2023-09-09T22:52:20.546 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-09T22:52:20.547 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/pdonnell-2023-09-09_19:00:20-fs-wip-batrick-testing-20230909.145638-distro-default-smithi/7392474 --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/postgres}} --name pdonnell-2023-09-09_19:00:20-fs-wip-batrick-testing-20230909.145638-distro-default-smithi --verbose --owner scheduled_pdonnell@teuthology -- /home/teuthworker/archive/pdonnell-2023-09-09_19:00:20-fs-wip-batrick-testing-20230909.145638-distro-default-smithi/7392474/orig.config.yaml 2023-09-09T22:52:20.613 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-09T22:52:20.708 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2023-09-09_19:00:20-fs-wip-batrick-testing-20230909.145638-distro-default-smithi/7392474 branch: wip-batrick-testing-20230909.145638 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/postgres}} email: pdonnell@redhat.com first_in_suite: false job_id: '7392474' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2023-09-09_19:00:20-fs-wip-batrick-testing-20230909.145638-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-batrick-testing-20230909.145638 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mds: mds_bal_replicate_threshold: 1 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 9670cca5bc8e17916865dc84939edfc10b1f2c00 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 9670cca5bc8e17916865dc84939edfc10b1f2c00 kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=crc - wsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20230909.145638 sha1: 9670cca5bc8e17916865dc84939edfc10b1f2c00 owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5964 sha1: 9670cca5bc8e17916865dc84939edfc10b1f2c00 sleep_before_teardown: 0 subset: 225/256 suite: fs suite_branch: wip-batrick-testing-20230909.145638 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_9670cca5bc8e17916865dc84939edfc10b1f2c00/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 9670cca5bc8e17916865dc84939edfc10b1f2c00 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi082.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi098.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi170.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 500 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-09-09_19:00:20 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.1394 2023-09-09T22:52:20.709 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_9670cca5bc8e17916865dc84939edfc10b1f2c00/qa; will attempt to use it 2023-09-09T22:52:20.710 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_9670cca5bc8e17916865dc84939edfc10b1f2c00/qa/tasks 2023-09-09T22:52:20.710 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-09-09T22:52:20.713 INFO:teuthology.task.internal:Checking packages... 2023-09-09T22:52:20.737 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '9670cca5bc8e17916865dc84939edfc10b1f2c00' 2023-09-09T22:52:20.737 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-09T22:52:20.738 INFO:teuthology.packaging:ref: None 2023-09-09T22:52:20.738 INFO:teuthology.packaging:tag: None 2023-09-09T22:52:20.738 INFO:teuthology.packaging:branch: wip-batrick-testing-20230909.145638 2023-09-09T22:52:20.739 INFO:teuthology.packaging:sha1: 9670cca5bc8e17916865dc84939edfc10b1f2c00 2023-09-09T22:52:20.739 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-batrick-testing-20230909.145638 2023-09-09T22:52:20.969 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6025.g9670cca5 2023-09-09T22:52:20.969 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-09-09T22:52:20.976 INFO:teuthology.task.internal:no buildpackages task found 2023-09-09T22:52:20.977 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-09-09T22:52:20.992 INFO:teuthology.task.internal:Saving configuration 2023-09-09T22:52:21.016 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-09-09T22:52:21.022 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-09T22:52:21.048 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-09-09_19:00:20-fs-wip-batrick-testing-20230909.145638-distro-default-smithi/7392474', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-09 22:47:03.551957', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-09T22:52:21.069 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi098.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-09-09_19:00:20-fs-wip-batrick-testing-20230909.145638-distro-default-smithi/7392474', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-09 22:47:03.553994', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-09T22:52:21.095 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-09-09_19:00:20-fs-wip-batrick-testing-20230909.145638-distro-default-smithi/7392474', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-09 22:47:03.555719', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-09T22:52:21.095 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-09-09T22:52:21.101 INFO:teuthology.task.internal:roles: ubuntu@smithi082.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-09-09T22:52:21.101 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-09-09T22:52:21.101 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-09-09T22:52:21.102 INFO:teuthology.run_tasks:Running task console_log... 2023-09-09T22:52:21.197 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f77d376fd30>, signals=[15]) 2023-09-09T22:52:21.200 INFO:teuthology.run_tasks:Running task internal.connect... 2023-09-09T22:52:21.206 INFO:teuthology.task.internal:Opening connections... 2023-09-09T22:52:21.207 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi082.front.sepia.ceph.com 2023-09-09T22:52:21.208 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-09T22:52:21.276 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi098.front.sepia.ceph.com 2023-09-09T22:52:21.278 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-09T22:52:21.345 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi170.front.sepia.ceph.com 2023-09-09T22:52:21.347 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-09T22:52:21.414 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-09-09T22:52:21.421 DEBUG:teuthology.orchestra.run.smithi082:> uname -m 2023-09-09T22:52:21.442 INFO:teuthology.orchestra.run.smithi082.stdout:x86_64 2023-09-09T22:52:21.442 DEBUG:teuthology.orchestra.run.smithi082:> cat /etc/os-release 2023-09-09T22:52:21.502 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="Red Hat Enterprise Linux" 2023-09-09T22:52:21.503 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="8.6 (Ootpa)" 2023-09-09T22:52:21.503 INFO:teuthology.orchestra.run.smithi082.stdout:ID="rhel" 2023-09-09T22:52:21.503 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE="fedora" 2023-09-09T22:52:21.503 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="8.6" 2023-09-09T22:52:21.503 INFO:teuthology.orchestra.run.smithi082.stdout:PLATFORM_ID="platform:el8" 2023-09-09T22:52:21.504 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-09T22:52:21.504 INFO:teuthology.orchestra.run.smithi082.stdout:ANSI_COLOR="0;31" 2023-09-09T22:52:21.504 INFO:teuthology.orchestra.run.smithi082.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-09T22:52:21.504 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://www.redhat.com/" 2023-09-09T22:52:21.504 INFO:teuthology.orchestra.run.smithi082.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-09T22:52:21.505 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-09T22:52:21.505 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T22:52:21.505 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-09T22:52:21.505 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-09T22:52:21.505 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-09T22:52:21.506 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-09T22:52:21.506 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server 2023-09-09T22:52:21.533 DEBUG:teuthology.orchestra.run.smithi098:> uname -m 2023-09-09T22:52:21.553 INFO:teuthology.orchestra.run.smithi098.stdout:x86_64 2023-09-09T22:52:21.554 DEBUG:teuthology.orchestra.run.smithi098:> cat /etc/os-release 2023-09-09T22:52:21.613 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="Red Hat Enterprise Linux" 2023-09-09T22:52:21.614 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="8.6 (Ootpa)" 2023-09-09T22:52:21.614 INFO:teuthology.orchestra.run.smithi098.stdout:ID="rhel" 2023-09-09T22:52:21.614 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="fedora" 2023-09-09T22:52:21.614 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="8.6" 2023-09-09T22:52:21.615 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el8" 2023-09-09T22:52:21.615 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-09T22:52:21.615 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31" 2023-09-09T22:52:21.615 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-09T22:52:21.615 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://www.redhat.com/" 2023-09-09T22:52:21.616 INFO:teuthology.orchestra.run.smithi098.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-09T22:52:21.616 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-09T22:52:21.616 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-09-09T22:52:21.616 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-09T22:52:21.616 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-09T22:52:21.617 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-09T22:52:21.617 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-09T22:52:21.617 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server 2023-09-09T22:52:21.641 DEBUG:teuthology.orchestra.run.smithi170:> uname -m 2023-09-09T22:52:21.663 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2023-09-09T22:52:21.663 DEBUG:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2023-09-09T22:52:21.722 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="Red Hat Enterprise Linux" 2023-09-09T22:52:21.723 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="8.6 (Ootpa)" 2023-09-09T22:52:21.723 INFO:teuthology.orchestra.run.smithi170.stdout:ID="rhel" 2023-09-09T22:52:21.723 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="fedora" 2023-09-09T22:52:21.723 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="8.6" 2023-09-09T22:52:21.724 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el8" 2023-09-09T22:52:21.724 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-09T22:52:21.724 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31" 2023-09-09T22:52:21.724 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-09T22:52:21.724 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://www.redhat.com/" 2023-09-09T22:52:21.725 INFO:teuthology.orchestra.run.smithi170.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-09T22:52:21.725 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-09T22:52:21.725 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-09T22:52:21.725 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-09T22:52:21.725 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-09T22:52:21.725 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-09T22:52:21.726 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-09T22:52:21.726 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2023-09-09T22:52:21.749 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-09-09T22:52:21.758 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-09-09T22:52:21.764 INFO:teuthology.task.internal:Checking for old test directory... 2023-09-09T22:52:21.765 DEBUG:teuthology.orchestra.run.smithi082:> test '!' -e /home/ubuntu/cephtest 2023-09-09T22:52:21.767 DEBUG:teuthology.orchestra.run.smithi098:> test '!' -e /home/ubuntu/cephtest 2023-09-09T22:52:21.769 DEBUG:teuthology.orchestra.run.smithi170:> test '!' -e /home/ubuntu/cephtest 2023-09-09T22:52:21.788 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-09-09T22:52:21.794 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-09-09T22:52:21.795 DEBUG:teuthology.orchestra.run.smithi082:> test -z $(ls -A /var/lib/ceph) 2023-09-09T22:52:21.826 DEBUG:teuthology.orchestra.run.smithi098:> test -z $(ls -A /var/lib/ceph) 2023-09-09T22:52:21.828 DEBUG:teuthology.orchestra.run.smithi170:> test -z $(ls -A /var/lib/ceph) 2023-09-09T22:52:21.859 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-09-09T22:52:21.942 INFO:teuthology.run_tasks:Running task kernel... 2023-09-09T22:52:21.957 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-09-09T22:52:21.958 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2023-09-09T22:52:21.958 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2023-09-09T22:52:21.959 DEBUG:teuthology.orchestra.run.smithi082:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-09T22:52:21.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-09T22:52:21.978 DEBUG:teuthology.orchestra.run.smithi082:> uname -r 2023-09-09T22:52:22.038 INFO:teuthology.orchestra.run.smithi082.stdout:4.18.0-372.9.1.el8.x86_64 2023-09-09T22:52:22.038 INFO:teuthology.task.kernel:Running kernel on smithi082: 4.18.0-372.9.1.el8.x86_64 2023-09-09T22:52:22.039 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum install -y kernel 2023-09-09T22:52:23.217 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-09-09T22:52:23.217 INFO:teuthology.orchestra.run.smithi082.stdout:Unable to read consumer identity 2023-09-09T22:52:24.619 INFO:teuthology.orchestra.run.smithi082.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2023-09-09T22:52:25.311 INFO:teuthology.orchestra.run.smithi082.stdout:Extra Packages for Enterprise Linux 24 MB/s | 16 MB 00:00 2023-09-09T22:52:28.802 INFO:teuthology.orchestra.run.smithi082.stdout:lab-extras 438 kB/s | 24 kB 00:00 2023-09-09T22:52:29.521 INFO:teuthology.orchestra.run.smithi082.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-09-09T22:52:29.572 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-09-09T22:52:29.573 INFO:teuthology.orchestra.run.smithi082.stdout:Nothing to do. 2023-09-09T22:52:29.574 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-09-09T22:52:29.696 DEBUG:teuthology.orchestra.run.smithi082:> echo no | sudo yum reinstall kernel || true 2023-09-09T22:52:30.026 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-09-09T22:52:30.026 INFO:teuthology.orchestra.run.smithi082.stdout:Unable to read consumer identity 2023-09-09T22:52:30.282 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:00:02 ago on Sat 09 Sep 2023 10:52:28 PM UTC. 2023-09-09T22:52:30.325 INFO:teuthology.orchestra.run.smithi082.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-09-09T22:52:30.373 INFO:teuthology.orchestra.run.smithi082.stderr:Error: No packages marked for reinstall. 2023-09-09T22:52:30.419 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum reinstall -y kernel || true 2023-09-09T22:52:30.747 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-09-09T22:52:30.748 INFO:teuthology.orchestra.run.smithi082.stdout:Unable to read consumer identity 2023-09-09T22:52:30.999 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:00:03 ago on Sat 09 Sep 2023 10:52:28 PM UTC. 2023-09-09T22:52:31.038 INFO:teuthology.orchestra.run.smithi082.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-09-09T22:52:31.064 INFO:teuthology.orchestra.run.smithi082.stderr:Error: No packages marked for reinstall. 2023-09-09T22:52:31.110 DEBUG:teuthology.orchestra.run.smithi082:> rpm -q kernel | sort -rV | head -n 1 2023-09-09T22:52:31.190 INFO:teuthology.orchestra.run.smithi082.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-09-09T22:52:31.191 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-09-09T22:52:31.191 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-09-09T22:52:31.191 DEBUG:teuthology.parallel:result is None 2023-09-09T22:52:31.192 INFO:teuthology.run_tasks:Running task internal.base... 2023-09-09T22:52:31.199 INFO:teuthology.task.internal:Creating test directory... 2023-09-09T22:52:31.199 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-09T22:52:31.202 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-09T22:52:31.204 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-09T22:52:31.224 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-09-09T22:52:31.231 INFO:teuthology.run_tasks:Running task internal.archive... 2023-09-09T22:52:31.240 INFO:teuthology.task.internal:Creating archive directory... 2023-09-09T22:52:31.241 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-09T22:52:31.259 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-09T22:52:31.265 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-09T22:52:31.300 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-09-09T22:52:31.307 INFO:teuthology.task.internal:Enabling coredump saving... 2023-09-09T22:52:31.308 DEBUG:teuthology.orchestra.run.smithi098:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-09T22:52:31.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-09T22:52:31.350 DEBUG:teuthology.orchestra.run.smithi170:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-09T22:52:31.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-09T22:52:31.370 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-09T22:52:31.372 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-09T22:52:31.393 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-09T22:52:31.420 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-09T22:52:31.437 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-09T22:52:31.458 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-09T22:52:31.459 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-09T22:52:31.474 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-09T22:52:31.475 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-09T22:52:31.477 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-09-09T22:52:31.484 INFO:teuthology.task.internal:Configuring sudo... 2023-09-09T22:52:31.485 DEBUG:teuthology.orchestra.run.smithi082:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-09T22:52:31.487 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-09T22:52:31.490 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-09T22:52:31.519 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-09-09T22:52:31.528 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-09-09T22:52:31.529 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-09T22:52:31.555 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-09T22:52:31.559 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-09T22:52:31.577 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-09T22:52:31.652 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-09T22:52:31.697 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-09T22:52:31.781 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-09T22:52:31.821 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-09-09T22:52:31.822 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-09T22:52:31.888 DEBUG:teuthology.orchestra.run.smithi098:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-09T22:52:31.934 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-09T22:52:31.990 DEBUG:teuthology.orchestra.run.smithi098:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-09T22:52:32.076 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-09T22:52:32.120 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-09-09T22:52:32.121 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-09T22:52:32.193 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-09T22:52:32.241 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-09T22:52:32.294 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-09T22:52:32.382 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-09T22:52:32.427 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-09-09T22:52:32.427 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-09T22:52:32.498 DEBUG:teuthology.orchestra.run.smithi082:> sudo service rsyslog restart 2023-09-09T22:52:32.501 DEBUG:teuthology.orchestra.run.smithi098:> sudo service rsyslog restart 2023-09-09T22:52:32.504 DEBUG:teuthology.orchestra.run.smithi170:> sudo service rsyslog restart 2023-09-09T22:52:32.564 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-09T22:52:32.587 INFO:teuthology.orchestra.run.smithi098.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-09T22:52:32.594 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-09T22:52:33.015 INFO:teuthology.run_tasks:Running task internal.timer... 2023-09-09T22:52:33.024 INFO:teuthology.task.internal:Starting timer... 2023-09-09T22:52:33.024 INFO:teuthology.run_tasks:Running task pcp... 2023-09-09T22:52:33.034 INFO:teuthology.run_tasks:Running task selinux... 2023-09-09T22:52:33.042 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-09-09T22:52:33.043 DEBUG:teuthology.orchestra.run.smithi082:> sudo service auditd rotate 2023-09-09T22:52:33.104 INFO:teuthology.orchestra.run.smithi082.stdout:Rotating logs: 2023-09-09T22:52:33.105 DEBUG:teuthology.orchestra.run.smithi098:> sudo service auditd rotate 2023-09-09T22:52:33.172 INFO:teuthology.orchestra.run.smithi098.stdout:Rotating logs: 2023-09-09T22:52:33.173 DEBUG:teuthology.orchestra.run.smithi170:> sudo service auditd rotate 2023-09-09T22:52:33.246 INFO:teuthology.orchestra.run.smithi170.stdout:Rotating logs: 2023-09-09T22:52:33.247 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-09-09T22:52:33.248 DEBUG:teuthology.orchestra.run.smithi082:> /usr/sbin/getenforce 2023-09-09T22:52:33.268 INFO:teuthology.orchestra.run.smithi082.stdout:Permissive 2023-09-09T22:52:33.269 DEBUG:teuthology.orchestra.run.smithi098:> /usr/sbin/getenforce 2023-09-09T22:52:33.303 INFO:teuthology.orchestra.run.smithi098.stdout:Permissive 2023-09-09T22:52:33.304 DEBUG:teuthology.orchestra.run.smithi170:> /usr/sbin/getenforce 2023-09-09T22:52:33.334 INFO:teuthology.orchestra.run.smithi170.stdout:Permissive 2023-09-09T22:52:33.335 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi082.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi098.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi170.front.sepia.ceph.com': 'permissive'} 2023-09-09T22:52:33.335 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-09T22:52:33.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-09T22:52:33.364 DEBUG:teuthology.orchestra.run.smithi098:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-09T22:52:33.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-09T22:52:33.392 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-09T22:52:33.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-09T22:52:33.419 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-09-09T22:52:33.420 DEBUG:teuthology.orchestra.run.smithi082:> sudo /usr/sbin/setenforce permissive 2023-09-09T22:52:33.447 DEBUG:teuthology.orchestra.run.smithi098:> sudo /usr/sbin/setenforce permissive 2023-09-09T22:52:33.482 DEBUG:teuthology.orchestra.run.smithi170:> sudo /usr/sbin/setenforce permissive 2023-09-09T22:52:33.509 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-09-09T22:52:33.519 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-09-09T22:52:33.520 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-09-09T22:52:33.536 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-09-09T22:52:33.537 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi082.front.sepia.ceph.com,smithi098.front.sepia.ceph.com,smithi170.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-09-09T23:01:37.173 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi082.front.sepia.ceph.com'), Remote(name='ubuntu@smithi098.front.sepia.ceph.com'), Remote(name='ubuntu@smithi170.front.sepia.ceph.com')] 2023-09-09T23:01:37.174 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2023-09-09T23:01:37.175 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-09T23:01:37.244 DEBUG:teuthology.orchestra.run.smithi082:> true 2023-09-09T23:01:37.311 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi082.front.sepia.ceph.com' 2023-09-09T23:01:37.312 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2023-09-09T23:01:37.313 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-09T23:01:37.381 DEBUG:teuthology.orchestra.run.smithi098:> true 2023-09-09T23:01:37.448 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi098.front.sepia.ceph.com' 2023-09-09T23:01:37.448 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2023-09-09T23:01:37.449 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-09T23:01:37.518 DEBUG:teuthology.orchestra.run.smithi170:> true 2023-09-09T23:01:37.592 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2023-09-09T23:01:37.593 INFO:teuthology.run_tasks:Running task clock... 2023-09-09T23:01:37.604 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-09-09T23:01:37.605 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-09T23:01:37.606 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-09T23:01:37.608 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-09T23:01:37.608 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-09T23:01:37.610 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-09T23:01:37.611 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-09T23:01:37.641 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-09T23:01:37.642 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-09T23:01:37.656 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-09T23:01:37.657 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-09T23:01:37.666 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-09T23:01:37.681 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-09T23:01:37.726 INFO:teuthology.orchestra.run.smithi082.stderr:sudo: ntpd: command not found 2023-09-09T23:01:37.738 INFO:teuthology.orchestra.run.smithi082.stdout:506 Cannot talk to daemon 2023-09-09T23:01:37.753 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-09T23:01:37.757 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: ntpd: command not found 2023-09-09T23:01:37.768 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-09T23:01:37.769 INFO:teuthology.orchestra.run.smithi170.stdout:506 Cannot talk to daemon 2023-09-09T23:01:37.784 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-09T23:01:37.798 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-09T23:01:37.820 INFO:teuthology.orchestra.run.smithi098.stderr:sudo: ntpd: command not found 2023-09-09T23:01:37.832 INFO:teuthology.orchestra.run.smithi098.stdout:506 Cannot talk to daemon 2023-09-09T23:01:37.840 INFO:teuthology.orchestra.run.smithi082.stderr:bash: ntpq: command not found 2023-09-09T23:01:37.843 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-09T23:01:37.844 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2023-09-09T23:01:37.844 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-09T23:01:37.844 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-09T23:01:37.844 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-09T23:01:37.844 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-09T23:01:37.846 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-09T23:01:37.860 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-09T23:01:37.870 INFO:teuthology.orchestra.run.smithi170.stderr:bash: ntpq: command not found 2023-09-09T23:01:37.874 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-09T23:01:37.874 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2023-09-09T23:01:37.874 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-09T23:01:37.874 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-09T23:01:37.874 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-09T23:01:37.875 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-09T23:01:37.940 INFO:teuthology.orchestra.run.smithi098.stderr:bash: ntpq: command not found 2023-09-09T23:01:37.943 INFO:teuthology.orchestra.run.smithi098.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-09T23:01:37.944 INFO:teuthology.orchestra.run.smithi098.stdout:=============================================================================== 2023-09-09T23:01:37.944 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-09T23:01:37.944 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-09T23:01:37.944 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-09T23:01:37.944 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-09T23:01:37.945 INFO:teuthology.run_tasks:Running task pexec... 2023-09-09T23:01:37.957 INFO:teuthology.task.pexec:Executing custom commands... 2023-09-09T23:01:37.958 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi082.front.sepia.ceph.com 2023-09-09T23:01:37.958 DEBUG:teuthology.orchestra.run.smithi082:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-09-09T23:01:37.958 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi098.front.sepia.ceph.com 2023-09-09T23:01:37.959 DEBUG:teuthology.orchestra.run.smithi098:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-09-09T23:01:37.959 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi170.front.sepia.ceph.com 2023-09-09T23:01:37.959 DEBUG:teuthology.orchestra.run.smithi170:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-09-09T23:01:38.330 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-09-09T23:01:38.331 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2023-09-09T23:01:38.359 INFO:teuthology.orchestra.run.smithi098.stdout:Updating Subscription Management repositories. 2023-09-09T23:01:39.143 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:01:29 ago on Sat 09 Sep 2023 11:00:09 PM UTC. 2023-09-09T23:01:39.151 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:01:38 ago on Sat 09 Sep 2023 11:00:00 PM UTC. 2023-09-09T23:01:39.167 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:01:49 ago on Sat 09 Sep 2023 10:59:49 PM UTC. 2023-09-09T23:01:40.883 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2023-09-09T23:01:40.884 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2023-09-09T23:01:40.884 INFO:teuthology.orchestra.run.smithi170.stdout: Package Architecture Version Repository Size 2023-09-09T23:01:40.885 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2023-09-09T23:01:40.885 INFO:teuthology.orchestra.run.smithi170.stdout:Resetting modules: 2023-09-09T23:01:40.885 INFO:teuthology.orchestra.run.smithi170.stdout: container-tools 2023-09-09T23:01:40.885 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-09T23:01:40.885 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2023-09-09T23:01:40.886 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2023-09-09T23:01:40.886 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-09T23:01:40.898 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2023-09-09T23:01:40.899 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2023-09-09T23:01:40.900 INFO:teuthology.orchestra.run.smithi098.stdout: Package Architecture Version Repository Size 2023-09-09T23:01:40.900 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2023-09-09T23:01:40.900 INFO:teuthology.orchestra.run.smithi098.stdout:Resetting modules: 2023-09-09T23:01:40.900 INFO:teuthology.orchestra.run.smithi098.stdout: container-tools 2023-09-09T23:01:40.900 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-09-09T23:01:40.901 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2023-09-09T23:01:40.901 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2023-09-09T23:01:40.901 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-09-09T23:01:40.906 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-09-09T23:01:40.908 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-09-09T23:01:40.908 INFO:teuthology.orchestra.run.smithi082.stdout: Package Architecture Version Repository Size 2023-09-09T23:01:40.908 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-09-09T23:01:40.908 INFO:teuthology.orchestra.run.smithi082.stdout:Resetting modules: 2023-09-09T23:01:40.909 INFO:teuthology.orchestra.run.smithi082.stdout: container-tools 2023-09-09T23:01:40.909 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:01:40.909 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2023-09-09T23:01:40.909 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-09-09T23:01:40.910 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:01:41.842 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:01:32 ago on Sat 09 Sep 2023 11:00:09 PM UTC. 2023-09-09T23:01:41.846 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:01:41 ago on Sat 09 Sep 2023 11:00:00 PM UTC. 2023-09-09T23:01:41.869 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:01:52 ago on Sat 09 Sep 2023 10:59:49 PM UTC. 2023-09-09T23:01:44.572 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2023-09-09T23:01:44.572 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2023-09-09T23:01:44.630 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-09-09T23:01:45.043 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2023-09-09T23:01:45.046 INFO:teuthology.orchestra.run.smithi098.stdout:Updating Subscription Management repositories. 2023-09-09T23:01:45.101 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-09-09T23:01:45.833 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:01:36 ago on Sat 09 Sep 2023 11:00:09 PM UTC. 2023-09-09T23:01:45.843 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:01:45 ago on Sat 09 Sep 2023 11:00:00 PM UTC. 2023-09-09T23:01:45.891 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:01:56 ago on Sat 09 Sep 2023 10:59:49 PM UTC. 2023-09-09T23:01:47.568 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2023-09-09T23:01:47.579 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2023-09-09T23:01:47.607 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-09-09T23:01:47.661 INFO:teuthology.orchestra.run.smithi098.stdout:============================================================================================================ 2023-09-09T23:01:47.661 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repository Size 2023-09-09T23:01:47.662 INFO:teuthology.orchestra.run.smithi098.stdout:============================================================================================================ 2023-09-09T23:01:47.662 INFO:teuthology.orchestra.run.smithi098.stdout:Installing group/module packages: 2023-09-09T23:01:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-09-09T23:01:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-09-09T23:01:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-09-09T23:01:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-09-09T23:01:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-09-09T23:01:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-09-09T23:01:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-09-09T23:01:47.664 INFO:teuthology.orchestra.run.smithi098.stdout:Installing dependencies: 2023-09-09T23:01:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-09-09T23:01:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-09-09T23:01:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-09-09T23:01:47.664 INFO:teuthology.orchestra.run.smithi098.stdout:Installing module profiles: 2023-09-09T23:01:47.665 INFO:teuthology.orchestra.run.smithi098.stdout: container-tools/common 2023-09-09T23:01:47.665 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-09-09T23:01:47.665 INFO:teuthology.orchestra.run.smithi098.stdout:Enabling module streams: 2023-09-09T23:01:47.665 INFO:teuthology.orchestra.run.smithi098.stdout: container-tools rhel8 2023-09-09T23:01:47.665 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-09-09T23:01:47.666 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2023-09-09T23:01:47.666 INFO:teuthology.orchestra.run.smithi098.stdout:============================================================================================================ 2023-09-09T23:01:47.666 INFO:teuthology.orchestra.run.smithi098.stdout:Install 10 Packages 2023-09-09T23:01:47.666 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-09-09T23:01:47.667 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 18 M 2023-09-09T23:01:47.667 INFO:teuthology.orchestra.run.smithi098.stdout:Installed size: 65 M 2023-09-09T23:01:47.667 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2023-09-09T23:01:47.672 INFO:teuthology.orchestra.run.smithi170.stdout:============================================================================================================ 2023-09-09T23:01:47.673 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2023-09-09T23:01:47.673 INFO:teuthology.orchestra.run.smithi170.stdout:============================================================================================================ 2023-09-09T23:01:47.673 INFO:teuthology.orchestra.run.smithi170.stdout:Installing group/module packages: 2023-09-09T23:01:47.673 INFO:teuthology.orchestra.run.smithi170.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-09-09T23:01:47.674 INFO:teuthology.orchestra.run.smithi170.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-09-09T23:01:47.674 INFO:teuthology.orchestra.run.smithi170.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-09-09T23:01:47.674 INFO:teuthology.orchestra.run.smithi170.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-09-09T23:01:47.674 INFO:teuthology.orchestra.run.smithi170.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-09-09T23:01:47.674 INFO:teuthology.orchestra.run.smithi170.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-09-09T23:01:47.675 INFO:teuthology.orchestra.run.smithi170.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-09-09T23:01:47.675 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2023-09-09T23:01:47.675 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-09-09T23:01:47.675 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-09-09T23:01:47.675 INFO:teuthology.orchestra.run.smithi170.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-09-09T23:01:47.676 INFO:teuthology.orchestra.run.smithi170.stdout:Installing module profiles: 2023-09-09T23:01:47.676 INFO:teuthology.orchestra.run.smithi170.stdout: container-tools/common 2023-09-09T23:01:47.676 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-09T23:01:47.676 INFO:teuthology.orchestra.run.smithi170.stdout:Enabling module streams: 2023-09-09T23:01:47.676 INFO:teuthology.orchestra.run.smithi170.stdout: container-tools rhel8 2023-09-09T23:01:47.676 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-09T23:01:47.677 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2023-09-09T23:01:47.677 INFO:teuthology.orchestra.run.smithi170.stdout:============================================================================================================ 2023-09-09T23:01:47.677 INFO:teuthology.orchestra.run.smithi170.stdout:Install 10 Packages 2023-09-09T23:01:47.677 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-09T23:01:47.678 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 18 M 2023-09-09T23:01:47.678 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 65 M 2023-09-09T23:01:47.678 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2023-09-09T23:01:47.699 INFO:teuthology.orchestra.run.smithi082.stdout:============================================================================================================ 2023-09-09T23:01:47.699 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2023-09-09T23:01:47.699 INFO:teuthology.orchestra.run.smithi082.stdout:============================================================================================================ 2023-09-09T23:01:47.699 INFO:teuthology.orchestra.run.smithi082.stdout:Installing group/module packages: 2023-09-09T23:01:47.700 INFO:teuthology.orchestra.run.smithi082.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-09-09T23:01:47.700 INFO:teuthology.orchestra.run.smithi082.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-09-09T23:01:47.700 INFO:teuthology.orchestra.run.smithi082.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-09-09T23:01:47.700 INFO:teuthology.orchestra.run.smithi082.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-09-09T23:01:47.700 INFO:teuthology.orchestra.run.smithi082.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-09-09T23:01:47.701 INFO:teuthology.orchestra.run.smithi082.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-09-09T23:01:47.701 INFO:teuthology.orchestra.run.smithi082.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-09-09T23:01:47.701 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2023-09-09T23:01:47.701 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-09-09T23:01:47.701 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-09-09T23:01:47.702 INFO:teuthology.orchestra.run.smithi082.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-09-09T23:01:47.702 INFO:teuthology.orchestra.run.smithi082.stdout:Installing module profiles: 2023-09-09T23:01:47.702 INFO:teuthology.orchestra.run.smithi082.stdout: container-tools/common 2023-09-09T23:01:47.702 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:01:47.702 INFO:teuthology.orchestra.run.smithi082.stdout:Enabling module streams: 2023-09-09T23:01:47.703 INFO:teuthology.orchestra.run.smithi082.stdout: container-tools rhel8 2023-09-09T23:01:47.703 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:01:47.703 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2023-09-09T23:01:47.703 INFO:teuthology.orchestra.run.smithi082.stdout:============================================================================================================ 2023-09-09T23:01:47.703 INFO:teuthology.orchestra.run.smithi082.stdout:Install 10 Packages 2023-09-09T23:01:47.704 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:01:47.704 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 18 M 2023-09-09T23:01:47.704 INFO:teuthology.orchestra.run.smithi082.stdout:Installed size: 65 M 2023-09-09T23:01:47.705 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2023-09-09T23:01:47.744 INFO:teuthology.orchestra.run.smithi098.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.1 MB/s | 94 kB 00:00 2023-09-09T23:01:47.765 INFO:teuthology.orchestra.run.smithi170.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 455 kB/s | 41 kB 00:00 2023-09-09T23:01:47.777 INFO:teuthology.orchestra.run.smithi098.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 219 kB/s | 25 kB 00:00 2023-09-09T23:01:47.780 INFO:teuthology.orchestra.run.smithi082.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.2 MB/s | 94 kB 00:00 2023-09-09T23:01:47.794 INFO:teuthology.orchestra.run.smithi170.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 209 kB/s | 25 kB 00:00 2023-09-09T23:01:47.803 INFO:teuthology.orchestra.run.smithi098.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 294 kB/s | 41 kB 00:00 2023-09-09T23:01:47.805 INFO:teuthology.orchestra.run.smithi082.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 239 kB/s | 25 kB 00:00 2023-09-09T23:01:47.819 INFO:teuthology.orchestra.run.smithi170.stdout:(3/10): udica-0.2.6-4.module+el8.6.0+16771+28df 916 kB/s | 49 kB 00:00 2023-09-09T23:01:47.828 INFO:teuthology.orchestra.run.smithi098.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 588 kB/s | 49 kB 00:00 2023-09-09T23:01:47.831 INFO:teuthology.orchestra.run.smithi082.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 316 kB/s | 41 kB 00:00 2023-09-09T23:01:47.844 INFO:teuthology.orchestra.run.smithi170.stdout:(4/10): python3-pyxdg-0.25-16.el8.noarch.rpm 555 kB/s | 94 kB 00:00 2023-09-09T23:01:47.856 INFO:teuthology.orchestra.run.smithi082.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 655 kB/s | 49 kB 00:00 2023-09-09T23:01:47.870 INFO:teuthology.orchestra.run.smithi098.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 5.6 MB/s | 524 kB 00:00 2023-09-09T23:01:47.878 INFO:teuthology.orchestra.run.smithi170.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-09-09T23:01:47.895 INFO:teuthology.orchestra.run.smithi098.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.2 MB/s | 149 kB 00:00 2023-09-09T23:01:47.898 INFO:teuthology.orchestra.run.smithi082.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 5.6 MB/s | 524 kB 00:00 2023-09-09T23:01:47.912 INFO:teuthology.orchestra.run.smithi170.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.2 MB/s | 149 kB 00:00 2023-09-09T23:01:47.937 INFO:teuthology.orchestra.run.smithi170.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 3.5 MB/s | 213 kB 00:00 2023-09-09T23:01:47.948 INFO:teuthology.orchestra.run.smithi082.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.6 MB/s | 149 kB 00:00 2023-09-09T23:01:47.954 INFO:teuthology.orchestra.run.smithi098.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 2.5 MB/s | 213 kB 00:00 2023-09-09T23:01:47.973 INFO:teuthology.orchestra.run.smithi082.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 2.8 MB/s | 213 kB 00:00 2023-09-09T23:01:48.029 INFO:teuthology.orchestra.run.smithi170.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 24 MB/s | 2.2 MB 00:00 2023-09-09T23:01:48.046 INFO:teuthology.orchestra.run.smithi098.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 24 MB/s | 2.2 MB 00:00 2023-09-09T23:01:48.074 INFO:teuthology.orchestra.run.smithi082.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 22 MB/s | 2.2 MB 00:00 2023-09-09T23:01:48.146 INFO:teuthology.orchestra.run.smithi170.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 30 MB/s | 7.0 MB 00:00 2023-09-09T23:01:48.156 INFO:teuthology.orchestra.run.smithi098.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 27 MB/s | 7.0 MB 00:00 2023-09-09T23:01:48.191 INFO:teuthology.orchestra.run.smithi082.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 29 MB/s | 7.0 MB 00:00 2023-09-09T23:01:48.263 INFO:teuthology.orchestra.run.smithi170.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 18 MB/s | 8.0 MB 00:00 2023-09-09T23:01:48.263 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2023-09-09T23:01:48.264 INFO:teuthology.orchestra.run.smithi170.stdout:Total 31 MB/s | 18 MB 00:00 2023-09-09T23:01:48.267 INFO:teuthology.orchestra.run.smithi098.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 17 MB/s | 8.0 MB 00:00 2023-09-09T23:01:48.268 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2023-09-09T23:01:48.268 INFO:teuthology.orchestra.run.smithi098.stdout:Total 30 MB/s | 18 MB 00:00 2023-09-09T23:01:48.282 INFO:teuthology.orchestra.run.smithi082.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 18 MB/s | 8.0 MB 00:00 2023-09-09T23:01:48.283 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2023-09-09T23:01:48.283 INFO:teuthology.orchestra.run.smithi082.stdout:Total 31 MB/s | 18 MB 00:00 2023-09-09T23:01:48.467 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2023-09-09T23:01:48.468 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2023-09-09T23:01:48.484 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2023-09-09T23:01:48.486 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2023-09-09T23:01:48.486 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2023-09-09T23:01:48.487 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2023-09-09T23:01:48.487 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2023-09-09T23:01:48.502 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2023-09-09T23:01:48.503 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2023-09-09T23:01:48.668 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2023-09-09T23:01:48.669 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2023-09-09T23:01:48.672 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2023-09-09T23:01:48.673 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2023-09-09T23:01:48.685 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2023-09-09T23:01:48.689 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2023-09-09T23:01:49.194 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2023-09-09T23:01:49.209 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2023-09-09T23:01:49.220 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2023-09-09T23:01:49.371 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-09-09T23:01:49.401 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-09-09T23:01:49.436 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-09-09T23:01:49.538 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-09T23:01:49.599 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-09T23:01:49.602 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-09T23:01:49.739 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-09-09T23:01:49.777 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-09-09T23:01:49.830 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-09-09T23:01:49.934 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-09-09T23:01:49.962 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-09-09T23:01:50.074 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-09-09T23:01:50.283 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-09-09T23:01:50.335 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-09-09T23:01:50.421 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-09-09T23:01:51.095 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-09-09T23:01:51.106 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-09-09T23:01:51.183 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-09-09T23:01:51.961 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-09-09T23:01:51.981 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-09-09T23:01:52.082 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-09-09T23:01:52.166 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-09-09T23:01:52.219 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-09-09T23:01:52.268 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-09-09T23:01:52.353 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-09-09T23:01:52.381 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-09-09T23:01:52.467 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-09-09T23:01:52.548 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-09T23:01:52.591 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-09T23:01:52.644 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-09T23:01:52.819 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-09T23:01:52.819 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-09-09T23:01:52.819 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-09T23:01:52.820 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-09-09T23:01:52.820 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-09-09T23:01:52.820 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-09-09T23:01:52.820 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-09-09T23:01:52.820 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-09-09T23:01:52.821 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-09-09T23:01:52.821 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-09-09T23:01:52.842 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-09T23:01:52.842 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-09-09T23:01:52.843 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-09T23:01:52.843 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-09-09T23:01:52.843 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-09-09T23:01:52.843 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-09-09T23:01:52.844 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-09-09T23:01:52.844 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-09-09T23:01:52.844 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-09-09T23:01:52.844 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-09-09T23:01:52.892 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-09T23:01:52.893 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-09-09T23:01:52.893 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-09T23:01:52.893 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-09-09T23:01:52.894 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-09-09T23:01:52.894 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-09-09T23:01:52.894 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-09-09T23:01:52.894 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-09-09T23:01:52.894 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-09-09T23:01:52.895 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-09-09T23:01:53.975 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-09-09T23:01:53.975 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:02:04 ago on Sat 09 Sep 2023 10:59:49 PM UTC. 2023-09-09T23:01:53.991 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-09-09T23:01:53.991 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:01:53 ago on Sat 09 Sep 2023 11:00:00 PM UTC. 2023-09-09T23:01:53.998 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-09-09T23:01:53.999 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:01:44 ago on Sat 09 Sep 2023 11:00:09 PM UTC. 2023-09-09T23:01:57.019 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2023-09-09T23:01:57.210 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-09T23:01:57.211 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2023-09-09T23:01:57.211 INFO:teuthology.orchestra.run.smithi170.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-09T23:01:57.211 INFO:teuthology.orchestra.run.smithi170.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-09-09T23:01:57.211 INFO:teuthology.orchestra.run.smithi170.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-09T23:01:57.212 INFO:teuthology.orchestra.run.smithi170.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-09-09T23:01:57.212 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-09-09T23:01:57.212 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-09-09T23:01:57.212 INFO:teuthology.orchestra.run.smithi170.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-09T23:01:57.212 INFO:teuthology.orchestra.run.smithi170.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-09-09T23:01:57.213 INFO:teuthology.orchestra.run.smithi170.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-09-09T23:01:57.213 INFO:teuthology.orchestra.run.smithi170.stdout: yajl-2.1.0-10.el8.x86_64 2023-09-09T23:01:57.213 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-09T23:01:57.213 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2023-09-09T23:01:57.364 DEBUG:teuthology.parallel:result is None 2023-09-09T23:01:57.931 INFO:teuthology.orchestra.run.smithi082.stdout:Installed products updated. 2023-09-09T23:01:58.126 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:01:58.126 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2023-09-09T23:01:58.127 INFO:teuthology.orchestra.run.smithi082.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-09T23:01:58.127 INFO:teuthology.orchestra.run.smithi082.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-09-09T23:01:58.127 INFO:teuthology.orchestra.run.smithi082.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-09T23:01:58.127 INFO:teuthology.orchestra.run.smithi082.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-09-09T23:01:58.127 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-09-09T23:01:58.127 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-09-09T23:01:58.128 INFO:teuthology.orchestra.run.smithi082.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-09T23:01:58.128 INFO:teuthology.orchestra.run.smithi082.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-09-09T23:01:58.128 INFO:teuthology.orchestra.run.smithi082.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-09-09T23:01:58.128 INFO:teuthology.orchestra.run.smithi082.stdout: yajl-2.1.0-10.el8.x86_64 2023-09-09T23:01:58.128 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:01:58.129 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-09-09T23:01:58.244 INFO:teuthology.orchestra.run.smithi098.stdout:Installed products updated. 2023-09-09T23:01:58.265 DEBUG:teuthology.parallel:result is None 2023-09-09T23:01:58.430 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-09-09T23:01:58.430 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2023-09-09T23:01:58.431 INFO:teuthology.orchestra.run.smithi098.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-09T23:01:58.431 INFO:teuthology.orchestra.run.smithi098.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-09-09T23:01:58.431 INFO:teuthology.orchestra.run.smithi098.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-09T23:01:58.431 INFO:teuthology.orchestra.run.smithi098.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-09-09T23:01:58.431 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-09-09T23:01:58.431 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-09-09T23:01:58.432 INFO:teuthology.orchestra.run.smithi098.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-09T23:01:58.432 INFO:teuthology.orchestra.run.smithi098.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-09-09T23:01:58.432 INFO:teuthology.orchestra.run.smithi098.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-09-09T23:01:58.432 INFO:teuthology.orchestra.run.smithi098.stdout: yajl-2.1.0-10.el8.x86_64 2023-09-09T23:01:58.432 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-09-09T23:01:58.433 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2023-09-09T23:01:58.572 DEBUG:teuthology.parallel:result is None 2023-09-09T23:01:58.573 INFO:teuthology.run_tasks:Running task install... 2023-09-09T23:01:58.580 DEBUG:teuthology.task.install:project ceph 2023-09-09T23:01:58.580 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9670cca5bc8e17916865dc84939edfc10b1f2c00'}} 2023-09-09T23:01:58.581 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9670cca5bc8e17916865dc84939edfc10b1f2c00'} 2023-09-09T23:01:58.581 INFO:teuthology.task.install:Using flavor: default 2023-09-09T23:01:58.587 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-09-09T23:01:58.587 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-09-09T23:01:58.588 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9670cca5bc8e17916865dc84939edfc10b1f2c00', 'tag': None, 'wait_for_package': False} 2023-09-09T23:01:58.588 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=9670cca5bc8e17916865dc84939edfc10b1f2c00 2023-09-09T23:01:58.590 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9670cca5bc8e17916865dc84939edfc10b1f2c00', 'tag': None, 'wait_for_package': False} 2023-09-09T23:01:58.591 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=9670cca5bc8e17916865dc84939edfc10b1f2c00 2023-09-09T23:01:58.592 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9670cca5bc8e17916865dc84939edfc10b1f2c00', 'tag': None, 'wait_for_package': False} 2023-09-09T23:01:58.592 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=9670cca5bc8e17916865dc84939edfc10b1f2c00 2023-09-09T23:01:58.734 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/flavors/default/ 2023-09-09T23:01:58.735 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6025.g9670cca5 2023-09-09T23:01:58.804 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/flavors/default/ 2023-09-09T23:01:58.805 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6025.g9670cca5 2023-09-09T23:01:58.812 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/flavors/default/ 2023-09-09T23:01:58.812 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6025.g9670cca5 2023-09-09T23:01:58.853 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-09T23:01:58.854 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-09-09T23:01:58.854 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-09T23:01:58.887 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-09T23:01:58.888 DEBUG:teuthology.orchestra.run.smithi170:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9670cca5bc8e17916865dc84939edfc10b1f2c00/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-09T23:01:58.919 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-09T23:01:58.919 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-09-09T23:01:58.919 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-09T23:01:58.947 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-09T23:01:58.947 DEBUG:teuthology.orchestra.run.smithi098:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9670cca5bc8e17916865dc84939edfc10b1f2c00/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-09T23:01:58.959 DEBUG:teuthology.orchestra.run.smithi170:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-09T23:01:58.972 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-09T23:01:58.972 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-09-09T23:01:58.973 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-09T23:01:59.003 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-09T23:01:59.003 DEBUG:teuthology.orchestra.run.smithi082:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9670cca5bc8e17916865dc84939edfc10b1f2c00/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-09T23:01:59.014 DEBUG:teuthology.orchestra.run.smithi098:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-09T23:01:59.039 DEBUG:teuthology.orchestra.run.smithi170:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-09T23:01:59.072 DEBUG:teuthology.orchestra.run.smithi082:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-09T23:01:59.091 DEBUG:teuthology.orchestra.run.smithi098:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-09T23:01:59.106 INFO:teuthology.orchestra.run.smithi170.stdout:check_obsoletes = 1 2023-09-09T23:01:59.107 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum clean all 2023-09-09T23:01:59.151 DEBUG:teuthology.orchestra.run.smithi082:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-09T23:01:59.157 INFO:teuthology.orchestra.run.smithi098.stdout:check_obsoletes = 1 2023-09-09T23:01:59.159 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum clean all 2023-09-09T23:01:59.218 INFO:teuthology.orchestra.run.smithi082.stdout:check_obsoletes = 1 2023-09-09T23:01:59.219 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum clean all 2023-09-09T23:01:59.480 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2023-09-09T23:01:59.528 INFO:teuthology.orchestra.run.smithi098.stdout:Updating Subscription Management repositories. 2023-09-09T23:01:59.590 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-09-09T23:02:00.420 INFO:teuthology.orchestra.run.smithi170.stdout:44 files removed 2023-09-09T23:02:00.470 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-09T23:02:00.479 INFO:teuthology.orchestra.run.smithi098.stdout:44 files removed 2023-09-09T23:02:00.503 INFO:teuthology.orchestra.run.smithi082.stdout:44 files removed 2023-09-09T23:02:00.531 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-09T23:02:00.552 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-09T23:02:00.801 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2023-09-09T23:02:00.862 INFO:teuthology.orchestra.run.smithi098.stdout:Updating Subscription Management repositories. 2023-09-09T23:02:00.882 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-09-09T23:02:02.074 INFO:teuthology.orchestra.run.smithi170.stdout:ceph packages for x86_64 266 kB/s | 81 kB 00:00 2023-09-09T23:02:02.253 INFO:teuthology.orchestra.run.smithi098.stdout:ceph packages for x86_64 290 kB/s | 81 kB 00:00 2023-09-09T23:02:02.402 INFO:teuthology.orchestra.run.smithi082.stdout:ceph packages for x86_64 224 kB/s | 81 kB 00:00 2023-09-09T23:02:02.433 INFO:teuthology.orchestra.run.smithi170.stdout:ceph noarch packages 29 kB/s | 9.4 kB 00:00 2023-09-09T23:02:02.520 INFO:teuthology.orchestra.run.smithi098.stdout:ceph noarch packages 40 kB/s | 9.4 kB 00:00 2023-09-09T23:02:02.703 INFO:teuthology.orchestra.run.smithi082.stdout:ceph noarch packages 35 kB/s | 9.4 kB 00:00 2023-09-09T23:02:02.792 INFO:teuthology.orchestra.run.smithi170.stdout:ceph source packages 4.4 kB/s | 1.5 kB 00:00 2023-09-09T23:02:02.837 INFO:teuthology.orchestra.run.smithi098.stdout:ceph source packages 5.0 kB/s | 1.5 kB 00:00 2023-09-09T23:02:02.968 INFO:teuthology.orchestra.run.smithi170.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2023-09-09T23:02:03.005 INFO:teuthology.orchestra.run.smithi098.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-09-09T23:02:03.062 INFO:teuthology.orchestra.run.smithi082.stdout:ceph source packages 4.4 kB/s | 1.5 kB 00:00 2023-09-09T23:02:03.204 INFO:teuthology.orchestra.run.smithi082.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-09-09T23:02:04.952 INFO:teuthology.orchestra.run.smithi170.stdout:Extra Packages for Enterprise Linux 8.1 MB/s | 16 MB 00:01 2023-09-09T23:02:05.306 INFO:teuthology.orchestra.run.smithi098.stdout:Extra Packages for Enterprise Linux 6.9 MB/s | 16 MB 00:02 2023-09-09T23:02:07.090 INFO:teuthology.orchestra.run.smithi082.stdout:Extra Packages for Enterprise Linux 4.1 MB/s | 16 MB 00:03 2023-09-09T23:02:08.402 INFO:teuthology.orchestra.run.smithi170.stdout:lab-extras 395 kB/s | 24 kB 00:00 2023-09-09T23:02:08.756 INFO:teuthology.orchestra.run.smithi098.stdout:lab-extras 469 kB/s | 24 kB 00:00 2023-09-09T23:02:09.395 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 58 MB/s | 53 MB 00:00 2023-09-09T23:02:09.942 INFO:teuthology.orchestra.run.smithi098.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 48 MB/s | 53 MB 00:01 2023-09-09T23:02:10.587 INFO:teuthology.orchestra.run.smithi082.stdout:lab-extras 571 kB/s | 24 kB 00:00 2023-09-09T23:02:11.631 INFO:teuthology.orchestra.run.smithi082.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 49 MB/s | 47 MB 00:00 2023-09-09T23:02:17.672 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 50 MB/s | 47 MB 00:00 2023-09-09T23:02:18.568 INFO:teuthology.orchestra.run.smithi098.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 41 MB/s | 47 MB 00:01 2023-09-09T23:02:19.889 INFO:teuthology.orchestra.run.smithi082.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 45 MB/s | 53 MB 00:01 2023-09-09T23:02:28.590 INFO:teuthology.orchestra.run.smithi170.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-09T23:02:28.590 INFO:teuthology.orchestra.run.smithi170.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-09T23:02:28.605 INFO:teuthology.orchestra.run.smithi170.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-09-09T23:02:28.606 INFO:teuthology.orchestra.run.smithi170.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-09-09T23:02:28.607 INFO:teuthology.orchestra.run.smithi170.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-09-09T23:02:28.609 INFO:teuthology.orchestra.run.smithi170.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-09-09T23:02:28.611 INFO:teuthology.orchestra.run.smithi170.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-09-09T23:02:28.614 INFO:teuthology.orchestra.run.smithi170.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-09-09T23:02:28.615 INFO:teuthology.orchestra.run.smithi170.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-09-09T23:02:28.749 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2023-09-09T23:02:28.761 INFO:teuthology.orchestra.run.smithi170.stdout:=========================================================================================================================== 2023-09-09T23:02:28.762 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2023-09-09T23:02:28.762 INFO:teuthology.orchestra.run.smithi170.stdout:=========================================================================================================================== 2023-09-09T23:02:28.762 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2023-09-09T23:02:28.762 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-09-09T23:02:28.763 INFO:teuthology.orchestra.run.smithi170.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-09-09T23:02:28.763 INFO:teuthology.orchestra.run.smithi170.stdout: ceph x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 6.4 k 2023-09-09T23:02:28.763 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 5.4 M 2023-09-09T23:02:28.763 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 907 k 2023-09-09T23:02:28.763 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 167 k 2023-09-09T23:02:28.764 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 1.6 M 2023-09-09T23:02:28.764 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 132 k 2023-09-09T23:02:28.764 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 1.7 M 2023-09-09T23:02:28.764 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 7.4 M 2023-09-09T23:02:28.764 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 47 k 2023-09-09T23:02:28.765 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 14 M 2023-09-09T23:02:28.765 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 49 M 2023-09-09T23:02:28.765 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 261 k 2023-09-09T23:02:28.765 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 176 k 2023-09-09T23:02:28.765 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-mirror x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 260 k 2023-09-09T23:02:28.766 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-top noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 18 k 2023-09-09T23:02:28.766 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-09-09T23:02:28.766 INFO:teuthology.orchestra.run.smithi170.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-09-09T23:02:28.766 INFO:teuthology.orchestra.run.smithi170.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-09-09T23:02:28.766 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 31 k 2023-09-09T23:02:28.767 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 827 k 2023-09-09T23:02:28.767 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 139 k 2023-09-09T23:02:28.767 INFO:teuthology.orchestra.run.smithi170.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-09-09T23:02:28.767 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-09-09T23:02:28.767 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-09-09T23:02:28.768 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-09-09T23:02:28.768 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-09-09T23:02:28.768 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-09-09T23:02:28.768 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 197 k 2023-09-09T23:02:28.768 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 372 k 2023-09-09T23:02:28.769 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 364 k 2023-09-09T23:02:28.769 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 107 k 2023-09-09T23:02:28.769 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 94 k 2023-09-09T23:02:28.769 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 3.6 M 2023-09-09T23:02:28.769 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 179 k 2023-09-09T23:02:28.769 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-09-09T23:02:28.770 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrading: 2023-09-09T23:02:28.770 INFO:teuthology.orchestra.run.smithi170.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-09-09T23:02:28.770 INFO:teuthology.orchestra.run.smithi170.stdout: librados2 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 3.7 M 2023-09-09T23:02:28.770 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 3.7 M 2023-09-09T23:02:28.771 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2023-09-09T23:02:28.771 INFO:teuthology.orchestra.run.smithi170.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-09-09T23:02:28.771 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 22 M 2023-09-09T23:02:28.771 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 23 k 2023-09-09T23:02:28.771 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 2.3 M 2023-09-09T23:02:28.771 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 242 k 2023-09-09T23:02:28.772 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 4.6 M 2023-09-09T23:02:28.772 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 18 M 2023-09-09T23:02:28.772 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 15 k 2023-09-09T23:02:28.772 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 24 k 2023-09-09T23:02:28.772 INFO:teuthology.orchestra.run.smithi170.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-09-09T23:02:28.773 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-09T23:02:28.773 INFO:teuthology.orchestra.run.smithi170.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-09-09T23:02:28.773 INFO:teuthology.orchestra.run.smithi170.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-09-09T23:02:28.773 INFO:teuthology.orchestra.run.smithi170.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-09-09T23:02:28.773 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 184 k 2023-09-09T23:02:28.774 INFO:teuthology.orchestra.run.smithi170.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-09-09T23:02:28.774 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-09-09T23:02:28.774 INFO:teuthology.orchestra.run.smithi170.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-09T23:02:28.774 INFO:teuthology.orchestra.run.smithi170.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-09-09T23:02:28.774 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-09-09T23:02:28.774 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-09-09T23:02:28.775 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 519 k 2023-09-09T23:02:28.775 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-09-09T23:02:28.775 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 6.8 M 2023-09-09T23:02:28.775 INFO:teuthology.orchestra.run.smithi170.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-09-09T23:02:28.775 INFO:teuthology.orchestra.run.smithi170.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-09-09T23:02:28.776 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-09T23:02:28.776 INFO:teuthology.orchestra.run.smithi170.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-09-09T23:02:28.776 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-09-09T23:02:28.776 INFO:teuthology.orchestra.run.smithi170.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-09-09T23:02:28.776 INFO:teuthology.orchestra.run.smithi170.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-09-09T23:02:28.776 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-09-09T23:02:28.777 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-09-09T23:02:28.777 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-09-09T23:02:28.777 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-09-09T23:02:28.777 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-09T23:02:28.777 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-09-09T23:02:28.778 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-09T23:02:28.778 INFO:teuthology.orchestra.run.smithi170.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-09T23:02:28.778 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-09T23:02:28.778 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 45 k 2023-09-09T23:02:28.778 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 126 k 2023-09-09T23:02:28.779 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-09T23:02:28.779 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-09-09T23:02:28.779 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-09T23:02:28.779 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-09T23:02:28.779 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-09-09T23:02:28.780 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-09T23:02:28.780 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-09T23:02:28.780 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-09T23:02:28.780 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-09-09T23:02:28.780 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-09-09T23:02:28.780 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-09-09T23:02:28.781 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-09T23:02:28.781 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-09T23:02:28.781 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-09T23:02:28.781 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-09-09T23:02:28.781 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-09-09T23:02:28.781 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-09T23:02:28.782 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-09T23:02:28.782 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-09-09T23:02:28.782 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-09-09T23:02:28.782 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-09-09T23:02:28.782 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-09T23:02:28.783 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-09T23:02:28.783 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-09-09T23:02:28.783 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-09-09T23:02:28.783 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-09-09T23:02:28.783 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-09-09T23:02:28.783 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-09-09T23:02:28.784 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-09-09T23:02:28.784 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-09T23:02:28.784 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-09T23:02:28.784 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-09T23:02:28.784 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-09T23:02:28.785 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-09-09T23:02:28.785 INFO:teuthology.orchestra.run.smithi170.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-09T23:02:28.785 INFO:teuthology.orchestra.run.smithi170.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-09T23:02:28.785 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-09T23:02:28.785 INFO:teuthology.orchestra.run.smithi170.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-09T23:02:28.785 INFO:teuthology.orchestra.run.smithi170.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-09T23:02:28.786 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-09T23:02:28.786 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-09T23:02:28.786 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-09T23:02:28.786 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-09-09T23:02:28.786 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-09T23:02:28.786 INFO:teuthology.orchestra.run.smithi170.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-09-09T23:02:28.787 INFO:teuthology.orchestra.run.smithi170.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-09T23:02:28.787 INFO:teuthology.orchestra.run.smithi170.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-09-09T23:02:28.787 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-09-09T23:02:28.787 INFO:teuthology.orchestra.run.smithi170.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-09-09T23:02:28.787 INFO:teuthology.orchestra.run.smithi170.stdout:Enabling module streams: 2023-09-09T23:02:28.788 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql 10 2023-09-09T23:02:28.788 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-09T23:02:28.788 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2023-09-09T23:02:28.788 INFO:teuthology.orchestra.run.smithi170.stdout:=========================================================================================================================== 2023-09-09T23:02:28.788 INFO:teuthology.orchestra.run.smithi170.stdout:Install 128 Packages 2023-09-09T23:02:28.789 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrade 3 Packages 2023-09-09T23:02:28.789 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-09T23:02:28.789 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 208 M 2023-09-09T23:02:28.789 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2023-09-09T23:02:29.072 INFO:teuthology.orchestra.run.smithi170.stdout:(1/131): ceph-18.0.0-6025.g9670cca5.el8.x86_64. 51 kB/s | 6.4 kB 00:00 2023-09-09T23:02:29.280 INFO:teuthology.orchestra.run.smithi170.stdout:(2/131): ceph-fuse-18.0.0-6025.g9670cca5.el8.x8 4.3 MB/s | 907 kB 00:00 2023-09-09T23:02:29.347 INFO:teuthology.orchestra.run.smithi170.stdout:(3/131): ceph-immutable-object-cache-18.0.0-602 2.4 MB/s | 167 kB 00:00 2023-09-09T23:02:29.448 INFO:teuthology.orchestra.run.smithi170.stdout:(4/131): ceph-base-18.0.0-6025.g9670cca5.el8.x8 11 MB/s | 5.4 MB 00:00 2023-09-09T23:02:29.535 INFO:teuthology.orchestra.run.smithi098.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-09T23:02:29.536 INFO:teuthology.orchestra.run.smithi098.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-09T23:02:29.550 INFO:teuthology.orchestra.run.smithi098.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-09-09T23:02:29.551 INFO:teuthology.orchestra.run.smithi098.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-09-09T23:02:29.552 INFO:teuthology.orchestra.run.smithi098.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-09-09T23:02:29.553 INFO:teuthology.orchestra.run.smithi098.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-09-09T23:02:29.556 INFO:teuthology.orchestra.run.smithi098.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-09-09T23:02:29.559 INFO:teuthology.orchestra.run.smithi098.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-09-09T23:02:29.560 INFO:teuthology.orchestra.run.smithi098.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-09-09T23:02:29.573 INFO:teuthology.orchestra.run.smithi170.stdout:(5/131): ceph-mds-18.0.0-6025.g9670cca5.el8.x86 10 MB/s | 2.3 MB 00:00 2023-09-09T23:02:29.615 INFO:teuthology.orchestra.run.smithi170.stdout:(6/131): ceph-mgr-18.0.0-6025.g9670cca5.el8.x86 9.7 MB/s | 1.6 MB 00:00 2023-09-09T23:02:29.689 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2023-09-09T23:02:29.702 INFO:teuthology.orchestra.run.smithi098.stdout:=========================================================================================================================== 2023-09-09T23:02:29.702 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repository Size 2023-09-09T23:02:29.702 INFO:teuthology.orchestra.run.smithi098.stdout:=========================================================================================================================== 2023-09-09T23:02:29.702 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2023-09-09T23:02:29.702 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-09-09T23:02:29.703 INFO:teuthology.orchestra.run.smithi098.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-09-09T23:02:29.703 INFO:teuthology.orchestra.run.smithi098.stdout: ceph x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 6.4 k 2023-09-09T23:02:29.703 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-base x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 5.4 M 2023-09-09T23:02:29.703 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-fuse x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 907 k 2023-09-09T23:02:29.703 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 167 k 2023-09-09T23:02:29.704 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 1.6 M 2023-09-09T23:02:29.704 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 132 k 2023-09-09T23:02:29.704 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 1.7 M 2023-09-09T23:02:29.704 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 7.4 M 2023-09-09T23:02:29.704 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-rook noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 47 k 2023-09-09T23:02:29.705 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-radosgw x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 14 M 2023-09-09T23:02:29.705 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-test x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 49 M 2023-09-09T23:02:29.705 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-volume noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 261 k 2023-09-09T23:02:29.705 INFO:teuthology.orchestra.run.smithi098.stdout: cephadm noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 176 k 2023-09-09T23:02:29.705 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-mirror x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 260 k 2023-09-09T23:02:29.705 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-top noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 18 k 2023-09-09T23:02:29.706 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-09-09T23:02:29.706 INFO:teuthology.orchestra.run.smithi098.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-09-09T23:02:29.706 INFO:teuthology.orchestra.run.smithi098.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-09-09T23:02:29.706 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs-devel x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 31 k 2023-09-09T23:02:29.706 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs2 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 827 k 2023-09-09T23:02:29.707 INFO:teuthology.orchestra.run.smithi098.stdout: librados-devel x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 139 k 2023-09-09T23:02:29.707 INFO:teuthology.orchestra.run.smithi098.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-09-09T23:02:29.707 INFO:teuthology.orchestra.run.smithi098.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-09-09T23:02:29.707 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-09-09T23:02:29.707 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-09-09T23:02:29.708 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-09-09T23:02:29.708 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-09-09T23:02:29.708 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cephfs x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 197 k 2023-09-09T23:02:29.708 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rados x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 372 k 2023-09-09T23:02:29.708 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rbd x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 364 k 2023-09-09T23:02:29.709 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rgw x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 107 k 2023-09-09T23:02:29.709 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-fuse x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 94 k 2023-09-09T23:02:29.709 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-mirror x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 3.6 M 2023-09-09T23:02:29.709 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-nbd x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 179 k 2023-09-09T23:02:29.709 INFO:teuthology.orchestra.run.smithi098.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-09-09T23:02:29.710 INFO:teuthology.orchestra.run.smithi098.stdout:Upgrading: 2023-09-09T23:02:29.710 INFO:teuthology.orchestra.run.smithi098.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-09-09T23:02:29.710 INFO:teuthology.orchestra.run.smithi098.stdout: librados2 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 3.7 M 2023-09-09T23:02:29.710 INFO:teuthology.orchestra.run.smithi098.stdout: librbd1 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 3.7 M 2023-09-09T23:02:29.710 INFO:teuthology.orchestra.run.smithi098.stdout:Installing dependencies: 2023-09-09T23:02:29.711 INFO:teuthology.orchestra.run.smithi098.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-09-09T23:02:29.711 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-common x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 22 M 2023-09-09T23:02:29.711 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 23 k 2023-09-09T23:02:29.711 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mds x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 2.3 M 2023-09-09T23:02:29.711 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 242 k 2023-09-09T23:02:29.711 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mon x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 4.6 M 2023-09-09T23:02:29.712 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-osd x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 18 M 2023-09-09T23:02:29.712 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 15 k 2023-09-09T23:02:29.712 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-selinux x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 24 k 2023-09-09T23:02:29.712 INFO:teuthology.orchestra.run.smithi098.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-09-09T23:02:29.712 INFO:teuthology.orchestra.run.smithi098.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-09T23:02:29.713 INFO:teuthology.orchestra.run.smithi098.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-09-09T23:02:29.713 INFO:teuthology.orchestra.run.smithi098.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-09-09T23:02:29.713 INFO:teuthology.orchestra.run.smithi098.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-09-09T23:02:29.713 INFO:teuthology.orchestra.run.smithi098.stdout: libcephsqlite x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 184 k 2023-09-09T23:02:29.713 INFO:teuthology.orchestra.run.smithi098.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-09-09T23:02:29.714 INFO:teuthology.orchestra.run.smithi098.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-09-09T23:02:29.714 INFO:teuthology.orchestra.run.smithi098.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-09T23:02:29.714 INFO:teuthology.orchestra.run.smithi098.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-09-09T23:02:29.714 INFO:teuthology.orchestra.run.smithi098.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-09-09T23:02:29.714 INFO:teuthology.orchestra.run.smithi098.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-09-09T23:02:29.714 INFO:teuthology.orchestra.run.smithi098.stdout: libradosstriper1 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 519 k 2023-09-09T23:02:29.715 INFO:teuthology.orchestra.run.smithi098.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-09-09T23:02:29.715 INFO:teuthology.orchestra.run.smithi098.stdout: librgw2 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 6.8 M 2023-09-09T23:02:29.715 INFO:teuthology.orchestra.run.smithi098.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-09-09T23:02:29.715 INFO:teuthology.orchestra.run.smithi098.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-09-09T23:02:29.715 INFO:teuthology.orchestra.run.smithi098.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-09T23:02:29.716 INFO:teuthology.orchestra.run.smithi098.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-09-09T23:02:29.716 INFO:teuthology.orchestra.run.smithi098.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-09-09T23:02:29.716 INFO:teuthology.orchestra.run.smithi098.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-09-09T23:02:29.716 INFO:teuthology.orchestra.run.smithi098.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-09-09T23:02:29.716 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-09-09T23:02:29.716 INFO:teuthology.orchestra.run.smithi098.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-09-09T23:02:29.717 INFO:teuthology.orchestra.run.smithi098.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-09-09T23:02:29.717 INFO:teuthology.orchestra.run.smithi098.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-09-09T23:02:29.717 INFO:teuthology.orchestra.run.smithi098.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-09T23:02:29.717 INFO:teuthology.orchestra.run.smithi098.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-09-09T23:02:29.717 INFO:teuthology.orchestra.run.smithi098.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-09T23:02:29.718 INFO:teuthology.orchestra.run.smithi098.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-09T23:02:29.718 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-09T23:02:29.718 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-argparse x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 45 k 2023-09-09T23:02:29.718 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-common x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 126 k 2023-09-09T23:02:29.718 INFO:teuthology.orchestra.run.smithi098.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-09T23:02:29.719 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-09-09T23:02:29.719 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-09T23:02:29.719 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-09T23:02:29.719 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-09-09T23:02:29.719 INFO:teuthology.orchestra.run.smithi098.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-09T23:02:29.720 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-09T23:02:29.720 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-09T23:02:29.720 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-09-09T23:02:29.720 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-09-09T23:02:29.720 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-09-09T23:02:29.720 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-09T23:02:29.721 INFO:teuthology.orchestra.run.smithi098.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-09T23:02:29.721 INFO:teuthology.orchestra.run.smithi098.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-09T23:02:29.721 INFO:teuthology.orchestra.run.smithi098.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-09-09T23:02:29.721 INFO:teuthology.orchestra.run.smithi098.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-09-09T23:02:29.721 INFO:teuthology.orchestra.run.smithi098.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-09T23:02:29.722 INFO:teuthology.orchestra.run.smithi098.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-09T23:02:29.722 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-09-09T23:02:29.722 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-09-09T23:02:29.722 INFO:teuthology.orchestra.run.smithi098.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-09-09T23:02:29.722 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-09T23:02:29.723 INFO:teuthology.orchestra.run.smithi098.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-09T23:02:29.723 INFO:teuthology.orchestra.run.smithi098.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-09-09T23:02:29.723 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-09-09T23:02:29.723 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-09-09T23:02:29.723 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-09-09T23:02:29.724 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-09-09T23:02:29.724 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-09-09T23:02:29.724 INFO:teuthology.orchestra.run.smithi098.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-09T23:02:29.724 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-09T23:02:29.724 INFO:teuthology.orchestra.run.smithi098.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-09T23:02:29.725 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-09T23:02:29.725 INFO:teuthology.orchestra.run.smithi098.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-09-09T23:02:29.725 INFO:teuthology.orchestra.run.smithi098.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-09T23:02:29.725 INFO:teuthology.orchestra.run.smithi098.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-09T23:02:29.725 INFO:teuthology.orchestra.run.smithi098.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-09T23:02:29.726 INFO:teuthology.orchestra.run.smithi098.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-09T23:02:29.726 INFO:teuthology.orchestra.run.smithi098.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-09T23:02:29.726 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-09T23:02:29.726 INFO:teuthology.orchestra.run.smithi098.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-09T23:02:29.727 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-09T23:02:29.727 INFO:teuthology.orchestra.run.smithi098.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-09-09T23:02:29.727 INFO:teuthology.orchestra.run.smithi098.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-09T23:02:29.727 INFO:teuthology.orchestra.run.smithi098.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-09-09T23:02:29.727 INFO:teuthology.orchestra.run.smithi098.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-09T23:02:29.727 INFO:teuthology.orchestra.run.smithi098.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-09-09T23:02:29.728 INFO:teuthology.orchestra.run.smithi098.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-09-09T23:02:29.728 INFO:teuthology.orchestra.run.smithi098.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-09-09T23:02:29.728 INFO:teuthology.orchestra.run.smithi098.stdout:Enabling module streams: 2023-09-09T23:02:29.728 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql 10 2023-09-09T23:02:29.728 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-09-09T23:02:29.729 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2023-09-09T23:02:29.729 INFO:teuthology.orchestra.run.smithi098.stdout:=========================================================================================================================== 2023-09-09T23:02:29.729 INFO:teuthology.orchestra.run.smithi098.stdout:Install 128 Packages 2023-09-09T23:02:29.729 INFO:teuthology.orchestra.run.smithi098.stdout:Upgrade 3 Packages 2023-09-09T23:02:29.729 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-09-09T23:02:29.730 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 208 M 2023-09-09T23:02:29.730 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2023-09-09T23:02:30.024 INFO:teuthology.orchestra.run.smithi170.stdout:(7/131): ceph-mon-18.0.0-6025.g9670cca5.el8.x86 10 MB/s | 4.6 MB 00:00 2023-09-09T23:02:30.035 INFO:teuthology.orchestra.run.smithi098.stdout:(1/131): ceph-18.0.0-6025.g9670cca5.el8.x86_64. 22 kB/s | 6.4 kB 00:00 2023-09-09T23:02:30.928 INFO:teuthology.orchestra.run.smithi098.stdout:(2/131): ceph-fuse-18.0.0-6025.g9670cca5.el8.x8 1.0 MB/s | 907 kB 00:00 2023-09-09T23:02:31.011 INFO:teuthology.orchestra.run.smithi098.stdout:(3/131): ceph-immutable-object-cache-18.0.0-602 2.0 MB/s | 167 kB 00:00 2023-09-09T23:02:31.024 INFO:teuthology.orchestra.run.smithi170.stdout:(8/131): ceph-osd-18.0.0-6025.g9670cca5.el8.x86 13 MB/s | 18 MB 00:01 2023-09-09T23:02:31.050 INFO:teuthology.orchestra.run.smithi082.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-09T23:02:31.051 INFO:teuthology.orchestra.run.smithi082.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-09T23:02:31.066 INFO:teuthology.orchestra.run.smithi082.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-09-09T23:02:31.067 INFO:teuthology.orchestra.run.smithi082.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-09-09T23:02:31.068 INFO:teuthology.orchestra.run.smithi082.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-09-09T23:02:31.069 INFO:teuthology.orchestra.run.smithi082.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-09-09T23:02:31.071 INFO:teuthology.orchestra.run.smithi082.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-09-09T23:02:31.074 INFO:teuthology.orchestra.run.smithi082.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-09-09T23:02:31.075 INFO:teuthology.orchestra.run.smithi082.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-09-09T23:02:31.091 INFO:teuthology.orchestra.run.smithi170.stdout:(9/131): ceph-selinux-18.0.0-6025.g9670cca5.el8 365 kB/s | 24 kB 00:00 2023-09-09T23:02:31.210 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-09-09T23:02:31.222 INFO:teuthology.orchestra.run.smithi082.stdout:=========================================================================================================================== 2023-09-09T23:02:31.222 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2023-09-09T23:02:31.222 INFO:teuthology.orchestra.run.smithi082.stdout:=========================================================================================================================== 2023-09-09T23:02:31.222 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2023-09-09T23:02:31.223 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-09-09T23:02:31.223 INFO:teuthology.orchestra.run.smithi082.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-09-09T23:02:31.223 INFO:teuthology.orchestra.run.smithi082.stdout: ceph x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 6.4 k 2023-09-09T23:02:31.223 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 5.4 M 2023-09-09T23:02:31.223 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-fuse x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 907 k 2023-09-09T23:02:31.224 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 167 k 2023-09-09T23:02:31.224 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 1.6 M 2023-09-09T23:02:31.224 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 132 k 2023-09-09T23:02:31.224 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 1.7 M 2023-09-09T23:02:31.224 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 7.4 M 2023-09-09T23:02:31.225 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-rook noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 47 k 2023-09-09T23:02:31.225 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-radosgw x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 14 M 2023-09-09T23:02:31.225 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-test x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 49 M 2023-09-09T23:02:31.225 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-volume noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 261 k 2023-09-09T23:02:31.225 INFO:teuthology.orchestra.run.smithi082.stdout: cephadm noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 176 k 2023-09-09T23:02:31.226 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-mirror x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 260 k 2023-09-09T23:02:31.226 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-top noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 18 k 2023-09-09T23:02:31.226 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-09-09T23:02:31.226 INFO:teuthology.orchestra.run.smithi082.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-09-09T23:02:31.226 INFO:teuthology.orchestra.run.smithi082.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-09-09T23:02:31.227 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs-devel x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 31 k 2023-09-09T23:02:31.227 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs2 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 827 k 2023-09-09T23:02:31.227 INFO:teuthology.orchestra.run.smithi082.stdout: librados-devel x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 139 k 2023-09-09T23:02:31.227 INFO:teuthology.orchestra.run.smithi082.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-09-09T23:02:31.228 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-09-09T23:02:31.228 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-09-09T23:02:31.228 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-09-09T23:02:31.228 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-09-09T23:02:31.228 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-09-09T23:02:31.229 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cephfs x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 197 k 2023-09-09T23:02:31.229 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 372 k 2023-09-09T23:02:31.229 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rbd x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 364 k 2023-09-09T23:02:31.229 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rgw x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 107 k 2023-09-09T23:02:31.229 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-fuse x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 94 k 2023-09-09T23:02:31.229 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-mirror x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 3.6 M 2023-09-09T23:02:31.230 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-nbd x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 179 k 2023-09-09T23:02:31.230 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-09-09T23:02:31.230 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrading: 2023-09-09T23:02:31.230 INFO:teuthology.orchestra.run.smithi082.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-09-09T23:02:31.230 INFO:teuthology.orchestra.run.smithi082.stdout: librados2 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 3.7 M 2023-09-09T23:02:31.231 INFO:teuthology.orchestra.run.smithi082.stdout: librbd1 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 3.7 M 2023-09-09T23:02:31.231 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2023-09-09T23:02:31.231 INFO:teuthology.orchestra.run.smithi082.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-09-09T23:02:31.231 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-common x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 22 M 2023-09-09T23:02:31.231 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 23 k 2023-09-09T23:02:31.232 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mds x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 2.3 M 2023-09-09T23:02:31.232 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 242 k 2023-09-09T23:02:31.232 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mon x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 4.6 M 2023-09-09T23:02:31.232 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-osd x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 18 M 2023-09-09T23:02:31.232 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6025.g9670cca5.el8 ceph-noarch 15 k 2023-09-09T23:02:31.233 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-selinux x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 24 k 2023-09-09T23:02:31.233 INFO:teuthology.orchestra.run.smithi082.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-09-09T23:02:31.233 INFO:teuthology.orchestra.run.smithi082.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-09T23:02:31.233 INFO:teuthology.orchestra.run.smithi082.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-09-09T23:02:31.233 INFO:teuthology.orchestra.run.smithi082.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-09-09T23:02:31.234 INFO:teuthology.orchestra.run.smithi082.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-09-09T23:02:31.234 INFO:teuthology.orchestra.run.smithi082.stdout: libcephsqlite x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 184 k 2023-09-09T23:02:31.234 INFO:teuthology.orchestra.run.smithi082.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-09-09T23:02:31.234 INFO:teuthology.orchestra.run.smithi082.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-09-09T23:02:31.234 INFO:teuthology.orchestra.run.smithi082.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-09T23:02:31.235 INFO:teuthology.orchestra.run.smithi082.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-09-09T23:02:31.235 INFO:teuthology.orchestra.run.smithi082.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-09-09T23:02:31.235 INFO:teuthology.orchestra.run.smithi082.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-09-09T23:02:31.235 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 519 k 2023-09-09T23:02:31.235 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-09-09T23:02:31.236 INFO:teuthology.orchestra.run.smithi082.stdout: librgw2 x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 6.8 M 2023-09-09T23:02:31.236 INFO:teuthology.orchestra.run.smithi082.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-09-09T23:02:31.236 INFO:teuthology.orchestra.run.smithi082.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-09-09T23:02:31.236 INFO:teuthology.orchestra.run.smithi082.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-09T23:02:31.236 INFO:teuthology.orchestra.run.smithi082.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-09-09T23:02:31.237 INFO:teuthology.orchestra.run.smithi082.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-09-09T23:02:31.237 INFO:teuthology.orchestra.run.smithi082.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-09-09T23:02:31.237 INFO:teuthology.orchestra.run.smithi082.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-09-09T23:02:31.237 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-09-09T23:02:31.237 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-09-09T23:02:31.238 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-09-09T23:02:31.238 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-09-09T23:02:31.238 INFO:teuthology.orchestra.run.smithi082.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-09T23:02:31.238 INFO:teuthology.orchestra.run.smithi082.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-09-09T23:02:31.238 INFO:teuthology.orchestra.run.smithi082.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-09T23:02:31.238 INFO:teuthology.orchestra.run.smithi082.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-09T23:02:31.239 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-09T23:02:31.239 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 45 k 2023-09-09T23:02:31.239 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-common x86_64 2:18.0.0-6025.g9670cca5.el8 ceph 126 k 2023-09-09T23:02:31.239 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-09T23:02:31.239 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-09-09T23:02:31.240 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-09T23:02:31.240 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-09T23:02:31.240 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-09-09T23:02:31.240 INFO:teuthology.orchestra.run.smithi082.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-09T23:02:31.240 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-09T23:02:31.241 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-09T23:02:31.241 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-09-09T23:02:31.241 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-09-09T23:02:31.241 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-09-09T23:02:31.241 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-09T23:02:31.242 INFO:teuthology.orchestra.run.smithi082.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-09T23:02:31.242 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-09T23:02:31.242 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-09-09T23:02:31.242 INFO:teuthology.orchestra.run.smithi082.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-09-09T23:02:31.242 INFO:teuthology.orchestra.run.smithi082.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-09T23:02:31.242 INFO:teuthology.orchestra.run.smithi082.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-09T23:02:31.243 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-09-09T23:02:31.243 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-09-09T23:02:31.243 INFO:teuthology.orchestra.run.smithi082.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-09-09T23:02:31.243 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-09T23:02:31.244 INFO:teuthology.orchestra.run.smithi082.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-09T23:02:31.244 INFO:teuthology.orchestra.run.smithi082.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-09-09T23:02:31.244 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-09-09T23:02:31.244 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-09-09T23:02:31.244 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-09-09T23:02:31.245 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-09-09T23:02:31.245 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-09-09T23:02:31.245 INFO:teuthology.orchestra.run.smithi082.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-09T23:02:31.245 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-09T23:02:31.245 INFO:teuthology.orchestra.run.smithi082.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-09T23:02:31.246 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-09T23:02:31.246 INFO:teuthology.orchestra.run.smithi082.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-09-09T23:02:31.246 INFO:teuthology.orchestra.run.smithi082.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-09T23:02:31.246 INFO:teuthology.orchestra.run.smithi082.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-09T23:02:31.246 INFO:teuthology.orchestra.run.smithi082.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-09T23:02:31.247 INFO:teuthology.orchestra.run.smithi082.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-09T23:02:31.247 INFO:teuthology.orchestra.run.smithi082.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-09T23:02:31.247 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-09T23:02:31.247 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-09T23:02:31.247 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-09T23:02:31.247 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-09-09T23:02:31.248 INFO:teuthology.orchestra.run.smithi082.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-09T23:02:31.248 INFO:teuthology.orchestra.run.smithi082.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-09-09T23:02:31.248 INFO:teuthology.orchestra.run.smithi082.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-09T23:02:31.248 INFO:teuthology.orchestra.run.smithi082.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-09-09T23:02:31.248 INFO:teuthology.orchestra.run.smithi082.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-09-09T23:02:31.249 INFO:teuthology.orchestra.run.smithi082.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-09-09T23:02:31.249 INFO:teuthology.orchestra.run.smithi082.stdout:Enabling module streams: 2023-09-09T23:02:31.249 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql 10 2023-09-09T23:02:31.249 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:02:31.249 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2023-09-09T23:02:31.249 INFO:teuthology.orchestra.run.smithi082.stdout:=========================================================================================================================== 2023-09-09T23:02:31.250 INFO:teuthology.orchestra.run.smithi082.stdout:Install 128 Packages 2023-09-09T23:02:31.250 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrade 3 Packages 2023-09-09T23:02:31.250 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:02:31.251 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 208 M 2023-09-09T23:02:31.251 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2023-09-09T23:02:31.450 INFO:teuthology.orchestra.run.smithi170.stdout:(10/131): ceph-radosgw-18.0.0-6025.g9670cca5.el 9.6 MB/s | 14 MB 00:01 2023-09-09T23:02:31.533 INFO:teuthology.orchestra.run.smithi170.stdout:(11/131): cephfs-mirror-18.0.0-6025.g9670cca5.e 3.1 MB/s | 260 kB 00:00 2023-09-09T23:02:31.572 INFO:teuthology.orchestra.run.smithi082.stdout:(1/131): ceph-18.0.0-6025.g9670cca5.el8.x86_64. 24 kB/s | 6.4 kB 00:00 2023-09-09T23:02:31.608 INFO:teuthology.orchestra.run.smithi170.stdout:(12/131): libcephfs-devel-18.0.0-6025.g9670cca5 415 kB/s | 31 kB 00:00 2023-09-09T23:02:31.784 INFO:teuthology.orchestra.run.smithi170.stdout:(13/131): libcephfs2-18.0.0-6025.g9670cca5.el8. 4.6 MB/s | 827 kB 00:00 2023-09-09T23:02:31.828 INFO:teuthology.orchestra.run.smithi098.stdout:(4/131): ceph-mds-18.0.0-6025.g9670cca5.el8.x86 2.9 MB/s | 2.3 MB 00:00 2023-09-09T23:02:31.859 INFO:teuthology.orchestra.run.smithi170.stdout:(14/131): libcephsqlite-18.0.0-6025.g9670cca5.e 2.4 MB/s | 184 kB 00:00 2023-09-09T23:02:31.943 INFO:teuthology.orchestra.run.smithi170.stdout:(15/131): librados-devel-18.0.0-6025.g9670cca5. 1.6 MB/s | 139 kB 00:00 2023-09-09T23:02:32.135 INFO:teuthology.orchestra.run.smithi170.stdout:(16/131): libradosstriper1-18.0.0-6025.g9670cca 2.6 MB/s | 519 kB 00:00 2023-09-09T23:02:32.412 INFO:teuthology.orchestra.run.smithi098.stdout:(5/131): ceph-mgr-18.0.0-6025.g9670cca5.el8.x86 2.8 MB/s | 1.6 MB 00:00 2023-09-09T23:02:32.617 INFO:teuthology.orchestra.run.smithi082.stdout:(2/131): ceph-fuse-18.0.0-6025.g9670cca5.el8.x8 868 kB/s | 907 kB 00:01 2023-09-09T23:02:32.823 INFO:teuthology.orchestra.run.smithi082.stdout:(3/131): ceph-immutable-object-cache-18.0.0-602 815 kB/s | 167 kB 00:00 2023-09-09T23:02:33.210 INFO:teuthology.orchestra.run.smithi170.stdout:(17/131): librgw2-18.0.0-6025.g9670cca5.el8.x86 6.3 MB/s | 6.8 MB 00:01 2023-09-09T23:02:33.286 INFO:teuthology.orchestra.run.smithi170.stdout:(18/131): python3-ceph-argparse-18.0.0-6025.g96 598 kB/s | 45 kB 00:00 2023-09-09T23:02:33.416 INFO:teuthology.orchestra.run.smithi170.stdout:(19/131): python3-ceph-common-18.0.0-6025.g9670 969 kB/s | 126 kB 00:00 2023-09-09T23:02:33.519 INFO:teuthology.orchestra.run.smithi170.stdout:(20/131): python3-cephfs-18.0.0-6025.g9670cca5. 1.9 MB/s | 197 kB 00:00 2023-09-09T23:02:33.553 INFO:teuthology.orchestra.run.smithi098.stdout:(6/131): ceph-base-18.0.0-6025.g9670cca5.el8.x8 1.4 MB/s | 5.4 MB 00:03 2023-09-09T23:02:33.603 INFO:teuthology.orchestra.run.smithi170.stdout:(21/131): python3-rados-18.0.0-6025.g9670cca5.e 4.4 MB/s | 372 kB 00:00 2023-09-09T23:02:33.979 INFO:teuthology.orchestra.run.smithi098.stdout:(7/131): ceph-mon-18.0.0-6025.g9670cca5.el8.x86 2.9 MB/s | 4.6 MB 00:01 2023-09-09T23:02:34.020 INFO:teuthology.orchestra.run.smithi170.stdout:(22/131): python3-rbd-18.0.0-6025.g9670cca5.el8 874 kB/s | 364 kB 00:00 2023-09-09T23:02:34.048 INFO:teuthology.orchestra.run.smithi082.stdout:(4/131): ceph-mds-18.0.0-6025.g9670cca5.el8.x86 1.9 MB/s | 2.3 MB 00:01 2023-09-09T23:02:34.110 INFO:teuthology.orchestra.run.smithi170.stdout:(23/131): python3-rgw-18.0.0-6025.g9670cca5.el8 1.2 MB/s | 107 kB 00:00 2023-09-09T23:02:34.140 INFO:teuthology.orchestra.run.smithi082.stdout:(5/131): ceph-base-18.0.0-6025.g9670cca5.el8.x8 1.9 MB/s | 5.4 MB 00:02 2023-09-09T23:02:34.179 INFO:teuthology.orchestra.run.smithi170.stdout:(24/131): rbd-fuse-18.0.0-6025.g9670cca5.el8.x8 1.3 MB/s | 94 kB 00:00 2023-09-09T23:02:34.623 INFO:teuthology.orchestra.run.smithi082.stdout:(6/131): ceph-mgr-18.0.0-6025.g9670cca5.el8.x86 2.8 MB/s | 1.6 MB 00:00 2023-09-09T23:02:35.087 INFO:teuthology.orchestra.run.smithi170.stdout:(25/131): rbd-mirror-18.0.0-6025.g9670cca5.el8. 3.9 MB/s | 3.6 MB 00:00 2023-09-09T23:02:35.180 INFO:teuthology.orchestra.run.smithi170.stdout:(26/131): rbd-nbd-18.0.0-6025.g9670cca5.el8.x86 1.9 MB/s | 179 kB 00:00 2023-09-09T23:02:35.263 INFO:teuthology.orchestra.run.smithi170.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6025.g 277 kB/s | 23 kB 00:00 2023-09-09T23:02:35.380 INFO:teuthology.orchestra.run.smithi170.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6025.g9670cca 1.1 MB/s | 132 kB 00:00 2023-09-09T23:02:35.712 INFO:teuthology.orchestra.run.smithi098.stdout:(8/131): ceph-common-18.0.0-6025.g9670cca5.el8. 3.8 MB/s | 22 MB 00:05 2023-09-09T23:02:35.806 INFO:teuthology.orchestra.run.smithi170.stdout:(29/131): ceph-mgr-dashboard-18.0.0-6025.g9670c 4.1 MB/s | 1.7 MB 00:00 2023-09-09T23:02:35.853 INFO:teuthology.orchestra.run.smithi098.stdout:(9/131): ceph-selinux-18.0.0-6025.g9670cca5.el8 174 kB/s | 24 kB 00:00 2023-09-09T23:02:36.624 INFO:teuthology.orchestra.run.smithi082.stdout:(7/131): ceph-mon-18.0.0-6025.g9670cca5.el8.x86 1.9 MB/s | 4.6 MB 00:02 2023-09-09T23:02:36.731 INFO:teuthology.orchestra.run.smithi170.stdout:(30/131): ceph-common-18.0.0-6025.g9670cca5.el8 2.9 MB/s | 22 MB 00:07 2023-09-09T23:02:36.915 INFO:teuthology.orchestra.run.smithi170.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6025.g96 1.3 MB/s | 242 kB 00:00 2023-09-09T23:02:37.023 INFO:teuthology.orchestra.run.smithi170.stdout:(32/131): ceph-mgr-rook-18.0.0-6025.g9670cca5.e 438 kB/s | 47 kB 00:00 2023-09-09T23:02:37.132 INFO:teuthology.orchestra.run.smithi170.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6025.g9 134 kB/s | 15 kB 00:00 2023-09-09T23:02:37.324 INFO:teuthology.orchestra.run.smithi170.stdout:(34/131): ceph-volume-18.0.0-6025.g9670cca5.el8 1.3 MB/s | 261 kB 00:00 2023-09-09T23:02:37.508 INFO:teuthology.orchestra.run.smithi170.stdout:(35/131): cephadm-18.0.0-6025.g9670cca5.el8.noa 961 kB/s | 176 kB 00:00 2023-09-09T23:02:37.583 INFO:teuthology.orchestra.run.smithi170.stdout:(36/131): cephfs-top-18.0.0-6025.g9670cca5.el8. 240 kB/s | 18 kB 00:00 2023-09-09T23:02:37.640 INFO:teuthology.orchestra.run.smithi082.stdout:(8/131): ceph-osd-18.0.0-6025.g9670cca5.el8.x86 5.9 MB/s | 18 MB 00:03 2023-09-09T23:02:37.732 INFO:teuthology.orchestra.run.smithi082.stdout:(9/131): ceph-selinux-18.0.0-6025.g9670cca5.el8 266 kB/s | 24 kB 00:00 2023-09-09T23:02:37.750 INFO:teuthology.orchestra.run.smithi170.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.8 MB/s | 306 kB 00:00 2023-09-09T23:02:37.800 INFO:teuthology.orchestra.run.smithi170.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 1.2 MB/s | 59 kB 00:00 2023-09-09T23:02:37.851 INFO:teuthology.orchestra.run.smithi170.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.5 MB/s | 75 kB 00:00 2023-09-09T23:02:37.930 INFO:teuthology.orchestra.run.smithi098.stdout:(10/131): ceph-radosgw-18.0.0-6025.g9670cca5.el 3.5 MB/s | 14 MB 00:03 2023-09-09T23:02:37.951 INFO:teuthology.orchestra.run.smithi170.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 4.6 MB/s | 476 kB 00:00 2023-09-09T23:02:38.001 INFO:teuthology.orchestra.run.smithi170.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 888 kB/s | 44 kB 00:00 2023-09-09T23:02:38.051 INFO:teuthology.orchestra.run.smithi170.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 3.6 MB/s | 185 kB 00:00 2023-09-09T23:02:38.080 INFO:teuthology.orchestra.run.smithi098.stdout:(11/131): cephfs-mirror-18.0.0-6025.g9670cca5.e 1.7 MB/s | 260 kB 00:00 2023-09-09T23:02:38.093 INFO:teuthology.orchestra.run.smithi170.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 790 kB/s | 33 kB 00:00 2023-09-09T23:02:38.135 INFO:teuthology.orchestra.run.smithi170.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 392 kB/s | 16 kB 00:00 2023-09-09T23:02:38.163 INFO:teuthology.orchestra.run.smithi098.stdout:(12/131): libcephfs-devel-18.0.0-6025.g9670cca5 373 kB/s | 31 kB 00:00 2023-09-09T23:02:38.186 INFO:teuthology.orchestra.run.smithi170.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 3.4 MB/s | 173 kB 00:00 2023-09-09T23:02:38.253 INFO:teuthology.orchestra.run.smithi170.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 5.6 MB/s | 384 kB 00:00 2023-09-09T23:02:38.295 INFO:teuthology.orchestra.run.smithi170.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.9 MB/s | 81 kB 00:00 2023-09-09T23:02:38.328 INFO:teuthology.orchestra.run.smithi170.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 335 kB/s | 11 kB 00:00 2023-09-09T23:02:38.362 INFO:teuthology.orchestra.run.smithi170.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 528 kB/s | 18 kB 00:00 2023-09-09T23:02:38.480 INFO:teuthology.orchestra.run.smithi098.stdout:(13/131): libcephfs2-18.0.0-6025.g9670cca5.el8. 2.6 MB/s | 827 kB 00:00 2023-09-09T23:02:38.487 INFO:teuthology.orchestra.run.smithi170.stdout:(50/131): python3-kubernetes-11.0.0-6.el8.noarc 9.3 MB/s | 1.2 MB 00:00 2023-09-09T23:02:38.521 INFO:teuthology.orchestra.run.smithi170.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 1.4 MB/s | 49 kB 00:00 2023-09-09T23:02:38.579 INFO:teuthology.orchestra.run.smithi170.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 1.0 MB/s | 59 kB 00:00 2023-09-09T23:02:38.591 INFO:teuthology.orchestra.run.smithi098.stdout:(14/131): libcephsqlite-18.0.0-6025.g9670cca5.e 1.6 MB/s | 184 kB 00:00 2023-09-09T23:02:38.622 INFO:teuthology.orchestra.run.smithi170.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.4 MB/s | 60 kB 00:00 2023-09-09T23:02:38.672 INFO:teuthology.orchestra.run.smithi170.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 5.5 MB/s | 283 kB 00:00 2023-09-09T23:02:38.681 INFO:teuthology.orchestra.run.smithi098.stdout:(15/131): librados-devel-18.0.0-6025.g9670cca5. 1.5 MB/s | 139 kB 00:00 2023-09-09T23:02:38.714 INFO:teuthology.orchestra.run.smithi170.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 374 kB/s | 16 kB 00:00 2023-09-09T23:02:38.806 INFO:teuthology.orchestra.run.smithi170.stdout:(56/131): ceph-mgr-diskprediction-local-18.0.0- 2.5 MB/s | 7.4 MB 00:02 2023-09-09T23:02:38.831 INFO:teuthology.orchestra.run.smithi170.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 287 kB/s | 34 kB 00:00 2023-09-09T23:02:38.873 INFO:teuthology.orchestra.run.smithi170.stdout:(58/131): python3-rsa-4.9-2.el8.noarch.rpm 1.5 MB/s | 64 kB 00:00 2023-09-09T23:02:38.915 INFO:teuthology.orchestra.run.smithi170.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 451 kB/s | 19 kB 00:00 2023-09-09T23:02:38.953 INFO:teuthology.orchestra.run.smithi098.stdout:(16/131): libradosstriper1-18.0.0-6025.g9670cca 1.9 MB/s | 519 kB 00:00 2023-09-09T23:02:38.957 INFO:teuthology.orchestra.run.smithi170.stdout:(60/131): python3-singledispatch-3.4.0.3-18.el8 576 kB/s | 24 kB 00:00 2023-09-09T23:02:38.983 INFO:teuthology.orchestra.run.smithi170.stdout:(61/131): python3-routes-2.4.1-12.el8.noarch.rp 1.1 MB/s | 196 kB 00:00 2023-09-09T23:02:39.008 INFO:teuthology.orchestra.run.smithi170.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 582 kB/s | 29 kB 00:00 2023-09-09T23:02:39.033 INFO:teuthology.orchestra.run.smithi170.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 550 kB/s | 27 kB 00:00 2023-09-09T23:02:39.058 INFO:teuthology.orchestra.run.smithi170.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 5.0 MB/s | 255 kB 00:00 2023-09-09T23:02:39.100 INFO:teuthology.orchestra.run.smithi170.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 3.7 MB/s | 251 kB 00:00 2023-09-09T23:02:39.126 INFO:teuthology.orchestra.run.smithi170.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 916 kB/s | 61 kB 00:00 2023-09-09T23:02:39.143 INFO:teuthology.orchestra.run.smithi170.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 2.0 MB/s | 85 kB 00:00 2023-09-09T23:02:39.168 INFO:teuthology.orchestra.run.smithi170.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 546 kB/s | 23 kB 00:00 2023-09-09T23:02:39.193 INFO:teuthology.orchestra.run.smithi170.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-09-09T23:02:39.252 INFO:teuthology.orchestra.run.smithi170.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 802 kB/s | 47 kB 00:00 2023-09-09T23:02:39.294 INFO:teuthology.orchestra.run.smithi170.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 4.7 MB/s | 200 kB 00:00 2023-09-09T23:02:39.336 INFO:teuthology.orchestra.run.smithi170.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-09-09T23:02:39.770 INFO:teuthology.orchestra.run.smithi170.stdout:(73/131): ceph-test-18.0.0-6025.g9670cca5.el8.x 5.6 MB/s | 49 MB 00:08 2023-09-09T23:02:39.812 INFO:teuthology.orchestra.run.smithi170.stdout:(74/131): thrift-0.13.0-2.el8.x86_64.rpm 2.6 MB/s | 1.7 MB 00:00 2023-09-09T23:02:39.868 INFO:teuthology.orchestra.run.smithi170.stdout:(75/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 414 kB/s | 220 kB 00:00 2023-09-09T23:02:39.904 INFO:teuthology.orchestra.run.smithi170.stdout:(76/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 360 kB/s | 48 kB 00:00 2023-09-09T23:02:39.929 INFO:teuthology.orchestra.run.smithi170.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 774 kB/s | 43 kB 00:00 2023-09-09T23:02:39.954 INFO:teuthology.orchestra.run.smithi170.stdout:(78/131): libsepol-devel-2.9-3.el8.x86_64.rpm 636 kB/s | 87 kB 00:00 2023-09-09T23:02:39.980 INFO:teuthology.orchestra.run.smithi170.stdout:(79/131): python3-pycparser-2.14-14.el8.noarch. 1.4 MB/s | 109 kB 00:00 2023-09-09T23:02:40.005 INFO:teuthology.orchestra.run.smithi170.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.0 MB/s | 229 kB 00:00 2023-09-09T23:02:40.030 INFO:teuthology.orchestra.run.smithi170.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 370 kB/s | 19 kB 00:00 2023-09-09T23:02:40.056 INFO:teuthology.orchestra.run.smithi170.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-09-09T23:02:40.089 INFO:teuthology.orchestra.run.smithi170.stdout:(83/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.1 MB/s | 560 kB 00:00 2023-09-09T23:02:40.114 INFO:teuthology.orchestra.run.smithi170.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 307 kB/s | 18 kB 00:00 2023-09-09T23:02:40.148 INFO:teuthology.orchestra.run.smithi170.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 4.7 MB/s | 559 kB 00:00 2023-09-09T23:02:40.174 INFO:teuthology.orchestra.run.smithi170.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-09-09T23:02:40.207 INFO:teuthology.orchestra.run.smithi170.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.0 MB/s | 605 kB 00:00 2023-09-09T23:02:40.232 INFO:teuthology.orchestra.run.smithi170.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.2 MB/s | 72 kB 00:00 2023-09-09T23:02:40.258 INFO:teuthology.orchestra.run.smithi170.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-09-09T23:02:40.275 INFO:teuthology.orchestra.run.smithi170.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.4 MB/s | 60 kB 00:00 2023-09-09T23:02:40.308 INFO:teuthology.orchestra.run.smithi170.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-09-09T23:02:40.325 INFO:teuthology.orchestra.run.smithi170.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-09-09T23:02:40.342 INFO:teuthology.orchestra.run.smithi170.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 1.1 MB/s | 39 kB 00:00 2023-09-09T23:02:40.376 INFO:teuthology.orchestra.run.smithi170.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 13 MB/s | 644 kB 00:00 2023-09-09T23:02:40.401 INFO:teuthology.orchestra.run.smithi170.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-09-09T23:02:40.426 INFO:teuthology.orchestra.run.smithi170.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-09-09T23:02:40.504 INFO:teuthology.orchestra.run.smithi170.stdout:(97/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 6.5 MB/s | 2.3 MB 00:00 2023-09-09T23:02:40.535 INFO:teuthology.orchestra.run.smithi170.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.0 MB/s | 110 kB 00:00 2023-09-09T23:02:40.560 INFO:teuthology.orchestra.run.smithi170.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 970 kB/s | 54 kB 00:00 2023-09-09T23:02:40.585 INFO:teuthology.orchestra.run.smithi170.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.0 MB/s | 259 kB 00:00 2023-09-09T23:02:40.619 INFO:teuthology.orchestra.run.smithi170.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-09-09T23:02:40.645 INFO:teuthology.orchestra.run.smithi170.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 662 kB/s | 39 kB 00:00 2023-09-09T23:02:40.678 INFO:teuthology.orchestra.run.smithi170.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 11 MB/s | 688 kB 00:00 2023-09-09T23:02:40.712 INFO:teuthology.orchestra.run.smithi170.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.9 MB/s | 538 kB 00:00 2023-09-09T23:02:40.737 INFO:teuthology.orchestra.run.smithi170.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-09-09T23:02:40.762 INFO:teuthology.orchestra.run.smithi170.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-09-09T23:02:40.797 INFO:teuthology.orchestra.run.smithi170.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-09-09T23:02:40.830 INFO:teuthology.orchestra.run.smithi170.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 794 kB/s | 27 kB 00:00 2023-09-09T23:02:40.871 INFO:teuthology.orchestra.run.smithi170.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-09-09T23:02:40.914 INFO:teuthology.orchestra.run.smithi170.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.0 MB/s | 44 kB 00:00 2023-09-09T23:02:41.015 INFO:teuthology.orchestra.run.smithi170.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 19 MB/s | 4.8 MB 00:00 2023-09-09T23:02:41.065 INFO:teuthology.orchestra.run.smithi170.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 9.0 MB/s | 457 kB 00:00 2023-09-09T23:02:41.106 INFO:teuthology.orchestra.run.smithi170.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-09-09T23:02:41.182 INFO:teuthology.orchestra.run.smithi170.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 14 MB/s | 3.7 MB 00:00 2023-09-09T23:02:41.207 INFO:teuthology.orchestra.run.smithi170.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 637 kB/s | 64 kB 00:00 2023-09-09T23:02:41.232 INFO:teuthology.orchestra.run.smithi170.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-09-09T23:02:41.250 INFO:teuthology.orchestra.run.smithi170.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 476 kB/s | 20 kB 00:00 2023-09-09T23:02:41.274 INFO:teuthology.orchestra.run.smithi170.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.4 MB/s | 103 kB 00:00 2023-09-09T23:02:41.392 INFO:teuthology.orchestra.run.smithi170.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 4.9 MB/s | 709 kB 00:00 2023-09-09T23:02:41.592 INFO:teuthology.orchestra.run.smithi170.stdout:(120/131): python3-scipy-1.0.0-21.module+el8.5. 11 MB/s | 14 MB 00:01 2023-09-09T23:02:41.618 INFO:teuthology.orchestra.run.smithi170.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 877 kB/s | 198 kB 00:00 2023-09-09T23:02:41.659 INFO:teuthology.orchestra.run.smithi170.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-09-09T23:02:41.701 INFO:teuthology.orchestra.run.smithi170.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-09-09T23:02:41.818 INFO:teuthology.orchestra.run.smithi170.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 8.9 MB/s | 4.9 MB 00:00 2023-09-09T23:02:41.894 INFO:teuthology.orchestra.run.smithi170.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 16 MB/s | 4.7 MB 00:00 2023-09-09T23:02:41.927 INFO:teuthology.orchestra.run.smithi170.stdout:(126/131): postgresql-contrib-10.21-2.module+el 7.3 MB/s | 808 kB 00:00 2023-09-09T23:02:41.986 INFO:teuthology.orchestra.run.smithi170.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 16 MB/s | 1.5 MB 00:00 2023-09-09T23:02:42.061 INFO:teuthology.orchestra.run.smithi170.stdout:(128/131): postgresql-server-10.21-2.module+el8 14 MB/s | 5.1 MB 00:00 2023-09-09T23:02:42.120 INFO:teuthology.orchestra.run.smithi170.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-09-09T23:02:42.365 INFO:teuthology.orchestra.run.smithi098.stdout:(17/131): librgw2-18.0.0-6025.g9670cca5.el8.x86 2.0 MB/s | 6.8 MB 00:03 2023-09-09T23:02:42.465 INFO:teuthology.orchestra.run.smithi098.stdout:(18/131): python3-ceph-argparse-18.0.0-6025.g96 447 kB/s | 45 kB 00:00 2023-09-09T23:02:42.774 INFO:teuthology.orchestra.run.smithi098.stdout:(19/131): python3-ceph-common-18.0.0-6025.g9670 407 kB/s | 126 kB 00:00 2023-09-09T23:02:43.108 INFO:teuthology.orchestra.run.smithi098.stdout:(20/131): ceph-osd-18.0.0-6025.g9670cca5.el8.x8 1.9 MB/s | 18 MB 00:09 2023-09-09T23:02:43.132 INFO:teuthology.orchestra.run.smithi098.stdout:(21/131): python3-cephfs-18.0.0-6025.g9670cca5. 549 kB/s | 197 kB 00:00 2023-09-09T23:02:43.516 INFO:teuthology.orchestra.run.smithi098.stdout:(22/131): python3-rbd-18.0.0-6025.g9670cca5.el8 950 kB/s | 364 kB 00:00 2023-09-09T23:02:43.566 INFO:teuthology.orchestra.run.smithi098.stdout:(23/131): python3-rados-18.0.0-6025.g9670cca5.e 811 kB/s | 372 kB 00:00 2023-09-09T23:02:43.662 INFO:teuthology.orchestra.run.smithi170.stdout:(130/131): librados2-18.0.0-6025.g9670cca5.el8. 2.1 MB/s | 3.7 MB 00:01 2023-09-09T23:02:43.675 INFO:teuthology.orchestra.run.smithi098.stdout:(24/131): python3-rgw-18.0.0-6025.g9670cca5.el8 672 kB/s | 107 kB 00:00 2023-09-09T23:02:43.718 INFO:teuthology.orchestra.run.smithi098.stdout:(25/131): rbd-fuse-18.0.0-6025.g9670cca5.el8.x8 622 kB/s | 94 kB 00:00 2023-09-09T23:02:43.828 INFO:teuthology.orchestra.run.smithi170.stdout:(131/131): librbd1-18.0.0-6025.g9670cca5.el8.x8 2.0 MB/s | 3.7 MB 00:01 2023-09-09T23:02:43.831 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2023-09-09T23:02:43.831 INFO:teuthology.orchestra.run.smithi170.stdout:Total 14 MB/s | 208 MB 00:15 2023-09-09T23:02:43.926 INFO:teuthology.orchestra.run.smithi098.stdout:(26/131): rbd-nbd-18.0.0-6025.g9670cca5.el8.x86 860 kB/s | 179 kB 00:00 2023-09-09T23:02:43.985 INFO:teuthology.orchestra.run.smithi098.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6025.g 395 kB/s | 23 kB 00:00 2023-09-09T23:02:44.144 INFO:teuthology.orchestra.run.smithi098.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6025.g9670cca 834 kB/s | 132 kB 00:00 2023-09-09T23:02:44.649 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2023-09-09T23:02:44.767 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2023-09-09T23:02:44.768 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2023-09-09T23:02:45.485 INFO:teuthology.orchestra.run.smithi098.stdout:(29/131): ceph-mgr-dashboard-18.0.0-6025.g9670c 1.3 MB/s | 1.7 MB 00:01 2023-09-09T23:02:45.824 INFO:teuthology.orchestra.run.smithi082.stdout:(10/131): ceph-radosgw-18.0.0-6025.g9670cca5.el 1.5 MB/s | 14 MB 00:09 2023-09-09T23:02:45.885 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2023-09-09T23:02:45.888 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2023-09-09T23:02:46.048 INFO:teuthology.orchestra.run.smithi082.stdout:(11/131): cephfs-mirror-18.0.0-6025.g9670cca5.e 1.1 MB/s | 260 kB 00:00 2023-09-09T23:02:46.140 INFO:teuthology.orchestra.run.smithi082.stdout:(12/131): libcephfs-devel-18.0.0-6025.g9670cca5 339 kB/s | 31 kB 00:00 2023-09-09T23:02:46.210 INFO:teuthology.orchestra.run.smithi098.stdout:(30/131): rbd-mirror-18.0.0-6025.g9670cca5.el8. 1.4 MB/s | 3.6 MB 00:02 2023-09-09T23:02:46.477 INFO:teuthology.orchestra.run.smithi098.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6025.g96 907 kB/s | 242 kB 00:00 2023-09-09T23:02:46.569 INFO:teuthology.orchestra.run.smithi098.stdout:(32/131): ceph-mgr-rook-18.0.0-6025.g9670cca5.e 517 kB/s | 47 kB 00:00 2023-09-09T23:02:46.653 INFO:teuthology.orchestra.run.smithi098.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6025.g9 174 kB/s | 15 kB 00:00 2023-09-09T23:02:46.757 INFO:teuthology.orchestra.run.smithi082.stdout:(13/131): libcephfs2-18.0.0-6025.g9670cca5.el8. 1.3 MB/s | 827 kB 00:00 2023-09-09T23:02:46.903 INFO:teuthology.orchestra.run.smithi098.stdout:(34/131): ceph-volume-18.0.0-6025.g9670cca5.el8 1.0 MB/s | 261 kB 00:00 2023-09-09T23:02:46.966 INFO:teuthology.orchestra.run.smithi082.stdout:(14/131): libcephsqlite-18.0.0-6025.g9670cca5.e 881 kB/s | 184 kB 00:00 2023-09-09T23:02:47.079 INFO:teuthology.orchestra.run.smithi098.stdout:(35/131): cephadm-18.0.0-6025.g9670cca5.el8.noa 1.0 MB/s | 176 kB 00:00 2023-09-09T23:02:47.179 INFO:teuthology.orchestra.run.smithi098.stdout:(36/131): cephfs-top-18.0.0-6025.g9670cca5.el8. 180 kB/s | 18 kB 00:00 2023-09-09T23:02:47.200 INFO:teuthology.orchestra.run.smithi082.stdout:(15/131): librados-devel-18.0.0-6025.g9670cca5. 594 kB/s | 139 kB 00:00 2023-09-09T23:02:47.654 INFO:teuthology.orchestra.run.smithi098.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 645 kB/s | 306 kB 00:00 2023-09-09T23:02:47.658 INFO:teuthology.orchestra.run.smithi082.stdout:(16/131): libradosstriper1-18.0.0-6025.g9670cca 1.1 MB/s | 519 kB 00:00 2023-09-09T23:02:47.763 INFO:teuthology.orchestra.run.smithi098.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 542 kB/s | 59 kB 00:00 2023-09-09T23:02:47.922 INFO:teuthology.orchestra.run.smithi098.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 474 kB/s | 75 kB 00:00 2023-09-09T23:02:48.522 INFO:teuthology.orchestra.run.smithi098.stdout:(40/131): ceph-test-18.0.0-6025.g9670cca5.el8.x 3.8 MB/s | 49 MB 00:12 2023-09-09T23:02:48.547 INFO:teuthology.orchestra.run.smithi098.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 761 kB/s | 476 kB 00:00 2023-09-09T23:02:48.647 INFO:teuthology.orchestra.run.smithi098.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.8 MB/s | 185 kB 00:00 2023-09-09T23:02:48.748 INFO:teuthology.orchestra.run.smithi098.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 330 kB/s | 33 kB 00:00 2023-09-09T23:02:48.865 INFO:teuthology.orchestra.run.smithi098.stdout:(44/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 127 kB/s | 44 kB 00:00 2023-09-09T23:02:48.898 INFO:teuthology.orchestra.run.smithi098.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 108 kB/s | 16 kB 00:00 2023-09-09T23:02:48.999 INFO:teuthology.orchestra.run.smithi098.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.8 MB/s | 384 kB 00:00 2023-09-09T23:02:49.074 INFO:teuthology.orchestra.run.smithi098.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 825 kB/s | 173 kB 00:00 2023-09-09T23:02:49.107 INFO:teuthology.orchestra.run.smithi098.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 749 kB/s | 81 kB 00:00 2023-09-09T23:02:49.166 INFO:teuthology.orchestra.run.smithi098.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 121 kB/s | 11 kB 00:00 2023-09-09T23:02:49.191 INFO:teuthology.orchestra.run.smithi082.stdout:(17/131): ceph-common-18.0.0-6025.g9670cca5.el8 1.3 MB/s | 22 MB 00:17 2023-09-09T23:02:49.233 INFO:teuthology.orchestra.run.smithi098.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 140 kB/s | 18 kB 00:00 2023-09-09T23:02:49.308 INFO:teuthology.orchestra.run.smithi082.stdout:(18/131): python3-ceph-argparse-18.0.0-6025.g96 384 kB/s | 45 kB 00:00 2023-09-09T23:02:49.434 INFO:teuthology.orchestra.run.smithi098.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 243 kB/s | 49 kB 00:00 2023-09-09T23:02:49.517 INFO:teuthology.orchestra.run.smithi082.stdout:(19/131): python3-ceph-common-18.0.0-6025.g9670 603 kB/s | 126 kB 00:00 2023-09-09T23:02:49.547 INFO:teuthology.orchestra.run.smithi098.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 3.1 MB/s | 1.2 MB 00:00 2023-09-09T23:02:49.576 INFO:teuthology.orchestra.run.smithi098.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 413 kB/s | 59 kB 00:00 2023-09-09T23:02:49.668 INFO:teuthology.orchestra.run.smithi098.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 499 kB/s | 60 kB 00:00 2023-09-09T23:02:49.693 INFO:teuthology.orchestra.run.smithi098.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.4 MB/s | 283 kB 00:00 2023-09-09T23:02:49.717 INFO:teuthology.orchestra.run.smithi082.stdout:(20/131): python3-cephfs-18.0.0-6025.g9670cca5. 985 kB/s | 197 kB 00:00 2023-09-09T23:02:49.760 INFO:teuthology.orchestra.run.smithi098.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 170 kB/s | 16 kB 00:00 2023-09-09T23:02:49.786 INFO:teuthology.orchestra.run.smithi098.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 365 kB/s | 34 kB 00:00 2023-09-09T23:02:49.861 INFO:teuthology.orchestra.run.smithi098.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.9 MB/s | 196 kB 00:00 2023-09-09T23:02:49.936 INFO:teuthology.orchestra.run.smithi098.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 427 kB/s | 64 kB 00:00 2023-09-09T23:02:49.953 INFO:teuthology.orchestra.run.smithi098.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 203 kB/s | 19 kB 00:00 2023-09-09T23:02:49.959 INFO:teuthology.orchestra.run.smithi082.stdout:(21/131): python3-rados-18.0.0-6025.g9670cca5.e 1.5 MB/s | 372 kB 00:00 2023-09-09T23:02:50.045 INFO:teuthology.orchestra.run.smithi098.stdout:(61/131): ceph-mgr-diskprediction-local-18.0.0- 1.6 MB/s | 7.4 MB 00:04 2023-09-09T23:02:50.062 INFO:teuthology.orchestra.run.smithi098.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 190 kB/s | 24 kB 00:00 2023-09-09T23:02:50.079 INFO:teuthology.orchestra.run.smithi098.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 233 kB/s | 29 kB 00:00 2023-09-09T23:02:50.187 INFO:teuthology.orchestra.run.smithi098.stdout:(64/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.3 MB/s | 251 kB 00:00 2023-09-09T23:02:50.213 INFO:teuthology.orchestra.run.smithi098.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 1.9 MB/s | 255 kB 00:00 2023-09-09T23:02:50.280 INFO:teuthology.orchestra.run.smithi098.stdout:(66/131): python3-trustme-0.6.0-4.el8.noarch.rp 117 kB/s | 27 kB 00:00 2023-09-09T23:02:50.305 INFO:teuthology.orchestra.run.smithi098.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 522 kB/s | 61 kB 00:00 2023-09-09T23:02:50.331 INFO:teuthology.orchestra.run.smithi098.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 728 kB/s | 85 kB 00:00 2023-09-09T23:02:50.356 INFO:teuthology.orchestra.run.smithi098.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-09-09T23:02:50.373 INFO:teuthology.orchestra.run.smithi098.stdout:(70/131): python3-zc-lockfile-2.0-2.el8.noarch. 247 kB/s | 23 kB 00:00 2023-09-09T23:02:50.431 INFO:teuthology.orchestra.run.smithi098.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 622 kB/s | 47 kB 00:00 2023-09-09T23:02:50.457 INFO:teuthology.orchestra.run.smithi098.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.3 MB/s | 200 kB 00:00 2023-09-09T23:02:50.467 INFO:teuthology.orchestra.run.smithi082.stdout:(22/131): python3-rbd-18.0.0-6025.g9670cca5.el8 716 kB/s | 364 kB 00:00 2023-09-09T23:02:50.482 INFO:teuthology.orchestra.run.smithi098.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 847 kB/s | 43 kB 00:00 2023-09-09T23:02:50.507 INFO:teuthology.orchestra.run.smithi098.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-09-09T23:02:50.533 INFO:teuthology.orchestra.run.smithi098.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 959 kB/s | 48 kB 00:00 2023-09-09T23:02:50.558 INFO:teuthology.orchestra.run.smithi098.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-09-09T23:02:50.608 INFO:teuthology.orchestra.run.smithi098.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 567 kB/s | 43 kB 00:00 2023-09-09T23:02:50.634 INFO:teuthology.orchestra.run.smithi098.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 1.4 MB/s | 109 kB 00:00 2023-09-09T23:02:50.659 INFO:teuthology.orchestra.run.smithi098.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-09-09T23:02:50.660 INFO:teuthology.orchestra.run.smithi082.stdout:(23/131): python3-rgw-18.0.0-6025.g9670cca5.el8 556 kB/s | 107 kB 00:00 2023-09-09T23:02:50.693 INFO:teuthology.orchestra.run.smithi098.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.3 MB/s | 560 kB 00:00 2023-09-09T23:02:50.718 INFO:teuthology.orchestra.run.smithi098.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 319 kB/s | 19 kB 00:00 2023-09-09T23:02:50.743 INFO:teuthology.orchestra.run.smithi082.stdout:(24/131): rbd-fuse-18.0.0-6025.g9670cca5.el8.x8 1.1 MB/s | 94 kB 00:00 2023-09-09T23:02:50.760 INFO:teuthology.orchestra.run.smithi098.stdout:(82/131): thrift-0.13.0-2.el8.x86_64.rpm 3.8 MB/s | 1.7 MB 00:00 2023-09-09T23:02:50.785 INFO:teuthology.orchestra.run.smithi098.stdout:(83/131): python3-oauthlib-2.1.0-1.el8.noarch.r 1.6 MB/s | 155 kB 00:00 2023-09-09T23:02:50.819 INFO:teuthology.orchestra.run.smithi098.stdout:(84/131): python3-cryptography-3.2.1-5.el8.x86_ 5.4 MB/s | 559 kB 00:00 2023-09-09T23:02:51.158 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2023-09-09T23:02:51.191 INFO:teuthology.orchestra.run.smithi098.stdout:(85/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 41 kB/s | 18 kB 00:00 2023-09-09T23:02:51.220 INFO:teuthology.orchestra.run.smithi098.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 593 kB/s | 238 kB 00:00 2023-09-09T23:02:51.253 INFO:teuthology.orchestra.run.smithi098.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 1.3 MB/s | 605 kB 00:00 2023-09-09T23:02:51.279 INFO:teuthology.orchestra.run.smithi098.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.2 MB/s | 72 kB 00:00 2023-09-09T23:02:51.328 INFO:teuthology.orchestra.run.smithi098.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2023-09-09T23:02:51.338 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-09-09T23:02:51.362 INFO:teuthology.orchestra.run.smithi098.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 717 kB/s | 60 kB 00:00 2023-09-09T23:02:51.388 INFO:teuthology.orchestra.run.smithi098.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.6 MB/s | 98 kB 00:00 2023-09-09T23:02:51.413 INFO:teuthology.orchestra.run.smithi098.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-09-09T23:02:51.438 INFO:teuthology.orchestra.run.smithi098.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 767 kB/s | 39 kB 00:00 2023-09-09T23:02:51.460 INFO:teuthology.orchestra.run.smithi082.stdout:(25/131): ceph-test-18.0.0-6025.g9670cca5.el8.x 3.5 MB/s | 49 MB 00:13 2023-09-09T23:02:51.472 INFO:teuthology.orchestra.run.smithi098.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-09-09T23:02:51.497 INFO:teuthology.orchestra.run.smithi098.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-09-09T23:02:51.539 INFO:teuthology.orchestra.run.smithi098.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.0 MB/s | 138 kB 00:00 2023-09-09T23:02:51.555 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-09-09T23:02:51.589 INFO:teuthology.orchestra.run.smithi098.stdout:(97/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 5.8 MB/s | 2.3 MB 00:00 2023-09-09T23:02:51.615 INFO:teuthology.orchestra.run.smithi098.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.4 MB/s | 110 kB 00:00 2023-09-09T23:02:51.640 INFO:teuthology.orchestra.run.smithi098.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-09-09T23:02:51.690 INFO:teuthology.orchestra.run.smithi098.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.4 MB/s | 259 kB 00:00 2023-09-09T23:02:51.693 INFO:teuthology.orchestra.run.smithi082.stdout:(26/131): librgw2-18.0.0-6025.g9670cca5.el8.x86 1.7 MB/s | 6.8 MB 00:04 2023-09-09T23:02:51.719 INFO:teuthology.orchestra.run.smithi082.stdout:(27/131): rbd-nbd-18.0.0-6025.g9670cca5.el8.x86 693 kB/s | 179 kB 00:00 2023-09-09T23:02:51.724 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-09-09T23:02:51.724 INFO:teuthology.orchestra.run.smithi098.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 3.8 MB/s | 323 kB 00:00 2023-09-09T23:02:51.741 INFO:teuthology.orchestra.run.smithi098.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 774 kB/s | 39 kB 00:00 2023-09-09T23:02:51.775 INFO:teuthology.orchestra.run.smithi098.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-09-09T23:02:51.777 INFO:teuthology.orchestra.run.smithi082.stdout:(28/131): ceph-grafana-dashboards-18.0.0-6025.g 277 kB/s | 23 kB 00:00 2023-09-09T23:02:51.803 INFO:teuthology.orchestra.run.smithi082.stdout:(29/131): ceph-mgr-cephadm-18.0.0-6025.g9670cca 1.5 MB/s | 132 kB 00:00 2023-09-09T23:02:51.808 INFO:teuthology.orchestra.run.smithi098.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.8 MB/s | 538 kB 00:00 2023-09-09T23:02:51.822 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-09-09T23:02:51.842 INFO:teuthology.orchestra.run.smithi098.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2023-09-09T23:02:51.867 INFO:teuthology.orchestra.run.smithi098.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2023-09-09T23:02:51.901 INFO:teuthology.orchestra.run.smithi098.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-09-09T23:02:51.935 INFO:teuthology.orchestra.run.smithi098.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 798 kB/s | 27 kB 00:00 2023-09-09T23:02:51.977 INFO:teuthology.orchestra.run.smithi098.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-09-09T23:02:52.035 INFO:teuthology.orchestra.run.smithi098.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 754 kB/s | 44 kB 00:00 2023-09-09T23:02:52.066 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-09-09T23:02:52.119 INFO:teuthology.orchestra.run.smithi098.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 19 MB/s | 4.8 MB 00:00 2023-09-09T23:02:52.161 INFO:teuthology.orchestra.run.smithi098.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 11 MB/s | 457 kB 00:00 2023-09-09T23:02:52.203 INFO:teuthology.orchestra.run.smithi098.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-09-09T23:02:52.261 INFO:teuthology.orchestra.run.smithi098.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 17 MB/s | 3.7 MB 00:00 2023-09-09T23:02:52.287 INFO:teuthology.orchestra.run.smithi098.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 766 kB/s | 64 kB 00:00 2023-09-09T23:02:52.304 INFO:teuthology.orchestra.run.smithi098.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.9 MB/s | 126 kB 00:00 2023-09-09T23:02:52.362 INFO:teuthology.orchestra.run.smithi098.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 266 kB/s | 20 kB 00:00 2023-09-09T23:02:52.388 INFO:teuthology.orchestra.run.smithi098.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 1.2 MB/s | 103 kB 00:00 2023-09-09T23:02:52.430 INFO:teuthology.orchestra.run.smithi098.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-09-09T23:02:52.476 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-09-09T23:02:52.580 INFO:teuthology.orchestra.run.smithi098.stdout:(120/131): python3-scipy-1.0.0-21.module+el8.5. 13 MB/s | 14 MB 00:01 2023-09-09T23:02:52.605 INFO:teuthology.orchestra.run.smithi098.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 1.1 MB/s | 198 kB 00:00 2023-09-09T23:02:52.647 INFO:teuthology.orchestra.run.smithi098.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-09-09T23:02:52.723 INFO:teuthology.orchestra.run.smithi098.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 4.6 MB/s | 354 kB 00:00 2023-09-09T23:02:52.774 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librados2-2:18.0.0-6025.g9670cca5.el8.x86_64 5/134 2023-09-09T23:02:52.807 INFO:teuthology.orchestra.run.smithi098.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 12 MB/s | 4.9 MB 00:00 2023-09-09T23:02:52.878 INFO:teuthology.orchestra.run.smithi082.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6025.g9670c 1.6 MB/s | 1.7 MB 00:01 2023-09-09T23:02:52.890 INFO:teuthology.orchestra.run.smithi098.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2023-09-09T23:02:52.924 INFO:teuthology.orchestra.run.smithi098.stdout:(126/131): postgresql-contrib-10.21-2.module+el 6.8 MB/s | 808 kB 00:00 2023-09-09T23:02:52.982 INFO:teuthology.orchestra.run.smithi098.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 16 MB/s | 1.5 MB 00:00 2023-09-09T23:02:53.086 INFO:teuthology.orchestra.run.smithi082.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6025.g96 1.1 MB/s | 242 kB 00:00 2023-09-09T23:02:53.091 INFO:teuthology.orchestra.run.smithi098.stdout:(128/131): postgresql-server-10.21-2.module+el8 14 MB/s | 5.1 MB 00:00 2023-09-09T23:02:53.149 INFO:teuthology.orchestra.run.smithi098.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-09-09T23:02:53.183 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-2:18.0.0-6025.g9670cca5.el8.x86_64 5/134 2023-09-09T23:02:53.203 INFO:teuthology.orchestra.run.smithi082.stdout:(32/131): ceph-mgr-rook-18.0.0-6025.g9670cca5.e 406 kB/s | 47 kB 00:00 2023-09-09T23:02:53.320 INFO:teuthology.orchestra.run.smithi082.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6025.g9 125 kB/s | 15 kB 00:00 2023-09-09T23:02:53.390 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librbd1-2:18.0.0-6025.g9670cca5.el8.x86_64 6/134 2023-09-09T23:02:53.524 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-2:18.0.0-6025.g9670cca5.el8.x86_64 6/134 2023-09-09T23:02:53.529 INFO:teuthology.orchestra.run.smithi082.stdout:(34/131): ceph-volume-18.0.0-6025.g9670cca5.el8 1.2 MB/s | 261 kB 00:00 2023-09-09T23:02:53.612 INFO:teuthology.orchestra.run.smithi082.stdout:(35/131): rbd-mirror-18.0.0-6025.g9670cca5.el8. 1.2 MB/s | 3.6 MB 00:02 2023-09-09T23:02:53.713 INFO:teuthology.orchestra.run.smithi082.stdout:(36/131): cephadm-18.0.0-6025.g9670cca5.el8.noa 960 kB/s | 176 kB 00:00 2023-09-09T23:02:53.738 INFO:teuthology.orchestra.run.smithi082.stdout:(37/131): cephfs-top-18.0.0-6025.g9670cca5.el8. 143 kB/s | 18 kB 00:00 2023-09-09T23:02:53.740 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs2-2:18.0.0-6025.g9670cca5.el8.x86_64 7/134 2023-09-09T23:02:53.841 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephfs2-2:18.0.0-6025.g9670cca5.el8.x86_64 7/134 2023-09-09T23:02:53.863 INFO:teuthology.orchestra.run.smithi082.stdout:(38/131): ceph-mgr-diskprediction-local-18.0.0- 3.6 MB/s | 7.4 MB 00:02 2023-09-09T23:02:54.028 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rados-2:18.0.0-6025.g9670cca5.el8.x86_64 8/134 2023-09-09T23:02:54.182 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-09-09T23:02:54.213 INFO:teuthology.orchestra.run.smithi082.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 124 kB/s | 59 kB 00:00 2023-09-09T23:02:54.314 INFO:teuthology.orchestra.run.smithi082.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 170 kB/s | 75 kB 00:00 2023-09-09T23:02:54.389 INFO:teuthology.orchestra.run.smithi082.stdout:(41/131): gperftools-libs-2.7-9.el8.x86_64.rpm 453 kB/s | 306 kB 00:00 2023-09-09T23:02:54.419 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-09-09T23:02:54.431 INFO:teuthology.orchestra.run.smithi082.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 373 kB/s | 44 kB 00:00 2023-09-09T23:02:54.523 INFO:teuthology.orchestra.run.smithi082.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.4 MB/s | 185 kB 00:00 2023-09-09T23:02:54.548 INFO:teuthology.orchestra.run.smithi082.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 282 kB/s | 33 kB 00:00 2023-09-09T23:02:54.590 INFO:teuthology.orchestra.run.smithi082.stdout:(45/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.2 MB/s | 476 kB 00:00 2023-09-09T23:02:54.615 INFO:teuthology.orchestra.run.smithi082.stdout:(46/131): python3-certifi-2018.10.15-7.el8.noar 177 kB/s | 16 kB 00:00 2023-09-09T23:02:54.618 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-09-09T23:02:54.641 INFO:teuthology.orchestra.run.smithi098.stdout:(130/131): librbd1-18.0.0-6025.g9670cca5.el8.x8 2.2 MB/s | 3.7 MB 00:01 2023-09-09T23:02:54.732 INFO:teuthology.orchestra.run.smithi082.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 696 kB/s | 81 kB 00:00 2023-09-09T23:02:54.758 INFO:teuthology.orchestra.run.smithi082.stdout:(48/131): python3-cheroot-8.5.2-1.el8.noarch.rp 825 kB/s | 173 kB 00:00 2023-09-09T23:02:54.792 INFO:teuthology.orchestra.run.smithi082.stdout:(49/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.9 MB/s | 384 kB 00:00 2023-09-09T23:02:54.833 INFO:teuthology.orchestra.run.smithi082.stdout:(50/131): python3-jaraco-6.2-6.el8.noarch.rpm 110 kB/s | 11 kB 00:00 2023-09-09T23:02:54.858 INFO:teuthology.orchestra.run.smithi082.stdout:(51/131): python3-jaraco-functools-2.0-4.el8.no 175 kB/s | 18 kB 00:00 2023-09-09T23:02:54.883 INFO:teuthology.orchestra.run.smithi098.stdout:(131/131): librados2-18.0.0-6025.g9670cca5.el8. 1.9 MB/s | 3.7 MB 00:01 2023-09-09T23:02:54.885 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2023-09-09T23:02:54.886 INFO:teuthology.orchestra.run.smithi098.stdout:Total 8.2 MB/s | 208 MB 00:25 2023-09-09T23:02:54.975 INFO:teuthology.orchestra.run.smithi082.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 343 kB/s | 49 kB 00:00 2023-09-09T23:02:54.987 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-09-09T23:02:55.001 INFO:teuthology.orchestra.run.smithi082.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 414 kB/s | 59 kB 00:00 2023-09-09T23:02:55.151 INFO:teuthology.orchestra.run.smithi082.stdout:(54/131): python3-kubernetes-11.0.0-6.el8.noarc 3.2 MB/s | 1.2 MB 00:00 2023-09-09T23:02:55.251 INFO:teuthology.orchestra.run.smithi082.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 156 kB/s | 16 kB 00:00 2023-09-09T23:02:55.256 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-09-09T23:02:55.360 INFO:teuthology.orchestra.run.smithi082.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 310 kB/s | 34 kB 00:00 2023-09-09T23:02:55.415 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephsqlite-2:18.0.0-6025.g9670cca5.el8.x86_64 14/134 2023-09-09T23:02:55.526 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6025.g9670cca5.el8.x86_64 14/134 2023-09-09T23:02:55.569 INFO:teuthology.orchestra.run.smithi082.stdout:(57/131): python3-routes-2.4.1-12.el8.noarch.rp 943 kB/s | 196 kB 00:00 2023-09-09T23:02:55.594 INFO:teuthology.orchestra.run.smithi082.stdout:(58/131): python3-natsort-7.1.1-2.el8.noarch.rp 97 kB/s | 60 kB 00:00 2023-09-09T23:02:55.619 INFO:teuthology.orchestra.run.smithi082.stdout:(59/131): python3-pecan-1.3.2-9.el8.noarch.rpm 458 kB/s | 283 kB 00:00 2023-09-09T23:02:55.681 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libradosstriper1-2:18.0.0-6025.g9670cca5.el8.x86 15/134 2023-09-09T23:02:55.714 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2023-09-09T23:02:55.786 INFO:teuthology.orchestra.run.smithi082.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 98 kB/s | 19 kB 00:00 2023-09-09T23:02:55.811 INFO:teuthology.orchestra.run.smithi082.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 125 kB/s | 24 kB 00:00 2023-09-09T23:02:55.832 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6025.g9670cca5.el8.x86 15/134 2023-09-09T23:02:55.834 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2023-09-09T23:02:55.834 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2023-09-09T23:02:55.836 INFO:teuthology.orchestra.run.smithi082.stdout:(62/131): python3-rsa-4.9-2.el8.noarch.rpm 240 kB/s | 64 kB 00:00 2023-09-09T23:02:55.895 INFO:teuthology.orchestra.run.smithi082.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 268 kB/s | 29 kB 00:00 2023-09-09T23:02:55.920 INFO:teuthology.orchestra.run.smithi082.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 253 kB/s | 27 kB 00:00 2023-09-09T23:02:55.954 INFO:teuthology.orchestra.run.smithi082.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 2.1 MB/s | 255 kB 00:00 2023-09-09T23:02:56.030 INFO:teuthology.orchestra.run.smithi082.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 562 kB/s | 61 kB 00:00 2023-09-09T23:02:56.054 INFO:teuthology.orchestra.run.smithi082.stdout:(67/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.5 MB/s | 251 kB 00:00 2023-09-09T23:02:56.077 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-09-09T23:02:56.088 INFO:teuthology.orchestra.run.smithi082.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 638 kB/s | 85 kB 00:00 2023-09-09T23:02:56.113 INFO:teuthology.orchestra.run.smithi082.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-09-09T23:02:56.139 INFO:teuthology.orchestra.run.smithi082.stdout:(70/131): python3-zc-lockfile-2.0-2.el8.noarch. 210 kB/s | 23 kB 00:00 2023-09-09T23:02:56.181 INFO:teuthology.orchestra.run.smithi082.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-09-09T23:02:56.223 INFO:teuthology.orchestra.run.smithi082.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-09-09T23:02:56.239 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-09-09T23:02:56.265 INFO:teuthology.orchestra.run.smithi082.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-09-09T23:02:56.307 INFO:teuthology.orchestra.run.smithi082.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-09-09T23:02:56.340 INFO:teuthology.orchestra.run.smithi082.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 1.1 MB/s | 39 kB 00:00 2023-09-09T23:02:56.390 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-09-09T23:02:56.399 INFO:teuthology.orchestra.run.smithi082.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 12 MB/s | 688 kB 00:00 2023-09-09T23:02:56.449 INFO:teuthology.orchestra.run.smithi082.stdout:(77/131): thrift-0.13.0-2.el8.x86_64.rpm 4.4 MB/s | 1.7 MB 00:00 2023-09-09T23:02:56.474 INFO:teuthology.orchestra.run.smithi082.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.0 MB/s | 538 kB 00:00 2023-09-09T23:02:56.539 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-09-09T23:02:56.545 INFO:teuthology.orchestra.run.smithi082.stdout:(79/131): oniguruma-6.8.2-2.el8.x86_64.rpm 2.8 MB/s | 187 kB 00:00 2023-09-09T23:02:56.575 INFO:teuthology.orchestra.run.smithi082.stdout:(80/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.2 MB/s | 157 kB 00:00 2023-09-09T23:02:56.609 INFO:teuthology.orchestra.run.smithi082.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 11 MB/s | 713 kB 00:00 2023-09-09T23:02:56.642 INFO:teuthology.orchestra.run.smithi082.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 799 kB/s | 27 kB 00:00 2023-09-09T23:02:56.682 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-09-09T23:02:56.684 INFO:teuthology.orchestra.run.smithi082.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-09-09T23:02:56.818 INFO:teuthology.orchestra.run.smithi082.stdout:(84/131): python3-scipy-1.0.0-21.module+el8.5.0 19 MB/s | 14 MB 00:00 2023-09-09T23:02:56.823 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-09-09T23:02:56.835 INFO:teuthology.orchestra.run.smithi082.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 292 kB/s | 44 kB 00:00 2023-09-09T23:02:56.918 INFO:teuthology.orchestra.run.smithi082.stdout:(86/131): python3-babel-2.5.1-7.el8.noarch.rpm 14 MB/s | 4.8 MB 00:00 2023-09-09T23:02:56.952 INFO:teuthology.orchestra.run.smithi082.stdout:(87/131): python3-werkzeug-0.12.2-4.el8.noarch. 3.8 MB/s | 457 kB 00:00 2023-09-09T23:02:56.966 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2023-09-09T23:02:56.970 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2023-09-09T23:02:56.977 INFO:teuthology.orchestra.run.smithi082.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 3.8 MB/s | 225 kB 00:00 2023-09-09T23:02:56.994 INFO:teuthology.orchestra.run.smithi082.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2023-09-09T23:02:57.019 INFO:teuthology.orchestra.run.smithi082.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.9 MB/s | 126 kB 00:00 2023-09-09T23:02:57.044 INFO:teuthology.orchestra.run.smithi082.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 398 kB/s | 20 kB 00:00 2023-09-09T23:02:57.070 INFO:teuthology.orchestra.run.smithi082.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-09-09T23:02:57.104 INFO:teuthology.orchestra.run.smithi082.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-09-09T23:02:57.171 INFO:teuthology.orchestra.run.smithi082.stdout:(94/131): python3-numpy-1.14.3-10.el8.x86_64.rp 11 MB/s | 3.7 MB 00:00 2023-09-09T23:02:57.204 INFO:teuthology.orchestra.run.smithi082.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 1.9 MB/s | 198 kB 00:00 2023-09-09T23:02:57.246 INFO:teuthology.orchestra.run.smithi082.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-09-09T23:02:57.314 INFO:teuthology.orchestra.run.smithi082.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 5.1 MB/s | 354 kB 00:00 2023-09-09T23:02:57.405 INFO:teuthology.orchestra.run.smithi082.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 15 MB/s | 4.9 MB 00:00 2023-09-09T23:02:57.438 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-09-09T23:02:57.472 INFO:teuthology.orchestra.run.smithi082.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 16 MB/s | 4.7 MB 00:00 2023-09-09T23:02:57.506 INFO:teuthology.orchestra.run.smithi082.stdout:(100/131): postgresql-contrib-10.21-2.module+el 7.9 MB/s | 808 kB 00:00 2023-09-09T23:02:57.539 INFO:teuthology.orchestra.run.smithi082.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2023-09-09T23:02:57.581 INFO:teuthology.orchestra.run.smithi082.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 14 MB/s | 1.5 MB 00:00 2023-09-09T23:02:57.590 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-09-09T23:02:57.606 INFO:teuthology.orchestra.run.smithi082.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.9 MB/s | 200 kB 00:00 2023-09-09T23:02:57.632 INFO:teuthology.orchestra.run.smithi082.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 849 kB/s | 43 kB 00:00 2023-09-09T23:02:57.657 INFO:teuthology.orchestra.run.smithi082.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-09-09T23:02:57.682 INFO:teuthology.orchestra.run.smithi082.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 956 kB/s | 48 kB 00:00 2023-09-09T23:02:57.708 INFO:teuthology.orchestra.run.smithi082.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-09-09T23:02:57.733 INFO:teuthology.orchestra.run.smithi082.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 850 kB/s | 43 kB 00:00 2023-09-09T23:02:57.758 INFO:teuthology.orchestra.run.smithi082.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-09-09T23:02:57.784 INFO:teuthology.orchestra.run.smithi082.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-09-09T23:02:57.850 INFO:teuthology.orchestra.run.smithi082.stdout:(111/131): postgresql-server-10.21-2.module+el8 9.5 MB/s | 5.1 MB 00:00 2023-09-09T23:02:57.876 INFO:teuthology.orchestra.run.smithi082.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 203 kB/s | 19 kB 00:00 2023-09-09T23:02:57.901 INFO:teuthology.orchestra.run.smithi082.stdout:(113/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.8 MB/s | 560 kB 00:00 2023-09-09T23:02:57.926 INFO:teuthology.orchestra.run.smithi082.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 2.0 MB/s | 155 kB 00:00 2023-09-09T23:02:57.952 INFO:teuthology.orchestra.run.smithi082.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 357 kB/s | 18 kB 00:00 2023-09-09T23:02:57.985 INFO:teuthology.orchestra.run.smithi082.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 5.0 MB/s | 559 kB 00:00 2023-09-09T23:02:58.010 INFO:teuthology.orchestra.run.smithi082.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 7.1 MB/s | 605 kB 00:00 2023-09-09T23:02:58.036 INFO:teuthology.orchestra.run.smithi082.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.8 MB/s | 238 kB 00:00 2023-09-09T23:02:58.092 INFO:teuthology.orchestra.run.smithi082.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 887 kB/s | 72 kB 00:00 2023-09-09T23:02:58.119 INFO:teuthology.orchestra.run.smithi082.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 990 kB/s | 83 kB 00:00 2023-09-09T23:02:58.145 INFO:teuthology.orchestra.run.smithi082.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.1 MB/s | 60 kB 00:00 2023-09-09T23:02:58.170 INFO:teuthology.orchestra.run.smithi082.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-09-09T23:02:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-09-09T23:02:58.195 INFO:teuthology.orchestra.run.smithi082.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-09-09T23:02:58.221 INFO:teuthology.orchestra.run.smithi082.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 766 kB/s | 39 kB 00:00 2023-09-09T23:02:58.255 INFO:teuthology.orchestra.run.smithi082.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 11 MB/s | 644 kB 00:00 2023-09-09T23:02:58.280 INFO:teuthology.orchestra.run.smithi082.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.9 MB/s | 171 kB 00:00 2023-09-09T23:02:58.305 INFO:teuthology.orchestra.run.smithi082.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-09-09T23:02:58.356 INFO:teuthology.orchestra.run.smithi082.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.3 MB/s | 2.3 MB 00:00 2023-09-09T23:02:58.381 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-09-09T23:02:58.414 INFO:teuthology.orchestra.run.smithi082.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-09-09T23:02:58.981 INFO:teuthology.orchestra.run.smithi082.stdout:(130/131): librados2-18.0.0-6025.g9670cca5.el8. 5.3 MB/s | 3.7 MB 00:00 2023-09-09T23:02:59.066 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-09-09T23:02:59.313 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-09-09T23:02:59.389 INFO:teuthology.orchestra.run.smithi082.stdout:(131/131): librbd1-18.0.0-6025.g9670cca5.el8.x8 3.4 MB/s | 3.7 MB 00:01 2023-09-09T23:02:59.394 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2023-09-09T23:02:59.394 INFO:teuthology.orchestra.run.smithi082.stdout:Total 7.4 MB/s | 208 MB 00:28 2023-09-09T23:02:59.496 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-09-09T23:02:59.724 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-09-09T23:02:59.904 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-common-2:18.0.0-6025.g9670cca5.el8. 26/134 2023-09-09T23:03:00.034 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-argparse-2:18.0.0-6025.g9670cca5.el 27/134 2023-09-09T23:03:00.208 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2023-09-09T23:03:00.244 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cephfs-2:18.0.0-6025.g9670cca5.el8.x86_6 28/134 2023-09-09T23:03:00.327 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2023-09-09T23:03:00.328 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2023-09-09T23:03:00.472 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-09-09T23:03:01.455 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2023-09-09T23:03:01.459 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2023-09-09T23:03:02.025 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2023-09-09T23:03:02.236 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-09-09T23:03:02.244 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-09-09T23:03:02.435 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-09-09T23:03:02.605 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-09-09T23:03:02.715 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-09-09T23:03:02.938 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-09-09T23:03:03.243 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-09-09T23:03:03.339 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-09-09T23:03:03.561 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-09-09T23:03:03.587 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : librados2-2:18.0.0-6025.g9670cca5.el8.x86_64 5/134 2023-09-09T23:03:03.833 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-09-09T23:03:03.979 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librados2-2:18.0.0-6025.g9670cca5.el8.x86_64 5/134 2023-09-09T23:03:04.055 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-09-09T23:03:04.154 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : librbd1-2:18.0.0-6025.g9670cca5.el8.x86_64 6/134 2023-09-09T23:03:04.271 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librbd1-2:18.0.0-6025.g9670cca5.el8.x86_64 6/134 2023-09-09T23:03:04.275 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-09-09T23:03:04.454 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephfs2-2:18.0.0-6025.g9670cca5.el8.x86_64 7/134 2023-09-09T23:03:04.499 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-09-09T23:03:04.539 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libcephfs2-2:18.0.0-6025.g9670cca5.el8.x86_64 7/134 2023-09-09T23:03:04.669 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rbd-2:18.0.0-6025.g9670cca5.el8.x86_64 36/134 2023-09-09T23:03:04.734 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rados-2:18.0.0-6025.g9670cca5.el8.x86_64 8/134 2023-09-09T23:03:04.837 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librados-devel-2:18.0.0-6025.g9670cca5.el8.x86_6 37/134 2023-09-09T23:03:04.893 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-09-09T23:03:04.979 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-09-09T23:03:05.125 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-09-09T23:03:05.140 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-09-09T23:03:05.321 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-09-09T23:03:05.374 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-09-09T23:03:05.375 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-09-09T23:03:05.600 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-09-09T23:03:05.780 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-09-09T23:03:05.875 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-09-09T23:03:06.022 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-09-09T23:03:06.053 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-09-09T23:03:06.103 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-09-09T23:03:06.212 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephsqlite-2:18.0.0-6025.g9670cca5.el8.x86_64 14/134 2023-09-09T23:03:06.259 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2023-09-09T23:03:06.318 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-09-09T23:03:06.349 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6025.g9670cca5.el8.x86_64 14/134 2023-09-09T23:03:06.416 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-09-09T23:03:06.495 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libradosstriper1-2:18.0.0-6025.g9670cca5.el8.x86 15/134 2023-09-09T23:03:06.508 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-09-09T23:03:06.625 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-09-09T23:03:06.638 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6025.g9670cca5.el8.x86 15/134 2023-09-09T23:03:06.662 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-09-09T23:03:06.824 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-09-09T23:03:06.825 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-09-09T23:03:06.857 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-09-09T23:03:06.977 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-09-09T23:03:06.990 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-09-09T23:03:07.055 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-09-09T23:03:07.097 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-09-09T23:03:07.167 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-09-09T23:03:07.257 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-09-09T23:03:07.362 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-09-09T23:03:07.368 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-09-09T23:03:07.412 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-09-09T23:03:07.536 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-09-09T23:03:07.552 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-09-09T23:03:07.745 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-09-09T23:03:07.779 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-09-09T23:03:07.894 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-09-09T23:03:07.986 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : librados2-2:18.0.0-6025.g9670cca5.el8.x86_64 6/134 2023-09-09T23:03:08.206 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-09-09T23:03:08.360 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-09-09T23:03:08.426 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados2-2:18.0.0-6025.g9670cca5.el8.x86_64 6/134 2023-09-09T23:03:08.449 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-09-09T23:03:08.529 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-09-09T23:03:08.618 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : librbd1-2:18.0.0-6025.g9670cca5.el8.x86_64 7/134 2023-09-09T23:03:08.658 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-09-09T23:03:08.759 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librbd1-2:18.0.0-6025.g9670cca5.el8.x86_64 7/134 2023-09-09T23:03:08.884 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-09-09T23:03:08.960 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephfs2-2:18.0.0-6025.g9670cca5.el8.x86_64 8/134 2023-09-09T23:03:09.061 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephfs2-2:18.0.0-6025.g9670cca5.el8.x86_64 8/134 2023-09-09T23:03:09.126 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-09-09T23:03:09.204 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-09-09T23:03:09.243 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rados-2:18.0.0-6025.g9670cca5.el8.x86_64 9/134 2023-09-09T23:03:09.303 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-09-09T23:03:09.404 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-09-09T23:03:09.462 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-09-09T23:03:09.588 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-09-09T23:03:09.696 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-09-09T23:03:09.825 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-09-09T23:03:09.990 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-09-09T23:03:10.048 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-09-09T23:03:10.079 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-09-09T23:03:10.157 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-09-09T23:03:10.302 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-09-09T23:03:10.308 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-09-09T23:03:10.476 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephsqlite-2:18.0.0-6025.g9670cca5.el8.x86_64 14/134 2023-09-09T23:03:10.484 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-09-09T23:03:10.520 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-09-09T23:03:10.586 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6025.g9670cca5.el8.x86_64 14/134 2023-09-09T23:03:10.726 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libradosstriper1-2:18.0.0-6025.g9670cca5.el8.x86 15/134 2023-09-09T23:03:10.743 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-09-09T23:03:10.800 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-09-09T23:03:10.806 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6025.g9670cca5.el8.x86 15/134 2023-09-09T23:03:10.959 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-09-09T23:03:10.967 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-09-09T23:03:10.968 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ceph-common-2:18.0.0-6025.g9670cca5.el8. 26/134 2023-09-09T23:03:11.108 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-09-09T23:03:11.132 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ceph-argparse-2:18.0.0-6025.g9670cca5.el 27/134 2023-09-09T23:03:11.197 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-09-09T23:03:11.293 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-09-09T23:03:11.345 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-09-09T23:03:11.356 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cephfs-2:18.0.0-6025.g9670cca5.el8.x86_6 28/134 2023-09-09T23:03:11.523 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-09-09T23:03:11.587 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-09-09T23:03:11.804 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-09-09T23:03:11.912 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-09-09T23:03:12.044 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-09-09T23:03:12.101 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-09-09T23:03:12.199 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-09-09T23:03:12.390 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-09-09T23:03:12.581 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-09-09T23:03:12.694 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-09-09T23:03:12.791 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-09-09T23:03:12.876 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-09-09T23:03:13.017 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-09-09T23:03:13.182 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-09-09T23:03:13.318 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-09-09T23:03:13.371 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-09-09T23:03:13.544 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-09-09T23:03:13.590 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-09-09T23:03:13.799 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-09-09T23:03:13.832 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-09-09T23:03:13.993 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-09-09T23:03:14.051 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-09-09T23:03:14.251 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-09-09T23:03:14.296 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-09-09T23:03:14.395 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-09-09T23:03:14.490 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-09-09T23:03:14.503 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-09-09T23:03:14.669 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-09-09T23:03:14.671 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-09-09T23:03:14.735 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-09-09T23:03:14.879 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-09-09T23:03:14.927 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-09-09T23:03:15.030 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-09-09T23:03:15.115 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ceph-common-2:18.0.0-6025.g9670cca5.el8. 26/134 2023-09-09T23:03:15.236 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ceph-argparse-2:18.0.0-6025.g9670cca5.el 27/134 2023-09-09T23:03:15.282 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-09-09T23:03:15.498 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-09-09T23:03:15.712 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-09-09T23:03:15.925 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rbd-2:18.0.0-6025.g9670cca5.el8.x86_64 36/134 2023-09-09T23:03:16.015 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-09-09T23:03:16.067 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librados-devel-2:18.0.0-6025.g9670cca5.el8.x86_6 37/134 2023-09-09T23:03:16.142 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cephfs-2:18.0.0-6025.g9670cca5.el8.x86_6 28/134 2023-09-09T23:03:16.243 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-09-09T23:03:16.332 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-09-09T23:03:16.387 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-09-09T23:03:16.447 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-09-09T23:03:16.594 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-09-09T23:03:16.603 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-09-09T23:03:16.648 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-09-09T23:03:16.752 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-09-09T23:03:16.877 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-09-09T23:03:16.913 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-09-09T23:03:17.029 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-09-09T23:03:17.106 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-09-09T23:03:17.186 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-09-09T23:03:17.191 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-09-09T23:03:17.277 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-09-09T23:03:17.352 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-09-09T23:03:17.394 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-09-09T23:03:17.901 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-09-09T23:03:18.107 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-09-09T23:03:18.215 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-09-09T23:03:18.261 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-09-09T23:03:18.386 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librgw2-2:18.0.0-6025.g9670cca5.el8.x86_64 92/134 2023-09-09T23:03:18.460 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librgw2-2:18.0.0-6025.g9670cca5.el8.x86_64 92/134 2023-09-09T23:03:18.463 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-09-09T23:03:18.630 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-09-09T23:03:18.636 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rgw-2:18.0.0-6025.g9670cca5.el8.x86_64 93/134 2023-09-09T23:03:18.806 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-09-09T23:03:18.945 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-09-09T23:03:19.010 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-09-09T23:03:19.225 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-09-09T23:03:19.266 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-09-09T23:03:19.468 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-09-09T23:03:19.604 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-09-09T23:03:19.676 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-09-09T23:03:19.855 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rbd-2:18.0.0-6025.g9670cca5.el8.x86_64 36/134 2023-09-09T23:03:19.912 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-09-09T23:03:20.028 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librados-devel-2:18.0.0-6025.g9670cca5.el8.x86_6 37/134 2023-09-09T23:03:20.155 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-09-09T23:03:20.189 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-09-09T23:03:20.374 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-09-09T23:03:20.392 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-09-09T23:03:20.585 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-09-09T23:03:20.608 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-09-09T23:03:20.735 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-09-09T23:03:20.945 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-09-09T23:03:20.952 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-09-09T23:03:21.168 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-09-09T23:03:21.185 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-09-09T23:03:21.361 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-09-09T23:03:21.523 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-09-09T23:03:21.615 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-09-09T23:03:21.794 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-09-09T23:03:21.822 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-09-09T23:03:21.996 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-09-09T23:03:22.287 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-09-09T23:03:22.322 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-09-09T23:03:22.347 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 94/134 2023-09-09T23:03:22.491 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-09-09T23:03:22.569 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 94/134 2023-09-09T23:03:22.608 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-09-09T23:03:22.763 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-09-09T23:03:22.857 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-09-09T23:03:23.077 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-09-09T23:03:23.086 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-09-09T23:03:23.149 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 94/134 2023-09-09T23:03:23.273 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-09-09T23:03:23.276 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-09-09T23:03:23.454 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-09-09T23:03:23.465 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-09-09T23:03:23.501 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-base-2:18.0.0-6025.g9670cca5.el8.x86_64 95/134 2023-09-09T23:03:23.588 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-base-2:18.0.0-6025.g9670cca5.el8.x86_64 95/134 2023-09-09T23:03:23.668 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-09-09T23:03:23.727 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-selinux-2:18.0.0-6025.g9670cca5.el8.x86_64 96/134 2023-09-09T23:03:23.737 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-09-09T23:03:23.851 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-09-09T23:03:23.983 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-09-09T23:03:24.026 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-09-09T23:03:24.122 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-09-09T23:03:24.278 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-09-09T23:03:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-09-09T23:03:24.468 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-09-09T23:03:24.594 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-09-09T23:03:24.704 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-09-09T23:03:24.835 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-09-09T23:03:24.879 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-09-09T23:03:25.088 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-09-09T23:03:25.088 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-09-09T23:03:25.224 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-09-09T23:03:25.246 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-09-09T23:03:25.407 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-09-09T23:03:25.482 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-09-09T23:03:25.564 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-09-09T23:03:25.636 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-09-09T23:03:25.739 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-09-09T23:03:25.894 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-09-09T23:03:25.916 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-09-09T23:03:26.077 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-09-09T23:03:26.184 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-09-09T23:03:26.231 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-09-09T23:03:26.306 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-09-09T23:03:26.425 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-09-09T23:03:26.552 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-09-09T23:03:26.575 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-09-09T23:03:26.740 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-09-09T23:03:26.816 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-09-09T23:03:26.897 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-09-09T23:03:27.050 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-09-09T23:03:27.281 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-09-09T23:03:27.504 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-09-09T23:03:27.635 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-09-09T23:03:27.774 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-09-09T23:03:27.963 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-09-09T23:03:27.971 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-09-09T23:03:28.201 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-09-09T23:03:28.330 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-09-09T23:03:28.373 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-09-09T23:03:28.541 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-09-09T23:03:28.593 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-09-09T23:03:28.686 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-09-09T23:03:28.895 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-09-09T23:03:28.945 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-09-09T23:03:29.183 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-09-09T23:03:29.197 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-09-09T23:03:29.361 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-09-09T23:03:29.402 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-09-09T23:03:29.602 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-09-09T23:03:29.796 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-09-09T23:03:29.953 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-09-09T23:03:30.136 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-09-09T23:03:30.220 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-09-09T23:03:30.375 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librgw2-2:18.0.0-6025.g9670cca5.el8.x86_64 92/134 2023-09-09T23:03:30.436 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librgw2-2:18.0.0-6025.g9670cca5.el8.x86_64 92/134 2023-09-09T23:03:30.635 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rgw-2:18.0.0-6025.g9670cca5.el8.x86_64 93/134 2023-09-09T23:03:31.250 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-09-09T23:03:31.508 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-09-09T23:03:31.737 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-09-09T23:03:32.038 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-09-09T23:03:32.310 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-09-09T23:03:32.486 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-09-09T23:03:33.350 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-09-09T23:03:33.529 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librgw2-2:18.0.0-6025.g9670cca5.el8.x86_64 92/134 2023-09-09T23:03:33.599 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librgw2-2:18.0.0-6025.g9670cca5.el8.x86_64 92/134 2023-09-09T23:03:33.788 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rgw-2:18.0.0-6025.g9670cca5.el8.x86_64 93/134 2023-09-09T23:03:34.339 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 94/134 2023-09-09T23:03:34.526 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 94/134 2023-09-09T23:03:35.120 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 94/134 2023-09-09T23:03:35.421 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-base-2:18.0.0-6025.g9670cca5.el8.x86_64 95/134 2023-09-09T23:03:35.489 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-base-2:18.0.0-6025.g9670cca5.el8.x86_64 95/134 2023-09-09T23:03:35.617 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-selinux-2:18.0.0-6025.g9670cca5.el8.x86_64 96/134 2023-09-09T23:03:37.451 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 94/134 2023-09-09T23:03:37.629 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 94/134 2023-09-09T23:03:38.210 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 94/134 2023-09-09T23:03:38.479 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-base-2:18.0.0-6025.g9670cca5.el8.x86_64 95/134 2023-09-09T23:03:38.542 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-base-2:18.0.0-6025.g9670cca5.el8.x86_64 95/134 2023-09-09T23:03:38.679 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-selinux-2:18.0.0-6025.g9670cca5.el8.x86_64 96/134 2023-09-09T23:03:42.617 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6025.g9670cca5.el8.x86_64 96/134 2023-09-09T23:03:42.618 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /sys 2023-09-09T23:03:42.618 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /proc 2023-09-09T23:03:42.618 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /dev 2023-09-09T23:03:42.618 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /run 2023-09-09T23:03:42.619 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /mnt 2023-09-09T23:03:42.619 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /var/tmp 2023-09-09T23:03:42.619 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /home 2023-09-09T23:03:42.619 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /tmp 2023-09-09T23:03:42.619 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /dev 2023-09-09T23:03:42.620 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-09T23:03:43.000 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-volume-2:18.0.0-6025.g9670cca5.el8.noarch 97/134 2023-09-09T23:03:44.593 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-volume-2:18.0.0-6025.g9670cca5.el8.noarch 97/134 2023-09-09T23:03:44.739 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-osd-2:18.0.0-6025.g9670cca5.el8.x86_64 98/134 2023-09-09T23:03:44.962 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-osd-2:18.0.0-6025.g9670cca5.el8.x86_64 98/134 2023-09-09T23:03:45.132 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mds-2:18.0.0-6025.g9670cca5.el8.x86_64 99/134 2023-09-09T23:03:45.586 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mds-2:18.0.0-6025.g9670cca5.el8.x86_64 99/134 2023-09-09T23:03:45.732 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mon-2:18.0.0-6025.g9670cca5.el8.x86_64 100/134 2023-09-09T23:03:45.749 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mon-2:18.0.0-6025.g9670cca5.el8.x86_64 100/134 2023-09-09T23:03:47.439 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 101/134 2023-09-09T23:03:47.650 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 101/134 2023-09-09T23:03:47.771 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6025.g9670cca5.e 102/134 2023-09-09T23:03:47.991 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6025.g9670cca5. 103/134 2023-09-09T23:03:48.190 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6025.g9670cca5.el8.noa 104/134 2023-09-09T23:03:48.410 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6025.g9670cca5.el8.noa 104/134 2023-09-09T23:03:48.721 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6025.g9670cca5.el8.n 105/134 2023-09-09T23:03:49.684 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6025.g9670cca5.el8.n 105/134 2023-09-09T23:03:49.848 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6025.g967 106/134 2023-09-09T23:03:49.939 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6025.g967 106/134 2023-09-09T23:03:50.271 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6025.g9670cca5.el 107/134 2023-09-09T23:03:50.494 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-2:18.0.0-6025.g9670cca5.el8.x86_64 108/134 2023-09-09T23:03:50.529 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6025.g9670cca5.el8.x86_64 108/134 2023-09-09T23:03:50.714 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-rook-2:18.0.0-6025.g9670cca5.el8.noarch 109/134 2023-09-09T23:03:50.725 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6025.g9670cca5.el8.noarch 109/134 2023-09-09T23:03:52.042 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-2:18.0.0-6025.g9670cca5.el8.x86_64 110/134 2023-09-09T23:03:52.252 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-radosgw-2:18.0.0-6025.g9670cca5.el8.x86_64 111/134 2023-09-09T23:03:52.286 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6025.g9670cca5.el8.x86_64 111/134 2023-09-09T23:03:52.439 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6025.g9670c 112/134 2023-09-09T23:03:52.481 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6025.g9670c 112/134 2023-09-09T23:03:52.656 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 113/134 2023-09-09T23:03:53.015 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 113/134 2023-09-09T23:03:53.164 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 114/134 2023-09-09T23:03:54.395 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6025.g9670cca5.el8.x86_64 96/134 2023-09-09T23:03:54.395 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /sys 2023-09-09T23:03:54.396 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /proc 2023-09-09T23:03:54.396 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /dev 2023-09-09T23:03:54.396 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /run 2023-09-09T23:03:54.396 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /mnt 2023-09-09T23:03:54.396 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /var/tmp 2023-09-09T23:03:54.397 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /home 2023-09-09T23:03:54.397 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /tmp 2023-09-09T23:03:54.397 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /dev 2023-09-09T23:03:54.397 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-09-09T23:03:54.756 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-volume-2:18.0.0-6025.g9670cca5.el8.noarch 97/134 2023-09-09T23:03:56.346 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-volume-2:18.0.0-6025.g9670cca5.el8.noarch 97/134 2023-09-09T23:03:56.543 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-osd-2:18.0.0-6025.g9670cca5.el8.x86_64 98/134 2023-09-09T23:03:56.765 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-osd-2:18.0.0-6025.g9670cca5.el8.x86_64 98/134 2023-09-09T23:03:56.964 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mds-2:18.0.0-6025.g9670cca5.el8.x86_64 99/134 2023-09-09T23:03:57.418 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mds-2:18.0.0-6025.g9670cca5.el8.x86_64 99/134 2023-09-09T23:03:57.597 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mon-2:18.0.0-6025.g9670cca5.el8.x86_64 100/134 2023-09-09T23:03:57.614 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mon-2:18.0.0-6025.g9670cca5.el8.x86_64 100/134 2023-09-09T23:03:57.673 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6025.g9670cca5.el8.x86_64 96/134 2023-09-09T23:03:57.674 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /sys 2023-09-09T23:03:57.674 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /proc 2023-09-09T23:03:57.674 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /dev 2023-09-09T23:03:57.674 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /run 2023-09-09T23:03:57.674 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /mnt 2023-09-09T23:03:57.675 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /var/tmp 2023-09-09T23:03:57.675 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /home 2023-09-09T23:03:57.675 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /tmp 2023-09-09T23:03:57.675 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /dev 2023-09-09T23:03:57.675 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:03:57.796 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 114/134 2023-09-09T23:03:58.008 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-volume-2:18.0.0-6025.g9670cca5.el8.noarch 97/134 2023-09-09T23:03:58.667 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-test-2:18.0.0-6025.g9670cca5.el8.x86_64 115/134 2023-09-09T23:03:58.982 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-09-09T23:03:59.174 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 117/134 2023-09-09T23:03:59.246 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 101/134 2023-09-09T23:03:59.419 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-09-09T23:03:59.437 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 101/134 2023-09-09T23:03:59.553 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6025.g9670cca5.e 102/134 2023-09-09T23:03:59.562 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-09-09T23:03:59.597 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-volume-2:18.0.0-6025.g9670cca5.el8.noarch 97/134 2023-09-09T23:03:59.749 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-osd-2:18.0.0-6025.g9670cca5.el8.x86_64 98/134 2023-09-09T23:03:59.754 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-09-09T23:03:59.780 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6025.g9670cca5. 103/134 2023-09-09T23:03:59.971 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-osd-2:18.0.0-6025.g9670cca5.el8.x86_64 98/134 2023-09-09T23:03:59.992 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-09-09T23:04:00.005 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6025.g9670cca5.el8.noa 104/134 2023-09-09T23:04:00.157 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mds-2:18.0.0-6025.g9670cca5.el8.x86_64 99/134 2023-09-09T23:04:00.175 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs-devel-2:18.0.0-6025.g9670cca5.el8.x86_ 121/134 2023-09-09T23:04:00.225 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6025.g9670cca5.el8.noa 104/134 2023-09-09T23:04:00.355 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-09T23:04:00.517 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6025.g9670cca5.el8.n 105/134 2023-09-09T23:04:00.611 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mds-2:18.0.0-6025.g9670cca5.el8.x86_64 99/134 2023-09-09T23:04:00.748 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mon-2:18.0.0-6025.g9670cca5.el8.x86_64 100/134 2023-09-09T23:04:00.765 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mon-2:18.0.0-6025.g9670cca5.el8.x86_64 100/134 2023-09-09T23:04:01.479 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6025.g9670cca5.el8.n 105/134 2023-09-09T23:04:01.705 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6025.g967 106/134 2023-09-09T23:04:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6025.g967 106/134 2023-09-09T23:04:02.153 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6025.g9670cca5.el 107/134 2023-09-09T23:04:02.411 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-2:18.0.0-6025.g9670cca5.el8.x86_64 108/134 2023-09-09T23:04:02.419 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 101/134 2023-09-09T23:04:02.446 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6025.g9670cca5.el8.x86_64 108/134 2023-09-09T23:04:02.602 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-09T23:04:02.613 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-rook-2:18.0.0-6025.g9670cca5.el8.noarch 109/134 2023-09-09T23:04:02.624 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6025.g9670cca5.el8.noarch 109/134 2023-09-09T23:04:02.626 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 101/134 2023-09-09T23:04:02.779 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6025.g9670cca5.e 102/134 2023-09-09T23:04:02.896 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-09T23:04:02.915 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-09T23:04:02.980 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6025.g9670cca5. 103/134 2023-09-09T23:04:03.117 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-top-2:18.0.0-6025.g9670cca5.el8.noarch 124/134 2023-09-09T23:04:03.156 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6025.g9670cca5.el8.noa 104/134 2023-09-09T23:04:03.353 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 125/134 2023-09-09T23:04:03.376 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6025.g9670cca5.el8.noa 104/134 2023-09-09T23:04:03.675 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-nbd-2:18.0.0-6025.g9670cca5.el8.x86_64 126/134 2023-09-09T23:04:03.681 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6025.g9670cca5.el8.n 105/134 2023-09-09T23:04:03.959 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-2:18.0.0-6025.g9670cca5.el8.x86_64 110/134 2023-09-09T23:04:04.016 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-09-09T23:04:04.168 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-radosgw-2:18.0.0-6025.g9670cca5.el8.x86_64 111/134 2023-09-09T23:04:04.201 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6025.g9670cca5.el8.x86_64 111/134 2023-09-09T23:04:04.212 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-09-09T23:04:04.293 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-09-09T23:04:04.346 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6025.g9670c 112/134 2023-09-09T23:04:04.388 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6025.g9670c 112/134 2023-09-09T23:04:04.487 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-09-09T23:04:04.500 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-09-09T23:04:04.571 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 113/134 2023-09-09T23:04:04.637 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-09-09T23:04:04.639 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6025.g9670cca5.el8.n 105/134 2023-09-09T23:04:04.645 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-09-09T23:04:04.814 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-09-09T23:04:04.814 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-09T23:04:04.830 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6025.g967 106/134 2023-09-09T23:04:04.920 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6025.g967 106/134 2023-09-09T23:04:04.929 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 113/134 2023-09-09T23:04:05.007 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-09T23:04:05.007 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-09T23:04:05.121 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 114/134 2023-09-09T23:04:05.259 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-09T23:04:05.289 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6025.g9670cca5.el 107/134 2023-09-09T23:04:05.492 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-2:18.0.0-6025.g9670cca5.el8.x86_64 108/134 2023-09-09T23:04:05.502 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-09T23:04:05.527 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6025.g9670cca5.el8.x86_64 108/134 2023-09-09T23:04:05.664 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-rook-2:18.0.0-6025.g9670cca5.el8.noarch 109/134 2023-09-09T23:04:05.675 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6025.g9670cca5.el8.noarch 109/134 2023-09-09T23:04:05.993 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-09-09T23:04:05.994 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-2:18.0.0-6025.g9670cca5.el8.x86_64 1/134 2023-09-09T23:04:05.994 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-base-2:18.0.0-6025.g9670cca5.el8.x86_64 2/134 2023-09-09T23:04:05.994 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 3/134 2023-09-09T23:04:05.994 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 4/134 2023-09-09T23:04:05.994 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6025.g9670c 5/134 2023-09-09T23:04:05.995 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mds-2:18.0.0-6025.g9670cca5.el8.x86_64 6/134 2023-09-09T23:04:05.995 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-2:18.0.0-6025.g9670cca5.el8.x86_64 7/134 2023-09-09T23:04:05.995 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mon-2:18.0.0-6025.g9670cca5.el8.x86_64 8/134 2023-09-09T23:04:05.995 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-osd-2:18.0.0-6025.g9670cca5.el8.x86_64 9/134 2023-09-09T23:04:05.995 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-radosgw-2:18.0.0-6025.g9670cca5.el8.x86_64 10/134 2023-09-09T23:04:05.996 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-selinux-2:18.0.0-6025.g9670cca5.el8.x86_64 11/134 2023-09-09T23:04:05.996 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-test-2:18.0.0-6025.g9670cca5.el8.x86_64 12/134 2023-09-09T23:04:05.996 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 13/134 2023-09-09T23:04:05.996 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs-devel-2:18.0.0-6025.g9670cca5.el8.x86_ 14/134 2023-09-09T23:04:05.996 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs2-2:18.0.0-6025.g9670cca5.el8.x86_64 15/134 2023-09-09T23:04:05.997 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephsqlite-2:18.0.0-6025.g9670cca5.el8.x86_64 16/134 2023-09-09T23:04:05.997 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados-devel-2:18.0.0-6025.g9670cca5.el8.x86_6 17/134 2023-09-09T23:04:05.997 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libradosstriper1-2:18.0.0-6025.g9670cca5.el8.x86 18/134 2023-09-09T23:04:05.997 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librgw2-2:18.0.0-6025.g9670cca5.el8.x86_64 19/134 2023-09-09T23:04:05.997 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6025.g9670cca5.el 20/134 2023-09-09T23:04:05.998 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-common-2:18.0.0-6025.g9670cca5.el8. 21/134 2023-09-09T23:04:05.998 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cephfs-2:18.0.0-6025.g9670cca5.el8.x86_6 22/134 2023-09-09T23:04:05.998 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rados-2:18.0.0-6025.g9670cca5.el8.x86_64 23/134 2023-09-09T23:04:05.998 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rbd-2:18.0.0-6025.g9670cca5.el8.x86_64 24/134 2023-09-09T23:04:05.998 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rgw-2:18.0.0-6025.g9670cca5.el8.x86_64 25/134 2023-09-09T23:04:05.999 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 26/134 2023-09-09T23:04:05.999 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 27/134 2023-09-09T23:04:05.999 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-nbd-2:18.0.0-6025.g9670cca5.el8.x86_64 28/134 2023-09-09T23:04:06.008 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6025.g9670cca5. 29/134 2023-09-09T23:04:06.008 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6025.g9670cca5.el8.noa 30/134 2023-09-09T23:04:06.008 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6025.g9670cca5.el8.n 31/134 2023-09-09T23:04:06.009 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6025.g967 32/134 2023-09-09T23:04:06.009 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6025.g9670cca5.el 33/134 2023-09-09T23:04:06.009 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6025.g9670cca5.el8.noarch 34/134 2023-09-09T23:04:06.009 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6025.g9670cca5.e 35/134 2023-09-09T23:04:06.010 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-volume-2:18.0.0-6025.g9670cca5.el8.noarch 36/134 2023-09-09T23:04:06.010 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 37/134 2023-09-09T23:04:06.010 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-top-2:18.0.0-6025.g9670cca5.el8.noarch 38/134 2023-09-09T23:04:06.010 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-09-09T23:04:06.010 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-09-09T23:04:06.011 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-09-09T23:04:06.011 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-09-09T23:04:06.011 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-09-09T23:04:06.011 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-09-09T23:04:06.011 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-09-09T23:04:06.012 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-09-09T23:04:06.012 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-09-09T23:04:06.012 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-09-09T23:04:06.012 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-09-09T23:04:06.012 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-09-09T23:04:06.012 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-09-09T23:04:06.013 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-09-09T23:04:06.013 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-09-09T23:04:06.013 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-09-09T23:04:06.013 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-09-09T23:04:06.013 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-09-09T23:04:06.014 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-09-09T23:04:06.014 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-09-09T23:04:06.014 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-09-09T23:04:06.014 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-09-09T23:04:06.015 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-09-09T23:04:06.015 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-09-09T23:04:06.015 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-09-09T23:04:06.015 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-09-09T23:04:06.015 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-09-09T23:04:06.016 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-09-09T23:04:06.016 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-09-09T23:04:06.016 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-09-09T23:04:06.016 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-09-09T23:04:06.016 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-09-09T23:04:06.017 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-09-09T23:04:06.017 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-09-09T23:04:06.017 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-09-09T23:04:06.017 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-09-09T23:04:06.017 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-09-09T23:04:06.018 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-09-09T23:04:06.018 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-09-09T23:04:06.018 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-09-09T23:04:06.018 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-09-09T23:04:06.018 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-09-09T23:04:06.019 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-09-09T23:04:06.019 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-09-09T23:04:06.019 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-09-09T23:04:06.019 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-09-09T23:04:06.020 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-09-09T23:04:06.020 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-09-09T23:04:06.020 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-09-09T23:04:06.020 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-09-09T23:04:06.020 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-09-09T23:04:06.020 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-09-09T23:04:06.021 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-09-09T23:04:06.021 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-09-09T23:04:06.021 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-09-09T23:04:06.021 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-09-09T23:04:06.022 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-09-09T23:04:06.022 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-09-09T23:04:06.022 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-09-09T23:04:06.022 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-09-09T23:04:06.023 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-09-09T23:04:06.023 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-09-09T23:04:06.023 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-09-09T23:04:06.023 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-09-09T23:04:06.023 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-09-09T23:04:06.023 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-09-09T23:04:06.024 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-09-09T23:04:06.024 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-09-09T23:04:06.024 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-09-09T23:04:06.024 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-09-09T23:04:06.025 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-09-09T23:04:06.025 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-09-09T23:04:06.025 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-09-09T23:04:06.025 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-09-09T23:04:06.025 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-09-09T23:04:06.025 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-09-09T23:04:06.026 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-09-09T23:04:06.026 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-09-09T23:04:06.026 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-09-09T23:04:06.026 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-09-09T23:04:06.026 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-09-09T23:04:06.027 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-09-09T23:04:06.027 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-09-09T23:04:06.027 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-09-09T23:04:06.027 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-09-09T23:04:06.027 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-09-09T23:04:06.028 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-09-09T23:04:06.028 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-09-09T23:04:06.028 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-09-09T23:04:06.028 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-09-09T23:04:06.028 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-2:18.0.0-6025.g9670cca5.el8.x86_64 129/134 2023-09-09T23:04:06.029 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-09-09T23:04:06.029 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-2:18.0.0-6025.g9670cca5.el8.x86_64 131/134 2023-09-09T23:04:06.029 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-09T23:04:06.029 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-09-09T23:04:07.020 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-2:18.0.0-6025.g9670cca5.el8.x86_64 110/134 2023-09-09T23:04:07.188 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-radosgw-2:18.0.0-6025.g9670cca5.el8.x86_64 111/134 2023-09-09T23:04:07.221 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6025.g9670cca5.el8.x86_64 111/134 2023-09-09T23:04:07.422 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6025.g9670c 112/134 2023-09-09T23:04:07.464 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6025.g9670c 112/134 2023-09-09T23:04:07.605 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 113/134 2023-09-09T23:04:07.963 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 113/134 2023-09-09T23:04:08.147 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 114/134 2023-09-09T23:04:08.518 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-09T23:04:08.519 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:01:52 ago on Sat 09 Sep 2023 11:02:16 PM UTC. 2023-09-09T23:04:09.750 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 114/134 2023-09-09T23:04:10.649 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-test-2:18.0.0-6025.g9670cca5.el8.x86_64 115/134 2023-09-09T23:04:11.020 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-09-09T23:04:11.156 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2023-09-09T23:04:11.232 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 117/134 2023-09-09T23:04:11.370 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-09T23:04:11.370 INFO:teuthology.orchestra.run.smithi170.stdout:Upgraded: 2023-09-09T23:04:11.370 INFO:teuthology.orchestra.run.smithi170.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-09-09T23:04:11.371 INFO:teuthology.orchestra.run.smithi170.stdout: librados2-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.371 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.371 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2023-09-09T23:04:11.371 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-09-09T23:04:11.371 INFO:teuthology.orchestra.run.smithi170.stdout: automake-1.16.1-7.el8.noarch 2023-09-09T23:04:11.372 INFO:teuthology.orchestra.run.smithi170.stdout: bison-3.0.4-10.el8.x86_64 2023-09-09T23:04:11.372 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.372 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.372 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.372 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.373 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:11.373 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.373 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.373 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.373 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:11.374 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:11.374 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:11.374 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:11.374 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:11.374 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.375 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.375 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:11.375 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.375 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.375 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.375 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:11.376 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:11.376 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.376 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-top-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:11.376 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-09-09T23:04:11.376 INFO:teuthology.orchestra.run.smithi170.stdout: flex-2.6.1-9.el8.x86_64 2023-09-09T23:04:11.377 INFO:teuthology.orchestra.run.smithi170.stdout: fuse-2.9.7-15.el8.x86_64 2023-09-09T23:04:11.377 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-09T23:04:11.377 INFO:teuthology.orchestra.run.smithi170.stdout: jq-1.6-3.el8.x86_64 2023-09-09T23:04:11.377 INFO:teuthology.orchestra.run.smithi170.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-09T23:04:11.377 INFO:teuthology.orchestra.run.smithi170.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-09-09T23:04:11.378 INFO:teuthology.orchestra.run.smithi170.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-09-09T23:04:11.378 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.378 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.378 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.378 INFO:teuthology.orchestra.run.smithi170.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-09-09T23:04:11.379 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-09-09T23:04:11.379 INFO:teuthology.orchestra.run.smithi170.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-09T23:04:11.379 INFO:teuthology.orchestra.run.smithi170.stdout: libpq-13.5-1.el8.x86_64 2023-09-09T23:04:11.379 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-09-09T23:04:11.379 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-09-09T23:04:11.380 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.380 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.380 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-09T23:04:11.380 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.380 INFO:teuthology.orchestra.run.smithi170.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-09-09T23:04:11.381 INFO:teuthology.orchestra.run.smithi170.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-09T23:04:11.381 INFO:teuthology.orchestra.run.smithi170.stdout: libtool-2.4.6-25.el8.x86_64 2023-09-09T23:04:11.381 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-09T23:04:11.381 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-09-09T23:04:11.382 INFO:teuthology.orchestra.run.smithi170.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-09-09T23:04:11.382 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-09T23:04:11.382 INFO:teuthology.orchestra.run.smithi170.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-09T23:04:11.382 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-0.3.15-3.el8.x86_64 2023-09-09T23:04:11.382 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-09-09T23:04:11.383 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-09-09T23:04:11.383 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-09-09T23:04:11.383 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-09-09T23:04:11.383 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-09-09T23:04:11.383 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-09T23:04:11.384 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-09T23:04:11.384 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-09T23:04:11.384 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-09T23:04:11.384 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-09T23:04:11.385 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-09T23:04:11.385 INFO:teuthology.orchestra.run.smithi170.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-09T23:04:11.385 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-09T23:04:11.385 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.385 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.386 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.386 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-09T23:04:11.386 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-09-09T23:04:11.386 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-09T23:04:11.386 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-09T23:04:11.387 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-09-09T23:04:11.387 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-09T23:04:11.387 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-09T23:04:11.387 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-09T23:04:11.387 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-09T23:04:11.388 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-09T23:04:11.388 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-09T23:04:11.388 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-09T23:04:11.388 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-09T23:04:11.388 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-09T23:04:11.389 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako-1.0.6-13.el8.noarch 2023-09-09T23:04:11.389 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-09T23:04:11.389 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-09T23:04:11.389 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-09T23:04:11.389 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-09T23:04:11.390 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-09T23:04:11.390 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-09T23:04:11.390 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-09T23:04:11.390 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend-2.6-1.el8.noarch 2023-09-09T23:04:11.390 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-09T23:04:11.391 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-09T23:04:11.391 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-09T23:04:11.391 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-09T23:04:11.391 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-09T23:04:11.391 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz-2017.2-9.el8.noarch 2023-09-09T23:04:11.392 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.392 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.392 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-09T23:04:11.392 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-09T23:04:11.392 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.393 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-09T23:04:11.393 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-09T23:04:11.393 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-09-09T23:04:11.393 INFO:teuthology.orchestra.run.smithi170.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-09T23:04:11.393 INFO:teuthology.orchestra.run.smithi170.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-09T23:04:11.393 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-09T23:04:11.394 INFO:teuthology.orchestra.run.smithi170.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-09T23:04:11.394 INFO:teuthology.orchestra.run.smithi170.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-09T23:04:11.394 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-09T23:04:11.394 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-09T23:04:11.394 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-09T23:04:11.394 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-09T23:04:11.395 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-09T23:04:11.395 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.395 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.395 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:11.395 INFO:teuthology.orchestra.run.smithi170.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-09T23:04:11.395 INFO:teuthology.orchestra.run.smithi170.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-09T23:04:11.396 INFO:teuthology.orchestra.run.smithi170.stdout: uuid-1.6.2-43.el8.x86_64 2023-09-09T23:04:11.396 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-09-09T23:04:11.396 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-09T23:04:11.396 INFO:teuthology.orchestra.run.smithi170.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-09-09T23:04:11.397 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-09T23:04:11.397 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2023-09-09T23:04:11.488 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-09-09T23:04:11.578 DEBUG:teuthology.parallel:result is None 2023-09-09T23:04:11.670 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-09-09T23:04:11.861 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-09-09T23:04:12.108 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-09-09T23:04:12.285 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephfs-devel-2:18.0.0-6025.g9670cca5.el8.x86_ 121/134 2023-09-09T23:04:12.438 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-09T23:04:12.766 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 114/134 2023-09-09T23:04:13.031 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-test-2:18.0.0-6025.g9670cca5.el8.x86_64 115/134 2023-09-09T23:04:13.179 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-09-09T23:04:13.368 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-09-09T23:04:14.261 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-09-09T23:04:14.590 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-09-09T23:04:14.637 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-09T23:04:14.756 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 119/134 2023-09-09T23:04:14.895 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-09T23:04:14.912 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-09T23:04:14.966 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-09-09T23:04:15.103 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-top-2:18.0.0-6025.g9670cca5.el8.noarch 124/134 2023-09-09T23:04:15.116 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephfs-devel-2:18.0.0-6025.g9670cca5.el8.x86_ 121/134 2023-09-09T23:04:15.254 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-09T23:04:15.428 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 125/134 2023-09-09T23:04:15.763 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-nbd-2:18.0.0-6025.g9670cca5.el8.x86_64 126/134 2023-09-09T23:04:16.149 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-09-09T23:04:16.353 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-09-09T23:04:16.437 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-09-09T23:04:16.628 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-09-09T23:04:16.640 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-09-09T23:04:16.828 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-09-09T23:04:16.835 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-09-09T23:04:16.972 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-09-09T23:04:16.972 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-09T23:04:17.190 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-09T23:04:17.190 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-09T23:04:17.434 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-09T23:04:17.472 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-09T23:04:17.610 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-09T23:04:17.728 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-09T23:04:17.745 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-09T23:04:17.895 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-top-2:18.0.0-6025.g9670cca5.el8.noarch 124/134 2023-09-09T23:04:18.136 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-09-09T23:04:18.136 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-2:18.0.0-6025.g9670cca5.el8.x86_64 1/134 2023-09-09T23:04:18.137 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-base-2:18.0.0-6025.g9670cca5.el8.x86_64 2/134 2023-09-09T23:04:18.137 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 3/134 2023-09-09T23:04:18.137 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 4/134 2023-09-09T23:04:18.137 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6025.g9670c 5/134 2023-09-09T23:04:18.137 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mds-2:18.0.0-6025.g9670cca5.el8.x86_64 6/134 2023-09-09T23:04:18.138 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-2:18.0.0-6025.g9670cca5.el8.x86_64 7/134 2023-09-09T23:04:18.138 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mon-2:18.0.0-6025.g9670cca5.el8.x86_64 8/134 2023-09-09T23:04:18.138 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-osd-2:18.0.0-6025.g9670cca5.el8.x86_64 9/134 2023-09-09T23:04:18.138 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-radosgw-2:18.0.0-6025.g9670cca5.el8.x86_64 10/134 2023-09-09T23:04:18.138 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-selinux-2:18.0.0-6025.g9670cca5.el8.x86_64 11/134 2023-09-09T23:04:18.139 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-test-2:18.0.0-6025.g9670cca5.el8.x86_64 12/134 2023-09-09T23:04:18.139 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 13/134 2023-09-09T23:04:18.139 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephfs-devel-2:18.0.0-6025.g9670cca5.el8.x86_ 14/134 2023-09-09T23:04:18.139 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephfs2-2:18.0.0-6025.g9670cca5.el8.x86_64 15/134 2023-09-09T23:04:18.139 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephsqlite-2:18.0.0-6025.g9670cca5.el8.x86_64 16/134 2023-09-09T23:04:18.140 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados-devel-2:18.0.0-6025.g9670cca5.el8.x86_6 17/134 2023-09-09T23:04:18.140 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libradosstriper1-2:18.0.0-6025.g9670cca5.el8.x86 18/134 2023-09-09T23:04:18.140 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librgw2-2:18.0.0-6025.g9670cca5.el8.x86_64 19/134 2023-09-09T23:04:18.140 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6025.g9670cca5.el 20/134 2023-09-09T23:04:18.140 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ceph-common-2:18.0.0-6025.g9670cca5.el8. 21/134 2023-09-09T23:04:18.141 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cephfs-2:18.0.0-6025.g9670cca5.el8.x86_6 22/134 2023-09-09T23:04:18.141 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rados-2:18.0.0-6025.g9670cca5.el8.x86_64 23/134 2023-09-09T23:04:18.141 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rbd-2:18.0.0-6025.g9670cca5.el8.x86_64 24/134 2023-09-09T23:04:18.141 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rgw-2:18.0.0-6025.g9670cca5.el8.x86_64 25/134 2023-09-09T23:04:18.141 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 26/134 2023-09-09T23:04:18.141 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 27/134 2023-09-09T23:04:18.142 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-nbd-2:18.0.0-6025.g9670cca5.el8.x86_64 28/134 2023-09-09T23:04:18.142 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6025.g9670cca5. 29/134 2023-09-09T23:04:18.142 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6025.g9670cca5.el8.noa 30/134 2023-09-09T23:04:18.142 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6025.g9670cca5.el8.n 31/134 2023-09-09T23:04:18.142 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6025.g967 32/134 2023-09-09T23:04:18.143 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6025.g9670cca5.el 33/134 2023-09-09T23:04:18.143 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6025.g9670cca5.el8.noarch 34/134 2023-09-09T23:04:18.143 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6025.g9670cca5.e 35/134 2023-09-09T23:04:18.143 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-volume-2:18.0.0-6025.g9670cca5.el8.noarch 36/134 2023-09-09T23:04:18.143 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 37/134 2023-09-09T23:04:18.144 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-top-2:18.0.0-6025.g9670cca5.el8.noarch 38/134 2023-09-09T23:04:18.144 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-09-09T23:04:18.144 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-09-09T23:04:18.144 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-09-09T23:04:18.144 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-09-09T23:04:18.145 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-09-09T23:04:18.145 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-09-09T23:04:18.145 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-09-09T23:04:18.145 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-09-09T23:04:18.145 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-09-09T23:04:18.146 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-09-09T23:04:18.146 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-09-09T23:04:18.146 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-09-09T23:04:18.146 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-09-09T23:04:18.146 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-09-09T23:04:18.147 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-09-09T23:04:18.147 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-09-09T23:04:18.147 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-09-09T23:04:18.147 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-09-09T23:04:18.147 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-09-09T23:04:18.148 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-09-09T23:04:18.148 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-09-09T23:04:18.148 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-09-09T23:04:18.148 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-09-09T23:04:18.148 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-09-09T23:04:18.149 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-09-09T23:04:18.149 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-09-09T23:04:18.149 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-09-09T23:04:18.149 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-09-09T23:04:18.149 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-09-09T23:04:18.150 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-09-09T23:04:18.150 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-09-09T23:04:18.150 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-09-09T23:04:18.151 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-09-09T23:04:18.151 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-09-09T23:04:18.151 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-09-09T23:04:18.151 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-09-09T23:04:18.151 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-09-09T23:04:18.151 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-09-09T23:04:18.152 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-09-09T23:04:18.152 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-09-09T23:04:18.152 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-09-09T23:04:18.152 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-09-09T23:04:18.152 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-09-09T23:04:18.153 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-09-09T23:04:18.153 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-09-09T23:04:18.153 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-09-09T23:04:18.153 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-09-09T23:04:18.153 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-09-09T23:04:18.154 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-09-09T23:04:18.154 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-09-09T23:04:18.154 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-09-09T23:04:18.154 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-09-09T23:04:18.154 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-09-09T23:04:18.154 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-09-09T23:04:18.155 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-09-09T23:04:18.155 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-09-09T23:04:18.155 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-09-09T23:04:18.155 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-09-09T23:04:18.155 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-09-09T23:04:18.156 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-09-09T23:04:18.156 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-09-09T23:04:18.156 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-09-09T23:04:18.156 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-09-09T23:04:18.157 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-09-09T23:04:18.157 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-09-09T23:04:18.157 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-09-09T23:04:18.157 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-09-09T23:04:18.157 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-09-09T23:04:18.157 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-09-09T23:04:18.158 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-09-09T23:04:18.158 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-09-09T23:04:18.158 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-09-09T23:04:18.158 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-09-09T23:04:18.158 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-09-09T23:04:18.159 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-09-09T23:04:18.159 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-09-09T23:04:18.159 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-09-09T23:04:18.159 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-09-09T23:04:18.159 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-09-09T23:04:18.160 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-09-09T23:04:18.160 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-09-09T23:04:18.160 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-09-09T23:04:18.160 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-09-09T23:04:18.160 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-09-09T23:04:18.160 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-09-09T23:04:18.161 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-09-09T23:04:18.161 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-09-09T23:04:18.161 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-09-09T23:04:18.161 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-09-09T23:04:18.161 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-09-09T23:04:18.162 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados2-2:18.0.0-6025.g9670cca5.el8.x86_64 129/134 2023-09-09T23:04:18.162 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-09-09T23:04:18.162 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librbd1-2:18.0.0-6025.g9670cca5.el8.x86_64 131/134 2023-09-09T23:04:18.162 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-09T23:04:18.162 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-09-09T23:04:18.219 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 125/134 2023-09-09T23:04:18.487 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-nbd-2:18.0.0-6025.g9670cca5.el8.x86_64 126/134 2023-09-09T23:04:18.829 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-09-09T23:04:18.978 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-09T23:04:18.986 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-09T23:04:19.177 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-09-09T23:04:19.369 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-09-09T23:04:19.448 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-09-09T23:04:19.661 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-09-09T23:04:19.673 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-09-09T23:04:19.673 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-09T23:04:19.855 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-09T23:04:19.855 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-09T23:04:20.091 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-09T23:04:20.285 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-09T23:04:20.665 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-09T23:04:20.665 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:02:03 ago on Sat 09 Sep 2023 11:02:17 PM UTC. 2023-09-09T23:04:20.806 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-09-09T23:04:20.806 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-2:18.0.0-6025.g9670cca5.el8.x86_64 1/134 2023-09-09T23:04:20.807 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-base-2:18.0.0-6025.g9670cca5.el8.x86_64 2/134 2023-09-09T23:04:20.807 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 3/134 2023-09-09T23:04:20.807 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 4/134 2023-09-09T23:04:20.807 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6025.g9670c 5/134 2023-09-09T23:04:20.808 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mds-2:18.0.0-6025.g9670cca5.el8.x86_64 6/134 2023-09-09T23:04:20.808 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-2:18.0.0-6025.g9670cca5.el8.x86_64 7/134 2023-09-09T23:04:20.808 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mon-2:18.0.0-6025.g9670cca5.el8.x86_64 8/134 2023-09-09T23:04:20.808 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-osd-2:18.0.0-6025.g9670cca5.el8.x86_64 9/134 2023-09-09T23:04:20.809 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-radosgw-2:18.0.0-6025.g9670cca5.el8.x86_64 10/134 2023-09-09T23:04:20.809 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-selinux-2:18.0.0-6025.g9670cca5.el8.x86_64 11/134 2023-09-09T23:04:20.809 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-test-2:18.0.0-6025.g9670cca5.el8.x86_64 12/134 2023-09-09T23:04:20.809 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 13/134 2023-09-09T23:04:20.809 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephfs-devel-2:18.0.0-6025.g9670cca5.el8.x86_ 14/134 2023-09-09T23:04:20.810 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephfs2-2:18.0.0-6025.g9670cca5.el8.x86_64 15/134 2023-09-09T23:04:20.810 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephsqlite-2:18.0.0-6025.g9670cca5.el8.x86_64 16/134 2023-09-09T23:04:20.810 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados-devel-2:18.0.0-6025.g9670cca5.el8.x86_6 17/134 2023-09-09T23:04:20.811 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libradosstriper1-2:18.0.0-6025.g9670cca5.el8.x86 18/134 2023-09-09T23:04:20.811 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librgw2-2:18.0.0-6025.g9670cca5.el8.x86_64 19/134 2023-09-09T23:04:20.811 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6025.g9670cca5.el 20/134 2023-09-09T23:04:20.811 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ceph-common-2:18.0.0-6025.g9670cca5.el8. 21/134 2023-09-09T23:04:20.811 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cephfs-2:18.0.0-6025.g9670cca5.el8.x86_6 22/134 2023-09-09T23:04:20.812 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rados-2:18.0.0-6025.g9670cca5.el8.x86_64 23/134 2023-09-09T23:04:20.812 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rbd-2:18.0.0-6025.g9670cca5.el8.x86_64 24/134 2023-09-09T23:04:20.812 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rgw-2:18.0.0-6025.g9670cca5.el8.x86_64 25/134 2023-09-09T23:04:20.812 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 26/134 2023-09-09T23:04:20.813 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 27/134 2023-09-09T23:04:20.813 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-nbd-2:18.0.0-6025.g9670cca5.el8.x86_64 28/134 2023-09-09T23:04:20.813 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6025.g9670cca5. 29/134 2023-09-09T23:04:20.813 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6025.g9670cca5.el8.noa 30/134 2023-09-09T23:04:20.814 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6025.g9670cca5.el8.n 31/134 2023-09-09T23:04:20.814 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6025.g967 32/134 2023-09-09T23:04:20.814 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6025.g9670cca5.el 33/134 2023-09-09T23:04:20.814 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6025.g9670cca5.el8.noarch 34/134 2023-09-09T23:04:20.815 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6025.g9670cca5.e 35/134 2023-09-09T23:04:20.815 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-volume-2:18.0.0-6025.g9670cca5.el8.noarch 36/134 2023-09-09T23:04:20.815 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 37/134 2023-09-09T23:04:20.815 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-top-2:18.0.0-6025.g9670cca5.el8.noarch 38/134 2023-09-09T23:04:20.816 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-09-09T23:04:20.816 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-09-09T23:04:20.816 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-09-09T23:04:20.816 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-09-09T23:04:20.817 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-09-09T23:04:20.817 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-09-09T23:04:20.817 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-09-09T23:04:20.818 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-09-09T23:04:20.818 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-09-09T23:04:20.818 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-09-09T23:04:20.818 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-09-09T23:04:20.818 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-09-09T23:04:20.819 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-09-09T23:04:20.819 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-09-09T23:04:20.819 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-09-09T23:04:20.819 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-09-09T23:04:20.819 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-09-09T23:04:20.819 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-09-09T23:04:20.820 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-09-09T23:04:20.820 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-09-09T23:04:20.820 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-09-09T23:04:20.820 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-09-09T23:04:20.820 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-09-09T23:04:20.821 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-09-09T23:04:20.821 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-09-09T23:04:20.821 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-09-09T23:04:20.821 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-09-09T23:04:20.821 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-09-09T23:04:20.821 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-09-09T23:04:20.822 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-09-09T23:04:20.822 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-09-09T23:04:20.822 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-09-09T23:04:20.822 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-09-09T23:04:20.822 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-09-09T23:04:20.822 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-09-09T23:04:20.823 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-09-09T23:04:20.823 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-09-09T23:04:20.823 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-09-09T23:04:20.823 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-09-09T23:04:20.823 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-09-09T23:04:20.824 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-09-09T23:04:20.824 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-09-09T23:04:20.824 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-09-09T23:04:20.824 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-09-09T23:04:20.824 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-09-09T23:04:20.825 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-09-09T23:04:20.825 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-09-09T23:04:20.825 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-09-09T23:04:20.825 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-09-09T23:04:20.825 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-09-09T23:04:20.825 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-09-09T23:04:20.826 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-09-09T23:04:20.826 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-09-09T23:04:20.826 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-09-09T23:04:20.826 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-09-09T23:04:20.826 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-09-09T23:04:20.827 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-09-09T23:04:20.827 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-09-09T23:04:20.827 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-09-09T23:04:20.827 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-09-09T23:04:20.828 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-09-09T23:04:20.828 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-09-09T23:04:20.828 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-09-09T23:04:20.828 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-09-09T23:04:20.828 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-09-09T23:04:20.829 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-09-09T23:04:20.829 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-09-09T23:04:20.829 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-09-09T23:04:20.829 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-09-09T23:04:20.829 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-09-09T23:04:20.830 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-09-09T23:04:20.830 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-09-09T23:04:20.830 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-09-09T23:04:20.830 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-09-09T23:04:20.830 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-09-09T23:04:20.831 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-09-09T23:04:20.831 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-09-09T23:04:20.831 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-09-09T23:04:20.831 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-09-09T23:04:20.831 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-09-09T23:04:20.831 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-09-09T23:04:20.832 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-09-09T23:04:20.832 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-09-09T23:04:20.832 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-09T23:04:20.832 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-09-09T23:04:20.833 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-09-09T23:04:20.833 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-09-09T23:04:20.833 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-09-09T23:04:20.833 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-09-09T23:04:20.833 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-09T23:04:20.833 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados2-2:18.0.0-6025.g9670cca5.el8.x86_64 129/134 2023-09-09T23:04:20.834 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-09-09T23:04:20.834 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librbd1-2:18.0.0-6025.g9670cca5.el8.x86_64 131/134 2023-09-09T23:04:20.834 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-09T23:04:20.834 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-09-09T23:04:23.107 INFO:teuthology.orchestra.run.smithi098.stdout:Installed products updated. 2023-09-09T23:04:23.271 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-09T23:04:23.271 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:02:04 ago on Sat 09 Sep 2023 11:02:19 PM UTC. 2023-09-09T23:04:23.317 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-09-09T23:04:23.318 INFO:teuthology.orchestra.run.smithi098.stdout:Upgraded: 2023-09-09T23:04:23.318 INFO:teuthology.orchestra.run.smithi098.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-09-09T23:04:23.318 INFO:teuthology.orchestra.run.smithi098.stdout: librados2-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.318 INFO:teuthology.orchestra.run.smithi098.stdout: librbd1-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.319 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2023-09-09T23:04:23.319 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-09-09T23:04:23.319 INFO:teuthology.orchestra.run.smithi098.stdout: automake-1.16.1-7.el8.noarch 2023-09-09T23:04:23.319 INFO:teuthology.orchestra.run.smithi098.stdout: bison-3.0.4-10.el8.x86_64 2023-09-09T23:04:23.319 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.320 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-base-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.320 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.320 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.320 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-grafana-dashboards-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:23.320 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-immutable-object-cache-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.320 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mds-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.321 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.321 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:23.321 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-dashboard-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:23.321 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:23.321 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-modules-core-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:23.322 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-rook-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:23.322 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mon-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.322 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-osd-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.322 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-prometheus-alerts-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:23.323 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-radosgw-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.323 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-selinux-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.323 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-test-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.323 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-volume-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:23.323 INFO:teuthology.orchestra.run.smithi098.stdout: cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:23.329 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.329 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-top-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:23.330 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-09-09T23:04:23.330 INFO:teuthology.orchestra.run.smithi098.stdout: flex-2.6.1-9.el8.x86_64 2023-09-09T23:04:23.330 INFO:teuthology.orchestra.run.smithi098.stdout: fuse-2.9.7-15.el8.x86_64 2023-09-09T23:04:23.330 INFO:teuthology.orchestra.run.smithi098.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-09T23:04:23.330 INFO:teuthology.orchestra.run.smithi098.stdout: jq-1.6-3.el8.x86_64 2023-09-09T23:04:23.331 INFO:teuthology.orchestra.run.smithi098.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-09T23:04:23.331 INFO:teuthology.orchestra.run.smithi098.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-09-09T23:04:23.331 INFO:teuthology.orchestra.run.smithi098.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-09-09T23:04:23.331 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs-devel-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.331 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs2-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.332 INFO:teuthology.orchestra.run.smithi098.stdout: libcephsqlite-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.332 INFO:teuthology.orchestra.run.smithi098.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-09-09T23:04:23.332 INFO:teuthology.orchestra.run.smithi098.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-09-09T23:04:23.332 INFO:teuthology.orchestra.run.smithi098.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-09T23:04:23.332 INFO:teuthology.orchestra.run.smithi098.stdout: libpq-13.5-1.el8.x86_64 2023-09-09T23:04:23.333 INFO:teuthology.orchestra.run.smithi098.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-09-09T23:04:23.333 INFO:teuthology.orchestra.run.smithi098.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-09-09T23:04:23.333 INFO:teuthology.orchestra.run.smithi098.stdout: librados-devel-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.333 INFO:teuthology.orchestra.run.smithi098.stdout: libradosstriper1-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.333 INFO:teuthology.orchestra.run.smithi098.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-09T23:04:23.334 INFO:teuthology.orchestra.run.smithi098.stdout: librgw2-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.334 INFO:teuthology.orchestra.run.smithi098.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-09-09T23:04:23.334 INFO:teuthology.orchestra.run.smithi098.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-09T23:04:23.334 INFO:teuthology.orchestra.run.smithi098.stdout: libtool-2.4.6-25.el8.x86_64 2023-09-09T23:04:23.334 INFO:teuthology.orchestra.run.smithi098.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-09T23:04:23.335 INFO:teuthology.orchestra.run.smithi098.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-09-09T23:04:23.335 INFO:teuthology.orchestra.run.smithi098.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-09-09T23:04:23.335 INFO:teuthology.orchestra.run.smithi098.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-09T23:04:23.335 INFO:teuthology.orchestra.run.smithi098.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-09T23:04:23.335 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-0.3.15-3.el8.x86_64 2023-09-09T23:04:23.336 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-09-09T23:04:23.336 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-09-09T23:04:23.336 INFO:teuthology.orchestra.run.smithi098.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-09-09T23:04:23.336 INFO:teuthology.orchestra.run.smithi098.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-09-09T23:04:23.336 INFO:teuthology.orchestra.run.smithi098.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-09-09T23:04:23.336 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-09T23:04:23.337 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-09T23:04:23.337 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-09T23:04:23.337 INFO:teuthology.orchestra.run.smithi098.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-09T23:04:23.337 INFO:teuthology.orchestra.run.smithi098.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-09T23:04:23.337 INFO:teuthology.orchestra.run.smithi098.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-09T23:04:23.338 INFO:teuthology.orchestra.run.smithi098.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-09T23:04:23.338 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-09T23:04:23.338 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-argparse-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.338 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.338 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cephfs-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.339 INFO:teuthology.orchestra.run.smithi098.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-09T23:04:23.339 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-09-09T23:04:23.339 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-09T23:04:23.339 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-09T23:04:23.339 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-09-09T23:04:23.340 INFO:teuthology.orchestra.run.smithi098.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-09T23:04:23.340 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-09T23:04:23.340 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-09T23:04:23.340 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-09T23:04:23.340 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-09T23:04:23.341 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-09T23:04:23.341 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-09T23:04:23.341 INFO:teuthology.orchestra.run.smithi098.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-09T23:04:23.341 INFO:teuthology.orchestra.run.smithi098.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-09T23:04:23.341 INFO:teuthology.orchestra.run.smithi098.stdout: python3-mako-1.0.6-13.el8.noarch 2023-09-09T23:04:23.342 INFO:teuthology.orchestra.run.smithi098.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-09T23:04:23.342 INFO:teuthology.orchestra.run.smithi098.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-09T23:04:23.342 INFO:teuthology.orchestra.run.smithi098.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-09T23:04:23.342 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-09T23:04:23.342 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-09T23:04:23.343 INFO:teuthology.orchestra.run.smithi098.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-09T23:04:23.343 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-09T23:04:23.343 INFO:teuthology.orchestra.run.smithi098.stdout: python3-portend-2.6-1.el8.noarch 2023-09-09T23:04:23.343 INFO:teuthology.orchestra.run.smithi098.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-09T23:04:23.343 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-09T23:04:23.343 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-09T23:04:23.344 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-09T23:04:23.344 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-09T23:04:23.344 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytz-2017.2-9.el8.noarch 2023-09-09T23:04:23.344 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rados-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.344 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rbd-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.345 INFO:teuthology.orchestra.run.smithi098.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-09T23:04:23.345 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-09T23:04:23.345 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rgw-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.345 INFO:teuthology.orchestra.run.smithi098.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-09T23:04:23.345 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-09T23:04:23.346 INFO:teuthology.orchestra.run.smithi098.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-09-09T23:04:23.346 INFO:teuthology.orchestra.run.smithi098.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-09T23:04:23.346 INFO:teuthology.orchestra.run.smithi098.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-09T23:04:23.346 INFO:teuthology.orchestra.run.smithi098.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-09T23:04:23.346 INFO:teuthology.orchestra.run.smithi098.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-09T23:04:23.347 INFO:teuthology.orchestra.run.smithi098.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-09T23:04:23.347 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-09T23:04:23.347 INFO:teuthology.orchestra.run.smithi098.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-09T23:04:23.347 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-09T23:04:23.347 INFO:teuthology.orchestra.run.smithi098.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-09T23:04:23.348 INFO:teuthology.orchestra.run.smithi098.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-09T23:04:23.348 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.348 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.348 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-nbd-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:23.348 INFO:teuthology.orchestra.run.smithi098.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-09T23:04:23.349 INFO:teuthology.orchestra.run.smithi098.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-09T23:04:23.349 INFO:teuthology.orchestra.run.smithi098.stdout: uuid-1.6.2-43.el8.x86_64 2023-09-09T23:04:23.349 INFO:teuthology.orchestra.run.smithi098.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-09-09T23:04:23.349 INFO:teuthology.orchestra.run.smithi098.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-09T23:04:23.350 INFO:teuthology.orchestra.run.smithi098.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-09-09T23:04:23.350 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-09-09T23:04:23.350 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2023-09-09T23:04:23.505 DEBUG:teuthology.parallel:result is None 2023-09-09T23:04:25.827 INFO:teuthology.orchestra.run.smithi082.stdout:Installed products updated. 2023-09-09T23:04:26.041 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:04:26.041 INFO:teuthology.orchestra.run.smithi082.stdout:Upgraded: 2023-09-09T23:04:26.041 INFO:teuthology.orchestra.run.smithi082.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-09-09T23:04:26.041 INFO:teuthology.orchestra.run.smithi082.stdout: librados2-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.042 INFO:teuthology.orchestra.run.smithi082.stdout: librbd1-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.042 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2023-09-09T23:04:26.042 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-09-09T23:04:26.042 INFO:teuthology.orchestra.run.smithi082.stdout: automake-1.16.1-7.el8.noarch 2023-09-09T23:04:26.043 INFO:teuthology.orchestra.run.smithi082.stdout: bison-3.0.4-10.el8.x86_64 2023-09-09T23:04:26.043 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.043 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.043 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.044 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.044 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-grafana-dashboards-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:26.044 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-immutable-object-cache-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.044 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mds-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.044 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.045 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:26.045 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:26.045 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:26.045 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:26.046 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-rook-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:26.046 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mon-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.046 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-osd-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.046 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-prometheus-alerts-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:26.047 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-radosgw-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.047 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-selinux-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.047 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-test-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.047 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-volume-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:26.048 INFO:teuthology.orchestra.run.smithi082.stdout: cephadm-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:26.048 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.048 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-top-2:18.0.0-6025.g9670cca5.el8.noarch 2023-09-09T23:04:26.048 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-09-09T23:04:26.048 INFO:teuthology.orchestra.run.smithi082.stdout: flex-2.6.1-9.el8.x86_64 2023-09-09T23:04:26.049 INFO:teuthology.orchestra.run.smithi082.stdout: fuse-2.9.7-15.el8.x86_64 2023-09-09T23:04:26.049 INFO:teuthology.orchestra.run.smithi082.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-09T23:04:26.049 INFO:teuthology.orchestra.run.smithi082.stdout: jq-1.6-3.el8.x86_64 2023-09-09T23:04:26.049 INFO:teuthology.orchestra.run.smithi082.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-09T23:04:26.049 INFO:teuthology.orchestra.run.smithi082.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-09-09T23:04:26.050 INFO:teuthology.orchestra.run.smithi082.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-09-09T23:04:26.050 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs-devel-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.050 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs2-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.050 INFO:teuthology.orchestra.run.smithi082.stdout: libcephsqlite-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.051 INFO:teuthology.orchestra.run.smithi082.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-09-09T23:04:26.051 INFO:teuthology.orchestra.run.smithi082.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-09-09T23:04:26.051 INFO:teuthology.orchestra.run.smithi082.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-09T23:04:26.051 INFO:teuthology.orchestra.run.smithi082.stdout: libpq-13.5-1.el8.x86_64 2023-09-09T23:04:26.051 INFO:teuthology.orchestra.run.smithi082.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-09-09T23:04:26.052 INFO:teuthology.orchestra.run.smithi082.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-09-09T23:04:26.052 INFO:teuthology.orchestra.run.smithi082.stdout: librados-devel-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.052 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.052 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-09T23:04:26.053 INFO:teuthology.orchestra.run.smithi082.stdout: librgw2-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.053 INFO:teuthology.orchestra.run.smithi082.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-09-09T23:04:26.053 INFO:teuthology.orchestra.run.smithi082.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-09T23:04:26.053 INFO:teuthology.orchestra.run.smithi082.stdout: libtool-2.4.6-25.el8.x86_64 2023-09-09T23:04:26.053 INFO:teuthology.orchestra.run.smithi082.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-09T23:04:26.054 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-09-09T23:04:26.054 INFO:teuthology.orchestra.run.smithi082.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-09-09T23:04:26.054 INFO:teuthology.orchestra.run.smithi082.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-09T23:04:26.055 INFO:teuthology.orchestra.run.smithi082.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-09T23:04:26.055 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-0.3.15-3.el8.x86_64 2023-09-09T23:04:26.055 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-09-09T23:04:26.055 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-09-09T23:04:26.055 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-09-09T23:04:26.056 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-09-09T23:04:26.056 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-09-09T23:04:26.056 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-09T23:04:26.056 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-09T23:04:26.056 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-09T23:04:26.057 INFO:teuthology.orchestra.run.smithi082.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-09T23:04:26.057 INFO:teuthology.orchestra.run.smithi082.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-09T23:04:26.057 INFO:teuthology.orchestra.run.smithi082.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-09T23:04:26.057 INFO:teuthology.orchestra.run.smithi082.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-09T23:04:26.058 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-09T23:04:26.058 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.058 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-common-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.058 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cephfs-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.058 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-09T23:04:26.059 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-09-09T23:04:26.059 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-09T23:04:26.059 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-09T23:04:26.059 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-09-09T23:04:26.059 INFO:teuthology.orchestra.run.smithi082.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-09T23:04:26.060 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-09T23:04:26.060 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-09T23:04:26.060 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-09T23:04:26.060 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-09T23:04:26.061 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-09T23:04:26.061 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-09T23:04:26.061 INFO:teuthology.orchestra.run.smithi082.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-09T23:04:26.061 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-09T23:04:26.061 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako-1.0.6-13.el8.noarch 2023-09-09T23:04:26.062 INFO:teuthology.orchestra.run.smithi082.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-09T23:04:26.062 INFO:teuthology.orchestra.run.smithi082.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-09T23:04:26.062 INFO:teuthology.orchestra.run.smithi082.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-09T23:04:26.062 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-09T23:04:26.063 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-09T23:04:26.063 INFO:teuthology.orchestra.run.smithi082.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-09T23:04:26.063 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-09T23:04:26.063 INFO:teuthology.orchestra.run.smithi082.stdout: python3-portend-2.6-1.el8.noarch 2023-09-09T23:04:26.063 INFO:teuthology.orchestra.run.smithi082.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-09T23:04:26.064 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-09T23:04:26.064 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-09T23:04:26.064 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-09T23:04:26.065 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-09T23:04:26.065 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytz-2017.2-9.el8.noarch 2023-09-09T23:04:26.065 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.065 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rbd-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.065 INFO:teuthology.orchestra.run.smithi082.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-09T23:04:26.066 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-09T23:04:26.066 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rgw-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.066 INFO:teuthology.orchestra.run.smithi082.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-09T23:04:26.066 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-09T23:04:26.066 INFO:teuthology.orchestra.run.smithi082.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-09-09T23:04:26.067 INFO:teuthology.orchestra.run.smithi082.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-09T23:04:26.067 INFO:teuthology.orchestra.run.smithi082.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-09T23:04:26.067 INFO:teuthology.orchestra.run.smithi082.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-09T23:04:26.067 INFO:teuthology.orchestra.run.smithi082.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-09T23:04:26.067 INFO:teuthology.orchestra.run.smithi082.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-09T23:04:26.068 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-09T23:04:26.068 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-09T23:04:26.068 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-09T23:04:26.068 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-09T23:04:26.068 INFO:teuthology.orchestra.run.smithi082.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-09T23:04:26.069 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-fuse-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.069 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-mirror-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.069 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-nbd-2:18.0.0-6025.g9670cca5.el8.x86_64 2023-09-09T23:04:26.069 INFO:teuthology.orchestra.run.smithi082.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-09T23:04:26.070 INFO:teuthology.orchestra.run.smithi082.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-09T23:04:26.070 INFO:teuthology.orchestra.run.smithi082.stdout: uuid-1.6.2-43.el8.x86_64 2023-09-09T23:04:26.070 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-09-09T23:04:26.070 INFO:teuthology.orchestra.run.smithi082.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-09T23:04:26.071 INFO:teuthology.orchestra.run.smithi082.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-09-09T23:04:26.071 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:04:26.071 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-09-09T23:04:26.241 DEBUG:teuthology.parallel:result is None 2023-09-09T23:04:26.241 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=9670cca5bc8e17916865dc84939edfc10b1f2c00 2023-09-09T23:04:26.461 DEBUG:teuthology.orchestra.run.smithi082:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-09T23:04:26.485 INFO:teuthology.orchestra.run.smithi082.stdout:18.0.0-6025.g9670cca5.el8 2023-09-09T23:04:26.486 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6025.g9670cca5.el8 2023-09-09T23:04:26.487 INFO:teuthology.task.install:The correct ceph version 18.0.0-6025.g9670cca5 is installed. 2023-09-09T23:04:26.487 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=9670cca5bc8e17916865dc84939edfc10b1f2c00 2023-09-09T23:04:26.630 DEBUG:teuthology.orchestra.run.smithi098:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-09T23:04:26.659 INFO:teuthology.orchestra.run.smithi098.stdout:18.0.0-6025.g9670cca5.el8 2023-09-09T23:04:26.659 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6025.g9670cca5.el8 2023-09-09T23:04:26.660 INFO:teuthology.task.install:The correct ceph version 18.0.0-6025.g9670cca5 is installed. 2023-09-09T23:04:26.660 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=9670cca5bc8e17916865dc84939edfc10b1f2c00 2023-09-09T23:04:26.879 DEBUG:teuthology.orchestra.run.smithi170:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-09T23:04:26.910 INFO:teuthology.orchestra.run.smithi170.stdout:18.0.0-6025.g9670cca5.el8 2023-09-09T23:04:26.910 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6025.g9670cca5.el8 2023-09-09T23:04:26.910 INFO:teuthology.task.install:The correct ceph version 18.0.0-6025.g9670cca5 is installed. 2023-09-09T23:04:26.911 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-09-09T23:04:26.911 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-09-09T23:04:26.912 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-09T23:04:26.954 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-09-09T23:04:26.954 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-09T23:04:27.001 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-09-09T23:04:27.002 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-09T23:04:27.047 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-09-09T23:04:27.048 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-09-09T23:04:27.048 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/daemon-helper 2023-09-09T23:04:27.076 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-09T23:04:27.143 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-09-09T23:04:27.143 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/daemon-helper 2023-09-09T23:04:27.174 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-09T23:04:27.245 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-09-09T23:04:27.245 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/daemon-helper 2023-09-09T23:04:27.277 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-09T23:04:27.347 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-09-09T23:04:27.348 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-09-09T23:04:27.348 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-09T23:04:27.374 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-09T23:04:27.439 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-09-09T23:04:27.440 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-09T23:04:27.472 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-09T23:04:27.542 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-09-09T23:04:27.543 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-09T23:04:27.573 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-09T23:04:27.642 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-09-09T23:04:27.643 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-09-09T23:04:27.643 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/stdin-killer 2023-09-09T23:04:27.669 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-09T23:04:27.736 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-09-09T23:04:27.736 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/stdin-killer 2023-09-09T23:04:27.767 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-09T23:04:27.838 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-09-09T23:04:27.838 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/stdin-killer 2023-09-09T23:04:27.868 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-09T23:04:27.937 INFO:teuthology.run_tasks:Running task cephadm... 2023-09-09T23:04:28.017 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mds': {'mds_bal_replicate_threshold': 1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '9670cca5bc8e17916865dc84939edfc10b1f2c00', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2023-09-09T23:04:28.017 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 2023-09-09T23:04:28.018 INFO:tasks.cephadm:Cluster fsid is 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da 2023-09-09T23:04:28.018 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-09-09T23:04:28.019 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.82', 'mon.b': '172.21.15.98', 'mon.c': '172.21.15.170'} 2023-09-09T23:04:28.019 INFO:tasks.cephadm:First mon is mon.a on smithi082 2023-09-09T23:04:28.020 INFO:tasks.cephadm:First mgr is x 2023-09-09T23:04:28.020 INFO:tasks.cephadm:Normalizing hostnames... 2023-09-09T23:04:28.020 DEBUG:teuthology.orchestra.run.smithi082:> sudo hostname $(hostname -s) 2023-09-09T23:04:28.048 DEBUG:teuthology.orchestra.run.smithi098:> sudo hostname $(hostname -s) 2023-09-09T23:04:28.081 DEBUG:teuthology.orchestra.run.smithi170:> sudo hostname $(hostname -s) 2023-09-09T23:04:28.111 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-09-09T23:04:28.111 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=9670cca5bc8e17916865dc84939edfc10b1f2c00 2023-09-09T23:04:28.332 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '9670cca5bc8e17916865dc84939edfc10b1f2c00', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/72502/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-6025-g9670cca5', 'node_name': '172.21.2.6+braggi06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-6025.g9670cca5'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-09-09 16:32:07.237982', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20230909.145638', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2023-09-09T23:04:28.475 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-batrick-testing-20230909.145638, sha1 9670cca5bc8e17916865dc84939edfc10b1f2c00 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=9670cca5bc8e17916865dc84939edfc10b1f2c00 2023-09-09T23:04:28.476 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/x86_64/flavors/default/cephadm 2023-09-09T23:04:28.476 DEBUG:teuthology.orchestra.run.smithi082:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-09-09T23:04:28.868 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-rw-r--. 1 ubuntu ubuntu 724179 Sep 9 23:04 /home/ubuntu/cephtest/cephadm 2023-09-09T23:04:28.869 DEBUG:teuthology.orchestra.run.smithi098:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-09-09T23:04:29.226 INFO:teuthology.orchestra.run.smithi098.stdout:-rw-rw-r--. 1 ubuntu ubuntu 724179 Sep 9 23:04 /home/ubuntu/cephtest/cephadm 2023-09-09T23:04:29.227 DEBUG:teuthology.orchestra.run.smithi170:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20230909.145638/9670cca5bc8e17916865dc84939edfc10b1f2c00/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-09-09T23:04:29.753 INFO:teuthology.orchestra.run.smithi170.stdout:-rw-rw-r--. 1 ubuntu ubuntu 724179 Sep 9 23:04 /home/ubuntu/cephtest/cephadm 2023-09-09T23:04:29.754 DEBUG:teuthology.orchestra.run.smithi082:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-09-09T23:04:29.775 DEBUG:teuthology.orchestra.run.smithi098:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-09-09T23:04:29.796 DEBUG:teuthology.orchestra.run.smithi170:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-09-09T23:04:29.842 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 on all hosts... 2023-09-09T23:04:29.843 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 pull 2023-09-09T23:04:29.846 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 pull 2023-09-09T23:04:29.849 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 pull 2023-09-09T23:04:33.141 INFO:teuthology.orchestra.run.smithi170.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00... 2023-09-09T23:04:35.097 INFO:teuthology.orchestra.run.smithi082.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00... 2023-09-09T23:04:35.388 INFO:teuthology.orchestra.run.smithi098.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00... 2023-09-09T23:05:02.480 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2023-09-09T23:05:02.481 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_version": "ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev)", 2023-09-09T23:05:02.481 INFO:teuthology.orchestra.run.smithi098.stdout: "image_id": "20c3861ef6c981f72050dd1dd3edbdc62747039bda543319e240a6e02f850eea", 2023-09-09T23:05:02.482 INFO:teuthology.orchestra.run.smithi098.stdout: "repo_digests": [ 2023-09-09T23:05:02.482 INFO:teuthology.orchestra.run.smithi098.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8bb87e43f4b5fd23ed47b24bb48a71d3a37a4d237bf8cf5ca0615963ecee90da" 2023-09-09T23:05:02.483 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2023-09-09T23:05:02.483 INFO:teuthology.orchestra.run.smithi098.stdout:} 2023-09-09T23:05:03.122 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-09-09T23:05:03.123 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_version": "ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev)", 2023-09-09T23:05:03.123 INFO:teuthology.orchestra.run.smithi170.stdout: "image_id": "20c3861ef6c981f72050dd1dd3edbdc62747039bda543319e240a6e02f850eea", 2023-09-09T23:05:03.123 INFO:teuthology.orchestra.run.smithi170.stdout: "repo_digests": [ 2023-09-09T23:05:03.123 INFO:teuthology.orchestra.run.smithi170.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8bb87e43f4b5fd23ed47b24bb48a71d3a37a4d237bf8cf5ca0615963ecee90da" 2023-09-09T23:05:03.124 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2023-09-09T23:05:03.124 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-09-09T23:05:04.595 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:05:04.595 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev)", 2023-09-09T23:05:04.595 INFO:teuthology.orchestra.run.smithi082.stdout: "image_id": "20c3861ef6c981f72050dd1dd3edbdc62747039bda543319e240a6e02f850eea", 2023-09-09T23:05:04.595 INFO:teuthology.orchestra.run.smithi082.stdout: "repo_digests": [ 2023-09-09T23:05:04.596 INFO:teuthology.orchestra.run.smithi082.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8bb87e43f4b5fd23ed47b24bb48a71d3a37a4d237bf8cf5ca0615963ecee90da" 2023-09-09T23:05:04.596 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2023-09-09T23:05:04.596 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:05:04.615 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /etc/ceph 2023-09-09T23:05:04.645 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /etc/ceph 2023-09-09T23:05:04.682 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /etc/ceph 2023-09-09T23:05:04.718 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 777 /etc/ceph 2023-09-09T23:05:04.744 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 777 /etc/ceph 2023-09-09T23:05:04.775 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 777 /etc/ceph 2023-09-09T23:05:04.805 INFO:tasks.cephadm:Writing seed config... 2023-09-09T23:05:04.806 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-09-09T23:05:04.807 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2023-09-09T23:05:04.807 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-09-09T23:05:04.807 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-09-09T23:05:04.807 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-09-09T23:05:04.808 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-09-09T23:05:04.808 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-09-09T23:05:04.808 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-09-09T23:05:04.808 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-09-09T23:05:04.809 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2023-09-09T23:05:04.809 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2023-09-09T23:05:04.809 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2023-09-09T23:05:04.809 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-09-09T23:05:04.810 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-09-09T23:05:04.810 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2023-09-09T23:05:04.810 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2023-09-09T23:05:04.811 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-09-09T23:05:04.811 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-09-09T23:05:04.811 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2023-09-09T23:05:04.811 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-09-09T23:05:04.812 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-09-09T23:05:04.812 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-09-09T23:05:04.812 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-09-09T23:05:04.812 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-09-09T23:05:04.813 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-09-09T23:05:04.813 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-09-09T23:05:04.814 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-09-09T23:05:04.814 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-09-09T23:05:04.830 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [mds] mds_bal_replicate_threshold = 1 2023-09-09T23:05:04.831 DEBUG:teuthology.orchestra.run.smithi082:mon.a> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@mon.a.service 2023-09-09T23:05:04.873 DEBUG:teuthology.orchestra.run.smithi082:mgr.x> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@mgr.x.service 2023-09-09T23:05:04.898 INFO:journalctl@ceph.mon.a.smithi082.stdout:-- Logs begin at Sat 2023-09-09 22:51:24 UTC. -- 2023-09-09T23:05:04.899 INFO:tasks.cephadm:Bootstrapping... 2023-09-09T23:05:04.899 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 -v bootstrap --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.82 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-09-09T23:05:04.923 INFO:journalctl@ceph.mgr.x.smithi082.stdout:-- Logs begin at Sat 2023-09-09 22:51:24 UTC. -- 2023-09-09T23:05:05.058 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2023-09-09T23:05:05.059 INFO:teuthology.orchestra.run.smithi082.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00', '-v', 'bootstrap', '--fsid', '3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.82', '--skip-admin-label'] 2023-09-09T23:05:05.204 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stdout 4.1.1 2023-09-09T23:05:05.205 INFO:teuthology.orchestra.run.smithi082.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-09-09T23:05:05.205 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying podman|docker is present... 2023-09-09T23:05:05.346 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stdout 4.1.1 2023-09-09T23:05:05.346 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying lvm2 is present... 2023-09-09T23:05:05.346 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying time synchronization is in place... 2023-09-09T23:05:05.353 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-09-09T23:05:05.353 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-09-09T23:05:05.358 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-09-09T23:05:05.359 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2023-09-09T23:05:05.364 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout enabled 2023-09-09T23:05:05.370 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout active 2023-09-09T23:05:05.370 INFO:teuthology.orchestra.run.smithi082.stdout:Unit chronyd.service is enabled and running 2023-09-09T23:05:05.371 INFO:teuthology.orchestra.run.smithi082.stdout:Repeating the final host check... 2023-09-09T23:05:05.514 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stdout 4.1.1 2023-09-09T23:05:05.514 INFO:teuthology.orchestra.run.smithi082.stdout:podman (/bin/podman) version 4.1.1 is present 2023-09-09T23:05:05.514 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl is present 2023-09-09T23:05:05.515 INFO:teuthology.orchestra.run.smithi082.stdout:lvcreate is present 2023-09-09T23:05:05.520 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-09-09T23:05:05.520 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-09-09T23:05:05.525 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-09-09T23:05:05.526 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2023-09-09T23:05:05.531 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout enabled 2023-09-09T23:05:05.537 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout active 2023-09-09T23:05:05.538 INFO:teuthology.orchestra.run.smithi082.stdout:Unit chronyd.service is enabled and running 2023-09-09T23:05:05.538 INFO:teuthology.orchestra.run.smithi082.stdout:Host looks OK 2023-09-09T23:05:05.538 INFO:teuthology.orchestra.run.smithi082.stdout:Cluster fsid: 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da 2023-09-09T23:05:05.538 INFO:teuthology.orchestra.run.smithi082.stdout:Acquiring lock 139752889476712 on /run/cephadm/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da.lock 2023-09-09T23:05:05.539 INFO:teuthology.orchestra.run.smithi082.stdout:Lock 139752889476712 acquired on /run/cephadm/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da.lock 2023-09-09T23:05:05.539 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying IP 172.21.15.82 port 3300 ... 2023-09-09T23:05:05.539 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying IP 172.21.15.82 port 6789 ... 2023-09-09T23:05:05.540 INFO:teuthology.orchestra.run.smithi082.stdout:Base mon IP(s) is [172.21.15.82:3300, 172.21.15.82:6789], mon addrv is [v2:172.21.15.82:3300,v1:172.21.15.82:6789] 2023-09-09T23:05:05.542 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2023-09-09T23:05:05.543 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.82 metric 100 2023-09-09T23:05:05.544 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-09-09T23:05:05.544 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2023-09-09T23:05:05.545 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2023-09-09T23:05:05.547 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-09-09T23:05:05.547 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-09-09T23:05:05.547 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-09-09T23:05:05.547 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-09-09T23:05:05.548 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout inet6 fe80::164c:74f1:387:f0a9/64 scope link noprefixroute 2023-09-09T23:05:05.548 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-09-09T23:05:05.549 INFO:teuthology.orchestra.run.smithi082.stdout:Mon IP `172.21.15.82` is in CIDR network `172.21.0.0/20` 2023-09-09T23:05:05.549 INFO:teuthology.orchestra.run.smithi082.stdout:Mon IP `172.21.15.82` is in CIDR network `172.21.0.0/20` 2023-09-09T23:05:05.549 INFO:teuthology.orchestra.run.smithi082.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-09-09T23:05:05.549 INFO:teuthology.orchestra.run.smithi082.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-09-09T23:05:05.550 INFO:teuthology.orchestra.run.smithi082.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00... 2023-09-09T23:05:06.392 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stdout 20c3861ef6c981f72050dd1dd3edbdc62747039bda543319e240a6e02f850eea 2023-09-09T23:05:06.393 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00... 2023-09-09T23:05:06.393 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Getting image source signatures 2023-09-09T23:05:06.393 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Copying blob sha256:2d7a35eed4a09aaaa91d235ae74efd6175638ed8fde15ffafe0c3c53db478b03 2023-09-09T23:05:06.393 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Copying blob sha256:d3e1674b1d277d1d98aa3561ad0e3209c8e2f26183d8e8b51d0d9fbac358121d 2023-09-09T23:05:06.394 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Copying config sha256:20c3861ef6c981f72050dd1dd3edbdc62747039bda543319e240a6e02f850eea 2023-09-09T23:05:06.394 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Writing manifest to image destination 2023-09-09T23:05:06.394 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Storing signatures 2023-09-09T23:05:07.145 INFO:teuthology.orchestra.run.smithi082.stdout:ceph: stdout ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-09T23:05:07.145 INFO:teuthology.orchestra.run.smithi082.stdout:Ceph version: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev) 2023-09-09T23:05:07.145 INFO:teuthology.orchestra.run.smithi082.stdout:Extracting ceph user uid/gid from container image... 2023-09-09T23:05:07.791 INFO:teuthology.orchestra.run.smithi082.stdout:stat: stdout 167 167 2023-09-09T23:05:07.791 INFO:teuthology.orchestra.run.smithi082.stdout:Creating initial keys... 2023-09-09T23:05:08.439 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph-authtool: stdout AQAk+vxkY38OCBAAfnxL/Ic8ZIdDOH26jvVZgA== 2023-09-09T23:05:09.086 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph-authtool: stdout AQAk+vxk3r2RLRAA+t3uFOASbQNjzZRNbCcFSw== 2023-09-09T23:05:09.737 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph-authtool: stdout AQAl+vxkD6BsGBAAwMy7F+s4stznwlvULY036w== 2023-09-09T23:05:09.737 INFO:teuthology.orchestra.run.smithi082.stdout:Creating initial monmap... 2023-09-09T23:05:10.385 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-09-09T23:05:10.385 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-09-09T23:05:10.386 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da 2023-09-09T23:05:10.386 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-09-09T23:05:10.386 INFO:teuthology.orchestra.run.smithi082.stdout:monmaptool for a [v2:172.21.15.82:3300,v1:172.21.15.82:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-09-09T23:05:10.386 INFO:teuthology.orchestra.run.smithi082.stdout:setting min_mon_release = pacific 2023-09-09T23:05:10.386 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: set fsid to 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da 2023-09-09T23:05:10.387 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-09-09T23:05:10.387 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:05:10.387 INFO:teuthology.orchestra.run.smithi082.stdout:Creating mon... 2023-09-09T23:05:12.232 INFO:teuthology.orchestra.run.smithi082.stdout:create mon.a on 2023-09-09T23:05:12.536 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-09-09T23:05:12.624 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-09-09T23:05:13.027 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da.target → /etc/systemd/system/ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da.target. 2023-09-09T23:05:13.028 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da.target → /etc/systemd/system/ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da.target. 2023-09-09T23:05:13.547 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@mon.a 2023-09-09T23:05:13.547 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@mon.a.service: Unit ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@mon.a.service not loaded. 2023-09-09T23:05:13.632 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da.target.wants/ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@mon.a.service → /etc/systemd/system/ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@.service. 2023-09-09T23:05:13.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:13 smithi082 systemd[1]: Starting Ceph mon.a for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da... 2023-09-09T23:05:14.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:13 smithi082 podman[98142]: 2023-09-09T23:05:14.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:14 smithi082 bash[98142]: d867f15273f627c2bbb912b2992ae9d7559c4cf33fc65aaf246c4ad0d2b564fc 2023-09-09T23:05:14.236 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-09T23:05:14.237 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout disabled 2023-09-09T23:05:14.243 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-09T23:05:14.243 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2023-09-09T23:05:14.243 INFO:teuthology.orchestra.run.smithi082.stdout:firewalld.service is not enabled 2023-09-09T23:05:14.244 INFO:teuthology.orchestra.run.smithi082.stdout:Not possible to enable service . firewalld.service is not available 2023-09-09T23:05:14.244 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mon to start... 2023-09-09T23:05:14.245 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mon... 2023-09-09T23:05:14.489 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:14 smithi082 ceph-mon[98183]: mkfs 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da 2023-09-09T23:05:14.489 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:14 smithi082 ceph-mon[98183]: mon.a is new leader, mons a in quorum (ranks 0) 2023-09-09T23:05:14.489 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:14 smithi082 systemd[1]: Started Ceph mon.a for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da. 2023-09-09T23:05:15.093 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout cluster: 2023-09-09T23:05:15.094 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout id: 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da 2023-09-09T23:05:15.094 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-09-09T23:05:15.094 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2023-09-09T23:05:15.095 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout services: 2023-09-09T23:05:15.095 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.556322s) 2023-09-09T23:05:15.095 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-09-09T23:05:15.096 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-09-09T23:05:15.096 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2023-09-09T23:05:15.096 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout data: 2023-09-09T23:05:15.097 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-09-09T23:05:15.097 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-09-09T23:05:15.098 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-09-09T23:05:15.098 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout pgs: 2023-09-09T23:05:15.098 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2023-09-09T23:05:15.099 INFO:teuthology.orchestra.run.smithi082.stdout:mon is available 2023-09-09T23:05:15.099 INFO:teuthology.orchestra.run.smithi082.stdout:Assimilating anything we can from ceph.conf... 2023-09-09T23:05:15.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:15 smithi082 ceph-mon[98183]: mon.a is new leader, mons a in quorum (ranks 0) 2023-09-09T23:05:15.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:15 smithi082 ceph-mon[98183]: monmap e1: 1 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0]} removed_ranks: {} 2023-09-09T23:05:15.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:15 smithi082 ceph-mon[98183]: fsmap 2023-09-09T23:05:15.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:15 smithi082 ceph-mon[98183]: osdmap e1: 0 total, 0 up, 0 in 2023-09-09T23:05:15.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:15 smithi082 ceph-mon[98183]: mgrmap e1: no daemons active 2023-09-09T23:05:15.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:15 smithi082 ceph-mon[98183]: from='client.? 172.21.15.82:0/3221993670' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-09-09T23:05:15.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2023-09-09T23:05:15.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [global] 2023-09-09T23:05:15.935 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout fsid = 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da 2023-09-09T23:05:15.936 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.82:3300,v1:172.21.15.82:6789] 2023-09-09T23:05:15.936 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-09-09T23:05:15.936 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-09-09T23:05:15.936 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-09-09T23:05:15.937 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-09-09T23:05:15.937 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2023-09-09T23:05:15.937 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [mgr] 2023-09-09T23:05:15.937 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-09-09T23:05:15.937 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2023-09-09T23:05:15.938 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [osd] 2023-09-09T23:05:15.938 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-09-09T23:05:15.938 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-09-09T23:05:15.938 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-09-09T23:05:15.938 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-09-09T23:05:15.939 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-09-09T23:05:15.939 INFO:teuthology.orchestra.run.smithi082.stdout:Generating new minimal ceph.conf... 2023-09-09T23:05:16.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:16 smithi082 ceph-mon[98183]: from='client.? 172.21.15.82:0/1164404587' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-09-09T23:05:16.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:16 smithi082 ceph-mon[98183]: from='client.? 172.21.15.82:0/1164404587' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-09-09T23:05:16.775 INFO:teuthology.orchestra.run.smithi082.stdout:Restarting the monitor... 2023-09-09T23:05:17.123 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:16 smithi082 systemd[1]: Stopping Ceph mon.a for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da... 2023-09-09T23:05:17.374 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:17 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mon-a[98158]: 2023-09-09T23:05:17.123+0000 7f391d500700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-09T23:05:17.375 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:17 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mon-a[98158]: 2023-09-09T23:05:17.123+0000 7f391d500700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-09-09T23:05:17.375 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:17 smithi082 bash[98520]: ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mon-a 2023-09-09T23:05:17.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:17 smithi082 systemd[1]: ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@mon.a.service: Succeeded. 2023-09-09T23:05:17.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:17 smithi082 systemd[1]: Stopped Ceph mon.a for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da. 2023-09-09T23:05:17.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:17 smithi082 systemd[1]: Starting Ceph mon.a for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da... 2023-09-09T23:05:18.160 INFO:teuthology.orchestra.run.smithi082.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-09-09T23:05:18.229 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:17 smithi082 podman[98654]: 2023-09-09T23:05:18.229 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 bash[98654]: bf092fc33c238c71954a0736421c00ed0e2e991bdba71a6cb3a882c58ee7e7b6 2023-09-09T23:05:18.229 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: set uid:gid to 167:167 (ceph:ceph) 2023-09-09T23:05:18.229 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev), process ceph-mon, pid 7 2023-09-09T23:05:18.230 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: pidfile_write: ignore empty --pid-file 2023-09-09T23:05:18.230 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: load: jerasure load: lrc 2023-09-09T23:05:18.230 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: RocksDB version: 7.9.2 2023-09-09T23:05:18.230 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Git sha 0 2023-09-09T23:05:18.230 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Compile date 2023-09-09 15:16:58 2023-09-09T23:05:18.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: DB SUMMARY 2023-09-09T23:05:18.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: DB Session ID: 3435MRQ20N68V349WOIC 2023-09-09T23:05:18.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: CURRENT file: CURRENT 2023-09-09T23:05:18.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: IDENTITY file: IDENTITY 2023-09-09T23:05:18.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-09-09T23:05:18.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-09-09T23:05:18.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 89112 ; 2023-09-09T23:05:18.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.error_if_exists: 0 2023-09-09T23:05:18.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.create_if_missing: 0 2023-09-09T23:05:18.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.paranoid_checks: 1 2023-09-09T23:05:18.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.flush_verify_memtable_count: 1 2023-09-09T23:05:18.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-09-09T23:05:18.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-09-09T23:05:18.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.env: 0x562d8db9cda0 2023-09-09T23:05:18.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.fs: PosixFileSystem 2023-09-09T23:05:18.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.info_log: 0x562d8f35d4e0 2023-09-09T23:05:18.234 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.max_file_opening_threads: 16 2023-09-09T23:05:18.234 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.statistics: (nil) 2023-09-09T23:05:18.234 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.use_fsync: 0 2023-09-09T23:05:18.234 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.max_log_file_size: 0 2023-09-09T23:05:18.235 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-09-09T23:05:18.235 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.log_file_time_to_roll: 0 2023-09-09T23:05:18.235 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.keep_log_file_num: 1000 2023-09-09T23:05:18.235 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.recycle_log_file_num: 0 2023-09-09T23:05:18.235 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.allow_fallocate: 1 2023-09-09T23:05:18.236 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.allow_mmap_reads: 0 2023-09-09T23:05:18.236 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.allow_mmap_writes: 0 2023-09-09T23:05:18.236 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.use_direct_reads: 0 2023-09-09T23:05:18.236 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-09-09T23:05:18.236 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.create_missing_column_families: 0 2023-09-09T23:05:18.237 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.db_log_dir: 2023-09-09T23:05:18.237 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.wal_dir: 2023-09-09T23:05:18.237 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.table_cache_numshardbits: 6 2023-09-09T23:05:18.237 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.WAL_ttl_seconds: 0 2023-09-09T23:05:18.237 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.WAL_size_limit_MB: 0 2023-09-09T23:05:18.237 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-09-09T23:05:18.238 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-09-09T23:05:18.238 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.is_fd_close_on_exec: 1 2023-09-09T23:05:18.238 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.advise_random_on_open: 1 2023-09-09T23:05:18.238 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.db_write_buffer_size: 0 2023-09-09T23:05:18.238 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.write_buffer_manager: 0x562d8f3174a0 2023-09-09T23:05:18.239 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-09-09T23:05:18.239 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-09-09T23:05:18.239 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.use_adaptive_mutex: 0 2023-09-09T23:05:18.239 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.rate_limiter: (nil) 2023-09-09T23:05:18.239 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-09-09T23:05:18.240 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.wal_recovery_mode: 2 2023-09-09T23:05:18.240 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.enable_thread_tracking: 0 2023-09-09T23:05:18.240 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.enable_pipelined_write: 0 2023-09-09T23:05:18.240 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.unordered_write: 0 2023-09-09T23:05:18.240 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-09-09T23:05:18.240 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-09-09T23:05:18.241 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-09-09T23:05:18.241 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-09-09T23:05:18.241 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.row_cache: None 2023-09-09T23:05:18.241 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.wal_filter: None 2023-09-09T23:05:18.241 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-09-09T23:05:18.242 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.allow_ingest_behind: 0 2023-09-09T23:05:18.242 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.two_write_queues: 0 2023-09-09T23:05:18.242 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.manual_wal_flush: 0 2023-09-09T23:05:18.242 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.wal_compression: 0 2023-09-09T23:05:18.242 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.atomic_flush: 0 2023-09-09T23:05:18.242 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-09-09T23:05:18.243 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.persist_stats_to_disk: 0 2023-09-09T23:05:18.243 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.write_dbid_to_manifest: 0 2023-09-09T23:05:18.243 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.log_readahead_size: 0 2023-09-09T23:05:18.243 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-09-09T23:05:18.243 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.best_efforts_recovery: 0 2023-09-09T23:05:18.244 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-09-09T23:05:18.244 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-09-09T23:05:18.244 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.allow_data_in_errors: 0 2023-09-09T23:05:18.244 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.db_host_id: __hostname__ 2023-09-09T23:05:18.244 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.enforce_single_del_contracts: true 2023-09-09T23:05:18.244 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.max_background_jobs: 2 2023-09-09T23:05:18.245 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.max_background_compactions: -1 2023-09-09T23:05:18.245 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.max_subcompactions: 1 2023-09-09T23:05:18.245 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-09-09T23:05:18.245 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-09-09T23:05:18.245 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.delayed_write_rate : 16777216 2023-09-09T23:05:18.246 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.max_total_wal_size: 0 2023-09-09T23:05:18.246 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-09-09T23:05:18.246 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.stats_dump_period_sec: 600 2023-09-09T23:05:18.246 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.stats_persist_period_sec: 600 2023-09-09T23:05:18.246 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-09-09T23:05:18.246 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.max_open_files: -1 2023-09-09T23:05:18.247 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.bytes_per_sync: 0 2023-09-09T23:05:18.247 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.wal_bytes_per_sync: 0 2023-09-09T23:05:18.247 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.strict_bytes_per_sync: 0 2023-09-09T23:05:18.247 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.compaction_readahead_size: 0 2023-09-09T23:05:18.247 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.max_background_flushes: -1 2023-09-09T23:05:18.248 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Compression algorithms supported: 2023-09-09T23:05:18.248 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-09-09T23:05:18.248 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: kZSTD supported: 0 2023-09-09T23:05:18.248 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: kXpressCompression supported: 0 2023-09-09T23:05:18.248 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: kLZ4HCCompression supported: 1 2023-09-09T23:05:18.249 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: kZlibCompression supported: 1 2023-09-09T23:05:18.249 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: kSnappyCompression supported: 1 2023-09-09T23:05:18.249 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: kLZ4Compression supported: 1 2023-09-09T23:05:18.249 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: kBZip2Compression supported: 0 2023-09-09T23:05:18.249 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Fast CRC32 supported: Supported on x86 2023-09-09T23:05:18.249 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: DMutex implementation: pthread_mutex_t 2023-09-09T23:05:18.250 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-09-09T23:05:18.250 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-09-09T23:05:18.250 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-09-09T23:05:18.250 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.merge_operator: 2023-09-09T23:05:18.250 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.compaction_filter: None 2023-09-09T23:05:18.251 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.compaction_filter_factory: None 2023-09-09T23:05:18.251 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.sst_partitioner_factory: None 2023-09-09T23:05:18.251 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.memtable_factory: SkipListFactory 2023-09-09T23:05:18.251 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.table_factory: BlockBasedTable 2023-09-09T23:05:18.251 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562d8f35d5e0) 2023-09-09T23:05:18.251 INFO:journalctl@ceph.mon.a.smithi082.stdout: cache_index_and_filter_blocks: 1 2023-09-09T23:05:18.252 INFO:journalctl@ceph.mon.a.smithi082.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-09-09T23:05:18.252 INFO:journalctl@ceph.mon.a.smithi082.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-09-09T23:05:18.252 INFO:journalctl@ceph.mon.a.smithi082.stdout: pin_top_level_index_and_filter: 1 2023-09-09T23:05:18.252 INFO:journalctl@ceph.mon.a.smithi082.stdout: index_type: 0 2023-09-09T23:05:18.252 INFO:journalctl@ceph.mon.a.smithi082.stdout: data_block_index_type: 0 2023-09-09T23:05:18.253 INFO:journalctl@ceph.mon.a.smithi082.stdout: index_shortening: 1 2023-09-09T23:05:18.253 INFO:journalctl@ceph.mon.a.smithi082.stdout: data_block_hash_table_util_ratio: 0.750000 2023-09-09T23:05:18.253 INFO:journalctl@ceph.mon.a.smithi082.stdout: checksum: 4 2023-09-09T23:05:18.253 INFO:journalctl@ceph.mon.a.smithi082.stdout: no_block_cache: 0 2023-09-09T23:05:18.253 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_cache: 0x562d8f3b51f0 2023-09-09T23:05:18.253 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_cache_name: BinnedLRUCache 2023-09-09T23:05:18.254 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_cache_options: 2023-09-09T23:05:18.254 INFO:journalctl@ceph.mon.a.smithi082.stdout: capacity : 536870912 2023-09-09T23:05:18.254 INFO:journalctl@ceph.mon.a.smithi082.stdout: num_shard_bits : 4 2023-09-09T23:05:18.254 INFO:journalctl@ceph.mon.a.smithi082.stdout: strict_capacity_limit : 0 2023-09-09T23:05:18.254 INFO:journalctl@ceph.mon.a.smithi082.stdout: high_pri_pool_ratio: 0.000 2023-09-09T23:05:18.255 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_cache_compressed: (nil) 2023-09-09T23:05:18.255 INFO:journalctl@ceph.mon.a.smithi082.stdout: persistent_cache: (nil) 2023-09-09T23:05:18.255 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_size: 4096 2023-09-09T23:05:18.255 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_size_deviation: 10 2023-09-09T23:05:18.255 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_restart_interval: 16 2023-09-09T23:05:18.255 INFO:journalctl@ceph.mon.a.smithi082.stdout: index_block_restart_interval: 1 2023-09-09T23:05:18.256 INFO:journalctl@ceph.mon.a.smithi082.stdout: metadata_block_size: 4096 2023-09-09T23:05:18.256 INFO:journalctl@ceph.mon.a.smithi082.stdout: partition_filters: 0 2023-09-09T23:05:18.256 INFO:journalctl@ceph.mon.a.smithi082.stdout: use_delta_encoding: 1 2023-09-09T23:05:18.256 INFO:journalctl@ceph.mon.a.smithi082.stdout: filter_policy: bloomfilter 2023-09-09T23:05:18.256 INFO:journalctl@ceph.mon.a.smithi082.stdout: whole_key_filtering: 1 2023-09-09T23:05:18.257 INFO:journalctl@ceph.mon.a.smithi082.stdout: verify_compression: 0 2023-09-09T23:05:18.257 INFO:journalctl@ceph.mon.a.smithi082.stdout: read_amp_bytes_per_bit: 0 2023-09-09T23:05:18.257 INFO:journalctl@ceph.mon.a.smithi082.stdout: format_version: 5 2023-09-09T23:05:18.257 INFO:journalctl@ceph.mon.a.smithi082.stdout: enable_index_compression: 1 2023-09-09T23:05:18.257 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_align: 0 2023-09-09T23:05:18.257 INFO:journalctl@ceph.mon.a.smithi082.stdout: max_auto_readahead_size: 262144 2023-09-09T23:05:18.258 INFO:journalctl@ceph.mon.a.smithi082.stdout: prepopulate_block_cache: 0 2023-09-09T23:05:18.258 INFO:journalctl@ceph.mon.a.smithi082.stdout: initial_auto_readahead_size: 8192 2023-09-09T23:05:18.258 INFO:journalctl@ceph.mon.a.smithi082.stdout: num_file_reads_for_auto_readahead: 2 2023-09-09T23:05:18.258 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.write_buffer_size: 33554432 2023-09-09T23:05:18.258 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.max_write_buffer_number: 2 2023-09-09T23:05:18.259 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.compression: NoCompression 2023-09-09T23:05:18.259 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.bottommost_compression: Disabled 2023-09-09T23:05:18.259 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.prefix_extractor: nullptr 2023-09-09T23:05:18.259 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-09-09T23:05:18.259 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.num_levels: 7 2023-09-09T23:05:18.260 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-09-09T23:05:18.260 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-09-09T23:05:18.260 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-09-09T23:05:18.260 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-09-09T23:05:18.260 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-09-09T23:05:18.260 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-09-09T23:05:18.261 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-09-09T23:05:18.261 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-09-09T23:05:18.261 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-09-09T23:05:18.261 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-09-09T23:05:18.261 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-09-09T23:05:18.262 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-09-09T23:05:18.262 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.compression_opts.window_bits: -14 2023-09-09T23:05:18.262 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.compression_opts.level: 32767 2023-09-09T23:05:18.262 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.compression_opts.strategy: 0 2023-09-09T23:05:18.262 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-09-09T23:05:18.263 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-09-09T23:05:18.263 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-09-09T23:05:18.263 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-09-09T23:05:18.263 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.compression_opts.enabled: false 2023-09-09T23:05:18.263 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-09-09T23:05:18.263 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-09-09T23:05:18.264 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-09-09T23:05:18.264 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-09-09T23:05:18.264 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.target_file_size_base: 67108864 2023-09-09T23:05:18.264 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.target_file_size_multiplier: 1 2023-09-09T23:05:18.264 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-09-09T23:05:18.265 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-09-09T23:05:18.265 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-09-09T23:05:18.265 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-09-09T23:05:18.265 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-09-09T23:05:18.265 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-09-09T23:05:18.266 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-09-09T23:05:18.266 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-09-09T23:05:18.266 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-09-09T23:05:18.266 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-09-09T23:05:18.266 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-09-09T23:05:18.266 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-09-09T23:05:18.267 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-09-09T23:05:18.267 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.arena_block_size: 1048576 2023-09-09T23:05:18.267 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-09-09T23:05:18.267 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-09-09T23:05:18.267 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.disable_auto_compactions: 0 2023-09-09T23:05:18.268 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-09-09T23:05:18.268 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-09-09T23:05:18.269 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-09-09T23:05:18.269 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-09-09T23:05:18.269 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-09-09T23:05:18.269 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-09-09T23:05:18.269 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-09-09T23:05:18.270 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-09-09T23:05:18.270 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-09-09T23:05:18.270 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-09-09T23:05:18.270 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.table_properties_collectors: 2023-09-09T23:05:18.270 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.inplace_update_support: 0 2023-09-09T23:05:18.271 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.inplace_update_num_locks: 10000 2023-09-09T23:05:18.271 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-09-09T23:05:18.271 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-09-09T23:05:18.271 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.memtable_huge_page_size: 0 2023-09-09T23:05:18.271 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.bloom_locality: 0 2023-09-09T23:05:18.272 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.max_successive_merges: 0 2023-09-09T23:05:18.272 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.optimize_filters_for_hits: 0 2023-09-09T23:05:18.272 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.paranoid_file_checks: 0 2023-09-09T23:05:18.272 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.force_consistency_checks: 1 2023-09-09T23:05:18.272 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.report_bg_io_stats: 0 2023-09-09T23:05:18.273 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.ttl: 2592000 2023-09-09T23:05:18.273 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.periodic_compaction_seconds: 0 2023-09-09T23:05:18.273 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-09-09T23:05:18.273 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-09-09T23:05:18.273 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.enable_blob_files: false 2023-09-09T23:05:18.273 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.min_blob_size: 0 2023-09-09T23:05:18.274 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.blob_file_size: 268435456 2023-09-09T23:05:18.274 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.blob_compression_type: NoCompression 2023-09-09T23:05:18.274 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.enable_blob_garbage_collection: false 2023-09-09T23:05:18.274 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-09-09T23:05:18.274 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-09-09T23:05:18.275 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-09-09T23:05:18.275 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.blob_file_starting_level: 0 2023-09-09T23:05:18.275 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-09-09T23:05:18.275 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-09-09T23:05:18.275 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-09-09T23:05:18.276 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5b642798-4023-4579-9273-00ce834e814b 2023-09-09T23:05:18.276 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694300718155801, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-09-09T23:05:18.276 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-09-09T23:05:18.276 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694300718156734, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 85407, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 271, "table_properties": {"data_size": 83488, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11504, "raw_average_key_size": 49, "raw_value_size": 77026, "raw_average_value_size": 329, "num_data_blocks": 11, "num_entries": 234, "num_filter_entries": 234, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1694300718, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5b642798-4023-4579-9273-00ce834e814b", "db_session_id": "3435MRQ20N68V349WOIC", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-09-09T23:05:18.277 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694300718156803, "job": 1, "event": "recovery_finished"} 2023-09-09T23:05:18.277 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-09-09T23:05:18.277 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-09-09T23:05:18.277 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562d8f450000 2023-09-09T23:05:18.277 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: DB pointer 0x562d8f43c000 2023-09-09T23:05:18.277 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-09-09T23:05:18.278 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-09-09T23:05:18.278 INFO:journalctl@ceph.mon.a.smithi082.stdout: ** DB Stats ** 2023-09-09T23:05:18.278 INFO:journalctl@ceph.mon.a.smithi082.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-09T23:05:18.278 INFO:journalctl@ceph.mon.a.smithi082.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-09-09T23:05:18.278 INFO:journalctl@ceph.mon.a.smithi082.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-09T23:05:18.279 INFO:journalctl@ceph.mon.a.smithi082.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-09T23:05:18.279 INFO:journalctl@ceph.mon.a.smithi082.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-09-09T23:05:18.279 INFO:journalctl@ceph.mon.a.smithi082.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-09T23:05:18.279 INFO:journalctl@ceph.mon.a.smithi082.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-09T23:05:18.279 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2023-09-09T23:05:18.280 INFO:journalctl@ceph.mon.a.smithi082.stdout: ** Compaction Stats [default] ** 2023-09-09T23:05:18.280 INFO:journalctl@ceph.mon.a.smithi082.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-09T23:05:18.280 INFO:journalctl@ceph.mon.a.smithi082.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-09-09T23:05:18.280 INFO:journalctl@ceph.mon.a.smithi082.stdout: L0 2/0 85.24 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 130.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-09T23:05:18.280 INFO:journalctl@ceph.mon.a.smithi082.stdout: Sum 2/0 85.24 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 130.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-09T23:05:18.280 INFO:journalctl@ceph.mon.a.smithi082.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 130.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-09T23:05:18.281 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2023-09-09T23:05:18.282 INFO:journalctl@ceph.mon.a.smithi082.stdout: ** Compaction Stats [default] ** 2023-09-09T23:05:18.282 INFO:journalctl@ceph.mon.a.smithi082.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-09T23:05:18.282 INFO:journalctl@ceph.mon.a.smithi082.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-09-09T23:05:18.282 INFO:journalctl@ceph.mon.a.smithi082.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 130.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-09T23:05:18.283 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2023-09-09T23:05:18.283 INFO:journalctl@ceph.mon.a.smithi082.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-09-09T23:05:18.283 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2023-09-09T23:05:18.283 INFO:journalctl@ceph.mon.a.smithi082.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-09T23:05:18.283 INFO:journalctl@ceph.mon.a.smithi082.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-09-09T23:05:18.283 INFO:journalctl@ceph.mon.a.smithi082.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-09-09T23:05:18.284 INFO:journalctl@ceph.mon.a.smithi082.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-09-09T23:05:18.284 INFO:journalctl@ceph.mon.a.smithi082.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-09-09T23:05:18.284 INFO:journalctl@ceph.mon.a.smithi082.stdout: AddFile(Keys): cumulative 0, interval 0 2023-09-09T23:05:18.284 INFO:journalctl@ceph.mon.a.smithi082.stdout: Cumulative compaction: 0.00 GB write, 28.13 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-09T23:05:18.284 INFO:journalctl@ceph.mon.a.smithi082.stdout: Interval compaction: 0.00 GB write, 28.13 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-09T23:05:18.285 INFO:journalctl@ceph.mon.a.smithi082.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-09-09T23:05:18.285 INFO:journalctl@ceph.mon.a.smithi082.stdout: Block cache BinnedLRUCache@0x562d8f3b51f0#7 capacity: 512.00 MB usage: 1.30 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-09-09T23:05:18.285 INFO:journalctl@ceph.mon.a.smithi082.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-09-09T23:05:18.285 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2023-09-09T23:05:18.285 INFO:journalctl@ceph.mon.a.smithi082.stdout: ** File Read Latency Histogram By Level [default] ** 2023-09-09T23:05:18.286 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: starting mon.a rank 0 at public addrs [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] at bind addrs [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da 2023-09-09T23:05:18.286 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: mon.a@-1(???) e1 preinit fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da 2023-09-09T23:05:18.286 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: mon.a@-1(???).mds e1 new map 2023-09-09T23:05:18.286 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: mon.a@-1(???).mds e1 print_map 2023-09-09T23:05:18.286 INFO:journalctl@ceph.mon.a.smithi082.stdout: e1 2023-09-09T23:05:18.287 INFO:journalctl@ceph.mon.a.smithi082.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-09-09T23:05:18.287 INFO:journalctl@ceph.mon.a.smithi082.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-09T23:05:18.287 INFO:journalctl@ceph.mon.a.smithi082.stdout: legacy client fscid: -1 2023-09-09T23:05:18.287 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2023-09-09T23:05:18.287 INFO:journalctl@ceph.mon.a.smithi082.stdout: No filesystems configured 2023-09-09T23:05:18.288 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-09-09T23:05:18.288 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-09-09T23:05:18.288 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-09-09T23:05:18.288 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-09-09T23:05:18.288 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-09-09T23:05:18.288 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 systemd[1]: Started Ceph mon.a for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da. 2023-09-09T23:05:18.289 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: expand_channel_meta expand map: {default=false} 2023-09-09T23:05:18.289 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: expand_channel_meta from 'false' to 'false' 2023-09-09T23:05:18.289 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: expand_channel_meta expanded map: {default=false} 2023-09-09T23:05:18.289 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: expand_channel_meta expand map: {default=info} 2023-09-09T23:05:18.289 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: expand_channel_meta from 'info' to 'info' 2023-09-09T23:05:18.290 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: expand_channel_meta expanded map: {default=info} 2023-09-09T23:05:18.290 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: expand_channel_meta expand map: {default=daemon} 2023-09-09T23:05:18.290 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: expand_channel_meta from 'daemon' to 'daemon' 2023-09-09T23:05:18.290 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: expand_channel_meta expanded map: {default=daemon} 2023-09-09T23:05:18.290 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: expand_channel_meta expand map: {default=debug} 2023-09-09T23:05:18.291 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: expand_channel_meta from 'debug' to 'debug' 2023-09-09T23:05:18.291 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: expand_channel_meta expanded map: {default=debug} 2023-09-09T23:05:18.291 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: mon.a is new leader, mons a in quorum (ranks 0) 2023-09-09T23:05:18.291 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: monmap e1: 1 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0]} removed_ranks: {} 2023-09-09T23:05:18.291 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: fsmap 2023-09-09T23:05:18.292 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: osdmap e1: 0 total, 0 up, 0 in 2023-09-09T23:05:18.292 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:18 smithi082 ceph-mon[98694]: mgrmap e1: no daemons active 2023-09-09T23:05:19.106 INFO:teuthology.orchestra.run.smithi082.stdout:Wrote config to /etc/ceph/ceph.conf 2023-09-09T23:05:19.134 INFO:teuthology.orchestra.run.smithi082.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-09-09T23:05:19.134 INFO:teuthology.orchestra.run.smithi082.stdout:Creating mgr... 2023-09-09T23:05:19.134 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying port 0.0.0.0:9283 ... 2023-09-09T23:05:19.135 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying port 0.0.0.0:8765 ... 2023-09-09T23:05:19.329 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@mgr.x 2023-09-09T23:05:19.329 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@mgr.x.service: Unit ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@mgr.x.service not loaded. 2023-09-09T23:05:19.416 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da.target.wants/ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@mgr.x.service → /etc/systemd/system/ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@.service. 2023-09-09T23:05:19.703 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:19 smithi082 systemd[1]: Starting Ceph mgr.x for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da... 2023-09-09T23:05:19.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:19 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/4180158955' entity='client.admin' 2023-09-09T23:05:19.956 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:19 smithi082 podman[98922]: 2023-09-09T23:05:19.956 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:19 smithi082 bash[98922]: aac272d167136217742a79fc93c4f55132ca197e5ab45ea8461b3cf28e2197b1 2023-09-09T23:05:20.018 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-09T23:05:20.018 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout disabled 2023-09-09T23:05:20.025 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-09T23:05:20.026 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2023-09-09T23:05:20.026 INFO:teuthology.orchestra.run.smithi082.stdout:firewalld.service is not enabled 2023-09-09T23:05:20.026 INFO:teuthology.orchestra.run.smithi082.stdout:Not possible to enable service . firewalld.service is not available 2023-09-09T23:05:20.034 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-09T23:05:20.034 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout disabled 2023-09-09T23:05:20.042 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-09T23:05:20.042 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2023-09-09T23:05:20.043 INFO:teuthology.orchestra.run.smithi082.stdout:firewalld.service is not enabled 2023-09-09T23:05:20.043 INFO:teuthology.orchestra.run.smithi082.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-09-09T23:05:20.043 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mgr to start... 2023-09-09T23:05:20.043 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mgr... 2023-09-09T23:05:20.290 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:20 smithi082 systemd[1]: Started Ceph mgr.x for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da. 2023-09-09T23:05:20.290 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:20 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:20.064+0000 7f6f94ba0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-09T23:05:20.291 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:20 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:20.148+0000 7f6f94ba0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-09T23:05:20.291 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:20 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:20.239+0000 7f6f94ba0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-09T23:05:20.692 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:20 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:20.340+0000 7f6f94ba0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-09T23:05:20.692 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:20 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:20.518+0000 7f6f94ba0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-09T23:05:20.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:20 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1959050492' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-09T23:05:20.955 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:20 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:20.904+0000 7f6f94ba0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-09T23:05:21.092 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2023-09-09T23:05:21.093 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2023-09-09T23:05:21.093 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsid": "3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da", 2023-09-09T23:05:21.093 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "health": { 2023-09-09T23:05:21.093 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-09T23:05:21.094 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-09T23:05:21.094 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-09T23:05:21.094 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-09-09T23:05:21.094 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-09T23:05:21.094 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-09T23:05:21.095 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 0 2023-09-09T23:05:21.095 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2023-09-09T23:05:21.095 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-09T23:05:21.095 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "a" 2023-09-09T23:05:21.095 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2023-09-09T23:05:21.096 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2023-09-09T23:05:21.096 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-09T23:05:21.096 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-09T23:05:21.096 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-09T23:05:21.096 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-09T23:05:21.096 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-09-09T23:05:21.097 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-09T23:05:21.097 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-09T23:05:21.097 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-09T23:05:21.100 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-09T23:05:21.101 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-09T23:05:21.101 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-09T23:05:21.101 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-09T23:05:21.101 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-09T23:05:21.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-09-09T23:05:21.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-09T23:05:21.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-09T23:05:21.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-09T23:05:21.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-09T23:05:21.103 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-09T23:05:21.103 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-09T23:05:21.103 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-09T23:05:21.103 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-09T23:05:21.103 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-09T23:05:21.103 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-09-09T23:05:21.104 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-09T23:05:21.104 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-09T23:05:21.104 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-09T23:05:21.104 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-09T23:05:21.104 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-09-09T23:05:21.105 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-09T23:05:21.105 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": false, 2023-09-09T23:05:21.105 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-09T23:05:21.105 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-09T23:05:21.105 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-09T23:05:21.106 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-09T23:05:21.106 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "restful" 2023-09-09T23:05:21.106 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2023-09-09T23:05:21.106 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-09T23:05:21.106 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-09-09T23:05:21.107 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-09T23:05:21.107 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-09T23:05:21.107 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modified": "2023-09-09T23:05:14.222684+0000", 2023-09-09T23:05:21.107 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-09T23:05:21.107 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-09-09T23:05:21.107 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-09T23:05:21.108 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2023-09-09T23:05:21.108 INFO:teuthology.orchestra.run.smithi082.stdout:mgr not available, waiting (1/15)... 2023-09-09T23:05:21.455 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:21 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:21.035+0000 7f6f94ba0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-09T23:05:21.455 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:21 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:21.129+0000 7f6f94ba0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-09T23:05:21.455 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:21 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:21.223+0000 7f6f94ba0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-09T23:05:21.955 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:21 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:21.640+0000 7f6f94ba0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-09T23:05:22.705 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:22 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:22.303+0000 7f6f94ba0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-09T23:05:22.705 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:22 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:22.384+0000 7f6f94ba0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-09T23:05:23.427 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:23 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:23.152+0000 7f6f94ba0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-09T23:05:23.428 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:23 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:23.244+0000 7f6f94ba0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-09T23:05:23.695 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:23 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:23.428+0000 7f6f94ba0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-09T23:05:23.955 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:23 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:23.723+0000 7f6f94ba0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-09T23:05:24.037 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2023-09-09T23:05:24.040 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2023-09-09T23:05:24.040 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsid": "3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da", 2023-09-09T23:05:24.040 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "health": { 2023-09-09T23:05:24.040 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-09T23:05:24.040 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-09T23:05:24.041 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-09T23:05:24.041 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-09-09T23:05:24.041 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-09T23:05:24.041 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-09T23:05:24.041 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 0 2023-09-09T23:05:24.042 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2023-09-09T23:05:24.042 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-09T23:05:24.042 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "a" 2023-09-09T23:05:24.042 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2023-09-09T23:05:24.043 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2023-09-09T23:05:24.043 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-09T23:05:24.043 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-09T23:05:24.043 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-09T23:05:24.043 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-09T23:05:24.044 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-09-09T23:05:24.044 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-09T23:05:24.044 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-09T23:05:24.044 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-09T23:05:24.044 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-09T23:05:24.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-09T23:05:24.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-09T23:05:24.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-09T23:05:24.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-09T23:05:24.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-09-09T23:05:24.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-09T23:05:24.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-09T23:05:24.050 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-09T23:05:24.050 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-09T23:05:24.050 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-09T23:05:24.050 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-09T23:05:24.050 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-09T23:05:24.051 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-09T23:05:24.051 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-09T23:05:24.051 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-09-09T23:05:24.051 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-09T23:05:24.051 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-09T23:05:24.052 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-09T23:05:24.052 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-09T23:05:24.052 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-09-09T23:05:24.052 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-09T23:05:24.052 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": false, 2023-09-09T23:05:24.053 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-09T23:05:24.053 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-09T23:05:24.053 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-09T23:05:24.053 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-09T23:05:24.053 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "restful" 2023-09-09T23:05:24.053 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2023-09-09T23:05:24.054 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-09T23:05:24.054 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-09-09T23:05:24.054 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-09T23:05:24.054 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-09T23:05:24.054 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modified": "2023-09-09T23:05:14.222684+0000", 2023-09-09T23:05:24.055 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-09T23:05:24.055 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-09-09T23:05:24.055 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-09T23:05:24.055 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2023-09-09T23:05:24.055 INFO:teuthology.orchestra.run.smithi082.stdout:mgr not available, waiting (2/15)... 2023-09-09T23:05:24.370 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:24 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1935664606' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-09T23:05:24.370 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:23 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:23.956+0000 7f6f94ba0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-09T23:05:24.371 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:24 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:24.109+0000 7f6f94ba0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-09T23:05:25.387 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:24 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:24.371+0000 7f6f94ba0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-09T23:05:25.387 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:25 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:25.065+0000 7f6f94ba0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-09T23:05:25.387 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:25 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:25.197+0000 7f6f94ba0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-09T23:05:25.705 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:25 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:25.388+0000 7f6f94ba0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-09T23:05:25.705 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:25 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:25.500+0000 7f6f94ba0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-09T23:05:26.205 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:25 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:25.714+0000 7f6f94ba0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-09T23:05:26.955 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:26 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:26.612+0000 7f6f94ba0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-09T23:05:26.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:26 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/716690624' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-09T23:05:26.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:26 smithi082 ceph-mon[98694]: Activating manager daemon x 2023-09-09T23:05:26.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:26 smithi082 ceph-mon[98694]: mgrmap e2: x(active, starting, since 0.00298804s) 2023-09-09T23:05:26.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:26 smithi082 ceph-mon[98694]: from='mgr.14100 172.21.15.82:0/1414338373' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-09T23:05:26.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:26 smithi082 ceph-mon[98694]: from='mgr.14100 172.21.15.82:0/1414338373' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-09T23:05:26.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:26 smithi082 ceph-mon[98694]: from='mgr.14100 172.21.15.82:0/1414338373' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-09T23:05:26.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:26 smithi082 ceph-mon[98694]: from='mgr.14100 172.21.15.82:0/1414338373' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-09T23:05:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:26 smithi082 ceph-mon[98694]: from='mgr.14100 172.21.15.82:0/1414338373' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-09T23:05:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:26 smithi082 ceph-mon[98694]: Manager daemon x is now available 2023-09-09T23:05:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:26 smithi082 ceph-mon[98694]: from='mgr.14100 172.21.15.82:0/1414338373' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-09T23:05:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:26 smithi082 ceph-mon[98694]: from='mgr.14100 172.21.15.82:0/1414338373' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-09T23:05:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:26 smithi082 ceph-mon[98694]: from='mgr.14100 172.21.15.82:0/1414338373' entity='mgr.x' 2023-09-09T23:05:26.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:26 smithi082 ceph-mon[98694]: from='mgr.14100 172.21.15.82:0/1414338373' entity='mgr.x' 2023-09-09T23:05:26.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:26 smithi082 ceph-mon[98694]: from='mgr.14100 172.21.15.82:0/1414338373' entity='mgr.x' 2023-09-09T23:05:26.987 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2023-09-09T23:05:26.988 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2023-09-09T23:05:26.988 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsid": "3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da", 2023-09-09T23:05:26.988 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "health": { 2023-09-09T23:05:26.988 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-09T23:05:26.989 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-09T23:05:26.989 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-09T23:05:26.989 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-09-09T23:05:26.989 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-09T23:05:26.989 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-09T23:05:26.990 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 0 2023-09-09T23:05:26.990 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2023-09-09T23:05:26.990 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-09T23:05:26.990 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "a" 2023-09-09T23:05:26.990 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2023-09-09T23:05:26.991 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2023-09-09T23:05:26.991 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-09T23:05:26.991 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-09T23:05:26.991 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-09T23:05:26.991 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-09T23:05:26.991 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-09-09T23:05:26.992 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-09T23:05:26.992 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-09T23:05:26.992 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-09T23:05:26.995 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-09T23:05:26.996 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-09T23:05:26.996 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-09T23:05:26.996 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-09T23:05:26.996 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-09T23:05:26.996 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-09-09T23:05:26.997 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-09T23:05:26.997 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-09T23:05:26.997 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-09T23:05:26.997 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-09T23:05:26.997 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-09T23:05:26.998 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-09T23:05:26.998 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-09T23:05:26.998 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-09T23:05:26.998 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-09T23:05:26.998 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-09-09T23:05:26.999 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-09T23:05:26.999 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-09T23:05:26.999 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-09T23:05:26.999 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-09T23:05:26.999 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-09-09T23:05:26.999 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-09T23:05:27.000 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": false, 2023-09-09T23:05:27.000 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-09T23:05:27.000 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-09T23:05:27.000 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-09T23:05:27.000 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-09T23:05:27.001 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "restful" 2023-09-09T23:05:27.001 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2023-09-09T23:05:27.001 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-09T23:05:27.001 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-09-09T23:05:27.001 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-09T23:05:27.001 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-09T23:05:27.002 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modified": "2023-09-09T23:05:14.222684+0000", 2023-09-09T23:05:27.002 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-09T23:05:27.002 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-09-09T23:05:27.002 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-09T23:05:27.002 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2023-09-09T23:05:27.003 INFO:teuthology.orchestra.run.smithi082.stdout:mgr not available, waiting (3/15)... 2023-09-09T23:05:28.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:28 smithi082 ceph-mon[98694]: mgrmap e3: x(active, since 1.0072s) 2023-09-09T23:05:30.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:30 smithi082 ceph-mon[98694]: mgrmap e4: x(active, since 3s) 2023-09-09T23:05:31.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:31 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3313661977' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-09T23:05:33.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2023-09-09T23:05:33.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2023-09-09T23:05:33.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsid": "3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da", 2023-09-09T23:05:33.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "health": { 2023-09-09T23:05:33.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-09T23:05:33.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-09T23:05:33.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-09T23:05:33.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-09-09T23:05:33.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-09T23:05:33.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-09T23:05:33.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 0 2023-09-09T23:05:33.049 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2023-09-09T23:05:33.050 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-09T23:05:33.050 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "a" 2023-09-09T23:05:33.050 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2023-09-09T23:05:33.050 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2023-09-09T23:05:33.050 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-09T23:05:33.051 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-09T23:05:33.051 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-09T23:05:33.051 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-09T23:05:33.051 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-09-09T23:05:33.051 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-09T23:05:33.051 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-09T23:05:33.052 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-09T23:05:33.052 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-09T23:05:33.052 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-09T23:05:33.052 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-09T23:05:33.055 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-09T23:05:33.056 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-09T23:05:33.056 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-09-09T23:05:33.056 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-09T23:05:33.056 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-09T23:05:33.056 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-09T23:05:33.057 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-09T23:05:33.057 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-09T23:05:33.057 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-09T23:05:33.057 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-09T23:05:33.057 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-09T23:05:33.058 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-09T23:05:33.058 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-09-09T23:05:33.058 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-09T23:05:33.058 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-09T23:05:33.058 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-09T23:05:33.059 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-09T23:05:33.059 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-09-09T23:05:33.059 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-09T23:05:33.059 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": true, 2023-09-09T23:05:33.059 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-09T23:05:33.059 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-09T23:05:33.060 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-09T23:05:33.060 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-09T23:05:33.060 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "restful" 2023-09-09T23:05:33.060 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2023-09-09T23:05:33.060 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-09T23:05:33.061 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-09-09T23:05:33.061 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-09T23:05:33.061 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-09T23:05:33.061 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modified": "2023-09-09T23:05:14.222684+0000", 2023-09-09T23:05:33.061 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-09T23:05:33.061 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-09-09T23:05:33.062 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-09T23:05:33.062 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2023-09-09T23:05:33.062 INFO:teuthology.orchestra.run.smithi082.stdout:mgr is available 2023-09-09T23:05:36.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:35 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2559520138' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-09-09T23:05:37.908 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2023-09-09T23:05:37.908 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [global] 2023-09-09T23:05:37.908 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout fsid = 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da 2023-09-09T23:05:37.909 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-09-09T23:05:37.909 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-09-09T23:05:37.909 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-09-09T23:05:37.909 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-09-09T23:05:37.909 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2023-09-09T23:05:37.910 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [mgr] 2023-09-09T23:05:37.910 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-09-09T23:05:37.910 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2023-09-09T23:05:37.910 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [osd] 2023-09-09T23:05:37.910 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-09-09T23:05:37.911 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-09-09T23:05:37.911 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-09-09T23:05:37.911 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-09-09T23:05:37.911 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-09-09T23:05:37.911 INFO:teuthology.orchestra.run.smithi082.stdout:Enabling cephadm module... 2023-09-09T23:05:40.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:39 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/4086939398' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-09-09T23:05:40.205 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:39 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: ignoring --setuser ceph since I am not root 2023-09-09T23:05:40.206 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:39 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: ignoring --setgroup ceph since I am not root 2023-09-09T23:05:40.206 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:40 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:40.098+0000 7f0e74740200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-09T23:05:40.206 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:40 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:40.179+0000 7f0e74740200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-09T23:05:40.534 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:40 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:40.270+0000 7f0e74740200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-09T23:05:40.534 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:40 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:40.363+0000 7f0e74740200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-09T23:05:40.920 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:40 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:40.543+0000 7f0e74740200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-09T23:05:40.921 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:40 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:40.921+0000 7f0e74740200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-09T23:05:41.202 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:40 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/4086939398' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-09-09T23:05:41.203 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:40 smithi082 ceph-mon[98694]: mgrmap e5: x(active, since 13s) 2023-09-09T23:05:41.203 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:41 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:41.053+0000 7f0e74740200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-09T23:05:41.204 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:41 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:41.149+0000 7f0e74740200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-09T23:05:41.396 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2023-09-09T23:05:41.397 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-09-09T23:05:41.398 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": true, 2023-09-09T23:05:41.398 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-09-09T23:05:41.398 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-09-09T23:05:41.398 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2023-09-09T23:05:41.398 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for the mgr to restart... 2023-09-09T23:05:41.399 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mgr epoch 5... 2023-09-09T23:05:41.455 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:41 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:41.248+0000 7f0e74740200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-09T23:05:41.936 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:41 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:41.685+0000 7f0e74740200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-09T23:05:42.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:41 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3653650098' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-09-09T23:05:42.705 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:42 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:42.346+0000 7f0e74740200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-09T23:05:42.705 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:42 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:42.428+0000 7f0e74740200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-09T23:05:43.448 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:43 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:43.188+0000 7f0e74740200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-09T23:05:43.449 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:43 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:43.278+0000 7f0e74740200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-09T23:05:43.705 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:43 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:43.449+0000 7f0e74740200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-09T23:05:44.095 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:43 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:43.725+0000 7f0e74740200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-09T23:05:44.095 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:43 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:43.948+0000 7f0e74740200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-09T23:05:44.705 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:44 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:44.097+0000 7f0e74740200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-09T23:05:44.705 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:44 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:44.357+0000 7f0e74740200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-09T23:05:45.363 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:45 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:45.046+0000 7f0e74740200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-09T23:05:45.363 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:45 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:45.175+0000 7f0e74740200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-09T23:05:45.363 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:45 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:45.365+0000 7f0e74740200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-09T23:05:45.688 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:45 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:45.477+0000 7f0e74740200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-09T23:05:45.955 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:45 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:45.690+0000 7f0e74740200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-09T23:05:46.955 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:46 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:05:46.537+0000 7f0e74740200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-09T23:05:46.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:46 smithi082 ceph-mon[98694]: Active manager daemon x restarted 2023-09-09T23:05:46.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:46 smithi082 ceph-mon[98694]: Activating manager daemon x 2023-09-09T23:05:46.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:46 smithi082 ceph-mon[98694]: osdmap e2: 0 total, 0 up, 0 in 2023-09-09T23:05:46.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:46 smithi082 ceph-mon[98694]: mgrmap e6: x(active, starting, since 0.00328761s) 2023-09-09T23:05:46.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:46 smithi082 ceph-mon[98694]: from='mgr.14120 172.21.15.82:0/688015679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-09T23:05:46.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:46 smithi082 ceph-mon[98694]: from='mgr.14120 172.21.15.82:0/688015679' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-09T23:05:46.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:46 smithi082 ceph-mon[98694]: from='mgr.14120 172.21.15.82:0/688015679' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-09T23:05:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:46 smithi082 ceph-mon[98694]: from='mgr.14120 172.21.15.82:0/688015679' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-09T23:05:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:46 smithi082 ceph-mon[98694]: from='mgr.14120 172.21.15.82:0/688015679' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-09T23:05:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:46 smithi082 ceph-mon[98694]: Manager daemon x is now available 2023-09-09T23:05:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:46 smithi082 ceph-mon[98694]: from='mgr.14120 172.21.15.82:0/688015679' entity='mgr.x' 2023-09-09T23:05:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:46 smithi082 ceph-mon[98694]: from='mgr.14120 172.21.15.82:0/688015679' entity='mgr.x' 2023-09-09T23:05:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:46 smithi082 ceph-mon[98694]: from='mgr.14120 172.21.15.82:0/688015679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:05:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:46 smithi082 ceph-mon[98694]: from='mgr.14120 172.21.15.82:0/688015679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:05:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:46 smithi082 ceph-mon[98694]: from='mgr.14120 172.21.15.82:0/688015679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:05:47.906 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:47 smithi082 ceph-mon[98694]: Found migration_current of "None". Setting to last migration. 2023-09-09T23:05:47.906 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:47 smithi082 ceph-mon[98694]: from='mgr.14120 172.21.15.82:0/688015679' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-09T23:05:47.906 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:47 smithi082 ceph-mon[98694]: from='mgr.14120 172.21.15.82:0/688015679' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-09T23:05:47.907 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:47 smithi082 ceph-mon[98694]: from='mgr.14120 172.21.15.82:0/688015679' entity='mgr.x' 2023-09-09T23:05:47.907 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:47 smithi082 ceph-mon[98694]: from='mgr.14120 172.21.15.82:0/688015679' entity='mgr.x' 2023-09-09T23:05:47.907 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:47 smithi082 ceph-mon[98694]: mgrmap e7: x(active, since 1.00459s) 2023-09-09T23:05:47.951 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2023-09-09T23:05:47.952 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-09-09T23:05:47.952 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "initialized": true 2023-09-09T23:05:47.952 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2023-09-09T23:05:47.952 INFO:teuthology.orchestra.run.smithi082.stdout:mgr epoch 5 is available 2023-09-09T23:05:47.952 INFO:teuthology.orchestra.run.smithi082.stdout:Setting orchestrator backend to cephadm... 2023-09-09T23:05:48.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:48 smithi082 ceph-mon[98694]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-09-09T23:05:48.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:48 smithi082 ceph-mon[98694]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-09-09T23:05:48.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:48 smithi082 ceph-mon[98694]: [09/Sep/2023:23:05:47] ENGINE Bus STARTING 2023-09-09T23:05:48.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:48 smithi082 ceph-mon[98694]: [09/Sep/2023:23:05:47] ENGINE Serving on https://172.21.15.82:7150 2023-09-09T23:05:48.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:48 smithi082 ceph-mon[98694]: [09/Sep/2023:23:05:47] ENGINE Serving on http://172.21.15.82:8765 2023-09-09T23:05:48.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:48 smithi082 ceph-mon[98694]: [09/Sep/2023:23:05:47] ENGINE Bus STARTED 2023-09-09T23:05:48.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:48 smithi082 ceph-mon[98694]: from='mgr.14120 172.21.15.82:0/688015679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:05:49.919 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:49 smithi082 ceph-mon[98694]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:05:49.920 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:49 smithi082 ceph-mon[98694]: from='mgr.14120 172.21.15.82:0/688015679' entity='mgr.x' 2023-09-09T23:05:49.920 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:49 smithi082 ceph-mon[98694]: from='mgr.14120 172.21.15.82:0/688015679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:05:49.920 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:49 smithi082 ceph-mon[98694]: mgrmap e8: x(active, since 2s) 2023-09-09T23:05:49.931 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout value unchanged 2023-09-09T23:05:49.931 INFO:teuthology.orchestra.run.smithi082.stdout:Generating ssh key... 2023-09-09T23:05:50.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:50 smithi082 ceph-mon[98694]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:05:51.575 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:51 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: Generating public/private rsa key pair. 2023-09-09T23:05:51.576 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:51 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: Your identification has been saved in /tmp/tmpcw0lg83u/key. 2023-09-09T23:05:51.576 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:51 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: Your public key has been saved in /tmp/tmpcw0lg83u/key.pub. 2023-09-09T23:05:51.576 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:51 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: The key fingerprint is: 2023-09-09T23:05:51.576 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:51 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: SHA256:HpW7LPVEgdYCdO6HK5oCwdZ3oVkP32DUEi3+tp+Xvqk ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da 2023-09-09T23:05:51.577 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:51 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: The key's randomart image is: 2023-09-09T23:05:51.577 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:51 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: +---[RSA 3072]----+ 2023-09-09T23:05:51.577 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:51 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: | .oo+*. | 2023-09-09T23:05:51.577 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:51 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: | +oO.+. | 2023-09-09T23:05:51.577 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:51 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: | . . + O+*. | 2023-09-09T23:05:51.577 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:51 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: | + . + .o++. | 2023-09-09T23:05:51.578 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:51 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: | . . . .S =.o | 2023-09-09T23:05:51.578 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:51 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: | . . + *o | 2023-09-09T23:05:51.578 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:51 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: | . + +... .| 2023-09-09T23:05:51.578 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:51 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: | . o o . oo| 2023-09-09T23:05:51.578 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:51 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: | .o E==.| 2023-09-09T23:05:51.579 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:05:51 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: +----[SHA256]-----+ 2023-09-09T23:05:51.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:51 smithi082 ceph-mon[98694]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:05:51.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:51 smithi082 ceph-mon[98694]: Generating ssh key... 2023-09-09T23:05:51.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:51 smithi082 ceph-mon[98694]: from='mgr.14120 172.21.15.82:0/688015679' entity='mgr.x' 2023-09-09T23:05:51.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:51 smithi082 ceph-mon[98694]: from='mgr.14120 172.21.15.82:0/688015679' entity='mgr.x' 2023-09-09T23:05:52.611 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5deT51qqjCDVYYSH5A0a2e2VZVZF4MVOTAj0P90zry43sNlXz6LvqZtTpVgTG+wHyrlYxMvDo/Z47kr0YJwpgOZM5I/ZBICuldwMzaFSDhHTKODROdh/ThJYhL21iu+mxPvEjHKZxQwIYJE99UJNUUoYsE2CKiSHJRfQUBMSeON4vP9hn3GtPIRVCvbHD8Muw+jh5Zjw8dNWUtT6lcMgEATFqu6uvylE7r10GNSRb1uQ0W1zEIC/d/cBPNS4+m9YBz5P+Vm8QvBsF0UpSuVdkELQ34AHDKpMmU10Yk1FVDBrbRWzvy8+9kYpNFTtkpQQgE4r2aDo2p197UAWk7dZODbr5Nb4yDqonbjFlMWQDTPIJdYSYk403E2uJS1kLOK/zrdgrza6Du//iBF/dUoT6axdhDhf9tSTU8dDnOOAy2FI0lsGobbdxOVF4XGpRbVrhCrFFWsFcDJrC9ALzqqvlq+LuA6BiFvTbTe8qYEfLCmLpXlVAl7C/7pDuPjf38es= ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da 2023-09-09T23:05:52.611 INFO:teuthology.orchestra.run.smithi082.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-09-09T23:05:52.612 INFO:teuthology.orchestra.run.smithi082.stdout:Adding key to root@localhost authorized_keys... 2023-09-09T23:05:52.613 INFO:teuthology.orchestra.run.smithi082.stdout:Adding host smithi082... 2023-09-09T23:05:53.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:53 smithi082 ceph-mon[98694]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:05:54.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:54 smithi082 ceph-mon[98694]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi082", "addr": "172.21.15.82", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:05:54.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:54 smithi082 ceph-mon[98694]: Deploying cephadm binary to smithi082 2023-09-09T23:05:55.964 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout Added host 'smithi082' with addr '172.21.15.82' 2023-09-09T23:05:55.965 INFO:teuthology.orchestra.run.smithi082.stdout:Deploying unmanaged mon service... 2023-09-09T23:05:56.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:56 smithi082 ceph-mon[98694]: from='mgr.14120 172.21.15.82:0/688015679' entity='mgr.x' 2023-09-09T23:05:56.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:56 smithi082 ceph-mon[98694]: Added host smithi082 2023-09-09T23:05:56.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:56 smithi082 ceph-mon[98694]: from='mgr.14120 172.21.15.82:0/688015679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:05:57.308 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-09-09T23:05:57.308 INFO:teuthology.orchestra.run.smithi082.stdout:Deploying unmanaged mgr service... 2023-09-09T23:05:57.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:57 smithi082 ceph-mon[98694]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:05:57.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:57 smithi082 ceph-mon[98694]: Saving service mon spec with placement count:5 2023-09-09T23:05:57.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:57 smithi082 ceph-mon[98694]: from='mgr.14120 172.21.15.82:0/688015679' entity='mgr.x' 2023-09-09T23:05:57.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:57 smithi082 ceph-mon[98694]: from='mgr.14120 172.21.15.82:0/688015679' entity='mgr.x' 2023-09-09T23:05:58.448 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-09-09T23:05:59.203 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:58 smithi082 ceph-mon[98694]: from='mgr.14120 172.21.15.82:0/688015679' entity='mgr.x' 2023-09-09T23:05:59.203 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:58 smithi082 ceph-mon[98694]: from='mgr.14120 172.21.15.82:0/688015679' entity='mgr.x' 2023-09-09T23:06:00.123 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:59 smithi082 ceph-mon[98694]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:06:00.124 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:59 smithi082 ceph-mon[98694]: Saving service mgr spec with placement count:2 2023-09-09T23:06:00.124 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:59 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/4135678147' entity='client.admin' 2023-09-09T23:06:00.124 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:05:59 smithi082 ceph-mon[98694]: from='mgr.14120 172.21.15.82:0/688015679' entity='mgr.x' 2023-09-09T23:06:00.582 INFO:teuthology.orchestra.run.smithi082.stdout:Enabling the dashboard module... 2023-09-09T23:06:01.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:01 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/318005691' entity='client.admin' 2023-09-09T23:06:01.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:01 smithi082 ceph-mon[98694]: from='mgr.14120 172.21.15.82:0/688015679' entity='mgr.x' 2023-09-09T23:06:02.448 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:02 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3197151207' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-09-09T23:06:02.449 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:06:02 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: ignoring --setuser ceph since I am not root 2023-09-09T23:06:02.449 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:06:02 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: ignoring --setgroup ceph since I am not root 2023-09-09T23:06:02.449 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:06:02 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:06:02.370+0000 7f41dcf4c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-09T23:06:02.449 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:06:02 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:06:02.450+0000 7f41dcf4c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-09T23:06:02.705 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:06:02 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:06:02.543+0000 7f41dcf4c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-09T23:06:02.705 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:06:02 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:06:02.642+0000 7f41dcf4c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-09T23:06:03.196 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:06:02 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:06:02.823+0000 7f41dcf4c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-09T23:06:03.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:03 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3197151207' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-09-09T23:06:03.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:03 smithi082 ceph-mon[98694]: mgrmap e9: x(active, since 15s) 2023-09-09T23:06:03.456 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:06:03 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:06:03.196+0000 7f41dcf4c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-09T23:06:03.456 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:06:03 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:06:03.330+0000 7f41dcf4c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-09T23:06:03.456 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:06:03 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:06:03.429+0000 7f41dcf4c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-09T23:06:03.670 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2023-09-09T23:06:03.671 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-09-09T23:06:03.671 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": true, 2023-09-09T23:06:03.671 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-09-09T23:06:03.671 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-09-09T23:06:03.672 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2023-09-09T23:06:03.672 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for the mgr to restart... 2023-09-09T23:06:03.672 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mgr epoch 9... 2023-09-09T23:06:03.776 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:06:03 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:06:03.526+0000 7f41dcf4c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-09T23:06:04.196 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:06:03 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:06:03.971+0000 7f41dcf4c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-09T23:06:04.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:04 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1679180999' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-09-09T23:06:04.955 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:06:04 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:06:04.633+0000 7f41dcf4c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-09T23:06:04.955 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:06:04 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:06:04.714+0000 7f41dcf4c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-09T23:06:05.742 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:06:05 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:06:05.480+0000 7f41dcf4c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-09T23:06:05.743 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:06:05 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:06:05.570+0000 7f41dcf4c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-09T23:06:06.024 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:06:05 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:06:05.744+0000 7f41dcf4c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-09T23:06:06.408 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:06:06 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:06:06.025+0000 7f41dcf4c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-09T23:06:06.408 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:06:06 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:06:06.261+0000 7f41dcf4c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-09T23:06:06.670 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:06:06 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:06:06.409+0000 7f41dcf4c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-09T23:06:06.955 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:06:06 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:06:06.671+0000 7f41dcf4c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-09T23:06:07.697 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:06:07 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:06:07.370+0000 7f41dcf4c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-09T23:06:07.697 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:06:07 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:06:07.502+0000 7f41dcf4c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-09T23:06:07.955 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:06:07 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:06:07.696+0000 7f41dcf4c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-09T23:06:07.955 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:06:07 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:06:07.809+0000 7f41dcf4c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-09T23:06:08.455 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:06:08 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:06:08.029+0000 7f41dcf4c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-09T23:06:09.167 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:06:08 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:06:08.888+0000 7f41dcf4c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-09T23:06:09.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:09 smithi082 ceph-mon[98694]: Active manager daemon x restarted 2023-09-09T23:06:09.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:09 smithi082 ceph-mon[98694]: Activating manager daemon x 2023-09-09T23:06:09.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:09 smithi082 ceph-mon[98694]: osdmap e3: 0 total, 0 up, 0 in 2023-09-09T23:06:09.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:09 smithi082 ceph-mon[98694]: mgrmap e10: x(active, starting, since 0.00314049s) 2023-09-09T23:06:09.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-09T23:06:09.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-09T23:06:09.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-09T23:06:09.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-09T23:06:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-09T23:06:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:09 smithi082 ceph-mon[98694]: Manager daemon x is now available 2023-09-09T23:06:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:06:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:06:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-09T23:06:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-09T23:06:10.626 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2023-09-09T23:06:10.626 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-09-09T23:06:10.626 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "initialized": true 2023-09-09T23:06:10.627 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2023-09-09T23:06:10.627 INFO:teuthology.orchestra.run.smithi082.stdout:mgr epoch 9 is available 2023-09-09T23:06:10.627 INFO:teuthology.orchestra.run.smithi082.stdout:Generating a dashboard self-signed certificate... 2023-09-09T23:06:11.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:10 smithi082 ceph-mon[98694]: [09/Sep/2023:23:06:09] ENGINE Bus STARTING 2023-09-09T23:06:11.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:10 smithi082 ceph-mon[98694]: [09/Sep/2023:23:06:09] ENGINE Serving on http://172.21.15.82:8765 2023-09-09T23:06:11.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:10 smithi082 ceph-mon[98694]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-09-09T23:06:11.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:10 smithi082 ceph-mon[98694]: mgrmap e11: x(active, since 1.00539s) 2023-09-09T23:06:11.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:10 smithi082 ceph-mon[98694]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-09-09T23:06:11.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:10 smithi082 ceph-mon[98694]: [09/Sep/2023:23:06:09] ENGINE Serving on https://172.21.15.82:7150 2023-09-09T23:06:11.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:10 smithi082 ceph-mon[98694]: [09/Sep/2023:23:06:09] ENGINE Bus STARTED 2023-09-09T23:06:11.966 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-09-09T23:06:11.967 INFO:teuthology.orchestra.run.smithi082.stdout:Creating initial admin user... 2023-09-09T23:06:12.193 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:11 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:12.194 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:11 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:12.194 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:11 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:12.194 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:11 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:12.194 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:11 smithi082 ceph-mon[98694]: mgrmap e12: x(active, since 3s) 2023-09-09T23:06:13.121 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:12 smithi082 ceph-mon[98694]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:06:13.313 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$WVn6TprJtIQ2z66Kd4JED.0DsemfX7smg6zR/TtLDXN0oloouWP1O", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1694300772, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-09-09T23:06:13.313 INFO:teuthology.orchestra.run.smithi082.stdout:Fetching dashboard port number... 2023-09-09T23:06:14.234 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:13 smithi082 ceph-mon[98694]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:06:14.234 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:13 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:14.555 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 8443 2023-09-09T23:06:14.564 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-09T23:06:14.564 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout disabled 2023-09-09T23:06:14.571 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-09T23:06:14.571 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2023-09-09T23:06:14.572 INFO:teuthology.orchestra.run.smithi082.stdout:firewalld.service is not enabled 2023-09-09T23:06:14.572 INFO:teuthology.orchestra.run.smithi082.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-09-09T23:06:14.573 INFO:teuthology.orchestra.run.smithi082.stdout:Ceph Dashboard is now available at: 2023-09-09T23:06:14.573 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:06:14.573 INFO:teuthology.orchestra.run.smithi082.stdout: URL: https://smithi082:8443/ 2023-09-09T23:06:14.573 INFO:teuthology.orchestra.run.smithi082.stdout: User: admin 2023-09-09T23:06:14.574 INFO:teuthology.orchestra.run.smithi082.stdout: Password: 3azhj0aend 2023-09-09T23:06:14.574 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:06:14.574 INFO:teuthology.orchestra.run.smithi082.stdout:Saving cluster configuration to /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config directory 2023-09-09T23:06:14.575 INFO:teuthology.orchestra.run.smithi082.stdout:Enabling autotune for osd_memory_target 2023-09-09T23:06:15.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:14 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2741461275' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-09-09T23:06:16.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:15 smithi082 ceph-mon[98694]: mgrmap e13: x(active, since 6s) 2023-09-09T23:06:16.655 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-09-09T23:06:16.655 INFO:teuthology.orchestra.run.smithi082.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-09-09T23:06:16.655 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:06:16.656 INFO:teuthology.orchestra.run.smithi082.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-09-09T23:06:16.656 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:06:16.656 INFO:teuthology.orchestra.run.smithi082.stdout:Or, if you are only running a single cluster on this host: 2023-09-09T23:06:16.656 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:06:16.656 INFO:teuthology.orchestra.run.smithi082.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-09-09T23:06:16.657 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:06:16.657 INFO:teuthology.orchestra.run.smithi082.stdout:Please consider enabling telemetry to help improve Ceph: 2023-09-09T23:06:16.657 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:06:16.657 INFO:teuthology.orchestra.run.smithi082.stdout: ceph telemetry on 2023-09-09T23:06:16.657 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:06:16.657 INFO:teuthology.orchestra.run.smithi082.stdout:For more information see: 2023-09-09T23:06:16.658 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:06:16.658 INFO:teuthology.orchestra.run.smithi082.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-09-09T23:06:16.658 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:06:16.658 INFO:teuthology.orchestra.run.smithi082.stdout:Bootstrap complete. 2023-09-09T23:06:16.690 INFO:tasks.cephadm:Fetching config... 2023-09-09T23:06:16.691 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-09-09T23:06:16.691 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-09-09T23:06:16.708 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-09-09T23:06:16.708 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-09-09T23:06:16.708 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-09-09T23:06:16.766 INFO:tasks.cephadm:Fetching mon keyring... 2023-09-09T23:06:16.767 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-09-09T23:06:16.767 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/keyring of=/dev/stdout 2023-09-09T23:06:16.835 INFO:tasks.cephadm:Fetching pub ssh key... 2023-09-09T23:06:16.836 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-09-09T23:06:16.836 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-09-09T23:06:16.892 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-09-09T23:06:16.893 DEBUG:teuthology.orchestra.run.smithi082:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5deT51qqjCDVYYSH5A0a2e2VZVZF4MVOTAj0P90zry43sNlXz6LvqZtTpVgTG+wHyrlYxMvDo/Z47kr0YJwpgOZM5I/ZBICuldwMzaFSDhHTKODROdh/ThJYhL21iu+mxPvEjHKZxQwIYJE99UJNUUoYsE2CKiSHJRfQUBMSeON4vP9hn3GtPIRVCvbHD8Muw+jh5Zjw8dNWUtT6lcMgEATFqu6uvylE7r10GNSRb1uQ0W1zEIC/d/cBPNS4+m9YBz5P+Vm8QvBsF0UpSuVdkELQ34AHDKpMmU10Yk1FVDBrbRWzvy8+9kYpNFTtkpQQgE4r2aDo2p197UAWk7dZODbr5Nb4yDqonbjFlMWQDTPIJdYSYk403E2uJS1kLOK/zrdgrza6Du//iBF/dUoT6axdhDhf9tSTU8dDnOOAy2FI0lsGobbdxOVF4XGpRbVrhCrFFWsFcDJrC9ALzqqvlq+LuA6BiFvTbTe8qYEfLCmLpXlVAl7C/7pDuPjf38es= ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-09-09T23:06:16.974 INFO:teuthology.orchestra.run.smithi082.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5deT51qqjCDVYYSH5A0a2e2VZVZF4MVOTAj0P90zry43sNlXz6LvqZtTpVgTG+wHyrlYxMvDo/Z47kr0YJwpgOZM5I/ZBICuldwMzaFSDhHTKODROdh/ThJYhL21iu+mxPvEjHKZxQwIYJE99UJNUUoYsE2CKiSHJRfQUBMSeON4vP9hn3GtPIRVCvbHD8Muw+jh5Zjw8dNWUtT6lcMgEATFqu6uvylE7r10GNSRb1uQ0W1zEIC/d/cBPNS4+m9YBz5P+Vm8QvBsF0UpSuVdkELQ34AHDKpMmU10Yk1FVDBrbRWzvy8+9kYpNFTtkpQQgE4r2aDo2p197UAWk7dZODbr5Nb4yDqonbjFlMWQDTPIJdYSYk403E2uJS1kLOK/zrdgrza6Du//iBF/dUoT6axdhDhf9tSTU8dDnOOAy2FI0lsGobbdxOVF4XGpRbVrhCrFFWsFcDJrC9ALzqqvlq+LuA6BiFvTbTe8qYEfLCmLpXlVAl7C/7pDuPjf38es= ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da 2023-09-09T23:06:16.986 DEBUG:teuthology.orchestra.run.smithi098:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5deT51qqjCDVYYSH5A0a2e2VZVZF4MVOTAj0P90zry43sNlXz6LvqZtTpVgTG+wHyrlYxMvDo/Z47kr0YJwpgOZM5I/ZBICuldwMzaFSDhHTKODROdh/ThJYhL21iu+mxPvEjHKZxQwIYJE99UJNUUoYsE2CKiSHJRfQUBMSeON4vP9hn3GtPIRVCvbHD8Muw+jh5Zjw8dNWUtT6lcMgEATFqu6uvylE7r10GNSRb1uQ0W1zEIC/d/cBPNS4+m9YBz5P+Vm8QvBsF0UpSuVdkELQ34AHDKpMmU10Yk1FVDBrbRWzvy8+9kYpNFTtkpQQgE4r2aDo2p197UAWk7dZODbr5Nb4yDqonbjFlMWQDTPIJdYSYk403E2uJS1kLOK/zrdgrza6Du//iBF/dUoT6axdhDhf9tSTU8dDnOOAy2FI0lsGobbdxOVF4XGpRbVrhCrFFWsFcDJrC9ALzqqvlq+LuA6BiFvTbTe8qYEfLCmLpXlVAl7C/7pDuPjf38es= ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-09-09T23:06:17.031 INFO:teuthology.orchestra.run.smithi098.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5deT51qqjCDVYYSH5A0a2e2VZVZF4MVOTAj0P90zry43sNlXz6LvqZtTpVgTG+wHyrlYxMvDo/Z47kr0YJwpgOZM5I/ZBICuldwMzaFSDhHTKODROdh/ThJYhL21iu+mxPvEjHKZxQwIYJE99UJNUUoYsE2CKiSHJRfQUBMSeON4vP9hn3GtPIRVCvbHD8Muw+jh5Zjw8dNWUtT6lcMgEATFqu6uvylE7r10GNSRb1uQ0W1zEIC/d/cBPNS4+m9YBz5P+Vm8QvBsF0UpSuVdkELQ34AHDKpMmU10Yk1FVDBrbRWzvy8+9kYpNFTtkpQQgE4r2aDo2p197UAWk7dZODbr5Nb4yDqonbjFlMWQDTPIJdYSYk403E2uJS1kLOK/zrdgrza6Du//iBF/dUoT6axdhDhf9tSTU8dDnOOAy2FI0lsGobbdxOVF4XGpRbVrhCrFFWsFcDJrC9ALzqqvlq+LuA6BiFvTbTe8qYEfLCmLpXlVAl7C/7pDuPjf38es= ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da 2023-09-09T23:06:17.044 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5deT51qqjCDVYYSH5A0a2e2VZVZF4MVOTAj0P90zry43sNlXz6LvqZtTpVgTG+wHyrlYxMvDo/Z47kr0YJwpgOZM5I/ZBICuldwMzaFSDhHTKODROdh/ThJYhL21iu+mxPvEjHKZxQwIYJE99UJNUUoYsE2CKiSHJRfQUBMSeON4vP9hn3GtPIRVCvbHD8Muw+jh5Zjw8dNWUtT6lcMgEATFqu6uvylE7r10GNSRb1uQ0W1zEIC/d/cBPNS4+m9YBz5P+Vm8QvBsF0UpSuVdkELQ34AHDKpMmU10Yk1FVDBrbRWzvy8+9kYpNFTtkpQQgE4r2aDo2p197UAWk7dZODbr5Nb4yDqonbjFlMWQDTPIJdYSYk403E2uJS1kLOK/zrdgrza6Du//iBF/dUoT6axdhDhf9tSTU8dDnOOAy2FI0lsGobbdxOVF4XGpRbVrhCrFFWsFcDJrC9ALzqqvlq+LuA6BiFvTbTe8qYEfLCmLpXlVAl7C/7pDuPjf38es= ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-09-09T23:06:17.090 INFO:teuthology.orchestra.run.smithi170.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5deT51qqjCDVYYSH5A0a2e2VZVZF4MVOTAj0P90zry43sNlXz6LvqZtTpVgTG+wHyrlYxMvDo/Z47kr0YJwpgOZM5I/ZBICuldwMzaFSDhHTKODROdh/ThJYhL21iu+mxPvEjHKZxQwIYJE99UJNUUoYsE2CKiSHJRfQUBMSeON4vP9hn3GtPIRVCvbHD8Muw+jh5Zjw8dNWUtT6lcMgEATFqu6uvylE7r10GNSRb1uQ0W1zEIC/d/cBPNS4+m9YBz5P+Vm8QvBsF0UpSuVdkELQ34AHDKpMmU10Yk1FVDBrbRWzvy8+9kYpNFTtkpQQgE4r2aDo2p197UAWk7dZODbr5Nb4yDqonbjFlMWQDTPIJdYSYk403E2uJS1kLOK/zrdgrza6Du//iBF/dUoT6axdhDhf9tSTU8dDnOOAy2FI0lsGobbdxOVF4XGpRbVrhCrFFWsFcDJrC9ALzqqvlq+LuA6BiFvTbTe8qYEfLCmLpXlVAl7C/7pDuPjf38es= ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da 2023-09-09T23:06:17.102 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-09-09T23:06:17.658 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:06:17.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:17 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3988769992' entity='client.admin' 2023-09-09T23:06:19.661 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-09-09T23:06:19.662 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-09-09T23:06:20.220 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:06:20.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:20 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2811446740' entity='client.admin' 2023-09-09T23:06:20.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:20 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:20.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:20 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:20.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:20 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:06:20.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:20 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:20.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:20 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:06:20.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:20 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:21.977 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi098 2023-09-09T23:06:21.978 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-09-09T23:06:21.978 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/etc/ceph/ceph.conf 2023-09-09T23:06:21.999 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-09-09T23:06:22.000 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-09T23:06:22.060 INFO:tasks.cephadm:Adding host smithi098 to orchestrator... 2023-09-09T23:06:22.060 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph orch host add smithi098 2023-09-09T23:06:22.622 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:06:22.840 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:22 smithi082 ceph-mon[98694]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:06:22.840 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:22 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:22.841 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:22 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:06:22.841 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:22 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:06:22.841 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:22 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:06:22.841 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:22 smithi082 ceph-mon[98694]: Updating smithi082:/etc/ceph/ceph.conf 2023-09-09T23:06:22.841 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:22 smithi082 ceph-mon[98694]: Updating smithi082:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-09T23:06:22.842 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:22 smithi082 ceph-mon[98694]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2023-09-09T23:06:22.842 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:22 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:22.842 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:22 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:22.842 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:22 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:24.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:24 smithi082 ceph-mon[98694]: Updating smithi082:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-09T23:06:25.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:25 smithi082 ceph-mon[98694]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi098", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:06:26.317 INFO:teuthology.orchestra.run.smithi082.stdout:Added host 'smithi098' with addr '172.21.15.98' 2023-09-09T23:06:26.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:26 smithi082 ceph-mon[98694]: Deploying cephadm binary to smithi098 2023-09-09T23:06:26.668 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph orch host ls --format=json 2023-09-09T23:06:27.222 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:06:27.595 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:27 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:27.595 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:27 smithi082 ceph-mon[98694]: Added host smithi098 2023-09-09T23:06:27.596 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:27 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:06:27.596 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:27 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:28.613 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:06:28.613 INFO:teuthology.orchestra.run.smithi082.stdout:[{"addr": "172.21.15.82", "hostname": "smithi082", "labels": [], "status": ""}, {"addr": "172.21.15.98", "hostname": "smithi098", "labels": [], "status": ""}] 2023-09-09T23:06:28.638 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:28 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:28.964 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi170 2023-09-09T23:06:28.965 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-09-09T23:06:28.965 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/etc/ceph/ceph.conf 2023-09-09T23:06:28.987 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-09-09T23:06:28.987 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-09T23:06:29.046 INFO:tasks.cephadm:Adding host smithi170 to orchestrator... 2023-09-09T23:06:29.047 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph orch host add smithi170 2023-09-09T23:06:29.599 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:06:29.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:29 smithi082 ceph-mon[98694]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-09T23:06:29.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:29 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:29.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:29 smithi082 ceph-mon[98694]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:32.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:31 smithi082 ceph-mon[98694]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:06:32.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:31 smithi082 ceph-mon[98694]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:32.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:31 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:32.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:31 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:32.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:31 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:31 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:31 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi098", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:06:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:31 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:06:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:31 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:06:33.190 INFO:teuthology.orchestra.run.smithi082.stdout:Added host 'smithi170' with addr '172.21.15.170' 2023-09-09T23:06:33.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:32 smithi082 ceph-mon[98694]: Deploying cephadm binary to smithi170 2023-09-09T23:06:33.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:32 smithi082 ceph-mon[98694]: Updating smithi098:/etc/ceph/ceph.conf 2023-09-09T23:06:33.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:32 smithi082 ceph-mon[98694]: Updating smithi098:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-09T23:06:33.546 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph orch host ls --format=json 2023-09-09T23:06:34.100 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:06:34.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:34 smithi082 ceph-mon[98694]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2023-09-09T23:06:34.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:34 smithi082 ceph-mon[98694]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:34.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:34 smithi082 ceph-mon[98694]: Updating smithi098:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-09T23:06:34.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:34 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:34.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:34 smithi082 ceph-mon[98694]: Added host smithi170 2023-09-09T23:06:34.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:34 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:34.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:34 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:34.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:34 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:34 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:06:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:34 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:35.475 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:06:35.475 INFO:teuthology.orchestra.run.smithi082.stdout:[{"addr": "172.21.15.82", "hostname": "smithi082", "labels": [], "status": ""}, {"addr": "172.21.15.98", "hostname": "smithi098", "labels": [], "status": ""}, {"addr": "172.21.15.170", "hostname": "smithi170", "labels": [], "status": ""}] 2023-09-09T23:06:35.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:35 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:35.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:35 smithi082 ceph-mon[98694]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:35.853 INFO:tasks.cephadm:Setting crush tunables to default 2023-09-09T23:06:35.853 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph osd crush tunables default 2023-09-09T23:06:36.405 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:06:36.856 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:36 smithi082 ceph-mon[98694]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-09T23:06:36.857 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:36 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:37.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:37 smithi082 ceph-mon[98694]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:38.630 INFO:teuthology.orchestra.run.smithi082.stderr:adjusted tunables profile to default 2023-09-09T23:06:38.944 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:38 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/620899770' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-09-09T23:06:38.945 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:38 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:38.945 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:38 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:38.945 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:38 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:38.945 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:38 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:38.946 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:38 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi170", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:06:38.946 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:38 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:06:38.946 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:38 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:06:38.963 INFO:tasks.cephadm:Adding mon.a on smithi082 2023-09-09T23:06:38.964 INFO:tasks.cephadm:Adding mon.b on smithi098 2023-09-09T23:06:38.964 INFO:tasks.cephadm:Adding mon.c on smithi170 2023-09-09T23:06:38.964 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph orch apply mon '3;smithi082:172.21.15.82=a;smithi098:172.21.15.98=b;smithi170:172.21.15.170=c' 2023-09-09T23:06:39.551 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-09T23:06:39.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:39 smithi082 ceph-mon[98694]: Updating smithi170:/etc/ceph/ceph.conf 2023-09-09T23:06:39.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:39 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/620899770' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-09-09T23:06:39.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:39 smithi082 ceph-mon[98694]: osdmap e4: 0 total, 0 up, 0 in 2023-09-09T23:06:39.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:39 smithi082 ceph-mon[98694]: Updating smithi170:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-09T23:06:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:39 smithi082 ceph-mon[98694]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2023-09-09T23:06:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:39 smithi082 ceph-mon[98694]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:39 smithi082 ceph-mon[98694]: Updating smithi170:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-09T23:06:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:39 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:39 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:39 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:40.968 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled mon update... 2023-09-09T23:06:41.418 DEBUG:teuthology.orchestra.run.smithi098:mon.b> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@mon.b.service 2023-09-09T23:06:41.421 DEBUG:teuthology.orchestra.run.smithi170:mon.c> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@mon.c.service 2023-09-09T23:06:41.424 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-09-09T23:06:41.424 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph mon dump -f json 2023-09-09T23:06:41.456 INFO:journalctl@ceph.mon.c.smithi170.stdout:-- Logs begin at Sat 2023-09-09 22:51:11 UTC. -- 2023-09-09T23:06:41.457 INFO:journalctl@ceph.mon.b.smithi098.stdout:-- Logs begin at Sat 2023-09-09 22:51:28 UTC. -- 2023-09-09T23:06:42.278 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-09T23:06:42.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:41 smithi082 ceph-mon[98694]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:42.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:41 smithi082 ceph-mon[98694]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi082:172.21.15.82=a;smithi098:172.21.15.98=b;smithi170:172.21.15.170=c", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:06:42.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:41 smithi082 ceph-mon[98694]: Saving service mon spec with placement smithi082:172.21.15.82=a;smithi098:172.21.15.98=b;smithi170:172.21.15.170=c;count:3 2023-09-09T23:06:42.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:41 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:42.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:41 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:06:42.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:41 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:06:42.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:41 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:06:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:41 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:41 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-09T23:06:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:41 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:06:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:41 smithi082 ceph-mon[98694]: Deploying daemon mon.c on smithi170 2023-09-09T23:06:43.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:43 smithi170 systemd[1]: Starting Ceph mon.c for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da... 2023-09-09T23:06:44.140 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-09T23:06:44.140 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":1,"fsid":"3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da","modified":"2023-09-09T23:05:10.079367Z","created":"2023-09-09T23:05:10.079367Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:3300","nonce":0},{"type":"v1","addr":"172.21.15.82:6789","nonce":0}]},"addr":"172.21.15.82:6789/0","public_addr":"172.21.15.82:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-09-09T23:06:44.141 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 1 2023-09-09T23:06:44.405 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 podman[102778]: 2023-09-09T23:06:44.405 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: set uid:gid to 167:167 (ceph:ceph) 2023-09-09T23:06:44.405 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev), process ceph-mon, pid 7 2023-09-09T23:06:44.405 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: pidfile_write: ignore empty --pid-file 2023-09-09T23:06:44.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: load: jerasure load: lrc 2023-09-09T23:06:44.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: RocksDB version: 7.9.2 2023-09-09T23:06:44.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Git sha 0 2023-09-09T23:06:44.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Compile date 2023-09-09 15:16:58 2023-09-09T23:06:44.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: DB SUMMARY 2023-09-09T23:06:44.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: DB Session ID: Y5EXH4ZMVBHLBHL8LXEH 2023-09-09T23:06:44.407 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: CURRENT file: CURRENT 2023-09-09T23:06:44.407 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: IDENTITY file: IDENTITY 2023-09-09T23:06:44.407 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-09-09T23:06:44.407 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-09-09T23:06:44.407 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-09-09T23:06:44.408 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.error_if_exists: 0 2023-09-09T23:06:44.408 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.create_if_missing: 0 2023-09-09T23:06:44.408 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.paranoid_checks: 1 2023-09-09T23:06:44.408 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.flush_verify_memtable_count: 1 2023-09-09T23:06:44.408 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-09-09T23:06:44.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-09-09T23:06:44.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.env: 0x55d83fda2da0 2023-09-09T23:06:44.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.fs: PosixFileSystem 2023-09-09T23:06:44.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.info_log: 0x55d8414e34e0 2023-09-09T23:06:44.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.max_file_opening_threads: 16 2023-09-09T23:06:44.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.statistics: (nil) 2023-09-09T23:06:44.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.use_fsync: 0 2023-09-09T23:06:44.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.max_log_file_size: 0 2023-09-09T23:06:44.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-09-09T23:06:44.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.log_file_time_to_roll: 0 2023-09-09T23:06:44.410 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.keep_log_file_num: 1000 2023-09-09T23:06:44.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.recycle_log_file_num: 0 2023-09-09T23:06:44.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.allow_fallocate: 1 2023-09-09T23:06:44.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.allow_mmap_reads: 0 2023-09-09T23:06:44.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.allow_mmap_writes: 0 2023-09-09T23:06:44.411 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.use_direct_reads: 0 2023-09-09T23:06:44.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-09-09T23:06:44.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.create_missing_column_families: 0 2023-09-09T23:06:44.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.db_log_dir: 2023-09-09T23:06:44.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.wal_dir: 2023-09-09T23:06:44.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.table_cache_numshardbits: 6 2023-09-09T23:06:44.413 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.WAL_ttl_seconds: 0 2023-09-09T23:06:44.413 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.WAL_size_limit_MB: 0 2023-09-09T23:06:44.413 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-09-09T23:06:44.413 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-09-09T23:06:44.413 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.is_fd_close_on_exec: 1 2023-09-09T23:06:44.413 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.advise_random_on_open: 1 2023-09-09T23:06:44.414 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.db_write_buffer_size: 0 2023-09-09T23:06:44.414 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.write_buffer_manager: 0x55d84149d4a0 2023-09-09T23:06:44.414 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-09-09T23:06:44.414 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-09-09T23:06:44.414 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.use_adaptive_mutex: 0 2023-09-09T23:06:44.415 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.rate_limiter: (nil) 2023-09-09T23:06:44.415 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-09-09T23:06:44.415 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.wal_recovery_mode: 2 2023-09-09T23:06:44.415 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.enable_thread_tracking: 0 2023-09-09T23:06:44.415 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.enable_pipelined_write: 0 2023-09-09T23:06:44.416 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.unordered_write: 0 2023-09-09T23:06:44.416 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-09-09T23:06:44.416 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-09-09T23:06:44.416 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-09-09T23:06:44.416 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-09-09T23:06:44.417 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.row_cache: None 2023-09-09T23:06:44.417 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.wal_filter: None 2023-09-09T23:06:44.417 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-09-09T23:06:44.417 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.allow_ingest_behind: 0 2023-09-09T23:06:44.417 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.two_write_queues: 0 2023-09-09T23:06:44.417 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.manual_wal_flush: 0 2023-09-09T23:06:44.418 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.wal_compression: 0 2023-09-09T23:06:44.418 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.atomic_flush: 0 2023-09-09T23:06:44.418 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-09-09T23:06:44.418 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.persist_stats_to_disk: 0 2023-09-09T23:06:44.418 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.write_dbid_to_manifest: 0 2023-09-09T23:06:44.419 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.log_readahead_size: 0 2023-09-09T23:06:44.419 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-09-09T23:06:44.419 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.best_efforts_recovery: 0 2023-09-09T23:06:44.419 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-09-09T23:06:44.419 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-09-09T23:06:44.419 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.allow_data_in_errors: 0 2023-09-09T23:06:44.420 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.db_host_id: __hostname__ 2023-09-09T23:06:44.420 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.enforce_single_del_contracts: true 2023-09-09T23:06:44.420 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.max_background_jobs: 2 2023-09-09T23:06:44.420 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.max_background_compactions: -1 2023-09-09T23:06:44.420 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.max_subcompactions: 1 2023-09-09T23:06:44.421 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-09-09T23:06:44.421 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-09-09T23:06:44.421 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.delayed_write_rate : 16777216 2023-09-09T23:06:44.421 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.max_total_wal_size: 0 2023-09-09T23:06:44.421 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-09-09T23:06:44.422 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.stats_dump_period_sec: 600 2023-09-09T23:06:44.422 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.stats_persist_period_sec: 600 2023-09-09T23:06:44.422 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-09-09T23:06:44.422 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.max_open_files: -1 2023-09-09T23:06:44.422 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.bytes_per_sync: 0 2023-09-09T23:06:44.423 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.wal_bytes_per_sync: 0 2023-09-09T23:06:44.423 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.strict_bytes_per_sync: 0 2023-09-09T23:06:44.423 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.compaction_readahead_size: 0 2023-09-09T23:06:44.423 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.max_background_flushes: -1 2023-09-09T23:06:44.423 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Compression algorithms supported: 2023-09-09T23:06:44.423 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-09-09T23:06:44.424 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: kZSTD supported: 0 2023-09-09T23:06:44.424 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: kXpressCompression supported: 0 2023-09-09T23:06:44.424 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: kLZ4HCCompression supported: 1 2023-09-09T23:06:44.424 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: kZlibCompression supported: 1 2023-09-09T23:06:44.424 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: kSnappyCompression supported: 1 2023-09-09T23:06:44.424 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: kLZ4Compression supported: 1 2023-09-09T23:06:44.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: kBZip2Compression supported: 0 2023-09-09T23:06:44.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Fast CRC32 supported: Supported on x86 2023-09-09T23:06:44.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: DMutex implementation: pthread_mutex_t 2023-09-09T23:06:44.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-09-09T23:06:44.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-09-09T23:06:44.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-09-09T23:06:44.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.merge_operator: 2023-09-09T23:06:44.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.compaction_filter: None 2023-09-09T23:06:44.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.compaction_filter_factory: None 2023-09-09T23:06:44.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.sst_partitioner_factory: None 2023-09-09T23:06:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.memtable_factory: SkipListFactory 2023-09-09T23:06:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.table_factory: BlockBasedTable 2023-09-09T23:06:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d8414e35e0) 2023-09-09T23:06:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout: cache_index_and_filter_blocks: 1 2023-09-09T23:06:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-09-09T23:06:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-09-09T23:06:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout: pin_top_level_index_and_filter: 1 2023-09-09T23:06:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout: index_type: 0 2023-09-09T23:06:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout: data_block_index_type: 0 2023-09-09T23:06:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout: index_shortening: 1 2023-09-09T23:06:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout: data_block_hash_table_util_ratio: 0.750000 2023-09-09T23:06:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout: checksum: 4 2023-09-09T23:06:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout: no_block_cache: 0 2023-09-09T23:06:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_cache: 0x55d84153b1f0 2023-09-09T23:06:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_cache_name: BinnedLRUCache 2023-09-09T23:06:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_cache_options: 2023-09-09T23:06:44.430 INFO:journalctl@ceph.mon.c.smithi170.stdout: capacity : 536870912 2023-09-09T23:06:44.430 INFO:journalctl@ceph.mon.c.smithi170.stdout: num_shard_bits : 4 2023-09-09T23:06:44.430 INFO:journalctl@ceph.mon.c.smithi170.stdout: strict_capacity_limit : 0 2023-09-09T23:06:44.430 INFO:journalctl@ceph.mon.c.smithi170.stdout: high_pri_pool_ratio: 0.000 2023-09-09T23:06:44.430 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_cache_compressed: (nil) 2023-09-09T23:06:44.431 INFO:journalctl@ceph.mon.c.smithi170.stdout: persistent_cache: (nil) 2023-09-09T23:06:44.431 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_size: 4096 2023-09-09T23:06:44.431 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_size_deviation: 10 2023-09-09T23:06:44.431 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_restart_interval: 16 2023-09-09T23:06:44.431 INFO:journalctl@ceph.mon.c.smithi170.stdout: index_block_restart_interval: 1 2023-09-09T23:06:44.432 INFO:journalctl@ceph.mon.c.smithi170.stdout: metadata_block_size: 4096 2023-09-09T23:06:44.432 INFO:journalctl@ceph.mon.c.smithi170.stdout: partition_filters: 0 2023-09-09T23:06:44.432 INFO:journalctl@ceph.mon.c.smithi170.stdout: use_delta_encoding: 1 2023-09-09T23:06:44.432 INFO:journalctl@ceph.mon.c.smithi170.stdout: filter_policy: bloomfilter 2023-09-09T23:06:44.432 INFO:journalctl@ceph.mon.c.smithi170.stdout: whole_key_filtering: 1 2023-09-09T23:06:44.432 INFO:journalctl@ceph.mon.c.smithi170.stdout: verify_compression: 0 2023-09-09T23:06:44.433 INFO:journalctl@ceph.mon.c.smithi170.stdout: read_amp_bytes_per_bit: 0 2023-09-09T23:06:44.433 INFO:journalctl@ceph.mon.c.smithi170.stdout: format_version: 5 2023-09-09T23:06:44.433 INFO:journalctl@ceph.mon.c.smithi170.stdout: enable_index_compression: 1 2023-09-09T23:06:44.433 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_align: 0 2023-09-09T23:06:44.433 INFO:journalctl@ceph.mon.c.smithi170.stdout: max_auto_readahead_size: 262144 2023-09-09T23:06:44.434 INFO:journalctl@ceph.mon.c.smithi170.stdout: prepopulate_block_cache: 0 2023-09-09T23:06:44.434 INFO:journalctl@ceph.mon.c.smithi170.stdout: initial_auto_readahead_size: 8192 2023-09-09T23:06:44.434 INFO:journalctl@ceph.mon.c.smithi170.stdout: num_file_reads_for_auto_readahead: 2 2023-09-09T23:06:44.434 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.write_buffer_size: 33554432 2023-09-09T23:06:44.434 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.max_write_buffer_number: 2 2023-09-09T23:06:44.434 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.compression: NoCompression 2023-09-09T23:06:44.435 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.bottommost_compression: Disabled 2023-09-09T23:06:44.435 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.prefix_extractor: nullptr 2023-09-09T23:06:44.435 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-09-09T23:06:44.435 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.num_levels: 7 2023-09-09T23:06:44.435 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-09-09T23:06:44.436 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-09-09T23:06:44.436 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-09-09T23:06:44.436 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-09-09T23:06:44.436 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-09-09T23:06:44.436 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-09-09T23:06:44.436 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-09-09T23:06:44.437 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-09-09T23:06:44.437 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-09-09T23:06:44.437 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-09-09T23:06:44.437 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-09-09T23:06:44.437 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-09-09T23:06:44.438 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.compression_opts.window_bits: -14 2023-09-09T23:06:44.438 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.compression_opts.level: 32767 2023-09-09T23:06:44.438 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.compression_opts.strategy: 0 2023-09-09T23:06:44.438 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-09-09T23:06:44.438 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-09-09T23:06:44.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-09-09T23:06:44.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-09-09T23:06:44.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.compression_opts.enabled: false 2023-09-09T23:06:44.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-09-09T23:06:44.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-09-09T23:06:44.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-09-09T23:06:44.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-09-09T23:06:44.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.target_file_size_base: 67108864 2023-09-09T23:06:44.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.target_file_size_multiplier: 1 2023-09-09T23:06:44.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-09-09T23:06:44.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-09-09T23:06:44.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-09-09T23:06:44.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-09-09T23:06:44.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-09-09T23:06:44.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-09-09T23:06:44.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-09-09T23:06:44.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-09-09T23:06:44.442 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-09-09T23:06:44.442 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-09-09T23:06:44.442 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-09-09T23:06:44.442 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-09-09T23:06:44.442 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-09-09T23:06:44.443 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.arena_block_size: 1048576 2023-09-09T23:06:44.443 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-09-09T23:06:44.443 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-09-09T23:06:44.443 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.disable_auto_compactions: 0 2023-09-09T23:06:44.443 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-09-09T23:06:44.444 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-09-09T23:06:44.445 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-09-09T23:06:44.445 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-09-09T23:06:44.445 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-09-09T23:06:44.445 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-09-09T23:06:44.445 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-09-09T23:06:44.446 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-09-09T23:06:44.446 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-09-09T23:06:44.446 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-09-09T23:06:44.446 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.table_properties_collectors: 2023-09-09T23:06:44.446 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.inplace_update_support: 0 2023-09-09T23:06:44.447 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.inplace_update_num_locks: 10000 2023-09-09T23:06:44.447 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-09-09T23:06:44.447 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-09-09T23:06:44.447 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.memtable_huge_page_size: 0 2023-09-09T23:06:44.447 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.bloom_locality: 0 2023-09-09T23:06:44.447 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.max_successive_merges: 0 2023-09-09T23:06:44.448 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.optimize_filters_for_hits: 0 2023-09-09T23:06:44.448 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.paranoid_file_checks: 0 2023-09-09T23:06:44.448 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.force_consistency_checks: 1 2023-09-09T23:06:44.448 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.report_bg_io_stats: 0 2023-09-09T23:06:44.448 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.ttl: 2592000 2023-09-09T23:06:44.449 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.periodic_compaction_seconds: 0 2023-09-09T23:06:44.449 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-09-09T23:06:44.449 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-09-09T23:06:44.449 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.enable_blob_files: false 2023-09-09T23:06:44.449 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.min_blob_size: 0 2023-09-09T23:06:44.449 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.blob_file_size: 268435456 2023-09-09T23:06:44.450 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.blob_compression_type: NoCompression 2023-09-09T23:06:44.450 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.enable_blob_garbage_collection: false 2023-09-09T23:06:44.450 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-09-09T23:06:44.450 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-09-09T23:06:44.450 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-09-09T23:06:44.451 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.blob_file_starting_level: 0 2023-09-09T23:06:44.451 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-09-09T23:06:44.451 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-09-09T23:06:44.451 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-09-09T23:06:44.451 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b6bb205a-63ba-4ae1-a80a-5fdcac14f4a4 2023-09-09T23:06:44.451 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694300804265594, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-09-09T23:06:44.452 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-09-09T23:06:44.452 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694300804266016, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1694300804, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b6bb205a-63ba-4ae1-a80a-5fdcac14f4a4", "db_session_id": "Y5EXH4ZMVBHLBHL8LXEH", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-09-09T23:06:44.452 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694300804266080, "job": 1, "event": "recovery_finished"} 2023-09-09T23:06:44.452 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-09-09T23:06:44.452 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-09-09T23:06:44.453 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d8415d6000 2023-09-09T23:06:44.453 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: DB pointer 0x55d8415c2000 2023-09-09T23:06:44.453 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-09-09T23:06:44.453 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-09-09T23:06:44.453 INFO:journalctl@ceph.mon.c.smithi170.stdout: ** DB Stats ** 2023-09-09T23:06:44.454 INFO:journalctl@ceph.mon.c.smithi170.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-09T23:06:44.454 INFO:journalctl@ceph.mon.c.smithi170.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-09-09T23:06:44.454 INFO:journalctl@ceph.mon.c.smithi170.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-09T23:06:44.454 INFO:journalctl@ceph.mon.c.smithi170.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-09T23:06:44.454 INFO:journalctl@ceph.mon.c.smithi170.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-09-09T23:06:44.454 INFO:journalctl@ceph.mon.c.smithi170.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-09T23:06:44.455 INFO:journalctl@ceph.mon.c.smithi170.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-09T23:06:44.455 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2023-09-09T23:06:44.455 INFO:journalctl@ceph.mon.c.smithi170.stdout: ** Compaction Stats [default] ** 2023-09-09T23:06:44.455 INFO:journalctl@ceph.mon.c.smithi170.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-09T23:06:44.455 INFO:journalctl@ceph.mon.c.smithi170.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-09-09T23:06:44.456 INFO:journalctl@ceph.mon.c.smithi170.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-09T23:06:44.456 INFO:journalctl@ceph.mon.c.smithi170.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-09T23:06:44.456 INFO:journalctl@ceph.mon.c.smithi170.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-09T23:06:44.456 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2023-09-09T23:06:44.456 INFO:journalctl@ceph.mon.c.smithi170.stdout: ** Compaction Stats [default] ** 2023-09-09T23:06:44.457 INFO:journalctl@ceph.mon.c.smithi170.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-09T23:06:44.457 INFO:journalctl@ceph.mon.c.smithi170.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-09-09T23:06:44.457 INFO:journalctl@ceph.mon.c.smithi170.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-09T23:06:44.457 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2023-09-09T23:06:44.457 INFO:journalctl@ceph.mon.c.smithi170.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-09-09T23:06:44.458 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2023-09-09T23:06:44.458 INFO:journalctl@ceph.mon.c.smithi170.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-09T23:06:44.458 INFO:journalctl@ceph.mon.c.smithi170.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-09-09T23:06:44.458 INFO:journalctl@ceph.mon.c.smithi170.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-09-09T23:06:44.458 INFO:journalctl@ceph.mon.c.smithi170.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-09-09T23:06:44.458 INFO:journalctl@ceph.mon.c.smithi170.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-09-09T23:06:44.459 INFO:journalctl@ceph.mon.c.smithi170.stdout: AddFile(Keys): cumulative 0, interval 0 2023-09-09T23:06:44.459 INFO:journalctl@ceph.mon.c.smithi170.stdout: Cumulative compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-09T23:06:44.459 INFO:journalctl@ceph.mon.c.smithi170.stdout: Interval compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-09T23:06:44.459 INFO:journalctl@ceph.mon.c.smithi170.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-09-09T23:06:44.459 INFO:journalctl@ceph.mon.c.smithi170.stdout: Block cache BinnedLRUCache@0x55d84153b1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-09-09T23:06:44.459 INFO:journalctl@ceph.mon.c.smithi170.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-09-09T23:06:44.460 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2023-09-09T23:06:44.460 INFO:journalctl@ceph.mon.c.smithi170.stdout: ** File Read Latency Histogram By Level [default] ** 2023-09-09T23:06:44.460 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-09-09T23:06:44.460 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: using public_addr v2:172.21.15.170:0/0 -> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] 2023-09-09T23:06:44.460 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: starting mon.c rank -1 at public addrs [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] at bind addrs [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da 2023-09-09T23:06:44.461 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: mon.c@-1(???) e0 preinit fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da 2023-09-09T23:06:44.461 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: mon.c@-1(synchronizing).mds e1 new map 2023-09-09T23:06:44.461 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: mon.c@-1(synchronizing).mds e1 print_map 2023-09-09T23:06:44.461 INFO:journalctl@ceph.mon.c.smithi170.stdout: e1 2023-09-09T23:06:44.461 INFO:journalctl@ceph.mon.c.smithi170.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-09-09T23:06:44.461 INFO:journalctl@ceph.mon.c.smithi170.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-09T23:06:44.462 INFO:journalctl@ceph.mon.c.smithi170.stdout: legacy client fscid: -1 2023-09-09T23:06:44.462 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2023-09-09T23:06:44.462 INFO:journalctl@ceph.mon.c.smithi170.stdout: No filesystems configured 2023-09-09T23:06:44.462 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-09-09T23:06:44.462 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-09-09T23:06:44.463 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-09-09T23:06:44.463 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-09-09T23:06:44.463 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-09-09T23:06:44.463 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-09-09T23:06:44.463 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-09-09T23:06:44.464 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-09T23:06:44.464 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-09T23:06:44.464 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-09T23:06:44.464 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:44.464 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:44.464 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-09T23:06:44.465 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:44.465 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:44.465 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/620899770' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-09-09T23:06:44.465 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:44.465 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:44.465 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:44.466 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:44.466 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi170", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:06:44.466 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:06:44.466 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:06:44.466 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: Updating smithi170:/etc/ceph/ceph.conf 2023-09-09T23:06:44.467 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/620899770' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-09-09T23:06:44.467 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: osdmap e4: 0 total, 0 up, 0 in 2023-09-09T23:06:44.467 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: Updating smithi170:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-09T23:06:44.467 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2023-09-09T23:06:44.467 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:44.467 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: Updating smithi170:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-09T23:06:44.468 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:44.468 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:44.468 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:44.468 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:44.468 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi082:172.21.15.82=a;smithi098:172.21.15.98=b;smithi170:172.21.15.170=c", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:06:44.469 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: Saving service mon spec with placement smithi082:172.21.15.82=a;smithi098:172.21.15.98=b;smithi170:172.21.15.170=c;count:3 2023-09-09T23:06:44.469 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:44.469 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:06:44.469 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:06:44.469 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:06:44.470 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:44.470 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-09T23:06:44.470 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:06:44.470 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: Deploying daemon mon.c on smithi170 2023-09-09T23:06:44.470 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:44.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: from='client.? 172.21.15.170:0/1347188464' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-09T23:06:44.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-09-09T23:06:44.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: expand_channel_meta expand map: {default=false} 2023-09-09T23:06:44.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: expand_channel_meta from 'false' to 'false' 2023-09-09T23:06:44.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: expand_channel_meta expanded map: {default=false} 2023-09-09T23:06:44.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: expand_channel_meta expand map: {default=info} 2023-09-09T23:06:44.472 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: expand_channel_meta from 'info' to 'info' 2023-09-09T23:06:44.472 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: expand_channel_meta expanded map: {default=info} 2023-09-09T23:06:44.472 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: expand_channel_meta expand map: {default=daemon} 2023-09-09T23:06:44.472 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: expand_channel_meta from 'daemon' to 'daemon' 2023-09-09T23:06:44.472 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: expand_channel_meta expanded map: {default=daemon} 2023-09-09T23:06:44.473 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: expand_channel_meta expand map: {default=debug} 2023-09-09T23:06:44.473 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: expand_channel_meta from 'debug' to 'debug' 2023-09-09T23:06:44.473 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: expand_channel_meta expanded map: {default=debug} 2023-09-09T23:06:44.473 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 ceph-mon[102832]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-09-09T23:06:44.473 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 bash[102778]: 50a05b43745bbab0c4bd3b65c20f73d72c6636b347920fd34e23e676f6c81d6d 2023-09-09T23:06:44.474 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:44 smithi082 ceph-mon[98694]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:44.474 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:44 smithi082 ceph-mon[98694]: from='client.? 172.21.15.170:0/1347188464' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-09T23:06:44.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:44 smithi170 systemd[1]: Started Ceph mon.c for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da. 2023-09-09T23:06:45.519 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-09-09T23:06:45.520 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph mon dump -f json 2023-09-09T23:06:46.087 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-09T23:06:49.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:49 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-09T23:06:49.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:49 smithi170 ceph-mon[102832]: mon.a calling monitor election 2023-09-09T23:06:49.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:49 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:06:49.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:49 smithi170 ceph-mon[102832]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:49 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:06:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:49 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:06:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:49 smithi170 ceph-mon[102832]: mon.c calling monitor election 2023-09-09T23:06:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:49 smithi170 ceph-mon[102832]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:49 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:06:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:49 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:06:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:49 smithi170 ceph-mon[102832]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:49 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:06:49.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:49 smithi170 ceph-mon[102832]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-09T23:06:49.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:49 smithi170 ceph-mon[102832]: monmap e2: 2 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} 2023-09-09T23:06:49.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:49 smithi170 ceph-mon[102832]: fsmap 2023-09-09T23:06:49.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:49 smithi170 ceph-mon[102832]: osdmap e4: 0 total, 0 up, 0 in 2023-09-09T23:06:49.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:49 smithi170 ceph-mon[102832]: mgrmap e13: x(active, since 40s) 2023-09-09T23:06:49.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:49 smithi170 ceph-mon[102832]: overall HEALTH_OK 2023-09-09T23:06:49.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:49 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:49.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:49 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:49.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:49 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:49.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:49 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-09T23:06:49.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:49 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:06:49.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:49 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-09T23:06:49.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:49 smithi082 ceph-mon[98694]: mon.a calling monitor election 2023-09-09T23:06:49.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:49 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:06:49.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:49 smithi082 ceph-mon[98694]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:49.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:49 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:06:49.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:49 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:06:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:49 smithi082 ceph-mon[98694]: mon.c calling monitor election 2023-09-09T23:06:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:49 smithi082 ceph-mon[98694]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:49 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:06:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:49 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:06:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:49 smithi082 ceph-mon[98694]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:49 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:06:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:49 smithi082 ceph-mon[98694]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-09T23:06:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:49 smithi082 ceph-mon[98694]: monmap e2: 2 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} 2023-09-09T23:06:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:49 smithi082 ceph-mon[98694]: fsmap 2023-09-09T23:06:49.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:49 smithi082 ceph-mon[98694]: osdmap e4: 0 total, 0 up, 0 in 2023-09-09T23:06:49.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:49 smithi082 ceph-mon[98694]: mgrmap e13: x(active, since 40s) 2023-09-09T23:06:49.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:49 smithi082 ceph-mon[98694]: overall HEALTH_OK 2023-09-09T23:06:49.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:49 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:49.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:49 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:49.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:49 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:49.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:49 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-09T23:06:49.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:49 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:06:50.459 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-09T23:06:50.460 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":2,"fsid":"3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da","modified":"2023-09-09T23:06:44.288337Z","created":"2023-09-09T23:05:10.079367Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:3300","nonce":0},{"type":"v1","addr":"172.21.15.82:6789","nonce":0}]},"addr":"172.21.15.82:6789/0","public_addr":"172.21.15.82:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:3300","nonce":0},{"type":"v1","addr":"172.21.15.170:6789","nonce":0}]},"addr":"172.21.15.170:6789/0","public_addr":"172.21.15.170:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-09-09T23:06:50.461 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 2 2023-09-09T23:06:50.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:50 smithi170 ceph-mon[102832]: Deploying daemon mon.b on smithi098 2023-09-09T23:06:50.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:50 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:06:50.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:50 smithi082 ceph-mon[98694]: Deploying daemon mon.b on smithi098 2023-09-09T23:06:50.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:50 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:06:51.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:51 smithi170 ceph-mon[102832]: from='client.? 172.21.15.170:0/3496954253' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-09T23:06:51.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:51 smithi170 ceph-mon[102832]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:51.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:51 smithi082 ceph-mon[98694]: from='client.? 172.21.15.170:0/3496954253' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-09T23:06:51.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:51 smithi082 ceph-mon[98694]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:51.835 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-09-09T23:06:51.836 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph mon dump -f json 2023-09-09T23:06:52.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:51 smithi098 systemd[1]: Starting Ceph mon.b for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da... 2023-09-09T23:06:52.400 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-09T23:06:52.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 podman[103084]: 2023-09-09T23:06:52.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: set uid:gid to 167:167 (ceph:ceph) 2023-09-09T23:06:52.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: ceph version 18.0.0-6025-g9670cca5 (9670cca5bc8e17916865dc84939edfc10b1f2c00) reef (dev), process ceph-mon, pid 7 2023-09-09T23:06:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: pidfile_write: ignore empty --pid-file 2023-09-09T23:06:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 bash[103084]: 0ac2ec160985c836638cdbbfe683261981ffbc6e8d244cf0a63a37b20883303f 2023-09-09T23:06:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: load: jerasure load: lrc 2023-09-09T23:06:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: RocksDB version: 7.9.2 2023-09-09T23:06:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Git sha 0 2023-09-09T23:06:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Compile date 2023-09-09 15:16:58 2023-09-09T23:06:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: DB SUMMARY 2023-09-09T23:06:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: DB Session ID: NENA3G1U5015FQ8YVY4G 2023-09-09T23:06:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: CURRENT file: CURRENT 2023-09-09T23:06:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: IDENTITY file: IDENTITY 2023-09-09T23:06:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-09-09T23:06:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-09-09T23:06:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-09-09T23:06:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.error_if_exists: 0 2023-09-09T23:06:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.create_if_missing: 0 2023-09-09T23:06:52.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.paranoid_checks: 1 2023-09-09T23:06:52.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.flush_verify_memtable_count: 1 2023-09-09T23:06:52.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-09-09T23:06:52.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-09-09T23:06:52.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.env: 0x5645db8e5da0 2023-09-09T23:06:52.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.fs: PosixFileSystem 2023-09-09T23:06:52.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.info_log: 0x5645dd1894e0 2023-09-09T23:06:52.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.max_file_opening_threads: 16 2023-09-09T23:06:52.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.statistics: (nil) 2023-09-09T23:06:52.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.use_fsync: 0 2023-09-09T23:06:52.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.max_log_file_size: 0 2023-09-09T23:06:52.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-09-09T23:06:52.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.log_file_time_to_roll: 0 2023-09-09T23:06:52.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.keep_log_file_num: 1000 2023-09-09T23:06:52.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.recycle_log_file_num: 0 2023-09-09T23:06:52.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.allow_fallocate: 1 2023-09-09T23:06:52.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.allow_mmap_reads: 0 2023-09-09T23:06:52.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.allow_mmap_writes: 0 2023-09-09T23:06:52.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.use_direct_reads: 0 2023-09-09T23:06:52.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-09-09T23:06:52.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.create_missing_column_families: 0 2023-09-09T23:06:52.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.db_log_dir: 2023-09-09T23:06:52.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.wal_dir: 2023-09-09T23:06:52.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.table_cache_numshardbits: 6 2023-09-09T23:06:52.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.WAL_ttl_seconds: 0 2023-09-09T23:06:52.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.WAL_size_limit_MB: 0 2023-09-09T23:06:52.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-09-09T23:06:52.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-09-09T23:06:52.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.is_fd_close_on_exec: 1 2023-09-09T23:06:52.762 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.advise_random_on_open: 1 2023-09-09T23:06:52.762 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.db_write_buffer_size: 0 2023-09-09T23:06:52.762 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.write_buffer_manager: 0x5645dd1434a0 2023-09-09T23:06:52.762 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-09-09T23:06:52.762 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-09-09T23:06:52.763 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.use_adaptive_mutex: 0 2023-09-09T23:06:52.763 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.rate_limiter: (nil) 2023-09-09T23:06:52.763 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-09-09T23:06:52.763 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.wal_recovery_mode: 2 2023-09-09T23:06:52.763 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.enable_thread_tracking: 0 2023-09-09T23:06:52.763 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.enable_pipelined_write: 0 2023-09-09T23:06:52.764 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.unordered_write: 0 2023-09-09T23:06:52.764 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-09-09T23:06:52.764 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-09-09T23:06:52.764 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-09-09T23:06:52.764 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-09-09T23:06:52.765 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.row_cache: None 2023-09-09T23:06:52.765 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.wal_filter: None 2023-09-09T23:06:52.765 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-09-09T23:06:52.765 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.allow_ingest_behind: 0 2023-09-09T23:06:52.765 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.two_write_queues: 0 2023-09-09T23:06:52.766 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.manual_wal_flush: 0 2023-09-09T23:06:52.766 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.wal_compression: 0 2023-09-09T23:06:52.766 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.atomic_flush: 0 2023-09-09T23:06:52.766 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-09-09T23:06:52.766 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.persist_stats_to_disk: 0 2023-09-09T23:06:52.767 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.write_dbid_to_manifest: 0 2023-09-09T23:06:52.767 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.log_readahead_size: 0 2023-09-09T23:06:52.767 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-09-09T23:06:52.767 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.best_efforts_recovery: 0 2023-09-09T23:06:52.767 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-09-09T23:06:52.768 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-09-09T23:06:52.768 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.allow_data_in_errors: 0 2023-09-09T23:06:52.768 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.db_host_id: __hostname__ 2023-09-09T23:06:52.768 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.enforce_single_del_contracts: true 2023-09-09T23:06:52.768 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.max_background_jobs: 2 2023-09-09T23:06:52.769 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.max_background_compactions: -1 2023-09-09T23:06:52.769 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.max_subcompactions: 1 2023-09-09T23:06:52.769 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-09-09T23:06:52.769 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-09-09T23:06:52.769 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.delayed_write_rate : 16777216 2023-09-09T23:06:52.770 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.max_total_wal_size: 0 2023-09-09T23:06:52.770 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-09-09T23:06:52.770 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.stats_dump_period_sec: 600 2023-09-09T23:06:52.770 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.stats_persist_period_sec: 600 2023-09-09T23:06:52.770 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-09-09T23:06:52.771 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.max_open_files: -1 2023-09-09T23:06:52.771 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.bytes_per_sync: 0 2023-09-09T23:06:52.771 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.wal_bytes_per_sync: 0 2023-09-09T23:06:52.771 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.strict_bytes_per_sync: 0 2023-09-09T23:06:52.771 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.compaction_readahead_size: 0 2023-09-09T23:06:52.771 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.max_background_flushes: -1 2023-09-09T23:06:52.772 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Compression algorithms supported: 2023-09-09T23:06:52.772 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-09-09T23:06:52.772 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: kZSTD supported: 0 2023-09-09T23:06:52.772 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: kXpressCompression supported: 0 2023-09-09T23:06:52.772 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: kLZ4HCCompression supported: 1 2023-09-09T23:06:52.773 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: kZlibCompression supported: 1 2023-09-09T23:06:52.773 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: kSnappyCompression supported: 1 2023-09-09T23:06:52.773 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: kLZ4Compression supported: 1 2023-09-09T23:06:52.773 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: kBZip2Compression supported: 0 2023-09-09T23:06:52.773 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Fast CRC32 supported: Supported on x86 2023-09-09T23:06:52.773 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: DMutex implementation: pthread_mutex_t 2023-09-09T23:06:52.774 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-09-09T23:06:52.774 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-09-09T23:06:52.774 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-09-09T23:06:52.774 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.merge_operator: 2023-09-09T23:06:52.774 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.compaction_filter: None 2023-09-09T23:06:52.775 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.compaction_filter_factory: None 2023-09-09T23:06:52.775 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.sst_partitioner_factory: None 2023-09-09T23:06:52.775 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.memtable_factory: SkipListFactory 2023-09-09T23:06:52.775 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.table_factory: BlockBasedTable 2023-09-09T23:06:52.775 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5645dd1895e0) 2023-09-09T23:06:52.776 INFO:journalctl@ceph.mon.b.smithi098.stdout: cache_index_and_filter_blocks: 1 2023-09-09T23:06:52.776 INFO:journalctl@ceph.mon.b.smithi098.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-09-09T23:06:52.776 INFO:journalctl@ceph.mon.b.smithi098.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-09-09T23:06:52.776 INFO:journalctl@ceph.mon.b.smithi098.stdout: pin_top_level_index_and_filter: 1 2023-09-09T23:06:52.776 INFO:journalctl@ceph.mon.b.smithi098.stdout: index_type: 0 2023-09-09T23:06:52.777 INFO:journalctl@ceph.mon.b.smithi098.stdout: data_block_index_type: 0 2023-09-09T23:06:52.777 INFO:journalctl@ceph.mon.b.smithi098.stdout: index_shortening: 1 2023-09-09T23:06:52.777 INFO:journalctl@ceph.mon.b.smithi098.stdout: data_block_hash_table_util_ratio: 0.750000 2023-09-09T23:06:52.777 INFO:journalctl@ceph.mon.b.smithi098.stdout: checksum: 4 2023-09-09T23:06:52.777 INFO:journalctl@ceph.mon.b.smithi098.stdout: no_block_cache: 0 2023-09-09T23:06:52.777 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_cache: 0x5645dd1e11f0 2023-09-09T23:06:52.778 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_cache_name: BinnedLRUCache 2023-09-09T23:06:52.778 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_cache_options: 2023-09-09T23:06:52.778 INFO:journalctl@ceph.mon.b.smithi098.stdout: capacity : 536870912 2023-09-09T23:06:52.778 INFO:journalctl@ceph.mon.b.smithi098.stdout: num_shard_bits : 4 2023-09-09T23:06:52.778 INFO:journalctl@ceph.mon.b.smithi098.stdout: strict_capacity_limit : 0 2023-09-09T23:06:52.779 INFO:journalctl@ceph.mon.b.smithi098.stdout: high_pri_pool_ratio: 0.000 2023-09-09T23:06:52.779 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_cache_compressed: (nil) 2023-09-09T23:06:52.779 INFO:journalctl@ceph.mon.b.smithi098.stdout: persistent_cache: (nil) 2023-09-09T23:06:52.779 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_size: 4096 2023-09-09T23:06:52.779 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_size_deviation: 10 2023-09-09T23:06:52.779 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_restart_interval: 16 2023-09-09T23:06:52.780 INFO:journalctl@ceph.mon.b.smithi098.stdout: index_block_restart_interval: 1 2023-09-09T23:06:52.780 INFO:journalctl@ceph.mon.b.smithi098.stdout: metadata_block_size: 4096 2023-09-09T23:06:52.780 INFO:journalctl@ceph.mon.b.smithi098.stdout: partition_filters: 0 2023-09-09T23:06:52.780 INFO:journalctl@ceph.mon.b.smithi098.stdout: use_delta_encoding: 1 2023-09-09T23:06:52.780 INFO:journalctl@ceph.mon.b.smithi098.stdout: filter_policy: bloomfilter 2023-09-09T23:06:52.781 INFO:journalctl@ceph.mon.b.smithi098.stdout: whole_key_filtering: 1 2023-09-09T23:06:52.781 INFO:journalctl@ceph.mon.b.smithi098.stdout: verify_compression: 0 2023-09-09T23:06:52.781 INFO:journalctl@ceph.mon.b.smithi098.stdout: read_amp_bytes_per_bit: 0 2023-09-09T23:06:52.781 INFO:journalctl@ceph.mon.b.smithi098.stdout: format_version: 5 2023-09-09T23:06:52.781 INFO:journalctl@ceph.mon.b.smithi098.stdout: enable_index_compression: 1 2023-09-09T23:06:52.782 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_align: 0 2023-09-09T23:06:52.782 INFO:journalctl@ceph.mon.b.smithi098.stdout: max_auto_readahead_size: 262144 2023-09-09T23:06:52.782 INFO:journalctl@ceph.mon.b.smithi098.stdout: prepopulate_block_cache: 0 2023-09-09T23:06:52.782 INFO:journalctl@ceph.mon.b.smithi098.stdout: initial_auto_readahead_size: 8192 2023-09-09T23:06:52.782 INFO:journalctl@ceph.mon.b.smithi098.stdout: num_file_reads_for_auto_readahead: 2 2023-09-09T23:06:52.783 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.write_buffer_size: 33554432 2023-09-09T23:06:52.783 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.max_write_buffer_number: 2 2023-09-09T23:06:52.783 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.compression: NoCompression 2023-09-09T23:06:52.783 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.bottommost_compression: Disabled 2023-09-09T23:06:52.783 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.prefix_extractor: nullptr 2023-09-09T23:06:52.783 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-09-09T23:06:52.784 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.num_levels: 7 2023-09-09T23:06:52.784 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-09-09T23:06:52.784 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-09-09T23:06:52.784 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-09-09T23:06:52.784 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-09-09T23:06:52.785 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-09-09T23:06:52.785 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-09-09T23:06:52.786 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-09-09T23:06:52.786 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-09-09T23:06:52.787 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-09-09T23:06:52.787 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-09-09T23:06:52.787 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-09-09T23:06:52.787 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-09-09T23:06:52.787 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.compression_opts.window_bits: -14 2023-09-09T23:06:52.787 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.compression_opts.level: 32767 2023-09-09T23:06:52.788 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.compression_opts.strategy: 0 2023-09-09T23:06:52.788 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-09-09T23:06:52.788 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-09-09T23:06:52.788 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-09-09T23:06:52.788 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-09-09T23:06:52.789 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.compression_opts.enabled: false 2023-09-09T23:06:52.789 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-09-09T23:06:52.789 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-09-09T23:06:52.789 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-09-09T23:06:52.789 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-09-09T23:06:52.790 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.target_file_size_base: 67108864 2023-09-09T23:06:52.790 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.target_file_size_multiplier: 1 2023-09-09T23:06:52.790 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-09-09T23:06:52.790 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-09-09T23:06:52.790 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-09-09T23:06:52.791 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-09-09T23:06:52.791 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-09-09T23:06:52.791 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-09-09T23:06:52.791 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-09-09T23:06:52.791 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-09-09T23:06:52.792 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-09-09T23:06:52.792 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-09-09T23:06:52.792 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-09-09T23:06:52.792 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-09-09T23:06:52.792 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-09-09T23:06:52.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.arena_block_size: 1048576 2023-09-09T23:06:52.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-09-09T23:06:52.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-09-09T23:06:52.793 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.disable_auto_compactions: 0 2023-09-09T23:06:52.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-09-09T23:06:52.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-09-09T23:06:52.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-09-09T23:06:52.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-09-09T23:06:52.794 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-09-09T23:06:52.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-09-09T23:06:52.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-09-09T23:06:52.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-09-09T23:06:52.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-09-09T23:06:52.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-09-09T23:06:52.795 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.table_properties_collectors: 2023-09-09T23:06:52.796 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.inplace_update_support: 0 2023-09-09T23:06:52.796 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.inplace_update_num_locks: 10000 2023-09-09T23:06:52.796 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-09-09T23:06:52.797 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-09-09T23:06:52.797 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.memtable_huge_page_size: 0 2023-09-09T23:06:52.797 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.bloom_locality: 0 2023-09-09T23:06:52.797 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.max_successive_merges: 0 2023-09-09T23:06:52.798 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.optimize_filters_for_hits: 0 2023-09-09T23:06:52.798 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.paranoid_file_checks: 0 2023-09-09T23:06:52.798 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.force_consistency_checks: 1 2023-09-09T23:06:52.798 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.report_bg_io_stats: 0 2023-09-09T23:06:52.799 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.ttl: 2592000 2023-09-09T23:06:52.799 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.periodic_compaction_seconds: 0 2023-09-09T23:06:52.799 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-09-09T23:06:52.799 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-09-09T23:06:52.799 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.enable_blob_files: false 2023-09-09T23:06:52.800 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.min_blob_size: 0 2023-09-09T23:06:52.800 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.blob_file_size: 268435456 2023-09-09T23:06:52.800 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.blob_compression_type: NoCompression 2023-09-09T23:06:52.800 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.enable_blob_garbage_collection: false 2023-09-09T23:06:52.800 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-09-09T23:06:52.801 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-09-09T23:06:52.801 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-09-09T23:06:52.801 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.blob_file_starting_level: 0 2023-09-09T23:06:52.801 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-09-09T23:06:52.801 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-09-09T23:06:52.802 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-09-09T23:06:52.802 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6e27ee40-c49e-443b-80ab-1430c33d42ec 2023-09-09T23:06:52.802 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694300812426882, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-09-09T23:06:52.802 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-09-09T23:06:52.802 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694300812427451, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1694300812, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6e27ee40-c49e-443b-80ab-1430c33d42ec", "db_session_id": "NENA3G1U5015FQ8YVY4G", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-09-09T23:06:52.803 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: EVENT_LOG_v1 {"time_micros": 1694300812427538, "job": 1, "event": "recovery_finished"} 2023-09-09T23:06:52.803 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-09-09T23:06:52.803 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-09-09T23:06:52.803 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5645dd27c000 2023-09-09T23:06:52.803 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: DB pointer 0x5645dd268000 2023-09-09T23:06:52.804 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-09-09T23:06:52.804 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-09-09T23:06:52.804 INFO:journalctl@ceph.mon.b.smithi098.stdout: ** DB Stats ** 2023-09-09T23:06:52.804 INFO:journalctl@ceph.mon.b.smithi098.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-09T23:06:52.804 INFO:journalctl@ceph.mon.b.smithi098.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-09-09T23:06:52.805 INFO:journalctl@ceph.mon.b.smithi098.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-09T23:06:52.805 INFO:journalctl@ceph.mon.b.smithi098.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-09T23:06:52.805 INFO:journalctl@ceph.mon.b.smithi098.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-09-09T23:06:52.805 INFO:journalctl@ceph.mon.b.smithi098.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-09T23:06:52.805 INFO:journalctl@ceph.mon.b.smithi098.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-09T23:06:52.806 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2023-09-09T23:06:52.806 INFO:journalctl@ceph.mon.b.smithi098.stdout: ** Compaction Stats [default] ** 2023-09-09T23:06:52.806 INFO:journalctl@ceph.mon.b.smithi098.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-09T23:06:52.806 INFO:journalctl@ceph.mon.b.smithi098.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-09-09T23:06:52.806 INFO:journalctl@ceph.mon.b.smithi098.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-09T23:06:52.807 INFO:journalctl@ceph.mon.b.smithi098.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-09T23:06:52.807 INFO:journalctl@ceph.mon.b.smithi098.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-09T23:06:52.807 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2023-09-09T23:06:52.807 INFO:journalctl@ceph.mon.b.smithi098.stdout: ** Compaction Stats [default] ** 2023-09-09T23:06:52.807 INFO:journalctl@ceph.mon.b.smithi098.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-09T23:06:52.808 INFO:journalctl@ceph.mon.b.smithi098.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-09-09T23:06:52.808 INFO:journalctl@ceph.mon.b.smithi098.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-09T23:06:52.808 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2023-09-09T23:06:52.808 INFO:journalctl@ceph.mon.b.smithi098.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-09-09T23:06:52.808 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2023-09-09T23:06:52.809 INFO:journalctl@ceph.mon.b.smithi098.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-09T23:06:52.809 INFO:journalctl@ceph.mon.b.smithi098.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-09-09T23:06:52.809 INFO:journalctl@ceph.mon.b.smithi098.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-09-09T23:06:52.809 INFO:journalctl@ceph.mon.b.smithi098.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-09-09T23:06:52.809 INFO:journalctl@ceph.mon.b.smithi098.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-09-09T23:06:52.810 INFO:journalctl@ceph.mon.b.smithi098.stdout: AddFile(Keys): cumulative 0, interval 0 2023-09-09T23:06:52.810 INFO:journalctl@ceph.mon.b.smithi098.stdout: Cumulative compaction: 0.00 GB write, 0.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-09T23:06:52.810 INFO:journalctl@ceph.mon.b.smithi098.stdout: Interval compaction: 0.00 GB write, 0.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-09T23:06:52.810 INFO:journalctl@ceph.mon.b.smithi098.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-09-09T23:06:52.810 INFO:journalctl@ceph.mon.b.smithi098.stdout: Block cache BinnedLRUCache@0x5645dd1e11f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2023-09-09T23:06:52.810 INFO:journalctl@ceph.mon.b.smithi098.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-09-09T23:06:52.811 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2023-09-09T23:06:52.811 INFO:journalctl@ceph.mon.b.smithi098.stdout: ** File Read Latency Histogram By Level [default] ** 2023-09-09T23:06:52.811 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-09-09T23:06:52.811 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: using public_addr v2:172.21.15.98:0/0 -> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] 2023-09-09T23:06:52.811 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: starting mon.b rank -1 at public addrs [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] at bind addrs [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da 2023-09-09T23:06:52.812 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: mon.b@-1(???) e0 preinit fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da 2023-09-09T23:06:52.812 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: mon.b@-1(synchronizing).mds e1 new map 2023-09-09T23:06:52.812 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: mon.b@-1(synchronizing).mds e1 print_map 2023-09-09T23:06:52.812 INFO:journalctl@ceph.mon.b.smithi098.stdout: e1 2023-09-09T23:06:52.812 INFO:journalctl@ceph.mon.b.smithi098.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-09-09T23:06:52.813 INFO:journalctl@ceph.mon.b.smithi098.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-09T23:06:52.813 INFO:journalctl@ceph.mon.b.smithi098.stdout: legacy client fscid: -1 2023-09-09T23:06:52.813 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2023-09-09T23:06:52.813 INFO:journalctl@ceph.mon.b.smithi098.stdout: No filesystems configured 2023-09-09T23:06:52.814 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-09-09T23:06:52.814 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-09-09T23:06:52.814 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-09-09T23:06:52.814 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-09-09T23:06:52.814 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-09-09T23:06:52.815 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-09-09T23:06:52.815 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-09-09T23:06:52.815 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-09T23:06:52.815 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-09T23:06:52.816 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-09T23:06:52.816 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:52.816 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:52.816 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-09T23:06:52.816 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:52.817 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:52.817 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/620899770' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-09-09T23:06:52.817 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:52.817 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:52.817 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:52.818 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:52.818 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi170", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:06:52.818 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:06:52.818 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:06:52.818 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: Updating smithi170:/etc/ceph/ceph.conf 2023-09-09T23:06:52.819 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/620899770' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-09-09T23:06:52.819 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: osdmap e4: 0 total, 0 up, 0 in 2023-09-09T23:06:52.819 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: Updating smithi170:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-09T23:06:52.819 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2023-09-09T23:06:52.819 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:52.819 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: Updating smithi170:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-09T23:06:52.820 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:52.820 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:52.820 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:52.820 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:52.820 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi082:172.21.15.82=a;smithi098:172.21.15.98=b;smithi170:172.21.15.170=c", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:06:52.821 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: Saving service mon spec with placement smithi082:172.21.15.82=a;smithi098:172.21.15.98=b;smithi170:172.21.15.170=c;count:3 2023-09-09T23:06:52.821 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:52.821 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:06:52.821 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:06:52.821 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:06:52.822 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:52.822 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-09T23:06:52.822 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:06:52.822 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: Deploying daemon mon.c on smithi170 2023-09-09T23:06:52.822 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:52.823 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='client.? 172.21.15.170:0/1347188464' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-09T23:06:52.823 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-09T23:06:52.823 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: mon.a calling monitor election 2023-09-09T23:06:52.823 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:06:52.823 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:52.824 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:06:52.824 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:06:52.824 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: mon.c calling monitor election 2023-09-09T23:06:52.824 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:52.824 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:06:52.824 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:06:52.825 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:52.825 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:06:52.825 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-09T23:06:52.825 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: monmap e2: 2 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} 2023-09-09T23:06:52.825 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: fsmap 2023-09-09T23:06:52.826 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: osdmap e4: 0 total, 0 up, 0 in 2023-09-09T23:06:52.826 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: mgrmap e13: x(active, since 40s) 2023-09-09T23:06:52.826 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: overall HEALTH_OK 2023-09-09T23:06:52.826 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:52.826 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:52.827 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:52.827 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-09T23:06:52.827 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:06:52.827 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: Deploying daemon mon.b on smithi098 2023-09-09T23:06:52.827 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:06:52.827 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: from='client.? 172.21.15.170:0/3496954253' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-09T23:06:52.828 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:52.828 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-09-09T23:06:52.828 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: expand_channel_meta expand map: {default=false} 2023-09-09T23:06:52.828 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: expand_channel_meta from 'false' to 'false' 2023-09-09T23:06:52.828 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: expand_channel_meta expanded map: {default=false} 2023-09-09T23:06:52.829 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: expand_channel_meta expand map: {default=info} 2023-09-09T23:06:52.829 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: expand_channel_meta from 'info' to 'info' 2023-09-09T23:06:52.829 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: expand_channel_meta expanded map: {default=info} 2023-09-09T23:06:52.829 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: expand_channel_meta expand map: {default=daemon} 2023-09-09T23:06:52.829 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: expand_channel_meta from 'daemon' to 'daemon' 2023-09-09T23:06:52.829 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: expand_channel_meta expanded map: {default=daemon} 2023-09-09T23:06:52.830 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: expand_channel_meta expand map: {default=debug} 2023-09-09T23:06:52.830 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: expand_channel_meta from 'debug' to 'debug' 2023-09-09T23:06:52.830 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: expand_channel_meta expanded map: {default=debug} 2023-09-09T23:06:52.830 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 ceph-mon[103124]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-09-09T23:06:52.830 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:52 smithi098 systemd[1]: Started Ceph mon.b for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da. 2023-09-09T23:06:57.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:57 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-09T23:06:57.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:57 smithi170 ceph-mon[102832]: mon.c calling monitor election 2023-09-09T23:06:57.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:57 smithi170 ceph-mon[102832]: mon.a calling monitor election 2023-09-09T23:06:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:57 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:06:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:57 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:06:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:57 smithi170 ceph-mon[102832]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:57 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:06:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:57 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:06:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:57 smithi170 ceph-mon[102832]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:57 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:06:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:57 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:06:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:57 smithi170 ceph-mon[102832]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:57 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:06:57.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:57 smithi170 ceph-mon[102832]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-09T23:06:57.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:57 smithi170 ceph-mon[102832]: monmap e3: 3 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],b=[v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} 2023-09-09T23:06:57.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:57 smithi170 ceph-mon[102832]: fsmap 2023-09-09T23:06:57.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:57 smithi170 ceph-mon[102832]: osdmap e4: 0 total, 0 up, 0 in 2023-09-09T23:06:57.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:57 smithi170 ceph-mon[102832]: mgrmap e13: x(active, since 48s) 2023-09-09T23:06:57.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:57 smithi170 ceph-mon[102832]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-09-09T23:06:57.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:57 smithi170 ceph-mon[102832]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-09-09T23:06:57.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:57 smithi170 ceph-mon[102832]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-09-09T23:06:57.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:57 smithi170 ceph-mon[102832]: mon.b (rank 2) addr [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] is down (out of quorum) 2023-09-09T23:06:57.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:57 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:57.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:57 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:57.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:57 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:57.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:57 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:57.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:57 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-09T23:06:57.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:57 smithi082 ceph-mon[98694]: mon.c calling monitor election 2023-09-09T23:06:57.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:57 smithi082 ceph-mon[98694]: mon.a calling monitor election 2023-09-09T23:06:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:57 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:06:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:57 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:06:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:57 smithi082 ceph-mon[98694]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:57 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:06:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:57 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:06:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:57 smithi082 ceph-mon[98694]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:57 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:06:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:57 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:06:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:57 smithi082 ceph-mon[98694]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:57 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:06:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:57 smithi082 ceph-mon[98694]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-09T23:06:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:57 smithi082 ceph-mon[98694]: monmap e3: 3 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],b=[v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} 2023-09-09T23:06:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:57 smithi082 ceph-mon[98694]: fsmap 2023-09-09T23:06:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:57 smithi082 ceph-mon[98694]: osdmap e4: 0 total, 0 up, 0 in 2023-09-09T23:06:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:57 smithi082 ceph-mon[98694]: mgrmap e13: x(active, since 48s) 2023-09-09T23:06:57.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:57 smithi082 ceph-mon[98694]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-09-09T23:06:57.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:57 smithi082 ceph-mon[98694]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-09-09T23:06:57.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:57 smithi082 ceph-mon[98694]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-09-09T23:06:57.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:57 smithi082 ceph-mon[98694]: mon.b (rank 2) addr [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] is down (out of quorum) 2023-09-09T23:06:57.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:57 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:57.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:57 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:57.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:57 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:57.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:57 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:58.273 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-09-09T23:06:58.273 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":3,"fsid":"3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da","modified":"2023-09-09T23:06:52.452852Z","created":"2023-09-09T23:05:10.079367Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:3300","nonce":0},{"type":"v1","addr":"172.21.15.82:6789","nonce":0}]},"addr":"172.21.15.82:6789/0","public_addr":"172.21.15.82:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:3300","nonce":0},{"type":"v1","addr":"172.21.15.170:6789","nonce":0}]},"addr":"172.21.15.170:6789/0","public_addr":"172.21.15.170:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:3300","nonce":0},{"type":"v1","addr":"172.21.15.98:6789","nonce":0}]},"addr":"172.21.15.98:6789/0","public_addr":"172.21.15.98:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-09-09T23:06:58.275 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 3 2023-09-09T23:06:58.567 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:58 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:06:58.568 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:58 smithi170 ceph-mon[102832]: from='client.? 172.21.15.170:0/2733082560' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-09T23:06:58.568 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:06:58 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:06:58.664 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-09-09T23:06:58.664 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph config generate-minimal-conf 2023-09-09T23:06:58.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:58 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:06:58.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:58 smithi082 ceph-mon[98694]: from='client.? 172.21.15.170:0/2733082560' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-09T23:06:58.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:06:58 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:06:59.233 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:06:59.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-09T23:06:59.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: mon.c calling monitor election 2023-09-09T23:06:59.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: mon.a calling monitor election 2023-09-09T23:06:59.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:06:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:06:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:06:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:06:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:06:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:06:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:06:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:06:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-09T23:06:59.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: monmap e3: 3 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],b=[v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} 2023-09-09T23:06:59.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: fsmap 2023-09-09T23:06:59.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: osdmap e4: 0 total, 0 up, 0 in 2023-09-09T23:06:59.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: mgrmap e13: x(active, since 48s) 2023-09-09T23:06:59.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-09-09T23:06:59.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-09-09T23:06:59.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-09-09T23:06:59.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: mon.b (rank 2) addr [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] is down (out of quorum) 2023-09-09T23:06:59.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:59.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:59.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:59.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:06:59.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:06:59.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: from='client.? 172.21.15.170:0/2733082560' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-09T23:06:59.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:06:59 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:07:00.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: mon.b calling monitor election 2023-09-09T23:07:00.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: Updating smithi082:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-09T23:07:00.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: Updating smithi098:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-09T23:07:00.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: Updating smithi170:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-09T23:07:00.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: mon.b calling monitor election 2023-09-09T23:07:00.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: mon.c calling monitor election 2023-09-09T23:07:00.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: mon.a calling monitor election 2023-09-09T23:07:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-09-09T23:07:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: monmap e3: 3 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],b=[v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} 2023-09-09T23:07:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: fsmap 2023-09-09T23:07:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: osdmap e4: 0 total, 0 up, 0 in 2023-09-09T23:07:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: mgrmap e13: x(active, since 50s) 2023-09-09T23:07:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-09-09T23:07:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: Cluster is now healthy 2023-09-09T23:07:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: overall HEALTH_OK 2023-09-09T23:07:00.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: Reconfiguring mon.a (unknown last config time)... 2023-09-09T23:07:00.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-09T23:07:00.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-09T23:07:00.462 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:00.462 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:00 smithi082 ceph-mon[98694]: Reconfiguring daemon mon.a on smithi082 2023-09-09T23:07:00.637 INFO:teuthology.orchestra.run.smithi082.stdout:# minimal ceph.conf for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da 2023-09-09T23:07:00.638 INFO:teuthology.orchestra.run.smithi082.stdout:[global] 2023-09-09T23:07:00.638 INFO:teuthology.orchestra.run.smithi082.stdout: fsid = 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da 2023-09-09T23:07:00.638 INFO:teuthology.orchestra.run.smithi082.stdout: mon_host = [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] 2023-09-09T23:07:00.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: mon.b calling monitor election 2023-09-09T23:07:00.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: Updating smithi082:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-09T23:07:00.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: Updating smithi098:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-09T23:07:00.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: Updating smithi170:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-09T23:07:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: mon.b calling monitor election 2023-09-09T23:07:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: mon.c calling monitor election 2023-09-09T23:07:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: mon.a calling monitor election 2023-09-09T23:07:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-09-09T23:07:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: monmap e3: 3 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],b=[v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} 2023-09-09T23:07:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: fsmap 2023-09-09T23:07:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: osdmap e4: 0 total, 0 up, 0 in 2023-09-09T23:07:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: mgrmap e13: x(active, since 50s) 2023-09-09T23:07:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-09-09T23:07:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: Cluster is now healthy 2023-09-09T23:07:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: overall HEALTH_OK 2023-09-09T23:07:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: Reconfiguring mon.a (unknown last config time)... 2023-09-09T23:07:00.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-09T23:07:00.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-09T23:07:00.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:00.682 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:00 smithi170 ceph-mon[102832]: Reconfiguring daemon mon.a on smithi082 2023-09-09T23:07:00.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: mon.b calling monitor election 2023-09-09T23:07:00.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: Updating smithi082:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-09T23:07:00.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: Updating smithi098:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-09T23:07:00.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: Updating smithi170:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-09T23:07:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: mon.b calling monitor election 2023-09-09T23:07:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: mon.c calling monitor election 2023-09-09T23:07:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: mon.a calling monitor election 2023-09-09T23:07:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-09-09T23:07:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: monmap e3: 3 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],b=[v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} 2023-09-09T23:07:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: fsmap 2023-09-09T23:07:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: osdmap e4: 0 total, 0 up, 0 in 2023-09-09T23:07:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: mgrmap e13: x(active, since 50s) 2023-09-09T23:07:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-09-09T23:07:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: Cluster is now healthy 2023-09-09T23:07:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: overall HEALTH_OK 2023-09-09T23:07:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:00.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: Reconfiguring mon.a (unknown last config time)... 2023-09-09T23:07:00.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-09T23:07:00.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-09T23:07:00.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:00.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:00 smithi098 ceph-mon[103124]: Reconfiguring daemon mon.a on smithi082 2023-09-09T23:07:01.079 INFO:tasks.cephadm:['ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'] 2023-09-09T23:07:01.080 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph config set client client_mount_timeout 600 2023-09-09T23:07:01.082 INFO:tasks.cephadm:['ceph', 'config', 'set', 'client', 'debug_client', '20'] 2023-09-09T23:07:01.082 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph config set client debug_client 20 2023-09-09T23:07:01.125 INFO:tasks.cephadm:['ceph', 'config', 'set', 'client', 'debug_ms', '1'] 2023-09-09T23:07:01.125 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph config set client debug_ms 1 2023-09-09T23:07:01.168 INFO:tasks.cephadm:['ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'] 2023-09-09T23:07:01.169 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph config set client rados_mon_op_timeout 900 2023-09-09T23:07:01.211 INFO:tasks.cephadm:['ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'] 2023-09-09T23:07:01.211 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph config set client rados_osd_op_timeout 900 2023-09-09T23:07:01.254 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'debug_mds', '20'] 2023-09-09T23:07:01.255 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph config set mds debug_mds 20 2023-09-09T23:07:01.298 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'] 2023-09-09T23:07:01.299 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph config set mds debug_mds_balancer 20 2023-09-09T23:07:01.341 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'debug_ms', '1'] 2023-09-09T23:07:01.342 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph config set mds debug_ms 1 2023-09-09T23:07:01.385 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:01 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:07:01.385 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:01 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/4292385462' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:01.386 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:01 smithi082 ceph-mon[98694]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:01.386 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:01 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:01.386 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:01 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:01.387 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:01 smithi082 ceph-mon[98694]: Reconfiguring mon.b (monmap changed)... 2023-09-09T23:07:01.387 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:01 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-09T23:07:01.387 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:01 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-09T23:07:01.387 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:01 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:01.388 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:01 smithi082 ceph-mon[98694]: Reconfiguring daemon mon.b on smithi098 2023-09-09T23:07:01.389 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'] 2023-09-09T23:07:01.390 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_bal_fragment_size_max 10000 2023-09-09T23:07:01.435 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'] 2023-09-09T23:07:01.436 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_bal_merge_size 5 2023-09-09T23:07:01.484 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'] 2023-09-09T23:07:01.485 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_bal_split_bits 3 2023-09-09T23:07:01.534 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'] 2023-09-09T23:07:01.535 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_bal_split_size 100 2023-09-09T23:07:01.584 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'] 2023-09-09T23:07:01.585 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_debug_frag True 2023-09-09T23:07:01.632 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'] 2023-09-09T23:07:01.632 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_debug_scatterstat True 2023-09-09T23:07:01.633 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:01 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:07:01.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:01 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/4292385462' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:01.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:01 smithi098 ceph-mon[103124]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:01.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:01 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:01.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:01 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:01.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:01 smithi098 ceph-mon[103124]: Reconfiguring mon.b (monmap changed)... 2023-09-09T23:07:01.635 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:01 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-09T23:07:01.635 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:01 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-09T23:07:01.635 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:01 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:01.635 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:01 smithi098 ceph-mon[103124]: Reconfiguring daemon mon.b on smithi098 2023-09-09T23:07:01.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:01 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:07:01.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:01 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/4292385462' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:01.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:01 smithi170 ceph-mon[102832]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:01.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:01 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:01.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:01 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:01.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:01 smithi170 ceph-mon[102832]: Reconfiguring mon.b (monmap changed)... 2023-09-09T23:07:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:01 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-09T23:07:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:01 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-09T23:07:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:01 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:01 smithi170 ceph-mon[102832]: Reconfiguring daemon mon.b on smithi098 2023-09-09T23:07:01.690 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'] 2023-09-09T23:07:01.690 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_op_complaint_time 180 2023-09-09T23:07:01.743 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'] 2023-09-09T23:07:01.743 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph config set mds mds_verify_scatter True 2023-09-09T23:07:01.744 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:07:01.790 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'] 2023-09-09T23:07:01.791 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph config set mds osd_op_complaint_time 180 2023-09-09T23:07:01.831 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:07:01.833 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'] 2023-09-09T23:07:01.833 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph config set mds rados_mon_op_timeout 900 2023-09-09T23:07:01.879 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'] 2023-09-09T23:07:01.880 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph config set mds rados_osd_op_timeout 900 2023-09-09T23:07:01.922 INFO:tasks.cephadm:['ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'] 2023-09-09T23:07:01.923 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph config set mon mon_op_complaint_time 120 2023-09-09T23:07:01.969 INFO:tasks.cephadm:['ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'] 2023-09-09T23:07:01.969 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph config set osd osd_op_complaint_time 180 2023-09-09T23:07:02.004 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:07:02.006 DEBUG:tasks.cephadm:set 21 configs 2023-09-09T23:07:02.007 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:07:02.142 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:07:02.948 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:07:02.949 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:07:02.950 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:07:03.390 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:03 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:03.390 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:03 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:03.391 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:03 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-09T23:07:03.391 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:03 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-09T23:07:03.391 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:03 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:03.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:03 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:03.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:03 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:03.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:03 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-09T23:07:03.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:03 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-09T23:07:03.472 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:03 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:03.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:03 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:03.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:03 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:03.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:03 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-09T23:07:03.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:03 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-09T23:07:03.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:03 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:03.571 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:07:04.298 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:04 smithi082 ceph-mon[98694]: Reconfiguring mon.c (monmap changed)... 2023-09-09T23:07:04.298 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:04 smithi082 ceph-mon[98694]: Reconfiguring daemon mon.c on smithi170 2023-09-09T23:07:04.299 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:04 smithi082 ceph-mon[98694]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:04.299 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:04 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:04.299 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:04 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:04.299 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:04 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:07:04.299 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:04 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:04.299 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:04 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:07:04.300 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:04 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:04.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:04 smithi098 ceph-mon[103124]: Reconfiguring mon.c (monmap changed)... 2023-09-09T23:07:04.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:04 smithi098 ceph-mon[103124]: Reconfiguring daemon mon.c on smithi170 2023-09-09T23:07:04.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:04 smithi098 ceph-mon[103124]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:04.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:04 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:04.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:04 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:04 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:07:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:04 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:04 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:07:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:04 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:04.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:04 smithi170 ceph-mon[102832]: Reconfiguring mon.c (monmap changed)... 2023-09-09T23:07:04.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:04 smithi170 ceph-mon[102832]: Reconfiguring daemon mon.c on smithi170 2023-09-09T23:07:04.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:04 smithi170 ceph-mon[102832]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:04.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:04 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:04.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:04 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:04 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:07:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:04 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:04 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:07:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:04 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:05.202 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:07:05.203 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:07:05.925 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:07:06.172 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:07:06.431 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:07:06.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:06 smithi082 ceph-mon[98694]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:06.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:06 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3872110053' entity='client.admin' 2023-09-09T23:07:06.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:06 smithi098 ceph-mon[103124]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:06.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:06 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3872110053' entity='client.admin' 2023-09-09T23:07:06.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:06 smithi170 ceph-mon[102832]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:06.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:06 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3872110053' entity='client.admin' 2023-09-09T23:07:08.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:07 smithi082 ceph-mon[98694]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:08.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:07 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3508408746' entity='client.admin' 2023-09-09T23:07:08.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:07 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/324274240' entity='client.admin' 2023-09-09T23:07:08.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:07 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3334810551' entity='client.admin' 2023-09-09T23:07:08.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:07 smithi098 ceph-mon[103124]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:08.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:07 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3508408746' entity='client.admin' 2023-09-09T23:07:08.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:07 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/324274240' entity='client.admin' 2023-09-09T23:07:08.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:07 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3334810551' entity='client.admin' 2023-09-09T23:07:08.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:07 smithi170 ceph-mon[102832]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:08.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:07 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3508408746' entity='client.admin' 2023-09-09T23:07:08.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:07 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/324274240' entity='client.admin' 2023-09-09T23:07:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:07 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3334810551' entity='client.admin' 2023-09-09T23:07:08.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:08.480+0000 7ff21a8d7700 1 -- 172.21.15.82:0/2154237427 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff1fc038630 msgr2=0x7ff1fc03aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:08.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:08.480+0000 7ff21a8d7700 1 --2- 172.21.15.82:0/2154237427 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff1fc038630 0x7ff1fc03aaf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff20400b5c0 tx=0x7ff204005960 comp rx=0 tx=0).stop 2023-09-09T23:07:08.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:08.481+0000 7ff21a8d7700 1 -- 172.21.15.82:0/2154237427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2140ff760 msgr2=0x7ff214196a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:08.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:08.481+0000 7ff21a8d7700 1 --2- 172.21.15.82:0/2154237427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2140ff760 0x7ff214196a60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff20800d950 tx=0x7ff20800dc60 comp rx=0 tx=0).stop 2023-09-09T23:07:08.481 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:08.482+0000 7ff21a8d7700 1 -- 172.21.15.82:0/2154237427 shutdown_connections 2023-09-09T23:07:08.481 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:08.482+0000 7ff21a8d7700 1 --2- 172.21.15.82:0/2154237427 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff1fc038630 0x7ff1fc03aaf0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:08.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:08.482+0000 7ff21a8d7700 1 --2- 172.21.15.82:0/2154237427 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff214105950 0x7ff21419aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:08.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:08.482+0000 7ff21a8d7700 1 --2- 172.21.15.82:0/2154237427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2140fedb0 0x7ff214196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:08.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:08.482+0000 7ff21a8d7700 1 --2- 172.21.15.82:0/2154237427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2140ff760 0x7ff214196a60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:08.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:08.482+0000 7ff21a8d7700 1 -- 172.21.15.82:0/2154237427 >> 172.21.15.82:0/2154237427 conn(0x7ff214075a70 msgr2=0x7ff214103800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:08.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:08.482+0000 7ff21a8d7700 1 -- 172.21.15.82:0/2154237427 shutdown_connections 2023-09-09T23:07:08.483 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:08.482+0000 7ff21a8d7700 1 -- 172.21.15.82:0/2154237427 wait complete. 2023-09-09T23:07:08.521 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:08.520+0000 7f627d20b700 1 -- 172.21.15.82:0/3255648286 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) v1 -- 0x7f627819ad30 con 0x7f6278101840 2023-09-09T23:07:08.524 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:08.525+0000 7f625ffff700 1 -- 172.21.15.82:0/3255648286 <== mon.2 v2:172.21.15.98:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v16)=0 v16) v1 ==== 106+0+0 (secure 0 0 0) 0x7f627001cbb0 con 0x7f6278101840 2023-09-09T23:07:08.526 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:08.527+0000 7f627d20b700 1 -- 172.21.15.82:0/3255648286 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f6260049640 msgr2=0x7f626004bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:08.526 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:08.527+0000 7f627d20b700 1 --2- 172.21.15.82:0/3255648286 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f6260049640 0x7f626004bb00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f626800b2e0 tx=0x7f6268005fd0 comp rx=0 tx=0).stop 2023-09-09T23:07:08.526 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:08.527+0000 7f627d20b700 1 -- 172.21.15.82:0/3255648286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6278101840 msgr2=0x7f6278199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:08.527 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:08.527+0000 7f627d20b700 1 --2- 172.21.15.82:0/3255648286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6278101840 0x7f6278199f50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f627000ef30 tx=0x7f627000c3a0 comp rx=0 tx=0).stop 2023-09-09T23:07:08.527 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:08.527+0000 7f627d20b700 1 -- 172.21.15.82:0/3255648286 shutdown_connections 2023-09-09T23:07:08.527 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:08.527+0000 7f627d20b700 1 --2- 172.21.15.82:0/3255648286 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f6260049640 0x7f626004bb00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:08.527 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:08.527+0000 7f627d20b700 1 --2- 172.21.15.82:0/3255648286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6278101840 0x7f6278199f50 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:08.527 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:08.527+0000 7f627d20b700 1 --2- 172.21.15.82:0/3255648286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62780ff880 0x7f62781955a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:08.528 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:08.527+0000 7f627d20b700 1 --2- 172.21.15.82:0/3255648286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62781001a0 0x7f6278195ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:08.528 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:08.527+0000 7f627d20b700 1 -- 172.21.15.82:0/3255648286 >> 172.21.15.82:0/3255648286 conn(0x7f62780765f0 msgr2=0x7f62780fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:08.528 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:08.527+0000 7f627d20b700 1 -- 172.21.15.82:0/3255648286 shutdown_connections 2023-09-09T23:07:08.528 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:08.527+0000 7f627d20b700 1 -- 172.21.15.82:0/3255648286 wait complete. 2023-09-09T23:07:09.295 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:09 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3198933057' entity='client.admin' 2023-09-09T23:07:09.295 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:09 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2154237427' entity='client.admin' 2023-09-09T23:07:09.296 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:09 smithi082 ceph-mon[98694]: from='client.? ' entity='client.admin' 2023-09-09T23:07:09.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:09 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3198933057' entity='client.admin' 2023-09-09T23:07:09.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:09 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2154237427' entity='client.admin' 2023-09-09T23:07:09.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:09 smithi098 ceph-mon[103124]: from='client.? ' entity='client.admin' 2023-09-09T23:07:09.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:09 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3198933057' entity='client.admin' 2023-09-09T23:07:09.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:09 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2154237427' entity='client.admin' 2023-09-09T23:07:09.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:09 smithi170 ceph-mon[102832]: from='client.? ' entity='client.admin' 2023-09-09T23:07:10.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:10 smithi098 ceph-mon[103124]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:10.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.581+0000 7fad1879c700 1 -- 172.21.15.82:0/1240601588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad10101840 msgr2=0x7fad1010c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:10.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.581+0000 7fad1879c700 1 --2- 172.21.15.82:0/1240601588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad10101840 0x7fad1010c4d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fad0800b0a0 tx=0x7fad0800b3b0 comp rx=0 tx=0).stop 2023-09-09T23:07:10.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.581+0000 7fad1879c700 1 -- 172.21.15.82:0/1240601588 shutdown_connections 2023-09-09T23:07:10.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.581+0000 7fad1879c700 1 --2- 172.21.15.82:0/1240601588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad10101840 0x7fad1010c4d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:10.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.581+0000 7fad1879c700 1 --2- 172.21.15.82:0/1240601588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad100ff880 0x7fad100ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:10.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.581+0000 7fad1879c700 1 --2- 172.21.15.82:0/1240601588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad101001a0 0x7fad10101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:10.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.581+0000 7fad1879c700 1 -- 172.21.15.82:0/1240601588 >> 172.21.15.82:0/1240601588 conn(0x7fad100765f0 msgr2=0x7fad10076a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:10.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.581+0000 7fad1879c700 1 -- 172.21.15.82:0/1240601588 shutdown_connections 2023-09-09T23:07:10.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.581+0000 7fad1879c700 1 -- 172.21.15.82:0/1240601588 wait complete. 2023-09-09T23:07:10.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.581+0000 7fad1879c700 1 Processor -- start 2023-09-09T23:07:10.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.581+0000 7fad1879c700 1 -- start start 2023-09-09T23:07:10.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.582+0000 7fad1879c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad100ff880 0x7fad1019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:10.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.582+0000 7fad1879c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad101001a0 0x7fad1019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:10.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.582+0000 7fad1879c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad10101840 0x7fad10196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:10.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.582+0000 7fad1879c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad10196dc0 con 0x7fad100ff880 2023-09-09T23:07:10.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.582+0000 7fad1879c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad10196f30 con 0x7fad101001a0 2023-09-09T23:07:10.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.582+0000 7fad1879c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad101970f0 con 0x7fad10101840 2023-09-09T23:07:10.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.582+0000 7fad16538700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad100ff880 0x7fad1019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:10.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.582+0000 7fad16538700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad100ff880 0x7fad1019c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53338/0 (socket says 172.21.15.82:53338) 2023-09-09T23:07:10.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.582+0000 7fad16538700 1 -- 172.21.15.82:0/1848563690 learned_addr learned my addr 172.21.15.82:0/1848563690 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:07:10.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.582+0000 7fad16d39700 1 --2- 172.21.15.82:0/1848563690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad10101840 0x7fad10196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:10.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.582+0000 7fad16538700 1 -- 172.21.15.82:0/1848563690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad10101840 msgr2=0x7fad10196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:10.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.582+0000 7fad15d37700 1 --2- 172.21.15.82:0/1848563690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad101001a0 0x7fad1019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:10.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.582+0000 7fad16538700 1 --2- 172.21.15.82:0/1848563690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad10101840 0x7fad10196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:10.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.582+0000 7fad16538700 1 -- 172.21.15.82:0/1848563690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad101001a0 msgr2=0x7fad1019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:10.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.582+0000 7fad16538700 1 --2- 172.21.15.82:0/1848563690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad101001a0 0x7fad1019cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:10.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.582+0000 7fad16538700 1 -- 172.21.15.82:0/1848563690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad08009d00 con 0x7fad100ff880 2023-09-09T23:07:10.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.582+0000 7fad16538700 1 --2- 172.21.15.82:0/1848563690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad100ff880 0x7fad1019c6d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00b530 tx=0x7fad0c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:10.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.582+0000 7fad077fe700 1 -- 172.21.15.82:0/1848563690 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c011830 con 0x7fad100ff880 2023-09-09T23:07:10.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.582+0000 7fad077fe700 1 -- 172.21.15.82:0/1848563690 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad0c011e50 con 0x7fad100ff880 2023-09-09T23:07:10.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.582+0000 7fad077fe700 1 -- 172.21.15.82:0/1848563690 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c00f2e0 con 0x7fad100ff880 2023-09-09T23:07:10.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.582+0000 7fad1879c700 1 -- 172.21.15.82:0/1848563690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad101973d0 con 0x7fad100ff880 2023-09-09T23:07:10.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.582+0000 7fad1879c700 1 -- 172.21.15.82:0/1848563690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad101a3740 con 0x7fad100ff880 2023-09-09T23:07:10.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.583+0000 7fad077fe700 1 -- 172.21.15.82:0/1848563690 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7fad0c011990 con 0x7fad100ff880 2023-09-09T23:07:10.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.583+0000 7fad1879c700 1 -- 172.21.15.82:0/1848563690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad1004e300 con 0x7fad100ff880 2023-09-09T23:07:10.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.583+0000 7fad077fe700 1 --2- 172.21.15.82:0/1848563690 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7facfc038630 0x7facfc03aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:10.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.584+0000 7fad077fe700 1 -- 172.21.15.82:0/1848563690 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fad0c04a900 con 0x7fad100ff880 2023-09-09T23:07:10.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.584+0000 7fad15d37700 1 --2- 172.21.15.82:0/1848563690 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7facfc038630 0x7facfc03aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:10.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.584+0000 7fad15d37700 1 --2- 172.21.15.82:0/1848563690 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7facfc038630 0x7facfc03aaf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fad10197bc0 tx=0x7fad00005cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:10.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.588+0000 7fad077fe700 1 -- 172.21.15.82:0/1848563690 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fad0c021950 con 0x7fad100ff880 2023-09-09T23:07:10.594 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:07:10.621 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:07:10.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:10 smithi170 ceph-mon[102832]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:10.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:10 smithi082 ceph-mon[98694]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:10.806 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.807+0000 7fad1879c700 1 -- 172.21.15.82:0/1848563690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) v1 -- 0x7fad10197c60 con 0x7fad100ff880 2023-09-09T23:07:10.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.810+0000 7fad077fe700 1 -- 172.21.15.82:0/1848563690 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v17)=0 v17) v1 ==== 116+0+0 (secure 0 0 0) 0x7fad0c015030 con 0x7fad100ff880 2023-09-09T23:07:10.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.813+0000 7fad1879c700 1 -- 172.21.15.82:0/1848563690 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7facfc038630 msgr2=0x7facfc03aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:10.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.813+0000 7fad1879c700 1 --2- 172.21.15.82:0/1848563690 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7facfc038630 0x7facfc03aaf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fad10197bc0 tx=0x7fad00005cf0 comp rx=0 tx=0).stop 2023-09-09T23:07:10.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.813+0000 7fad1879c700 1 -- 172.21.15.82:0/1848563690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad100ff880 msgr2=0x7fad1019c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:10.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.813+0000 7fad1879c700 1 --2- 172.21.15.82:0/1848563690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad100ff880 0x7fad1019c6d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00b530 tx=0x7fad0c00b8f0 comp rx=0 tx=0).stop 2023-09-09T23:07:10.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.813+0000 7fad1879c700 1 -- 172.21.15.82:0/1848563690 shutdown_connections 2023-09-09T23:07:10.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.813+0000 7fad1879c700 1 --2- 172.21.15.82:0/1848563690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad10101840 0x7fad10196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:10.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.813+0000 7fad1879c700 1 --2- 172.21.15.82:0/1848563690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad100ff880 0x7fad1019c6d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:10.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.813+0000 7fad1879c700 1 --2- 172.21.15.82:0/1848563690 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7facfc038630 0x7facfc03aaf0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:10.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.813+0000 7fad1879c700 1 --2- 172.21.15.82:0/1848563690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad101001a0 0x7fad1019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:10.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.813+0000 7fad1879c700 1 -- 172.21.15.82:0/1848563690 >> 172.21.15.82:0/1848563690 conn(0x7fad100765f0 msgr2=0x7fad10104640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:10.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.813+0000 7fad1879c700 1 -- 172.21.15.82:0/1848563690 shutdown_connections 2023-09-09T23:07:10.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:10.813+0000 7fad1879c700 1 -- 172.21.15.82:0/1848563690 wait complete. 2023-09-09T23:07:11.457 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:07:11.464 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:07:11.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.580+0000 7f8cd7460700 1 -- 172.21.15.82:0/1291494569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cd0100fe0 msgr2=0x7f8cd010d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:11.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.580+0000 7f8cd7460700 1 --2- 172.21.15.82:0/1291494569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cd0100fe0 0x7f8cd010d4f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8cc800b0a0 tx=0x7f8cc800b3b0 comp rx=0 tx=0).stop 2023-09-09T23:07:11.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.580+0000 7f8cd7460700 1 -- 172.21.15.82:0/1291494569 shutdown_connections 2023-09-09T23:07:11.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.580+0000 7f8cd7460700 1 --2- 172.21.15.82:0/1291494569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cd0100fe0 0x7f8cd010d4f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:11.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.580+0000 7f8cd7460700 1 --2- 172.21.15.82:0/1291494569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cd0106620 0x7f8cd0106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:11.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.580+0000 7f8cd7460700 1 --2- 172.21.15.82:0/1291494569 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8cd0100620 0x7f8cd0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:11.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.580+0000 7f8cd7460700 1 -- 172.21.15.82:0/1291494569 >> 172.21.15.82:0/1291494569 conn(0x7f8cd00fc0f0 msgr2=0x7f8cd00fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:11.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.580+0000 7f8cd7460700 1 -- 172.21.15.82:0/1291494569 shutdown_connections 2023-09-09T23:07:11.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.580+0000 7f8cd7460700 1 -- 172.21.15.82:0/1291494569 wait complete. 2023-09-09T23:07:11.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.581+0000 7f8cd7460700 1 Processor -- start 2023-09-09T23:07:11.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.581+0000 7f8cd7460700 1 -- start start 2023-09-09T23:07:11.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.581+0000 7f8cd7460700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cd0100620 0x7f8cd019c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:11.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.581+0000 7f8cd7460700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cd0100fe0 0x7f8cd019cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:11.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.581+0000 7f8cd7460700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8cd0106620 0x7f8cd0196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:11.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.581+0000 7f8cd7460700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd0196c80 con 0x7f8cd0100620 2023-09-09T23:07:11.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.581+0000 7f8cd7460700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd0196df0 con 0x7f8cd0106620 2023-09-09T23:07:11.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.581+0000 7f8cd7460700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd0196fb0 con 0x7f8cd0100fe0 2023-09-09T23:07:11.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.581+0000 7f8cd51fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cd0100620 0x7f8cd019c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:11.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.581+0000 7f8cd51fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cd0100620 0x7f8cd019c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53350/0 (socket says 172.21.15.82:53350) 2023-09-09T23:07:11.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.581+0000 7f8cd51fc700 1 -- 172.21.15.82:0/2858897293 learned_addr learned my addr 172.21.15.82:0/2858897293 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:07:11.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.582+0000 7f8cd59fd700 1 --2- 172.21.15.82:0/2858897293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8cd0106620 0x7f8cd0196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:11.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.582+0000 7f8cd49fb700 1 --2- 172.21.15.82:0/2858897293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cd0100fe0 0x7f8cd019cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:11.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.582+0000 7f8cd51fc700 1 -- 172.21.15.82:0/2858897293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cd0100fe0 msgr2=0x7f8cd019cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:11.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.582+0000 7f8cd51fc700 1 --2- 172.21.15.82:0/2858897293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cd0100fe0 0x7f8cd019cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:11.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.582+0000 7f8cd51fc700 1 -- 172.21.15.82:0/2858897293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8cd0106620 msgr2=0x7f8cd0196610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:11.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.582+0000 7f8cd51fc700 1 --2- 172.21.15.82:0/2858897293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8cd0106620 0x7f8cd0196610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:11.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.582+0000 7f8cd51fc700 1 -- 172.21.15.82:0/2858897293 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc8009d00 con 0x7f8cd0100620 2023-09-09T23:07:11.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.582+0000 7f8cd49fb700 1 --2- 172.21.15.82:0/2858897293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cd0100fe0 0x7f8cd019cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:07:11.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.582+0000 7f8cd51fc700 1 --2- 172.21.15.82:0/2858897293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cd0100620 0x7f8cd019c590 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00c9b0 tx=0x7f8ccc00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:11.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.582+0000 7f8cc67fc700 1 -- 172.21.15.82:0/2858897293 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc007b00 con 0x7f8cd0100620 2023-09-09T23:07:11.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.582+0000 7f8cc67fc700 1 -- 172.21.15.82:0/2858897293 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ccc007c60 con 0x7f8cd0100620 2023-09-09T23:07:11.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.582+0000 7f8cd7460700 1 -- 172.21.15.82:0/2858897293 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd0197290 con 0x7f8cd0100620 2023-09-09T23:07:11.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.582+0000 7f8cc67fc700 1 -- 172.21.15.82:0/2858897293 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc018660 con 0x7f8cd0100620 2023-09-09T23:07:11.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.582+0000 7f8cd7460700 1 -- 172.21.15.82:0/2858897293 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd01a3580 con 0x7f8cd0100620 2023-09-09T23:07:11.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.583+0000 7f8cd7460700 1 -- 172.21.15.82:0/2858897293 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cd004e300 con 0x7f8cd0100620 2023-09-09T23:07:11.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.583+0000 7f8cc67fc700 1 -- 172.21.15.82:0/2858897293 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f8ccc018880 con 0x7f8cd0100620 2023-09-09T23:07:11.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.583+0000 7f8cc67fc700 1 --2- 172.21.15.82:0/2858897293 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f8cbc038630 0x7f8cbc03aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:11.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.583+0000 7f8cc67fc700 1 -- 172.21.15.82:0/2858897293 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8ccc04ae00 con 0x7f8cd0100620 2023-09-09T23:07:11.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.584+0000 7f8cd49fb700 1 --2- 172.21.15.82:0/2858897293 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f8cbc038630 0x7f8cbc03aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:11.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.584+0000 7f8cd49fb700 1 --2- 172.21.15.82:0/2858897293 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f8cbc038630 0x7f8cbc03aaf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0005fd0 tx=0x7f8cc0005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:11.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.592+0000 7f8cc67fc700 1 -- 172.21.15.82:0/2858897293 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8ccc00db80 con 0x7f8cd0100620 2023-09-09T23:07:11.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.655+0000 7f68cab41700 1 -- 172.21.15.82:0/3864124475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68c4106770 msgr2=0x7f68c4106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:11.659 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.655+0000 7f68cab41700 1 --2- 172.21.15.82:0/3864124475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68c4106770 0x7f68c4106b50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009990 tx=0x7f68b8009ca0 comp rx=0 tx=0).stop 2023-09-09T23:07:11.659 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.656+0000 7f68cab41700 1 -- 172.21.15.82:0/3864124475 shutdown_connections 2023-09-09T23:07:11.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.656+0000 7f68cab41700 1 --2- 172.21.15.82:0/3864124475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68c4068ae0 0x7f68c410d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:11.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.656+0000 7f68cab41700 1 --2- 172.21.15.82:0/3864124475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68c4106770 0x7f68c4106b50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:11.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.656+0000 7f68cab41700 1 --2- 172.21.15.82:0/3864124475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68c4068120 0x7f68c40685a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:11.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.656+0000 7f68cab41700 1 -- 172.21.15.82:0/3864124475 >> 172.21.15.82:0/3864124475 conn(0x7f68c4076600 msgr2=0x7f68c4076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:11.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.656+0000 7f68cab41700 1 -- 172.21.15.82:0/3864124475 shutdown_connections 2023-09-09T23:07:11.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.656+0000 7f68cab41700 1 -- 172.21.15.82:0/3864124475 wait complete. 2023-09-09T23:07:11.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.656+0000 7f68cab41700 1 Processor -- start 2023-09-09T23:07:11.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.656+0000 7f68cab41700 1 -- start start 2023-09-09T23:07:11.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.656+0000 7f68cab41700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68c4068120 0x7f68c40ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:11.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.656+0000 7f68cab41700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68c4068ae0 0x7f68c40ffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:11.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.656+0000 7f68cab41700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68c4106770 0x7f68c41a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:11.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.656+0000 7f68cab41700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c41003e0 con 0x7f68c4068ae0 2023-09-09T23:07:11.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.656+0000 7f68cab41700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c41a4df0 con 0x7f68c4068120 2023-09-09T23:07:11.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.656+0000 7f68cab41700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c41a4f60 con 0x7f68c4106770 2023-09-09T23:07:11.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.657+0000 7f68c3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68c4068ae0 0x7f68c40ffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:11.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.657+0000 7f68c88dd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68c4068120 0x7f68c40ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:11.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.657+0000 7f68c3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68c4068ae0 0x7f68c40ffaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53366/0 (socket says 172.21.15.82:53366) 2023-09-09T23:07:11.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.657+0000 7f68c3fff700 1 -- 172.21.15.82:0/859974193 learned_addr learned my addr 172.21.15.82:0/859974193 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:07:11.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.657+0000 7f68c90de700 1 --2- 172.21.15.82:0/859974193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68c4106770 0x7f68c41a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:11.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.657+0000 7f68c3fff700 1 -- 172.21.15.82:0/859974193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68c4106770 msgr2=0x7f68c41a48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:11.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.657+0000 7f68c3fff700 1 --2- 172.21.15.82:0/859974193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68c4106770 0x7f68c41a48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:11.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.657+0000 7f68c3fff700 1 -- 172.21.15.82:0/859974193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68c4068120 msgr2=0x7f68c40ff5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:11.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.657+0000 7f68c3fff700 1 --2- 172.21.15.82:0/859974193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68c4068120 0x7f68c40ff5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:11.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.657+0000 7f68c3fff700 1 -- 172.21.15.82:0/859974193 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b80095f0 con 0x7f68c4068ae0 2023-09-09T23:07:11.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.657+0000 7f68c3fff700 1 --2- 172.21.15.82:0/859974193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68c4068ae0 0x7f68c40ffaf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f68b000e9a0 tx=0x7f68b000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:11.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.657+0000 7f68c1ffb700 1 -- 172.21.15.82:0/859974193 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b0009790 con 0x7f68c4068ae0 2023-09-09T23:07:11.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.657+0000 7f68c1ffb700 1 -- 172.21.15.82:0/859974193 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68b000cbd0 con 0x7f68c4068ae0 2023-09-09T23:07:11.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.657+0000 7f68cab41700 1 -- 172.21.15.82:0/859974193 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c41a51e0 con 0x7f68c4068ae0 2023-09-09T23:07:11.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.657+0000 7f68c1ffb700 1 -- 172.21.15.82:0/859974193 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b0018680 con 0x7f68c4068ae0 2023-09-09T23:07:11.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.658+0000 7f68cab41700 1 -- 172.21.15.82:0/859974193 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c41a5730 con 0x7f68c4068ae0 2023-09-09T23:07:11.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.658+0000 7f68c1ffb700 1 -- 172.21.15.82:0/859974193 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f68b00187e0 con 0x7f68c4068ae0 2023-09-09T23:07:11.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.659+0000 7f68c1ffb700 1 --2- 172.21.15.82:0/859974193 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f68b4038630 0x7f68b403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:11.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.659+0000 7f68cab41700 1 -- 172.21.15.82:0/859974193 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c404e300 con 0x7f68c4068ae0 2023-09-09T23:07:11.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.659+0000 7f68c1ffb700 1 -- 172.21.15.82:0/859974193 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f68b0014070 con 0x7f68c4068ae0 2023-09-09T23:07:11.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.659+0000 7f68c88dd700 1 --2- 172.21.15.82:0/859974193 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f68b4038630 0x7f68b403aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:11.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.659+0000 7f68c88dd700 1 --2- 172.21.15.82:0/859974193 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f68b4038630 0x7f68b403aaf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f68b800b2e0 tx=0x7f68b8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:11.672 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.663+0000 7f68c1ffb700 1 -- 172.21.15.82:0/859974193 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f68b00173d0 con 0x7f68c4068ae0 2023-09-09T23:07:11.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.794+0000 7f8cd7460700 1 -- 172.21.15.82:0/2858897293 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) v1 -- 0x7f8cd01106a0 con 0x7f8cd0100620 2023-09-09T23:07:11.795 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.797+0000 7f8cc67fc700 1 -- 172.21.15.82:0/2858897293 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v18)=0 v18) v1 ==== 108+0+0 (secure 0 0 0) 0x7f8ccc00d950 con 0x7f8cd0100620 2023-09-09T23:07:11.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.799+0000 7f8cd7460700 1 -- 172.21.15.82:0/2858897293 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f8cbc038630 msgr2=0x7f8cbc03aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:11.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.799+0000 7f8cd7460700 1 --2- 172.21.15.82:0/2858897293 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f8cbc038630 0x7f8cbc03aaf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0005fd0 tx=0x7f8cc0005e20 comp rx=0 tx=0).stop 2023-09-09T23:07:11.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.799+0000 7f8cd7460700 1 -- 172.21.15.82:0/2858897293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cd0100620 msgr2=0x7f8cd019c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:11.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.799+0000 7f8cd7460700 1 --2- 172.21.15.82:0/2858897293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cd0100620 0x7f8cd019c590 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00c9b0 tx=0x7f8ccc00cd70 comp rx=0 tx=0).stop 2023-09-09T23:07:11.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.799+0000 7f8cd7460700 1 -- 172.21.15.82:0/2858897293 shutdown_connections 2023-09-09T23:07:11.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.799+0000 7f8cd7460700 1 --2- 172.21.15.82:0/2858897293 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f8cbc038630 0x7f8cbc03aaf0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:11.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.799+0000 7f8cd7460700 1 --2- 172.21.15.82:0/2858897293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8cd0106620 0x7f8cd0196610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:11.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.799+0000 7f8cd7460700 1 --2- 172.21.15.82:0/2858897293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cd0100620 0x7f8cd019c590 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:11.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.799+0000 7f8cd7460700 1 --2- 172.21.15.82:0/2858897293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cd0100fe0 0x7f8cd019cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:11.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.799+0000 7f8cd7460700 1 -- 172.21.15.82:0/2858897293 >> 172.21.15.82:0/2858897293 conn(0x7f8cd00fc0f0 msgr2=0x7f8cd01074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:11.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.799+0000 7f8cd7460700 1 -- 172.21.15.82:0/2858897293 shutdown_connections 2023-09-09T23:07:11.801 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.799+0000 7f8cd7460700 1 -- 172.21.15.82:0/2858897293 wait complete. 2023-09-09T23:07:11.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.868+0000 7f68cab41700 1 -- 172.21.15.82:0/859974193 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) v1 -- 0x7f68c4110760 con 0x7f68c4068ae0 2023-09-09T23:07:11.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.871+0000 7f68c1ffb700 1 -- 172.21.15.82:0/859974193 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v19)=0 v19) v1 ==== 122+0+0 (secure 0 0 0) 0x7f68b0029070 con 0x7f68c4068ae0 2023-09-09T23:07:11.874 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.875+0000 7f68cab41700 1 -- 172.21.15.82:0/859974193 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f68b4038630 msgr2=0x7f68b403aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:11.874 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.875+0000 7f68cab41700 1 --2- 172.21.15.82:0/859974193 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f68b4038630 0x7f68b403aaf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f68b800b2e0 tx=0x7f68b8005fd0 comp rx=0 tx=0).stop 2023-09-09T23:07:11.874 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.875+0000 7f68cab41700 1 -- 172.21.15.82:0/859974193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68c4068ae0 msgr2=0x7f68c40ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:11.874 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.875+0000 7f68cab41700 1 --2- 172.21.15.82:0/859974193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68c4068ae0 0x7f68c40ffaf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f68b000e9a0 tx=0x7f68b000ecb0 comp rx=0 tx=0).stop 2023-09-09T23:07:11.875 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.875+0000 7f68cab41700 1 -- 172.21.15.82:0/859974193 shutdown_connections 2023-09-09T23:07:11.875 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.875+0000 7f68cab41700 1 --2- 172.21.15.82:0/859974193 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f68b4038630 0x7f68b403aaf0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:11.875 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.875+0000 7f68cab41700 1 --2- 172.21.15.82:0/859974193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68c4068120 0x7f68c40ff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:11.875 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.875+0000 7f68cab41700 1 --2- 172.21.15.82:0/859974193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68c4106770 0x7f68c41a48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:11.875 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.875+0000 7f68cab41700 1 --2- 172.21.15.82:0/859974193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68c4068ae0 0x7f68c40ffaf0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:11.875 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.875+0000 7f68cab41700 1 -- 172.21.15.82:0/859974193 >> 172.21.15.82:0/859974193 conn(0x7f68c4076600 msgr2=0x7f68c40fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:11.876 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.875+0000 7f68cab41700 1 -- 172.21.15.82:0/859974193 shutdown_connections 2023-09-09T23:07:11.876 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:11.875+0000 7f68cab41700 1 -- 172.21.15.82:0/859974193 wait complete. 2023-09-09T23:07:12.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.081+0000 7fc9179d1700 1 -- 172.21.15.82:0/331161620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc910100620 msgr2=0x7fc910100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:12.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.081+0000 7fc9179d1700 1 --2- 172.21.15.82:0/331161620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc910100620 0x7fc910100aa0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc900009990 tx=0x7fc900009ca0 comp rx=0 tx=0).stop 2023-09-09T23:07:12.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.081+0000 7fc9179d1700 1 -- 172.21.15.82:0/331161620 shutdown_connections 2023-09-09T23:07:12.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.081+0000 7fc9179d1700 1 --2- 172.21.15.82:0/331161620 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc910100fe0 0x7fc91010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:12.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.081+0000 7fc9179d1700 1 --2- 172.21.15.82:0/331161620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc910106620 0x7fc910106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:12.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.081+0000 7fc9179d1700 1 --2- 172.21.15.82:0/331161620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc910100620 0x7fc910100aa0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:12.117 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.081+0000 7fc9179d1700 1 -- 172.21.15.82:0/331161620 >> 172.21.15.82:0/331161620 conn(0x7fc9100fc0f0 msgr2=0x7fc9100fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:12.117 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.081+0000 7fc9179d1700 1 -- 172.21.15.82:0/331161620 shutdown_connections 2023-09-09T23:07:12.117 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.081+0000 7fc9179d1700 1 -- 172.21.15.82:0/331161620 wait complete. 2023-09-09T23:07:12.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.082+0000 7fc9179d1700 1 Processor -- start 2023-09-09T23:07:12.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.082+0000 7fc9179d1700 1 -- start start 2023-09-09T23:07:12.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.082+0000 7fc9179d1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc910100620 0x7fc910196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:12.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.082+0000 7fc9179d1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc910100fe0 0x7fc910196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:12.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.082+0000 7fc9179d1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc910106620 0x7fc910199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:12.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.082+0000 7fc9179d1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91019a590 con 0x7fc910100fe0 2023-09-09T23:07:12.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.082+0000 7fc9179d1700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91019a700 con 0x7fc910100620 2023-09-09T23:07:12.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.082+0000 7fc9179d1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91019a8c0 con 0x7fc910106620 2023-09-09T23:07:12.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.082+0000 7fc914f6c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc910100fe0 0x7fc910196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:12.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.082+0000 7fc914f6c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc910100fe0 0x7fc910196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53380/0 (socket says 172.21.15.82:53380) 2023-09-09T23:07:12.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.082+0000 7fc914f6c700 1 -- 172.21.15.82:0/1623196932 learned_addr learned my addr 172.21.15.82:0/1623196932 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:07:12.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.082+0000 7fc91576d700 1 --2- 172.21.15.82:0/1623196932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc910100620 0x7fc910196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:12.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.082+0000 7fc914f6c700 1 -- 172.21.15.82:0/1623196932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc910106620 msgr2=0x7fc910199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:12.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.082+0000 7fc914f6c700 1 --2- 172.21.15.82:0/1623196932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc910106620 0x7fc910199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:12.122 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.082+0000 7fc914f6c700 1 -- 172.21.15.82:0/1623196932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc910100620 msgr2=0x7fc910196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:12.122 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.082+0000 7fc914f6c700 1 --2- 172.21.15.82:0/1623196932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc910100620 0x7fc910196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:12.122 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.082+0000 7fc914f6c700 1 -- 172.21.15.82:0/1623196932 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9000095f0 con 0x7fc910100fe0 2023-09-09T23:07:12.123 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.082+0000 7fc914f6c700 1 --2- 172.21.15.82:0/1623196932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc910100fe0 0x7fc910196bd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc9000095c0 tx=0x7fc90000f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:12.123 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.083+0000 7fc9067fc700 1 -- 172.21.15.82:0/1623196932 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90001d070 con 0x7fc910100fe0 2023-09-09T23:07:12.123 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.083+0000 7fc9067fc700 1 -- 172.21.15.82:0/1623196932 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc90000fb50 con 0x7fc910100fe0 2023-09-09T23:07:12.123 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.083+0000 7fc9067fc700 1 -- 172.21.15.82:0/1623196932 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9000173f0 con 0x7fc910100fe0 2023-09-09T23:07:12.124 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.083+0000 7fc9179d1700 1 -- 172.21.15.82:0/1623196932 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc91019ab40 con 0x7fc910100fe0 2023-09-09T23:07:12.124 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.083+0000 7fc9179d1700 1 -- 172.21.15.82:0/1623196932 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc91019ae10 con 0x7fc910100fe0 2023-09-09T23:07:12.124 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.083+0000 7fc9067fc700 1 -- 172.21.15.82:0/1623196932 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7fc900017550 con 0x7fc910100fe0 2023-09-09T23:07:12.124 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.084+0000 7fc9179d1700 1 -- 172.21.15.82:0/1623196932 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc91004e300 con 0x7fc910100fe0 2023-09-09T23:07:12.124 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.084+0000 7fc9067fc700 1 --2- 172.21.15.82:0/1623196932 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fc8fc038630 0x7fc8fc03aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:12.125 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.084+0000 7fc9067fc700 1 -- 172.21.15.82:0/1623196932 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc90004c1c0 con 0x7fc910100fe0 2023-09-09T23:07:12.125 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.084+0000 7fc91576d700 1 --2- 172.21.15.82:0/1623196932 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fc8fc038630 0x7fc8fc03aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:12.125 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.084+0000 7fc91576d700 1 --2- 172.21.15.82:0/1623196932 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fc8fc038630 0x7fc8fc03aaf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc90c005950 tx=0x7fc90c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:12.125 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.088+0000 7fc9067fc700 1 -- 172.21.15.82:0/1623196932 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc90002b3b0 con 0x7fc910100fe0 2023-09-09T23:07:12.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:11 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1848563690' entity='client.admin' 2023-09-09T23:07:12.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:11 smithi170 ceph-mon[102832]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:12.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:11 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2858897293' entity='client.admin' 2023-09-09T23:07:12.182 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:11 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1848563690' entity='client.admin' 2023-09-09T23:07:12.182 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:11 smithi082 ceph-mon[98694]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:12.182 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:11 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2858897293' entity='client.admin' 2023-09-09T23:07:12.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:11 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1848563690' entity='client.admin' 2023-09-09T23:07:12.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:11 smithi098 ceph-mon[103124]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:12.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:11 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2858897293' entity='client.admin' 2023-09-09T23:07:12.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.295+0000 7fc9179d1700 1 -- 172.21.15.82:0/1623196932 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) v1 -- 0x7fc9101979f0 con 0x7fc910100fe0 2023-09-09T23:07:12.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.300+0000 7fc9067fc700 1 -- 172.21.15.82:0/1623196932 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v20)=0 v20) v1 ==== 116+0+0 (secure 0 0 0) 0x7fc9000270b0 con 0x7fc910100fe0 2023-09-09T23:07:12.301 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.302+0000 7fc9179d1700 1 -- 172.21.15.82:0/1623196932 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fc8fc038630 msgr2=0x7fc8fc03aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:12.301 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.302+0000 7fc9179d1700 1 --2- 172.21.15.82:0/1623196932 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fc8fc038630 0x7fc8fc03aaf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc90c005950 tx=0x7fc90c00b410 comp rx=0 tx=0).stop 2023-09-09T23:07:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.302+0000 7fc9179d1700 1 -- 172.21.15.82:0/1623196932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc910100fe0 msgr2=0x7fc910196bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.302+0000 7fc9179d1700 1 --2- 172.21.15.82:0/1623196932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc910100fe0 0x7fc910196bd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc9000095c0 tx=0x7fc90000f710 comp rx=0 tx=0).stop 2023-09-09T23:07:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.302+0000 7fc9179d1700 1 -- 172.21.15.82:0/1623196932 shutdown_connections 2023-09-09T23:07:12.303 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.302+0000 7fc9179d1700 1 --2- 172.21.15.82:0/1623196932 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fc8fc038630 0x7fc8fc03aaf0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:12.303 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.302+0000 7fc9179d1700 1 --2- 172.21.15.82:0/1623196932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc910100620 0x7fc910196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:12.304 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.302+0000 7fc9179d1700 1 --2- 172.21.15.82:0/1623196932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc910106620 0x7fc910199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:12.304 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.302+0000 7fc9179d1700 1 --2- 172.21.15.82:0/1623196932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc910100fe0 0x7fc910196bd0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:12.304 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.302+0000 7fc9179d1700 1 -- 172.21.15.82:0/1623196932 >> 172.21.15.82:0/1623196932 conn(0x7fc9100fc0f0 msgr2=0x7fc9100fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:12.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.302+0000 7fc9179d1700 1 -- 172.21.15.82:0/1623196932 shutdown_connections 2023-09-09T23:07:12.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.302+0000 7fc9179d1700 1 -- 172.21.15.82:0/1623196932 wait complete. 2023-09-09T23:07:12.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.432+0000 7fdac5b88700 1 -- 172.21.15.82:0/812198732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdac01055e0 msgr2=0x7fdac01059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:12.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.432+0000 7fdac5b88700 1 --2- 172.21.15.82:0/812198732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdac01055e0 0x7fdac01059c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdab00099c0 tx=0x7fdab0009cd0 comp rx=0 tx=0).stop 2023-09-09T23:07:12.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.432+0000 7fdac5b88700 1 -- 172.21.15.82:0/812198732 shutdown_connections 2023-09-09T23:07:12.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.432+0000 7fdac5b88700 1 --2- 172.21.15.82:0/812198732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdac00ffef0 0x7fdac010c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:12.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.432+0000 7fdac5b88700 1 --2- 172.21.15.82:0/812198732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdac01055e0 0x7fdac01059c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:12.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.432+0000 7fdac5b88700 1 --2- 172.21.15.82:0/812198732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdac00ff530 0x7fdac00ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:12.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.432+0000 7fdac5b88700 1 -- 172.21.15.82:0/812198732 >> 172.21.15.82:0/812198732 conn(0x7fdac00fafc0 msgr2=0x7fdac00fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:12.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.432+0000 7fdac5b88700 1 -- 172.21.15.82:0/812198732 shutdown_connections 2023-09-09T23:07:12.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.432+0000 7fdac5b88700 1 -- 172.21.15.82:0/812198732 wait complete. 2023-09-09T23:07:12.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.433+0000 7fdac5b88700 1 Processor -- start 2023-09-09T23:07:12.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.433+0000 7fdac5b88700 1 -- start start 2023-09-09T23:07:12.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.434+0000 7fdac5b88700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdac00ff530 0x7fdac0197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:12.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.434+0000 7fdac5b88700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdac00ffef0 0x7fdac01979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:12.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.434+0000 7fdac5b88700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdac01055e0 0x7fdac01a2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:12.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.434+0000 7fdac5b88700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac01982d0 con 0x7fdac01055e0 2023-09-09T23:07:12.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.434+0000 7fdac5b88700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac01a2a40 con 0x7fdac00ffef0 2023-09-09T23:07:12.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.434+0000 7fdac5b88700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac01a2bb0 con 0x7fdac00ff530 2023-09-09T23:07:12.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.434+0000 7fdabffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdac01055e0 0x7fdac01a2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:12.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.434+0000 7fdabffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdac01055e0 0x7fdac01a2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53396/0 (socket says 172.21.15.82:53396) 2023-09-09T23:07:12.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.434+0000 7fdabffff700 1 -- 172.21.15.82:0/1681338 learned_addr learned my addr 172.21.15.82:0/1681338 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:07:12.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.434+0000 7fdabf7fe700 1 --2- 172.21.15.82:0/1681338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdac00ff530 0x7fdac0197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:12.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.434+0000 7fdabeffd700 1 --2- 172.21.15.82:0/1681338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdac00ffef0 0x7fdac01979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:12.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.434+0000 7fdabffff700 1 -- 172.21.15.82:0/1681338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdac00ff530 msgr2=0x7fdac0197480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:12.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.434+0000 7fdabffff700 1 --2- 172.21.15.82:0/1681338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdac00ff530 0x7fdac0197480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:12.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.434+0000 7fdabffff700 1 -- 172.21.15.82:0/1681338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdac00ffef0 msgr2=0x7fdac01979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:12.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.434+0000 7fdabffff700 1 --2- 172.21.15.82:0/1681338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdac00ffef0 0x7fdac01979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:12.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.434+0000 7fdabffff700 1 -- 172.21.15.82:0/1681338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab00095f0 con 0x7fdac01055e0 2023-09-09T23:07:12.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.434+0000 7fdabffff700 1 --2- 172.21.15.82:0/1681338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdac01055e0 0x7fdac01a2500 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdab8007ee0 tx=0x7fdab800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:12.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.435+0000 7fdabcff9700 1 -- 172.21.15.82:0/1681338 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab8017070 con 0x7fdac01055e0 2023-09-09T23:07:12.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.435+0000 7fdabcff9700 1 -- 172.21.15.82:0/1681338 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdab800f040 con 0x7fdac01055e0 2023-09-09T23:07:12.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.435+0000 7fdac5b88700 1 -- 172.21.15.82:0/1681338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac01a2e90 con 0x7fdac01055e0 2023-09-09T23:07:12.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.435+0000 7fdabcff9700 1 -- 172.21.15.82:0/1681338 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab8005580 con 0x7fdac01055e0 2023-09-09T23:07:12.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.435+0000 7fdac5b88700 1 -- 172.21.15.82:0/1681338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac01a3360 con 0x7fdac01055e0 2023-09-09T23:07:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.436+0000 7fdabcff9700 1 -- 172.21.15.82:0/1681338 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7fdab80056e0 con 0x7fdac01055e0 2023-09-09T23:07:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.436+0000 7fdabcff9700 1 --2- 172.21.15.82:0/1681338 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fdaa8038630 0x7fdaa803aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.436+0000 7fdabcff9700 1 -- 172.21.15.82:0/1681338 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdab804a680 con 0x7fdac01055e0 2023-09-09T23:07:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.436+0000 7fdabf7fe700 1 --2- 172.21.15.82:0/1681338 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fdaa8038630 0x7fdaa803aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.436+0000 7fdac5b88700 1 -- 172.21.15.82:0/1681338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac004e300 con 0x7fdac01055e0 2023-09-09T23:07:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.436+0000 7fdabf7fe700 1 --2- 172.21.15.82:0/1681338 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fdaa8038630 0x7fdaa803aaf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdab0000c00 tx=0x7fdab0003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:12.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.441+0000 7fdabcff9700 1 -- 172.21.15.82:0/1681338 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fdab801ccb0 con 0x7fdac01055e0 2023-09-09T23:07:12.646 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.647+0000 7fdac5b88700 1 -- 172.21.15.82:0/1681338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) v1 -- 0x7fdac010f4f0 con 0x7fdac01055e0 2023-09-09T23:07:12.650 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.652+0000 7fdabcff9700 1 -- 172.21.15.82:0/1681338 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v21)=0 v21) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdab801aba0 con 0x7fdac01055e0 2023-09-09T23:07:12.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.654+0000 7fdac5b88700 1 -- 172.21.15.82:0/1681338 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fdaa8038630 msgr2=0x7fdaa803aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:12.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.654+0000 7fdac5b88700 1 --2- 172.21.15.82:0/1681338 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fdaa8038630 0x7fdaa803aaf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdab0000c00 tx=0x7fdab0003680 comp rx=0 tx=0).stop 2023-09-09T23:07:12.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.654+0000 7fdac5b88700 1 -- 172.21.15.82:0/1681338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdac01055e0 msgr2=0x7fdac01a2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:12.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.654+0000 7fdac5b88700 1 --2- 172.21.15.82:0/1681338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdac01055e0 0x7fdac01a2500 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdab8007ee0 tx=0x7fdab800d3b0 comp rx=0 tx=0).stop 2023-09-09T23:07:12.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.654+0000 7fdac5b88700 1 -- 172.21.15.82:0/1681338 shutdown_connections 2023-09-09T23:07:12.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.654+0000 7fdac5b88700 1 --2- 172.21.15.82:0/1681338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdac01055e0 0x7fdac01a2500 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:12.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.654+0000 7fdac5b88700 1 --2- 172.21.15.82:0/1681338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdac00ff530 0x7fdac0197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:12.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.654+0000 7fdac5b88700 1 --2- 172.21.15.82:0/1681338 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fdaa8038630 0x7fdaa803aaf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:12.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.654+0000 7fdac5b88700 1 --2- 172.21.15.82:0/1681338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdac00ffef0 0x7fdac01979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:12.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.655+0000 7fdac5b88700 1 -- 172.21.15.82:0/1681338 >> 172.21.15.82:0/1681338 conn(0x7fdac00fafc0 msgr2=0x7fdac00fc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:12.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.655+0000 7fdac5b88700 1 -- 172.21.15.82:0/1681338 shutdown_connections 2023-09-09T23:07:12.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.655+0000 7fdac5b88700 1 -- 172.21.15.82:0/1681338 wait complete. 2023-09-09T23:07:13.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:12 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/859974193' entity='client.admin' 2023-09-09T23:07:13.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:12 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1623196932' entity='client.admin' 2023-09-09T23:07:13.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:12 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1681338' entity='client.admin' 2023-09-09T23:07:13.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:12 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2425727186' entity='client.admin' 2023-09-09T23:07:13.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:12 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/859974193' entity='client.admin' 2023-09-09T23:07:13.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:12 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1623196932' entity='client.admin' 2023-09-09T23:07:13.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:12 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1681338' entity='client.admin' 2023-09-09T23:07:13.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:12 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2425727186' entity='client.admin' 2023-09-09T23:07:13.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:12 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/859974193' entity='client.admin' 2023-09-09T23:07:13.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:12 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1623196932' entity='client.admin' 2023-09-09T23:07:13.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:12 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1681338' entity='client.admin' 2023-09-09T23:07:13.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:12 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2425727186' entity='client.admin' 2023-09-09T23:07:13.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.585+0000 7f28bf9e7700 1 -- 172.21.15.82:0/3243647168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28b8105950 msgr2=0x7f28b810d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:13.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.585+0000 7f28bf9e7700 1 --2- 172.21.15.82:0/3243647168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28b8105950 0x7f28b810d560 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f28b000b0a0 tx=0x7f28b000b3b0 comp rx=0 tx=0).stop 2023-09-09T23:07:13.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.585+0000 7f28bf9e7700 1 -- 172.21.15.82:0/3243647168 shutdown_connections 2023-09-09T23:07:13.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.585+0000 7f28bf9e7700 1 --2- 172.21.15.82:0/3243647168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28b80fedb0 0x7f28b80ff190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:13.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.585+0000 7f28bf9e7700 1 --2- 172.21.15.82:0/3243647168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28b8105950 0x7f28b810d560 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:13.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.585+0000 7f28bf9e7700 1 --2- 172.21.15.82:0/3243647168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28b80ff760 0x7f28b8105220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:13.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.585+0000 7f28bf9e7700 1 -- 172.21.15.82:0/3243647168 >> 172.21.15.82:0/3243647168 conn(0x7f28b8075a70 msgr2=0x7f28b80fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:13.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.585+0000 7f28bf9e7700 1 -- 172.21.15.82:0/3243647168 shutdown_connections 2023-09-09T23:07:13.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.585+0000 7f28bf9e7700 1 -- 172.21.15.82:0/3243647168 wait complete. 2023-09-09T23:07:13.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.586+0000 7f28bf9e7700 1 Processor -- start 2023-09-09T23:07:13.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.586+0000 7f28bf9e7700 1 -- start start 2023-09-09T23:07:13.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.586+0000 7f28bf9e7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28b80fedb0 0x7f28b819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:13.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.586+0000 7f28bf9e7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28b80ff760 0x7f28b819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:13.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.586+0000 7f28bf9e7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28b8105950 0x7f28b8196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:13.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.586+0000 7f28bf9e7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b8196c70 con 0x7f28b80fedb0 2023-09-09T23:07:13.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.586+0000 7f28bf9e7700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b8196de0 con 0x7f28b80ff760 2023-09-09T23:07:13.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.586+0000 7f28bf9e7700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b8196fa0 con 0x7f28b8105950 2023-09-09T23:07:13.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.586+0000 7f28bd783700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28b80fedb0 0x7f28b819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:13.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.586+0000 7f28bd783700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28b80fedb0 0x7f28b819c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53406/0 (socket says 172.21.15.82:53406) 2023-09-09T23:07:13.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.586+0000 7f28bd783700 1 -- 172.21.15.82:0/2425727186 learned_addr learned my addr 172.21.15.82:0/2425727186 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:07:13.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.586+0000 7f28bcf82700 1 --2- 172.21.15.82:0/2425727186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28b80ff760 0x7f28b819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:13.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.587+0000 7f28bd783700 1 -- 172.21.15.82:0/2425727186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28b8105950 msgr2=0x7f28b8196600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:13.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.587+0000 7f28bd783700 1 --2- 172.21.15.82:0/2425727186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28b8105950 0x7f28b8196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:13.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.587+0000 7f28bd783700 1 -- 172.21.15.82:0/2425727186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28b80ff760 msgr2=0x7f28b819cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:13.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.587+0000 7f28bd783700 1 --2- 172.21.15.82:0/2425727186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28b80ff760 0x7f28b819cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:13.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.587+0000 7f28bd783700 1 -- 172.21.15.82:0/2425727186 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b0009d00 con 0x7f28b80fedb0 2023-09-09T23:07:13.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.587+0000 7f28bd783700 1 --2- 172.21.15.82:0/2425727186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28b80fedb0 0x7f28b819c580 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f28b400d950 tx=0x7f28b400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:13.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.587+0000 7f28ae7fc700 1 -- 172.21.15.82:0/2425727186 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b4009940 con 0x7f28b80fedb0 2023-09-09T23:07:13.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.587+0000 7f28ae7fc700 1 -- 172.21.15.82:0/2425727186 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28b4010460 con 0x7f28b80fedb0 2023-09-09T23:07:13.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.587+0000 7f28bf9e7700 1 -- 172.21.15.82:0/2425727186 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b8197280 con 0x7f28b80fedb0 2023-09-09T23:07:13.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.587+0000 7f28ae7fc700 1 -- 172.21.15.82:0/2425727186 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b400f620 con 0x7f28b80fedb0 2023-09-09T23:07:13.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.587+0000 7f28bf9e7700 1 -- 172.21.15.82:0/2425727186 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b81a3570 con 0x7f28b80fedb0 2023-09-09T23:07:13.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.588+0000 7f28ae7fc700 1 -- 172.21.15.82:0/2425727186 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f28b40108d0 con 0x7f28b80fedb0 2023-09-09T23:07:13.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.588+0000 7f28bf9e7700 1 -- 172.21.15.82:0/2425727186 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28b8108bb0 con 0x7f28b80fedb0 2023-09-09T23:07:13.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.588+0000 7f28ae7fc700 1 --2- 172.21.15.82:0/2425727186 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f28a4038630 0x7f28a403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:13.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.588+0000 7f28ae7fc700 1 -- 172.21.15.82:0/2425727186 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f28b404ac50 con 0x7f28b80fedb0 2023-09-09T23:07:13.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.588+0000 7f28bcf82700 1 --2- 172.21.15.82:0/2425727186 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f28a4038630 0x7f28a403aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:13.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.589+0000 7f28bcf82700 1 --2- 172.21.15.82:0/2425727186 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f28a4038630 0x7f28a403aaf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f28b80fe0c0 tx=0x7f28a8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:13.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.593+0000 7f28ae7fc700 1 -- 172.21.15.82:0/2425727186 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f28b4050020 con 0x7f28b80fedb0 2023-09-09T23:07:13.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.810+0000 7f28bf9e7700 1 -- 172.21.15.82:0/2425727186 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) v1 -- 0x7f28b804e300 con 0x7f28b80fedb0 2023-09-09T23:07:13.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.815+0000 7f28ae7fc700 1 -- 172.21.15.82:0/2425727186 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v22)=0 v22) v1 ==== 121+0+0 (secure 0 0 0) 0x7f28b4016070 con 0x7f28b80fedb0 2023-09-09T23:07:13.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.817+0000 7f28bf9e7700 1 -- 172.21.15.82:0/2425727186 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f28a4038630 msgr2=0x7f28a403aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:13.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.817+0000 7f28bf9e7700 1 --2- 172.21.15.82:0/2425727186 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f28a4038630 0x7f28a403aaf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f28b80fe0c0 tx=0x7f28a8009190 comp rx=0 tx=0).stop 2023-09-09T23:07:13.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.817+0000 7f28bf9e7700 1 -- 172.21.15.82:0/2425727186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28b80fedb0 msgr2=0x7f28b819c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:13.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.817+0000 7f28bf9e7700 1 --2- 172.21.15.82:0/2425727186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28b80fedb0 0x7f28b819c580 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f28b400d950 tx=0x7f28b400dc60 comp rx=0 tx=0).stop 2023-09-09T23:07:13.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.817+0000 7f28bf9e7700 1 -- 172.21.15.82:0/2425727186 shutdown_connections 2023-09-09T23:07:13.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.817+0000 7f28bf9e7700 1 --2- 172.21.15.82:0/2425727186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28b8105950 0x7f28b8196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:13.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.817+0000 7f28bf9e7700 1 --2- 172.21.15.82:0/2425727186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28b80fedb0 0x7f28b819c580 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:13.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.817+0000 7f28bf9e7700 1 --2- 172.21.15.82:0/2425727186 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f28a4038630 0x7f28a403aaf0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:13.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.817+0000 7f28bf9e7700 1 --2- 172.21.15.82:0/2425727186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28b80ff760 0x7f28b819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:13.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.817+0000 7f28bf9e7700 1 -- 172.21.15.82:0/2425727186 >> 172.21.15.82:0/2425727186 conn(0x7f28b8075a70 msgr2=0x7f28b8103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:13.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.817+0000 7f28bf9e7700 1 -- 172.21.15.82:0/2425727186 shutdown_connections 2023-09-09T23:07:13.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:12.817+0000 7f28bf9e7700 1 -- 172.21.15.82:0/2425727186 wait complete. 2023-09-09T23:07:13.979 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:07:14.320 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:07:14.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.360+0000 7f5d597b4700 1 -- 172.21.15.82:0/4177975125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d540ff8a0 msgr2=0x7f5d541038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:14.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.360+0000 7f5d597b4700 1 --2- 172.21.15.82:0/4177975125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d540ff8a0 0x7f5d541038f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400aaa0 tx=0x7f5d4400adb0 comp rx=0 tx=0).stop 2023-09-09T23:07:14.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.360+0000 7f5d597b4700 1 -- 172.21.15.82:0/4177975125 shutdown_connections 2023-09-09T23:07:14.360 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.360+0000 7f5d597b4700 1 --2- 172.21.15.82:0/4177975125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d54103fd0 0x7f5d5410bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:14.360 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.360+0000 7f5d597b4700 1 --2- 172.21.15.82:0/4177975125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d540feef0 0x7f5d540ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:14.360 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.360+0000 7f5d597b4700 1 --2- 172.21.15.82:0/4177975125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d540ff8a0 0x7f5d541038f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:14.360 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.360+0000 7f5d597b4700 1 -- 172.21.15.82:0/4177975125 >> 172.21.15.82:0/4177975125 conn(0x7f5d54075a70 msgr2=0x7f5d540fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:14.360 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.360+0000 7f5d597b4700 1 -- 172.21.15.82:0/4177975125 shutdown_connections 2023-09-09T23:07:14.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.360+0000 7f5d597b4700 1 -- 172.21.15.82:0/4177975125 wait complete. 2023-09-09T23:07:14.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.361+0000 7f5d597b4700 1 Processor -- start 2023-09-09T23:07:14.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.361+0000 7f5d597b4700 1 -- start start 2023-09-09T23:07:14.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.361+0000 7f5d597b4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d540feef0 0x7f5d54194520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:14.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.361+0000 7f5d597b4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d540ff8a0 0x7f5d54194a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:14.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.361+0000 7f5d597b4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d54103fd0 0x7f5d54198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:14.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.361+0000 7f5d597b4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d541994e0 con 0x7f5d540feef0 2023-09-09T23:07:14.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.361+0000 7f5d597b4700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d54199650 con 0x7f5d540ff8a0 2023-09-09T23:07:14.365 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.361+0000 7f5d597b4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d54199810 con 0x7f5d54103fd0 2023-09-09T23:07:14.365 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.361+0000 7f5d52ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d540feef0 0x7f5d54194520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:14.365 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.362+0000 7f5d52ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d540feef0 0x7f5d54194520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53418/0 (socket says 172.21.15.82:53418) 2023-09-09T23:07:14.365 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.362+0000 7f5d52ffd700 1 -- 172.21.15.82:0/866760154 learned_addr learned my addr 172.21.15.82:0/866760154 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:07:14.365 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.362+0000 7f5d527fc700 1 --2- 172.21.15.82:0/866760154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d540ff8a0 0x7f5d54194a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:14.366 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.362+0000 7f5d537fe700 1 --2- 172.21.15.82:0/866760154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d54103fd0 0x7f5d54198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:14.366 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.362+0000 7f5d52ffd700 1 -- 172.21.15.82:0/866760154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d54103fd0 msgr2=0x7f5d54198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:14.366 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.362+0000 7f5d52ffd700 1 --2- 172.21.15.82:0/866760154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d54103fd0 0x7f5d54198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:14.366 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.362+0000 7f5d52ffd700 1 -- 172.21.15.82:0/866760154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d540ff8a0 msgr2=0x7f5d54194a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:14.366 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.362+0000 7f5d52ffd700 1 --2- 172.21.15.82:0/866760154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d540ff8a0 0x7f5d54194a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:14.367 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.362+0000 7f5d52ffd700 1 -- 172.21.15.82:0/866760154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4400a700 con 0x7f5d540feef0 2023-09-09T23:07:14.367 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.362+0000 7f5d52ffd700 1 --2- 172.21.15.82:0/866760154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d540feef0 0x7f5d54194520 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5d4800d9e0 tx=0x7f5d4800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:14.367 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.362+0000 7f5d3bfff700 1 -- 172.21.15.82:0/866760154 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4800ec50 con 0x7f5d540feef0 2023-09-09T23:07:14.367 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.362+0000 7f5d3bfff700 1 -- 172.21.15.82:0/866760154 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d4800edb0 con 0x7f5d540feef0 2023-09-09T23:07:14.367 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.362+0000 7f5d597b4700 1 -- 172.21.15.82:0/866760154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d54199af0 con 0x7f5d540feef0 2023-09-09T23:07:14.367 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.362+0000 7f5d3bfff700 1 -- 172.21.15.82:0/866760154 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d48014730 con 0x7f5d540feef0 2023-09-09T23:07:14.368 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.363+0000 7f5d597b4700 1 -- 172.21.15.82:0/866760154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d541a1350 con 0x7f5d540feef0 2023-09-09T23:07:14.368 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.364+0000 7f5d3bfff700 1 -- 172.21.15.82:0/866760154 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f5d48016a90 con 0x7f5d540feef0 2023-09-09T23:07:14.368 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.364+0000 7f5d597b4700 1 -- 172.21.15.82:0/866760154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d5404e300 con 0x7f5d540feef0 2023-09-09T23:07:14.368 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.364+0000 7f5d3bfff700 1 --2- 172.21.15.82:0/866760154 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5d3c038630 0x7f5d3c03aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:14.368 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.364+0000 7f5d3bfff700 1 -- 172.21.15.82:0/866760154 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5d4804c750 con 0x7f5d540feef0 2023-09-09T23:07:14.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.364+0000 7f5d527fc700 1 --2- 172.21.15.82:0/866760154 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5d3c038630 0x7f5d3c03aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:14.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.364+0000 7f5d527fc700 1 --2- 172.21.15.82:0/866760154 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5d3c038630 0x7f5d3c03aaf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5d54195b40 tx=0x7f5d44005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:14.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.369+0000 7f5d3bfff700 1 -- 172.21.15.82:0/866760154 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5d4801d2c0 con 0x7f5d540feef0 2023-09-09T23:07:14.373 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:14 smithi082 ceph-mon[98694]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:14.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:14 smithi098 ceph-mon[103124]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:14.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.578+0000 7f5d597b4700 1 -- 172.21.15.82:0/866760154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) v1 -- 0x7f5d54195880 con 0x7f5d540feef0 2023-09-09T23:07:14.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.581+0000 7f5d3bfff700 1 -- 172.21.15.82:0/866760154 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v23)=0 v23) v1 ==== 127+0+0 (secure 0 0 0) 0x7f5d48015b90 con 0x7f5d540feef0 2023-09-09T23:07:14.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.583+0000 7f5d597b4700 1 -- 172.21.15.82:0/866760154 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5d3c038630 msgr2=0x7f5d3c03aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:14.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.583+0000 7f5d597b4700 1 --2- 172.21.15.82:0/866760154 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5d3c038630 0x7f5d3c03aaf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5d54195b40 tx=0x7f5d44005fd0 comp rx=0 tx=0).stop 2023-09-09T23:07:14.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.583+0000 7f5d597b4700 1 -- 172.21.15.82:0/866760154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d540feef0 msgr2=0x7f5d54194520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:14.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.583+0000 7f5d597b4700 1 --2- 172.21.15.82:0/866760154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d540feef0 0x7f5d54194520 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5d4800d9e0 tx=0x7f5d4800dcf0 comp rx=0 tx=0).stop 2023-09-09T23:07:14.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.583+0000 7f5d597b4700 1 -- 172.21.15.82:0/866760154 shutdown_connections 2023-09-09T23:07:14.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.583+0000 7f5d597b4700 1 --2- 172.21.15.82:0/866760154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d54103fd0 0x7f5d54198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:14.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.583+0000 7f5d597b4700 1 --2- 172.21.15.82:0/866760154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d540feef0 0x7f5d54194520 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:14.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.583+0000 7f5d597b4700 1 --2- 172.21.15.82:0/866760154 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5d3c038630 0x7f5d3c03aaf0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:14.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.583+0000 7f5d597b4700 1 --2- 172.21.15.82:0/866760154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d540ff8a0 0x7f5d54194a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:14.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.583+0000 7f5d597b4700 1 -- 172.21.15.82:0/866760154 >> 172.21.15.82:0/866760154 conn(0x7f5d54075a70 msgr2=0x7f5d54103650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:14.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.583+0000 7f5d597b4700 1 -- 172.21.15.82:0/866760154 shutdown_connections 2023-09-09T23:07:14.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:14.583+0000 7f5d597b4700 1 -- 172.21.15.82:0/866760154 wait complete. 2023-09-09T23:07:14.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:14 smithi170 ceph-mon[102832]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:15.489 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.491+0000 7f8c4dbfc700 1 -- 172.21.15.82:0/2379374157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c481055e0 msgr2=0x7f8c481059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:15.490 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.491+0000 7f8c4dbfc700 1 --2- 172.21.15.82:0/2379374157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c481055e0 0x7f8c481059c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8c380099c0 tx=0x7f8c38009cd0 comp rx=0 tx=0).stop 2023-09-09T23:07:15.490 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.491+0000 7f8c4dbfc700 1 -- 172.21.15.82:0/2379374157 shutdown_connections 2023-09-09T23:07:15.490 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.491+0000 7f8c4dbfc700 1 --2- 172.21.15.82:0/2379374157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c48100640 0x7f8c4810c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:15.491 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.491+0000 7f8c4dbfc700 1 --2- 172.21.15.82:0/2379374157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c481055e0 0x7f8c481059c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:15.491 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.491+0000 7f8c4dbfc700 1 --2- 172.21.15.82:0/2379374157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c480ffc80 0x7f8c48100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:15.491 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.491+0000 7f8c4dbfc700 1 -- 172.21.15.82:0/2379374157 >> 172.21.15.82:0/2379374157 conn(0x7f8c48075a70 msgr2=0x7f8c480fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:15.491 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.491+0000 7f8c4dbfc700 1 -- 172.21.15.82:0/2379374157 shutdown_connections 2023-09-09T23:07:15.491 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.491+0000 7f8c4dbfc700 1 -- 172.21.15.82:0/2379374157 wait complete. 2023-09-09T23:07:15.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.491+0000 7f8c4dbfc700 1 Processor -- start 2023-09-09T23:07:15.497 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.492+0000 7f8c4dbfc700 1 -- start start 2023-09-09T23:07:15.497 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.492+0000 7f8c4dbfc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c480ffc80 0x7f8c48106450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:15.497 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.492+0000 7f8c4dbfc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c48100640 0x7f8c48106990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:15.498 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.492+0000 7f8c4dbfc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c481055e0 0x7f8c48106f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:15.498 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.492+0000 7f8c4dbfc700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c481a4b80 con 0x7f8c48100640 2023-09-09T23:07:15.498 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.492+0000 7f8c4dbfc700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c481a4cf0 con 0x7f8c481055e0 2023-09-09T23:07:15.498 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.492+0000 7f8c4dbfc700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c481a4eb0 con 0x7f8c480ffc80 2023-09-09T23:07:15.498 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.492+0000 7f8c46ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c48100640 0x7f8c48106990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:15.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.492+0000 7f8c46ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c48100640 0x7f8c48106990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53432/0 (socket says 172.21.15.82:53432) 2023-09-09T23:07:15.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.492+0000 7f8c46ffd700 1 -- 172.21.15.82:0/452822771 learned_addr learned my addr 172.21.15.82:0/452822771 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:07:15.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.492+0000 7f8c477fe700 1 --2- 172.21.15.82:0/452822771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c480ffc80 0x7f8c48106450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:15.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.492+0000 7f8c47fff700 1 --2- 172.21.15.82:0/452822771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c481055e0 0x7f8c48106f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:15.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.492+0000 7f8c46ffd700 1 -- 172.21.15.82:0/452822771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c480ffc80 msgr2=0x7f8c48106450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:15.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.492+0000 7f8c46ffd700 1 --2- 172.21.15.82:0/452822771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c480ffc80 0x7f8c48106450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:15.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.492+0000 7f8c46ffd700 1 -- 172.21.15.82:0/452822771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c481055e0 msgr2=0x7f8c48106f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:15.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.492+0000 7f8c46ffd700 1 --2- 172.21.15.82:0/452822771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c481055e0 0x7f8c48106f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:15.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.492+0000 7f8c46ffd700 1 -- 172.21.15.82:0/452822771 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c380095f0 con 0x7f8c48100640 2023-09-09T23:07:15.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.492+0000 7f8c477fe700 1 --2- 172.21.15.82:0/452822771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c480ffc80 0x7f8c48106450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:07:15.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.492+0000 7f8c47fff700 1 --2- 172.21.15.82:0/452822771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c481055e0 0x7f8c48106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:07:15.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.492+0000 7f8c46ffd700 1 --2- 172.21.15.82:0/452822771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c48100640 0x7f8c48106990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00b530 tx=0x7f8c3c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:15.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.493+0000 7f8c44ff9700 1 -- 172.21.15.82:0/452822771 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c011830 con 0x7f8c48100640 2023-09-09T23:07:15.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.493+0000 7f8c44ff9700 1 -- 172.21.15.82:0/452822771 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c3c011e50 con 0x7f8c48100640 2023-09-09T23:07:15.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.493+0000 7f8c44ff9700 1 -- 172.21.15.82:0/452822771 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c00f2e0 con 0x7f8c48100640 2023-09-09T23:07:15.502 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.493+0000 7f8c4dbfc700 1 -- 172.21.15.82:0/452822771 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c481a5190 con 0x7f8c48100640 2023-09-09T23:07:15.502 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.493+0000 7f8c4dbfc700 1 -- 172.21.15.82:0/452822771 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c481a56c0 con 0x7f8c48100640 2023-09-09T23:07:15.502 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.494+0000 7f8c44ff9700 1 -- 172.21.15.82:0/452822771 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f8c3c00f440 con 0x7f8c48100640 2023-09-09T23:07:15.502 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.494+0000 7f8c4dbfc700 1 -- 172.21.15.82:0/452822771 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c4804e300 con 0x7f8c48100640 2023-09-09T23:07:15.502 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.494+0000 7f8c44ff9700 1 --2- 172.21.15.82:0/452822771 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f8c30038630 0x7f8c3003aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:15.503 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.494+0000 7f8c44ff9700 1 -- 172.21.15.82:0/452822771 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8c3c04aa40 con 0x7f8c48100640 2023-09-09T23:07:15.503 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.494+0000 7f8c477fe700 1 --2- 172.21.15.82:0/452822771 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f8c30038630 0x7f8c3003aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:15.503 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.494+0000 7f8c477fe700 1 --2- 172.21.15.82:0/452822771 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f8c30038630 0x7f8c3003aaf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8c38000c00 tx=0x7f8c38005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:15.503 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.498+0000 7f8c44ff9700 1 -- 172.21.15.82:0/452822771 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8c3c021af0 con 0x7f8c48100640 2023-09-09T23:07:15.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.693+0000 7f8c4dbfc700 1 -- 172.21.15.82:0/452822771 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) v1 -- 0x7f8c48107980 con 0x7f8c48100640 2023-09-09T23:07:15.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.696+0000 7f8c44ff9700 1 -- 172.21.15.82:0/452822771 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v24)=0 v24) v1 ==== 117+0+0 (secure 0 0 0) 0x7f8c3c015030 con 0x7f8c48100640 2023-09-09T23:07:15.697 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.698+0000 7f8c4dbfc700 1 -- 172.21.15.82:0/452822771 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f8c30038630 msgr2=0x7f8c3003aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:15.698 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.698+0000 7f8c4dbfc700 1 --2- 172.21.15.82:0/452822771 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f8c30038630 0x7f8c3003aaf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8c38000c00 tx=0x7f8c38005fb0 comp rx=0 tx=0).stop 2023-09-09T23:07:15.698 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.698+0000 7f8c4dbfc700 1 -- 172.21.15.82:0/452822771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c48100640 msgr2=0x7f8c48106990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:15.698 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.698+0000 7f8c4dbfc700 1 --2- 172.21.15.82:0/452822771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c48100640 0x7f8c48106990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00b530 tx=0x7f8c3c00b8f0 comp rx=0 tx=0).stop 2023-09-09T23:07:15.698 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.698+0000 7f8c4dbfc700 1 -- 172.21.15.82:0/452822771 shutdown_connections 2023-09-09T23:07:15.698 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.698+0000 7f8c4dbfc700 1 --2- 172.21.15.82:0/452822771 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f8c30038630 0x7f8c3003aaf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:15.699 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.698+0000 7f8c4dbfc700 1 --2- 172.21.15.82:0/452822771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c481055e0 0x7f8c48106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:15.699 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.698+0000 7f8c4dbfc700 1 --2- 172.21.15.82:0/452822771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c480ffc80 0x7f8c48106450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:15.699 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.698+0000 7f8c4dbfc700 1 --2- 172.21.15.82:0/452822771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c48100640 0x7f8c48106990 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:15.699 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.698+0000 7f8c4dbfc700 1 -- 172.21.15.82:0/452822771 >> 172.21.15.82:0/452822771 conn(0x7f8c48075a70 msgr2=0x7f8c480fbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:15.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.699+0000 7f8c4dbfc700 1 -- 172.21.15.82:0/452822771 shutdown_connections 2023-09-09T23:07:15.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:15.699+0000 7f8c4dbfc700 1 -- 172.21.15.82:0/452822771 wait complete. 2023-09-09T23:07:15.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:15 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/866760154' entity='client.admin' 2023-09-09T23:07:15.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:15 smithi082 ceph-mon[98694]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:15.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:15 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/866760154' entity='client.admin' 2023-09-09T23:07:15.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:15 smithi170 ceph-mon[102832]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:15.959 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:07:16.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:15 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/866760154' entity='client.admin' 2023-09-09T23:07:16.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:15 smithi098 ceph-mon[103124]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:16.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.472+0000 7fa7f1065700 1 -- 172.21.15.82:0/753755577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7ec0ff570 msgr2=0x7fa7ec0ff9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:16.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.472+0000 7fa7f1065700 1 --2- 172.21.15.82:0/753755577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7ec0ff570 0x7fa7ec0ff9f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00099c0 tx=0x7fa7e0009cd0 comp rx=0 tx=0).stop 2023-09-09T23:07:16.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.472+0000 7fa7f1065700 1 -- 172.21.15.82:0/753755577 shutdown_connections 2023-09-09T23:07:16.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.472+0000 7fa7f1065700 1 --2- 172.21.15.82:0/753755577 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7ec0fff30 0x7fa7ec10c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:16.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.472+0000 7fa7f1065700 1 --2- 172.21.15.82:0/753755577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7ec105610 0x7fa7ec1059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:16.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.472+0000 7fa7f1065700 1 --2- 172.21.15.82:0/753755577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7ec0ff570 0x7fa7ec0ff9f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:16.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.472+0000 7fa7f1065700 1 -- 172.21.15.82:0/753755577 >> 172.21.15.82:0/753755577 conn(0x7fa7ec076680 msgr2=0x7fa7ec076a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:16.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.472+0000 7fa7f1065700 1 -- 172.21.15.82:0/753755577 shutdown_connections 2023-09-09T23:07:16.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.472+0000 7fa7f1065700 1 -- 172.21.15.82:0/753755577 wait complete. 2023-09-09T23:07:16.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.473+0000 7fa7f1065700 1 Processor -- start 2023-09-09T23:07:16.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.474+0000 7fa7f1065700 1 -- start start 2023-09-09T23:07:16.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.474+0000 7fa7f1065700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7ec0ff570 0x7fa7ec19c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:16.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.474+0000 7fa7f1065700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7ec0fff30 0x7fa7ec19c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:16.477 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.474+0000 7fa7f1065700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7ec105610 0x7fa7ec196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:16.477 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.474+0000 7fa7f1065700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec1973d0 con 0x7fa7ec0fff30 2023-09-09T23:07:16.477 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.474+0000 7fa7f1065700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec197540 con 0x7fa7ec0ff570 2023-09-09T23:07:16.477 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.474+0000 7fa7f1065700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec197700 con 0x7fa7ec105610 2023-09-09T23:07:16.477 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.474+0000 7fa7ea59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7ec0fff30 0x7fa7ec19c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:16.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.474+0000 7fa7ea59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7ec0fff30 0x7fa7ec19c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53446/0 (socket says 172.21.15.82:53446) 2023-09-09T23:07:16.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.474+0000 7fa7ea59c700 1 -- 172.21.15.82:0/2112188205 learned_addr learned my addr 172.21.15.82:0/2112188205 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:07:16.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.474+0000 7fa7ead9d700 1 --2- 172.21.15.82:0/2112188205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7ec0ff570 0x7fa7ec19c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:16.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.474+0000 7fa7eb59e700 1 --2- 172.21.15.82:0/2112188205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7ec105610 0x7fa7ec196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:16.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.474+0000 7fa7ea59c700 1 -- 172.21.15.82:0/2112188205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7ec105610 msgr2=0x7fa7ec196d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:16.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.474+0000 7fa7ea59c700 1 --2- 172.21.15.82:0/2112188205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7ec105610 0x7fa7ec196d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:16.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.474+0000 7fa7ea59c700 1 -- 172.21.15.82:0/2112188205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7ec0ff570 msgr2=0x7fa7ec19c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:16.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.474+0000 7fa7ea59c700 1 --2- 172.21.15.82:0/2112188205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7ec0ff570 0x7fa7ec19c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:16.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.474+0000 7fa7ea59c700 1 -- 172.21.15.82:0/2112188205 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e00095f0 con 0x7fa7ec0fff30 2023-09-09T23:07:16.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.474+0000 7fa7eb59e700 1 --2- 172.21.15.82:0/2112188205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7ec105610 0x7fa7ec196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:07:16.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.474+0000 7fa7ead9d700 1 --2- 172.21.15.82:0/2112188205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7ec0ff570 0x7fa7ec19c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:07:16.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.474+0000 7fa7ea59c700 1 --2- 172.21.15.82:0/2112188205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7ec0fff30 0x7fa7ec19c9f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0009990 tx=0x7fa7e0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:16.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.475+0000 7fa7d3fff700 1 -- 172.21.15.82:0/2112188205 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e001d070 con 0x7fa7ec0fff30 2023-09-09T23:07:16.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.475+0000 7fa7d3fff700 1 -- 172.21.15.82:0/2112188205 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7e0004010 con 0x7fa7ec0fff30 2023-09-09T23:07:16.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.475+0000 7fa7f1065700 1 -- 172.21.15.82:0/2112188205 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7ec197980 con 0x7fa7ec0fff30 2023-09-09T23:07:16.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.475+0000 7fa7d3fff700 1 -- 172.21.15.82:0/2112188205 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0022d40 con 0x7fa7ec0fff30 2023-09-09T23:07:16.481 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.475+0000 7fa7f1065700 1 -- 172.21.15.82:0/2112188205 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7ec1a34a0 con 0x7fa7ec0fff30 2023-09-09T23:07:16.481 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.475+0000 7fa7d3fff700 1 -- 172.21.15.82:0/2112188205 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7fa7e000f460 con 0x7fa7ec0fff30 2023-09-09T23:07:16.481 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.476+0000 7fa7d3fff700 1 --2- 172.21.15.82:0/2112188205 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa7d4038630 0x7fa7d403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:16.481 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.476+0000 7fa7f1065700 1 -- 172.21.15.82:0/2112188205 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7ec04e300 con 0x7fa7ec0fff30 2023-09-09T23:07:16.481 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.476+0000 7fa7ead9d700 1 --2- 172.21.15.82:0/2112188205 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa7d4038630 0x7fa7d403aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:16.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.476+0000 7fa7d3fff700 1 -- 172.21.15.82:0/2112188205 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa7e004c300 con 0x7fa7ec0fff30 2023-09-09T23:07:16.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.476+0000 7fa7ead9d700 1 --2- 172.21.15.82:0/2112188205 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa7d4038630 0x7fa7d403aaf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc005950 tx=0x7fa7dc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:16.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.480+0000 7fa7d3fff700 1 -- 172.21.15.82:0/2112188205 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa7e0027080 con 0x7fa7ec0fff30 2023-09-09T23:07:16.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.695+0000 7fa7f1065700 1 -- 172.21.15.82:0/2112188205 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7fa7ec10f520 con 0x7fa7ec0fff30 2023-09-09T23:07:16.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.702+0000 7fa7d3fff700 1 -- 172.21.15.82:0/2112188205 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v25)=0 v25) v1 ==== 121+0+0 (secure 0 0 0) 0x7fa7e0016990 con 0x7fa7ec0fff30 2023-09-09T23:07:16.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.706+0000 7fa7f1065700 1 -- 172.21.15.82:0/2112188205 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa7d4038630 msgr2=0x7fa7d403aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:16.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.706+0000 7fa7f1065700 1 --2- 172.21.15.82:0/2112188205 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa7d4038630 0x7fa7d403aaf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc005950 tx=0x7fa7dc0058e0 comp rx=0 tx=0).stop 2023-09-09T23:07:16.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.706+0000 7fa7f1065700 1 -- 172.21.15.82:0/2112188205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7ec0fff30 msgr2=0x7fa7ec19c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:16.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.706+0000 7fa7f1065700 1 --2- 172.21.15.82:0/2112188205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7ec0fff30 0x7fa7ec19c9f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0009990 tx=0x7fa7e0009580 comp rx=0 tx=0).stop 2023-09-09T23:07:16.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.706+0000 7fa7f1065700 1 -- 172.21.15.82:0/2112188205 shutdown_connections 2023-09-09T23:07:16.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.706+0000 7fa7f1065700 1 --2- 172.21.15.82:0/2112188205 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa7d4038630 0x7fa7d403aaf0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:16.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.706+0000 7fa7f1065700 1 --2- 172.21.15.82:0/2112188205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7ec0ff570 0x7fa7ec19c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:16.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.706+0000 7fa7f1065700 1 --2- 172.21.15.82:0/2112188205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7ec105610 0x7fa7ec196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:16.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.706+0000 7fa7f1065700 1 --2- 172.21.15.82:0/2112188205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7ec0fff30 0x7fa7ec19c9f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:16.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.706+0000 7fa7f1065700 1 -- 172.21.15.82:0/2112188205 >> 172.21.15.82:0/2112188205 conn(0x7fa7ec076680 msgr2=0x7fa7ec0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:16.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.706+0000 7fa7f1065700 1 -- 172.21.15.82:0/2112188205 shutdown_connections 2023-09-09T23:07:16.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:16.706+0000 7fa7f1065700 1 -- 172.21.15.82:0/2112188205 wait complete. 2023-09-09T23:07:16.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:16 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/452822771' entity='client.admin' 2023-09-09T23:07:17.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:16 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/452822771' entity='client.admin' 2023-09-09T23:07:17.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:16 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/452822771' entity='client.admin' 2023-09-09T23:07:17.386 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.387+0000 7facfacc5700 1 -- 172.21.15.82:0/3045908396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facf4101250 msgr2=0x7facf410d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:17.386 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.387+0000 7facfacc5700 1 --2- 172.21.15.82:0/3045908396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facf4101250 0x7facf410d4f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7face4009990 tx=0x7face4009ca0 comp rx=0 tx=0).stop 2023-09-09T23:07:17.386 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.387+0000 7facfacc5700 1 -- 172.21.15.82:0/3045908396 shutdown_connections 2023-09-09T23:07:17.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.387+0000 7facfacc5700 1 --2- 172.21.15.82:0/3045908396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facf4101250 0x7facf410d4f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:17.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.387+0000 7facfacc5700 1 --2- 172.21.15.82:0/3045908396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facf4106760 0x7facf4106b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:17.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.387+0000 7facfacc5700 1 --2- 172.21.15.82:0/3045908396 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facf4100890 0x7facf4100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:17.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.387+0000 7facfacc5700 1 -- 172.21.15.82:0/3045908396 >> 172.21.15.82:0/3045908396 conn(0x7facf4078620 msgr2=0x7facf4078a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:17.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.387+0000 7facfacc5700 1 -- 172.21.15.82:0/3045908396 shutdown_connections 2023-09-09T23:07:17.388 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.387+0000 7facfacc5700 1 -- 172.21.15.82:0/3045908396 wait complete. 2023-09-09T23:07:17.390 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.388+0000 7facfacc5700 1 Processor -- start 2023-09-09T23:07:17.390 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.388+0000 7facfacc5700 1 -- start start 2023-09-09T23:07:17.390 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.388+0000 7facfacc5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facf4100890 0x7facf4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:17.390 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.388+0000 7facfacc5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facf4101250 0x7facf4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:17.390 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.388+0000 7facfacc5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facf4106760 0x7facf4073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:17.391 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.388+0000 7facfacc5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf41a4b10 con 0x7facf4100890 2023-09-09T23:07:17.391 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.388+0000 7facfacc5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf41a4c80 con 0x7facf4101250 2023-09-09T23:07:17.391 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.388+0000 7facfacc5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf41a4e40 con 0x7facf4106760 2023-09-09T23:07:17.391 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.388+0000 7facf8a61700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facf4100890 0x7facf4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:17.391 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.388+0000 7facf8a61700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facf4100890 0x7facf4072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53472/0 (socket says 172.21.15.82:53472) 2023-09-09T23:07:17.392 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.388+0000 7facf8a61700 1 -- 172.21.15.82:0/2963707038 learned_addr learned my addr 172.21.15.82:0/2963707038 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:07:17.392 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.388+0000 7facf9262700 1 --2- 172.21.15.82:0/2963707038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facf4106760 0x7facf4073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:17.392 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.388+0000 7facf3fff700 1 --2- 172.21.15.82:0/2963707038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facf4101250 0x7facf4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:17.392 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.388+0000 7facf8a61700 1 -- 172.21.15.82:0/2963707038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facf4106760 msgr2=0x7facf4073380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:17.392 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.388+0000 7facf8a61700 1 --2- 172.21.15.82:0/2963707038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facf4106760 0x7facf4073380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:17.393 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.388+0000 7facf8a61700 1 -- 172.21.15.82:0/2963707038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facf4101250 msgr2=0x7facf4072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:17.393 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.388+0000 7facf8a61700 1 --2- 172.21.15.82:0/2963707038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facf4101250 0x7facf4072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:17.393 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.388+0000 7facf8a61700 1 -- 172.21.15.82:0/2963707038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face40095f0 con 0x7facf4100890 2023-09-09T23:07:17.393 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.388+0000 7facf3fff700 1 --2- 172.21.15.82:0/2963707038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facf4101250 0x7facf4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:07:17.393 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.388+0000 7facf8a61700 1 --2- 172.21.15.82:0/2963707038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facf4100890 0x7facf4072900 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7facec0060b0 tx=0x7facec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:17.393 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.389+0000 7facf1ffb700 1 -- 172.21.15.82:0/2963707038 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec00ddd0 con 0x7facf4100890 2023-09-09T23:07:17.394 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.389+0000 7facf1ffb700 1 -- 172.21.15.82:0/2963707038 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facec00f040 con 0x7facf4100890 2023-09-09T23:07:17.394 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.389+0000 7facfacc5700 1 -- 172.21.15.82:0/2963707038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf41a5120 con 0x7facf4100890 2023-09-09T23:07:17.394 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.389+0000 7facf1ffb700 1 -- 172.21.15.82:0/2963707038 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec014690 con 0x7facf4100890 2023-09-09T23:07:17.394 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.389+0000 7facfacc5700 1 -- 172.21.15.82:0/2963707038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf4196620 con 0x7facf4100890 2023-09-09T23:07:17.394 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.390+0000 7facfacc5700 1 -- 172.21.15.82:0/2963707038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facf404e300 con 0x7facf4100890 2023-09-09T23:07:17.395 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.390+0000 7facf1ffb700 1 -- 172.21.15.82:0/2963707038 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7facec015400 con 0x7facf4100890 2023-09-09T23:07:17.395 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.390+0000 7facf1ffb700 1 --2- 172.21.15.82:0/2963707038 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7face0040e40 0x7face0043300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:17.395 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.390+0000 7facf1ffb700 1 -- 172.21.15.82:0/2963707038 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7facec04a7e0 con 0x7facf4100890 2023-09-09T23:07:17.395 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.390+0000 7facf3fff700 1 --2- 172.21.15.82:0/2963707038 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7face0040e40 0x7face0043300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:17.395 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.391+0000 7facf3fff700 1 --2- 172.21.15.82:0/2963707038 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7face0040e40 0x7face0043300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7facdc005fd0 tx=0x7facdc005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:17.396 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.395+0000 7facf1ffb700 1 -- 172.21.15.82:0/2963707038 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7facec01c6b0 con 0x7facf4100890 2023-09-09T23:07:17.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.594+0000 7facfacc5700 1 -- 172.21.15.82:0/2963707038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) v1 -- 0x7facf4073de0 con 0x7facf4100890 2023-09-09T23:07:17.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.597+0000 7facf1ffb700 1 -- 172.21.15.82:0/2963707038 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v27)=0 v27) v1 ==== 119+0+0 (secure 0 0 0) 0x7facec01a030 con 0x7facf4100890 2023-09-09T23:07:17.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.599+0000 7facfacc5700 1 -- 172.21.15.82:0/2963707038 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7face0040e40 msgr2=0x7face0043300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:17.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.599+0000 7facfacc5700 1 --2- 172.21.15.82:0/2963707038 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7face0040e40 0x7face0043300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7facdc005fd0 tx=0x7facdc005ee0 comp rx=0 tx=0).stop 2023-09-09T23:07:17.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.599+0000 7facfacc5700 1 -- 172.21.15.82:0/2963707038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facf4100890 msgr2=0x7facf4072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:17.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.599+0000 7facfacc5700 1 --2- 172.21.15.82:0/2963707038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facf4100890 0x7facf4072900 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7facec0060b0 tx=0x7facec00d3b0 comp rx=0 tx=0).stop 2023-09-09T23:07:17.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.600+0000 7facfacc5700 1 -- 172.21.15.82:0/2963707038 shutdown_connections 2023-09-09T23:07:17.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.600+0000 7facfacc5700 1 --2- 172.21.15.82:0/2963707038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facf4106760 0x7facf4073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:17.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.600+0000 7facfacc5700 1 --2- 172.21.15.82:0/2963707038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facf4100890 0x7facf4072900 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:17.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.600+0000 7facfacc5700 1 --2- 172.21.15.82:0/2963707038 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7face0040e40 0x7face0043300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:17.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.600+0000 7facfacc5700 1 --2- 172.21.15.82:0/2963707038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facf4101250 0x7facf4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:17.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.600+0000 7facfacc5700 1 -- 172.21.15.82:0/2963707038 >> 172.21.15.82:0/2963707038 conn(0x7facf4078620 msgr2=0x7facf40ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:17.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.600+0000 7facfacc5700 1 -- 172.21.15.82:0/2963707038 shutdown_connections 2023-09-09T23:07:17.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.600+0000 7facfacc5700 1 -- 172.21.15.82:0/2963707038 wait complete. 2023-09-09T23:07:17.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:17 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2112188205' entity='client.admin' 2023-09-09T23:07:17.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:17 smithi082 ceph-mon[98694]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:17.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:17 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3183540871' entity='client.admin' 2023-09-09T23:07:17.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:17 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2963707038' entity='client.admin' 2023-09-09T23:07:18.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:17 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2112188205' entity='client.admin' 2023-09-09T23:07:18.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:17 smithi098 ceph-mon[103124]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:18.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:17 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3183540871' entity='client.admin' 2023-09-09T23:07:18.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:17 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2963707038' entity='client.admin' 2023-09-09T23:07:18.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.249+0000 7f19f2dca700 1 -- 172.21.15.82:0/3588372009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ec106760 msgr2=0x7f19ec106b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:18.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.249+0000 7f19f2dca700 1 --2- 172.21.15.82:0/3588372009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ec106760 0x7f19ec106b40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009990 tx=0x7f19e0009ca0 comp rx=0 tx=0).stop 2023-09-09T23:07:18.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.249+0000 7f19f2dca700 1 -- 172.21.15.82:0/3588372009 shutdown_connections 2023-09-09T23:07:18.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.249+0000 7f19f2dca700 1 --2- 172.21.15.82:0/3588372009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19ec101250 0x7f19ec10d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:18.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.249+0000 7f19f2dca700 1 --2- 172.21.15.82:0/3588372009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ec106760 0x7f19ec106b40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:18.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.249+0000 7f19f2dca700 1 --2- 172.21.15.82:0/3588372009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19ec100890 0x7f19ec100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:18.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.249+0000 7f19f2dca700 1 -- 172.21.15.82:0/3588372009 >> 172.21.15.82:0/3588372009 conn(0x7f19ec078620 msgr2=0x7f19ec078a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:18.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.249+0000 7f19f2dca700 1 -- 172.21.15.82:0/3588372009 shutdown_connections 2023-09-09T23:07:18.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.249+0000 7f19f2dca700 1 -- 172.21.15.82:0/3588372009 wait complete. 2023-09-09T23:07:18.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.250+0000 7f19f2dca700 1 Processor -- start 2023-09-09T23:07:18.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.250+0000 7f19f2dca700 1 -- start start 2023-09-09T23:07:18.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.250+0000 7f19f2dca700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19ec100890 0x7f19ec1966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:18.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.250+0000 7f19f2dca700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19ec101250 0x7f19ec196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:18.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.250+0000 7f19f2dca700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ec106760 0x7f19ec199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:18.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.250+0000 7f19f2dca700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec19a5c0 con 0x7f19ec106760 2023-09-09T23:07:18.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.250+0000 7f19f2dca700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec19a730 con 0x7f19ec100890 2023-09-09T23:07:18.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.250+0000 7f19f2dca700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec19a8f0 con 0x7f19ec101250 2023-09-09T23:07:18.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.250+0000 7f19f1367700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ec106760 0x7f19ec199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:18.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.250+0000 7f19f1367700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ec106760 0x7f19ec199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53460/0 (socket says 172.21.15.82:53460) 2023-09-09T23:07:18.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.250+0000 7f19f1367700 1 -- 172.21.15.82:0/3183540871 learned_addr learned my addr 172.21.15.82:0/3183540871 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:07:18.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.250+0000 7f19ebfff700 1 --2- 172.21.15.82:0/3183540871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19ec101250 0x7f19ec196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:18.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.250+0000 7f19f0b66700 1 --2- 172.21.15.82:0/3183540871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19ec100890 0x7f19ec1966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:18.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.250+0000 7f19f1367700 1 -- 172.21.15.82:0/3183540871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19ec101250 msgr2=0x7f19ec196c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:18.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.250+0000 7f19f1367700 1 --2- 172.21.15.82:0/3183540871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19ec101250 0x7f19ec196c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:18.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.250+0000 7f19f1367700 1 -- 172.21.15.82:0/3183540871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19ec100890 msgr2=0x7f19ec1966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:18.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.250+0000 7f19f1367700 1 --2- 172.21.15.82:0/3183540871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19ec100890 0x7f19ec1966c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:18.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.250+0000 7f19f1367700 1 -- 172.21.15.82:0/3183540871 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e00095f0 con 0x7f19ec106760 2023-09-09T23:07:18.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.250+0000 7f19f0b66700 1 --2- 172.21.15.82:0/3183540871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19ec100890 0x7f19ec1966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:07:18.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.250+0000 7f19ebfff700 1 --2- 172.21.15.82:0/3183540871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19ec101250 0x7f19ec196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:07:18.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.251+0000 7f19f1367700 1 --2- 172.21.15.82:0/3183540871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ec106760 0x7f19ec199fc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f19e400ef30 tx=0x7f19e400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:18.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.251+0000 7f19e9ffb700 1 -- 172.21.15.82:0/3183540871 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4018070 con 0x7f19ec106760 2023-09-09T23:07:18.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.251+0000 7f19e9ffb700 1 -- 172.21.15.82:0/3183540871 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19e400f040 con 0x7f19ec106760 2023-09-09T23:07:18.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.251+0000 7f19f2dca700 1 -- 172.21.15.82:0/3183540871 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ec19ab70 con 0x7f19ec106760 2023-09-09T23:07:18.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.251+0000 7f19e9ffb700 1 -- 172.21.15.82:0/3183540871 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4003dc0 con 0x7f19ec106760 2023-09-09T23:07:18.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.251+0000 7f19f2dca700 1 -- 172.21.15.82:0/3183540871 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ec1a3520 con 0x7f19ec106760 2023-09-09T23:07:18.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.252+0000 7f19e9ffb700 1 -- 172.21.15.82:0/3183540871 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f19e400caa0 con 0x7f19ec106760 2023-09-09T23:07:18.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.252+0000 7f19e9ffb700 1 --2- 172.21.15.82:0/3183540871 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f19dc038630 0x7f19dc03aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:18.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.252+0000 7f19f2dca700 1 -- 172.21.15.82:0/3183540871 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ec04e300 con 0x7f19ec106760 2023-09-09T23:07:18.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.252+0000 7f19e9ffb700 1 -- 172.21.15.82:0/3183540871 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f19e404aa90 con 0x7f19ec106760 2023-09-09T23:07:18.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.252+0000 7f19f0b66700 1 --2- 172.21.15.82:0/3183540871 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f19dc038630 0x7f19dc03aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:18.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.252+0000 7f19f0b66700 1 --2- 172.21.15.82:0/3183540871 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f19dc038630 0x7f19dc03aaf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f19e000b2e0 tx=0x7f19e0005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:18.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.257+0000 7f19e9ffb700 1 -- 172.21.15.82:0/3183540871 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f19e400cdf0 con 0x7f19ec106760 2023-09-09T23:07:18.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.461+0000 7f19f2dca700 1 -- 172.21.15.82:0/3183540871 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) v1 -- 0x7f19ec197a20 con 0x7f19ec106760 2023-09-09T23:07:18.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.464+0000 7f19e9ffb700 1 -- 172.21.15.82:0/3183540871 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v26)=0 v26) v1 ==== 115+0+0 (secure 0 0 0) 0x7f19e40133e0 con 0x7f19ec106760 2023-09-09T23:07:18.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.466+0000 7f19f2dca700 1 -- 172.21.15.82:0/3183540871 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f19dc038630 msgr2=0x7f19dc03aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:18.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.466+0000 7f19f2dca700 1 --2- 172.21.15.82:0/3183540871 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f19dc038630 0x7f19dc03aaf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f19e000b2e0 tx=0x7f19e0005e80 comp rx=0 tx=0).stop 2023-09-09T23:07:18.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.466+0000 7f19f2dca700 1 -- 172.21.15.82:0/3183540871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ec106760 msgr2=0x7f19ec199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:18.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.466+0000 7f19f2dca700 1 --2- 172.21.15.82:0/3183540871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ec106760 0x7f19ec199fc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f19e400ef30 tx=0x7f19e400c450 comp rx=0 tx=0).stop 2023-09-09T23:07:18.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.466+0000 7f19f2dca700 1 -- 172.21.15.82:0/3183540871 shutdown_connections 2023-09-09T23:07:18.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.466+0000 7f19f2dca700 1 --2- 172.21.15.82:0/3183540871 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f19dc038630 0x7f19dc03aaf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:18.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.466+0000 7f19f2dca700 1 --2- 172.21.15.82:0/3183540871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19ec100890 0x7f19ec1966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:18.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.466+0000 7f19f2dca700 1 --2- 172.21.15.82:0/3183540871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ec106760 0x7f19ec199fc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:18.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.466+0000 7f19f2dca700 1 --2- 172.21.15.82:0/3183540871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19ec101250 0x7f19ec196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:18.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.466+0000 7f19f2dca700 1 -- 172.21.15.82:0/3183540871 >> 172.21.15.82:0/3183540871 conn(0x7f19ec078620 msgr2=0x7f19ec0ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:18.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.466+0000 7f19f2dca700 1 -- 172.21.15.82:0/3183540871 shutdown_connections 2023-09-09T23:07:18.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:17.466+0000 7f19f2dca700 1 -- 172.21.15.82:0/3183540871 wait complete. 2023-09-09T23:07:18.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:17 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2112188205' entity='client.admin' 2023-09-09T23:07:18.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:17 smithi170 ceph-mon[102832]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:18.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:17 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3183540871' entity='client.admin' 2023-09-09T23:07:18.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:17 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2963707038' entity='client.admin' 2023-09-09T23:07:18.805 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.729+0000 7f34347c7700 1 -- 172.21.15.82:0/2199505081 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f342c0ffee0 msgr2=0x7f342c10c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:18.805 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.729+0000 7f34347c7700 1 --2- 172.21.15.82:0/2199505081 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f342c0ffee0 0x7f342c10c3f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f342400b0a0 tx=0x7f342400b3b0 comp rx=0 tx=0).stop 2023-09-09T23:07:18.805 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.729+0000 7f34347c7700 1 -- 172.21.15.82:0/2199505081 shutdown_connections 2023-09-09T23:07:18.805 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.729+0000 7f34347c7700 1 --2- 172.21.15.82:0/2199505081 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f342c0ffee0 0x7f342c10c3f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:18.806 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.729+0000 7f34347c7700 1 --2- 172.21.15.82:0/2199505081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f342c104530 0x7f342c104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:18.806 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.729+0000 7f34347c7700 1 --2- 172.21.15.82:0/2199505081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f342c0ff520 0x7f342c0ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:18.806 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.729+0000 7f34347c7700 1 -- 172.21.15.82:0/2199505081 >> 172.21.15.82:0/2199505081 conn(0x7f342c076600 msgr2=0x7f342c076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:18.806 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.729+0000 7f34347c7700 1 -- 172.21.15.82:0/2199505081 shutdown_connections 2023-09-09T23:07:18.807 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.729+0000 7f34347c7700 1 -- 172.21.15.82:0/2199505081 wait complete. 2023-09-09T23:07:18.807 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.730+0000 7f34347c7700 1 Processor -- start 2023-09-09T23:07:18.807 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.730+0000 7f34347c7700 1 -- start start 2023-09-09T23:07:18.807 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.730+0000 7f34347c7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f342c0ff520 0x7f342c19c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:18.807 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.730+0000 7f34347c7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f342c0ffee0 0x7f342c19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:18.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.730+0000 7f34347c7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f342c104530 0x7f342c196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:18.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.730+0000 7f34347c7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c196db0 con 0x7f342c0ffee0 2023-09-09T23:07:18.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.730+0000 7f34347c7700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c196f20 con 0x7f342c104530 2023-09-09T23:07:18.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.730+0000 7f34347c7700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c1970e0 con 0x7f342c0ff520 2023-09-09T23:07:18.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.730+0000 7f3431d62700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f342c0ffee0 0x7f342c19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:18.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.730+0000 7f3431d62700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f342c0ffee0 0x7f342c19cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53484/0 (socket says 172.21.15.82:53484) 2023-09-09T23:07:18.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.730+0000 7f3431d62700 1 -- 172.21.15.82:0/234892624 learned_addr learned my addr 172.21.15.82:0/234892624 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:07:18.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.730+0000 7f3432563700 1 --2- 172.21.15.82:0/234892624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f342c0ff520 0x7f342c19c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:18.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.730+0000 7f3432d64700 1 --2- 172.21.15.82:0/234892624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f342c104530 0x7f342c196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:18.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.730+0000 7f3431d62700 1 -- 172.21.15.82:0/234892624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f342c0ff520 msgr2=0x7f342c19c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:18.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.730+0000 7f3431d62700 1 --2- 172.21.15.82:0/234892624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f342c0ff520 0x7f342c19c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:18.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.730+0000 7f3431d62700 1 -- 172.21.15.82:0/234892624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f342c104530 msgr2=0x7f342c196600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:18.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.730+0000 7f3431d62700 1 --2- 172.21.15.82:0/234892624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f342c104530 0x7f342c196600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:18.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.730+0000 7f3431d62700 1 -- 172.21.15.82:0/234892624 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3424009d00 con 0x7f342c0ffee0 2023-09-09T23:07:18.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.730+0000 7f3432d64700 1 --2- 172.21.15.82:0/234892624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f342c104530 0x7f342c196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:07:18.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.730+0000 7f3432563700 1 --2- 172.21.15.82:0/234892624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f342c0ff520 0x7f342c19c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:07:18.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.731+0000 7f3431d62700 1 --2- 172.21.15.82:0/234892624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f342c0ffee0 0x7f342c19cb50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f341c00d950 tx=0x7f341c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:18.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.731+0000 7f34237fe700 1 -- 172.21.15.82:0/234892624 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c009b40 con 0x7f342c0ffee0 2023-09-09T23:07:18.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.731+0000 7f34347c7700 1 -- 172.21.15.82:0/234892624 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f342c1973c0 con 0x7f342c0ffee0 2023-09-09T23:07:18.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.731+0000 7f34237fe700 1 -- 172.21.15.82:0/234892624 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f341c009ca0 con 0x7f342c0ffee0 2023-09-09T23:07:18.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.731+0000 7f34237fe700 1 -- 172.21.15.82:0/234892624 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c00f620 con 0x7f342c0ffee0 2023-09-09T23:07:18.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.731+0000 7f34347c7700 1 -- 172.21.15.82:0/234892624 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f342c101280 con 0x7f342c0ffee0 2023-09-09T23:07:18.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.732+0000 7f34237fe700 1 -- 172.21.15.82:0/234892624 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f341c00f780 con 0x7f342c0ffee0 2023-09-09T23:07:18.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.732+0000 7f34237fe700 1 --2- 172.21.15.82:0/234892624 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3418038630 0x7f341803aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:18.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.732+0000 7f34237fe700 1 -- 172.21.15.82:0/234892624 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f341c04b1b0 con 0x7f342c0ffee0 2023-09-09T23:07:18.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.732+0000 7f34347c7700 1 -- 172.21.15.82:0/234892624 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f342c04e300 con 0x7f342c0ffee0 2023-09-09T23:07:18.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.732+0000 7f3432563700 1 --2- 172.21.15.82:0/234892624 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3418038630 0x7f341803aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:18.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.732+0000 7f3432563700 1 --2- 172.21.15.82:0/234892624 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3418038630 0x7f341803aaf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f342c069c10 tx=0x7f3428005f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:18.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.737+0000 7f34237fe700 1 -- 172.21.15.82:0/234892624 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f341c016070 con 0x7f342c0ffee0 2023-09-09T23:07:18.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.933+0000 7f34347c7700 1 -- 172.21.15.82:0/234892624 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) v1 -- 0x7f342c0008d0 con 0x7f342c0ffee0 2023-09-09T23:07:18.934 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.936+0000 7f34237fe700 1 -- 172.21.15.82:0/234892624 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v28)=0 v28) v1 ==== 120+0+0 (secure 0 0 0) 0x7f341c024490 con 0x7f342c0ffee0 2023-09-09T23:07:18.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.938+0000 7f34347c7700 1 -- 172.21.15.82:0/234892624 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3418038630 msgr2=0x7f341803aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:18.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.938+0000 7f34347c7700 1 --2- 172.21.15.82:0/234892624 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3418038630 0x7f341803aaf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f342c069c10 tx=0x7f3428005f70 comp rx=0 tx=0).stop 2023-09-09T23:07:18.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.939+0000 7f34347c7700 1 -- 172.21.15.82:0/234892624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f342c0ffee0 msgr2=0x7f342c19cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:18.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.939+0000 7f34347c7700 1 --2- 172.21.15.82:0/234892624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f342c0ffee0 0x7f342c19cb50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f341c00d950 tx=0x7f341c00dd10 comp rx=0 tx=0).stop 2023-09-09T23:07:18.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.939+0000 7f34347c7700 1 -- 172.21.15.82:0/234892624 shutdown_connections 2023-09-09T23:07:18.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.939+0000 7f34347c7700 1 --2- 172.21.15.82:0/234892624 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3418038630 0x7f341803aaf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:18.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.939+0000 7f34347c7700 1 --2- 172.21.15.82:0/234892624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f342c104530 0x7f342c196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:18.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.939+0000 7f34347c7700 1 --2- 172.21.15.82:0/234892624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f342c0ff520 0x7f342c19c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:18.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.939+0000 7f34347c7700 1 --2- 172.21.15.82:0/234892624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f342c0ffee0 0x7f342c19cb50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:18.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.939+0000 7f34347c7700 1 -- 172.21.15.82:0/234892624 >> 172.21.15.82:0/234892624 conn(0x7f342c076600 msgr2=0x7f342c0680c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:18.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.939+0000 7f34347c7700 1 -- 172.21.15.82:0/234892624 shutdown_connections 2023-09-09T23:07:18.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:18.939+0000 7f34347c7700 1 -- 172.21.15.82:0/234892624 wait complete. 2023-09-09T23:07:19.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.213+0000 7f3bb1c1a700 1 -- 172.21.15.82:0/2668931843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bac068ae0 msgr2=0x7f3bac10d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:19.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.213+0000 7f3bb1c1a700 1 --2- 172.21.15.82:0/2668931843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bac068ae0 0x7f3bac10d440 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400b0a0 tx=0x7f3ba400b3b0 comp rx=0 tx=0).stop 2023-09-09T23:07:19.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.213+0000 7f3bb1c1a700 1 -- 172.21.15.82:0/2668931843 shutdown_connections 2023-09-09T23:07:19.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.213+0000 7f3bb1c1a700 1 --2- 172.21.15.82:0/2668931843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bac106770 0x7f3bac106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:19.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.213+0000 7f3bb1c1a700 1 --2- 172.21.15.82:0/2668931843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bac068ae0 0x7f3bac10d440 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:19.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.213+0000 7f3bb1c1a700 1 --2- 172.21.15.82:0/2668931843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bac068120 0x7f3bac0685a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:19.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.213+0000 7f3bb1c1a700 1 -- 172.21.15.82:0/2668931843 >> 172.21.15.82:0/2668931843 conn(0x7f3bac076600 msgr2=0x7f3bac076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:19.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.213+0000 7f3bb1c1a700 1 -- 172.21.15.82:0/2668931843 shutdown_connections 2023-09-09T23:07:19.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.213+0000 7f3bb1c1a700 1 -- 172.21.15.82:0/2668931843 wait complete. 2023-09-09T23:07:19.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.214+0000 7f3bb1c1a700 1 Processor -- start 2023-09-09T23:07:19.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.214+0000 7f3bb1c1a700 1 -- start start 2023-09-09T23:07:19.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.214+0000 7f3bb1c1a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bac068120 0x7f3bac0ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:19.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.214+0000 7f3bb1c1a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bac068ae0 0x7f3bac0ffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:19.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.214+0000 7f3bb1c1a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bac106770 0x7f3bac100070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:19.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.214+0000 7f3bb1c1a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bac1a4d00 con 0x7f3bac106770 2023-09-09T23:07:19.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.214+0000 7f3bb1c1a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bac1a4e70 con 0x7f3bac068120 2023-09-09T23:07:19.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.214+0000 7f3bb1c1a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bac1a5030 con 0x7f3bac068ae0 2023-09-09T23:07:19.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.214+0000 7f3babfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bac106770 0x7f3bac100070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:19.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.214+0000 7f3babfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bac106770 0x7f3bac100070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53498/0 (socket says 172.21.15.82:53498) 2023-09-09T23:07:19.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.214+0000 7f3babfff700 1 -- 172.21.15.82:0/266669259 learned_addr learned my addr 172.21.15.82:0/266669259 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:07:19.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.214+0000 7f3baaffd700 1 --2- 172.21.15.82:0/266669259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bac068ae0 0x7f3bac0ffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:19.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.214+0000 7f3bab7fe700 1 --2- 172.21.15.82:0/266669259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bac068120 0x7f3bac0ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:19.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.214+0000 7f3babfff700 1 -- 172.21.15.82:0/266669259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bac068ae0 msgr2=0x7f3bac0ffaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:19.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.214+0000 7f3babfff700 1 --2- 172.21.15.82:0/266669259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bac068ae0 0x7f3bac0ffaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:19.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.215+0000 7f3babfff700 1 -- 172.21.15.82:0/266669259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bac068120 msgr2=0x7f3bac0ff5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:19.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.215+0000 7f3babfff700 1 --2- 172.21.15.82:0/266669259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bac068120 0x7f3bac0ff5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:19.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.215+0000 7f3babfff700 1 -- 172.21.15.82:0/266669259 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba4009d00 con 0x7f3bac106770 2023-09-09T23:07:19.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.215+0000 7f3babfff700 1 --2- 172.21.15.82:0/266669259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bac106770 0x7f3bac100070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400b830 tx=0x7f3ba40078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:19.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.215+0000 7f3ba8ff9700 1 -- 172.21.15.82:0/266669259 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba400e040 con 0x7f3bac106770 2023-09-09T23:07:19.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.215+0000 7f3ba8ff9700 1 -- 172.21.15.82:0/266669259 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ba4004ce0 con 0x7f3bac106770 2023-09-09T23:07:19.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.215+0000 7f3bb1c1a700 1 -- 172.21.15.82:0/266669259 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bac1a52b0 con 0x7f3bac106770 2023-09-09T23:07:19.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.215+0000 7f3ba8ff9700 1 -- 172.21.15.82:0/266669259 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba401d480 con 0x7f3bac106770 2023-09-09T23:07:19.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.215+0000 7f3bb1c1a700 1 -- 172.21.15.82:0/266669259 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bac1a5770 con 0x7f3bac106770 2023-09-09T23:07:19.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.216+0000 7f3ba8ff9700 1 -- 172.21.15.82:0/266669259 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f3ba401d5e0 con 0x7f3bac106770 2023-09-09T23:07:19.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.216+0000 7f3ba8ff9700 1 --2- 172.21.15.82:0/266669259 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3b94038630 0x7f3b9403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:19.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.216+0000 7f3ba8ff9700 1 -- 172.21.15.82:0/266669259 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3ba404c520 con 0x7f3bac106770 2023-09-09T23:07:19.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.216+0000 7f3bab7fe700 1 --2- 172.21.15.82:0/266669259 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3b94038630 0x7f3b9403aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:19.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.216+0000 7f3bb1c1a700 1 -- 172.21.15.82:0/266669259 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bac04e300 con 0x7f3bac106770 2023-09-09T23:07:19.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.216+0000 7f3bab7fe700 1 --2- 172.21.15.82:0/266669259 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3b94038630 0x7f3b9403aaf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c005950 tx=0x7f3b9c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:19.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.221+0000 7f3ba8ff9700 1 -- 172.21.15.82:0/266669259 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3ba4026460 con 0x7f3bac106770 2023-09-09T23:07:19.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.272+0000 7f8049631700 1 -- 172.21.15.82:0/3229644710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f804410d090 msgr2=0x7f804410f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:19.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.272+0000 7f8049631700 1 --2- 172.21.15.82:0/3229644710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f804410d090 0x7f804410f480 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f803c00b0a0 tx=0x7f803c00b3b0 comp rx=0 tx=0).stop 2023-09-09T23:07:19.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.272+0000 7f8049631700 1 -- 172.21.15.82:0/3229644710 shutdown_connections 2023-09-09T23:07:19.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.272+0000 7f8049631700 1 --2- 172.21.15.82:0/3229644710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8044100020 0x7f8044100400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:19.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.273+0000 7f8049631700 1 --2- 172.21.15.82:0/3229644710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f804410d090 0x7f804410f480 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:19.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.273+0000 7f8049631700 1 --2- 172.21.15.82:0/3229644710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8044100940 0x7f804410cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:19.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.273+0000 7f8049631700 1 -- 172.21.15.82:0/3229644710 >> 172.21.15.82:0/3229644710 conn(0x7f8044076600 msgr2=0x7f8044076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:19.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.273+0000 7f8049631700 1 -- 172.21.15.82:0/3229644710 shutdown_connections 2023-09-09T23:07:19.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.273+0000 7f8049631700 1 -- 172.21.15.82:0/3229644710 wait complete. 2023-09-09T23:07:19.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.273+0000 7f8049631700 1 Processor -- start 2023-09-09T23:07:19.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.274+0000 7f8049631700 1 -- start start 2023-09-09T23:07:19.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.274+0000 7f8049631700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8044100020 0x7f804419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:19.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.274+0000 7f8049631700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8044100940 0x7f804419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:19.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.274+0000 7f8049631700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f804410d090 0x7f8044198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:19.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.274+0000 7f8049631700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8044198ef0 con 0x7f804410d090 2023-09-09T23:07:19.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.274+0000 7f8049631700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8044199060 con 0x7f8044100020 2023-09-09T23:07:19.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.274+0000 7f8049631700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8044199220 con 0x7f8044100940 2023-09-09T23:07:19.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.274+0000 7f80437fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f804410d090 0x7f8044198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:19.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.274+0000 7f80437fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f804410d090 0x7f8044198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53512/0 (socket says 172.21.15.82:53512) 2023-09-09T23:07:19.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.274+0000 7f80437fe700 1 -- 172.21.15.82:0/4088791567 learned_addr learned my addr 172.21.15.82:0/4088791567 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:07:19.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.274+0000 7f8042ffd700 1 --2- 172.21.15.82:0/4088791567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8044100020 0x7f804419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:19.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.274+0000 7f80427fc700 1 --2- 172.21.15.82:0/4088791567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8044100940 0x7f804419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:19.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.274+0000 7f80437fe700 1 -- 172.21.15.82:0/4088791567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8044100940 msgr2=0x7f804419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:19.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.274+0000 7f80437fe700 1 --2- 172.21.15.82:0/4088791567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8044100940 0x7f804419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:19.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.274+0000 7f80437fe700 1 -- 172.21.15.82:0/4088791567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8044100020 msgr2=0x7f804419e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:19.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.274+0000 7f80437fe700 1 --2- 172.21.15.82:0/4088791567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8044100020 0x7f804419e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:19.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.274+0000 7f80437fe700 1 -- 172.21.15.82:0/4088791567 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803c009d00 con 0x7f804410d090 2023-09-09T23:07:19.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.274+0000 7f80427fc700 1 --2- 172.21.15.82:0/4088791567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8044100940 0x7f804419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:07:19.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.274+0000 7f80437fe700 1 --2- 172.21.15.82:0/4088791567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f804410d090 0x7f8044198880 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f803c00b780 tx=0x7f803c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:19.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.275+0000 7f802bfff700 1 -- 172.21.15.82:0/4088791567 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c00e040 con 0x7f804410d090 2023-09-09T23:07:19.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.275+0000 7f8049631700 1 -- 172.21.15.82:0/4088791567 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80441994a0 con 0x7f804410d090 2023-09-09T23:07:19.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.275+0000 7f802bfff700 1 -- 172.21.15.82:0/4088791567 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f803c003c00 con 0x7f804410d090 2023-09-09T23:07:19.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.275+0000 7f802bfff700 1 -- 172.21.15.82:0/4088791567 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c01b6c0 con 0x7f804410d090 2023-09-09T23:07:19.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.275+0000 7f8049631700 1 -- 172.21.15.82:0/4088791567 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80441a57f0 con 0x7f804410d090 2023-09-09T23:07:19.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.276+0000 7f802bfff700 1 -- 172.21.15.82:0/4088791567 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f803c01b8e0 con 0x7f804410d090 2023-09-09T23:07:19.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.276+0000 7f8049631700 1 -- 172.21.15.82:0/4088791567 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f804404e300 con 0x7f804410d090 2023-09-09T23:07:19.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.276+0000 7f802bfff700 1 --2- 172.21.15.82:0/4088791567 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f802c038630 0x7f802c03aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:19.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.276+0000 7f802bfff700 1 -- 172.21.15.82:0/4088791567 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f803c04bf40 con 0x7f804410d090 2023-09-09T23:07:19.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.276+0000 7f8042ffd700 1 --2- 172.21.15.82:0/4088791567 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f802c038630 0x7f802c03aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:19.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.276+0000 7f8042ffd700 1 --2- 172.21.15.82:0/4088791567 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f802c038630 0x7f802c03aaf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8034005fd0 tx=0x7f8034005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:19.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.285+0000 7f802bfff700 1 -- 172.21.15.82:0/4088791567 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f803c011390 con 0x7f804410d090 2023-09-09T23:07:19.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.422+0000 7f3bb1c1a700 1 -- 172.21.15.82:0/266669259 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) v1 -- 0x7f3bac110620 con 0x7f3bac106770 2023-09-09T23:07:19.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.425+0000 7f3ba8ff9700 1 -- 172.21.15.82:0/266669259 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v29)=0 v29) v1 ==== 121+0+0 (secure 0 0 0) 0x7f3ba40350c0 con 0x7f3bac106770 2023-09-09T23:07:19.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.428+0000 7f3bb1c1a700 1 -- 172.21.15.82:0/266669259 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3b94038630 msgr2=0x7f3b9403aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:19.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.428+0000 7f3bb1c1a700 1 --2- 172.21.15.82:0/266669259 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3b94038630 0x7f3b9403aaf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c005950 tx=0x7f3b9c0058e0 comp rx=0 tx=0).stop 2023-09-09T23:07:19.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.428+0000 7f3bb1c1a700 1 -- 172.21.15.82:0/266669259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bac106770 msgr2=0x7f3bac100070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:19.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.428+0000 7f3bb1c1a700 1 --2- 172.21.15.82:0/266669259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bac106770 0x7f3bac100070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400b830 tx=0x7f3ba40078c0 comp rx=0 tx=0).stop 2023-09-09T23:07:19.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.428+0000 7f3bb1c1a700 1 -- 172.21.15.82:0/266669259 shutdown_connections 2023-09-09T23:07:19.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.428+0000 7f3bb1c1a700 1 --2- 172.21.15.82:0/266669259 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3b94038630 0x7f3b9403aaf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:19.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.428+0000 7f3bb1c1a700 1 --2- 172.21.15.82:0/266669259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bac068120 0x7f3bac0ff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:19.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.428+0000 7f3bb1c1a700 1 --2- 172.21.15.82:0/266669259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bac106770 0x7f3bac100070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:19.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.428+0000 7f3bb1c1a700 1 --2- 172.21.15.82:0/266669259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bac068ae0 0x7f3bac0ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:19.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.428+0000 7f3bb1c1a700 1 -- 172.21.15.82:0/266669259 >> 172.21.15.82:0/266669259 conn(0x7f3bac076600 msgr2=0x7f3bac0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:19.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.428+0000 7f3bb1c1a700 1 -- 172.21.15.82:0/266669259 shutdown_connections 2023-09-09T23:07:19.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.428+0000 7f3bb1c1a700 1 -- 172.21.15.82:0/266669259 wait complete. 2023-09-09T23:07:19.491 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.492+0000 7f8049631700 1 -- 172.21.15.82:0/4088791567 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7f8044199d90 con 0x7f804410d090 2023-09-09T23:07:19.495 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.496+0000 7f802bfff700 1 -- 172.21.15.82:0/4088791567 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v30)=0 v30) v1 ==== 121+0+0 (secure 0 0 0) 0x7f803c025b20 con 0x7f804410d090 2023-09-09T23:07:19.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.500+0000 7f8049631700 1 -- 172.21.15.82:0/4088791567 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f802c038630 msgr2=0x7f802c03aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:19.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.500+0000 7f8049631700 1 --2- 172.21.15.82:0/4088791567 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f802c038630 0x7f802c03aaf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8034005fd0 tx=0x7f8034005f60 comp rx=0 tx=0).stop 2023-09-09T23:07:19.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.500+0000 7f8049631700 1 -- 172.21.15.82:0/4088791567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f804410d090 msgr2=0x7f8044198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:19.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.500+0000 7f8049631700 1 --2- 172.21.15.82:0/4088791567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f804410d090 0x7f8044198880 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f803c00b780 tx=0x7f803c007a40 comp rx=0 tx=0).stop 2023-09-09T23:07:19.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.500+0000 7f8049631700 1 -- 172.21.15.82:0/4088791567 shutdown_connections 2023-09-09T23:07:19.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.500+0000 7f8049631700 1 --2- 172.21.15.82:0/4088791567 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f802c038630 0x7f802c03aaf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:19.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.500+0000 7f8049631700 1 --2- 172.21.15.82:0/4088791567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8044100020 0x7f804419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:19.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.500+0000 7f8049631700 1 --2- 172.21.15.82:0/4088791567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f804410d090 0x7f8044198880 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:19.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.500+0000 7f8049631700 1 --2- 172.21.15.82:0/4088791567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8044100940 0x7f804419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:19.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.500+0000 7f8049631700 1 -- 172.21.15.82:0/4088791567 >> 172.21.15.82:0/4088791567 conn(0x7f8044076600 msgr2=0x7f804410e400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:19.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.500+0000 7f8049631700 1 -- 172.21.15.82:0/4088791567 shutdown_connections 2023-09-09T23:07:19.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:19.500+0000 7f8049631700 1 -- 172.21.15.82:0/4088791567 wait complete. 2023-09-09T23:07:20.155 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:19 smithi082 ceph-mon[98694]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:20.155 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:19 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/234892624' entity='client.admin' 2023-09-09T23:07:20.155 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:19 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/266669259' entity='client.admin' 2023-09-09T23:07:20.156 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:19 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/4088791567' entity='client.admin' 2023-09-09T23:07:20.183 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-09-09T23:07:20.184 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-09-09T23:07:20.184 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.conf 2023-09-09T23:07:20.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:19 smithi098 ceph-mon[103124]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:20.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:19 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/234892624' entity='client.admin' 2023-09-09T23:07:20.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:19 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/266669259' entity='client.admin' 2023-09-09T23:07:20.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:19 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/4088791567' entity='client.admin' 2023-09-09T23:07:20.253 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-09-09T23:07:20.254 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-09T23:07:20.320 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-09-09T23:07:20.320 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/ceph/ceph.conf 2023-09-09T23:07:20.358 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-09-09T23:07:20.359 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-09T23:07:20.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:19 smithi170 ceph-mon[102832]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:20.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:19 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/234892624' entity='client.admin' 2023-09-09T23:07:20.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:19 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/266669259' entity='client.admin' 2023-09-09T23:07:20.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:19 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/4088791567' entity='client.admin' 2023-09-09T23:07:20.429 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-09-09T23:07:20.429 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.conf 2023-09-09T23:07:20.466 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-09-09T23:07:20.467 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-09T23:07:20.536 INFO:tasks.cephadm:Adding mgr.x on smithi082 2023-09-09T23:07:20.537 INFO:tasks.cephadm:Adding mgr.y on smithi098 2023-09-09T23:07:20.537 INFO:tasks.cephadm:Adding mgr.z on smithi170 2023-09-09T23:07:20.538 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph orch apply mgr '3;smithi082=x;smithi098=y;smithi170=z' 2023-09-09T23:07:21.130 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-09T23:07:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.180+0000 7f3afa05a700 1 -- 172.21.15.170:0/1828629182 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3af4101130 msgr2=0x7f3af41079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.180+0000 7f3afa05a700 1 --2- 172.21.15.170:0/1828629182 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3af4101130 0x7f3af41079e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3aec00b0a0 tx=0x7f3aec00b3b0 comp rx=0 tx=0).stop 2023-09-09T23:07:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.180+0000 7f3afa05a700 1 -- 172.21.15.170:0/1828629182 shutdown_connections 2023-09-09T23:07:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.180+0000 7f3afa05a700 1 --2- 172.21.15.170:0/1828629182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3af40ff590 0x7f3af40ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.180+0000 7f3afa05a700 1 --2- 172.21.15.170:0/1828629182 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3af4101130 0x7f3af41079e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.180+0000 7f3afa05a700 1 --2- 172.21.15.170:0/1828629182 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3af4100790 0x7f3af4100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.180+0000 7f3afa05a700 1 -- 172.21.15.170:0/1828629182 >> 172.21.15.170:0/1828629182 conn(0x7f3af40fab10 msgr2=0x7f3af40fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.180+0000 7f3afa05a700 1 -- 172.21.15.170:0/1828629182 shutdown_connections 2023-09-09T23:07:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.180+0000 7f3afa05a700 1 -- 172.21.15.170:0/1828629182 wait complete. 2023-09-09T23:07:22.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.181+0000 7f3afa05a700 1 Processor -- start 2023-09-09T23:07:22.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.181+0000 7f3afa05a700 1 -- start start 2023-09-09T23:07:22.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.181+0000 7f3afa05a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3af40ff590 0x7f3af4196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:22.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.181+0000 7f3afa05a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3af4100790 0x7f3af4196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:22.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.181+0000 7f3afa05a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3af4101130 0x7f3af419de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:22.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.182+0000 7f3afa05a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af419e3e0 con 0x7f3af4100790 2023-09-09T23:07:22.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.182+0000 7f3afa05a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af419e520 con 0x7f3af40ff590 2023-09-09T23:07:22.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.182+0000 7f3afa05a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af419e6b0 con 0x7f3af4101130 2023-09-09T23:07:22.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.182+0000 7f3af3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3af4101130 0x7f3af419de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:22.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.182+0000 7f3af3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3af4101130 0x7f3af419de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34338/0 (socket says 172.21.15.170:34338) 2023-09-09T23:07:22.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.182+0000 7f3af3fff700 1 -- 172.21.15.170:0/2333613527 learned_addr learned my addr 172.21.15.170:0/2333613527 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-09-09T23:07:22.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.182+0000 7f3af37fe700 1 --2- 172.21.15.170:0/2333613527 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3af40ff590 0x7f3af4196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:22.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.182+0000 7f3af2ffd700 1 --2- 172.21.15.170:0/2333613527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3af4100790 0x7f3af4196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:22.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.182+0000 7f3af3fff700 1 -- 172.21.15.170:0/2333613527 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3af40ff590 msgr2=0x7f3af4196850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:22.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.182+0000 7f3af3fff700 1 --2- 172.21.15.170:0/2333613527 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3af40ff590 0x7f3af4196850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:22.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.182+0000 7f3af3fff700 1 -- 172.21.15.170:0/2333613527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3af4100790 msgr2=0x7f3af4196d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:22.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.182+0000 7f3af3fff700 1 --2- 172.21.15.170:0/2333613527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3af4100790 0x7f3af4196d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:22.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.182+0000 7f3af3fff700 1 -- 172.21.15.170:0/2333613527 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aec009d00 con 0x7f3af4101130 2023-09-09T23:07:22.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.182+0000 7f3af3fff700 1 --2- 172.21.15.170:0/2333613527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3af4101130 0x7f3af419de10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3aec004d00 tx=0x7f3aec004de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:22.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.182+0000 7f3af0ff9700 1 -- 172.21.15.170:0/2333613527 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec00e040 con 0x7f3af4101130 2023-09-09T23:07:22.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.182+0000 7f3af0ff9700 1 -- 172.21.15.170:0/2333613527 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3aec01bd20 con 0x7f3af4101130 2023-09-09T23:07:22.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.182+0000 7f3afa05a700 1 -- 172.21.15.170:0/2333613527 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af419e930 con 0x7f3af4101130 2023-09-09T23:07:22.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.182+0000 7f3af0ff9700 1 -- 172.21.15.170:0/2333613527 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec005920 con 0x7f3af4101130 2023-09-09T23:07:22.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.182+0000 7f3afa05a700 1 -- 172.21.15.170:0/2333613527 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af419eda0 con 0x7f3af4101130 2023-09-09T23:07:22.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.183+0000 7f3af0ff9700 1 -- 172.21.15.170:0/2333613527 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7f3aec025410 con 0x7f3af4101130 2023-09-09T23:07:22.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.183+0000 7f3afa05a700 1 -- 172.21.15.170:0/2333613527 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af410aeb0 con 0x7f3af4101130 2023-09-09T23:07:22.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.183+0000 7f3af0ff9700 1 --2- 172.21.15.170:0/2333613527 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3ae0038630 0x7f3ae003aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:22.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.183+0000 7f3af0ff9700 1 -- 172.21.15.170:0/2333613527 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3aec04cc70 con 0x7f3af4101130 2023-09-09T23:07:22.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.184+0000 7f3af37fe700 1 --2- 172.21.15.170:0/2333613527 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3ae0038630 0x7f3ae003aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:22.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.184+0000 7f3af37fe700 1 --2- 172.21.15.170:0/2333613527 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3ae0038630 0x7f3ae003aaf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3adc006fd0 tx=0x7f3adc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:22.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.188+0000 7f3af0ff9700 1 -- 172.21.15.170:0/2333613527 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3aec011980 con 0x7f3af4101130 2023-09-09T23:07:22.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:21 smithi082 ceph-mon[98694]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:22.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:21 smithi098 ceph-mon[103124]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:22.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.388+0000 7f3afa05a700 1 -- 172.21.15.170:0/2333613527 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi082=x;smithi098=y;smithi170=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f3af40fbde0 con 0x7f3ae0038630 2023-09-09T23:07:22.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.395+0000 7f3af0ff9700 1 -- 172.21.15.170:0/2333613527 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f3af40fbde0 con 0x7f3ae0038630 2023-09-09T23:07:22.393 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled mgr update... 2023-09-09T23:07:22.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.397+0000 7f3afa05a700 1 -- 172.21.15.170:0/2333613527 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3ae0038630 msgr2=0x7f3ae003aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:22.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.397+0000 7f3afa05a700 1 --2- 172.21.15.170:0/2333613527 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3ae0038630 0x7f3ae003aaf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3adc006fd0 tx=0x7f3adc008040 comp rx=0 tx=0).stop 2023-09-09T23:07:22.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.397+0000 7f3afa05a700 1 -- 172.21.15.170:0/2333613527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3af4101130 msgr2=0x7f3af419de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:22.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.397+0000 7f3afa05a700 1 --2- 172.21.15.170:0/2333613527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3af4101130 0x7f3af419de10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3aec004d00 tx=0x7f3aec004de0 comp rx=0 tx=0).stop 2023-09-09T23:07:22.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.397+0000 7f3afa05a700 1 -- 172.21.15.170:0/2333613527 shutdown_connections 2023-09-09T23:07:22.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.397+0000 7f3afa05a700 1 --2- 172.21.15.170:0/2333613527 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3ae0038630 0x7f3ae003aaf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:22.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.397+0000 7f3afa05a700 1 --2- 172.21.15.170:0/2333613527 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3af40ff590 0x7f3af4196850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:22.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.397+0000 7f3afa05a700 1 --2- 172.21.15.170:0/2333613527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3af4101130 0x7f3af419de10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:22.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.397+0000 7f3afa05a700 1 --2- 172.21.15.170:0/2333613527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3af4100790 0x7f3af4196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:22.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.397+0000 7f3afa05a700 1 -- 172.21.15.170:0/2333613527 >> 172.21.15.170:0/2333613527 conn(0x7f3af40fab10 msgr2=0x7f3af40fb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:22.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.397+0000 7f3afa05a700 1 -- 172.21.15.170:0/2333613527 shutdown_connections 2023-09-09T23:07:22.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:07:22.397+0000 7f3afa05a700 1 -- 172.21.15.170:0/2333613527 wait complete. 2023-09-09T23:07:22.415 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:21 smithi170 ceph-mon[102832]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:22.772 DEBUG:teuthology.orchestra.run.smithi098:mgr.y> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@mgr.y.service 2023-09-09T23:07:22.774 DEBUG:teuthology.orchestra.run.smithi170:mgr.z> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@mgr.z.service 2023-09-09T23:07:22.777 INFO:tasks.cephadm:Deploying OSDs... 2023-09-09T23:07:22.778 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-09-09T23:07:22.778 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/scratch_devs of=/dev/stdout 2023-09-09T23:07:22.796 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-09T23:07:22.796 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_1 2023-09-09T23:07:22.806 INFO:journalctl@ceph.mgr.y.smithi098.stdout:-- Logs begin at Sat 2023-09-09 22:51:28 UTC. -- 2023-09-09T23:07:22.806 INFO:journalctl@ceph.mgr.z.smithi170.stdout:-- Logs begin at Sat 2023-09-09 22:51:11 UTC. -- 2023-09-09T23:07:22.853 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-09T23:07:22.853 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-09T23:07:22.853 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 388206 Links: 1 2023-09-09T23:07:22.854 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-09T23:07:22.854 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2023-09-09T23:07:22.854 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2023-09-09 23:06:19.169684031 +0000 2023-09-09T23:07:22.854 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2023-09-09 23:00:43.395182199 +0000 2023-09-09T23:07:22.854 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2023-09-09 23:00:43.395182199 +0000 2023-09-09T23:07:22.855 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: - 2023-09-09T23:07:22.855 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-09T23:07:22.920 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2023-09-09T23:07:22.921 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2023-09-09T23:07:22.921 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000217421 s, 2.4 MB/s 2023-09-09T23:07:22.922 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-09T23:07:22.980 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_2 2023-09-09T23:07:23.037 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-09T23:07:23.037 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-09T23:07:23.038 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 384940 Links: 1 2023-09-09T23:07:23.038 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-09T23:07:23.038 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2023-09-09T23:07:23.038 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2023-09-09 23:06:19.225682968 +0000 2023-09-09T23:07:23.038 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2023-09-09 23:00:43.922171989 +0000 2023-09-09T23:07:23.039 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2023-09-09 23:00:43.922171989 +0000 2023-09-09T23:07:23.039 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: - 2023-09-09T23:07:23.039 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-09T23:07:23.102 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2023-09-09T23:07:23.103 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2023-09-09T23:07:23.103 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000275719 s, 1.9 MB/s 2023-09-09T23:07:23.104 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-09T23:07:23.162 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_3 2023-09-09T23:07:23.219 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-09T23:07:23.220 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-09T23:07:23.220 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 392247 Links: 1 2023-09-09T23:07:23.220 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-09T23:07:23.221 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2023-09-09T23:07:23.221 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2023-09-09 23:06:19.291681715 +0000 2023-09-09T23:07:23.221 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2023-09-09 23:00:44.418162380 +0000 2023-09-09T23:07:23.221 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2023-09-09 23:00:44.418162380 +0000 2023-09-09T23:07:23.221 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: - 2023-09-09T23:07:23.222 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-09T23:07:23.284 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2023-09-09T23:07:23.284 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2023-09-09T23:07:23.285 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000189678 s, 2.7 MB/s 2023-09-09T23:07:23.286 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-09T23:07:23.343 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_4 2023-09-09T23:07:23.400 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-09T23:07:23.401 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-09T23:07:23.401 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 390306 Links: 1 2023-09-09T23:07:23.401 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-09T23:07:23.402 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2023-09-09T23:07:23.402 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2023-09-09 23:06:19.352680557 +0000 2023-09-09T23:07:23.402 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2023-09-09 23:00:44.922152616 +0000 2023-09-09T23:07:23.402 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2023-09-09 23:00:44.922152616 +0000 2023-09-09T23:07:23.402 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: - 2023-09-09T23:07:23.403 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-09T23:07:23.465 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2023-09-09T23:07:23.466 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2023-09-09T23:07:23.466 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000237953 s, 2.2 MB/s 2023-09-09T23:07:23.467 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-09T23:07:23.525 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2023-09-09T23:07:23.525 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/scratch_devs of=/dev/stdout 2023-09-09T23:07:23.546 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-09T23:07:23.547 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_1 2023-09-09T23:07:23.606 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-09T23:07:23.607 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-09T23:07:23.607 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 6h/6d Inode: 388491 Links: 1 2023-09-09T23:07:23.607 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-09T23:07:23.607 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2023-09-09T23:07:23.607 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2023-09-09 23:06:51.859332604 +0000 2023-09-09T23:07:23.608 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2023-09-09 23:00:58.482753872 +0000 2023-09-09T23:07:23.608 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2023-09-09 23:00:58.482753872 +0000 2023-09-09T23:07:23.608 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: - 2023-09-09T23:07:23.608 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-09T23:07:23.673 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:23 smithi098 ceph-mon[103124]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi082=x;smithi098=y;smithi170=z", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:07:23.673 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:23 smithi098 ceph-mon[103124]: Saving service mgr spec with placement smithi082=x;smithi098=y;smithi170=z;count:3 2023-09-09T23:07:23.674 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:23 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:23.674 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:23 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:07:23.674 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:23 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:23.674 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:23 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:07:23.674 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:23 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:23.675 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:23 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-09T23:07:23.675 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:23 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-09T23:07:23.675 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:23 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-09T23:07:23.675 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:23 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:23.675 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:23 smithi098 ceph-mon[103124]: Deploying daemon mgr.z on smithi170 2023-09-09T23:07:23.676 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:23 smithi098 ceph-mon[103124]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:23.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:23 smithi170 ceph-mon[102832]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi082=x;smithi098=y;smithi170=z", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:07:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:23 smithi170 ceph-mon[102832]: Saving service mgr spec with placement smithi082=x;smithi098=y;smithi170=z;count:3 2023-09-09T23:07:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:23 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:23 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:07:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:23 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:23 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:07:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:23 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:23 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-09T23:07:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:23 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-09T23:07:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:23 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-09T23:07:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:23 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:23.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:23 smithi170 ceph-mon[102832]: Deploying daemon mgr.z on smithi170 2023-09-09T23:07:23.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:23 smithi170 ceph-mon[102832]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:23.679 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2023-09-09T23:07:23.679 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2023-09-09T23:07:23.679 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000256531 s, 2.0 MB/s 2023-09-09T23:07:23.680 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-09T23:07:23.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:23 smithi082 ceph-mon[98694]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi082=x;smithi098=y;smithi170=z", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:07:23.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:23 smithi082 ceph-mon[98694]: Saving service mgr spec with placement smithi082=x;smithi098=y;smithi170=z;count:3 2023-09-09T23:07:23.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:23 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:23.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:23 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:07:23.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:23 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:23.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:23 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:07:23.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:23 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:23.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:23 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-09T23:07:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:23 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-09T23:07:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:23 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-09T23:07:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:23 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:23 smithi082 ceph-mon[98694]: Deploying daemon mgr.z on smithi170 2023-09-09T23:07:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:23 smithi082 ceph-mon[98694]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:23.743 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_2 2023-09-09T23:07:23.817 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-09T23:07:23.818 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-09T23:07:23.818 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 6h/6d Inode: 388559 Links: 1 2023-09-09T23:07:23.818 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-09T23:07:23.818 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2023-09-09T23:07:23.819 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2023-09-09 23:06:51.859332604 +0000 2023-09-09T23:07:23.819 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2023-09-09 23:00:58.979747649 +0000 2023-09-09T23:07:23.819 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2023-09-09 23:00:58.979747649 +0000 2023-09-09T23:07:23.819 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: - 2023-09-09T23:07:23.820 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-09T23:07:23.891 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2023-09-09T23:07:23.892 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2023-09-09T23:07:23.892 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.00026239 s, 2.0 MB/s 2023-09-09T23:07:23.893 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-09T23:07:23.955 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_3 2023-09-09T23:07:24.014 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-09T23:07:24.015 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-09T23:07:24.015 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 6h/6d Inode: 376483 Links: 1 2023-09-09T23:07:24.015 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-09T23:07:24.015 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2023-09-09T23:07:24.015 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2023-09-09 23:06:51.859332604 +0000 2023-09-09T23:07:24.016 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2023-09-09 23:00:59.478741402 +0000 2023-09-09T23:07:24.016 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2023-09-09 23:00:59.478741402 +0000 2023-09-09T23:07:24.016 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: - 2023-09-09T23:07:24.016 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-09T23:07:24.081 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2023-09-09T23:07:24.081 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2023-09-09T23:07:24.081 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.00031311 s, 1.6 MB/s 2023-09-09T23:07:24.082 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-09T23:07:24.141 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_4 2023-09-09T23:07:24.202 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-09T23:07:24.203 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-09T23:07:24.203 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 6h/6d Inode: 391361 Links: 1 2023-09-09T23:07:24.203 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-09T23:07:24.203 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2023-09-09T23:07:24.203 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2023-09-09 23:06:51.860332591 +0000 2023-09-09T23:07:24.204 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2023-09-09 23:00:59.976735167 +0000 2023-09-09T23:07:24.204 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2023-09-09 23:00:59.976735167 +0000 2023-09-09T23:07:24.204 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: - 2023-09-09T23:07:24.204 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-09T23:07:24.269 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2023-09-09T23:07:24.270 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2023-09-09T23:07:24.270 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000178088 s, 2.9 MB/s 2023-09-09T23:07:24.271 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-09T23:07:24.331 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-09-09T23:07:24.331 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/scratch_devs of=/dev/stdout 2023-09-09T23:07:24.350 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-09T23:07:24.351 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_1 2023-09-09T23:07:24.411 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-09T23:07:24.411 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-09T23:07:24.411 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 387728 Links: 1 2023-09-09T23:07:24.412 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-09T23:07:24.412 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2023-09-09T23:07:24.412 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2023-09-09 23:07:24.315320889 +0000 2023-09-09T23:07:24.412 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2023-09-09 23:01:09.480377509 +0000 2023-09-09T23:07:24.412 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2023-09-09 23:01:09.480377509 +0000 2023-09-09T23:07:24.413 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2023-09-09T23:07:24.413 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-09T23:07:24.482 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2023-09-09T23:07:24.482 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2023-09-09T23:07:24.483 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 9.8473e-05 s, 5.2 MB/s 2023-09-09T23:07:24.484 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-09T23:07:24.543 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_2 2023-09-09T23:07:24.601 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-09T23:07:24.601 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-09T23:07:24.601 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 379828 Links: 1 2023-09-09T23:07:24.601 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-09T23:07:24.602 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2023-09-09T23:07:24.602 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2023-09-09 23:07:24.315320889 +0000 2023-09-09T23:07:24.602 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2023-09-09 23:01:09.982365614 +0000 2023-09-09T23:07:24.602 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2023-09-09 23:01:09.982365614 +0000 2023-09-09T23:07:24.602 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2023-09-09T23:07:24.603 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-09T23:07:24.638 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:24 smithi170 systemd[1]: Starting Ceph mgr.z for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da... 2023-09-09T23:07:24.664 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2023-09-09T23:07:24.665 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2023-09-09T23:07:24.665 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000128403 s, 4.0 MB/s 2023-09-09T23:07:24.666 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-09T23:07:24.725 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_3 2023-09-09T23:07:24.783 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-09T23:07:24.784 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-09T23:07:24.784 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 386393 Links: 1 2023-09-09T23:07:24.784 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-09T23:07:24.784 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2023-09-09T23:07:24.784 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2023-09-09 23:07:24.316320866 +0000 2023-09-09T23:07:24.785 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2023-09-09 23:01:10.497353412 +0000 2023-09-09T23:07:24.785 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2023-09-09 23:01:10.497353412 +0000 2023-09-09T23:07:24.785 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2023-09-09T23:07:24.785 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-09T23:07:24.850 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2023-09-09T23:07:24.850 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2023-09-09T23:07:24.850 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000110207 s, 4.6 MB/s 2023-09-09T23:07:24.851 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-09T23:07:24.909 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_4 2023-09-09T23:07:24.925 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:24 smithi170 podman[104768]: 2023-09-09T23:07:24.925 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:24 smithi170 bash[104768]: 9f7feac68312bc1a4e9ee4743481ce57e6376f77ce556d48f6d39482c824829b 2023-09-09T23:07:24.926 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:24 smithi170 ceph-mgr[104931]: -- 172.21.15.170:0/1533765047 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56307eb3f1e0 con 0x56307eb14800 2023-09-09T23:07:24.943 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-09T23:07:24.944 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-09T23:07:24.944 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 390816 Links: 1 2023-09-09T23:07:24.944 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-09T23:07:24.944 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2023-09-09T23:07:24.945 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2023-09-09 23:07:24.316320866 +0000 2023-09-09T23:07:24.945 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2023-09-09 23:01:10.986341825 +0000 2023-09-09T23:07:24.945 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2023-09-09 23:01:10.986341825 +0000 2023-09-09T23:07:24.945 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2023-09-09T23:07:24.945 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-09T23:07:25.011 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2023-09-09T23:07:25.012 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2023-09-09T23:07:25.012 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000124594 s, 4.1 MB/s 2023-09-09T23:07:25.013 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-09T23:07:25.070 INFO:tasks.cephadm:Deploying osd.0 on smithi082 with /dev/vg_nvme/lv_4... 2023-09-09T23:07:25.071 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-09-09T23:07:25.265 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:24 smithi170 systemd[1]: Started Ceph mgr.z for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da. 2023-09-09T23:07:25.266 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:25 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:07:24.999+0000 7f2c46816200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-09T23:07:25.266 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:25 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:07:25.087+0000 7f2c46816200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-09T23:07:25.266 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:25 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:07:25.175+0000 7f2c46816200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-09T23:07:25.266 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:25 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:07:25.267+0000 7f2c46816200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-09T23:07:25.627 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:07:25.675 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:25 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:07:25.436+0000 7f2c46816200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-09T23:07:26.107 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:25 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:07:25.796+0000 7f2c46816200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-09T23:07:26.107 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:25 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:07:25.925+0000 7f2c46816200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-09T23:07:26.108 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:26 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:07:26.013+0000 7f2c46816200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-09T23:07:26.108 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:26 smithi170 ceph-mon[102832]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:26.108 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:26 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:26.108 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:26 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:26.109 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:26 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:26.109 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:26 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-09T23:07:26.109 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:26 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-09T23:07:26.109 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:26 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-09T23:07:26.110 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:26 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:26.110 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:26 smithi170 ceph-mon[102832]: Deploying daemon mgr.y on smithi098 2023-09-09T23:07:26.146 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:26 smithi082 ceph-mon[98694]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:26.146 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:26 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:26.146 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:26 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:26.147 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:26 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:26.147 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:26 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-09T23:07:26.147 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:26 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-09T23:07:26.147 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:26 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-09T23:07:26.147 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:26 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:26.148 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:26 smithi082 ceph-mon[98694]: Deploying daemon mgr.y on smithi098 2023-09-09T23:07:26.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:26 smithi098 ceph-mon[103124]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:26.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:26 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:26.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:26 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:26.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:26 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:26.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:26 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-09T23:07:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:26 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-09T23:07:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:26 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-09T23:07:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:26 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:26 smithi098 ceph-mon[103124]: Deploying daemon mgr.y on smithi098 2023-09-09T23:07:26.752 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:26 smithi098 systemd[1]: Starting Ceph mgr.y for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da... 2023-09-09T23:07:26.925 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:26 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:07:26.109+0000 7f2c46816200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-09T23:07:26.925 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:26 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:07:26.527+0000 7f2c46816200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-09T23:07:27.072 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:26 smithi098 podman[104587]: 2023-09-09T23:07:27.117 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-09-09T23:07:27.118 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-09-09T23:07:27.118 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2023-09-09T23:07:27.118 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2023-09-09T23:07:27.118 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111909 s, 937 MB/s 2023-09-09T23:07:27.118 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2023-09-09T23:07:27.374 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:27 smithi098 bash[104587]: ed48cd7633907bc9b9d3d0d619da435f8e48bad361630fc83d1e6860a695542d 2023-09-09T23:07:27.374 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:27 smithi098 ceph-mgr[104628]: -- 172.21.15.98:0/2589760970 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5640659fb1e0 con 0x5640659d0c00 2023-09-09T23:07:27.375 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:27 smithi098 systemd[1]: Started Ceph mgr.y for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da. 2023-09-09T23:07:27.375 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:27 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:07:27.201+0000 7faf81729200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-09T23:07:27.375 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:27 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:07:27.282+0000 7faf81729200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-09T23:07:27.375 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:27 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:07:27.375+0000 7faf81729200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-09T23:07:27.467 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi082:vg_nvme/lv_4 2023-09-09T23:07:27.652 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:27 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:07:27.474+0000 7faf81729200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-09T23:07:27.675 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:27 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:07:27.190+0000 7f2c46816200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-09T23:07:27.675 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:27 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:07:27.271+0000 7f2c46816200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-09T23:07:28.001 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:27 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:07:27.653+0000 7faf81729200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-09T23:07:28.021 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:07:28.366 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:28 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:07:28.032+0000 7faf81729200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-09T23:07:28.366 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:28 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:07:28.168+0000 7faf81729200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-09T23:07:28.367 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:28 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:07:28.265+0000 7faf81729200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-09T23:07:28.367 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:28 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:07:28.367+0000 7faf81729200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-09T23:07:28.367 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:28 smithi098 ceph-mon[103124]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:28.367 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:28 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:28.367 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:28 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:28.368 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:28 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:28.368 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:28 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:28.368 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:28 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:07:28.368 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:28 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:07:28.089+0000 7f2c46816200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-09T23:07:28.369 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:28 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:07:28.180+0000 7f2c46816200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-09T23:07:28.369 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:28 smithi170 ceph-mon[102832]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:28.369 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:28 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:28.369 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:28 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:28.370 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:28 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:28.370 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:28 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:28.370 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:28 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:07:28.415 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:28 smithi082 ceph-mon[98694]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:28.415 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:28 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:28.415 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:28 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:28.416 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:28 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:28.416 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:28 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:28.416 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:28 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:07:28.675 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:28 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:07:28.369+0000 7f2c46816200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-09T23:07:29.066 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:28 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:07:28.688+0000 7f2c46816200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-09T23:07:29.066 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:28 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:07:28.920+0000 7f2c46816200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-09T23:07:29.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.202+0000 7fc5a4e50700 1 -- 172.21.15.82:0/742449257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a0106380 msgr2=0x7fc5a01087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:29.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.202+0000 7fc5a4e50700 1 --2- 172.21.15.82:0/742449257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a0106380 0x7fc5a01087f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc59800b0a0 tx=0x7fc59800b3b0 comp rx=0 tx=0).stop 2023-09-09T23:07:29.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.202+0000 7fc5a4e50700 1 -- 172.21.15.82:0/742449257 shutdown_connections 2023-09-09T23:07:29.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.202+0000 7fc5a4e50700 1 --2- 172.21.15.82:0/742449257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a0106380 0x7fc5a01087f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:29.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.202+0000 7fc5a4e50700 1 --2- 172.21.15.82:0/742449257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5a0101120 0x7fc5a0103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:29.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.202+0000 7fc5a4e50700 1 --2- 172.21.15.82:0/742449257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5a0103a50 0x7fc5a0105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:29.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.202+0000 7fc5a4e50700 1 -- 172.21.15.82:0/742449257 >> 172.21.15.82:0/742449257 conn(0x7fc5a00fab10 msgr2=0x7fc5a00fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:29.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.202+0000 7fc5a4e50700 1 -- 172.21.15.82:0/742449257 shutdown_connections 2023-09-09T23:07:29.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.202+0000 7fc5a4e50700 1 -- 172.21.15.82:0/742449257 wait complete. 2023-09-09T23:07:29.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.203+0000 7fc5a4e50700 1 Processor -- start 2023-09-09T23:07:29.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.203+0000 7fc5a4e50700 1 -- start start 2023-09-09T23:07:29.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.203+0000 7fc5a4e50700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5a0101120 0x7fc5a0196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:29.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.203+0000 7fc5a4e50700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a0103a50 0x7fc5a01969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:29.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.203+0000 7fc5a4e50700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5a0106380 0x7fc5a019da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:29.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.203+0000 7fc5a4e50700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a019dff0 con 0x7fc5a0103a50 2023-09-09T23:07:29.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.203+0000 7fc5a4e50700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a019e130 con 0x7fc5a0106380 2023-09-09T23:07:29.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.203+0000 7fc5a4e50700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a019e2c0 con 0x7fc5a0101120 2023-09-09T23:07:29.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.203+0000 7fc59dd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a0103a50 0x7fc5a01969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:29.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.203+0000 7fc59dd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a0103a50 0x7fc5a01969a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53522/0 (socket says 172.21.15.82:53522) 2023-09-09T23:07:29.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.203+0000 7fc59dd9b700 1 -- 172.21.15.82:0/1583317114 learned_addr learned my addr 172.21.15.82:0/1583317114 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:07:29.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.203+0000 7fc59e59c700 1 --2- 172.21.15.82:0/1583317114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5a0101120 0x7fc5a0196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:29.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.203+0000 7fc59ed9d700 1 --2- 172.21.15.82:0/1583317114 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5a0106380 0x7fc5a019da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:29.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.203+0000 7fc59dd9b700 1 -- 172.21.15.82:0/1583317114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5a0101120 msgr2=0x7fc5a0196460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:29.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.203+0000 7fc59dd9b700 1 --2- 172.21.15.82:0/1583317114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5a0101120 0x7fc5a0196460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:29.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.203+0000 7fc59dd9b700 1 -- 172.21.15.82:0/1583317114 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5a0106380 msgr2=0x7fc5a019da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:29.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.203+0000 7fc59dd9b700 1 --2- 172.21.15.82:0/1583317114 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5a0106380 0x7fc5a019da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:29.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.203+0000 7fc59dd9b700 1 -- 172.21.15.82:0/1583317114 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc598009d00 con 0x7fc5a0103a50 2023-09-09T23:07:29.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.204+0000 7fc59dd9b700 1 --2- 172.21.15.82:0/1583317114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a0103a50 0x7fc5a01969a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc59000d9b0 tx=0x7fc59000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:29.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.204+0000 7fc58f7fe700 1 -- 172.21.15.82:0/1583317114 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc590004220 con 0x7fc5a0103a50 2023-09-09T23:07:29.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.204+0000 7fc58f7fe700 1 -- 172.21.15.82:0/1583317114 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc590007640 con 0x7fc5a0103a50 2023-09-09T23:07:29.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.204+0000 7fc58f7fe700 1 -- 172.21.15.82:0/1583317114 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc590005590 con 0x7fc5a0103a50 2023-09-09T23:07:29.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.204+0000 7fc5a4e50700 1 -- 172.21.15.82:0/1583317114 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a019e5a0 con 0x7fc5a0103a50 2023-09-09T23:07:29.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.204+0000 7fc5a4e50700 1 -- 172.21.15.82:0/1583317114 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a019ea50 con 0x7fc5a0103a50 2023-09-09T23:07:29.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.205+0000 7fc58f7fe700 1 -- 172.21.15.82:0/1583317114 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45387+0+0 (secure 0 0 0) 0x7fc59000b750 con 0x7fc5a0103a50 2023-09-09T23:07:29.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.205+0000 7fc58f7fe700 1 --2- 172.21.15.82:0/1583317114 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fc588038630 0x7fc58803aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:29.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.205+0000 7fc58f7fe700 1 -- 172.21.15.82:0/1583317114 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc59004ac60 con 0x7fc5a0103a50 2023-09-09T23:07:29.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.205+0000 7fc59e59c700 1 --2- 172.21.15.82:0/1583317114 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fc588038630 0x7fc58803aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:29.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.205+0000 7fc5a4e50700 1 -- 172.21.15.82:0/1583317114 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a00fc690 con 0x7fc5a0103a50 2023-09-09T23:07:29.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.205+0000 7fc59e59c700 1 --2- 172.21.15.82:0/1583317114 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fc588038630 0x7fc58803aaf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc594005950 tx=0x7fc59400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:29.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.210+0000 7fc58f7fe700 1 -- 172.21.15.82:0/1583317114 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc590019730 con 0x7fc5a0103a50 2023-09-09T23:07:29.251 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:28 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:07:28.810+0000 7faf81729200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-09T23:07:29.326 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:29 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:07:29.068+0000 7f2c46816200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-09T23:07:29.411 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:29.413+0000 7fc5a4e50700 1 -- 172.21.15.82:0/1583317114 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5a010c0a0 con 0x7fc588038630 2023-09-09T23:07:29.675 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:29 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:07:29.328+0000 7f2c46816200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-09T23:07:29.751 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:29 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:07:29.500+0000 7faf81729200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-09T23:07:29.752 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:29 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:07:29.582+0000 7faf81729200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-09T23:07:30.164 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:30 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:07:30.034+0000 7f2c46816200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-09T23:07:30.164 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:29 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:30.165 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:29 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:30.165 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:29 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:30.165 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:29 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:30.165 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:29 smithi170 ceph-mon[102832]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:30.166 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:29 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:30.166 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:29 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:30.166 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:29 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:07:30.166 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:29 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:30.166 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:29 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:07:30.167 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:29 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:07:30.167 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:29 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:30.167 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:29 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-09T23:07:30.167 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:29 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-09T23:07:30.167 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:29 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:30.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:29 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:30.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:29 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:30.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:29 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:30.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:29 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:30.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:29 smithi082 ceph-mon[98694]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:30.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:29 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:30.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:29 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:30.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:29 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:07:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:29 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:29 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:07:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:29 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:07:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:29 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:29 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-09T23:07:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:29 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-09T23:07:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:29 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:30.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:29 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:30.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:29 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:30.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:29 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:30.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:29 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:30.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:29 smithi098 ceph-mon[103124]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:30.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:29 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:29 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:29 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:07:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:29 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:29 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:07:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:29 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:07:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:29 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:29 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-09T23:07:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:29 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-09T23:07:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:29 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:30.425 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:30 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:07:30.166+0000 7f2c46816200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-09T23:07:30.425 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:30 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:07:30.358+0000 7f2c46816200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-09T23:07:30.604 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:30 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:07:30.345+0000 7faf81729200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-09T23:07:30.604 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:30 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:07:30.434+0000 7faf81729200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-09T23:07:30.837 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:30 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:07:30.470+0000 7f2c46816200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-09T23:07:30.838 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:30 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:07:30.685+0000 7f2c46816200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-09T23:07:30.884 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:30 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:07:30.605+0000 7faf81729200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-09T23:07:30.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:30 smithi098 ceph-mon[103124]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:07:30.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:30 smithi098 ceph-mon[103124]: Reconfiguring mgr.x (unknown last config time)... 2023-09-09T23:07:30.884 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:30 smithi098 ceph-mon[103124]: Reconfiguring daemon mgr.x on smithi082 2023-09-09T23:07:30.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:30 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:30.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:30 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:30.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:30 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:07:30.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:30 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:30.885 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:30 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:07:30.886 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:30 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:31.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:30 smithi170 ceph-mon[102832]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:07:31.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:30 smithi170 ceph-mon[102832]: Reconfiguring mgr.x (unknown last config time)... 2023-09-09T23:07:31.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:30 smithi170 ceph-mon[102832]: Reconfiguring daemon mgr.x on smithi082 2023-09-09T23:07:31.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:30 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:31.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:30 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:31.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:30 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:07:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:30 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:30 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:07:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:30 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:31.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:30 smithi082 ceph-mon[98694]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:07:31.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:30 smithi082 ceph-mon[98694]: Reconfiguring mgr.x (unknown last config time)... 2023-09-09T23:07:31.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:30 smithi082 ceph-mon[98694]: Reconfiguring daemon mgr.x on smithi082 2023-09-09T23:07:31.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:30 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:31.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:30 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:31.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:30 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:07:31.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:30 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:31.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:30 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:07:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:30 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:31.251 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:30 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:07:30.885+0000 7faf81729200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-09T23:07:31.252 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:31 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:07:31.113+0000 7faf81729200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-09T23:07:31.518 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:31 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:07:31.261+0000 7faf81729200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-09T23:07:31.843 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:07:31 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:07:31.552+0000 7f2c46816200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-09T23:07:31.844 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:31 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:07:31.520+0000 7faf81729200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-09T23:07:31.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:31.848+0000 7fc58f7fe700 1 -- 172.21.15.82:0/1583317114 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 14) v1 ==== 90295+0+0 (secure 0 0 0) 0x7fc590003810 con 0x7fc5a0103a50 2023-09-09T23:07:32.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:31 smithi170 ceph-mon[102832]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:32.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:31 smithi170 ceph-mon[102832]: Standby manager daemon z started 2023-09-09T23:07:32.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:31 smithi170 ceph-mon[102832]: from='mgr.? 172.21.15.170:0/2096552539' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-09T23:07:32.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:31 smithi170 ceph-mon[102832]: from='mgr.? 172.21.15.170:0/2096552539' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-09T23:07:32.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:31 smithi170 ceph-mon[102832]: from='mgr.? 172.21.15.170:0/2096552539' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-09T23:07:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:31 smithi170 ceph-mon[102832]: from='mgr.? 172.21.15.170:0/2096552539' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-09T23:07:32.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:31 smithi082 ceph-mon[98694]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:32.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:31 smithi082 ceph-mon[98694]: Standby manager daemon z started 2023-09-09T23:07:32.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:31 smithi082 ceph-mon[98694]: from='mgr.? 172.21.15.170:0/2096552539' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-09T23:07:32.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:31 smithi082 ceph-mon[98694]: from='mgr.? 172.21.15.170:0/2096552539' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-09T23:07:32.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:31 smithi082 ceph-mon[98694]: from='mgr.? 172.21.15.170:0/2096552539' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-09T23:07:32.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:31 smithi082 ceph-mon[98694]: from='mgr.? 172.21.15.170:0/2096552539' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-09T23:07:32.220 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:31 smithi098 ceph-mon[103124]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:32.220 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:31 smithi098 ceph-mon[103124]: Standby manager daemon z started 2023-09-09T23:07:32.221 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:31 smithi098 ceph-mon[103124]: from='mgr.? 172.21.15.170:0/2096552539' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-09T23:07:32.221 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:31 smithi098 ceph-mon[103124]: from='mgr.? 172.21.15.170:0/2096552539' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-09T23:07:32.221 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:31 smithi098 ceph-mon[103124]: from='mgr.? 172.21.15.170:0/2096552539' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-09T23:07:32.221 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:31 smithi098 ceph-mon[103124]: from='mgr.? 172.21.15.170:0/2096552539' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-09T23:07:32.501 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:32 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:07:32.221+0000 7faf81729200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-09T23:07:32.502 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:32 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:07:32.353+0000 7faf81729200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-09T23:07:32.868 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:32 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:07:32.543+0000 7faf81729200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-09T23:07:32.868 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:32 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:07:32.655+0000 7faf81729200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-09T23:07:33.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:32 smithi170 ceph-mon[102832]: mgrmap e14: x(active, since 82s), standbys: z 2023-09-09T23:07:33.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:32 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-09T23:07:33.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:32 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/4031747129' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1277eb17-a2e7-4e10-9832-f12a5e93ffde"}]: dispatch 2023-09-09T23:07:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:32 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/4031747129' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1277eb17-a2e7-4e10-9832-f12a5e93ffde"}]': finished 2023-09-09T23:07:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:32 smithi170 ceph-mon[102832]: osdmap e5: 1 total, 0 up, 1 in 2023-09-09T23:07:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:32 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-09T23:07:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:32 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/750350506' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:07:33.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:32 smithi082 ceph-mon[98694]: mgrmap e14: x(active, since 82s), standbys: z 2023-09-09T23:07:33.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:32 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-09T23:07:33.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:32 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/4031747129' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1277eb17-a2e7-4e10-9832-f12a5e93ffde"}]: dispatch 2023-09-09T23:07:33.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:32 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/4031747129' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1277eb17-a2e7-4e10-9832-f12a5e93ffde"}]': finished 2023-09-09T23:07:33.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:32 smithi082 ceph-mon[98694]: osdmap e5: 1 total, 0 up, 1 in 2023-09-09T23:07:33.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:32 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-09T23:07:33.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:32 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/750350506' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:07:33.251 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:32 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:07:32.869+0000 7faf81729200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-09T23:07:33.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:32 smithi098 ceph-mon[103124]: mgrmap e14: x(active, since 82s), standbys: z 2023-09-09T23:07:33.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:32 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-09T23:07:33.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:32 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/4031747129' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1277eb17-a2e7-4e10-9832-f12a5e93ffde"}]: dispatch 2023-09-09T23:07:33.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:32 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/4031747129' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1277eb17-a2e7-4e10-9832-f12a5e93ffde"}]': finished 2023-09-09T23:07:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:32 smithi098 ceph-mon[103124]: osdmap e5: 1 total, 0 up, 1 in 2023-09-09T23:07:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:32 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-09T23:07:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:32 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/750350506' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:07:34.001 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:07:33 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:07:33.725+0000 7faf81729200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-09T23:07:34.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:34.192+0000 7fc58f7fe700 1 -- 172.21.15.82:0/1583317114 <== mon.0 v2:172.21.15.82:3300/0 8 ==== mgrmap(e 15) v1 ==== 135203+0+0 (secure 0 0 0) 0x7fc5900193b0 con 0x7fc5a0103a50 2023-09-09T23:07:34.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:34 smithi082 ceph-mon[98694]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:34.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:34 smithi082 ceph-mon[98694]: Standby manager daemon y started 2023-09-09T23:07:34.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:34 smithi082 ceph-mon[98694]: from='mgr.? 172.21.15.98:0/3460325973' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-09T23:07:34.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:34 smithi082 ceph-mon[98694]: from='mgr.? 172.21.15.98:0/3460325973' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-09T23:07:34.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:34 smithi082 ceph-mon[98694]: from='mgr.? 172.21.15.98:0/3460325973' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-09T23:07:34.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:34 smithi082 ceph-mon[98694]: from='mgr.? 172.21.15.98:0/3460325973' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-09T23:07:34.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:34 smithi098 ceph-mon[103124]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:34.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:34 smithi098 ceph-mon[103124]: Standby manager daemon y started 2023-09-09T23:07:34.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:34 smithi098 ceph-mon[103124]: from='mgr.? 172.21.15.98:0/3460325973' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-09T23:07:34.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:34 smithi098 ceph-mon[103124]: from='mgr.? 172.21.15.98:0/3460325973' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-09T23:07:34.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:34 smithi098 ceph-mon[103124]: from='mgr.? 172.21.15.98:0/3460325973' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-09T23:07:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:34 smithi098 ceph-mon[103124]: from='mgr.? 172.21.15.98:0/3460325973' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-09T23:07:34.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:34 smithi170 ceph-mon[102832]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:34.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:34 smithi170 ceph-mon[102832]: Standby manager daemon y started 2023-09-09T23:07:34.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:34 smithi170 ceph-mon[102832]: from='mgr.? 172.21.15.98:0/3460325973' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-09T23:07:34.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:34 smithi170 ceph-mon[102832]: from='mgr.? 172.21.15.98:0/3460325973' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-09T23:07:34.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:34 smithi170 ceph-mon[102832]: from='mgr.? 172.21.15.98:0/3460325973' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-09T23:07:34.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:34 smithi170 ceph-mon[102832]: from='mgr.? 172.21.15.98:0/3460325973' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-09T23:07:35.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:35 smithi082 ceph-mon[98694]: mgrmap e15: x(active, since 85s), standbys: z, y 2023-09-09T23:07:35.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:35 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-09T23:07:35.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:35 smithi098 ceph-mon[103124]: mgrmap e15: x(active, since 85s), standbys: z, y 2023-09-09T23:07:35.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:35 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-09T23:07:35.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:35 smithi170 ceph-mon[102832]: mgrmap e15: x(active, since 85s), standbys: z, y 2023-09-09T23:07:35.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:35 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-09T23:07:36.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:36 smithi082 ceph-mon[98694]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:36.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:36 smithi098 ceph-mon[103124]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:36.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:36 smithi170 ceph-mon[102832]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:38.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:38 smithi082 ceph-mon[98694]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:38.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:38 smithi098 ceph-mon[103124]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:38.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:38 smithi170 ceph-mon[102832]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:39.434 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:39 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-09-09T23:07:39.434 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:39 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:39.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:39 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-09-09T23:07:39.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:39 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:39.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:39 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-09-09T23:07:39.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:39 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:40.332 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:40 smithi082 ceph-mon[98694]: Deploying daemon osd.0 on smithi082 2023-09-09T23:07:40.332 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:40 smithi082 ceph-mon[98694]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:40.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:40 smithi098 ceph-mon[103124]: Deploying daemon osd.0 on smithi082 2023-09-09T23:07:40.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:40 smithi098 ceph-mon[103124]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:40.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:40 smithi170 ceph-mon[102832]: Deploying daemon osd.0 on smithi082 2023-09-09T23:07:40.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:40 smithi170 ceph-mon[102832]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:42.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:42 smithi082 ceph-mon[98694]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:42.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:42 smithi098 ceph-mon[103124]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:42.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:42 smithi170 ceph-mon[102832]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:43.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:43 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:43.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:43 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:43.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:43 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:07:43.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:43 smithi082 ceph-mon[98694]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:43.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:43 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:43.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:43 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:43.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:43 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:07:43.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:43 smithi170 ceph-mon[102832]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:44.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:43 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:44.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:43 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:44.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:43 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:07:44.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:43 smithi098 ceph-mon[103124]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:45.159 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:45.162+0000 7fc58f7fe700 1 -- 172.21.15.82:0/1583317114 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc594015070 con 0x7fc588038630 2023-09-09T23:07:45.160 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 0 on host 'smithi082' 2023-09-09T23:07:45.162 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:45.164+0000 7fc5a4e50700 1 -- 172.21.15.82:0/1583317114 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fc588038630 msgr2=0x7fc58803aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:45.163 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:45.164+0000 7fc5a4e50700 1 --2- 172.21.15.82:0/1583317114 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fc588038630 0x7fc58803aaf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc594005950 tx=0x7fc59400b410 comp rx=0 tx=0).stop 2023-09-09T23:07:45.163 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:45.164+0000 7fc5a4e50700 1 -- 172.21.15.82:0/1583317114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a0103a50 msgr2=0x7fc5a01969a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:45.163 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:45.164+0000 7fc5a4e50700 1 --2- 172.21.15.82:0/1583317114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a0103a50 0x7fc5a01969a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc59000d9b0 tx=0x7fc59000dcc0 comp rx=0 tx=0).stop 2023-09-09T23:07:45.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:45.164+0000 7fc5a4e50700 1 -- 172.21.15.82:0/1583317114 shutdown_connections 2023-09-09T23:07:45.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:45.164+0000 7fc5a4e50700 1 --2- 172.21.15.82:0/1583317114 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fc588038630 0x7fc58803aaf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:45.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:45.164+0000 7fc5a4e50700 1 --2- 172.21.15.82:0/1583317114 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5a0106380 0x7fc5a019da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:45.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:45.164+0000 7fc5a4e50700 1 --2- 172.21.15.82:0/1583317114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5a0101120 0x7fc5a0196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:45.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:45.164+0000 7fc5a4e50700 1 --2- 172.21.15.82:0/1583317114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a0103a50 0x7fc5a01969a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:45.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:45.164+0000 7fc5a4e50700 1 -- 172.21.15.82:0/1583317114 >> 172.21.15.82:0/1583317114 conn(0x7fc5a00fab10 msgr2=0x7fc5a0101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:45.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:45.164+0000 7fc5a4e50700 1 -- 172.21.15.82:0/1583317114 shutdown_connections 2023-09-09T23:07:45.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:07:45.164+0000 7fc5a4e50700 1 -- 172.21.15.82:0/1583317114 wait complete. 2023-09-09T23:07:45.619 DEBUG:teuthology.orchestra.run.smithi082:osd.0> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@osd.0.service 2023-09-09T23:07:45.622 INFO:tasks.cephadm:Deploying osd.1 on smithi098 with /dev/vg_nvme/lv_4... 2023-09-09T23:07:45.622 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-09-09T23:07:45.652 INFO:journalctl@ceph.osd.0.smithi082.stdout:-- Logs begin at Sat 2023-09-09 22:51:24 UTC. -- 2023-09-09T23:07:45.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:45 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:45.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:45 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:45.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:45 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:45.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:45 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:07:45.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:45 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:45.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:45 smithi082 ceph-mon[98694]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:45.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:45 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:45.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:45 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:45.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:45 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:07:45.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:45 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:45.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:45 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:45.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:45 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:45.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:45 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:07:45.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:45 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:45.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:45 smithi098 ceph-mon[103124]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:45 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:45 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:45 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:07:45.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:45 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:45.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:45 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:45.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:45 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:45.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:45 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:07:45.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:45 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:45.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:45 smithi170 ceph-mon[102832]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:45 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:45 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:45 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:07:46.189 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-09T23:07:46.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:46 smithi082 ceph-mon[98694]: from='osd.0 [v2:172.21.15.82:6802/3197434929,v1:172.21.15.82:6803/3197434929]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-09T23:07:46.705 INFO:journalctl@ceph.osd.0.smithi082.stdout:Sep 09 23:07:46 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-0[121793]: 2023-09-09T23:07:46.541+0000 7f9a6cad4640 -1 osd.0 0 log_to_monitors true 2023-09-09T23:07:46.745 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:46 smithi098 ceph-mon[103124]: from='osd.0 [v2:172.21.15.82:6802/3197434929,v1:172.21.15.82:6803/3197434929]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-09T23:07:46.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:46 smithi170 ceph-mon[102832]: from='osd.0 [v2:172.21.15.82:6802/3197434929,v1:172.21.15.82:6803/3197434929]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-09T23:07:47.717 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-09-09T23:07:47.717 INFO:teuthology.orchestra.run.smithi098.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-09-09T23:07:47.717 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10+0 records in 2023-09-09T23:07:47.717 INFO:teuthology.orchestra.run.smithi098.stderr:10+0 records out 2023-09-09T23:07:47.718 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112164 s, 935 MB/s 2023-09-09T23:07:47.718 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping successful for: 2023-09-09T23:07:47.871 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:47 smithi098 ceph-mon[103124]: from='osd.0 [v2:172.21.15.82:6802/3197434929,v1:172.21.15.82:6803/3197434929]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-09-09T23:07:47.871 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:47 smithi098 ceph-mon[103124]: osdmap e6: 1 total, 0 up, 1 in 2023-09-09T23:07:47.871 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:47 smithi098 ceph-mon[103124]: from='osd.0 [v2:172.21.15.82:6802/3197434929,v1:172.21.15.82:6803/3197434929]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-09-09T23:07:47.871 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:47 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-09T23:07:47.872 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:47 smithi098 ceph-mon[103124]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:47.872 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:47 smithi098 ceph-mon[103124]: from='osd.0 [v2:172.21.15.82:6802/3197434929,v1:172.21.15.82:6803/3197434929]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-09-09T23:07:47.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:47 smithi170 ceph-mon[102832]: from='osd.0 [v2:172.21.15.82:6802/3197434929,v1:172.21.15.82:6803/3197434929]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-09-09T23:07:47.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:47 smithi170 ceph-mon[102832]: osdmap e6: 1 total, 0 up, 1 in 2023-09-09T23:07:47.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:47 smithi170 ceph-mon[102832]: from='osd.0 [v2:172.21.15.82:6802/3197434929,v1:172.21.15.82:6803/3197434929]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-09-09T23:07:47.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:47 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-09T23:07:47.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:47 smithi170 ceph-mon[102832]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:47.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:47 smithi170 ceph-mon[102832]: from='osd.0 [v2:172.21.15.82:6802/3197434929,v1:172.21.15.82:6803/3197434929]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-09-09T23:07:47.955 INFO:journalctl@ceph.osd.0.smithi082.stdout:Sep 09 23:07:47 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-0[121793]: 2023-09-09T23:07:47.604+0000 7f9a6194a700 -1 osd.0 0 waiting for initial osdmap 2023-09-09T23:07:47.955 INFO:journalctl@ceph.osd.0.smithi082.stdout:Sep 09 23:07:47 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-0[121793]: 2023-09-09T23:07:47.610+0000 7f9a5bf3c700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-09T23:07:47.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:47 smithi082 ceph-mon[98694]: from='osd.0 [v2:172.21.15.82:6802/3197434929,v1:172.21.15.82:6803/3197434929]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-09-09T23:07:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:47 smithi082 ceph-mon[98694]: osdmap e6: 1 total, 0 up, 1 in 2023-09-09T23:07:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:47 smithi082 ceph-mon[98694]: from='osd.0 [v2:172.21.15.82:6802/3197434929,v1:172.21.15.82:6803/3197434929]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-09-09T23:07:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:47 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-09T23:07:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:47 smithi082 ceph-mon[98694]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:47 smithi082 ceph-mon[98694]: from='osd.0 [v2:172.21.15.82:6802/3197434929,v1:172.21.15.82:6803/3197434929]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-09-09T23:07:48.141 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi098:vg_nvme/lv_4 2023-09-09T23:07:48.698 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-09T23:07:48.901 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:48 smithi098 ceph-mon[103124]: osdmap e7: 1 total, 0 up, 1 in 2023-09-09T23:07:48.902 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:48 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-09T23:07:48.902 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:48 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-09T23:07:48.902 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:48 smithi098 ceph-mon[103124]: Detected new or changed devices on smithi082 2023-09-09T23:07:48.902 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:48 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:48.903 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:48 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:48.903 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:48 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:07:48.903 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:48 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 17168M 2023-09-09T23:07:48.903 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:48 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:48.903 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:48 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:48.904 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:48 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:07:48.904 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:48 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:48.904 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:48 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-09T23:07:48.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:48 smithi170 ceph-mon[102832]: osdmap e7: 1 total, 0 up, 1 in 2023-09-09T23:07:48.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:48 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-09T23:07:48.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:48 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-09T23:07:48.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:48 smithi170 ceph-mon[102832]: Detected new or changed devices on smithi082 2023-09-09T23:07:48.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:48 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:48.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:48 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:48.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:48 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:07:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:48 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 17168M 2023-09-09T23:07:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:48 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:48 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:48 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:07:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:48 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:48 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-09T23:07:48.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:48 smithi082 ceph-mon[98694]: osdmap e7: 1 total, 0 up, 1 in 2023-09-09T23:07:48.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:48 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-09T23:07:48.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:48 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-09T23:07:48.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:48 smithi082 ceph-mon[98694]: Detected new or changed devices on smithi082 2023-09-09T23:07:48.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:48 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:48.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:48 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:48.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:48 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:07:48.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:48 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 17168M 2023-09-09T23:07:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:48 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:48 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:48 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:07:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:48 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:07:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:48 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-09T23:07:49.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:49 smithi170 ceph-mon[102832]: purged_snaps scrub starts 2023-09-09T23:07:49.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:49 smithi170 ceph-mon[102832]: purged_snaps scrub ok 2023-09-09T23:07:49.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:49 smithi170 ceph-mon[102832]: osd.0 [v2:172.21.15.82:6802/3197434929,v1:172.21.15.82:6803/3197434929] boot 2023-09-09T23:07:49.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:49 smithi170 ceph-mon[102832]: osdmap e8: 1 total, 1 up, 1 in 2023-09-09T23:07:49.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:49 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-09T23:07:49.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:49 smithi170 ceph-mon[102832]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:49.928 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.928+0000 7f6c13b8c700 1 -- 172.21.15.98:0/3689741927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c0c0fe950 msgr2=0x7f6c0c0fed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:49.929 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.928+0000 7f6c13b8c700 1 --2- 172.21.15.98:0/3689741927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c0c0fe950 0x7f6c0c0fed70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6c08009990 tx=0x7f6c08009ca0 comp rx=0 tx=0).stop 2023-09-09T23:07:49.929 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.928+0000 7f6c13b8c700 1 -- 172.21.15.98:0/3689741927 shutdown_connections 2023-09-09T23:07:49.929 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.928+0000 7f6c13b8c700 1 --2- 172.21.15.98:0/3689741927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c0c0fe950 0x7f6c0c0fed70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:49.929 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.928+0000 7f6c13b8c700 1 --2- 172.21.15.98:0/3689741927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c0c1022d0 0x7f6c0c108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:49.930 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.928+0000 7f6c13b8c700 1 --2- 172.21.15.98:0/3689741927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c0c101930 0x7f6c0c101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:49.930 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.928+0000 7f6c13b8c700 1 -- 172.21.15.98:0/3689741927 >> 172.21.15.98:0/3689741927 conn(0x7f6c0c0fa2b0 msgr2=0x7f6c0c0fc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:07:49.930 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.929+0000 7f6c13b8c700 1 -- 172.21.15.98:0/3689741927 shutdown_connections 2023-09-09T23:07:49.930 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.929+0000 7f6c13b8c700 1 -- 172.21.15.98:0/3689741927 wait complete. 2023-09-09T23:07:49.930 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.929+0000 7f6c13b8c700 1 Processor -- start 2023-09-09T23:07:49.931 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.929+0000 7f6c13b8c700 1 -- start start 2023-09-09T23:07:49.931 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.929+0000 7f6c13b8c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c0c0fe950 0x7f6c0c196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:49.931 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.929+0000 7f6c13b8c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c0c101930 0x7f6c0c196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:49.931 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.929+0000 7f6c13b8c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c0c1022d0 0x7f6c0c19d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:49.931 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.929+0000 7f6c13b8c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c19dfa0 con 0x7f6c0c0fe950 2023-09-09T23:07:49.932 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.929+0000 7f6c13b8c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c19e0e0 con 0x7f6c0c101930 2023-09-09T23:07:49.932 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.929+0000 7f6c13b8c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c19e270 con 0x7f6c0c1022d0 2023-09-09T23:07:49.932 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.929+0000 7f6c11127700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c0c101930 0x7f6c0c196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:49.932 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.929+0000 7f6c11127700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c0c101930 0x7f6c0c196950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40630/0 (socket says 172.21.15.98:40630) 2023-09-09T23:07:49.932 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.929+0000 7f6c11127700 1 -- 172.21.15.98:0/2620220383 learned_addr learned my addr 172.21.15.98:0/2620220383 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-09-09T23:07:49.933 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.930+0000 7f6c11127700 1 -- 172.21.15.98:0/2620220383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c0c1022d0 msgr2=0x7f6c0c19d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:49.933 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.930+0000 7f6c11127700 1 --2- 172.21.15.98:0/2620220383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c0c1022d0 0x7f6c0c19d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:49.933 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.930+0000 7f6c11928700 1 --2- 172.21.15.98:0/2620220383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c0c0fe950 0x7f6c0c196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:49.933 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.930+0000 7f6c11127700 1 -- 172.21.15.98:0/2620220383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c0c0fe950 msgr2=0x7f6c0c196410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:07:49.933 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.930+0000 7f6c11127700 1 --2- 172.21.15.98:0/2620220383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c0c0fe950 0x7f6c0c196410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:07:49.934 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.930+0000 7f6c11127700 1 -- 172.21.15.98:0/2620220383 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c080095f0 con 0x7f6c0c101930 2023-09-09T23:07:49.934 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.930+0000 7f6c11127700 1 --2- 172.21.15.98:0/2620220383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c0c101930 0x7f6c0c196950 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00c920 tx=0x7f6bfc00cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:49.934 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.930+0000 7f6c02ffd700 1 -- 172.21.15.98:0/2620220383 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc007b00 con 0x7f6c0c101930 2023-09-09T23:07:49.934 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.930+0000 7f6c02ffd700 1 -- 172.21.15.98:0/2620220383 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bfc004cf0 con 0x7f6c0c101930 2023-09-09T23:07:49.934 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.930+0000 7f6c13b8c700 1 -- 172.21.15.98:0/2620220383 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c0c0ff840 con 0x7f6c0c101930 2023-09-09T23:07:49.935 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.930+0000 7f6c02ffd700 1 -- 172.21.15.98:0/2620220383 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc0055e0 con 0x7f6c0c101930 2023-09-09T23:07:49.936 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.930+0000 7f6c13b8c700 1 -- 172.21.15.98:0/2620220383 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0c0ffd40 con 0x7f6c0c101930 2023-09-09T23:07:49.936 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.931+0000 7f6c13b8c700 1 -- 172.21.15.98:0/2620220383 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c0c107d20 con 0x7f6c0c101930 2023-09-09T23:07:49.936 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.932+0000 7f6c02ffd700 1 -- 172.21.15.98:0/2620220383 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 15) v1 ==== 135203+0+0 (secure 0 0 0) 0x7f6bfc007c60 con 0x7f6c0c101930 2023-09-09T23:07:49.936 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.932+0000 7f6c02ffd700 1 --2- 172.21.15.98:0/2620220383 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f6bf80a05b0 0x7f6bf80a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:07:49.936 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.933+0000 7f6c02ffd700 1 -- 172.21.15.98:0/2620220383 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f6bfc0c99d0 con 0x7f6c0c101930 2023-09-09T23:07:49.937 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.933+0000 7f6c11928700 1 --2- 172.21.15.98:0/2620220383 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f6bf80a05b0 0x7f6bf80a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:07:49.937 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.933+0000 7f6c11928700 1 --2- 172.21.15.98:0/2620220383 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f6bf80a05b0 0x7f6bf80a2a70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6c08000c00 tx=0x7f6c08005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:07:49.937 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:49.937+0000 7f6c02ffd700 1 -- 172.21.15.98:0/2620220383 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6bfc094f70 con 0x7f6c0c101930 2023-09-09T23:07:49.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:49 smithi082 ceph-mon[98694]: purged_snaps scrub starts 2023-09-09T23:07:49.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:49 smithi082 ceph-mon[98694]: purged_snaps scrub ok 2023-09-09T23:07:49.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:49 smithi082 ceph-mon[98694]: osd.0 [v2:172.21.15.82:6802/3197434929,v1:172.21.15.82:6803/3197434929] boot 2023-09-09T23:07:49.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:49 smithi082 ceph-mon[98694]: osdmap e8: 1 total, 1 up, 1 in 2023-09-09T23:07:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:49 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-09T23:07:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:49 smithi082 ceph-mon[98694]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:50.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:49 smithi098 ceph-mon[103124]: purged_snaps scrub starts 2023-09-09T23:07:50.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:49 smithi098 ceph-mon[103124]: purged_snaps scrub ok 2023-09-09T23:07:50.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:49 smithi098 ceph-mon[103124]: osd.0 [v2:172.21.15.82:6802/3197434929,v1:172.21.15.82:6803/3197434929] boot 2023-09-09T23:07:50.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:49 smithi098 ceph-mon[103124]: osdmap e8: 1 total, 1 up, 1 in 2023-09-09T23:07:50.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:49 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-09T23:07:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:49 smithi098 ceph-mon[103124]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-09T23:07:50.130 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:07:50.131+0000 7f6c13b8c700 1 -- 172.21.15.98:0/2620220383 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c0c02cff0 con 0x7f6bf80a05b0 2023-09-09T23:07:50.892 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:50 smithi098 ceph-mon[103124]: osdmap e9: 1 total, 1 up, 1 in 2023-09-09T23:07:50.893 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:50 smithi098 ceph-mon[103124]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:07:50.893 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:50 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:07:50.893 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:50 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:07:50.893 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:50 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:50.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:50 smithi170 ceph-mon[102832]: osdmap e9: 1 total, 1 up, 1 in 2023-09-09T23:07:50.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:50 smithi170 ceph-mon[102832]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:07:50.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:50 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:07:50.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:50 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:07:50.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:50 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:50.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:50 smithi082 ceph-mon[98694]: osdmap e9: 1 total, 1 up, 1 in 2023-09-09T23:07:50.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:50 smithi082 ceph-mon[98694]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:07:50.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:50 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:07:50.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:50 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:07:50.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:50 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:51.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:51 smithi170 ceph-mon[102832]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:07:51.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:51 smithi082 ceph-mon[98694]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:07:52.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:51 smithi098 ceph-mon[103124]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:07:52.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:52 smithi170 ceph-mon[102832]: from='client.? 172.21.15.98:0/2137477649' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f68723fd-0908-4e21-ae8e-1de1c3a9a65c"}]: dispatch 2023-09-09T23:07:52.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:52 smithi170 ceph-mon[102832]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f68723fd-0908-4e21-ae8e-1de1c3a9a65c"}]: dispatch 2023-09-09T23:07:52.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:52 smithi170 ceph-mon[102832]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f68723fd-0908-4e21-ae8e-1de1c3a9a65c"}]': finished 2023-09-09T23:07:52.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:52 smithi170 ceph-mon[102832]: osdmap e10: 2 total, 1 up, 2 in 2023-09-09T23:07:52.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:52 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-09T23:07:52.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:52 smithi082 ceph-mon[98694]: from='client.? 172.21.15.98:0/2137477649' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f68723fd-0908-4e21-ae8e-1de1c3a9a65c"}]: dispatch 2023-09-09T23:07:52.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:52 smithi082 ceph-mon[98694]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f68723fd-0908-4e21-ae8e-1de1c3a9a65c"}]: dispatch 2023-09-09T23:07:52.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:52 smithi082 ceph-mon[98694]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f68723fd-0908-4e21-ae8e-1de1c3a9a65c"}]': finished 2023-09-09T23:07:52.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:52 smithi082 ceph-mon[98694]: osdmap e10: 2 total, 1 up, 2 in 2023-09-09T23:07:52.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:52 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-09T23:07:53.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:52 smithi098 ceph-mon[103124]: from='client.? 172.21.15.98:0/2137477649' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f68723fd-0908-4e21-ae8e-1de1c3a9a65c"}]: dispatch 2023-09-09T23:07:53.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:52 smithi098 ceph-mon[103124]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f68723fd-0908-4e21-ae8e-1de1c3a9a65c"}]: dispatch 2023-09-09T23:07:53.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:52 smithi098 ceph-mon[103124]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f68723fd-0908-4e21-ae8e-1de1c3a9a65c"}]': finished 2023-09-09T23:07:53.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:52 smithi098 ceph-mon[103124]: osdmap e10: 2 total, 1 up, 2 in 2023-09-09T23:07:53.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-09T23:07:53.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:53 smithi170 ceph-mon[102832]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:07:53.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:53 smithi170 ceph-mon[102832]: from='client.? 172.21.15.98:0/3410093730' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:07:53.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:53 smithi082 ceph-mon[98694]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:07:53.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:53 smithi082 ceph-mon[98694]: from='client.? 172.21.15.98:0/3410093730' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:07:54.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:53 smithi098 ceph-mon[103124]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:07:54.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:53 smithi098 ceph-mon[103124]: from='client.? 172.21.15.98:0/3410093730' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:07:56.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:55 smithi098 ceph-mon[103124]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:07:56.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:55 smithi170 ceph-mon[102832]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:07:56.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:55 smithi082 ceph-mon[98694]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:07:58.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:57 smithi098 ceph-mon[103124]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:07:58.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:57 smithi170 ceph-mon[102832]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:07:58.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:57 smithi082 ceph-mon[98694]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:07:59.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:59 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-09-09T23:07:59.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:07:59 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:59.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:59 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-09-09T23:07:59.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:07:59 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:07:59.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:59 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-09-09T23:07:59.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:07:59 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:00.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:00 smithi082 ceph-mon[98694]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:08:00.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:00 smithi082 ceph-mon[98694]: Deploying daemon osd.1 on smithi098 2023-09-09T23:08:00.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:00 smithi098 ceph-mon[103124]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:08:00.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:00 smithi098 ceph-mon[103124]: Deploying daemon osd.1 on smithi098 2023-09-09T23:08:00.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:00 smithi170 ceph-mon[102832]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:08:00.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:00 smithi170 ceph-mon[102832]: Deploying daemon osd.1 on smithi098 2023-09-09T23:08:02.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:02 smithi082 ceph-mon[98694]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:08:02.490 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:02 smithi098 ceph-mon[103124]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:08:02.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:02 smithi170 ceph-mon[102832]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:08:04.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:04 smithi082 ceph-mon[98694]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:08:04.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:04 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:08:04.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:04 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:04.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:04 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:04.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:04 smithi098 ceph-mon[103124]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:08:04.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:04 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:08:04.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:04 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:04.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:04 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:04.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:04 smithi170 ceph-mon[102832]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:08:04.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:04 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:08:04.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:04 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:04.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:04 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:06.422 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:06 smithi098 ceph-mon[103124]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:08:06.422 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:06 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:06.422 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:06 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:06.423 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:06 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:06.423 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:06 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:08:06.423 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:06 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:06.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:06 smithi082 ceph-mon[98694]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:08:06.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:06 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:06.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:06 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:06.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:06 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:06.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:06 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:08:06.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:06 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:06.539 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 1 on host 'smithi098' 2023-09-09T23:08:06.540 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:06.540+0000 7f6c02ffd700 1 -- 172.21.15.98:0/2620220383 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6c0c02cff0 con 0x7f6bf80a05b0 2023-09-09T23:08:06.541 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:06.542+0000 7f6c13b8c700 1 -- 172.21.15.98:0/2620220383 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f6bf80a05b0 msgr2=0x7f6bf80a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:08:06.542 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:06.542+0000 7f6c13b8c700 1 --2- 172.21.15.98:0/2620220383 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f6bf80a05b0 0x7f6bf80a2a70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6c08000c00 tx=0x7f6c08005fb0 comp rx=0 tx=0).stop 2023-09-09T23:08:06.542 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:06.542+0000 7f6c13b8c700 1 -- 172.21.15.98:0/2620220383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c0c101930 msgr2=0x7f6c0c196950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:08:06.542 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:06.542+0000 7f6c13b8c700 1 --2- 172.21.15.98:0/2620220383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c0c101930 0x7f6c0c196950 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00c920 tx=0x7f6bfc00cce0 comp rx=0 tx=0).stop 2023-09-09T23:08:06.543 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:06.543+0000 7f6c13b8c700 1 -- 172.21.15.98:0/2620220383 shutdown_connections 2023-09-09T23:08:06.543 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:06.543+0000 7f6c13b8c700 1 --2- 172.21.15.98:0/2620220383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c0c1022d0 0x7f6c0c19d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:08:06.543 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:06.543+0000 7f6c13b8c700 1 --2- 172.21.15.98:0/2620220383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c0c0fe950 0x7f6c0c196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:08:06.543 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:06.543+0000 7f6c13b8c700 1 --2- 172.21.15.98:0/2620220383 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f6bf80a05b0 0x7f6bf80a2a70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:08:06.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:06.543+0000 7f6c13b8c700 1 --2- 172.21.15.98:0/2620220383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c0c101930 0x7f6c0c196950 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:08:06.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:06.543+0000 7f6c13b8c700 1 -- 172.21.15.98:0/2620220383 >> 172.21.15.98:0/2620220383 conn(0x7f6c0c0fa2b0 msgr2=0x7f6c0c104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:08:06.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:06.543+0000 7f6c13b8c700 1 -- 172.21.15.98:0/2620220383 shutdown_connections 2023-09-09T23:08:06.545 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:06.543+0000 7f6c13b8c700 1 -- 172.21.15.98:0/2620220383 wait complete. 2023-09-09T23:08:06.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:06 smithi170 ceph-mon[102832]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:08:06.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:06 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:06.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:06 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:06.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:06 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:06.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:06 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:08:06.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:06 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:06.885 DEBUG:teuthology.orchestra.run.smithi098:osd.1> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@osd.1.service 2023-09-09T23:08:06.888 INFO:tasks.cephadm:Deploying osd.2 on smithi170 with /dev/vg_nvme/lv_4... 2023-09-09T23:08:06.889 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-09-09T23:08:06.918 INFO:journalctl@ceph.osd.1.smithi098.stdout:-- Logs begin at Sat 2023-09-09 22:51:28 UTC. -- 2023-09-09T23:08:07.450 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-09T23:08:07.643 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 09 23:08:07 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-1[108236]: 2023-09-09T23:08:07.373+0000 7f70520d8640 -1 osd.1 0 log_to_monitors true 2023-09-09T23:08:07.643 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:07 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:07.643 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:07 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:08:07.644 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:07 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:07.644 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:07 smithi098 ceph-mon[103124]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:08:07.644 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:07 smithi098 ceph-mon[103124]: from='osd.1 [v2:172.21.15.98:6800/3528641314,v1:172.21.15.98:6801/3528641314]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-09T23:08:07.644 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:07 smithi098 ceph-mon[103124]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-09T23:08:07.812 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:07 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:07.813 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:07 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:08:07.813 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:07 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:07.814 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:07 smithi170 ceph-mon[102832]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:08:07.814 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:07 smithi170 ceph-mon[102832]: from='osd.1 [v2:172.21.15.98:6800/3528641314,v1:172.21.15.98:6801/3528641314]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-09T23:08:07.815 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:07 smithi170 ceph-mon[102832]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-09T23:08:07.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:07 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:07 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:08:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:07 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:07 smithi082 ceph-mon[98694]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:08:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:07 smithi082 ceph-mon[98694]: from='osd.1 [v2:172.21.15.98:6800/3528641314,v1:172.21.15.98:6801/3528641314]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-09T23:08:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:07 smithi082 ceph-mon[98694]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-09T23:08:08.911 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-09-09T23:08:08.912 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-09-09T23:08:08.912 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2023-09-09T23:08:08.912 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2023-09-09T23:08:08.912 INFO:teuthology.orchestra.run.smithi170.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00823062 s, 1.3 GB/s 2023-09-09T23:08:08.913 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2023-09-09T23:08:08.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:08 smithi170 ceph-mon[102832]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-09-09T23:08:08.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:08 smithi170 ceph-mon[102832]: from='osd.1 [v2:172.21.15.98:6800/3528641314,v1:172.21.15.98:6801/3528641314]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-09T23:08:08.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:08 smithi170 ceph-mon[102832]: osdmap e11: 2 total, 1 up, 2 in 2023-09-09T23:08:08.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:08 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-09T23:08:08.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:08 smithi170 ceph-mon[102832]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-09T23:08:08.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:08 smithi082 ceph-mon[98694]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-09-09T23:08:08.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:08 smithi082 ceph-mon[98694]: from='osd.1 [v2:172.21.15.98:6800/3528641314,v1:172.21.15.98:6801/3528641314]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-09T23:08:08.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:08 smithi082 ceph-mon[98694]: osdmap e11: 2 total, 1 up, 2 in 2023-09-09T23:08:08.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:08 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-09T23:08:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:08 smithi082 ceph-mon[98694]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-09T23:08:09.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:08 smithi098 ceph-mon[103124]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-09-09T23:08:09.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:08 smithi098 ceph-mon[103124]: from='osd.1 [v2:172.21.15.98:6800/3528641314,v1:172.21.15.98:6801/3528641314]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-09T23:08:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:08 smithi098 ceph-mon[103124]: osdmap e11: 2 total, 1 up, 2 in 2023-09-09T23:08:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:08 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-09T23:08:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:08 smithi098 ceph-mon[103124]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-09T23:08:09.003 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 09 23:08:08 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-1[108236]: 2023-09-09T23:08:08.512+0000 7f7046f4e700 -1 osd.1 0 waiting for initial osdmap 2023-09-09T23:08:09.004 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 09 23:08:08 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-1[108236]: 2023-09-09T23:08:08.515+0000 7f7040d3f700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-09T23:08:09.307 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi170:vg_nvme/lv_4 2023-09-09T23:08:09.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:09 smithi170 ceph-mon[102832]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-09-09T23:08:09.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:09 smithi170 ceph-mon[102832]: osdmap e12: 2 total, 1 up, 2 in 2023-09-09T23:08:09.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:09 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-09T23:08:09.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:09 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-09T23:08:09.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:09 smithi170 ceph-mon[102832]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:08:09.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:09 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:09 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:09 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:08:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:09 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:09 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:09 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:08:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:09 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:09.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:09 smithi098 ceph-mon[103124]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-09-09T23:08:09.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:09 smithi098 ceph-mon[103124]: osdmap e12: 2 total, 1 up, 2 in 2023-09-09T23:08:09.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:09 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-09T23:08:09.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:09 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-09T23:08:09.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:09 smithi098 ceph-mon[103124]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:08:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:09 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:09 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:09 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:08:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:09 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:09 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:09 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:08:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:09 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:09.862 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-09T23:08:09.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:09 smithi082 ceph-mon[98694]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-09-09T23:08:09.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:09 smithi082 ceph-mon[98694]: osdmap e12: 2 total, 1 up, 2 in 2023-09-09T23:08:09.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-09T23:08:09.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-09T23:08:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:09 smithi082 ceph-mon[98694]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-09T23:08:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:08:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:08:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:10.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:10 smithi170 ceph-mon[102832]: purged_snaps scrub starts 2023-09-09T23:08:10.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:10 smithi170 ceph-mon[102832]: purged_snaps scrub ok 2023-09-09T23:08:10.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:10 smithi170 ceph-mon[102832]: Detected new or changed devices on smithi098 2023-09-09T23:08:10.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:10 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 17168M 2023-09-09T23:08:10.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:10 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-09T23:08:10.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:10 smithi170 ceph-mon[102832]: osd.1 [v2:172.21.15.98:6800/3528641314,v1:172.21.15.98:6801/3528641314] boot 2023-09-09T23:08:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:10 smithi170 ceph-mon[102832]: osdmap e13: 2 total, 2 up, 2 in 2023-09-09T23:08:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:10 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-09T23:08:10.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:10 smithi082 ceph-mon[98694]: purged_snaps scrub starts 2023-09-09T23:08:10.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:10 smithi082 ceph-mon[98694]: purged_snaps scrub ok 2023-09-09T23:08:10.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:10 smithi082 ceph-mon[98694]: Detected new or changed devices on smithi098 2023-09-09T23:08:10.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:10 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 17168M 2023-09-09T23:08:10.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:10 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-09T23:08:10.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:10 smithi082 ceph-mon[98694]: osd.1 [v2:172.21.15.98:6800/3528641314,v1:172.21.15.98:6801/3528641314] boot 2023-09-09T23:08:10.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:10 smithi082 ceph-mon[98694]: osdmap e13: 2 total, 2 up, 2 in 2023-09-09T23:08:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:10 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-09T23:08:11.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:10 smithi098 ceph-mon[103124]: purged_snaps scrub starts 2023-09-09T23:08:11.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:10 smithi098 ceph-mon[103124]: purged_snaps scrub ok 2023-09-09T23:08:11.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:10 smithi098 ceph-mon[103124]: Detected new or changed devices on smithi098 2023-09-09T23:08:11.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:10 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 17168M 2023-09-09T23:08:11.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:10 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-09T23:08:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:10 smithi098 ceph-mon[103124]: osd.1 [v2:172.21.15.98:6800/3528641314,v1:172.21.15.98:6801/3528641314] boot 2023-09-09T23:08:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:10 smithi098 ceph-mon[103124]: osdmap e13: 2 total, 2 up, 2 in 2023-09-09T23:08:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:10 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-09T23:08:11.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.029+0000 7fe6bdae8700 1 -- 172.21.15.170:0/1686754864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6b8101130 msgr2=0x7fe6b81079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:08:11.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.029+0000 7fe6bdae8700 1 --2- 172.21.15.170:0/1686754864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6b8101130 0x7fe6b81079e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe6b000b0a0 tx=0x7fe6b000b3b0 comp rx=0 tx=0).stop 2023-09-09T23:08:11.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.029+0000 7fe6bdae8700 1 -- 172.21.15.170:0/1686754864 shutdown_connections 2023-09-09T23:08:11.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.029+0000 7fe6bdae8700 1 --2- 172.21.15.170:0/1686754864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6b80ff590 0x7fe6b80ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:08:11.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.029+0000 7fe6bdae8700 1 --2- 172.21.15.170:0/1686754864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6b8101130 0x7fe6b81079e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:08:11.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.029+0000 7fe6bdae8700 1 --2- 172.21.15.170:0/1686754864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6b8100790 0x7fe6b8100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:08:11.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.029+0000 7fe6bdae8700 1 -- 172.21.15.170:0/1686754864 >> 172.21.15.170:0/1686754864 conn(0x7fe6b80fab10 msgr2=0x7fe6b80fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:08:11.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.029+0000 7fe6bdae8700 1 -- 172.21.15.170:0/1686754864 shutdown_connections 2023-09-09T23:08:11.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.029+0000 7fe6bdae8700 1 -- 172.21.15.170:0/1686754864 wait complete. 2023-09-09T23:08:11.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.030+0000 7fe6bdae8700 1 Processor -- start 2023-09-09T23:08:11.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.030+0000 7fe6bdae8700 1 -- start start 2023-09-09T23:08:11.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.030+0000 7fe6bdae8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6b80ff590 0x7fe6b81952d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:08:11.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.031+0000 7fe6bdae8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6b8100790 0x7fe6b8195810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:08:11.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.031+0000 7fe6bdae8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6b8101130 0x7fe6b819c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:08:11.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.031+0000 7fe6bdae8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b819ce60 con 0x7fe6b80ff590 2023-09-09T23:08:11.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.031+0000 7fe6bdae8700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b819cfa0 con 0x7fe6b8101130 2023-09-09T23:08:11.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.031+0000 7fe6bdae8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b819d130 con 0x7fe6b8100790 2023-09-09T23:08:11.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.031+0000 7fe6b6ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6b8100790 0x7fe6b8195810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:08:11.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.031+0000 7fe6b6ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6b8100790 0x7fe6b8195810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34368/0 (socket says 172.21.15.170:34368) 2023-09-09T23:08:11.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.031+0000 7fe6b6ffd700 1 -- 172.21.15.170:0/2025350932 learned_addr learned my addr 172.21.15.170:0/2025350932 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-09-09T23:08:11.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.031+0000 7fe6b77fe700 1 --2- 172.21.15.170:0/2025350932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6b80ff590 0x7fe6b81952d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:08:11.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.031+0000 7fe6b6ffd700 1 -- 172.21.15.170:0/2025350932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6b8101130 msgr2=0x7fe6b819c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:08:11.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.031+0000 7fe6b6ffd700 1 --2- 172.21.15.170:0/2025350932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6b8101130 0x7fe6b819c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:08:11.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.031+0000 7fe6b6ffd700 1 -- 172.21.15.170:0/2025350932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6b80ff590 msgr2=0x7fe6b81952d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:08:11.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.031+0000 7fe6b6ffd700 1 --2- 172.21.15.170:0/2025350932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6b80ff590 0x7fe6b81952d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:08:11.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.031+0000 7fe6b6ffd700 1 -- 172.21.15.170:0/2025350932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b0009d00 con 0x7fe6b8100790 2023-09-09T23:08:11.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.031+0000 7fe6b77fe700 1 --2- 172.21.15.170:0/2025350932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6b80ff590 0x7fe6b81952d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:08:11.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.031+0000 7fe6b6ffd700 1 --2- 172.21.15.170:0/2025350932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6b8100790 0x7fe6b8195810 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800c9e0 tx=0x7fe6a800cda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:08:11.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.031+0000 7fe6b4ff9700 1 -- 172.21.15.170:0/2025350932 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8007ac0 con 0x7fe6b8100790 2023-09-09T23:08:11.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.031+0000 7fe6b4ff9700 1 -- 172.21.15.170:0/2025350932 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6a8007c20 con 0x7fe6b8100790 2023-09-09T23:08:11.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.031+0000 7fe6bdae8700 1 -- 172.21.15.170:0/2025350932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b819d410 con 0x7fe6b8100790 2023-09-09T23:08:11.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.031+0000 7fe6b4ff9700 1 -- 172.21.15.170:0/2025350932 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8018660 con 0x7fe6b8100790 2023-09-09T23:08:11.036 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.031+0000 7fe6bdae8700 1 -- 172.21.15.170:0/2025350932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b819d960 con 0x7fe6b8100790 2023-09-09T23:08:11.036 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.032+0000 7fe6bdae8700 1 -- 172.21.15.170:0/2025350932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b810adf0 con 0x7fe6b8100790 2023-09-09T23:08:11.036 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.033+0000 7fe6b4ff9700 1 -- 172.21.15.170:0/2025350932 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 15) v1 ==== 135203+0+0 (secure 0 0 0) 0x7fe6a800f450 con 0x7fe6b8100790 2023-09-09T23:08:11.037 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.034+0000 7fe6b4ff9700 1 --2- 172.21.15.170:0/2025350932 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fe6a40a0630 0x7fe6a40a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:08:11.037 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.034+0000 7fe6b4ff9700 1 -- 172.21.15.170:0/2025350932 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7fe6a80ca280 con 0x7fe6b8100790 2023-09-09T23:08:11.037 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.034+0000 7fe6b77fe700 1 --2- 172.21.15.170:0/2025350932 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fe6a40a0630 0x7fe6a40a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:08:11.037 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.034+0000 7fe6b77fe700 1 --2- 172.21.15.170:0/2025350932 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fe6a40a0630 0x7fe6a40a2af0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0005950 tx=0x7fe6a00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:08:11.038 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.039+0000 7fe6b4ff9700 1 -- 172.21.15.170:0/2025350932 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe6a8095600 con 0x7fe6b8100790 2023-09-09T23:08:11.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:11.234+0000 7fe6bdae8700 1 -- 172.21.15.170:0/2025350932 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6b8060a40 con 0x7fe6a40a0630 2023-09-09T23:08:11.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:11 smithi170 ceph-mon[102832]: osdmap e14: 2 total, 2 up, 2 in 2023-09-09T23:08:11.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:11 smithi170 ceph-mon[102832]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:11.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:11 smithi170 ceph-mon[102832]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:08:11.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:11 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:08:11.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:11 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:08:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:11 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:11.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:11 smithi082 ceph-mon[98694]: osdmap e14: 2 total, 2 up, 2 in 2023-09-09T23:08:11.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:11 smithi082 ceph-mon[98694]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:11.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:11 smithi082 ceph-mon[98694]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:08:11.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:11 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:08:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:11 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:08:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:11 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:12.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:11 smithi098 ceph-mon[103124]: osdmap e14: 2 total, 2 up, 2 in 2023-09-09T23:08:12.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:11 smithi098 ceph-mon[103124]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:12.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:11 smithi098 ceph-mon[103124]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:08:12.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:11 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:08:12.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:11 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:08:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:11 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:14.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:14 smithi082 ceph-mon[98694]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:14.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:14 smithi082 ceph-mon[98694]: from='client.? 172.21.15.170:0/1054352788' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06f89c53-1419-4f52-acad-6e92916b476f"}]: dispatch 2023-09-09T23:08:14.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:14 smithi082 ceph-mon[98694]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06f89c53-1419-4f52-acad-6e92916b476f"}]: dispatch 2023-09-09T23:08:14.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:14 smithi082 ceph-mon[98694]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06f89c53-1419-4f52-acad-6e92916b476f"}]': finished 2023-09-09T23:08:14.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:14 smithi082 ceph-mon[98694]: osdmap e15: 3 total, 2 up, 3 in 2023-09-09T23:08:14.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:14 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-09T23:08:14.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:14 smithi082 ceph-mon[98694]: from='client.? 172.21.15.170:0/3926808819' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:08:14.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:14 smithi098 ceph-mon[103124]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:14.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:14 smithi098 ceph-mon[103124]: from='client.? 172.21.15.170:0/1054352788' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06f89c53-1419-4f52-acad-6e92916b476f"}]: dispatch 2023-09-09T23:08:14.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:14 smithi098 ceph-mon[103124]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06f89c53-1419-4f52-acad-6e92916b476f"}]: dispatch 2023-09-09T23:08:14.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:14 smithi098 ceph-mon[103124]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06f89c53-1419-4f52-acad-6e92916b476f"}]': finished 2023-09-09T23:08:14.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:14 smithi098 ceph-mon[103124]: osdmap e15: 3 total, 2 up, 3 in 2023-09-09T23:08:14.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:14 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-09T23:08:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:14 smithi098 ceph-mon[103124]: from='client.? 172.21.15.170:0/3926808819' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:08:14.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:14 smithi170 ceph-mon[102832]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:14.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:14 smithi170 ceph-mon[102832]: from='client.? 172.21.15.170:0/1054352788' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06f89c53-1419-4f52-acad-6e92916b476f"}]: dispatch 2023-09-09T23:08:14.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:14 smithi170 ceph-mon[102832]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06f89c53-1419-4f52-acad-6e92916b476f"}]: dispatch 2023-09-09T23:08:14.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:14 smithi170 ceph-mon[102832]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06f89c53-1419-4f52-acad-6e92916b476f"}]': finished 2023-09-09T23:08:14.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:14 smithi170 ceph-mon[102832]: osdmap e15: 3 total, 2 up, 3 in 2023-09-09T23:08:14.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:14 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-09T23:08:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:14 smithi170 ceph-mon[102832]: from='client.? 172.21.15.170:0/3926808819' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:08:16.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:16 smithi082 ceph-mon[98694]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:16.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:16 smithi098 ceph-mon[103124]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:16.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:16 smithi170 ceph-mon[102832]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:18.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:18 smithi082 ceph-mon[98694]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:18.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:18 smithi098 ceph-mon[103124]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:18.530 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:18 smithi170 ceph-mon[102832]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:20.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:20 smithi082 ceph-mon[98694]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:20.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:20 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-09-09T23:08:20.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:20 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:20.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:20 smithi098 ceph-mon[103124]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:20.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:20 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-09-09T23:08:20.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:20 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:20.503 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:20 smithi170 ceph-mon[102832]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:20.503 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:20 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-09-09T23:08:20.504 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:20 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:21.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:21 smithi082 ceph-mon[98694]: Deploying daemon osd.2 on smithi170 2023-09-09T23:08:21.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:21 smithi098 ceph-mon[103124]: Deploying daemon osd.2 on smithi170 2023-09-09T23:08:21.667 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:21 smithi170 ceph-mon[102832]: Deploying daemon osd.2 on smithi170 2023-09-09T23:08:22.284 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:22 smithi170 ceph-mon[102832]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:22.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:22 smithi098 ceph-mon[103124]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:22.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:22 smithi082 ceph-mon[98694]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:24.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:24 smithi098 ceph-mon[103124]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:24.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:24 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:24.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:24 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:24.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:24 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:08:24.516 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:24 smithi170 ceph-mon[102832]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:24.517 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:24 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:24.517 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:24 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:24.517 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:24 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:08:24.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:24 smithi082 ceph-mon[98694]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:24.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:24 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:24.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:24 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:24.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:24 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:08:26.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:26 smithi098 ceph-mon[103124]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:26.605 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:26 smithi170 ceph-mon[102832]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:26.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:26 smithi082 ceph-mon[98694]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:26.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:26.741+0000 7fe6b4ff9700 1 -- 172.21.15.170:0/2025350932 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe6b8060a40 con 0x7fe6a40a0630 2023-09-09T23:08:26.740 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 2 on host 'smithi170' 2023-09-09T23:08:26.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:26.743+0000 7fe6bdae8700 1 -- 172.21.15.170:0/2025350932 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fe6a40a0630 msgr2=0x7fe6a40a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:08:26.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:26.743+0000 7fe6bdae8700 1 --2- 172.21.15.170:0/2025350932 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fe6a40a0630 0x7fe6a40a2af0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0005950 tx=0x7fe6a00058e0 comp rx=0 tx=0).stop 2023-09-09T23:08:26.743 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:26.743+0000 7fe6bdae8700 1 -- 172.21.15.170:0/2025350932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6b8100790 msgr2=0x7fe6b8195810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:08:26.743 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:26.743+0000 7fe6bdae8700 1 --2- 172.21.15.170:0/2025350932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6b8100790 0x7fe6b8195810 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800c9e0 tx=0x7fe6a800cda0 comp rx=0 tx=0).stop 2023-09-09T23:08:26.743 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:26.743+0000 7fe6bdae8700 1 -- 172.21.15.170:0/2025350932 shutdown_connections 2023-09-09T23:08:26.743 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:26.743+0000 7fe6bdae8700 1 --2- 172.21.15.170:0/2025350932 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fe6a40a0630 0x7fe6a40a2af0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:08:26.743 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:26.743+0000 7fe6bdae8700 1 --2- 172.21.15.170:0/2025350932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6b8101130 0x7fe6b819c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:08:26.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:26.743+0000 7fe6bdae8700 1 --2- 172.21.15.170:0/2025350932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6b80ff590 0x7fe6b81952d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:08:26.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:26.743+0000 7fe6bdae8700 1 --2- 172.21.15.170:0/2025350932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6b8100790 0x7fe6b8195810 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:08:26.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:26.743+0000 7fe6bdae8700 1 -- 172.21.15.170:0/2025350932 >> 172.21.15.170:0/2025350932 conn(0x7fe6b80fab10 msgr2=0x7fe6b80fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:08:26.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:26.743+0000 7fe6bdae8700 1 -- 172.21.15.170:0/2025350932 shutdown_connections 2023-09-09T23:08:26.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:08:26.744+0000 7fe6bdae8700 1 -- 172.21.15.170:0/2025350932 wait complete. 2023-09-09T23:08:27.138 DEBUG:teuthology.orchestra.run.smithi170:osd.2> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@osd.2.service 2023-09-09T23:08:27.141 INFO:tasks.cephadm:Deploying osd.3 on smithi082 with /dev/vg_nvme/lv_3... 2023-09-09T23:08:27.142 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-09-09T23:08:27.174 INFO:journalctl@ceph.osd.2.smithi170.stdout:-- Logs begin at Sat 2023-09-09 22:51:11 UTC. -- 2023-09-09T23:08:27.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:27 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:27.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:27 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:27.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:27 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:27.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:27 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:08:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:27 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:27 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:27 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:08:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:27 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:27.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:27 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:27.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:27 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:27.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:27 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:27.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:27 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:08:27.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:27 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:27.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:27 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:27.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:27 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:08:27.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:27 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:27.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:27 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:27.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:27 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:27.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:27 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:27.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:27 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:08:27.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:27 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:27 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:27 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:08:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:27 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:27.709 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:08:28.222 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 09 23:08:28 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-2[108651]: 2023-09-09T23:08:28.053+0000 7f7717bc1640 -1 osd.2 0 log_to_monitors true 2023-09-09T23:08:28.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:28 smithi082 ceph-mon[98694]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:28.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:28 smithi082 ceph-mon[98694]: from='osd.2 [v2:172.21.15.170:6800/4006414551,v1:172.21.15.170:6801/4006414551]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-09T23:08:28.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:28 smithi098 ceph-mon[103124]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:28.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:28 smithi098 ceph-mon[103124]: from='osd.2 [v2:172.21.15.170:6800/4006414551,v1:172.21.15.170:6801/4006414551]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-09T23:08:28.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:28 smithi170 ceph-mon[102832]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:28.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:28 smithi170 ceph-mon[102832]: from='osd.2 [v2:172.21.15.170:6800/4006414551,v1:172.21.15.170:6801/4006414551]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-09T23:08:29.193 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-09-09T23:08:29.193 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-09-09T23:08:29.194 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2023-09-09T23:08:29.194 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2023-09-09T23:08:29.194 INFO:teuthology.orchestra.run.smithi082.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111233 s, 943 MB/s 2023-09-09T23:08:29.194 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2023-09-09T23:08:29.562 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi082:vg_nvme/lv_3 2023-09-09T23:08:29.925 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 09 23:08:29 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-2[108651]: 2023-09-09T23:08:29.709+0000 7f770ca37700 -1 osd.2 0 waiting for initial osdmap 2023-09-09T23:08:29.926 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 09 23:08:29 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-2[108651]: 2023-09-09T23:08:29.718+0000 7f770902d700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-09T23:08:29.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:29 smithi170 ceph-mon[102832]: from='osd.2 [v2:172.21.15.170:6800/4006414551,v1:172.21.15.170:6801/4006414551]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-09-09T23:08:29.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:29 smithi170 ceph-mon[102832]: osdmap e16: 3 total, 2 up, 3 in 2023-09-09T23:08:29.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:29 smithi170 ceph-mon[102832]: from='osd.2 [v2:172.21.15.170:6800/4006414551,v1:172.21.15.170:6801/4006414551]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-09T23:08:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:29 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-09T23:08:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:29 smithi170 ceph-mon[102832]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:29.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:29 smithi082 ceph-mon[98694]: from='osd.2 [v2:172.21.15.170:6800/4006414551,v1:172.21.15.170:6801/4006414551]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-09-09T23:08:29.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:29 smithi082 ceph-mon[98694]: osdmap e16: 3 total, 2 up, 3 in 2023-09-09T23:08:29.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:29 smithi082 ceph-mon[98694]: from='osd.2 [v2:172.21.15.170:6800/4006414551,v1:172.21.15.170:6801/4006414551]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-09T23:08:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:29 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-09T23:08:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:29 smithi082 ceph-mon[98694]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:30.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:29 smithi098 ceph-mon[103124]: from='osd.2 [v2:172.21.15.170:6800/4006414551,v1:172.21.15.170:6801/4006414551]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-09-09T23:08:30.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:29 smithi098 ceph-mon[103124]: osdmap e16: 3 total, 2 up, 3 in 2023-09-09T23:08:30.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:29 smithi098 ceph-mon[103124]: from='osd.2 [v2:172.21.15.170:6800/4006414551,v1:172.21.15.170:6801/4006414551]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-09T23:08:30.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:29 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-09T23:08:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:29 smithi098 ceph-mon[103124]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:30.120 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:08:30.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:30 smithi082 ceph-mon[98694]: from='osd.2 [v2:172.21.15.170:6800/4006414551,v1:172.21.15.170:6801/4006414551]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-09-09T23:08:30.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:30 smithi082 ceph-mon[98694]: osdmap e17: 3 total, 2 up, 3 in 2023-09-09T23:08:30.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:30 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-09T23:08:30.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:30 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-09T23:08:30.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:30 smithi082 ceph-mon[98694]: Detected new or changed devices on smithi170 2023-09-09T23:08:30.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:30 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:30.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:30 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:30.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:30 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:08:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:30 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 17168M 2023-09-09T23:08:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:30 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:30 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:30 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:08:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:30 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:31.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:30 smithi098 ceph-mon[103124]: from='osd.2 [v2:172.21.15.170:6800/4006414551,v1:172.21.15.170:6801/4006414551]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-09-09T23:08:31.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:30 smithi098 ceph-mon[103124]: osdmap e17: 3 total, 2 up, 3 in 2023-09-09T23:08:31.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:30 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-09T23:08:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:30 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-09T23:08:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:30 smithi098 ceph-mon[103124]: Detected new or changed devices on smithi170 2023-09-09T23:08:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:30 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:30 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:30 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:08:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:30 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 17168M 2023-09-09T23:08:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:30 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:30 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:30 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:08:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:30 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:31.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:30 smithi170 ceph-mon[102832]: from='osd.2 [v2:172.21.15.170:6800/4006414551,v1:172.21.15.170:6801/4006414551]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-09-09T23:08:31.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:30 smithi170 ceph-mon[102832]: osdmap e17: 3 total, 2 up, 3 in 2023-09-09T23:08:31.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:30 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-09T23:08:31.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:30 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-09T23:08:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:30 smithi170 ceph-mon[102832]: Detected new or changed devices on smithi170 2023-09-09T23:08:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:30 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:30 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:30 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:08:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:30 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 17168M 2023-09-09T23:08:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:30 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:30 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:30 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:08:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:30 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:31.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.281+0000 7f023d1bb700 1 -- 172.21.15.82:0/1224620150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0238101180 msgr2=0x7f0238103570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:08:31.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.281+0000 7f023d1bb700 1 --2- 172.21.15.82:0/1224620150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0238101180 0x7f0238103570 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f022c009990 tx=0x7f022c009ca0 comp rx=0 tx=0).stop 2023-09-09T23:08:31.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.281+0000 7f023d1bb700 1 -- 172.21.15.82:0/1224620150 shutdown_connections 2023-09-09T23:08:31.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.281+0000 7f023d1bb700 1 --2- 172.21.15.82:0/1224620150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02381063e0 0x7f02381087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:08:31.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.281+0000 7f023d1bb700 1 --2- 172.21.15.82:0/1224620150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0238101180 0x7f0238103570 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:08:31.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.281+0000 7f023d1bb700 1 --2- 172.21.15.82:0/1224620150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0238103ab0 0x7f0238105ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:08:31.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.281+0000 7f023d1bb700 1 -- 172.21.15.82:0/1224620150 >> 172.21.15.82:0/1224620150 conn(0x7f02380fab10 msgr2=0x7f02380fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:08:31.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.281+0000 7f023d1bb700 1 -- 172.21.15.82:0/1224620150 shutdown_connections 2023-09-09T23:08:31.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.281+0000 7f023d1bb700 1 -- 172.21.15.82:0/1224620150 wait complete. 2023-09-09T23:08:31.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.281+0000 7f023d1bb700 1 Processor -- start 2023-09-09T23:08:31.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.282+0000 7f023d1bb700 1 -- start start 2023-09-09T23:08:31.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.282+0000 7f023d1bb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0238101180 0x7f0238196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:08:31.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.282+0000 7f023d1bb700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0238103ab0 0x7f0238196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:08:31.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.282+0000 7f023d1bb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02381063e0 0x7f023819d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:08:31.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.282+0000 7f023d1bb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023819dec0 con 0x7f0238101180 2023-09-09T23:08:31.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.282+0000 7f023d1bb700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023819e000 con 0x7f0238103ab0 2023-09-09T23:08:31.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.282+0000 7f023d1bb700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023819e190 con 0x7f02381063e0 2023-09-09T23:08:31.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.282+0000 7f0236d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0238101180 0x7f0238196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:08:31.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.282+0000 7f0236d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0238101180 0x7f0238196330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53590/0 (socket says 172.21.15.82:53590) 2023-09-09T23:08:31.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.282+0000 7f0236d9d700 1 -- 172.21.15.82:0/3593169832 learned_addr learned my addr 172.21.15.82:0/3593169832 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:08:31.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.282+0000 7f023759e700 1 --2- 172.21.15.82:0/3593169832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02381063e0 0x7f023819d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:08:31.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.282+0000 7f023659c700 1 --2- 172.21.15.82:0/3593169832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0238103ab0 0x7f0238196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:08:31.286 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.282+0000 7f0236d9d700 1 -- 172.21.15.82:0/3593169832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02381063e0 msgr2=0x7f023819d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:08:31.286 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.282+0000 7f0236d9d700 1 --2- 172.21.15.82:0/3593169832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02381063e0 0x7f023819d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:08:31.286 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.282+0000 7f0236d9d700 1 -- 172.21.15.82:0/3593169832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0238103ab0 msgr2=0x7f0238196870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:08:31.287 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.282+0000 7f0236d9d700 1 --2- 172.21.15.82:0/3593169832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0238103ab0 0x7f0238196870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:08:31.287 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.282+0000 7f0236d9d700 1 -- 172.21.15.82:0/3593169832 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022c0095f0 con 0x7f0238101180 2023-09-09T23:08:31.287 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.282+0000 7f023659c700 1 --2- 172.21.15.82:0/3593169832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0238103ab0 0x7f0238196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:08:31.287 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.283+0000 7f0236d9d700 1 --2- 172.21.15.82:0/3593169832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0238101180 0x7f0238196330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f022c000c00 tx=0x7f022c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:08:31.288 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.283+0000 7f021ffff700 1 -- 172.21.15.82:0/3593169832 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022c01d070 con 0x7f0238101180 2023-09-09T23:08:31.288 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.283+0000 7f023d1bb700 1 -- 172.21.15.82:0/3593169832 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f023819e410 con 0x7f0238101180 2023-09-09T23:08:31.288 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.283+0000 7f021ffff700 1 -- 172.21.15.82:0/3593169832 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f022c022470 con 0x7f0238101180 2023-09-09T23:08:31.288 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.283+0000 7f021ffff700 1 -- 172.21.15.82:0/3593169832 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022c017650 con 0x7f0238101180 2023-09-09T23:08:31.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.283+0000 7f023d1bb700 1 -- 172.21.15.82:0/3593169832 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f023819e880 con 0x7f0238101180 2023-09-09T23:08:31.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.284+0000 7f023d1bb700 1 -- 172.21.15.82:0/3593169832 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02380fc690 con 0x7f0238101180 2023-09-09T23:08:31.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.285+0000 7f021ffff700 1 -- 172.21.15.82:0/3593169832 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 15) v1 ==== 135203+0+0 (secure 0 0 0) 0x7f022c0225e0 con 0x7f0238101180 2023-09-09T23:08:31.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.285+0000 7f021ffff700 1 --2- 172.21.15.82:0/3593169832 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f02200a0630 0x7f02200a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:08:31.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.286+0000 7f023659c700 1 --2- 172.21.15.82:0/3593169832 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f02200a0630 0x7f02200a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:08:31.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.286+0000 7f021ffff700 1 -- 172.21.15.82:0/3593169832 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f022c0cbb30 con 0x7f0238101180 2023-09-09T23:08:31.292 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.286+0000 7f023659c700 1 --2- 172.21.15.82:0/3593169832 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f02200a0630 0x7f02200a2af0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0228005950 tx=0x7f02280058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:08:31.292 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.289+0000 7f021ffff700 1 -- 172.21.15.82:0/3593169832 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f022c096c80 con 0x7f0238101180 2023-09-09T23:08:31.486 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:31.488+0000 7f023d1bb700 1 -- 172.21.15.82:0/3593169832 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f023802cf10 con 0x7f02200a0630 2023-09-09T23:08:32.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:31 smithi098 ceph-mon[103124]: purged_snaps scrub starts 2023-09-09T23:08:32.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:31 smithi098 ceph-mon[103124]: purged_snaps scrub ok 2023-09-09T23:08:32.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:31 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-09T23:08:32.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:31 smithi098 ceph-mon[103124]: osd.2 [v2:172.21.15.170:6800/4006414551,v1:172.21.15.170:6801/4006414551] boot 2023-09-09T23:08:32.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:31 smithi098 ceph-mon[103124]: osdmap e18: 3 total, 3 up, 3 in 2023-09-09T23:08:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:31 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-09T23:08:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:31 smithi098 ceph-mon[103124]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:31 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-09-09T23:08:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:31 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:08:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:31 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:08:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:31 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:32.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:31 smithi170 ceph-mon[102832]: purged_snaps scrub starts 2023-09-09T23:08:32.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:31 smithi170 ceph-mon[102832]: purged_snaps scrub ok 2023-09-09T23:08:32.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:31 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-09T23:08:32.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:31 smithi170 ceph-mon[102832]: osd.2 [v2:172.21.15.170:6800/4006414551,v1:172.21.15.170:6801/4006414551] boot 2023-09-09T23:08:32.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:31 smithi170 ceph-mon[102832]: osdmap e18: 3 total, 3 up, 3 in 2023-09-09T23:08:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:31 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-09T23:08:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:31 smithi170 ceph-mon[102832]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:31 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-09-09T23:08:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:31 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:08:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:31 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:08:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:31 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:32.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:31 smithi082 ceph-mon[98694]: purged_snaps scrub starts 2023-09-09T23:08:32.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:31 smithi082 ceph-mon[98694]: purged_snaps scrub ok 2023-09-09T23:08:32.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:31 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-09T23:08:32.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:31 smithi082 ceph-mon[98694]: osd.2 [v2:172.21.15.170:6800/4006414551,v1:172.21.15.170:6801/4006414551] boot 2023-09-09T23:08:32.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:31 smithi082 ceph-mon[98694]: osdmap e18: 3 total, 3 up, 3 in 2023-09-09T23:08:32.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:31 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-09T23:08:32.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:31 smithi082 ceph-mon[98694]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-09T23:08:32.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:31 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-09-09T23:08:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:31 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:08:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:31 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:08:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:31 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:32.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:32 smithi082 ceph-mon[98694]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:08:32.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:32 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-09-09T23:08:32.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:32 smithi082 ceph-mon[98694]: osdmap e19: 3 total, 3 up, 3 in 2023-09-09T23:08:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:32 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-09-09T23:08:33.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:32 smithi098 ceph-mon[103124]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:08:33.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:32 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-09-09T23:08:33.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:32 smithi098 ceph-mon[103124]: osdmap e19: 3 total, 3 up, 3 in 2023-09-09T23:08:33.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:32 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-09-09T23:08:33.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:32 smithi170 ceph-mon[102832]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:08:33.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:32 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-09-09T23:08:33.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:32 smithi170 ceph-mon[102832]: osdmap e19: 3 total, 3 up, 3 in 2023-09-09T23:08:33.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:32 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-09-09T23:08:33.394 INFO:journalctl@ceph.osd.0.smithi082.stdout:Sep 09 23:08:33 smithi082 sudo[127580]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-09T23:08:33.395 INFO:journalctl@ceph.osd.0.smithi082.stdout:Sep 09 23:08:33 smithi082 sudo[127580]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-09T23:08:33.395 INFO:journalctl@ceph.osd.0.smithi082.stdout:Sep 09 23:08:33 smithi082 sudo[127580]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-09T23:08:33.395 INFO:journalctl@ceph.osd.0.smithi082.stdout:Sep 09 23:08:33 smithi082 sudo[127580]: pam_unix(sudo:session): session closed for user root 2023-09-09T23:08:33.705 INFO:journalctl@ceph.osd.0.smithi082.stdout:Sep 09 23:08:33 smithi082 sudo[127593]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-09T23:08:33.705 INFO:journalctl@ceph.osd.0.smithi082.stdout:Sep 09 23:08:33 smithi082 sudo[127593]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-09T23:08:33.705 INFO:journalctl@ceph.osd.0.smithi082.stdout:Sep 09 23:08:33 smithi082 sudo[127593]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-09T23:08:33.706 INFO:journalctl@ceph.osd.0.smithi082.stdout:Sep 09 23:08:33 smithi082 sudo[127593]: pam_unix(sudo:session): session closed for user root 2023-09-09T23:08:33.922 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:33 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-09-09T23:08:33.922 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:33 smithi098 ceph-mon[103124]: osdmap e20: 3 total, 3 up, 3 in 2023-09-09T23:08:33.923 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:33 smithi098 ceph-mon[103124]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-09T23:08:33.923 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:33 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:33.923 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 09 23:08:33 smithi098 sudo[112962]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-09T23:08:33.923 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 09 23:08:33 smithi098 sudo[112962]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-09T23:08:33.924 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 09 23:08:33 smithi098 sudo[112962]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-09T23:08:33.924 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 09 23:08:33 smithi098 sudo[112962]: pam_unix(sudo:session): session closed for user root 2023-09-09T23:08:34.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:33 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-09-09T23:08:34.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:33 smithi170 ceph-mon[102832]: osdmap e20: 3 total, 3 up, 3 in 2023-09-09T23:08:34.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:33 smithi170 ceph-mon[102832]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-09T23:08:34.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:33 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:34.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:33 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-09-09T23:08:34.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:33 smithi082 ceph-mon[98694]: osdmap e20: 3 total, 3 up, 3 in 2023-09-09T23:08:34.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:33 smithi082 ceph-mon[98694]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-09T23:08:34.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:33 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:34.251 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 09 23:08:33 smithi098 sudo[112965]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-09T23:08:34.252 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 09 23:08:33 smithi098 sudo[112965]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-09T23:08:34.252 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 09 23:08:33 smithi098 sudo[112965]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-09T23:08:34.252 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 09 23:08:33 smithi098 sudo[112965]: pam_unix(sudo:session): session closed for user root 2023-09-09T23:08:34.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:34.434+0000 7f021ffff700 1 -- 172.21.15.82:0/3593169832 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f022c097050 con 0x7f0238101180 2023-09-09T23:08:34.439 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 09 23:08:34 smithi170 sudo[114096]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-09T23:08:34.440 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 09 23:08:34 smithi170 sudo[114096]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-09T23:08:34.440 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 09 23:08:34 smithi170 sudo[114096]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-09T23:08:34.440 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 09 23:08:34 smithi170 sudo[114096]: pam_unix(sudo:session): session closed for user root 2023-09-09T23:08:34.925 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 09 23:08:34 smithi170 sudo[114099]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-09T23:08:34.926 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 09 23:08:34 smithi170 sudo[114099]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-09T23:08:34.926 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 09 23:08:34 smithi170 sudo[114099]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-09T23:08:34.926 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 09 23:08:34 smithi170 sudo[114099]: pam_unix(sudo:session): session closed for user root 2023-09-09T23:08:34.962 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:34 smithi082 sudo[127664]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-09T23:08:34.962 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:34 smithi082 sudo[127664]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-09T23:08:34.963 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:34 smithi082 sudo[127664]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-09T23:08:34.963 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:34 smithi082 sudo[127664]: pam_unix(sudo:session): session closed for user root 2023-09-09T23:08:35.223 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:34 smithi170 ceph-mon[102832]: osdmap e21: 3 total, 3 up, 3 in 2023-09-09T23:08:35.224 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:34 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1961137089' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fedbd4c9-6852-42b5-8f3a-ba8d9bfe5d59"}]: dispatch 2023-09-09T23:08:35.224 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:34 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1961137089' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fedbd4c9-6852-42b5-8f3a-ba8d9bfe5d59"}]': finished 2023-09-09T23:08:35.224 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:34 smithi170 ceph-mon[102832]: osdmap e22: 4 total, 3 up, 4 in 2023-09-09T23:08:35.224 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:34 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-09T23:08:35.225 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:34 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:35.225 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:34 smithi170 ceph-mon[102832]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-09-09T23:08:35.225 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:34 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:35.225 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:34 smithi170 ceph-mon[102832]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-09T23:08:35.225 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:35 smithi170 sudo[114102]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-09T23:08:35.226 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:35 smithi170 sudo[114102]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-09T23:08:35.226 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:35 smithi170 sudo[114102]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-09T23:08:35.226 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:35 smithi170 sudo[114102]: pam_unix(sudo:session): session closed for user root 2023-09-09T23:08:35.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:34 smithi098 ceph-mon[103124]: osdmap e21: 3 total, 3 up, 3 in 2023-09-09T23:08:35.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:34 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1961137089' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fedbd4c9-6852-42b5-8f3a-ba8d9bfe5d59"}]: dispatch 2023-09-09T23:08:35.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:34 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1961137089' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fedbd4c9-6852-42b5-8f3a-ba8d9bfe5d59"}]': finished 2023-09-09T23:08:35.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:34 smithi098 ceph-mon[103124]: osdmap e22: 4 total, 3 up, 4 in 2023-09-09T23:08:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:34 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-09T23:08:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:34 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:34 smithi098 ceph-mon[103124]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-09-09T23:08:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:34 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:34 smithi098 ceph-mon[103124]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-09T23:08:35.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:34 smithi082 sudo[127667]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-09T23:08:35.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:34 smithi082 sudo[127667]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-09T23:08:35.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:34 smithi082 sudo[127667]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-09T23:08:35.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:34 smithi082 sudo[127667]: pam_unix(sudo:session): session closed for user root 2023-09-09T23:08:35.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:34 smithi082 ceph-mon[98694]: osdmap e21: 3 total, 3 up, 3 in 2023-09-09T23:08:35.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:34 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1961137089' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fedbd4c9-6852-42b5-8f3a-ba8d9bfe5d59"}]: dispatch 2023-09-09T23:08:35.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:34 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1961137089' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fedbd4c9-6852-42b5-8f3a-ba8d9bfe5d59"}]': finished 2023-09-09T23:08:35.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:34 smithi082 ceph-mon[98694]: osdmap e22: 4 total, 3 up, 4 in 2023-09-09T23:08:35.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:34 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-09T23:08:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:34 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:34 smithi082 ceph-mon[98694]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-09-09T23:08:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:34 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:34 smithi082 ceph-mon[98694]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-09T23:08:35.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:35 smithi170 sudo[114105]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-09T23:08:35.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:35 smithi170 sudo[114105]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-09T23:08:35.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:35 smithi170 sudo[114105]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-09T23:08:35.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:35 smithi170 sudo[114105]: pam_unix(sudo:session): session closed for user root 2023-09-09T23:08:35.952 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:35 smithi098 sudo[112976]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-09T23:08:35.952 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:35 smithi098 sudo[112976]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-09T23:08:35.952 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:35 smithi098 sudo[112976]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-09T23:08:35.953 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:35 smithi098 sudo[112976]: pam_unix(sudo:session): session closed for user root 2023-09-09T23:08:36.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:35 smithi098 sudo[112979]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-09T23:08:36.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:35 smithi098 sudo[112979]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-09T23:08:36.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:35 smithi098 sudo[112979]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-09T23:08:36.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:35 smithi098 sudo[112979]: pam_unix(sudo:session): session closed for user root 2023-09-09T23:08:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:35 smithi098 ceph-mon[103124]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-09T23:08:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:35 smithi098 ceph-mon[103124]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-09T23:08:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:35 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-09T23:08:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:35 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3332992295' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:08:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:35 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:08:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:35 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:08:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:35 smithi098 ceph-mon[103124]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-09T23:08:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:35 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-09T23:08:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:35 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:08:36.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:35 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:08:36.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:35 smithi098 ceph-mon[103124]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-09T23:08:36.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:35 smithi098 ceph-mon[103124]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-09T23:08:36.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:35 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-09T23:08:36.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:35 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:08:36.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:35 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:08:36.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:35 smithi098 ceph-mon[103124]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-09T23:08:36.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:35 smithi170 ceph-mon[102832]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-09T23:08:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:35 smithi170 ceph-mon[102832]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-09T23:08:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:35 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-09T23:08:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:35 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3332992295' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:08:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:35 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:08:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:35 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:08:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:35 smithi170 ceph-mon[102832]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-09T23:08:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:35 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-09T23:08:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:35 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:08:36.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:35 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:08:36.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:35 smithi170 ceph-mon[102832]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-09T23:08:36.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:35 smithi170 ceph-mon[102832]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-09T23:08:36.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:35 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-09T23:08:36.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:35 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:08:36.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:35 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:08:36.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:35 smithi170 ceph-mon[102832]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-09T23:08:36.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:35 smithi082 ceph-mon[98694]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-09T23:08:36.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:35 smithi082 ceph-mon[98694]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-09T23:08:36.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:35 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-09T23:08:36.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:35 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3332992295' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:08:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:35 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:08:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:35 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:08:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:35 smithi082 ceph-mon[98694]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-09T23:08:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:35 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-09T23:08:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:35 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:08:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:35 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:08:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:35 smithi082 ceph-mon[98694]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-09T23:08:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:35 smithi082 ceph-mon[98694]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-09T23:08:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:35 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-09T23:08:36.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:35 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:08:36.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:35 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:08:36.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:35 smithi082 ceph-mon[98694]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-09T23:08:38.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:37 smithi098 ceph-mon[103124]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-09T23:08:38.289 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:37 smithi082 ceph-mon[98694]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-09T23:08:38.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:37 smithi170 ceph-mon[102832]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-09T23:08:40.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:40 smithi082 ceph-mon[98694]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-09T23:08:40.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:40 smithi098 ceph-mon[103124]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-09T23:08:40.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:40 smithi170 ceph-mon[102832]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-09T23:08:41.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:41 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-09-09T23:08:41.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:41 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:41.542 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:41 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-09-09T23:08:41.543 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:41 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:41.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:41 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-09-09T23:08:41.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:41 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:42.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:42 smithi098 ceph-mon[103124]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2023-09-09T23:08:42.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:42 smithi098 ceph-mon[103124]: Deploying daemon osd.3 on smithi082 2023-09-09T23:08:42.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:42 smithi082 ceph-mon[98694]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2023-09-09T23:08:42.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:42 smithi082 ceph-mon[98694]: Deploying daemon osd.3 on smithi082 2023-09-09T23:08:42.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:42 smithi170 ceph-mon[102832]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2023-09-09T23:08:42.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:42 smithi170 ceph-mon[102832]: Deploying daemon osd.3 on smithi082 2023-09-09T23:08:44.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:44 smithi082 ceph-mon[98694]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-09-09T23:08:44.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:44 smithi098 ceph-mon[103124]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-09-09T23:08:44.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:44 smithi170 ceph-mon[102832]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-09-09T23:08:46.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:45 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:46.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:45 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:46.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:45 smithi170 ceph-mon[102832]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-09-09T23:08:46.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:45 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:08:46.195 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:45 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:46.195 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:45 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:46.195 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:45 smithi082 ceph-mon[98694]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-09-09T23:08:46.196 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:45 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:08:46.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:45 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:46.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:45 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:46.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:45 smithi098 ceph-mon[103124]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-09-09T23:08:46.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:45 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:08:47.835 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 3 on host 'smithi082' 2023-09-09T23:08:47.836 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:47.836+0000 7f021ffff700 1 -- 172.21.15.82:0/3593169832 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f023802cf10 con 0x7f02200a0630 2023-09-09T23:08:47.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:47.839+0000 7f023d1bb700 1 -- 172.21.15.82:0/3593169832 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f02200a0630 msgr2=0x7f02200a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:08:47.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:47.839+0000 7f023d1bb700 1 --2- 172.21.15.82:0/3593169832 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f02200a0630 0x7f02200a2af0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0228005950 tx=0x7f02280058e0 comp rx=0 tx=0).stop 2023-09-09T23:08:47.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:47.839+0000 7f023d1bb700 1 -- 172.21.15.82:0/3593169832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0238101180 msgr2=0x7f0238196330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:08:47.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:47.839+0000 7f023d1bb700 1 --2- 172.21.15.82:0/3593169832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0238101180 0x7f0238196330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f022c000c00 tx=0x7f022c00f690 comp rx=0 tx=0).stop 2023-09-09T23:08:47.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:47.839+0000 7f023d1bb700 1 -- 172.21.15.82:0/3593169832 shutdown_connections 2023-09-09T23:08:47.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:47.839+0000 7f023d1bb700 1 --2- 172.21.15.82:0/3593169832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02381063e0 0x7f023819d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:08:47.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:47.839+0000 7f023d1bb700 1 --2- 172.21.15.82:0/3593169832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0238101180 0x7f0238196330 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:08:47.840 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:47.839+0000 7f023d1bb700 1 --2- 172.21.15.82:0/3593169832 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f02200a0630 0x7f02200a2af0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:08:47.840 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:47.839+0000 7f023d1bb700 1 --2- 172.21.15.82:0/3593169832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0238103ab0 0x7f0238196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:08:47.840 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:47.839+0000 7f023d1bb700 1 -- 172.21.15.82:0/3593169832 >> 172.21.15.82:0/3593169832 conn(0x7f02380fab10 msgr2=0x7f0238101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:08:47.840 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:47.839+0000 7f023d1bb700 1 -- 172.21.15.82:0/3593169832 shutdown_connections 2023-09-09T23:08:47.840 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:08:47.840+0000 7f023d1bb700 1 -- 172.21.15.82:0/3593169832 wait complete. 2023-09-09T23:08:48.197 DEBUG:teuthology.orchestra.run.smithi082:osd.3> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@osd.3.service 2023-09-09T23:08:48.199 INFO:tasks.cephadm:Deploying osd.4 on smithi098 with /dev/vg_nvme/lv_3... 2023-09-09T23:08:48.199 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-09-09T23:08:48.230 INFO:journalctl@ceph.osd.3.smithi082.stdout:-- Logs begin at Sat 2023-09-09 22:51:24 UTC. -- 2023-09-09T23:08:48.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:48 smithi170 ceph-mon[102832]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-09-09T23:08:48.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:48 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:48.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:48 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:48.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:48 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:48.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:48 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:08:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:48 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:48 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:48 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:48 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:08:48.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:48 smithi082 ceph-mon[98694]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-09-09T23:08:48.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:48 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:48.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:48 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:48.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:48 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:48.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:48 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:08:48.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:48 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:48.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:48 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:48.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:48 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:48.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:48 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:08:48.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:48 smithi098 ceph-mon[103124]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-09-09T23:08:48.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:48 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:48.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:48 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:48.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:48 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:48.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:48 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:08:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:48 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:48 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:48 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:48 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:08:48.762 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-09T23:08:48.955 INFO:journalctl@ceph.osd.3.smithi082.stdout:Sep 09 23:08:48 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-3[129868]: 2023-09-09T23:08:48.876+0000 7fc7269c6640 -1 osd.3 0 log_to_monitors true 2023-09-09T23:08:49.250 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:49 smithi082 ceph-mon[98694]: from='osd.3 [v2:172.21.15.82:6810/3647629241,v1:172.21.15.82:6811/3647629241]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-09T23:08:49.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:49 smithi098 ceph-mon[103124]: from='osd.3 [v2:172.21.15.82:6810/3647629241,v1:172.21.15.82:6811/3647629241]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-09T23:08:49.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:49 smithi170 ceph-mon[102832]: from='osd.3 [v2:172.21.15.82:6810/3647629241,v1:172.21.15.82:6811/3647629241]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-09T23:08:50.249 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-09-09T23:08:50.250 INFO:teuthology.orchestra.run.smithi098.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-09-09T23:08:50.250 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10+0 records in 2023-09-09T23:08:50.250 INFO:teuthology.orchestra.run.smithi098.stderr:10+0 records out 2023-09-09T23:08:50.250 INFO:teuthology.orchestra.run.smithi098.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112381 s, 933 MB/s 2023-09-09T23:08:50.251 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping successful for: 2023-09-09T23:08:50.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:50 smithi098 ceph-mon[103124]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-09T23:08:50.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:50 smithi098 ceph-mon[103124]: from='osd.3 [v2:172.21.15.82:6810/3647629241,v1:172.21.15.82:6811/3647629241]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-09-09T23:08:50.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:50 smithi098 ceph-mon[103124]: osdmap e23: 4 total, 3 up, 4 in 2023-09-09T23:08:50.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:50 smithi098 ceph-mon[103124]: from='osd.3 [v2:172.21.15.82:6810/3647629241,v1:172.21.15.82:6811/3647629241]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-09-09T23:08:50.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:50 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-09T23:08:50.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:50 smithi170 ceph-mon[102832]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-09T23:08:50.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:50 smithi170 ceph-mon[102832]: from='osd.3 [v2:172.21.15.82:6810/3647629241,v1:172.21.15.82:6811/3647629241]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-09-09T23:08:50.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:50 smithi170 ceph-mon[102832]: osdmap e23: 4 total, 3 up, 4 in 2023-09-09T23:08:50.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:50 smithi170 ceph-mon[102832]: from='osd.3 [v2:172.21.15.82:6810/3647629241,v1:172.21.15.82:6811/3647629241]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-09-09T23:08:50.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:50 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-09T23:08:50.686 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:50 smithi082 ceph-mon[98694]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-09-09T23:08:50.686 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:50 smithi082 ceph-mon[98694]: from='osd.3 [v2:172.21.15.82:6810/3647629241,v1:172.21.15.82:6811/3647629241]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-09-09T23:08:50.686 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:50 smithi082 ceph-mon[98694]: osdmap e23: 4 total, 3 up, 4 in 2023-09-09T23:08:50.686 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:50 smithi082 ceph-mon[98694]: from='osd.3 [v2:172.21.15.82:6810/3647629241,v1:172.21.15.82:6811/3647629241]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-09-09T23:08:50.687 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:50 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-09T23:08:50.687 INFO:journalctl@ceph.osd.3.smithi082.stdout:Sep 09 23:08:50 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-3[129868]: 2023-09-09T23:08:50.217+0000 7fc71b83c700 -1 osd.3 0 waiting for initial osdmap 2023-09-09T23:08:50.687 INFO:journalctl@ceph.osd.3.smithi082.stdout:Sep 09 23:08:50 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-3[129868]: 2023-09-09T23:08:50.225+0000 7fc715e2e700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-09T23:08:50.712 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi098:vg_nvme/lv_3 2023-09-09T23:08:51.264 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-09T23:08:51.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:51 smithi098 ceph-mon[103124]: from='osd.3 [v2:172.21.15.82:6810/3647629241,v1:172.21.15.82:6811/3647629241]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-09-09T23:08:51.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:51 smithi098 ceph-mon[103124]: osdmap e24: 4 total, 3 up, 4 in 2023-09-09T23:08:51.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:51 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-09T23:08:51.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:51 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-09T23:08:51.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:51 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:51.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:51 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:51 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:08:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:51 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:08:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:51 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:51 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:51 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:08:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:51 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:51 smithi098 ceph-mon[103124]: osd.3 [v2:172.21.15.82:6810/3647629241,v1:172.21.15.82:6811/3647629241] boot 2023-09-09T23:08:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:51 smithi098 ceph-mon[103124]: osdmap e25: 4 total, 4 up, 4 in 2023-09-09T23:08:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:51 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-09T23:08:51.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:51 smithi170 ceph-mon[102832]: from='osd.3 [v2:172.21.15.82:6810/3647629241,v1:172.21.15.82:6811/3647629241]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-09-09T23:08:51.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:51 smithi170 ceph-mon[102832]: osdmap e24: 4 total, 3 up, 4 in 2023-09-09T23:08:51.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:51 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-09T23:08:51.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:51 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-09T23:08:51.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:51 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:51.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:51 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:51 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:08:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:51 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:08:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:51 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:51 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:51 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:08:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:51 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:51 smithi170 ceph-mon[102832]: osd.3 [v2:172.21.15.82:6810/3647629241,v1:172.21.15.82:6811/3647629241] boot 2023-09-09T23:08:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:51 smithi170 ceph-mon[102832]: osdmap e25: 4 total, 4 up, 4 in 2023-09-09T23:08:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:51 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-09T23:08:51.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:51 smithi082 ceph-mon[98694]: from='osd.3 [v2:172.21.15.82:6810/3647629241,v1:172.21.15.82:6811/3647629241]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-09-09T23:08:51.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:51 smithi082 ceph-mon[98694]: osdmap e24: 4 total, 3 up, 4 in 2023-09-09T23:08:51.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:51 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-09T23:08:51.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:51 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-09T23:08:51.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:51 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:51.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:51 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:51.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:51 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:08:51.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:51 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:08:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:51 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:51 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:51 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:08:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:51 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:08:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:51 smithi082 ceph-mon[98694]: osd.3 [v2:172.21.15.82:6810/3647629241,v1:172.21.15.82:6811/3647629241] boot 2023-09-09T23:08:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:51 smithi082 ceph-mon[98694]: osdmap e25: 4 total, 4 up, 4 in 2023-09-09T23:08:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:51 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-09T23:08:52.442 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.441+0000 7f6e26573700 1 -- 172.21.15.98:0/1632267782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e201006c0 msgr2=0x7f6e20100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:08:52.442 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.441+0000 7f6e26573700 1 --2- 172.21.15.98:0/1632267782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e201006c0 0x7f6e20100ae0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6e140099c0 tx=0x7f6e14009cd0 comp rx=0 tx=0).stop 2023-09-09T23:08:52.443 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.442+0000 7f6e26573700 1 -- 172.21.15.98:0/1632267782 shutdown_connections 2023-09-09T23:08:52.443 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.442+0000 7f6e26573700 1 --2- 172.21.15.98:0/1632267782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e201006c0 0x7f6e20100ae0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:08:52.443 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.442+0000 7f6e26573700 1 --2- 172.21.15.98:0/1632267782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e20102260 0x7f6e20108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:08:52.443 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.442+0000 7f6e26573700 1 --2- 172.21.15.98:0/1632267782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e201018c0 0x7f6e20101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:08:52.443 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.442+0000 7f6e26573700 1 -- 172.21.15.98:0/1632267782 >> 172.21.15.98:0/1632267782 conn(0x7f6e200fbc80 msgr2=0x7f6e200fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:08:52.444 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.442+0000 7f6e26573700 1 -- 172.21.15.98:0/1632267782 shutdown_connections 2023-09-09T23:08:52.444 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.442+0000 7f6e26573700 1 -- 172.21.15.98:0/1632267782 wait complete. 2023-09-09T23:08:52.444 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.442+0000 7f6e26573700 1 Processor -- start 2023-09-09T23:08:52.444 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.442+0000 7f6e26573700 1 -- start start 2023-09-09T23:08:52.444 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.443+0000 7f6e26573700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e201006c0 0x7f6e20196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:08:52.444 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.443+0000 7f6e26573700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e201018c0 0x7f6e20196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:08:52.445 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.443+0000 7f6e26573700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e20102260 0x7f6e2019da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:08:52.445 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.443+0000 7f6e26573700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2019dfe0 con 0x7f6e201018c0 2023-09-09T23:08:52.445 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.443+0000 7f6e26573700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2019e120 con 0x7f6e201006c0 2023-09-09T23:08:52.445 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.443+0000 7f6e26573700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2019e2b0 con 0x7f6e20102260 2023-09-09T23:08:52.445 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.443+0000 7f6e1ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e201006c0 0x7f6e20196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:08:52.446 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.443+0000 7f6e1ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e201006c0 0x7f6e20196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40704/0 (socket says 172.21.15.98:40704) 2023-09-09T23:08:52.446 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.443+0000 7f6e1ffff700 1 -- 172.21.15.98:0/1130079687 learned_addr learned my addr 172.21.15.98:0/1130079687 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-09-09T23:08:52.446 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.443+0000 7f6e1f7fe700 1 --2- 172.21.15.98:0/1130079687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e201018c0 0x7f6e20196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:08:52.446 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.443+0000 7f6e24b10700 1 --2- 172.21.15.98:0/1130079687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e20102260 0x7f6e2019da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:08:52.446 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.443+0000 7f6e1ffff700 1 -- 172.21.15.98:0/1130079687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e20102260 msgr2=0x7f6e2019da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:08:52.447 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.443+0000 7f6e1ffff700 1 --2- 172.21.15.98:0/1130079687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e20102260 0x7f6e2019da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:08:52.447 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.443+0000 7f6e1ffff700 1 -- 172.21.15.98:0/1130079687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e201018c0 msgr2=0x7f6e20196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:08:52.447 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.443+0000 7f6e1ffff700 1 --2- 172.21.15.98:0/1130079687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e201018c0 0x7f6e20196990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:08:52.447 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.443+0000 7f6e1ffff700 1 -- 172.21.15.98:0/1130079687 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e140095f0 con 0x7f6e201006c0 2023-09-09T23:08:52.447 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.443+0000 7f6e1ffff700 1 --2- 172.21.15.98:0/1130079687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e201006c0 0x7f6e20196450 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6e14009990 tx=0x7f6e14003af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:08:52.448 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.443+0000 7f6e1d7fa700 1 -- 172.21.15.98:0/1130079687 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1401d070 con 0x7f6e201006c0 2023-09-09T23:08:52.448 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.443+0000 7f6e1d7fa700 1 -- 172.21.15.98:0/1130079687 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e14004640 con 0x7f6e201006c0 2023-09-09T23:08:52.448 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.443+0000 7f6e1d7fa700 1 -- 172.21.15.98:0/1130079687 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1400f460 con 0x7f6e201006c0 2023-09-09T23:08:52.448 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.443+0000 7f6e26573700 1 -- 172.21.15.98:0/1130079687 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e2019e530 con 0x7f6e201006c0 2023-09-09T23:08:52.449 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.443+0000 7f6e26573700 1 -- 172.21.15.98:0/1130079687 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e2019ea00 con 0x7f6e201006c0 2023-09-09T23:08:52.450 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.444+0000 7f6e26573700 1 -- 172.21.15.98:0/1130079687 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e20107cc0 con 0x7f6e201006c0 2023-09-09T23:08:52.450 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.445+0000 7f6e1d7fa700 1 -- 172.21.15.98:0/1130079687 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f6e14003fa0 con 0x7f6e201006c0 2023-09-09T23:08:52.450 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.446+0000 7f6e1d7fa700 1 --2- 172.21.15.98:0/1130079687 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f6e080a0640 0x7f6e080a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:08:52.450 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.446+0000 7f6e1d7fa700 1 -- 172.21.15.98:0/1130079687 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f6e140cbf00 con 0x7f6e201006c0 2023-09-09T23:08:52.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.446+0000 7f6e1f7fe700 1 --2- 172.21.15.98:0/1130079687 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f6e080a0640 0x7f6e080a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:08:52.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.447+0000 7f6e1f7fe700 1 --2- 172.21.15.98:0/1130079687 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f6e080a0640 0x7f6e080a2b00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6e10009a10 tx=0x7f6e10009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:08:52.451 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.450+0000 7f6e1d7fa700 1 -- 172.21.15.98:0/1130079687 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6e14096cc0 con 0x7f6e201006c0 2023-09-09T23:08:52.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:52 smithi098 ceph-mon[103124]: purged_snaps scrub starts 2023-09-09T23:08:52.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:52 smithi098 ceph-mon[103124]: purged_snaps scrub ok 2023-09-09T23:08:52.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:52 smithi098 ceph-mon[103124]: Detected new or changed devices on smithi082 2023-09-09T23:08:52.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:52 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 8584M 2023-09-09T23:08:52.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:52 smithi098 ceph-mon[103124]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-09T23:08:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:52 smithi098 ceph-mon[103124]: osdmap e26: 4 total, 4 up, 4 in 2023-09-09T23:08:52.646 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:08:52.648+0000 7f6e26573700 1 -- 172.21.15.98:0/1130079687 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e2002cec0 con 0x7f6e080a0640 2023-09-09T23:08:52.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:52 smithi170 ceph-mon[102832]: purged_snaps scrub starts 2023-09-09T23:08:52.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:52 smithi170 ceph-mon[102832]: purged_snaps scrub ok 2023-09-09T23:08:52.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:52 smithi170 ceph-mon[102832]: Detected new or changed devices on smithi082 2023-09-09T23:08:52.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:52 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 8584M 2023-09-09T23:08:52.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:52 smithi170 ceph-mon[102832]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-09T23:08:52.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:52 smithi170 ceph-mon[102832]: osdmap e26: 4 total, 4 up, 4 in 2023-09-09T23:08:52.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:52 smithi082 ceph-mon[98694]: purged_snaps scrub starts 2023-09-09T23:08:52.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:52 smithi082 ceph-mon[98694]: purged_snaps scrub ok 2023-09-09T23:08:52.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:52 smithi082 ceph-mon[98694]: Detected new or changed devices on smithi082 2023-09-09T23:08:52.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:52 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 8584M 2023-09-09T23:08:52.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:52 smithi082 ceph-mon[98694]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-09T23:08:52.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:52 smithi082 ceph-mon[98694]: osdmap e26: 4 total, 4 up, 4 in 2023-09-09T23:08:53.475 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:53 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:08:53.475 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:53 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:08:53.475 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:53 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:53.476 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:53 smithi098 ceph-mon[103124]: osdmap e27: 4 total, 4 up, 4 in 2023-09-09T23:08:53.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:53 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:08:53.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:53 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:08:53.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:53 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:53.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:53 smithi170 ceph-mon[102832]: osdmap e27: 4 total, 4 up, 4 in 2023-09-09T23:08:53.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:53 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:08:53.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:53 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:08:53.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:53 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:08:53.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:53 smithi082 ceph-mon[98694]: osdmap e27: 4 total, 4 up, 4 in 2023-09-09T23:08:54.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:54 smithi098 ceph-mon[103124]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:08:54.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:54 smithi098 ceph-mon[103124]: pgmap v98: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-09-09T23:08:54.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:54 smithi170 ceph-mon[102832]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:08:54.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:54 smithi170 ceph-mon[102832]: pgmap v98: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-09-09T23:08:54.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:54 smithi082 ceph-mon[98694]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:08:54.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:54 smithi082 ceph-mon[98694]: pgmap v98: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-09-09T23:08:55.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:55 smithi098 ceph-mon[103124]: osdmap e28: 4 total, 4 up, 4 in 2023-09-09T23:08:55.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:55 smithi098 ceph-mon[103124]: from='client.? 172.21.15.98:0/2898957486' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a71f3084-6919-4161-87bf-345f3355d8c2"}]: dispatch 2023-09-09T23:08:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:55 smithi098 ceph-mon[103124]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a71f3084-6919-4161-87bf-345f3355d8c2"}]: dispatch 2023-09-09T23:08:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:55 smithi098 ceph-mon[103124]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a71f3084-6919-4161-87bf-345f3355d8c2"}]': finished 2023-09-09T23:08:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:55 smithi098 ceph-mon[103124]: osdmap e29: 5 total, 4 up, 5 in 2023-09-09T23:08:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:55 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-09T23:08:55.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:55 smithi170 ceph-mon[102832]: osdmap e28: 4 total, 4 up, 4 in 2023-09-09T23:08:55.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:55 smithi170 ceph-mon[102832]: from='client.? 172.21.15.98:0/2898957486' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a71f3084-6919-4161-87bf-345f3355d8c2"}]: dispatch 2023-09-09T23:08:55.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:55 smithi170 ceph-mon[102832]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a71f3084-6919-4161-87bf-345f3355d8c2"}]: dispatch 2023-09-09T23:08:55.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:55 smithi170 ceph-mon[102832]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a71f3084-6919-4161-87bf-345f3355d8c2"}]': finished 2023-09-09T23:08:55.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:55 smithi170 ceph-mon[102832]: osdmap e29: 5 total, 4 up, 5 in 2023-09-09T23:08:55.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:55 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-09T23:08:55.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:55 smithi082 ceph-mon[98694]: osdmap e28: 4 total, 4 up, 4 in 2023-09-09T23:08:55.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:55 smithi082 ceph-mon[98694]: from='client.? 172.21.15.98:0/2898957486' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a71f3084-6919-4161-87bf-345f3355d8c2"}]: dispatch 2023-09-09T23:08:55.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:55 smithi082 ceph-mon[98694]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a71f3084-6919-4161-87bf-345f3355d8c2"}]: dispatch 2023-09-09T23:08:55.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:55 smithi082 ceph-mon[98694]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a71f3084-6919-4161-87bf-345f3355d8c2"}]': finished 2023-09-09T23:08:55.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:55 smithi082 ceph-mon[98694]: osdmap e29: 5 total, 4 up, 5 in 2023-09-09T23:08:55.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:55 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-09T23:08:56.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:56 smithi098 ceph-mon[103124]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-09T23:08:56.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:56 smithi098 ceph-mon[103124]: from='client.? 172.21.15.98:0/775101851' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:08:56.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:56 smithi170 ceph-mon[102832]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-09T23:08:56.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:56 smithi170 ceph-mon[102832]: from='client.? 172.21.15.98:0/775101851' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:08:56.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:56 smithi082 ceph-mon[98694]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-09T23:08:56.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:56 smithi082 ceph-mon[98694]: from='client.? 172.21.15.98:0/775101851' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:08:58.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:08:58 smithi098 ceph-mon[103124]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2023-09-09T23:08:58.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:08:58 smithi170 ceph-mon[102832]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2023-09-09T23:08:58.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:08:58 smithi082 ceph-mon[98694]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2023-09-09T23:09:00.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:00 smithi098 ceph-mon[103124]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-09T23:09:00.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:00 smithi170 ceph-mon[102832]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-09T23:09:00.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:00 smithi082 ceph-mon[98694]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-09T23:09:02.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:02 smithi098 ceph-mon[103124]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2023-09-09T23:09:02.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:02 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-09-09T23:09:02.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:02 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:02.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:02 smithi170 ceph-mon[102832]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2023-09-09T23:09:02.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:02 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-09-09T23:09:02.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:02 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:02.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:02 smithi082 ceph-mon[98694]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2023-09-09T23:09:02.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:02 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-09-09T23:09:02.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:02 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:03.547 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:03 smithi098 ceph-mon[103124]: Deploying daemon osd.4 on smithi098 2023-09-09T23:09:03.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:03 smithi170 ceph-mon[102832]: Deploying daemon osd.4 on smithi098 2023-09-09T23:09:03.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:03 smithi082 ceph-mon[98694]: Deploying daemon osd.4 on smithi098 2023-09-09T23:09:04.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:04 smithi098 ceph-mon[103124]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-09-09T23:09:04.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:04 smithi170 ceph-mon[102832]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-09-09T23:09:04.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:04 smithi082 ceph-mon[98694]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-09-09T23:09:06.477 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:06 smithi098 ceph-mon[103124]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-09T23:09:06.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:06 smithi170 ceph-mon[102832]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-09T23:09:06.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:06 smithi082 ceph-mon[98694]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-09T23:09:07.659 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:07 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:07.660 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:07 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:07.660 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:07 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:09:07.660 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:07 smithi098 ceph-mon[103124]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-09-09T23:09:07.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:07 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:07.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:07 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:07.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:07 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:09:07.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:07 smithi170 ceph-mon[102832]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-09-09T23:09:07.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:07 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:07.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:07 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:07.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:07 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:09:07.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:07 smithi082 ceph-mon[98694]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-09-09T23:09:09.105 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:09.107+0000 7f6e1d7fa700 1 -- 172.21.15.98:0/1130079687 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6e2002cec0 con 0x7f6e080a0640 2023-09-09T23:09:09.106 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 4 on host 'smithi098' 2023-09-09T23:09:09.107 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:09.109+0000 7f6e26573700 1 -- 172.21.15.98:0/1130079687 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f6e080a0640 msgr2=0x7f6e080a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:09:09.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:09.109+0000 7f6e26573700 1 --2- 172.21.15.98:0/1130079687 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f6e080a0640 0x7f6e080a2b00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6e10009a10 tx=0x7f6e10009190 comp rx=0 tx=0).stop 2023-09-09T23:09:09.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:09.109+0000 7f6e26573700 1 -- 172.21.15.98:0/1130079687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e201006c0 msgr2=0x7f6e20196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:09:09.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:09.109+0000 7f6e26573700 1 --2- 172.21.15.98:0/1130079687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e201006c0 0x7f6e20196450 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6e14009990 tx=0x7f6e14003af0 comp rx=0 tx=0).stop 2023-09-09T23:09:09.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:09.109+0000 7f6e26573700 1 -- 172.21.15.98:0/1130079687 shutdown_connections 2023-09-09T23:09:09.108 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:09.109+0000 7f6e26573700 1 --2- 172.21.15.98:0/1130079687 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f6e080a0640 0x7f6e080a2b00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:09:09.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:09.109+0000 7f6e26573700 1 --2- 172.21.15.98:0/1130079687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e201006c0 0x7f6e20196450 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:09:09.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:09.109+0000 7f6e26573700 1 --2- 172.21.15.98:0/1130079687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e20102260 0x7f6e2019da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:09:09.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:09.109+0000 7f6e26573700 1 --2- 172.21.15.98:0/1130079687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e201018c0 0x7f6e20196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:09:09.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:09.109+0000 7f6e26573700 1 -- 172.21.15.98:0/1130079687 >> 172.21.15.98:0/1130079687 conn(0x7f6e200fbc80 msgr2=0x7f6e20104e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:09:09.109 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:09.109+0000 7f6e26573700 1 -- 172.21.15.98:0/1130079687 shutdown_connections 2023-09-09T23:09:09.110 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:09.109+0000 7f6e26573700 1 -- 172.21.15.98:0/1130079687 wait complete. 2023-09-09T23:09:09.552 DEBUG:teuthology.orchestra.run.smithi098:osd.4> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@osd.4.service 2023-09-09T23:09:09.555 INFO:tasks.cephadm:Deploying osd.5 on smithi170 with /dev/vg_nvme/lv_3... 2023-09-09T23:09:09.555 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-09-09T23:09:09.583 INFO:journalctl@ceph.osd.4.smithi098.stdout:-- Logs begin at Sat 2023-09-09 22:51:28 UTC. -- 2023-09-09T23:09:09.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:09 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:09.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:09 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:09.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:09 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:09.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:09 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:09:09.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:09 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:09.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:09 smithi170 ceph-mon[102832]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-09T23:09:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:09 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:09 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:09 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:09:10.121 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-09T23:09:10.159 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:09 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:10.159 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:09 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:10.160 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:09 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:10.160 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:09 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:09:10.160 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:09 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:10.160 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:09 smithi098 ceph-mon[103124]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-09T23:09:10.160 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:09 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:10.161 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:09 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:10.161 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:09 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:09:10.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:10.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:10.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:10.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:09:10.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:10.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:09 smithi082 ceph-mon[98694]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-09T23:09:10.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:10.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:09:10.673 INFO:journalctl@ceph.osd.4.smithi098.stdout:Sep 09 23:09:10 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-4[116158]: 2023-09-09T23:09:10.410+0000 7f5800d4e640 -1 osd.4 0 log_to_monitors true 2023-09-09T23:09:11.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:10 smithi098 ceph-mon[103124]: from='osd.4 [v2:172.21.15.98:6808/2507035643,v1:172.21.15.98:6809/2507035643]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-09T23:09:11.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:10 smithi098 ceph-mon[103124]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-09T23:09:11.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:10 smithi170 ceph-mon[102832]: from='osd.4 [v2:172.21.15.98:6808/2507035643,v1:172.21.15.98:6809/2507035643]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-09T23:09:11.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:10 smithi170 ceph-mon[102832]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-09T23:09:11.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:10 smithi082 ceph-mon[98694]: from='osd.4 [v2:172.21.15.98:6808/2507035643,v1:172.21.15.98:6809/2507035643]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-09T23:09:11.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:10 smithi082 ceph-mon[98694]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-09T23:09:11.478 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-09-09T23:09:11.478 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-09-09T23:09:11.478 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2023-09-09T23:09:11.479 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2023-09-09T23:09:11.479 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00883169 s, 1.2 GB/s 2023-09-09T23:09:11.479 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2023-09-09T23:09:11.864 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi170:vg_nvme/lv_3 2023-09-09T23:09:12.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:11 smithi170 ceph-mon[102832]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-09-09T23:09:12.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:11 smithi170 ceph-mon[102832]: osdmap e30: 5 total, 4 up, 5 in 2023-09-09T23:09:12.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:11 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-09T23:09:12.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:11 smithi170 ceph-mon[102832]: from='osd.4 [v2:172.21.15.98:6808/2507035643,v1:172.21.15.98:6809/2507035643]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-09T23:09:12.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:11 smithi170 ceph-mon[102832]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-09T23:09:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:11 smithi170 ceph-mon[102832]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-09T23:09:12.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:11 smithi082 ceph-mon[98694]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-09-09T23:09:12.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:11 smithi082 ceph-mon[98694]: osdmap e30: 5 total, 4 up, 5 in 2023-09-09T23:09:12.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:11 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-09T23:09:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:11 smithi082 ceph-mon[98694]: from='osd.4 [v2:172.21.15.98:6808/2507035643,v1:172.21.15.98:6809/2507035643]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-09T23:09:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:11 smithi082 ceph-mon[98694]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-09T23:09:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:11 smithi082 ceph-mon[98694]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-09T23:09:12.252 INFO:journalctl@ceph.osd.4.smithi098.stdout:Sep 09 23:09:11 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-4[116158]: 2023-09-09T23:09:11.837+0000 7f57f5bc4700 -1 osd.4 0 waiting for initial osdmap 2023-09-09T23:09:12.252 INFO:journalctl@ceph.osd.4.smithi098.stdout:Sep 09 23:09:11 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-4[116158]: 2023-09-09T23:09:11.843+0000 7f57f11b8700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-09T23:09:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:11 smithi098 ceph-mon[103124]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-09-09T23:09:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:11 smithi098 ceph-mon[103124]: osdmap e30: 5 total, 4 up, 5 in 2023-09-09T23:09:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:11 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-09T23:09:12.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:11 smithi098 ceph-mon[103124]: from='osd.4 [v2:172.21.15.98:6808/2507035643,v1:172.21.15.98:6809/2507035643]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-09T23:09:12.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:11 smithi098 ceph-mon[103124]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-09T23:09:12.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:11 smithi098 ceph-mon[103124]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-09T23:09:12.418 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-09T23:09:12.838 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:12 smithi170 ceph-mon[102832]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-09-09T23:09:12.838 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:12 smithi170 ceph-mon[102832]: osdmap e31: 5 total, 4 up, 5 in 2023-09-09T23:09:12.839 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:12 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-09T23:09:13.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:12 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-09T23:09:13.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:12 smithi170 ceph-mon[102832]: Detected new or changed devices on smithi098 2023-09-09T23:09:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:12 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:12 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:12 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:09:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:12 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:09:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:12 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 8584M 2023-09-09T23:09:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:12 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:12 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:12 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:09:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:12 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:13.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:12 smithi082 ceph-mon[98694]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-09-09T23:09:13.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:12 smithi082 ceph-mon[98694]: osdmap e31: 5 total, 4 up, 5 in 2023-09-09T23:09:13.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:12 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-09T23:09:13.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:12 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-09T23:09:13.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:12 smithi082 ceph-mon[98694]: Detected new or changed devices on smithi098 2023-09-09T23:09:13.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:12 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:13.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:12 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:13.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:12 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:09:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:12 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:09:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:12 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 8584M 2023-09-09T23:09:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:12 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:12 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:12 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:09:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:12 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:13.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:12 smithi098 ceph-mon[103124]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-09-09T23:09:13.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:12 smithi098 ceph-mon[103124]: osdmap e31: 5 total, 4 up, 5 in 2023-09-09T23:09:13.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:12 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-09T23:09:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:12 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-09T23:09:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:12 smithi098 ceph-mon[103124]: Detected new or changed devices on smithi098 2023-09-09T23:09:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:12 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:12 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:12 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:09:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:12 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:09:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:12 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 8584M 2023-09-09T23:09:13.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:12 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:13.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:12 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:13.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:12 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:09:13.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:12 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:13.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.494+0000 7f34f5c40700 1 -- 172.21.15.170:0/2885226501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34f0101130 msgr2=0x7f34f01079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:09:13.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.494+0000 7f34f5c40700 1 --2- 172.21.15.170:0/2885226501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34f0101130 0x7f34f01079e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f34e800b0a0 tx=0x7f34e800b3b0 comp rx=0 tx=0).stop 2023-09-09T23:09:13.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.494+0000 7f34f5c40700 1 -- 172.21.15.170:0/2885226501 shutdown_connections 2023-09-09T23:09:13.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.494+0000 7f34f5c40700 1 --2- 172.21.15.170:0/2885226501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34f0101130 0x7f34f01079e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:09:13.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.494+0000 7f34f5c40700 1 --2- 172.21.15.170:0/2885226501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34f00ff590 0x7f34f00ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:09:13.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.494+0000 7f34f5c40700 1 --2- 172.21.15.170:0/2885226501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34f0100790 0x7f34f0100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:09:13.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.494+0000 7f34f5c40700 1 -- 172.21.15.170:0/2885226501 >> 172.21.15.170:0/2885226501 conn(0x7f34f00fab10 msgr2=0x7f34f00fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:09:13.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.494+0000 7f34f5c40700 1 -- 172.21.15.170:0/2885226501 shutdown_connections 2023-09-09T23:09:13.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.494+0000 7f34f5c40700 1 -- 172.21.15.170:0/2885226501 wait complete. 2023-09-09T23:09:13.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.495+0000 7f34f5c40700 1 Processor -- start 2023-09-09T23:09:13.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.495+0000 7f34f5c40700 1 -- start start 2023-09-09T23:09:13.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.496+0000 7f34f5c40700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34f00ff590 0x7f34f0196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:09:13.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.496+0000 7f34f5c40700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34f0100790 0x7f34f0196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:09:13.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.496+0000 7f34f5c40700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34f0101130 0x7f34f019d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:09:13.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.496+0000 7f34f5c40700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f019dfa0 con 0x7f34f0101130 2023-09-09T23:09:13.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.496+0000 7f34f5c40700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f019e0e0 con 0x7f34f0100790 2023-09-09T23:09:13.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.496+0000 7f34f5c40700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f019e270 con 0x7f34f00ff590 2023-09-09T23:09:13.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.496+0000 7f34ef7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34f00ff590 0x7f34f0196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:09:13.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.496+0000 7f34ef7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34f00ff590 0x7f34f0196410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34450/0 (socket says 172.21.15.170:34450) 2023-09-09T23:09:13.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.496+0000 7f34ef7fe700 1 -- 172.21.15.170:0/3970505676 learned_addr learned my addr 172.21.15.170:0/3970505676 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-09-09T23:09:13.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.496+0000 7f34effff700 1 --2- 172.21.15.170:0/3970505676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34f0101130 0x7f34f019d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:09:13.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.496+0000 7f34ef7fe700 1 -- 172.21.15.170:0/3970505676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34f0100790 msgr2=0x7f34f0196950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:09:13.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.496+0000 7f34ef7fe700 1 --2- 172.21.15.170:0/3970505676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34f0100790 0x7f34f0196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:09:13.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.496+0000 7f34ef7fe700 1 -- 172.21.15.170:0/3970505676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34f0101130 msgr2=0x7f34f019d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:09:13.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.496+0000 7f34ef7fe700 1 --2- 172.21.15.170:0/3970505676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34f0101130 0x7f34f019d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:09:13.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.496+0000 7f34ef7fe700 1 -- 172.21.15.170:0/3970505676 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e8009d00 con 0x7f34f00ff590 2023-09-09T23:09:13.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.496+0000 7f34effff700 1 --2- 172.21.15.170:0/3970505676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34f0101130 0x7f34f019d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:09:13.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.496+0000 7f34ef7fe700 1 --2- 172.21.15.170:0/3970505676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34f00ff590 0x7f34f0196410 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f34d800c9a0 tx=0x7f34d800cd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:09:13.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.496+0000 7f34ecff9700 1 -- 172.21.15.170:0/3970505676 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d8004220 con 0x7f34f00ff590 2023-09-09T23:09:13.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.496+0000 7f34f5c40700 1 -- 172.21.15.170:0/3970505676 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f019e550 con 0x7f34f00ff590 2023-09-09T23:09:13.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.496+0000 7f34ecff9700 1 -- 172.21.15.170:0/3970505676 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34d8007d20 con 0x7f34f00ff590 2023-09-09T23:09:13.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.496+0000 7f34ecff9700 1 -- 172.21.15.170:0/3970505676 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d8003d00 con 0x7f34f00ff590 2023-09-09T23:09:13.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.496+0000 7f34f5c40700 1 -- 172.21.15.170:0/3970505676 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f019eaa0 con 0x7f34f00ff590 2023-09-09T23:09:13.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.497+0000 7f34f5c40700 1 -- 172.21.15.170:0/3970505676 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f0106170 con 0x7f34f00ff590 2023-09-09T23:09:13.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.498+0000 7f34ecff9700 1 -- 172.21.15.170:0/3970505676 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f34d8004380 con 0x7f34f00ff590 2023-09-09T23:09:13.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.499+0000 7f34ecff9700 1 --2- 172.21.15.170:0/3970505676 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f34dc0a0640 0x7f34dc0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:09:13.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.499+0000 7f34ecff9700 1 -- 172.21.15.170:0/3970505676 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f34d80ca740 con 0x7f34f00ff590 2023-09-09T23:09:13.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.499+0000 7f34eeffd700 1 --2- 172.21.15.170:0/3970505676 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f34dc0a0640 0x7f34dc0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:09:13.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.500+0000 7f34eeffd700 1 --2- 172.21.15.170:0/3970505676 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f34dc0a0640 0x7f34dc0a2b00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f34e0005950 tx=0x7f34e00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:09:13.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.503+0000 7f34ecff9700 1 -- 172.21.15.170:0/3970505676 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f34d80952e0 con 0x7f34f00ff590 2023-09-09T23:09:13.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:13.700+0000 7f34f5c40700 1 -- 172.21.15.170:0/3970505676 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f34f002d0c0 con 0x7f34dc0a0640 2023-09-09T23:09:14.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:13 smithi170 ceph-mon[102832]: purged_snaps scrub starts 2023-09-09T23:09:14.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:13 smithi170 ceph-mon[102832]: purged_snaps scrub ok 2023-09-09T23:09:14.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:13 smithi170 ceph-mon[102832]: osd.4 [v2:172.21.15.98:6808/2507035643,v1:172.21.15.98:6809/2507035643] boot 2023-09-09T23:09:14.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:13 smithi170 ceph-mon[102832]: osdmap e32: 5 total, 5 up, 5 in 2023-09-09T23:09:14.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:13 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-09T23:09:14.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:13 smithi170 ceph-mon[102832]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-09T23:09:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:13 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:09:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:13 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:09:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:13 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:14.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:13 smithi082 ceph-mon[98694]: purged_snaps scrub starts 2023-09-09T23:09:14.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:13 smithi082 ceph-mon[98694]: purged_snaps scrub ok 2023-09-09T23:09:14.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:13 smithi082 ceph-mon[98694]: osd.4 [v2:172.21.15.98:6808/2507035643,v1:172.21.15.98:6809/2507035643] boot 2023-09-09T23:09:14.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:13 smithi082 ceph-mon[98694]: osdmap e32: 5 total, 5 up, 5 in 2023-09-09T23:09:14.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:13 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-09T23:09:14.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:13 smithi082 ceph-mon[98694]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-09T23:09:14.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:13 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:09:14.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:13 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:09:14.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:13 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:14.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:13 smithi098 ceph-mon[103124]: purged_snaps scrub starts 2023-09-09T23:09:14.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:13 smithi098 ceph-mon[103124]: purged_snaps scrub ok 2023-09-09T23:09:14.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:13 smithi098 ceph-mon[103124]: osd.4 [v2:172.21.15.98:6808/2507035643,v1:172.21.15.98:6809/2507035643] boot 2023-09-09T23:09:14.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:13 smithi098 ceph-mon[103124]: osdmap e32: 5 total, 5 up, 5 in 2023-09-09T23:09:14.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:13 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-09T23:09:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:13 smithi098 ceph-mon[103124]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-09T23:09:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:13 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:09:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:13 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:09:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:13 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:15.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:15 smithi098 ceph-mon[103124]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:09:15.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:15 smithi098 ceph-mon[103124]: osdmap e33: 5 total, 5 up, 5 in 2023-09-09T23:09:15.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:15 smithi170 ceph-mon[102832]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:09:15.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:15 smithi170 ceph-mon[102832]: osdmap e33: 5 total, 5 up, 5 in 2023-09-09T23:09:15.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:15 smithi082 ceph-mon[98694]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:09:15.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:15 smithi082 ceph-mon[98694]: osdmap e33: 5 total, 5 up, 5 in 2023-09-09T23:09:16.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:16 smithi098 ceph-mon[103124]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:16.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:16 smithi098 ceph-mon[103124]: from='client.? 172.21.15.170:0/3904332635' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f76058e9-a90e-476e-87f0-3c6723795e26"}]: dispatch 2023-09-09T23:09:16.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:16 smithi098 ceph-mon[103124]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f76058e9-a90e-476e-87f0-3c6723795e26"}]: dispatch 2023-09-09T23:09:16.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:16 smithi098 ceph-mon[103124]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f76058e9-a90e-476e-87f0-3c6723795e26"}]': finished 2023-09-09T23:09:16.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:16 smithi098 ceph-mon[103124]: osdmap e34: 6 total, 5 up, 6 in 2023-09-09T23:09:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:16 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-09T23:09:16.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:16 smithi170 ceph-mon[102832]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:16.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:16 smithi170 ceph-mon[102832]: from='client.? 172.21.15.170:0/3904332635' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f76058e9-a90e-476e-87f0-3c6723795e26"}]: dispatch 2023-09-09T23:09:16.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:16 smithi170 ceph-mon[102832]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f76058e9-a90e-476e-87f0-3c6723795e26"}]: dispatch 2023-09-09T23:09:16.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:16 smithi170 ceph-mon[102832]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f76058e9-a90e-476e-87f0-3c6723795e26"}]': finished 2023-09-09T23:09:16.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:16 smithi170 ceph-mon[102832]: osdmap e34: 6 total, 5 up, 6 in 2023-09-09T23:09:16.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:16 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-09T23:09:16.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:16 smithi082 ceph-mon[98694]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:16.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:16 smithi082 ceph-mon[98694]: from='client.? 172.21.15.170:0/3904332635' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f76058e9-a90e-476e-87f0-3c6723795e26"}]: dispatch 2023-09-09T23:09:16.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:16 smithi082 ceph-mon[98694]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f76058e9-a90e-476e-87f0-3c6723795e26"}]: dispatch 2023-09-09T23:09:16.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:16 smithi082 ceph-mon[98694]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f76058e9-a90e-476e-87f0-3c6723795e26"}]': finished 2023-09-09T23:09:16.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:16 smithi082 ceph-mon[98694]: osdmap e34: 6 total, 5 up, 6 in 2023-09-09T23:09:16.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:16 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-09T23:09:17.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:17 smithi098 ceph-mon[103124]: from='client.? 172.21.15.170:0/454161192' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:09:17.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:17 smithi170 ceph-mon[102832]: from='client.? 172.21.15.170:0/454161192' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:09:17.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:17 smithi082 ceph-mon[98694]: from='client.? 172.21.15.170:0/454161192' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:09:18.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:18 smithi098 ceph-mon[103124]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:18.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:18 smithi170 ceph-mon[102832]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:18.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:18 smithi082 ceph-mon[98694]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:20.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:20 smithi170 ceph-mon[102832]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:20.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:20 smithi098 ceph-mon[103124]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:20.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:20 smithi082 ceph-mon[98694]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:22.300 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:22 smithi170 ceph-mon[102832]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:22.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:22 smithi098 ceph-mon[103124]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:22.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:22 smithi082 ceph-mon[98694]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:23.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:23 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-09-09T23:09:23.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:23 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:23.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:23 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-09-09T23:09:23.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:23 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:23.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:23 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-09-09T23:09:23.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:23 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:24.391 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:24 smithi170 ceph-mon[102832]: Deploying daemon osd.5 on smithi170 2023-09-09T23:09:24.391 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:24 smithi170 ceph-mon[102832]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:24.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:24 smithi098 ceph-mon[103124]: Deploying daemon osd.5 on smithi170 2023-09-09T23:09:24.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:24 smithi098 ceph-mon[103124]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:24.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:24 smithi082 ceph-mon[98694]: Deploying daemon osd.5 on smithi170 2023-09-09T23:09:24.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:24 smithi082 ceph-mon[98694]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:26.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:26 smithi098 ceph-mon[103124]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:26.523 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:26 smithi170 ceph-mon[102832]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:26.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:26 smithi082 ceph-mon[98694]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:27.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:27 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:27.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:27 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:27.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:27 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:09:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:27 smithi170 ceph-mon[102832]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:27.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:27 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:27.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:27 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:27.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:27 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:09:27.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:27 smithi082 ceph-mon[98694]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:28.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:27 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:28.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:27 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:28.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:27 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:09:28.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:27 smithi098 ceph-mon[103124]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:29.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:29.211+0000 7f34ecff9700 1 -- 172.21.15.170:0/3970505676 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f34f002d0c0 con 0x7f34dc0a0640 2023-09-09T23:09:29.210 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 5 on host 'smithi170' 2023-09-09T23:09:29.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:29.214+0000 7f34f5c40700 1 -- 172.21.15.170:0/3970505676 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f34dc0a0640 msgr2=0x7f34dc0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:09:29.213 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:29.214+0000 7f34f5c40700 1 --2- 172.21.15.170:0/3970505676 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f34dc0a0640 0x7f34dc0a2b00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f34e0005950 tx=0x7f34e00058e0 comp rx=0 tx=0).stop 2023-09-09T23:09:29.213 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:29.215+0000 7f34f5c40700 1 -- 172.21.15.170:0/3970505676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34f00ff590 msgr2=0x7f34f0196410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:09:29.213 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:29.215+0000 7f34f5c40700 1 --2- 172.21.15.170:0/3970505676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34f00ff590 0x7f34f0196410 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f34d800c9a0 tx=0x7f34d800cd60 comp rx=0 tx=0).stop 2023-09-09T23:09:29.213 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:29.215+0000 7f34f5c40700 1 -- 172.21.15.170:0/3970505676 shutdown_connections 2023-09-09T23:09:29.213 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:29.215+0000 7f34f5c40700 1 --2- 172.21.15.170:0/3970505676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34f0101130 0x7f34f019d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:09:29.213 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:29.215+0000 7f34f5c40700 1 --2- 172.21.15.170:0/3970505676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34f00ff590 0x7f34f0196410 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:09:29.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:29.215+0000 7f34f5c40700 1 --2- 172.21.15.170:0/3970505676 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f34dc0a0640 0x7f34dc0a2b00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:09:29.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:29.215+0000 7f34f5c40700 1 --2- 172.21.15.170:0/3970505676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34f0100790 0x7f34f0196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:09:29.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:29.215+0000 7f34f5c40700 1 -- 172.21.15.170:0/3970505676 >> 172.21.15.170:0/3970505676 conn(0x7f34f00fab10 msgr2=0x7f34f00fc760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:09:29.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:29.215+0000 7f34f5c40700 1 -- 172.21.15.170:0/3970505676 shutdown_connections 2023-09-09T23:09:29.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:09:29.215+0000 7f34f5c40700 1 -- 172.21.15.170:0/3970505676 wait complete. 2023-09-09T23:09:29.607 DEBUG:teuthology.orchestra.run.smithi170:osd.5> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@osd.5.service 2023-09-09T23:09:29.610 INFO:tasks.cephadm:Deploying osd.6 on smithi082 with /dev/vg_nvme/lv_2... 2023-09-09T23:09:29.610 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-09-09T23:09:29.651 INFO:journalctl@ceph.osd.5.smithi170.stdout:-- Logs begin at Sat 2023-09-09 22:51:11 UTC. -- 2023-09-09T23:09:30.189 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:09:30.406 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:30 smithi082 ceph-mon[98694]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:30.407 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:30 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:30.407 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:30 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:30.407 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:30 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:30.407 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:30 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:30.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:30 smithi170 ceph-mon[102832]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:30.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:30 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:30.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:30 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:30.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:30 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:30.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:30 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:30.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:30 smithi098 ceph-mon[103124]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:30.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:30 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:30.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:30 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:30.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:30 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:30.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:30 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:30.892 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 09 23:09:30 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-5[117318]: 2023-09-09T23:09:30.494+0000 7fe8c6f32640 -1 osd.5 0 log_to_monitors true 2023-09-09T23:09:31.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:31 smithi170 ceph-mon[102832]: from='osd.5 [v2:172.21.15.170:6808/775005168,v1:172.21.15.170:6809/775005168]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-09T23:09:31.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:31 smithi170 ceph-mon[102832]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-09T23:09:31.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:31 smithi082 ceph-mon[98694]: from='osd.5 [v2:172.21.15.170:6808/775005168,v1:172.21.15.170:6809/775005168]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-09T23:09:31.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:31 smithi082 ceph-mon[98694]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-09T23:09:31.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:31 smithi098 ceph-mon[103124]: from='osd.5 [v2:172.21.15.170:6808/775005168,v1:172.21.15.170:6809/775005168]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-09T23:09:31.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:31 smithi098 ceph-mon[103124]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-09T23:09:31.715 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-09-09T23:09:31.716 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-09-09T23:09:31.716 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2023-09-09T23:09:31.716 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2023-09-09T23:09:31.716 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126895 s, 826 MB/s 2023-09-09T23:09:31.716 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2023-09-09T23:09:32.145 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi082:vg_nvme/lv_2 2023-09-09T23:09:32.425 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 09 23:09:32 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-5[117318]: 2023-09-09T23:09:32.172+0000 7fe8bbda8700 -1 osd.5 0 waiting for initial osdmap 2023-09-09T23:09:32.426 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 09 23:09:32 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-5[117318]: 2023-09-09T23:09:32.179+0000 7fe8b639a700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-09T23:09:32.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:32 smithi170 ceph-mon[102832]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:32.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:32 smithi170 ceph-mon[102832]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-09-09T23:09:32.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:32 smithi170 ceph-mon[102832]: osdmap e35: 6 total, 5 up, 6 in 2023-09-09T23:09:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:32 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-09T23:09:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:32 smithi170 ceph-mon[102832]: from='osd.5 [v2:172.21.15.170:6808/775005168,v1:172.21.15.170:6809/775005168]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-09T23:09:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:32 smithi170 ceph-mon[102832]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-09T23:09:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:32 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:32 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:32 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:09:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:32 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:09:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:32 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:32 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:32 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:09:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:32 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:32.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:32 smithi082 ceph-mon[98694]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:32.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:32 smithi082 ceph-mon[98694]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-09-09T23:09:32.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:32 smithi082 ceph-mon[98694]: osdmap e35: 6 total, 5 up, 6 in 2023-09-09T23:09:32.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:32 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-09T23:09:32.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:32 smithi082 ceph-mon[98694]: from='osd.5 [v2:172.21.15.170:6808/775005168,v1:172.21.15.170:6809/775005168]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-09T23:09:32.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:32 smithi082 ceph-mon[98694]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-09T23:09:32.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:32 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:32.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:32 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:32 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:09:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:32 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:09:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:32 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:32 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:32 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:09:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:32 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:32.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:32 smithi098 ceph-mon[103124]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:32.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:32 smithi098 ceph-mon[103124]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-09-09T23:09:32.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:32 smithi098 ceph-mon[103124]: osdmap e35: 6 total, 5 up, 6 in 2023-09-09T23:09:32.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:32 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-09T23:09:32.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:32 smithi098 ceph-mon[103124]: from='osd.5 [v2:172.21.15.170:6808/775005168,v1:172.21.15.170:6809/775005168]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-09T23:09:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:32 smithi098 ceph-mon[103124]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-09T23:09:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:32 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:32 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:32 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:09:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:32 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:09:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:32 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:32 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:32 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:09:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:32 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:32.707 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:09:33.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:33 smithi082 ceph-mon[98694]: Detected new or changed devices on smithi170 2023-09-09T23:09:33.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:33 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 8584M 2023-09-09T23:09:33.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:33 smithi082 ceph-mon[98694]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-09-09T23:09:33.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:33 smithi082 ceph-mon[98694]: osdmap e36: 6 total, 5 up, 6 in 2023-09-09T23:09:33.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:33 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-09T23:09:33.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:33 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-09T23:09:33.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:33 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:09:33.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:33 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:33 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:09:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:33 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:33 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-09T23:09:33.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:33 smithi098 ceph-mon[103124]: Detected new or changed devices on smithi170 2023-09-09T23:09:33.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:33 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 8584M 2023-09-09T23:09:33.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:33 smithi098 ceph-mon[103124]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-09-09T23:09:33.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:33 smithi098 ceph-mon[103124]: osdmap e36: 6 total, 5 up, 6 in 2023-09-09T23:09:33.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:33 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-09T23:09:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:33 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-09T23:09:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:33 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:09:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:33 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:33 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:09:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:33 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:33 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-09T23:09:33.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:33 smithi170 ceph-mon[102832]: Detected new or changed devices on smithi170 2023-09-09T23:09:33.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:33 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 8584M 2023-09-09T23:09:33.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:33 smithi170 ceph-mon[102832]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-09-09T23:09:33.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:33 smithi170 ceph-mon[102832]: osdmap e36: 6 total, 5 up, 6 in 2023-09-09T23:09:33.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:33 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-09T23:09:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:33 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-09T23:09:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:33 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:09:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:33 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:33 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:09:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:33 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:33.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:33 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-09T23:09:33.935 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.934+0000 7f0f3d9b9700 1 -- 172.21.15.82:0/3605909986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f380ff590 msgr2=0x7f0f380ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:09:33.935 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.934+0000 7f0f3d9b9700 1 --2- 172.21.15.82:0/3605909986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f380ff590 0x7f0f380ff9b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c0099c0 tx=0x7f0f2c009cd0 comp rx=0 tx=0).stop 2023-09-09T23:09:33.935 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.934+0000 7f0f3d9b9700 1 -- 172.21.15.82:0/3605909986 shutdown_connections 2023-09-09T23:09:33.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.934+0000 7f0f3d9b9700 1 --2- 172.21.15.82:0/3605909986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f38101130 0x7f0f381079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:09:33.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.934+0000 7f0f3d9b9700 1 --2- 172.21.15.82:0/3605909986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f380ff590 0x7f0f380ff9b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:09:33.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.934+0000 7f0f3d9b9700 1 --2- 172.21.15.82:0/3605909986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f38100790 0x7f0f38100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:09:33.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.934+0000 7f0f3d9b9700 1 -- 172.21.15.82:0/3605909986 >> 172.21.15.82:0/3605909986 conn(0x7f0f380fab10 msgr2=0x7f0f380fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:09:33.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.934+0000 7f0f3d9b9700 1 -- 172.21.15.82:0/3605909986 shutdown_connections 2023-09-09T23:09:33.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.934+0000 7f0f3d9b9700 1 -- 172.21.15.82:0/3605909986 wait complete. 2023-09-09T23:09:33.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.934+0000 7f0f3d9b9700 1 Processor -- start 2023-09-09T23:09:33.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.935+0000 7f0f3d9b9700 1 -- start start 2023-09-09T23:09:33.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.935+0000 7f0f3d9b9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f380ff590 0x7f0f38195350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:09:33.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.935+0000 7f0f3d9b9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f38100790 0x7f0f38195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:09:33.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.935+0000 7f0f3d9b9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f38101130 0x7f0f3819c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:09:33.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.935+0000 7f0f3d9b9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3819cee0 con 0x7f0f380ff590 2023-09-09T23:09:33.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.935+0000 7f0f3d9b9700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3819d020 con 0x7f0f38101130 2023-09-09T23:09:33.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.935+0000 7f0f3d9b9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3819d1b0 con 0x7f0f38100790 2023-09-09T23:09:33.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.935+0000 7f0f36ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f380ff590 0x7f0f38195350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:09:33.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.935+0000 7f0f36ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f380ff590 0x7f0f38195350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53724/0 (socket says 172.21.15.82:53724) 2023-09-09T23:09:33.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.935+0000 7f0f36ffd700 1 -- 172.21.15.82:0/1049218636 learned_addr learned my addr 172.21.15.82:0/1049218636 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:09:33.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.935+0000 7f0f367fc700 1 --2- 172.21.15.82:0/1049218636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f38100790 0x7f0f38195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:09:33.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.935+0000 7f0f36ffd700 1 -- 172.21.15.82:0/1049218636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f38100790 msgr2=0x7f0f38195890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:09:33.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.935+0000 7f0f377fe700 1 --2- 172.21.15.82:0/1049218636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f38101130 0x7f0f3819c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:09:33.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.935+0000 7f0f36ffd700 1 --2- 172.21.15.82:0/1049218636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f38100790 0x7f0f38195890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:09:33.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.935+0000 7f0f36ffd700 1 -- 172.21.15.82:0/1049218636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f38101130 msgr2=0x7f0f3819c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:09:33.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.935+0000 7f0f36ffd700 1 --2- 172.21.15.82:0/1049218636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f38101130 0x7f0f3819c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:09:33.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.935+0000 7f0f36ffd700 1 -- 172.21.15.82:0/1049218636 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2c0095f0 con 0x7f0f380ff590 2023-09-09T23:09:33.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.935+0000 7f0f367fc700 1 --2- 172.21.15.82:0/1049218636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f38100790 0x7f0f38195890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:09:33.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.935+0000 7f0f377fe700 1 --2- 172.21.15.82:0/1049218636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f38101130 0x7f0f3819c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:09:33.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.935+0000 7f0f36ffd700 1 --2- 172.21.15.82:0/1049218636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f380ff590 0x7f0f38195350 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c0098e0 tx=0x7f0f2c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:09:33.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.936+0000 7f0f3c9b7700 1 -- 172.21.15.82:0/1049218636 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c01d070 con 0x7f0f380ff590 2023-09-09T23:09:33.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.936+0000 7f0f3c9b7700 1 -- 172.21.15.82:0/1049218636 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f2c004640 con 0x7f0f380ff590 2023-09-09T23:09:33.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.936+0000 7f0f3d9b9700 1 -- 172.21.15.82:0/1049218636 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f3819d430 con 0x7f0f380ff590 2023-09-09T23:09:33.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.936+0000 7f0f3c9b7700 1 -- 172.21.15.82:0/1049218636 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c00f460 con 0x7f0f380ff590 2023-09-09T23:09:33.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.936+0000 7f0f3d9b9700 1 -- 172.21.15.82:0/1049218636 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f3819d9a0 con 0x7f0f380ff590 2023-09-09T23:09:33.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.937+0000 7f0f3d9b9700 1 -- 172.21.15.82:0/1049218636 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f38106b80 con 0x7f0f380ff590 2023-09-09T23:09:33.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.938+0000 7f0f3c9b7700 1 -- 172.21.15.82:0/1049218636 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f0f2c004080 con 0x7f0f380ff590 2023-09-09T23:09:33.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.939+0000 7f0f3c9b7700 1 --2- 172.21.15.82:0/1049218636 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0f200a06c0 0x7f0f200a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:09:33.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.939+0000 7f0f367fc700 1 --2- 172.21.15.82:0/1049218636 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0f200a06c0 0x7f0f200a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:09:33.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.939+0000 7f0f3c9b7700 1 -- 172.21.15.82:0/1049218636 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f0f2c0cc580 con 0x7f0f380ff590 2023-09-09T23:09:33.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.939+0000 7f0f367fc700 1 --2- 172.21.15.82:0/1049218636 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0f200a06c0 0x7f0f200a2b80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0f28007900 tx=0x7f0f28008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:09:33.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:33.943+0000 7f0f3c9b7700 1 -- 172.21.15.82:0/1049218636 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0f2c096fe0 con 0x7f0f380ff590 2023-09-09T23:09:34.142 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:34.143+0000 7f0f3d9b9700 1 -- 172.21.15.82:0/1049218636 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f3802cf80 con 0x7f0f200a06c0 2023-09-09T23:09:34.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:34 smithi082 ceph-mon[98694]: purged_snaps scrub starts 2023-09-09T23:09:34.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:34 smithi082 ceph-mon[98694]: purged_snaps scrub ok 2023-09-09T23:09:34.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:34 smithi082 ceph-mon[98694]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:34.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:34 smithi082 ceph-mon[98694]: osd.5 [v2:172.21.15.170:6808/775005168,v1:172.21.15.170:6809/775005168] boot 2023-09-09T23:09:34.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:34 smithi082 ceph-mon[98694]: osdmap e37: 6 total, 6 up, 6 in 2023-09-09T23:09:34.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:34 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-09T23:09:34.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:34 smithi082 ceph-mon[98694]: osdmap e38: 6 total, 6 up, 6 in 2023-09-09T23:09:34.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:34 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:09:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:34 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:09:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:34 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:34.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:34 smithi098 ceph-mon[103124]: purged_snaps scrub starts 2023-09-09T23:09:34.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:34 smithi098 ceph-mon[103124]: purged_snaps scrub ok 2023-09-09T23:09:34.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:34 smithi098 ceph-mon[103124]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:34.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:34 smithi098 ceph-mon[103124]: osd.5 [v2:172.21.15.170:6808/775005168,v1:172.21.15.170:6809/775005168] boot 2023-09-09T23:09:34.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:34 smithi098 ceph-mon[103124]: osdmap e37: 6 total, 6 up, 6 in 2023-09-09T23:09:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:34 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-09T23:09:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:34 smithi098 ceph-mon[103124]: osdmap e38: 6 total, 6 up, 6 in 2023-09-09T23:09:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:34 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:09:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:34 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:09:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:34 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:34.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:34 smithi170 ceph-mon[102832]: purged_snaps scrub starts 2023-09-09T23:09:34.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:34 smithi170 ceph-mon[102832]: purged_snaps scrub ok 2023-09-09T23:09:34.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:34 smithi170 ceph-mon[102832]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-09T23:09:34.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:34 smithi170 ceph-mon[102832]: osd.5 [v2:172.21.15.170:6808/775005168,v1:172.21.15.170:6809/775005168] boot 2023-09-09T23:09:34.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:34 smithi170 ceph-mon[102832]: osdmap e37: 6 total, 6 up, 6 in 2023-09-09T23:09:34.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:34 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-09T23:09:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:34 smithi170 ceph-mon[102832]: osdmap e38: 6 total, 6 up, 6 in 2023-09-09T23:09:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:34 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:09:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:34 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:09:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:34 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:35.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:35 smithi098 ceph-mon[103124]: from='client.14466 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:09:35.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:35 smithi098 ceph-mon[103124]: osdmap e39: 6 total, 6 up, 6 in 2023-09-09T23:09:35.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:35 smithi170 ceph-mon[102832]: from='client.14466 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:09:35.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:35 smithi170 ceph-mon[102832]: osdmap e39: 6 total, 6 up, 6 in 2023-09-09T23:09:35.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:35 smithi082 ceph-mon[98694]: from='client.14466 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:09:35.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:35 smithi082 ceph-mon[98694]: osdmap e39: 6 total, 6 up, 6 in 2023-09-09T23:09:36.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:36 smithi098 ceph-mon[103124]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-09T23:09:36.528 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:36 smithi098 ceph-mon[103124]: osdmap e40: 6 total, 6 up, 6 in 2023-09-09T23:09:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:36 smithi170 ceph-mon[102832]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-09T23:09:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:36 smithi170 ceph-mon[102832]: osdmap e40: 6 total, 6 up, 6 in 2023-09-09T23:09:36.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:36 smithi082 ceph-mon[98694]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-09T23:09:36.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:36 smithi082 ceph-mon[98694]: osdmap e40: 6 total, 6 up, 6 in 2023-09-09T23:09:37.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:37 smithi098 ceph-mon[103124]: osdmap e41: 6 total, 6 up, 6 in 2023-09-09T23:09:37.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:37 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1733576729' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25940be0-c1bd-4c72-95bf-fd8367aa2f08"}]: dispatch 2023-09-09T23:09:37.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:37 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1733576729' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "25940be0-c1bd-4c72-95bf-fd8367aa2f08"}]': finished 2023-09-09T23:09:37.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:37 smithi098 ceph-mon[103124]: osdmap e42: 7 total, 6 up, 7 in 2023-09-09T23:09:37.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:37 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-09T23:09:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:37 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2549702888' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:09:37.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:37 smithi170 ceph-mon[102832]: osdmap e41: 6 total, 6 up, 6 in 2023-09-09T23:09:37.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:37 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1733576729' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25940be0-c1bd-4c72-95bf-fd8367aa2f08"}]: dispatch 2023-09-09T23:09:37.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:37 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1733576729' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "25940be0-c1bd-4c72-95bf-fd8367aa2f08"}]': finished 2023-09-09T23:09:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:37 smithi170 ceph-mon[102832]: osdmap e42: 7 total, 6 up, 7 in 2023-09-09T23:09:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:37 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-09T23:09:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:37 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2549702888' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:09:37.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:37 smithi082 ceph-mon[98694]: osdmap e41: 6 total, 6 up, 6 in 2023-09-09T23:09:37.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:37 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1733576729' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25940be0-c1bd-4c72-95bf-fd8367aa2f08"}]: dispatch 2023-09-09T23:09:37.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:37 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1733576729' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "25940be0-c1bd-4c72-95bf-fd8367aa2f08"}]': finished 2023-09-09T23:09:37.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:37 smithi082 ceph-mon[98694]: osdmap e42: 7 total, 6 up, 7 in 2023-09-09T23:09:37.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:37 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-09T23:09:37.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:37 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2549702888' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:09:38.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:38 smithi098 ceph-mon[103124]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 154 KiB/s, 0 objects/s recovering 2023-09-09T23:09:38.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:38 smithi170 ceph-mon[102832]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 154 KiB/s, 0 objects/s recovering 2023-09-09T23:09:38.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:38 smithi082 ceph-mon[98694]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 154 KiB/s, 0 objects/s recovering 2023-09-09T23:09:40.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:40 smithi098 ceph-mon[103124]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 122 KiB/s, 0 objects/s recovering 2023-09-09T23:09:40.599 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:40 smithi082 ceph-mon[98694]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 122 KiB/s, 0 objects/s recovering 2023-09-09T23:09:40.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:40 smithi170 ceph-mon[102832]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 122 KiB/s, 0 objects/s recovering 2023-09-09T23:09:42.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:42 smithi082 ceph-mon[98694]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-09T23:09:42.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:42 smithi098 ceph-mon[103124]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-09T23:09:42.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:42 smithi170 ceph-mon[102832]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-09T23:09:44.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:44 smithi082 ceph-mon[98694]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2023-09-09T23:09:44.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:44 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-09-09T23:09:44.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:44 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:44.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:44 smithi082 ceph-mon[98694]: Deploying daemon osd.6 on smithi082 2023-09-09T23:09:44.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:44 smithi098 ceph-mon[103124]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2023-09-09T23:09:44.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:44 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-09-09T23:09:44.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:44 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:44.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:44 smithi098 ceph-mon[103124]: Deploying daemon osd.6 on smithi082 2023-09-09T23:09:44.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:44 smithi170 ceph-mon[102832]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2023-09-09T23:09:44.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:44 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-09-09T23:09:44.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:44 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:44.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:44 smithi170 ceph-mon[102832]: Deploying daemon osd.6 on smithi082 2023-09-09T23:09:46.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:46 smithi082 ceph-mon[98694]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-09T23:09:46.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:46 smithi098 ceph-mon[103124]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-09T23:09:46.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:46 smithi170 ceph-mon[102832]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-09T23:09:48.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:48 smithi098 ceph-mon[103124]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-09T23:09:48.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:48 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:48 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:09:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:48 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:48.598 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:48 smithi082 ceph-mon[98694]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-09T23:09:48.598 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:48 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:48.598 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:48 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:09:48.598 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:48 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:48.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:48 smithi170 ceph-mon[102832]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-09T23:09:48.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:48 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:48.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:48 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:09:48.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:48 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:49.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:49.947+0000 7f0f3c9b7700 1 -- 172.21.15.82:0/1049218636 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0f3802cf80 con 0x7f0f200a06c0 2023-09-09T23:09:49.947 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 6 on host 'smithi082' 2023-09-09T23:09:49.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:49.950+0000 7f0f3d9b9700 1 -- 172.21.15.82:0/1049218636 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0f200a06c0 msgr2=0x7f0f200a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:09:49.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:49.950+0000 7f0f3d9b9700 1 --2- 172.21.15.82:0/1049218636 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0f200a06c0 0x7f0f200a2b80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0f28007900 tx=0x7f0f28008040 comp rx=0 tx=0).stop 2023-09-09T23:09:49.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:49.950+0000 7f0f3d9b9700 1 -- 172.21.15.82:0/1049218636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f380ff590 msgr2=0x7f0f38195350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:09:49.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:49.950+0000 7f0f3d9b9700 1 --2- 172.21.15.82:0/1049218636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f380ff590 0x7f0f38195350 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c0098e0 tx=0x7f0f2c003ba0 comp rx=0 tx=0).stop 2023-09-09T23:09:49.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:49.950+0000 7f0f3d9b9700 1 -- 172.21.15.82:0/1049218636 shutdown_connections 2023-09-09T23:09:49.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:49.950+0000 7f0f3d9b9700 1 --2- 172.21.15.82:0/1049218636 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0f200a06c0 0x7f0f200a2b80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:09:49.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:49.950+0000 7f0f3d9b9700 1 --2- 172.21.15.82:0/1049218636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f38101130 0x7f0f3819c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:09:49.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:49.950+0000 7f0f3d9b9700 1 --2- 172.21.15.82:0/1049218636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f380ff590 0x7f0f38195350 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:09:49.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:49.950+0000 7f0f3d9b9700 1 --2- 172.21.15.82:0/1049218636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f38100790 0x7f0f38195890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:09:49.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:49.950+0000 7f0f3d9b9700 1 -- 172.21.15.82:0/1049218636 >> 172.21.15.82:0/1049218636 conn(0x7f0f380fab10 msgr2=0x7f0f38103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:09:49.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:49.950+0000 7f0f3d9b9700 1 -- 172.21.15.82:0/1049218636 shutdown_connections 2023-09-09T23:09:49.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:09:49.950+0000 7f0f3d9b9700 1 -- 172.21.15.82:0/1049218636 wait complete. 2023-09-09T23:09:50.288 DEBUG:teuthology.orchestra.run.smithi082:osd.6> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@osd.6.service 2023-09-09T23:09:50.291 INFO:tasks.cephadm:Deploying osd.7 on smithi098 with /dev/vg_nvme/lv_2... 2023-09-09T23:09:50.291 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-09-09T23:09:50.321 INFO:journalctl@ceph.osd.6.smithi082.stdout:-- Logs begin at Sat 2023-09-09 22:51:24 UTC. -- 2023-09-09T23:09:50.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:50 smithi098 ceph-mon[103124]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-09T23:09:50.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:50 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:50.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:50 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:50.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:50 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:50.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:50 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:09:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:50 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:50 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:50 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:50 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:09:50.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:50 smithi170 ceph-mon[102832]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-09T23:09:50.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:50 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:50.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:50 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:50.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:50 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:50.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:50 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:09:50.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:50 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:50 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:50 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:50 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:09:50.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:50 smithi082 ceph-mon[98694]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-09T23:09:50.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:50 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:50.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:50 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:50.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:50 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:50.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:50 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:09:50.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:50 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:50.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:50 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:50.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:50 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:50.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:50 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:09:50.865 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-09T23:09:51.561 INFO:journalctl@ceph.osd.6.smithi082.stdout:Sep 09 23:09:51 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-6[137809]: 2023-09-09T23:09:51.367+0000 7f3fab638640 -1 osd.6 0 log_to_monitors true 2023-09-09T23:09:52.382 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-09-09T23:09:52.382 INFO:teuthology.orchestra.run.smithi098.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-09-09T23:09:52.383 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10+0 records in 2023-09-09T23:09:52.383 INFO:teuthology.orchestra.run.smithi098.stderr:10+0 records out 2023-09-09T23:09:52.383 INFO:teuthology.orchestra.run.smithi098.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.015176 s, 691 MB/s 2023-09-09T23:09:52.383 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping successful for: 2023-09-09T23:09:52.493 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:52 smithi082 ceph-mon[98694]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-09T23:09:52.493 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:52 smithi082 ceph-mon[98694]: from='osd.6 [v2:172.21.15.82:6818/3008405982,v1:172.21.15.82:6819/3008405982]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-09T23:09:52.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:52 smithi098 ceph-mon[103124]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-09T23:09:52.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:52 smithi098 ceph-mon[103124]: from='osd.6 [v2:172.21.15.82:6818/3008405982,v1:172.21.15.82:6819/3008405982]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-09T23:09:52.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:52 smithi170 ceph-mon[102832]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-09T23:09:52.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:52 smithi170 ceph-mon[102832]: from='osd.6 [v2:172.21.15.82:6818/3008405982,v1:172.21.15.82:6819/3008405982]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-09T23:09:52.820 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi098:vg_nvme/lv_2 2023-09-09T23:09:53.378 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-09T23:09:53.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:53 smithi098 ceph-mon[103124]: from='osd.6 [v2:172.21.15.82:6818/3008405982,v1:172.21.15.82:6819/3008405982]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-09-09T23:09:53.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:53 smithi098 ceph-mon[103124]: osdmap e43: 7 total, 6 up, 7 in 2023-09-09T23:09:53.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:53 smithi098 ceph-mon[103124]: from='osd.6 [v2:172.21.15.82:6818/3008405982,v1:172.21.15.82:6819/3008405982]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-09-09T23:09:53.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:53 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-09T23:09:53.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:53 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:53 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:53 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:09:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:53 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:09:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:53 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:09:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:53 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:53 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:53 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:09:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:53 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:53 smithi098 ceph-mon[103124]: from='osd.6 [v2:172.21.15.82:6818/3008405982,v1:172.21.15.82:6819/3008405982]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-09-09T23:09:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:53 smithi098 ceph-mon[103124]: osdmap e44: 7 total, 6 up, 7 in 2023-09-09T23:09:53.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:53 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-09T23:09:53.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:53 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-09T23:09:53.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:53 smithi170 ceph-mon[102832]: from='osd.6 [v2:172.21.15.82:6818/3008405982,v1:172.21.15.82:6819/3008405982]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-09-09T23:09:53.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:53 smithi170 ceph-mon[102832]: osdmap e43: 7 total, 6 up, 7 in 2023-09-09T23:09:53.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:53 smithi170 ceph-mon[102832]: from='osd.6 [v2:172.21.15.82:6818/3008405982,v1:172.21.15.82:6819/3008405982]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-09-09T23:09:53.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:53 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-09T23:09:53.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:53 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:53 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:53 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:09:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:53 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:09:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:53 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:09:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:53 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:53 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:53 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:09:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:53 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:53 smithi170 ceph-mon[102832]: from='osd.6 [v2:172.21.15.82:6818/3008405982,v1:172.21.15.82:6819/3008405982]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-09-09T23:09:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:53 smithi170 ceph-mon[102832]: osdmap e44: 7 total, 6 up, 7 in 2023-09-09T23:09:53.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:53 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-09T23:09:53.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:53 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-09T23:09:53.705 INFO:journalctl@ceph.osd.6.smithi082.stdout:Sep 09 23:09:53 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-6[137809]: 2023-09-09T23:09:53.235+0000 7f3fa04ae700 -1 osd.6 0 waiting for initial osdmap 2023-09-09T23:09:53.705 INFO:journalctl@ceph.osd.6.smithi082.stdout:Sep 09 23:09:53 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-6[137809]: 2023-09-09T23:09:53.245+0000 7f3f9baa2700 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-09T23:09:53.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:53 smithi082 ceph-mon[98694]: from='osd.6 [v2:172.21.15.82:6818/3008405982,v1:172.21.15.82:6819/3008405982]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-09-09T23:09:53.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:53 smithi082 ceph-mon[98694]: osdmap e43: 7 total, 6 up, 7 in 2023-09-09T23:09:53.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:53 smithi082 ceph-mon[98694]: from='osd.6 [v2:172.21.15.82:6818/3008405982,v1:172.21.15.82:6819/3008405982]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-09-09T23:09:53.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:53 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-09T23:09:53.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:53 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:53 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:53 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:09:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:53 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:09:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:53 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:09:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:53 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:53 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:53 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:09:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:53 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:09:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:53 smithi082 ceph-mon[98694]: from='osd.6 [v2:172.21.15.82:6818/3008405982,v1:172.21.15.82:6819/3008405982]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-09-09T23:09:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:53 smithi082 ceph-mon[98694]: osdmap e44: 7 total, 6 up, 7 in 2023-09-09T23:09:53.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:53 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-09T23:09:53.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:53 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-09T23:09:54.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:54 smithi098 ceph-mon[103124]: Detected new or changed devices on smithi082 2023-09-09T23:09:54.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:54 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 5722M 2023-09-09T23:09:54.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:54 smithi098 ceph-mon[103124]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-09T23:09:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:54 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-09T23:09:54.542 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.544+0000 7f33f640e700 1 -- 172.21.15.98:0/3096993075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33f01011a0 msgr2=0x7f33f0103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:09:54.543 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.544+0000 7f33f640e700 1 --2- 172.21.15.98:0/3096993075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33f01011a0 0x7f33f0103590 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f33e0009990 tx=0x7f33e0009ca0 comp rx=0 tx=0).stop 2023-09-09T23:09:54.543 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.545+0000 7f33f640e700 1 -- 172.21.15.98:0/3096993075 shutdown_connections 2023-09-09T23:09:54.543 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.545+0000 7f33f640e700 1 --2- 172.21.15.98:0/3096993075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33f01011a0 0x7f33f0103590 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:09:54.543 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.545+0000 7f33f640e700 1 --2- 172.21.15.98:0/3096993075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33f0106400 0x7f33f01087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:09:54.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.545+0000 7f33f640e700 1 --2- 172.21.15.98:0/3096993075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33f0103ad0 0x7f33f0105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:09:54.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.545+0000 7f33f640e700 1 -- 172.21.15.98:0/3096993075 >> 172.21.15.98:0/3096993075 conn(0x7f33f00fab10 msgr2=0x7f33f00fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:09:54.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.545+0000 7f33f640e700 1 -- 172.21.15.98:0/3096993075 shutdown_connections 2023-09-09T23:09:54.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.545+0000 7f33f640e700 1 -- 172.21.15.98:0/3096993075 wait complete. 2023-09-09T23:09:54.544 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.545+0000 7f33f640e700 1 Processor -- start 2023-09-09T23:09:54.545 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.545+0000 7f33f640e700 1 -- start start 2023-09-09T23:09:54.547 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.545+0000 7f33f640e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33f01011a0 0x7f33f01963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:09:54.547 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.545+0000 7f33f640e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33f0103ad0 0x7f33f0196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:09:54.547 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.546+0000 7f33f640e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33f0106400 0x7f33f019d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:09:54.547 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.546+0000 7f33f640e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f019df70 con 0x7f33f0103ad0 2023-09-09T23:09:54.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.546+0000 7f33f640e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f019e0b0 con 0x7f33f01011a0 2023-09-09T23:09:54.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.546+0000 7f33f640e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f019e240 con 0x7f33f0106400 2023-09-09T23:09:54.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.546+0000 7f33effff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33f01011a0 0x7f33f01963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:09:54.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.546+0000 7f33effff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33f01011a0 0x7f33f01963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40810/0 (socket says 172.21.15.98:40810) 2023-09-09T23:09:54.548 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.546+0000 7f33effff700 1 -- 172.21.15.98:0/2585350934 learned_addr learned my addr 172.21.15.98:0/2585350934 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-09-09T23:09:54.549 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.546+0000 7f33ef7fe700 1 --2- 172.21.15.98:0/2585350934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33f0103ad0 0x7f33f0196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:09:54.549 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.546+0000 7f33effff700 1 -- 172.21.15.98:0/2585350934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33f0106400 msgr2=0x7f33f019d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:09:54.549 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.546+0000 7f33effff700 1 --2- 172.21.15.98:0/2585350934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33f0106400 0x7f33f019d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:09:54.549 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.546+0000 7f33effff700 1 -- 172.21.15.98:0/2585350934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33f0103ad0 msgr2=0x7f33f0196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:09:54.549 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.546+0000 7f33effff700 1 --2- 172.21.15.98:0/2585350934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33f0103ad0 0x7f33f0196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:09:54.549 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.546+0000 7f33effff700 1 -- 172.21.15.98:0/2585350934 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e00095f0 con 0x7f33f01011a0 2023-09-09T23:09:54.550 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.546+0000 7f33ef7fe700 1 --2- 172.21.15.98:0/2585350934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33f0103ad0 0x7f33f0196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:09:54.550 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.546+0000 7f33effff700 1 --2- 172.21.15.98:0/2585350934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33f01011a0 0x7f33f01963e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f33e0000c00 tx=0x7f33e000f710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:09:54.550 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.546+0000 7f33ed7fa700 1 -- 172.21.15.98:0/2585350934 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e001d070 con 0x7f33f01011a0 2023-09-09T23:09:54.550 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.546+0000 7f33ed7fa700 1 -- 172.21.15.98:0/2585350934 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33e000fc00 con 0x7f33f01011a0 2023-09-09T23:09:54.550 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.546+0000 7f33ed7fa700 1 -- 172.21.15.98:0/2585350934 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e0017650 con 0x7f33f01011a0 2023-09-09T23:09:54.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.546+0000 7f33f640e700 1 -- 172.21.15.98:0/2585350934 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f019e4c0 con 0x7f33f01011a0 2023-09-09T23:09:54.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.546+0000 7f33f640e700 1 -- 172.21.15.98:0/2585350934 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f019e960 con 0x7f33f01011a0 2023-09-09T23:09:54.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.548+0000 7f33f640e700 1 -- 172.21.15.98:0/2585350934 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33f00fc690 con 0x7f33f01011a0 2023-09-09T23:09:54.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.548+0000 7f33ed7fa700 1 -- 172.21.15.98:0/2585350934 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f33e0022470 con 0x7f33f01011a0 2023-09-09T23:09:54.551 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.549+0000 7f33ed7fa700 1 --2- 172.21.15.98:0/2585350934 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f33d80a06c0 0x7f33d80a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:09:54.552 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.549+0000 7f33ed7fa700 1 -- 172.21.15.98:0/2585350934 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f33e00cc110 con 0x7f33f01011a0 2023-09-09T23:09:54.552 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.549+0000 7f33ef7fe700 1 --2- 172.21.15.98:0/2585350934 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f33d80a06c0 0x7f33d80a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:09:54.552 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.550+0000 7f33ef7fe700 1 --2- 172.21.15.98:0/2585350934 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f33d80a06c0 0x7f33d80a2b80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f33e4005fd0 tx=0x7f33e4005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:09:54.553 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.553+0000 7f33ed7fa700 1 -- 172.21.15.98:0/2585350934 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f33e00969e0 con 0x7f33f01011a0 2023-09-09T23:09:54.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:54 smithi170 ceph-mon[102832]: Detected new or changed devices on smithi082 2023-09-09T23:09:54.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:54 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 5722M 2023-09-09T23:09:54.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:54 smithi170 ceph-mon[102832]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-09T23:09:54.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:54 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-09T23:09:54.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:54 smithi082 ceph-mon[98694]: Detected new or changed devices on smithi082 2023-09-09T23:09:54.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:54 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 5722M 2023-09-09T23:09:54.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:54 smithi082 ceph-mon[98694]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-09T23:09:54.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:54 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-09T23:09:54.747 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:09:54.750+0000 7f33f640e700 1 -- 172.21.15.98:0/2585350934 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f33f002cf10 con 0x7f33d80a06c0 2023-09-09T23:09:55.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:55 smithi098 ceph-mon[103124]: purged_snaps scrub starts 2023-09-09T23:09:55.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:55 smithi098 ceph-mon[103124]: purged_snaps scrub ok 2023-09-09T23:09:55.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:55 smithi098 ceph-mon[103124]: osd.6 [v2:172.21.15.82:6818/3008405982,v1:172.21.15.82:6819/3008405982] boot 2023-09-09T23:09:55.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:55 smithi098 ceph-mon[103124]: osdmap e45: 7 total, 7 up, 7 in 2023-09-09T23:09:55.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:55 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-09T23:09:55.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:55 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:09:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:55 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:09:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:55 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:55.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:55 smithi170 ceph-mon[102832]: purged_snaps scrub starts 2023-09-09T23:09:55.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:55 smithi170 ceph-mon[102832]: purged_snaps scrub ok 2023-09-09T23:09:55.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:55 smithi170 ceph-mon[102832]: osd.6 [v2:172.21.15.82:6818/3008405982,v1:172.21.15.82:6819/3008405982] boot 2023-09-09T23:09:55.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:55 smithi170 ceph-mon[102832]: osdmap e45: 7 total, 7 up, 7 in 2023-09-09T23:09:55.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:55 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-09T23:09:55.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:55 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:09:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:55 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:09:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:55 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:55.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:55 smithi082 ceph-mon[98694]: purged_snaps scrub starts 2023-09-09T23:09:55.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:55 smithi082 ceph-mon[98694]: purged_snaps scrub ok 2023-09-09T23:09:55.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:55 smithi082 ceph-mon[98694]: osd.6 [v2:172.21.15.82:6818/3008405982,v1:172.21.15.82:6819/3008405982] boot 2023-09-09T23:09:55.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:55 smithi082 ceph-mon[98694]: osdmap e45: 7 total, 7 up, 7 in 2023-09-09T23:09:55.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:55 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-09T23:09:55.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:55 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:09:55.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:55 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:09:55.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:55 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:09:56.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:56 smithi098 ceph-mon[103124]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:09:56.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:56 smithi098 ceph-mon[103124]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-09T23:09:56.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:56 smithi098 ceph-mon[103124]: osdmap e46: 7 total, 7 up, 7 in 2023-09-09T23:09:56.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:56 smithi170 ceph-mon[102832]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:09:56.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:56 smithi170 ceph-mon[102832]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-09T23:09:56.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:56 smithi170 ceph-mon[102832]: osdmap e46: 7 total, 7 up, 7 in 2023-09-09T23:09:56.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:56 smithi082 ceph-mon[98694]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:09:56.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:56 smithi082 ceph-mon[98694]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-09T23:09:56.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:56 smithi082 ceph-mon[98694]: osdmap e46: 7 total, 7 up, 7 in 2023-09-09T23:09:57.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:57 smithi170 ceph-mon[102832]: from='client.? 172.21.15.98:0/181603829' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f651bb8c-0b25-42d5-b028-58563f4c27f8"}]: dispatch 2023-09-09T23:09:57.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:57 smithi170 ceph-mon[102832]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f651bb8c-0b25-42d5-b028-58563f4c27f8"}]: dispatch 2023-09-09T23:09:57.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:57 smithi170 ceph-mon[102832]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f651bb8c-0b25-42d5-b028-58563f4c27f8"}]': finished 2023-09-09T23:09:57.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:57 smithi170 ceph-mon[102832]: osdmap e47: 8 total, 7 up, 8 in 2023-09-09T23:09:57.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:57 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-09T23:09:57.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:57 smithi082 ceph-mon[98694]: from='client.? 172.21.15.98:0/181603829' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f651bb8c-0b25-42d5-b028-58563f4c27f8"}]: dispatch 2023-09-09T23:09:57.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:57 smithi082 ceph-mon[98694]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f651bb8c-0b25-42d5-b028-58563f4c27f8"}]: dispatch 2023-09-09T23:09:57.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:57 smithi082 ceph-mon[98694]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f651bb8c-0b25-42d5-b028-58563f4c27f8"}]': finished 2023-09-09T23:09:57.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:57 smithi082 ceph-mon[98694]: osdmap e47: 8 total, 7 up, 8 in 2023-09-09T23:09:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:57 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-09T23:09:57.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:57 smithi098 ceph-mon[103124]: from='client.? 172.21.15.98:0/181603829' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f651bb8c-0b25-42d5-b028-58563f4c27f8"}]: dispatch 2023-09-09T23:09:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:57 smithi098 ceph-mon[103124]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f651bb8c-0b25-42d5-b028-58563f4c27f8"}]: dispatch 2023-09-09T23:09:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:57 smithi098 ceph-mon[103124]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f651bb8c-0b25-42d5-b028-58563f4c27f8"}]': finished 2023-09-09T23:09:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:57 smithi098 ceph-mon[103124]: osdmap e47: 8 total, 7 up, 8 in 2023-09-09T23:09:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:57 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-09T23:09:58.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:58 smithi170 ceph-mon[102832]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:09:58.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:09:58 smithi170 ceph-mon[102832]: from='client.? 172.21.15.98:0/2270833285' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:09:58.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:58 smithi082 ceph-mon[98694]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:09:58.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:09:58 smithi082 ceph-mon[98694]: from='client.? 172.21.15.98:0/2270833285' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:09:58.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:58 smithi098 ceph-mon[103124]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:09:58.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:09:58 smithi098 ceph-mon[103124]: from='client.? 172.21.15.98:0/2270833285' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:10:00.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:00 smithi098 ceph-mon[103124]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:10:00.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:00 smithi098 ceph-mon[103124]: overall HEALTH_OK 2023-09-09T23:10:00.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:00 smithi170 ceph-mon[102832]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:10:00.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:00 smithi170 ceph-mon[102832]: overall HEALTH_OK 2023-09-09T23:10:00.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:00 smithi082 ceph-mon[98694]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:10:00.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:00 smithi082 ceph-mon[98694]: overall HEALTH_OK 2023-09-09T23:10:02.587 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:02 smithi098 ceph-mon[103124]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:10:02.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:02 smithi170 ceph-mon[102832]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:10:02.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:02 smithi082 ceph-mon[98694]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:10:04.666 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:04 smithi098 ceph-mon[103124]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:10:04.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:04 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-09-09T23:10:04.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:04 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:04.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:04 smithi170 ceph-mon[102832]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:10:04.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:04 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-09-09T23:10:04.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:04 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:04.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:04 smithi082 ceph-mon[98694]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:10:04.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:04 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-09-09T23:10:04.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:04 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:05.565 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:05 smithi098 ceph-mon[103124]: Deploying daemon osd.7 on smithi098 2023-09-09T23:10:05.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:05 smithi170 ceph-mon[102832]: Deploying daemon osd.7 on smithi098 2023-09-09T23:10:05.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:05 smithi082 ceph-mon[98694]: Deploying daemon osd.7 on smithi098 2023-09-09T23:10:06.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:06 smithi098 ceph-mon[103124]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:10:06.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:06 smithi170 ceph-mon[102832]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:10:06.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:06 smithi082 ceph-mon[98694]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:10:08.479 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:08 smithi098 ceph-mon[103124]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:10:08.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:08 smithi170 ceph-mon[102832]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:10:08.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:08 smithi082 ceph-mon[98694]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:10:09.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:09 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:09.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:09 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:09.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:09 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:10:09.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:09 smithi098 ceph-mon[103124]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:10:09.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:09 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:09.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:09 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:09.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:09 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:10:09.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:09 smithi170 ceph-mon[102832]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:10:09.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:09.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:09.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:10:09.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:09 smithi082 ceph-mon[98694]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:10:11.058 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 7 on host 'smithi098' 2023-09-09T23:10:11.059 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:11.059+0000 7f33ed7fa700 1 -- 172.21.15.98:0/2585350934 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f33f002cf10 con 0x7f33d80a06c0 2023-09-09T23:10:11.060 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:11.061+0000 7f33f640e700 1 -- 172.21.15.98:0/2585350934 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f33d80a06c0 msgr2=0x7f33d80a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:10:11.060 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:11.061+0000 7f33f640e700 1 --2- 172.21.15.98:0/2585350934 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f33d80a06c0 0x7f33d80a2b80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f33e4005fd0 tx=0x7f33e4005f60 comp rx=0 tx=0).stop 2023-09-09T23:10:11.060 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:11.061+0000 7f33f640e700 1 -- 172.21.15.98:0/2585350934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33f01011a0 msgr2=0x7f33f01963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:10:11.061 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:11.061+0000 7f33f640e700 1 --2- 172.21.15.98:0/2585350934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33f01011a0 0x7f33f01963e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f33e0000c00 tx=0x7f33e000f710 comp rx=0 tx=0).stop 2023-09-09T23:10:11.061 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:11.062+0000 7f33f640e700 1 -- 172.21.15.98:0/2585350934 shutdown_connections 2023-09-09T23:10:11.061 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:11.062+0000 7f33f640e700 1 --2- 172.21.15.98:0/2585350934 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f33d80a06c0 0x7f33d80a2b80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:10:11.061 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:11.062+0000 7f33f640e700 1 --2- 172.21.15.98:0/2585350934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33f01011a0 0x7f33f01963e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:10:11.061 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:11.062+0000 7f33f640e700 1 --2- 172.21.15.98:0/2585350934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33f0106400 0x7f33f019d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:10:11.062 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:11.062+0000 7f33f640e700 1 --2- 172.21.15.98:0/2585350934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33f0103ad0 0x7f33f0196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:10:11.062 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:11.062+0000 7f33f640e700 1 -- 172.21.15.98:0/2585350934 >> 172.21.15.98:0/2585350934 conn(0x7f33f00fab10 msgr2=0x7f33f0101e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:10:11.062 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:11.062+0000 7f33f640e700 1 -- 172.21.15.98:0/2585350934 shutdown_connections 2023-09-09T23:10:11.062 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:11.062+0000 7f33f640e700 1 -- 172.21.15.98:0/2585350934 wait complete. 2023-09-09T23:10:11.459 DEBUG:teuthology.orchestra.run.smithi098:osd.7> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@osd.7.service 2023-09-09T23:10:11.461 INFO:tasks.cephadm:Deploying osd.8 on smithi170 with /dev/vg_nvme/lv_2... 2023-09-09T23:10:11.462 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-09-09T23:10:11.490 INFO:journalctl@ceph.osd.7.smithi098.stdout:-- Logs begin at Sat 2023-09-09 22:51:28 UTC. -- 2023-09-09T23:10:12.029 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-09T23:10:12.165 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:11 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:12.165 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:11 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:12.165 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:11 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:12.165 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:11 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:10:12.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:11 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:12.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:11 smithi098 ceph-mon[103124]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:10:12.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:11 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:12.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:11 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:12.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:11 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:10:12.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:11 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:12.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:11 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:12.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:11 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:12.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:11 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:10:12.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:11 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:12.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:11 smithi170 ceph-mon[102832]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:10:12.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:11 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:11 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:11 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:10:12.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:11 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:12.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:11 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:12.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:11 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:12.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:11 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:10:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:11 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:11 smithi082 ceph-mon[98694]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:10:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:11 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:11 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:11 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:10:12.752 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 09 23:10:12 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-7[124140]: 2023-09-09T23:10:12.453+0000 7fe1a556e640 -1 osd.7 0 log_to_monitors true 2023-09-09T23:10:13.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:12 smithi082 ceph-mon[98694]: from='osd.7 [v2:172.21.15.98:6816/3547615284,v1:172.21.15.98:6817/3547615284]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-09T23:10:13.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:12 smithi082 ceph-mon[98694]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-09T23:10:13.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:12 smithi098 ceph-mon[103124]: from='osd.7 [v2:172.21.15.98:6816/3547615284,v1:172.21.15.98:6817/3547615284]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-09T23:10:13.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:12 smithi098 ceph-mon[103124]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-09T23:10:13.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:12 smithi170 ceph-mon[102832]: from='osd.7 [v2:172.21.15.98:6816/3547615284,v1:172.21.15.98:6817/3547615284]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-09T23:10:13.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:12 smithi170 ceph-mon[102832]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-09T23:10:13.554 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-09-09T23:10:13.555 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-09-09T23:10:13.555 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2023-09-09T23:10:13.555 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2023-09-09T23:10:13.555 INFO:teuthology.orchestra.run.smithi170.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124151 s, 845 MB/s 2023-09-09T23:10:13.556 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2023-09-09T23:10:13.969 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi170:vg_nvme/lv_2 2023-09-09T23:10:14.188 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:13 smithi098 ceph-mon[103124]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-09-09T23:10:14.189 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:13 smithi098 ceph-mon[103124]: osdmap e48: 8 total, 7 up, 8 in 2023-09-09T23:10:14.189 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:13 smithi098 ceph-mon[103124]: from='osd.7 [v2:172.21.15.98:6816/3547615284,v1:172.21.15.98:6817/3547615284]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-09T23:10:14.189 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:13 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-09T23:10:14.189 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:13 smithi098 ceph-mon[103124]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-09T23:10:14.189 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:13 smithi098 ceph-mon[103124]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:10:14.190 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 09 23:10:13 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-7[124140]: 2023-09-09T23:10:13.946+0000 7fe19a3e4700 -1 osd.7 0 waiting for initial osdmap 2023-09-09T23:10:14.190 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 09 23:10:13 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-7[124140]: 2023-09-09T23:10:13.955+0000 7fe1949d6700 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-09T23:10:14.204 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:13 smithi082 ceph-mon[98694]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-09-09T23:10:14.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:13 smithi082 ceph-mon[98694]: osdmap e48: 8 total, 7 up, 8 in 2023-09-09T23:10:14.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:13 smithi082 ceph-mon[98694]: from='osd.7 [v2:172.21.15.98:6816/3547615284,v1:172.21.15.98:6817/3547615284]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-09T23:10:14.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:13 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-09T23:10:14.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:13 smithi082 ceph-mon[98694]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-09T23:10:14.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:13 smithi082 ceph-mon[98694]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:10:14.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:13 smithi170 ceph-mon[102832]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-09-09T23:10:14.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:13 smithi170 ceph-mon[102832]: osdmap e48: 8 total, 7 up, 8 in 2023-09-09T23:10:14.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:13 smithi170 ceph-mon[102832]: from='osd.7 [v2:172.21.15.98:6816/3547615284,v1:172.21.15.98:6817/3547615284]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-09T23:10:14.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:13 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-09T23:10:14.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:13 smithi170 ceph-mon[102832]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-09T23:10:14.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:13 smithi170 ceph-mon[102832]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:10:14.521 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-09T23:10:15.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:14 smithi082 ceph-mon[98694]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-09-09T23:10:15.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:14 smithi082 ceph-mon[98694]: osdmap e49: 8 total, 7 up, 8 in 2023-09-09T23:10:15.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:14 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-09T23:10:15.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:14 smithi082 ceph-mon[98694]: Detected new or changed devices on smithi098 2023-09-09T23:10:15.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:14 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:15.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:14 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:15.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:14 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:15.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:14 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:14 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:14 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 5722M 2023-09-09T23:10:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:14 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:14 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:14 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:10:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:14 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:15.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:14 smithi098 ceph-mon[103124]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-09-09T23:10:15.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:14 smithi098 ceph-mon[103124]: osdmap e49: 8 total, 7 up, 8 in 2023-09-09T23:10:15.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:14 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-09T23:10:15.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:14 smithi098 ceph-mon[103124]: Detected new or changed devices on smithi098 2023-09-09T23:10:15.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:14 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:15.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:14 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:14 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:14 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:14 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:14 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 5722M 2023-09-09T23:10:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:14 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:14 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:14 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:10:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:14 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:15.402 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:14 smithi170 ceph-mon[102832]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-09-09T23:10:15.403 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:14 smithi170 ceph-mon[102832]: osdmap e49: 8 total, 7 up, 8 in 2023-09-09T23:10:15.403 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:14 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-09T23:10:15.403 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:14 smithi170 ceph-mon[102832]: Detected new or changed devices on smithi098 2023-09-09T23:10:15.403 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:14 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:15.404 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:14 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:15.404 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:14 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:15.404 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:14 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:15.404 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:14 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:15.404 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:14 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 5722M 2023-09-09T23:10:15.405 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:14 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:15.405 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:14 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:15.405 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:14 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:10:15.405 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:14 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:15.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.584+0000 7fdc5d9e0700 1 -- 172.21.15.170:0/3351366240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc580ff860 msgr2=0x7fdc580ffce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:10:15.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.584+0000 7fdc5d9e0700 1 --2- 172.21.15.170:0/3351366240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc580ff860 0x7fdc580ffce0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdc48009990 tx=0x7fdc48009ca0 comp rx=0 tx=0).stop 2023-09-09T23:10:15.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.584+0000 7fdc5d9e0700 1 -- 172.21.15.170:0/3351366240 shutdown_connections 2023-09-09T23:10:15.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.584+0000 7fdc5d9e0700 1 --2- 172.21.15.170:0/3351366240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc58100220 0x7fdc58107ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:10:15.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.584+0000 7fdc5d9e0700 1 --2- 172.21.15.170:0/3351366240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc58100c00 0x7fdc58101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:10:15.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.584+0000 7fdc5d9e0700 1 --2- 172.21.15.170:0/3351366240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc580ff860 0x7fdc580ffce0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:10:15.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.584+0000 7fdc5d9e0700 1 -- 172.21.15.170:0/3351366240 >> 172.21.15.170:0/3351366240 conn(0x7fdc580fa310 msgr2=0x7fdc580fc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:10:15.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.584+0000 7fdc5d9e0700 1 -- 172.21.15.170:0/3351366240 shutdown_connections 2023-09-09T23:10:15.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.584+0000 7fdc5d9e0700 1 -- 172.21.15.170:0/3351366240 wait complete. 2023-09-09T23:10:15.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.585+0000 7fdc5d9e0700 1 Processor -- start 2023-09-09T23:10:15.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.585+0000 7fdc5d9e0700 1 -- start start 2023-09-09T23:10:15.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.585+0000 7fdc5d9e0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc580ff860 0x7fdc580ff2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:10:15.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.585+0000 7fdc5d9e0700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc58100220 0x7fdc5819b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:10:15.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.585+0000 7fdc5d9e0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc58100c00 0x7fdc5819d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:10:15.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.585+0000 7fdc5d9e0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5819ded0 con 0x7fdc580ff860 2023-09-09T23:10:15.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.585+0000 7fdc5d9e0700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5819e040 con 0x7fdc58100220 2023-09-09T23:10:15.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.585+0000 7fdc5d9e0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5819e200 con 0x7fdc58100c00 2023-09-09T23:10:15.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.585+0000 7fdc577fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc58100c00 0x7fdc5819d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:10:15.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.585+0000 7fdc577fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc58100c00 0x7fdc5819d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34572/0 (socket says 172.21.15.170:34572) 2023-09-09T23:10:15.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.585+0000 7fdc567fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc58100220 0x7fdc5819b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:10:15.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.585+0000 7fdc577fe700 1 -- 172.21.15.170:0/600264200 learned_addr learned my addr 172.21.15.170:0/600264200 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-09-09T23:10:15.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.586+0000 7fdc56ffd700 1 --2- 172.21.15.170:0/600264200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc580ff860 0x7fdc580ff2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:10:15.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.586+0000 7fdc577fe700 1 -- 172.21.15.170:0/600264200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc58100220 msgr2=0x7fdc5819b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:10:15.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.586+0000 7fdc577fe700 1 --2- 172.21.15.170:0/600264200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc58100220 0x7fdc5819b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:10:15.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.586+0000 7fdc577fe700 1 -- 172.21.15.170:0/600264200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc580ff860 msgr2=0x7fdc580ff2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:10:15.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.586+0000 7fdc577fe700 1 --2- 172.21.15.170:0/600264200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc580ff860 0x7fdc580ff2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:10:15.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.586+0000 7fdc577fe700 1 -- 172.21.15.170:0/600264200 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc480095f0 con 0x7fdc58100c00 2023-09-09T23:10:15.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.586+0000 7fdc577fe700 1 --2- 172.21.15.170:0/600264200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc58100c00 0x7fdc5819d730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000ef30 tx=0x7fdc5000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:10:15.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.586+0000 7fdc5c9de700 1 -- 172.21.15.170:0/600264200 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc50018070 con 0x7fdc58100c00 2023-09-09T23:10:15.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.586+0000 7fdc5d9e0700 1 -- 172.21.15.170:0/600264200 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc5819e480 con 0x7fdc58100c00 2023-09-09T23:10:15.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.586+0000 7fdc5c9de700 1 -- 172.21.15.170:0/600264200 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc5000f040 con 0x7fdc58100c00 2023-09-09T23:10:15.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.586+0000 7fdc5c9de700 1 -- 172.21.15.170:0/600264200 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5000cdb0 con 0x7fdc58100c00 2023-09-09T23:10:15.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.586+0000 7fdc5d9e0700 1 -- 172.21.15.170:0/600264200 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc5819e9d0 con 0x7fdc58100c00 2023-09-09T23:10:15.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.587+0000 7fdc5d9e0700 1 -- 172.21.15.170:0/600264200 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc58106bf0 con 0x7fdc58100c00 2023-09-09T23:10:15.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.589+0000 7fdc5c9de700 1 -- 172.21.15.170:0/600264200 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fdc50003a00 con 0x7fdc58100c00 2023-09-09T23:10:15.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.589+0000 7fdc5c9de700 1 --2- 172.21.15.170:0/600264200 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fdc400a0640 0x7fdc400a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:10:15.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.590+0000 7fdc5c9de700 1 -- 172.21.15.170:0/600264200 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5430+0+0 (secure 0 0 0) 0x7fdc500cb040 con 0x7fdc58100c00 2023-09-09T23:10:15.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.590+0000 7fdc56ffd700 1 --2- 172.21.15.170:0/600264200 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fdc400a0640 0x7fdc400a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:10:15.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.590+0000 7fdc56ffd700 1 --2- 172.21.15.170:0/600264200 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fdc400a0640 0x7fdc400a2b00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c005fd0 tx=0x7fdc4c005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:10:15.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.593+0000 7fdc5c9de700 1 -- 172.21.15.170:0/600264200 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fdc50095740 con 0x7fdc58100c00 2023-09-09T23:10:15.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:15.790+0000 7fdc5d9e0700 1 -- 172.21.15.170:0/600264200 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc5802cf60 con 0x7fdc400a0640 2023-09-09T23:10:16.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:15 smithi082 ceph-mon[98694]: purged_snaps scrub starts 2023-09-09T23:10:16.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:15 smithi082 ceph-mon[98694]: purged_snaps scrub ok 2023-09-09T23:10:16.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:15 smithi082 ceph-mon[98694]: osd.7 [v2:172.21.15.98:6816/3547615284,v1:172.21.15.98:6817/3547615284] boot 2023-09-09T23:10:16.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:15 smithi082 ceph-mon[98694]: osdmap e50: 8 total, 8 up, 8 in 2023-09-09T23:10:16.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:15 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-09T23:10:16.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:15 smithi082 ceph-mon[98694]: pgmap v163: 1 pgs: 1 unknown; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:10:16.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:15 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:10:16.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:15 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:10:16.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:15 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:15 smithi082 ceph-mon[98694]: osdmap e51: 8 total, 8 up, 8 in 2023-09-09T23:10:16.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:15 smithi098 ceph-mon[103124]: purged_snaps scrub starts 2023-09-09T23:10:16.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:15 smithi098 ceph-mon[103124]: purged_snaps scrub ok 2023-09-09T23:10:16.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:15 smithi098 ceph-mon[103124]: osd.7 [v2:172.21.15.98:6816/3547615284,v1:172.21.15.98:6817/3547615284] boot 2023-09-09T23:10:16.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:15 smithi098 ceph-mon[103124]: osdmap e50: 8 total, 8 up, 8 in 2023-09-09T23:10:16.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:15 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-09T23:10:16.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:15 smithi098 ceph-mon[103124]: pgmap v163: 1 pgs: 1 unknown; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:10:16.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:15 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:10:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:15 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:10:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:15 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:15 smithi098 ceph-mon[103124]: osdmap e51: 8 total, 8 up, 8 in 2023-09-09T23:10:16.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:15 smithi170 ceph-mon[102832]: purged_snaps scrub starts 2023-09-09T23:10:16.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:15 smithi170 ceph-mon[102832]: purged_snaps scrub ok 2023-09-09T23:10:16.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:15 smithi170 ceph-mon[102832]: osd.7 [v2:172.21.15.98:6816/3547615284,v1:172.21.15.98:6817/3547615284] boot 2023-09-09T23:10:16.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:15 smithi170 ceph-mon[102832]: osdmap e50: 8 total, 8 up, 8 in 2023-09-09T23:10:16.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:15 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-09T23:10:16.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:15 smithi170 ceph-mon[102832]: pgmap v163: 1 pgs: 1 unknown; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-09T23:10:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:15 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:10:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:15 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:10:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:15 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:15 smithi170 ceph-mon[102832]: osdmap e51: 8 total, 8 up, 8 in 2023-09-09T23:10:17.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:16 smithi170 ceph-mon[102832]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:10:17.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:16 smithi170 ceph-mon[102832]: osdmap e52: 8 total, 8 up, 8 in 2023-09-09T23:10:17.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:16 smithi082 ceph-mon[98694]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:10:17.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:16 smithi082 ceph-mon[98694]: osdmap e52: 8 total, 8 up, 8 in 2023-09-09T23:10:17.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:16 smithi098 ceph-mon[103124]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:10:17.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:16 smithi098 ceph-mon[103124]: osdmap e52: 8 total, 8 up, 8 in 2023-09-09T23:10:18.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:17 smithi098 ceph-mon[103124]: pgmap v166: 1 pgs: 1 unknown; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:18.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:17 smithi170 ceph-mon[102832]: pgmap v166: 1 pgs: 1 unknown; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:18.454 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:17 smithi082 ceph-mon[98694]: pgmap v166: 1 pgs: 1 unknown; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:19.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:18 smithi098 ceph-mon[103124]: osdmap e53: 8 total, 8 up, 8 in 2023-09-09T23:10:19.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:18 smithi098 ceph-mon[103124]: from='client.? 172.21.15.170:0/1252395697' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23b27c86-8c98-441e-b7ab-d57bc584e58f"}]: dispatch 2023-09-09T23:10:19.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:18 smithi098 ceph-mon[103124]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23b27c86-8c98-441e-b7ab-d57bc584e58f"}]: dispatch 2023-09-09T23:10:19.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:18 smithi098 ceph-mon[103124]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23b27c86-8c98-441e-b7ab-d57bc584e58f"}]': finished 2023-09-09T23:10:19.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:18 smithi098 ceph-mon[103124]: osdmap e54: 9 total, 8 up, 9 in 2023-09-09T23:10:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:18 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-09T23:10:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:18 smithi098 ceph-mon[103124]: from='client.? 172.21.15.170:0/1332976921' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:10:19.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:18 smithi170 ceph-mon[102832]: osdmap e53: 8 total, 8 up, 8 in 2023-09-09T23:10:19.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:18 smithi170 ceph-mon[102832]: from='client.? 172.21.15.170:0/1252395697' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23b27c86-8c98-441e-b7ab-d57bc584e58f"}]: dispatch 2023-09-09T23:10:19.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:18 smithi170 ceph-mon[102832]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23b27c86-8c98-441e-b7ab-d57bc584e58f"}]: dispatch 2023-09-09T23:10:19.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:18 smithi170 ceph-mon[102832]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23b27c86-8c98-441e-b7ab-d57bc584e58f"}]': finished 2023-09-09T23:10:19.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:18 smithi170 ceph-mon[102832]: osdmap e54: 9 total, 8 up, 9 in 2023-09-09T23:10:19.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:18 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-09T23:10:19.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:18 smithi170 ceph-mon[102832]: from='client.? 172.21.15.170:0/1332976921' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:10:19.454 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:18 smithi082 ceph-mon[98694]: osdmap e53: 8 total, 8 up, 8 in 2023-09-09T23:10:19.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:18 smithi082 ceph-mon[98694]: from='client.? 172.21.15.170:0/1252395697' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23b27c86-8c98-441e-b7ab-d57bc584e58f"}]: dispatch 2023-09-09T23:10:19.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:18 smithi082 ceph-mon[98694]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "23b27c86-8c98-441e-b7ab-d57bc584e58f"}]: dispatch 2023-09-09T23:10:19.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:18 smithi082 ceph-mon[98694]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "23b27c86-8c98-441e-b7ab-d57bc584e58f"}]': finished 2023-09-09T23:10:19.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:18 smithi082 ceph-mon[98694]: osdmap e54: 9 total, 8 up, 9 in 2023-09-09T23:10:19.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:18 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-09T23:10:19.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:18 smithi082 ceph-mon[98694]: from='client.? 172.21.15.170:0/1332976921' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:10:20.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:20 smithi098 ceph-mon[103124]: pgmap v169: 1 pgs: 1 unknown; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:20.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:20 smithi170 ceph-mon[102832]: pgmap v169: 1 pgs: 1 unknown; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:20.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:20 smithi082 ceph-mon[98694]: pgmap v169: 1 pgs: 1 unknown; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:22.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:22 smithi098 ceph-mon[103124]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:22.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:22 smithi170 ceph-mon[102832]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:22.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:22 smithi082 ceph-mon[98694]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:24.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:24 smithi098 ceph-mon[103124]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:24.590 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:24 smithi170 ceph-mon[102832]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:24.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:24 smithi082 ceph-mon[98694]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:25.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:25 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-09-09T23:10:25.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:25 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:25.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:25 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-09-09T23:10:25.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:25 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:25.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:25 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-09-09T23:10:25.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:25 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:26.422 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:26 smithi170 ceph-mon[102832]: Deploying daemon osd.8 on smithi170 2023-09-09T23:10:26.423 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:26 smithi170 ceph-mon[102832]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:26.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:26 smithi098 ceph-mon[103124]: Deploying daemon osd.8 on smithi170 2023-09-09T23:10:26.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:26 smithi098 ceph-mon[103124]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:26.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:26 smithi082 ceph-mon[98694]: Deploying daemon osd.8 on smithi170 2023-09-09T23:10:26.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:26 smithi082 ceph-mon[98694]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:28.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:28 smithi098 ceph-mon[103124]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:28.617 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:28 smithi170 ceph-mon[102832]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:28.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:28 smithi082 ceph-mon[98694]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:29.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:29 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:29.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:29 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:29.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:29 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:10:29.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:29 smithi170 ceph-mon[102832]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:30.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:29 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:30.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:29 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:30.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:29 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:10:30.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:29 smithi082 ceph-mon[98694]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:30.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:29 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:30.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:29 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:30.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:29 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:10:30.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:29 smithi098 ceph-mon[103124]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:31.450 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:31.451+0000 7fdc5c9de700 1 -- 172.21.15.170:0/600264200 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fdc5802cf60 con 0x7fdc400a0640 2023-09-09T23:10:31.451 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 8 on host 'smithi170' 2023-09-09T23:10:31.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:31.453+0000 7fdc5d9e0700 1 -- 172.21.15.170:0/600264200 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fdc400a0640 msgr2=0x7fdc400a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:10:31.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:31.453+0000 7fdc5d9e0700 1 --2- 172.21.15.170:0/600264200 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fdc400a0640 0x7fdc400a2b00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c005fd0 tx=0x7fdc4c005dd0 comp rx=0 tx=0).stop 2023-09-09T23:10:31.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:31.453+0000 7fdc5d9e0700 1 -- 172.21.15.170:0/600264200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc58100c00 msgr2=0x7fdc5819d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:10:31.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:31.453+0000 7fdc5d9e0700 1 --2- 172.21.15.170:0/600264200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc58100c00 0x7fdc5819d730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000ef30 tx=0x7fdc5000c3a0 comp rx=0 tx=0).stop 2023-09-09T23:10:31.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:31.453+0000 7fdc5d9e0700 1 -- 172.21.15.170:0/600264200 shutdown_connections 2023-09-09T23:10:31.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:31.453+0000 7fdc5d9e0700 1 --2- 172.21.15.170:0/600264200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc58100c00 0x7fdc5819d730 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:10:31.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:31.453+0000 7fdc5d9e0700 1 --2- 172.21.15.170:0/600264200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc580ff860 0x7fdc580ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:10:31.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:31.453+0000 7fdc5d9e0700 1 --2- 172.21.15.170:0/600264200 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fdc400a0640 0x7fdc400a2b00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:10:31.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:31.453+0000 7fdc5d9e0700 1 --2- 172.21.15.170:0/600264200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc58100220 0x7fdc5819b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:10:31.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:31.453+0000 7fdc5d9e0700 1 -- 172.21.15.170:0/600264200 >> 172.21.15.170:0/600264200 conn(0x7fdc580fa310 msgr2=0x7fdc58103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:10:31.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:31.454+0000 7fdc5d9e0700 1 -- 172.21.15.170:0/600264200 shutdown_connections 2023-09-09T23:10:31.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:10:31.454+0000 7fdc5d9e0700 1 -- 172.21.15.170:0/600264200 wait complete. 2023-09-09T23:10:31.799 DEBUG:teuthology.orchestra.run.smithi170:osd.8> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@osd.8.service 2023-09-09T23:10:31.802 INFO:tasks.cephadm:Deploying osd.9 on smithi082 with /dev/vg_nvme/lv_1... 2023-09-09T23:10:31.802 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-09-09T23:10:31.831 INFO:journalctl@ceph.osd.8.smithi170.stdout:-- Logs begin at Sat 2023-09-09 22:51:11 UTC. -- 2023-09-09T23:10:32.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:32 smithi170 ceph-mon[102832]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:32.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:32 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:32.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:32 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:32.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:32 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:32.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:32 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:32.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:32 smithi082 ceph-mon[98694]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:32.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:32 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:32.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:32 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:32.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:32 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:32.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:32 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:32.369 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:10:32.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:32 smithi098 ceph-mon[103124]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:32.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:32 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:32.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:32 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:32.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:32 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:32.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:32 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:33.012 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 09 23:10:32 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-8[125297]: 2023-09-09T23:10:32.814+0000 7f5d3a572640 -1 osd.8 0 log_to_monitors true 2023-09-09T23:10:33.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:33 smithi082 ceph-mon[98694]: from='osd.8 [v2:172.21.15.170:6816/632038848,v1:172.21.15.170:6817/632038848]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-09T23:10:33.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:33 smithi082 ceph-mon[98694]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-09T23:10:33.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:33 smithi170 ceph-mon[102832]: from='osd.8 [v2:172.21.15.170:6816/632038848,v1:172.21.15.170:6817/632038848]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-09T23:10:33.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:33 smithi170 ceph-mon[102832]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-09T23:10:33.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:33 smithi098 ceph-mon[103124]: from='osd.8 [v2:172.21.15.170:6816/632038848,v1:172.21.15.170:6817/632038848]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-09T23:10:33.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:33 smithi098 ceph-mon[103124]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-09T23:10:33.824 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-09-09T23:10:33.824 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-09-09T23:10:33.824 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2023-09-09T23:10:33.824 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2023-09-09T23:10:33.825 INFO:teuthology.orchestra.run.smithi082.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110586 s, 948 MB/s 2023-09-09T23:10:33.825 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2023-09-09T23:10:34.148 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:34 smithi082 ceph-mon[98694]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:34.148 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:34 smithi082 ceph-mon[98694]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-09-09T23:10:34.148 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:34 smithi082 ceph-mon[98694]: osdmap e55: 9 total, 8 up, 9 in 2023-09-09T23:10:34.148 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:34 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-09T23:10:34.149 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:34 smithi082 ceph-mon[98694]: from='osd.8 [v2:172.21.15.170:6816/632038848,v1:172.21.15.170:6817/632038848]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-09T23:10:34.149 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:34 smithi082 ceph-mon[98694]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-09T23:10:34.213 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi082:vg_nvme/lv_1 2023-09-09T23:10:34.383 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 09 23:10:34 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-8[125297]: 2023-09-09T23:10:34.020+0000 7f5d2f3e8700 -1 osd.8 0 waiting for initial osdmap 2023-09-09T23:10:34.384 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 09 23:10:34 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-8[125297]: 2023-09-09T23:10:34.038+0000 7f5d299da700 -1 osd.8 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-09T23:10:34.384 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:34 smithi170 ceph-mon[102832]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:34.384 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:34 smithi170 ceph-mon[102832]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-09-09T23:10:34.384 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:34 smithi170 ceph-mon[102832]: osdmap e55: 9 total, 8 up, 9 in 2023-09-09T23:10:34.384 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:34 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-09T23:10:34.385 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:34 smithi170 ceph-mon[102832]: from='osd.8 [v2:172.21.15.170:6816/632038848,v1:172.21.15.170:6817/632038848]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-09T23:10:34.385 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:34 smithi170 ceph-mon[102832]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-09T23:10:34.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:34 smithi098 ceph-mon[103124]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:34.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:34 smithi098 ceph-mon[103124]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-09-09T23:10:34.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:34 smithi098 ceph-mon[103124]: osdmap e55: 9 total, 8 up, 9 in 2023-09-09T23:10:34.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:34 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-09T23:10:34.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:34 smithi098 ceph-mon[103124]: from='osd.8 [v2:172.21.15.170:6816/632038848,v1:172.21.15.170:6817/632038848]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-09T23:10:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:34 smithi098 ceph-mon[103124]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-09T23:10:34.774 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:10:35.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:35 smithi082 ceph-mon[98694]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-09-09T23:10:35.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:35 smithi082 ceph-mon[98694]: osdmap e56: 9 total, 8 up, 9 in 2023-09-09T23:10:35.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:35 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-09T23:10:35.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:35 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-09T23:10:35.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:35 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:35.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:35 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:35.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:35 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:35.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:35 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:35 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:35 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:35 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:35 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:10:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:35 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:35 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:10:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:35 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:35 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:10:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:35 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:35 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-09T23:10:35.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:35 smithi170 ceph-mon[102832]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-09-09T23:10:35.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:35 smithi170 ceph-mon[102832]: osdmap e56: 9 total, 8 up, 9 in 2023-09-09T23:10:35.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:35 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-09T23:10:35.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:35 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-09T23:10:35.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:35 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:35.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:35 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:35 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:35 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:35 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:35 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:35 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:35 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:10:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:35 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:35 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:10:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:35 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:35 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:10:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:35 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:35.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:35 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-09T23:10:35.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:35 smithi098 ceph-mon[103124]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-09-09T23:10:35.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:35 smithi098 ceph-mon[103124]: osdmap e56: 9 total, 8 up, 9 in 2023-09-09T23:10:35.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:35 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-09T23:10:35.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:35 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-09T23:10:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:35 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:35 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:35 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:35 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:35 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:35 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:35 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:35 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:10:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:35 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:35 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:10:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:35 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:35.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:35 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:10:35.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:35 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:35.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:35 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-09T23:10:35.860 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.860+0000 7fdece7df700 1 -- 172.21.15.82:0/2531941402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec80ff7e0 msgr2=0x7fdec80ffc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:10:35.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.860+0000 7fdece7df700 1 --2- 172.21.15.82:0/2531941402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec80ff7e0 0x7fdec80ffc60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8009990 tx=0x7fdeb8009ca0 comp rx=0 tx=0).stop 2023-09-09T23:10:35.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.860+0000 7fdece7df700 1 -- 172.21.15.82:0/2531941402 shutdown_connections 2023-09-09T23:10:35.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.860+0000 7fdece7df700 1 --2- 172.21.15.82:0/2531941402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdec81001a0 0x7fdec8107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:10:35.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.860+0000 7fdece7df700 1 --2- 172.21.15.82:0/2531941402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdec8100c10 0x7fdec8101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:10:35.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.860+0000 7fdece7df700 1 --2- 172.21.15.82:0/2531941402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec80ff7e0 0x7fdec80ffc60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:10:35.862 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.860+0000 7fdece7df700 1 -- 172.21.15.82:0/2531941402 >> 172.21.15.82:0/2531941402 conn(0x7fdec80fa2b0 msgr2=0x7fdec80fc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:10:35.862 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.860+0000 7fdece7df700 1 -- 172.21.15.82:0/2531941402 shutdown_connections 2023-09-09T23:10:35.862 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.860+0000 7fdece7df700 1 -- 172.21.15.82:0/2531941402 wait complete. 2023-09-09T23:10:35.862 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.860+0000 7fdece7df700 1 Processor -- start 2023-09-09T23:10:35.862 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.860+0000 7fdece7df700 1 -- start start 2023-09-09T23:10:35.863 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.861+0000 7fdece7df700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdec80ff7e0 0x7fdec8196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:10:35.863 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.861+0000 7fdece7df700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdec81001a0 0x7fdec81968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:10:35.863 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.861+0000 7fdece7df700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec8100c10 0x7fdec819d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:10:35.863 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.861+0000 7fdece7df700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec819df20 con 0x7fdec8100c10 2023-09-09T23:10:35.863 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.861+0000 7fdece7df700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec819e060 con 0x7fdec81001a0 2023-09-09T23:10:35.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.861+0000 7fdece7df700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec819e1f0 con 0x7fdec80ff7e0 2023-09-09T23:10:35.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.861+0000 7fdeccd7c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec8100c10 0x7fdec819d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:10:35.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.861+0000 7fdeccd7c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec8100c10 0x7fdec819d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53874/0 (socket says 172.21.15.82:53874) 2023-09-09T23:10:35.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.861+0000 7fdeccd7c700 1 -- 172.21.15.82:0/603659644 learned_addr learned my addr 172.21.15.82:0/603659644 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:10:35.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.861+0000 7fdec7fff700 1 --2- 172.21.15.82:0/603659644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdec80ff7e0 0x7fdec8196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:10:35.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.861+0000 7fdec77fe700 1 --2- 172.21.15.82:0/603659644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdec81001a0 0x7fdec81968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:10:35.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.861+0000 7fdeccd7c700 1 -- 172.21.15.82:0/603659644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdec80ff7e0 msgr2=0x7fdec8196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:10:35.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.861+0000 7fdeccd7c700 1 --2- 172.21.15.82:0/603659644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdec80ff7e0 0x7fdec8196390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:10:35.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.861+0000 7fdeccd7c700 1 -- 172.21.15.82:0/603659644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdec81001a0 msgr2=0x7fdec81968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:10:35.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.861+0000 7fdeccd7c700 1 --2- 172.21.15.82:0/603659644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdec81001a0 0x7fdec81968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:10:35.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.861+0000 7fdeccd7c700 1 -- 172.21.15.82:0/603659644 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb80095f0 con 0x7fdec8100c10 2023-09-09T23:10:35.866 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.861+0000 7fdec77fe700 1 --2- 172.21.15.82:0/603659644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdec81001a0 0x7fdec81968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:10:35.866 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.861+0000 7fdeccd7c700 1 --2- 172.21.15.82:0/603659644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec8100c10 0x7fdec819d950 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdec000add0 tx=0x7fdec000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:10:35.866 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.861+0000 7fdec57fa700 1 -- 172.21.15.82:0/603659644 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec000f070 con 0x7fdec8100c10 2023-09-09T23:10:35.866 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.861+0000 7fdec57fa700 1 -- 172.21.15.82:0/603659644 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdec0007b30 con 0x7fdec8100c10 2023-09-09T23:10:35.866 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.861+0000 7fdec57fa700 1 -- 172.21.15.82:0/603659644 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec0005310 con 0x7fdec8100c10 2023-09-09T23:10:35.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.861+0000 7fdece7df700 1 -- 172.21.15.82:0/603659644 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec819e4d0 con 0x7fdec8100c10 2023-09-09T23:10:35.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.862+0000 7fdece7df700 1 -- 172.21.15.82:0/603659644 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec819e980 con 0x7fdec8100c10 2023-09-09T23:10:35.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.863+0000 7fdece7df700 1 -- 172.21.15.82:0/603659644 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec810ae10 con 0x7fdec8100c10 2023-09-09T23:10:35.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.864+0000 7fdec57fa700 1 -- 172.21.15.82:0/603659644 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fdec000c940 con 0x7fdec8100c10 2023-09-09T23:10:35.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.864+0000 7fdec57fa700 1 --2- 172.21.15.82:0/603659644 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fdeb00a0640 0x7fdeb00a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:10:35.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.864+0000 7fdec7fff700 1 --2- 172.21.15.82:0/603659644 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fdeb00a0640 0x7fdeb00a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:10:35.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.864+0000 7fdec57fa700 1 -- 172.21.15.82:0/603659644 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5862+0+0 (secure 0 0 0) 0x7fdec00cadd0 con 0x7fdec8100c10 2023-09-09T23:10:35.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.865+0000 7fdec7fff700 1 --2- 172.21.15.82:0/603659644 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fdeb00a0640 0x7fdeb00a2b00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdebc005950 tx=0x7fdebc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:10:35.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:35.868+0000 7fdec57fa700 1 -- 172.21.15.82:0/603659644 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fdec0095320 con 0x7fdec8100c10 2023-09-09T23:10:36.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:36.069+0000 7fdece7df700 1 -- 172.21.15.82:0/603659644 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fdec80fee40 con 0x7fdeb00a0640 2023-09-09T23:10:36.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:36 smithi170 ceph-mon[102832]: purged_snaps scrub starts 2023-09-09T23:10:36.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:36 smithi170 ceph-mon[102832]: purged_snaps scrub ok 2023-09-09T23:10:36.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:36 smithi170 ceph-mon[102832]: Detected new or changed devices on smithi170 2023-09-09T23:10:36.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:36 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 5722M 2023-09-09T23:10:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:36 smithi170 ceph-mon[102832]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:36 smithi170 ceph-mon[102832]: osd.8 [v2:172.21.15.170:6816/632038848,v1:172.21.15.170:6817/632038848] boot 2023-09-09T23:10:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:36 smithi170 ceph-mon[102832]: osdmap e57: 9 total, 9 up, 9 in 2023-09-09T23:10:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:36 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-09T23:10:36.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:36 smithi082 ceph-mon[98694]: purged_snaps scrub starts 2023-09-09T23:10:36.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:36 smithi082 ceph-mon[98694]: purged_snaps scrub ok 2023-09-09T23:10:36.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:36 smithi082 ceph-mon[98694]: Detected new or changed devices on smithi170 2023-09-09T23:10:36.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:36 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 5722M 2023-09-09T23:10:36.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:36 smithi082 ceph-mon[98694]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:36.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:36 smithi082 ceph-mon[98694]: osd.8 [v2:172.21.15.170:6816/632038848,v1:172.21.15.170:6817/632038848] boot 2023-09-09T23:10:36.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:36 smithi082 ceph-mon[98694]: osdmap e57: 9 total, 9 up, 9 in 2023-09-09T23:10:36.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:36 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-09T23:10:36.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:36 smithi098 ceph-mon[103124]: purged_snaps scrub starts 2023-09-09T23:10:36.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:36 smithi098 ceph-mon[103124]: purged_snaps scrub ok 2023-09-09T23:10:36.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:36 smithi098 ceph-mon[103124]: Detected new or changed devices on smithi170 2023-09-09T23:10:36.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:36 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 5722M 2023-09-09T23:10:36.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:36 smithi098 ceph-mon[103124]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-09T23:10:36.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:36 smithi098 ceph-mon[103124]: osd.8 [v2:172.21.15.170:6816/632038848,v1:172.21.15.170:6817/632038848] boot 2023-09-09T23:10:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:36 smithi098 ceph-mon[103124]: osdmap e57: 9 total, 9 up, 9 in 2023-09-09T23:10:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:36 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-09T23:10:37.128 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:37 smithi082 ceph-mon[98694]: osdmap e58: 9 total, 9 up, 9 in 2023-09-09T23:10:37.128 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:37 smithi082 ceph-mon[98694]: from='client.14529 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:10:37.128 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:37 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:10:37.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:37 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:10:37.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:37 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:37.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:37 smithi170 ceph-mon[102832]: osdmap e58: 9 total, 9 up, 9 in 2023-09-09T23:10:37.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:37 smithi170 ceph-mon[102832]: from='client.14529 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:10:37.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:37 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:10:37.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:37 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:10:37.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:37 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:37.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:37 smithi098 ceph-mon[103124]: osdmap e58: 9 total, 9 up, 9 in 2023-09-09T23:10:37.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:37 smithi098 ceph-mon[103124]: from='client.14529 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:10:37.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:37 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:10:37.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:37 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:10:37.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:37 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:38.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:38 smithi170 ceph-mon[102832]: pgmap v182: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-09T23:10:38.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:38 smithi170 ceph-mon[102832]: osdmap e59: 9 total, 9 up, 9 in 2023-09-09T23:10:38.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:38 smithi082 ceph-mon[98694]: pgmap v182: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-09T23:10:38.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:38 smithi082 ceph-mon[98694]: osdmap e59: 9 total, 9 up, 9 in 2023-09-09T23:10:38.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:38 smithi098 ceph-mon[103124]: pgmap v182: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-09T23:10:38.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:38 smithi098 ceph-mon[103124]: osdmap e59: 9 total, 9 up, 9 in 2023-09-09T23:10:39.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:39 smithi170 ceph-mon[102832]: osdmap e60: 9 total, 9 up, 9 in 2023-09-09T23:10:39.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:39 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/4211594263' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "383b8989-4922-4eee-9c77-f198258ac243"}]: dispatch 2023-09-09T23:10:39.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:39 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/4211594263' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "383b8989-4922-4eee-9c77-f198258ac243"}]': finished 2023-09-09T23:10:39.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:39 smithi170 ceph-mon[102832]: osdmap e61: 10 total, 9 up, 10 in 2023-09-09T23:10:39.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:39 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-09T23:10:39.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:39 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2439017139' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:10:39.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:39 smithi082 ceph-mon[98694]: osdmap e60: 9 total, 9 up, 9 in 2023-09-09T23:10:39.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:39 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/4211594263' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "383b8989-4922-4eee-9c77-f198258ac243"}]: dispatch 2023-09-09T23:10:39.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:39 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/4211594263' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "383b8989-4922-4eee-9c77-f198258ac243"}]': finished 2023-09-09T23:10:39.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:39 smithi082 ceph-mon[98694]: osdmap e61: 10 total, 9 up, 10 in 2023-09-09T23:10:39.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:39 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-09T23:10:39.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:39 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2439017139' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:10:39.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:39 smithi098 ceph-mon[103124]: osdmap e60: 9 total, 9 up, 9 in 2023-09-09T23:10:39.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:39 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/4211594263' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "383b8989-4922-4eee-9c77-f198258ac243"}]: dispatch 2023-09-09T23:10:39.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:39 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/4211594263' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "383b8989-4922-4eee-9c77-f198258ac243"}]': finished 2023-09-09T23:10:39.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:39 smithi098 ceph-mon[103124]: osdmap e61: 10 total, 9 up, 10 in 2023-09-09T23:10:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:39 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-09T23:10:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:39 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2439017139' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:10:40.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:40 smithi170 ceph-mon[102832]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-09T23:10:40.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:40 smithi082 ceph-mon[98694]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-09T23:10:40.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:40 smithi098 ceph-mon[103124]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-09T23:10:42.343 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:42 smithi082 ceph-mon[98694]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-09-09T23:10:42.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:42 smithi170 ceph-mon[102832]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-09-09T23:10:42.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:42 smithi098 ceph-mon[103124]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-09-09T23:10:44.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:44 smithi082 ceph-mon[98694]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-09T23:10:44.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:44 smithi098 ceph-mon[103124]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-09T23:10:44.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:44 smithi170 ceph-mon[102832]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-09T23:10:45.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:45 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-09-09T23:10:45.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:45 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:45.558 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:45 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-09-09T23:10:45.558 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:45 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:45.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:45 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-09-09T23:10:45.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:45 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:46.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:46 smithi098 ceph-mon[103124]: Deploying daemon osd.9 on smithi082 2023-09-09T23:10:46.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:46 smithi098 ceph-mon[103124]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-09-09T23:10:46.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:46 smithi170 ceph-mon[102832]: Deploying daemon osd.9 on smithi082 2023-09-09T23:10:46.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:46 smithi170 ceph-mon[102832]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-09-09T23:10:46.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:46 smithi082 ceph-mon[98694]: Deploying daemon osd.9 on smithi082 2023-09-09T23:10:46.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:46 smithi082 ceph-mon[98694]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-09-09T23:10:48.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:48 smithi098 ceph-mon[103124]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-09-09T23:10:48.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:48 smithi170 ceph-mon[102832]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-09-09T23:10:48.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:48 smithi082 ceph-mon[98694]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-09-09T23:10:50.308 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:50 smithi082 ceph-mon[98694]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-09-09T23:10:50.308 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:50 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:50.308 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:50 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:10:50.309 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:50 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:50.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:50 smithi170 ceph-mon[102832]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-09-09T23:10:50.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:50 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:50.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:50 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:10:50.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:50 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:50.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:50 smithi098 ceph-mon[103124]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-09-09T23:10:50.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:50 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:50.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:50 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:10:50.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:50 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:51.568 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 9 on host 'smithi082' 2023-09-09T23:10:51.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:51.569+0000 7fdec57fa700 1 -- 172.21.15.82:0/603659644 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fdec80fee40 con 0x7fdeb00a0640 2023-09-09T23:10:51.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:51.571+0000 7fdece7df700 1 -- 172.21.15.82:0/603659644 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fdeb00a0640 msgr2=0x7fdeb00a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:10:51.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:51.571+0000 7fdece7df700 1 --2- 172.21.15.82:0/603659644 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fdeb00a0640 0x7fdeb00a2b00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdebc005950 tx=0x7fdebc00a300 comp rx=0 tx=0).stop 2023-09-09T23:10:51.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:51.571+0000 7fdece7df700 1 -- 172.21.15.82:0/603659644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec8100c10 msgr2=0x7fdec819d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:10:51.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:51.571+0000 7fdece7df700 1 --2- 172.21.15.82:0/603659644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec8100c10 0x7fdec819d950 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdec000add0 tx=0x7fdec000c330 comp rx=0 tx=0).stop 2023-09-09T23:10:51.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:51.572+0000 7fdece7df700 1 -- 172.21.15.82:0/603659644 shutdown_connections 2023-09-09T23:10:51.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:51.572+0000 7fdece7df700 1 --2- 172.21.15.82:0/603659644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdec8100c10 0x7fdec819d950 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:10:51.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:51.572+0000 7fdece7df700 1 --2- 172.21.15.82:0/603659644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdec80ff7e0 0x7fdec8196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:10:51.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:51.572+0000 7fdece7df700 1 --2- 172.21.15.82:0/603659644 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fdeb00a0640 0x7fdeb00a2b00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:10:51.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:51.572+0000 7fdece7df700 1 --2- 172.21.15.82:0/603659644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdec81001a0 0x7fdec81968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:10:51.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:51.572+0000 7fdece7df700 1 -- 172.21.15.82:0/603659644 >> 172.21.15.82:0/603659644 conn(0x7fdec80fa2b0 msgr2=0x7fdec80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:10:51.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:51.572+0000 7fdece7df700 1 -- 172.21.15.82:0/603659644 shutdown_connections 2023-09-09T23:10:51.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:10:51.572+0000 7fdece7df700 1 -- 172.21.15.82:0/603659644 wait complete. 2023-09-09T23:10:51.953 DEBUG:teuthology.orchestra.run.smithi082:osd.9> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@osd.9.service 2023-09-09T23:10:51.955 INFO:tasks.cephadm:Deploying osd.10 on smithi098 with /dev/vg_nvme/lv_1... 2023-09-09T23:10:51.956 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-09-09T23:10:51.989 INFO:journalctl@ceph.osd.9.smithi082.stdout:-- Logs begin at Sat 2023-09-09 22:51:24 UTC. -- 2023-09-09T23:10:52.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:52 smithi098 ceph-mon[103124]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-09T23:10:52.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:52.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:52.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:52.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:52.390 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:52 smithi082 ceph-mon[98694]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-09T23:10:52.390 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:52 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:52.391 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:52 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:52.391 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:52 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:52.391 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:52 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:52.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:52 smithi170 ceph-mon[102832]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-09T23:10:52.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:52 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:52.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:52 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:52.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:52 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:52.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:52 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:52.520 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-09T23:10:53.276 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:53 smithi098 ceph-mon[103124]: from='osd.9 [v2:172.21.15.82:6826/3354242311,v1:172.21.15.82:6827/3354242311]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-09T23:10:53.276 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:53 smithi098 ceph-mon[103124]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-09T23:10:53.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:53 smithi170 ceph-mon[102832]: from='osd.9 [v2:172.21.15.82:6826/3354242311,v1:172.21.15.82:6827/3354242311]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-09T23:10:53.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:53 smithi170 ceph-mon[102832]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-09T23:10:53.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:53 smithi082 ceph-mon[98694]: from='osd.9 [v2:172.21.15.82:6826/3354242311,v1:172.21.15.82:6827/3354242311]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-09T23:10:53.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:53 smithi082 ceph-mon[98694]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-09T23:10:53.455 INFO:journalctl@ceph.osd.9.smithi082.stdout:Sep 09 23:10:53 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-9[146623]: 2023-09-09T23:10:53.073+0000 7fe8c2add640 -1 osd.9 0 log_to_monitors true 2023-09-09T23:10:54.032 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-09-09T23:10:54.032 INFO:teuthology.orchestra.run.smithi098.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-09-09T23:10:54.032 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10+0 records in 2023-09-09T23:10:54.033 INFO:teuthology.orchestra.run.smithi098.stderr:10+0 records out 2023-09-09T23:10:54.033 INFO:teuthology.orchestra.run.smithi098.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111709 s, 939 MB/s 2023-09-09T23:10:54.033 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping successful for: 2023-09-09T23:10:54.205 INFO:journalctl@ceph.osd.9.smithi082.stdout:Sep 09 23:10:54 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-9[146623]: 2023-09-09T23:10:54.149+0000 7fe8b7953700 -1 osd.9 0 waiting for initial osdmap 2023-09-09T23:10:54.206 INFO:journalctl@ceph.osd.9.smithi082.stdout:Sep 09 23:10:54 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-9[146623]: 2023-09-09T23:10:54.161+0000 7fe8b1f45700 -1 osd.9 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-09T23:10:54.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:54 smithi082 ceph-mon[98694]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-09T23:10:54.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:54 smithi082 ceph-mon[98694]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-09-09T23:10:54.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:54 smithi082 ceph-mon[98694]: osdmap e62: 10 total, 9 up, 10 in 2023-09-09T23:10:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:54 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-09T23:10:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:54 smithi082 ceph-mon[98694]: from='osd.9 [v2:172.21.15.82:6826/3354242311,v1:172.21.15.82:6827/3354242311]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-09-09T23:10:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:54 smithi082 ceph-mon[98694]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-09-09T23:10:54.379 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:54 smithi098 ceph-mon[103124]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-09T23:10:54.380 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:54 smithi098 ceph-mon[103124]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-09-09T23:10:54.380 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:54 smithi098 ceph-mon[103124]: osdmap e62: 10 total, 9 up, 10 in 2023-09-09T23:10:54.380 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:54 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-09T23:10:54.380 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:54 smithi098 ceph-mon[103124]: from='osd.9 [v2:172.21.15.82:6826/3354242311,v1:172.21.15.82:6827/3354242311]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-09-09T23:10:54.380 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:54 smithi098 ceph-mon[103124]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-09-09T23:10:54.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:54 smithi170 ceph-mon[102832]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-09T23:10:54.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:54 smithi170 ceph-mon[102832]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-09-09T23:10:54.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:54 smithi170 ceph-mon[102832]: osdmap e62: 10 total, 9 up, 10 in 2023-09-09T23:10:54.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:54 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-09T23:10:54.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:54 smithi170 ceph-mon[102832]: from='osd.9 [v2:172.21.15.82:6826/3354242311,v1:172.21.15.82:6827/3354242311]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-09-09T23:10:54.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:54 smithi170 ceph-mon[102832]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-09-09T23:10:54.464 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi098:vg_nvme/lv_1 2023-09-09T23:10:55.018 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.b/config 2023-09-09T23:10:55.407 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:55 smithi098 ceph-mon[103124]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-09-09T23:10:55.408 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:55 smithi098 ceph-mon[103124]: osdmap e63: 10 total, 9 up, 10 in 2023-09-09T23:10:55.408 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:55 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-09T23:10:55.408 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:55 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-09T23:10:55.408 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:55 smithi098 ceph-mon[103124]: Detected new or changed devices on smithi082 2023-09-09T23:10:55.409 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:55 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:55.409 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:55 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:55.409 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:55 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:55.409 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:55 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:55.410 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:55 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:55.410 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:55 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:55.410 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:55 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:55.410 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:55 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:55.411 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:55 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:10:55.411 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:55 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:55.411 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:55 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:10:55.411 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:55 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:55.411 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:55 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:10:55.412 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:55 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:55.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:55 smithi170 ceph-mon[102832]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-09-09T23:10:55.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:55 smithi170 ceph-mon[102832]: osdmap e63: 10 total, 9 up, 10 in 2023-09-09T23:10:55.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:55 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-09T23:10:55.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:55 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-09T23:10:55.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:55 smithi170 ceph-mon[102832]: Detected new or changed devices on smithi082 2023-09-09T23:10:55.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:55 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:55 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:55 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:55 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:55 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:55 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:55 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:55 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:55 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:10:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:55 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:55.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:55 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:10:55.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:55 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:55.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:55 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:10:55.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:55 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:55.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:55 smithi082 ceph-mon[98694]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-09-09T23:10:55.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:55 smithi082 ceph-mon[98694]: osdmap e63: 10 total, 9 up, 10 in 2023-09-09T23:10:55.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:55 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-09T23:10:55.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:55 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-09T23:10:55.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:55 smithi082 ceph-mon[98694]: Detected new or changed devices on smithi082 2023-09-09T23:10:55.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:55 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:55 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:55 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:55 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:55 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:55 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:10:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:55 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:55 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:55 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:10:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:55 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:55 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:10:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:55 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:55.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:55 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:10:55.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:55 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:10:56.226 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.222+0000 7fec92ea9700 1 -- 172.21.15.98:0/733172416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec8c1063c0 msgr2=0x7fec8c1087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:10:56.226 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.222+0000 7fec92ea9700 1 --2- 172.21.15.98:0/733172416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec8c1063c0 0x7fec8c1087f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fec8400b0a0 tx=0x7fec8400b3b0 comp rx=0 tx=0).stop 2023-09-09T23:10:56.227 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.222+0000 7fec92ea9700 1 -- 172.21.15.98:0/733172416 shutdown_connections 2023-09-09T23:10:56.227 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.222+0000 7fec92ea9700 1 --2- 172.21.15.98:0/733172416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec8c1063c0 0x7fec8c1087f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:10:56.227 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.222+0000 7fec92ea9700 1 --2- 172.21.15.98:0/733172416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec8c101160 0x7fec8c103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:10:56.227 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.222+0000 7fec92ea9700 1 --2- 172.21.15.98:0/733172416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec8c103a90 0x7fec8c105e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:10:56.227 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.222+0000 7fec92ea9700 1 -- 172.21.15.98:0/733172416 >> 172.21.15.98:0/733172416 conn(0x7fec8c0fab10 msgr2=0x7fec8c0fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:10:56.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.222+0000 7fec92ea9700 1 -- 172.21.15.98:0/733172416 shutdown_connections 2023-09-09T23:10:56.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.222+0000 7fec92ea9700 1 -- 172.21.15.98:0/733172416 wait complete. 2023-09-09T23:10:56.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.223+0000 7fec92ea9700 1 Processor -- start 2023-09-09T23:10:56.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.223+0000 7fec92ea9700 1 -- start start 2023-09-09T23:10:56.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.223+0000 7fec92ea9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec8c101160 0x7fec8c196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:10:56.228 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.223+0000 7fec92ea9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec8c103a90 0x7fec8c1969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:10:56.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.223+0000 7fec92ea9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec8c1063c0 0x7fec8c19da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:10:56.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.223+0000 7fec92ea9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c19e000 con 0x7fec8c103a90 2023-09-09T23:10:56.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.223+0000 7fec92ea9700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c19e140 con 0x7fec8c1063c0 2023-09-09T23:10:56.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.223+0000 7fec92ea9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c19e2d0 con 0x7fec8c101160 2023-09-09T23:10:56.229 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.223+0000 7fec91446700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec8c1063c0 0x7fec8c19da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:10:56.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.223+0000 7fec91446700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec8c1063c0 0x7fec8c19da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40964/0 (socket says 172.21.15.98:40964) 2023-09-09T23:10:56.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.223+0000 7fec91446700 1 -- 172.21.15.98:0/80995412 learned_addr learned my addr 172.21.15.98:0/80995412 (peer_addr_for_me v2:172.21.15.98:0/0) 2023-09-09T23:10:56.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.224+0000 7fec90c45700 1 --2- 172.21.15.98:0/80995412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec8c101160 0x7fec8c196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:10:56.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.224+0000 7fec8bfff700 1 --2- 172.21.15.98:0/80995412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec8c103a90 0x7fec8c1969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:10:56.230 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.224+0000 7fec91446700 1 -- 172.21.15.98:0/80995412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec8c101160 msgr2=0x7fec8c196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:10:56.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.224+0000 7fec91446700 1 --2- 172.21.15.98:0/80995412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec8c101160 0x7fec8c196470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:10:56.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.224+0000 7fec91446700 1 -- 172.21.15.98:0/80995412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec8c103a90 msgr2=0x7fec8c1969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:10:56.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.224+0000 7fec91446700 1 --2- 172.21.15.98:0/80995412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec8c103a90 0x7fec8c1969b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:10:56.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.224+0000 7fec91446700 1 -- 172.21.15.98:0/80995412 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec84009d00 con 0x7fec8c1063c0 2023-09-09T23:10:56.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.224+0000 7fec90c45700 1 --2- 172.21.15.98:0/80995412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec8c101160 0x7fec8c196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:10:56.231 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.224+0000 7fec91446700 1 --2- 172.21.15.98:0/80995412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec8c1063c0 0x7fec8c19da30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fec8400b830 tx=0x7fec84004b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:10:56.232 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.224+0000 7fec89ffb700 1 -- 172.21.15.98:0/80995412 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8400e040 con 0x7fec8c1063c0 2023-09-09T23:10:56.232 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.224+0000 7fec89ffb700 1 -- 172.21.15.98:0/80995412 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec8401db40 con 0x7fec8c1063c0 2023-09-09T23:10:56.232 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.224+0000 7fec92ea9700 1 -- 172.21.15.98:0/80995412 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec8c19e550 con 0x7fec8c1063c0 2023-09-09T23:10:56.232 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.224+0000 7fec89ffb700 1 -- 172.21.15.98:0/80995412 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8401bd40 con 0x7fec8c1063c0 2023-09-09T23:10:56.232 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.224+0000 7fec92ea9700 1 -- 172.21.15.98:0/80995412 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec8c19ea90 con 0x7fec8c1063c0 2023-09-09T23:10:56.233 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.225+0000 7fec92ea9700 1 -- 172.21.15.98:0/80995412 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec8c0fc690 con 0x7fec8c1063c0 2023-09-09T23:10:56.233 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.226+0000 7fec89ffb700 1 -- 172.21.15.98:0/80995412 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fec8401dcb0 con 0x7fec8c1063c0 2023-09-09T23:10:56.233 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.227+0000 7fec89ffb700 1 --2- 172.21.15.98:0/80995412 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fec7c0a06c0 0x7fec7c0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:10:56.233 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.227+0000 7fec89ffb700 1 -- 172.21.15.98:0/80995412 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fec840cd7a0 con 0x7fec8c1063c0 2023-09-09T23:10:56.233 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.227+0000 7fec90c45700 1 --2- 172.21.15.98:0/80995412 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fec7c0a06c0 0x7fec7c0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:10:56.234 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.228+0000 7fec90c45700 1 --2- 172.21.15.98:0/80995412 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fec7c0a06c0 0x7fec7c0a2b80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fec80005fd0 tx=0x7fec80005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:10:56.234 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.231+0000 7fec89ffb700 1 -- 172.21.15.98:0/80995412 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fec84097b60 con 0x7fec8c1063c0 2023-09-09T23:10:56.425 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:10:56.426+0000 7fec92ea9700 1 -- 172.21.15.98:0/80995412 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fec8c02d030 con 0x7fec7c0a06c0 2023-09-09T23:10:56.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:56 smithi170 ceph-mon[102832]: purged_snaps scrub starts 2023-09-09T23:10:56.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:56 smithi170 ceph-mon[102832]: purged_snaps scrub ok 2023-09-09T23:10:56.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:56 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 4292M 2023-09-09T23:10:56.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:56 smithi170 ceph-mon[102832]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-09T23:10:56.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:56 smithi170 ceph-mon[102832]: osd.9 [v2:172.21.15.82:6826/3354242311,v1:172.21.15.82:6827/3354242311] boot 2023-09-09T23:10:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:56 smithi170 ceph-mon[102832]: osdmap e64: 10 total, 10 up, 10 in 2023-09-09T23:10:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:56 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-09T23:10:56.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:56 smithi082 ceph-mon[98694]: purged_snaps scrub starts 2023-09-09T23:10:56.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:56 smithi082 ceph-mon[98694]: purged_snaps scrub ok 2023-09-09T23:10:56.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:56 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 4292M 2023-09-09T23:10:56.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:56 smithi082 ceph-mon[98694]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-09T23:10:56.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:56 smithi082 ceph-mon[98694]: osd.9 [v2:172.21.15.82:6826/3354242311,v1:172.21.15.82:6827/3354242311] boot 2023-09-09T23:10:56.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:56 smithi082 ceph-mon[98694]: osdmap e64: 10 total, 10 up, 10 in 2023-09-09T23:10:56.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:56 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-09T23:10:56.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:56 smithi098 ceph-mon[103124]: purged_snaps scrub starts 2023-09-09T23:10:56.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:56 smithi098 ceph-mon[103124]: purged_snaps scrub ok 2023-09-09T23:10:56.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:56 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 4292M 2023-09-09T23:10:56.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:56 smithi098 ceph-mon[103124]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-09T23:10:56.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:56 smithi098 ceph-mon[103124]: osd.9 [v2:172.21.15.82:6826/3354242311,v1:172.21.15.82:6827/3354242311] boot 2023-09-09T23:10:56.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:56 smithi098 ceph-mon[103124]: osdmap e64: 10 total, 10 up, 10 in 2023-09-09T23:10:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:56 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-09T23:10:57.423 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:57 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:10:57.423 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:57 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:10:57.423 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:57 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:57.424 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:57 smithi098 ceph-mon[103124]: osdmap e65: 10 total, 10 up, 10 in 2023-09-09T23:10:57.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:57 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:10:57.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:57 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:10:57.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:57 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:57.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:57 smithi170 ceph-mon[102832]: osdmap e65: 10 total, 10 up, 10 in 2023-09-09T23:10:57.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:57 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:10:57.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:57 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:10:57.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:57 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:10:57.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:57 smithi082 ceph-mon[98694]: osdmap e65: 10 total, 10 up, 10 in 2023-09-09T23:10:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:58 smithi170 ceph-mon[102832]: from='client.24416 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:10:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:58 smithi170 ceph-mon[102832]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:10:58.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:58 smithi082 ceph-mon[98694]: from='client.24416 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:10:58.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:58 smithi082 ceph-mon[98694]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:10:58.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:58 smithi098 ceph-mon[103124]: from='client.24416 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:10:58.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:58 smithi098 ceph-mon[103124]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:10:59.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:59 smithi098 ceph-mon[103124]: from='client.? 172.21.15.98:0/4031434256' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33a59389-7ced-42d5-bf54-2071317132d8"}]: dispatch 2023-09-09T23:10:59.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:59 smithi098 ceph-mon[103124]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33a59389-7ced-42d5-bf54-2071317132d8"}]: dispatch 2023-09-09T23:10:59.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:59 smithi098 ceph-mon[103124]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33a59389-7ced-42d5-bf54-2071317132d8"}]': finished 2023-09-09T23:10:59.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:59 smithi098 ceph-mon[103124]: osdmap e66: 11 total, 10 up, 11 in 2023-09-09T23:10:59.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:10:59 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-09T23:10:59.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:59 smithi170 ceph-mon[102832]: from='client.? 172.21.15.98:0/4031434256' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33a59389-7ced-42d5-bf54-2071317132d8"}]: dispatch 2023-09-09T23:10:59.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:59 smithi170 ceph-mon[102832]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33a59389-7ced-42d5-bf54-2071317132d8"}]: dispatch 2023-09-09T23:10:59.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:59 smithi170 ceph-mon[102832]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33a59389-7ced-42d5-bf54-2071317132d8"}]': finished 2023-09-09T23:10:59.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:59 smithi170 ceph-mon[102832]: osdmap e66: 11 total, 10 up, 11 in 2023-09-09T23:10:59.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:10:59 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-09T23:10:59.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:59 smithi082 ceph-mon[98694]: from='client.? 172.21.15.98:0/4031434256' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33a59389-7ced-42d5-bf54-2071317132d8"}]: dispatch 2023-09-09T23:10:59.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:59 smithi082 ceph-mon[98694]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33a59389-7ced-42d5-bf54-2071317132d8"}]: dispatch 2023-09-09T23:10:59.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:59 smithi082 ceph-mon[98694]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33a59389-7ced-42d5-bf54-2071317132d8"}]': finished 2023-09-09T23:10:59.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:59 smithi082 ceph-mon[98694]: osdmap e66: 11 total, 10 up, 11 in 2023-09-09T23:10:59.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:10:59 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-09T23:11:00.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:00 smithi098 ceph-mon[103124]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:11:00.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:00 smithi098 ceph-mon[103124]: from='client.? 172.21.15.98:0/4279108121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:11:00.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:00 smithi170 ceph-mon[102832]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:11:00.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:00 smithi170 ceph-mon[102832]: from='client.? 172.21.15.98:0/4279108121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:11:00.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:00 smithi082 ceph-mon[98694]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:11:00.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:00 smithi082 ceph-mon[98694]: from='client.? 172.21.15.98:0/4279108121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:11:02.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:02 smithi098 ceph-mon[103124]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:11:02.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:02 smithi170 ceph-mon[102832]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:11:02.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:02 smithi082 ceph-mon[98694]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:11:04.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:04 smithi098 ceph-mon[103124]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:11:05.556 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:04 smithi082 ceph-mon[98694]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:11:05.556 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:04 smithi170 ceph-mon[102832]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:11:06.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:06 smithi098 ceph-mon[103124]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:11:06.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:06 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-09-09T23:11:06.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:06 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:06.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:06 smithi170 ceph-mon[102832]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:11:06.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:06 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-09-09T23:11:06.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:06 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:06.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:06 smithi082 ceph-mon[98694]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:11:06.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:06 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-09-09T23:11:06.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:06 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:07.398 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:07 smithi098 ceph-mon[103124]: Deploying daemon osd.10 on smithi098 2023-09-09T23:11:07.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:07 smithi170 ceph-mon[102832]: Deploying daemon osd.10 on smithi098 2023-09-09T23:11:07.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:07 smithi082 ceph-mon[98694]: Deploying daemon osd.10 on smithi098 2023-09-09T23:11:08.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:08 smithi098 ceph-mon[103124]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:11:08.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:08 smithi170 ceph-mon[102832]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:11:08.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:08 smithi082 ceph-mon[98694]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:11:10.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:10 smithi170 ceph-mon[102832]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:11:10.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:10 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:10.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:10 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:10.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:10 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:11:10.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:10 smithi082 ceph-mon[98694]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:11:10.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:10 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:10.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:10 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:10.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:10 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:11:10.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:10 smithi098 ceph-mon[103124]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:11:10.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:10 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:10.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:10 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:10.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:10 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:11:12.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:12 smithi170 ceph-mon[102832]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:11:12.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:12 smithi082 ceph-mon[98694]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:11:12.736 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:12 smithi098 ceph-mon[103124]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:11:12.784 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:11:12.787+0000 7fec89ffb700 1 -- 172.21.15.98:0/80995412 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fec8c02d030 con 0x7fec7c0a06c0 2023-09-09T23:11:12.785 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 10 on host 'smithi098' 2023-09-09T23:11:12.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:11:12.789+0000 7fec92ea9700 1 -- 172.21.15.98:0/80995412 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fec7c0a06c0 msgr2=0x7fec7c0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:12.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:11:12.789+0000 7fec92ea9700 1 --2- 172.21.15.98:0/80995412 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fec7c0a06c0 0x7fec7c0a2b80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fec80005fd0 tx=0x7fec80005e20 comp rx=0 tx=0).stop 2023-09-09T23:11:12.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:11:12.789+0000 7fec92ea9700 1 -- 172.21.15.98:0/80995412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec8c1063c0 msgr2=0x7fec8c19da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:12.787 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:11:12.789+0000 7fec92ea9700 1 --2- 172.21.15.98:0/80995412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec8c1063c0 0x7fec8c19da30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fec8400b830 tx=0x7fec84004b00 comp rx=0 tx=0).stop 2023-09-09T23:11:12.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:11:12.789+0000 7fec92ea9700 1 -- 172.21.15.98:0/80995412 shutdown_connections 2023-09-09T23:11:12.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:11:12.789+0000 7fec92ea9700 1 --2- 172.21.15.98:0/80995412 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fec7c0a06c0 0x7fec7c0a2b80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:12.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:11:12.789+0000 7fec92ea9700 1 --2- 172.21.15.98:0/80995412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec8c1063c0 0x7fec8c19da30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:12.788 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:11:12.789+0000 7fec92ea9700 1 --2- 172.21.15.98:0/80995412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec8c101160 0x7fec8c196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:12.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:11:12.789+0000 7fec92ea9700 1 --2- 172.21.15.98:0/80995412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec8c103a90 0x7fec8c1969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:12.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:11:12.789+0000 7fec92ea9700 1 -- 172.21.15.98:0/80995412 >> 172.21.15.98:0/80995412 conn(0x7fec8c0fab10 msgr2=0x7fec8c101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:12.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:11:12.789+0000 7fec92ea9700 1 -- 172.21.15.98:0/80995412 shutdown_connections 2023-09-09T23:11:12.789 INFO:teuthology.orchestra.run.smithi098.stderr:2023-09-09T23:11:12.789+0000 7fec92ea9700 1 -- 172.21.15.98:0/80995412 wait complete. 2023-09-09T23:11:13.216 DEBUG:teuthology.orchestra.run.smithi098:osd.10> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@osd.10.service 2023-09-09T23:11:13.218 INFO:tasks.cephadm:Deploying osd.11 on smithi170 with /dev/vg_nvme/lv_1... 2023-09-09T23:11:13.219 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-09-09T23:11:13.253 INFO:journalctl@ceph.osd.10.smithi098.stdout:-- Logs begin at Sat 2023-09-09 22:51:28 UTC. -- 2023-09-09T23:11:13.785 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-09T23:11:14.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:13 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:14.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:13 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:14.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:13 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:14.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:13 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:14.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:13 smithi098 ceph-mon[103124]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:11:14.033 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:13 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:14.033 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:13 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:14.034 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:13 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:14.034 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:13 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:14.034 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:13 smithi170 ceph-mon[102832]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:11:14.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:13 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:14.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:13 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:14.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:13 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:14.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:13 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:14.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:13 smithi082 ceph-mon[98694]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:11:14.273 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 09 23:11:14 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-10[132273]: 2023-09-09T23:11:14.004+0000 7f78f07e5640 -1 osd.10 0 log_to_monitors true 2023-09-09T23:11:15.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:14 smithi170 ceph-mon[102832]: from='osd.10 [v2:172.21.15.98:6824/2069114959,v1:172.21.15.98:6825/2069114959]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-09T23:11:15.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:14 smithi170 ceph-mon[102832]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-09T23:11:15.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:14 smithi082 ceph-mon[98694]: from='osd.10 [v2:172.21.15.98:6824/2069114959,v1:172.21.15.98:6825/2069114959]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-09T23:11:15.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:14 smithi082 ceph-mon[98694]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-09T23:11:15.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:14 smithi098 ceph-mon[103124]: from='osd.10 [v2:172.21.15.98:6824/2069114959,v1:172.21.15.98:6825/2069114959]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-09T23:11:15.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:14 smithi098 ceph-mon[103124]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-09T23:11:15.267 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-09-09T23:11:15.267 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-09-09T23:11:15.268 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2023-09-09T23:11:15.268 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2023-09-09T23:11:15.268 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00863891 s, 1.2 GB/s 2023-09-09T23:11:15.268 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2023-09-09T23:11:15.638 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph orch daemon add osd smithi170:vg_nvme/lv_1 2023-09-09T23:11:16.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:15 smithi170 ceph-mon[102832]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-09-09T23:11:16.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:15 smithi170 ceph-mon[102832]: osdmap e67: 11 total, 10 up, 11 in 2023-09-09T23:11:16.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:15 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-09T23:11:16.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:15 smithi170 ceph-mon[102832]: from='osd.10 [v2:172.21.15.98:6824/2069114959,v1:172.21.15.98:6825/2069114959]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-09T23:11:16.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:15 smithi170 ceph-mon[102832]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-09T23:11:16.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:15 smithi170 ceph-mon[102832]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:11:16.192 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-09T23:11:16.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:15 smithi082 ceph-mon[98694]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-09-09T23:11:16.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:15 smithi082 ceph-mon[98694]: osdmap e67: 11 total, 10 up, 11 in 2023-09-09T23:11:16.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:15 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-09T23:11:16.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:15 smithi082 ceph-mon[98694]: from='osd.10 [v2:172.21.15.98:6824/2069114959,v1:172.21.15.98:6825/2069114959]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-09T23:11:16.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:15 smithi082 ceph-mon[98694]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-09T23:11:16.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:15 smithi082 ceph-mon[98694]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:11:16.251 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 09 23:11:15 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-10[132273]: 2023-09-09T23:11:15.921+0000 7f78e565b700 -1 osd.10 0 waiting for initial osdmap 2023-09-09T23:11:16.252 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 09 23:11:15 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-10[132273]: 2023-09-09T23:11:15.938+0000 7f78dfc4d700 -1 osd.10 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-09T23:11:16.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:15 smithi098 ceph-mon[103124]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-09-09T23:11:16.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:15 smithi098 ceph-mon[103124]: osdmap e67: 11 total, 10 up, 11 in 2023-09-09T23:11:16.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:15 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-09T23:11:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:15 smithi098 ceph-mon[103124]: from='osd.10 [v2:172.21.15.98:6824/2069114959,v1:172.21.15.98:6825/2069114959]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-09T23:11:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:15 smithi098 ceph-mon[103124]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2023-09-09T23:11:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:15 smithi098 ceph-mon[103124]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-09T23:11:17.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:16 smithi170 ceph-mon[102832]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-09-09T23:11:17.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:16 smithi170 ceph-mon[102832]: osdmap e68: 11 total, 10 up, 11 in 2023-09-09T23:11:17.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:16 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-09T23:11:17.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:16 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-09T23:11:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:16 smithi170 ceph-mon[102832]: Detected new or changed devices on smithi098 2023-09-09T23:11:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:16 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:16 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:16 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:11:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:16 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:11:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:16 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:11:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:16 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:11:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:16 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 4292M 2023-09-09T23:11:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:16 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:16 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:17.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:16 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:11:17.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:16 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:17.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:16 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:11:17.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:16 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:17.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:16 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:11:17.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:16 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:17.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:16 smithi082 ceph-mon[98694]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-09-09T23:11:17.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:16 smithi082 ceph-mon[98694]: osdmap e68: 11 total, 10 up, 11 in 2023-09-09T23:11:17.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:16 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-09T23:11:17.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:16 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-09T23:11:17.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:16 smithi082 ceph-mon[98694]: Detected new or changed devices on smithi098 2023-09-09T23:11:17.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:16 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:17.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:16 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:17.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:16 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:11:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:16 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:11:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:16 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:11:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:16 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:11:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:16 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 4292M 2023-09-09T23:11:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:16 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:16 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:16 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:11:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:16 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:16 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:11:17.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:16 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:17.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:16 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:11:17.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:16 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:17.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:16 smithi098 ceph-mon[103124]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2023-09-09T23:11:17.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:16 smithi098 ceph-mon[103124]: osdmap e68: 11 total, 10 up, 11 in 2023-09-09T23:11:17.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:16 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-09T23:11:17.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:16 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-09T23:11:17.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:16 smithi098 ceph-mon[103124]: Detected new or changed devices on smithi098 2023-09-09T23:11:17.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:16 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:16 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:16 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:11:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:16 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:11:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:16 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:11:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:16 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:11:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:16 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 4292M 2023-09-09T23:11:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:16 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:16 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:16 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:11:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:16 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:16 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:11:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:16 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:16 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:11:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:16 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:17.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.408+0000 7fa4a2d02700 1 -- 172.21.15.170:0/3346057825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa49c102390 msgr2=0x7fa49c108c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:17.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.408+0000 7fa4a2d02700 1 --2- 172.21.15.170:0/3346057825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa49c102390 0x7fa49c108c40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa49400b0a0 tx=0x7fa49400b3b0 comp rx=0 tx=0).stop 2023-09-09T23:11:17.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.408+0000 7fa4a2d02700 1 -- 172.21.15.170:0/3346057825 shutdown_connections 2023-09-09T23:11:17.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.408+0000 7fa4a2d02700 1 --2- 172.21.15.170:0/3346057825 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa49c0fe970 0x7fa49c0fed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:17.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.408+0000 7fa4a2d02700 1 --2- 172.21.15.170:0/3346057825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa49c102390 0x7fa49c108c40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:17.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.408+0000 7fa4a2d02700 1 --2- 172.21.15.170:0/3346057825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa49c1019f0 0x7fa49c101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:17.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.408+0000 7fa4a2d02700 1 -- 172.21.15.170:0/3346057825 >> 172.21.15.170:0/3346057825 conn(0x7fa49c0fa2b0 msgr2=0x7fa49c0fc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:17.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.408+0000 7fa4a2d02700 1 -- 172.21.15.170:0/3346057825 shutdown_connections 2023-09-09T23:11:17.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.408+0000 7fa4a2d02700 1 -- 172.21.15.170:0/3346057825 wait complete. 2023-09-09T23:11:17.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.409+0000 7fa4a2d02700 1 Processor -- start 2023-09-09T23:11:17.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.409+0000 7fa4a2d02700 1 -- start start 2023-09-09T23:11:17.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.409+0000 7fa4a2d02700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa49c0fe970 0x7fa49c101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:17.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.409+0000 7fa4a2d02700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa49c1019f0 0x7fa49c0ff9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:17.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.409+0000 7fa4a2d02700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa49c102390 0x7fa49c0ffee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:17.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.409+0000 7fa4a2d02700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c100660 con 0x7fa49c1019f0 2023-09-09T23:11:17.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.409+0000 7fa4a2d02700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c1007d0 con 0x7fa49c102390 2023-09-09T23:11:17.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.409+0000 7fa4a2d02700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c1a0420 con 0x7fa49c0fe970 2023-09-09T23:11:17.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.409+0000 7fa4a0a9e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa49c0fe970 0x7fa49c101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:17.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.409+0000 7fa4a0a9e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa49c0fe970 0x7fa49c101350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34726/0 (socket says 172.21.15.170:34726) 2023-09-09T23:11:17.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.409+0000 7fa4a0a9e700 1 -- 172.21.15.170:0/1831056136 learned_addr learned my addr 172.21.15.170:0/1831056136 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-09-09T23:11:17.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.409+0000 7fa4a129f700 1 --2- 172.21.15.170:0/1831056136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa49c102390 0x7fa49c0ffee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:17.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.409+0000 7fa4a0a9e700 1 -- 172.21.15.170:0/1831056136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa49c102390 msgr2=0x7fa49c0ffee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:17.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.409+0000 7fa4a0a9e700 1 --2- 172.21.15.170:0/1831056136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa49c102390 0x7fa49c0ffee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:17.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.409+0000 7fa4a0a9e700 1 -- 172.21.15.170:0/1831056136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa49c1019f0 msgr2=0x7fa49c0ff9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:17.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.409+0000 7fa4a0a9e700 1 --2- 172.21.15.170:0/1831056136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa49c1019f0 0x7fa49c0ff9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:17.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.409+0000 7fa4a0a9e700 1 -- 172.21.15.170:0/1831056136 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa494009d00 con 0x7fa49c0fe970 2023-09-09T23:11:17.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.409+0000 7fa4a129f700 1 --2- 172.21.15.170:0/1831056136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa49c102390 0x7fa49c0ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:11:17.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.409+0000 7fa4a0a9e700 1 --2- 172.21.15.170:0/1831056136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa49c0fe970 0x7fa49c101350 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa49000d980 tx=0x7fa49000dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:11:17.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.409+0000 7fa499ffb700 1 -- 172.21.15.170:0/1831056136 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49000de60 con 0x7fa49c0fe970 2023-09-09T23:11:17.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.409+0000 7fa499ffb700 1 -- 172.21.15.170:0/1831056136 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4900044e0 con 0x7fa49c0fe970 2023-09-09T23:11:17.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.410+0000 7fa4a2d02700 1 -- 172.21.15.170:0/1831056136 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa49c1a0700 con 0x7fa49c0fe970 2023-09-09T23:11:17.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.410+0000 7fa499ffb700 1 -- 172.21.15.170:0/1831056136 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa490003be0 con 0x7fa49c0fe970 2023-09-09T23:11:17.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.410+0000 7fa4a2d02700 1 -- 172.21.15.170:0/1831056136 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49c1a0c50 con 0x7fa49c0fe970 2023-09-09T23:11:17.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.411+0000 7fa4a2d02700 1 -- 172.21.15.170:0/1831056136 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa49c1076e0 con 0x7fa49c0fe970 2023-09-09T23:11:17.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.412+0000 7fa499ffb700 1 -- 172.21.15.170:0/1831056136 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fa490009bb0 con 0x7fa49c0fe970 2023-09-09T23:11:17.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.412+0000 7fa499ffb700 1 --2- 172.21.15.170:0/1831056136 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa48c0a06c0 0x7fa48c0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:17.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.412+0000 7fa499ffb700 1 -- 172.21.15.170:0/1831056136 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6727+0+0 (secure 0 0 0) 0x7fa4900cb300 con 0x7fa49c0fe970 2023-09-09T23:11:17.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.413+0000 7fa49bfff700 1 --2- 172.21.15.170:0/1831056136 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa48c0a06c0 0x7fa48c0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:17.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.413+0000 7fa49bfff700 1 --2- 172.21.15.170:0/1831056136 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa48c0a06c0 0x7fa48c0a2b80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa488005fd0 tx=0x7fa488005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:11:17.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.417+0000 7fa499ffb700 1 -- 172.21.15.170:0/1831056136 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa4900953c0 con 0x7fa49c0fe970 2023-09-09T23:11:17.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:17.612+0000 7fa4a2d02700 1 -- 172.21.15.170:0/1831056136 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa49c02d000 con 0x7fa48c0a06c0 2023-09-09T23:11:18.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:17 smithi082 ceph-mon[98694]: purged_snaps scrub starts 2023-09-09T23:11:18.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:17 smithi082 ceph-mon[98694]: purged_snaps scrub ok 2023-09-09T23:11:18.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:17 smithi082 ceph-mon[98694]: osd.10 [v2:172.21.15.98:6824/2069114959,v1:172.21.15.98:6825/2069114959] boot 2023-09-09T23:11:18.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:17 smithi082 ceph-mon[98694]: osdmap e69: 11 total, 11 up, 11 in 2023-09-09T23:11:18.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:17 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-09T23:11:18.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:17 smithi082 ceph-mon[98694]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-09T23:11:18.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:17 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:11:18.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:17 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:11:18.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:17 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:18.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:17 smithi098 ceph-mon[103124]: purged_snaps scrub starts 2023-09-09T23:11:18.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:17 smithi098 ceph-mon[103124]: purged_snaps scrub ok 2023-09-09T23:11:18.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:17 smithi098 ceph-mon[103124]: osd.10 [v2:172.21.15.98:6824/2069114959,v1:172.21.15.98:6825/2069114959] boot 2023-09-09T23:11:18.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:17 smithi098 ceph-mon[103124]: osdmap e69: 11 total, 11 up, 11 in 2023-09-09T23:11:18.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:17 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-09T23:11:18.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:17 smithi098 ceph-mon[103124]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-09T23:11:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:17 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:11:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:17 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:11:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:17 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:18.423 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:17 smithi170 ceph-mon[102832]: purged_snaps scrub starts 2023-09-09T23:11:18.424 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:17 smithi170 ceph-mon[102832]: purged_snaps scrub ok 2023-09-09T23:11:18.424 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:17 smithi170 ceph-mon[102832]: osd.10 [v2:172.21.15.98:6824/2069114959,v1:172.21.15.98:6825/2069114959] boot 2023-09-09T23:11:18.424 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:17 smithi170 ceph-mon[102832]: osdmap e69: 11 total, 11 up, 11 in 2023-09-09T23:11:18.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:17 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-09T23:11:18.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:17 smithi170 ceph-mon[102832]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-09T23:11:18.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:17 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-09T23:11:18.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:17 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-09T23:11:18.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:17 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:19.169 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:18 smithi170 ceph-mon[102832]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:11:19.169 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:18 smithi170 ceph-mon[102832]: osdmap e70: 11 total, 11 up, 11 in 2023-09-09T23:11:19.169 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:18 smithi170 ceph-mon[102832]: osdmap e71: 11 total, 11 up, 11 in 2023-09-09T23:11:19.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:18 smithi082 ceph-mon[98694]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:11:19.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:18 smithi082 ceph-mon[98694]: osdmap e70: 11 total, 11 up, 11 in 2023-09-09T23:11:19.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:18 smithi082 ceph-mon[98694]: osdmap e71: 11 total, 11 up, 11 in 2023-09-09T23:11:19.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:18 smithi098 ceph-mon[103124]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:11:19.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:18 smithi098 ceph-mon[103124]: osdmap e70: 11 total, 11 up, 11 in 2023-09-09T23:11:19.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:18 smithi098 ceph-mon[103124]: osdmap e71: 11 total, 11 up, 11 in 2023-09-09T23:11:20.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:19 smithi082 ceph-mon[98694]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-09T23:11:20.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:19 smithi098 ceph-mon[103124]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-09T23:11:20.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:19 smithi170 ceph-mon[102832]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-09-09T23:11:21.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:20 smithi082 ceph-mon[98694]: from='client.? 172.21.15.170:0/4137394900' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7a71f5c-85e2-4fec-be4b-95be0788b302"}]: dispatch 2023-09-09T23:11:21.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:20 smithi082 ceph-mon[98694]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7a71f5c-85e2-4fec-be4b-95be0788b302"}]: dispatch 2023-09-09T23:11:21.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:20 smithi082 ceph-mon[98694]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7a71f5c-85e2-4fec-be4b-95be0788b302"}]': finished 2023-09-09T23:11:21.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:20 smithi082 ceph-mon[98694]: osdmap e72: 12 total, 11 up, 12 in 2023-09-09T23:11:21.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:20 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-09T23:11:21.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:20 smithi082 ceph-mon[98694]: from='client.? 172.21.15.170:0/3584697819' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:11:21.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:20 smithi098 ceph-mon[103124]: from='client.? 172.21.15.170:0/4137394900' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7a71f5c-85e2-4fec-be4b-95be0788b302"}]: dispatch 2023-09-09T23:11:21.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:20 smithi098 ceph-mon[103124]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7a71f5c-85e2-4fec-be4b-95be0788b302"}]: dispatch 2023-09-09T23:11:21.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:20 smithi098 ceph-mon[103124]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7a71f5c-85e2-4fec-be4b-95be0788b302"}]': finished 2023-09-09T23:11:21.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:20 smithi098 ceph-mon[103124]: osdmap e72: 12 total, 11 up, 12 in 2023-09-09T23:11:21.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:20 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-09T23:11:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:20 smithi098 ceph-mon[103124]: from='client.? 172.21.15.170:0/3584697819' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:11:21.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:20 smithi170 ceph-mon[102832]: from='client.? 172.21.15.170:0/4137394900' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7a71f5c-85e2-4fec-be4b-95be0788b302"}]: dispatch 2023-09-09T23:11:21.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:20 smithi170 ceph-mon[102832]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7a71f5c-85e2-4fec-be4b-95be0788b302"}]: dispatch 2023-09-09T23:11:21.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:20 smithi170 ceph-mon[102832]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7a71f5c-85e2-4fec-be4b-95be0788b302"}]': finished 2023-09-09T23:11:21.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:20 smithi170 ceph-mon[102832]: osdmap e72: 12 total, 11 up, 12 in 2023-09-09T23:11:21.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:20 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-09T23:11:21.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:20 smithi170 ceph-mon[102832]: from='client.? 172.21.15.170:0/3584697819' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-09T23:11:22.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:21 smithi082 ceph-mon[98694]: osdmap e73: 12 total, 11 up, 12 in 2023-09-09T23:11:22.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:21 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-09T23:11:22.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:21 smithi082 ceph-mon[98694]: pgmap v219: 1 pgs: 1 peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-09T23:11:22.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:21 smithi098 ceph-mon[103124]: osdmap e73: 12 total, 11 up, 12 in 2023-09-09T23:11:22.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:21 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-09T23:11:22.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:21 smithi098 ceph-mon[103124]: pgmap v219: 1 pgs: 1 peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-09T23:11:22.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:21 smithi170 ceph-mon[102832]: osdmap e73: 12 total, 11 up, 12 in 2023-09-09T23:11:22.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:21 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-09T23:11:22.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:21 smithi170 ceph-mon[102832]: pgmap v219: 1 pgs: 1 peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-09T23:11:24.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:24 smithi098 ceph-mon[103124]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-09-09T23:11:24.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:24 smithi170 ceph-mon[102832]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-09-09T23:11:24.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:24 smithi082 ceph-mon[98694]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-09-09T23:11:26.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:26 smithi098 ceph-mon[103124]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-09-09T23:11:26.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:26 smithi170 ceph-mon[102832]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-09-09T23:11:26.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:26 smithi082 ceph-mon[98694]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-09-09T23:11:27.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:27 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-09-09T23:11:27.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:27 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:27.521 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:27 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-09-09T23:11:27.521 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:27 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:27.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:27 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-09-09T23:11:27.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:27 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:28.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:28 smithi098 ceph-mon[103124]: Deploying daemon osd.11 on smithi170 2023-09-09T23:11:28.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:28 smithi098 ceph-mon[103124]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-09T23:11:28.550 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:28 smithi170 ceph-mon[102832]: Deploying daemon osd.11 on smithi170 2023-09-09T23:11:28.550 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:28 smithi170 ceph-mon[102832]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-09T23:11:28.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:28 smithi082 ceph-mon[98694]: Deploying daemon osd.11 on smithi170 2023-09-09T23:11:28.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:28 smithi082 ceph-mon[98694]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-09T23:11:30.514 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:30 smithi170 ceph-mon[102832]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-09-09T23:11:30.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:30 smithi082 ceph-mon[98694]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-09-09T23:11:30.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:30 smithi098 ceph-mon[103124]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-09-09T23:11:31.800 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:31 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:31.800 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:31 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:31.800 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:31 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:11:31.801 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:31 smithi170 ceph-mon[102832]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-09T23:11:31.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:31 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:31.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:31 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:31.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:31 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:11:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:31 smithi082 ceph-mon[98694]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-09T23:11:32.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:31 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:32.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:31 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:32.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:31 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:11:32.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:31 smithi098 ceph-mon[103124]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-09T23:11:33.470 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:33.471+0000 7fa499ffb700 1 -- 172.21.15.170:0/1831056136 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fa49c02d000 con 0x7fa48c0a06c0 2023-09-09T23:11:33.471 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 11 on host 'smithi170' 2023-09-09T23:11:33.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:33.474+0000 7fa4a2d02700 1 -- 172.21.15.170:0/1831056136 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa48c0a06c0 msgr2=0x7fa48c0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:33.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:33.474+0000 7fa4a2d02700 1 --2- 172.21.15.170:0/1831056136 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa48c0a06c0 0x7fa48c0a2b80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa488005fd0 tx=0x7fa488005d00 comp rx=0 tx=0).stop 2023-09-09T23:11:33.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:33.474+0000 7fa4a2d02700 1 -- 172.21.15.170:0/1831056136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa49c0fe970 msgr2=0x7fa49c101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:33.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:33.474+0000 7fa4a2d02700 1 --2- 172.21.15.170:0/1831056136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa49c0fe970 0x7fa49c101350 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa49000d980 tx=0x7fa49000dd40 comp rx=0 tx=0).stop 2023-09-09T23:11:33.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:33.474+0000 7fa4a2d02700 1 -- 172.21.15.170:0/1831056136 shutdown_connections 2023-09-09T23:11:33.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:33.474+0000 7fa4a2d02700 1 --2- 172.21.15.170:0/1831056136 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa48c0a06c0 0x7fa48c0a2b80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:33.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:33.474+0000 7fa4a2d02700 1 --2- 172.21.15.170:0/1831056136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa49c102390 0x7fa49c0ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:33.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:33.474+0000 7fa4a2d02700 1 --2- 172.21.15.170:0/1831056136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa49c0fe970 0x7fa49c101350 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:33.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:33.474+0000 7fa4a2d02700 1 --2- 172.21.15.170:0/1831056136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa49c1019f0 0x7fa49c0ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:33.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:33.474+0000 7fa4a2d02700 1 -- 172.21.15.170:0/1831056136 >> 172.21.15.170:0/1831056136 conn(0x7fa49c0fa2b0 msgr2=0x7fa49c0fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:33.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:33.474+0000 7fa4a2d02700 1 -- 172.21.15.170:0/1831056136 shutdown_connections 2023-09-09T23:11:33.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:33.474+0000 7fa4a2d02700 1 -- 172.21.15.170:0/1831056136 wait complete. 2023-09-09T23:11:33.884 DEBUG:teuthology.orchestra.run.smithi170:osd.11> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@osd.11.service 2023-09-09T23:11:33.886 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-09-09T23:11:33.887 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph osd stat -f json 2023-09-09T23:11:33.918 INFO:journalctl@ceph.osd.11.smithi170.stdout:-- Logs begin at Sat 2023-09-09 22:51:11 UTC. -- 2023-09-09T23:11:34.455 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:11:34.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:34 smithi098 ceph-mon[103124]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-09T23:11:34.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:34 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:34.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:34 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:34.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:34 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:34.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:34 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:34.585 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:34 smithi170 ceph-mon[102832]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-09T23:11:34.585 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:34 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:34.586 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:34 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:34.586 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:34 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:34.586 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:34 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:34.646 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:34 smithi082 ceph-mon[98694]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-09T23:11:34.646 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:34 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:34.646 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:34 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:34.646 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:34 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:34.647 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:34 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:34.906 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 09 23:11:34 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-11[133461]: 2023-09-09T23:11:34.676+0000 7fd77e4fd640 -1 osd.11 0 log_to_monitors true 2023-09-09T23:11:35.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:35 smithi170 ceph-mon[102832]: from='osd.11 [v2:172.21.15.170:6824/146964512,v1:172.21.15.170:6825/146964512]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-09T23:11:35.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:35 smithi170 ceph-mon[102832]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-09T23:11:35.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:35 smithi098 ceph-mon[103124]: from='osd.11 [v2:172.21.15.170:6824/146964512,v1:172.21.15.170:6825/146964512]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-09T23:11:35.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:35 smithi098 ceph-mon[103124]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-09T23:11:35.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.584+0000 7fa2d7a44700 1 -- 172.21.15.82:0/2281677837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2d0101140 msgr2=0x7fa2d01079f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:35.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.584+0000 7fa2d7a44700 1 --2- 172.21.15.82:0/2281677837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2d0101140 0x7fa2d01079f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800b0d0 tx=0x7fa2c800b3e0 comp rx=0 tx=0).stop 2023-09-09T23:11:35.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.584+0000 7fa2d7a44700 1 -- 172.21.15.82:0/2281677837 shutdown_connections 2023-09-09T23:11:35.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.584+0000 7fa2d7a44700 1 --2- 172.21.15.82:0/2281677837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2d0101140 0x7fa2d01079f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:35.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.584+0000 7fa2d7a44700 1 --2- 172.21.15.82:0/2281677837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2d00ff5a0 0x7fa2d00ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:35.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.584+0000 7fa2d7a44700 1 --2- 172.21.15.82:0/2281677837 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2d01007a0 0x7fa2d0100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:35.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.584+0000 7fa2d7a44700 1 -- 172.21.15.82:0/2281677837 >> 172.21.15.82:0/2281677837 conn(0x7fa2d00fab40 msgr2=0x7fa2d00fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:35.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.584+0000 7fa2d7a44700 1 -- 172.21.15.82:0/2281677837 shutdown_connections 2023-09-09T23:11:35.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.584+0000 7fa2d7a44700 1 -- 172.21.15.82:0/2281677837 wait complete. 2023-09-09T23:11:35.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.585+0000 7fa2d7a44700 1 Processor -- start 2023-09-09T23:11:35.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.585+0000 7fa2d7a44700 1 -- start start 2023-09-09T23:11:35.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.585+0000 7fa2d7a44700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2d00ff5a0 0x7fa2d01963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:35.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.585+0000 7fa2d7a44700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2d01007a0 0x7fa2d01968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:35.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.585+0000 7fa2d7a44700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2d0101140 0x7fa2d019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:35.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.585+0000 7fa2d7a44700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d019df40 con 0x7fa2d00ff5a0 2023-09-09T23:11:35.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.585+0000 7fa2d7a44700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d019e080 con 0x7fa2d01007a0 2023-09-09T23:11:35.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.585+0000 7fa2d7a44700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d019e210 con 0x7fa2d0101140 2023-09-09T23:11:35.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.585+0000 7fa2d57e0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2d00ff5a0 0x7fa2d01963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:35.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.585+0000 7fa2d57e0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2d00ff5a0 0x7fa2d01963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54032/0 (socket says 172.21.15.82:54032) 2023-09-09T23:11:35.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.585+0000 7fa2d57e0700 1 -- 172.21.15.82:0/1418285965 learned_addr learned my addr 172.21.15.82:0/1418285965 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:11:35.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.585+0000 7fa2d5fe1700 1 --2- 172.21.15.82:0/1418285965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2d0101140 0x7fa2d019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:35.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.585+0000 7fa2d4fdf700 1 --2- 172.21.15.82:0/1418285965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2d01007a0 0x7fa2d01968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:35.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.586+0000 7fa2d57e0700 1 -- 172.21.15.82:0/1418285965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2d0101140 msgr2=0x7fa2d019d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:35.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.586+0000 7fa2d57e0700 1 --2- 172.21.15.82:0/1418285965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2d0101140 0x7fa2d019d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:35.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.586+0000 7fa2d57e0700 1 -- 172.21.15.82:0/1418285965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2d01007a0 msgr2=0x7fa2d01968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:35.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.586+0000 7fa2d57e0700 1 --2- 172.21.15.82:0/1418285965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2d01007a0 0x7fa2d01968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:35.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.586+0000 7fa2d57e0700 1 -- 172.21.15.82:0/1418285965 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c8009d00 con 0x7fa2d00ff5a0 2023-09-09T23:11:35.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.586+0000 7fa2d57e0700 1 --2- 172.21.15.82:0/1418285965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2d00ff5a0 0x7fa2d01963b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa2c000e9a0 tx=0x7fa2c000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:11:35.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.586+0000 7fa2c67fc700 1 -- 172.21.15.82:0/1418285965 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0009790 con 0x7fa2d00ff5a0 2023-09-09T23:11:35.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.586+0000 7fa2c67fc700 1 -- 172.21.15.82:0/1418285965 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2c000c810 con 0x7fa2d00ff5a0 2023-09-09T23:11:35.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.586+0000 7fa2d7a44700 1 -- 172.21.15.82:0/1418285965 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d019e4f0 con 0x7fa2d00ff5a0 2023-09-09T23:11:35.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.586+0000 7fa2c67fc700 1 -- 172.21.15.82:0/1418285965 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0003c70 con 0x7fa2d00ff5a0 2023-09-09T23:11:35.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.586+0000 7fa2d7a44700 1 -- 172.21.15.82:0/1418285965 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d019e9f0 con 0x7fa2d00ff5a0 2023-09-09T23:11:35.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.587+0000 7fa2d7a44700 1 -- 172.21.15.82:0/1418285965 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d010bec0 con 0x7fa2d00ff5a0 2023-09-09T23:11:35.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.588+0000 7fa2c67fc700 1 -- 172.21.15.82:0/1418285965 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fa2c000cc80 con 0x7fa2d00ff5a0 2023-09-09T23:11:35.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.589+0000 7fa2c67fc700 1 --2- 172.21.15.82:0/1418285965 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa2bc0a0640 0x7fa2bc0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:35.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.589+0000 7fa2d4fdf700 1 --2- 172.21.15.82:0/1418285965 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa2bc0a0640 0x7fa2bc0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:35.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.589+0000 7fa2c67fc700 1 -- 172.21.15.82:0/1418285965 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6831+0+0 (secure 0 0 0) 0x7fa2c0014070 con 0x7fa2d00ff5a0 2023-09-09T23:11:35.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.589+0000 7fa2d4fdf700 1 --2- 172.21.15.82:0/1418285965 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa2bc0a0640 0x7fa2bc0a2b00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00ac50 tx=0x7fa2cc00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:11:35.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.592+0000 7fa2c67fc700 1 -- 172.21.15.82:0/1418285965 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa2c0095950 con 0x7fa2d00ff5a0 2023-09-09T23:11:35.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:35 smithi082 ceph-mon[98694]: from='osd.11 [v2:172.21.15.170:6824/146964512,v1:172.21.15.170:6825/146964512]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-09T23:11:35.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:35 smithi082 ceph-mon[98694]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-09T23:11:35.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.781+0000 7fa2d7a44700 1 -- 172.21.15.82:0/1418285965 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fa2d010c0d0 con 0x7fa2d00ff5a0 2023-09-09T23:11:35.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.781+0000 7fa2c67fc700 1 -- 172.21.15.82:0/1418285965 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) v1 ==== 74+0+133 (secure 0 0 0) 0x7fa2c0082e60 con 0x7fa2d00ff5a0 2023-09-09T23:11:35.780 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:11:35.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.783+0000 7fa2d7a44700 1 -- 172.21.15.82:0/1418285965 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa2bc0a0640 msgr2=0x7fa2bc0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:35.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.783+0000 7fa2d7a44700 1 --2- 172.21.15.82:0/1418285965 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa2bc0a0640 0x7fa2bc0a2b00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00ac50 tx=0x7fa2cc00a380 comp rx=0 tx=0).stop 2023-09-09T23:11:35.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.783+0000 7fa2d7a44700 1 -- 172.21.15.82:0/1418285965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2d00ff5a0 msgr2=0x7fa2d01963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:35.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.783+0000 7fa2d7a44700 1 --2- 172.21.15.82:0/1418285965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2d00ff5a0 0x7fa2d01963b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa2c000e9a0 tx=0x7fa2c000ecb0 comp rx=0 tx=0).stop 2023-09-09T23:11:35.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.784+0000 7fa2d7a44700 1 -- 172.21.15.82:0/1418285965 shutdown_connections 2023-09-09T23:11:35.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.784+0000 7fa2d7a44700 1 --2- 172.21.15.82:0/1418285965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2d0101140 0x7fa2d019d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:35.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.784+0000 7fa2d7a44700 1 --2- 172.21.15.82:0/1418285965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2d00ff5a0 0x7fa2d01963b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:35.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.784+0000 7fa2d7a44700 1 --2- 172.21.15.82:0/1418285965 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa2bc0a0640 0x7fa2bc0a2b00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:35.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.784+0000 7fa2d7a44700 1 --2- 172.21.15.82:0/1418285965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2d01007a0 0x7fa2d01968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:35.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.784+0000 7fa2d7a44700 1 -- 172.21.15.82:0/1418285965 >> 172.21.15.82:0/1418285965 conn(0x7fa2d00fab40 msgr2=0x7fa2d00fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:35.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.784+0000 7fa2d7a44700 1 -- 172.21.15.82:0/1418285965 shutdown_connections 2023-09-09T23:11:35.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:35.784+0000 7fa2d7a44700 1 -- 172.21.15.82:0/1418285965 wait complete. 2023-09-09T23:11:36.110 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":74,"num_osds":12,"num_up_osds":11,"osd_up_since":1694301076,"num_in_osds":12,"osd_in_since":1694301079,"num_remapped_pgs":0} 2023-09-09T23:11:36.256 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:36 smithi170 ceph-mon[102832]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-09T23:11:36.256 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:36 smithi170 ceph-mon[102832]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-09-09T23:11:36.256 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:36 smithi170 ceph-mon[102832]: from='osd.11 [v2:172.21.15.170:6824/146964512,v1:172.21.15.170:6825/146964512]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-09T23:11:36.257 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:36 smithi170 ceph-mon[102832]: osdmap e74: 12 total, 11 up, 12 in 2023-09-09T23:11:36.257 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:36 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-09T23:11:36.257 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:36 smithi170 ceph-mon[102832]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-09T23:11:36.257 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:36 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1418285965' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-09T23:11:36.454 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:36 smithi082 ceph-mon[98694]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-09T23:11:36.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:36 smithi082 ceph-mon[98694]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-09-09T23:11:36.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:36 smithi082 ceph-mon[98694]: from='osd.11 [v2:172.21.15.170:6824/146964512,v1:172.21.15.170:6825/146964512]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-09T23:11:36.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:36 smithi082 ceph-mon[98694]: osdmap e74: 12 total, 11 up, 12 in 2023-09-09T23:11:36.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:36 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-09T23:11:36.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:36 smithi082 ceph-mon[98694]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-09T23:11:36.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:36 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1418285965' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-09T23:11:36.675 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 09 23:11:36 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-11[133461]: 2023-09-09T23:11:36.257+0000 7fd773373700 -1 osd.11 0 waiting for initial osdmap 2023-09-09T23:11:36.676 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 09 23:11:36 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-11[133461]: 2023-09-09T23:11:36.272+0000 7fd76e166700 -1 osd.11 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-09T23:11:36.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:36 smithi098 ceph-mon[103124]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-09T23:11:36.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:36 smithi098 ceph-mon[103124]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-09-09T23:11:36.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:36 smithi098 ceph-mon[103124]: from='osd.11 [v2:172.21.15.170:6824/146964512,v1:172.21.15.170:6825/146964512]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-09T23:11:36.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:36 smithi098 ceph-mon[103124]: osdmap e74: 12 total, 11 up, 12 in 2023-09-09T23:11:36.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:36 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-09T23:11:36.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:36 smithi098 ceph-mon[103124]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-09-09T23:11:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:36 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1418285965' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-09T23:11:37.112 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph osd stat -f json 2023-09-09T23:11:37.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:37 smithi082 ceph-mon[98694]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-09-09T23:11:37.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:37 smithi082 ceph-mon[98694]: osdmap e75: 12 total, 11 up, 12 in 2023-09-09T23:11:37.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:37 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-09T23:11:37.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:37 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-09T23:11:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:37 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:37 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:37 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:11:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:37 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:11:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:37 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:11:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:37 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:11:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:37 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:37 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:37 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:11:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:37 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:37.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:37 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:11:37.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:37 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:37.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:37 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:11:37.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:37 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:37.675 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:11:37.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:37 smithi170 ceph-mon[102832]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-09-09T23:11:37.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:37 smithi170 ceph-mon[102832]: osdmap e75: 12 total, 11 up, 12 in 2023-09-09T23:11:37.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:37 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-09T23:11:37.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:37 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-09T23:11:37.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:37 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:37 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:37 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:11:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:37 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:11:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:37 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:11:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:37 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:11:37.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:37 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:37.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:37 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:37.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:37 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:11:37.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:37 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:37.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:37 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:11:37.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:37 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:37.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:37 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:11:37.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:37 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:37.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:37 smithi098 ceph-mon[103124]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-09-09T23:11:37.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:37 smithi098 ceph-mon[103124]: osdmap e75: 12 total, 11 up, 12 in 2023-09-09T23:11:37.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:37 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-09T23:11:37.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:37 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-09T23:11:37.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:37 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:37.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:37 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:37 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:11:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:37 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:11:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:37 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:11:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:37 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:11:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:37 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:37 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:37 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:11:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:37 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:37 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:11:37.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:37 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:37.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:37 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:11:37.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:37 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:38.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:38 smithi082 ceph-mon[98694]: purged_snaps scrub starts 2023-09-09T23:11:38.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:38 smithi082 ceph-mon[98694]: purged_snaps scrub ok 2023-09-09T23:11:38.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:38 smithi082 ceph-mon[98694]: Detected new or changed devices on smithi170 2023-09-09T23:11:38.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:38 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 4292M 2023-09-09T23:11:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:38 smithi082 ceph-mon[98694]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-09T23:11:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:38 smithi082 ceph-mon[98694]: osd.11 [v2:172.21.15.170:6824/146964512,v1:172.21.15.170:6825/146964512] boot 2023-09-09T23:11:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:38 smithi082 ceph-mon[98694]: osdmap e76: 12 total, 12 up, 12 in 2023-09-09T23:11:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:38 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-09T23:11:38.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:38 smithi170 ceph-mon[102832]: purged_snaps scrub starts 2023-09-09T23:11:38.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:38 smithi170 ceph-mon[102832]: purged_snaps scrub ok 2023-09-09T23:11:38.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:38 smithi170 ceph-mon[102832]: Detected new or changed devices on smithi170 2023-09-09T23:11:38.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:38 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 4292M 2023-09-09T23:11:38.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:38 smithi170 ceph-mon[102832]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-09T23:11:38.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:38 smithi170 ceph-mon[102832]: osd.11 [v2:172.21.15.170:6824/146964512,v1:172.21.15.170:6825/146964512] boot 2023-09-09T23:11:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:38 smithi170 ceph-mon[102832]: osdmap e76: 12 total, 12 up, 12 in 2023-09-09T23:11:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:38 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-09T23:11:38.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:38 smithi098 ceph-mon[103124]: purged_snaps scrub starts 2023-09-09T23:11:38.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:38 smithi098 ceph-mon[103124]: purged_snaps scrub ok 2023-09-09T23:11:38.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:38 smithi098 ceph-mon[103124]: Detected new or changed devices on smithi170 2023-09-09T23:11:38.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:38 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 4292M 2023-09-09T23:11:38.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:38 smithi098 ceph-mon[103124]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-09T23:11:38.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:38 smithi098 ceph-mon[103124]: osd.11 [v2:172.21.15.170:6824/146964512,v1:172.21.15.170:6825/146964512] boot 2023-09-09T23:11:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:38 smithi098 ceph-mon[103124]: osdmap e76: 12 total, 12 up, 12 in 2023-09-09T23:11:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:38 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-09T23:11:38.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.791+0000 7f8d23fc6700 1 -- 172.21.15.82:0/266464195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d1c1018d0 msgr2=0x7f8d1c101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:38.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.791+0000 7f8d23fc6700 1 --2- 172.21.15.82:0/266464195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d1c1018d0 0x7f8d1c101d30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8d180099c0 tx=0x7f8d18009cd0 comp rx=0 tx=0).stop 2023-09-09T23:11:38.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.791+0000 7f8d23fc6700 1 -- 172.21.15.82:0/266464195 shutdown_connections 2023-09-09T23:11:38.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.791+0000 7f8d23fc6700 1 --2- 172.21.15.82:0/266464195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d1c1006d0 0x7f8d1c100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:38.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.791+0000 7f8d23fc6700 1 --2- 172.21.15.82:0/266464195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d1c102270 0x7f8d1c108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:38.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.791+0000 7f8d23fc6700 1 --2- 172.21.15.82:0/266464195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d1c1018d0 0x7f8d1c101d30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:38.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.791+0000 7f8d23fc6700 1 -- 172.21.15.82:0/266464195 >> 172.21.15.82:0/266464195 conn(0x7f8d1c0fbc70 msgr2=0x7f8d1c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:38.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.792+0000 7f8d23fc6700 1 -- 172.21.15.82:0/266464195 shutdown_connections 2023-09-09T23:11:38.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.792+0000 7f8d23fc6700 1 -- 172.21.15.82:0/266464195 wait complete. 2023-09-09T23:11:38.794 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.792+0000 7f8d23fc6700 1 Processor -- start 2023-09-09T23:11:38.795 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.792+0000 7f8d23fc6700 1 -- start start 2023-09-09T23:11:38.795 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.792+0000 7f8d23fc6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d1c1006d0 0x7f8d1c1962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:38.795 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.792+0000 7f8d23fc6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d1c1018d0 0x7f8d1c1967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:38.795 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.792+0000 7f8d23fc6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d1c102270 0x7f8d1c19d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:38.796 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.792+0000 7f8d23fc6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c19de40 con 0x7f8d1c1006d0 2023-09-09T23:11:38.796 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.792+0000 7f8d23fc6700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c19df80 con 0x7f8d1c102270 2023-09-09T23:11:38.796 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.792+0000 7f8d23fc6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c19e110 con 0x7f8d1c1018d0 2023-09-09T23:11:38.796 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.792+0000 7f8d21d62700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d1c1006d0 0x7f8d1c1962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:38.796 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.792+0000 7f8d21d62700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d1c1006d0 0x7f8d1c1962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54050/0 (socket says 172.21.15.82:54050) 2023-09-09T23:11:38.797 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.792+0000 7f8d21d62700 1 -- 172.21.15.82:0/243596524 learned_addr learned my addr 172.21.15.82:0/243596524 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:11:38.797 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.793+0000 7f8d22563700 1 --2- 172.21.15.82:0/243596524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d1c102270 0x7f8d1c19d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:38.797 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.793+0000 7f8d21561700 1 --2- 172.21.15.82:0/243596524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d1c1018d0 0x7f8d1c1967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:38.797 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.793+0000 7f8d21d62700 1 -- 172.21.15.82:0/243596524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d1c1018d0 msgr2=0x7f8d1c1967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:38.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.793+0000 7f8d21d62700 1 --2- 172.21.15.82:0/243596524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d1c1018d0 0x7f8d1c1967f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:38.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.793+0000 7f8d21d62700 1 -- 172.21.15.82:0/243596524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d1c102270 msgr2=0x7f8d1c19d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:38.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.793+0000 7f8d21d62700 1 --2- 172.21.15.82:0/243596524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d1c102270 0x7f8d1c19d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:38.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.793+0000 7f8d21d62700 1 -- 172.21.15.82:0/243596524 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d180095f0 con 0x7f8d1c1006d0 2023-09-09T23:11:38.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.793+0000 7f8d21d62700 1 --2- 172.21.15.82:0/243596524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d1c1006d0 0x7f8d1c1962b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00c9c0 tx=0x7f8d0c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:11:38.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.793+0000 7f8d12ffd700 1 -- 172.21.15.82:0/243596524 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c007ac0 con 0x7f8d1c1006d0 2023-09-09T23:11:38.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.793+0000 7f8d23fc6700 1 -- 172.21.15.82:0/243596524 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d1c19e3f0 con 0x7f8d1c1006d0 2023-09-09T23:11:38.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.793+0000 7f8d12ffd700 1 -- 172.21.15.82:0/243596524 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d0c007c20 con 0x7f8d1c1006d0 2023-09-09T23:11:38.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.793+0000 7f8d12ffd700 1 -- 172.21.15.82:0/243596524 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c018660 con 0x7f8d1c1006d0 2023-09-09T23:11:38.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.793+0000 7f8d23fc6700 1 -- 172.21.15.82:0/243596524 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d1c19e8f0 con 0x7f8d1c1006d0 2023-09-09T23:11:38.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.794+0000 7f8d23fc6700 1 -- 172.21.15.82:0/243596524 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d1c10bed0 con 0x7f8d1c1006d0 2023-09-09T23:11:38.801 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.796+0000 7f8d12ffd700 1 -- 172.21.15.82:0/243596524 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f8d0c004020 con 0x7f8d1c1006d0 2023-09-09T23:11:38.801 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.796+0000 7f8d12ffd700 1 --2- 172.21.15.82:0/243596524 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f8d080a0640 0x7f8d080a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:38.801 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.796+0000 7f8d21561700 1 --2- 172.21.15.82:0/243596524 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f8d080a0640 0x7f8d080a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:38.801 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.796+0000 7f8d12ffd700 1 -- 172.21.15.82:0/243596524 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8d0c0cb780 con 0x7f8d1c1006d0 2023-09-09T23:11:38.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.796+0000 7f8d21561700 1 --2- 172.21.15.82:0/243596524 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f8d080a0640 0x7f8d080a2b00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8d180098e0 tx=0x7f8d1800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:11:38.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.799+0000 7f8d12ffd700 1 -- 172.21.15.82:0/243596524 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8d0c0957e0 con 0x7f8d1c1006d0 2023-09-09T23:11:38.991 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.992+0000 7f8d23fc6700 1 -- 172.21.15.82:0/243596524 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f8d1c10c110 con 0x7f8d1c1006d0 2023-09-09T23:11:38.991 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.992+0000 7f8d12ffd700 1 -- 172.21.15.82:0/243596524 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) v1 ==== 74+0+133 (secure 0 0 0) 0x7f8d0c098e00 con 0x7f8d1c1006d0 2023-09-09T23:11:38.992 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:11:38.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.995+0000 7f8d23fc6700 1 -- 172.21.15.82:0/243596524 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f8d080a0640 msgr2=0x7f8d080a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:38.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.995+0000 7f8d23fc6700 1 --2- 172.21.15.82:0/243596524 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f8d080a0640 0x7f8d080a2b00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8d180098e0 tx=0x7f8d1800b540 comp rx=0 tx=0).stop 2023-09-09T23:11:38.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.995+0000 7f8d23fc6700 1 -- 172.21.15.82:0/243596524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d1c1006d0 msgr2=0x7f8d1c1962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:38.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.995+0000 7f8d23fc6700 1 --2- 172.21.15.82:0/243596524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d1c1006d0 0x7f8d1c1962b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00c9c0 tx=0x7f8d0c00ccd0 comp rx=0 tx=0).stop 2023-09-09T23:11:38.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.995+0000 7f8d23fc6700 1 -- 172.21.15.82:0/243596524 shutdown_connections 2023-09-09T23:11:38.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.995+0000 7f8d23fc6700 1 --2- 172.21.15.82:0/243596524 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f8d080a0640 0x7f8d080a2b00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:38.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.995+0000 7f8d23fc6700 1 --2- 172.21.15.82:0/243596524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d1c102270 0x7f8d1c19d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:38.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.995+0000 7f8d23fc6700 1 --2- 172.21.15.82:0/243596524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d1c1006d0 0x7f8d1c1962b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:38.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.995+0000 7f8d23fc6700 1 --2- 172.21.15.82:0/243596524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d1c1018d0 0x7f8d1c1967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:38.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.995+0000 7f8d23fc6700 1 -- 172.21.15.82:0/243596524 >> 172.21.15.82:0/243596524 conn(0x7f8d1c0fbc70 msgr2=0x7f8d1c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:38.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.995+0000 7f8d23fc6700 1 -- 172.21.15.82:0/243596524 shutdown_connections 2023-09-09T23:11:38.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:38.995+0000 7f8d23fc6700 1 -- 172.21.15.82:0/243596524 wait complete. 2023-09-09T23:11:39.313 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":77,"num_osds":12,"num_up_osds":12,"osd_up_since":1694301097,"num_in_osds":12,"osd_in_since":1694301079,"num_remapped_pgs":0} 2023-09-09T23:11:39.314 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph osd dump --format=json 2023-09-09T23:11:39.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:39 smithi082 ceph-mon[98694]: osdmap e77: 12 total, 12 up, 12 in 2023-09-09T23:11:39.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:39 smithi082 ceph-mon[98694]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:39.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:39 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/243596524' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-09T23:11:39.868 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:11:39.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:39 smithi170 ceph-mon[102832]: osdmap e77: 12 total, 12 up, 12 in 2023-09-09T23:11:39.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:39 smithi170 ceph-mon[102832]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:39.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:39 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/243596524' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-09T23:11:40.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:39 smithi098 ceph-mon[103124]: osdmap e77: 12 total, 12 up, 12 in 2023-09-09T23:11:40.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:39 smithi098 ceph-mon[103124]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:40.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:39 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/243596524' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-09T23:11:40.898 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.899+0000 7f58c2f12700 1 -- 172.21.15.82:0/683030293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58bc101840 msgr2=0x7f58bc10c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:40.899 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.899+0000 7f58c2f12700 1 --2- 172.21.15.82:0/683030293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58bc101840 0x7f58bc10c4d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f58b400b0a0 tx=0x7f58b400b3b0 comp rx=0 tx=0).stop 2023-09-09T23:11:40.899 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.899+0000 7f58c2f12700 1 -- 172.21.15.82:0/683030293 shutdown_connections 2023-09-09T23:11:40.899 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.899+0000 7f58c2f12700 1 --2- 172.21.15.82:0/683030293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58bc0ff880 0x7f58bc0ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:40.899 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.899+0000 7f58c2f12700 1 --2- 172.21.15.82:0/683030293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58bc101840 0x7f58bc10c4d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:40.899 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.899+0000 7f58c2f12700 1 --2- 172.21.15.82:0/683030293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58bc1001a0 0x7f58bc101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:40.900 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.899+0000 7f58c2f12700 1 -- 172.21.15.82:0/683030293 >> 172.21.15.82:0/683030293 conn(0x7f58bc0765f0 msgr2=0x7f58bc076a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:40.900 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.899+0000 7f58c2f12700 1 -- 172.21.15.82:0/683030293 shutdown_connections 2023-09-09T23:11:40.900 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.899+0000 7f58c2f12700 1 -- 172.21.15.82:0/683030293 wait complete. 2023-09-09T23:11:40.900 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.900+0000 7f58c2f12700 1 Processor -- start 2023-09-09T23:11:40.900 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.900+0000 7f58c2f12700 1 -- start start 2023-09-09T23:11:40.902 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.900+0000 7f58c2f12700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58bc0ff880 0x7f58bc19b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:40.903 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.900+0000 7f58c2f12700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58bc1001a0 0x7f58bc19bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:40.903 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.900+0000 7f58c2f12700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58bc101840 0x7f58bc195670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:40.903 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.900+0000 7f58c2f12700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc195ce0 con 0x7f58bc101840 2023-09-09T23:11:40.903 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.900+0000 7f58c2f12700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc195e50 con 0x7f58bc0ff880 2023-09-09T23:11:40.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.900+0000 7f58c2f12700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc196010 con 0x7f58bc1001a0 2023-09-09T23:11:40.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.900+0000 7f58c14af700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58bc101840 0x7f58bc195670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:40.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.900+0000 7f58c14af700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58bc101840 0x7f58bc195670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54076/0 (socket says 172.21.15.82:54076) 2023-09-09T23:11:40.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.900+0000 7f58c14af700 1 -- 172.21.15.82:0/70348223 learned_addr learned my addr 172.21.15.82:0/70348223 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:11:40.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.900+0000 7f58c0cae700 1 --2- 172.21.15.82:0/70348223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58bc0ff880 0x7f58bc19b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:40.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.901+0000 7f58bbfff700 1 --2- 172.21.15.82:0/70348223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58bc1001a0 0x7f58bc19bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:40.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.901+0000 7f58c14af700 1 -- 172.21.15.82:0/70348223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58bc1001a0 msgr2=0x7f58bc19bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:40.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.901+0000 7f58c14af700 1 --2- 172.21.15.82:0/70348223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58bc1001a0 0x7f58bc19bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:40.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.901+0000 7f58c14af700 1 -- 172.21.15.82:0/70348223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58bc0ff880 msgr2=0x7f58bc19b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:40.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.901+0000 7f58c14af700 1 --2- 172.21.15.82:0/70348223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58bc0ff880 0x7f58bc19b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:40.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.901+0000 7f58c14af700 1 -- 172.21.15.82:0/70348223 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b4009d00 con 0x7f58bc101840 2023-09-09T23:11:40.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.901+0000 7f58bbfff700 1 --2- 172.21.15.82:0/70348223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58bc1001a0 0x7f58bc19bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:11:40.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.901+0000 7f58c14af700 1 --2- 172.21.15.82:0/70348223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58bc101840 0x7f58bc195670 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f58b400b780 tx=0x7f58b4007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:11:40.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.901+0000 7f58b9ffb700 1 -- 172.21.15.82:0/70348223 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b400e040 con 0x7f58bc101840 2023-09-09T23:11:40.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.901+0000 7f58b9ffb700 1 -- 172.21.15.82:0/70348223 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58b4004e60 con 0x7f58bc101840 2023-09-09T23:11:40.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.901+0000 7f58c2f12700 1 -- 172.21.15.82:0/70348223 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58bc196290 con 0x7f58bc101840 2023-09-09T23:11:40.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.901+0000 7f58b9ffb700 1 -- 172.21.15.82:0/70348223 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b40052a0 con 0x7f58bc101840 2023-09-09T23:11:40.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.901+0000 7f58c2f12700 1 -- 172.21.15.82:0/70348223 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58bc1a25e0 con 0x7f58bc101840 2023-09-09T23:11:40.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.902+0000 7f58c2f12700 1 -- 172.21.15.82:0/70348223 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58bc04e300 con 0x7f58bc101840 2023-09-09T23:11:40.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.903+0000 7f58b9ffb700 1 -- 172.21.15.82:0/70348223 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f58b401d470 con 0x7f58bc101840 2023-09-09T23:11:40.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.904+0000 7f58b9ffb700 1 --2- 172.21.15.82:0/70348223 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f58ac0a0640 0x7f58ac0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:40.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.904+0000 7f58c0cae700 1 --2- 172.21.15.82:0/70348223 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f58ac0a0640 0x7f58ac0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:40.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.904+0000 7f58b9ffb700 1 -- 172.21.15.82:0/70348223 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f58b40cd010 con 0x7f58bc101840 2023-09-09T23:11:40.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.904+0000 7f58c0cae700 1 --2- 172.21.15.82:0/70348223 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f58ac0a0640 0x7f58ac0a2b00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f58b0009c60 tx=0x7f58b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:11:40.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:40.907+0000 7f58b9ffb700 1 -- 172.21.15.82:0/70348223 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f58b4097070 con 0x7f58bc101840 2023-09-09T23:11:41.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:41.098+0000 7f58c2f12700 1 -- 172.21.15.82:0/70348223 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58bc196b80 con 0x7f58bc101840 2023-09-09T23:11:41.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:41.099+0000 7f58b9ffb700 1 -- 172.21.15.82:0/70348223 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18374 (secure 0 0 0) 0x7f58b4025e90 con 0x7f58bc101840 2023-09-09T23:11:41.098 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:11:41.098 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":77,"fsid":"3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da","created":"2023-09-09T23:05:14.222190+0000","modified":"2023-09-09T23:11:38.498415+0000","last_up_change":"2023-09-09T23:11:37.253759+0000","last_in_change":"2023-09-09T23:11:19.933812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-09T23:08:31.234958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1277eb17-a2e7-4e10-9832-f12a5e93ffde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6803","nonce":3197434929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6805","nonce":3197434929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6809","nonce":3197434929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6807","nonce":3197434929}]},"public_addr":"172.21.15.82:6803/3197434929","cluster_addr":"172.21.15.82:6805/3197434929","heartbeat_back_addr":"172.21.15.82:6809/3197434929","heartbeat_front_addr":"172.21.15.82:6807/3197434929","state":["exists","up"]},{"osd":1,"uuid":"f68723fd-0908-4e21-ae8e-1de1c3a9a65c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6801","nonce":3528641314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6803","nonce":3528641314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6807","nonce":3528641314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6805","nonce":3528641314}]},"public_addr":"172.21.15.98:6801/3528641314","cluster_addr":"172.21.15.98:6803/3528641314","heartbeat_back_addr":"172.21.15.98:6807/3528641314","heartbeat_front_addr":"172.21.15.98:6805/3528641314","state":["exists","up"]},{"osd":2,"uuid":"06f89c53-1419-4f52-acad-6e92916b476f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6801","nonce":4006414551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6803","nonce":4006414551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6807","nonce":4006414551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6805","nonce":4006414551}]},"public_addr":"172.21.15.170:6801/4006414551","cluster_addr":"172.21.15.170:6803/4006414551","heartbeat_back_addr":"172.21.15.170:6807/4006414551","heartbeat_front_addr":"172.21.15.170:6805/4006414551","state":["exists","up"]},{"osd":3,"uuid":"fedbd4c9-6852-42b5-8f3a-ba8d9bfe5d59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6811","nonce":3647629241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6813","nonce":3647629241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6817","nonce":3647629241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6815","nonce":3647629241}]},"public_addr":"172.21.15.82:6811/3647629241","cluster_addr":"172.21.15.82:6813/3647629241","heartbeat_back_addr":"172.21.15.82:6817/3647629241","heartbeat_front_addr":"172.21.15.82:6815/3647629241","state":["exists","up"]},{"osd":4,"uuid":"a71f3084-6919-4161-87bf-345f3355d8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6809","nonce":2507035643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6811","nonce":2507035643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6815","nonce":2507035643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6813","nonce":2507035643}]},"public_addr":"172.21.15.98:6809/2507035643","cluster_addr":"172.21.15.98:6811/2507035643","heartbeat_back_addr":"172.21.15.98:6815/2507035643","heartbeat_front_addr":"172.21.15.98:6813/2507035643","state":["exists","up"]},{"osd":5,"uuid":"f76058e9-a90e-476e-87f0-3c6723795e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6809","nonce":775005168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6811","nonce":775005168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6815","nonce":775005168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6813","nonce":775005168}]},"public_addr":"172.21.15.170:6809/775005168","cluster_addr":"172.21.15.170:6811/775005168","heartbeat_back_addr":"172.21.15.170:6815/775005168","heartbeat_front_addr":"172.21.15.170:6813/775005168","state":["exists","up"]},{"osd":6,"uuid":"25940be0-c1bd-4c72-95bf-fd8367aa2f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6819","nonce":3008405982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6821","nonce":3008405982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6825","nonce":3008405982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6823","nonce":3008405982}]},"public_addr":"172.21.15.82:6819/3008405982","cluster_addr":"172.21.15.82:6821/3008405982","heartbeat_back_addr":"172.21.15.82:6825/3008405982","heartbeat_front_addr":"172.21.15.82:6823/3008405982","state":["exists","up"]},{"osd":7,"uuid":"f651bb8c-0b25-42d5-b028-58563f4c27f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6817","nonce":3547615284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6819","nonce":3547615284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6823","nonce":3547615284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6821","nonce":3547615284}]},"public_addr":"172.21.15.98:6817/3547615284","cluster_addr":"172.21.15.98:6819/3547615284","heartbeat_back_addr":"172.21.15.98:6823/3547615284","heartbeat_front_addr":"172.21.15.98:6821/3547615284","state":["exists","up"]},{"osd":8,"uuid":"23b27c86-8c98-441e-b7ab-d57bc584e58f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6817","nonce":632038848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6819","nonce":632038848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6823","nonce":632038848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6821","nonce":632038848}]},"public_addr":"172.21.15.170:6817/632038848","cluster_addr":"172.21.15.170:6819/632038848","heartbeat_back_addr":"172.21.15.170:6823/632038848","heartbeat_front_addr":"172.21.15.170:6821/632038848","state":["exists","up"]},{"osd":9,"uuid":"383b8989-4922-4eee-9c77-f198258ac243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6827","nonce":3354242311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6829","nonce":3354242311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6833","nonce":3354242311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6831","nonce":3354242311}]},"public_addr":"172.21.15.82:6827/3354242311","cluster_addr":"172.21.15.82:6829/3354242311","heartbeat_back_addr":"172.21.15.82:6833/3354242311","heartbeat_front_addr":"172.21.15.82:6831/3354242311","state":["exists","up"]},{"osd":10,"uuid":"33a59389-7ced-42d5-bf54-2071317132d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6825","nonce":2069114959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6827","nonce":2069114959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6831","nonce":2069114959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6829","nonce":2069114959}]},"public_addr":"172.21.15.98:6825/2069114959","cluster_addr":"172.21.15.98:6827/2069114959","heartbeat_back_addr":"172.21.15.98:6831/2069114959","heartbeat_front_addr":"172.21.15.98:6829/2069114959","state":["exists","up"]},{"osd":11,"uuid":"b7a71f5c-85e2-4fec-be4b-95be0788b302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6825","nonce":146964512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6827","nonce":146964512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6831","nonce":146964512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6829","nonce":146964512}]},"public_addr":"172.21.15.170:6825/146964512","cluster_addr":"172.21.15.170:6827/146964512","heartbeat_back_addr":"172.21.15.170:6831/146964512","heartbeat_front_addr":"172.21.15.170:6829/146964512","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:07:47.563043+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:08.359260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:29.070374+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:49.897775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:11.434248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:31.538464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:52.377109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:13.452968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:33.806897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:54.083177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:14.980087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:35.690089+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4013747788":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6800/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/2823341464":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/75116806":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/1037625632":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/3142088817":"2023-09-10T23:05:46.550354+0000","172.21.15.82:6800/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/1679622963":"2023-09-10T23:05:46.550354+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-09T23:11:41.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:41.101+0000 7f58c2f12700 1 -- 172.21.15.82:0/70348223 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f58ac0a0640 msgr2=0x7f58ac0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:41.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:41.101+0000 7f58c2f12700 1 --2- 172.21.15.82:0/70348223 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f58ac0a0640 0x7f58ac0a2b00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f58b0009c60 tx=0x7f58b0009260 comp rx=0 tx=0).stop 2023-09-09T23:11:41.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:41.101+0000 7f58c2f12700 1 -- 172.21.15.82:0/70348223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58bc101840 msgr2=0x7f58bc195670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:41.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:41.101+0000 7f58c2f12700 1 --2- 172.21.15.82:0/70348223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58bc101840 0x7f58bc195670 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f58b400b780 tx=0x7f58b4007870 comp rx=0 tx=0).stop 2023-09-09T23:11:41.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:41.101+0000 7f58c2f12700 1 -- 172.21.15.82:0/70348223 shutdown_connections 2023-09-09T23:11:41.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:41.101+0000 7f58c2f12700 1 --2- 172.21.15.82:0/70348223 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f58ac0a0640 0x7f58ac0a2b00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:41.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:41.101+0000 7f58c2f12700 1 --2- 172.21.15.82:0/70348223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58bc0ff880 0x7f58bc19b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:41.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:41.101+0000 7f58c2f12700 1 --2- 172.21.15.82:0/70348223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58bc101840 0x7f58bc195670 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:41.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:41.101+0000 7f58c2f12700 1 --2- 172.21.15.82:0/70348223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58bc1001a0 0x7f58bc19bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:41.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:41.101+0000 7f58c2f12700 1 -- 172.21.15.82:0/70348223 >> 172.21.15.82:0/70348223 conn(0x7f58bc0765f0 msgr2=0x7f58bc0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:41.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:41.101+0000 7f58c2f12700 1 -- 172.21.15.82:0/70348223 shutdown_connections 2023-09-09T23:11:41.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:41.101+0000 7f58c2f12700 1 -- 172.21.15.82:0/70348223 wait complete. 2023-09-09T23:11:41.411 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-09T23:08:31.234958+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-09T23:11:41.412 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph osd pool get .mgr pg_num 2023-09-09T23:11:41.965 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:11:42.343 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:42 smithi082 ceph-mon[98694]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:42.343 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:42 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/70348223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:11:42.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:42 smithi170 ceph-mon[102832]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:42.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:42 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/70348223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:11:42.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:42 smithi098 ceph-mon[103124]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:42.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:42 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/70348223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:11:43.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.019+0000 7fbda8ea0700 1 -- 172.21.15.82:0/1346535192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbda4104530 msgr2=0x7fbda4104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:43.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.019+0000 7fbda8ea0700 1 --2- 172.21.15.82:0/1346535192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbda4104530 0x7fbda4104910 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009990 tx=0x7fbd98009ca0 comp rx=0 tx=0).stop 2023-09-09T23:11:43.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.019+0000 7fbda8ea0700 1 -- 172.21.15.82:0/1346535192 shutdown_connections 2023-09-09T23:11:43.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.019+0000 7fbda8ea0700 1 --2- 172.21.15.82:0/1346535192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbda40ffee0 0x7fbda410c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:43.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.019+0000 7fbda8ea0700 1 --2- 172.21.15.82:0/1346535192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbda4104530 0x7fbda4104910 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:43.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.019+0000 7fbda8ea0700 1 --2- 172.21.15.82:0/1346535192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbda40ff520 0x7fbda40ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:43.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.019+0000 7fbda8ea0700 1 -- 172.21.15.82:0/1346535192 >> 172.21.15.82:0/1346535192 conn(0x7fbda4076600 msgr2=0x7fbda4076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:43.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.019+0000 7fbda8ea0700 1 -- 172.21.15.82:0/1346535192 shutdown_connections 2023-09-09T23:11:43.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.019+0000 7fbda8ea0700 1 -- 172.21.15.82:0/1346535192 wait complete. 2023-09-09T23:11:43.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.020+0000 7fbda8ea0700 1 Processor -- start 2023-09-09T23:11:43.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.020+0000 7fbda8ea0700 1 -- start start 2023-09-09T23:11:43.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.020+0000 7fbda8ea0700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbda40ff520 0x7fbda4197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:43.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.020+0000 7fbda8ea0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbda40ffee0 0x7fbda4197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:43.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.020+0000 7fbda8ea0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbda4104530 0x7fbda41a25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:43.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.020+0000 7fbda8ea0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda41a2b10 con 0x7fbda4104530 2023-09-09T23:11:43.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.020+0000 7fbda8ea0700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda41a2c50 con 0x7fbda40ff520 2023-09-09T23:11:43.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.020+0000 7fbda8ea0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda41a2d90 con 0x7fbda40ffee0 2023-09-09T23:11:43.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.020+0000 7fbda2d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbda4104530 0x7fbda41a25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:43.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.020+0000 7fbda2d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbda4104530 0x7fbda41a25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54088/0 (socket says 172.21.15.82:54088) 2023-09-09T23:11:43.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.020+0000 7fbda2d9d700 1 -- 172.21.15.82:0/2810784815 learned_addr learned my addr 172.21.15.82:0/2810784815 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:11:43.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.020+0000 7fbda1d9b700 1 --2- 172.21.15.82:0/2810784815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbda40ffee0 0x7fbda4197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:43.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.020+0000 7fbda259c700 1 --2- 172.21.15.82:0/2810784815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbda40ff520 0x7fbda4197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:43.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.020+0000 7fbda2d9d700 1 -- 172.21.15.82:0/2810784815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbda40ffee0 msgr2=0x7fbda4197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:43.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.020+0000 7fbda2d9d700 1 --2- 172.21.15.82:0/2810784815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbda40ffee0 0x7fbda4197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:43.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.020+0000 7fbda2d9d700 1 -- 172.21.15.82:0/2810784815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbda40ff520 msgr2=0x7fbda4197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:43.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.020+0000 7fbda2d9d700 1 --2- 172.21.15.82:0/2810784815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbda40ff520 0x7fbda4197550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:43.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.020+0000 7fbda2d9d700 1 -- 172.21.15.82:0/2810784815 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd980095f0 con 0x7fbda4104530 2023-09-09T23:11:43.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.020+0000 7fbda259c700 1 --2- 172.21.15.82:0/2810784815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbda40ff520 0x7fbda4197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:11:43.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.021+0000 7fbda2d9d700 1 --2- 172.21.15.82:0/2810784815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbda4104530 0x7fbda41a25d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00ab50 tx=0x7fbd9c00ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:11:43.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.021+0000 7fbd937fe700 1 -- 172.21.15.82:0/2810784815 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c00f070 con 0x7fbda4104530 2023-09-09T23:11:43.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.021+0000 7fbd937fe700 1 -- 172.21.15.82:0/2810784815 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd9c007a80 con 0x7fbda4104530 2023-09-09T23:11:43.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.021+0000 7fbd937fe700 1 -- 172.21.15.82:0/2810784815 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c005280 con 0x7fbda4104530 2023-09-09T23:11:43.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.021+0000 7fbda8ea0700 1 -- 172.21.15.82:0/2810784815 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda41a2ff0 con 0x7fbda4104530 2023-09-09T23:11:43.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.021+0000 7fbda8ea0700 1 -- 172.21.15.82:0/2810784815 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda41a3460 con 0x7fbda4104530 2023-09-09T23:11:43.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.022+0000 7fbda8ea0700 1 -- 172.21.15.82:0/2810784815 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda404e300 con 0x7fbda4104530 2023-09-09T23:11:43.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.023+0000 7fbd937fe700 1 -- 172.21.15.82:0/2810784815 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fbd9c00c620 con 0x7fbda4104530 2023-09-09T23:11:43.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.023+0000 7fbd937fe700 1 --2- 172.21.15.82:0/2810784815 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fbd8c0a0640 0x7fbd8c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:43.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.023+0000 7fbda259c700 1 --2- 172.21.15.82:0/2810784815 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fbd8c0a0640 0x7fbd8c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:43.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.023+0000 7fbd937fe700 1 -- 172.21.15.82:0/2810784815 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbd9c0caa90 con 0x7fbda4104530 2023-09-09T23:11:43.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.024+0000 7fbda259c700 1 --2- 172.21.15.82:0/2810784815 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fbd8c0a0640 0x7fbd8c0a2b00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800b5c0 tx=0x7fbd98005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:11:43.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.029+0000 7fbd937fe700 1 -- 172.21.15.82:0/2810784815 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fbd9c094af0 con 0x7fbda4104530 2023-09-09T23:11:43.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.218+0000 7fbda8ea0700 1 -- 172.21.15.82:0/2810784815 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbda410f590 con 0x7fbda4104530 2023-09-09T23:11:43.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.218+0000 7fbd937fe700 1 -- 172.21.15.82:0/2810784815 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbd9c098110 con 0x7fbda4104530 2023-09-09T23:11:43.217 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2023-09-09T23:11:43.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.220+0000 7fbda8ea0700 1 -- 172.21.15.82:0/2810784815 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fbd8c0a0640 msgr2=0x7fbd8c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:43.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.220+0000 7fbda8ea0700 1 --2- 172.21.15.82:0/2810784815 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fbd8c0a0640 0x7fbd8c0a2b00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800b5c0 tx=0x7fbd98005fb0 comp rx=0 tx=0).stop 2023-09-09T23:11:43.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.220+0000 7fbda8ea0700 1 -- 172.21.15.82:0/2810784815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbda4104530 msgr2=0x7fbda41a25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:43.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.220+0000 7fbda8ea0700 1 --2- 172.21.15.82:0/2810784815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbda4104530 0x7fbda41a25d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00ab50 tx=0x7fbd9c00ae60 comp rx=0 tx=0).stop 2023-09-09T23:11:43.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.220+0000 7fbda8ea0700 1 -- 172.21.15.82:0/2810784815 shutdown_connections 2023-09-09T23:11:43.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.220+0000 7fbda8ea0700 1 --2- 172.21.15.82:0/2810784815 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fbd8c0a0640 0x7fbd8c0a2b00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:43.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.220+0000 7fbda8ea0700 1 --2- 172.21.15.82:0/2810784815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbda40ff520 0x7fbda4197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:43.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.220+0000 7fbda8ea0700 1 --2- 172.21.15.82:0/2810784815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbda4104530 0x7fbda41a25d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:43.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.220+0000 7fbda8ea0700 1 --2- 172.21.15.82:0/2810784815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbda40ffee0 0x7fbda4197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:43.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.220+0000 7fbda8ea0700 1 -- 172.21.15.82:0/2810784815 >> 172.21.15.82:0/2810784815 conn(0x7fbda4076600 msgr2=0x7fbda40fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:43.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.221+0000 7fbda8ea0700 1 -- 172.21.15.82:0/2810784815 shutdown_connections 2023-09-09T23:11:43.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:43.221+0000 7fbda8ea0700 1 -- 172.21.15.82:0/2810784815 wait complete. 2023-09-09T23:11:43.518 INFO:tasks.cephadm:Adding mds.a on smithi082 2023-09-09T23:11:43.519 INFO:tasks.cephadm:Adding mds.d on smithi082 2023-09-09T23:11:43.519 INFO:tasks.cephadm:Adding mds.g on smithi082 2023-09-09T23:11:43.520 INFO:tasks.cephadm:Adding mds.j on smithi082 2023-09-09T23:11:43.520 INFO:tasks.cephadm:Adding mds.b on smithi098 2023-09-09T23:11:43.520 INFO:tasks.cephadm:Adding mds.e on smithi098 2023-09-09T23:11:43.521 INFO:tasks.cephadm:Adding mds.h on smithi098 2023-09-09T23:11:43.521 INFO:tasks.cephadm:Adding mds.k on smithi098 2023-09-09T23:11:43.521 INFO:tasks.cephadm:Adding mds.c on smithi170 2023-09-09T23:11:43.521 INFO:tasks.cephadm:Adding mds.f on smithi170 2023-09-09T23:11:43.522 INFO:tasks.cephadm:Adding mds.i on smithi170 2023-09-09T23:11:43.522 INFO:tasks.cephadm:Adding mds.l on smithi170 2023-09-09T23:11:43.522 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph orch apply mds all '12;smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l' 2023-09-09T23:11:44.093 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.c/config 2023-09-09T23:11:44.287 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:44 smithi170 ceph-mon[102832]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:44.288 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:44 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2810784815' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:11:44.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:44 smithi082 ceph-mon[98694]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:44.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:44 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2810784815' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:11:44.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:44 smithi098 ceph-mon[103124]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:44.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:44 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2810784815' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:11:45.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.253+0000 7ff95ea9c700 1 -- 172.21.15.170:0/391783189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff958103ab0 msgr2=0x7ff958105ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:45.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.253+0000 7ff95ea9c700 1 --2- 172.21.15.170:0/391783189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff958103ab0 0x7ff958105ea0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff944009990 tx=0x7ff944009ca0 comp rx=0 tx=0).stop 2023-09-09T23:11:45.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.254+0000 7ff95ea9c700 1 -- 172.21.15.170:0/391783189 shutdown_connections 2023-09-09T23:11:45.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.254+0000 7ff95ea9c700 1 --2- 172.21.15.170:0/391783189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9581063e0 0x7ff9581087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:45.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.254+0000 7ff95ea9c700 1 --2- 172.21.15.170:0/391783189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff958101180 0x7ff958103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:45.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.254+0000 7ff95ea9c700 1 --2- 172.21.15.170:0/391783189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff958103ab0 0x7ff958105ea0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:45.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.254+0000 7ff95ea9c700 1 -- 172.21.15.170:0/391783189 >> 172.21.15.170:0/391783189 conn(0x7ff9580fab10 msgr2=0x7ff9580fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:45.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.254+0000 7ff95ea9c700 1 -- 172.21.15.170:0/391783189 shutdown_connections 2023-09-09T23:11:45.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.254+0000 7ff95ea9c700 1 -- 172.21.15.170:0/391783189 wait complete. 2023-09-09T23:11:45.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.254+0000 7ff95ea9c700 1 Processor -- start 2023-09-09T23:11:45.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.254+0000 7ff95ea9c700 1 -- start start 2023-09-09T23:11:45.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.254+0000 7ff95ea9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff958101180 0x7ff958196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:45.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.254+0000 7ff95ea9c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff958103ab0 0x7ff9581968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:45.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.254+0000 7ff95ea9c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9581063e0 0x7ff95819d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:45.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.254+0000 7ff95ea9c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95819df00 con 0x7ff958103ab0 2023-09-09T23:11:45.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.254+0000 7ff95ea9c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95819e040 con 0x7ff9581063e0 2023-09-09T23:11:45.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.254+0000 7ff95ea9c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95819e1d0 con 0x7ff958101180 2023-09-09T23:11:45.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.254+0000 7ff95c838700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff958101180 0x7ff958196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:45.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.255+0000 7ff95c838700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff958101180 0x7ff958196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34852/0 (socket says 172.21.15.170:34852) 2023-09-09T23:11:45.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.255+0000 7ff95c838700 1 -- 172.21.15.170:0/3051279564 learned_addr learned my addr 172.21.15.170:0/3051279564 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-09-09T23:11:45.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.255+0000 7ff957fff700 1 --2- 172.21.15.170:0/3051279564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff958103ab0 0x7ff9581968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:45.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.255+0000 7ff95c838700 1 -- 172.21.15.170:0/3051279564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9581063e0 msgr2=0x7ff95819d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:45.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.255+0000 7ff95c838700 1 --2- 172.21.15.170:0/3051279564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9581063e0 0x7ff95819d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:45.258 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.255+0000 7ff95c838700 1 -- 172.21.15.170:0/3051279564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff958103ab0 msgr2=0x7ff9581968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:45.258 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.255+0000 7ff95c838700 1 --2- 172.21.15.170:0/3051279564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff958103ab0 0x7ff9581968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:45.258 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.255+0000 7ff95c838700 1 -- 172.21.15.170:0/3051279564 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9440095f0 con 0x7ff958101180 2023-09-09T23:11:45.258 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.255+0000 7ff95c838700 1 --2- 172.21.15.170:0/3051279564 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff958101180 0x7ff958196370 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00d980 tx=0x7ff94c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:11:45.258 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.255+0000 7ff955ffb700 1 -- 172.21.15.170:0/3051279564 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c009b40 con 0x7ff958101180 2023-09-09T23:11:45.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.255+0000 7ff95ea9c700 1 -- 172.21.15.170:0/3051279564 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9580fefd0 con 0x7ff958101180 2023-09-09T23:11:45.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.255+0000 7ff955ffb700 1 -- 172.21.15.170:0/3051279564 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff94c00b6e0 con 0x7ff958101180 2023-09-09T23:11:45.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.255+0000 7ff955ffb700 1 -- 172.21.15.170:0/3051279564 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c00f620 con 0x7ff958101180 2023-09-09T23:11:45.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.255+0000 7ff95ea9c700 1 -- 172.21.15.170:0/3051279564 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9580ff520 con 0x7ff958101180 2023-09-09T23:11:45.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.256+0000 7ff95ea9c700 1 -- 172.21.15.170:0/3051279564 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9580fc690 con 0x7ff958101180 2023-09-09T23:11:45.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.257+0000 7ff955ffb700 1 -- 172.21.15.170:0/3051279564 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7ff94c010460 con 0x7ff958101180 2023-09-09T23:11:45.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.258+0000 7ff955ffb700 1 --2- 172.21.15.170:0/3051279564 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff9480a0640 0x7ff9480a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:45.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.258+0000 7ff955ffb700 1 -- 172.21.15.170:0/3051279564 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff94c0cb950 con 0x7ff958101180 2023-09-09T23:11:45.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.258+0000 7ff957fff700 1 --2- 172.21.15.170:0/3051279564 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff9480a0640 0x7ff9480a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:45.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.258+0000 7ff957fff700 1 --2- 172.21.15.170:0/3051279564 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff9480a0640 0x7ff9480a2b00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff94400b5c0 tx=0x7ff944005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:11:45.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.262+0000 7ff955ffb700 1 -- 172.21.15.170:0/3051279564 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff94c095920 con 0x7ff958101180 2023-09-09T23:11:45.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.463+0000 7ff95ea9c700 1 -- 172.21.15.170:0/3051279564 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}) v1 -- 0x7ff95802cf40 con 0x7ff9480a0640 2023-09-09T23:11:45.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.473+0000 7ff955ffb700 1 -- 172.21.15.170:0/3051279564 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7ff95802cf40 con 0x7ff9480a0640 2023-09-09T23:11:45.472 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled mds.all update... 2023-09-09T23:11:45.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.475+0000 7ff95ea9c700 1 -- 172.21.15.170:0/3051279564 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff9480a0640 msgr2=0x7ff9480a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:45.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.475+0000 7ff95ea9c700 1 --2- 172.21.15.170:0/3051279564 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff9480a0640 0x7ff9480a2b00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff94400b5c0 tx=0x7ff944005fd0 comp rx=0 tx=0).stop 2023-09-09T23:11:45.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.475+0000 7ff95ea9c700 1 -- 172.21.15.170:0/3051279564 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff958101180 msgr2=0x7ff958196370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:45.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.475+0000 7ff95ea9c700 1 --2- 172.21.15.170:0/3051279564 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff958101180 0x7ff958196370 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00d980 tx=0x7ff94c00dc90 comp rx=0 tx=0).stop 2023-09-09T23:11:45.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.475+0000 7ff95ea9c700 1 -- 172.21.15.170:0/3051279564 shutdown_connections 2023-09-09T23:11:45.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.475+0000 7ff95ea9c700 1 --2- 172.21.15.170:0/3051279564 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff9480a0640 0x7ff9480a2b00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:45.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.475+0000 7ff95ea9c700 1 --2- 172.21.15.170:0/3051279564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9581063e0 0x7ff95819d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:45.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.475+0000 7ff95ea9c700 1 --2- 172.21.15.170:0/3051279564 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff958101180 0x7ff958196370 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:45.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.475+0000 7ff95ea9c700 1 --2- 172.21.15.170:0/3051279564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff958103ab0 0x7ff9581968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:45.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.475+0000 7ff95ea9c700 1 -- 172.21.15.170:0/3051279564 >> 172.21.15.170:0/3051279564 conn(0x7ff9580fab10 msgr2=0x7ff958101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:45.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.475+0000 7ff95ea9c700 1 -- 172.21.15.170:0/3051279564 shutdown_connections 2023-09-09T23:11:45.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-09-09T23:11:45.475+0000 7ff95ea9c700 1 -- 172.21.15.170:0/3051279564 wait complete. 2023-09-09T23:11:46.041 DEBUG:teuthology.orchestra.run.smithi082:mds.a> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@mds.a.service 2023-09-09T23:11:46.044 DEBUG:teuthology.orchestra.run.smithi082:mds.d> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@mds.d.service 2023-09-09T23:11:46.080 INFO:journalctl@ceph.mds.a.smithi082.stdout:-- Logs begin at Sat 2023-09-09 22:51:24 UTC. -- 2023-09-09T23:11:46.081 DEBUG:teuthology.orchestra.run.smithi082:mds.g> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@mds.g.service 2023-09-09T23:11:46.110 INFO:journalctl@ceph.mds.d.smithi082.stdout:-- Logs begin at Sat 2023-09-09 22:51:24 UTC. -- 2023-09-09T23:11:46.111 DEBUG:teuthology.orchestra.run.smithi082:mds.j> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@mds.j.service 2023-09-09T23:11:46.138 INFO:journalctl@ceph.mds.g.smithi082.stdout:-- Logs begin at Sat 2023-09-09 22:51:24 UTC. -- 2023-09-09T23:11:46.139 DEBUG:teuthology.orchestra.run.smithi098:mds.b> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@mds.b.service 2023-09-09T23:11:46.142 DEBUG:teuthology.orchestra.run.smithi098:mds.e> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@mds.e.service 2023-09-09T23:11:46.165 INFO:journalctl@ceph.mds.j.smithi082.stdout:-- Logs begin at Sat 2023-09-09 22:51:24 UTC. -- 2023-09-09T23:11:46.179 INFO:journalctl@ceph.mds.b.smithi098.stdout:-- Logs begin at Sat 2023-09-09 22:51:28 UTC. -- 2023-09-09T23:11:46.181 DEBUG:teuthology.orchestra.run.smithi098:mds.h> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@mds.h.service 2023-09-09T23:11:46.210 INFO:journalctl@ceph.mds.e.smithi098.stdout:-- Logs begin at Sat 2023-09-09 22:51:28 UTC. -- 2023-09-09T23:11:46.211 DEBUG:teuthology.orchestra.run.smithi098:mds.k> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@mds.k.service 2023-09-09T23:11:46.239 INFO:journalctl@ceph.mds.h.smithi098.stdout:-- Logs begin at Sat 2023-09-09 22:51:28 UTC. -- 2023-09-09T23:11:46.240 DEBUG:teuthology.orchestra.run.smithi170:mds.c> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@mds.c.service 2023-09-09T23:11:46.242 DEBUG:teuthology.orchestra.run.smithi170:mds.f> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@mds.f.service 2023-09-09T23:11:46.266 INFO:journalctl@ceph.mds.k.smithi098.stdout:-- Logs begin at Sat 2023-09-09 22:51:28 UTC. -- 2023-09-09T23:11:46.270 INFO:journalctl@ceph.mds.c.smithi170.stdout:-- Logs begin at Sat 2023-09-09 22:51:11 UTC. -- 2023-09-09T23:11:46.271 DEBUG:teuthology.orchestra.run.smithi170:mds.i> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@mds.i.service 2023-09-09T23:11:46.292 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:46 smithi170 ceph-mon[102832]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:46.292 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:46 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:46.293 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:46 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:11:46.293 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:46 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:46.293 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:46 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:11:46.293 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:46 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:46.293 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:46 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:11:46.294 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:46 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:11:46.294 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:46 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:46.295 DEBUG:teuthology.orchestra.run.smithi170:mds.l> sudo journalctl -f -n 0 -u ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da@mds.l.service 2023-09-09T23:11:46.297 INFO:journalctl@ceph.mds.f.smithi170.stdout:-- Logs begin at Sat 2023-09-09 22:51:11 UTC. -- 2023-09-09T23:11:46.298 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-09-09T23:11:46.299 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-09T23:11:46.321 INFO:journalctl@ceph.mds.i.smithi170.stdout:-- Logs begin at Sat 2023-09-09 22:51:11 UTC. -- 2023-09-09T23:11:46.324 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:46 smithi082 ceph-mon[98694]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:46.324 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:46 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:46.324 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:46 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:11:46.324 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:46 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:46.325 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:46 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:11:46.325 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:46 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:46.325 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:46 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:11:46.325 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:46 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:11:46.325 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:46 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:46.326 INFO:journalctl@ceph.mds.l.smithi170.stdout:-- Logs begin at Sat 2023-09-09 22:51:11 UTC. -- 2023-09-09T23:11:46.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.435+0000 7f2fe0cad700 1 -- 172.21.15.82:0/3356942912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc102260 msgr2=0x7f2fdc108b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:46.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.435+0000 7f2fe0cad700 1 --2- 172.21.15.82:0/3356942912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc102260 0x7f2fdc108b10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400b0a0 tx=0x7f2fd400b3b0 comp rx=0 tx=0).stop 2023-09-09T23:11:46.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.435+0000 7f2fe0cad700 1 -- 172.21.15.82:0/3356942912 shutdown_connections 2023-09-09T23:11:46.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.435+0000 7f2fe0cad700 1 --2- 172.21.15.82:0/3356942912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc102260 0x7f2fdc108b10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:46.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.435+0000 7f2fe0cad700 1 --2- 172.21.15.82:0/3356942912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fdc0fe8f0 0x7f2fdc0fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:46.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.435+0000 7f2fe0cad700 1 --2- 172.21.15.82:0/3356942912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fdc1018c0 0x7f2fdc101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:46.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.435+0000 7f2fe0cad700 1 -- 172.21.15.82:0/3356942912 >> 172.21.15.82:0/3356942912 conn(0x7f2fdc0fa200 msgr2=0x7f2fdc0fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:46.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.435+0000 7f2fe0cad700 1 -- 172.21.15.82:0/3356942912 shutdown_connections 2023-09-09T23:11:46.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.435+0000 7f2fe0cad700 1 -- 172.21.15.82:0/3356942912 wait complete. 2023-09-09T23:11:46.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.436+0000 7f2fe0cad700 1 Processor -- start 2023-09-09T23:11:46.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.436+0000 7f2fe0cad700 1 -- start start 2023-09-09T23:11:46.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.436+0000 7f2fe0cad700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc0fe8f0 0x7f2fdc196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:46.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.436+0000 7f2fe0cad700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fdc1018c0 0x7f2fdc1968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:46.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.436+0000 7f2fe0cad700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fdc102260 0x7f2fdc19d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:46.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.436+0000 7f2fe0cad700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc19df00 con 0x7f2fdc0fe8f0 2023-09-09T23:11:46.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.436+0000 7f2fe0cad700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc19e040 con 0x7f2fdc102260 2023-09-09T23:11:46.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.436+0000 7f2fe0cad700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc19e1d0 con 0x7f2fdc1018c0 2023-09-09T23:11:46.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.436+0000 7f2fda59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc0fe8f0 0x7f2fdc196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:46.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.436+0000 7f2fda59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc0fe8f0 0x7f2fdc196380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54098/0 (socket says 172.21.15.82:54098) 2023-09-09T23:11:46.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.436+0000 7f2fda59c700 1 -- 172.21.15.82:0/1108451767 learned_addr learned my addr 172.21.15.82:0/1108451767 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:11:46.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.436+0000 7f2fd9d9b700 1 --2- 172.21.15.82:0/1108451767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fdc1018c0 0x7f2fdc1968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:46.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.436+0000 7f2fdad9d700 1 --2- 172.21.15.82:0/1108451767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fdc102260 0x7f2fdc19d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:46.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.436+0000 7f2fda59c700 1 -- 172.21.15.82:0/1108451767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fdc1018c0 msgr2=0x7f2fdc1968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:46.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.436+0000 7f2fda59c700 1 --2- 172.21.15.82:0/1108451767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fdc1018c0 0x7f2fdc1968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:46.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.436+0000 7f2fda59c700 1 -- 172.21.15.82:0/1108451767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fdc102260 msgr2=0x7f2fdc19d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:46.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.436+0000 7f2fda59c700 1 --2- 172.21.15.82:0/1108451767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fdc102260 0x7f2fdc19d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:46.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.436+0000 7f2fda59c700 1 -- 172.21.15.82:0/1108451767 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd4009d00 con 0x7f2fdc0fe8f0 2023-09-09T23:11:46.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.436+0000 7f2fdad9d700 1 --2- 172.21.15.82:0/1108451767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fdc102260 0x7f2fdc19d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:11:46.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.436+0000 7f2fda59c700 1 --2- 172.21.15.82:0/1108451767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc0fe8f0 0x7f2fdc196380 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00e970 tx=0x7f2fcc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:11:46.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.436+0000 7f2fcb7fe700 1 -- 172.21.15.82:0/1108451767 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc009790 con 0x7f2fdc0fe8f0 2023-09-09T23:11:46.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.436+0000 7f2fcb7fe700 1 -- 172.21.15.82:0/1108451767 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fcc00eec0 con 0x7f2fdc0fe8f0 2023-09-09T23:11:46.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.436+0000 7f2fcb7fe700 1 -- 172.21.15.82:0/1108451767 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc003cf0 con 0x7f2fdc0fe8f0 2023-09-09T23:11:46.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.437+0000 7f2fe0cad700 1 -- 172.21.15.82:0/1108451767 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fdc19e4b0 con 0x7f2fdc0fe8f0 2023-09-09T23:11:46.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.437+0000 7f2fe0cad700 1 -- 172.21.15.82:0/1108451767 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fdc19e9c0 con 0x7f2fdc0fe8f0 2023-09-09T23:11:46.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.438+0000 7f2fe0cad700 1 -- 172.21.15.82:0/1108451767 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fdc10be40 con 0x7f2fdc0fe8f0 2023-09-09T23:11:46.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.439+0000 7f2fcb7fe700 1 -- 172.21.15.82:0/1108451767 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f2fcc00ccd0 con 0x7f2fdc0fe8f0 2023-09-09T23:11:46.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.439+0000 7f2fcb7fe700 1 --2- 172.21.15.82:0/1108451767 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f2fc40a0710 0x7f2fc40a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:46.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.439+0000 7f2fd9d9b700 1 --2- 172.21.15.82:0/1108451767 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f2fc40a0710 0x7f2fc40a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:46.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.439+0000 7f2fcb7fe700 1 -- 172.21.15.82:0/1108451767 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2fcc014070 con 0x7f2fdc0fe8f0 2023-09-09T23:11:46.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.440+0000 7f2fd9d9b700 1 --2- 172.21.15.82:0/1108451767 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f2fc40a0710 0x7f2fc40a2bd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0009c60 tx=0x7f2fd0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:11:46.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.444+0000 7f2fcb7fe700 1 -- 172.21.15.82:0/1108451767 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2fcc0958b0 con 0x7f2fdc0fe8f0 2023-09-09T23:11:46.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:46 smithi098 ceph-mon[103124]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:46.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:46 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:46.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:46 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:11:46.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:46 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:46.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:46 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:11:46.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:46 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:46 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:11:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:46 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:11:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:46 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:46.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.636+0000 7f2fe0cad700 1 -- 172.21.15.82:0/1108451767 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2fdc10c0b0 con 0x7f2fdc0fe8f0 2023-09-09T23:11:46.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.638+0000 7f2fcb7fe700 1 -- 172.21.15.82:0/1108451767 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18374 (secure 0 0 0) 0x7f2fcc098ed0 con 0x7f2fdc0fe8f0 2023-09-09T23:11:46.636 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:11:46.637 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":77,"fsid":"3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da","created":"2023-09-09T23:05:14.222190+0000","modified":"2023-09-09T23:11:38.498415+0000","last_up_change":"2023-09-09T23:11:37.253759+0000","last_in_change":"2023-09-09T23:11:19.933812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-09T23:08:31.234958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1277eb17-a2e7-4e10-9832-f12a5e93ffde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6803","nonce":3197434929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6805","nonce":3197434929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6809","nonce":3197434929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6807","nonce":3197434929}]},"public_addr":"172.21.15.82:6803/3197434929","cluster_addr":"172.21.15.82:6805/3197434929","heartbeat_back_addr":"172.21.15.82:6809/3197434929","heartbeat_front_addr":"172.21.15.82:6807/3197434929","state":["exists","up"]},{"osd":1,"uuid":"f68723fd-0908-4e21-ae8e-1de1c3a9a65c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6801","nonce":3528641314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6803","nonce":3528641314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6807","nonce":3528641314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6805","nonce":3528641314}]},"public_addr":"172.21.15.98:6801/3528641314","cluster_addr":"172.21.15.98:6803/3528641314","heartbeat_back_addr":"172.21.15.98:6807/3528641314","heartbeat_front_addr":"172.21.15.98:6805/3528641314","state":["exists","up"]},{"osd":2,"uuid":"06f89c53-1419-4f52-acad-6e92916b476f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6801","nonce":4006414551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6803","nonce":4006414551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6807","nonce":4006414551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6805","nonce":4006414551}]},"public_addr":"172.21.15.170:6801/4006414551","cluster_addr":"172.21.15.170:6803/4006414551","heartbeat_back_addr":"172.21.15.170:6807/4006414551","heartbeat_front_addr":"172.21.15.170:6805/4006414551","state":["exists","up"]},{"osd":3,"uuid":"fedbd4c9-6852-42b5-8f3a-ba8d9bfe5d59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6811","nonce":3647629241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6813","nonce":3647629241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6817","nonce":3647629241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6815","nonce":3647629241}]},"public_addr":"172.21.15.82:6811/3647629241","cluster_addr":"172.21.15.82:6813/3647629241","heartbeat_back_addr":"172.21.15.82:6817/3647629241","heartbeat_front_addr":"172.21.15.82:6815/3647629241","state":["exists","up"]},{"osd":4,"uuid":"a71f3084-6919-4161-87bf-345f3355d8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6809","nonce":2507035643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6811","nonce":2507035643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6815","nonce":2507035643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6813","nonce":2507035643}]},"public_addr":"172.21.15.98:6809/2507035643","cluster_addr":"172.21.15.98:6811/2507035643","heartbeat_back_addr":"172.21.15.98:6815/2507035643","heartbeat_front_addr":"172.21.15.98:6813/2507035643","state":["exists","up"]},{"osd":5,"uuid":"f76058e9-a90e-476e-87f0-3c6723795e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6809","nonce":775005168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6811","nonce":775005168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6815","nonce":775005168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6813","nonce":775005168}]},"public_addr":"172.21.15.170:6809/775005168","cluster_addr":"172.21.15.170:6811/775005168","heartbeat_back_addr":"172.21.15.170:6815/775005168","heartbeat_front_addr":"172.21.15.170:6813/775005168","state":["exists","up"]},{"osd":6,"uuid":"25940be0-c1bd-4c72-95bf-fd8367aa2f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6819","nonce":3008405982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6821","nonce":3008405982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6825","nonce":3008405982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6823","nonce":3008405982}]},"public_addr":"172.21.15.82:6819/3008405982","cluster_addr":"172.21.15.82:6821/3008405982","heartbeat_back_addr":"172.21.15.82:6825/3008405982","heartbeat_front_addr":"172.21.15.82:6823/3008405982","state":["exists","up"]},{"osd":7,"uuid":"f651bb8c-0b25-42d5-b028-58563f4c27f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6817","nonce":3547615284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6819","nonce":3547615284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6823","nonce":3547615284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6821","nonce":3547615284}]},"public_addr":"172.21.15.98:6817/3547615284","cluster_addr":"172.21.15.98:6819/3547615284","heartbeat_back_addr":"172.21.15.98:6823/3547615284","heartbeat_front_addr":"172.21.15.98:6821/3547615284","state":["exists","up"]},{"osd":8,"uuid":"23b27c86-8c98-441e-b7ab-d57bc584e58f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6817","nonce":632038848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6819","nonce":632038848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6823","nonce":632038848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6821","nonce":632038848}]},"public_addr":"172.21.15.170:6817/632038848","cluster_addr":"172.21.15.170:6819/632038848","heartbeat_back_addr":"172.21.15.170:6823/632038848","heartbeat_front_addr":"172.21.15.170:6821/632038848","state":["exists","up"]},{"osd":9,"uuid":"383b8989-4922-4eee-9c77-f198258ac243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6827","nonce":3354242311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6829","nonce":3354242311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6833","nonce":3354242311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6831","nonce":3354242311}]},"public_addr":"172.21.15.82:6827/3354242311","cluster_addr":"172.21.15.82:6829/3354242311","heartbeat_back_addr":"172.21.15.82:6833/3354242311","heartbeat_front_addr":"172.21.15.82:6831/3354242311","state":["exists","up"]},{"osd":10,"uuid":"33a59389-7ced-42d5-bf54-2071317132d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6825","nonce":2069114959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6827","nonce":2069114959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6831","nonce":2069114959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6829","nonce":2069114959}]},"public_addr":"172.21.15.98:6825/2069114959","cluster_addr":"172.21.15.98:6827/2069114959","heartbeat_back_addr":"172.21.15.98:6831/2069114959","heartbeat_front_addr":"172.21.15.98:6829/2069114959","state":["exists","up"]},{"osd":11,"uuid":"b7a71f5c-85e2-4fec-be4b-95be0788b302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6825","nonce":146964512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6827","nonce":146964512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6831","nonce":146964512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6829","nonce":146964512}]},"public_addr":"172.21.15.170:6825/146964512","cluster_addr":"172.21.15.170:6827/146964512","heartbeat_back_addr":"172.21.15.170:6831/146964512","heartbeat_front_addr":"172.21.15.170:6829/146964512","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:07:47.563043+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:08.359260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:29.070374+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:49.897775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:11.434248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:31.538464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:52.377109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:13.452968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:33.806897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:54.083177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:14.980087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:35.690089+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4013747788":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6800/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/2823341464":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/75116806":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/1037625632":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/3142088817":"2023-09-10T23:05:46.550354+0000","172.21.15.82:6800/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/1679622963":"2023-09-10T23:05:46.550354+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-09T23:11:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.640+0000 7f2fe0cad700 1 -- 172.21.15.82:0/1108451767 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f2fc40a0710 msgr2=0x7f2fc40a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.640+0000 7f2fe0cad700 1 --2- 172.21.15.82:0/1108451767 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f2fc40a0710 0x7f2fc40a2bd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0009c60 tx=0x7f2fd0009260 comp rx=0 tx=0).stop 2023-09-09T23:11:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.640+0000 7f2fe0cad700 1 -- 172.21.15.82:0/1108451767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc0fe8f0 msgr2=0x7f2fdc196380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.640+0000 7f2fe0cad700 1 --2- 172.21.15.82:0/1108451767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc0fe8f0 0x7f2fdc196380 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00e970 tx=0x7f2fcc00ec80 comp rx=0 tx=0).stop 2023-09-09T23:11:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.640+0000 7f2fe0cad700 1 -- 172.21.15.82:0/1108451767 shutdown_connections 2023-09-09T23:11:46.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.640+0000 7f2fe0cad700 1 --2- 172.21.15.82:0/1108451767 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f2fc40a0710 0x7f2fc40a2bd0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:46.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.640+0000 7f2fe0cad700 1 --2- 172.21.15.82:0/1108451767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fdc102260 0x7f2fdc19d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:46.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.640+0000 7f2fe0cad700 1 --2- 172.21.15.82:0/1108451767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc0fe8f0 0x7f2fdc196380 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:46.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.640+0000 7f2fe0cad700 1 --2- 172.21.15.82:0/1108451767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fdc1018c0 0x7f2fdc1968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:46.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.640+0000 7f2fe0cad700 1 -- 172.21.15.82:0/1108451767 >> 172.21.15.82:0/1108451767 conn(0x7f2fdc0fa200 msgr2=0x7f2fdc0fc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:46.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.640+0000 7f2fe0cad700 1 -- 172.21.15.82:0/1108451767 shutdown_connections 2023-09-09T23:11:46.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.640+0000 7f2fe0cad700 1 -- 172.21.15.82:0/1108451767 wait complete. 2023-09-09T23:11:46.651 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-09T23:08:31.234958+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-09T23:11:46.652 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-09T23:11:46.816 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.816+0000 7f4047883700 1 -- 172.21.15.82:0/3484674638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4040101870 msgr2=0x7f4040101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:46.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.816+0000 7f4047883700 1 --2- 172.21.15.82:0/3484674638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4040101870 0x7f4040101cd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f403c009990 tx=0x7f403c009ca0 comp rx=0 tx=0).stop 2023-09-09T23:11:46.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.816+0000 7f4047883700 1 -- 172.21.15.82:0/3484674638 shutdown_connections 2023-09-09T23:11:46.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.816+0000 7f4047883700 1 --2- 172.21.15.82:0/3484674638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4040102210 0x7f4040108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:46.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.816+0000 7f4047883700 1 --2- 172.21.15.82:0/3484674638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40400fe8a0 0x7f40400fecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:46.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.816+0000 7f4047883700 1 --2- 172.21.15.82:0/3484674638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4040101870 0x7f4040101cd0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:46.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.816+0000 7f4047883700 1 -- 172.21.15.82:0/3484674638 >> 172.21.15.82:0/3484674638 conn(0x7f40400fa1d0 msgr2=0x7f40400fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:46.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.816+0000 7f4047883700 1 -- 172.21.15.82:0/3484674638 shutdown_connections 2023-09-09T23:11:46.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.816+0000 7f4047883700 1 -- 172.21.15.82:0/3484674638 wait complete. 2023-09-09T23:11:46.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.817+0000 7f4047883700 1 Processor -- start 2023-09-09T23:11:46.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.817+0000 7f4047883700 1 -- start start 2023-09-09T23:11:46.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.817+0000 7f4047883700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40400fe8a0 0x7f40401011e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:46.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.817+0000 7f4047883700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4040101870 0x7f40400ff820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:46.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.817+0000 7f4047883700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4040102210 0x7f40400ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:46.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.817+0000 7f4047883700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4040100500 con 0x7f4040102210 2023-09-09T23:11:46.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.817+0000 7f4047883700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4040100670 con 0x7f40400fe8a0 2023-09-09T23:11:46.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.817+0000 7f4047883700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40401a0370 con 0x7f4040101870 2023-09-09T23:11:46.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.817+0000 7f4045e20700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4040102210 0x7f40400ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:46.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.817+0000 7f4045e20700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4040102210 0x7f40400ffd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54116/0 (socket says 172.21.15.82:54116) 2023-09-09T23:11:46.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.817+0000 7f4045e20700 1 -- 172.21.15.82:0/1172543688 learned_addr learned my addr 172.21.15.82:0/1172543688 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:11:46.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.817+0000 7f4045e20700 1 -- 172.21.15.82:0/1172543688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4040101870 msgr2=0x7f40400ff820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:46.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.817+0000 7f4045e20700 1 --2- 172.21.15.82:0/1172543688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4040101870 0x7f40400ff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:46.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.817+0000 7f4045e20700 1 -- 172.21.15.82:0/1172543688 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40400fe8a0 msgr2=0x7f40401011e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:46.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.817+0000 7f4045e20700 1 --2- 172.21.15.82:0/1172543688 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40400fe8a0 0x7f40401011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:46.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.817+0000 7f4045e20700 1 -- 172.21.15.82:0/1172543688 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403c0095f0 con 0x7f4040102210 2023-09-09T23:11:46.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.817+0000 7f4045e20700 1 --2- 172.21.15.82:0/1172543688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4040102210 0x7f40400ffd60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f403800ada0 tx=0x7f403800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:11:46.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.818+0000 7f40327fc700 1 -- 172.21.15.82:0/1172543688 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403800f070 con 0x7f4040102210 2023-09-09T23:11:46.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.818+0000 7f40327fc700 1 -- 172.21.15.82:0/1172543688 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4038007b30 con 0x7f4040102210 2023-09-09T23:11:46.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.818+0000 7f4047883700 1 -- 172.21.15.82:0/1172543688 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40401a0650 con 0x7f4040102210 2023-09-09T23:11:46.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.818+0000 7f40327fc700 1 -- 172.21.15.82:0/1172543688 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4038005440 con 0x7f4040102210 2023-09-09T23:11:46.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.818+0000 7f4047883700 1 -- 172.21.15.82:0/1172543688 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40401a0b60 con 0x7f4040102210 2023-09-09T23:11:46.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.819+0000 7f4047883700 1 -- 172.21.15.82:0/1172543688 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f404010bdf0 con 0x7f4040102210 2023-09-09T23:11:46.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.820+0000 7f40327fc700 1 -- 172.21.15.82:0/1172543688 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f4038004ad0 con 0x7f4040102210 2023-09-09T23:11:46.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.820+0000 7f40327fc700 1 --2- 172.21.15.82:0/1172543688 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f402c0a06c0 0x7f402c0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:46.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.821+0000 7f404561f700 1 --2- 172.21.15.82:0/1172543688 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f402c0a06c0 0x7f402c0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:46.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.821+0000 7f40327fc700 1 -- 172.21.15.82:0/1172543688 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f40380cb300 con 0x7f4040102210 2023-09-09T23:11:46.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.821+0000 7f404561f700 1 --2- 172.21.15.82:0/1172543688 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f402c0a06c0 0x7f402c0a2b80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4040067560 tx=0x7f4034009470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:11:46.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:46.824+0000 7f40327fc700 1 -- 172.21.15.82:0/1172543688 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4038095360 con 0x7f4040102210 2023-09-09T23:11:47.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.015+0000 7f4047883700 1 -- 172.21.15.82:0/1172543688 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4040107630 con 0x7f4040102210 2023-09-09T23:11:47.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.016+0000 7f40327fc700 1 -- 172.21.15.82:0/1172543688 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f403801a310 con 0x7f4040102210 2023-09-09T23:11:47.015 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2023-09-09T23:11:47.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.018+0000 7f4047883700 1 -- 172.21.15.82:0/1172543688 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f402c0a06c0 msgr2=0x7f402c0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:47.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.018+0000 7f4047883700 1 --2- 172.21.15.82:0/1172543688 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f402c0a06c0 0x7f402c0a2b80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4040067560 tx=0x7f4034009470 comp rx=0 tx=0).stop 2023-09-09T23:11:47.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.018+0000 7f4047883700 1 -- 172.21.15.82:0/1172543688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4040102210 msgr2=0x7f40400ffd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:47.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.018+0000 7f4047883700 1 --2- 172.21.15.82:0/1172543688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4040102210 0x7f40400ffd60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f403800ada0 tx=0x7f403800c330 comp rx=0 tx=0).stop 2023-09-09T23:11:47.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.018+0000 7f4047883700 1 -- 172.21.15.82:0/1172543688 shutdown_connections 2023-09-09T23:11:47.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.018+0000 7f4047883700 1 --2- 172.21.15.82:0/1172543688 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f402c0a06c0 0x7f402c0a2b80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:47.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.018+0000 7f4047883700 1 --2- 172.21.15.82:0/1172543688 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40400fe8a0 0x7f40401011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:47.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.018+0000 7f4047883700 1 --2- 172.21.15.82:0/1172543688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4040102210 0x7f40400ffd60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:47.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.018+0000 7f4047883700 1 --2- 172.21.15.82:0/1172543688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4040101870 0x7f40400ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:47.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.018+0000 7f4047883700 1 -- 172.21.15.82:0/1172543688 >> 172.21.15.82:0/1172543688 conn(0x7f40400fa1d0 msgr2=0x7f40400fc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:47.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.018+0000 7f4047883700 1 -- 172.21.15.82:0/1172543688 shutdown_connections 2023-09-09T23:11:47.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.018+0000 7f4047883700 1 -- 172.21.15.82:0/1172543688 wait complete. 2023-09-09T23:11:47.028 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-09T23:11:47.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.151+0000 7fb77fa64700 1 -- 172.21.15.82:0/2116841638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7781000b0 msgr2=0x7fb778107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:47.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.151+0000 7fb77fa64700 1 --2- 172.21.15.82:0/2116841638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7781000b0 0x7fb778107950 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb77000b0a0 tx=0x7fb77000b3b0 comp rx=0 tx=0).stop 2023-09-09T23:11:47.153 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.152+0000 7fb77fa64700 1 -- 172.21.15.82:0/2116841638 shutdown_connections 2023-09-09T23:11:47.153 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.152+0000 7fb77fa64700 1 --2- 172.21.15.82:0/2116841638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7781000b0 0x7fb778107950 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:47.153 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.152+0000 7fb77fa64700 1 --2- 172.21.15.82:0/2116841638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb778100aa0 0x7fb778100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:47.153 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.152+0000 7fb77fa64700 1 --2- 172.21.15.82:0/2116841638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7780ff6f0 0x7fb7780ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:47.153 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.152+0000 7fb77fa64700 1 -- 172.21.15.82:0/2116841638 >> 172.21.15.82:0/2116841638 conn(0x7fb7780fa1c0 msgr2=0x7fb7780fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:47.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.152+0000 7fb77fa64700 1 -- 172.21.15.82:0/2116841638 shutdown_connections 2023-09-09T23:11:47.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.152+0000 7fb77fa64700 1 -- 172.21.15.82:0/2116841638 wait complete. 2023-09-09T23:11:47.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.152+0000 7fb77fa64700 1 Processor -- start 2023-09-09T23:11:47.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.152+0000 7fb77fa64700 1 -- start start 2023-09-09T23:11:47.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.152+0000 7fb77fa64700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7780ff6f0 0x7fb778195230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:47.155 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.152+0000 7fb77fa64700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7781000b0 0x7fb778195770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:47.155 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.152+0000 7fb77fa64700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb778100aa0 0x7fb77819c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:47.155 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.152+0000 7fb77fa64700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77819cdb0 con 0x7fb7780ff6f0 2023-09-09T23:11:47.155 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.152+0000 7fb77fa64700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77819cef0 con 0x7fb7781000b0 2023-09-09T23:11:47.155 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.152+0000 7fb77fa64700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77819d080 con 0x7fb778100aa0 2023-09-09T23:11:47.156 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.153+0000 7fb77d800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7780ff6f0 0x7fb778195230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:47.156 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.153+0000 7fb77d800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7780ff6f0 0x7fb778195230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54126/0 (socket says 172.21.15.82:54126) 2023-09-09T23:11:47.156 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.153+0000 7fb77d800700 1 -- 172.21.15.82:0/1127005032 learned_addr learned my addr 172.21.15.82:0/1127005032 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:11:47.156 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.153+0000 7fb77e001700 1 --2- 172.21.15.82:0/1127005032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb778100aa0 0x7fb77819c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:47.156 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.153+0000 7fb77cfff700 1 --2- 172.21.15.82:0/1127005032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7781000b0 0x7fb778195770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:47.157 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.153+0000 7fb77d800700 1 -- 172.21.15.82:0/1127005032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb778100aa0 msgr2=0x7fb77819c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:47.157 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.153+0000 7fb77d800700 1 --2- 172.21.15.82:0/1127005032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb778100aa0 0x7fb77819c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:47.157 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.153+0000 7fb77d800700 1 -- 172.21.15.82:0/1127005032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7781000b0 msgr2=0x7fb778195770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:47.157 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.153+0000 7fb77d800700 1 --2- 172.21.15.82:0/1127005032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7781000b0 0x7fb778195770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:47.157 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.153+0000 7fb77d800700 1 -- 172.21.15.82:0/1127005032 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb770009d00 con 0x7fb7780ff6f0 2023-09-09T23:11:47.158 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.153+0000 7fb77d800700 1 --2- 172.21.15.82:0/1127005032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7780ff6f0 0x7fb778195230 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb77400d950 tx=0x7fb77400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:11:47.158 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.153+0000 7fb76e7fc700 1 -- 172.21.15.82:0/1127005032 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb774009940 con 0x7fb7780ff6f0 2023-09-09T23:11:47.158 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.153+0000 7fb76e7fc700 1 -- 172.21.15.82:0/1127005032 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb774010460 con 0x7fb7780ff6f0 2023-09-09T23:11:47.158 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.153+0000 7fb77fa64700 1 -- 172.21.15.82:0/1127005032 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb77819d360 con 0x7fb7780ff6f0 2023-09-09T23:11:47.158 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.153+0000 7fb76e7fc700 1 -- 172.21.15.82:0/1127005032 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77400f620 con 0x7fb7780ff6f0 2023-09-09T23:11:47.159 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.153+0000 7fb77fa64700 1 -- 172.21.15.82:0/1127005032 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb77819d870 con 0x7fb7780ff6f0 2023-09-09T23:11:47.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.154+0000 7fb77fa64700 1 -- 172.21.15.82:0/1127005032 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb778106440 con 0x7fb7780ff6f0 2023-09-09T23:11:47.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.155+0000 7fb76e7fc700 1 -- 172.21.15.82:0/1127005032 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fb7740108d0 con 0x7fb7780ff6f0 2023-09-09T23:11:47.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.156+0000 7fb76e7fc700 1 --2- 172.21.15.82:0/1127005032 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb7640a0690 0x7fb7640a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:47.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.156+0000 7fb77cfff700 1 --2- 172.21.15.82:0/1127005032 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb7640a0690 0x7fb7640a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:47.161 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.156+0000 7fb76e7fc700 1 -- 172.21.15.82:0/1127005032 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb7740cb550 con 0x7fb7780ff6f0 2023-09-09T23:11:47.161 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.156+0000 7fb77cfff700 1 --2- 172.21.15.82:0/1127005032 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb7640a0690 0x7fb7640a2b50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb768005950 tx=0x7fb7680058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:11:47.161 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.160+0000 7fb76e7fc700 1 -- 172.21.15.82:0/1127005032 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb774095520 con 0x7fb7780ff6f0 2023-09-09T23:11:47.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:47 smithi170 ceph-mon[102832]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:11:47.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:47 smithi170 ceph-mon[102832]: Saving service mds.all spec with placement smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2023-09-09T23:11:47.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:47 smithi170 ceph-mon[102832]: Deploying daemon mds.l on smithi170 2023-09-09T23:11:47.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:47 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1108451767' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:11:47.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:47 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1172543688' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:11:47.393 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.394+0000 7fb77fa64700 1 -- 172.21.15.82:0/1127005032 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb77810af60 con 0x7fb7780ff6f0 2023-09-09T23:11:47.393 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.395+0000 7fb76e7fc700 1 -- 172.21.15.82:0/1127005032 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7fb774098b40 con 0x7fb7780ff6f0 2023-09-09T23:11:47.394 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:11:47.394 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-09-09T23:11:47.395 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.397+0000 7fb77fa64700 1 -- 172.21.15.82:0/1127005032 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb7640a0690 msgr2=0x7fb7640a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:47.395 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.397+0000 7fb77fa64700 1 --2- 172.21.15.82:0/1127005032 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb7640a0690 0x7fb7640a2b50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb768005950 tx=0x7fb7680058e0 comp rx=0 tx=0).stop 2023-09-09T23:11:47.396 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.397+0000 7fb77fa64700 1 -- 172.21.15.82:0/1127005032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7780ff6f0 msgr2=0x7fb778195230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:47.396 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.397+0000 7fb77fa64700 1 --2- 172.21.15.82:0/1127005032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7780ff6f0 0x7fb778195230 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb77400d950 tx=0x7fb77400dc60 comp rx=0 tx=0).stop 2023-09-09T23:11:47.396 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.397+0000 7fb77fa64700 1 -- 172.21.15.82:0/1127005032 shutdown_connections 2023-09-09T23:11:47.396 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.397+0000 7fb77fa64700 1 --2- 172.21.15.82:0/1127005032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb778100aa0 0x7fb77819c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:47.396 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.397+0000 7fb77fa64700 1 --2- 172.21.15.82:0/1127005032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7780ff6f0 0x7fb778195230 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:47.397 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.397+0000 7fb77fa64700 1 --2- 172.21.15.82:0/1127005032 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb7640a0690 0x7fb7640a2b50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:47.397 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.397+0000 7fb77fa64700 1 --2- 172.21.15.82:0/1127005032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7781000b0 0x7fb778195770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:47.397 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.397+0000 7fb77fa64700 1 -- 172.21.15.82:0/1127005032 >> 172.21.15.82:0/1127005032 conn(0x7fb7780fa1c0 msgr2=0x7fb7780fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:47.397 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.397+0000 7fb77fa64700 1 -- 172.21.15.82:0/1127005032 shutdown_connections 2023-09-09T23:11:47.397 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:47.397+0000 7fb77fa64700 1 -- 172.21.15.82:0/1127005032 wait complete. 2023-09-09T23:11:47.398 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 1 2023-09-09T23:11:47.406 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:47 smithi082 ceph-mon[98694]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:11:47.407 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:47 smithi082 ceph-mon[98694]: Saving service mds.all spec with placement smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2023-09-09T23:11:47.407 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:47 smithi082 ceph-mon[98694]: Deploying daemon mds.l on smithi170 2023-09-09T23:11:47.407 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:47 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1108451767' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:11:47.407 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:47 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1172543688' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:11:47.494 INFO:journalctl@ceph.mds.l.smithi170.stdout:Sep 09 23:11:47 smithi170 systemd[1]: Starting Ceph mds.l for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da... 2023-09-09T23:11:47.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:47 smithi098 ceph-mon[103124]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:11:47.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:47 smithi098 ceph-mon[103124]: Saving service mds.all spec with placement smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2023-09-09T23:11:47.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:47 smithi098 ceph-mon[103124]: Deploying daemon mds.l on smithi170 2023-09-09T23:11:47.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:47 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1108451767' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:11:47.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:47 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1172543688' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:11:47.744 INFO:journalctl@ceph.mds.l.smithi170.stdout:Sep 09 23:11:47 smithi170 podman[138845]: 2023-09-09T23:11:47.745 INFO:journalctl@ceph.mds.l.smithi170.stdout:Sep 09 23:11:47 smithi170 bash[138845]: d67bbf0f7325f64f8b410f362cb2f615e812d52635e2268ee141a7ce7f15542a 2023-09-09T23:11:48.036 INFO:journalctl@ceph.mds.l.smithi170.stdout:Sep 09 23:11:47 smithi170 ceph-mds[138885]: -- 172.21.15.170:0/1574537349 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f3e7f1f1e0 con 0x55f3e7ef4800 2023-09-09T23:11:48.036 INFO:journalctl@ceph.mds.l.smithi170.stdout:Sep 09 23:11:47 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mds-l[138861]: starting mds.l at 2023-09-09T23:11:48.036 INFO:journalctl@ceph.mds.l.smithi170.stdout:Sep 09 23:11:47 smithi170 systemd[1]: Started Ceph mds.l for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da. 2023-09-09T23:11:48.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:48 smithi170 ceph-mon[102832]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:48.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:48 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1127005032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:11:48.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:48 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] up:boot 2023-09-09T23:11:48.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:48 smithi170 ceph-mon[102832]: fsmap 1 up:standby 2023-09-09T23:11:48.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:48 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-09T23:11:48.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:48 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:48.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:48 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:48 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:48 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:11:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:48 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:11:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:48 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:48.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:48 smithi082 ceph-mon[98694]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:48.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:48 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1127005032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:11:48.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:48 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] up:boot 2023-09-09T23:11:48.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:48 smithi082 ceph-mon[98694]: fsmap 1 up:standby 2023-09-09T23:11:48.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:48 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-09T23:11:48.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:48 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:48.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:48 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:48.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:48 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:48.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:48 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:11:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:48 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:11:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:48 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:48.478 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:48 smithi098 ceph-mon[103124]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:48.478 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:48 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1127005032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:11:48.478 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:48 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] up:boot 2023-09-09T23:11:48.478 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:48 smithi098 ceph-mon[103124]: fsmap 1 up:standby 2023-09-09T23:11:48.479 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:48 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-09T23:11:48.479 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:48 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:48.479 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:48 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:48.479 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:48 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:48.479 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:48 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:11:48.480 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:48 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:11:48.480 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:48 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:49.409 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-09T23:11:49.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:49 smithi098 ceph-mon[103124]: Deploying daemon mds.k on smithi098 2023-09-09T23:11:49.502 INFO:journalctl@ceph.mds.k.smithi098.stdout:Sep 09 23:11:49 smithi098 systemd[1]: Starting Ceph mds.k for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da... 2023-09-09T23:11:49.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.547+0000 7f3038753700 1 -- 172.21.15.82:0/3381672717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30300ffe60 msgr2=0x7f30301002c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:49.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.547+0000 7f3038753700 1 --2- 172.21.15.82:0/3381672717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30300ffe60 0x7f30301002c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f302c0099c0 tx=0x7f302c009cd0 comp rx=0 tx=0).stop 2023-09-09T23:11:49.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.547+0000 7f3038753700 1 -- 172.21.15.82:0/3381672717 shutdown_connections 2023-09-09T23:11:49.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.547+0000 7f3038753700 1 --2- 172.21.15.82:0/3381672717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30300fec70 0x7f30300ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:49.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.547+0000 7f3038753700 1 --2- 172.21.15.82:0/3381672717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3030100800 0x7f30301070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:49.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.547+0000 7f3038753700 1 --2- 172.21.15.82:0/3381672717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30300ffe60 0x7f30301002c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:49.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.547+0000 7f3038753700 1 -- 172.21.15.82:0/3381672717 >> 172.21.15.82:0/3381672717 conn(0x7f30300fa1c0 msgr2=0x7f30300fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:49.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.547+0000 7f3038753700 1 -- 172.21.15.82:0/3381672717 shutdown_connections 2023-09-09T23:11:49.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.547+0000 7f3038753700 1 -- 172.21.15.82:0/3381672717 wait complete. 2023-09-09T23:11:49.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.548+0000 7f3038753700 1 Processor -- start 2023-09-09T23:11:49.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.548+0000 7f3038753700 1 -- start start 2023-09-09T23:11:49.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.548+0000 7f3038753700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30300fec70 0x7f30301962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:49.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.548+0000 7f3038753700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30300ffe60 0x7f3030196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:49.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.548+0000 7f3038753700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3030100800 0x7f303019d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:49.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.548+0000 7f3038753700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303019de50 con 0x7f30300ffe60 2023-09-09T23:11:49.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.548+0000 7f3038753700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303019df90 con 0x7f30300fec70 2023-09-09T23:11:49.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.548+0000 7f3038753700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303019e120 con 0x7f3030100800 2023-09-09T23:11:49.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.548+0000 7f3035cee700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30300ffe60 0x7f3030196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:49.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.548+0000 7f30364ef700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30300fec70 0x7f30301962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:49.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.548+0000 7f3035cee700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30300ffe60 0x7f3030196810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54140/0 (socket says 172.21.15.82:54140) 2023-09-09T23:11:49.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.548+0000 7f3035cee700 1 -- 172.21.15.82:0/3334738432 learned_addr learned my addr 172.21.15.82:0/3334738432 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:11:49.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.548+0000 7f3036cf0700 1 --2- 172.21.15.82:0/3334738432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3030100800 0x7f303019d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:49.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.548+0000 7f3035cee700 1 -- 172.21.15.82:0/3334738432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3030100800 msgr2=0x7f303019d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:49.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.548+0000 7f3035cee700 1 --2- 172.21.15.82:0/3334738432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3030100800 0x7f303019d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:49.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.548+0000 7f3035cee700 1 -- 172.21.15.82:0/3334738432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30300fec70 msgr2=0x7f30301962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:49.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.548+0000 7f3035cee700 1 --2- 172.21.15.82:0/3334738432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30300fec70 0x7f30301962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:49.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.548+0000 7f3035cee700 1 -- 172.21.15.82:0/3334738432 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f302c0095f0 con 0x7f30300ffe60 2023-09-09T23:11:49.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.548+0000 7f30364ef700 1 --2- 172.21.15.82:0/3334738432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30300fec70 0x7f30301962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:11:49.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.548+0000 7f3035cee700 1 --2- 172.21.15.82:0/3334738432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30300ffe60 0x7f3030196810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f302c009990 tx=0x7f302c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:11:49.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.549+0000 7f30277fe700 1 -- 172.21.15.82:0/3334738432 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c01d070 con 0x7f30300ffe60 2023-09-09T23:11:49.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.549+0000 7f30277fe700 1 -- 172.21.15.82:0/3334738432 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f302c005030 con 0x7f30300ffe60 2023-09-09T23:11:49.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.549+0000 7f3038753700 1 -- 172.21.15.82:0/3334738432 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f303019e3a0 con 0x7f30300ffe60 2023-09-09T23:11:49.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.549+0000 7f30277fe700 1 -- 172.21.15.82:0/3334738432 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c004b30 con 0x7f30300ffe60 2023-09-09T23:11:49.559 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.549+0000 7f3038753700 1 -- 172.21.15.82:0/3334738432 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f303019e7c0 con 0x7f30300ffe60 2023-09-09T23:11:49.559 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.550+0000 7f3038753700 1 -- 172.21.15.82:0/3334738432 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f303010ad60 con 0x7f30300ffe60 2023-09-09T23:11:49.560 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.551+0000 7f30277fe700 1 -- 172.21.15.82:0/3334738432 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f302c005410 con 0x7f30300ffe60 2023-09-09T23:11:49.560 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.551+0000 7f30277fe700 1 --2- 172.21.15.82:0/3334738432 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f301c0a0690 0x7f301c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:49.560 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.551+0000 7f30277fe700 1 -- 172.21.15.82:0/3334738432 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f302c0cc950 con 0x7f30300ffe60 2023-09-09T23:11:49.560 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.551+0000 7f30364ef700 1 --2- 172.21.15.82:0/3334738432 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f301c0a0690 0x7f301c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:49.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.552+0000 7f30364ef700 1 --2- 172.21.15.82:0/3334738432 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f301c0a0690 0x7f301c0a2b50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f30300ffcc0 tx=0x7f3020009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:11:49.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.555+0000 7f30277fe700 1 -- 172.21.15.82:0/3334738432 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f302c0969b0 con 0x7f30300ffe60 2023-09-09T23:11:49.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:49 smithi170 ceph-mon[102832]: Deploying daemon mds.k on smithi098 2023-09-09T23:11:49.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:49 smithi082 ceph-mon[98694]: Deploying daemon mds.k on smithi098 2023-09-09T23:11:49.787 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.788+0000 7f3038753700 1 -- 172.21.15.82:0/3334738432 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f303019e9e0 con 0x7f30300ffe60 2023-09-09T23:11:49.787 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.789+0000 7f30277fe700 1 -- 172.21.15.82:0/3334738432 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f302c099fd0 con 0x7f30300ffe60 2023-09-09T23:11:49.788 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:11:49.788 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1931961940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1931961940},{"type":"v1","addr":"172.21.15.170:6833","nonce":1931961940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-09-09T23:11:49.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.791+0000 7f3038753700 1 -- 172.21.15.82:0/3334738432 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f301c0a0690 msgr2=0x7f301c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:49.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.791+0000 7f3038753700 1 --2- 172.21.15.82:0/3334738432 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f301c0a0690 0x7f301c0a2b50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f30300ffcc0 tx=0x7f3020009260 comp rx=0 tx=0).stop 2023-09-09T23:11:49.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.791+0000 7f3038753700 1 -- 172.21.15.82:0/3334738432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30300ffe60 msgr2=0x7f3030196810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:49.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.791+0000 7f3038753700 1 --2- 172.21.15.82:0/3334738432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30300ffe60 0x7f3030196810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f302c009990 tx=0x7f302c00f690 comp rx=0 tx=0).stop 2023-09-09T23:11:49.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.791+0000 7f3038753700 1 -- 172.21.15.82:0/3334738432 shutdown_connections 2023-09-09T23:11:49.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.791+0000 7f3038753700 1 --2- 172.21.15.82:0/3334738432 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f301c0a0690 0x7f301c0a2b50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:49.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.791+0000 7f3038753700 1 --2- 172.21.15.82:0/3334738432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30300fec70 0x7f30301962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:49.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.791+0000 7f3038753700 1 --2- 172.21.15.82:0/3334738432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3030100800 0x7f303019d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:49.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.791+0000 7f3038753700 1 --2- 172.21.15.82:0/3334738432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30300ffe60 0x7f3030196810 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:49.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.791+0000 7f3038753700 1 -- 172.21.15.82:0/3334738432 >> 172.21.15.82:0/3334738432 conn(0x7f30300fa1c0 msgr2=0x7f3030107a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:49.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.791+0000 7f3038753700 1 -- 172.21.15.82:0/3334738432 shutdown_connections 2023-09-09T23:11:49.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:49.791+0000 7f3038753700 1 -- 172.21.15.82:0/3334738432 wait complete. 2023-09-09T23:11:49.792 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 2 2023-09-09T23:11:49.878 INFO:journalctl@ceph.mds.k.smithi098.stdout:Sep 09 23:11:49 smithi098 podman[138290]: 2023-09-09T23:11:50.251 INFO:journalctl@ceph.mds.k.smithi098.stdout:Sep 09 23:11:49 smithi098 ceph-mds[138331]: -- 172.21.15.98:0/2076054636 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5567425171e0 con 0x5567424ed000 2023-09-09T23:11:50.252 INFO:journalctl@ceph.mds.k.smithi098.stdout:Sep 09 23:11:49 smithi098 bash[138290]: 2c5e8d7e40e00543751c3f33a2e5a5a19e632788419e4b359500b0c78c646f8c 2023-09-09T23:11:50.253 INFO:journalctl@ceph.mds.k.smithi098.stdout:Sep 09 23:11:49 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mds-k[138306]: starting mds.k at 2023-09-09T23:11:50.253 INFO:journalctl@ceph.mds.k.smithi098.stdout:Sep 09 23:11:49 smithi098 systemd[1]: Started Ceph mds.k for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da. 2023-09-09T23:11:50.575 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:50 smithi082 ceph-mon[98694]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:50.575 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:50 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3334738432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:11:50.576 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:50 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:50.576 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:50 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:50.576 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:50 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:50.576 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:50 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:11:50.577 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:50 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:11:50.577 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:50 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:50.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:50 smithi170 ceph-mon[102832]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:50.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:50 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3334738432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:11:50.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:50 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:50.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:50 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:50 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:50 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:11:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:50 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:11:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:50 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:50.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:50 smithi098 ceph-mon[103124]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:50.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:50 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3334738432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:11:50.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:50 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:50.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:50 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:50.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:50 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:50.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:50 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:11:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:50 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:11:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:50 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:51.455 INFO:journalctl@ceph.mds.j.smithi082.stdout:Sep 09 23:11:51 smithi082 systemd[1]: Starting Ceph mds.j for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da... 2023-09-09T23:11:51.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:51 smithi082 ceph-mon[98694]: Deploying daemon mds.j on smithi082 2023-09-09T23:11:51.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] up:boot 2023-09-09T23:11:51.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:51 smithi082 ceph-mon[98694]: fsmap 2 up:standby 2023-09-09T23:11:51.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:51 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-09T23:11:51.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:51 smithi170 ceph-mon[102832]: Deploying daemon mds.j on smithi082 2023-09-09T23:11:51.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] up:boot 2023-09-09T23:11:51.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:51 smithi170 ceph-mon[102832]: fsmap 2 up:standby 2023-09-09T23:11:51.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:51 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-09T23:11:51.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:51 smithi098 ceph-mon[103124]: Deploying daemon mds.j on smithi082 2023-09-09T23:11:51.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] up:boot 2023-09-09T23:11:51.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:51 smithi098 ceph-mon[103124]: fsmap 2 up:standby 2023-09-09T23:11:51.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:51 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-09T23:11:51.802 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-09T23:11:51.901 INFO:journalctl@ceph.mds.j.smithi082.stdout:Sep 09 23:11:51 smithi082 podman[153065]: 2023-09-09T23:11:51.901 INFO:journalctl@ceph.mds.j.smithi082.stdout:Sep 09 23:11:51 smithi082 bash[153065]: ff62a5ecd079ae981cc9e56c4a940a342d6abe47a207cc36fe2cc85a5fd35d55 2023-09-09T23:11:51.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.935+0000 7f4a5cd44700 1 -- 172.21.15.82:0/885137718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a58106300 msgr2=0x7f4a58108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:51.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.935+0000 7f4a5cd44700 1 --2- 172.21.15.82:0/885137718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a58106300 0x7f4a58108770 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4a5000b0a0 tx=0x7f4a5000b3b0 comp rx=0 tx=0).stop 2023-09-09T23:11:51.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.935+0000 7f4a5cd44700 1 -- 172.21.15.82:0/885137718 shutdown_connections 2023-09-09T23:11:51.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.935+0000 7f4a5cd44700 1 --2- 172.21.15.82:0/885137718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a58106300 0x7f4a58108770 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:51.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.935+0000 7f4a5cd44700 1 --2- 172.21.15.82:0/885137718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a581010a0 0x7f4a58103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:51.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.935+0000 7f4a5cd44700 1 --2- 172.21.15.82:0/885137718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a581039d0 0x7f4a58105dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:51.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.935+0000 7f4a5cd44700 1 -- 172.21.15.82:0/885137718 >> 172.21.15.82:0/885137718 conn(0x7f4a580faa90 msgr2=0x7f4a580fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:51.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.935+0000 7f4a5cd44700 1 -- 172.21.15.82:0/885137718 shutdown_connections 2023-09-09T23:11:51.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.935+0000 7f4a5cd44700 1 -- 172.21.15.82:0/885137718 wait complete. 2023-09-09T23:11:51.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.936+0000 7f4a5cd44700 1 Processor -- start 2023-09-09T23:11:51.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.936+0000 7f4a5cd44700 1 -- start start 2023-09-09T23:11:51.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.936+0000 7f4a5cd44700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a581010a0 0x7f4a58196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:51.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.936+0000 7f4a5cd44700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a581039d0 0x7f4a581968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:51.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.936+0000 7f4a5cd44700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a58106300 0x7f4a5819d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:51.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.936+0000 7f4a5cd44700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5819df00 con 0x7f4a581039d0 2023-09-09T23:11:51.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.936+0000 7f4a5cd44700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5819e040 con 0x7f4a581010a0 2023-09-09T23:11:51.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.936+0000 7f4a5cd44700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5819e1d0 con 0x7f4a58106300 2023-09-09T23:11:51.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.937+0000 7f4a55d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a581039d0 0x7f4a581968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:51.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.937+0000 7f4a5659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a581010a0 0x7f4a58196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:51.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.937+0000 7f4a55d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a581039d0 0x7f4a581968c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54162/0 (socket says 172.21.15.82:54162) 2023-09-09T23:11:51.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.937+0000 7f4a5659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a581010a0 0x7f4a58196380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:39724/0 (socket says 172.21.15.82:39724) 2023-09-09T23:11:51.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.937+0000 7f4a55d9b700 1 -- 172.21.15.82:0/3953004449 learned_addr learned my addr 172.21.15.82:0/3953004449 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:11:51.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.937+0000 7f4a56d9d700 1 --2- 172.21.15.82:0/3953004449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a58106300 0x7f4a5819d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:51.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.937+0000 7f4a55d9b700 1 -- 172.21.15.82:0/3953004449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a58106300 msgr2=0x7f4a5819d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:51.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.937+0000 7f4a55d9b700 1 --2- 172.21.15.82:0/3953004449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a58106300 0x7f4a5819d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:51.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.937+0000 7f4a55d9b700 1 -- 172.21.15.82:0/3953004449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a581010a0 msgr2=0x7f4a58196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:51.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.937+0000 7f4a55d9b700 1 --2- 172.21.15.82:0/3953004449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a581010a0 0x7f4a58196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:51.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.937+0000 7f4a55d9b700 1 -- 172.21.15.82:0/3953004449 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a50009d00 con 0x7f4a581039d0 2023-09-09T23:11:51.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.937+0000 7f4a5659c700 1 --2- 172.21.15.82:0/3953004449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a581010a0 0x7f4a58196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:11:51.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.937+0000 7f4a55d9b700 1 --2- 172.21.15.82:0/3953004449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a581039d0 0x7f4a581968c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800e9a0 tx=0x7f4a4800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:11:51.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.937+0000 7f4a477fe700 1 -- 172.21.15.82:0/3953004449 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a48009790 con 0x7f4a581039d0 2023-09-09T23:11:51.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.937+0000 7f4a5cd44700 1 -- 172.21.15.82:0/3953004449 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a5819e4b0 con 0x7f4a581039d0 2023-09-09T23:11:51.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.937+0000 7f4a477fe700 1 -- 172.21.15.82:0/3953004449 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a48010430 con 0x7f4a581039d0 2023-09-09T23:11:51.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.937+0000 7f4a477fe700 1 -- 172.21.15.82:0/3953004449 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a48018680 con 0x7f4a581039d0 2023-09-09T23:11:51.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.937+0000 7f4a5cd44700 1 -- 172.21.15.82:0/3953004449 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5819e9c0 con 0x7f4a581039d0 2023-09-09T23:11:51.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.938+0000 7f4a5cd44700 1 -- 172.21.15.82:0/3953004449 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a580fc610 con 0x7f4a581039d0 2023-09-09T23:11:51.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.939+0000 7f4a477fe700 1 -- 172.21.15.82:0/3953004449 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f4a480105a0 con 0x7f4a581039d0 2023-09-09T23:11:51.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.940+0000 7f4a477fe700 1 --2- 172.21.15.82:0/3953004449 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f4a400a0710 0x7f4a400a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:51.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.940+0000 7f4a5659c700 1 --2- 172.21.15.82:0/3953004449 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f4a400a0710 0x7f4a400a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:51.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.940+0000 7f4a477fe700 1 -- 172.21.15.82:0/3953004449 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4a48014070 con 0x7f4a581039d0 2023-09-09T23:11:51.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.940+0000 7f4a5659c700 1 --2- 172.21.15.82:0/3953004449 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f4a400a0710 0x7f4a400a2bd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c011490 tx=0x7f4a4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:11:51.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:51.943+0000 7f4a477fe700 1 -- 172.21.15.82:0/3953004449 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4a48095a50 con 0x7f4a581039d0 2023-09-09T23:11:52.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:52.202+0000 7f4a5cd44700 1 -- 172.21.15.82:0/3953004449 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a5802d020 con 0x7f4a581039d0 2023-09-09T23:11:52.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:52.203+0000 7f4a477fe700 1 -- 172.21.15.82:0/3953004449 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7f4a48099070 con 0x7f4a581039d0 2023-09-09T23:11:52.202 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:11:52.202 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1931961940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1931961940},{"type":"v1","addr":"172.21.15.170:6833","nonce":1931961940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3855146755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3855146755},{"type":"v1","addr":"172.21.15.98:6833","nonce":3855146755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-09-09T23:11:52.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:52.205+0000 7f4a5cd44700 1 -- 172.21.15.82:0/3953004449 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f4a400a0710 msgr2=0x7f4a400a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:52.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:52.205+0000 7f4a5cd44700 1 --2- 172.21.15.82:0/3953004449 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f4a400a0710 0x7f4a400a2bd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c011490 tx=0x7f4a4c009260 comp rx=0 tx=0).stop 2023-09-09T23:11:52.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:52.205+0000 7f4a5cd44700 1 -- 172.21.15.82:0/3953004449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a581039d0 msgr2=0x7f4a581968c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:52.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:52.205+0000 7f4a5cd44700 1 --2- 172.21.15.82:0/3953004449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a581039d0 0x7f4a581968c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800e9a0 tx=0x7f4a4800ecb0 comp rx=0 tx=0).stop 2023-09-09T23:11:52.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:52.205+0000 7f4a5cd44700 1 -- 172.21.15.82:0/3953004449 shutdown_connections 2023-09-09T23:11:52.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:52.205+0000 7f4a5cd44700 1 --2- 172.21.15.82:0/3953004449 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f4a400a0710 0x7f4a400a2bd0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:52.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:52.205+0000 7f4a5cd44700 1 --2- 172.21.15.82:0/3953004449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a581010a0 0x7f4a58196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:52.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:52.205+0000 7f4a5cd44700 1 --2- 172.21.15.82:0/3953004449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a58106300 0x7f4a5819d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:52.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:52.205+0000 7f4a5cd44700 1 --2- 172.21.15.82:0/3953004449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a581039d0 0x7f4a581968c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:52.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:52.205+0000 7f4a5cd44700 1 -- 172.21.15.82:0/3953004449 >> 172.21.15.82:0/3953004449 conn(0x7f4a580faa90 msgr2=0x7f4a58101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:52.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:52.205+0000 7f4a5cd44700 1 -- 172.21.15.82:0/3953004449 shutdown_connections 2023-09-09T23:11:52.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:52.205+0000 7f4a5cd44700 1 -- 172.21.15.82:0/3953004449 wait complete. 2023-09-09T23:11:52.206 INFO:journalctl@ceph.mds.j.smithi082.stdout:Sep 09 23:11:51 smithi082 ceph-mds[153129]: -- 172.21.15.82:0/691395873 <== mon.0 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5608589011e0 con 0x5608588d6800 2023-09-09T23:11:52.207 INFO:journalctl@ceph.mds.j.smithi082.stdout:Sep 09 23:11:51 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mds-j[153081]: starting mds.j at 2023-09-09T23:11:52.207 INFO:journalctl@ceph.mds.j.smithi082.stdout:Sep 09 23:11:51 smithi082 systemd[1]: Started Ceph mds.j for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da. 2023-09-09T23:11:52.207 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 3 2023-09-09T23:11:52.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:52 smithi170 ceph-mon[102832]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:52.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:52 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:52.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:52 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:52.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:52 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:52.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:52 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:11:52.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:52 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:11:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:52 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:52 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3953004449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:11:52.701 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:52 smithi082 ceph-mon[98694]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:52.701 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:52 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:52.701 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:52 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:52.702 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:52 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:52.702 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:52 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:11:52.702 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:52 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:11:52.702 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:52 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:52.703 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:52 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3953004449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:11:52.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:52 smithi098 ceph-mon[103124]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:52.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:52.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:52.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:52.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:11:52.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:11:52.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:52 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:52 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3953004449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:11:53.523 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:53 smithi082 ceph-mon[98694]: Deploying daemon mds.d on smithi082 2023-09-09T23:11:53.524 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.82:6834/2471432684,v1:172.21.15.82:6835/2471432684] up:boot 2023-09-09T23:11:53.524 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:53 smithi082 ceph-mon[98694]: fsmap 3 up:standby 2023-09-09T23:11:53.524 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:53 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-09T23:11:53.524 INFO:journalctl@ceph.mds.d.smithi082.stdout:Sep 09 23:11:53 smithi082 systemd[1]: Starting Ceph mds.d for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da... 2023-09-09T23:11:53.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:53 smithi170 ceph-mon[102832]: Deploying daemon mds.d on smithi082 2023-09-09T23:11:53.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.82:6834/2471432684,v1:172.21.15.82:6835/2471432684] up:boot 2023-09-09T23:11:53.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:53 smithi170 ceph-mon[102832]: fsmap 3 up:standby 2023-09-09T23:11:53.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:53 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-09T23:11:53.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:53 smithi098 ceph-mon[103124]: Deploying daemon mds.d on smithi082 2023-09-09T23:11:53.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.82:6834/2471432684,v1:172.21.15.82:6835/2471432684] up:boot 2023-09-09T23:11:53.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:53 smithi098 ceph-mon[103124]: fsmap 3 up:standby 2023-09-09T23:11:53.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:53 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-09T23:11:54.075 INFO:journalctl@ceph.mds.d.smithi082.stdout:Sep 09 23:11:53 smithi082 podman[153454]: 2023-09-09T23:11:54.217 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-09T23:11:54.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.349+0000 7f4e9c6ef700 1 -- 172.21.15.82:0/2336434992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e940ff6b0 msgr2=0x7f4e940ffb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:54.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.349+0000 7f4e9c6ef700 1 --2- 172.21.15.82:0/2336434992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e940ff6b0 0x7f4e940ffb30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4e90009990 tx=0x7f4e90009ca0 comp rx=0 tx=0).stop 2023-09-09T23:11:54.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.349+0000 7f4e9c6ef700 1 -- 172.21.15.82:0/2336434992 shutdown_connections 2023-09-09T23:11:54.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.349+0000 7f4e9c6ef700 1 --2- 172.21.15.82:0/2336434992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e94100af0 0x7f4e94100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:54.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.349+0000 7f4e9c6ef700 1 --2- 172.21.15.82:0/2336434992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e94100070 0x7f4e94107910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:54.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.349+0000 7f4e9c6ef700 1 --2- 172.21.15.82:0/2336434992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e940ff6b0 0x7f4e940ffb30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:54.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.349+0000 7f4e9c6ef700 1 -- 172.21.15.82:0/2336434992 >> 172.21.15.82:0/2336434992 conn(0x7f4e940fa140 msgr2=0x7f4e940fc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:54.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.349+0000 7f4e9c6ef700 1 -- 172.21.15.82:0/2336434992 shutdown_connections 2023-09-09T23:11:54.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.350+0000 7f4e9c6ef700 1 -- 172.21.15.82:0/2336434992 wait complete. 2023-09-09T23:11:54.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.350+0000 7f4e9c6ef700 1 Processor -- start 2023-09-09T23:11:54.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.350+0000 7f4e9c6ef700 1 -- start start 2023-09-09T23:11:54.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.350+0000 7f4e9c6ef700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e940ff6b0 0x7f4e941950b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:54.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.351+0000 7f4e9c6ef700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e94100070 0x7f4e941955f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:54.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.351+0000 7f4e9c6ef700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e94100af0 0x7f4e9419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:54.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.351+0000 7f4e9c6ef700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9419cc30 con 0x7f4e94100af0 2023-09-09T23:11:54.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.351+0000 7f4e9c6ef700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9419cd70 con 0x7f4e940ff6b0 2023-09-09T23:11:54.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.351+0000 7f4e9c6ef700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9419cf00 con 0x7f4e94100070 2023-09-09T23:11:54.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.351+0000 7f4e9ac8c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e94100af0 0x7f4e9419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:54.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.351+0000 7f4e9ac8c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e94100af0 0x7f4e9419c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54192/0 (socket says 172.21.15.82:54192) 2023-09-09T23:11:54.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.351+0000 7f4e9ac8c700 1 -- 172.21.15.82:0/1301914842 learned_addr learned my addr 172.21.15.82:0/1301914842 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:11:54.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.351+0000 7f4e9a48b700 1 --2- 172.21.15.82:0/1301914842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e940ff6b0 0x7f4e941950b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:54.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.351+0000 7f4e9ac8c700 1 -- 172.21.15.82:0/1301914842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e94100070 msgr2=0x7f4e941955f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:54.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.351+0000 7f4e9ac8c700 1 --2- 172.21.15.82:0/1301914842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e94100070 0x7f4e941955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:54.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.351+0000 7f4e9ac8c700 1 -- 172.21.15.82:0/1301914842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e940ff6b0 msgr2=0x7f4e941950b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:54.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.351+0000 7f4e9ac8c700 1 --2- 172.21.15.82:0/1301914842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e940ff6b0 0x7f4e941950b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:54.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.351+0000 7f4e9ac8c700 1 -- 172.21.15.82:0/1301914842 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e900095f0 con 0x7f4e94100af0 2023-09-09T23:11:54.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.351+0000 7f4e9ac8c700 1 --2- 172.21.15.82:0/1301914842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e94100af0 0x7f4e9419c660 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c007d70 tx=0x7f4e8c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:11:54.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.351+0000 7f4e877fe700 1 -- 172.21.15.82:0/1301914842 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c00ad40 con 0x7f4e94100af0 2023-09-09T23:11:54.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.351+0000 7f4e877fe700 1 -- 172.21.15.82:0/1301914842 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e8c0128e0 con 0x7f4e94100af0 2023-09-09T23:11:54.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.351+0000 7f4e9c6ef700 1 -- 172.21.15.82:0/1301914842 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e9419d1e0 con 0x7f4e94100af0 2023-09-09T23:11:54.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.351+0000 7f4e877fe700 1 -- 172.21.15.82:0/1301914842 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c0116b0 con 0x7f4e94100af0 2023-09-09T23:11:54.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.351+0000 7f4e9c6ef700 1 -- 172.21.15.82:0/1301914842 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e9419d6a0 con 0x7f4e94100af0 2023-09-09T23:11:54.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.352+0000 7f4e9c6ef700 1 -- 172.21.15.82:0/1301914842 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e94106a10 con 0x7f4e94100af0 2023-09-09T23:11:54.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.353+0000 7f4e877fe700 1 -- 172.21.15.82:0/1301914842 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f4e8c012420 con 0x7f4e94100af0 2023-09-09T23:11:54.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.354+0000 7f4e877fe700 1 --2- 172.21.15.82:0/1301914842 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f4e800a0690 0x7f4e800a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:54.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.354+0000 7f4e877fe700 1 -- 172.21.15.82:0/1301914842 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4e8c0cbd80 con 0x7f4e94100af0 2023-09-09T23:11:54.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.354+0000 7f4e9a48b700 1 --2- 172.21.15.82:0/1301914842 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f4e800a0690 0x7f4e800a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:54.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.354+0000 7f4e9a48b700 1 --2- 172.21.15.82:0/1301914842 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f4e800a0690 0x7f4e800a2b50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4e88009c60 tx=0x7f4e88009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:11:54.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.357+0000 7f4e877fe700 1 -- 172.21.15.82:0/1301914842 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4e8c095d50 con 0x7f4e94100af0 2023-09-09T23:11:54.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:54 smithi082 ceph-mon[98694]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:54.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:54 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:54.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:54 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:54.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:54 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:54.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:54 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:11:54.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:54 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:11:54.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:54 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:54.456 INFO:journalctl@ceph.mds.d.smithi082.stdout:Sep 09 23:11:54 smithi082 ceph-mds[153494]: -- 172.21.15.82:0/2950479858 <== mon.0 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564ce041d1e0 con 0x564ce03f3000 2023-09-09T23:11:54.457 INFO:journalctl@ceph.mds.d.smithi082.stdout:Sep 09 23:11:54 smithi082 bash[153454]: 1ded25eb76b9883395a8f6568a383b0ffc1dabc581f70053543e7e238b88fc6c 2023-09-09T23:11:54.457 INFO:journalctl@ceph.mds.d.smithi082.stdout:Sep 09 23:11:54 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mds-d[153470]: starting mds.d at 2023-09-09T23:11:54.457 INFO:journalctl@ceph.mds.d.smithi082.stdout:Sep 09 23:11:54 smithi082 systemd[1]: Started Ceph mds.d for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da. 2023-09-09T23:11:54.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.595+0000 7f4e9c6ef700 1 -- 172.21.15.82:0/1301914842 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e9402cd50 con 0x7f4e94100af0 2023-09-09T23:11:54.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.595+0000 7f4e877fe700 1 -- 172.21.15.82:0/1301914842 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3701 (secure 0 0 0) 0x7f4e8c016e90 con 0x7f4e94100af0 2023-09-09T23:11:54.593 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:11:54.594 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2471432684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2471432684},{"type":"v1","addr":"172.21.15.82:6835","nonce":2471432684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3579387137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3579387137},{"type":"v1","addr":"172.21.15.82:6837","nonce":3579387137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1931961940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1931961940},{"type":"v1","addr":"172.21.15.170:6833","nonce":1931961940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3855146755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3855146755},{"type":"v1","addr":"172.21.15.98:6833","nonce":3855146755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-09-09T23:11:54.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.597+0000 7f4e9c6ef700 1 -- 172.21.15.82:0/1301914842 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f4e800a0690 msgr2=0x7f4e800a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:54.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.597+0000 7f4e9c6ef700 1 --2- 172.21.15.82:0/1301914842 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f4e800a0690 0x7f4e800a2b50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4e88009c60 tx=0x7f4e88009260 comp rx=0 tx=0).stop 2023-09-09T23:11:54.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.597+0000 7f4e9c6ef700 1 -- 172.21.15.82:0/1301914842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e94100af0 msgr2=0x7f4e9419c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:54.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.597+0000 7f4e9c6ef700 1 --2- 172.21.15.82:0/1301914842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e94100af0 0x7f4e9419c660 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c007d70 tx=0x7f4e8c00a340 comp rx=0 tx=0).stop 2023-09-09T23:11:54.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.597+0000 7f4e9c6ef700 1 -- 172.21.15.82:0/1301914842 shutdown_connections 2023-09-09T23:11:54.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.597+0000 7f4e9c6ef700 1 --2- 172.21.15.82:0/1301914842 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f4e800a0690 0x7f4e800a2b50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:54.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.597+0000 7f4e9c6ef700 1 --2- 172.21.15.82:0/1301914842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e940ff6b0 0x7f4e941950b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:54.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.597+0000 7f4e9c6ef700 1 --2- 172.21.15.82:0/1301914842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e94100af0 0x7f4e9419c660 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:54.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.597+0000 7f4e9c6ef700 1 --2- 172.21.15.82:0/1301914842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e94100070 0x7f4e941955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:54.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.597+0000 7f4e9c6ef700 1 -- 172.21.15.82:0/1301914842 >> 172.21.15.82:0/1301914842 conn(0x7f4e940fa140 msgr2=0x7f4e94103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:54.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.597+0000 7f4e9c6ef700 1 -- 172.21.15.82:0/1301914842 shutdown_connections 2023-09-09T23:11:54.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:54.597+0000 7f4e9c6ef700 1 -- 172.21.15.82:0/1301914842 wait complete. 2023-09-09T23:11:54.598 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 5 2023-09-09T23:11:54.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:54 smithi170 ceph-mon[102832]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:54.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:54 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:54.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:54 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:54.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:54 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:54.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:54 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:11:54.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:54 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:11:54.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:54 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:54.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:54 smithi098 ceph-mon[103124]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:54.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:54 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:54.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:54 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:54.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:54 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:54.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:54 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:11:54.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:54 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:11:54.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:54 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:55.356 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:55 smithi098 ceph-mon[103124]: Deploying daemon mds.e on smithi098 2023-09-09T23:11:55.356 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.82:6836/3579387137,v1:172.21.15.82:6837/3579387137] up:boot 2023-09-09T23:11:55.356 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:55 smithi098 ceph-mon[103124]: fsmap 4 up:standby 2023-09-09T23:11:55.356 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:55 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-09T23:11:55.357 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:55 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1301914842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:11:55.666 INFO:journalctl@ceph.mds.e.smithi098.stdout:Sep 09 23:11:55 smithi098 systemd[1]: Starting Ceph mds.e for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da... 2023-09-09T23:11:55.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:55 smithi170 ceph-mon[102832]: Deploying daemon mds.e on smithi098 2023-09-09T23:11:55.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.82:6836/3579387137,v1:172.21.15.82:6837/3579387137] up:boot 2023-09-09T23:11:55.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:55 smithi170 ceph-mon[102832]: fsmap 4 up:standby 2023-09-09T23:11:55.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:55 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-09T23:11:55.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:55 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1301914842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:11:55.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:55 smithi082 ceph-mon[98694]: Deploying daemon mds.e on smithi098 2023-09-09T23:11:55.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.82:6836/3579387137,v1:172.21.15.82:6837/3579387137] up:boot 2023-09-09T23:11:55.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:55 smithi082 ceph-mon[98694]: fsmap 4 up:standby 2023-09-09T23:11:55.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:55 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-09T23:11:55.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:55 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1301914842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:11:56.252 INFO:journalctl@ceph.mds.e.smithi098.stdout:Sep 09 23:11:55 smithi098 podman[138631]: 2023-09-09T23:11:56.608 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-09T23:11:56.638 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:56 smithi082 ceph-mon[98694]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:56.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:56 smithi170 ceph-mon[102832]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:56.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.746+0000 7f037a015700 1 -- 172.21.15.82:0/1075406057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03740fec70 msgr2=0x7f03740ff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:56.745 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.746+0000 7f037a015700 1 --2- 172.21.15.82:0/1075406057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03740fec70 0x7f03740ff090 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f035c0099c0 tx=0x7f035c009cd0 comp rx=0 tx=0).stop 2023-09-09T23:11:56.745 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.746+0000 7f037a015700 1 -- 172.21.15.82:0/1075406057 shutdown_connections 2023-09-09T23:11:56.745 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.746+0000 7f037a015700 1 --2- 172.21.15.82:0/1075406057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0374100800 0x7f03741070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:56.745 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.746+0000 7f037a015700 1 --2- 172.21.15.82:0/1075406057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03740fec70 0x7f03740ff090 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:56.745 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.746+0000 7f037a015700 1 --2- 172.21.15.82:0/1075406057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03740ffe60 0x7f03741002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:56.746 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.746+0000 7f037a015700 1 -- 172.21.15.82:0/1075406057 >> 172.21.15.82:0/1075406057 conn(0x7f03740fa1c0 msgr2=0x7f03740fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:56.746 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.746+0000 7f037a015700 1 -- 172.21.15.82:0/1075406057 shutdown_connections 2023-09-09T23:11:56.746 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.746+0000 7f037a015700 1 -- 172.21.15.82:0/1075406057 wait complete. 2023-09-09T23:11:56.746 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.747+0000 7f037a015700 1 Processor -- start 2023-09-09T23:11:56.747 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.747+0000 7f037a015700 1 -- start start 2023-09-09T23:11:56.749 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.747+0000 7f037a015700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03740fec70 0x7f0374102c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:56.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.747+0000 7f037a015700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03740ffe60 0x7f0374101240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:56.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.747+0000 7f037a015700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0374100800 0x7f0374101780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:56.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.747+0000 7f037a015700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0374101f20 con 0x7f03740ffe60 2023-09-09T23:11:56.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.747+0000 7f037a015700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0374102090 con 0x7f03740fec70 2023-09-09T23:11:56.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.747+0000 7f037a015700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03741a02e0 con 0x7f0374100800 2023-09-09T23:11:56.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.747+0000 7f0372ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03740ffe60 0x7f0374101240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:56.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.747+0000 7f0372ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03740ffe60 0x7f0374101240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54202/0 (socket says 172.21.15.82:54202) 2023-09-09T23:11:56.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.747+0000 7f0372ffd700 1 -- 172.21.15.82:0/4033654452 learned_addr learned my addr 172.21.15.82:0/4033654452 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:11:56.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.747+0000 7f0373fff700 1 --2- 172.21.15.82:0/4033654452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0374100800 0x7f0374101780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:56.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.747+0000 7f03737fe700 1 --2- 172.21.15.82:0/4033654452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03740fec70 0x7f0374102c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:56.752 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.748+0000 7f0372ffd700 1 -- 172.21.15.82:0/4033654452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0374100800 msgr2=0x7f0374101780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:56.752 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.748+0000 7f0372ffd700 1 --2- 172.21.15.82:0/4033654452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0374100800 0x7f0374101780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:56.752 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.748+0000 7f0372ffd700 1 -- 172.21.15.82:0/4033654452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03740fec70 msgr2=0x7f0374102c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:56.752 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.748+0000 7f0372ffd700 1 --2- 172.21.15.82:0/4033654452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03740fec70 0x7f0374102c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:56.752 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.748+0000 7f0372ffd700 1 -- 172.21.15.82:0/4033654452 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035c0095f0 con 0x7f03740ffe60 2023-09-09T23:11:56.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.748+0000 7f0372ffd700 1 --2- 172.21.15.82:0/4033654452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03740ffe60 0x7f0374101240 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f036400e9a0 tx=0x7f036400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:11:56.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.748+0000 7f0370ff9700 1 -- 172.21.15.82:0/4033654452 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364009790 con 0x7f03740ffe60 2023-09-09T23:11:56.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.748+0000 7f0370ff9700 1 -- 172.21.15.82:0/4033654452 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f036400cbd0 con 0x7f03740ffe60 2023-09-09T23:11:56.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.748+0000 7f037a015700 1 -- 172.21.15.82:0/4033654452 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03741a05c0 con 0x7f03740ffe60 2023-09-09T23:11:56.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.748+0000 7f0370ff9700 1 -- 172.21.15.82:0/4033654452 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364018680 con 0x7f03740ffe60 2023-09-09T23:11:56.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.748+0000 7f037a015700 1 -- 172.21.15.82:0/4033654452 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03741a0b20 con 0x7f03740ffe60 2023-09-09T23:11:56.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.749+0000 7f037a015700 1 -- 172.21.15.82:0/4033654452 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f037410ad00 con 0x7f03740ffe60 2023-09-09T23:11:56.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.750+0000 7f0370ff9700 1 -- 172.21.15.82:0/4033654452 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f036400cd40 con 0x7f03740ffe60 2023-09-09T23:11:56.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.750+0000 7f0370ff9700 1 --2- 172.21.15.82:0/4033654452 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f03600a0710 0x7f03600a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:56.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.750+0000 7f03737fe700 1 --2- 172.21.15.82:0/4033654452 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f03600a0710 0x7f03600a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:56.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.750+0000 7f0370ff9700 1 -- 172.21.15.82:0/4033654452 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0364014070 con 0x7f03740ffe60 2023-09-09T23:11:56.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.751+0000 7f03737fe700 1 --2- 172.21.15.82:0/4033654452 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f03600a0710 0x7f03600a2bd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f035c009990 tx=0x7f035c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:11:56.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:56 smithi098 ceph-mon[103124]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.755+0000 7f0370ff9700 1 -- 172.21.15.82:0/4033654452 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0364051e10 con 0x7f03740ffe60 2023-09-09T23:11:56.756 INFO:journalctl@ceph.mds.e.smithi098.stdout:Sep 09 23:11:56 smithi098 ceph-mds[138671]: -- 172.21.15.98:0/3196579305 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e9467571e0 con 0x55e94672cc00 2023-09-09T23:11:56.757 INFO:journalctl@ceph.mds.e.smithi098.stdout:Sep 09 23:11:56 smithi098 bash[138631]: 07915a74d9acc3967215050b7f5f1707733c4f95ec843eae0d0bb922ec731c89 2023-09-09T23:11:56.757 INFO:journalctl@ceph.mds.e.smithi098.stdout:Sep 09 23:11:56 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mds-e[138647]: starting mds.e at 2023-09-09T23:11:56.757 INFO:journalctl@ceph.mds.e.smithi098.stdout:Sep 09 23:11:56 smithi098 systemd[1]: Started Ceph mds.e for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da. 2023-09-09T23:11:56.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.991+0000 7f037a015700 1 -- 172.21.15.82:0/4033654452 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f037410aec0 con 0x7f03740ffe60 2023-09-09T23:11:56.989 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.991+0000 7f0370ff9700 1 -- 172.21.15.82:0/4033654452 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3701 (secure 0 0 0) 0x7f0364095930 con 0x7f03740ffe60 2023-09-09T23:11:56.989 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:11:56.989 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2471432684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2471432684},{"type":"v1","addr":"172.21.15.82:6835","nonce":2471432684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3579387137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3579387137},{"type":"v1","addr":"172.21.15.82:6837","nonce":3579387137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1931961940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1931961940},{"type":"v1","addr":"172.21.15.170:6833","nonce":1931961940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3855146755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3855146755},{"type":"v1","addr":"172.21.15.98:6833","nonce":3855146755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-09-09T23:11:56.991 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.993+0000 7f037a015700 1 -- 172.21.15.82:0/4033654452 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f03600a0710 msgr2=0x7f03600a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:56.991 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.993+0000 7f037a015700 1 --2- 172.21.15.82:0/4033654452 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f03600a0710 0x7f03600a2bd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f035c009990 tx=0x7f035c01a040 comp rx=0 tx=0).stop 2023-09-09T23:11:56.991 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.993+0000 7f037a015700 1 -- 172.21.15.82:0/4033654452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03740ffe60 msgr2=0x7f0374101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:56.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.993+0000 7f037a015700 1 --2- 172.21.15.82:0/4033654452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03740ffe60 0x7f0374101240 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f036400e9a0 tx=0x7f036400ecb0 comp rx=0 tx=0).stop 2023-09-09T23:11:56.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.993+0000 7f037a015700 1 -- 172.21.15.82:0/4033654452 shutdown_connections 2023-09-09T23:11:56.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.993+0000 7f037a015700 1 --2- 172.21.15.82:0/4033654452 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f03600a0710 0x7f03600a2bd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:56.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.993+0000 7f037a015700 1 --2- 172.21.15.82:0/4033654452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03740fec70 0x7f0374102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:56.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.993+0000 7f037a015700 1 --2- 172.21.15.82:0/4033654452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0374100800 0x7f0374101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:56.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.993+0000 7f037a015700 1 --2- 172.21.15.82:0/4033654452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03740ffe60 0x7f0374101240 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:56.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.993+0000 7f037a015700 1 -- 172.21.15.82:0/4033654452 >> 172.21.15.82:0/4033654452 conn(0x7f03740fa1c0 msgr2=0x7f0374107a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:56.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.993+0000 7f037a015700 1 -- 172.21.15.82:0/4033654452 shutdown_connections 2023-09-09T23:11:56.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:56.993+0000 7f037a015700 1 -- 172.21.15.82:0/4033654452 wait complete. 2023-09-09T23:11:56.994 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 5 2023-09-09T23:11:57.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:57 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:57.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:57 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:57.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:57 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:57.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:57 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:11:57.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:57 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:11:57.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:57 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:57.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:57 smithi170 ceph-mon[102832]: Deploying daemon mds.f on smithi170 2023-09-09T23:11:57.649 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:57 smithi170 ceph-mon[102832]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:57.649 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:57 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/4033654452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:11:57.649 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:boot 2023-09-09T23:11:57.649 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:57 smithi170 ceph-mon[102832]: fsmap 5 up:standby 2023-09-09T23:11:57.649 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:57 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-09T23:11:57.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:57 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:57.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:57 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:57.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:57 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:57 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:11:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:57 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:11:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:57 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:57 smithi082 ceph-mon[98694]: Deploying daemon mds.f on smithi170 2023-09-09T23:11:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:57 smithi082 ceph-mon[98694]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:57 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/4033654452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:11:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:boot 2023-09-09T23:11:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:57 smithi082 ceph-mon[98694]: fsmap 5 up:standby 2023-09-09T23:11:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:57 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-09T23:11:57.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:57 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:57 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:57 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:57 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:11:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:57 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:11:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:57 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:57 smithi098 ceph-mon[103124]: Deploying daemon mds.f on smithi170 2023-09-09T23:11:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:57 smithi098 ceph-mon[103124]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:57 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/4033654452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:11:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:boot 2023-09-09T23:11:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:57 smithi098 ceph-mon[103124]: fsmap 5 up:standby 2023-09-09T23:11:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:57 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-09T23:11:57.926 INFO:journalctl@ceph.mds.f.smithi170.stdout:Sep 09 23:11:57 smithi170 systemd[1]: Starting Ceph mds.f for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da... 2023-09-09T23:11:58.461 INFO:journalctl@ceph.mds.f.smithi170.stdout:Sep 09 23:11:58 smithi170 podman[139185]: 2023-09-09T23:11:58.462 INFO:journalctl@ceph.mds.f.smithi170.stdout:Sep 09 23:11:58 smithi170 ceph-mds[139225]: -- 172.21.15.170:0/3824318005 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561e9dcf91e0 con 0x561e9dccf000 2023-09-09T23:11:58.925 INFO:journalctl@ceph.mds.f.smithi170.stdout:Sep 09 23:11:58 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mds-f[139201]: starting mds.f at 2023-09-09T23:11:58.925 INFO:journalctl@ceph.mds.f.smithi170.stdout:Sep 09 23:11:58 smithi170 bash[139185]: de1f3b85b2e91dde42c25c6b928602b06d48e1b10e9d00235818514a7c80ba00 2023-09-09T23:11:58.926 INFO:journalctl@ceph.mds.f.smithi170.stdout:Sep 09 23:11:58 smithi170 systemd[1]: Started Ceph mds.f for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da. 2023-09-09T23:11:59.004 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-09T23:11:59.140 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.140+0000 7f07a3215700 1 -- 172.21.15.82:0/3661751823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f079c100b70 msgr2=0x7f079c100f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:59.141 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.141+0000 7f07a3215700 1 --2- 172.21.15.82:0/3661751823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f079c100b70 0x7f079c100f90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0790009990 tx=0x7f0790009ca0 comp rx=0 tx=0).stop 2023-09-09T23:11:59.141 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.141+0000 7f07a3215700 1 -- 172.21.15.82:0/3661751823 shutdown_connections 2023-09-09T23:11:59.141 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.141+0000 7f07a3215700 1 --2- 172.21.15.82:0/3661751823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f079c1000f0 0x7f079c107990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:59.141 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.141+0000 7f07a3215700 1 --2- 172.21.15.82:0/3661751823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f079c100b70 0x7f079c100f90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:59.141 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.141+0000 7f07a3215700 1 --2- 172.21.15.82:0/3661751823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f079c0ff730 0x7f079c0ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:59.142 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.141+0000 7f07a3215700 1 -- 172.21.15.82:0/3661751823 >> 172.21.15.82:0/3661751823 conn(0x7f079c0fa1c0 msgr2=0x7f079c0fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:59.142 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.141+0000 7f07a3215700 1 -- 172.21.15.82:0/3661751823 shutdown_connections 2023-09-09T23:11:59.142 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.141+0000 7f07a3215700 1 -- 172.21.15.82:0/3661751823 wait complete. 2023-09-09T23:11:59.142 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.141+0000 7f07a3215700 1 Processor -- start 2023-09-09T23:11:59.142 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.141+0000 7f07a3215700 1 -- start start 2023-09-09T23:11:59.143 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.141+0000 7f07a3215700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f079c0ff730 0x7f079c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:59.143 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.141+0000 7f07a3215700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f079c1000f0 0x7f079c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:59.143 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.141+0000 7f07a3215700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f079c100b70 0x7f079c19fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:59.143 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.141+0000 7f07a3215700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079c1a00a0 con 0x7f079c1000f0 2023-09-09T23:11:59.143 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.141+0000 7f07a3215700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079c1a01e0 con 0x7f079c100b70 2023-09-09T23:11:59.144 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.141+0000 7f07a3215700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079c1a0370 con 0x7f079c0ff730 2023-09-09T23:11:59.144 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.141+0000 7f079bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f079c1000f0 0x7f079c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:59.144 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.142+0000 7f079bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f079c1000f0 0x7f079c198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54216/0 (socket says 172.21.15.82:54216) 2023-09-09T23:11:59.144 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.142+0000 7f079bfff700 1 -- 172.21.15.82:0/3756699049 learned_addr learned my addr 172.21.15.82:0/3756699049 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:11:59.144 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.142+0000 7f07a17b2700 1 --2- 172.21.15.82:0/3756699049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f079c100b70 0x7f079c19fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:59.145 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.142+0000 7f07a0fb1700 1 --2- 172.21.15.82:0/3756699049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f079c0ff730 0x7f079c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:59.145 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.142+0000 7f079bfff700 1 -- 172.21.15.82:0/3756699049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f079c0ff730 msgr2=0x7f079c198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:59.145 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.142+0000 7f079bfff700 1 --2- 172.21.15.82:0/3756699049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f079c0ff730 0x7f079c198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:59.145 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.142+0000 7f079bfff700 1 -- 172.21.15.82:0/3756699049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f079c100b70 msgr2=0x7f079c19fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:59.145 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.142+0000 7f079bfff700 1 --2- 172.21.15.82:0/3756699049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f079c100b70 0x7f079c19fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:59.145 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.142+0000 7f079bfff700 1 -- 172.21.15.82:0/3756699049 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07900095f0 con 0x7f079c1000f0 2023-09-09T23:11:59.146 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.142+0000 7f07a0fb1700 1 --2- 172.21.15.82:0/3756699049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f079c0ff730 0x7f079c198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:11:59.146 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.142+0000 7f07a17b2700 1 --2- 172.21.15.82:0/3756699049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f079c100b70 0x7f079c19fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:11:59.146 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.142+0000 7f079bfff700 1 --2- 172.21.15.82:0/3756699049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f079c1000f0 0x7f079c198a60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f078800d950 tx=0x7f078800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:11:59.146 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.142+0000 7f0799ffb700 1 -- 172.21.15.82:0/3756699049 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0788009b40 con 0x7f079c1000f0 2023-09-09T23:11:59.146 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.142+0000 7f0799ffb700 1 -- 172.21.15.82:0/3756699049 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0788009ca0 con 0x7f079c1000f0 2023-09-09T23:11:59.147 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.142+0000 7f07a3215700 1 -- 172.21.15.82:0/3756699049 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f079c1a0650 con 0x7f079c1000f0 2023-09-09T23:11:59.147 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.142+0000 7f0799ffb700 1 -- 172.21.15.82:0/3756699049 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078800f620 con 0x7f079c1000f0 2023-09-09T23:11:59.147 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.142+0000 7f07a3215700 1 -- 172.21.15.82:0/3756699049 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f079c1a0b60 con 0x7f079c1000f0 2023-09-09T23:11:59.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.143+0000 7f07a3215700 1 -- 172.21.15.82:0/3756699049 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f079c10ad20 con 0x7f079c1000f0 2023-09-09T23:11:59.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.144+0000 7f0799ffb700 1 -- 172.21.15.82:0/3756699049 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f0788010460 con 0x7f079c1000f0 2023-09-09T23:11:59.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.145+0000 7f0799ffb700 1 --2- 172.21.15.82:0/3756699049 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f078c0a0690 0x7f078c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:11:59.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.145+0000 7f07a0fb1700 1 --2- 172.21.15.82:0/3756699049 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f078c0a0690 0x7f078c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:11:59.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.145+0000 7f0799ffb700 1 -- 172.21.15.82:0/3756699049 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f07880cbb70 con 0x7f079c1000f0 2023-09-09T23:11:59.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.145+0000 7f07a0fb1700 1 --2- 172.21.15.82:0/3756699049 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f078c0a0690 0x7f078c0a2b50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f079000b5c0 tx=0x7f0790005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:11:59.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.149+0000 7f0799ffb700 1 -- 172.21.15.82:0/3756699049 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0788095aa0 con 0x7f079c1000f0 2023-09-09T23:11:59.384 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.387+0000 7f07a3215700 1 -- 172.21.15.82:0/3756699049 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f079c10af10 con 0x7f079c1000f0 2023-09-09T23:11:59.385 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.387+0000 7f0799ffb700 1 -- 172.21.15.82:0/3756699049 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5286 (secure 0 0 0) 0x7f07880990c0 con 0x7f079c1000f0 2023-09-09T23:11:59.385 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:11:59.386 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2471432684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2471432684},{"type":"v1","addr":"172.21.15.82:6835","nonce":2471432684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3579387137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3579387137},{"type":"v1","addr":"172.21.15.82:6837","nonce":3579387137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1931961940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1931961940},{"type":"v1","addr":"172.21.15.170:6833","nonce":1931961940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3855146755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3855146755},{"type":"v1","addr":"172.21.15.98:6833","nonce":3855146755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2313861511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2313861511},{"type":"v1","addr":"172.21.15.170:6835","nonce":2313861511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1356474828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1356474828},{"type":"v1","addr":"172.21.15.98:6835","nonce":1356474828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2023-09-09T23:11:59.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.390+0000 7f07a3215700 1 -- 172.21.15.82:0/3756699049 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f078c0a0690 msgr2=0x7f078c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:59.388 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.390+0000 7f07a3215700 1 --2- 172.21.15.82:0/3756699049 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f078c0a0690 0x7f078c0a2b50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f079000b5c0 tx=0x7f0790005fb0 comp rx=0 tx=0).stop 2023-09-09T23:11:59.388 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.390+0000 7f07a3215700 1 -- 172.21.15.82:0/3756699049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f079c1000f0 msgr2=0x7f079c198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:11:59.388 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.390+0000 7f07a3215700 1 --2- 172.21.15.82:0/3756699049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f079c1000f0 0x7f079c198a60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f078800d950 tx=0x7f078800dd10 comp rx=0 tx=0).stop 2023-09-09T23:11:59.388 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.390+0000 7f07a3215700 1 -- 172.21.15.82:0/3756699049 shutdown_connections 2023-09-09T23:11:59.388 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.390+0000 7f07a3215700 1 --2- 172.21.15.82:0/3756699049 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f078c0a0690 0x7f078c0a2b50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:59.389 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.390+0000 7f07a3215700 1 --2- 172.21.15.82:0/3756699049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f079c100b70 0x7f079c19fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:59.389 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.390+0000 7f07a3215700 1 --2- 172.21.15.82:0/3756699049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f079c0ff730 0x7f079c198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:59.389 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.390+0000 7f07a3215700 1 --2- 172.21.15.82:0/3756699049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f079c1000f0 0x7f079c198a60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:11:59.389 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.390+0000 7f07a3215700 1 -- 172.21.15.82:0/3756699049 >> 172.21.15.82:0/3756699049 conn(0x7f079c0fa1c0 msgr2=0x7f079c0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:11:59.389 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.390+0000 7f07a3215700 1 -- 172.21.15.82:0/3756699049 shutdown_connections 2023-09-09T23:11:59.389 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:11:59.390+0000 7f07a3215700 1 -- 172.21.15.82:0/3756699049 wait complete. 2023-09-09T23:11:59.390 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 7 2023-09-09T23:11:59.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:59 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:59.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:59 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:59.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:59 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:59.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:59 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:11:59.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:59 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:11:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:59 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:59 smithi082 ceph-mon[98694]: Deploying daemon mds.c on smithi170 2023-09-09T23:11:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:59 smithi082 ceph-mon[98694]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:59 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.170:6834/2313861511,v1:172.21.15.170:6835/2313861511] up:boot 2023-09-09T23:11:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:59 smithi082 ceph-mon[98694]: fsmap 6 up:standby 2023-09-09T23:11:59.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:59 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-09T23:11:59.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:11:59 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3756699049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:11:59.855 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:59 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:59.856 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:59 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:59.856 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:59 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:11:59.856 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:59 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:11:59.857 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:59 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:11:59.857 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:59 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:11:59.857 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:59 smithi170 ceph-mon[102832]: Deploying daemon mds.c on smithi170 2023-09-09T23:11:59.857 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:59 smithi170 ceph-mon[102832]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:11:59.858 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:59 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.170:6834/2313861511,v1:172.21.15.170:6835/2313861511] up:boot 2023-09-09T23:11:59.858 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:59 smithi170 ceph-mon[102832]: fsmap 6 up:standby 2023-09-09T23:11:59.858 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:59 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-09T23:11:59.858 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:11:59 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3756699049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:12:00.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:59 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:00.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:59 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:00.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:59 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:00.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:59 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:12:00.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:59 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:12:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:59 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:12:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:59 smithi098 ceph-mon[103124]: Deploying daemon mds.c on smithi170 2023-09-09T23:12:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:59 smithi098 ceph-mon[103124]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:12:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:59 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.170:6834/2313861511,v1:172.21.15.170:6835/2313861511] up:boot 2023-09-09T23:12:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:59 smithi098 ceph-mon[103124]: fsmap 6 up:standby 2023-09-09T23:12:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:59 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-09T23:12:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:11:59 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3756699049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:12:00.417 INFO:journalctl@ceph.mds.c.smithi170.stdout:Sep 09 23:12:00 smithi170 systemd[1]: Starting Ceph mds.c for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da... 2023-09-09T23:12:00.675 INFO:journalctl@ceph.mds.c.smithi170.stdout:Sep 09 23:12:00 smithi170 podman[139531]: 2023-09-09T23:12:01.176 INFO:journalctl@ceph.mds.c.smithi170.stdout:Sep 09 23:12:00 smithi170 ceph-mds[139571]: -- 172.21.15.170:0/860974823 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560de180d1e0 con 0x560de17e2c00 2023-09-09T23:12:01.177 INFO:journalctl@ceph.mds.c.smithi170.stdout:Sep 09 23:12:00 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mds-c[139547]: starting mds.c at 2023-09-09T23:12:01.177 INFO:journalctl@ceph.mds.c.smithi170.stdout:Sep 09 23:12:00 smithi170 bash[139531]: e96b4b244f2a99c460adc54c74902fd7bdf7b45ca900d43315f6033c9d576764 2023-09-09T23:12:01.177 INFO:journalctl@ceph.mds.c.smithi170.stdout:Sep 09 23:12:00 smithi170 systemd[1]: Started Ceph mds.c for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da. 2023-09-09T23:12:01.400 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-09T23:12:01.540 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.537+0000 7fa563494700 1 -- 172.21.15.82:0/279912818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa55c100b70 msgr2=0x7fa55c100f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:01.541 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.537+0000 7fa563494700 1 --2- 172.21.15.82:0/279912818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa55c100b70 0x7fa55c100f90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa550009990 tx=0x7fa550009ca0 comp rx=0 tx=0).stop 2023-09-09T23:12:01.541 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.537+0000 7fa563494700 1 -- 172.21.15.82:0/279912818 shutdown_connections 2023-09-09T23:12:01.541 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.537+0000 7fa563494700 1 --2- 172.21.15.82:0/279912818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa55c1000f0 0x7fa55c107990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:01.541 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.537+0000 7fa563494700 1 --2- 172.21.15.82:0/279912818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa55c100b70 0x7fa55c100f90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:01.542 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.537+0000 7fa563494700 1 --2- 172.21.15.82:0/279912818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa55c0ff730 0x7fa55c0ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:01.542 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.537+0000 7fa563494700 1 -- 172.21.15.82:0/279912818 >> 172.21.15.82:0/279912818 conn(0x7fa55c0fa1c0 msgr2=0x7fa55c0fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:01.542 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.537+0000 7fa563494700 1 -- 172.21.15.82:0/279912818 shutdown_connections 2023-09-09T23:12:01.543 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.537+0000 7fa563494700 1 -- 172.21.15.82:0/279912818 wait complete. 2023-09-09T23:12:01.543 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.538+0000 7fa563494700 1 Processor -- start 2023-09-09T23:12:01.543 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.538+0000 7fa563494700 1 -- start start 2023-09-09T23:12:01.543 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.538+0000 7fa563494700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa55c0ff730 0x7fa55c195170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:01.543 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.538+0000 7fa563494700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa55c1000f0 0x7fa55c1956b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:01.544 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.538+0000 7fa563494700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa55c100b70 0x7fa55c19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:01.544 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.538+0000 7fa563494700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c19ccf0 con 0x7fa55c1000f0 2023-09-09T23:12:01.544 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.538+0000 7fa563494700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c19ce30 con 0x7fa55c0ff730 2023-09-09T23:12:01.544 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.538+0000 7fa563494700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c19cfc0 con 0x7fa55c100b70 2023-09-09T23:12:01.545 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.538+0000 7fa560a2f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa55c1000f0 0x7fa55c1956b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:01.545 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.538+0000 7fa560a2f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa55c1000f0 0x7fa55c1956b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54230/0 (socket says 172.21.15.82:54230) 2023-09-09T23:12:01.545 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.538+0000 7fa560a2f700 1 -- 172.21.15.82:0/2739769042 learned_addr learned my addr 172.21.15.82:0/2739769042 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:01.545 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.538+0000 7fa561a31700 1 --2- 172.21.15.82:0/2739769042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa55c100b70 0x7fa55c19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:01.545 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.538+0000 7fa561230700 1 --2- 172.21.15.82:0/2739769042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa55c0ff730 0x7fa55c195170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:01.546 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.539+0000 7fa560a2f700 1 -- 172.21.15.82:0/2739769042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa55c100b70 msgr2=0x7fa55c19c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:01.546 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.539+0000 7fa560a2f700 1 --2- 172.21.15.82:0/2739769042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa55c100b70 0x7fa55c19c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:01.546 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.539+0000 7fa560a2f700 1 -- 172.21.15.82:0/2739769042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa55c0ff730 msgr2=0x7fa55c195170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:01.546 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.539+0000 7fa560a2f700 1 --2- 172.21.15.82:0/2739769042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa55c0ff730 0x7fa55c195170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:01.547 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.539+0000 7fa560a2f700 1 -- 172.21.15.82:0/2739769042 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5500095f0 con 0x7fa55c1000f0 2023-09-09T23:12:01.547 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.539+0000 7fa561230700 1 --2- 172.21.15.82:0/2739769042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa55c0ff730 0x7fa55c195170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:12:01.547 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.539+0000 7fa560a2f700 1 --2- 172.21.15.82:0/2739769042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa55c1000f0 0x7fa55c1956b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa558007b50 tx=0x7fa558007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:01.547 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.539+0000 7fa54e7fc700 1 -- 172.21.15.82:0/2739769042 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa558011070 con 0x7fa55c1000f0 2023-09-09T23:12:01.547 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.539+0000 7fa54e7fc700 1 -- 172.21.15.82:0/2739769042 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa55800c2e0 con 0x7fa55c1000f0 2023-09-09T23:12:01.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.539+0000 7fa563494700 1 -- 172.21.15.82:0/2739769042 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa55c0fe680 con 0x7fa55c1000f0 2023-09-09T23:12:01.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.539+0000 7fa54e7fc700 1 -- 172.21.15.82:0/2739769042 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55800e500 con 0x7fa55c1000f0 2023-09-09T23:12:01.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.539+0000 7fa563494700 1 -- 172.21.15.82:0/2739769042 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa55c0febe0 con 0x7fa55c1000f0 2023-09-09T23:12:01.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.540+0000 7fa563494700 1 -- 172.21.15.82:0/2739769042 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa55c10ad00 con 0x7fa55c1000f0 2023-09-09T23:12:01.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.541+0000 7fa54e7fc700 1 -- 172.21.15.82:0/2739769042 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fa558004750 con 0x7fa55c1000f0 2023-09-09T23:12:01.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.541+0000 7fa54e7fc700 1 --2- 172.21.15.82:0/2739769042 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa5480a0790 0x7fa5480a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:01.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.541+0000 7fa561230700 1 --2- 172.21.15.82:0/2739769042 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa5480a0790 0x7fa5480a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:01.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.542+0000 7fa54e7fc700 1 -- 172.21.15.82:0/2739769042 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa5580ccf60 con 0x7fa55c1000f0 2023-09-09T23:12:01.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.542+0000 7fa561230700 1 --2- 172.21.15.82:0/2739769042 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa5480a0790 0x7fa5480a2c50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa55000b5c0 tx=0x7fa550005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:01.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.545+0000 7fa54e7fc700 1 -- 172.21.15.82:0/2739769042 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa5580f9e10 con 0x7fa55c1000f0 2023-09-09T23:12:01.778 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.781+0000 7fa563494700 1 -- 172.21.15.82:0/2739769042 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa55c10aec0 con 0x7fa55c1000f0 2023-09-09T23:12:01.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.781+0000 7fa54e7fc700 1 -- 172.21.15.82:0/2739769042 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6080 (secure 0 0 0) 0x7fa558096f20 con 0x7fa55c1000f0 2023-09-09T23:12:01.779 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:12:01.780 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2471432684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2471432684},{"type":"v1","addr":"172.21.15.82:6835","nonce":2471432684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3579387137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3579387137},{"type":"v1","addr":"172.21.15.82:6837","nonce":3579387137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1931961940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1931961940},{"type":"v1","addr":"172.21.15.170:6833","nonce":1931961940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3855146755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3855146755},{"type":"v1","addr":"172.21.15.98:6833","nonce":3855146755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2313861511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2313861511},{"type":"v1","addr":"172.21.15.170:6835","nonce":2313861511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/4006360308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":4006360308},{"type":"v1","addr":"172.21.15.170:6837","nonce":4006360308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1356474828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1356474828},{"type":"v1","addr":"172.21.15.98:6835","nonce":1356474828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2023-09-09T23:12:01.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.783+0000 7fa563494700 1 -- 172.21.15.82:0/2739769042 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa5480a0790 msgr2=0x7fa5480a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:01.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.783+0000 7fa563494700 1 --2- 172.21.15.82:0/2739769042 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa5480a0790 0x7fa5480a2c50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa55000b5c0 tx=0x7fa550005fb0 comp rx=0 tx=0).stop 2023-09-09T23:12:01.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.783+0000 7fa563494700 1 -- 172.21.15.82:0/2739769042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa55c1000f0 msgr2=0x7fa55c1956b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:01.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.784+0000 7fa563494700 1 --2- 172.21.15.82:0/2739769042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa55c1000f0 0x7fa55c1956b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa558007b50 tx=0x7fa558007e60 comp rx=0 tx=0).stop 2023-09-09T23:12:01.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.784+0000 7fa563494700 1 -- 172.21.15.82:0/2739769042 shutdown_connections 2023-09-09T23:12:01.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.784+0000 7fa563494700 1 --2- 172.21.15.82:0/2739769042 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa5480a0790 0x7fa5480a2c50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:01.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.784+0000 7fa563494700 1 --2- 172.21.15.82:0/2739769042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa55c0ff730 0x7fa55c195170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:01.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.784+0000 7fa563494700 1 --2- 172.21.15.82:0/2739769042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa55c100b70 0x7fa55c19c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:01.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.784+0000 7fa563494700 1 --2- 172.21.15.82:0/2739769042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa55c1000f0 0x7fa55c1956b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:01.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.784+0000 7fa563494700 1 -- 172.21.15.82:0/2739769042 >> 172.21.15.82:0/2739769042 conn(0x7fa55c0fa1c0 msgr2=0x7fa55c0fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:01.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.784+0000 7fa563494700 1 -- 172.21.15.82:0/2739769042 shutdown_connections 2023-09-09T23:12:01.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:01.784+0000 7fa563494700 1 -- 172.21.15.82:0/2739769042 wait complete. 2023-09-09T23:12:01.784 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 8 2023-09-09T23:12:02.080 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:01 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:02.080 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:01 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.170:6836/4006360308,v1:172.21.15.170:6837/4006360308] up:boot 2023-09-09T23:12:02.080 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:01 smithi098 ceph-mon[103124]: fsmap 7 up:standby 2023-09-09T23:12:02.081 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:01 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:02.081 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:01 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-09T23:12:02.081 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:01 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:02.081 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:01 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:12:02.081 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:01 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:12:02.082 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:01 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:12:02.082 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:01 smithi098 ceph-mon[103124]: Deploying daemon mds.h on smithi098 2023-09-09T23:12:02.082 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:01 smithi098 ceph-mon[103124]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:12:02.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:01 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:02.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:01 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.170:6836/4006360308,v1:172.21.15.170:6837/4006360308] up:boot 2023-09-09T23:12:02.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:01 smithi170 ceph-mon[102832]: fsmap 7 up:standby 2023-09-09T23:12:02.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:01 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:02.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:01 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-09T23:12:02.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:01 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:01 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:12:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:01 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:12:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:01 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:12:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:01 smithi170 ceph-mon[102832]: Deploying daemon mds.h on smithi098 2023-09-09T23:12:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:01 smithi170 ceph-mon[102832]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:12:02.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:01 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:02.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:01 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.170:6836/4006360308,v1:172.21.15.170:6837/4006360308] up:boot 2023-09-09T23:12:02.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:01 smithi082 ceph-mon[98694]: fsmap 7 up:standby 2023-09-09T23:12:02.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:01 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:02.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:01 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-09T23:12:02.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:01 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:02.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:01 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:12:02.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:01 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:12:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:01 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:12:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:01 smithi082 ceph-mon[98694]: Deploying daemon mds.h on smithi098 2023-09-09T23:12:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:01 smithi082 ceph-mon[98694]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:12:02.501 INFO:journalctl@ceph.mds.h.smithi098.stdout:Sep 09 23:12:02 smithi098 systemd[1]: Starting Ceph mds.h for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da... 2023-09-09T23:12:02.859 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:02 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2739769042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:12:02.860 INFO:journalctl@ceph.mds.h.smithi098.stdout:Sep 09 23:12:02 smithi098 podman[138972]: 2023-09-09T23:12:03.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:02 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2739769042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:12:03.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:02 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2739769042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:12:03.251 INFO:journalctl@ceph.mds.h.smithi098.stdout:Sep 09 23:12:02 smithi098 ceph-mds[139013]: -- 172.21.15.98:0/4099978273 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559edb4211e0 con 0x559edb3f7000 2023-09-09T23:12:03.251 INFO:journalctl@ceph.mds.h.smithi098.stdout:Sep 09 23:12:02 smithi098 bash[138972]: 7d1a5706bd5b70cefd74246d07821b00f4b3a29568c12812331203d2296e97dd 2023-09-09T23:12:03.252 INFO:journalctl@ceph.mds.h.smithi098.stdout:Sep 09 23:12:02 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mds-h[138988]: starting mds.h at 2023-09-09T23:12:03.252 INFO:journalctl@ceph.mds.h.smithi098.stdout:Sep 09 23:12:02 smithi098 systemd[1]: Started Ceph mds.h for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da. 2023-09-09T23:12:03.794 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-09T23:12:03.934 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.934+0000 7fa23c4d8700 1 -- 172.21.15.82:0/2856269637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa234100610 msgr2=0x7fa234100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:03.934 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.934+0000 7fa23c4d8700 1 --2- 172.21.15.82:0/2856269637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa234100610 0x7fa234100a30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa2240099c0 tx=0x7fa224009cd0 comp rx=0 tx=0).stop 2023-09-09T23:12:03.934 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.934+0000 7fa23c4d8700 1 -- 172.21.15.82:0/2856269637 shutdown_connections 2023-09-09T23:12:03.934 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.934+0000 7fa23c4d8700 1 --2- 172.21.15.82:0/2856269637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2341021a0 0x7fa234108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:03.935 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.934+0000 7fa23c4d8700 1 --2- 172.21.15.82:0/2856269637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa234100610 0x7fa234100a30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:03.935 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.934+0000 7fa23c4d8700 1 --2- 172.21.15.82:0/2856269637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa234101800 0x7fa234101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:03.935 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.934+0000 7fa23c4d8700 1 -- 172.21.15.82:0/2856269637 >> 172.21.15.82:0/2856269637 conn(0x7fa2340fbb40 msgr2=0x7fa2340fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:03.935 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.934+0000 7fa23c4d8700 1 -- 172.21.15.82:0/2856269637 shutdown_connections 2023-09-09T23:12:03.935 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.934+0000 7fa23c4d8700 1 -- 172.21.15.82:0/2856269637 wait complete. 2023-09-09T23:12:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.935+0000 7fa23c4d8700 1 Processor -- start 2023-09-09T23:12:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.935+0000 7fa23c4d8700 1 -- start start 2023-09-09T23:12:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.935+0000 7fa23c4d8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa234100610 0x7fa2340743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.935+0000 7fa23c4d8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa234101800 0x7fa2340729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.935+0000 7fa23c4d8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2341021a0 0x7fa234072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:03.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.935+0000 7fa23c4d8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2340736d0 con 0x7fa234101800 2023-09-09T23:12:03.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.935+0000 7fa23c4d8700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa234073840 con 0x7fa2341021a0 2023-09-09T23:12:03.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.935+0000 7fa23c4d8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2341a0380 con 0x7fa234100610 2023-09-09T23:12:03.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.935+0000 7fa239a73700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa234101800 0x7fa2340729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:03.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.935+0000 7fa239a73700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa234101800 0x7fa2340729f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54244/0 (socket says 172.21.15.82:54244) 2023-09-09T23:12:03.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.935+0000 7fa239a73700 1 -- 172.21.15.82:0/2307384492 learned_addr learned my addr 172.21.15.82:0/2307384492 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:03.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.935+0000 7fa23a274700 1 --2- 172.21.15.82:0/2307384492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa234100610 0x7fa2340743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:03.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.935+0000 7fa239a73700 1 -- 172.21.15.82:0/2307384492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa234100610 msgr2=0x7fa2340743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:03.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.935+0000 7fa239a73700 1 --2- 172.21.15.82:0/2307384492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa234100610 0x7fa2340743b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:03.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.935+0000 7fa239a73700 1 -- 172.21.15.82:0/2307384492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2341021a0 msgr2=0x7fa234072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:03.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.935+0000 7fa239a73700 1 --2- 172.21.15.82:0/2307384492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2341021a0 0x7fa234072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:03.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.935+0000 7fa239a73700 1 -- 172.21.15.82:0/2307384492 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2240095f0 con 0x7fa234101800 2023-09-09T23:12:03.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.935+0000 7fa239a73700 1 --2- 172.21.15.82:0/2307384492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa234101800 0x7fa2340729f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa23000d950 tx=0x7fa23000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:03.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.935+0000 7fa22b7fe700 1 -- 172.21.15.82:0/2307384492 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa230009940 con 0x7fa234101800 2023-09-09T23:12:03.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.936+0000 7fa22b7fe700 1 -- 172.21.15.82:0/2307384492 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa230010460 con 0x7fa234101800 2023-09-09T23:12:03.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.936+0000 7fa23c4d8700 1 -- 172.21.15.82:0/2307384492 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2341a0660 con 0x7fa234101800 2023-09-09T23:12:03.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.936+0000 7fa22b7fe700 1 -- 172.21.15.82:0/2307384492 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23000f620 con 0x7fa234101800 2023-09-09T23:12:03.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.936+0000 7fa23c4d8700 1 -- 172.21.15.82:0/2307384492 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2341a0b70 con 0x7fa234101800 2023-09-09T23:12:03.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.936+0000 7fa23c4d8700 1 -- 172.21.15.82:0/2307384492 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa234107bb0 con 0x7fa234101800 2023-09-09T23:12:03.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.937+0000 7fa22b7fe700 1 -- 172.21.15.82:0/2307384492 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fa2300108d0 con 0x7fa234101800 2023-09-09T23:12:03.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.938+0000 7fa22b7fe700 1 --2- 172.21.15.82:0/2307384492 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa2200a0690 0x7fa2200a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:03.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.938+0000 7fa22b7fe700 1 -- 172.21.15.82:0/2307384492 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa2300cb570 con 0x7fa234101800 2023-09-09T23:12:03.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.938+0000 7fa23a274700 1 --2- 172.21.15.82:0/2307384492 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa2200a0690 0x7fa2200a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:03.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.938+0000 7fa23a274700 1 --2- 172.21.15.82:0/2307384492 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa2200a0690 0x7fa2200a2b50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa224009f50 tx=0x7fa22400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:03.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:03.941+0000 7fa22b7fe700 1 -- 172.21.15.82:0/2307384492 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa230095540 con 0x7fa234101800 2023-09-09T23:12:04.175 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:04.178+0000 7fa23c4d8700 1 -- 172.21.15.82:0/2307384492 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa23410be40 con 0x7fa234101800 2023-09-09T23:12:04.176 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:04.179+0000 7fa22b7fe700 1 -- 172.21.15.82:0/2307384492 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6871 (secure 0 0 0) 0x7fa230098b60 con 0x7fa234101800 2023-09-09T23:12:04.177 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:12:04.177 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2471432684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2471432684},{"type":"v1","addr":"172.21.15.82:6835","nonce":2471432684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3579387137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3579387137},{"type":"v1","addr":"172.21.15.82:6837","nonce":3579387137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1931961940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1931961940},{"type":"v1","addr":"172.21.15.170:6833","nonce":1931961940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3855146755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3855146755},{"type":"v1","addr":"172.21.15.98:6833","nonce":3855146755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2313861511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2313861511},{"type":"v1","addr":"172.21.15.170:6835","nonce":2313861511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/4006360308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":4006360308},{"type":"v1","addr":"172.21.15.170:6837","nonce":4006360308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1356474828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1356474828},{"type":"v1","addr":"172.21.15.98:6835","nonce":1356474828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2019449840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2019449840},{"type":"v1","addr":"172.21.15.98:6837","nonce":2019449840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-09-09T23:12:04.179 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:04.181+0000 7fa23c4d8700 1 -- 172.21.15.82:0/2307384492 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa2200a0690 msgr2=0x7fa2200a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:04.179 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:04.181+0000 7fa23c4d8700 1 --2- 172.21.15.82:0/2307384492 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa2200a0690 0x7fa2200a2b50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa224009f50 tx=0x7fa22400b540 comp rx=0 tx=0).stop 2023-09-09T23:12:04.179 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:04.181+0000 7fa23c4d8700 1 -- 172.21.15.82:0/2307384492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa234101800 msgr2=0x7fa2340729f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:04.179 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:04.181+0000 7fa23c4d8700 1 --2- 172.21.15.82:0/2307384492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa234101800 0x7fa2340729f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa23000d950 tx=0x7fa23000dc60 comp rx=0 tx=0).stop 2023-09-09T23:12:04.180 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:04.181+0000 7fa23c4d8700 1 -- 172.21.15.82:0/2307384492 shutdown_connections 2023-09-09T23:12:04.180 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:04.181+0000 7fa23c4d8700 1 --2- 172.21.15.82:0/2307384492 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa2200a0690 0x7fa2200a2b50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:04.180 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:04.181+0000 7fa23c4d8700 1 --2- 172.21.15.82:0/2307384492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2341021a0 0x7fa234072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:04.180 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:04.181+0000 7fa23c4d8700 1 --2- 172.21.15.82:0/2307384492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa234100610 0x7fa2340743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:04.180 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:04.181+0000 7fa23c4d8700 1 --2- 172.21.15.82:0/2307384492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa234101800 0x7fa2340729f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:04.181 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:04.181+0000 7fa23c4d8700 1 -- 172.21.15.82:0/2307384492 >> 172.21.15.82:0/2307384492 conn(0x7fa2340fbb40 msgr2=0x7fa234104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:04.181 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:04.181+0000 7fa23c4d8700 1 -- 172.21.15.82:0/2307384492 shutdown_connections 2023-09-09T23:12:04.181 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:04.181+0000 7fa23c4d8700 1 -- 172.21.15.82:0/2307384492 wait complete. 2023-09-09T23:12:04.181 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 9 2023-09-09T23:12:04.190 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:03 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:04.190 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:03 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:04.190 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:03 smithi082 ceph-mon[98694]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:12:04.190 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:03 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:04.191 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:03 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:12:04.191 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:03 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:12:04.191 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:03 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:12:04.191 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:03 smithi082 ceph-mon[98694]: Deploying daemon mds.i on smithi170 2023-09-09T23:12:04.192 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:03 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:boot 2023-09-09T23:12:04.192 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:03 smithi082 ceph-mon[98694]: fsmap 8 up:standby 2023-09-09T23:12:04.192 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:03 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-09T23:12:04.226 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:03 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:04.226 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:03 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:04.226 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:03 smithi170 ceph-mon[102832]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:12:04.227 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:03 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:04.227 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:03 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:12:04.227 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:03 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:12:04.227 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:03 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:12:04.227 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:03 smithi170 ceph-mon[102832]: Deploying daemon mds.i on smithi170 2023-09-09T23:12:04.228 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:03 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:boot 2023-09-09T23:12:04.228 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:03 smithi170 ceph-mon[102832]: fsmap 8 up:standby 2023-09-09T23:12:04.228 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:03 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-09T23:12:04.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:03 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:04.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:03 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:04.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:03 smithi098 ceph-mon[103124]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:12:04.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:03 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:04.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:03 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:12:04.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:03 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:12:04.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:03 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:12:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:03 smithi098 ceph-mon[103124]: Deploying daemon mds.i on smithi170 2023-09-09T23:12:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:03 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:boot 2023-09-09T23:12:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:03 smithi098 ceph-mon[103124]: fsmap 8 up:standby 2023-09-09T23:12:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:03 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-09T23:12:04.874 INFO:journalctl@ceph.mds.i.smithi170.stdout:Sep 09 23:12:04 smithi170 systemd[1]: Starting Ceph mds.i for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da... 2023-09-09T23:12:05.125 INFO:journalctl@ceph.mds.i.smithi170.stdout:Sep 09 23:12:04 smithi170 podman[139874]: 2023-09-09T23:12:05.126 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:04 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2307384492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:12:05.204 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:04 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2307384492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:12:05.251 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:04 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2307384492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:12:05.425 INFO:journalctl@ceph.mds.i.smithi170.stdout:Sep 09 23:12:05 smithi170 ceph-mds[139914]: -- 172.21.15.170:0/1710382623 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5559f34711e0 con 0x5559f3446c00 2023-09-09T23:12:05.425 INFO:journalctl@ceph.mds.i.smithi170.stdout:Sep 09 23:12:05 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mds-i[139890]: starting mds.i at 2023-09-09T23:12:05.426 INFO:journalctl@ceph.mds.i.smithi170.stdout:Sep 09 23:12:05 smithi170 bash[139874]: ae36f5b1195c1a3493815d1edbef8c4a7d5d6a706a53d74995ede14861c25daa 2023-09-09T23:12:05.426 INFO:journalctl@ceph.mds.i.smithi170.stdout:Sep 09 23:12:05 smithi170 systemd[1]: Started Ceph mds.i for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da. 2023-09-09T23:12:06.194 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-09T23:12:06.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.324+0000 7f1a36972700 1 -- 172.21.15.82:0/3335285311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a301010c0 msgr2=0x7f1a30107970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:06.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.324+0000 7f1a36972700 1 --2- 172.21.15.82:0/3335285311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a301010c0 0x7f1a30107970 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1a2800b0d0 tx=0x7f1a2800b3e0 comp rx=0 tx=0).stop 2023-09-09T23:12:06.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.324+0000 7f1a36972700 1 -- 172.21.15.82:0/3335285311 shutdown_connections 2023-09-09T23:12:06.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.324+0000 7f1a36972700 1 --2- 172.21.15.82:0/3335285311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a301010c0 0x7f1a30107970 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:06.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.324+0000 7f1a36972700 1 --2- 172.21.15.82:0/3335285311 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a300ff530 0x7f1a300ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:06.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.324+0000 7f1a36972700 1 --2- 172.21.15.82:0/3335285311 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a30100720 0x7f1a30100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:06.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.324+0000 7f1a36972700 1 -- 172.21.15.82:0/3335285311 >> 172.21.15.82:0/3335285311 conn(0x7f1a300faa80 msgr2=0x7f1a300fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:06.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.324+0000 7f1a36972700 1 -- 172.21.15.82:0/3335285311 shutdown_connections 2023-09-09T23:12:06.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.324+0000 7f1a36972700 1 -- 172.21.15.82:0/3335285311 wait complete. 2023-09-09T23:12:06.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.325+0000 7f1a36972700 1 Processor -- start 2023-09-09T23:12:06.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.325+0000 7f1a36972700 1 -- start start 2023-09-09T23:12:06.591 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:06 smithi098 ceph-mon[103124]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:12:06.592 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:06 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:06.592 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:06 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:06.592 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:06 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:06.592 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:06 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:12:06.593 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:06 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:12:06.593 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:06 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:12:06.593 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:06 smithi098 ceph-mon[103124]: Deploying daemon mds.g on smithi082 2023-09-09T23:12:06.593 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] up:boot 2023-09-09T23:12:06.594 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:06 smithi098 ceph-mon[103124]: fsmap 9 up:standby 2023-09-09T23:12:06.594 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:06 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-09T23:12:06.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.325+0000 7f1a36972700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a300ff530 0x7f1a30069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:06.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.325+0000 7f1a36972700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a30100720 0x7f1a3019b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:06.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.325+0000 7f1a36972700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a301010c0 0x7f1a3019d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:06.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.325+0000 7f1a36972700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3019de90 con 0x7f1a301010c0 2023-09-09T23:12:06.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.325+0000 7f1a36972700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3019e000 con 0x7f1a30100720 2023-09-09T23:12:06.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.325+0000 7f1a36972700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3019e1c0 con 0x7f1a300ff530 2023-09-09T23:12:06.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.325+0000 7f1a34f0f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a301010c0 0x7f1a3019d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:06.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.325+0000 7f1a34f0f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a301010c0 0x7f1a3019d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54258/0 (socket says 172.21.15.82:54258) 2023-09-09T23:12:06.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.325+0000 7f1a34f0f700 1 -- 172.21.15.82:0/3787518279 learned_addr learned my addr 172.21.15.82:0/3787518279 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:06.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.325+0000 7f1a27fff700 1 --2- 172.21.15.82:0/3787518279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a30100720 0x7f1a3019b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:06.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.325+0000 7f1a2ffff700 1 --2- 172.21.15.82:0/3787518279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a300ff530 0x7f1a30069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:06.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.325+0000 7f1a34f0f700 1 -- 172.21.15.82:0/3787518279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a300ff530 msgr2=0x7f1a30069530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:06.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.325+0000 7f1a34f0f700 1 --2- 172.21.15.82:0/3787518279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a300ff530 0x7f1a30069530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:06.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.325+0000 7f1a34f0f700 1 -- 172.21.15.82:0/3787518279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a30100720 msgr2=0x7f1a3019b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:06.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.325+0000 7f1a34f0f700 1 --2- 172.21.15.82:0/3787518279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a30100720 0x7f1a3019b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:06.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.325+0000 7f1a34f0f700 1 -- 172.21.15.82:0/3787518279 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a28009d00 con 0x7f1a301010c0 2023-09-09T23:12:06.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.325+0000 7f1a34f0f700 1 --2- 172.21.15.82:0/3787518279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a301010c0 0x7f1a3019d6f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1a28007a00 tx=0x7f1a28009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:06.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.326+0000 7f1a2dffb700 1 -- 172.21.15.82:0/3787518279 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2800e040 con 0x7f1a301010c0 2023-09-09T23:12:06.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.326+0000 7f1a2dffb700 1 -- 172.21.15.82:0/3787518279 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a28003e10 con 0x7f1a301010c0 2023-09-09T23:12:06.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.326+0000 7f1a36972700 1 -- 172.21.15.82:0/3787518279 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a3019e440 con 0x7f1a301010c0 2023-09-09T23:12:06.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.326+0000 7f1a2dffb700 1 -- 172.21.15.82:0/3787518279 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a28005610 con 0x7f1a301010c0 2023-09-09T23:12:06.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.326+0000 7f1a36972700 1 -- 172.21.15.82:0/3787518279 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a3019e860 con 0x7f1a301010c0 2023-09-09T23:12:06.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.327+0000 7f1a36972700 1 -- 172.21.15.82:0/3787518279 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a301069e0 con 0x7f1a301010c0 2023-09-09T23:12:06.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.328+0000 7f1a2dffb700 1 -- 172.21.15.82:0/3787518279 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f1a28005770 con 0x7f1a301010c0 2023-09-09T23:12:06.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.328+0000 7f1a2dffb700 1 --2- 172.21.15.82:0/3787518279 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1a1c0a0710 0x7f1a1c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:06.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.328+0000 7f1a2ffff700 1 --2- 172.21.15.82:0/3787518279 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1a1c0a0710 0x7f1a1c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:06.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.328+0000 7f1a2dffb700 1 -- 172.21.15.82:0/3787518279 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1a280cd750 con 0x7f1a301010c0 2023-09-09T23:12:06.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.329+0000 7f1a2ffff700 1 --2- 172.21.15.82:0/3787518279 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1a1c0a0710 0x7f1a1c0a2bd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1a30100580 tx=0x7f1a20005cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:06.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.332+0000 7f1a2dffb700 1 -- 172.21.15.82:0/3787518279 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1a28097710 con 0x7f1a301010c0 2023-09-09T23:12:06.603 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:06 smithi082 ceph-mon[98694]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:12:06.604 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:06 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:06.604 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:06 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:06.604 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:06 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:06.604 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:06 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:12:06.604 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:06 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:12:06.604 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:06 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:12:06.605 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:06 smithi082 ceph-mon[98694]: Deploying daemon mds.g on smithi082 2023-09-09T23:12:06.605 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] up:boot 2023-09-09T23:12:06.605 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:06 smithi082 ceph-mon[98694]: fsmap 9 up:standby 2023-09-09T23:12:06.605 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:06 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-09T23:12:06.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.592+0000 7f1a36972700 1 -- 172.21.15.82:0/3787518279 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a3002d040 con 0x7f1a301010c0 2023-09-09T23:12:06.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.593+0000 7f1a2dffb700 1 -- 172.21.15.82:0/3787518279 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7667 (secure 0 0 0) 0x7f1a2809ad30 con 0x7f1a301010c0 2023-09-09T23:12:06.606 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:12:06.607 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2471432684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2471432684},{"type":"v1","addr":"172.21.15.82:6835","nonce":2471432684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3579387137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3579387137},{"type":"v1","addr":"172.21.15.82:6837","nonce":3579387137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1931961940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1931961940},{"type":"v1","addr":"172.21.15.170:6833","nonce":1931961940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3855146755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3855146755},{"type":"v1","addr":"172.21.15.98:6833","nonce":3855146755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2313861511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2313861511},{"type":"v1","addr":"172.21.15.170:6835","nonce":2313861511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/4006360308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":4006360308},{"type":"v1","addr":"172.21.15.170:6837","nonce":4006360308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1356474828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1356474828},{"type":"v1","addr":"172.21.15.98:6835","nonce":1356474828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24523,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/3858369735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3858369735},{"type":"v1","addr":"172.21.15.170:6839","nonce":3858369735}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2019449840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2019449840},{"type":"v1","addr":"172.21.15.98:6837","nonce":2019449840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-09-09T23:12:06.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.595+0000 7f1a36972700 1 -- 172.21.15.82:0/3787518279 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1a1c0a0710 msgr2=0x7f1a1c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:06.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.595+0000 7f1a36972700 1 --2- 172.21.15.82:0/3787518279 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1a1c0a0710 0x7f1a1c0a2bd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1a30100580 tx=0x7f1a20005cf0 comp rx=0 tx=0).stop 2023-09-09T23:12:06.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.595+0000 7f1a36972700 1 -- 172.21.15.82:0/3787518279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a301010c0 msgr2=0x7f1a3019d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:06.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.595+0000 7f1a36972700 1 --2- 172.21.15.82:0/3787518279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a301010c0 0x7f1a3019d6f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1a28007a00 tx=0x7f1a28009570 comp rx=0 tx=0).stop 2023-09-09T23:12:06.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.595+0000 7f1a36972700 1 -- 172.21.15.82:0/3787518279 shutdown_connections 2023-09-09T23:12:06.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.595+0000 7f1a36972700 1 --2- 172.21.15.82:0/3787518279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a301010c0 0x7f1a3019d6f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:06.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.595+0000 7f1a36972700 1 --2- 172.21.15.82:0/3787518279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a300ff530 0x7f1a30069530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:06.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.595+0000 7f1a36972700 1 --2- 172.21.15.82:0/3787518279 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1a1c0a0710 0x7f1a1c0a2bd0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:06.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.595+0000 7f1a36972700 1 --2- 172.21.15.82:0/3787518279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a30100720 0x7f1a3019b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:06.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.595+0000 7f1a36972700 1 -- 172.21.15.82:0/3787518279 >> 172.21.15.82:0/3787518279 conn(0x7f1a300faa80 msgr2=0x7f1a30103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:06.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.596+0000 7f1a36972700 1 -- 172.21.15.82:0/3787518279 shutdown_connections 2023-09-09T23:12:06.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:06.596+0000 7f1a36972700 1 -- 172.21.15.82:0/3787518279 wait complete. 2023-09-09T23:12:06.612 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 10 2023-09-09T23:12:06.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:06 smithi170 ceph-mon[102832]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:12:06.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:06 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:06.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:06 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:06.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:06 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:06 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:12:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:06 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:12:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:06 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:12:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:06 smithi170 ceph-mon[102832]: Deploying daemon mds.g on smithi082 2023-09-09T23:12:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] up:boot 2023-09-09T23:12:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:06 smithi170 ceph-mon[102832]: fsmap 9 up:standby 2023-09-09T23:12:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:06 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-09T23:12:06.955 INFO:journalctl@ceph.mds.g.smithi082.stdout:Sep 09 23:12:06 smithi082 systemd[1]: Starting Ceph mds.g for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da... 2023-09-09T23:12:07.236 INFO:journalctl@ceph.mds.g.smithi082.stdout:Sep 09 23:12:06 smithi082 podman[154064]: 2023-09-09T23:12:07.236 INFO:journalctl@ceph.mds.g.smithi082.stdout:Sep 09 23:12:07 smithi082 ceph-mds[154104]: -- 172.21.15.82:0/2794462159 <== mon.0 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cc147311e0 con 0x55cc14706c00 2023-09-09T23:12:07.237 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:07 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3787518279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:12:07.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:07 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3787518279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:12:07.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:07 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3787518279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:12:07.705 INFO:journalctl@ceph.mds.g.smithi082.stdout:Sep 09 23:12:07 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mds-g[154080]: starting mds.g at 2023-09-09T23:12:07.705 INFO:journalctl@ceph.mds.g.smithi082.stdout:Sep 09 23:12:07 smithi082 bash[154064]: 78166051d7ce231e2c4b7180d9a909b3923d949b2c1da854cee629683bc4e1a6 2023-09-09T23:12:07.705 INFO:journalctl@ceph.mds.g.smithi082.stdout:Sep 09 23:12:07 smithi082 systemd[1]: Started Ceph mds.g for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da. 2023-09-09T23:12:08.332 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:08 smithi082 ceph-mon[98694]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:12:08.332 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:08 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:08.333 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:08 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:08.333 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:08 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:08.333 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:08 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:12:08.333 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:08 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:12:08.334 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:08 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:12:08.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:08 smithi098 ceph-mon[103124]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:12:08.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:08 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:08.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:08 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:08.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:08 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:08.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:08 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:12:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:08 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:12:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:08 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:12:08.613 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-09T23:12:08.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:08 smithi170 ceph-mon[102832]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:12:08.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:08 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:08.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:08 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:08.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:08 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:08.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:08 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:12:08.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:08 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:12:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:08 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:12:08.752 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.755+0000 7f1a73939700 1 -- 172.21.15.82:0/2052572173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a6c106300 msgr2=0x7f1a6c108730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:08.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.755+0000 7f1a73939700 1 --2- 172.21.15.82:0/2052572173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a6c106300 0x7f1a6c108730 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1a6400b0a0 tx=0x7f1a6400b3b0 comp rx=0 tx=0).stop 2023-09-09T23:12:08.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.755+0000 7f1a73939700 1 -- 172.21.15.82:0/2052572173 shutdown_connections 2023-09-09T23:12:08.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.755+0000 7f1a73939700 1 --2- 172.21.15.82:0/2052572173 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a6c1010a0 0x7f1a6c103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:08.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.755+0000 7f1a73939700 1 --2- 172.21.15.82:0/2052572173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a6c106300 0x7f1a6c108730 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:08.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.755+0000 7f1a73939700 1 --2- 172.21.15.82:0/2052572173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a6c1039d0 0x7f1a6c105dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:08.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.755+0000 7f1a73939700 1 -- 172.21.15.82:0/2052572173 >> 172.21.15.82:0/2052572173 conn(0x7f1a6c0faa30 msgr2=0x7f1a6c0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:08.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.755+0000 7f1a73939700 1 -- 172.21.15.82:0/2052572173 shutdown_connections 2023-09-09T23:12:08.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.755+0000 7f1a73939700 1 -- 172.21.15.82:0/2052572173 wait complete. 2023-09-09T23:12:08.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.755+0000 7f1a73939700 1 Processor -- start 2023-09-09T23:12:08.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.755+0000 7f1a73939700 1 -- start start 2023-09-09T23:12:08.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.756+0000 7f1a73939700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a6c1010a0 0x7f1a6c1009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:08.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.756+0000 7f1a73939700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a6c1039d0 0x7f1a6c0ff050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:08.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.756+0000 7f1a73939700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a6c106300 0x7f1a6c0ff590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:08.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.756+0000 7f1a73939700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c0ffd30 con 0x7f1a6c1039d0 2023-09-09T23:12:08.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.756+0000 7f1a73939700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c0ffea0 con 0x7f1a6c106300 2023-09-09T23:12:08.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.756+0000 7f1a73939700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c1a0460 con 0x7f1a6c1010a0 2023-09-09T23:12:08.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.756+0000 7f1a70ed4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a6c1039d0 0x7f1a6c0ff050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:08.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.756+0000 7f1a70ed4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a6c1039d0 0x7f1a6c0ff050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54286/0 (socket says 172.21.15.82:54286) 2023-09-09T23:12:08.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.756+0000 7f1a70ed4700 1 -- 172.21.15.82:0/1137629412 learned_addr learned my addr 172.21.15.82:0/1137629412 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:08.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.756+0000 7f1a71ed6700 1 --2- 172.21.15.82:0/1137629412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a6c106300 0x7f1a6c0ff590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:08.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.756+0000 7f1a716d5700 1 --2- 172.21.15.82:0/1137629412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a6c1010a0 0x7f1a6c1009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:08.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.756+0000 7f1a70ed4700 1 -- 172.21.15.82:0/1137629412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a6c1010a0 msgr2=0x7f1a6c1009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:08.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.756+0000 7f1a70ed4700 1 --2- 172.21.15.82:0/1137629412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a6c1010a0 0x7f1a6c1009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:08.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.756+0000 7f1a70ed4700 1 -- 172.21.15.82:0/1137629412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a6c106300 msgr2=0x7f1a6c0ff590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:08.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.756+0000 7f1a70ed4700 1 --2- 172.21.15.82:0/1137629412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a6c106300 0x7f1a6c0ff590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:08.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.756+0000 7f1a70ed4700 1 -- 172.21.15.82:0/1137629412 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a64009d00 con 0x7f1a6c1039d0 2023-09-09T23:12:08.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.756+0000 7f1a71ed6700 1 --2- 172.21.15.82:0/1137629412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a6c106300 0x7f1a6c0ff590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:12:08.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.756+0000 7f1a70ed4700 1 --2- 172.21.15.82:0/1137629412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a6c1039d0 0x7f1a6c0ff050 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1a6800c920 tx=0x7f1a6800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:08.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.756+0000 7f1a5e7fc700 1 -- 172.21.15.82:0/1137629412 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a68007aa0 con 0x7f1a6c1039d0 2023-09-09T23:12:08.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.756+0000 7f1a5e7fc700 1 -- 172.21.15.82:0/1137629412 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a68004cf0 con 0x7f1a6c1039d0 2023-09-09T23:12:08.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.756+0000 7f1a73939700 1 -- 172.21.15.82:0/1137629412 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a6c1a06f0 con 0x7f1a6c1039d0 2023-09-09T23:12:08.762 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.756+0000 7f1a5e7fc700 1 -- 172.21.15.82:0/1137629412 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a68005550 con 0x7f1a6c1039d0 2023-09-09T23:12:08.762 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.756+0000 7f1a73939700 1 -- 172.21.15.82:0/1137629412 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a6c1a0c50 con 0x7f1a6c1039d0 2023-09-09T23:12:08.762 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.757+0000 7f1a73939700 1 -- 172.21.15.82:0/1137629412 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a6c0fc5b0 con 0x7f1a6c1039d0 2023-09-09T23:12:08.762 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.758+0000 7f1a5e7fc700 1 -- 172.21.15.82:0/1137629412 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f1a68004750 con 0x7f1a6c1039d0 2023-09-09T23:12:08.762 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.759+0000 7f1a5e7fc700 1 --2- 172.21.15.82:0/1137629412 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1a580a0710 0x7f1a580a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:08.763 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.759+0000 7f1a716d5700 1 --2- 172.21.15.82:0/1137629412 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1a580a0710 0x7f1a580a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:08.763 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.759+0000 7f1a5e7fc700 1 -- 172.21.15.82:0/1137629412 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1a680cb620 con 0x7f1a6c1039d0 2023-09-09T23:12:08.763 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.759+0000 7f1a716d5700 1 --2- 172.21.15.82:0/1137629412 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1a580a0710 0x7f1a580a2bd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1a60005950 tx=0x7f1a600058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:08.763 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:08.763+0000 7f1a5e7fc700 1 -- 172.21.15.82:0/1137629412 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1a68095670 con 0x7f1a6c1039d0 2023-09-09T23:12:09.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:09.024+0000 7f1a73939700 1 -- 172.21.15.82:0/1137629412 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a6c10c080 con 0x7f1a6c1039d0 2023-09-09T23:12:09.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:09.025+0000 7f1a5e7fc700 1 -- 172.21.15.82:0/1137629412 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8459 (secure 0 0 0) 0x7f1a68098c90 con 0x7f1a6c1039d0 2023-09-09T23:12:09.024 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:12:09.024 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2471432684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2471432684},{"type":"v1","addr":"172.21.15.82:6835","nonce":2471432684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3579387137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3579387137},{"type":"v1","addr":"172.21.15.82:6837","nonce":3579387137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14712,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/2127301345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2127301345},{"type":"v1","addr":"172.21.15.82:6839","nonce":2127301345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1931961940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1931961940},{"type":"v1","addr":"172.21.15.170:6833","nonce":1931961940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3855146755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3855146755},{"type":"v1","addr":"172.21.15.98:6833","nonce":3855146755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2313861511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2313861511},{"type":"v1","addr":"172.21.15.170:6835","nonce":2313861511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/4006360308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":4006360308},{"type":"v1","addr":"172.21.15.170:6837","nonce":4006360308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1356474828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1356474828},{"type":"v1","addr":"172.21.15.98:6835","nonce":1356474828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24523,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/3858369735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3858369735},{"type":"v1","addr":"172.21.15.170:6839","nonce":3858369735}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2019449840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2019449840},{"type":"v1","addr":"172.21.15.98:6837","nonce":2019449840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-09-09T23:12:09.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:09.028+0000 7f1a73939700 1 -- 172.21.15.82:0/1137629412 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1a580a0710 msgr2=0x7f1a580a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:09.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:09.028+0000 7f1a73939700 1 --2- 172.21.15.82:0/1137629412 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1a580a0710 0x7f1a580a2bd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1a60005950 tx=0x7f1a600058e0 comp rx=0 tx=0).stop 2023-09-09T23:12:09.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:09.028+0000 7f1a73939700 1 -- 172.21.15.82:0/1137629412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a6c1039d0 msgr2=0x7f1a6c0ff050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:09.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:09.028+0000 7f1a73939700 1 --2- 172.21.15.82:0/1137629412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a6c1039d0 0x7f1a6c0ff050 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1a6800c920 tx=0x7f1a6800cce0 comp rx=0 tx=0).stop 2023-09-09T23:12:09.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:09.028+0000 7f1a73939700 1 -- 172.21.15.82:0/1137629412 shutdown_connections 2023-09-09T23:12:09.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:09.028+0000 7f1a73939700 1 --2- 172.21.15.82:0/1137629412 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1a580a0710 0x7f1a580a2bd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:09.029 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:09.028+0000 7f1a73939700 1 --2- 172.21.15.82:0/1137629412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a6c106300 0x7f1a6c0ff590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:09.029 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:09.028+0000 7f1a73939700 1 --2- 172.21.15.82:0/1137629412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a6c1010a0 0x7f1a6c1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:09.030 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:09.028+0000 7f1a73939700 1 --2- 172.21.15.82:0/1137629412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a6c1039d0 0x7f1a6c0ff050 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:09.030 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:09.028+0000 7f1a73939700 1 -- 172.21.15.82:0/1137629412 >> 172.21.15.82:0/1137629412 conn(0x7f1a6c0faa30 msgr2=0x7f1a6c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:09.030 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:09.028+0000 7f1a73939700 1 -- 172.21.15.82:0/1137629412 shutdown_connections 2023-09-09T23:12:09.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:09.028+0000 7f1a73939700 1 -- 172.21.15.82:0/1137629412 wait complete. 2023-09-09T23:12:09.031 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 11 2023-09-09T23:12:09.183 INFO:journalctl@ceph.mds.a.smithi082.stdout:Sep 09 23:12:08 smithi082 systemd[1]: Starting Ceph mds.a for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da... 2023-09-09T23:12:09.455 INFO:journalctl@ceph.mds.a.smithi082.stdout:Sep 09 23:12:09 smithi082 podman[154448]: 2023-09-09T23:12:09.455 INFO:journalctl@ceph.mds.a.smithi082.stdout:Sep 09 23:12:09 smithi082 ceph-mds[154488]: -- 172.21.15.82:0/2224873665 <== mon.0 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557b196171e0 con 0x557b195ed000 2023-09-09T23:12:09.455 INFO:journalctl@ceph.mds.a.smithi082.stdout:Sep 09 23:12:09 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mds-a[154464]: starting mds.a at 2023-09-09T23:12:09.455 INFO:journalctl@ceph.mds.a.smithi082.stdout:Sep 09 23:12:09 smithi082 bash[154448]: 0d3c44e140412d99bd9dd9539a2f60611d799bdddd92bd79e9c45febc954a3d9 2023-09-09T23:12:09.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:09 smithi082 ceph-mon[98694]: Deploying daemon mds.a on smithi082 2023-09-09T23:12:09.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:09 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.82:6838/2127301345,v1:172.21.15.82:6839/2127301345] up:boot 2023-09-09T23:12:09.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:09 smithi082 ceph-mon[98694]: fsmap 10 up:standby 2023-09-09T23:12:09.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-09T23:12:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:09 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1137629412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:12:09.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:09 smithi098 ceph-mon[103124]: Deploying daemon mds.a on smithi082 2023-09-09T23:12:09.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:09 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.82:6838/2127301345,v1:172.21.15.82:6839/2127301345] up:boot 2023-09-09T23:12:09.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:09 smithi098 ceph-mon[103124]: fsmap 10 up:standby 2023-09-09T23:12:09.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:09 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-09T23:12:09.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:09 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1137629412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:12:09.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:09 smithi170 ceph-mon[102832]: Deploying daemon mds.a on smithi082 2023-09-09T23:12:09.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:09 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.82:6838/2127301345,v1:172.21.15.82:6839/2127301345] up:boot 2023-09-09T23:12:09.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:09 smithi170 ceph-mon[102832]: fsmap 10 up:standby 2023-09-09T23:12:09.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:09 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-09T23:12:09.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:09 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1137629412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:12:09.955 INFO:journalctl@ceph.mds.a.smithi082.stdout:Sep 09 23:12:09 smithi082 systemd[1]: Started Ceph mds.a for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da. 2023-09-09T23:12:10.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:10 smithi098 ceph-mon[103124]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:12:10.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:10 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:10.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:10 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:10.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:10 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:10.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:10 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:12:10.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:10 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:12:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:10 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:12:10.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:10 smithi170 ceph-mon[102832]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:12:10.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:10 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:10.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:10 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:10.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:10 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:10.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:10 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:12:10.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:10 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:12:10.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:10 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:12:10.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:10 smithi082 ceph-mon[98694]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:12:10.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:10 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:10.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:10 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:10.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:10 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:10.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:10 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-09T23:12:10.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:10 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-09T23:12:10.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:10 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:12:11.041 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-09T23:12:11.181 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.183+0000 7f03c65dd700 1 -- 172.21.15.82:0/3996103251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03c0100720 msgr2=0x7f03c0100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:11.181 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.183+0000 7f03c65dd700 1 --2- 172.21.15.82:0/3996103251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03c0100720 0x7f03c0100b80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f03b00099c0 tx=0x7f03b0009cd0 comp rx=0 tx=0).stop 2023-09-09T23:12:11.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.183+0000 7f03c65dd700 1 -- 172.21.15.82:0/3996103251 shutdown_connections 2023-09-09T23:12:11.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.183+0000 7f03c65dd700 1 --2- 172.21.15.82:0/3996103251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03c00ff530 0x7f03c00ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:11.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.183+0000 7f03c65dd700 1 --2- 172.21.15.82:0/3996103251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03c01010c0 0x7f03c0107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:11.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.183+0000 7f03c65dd700 1 --2- 172.21.15.82:0/3996103251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03c0100720 0x7f03c0100b80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:11.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.183+0000 7f03c65dd700 1 -- 172.21.15.82:0/3996103251 >> 172.21.15.82:0/3996103251 conn(0x7f03c00faa80 msgr2=0x7f03c00fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:11.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.183+0000 7f03c65dd700 1 -- 172.21.15.82:0/3996103251 shutdown_connections 2023-09-09T23:12:11.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.183+0000 7f03c65dd700 1 -- 172.21.15.82:0/3996103251 wait complete. 2023-09-09T23:12:11.186 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.184+0000 7f03c65dd700 1 Processor -- start 2023-09-09T23:12:11.186 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.184+0000 7f03c65dd700 1 -- start start 2023-09-09T23:12:11.186 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.184+0000 7f03c65dd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03c00ff530 0x7f03c0069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:11.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.184+0000 7f03c65dd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03c0100720 0x7f03c019b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:11.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.184+0000 7f03c65dd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03c01010c0 0x7f03c019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:11.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.184+0000 7f03c65dd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c019de40 con 0x7f03c00ff530 2023-09-09T23:12:11.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.184+0000 7f03c65dd700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c019dfb0 con 0x7f03c0100720 2023-09-09T23:12:11.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.184+0000 7f03c65dd700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c019e170 con 0x7f03c01010c0 2023-09-09T23:12:11.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.184+0000 7f03bffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03c00ff530 0x7f03c0069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:11.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.184+0000 7f03bffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03c00ff530 0x7f03c0069550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54310/0 (socket says 172.21.15.82:54310) 2023-09-09T23:12:11.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.184+0000 7f03bf7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03c0100720 0x7f03c019b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:11.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.184+0000 7f03bffff700 1 -- 172.21.15.82:0/445039105 learned_addr learned my addr 172.21.15.82:0/445039105 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:11.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.184+0000 7f03c4b7a700 1 --2- 172.21.15.82:0/445039105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03c01010c0 0x7f03c019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:11.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.185+0000 7f03bffff700 1 -- 172.21.15.82:0/445039105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03c01010c0 msgr2=0x7f03c019d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:11.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.185+0000 7f03bffff700 1 --2- 172.21.15.82:0/445039105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03c01010c0 0x7f03c019d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:11.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.185+0000 7f03bffff700 1 -- 172.21.15.82:0/445039105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03c0100720 msgr2=0x7f03c019b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:11.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.185+0000 7f03bffff700 1 --2- 172.21.15.82:0/445039105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03c0100720 0x7f03c019b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:11.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.185+0000 7f03bffff700 1 -- 172.21.15.82:0/445039105 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b00095f0 con 0x7f03c00ff530 2023-09-09T23:12:11.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.185+0000 7f03bf7fe700 1 --2- 172.21.15.82:0/445039105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03c0100720 0x7f03c019b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:12:11.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.185+0000 7f03bffff700 1 --2- 172.21.15.82:0/445039105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03c00ff530 0x7f03c0069550 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f03a800b500 tx=0x7f03a800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:11.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.185+0000 7f03bd7fa700 1 -- 172.21.15.82:0/445039105 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8011830 con 0x7f03c00ff530 2023-09-09T23:12:11.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.185+0000 7f03c65dd700 1 -- 172.21.15.82:0/445039105 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c019e450 con 0x7f03c00ff530 2023-09-09T23:12:11.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.185+0000 7f03bd7fa700 1 -- 172.21.15.82:0/445039105 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03a8011e50 con 0x7f03c00ff530 2023-09-09T23:12:11.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.185+0000 7f03bd7fa700 1 -- 172.21.15.82:0/445039105 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8010dc0 con 0x7f03c00ff530 2023-09-09T23:12:11.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.185+0000 7f03c65dd700 1 -- 172.21.15.82:0/445039105 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c019e930 con 0x7f03c00ff530 2023-09-09T23:12:11.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.186+0000 7f03c65dd700 1 -- 172.21.15.82:0/445039105 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c010ade0 con 0x7f03c00ff530 2023-09-09T23:12:11.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.187+0000 7f03bd7fa700 1 -- 172.21.15.82:0/445039105 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f03a8010380 con 0x7f03c00ff530 2023-09-09T23:12:11.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.188+0000 7f03bd7fa700 1 --2- 172.21.15.82:0/445039105 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f03ac0a0690 0x7f03ac0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:11.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.188+0000 7f03bf7fe700 1 --2- 172.21.15.82:0/445039105 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f03ac0a0690 0x7f03ac0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:11.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.188+0000 7f03bd7fa700 1 -- 172.21.15.82:0/445039105 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f03a80cb6a0 con 0x7f03c00ff530 2023-09-09T23:12:11.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.188+0000 7f03bf7fe700 1 --2- 172.21.15.82:0/445039105 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f03ac0a0690 0x7f03ac0a2b50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f03b000b580 tx=0x7f03b0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:11.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.191+0000 7f03bd7fa700 1 -- 172.21.15.82:0/445039105 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f03a8095670 con 0x7f03c00ff530 2023-09-09T23:12:11.251 INFO:journalctl@ceph.mds.b.smithi098.stdout:Sep 09 23:12:11 smithi098 systemd[1]: Starting Ceph mds.b for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da... 2023-09-09T23:12:11.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.427+0000 7f03c65dd700 1 -- 172.21.15.82:0/445039105 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03c0106390 con 0x7f03c00ff530 2023-09-09T23:12:11.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.427+0000 7f03bd7fa700 1 -- 172.21.15.82:0/445039105 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9251 (secure 0 0 0) 0x7f03a8098c90 con 0x7f03c00ff530 2023-09-09T23:12:11.425 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:12:11.425 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2471432684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2471432684},{"type":"v1","addr":"172.21.15.82:6835","nonce":2471432684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3579387137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3579387137},{"type":"v1","addr":"172.21.15.82:6837","nonce":3579387137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14712,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/2127301345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2127301345},{"type":"v1","addr":"172.21.15.82:6839","nonce":2127301345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14724,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/4143604488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":4143604488},{"type":"v1","addr":"172.21.15.82:6841","nonce":4143604488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1931961940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1931961940},{"type":"v1","addr":"172.21.15.170:6833","nonce":1931961940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3855146755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3855146755},{"type":"v1","addr":"172.21.15.98:6833","nonce":3855146755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2313861511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2313861511},{"type":"v1","addr":"172.21.15.170:6835","nonce":2313861511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/4006360308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":4006360308},{"type":"v1","addr":"172.21.15.170:6837","nonce":4006360308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1356474828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1356474828},{"type":"v1","addr":"172.21.15.98:6835","nonce":1356474828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24523,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/3858369735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3858369735},{"type":"v1","addr":"172.21.15.170:6839","nonce":3858369735}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2019449840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2019449840},{"type":"v1","addr":"172.21.15.98:6837","nonce":2019449840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-09-09T23:12:11.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.429+0000 7f03c65dd700 1 -- 172.21.15.82:0/445039105 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f03ac0a0690 msgr2=0x7f03ac0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:11.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.429+0000 7f03c65dd700 1 --2- 172.21.15.82:0/445039105 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f03ac0a0690 0x7f03ac0a2b50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f03b000b580 tx=0x7f03b0000bc0 comp rx=0 tx=0).stop 2023-09-09T23:12:11.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.429+0000 7f03c65dd700 1 -- 172.21.15.82:0/445039105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03c00ff530 msgr2=0x7f03c0069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:11.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.429+0000 7f03c65dd700 1 --2- 172.21.15.82:0/445039105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03c00ff530 0x7f03c0069550 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f03a800b500 tx=0x7f03a800b810 comp rx=0 tx=0).stop 2023-09-09T23:12:11.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.430+0000 7f03c65dd700 1 -- 172.21.15.82:0/445039105 shutdown_connections 2023-09-09T23:12:11.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.430+0000 7f03c65dd700 1 --2- 172.21.15.82:0/445039105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03c01010c0 0x7f03c019d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:11.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.430+0000 7f03c65dd700 1 --2- 172.21.15.82:0/445039105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03c00ff530 0x7f03c0069550 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:11.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.430+0000 7f03c65dd700 1 --2- 172.21.15.82:0/445039105 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f03ac0a0690 0x7f03ac0a2b50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:11.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.430+0000 7f03c65dd700 1 --2- 172.21.15.82:0/445039105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03c0100720 0x7f03c019b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:11.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.430+0000 7f03c65dd700 1 -- 172.21.15.82:0/445039105 >> 172.21.15.82:0/445039105 conn(0x7f03c00faa80 msgr2=0x7f03c00fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:11.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.430+0000 7f03c65dd700 1 -- 172.21.15.82:0/445039105 shutdown_connections 2023-09-09T23:12:11.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:11.430+0000 7f03c65dd700 1 -- 172.21.15.82:0/445039105 wait complete. 2023-09-09T23:12:11.430 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 12 2023-09-09T23:12:11.438 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:11 smithi082 ceph-mon[98694]: Deploying daemon mds.b on smithi098 2023-09-09T23:12:11.438 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:11 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.82:6840/4143604488,v1:172.21.15.82:6841/4143604488] up:boot 2023-09-09T23:12:11.439 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:11 smithi082 ceph-mon[98694]: fsmap 11 up:standby 2023-09-09T23:12:11.439 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:11 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-09T23:12:11.560 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:11 smithi098 ceph-mon[103124]: Deploying daemon mds.b on smithi098 2023-09-09T23:12:11.560 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:11 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.82:6840/4143604488,v1:172.21.15.82:6841/4143604488] up:boot 2023-09-09T23:12:11.561 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:11 smithi098 ceph-mon[103124]: fsmap 11 up:standby 2023-09-09T23:12:11.561 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:11 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-09T23:12:11.561 INFO:journalctl@ceph.mds.b.smithi098.stdout:Sep 09 23:12:11 smithi098 podman[139317]: 2023-09-09T23:12:11.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:11 smithi170 ceph-mon[102832]: Deploying daemon mds.b on smithi098 2023-09-09T23:12:11.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:11 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.82:6840/4143604488,v1:172.21.15.82:6841/4143604488] up:boot 2023-09-09T23:12:11.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:11 smithi170 ceph-mon[102832]: fsmap 11 up:standby 2023-09-09T23:12:11.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:11 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-09T23:12:12.001 INFO:journalctl@ceph.mds.b.smithi098.stdout:Sep 09 23:12:11 smithi098 ceph-mds[139357]: -- 172.21.15.98:0/1095493072 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55eb2a58d1e0 con 0x55eb2a562c00 2023-09-09T23:12:12.001 INFO:journalctl@ceph.mds.b.smithi098.stdout:Sep 09 23:12:11 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mds-b[139333]: starting mds.b at 2023-09-09T23:12:12.002 INFO:journalctl@ceph.mds.b.smithi098.stdout:Sep 09 23:12:11 smithi098 bash[139317]: a777e31dd425a20e549b6ee1e60d45a5dc49c90bb4ad930c7c6d8f990aea5922 2023-09-09T23:12:12.002 INFO:journalctl@ceph.mds.b.smithi098.stdout:Sep 09 23:12:11 smithi098 systemd[1]: Started Ceph mds.b for 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da. 2023-09-09T23:12:12.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:12 smithi170 ceph-mon[102832]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:12:12.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:12 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/445039105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:12:12.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:12 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:12 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:12 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:12 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:12 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:12 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:12:12.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:12 smithi082 ceph-mon[98694]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:12:12.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:12 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/445039105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:12:12.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:12 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:12.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:12 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:12.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:12 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:12.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:12 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:12 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:12 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:12:12.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:12 smithi098 ceph-mon[103124]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:12:12.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:12 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/445039105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:12:12.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:12 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:12.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:12 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:12.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:12 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:12.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:12 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:12.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:12 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:12.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:12 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:12:13.441 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-09T23:12:13.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.573+0000 7f0194e11700 1 -- 172.21.15.82:0/1495707039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01901039c0 msgr2=0x7f0190105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:13.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.573+0000 7f0194e11700 1 --2- 172.21.15.82:0/1495707039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01901039c0 0x7f0190105db0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0184009990 tx=0x7f0184009ca0 comp rx=0 tx=0).stop 2023-09-09T23:12:13.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.573+0000 7f0194e11700 1 -- 172.21.15.82:0/1495707039 shutdown_connections 2023-09-09T23:12:13.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.573+0000 7f0194e11700 1 --2- 172.21.15.82:0/1495707039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0190101090 0x7f0190103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:13.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.573+0000 7f0194e11700 1 --2- 172.21.15.82:0/1495707039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01901062f0 0x7f0190108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:13.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.573+0000 7f0194e11700 1 --2- 172.21.15.82:0/1495707039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01901039c0 0x7f0190105db0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:13.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.573+0000 7f0194e11700 1 -- 172.21.15.82:0/1495707039 >> 172.21.15.82:0/1495707039 conn(0x7f01900faa60 msgr2=0x7f01900fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:13.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.573+0000 7f0194e11700 1 -- 172.21.15.82:0/1495707039 shutdown_connections 2023-09-09T23:12:13.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.573+0000 7f0194e11700 1 -- 172.21.15.82:0/1495707039 wait complete. 2023-09-09T23:12:13.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.574+0000 7f0194e11700 1 Processor -- start 2023-09-09T23:12:13.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.574+0000 7f0194e11700 1 -- start start 2023-09-09T23:12:13.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.574+0000 7f0194e11700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0190101090 0x7f0190196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:13.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.574+0000 7f0194e11700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01901039c0 0x7f0190196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:13.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.574+0000 7f0194e11700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01901062f0 0x7f019019d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:13.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.574+0000 7f0194e11700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019019dda0 con 0x7f01901039c0 2023-09-09T23:12:13.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.574+0000 7f0194e11700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019019dee0 con 0x7f01901062f0 2023-09-09T23:12:13.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.574+0000 7f0194e11700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019019e070 con 0x7f0190101090 2023-09-09T23:12:13.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.574+0000 7f018dd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01901039c0 0x7f0190196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:13.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.574+0000 7f018e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0190101090 0x7f0190196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:13.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.574+0000 7f018dd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01901039c0 0x7f0190196760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54326/0 (socket says 172.21.15.82:54326) 2023-09-09T23:12:13.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.574+0000 7f018dd9b700 1 -- 172.21.15.82:0/2914927331 learned_addr learned my addr 172.21.15.82:0/2914927331 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:13.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.574+0000 7f018ed9d700 1 --2- 172.21.15.82:0/2914927331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01901062f0 0x7f019019d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:13.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.574+0000 7f018dd9b700 1 -- 172.21.15.82:0/2914927331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0190101090 msgr2=0x7f0190196220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:13.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.574+0000 7f018dd9b700 1 --2- 172.21.15.82:0/2914927331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0190101090 0x7f0190196220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:13.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.574+0000 7f018dd9b700 1 -- 172.21.15.82:0/2914927331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01901062f0 msgr2=0x7f019019d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:13.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.574+0000 7f018dd9b700 1 --2- 172.21.15.82:0/2914927331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01901062f0 0x7f019019d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:13.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.574+0000 7f018dd9b700 1 -- 172.21.15.82:0/2914927331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01840095f0 con 0x7f01901039c0 2023-09-09T23:12:13.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.574+0000 7f018ed9d700 1 --2- 172.21.15.82:0/2914927331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01901062f0 0x7f019019d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:12:13.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.575+0000 7f018dd9b700 1 --2- 172.21.15.82:0/2914927331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01901039c0 0x7f0190196760 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0184000c00 tx=0x7f018400f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:13.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.575+0000 7f017f7fe700 1 -- 172.21.15.82:0/2914927331 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018401d070 con 0x7f01901039c0 2023-09-09T23:12:13.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.575+0000 7f0194e11700 1 -- 172.21.15.82:0/2914927331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f019019e2f0 con 0x7f01901039c0 2023-09-09T23:12:13.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.575+0000 7f017f7fe700 1 -- 172.21.15.82:0/2914927331 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f018400fb50 con 0x7f01901039c0 2023-09-09T23:12:13.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.575+0000 7f017f7fe700 1 -- 172.21.15.82:0/2914927331 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0184017650 con 0x7f01901039c0 2023-09-09T23:12:13.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.575+0000 7f0194e11700 1 -- 172.21.15.82:0/2914927331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f019019e7a0 con 0x7f01901039c0 2023-09-09T23:12:13.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.576+0000 7f0194e11700 1 -- 172.21.15.82:0/2914927331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f019010be30 con 0x7f01901039c0 2023-09-09T23:12:13.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.577+0000 7f017f7fe700 1 -- 172.21.15.82:0/2914927331 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f018400fcc0 con 0x7f01901039c0 2023-09-09T23:12:13.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.578+0000 7f017f7fe700 1 --2- 172.21.15.82:0/2914927331 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f01780a0690 0x7f01780a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:13.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.578+0000 7f018e59c700 1 --2- 172.21.15.82:0/2914927331 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f01780a0690 0x7f01780a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:13.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.578+0000 7f017f7fe700 1 -- 172.21.15.82:0/2914927331 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f01840ccdd0 con 0x7f01901039c0 2023-09-09T23:12:13.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.578+0000 7f018e59c700 1 --2- 172.21.15.82:0/2914927331 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f01780a0690 0x7f01780a2b50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0180005950 tx=0x7f018000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:13.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.581+0000 7f017f7fe700 1 -- 172.21.15.82:0/2914927331 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0184096e30 con 0x7f01901039c0 2023-09-09T23:12:13.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:boot 2023-09-09T23:12:13.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:13 smithi170 ceph-mon[102832]: fsmap 12 up:standby 2023-09-09T23:12:13.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:13 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-09T23:12:13.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:boot 2023-09-09T23:12:13.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:13 smithi082 ceph-mon[98694]: fsmap 12 up:standby 2023-09-09T23:12:13.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:13 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-09T23:12:13.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:boot 2023-09-09T23:12:13.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:13 smithi098 ceph-mon[103124]: fsmap 12 up:standby 2023-09-09T23:12:13.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:13 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-09T23:12:13.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.832+0000 7f0194e11700 1 -- 172.21.15.82:0/2914927331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f019010c020 con 0x7f01901039c0 2023-09-09T23:12:13.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.833+0000 7f017f7fe700 1 -- 172.21.15.82:0/2914927331 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10043 (secure 0 0 0) 0x7f018409a450 con 0x7f01901039c0 2023-09-09T23:12:13.832 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:12:13.832 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2471432684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2471432684},{"type":"v1","addr":"172.21.15.82:6835","nonce":2471432684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3579387137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3579387137},{"type":"v1","addr":"172.21.15.82:6837","nonce":3579387137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14712,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/2127301345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2127301345},{"type":"v1","addr":"172.21.15.82:6839","nonce":2127301345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14724,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/4143604488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":4143604488},{"type":"v1","addr":"172.21.15.82:6841","nonce":4143604488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1931961940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1931961940},{"type":"v1","addr":"172.21.15.170:6833","nonce":1931961940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24491,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3855146755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3855146755},{"type":"v1","addr":"172.21.15.98:6833","nonce":3855146755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2313861511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2313861511},{"type":"v1","addr":"172.21.15.170:6835","nonce":2313861511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/4006360308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":4006360308},{"type":"v1","addr":"172.21.15.170:6837","nonce":4006360308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1356474828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1356474828},{"type":"v1","addr":"172.21.15.98:6835","nonce":1356474828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24523,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/3858369735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3858369735},{"type":"v1","addr":"172.21.15.170:6839","nonce":3858369735}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2019449840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2019449840},{"type":"v1","addr":"172.21.15.98:6837","nonce":2019449840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24566,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6839/2785086371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2785086371},{"type":"v1","addr":"172.21.15.98:6839","nonce":2785086371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2023-09-09T23:12:13.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.835+0000 7f0194e11700 1 -- 172.21.15.82:0/2914927331 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f01780a0690 msgr2=0x7f01780a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:13.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.835+0000 7f0194e11700 1 --2- 172.21.15.82:0/2914927331 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f01780a0690 0x7f01780a2b50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0180005950 tx=0x7f018000a300 comp rx=0 tx=0).stop 2023-09-09T23:12:13.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.835+0000 7f0194e11700 1 -- 172.21.15.82:0/2914927331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01901039c0 msgr2=0x7f0190196760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:13.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.835+0000 7f0194e11700 1 --2- 172.21.15.82:0/2914927331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01901039c0 0x7f0190196760 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0184000c00 tx=0x7f018400f710 comp rx=0 tx=0).stop 2023-09-09T23:12:13.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.835+0000 7f0194e11700 1 -- 172.21.15.82:0/2914927331 shutdown_connections 2023-09-09T23:12:13.836 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.835+0000 7f0194e11700 1 --2- 172.21.15.82:0/2914927331 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f01780a0690 0x7f01780a2b50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:13.836 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.835+0000 7f0194e11700 1 --2- 172.21.15.82:0/2914927331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01901062f0 0x7f019019d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:13.836 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.835+0000 7f0194e11700 1 --2- 172.21.15.82:0/2914927331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0190101090 0x7f0190196220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:13.836 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.835+0000 7f0194e11700 1 --2- 172.21.15.82:0/2914927331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01901039c0 0x7f0190196760 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:13.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.835+0000 7f0194e11700 1 -- 172.21.15.82:0/2914927331 >> 172.21.15.82:0/2914927331 conn(0x7f01900faa60 msgr2=0x7f01901024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:13.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.836+0000 7f0194e11700 1 -- 172.21.15.82:0/2914927331 shutdown_connections 2023-09-09T23:12:13.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.836+0000 7f0194e11700 1 -- 172.21.15.82:0/2914927331 wait complete. 2023-09-09T23:12:13.838 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 13 2023-09-09T23:12:13.847 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-09T23:12:13.980 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.978+0000 7f04c7f22700 1 -- 172.21.15.82:0/2087689992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04c01010c0 msgr2=0x7f04c01034b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:13.980 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.978+0000 7f04c7f22700 1 --2- 172.21.15.82:0/2087689992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04c01010c0 0x7f04c01034b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f04bc009990 tx=0x7f04bc009ca0 comp rx=0 tx=0).stop 2023-09-09T23:12:13.980 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.978+0000 7f04c7f22700 1 -- 172.21.15.82:0/2087689992 shutdown_connections 2023-09-09T23:12:13.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.978+0000 7f04c7f22700 1 --2- 172.21.15.82:0/2087689992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04c0106320 0x7f04c0108730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:13.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.978+0000 7f04c7f22700 1 --2- 172.21.15.82:0/2087689992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04c01010c0 0x7f04c01034b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:13.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.978+0000 7f04c7f22700 1 --2- 172.21.15.82:0/2087689992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04c01039f0 0x7f04c0105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:13.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.978+0000 7f04c7f22700 1 -- 172.21.15.82:0/2087689992 >> 172.21.15.82:0/2087689992 conn(0x7f04c00faa30 msgr2=0x7f04c00fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:13.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.978+0000 7f04c7f22700 1 -- 172.21.15.82:0/2087689992 shutdown_connections 2023-09-09T23:12:13.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.978+0000 7f04c7f22700 1 -- 172.21.15.82:0/2087689992 wait complete. 2023-09-09T23:12:13.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.978+0000 7f04c7f22700 1 Processor -- start 2023-09-09T23:12:13.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.979+0000 7f04c7f22700 1 -- start start 2023-09-09T23:12:13.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.979+0000 7f04c7f22700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04c01010c0 0x7f04c01962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:13.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.979+0000 7f04c7f22700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04c01039f0 0x7f04c01967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:13.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.979+0000 7f04c7f22700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04c0106320 0x7f04c019d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:13.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.979+0000 7f04c7f22700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c019de20 con 0x7f04c01010c0 2023-09-09T23:12:13.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.979+0000 7f04c7f22700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c019df60 con 0x7f04c01039f0 2023-09-09T23:12:13.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.979+0000 7f04c7f22700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c019e0f0 con 0x7f04c0106320 2023-09-09T23:12:13.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.979+0000 7f04c5cbe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04c01010c0 0x7f04c01962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:13.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.979+0000 7f04c5cbe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04c01010c0 0x7f04c01962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54338/0 (socket says 172.21.15.82:54338) 2023-09-09T23:12:13.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.979+0000 7f04c5cbe700 1 -- 172.21.15.82:0/1009385674 learned_addr learned my addr 172.21.15.82:0/1009385674 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:13.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.979+0000 7f04c64bf700 1 --2- 172.21.15.82:0/1009385674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04c0106320 0x7f04c019d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:13.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.979+0000 7f04c54bd700 1 --2- 172.21.15.82:0/1009385674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04c01039f0 0x7f04c01967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:13.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.979+0000 7f04c5cbe700 1 -- 172.21.15.82:0/1009385674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04c0106320 msgr2=0x7f04c019d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:13.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.979+0000 7f04c5cbe700 1 --2- 172.21.15.82:0/1009385674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04c0106320 0x7f04c019d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:13.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.979+0000 7f04c5cbe700 1 -- 172.21.15.82:0/1009385674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04c01039f0 msgr2=0x7f04c01967e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:13.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.979+0000 7f04c5cbe700 1 --2- 172.21.15.82:0/1009385674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04c01039f0 0x7f04c01967e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:13.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.979+0000 7f04c5cbe700 1 -- 172.21.15.82:0/1009385674 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04bc0095f0 con 0x7f04c01010c0 2023-09-09T23:12:13.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.979+0000 7f04c5cbe700 1 --2- 172.21.15.82:0/1009385674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04c01010c0 0x7f04c01962a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f04bc004730 tx=0x7f04bc004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:13.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.979+0000 7f04b6ffd700 1 -- 172.21.15.82:0/1009385674 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc01d070 con 0x7f04c01010c0 2023-09-09T23:12:13.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.979+0000 7f04b6ffd700 1 -- 172.21.15.82:0/1009385674 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04bc004c90 con 0x7f04c01010c0 2023-09-09T23:12:13.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.979+0000 7f04b6ffd700 1 -- 172.21.15.82:0/1009385674 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc0054b0 con 0x7f04c01010c0 2023-09-09T23:12:13.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.979+0000 7f04c7f22700 1 -- 172.21.15.82:0/1009385674 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c019e370 con 0x7f04c01010c0 2023-09-09T23:12:13.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.979+0000 7f04c7f22700 1 -- 172.21.15.82:0/1009385674 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c019e730 con 0x7f04c01010c0 2023-09-09T23:12:13.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.980+0000 7f04c7f22700 1 -- 172.21.15.82:0/1009385674 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04c010be30 con 0x7f04c01010c0 2023-09-09T23:12:13.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.981+0000 7f04b6ffd700 1 -- 172.21.15.82:0/1009385674 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f04bc00fbc0 con 0x7f04c01010c0 2023-09-09T23:12:13.989 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.982+0000 7f04b6ffd700 1 --2- 172.21.15.82:0/1009385674 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f04ac0a0710 0x7f04ac0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:13.990 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.982+0000 7f04c54bd700 1 --2- 172.21.15.82:0/1009385674 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f04ac0a0710 0x7f04ac0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:13.990 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.982+0000 7f04b6ffd700 1 -- 172.21.15.82:0/1009385674 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f04bc0ccd70 con 0x7f04c01010c0 2023-09-09T23:12:13.990 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.982+0000 7f04c54bd700 1 --2- 172.21.15.82:0/1009385674 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f04ac0a0710 0x7f04ac0a2bd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f04b0007c60 tx=0x7f04b00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:13.990 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:13.985+0000 7f04b6ffd700 1 -- 172.21.15.82:0/1009385674 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f04bc096dd0 con 0x7f04c01010c0 2023-09-09T23:12:14.181 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.182+0000 7f04c7f22700 1 -- 172.21.15.82:0/1009385674 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04c019e9b0 con 0x7f04c01010c0 2023-09-09T23:12:14.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.183+0000 7f04b6ffd700 1 -- 172.21.15.82:0/1009385674 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18374 (secure 0 0 0) 0x7f04bc09a3f0 con 0x7f04c01010c0 2023-09-09T23:12:14.182 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:12:14.183 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":77,"fsid":"3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da","created":"2023-09-09T23:05:14.222190+0000","modified":"2023-09-09T23:11:38.498415+0000","last_up_change":"2023-09-09T23:11:37.253759+0000","last_in_change":"2023-09-09T23:11:19.933812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-09T23:08:31.234958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1277eb17-a2e7-4e10-9832-f12a5e93ffde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6803","nonce":3197434929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6805","nonce":3197434929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6809","nonce":3197434929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6807","nonce":3197434929}]},"public_addr":"172.21.15.82:6803/3197434929","cluster_addr":"172.21.15.82:6805/3197434929","heartbeat_back_addr":"172.21.15.82:6809/3197434929","heartbeat_front_addr":"172.21.15.82:6807/3197434929","state":["exists","up"]},{"osd":1,"uuid":"f68723fd-0908-4e21-ae8e-1de1c3a9a65c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6801","nonce":3528641314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6803","nonce":3528641314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6807","nonce":3528641314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6805","nonce":3528641314}]},"public_addr":"172.21.15.98:6801/3528641314","cluster_addr":"172.21.15.98:6803/3528641314","heartbeat_back_addr":"172.21.15.98:6807/3528641314","heartbeat_front_addr":"172.21.15.98:6805/3528641314","state":["exists","up"]},{"osd":2,"uuid":"06f89c53-1419-4f52-acad-6e92916b476f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6801","nonce":4006414551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6803","nonce":4006414551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6807","nonce":4006414551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6805","nonce":4006414551}]},"public_addr":"172.21.15.170:6801/4006414551","cluster_addr":"172.21.15.170:6803/4006414551","heartbeat_back_addr":"172.21.15.170:6807/4006414551","heartbeat_front_addr":"172.21.15.170:6805/4006414551","state":["exists","up"]},{"osd":3,"uuid":"fedbd4c9-6852-42b5-8f3a-ba8d9bfe5d59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6811","nonce":3647629241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6813","nonce":3647629241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6817","nonce":3647629241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6815","nonce":3647629241}]},"public_addr":"172.21.15.82:6811/3647629241","cluster_addr":"172.21.15.82:6813/3647629241","heartbeat_back_addr":"172.21.15.82:6817/3647629241","heartbeat_front_addr":"172.21.15.82:6815/3647629241","state":["exists","up"]},{"osd":4,"uuid":"a71f3084-6919-4161-87bf-345f3355d8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6809","nonce":2507035643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6811","nonce":2507035643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6815","nonce":2507035643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6813","nonce":2507035643}]},"public_addr":"172.21.15.98:6809/2507035643","cluster_addr":"172.21.15.98:6811/2507035643","heartbeat_back_addr":"172.21.15.98:6815/2507035643","heartbeat_front_addr":"172.21.15.98:6813/2507035643","state":["exists","up"]},{"osd":5,"uuid":"f76058e9-a90e-476e-87f0-3c6723795e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6809","nonce":775005168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6811","nonce":775005168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6815","nonce":775005168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6813","nonce":775005168}]},"public_addr":"172.21.15.170:6809/775005168","cluster_addr":"172.21.15.170:6811/775005168","heartbeat_back_addr":"172.21.15.170:6815/775005168","heartbeat_front_addr":"172.21.15.170:6813/775005168","state":["exists","up"]},{"osd":6,"uuid":"25940be0-c1bd-4c72-95bf-fd8367aa2f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6819","nonce":3008405982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6821","nonce":3008405982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6825","nonce":3008405982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6823","nonce":3008405982}]},"public_addr":"172.21.15.82:6819/3008405982","cluster_addr":"172.21.15.82:6821/3008405982","heartbeat_back_addr":"172.21.15.82:6825/3008405982","heartbeat_front_addr":"172.21.15.82:6823/3008405982","state":["exists","up"]},{"osd":7,"uuid":"f651bb8c-0b25-42d5-b028-58563f4c27f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6817","nonce":3547615284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6819","nonce":3547615284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6823","nonce":3547615284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6821","nonce":3547615284}]},"public_addr":"172.21.15.98:6817/3547615284","cluster_addr":"172.21.15.98:6819/3547615284","heartbeat_back_addr":"172.21.15.98:6823/3547615284","heartbeat_front_addr":"172.21.15.98:6821/3547615284","state":["exists","up"]},{"osd":8,"uuid":"23b27c86-8c98-441e-b7ab-d57bc584e58f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6817","nonce":632038848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6819","nonce":632038848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6823","nonce":632038848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6821","nonce":632038848}]},"public_addr":"172.21.15.170:6817/632038848","cluster_addr":"172.21.15.170:6819/632038848","heartbeat_back_addr":"172.21.15.170:6823/632038848","heartbeat_front_addr":"172.21.15.170:6821/632038848","state":["exists","up"]},{"osd":9,"uuid":"383b8989-4922-4eee-9c77-f198258ac243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6827","nonce":3354242311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6829","nonce":3354242311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6833","nonce":3354242311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6831","nonce":3354242311}]},"public_addr":"172.21.15.82:6827/3354242311","cluster_addr":"172.21.15.82:6829/3354242311","heartbeat_back_addr":"172.21.15.82:6833/3354242311","heartbeat_front_addr":"172.21.15.82:6831/3354242311","state":["exists","up"]},{"osd":10,"uuid":"33a59389-7ced-42d5-bf54-2071317132d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6825","nonce":2069114959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6827","nonce":2069114959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6831","nonce":2069114959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6829","nonce":2069114959}]},"public_addr":"172.21.15.98:6825/2069114959","cluster_addr":"172.21.15.98:6827/2069114959","heartbeat_back_addr":"172.21.15.98:6831/2069114959","heartbeat_front_addr":"172.21.15.98:6829/2069114959","state":["exists","up"]},{"osd":11,"uuid":"b7a71f5c-85e2-4fec-be4b-95be0788b302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6825","nonce":146964512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6827","nonce":146964512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6831","nonce":146964512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6829","nonce":146964512}]},"public_addr":"172.21.15.170:6825/146964512","cluster_addr":"172.21.15.170:6827/146964512","heartbeat_back_addr":"172.21.15.170:6831/146964512","heartbeat_front_addr":"172.21.15.170:6829/146964512","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:07:47.563043+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:08.359260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:29.070374+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:49.897775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:11.434248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:31.538464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:52.377109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:13.452968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:33.806897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:54.083177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:14.980087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:35.690089+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4013747788":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6800/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/2823341464":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/75116806":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/1037625632":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/3142088817":"2023-09-10T23:05:46.550354+0000","172.21.15.82:6800/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/1679622963":"2023-09-10T23:05:46.550354+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-09T23:12:14.185 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.186+0000 7f04c7f22700 1 -- 172.21.15.82:0/1009385674 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f04ac0a0710 msgr2=0x7f04ac0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:14.185 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.186+0000 7f04c7f22700 1 --2- 172.21.15.82:0/1009385674 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f04ac0a0710 0x7f04ac0a2bd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f04b0007c60 tx=0x7f04b00073d0 comp rx=0 tx=0).stop 2023-09-09T23:12:14.185 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.186+0000 7f04c7f22700 1 -- 172.21.15.82:0/1009385674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04c01010c0 msgr2=0x7f04c01962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:14.185 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.186+0000 7f04c7f22700 1 --2- 172.21.15.82:0/1009385674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04c01010c0 0x7f04c01962a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f04bc004730 tx=0x7f04bc004760 comp rx=0 tx=0).stop 2023-09-09T23:12:14.186 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.186+0000 7f04c7f22700 1 -- 172.21.15.82:0/1009385674 shutdown_connections 2023-09-09T23:12:14.186 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.186+0000 7f04c7f22700 1 --2- 172.21.15.82:0/1009385674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04c0106320 0x7f04c019d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:14.186 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.186+0000 7f04c7f22700 1 --2- 172.21.15.82:0/1009385674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04c01010c0 0x7f04c01962a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:14.186 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.186+0000 7f04c7f22700 1 --2- 172.21.15.82:0/1009385674 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f04ac0a0710 0x7f04ac0a2bd0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:14.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.186+0000 7f04c7f22700 1 --2- 172.21.15.82:0/1009385674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04c01039f0 0x7f04c01967e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:14.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.186+0000 7f04c7f22700 1 -- 172.21.15.82:0/1009385674 >> 172.21.15.82:0/1009385674 conn(0x7f04c00faa30 msgr2=0x7f04c0102590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:14.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.186+0000 7f04c7f22700 1 -- 172.21.15.82:0/1009385674 shutdown_connections 2023-09-09T23:12:14.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.186+0000 7f04c7f22700 1 -- 172.21.15.82:0/1009385674 wait complete. 2023-09-09T23:12:14.197 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-09T23:08:31.234958+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-09T23:12:14.198 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-09T23:12:14.329 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.329+0000 7f98e7b61700 1 -- 172.21.15.82:0/3449994851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98e0100600 msgr2=0x7f98e0100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:14.330 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.329+0000 7f98e7b61700 1 --2- 172.21.15.82:0/3449994851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98e0100600 0x7f98e0100a20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f98d00099c0 tx=0x7f98d0009cd0 comp rx=0 tx=0).stop 2023-09-09T23:12:14.330 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.329+0000 7f98e7b61700 1 -- 172.21.15.82:0/3449994851 shutdown_connections 2023-09-09T23:12:14.330 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.329+0000 7f98e7b61700 1 --2- 172.21.15.82:0/3449994851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98e0102190 0x7f98e0108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:14.330 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.329+0000 7f98e7b61700 1 --2- 172.21.15.82:0/3449994851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98e0100600 0x7f98e0100a20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:14.331 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.329+0000 7f98e7b61700 1 --2- 172.21.15.82:0/3449994851 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98e01017f0 0x7f98e0101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:14.331 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.329+0000 7f98e7b61700 1 -- 172.21.15.82:0/3449994851 >> 172.21.15.82:0/3449994851 conn(0x7f98e00fbb50 msgr2=0x7f98e00fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:14.331 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.329+0000 7f98e7b61700 1 -- 172.21.15.82:0/3449994851 shutdown_connections 2023-09-09T23:12:14.331 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.329+0000 7f98e7b61700 1 -- 172.21.15.82:0/3449994851 wait complete. 2023-09-09T23:12:14.331 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.330+0000 7f98e7b61700 1 Processor -- start 2023-09-09T23:12:14.332 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.330+0000 7f98e7b61700 1 -- start start 2023-09-09T23:12:14.332 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.330+0000 7f98e7b61700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98e0100600 0x7f98e00743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:14.332 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.330+0000 7f98e7b61700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98e01017f0 0x7f98e00729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:14.332 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.330+0000 7f98e7b61700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98e0102190 0x7f98e0072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:14.332 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.330+0000 7f98e7b61700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e00736d0 con 0x7f98e0100600 2023-09-09T23:12:14.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.330+0000 7f98e7b61700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e0073840 con 0x7f98e0102190 2023-09-09T23:12:14.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.330+0000 7f98e7b61700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e01a0410 con 0x7f98e01017f0 2023-09-09T23:12:14.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.330+0000 7f98e58fd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98e0100600 0x7f98e00743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:14.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.330+0000 7f98e58fd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98e0100600 0x7f98e00743b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54352/0 (socket says 172.21.15.82:54352) 2023-09-09T23:12:14.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.330+0000 7f98e58fd700 1 -- 172.21.15.82:0/1919413983 learned_addr learned my addr 172.21.15.82:0/1919413983 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:14.334 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.330+0000 7f98e60fe700 1 --2- 172.21.15.82:0/1919413983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98e0102190 0x7f98e0072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:14.334 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.330+0000 7f98e50fc700 1 --2- 172.21.15.82:0/1919413983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98e01017f0 0x7f98e00729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:14.334 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.330+0000 7f98e58fd700 1 -- 172.21.15.82:0/1919413983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98e01017f0 msgr2=0x7f98e00729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:14.334 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.330+0000 7f98e58fd700 1 --2- 172.21.15.82:0/1919413983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98e01017f0 0x7f98e00729f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:14.334 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.330+0000 7f98e58fd700 1 -- 172.21.15.82:0/1919413983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98e0102190 msgr2=0x7f98e0072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:14.335 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.330+0000 7f98e58fd700 1 --2- 172.21.15.82:0/1919413983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98e0102190 0x7f98e0072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:14.335 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.330+0000 7f98e58fd700 1 -- 172.21.15.82:0/1919413983 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d00095f0 con 0x7f98e0100600 2023-09-09T23:12:14.335 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.330+0000 7f98e58fd700 1 --2- 172.21.15.82:0/1919413983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98e0100600 0x7f98e00743b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009990 tx=0x7f98d0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:14.335 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.330+0000 7f98d6ffd700 1 -- 172.21.15.82:0/1919413983 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d001d070 con 0x7f98e0100600 2023-09-09T23:12:14.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.330+0000 7f98d6ffd700 1 -- 172.21.15.82:0/1919413983 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98d0004200 con 0x7f98e0100600 2023-09-09T23:12:14.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.330+0000 7f98e7b61700 1 -- 172.21.15.82:0/1919413983 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e01a0690 con 0x7f98e0100600 2023-09-09T23:12:14.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.330+0000 7f98d6ffd700 1 -- 172.21.15.82:0/1919413983 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d0022e30 con 0x7f98e0100600 2023-09-09T23:12:14.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.330+0000 7f98e7b61700 1 -- 172.21.15.82:0/1919413983 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e01a0b40 con 0x7f98e0100600 2023-09-09T23:12:14.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.331+0000 7f98e7b61700 1 -- 172.21.15.82:0/1919413983 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e0107c90 con 0x7f98e0100600 2023-09-09T23:12:14.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.332+0000 7f98d6ffd700 1 -- 172.21.15.82:0/1919413983 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f98d0003d40 con 0x7f98e0100600 2023-09-09T23:12:14.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.333+0000 7f98d6ffd700 1 --2- 172.21.15.82:0/1919413983 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f98cc0a0640 0x7f98cc0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:14.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.333+0000 7f98e50fc700 1 --2- 172.21.15.82:0/1919413983 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f98cc0a0640 0x7f98cc0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:14.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.333+0000 7f98d6ffd700 1 -- 172.21.15.82:0/1919413983 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f98d00cc9c0 con 0x7f98e0100600 2023-09-09T23:12:14.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.333+0000 7f98e50fc700 1 --2- 172.21.15.82:0/1919413983 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f98cc0a0640 0x7f98cc0a2b00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f98dc005950 tx=0x7f98dc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:14.340 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.337+0000 7f98d6ffd700 1 -- 172.21.15.82:0/1919413983 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f98d0096a20 con 0x7f98e0100600 2023-09-09T23:12:14.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:14 smithi082 ceph-mon[98694]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:12:14.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:14 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2914927331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:12:14.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:14 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:14.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:14 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1009385674' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:12:14.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.568+0000 7f98e7b61700 1 -- 172.21.15.82:0/1919413983 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f98e010c0a0 con 0x7f98e0100600 2023-09-09T23:12:14.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.568+0000 7f98d6ffd700 1 -- 172.21.15.82:0/1919413983 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f98d009a040 con 0x7f98e0100600 2023-09-09T23:12:14.567 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2023-09-09T23:12:14.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.570+0000 7f98e7b61700 1 -- 172.21.15.82:0/1919413983 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f98cc0a0640 msgr2=0x7f98cc0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:14.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.570+0000 7f98e7b61700 1 --2- 172.21.15.82:0/1919413983 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f98cc0a0640 0x7f98cc0a2b00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f98dc005950 tx=0x7f98dc0058e0 comp rx=0 tx=0).stop 2023-09-09T23:12:14.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.570+0000 7f98e7b61700 1 -- 172.21.15.82:0/1919413983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98e0100600 msgr2=0x7f98e00743b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:14.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.570+0000 7f98e7b61700 1 --2- 172.21.15.82:0/1919413983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98e0100600 0x7f98e00743b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009990 tx=0x7f98d0005e80 comp rx=0 tx=0).stop 2023-09-09T23:12:14.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.570+0000 7f98e7b61700 1 -- 172.21.15.82:0/1919413983 shutdown_connections 2023-09-09T23:12:14.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.570+0000 7f98e7b61700 1 --2- 172.21.15.82:0/1919413983 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f98cc0a0640 0x7f98cc0a2b00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:14.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.570+0000 7f98e7b61700 1 --2- 172.21.15.82:0/1919413983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98e0102190 0x7f98e0072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:14.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.570+0000 7f98e7b61700 1 --2- 172.21.15.82:0/1919413983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98e0100600 0x7f98e00743b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:14.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.570+0000 7f98e7b61700 1 --2- 172.21.15.82:0/1919413983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98e01017f0 0x7f98e00729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:14.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.570+0000 7f98e7b61700 1 -- 172.21.15.82:0/1919413983 >> 172.21.15.82:0/1919413983 conn(0x7f98e00fbb50 msgr2=0x7f98e0104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:14.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.570+0000 7f98e7b61700 1 -- 172.21.15.82:0/1919413983 shutdown_connections 2023-09-09T23:12:14.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.570+0000 7f98e7b61700 1 -- 172.21.15.82:0/1919413983 wait complete. 2023-09-09T23:12:14.584 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-09T23:12:14.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:14 smithi170 ceph-mon[102832]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:12:14.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:14 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2914927331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:12:14.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:14 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:14.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:14 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1009385674' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:12:14.721 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.722+0000 7f7be26f8700 1 -- 172.21.15.82:0/2046363853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bdc100610 msgr2=0x7f7bdc100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:14.722 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.722+0000 7f7be26f8700 1 --2- 172.21.15.82:0/2046363853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bdc100610 0x7f7bdc100a30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0099c0 tx=0x7f7bcc009cd0 comp rx=0 tx=0).stop 2023-09-09T23:12:14.722 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.722+0000 7f7be26f8700 1 -- 172.21.15.82:0/2046363853 shutdown_connections 2023-09-09T23:12:14.722 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.722+0000 7f7be26f8700 1 --2- 172.21.15.82:0/2046363853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7bdc1021a0 0x7f7bdc108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:14.722 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.722+0000 7f7be26f8700 1 --2- 172.21.15.82:0/2046363853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bdc100610 0x7f7bdc100a30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:14.722 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.722+0000 7f7be26f8700 1 --2- 172.21.15.82:0/2046363853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bdc101800 0x7f7bdc101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:14.723 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.722+0000 7f7be26f8700 1 -- 172.21.15.82:0/2046363853 >> 172.21.15.82:0/2046363853 conn(0x7f7bdc0fbb80 msgr2=0x7f7bdc0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:14.723 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.722+0000 7f7be26f8700 1 -- 172.21.15.82:0/2046363853 shutdown_connections 2023-09-09T23:12:14.723 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.722+0000 7f7be26f8700 1 -- 172.21.15.82:0/2046363853 wait complete. 2023-09-09T23:12:14.723 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.723+0000 7f7be26f8700 1 Processor -- start 2023-09-09T23:12:14.723 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.723+0000 7f7be26f8700 1 -- start start 2023-09-09T23:12:14.725 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.723+0000 7f7be26f8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bdc100610 0x7f7bdc0743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:14.726 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.723+0000 7f7be26f8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7bdc101800 0x7f7bdc0729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:14.726 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.723+0000 7f7be26f8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bdc1021a0 0x7f7bdc072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:14.726 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.723+0000 7f7be26f8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc0736d0 con 0x7f7bdc1021a0 2023-09-09T23:12:14.726 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.723+0000 7f7be26f8700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc073840 con 0x7f7bdc100610 2023-09-09T23:12:14.726 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.723+0000 7f7be26f8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc1a0370 con 0x7f7bdc101800 2023-09-09T23:12:14.727 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.723+0000 7f7be0c95700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bdc1021a0 0x7f7bdc072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:14.727 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.723+0000 7f7be0c95700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bdc1021a0 0x7f7bdc072f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54370/0 (socket says 172.21.15.82:54370) 2023-09-09T23:12:14.727 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.723+0000 7f7be0c95700 1 -- 172.21.15.82:0/4119477920 learned_addr learned my addr 172.21.15.82:0/4119477920 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:14.727 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.723+0000 7f7bdbfff700 1 --2- 172.21.15.82:0/4119477920 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bdc100610 0x7f7bdc0743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:14.727 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.723+0000 7f7bdb7fe700 1 --2- 172.21.15.82:0/4119477920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7bdc101800 0x7f7bdc0729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:14.727 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.724+0000 7f7be0c95700 1 -- 172.21.15.82:0/4119477920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7bdc101800 msgr2=0x7f7bdc0729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:14.728 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.724+0000 7f7be0c95700 1 --2- 172.21.15.82:0/4119477920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7bdc101800 0x7f7bdc0729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:14.728 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.724+0000 7f7be0c95700 1 -- 172.21.15.82:0/4119477920 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bdc100610 msgr2=0x7f7bdc0743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:14.728 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.724+0000 7f7be0c95700 1 --2- 172.21.15.82:0/4119477920 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bdc100610 0x7f7bdc0743b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:14.728 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.724+0000 7f7be0c95700 1 -- 172.21.15.82:0/4119477920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc0095f0 con 0x7f7bdc1021a0 2023-09-09T23:12:14.728 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.724+0000 7f7bdbfff700 1 --2- 172.21.15.82:0/4119477920 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bdc100610 0x7f7bdc0743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:12:14.729 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.724+0000 7f7be0c95700 1 --2- 172.21.15.82:0/4119477920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bdc1021a0 0x7f7bdc072f30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7bd400ef30 tx=0x7f7bd400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:14.729 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.727+0000 7f7bd97fa700 1 -- 172.21.15.82:0/4119477920 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd4018070 con 0x7f7bdc1021a0 2023-09-09T23:12:14.729 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.727+0000 7f7be26f8700 1 -- 172.21.15.82:0/4119477920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bdc1a05f0 con 0x7f7bdc1021a0 2023-09-09T23:12:14.729 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.727+0000 7f7bd97fa700 1 -- 172.21.15.82:0/4119477920 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bd400f040 con 0x7f7bdc1021a0 2023-09-09T23:12:14.729 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.727+0000 7f7bd97fa700 1 -- 172.21.15.82:0/4119477920 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd400cd00 con 0x7f7bdc1021a0 2023-09-09T23:12:14.730 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.727+0000 7f7be26f8700 1 -- 172.21.15.82:0/4119477920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bdc1a0b00 con 0x7f7bdc1021a0 2023-09-09T23:12:14.731 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.728+0000 7f7be26f8700 1 -- 172.21.15.82:0/4119477920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bdc10be80 con 0x7f7bdc1021a0 2023-09-09T23:12:14.731 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.729+0000 7f7bd97fa700 1 -- 172.21.15.82:0/4119477920 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f7bd4003a00 con 0x7f7bdc1021a0 2023-09-09T23:12:14.731 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.729+0000 7f7bd97fa700 1 --2- 172.21.15.82:0/4119477920 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f7bc40a0690 0x7f7bc40a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:14.731 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.729+0000 7f7bdbfff700 1 --2- 172.21.15.82:0/4119477920 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f7bc40a0690 0x7f7bc40a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:14.731 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.729+0000 7f7bd97fa700 1 -- 172.21.15.82:0/4119477920 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7bd40cbbc0 con 0x7f7bdc1021a0 2023-09-09T23:12:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.730+0000 7f7bdbfff700 1 --2- 172.21.15.82:0/4119477920 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f7bc40a0690 0x7f7bc40a2b50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc009990 tx=0x7f7bcc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.733+0000 7f7bd97fa700 1 -- 172.21.15.82:0/4119477920 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7bd4095c20 con 0x7f7bdc1021a0 2023-09-09T23:12:14.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:14 smithi098 ceph-mon[103124]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:12:14.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:14 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2914927331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:12:14.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:14 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:14.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:14 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1009385674' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:12:14.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.927+0000 7f7be26f8700 1 -- 172.21.15.82:0/4119477920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7bdc10c090 con 0x7f7bdc1021a0 2023-09-09T23:12:14.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.927+0000 7f7bd97fa700 1 -- 172.21.15.82:0/4119477920 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) v1 ==== 84+0+65 (secure 0 0 0) 0x7f7bd4099240 con 0x7f7bdc1021a0 2023-09-09T23:12:14.927 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:12:14.927 INFO:teuthology.orchestra.run.smithi082.stdout:[ 2023-09-09T23:12:14.927 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-09-09T23:12:14.927 INFO:teuthology.orchestra.run.smithi082.stdout: "poolnum": 1, 2023-09-09T23:12:14.927 INFO:teuthology.orchestra.run.smithi082.stdout: "poolname": ".mgr" 2023-09-09T23:12:14.928 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:12:14.928 INFO:teuthology.orchestra.run.smithi082.stdout:] 2023-09-09T23:12:14.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.930+0000 7f7be26f8700 1 -- 172.21.15.82:0/4119477920 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f7bc40a0690 msgr2=0x7f7bc40a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:14.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.930+0000 7f7be26f8700 1 --2- 172.21.15.82:0/4119477920 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f7bc40a0690 0x7f7bc40a2b50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc009990 tx=0x7f7bcc01a040 comp rx=0 tx=0).stop 2023-09-09T23:12:14.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.930+0000 7f7be26f8700 1 -- 172.21.15.82:0/4119477920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bdc1021a0 msgr2=0x7f7bdc072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:14.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.930+0000 7f7be26f8700 1 --2- 172.21.15.82:0/4119477920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bdc1021a0 0x7f7bdc072f30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7bd400ef30 tx=0x7f7bd400c3a0 comp rx=0 tx=0).stop 2023-09-09T23:12:14.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.930+0000 7f7be26f8700 1 -- 172.21.15.82:0/4119477920 shutdown_connections 2023-09-09T23:12:14.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.930+0000 7f7be26f8700 1 --2- 172.21.15.82:0/4119477920 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f7bc40a0690 0x7f7bc40a2b50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:14.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.930+0000 7f7be26f8700 1 --2- 172.21.15.82:0/4119477920 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bdc100610 0x7f7bdc0743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:14.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.930+0000 7f7be26f8700 1 --2- 172.21.15.82:0/4119477920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bdc1021a0 0x7f7bdc072f30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:14.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.930+0000 7f7be26f8700 1 --2- 172.21.15.82:0/4119477920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7bdc101800 0x7f7bdc0729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:14.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.930+0000 7f7be26f8700 1 -- 172.21.15.82:0/4119477920 >> 172.21.15.82:0/4119477920 conn(0x7f7bdc0fbb80 msgr2=0x7f7bdc0fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:14.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.930+0000 7f7be26f8700 1 -- 172.21.15.82:0/4119477920 shutdown_connections 2023-09-09T23:12:14.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:14.930+0000 7f7be26f8700 1 -- 172.21.15.82:0/4119477920 wait complete. 2023-09-09T23:12:14.940 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-09T23:12:14.941 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-09T23:12:15.067 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.068+0000 7f573645d700 1 -- 172.21.15.82:0/1213718223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5730101810 msgr2=0x7f5730101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:15.067 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.068+0000 7f573645d700 1 --2- 172.21.15.82:0/1213718223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5730101810 0x7f5730101c70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f57240099c0 tx=0x7f5724009cd0 comp rx=0 tx=0).stop 2023-09-09T23:12:15.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.068+0000 7f573645d700 1 -- 172.21.15.82:0/1213718223 shutdown_connections 2023-09-09T23:12:15.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.068+0000 7f573645d700 1 --2- 172.21.15.82:0/1213718223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57301021b0 0x7f5730108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:15.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.068+0000 7f573645d700 1 --2- 172.21.15.82:0/1213718223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5730100620 0x7f5730100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:15.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.068+0000 7f573645d700 1 --2- 172.21.15.82:0/1213718223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5730101810 0x7f5730101c70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:15.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.068+0000 7f573645d700 1 -- 172.21.15.82:0/1213718223 >> 172.21.15.82:0/1213718223 conn(0x7f57300fbb70 msgr2=0x7f57300fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:15.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.068+0000 7f573645d700 1 -- 172.21.15.82:0/1213718223 shutdown_connections 2023-09-09T23:12:15.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.068+0000 7f573645d700 1 -- 172.21.15.82:0/1213718223 wait complete. 2023-09-09T23:12:15.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.069+0000 7f573645d700 1 Processor -- start 2023-09-09T23:12:15.072 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.069+0000 7f573645d700 1 -- start start 2023-09-09T23:12:15.072 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.069+0000 7f573645d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5730100620 0x7f5730196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:15.072 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.069+0000 7f573645d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5730101810 0x7f5730196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:15.072 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.069+0000 7f573645d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57301021b0 0x7f573019d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:15.072 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.069+0000 7f573645d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573019dec0 con 0x7f5730101810 2023-09-09T23:12:15.073 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.069+0000 7f573645d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573019e000 con 0x7f57301021b0 2023-09-09T23:12:15.073 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.069+0000 7f573645d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573019e190 con 0x7f5730100620 2023-09-09T23:12:15.073 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.070+0000 7f572f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5730101810 0x7f5730196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:15.073 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.070+0000 7f572f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5730101810 0x7f5730196880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54382/0 (socket says 172.21.15.82:54382) 2023-09-09T23:12:15.073 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.070+0000 7f572f7fe700 1 -- 172.21.15.82:0/1004837068 learned_addr learned my addr 172.21.15.82:0/1004837068 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:15.074 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.070+0000 7f57349fa700 1 --2- 172.21.15.82:0/1004837068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57301021b0 0x7f573019d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:15.074 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.070+0000 7f572ffff700 1 --2- 172.21.15.82:0/1004837068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5730100620 0x7f5730196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:15.074 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.070+0000 7f572f7fe700 1 -- 172.21.15.82:0/1004837068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5730100620 msgr2=0x7f5730196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:15.074 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.070+0000 7f572f7fe700 1 --2- 172.21.15.82:0/1004837068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5730100620 0x7f5730196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:15.074 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.070+0000 7f572f7fe700 1 -- 172.21.15.82:0/1004837068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57301021b0 msgr2=0x7f573019d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:15.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.070+0000 7f572f7fe700 1 --2- 172.21.15.82:0/1004837068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57301021b0 0x7f573019d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:15.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.070+0000 7f572f7fe700 1 -- 172.21.15.82:0/1004837068 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57240095f0 con 0x7f5730101810 2023-09-09T23:12:15.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.070+0000 7f572f7fe700 1 --2- 172.21.15.82:0/1004837068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5730101810 0x7f5730196880 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f572400c010 tx=0x7f5724009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:15.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.070+0000 7f572d7fa700 1 -- 172.21.15.82:0/1004837068 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572401d070 con 0x7f5730101810 2023-09-09T23:12:15.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.070+0000 7f573645d700 1 -- 172.21.15.82:0/1004837068 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f573019e410 con 0x7f5730101810 2023-09-09T23:12:15.076 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.070+0000 7f572d7fa700 1 -- 172.21.15.82:0/1004837068 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5724003f90 con 0x7f5730101810 2023-09-09T23:12:15.076 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.070+0000 7f572d7fa700 1 -- 172.21.15.82:0/1004837068 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5724022d50 con 0x7f5730101810 2023-09-09T23:12:15.076 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.070+0000 7f573645d700 1 -- 172.21.15.82:0/1004837068 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f573019e830 con 0x7f5730101810 2023-09-09T23:12:15.076 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.071+0000 7f573645d700 1 -- 172.21.15.82:0/1004837068 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5730107be0 con 0x7f5730101810 2023-09-09T23:12:15.076 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.072+0000 7f572d7fa700 1 -- 172.21.15.82:0/1004837068 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f5724022470 con 0x7f5730101810 2023-09-09T23:12:15.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.073+0000 7f572d7fa700 1 --2- 172.21.15.82:0/1004837068 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f57180a0640 0x7f57180a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:15.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.073+0000 7f572ffff700 1 --2- 172.21.15.82:0/1004837068 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f57180a0640 0x7f57180a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:15.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.073+0000 7f572d7fa700 1 -- 172.21.15.82:0/1004837068 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f57240cd1b0 con 0x7f5730101810 2023-09-09T23:12:15.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.073+0000 7f572ffff700 1 --2- 172.21.15.82:0/1004837068 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f57180a0640 0x7f57180a2b00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5730101670 tx=0x7f572000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:15.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.076+0000 7f572d7fa700 1 -- 172.21.15.82:0/1004837068 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5724097210 con 0x7f5730101810 2023-09-09T23:12:15.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:15.279+0000 7f573645d700 1 -- 172.21.15.82:0/1004837068 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f573002d050 con 0x7f5730101810 2023-09-09T23:12:15.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:15 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1919413983' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:12:15.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:15 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:15.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:15 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:15.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:15 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:15.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:15 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:15.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:15 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:15.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:15 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:15 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/4119477920' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-09-09T23:12:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:15 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:12:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:15 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:12:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:15 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:15 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:12:15.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:15 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1919413983' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:12:15.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:15 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:15.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:15 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:15.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:15 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:15.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:15 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:15 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:15 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:15 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/4119477920' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-09-09T23:12:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:15 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:12:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:15 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:12:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:15 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:15 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:12:16.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:15 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1919413983' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:12:16.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:15 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:16.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:15 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:16.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:15 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:16.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:15 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:16.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:15 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:16.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:15 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:15 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/4119477920' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-09-09T23:12:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:15 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:12:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:15 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:12:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:15 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:15 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:12:16.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.151+0000 7f572d7fa700 1 -- 172.21.15.82:0/1004837068 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) v1 ==== 138+0+0 (secure 0 0 0) 0x7f572409a830 con 0x7f5730101810 2023-09-09T23:12:16.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.154+0000 7f573645d700 1 -- 172.21.15.82:0/1004837068 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f57180a0640 msgr2=0x7f57180a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:16.153 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.154+0000 7f573645d700 1 --2- 172.21.15.82:0/1004837068 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f57180a0640 0x7f57180a2b00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5730101670 tx=0x7f572000b410 comp rx=0 tx=0).stop 2023-09-09T23:12:16.153 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.154+0000 7f573645d700 1 -- 172.21.15.82:0/1004837068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5730101810 msgr2=0x7f5730196880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:16.153 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.154+0000 7f573645d700 1 --2- 172.21.15.82:0/1004837068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5730101810 0x7f5730196880 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f572400c010 tx=0x7f5724009580 comp rx=0 tx=0).stop 2023-09-09T23:12:16.153 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.154+0000 7f573645d700 1 -- 172.21.15.82:0/1004837068 shutdown_connections 2023-09-09T23:12:16.153 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.154+0000 7f573645d700 1 --2- 172.21.15.82:0/1004837068 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f57180a0640 0x7f57180a2b00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:16.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.154+0000 7f573645d700 1 --2- 172.21.15.82:0/1004837068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57301021b0 0x7f573019d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:16.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.154+0000 7f573645d700 1 --2- 172.21.15.82:0/1004837068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5730100620 0x7f5730196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:16.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.154+0000 7f573645d700 1 --2- 172.21.15.82:0/1004837068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5730101810 0x7f5730196880 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:16.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.154+0000 7f573645d700 1 -- 172.21.15.82:0/1004837068 >> 172.21.15.82:0/1004837068 conn(0x7f57300fbb70 msgr2=0x7f5730104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:16.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.154+0000 7f573645d700 1 -- 172.21.15.82:0/1004837068 shutdown_connections 2023-09-09T23:12:16.155 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.155+0000 7f573645d700 1 -- 172.21.15.82:0/1004837068 wait complete. 2023-09-09T23:12:16.158 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' created 2023-09-09T23:12:16.170 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-09-09T23:12:16.311 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.312+0000 7f1bf107c700 1 -- 172.21.15.82:0/2785265983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bec0ff7c0 msgr2=0x7f1bec0ffc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:16.311 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.312+0000 7f1bf107c700 1 --2- 172.21.15.82:0/2785265983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bec0ff7c0 0x7f1bec0ffc40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1be0009990 tx=0x7f1be0009ca0 comp rx=0 tx=0).stop 2023-09-09T23:12:16.311 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.312+0000 7f1bf107c700 1 -- 172.21.15.82:0/2785265983 shutdown_connections 2023-09-09T23:12:16.312 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.312+0000 7f1bf107c700 1 --2- 172.21.15.82:0/2785265983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bec0fee60 0x7f1bec0ff280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:16.312 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.312+0000 7f1bf107c700 1 --2- 172.21.15.82:0/2785265983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bec100180 0x7f1bec100600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:16.312 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.312+0000 7f1bf107c700 1 --2- 172.21.15.82:0/2785265983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bec0ff7c0 0x7f1bec0ffc40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:16.312 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.312+0000 7f1bf107c700 1 -- 172.21.15.82:0/2785265983 >> 172.21.15.82:0/2785265983 conn(0x7f1bec0faa60 msgr2=0x7f1bec0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:16.312 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.312+0000 7f1bf107c700 1 -- 172.21.15.82:0/2785265983 shutdown_connections 2023-09-09T23:12:16.313 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.312+0000 7f1bf107c700 1 -- 172.21.15.82:0/2785265983 wait complete. 2023-09-09T23:12:16.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.313+0000 7f1bf107c700 1 Processor -- start 2023-09-09T23:12:16.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.313+0000 7f1bf107c700 1 -- start start 2023-09-09T23:12:16.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.313+0000 7f1bf107c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bec0fee60 0x7f1bec198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:16.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.313+0000 7f1bf107c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bec0ff7c0 0x7f1bec1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:16.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.313+0000 7f1bf107c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bec100180 0x7f1bec19fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:16.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.313+0000 7f1bf107c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec1a0010 con 0x7f1bec100180 2023-09-09T23:12:16.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.313+0000 7f1bf107c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec1a0150 con 0x7f1bec0fee60 2023-09-09T23:12:16.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.313+0000 7f1bf107c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec1a02e0 con 0x7f1bec0ff7c0 2023-09-09T23:12:16.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.314+0000 7f1beb59e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bec100180 0x7f1bec19fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:16.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.314+0000 7f1bead9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bec0fee60 0x7f1bec198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:16.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.314+0000 7f1bead9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bec0fee60 0x7f1bec198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:39958/0 (socket says 172.21.15.82:39958) 2023-09-09T23:12:16.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.314+0000 7f1beb59e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bec100180 0x7f1bec19fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54398/0 (socket says 172.21.15.82:54398) 2023-09-09T23:12:16.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.314+0000 7f1bead9d700 1 -- 172.21.15.82:0/417349896 learned_addr learned my addr 172.21.15.82:0/417349896 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:16.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.314+0000 7f1bea59c700 1 --2- 172.21.15.82:0/417349896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bec0ff7c0 0x7f1bec1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:16.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.314+0000 7f1beb59e700 1 -- 172.21.15.82:0/417349896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bec0ff7c0 msgr2=0x7f1bec1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:16.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.314+0000 7f1beb59e700 1 --2- 172.21.15.82:0/417349896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bec0ff7c0 0x7f1bec1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:16.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.314+0000 7f1beb59e700 1 -- 172.21.15.82:0/417349896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bec0fee60 msgr2=0x7f1bec198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:16.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.314+0000 7f1beb59e700 1 --2- 172.21.15.82:0/417349896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bec0fee60 0x7f1bec198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:16.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.314+0000 7f1beb59e700 1 -- 172.21.15.82:0/417349896 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be00095f0 con 0x7f1bec100180 2023-09-09T23:12:16.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.314+0000 7f1bead9d700 1 --2- 172.21.15.82:0/417349896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bec0fee60 0x7f1bec198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:12:16.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.314+0000 7f1bea59c700 1 --2- 172.21.15.82:0/417349896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bec0ff7c0 0x7f1bec1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:12:16.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.314+0000 7f1beb59e700 1 --2- 172.21.15.82:0/417349896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bec100180 0x7f1bec19fa40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1be400efc0 tx=0x7f1be400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:16.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.314+0000 7f1bd3fff700 1 -- 172.21.15.82:0/417349896 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be4018070 con 0x7f1bec100180 2023-09-09T23:12:16.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.314+0000 7f1bf107c700 1 -- 172.21.15.82:0/417349896 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bec1a0560 con 0x7f1bec100180 2023-09-09T23:12:16.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.314+0000 7f1bd3fff700 1 -- 172.21.15.82:0/417349896 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1be400f040 con 0x7f1bec100180 2023-09-09T23:12:16.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.314+0000 7f1bd3fff700 1 -- 172.21.15.82:0/417349896 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be400cd40 con 0x7f1bec100180 2023-09-09T23:12:16.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.314+0000 7f1bf107c700 1 -- 172.21.15.82:0/417349896 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bec1a0a70 con 0x7f1bec100180 2023-09-09T23:12:16.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.315+0000 7f1bf107c700 1 -- 172.21.15.82:0/417349896 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bec109c40 con 0x7f1bec100180 2023-09-09T23:12:16.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.318+0000 7f1bd3fff700 1 -- 172.21.15.82:0/417349896 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f1be4003a00 con 0x7f1bec100180 2023-09-09T23:12:16.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.318+0000 7f1bd3fff700 1 --2- 172.21.15.82:0/417349896 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1bd40a0640 0x7f1bd40a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:16.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.318+0000 7f1bd3fff700 1 -- 172.21.15.82:0/417349896 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f1be40cbde0 con 0x7f1bec100180 2023-09-09T23:12:16.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.318+0000 7f1bead9d700 1 --2- 172.21.15.82:0/417349896 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1bd40a0640 0x7f1bd40a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:16.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.319+0000 7f1bead9d700 1 --2- 172.21.15.82:0/417349896 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1bd40a0640 0x7f1bd40a2b00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc011550 tx=0x7f1bdc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:16.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.321+0000 7f1bd3fff700 1 -- 172.21.15.82:0/417349896 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1be4095cd0 con 0x7f1bec100180 2023-09-09T23:12:16.536 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:16.538+0000 7f1bf107c700 1 -- 172.21.15.82:0/417349896 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1bec10e0c0 con 0x7f1bec100180 2023-09-09T23:12:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:16 smithi170 ceph-mon[102832]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:12:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:16 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1004837068' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-09T23:12:16.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:16 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1004837068' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-09-09T23:12:16.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:16 smithi170 ceph-mon[102832]: osdmap e78: 12 total, 12 up, 12 in 2023-09-09T23:12:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:16 smithi082 ceph-mon[98694]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:12:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:16 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1004837068' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-09T23:12:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:16 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1004837068' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-09-09T23:12:16.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:16 smithi082 ceph-mon[98694]: osdmap e78: 12 total, 12 up, 12 in 2023-09-09T23:12:17.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:16 smithi098 ceph-mon[103124]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:12:17.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:16 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1004837068' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-09T23:12:17.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:16 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1004837068' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-09-09T23:12:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:16 smithi098 ceph-mon[103124]: osdmap e78: 12 total, 12 up, 12 in 2023-09-09T23:12:17.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.154+0000 7f1bd3fff700 1 -- 172.21.15.82:0/417349896 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v79) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1be40992f0 con 0x7f1bec100180 2023-09-09T23:12:17.155 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.157+0000 7f1bf107c700 1 -- 172.21.15.82:0/417349896 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1bd40a0640 msgr2=0x7f1bd40a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:17.156 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.157+0000 7f1bf107c700 1 --2- 172.21.15.82:0/417349896 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1bd40a0640 0x7f1bd40a2b00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc011550 tx=0x7f1bdc009260 comp rx=0 tx=0).stop 2023-09-09T23:12:17.156 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.157+0000 7f1bf107c700 1 -- 172.21.15.82:0/417349896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bec100180 msgr2=0x7f1bec19fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:17.156 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.157+0000 7f1bf107c700 1 --2- 172.21.15.82:0/417349896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bec100180 0x7f1bec19fa40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1be400efc0 tx=0x7f1be400c3a0 comp rx=0 tx=0).stop 2023-09-09T23:12:17.156 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.157+0000 7f1bf107c700 1 -- 172.21.15.82:0/417349896 shutdown_connections 2023-09-09T23:12:17.156 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.157+0000 7f1bf107c700 1 --2- 172.21.15.82:0/417349896 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1bd40a0640 0x7f1bd40a2b00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:17.157 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.157+0000 7f1bf107c700 1 --2- 172.21.15.82:0/417349896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bec0fee60 0x7f1bec198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:17.157 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.157+0000 7f1bf107c700 1 --2- 172.21.15.82:0/417349896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bec100180 0x7f1bec19fa40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:17.157 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.157+0000 7f1bf107c700 1 --2- 172.21.15.82:0/417349896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bec0ff7c0 0x7f1bec1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:17.157 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.157+0000 7f1bf107c700 1 -- 172.21.15.82:0/417349896 >> 172.21.15.82:0/417349896 conn(0x7f1bec0faa60 msgr2=0x7f1bec0fc780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:17.158 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.157+0000 7f1bf107c700 1 -- 172.21.15.82:0/417349896 shutdown_connections 2023-09-09T23:12:17.158 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.159+0000 7f1bf107c700 1 -- 172.21.15.82:0/417349896 wait complete. 2023-09-09T23:12:17.168 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' created 2023-09-09T23:12:17.190 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-09T23:12:17.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.324+0000 7f5b6213c700 1 -- 172.21.15.82:0/1861187382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b5c1006e0 msgr2=0x7f5b5c100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:17.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.324+0000 7f5b6213c700 1 --2- 172.21.15.82:0/1861187382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b5c1006e0 0x7f5b5c100b40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c0099c0 tx=0x7f5b4c009cd0 comp rx=0 tx=0).stop 2023-09-09T23:12:17.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.324+0000 7f5b6213c700 1 -- 172.21.15.82:0/1861187382 shutdown_connections 2023-09-09T23:12:17.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.324+0000 7f5b6213c700 1 --2- 172.21.15.82:0/1861187382 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b5c0ff4f0 0x7f5b5c0ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:17.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.324+0000 7f5b6213c700 1 --2- 172.21.15.82:0/1861187382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b5c101080 0x7f5b5c107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:17.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.324+0000 7f5b6213c700 1 --2- 172.21.15.82:0/1861187382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b5c1006e0 0x7f5b5c100b40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:17.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.324+0000 7f5b6213c700 1 -- 172.21.15.82:0/1861187382 >> 172.21.15.82:0/1861187382 conn(0x7f5b5c0faa60 msgr2=0x7f5b5c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:17.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.324+0000 7f5b6213c700 1 -- 172.21.15.82:0/1861187382 shutdown_connections 2023-09-09T23:12:17.325 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.324+0000 7f5b6213c700 1 -- 172.21.15.82:0/1861187382 wait complete. 2023-09-09T23:12:17.330 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.325+0000 7f5b6213c700 1 Processor -- start 2023-09-09T23:12:17.330 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.325+0000 7f5b6213c700 1 -- start start 2023-09-09T23:12:17.331 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.325+0000 7f5b6213c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b5c0ff4f0 0x7f5b5c195240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:17.331 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.325+0000 7f5b6213c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b5c1006e0 0x7f5b5c195780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:17.331 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.325+0000 7f5b6213c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b5c101080 0x7f5b5c19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:17.331 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.325+0000 7f5b6213c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c19cdc0 con 0x7f5b5c1006e0 2023-09-09T23:12:17.331 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.325+0000 7f5b6213c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c19cf00 con 0x7f5b5c101080 2023-09-09T23:12:17.332 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.325+0000 7f5b6213c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c19d090 con 0x7f5b5c0ff4f0 2023-09-09T23:12:17.332 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.325+0000 7f5b5affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b5c1006e0 0x7f5b5c195780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:17.332 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.325+0000 7f5b5affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b5c1006e0 0x7f5b5c195780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54414/0 (socket says 172.21.15.82:54414) 2023-09-09T23:12:17.332 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.325+0000 7f5b5affd700 1 -- 172.21.15.82:0/2378038692 learned_addr learned my addr 172.21.15.82:0/2378038692 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:17.332 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.325+0000 7f5b5b7fe700 1 --2- 172.21.15.82:0/2378038692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b5c0ff4f0 0x7f5b5c195240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:17.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.325+0000 7f5b5bfff700 1 --2- 172.21.15.82:0/2378038692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b5c101080 0x7f5b5c19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:17.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.326+0000 7f5b5affd700 1 -- 172.21.15.82:0/2378038692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b5c0ff4f0 msgr2=0x7f5b5c195240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:17.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.326+0000 7f5b5affd700 1 --2- 172.21.15.82:0/2378038692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b5c0ff4f0 0x7f5b5c195240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:17.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.326+0000 7f5b5affd700 1 -- 172.21.15.82:0/2378038692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b5c101080 msgr2=0x7f5b5c19c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:17.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.326+0000 7f5b5affd700 1 --2- 172.21.15.82:0/2378038692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b5c101080 0x7f5b5c19c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:17.334 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.326+0000 7f5b5affd700 1 -- 172.21.15.82:0/2378038692 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c0095f0 con 0x7f5b5c1006e0 2023-09-09T23:12:17.334 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.326+0000 7f5b5affd700 1 --2- 172.21.15.82:0/2378038692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b5c1006e0 0x7f5b5c195780 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009990 tx=0x7f5b4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:17.334 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.326+0000 7f5b58ff9700 1 -- 172.21.15.82:0/2378038692 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c01d070 con 0x7f5b5c1006e0 2023-09-09T23:12:17.334 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.326+0000 7f5b6213c700 1 -- 172.21.15.82:0/2378038692 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b5c19d310 con 0x7f5b5c1006e0 2023-09-09T23:12:17.334 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.326+0000 7f5b58ff9700 1 -- 172.21.15.82:0/2378038692 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b4c004010 con 0x7f5b5c1006e0 2023-09-09T23:12:17.335 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.326+0000 7f5b58ff9700 1 -- 172.21.15.82:0/2378038692 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c022d40 con 0x7f5b5c1006e0 2023-09-09T23:12:17.335 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.326+0000 7f5b6213c700 1 -- 172.21.15.82:0/2378038692 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b5c19d7c0 con 0x7f5b5c1006e0 2023-09-09T23:12:17.335 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.327+0000 7f5b6213c700 1 -- 172.21.15.82:0/2378038692 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b5c10ad80 con 0x7f5b5c1006e0 2023-09-09T23:12:17.335 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.328+0000 7f5b58ff9700 1 -- 172.21.15.82:0/2378038692 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f5b4c022470 con 0x7f5b5c1006e0 2023-09-09T23:12:17.335 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.329+0000 7f5b58ff9700 1 --2- 172.21.15.82:0/2378038692 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5b440a0640 0x7f5b440a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:17.335 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.329+0000 7f5b5b7fe700 1 --2- 172.21.15.82:0/2378038692 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5b440a0640 0x7f5b440a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:17.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.329+0000 7f5b58ff9700 1 -- 172.21.15.82:0/2378038692 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f5b4c0cd140 con 0x7f5b5c1006e0 2023-09-09T23:12:17.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.329+0000 7f5b5b7fe700 1 --2- 172.21.15.82:0/2378038692 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5b440a0640 0x7f5b440a2b00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c100540 tx=0x7f5b5000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:17.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.333+0000 7f5b58ff9700 1 -- 172.21.15.82:0/2378038692 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5b4c096eb0 con 0x7f5b5c1006e0 2023-09-09T23:12:17.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:17.583+0000 7f5b6213c700 1 -- 172.21.15.82:0/2378038692 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5b5c106410 con 0x7f5b5c1006e0 2023-09-09T23:12:17.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:17 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/417349896' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-09T23:12:17.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:17 smithi170 ceph-mon[102832]: pgmap v252: 33 pgs: 11 creating+peering, 1 active+clean, 21 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:12:17.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:17 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/417349896' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-09-09T23:12:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:17 smithi170 ceph-mon[102832]: osdmap e79: 12 total, 12 up, 12 in 2023-09-09T23:12:17.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:17 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/417349896' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-09T23:12:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:17 smithi082 ceph-mon[98694]: pgmap v252: 33 pgs: 11 creating+peering, 1 active+clean, 21 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:12:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:17 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/417349896' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-09-09T23:12:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:17 smithi082 ceph-mon[98694]: osdmap e79: 12 total, 12 up, 12 in 2023-09-09T23:12:18.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:17 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/417349896' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-09T23:12:18.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:17 smithi098 ceph-mon[103124]: pgmap v252: 33 pgs: 11 creating+peering, 1 active+clean, 21 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:12:18.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:17 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/417349896' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-09-09T23:12:18.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:17 smithi098 ceph-mon[103124]: osdmap e79: 12 total, 12 up, 12 in 2023-09-09T23:12:18.531 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.532+0000 7f5b58ff9700 1 -- 172.21.15.82:0/2378038692 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f5b4c09a4d0 con 0x7f5b5c1006e0 2023-09-09T23:12:18.533 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.534+0000 7f5b6213c700 1 -- 172.21.15.82:0/2378038692 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5b440a0640 msgr2=0x7f5b440a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:18.533 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.534+0000 7f5b6213c700 1 --2- 172.21.15.82:0/2378038692 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5b440a0640 0x7f5b440a2b00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c100540 tx=0x7f5b5000b410 comp rx=0 tx=0).stop 2023-09-09T23:12:18.534 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.534+0000 7f5b6213c700 1 -- 172.21.15.82:0/2378038692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b5c1006e0 msgr2=0x7f5b5c195780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:18.534 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.534+0000 7f5b6213c700 1 --2- 172.21.15.82:0/2378038692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b5c1006e0 0x7f5b5c195780 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009990 tx=0x7f5b4c004730 comp rx=0 tx=0).stop 2023-09-09T23:12:18.534 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.534+0000 7f5b6213c700 1 -- 172.21.15.82:0/2378038692 shutdown_connections 2023-09-09T23:12:18.534 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.534+0000 7f5b6213c700 1 --2- 172.21.15.82:0/2378038692 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5b440a0640 0x7f5b440a2b00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:18.534 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.534+0000 7f5b6213c700 1 --2- 172.21.15.82:0/2378038692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b5c101080 0x7f5b5c19c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:18.535 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.534+0000 7f5b6213c700 1 --2- 172.21.15.82:0/2378038692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b5c0ff4f0 0x7f5b5c195240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:18.535 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.534+0000 7f5b6213c700 1 --2- 172.21.15.82:0/2378038692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b5c1006e0 0x7f5b5c195780 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:18.535 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.534+0000 7f5b6213c700 1 -- 172.21.15.82:0/2378038692 >> 172.21.15.82:0/2378038692 conn(0x7f5b5c0faa60 msgr2=0x7f5b5c0fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:18.535 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.535+0000 7f5b6213c700 1 -- 172.21.15.82:0/2378038692 shutdown_connections 2023-09-09T23:12:18.535 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.535+0000 7f5b6213c700 1 -- 172.21.15.82:0/2378038692 wait complete. 2023-09-09T23:12:18.536 INFO:teuthology.orchestra.run.smithi082.stderr:new fs with metadata pool 2 and data pool 3 2023-09-09T23:12:18.549 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-09T23:12:18.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.672+0000 7f3953c6c700 1 -- 172.21.15.82:0/2842231211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f394c101070 msgr2=0x7f394c107920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:18.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.672+0000 7f3953c6c700 1 --2- 172.21.15.82:0/2842231211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f394c101070 0x7f394c107920 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f394400b0d0 tx=0x7f394400b3e0 comp rx=0 tx=0).stop 2023-09-09T23:12:18.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.673+0000 7f3953c6c700 1 -- 172.21.15.82:0/2842231211 shutdown_connections 2023-09-09T23:12:18.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.673+0000 7f3953c6c700 1 --2- 172.21.15.82:0/2842231211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f394c101070 0x7f394c107920 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:18.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.673+0000 7f3953c6c700 1 --2- 172.21.15.82:0/2842231211 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f394c0ff4e0 0x7f394c0ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:18.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.673+0000 7f3953c6c700 1 --2- 172.21.15.82:0/2842231211 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f394c1006d0 0x7f394c100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:18.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.673+0000 7f3953c6c700 1 -- 172.21.15.82:0/2842231211 >> 172.21.15.82:0/2842231211 conn(0x7f394c0faa30 msgr2=0x7f394c0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:18.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.673+0000 7f3953c6c700 1 -- 172.21.15.82:0/2842231211 shutdown_connections 2023-09-09T23:12:18.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.673+0000 7f3953c6c700 1 -- 172.21.15.82:0/2842231211 wait complete. 2023-09-09T23:12:18.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.673+0000 7f3953c6c700 1 Processor -- start 2023-09-09T23:12:18.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.673+0000 7f3953c6c700 1 -- start start 2023-09-09T23:12:18.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.673+0000 7f3953c6c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f394c0ff4e0 0x7f394c0682f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:18.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.673+0000 7f3953c6c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f394c1006d0 0x7f394c19b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:18.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.673+0000 7f3953c6c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f394c101070 0x7f394c19d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:18.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.673+0000 7f3953c6c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c19dee0 con 0x7f394c0ff4e0 2023-09-09T23:12:18.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.673+0000 7f3953c6c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c19e050 con 0x7f394c1006d0 2023-09-09T23:12:18.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.673+0000 7f3953c6c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c19e210 con 0x7f394c101070 2023-09-09T23:12:18.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.673+0000 7f3951a08700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f394c0ff4e0 0x7f394c0682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:18.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.673+0000 7f3951a08700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f394c0ff4e0 0x7f394c0682f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54430/0 (socket says 172.21.15.82:54430) 2023-09-09T23:12:18.678 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.674+0000 7f3951a08700 1 -- 172.21.15.82:0/3021085440 learned_addr learned my addr 172.21.15.82:0/3021085440 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:18.678 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.674+0000 7f3951207700 1 --2- 172.21.15.82:0/3021085440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f394c1006d0 0x7f394c19b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:18.678 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.674+0000 7f3952209700 1 --2- 172.21.15.82:0/3021085440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f394c101070 0x7f394c19d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:18.678 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.674+0000 7f3951a08700 1 -- 172.21.15.82:0/3021085440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f394c101070 msgr2=0x7f394c19d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:18.678 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.674+0000 7f3951a08700 1 --2- 172.21.15.82:0/3021085440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f394c101070 0x7f394c19d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:18.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.674+0000 7f3951a08700 1 -- 172.21.15.82:0/3021085440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f394c1006d0 msgr2=0x7f394c19b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:18.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.674+0000 7f3951a08700 1 --2- 172.21.15.82:0/3021085440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f394c1006d0 0x7f394c19b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:18.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.674+0000 7f3951a08700 1 -- 172.21.15.82:0/3021085440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3944009d00 con 0x7f394c0ff4e0 2023-09-09T23:12:18.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.674+0000 7f3952209700 1 --2- 172.21.15.82:0/3021085440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f394c101070 0x7f394c19d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:12:18.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.674+0000 7f3951a08700 1 --2- 172.21.15.82:0/3021085440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f394c0ff4e0 0x7f394c0682f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f394800d980 tx=0x7f394800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:18.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.674+0000 7f3942ffd700 1 -- 172.21.15.82:0/3021085440 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394800b950 con 0x7f394c0ff4e0 2023-09-09T23:12:18.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.674+0000 7f3942ffd700 1 -- 172.21.15.82:0/3021085440 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3948007620 con 0x7f394c0ff4e0 2023-09-09T23:12:18.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.674+0000 7f3953c6c700 1 -- 172.21.15.82:0/3021085440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f394c19e4f0 con 0x7f394c0ff4e0 2023-09-09T23:12:18.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.674+0000 7f3942ffd700 1 -- 172.21.15.82:0/3021085440 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394800f3c0 con 0x7f394c0ff4e0 2023-09-09T23:12:18.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.674+0000 7f3953c6c700 1 -- 172.21.15.82:0/3021085440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f394c19ea50 con 0x7f394c0ff4e0 2023-09-09T23:12:18.681 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.675+0000 7f3953c6c700 1 -- 172.21.15.82:0/3021085440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f394c10ad50 con 0x7f394c0ff4e0 2023-09-09T23:12:18.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.676+0000 7f3942ffd700 1 -- 172.21.15.82:0/3021085440 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f3948010640 con 0x7f394c0ff4e0 2023-09-09T23:12:18.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.677+0000 7f3942ffd700 1 --2- 172.21.15.82:0/3021085440 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f39380a0710 0x7f39380a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:18.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.677+0000 7f3951207700 1 --2- 172.21.15.82:0/3021085440 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f39380a0710 0x7f39380a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:18.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.677+0000 7f3942ffd700 1 -- 172.21.15.82:0/3021085440 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f39480cc160 con 0x7f394c0ff4e0 2023-09-09T23:12:18.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.677+0000 7f3951207700 1 --2- 172.21.15.82:0/3021085440 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f39380a0710 0x7f39380a2bd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f394c068060 tx=0x7f393c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:18.683 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.680+0000 7f3942ffd700 1 -- 172.21.15.82:0/3021085440 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3948095de0 con 0x7f394c0ff4e0 2023-09-09T23:12:18.871 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.873+0000 7f3953c6c700 1 -- 172.21.15.82:0/3021085440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f394c10af60 con 0x7f394c0ff4e0 2023-09-09T23:12:18.873 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.874+0000 7f3942ffd700 1 -- 172.21.15.82:0/3021085440 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+22084 (secure 0 0 0) 0x7f3948016090 con 0x7f394c0ff4e0 2023-09-09T23:12:18.873 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:12:18.873 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":80,"fsid":"3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da","created":"2023-09-09T23:05:14.222190+0000","modified":"2023-09-09T23:12:17.585669+0000","last_up_change":"2023-09-09T23:11:37.253759+0000","last_in_change":"2023-09-09T23:11:19.933812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-09T23:08:31.234958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-09T23:12:15.524887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-09T23:12:16.785403+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1277eb17-a2e7-4e10-9832-f12a5e93ffde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6803","nonce":3197434929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6805","nonce":3197434929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6809","nonce":3197434929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6807","nonce":3197434929}]},"public_addr":"172.21.15.82:6803/3197434929","cluster_addr":"172.21.15.82:6805/3197434929","heartbeat_back_addr":"172.21.15.82:6809/3197434929","heartbeat_front_addr":"172.21.15.82:6807/3197434929","state":["exists","up"]},{"osd":1,"uuid":"f68723fd-0908-4e21-ae8e-1de1c3a9a65c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6801","nonce":3528641314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6803","nonce":3528641314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6807","nonce":3528641314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6805","nonce":3528641314}]},"public_addr":"172.21.15.98:6801/3528641314","cluster_addr":"172.21.15.98:6803/3528641314","heartbeat_back_addr":"172.21.15.98:6807/3528641314","heartbeat_front_addr":"172.21.15.98:6805/3528641314","state":["exists","up"]},{"osd":2,"uuid":"06f89c53-1419-4f52-acad-6e92916b476f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6801","nonce":4006414551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6803","nonce":4006414551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6807","nonce":4006414551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6805","nonce":4006414551}]},"public_addr":"172.21.15.170:6801/4006414551","cluster_addr":"172.21.15.170:6803/4006414551","heartbeat_back_addr":"172.21.15.170:6807/4006414551","heartbeat_front_addr":"172.21.15.170:6805/4006414551","state":["exists","up"]},{"osd":3,"uuid":"fedbd4c9-6852-42b5-8f3a-ba8d9bfe5d59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6811","nonce":3647629241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6813","nonce":3647629241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6817","nonce":3647629241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6815","nonce":3647629241}]},"public_addr":"172.21.15.82:6811/3647629241","cluster_addr":"172.21.15.82:6813/3647629241","heartbeat_back_addr":"172.21.15.82:6817/3647629241","heartbeat_front_addr":"172.21.15.82:6815/3647629241","state":["exists","up"]},{"osd":4,"uuid":"a71f3084-6919-4161-87bf-345f3355d8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6809","nonce":2507035643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6811","nonce":2507035643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6815","nonce":2507035643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6813","nonce":2507035643}]},"public_addr":"172.21.15.98:6809/2507035643","cluster_addr":"172.21.15.98:6811/2507035643","heartbeat_back_addr":"172.21.15.98:6815/2507035643","heartbeat_front_addr":"172.21.15.98:6813/2507035643","state":["exists","up"]},{"osd":5,"uuid":"f76058e9-a90e-476e-87f0-3c6723795e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6809","nonce":775005168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6811","nonce":775005168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6815","nonce":775005168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6813","nonce":775005168}]},"public_addr":"172.21.15.170:6809/775005168","cluster_addr":"172.21.15.170:6811/775005168","heartbeat_back_addr":"172.21.15.170:6815/775005168","heartbeat_front_addr":"172.21.15.170:6813/775005168","state":["exists","up"]},{"osd":6,"uuid":"25940be0-c1bd-4c72-95bf-fd8367aa2f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6819","nonce":3008405982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6821","nonce":3008405982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6825","nonce":3008405982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6823","nonce":3008405982}]},"public_addr":"172.21.15.82:6819/3008405982","cluster_addr":"172.21.15.82:6821/3008405982","heartbeat_back_addr":"172.21.15.82:6825/3008405982","heartbeat_front_addr":"172.21.15.82:6823/3008405982","state":["exists","up"]},{"osd":7,"uuid":"f651bb8c-0b25-42d5-b028-58563f4c27f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6817","nonce":3547615284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6819","nonce":3547615284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6823","nonce":3547615284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6821","nonce":3547615284}]},"public_addr":"172.21.15.98:6817/3547615284","cluster_addr":"172.21.15.98:6819/3547615284","heartbeat_back_addr":"172.21.15.98:6823/3547615284","heartbeat_front_addr":"172.21.15.98:6821/3547615284","state":["exists","up"]},{"osd":8,"uuid":"23b27c86-8c98-441e-b7ab-d57bc584e58f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6817","nonce":632038848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6819","nonce":632038848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6823","nonce":632038848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6821","nonce":632038848}]},"public_addr":"172.21.15.170:6817/632038848","cluster_addr":"172.21.15.170:6819/632038848","heartbeat_back_addr":"172.21.15.170:6823/632038848","heartbeat_front_addr":"172.21.15.170:6821/632038848","state":["exists","up"]},{"osd":9,"uuid":"383b8989-4922-4eee-9c77-f198258ac243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6827","nonce":3354242311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6829","nonce":3354242311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6833","nonce":3354242311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6831","nonce":3354242311}]},"public_addr":"172.21.15.82:6827/3354242311","cluster_addr":"172.21.15.82:6829/3354242311","heartbeat_back_addr":"172.21.15.82:6833/3354242311","heartbeat_front_addr":"172.21.15.82:6831/3354242311","state":["exists","up"]},{"osd":10,"uuid":"33a59389-7ced-42d5-bf54-2071317132d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6825","nonce":2069114959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6827","nonce":2069114959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6831","nonce":2069114959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6829","nonce":2069114959}]},"public_addr":"172.21.15.98:6825/2069114959","cluster_addr":"172.21.15.98:6827/2069114959","heartbeat_back_addr":"172.21.15.98:6831/2069114959","heartbeat_front_addr":"172.21.15.98:6829/2069114959","state":["exists","up"]},{"osd":11,"uuid":"b7a71f5c-85e2-4fec-be4b-95be0788b302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6825","nonce":146964512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6827","nonce":146964512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6831","nonce":146964512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6829","nonce":146964512}]},"public_addr":"172.21.15.170:6825/146964512","cluster_addr":"172.21.15.170:6827/146964512","heartbeat_back_addr":"172.21.15.170:6831/146964512","heartbeat_front_addr":"172.21.15.170:6829/146964512","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:07:47.563043+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:08.359260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:29.070374+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:49.897775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:11.434248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:31.538464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:52.377109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:13.452968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:33.806897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:54.083177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:14.980087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:35.690089+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4013747788":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6800/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/2823341464":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/75116806":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/1037625632":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/3142088817":"2023-09-10T23:05:46.550354+0000","172.21.15.82:6800/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/1679622963":"2023-09-10T23:05:46.550354+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-09T23:12:18.875 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.876+0000 7f3953c6c700 1 -- 172.21.15.82:0/3021085440 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f39380a0710 msgr2=0x7f39380a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:18.875 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.876+0000 7f3953c6c700 1 --2- 172.21.15.82:0/3021085440 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f39380a0710 0x7f39380a2bd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f394c068060 tx=0x7f393c00b410 comp rx=0 tx=0).stop 2023-09-09T23:12:18.876 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.876+0000 7f3953c6c700 1 -- 172.21.15.82:0/3021085440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f394c0ff4e0 msgr2=0x7f394c0682f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:18.876 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.876+0000 7f3953c6c700 1 --2- 172.21.15.82:0/3021085440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f394c0ff4e0 0x7f394c0682f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f394800d980 tx=0x7f394800dc90 comp rx=0 tx=0).stop 2023-09-09T23:12:18.876 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.876+0000 7f3953c6c700 1 -- 172.21.15.82:0/3021085440 shutdown_connections 2023-09-09T23:12:18.876 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.876+0000 7f3953c6c700 1 --2- 172.21.15.82:0/3021085440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f394c101070 0x7f394c19d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:18.876 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.876+0000 7f3953c6c700 1 --2- 172.21.15.82:0/3021085440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f394c0ff4e0 0x7f394c0682f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:18.876 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.876+0000 7f3953c6c700 1 --2- 172.21.15.82:0/3021085440 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f39380a0710 0x7f39380a2bd0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:18.877 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.876+0000 7f3953c6c700 1 --2- 172.21.15.82:0/3021085440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f394c1006d0 0x7f394c19b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:18.877 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.876+0000 7f3953c6c700 1 -- 172.21.15.82:0/3021085440 >> 172.21.15.82:0/3021085440 conn(0x7f394c0faa30 msgr2=0x7f394c0fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:18.877 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.876+0000 7f3953c6c700 1 -- 172.21.15.82:0/3021085440 shutdown_connections 2023-09-09T23:12:18.877 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:18.876+0000 7f3953c6c700 1 -- 172.21.15.82:0/3021085440 wait complete. 2023-09-09T23:12:18.886 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:18 smithi082 ceph-mon[98694]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-09T23:12:18.886 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:18 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2378038692' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-09T23:12:18.887 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:18 smithi082 ceph-mon[98694]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-09-09T23:12:18.887 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:18 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:18.887 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:18 smithi082 ceph-mon[98694]: osdmap e80: 12 total, 12 up, 12 in 2023-09-09T23:12:18.887 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:18 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:18.888 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:18 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:18.888 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:18 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:18.888 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:18 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:18.888 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:18 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:18.888 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:18 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:12:18.888 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:18 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:12:18.889 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:18 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:18.890 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-09T23:12:18.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:18 smithi170 ceph-mon[102832]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-09T23:12:18.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:18 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2378038692' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-09T23:12:18.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:18 smithi170 ceph-mon[102832]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-09-09T23:12:18.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:18 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:18.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:18 smithi170 ceph-mon[102832]: osdmap e80: 12 total, 12 up, 12 in 2023-09-09T23:12:18.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:18 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:18 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:18 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:18 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:18 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:18 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:12:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:18 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:12:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:18 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:19.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:18 smithi098 ceph-mon[103124]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-09T23:12:19.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:18 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2378038692' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-09T23:12:19.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:18 smithi098 ceph-mon[103124]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-09-09T23:12:19.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:18 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:18 smithi098 ceph-mon[103124]: osdmap e80: 12 total, 12 up, 12 in 2023-09-09T23:12:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:18 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:18 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:18 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:18 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:18 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:18 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:12:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:18 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:12:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:18 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:12:19.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.014+0000 7f70efd43700 1 -- 172.21.15.82:0/3691805509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70e81000b0 msgr2=0x7f70e8107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:19.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.014+0000 7f70efd43700 1 --2- 172.21.15.82:0/3691805509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70e81000b0 0x7f70e8107950 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f70e000b0a0 tx=0x7f70e000b3b0 comp rx=0 tx=0).stop 2023-09-09T23:12:19.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.014+0000 7f70efd43700 1 -- 172.21.15.82:0/3691805509 shutdown_connections 2023-09-09T23:12:19.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.014+0000 7f70efd43700 1 --2- 172.21.15.82:0/3691805509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70e81000b0 0x7f70e8107950 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:19.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.014+0000 7f70efd43700 1 --2- 172.21.15.82:0/3691805509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70e8100aa0 0x7f70e8100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:19.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.014+0000 7f70efd43700 1 --2- 172.21.15.82:0/3691805509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70e80ff6f0 0x7f70e80ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:19.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.014+0000 7f70efd43700 1 -- 172.21.15.82:0/3691805509 >> 172.21.15.82:0/3691805509 conn(0x7f70e80fa1c0 msgr2=0x7f70e80fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:19.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.014+0000 7f70efd43700 1 -- 172.21.15.82:0/3691805509 shutdown_connections 2023-09-09T23:12:19.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.014+0000 7f70efd43700 1 -- 172.21.15.82:0/3691805509 wait complete. 2023-09-09T23:12:19.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.015+0000 7f70efd43700 1 Processor -- start 2023-09-09T23:12:19.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.015+0000 7f70efd43700 1 -- start start 2023-09-09T23:12:19.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.015+0000 7f70efd43700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70e80ff6f0 0x7f70e81963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:19.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.015+0000 7f70efd43700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70e81000b0 0x7f70e81968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:19.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.015+0000 7f70efd43700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70e8100aa0 0x7f70e819d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:19.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.015+0000 7f70efd43700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e819ded0 con 0x7f70e81000b0 2023-09-09T23:12:19.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.015+0000 7f70efd43700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e819e010 con 0x7f70e8100aa0 2023-09-09T23:12:19.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.015+0000 7f70efd43700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e819e1a0 con 0x7f70e80ff6f0 2023-09-09T23:12:19.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.015+0000 7f70ed2de700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70e81000b0 0x7f70e81968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:19.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.015+0000 7f70ed2de700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70e81000b0 0x7f70e81968e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54446/0 (socket says 172.21.15.82:54446) 2023-09-09T23:12:19.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.015+0000 7f70ed2de700 1 -- 172.21.15.82:0/2334147986 learned_addr learned my addr 172.21.15.82:0/2334147986 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:19.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.015+0000 7f70edadf700 1 --2- 172.21.15.82:0/2334147986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70e80ff6f0 0x7f70e81963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:19.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.015+0000 7f70ee2e0700 1 --2- 172.21.15.82:0/2334147986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70e8100aa0 0x7f70e819d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:19.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.015+0000 7f70ed2de700 1 -- 172.21.15.82:0/2334147986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70e80ff6f0 msgr2=0x7f70e81963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:19.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.015+0000 7f70ed2de700 1 --2- 172.21.15.82:0/2334147986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70e80ff6f0 0x7f70e81963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:19.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.015+0000 7f70ed2de700 1 -- 172.21.15.82:0/2334147986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70e8100aa0 msgr2=0x7f70e819d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:19.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.015+0000 7f70ed2de700 1 --2- 172.21.15.82:0/2334147986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70e8100aa0 0x7f70e819d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:19.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.015+0000 7f70ed2de700 1 -- 172.21.15.82:0/2334147986 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e0009d00 con 0x7f70e81000b0 2023-09-09T23:12:19.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.016+0000 7f70ed2de700 1 --2- 172.21.15.82:0/2334147986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70e81000b0 0x7f70e81968e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f70e400d950 tx=0x7f70e400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:19.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.016+0000 7f70daffd700 1 -- 172.21.15.82:0/2334147986 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e4009940 con 0x7f70e81000b0 2023-09-09T23:12:19.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.016+0000 7f70daffd700 1 -- 172.21.15.82:0/2334147986 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70e4010460 con 0x7f70e81000b0 2023-09-09T23:12:19.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.016+0000 7f70daffd700 1 -- 172.21.15.82:0/2334147986 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e400f620 con 0x7f70e81000b0 2023-09-09T23:12:19.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.016+0000 7f70efd43700 1 -- 172.21.15.82:0/2334147986 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e819e480 con 0x7f70e81000b0 2023-09-09T23:12:19.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.016+0000 7f70efd43700 1 -- 172.21.15.82:0/2334147986 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e819e940 con 0x7f70e81000b0 2023-09-09T23:12:19.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.017+0000 7f70efd43700 1 -- 172.21.15.82:0/2334147986 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e810acf0 con 0x7f70e81000b0 2023-09-09T23:12:19.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.018+0000 7f70daffd700 1 -- 172.21.15.82:0/2334147986 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f70e40108d0 con 0x7f70e81000b0 2023-09-09T23:12:19.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.018+0000 7f70daffd700 1 --2- 172.21.15.82:0/2334147986 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f70d40a0710 0x7f70d40a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:19.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.019+0000 7f70edadf700 1 --2- 172.21.15.82:0/2334147986 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f70d40a0710 0x7f70d40a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:19.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.019+0000 7f70daffd700 1 -- 172.21.15.82:0/2334147986 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f70e40cb960 con 0x7f70e81000b0 2023-09-09T23:12:19.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.019+0000 7f70edadf700 1 --2- 172.21.15.82:0/2334147986 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f70d40a0710 0x7f70d40a2bd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f70dc007be0 tx=0x7f70dc0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:19.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.022+0000 7f70daffd700 1 -- 172.21.15.82:0/2334147986 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f70e4095540 con 0x7f70e81000b0 2023-09-09T23:12:19.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.214+0000 7f70efd43700 1 -- 172.21.15.82:0/2334147986 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70e819eb60 con 0x7f70e81000b0 2023-09-09T23:12:19.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.216+0000 7f70daffd700 1 -- 172.21.15.82:0/2334147986 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22071 (secure 0 0 0) 0x7f70e4016090 con 0x7f70e81000b0 2023-09-09T23:12:19.215 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:12:19.215 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":81,"fsid":"3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da","created":"2023-09-09T23:05:14.222190+0000","modified":"2023-09-09T23:12:18.894339+0000","last_up_change":"2023-09-09T23:11:37.253759+0000","last_in_change":"2023-09-09T23:11:19.933812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-09T23:08:31.234958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-09T23:12:15.524887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-09T23:12:16.785403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1277eb17-a2e7-4e10-9832-f12a5e93ffde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6803","nonce":3197434929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6805","nonce":3197434929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6809","nonce":3197434929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6807","nonce":3197434929}]},"public_addr":"172.21.15.82:6803/3197434929","cluster_addr":"172.21.15.82:6805/3197434929","heartbeat_back_addr":"172.21.15.82:6809/3197434929","heartbeat_front_addr":"172.21.15.82:6807/3197434929","state":["exists","up"]},{"osd":1,"uuid":"f68723fd-0908-4e21-ae8e-1de1c3a9a65c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6801","nonce":3528641314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6803","nonce":3528641314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6807","nonce":3528641314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6805","nonce":3528641314}]},"public_addr":"172.21.15.98:6801/3528641314","cluster_addr":"172.21.15.98:6803/3528641314","heartbeat_back_addr":"172.21.15.98:6807/3528641314","heartbeat_front_addr":"172.21.15.98:6805/3528641314","state":["exists","up"]},{"osd":2,"uuid":"06f89c53-1419-4f52-acad-6e92916b476f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6801","nonce":4006414551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6803","nonce":4006414551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6807","nonce":4006414551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6805","nonce":4006414551}]},"public_addr":"172.21.15.170:6801/4006414551","cluster_addr":"172.21.15.170:6803/4006414551","heartbeat_back_addr":"172.21.15.170:6807/4006414551","heartbeat_front_addr":"172.21.15.170:6805/4006414551","state":["exists","up"]},{"osd":3,"uuid":"fedbd4c9-6852-42b5-8f3a-ba8d9bfe5d59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6811","nonce":3647629241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6813","nonce":3647629241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6817","nonce":3647629241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6815","nonce":3647629241}]},"public_addr":"172.21.15.82:6811/3647629241","cluster_addr":"172.21.15.82:6813/3647629241","heartbeat_back_addr":"172.21.15.82:6817/3647629241","heartbeat_front_addr":"172.21.15.82:6815/3647629241","state":["exists","up"]},{"osd":4,"uuid":"a71f3084-6919-4161-87bf-345f3355d8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6809","nonce":2507035643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6811","nonce":2507035643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6815","nonce":2507035643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6813","nonce":2507035643}]},"public_addr":"172.21.15.98:6809/2507035643","cluster_addr":"172.21.15.98:6811/2507035643","heartbeat_back_addr":"172.21.15.98:6815/2507035643","heartbeat_front_addr":"172.21.15.98:6813/2507035643","state":["exists","up"]},{"osd":5,"uuid":"f76058e9-a90e-476e-87f0-3c6723795e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6809","nonce":775005168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6811","nonce":775005168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6815","nonce":775005168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6813","nonce":775005168}]},"public_addr":"172.21.15.170:6809/775005168","cluster_addr":"172.21.15.170:6811/775005168","heartbeat_back_addr":"172.21.15.170:6815/775005168","heartbeat_front_addr":"172.21.15.170:6813/775005168","state":["exists","up"]},{"osd":6,"uuid":"25940be0-c1bd-4c72-95bf-fd8367aa2f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6819","nonce":3008405982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6821","nonce":3008405982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6825","nonce":3008405982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6823","nonce":3008405982}]},"public_addr":"172.21.15.82:6819/3008405982","cluster_addr":"172.21.15.82:6821/3008405982","heartbeat_back_addr":"172.21.15.82:6825/3008405982","heartbeat_front_addr":"172.21.15.82:6823/3008405982","state":["exists","up"]},{"osd":7,"uuid":"f651bb8c-0b25-42d5-b028-58563f4c27f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6817","nonce":3547615284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6819","nonce":3547615284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6823","nonce":3547615284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6821","nonce":3547615284}]},"public_addr":"172.21.15.98:6817/3547615284","cluster_addr":"172.21.15.98:6819/3547615284","heartbeat_back_addr":"172.21.15.98:6823/3547615284","heartbeat_front_addr":"172.21.15.98:6821/3547615284","state":["exists","up"]},{"osd":8,"uuid":"23b27c86-8c98-441e-b7ab-d57bc584e58f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6817","nonce":632038848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6819","nonce":632038848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6823","nonce":632038848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6821","nonce":632038848}]},"public_addr":"172.21.15.170:6817/632038848","cluster_addr":"172.21.15.170:6819/632038848","heartbeat_back_addr":"172.21.15.170:6823/632038848","heartbeat_front_addr":"172.21.15.170:6821/632038848","state":["exists","up"]},{"osd":9,"uuid":"383b8989-4922-4eee-9c77-f198258ac243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6827","nonce":3354242311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6829","nonce":3354242311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6833","nonce":3354242311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6831","nonce":3354242311}]},"public_addr":"172.21.15.82:6827/3354242311","cluster_addr":"172.21.15.82:6829/3354242311","heartbeat_back_addr":"172.21.15.82:6833/3354242311","heartbeat_front_addr":"172.21.15.82:6831/3354242311","state":["exists","up"]},{"osd":10,"uuid":"33a59389-7ced-42d5-bf54-2071317132d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6825","nonce":2069114959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6827","nonce":2069114959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6831","nonce":2069114959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6829","nonce":2069114959}]},"public_addr":"172.21.15.98:6825/2069114959","cluster_addr":"172.21.15.98:6827/2069114959","heartbeat_back_addr":"172.21.15.98:6831/2069114959","heartbeat_front_addr":"172.21.15.98:6829/2069114959","state":["exists","up"]},{"osd":11,"uuid":"b7a71f5c-85e2-4fec-be4b-95be0788b302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6825","nonce":146964512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6827","nonce":146964512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6831","nonce":146964512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6829","nonce":146964512}]},"public_addr":"172.21.15.170:6825/146964512","cluster_addr":"172.21.15.170:6827/146964512","heartbeat_back_addr":"172.21.15.170:6831/146964512","heartbeat_front_addr":"172.21.15.170:6829/146964512","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:07:47.563043+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:08.359260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:29.070374+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:49.897775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:11.434248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:31.538464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:52.377109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:13.452968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:33.806897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:54.083177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:14.980087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:35.690089+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4013747788":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6800/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/2823341464":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/75116806":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/1037625632":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/3142088817":"2023-09-10T23:05:46.550354+0000","172.21.15.82:6800/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/1679622963":"2023-09-10T23:05:46.550354+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-09T23:12:19.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.218+0000 7f70efd43700 1 -- 172.21.15.82:0/2334147986 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f70d40a0710 msgr2=0x7f70d40a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:19.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.218+0000 7f70efd43700 1 --2- 172.21.15.82:0/2334147986 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f70d40a0710 0x7f70d40a2bd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f70dc007be0 tx=0x7f70dc0073d0 comp rx=0 tx=0).stop 2023-09-09T23:12:19.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.218+0000 7f70efd43700 1 -- 172.21.15.82:0/2334147986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70e81000b0 msgr2=0x7f70e81968e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:19.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.218+0000 7f70efd43700 1 --2- 172.21.15.82:0/2334147986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70e81000b0 0x7f70e81968e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f70e400d950 tx=0x7f70e400dc60 comp rx=0 tx=0).stop 2023-09-09T23:12:19.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.218+0000 7f70efd43700 1 -- 172.21.15.82:0/2334147986 shutdown_connections 2023-09-09T23:12:19.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.218+0000 7f70efd43700 1 --2- 172.21.15.82:0/2334147986 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f70d40a0710 0x7f70d40a2bd0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:19.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.218+0000 7f70efd43700 1 --2- 172.21.15.82:0/2334147986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70e8100aa0 0x7f70e819d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:19.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.218+0000 7f70efd43700 1 --2- 172.21.15.82:0/2334147986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70e80ff6f0 0x7f70e81963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:19.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.218+0000 7f70efd43700 1 --2- 172.21.15.82:0/2334147986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70e81000b0 0x7f70e81968e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:19.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.218+0000 7f70efd43700 1 -- 172.21.15.82:0/2334147986 >> 172.21.15.82:0/2334147986 conn(0x7f70e80fa1c0 msgr2=0x7f70e80fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:19.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.218+0000 7f70efd43700 1 -- 172.21.15.82:0/2334147986 shutdown_connections 2023-09-09T23:12:19.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.218+0000 7f70efd43700 1 -- 172.21.15.82:0/2334147986 wait complete. 2023-09-09T23:12:19.229 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-09T23:12:19.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.356+0000 7fb6cfe63700 1 -- 172.21.15.82:0/2725308413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6c81010b0 msgr2=0x7fb6c8107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:19.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.356+0000 7fb6cfe63700 1 --2- 172.21.15.82:0/2725308413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6c81010b0 0x7fb6c8107960 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000b0d0 tx=0x7fb6c000b3e0 comp rx=0 tx=0).stop 2023-09-09T23:12:19.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.356+0000 7fb6cfe63700 1 -- 172.21.15.82:0/2725308413 shutdown_connections 2023-09-09T23:12:19.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.356+0000 7fb6cfe63700 1 --2- 172.21.15.82:0/2725308413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6c81010b0 0x7fb6c8107960 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:19.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.356+0000 7fb6cfe63700 1 --2- 172.21.15.82:0/2725308413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6c80ff520 0x7fb6c80ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:19.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.356+0000 7fb6cfe63700 1 --2- 172.21.15.82:0/2725308413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6c8100710 0x7fb6c8100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:19.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.356+0000 7fb6cfe63700 1 -- 172.21.15.82:0/2725308413 >> 172.21.15.82:0/2725308413 conn(0x7fb6c80faa90 msgr2=0x7fb6c80fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:19.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.356+0000 7fb6cfe63700 1 -- 172.21.15.82:0/2725308413 shutdown_connections 2023-09-09T23:12:19.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.356+0000 7fb6cfe63700 1 -- 172.21.15.82:0/2725308413 wait complete. 2023-09-09T23:12:19.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.357+0000 7fb6cfe63700 1 Processor -- start 2023-09-09T23:12:19.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.357+0000 7fb6cfe63700 1 -- start start 2023-09-09T23:12:19.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.357+0000 7fb6cfe63700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6c80ff520 0x7fb6c80694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:19.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.357+0000 7fb6cfe63700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6c8100710 0x7fb6c819b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:19.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.357+0000 7fb6cfe63700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6c81010b0 0x7fb6c819d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:19.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.357+0000 7fb6cfe63700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c819dfe0 con 0x7fb6c81010b0 2023-09-09T23:12:19.360 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.357+0000 7fb6cfe63700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c819e150 con 0x7fb6c80ff520 2023-09-09T23:12:19.360 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.357+0000 7fb6cfe63700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c819e310 con 0x7fb6c8100710 2023-09-09T23:12:19.360 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.357+0000 7fb6ce400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6c81010b0 0x7fb6c819d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:19.360 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.357+0000 7fb6ce400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6c81010b0 0x7fb6c819d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54460/0 (socket says 172.21.15.82:54460) 2023-09-09T23:12:19.360 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.357+0000 7fb6ce400700 1 -- 172.21.15.82:0/4240182933 learned_addr learned my addr 172.21.15.82:0/4240182933 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:19.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.357+0000 7fb6cd3fe700 1 --2- 172.21.15.82:0/4240182933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6c8100710 0x7fb6c819b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:19.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.357+0000 7fb6cdbff700 1 --2- 172.21.15.82:0/4240182933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6c80ff520 0x7fb6c80694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:19.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.357+0000 7fb6ce400700 1 -- 172.21.15.82:0/4240182933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6c8100710 msgr2=0x7fb6c819b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:19.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.357+0000 7fb6ce400700 1 --2- 172.21.15.82:0/4240182933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6c8100710 0x7fb6c819b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:19.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.357+0000 7fb6ce400700 1 -- 172.21.15.82:0/4240182933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6c80ff520 msgr2=0x7fb6c80694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:19.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.357+0000 7fb6ce400700 1 --2- 172.21.15.82:0/4240182933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6c80ff520 0x7fb6c80694a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:19.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.357+0000 7fb6ce400700 1 -- 172.21.15.82:0/4240182933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c0009d00 con 0x7fb6c81010b0 2023-09-09T23:12:19.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.357+0000 7fb6ce400700 1 --2- 172.21.15.82:0/4240182933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6c81010b0 0x7fb6c819d810 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0007a00 tx=0x7fb6c0009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:19.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.357+0000 7fb6beffd700 1 -- 172.21.15.82:0/4240182933 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c000e040 con 0x7fb6c81010b0 2023-09-09T23:12:19.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.357+0000 7fb6beffd700 1 -- 172.21.15.82:0/4240182933 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6c0003e10 con 0x7fb6c81010b0 2023-09-09T23:12:19.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.358+0000 7fb6cfe63700 1 -- 172.21.15.82:0/4240182933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c819e590 con 0x7fb6c81010b0 2023-09-09T23:12:19.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.358+0000 7fb6beffd700 1 -- 172.21.15.82:0/4240182933 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0005610 con 0x7fb6c81010b0 2023-09-09T23:12:19.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.358+0000 7fb6cfe63700 1 -- 172.21.15.82:0/4240182933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c819ea10 con 0x7fb6c81010b0 2023-09-09T23:12:19.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.358+0000 7fb6cfe63700 1 -- 172.21.15.82:0/4240182933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6c8106ac0 con 0x7fb6c81010b0 2023-09-09T23:12:19.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.359+0000 7fb6beffd700 1 -- 172.21.15.82:0/4240182933 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fb6c0005770 con 0x7fb6c81010b0 2023-09-09T23:12:19.365 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.360+0000 7fb6beffd700 1 --2- 172.21.15.82:0/4240182933 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb6b40a0740 0x7fb6b40a2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:19.365 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.360+0000 7fb6cdbff700 1 --2- 172.21.15.82:0/4240182933 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb6b40a0740 0x7fb6b40a2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:19.365 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.360+0000 7fb6beffd700 1 -- 172.21.15.82:0/4240182933 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb6c00cd9a0 con 0x7fb6c81010b0 2023-09-09T23:12:19.365 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.360+0000 7fb6cdbff700 1 --2- 172.21.15.82:0/4240182933 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb6b40a0740 0x7fb6b40a2c00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8100570 tx=0x7fb6b800bef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:19.365 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.363+0000 7fb6beffd700 1 -- 172.21.15.82:0/4240182933 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb6c0097610 con 0x7fb6c81010b0 2023-09-09T23:12:19.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:19.602+0000 7fb6cfe63700 1 -- 172.21.15.82:0/4240182933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb6c802d050 con 0x7fb6c81010b0 2023-09-09T23:12:19.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:19 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2378038692' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-09-09T23:12:19.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:19 smithi170 ceph-mon[102832]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-09-09T23:12:19.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:19 smithi170 ceph-mon[102832]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-09-09T23:12:19.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:19 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3021085440' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:12:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:19 smithi170 ceph-mon[102832]: osdmap e81: 12 total, 12 up, 12 in 2023-09-09T23:12:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:19 smithi170 ceph-mon[102832]: pgmap v256: 97 pgs: 11 creating+peering, 13 active+clean, 73 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-09T23:12:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:19 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2334147986' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:12:19.954 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:19 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2378038692' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-09-09T23:12:19.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:19 smithi082 ceph-mon[98694]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-09-09T23:12:19.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:19 smithi082 ceph-mon[98694]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-09-09T23:12:19.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:19 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3021085440' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:12:19.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:19 smithi082 ceph-mon[98694]: osdmap e81: 12 total, 12 up, 12 in 2023-09-09T23:12:19.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:19 smithi082 ceph-mon[98694]: pgmap v256: 97 pgs: 11 creating+peering, 13 active+clean, 73 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-09T23:12:19.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:19 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2334147986' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:12:20.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:19 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2378038692' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-09-09T23:12:20.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:19 smithi098 ceph-mon[103124]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-09-09T23:12:20.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:19 smithi098 ceph-mon[103124]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-09-09T23:12:20.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:19 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3021085440' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:12:20.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:19 smithi098 ceph-mon[103124]: osdmap e81: 12 total, 12 up, 12 in 2023-09-09T23:12:20.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:19 smithi098 ceph-mon[103124]: pgmap v256: 97 pgs: 11 creating+peering, 13 active+clean, 73 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-09-09T23:12:20.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:19 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2334147986' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:12:20.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.550+0000 7fb6beffd700 1 -- 172.21.15.82:0/4240182933 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb6c009ac30 con 0x7fb6c81010b0 2023-09-09T23:12:20.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.553+0000 7fb6cfe63700 1 -- 172.21.15.82:0/4240182933 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb6b40a0740 msgr2=0x7fb6b40a2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:20.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.553+0000 7fb6cfe63700 1 --2- 172.21.15.82:0/4240182933 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb6b40a0740 0x7fb6b40a2c00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8100570 tx=0x7fb6b800bef0 comp rx=0 tx=0).stop 2023-09-09T23:12:20.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.553+0000 7fb6cfe63700 1 -- 172.21.15.82:0/4240182933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6c81010b0 msgr2=0x7fb6c819d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:20.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.553+0000 7fb6cfe63700 1 --2- 172.21.15.82:0/4240182933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6c81010b0 0x7fb6c819d810 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0007a00 tx=0x7fb6c0009570 comp rx=0 tx=0).stop 2023-09-09T23:12:20.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.553+0000 7fb6cfe63700 1 -- 172.21.15.82:0/4240182933 shutdown_connections 2023-09-09T23:12:20.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.553+0000 7fb6cfe63700 1 --2- 172.21.15.82:0/4240182933 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb6b40a0740 0x7fb6b40a2c00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:20.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.553+0000 7fb6cfe63700 1 --2- 172.21.15.82:0/4240182933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6c80ff520 0x7fb6c80694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:20.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.553+0000 7fb6cfe63700 1 --2- 172.21.15.82:0/4240182933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6c81010b0 0x7fb6c819d810 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:20.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.553+0000 7fb6cfe63700 1 --2- 172.21.15.82:0/4240182933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6c8100710 0x7fb6c819b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:20.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.553+0000 7fb6cfe63700 1 -- 172.21.15.82:0/4240182933 >> 172.21.15.82:0/4240182933 conn(0x7fb6c80faa90 msgr2=0x7fb6c8103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:20.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.553+0000 7fb6cfe63700 1 -- 172.21.15.82:0/4240182933 shutdown_connections 2023-09-09T23:12:20.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.553+0000 7fb6cfe63700 1 -- 172.21.15.82:0/4240182933 wait complete. 2023-09-09T23:12:20.566 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2023-09-09T23:12:20.567 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2023-09-09T23:12:20.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.699+0000 7feffe53b700 1 -- 172.21.15.82:0/2190499857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feff8102190 msgr2=0x7feff8108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:20.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.699+0000 7feffe53b700 1 --2- 172.21.15.82:0/2190499857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feff8102190 0x7feff8108a40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7feff000b0d0 tx=0x7feff000b3e0 comp rx=0 tx=0).stop 2023-09-09T23:12:20.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.699+0000 7feffe53b700 1 -- 172.21.15.82:0/2190499857 shutdown_connections 2023-09-09T23:12:20.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.699+0000 7feffe53b700 1 --2- 172.21.15.82:0/2190499857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feff8100600 0x7feff8100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:20.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.699+0000 7feffe53b700 1 --2- 172.21.15.82:0/2190499857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feff8102190 0x7feff8108a40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:20.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.699+0000 7feffe53b700 1 --2- 172.21.15.82:0/2190499857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feff81017f0 0x7feff8101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:20.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.699+0000 7feffe53b700 1 -- 172.21.15.82:0/2190499857 >> 172.21.15.82:0/2190499857 conn(0x7feff80fbb50 msgr2=0x7feff80fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:20.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.699+0000 7feffe53b700 1 -- 172.21.15.82:0/2190499857 shutdown_connections 2023-09-09T23:12:20.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.699+0000 7feffe53b700 1 -- 172.21.15.82:0/2190499857 wait complete. 2023-09-09T23:12:20.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.700+0000 7feffe53b700 1 Processor -- start 2023-09-09T23:12:20.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.700+0000 7feffe53b700 1 -- start start 2023-09-09T23:12:20.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.700+0000 7feffe53b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feff8100600 0x7feff81963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:20.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.700+0000 7feffe53b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feff81017f0 0x7feff8196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:20.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.700+0000 7feffe53b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feff8102190 0x7feff819d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:20.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.700+0000 7feffe53b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff819df40 con 0x7feff81017f0 2023-09-09T23:12:20.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.700+0000 7feffe53b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff819e080 con 0x7feff8100600 2023-09-09T23:12:20.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.700+0000 7feffe53b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff819e210 con 0x7feff8102190 2023-09-09T23:12:20.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.700+0000 7feff77fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feff81017f0 0x7feff8196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:20.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.700+0000 7feff77fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feff81017f0 0x7feff8196900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54472/0 (socket says 172.21.15.82:54472) 2023-09-09T23:12:20.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.700+0000 7feff77fe700 1 -- 172.21.15.82:0/4187256495 learned_addr learned my addr 172.21.15.82:0/4187256495 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:20.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.700+0000 7feff7fff700 1 --2- 172.21.15.82:0/4187256495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feff8100600 0x7feff81963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:20.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.700+0000 7feffcad8700 1 --2- 172.21.15.82:0/4187256495 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feff8102190 0x7feff819d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:20.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.700+0000 7feff77fe700 1 -- 172.21.15.82:0/4187256495 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feff8102190 msgr2=0x7feff819d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:20.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.700+0000 7feff77fe700 1 --2- 172.21.15.82:0/4187256495 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feff8102190 0x7feff819d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:20.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.700+0000 7feff77fe700 1 -- 172.21.15.82:0/4187256495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feff8100600 msgr2=0x7feff81963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:20.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.700+0000 7feff77fe700 1 --2- 172.21.15.82:0/4187256495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feff8100600 0x7feff81963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:20.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.700+0000 7feff77fe700 1 -- 172.21.15.82:0/4187256495 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff0009d00 con 0x7feff81017f0 2023-09-09T23:12:20.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.700+0000 7feffcad8700 1 --2- 172.21.15.82:0/4187256495 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feff8102190 0x7feff819d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:12:20.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.700+0000 7feff77fe700 1 --2- 172.21.15.82:0/4187256495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feff81017f0 0x7feff8196900 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fefe800c920 tx=0x7fefe800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:20.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.701+0000 7feff57fa700 1 -- 172.21.15.82:0/4187256495 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe8007b00 con 0x7feff81017f0 2023-09-09T23:12:20.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.701+0000 7feff57fa700 1 -- 172.21.15.82:0/4187256495 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefe800ce70 con 0x7feff81017f0 2023-09-09T23:12:20.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.701+0000 7feffe53b700 1 -- 172.21.15.82:0/4187256495 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff819e4f0 con 0x7feff81017f0 2023-09-09T23:12:20.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.701+0000 7feff57fa700 1 -- 172.21.15.82:0/4187256495 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe80054e0 con 0x7feff81017f0 2023-09-09T23:12:20.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.701+0000 7feffe53b700 1 -- 172.21.15.82:0/4187256495 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff819ea00 con 0x7feff81017f0 2023-09-09T23:12:20.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.702+0000 7feffe53b700 1 -- 172.21.15.82:0/4187256495 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff8107b50 con 0x7feff81017f0 2023-09-09T23:12:20.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.703+0000 7feff57fa700 1 -- 172.21.15.82:0/4187256495 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fefe8004750 con 0x7feff81017f0 2023-09-09T23:12:20.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.703+0000 7feff57fa700 1 --2- 172.21.15.82:0/4187256495 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fefe00a0640 0x7fefe00a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:20.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.703+0000 7feff7fff700 1 --2- 172.21.15.82:0/4187256495 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fefe00a0640 0x7fefe00a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:20.709 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.703+0000 7feff57fa700 1 -- 172.21.15.82:0/4187256495 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fefe80cb600 con 0x7feff81017f0 2023-09-09T23:12:20.709 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.704+0000 7feff7fff700 1 --2- 172.21.15.82:0/4187256495 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fefe00a0640 0x7fefe00a2b00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7feff8101650 tx=0x7fefec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:20.709 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.707+0000 7feff57fa700 1 -- 172.21.15.82:0/4187256495 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fefe8095280 con 0x7feff81017f0 2023-09-09T23:12:20.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:12:20.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:20 smithi170 ceph-mon[102832]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-09T23:12:20.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:20 smithi170 ceph-mon[102832]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-09T23:12:20.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:20 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/4240182933' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-09T23:12:20.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:20 smithi170 ceph-mon[102832]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-09-09T23:12:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:20 smithi170 ceph-mon[102832]: Cluster is now healthy 2023-09-09T23:12:20.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:20.945+0000 7feffe53b700 1 -- 172.21.15.82:0/4187256495 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7feff810be20 con 0x7feff81017f0 2023-09-09T23:12:20.954 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:12:20.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:20 smithi082 ceph-mon[98694]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-09T23:12:20.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:20 smithi082 ceph-mon[98694]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-09T23:12:20.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:20 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/4240182933' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-09T23:12:20.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:20 smithi082 ceph-mon[98694]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-09-09T23:12:20.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:20 smithi082 ceph-mon[98694]: Cluster is now healthy 2023-09-09T23:12:21.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:12:21.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:20 smithi098 ceph-mon[103124]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-09T23:12:21.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:20 smithi098 ceph-mon[103124]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-09T23:12:21.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:20 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/4240182933' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-09T23:12:21.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:20 smithi098 ceph-mon[103124]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-09-09T23:12:21.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:20 smithi098 ceph-mon[103124]: Cluster is now healthy 2023-09-09T23:12:21.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.555+0000 7feff57fa700 1 -- 172.21.15.82:0/4187256495 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7fefe80988a0 con 0x7feff81017f0 2023-09-09T23:12:21.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.557+0000 7feffe53b700 1 -- 172.21.15.82:0/4187256495 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fefe00a0640 msgr2=0x7fefe00a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:21.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.557+0000 7feffe53b700 1 --2- 172.21.15.82:0/4187256495 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fefe00a0640 0x7fefe00a2b00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7feff8101650 tx=0x7fefec009260 comp rx=0 tx=0).stop 2023-09-09T23:12:21.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.557+0000 7feffe53b700 1 -- 172.21.15.82:0/4187256495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feff81017f0 msgr2=0x7feff8196900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:21.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.557+0000 7feffe53b700 1 --2- 172.21.15.82:0/4187256495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feff81017f0 0x7feff8196900 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fefe800c920 tx=0x7fefe800cc30 comp rx=0 tx=0).stop 2023-09-09T23:12:21.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.557+0000 7feffe53b700 1 -- 172.21.15.82:0/4187256495 shutdown_connections 2023-09-09T23:12:21.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.557+0000 7feffe53b700 1 --2- 172.21.15.82:0/4187256495 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fefe00a0640 0x7fefe00a2b00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:21.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.557+0000 7feffe53b700 1 --2- 172.21.15.82:0/4187256495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feff8100600 0x7feff81963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:21.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.557+0000 7feffe53b700 1 --2- 172.21.15.82:0/4187256495 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feff8102190 0x7feff819d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:21.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.557+0000 7feffe53b700 1 --2- 172.21.15.82:0/4187256495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feff81017f0 0x7feff8196900 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:21.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.557+0000 7feffe53b700 1 -- 172.21.15.82:0/4187256495 >> 172.21.15.82:0/4187256495 conn(0x7feff80fbb50 msgr2=0x7feff8104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:21.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.557+0000 7feffe53b700 1 -- 172.21.15.82:0/4187256495 shutdown_connections 2023-09-09T23:12:21.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.558+0000 7feffe53b700 1 -- 172.21.15.82:0/4187256495 wait complete. 2023-09-09T23:12:21.571 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-09-09T23:12:21.699 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.697+0000 7efd15d4d700 1 -- 172.21.15.82:0/3184188917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd10101070 msgr2=0x7efd10103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:21.699 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.697+0000 7efd15d4d700 1 --2- 172.21.15.82:0/3184188917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd10101070 0x7efd10103460 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efd00009990 tx=0x7efd00009ca0 comp rx=0 tx=0).stop 2023-09-09T23:12:21.699 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.697+0000 7efd15d4d700 1 -- 172.21.15.82:0/3184188917 shutdown_connections 2023-09-09T23:12:21.699 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.697+0000 7efd15d4d700 1 --2- 172.21.15.82:0/3184188917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd101062d0 0x7efd10108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:21.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.697+0000 7efd15d4d700 1 --2- 172.21.15.82:0/3184188917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd10101070 0x7efd10103460 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:21.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.697+0000 7efd15d4d700 1 --2- 172.21.15.82:0/3184188917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd101039a0 0x7efd10105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:21.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.697+0000 7efd15d4d700 1 -- 172.21.15.82:0/3184188917 >> 172.21.15.82:0/3184188917 conn(0x7efd100faa60 msgr2=0x7efd100fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:21.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.697+0000 7efd15d4d700 1 -- 172.21.15.82:0/3184188917 shutdown_connections 2023-09-09T23:12:21.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.697+0000 7efd15d4d700 1 -- 172.21.15.82:0/3184188917 wait complete. 2023-09-09T23:12:21.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.698+0000 7efd15d4d700 1 Processor -- start 2023-09-09T23:12:21.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.698+0000 7efd15d4d700 1 -- start start 2023-09-09T23:12:21.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.698+0000 7efd15d4d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd10101070 0x7efd101009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:21.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.698+0000 7efd15d4d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd101039a0 0x7efd100ff020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:21.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.698+0000 7efd15d4d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd101062d0 0x7efd100ff560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:21.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.698+0000 7efd15d4d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd100ffd00 con 0x7efd10101070 2023-09-09T23:12:21.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.698+0000 7efd15d4d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd100ffe70 con 0x7efd101039a0 2023-09-09T23:12:21.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.698+0000 7efd15d4d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd101a02d0 con 0x7efd101062d0 2023-09-09T23:12:21.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.698+0000 7efd0f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd10101070 0x7efd101009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:21.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.698+0000 7efd0f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd10101070 0x7efd101009e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54482/0 (socket says 172.21.15.82:54482) 2023-09-09T23:12:21.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.698+0000 7efd0f7fe700 1 -- 172.21.15.82:0/2680017443 learned_addr learned my addr 172.21.15.82:0/2680017443 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:21.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.698+0000 7efd0effd700 1 --2- 172.21.15.82:0/2680017443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd101039a0 0x7efd100ff020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:21.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.699+0000 7efd0ffff700 1 --2- 172.21.15.82:0/2680017443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd101062d0 0x7efd100ff560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:21.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.699+0000 7efd0f7fe700 1 -- 172.21.15.82:0/2680017443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd101062d0 msgr2=0x7efd100ff560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:21.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.699+0000 7efd0f7fe700 1 --2- 172.21.15.82:0/2680017443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd101062d0 0x7efd100ff560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:21.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.699+0000 7efd0f7fe700 1 -- 172.21.15.82:0/2680017443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd101039a0 msgr2=0x7efd100ff020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:21.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.699+0000 7efd0f7fe700 1 --2- 172.21.15.82:0/2680017443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd101039a0 0x7efd100ff020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:21.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.699+0000 7efd0f7fe700 1 -- 172.21.15.82:0/2680017443 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd000095f0 con 0x7efd10101070 2023-09-09T23:12:21.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.699+0000 7efd0ffff700 1 --2- 172.21.15.82:0/2680017443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd101062d0 0x7efd100ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:12:21.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.699+0000 7efd0f7fe700 1 --2- 172.21.15.82:0/2680017443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd10101070 0x7efd101009e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efd000098e0 tx=0x7efd0000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:21.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.699+0000 7efd0cff9700 1 -- 172.21.15.82:0/2680017443 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0001d070 con 0x7efd10101070 2023-09-09T23:12:21.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.699+0000 7efd0cff9700 1 -- 172.21.15.82:0/2680017443 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd00022470 con 0x7efd10101070 2023-09-09T23:12:21.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.699+0000 7efd15d4d700 1 -- 172.21.15.82:0/2680017443 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd101a0550 con 0x7efd10101070 2023-09-09T23:12:21.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.699+0000 7efd0cff9700 1 -- 172.21.15.82:0/2680017443 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd00017650 con 0x7efd10101070 2023-09-09T23:12:21.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.699+0000 7efd15d4d700 1 -- 172.21.15.82:0/2680017443 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd101a09d0 con 0x7efd10101070 2023-09-09T23:12:21.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.700+0000 7efd15d4d700 1 -- 172.21.15.82:0/2680017443 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd100fc5e0 con 0x7efd10101070 2023-09-09T23:12:21.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.701+0000 7efd0cff9700 1 -- 172.21.15.82:0/2680017443 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7efd000225e0 con 0x7efd10101070 2023-09-09T23:12:21.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.702+0000 7efd0cff9700 1 --2- 172.21.15.82:0/2680017443 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7efcf80a06c0 0x7efcf80a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:21.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.702+0000 7efd0effd700 1 --2- 172.21.15.82:0/2680017443 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7efcf80a06c0 0x7efcf80a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:21.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.702+0000 7efd0cff9700 1 -- 172.21.15.82:0/2680017443 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7efd000ccd90 con 0x7efd10101070 2023-09-09T23:12:21.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.702+0000 7efd0effd700 1 --2- 172.21.15.82:0/2680017443 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7efcf80a06c0 0x7efcf80a2b80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efd0400ac50 tx=0x7efd0400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:21.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.706+0000 7efd0cff9700 1 -- 172.21.15.82:0/2680017443 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7efd00096a00 con 0x7efd10101070 2023-09-09T23:12:21.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:21 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/4240182933' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-09-09T23:12:21.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:21 smithi170 ceph-mon[102832]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-09T23:12:21.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:21 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/4187256495' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-09-09T23:12:21.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:21 smithi170 ceph-mon[102832]: pgmap v257: 97 pgs: 11 creating+peering, 58 active+clean, 28 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 424 B/s wr, 2 op/s 2023-09-09T23:12:21.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:21.939+0000 7efd15d4d700 1 -- 172.21.15.82:0/2680017443 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7efd1010be10 con 0x7efd10101070 2023-09-09T23:12:21.954 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:21 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/4240182933' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-09-09T23:12:21.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:21 smithi082 ceph-mon[98694]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-09T23:12:21.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:21 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/4187256495' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-09-09T23:12:21.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:21 smithi082 ceph-mon[98694]: pgmap v257: 97 pgs: 11 creating+peering, 58 active+clean, 28 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 424 B/s wr, 2 op/s 2023-09-09T23:12:22.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:21 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/4240182933' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-09-09T23:12:22.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:21 smithi098 ceph-mon[103124]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-09T23:12:22.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:21 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/4187256495' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-09-09T23:12:22.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:21 smithi098 ceph-mon[103124]: pgmap v257: 97 pgs: 11 creating+peering, 58 active+clean, 28 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 424 B/s wr, 2 op/s 2023-09-09T23:12:22.555 INFO:journalctl@ceph.mds.b.smithi098.stdout:Sep 09 23:12:22 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mds-b[139333]: 2023-09-09T23:12:22.272+0000 7fee2a4a6700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-09T23:12:22.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.564+0000 7efd0cff9700 1 -- 172.21.15.82:0/2680017443 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7efd0009a020 con 0x7efd10101070 2023-09-09T23:12:22.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.567+0000 7efd15d4d700 1 -- 172.21.15.82:0/2680017443 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7efcf80a06c0 msgr2=0x7efcf80a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:22.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.567+0000 7efd15d4d700 1 --2- 172.21.15.82:0/2680017443 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7efcf80a06c0 0x7efcf80a2b80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efd0400ac50 tx=0x7efd0400a380 comp rx=0 tx=0).stop 2023-09-09T23:12:22.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.567+0000 7efd15d4d700 1 -- 172.21.15.82:0/2680017443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd10101070 msgr2=0x7efd101009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:22.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.567+0000 7efd15d4d700 1 --2- 172.21.15.82:0/2680017443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd10101070 0x7efd101009e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efd000098e0 tx=0x7efd0000f690 comp rx=0 tx=0).stop 2023-09-09T23:12:22.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.567+0000 7efd15d4d700 1 -- 172.21.15.82:0/2680017443 shutdown_connections 2023-09-09T23:12:22.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.567+0000 7efd15d4d700 1 --2- 172.21.15.82:0/2680017443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd101062d0 0x7efd100ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:22.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.567+0000 7efd15d4d700 1 --2- 172.21.15.82:0/2680017443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd10101070 0x7efd101009e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:22.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.567+0000 7efd15d4d700 1 --2- 172.21.15.82:0/2680017443 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7efcf80a06c0 0x7efcf80a2b80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:22.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.567+0000 7efd15d4d700 1 --2- 172.21.15.82:0/2680017443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd101039a0 0x7efd100ff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:22.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.567+0000 7efd15d4d700 1 -- 172.21.15.82:0/2680017443 >> 172.21.15.82:0/2680017443 conn(0x7efd100faa60 msgr2=0x7efd10101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:22.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.567+0000 7efd15d4d700 1 -- 172.21.15.82:0/2680017443 shutdown_connections 2023-09-09T23:12:22.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.567+0000 7efd15d4d700 1 -- 172.21.15.82:0/2680017443 wait complete. 2023-09-09T23:12:22.582 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-09-09T23:12:22.709 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.709+0000 7effa311c700 1 -- 172.21.15.82:0/2131560832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff9c0ff520 msgr2=0x7eff9c0ff940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:22.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.709+0000 7effa311c700 1 --2- 172.21.15.82:0/2131560832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff9c0ff520 0x7eff9c0ff940 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7eff900099c0 tx=0x7eff90009cd0 comp rx=0 tx=0).stop 2023-09-09T23:12:22.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.709+0000 7effa311c700 1 -- 172.21.15.82:0/2131560832 shutdown_connections 2023-09-09T23:12:22.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.709+0000 7effa311c700 1 --2- 172.21.15.82:0/2131560832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff9c1010b0 0x7eff9c107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:22.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.709+0000 7effa311c700 1 --2- 172.21.15.82:0/2131560832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff9c0ff520 0x7eff9c0ff940 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:22.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.709+0000 7effa311c700 1 --2- 172.21.15.82:0/2131560832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff9c100710 0x7eff9c100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:22.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.709+0000 7effa311c700 1 -- 172.21.15.82:0/2131560832 >> 172.21.15.82:0/2131560832 conn(0x7eff9c0faa90 msgr2=0x7eff9c0fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:22.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.709+0000 7effa311c700 1 -- 172.21.15.82:0/2131560832 shutdown_connections 2023-09-09T23:12:22.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.709+0000 7effa311c700 1 -- 172.21.15.82:0/2131560832 wait complete. 2023-09-09T23:12:22.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.709+0000 7effa311c700 1 Processor -- start 2023-09-09T23:12:22.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.709+0000 7effa311c700 1 -- start start 2023-09-09T23:12:22.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.709+0000 7effa311c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff9c0ff520 0x7eff9c196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:22.713 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.709+0000 7effa311c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff9c100710 0x7eff9c196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:22.713 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.709+0000 7effa311c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff9c1010b0 0x7eff9c19d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:22.713 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.709+0000 7effa311c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c19de80 con 0x7eff9c0ff520 2023-09-09T23:12:22.713 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.709+0000 7effa311c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c19dfc0 con 0x7eff9c100710 2023-09-09T23:12:22.714 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.709+0000 7effa311c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c19e150 con 0x7eff9c1010b0 2023-09-09T23:12:22.714 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.709+0000 7effa0eb8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff9c0ff520 0x7eff9c196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:22.714 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.709+0000 7effa0eb8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff9c0ff520 0x7eff9c196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54496/0 (socket says 172.21.15.82:54496) 2023-09-09T23:12:22.714 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.709+0000 7effa0eb8700 1 -- 172.21.15.82:0/3571691758 learned_addr learned my addr 172.21.15.82:0/3571691758 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:22.715 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.710+0000 7effa16b9700 1 --2- 172.21.15.82:0/3571691758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff9c1010b0 0x7eff9c19d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:22.715 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.710+0000 7eff9bfff700 1 --2- 172.21.15.82:0/3571691758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff9c100710 0x7eff9c196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:22.715 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.710+0000 7effa0eb8700 1 -- 172.21.15.82:0/3571691758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff9c1010b0 msgr2=0x7eff9c19d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:22.715 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.710+0000 7effa0eb8700 1 --2- 172.21.15.82:0/3571691758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff9c1010b0 0x7eff9c19d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:22.715 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.710+0000 7effa0eb8700 1 -- 172.21.15.82:0/3571691758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff9c100710 msgr2=0x7eff9c196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:22.715 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.710+0000 7effa0eb8700 1 --2- 172.21.15.82:0/3571691758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff9c100710 0x7eff9c196840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:22.716 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.710+0000 7effa0eb8700 1 -- 172.21.15.82:0/3571691758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff900095f0 con 0x7eff9c0ff520 2023-09-09T23:12:22.716 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.710+0000 7effa0eb8700 1 --2- 172.21.15.82:0/3571691758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff9c0ff520 0x7eff9c196300 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7eff9000b580 tx=0x7eff90005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:22.716 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.710+0000 7eff99ffb700 1 -- 172.21.15.82:0/3571691758 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9001d070 con 0x7eff9c0ff520 2023-09-09T23:12:22.716 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.710+0000 7eff99ffb700 1 -- 172.21.15.82:0/3571691758 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff90004120 con 0x7eff9c0ff520 2023-09-09T23:12:22.717 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.710+0000 7eff99ffb700 1 -- 172.21.15.82:0/3571691758 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9000f460 con 0x7eff9c0ff520 2023-09-09T23:12:22.717 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.710+0000 7effa311c700 1 -- 172.21.15.82:0/3571691758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff9c19e3d0 con 0x7eff9c0ff520 2023-09-09T23:12:22.717 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.710+0000 7effa311c700 1 -- 172.21.15.82:0/3571691758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff9c19e7a0 con 0x7eff9c0ff520 2023-09-09T23:12:22.717 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.711+0000 7effa311c700 1 -- 172.21.15.82:0/3571691758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff9c10ad30 con 0x7eff9c0ff520 2023-09-09T23:12:22.719 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.712+0000 7eff99ffb700 1 -- 172.21.15.82:0/3571691758 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7eff90022600 con 0x7eff9c0ff520 2023-09-09T23:12:22.719 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.713+0000 7eff99ffb700 1 --2- 172.21.15.82:0/3571691758 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7eff8c0a06c0 0x7eff8c0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:22.719 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.713+0000 7eff9bfff700 1 --2- 172.21.15.82:0/3571691758 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7eff8c0a06c0 0x7eff8c0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:22.719 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.713+0000 7eff99ffb700 1 -- 172.21.15.82:0/3571691758 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7eff900cd780 con 0x7eff9c0ff520 2023-09-09T23:12:22.720 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.713+0000 7eff9bfff700 1 --2- 172.21.15.82:0/3571691758 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7eff8c0a06c0 0x7eff8c0a2b80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7eff88005950 tx=0x7eff880058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:22.720 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.716+0000 7eff99ffb700 1 -- 172.21.15.82:0/3571691758 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7eff900973f0 con 0x7eff9c0ff520 2023-09-09T23:12:22.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:22 smithi170 ceph-mon[102832]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-09-09T23:12:22.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:22 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/4187256495' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-09-09T23:12:22.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:22 smithi170 ceph-mon[102832]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-09-09T23:12:22.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:22 smithi170 ceph-mon[102832]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-09T23:12:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:22 smithi170 ceph-mon[102832]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2023-09-09T23:12:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:22 smithi170 ceph-mon[102832]: daemon mds.h is now active in filesystem cephfs as rank 1 2023-09-09T23:12:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:22 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2680017443' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-09T23:12:22.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:22.956+0000 7effa311c700 1 -- 172.21.15.82:0/3571691758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7eff9c19e9c0 con 0x7eff9c0ff520 2023-09-09T23:12:22.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:22 smithi082 ceph-mon[98694]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-09-09T23:12:22.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:22 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/4187256495' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-09-09T23:12:22.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:22 smithi082 ceph-mon[98694]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-09-09T23:12:22.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:22 smithi082 ceph-mon[98694]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-09T23:12:22.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:22 smithi082 ceph-mon[98694]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2023-09-09T23:12:22.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:22 smithi082 ceph-mon[98694]: daemon mds.h is now active in filesystem cephfs as rank 1 2023-09-09T23:12:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:22 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2680017443' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-09T23:12:23.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:22 smithi098 ceph-mon[103124]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-09-09T23:12:23.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:22 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/4187256495' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-09-09T23:12:23.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:22 smithi098 ceph-mon[103124]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-09-09T23:12:23.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:22 smithi098 ceph-mon[103124]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-09T23:12:23.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:22 smithi098 ceph-mon[103124]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2023-09-09T23:12:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:22 smithi098 ceph-mon[103124]: daemon mds.h is now active in filesystem cephfs as rank 1 2023-09-09T23:12:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:22 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2680017443' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-09T23:12:23.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.253+0000 7eff99ffb700 1 -- 172.21.15.82:0/3571691758 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7eff9009aa10 con 0x7eff9c0ff520 2023-09-09T23:12:23.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.255+0000 7effa311c700 1 -- 172.21.15.82:0/3571691758 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7eff8c0a06c0 msgr2=0x7eff8c0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:23.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.255+0000 7effa311c700 1 --2- 172.21.15.82:0/3571691758 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7eff8c0a06c0 0x7eff8c0a2b80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7eff88005950 tx=0x7eff880058e0 comp rx=0 tx=0).stop 2023-09-09T23:12:23.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.255+0000 7effa311c700 1 -- 172.21.15.82:0/3571691758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff9c0ff520 msgr2=0x7eff9c196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:23.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.255+0000 7effa311c700 1 --2- 172.21.15.82:0/3571691758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff9c0ff520 0x7eff9c196300 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7eff9000b580 tx=0x7eff90005ef0 comp rx=0 tx=0).stop 2023-09-09T23:12:23.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.255+0000 7effa311c700 1 -- 172.21.15.82:0/3571691758 shutdown_connections 2023-09-09T23:12:23.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.255+0000 7effa311c700 1 --2- 172.21.15.82:0/3571691758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff9c1010b0 0x7eff9c19d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:23.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.255+0000 7effa311c700 1 --2- 172.21.15.82:0/3571691758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff9c0ff520 0x7eff9c196300 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:23.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.255+0000 7effa311c700 1 --2- 172.21.15.82:0/3571691758 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7eff8c0a06c0 0x7eff8c0a2b80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:23.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.255+0000 7effa311c700 1 --2- 172.21.15.82:0/3571691758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff9c100710 0x7eff9c196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:23.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.255+0000 7effa311c700 1 -- 172.21.15.82:0/3571691758 >> 172.21.15.82:0/3571691758 conn(0x7eff9c0faa90 msgr2=0x7eff9c068950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:23.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.255+0000 7effa311c700 1 -- 172.21.15.82:0/3571691758 shutdown_connections 2023-09-09T23:12:23.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.255+0000 7effa311c700 1 -- 172.21.15.82:0/3571691758 wait complete. 2023-09-09T23:12:23.270 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2023-09-09T23:12:23.271 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-09-09T23:12:23.397 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.397+0000 7f69f2ced700 1 -- 172.21.15.82:0/2998524507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69ec0ff7a0 msgr2=0x7f69ec0ffc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:23.398 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.397+0000 7f69f2ced700 1 --2- 172.21.15.82:0/2998524507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69ec0ff7a0 0x7f69ec0ffc00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009990 tx=0x7f69d8009ca0 comp rx=0 tx=0).stop 2023-09-09T23:12:23.398 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.397+0000 7f69f2ced700 1 -- 172.21.15.82:0/2998524507 shutdown_connections 2023-09-09T23:12:23.398 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.397+0000 7f69f2ced700 1 --2- 172.21.15.82:0/2998524507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69ec100140 0x7f69ec107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:23.398 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.397+0000 7f69f2ced700 1 --2- 172.21.15.82:0/2998524507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69ec100a70 0x7f69ec100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:23.398 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.397+0000 7f69f2ced700 1 --2- 172.21.15.82:0/2998524507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69ec0ff7a0 0x7f69ec0ffc00 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:23.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.397+0000 7f69f2ced700 1 -- 172.21.15.82:0/2998524507 >> 172.21.15.82:0/2998524507 conn(0x7f69ec0fa1d0 msgr2=0x7f69ec0fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:23.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.397+0000 7f69f2ced700 1 -- 172.21.15.82:0/2998524507 shutdown_connections 2023-09-09T23:12:23.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.397+0000 7f69f2ced700 1 -- 172.21.15.82:0/2998524507 wait complete. 2023-09-09T23:12:23.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.397+0000 7f69f2ced700 1 Processor -- start 2023-09-09T23:12:23.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.397+0000 7f69f2ced700 1 -- start start 2023-09-09T23:12:23.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.397+0000 7f69f2ced700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69ec0ff7a0 0x7f69ec0ff200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:23.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.397+0000 7f69f2ced700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69ec100140 0x7f69ec19b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:23.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.397+0000 7f69f2ced700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69ec100a70 0x7f69ec19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:23.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.397+0000 7f69f2ced700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec19de30 con 0x7f69ec100140 2023-09-09T23:12:23.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.397+0000 7f69f2ced700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec19dfa0 con 0x7f69ec0ff7a0 2023-09-09T23:12:23.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.397+0000 7f69f2ced700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec19e160 con 0x7f69ec100a70 2023-09-09T23:12:23.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.397+0000 7f69ebfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69ec100140 0x7f69ec19b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:23.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.398+0000 7f69ebfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69ec100140 0x7f69ec19b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54512/0 (socket says 172.21.15.82:54512) 2023-09-09T23:12:23.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.398+0000 7f69ebfff700 1 -- 172.21.15.82:0/962792822 learned_addr learned my addr 172.21.15.82:0/962792822 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:23.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.398+0000 7f69f0a89700 1 --2- 172.21.15.82:0/962792822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69ec0ff7a0 0x7f69ec0ff200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:23.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.398+0000 7f69f128a700 1 --2- 172.21.15.82:0/962792822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69ec100a70 0x7f69ec19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:23.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.398+0000 7f69ebfff700 1 -- 172.21.15.82:0/962792822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69ec100a70 msgr2=0x7f69ec19d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:23.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.398+0000 7f69ebfff700 1 --2- 172.21.15.82:0/962792822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69ec100a70 0x7f69ec19d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:23.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.398+0000 7f69ebfff700 1 -- 172.21.15.82:0/962792822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69ec0ff7a0 msgr2=0x7f69ec0ff200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:23.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.398+0000 7f69ebfff700 1 --2- 172.21.15.82:0/962792822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69ec0ff7a0 0x7f69ec0ff200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:23.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.398+0000 7f69ebfff700 1 -- 172.21.15.82:0/962792822 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d80095f0 con 0x7f69ec100140 2023-09-09T23:12:23.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.398+0000 7f69f128a700 1 --2- 172.21.15.82:0/962792822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69ec100a70 0x7f69ec19d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:12:23.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.398+0000 7f69ebfff700 1 --2- 172.21.15.82:0/962792822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69ec100140 0x7f69ec19b050 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f69d8000c00 tx=0x7f69d800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:23.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.398+0000 7f69e9ffb700 1 -- 172.21.15.82:0/962792822 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d801d070 con 0x7f69ec100140 2023-09-09T23:12:23.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.398+0000 7f69f2ced700 1 -- 172.21.15.82:0/962792822 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69ec19e3e0 con 0x7f69ec100140 2023-09-09T23:12:23.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.398+0000 7f69e9ffb700 1 -- 172.21.15.82:0/962792822 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69d800fb50 con 0x7f69ec100140 2023-09-09T23:12:23.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.398+0000 7f69e9ffb700 1 -- 172.21.15.82:0/962792822 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d8017650 con 0x7f69ec100140 2023-09-09T23:12:23.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.398+0000 7f69f2ced700 1 -- 172.21.15.82:0/962792822 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69ec19e8e0 con 0x7f69ec100140 2023-09-09T23:12:23.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.399+0000 7f69f2ced700 1 -- 172.21.15.82:0/962792822 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69ec10ace0 con 0x7f69ec100140 2023-09-09T23:12:23.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.400+0000 7f69e9ffb700 1 -- 172.21.15.82:0/962792822 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f69d800fcc0 con 0x7f69ec100140 2023-09-09T23:12:23.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.401+0000 7f69e9ffb700 1 --2- 172.21.15.82:0/962792822 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f69dc0a0640 0x7f69dc0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:23.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.401+0000 7f69f0a89700 1 --2- 172.21.15.82:0/962792822 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f69dc0a0640 0x7f69dc0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:23.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.401+0000 7f69e9ffb700 1 -- 172.21.15.82:0/962792822 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f69d80cd120 con 0x7f69ec100140 2023-09-09T23:12:23.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.401+0000 7f69f0a89700 1 --2- 172.21.15.82:0/962792822 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f69dc0a0640 0x7f69dc0a2b00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f69ec0674e0 tx=0x7f69e000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:23.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.404+0000 7f69e9ffb700 1 -- 172.21.15.82:0/962792822 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f69d8096e30 con 0x7f69ec100140 2023-09-09T23:12:23.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:23.628+0000 7f69f2ced700 1 -- 172.21.15.82:0/962792822 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f69ec060a40 con 0x7f69dc0a0640 2023-09-09T23:12:23.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:23 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2680017443' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-09-09T23:12:23.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:12:23.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:23 smithi170 ceph-mon[102832]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-09-09T23:12:23.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:23 smithi170 ceph-mon[102832]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2023-09-09T23:12:23.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:23 smithi170 ceph-mon[102832]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2023-09-09T23:12:23.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:23 smithi170 ceph-mon[102832]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-09-09T23:12:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:23 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3571691758' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-09T23:12:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:23 smithi170 ceph-mon[102832]: pgmap v258: 97 pgs: 97 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2023-09-09T23:12:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:23 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3571691758' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-09-09T23:12:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] up:active 2023-09-09T23:12:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:23 smithi170 ceph-mon[102832]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-09-09T23:12:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:23 smithi170 ceph-mon[102832]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-09T23:12:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:23 smithi170 ceph-mon[102832]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2023-09-09T23:12:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:23 smithi170 ceph-mon[102832]: daemon mds.k is now active in filesystem cephfs as rank 3 2023-09-09T23:12:23.955 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:12:23 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:12:23.632+0000 7f413fb46700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-09T23:12:23.955 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:12:23 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:12:23.632+0000 7f413fb46700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-09T23:12:23.955 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:12:23 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:12:23.632+0000 7f413fb46700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-09T23:12:23.955 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:12:23 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:12:23.633+0000 7f413fb46700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-09T23:12:23.956 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:12:23 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:12:23.633+0000 7f413fb46700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-09T23:12:23.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:23 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2680017443' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-09-09T23:12:23.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:12:23.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:23 smithi082 ceph-mon[98694]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-09-09T23:12:23.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:23 smithi082 ceph-mon[98694]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2023-09-09T23:12:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:23 smithi082 ceph-mon[98694]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2023-09-09T23:12:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:23 smithi082 ceph-mon[98694]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-09-09T23:12:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:23 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3571691758' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-09T23:12:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:23 smithi082 ceph-mon[98694]: pgmap v258: 97 pgs: 97 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2023-09-09T23:12:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:23 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3571691758' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-09-09T23:12:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] up:active 2023-09-09T23:12:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:23 smithi082 ceph-mon[98694]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-09-09T23:12:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:23 smithi082 ceph-mon[98694]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-09T23:12:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:23 smithi082 ceph-mon[98694]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2023-09-09T23:12:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:23 smithi082 ceph-mon[98694]: daemon mds.k is now active in filesystem cephfs as rank 3 2023-09-09T23:12:24.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:23 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2680017443' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-09-09T23:12:24.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:12:24.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:23 smithi098 ceph-mon[103124]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-09-09T23:12:24.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:23 smithi098 ceph-mon[103124]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2023-09-09T23:12:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:23 smithi098 ceph-mon[103124]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2023-09-09T23:12:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:23 smithi098 ceph-mon[103124]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-09-09T23:12:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:23 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3571691758' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-09T23:12:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:23 smithi098 ceph-mon[103124]: pgmap v258: 97 pgs: 97 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2023-09-09T23:12:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:23 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3571691758' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-09-09T23:12:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] up:active 2023-09-09T23:12:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:23 smithi098 ceph-mon[103124]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-09-09T23:12:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:23 smithi098 ceph-mon[103124]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-09-09T23:12:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:23 smithi098 ceph-mon[103124]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2023-09-09T23:12:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:23 smithi098 ceph-mon[103124]: daemon mds.k is now active in filesystem cephfs as rank 3 2023-09-09T23:12:24.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.299+0000 7f69e9ffb700 1 -- 172.21.15.82:0/962792822 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f69ec060a40 con 0x7f69dc0a0640 2023-09-09T23:12:24.301 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.302+0000 7f69f2ced700 1 -- 172.21.15.82:0/962792822 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f69dc0a0640 msgr2=0x7f69dc0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:24.301 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.302+0000 7f69f2ced700 1 --2- 172.21.15.82:0/962792822 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f69dc0a0640 0x7f69dc0a2b00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f69ec0674e0 tx=0x7f69e000b410 comp rx=0 tx=0).stop 2023-09-09T23:12:24.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.302+0000 7f69f2ced700 1 -- 172.21.15.82:0/962792822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69ec100140 msgr2=0x7f69ec19b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:24.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.302+0000 7f69f2ced700 1 --2- 172.21.15.82:0/962792822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69ec100140 0x7f69ec19b050 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f69d8000c00 tx=0x7f69d800f710 comp rx=0 tx=0).stop 2023-09-09T23:12:24.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.302+0000 7f69f2ced700 1 -- 172.21.15.82:0/962792822 shutdown_connections 2023-09-09T23:12:24.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.302+0000 7f69f2ced700 1 --2- 172.21.15.82:0/962792822 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f69dc0a0640 0x7f69dc0a2b00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:24.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.302+0000 7f69f2ced700 1 --2- 172.21.15.82:0/962792822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69ec0ff7a0 0x7f69ec0ff200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:24.303 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.302+0000 7f69f2ced700 1 --2- 172.21.15.82:0/962792822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69ec100a70 0x7f69ec19d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:24.303 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.302+0000 7f69f2ced700 1 --2- 172.21.15.82:0/962792822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69ec100140 0x7f69ec19b050 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:24.303 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.302+0000 7f69f2ced700 1 -- 172.21.15.82:0/962792822 >> 172.21.15.82:0/962792822 conn(0x7f69ec0fa1d0 msgr2=0x7f69ec0fc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:24.303 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.302+0000 7f69f2ced700 1 -- 172.21.15.82:0/962792822 shutdown_connections 2023-09-09T23:12:24.303 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.302+0000 7f69f2ced700 1 -- 172.21.15.82:0/962792822 wait complete. 2023-09-09T23:12:24.313 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-09-09T23:12:24.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.439+0000 7fe8f816b700 1 -- 172.21.15.82:0/3422208854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8f0103a10 msgr2=0x7fe8f0105e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:24.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.439+0000 7fe8f816b700 1 --2- 172.21.15.82:0/3422208854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8f0103a10 0x7fe8f0105e00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009990 tx=0x7fe8ec009ca0 comp rx=0 tx=0).stop 2023-09-09T23:12:24.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.439+0000 7fe8f816b700 1 -- 172.21.15.82:0/3422208854 shutdown_connections 2023-09-09T23:12:24.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.439+0000 7fe8f816b700 1 --2- 172.21.15.82:0/3422208854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8f01010e0 0x7fe8f01034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:24.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.439+0000 7fe8f816b700 1 --2- 172.21.15.82:0/3422208854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f0106340 0x7fe8f01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:24.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.439+0000 7fe8f816b700 1 --2- 172.21.15.82:0/3422208854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8f0103a10 0x7fe8f0105e00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:24.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.439+0000 7fe8f816b700 1 -- 172.21.15.82:0/3422208854 >> 172.21.15.82:0/3422208854 conn(0x7fe8f00faad0 msgr2=0x7fe8f00fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:24.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.439+0000 7fe8f816b700 1 -- 172.21.15.82:0/3422208854 shutdown_connections 2023-09-09T23:12:24.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.439+0000 7fe8f816b700 1 -- 172.21.15.82:0/3422208854 wait complete. 2023-09-09T23:12:24.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.440+0000 7fe8f816b700 1 Processor -- start 2023-09-09T23:12:24.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.440+0000 7fe8f816b700 1 -- start start 2023-09-09T23:12:24.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.440+0000 7fe8f816b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f01010e0 0x7fe8f01962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:24.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.440+0000 7fe8f816b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8f0103a10 0x7fe8f01967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:24.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.440+0000 7fe8f816b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8f0106340 0x7fe8f019d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:24.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.440+0000 7fe8f816b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f019de20 con 0x7fe8f0106340 2023-09-09T23:12:24.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.440+0000 7fe8f816b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f019df60 con 0x7fe8f0103a10 2023-09-09T23:12:24.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.440+0000 7fe8f816b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f019e0f0 con 0x7fe8f01010e0 2023-09-09T23:12:24.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.440+0000 7fe8f6708700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8f0106340 0x7fe8f019d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:24.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.440+0000 7fe8f6708700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8f0106340 0x7fe8f019d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54548/0 (socket says 172.21.15.82:54548) 2023-09-09T23:12:24.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.440+0000 7fe8f6708700 1 -- 172.21.15.82:0/454694217 learned_addr learned my addr 172.21.15.82:0/454694217 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:24.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.440+0000 7fe8f5f07700 1 --2- 172.21.15.82:0/454694217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f01010e0 0x7fe8f01962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:24.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.440+0000 7fe8f5706700 1 --2- 172.21.15.82:0/454694217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8f0103a10 0x7fe8f01967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:24.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.440+0000 7fe8f6708700 1 -- 172.21.15.82:0/454694217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f01010e0 msgr2=0x7fe8f01962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:24.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.440+0000 7fe8f6708700 1 --2- 172.21.15.82:0/454694217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f01010e0 0x7fe8f01962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:24.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.440+0000 7fe8f6708700 1 -- 172.21.15.82:0/454694217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8f0103a10 msgr2=0x7fe8f01967e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:24.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.440+0000 7fe8f6708700 1 --2- 172.21.15.82:0/454694217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8f0103a10 0x7fe8f01967e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:24.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.440+0000 7fe8f6708700 1 -- 172.21.15.82:0/454694217 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ec0095f0 con 0x7fe8f0106340 2023-09-09T23:12:24.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.440+0000 7fe8f5706700 1 --2- 172.21.15.82:0/454694217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8f0103a10 0x7fe8f01967e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:12:24.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.441+0000 7fe8f6708700 1 --2- 172.21.15.82:0/454694217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8f0106340 0x7fe8f019d850 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8007ee0 tx=0x7fe8e800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:24.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.441+0000 7fe8e6ffd700 1 -- 172.21.15.82:0/454694217 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8017070 con 0x7fe8f0106340 2023-09-09T23:12:24.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.441+0000 7fe8e6ffd700 1 -- 172.21.15.82:0/454694217 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8e800f040 con 0x7fe8f0106340 2023-09-09T23:12:24.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.441+0000 7fe8f816b700 1 -- 172.21.15.82:0/454694217 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f019e3d0 con 0x7fe8f0106340 2023-09-09T23:12:24.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.441+0000 7fe8e6ffd700 1 -- 172.21.15.82:0/454694217 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8005740 con 0x7fe8f0106340 2023-09-09T23:12:24.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.441+0000 7fe8f816b700 1 -- 172.21.15.82:0/454694217 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f019e8e0 con 0x7fe8f0106340 2023-09-09T23:12:24.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.442+0000 7fe8f816b700 1 -- 172.21.15.82:0/454694217 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8f010be10 con 0x7fe8f0106340 2023-09-09T23:12:24.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.443+0000 7fe8e6ffd700 1 -- 172.21.15.82:0/454694217 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fe8e80058a0 con 0x7fe8f0106340 2023-09-09T23:12:24.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.444+0000 7fe8e6ffd700 1 --2- 172.21.15.82:0/454694217 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fe8dc0a06c0 0x7fe8dc0a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:24.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.444+0000 7fe8f5f07700 1 --2- 172.21.15.82:0/454694217 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fe8dc0a06c0 0x7fe8dc0a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:24.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.444+0000 7fe8e6ffd700 1 -- 172.21.15.82:0/454694217 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe8e80cb8c0 con 0x7fe8f0106340 2023-09-09T23:12:24.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.444+0000 7fe8f5f07700 1 --2- 172.21.15.82:0/454694217 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fe8dc0a06c0 0x7fe8dc0a2b80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0005950 tx=0x7fe8e00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:24.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.447+0000 7fe8e6ffd700 1 -- 172.21.15.82:0/454694217 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe8e8095540 con 0x7fe8f0106340 2023-09-09T23:12:24.672 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.674+0000 7fe8f816b700 1 -- 172.21.15.82:0/454694217 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8f0102ca0 con 0x7fe8dc0a06c0 2023-09-09T23:12:24.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.678+0000 7fe8e6ffd700 1 -- 172.21.15.82:0/454694217 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fe8f0102ca0 con 0x7fe8dc0a06c0 2023-09-09T23:12:24.676 INFO:teuthology.orchestra.run.smithi082.stdout:/volumes/_nogroup/sv_0/d6b74fb9-77ae-40fc-8c3b-9050778d1e07 2023-09-09T23:12:24.678 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.680+0000 7fe8f816b700 1 -- 172.21.15.82:0/454694217 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fe8dc0a06c0 msgr2=0x7fe8dc0a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:24.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.680+0000 7fe8f816b700 1 --2- 172.21.15.82:0/454694217 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fe8dc0a06c0 0x7fe8dc0a2b80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0005950 tx=0x7fe8e00058e0 comp rx=0 tx=0).stop 2023-09-09T23:12:24.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.680+0000 7fe8f816b700 1 -- 172.21.15.82:0/454694217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8f0106340 msgr2=0x7fe8f019d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:24.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.680+0000 7fe8f816b700 1 --2- 172.21.15.82:0/454694217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8f0106340 0x7fe8f019d850 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8007ee0 tx=0x7fe8e800d3b0 comp rx=0 tx=0).stop 2023-09-09T23:12:24.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.680+0000 7fe8f816b700 1 -- 172.21.15.82:0/454694217 shutdown_connections 2023-09-09T23:12:24.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.680+0000 7fe8f816b700 1 --2- 172.21.15.82:0/454694217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8f0106340 0x7fe8f019d850 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:24.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.680+0000 7fe8f816b700 1 --2- 172.21.15.82:0/454694217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f01010e0 0x7fe8f01962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:24.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.680+0000 7fe8f816b700 1 --2- 172.21.15.82:0/454694217 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fe8dc0a06c0 0x7fe8dc0a2b80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:24.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.680+0000 7fe8f816b700 1 --2- 172.21.15.82:0/454694217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8f0103a10 0x7fe8f01967e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:24.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.680+0000 7fe8f816b700 1 -- 172.21.15.82:0/454694217 >> 172.21.15.82:0/454694217 conn(0x7fe8f00faad0 msgr2=0x7fe8f0102590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:24.681 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.680+0000 7fe8f816b700 1 -- 172.21.15.82:0/454694217 shutdown_connections 2023-09-09T23:12:24.681 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.680+0000 7fe8f816b700 1 -- 172.21.15.82:0/454694217 wait complete. 2023-09-09T23:12:24.691 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-09-09T23:12:24.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.817+0000 7ff1d85a4700 1 -- 172.21.15.82:0/407227096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1d0100140 msgr2=0x7ff1d01079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:24.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.817+0000 7ff1d85a4700 1 --2- 172.21.15.82:0/407227096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1d0100140 0x7ff1d01079e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800b0a0 tx=0x7ff1c800b3b0 comp rx=0 tx=0).stop 2023-09-09T23:12:24.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.818+0000 7ff1d85a4700 1 -- 172.21.15.82:0/407227096 shutdown_connections 2023-09-09T23:12:24.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.818+0000 7ff1d85a4700 1 --2- 172.21.15.82:0/407227096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1d0100ae0 0x7ff1d0100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:24.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.818+0000 7ff1d85a4700 1 --2- 172.21.15.82:0/407227096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1d0100140 0x7ff1d01079e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:24.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.818+0000 7ff1d85a4700 1 --2- 172.21.15.82:0/407227096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1d00ff780 0x7ff1d00ffc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:24.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.818+0000 7ff1d85a4700 1 -- 172.21.15.82:0/407227096 >> 172.21.15.82:0/407227096 conn(0x7ff1d00fa1d0 msgr2=0x7ff1d00fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:24.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.818+0000 7ff1d85a4700 1 -- 172.21.15.82:0/407227096 shutdown_connections 2023-09-09T23:12:24.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.818+0000 7ff1d85a4700 1 -- 172.21.15.82:0/407227096 wait complete. 2023-09-09T23:12:24.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.818+0000 7ff1d85a4700 1 Processor -- start 2023-09-09T23:12:24.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.818+0000 7ff1d85a4700 1 -- start start 2023-09-09T23:12:24.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.818+0000 7ff1d85a4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1d00ff780 0x7ff1d00ff200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:24.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.818+0000 7ff1d85a4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1d0100140 0x7ff1d019b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:24.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.818+0000 7ff1d85a4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1d0100ae0 0x7ff1d019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:24.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.818+0000 7ff1d85a4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d019de90 con 0x7ff1d0100140 2023-09-09T23:12:24.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.818+0000 7ff1d85a4700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d019e000 con 0x7ff1d00ff780 2023-09-09T23:12:24.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.818+0000 7ff1d85a4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d019e1c0 con 0x7ff1d0100ae0 2023-09-09T23:12:24.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.818+0000 7ff1d5b3f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1d0100140 0x7ff1d019b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:24.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.819+0000 7ff1d5b3f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1d0100140 0x7ff1d019b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54560/0 (socket says 172.21.15.82:54560) 2023-09-09T23:12:24.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.819+0000 7ff1d5b3f700 1 -- 172.21.15.82:0/647683467 learned_addr learned my addr 172.21.15.82:0/647683467 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:24.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.819+0000 7ff1d6340700 1 --2- 172.21.15.82:0/647683467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1d00ff780 0x7ff1d00ff200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:24.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.819+0000 7ff1d6b41700 1 --2- 172.21.15.82:0/647683467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1d0100ae0 0x7ff1d019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:24.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.819+0000 7ff1d5b3f700 1 -- 172.21.15.82:0/647683467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1d0100ae0 msgr2=0x7ff1d019d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:24.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.819+0000 7ff1d5b3f700 1 --2- 172.21.15.82:0/647683467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1d0100ae0 0x7ff1d019d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:24.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.819+0000 7ff1d5b3f700 1 -- 172.21.15.82:0/647683467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1d00ff780 msgr2=0x7ff1d00ff200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:24.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.819+0000 7ff1d5b3f700 1 --2- 172.21.15.82:0/647683467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1d00ff780 0x7ff1d00ff200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:24.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.819+0000 7ff1d5b3f700 1 -- 172.21.15.82:0/647683467 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c8009d00 con 0x7ff1d0100140 2023-09-09T23:12:24.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.819+0000 7ff1d6340700 1 --2- 172.21.15.82:0/647683467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1d00ff780 0x7ff1d00ff200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:12:24.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.819+0000 7ff1d6b41700 1 --2- 172.21.15.82:0/647683467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1d0100ae0 0x7ff1d019d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:12:24.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.819+0000 7ff1d5b3f700 1 --2- 172.21.15.82:0/647683467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1d0100140 0x7ff1d019b070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000c990 tx=0x7ff1c000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:24.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.819+0000 7ff1c77fe700 1 -- 172.21.15.82:0/647683467 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0007b00 con 0x7ff1d0100140 2023-09-09T23:12:24.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.819+0000 7ff1c77fe700 1 -- 172.21.15.82:0/647683467 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1c0007c60 con 0x7ff1d0100140 2023-09-09T23:12:24.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.819+0000 7ff1d85a4700 1 -- 172.21.15.82:0/647683467 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d019e4a0 con 0x7ff1d0100140 2023-09-09T23:12:24.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.819+0000 7ff1c77fe700 1 -- 172.21.15.82:0/647683467 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0018660 con 0x7ff1d0100140 2023-09-09T23:12:24.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.819+0000 7ff1d85a4700 1 -- 172.21.15.82:0/647683467 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d019e9b0 con 0x7ff1d0100140 2023-09-09T23:12:24.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.820+0000 7ff1d85a4700 1 -- 172.21.15.82:0/647683467 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1d0106420 con 0x7ff1d0100140 2023-09-09T23:12:24.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.821+0000 7ff1c77fe700 1 -- 172.21.15.82:0/647683467 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7ff1c0004020 con 0x7ff1d0100140 2023-09-09T23:12:24.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.822+0000 7ff1c77fe700 1 --2- 172.21.15.82:0/647683467 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff1bc0a0640 0x7ff1bc0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:24.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.822+0000 7ff1d6340700 1 --2- 172.21.15.82:0/647683467 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff1bc0a0640 0x7ff1bc0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:24.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.822+0000 7ff1c77fe700 1 -- 172.21.15.82:0/647683467 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff1c00cbb50 con 0x7ff1d0100140 2023-09-09T23:12:24.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.822+0000 7ff1d6340700 1 --2- 172.21.15.82:0/647683467 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff1bc0a0640 0x7ff1bc0a2b00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0067500 tx=0x7ff1cc00b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:24.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:24.825+0000 7ff1c77fe700 1 -- 172.21.15.82:0/647683467 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff1c0095860 con 0x7ff1d0100140 2023-09-09T23:12:25.045 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.046+0000 7ff1d85a4700 1 -- 172.21.15.82:0/647683467 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1d002cfc0 con 0x7ff1bc0a0640 2023-09-09T23:12:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.071+0000 7ff1c77fe700 1 -- 172.21.15.82:0/647683467 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff1d002cfc0 con 0x7ff1bc0a0640 2023-09-09T23:12:25.072 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.073+0000 7ff1d85a4700 1 -- 172.21.15.82:0/647683467 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff1bc0a0640 msgr2=0x7ff1bc0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:25.073 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.073+0000 7ff1d85a4700 1 --2- 172.21.15.82:0/647683467 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff1bc0a0640 0x7ff1bc0a2b00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0067500 tx=0x7ff1cc00b5f0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.073 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.073+0000 7ff1d85a4700 1 -- 172.21.15.82:0/647683467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1d0100140 msgr2=0x7ff1d019b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:25.073 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.073+0000 7ff1d85a4700 1 --2- 172.21.15.82:0/647683467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1d0100140 0x7ff1d019b070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000c990 tx=0x7ff1c000cd50 comp rx=0 tx=0).stop 2023-09-09T23:12:25.073 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.073+0000 7ff1d85a4700 1 -- 172.21.15.82:0/647683467 shutdown_connections 2023-09-09T23:12:25.074 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.073+0000 7ff1d85a4700 1 --2- 172.21.15.82:0/647683467 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff1bc0a0640 0x7ff1bc0a2b00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.074 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.073+0000 7ff1d85a4700 1 --2- 172.21.15.82:0/647683467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1d00ff780 0x7ff1d00ff200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.074 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.073+0000 7ff1d85a4700 1 --2- 172.21.15.82:0/647683467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1d0100ae0 0x7ff1d019d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.074 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.073+0000 7ff1d85a4700 1 --2- 172.21.15.82:0/647683467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1d0100140 0x7ff1d019b070 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.073+0000 7ff1d85a4700 1 -- 172.21.15.82:0/647683467 >> 172.21.15.82:0/647683467 conn(0x7ff1d00fa1d0 msgr2=0x7ff1d00fc400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:25.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.073+0000 7ff1d85a4700 1 -- 172.21.15.82:0/647683467 shutdown_connections 2023-09-09T23:12:25.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.073+0000 7ff1d85a4700 1 -- 172.21.15.82:0/647683467 wait complete. 2023-09-09T23:12:25.084 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-09-09T23:12:25.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.211+0000 7f1aff915700 1 -- 172.21.15.82:0/3686896960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af80ff520 msgr2=0x7f1af80ff940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:25.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.211+0000 7f1aff915700 1 --2- 172.21.15.82:0/3686896960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af80ff520 0x7f1af80ff940 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1ae80099c0 tx=0x7f1ae8009cd0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.211+0000 7f1aff915700 1 -- 172.21.15.82:0/3686896960 shutdown_connections 2023-09-09T23:12:25.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.211+0000 7f1aff915700 1 --2- 172.21.15.82:0/3686896960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1af81010b0 0x7f1af8107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.211+0000 7f1aff915700 1 --2- 172.21.15.82:0/3686896960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af80ff520 0x7f1af80ff940 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.211+0000 7f1aff915700 1 --2- 172.21.15.82:0/3686896960 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1af8100710 0x7f1af8100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.211+0000 7f1aff915700 1 -- 172.21.15.82:0/3686896960 >> 172.21.15.82:0/3686896960 conn(0x7f1af80faa90 msgr2=0x7f1af80fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:25.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.211+0000 7f1aff915700 1 -- 172.21.15.82:0/3686896960 shutdown_connections 2023-09-09T23:12:25.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.211+0000 7f1aff915700 1 -- 172.21.15.82:0/3686896960 wait complete. 2023-09-09T23:12:25.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.212+0000 7f1aff915700 1 Processor -- start 2023-09-09T23:12:25.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.212+0000 7f1aff915700 1 -- start start 2023-09-09T23:12:25.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.212+0000 7f1aff915700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1af80ff520 0x7f1af8195190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:25.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.212+0000 7f1aff915700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1af8100710 0x7f1af81956d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:25.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.212+0000 7f1aff915700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af81010b0 0x7f1af819c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:25.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.212+0000 7f1aff915700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af819cd10 con 0x7f1af81010b0 2023-09-09T23:12:25.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.212+0000 7f1aff915700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af819ce50 con 0x7f1af8100710 2023-09-09T23:12:25.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.212+0000 7f1aff915700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af819cfe0 con 0x7f1af80ff520 2023-09-09T23:12:25.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.212+0000 7f1afdeb2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af81010b0 0x7f1af819c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:25.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.212+0000 7f1afdeb2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af81010b0 0x7f1af819c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54578/0 (socket says 172.21.15.82:54578) 2023-09-09T23:12:25.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.212+0000 7f1afd6b1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1af80ff520 0x7f1af8195190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:25.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.212+0000 7f1afdeb2700 1 -- 172.21.15.82:0/1679791830 learned_addr learned my addr 172.21.15.82:0/1679791830 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:25.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.212+0000 7f1afceb0700 1 --2- 172.21.15.82:0/1679791830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1af8100710 0x7f1af81956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:25.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.212+0000 7f1afdeb2700 1 -- 172.21.15.82:0/1679791830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1af80ff520 msgr2=0x7f1af8195190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:25.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.213+0000 7f1afdeb2700 1 --2- 172.21.15.82:0/1679791830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1af80ff520 0x7f1af8195190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.213+0000 7f1afdeb2700 1 -- 172.21.15.82:0/1679791830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1af8100710 msgr2=0x7f1af81956d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:25.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.213+0000 7f1afdeb2700 1 --2- 172.21.15.82:0/1679791830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1af8100710 0x7f1af81956d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.213+0000 7f1afdeb2700 1 -- 172.21.15.82:0/1679791830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae80095f0 con 0x7f1af81010b0 2023-09-09T23:12:25.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.213+0000 7f1afdeb2700 1 --2- 172.21.15.82:0/1679791830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af81010b0 0x7f1af819c740 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1af0007970 tx=0x7f1af0007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:25.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.213+0000 7f1aee7fc700 1 -- 172.21.15.82:0/1679791830 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af0017070 con 0x7f1af81010b0 2023-09-09T23:12:25.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.213+0000 7f1aee7fc700 1 -- 172.21.15.82:0/1679791830 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1af000f040 con 0x7f1af81010b0 2023-09-09T23:12:25.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.213+0000 7f1aff915700 1 -- 172.21.15.82:0/1679791830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af819d2c0 con 0x7f1af81010b0 2023-09-09T23:12:25.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.213+0000 7f1aee7fc700 1 -- 172.21.15.82:0/1679791830 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af0013a20 con 0x7f1af81010b0 2023-09-09T23:12:25.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.213+0000 7f1aff915700 1 -- 172.21.15.82:0/1679791830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af819d7d0 con 0x7f1af81010b0 2023-09-09T23:12:25.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.214+0000 7f1aff915700 1 -- 172.21.15.82:0/1679791830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af8106a60 con 0x7f1af81010b0 2023-09-09T23:12:25.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.215+0000 7f1aee7fc700 1 -- 172.21.15.82:0/1679791830 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f1af000dd30 con 0x7f1af81010b0 2023-09-09T23:12:25.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.215+0000 7f1aee7fc700 1 --2- 172.21.15.82:0/1679791830 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1ae40a0640 0x7f1ae40a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:25.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.215+0000 7f1afd6b1700 1 --2- 172.21.15.82:0/1679791830 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1ae40a0640 0x7f1ae40a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:25.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.215+0000 7f1aee7fc700 1 -- 172.21.15.82:0/1679791830 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1af00cb120 con 0x7f1af81010b0 2023-09-09T23:12:25.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.216+0000 7f1afd6b1700 1 --2- 172.21.15.82:0/1679791830 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1ae40a0640 0x7f1ae40a2b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009f50 tx=0x7f1ae800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:25.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.219+0000 7f1aee7fc700 1 -- 172.21.15.82:0/1679791830 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1af0094da0 con 0x7f1af81010b0 2023-09-09T23:12:25.286 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.287+0000 7f1aee7fc700 1 -- 172.21.15.82:0/1679791830 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f1af0095170 con 0x7f1af81010b0 2023-09-09T23:12:25.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:25 smithi082 ceph-mon[98694]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:12:25.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:25 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] up:active 2023-09-09T23:12:25.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:25 smithi082 ceph-mon[98694]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-09-09T23:12:25.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:25 smithi082 ceph-mon[98694]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-09-09T23:12:25.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:25 smithi082 ceph-mon[98694]: Cluster is now healthy 2023-09-09T23:12:25.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:25 smithi082 ceph-mon[98694]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2023-09-09T23:12:25.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:25 smithi082 ceph-mon[98694]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2023-09-09T23:12:25.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:25 smithi082 ceph-mon[98694]: daemon mds.l is now active in filesystem cephfs as rank 4 2023-09-09T23:12:25.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.456+0000 7f1aff915700 1 -- 172.21.15.82:0/1679791830 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1af802d030 con 0x7f1ae40a0640 2023-09-09T23:12:25.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.460+0000 7f1aee7fc700 1 -- 172.21.15.82:0/1679791830 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f1af802d030 con 0x7f1ae40a0640 2023-09-09T23:12:25.459 INFO:teuthology.orchestra.run.smithi082.stdout:/volumes/_nogroup/sv_1/0a302eaa-eeda-4440-b80e-72c42f5e6ec4 2023-09-09T23:12:25.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.462+0000 7f1aff915700 1 -- 172.21.15.82:0/1679791830 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1ae40a0640 msgr2=0x7f1ae40a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:25.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.462+0000 7f1aff915700 1 --2- 172.21.15.82:0/1679791830 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1ae40a0640 0x7f1ae40a2b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009f50 tx=0x7f1ae800b540 comp rx=0 tx=0).stop 2023-09-09T23:12:25.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.462+0000 7f1aff915700 1 -- 172.21.15.82:0/1679791830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af81010b0 msgr2=0x7f1af819c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:25.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.462+0000 7f1aff915700 1 --2- 172.21.15.82:0/1679791830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af81010b0 0x7f1af819c740 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1af0007970 tx=0x7f1af0007c80 comp rx=0 tx=0).stop 2023-09-09T23:12:25.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.462+0000 7f1aff915700 1 -- 172.21.15.82:0/1679791830 shutdown_connections 2023-09-09T23:12:25.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.462+0000 7f1aff915700 1 --2- 172.21.15.82:0/1679791830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af81010b0 0x7f1af819c740 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.462+0000 7f1aff915700 1 --2- 172.21.15.82:0/1679791830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1af80ff520 0x7f1af8195190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.462+0000 7f1aff915700 1 --2- 172.21.15.82:0/1679791830 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1ae40a0640 0x7f1ae40a2b00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.462+0000 7f1aff915700 1 --2- 172.21.15.82:0/1679791830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1af8100710 0x7f1af81956d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.462+0000 7f1aff915700 1 -- 172.21.15.82:0/1679791830 >> 172.21.15.82:0/1679791830 conn(0x7f1af80faa90 msgr2=0x7f1af8103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:25.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.462+0000 7f1aff915700 1 -- 172.21.15.82:0/1679791830 shutdown_connections 2023-09-09T23:12:25.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.462+0000 7f1aff915700 1 -- 172.21.15.82:0/1679791830 wait complete. 2023-09-09T23:12:25.473 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-09T23:12:25.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.596+0000 7f425bc5d700 1 -- 172.21.15.82:0/1611747837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4254101810 msgr2=0x7f4254101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:25.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.596+0000 7f425bc5d700 1 --2- 172.21.15.82:0/1611747837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4254101810 0x7f4254101c70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f42500099c0 tx=0x7f4250009cd0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.596+0000 7f425bc5d700 1 -- 172.21.15.82:0/1611747837 shutdown_connections 2023-09-09T23:12:25.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.596+0000 7f425bc5d700 1 --2- 172.21.15.82:0/1611747837 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42541021b0 0x7f4254108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.596+0000 7f425bc5d700 1 --2- 172.21.15.82:0/1611747837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4254100620 0x7f4254100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.596+0000 7f425bc5d700 1 --2- 172.21.15.82:0/1611747837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4254101810 0x7f4254101c70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.596+0000 7f425bc5d700 1 -- 172.21.15.82:0/1611747837 >> 172.21.15.82:0/1611747837 conn(0x7f42540fbb70 msgr2=0x7f42540fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:25.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.596+0000 7f425bc5d700 1 -- 172.21.15.82:0/1611747837 shutdown_connections 2023-09-09T23:12:25.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.596+0000 7f425bc5d700 1 -- 172.21.15.82:0/1611747837 wait complete. 2023-09-09T23:12:25.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.596+0000 7f425bc5d700 1 Processor -- start 2023-09-09T23:12:25.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.596+0000 7f425bc5d700 1 -- start start 2023-09-09T23:12:25.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.596+0000 7f425bc5d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4254100620 0x7f42540743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:25.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.596+0000 7f425bc5d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4254101810 0x7f42540729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:25.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.596+0000 7f425bc5d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42541021b0 0x7f4254072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:25.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.597+0000 7f425bc5d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42540736d0 con 0x7f42541021b0 2023-09-09T23:12:25.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.597+0000 7f425bc5d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4254073840 con 0x7f4254100620 2023-09-09T23:12:25.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.597+0000 7f425bc5d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42541a0280 con 0x7f4254101810 2023-09-09T23:12:25.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.597+0000 7f425a1fa700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42541021b0 0x7f4254072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:25.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.597+0000 7f425a1fa700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42541021b0 0x7f4254072f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54614/0 (socket says 172.21.15.82:54614) 2023-09-09T23:12:25.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.597+0000 7f425a1fa700 1 -- 172.21.15.82:0/2972752265 learned_addr learned my addr 172.21.15.82:0/2972752265 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:25.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.597+0000 7f42591f8700 1 --2- 172.21.15.82:0/2972752265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4254101810 0x7f42540729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:25.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.597+0000 7f425a1fa700 1 -- 172.21.15.82:0/2972752265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4254101810 msgr2=0x7f42540729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:25.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.597+0000 7f425a1fa700 1 --2- 172.21.15.82:0/2972752265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4254101810 0x7f42540729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.597+0000 7f42599f9700 1 --2- 172.21.15.82:0/2972752265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4254100620 0x7f42540743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:25.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.597+0000 7f425a1fa700 1 -- 172.21.15.82:0/2972752265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4254100620 msgr2=0x7f42540743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:25.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.597+0000 7f425a1fa700 1 --2- 172.21.15.82:0/2972752265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4254100620 0x7f42540743b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.597+0000 7f425a1fa700 1 -- 172.21.15.82:0/2972752265 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42500095f0 con 0x7f42541021b0 2023-09-09T23:12:25.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.597+0000 7f42591f8700 1 --2- 172.21.15.82:0/2972752265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4254101810 0x7f42540729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:12:25.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.597+0000 7f42599f9700 1 --2- 172.21.15.82:0/2972752265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4254100620 0x7f42540743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:12:25.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.597+0000 7f425a1fa700 1 --2- 172.21.15.82:0/2972752265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42541021b0 0x7f4254072f30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f424c007d70 tx=0x7f424c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:25.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.597+0000 7f424affd700 1 -- 172.21.15.82:0/2972752265 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c00ad40 con 0x7f42541021b0 2023-09-09T23:12:25.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.597+0000 7f424affd700 1 -- 172.21.15.82:0/2972752265 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f424c0128e0 con 0x7f42541021b0 2023-09-09T23:12:25.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.597+0000 7f425bc5d700 1 -- 172.21.15.82:0/2972752265 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42541a0560 con 0x7f42541021b0 2023-09-09T23:12:25.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.597+0000 7f424affd700 1 -- 172.21.15.82:0/2972752265 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c011780 con 0x7f42541021b0 2023-09-09T23:12:25.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.597+0000 7f425bc5d700 1 -- 172.21.15.82:0/2972752265 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42541a0a20 con 0x7f42541021b0 2023-09-09T23:12:25.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.598+0000 7f425bc5d700 1 -- 172.21.15.82:0/2972752265 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4254107b10 con 0x7f42541021b0 2023-09-09T23:12:25.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.599+0000 7f424affd700 1 -- 172.21.15.82:0/2972752265 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f424c012a50 con 0x7f42541021b0 2023-09-09T23:12:25.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.600+0000 7f424affd700 1 --2- 172.21.15.82:0/2972752265 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f42400a0710 0x7f42400a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:25.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.600+0000 7f42599f9700 1 --2- 172.21.15.82:0/2972752265 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f42400a0710 0x7f42400a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:25.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.600+0000 7f424affd700 1 -- 172.21.15.82:0/2972752265 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f424c0cbe10 con 0x7f42541021b0 2023-09-09T23:12:25.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.600+0000 7f42599f9700 1 --2- 172.21.15.82:0/2972752265 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f42400a0710 0x7f42400a2bd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4244005fd0 tx=0x7f4244005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:25.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.604+0000 7f424affd700 1 -- 172.21.15.82:0/2972752265 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f424c095aa0 con 0x7f42541021b0 2023-09-09T23:12:25.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:25 smithi170 ceph-mon[102832]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:12:25.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:25 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] up:active 2023-09-09T23:12:25.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:25 smithi170 ceph-mon[102832]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-09-09T23:12:25.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:25 smithi170 ceph-mon[102832]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-09-09T23:12:25.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:25 smithi170 ceph-mon[102832]: Cluster is now healthy 2023-09-09T23:12:25.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:25 smithi170 ceph-mon[102832]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2023-09-09T23:12:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:25 smithi170 ceph-mon[102832]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2023-09-09T23:12:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:25 smithi170 ceph-mon[102832]: daemon mds.l is now active in filesystem cephfs as rank 4 2023-09-09T23:12:25.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:25 smithi098 ceph-mon[103124]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:12:25.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:25 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] up:active 2023-09-09T23:12:25.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:25 smithi098 ceph-mon[103124]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-09-09T23:12:25.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:25 smithi098 ceph-mon[103124]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-09-09T23:12:25.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:25 smithi098 ceph-mon[103124]: Cluster is now healthy 2023-09-09T23:12:25.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:25 smithi098 ceph-mon[103124]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2023-09-09T23:12:25.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:25 smithi098 ceph-mon[103124]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2023-09-09T23:12:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:25 smithi098 ceph-mon[103124]: daemon mds.l is now active in filesystem cephfs as rank 4 2023-09-09T23:12:25.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.839+0000 7f425bc5d700 1 -- 172.21.15.82:0/2972752265 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f425410c060 con 0x7f42541021b0 2023-09-09T23:12:25.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.839+0000 7f424affd700 1 -- 172.21.15.82:0/2972752265 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11368 (secure 0 0 0) 0x7f424c0990c0 con 0x7f42541021b0 2023-09-09T23:12:25.838 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:12:25.838 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2471432684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2471432684},{"type":"v1","addr":"172.21.15.82:6835","nonce":2471432684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3579387137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3579387137},{"type":"v1","addr":"172.21.15.82:6837","nonce":3579387137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-09T23:12:17.586036+0000","modified":"2023-09-09T23:12:25.285958+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24542,"mds_2":24523,"mds_3":24491,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14712":{"gid":14712,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/2127301345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2127301345},{"type":"v1","addr":"172.21.15.82:6839","nonce":2127301345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14724":{"gid":14724,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/4143604488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":4143604488},{"type":"v1","addr":"172.21.15.82:6841","nonce":4143604488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.170:6833/1931961940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1931961940},{"type":"v1","addr":"172.21.15.170:6833","nonce":1931961940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24491":{"gid":24491,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.98:6833/3855146755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3855146755},{"type":"v1","addr":"172.21.15.98:6833","nonce":3855146755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2313861511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2313861511},{"type":"v1","addr":"172.21.15.170:6835","nonce":2313861511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/4006360308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":4006360308},{"type":"v1","addr":"172.21.15.170:6837","nonce":4006360308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1356474828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1356474828},{"type":"v1","addr":"172.21.15.98:6835","nonce":1356474828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/3858369735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3858369735},{"type":"v1","addr":"172.21.15.170:6839","nonce":3858369735}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/2019449840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2019449840},{"type":"v1","addr":"172.21.15.98:6837","nonce":2019449840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2785086371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2785086371},{"type":"v1","addr":"172.21.15.98:6839","nonce":2785086371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-09T23:12:25.840 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.841+0000 7f425bc5d700 1 -- 172.21.15.82:0/2972752265 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f42400a0710 msgr2=0x7f42400a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:25.840 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.841+0000 7f425bc5d700 1 --2- 172.21.15.82:0/2972752265 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f42400a0710 0x7f42400a2bd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4244005fd0 tx=0x7f4244005f00 comp rx=0 tx=0).stop 2023-09-09T23:12:25.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.841+0000 7f425bc5d700 1 -- 172.21.15.82:0/2972752265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42541021b0 msgr2=0x7f4254072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:25.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.841+0000 7f425bc5d700 1 --2- 172.21.15.82:0/2972752265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42541021b0 0x7f4254072f30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f424c007d70 tx=0x7f424c00a340 comp rx=0 tx=0).stop 2023-09-09T23:12:25.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.841+0000 7f425bc5d700 1 -- 172.21.15.82:0/2972752265 shutdown_connections 2023-09-09T23:12:25.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.841+0000 7f425bc5d700 1 --2- 172.21.15.82:0/2972752265 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f42400a0710 0x7f42400a2bd0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.841+0000 7f425bc5d700 1 --2- 172.21.15.82:0/2972752265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4254100620 0x7f42540743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.841+0000 7f425bc5d700 1 --2- 172.21.15.82:0/2972752265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42541021b0 0x7f4254072f30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.841+0000 7f425bc5d700 1 --2- 172.21.15.82:0/2972752265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4254101810 0x7f42540729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.841+0000 7f425bc5d700 1 -- 172.21.15.82:0/2972752265 >> 172.21.15.82:0/2972752265 conn(0x7f42540fbb70 msgr2=0x7f4254104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:25.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.842+0000 7f425bc5d700 1 -- 172.21.15.82:0/2972752265 shutdown_connections 2023-09-09T23:12:25.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.842+0000 7f425bc5d700 1 -- 172.21.15.82:0/2972752265 wait complete. 2023-09-09T23:12:25.843 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 27 2023-09-09T23:12:25.854 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-09T23:12:25.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.978+0000 7fc4fe6f8700 1 -- 172.21.15.82:0/2275317194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4f8101800 msgr2=0x7fc4f8101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:25.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.978+0000 7fc4fe6f8700 1 --2- 172.21.15.82:0/2275317194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4f8101800 0x7fc4f8101c60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc4e00099c0 tx=0x7fc4e0009cd0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.980 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.978+0000 7fc4fe6f8700 1 -- 172.21.15.82:0/2275317194 shutdown_connections 2023-09-09T23:12:25.980 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.978+0000 7fc4fe6f8700 1 --2- 172.21.15.82:0/2275317194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4f8100610 0x7fc4f8100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.980 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.978+0000 7fc4fe6f8700 1 --2- 172.21.15.82:0/2275317194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4f81021a0 0x7fc4f8108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.980 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.978+0000 7fc4fe6f8700 1 --2- 172.21.15.82:0/2275317194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4f8101800 0x7fc4f8101c60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.978+0000 7fc4fe6f8700 1 -- 172.21.15.82:0/2275317194 >> 172.21.15.82:0/2275317194 conn(0x7fc4f80fbb40 msgr2=0x7fc4f80fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:25.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.978+0000 7fc4fe6f8700 1 -- 172.21.15.82:0/2275317194 shutdown_connections 2023-09-09T23:12:25.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.978+0000 7fc4fe6f8700 1 -- 172.21.15.82:0/2275317194 wait complete. 2023-09-09T23:12:25.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.979+0000 7fc4fe6f8700 1 Processor -- start 2023-09-09T23:12:25.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.979+0000 7fc4fe6f8700 1 -- start start 2023-09-09T23:12:25.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.979+0000 7fc4fe6f8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4f8100610 0x7fc4f8196350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:25.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.979+0000 7fc4fe6f8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4f8101800 0x7fc4f8196890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:25.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.979+0000 7fc4fe6f8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4f81021a0 0x7fc4f819d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:25.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.979+0000 7fc4fe6f8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f819ded0 con 0x7fc4f8100610 2023-09-09T23:12:25.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.979+0000 7fc4fe6f8700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f819e010 con 0x7fc4f8101800 2023-09-09T23:12:25.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.979+0000 7fc4fe6f8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f819e1a0 con 0x7fc4f81021a0 2023-09-09T23:12:25.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.979+0000 7fc4f7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4f8100610 0x7fc4f8196350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:25.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.979+0000 7fc4f7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4f8100610 0x7fc4f8196350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54624/0 (socket says 172.21.15.82:54624) 2023-09-09T23:12:25.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.979+0000 7fc4f7fff700 1 -- 172.21.15.82:0/764753975 learned_addr learned my addr 172.21.15.82:0/764753975 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:25.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.979+0000 7fc4effff700 1 --2- 172.21.15.82:0/764753975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4f8101800 0x7fc4f8196890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:25.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.979+0000 7fc4f7fff700 1 -- 172.21.15.82:0/764753975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4f81021a0 msgr2=0x7fc4f819d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:25.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.979+0000 7fc4fcc95700 1 --2- 172.21.15.82:0/764753975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4f81021a0 0x7fc4f819d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:25.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.979+0000 7fc4f7fff700 1 --2- 172.21.15.82:0/764753975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4f81021a0 0x7fc4f819d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.980+0000 7fc4f7fff700 1 -- 172.21.15.82:0/764753975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4f8101800 msgr2=0x7fc4f8196890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:25.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.980+0000 7fc4f7fff700 1 --2- 172.21.15.82:0/764753975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4f8101800 0x7fc4f8196890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:25.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.980+0000 7fc4f7fff700 1 -- 172.21.15.82:0/764753975 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e00095f0 con 0x7fc4f8100610 2023-09-09T23:12:25.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.980+0000 7fc4effff700 1 --2- 172.21.15.82:0/764753975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4f8101800 0x7fc4f8196890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:12:25.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.980+0000 7fc4f7fff700 1 --2- 172.21.15.82:0/764753975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4f8100610 0x7fc4f8196350 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc4e800b500 tx=0x7fc4e800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:25.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.980+0000 7fc4f5ffb700 1 -- 172.21.15.82:0/764753975 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8011830 con 0x7fc4f8100610 2023-09-09T23:12:25.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.980+0000 7fc4f5ffb700 1 -- 172.21.15.82:0/764753975 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4e8011e50 con 0x7fc4f8100610 2023-09-09T23:12:25.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.980+0000 7fc4f5ffb700 1 -- 172.21.15.82:0/764753975 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e800f2e0 con 0x7fc4f8100610 2023-09-09T23:12:25.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.980+0000 7fc4fe6f8700 1 -- 172.21.15.82:0/764753975 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f819e480 con 0x7fc4f8100610 2023-09-09T23:12:25.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.980+0000 7fc4fe6f8700 1 -- 172.21.15.82:0/764753975 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f819e9c0 con 0x7fc4f8100610 2023-09-09T23:12:25.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.981+0000 7fc4fe6f8700 1 -- 172.21.15.82:0/764753975 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f810bf40 con 0x7fc4f8100610 2023-09-09T23:12:25.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.982+0000 7fc4f5ffb700 1 -- 172.21.15.82:0/764753975 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fc4e8010580 con 0x7fc4f8100610 2023-09-09T23:12:25.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.982+0000 7fc4f5ffb700 1 --2- 172.21.15.82:0/764753975 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fc4e40a0710 0x7fc4e40a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:25.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.982+0000 7fc4f5ffb700 1 -- 172.21.15.82:0/764753975 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc4e80cbad0 con 0x7fc4f8100610 2023-09-09T23:12:25.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.982+0000 7fc4effff700 1 --2- 172.21.15.82:0/764753975 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fc4e40a0710 0x7fc4e40a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:25.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.983+0000 7fc4effff700 1 --2- 172.21.15.82:0/764753975 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fc4e40a0710 0x7fc4e40a2bd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0006010 tx=0x7fc4e0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:25.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:25.986+0000 7fc4f5ffb700 1 -- 172.21.15.82:0/764753975 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc4e8095760 con 0x7fc4f8100610 2023-09-09T23:12:26.176 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.178+0000 7fc4fe6f8700 1 -- 172.21.15.82:0/764753975 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4f810c180 con 0x7fc4f8100610 2023-09-09T23:12:26.178 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.179+0000 7fc4f5ffb700 1 -- 172.21.15.82:0/764753975 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22071 (secure 0 0 0) 0x7fc4e8098d80 con 0x7fc4f8100610 2023-09-09T23:12:26.178 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:12:26.179 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":81,"fsid":"3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da","created":"2023-09-09T23:05:14.222190+0000","modified":"2023-09-09T23:12:18.894339+0000","last_up_change":"2023-09-09T23:11:37.253759+0000","last_in_change":"2023-09-09T23:11:19.933812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-09T23:08:31.234958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-09T23:12:15.524887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-09T23:12:16.785403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1277eb17-a2e7-4e10-9832-f12a5e93ffde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6803","nonce":3197434929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6805","nonce":3197434929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6809","nonce":3197434929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6807","nonce":3197434929}]},"public_addr":"172.21.15.82:6803/3197434929","cluster_addr":"172.21.15.82:6805/3197434929","heartbeat_back_addr":"172.21.15.82:6809/3197434929","heartbeat_front_addr":"172.21.15.82:6807/3197434929","state":["exists","up"]},{"osd":1,"uuid":"f68723fd-0908-4e21-ae8e-1de1c3a9a65c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6801","nonce":3528641314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6803","nonce":3528641314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6807","nonce":3528641314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6805","nonce":3528641314}]},"public_addr":"172.21.15.98:6801/3528641314","cluster_addr":"172.21.15.98:6803/3528641314","heartbeat_back_addr":"172.21.15.98:6807/3528641314","heartbeat_front_addr":"172.21.15.98:6805/3528641314","state":["exists","up"]},{"osd":2,"uuid":"06f89c53-1419-4f52-acad-6e92916b476f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6801","nonce":4006414551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6803","nonce":4006414551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6807","nonce":4006414551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6805","nonce":4006414551}]},"public_addr":"172.21.15.170:6801/4006414551","cluster_addr":"172.21.15.170:6803/4006414551","heartbeat_back_addr":"172.21.15.170:6807/4006414551","heartbeat_front_addr":"172.21.15.170:6805/4006414551","state":["exists","up"]},{"osd":3,"uuid":"fedbd4c9-6852-42b5-8f3a-ba8d9bfe5d59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6811","nonce":3647629241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6813","nonce":3647629241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6817","nonce":3647629241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6815","nonce":3647629241}]},"public_addr":"172.21.15.82:6811/3647629241","cluster_addr":"172.21.15.82:6813/3647629241","heartbeat_back_addr":"172.21.15.82:6817/3647629241","heartbeat_front_addr":"172.21.15.82:6815/3647629241","state":["exists","up"]},{"osd":4,"uuid":"a71f3084-6919-4161-87bf-345f3355d8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6809","nonce":2507035643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6811","nonce":2507035643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6815","nonce":2507035643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6813","nonce":2507035643}]},"public_addr":"172.21.15.98:6809/2507035643","cluster_addr":"172.21.15.98:6811/2507035643","heartbeat_back_addr":"172.21.15.98:6815/2507035643","heartbeat_front_addr":"172.21.15.98:6813/2507035643","state":["exists","up"]},{"osd":5,"uuid":"f76058e9-a90e-476e-87f0-3c6723795e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6809","nonce":775005168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6811","nonce":775005168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6815","nonce":775005168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6813","nonce":775005168}]},"public_addr":"172.21.15.170:6809/775005168","cluster_addr":"172.21.15.170:6811/775005168","heartbeat_back_addr":"172.21.15.170:6815/775005168","heartbeat_front_addr":"172.21.15.170:6813/775005168","state":["exists","up"]},{"osd":6,"uuid":"25940be0-c1bd-4c72-95bf-fd8367aa2f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6819","nonce":3008405982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6821","nonce":3008405982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6825","nonce":3008405982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6823","nonce":3008405982}]},"public_addr":"172.21.15.82:6819/3008405982","cluster_addr":"172.21.15.82:6821/3008405982","heartbeat_back_addr":"172.21.15.82:6825/3008405982","heartbeat_front_addr":"172.21.15.82:6823/3008405982","state":["exists","up"]},{"osd":7,"uuid":"f651bb8c-0b25-42d5-b028-58563f4c27f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6817","nonce":3547615284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6819","nonce":3547615284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6823","nonce":3547615284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6821","nonce":3547615284}]},"public_addr":"172.21.15.98:6817/3547615284","cluster_addr":"172.21.15.98:6819/3547615284","heartbeat_back_addr":"172.21.15.98:6823/3547615284","heartbeat_front_addr":"172.21.15.98:6821/3547615284","state":["exists","up"]},{"osd":8,"uuid":"23b27c86-8c98-441e-b7ab-d57bc584e58f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6817","nonce":632038848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6819","nonce":632038848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6823","nonce":632038848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6821","nonce":632038848}]},"public_addr":"172.21.15.170:6817/632038848","cluster_addr":"172.21.15.170:6819/632038848","heartbeat_back_addr":"172.21.15.170:6823/632038848","heartbeat_front_addr":"172.21.15.170:6821/632038848","state":["exists","up"]},{"osd":9,"uuid":"383b8989-4922-4eee-9c77-f198258ac243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6827","nonce":3354242311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6829","nonce":3354242311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6833","nonce":3354242311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6831","nonce":3354242311}]},"public_addr":"172.21.15.82:6827/3354242311","cluster_addr":"172.21.15.82:6829/3354242311","heartbeat_back_addr":"172.21.15.82:6833/3354242311","heartbeat_front_addr":"172.21.15.82:6831/3354242311","state":["exists","up"]},{"osd":10,"uuid":"33a59389-7ced-42d5-bf54-2071317132d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6825","nonce":2069114959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6827","nonce":2069114959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6831","nonce":2069114959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6829","nonce":2069114959}]},"public_addr":"172.21.15.98:6825/2069114959","cluster_addr":"172.21.15.98:6827/2069114959","heartbeat_back_addr":"172.21.15.98:6831/2069114959","heartbeat_front_addr":"172.21.15.98:6829/2069114959","state":["exists","up"]},{"osd":11,"uuid":"b7a71f5c-85e2-4fec-be4b-95be0788b302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6825","nonce":146964512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6827","nonce":146964512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6831","nonce":146964512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6829","nonce":146964512}]},"public_addr":"172.21.15.170:6825/146964512","cluster_addr":"172.21.15.170:6827/146964512","heartbeat_back_addr":"172.21.15.170:6831/146964512","heartbeat_front_addr":"172.21.15.170:6829/146964512","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:07:47.563043+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:08.359260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:29.070374+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:49.897775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:11.434248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:31.538464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:52.377109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:13.452968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:33.806897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:54.083177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:14.980087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:35.690089+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4013747788":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6800/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/2823341464":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/75116806":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/1037625632":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/3142088817":"2023-09-10T23:05:46.550354+0000","172.21.15.82:6800/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/1679622963":"2023-09-10T23:05:46.550354+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-09T23:12:26.180 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.182+0000 7fc4fe6f8700 1 -- 172.21.15.82:0/764753975 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fc4e40a0710 msgr2=0x7fc4e40a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:26.181 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.182+0000 7fc4fe6f8700 1 --2- 172.21.15.82:0/764753975 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fc4e40a0710 0x7fc4e40a2bd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0006010 tx=0x7fc4e0000bc0 comp rx=0 tx=0).stop 2023-09-09T23:12:26.181 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.182+0000 7fc4fe6f8700 1 -- 172.21.15.82:0/764753975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4f8100610 msgr2=0x7fc4f8196350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:26.181 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.182+0000 7fc4fe6f8700 1 --2- 172.21.15.82:0/764753975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4f8100610 0x7fc4f8196350 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc4e800b500 tx=0x7fc4e800b8c0 comp rx=0 tx=0).stop 2023-09-09T23:12:26.181 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.182+0000 7fc4fe6f8700 1 -- 172.21.15.82:0/764753975 shutdown_connections 2023-09-09T23:12:26.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.182+0000 7fc4fe6f8700 1 --2- 172.21.15.82:0/764753975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4f81021a0 0x7fc4f819d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:26.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.182+0000 7fc4fe6f8700 1 --2- 172.21.15.82:0/764753975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4f8100610 0x7fc4f8196350 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:26.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.182+0000 7fc4fe6f8700 1 --2- 172.21.15.82:0/764753975 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fc4e40a0710 0x7fc4e40a2bd0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:26.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.182+0000 7fc4fe6f8700 1 --2- 172.21.15.82:0/764753975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4f8101800 0x7fc4f8196890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:26.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.182+0000 7fc4fe6f8700 1 -- 172.21.15.82:0/764753975 >> 172.21.15.82:0/764753975 conn(0x7fc4f80fbb40 msgr2=0x7fc4f80fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:26.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.182+0000 7fc4fe6f8700 1 -- 172.21.15.82:0/764753975 shutdown_connections 2023-09-09T23:12:26.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.182+0000 7fc4fe6f8700 1 -- 172.21.15.82:0/764753975 wait complete. 2023-09-09T23:12:26.193 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-09T23:12:26.193 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-09T23:12:26.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.317+0000 7f0a399fa700 1 -- 172.21.15.82:0/355118842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a34106360 msgr2=0x7f0a34108750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:26.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.317+0000 7f0a399fa700 1 --2- 172.21.15.82:0/355118842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a34106360 0x7f0a34108750 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00b0a0 tx=0x7f0a2c00b3b0 comp rx=0 tx=0).stop 2023-09-09T23:12:26.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.317+0000 7f0a399fa700 1 -- 172.21.15.82:0/355118842 shutdown_connections 2023-09-09T23:12:26.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.317+0000 7f0a399fa700 1 --2- 172.21.15.82:0/355118842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a34101100 0x7f0a341034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:26.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.317+0000 7f0a399fa700 1 --2- 172.21.15.82:0/355118842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a34106360 0x7f0a34108750 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:26.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.317+0000 7f0a399fa700 1 --2- 172.21.15.82:0/355118842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a34103a30 0x7f0a34105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:26.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.317+0000 7f0a399fa700 1 -- 172.21.15.82:0/355118842 >> 172.21.15.82:0/355118842 conn(0x7f0a340faa30 msgr2=0x7f0a340fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:26.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.317+0000 7f0a399fa700 1 -- 172.21.15.82:0/355118842 shutdown_connections 2023-09-09T23:12:26.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.317+0000 7f0a399fa700 1 -- 172.21.15.82:0/355118842 wait complete. 2023-09-09T23:12:26.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.318+0000 7f0a399fa700 1 Processor -- start 2023-09-09T23:12:26.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.318+0000 7f0a399fa700 1 -- start start 2023-09-09T23:12:26.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.318+0000 7f0a399fa700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a34101100 0x7f0a341963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:26.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.318+0000 7f0a399fa700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a34103a30 0x7f0a34196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:26.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.318+0000 7f0a399fa700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a34106360 0x7f0a3419d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:26.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.318+0000 7f0a399fa700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3419df60 con 0x7f0a34103a30 2023-09-09T23:12:26.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.318+0000 7f0a399fa700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3419e0a0 con 0x7f0a34106360 2023-09-09T23:12:26.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.318+0000 7f0a399fa700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3419e230 con 0x7f0a34101100 2023-09-09T23:12:26.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.318+0000 7f0a327fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a34103a30 0x7f0a34196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:26.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.318+0000 7f0a327fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a34103a30 0x7f0a34196920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54640/0 (socket says 172.21.15.82:54640) 2023-09-09T23:12:26.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.318+0000 7f0a327fc700 1 -- 172.21.15.82:0/3779831358 learned_addr learned my addr 172.21.15.82:0/3779831358 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:26.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.318+0000 7f0a32ffd700 1 --2- 172.21.15.82:0/3779831358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a34101100 0x7f0a341963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:26.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.318+0000 7f0a327fc700 1 -- 172.21.15.82:0/3779831358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a34101100 msgr2=0x7f0a341963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:26.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.318+0000 7f0a327fc700 1 --2- 172.21.15.82:0/3779831358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a34101100 0x7f0a341963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:26.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.318+0000 7f0a327fc700 1 -- 172.21.15.82:0/3779831358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a34106360 msgr2=0x7f0a3419d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:26.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.318+0000 7f0a337fe700 1 --2- 172.21.15.82:0/3779831358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a34106360 0x7f0a3419d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:26.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.318+0000 7f0a327fc700 1 --2- 172.21.15.82:0/3779831358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a34106360 0x7f0a3419d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:26.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.318+0000 7f0a327fc700 1 -- 172.21.15.82:0/3779831358 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a2c009d00 con 0x7f0a34103a30 2023-09-09T23:12:26.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.318+0000 7f0a327fc700 1 --2- 172.21.15.82:0/3779831358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a34103a30 0x7f0a34196920 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0a2400b530 tx=0x7f0a2400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:26.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.319+0000 7f0a389f8700 1 -- 172.21.15.82:0/3779831358 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a24011830 con 0x7f0a34103a30 2023-09-09T23:12:26.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.319+0000 7f0a399fa700 1 -- 172.21.15.82:0/3779831358 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3419e510 con 0x7f0a34103a30 2023-09-09T23:12:26.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.319+0000 7f0a389f8700 1 -- 172.21.15.82:0/3779831358 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a24011e50 con 0x7f0a34103a30 2023-09-09T23:12:26.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.319+0000 7f0a389f8700 1 -- 172.21.15.82:0/3779831358 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2400f2e0 con 0x7f0a34103a30 2023-09-09T23:12:26.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.319+0000 7f0a399fa700 1 -- 172.21.15.82:0/3779831358 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3419eaf0 con 0x7f0a34103a30 2023-09-09T23:12:26.325 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.320+0000 7f0a399fa700 1 -- 172.21.15.82:0/3779831358 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a340fc5b0 con 0x7f0a34103a30 2023-09-09T23:12:26.326 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.321+0000 7f0a389f8700 1 -- 172.21.15.82:0/3779831358 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f0a24010580 con 0x7f0a34103a30 2023-09-09T23:12:26.326 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.321+0000 7f0a389f8700 1 --2- 172.21.15.82:0/3779831358 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0a1c0a0710 0x7f0a1c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:26.326 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.321+0000 7f0a32ffd700 1 --2- 172.21.15.82:0/3779831358 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0a1c0a0710 0x7f0a1c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:26.326 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.321+0000 7f0a389f8700 1 -- 172.21.15.82:0/3779831358 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0a240cbb00 con 0x7f0a34103a30 2023-09-09T23:12:26.326 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.322+0000 7f0a32ffd700 1 --2- 172.21.15.82:0/3779831358 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0a1c0a0710 0x7f0a1c0a2bd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0a28009bf0 tx=0x7f0a28009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:26.327 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.325+0000 7f0a389f8700 1 -- 172.21.15.82:0/3779831358 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0a24095790 con 0x7f0a34103a30 2023-09-09T23:12:26.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:26 smithi082 ceph-mon[98694]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:12:26.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:26 smithi082 ceph-mon[98694]: pgmap v259: 97 pgs: 97 active+clean; 582 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 4.7 KiB/s wr, 13 op/s 2023-09-09T23:12:26.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:26 smithi082 ceph-mon[98694]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:12:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:26 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] up:active 2023-09-09T23:12:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2023-09-09T23:12:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:26 smithi082 ceph-mon[98694]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-09-09T23:12:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:12:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:26 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2972752265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:12:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:26 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/764753975' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:12:26.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.514+0000 7f0a399fa700 1 -- 172.21.15.82:0/3779831358 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0a3410be40 con 0x7f0a1c0a0710 2023-09-09T23:12:26.523 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.524+0000 7f0a389f8700 1 -- 172.21.15.82:0/3779831358 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335938 (secure 0 0 0) 0x7f0a3410be40 con 0x7f0a1c0a0710 2023-09-09T23:12:26.527 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:12:26.530 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.528+0000 7f0a399fa700 1 -- 172.21.15.82:0/3779831358 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0a1c0a0710 msgr2=0x7f0a1c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:26.530 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.528+0000 7f0a399fa700 1 --2- 172.21.15.82:0/3779831358 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0a1c0a0710 0x7f0a1c0a2bd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0a28009bf0 tx=0x7f0a28009260 comp rx=0 tx=0).stop 2023-09-09T23:12:26.531 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.528+0000 7f0a399fa700 1 -- 172.21.15.82:0/3779831358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a34103a30 msgr2=0x7f0a34196920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:26.531 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.528+0000 7f0a399fa700 1 --2- 172.21.15.82:0/3779831358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a34103a30 0x7f0a34196920 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0a2400b530 tx=0x7f0a2400b8f0 comp rx=0 tx=0).stop 2023-09-09T23:12:26.531 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.528+0000 7f0a399fa700 1 -- 172.21.15.82:0/3779831358 shutdown_connections 2023-09-09T23:12:26.531 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.528+0000 7f0a399fa700 1 --2- 172.21.15.82:0/3779831358 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0a1c0a0710 0x7f0a1c0a2bd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:26.532 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.528+0000 7f0a399fa700 1 --2- 172.21.15.82:0/3779831358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a34106360 0x7f0a3419d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:26.532 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.528+0000 7f0a399fa700 1 --2- 172.21.15.82:0/3779831358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a34101100 0x7f0a341963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:26.532 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.528+0000 7f0a399fa700 1 --2- 172.21.15.82:0/3779831358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a34103a30 0x7f0a34196920 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:26.532 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.528+0000 7f0a399fa700 1 -- 172.21.15.82:0/3779831358 >> 172.21.15.82:0/3779831358 conn(0x7f0a340faa30 msgr2=0x7f0a34101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:26.532 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.528+0000 7f0a399fa700 1 -- 172.21.15.82:0/3779831358 shutdown_connections 2023-09-09T23:12:26.533 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.528+0000 7f0a399fa700 1 -- 172.21.15.82:0/3779831358 wait complete. 2023-09-09T23:12:26.533 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-09-09T23:12:26.539 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2023-09-09T23:12:24.979498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596366,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":148,"num_read_kb":129,"num_write":275,"num_write_kb":2975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":275,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336328,"kb_used_data":12084,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124605496,"statfs":{"total":1151940427776,"available":1151596027904,"internally_reserved":0,"allocated":12374016,"data_stored":6768560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5998,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":94,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.823662"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.988649+0000","last_change":"2023-09-09T23:12:17.903811+0000","last_active":"2023-09-09T23:12:18.988649+0000","last_peered":"2023-09-09T23:12:18.988649+0000","last_clean":"2023-09-09T23:12:18.988649+0000","last_became_active":"2023-09-09T23:12:17.903386+0000","last_became_peered":"2023-09-09T23:12:17.903386+0000","last_unstale":"2023-09-09T23:12:18.988649+0000","last_undegraded":"2023-09-09T23:12:18.988649+0000","last_fullsized":"2023-09-09T23:12:18.988649+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:36:45.122068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.485007+0000","last_change":"2023-09-09T23:12:17.897915+0000","last_active":"2023-09-09T23:12:19.485007+0000","last_peered":"2023-09-09T23:12:19.485007+0000","last_clean":"2023-09-09T23:12:19.485007+0000","last_became_active":"2023-09-09T23:12:17.896727+0000","last_became_peered":"2023-09-09T23:12:17.896727+0000","last_unstale":"2023-09-09T23:12:19.485007+0000","last_undegraded":"2023-09-09T23:12:19.485007+0000","last_fullsized":"2023-09-09T23:12:19.485007+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:36:03.053572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.485083+0000","last_change":"2023-09-09T23:12:17.907957+0000","last_active":"2023-09-09T23:12:19.485083+0000","last_peered":"2023-09-09T23:12:19.485083+0000","last_clean":"2023-09-09T23:12:19.485083+0000","last_became_active":"2023-09-09T23:12:17.907625+0000","last_became_peered":"2023-09-09T23:12:17.907625+0000","last_unstale":"2023-09-09T23:12:19.485083+0000","last_undegraded":"2023-09-09T23:12:19.485083+0000","last_fullsized":"2023-09-09T23:12:19.485083+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:26:56.520987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.045364+0000","last_change":"2023-09-09T23:12:17.903489+0000","last_active":"2023-09-09T23:12:20.045364+0000","last_peered":"2023-09-09T23:12:20.045364+0000","last_clean":"2023-09-09T23:12:20.045364+0000","last_became_active":"2023-09-09T23:12:17.901184+0000","last_became_peered":"2023-09-09T23:12:17.901184+0000","last_unstale":"2023-09-09T23:12:20.045364+0000","last_undegraded":"2023-09-09T23:12:20.045364+0000","last_fullsized":"2023-09-09T23:12:20.045364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:42:01.899998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897810+0000","last_change":"2023-09-09T23:12:17.899487+0000","last_active":"2023-09-09T23:12:18.897810+0000","last_peered":"2023-09-09T23:12:18.897810+0000","last_clean":"2023-09-09T23:12:18.897810+0000","last_became_active":"2023-09-09T23:12:17.899083+0000","last_became_peered":"2023-09-09T23:12:17.899083+0000","last_unstale":"2023-09-09T23:12:18.897810+0000","last_undegraded":"2023-09-09T23:12:18.897810+0000","last_fullsized":"2023-09-09T23:12:18.897810+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:15:20.109849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690024+0000","last_change":"2023-09-09T23:12:17.901554+0000","last_active":"2023-09-09T23:12:20.690024+0000","last_peered":"2023-09-09T23:12:20.690024+0000","last_clean":"2023-09-09T23:12:20.690024+0000","last_became_active":"2023-09-09T23:12:17.901246+0000","last_became_peered":"2023-09-09T23:12:17.901246+0000","last_unstale":"2023-09-09T23:12:20.690024+0000","last_undegraded":"2023-09-09T23:12:20.690024+0000","last_fullsized":"2023-09-09T23:12:20.690024+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:19:08.795927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689639+0000","last_change":"2023-09-09T23:12:17.906989+0000","last_active":"2023-09-09T23:12:20.689639+0000","last_peered":"2023-09-09T23:12:20.689639+0000","last_clean":"2023-09-09T23:12:20.689639+0000","last_became_active":"2023-09-09T23:12:17.906704+0000","last_became_peered":"2023-09-09T23:12:17.906704+0000","last_unstale":"2023-09-09T23:12:20.689639+0000","last_undegraded":"2023-09-09T23:12:20.689639+0000","last_fullsized":"2023-09-09T23:12:20.689639+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:41:33.612168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897513+0000","last_change":"2023-09-09T23:12:17.906269+0000","last_active":"2023-09-09T23:12:18.897513+0000","last_peered":"2023-09-09T23:12:18.897513+0000","last_clean":"2023-09-09T23:12:18.897513+0000","last_became_active":"2023-09-09T23:12:17.906017+0000","last_became_peered":"2023-09-09T23:12:17.906017+0000","last_unstale":"2023-09-09T23:12:18.897513+0000","last_undegraded":"2023-09-09T23:12:18.897513+0000","last_fullsized":"2023-09-09T23:12:18.897513+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:50:21.893506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.520143+0000","last_change":"2023-09-09T23:12:17.905191+0000","last_active":"2023-09-09T23:12:19.520143+0000","last_peered":"2023-09-09T23:12:19.520143+0000","last_clean":"2023-09-09T23:12:19.520143+0000","last_became_active":"2023-09-09T23:12:17.904813+0000","last_became_peered":"2023-09-09T23:12:17.904813+0000","last_unstale":"2023-09-09T23:12:19.520143+0000","last_undegraded":"2023-09-09T23:12:19.520143+0000","last_fullsized":"2023-09-09T23:12:19.520143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:29:54.722210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690667+0000","last_change":"2023-09-09T23:12:17.899980+0000","last_active":"2023-09-09T23:12:20.690667+0000","last_peered":"2023-09-09T23:12:20.690667+0000","last_clean":"2023-09-09T23:12:20.690667+0000","last_became_active":"2023-09-09T23:12:17.899692+0000","last_became_peered":"2023-09-09T23:12:17.899692+0000","last_unstale":"2023-09-09T23:12:20.690667+0000","last_undegraded":"2023-09-09T23:12:20.690667+0000","last_fullsized":"2023-09-09T23:12:20.690667+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:25:28.056684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690090+0000","last_change":"2023-09-09T23:12:17.907010+0000","last_active":"2023-09-09T23:12:20.690090+0000","last_peered":"2023-09-09T23:12:20.690090+0000","last_clean":"2023-09-09T23:12:20.690090+0000","last_became_active":"2023-09-09T23:12:17.906565+0000","last_became_peered":"2023-09-09T23:12:17.906565+0000","last_unstale":"2023-09-09T23:12:20.690090+0000","last_undegraded":"2023-09-09T23:12:20.690090+0000","last_fullsized":"2023-09-09T23:12:20.690090+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:08:03.733608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.045231+0000","last_change":"2023-09-09T23:12:17.902889+0000","last_active":"2023-09-09T23:12:20.045231+0000","last_peered":"2023-09-09T23:12:20.045231+0000","last_clean":"2023-09-09T23:12:20.045231+0000","last_became_active":"2023-09-09T23:12:17.900747+0000","last_became_peered":"2023-09-09T23:12:17.900747+0000","last_unstale":"2023-09-09T23:12:20.045231+0000","last_undegraded":"2023-09-09T23:12:20.045231+0000","last_fullsized":"2023-09-09T23:12:20.045231+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:01:29.195275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:17.899880+0000","last_change":"2023-09-09T23:12:17.897844+0000","last_active":"2023-09-09T23:12:17.899880+0000","last_peered":"2023-09-09T23:12:17.899880+0000","last_clean":"2023-09-09T23:12:17.899880+0000","last_became_active":"2023-09-09T23:12:17.897529+0000","last_became_peered":"2023-09-09T23:12:17.897529+0000","last_unstale":"2023-09-09T23:12:17.899880+0000","last_undegraded":"2023-09-09T23:12:17.899880+0000","last_fullsized":"2023-09-09T23:12:17.899880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:35:48.046254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.484862+0000","last_change":"2023-09-09T23:12:17.899255+0000","last_active":"2023-09-09T23:12:19.484862+0000","last_peered":"2023-09-09T23:12:19.484862+0000","last_clean":"2023-09-09T23:12:19.484862+0000","last_became_active":"2023-09-09T23:12:17.898932+0000","last_became_peered":"2023-09-09T23:12:17.898932+0000","last_unstale":"2023-09-09T23:12:19.484862+0000","last_undegraded":"2023-09-09T23:12:19.484862+0000","last_fullsized":"2023-09-09T23:12:19.484862+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:10:56.864588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.484825+0000","last_change":"2023-09-09T23:12:17.901083+0000","last_active":"2023-09-09T23:12:19.484825+0000","last_peered":"2023-09-09T23:12:19.484825+0000","last_clean":"2023-09-09T23:12:19.484825+0000","last_became_active":"2023-09-09T23:12:17.900744+0000","last_became_peered":"2023-09-09T23:12:17.900744+0000","last_unstale":"2023-09-09T23:12:19.484825+0000","last_undegraded":"2023-09-09T23:12:19.484825+0000","last_fullsized":"2023-09-09T23:12:19.484825+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:37:11.073786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.519605+0000","last_change":"2023-09-09T23:12:17.901962+0000","last_active":"2023-09-09T23:12:19.519605+0000","last_peered":"2023-09-09T23:12:19.519605+0000","last_clean":"2023-09-09T23:12:19.519605+0000","last_became_active":"2023-09-09T23:12:17.901638+0000","last_became_peered":"2023-09-09T23:12:17.901638+0000","last_unstale":"2023-09-09T23:12:19.519605+0000","last_undegraded":"2023-09-09T23:12:19.519605+0000","last_fullsized":"2023-09-09T23:12:19.519605+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:34:52.083262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.988721+0000","last_change":"2023-09-09T23:12:17.904424+0000","last_active":"2023-09-09T23:12:18.988721+0000","last_peered":"2023-09-09T23:12:18.988721+0000","last_clean":"2023-09-09T23:12:18.988721+0000","last_became_active":"2023-09-09T23:12:17.903879+0000","last_became_peered":"2023-09-09T23:12:17.903879+0000","last_unstale":"2023-09-09T23:12:18.988721+0000","last_undegraded":"2023-09-09T23:12:18.988721+0000","last_fullsized":"2023-09-09T23:12:18.988721+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:56:42.468630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898791+0000","last_change":"2023-09-09T23:12:17.905065+0000","last_active":"2023-09-09T23:12:18.898791+0000","last_peered":"2023-09-09T23:12:18.898791+0000","last_clean":"2023-09-09T23:12:18.898791+0000","last_became_active":"2023-09-09T23:12:17.904646+0000","last_became_peered":"2023-09-09T23:12:17.904646+0000","last_unstale":"2023-09-09T23:12:18.898791+0000","last_undegraded":"2023-09-09T23:12:18.898791+0000","last_fullsized":"2023-09-09T23:12:18.898791+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:11:14.705884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690263+0000","last_change":"2023-09-09T23:12:17.899751+0000","last_active":"2023-09-09T23:12:20.690263+0000","last_peered":"2023-09-09T23:12:20.690263+0000","last_clean":"2023-09-09T23:12:20.690263+0000","last_became_active":"2023-09-09T23:12:17.899410+0000","last_became_peered":"2023-09-09T23:12:17.899410+0000","last_unstale":"2023-09-09T23:12:20.690263+0000","last_undegraded":"2023-09-09T23:12:20.690263+0000","last_fullsized":"2023-09-09T23:12:20.690263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:23:06.098276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'4","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.544646+0000","last_change":"2023-09-09T23:12:17.175671+0000","last_active":"2023-09-09T23:12:18.544646+0000","last_peered":"2023-09-09T23:12:18.544646+0000","last_clean":"2023-09-09T23:12:18.544646+0000","last_became_active":"2023-09-09T23:12:17.175327+0000","last_became_peered":"2023-09-09T23:12:17.175327+0000","last_unstale":"2023-09-09T23:12:18.544646+0000","last_undegraded":"2023-09-09T23:12:18.544646+0000","last_fullsized":"2023-09-09T23:12:18.544646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:38:57.724260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898994+0000","last_change":"2023-09-09T23:12:17.906338+0000","last_active":"2023-09-09T23:12:18.898994+0000","last_peered":"2023-09-09T23:12:18.898994+0000","last_clean":"2023-09-09T23:12:18.898994+0000","last_became_active":"2023-09-09T23:12:17.906094+0000","last_became_peered":"2023-09-09T23:12:17.906094+0000","last_unstale":"2023-09-09T23:12:18.898994+0000","last_undegraded":"2023-09-09T23:12:18.898994+0000","last_fullsized":"2023-09-09T23:12:18.898994+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:43:18.183144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.542430+0000","last_change":"2023-09-09T23:12:17.167763+0000","last_active":"2023-09-09T23:12:18.542430+0000","last_peered":"2023-09-09T23:12:18.542430+0000","last_clean":"2023-09-09T23:12:18.542430+0000","last_became_active":"2023-09-09T23:12:17.167455+0000","last_became_peered":"2023-09-09T23:12:17.167455+0000","last_unstale":"2023-09-09T23:12:18.542430+0000","last_undegraded":"2023-09-09T23:12:18.542430+0000","last_fullsized":"2023-09-09T23:12:18.542430+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:15:23.415792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.485136+0000","last_change":"2023-09-09T23:12:17.907936+0000","last_active":"2023-09-09T23:12:19.485136+0000","last_peered":"2023-09-09T23:12:19.485136+0000","last_clean":"2023-09-09T23:12:19.485136+0000","last_became_active":"2023-09-09T23:12:17.907588+0000","last_became_peered":"2023-09-09T23:12:17.907588+0000","last_unstale":"2023-09-09T23:12:19.485136+0000","last_undegraded":"2023-09-09T23:12:19.485136+0000","last_fullsized":"2023-09-09T23:12:19.485136+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:57:30.910812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'2","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898138+0000","last_change":"2023-09-09T23:12:17.172481+0000","last_active":"2023-09-09T23:12:18.898138+0000","last_peered":"2023-09-09T23:12:18.898138+0000","last_clean":"2023-09-09T23:12:18.898138+0000","last_became_active":"2023-09-09T23:12:17.172178+0000","last_became_peered":"2023-09-09T23:12:17.172178+0000","last_unstale":"2023-09-09T23:12:18.898138+0000","last_undegraded":"2023-09-09T23:12:18.898138+0000","last_fullsized":"2023-09-09T23:12:18.898138+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:08:55.328423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.519862+0000","last_change":"2023-09-09T23:12:17.900155+0000","last_active":"2023-09-09T23:12:19.519862+0000","last_peered":"2023-09-09T23:12:19.519862+0000","last_clean":"2023-09-09T23:12:19.519862+0000","last_became_active":"2023-09-09T23:12:17.899862+0000","last_became_peered":"2023-09-09T23:12:17.899862+0000","last_unstale":"2023-09-09T23:12:19.519862+0000","last_undegraded":"2023-09-09T23:12:19.519862+0000","last_fullsized":"2023-09-09T23:12:19.519862+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:57:14.949174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897973+0000","last_change":"2023-09-09T23:12:17.171890+0000","last_active":"2023-09-09T23:12:18.897973+0000","last_peered":"2023-09-09T23:12:18.897973+0000","last_clean":"2023-09-09T23:12:18.897973+0000","last_became_active":"2023-09-09T23:12:17.171496+0000","last_became_peered":"2023-09-09T23:12:17.171496+0000","last_unstale":"2023-09-09T23:12:18.897973+0000","last_undegraded":"2023-09-09T23:12:18.897973+0000","last_fullsized":"2023-09-09T23:12:18.897973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:38:25.704989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689810+0000","last_change":"2023-09-09T23:12:17.901067+0000","last_active":"2023-09-09T23:12:20.689810+0000","last_peered":"2023-09-09T23:12:20.689810+0000","last_clean":"2023-09-09T23:12:20.689810+0000","last_became_active":"2023-09-09T23:12:17.900069+0000","last_became_peered":"2023-09-09T23:12:17.900069+0000","last_unstale":"2023-09-09T23:12:20.689810+0000","last_undegraded":"2023-09-09T23:12:20.689810+0000","last_fullsized":"2023-09-09T23:12:20.689810+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:59:55.356235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:23.265413+0000","last_change":"2023-09-09T23:12:17.169474+0000","last_active":"2023-09-09T23:12:23.265413+0000","last_peered":"2023-09-09T23:12:23.265413+0000","last_clean":"2023-09-09T23:12:23.265413+0000","last_became_active":"2023-09-09T23:12:17.169186+0000","last_became_peered":"2023-09-09T23:12:17.169186+0000","last_unstale":"2023-09-09T23:12:23.265413+0000","last_undegraded":"2023-09-09T23:12:23.265413+0000","last_fullsized":"2023-09-09T23:12:23.265413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:47:13.027949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897942+0000","last_change":"2023-09-09T23:12:17.901605+0000","last_active":"2023-09-09T23:12:18.897942+0000","last_peered":"2023-09-09T23:12:18.897942+0000","last_clean":"2023-09-09T23:12:18.897942+0000","last_became_active":"2023-09-09T23:12:17.901287+0000","last_became_peered":"2023-09-09T23:12:17.901287+0000","last_unstale":"2023-09-09T23:12:18.897942+0000","last_undegraded":"2023-09-09T23:12:18.897942+0000","last_fullsized":"2023-09-09T23:12:18.897942+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:16:41.570750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.543966+0000","last_change":"2023-09-09T23:12:17.171797+0000","last_active":"2023-09-09T23:12:18.543966+0000","last_peered":"2023-09-09T23:12:18.543966+0000","last_clean":"2023-09-09T23:12:18.543966+0000","last_became_active":"2023-09-09T23:12:17.171521+0000","last_became_peered":"2023-09-09T23:12:17.171521+0000","last_unstale":"2023-09-09T23:12:18.543966+0000","last_undegraded":"2023-09-09T23:12:18.543966+0000","last_fullsized":"2023-09-09T23:12:18.543966+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:07:51.951393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690229+0000","last_change":"2023-09-09T23:12:17.900284+0000","last_active":"2023-09-09T23:12:20.690229+0000","last_peered":"2023-09-09T23:12:20.690229+0000","last_clean":"2023-09-09T23:12:20.690229+0000","last_became_active":"2023-09-09T23:12:17.899975+0000","last_became_peered":"2023-09-09T23:12:17.899975+0000","last_unstale":"2023-09-09T23:12:20.690229+0000","last_undegraded":"2023-09-09T23:12:20.690229+0000","last_fullsized":"2023-09-09T23:12:20.690229+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:10:56.723338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:17.891006+0000","last_change":"2023-09-09T23:12:17.175328+0000","last_active":"2023-09-09T23:12:17.891006+0000","last_peered":"2023-09-09T23:12:17.891006+0000","last_clean":"2023-09-09T23:12:17.891006+0000","last_became_active":"2023-09-09T23:12:17.174977+0000","last_became_peered":"2023-09-09T23:12:17.174977+0000","last_unstale":"2023-09-09T23:12:17.891006+0000","last_undegraded":"2023-09-09T23:12:17.891006+0000","last_fullsized":"2023-09-09T23:12:17.891006+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:01:36.272333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897568+0000","last_change":"2023-09-09T23:12:17.902982+0000","last_active":"2023-09-09T23:12:18.897568+0000","last_peered":"2023-09-09T23:12:18.897568+0000","last_clean":"2023-09-09T23:12:18.897568+0000","last_became_active":"2023-09-09T23:12:17.902677+0000","last_became_peered":"2023-09-09T23:12:17.902677+0000","last_unstale":"2023-09-09T23:12:18.897568+0000","last_undegraded":"2023-09-09T23:12:18.897568+0000","last_fullsized":"2023-09-09T23:12:18.897568+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:08:31.507899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'2","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:21.572877+0000","last_change":"2023-09-09T23:12:17.170099+0000","last_active":"2023-09-09T23:12:21.572877+0000","last_peered":"2023-09-09T23:12:21.572877+0000","last_clean":"2023-09-09T23:12:21.572877+0000","last_became_active":"2023-09-09T23:12:17.169827+0000","last_became_peered":"2023-09-09T23:12:17.169827+0000","last_unstale":"2023-09-09T23:12:21.572877+0000","last_undegraded":"2023-09-09T23:12:21.572877+0000","last_fullsized":"2023-09-09T23:12:21.572877+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:16:34.693568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898646+0000","last_change":"2023-09-09T23:12:17.903735+0000","last_active":"2023-09-09T23:12:18.898646+0000","last_peered":"2023-09-09T23:12:18.898646+0000","last_clean":"2023-09-09T23:12:18.898646+0000","last_became_active":"2023-09-09T23:12:17.903410+0000","last_became_peered":"2023-09-09T23:12:17.903410+0000","last_unstale":"2023-09-09T23:12:18.898646+0000","last_undegraded":"2023-09-09T23:12:18.898646+0000","last_fullsized":"2023-09-09T23:12:18.898646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:00:47.664613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:21.573928+0000","last_change":"2023-09-09T23:12:17.176004+0000","last_active":"2023-09-09T23:12:21.573928+0000","last_peered":"2023-09-09T23:12:21.573928+0000","last_clean":"2023-09-09T23:12:21.573928+0000","last_became_active":"2023-09-09T23:12:17.175514+0000","last_became_peered":"2023-09-09T23:12:17.175514+0000","last_unstale":"2023-09-09T23:12:21.573928+0000","last_undegraded":"2023-09-09T23:12:21.573928+0000","last_fullsized":"2023-09-09T23:12:21.573928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:43:54.587724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.520135+0000","last_change":"2023-09-09T23:12:17.900943+0000","last_active":"2023-09-09T23:12:19.520135+0000","last_peered":"2023-09-09T23:12:19.520135+0000","last_clean":"2023-09-09T23:12:19.520135+0000","last_became_active":"2023-09-09T23:12:17.900662+0000","last_became_peered":"2023-09-09T23:12:17.900662+0000","last_unstale":"2023-09-09T23:12:19.520135+0000","last_undegraded":"2023-09-09T23:12:19.520135+0000","last_fullsized":"2023-09-09T23:12:19.520135+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:39:41.480033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:24.280105+0000","last_change":"2023-09-09T23:12:17.172458+0000","last_active":"2023-09-09T23:12:24.280105+0000","last_peered":"2023-09-09T23:12:24.280105+0000","last_clean":"2023-09-09T23:12:24.280105+0000","last_became_active":"2023-09-09T23:12:17.172119+0000","last_became_peered":"2023-09-09T23:12:17.172119+0000","last_unstale":"2023-09-09T23:12:24.280105+0000","last_undegraded":"2023-09-09T23:12:24.280105+0000","last_fullsized":"2023-09-09T23:12:24.280105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:31:27.821245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.485149+0000","last_change":"2023-09-09T23:12:17.903252+0000","last_active":"2023-09-09T23:12:19.485149+0000","last_peered":"2023-09-09T23:12:19.485149+0000","last_clean":"2023-09-09T23:12:19.485149+0000","last_became_active":"2023-09-09T23:12:17.902466+0000","last_became_peered":"2023-09-09T23:12:17.902466+0000","last_unstale":"2023-09-09T23:12:19.485149+0000","last_undegraded":"2023-09-09T23:12:19.485149+0000","last_fullsized":"2023-09-09T23:12:19.485149+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:28:07.535937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:23.264739+0000","last_change":"2023-09-09T23:12:17.165106+0000","last_active":"2023-09-09T23:12:23.264739+0000","last_peered":"2023-09-09T23:12:23.264739+0000","last_clean":"2023-09-09T23:12:23.264739+0000","last_became_active":"2023-09-09T23:12:17.164783+0000","last_became_peered":"2023-09-09T23:12:17.164783+0000","last_unstale":"2023-09-09T23:12:23.264739+0000","last_undegraded":"2023-09-09T23:12:23.264739+0000","last_fullsized":"2023-09-09T23:12:23.264739+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:02:14.984870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.519952+0000","last_change":"2023-09-09T23:12:17.899473+0000","last_active":"2023-09-09T23:12:19.519952+0000","last_peered":"2023-09-09T23:12:19.519952+0000","last_clean":"2023-09-09T23:12:19.519952+0000","last_became_active":"2023-09-09T23:12:17.899142+0000","last_became_peered":"2023-09-09T23:12:17.899142+0000","last_unstale":"2023-09-09T23:12:19.519952+0000","last_undegraded":"2023-09-09T23:12:19.519952+0000","last_fullsized":"2023-09-09T23:12:19.519952+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:50:30.261873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'8","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:21.574929+0000","last_change":"2023-09-09T23:12:17.172560+0000","last_active":"2023-09-09T23:12:21.574929+0000","last_peered":"2023-09-09T23:12:21.574929+0000","last_clean":"2023-09-09T23:12:21.574929+0000","last_became_active":"2023-09-09T23:12:17.172213+0000","last_became_peered":"2023-09-09T23:12:17.172213+0000","last_unstale":"2023-09-09T23:12:21.574929+0000","last_undegraded":"2023-09-09T23:12:21.574929+0000","last_fullsized":"2023-09-09T23:12:21.574929+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:30:00.206901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898030+0000","last_change":"2023-09-09T23:12:17.898279+0000","last_active":"2023-09-09T23:12:18.898030+0000","last_peered":"2023-09-09T23:12:18.898030+0000","last_clean":"2023-09-09T23:12:18.898030+0000","last_became_active":"2023-09-09T23:12:17.898041+0000","last_became_peered":"2023-09-09T23:12:17.898041+0000","last_unstale":"2023-09-09T23:12:18.898030+0000","last_undegraded":"2023-09-09T23:12:18.898030+0000","last_fullsized":"2023-09-09T23:12:18.898030+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:01:10.315445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'4","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:21.573959+0000","last_change":"2023-09-09T23:12:17.172079+0000","last_active":"2023-09-09T23:12:21.573959+0000","last_peered":"2023-09-09T23:12:21.573959+0000","last_clean":"2023-09-09T23:12:21.573959+0000","last_became_active":"2023-09-09T23:12:17.171779+0000","last_became_peered":"2023-09-09T23:12:17.171779+0000","last_unstale":"2023-09-09T23:12:21.573959+0000","last_undegraded":"2023-09-09T23:12:21.573959+0000","last_fullsized":"2023-09-09T23:12:21.573959+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:49:50.287124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:17.907644+0000","last_change":"2023-09-09T23:12:17.904087+0000","last_active":"2023-09-09T23:12:17.907644+0000","last_peered":"2023-09-09T23:12:17.907644+0000","last_clean":"2023-09-09T23:12:17.907644+0000","last_became_active":"2023-09-09T23:12:17.903670+0000","last_became_peered":"2023-09-09T23:12:17.903670+0000","last_unstale":"2023-09-09T23:12:17.907644+0000","last_undegraded":"2023-09-09T23:12:17.907644+0000","last_fullsized":"2023-09-09T23:12:17.907644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:31:28.511653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'4","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:21.573310+0000","last_change":"2023-09-09T23:12:17.170594+0000","last_active":"2023-09-09T23:12:21.573310+0000","last_peered":"2023-09-09T23:12:21.573310+0000","last_clean":"2023-09-09T23:12:21.573310+0000","last_became_active":"2023-09-09T23:12:17.170223+0000","last_became_peered":"2023-09-09T23:12:17.170223+0000","last_unstale":"2023-09-09T23:12:21.573310+0000","last_undegraded":"2023-09-09T23:12:21.573310+0000","last_fullsized":"2023-09-09T23:12:21.573310+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:30:09.222947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689977+0000","last_change":"2023-09-09T23:12:17.901047+0000","last_active":"2023-09-09T23:12:20.689977+0000","last_peered":"2023-09-09T23:12:20.689977+0000","last_clean":"2023-09-09T23:12:20.689977+0000","last_became_active":"2023-09-09T23:12:17.900129+0000","last_became_peered":"2023-09-09T23:12:17.900129+0000","last_unstale":"2023-09-09T23:12:20.689977+0000","last_undegraded":"2023-09-09T23:12:20.689977+0000","last_fullsized":"2023-09-09T23:12:20.689977+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:59:36.338392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:23.271984+0000","last_change":"2023-09-09T23:12:17.169495+0000","last_active":"2023-09-09T23:12:23.271984+0000","last_peered":"2023-09-09T23:12:23.271984+0000","last_clean":"2023-09-09T23:12:23.271984+0000","last_became_active":"2023-09-09T23:12:17.169170+0000","last_became_peered":"2023-09-09T23:12:17.169170+0000","last_unstale":"2023-09-09T23:12:23.271984+0000","last_undegraded":"2023-09-09T23:12:23.271984+0000","last_fullsized":"2023-09-09T23:12:23.271984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:36:23.765574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689415+0000","last_change":"2023-09-09T23:12:17.907826+0000","last_active":"2023-09-09T23:12:20.689415+0000","last_peered":"2023-09-09T23:12:20.689415+0000","last_clean":"2023-09-09T23:12:20.689415+0000","last_became_active":"2023-09-09T23:12:17.907526+0000","last_became_peered":"2023-09-09T23:12:17.907526+0000","last_unstale":"2023-09-09T23:12:20.689415+0000","last_undegraded":"2023-09-09T23:12:20.689415+0000","last_fullsized":"2023-09-09T23:12:20.689415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:17:20.587253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'8","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897643+0000","last_change":"2023-09-09T23:12:17.170139+0000","last_active":"2023-09-09T23:12:18.897643+0000","last_peered":"2023-09-09T23:12:18.897643+0000","last_clean":"2023-09-09T23:12:18.897643+0000","last_became_active":"2023-09-09T23:12:17.169797+0000","last_became_peered":"2023-09-09T23:12:17.169797+0000","last_unstale":"2023-09-09T23:12:18.897643+0000","last_undegraded":"2023-09-09T23:12:18.897643+0000","last_fullsized":"2023-09-09T23:12:18.897643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:59:44.668114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689753+0000","last_change":"2023-09-09T23:12:17.902567+0000","last_active":"2023-09-09T23:12:20.689753+0000","last_peered":"2023-09-09T23:12:20.689753+0000","last_clean":"2023-09-09T23:12:20.689753+0000","last_became_active":"2023-09-09T23:12:17.902229+0000","last_became_peered":"2023-09-09T23:12:17.902229+0000","last_unstale":"2023-09-09T23:12:20.689753+0000","last_undegraded":"2023-09-09T23:12:20.689753+0000","last_fullsized":"2023-09-09T23:12:20.689753+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:49:50.049925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897980+0000","last_change":"2023-09-09T23:12:17.171143+0000","last_active":"2023-09-09T23:12:18.897980+0000","last_peered":"2023-09-09T23:12:18.897980+0000","last_clean":"2023-09-09T23:12:18.897980+0000","last_became_active":"2023-09-09T23:12:17.170847+0000","last_became_peered":"2023-09-09T23:12:17.170847+0000","last_unstale":"2023-09-09T23:12:18.897980+0000","last_undegraded":"2023-09-09T23:12:18.897980+0000","last_fullsized":"2023-09-09T23:12:18.897980+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:35:55.570394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.988514+0000","last_change":"2023-09-09T23:12:17.900284+0000","last_active":"2023-09-09T23:12:18.988514+0000","last_peered":"2023-09-09T23:12:18.988514+0000","last_clean":"2023-09-09T23:12:18.988514+0000","last_became_active":"2023-09-09T23:12:17.899923+0000","last_became_peered":"2023-09-09T23:12:17.899923+0000","last_unstale":"2023-09-09T23:12:18.988514+0000","last_undegraded":"2023-09-09T23:12:18.988514+0000","last_fullsized":"2023-09-09T23:12:18.988514+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:32:32.230968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:17.889782+0000","last_change":"2023-09-09T23:12:17.176313+0000","last_active":"2023-09-09T23:12:17.889782+0000","last_peered":"2023-09-09T23:12:17.889782+0000","last_clean":"2023-09-09T23:12:17.889782+0000","last_became_active":"2023-09-09T23:12:17.175934+0000","last_became_peered":"2023-09-09T23:12:17.175934+0000","last_unstale":"2023-09-09T23:12:17.889782+0000","last_undegraded":"2023-09-09T23:12:17.889782+0000","last_fullsized":"2023-09-09T23:12:17.889782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:33:15.044986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690469+0000","last_change":"2023-09-09T23:12:17.900284+0000","last_active":"2023-09-09T23:12:20.690469+0000","last_peered":"2023-09-09T23:12:20.690469+0000","last_clean":"2023-09-09T23:12:20.690469+0000","last_became_active":"2023-09-09T23:12:17.899970+0000","last_became_peered":"2023-09-09T23:12:17.899970+0000","last_unstale":"2023-09-09T23:12:20.690469+0000","last_undegraded":"2023-09-09T23:12:20.690469+0000","last_fullsized":"2023-09-09T23:12:20.690469+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:01:37.415195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:24.280539+0000","last_change":"2023-09-09T23:12:17.170123+0000","last_active":"2023-09-09T23:12:24.280539+0000","last_peered":"2023-09-09T23:12:24.280539+0000","last_clean":"2023-09-09T23:12:24.280539+0000","last_became_active":"2023-09-09T23:12:17.169775+0000","last_became_peered":"2023-09-09T23:12:17.169775+0000","last_unstale":"2023-09-09T23:12:24.280539+0000","last_undegraded":"2023-09-09T23:12:24.280539+0000","last_fullsized":"2023-09-09T23:12:24.280539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:22:44.520489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690417+0000","last_change":"2023-09-09T23:12:17.894881+0000","last_active":"2023-09-09T23:12:20.690417+0000","last_peered":"2023-09-09T23:12:20.690417+0000","last_clean":"2023-09-09T23:12:20.690417+0000","last_became_active":"2023-09-09T23:12:17.894567+0000","last_became_peered":"2023-09-09T23:12:17.894567+0000","last_unstale":"2023-09-09T23:12:20.690417+0000","last_undegraded":"2023-09-09T23:12:20.690417+0000","last_fullsized":"2023-09-09T23:12:20.690417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:40:37.098780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689780+0000","last_change":"2023-09-09T23:12:17.903319+0000","last_active":"2023-09-09T23:12:20.689780+0000","last_peered":"2023-09-09T23:12:20.689780+0000","last_clean":"2023-09-09T23:12:20.689780+0000","last_became_active":"2023-09-09T23:12:17.902642+0000","last_became_peered":"2023-09-09T23:12:17.902642+0000","last_unstale":"2023-09-09T23:12:20.689780+0000","last_undegraded":"2023-09-09T23:12:20.689780+0000","last_fullsized":"2023-09-09T23:12:20.689780+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:03:29.562813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:17.890868+0000","last_change":"2023-09-09T23:12:17.172639+0000","last_active":"2023-09-09T23:12:17.890868+0000","last_peered":"2023-09-09T23:12:17.890868+0000","last_clean":"2023-09-09T23:12:17.890868+0000","last_became_active":"2023-09-09T23:12:17.172309+0000","last_became_peered":"2023-09-09T23:12:17.172309+0000","last_unstale":"2023-09-09T23:12:17.890868+0000","last_undegraded":"2023-09-09T23:12:17.890868+0000","last_fullsized":"2023-09-09T23:12:17.890868+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:05:56.160984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898148+0000","last_change":"2023-09-09T23:12:17.906676+0000","last_active":"2023-09-09T23:12:18.898148+0000","last_peered":"2023-09-09T23:12:18.898148+0000","last_clean":"2023-09-09T23:12:18.898148+0000","last_became_active":"2023-09-09T23:12:17.906381+0000","last_became_peered":"2023-09-09T23:12:17.906381+0000","last_unstale":"2023-09-09T23:12:18.898148+0000","last_undegraded":"2023-09-09T23:12:18.898148+0000","last_fullsized":"2023-09-09T23:12:18.898148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:42:28.033255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:23.271968+0000","last_change":"2023-09-09T23:12:17.166623+0000","last_active":"2023-09-09T23:12:23.271968+0000","last_peered":"2023-09-09T23:12:23.271968+0000","last_clean":"2023-09-09T23:12:23.271968+0000","last_became_active":"2023-09-09T23:12:17.166293+0000","last_became_peered":"2023-09-09T23:12:17.166293+0000","last_unstale":"2023-09-09T23:12:23.271968+0000","last_undegraded":"2023-09-09T23:12:23.271968+0000","last_fullsized":"2023-09-09T23:12:23.271968+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:52:51.267501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":317,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.963859+0000","last_change":"2023-09-09T23:11:20.948461+0000","last_active":"2023-09-09T23:12:19.963859+0000","last_peered":"2023-09-09T23:12:19.963859+0000","last_clean":"2023-09-09T23:12:19.963859+0000","last_became_active":"2023-09-09T23:11:20.947856+0000","last_became_peered":"2023-09-09T23:11:20.947856+0000","last_unstale":"2023-09-09T23:12:19.963859+0000","last_undegraded":"2023-09-09T23:12:19.963859+0000","last_fullsized":"2023-09-09T23:12:19.963859+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:08:31.737051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:08:31.737051+0000","last_clean_scrub_stamp":"2023-09-09T23:08:31.737051+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:20:16.293707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:17.908619+0000","last_change":"2023-09-09T23:12:17.907468+0000","last_active":"2023-09-09T23:12:17.908619+0000","last_peered":"2023-09-09T23:12:17.908619+0000","last_clean":"2023-09-09T23:12:17.908619+0000","last_became_active":"2023-09-09T23:12:17.907117+0000","last_became_peered":"2023-09-09T23:12:17.907117+0000","last_unstale":"2023-09-09T23:12:17.908619+0000","last_undegraded":"2023-09-09T23:12:17.908619+0000","last_fullsized":"2023-09-09T23:12:17.908619+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:56:38.086209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:22.583028+0000","last_change":"2023-09-09T23:12:17.171505+0000","last_active":"2023-09-09T23:12:22.583028+0000","last_peered":"2023-09-09T23:12:22.583028+0000","last_clean":"2023-09-09T23:12:22.583028+0000","last_became_active":"2023-09-09T23:12:17.171185+0000","last_became_peered":"2023-09-09T23:12:17.171185+0000","last_unstale":"2023-09-09T23:12:22.583028+0000","last_undegraded":"2023-09-09T23:12:22.583028+0000","last_fullsized":"2023-09-09T23:12:22.583028+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:07:54.271364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898098+0000","last_change":"2023-09-09T23:12:17.176450+0000","last_active":"2023-09-09T23:12:18.898098+0000","last_peered":"2023-09-09T23:12:18.898098+0000","last_clean":"2023-09-09T23:12:18.898098+0000","last_became_active":"2023-09-09T23:12:17.176217+0000","last_became_peered":"2023-09-09T23:12:17.176217+0000","last_unstale":"2023-09-09T23:12:18.898098+0000","last_undegraded":"2023-09-09T23:12:18.898098+0000","last_fullsized":"2023-09-09T23:12:18.898098+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:57:26.604261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:17.900130+0000","last_change":"2023-09-09T23:12:17.897612+0000","last_active":"2023-09-09T23:12:17.900130+0000","last_peered":"2023-09-09T23:12:17.900130+0000","last_clean":"2023-09-09T23:12:17.900130+0000","last_became_active":"2023-09-09T23:12:17.897292+0000","last_became_peered":"2023-09-09T23:12:17.897292+0000","last_unstale":"2023-09-09T23:12:17.900130+0000","last_undegraded":"2023-09-09T23:12:17.900130+0000","last_fullsized":"2023-09-09T23:12:17.900130+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:27:53.095238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:17.896214+0000","last_change":"2023-09-09T23:12:17.175682+0000","last_active":"2023-09-09T23:12:17.896214+0000","last_peered":"2023-09-09T23:12:17.896214+0000","last_clean":"2023-09-09T23:12:17.896214+0000","last_became_active":"2023-09-09T23:12:17.175291+0000","last_became_peered":"2023-09-09T23:12:17.175291+0000","last_unstale":"2023-09-09T23:12:17.896214+0000","last_undegraded":"2023-09-09T23:12:17.896214+0000","last_fullsized":"2023-09-09T23:12:17.896214+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:29:02.349319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.899164+0000","last_change":"2023-09-09T23:12:17.904153+0000","last_active":"2023-09-09T23:12:18.899164+0000","last_peered":"2023-09-09T23:12:18.899164+0000","last_clean":"2023-09-09T23:12:18.899164+0000","last_became_active":"2023-09-09T23:12:17.903804+0000","last_became_peered":"2023-09-09T23:12:17.903804+0000","last_unstale":"2023-09-09T23:12:18.899164+0000","last_undegraded":"2023-09-09T23:12:18.899164+0000","last_fullsized":"2023-09-09T23:12:18.899164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:39:34.046551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'5","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:21.575155+0000","last_change":"2023-09-09T23:12:17.169052+0000","last_active":"2023-09-09T23:12:21.575155+0000","last_peered":"2023-09-09T23:12:21.575155+0000","last_clean":"2023-09-09T23:12:21.575155+0000","last_became_active":"2023-09-09T23:12:17.168719+0000","last_became_peered":"2023-09-09T23:12:17.168719+0000","last_unstale":"2023-09-09T23:12:21.575155+0000","last_undegraded":"2023-09-09T23:12:21.575155+0000","last_fullsized":"2023-09-09T23:12:21.575155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:56:44.427954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.485050+0000","last_change":"2023-09-09T23:12:17.900945+0000","last_active":"2023-09-09T23:12:19.485050+0000","last_peered":"2023-09-09T23:12:19.485050+0000","last_clean":"2023-09-09T23:12:19.485050+0000","last_became_active":"2023-09-09T23:12:17.900602+0000","last_became_peered":"2023-09-09T23:12:17.900602+0000","last_unstale":"2023-09-09T23:12:19.485050+0000","last_undegraded":"2023-09-09T23:12:19.485050+0000","last_fullsized":"2023-09-09T23:12:19.485050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:22:10.336285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.519606+0000","last_change":"2023-09-09T23:12:17.166619+0000","last_active":"2023-09-09T23:12:19.519606+0000","last_peered":"2023-09-09T23:12:19.519606+0000","last_clean":"2023-09-09T23:12:19.519606+0000","last_became_active":"2023-09-09T23:12:17.166286+0000","last_became_peered":"2023-09-09T23:12:17.166286+0000","last_unstale":"2023-09-09T23:12:19.519606+0000","last_undegraded":"2023-09-09T23:12:19.519606+0000","last_fullsized":"2023-09-09T23:12:19.519606+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:41:40.713296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897742+0000","last_change":"2023-09-09T23:12:17.897131+0000","last_active":"2023-09-09T23:12:18.897742+0000","last_peered":"2023-09-09T23:12:18.897742+0000","last_clean":"2023-09-09T23:12:18.897742+0000","last_became_active":"2023-09-09T23:12:17.896873+0000","last_became_peered":"2023-09-09T23:12:17.896873+0000","last_unstale":"2023-09-09T23:12:18.897742+0000","last_undegraded":"2023-09-09T23:12:18.897742+0000","last_fullsized":"2023-09-09T23:12:18.897742+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:54:21.091418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:17.891648+0000","last_change":"2023-09-09T23:12:17.168868+0000","last_active":"2023-09-09T23:12:17.891648+0000","last_peered":"2023-09-09T23:12:17.891648+0000","last_clean":"2023-09-09T23:12:17.891648+0000","last_became_active":"2023-09-09T23:12:17.168572+0000","last_became_peered":"2023-09-09T23:12:17.168572+0000","last_unstale":"2023-09-09T23:12:17.891648+0000","last_undegraded":"2023-09-09T23:12:17.891648+0000","last_fullsized":"2023-09-09T23:12:17.891648+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:54:08.108023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.988326+0000","last_change":"2023-09-09T23:12:17.902887+0000","last_active":"2023-09-09T23:12:18.988326+0000","last_peered":"2023-09-09T23:12:18.988326+0000","last_clean":"2023-09-09T23:12:18.988326+0000","last_became_active":"2023-09-09T23:12:17.902507+0000","last_became_peered":"2023-09-09T23:12:17.902507+0000","last_unstale":"2023-09-09T23:12:18.988326+0000","last_undegraded":"2023-09-09T23:12:18.988326+0000","last_fullsized":"2023-09-09T23:12:18.988326+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:18:08.000184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:17.886594+0000","last_change":"2023-09-09T23:12:17.171379+0000","last_active":"2023-09-09T23:12:17.886594+0000","last_peered":"2023-09-09T23:12:17.886594+0000","last_clean":"2023-09-09T23:12:17.886594+0000","last_became_active":"2023-09-09T23:12:17.171024+0000","last_became_peered":"2023-09-09T23:12:17.171024+0000","last_unstale":"2023-09-09T23:12:17.886594+0000","last_undegraded":"2023-09-09T23:12:17.886594+0000","last_fullsized":"2023-09-09T23:12:17.886594+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:17:12.206768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898049+0000","last_change":"2023-09-09T23:12:17.905510+0000","last_active":"2023-09-09T23:12:18.898049+0000","last_peered":"2023-09-09T23:12:18.898049+0000","last_clean":"2023-09-09T23:12:18.898049+0000","last_became_active":"2023-09-09T23:12:17.905207+0000","last_became_peered":"2023-09-09T23:12:17.905207+0000","last_unstale":"2023-09-09T23:12:18.898049+0000","last_undegraded":"2023-09-09T23:12:18.898049+0000","last_fullsized":"2023-09-09T23:12:18.898049+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:48:20.167421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'2","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897752+0000","last_change":"2023-09-09T23:12:17.173764+0000","last_active":"2023-09-09T23:12:18.897752+0000","last_peered":"2023-09-09T23:12:18.897752+0000","last_clean":"2023-09-09T23:12:18.897752+0000","last_became_active":"2023-09-09T23:12:17.173470+0000","last_became_peered":"2023-09-09T23:12:17.173470+0000","last_unstale":"2023-09-09T23:12:18.897752+0000","last_undegraded":"2023-09-09T23:12:18.897752+0000","last_fullsized":"2023-09-09T23:12:18.897752+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:01:54.091664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690237+0000","last_change":"2023-09-09T23:12:17.906292+0000","last_active":"2023-09-09T23:12:20.690237+0000","last_peered":"2023-09-09T23:12:20.690237+0000","last_clean":"2023-09-09T23:12:20.690237+0000","last_became_active":"2023-09-09T23:12:17.905995+0000","last_became_peered":"2023-09-09T23:12:17.905995+0000","last_unstale":"2023-09-09T23:12:20.690237+0000","last_undegraded":"2023-09-09T23:12:20.690237+0000","last_fullsized":"2023-09-09T23:12:20.690237+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:52:29.972831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:17.886676+0000","last_change":"2023-09-09T23:12:17.173818+0000","last_active":"2023-09-09T23:12:17.886676+0000","last_peered":"2023-09-09T23:12:17.886676+0000","last_clean":"2023-09-09T23:12:17.886676+0000","last_became_active":"2023-09-09T23:12:17.173413+0000","last_became_peered":"2023-09-09T23:12:17.173413+0000","last_unstale":"2023-09-09T23:12:17.886676+0000","last_undegraded":"2023-09-09T23:12:17.886676+0000","last_fullsized":"2023-09-09T23:12:17.886676+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:55:28.856863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690266+0000","last_change":"2023-09-09T23:12:17.906833+0000","last_active":"2023-09-09T23:12:20.690266+0000","last_peered":"2023-09-09T23:12:20.690266+0000","last_clean":"2023-09-09T23:12:20.690266+0000","last_became_active":"2023-09-09T23:12:17.906138+0000","last_became_peered":"2023-09-09T23:12:17.906138+0000","last_unstale":"2023-09-09T23:12:20.690266+0000","last_undegraded":"2023-09-09T23:12:20.690266+0000","last_fullsized":"2023-09-09T23:12:20.690266+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:38:07.779167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:23.260259+0000","last_change":"2023-09-09T23:12:17.171338+0000","last_active":"2023-09-09T23:12:23.260259+0000","last_peered":"2023-09-09T23:12:23.260259+0000","last_clean":"2023-09-09T23:12:23.260259+0000","last_became_active":"2023-09-09T23:12:17.170613+0000","last_became_peered":"2023-09-09T23:12:17.170613+0000","last_unstale":"2023-09-09T23:12:23.260259+0000","last_undegraded":"2023-09-09T23:12:23.260259+0000","last_fullsized":"2023-09-09T23:12:23.260259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:24:07.794464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689730+0000","last_change":"2023-09-09T23:12:17.906470+0000","last_active":"2023-09-09T23:12:20.689730+0000","last_peered":"2023-09-09T23:12:20.689730+0000","last_clean":"2023-09-09T23:12:20.689730+0000","last_became_active":"2023-09-09T23:12:17.906170+0000","last_became_peered":"2023-09-09T23:12:17.906170+0000","last_unstale":"2023-09-09T23:12:20.689730+0000","last_undegraded":"2023-09-09T23:12:20.689730+0000","last_fullsized":"2023-09-09T23:12:20.689730+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:44:37.483541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.544399+0000","last_change":"2023-09-09T23:12:17.172142+0000","last_active":"2023-09-09T23:12:18.544399+0000","last_peered":"2023-09-09T23:12:18.544399+0000","last_clean":"2023-09-09T23:12:18.544399+0000","last_became_active":"2023-09-09T23:12:17.171703+0000","last_became_peered":"2023-09-09T23:12:17.171703+0000","last_unstale":"2023-09-09T23:12:18.544399+0000","last_undegraded":"2023-09-09T23:12:18.544399+0000","last_fullsized":"2023-09-09T23:12:18.544399+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:49:31.414385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689874+0000","last_change":"2023-09-09T23:12:17.900699+0000","last_active":"2023-09-09T23:12:20.689874+0000","last_peered":"2023-09-09T23:12:20.689874+0000","last_clean":"2023-09-09T23:12:20.689874+0000","last_became_active":"2023-09-09T23:12:17.900416+0000","last_became_peered":"2023-09-09T23:12:17.900416+0000","last_unstale":"2023-09-09T23:12:20.689874+0000","last_undegraded":"2023-09-09T23:12:20.689874+0000","last_fullsized":"2023-09-09T23:12:20.689874+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:54:29.328413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.988389+0000","last_change":"2023-09-09T23:12:17.900480+0000","last_active":"2023-09-09T23:12:18.988389+0000","last_peered":"2023-09-09T23:12:18.988389+0000","last_clean":"2023-09-09T23:12:18.988389+0000","last_became_active":"2023-09-09T23:12:17.900135+0000","last_became_peered":"2023-09-09T23:12:17.900135+0000","last_unstale":"2023-09-09T23:12:18.988389+0000","last_undegraded":"2023-09-09T23:12:18.988389+0000","last_fullsized":"2023-09-09T23:12:18.988389+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:25:44.254310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690446+0000","last_change":"2023-09-09T23:12:17.901302+0000","last_active":"2023-09-09T23:12:20.690446+0000","last_peered":"2023-09-09T23:12:20.690446+0000","last_clean":"2023-09-09T23:12:20.690446+0000","last_became_active":"2023-09-09T23:12:17.900982+0000","last_became_peered":"2023-09-09T23:12:17.900982+0000","last_unstale":"2023-09-09T23:12:20.690446+0000","last_undegraded":"2023-09-09T23:12:20.690446+0000","last_fullsized":"2023-09-09T23:12:20.690446+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:41:41.119614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689782+0000","last_change":"2023-09-09T23:12:17.904328+0000","last_active":"2023-09-09T23:12:20.689782+0000","last_peered":"2023-09-09T23:12:20.689782+0000","last_clean":"2023-09-09T23:12:20.689782+0000","last_became_active":"2023-09-09T23:12:17.904040+0000","last_became_peered":"2023-09-09T23:12:17.904040+0000","last_unstale":"2023-09-09T23:12:20.689782+0000","last_undegraded":"2023-09-09T23:12:20.689782+0000","last_fullsized":"2023-09-09T23:12:20.689782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:01:05.986097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.485015+0000","last_change":"2023-09-09T23:12:17.901737+0000","last_active":"2023-09-09T23:12:19.485015+0000","last_peered":"2023-09-09T23:12:19.485015+0000","last_clean":"2023-09-09T23:12:19.485015+0000","last_became_active":"2023-09-09T23:12:17.901452+0000","last_became_peered":"2023-09-09T23:12:17.901452+0000","last_unstale":"2023-09-09T23:12:19.485015+0000","last_undegraded":"2023-09-09T23:12:19.485015+0000","last_fullsized":"2023-09-09T23:12:19.485015+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:35:28.698304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:17.908698+0000","last_change":"2023-09-09T23:12:17.907433+0000","last_active":"2023-09-09T23:12:17.908698+0000","last_peered":"2023-09-09T23:12:17.908698+0000","last_clean":"2023-09-09T23:12:17.908698+0000","last_became_active":"2023-09-09T23:12:17.907127+0000","last_became_peered":"2023-09-09T23:12:17.907127+0000","last_unstale":"2023-09-09T23:12:17.908698+0000","last_undegraded":"2023-09-09T23:12:17.908698+0000","last_fullsized":"2023-09-09T23:12:17.908698+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:11:31.575509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.988586+0000","last_change":"2023-09-09T23:12:17.901373+0000","last_active":"2023-09-09T23:12:18.988586+0000","last_peered":"2023-09-09T23:12:18.988586+0000","last_clean":"2023-09-09T23:12:18.988586+0000","last_became_active":"2023-09-09T23:12:17.901080+0000","last_became_peered":"2023-09-09T23:12:17.901080+0000","last_unstale":"2023-09-09T23:12:18.988586+0000","last_undegraded":"2023-09-09T23:12:18.988586+0000","last_fullsized":"2023-09-09T23:12:18.988586+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:23:12.316659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690576+0000","last_change":"2023-09-09T23:12:17.896579+0000","last_active":"2023-09-09T23:12:20.690576+0000","last_peered":"2023-09-09T23:12:20.690576+0000","last_clean":"2023-09-09T23:12:20.690576+0000","last_became_active":"2023-09-09T23:12:17.896253+0000","last_became_peered":"2023-09-09T23:12:17.896253+0000","last_unstale":"2023-09-09T23:12:20.690576+0000","last_undegraded":"2023-09-09T23:12:20.690576+0000","last_fullsized":"2023-09-09T23:12:20.690576+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:30:20.249103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.899186+0000","last_change":"2023-09-09T23:12:17.903569+0000","last_active":"2023-09-09T23:12:18.899186+0000","last_peered":"2023-09-09T23:12:18.899186+0000","last_clean":"2023-09-09T23:12:18.899186+0000","last_became_active":"2023-09-09T23:12:17.903027+0000","last_became_peered":"2023-09-09T23:12:17.903027+0000","last_unstale":"2023-09-09T23:12:18.899186+0000","last_undegraded":"2023-09-09T23:12:18.899186+0000","last_fullsized":"2023-09-09T23:12:18.899186+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:38:46.392973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689604+0000","last_change":"2023-09-09T23:12:17.902944+0000","last_active":"2023-09-09T23:12:20.689604+0000","last_peered":"2023-09-09T23:12:20.689604+0000","last_clean":"2023-09-09T23:12:20.689604+0000","last_became_active":"2023-09-09T23:12:17.901812+0000","last_became_peered":"2023-09-09T23:12:17.901812+0000","last_unstale":"2023-09-09T23:12:20.689604+0000","last_undegraded":"2023-09-09T23:12:20.689604+0000","last_fullsized":"2023-09-09T23:12:20.689604+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:49:50.981693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897608+0000","last_change":"2023-09-09T23:12:17.897612+0000","last_active":"2023-09-09T23:12:18.897608+0000","last_peered":"2023-09-09T23:12:18.897608+0000","last_clean":"2023-09-09T23:12:18.897608+0000","last_became_active":"2023-09-09T23:12:17.897318+0000","last_became_peered":"2023-09-09T23:12:17.897318+0000","last_unstale":"2023-09-09T23:12:18.897608+0000","last_undegraded":"2023-09-09T23:12:18.897608+0000","last_fullsized":"2023-09-09T23:12:18.897608+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:17:38.687470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.519770+0000","last_change":"2023-09-09T23:12:17.903849+0000","last_active":"2023-09-09T23:12:19.519770+0000","last_peered":"2023-09-09T23:12:19.519770+0000","last_clean":"2023-09-09T23:12:19.519770+0000","last_became_active":"2023-09-09T23:12:17.903534+0000","last_became_peered":"2023-09-09T23:12:17.903534+0000","last_unstale":"2023-09-09T23:12:19.519770+0000","last_undegraded":"2023-09-09T23:12:19.519770+0000","last_fullsized":"2023-09-09T23:12:19.519770+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:06:18.073811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:17.906425+0000","last_change":"2023-09-09T23:12:17.899782+0000","last_active":"2023-09-09T23:12:17.906425+0000","last_peered":"2023-09-09T23:12:17.906425+0000","last_clean":"2023-09-09T23:12:17.906425+0000","last_became_active":"2023-09-09T23:12:17.899445+0000","last_became_peered":"2023-09-09T23:12:17.899445+0000","last_unstale":"2023-09-09T23:12:17.906425+0000","last_undegraded":"2023-09-09T23:12:17.906425+0000","last_fullsized":"2023-09-09T23:12:17.906425+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:20:44.292143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.988408+0000","last_change":"2023-09-09T23:12:17.898693+0000","last_active":"2023-09-09T23:12:18.988408+0000","last_peered":"2023-09-09T23:12:18.988408+0000","last_clean":"2023-09-09T23:12:18.988408+0000","last_became_active":"2023-09-09T23:12:17.898343+0000","last_became_peered":"2023-09-09T23:12:17.898343+0000","last_unstale":"2023-09-09T23:12:18.988408+0000","last_undegraded":"2023-09-09T23:12:18.988408+0000","last_fullsized":"2023-09-09T23:12:18.988408+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:37:43.229953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5998,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":94,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":17714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514507,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":416035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]}]},{"osd":10,"up_from":69,"seq":296352743439,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28472,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716680,"statfs":{"total":95995035648,"available":95965880320,"internally_reserved":0,"allocated":1486848,"data_stored":1008209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":1,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.308}]},{"osd":2,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.442}]},{"osd":3,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.45400000000000001}]},{"osd":4,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.33600000000000002}]},{"osd":5,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.628},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.38600000000000001}]},{"osd":6,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":8,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.56499999999999995}]},{"osd":9,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.59399999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]}]},{"osd":9,"up_from":64,"seq":274877906962,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":416617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:11:55 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.35399999999999998}]},{"osd":1,"last update":"Sat Sep 9 23:11:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.60899999999999999}]},{"osd":2,"last update":"Sat Sep 9 23:11:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.51900000000000002}]},{"osd":3,"last update":"Sat Sep 9 23:11:55 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.432},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.34100000000000003}]},{"osd":4,"last update":"Sat Sep 9 23:11:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.57799999999999996}]},{"osd":5,"last update":"Sat Sep 9 23:11:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.504},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.59399999999999997}]},{"osd":6,"last update":"Sat Sep 9 23:11:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.38},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.46100000000000002}]},{"osd":7,"last update":"Sat Sep 9 23:11:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.628},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.53800000000000003}]},{"osd":8,"last update":"Sat Sep 9 23:11:55 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.66400000000000003}]},{"osd":10,"last update":"Sat Sep 9 23:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.69199999999999995}]}]},{"osd":8,"up_from":57,"seq":244813135896,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":876544,"data_stored":416639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.38},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.52600000000000002}]},{"osd":1,"last update":"Sat Sep 9 23:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.32900000000000001}]},{"osd":2,"last update":"Sat Sep 9 23:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.42999999999999999}]},{"osd":3,"last update":"Sat Sep 9 23:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.504}]},{"osd":4,"last update":"Sat Sep 9 23:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.41899999999999998}]},{"osd":5,"last update":"Sat Sep 9 23:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.495}]},{"osd":6,"last update":"Sat Sep 9 23:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.55200000000000005}]},{"osd":7,"last update":"Sat Sep 9 23:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.60699999999999998}]},{"osd":9,"last update":"Sat Sep 9 23:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.56299999999999994}]},{"osd":10,"last update":"Sat Sep 9 23:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.59399999999999997}]}]},{"osd":7,"up_from":50,"seq":214748364828,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28464,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716688,"statfs":{"total":95995035648,"available":95965888512,"internally_reserved":0,"allocated":1482752,"data_stored":1008926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.439},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.33100000000000002}]},{"osd":1,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.435,"15min":0.432},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.60999999999999999,"5min":0.625,"15min":0.625},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.42699999999999999,"15min":0.41699999999999998},"min":{"1min":0.26200000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.48499999999999999}]},{"osd":2,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.77200000000000002,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.63700000000000001}]},{"osd":3,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.40600000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.626,"15min":0.625},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.79800000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.68300000000000005}]},{"osd":4,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52900000000000003,"15min":0.52300000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.76400000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.314,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.52100000000000002}]},{"osd":5,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.69399999999999995,"15min":0.69599999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86399999999999999,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.66900000000000004,"15min":0.66800000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.69999999999999996}]},{"osd":6,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.746,"15min":0.753},"min":{"1min":0.48899999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90900000000000003,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.753},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.72599999999999998,"15min":0.72899999999999998},"min":{"1min":0.46800000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92600000000000005,"5min":1.254,"15min":1.254},"last":0.65500000000000003}]},{"osd":8,"last update":"Sat Sep 9 23:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.86499999999999999}]},{"osd":9,"last update":"Sat Sep 9 23:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.85299999999999998}]},{"osd":10,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.88200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]}]},{"osd":6,"up_from":45,"seq":193273528352,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":856064,"data_stored":407465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.375,"15min":0.36799999999999999},"min":{"1min":0.27800000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.41199999999999998,"15min":0.40400000000000003},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.46700000000000003}]},{"osd":1,"last update":"Sat Sep 9 23:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.57199999999999995,"15min":0.56100000000000005},"min":{"1min":0.48199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.57999999999999996,"15min":0.56699999999999995},"min":{"1min":0.42699999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.56999999999999995}]},{"osd":2,"last update":"Sat Sep 9 23:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.621,"15min":0.61299999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.70199999999999996}]},{"osd":3,"last update":"Sat Sep 9 23:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.438,"15min":0.435},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.81599999999999995,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.55100000000000005}]},{"osd":4,"last update":"Sat Sep 9 23:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.443,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.96899999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.58999999999999997,"15min":0.57999999999999996},"min":{"1min":0.46700000000000003,"5min":0.437,"15min":0.437},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.71799999999999997}]},{"osd":5,"last update":"Sat Sep 9 23:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0289999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67000000000000004,"15min":0.66100000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.72699999999999998}]},{"osd":7,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93400000000000005,"5min":1.032,"15min":1.032},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.81899999999999995}]},{"osd":8,"last update":"Sat Sep 9 23:11:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.73999999999999999}]},{"osd":9,"last update":"Sat Sep 9 23:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.68700000000000006}]},{"osd":10,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.79000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":880640,"data_stored":417647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.51600000000000001,"15min":0.498},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.51400000000000001,"15min":0.48999999999999999},"min":{"1min":0.35099999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.47699999999999998}]},{"osd":2,"last update":"Sat Sep 9 23:11:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.51300000000000001,"15min":0.499},"min":{"1min":0.33400000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.53000000000000003,"15min":0.51400000000000001},"min":{"1min":0.39100000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.80600000000000005}]},{"osd":3,"last update":"Sat Sep 9 23:11:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.47299999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.60599999999999998}]},{"osd":4,"last update":"Sat Sep 9 23:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.438,"15min":0.41399999999999998},"min":{"1min":0.30499999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.48599999999999999,"15min":0.44800000000000001},"min":{"1min":0.28499999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.56599999999999995}]},{"osd":5,"last update":"Sat Sep 9 23:11:34 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.38600000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.57999999999999996,"15min":0.56599999999999995},"min":{"1min":0.43099999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.86399999999999999}]},{"osd":6,"last update":"Sat Sep 9 23:11:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.58499999999999996,"15min":0.56499999999999995},"min":{"1min":0.45300000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.40699999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.58699999999999997}]},{"osd":7,"last update":"Sat Sep 9 23:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.53100000000000003,"15min":0.51400000000000001},"min":{"1min":0.376,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52800000000000002,"15min":0.52000000000000002},"min":{"1min":0.33300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.54900000000000004}]},{"osd":8,"last update":"Sat Sep 9 23:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.81799999999999995}]},{"osd":9,"last update":"Sat Sep 9 23:11:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.749},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.65600000000000003}]},{"osd":10,"last update":"Sat Sep 9 23:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.629}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738425,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":418213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 9 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.53100000000000003,"15min":0.51100000000000001},"min":{"1min":0.39700000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.78000000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51400000000000001,"15min":0.5},"min":{"1min":0.374,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71799999999999997,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.442}]},{"osd":2,"last update":"Sat Sep 9 23:11:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53600000000000003,"15min":0.52800000000000002},"min":{"1min":0.42399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.53700000000000003,"15min":0.52400000000000002},"min":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.62}]},{"osd":3,"last update":"Sat Sep 9 23:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.39300000000000002,"15min":0.38900000000000001},"min":{"1min":0.27900000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.41599999999999998,"15min":0.40999999999999998},"min":{"1min":0.33500000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.35999999999999999}]},{"osd":4,"last update":"Sat Sep 9 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.54900000000000004,"15min":0.53300000000000003},"min":{"1min":0.42899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82999999999999996,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.497},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.375,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.39400000000000002}]},{"osd":5,"last update":"Sat Sep 9 23:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58699999999999997,"15min":0.58299999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56699999999999995,"15min":0.56499999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.63}]},{"osd":6,"last update":"Sat Sep 9 23:11:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51100000000000001,"15min":0.502},"min":{"1min":0.34300000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.51600000000000001,"15min":0.505},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.47199999999999998}]},{"osd":7,"last update":"Sat Sep 9 23:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.47599999999999998}]},{"osd":8,"last update":"Sat Sep 9 23:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.53200000000000003}]},{"osd":9,"last update":"Sat Sep 9 23:11:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.54300000000000004}]},{"osd":10,"last update":"Sat Sep 9 23:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.51300000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411377,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":884736,"data_stored":417681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52000000000000002,"15min":0.53100000000000003},"min":{"1min":0.32600000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81299999999999994,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.51200000000000001,"15min":0.51400000000000001},"min":{"1min":0.36599999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77600000000000002,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.39300000000000002}]},{"osd":1,"last update":"Sat Sep 9 23:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54400000000000004,"15min":0.53200000000000003},"min":{"1min":0.378,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77600000000000002,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.53100000000000003,"15min":0.50900000000000001},"min":{"1min":0.38,"5min":0.31,"15min":0.31},"max":{"1min":0.82199999999999995,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.53000000000000003}]},{"osd":3,"last update":"Sat Sep 9 23:11:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60299999999999998,"15min":0.60799999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85399999999999998,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.60599999999999998,"15min":0.61399999999999999},"min":{"1min":0.41699999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.86499999999999999,"5min":1.478,"15min":1.478},"last":0.55300000000000005}]},{"osd":4,"last update":"Sat Sep 9 23:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57299999999999995,"15min":0.57199999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.78000000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.626},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.85899999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.49099999999999999}]},{"osd":5,"last update":"Sat Sep 9 23:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.40600000000000003,"15min":0.39800000000000002},"min":{"1min":0.30199999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.505},{"interface":"front","average":{"1min":0.44,"5min":0.41899999999999998,"15min":0.41499999999999998},"min":{"1min":0.317,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.35899999999999999}]},{"osd":6,"last update":"Sat Sep 9 23:11:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.54300000000000004,"15min":0.52300000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.59899999999999998,"15min":0.58799999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.58699999999999997}]},{"osd":7,"last update":"Sat Sep 9 23:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.65700000000000003}]},{"osd":8,"last update":"Sat Sep 9 23:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.45400000000000001}]},{"osd":9,"last update":"Sat Sep 9 23:11:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.66000000000000003}]},{"osd":10,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.64000000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]}]},{"osd":3,"up_from":25,"seq":107374182444,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":872448,"data_stored":415808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.32700000000000001,"15min":0.313},"min":{"1min":0.28999999999999998,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.31},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.34899999999999998,"15min":0.33300000000000002},"min":{"1min":0.28699999999999998,"5min":0.222,"15min":0.222},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.28699999999999998}]},{"osd":1,"last update":"Sat Sep 9 23:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.53800000000000003,"15min":0.51600000000000001},"min":{"1min":0.39600000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.95799999999999996,"5min":1.091,"15min":1.091},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.34699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71099999999999997,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.48899999999999999}]},{"osd":2,"last update":"Sat Sep 9 23:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.53900000000000003,"15min":0.51900000000000002},"min":{"1min":0.441,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.75900000000000001,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.626},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53700000000000003,"15min":0.51700000000000002},"min":{"1min":0.44900000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.72799999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.58499999999999996}]},{"osd":4,"last update":"Sat Sep 9 23:11:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.51500000000000001,"15min":0.498},"min":{"1min":0.47299999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.52500000000000002,"15min":0.51300000000000001},"min":{"1min":0.41199999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.61299999999999999}]},{"osd":5,"last update":"Sat Sep 9 23:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.58899999999999997,"15min":0.58699999999999997},"min":{"1min":0.46200000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83699999999999997,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.61799999999999999,"15min":0.60599999999999998},"min":{"1min":0.51600000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95999999999999996,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.64800000000000002}]},{"osd":6,"last update":"Sat Sep 9 23:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.502,"15min":0.496},"min":{"1min":0.34899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.377},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.51800000000000002,"15min":0.50900000000000001},"min":{"1min":0.377,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.39500000000000002}]},{"osd":7,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66600000000000004,"15min":0.66400000000000003},"min":{"1min":0.45400000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69299999999999995,"15min":0.69799999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.501}]},{"osd":8,"last update":"Sat Sep 9 23:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.49199999999999999}]},{"osd":9,"last update":"Sat Sep 9 23:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.44},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.55600000000000005}]},{"osd":10,"last update":"Sat Sep 9 23:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.56899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]}]},{"osd":4,"up_from":32,"seq":137438953513,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":880640,"data_stored":416679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.503,"15min":0.47099999999999997},"min":{"1min":0.44900000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.52200000000000002,"15min":0.48799999999999999},"min":{"1min":0.41999999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.33300000000000002}]},{"osd":1,"last update":"Sat Sep 9 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.35899999999999999,"15min":0.33800000000000002},"min":{"1min":0.26100000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.52300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.38100000000000001,"15min":0.35699999999999998},"min":{"1min":0.27400000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.68999999999999995,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.48499999999999999}]},{"osd":2,"last update":"Sat Sep 9 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53000000000000003,"15min":0.51400000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.69399999999999995,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.54700000000000004,"15min":0.52200000000000002},"min":{"1min":0.377,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79700000000000004,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.78900000000000003}]},{"osd":3,"last update":"Sat Sep 9 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.52300000000000002,"15min":0.48599999999999999},"min":{"1min":0.47199999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.51700000000000002,"15min":0.49299999999999999},"min":{"1min":0.39200000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.45800000000000002}]},{"osd":5,"last update":"Sat Sep 9 23:11:33 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.873,"5min":0.879,"15min":0.879},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.56899999999999995,"15min":0.55100000000000005},"min":{"1min":0.495,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.90100000000000002}]},{"osd":6,"last update":"Sat Sep 9 23:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.63800000000000001,"15min":0.61499999999999999},"min":{"1min":0.56200000000000006,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.63500000000000001,"15min":0.61299999999999999},"min":{"1min":0.48999999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.56200000000000006}]},{"osd":7,"last update":"Sat Sep 9 23:12:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.83799999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.36699999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72399999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.51600000000000001}]},{"osd":8,"last update":"Sat Sep 9 23:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.81000000000000005}]},{"osd":9,"last update":"Sat Sep 9 23:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.69999999999999996}]},{"osd":10,"last update":"Sat Sep 9 23:12:20 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.57899999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]},{"osd":5,"up_from":37,"seq":158913789988,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28484,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716668,"statfs":{"total":95995035648,"available":95965868032,"internally_reserved":0,"allocated":1503232,"data_stored":1008641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.46800000000000003,"15min":0.45800000000000002},"min":{"1min":0.39100000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.70699999999999996,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.49299999999999999,"15min":0.48999999999999999},"min":{"1min":0.39200000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.60799999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.53400000000000003}]},{"osd":1,"last update":"Sat Sep 9 23:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.504,"15min":0.501},"min":{"1min":0.34599999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.50700000000000001,"15min":0.50600000000000001},"min":{"1min":0.36699999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.621}]},{"osd":2,"last update":"Sat Sep 9 23:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.374,"15min":0.36899999999999999},"min":{"1min":0.28999999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.28799999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.44800000000000001}]},{"osd":3,"last update":"Sat Sep 9 23:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.39400000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76100000000000001,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.75},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.52300000000000002,"15min":0.51700000000000002},"min":{"1min":0.34599999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.68500000000000005}]},{"osd":4,"last update":"Sat Sep 9 23:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.52800000000000002,"15min":0.52200000000000002},"min":{"1min":0.42099999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72999999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54000000000000004,"15min":0.53700000000000003},"min":{"1min":0.38800000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.51100000000000001}]},{"osd":6,"last update":"Sat Sep 9 23:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.54900000000000004,"15min":0.54300000000000004},"min":{"1min":0.41199999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.73099999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.59599999999999997,"15min":0.58799999999999997},"min":{"1min":0.46600000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.75900000000000001,"5min":0.998,"15min":0.998},"last":0.78000000000000003}]},{"osd":7,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.59599999999999997,"15min":0.58799999999999997},"min":{"1min":0.498,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79200000000000004,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82199999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.66300000000000003}]},{"osd":8,"last update":"Sat Sep 9 23:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.64200000000000002}]},{"osd":9,"last update":"Sat Sep 9 23:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.83299999999999996}]},{"osd":10,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.69699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-09T23:12:26.549 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-09T23:12:26.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:26 smithi170 ceph-mon[102832]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:12:26.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:26 smithi170 ceph-mon[102832]: pgmap v259: 97 pgs: 97 active+clean; 582 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 4.7 KiB/s wr, 13 op/s 2023-09-09T23:12:26.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:26 smithi170 ceph-mon[102832]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:12:26.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:26 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] up:active 2023-09-09T23:12:26.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2023-09-09T23:12:26.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:26 smithi170 ceph-mon[102832]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-09-09T23:12:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:12:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:26 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2972752265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:12:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:26 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/764753975' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:12:26.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.701+0000 7f1d0a01d700 1 -- 172.21.15.82:0/2346845788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d040ff7b0 msgr2=0x7f1d040ffc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:26.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.701+0000 7f1d0a01d700 1 --2- 172.21.15.82:0/2346845788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d040ff7b0 0x7f1d040ffc10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4009990 tx=0x7f1cf4009ca0 comp rx=0 tx=0).stop 2023-09-09T23:12:26.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.702+0000 7f1d0a01d700 1 -- 172.21.15.82:0/2346845788 shutdown_connections 2023-09-09T23:12:26.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.702+0000 7f1d0a01d700 1 --2- 172.21.15.82:0/2346845788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d04100150 0x7f1d04107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:26.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.702+0000 7f1d0a01d700 1 --2- 172.21.15.82:0/2346845788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d04100a80 0x7f1d04100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:26.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.702+0000 7f1d0a01d700 1 --2- 172.21.15.82:0/2346845788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d040ff7b0 0x7f1d040ffc10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:26.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.702+0000 7f1d0a01d700 1 -- 172.21.15.82:0/2346845788 >> 172.21.15.82:0/2346845788 conn(0x7f1d040fa1c0 msgr2=0x7f1d040fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:26.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.702+0000 7f1d0a01d700 1 -- 172.21.15.82:0/2346845788 shutdown_connections 2023-09-09T23:12:26.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.702+0000 7f1d0a01d700 1 -- 172.21.15.82:0/2346845788 wait complete. 2023-09-09T23:12:26.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.702+0000 7f1d0a01d700 1 Processor -- start 2023-09-09T23:12:26.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.702+0000 7f1d0a01d700 1 -- start start 2023-09-09T23:12:26.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.702+0000 7f1d0a01d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d040ff7b0 0x7f1d04196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:26.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.702+0000 7f1d0a01d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d04100150 0x7f1d041967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:26.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.702+0000 7f1d0a01d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d04100a80 0x7f1d0419d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:26.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.702+0000 7f1d0a01d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0419de10 con 0x7f1d04100a80 2023-09-09T23:12:26.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.702+0000 7f1d0a01d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0419df50 con 0x7f1d04100150 2023-09-09T23:12:26.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.702+0000 7f1d0a01d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0419e0e0 con 0x7f1d040ff7b0 2023-09-09T23:12:26.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.703+0000 7f1d03fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d04100a80 0x7f1d0419d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:26.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.703+0000 7f1d03fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d04100a80 0x7f1d0419d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54656/0 (socket says 172.21.15.82:54656) 2023-09-09T23:12:26.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.703+0000 7f1d03fff700 1 -- 172.21.15.82:0/2636558928 learned_addr learned my addr 172.21.15.82:0/2636558928 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:26.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.703+0000 7f1d037fe700 1 --2- 172.21.15.82:0/2636558928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d040ff7b0 0x7f1d04196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:26.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.703+0000 7f1d02ffd700 1 --2- 172.21.15.82:0/2636558928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d04100150 0x7f1d041967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:26.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.703+0000 7f1d03fff700 1 -- 172.21.15.82:0/2636558928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d040ff7b0 msgr2=0x7f1d04196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:26.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.703+0000 7f1d03fff700 1 --2- 172.21.15.82:0/2636558928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d040ff7b0 0x7f1d04196290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:26.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.703+0000 7f1d03fff700 1 -- 172.21.15.82:0/2636558928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d04100150 msgr2=0x7f1d041967d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:26.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.703+0000 7f1d03fff700 1 --2- 172.21.15.82:0/2636558928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d04100150 0x7f1d041967d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:26.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.703+0000 7f1d03fff700 1 -- 172.21.15.82:0/2636558928 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf40095f0 con 0x7f1d04100a80 2023-09-09T23:12:26.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.703+0000 7f1d02ffd700 1 --2- 172.21.15.82:0/2636558928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d04100150 0x7f1d041967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:12:26.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.703+0000 7f1d03fff700 1 --2- 172.21.15.82:0/2636558928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d04100a80 0x7f1d0419d840 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc007d70 tx=0x7f1cfc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:26.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.703+0000 7f1d00ff9700 1 -- 172.21.15.82:0/2636558928 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc00ad40 con 0x7f1d04100a80 2023-09-09T23:12:26.709 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.703+0000 7f1d00ff9700 1 -- 172.21.15.82:0/2636558928 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cfc0128e0 con 0x7f1d04100a80 2023-09-09T23:12:26.709 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.703+0000 7f1d0a01d700 1 -- 172.21.15.82:0/2636558928 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d0419e3c0 con 0x7f1d04100a80 2023-09-09T23:12:26.709 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.703+0000 7f1d00ff9700 1 -- 172.21.15.82:0/2636558928 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc0116b0 con 0x7f1d04100a80 2023-09-09T23:12:26.709 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.703+0000 7f1d0a01d700 1 -- 172.21.15.82:0/2636558928 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d0419e880 con 0x7f1d04100a80 2023-09-09T23:12:26.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.704+0000 7f1d0a01d700 1 -- 172.21.15.82:0/2636558928 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d04107610 con 0x7f1d04100a80 2023-09-09T23:12:26.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.705+0000 7f1d00ff9700 1 -- 172.21.15.82:0/2636558928 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f1cfc012420 con 0x7f1d04100a80 2023-09-09T23:12:26.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.706+0000 7f1d00ff9700 1 --2- 172.21.15.82:0/2636558928 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1cf00a0790 0x7f1cf00a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:26.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.706+0000 7f1d037fe700 1 --2- 172.21.15.82:0/2636558928 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1cf00a0790 0x7f1cf00a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:26.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.706+0000 7f1d00ff9700 1 -- 172.21.15.82:0/2636558928 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1cfc0cbdb0 con 0x7f1d04100a80 2023-09-09T23:12:26.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.706+0000 7f1d037fe700 1 --2- 172.21.15.82:0/2636558928 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1cf00a0790 0x7f1cf00a2c50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1d040674e0 tx=0x7f1cec009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:26.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.710+0000 7f1d00ff9700 1 -- 172.21.15.82:0/2636558928 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1cfc095940 con 0x7f1d04100a80 2023-09-09T23:12:26.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:26 smithi098 ceph-mon[103124]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:12:26.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:26 smithi098 ceph-mon[103124]: pgmap v259: 97 pgs: 97 active+clean; 582 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 4.7 KiB/s wr, 13 op/s 2023-09-09T23:12:26.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:26 smithi098 ceph-mon[103124]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:12:26.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:26 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] up:active 2023-09-09T23:12:26.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2023-09-09T23:12:26.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:26 smithi098 ceph-mon[103124]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-09-09T23:12:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:12:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:26 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2972752265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:12:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:26 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/764753975' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:12:26.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.896+0000 7f1d0a01d700 1 -- 172.21.15.82:0/2636558928 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1d0402cf80 con 0x7f1cf00a0790 2023-09-09T23:12:26.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.906+0000 7f1d00ff9700 1 -- 172.21.15.82:0/2636558928 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335938 (secure 0 0 0) 0x7f1d0402cf80 con 0x7f1cf00a0790 2023-09-09T23:12:26.908 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:12:26.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.909+0000 7f1d0a01d700 1 -- 172.21.15.82:0/2636558928 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1cf00a0790 msgr2=0x7f1cf00a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:26.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.909+0000 7f1d0a01d700 1 --2- 172.21.15.82:0/2636558928 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1cf00a0790 0x7f1cf00a2c50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1d040674e0 tx=0x7f1cec009190 comp rx=0 tx=0).stop 2023-09-09T23:12:26.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.909+0000 7f1d0a01d700 1 -- 172.21.15.82:0/2636558928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d04100a80 msgr2=0x7f1d0419d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:26.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.909+0000 7f1d0a01d700 1 --2- 172.21.15.82:0/2636558928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d04100a80 0x7f1d0419d840 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc007d70 tx=0x7f1cfc00a340 comp rx=0 tx=0).stop 2023-09-09T23:12:26.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.909+0000 7f1d0a01d700 1 -- 172.21.15.82:0/2636558928 shutdown_connections 2023-09-09T23:12:26.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.909+0000 7f1d0a01d700 1 --2- 172.21.15.82:0/2636558928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d04100a80 0x7f1d0419d840 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:26.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.909+0000 7f1d0a01d700 1 --2- 172.21.15.82:0/2636558928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d040ff7b0 0x7f1d04196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:26.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.909+0000 7f1d0a01d700 1 --2- 172.21.15.82:0/2636558928 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1cf00a0790 0x7f1cf00a2c50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:26.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.909+0000 7f1d0a01d700 1 --2- 172.21.15.82:0/2636558928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d04100150 0x7f1d041967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:26.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.909+0000 7f1d0a01d700 1 -- 172.21.15.82:0/2636558928 >> 172.21.15.82:0/2636558928 conn(0x7f1d040fa1c0 msgr2=0x7f1d040fc370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:26.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.909+0000 7f1d0a01d700 1 -- 172.21.15.82:0/2636558928 shutdown_connections 2023-09-09T23:12:26.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:26.909+0000 7f1d0a01d700 1 -- 172.21.15.82:0/2636558928 wait complete. 2023-09-09T23:12:26.913 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-09-09T23:12:26.920 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2023-09-09T23:12:24.979498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596366,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":148,"num_read_kb":129,"num_write":275,"num_write_kb":2975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":275,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336328,"kb_used_data":12084,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124605496,"statfs":{"total":1151940427776,"available":1151596027904,"internally_reserved":0,"allocated":12374016,"data_stored":6768560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5998,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":94,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.823662"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.988649+0000","last_change":"2023-09-09T23:12:17.903811+0000","last_active":"2023-09-09T23:12:18.988649+0000","last_peered":"2023-09-09T23:12:18.988649+0000","last_clean":"2023-09-09T23:12:18.988649+0000","last_became_active":"2023-09-09T23:12:17.903386+0000","last_became_peered":"2023-09-09T23:12:17.903386+0000","last_unstale":"2023-09-09T23:12:18.988649+0000","last_undegraded":"2023-09-09T23:12:18.988649+0000","last_fullsized":"2023-09-09T23:12:18.988649+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:36:45.122068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.485007+0000","last_change":"2023-09-09T23:12:17.897915+0000","last_active":"2023-09-09T23:12:19.485007+0000","last_peered":"2023-09-09T23:12:19.485007+0000","last_clean":"2023-09-09T23:12:19.485007+0000","last_became_active":"2023-09-09T23:12:17.896727+0000","last_became_peered":"2023-09-09T23:12:17.896727+0000","last_unstale":"2023-09-09T23:12:19.485007+0000","last_undegraded":"2023-09-09T23:12:19.485007+0000","last_fullsized":"2023-09-09T23:12:19.485007+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:36:03.053572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.485083+0000","last_change":"2023-09-09T23:12:17.907957+0000","last_active":"2023-09-09T23:12:19.485083+0000","last_peered":"2023-09-09T23:12:19.485083+0000","last_clean":"2023-09-09T23:12:19.485083+0000","last_became_active":"2023-09-09T23:12:17.907625+0000","last_became_peered":"2023-09-09T23:12:17.907625+0000","last_unstale":"2023-09-09T23:12:19.485083+0000","last_undegraded":"2023-09-09T23:12:19.485083+0000","last_fullsized":"2023-09-09T23:12:19.485083+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:26:56.520987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.045364+0000","last_change":"2023-09-09T23:12:17.903489+0000","last_active":"2023-09-09T23:12:20.045364+0000","last_peered":"2023-09-09T23:12:20.045364+0000","last_clean":"2023-09-09T23:12:20.045364+0000","last_became_active":"2023-09-09T23:12:17.901184+0000","last_became_peered":"2023-09-09T23:12:17.901184+0000","last_unstale":"2023-09-09T23:12:20.045364+0000","last_undegraded":"2023-09-09T23:12:20.045364+0000","last_fullsized":"2023-09-09T23:12:20.045364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:42:01.899998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897810+0000","last_change":"2023-09-09T23:12:17.899487+0000","last_active":"2023-09-09T23:12:18.897810+0000","last_peered":"2023-09-09T23:12:18.897810+0000","last_clean":"2023-09-09T23:12:18.897810+0000","last_became_active":"2023-09-09T23:12:17.899083+0000","last_became_peered":"2023-09-09T23:12:17.899083+0000","last_unstale":"2023-09-09T23:12:18.897810+0000","last_undegraded":"2023-09-09T23:12:18.897810+0000","last_fullsized":"2023-09-09T23:12:18.897810+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:15:20.109849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690024+0000","last_change":"2023-09-09T23:12:17.901554+0000","last_active":"2023-09-09T23:12:20.690024+0000","last_peered":"2023-09-09T23:12:20.690024+0000","last_clean":"2023-09-09T23:12:20.690024+0000","last_became_active":"2023-09-09T23:12:17.901246+0000","last_became_peered":"2023-09-09T23:12:17.901246+0000","last_unstale":"2023-09-09T23:12:20.690024+0000","last_undegraded":"2023-09-09T23:12:20.690024+0000","last_fullsized":"2023-09-09T23:12:20.690024+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:19:08.795927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689639+0000","last_change":"2023-09-09T23:12:17.906989+0000","last_active":"2023-09-09T23:12:20.689639+0000","last_peered":"2023-09-09T23:12:20.689639+0000","last_clean":"2023-09-09T23:12:20.689639+0000","last_became_active":"2023-09-09T23:12:17.906704+0000","last_became_peered":"2023-09-09T23:12:17.906704+0000","last_unstale":"2023-09-09T23:12:20.689639+0000","last_undegraded":"2023-09-09T23:12:20.689639+0000","last_fullsized":"2023-09-09T23:12:20.689639+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:41:33.612168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897513+0000","last_change":"2023-09-09T23:12:17.906269+0000","last_active":"2023-09-09T23:12:18.897513+0000","last_peered":"2023-09-09T23:12:18.897513+0000","last_clean":"2023-09-09T23:12:18.897513+0000","last_became_active":"2023-09-09T23:12:17.906017+0000","last_became_peered":"2023-09-09T23:12:17.906017+0000","last_unstale":"2023-09-09T23:12:18.897513+0000","last_undegraded":"2023-09-09T23:12:18.897513+0000","last_fullsized":"2023-09-09T23:12:18.897513+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:50:21.893506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.520143+0000","last_change":"2023-09-09T23:12:17.905191+0000","last_active":"2023-09-09T23:12:19.520143+0000","last_peered":"2023-09-09T23:12:19.520143+0000","last_clean":"2023-09-09T23:12:19.520143+0000","last_became_active":"2023-09-09T23:12:17.904813+0000","last_became_peered":"2023-09-09T23:12:17.904813+0000","last_unstale":"2023-09-09T23:12:19.520143+0000","last_undegraded":"2023-09-09T23:12:19.520143+0000","last_fullsized":"2023-09-09T23:12:19.520143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:29:54.722210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690667+0000","last_change":"2023-09-09T23:12:17.899980+0000","last_active":"2023-09-09T23:12:20.690667+0000","last_peered":"2023-09-09T23:12:20.690667+0000","last_clean":"2023-09-09T23:12:20.690667+0000","last_became_active":"2023-09-09T23:12:17.899692+0000","last_became_peered":"2023-09-09T23:12:17.899692+0000","last_unstale":"2023-09-09T23:12:20.690667+0000","last_undegraded":"2023-09-09T23:12:20.690667+0000","last_fullsized":"2023-09-09T23:12:20.690667+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:25:28.056684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690090+0000","last_change":"2023-09-09T23:12:17.907010+0000","last_active":"2023-09-09T23:12:20.690090+0000","last_peered":"2023-09-09T23:12:20.690090+0000","last_clean":"2023-09-09T23:12:20.690090+0000","last_became_active":"2023-09-09T23:12:17.906565+0000","last_became_peered":"2023-09-09T23:12:17.906565+0000","last_unstale":"2023-09-09T23:12:20.690090+0000","last_undegraded":"2023-09-09T23:12:20.690090+0000","last_fullsized":"2023-09-09T23:12:20.690090+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:08:03.733608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.045231+0000","last_change":"2023-09-09T23:12:17.902889+0000","last_active":"2023-09-09T23:12:20.045231+0000","last_peered":"2023-09-09T23:12:20.045231+0000","last_clean":"2023-09-09T23:12:20.045231+0000","last_became_active":"2023-09-09T23:12:17.900747+0000","last_became_peered":"2023-09-09T23:12:17.900747+0000","last_unstale":"2023-09-09T23:12:20.045231+0000","last_undegraded":"2023-09-09T23:12:20.045231+0000","last_fullsized":"2023-09-09T23:12:20.045231+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:01:29.195275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:17.899880+0000","last_change":"2023-09-09T23:12:17.897844+0000","last_active":"2023-09-09T23:12:17.899880+0000","last_peered":"2023-09-09T23:12:17.899880+0000","last_clean":"2023-09-09T23:12:17.899880+0000","last_became_active":"2023-09-09T23:12:17.897529+0000","last_became_peered":"2023-09-09T23:12:17.897529+0000","last_unstale":"2023-09-09T23:12:17.899880+0000","last_undegraded":"2023-09-09T23:12:17.899880+0000","last_fullsized":"2023-09-09T23:12:17.899880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:35:48.046254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.484862+0000","last_change":"2023-09-09T23:12:17.899255+0000","last_active":"2023-09-09T23:12:19.484862+0000","last_peered":"2023-09-09T23:12:19.484862+0000","last_clean":"2023-09-09T23:12:19.484862+0000","last_became_active":"2023-09-09T23:12:17.898932+0000","last_became_peered":"2023-09-09T23:12:17.898932+0000","last_unstale":"2023-09-09T23:12:19.484862+0000","last_undegraded":"2023-09-09T23:12:19.484862+0000","last_fullsized":"2023-09-09T23:12:19.484862+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:10:56.864588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.484825+0000","last_change":"2023-09-09T23:12:17.901083+0000","last_active":"2023-09-09T23:12:19.484825+0000","last_peered":"2023-09-09T23:12:19.484825+0000","last_clean":"2023-09-09T23:12:19.484825+0000","last_became_active":"2023-09-09T23:12:17.900744+0000","last_became_peered":"2023-09-09T23:12:17.900744+0000","last_unstale":"2023-09-09T23:12:19.484825+0000","last_undegraded":"2023-09-09T23:12:19.484825+0000","last_fullsized":"2023-09-09T23:12:19.484825+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:37:11.073786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.519605+0000","last_change":"2023-09-09T23:12:17.901962+0000","last_active":"2023-09-09T23:12:19.519605+0000","last_peered":"2023-09-09T23:12:19.519605+0000","last_clean":"2023-09-09T23:12:19.519605+0000","last_became_active":"2023-09-09T23:12:17.901638+0000","last_became_peered":"2023-09-09T23:12:17.901638+0000","last_unstale":"2023-09-09T23:12:19.519605+0000","last_undegraded":"2023-09-09T23:12:19.519605+0000","last_fullsized":"2023-09-09T23:12:19.519605+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:34:52.083262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.988721+0000","last_change":"2023-09-09T23:12:17.904424+0000","last_active":"2023-09-09T23:12:18.988721+0000","last_peered":"2023-09-09T23:12:18.988721+0000","last_clean":"2023-09-09T23:12:18.988721+0000","last_became_active":"2023-09-09T23:12:17.903879+0000","last_became_peered":"2023-09-09T23:12:17.903879+0000","last_unstale":"2023-09-09T23:12:18.988721+0000","last_undegraded":"2023-09-09T23:12:18.988721+0000","last_fullsized":"2023-09-09T23:12:18.988721+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:56:42.468630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898791+0000","last_change":"2023-09-09T23:12:17.905065+0000","last_active":"2023-09-09T23:12:18.898791+0000","last_peered":"2023-09-09T23:12:18.898791+0000","last_clean":"2023-09-09T23:12:18.898791+0000","last_became_active":"2023-09-09T23:12:17.904646+0000","last_became_peered":"2023-09-09T23:12:17.904646+0000","last_unstale":"2023-09-09T23:12:18.898791+0000","last_undegraded":"2023-09-09T23:12:18.898791+0000","last_fullsized":"2023-09-09T23:12:18.898791+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:11:14.705884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690263+0000","last_change":"2023-09-09T23:12:17.899751+0000","last_active":"2023-09-09T23:12:20.690263+0000","last_peered":"2023-09-09T23:12:20.690263+0000","last_clean":"2023-09-09T23:12:20.690263+0000","last_became_active":"2023-09-09T23:12:17.899410+0000","last_became_peered":"2023-09-09T23:12:17.899410+0000","last_unstale":"2023-09-09T23:12:20.690263+0000","last_undegraded":"2023-09-09T23:12:20.690263+0000","last_fullsized":"2023-09-09T23:12:20.690263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:23:06.098276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'4","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.544646+0000","last_change":"2023-09-09T23:12:17.175671+0000","last_active":"2023-09-09T23:12:18.544646+0000","last_peered":"2023-09-09T23:12:18.544646+0000","last_clean":"2023-09-09T23:12:18.544646+0000","last_became_active":"2023-09-09T23:12:17.175327+0000","last_became_peered":"2023-09-09T23:12:17.175327+0000","last_unstale":"2023-09-09T23:12:18.544646+0000","last_undegraded":"2023-09-09T23:12:18.544646+0000","last_fullsized":"2023-09-09T23:12:18.544646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:38:57.724260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898994+0000","last_change":"2023-09-09T23:12:17.906338+0000","last_active":"2023-09-09T23:12:18.898994+0000","last_peered":"2023-09-09T23:12:18.898994+0000","last_clean":"2023-09-09T23:12:18.898994+0000","last_became_active":"2023-09-09T23:12:17.906094+0000","last_became_peered":"2023-09-09T23:12:17.906094+0000","last_unstale":"2023-09-09T23:12:18.898994+0000","last_undegraded":"2023-09-09T23:12:18.898994+0000","last_fullsized":"2023-09-09T23:12:18.898994+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:43:18.183144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.542430+0000","last_change":"2023-09-09T23:12:17.167763+0000","last_active":"2023-09-09T23:12:18.542430+0000","last_peered":"2023-09-09T23:12:18.542430+0000","last_clean":"2023-09-09T23:12:18.542430+0000","last_became_active":"2023-09-09T23:12:17.167455+0000","last_became_peered":"2023-09-09T23:12:17.167455+0000","last_unstale":"2023-09-09T23:12:18.542430+0000","last_undegraded":"2023-09-09T23:12:18.542430+0000","last_fullsized":"2023-09-09T23:12:18.542430+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:15:23.415792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.485136+0000","last_change":"2023-09-09T23:12:17.907936+0000","last_active":"2023-09-09T23:12:19.485136+0000","last_peered":"2023-09-09T23:12:19.485136+0000","last_clean":"2023-09-09T23:12:19.485136+0000","last_became_active":"2023-09-09T23:12:17.907588+0000","last_became_peered":"2023-09-09T23:12:17.907588+0000","last_unstale":"2023-09-09T23:12:19.485136+0000","last_undegraded":"2023-09-09T23:12:19.485136+0000","last_fullsized":"2023-09-09T23:12:19.485136+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:57:30.910812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'2","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898138+0000","last_change":"2023-09-09T23:12:17.172481+0000","last_active":"2023-09-09T23:12:18.898138+0000","last_peered":"2023-09-09T23:12:18.898138+0000","last_clean":"2023-09-09T23:12:18.898138+0000","last_became_active":"2023-09-09T23:12:17.172178+0000","last_became_peered":"2023-09-09T23:12:17.172178+0000","last_unstale":"2023-09-09T23:12:18.898138+0000","last_undegraded":"2023-09-09T23:12:18.898138+0000","last_fullsized":"2023-09-09T23:12:18.898138+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:08:55.328423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.519862+0000","last_change":"2023-09-09T23:12:17.900155+0000","last_active":"2023-09-09T23:12:19.519862+0000","last_peered":"2023-09-09T23:12:19.519862+0000","last_clean":"2023-09-09T23:12:19.519862+0000","last_became_active":"2023-09-09T23:12:17.899862+0000","last_became_peered":"2023-09-09T23:12:17.899862+0000","last_unstale":"2023-09-09T23:12:19.519862+0000","last_undegraded":"2023-09-09T23:12:19.519862+0000","last_fullsized":"2023-09-09T23:12:19.519862+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:57:14.949174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897973+0000","last_change":"2023-09-09T23:12:17.171890+0000","last_active":"2023-09-09T23:12:18.897973+0000","last_peered":"2023-09-09T23:12:18.897973+0000","last_clean":"2023-09-09T23:12:18.897973+0000","last_became_active":"2023-09-09T23:12:17.171496+0000","last_became_peered":"2023-09-09T23:12:17.171496+0000","last_unstale":"2023-09-09T23:12:18.897973+0000","last_undegraded":"2023-09-09T23:12:18.897973+0000","last_fullsized":"2023-09-09T23:12:18.897973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:38:25.704989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689810+0000","last_change":"2023-09-09T23:12:17.901067+0000","last_active":"2023-09-09T23:12:20.689810+0000","last_peered":"2023-09-09T23:12:20.689810+0000","last_clean":"2023-09-09T23:12:20.689810+0000","last_became_active":"2023-09-09T23:12:17.900069+0000","last_became_peered":"2023-09-09T23:12:17.900069+0000","last_unstale":"2023-09-09T23:12:20.689810+0000","last_undegraded":"2023-09-09T23:12:20.689810+0000","last_fullsized":"2023-09-09T23:12:20.689810+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:59:55.356235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:23.265413+0000","last_change":"2023-09-09T23:12:17.169474+0000","last_active":"2023-09-09T23:12:23.265413+0000","last_peered":"2023-09-09T23:12:23.265413+0000","last_clean":"2023-09-09T23:12:23.265413+0000","last_became_active":"2023-09-09T23:12:17.169186+0000","last_became_peered":"2023-09-09T23:12:17.169186+0000","last_unstale":"2023-09-09T23:12:23.265413+0000","last_undegraded":"2023-09-09T23:12:23.265413+0000","last_fullsized":"2023-09-09T23:12:23.265413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:47:13.027949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897942+0000","last_change":"2023-09-09T23:12:17.901605+0000","last_active":"2023-09-09T23:12:18.897942+0000","last_peered":"2023-09-09T23:12:18.897942+0000","last_clean":"2023-09-09T23:12:18.897942+0000","last_became_active":"2023-09-09T23:12:17.901287+0000","last_became_peered":"2023-09-09T23:12:17.901287+0000","last_unstale":"2023-09-09T23:12:18.897942+0000","last_undegraded":"2023-09-09T23:12:18.897942+0000","last_fullsized":"2023-09-09T23:12:18.897942+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:16:41.570750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.543966+0000","last_change":"2023-09-09T23:12:17.171797+0000","last_active":"2023-09-09T23:12:18.543966+0000","last_peered":"2023-09-09T23:12:18.543966+0000","last_clean":"2023-09-09T23:12:18.543966+0000","last_became_active":"2023-09-09T23:12:17.171521+0000","last_became_peered":"2023-09-09T23:12:17.171521+0000","last_unstale":"2023-09-09T23:12:18.543966+0000","last_undegraded":"2023-09-09T23:12:18.543966+0000","last_fullsized":"2023-09-09T23:12:18.543966+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:07:51.951393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690229+0000","last_change":"2023-09-09T23:12:17.900284+0000","last_active":"2023-09-09T23:12:20.690229+0000","last_peered":"2023-09-09T23:12:20.690229+0000","last_clean":"2023-09-09T23:12:20.690229+0000","last_became_active":"2023-09-09T23:12:17.899975+0000","last_became_peered":"2023-09-09T23:12:17.899975+0000","last_unstale":"2023-09-09T23:12:20.690229+0000","last_undegraded":"2023-09-09T23:12:20.690229+0000","last_fullsized":"2023-09-09T23:12:20.690229+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:10:56.723338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:17.891006+0000","last_change":"2023-09-09T23:12:17.175328+0000","last_active":"2023-09-09T23:12:17.891006+0000","last_peered":"2023-09-09T23:12:17.891006+0000","last_clean":"2023-09-09T23:12:17.891006+0000","last_became_active":"2023-09-09T23:12:17.174977+0000","last_became_peered":"2023-09-09T23:12:17.174977+0000","last_unstale":"2023-09-09T23:12:17.891006+0000","last_undegraded":"2023-09-09T23:12:17.891006+0000","last_fullsized":"2023-09-09T23:12:17.891006+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:01:36.272333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897568+0000","last_change":"2023-09-09T23:12:17.902982+0000","last_active":"2023-09-09T23:12:18.897568+0000","last_peered":"2023-09-09T23:12:18.897568+0000","last_clean":"2023-09-09T23:12:18.897568+0000","last_became_active":"2023-09-09T23:12:17.902677+0000","last_became_peered":"2023-09-09T23:12:17.902677+0000","last_unstale":"2023-09-09T23:12:18.897568+0000","last_undegraded":"2023-09-09T23:12:18.897568+0000","last_fullsized":"2023-09-09T23:12:18.897568+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:08:31.507899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'2","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:21.572877+0000","last_change":"2023-09-09T23:12:17.170099+0000","last_active":"2023-09-09T23:12:21.572877+0000","last_peered":"2023-09-09T23:12:21.572877+0000","last_clean":"2023-09-09T23:12:21.572877+0000","last_became_active":"2023-09-09T23:12:17.169827+0000","last_became_peered":"2023-09-09T23:12:17.169827+0000","last_unstale":"2023-09-09T23:12:21.572877+0000","last_undegraded":"2023-09-09T23:12:21.572877+0000","last_fullsized":"2023-09-09T23:12:21.572877+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:16:34.693568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898646+0000","last_change":"2023-09-09T23:12:17.903735+0000","last_active":"2023-09-09T23:12:18.898646+0000","last_peered":"2023-09-09T23:12:18.898646+0000","last_clean":"2023-09-09T23:12:18.898646+0000","last_became_active":"2023-09-09T23:12:17.903410+0000","last_became_peered":"2023-09-09T23:12:17.903410+0000","last_unstale":"2023-09-09T23:12:18.898646+0000","last_undegraded":"2023-09-09T23:12:18.898646+0000","last_fullsized":"2023-09-09T23:12:18.898646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:00:47.664613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:21.573928+0000","last_change":"2023-09-09T23:12:17.176004+0000","last_active":"2023-09-09T23:12:21.573928+0000","last_peered":"2023-09-09T23:12:21.573928+0000","last_clean":"2023-09-09T23:12:21.573928+0000","last_became_active":"2023-09-09T23:12:17.175514+0000","last_became_peered":"2023-09-09T23:12:17.175514+0000","last_unstale":"2023-09-09T23:12:21.573928+0000","last_undegraded":"2023-09-09T23:12:21.573928+0000","last_fullsized":"2023-09-09T23:12:21.573928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:43:54.587724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.520135+0000","last_change":"2023-09-09T23:12:17.900943+0000","last_active":"2023-09-09T23:12:19.520135+0000","last_peered":"2023-09-09T23:12:19.520135+0000","last_clean":"2023-09-09T23:12:19.520135+0000","last_became_active":"2023-09-09T23:12:17.900662+0000","last_became_peered":"2023-09-09T23:12:17.900662+0000","last_unstale":"2023-09-09T23:12:19.520135+0000","last_undegraded":"2023-09-09T23:12:19.520135+0000","last_fullsized":"2023-09-09T23:12:19.520135+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:39:41.480033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:24.280105+0000","last_change":"2023-09-09T23:12:17.172458+0000","last_active":"2023-09-09T23:12:24.280105+0000","last_peered":"2023-09-09T23:12:24.280105+0000","last_clean":"2023-09-09T23:12:24.280105+0000","last_became_active":"2023-09-09T23:12:17.172119+0000","last_became_peered":"2023-09-09T23:12:17.172119+0000","last_unstale":"2023-09-09T23:12:24.280105+0000","last_undegraded":"2023-09-09T23:12:24.280105+0000","last_fullsized":"2023-09-09T23:12:24.280105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:31:27.821245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.485149+0000","last_change":"2023-09-09T23:12:17.903252+0000","last_active":"2023-09-09T23:12:19.485149+0000","last_peered":"2023-09-09T23:12:19.485149+0000","last_clean":"2023-09-09T23:12:19.485149+0000","last_became_active":"2023-09-09T23:12:17.902466+0000","last_became_peered":"2023-09-09T23:12:17.902466+0000","last_unstale":"2023-09-09T23:12:19.485149+0000","last_undegraded":"2023-09-09T23:12:19.485149+0000","last_fullsized":"2023-09-09T23:12:19.485149+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:28:07.535937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:23.264739+0000","last_change":"2023-09-09T23:12:17.165106+0000","last_active":"2023-09-09T23:12:23.264739+0000","last_peered":"2023-09-09T23:12:23.264739+0000","last_clean":"2023-09-09T23:12:23.264739+0000","last_became_active":"2023-09-09T23:12:17.164783+0000","last_became_peered":"2023-09-09T23:12:17.164783+0000","last_unstale":"2023-09-09T23:12:23.264739+0000","last_undegraded":"2023-09-09T23:12:23.264739+0000","last_fullsized":"2023-09-09T23:12:23.264739+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:02:14.984870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.519952+0000","last_change":"2023-09-09T23:12:17.899473+0000","last_active":"2023-09-09T23:12:19.519952+0000","last_peered":"2023-09-09T23:12:19.519952+0000","last_clean":"2023-09-09T23:12:19.519952+0000","last_became_active":"2023-09-09T23:12:17.899142+0000","last_became_peered":"2023-09-09T23:12:17.899142+0000","last_unstale":"2023-09-09T23:12:19.519952+0000","last_undegraded":"2023-09-09T23:12:19.519952+0000","last_fullsized":"2023-09-09T23:12:19.519952+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:50:30.261873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'8","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:21.574929+0000","last_change":"2023-09-09T23:12:17.172560+0000","last_active":"2023-09-09T23:12:21.574929+0000","last_peered":"2023-09-09T23:12:21.574929+0000","last_clean":"2023-09-09T23:12:21.574929+0000","last_became_active":"2023-09-09T23:12:17.172213+0000","last_became_peered":"2023-09-09T23:12:17.172213+0000","last_unstale":"2023-09-09T23:12:21.574929+0000","last_undegraded":"2023-09-09T23:12:21.574929+0000","last_fullsized":"2023-09-09T23:12:21.574929+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:30:00.206901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898030+0000","last_change":"2023-09-09T23:12:17.898279+0000","last_active":"2023-09-09T23:12:18.898030+0000","last_peered":"2023-09-09T23:12:18.898030+0000","last_clean":"2023-09-09T23:12:18.898030+0000","last_became_active":"2023-09-09T23:12:17.898041+0000","last_became_peered":"2023-09-09T23:12:17.898041+0000","last_unstale":"2023-09-09T23:12:18.898030+0000","last_undegraded":"2023-09-09T23:12:18.898030+0000","last_fullsized":"2023-09-09T23:12:18.898030+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:01:10.315445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'4","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:21.573959+0000","last_change":"2023-09-09T23:12:17.172079+0000","last_active":"2023-09-09T23:12:21.573959+0000","last_peered":"2023-09-09T23:12:21.573959+0000","last_clean":"2023-09-09T23:12:21.573959+0000","last_became_active":"2023-09-09T23:12:17.171779+0000","last_became_peered":"2023-09-09T23:12:17.171779+0000","last_unstale":"2023-09-09T23:12:21.573959+0000","last_undegraded":"2023-09-09T23:12:21.573959+0000","last_fullsized":"2023-09-09T23:12:21.573959+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:49:50.287124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:17.907644+0000","last_change":"2023-09-09T23:12:17.904087+0000","last_active":"2023-09-09T23:12:17.907644+0000","last_peered":"2023-09-09T23:12:17.907644+0000","last_clean":"2023-09-09T23:12:17.907644+0000","last_became_active":"2023-09-09T23:12:17.903670+0000","last_became_peered":"2023-09-09T23:12:17.903670+0000","last_unstale":"2023-09-09T23:12:17.907644+0000","last_undegraded":"2023-09-09T23:12:17.907644+0000","last_fullsized":"2023-09-09T23:12:17.907644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:31:28.511653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'4","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:21.573310+0000","last_change":"2023-09-09T23:12:17.170594+0000","last_active":"2023-09-09T23:12:21.573310+0000","last_peered":"2023-09-09T23:12:21.573310+0000","last_clean":"2023-09-09T23:12:21.573310+0000","last_became_active":"2023-09-09T23:12:17.170223+0000","last_became_peered":"2023-09-09T23:12:17.170223+0000","last_unstale":"2023-09-09T23:12:21.573310+0000","last_undegraded":"2023-09-09T23:12:21.573310+0000","last_fullsized":"2023-09-09T23:12:21.573310+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:30:09.222947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689977+0000","last_change":"2023-09-09T23:12:17.901047+0000","last_active":"2023-09-09T23:12:20.689977+0000","last_peered":"2023-09-09T23:12:20.689977+0000","last_clean":"2023-09-09T23:12:20.689977+0000","last_became_active":"2023-09-09T23:12:17.900129+0000","last_became_peered":"2023-09-09T23:12:17.900129+0000","last_unstale":"2023-09-09T23:12:20.689977+0000","last_undegraded":"2023-09-09T23:12:20.689977+0000","last_fullsized":"2023-09-09T23:12:20.689977+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:59:36.338392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:23.271984+0000","last_change":"2023-09-09T23:12:17.169495+0000","last_active":"2023-09-09T23:12:23.271984+0000","last_peered":"2023-09-09T23:12:23.271984+0000","last_clean":"2023-09-09T23:12:23.271984+0000","last_became_active":"2023-09-09T23:12:17.169170+0000","last_became_peered":"2023-09-09T23:12:17.169170+0000","last_unstale":"2023-09-09T23:12:23.271984+0000","last_undegraded":"2023-09-09T23:12:23.271984+0000","last_fullsized":"2023-09-09T23:12:23.271984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:36:23.765574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689415+0000","last_change":"2023-09-09T23:12:17.907826+0000","last_active":"2023-09-09T23:12:20.689415+0000","last_peered":"2023-09-09T23:12:20.689415+0000","last_clean":"2023-09-09T23:12:20.689415+0000","last_became_active":"2023-09-09T23:12:17.907526+0000","last_became_peered":"2023-09-09T23:12:17.907526+0000","last_unstale":"2023-09-09T23:12:20.689415+0000","last_undegraded":"2023-09-09T23:12:20.689415+0000","last_fullsized":"2023-09-09T23:12:20.689415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:17:20.587253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'8","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897643+0000","last_change":"2023-09-09T23:12:17.170139+0000","last_active":"2023-09-09T23:12:18.897643+0000","last_peered":"2023-09-09T23:12:18.897643+0000","last_clean":"2023-09-09T23:12:18.897643+0000","last_became_active":"2023-09-09T23:12:17.169797+0000","last_became_peered":"2023-09-09T23:12:17.169797+0000","last_unstale":"2023-09-09T23:12:18.897643+0000","last_undegraded":"2023-09-09T23:12:18.897643+0000","last_fullsized":"2023-09-09T23:12:18.897643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:59:44.668114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689753+0000","last_change":"2023-09-09T23:12:17.902567+0000","last_active":"2023-09-09T23:12:20.689753+0000","last_peered":"2023-09-09T23:12:20.689753+0000","last_clean":"2023-09-09T23:12:20.689753+0000","last_became_active":"2023-09-09T23:12:17.902229+0000","last_became_peered":"2023-09-09T23:12:17.902229+0000","last_unstale":"2023-09-09T23:12:20.689753+0000","last_undegraded":"2023-09-09T23:12:20.689753+0000","last_fullsized":"2023-09-09T23:12:20.689753+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:49:50.049925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897980+0000","last_change":"2023-09-09T23:12:17.171143+0000","last_active":"2023-09-09T23:12:18.897980+0000","last_peered":"2023-09-09T23:12:18.897980+0000","last_clean":"2023-09-09T23:12:18.897980+0000","last_became_active":"2023-09-09T23:12:17.170847+0000","last_became_peered":"2023-09-09T23:12:17.170847+0000","last_unstale":"2023-09-09T23:12:18.897980+0000","last_undegraded":"2023-09-09T23:12:18.897980+0000","last_fullsized":"2023-09-09T23:12:18.897980+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:35:55.570394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.988514+0000","last_change":"2023-09-09T23:12:17.900284+0000","last_active":"2023-09-09T23:12:18.988514+0000","last_peered":"2023-09-09T23:12:18.988514+0000","last_clean":"2023-09-09T23:12:18.988514+0000","last_became_active":"2023-09-09T23:12:17.899923+0000","last_became_peered":"2023-09-09T23:12:17.899923+0000","last_unstale":"2023-09-09T23:12:18.988514+0000","last_undegraded":"2023-09-09T23:12:18.988514+0000","last_fullsized":"2023-09-09T23:12:18.988514+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:32:32.230968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:17.889782+0000","last_change":"2023-09-09T23:12:17.176313+0000","last_active":"2023-09-09T23:12:17.889782+0000","last_peered":"2023-09-09T23:12:17.889782+0000","last_clean":"2023-09-09T23:12:17.889782+0000","last_became_active":"2023-09-09T23:12:17.175934+0000","last_became_peered":"2023-09-09T23:12:17.175934+0000","last_unstale":"2023-09-09T23:12:17.889782+0000","last_undegraded":"2023-09-09T23:12:17.889782+0000","last_fullsized":"2023-09-09T23:12:17.889782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:33:15.044986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690469+0000","last_change":"2023-09-09T23:12:17.900284+0000","last_active":"2023-09-09T23:12:20.690469+0000","last_peered":"2023-09-09T23:12:20.690469+0000","last_clean":"2023-09-09T23:12:20.690469+0000","last_became_active":"2023-09-09T23:12:17.899970+0000","last_became_peered":"2023-09-09T23:12:17.899970+0000","last_unstale":"2023-09-09T23:12:20.690469+0000","last_undegraded":"2023-09-09T23:12:20.690469+0000","last_fullsized":"2023-09-09T23:12:20.690469+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:01:37.415195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:24.280539+0000","last_change":"2023-09-09T23:12:17.170123+0000","last_active":"2023-09-09T23:12:24.280539+0000","last_peered":"2023-09-09T23:12:24.280539+0000","last_clean":"2023-09-09T23:12:24.280539+0000","last_became_active":"2023-09-09T23:12:17.169775+0000","last_became_peered":"2023-09-09T23:12:17.169775+0000","last_unstale":"2023-09-09T23:12:24.280539+0000","last_undegraded":"2023-09-09T23:12:24.280539+0000","last_fullsized":"2023-09-09T23:12:24.280539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:22:44.520489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690417+0000","last_change":"2023-09-09T23:12:17.894881+0000","last_active":"2023-09-09T23:12:20.690417+0000","last_peered":"2023-09-09T23:12:20.690417+0000","last_clean":"2023-09-09T23:12:20.690417+0000","last_became_active":"2023-09-09T23:12:17.894567+0000","last_became_peered":"2023-09-09T23:12:17.894567+0000","last_unstale":"2023-09-09T23:12:20.690417+0000","last_undegraded":"2023-09-09T23:12:20.690417+0000","last_fullsized":"2023-09-09T23:12:20.690417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:40:37.098780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689780+0000","last_change":"2023-09-09T23:12:17.903319+0000","last_active":"2023-09-09T23:12:20.689780+0000","last_peered":"2023-09-09T23:12:20.689780+0000","last_clean":"2023-09-09T23:12:20.689780+0000","last_became_active":"2023-09-09T23:12:17.902642+0000","last_became_peered":"2023-09-09T23:12:17.902642+0000","last_unstale":"2023-09-09T23:12:20.689780+0000","last_undegraded":"2023-09-09T23:12:20.689780+0000","last_fullsized":"2023-09-09T23:12:20.689780+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:03:29.562813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:17.890868+0000","last_change":"2023-09-09T23:12:17.172639+0000","last_active":"2023-09-09T23:12:17.890868+0000","last_peered":"2023-09-09T23:12:17.890868+0000","last_clean":"2023-09-09T23:12:17.890868+0000","last_became_active":"2023-09-09T23:12:17.172309+0000","last_became_peered":"2023-09-09T23:12:17.172309+0000","last_unstale":"2023-09-09T23:12:17.890868+0000","last_undegraded":"2023-09-09T23:12:17.890868+0000","last_fullsized":"2023-09-09T23:12:17.890868+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:05:56.160984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898148+0000","last_change":"2023-09-09T23:12:17.906676+0000","last_active":"2023-09-09T23:12:18.898148+0000","last_peered":"2023-09-09T23:12:18.898148+0000","last_clean":"2023-09-09T23:12:18.898148+0000","last_became_active":"2023-09-09T23:12:17.906381+0000","last_became_peered":"2023-09-09T23:12:17.906381+0000","last_unstale":"2023-09-09T23:12:18.898148+0000","last_undegraded":"2023-09-09T23:12:18.898148+0000","last_fullsized":"2023-09-09T23:12:18.898148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:42:28.033255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:23.271968+0000","last_change":"2023-09-09T23:12:17.166623+0000","last_active":"2023-09-09T23:12:23.271968+0000","last_peered":"2023-09-09T23:12:23.271968+0000","last_clean":"2023-09-09T23:12:23.271968+0000","last_became_active":"2023-09-09T23:12:17.166293+0000","last_became_peered":"2023-09-09T23:12:17.166293+0000","last_unstale":"2023-09-09T23:12:23.271968+0000","last_undegraded":"2023-09-09T23:12:23.271968+0000","last_fullsized":"2023-09-09T23:12:23.271968+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:52:51.267501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":317,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.963859+0000","last_change":"2023-09-09T23:11:20.948461+0000","last_active":"2023-09-09T23:12:19.963859+0000","last_peered":"2023-09-09T23:12:19.963859+0000","last_clean":"2023-09-09T23:12:19.963859+0000","last_became_active":"2023-09-09T23:11:20.947856+0000","last_became_peered":"2023-09-09T23:11:20.947856+0000","last_unstale":"2023-09-09T23:12:19.963859+0000","last_undegraded":"2023-09-09T23:12:19.963859+0000","last_fullsized":"2023-09-09T23:12:19.963859+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:08:31.737051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:08:31.737051+0000","last_clean_scrub_stamp":"2023-09-09T23:08:31.737051+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:20:16.293707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:17.908619+0000","last_change":"2023-09-09T23:12:17.907468+0000","last_active":"2023-09-09T23:12:17.908619+0000","last_peered":"2023-09-09T23:12:17.908619+0000","last_clean":"2023-09-09T23:12:17.908619+0000","last_became_active":"2023-09-09T23:12:17.907117+0000","last_became_peered":"2023-09-09T23:12:17.907117+0000","last_unstale":"2023-09-09T23:12:17.908619+0000","last_undegraded":"2023-09-09T23:12:17.908619+0000","last_fullsized":"2023-09-09T23:12:17.908619+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:56:38.086209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:22.583028+0000","last_change":"2023-09-09T23:12:17.171505+0000","last_active":"2023-09-09T23:12:22.583028+0000","last_peered":"2023-09-09T23:12:22.583028+0000","last_clean":"2023-09-09T23:12:22.583028+0000","last_became_active":"2023-09-09T23:12:17.171185+0000","last_became_peered":"2023-09-09T23:12:17.171185+0000","last_unstale":"2023-09-09T23:12:22.583028+0000","last_undegraded":"2023-09-09T23:12:22.583028+0000","last_fullsized":"2023-09-09T23:12:22.583028+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:07:54.271364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898098+0000","last_change":"2023-09-09T23:12:17.176450+0000","last_active":"2023-09-09T23:12:18.898098+0000","last_peered":"2023-09-09T23:12:18.898098+0000","last_clean":"2023-09-09T23:12:18.898098+0000","last_became_active":"2023-09-09T23:12:17.176217+0000","last_became_peered":"2023-09-09T23:12:17.176217+0000","last_unstale":"2023-09-09T23:12:18.898098+0000","last_undegraded":"2023-09-09T23:12:18.898098+0000","last_fullsized":"2023-09-09T23:12:18.898098+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:57:26.604261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:17.900130+0000","last_change":"2023-09-09T23:12:17.897612+0000","last_active":"2023-09-09T23:12:17.900130+0000","last_peered":"2023-09-09T23:12:17.900130+0000","last_clean":"2023-09-09T23:12:17.900130+0000","last_became_active":"2023-09-09T23:12:17.897292+0000","last_became_peered":"2023-09-09T23:12:17.897292+0000","last_unstale":"2023-09-09T23:12:17.900130+0000","last_undegraded":"2023-09-09T23:12:17.900130+0000","last_fullsized":"2023-09-09T23:12:17.900130+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:27:53.095238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:17.896214+0000","last_change":"2023-09-09T23:12:17.175682+0000","last_active":"2023-09-09T23:12:17.896214+0000","last_peered":"2023-09-09T23:12:17.896214+0000","last_clean":"2023-09-09T23:12:17.896214+0000","last_became_active":"2023-09-09T23:12:17.175291+0000","last_became_peered":"2023-09-09T23:12:17.175291+0000","last_unstale":"2023-09-09T23:12:17.896214+0000","last_undegraded":"2023-09-09T23:12:17.896214+0000","last_fullsized":"2023-09-09T23:12:17.896214+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:29:02.349319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.899164+0000","last_change":"2023-09-09T23:12:17.904153+0000","last_active":"2023-09-09T23:12:18.899164+0000","last_peered":"2023-09-09T23:12:18.899164+0000","last_clean":"2023-09-09T23:12:18.899164+0000","last_became_active":"2023-09-09T23:12:17.903804+0000","last_became_peered":"2023-09-09T23:12:17.903804+0000","last_unstale":"2023-09-09T23:12:18.899164+0000","last_undegraded":"2023-09-09T23:12:18.899164+0000","last_fullsized":"2023-09-09T23:12:18.899164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:39:34.046551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'5","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:21.575155+0000","last_change":"2023-09-09T23:12:17.169052+0000","last_active":"2023-09-09T23:12:21.575155+0000","last_peered":"2023-09-09T23:12:21.575155+0000","last_clean":"2023-09-09T23:12:21.575155+0000","last_became_active":"2023-09-09T23:12:17.168719+0000","last_became_peered":"2023-09-09T23:12:17.168719+0000","last_unstale":"2023-09-09T23:12:21.575155+0000","last_undegraded":"2023-09-09T23:12:21.575155+0000","last_fullsized":"2023-09-09T23:12:21.575155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:56:44.427954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.485050+0000","last_change":"2023-09-09T23:12:17.900945+0000","last_active":"2023-09-09T23:12:19.485050+0000","last_peered":"2023-09-09T23:12:19.485050+0000","last_clean":"2023-09-09T23:12:19.485050+0000","last_became_active":"2023-09-09T23:12:17.900602+0000","last_became_peered":"2023-09-09T23:12:17.900602+0000","last_unstale":"2023-09-09T23:12:19.485050+0000","last_undegraded":"2023-09-09T23:12:19.485050+0000","last_fullsized":"2023-09-09T23:12:19.485050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:22:10.336285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.519606+0000","last_change":"2023-09-09T23:12:17.166619+0000","last_active":"2023-09-09T23:12:19.519606+0000","last_peered":"2023-09-09T23:12:19.519606+0000","last_clean":"2023-09-09T23:12:19.519606+0000","last_became_active":"2023-09-09T23:12:17.166286+0000","last_became_peered":"2023-09-09T23:12:17.166286+0000","last_unstale":"2023-09-09T23:12:19.519606+0000","last_undegraded":"2023-09-09T23:12:19.519606+0000","last_fullsized":"2023-09-09T23:12:19.519606+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:41:40.713296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897742+0000","last_change":"2023-09-09T23:12:17.897131+0000","last_active":"2023-09-09T23:12:18.897742+0000","last_peered":"2023-09-09T23:12:18.897742+0000","last_clean":"2023-09-09T23:12:18.897742+0000","last_became_active":"2023-09-09T23:12:17.896873+0000","last_became_peered":"2023-09-09T23:12:17.896873+0000","last_unstale":"2023-09-09T23:12:18.897742+0000","last_undegraded":"2023-09-09T23:12:18.897742+0000","last_fullsized":"2023-09-09T23:12:18.897742+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:54:21.091418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:17.891648+0000","last_change":"2023-09-09T23:12:17.168868+0000","last_active":"2023-09-09T23:12:17.891648+0000","last_peered":"2023-09-09T23:12:17.891648+0000","last_clean":"2023-09-09T23:12:17.891648+0000","last_became_active":"2023-09-09T23:12:17.168572+0000","last_became_peered":"2023-09-09T23:12:17.168572+0000","last_unstale":"2023-09-09T23:12:17.891648+0000","last_undegraded":"2023-09-09T23:12:17.891648+0000","last_fullsized":"2023-09-09T23:12:17.891648+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:54:08.108023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.988326+0000","last_change":"2023-09-09T23:12:17.902887+0000","last_active":"2023-09-09T23:12:18.988326+0000","last_peered":"2023-09-09T23:12:18.988326+0000","last_clean":"2023-09-09T23:12:18.988326+0000","last_became_active":"2023-09-09T23:12:17.902507+0000","last_became_peered":"2023-09-09T23:12:17.902507+0000","last_unstale":"2023-09-09T23:12:18.988326+0000","last_undegraded":"2023-09-09T23:12:18.988326+0000","last_fullsized":"2023-09-09T23:12:18.988326+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:18:08.000184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:17.886594+0000","last_change":"2023-09-09T23:12:17.171379+0000","last_active":"2023-09-09T23:12:17.886594+0000","last_peered":"2023-09-09T23:12:17.886594+0000","last_clean":"2023-09-09T23:12:17.886594+0000","last_became_active":"2023-09-09T23:12:17.171024+0000","last_became_peered":"2023-09-09T23:12:17.171024+0000","last_unstale":"2023-09-09T23:12:17.886594+0000","last_undegraded":"2023-09-09T23:12:17.886594+0000","last_fullsized":"2023-09-09T23:12:17.886594+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:17:12.206768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898049+0000","last_change":"2023-09-09T23:12:17.905510+0000","last_active":"2023-09-09T23:12:18.898049+0000","last_peered":"2023-09-09T23:12:18.898049+0000","last_clean":"2023-09-09T23:12:18.898049+0000","last_became_active":"2023-09-09T23:12:17.905207+0000","last_became_peered":"2023-09-09T23:12:17.905207+0000","last_unstale":"2023-09-09T23:12:18.898049+0000","last_undegraded":"2023-09-09T23:12:18.898049+0000","last_fullsized":"2023-09-09T23:12:18.898049+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:48:20.167421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'2","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897752+0000","last_change":"2023-09-09T23:12:17.173764+0000","last_active":"2023-09-09T23:12:18.897752+0000","last_peered":"2023-09-09T23:12:18.897752+0000","last_clean":"2023-09-09T23:12:18.897752+0000","last_became_active":"2023-09-09T23:12:17.173470+0000","last_became_peered":"2023-09-09T23:12:17.173470+0000","last_unstale":"2023-09-09T23:12:18.897752+0000","last_undegraded":"2023-09-09T23:12:18.897752+0000","last_fullsized":"2023-09-09T23:12:18.897752+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:01:54.091664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690237+0000","last_change":"2023-09-09T23:12:17.906292+0000","last_active":"2023-09-09T23:12:20.690237+0000","last_peered":"2023-09-09T23:12:20.690237+0000","last_clean":"2023-09-09T23:12:20.690237+0000","last_became_active":"2023-09-09T23:12:17.905995+0000","last_became_peered":"2023-09-09T23:12:17.905995+0000","last_unstale":"2023-09-09T23:12:20.690237+0000","last_undegraded":"2023-09-09T23:12:20.690237+0000","last_fullsized":"2023-09-09T23:12:20.690237+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:52:29.972831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:17.886676+0000","last_change":"2023-09-09T23:12:17.173818+0000","last_active":"2023-09-09T23:12:17.886676+0000","last_peered":"2023-09-09T23:12:17.886676+0000","last_clean":"2023-09-09T23:12:17.886676+0000","last_became_active":"2023-09-09T23:12:17.173413+0000","last_became_peered":"2023-09-09T23:12:17.173413+0000","last_unstale":"2023-09-09T23:12:17.886676+0000","last_undegraded":"2023-09-09T23:12:17.886676+0000","last_fullsized":"2023-09-09T23:12:17.886676+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:55:28.856863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690266+0000","last_change":"2023-09-09T23:12:17.906833+0000","last_active":"2023-09-09T23:12:20.690266+0000","last_peered":"2023-09-09T23:12:20.690266+0000","last_clean":"2023-09-09T23:12:20.690266+0000","last_became_active":"2023-09-09T23:12:17.906138+0000","last_became_peered":"2023-09-09T23:12:17.906138+0000","last_unstale":"2023-09-09T23:12:20.690266+0000","last_undegraded":"2023-09-09T23:12:20.690266+0000","last_fullsized":"2023-09-09T23:12:20.690266+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:38:07.779167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:23.260259+0000","last_change":"2023-09-09T23:12:17.171338+0000","last_active":"2023-09-09T23:12:23.260259+0000","last_peered":"2023-09-09T23:12:23.260259+0000","last_clean":"2023-09-09T23:12:23.260259+0000","last_became_active":"2023-09-09T23:12:17.170613+0000","last_became_peered":"2023-09-09T23:12:17.170613+0000","last_unstale":"2023-09-09T23:12:23.260259+0000","last_undegraded":"2023-09-09T23:12:23.260259+0000","last_fullsized":"2023-09-09T23:12:23.260259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:24:07.794464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689730+0000","last_change":"2023-09-09T23:12:17.906470+0000","last_active":"2023-09-09T23:12:20.689730+0000","last_peered":"2023-09-09T23:12:20.689730+0000","last_clean":"2023-09-09T23:12:20.689730+0000","last_became_active":"2023-09-09T23:12:17.906170+0000","last_became_peered":"2023-09-09T23:12:17.906170+0000","last_unstale":"2023-09-09T23:12:20.689730+0000","last_undegraded":"2023-09-09T23:12:20.689730+0000","last_fullsized":"2023-09-09T23:12:20.689730+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:44:37.483541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.544399+0000","last_change":"2023-09-09T23:12:17.172142+0000","last_active":"2023-09-09T23:12:18.544399+0000","last_peered":"2023-09-09T23:12:18.544399+0000","last_clean":"2023-09-09T23:12:18.544399+0000","last_became_active":"2023-09-09T23:12:17.171703+0000","last_became_peered":"2023-09-09T23:12:17.171703+0000","last_unstale":"2023-09-09T23:12:18.544399+0000","last_undegraded":"2023-09-09T23:12:18.544399+0000","last_fullsized":"2023-09-09T23:12:18.544399+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:49:31.414385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689874+0000","last_change":"2023-09-09T23:12:17.900699+0000","last_active":"2023-09-09T23:12:20.689874+0000","last_peered":"2023-09-09T23:12:20.689874+0000","last_clean":"2023-09-09T23:12:20.689874+0000","last_became_active":"2023-09-09T23:12:17.900416+0000","last_became_peered":"2023-09-09T23:12:17.900416+0000","last_unstale":"2023-09-09T23:12:20.689874+0000","last_undegraded":"2023-09-09T23:12:20.689874+0000","last_fullsized":"2023-09-09T23:12:20.689874+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:54:29.328413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.988389+0000","last_change":"2023-09-09T23:12:17.900480+0000","last_active":"2023-09-09T23:12:18.988389+0000","last_peered":"2023-09-09T23:12:18.988389+0000","last_clean":"2023-09-09T23:12:18.988389+0000","last_became_active":"2023-09-09T23:12:17.900135+0000","last_became_peered":"2023-09-09T23:12:17.900135+0000","last_unstale":"2023-09-09T23:12:18.988389+0000","last_undegraded":"2023-09-09T23:12:18.988389+0000","last_fullsized":"2023-09-09T23:12:18.988389+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:25:44.254310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690446+0000","last_change":"2023-09-09T23:12:17.901302+0000","last_active":"2023-09-09T23:12:20.690446+0000","last_peered":"2023-09-09T23:12:20.690446+0000","last_clean":"2023-09-09T23:12:20.690446+0000","last_became_active":"2023-09-09T23:12:17.900982+0000","last_became_peered":"2023-09-09T23:12:17.900982+0000","last_unstale":"2023-09-09T23:12:20.690446+0000","last_undegraded":"2023-09-09T23:12:20.690446+0000","last_fullsized":"2023-09-09T23:12:20.690446+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:41:41.119614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689782+0000","last_change":"2023-09-09T23:12:17.904328+0000","last_active":"2023-09-09T23:12:20.689782+0000","last_peered":"2023-09-09T23:12:20.689782+0000","last_clean":"2023-09-09T23:12:20.689782+0000","last_became_active":"2023-09-09T23:12:17.904040+0000","last_became_peered":"2023-09-09T23:12:17.904040+0000","last_unstale":"2023-09-09T23:12:20.689782+0000","last_undegraded":"2023-09-09T23:12:20.689782+0000","last_fullsized":"2023-09-09T23:12:20.689782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:01:05.986097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.485015+0000","last_change":"2023-09-09T23:12:17.901737+0000","last_active":"2023-09-09T23:12:19.485015+0000","last_peered":"2023-09-09T23:12:19.485015+0000","last_clean":"2023-09-09T23:12:19.485015+0000","last_became_active":"2023-09-09T23:12:17.901452+0000","last_became_peered":"2023-09-09T23:12:17.901452+0000","last_unstale":"2023-09-09T23:12:19.485015+0000","last_undegraded":"2023-09-09T23:12:19.485015+0000","last_fullsized":"2023-09-09T23:12:19.485015+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:35:28.698304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:17.908698+0000","last_change":"2023-09-09T23:12:17.907433+0000","last_active":"2023-09-09T23:12:17.908698+0000","last_peered":"2023-09-09T23:12:17.908698+0000","last_clean":"2023-09-09T23:12:17.908698+0000","last_became_active":"2023-09-09T23:12:17.907127+0000","last_became_peered":"2023-09-09T23:12:17.907127+0000","last_unstale":"2023-09-09T23:12:17.908698+0000","last_undegraded":"2023-09-09T23:12:17.908698+0000","last_fullsized":"2023-09-09T23:12:17.908698+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:11:31.575509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.988586+0000","last_change":"2023-09-09T23:12:17.901373+0000","last_active":"2023-09-09T23:12:18.988586+0000","last_peered":"2023-09-09T23:12:18.988586+0000","last_clean":"2023-09-09T23:12:18.988586+0000","last_became_active":"2023-09-09T23:12:17.901080+0000","last_became_peered":"2023-09-09T23:12:17.901080+0000","last_unstale":"2023-09-09T23:12:18.988586+0000","last_undegraded":"2023-09-09T23:12:18.988586+0000","last_fullsized":"2023-09-09T23:12:18.988586+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:23:12.316659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690576+0000","last_change":"2023-09-09T23:12:17.896579+0000","last_active":"2023-09-09T23:12:20.690576+0000","last_peered":"2023-09-09T23:12:20.690576+0000","last_clean":"2023-09-09T23:12:20.690576+0000","last_became_active":"2023-09-09T23:12:17.896253+0000","last_became_peered":"2023-09-09T23:12:17.896253+0000","last_unstale":"2023-09-09T23:12:20.690576+0000","last_undegraded":"2023-09-09T23:12:20.690576+0000","last_fullsized":"2023-09-09T23:12:20.690576+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:30:20.249103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.899186+0000","last_change":"2023-09-09T23:12:17.903569+0000","last_active":"2023-09-09T23:12:18.899186+0000","last_peered":"2023-09-09T23:12:18.899186+0000","last_clean":"2023-09-09T23:12:18.899186+0000","last_became_active":"2023-09-09T23:12:17.903027+0000","last_became_peered":"2023-09-09T23:12:17.903027+0000","last_unstale":"2023-09-09T23:12:18.899186+0000","last_undegraded":"2023-09-09T23:12:18.899186+0000","last_fullsized":"2023-09-09T23:12:18.899186+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:38:46.392973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689604+0000","last_change":"2023-09-09T23:12:17.902944+0000","last_active":"2023-09-09T23:12:20.689604+0000","last_peered":"2023-09-09T23:12:20.689604+0000","last_clean":"2023-09-09T23:12:20.689604+0000","last_became_active":"2023-09-09T23:12:17.901812+0000","last_became_peered":"2023-09-09T23:12:17.901812+0000","last_unstale":"2023-09-09T23:12:20.689604+0000","last_undegraded":"2023-09-09T23:12:20.689604+0000","last_fullsized":"2023-09-09T23:12:20.689604+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:49:50.981693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897608+0000","last_change":"2023-09-09T23:12:17.897612+0000","last_active":"2023-09-09T23:12:18.897608+0000","last_peered":"2023-09-09T23:12:18.897608+0000","last_clean":"2023-09-09T23:12:18.897608+0000","last_became_active":"2023-09-09T23:12:17.897318+0000","last_became_peered":"2023-09-09T23:12:17.897318+0000","last_unstale":"2023-09-09T23:12:18.897608+0000","last_undegraded":"2023-09-09T23:12:18.897608+0000","last_fullsized":"2023-09-09T23:12:18.897608+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:17:38.687470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.519770+0000","last_change":"2023-09-09T23:12:17.903849+0000","last_active":"2023-09-09T23:12:19.519770+0000","last_peered":"2023-09-09T23:12:19.519770+0000","last_clean":"2023-09-09T23:12:19.519770+0000","last_became_active":"2023-09-09T23:12:17.903534+0000","last_became_peered":"2023-09-09T23:12:17.903534+0000","last_unstale":"2023-09-09T23:12:19.519770+0000","last_undegraded":"2023-09-09T23:12:19.519770+0000","last_fullsized":"2023-09-09T23:12:19.519770+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:06:18.073811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:17.906425+0000","last_change":"2023-09-09T23:12:17.899782+0000","last_active":"2023-09-09T23:12:17.906425+0000","last_peered":"2023-09-09T23:12:17.906425+0000","last_clean":"2023-09-09T23:12:17.906425+0000","last_became_active":"2023-09-09T23:12:17.899445+0000","last_became_peered":"2023-09-09T23:12:17.899445+0000","last_unstale":"2023-09-09T23:12:17.906425+0000","last_undegraded":"2023-09-09T23:12:17.906425+0000","last_fullsized":"2023-09-09T23:12:17.906425+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:20:44.292143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.988408+0000","last_change":"2023-09-09T23:12:17.898693+0000","last_active":"2023-09-09T23:12:18.988408+0000","last_peered":"2023-09-09T23:12:18.988408+0000","last_clean":"2023-09-09T23:12:18.988408+0000","last_became_active":"2023-09-09T23:12:17.898343+0000","last_became_peered":"2023-09-09T23:12:17.898343+0000","last_unstale":"2023-09-09T23:12:18.988408+0000","last_undegraded":"2023-09-09T23:12:18.988408+0000","last_fullsized":"2023-09-09T23:12:18.988408+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:37:43.229953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5998,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":94,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":17714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85,"ondisk_log_size":85,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514507,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":416035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]}]},{"osd":10,"up_from":69,"seq":296352743439,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28472,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716680,"statfs":{"total":95995035648,"available":95965880320,"internally_reserved":0,"allocated":1486848,"data_stored":1008209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":1,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.308}]},{"osd":2,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.442}]},{"osd":3,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.45400000000000001}]},{"osd":4,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.33600000000000002}]},{"osd":5,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.628},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.38600000000000001}]},{"osd":6,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":8,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.56499999999999995}]},{"osd":9,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.59399999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]}]},{"osd":9,"up_from":64,"seq":274877906962,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":416617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:11:55 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.35399999999999998}]},{"osd":1,"last update":"Sat Sep 9 23:11:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.60899999999999999}]},{"osd":2,"last update":"Sat Sep 9 23:11:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.51900000000000002}]},{"osd":3,"last update":"Sat Sep 9 23:11:55 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.432},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.34100000000000003}]},{"osd":4,"last update":"Sat Sep 9 23:11:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.57799999999999996}]},{"osd":5,"last update":"Sat Sep 9 23:11:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.504},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.59399999999999997}]},{"osd":6,"last update":"Sat Sep 9 23:11:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.38},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.46100000000000002}]},{"osd":7,"last update":"Sat Sep 9 23:11:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.628},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.53800000000000003}]},{"osd":8,"last update":"Sat Sep 9 23:11:55 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.66400000000000003}]},{"osd":10,"last update":"Sat Sep 9 23:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.69199999999999995}]}]},{"osd":8,"up_from":57,"seq":244813135896,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":876544,"data_stored":416639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.38},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.52600000000000002}]},{"osd":1,"last update":"Sat Sep 9 23:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.32900000000000001}]},{"osd":2,"last update":"Sat Sep 9 23:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.42999999999999999}]},{"osd":3,"last update":"Sat Sep 9 23:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.504}]},{"osd":4,"last update":"Sat Sep 9 23:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.41899999999999998}]},{"osd":5,"last update":"Sat Sep 9 23:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.495}]},{"osd":6,"last update":"Sat Sep 9 23:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.55200000000000005}]},{"osd":7,"last update":"Sat Sep 9 23:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.60699999999999998}]},{"osd":9,"last update":"Sat Sep 9 23:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.56299999999999994}]},{"osd":10,"last update":"Sat Sep 9 23:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.59399999999999997}]}]},{"osd":7,"up_from":50,"seq":214748364828,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28464,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716688,"statfs":{"total":95995035648,"available":95965888512,"internally_reserved":0,"allocated":1482752,"data_stored":1008926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.439},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.33100000000000002}]},{"osd":1,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.435,"15min":0.432},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.60999999999999999,"5min":0.625,"15min":0.625},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.42699999999999999,"15min":0.41699999999999998},"min":{"1min":0.26200000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.48499999999999999}]},{"osd":2,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.77200000000000002,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.63700000000000001}]},{"osd":3,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.40600000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.626,"15min":0.625},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.79800000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.68300000000000005}]},{"osd":4,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52900000000000003,"15min":0.52300000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.76400000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.314,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.52100000000000002}]},{"osd":5,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.69399999999999995,"15min":0.69599999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86399999999999999,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.66900000000000004,"15min":0.66800000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.69999999999999996}]},{"osd":6,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.746,"15min":0.753},"min":{"1min":0.48899999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90900000000000003,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.753},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.72599999999999998,"15min":0.72899999999999998},"min":{"1min":0.46800000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92600000000000005,"5min":1.254,"15min":1.254},"last":0.65500000000000003}]},{"osd":8,"last update":"Sat Sep 9 23:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.86499999999999999}]},{"osd":9,"last update":"Sat Sep 9 23:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.85299999999999998}]},{"osd":10,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.88200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]}]},{"osd":6,"up_from":45,"seq":193273528352,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":856064,"data_stored":407465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.375,"15min":0.36799999999999999},"min":{"1min":0.27800000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.41199999999999998,"15min":0.40400000000000003},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.46700000000000003}]},{"osd":1,"last update":"Sat Sep 9 23:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.57199999999999995,"15min":0.56100000000000005},"min":{"1min":0.48199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.57999999999999996,"15min":0.56699999999999995},"min":{"1min":0.42699999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.56999999999999995}]},{"osd":2,"last update":"Sat Sep 9 23:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.621,"15min":0.61299999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.70199999999999996}]},{"osd":3,"last update":"Sat Sep 9 23:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.438,"15min":0.435},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.81599999999999995,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.55100000000000005}]},{"osd":4,"last update":"Sat Sep 9 23:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.443,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.96899999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.58999999999999997,"15min":0.57999999999999996},"min":{"1min":0.46700000000000003,"5min":0.437,"15min":0.437},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.71799999999999997}]},{"osd":5,"last update":"Sat Sep 9 23:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0289999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67000000000000004,"15min":0.66100000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.72699999999999998}]},{"osd":7,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93400000000000005,"5min":1.032,"15min":1.032},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.81899999999999995}]},{"osd":8,"last update":"Sat Sep 9 23:11:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.73999999999999999}]},{"osd":9,"last update":"Sat Sep 9 23:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.68700000000000006}]},{"osd":10,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.79000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":880640,"data_stored":417647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.51600000000000001,"15min":0.498},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.51400000000000001,"15min":0.48999999999999999},"min":{"1min":0.35099999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.47699999999999998}]},{"osd":2,"last update":"Sat Sep 9 23:11:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.51300000000000001,"15min":0.499},"min":{"1min":0.33400000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.53000000000000003,"15min":0.51400000000000001},"min":{"1min":0.39100000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.80600000000000005}]},{"osd":3,"last update":"Sat Sep 9 23:11:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.47299999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.60599999999999998}]},{"osd":4,"last update":"Sat Sep 9 23:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.438,"15min":0.41399999999999998},"min":{"1min":0.30499999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.48599999999999999,"15min":0.44800000000000001},"min":{"1min":0.28499999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.56599999999999995}]},{"osd":5,"last update":"Sat Sep 9 23:11:34 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.38600000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.57999999999999996,"15min":0.56599999999999995},"min":{"1min":0.43099999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.86399999999999999}]},{"osd":6,"last update":"Sat Sep 9 23:11:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.58499999999999996,"15min":0.56499999999999995},"min":{"1min":0.45300000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.40699999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.58699999999999997}]},{"osd":7,"last update":"Sat Sep 9 23:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.53100000000000003,"15min":0.51400000000000001},"min":{"1min":0.376,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52800000000000002,"15min":0.52000000000000002},"min":{"1min":0.33300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.54900000000000004}]},{"osd":8,"last update":"Sat Sep 9 23:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.81799999999999995}]},{"osd":9,"last update":"Sat Sep 9 23:11:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.749},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.65600000000000003}]},{"osd":10,"last update":"Sat Sep 9 23:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.629}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738425,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":418213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 9 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.53100000000000003,"15min":0.51100000000000001},"min":{"1min":0.39700000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.78000000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51400000000000001,"15min":0.5},"min":{"1min":0.374,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71799999999999997,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.442}]},{"osd":2,"last update":"Sat Sep 9 23:11:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53600000000000003,"15min":0.52800000000000002},"min":{"1min":0.42399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.53700000000000003,"15min":0.52400000000000002},"min":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.62}]},{"osd":3,"last update":"Sat Sep 9 23:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.39300000000000002,"15min":0.38900000000000001},"min":{"1min":0.27900000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.41599999999999998,"15min":0.40999999999999998},"min":{"1min":0.33500000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.35999999999999999}]},{"osd":4,"last update":"Sat Sep 9 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.54900000000000004,"15min":0.53300000000000003},"min":{"1min":0.42899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82999999999999996,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.497},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.375,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.39400000000000002}]},{"osd":5,"last update":"Sat Sep 9 23:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58699999999999997,"15min":0.58299999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56699999999999995,"15min":0.56499999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.63}]},{"osd":6,"last update":"Sat Sep 9 23:11:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.51100000000000001,"15min":0.502},"min":{"1min":0.34300000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.51600000000000001,"15min":0.505},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.47199999999999998}]},{"osd":7,"last update":"Sat Sep 9 23:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.47599999999999998}]},{"osd":8,"last update":"Sat Sep 9 23:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.53200000000000003}]},{"osd":9,"last update":"Sat Sep 9 23:11:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.54300000000000004}]},{"osd":10,"last update":"Sat Sep 9 23:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.51300000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411377,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":884736,"data_stored":417681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52000000000000002,"15min":0.53100000000000003},"min":{"1min":0.32600000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81299999999999994,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.51200000000000001,"15min":0.51400000000000001},"min":{"1min":0.36599999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77600000000000002,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.39300000000000002}]},{"osd":1,"last update":"Sat Sep 9 23:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54400000000000004,"15min":0.53200000000000003},"min":{"1min":0.378,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77600000000000002,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.53100000000000003,"15min":0.50900000000000001},"min":{"1min":0.38,"5min":0.31,"15min":0.31},"max":{"1min":0.82199999999999995,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.53000000000000003}]},{"osd":3,"last update":"Sat Sep 9 23:11:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60299999999999998,"15min":0.60799999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85399999999999998,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.60599999999999998,"15min":0.61399999999999999},"min":{"1min":0.41699999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.86499999999999999,"5min":1.478,"15min":1.478},"last":0.55300000000000005}]},{"osd":4,"last update":"Sat Sep 9 23:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57299999999999995,"15min":0.57199999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.78000000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.626},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.85899999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.49099999999999999}]},{"osd":5,"last update":"Sat Sep 9 23:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.40600000000000003,"15min":0.39800000000000002},"min":{"1min":0.30199999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.505},{"interface":"front","average":{"1min":0.44,"5min":0.41899999999999998,"15min":0.41499999999999998},"min":{"1min":0.317,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.35899999999999999}]},{"osd":6,"last update":"Sat Sep 9 23:11:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.54300000000000004,"15min":0.52300000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.59899999999999998,"15min":0.58799999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.58699999999999997}]},{"osd":7,"last update":"Sat Sep 9 23:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.65700000000000003}]},{"osd":8,"last update":"Sat Sep 9 23:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.45400000000000001}]},{"osd":9,"last update":"Sat Sep 9 23:11:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.66000000000000003}]},{"osd":10,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.64000000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]}]},{"osd":3,"up_from":25,"seq":107374182444,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":872448,"data_stored":415808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.32700000000000001,"15min":0.313},"min":{"1min":0.28999999999999998,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.31},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.34899999999999998,"15min":0.33300000000000002},"min":{"1min":0.28699999999999998,"5min":0.222,"15min":0.222},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.28699999999999998}]},{"osd":1,"last update":"Sat Sep 9 23:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.53800000000000003,"15min":0.51600000000000001},"min":{"1min":0.39600000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.95799999999999996,"5min":1.091,"15min":1.091},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.34699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71099999999999997,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.48899999999999999}]},{"osd":2,"last update":"Sat Sep 9 23:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.53900000000000003,"15min":0.51900000000000002},"min":{"1min":0.441,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.75900000000000001,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.626},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53700000000000003,"15min":0.51700000000000002},"min":{"1min":0.44900000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.72799999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.58499999999999996}]},{"osd":4,"last update":"Sat Sep 9 23:11:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.51500000000000001,"15min":0.498},"min":{"1min":0.47299999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.52500000000000002,"15min":0.51300000000000001},"min":{"1min":0.41199999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.61299999999999999}]},{"osd":5,"last update":"Sat Sep 9 23:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.58899999999999997,"15min":0.58699999999999997},"min":{"1min":0.46200000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83699999999999997,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.61799999999999999,"15min":0.60599999999999998},"min":{"1min":0.51600000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95999999999999996,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.64800000000000002}]},{"osd":6,"last update":"Sat Sep 9 23:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.502,"15min":0.496},"min":{"1min":0.34899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.377},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.51800000000000002,"15min":0.50900000000000001},"min":{"1min":0.377,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.39500000000000002}]},{"osd":7,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66600000000000004,"15min":0.66400000000000003},"min":{"1min":0.45400000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69299999999999995,"15min":0.69799999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.501}]},{"osd":8,"last update":"Sat Sep 9 23:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.49199999999999999}]},{"osd":9,"last update":"Sat Sep 9 23:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.44},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.55600000000000005}]},{"osd":10,"last update":"Sat Sep 9 23:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.56899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]}]},{"osd":4,"up_from":32,"seq":137438953513,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":880640,"data_stored":416679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.503,"15min":0.47099999999999997},"min":{"1min":0.44900000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.52200000000000002,"15min":0.48799999999999999},"min":{"1min":0.41999999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.33300000000000002}]},{"osd":1,"last update":"Sat Sep 9 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.35899999999999999,"15min":0.33800000000000002},"min":{"1min":0.26100000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.52300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.38100000000000001,"15min":0.35699999999999998},"min":{"1min":0.27400000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.68999999999999995,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.48499999999999999}]},{"osd":2,"last update":"Sat Sep 9 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53000000000000003,"15min":0.51400000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.69399999999999995,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.54700000000000004,"15min":0.52200000000000002},"min":{"1min":0.377,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79700000000000004,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.78900000000000003}]},{"osd":3,"last update":"Sat Sep 9 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.52300000000000002,"15min":0.48599999999999999},"min":{"1min":0.47199999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.51700000000000002,"15min":0.49299999999999999},"min":{"1min":0.39200000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.45800000000000002}]},{"osd":5,"last update":"Sat Sep 9 23:11:33 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.873,"5min":0.879,"15min":0.879},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.56899999999999995,"15min":0.55100000000000005},"min":{"1min":0.495,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.90100000000000002}]},{"osd":6,"last update":"Sat Sep 9 23:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.63800000000000001,"15min":0.61499999999999999},"min":{"1min":0.56200000000000006,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.63500000000000001,"15min":0.61299999999999999},"min":{"1min":0.48999999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.56200000000000006}]},{"osd":7,"last update":"Sat Sep 9 23:12:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.83799999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.36699999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72399999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.51600000000000001}]},{"osd":8,"last update":"Sat Sep 9 23:11:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.81000000000000005}]},{"osd":9,"last update":"Sat Sep 9 23:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.69999999999999996}]},{"osd":10,"last update":"Sat Sep 9 23:12:20 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.57899999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]},{"osd":5,"up_from":37,"seq":158913789988,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28484,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716668,"statfs":{"total":95995035648,"available":95965868032,"internally_reserved":0,"allocated":1503232,"data_stored":1008641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.46800000000000003,"15min":0.45800000000000002},"min":{"1min":0.39100000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.70699999999999996,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.49299999999999999,"15min":0.48999999999999999},"min":{"1min":0.39200000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.60799999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.53400000000000003}]},{"osd":1,"last update":"Sat Sep 9 23:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.504,"15min":0.501},"min":{"1min":0.34599999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.50700000000000001,"15min":0.50600000000000001},"min":{"1min":0.36699999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.621}]},{"osd":2,"last update":"Sat Sep 9 23:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.374,"15min":0.36899999999999999},"min":{"1min":0.28999999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.28799999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.44800000000000001}]},{"osd":3,"last update":"Sat Sep 9 23:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.39400000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76100000000000001,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.75},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.52300000000000002,"15min":0.51700000000000002},"min":{"1min":0.34599999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.68500000000000005}]},{"osd":4,"last update":"Sat Sep 9 23:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.52800000000000002,"15min":0.52200000000000002},"min":{"1min":0.42099999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72999999999999998,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54000000000000004,"15min":0.53700000000000003},"min":{"1min":0.38800000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.51100000000000001}]},{"osd":6,"last update":"Sat Sep 9 23:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.54900000000000004,"15min":0.54300000000000004},"min":{"1min":0.41199999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.73099999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.59599999999999997,"15min":0.58799999999999997},"min":{"1min":0.46600000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.75900000000000001,"5min":0.998,"15min":0.998},"last":0.78000000000000003}]},{"osd":7,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.59599999999999997,"15min":0.58799999999999997},"min":{"1min":0.498,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79200000000000004,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82199999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.66300000000000003}]},{"osd":8,"last update":"Sat Sep 9 23:11:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.64200000000000002}]},{"osd":9,"last update":"Sat Sep 9 23:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.83299999999999996}]},{"osd":10,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.69699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-09T23:12:26.928 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-09T23:12:26.929 INFO:tasks.cephadm:Setting up client nodes... 2023-09-09T23:12:26.929 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-09-09T23:12:27.511 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:27.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:27 smithi170 ceph-mon[102832]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:12:27.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:27 smithi082 ceph-mon[98694]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:12:27.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:27 smithi098 ceph-mon[103124]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:12:27.751 INFO:journalctl@ceph.mds.b.smithi098.stdout:Sep 09 23:12:27 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mds-b[139333]: 2023-09-09T23:12:27.272+0000 7fee2a4a6700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2023-09-09T23:12:28.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:28 smithi082 ceph-mon[98694]: from='client.14862 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-09T23:12:28.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:28 smithi082 ceph-mon[98694]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-09T23:12:28.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:28 smithi082 ceph-mon[98694]: pgmap v260: 97 pgs: 97 active+clean; 623 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 11 KiB/s wr, 28 op/s 2023-09-09T23:12:28.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:28 smithi170 ceph-mon[102832]: from='client.14862 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-09T23:12:28.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:28 smithi170 ceph-mon[102832]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-09T23:12:28.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:28 smithi170 ceph-mon[102832]: pgmap v260: 97 pgs: 97 active+clean; 623 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 11 KiB/s wr, 28 op/s 2023-09-09T23:12:28.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.694+0000 7f37f7453700 1 -- 172.21.15.82:0/469482253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37f0103a80 msgr2=0x7f37f0105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:28.696 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.694+0000 7f37f7453700 1 --2- 172.21.15.82:0/469482253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37f0103a80 0x7f37f0105e70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f37ec009990 tx=0x7f37ec009ca0 comp rx=0 tx=0).stop 2023-09-09T23:12:28.696 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.694+0000 7f37f7453700 1 -- 172.21.15.82:0/469482253 shutdown_connections 2023-09-09T23:12:28.697 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.694+0000 7f37f7453700 1 --2- 172.21.15.82:0/469482253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37f0101150 0x7f37f0103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:28.697 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.694+0000 7f37f7453700 1 --2- 172.21.15.82:0/469482253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37f01063b0 0x7f37f0108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:28.697 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.694+0000 7f37f7453700 1 --2- 172.21.15.82:0/469482253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37f0103a80 0x7f37f0105e70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:28.698 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.694+0000 7f37f7453700 1 -- 172.21.15.82:0/469482253 >> 172.21.15.82:0/469482253 conn(0x7f37f00fab40 msgr2=0x7f37f00fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:28.698 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.694+0000 7f37f7453700 1 -- 172.21.15.82:0/469482253 shutdown_connections 2023-09-09T23:12:28.698 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.694+0000 7f37f7453700 1 -- 172.21.15.82:0/469482253 wait complete. 2023-09-09T23:12:28.699 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.694+0000 7f37f7453700 1 Processor -- start 2023-09-09T23:12:28.699 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.695+0000 7f37f7453700 1 -- start start 2023-09-09T23:12:28.699 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.695+0000 7f37f7453700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37f0101150 0x7f37f01962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:28.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.695+0000 7f37f7453700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37f0103a80 0x7f37f0196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:28.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.695+0000 7f37f7453700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37f01063b0 0x7f37f019d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:28.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.695+0000 7f37f7453700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f019de60 con 0x7f37f01063b0 2023-09-09T23:12:28.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.695+0000 7f37f7453700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f019dfa0 con 0x7f37f0101150 2023-09-09T23:12:28.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.695+0000 7f37f7453700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f019e130 con 0x7f37f0103a80 2023-09-09T23:12:28.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.695+0000 7f37f59f0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37f01063b0 0x7f37f019d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:28.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.695+0000 7f37f59f0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37f01063b0 0x7f37f019d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54664/0 (socket says 172.21.15.82:54664) 2023-09-09T23:12:28.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.695+0000 7f37f59f0700 1 -- 172.21.15.82:0/2716327371 learned_addr learned my addr 172.21.15.82:0/2716327371 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:28.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.695+0000 7f37f51ef700 1 --2- 172.21.15.82:0/2716327371 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37f0101150 0x7f37f01962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:28.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.695+0000 7f37f49ee700 1 --2- 172.21.15.82:0/2716327371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37f0103a80 0x7f37f0196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:28.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.695+0000 7f37f59f0700 1 -- 172.21.15.82:0/2716327371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37f0103a80 msgr2=0x7f37f0196810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:28.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.695+0000 7f37f59f0700 1 --2- 172.21.15.82:0/2716327371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37f0103a80 0x7f37f0196810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:28.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.695+0000 7f37f59f0700 1 -- 172.21.15.82:0/2716327371 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37f0101150 msgr2=0x7f37f01962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:28.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.695+0000 7f37f59f0700 1 --2- 172.21.15.82:0/2716327371 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37f0101150 0x7f37f01962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:28.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.695+0000 7f37f59f0700 1 -- 172.21.15.82:0/2716327371 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ec0095f0 con 0x7f37f01063b0 2023-09-09T23:12:28.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.695+0000 7f37f49ee700 1 --2- 172.21.15.82:0/2716327371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37f0103a80 0x7f37f0196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:12:28.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.695+0000 7f37f59f0700 1 --2- 172.21.15.82:0/2716327371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37f01063b0 0x7f37f019d890 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f37e8007d70 tx=0x7f37e800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:28.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.696+0000 7f37e67fc700 1 -- 172.21.15.82:0/2716327371 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e800ad00 con 0x7f37f01063b0 2023-09-09T23:12:28.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.696+0000 7f37f7453700 1 -- 172.21.15.82:0/2716327371 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f019e410 con 0x7f37f01063b0 2023-09-09T23:12:28.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.696+0000 7f37e67fc700 1 -- 172.21.15.82:0/2716327371 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37e800ae60 con 0x7f37f01063b0 2023-09-09T23:12:28.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.696+0000 7f37e67fc700 1 -- 172.21.15.82:0/2716327371 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e80116f0 con 0x7f37f01063b0 2023-09-09T23:12:28.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.696+0000 7f37f7453700 1 -- 172.21.15.82:0/2716327371 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f019e910 con 0x7f37f01063b0 2023-09-09T23:12:28.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.697+0000 7f37f7453700 1 -- 172.21.15.82:0/2716327371 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f00fc6c0 con 0x7f37f01063b0 2023-09-09T23:12:28.709 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.698+0000 7f37e67fc700 1 -- 172.21.15.82:0/2716327371 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f37e80129e0 con 0x7f37f01063b0 2023-09-09T23:12:28.709 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.698+0000 7f37e67fc700 1 --2- 172.21.15.82:0/2716327371 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f37dc0a0750 0x7f37dc0a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:28.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.698+0000 7f37e67fc700 1 -- 172.21.15.82:0/2716327371 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f37e80cbd00 con 0x7f37f01063b0 2023-09-09T23:12:28.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.698+0000 7f37f51ef700 1 --2- 172.21.15.82:0/2716327371 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f37dc0a0750 0x7f37dc0a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:28.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.699+0000 7f37f51ef700 1 --2- 172.21.15.82:0/2716327371 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f37dc0a0750 0x7f37dc0a2c10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f37e0005950 tx=0x7f37e00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:28.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.703+0000 7f37e67fc700 1 -- 172.21.15.82:0/2716327371 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f37e80cf030 con 0x7f37f01063b0 2023-09-09T23:12:28.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:28 smithi098 ceph-mon[103124]: from='client.14862 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-09T23:12:28.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:28 smithi098 ceph-mon[103124]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-09T23:12:28.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:28 smithi098 ceph-mon[103124]: pgmap v260: 97 pgs: 97 active+clean; 623 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 11 KiB/s wr, 28 op/s 2023-09-09T23:12:28.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.966+0000 7f37f7453700 1 -- 172.21.15.82:0/2716327371 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f37f010c0f0 con 0x7f37f01063b0 2023-09-09T23:12:28.967 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.969+0000 7f37e67fc700 1 -- 172.21.15.82:0/2716327371 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f37e8095990 con 0x7f37f01063b0 2023-09-09T23:12:28.967 INFO:teuthology.orchestra.run.smithi082.stdout:[client.0] 2023-09-09T23:12:28.968 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDc+/xk+qq7ORAALcv/kuVoliDIrS77Dp/aOA== 2023-09-09T23:12:28.970 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.971+0000 7f37f7453700 1 -- 172.21.15.82:0/2716327371 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f37dc0a0750 msgr2=0x7f37dc0a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:28.970 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.971+0000 7f37f7453700 1 --2- 172.21.15.82:0/2716327371 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f37dc0a0750 0x7f37dc0a2c10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f37e0005950 tx=0x7f37e00058e0 comp rx=0 tx=0).stop 2023-09-09T23:12:28.970 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.971+0000 7f37f7453700 1 -- 172.21.15.82:0/2716327371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37f01063b0 msgr2=0x7f37f019d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:28.970 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.971+0000 7f37f7453700 1 --2- 172.21.15.82:0/2716327371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37f01063b0 0x7f37f019d890 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f37e8007d70 tx=0x7f37e800a340 comp rx=0 tx=0).stop 2023-09-09T23:12:28.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.971+0000 7f37f7453700 1 -- 172.21.15.82:0/2716327371 shutdown_connections 2023-09-09T23:12:28.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.971+0000 7f37f7453700 1 --2- 172.21.15.82:0/2716327371 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f37dc0a0750 0x7f37dc0a2c10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:28.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.971+0000 7f37f7453700 1 --2- 172.21.15.82:0/2716327371 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37f0101150 0x7f37f01962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:28.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.971+0000 7f37f7453700 1 --2- 172.21.15.82:0/2716327371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37f01063b0 0x7f37f019d890 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:28.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.971+0000 7f37f7453700 1 --2- 172.21.15.82:0/2716327371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37f0103a80 0x7f37f0196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:28.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.971+0000 7f37f7453700 1 -- 172.21.15.82:0/2716327371 >> 172.21.15.82:0/2716327371 conn(0x7f37f00fab40 msgr2=0x7f37f0101dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:28.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.971+0000 7f37f7453700 1 -- 172.21.15.82:0/2716327371 shutdown_connections 2023-09-09T23:12:28.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:28.971+0000 7f37f7453700 1 -- 172.21.15.82:0/2716327371 wait complete. 2023-09-09T23:12:29.353 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-09-09T23:12:29.354 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-09-09T23:12:29.354 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-09-09T23:12:29.396 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph config log 1 --format=json 2023-09-09T23:12:29.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:29 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2716327371' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-09-09T23:12:29.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:29 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2716327371' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-09-09T23:12:29.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:29 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2716327371' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-09-09T23:12:29.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:29 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2716327371' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-09-09T23:12:29.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:29 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2716327371' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-09-09T23:12:29.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:29 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2716327371' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-09-09T23:12:29.987 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:30.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:30 smithi082 ceph-mon[98694]: pgmap v261: 97 pgs: 97 active+clean; 624 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 10 KiB/s wr, 26 op/s 2023-09-09T23:12:30.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:30 smithi170 ceph-mon[102832]: pgmap v261: 97 pgs: 97 active+clean; 624 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 10 KiB/s wr, 26 op/s 2023-09-09T23:12:30.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:30 smithi098 ceph-mon[103124]: pgmap v261: 97 pgs: 97 active+clean; 624 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 10 KiB/s wr, 26 op/s 2023-09-09T23:12:31.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.151+0000 7f9befc34700 1 -- 172.21.15.82:0/302359953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9be8100610 msgr2=0x7f9be8100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:31.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.151+0000 7f9befc34700 1 --2- 172.21.15.82:0/302359953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9be8100610 0x7f9be8100a90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8009990 tx=0x7f9bd8009ca0 comp rx=0 tx=0).stop 2023-09-09T23:12:31.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.151+0000 7f9befc34700 1 -- 172.21.15.82:0/302359953 shutdown_connections 2023-09-09T23:12:31.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.151+0000 7f9befc34700 1 --2- 172.21.15.82:0/302359953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9be8100fd0 0x7f9be810d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:31.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.151+0000 7f9befc34700 1 --2- 172.21.15.82:0/302359953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9be81066a0 0x7f9be8106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:31.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.151+0000 7f9befc34700 1 --2- 172.21.15.82:0/302359953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9be8100610 0x7f9be8100a90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:31.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.151+0000 7f9befc34700 1 -- 172.21.15.82:0/302359953 >> 172.21.15.82:0/302359953 conn(0x7f9be80fc0a0 msgr2=0x7f9be80fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:31.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.151+0000 7f9befc34700 1 -- 172.21.15.82:0/302359953 shutdown_connections 2023-09-09T23:12:31.153 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.151+0000 7f9befc34700 1 -- 172.21.15.82:0/302359953 wait complete. 2023-09-09T23:12:31.153 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.151+0000 7f9befc34700 1 Processor -- start 2023-09-09T23:12:31.153 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.151+0000 7f9befc34700 1 -- start start 2023-09-09T23:12:31.153 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.152+0000 7f9befc34700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9be8100610 0x7f9be81966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:31.153 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.152+0000 7f9befc34700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9be8100fd0 0x7f9be8196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:31.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.152+0000 7f9befc34700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9be81066a0 0x7f9be8199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:31.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.152+0000 7f9befc34700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be819a5a0 con 0x7f9be81066a0 2023-09-09T23:12:31.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.152+0000 7f9befc34700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be819a710 con 0x7f9be8100fd0 2023-09-09T23:12:31.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.152+0000 7f9befc34700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be819a8d0 con 0x7f9be8100610 2023-09-09T23:12:31.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.152+0000 7f9bee1d1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9be81066a0 0x7f9be8199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:31.155 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.152+0000 7f9bee1d1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9be81066a0 0x7f9be8199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54676/0 (socket says 172.21.15.82:54676) 2023-09-09T23:12:31.155 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.152+0000 7f9bee1d1700 1 -- 172.21.15.82:0/2420140192 learned_addr learned my addr 172.21.15.82:0/2420140192 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:31.155 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.152+0000 7f9bed1cf700 1 --2- 172.21.15.82:0/2420140192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9be8100fd0 0x7f9be8196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:31.155 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.152+0000 7f9bed9d0700 1 --2- 172.21.15.82:0/2420140192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9be8100610 0x7f9be81966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:31.155 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.152+0000 7f9bee1d1700 1 -- 172.21.15.82:0/2420140192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9be8100610 msgr2=0x7f9be81966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:31.156 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.152+0000 7f9bee1d1700 1 --2- 172.21.15.82:0/2420140192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9be8100610 0x7f9be81966a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:31.156 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.152+0000 7f9bee1d1700 1 -- 172.21.15.82:0/2420140192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9be8100fd0 msgr2=0x7f9be8196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:31.156 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.152+0000 7f9bee1d1700 1 --2- 172.21.15.82:0/2420140192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9be8100fd0 0x7f9be8196be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:31.156 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.152+0000 7f9bee1d1700 1 -- 172.21.15.82:0/2420140192 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd80095f0 con 0x7f9be81066a0 2023-09-09T23:12:31.156 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.152+0000 7f9bee1d1700 1 --2- 172.21.15.82:0/2420140192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9be81066a0 0x7f9be8199fa0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9be0007ee0 tx=0x7f9be000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:31.156 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.152+0000 7f9bdeffd700 1 -- 172.21.15.82:0/2420140192 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be0017070 con 0x7f9be81066a0 2023-09-09T23:12:31.157 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.152+0000 7f9bdeffd700 1 -- 172.21.15.82:0/2420140192 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9be000f040 con 0x7f9be81066a0 2023-09-09T23:12:31.157 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.152+0000 7f9befc34700 1 -- 172.21.15.82:0/2420140192 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be819abb0 con 0x7f9be81066a0 2023-09-09T23:12:31.157 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.152+0000 7f9bdeffd700 1 -- 172.21.15.82:0/2420140192 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be0005580 con 0x7f9be81066a0 2023-09-09T23:12:31.158 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.153+0000 7f9befc34700 1 -- 172.21.15.82:0/2420140192 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be81a3500 con 0x7f9be81066a0 2023-09-09T23:12:31.158 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.154+0000 7f9befc34700 1 -- 172.21.15.82:0/2420140192 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be804e300 con 0x7f9be81066a0 2023-09-09T23:12:31.159 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.155+0000 7f9bdeffd700 1 -- 172.21.15.82:0/2420140192 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f9be000db20 con 0x7f9be81066a0 2023-09-09T23:12:31.159 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.156+0000 7f9bdeffd700 1 --2- 172.21.15.82:0/2420140192 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f9bd40a0750 0x7f9bd40a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:31.159 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.156+0000 7f9bed9d0700 1 --2- 172.21.15.82:0/2420140192 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f9bd40a0750 0x7f9bd40a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:31.159 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.156+0000 7f9bdeffd700 1 -- 172.21.15.82:0/2420140192 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9be00cb8f0 con 0x7f9be81066a0 2023-09-09T23:12:31.159 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.156+0000 7f9bed9d0700 1 --2- 172.21.15.82:0/2420140192 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f9bd40a0750 0x7f9bd40a2c10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9be8101750 tx=0x7f9be400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:31.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.159+0000 7f9bdeffd700 1 -- 172.21.15.82:0/2420140192 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9be0095470 con 0x7f9be81066a0 2023-09-09T23:12:31.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.352+0000 7f9befc34700 1 -- 172.21.15.82:0/2420140192 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) v1 -- 0x7f9be81107a0 con 0x7f9be81066a0 2023-09-09T23:12:31.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.352+0000 7f9bdeffd700 1 -- 172.21.15.82:0/2420140192 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v43) v1 ==== 86+0+133 (secure 0 0 0) 0x7f9be0098a90 con 0x7f9be81066a0 2023-09-09T23:12:31.352 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:12:31.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.355+0000 7f9befc34700 1 -- 172.21.15.82:0/2420140192 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f9bd40a0750 msgr2=0x7f9bd40a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:31.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.355+0000 7f9befc34700 1 --2- 172.21.15.82:0/2420140192 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f9bd40a0750 0x7f9bd40a2c10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9be8101750 tx=0x7f9be400b410 comp rx=0 tx=0).stop 2023-09-09T23:12:31.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.355+0000 7f9befc34700 1 -- 172.21.15.82:0/2420140192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9be81066a0 msgr2=0x7f9be8199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:31.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.355+0000 7f9befc34700 1 --2- 172.21.15.82:0/2420140192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9be81066a0 0x7f9be8199fa0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9be0007ee0 tx=0x7f9be000d3b0 comp rx=0 tx=0).stop 2023-09-09T23:12:31.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.355+0000 7f9befc34700 1 -- 172.21.15.82:0/2420140192 shutdown_connections 2023-09-09T23:12:31.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.355+0000 7f9befc34700 1 --2- 172.21.15.82:0/2420140192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9be81066a0 0x7f9be8199fa0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:31.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.355+0000 7f9befc34700 1 --2- 172.21.15.82:0/2420140192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9be8100610 0x7f9be81966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:31.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.355+0000 7f9befc34700 1 --2- 172.21.15.82:0/2420140192 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f9bd40a0750 0x7f9bd40a2c10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:31.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.355+0000 7f9befc34700 1 --2- 172.21.15.82:0/2420140192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9be8100fd0 0x7f9be8196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:31.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.355+0000 7f9befc34700 1 -- 172.21.15.82:0/2420140192 >> 172.21.15.82:0/2420140192 conn(0x7f9be80fc0a0 msgr2=0x7f9be8104260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:31.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.355+0000 7f9befc34700 1 -- 172.21.15.82:0/2420140192 shutdown_connections 2023-09-09T23:12:31.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:31.355+0000 7f9befc34700 1 -- 172.21.15.82:0/2420140192 wait complete. 2023-09-09T23:12:31.740 INFO:teuthology.orchestra.run.smithi082.stdout:[{"version":43,"timestamp":"2023-09-09T23:12:11.812772+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2023-09-09T23:12:31.741 INFO:tasks.ceph_manager:config epoch is 43 2023-09-09T23:12:31.741 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-09-09T23:12:31.742 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-09-09T23:12:31.742 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph mgr dump --format=json 2023-09-09T23:12:32.296 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:32.552 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:32 smithi082 ceph-mon[98694]: pgmap v262: 97 pgs: 97 active+clean; 624 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.8 KiB/s wr, 24 op/s 2023-09-09T23:12:32.552 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:32 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2420140192' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-09-09T23:12:32.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:32 smithi170 ceph-mon[102832]: pgmap v262: 97 pgs: 97 active+clean; 624 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.8 KiB/s wr, 24 op/s 2023-09-09T23:12:32.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:32 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2420140192' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-09-09T23:12:32.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:32 smithi098 ceph-mon[103124]: pgmap v262: 97 pgs: 97 active+clean; 624 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.8 KiB/s wr, 24 op/s 2023-09-09T23:12:32.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:32 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2420140192' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-09-09T23:12:33.497 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.496+0000 7f78471e9700 1 -- 172.21.15.82:0/2279958401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7840100fe0 msgr2=0x7f784010d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:33.497 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.496+0000 7f78471e9700 1 --2- 172.21.15.82:0/2279958401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7840100fe0 0x7f784010d4f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f783800b0a0 tx=0x7f783800b3b0 comp rx=0 tx=0).stop 2023-09-09T23:12:33.497 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.496+0000 7f78471e9700 1 -- 172.21.15.82:0/2279958401 shutdown_connections 2023-09-09T23:12:33.497 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.496+0000 7f78471e9700 1 --2- 172.21.15.82:0/2279958401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7840106620 0x7f7840106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:33.498 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.496+0000 7f78471e9700 1 --2- 172.21.15.82:0/2279958401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7840100fe0 0x7f784010d4f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:33.498 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.496+0000 7f78471e9700 1 --2- 172.21.15.82:0/2279958401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7840100620 0x7f7840100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:33.498 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.496+0000 7f78471e9700 1 -- 172.21.15.82:0/2279958401 >> 172.21.15.82:0/2279958401 conn(0x7f78400fc0f0 msgr2=0x7f78400fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:33.498 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.496+0000 7f78471e9700 1 -- 172.21.15.82:0/2279958401 shutdown_connections 2023-09-09T23:12:33.498 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.496+0000 7f78471e9700 1 -- 172.21.15.82:0/2279958401 wait complete. 2023-09-09T23:12:33.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.497+0000 7f78471e9700 1 Processor -- start 2023-09-09T23:12:33.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.497+0000 7f78471e9700 1 -- start start 2023-09-09T23:12:33.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.497+0000 7f78471e9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7840100620 0x7f784019c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:33.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.497+0000 7f78471e9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7840100fe0 0x7f784019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:33.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.497+0000 7f78471e9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7840106620 0x7f78401965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:33.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.497+0000 7f78471e9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7840196c40 con 0x7f7840100620 2023-09-09T23:12:33.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.497+0000 7f78471e9700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7840196db0 con 0x7f7840100fe0 2023-09-09T23:12:33.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.497+0000 7f78471e9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7840196f70 con 0x7f7840106620 2023-09-09T23:12:33.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.497+0000 7f7844f85700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7840100620 0x7f784019c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:33.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.497+0000 7f7844f85700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7840100620 0x7f784019c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54686/0 (socket says 172.21.15.82:54686) 2023-09-09T23:12:33.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.497+0000 7f7844f85700 1 -- 172.21.15.82:0/4057972101 learned_addr learned my addr 172.21.15.82:0/4057972101 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:33.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.497+0000 7f7845786700 1 --2- 172.21.15.82:0/4057972101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7840106620 0x7f78401965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:33.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.497+0000 7f7844f85700 1 -- 172.21.15.82:0/4057972101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7840106620 msgr2=0x7f78401965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:33.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.497+0000 7f7844f85700 1 --2- 172.21.15.82:0/4057972101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7840106620 0x7f78401965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:33.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.497+0000 7f783ffff700 1 --2- 172.21.15.82:0/4057972101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7840100fe0 0x7f784019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:33.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.497+0000 7f7844f85700 1 -- 172.21.15.82:0/4057972101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7840100fe0 msgr2=0x7f784019ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:33.502 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.497+0000 7f7844f85700 1 --2- 172.21.15.82:0/4057972101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7840100fe0 0x7f784019ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:33.502 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.497+0000 7f7844f85700 1 -- 172.21.15.82:0/4057972101 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7838009d00 con 0x7f7840100620 2023-09-09T23:12:33.502 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.498+0000 7f7844f85700 1 --2- 172.21.15.82:0/4057972101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7840100620 0x7f784019c550 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f783400c9b0 tx=0x7f783400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:33.502 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.498+0000 7f783dffb700 1 -- 172.21.15.82:0/4057972101 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7834007b00 con 0x7f7840100620 2023-09-09T23:12:33.502 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.498+0000 7f783dffb700 1 -- 172.21.15.82:0/4057972101 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7834007c60 con 0x7f7840100620 2023-09-09T23:12:33.503 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.498+0000 7f783dffb700 1 -- 172.21.15.82:0/4057972101 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7834018660 con 0x7f7840100620 2023-09-09T23:12:33.503 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.498+0000 7f78471e9700 1 -- 172.21.15.82:0/4057972101 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7840197250 con 0x7f7840100620 2023-09-09T23:12:33.503 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.498+0000 7f78471e9700 1 -- 172.21.15.82:0/4057972101 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78401a3540 con 0x7f7840100620 2023-09-09T23:12:33.504 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.499+0000 7f78471e9700 1 -- 172.21.15.82:0/4057972101 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f784004e300 con 0x7f7840100620 2023-09-09T23:12:33.504 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.500+0000 7f783dffb700 1 -- 172.21.15.82:0/4057972101 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f7834018880 con 0x7f7840100620 2023-09-09T23:12:33.504 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.500+0000 7f783dffb700 1 --2- 172.21.15.82:0/4057972101 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f78300a0750 0x7f78300a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:33.504 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.500+0000 7f783ffff700 1 --2- 172.21.15.82:0/4057972101 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f78300a0750 0x7f78300a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:33.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.500+0000 7f783dffb700 1 -- 172.21.15.82:0/4057972101 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f78340cbaf0 con 0x7f7840100620 2023-09-09T23:12:33.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.501+0000 7f783ffff700 1 --2- 172.21.15.82:0/4057972101 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f78300a0750 0x7f78300a2c10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f782c005950 tx=0x7f782c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:33.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.505+0000 7f783dffb700 1 -- 172.21.15.82:0/4057972101 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7834095670 con 0x7f7840100620 2023-09-09T23:12:33.747 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.748+0000 7f78471e9700 1 -- 172.21.15.82:0/4057972101 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f78401105f0 con 0x7f7840100620 2023-09-09T23:12:33.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.753+0000 7f783dffb700 1 -- 172.21.15.82:0/4057972101 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+259019 (secure 0 0 0) 0x7f7834098c90 con 0x7f7840100620 2023-09-09T23:12:33.755 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:12:33.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.756+0000 7f78471e9700 1 -- 172.21.15.82:0/4057972101 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f78300a0750 msgr2=0x7f78300a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:33.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.756+0000 7f78471e9700 1 --2- 172.21.15.82:0/4057972101 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f78300a0750 0x7f78300a2c10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f782c005950 tx=0x7f782c0058e0 comp rx=0 tx=0).stop 2023-09-09T23:12:33.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.756+0000 7f78471e9700 1 -- 172.21.15.82:0/4057972101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7840100620 msgr2=0x7f784019c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:33.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.756+0000 7f78471e9700 1 --2- 172.21.15.82:0/4057972101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7840100620 0x7f784019c550 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f783400c9b0 tx=0x7f783400cd70 comp rx=0 tx=0).stop 2023-09-09T23:12:33.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.756+0000 7f78471e9700 1 -- 172.21.15.82:0/4057972101 shutdown_connections 2023-09-09T23:12:33.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.756+0000 7f78471e9700 1 --2- 172.21.15.82:0/4057972101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7840106620 0x7f78401965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:33.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.756+0000 7f78471e9700 1 --2- 172.21.15.82:0/4057972101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7840100620 0x7f784019c550 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:33.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.756+0000 7f78471e9700 1 --2- 172.21.15.82:0/4057972101 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f78300a0750 0x7f78300a2c10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:33.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.756+0000 7f78471e9700 1 --2- 172.21.15.82:0/4057972101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7840100fe0 0x7f784019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:33.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.756+0000 7f78471e9700 1 -- 172.21.15.82:0/4057972101 >> 172.21.15.82:0/4057972101 conn(0x7f78400fc0f0 msgr2=0x7f78400fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:33.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.756+0000 7f78471e9700 1 -- 172.21.15.82:0/4057972101 shutdown_connections 2023-09-09T23:12:33.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:33.757+0000 7f78471e9700 1 -- 172.21.15.82:0/4057972101 wait complete. 2023-09-09T23:12:34.153 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":17,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":1453069857},{"type":"v1","addr":"172.21.15.82:6801","nonce":1453069857}]},"active_addr":"172.21.15.82:6801/1453069857","active_change":"2023-09-09T23:06:08.903642+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24178,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24191,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.82:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":1770498756}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":2423813751}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":3103991753}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":2838103651}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":3664314932}]}]} 2023-09-09T23:12:34.160 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-09-09T23:12:34.160 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-09-09T23:12:34.160 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph osd dump --format=json 2023-09-09T23:12:34.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:34 smithi082 ceph-mon[98694]: pgmap v263: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 26 op/s 2023-09-09T23:12:34.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:34 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/4057972101' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-09-09T23:12:34.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:34 smithi170 ceph-mon[102832]: pgmap v263: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 26 op/s 2023-09-09T23:12:34.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:34 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/4057972101' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-09-09T23:12:34.716 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:34.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:34 smithi098 ceph-mon[103124]: pgmap v263: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.8 KiB/s wr, 26 op/s 2023-09-09T23:12:34.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:34 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/4057972101' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-09-09T23:12:35.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.920+0000 7f5b009e1700 1 -- 172.21.15.82:0/408186194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5afc101020 msgr2=0x7f5afc10d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:35.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.920+0000 7f5b009e1700 1 --2- 172.21.15.82:0/408186194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5afc101020 0x7f5afc10d530 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009990 tx=0x7f5aec009ca0 comp rx=0 tx=0).stop 2023-09-09T23:12:35.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.920+0000 7f5b009e1700 1 -- 172.21.15.82:0/408186194 shutdown_connections 2023-09-09T23:12:35.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.920+0000 7f5b009e1700 1 --2- 172.21.15.82:0/408186194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5afc106660 0x7f5afc106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:35.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.920+0000 7f5b009e1700 1 --2- 172.21.15.82:0/408186194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5afc101020 0x7f5afc10d530 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:35.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.920+0000 7f5b009e1700 1 --2- 172.21.15.82:0/408186194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5afc100660 0x7f5afc100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:35.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.920+0000 7f5b009e1700 1 -- 172.21.15.82:0/408186194 >> 172.21.15.82:0/408186194 conn(0x7f5afc0fc150 msgr2=0x7f5afc0fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:35.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.920+0000 7f5b009e1700 1 -- 172.21.15.82:0/408186194 shutdown_connections 2023-09-09T23:12:35.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.920+0000 7f5b009e1700 1 -- 172.21.15.82:0/408186194 wait complete. 2023-09-09T23:12:35.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.920+0000 7f5b009e1700 1 Processor -- start 2023-09-09T23:12:35.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.920+0000 7f5b009e1700 1 -- start start 2023-09-09T23:12:35.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.921+0000 7f5b009e1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5afc100660 0x7f5afc196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:35.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.921+0000 7f5b009e1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5afc101020 0x7f5afc196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:35.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.921+0000 7f5b009e1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5afc106660 0x7f5afc199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:35.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.921+0000 7f5b009e1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc19a580 con 0x7f5afc101020 2023-09-09T23:12:35.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.921+0000 7f5b009e1700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc19a6f0 con 0x7f5afc100660 2023-09-09T23:12:35.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.921+0000 7f5b009e1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc19a8b0 con 0x7f5afc106660 2023-09-09T23:12:35.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.921+0000 7f5af9d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5afc101020 0x7f5afc196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:35.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.921+0000 7f5af9d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5afc101020 0x7f5afc196bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54700/0 (socket says 172.21.15.82:54700) 2023-09-09T23:12:35.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.921+0000 7f5af9d9b700 1 -- 172.21.15.82:0/261399644 learned_addr learned my addr 172.21.15.82:0/261399644 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:35.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.921+0000 7f5afad9d700 1 --2- 172.21.15.82:0/261399644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5afc106660 0x7f5afc199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:35.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.921+0000 7f5afa59c700 1 --2- 172.21.15.82:0/261399644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5afc100660 0x7f5afc196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:35.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.921+0000 7f5af9d9b700 1 -- 172.21.15.82:0/261399644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5afc106660 msgr2=0x7f5afc199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:35.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.921+0000 7f5af9d9b700 1 --2- 172.21.15.82:0/261399644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5afc106660 0x7f5afc199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:35.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.921+0000 7f5af9d9b700 1 -- 172.21.15.82:0/261399644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5afc100660 msgr2=0x7f5afc196680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:35.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.921+0000 7f5af9d9b700 1 --2- 172.21.15.82:0/261399644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5afc100660 0x7f5afc196680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:35.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.921+0000 7f5af9d9b700 1 -- 172.21.15.82:0/261399644 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec0095f0 con 0x7f5afc101020 2023-09-09T23:12:35.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.921+0000 7f5afad9d700 1 --2- 172.21.15.82:0/261399644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5afc106660 0x7f5afc199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:12:35.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.921+0000 7f5afa59c700 1 --2- 172.21.15.82:0/261399644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5afc100660 0x7f5afc196680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:12:35.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.921+0000 7f5af9d9b700 1 --2- 172.21.15.82:0/261399644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5afc101020 0x7f5afc196bc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5af000d950 tx=0x7f5af000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:35.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.921+0000 7f5aeb7fe700 1 -- 172.21.15.82:0/261399644 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0009b40 con 0x7f5afc101020 2023-09-09T23:12:35.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.921+0000 7f5aeb7fe700 1 -- 172.21.15.82:0/261399644 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5af0009ca0 con 0x7f5afc101020 2023-09-09T23:12:35.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.921+0000 7f5b009e1700 1 -- 172.21.15.82:0/261399644 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5afc19ab90 con 0x7f5afc101020 2023-09-09T23:12:35.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.921+0000 7f5aeb7fe700 1 -- 172.21.15.82:0/261399644 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af000f620 con 0x7f5afc101020 2023-09-09T23:12:35.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.921+0000 7f5b009e1700 1 -- 172.21.15.82:0/261399644 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5afc1a34e0 con 0x7f5afc101020 2023-09-09T23:12:35.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.922+0000 7f5b009e1700 1 -- 172.21.15.82:0/261399644 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5afc04e300 con 0x7f5afc101020 2023-09-09T23:12:35.933 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.924+0000 7f5aeb7fe700 1 -- 172.21.15.82:0/261399644 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f5af0010460 con 0x7f5afc101020 2023-09-09T23:12:35.933 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.924+0000 7f5aeb7fe700 1 --2- 172.21.15.82:0/261399644 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5ae40a06d0 0x7f5ae40a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:35.934 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.924+0000 7f5afa59c700 1 --2- 172.21.15.82:0/261399644 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5ae40a06d0 0x7f5ae40a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:35.934 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.924+0000 7f5aeb7fe700 1 -- 172.21.15.82:0/261399644 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5af00cbf80 con 0x7f5afc101020 2023-09-09T23:12:35.934 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.924+0000 7f5afa59c700 1 --2- 172.21.15.82:0/261399644 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5ae40a06d0 0x7f5ae40a2b90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5af4009c50 tx=0x7f5af4009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:35.934 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:35.928+0000 7f5aeb7fe700 1 -- 172.21.15.82:0/261399644 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5af0095b00 con 0x7f5afc101020 2023-09-09T23:12:36.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:36.116+0000 7f5b009e1700 1 -- 172.21.15.82:0/261399644 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5afc1979e0 con 0x7f5afc101020 2023-09-09T23:12:36.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:36.118+0000 7f5aeb7fe700 1 -- 172.21.15.82:0/261399644 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22071 (secure 0 0 0) 0x7f5af0020090 con 0x7f5afc101020 2023-09-09T23:12:36.117 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:12:36.117 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":81,"fsid":"3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da","created":"2023-09-09T23:05:14.222190+0000","modified":"2023-09-09T23:12:18.894339+0000","last_up_change":"2023-09-09T23:11:37.253759+0000","last_in_change":"2023-09-09T23:11:19.933812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-09T23:08:31.234958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-09T23:12:15.524887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-09T23:12:16.785403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1277eb17-a2e7-4e10-9832-f12a5e93ffde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6803","nonce":3197434929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6805","nonce":3197434929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6809","nonce":3197434929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6807","nonce":3197434929}]},"public_addr":"172.21.15.82:6803/3197434929","cluster_addr":"172.21.15.82:6805/3197434929","heartbeat_back_addr":"172.21.15.82:6809/3197434929","heartbeat_front_addr":"172.21.15.82:6807/3197434929","state":["exists","up"]},{"osd":1,"uuid":"f68723fd-0908-4e21-ae8e-1de1c3a9a65c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6801","nonce":3528641314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6803","nonce":3528641314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6807","nonce":3528641314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6805","nonce":3528641314}]},"public_addr":"172.21.15.98:6801/3528641314","cluster_addr":"172.21.15.98:6803/3528641314","heartbeat_back_addr":"172.21.15.98:6807/3528641314","heartbeat_front_addr":"172.21.15.98:6805/3528641314","state":["exists","up"]},{"osd":2,"uuid":"06f89c53-1419-4f52-acad-6e92916b476f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6801","nonce":4006414551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6803","nonce":4006414551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6807","nonce":4006414551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6805","nonce":4006414551}]},"public_addr":"172.21.15.170:6801/4006414551","cluster_addr":"172.21.15.170:6803/4006414551","heartbeat_back_addr":"172.21.15.170:6807/4006414551","heartbeat_front_addr":"172.21.15.170:6805/4006414551","state":["exists","up"]},{"osd":3,"uuid":"fedbd4c9-6852-42b5-8f3a-ba8d9bfe5d59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6811","nonce":3647629241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6813","nonce":3647629241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6817","nonce":3647629241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6815","nonce":3647629241}]},"public_addr":"172.21.15.82:6811/3647629241","cluster_addr":"172.21.15.82:6813/3647629241","heartbeat_back_addr":"172.21.15.82:6817/3647629241","heartbeat_front_addr":"172.21.15.82:6815/3647629241","state":["exists","up"]},{"osd":4,"uuid":"a71f3084-6919-4161-87bf-345f3355d8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6809","nonce":2507035643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6811","nonce":2507035643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6815","nonce":2507035643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6813","nonce":2507035643}]},"public_addr":"172.21.15.98:6809/2507035643","cluster_addr":"172.21.15.98:6811/2507035643","heartbeat_back_addr":"172.21.15.98:6815/2507035643","heartbeat_front_addr":"172.21.15.98:6813/2507035643","state":["exists","up"]},{"osd":5,"uuid":"f76058e9-a90e-476e-87f0-3c6723795e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6809","nonce":775005168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6811","nonce":775005168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6815","nonce":775005168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6813","nonce":775005168}]},"public_addr":"172.21.15.170:6809/775005168","cluster_addr":"172.21.15.170:6811/775005168","heartbeat_back_addr":"172.21.15.170:6815/775005168","heartbeat_front_addr":"172.21.15.170:6813/775005168","state":["exists","up"]},{"osd":6,"uuid":"25940be0-c1bd-4c72-95bf-fd8367aa2f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6819","nonce":3008405982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6821","nonce":3008405982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6825","nonce":3008405982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6823","nonce":3008405982}]},"public_addr":"172.21.15.82:6819/3008405982","cluster_addr":"172.21.15.82:6821/3008405982","heartbeat_back_addr":"172.21.15.82:6825/3008405982","heartbeat_front_addr":"172.21.15.82:6823/3008405982","state":["exists","up"]},{"osd":7,"uuid":"f651bb8c-0b25-42d5-b028-58563f4c27f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6817","nonce":3547615284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6819","nonce":3547615284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6823","nonce":3547615284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6821","nonce":3547615284}]},"public_addr":"172.21.15.98:6817/3547615284","cluster_addr":"172.21.15.98:6819/3547615284","heartbeat_back_addr":"172.21.15.98:6823/3547615284","heartbeat_front_addr":"172.21.15.98:6821/3547615284","state":["exists","up"]},{"osd":8,"uuid":"23b27c86-8c98-441e-b7ab-d57bc584e58f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6817","nonce":632038848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6819","nonce":632038848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6823","nonce":632038848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6821","nonce":632038848}]},"public_addr":"172.21.15.170:6817/632038848","cluster_addr":"172.21.15.170:6819/632038848","heartbeat_back_addr":"172.21.15.170:6823/632038848","heartbeat_front_addr":"172.21.15.170:6821/632038848","state":["exists","up"]},{"osd":9,"uuid":"383b8989-4922-4eee-9c77-f198258ac243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6827","nonce":3354242311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6829","nonce":3354242311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6833","nonce":3354242311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6831","nonce":3354242311}]},"public_addr":"172.21.15.82:6827/3354242311","cluster_addr":"172.21.15.82:6829/3354242311","heartbeat_back_addr":"172.21.15.82:6833/3354242311","heartbeat_front_addr":"172.21.15.82:6831/3354242311","state":["exists","up"]},{"osd":10,"uuid":"33a59389-7ced-42d5-bf54-2071317132d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6825","nonce":2069114959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6827","nonce":2069114959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6831","nonce":2069114959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6829","nonce":2069114959}]},"public_addr":"172.21.15.98:6825/2069114959","cluster_addr":"172.21.15.98:6827/2069114959","heartbeat_back_addr":"172.21.15.98:6831/2069114959","heartbeat_front_addr":"172.21.15.98:6829/2069114959","state":["exists","up"]},{"osd":11,"uuid":"b7a71f5c-85e2-4fec-be4b-95be0788b302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6825","nonce":146964512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6827","nonce":146964512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6831","nonce":146964512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6829","nonce":146964512}]},"public_addr":"172.21.15.170:6825/146964512","cluster_addr":"172.21.15.170:6827/146964512","heartbeat_back_addr":"172.21.15.170:6831/146964512","heartbeat_front_addr":"172.21.15.170:6829/146964512","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:07:47.563043+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:08.359260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:29.070374+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:49.897775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:11.434248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:31.538464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:52.377109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:13.452968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:33.806897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:54.083177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:14.980087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:35.690089+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4013747788":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6800/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/2823341464":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/75116806":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/1037625632":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/3142088817":"2023-09-10T23:05:46.550354+0000","172.21.15.82:6800/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/1679622963":"2023-09-10T23:05:46.550354+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-09T23:12:36.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:36.120+0000 7f5b009e1700 1 -- 172.21.15.82:0/261399644 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5ae40a06d0 msgr2=0x7f5ae40a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:36.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:36.120+0000 7f5b009e1700 1 --2- 172.21.15.82:0/261399644 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5ae40a06d0 0x7f5ae40a2b90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5af4009c50 tx=0x7f5af4009040 comp rx=0 tx=0).stop 2023-09-09T23:12:36.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:36.120+0000 7f5b009e1700 1 -- 172.21.15.82:0/261399644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5afc101020 msgr2=0x7f5afc196bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:36.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:36.120+0000 7f5b009e1700 1 --2- 172.21.15.82:0/261399644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5afc101020 0x7f5afc196bc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5af000d950 tx=0x7f5af000dd10 comp rx=0 tx=0).stop 2023-09-09T23:12:36.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:36.120+0000 7f5b009e1700 1 -- 172.21.15.82:0/261399644 shutdown_connections 2023-09-09T23:12:36.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:36.120+0000 7f5b009e1700 1 --2- 172.21.15.82:0/261399644 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5ae40a06d0 0x7f5ae40a2b90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:36.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:36.120+0000 7f5b009e1700 1 --2- 172.21.15.82:0/261399644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5afc100660 0x7f5afc196680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:36.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:36.120+0000 7f5b009e1700 1 --2- 172.21.15.82:0/261399644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5afc106660 0x7f5afc199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:36.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:36.120+0000 7f5b009e1700 1 --2- 172.21.15.82:0/261399644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5afc101020 0x7f5afc196bc0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:36.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:36.120+0000 7f5b009e1700 1 -- 172.21.15.82:0/261399644 >> 172.21.15.82:0/261399644 conn(0x7f5afc0fc150 msgr2=0x7f5afc104170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:36.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:36.120+0000 7f5b009e1700 1 -- 172.21.15.82:0/261399644 shutdown_connections 2023-09-09T23:12:36.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:36.120+0000 7f5b009e1700 1 -- 172.21.15.82:0/261399644 wait complete. 2023-09-09T23:12:36.439 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:36 smithi082 ceph-mon[98694]: pgmap v264: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.7 KiB/s wr, 24 op/s 2023-09-09T23:12:36.440 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:36 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/261399644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:12:36.459 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-09-09T23:12:36.460 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph osd dump --format=json 2023-09-09T23:12:36.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:36 smithi170 ceph-mon[102832]: pgmap v264: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.7 KiB/s wr, 24 op/s 2023-09-09T23:12:36.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:36 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/261399644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:12:36.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:36 smithi098 ceph-mon[103124]: pgmap v264: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.7 KiB/s wr, 24 op/s 2023-09-09T23:12:36.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:36 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/261399644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:12:37.012 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:38.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.229+0000 7fcfaec3e700 1 -- 172.21.15.82:0/4179149272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfa80ffc00 msgr2=0x7fcfa8100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:38.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.229+0000 7fcfaec3e700 1 --2- 172.21.15.82:0/4179149272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfa80ffc00 0x7fcfa8100080 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcf94009990 tx=0x7fcf94009ca0 comp rx=0 tx=0).stop 2023-09-09T23:12:38.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.229+0000 7fcfaec3e700 1 -- 172.21.15.82:0/4179149272 shutdown_connections 2023-09-09T23:12:38.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.229+0000 7fcfaec3e700 1 --2- 172.21.15.82:0/4179149272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfa8105540 0x7fcfa8105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:38.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.229+0000 7fcfaec3e700 1 --2- 172.21.15.82:0/4179149272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfa81005c0 0x7fcfa810c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:38.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.229+0000 7fcfaec3e700 1 --2- 172.21.15.82:0/4179149272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfa80ffc00 0x7fcfa8100080 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:38.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.229+0000 7fcfaec3e700 1 -- 172.21.15.82:0/4179149272 >> 172.21.15.82:0/4179149272 conn(0x7fcfa8075a70 msgr2=0x7fcfa80fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:38.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.229+0000 7fcfaec3e700 1 -- 172.21.15.82:0/4179149272 shutdown_connections 2023-09-09T23:12:38.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.229+0000 7fcfaec3e700 1 -- 172.21.15.82:0/4179149272 wait complete. 2023-09-09T23:12:38.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.230+0000 7fcfaec3e700 1 Processor -- start 2023-09-09T23:12:38.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.230+0000 7fcfaec3e700 1 -- start start 2023-09-09T23:12:38.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.230+0000 7fcfaec3e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfa80ffc00 0x7fcfa819b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:38.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.230+0000 7fcfaec3e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfa81005c0 0x7fcfa819b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:38.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.230+0000 7fcfaec3e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfa8105540 0x7fcfa8195500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:38.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.230+0000 7fcfaec3e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa8195b60 con 0x7fcfa8105540 2023-09-09T23:12:38.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.230+0000 7fcfaec3e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa8195ca0 con 0x7fcfa81005c0 2023-09-09T23:12:38.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.230+0000 7fcfaec3e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa8195e60 con 0x7fcfa80ffc00 2023-09-09T23:12:38.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.230+0000 7fcfad1db700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfa8105540 0x7fcfa8195500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:38.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.230+0000 7fcfad1db700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfa8105540 0x7fcfa8195500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54712/0 (socket says 172.21.15.82:54712) 2023-09-09T23:12:38.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.230+0000 7fcfad1db700 1 -- 172.21.15.82:0/3992258035 learned_addr learned my addr 172.21.15.82:0/3992258035 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:38.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.230+0000 7fcfac9da700 1 --2- 172.21.15.82:0/3992258035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfa80ffc00 0x7fcfa819b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:38.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.230+0000 7fcfa7fff700 1 --2- 172.21.15.82:0/3992258035 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfa81005c0 0x7fcfa819b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:38.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.230+0000 7fcfad1db700 1 -- 172.21.15.82:0/3992258035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfa80ffc00 msgr2=0x7fcfa819b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:38.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.230+0000 7fcfad1db700 1 --2- 172.21.15.82:0/3992258035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfa80ffc00 0x7fcfa819b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:38.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.230+0000 7fcfad1db700 1 -- 172.21.15.82:0/3992258035 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfa81005c0 msgr2=0x7fcfa819b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:38.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.230+0000 7fcfad1db700 1 --2- 172.21.15.82:0/3992258035 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfa81005c0 0x7fcfa819b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:38.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.230+0000 7fcfad1db700 1 -- 172.21.15.82:0/3992258035 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf940095f0 con 0x7fcfa8105540 2023-09-09T23:12:38.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.230+0000 7fcfad1db700 1 --2- 172.21.15.82:0/3992258035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfa8105540 0x7fcfa8195500 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000b970 tx=0x7fcfa000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:38.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.231+0000 7fcfa5ffb700 1 -- 172.21.15.82:0/3992258035 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa000c930 con 0x7fcfa8105540 2023-09-09T23:12:38.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.231+0000 7fcfa5ffb700 1 -- 172.21.15.82:0/3992258035 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcfa0014440 con 0x7fcfa8105540 2023-09-09T23:12:38.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.231+0000 7fcfaec3e700 1 -- 172.21.15.82:0/3992258035 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa8196140 con 0x7fcfa8105540 2023-09-09T23:12:38.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.231+0000 7fcfa5ffb700 1 -- 172.21.15.82:0/3992258035 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa00075d0 con 0x7fcfa8105540 2023-09-09T23:12:38.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.231+0000 7fcfaec3e700 1 -- 172.21.15.82:0/3992258035 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa81a24c0 con 0x7fcfa8105540 2023-09-09T23:12:38.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.232+0000 7fcfaec3e700 1 -- 172.21.15.82:0/3992258035 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa804e300 con 0x7fcfa8105540 2023-09-09T23:12:38.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.233+0000 7fcfa5ffb700 1 -- 172.21.15.82:0/3992258035 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fcfa00145b0 con 0x7fcfa8105540 2023-09-09T23:12:38.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.233+0000 7fcfa5ffb700 1 --2- 172.21.15.82:0/3992258035 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcf980a0750 0x7fcf980a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:38.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.233+0000 7fcfac9da700 1 --2- 172.21.15.82:0/3992258035 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcf980a0750 0x7fcf980a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:38.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.233+0000 7fcfa5ffb700 1 -- 172.21.15.82:0/3992258035 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcfa0006e40 con 0x7fcfa8105540 2023-09-09T23:12:38.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.234+0000 7fcfac9da700 1 --2- 172.21.15.82:0/3992258035 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcf980a0750 0x7fcf980a2c10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c009a10 tx=0x7fcf9c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:38.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.237+0000 7fcfa5ffb700 1 -- 172.21.15.82:0/3992258035 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fcfa004ee10 con 0x7fcfa8105540 2023-09-09T23:12:38.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.429+0000 7fcfaec3e700 1 -- 172.21.15.82:0/3992258035 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfa810f610 con 0x7fcfa8105540 2023-09-09T23:12:38.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.431+0000 7fcfa5ffb700 1 -- 172.21.15.82:0/3992258035 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22071 (secure 0 0 0) 0x7fcfa0094d30 con 0x7fcfa8105540 2023-09-09T23:12:38.430 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:12:38.430 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":81,"fsid":"3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da","created":"2023-09-09T23:05:14.222190+0000","modified":"2023-09-09T23:12:18.894339+0000","last_up_change":"2023-09-09T23:11:37.253759+0000","last_in_change":"2023-09-09T23:11:19.933812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-09T23:08:31.234958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-09T23:12:15.524887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-09T23:12:16.785403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1277eb17-a2e7-4e10-9832-f12a5e93ffde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6803","nonce":3197434929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6805","nonce":3197434929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6809","nonce":3197434929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6807","nonce":3197434929}]},"public_addr":"172.21.15.82:6803/3197434929","cluster_addr":"172.21.15.82:6805/3197434929","heartbeat_back_addr":"172.21.15.82:6809/3197434929","heartbeat_front_addr":"172.21.15.82:6807/3197434929","state":["exists","up"]},{"osd":1,"uuid":"f68723fd-0908-4e21-ae8e-1de1c3a9a65c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6801","nonce":3528641314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6803","nonce":3528641314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6807","nonce":3528641314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6805","nonce":3528641314}]},"public_addr":"172.21.15.98:6801/3528641314","cluster_addr":"172.21.15.98:6803/3528641314","heartbeat_back_addr":"172.21.15.98:6807/3528641314","heartbeat_front_addr":"172.21.15.98:6805/3528641314","state":["exists","up"]},{"osd":2,"uuid":"06f89c53-1419-4f52-acad-6e92916b476f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6801","nonce":4006414551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6803","nonce":4006414551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6807","nonce":4006414551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6805","nonce":4006414551}]},"public_addr":"172.21.15.170:6801/4006414551","cluster_addr":"172.21.15.170:6803/4006414551","heartbeat_back_addr":"172.21.15.170:6807/4006414551","heartbeat_front_addr":"172.21.15.170:6805/4006414551","state":["exists","up"]},{"osd":3,"uuid":"fedbd4c9-6852-42b5-8f3a-ba8d9bfe5d59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6811","nonce":3647629241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6813","nonce":3647629241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6817","nonce":3647629241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6815","nonce":3647629241}]},"public_addr":"172.21.15.82:6811/3647629241","cluster_addr":"172.21.15.82:6813/3647629241","heartbeat_back_addr":"172.21.15.82:6817/3647629241","heartbeat_front_addr":"172.21.15.82:6815/3647629241","state":["exists","up"]},{"osd":4,"uuid":"a71f3084-6919-4161-87bf-345f3355d8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6809","nonce":2507035643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6811","nonce":2507035643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6815","nonce":2507035643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6813","nonce":2507035643}]},"public_addr":"172.21.15.98:6809/2507035643","cluster_addr":"172.21.15.98:6811/2507035643","heartbeat_back_addr":"172.21.15.98:6815/2507035643","heartbeat_front_addr":"172.21.15.98:6813/2507035643","state":["exists","up"]},{"osd":5,"uuid":"f76058e9-a90e-476e-87f0-3c6723795e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6809","nonce":775005168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6811","nonce":775005168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6815","nonce":775005168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6813","nonce":775005168}]},"public_addr":"172.21.15.170:6809/775005168","cluster_addr":"172.21.15.170:6811/775005168","heartbeat_back_addr":"172.21.15.170:6815/775005168","heartbeat_front_addr":"172.21.15.170:6813/775005168","state":["exists","up"]},{"osd":6,"uuid":"25940be0-c1bd-4c72-95bf-fd8367aa2f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6819","nonce":3008405982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6821","nonce":3008405982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6825","nonce":3008405982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6823","nonce":3008405982}]},"public_addr":"172.21.15.82:6819/3008405982","cluster_addr":"172.21.15.82:6821/3008405982","heartbeat_back_addr":"172.21.15.82:6825/3008405982","heartbeat_front_addr":"172.21.15.82:6823/3008405982","state":["exists","up"]},{"osd":7,"uuid":"f651bb8c-0b25-42d5-b028-58563f4c27f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6817","nonce":3547615284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6819","nonce":3547615284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6823","nonce":3547615284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6821","nonce":3547615284}]},"public_addr":"172.21.15.98:6817/3547615284","cluster_addr":"172.21.15.98:6819/3547615284","heartbeat_back_addr":"172.21.15.98:6823/3547615284","heartbeat_front_addr":"172.21.15.98:6821/3547615284","state":["exists","up"]},{"osd":8,"uuid":"23b27c86-8c98-441e-b7ab-d57bc584e58f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6817","nonce":632038848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6819","nonce":632038848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6823","nonce":632038848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6821","nonce":632038848}]},"public_addr":"172.21.15.170:6817/632038848","cluster_addr":"172.21.15.170:6819/632038848","heartbeat_back_addr":"172.21.15.170:6823/632038848","heartbeat_front_addr":"172.21.15.170:6821/632038848","state":["exists","up"]},{"osd":9,"uuid":"383b8989-4922-4eee-9c77-f198258ac243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6827","nonce":3354242311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6829","nonce":3354242311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6833","nonce":3354242311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6831","nonce":3354242311}]},"public_addr":"172.21.15.82:6827/3354242311","cluster_addr":"172.21.15.82:6829/3354242311","heartbeat_back_addr":"172.21.15.82:6833/3354242311","heartbeat_front_addr":"172.21.15.82:6831/3354242311","state":["exists","up"]},{"osd":10,"uuid":"33a59389-7ced-42d5-bf54-2071317132d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6825","nonce":2069114959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6827","nonce":2069114959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6831","nonce":2069114959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6829","nonce":2069114959}]},"public_addr":"172.21.15.98:6825/2069114959","cluster_addr":"172.21.15.98:6827/2069114959","heartbeat_back_addr":"172.21.15.98:6831/2069114959","heartbeat_front_addr":"172.21.15.98:6829/2069114959","state":["exists","up"]},{"osd":11,"uuid":"b7a71f5c-85e2-4fec-be4b-95be0788b302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6825","nonce":146964512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6827","nonce":146964512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6831","nonce":146964512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6829","nonce":146964512}]},"public_addr":"172.21.15.170:6825/146964512","cluster_addr":"172.21.15.170:6827/146964512","heartbeat_back_addr":"172.21.15.170:6831/146964512","heartbeat_front_addr":"172.21.15.170:6829/146964512","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:07:47.563043+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:08.359260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:29.070374+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:49.897775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:11.434248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:31.538464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:52.377109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:13.452968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:33.806897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:54.083177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:14.980087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:35.690089+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4013747788":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6800/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/2823341464":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/75116806":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/1037625632":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/3142088817":"2023-09-10T23:05:46.550354+0000","172.21.15.82:6800/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/1679622963":"2023-09-10T23:05:46.550354+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-09T23:12:38.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.433+0000 7fcfaec3e700 1 -- 172.21.15.82:0/3992258035 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcf980a0750 msgr2=0x7fcf980a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:38.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.433+0000 7fcfaec3e700 1 --2- 172.21.15.82:0/3992258035 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcf980a0750 0x7fcf980a2c10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c009a10 tx=0x7fcf9c009190 comp rx=0 tx=0).stop 2023-09-09T23:12:38.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.433+0000 7fcfaec3e700 1 -- 172.21.15.82:0/3992258035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfa8105540 msgr2=0x7fcfa8195500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:38.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.433+0000 7fcfaec3e700 1 --2- 172.21.15.82:0/3992258035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfa8105540 0x7fcfa8195500 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000b970 tx=0x7fcfa000bc80 comp rx=0 tx=0).stop 2023-09-09T23:12:38.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.433+0000 7fcfaec3e700 1 -- 172.21.15.82:0/3992258035 shutdown_connections 2023-09-09T23:12:38.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.433+0000 7fcfaec3e700 1 --2- 172.21.15.82:0/3992258035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfa8105540 0x7fcfa8195500 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:38.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.433+0000 7fcfaec3e700 1 --2- 172.21.15.82:0/3992258035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfa80ffc00 0x7fcfa819b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:38.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.433+0000 7fcfaec3e700 1 --2- 172.21.15.82:0/3992258035 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcf980a0750 0x7fcf980a2c10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:38.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.433+0000 7fcfaec3e700 1 --2- 172.21.15.82:0/3992258035 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfa81005c0 0x7fcfa819b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:38.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.433+0000 7fcfaec3e700 1 -- 172.21.15.82:0/3992258035 >> 172.21.15.82:0/3992258035 conn(0x7fcfa8075a70 msgr2=0x7fcfa80fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:38.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.433+0000 7fcfaec3e700 1 -- 172.21.15.82:0/3992258035 shutdown_connections 2023-09-09T23:12:38.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:38.433+0000 7fcfaec3e700 1 -- 172.21.15.82:0/3992258035 wait complete. 2023-09-09T23:12:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:38 smithi082 ceph-mon[98694]: pgmap v265: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 KiB/s wr, 23 op/s 2023-09-09T23:12:38.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:38 smithi170 ceph-mon[102832]: pgmap v265: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 KiB/s wr, 23 op/s 2023-09-09T23:12:38.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:38 smithi098 ceph-mon[103124]: pgmap v265: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 KiB/s wr, 23 op/s 2023-09-09T23:12:38.870 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph tell osd.0 flush_pg_stats 2023-09-09T23:12:38.871 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph tell osd.1 flush_pg_stats 2023-09-09T23:12:38.871 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph tell osd.2 flush_pg_stats 2023-09-09T23:12:38.872 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph tell osd.3 flush_pg_stats 2023-09-09T23:12:38.873 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph tell osd.4 flush_pg_stats 2023-09-09T23:12:38.874 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph tell osd.5 flush_pg_stats 2023-09-09T23:12:38.874 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph tell osd.6 flush_pg_stats 2023-09-09T23:12:38.875 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph tell osd.7 flush_pg_stats 2023-09-09T23:12:38.875 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph tell osd.8 flush_pg_stats 2023-09-09T23:12:38.876 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph tell osd.9 flush_pg_stats 2023-09-09T23:12:38.876 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph tell osd.10 flush_pg_stats 2023-09-09T23:12:38.877 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph tell osd.11 flush_pg_stats 2023-09-09T23:12:39.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:39 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3992258035' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:12:39.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:39 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3992258035' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:12:39.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:39 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3992258035' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:12:40.249 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:40.288 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:40.364 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:40.365 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:40.413 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:40.583 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:40.670 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:40 smithi082 ceph-mon[98694]: pgmap v266: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2023-09-09T23:12:40.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:40 smithi170 ceph-mon[102832]: pgmap v266: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2023-09-09T23:12:40.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:40 smithi098 ceph-mon[103124]: pgmap v266: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2023-09-09T23:12:40.754 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:40.754 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:41.021 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:41.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:41 smithi170 ceph-mon[102832]: pgmap v267: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-09-09T23:12:41.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:41 smithi082 ceph-mon[98694]: pgmap v267: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-09-09T23:12:41.739 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:41.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:41 smithi098 ceph-mon[103124]: pgmap v267: 97 pgs: 97 active+clean; 634 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-09-09T23:12:41.882 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:42.195 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:43.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.332+0000 7fad56c69700 1 -- 172.21.15.82:0/4278732329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad50106770 msgr2=0x7fad50106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:43.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.332+0000 7fad56c69700 1 --2- 172.21.15.82:0/4278732329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad50106770 0x7fad50106b50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fad44009990 tx=0x7fad44009ca0 comp rx=0 tx=0).stop 2023-09-09T23:12:43.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.332+0000 7fad56c69700 1 -- 172.21.15.82:0/4278732329 shutdown_connections 2023-09-09T23:12:43.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.332+0000 7fad56c69700 1 --2- 172.21.15.82:0/4278732329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad50068ae0 0x7fad5010d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:43.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.332+0000 7fad56c69700 1 --2- 172.21.15.82:0/4278732329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad50106770 0x7fad50106b50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:43.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.332+0000 7fad56c69700 1 --2- 172.21.15.82:0/4278732329 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad50068120 0x7fad500685a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:43.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.332+0000 7fad56c69700 1 -- 172.21.15.82:0/4278732329 >> 172.21.15.82:0/4278732329 conn(0x7fad50076600 msgr2=0x7fad50076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:43.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.332+0000 7fad56c69700 1 -- 172.21.15.82:0/4278732329 shutdown_connections 2023-09-09T23:12:43.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.332+0000 7fad56c69700 1 -- 172.21.15.82:0/4278732329 wait complete. 2023-09-09T23:12:43.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.332+0000 7fad56c69700 1 Processor -- start 2023-09-09T23:12:43.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.333+0000 7fad56c69700 1 -- start start 2023-09-09T23:12:43.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.333+0000 7fad56c69700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad50068120 0x7fad501966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:43.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.333+0000 7fad56c69700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad50068ae0 0x7fad50196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:43.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.333+0000 7fad56c69700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad50106770 0x7fad50199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:43.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.333+0000 7fad56c69700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5019a5d0 con 0x7fad50068120 2023-09-09T23:12:43.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.333+0000 7fad56c69700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5019a740 con 0x7fad50068ae0 2023-09-09T23:12:43.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.333+0000 7fad56c69700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5019a900 con 0x7fad50106770 2023-09-09T23:12:43.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.333+0000 7fad54a05700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad50068120 0x7fad501966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:43.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.333+0000 7fad54a05700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad50068120 0x7fad501966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54722/0 (socket says 172.21.15.82:54722) 2023-09-09T23:12:43.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.333+0000 7fad54a05700 1 -- 172.21.15.82:0/1613002161 learned_addr learned my addr 172.21.15.82:0/1613002161 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:43.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.333+0000 7fad55206700 1 --2- 172.21.15.82:0/1613002161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad50106770 0x7fad50199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:43.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.333+0000 7fad4ffff700 1 --2- 172.21.15.82:0/1613002161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad50068ae0 0x7fad50196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:43.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.333+0000 7fad54a05700 1 -- 172.21.15.82:0/1613002161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad50106770 msgr2=0x7fad50199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:43.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.333+0000 7fad54a05700 1 --2- 172.21.15.82:0/1613002161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad50106770 0x7fad50199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:43.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.333+0000 7fad54a05700 1 -- 172.21.15.82:0/1613002161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad50068ae0 msgr2=0x7fad50196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:43.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.333+0000 7fad54a05700 1 --2- 172.21.15.82:0/1613002161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad50068ae0 0x7fad50196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:43.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.333+0000 7fad54a05700 1 -- 172.21.15.82:0/1613002161 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad440095f0 con 0x7fad50068120 2023-09-09T23:12:43.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.333+0000 7fad4ffff700 1 --2- 172.21.15.82:0/1613002161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad50068ae0 0x7fad50196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:12:43.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.333+0000 7fad54a05700 1 --2- 172.21.15.82:0/1613002161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad50068120 0x7fad501966d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fad440098e0 tx=0x7fad4400f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:43.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.333+0000 7fad4dffb700 1 -- 172.21.15.82:0/1613002161 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4401d070 con 0x7fad50068120 2023-09-09T23:12:43.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.333+0000 7fad4dffb700 1 -- 172.21.15.82:0/1613002161 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad44005510 con 0x7fad50068120 2023-09-09T23:12:43.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.333+0000 7fad4dffb700 1 -- 172.21.15.82:0/1613002161 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad44004bd0 con 0x7fad50068120 2023-09-09T23:12:43.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.333+0000 7fad56c69700 1 -- 172.21.15.82:0/1613002161 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad5019ab80 con 0x7fad50068120 2023-09-09T23:12:43.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.334+0000 7fad56c69700 1 -- 172.21.15.82:0/1613002161 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad501a3360 con 0x7fad50068120 2023-09-09T23:12:43.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.334+0000 7fad56c69700 1 -- 172.21.15.82:0/1613002161 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fad501108e0 con 0x7fad50068120 2023-09-09T23:12:43.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.336+0000 7fad4dffb700 1 -- 172.21.15.82:0/1613002161 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fad44004d30 con 0x7fad50068120 2023-09-09T23:12:43.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.336+0000 7fad4dffb700 1 --2- 172.21.15.82:0/1613002161 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fad400a06d0 0x7fad400a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:43.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.336+0000 7fad4dffb700 1 -- 172.21.15.82:0/1613002161 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fad440ccdd0 con 0x7fad50068120 2023-09-09T23:12:43.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.336+0000 7fad4ffff700 1 --2- 172.21.15.82:0/1613002161 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fad400a06d0 0x7fad400a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:43.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.336+0000 7fad4dffb700 1 --2- 172.21.15.82:0/1613002161 >> [v2:172.21.15.170:6800/4006414551,v1:172.21.15.170:6801/4006414551] conn(0x7fad400a9eb0 0x7fad400ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:43.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.336+0000 7fad4dffb700 1 -- 172.21.15.82:0/1613002161 --> [v2:172.21.15.170:6800/4006414551,v1:172.21.15.170:6801/4006414551] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad400ac980 con 0x7fad400a9eb0 2023-09-09T23:12:43.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.336+0000 7fad4dffb700 1 -- 172.21.15.82:0/1613002161 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad440cd270 con 0x7fad50068120 2023-09-09T23:12:43.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.337+0000 7fad4ffff700 1 --2- 172.21.15.82:0/1613002161 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fad400a06d0 0x7fad400a2b90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fad3c005fd0 tx=0x7fad3c005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:43.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.337+0000 7fad55206700 1 --2- 172.21.15.82:0/1613002161 >> [v2:172.21.15.170:6800/4006414551,v1:172.21.15.170:6801/4006414551] conn(0x7fad400a9eb0 0x7fad400ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:43.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.337+0000 7fad55206700 1 --2- 172.21.15.82:0/1613002161 >> [v2:172.21.15.170:6800/4006414551,v1:172.21.15.170:6801/4006414551] conn(0x7fad400a9eb0 0x7fad400ac2d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:43.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.338+0000 7fad4dffb700 1 -- 172.21.15.82:0/1613002161 <== osd.2 v2:172.21.15.170:6800/4006414551 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fad400ac980 con 0x7fad400a9eb0 2023-09-09T23:12:43.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.354+0000 7fad56c69700 1 -- 172.21.15.82:0/1613002161 --> [v2:172.21.15.170:6800/4006414551,v1:172.21.15.170:6801/4006414551] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fad5004e300 con 0x7fad400a9eb0 2023-09-09T23:12:43.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.354+0000 7fad4dffb700 1 -- 172.21.15.82:0/1613002161 <== osd.2 v2:172.21.15.170:6800/4006414551 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fad5004e300 con 0x7fad400a9eb0 2023-09-09T23:12:43.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.355+0000 7fad56c69700 1 -- 172.21.15.82:0/1613002161 >> [v2:172.21.15.170:6800/4006414551,v1:172.21.15.170:6801/4006414551] conn(0x7fad400a9eb0 msgr2=0x7fad400ac2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:43.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.355+0000 7fad56c69700 1 --2- 172.21.15.82:0/1613002161 >> [v2:172.21.15.170:6800/4006414551,v1:172.21.15.170:6801/4006414551] conn(0x7fad400a9eb0 0x7fad400ac2d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:43.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.355+0000 7fad56c69700 1 -- 172.21.15.82:0/1613002161 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fad400a06d0 msgr2=0x7fad400a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:43.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.355+0000 7fad56c69700 1 --2- 172.21.15.82:0/1613002161 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fad400a06d0 0x7fad400a2b90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fad3c005fd0 tx=0x7fad3c005dc0 comp rx=0 tx=0).stop 2023-09-09T23:12:43.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.355+0000 7fad56c69700 1 -- 172.21.15.82:0/1613002161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad50068120 msgr2=0x7fad501966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:43.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.355+0000 7fad56c69700 1 --2- 172.21.15.82:0/1613002161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad50068120 0x7fad501966d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fad440098e0 tx=0x7fad4400f670 comp rx=0 tx=0).stop 2023-09-09T23:12:43.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.355+0000 7fad55206700 1 -- 172.21.15.82:0/1613002161 reap_dead start 2023-09-09T23:12:43.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.355+0000 7fad56c69700 1 -- 172.21.15.82:0/1613002161 shutdown_connections 2023-09-09T23:12:43.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.355+0000 7fad56c69700 1 -- 172.21.15.82:0/1613002161 >> 172.21.15.82:0/1613002161 conn(0x7fad50076600 msgr2=0x7fad500fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:43.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.355+0000 7fad56c69700 1 -- 172.21.15.82:0/1613002161 shutdown_connections 2023-09-09T23:12:43.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:43.355+0000 7fad56c69700 1 -- 172.21.15.82:0/1613002161 wait complete. 2023-09-09T23:12:44.452 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:44 smithi082 ceph-mon[98694]: pgmap v268: 97 pgs: 97 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-09-09T23:12:44.511 INFO:teuthology.orchestra.run.smithi082.stdout:77309411382 2023-09-09T23:12:44.512 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.2 2023-09-09T23:12:44.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:44 smithi170 ceph-mon[102832]: pgmap v268: 97 pgs: 97 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-09-09T23:12:44.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:44 smithi098 ceph-mon[103124]: pgmap v268: 97 pgs: 97 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-09-09T23:12:44.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.555+0000 7f001bfa8700 1 -- 172.21.15.82:0/3096792560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0014068120 msgr2=0x7f00140685a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:44.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.555+0000 7f001bfa8700 1 --2- 172.21.15.82:0/3096792560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0014068120 0x7f00140685a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0004009990 tx=0x7f0004009ca0 comp rx=0 tx=0).stop 2023-09-09T23:12:44.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.556+0000 7f001bfa8700 1 -- 172.21.15.82:0/3096792560 shutdown_connections 2023-09-09T23:12:44.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.556+0000 7f001bfa8700 1 --2- 172.21.15.82:0/3096792560 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0014106770 0x7f0014106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:44.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.556+0000 7f001bfa8700 1 --2- 172.21.15.82:0/3096792560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0014068ae0 0x7f001410d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:44.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.556+0000 7f001bfa8700 1 --2- 172.21.15.82:0/3096792560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0014068120 0x7f00140685a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:44.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.556+0000 7f001bfa8700 1 -- 172.21.15.82:0/3096792560 >> 172.21.15.82:0/3096792560 conn(0x7f0014076600 msgr2=0x7f0014076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:44.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.556+0000 7f001bfa8700 1 -- 172.21.15.82:0/3096792560 shutdown_connections 2023-09-09T23:12:44.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.556+0000 7f001bfa8700 1 -- 172.21.15.82:0/3096792560 wait complete. 2023-09-09T23:12:44.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.557+0000 7f001bfa8700 1 Processor -- start 2023-09-09T23:12:44.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.557+0000 7f001bfa8700 1 -- start start 2023-09-09T23:12:44.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.558+0000 7f001bfa8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0014068120 0x7f001419c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:44.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.558+0000 7f001bfa8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0014068ae0 0x7f001419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:44.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.558+0000 7f001bfa8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0014106770 0x7f0014196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:44.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.558+0000 7f001bfa8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0014196be0 con 0x7f0014106770 2023-09-09T23:12:44.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.558+0000 7f001bfa8700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0014196d50 con 0x7f0014068120 2023-09-09T23:12:44.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.558+0000 7f001bfa8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0014196f10 con 0x7f0014068ae0 2023-09-09T23:12:44.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.558+0000 7f001a545700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0014106770 0x7f0014196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:44.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.558+0000 7f001a545700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0014106770 0x7f0014196570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54740/0 (socket says 172.21.15.82:54740) 2023-09-09T23:12:44.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.558+0000 7f001a545700 1 -- 172.21.15.82:0/1226251770 learned_addr learned my addr 172.21.15.82:0/1226251770 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:44.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.558+0000 7f0019543700 1 --2- 172.21.15.82:0/1226251770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0014068ae0 0x7f001419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:44.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.558+0000 7f0019d44700 1 --2- 172.21.15.82:0/1226251770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0014068120 0x7f001419c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:44.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.558+0000 7f001a545700 1 -- 172.21.15.82:0/1226251770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0014068ae0 msgr2=0x7f001419ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:44.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.558+0000 7f001a545700 1 --2- 172.21.15.82:0/1226251770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0014068ae0 0x7f001419ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:44.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.558+0000 7f001a545700 1 -- 172.21.15.82:0/1226251770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0014068120 msgr2=0x7f001419c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:44.786 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.558+0000 7f001a545700 1 --2- 172.21.15.82:0/1226251770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0014068120 0x7f001419c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:44.786 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.558+0000 7f001a545700 1 -- 172.21.15.82:0/1226251770 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00040095f0 con 0x7f0014106770 2023-09-09T23:12:44.786 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.558+0000 7f0019d44700 1 --2- 172.21.15.82:0/1226251770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0014068120 0x7f001419c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:12:44.786 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.558+0000 7f001a545700 1 --2- 172.21.15.82:0/1226251770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0014106770 0x7f0014196570 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f000c00efc0 tx=0x7f000c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:44.786 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.558+0000 7f000affd700 1 -- 172.21.15.82:0/1226251770 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c018070 con 0x7f0014106770 2023-09-09T23:12:44.787 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.559+0000 7f001bfa8700 1 -- 172.21.15.82:0/1226251770 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0014197190 con 0x7f0014106770 2023-09-09T23:12:44.787 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.559+0000 7f000affd700 1 -- 172.21.15.82:0/1226251770 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f000c00f040 con 0x7f0014106770 2023-09-09T23:12:44.787 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.559+0000 7f000affd700 1 -- 172.21.15.82:0/1226251770 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c00ccc0 con 0x7f0014106770 2023-09-09T23:12:44.787 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.559+0000 7f001bfa8700 1 -- 172.21.15.82:0/1226251770 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00141a34e0 con 0x7f0014106770 2023-09-09T23:12:44.787 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.559+0000 7f001bfa8700 1 -- 172.21.15.82:0/1226251770 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f00141108e0 con 0x7f0014106770 2023-09-09T23:12:44.787 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.562+0000 7f000affd700 1 -- 172.21.15.82:0/1226251770 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f000c003a00 con 0x7f0014106770 2023-09-09T23:12:44.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.562+0000 7f000affd700 1 --2- 172.21.15.82:0/1226251770 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f00000a06d0 0x7f00000a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:44.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.562+0000 7f000affd700 1 -- 172.21.15.82:0/1226251770 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f000c0cbfa0 con 0x7f0014106770 2023-09-09T23:12:44.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.562+0000 7f000affd700 1 --2- 172.21.15.82:0/1226251770 >> [v2:172.21.15.82:6818/3008405982,v1:172.21.15.82:6819/3008405982] conn(0x7f00000a9eb0 0x7f00000ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:44.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.562+0000 7f000affd700 1 -- 172.21.15.82:0/1226251770 --> [v2:172.21.15.82:6818/3008405982,v1:172.21.15.82:6819/3008405982] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f00000ac980 con 0x7f00000a9eb0 2023-09-09T23:12:44.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.562+0000 7f000affd700 1 -- 172.21.15.82:0/1226251770 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f000c0cc440 con 0x7f0014106770 2023-09-09T23:12:44.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.563+0000 7f0019543700 1 --2- 172.21.15.82:0/1226251770 >> [v2:172.21.15.82:6818/3008405982,v1:172.21.15.82:6819/3008405982] conn(0x7f00000a9eb0 0x7f00000ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:44.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.563+0000 7f0019d44700 1 --2- 172.21.15.82:0/1226251770 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f00000a06d0 0x7f00000a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:44.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.563+0000 7f0019d44700 1 --2- 172.21.15.82:0/1226251770 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f00000a06d0 0x7f00000a2b90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f001000bb90 tx=0x7f0010005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:44.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.563+0000 7f0019543700 1 --2- 172.21.15.82:0/1226251770 >> [v2:172.21.15.82:6818/3008405982,v1:172.21.15.82:6819/3008405982] conn(0x7f00000a9eb0 0x7f00000ac2d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:44.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.564+0000 7f000affd700 1 -- 172.21.15.82:0/1226251770 <== osd.6 v2:172.21.15.82:6818/3008405982 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f00000ac980 con 0x7f00000a9eb0 2023-09-09T23:12:44.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.587+0000 7f001bfa8700 1 -- 172.21.15.82:0/1226251770 --> [v2:172.21.15.82:6818/3008405982,v1:172.21.15.82:6819/3008405982] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f001404e300 con 0x7f00000a9eb0 2023-09-09T23:12:44.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.587+0000 7f000affd700 1 -- 172.21.15.82:0/1226251770 <== osd.6 v2:172.21.15.82:6818/3008405982 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f001404e300 con 0x7f00000a9eb0 2023-09-09T23:12:44.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.587+0000 7f001bfa8700 1 -- 172.21.15.82:0/1226251770 >> [v2:172.21.15.82:6818/3008405982,v1:172.21.15.82:6819/3008405982] conn(0x7f00000a9eb0 msgr2=0x7f00000ac2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:44.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.587+0000 7f001bfa8700 1 --2- 172.21.15.82:0/1226251770 >> [v2:172.21.15.82:6818/3008405982,v1:172.21.15.82:6819/3008405982] conn(0x7f00000a9eb0 0x7f00000ac2d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:44.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.588+0000 7f001bfa8700 1 -- 172.21.15.82:0/1226251770 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f00000a06d0 msgr2=0x7f00000a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:44.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.588+0000 7f001bfa8700 1 --2- 172.21.15.82:0/1226251770 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f00000a06d0 0x7f00000a2b90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f001000bb90 tx=0x7f0010005d80 comp rx=0 tx=0).stop 2023-09-09T23:12:44.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.588+0000 7f001bfa8700 1 -- 172.21.15.82:0/1226251770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0014106770 msgr2=0x7f0014196570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:44.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.588+0000 7f001bfa8700 1 --2- 172.21.15.82:0/1226251770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0014106770 0x7f0014196570 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f000c00efc0 tx=0x7f000c00c3a0 comp rx=0 tx=0).stop 2023-09-09T23:12:44.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.588+0000 7f001a545700 1 -- 172.21.15.82:0/1226251770 reap_dead start 2023-09-09T23:12:44.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.588+0000 7f001bfa8700 1 -- 172.21.15.82:0/1226251770 shutdown_connections 2023-09-09T23:12:44.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.588+0000 7f001bfa8700 1 -- 172.21.15.82:0/1226251770 >> 172.21.15.82:0/1226251770 conn(0x7f0014076600 msgr2=0x7f00141042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:44.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.588+0000 7f001bfa8700 1 -- 172.21.15.82:0/1226251770 shutdown_connections 2023-09-09T23:12:44.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:44.588+0000 7f001bfa8700 1 -- 172.21.15.82:0/1226251770 wait complete. 2023-09-09T23:12:46.334 INFO:teuthology.orchestra.run.smithi082.stdout:193273528357 2023-09-09T23:12:46.334 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.6 2023-09-09T23:12:46.502 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:46 smithi082 ceph-mon[98694]: pgmap v269: 97 pgs: 97 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 426 B/s wr, 10 op/s 2023-09-09T23:12:46.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:46 smithi170 ceph-mon[102832]: pgmap v269: 97 pgs: 97 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 426 B/s wr, 10 op/s 2023-09-09T23:12:46.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:46 smithi098 ceph-mon[103124]: pgmap v269: 97 pgs: 97 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 426 B/s wr, 10 op/s 2023-09-09T23:12:47.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.003+0000 7fbbc850e700 1 -- 172.21.15.82:0/3900030220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbc010d060 msgr2=0x7fbbc010f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:47.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.004+0000 7fbbc850e700 1 --2- 172.21.15.82:0/3900030220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbc010d060 0x7fbbc010f450 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800b0a0 tx=0x7fbbb800b3b0 comp rx=0 tx=0).stop 2023-09-09T23:12:47.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.004+0000 7fbbc850e700 1 -- 172.21.15.82:0/3900030220 shutdown_connections 2023-09-09T23:12:47.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.004+0000 7fbbc850e700 1 --2- 172.21.15.82:0/3900030220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbc010d060 0x7fbbc010f450 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:47.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.004+0000 7fbbc850e700 1 --2- 172.21.15.82:0/3900030220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbc0068a80 0x7fbbc0068e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:47.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.004+0000 7fbbc850e700 1 --2- 172.21.15.82:0/3900030220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbc00693a0 0x7fbbc010cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:47.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.004+0000 7fbbc850e700 1 -- 172.21.15.82:0/3900030220 >> 172.21.15.82:0/3900030220 conn(0x7fbbc0076680 msgr2=0x7fbbc0076a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:47.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.004+0000 7fbbc850e700 1 -- 172.21.15.82:0/3900030220 shutdown_connections 2023-09-09T23:12:47.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.004+0000 7fbbc850e700 1 -- 172.21.15.82:0/3900030220 wait complete. 2023-09-09T23:12:47.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.004+0000 7fbbc850e700 1 Processor -- start 2023-09-09T23:12:47.340 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.004+0000 7fbbc850e700 1 -- start start 2023-09-09T23:12:47.340 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.005+0000 7fbbc850e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbc0068a80 0x7fbbc019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:47.340 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.005+0000 7fbbc850e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbc00693a0 0x7fbbc019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:47.340 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.005+0000 7fbbc850e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbc010d060 0x7fbbc0198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:47.340 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.005+0000 7fbbc850e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc0198e70 con 0x7fbbc0068a80 2023-09-09T23:12:47.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.005+0000 7fbbc850e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc0198fe0 con 0x7fbbc00693a0 2023-09-09T23:12:47.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.005+0000 7fbbc850e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc01991a0 con 0x7fbbc010d060 2023-09-09T23:12:47.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.005+0000 7fbbc62aa700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbc0068a80 0x7fbbc019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:47.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.005+0000 7fbbc62aa700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbc0068a80 0x7fbbc019e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54750/0 (socket says 172.21.15.82:54750) 2023-09-09T23:12:47.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.005+0000 7fbbc62aa700 1 -- 172.21.15.82:0/1378271718 learned_addr learned my addr 172.21.15.82:0/1378271718 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:47.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.005+0000 7fbbc5aa9700 1 --2- 172.21.15.82:0/1378271718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbc00693a0 0x7fbbc019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:47.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.005+0000 7fbbc62aa700 1 -- 172.21.15.82:0/1378271718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbc010d060 msgr2=0x7fbbc0198800 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-09T23:12:47.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.005+0000 7fbbc62aa700 1 --2- 172.21.15.82:0/1378271718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbc010d060 0x7fbbc0198800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:47.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.005+0000 7fbbc62aa700 1 -- 172.21.15.82:0/1378271718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbc00693a0 msgr2=0x7fbbc019ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:47.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.005+0000 7fbbc62aa700 1 --2- 172.21.15.82:0/1378271718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbc00693a0 0x7fbbc019ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:47.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.005+0000 7fbbc62aa700 1 -- 172.21.15.82:0/1378271718 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb8009d00 con 0x7fbbc0068a80 2023-09-09T23:12:47.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.005+0000 7fbbc62aa700 1 --2- 172.21.15.82:0/1378271718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbc0068a80 0x7fbbc019e780 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc00d950 tx=0x7fbbbc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:47.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.005+0000 7fbbb77fe700 1 -- 172.21.15.82:0/1378271718 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc009b40 con 0x7fbbc0068a80 2023-09-09T23:12:47.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.005+0000 7fbbc850e700 1 -- 172.21.15.82:0/1378271718 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbc0199480 con 0x7fbbc0068a80 2023-09-09T23:12:47.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.005+0000 7fbbb77fe700 1 -- 172.21.15.82:0/1378271718 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbbc009ca0 con 0x7fbbc0068a80 2023-09-09T23:12:47.344 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.005+0000 7fbbb77fe700 1 -- 172.21.15.82:0/1378271718 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc00f620 con 0x7fbbc0068a80 2023-09-09T23:12:47.344 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.006+0000 7fbbc850e700 1 -- 172.21.15.82:0/1378271718 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc01a5770 con 0x7fbbc0068a80 2023-09-09T23:12:47.344 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.006+0000 7fbbc850e700 1 -- 172.21.15.82:0/1378271718 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbbc004e300 con 0x7fbbc0068a80 2023-09-09T23:12:47.344 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.008+0000 7fbbb77fe700 1 -- 172.21.15.82:0/1378271718 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fbbbc010460 con 0x7fbbc0068a80 2023-09-09T23:12:47.344 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.009+0000 7fbbb77fe700 1 --2- 172.21.15.82:0/1378271718 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fbbac0a06d0 0x7fbbac0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:47.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.009+0000 7fbbc5aa9700 1 --2- 172.21.15.82:0/1378271718 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fbbac0a06d0 0x7fbbac0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:47.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.009+0000 7fbbb77fe700 1 -- 172.21.15.82:0/1378271718 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbbbc0cbbb0 con 0x7fbbc0068a80 2023-09-09T23:12:47.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.009+0000 7fbbb77fe700 1 --2- 172.21.15.82:0/1378271718 >> [v2:172.21.15.82:6802/3197434929,v1:172.21.15.82:6803/3197434929] conn(0x7fbbac0a9eb0 0x7fbbac0ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:47.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.009+0000 7fbbb77fe700 1 -- 172.21.15.82:0/1378271718 --> [v2:172.21.15.82:6802/3197434929,v1:172.21.15.82:6803/3197434929] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbbac0ac980 con 0x7fbbac0a9eb0 2023-09-09T23:12:47.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.009+0000 7fbbb77fe700 1 -- 172.21.15.82:0/1378271718 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbbc0ce2c0 con 0x7fbbc0068a80 2023-09-09T23:12:47.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.009+0000 7fbbc5aa9700 1 --2- 172.21.15.82:0/1378271718 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fbbac0a06d0 0x7fbbac0a2b90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0100bb0 tx=0x7fbbb000bef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:47.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.009+0000 7fbbc6aab700 1 --2- 172.21.15.82:0/1378271718 >> [v2:172.21.15.82:6802/3197434929,v1:172.21.15.82:6803/3197434929] conn(0x7fbbac0a9eb0 0x7fbbac0ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:47.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.010+0000 7fbbc6aab700 1 --2- 172.21.15.82:0/1378271718 >> [v2:172.21.15.82:6802/3197434929,v1:172.21.15.82:6803/3197434929] conn(0x7fbbac0a9eb0 0x7fbbac0ac2d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:47.347 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.010+0000 7fbbb77fe700 1 -- 172.21.15.82:0/1378271718 <== osd.0 v2:172.21.15.82:6802/3197434929 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fbbac0ac980 con 0x7fbbac0a9eb0 2023-09-09T23:12:47.347 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.027+0000 7fbbc850e700 1 -- 172.21.15.82:0/1378271718 --> [v2:172.21.15.82:6802/3197434929,v1:172.21.15.82:6803/3197434929] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbbc0112a90 con 0x7fbbac0a9eb0 2023-09-09T23:12:47.347 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.027+0000 7fbbb77fe700 1 -- 172.21.15.82:0/1378271718 <== osd.0 v2:172.21.15.82:6802/3197434929 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbbb80091d0 con 0x7fbbac0a9eb0 2023-09-09T23:12:47.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.027+0000 7fbbc850e700 1 -- 172.21.15.82:0/1378271718 >> [v2:172.21.15.82:6802/3197434929,v1:172.21.15.82:6803/3197434929] conn(0x7fbbac0a9eb0 msgr2=0x7fbbac0ac2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:47.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.027+0000 7fbbc850e700 1 --2- 172.21.15.82:0/1378271718 >> [v2:172.21.15.82:6802/3197434929,v1:172.21.15.82:6803/3197434929] conn(0x7fbbac0a9eb0 0x7fbbac0ac2d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:47.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.027+0000 7fbbc850e700 1 -- 172.21.15.82:0/1378271718 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fbbac0a06d0 msgr2=0x7fbbac0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:47.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.027+0000 7fbbc850e700 1 --2- 172.21.15.82:0/1378271718 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fbbac0a06d0 0x7fbbac0a2b90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0100bb0 tx=0x7fbbb000bef0 comp rx=0 tx=0).stop 2023-09-09T23:12:47.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.027+0000 7fbbc850e700 1 -- 172.21.15.82:0/1378271718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbc0068a80 msgr2=0x7fbbc019e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:47.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.027+0000 7fbbc850e700 1 --2- 172.21.15.82:0/1378271718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbc0068a80 0x7fbbc019e780 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc00d950 tx=0x7fbbbc00dd10 comp rx=0 tx=0).stop 2023-09-09T23:12:47.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.027+0000 7fbbc6aab700 1 -- 172.21.15.82:0/1378271718 reap_dead start 2023-09-09T23:12:47.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.028+0000 7fbbc850e700 1 -- 172.21.15.82:0/1378271718 shutdown_connections 2023-09-09T23:12:47.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.028+0000 7fbbc850e700 1 -- 172.21.15.82:0/1378271718 >> 172.21.15.82:0/1378271718 conn(0x7fbbc0076680 msgr2=0x7fbbc0106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:47.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.028+0000 7fbbc850e700 1 -- 172.21.15.82:0/1378271718 shutdown_connections 2023-09-09T23:12:47.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.028+0000 7fbbc850e700 1 -- 172.21.15.82:0/1378271718 wait complete. 2023-09-09T23:12:47.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.601+0000 7f2b236b2700 1 -- 172.21.15.82:0/4027138081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b1c0ffef0 msgr2=0x7f2b1c10c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:47.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.601+0000 7f2b236b2700 1 --2- 172.21.15.82:0/4027138081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b1c0ffef0 0x7f2b1c10c3b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400d360 tx=0x7f2b1400d670 comp rx=0 tx=0).stop 2023-09-09T23:12:47.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.601+0000 7f2b236b2700 1 -- 172.21.15.82:0/4027138081 shutdown_connections 2023-09-09T23:12:47.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.601+0000 7f2b236b2700 1 --2- 172.21.15.82:0/4027138081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b1c0ffef0 0x7f2b1c10c3b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:47.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.601+0000 7f2b236b2700 1 --2- 172.21.15.82:0/4027138081 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b1c1055e0 0x7f2b1c1059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:47.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.601+0000 7f2b236b2700 1 --2- 172.21.15.82:0/4027138081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b1c0ff530 0x7f2b1c0ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:47.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.601+0000 7f2b236b2700 1 -- 172.21.15.82:0/4027138081 >> 172.21.15.82:0/4027138081 conn(0x7f2b1c0fafc0 msgr2=0x7f2b1c0fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:47.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.601+0000 7f2b236b2700 1 -- 172.21.15.82:0/4027138081 shutdown_connections 2023-09-09T23:12:47.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.601+0000 7f2b236b2700 1 -- 172.21.15.82:0/4027138081 wait complete. 2023-09-09T23:12:47.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.602+0000 7f2b236b2700 1 Processor -- start 2023-09-09T23:12:47.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.602+0000 7f2b236b2700 1 -- start start 2023-09-09T23:12:47.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.602+0000 7f2b236b2700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b1c0ff530 0x7f2b1c1944d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:47.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.602+0000 7f2b236b2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b1c0ffef0 0x7f2b1c194a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:47.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.602+0000 7f2b236b2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b1c1055e0 0x7f2b1c198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:47.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.603+0000 7f2b236b2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c199490 con 0x7f2b1c1055e0 2023-09-09T23:12:47.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.603+0000 7f2b236b2700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c199600 con 0x7f2b1c0ff530 2023-09-09T23:12:47.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.603+0000 7f2b236b2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c1997c0 con 0x7f2b1c0ffef0 2023-09-09T23:12:47.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.603+0000 7f2b21c4f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b1c1055e0 0x7f2b1c198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:47.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.603+0000 7f2b21c4f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b1c1055e0 0x7f2b1c198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54782/0 (socket says 172.21.15.82:54782) 2023-09-09T23:12:47.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.603+0000 7f2b21c4f700 1 -- 172.21.15.82:0/323473222 learned_addr learned my addr 172.21.15.82:0/323473222 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:47.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.603+0000 7f2b2144e700 1 --2- 172.21.15.82:0/323473222 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b1c0ff530 0x7f2b1c1944d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:47.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.603+0000 7f2b20c4d700 1 --2- 172.21.15.82:0/323473222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b1c0ffef0 0x7f2b1c194a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:47.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.603+0000 7f2b21c4f700 1 -- 172.21.15.82:0/323473222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b1c0ffef0 msgr2=0x7f2b1c194a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:47.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.603+0000 7f2b21c4f700 1 --2- 172.21.15.82:0/323473222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b1c0ffef0 0x7f2b1c194a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:47.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.603+0000 7f2b21c4f700 1 -- 172.21.15.82:0/323473222 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b1c0ff530 msgr2=0x7f2b1c1944d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:47.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.603+0000 7f2b21c4f700 1 --2- 172.21.15.82:0/323473222 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b1c0ff530 0x7f2b1c1944d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:47.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.603+0000 7f2b21c4f700 1 -- 172.21.15.82:0/323473222 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b14009d00 con 0x7f2b1c1055e0 2023-09-09T23:12:47.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.603+0000 7f2b2144e700 1 --2- 172.21.15.82:0/323473222 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b1c0ff530 0x7f2b1c1944d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:12:47.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.603+0000 7f2b21c4f700 1 --2- 172.21.15.82:0/323473222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b1c1055e0 0x7f2b1c198e80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2b14011c20 tx=0x7f2b14011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:47.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.603+0000 7f2b0e7fc700 1 -- 172.21.15.82:0/323473222 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b14003c10 con 0x7f2b1c1055e0 2023-09-09T23:12:47.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.604+0000 7f2b0e7fc700 1 -- 172.21.15.82:0/323473222 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b14003d70 con 0x7f2b1c1055e0 2023-09-09T23:12:47.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.604+0000 7f2b0e7fc700 1 -- 172.21.15.82:0/323473222 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b140052d0 con 0x7f2b1c1055e0 2023-09-09T23:12:47.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.604+0000 7f2b236b2700 1 -- 172.21.15.82:0/323473222 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b1c199a40 con 0x7f2b1c1055e0 2023-09-09T23:12:47.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.604+0000 7f2b236b2700 1 -- 172.21.15.82:0/323473222 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b1c1a1130 con 0x7f2b1c1055e0 2023-09-09T23:12:47.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.605+0000 7f2b236b2700 1 -- 172.21.15.82:0/323473222 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2b1c068290 con 0x7f2b1c1055e0 2023-09-09T23:12:47.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.606+0000 7f2b0e7fc700 1 -- 172.21.15.82:0/323473222 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f2b140074e0 con 0x7f2b1c1055e0 2023-09-09T23:12:47.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.607+0000 7f2b0e7fc700 1 --2- 172.21.15.82:0/323473222 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f2b080a0750 0x7f2b080a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:47.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.607+0000 7f2b2144e700 1 --2- 172.21.15.82:0/323473222 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f2b080a0750 0x7f2b080a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:47.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.607+0000 7f2b0e7fc700 1 -- 172.21.15.82:0/323473222 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2b140cfb90 con 0x7f2b1c1055e0 2023-09-09T23:12:47.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.607+0000 7f2b0e7fc700 1 --2- 172.21.15.82:0/323473222 >> [v2:172.21.15.170:6824/146964512,v1:172.21.15.170:6825/146964512] conn(0x7f2b080a9f30 0x7f2b080ac350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:47.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.607+0000 7f2b0e7fc700 1 -- 172.21.15.82:0/323473222 --> [v2:172.21.15.170:6824/146964512,v1:172.21.15.170:6825/146964512] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2b080aca00 con 0x7f2b080a9f30 2023-09-09T23:12:47.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.607+0000 7f2b0e7fc700 1 -- 172.21.15.82:0/323473222 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b140cff40 con 0x7f2b1c1055e0 2023-09-09T23:12:47.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.607+0000 7f2b2144e700 1 --2- 172.21.15.82:0/323473222 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f2b080a0750 0x7f2b080a2c10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2b18011490 tx=0x7f2b18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:47.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.607+0000 7f2b20c4d700 1 --2- 172.21.15.82:0/323473222 >> [v2:172.21.15.170:6824/146964512,v1:172.21.15.170:6825/146964512] conn(0x7f2b080a9f30 0x7f2b080ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:47.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.608+0000 7f2b20c4d700 1 --2- 172.21.15.82:0/323473222 >> [v2:172.21.15.170:6824/146964512,v1:172.21.15.170:6825/146964512] conn(0x7f2b080a9f30 0x7f2b080ac350 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:47.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.609+0000 7f2b0e7fc700 1 -- 172.21.15.82:0/323473222 <== osd.11 v2:172.21.15.170:6824/146964512 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f2b080aca00 con 0x7f2b080a9f30 2023-09-09T23:12:47.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.625+0000 7f2b236b2700 1 -- 172.21.15.82:0/323473222 --> [v2:172.21.15.170:6824/146964512,v1:172.21.15.170:6825/146964512] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2b1c10e6b0 con 0x7f2b080a9f30 2023-09-09T23:12:47.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.625+0000 7f2b0e7fc700 1 -- 172.21.15.82:0/323473222 <== osd.11 v2:172.21.15.170:6824/146964512 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2b10009990 con 0x7f2b080a9f30 2023-09-09T23:12:47.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.626+0000 7f2b236b2700 1 -- 172.21.15.82:0/323473222 >> [v2:172.21.15.170:6824/146964512,v1:172.21.15.170:6825/146964512] conn(0x7f2b080a9f30 msgr2=0x7f2b080ac350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:47.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.626+0000 7f2b236b2700 1 --2- 172.21.15.82:0/323473222 >> [v2:172.21.15.170:6824/146964512,v1:172.21.15.170:6825/146964512] conn(0x7f2b080a9f30 0x7f2b080ac350 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:47.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.626+0000 7f2b236b2700 1 -- 172.21.15.82:0/323473222 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f2b080a0750 msgr2=0x7f2b080a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:47.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.626+0000 7f2b236b2700 1 --2- 172.21.15.82:0/323473222 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f2b080a0750 0x7f2b080a2c10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2b18011490 tx=0x7f2b18009260 comp rx=0 tx=0).stop 2023-09-09T23:12:47.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.626+0000 7f2b236b2700 1 -- 172.21.15.82:0/323473222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b1c1055e0 msgr2=0x7f2b1c198e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:47.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.626+0000 7f2b236b2700 1 --2- 172.21.15.82:0/323473222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b1c1055e0 0x7f2b1c198e80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2b14011c20 tx=0x7f2b14011c50 comp rx=0 tx=0).stop 2023-09-09T23:12:47.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.626+0000 7f2b21c4f700 1 -- 172.21.15.82:0/323473222 reap_dead start 2023-09-09T23:12:47.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.626+0000 7f2b236b2700 1 -- 172.21.15.82:0/323473222 shutdown_connections 2023-09-09T23:12:47.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.626+0000 7f2b236b2700 1 -- 172.21.15.82:0/323473222 >> 172.21.15.82:0/323473222 conn(0x7f2b1c0fafc0 msgr2=0x7f2b1c0fc520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:47.627 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.626+0000 7f2b236b2700 1 -- 172.21.15.82:0/323473222 shutdown_connections 2023-09-09T23:12:47.627 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.626+0000 7f2b236b2700 1 -- 172.21.15.82:0/323473222 wait complete. 2023-09-09T23:12:47.998 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.000+0000 7f5b11f54700 1 -- 172.21.15.82:0/3464662638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b0c100af0 msgr2=0x7f5b0c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:47.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.000+0000 7f5b11f54700 1 --2- 172.21.15.82:0/3464662638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b0c100af0 0x7f5b0c100ed0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5afc009990 tx=0x7f5afc009ca0 comp rx=0 tx=0).stop 2023-09-09T23:12:47.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.000+0000 7f5b11f54700 1 -- 172.21.15.82:0/3464662638 shutdown_connections 2023-09-09T23:12:47.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.000+0000 7f5b11f54700 1 --2- 172.21.15.82:0/3464662638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b0c105a20 0x7f5b0c10d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.000+0000 7f5b11f54700 1 --2- 172.21.15.82:0/3464662638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b0c100af0 0x7f5b0c100ed0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.000+0000 7f5b11f54700 1 --2- 172.21.15.82:0/3464662638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b0c1014a0 0x7f5b0c1052f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.000+0000 7f5b11f54700 1 -- 172.21.15.82:0/3464662638 >> 172.21.15.82:0/3464662638 conn(0x7f5b0c0785d0 msgr2=0x7f5b0c0789e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:48.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.000+0000 7f5b11f54700 1 -- 172.21.15.82:0/3464662638 shutdown_connections 2023-09-09T23:12:48.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.000+0000 7f5b11f54700 1 -- 172.21.15.82:0/3464662638 wait complete. 2023-09-09T23:12:48.001 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.000+0000 7f5b11f54700 1 Processor -- start 2023-09-09T23:12:48.003 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.001+0000 7f5b11f54700 1 -- start start 2023-09-09T23:12:48.004 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.001+0000 7f5b11f54700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b0c100af0 0x7f5b0c196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.004 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.001+0000 7f5b11f54700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b0c1014a0 0x7f5b0c196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.004 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.001+0000 7f5b11f54700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b0c105a20 0x7f5b0c19afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.004 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.001+0000 7f5b11f54700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c19b5c0 con 0x7f5b0c105a20 2023-09-09T23:12:48.004 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.001+0000 7f5b11f54700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c19b730 con 0x7f5b0c100af0 2023-09-09T23:12:48.005 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.001+0000 7f5b11f54700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c19b8f0 con 0x7f5b0c1014a0 2023-09-09T23:12:48.005 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.001+0000 7f5b0bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b0c105a20 0x7f5b0c19afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.005 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.001+0000 7f5b0bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b0c105a20 0x7f5b0c19afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54798/0 (socket says 172.21.15.82:54798) 2023-09-09T23:12:48.005 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.001+0000 7f5b0bfff700 1 -- 172.21.15.82:0/1204851807 learned_addr learned my addr 172.21.15.82:0/1204851807 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:48.005 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.001+0000 7f5b0affd700 1 --2- 172.21.15.82:0/1204851807 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b0c1014a0 0x7f5b0c196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.006 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.001+0000 7f5b0b7fe700 1 --2- 172.21.15.82:0/1204851807 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b0c100af0 0x7f5b0c196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.006 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.001+0000 7f5b0bfff700 1 -- 172.21.15.82:0/1204851807 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b0c1014a0 msgr2=0x7f5b0c196b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.006 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.001+0000 7f5b0bfff700 1 --2- 172.21.15.82:0/1204851807 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b0c1014a0 0x7f5b0c196b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.006 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.001+0000 7f5b0bfff700 1 -- 172.21.15.82:0/1204851807 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b0c100af0 msgr2=0x7f5b0c196600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.006 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.001+0000 7f5b0bfff700 1 --2- 172.21.15.82:0/1204851807 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b0c100af0 0x7f5b0c196600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.007 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.001+0000 7f5b0bfff700 1 -- 172.21.15.82:0/1204851807 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5afc0095f0 con 0x7f5b0c105a20 2023-09-09T23:12:48.007 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.001+0000 7f5b0affd700 1 --2- 172.21.15.82:0/1204851807 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b0c1014a0 0x7f5b0c196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:12:48.007 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.001+0000 7f5b0b7fe700 1 --2- 172.21.15.82:0/1204851807 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b0c100af0 0x7f5b0c196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:12:48.007 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.001+0000 7f5b0bfff700 1 --2- 172.21.15.82:0/1204851807 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b0c105a20 0x7f5b0c19afb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5b04007f10 tx=0x7f5b0400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:48.007 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.001+0000 7f5b08ff9700 1 -- 172.21.15.82:0/1204851807 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04017070 con 0x7f5b0c105a20 2023-09-09T23:12:48.008 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.001+0000 7f5b08ff9700 1 -- 172.21.15.82:0/1204851807 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b0400f040 con 0x7f5b0c105a20 2023-09-09T23:12:48.008 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.001+0000 7f5b11f54700 1 -- 172.21.15.82:0/1204851807 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b0c19bbd0 con 0x7f5b0c105a20 2023-09-09T23:12:48.008 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.002+0000 7f5b08ff9700 1 -- 172.21.15.82:0/1204851807 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04005740 con 0x7f5b0c105a20 2023-09-09T23:12:48.008 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.002+0000 7f5b11f54700 1 -- 172.21.15.82:0/1204851807 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0c1a3430 con 0x7f5b0c105a20 2023-09-09T23:12:48.008 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.002+0000 7f5b11f54700 1 -- 172.21.15.82:0/1204851807 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5b0c110890 con 0x7f5b0c105a20 2023-09-09T23:12:48.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.003+0000 7f5b08ff9700 1 -- 172.21.15.82:0/1204851807 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f5b0400dc60 con 0x7f5b0c105a20 2023-09-09T23:12:48.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.004+0000 7f5b08ff9700 1 --2- 172.21.15.82:0/1204851807 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5af40a0750 0x7f5af40a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.004+0000 7f5b08ff9700 1 -- 172.21.15.82:0/1204851807 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5b040cb2d0 con 0x7f5b0c105a20 2023-09-09T23:12:48.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.004+0000 7f5b0b7fe700 1 --2- 172.21.15.82:0/1204851807 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5af40a0750 0x7f5af40a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.004+0000 7f5b08ff9700 1 --2- 172.21.15.82:0/1204851807 >> [v2:172.21.15.98:6816/3547615284,v1:172.21.15.98:6817/3547615284] conn(0x7f5af40a9f30 0x7f5af40ac350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.010 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.004+0000 7f5b08ff9700 1 -- 172.21.15.82:0/1204851807 --> [v2:172.21.15.98:6816/3547615284,v1:172.21.15.98:6817/3547615284] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5af40aca00 con 0x7f5af40a9f30 2023-09-09T23:12:48.010 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.004+0000 7f5b08ff9700 1 -- 172.21.15.82:0/1204851807 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b040cd9e0 con 0x7f5b0c105a20 2023-09-09T23:12:48.010 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.004+0000 7f5b0b7fe700 1 --2- 172.21.15.82:0/1204851807 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5af40a0750 0x7f5af40a2c10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5afc000c00 tx=0x7f5afc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:48.010 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.004+0000 7f5b0affd700 1 --2- 172.21.15.82:0/1204851807 >> [v2:172.21.15.98:6816/3547615284,v1:172.21.15.98:6817/3547615284] conn(0x7f5af40a9f30 0x7f5af40ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.010 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.005+0000 7f5b0affd700 1 --2- 172.21.15.82:0/1204851807 >> [v2:172.21.15.98:6816/3547615284,v1:172.21.15.98:6817/3547615284] conn(0x7f5af40a9f30 0x7f5af40ac350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:48.011 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.006+0000 7f5b08ff9700 1 -- 172.21.15.82:0/1204851807 <== osd.7 v2:172.21.15.98:6816/3547615284 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f5af40aca00 con 0x7f5af40a9f30 2023-09-09T23:12:48.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.021+0000 7f5b11f54700 1 -- 172.21.15.82:0/1204851807 --> [v2:172.21.15.98:6816/3547615284,v1:172.21.15.98:6817/3547615284] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5b0c04e300 con 0x7f5af40a9f30 2023-09-09T23:12:48.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.022+0000 7f5b08ff9700 1 -- 172.21.15.82:0/1204851807 <== osd.7 v2:172.21.15.98:6816/3547615284 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5b0c04e300 con 0x7f5af40a9f30 2023-09-09T23:12:48.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.022+0000 7f5b11f54700 1 -- 172.21.15.82:0/1204851807 >> [v2:172.21.15.98:6816/3547615284,v1:172.21.15.98:6817/3547615284] conn(0x7f5af40a9f30 msgr2=0x7f5af40ac350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.022+0000 7f5b11f54700 1 --2- 172.21.15.82:0/1204851807 >> [v2:172.21.15.98:6816/3547615284,v1:172.21.15.98:6817/3547615284] conn(0x7f5af40a9f30 0x7f5af40ac350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.022+0000 7f5b11f54700 1 -- 172.21.15.82:0/1204851807 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5af40a0750 msgr2=0x7f5af40a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.022+0000 7f5b11f54700 1 --2- 172.21.15.82:0/1204851807 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5af40a0750 0x7f5af40a2c10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5afc000c00 tx=0x7f5afc005fb0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.022+0000 7f5b11f54700 1 -- 172.21.15.82:0/1204851807 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b0c105a20 msgr2=0x7f5b0c19afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.022+0000 7f5b11f54700 1 --2- 172.21.15.82:0/1204851807 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b0c105a20 0x7f5b0c19afb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5b04007f10 tx=0x7f5b0400d3b0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.022+0000 7f5b0bfff700 1 -- 172.21.15.82:0/1204851807 reap_dead start 2023-09-09T23:12:48.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.022+0000 7f5b11f54700 1 -- 172.21.15.82:0/1204851807 shutdown_connections 2023-09-09T23:12:48.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.022+0000 7f5b11f54700 1 -- 172.21.15.82:0/1204851807 >> 172.21.15.82:0/1204851807 conn(0x7f5b0c0785d0 msgr2=0x7f5b0c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:48.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.022+0000 7f5b11f54700 1 -- 172.21.15.82:0/1204851807 shutdown_connections 2023-09-09T23:12:48.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.022+0000 7f5b11f54700 1 -- 172.21.15.82:0/1204851807 wait complete. 2023-09-09T23:12:48.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.572+0000 7f5f72a7a700 1 -- 172.21.15.82:0/2495506615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f6c104530 msgr2=0x7f5f6c104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.572+0000 7f5f72a7a700 1 --2- 172.21.15.82:0/2495506615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f6c104530 0x7f5f6c104910 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5f60009990 tx=0x7f5f60009ca0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.572+0000 7f5f72a7a700 1 -- 172.21.15.82:0/2495506615 shutdown_connections 2023-09-09T23:12:48.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.572+0000 7f5f72a7a700 1 --2- 172.21.15.82:0/2495506615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f6c0ffee0 0x7f5f6c10c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.572+0000 7f5f72a7a700 1 --2- 172.21.15.82:0/2495506615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f6c104530 0x7f5f6c104910 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.572+0000 7f5f72a7a700 1 --2- 172.21.15.82:0/2495506615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f6c0ff520 0x7f5f6c0ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.572+0000 7f5f72a7a700 1 -- 172.21.15.82:0/2495506615 >> 172.21.15.82:0/2495506615 conn(0x7f5f6c076600 msgr2=0x7f5f6c076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:48.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.572+0000 7f5f72a7a700 1 -- 172.21.15.82:0/2495506615 shutdown_connections 2023-09-09T23:12:48.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.572+0000 7f5f72a7a700 1 -- 172.21.15.82:0/2495506615 wait complete. 2023-09-09T23:12:48.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.573+0000 7f5f72a7a700 1 Processor -- start 2023-09-09T23:12:48.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.573+0000 7f5f72a7a700 1 -- start start 2023-09-09T23:12:48.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.573+0000 7f5f72a7a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f6c0ff520 0x7f5f6c196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.573+0000 7f5f72a7a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f6c0ffee0 0x7f5f6c196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.573+0000 7f5f72a7a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f6c104530 0x7f5f6c19afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.573+0000 7f5f72a7a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c19b5f0 con 0x7f5f6c0ff520 2023-09-09T23:12:48.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.573+0000 7f5f72a7a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c19b760 con 0x7f5f6c104530 2023-09-09T23:12:48.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.573+0000 7f5f72a7a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c19b920 con 0x7f5f6c0ffee0 2023-09-09T23:12:48.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.573+0000 7f5f70816700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f6c0ff520 0x7f5f6c196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.573+0000 7f5f70816700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f6c0ff520 0x7f5f6c196630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54766/0 (socket says 172.21.15.82:54766) 2023-09-09T23:12:48.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.573+0000 7f5f70816700 1 -- 172.21.15.82:0/1910554780 learned_addr learned my addr 172.21.15.82:0/1910554780 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:48.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.573+0000 7f5f6bfff700 1 --2- 172.21.15.82:0/1910554780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f6c0ffee0 0x7f5f6c196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.573+0000 7f5f70816700 1 -- 172.21.15.82:0/1910554780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f6c0ffee0 msgr2=0x7f5f6c196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.573+0000 7f5f70816700 1 --2- 172.21.15.82:0/1910554780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f6c0ffee0 0x7f5f6c196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.573+0000 7f5f70816700 1 -- 172.21.15.82:0/1910554780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f6c104530 msgr2=0x7f5f6c19afe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-09T23:12:48.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.573+0000 7f5f70816700 1 --2- 172.21.15.82:0/1910554780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f6c104530 0x7f5f6c19afe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.573+0000 7f5f70816700 1 -- 172.21.15.82:0/1910554780 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f600095f0 con 0x7f5f6c0ff520 2023-09-09T23:12:48.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.574+0000 7f5f70816700 1 --2- 172.21.15.82:0/1910554780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f6c0ff520 0x7f5f6c196630 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5f600098e0 tx=0x7f5f6000f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:48.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.574+0000 7f5f69ffb700 1 -- 172.21.15.82:0/1910554780 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6001d070 con 0x7f5f6c0ff520 2023-09-09T23:12:48.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.574+0000 7f5f72a7a700 1 -- 172.21.15.82:0/1910554780 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f6c19bba0 con 0x7f5f6c0ff520 2023-09-09T23:12:48.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.574+0000 7f5f69ffb700 1 -- 172.21.15.82:0/1910554780 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f60005510 con 0x7f5f6c0ff520 2023-09-09T23:12:48.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.574+0000 7f5f69ffb700 1 -- 172.21.15.82:0/1910554780 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60004c40 con 0x7f5f6c0ff520 2023-09-09T23:12:48.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.574+0000 7f5f72a7a700 1 -- 172.21.15.82:0/1910554780 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f6c1a3460 con 0x7f5f6c0ff520 2023-09-09T23:12:48.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.576+0000 7f5f72a7a700 1 -- 172.21.15.82:0/1910554780 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5f6c04e300 con 0x7f5f6c0ff520 2023-09-09T23:12:48.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.576+0000 7f5f69ffb700 1 -- 172.21.15.82:0/1910554780 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f5f6000fc10 con 0x7f5f6c0ff520 2023-09-09T23:12:48.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.577+0000 7f5f69ffb700 1 --2- 172.21.15.82:0/1910554780 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5f5c0a0750 0x7f5f5c0a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.577+0000 7f5f6bfff700 1 --2- 172.21.15.82:0/1910554780 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5f5c0a0750 0x7f5f5c0a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.577+0000 7f5f69ffb700 1 -- 172.21.15.82:0/1910554780 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5f600cca30 con 0x7f5f6c0ff520 2023-09-09T23:12:48.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.577+0000 7f5f69ffb700 1 --2- 172.21.15.82:0/1910554780 >> [v2:172.21.15.98:6808/2507035643,v1:172.21.15.98:6809/2507035643] conn(0x7f5f5c0a9f30 0x7f5f5c0ac350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.577+0000 7f5f69ffb700 1 -- 172.21.15.82:0/1910554780 --> [v2:172.21.15.98:6808/2507035643,v1:172.21.15.98:6809/2507035643] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5f5c0aca00 con 0x7f5f5c0a9f30 2023-09-09T23:12:48.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.577+0000 7f5f69ffb700 1 -- 172.21.15.82:0/1910554780 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f600ceea0 con 0x7f5f6c0ff520 2023-09-09T23:12:48.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.577+0000 7f5f71017700 1 --2- 172.21.15.82:0/1910554780 >> [v2:172.21.15.98:6808/2507035643,v1:172.21.15.98:6809/2507035643] conn(0x7f5f5c0a9f30 0x7f5f5c0ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.578+0000 7f5f71017700 1 --2- 172.21.15.82:0/1910554780 >> [v2:172.21.15.98:6808/2507035643,v1:172.21.15.98:6809/2507035643] conn(0x7f5f5c0a9f30 0x7f5f5c0ac350 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:48.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.579+0000 7f5f69ffb700 1 -- 172.21.15.82:0/1910554780 <== osd.4 v2:172.21.15.98:6808/2507035643 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f5f5c0aca00 con 0x7f5f5c0a9f30 2023-09-09T23:12:48.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.579+0000 7f5f6bfff700 1 --2- 172.21.15.82:0/1910554780 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5f5c0a0750 0x7f5f5c0a2c10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5f58005950 tx=0x7f5f580058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:48.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.597+0000 7f5f72a7a700 1 -- 172.21.15.82:0/1910554780 --> [v2:172.21.15.98:6808/2507035643,v1:172.21.15.98:6809/2507035643] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5f6c10f750 con 0x7f5f5c0a9f30 2023-09-09T23:12:48.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.598+0000 7f5f69ffb700 1 -- 172.21.15.82:0/1910554780 <== osd.4 v2:172.21.15.98:6808/2507035643 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5f64008b10 con 0x7f5f5c0a9f30 2023-09-09T23:12:48.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.598+0000 7f5f72a7a700 1 -- 172.21.15.82:0/1910554780 >> [v2:172.21.15.98:6808/2507035643,v1:172.21.15.98:6809/2507035643] conn(0x7f5f5c0a9f30 msgr2=0x7f5f5c0ac350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.598+0000 7f5f72a7a700 1 --2- 172.21.15.82:0/1910554780 >> [v2:172.21.15.98:6808/2507035643,v1:172.21.15.98:6809/2507035643] conn(0x7f5f5c0a9f30 0x7f5f5c0ac350 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.599+0000 7f5f72a7a700 1 -- 172.21.15.82:0/1910554780 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5f5c0a0750 msgr2=0x7f5f5c0a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.599+0000 7f5f72a7a700 1 --2- 172.21.15.82:0/1910554780 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5f5c0a0750 0x7f5f5c0a2c10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5f58005950 tx=0x7f5f580058e0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.599+0000 7f5f72a7a700 1 -- 172.21.15.82:0/1910554780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f6c0ff520 msgr2=0x7f5f6c196630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.599+0000 7f5f72a7a700 1 --2- 172.21.15.82:0/1910554780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f6c0ff520 0x7f5f6c196630 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5f600098e0 tx=0x7f5f6000f720 comp rx=0 tx=0).stop 2023-09-09T23:12:48.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.599+0000 7f5f71017700 1 -- 172.21.15.82:0/1910554780 reap_dead start 2023-09-09T23:12:48.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.599+0000 7f5f72a7a700 1 -- 172.21.15.82:0/1910554780 shutdown_connections 2023-09-09T23:12:48.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.599+0000 7f5f72a7a700 1 -- 172.21.15.82:0/1910554780 >> 172.21.15.82:0/1910554780 conn(0x7f5f6c076600 msgr2=0x7f5f6c0680c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:48.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.599+0000 7f5f72a7a700 1 -- 172.21.15.82:0/1910554780 shutdown_connections 2023-09-09T23:12:48.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:47.600+0000 7f5f72a7a700 1 -- 172.21.15.82:0/1910554780 wait complete. 2023-09-09T23:12:48.497 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.493+0000 7fd152016700 1 -- 172.21.15.82:0/4189514772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd14c068120 msgr2=0x7fd14c0685a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.497 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.493+0000 7fd152016700 1 --2- 172.21.15.82:0/4189514772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd14c068120 0x7fd14c0685a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009990 tx=0x7fd13c009ca0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.498 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.493+0000 7fd152016700 1 -- 172.21.15.82:0/4189514772 shutdown_connections 2023-09-09T23:12:48.498 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.493+0000 7fd152016700 1 --2- 172.21.15.82:0/4189514772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd14c068ae0 0x7fd14c10d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.493+0000 7fd152016700 1 --2- 172.21.15.82:0/4189514772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd14c106770 0x7fd14c106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.493+0000 7fd152016700 1 --2- 172.21.15.82:0/4189514772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd14c068120 0x7fd14c0685a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.493+0000 7fd152016700 1 -- 172.21.15.82:0/4189514772 >> 172.21.15.82:0/4189514772 conn(0x7fd14c076600 msgr2=0x7fd14c076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:48.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.493+0000 7fd152016700 1 -- 172.21.15.82:0/4189514772 shutdown_connections 2023-09-09T23:12:48.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.493+0000 7fd152016700 1 -- 172.21.15.82:0/4189514772 wait complete. 2023-09-09T23:12:48.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.494+0000 7fd152016700 1 Processor -- start 2023-09-09T23:12:48.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.494+0000 7fd152016700 1 -- start start 2023-09-09T23:12:48.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.494+0000 7fd152016700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd14c068120 0x7fd14c0ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.494+0000 7fd152016700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd14c068ae0 0x7fd14c0ffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.494+0000 7fd152016700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd14c106770 0x7fd14c100070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.494+0000 7fd152016700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c1a4bb0 con 0x7fd14c068120 2023-09-09T23:12:48.502 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.494+0000 7fd152016700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c1a4d20 con 0x7fd14c068ae0 2023-09-09T23:12:48.502 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.494+0000 7fd152016700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c1a4ee0 con 0x7fd14c106770 2023-09-09T23:12:48.502 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.494+0000 7fd14b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd14c068120 0x7fd14c0ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.502 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.494+0000 7fd14b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd14c068120 0x7fd14c0ff5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54808/0 (socket says 172.21.15.82:54808) 2023-09-09T23:12:48.503 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.494+0000 7fd14b7fe700 1 -- 172.21.15.82:0/1774364009 learned_addr learned my addr 172.21.15.82:0/1774364009 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:48.503 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.494+0000 7fd14bfff700 1 --2- 172.21.15.82:0/1774364009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd14c106770 0x7fd14c100070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.503 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.494+0000 7fd14affd700 1 --2- 172.21.15.82:0/1774364009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd14c068ae0 0x7fd14c0ffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.503 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.494+0000 7fd14b7fe700 1 -- 172.21.15.82:0/1774364009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd14c106770 msgr2=0x7fd14c100070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.504 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.494+0000 7fd14b7fe700 1 --2- 172.21.15.82:0/1774364009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd14c106770 0x7fd14c100070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.504 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.494+0000 7fd14b7fe700 1 -- 172.21.15.82:0/1774364009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd14c068ae0 msgr2=0x7fd14c0ffaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.504 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.494+0000 7fd14b7fe700 1 --2- 172.21.15.82:0/1774364009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd14c068ae0 0x7fd14c0ffaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.504 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.494+0000 7fd14b7fe700 1 -- 172.21.15.82:0/1774364009 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd13c0095f0 con 0x7fd14c068120 2023-09-09T23:12:48.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.495+0000 7fd14b7fe700 1 --2- 172.21.15.82:0/1774364009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd14c068120 0x7fd14c0ff5b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd14000c920 tx=0x7fd14000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:48.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.495+0000 7fd148ff9700 1 -- 172.21.15.82:0/1774364009 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd140007b00 con 0x7fd14c068120 2023-09-09T23:12:48.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.495+0000 7fd152016700 1 -- 172.21.15.82:0/1774364009 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14c1a51c0 con 0x7fd14c068120 2023-09-09T23:12:48.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.495+0000 7fd148ff9700 1 -- 172.21.15.82:0/1774364009 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd14000ce70 con 0x7fd14c068120 2023-09-09T23:12:48.506 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.495+0000 7fd148ff9700 1 -- 172.21.15.82:0/1774364009 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd140005460 con 0x7fd14c068120 2023-09-09T23:12:48.506 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.495+0000 7fd152016700 1 -- 172.21.15.82:0/1774364009 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14c1a56c0 con 0x7fd14c068120 2023-09-09T23:12:48.506 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.496+0000 7fd152016700 1 -- 172.21.15.82:0/1774364009 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd14c1108e0 con 0x7fd14c068120 2023-09-09T23:12:48.506 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.497+0000 7fd148ff9700 1 -- 172.21.15.82:0/1774364009 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fd140004750 con 0x7fd14c068120 2023-09-09T23:12:48.506 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.497+0000 7fd148ff9700 1 --2- 172.21.15.82:0/1774364009 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd1340a06d0 0x7fd1340a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.497+0000 7fd14affd700 1 --2- 172.21.15.82:0/1774364009 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd1340a06d0 0x7fd1340a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.497+0000 7fd148ff9700 1 -- 172.21.15.82:0/1774364009 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd1400cbb30 con 0x7fd14c068120 2023-09-09T23:12:48.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.498+0000 7fd148ff9700 1 --2- 172.21.15.82:0/1774364009 >> [v2:172.21.15.82:6826/3354242311,v1:172.21.15.82:6827/3354242311] conn(0x7fd1340a9eb0 0x7fd1340ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.498+0000 7fd148ff9700 1 -- 172.21.15.82:0/1774364009 --> [v2:172.21.15.82:6826/3354242311,v1:172.21.15.82:6827/3354242311] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd1340ac980 con 0x7fd1340a9eb0 2023-09-09T23:12:48.508 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.498+0000 7fd148ff9700 1 -- 172.21.15.82:0/1774364009 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1400cbfd0 con 0x7fd14c068120 2023-09-09T23:12:48.508 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.498+0000 7fd14bfff700 1 --2- 172.21.15.82:0/1774364009 >> [v2:172.21.15.82:6826/3354242311,v1:172.21.15.82:6827/3354242311] conn(0x7fd1340a9eb0 0x7fd1340ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.508 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.498+0000 7fd14affd700 1 --2- 172.21.15.82:0/1774364009 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd1340a06d0 0x7fd1340a2b90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd13c006010 tx=0x7fd13c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:48.508 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.498+0000 7fd14bfff700 1 --2- 172.21.15.82:0/1774364009 >> [v2:172.21.15.82:6826/3354242311,v1:172.21.15.82:6827/3354242311] conn(0x7fd1340a9eb0 0x7fd1340ac2d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:48.508 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.499+0000 7fd148ff9700 1 -- 172.21.15.82:0/1774364009 <== osd.9 v2:172.21.15.82:6826/3354242311 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fd1340ac980 con 0x7fd1340a9eb0 2023-09-09T23:12:48.513 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.515+0000 7fd152016700 1 -- 172.21.15.82:0/1774364009 --> [v2:172.21.15.82:6826/3354242311,v1:172.21.15.82:6827/3354242311] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd14c04e300 con 0x7fd1340a9eb0 2023-09-09T23:12:48.514 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.515+0000 7fd148ff9700 1 -- 172.21.15.82:0/1774364009 <== osd.9 v2:172.21.15.82:6826/3354242311 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd14c04e300 con 0x7fd1340a9eb0 2023-09-09T23:12:48.515 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.516+0000 7fd152016700 1 -- 172.21.15.82:0/1774364009 >> [v2:172.21.15.82:6826/3354242311,v1:172.21.15.82:6827/3354242311] conn(0x7fd1340a9eb0 msgr2=0x7fd1340ac2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.515 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.516+0000 7fd152016700 1 --2- 172.21.15.82:0/1774364009 >> [v2:172.21.15.82:6826/3354242311,v1:172.21.15.82:6827/3354242311] conn(0x7fd1340a9eb0 0x7fd1340ac2d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.515 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.516+0000 7fd152016700 1 -- 172.21.15.82:0/1774364009 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd1340a06d0 msgr2=0x7fd1340a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.516 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.516+0000 7fd152016700 1 --2- 172.21.15.82:0/1774364009 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd1340a06d0 0x7fd1340a2b90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd13c006010 tx=0x7fd13c00b540 comp rx=0 tx=0).stop 2023-09-09T23:12:48.516 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.516+0000 7fd152016700 1 -- 172.21.15.82:0/1774364009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd14c068120 msgr2=0x7fd14c0ff5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.516 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.516+0000 7fd152016700 1 --2- 172.21.15.82:0/1774364009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd14c068120 0x7fd14c0ff5b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd14000c920 tx=0x7fd14000cc30 comp rx=0 tx=0).stop 2023-09-09T23:12:48.516 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.516+0000 7fd14bfff700 1 -- 172.21.15.82:0/1774364009 reap_dead start 2023-09-09T23:12:48.517 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.516+0000 7fd152016700 1 -- 172.21.15.82:0/1774364009 shutdown_connections 2023-09-09T23:12:48.517 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.516+0000 7fd152016700 1 -- 172.21.15.82:0/1774364009 >> 172.21.15.82:0/1774364009 conn(0x7fd14c076600 msgr2=0x7fd14c0fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:48.517 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.516+0000 7fd152016700 1 -- 172.21.15.82:0/1774364009 shutdown_connections 2023-09-09T23:12:48.517 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.516+0000 7fd152016700 1 -- 172.21.15.82:0/1774364009 wait complete. 2023-09-09T23:12:48.524 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:48 smithi082 ceph-mon[98694]: pgmap v270: 97 pgs: 97 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 426 B/s wr, 11 op/s 2023-09-09T23:12:48.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:48 smithi170 ceph-mon[102832]: pgmap v270: 97 pgs: 97 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 426 B/s wr, 11 op/s 2023-09-09T23:12:48.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:48 smithi098 ceph-mon[103124]: pgmap v270: 97 pgs: 97 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 426 B/s wr, 11 op/s 2023-09-09T23:12:48.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.783+0000 7fa6a015e700 1 -- 172.21.15.82:0/2535658839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6981006a0 msgr2=0x7fa69810c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.783+0000 7fa6a015e700 1 --2- 172.21.15.82:0/2535658839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6981006a0 0x7fa69810c3b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa69000b0d0 tx=0x7fa69000b3e0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.783+0000 7fa6a015e700 1 -- 172.21.15.82:0/2535658839 shutdown_connections 2023-09-09T23:12:48.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.783+0000 7fa6a015e700 1 --2- 172.21.15.82:0/2535658839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6981006a0 0x7fa69810c3b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.783+0000 7fa6a015e700 1 --2- 172.21.15.82:0/2535658839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa698105550 0x7fa698105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.783+0000 7fa6a015e700 1 --2- 172.21.15.82:0/2535658839 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6980ffce0 0x7fa698100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.783+0000 7fa6a015e700 1 -- 172.21.15.82:0/2535658839 >> 172.21.15.82:0/2535658839 conn(0x7fa698075a70 msgr2=0x7fa6980fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:48.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.783+0000 7fa6a015e700 1 -- 172.21.15.82:0/2535658839 shutdown_connections 2023-09-09T23:12:48.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.783+0000 7fa6a015e700 1 -- 172.21.15.82:0/2535658839 wait complete. 2023-09-09T23:12:48.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.784+0000 7fa6a015e700 1 Processor -- start 2023-09-09T23:12:48.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.784+0000 7fa6a015e700 1 -- start start 2023-09-09T23:12:48.787 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.784+0000 7fa6a015e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6980ffce0 0x7fa6981955a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.787 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.784+0000 7fa6a015e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6981006a0 0x7fa698195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.787 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.784+0000 7fa6a015e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa698105550 0x7fa698199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.784+0000 7fa6a015e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69819a560 con 0x7fa6980ffce0 2023-09-09T23:12:48.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.784+0000 7fa6a015e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69819a6d0 con 0x7fa6981006a0 2023-09-09T23:12:48.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.784+0000 7fa6a015e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69819a890 con 0x7fa698105550 2023-09-09T23:12:48.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.784+0000 7fa69defa700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6980ffce0 0x7fa6981955a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.785+0000 7fa69defa700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6980ffce0 0x7fa6981955a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54840/0 (socket says 172.21.15.82:54840) 2023-09-09T23:12:48.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.785+0000 7fa69defa700 1 -- 172.21.15.82:0/3094103846 learned_addr learned my addr 172.21.15.82:0/3094103846 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:48.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.785+0000 7fa69d6f9700 1 --2- 172.21.15.82:0/3094103846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6981006a0 0x7fa698195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.785+0000 7fa69e6fb700 1 --2- 172.21.15.82:0/3094103846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa698105550 0x7fa698199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.785+0000 7fa69defa700 1 -- 172.21.15.82:0/3094103846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa698105550 msgr2=0x7fa698199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.785+0000 7fa69defa700 1 --2- 172.21.15.82:0/3094103846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa698105550 0x7fa698199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.785+0000 7fa69defa700 1 -- 172.21.15.82:0/3094103846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6981006a0 msgr2=0x7fa698195ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.785+0000 7fa69defa700 1 --2- 172.21.15.82:0/3094103846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6981006a0 0x7fa698195ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.785+0000 7fa69defa700 1 -- 172.21.15.82:0/3094103846 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa690009d00 con 0x7fa6980ffce0 2023-09-09T23:12:48.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.785+0000 7fa69defa700 1 --2- 172.21.15.82:0/3094103846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6980ffce0 0x7fa6981955a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa69400c9b0 tx=0x7fa69400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:48.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.785+0000 7fa68effd700 1 -- 172.21.15.82:0/3094103846 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa694007b00 con 0x7fa6980ffce0 2023-09-09T23:12:48.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.785+0000 7fa68effd700 1 -- 172.21.15.82:0/3094103846 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa69400ce50 con 0x7fa6980ffce0 2023-09-09T23:12:48.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.785+0000 7fa68effd700 1 -- 172.21.15.82:0/3094103846 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa694018660 con 0x7fa6980ffce0 2023-09-09T23:12:48.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.785+0000 7fa6a015e700 1 -- 172.21.15.82:0/3094103846 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa69819ab70 con 0x7fa6980ffce0 2023-09-09T23:12:48.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.785+0000 7fa6a015e700 1 -- 172.21.15.82:0/3094103846 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6981a23d0 con 0x7fa6980ffce0 2023-09-09T23:12:48.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.786+0000 7fa6a015e700 1 -- 172.21.15.82:0/3094103846 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa69804e300 con 0x7fa6980ffce0 2023-09-09T23:12:48.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.788+0000 7fa68effd700 1 -- 172.21.15.82:0/3094103846 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fa69400f450 con 0x7fa6980ffce0 2023-09-09T23:12:48.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.788+0000 7fa68effd700 1 --2- 172.21.15.82:0/3094103846 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa6840a06d0 0x7fa6840a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.788+0000 7fa69d6f9700 1 --2- 172.21.15.82:0/3094103846 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa6840a06d0 0x7fa6840a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.788+0000 7fa68effd700 1 -- 172.21.15.82:0/3094103846 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa6940cbf10 con 0x7fa6980ffce0 2023-09-09T23:12:48.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.788+0000 7fa68effd700 1 --2- 172.21.15.82:0/3094103846 >> [v2:172.21.15.170:6816/632038848,v1:172.21.15.170:6817/632038848] conn(0x7fa6840a9eb0 0x7fa6840ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.788+0000 7fa68effd700 1 -- 172.21.15.82:0/3094103846 --> [v2:172.21.15.170:6816/632038848,v1:172.21.15.170:6817/632038848] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa6840ac980 con 0x7fa6840a9eb0 2023-09-09T23:12:48.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.788+0000 7fa68effd700 1 -- 172.21.15.82:0/3094103846 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6940cc3b0 con 0x7fa6980ffce0 2023-09-09T23:12:48.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.789+0000 7fa69d6f9700 1 --2- 172.21.15.82:0/3094103846 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa6840a06d0 0x7fa6840a2b90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa698196bc0 tx=0x7fa688009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:48.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.789+0000 7fa69e6fb700 1 --2- 172.21.15.82:0/3094103846 >> [v2:172.21.15.170:6816/632038848,v1:172.21.15.170:6817/632038848] conn(0x7fa6840a9eb0 0x7fa6840ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.794 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.789+0000 7fa69e6fb700 1 --2- 172.21.15.82:0/3094103846 >> [v2:172.21.15.170:6816/632038848,v1:172.21.15.170:6817/632038848] conn(0x7fa6840a9eb0 0x7fa6840ac2d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:48.794 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.790+0000 7fa68effd700 1 -- 172.21.15.82:0/3094103846 <== osd.8 v2:172.21.15.170:6816/632038848 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fa6840ac980 con 0x7fa6840a9eb0 2023-09-09T23:12:48.816 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.807+0000 7fa6a015e700 1 -- 172.21.15.82:0/3094103846 --> [v2:172.21.15.170:6816/632038848,v1:172.21.15.170:6817/632038848] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa69810f6d0 con 0x7fa6840a9eb0 2023-09-09T23:12:48.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.807+0000 7fa68effd700 1 -- 172.21.15.82:0/3094103846 <== osd.8 v2:172.21.15.170:6816/632038848 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa69810f6d0 con 0x7fa6840a9eb0 2023-09-09T23:12:48.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.808+0000 7fa6a015e700 1 -- 172.21.15.82:0/3094103846 >> [v2:172.21.15.170:6816/632038848,v1:172.21.15.170:6817/632038848] conn(0x7fa6840a9eb0 msgr2=0x7fa6840ac2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.808+0000 7fa6a015e700 1 --2- 172.21.15.82:0/3094103846 >> [v2:172.21.15.170:6816/632038848,v1:172.21.15.170:6817/632038848] conn(0x7fa6840a9eb0 0x7fa6840ac2d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.808+0000 7fa6a015e700 1 -- 172.21.15.82:0/3094103846 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa6840a06d0 msgr2=0x7fa6840a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.808+0000 7fa6a015e700 1 --2- 172.21.15.82:0/3094103846 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa6840a06d0 0x7fa6840a2b90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa698196bc0 tx=0x7fa688009260 comp rx=0 tx=0).stop 2023-09-09T23:12:48.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.808+0000 7fa6a015e700 1 -- 172.21.15.82:0/3094103846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6980ffce0 msgr2=0x7fa6981955a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.808+0000 7fa6a015e700 1 --2- 172.21.15.82:0/3094103846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6980ffce0 0x7fa6981955a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa69400c9b0 tx=0x7fa69400ccc0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.808+0000 7fa69e6fb700 1 -- 172.21.15.82:0/3094103846 reap_dead start 2023-09-09T23:12:48.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.808+0000 7fa6a015e700 1 -- 172.21.15.82:0/3094103846 shutdown_connections 2023-09-09T23:12:48.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.808+0000 7fa6a015e700 1 -- 172.21.15.82:0/3094103846 >> 172.21.15.82:0/3094103846 conn(0x7fa698075a70 msgr2=0x7fa698106360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:48.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.808+0000 7fa6a015e700 1 -- 172.21.15.82:0/3094103846 shutdown_connections 2023-09-09T23:12:48.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.808+0000 7fa6a015e700 1 -- 172.21.15.82:0/3094103846 wait complete. 2023-09-09T23:12:48.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.893+0000 7effcc688700 1 -- 172.21.15.82:0/585045982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effc4100870 msgr2=0x7effc41041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.893+0000 7effcc688700 1 --2- 172.21.15.82:0/585045982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effc4100870 0x7effc41041b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7effb4009990 tx=0x7effb4009ca0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.893+0000 7effcc688700 1 -- 172.21.15.82:0/585045982 shutdown_connections 2023-09-09T23:12:48.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.893+0000 7effcc688700 1 --2- 172.21.15.82:0/585045982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7effc41048e0 0x7effc410c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.893+0000 7effcc688700 1 --2- 172.21.15.82:0/585045982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effc40ffec0 0x7effc41002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.893+0000 7effcc688700 1 --2- 172.21.15.82:0/585045982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effc4100870 0x7effc41041b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.893+0000 7effcc688700 1 -- 172.21.15.82:0/585045982 >> 172.21.15.82:0/585045982 conn(0x7effc4075a70 msgr2=0x7effc40fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:48.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.893+0000 7effcc688700 1 -- 172.21.15.82:0/585045982 shutdown_connections 2023-09-09T23:12:48.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.893+0000 7effcc688700 1 -- 172.21.15.82:0/585045982 wait complete. 2023-09-09T23:12:48.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.894+0000 7effcc688700 1 Processor -- start 2023-09-09T23:12:48.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.894+0000 7effcc688700 1 -- start start 2023-09-09T23:12:48.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.895+0000 7effcc688700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effc40ffec0 0x7effc4195540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.895+0000 7effcc688700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7effc4100870 0x7effc4195a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.897 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.895+0000 7effcc688700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effc41048e0 0x7effc4199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.897 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.895+0000 7effcc688700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc419a500 con 0x7effc41048e0 2023-09-09T23:12:48.897 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.895+0000 7effcc688700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc419a670 con 0x7effc4100870 2023-09-09T23:12:48.897 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.895+0000 7effcc688700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc419a830 con 0x7effc40ffec0 2023-09-09T23:12:48.897 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.895+0000 7effcac25700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effc41048e0 0x7effc4199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.898 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.895+0000 7effcac25700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effc41048e0 0x7effc4199ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54860/0 (socket says 172.21.15.82:54860) 2023-09-09T23:12:48.898 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.895+0000 7effcac25700 1 -- 172.21.15.82:0/4275485270 learned_addr learned my addr 172.21.15.82:0/4275485270 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:48.898 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.895+0000 7effca424700 1 --2- 172.21.15.82:0/4275485270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effc40ffec0 0x7effc4195540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.898 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.895+0000 7effc9c23700 1 --2- 172.21.15.82:0/4275485270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7effc4100870 0x7effc4195a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.898 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.895+0000 7effcac25700 1 -- 172.21.15.82:0/4275485270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effc40ffec0 msgr2=0x7effc4195540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.899 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.895+0000 7effcac25700 1 --2- 172.21.15.82:0/4275485270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effc40ffec0 0x7effc4195540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.899 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.895+0000 7effcac25700 1 -- 172.21.15.82:0/4275485270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7effc4100870 msgr2=0x7effc4195a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.899 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.895+0000 7effcac25700 1 --2- 172.21.15.82:0/4275485270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7effc4100870 0x7effc4195a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.899 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.895+0000 7effcac25700 1 -- 172.21.15.82:0/4275485270 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb40095f0 con 0x7effc41048e0 2023-09-09T23:12:48.899 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.896+0000 7effcac25700 1 --2- 172.21.15.82:0/4275485270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effc41048e0 0x7effc4199ef0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7effbc007ee0 tx=0x7effbc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:48.900 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.896+0000 7effbb7fe700 1 -- 172.21.15.82:0/4275485270 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc017070 con 0x7effc41048e0 2023-09-09T23:12:48.900 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.896+0000 7effcc688700 1 -- 172.21.15.82:0/4275485270 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc419ab10 con 0x7effc41048e0 2023-09-09T23:12:48.900 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.896+0000 7effbb7fe700 1 -- 172.21.15.82:0/4275485270 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effbc00f040 con 0x7effc41048e0 2023-09-09T23:12:48.900 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.896+0000 7effbb7fe700 1 -- 172.21.15.82:0/4275485270 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc005580 con 0x7effc41048e0 2023-09-09T23:12:48.900 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.896+0000 7effcc688700 1 -- 172.21.15.82:0/4275485270 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc40fed20 con 0x7effc41048e0 2023-09-09T23:12:48.900 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.897+0000 7effcc688700 1 -- 172.21.15.82:0/4275485270 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7effa8000ff0 con 0x7effc41048e0 2023-09-09T23:12:48.902 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.898+0000 7effbb7fe700 1 -- 172.21.15.82:0/4275485270 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7effbc004ad0 con 0x7effc41048e0 2023-09-09T23:12:48.902 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.899+0000 7effbb7fe700 1 --2- 172.21.15.82:0/4275485270 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7effb00a06d0 0x7effb00a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.902 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.899+0000 7effca424700 1 --2- 172.21.15.82:0/4275485270 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7effb00a06d0 0x7effb00a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.902 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.899+0000 7effbb7fe700 1 -- 172.21.15.82:0/4275485270 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7effbc0cbc30 con 0x7effc41048e0 2023-09-09T23:12:48.903 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.899+0000 7effbb7fe700 1 --2- 172.21.15.82:0/4275485270 >> [v2:172.21.15.170:6808/775005168,v1:172.21.15.170:6809/775005168] conn(0x7effb00a9eb0 0x7effb00ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.903 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.899+0000 7effbb7fe700 1 -- 172.21.15.82:0/4275485270 --> [v2:172.21.15.170:6808/775005168,v1:172.21.15.170:6809/775005168] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7effb00ac980 con 0x7effb00a9eb0 2023-09-09T23:12:48.903 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.899+0000 7effbb7fe700 1 -- 172.21.15.82:0/4275485270 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7effbc0cc0d0 con 0x7effc41048e0 2023-09-09T23:12:48.903 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.899+0000 7effca424700 1 --2- 172.21.15.82:0/4275485270 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7effb00a06d0 0x7effb00a2b90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7effc0005950 tx=0x7effc000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:48.903 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.899+0000 7effc9c23700 1 --2- 172.21.15.82:0/4275485270 >> [v2:172.21.15.170:6808/775005168,v1:172.21.15.170:6809/775005168] conn(0x7effb00a9eb0 0x7effb00ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.900+0000 7effc9c23700 1 --2- 172.21.15.82:0/4275485270 >> [v2:172.21.15.170:6808/775005168,v1:172.21.15.170:6809/775005168] conn(0x7effb00a9eb0 0x7effb00ac2d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:48.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.901+0000 7effbb7fe700 1 -- 172.21.15.82:0/4275485270 <== osd.5 v2:172.21.15.170:6808/775005168 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7effb00ac980 con 0x7effb00a9eb0 2023-09-09T23:12:48.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.757+0000 7f1089eb6700 1 -- 172.21.15.82:0/3541930905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1084068ae0 msgr2=0x7f108410d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.757+0000 7f1089eb6700 1 --2- 172.21.15.82:0/3541930905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1084068ae0 0x7f108410d440 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f107c00b0a0 tx=0x7f107c00b3b0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.757+0000 7f1089eb6700 1 -- 172.21.15.82:0/3541930905 shutdown_connections 2023-09-09T23:12:48.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.757+0000 7f1089eb6700 1 --2- 172.21.15.82:0/3541930905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1084068ae0 0x7f108410d440 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.757+0000 7f1089eb6700 1 --2- 172.21.15.82:0/3541930905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1084106770 0x7f1084106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.757+0000 7f1089eb6700 1 --2- 172.21.15.82:0/3541930905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1084068120 0x7f10840685a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.757+0000 7f1089eb6700 1 -- 172.21.15.82:0/3541930905 >> 172.21.15.82:0/3541930905 conn(0x7f1084076600 msgr2=0x7f1084076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:48.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.757+0000 7f1089eb6700 1 -- 172.21.15.82:0/3541930905 shutdown_connections 2023-09-09T23:12:48.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.757+0000 7f1089eb6700 1 -- 172.21.15.82:0/3541930905 wait complete. 2023-09-09T23:12:48.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.758+0000 7f1089eb6700 1 Processor -- start 2023-09-09T23:12:48.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.758+0000 7f1089eb6700 1 -- start start 2023-09-09T23:12:48.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.758+0000 7f1089eb6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1084068120 0x7f10840ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.758+0000 7f1089eb6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1084068ae0 0x7f10840ffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.758+0000 7f1089eb6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1084106770 0x7f1084100070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.758+0000 7f1089eb6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10841a4bb0 con 0x7f1084068120 2023-09-09T23:12:48.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.758+0000 7f1089eb6700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10841a4d20 con 0x7f1084068ae0 2023-09-09T23:12:48.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.758+0000 7f1089eb6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10841a4ee0 con 0x7f1084106770 2023-09-09T23:12:48.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.758+0000 7f10837fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1084068120 0x7f10840ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.758+0000 7f10837fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1084068120 0x7f10840ff5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54824/0 (socket says 172.21.15.82:54824) 2023-09-09T23:12:48.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.758+0000 7f10837fe700 1 -- 172.21.15.82:0/185963745 learned_addr learned my addr 172.21.15.82:0/185963745 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:48.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.758+0000 7f1083fff700 1 --2- 172.21.15.82:0/185963745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1084106770 0x7f1084100070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.758+0000 7f1082ffd700 1 --2- 172.21.15.82:0/185963745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1084068ae0 0x7f10840ffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.759+0000 7f10837fe700 1 -- 172.21.15.82:0/185963745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1084106770 msgr2=0x7f1084100070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.759+0000 7f10837fe700 1 --2- 172.21.15.82:0/185963745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1084106770 0x7f1084100070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.759+0000 7f10837fe700 1 -- 172.21.15.82:0/185963745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1084068ae0 msgr2=0x7f10840ffaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.759+0000 7f10837fe700 1 --2- 172.21.15.82:0/185963745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1084068ae0 0x7f10840ffaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.759+0000 7f10837fe700 1 -- 172.21.15.82:0/185963745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107c009d00 con 0x7f1084068120 2023-09-09T23:12:48.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.759+0000 7f10837fe700 1 --2- 172.21.15.82:0/185963745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1084068120 0x7f10840ff5b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f107400c9b0 tx=0x7f107400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:48.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.759+0000 7f1080ff9700 1 -- 172.21.15.82:0/185963745 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1074007b00 con 0x7f1084068120 2023-09-09T23:12:48.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.759+0000 7f1089eb6700 1 -- 172.21.15.82:0/185963745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10841a51c0 con 0x7f1084068120 2023-09-09T23:12:48.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.759+0000 7f1080ff9700 1 -- 172.21.15.82:0/185963745 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f107400ce50 con 0x7f1084068120 2023-09-09T23:12:48.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.759+0000 7f1080ff9700 1 -- 172.21.15.82:0/185963745 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1074018660 con 0x7f1084068120 2023-09-09T23:12:48.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.759+0000 7f1089eb6700 1 -- 172.21.15.82:0/185963745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10841a5710 con 0x7f1084068120 2023-09-09T23:12:48.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.760+0000 7f1089eb6700 1 -- 172.21.15.82:0/185963745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f10841108e0 con 0x7f1084068120 2023-09-09T23:12:48.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.761+0000 7f1080ff9700 1 -- 172.21.15.82:0/185963745 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f107400f450 con 0x7f1084068120 2023-09-09T23:12:48.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.762+0000 7f1080ff9700 1 --2- 172.21.15.82:0/185963745 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f106c0a06d0 0x7f106c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.762+0000 7f1080ff9700 1 -- 172.21.15.82:0/185963745 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f10740cbea0 con 0x7f1084068120 2023-09-09T23:12:48.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.762+0000 7f1082ffd700 1 --2- 172.21.15.82:0/185963745 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f106c0a06d0 0x7f106c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.762+0000 7f1080ff9700 1 --2- 172.21.15.82:0/185963745 >> [v2:172.21.15.98:6800/3528641314,v1:172.21.15.98:6801/3528641314] conn(0x7f106c0a9eb0 0x7f106c0ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:48.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.762+0000 7f1080ff9700 1 -- 172.21.15.82:0/185963745 --> [v2:172.21.15.98:6800/3528641314,v1:172.21.15.98:6801/3528641314] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f106c0ac980 con 0x7f106c0a9eb0 2023-09-09T23:12:48.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.762+0000 7f1080ff9700 1 -- 172.21.15.82:0/185963745 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10740cc340 con 0x7f1084068120 2023-09-09T23:12:48.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.762+0000 7f1082ffd700 1 --2- 172.21.15.82:0/185963745 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f106c0a06d0 0x7f106c0a2b90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1084100da0 tx=0x7f107800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:48.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.762+0000 7f1083fff700 1 --2- 172.21.15.82:0/185963745 >> [v2:172.21.15.98:6800/3528641314,v1:172.21.15.98:6801/3528641314] conn(0x7f106c0a9eb0 0x7f106c0ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:48.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.763+0000 7f1083fff700 1 --2- 172.21.15.82:0/185963745 >> [v2:172.21.15.98:6800/3528641314,v1:172.21.15.98:6801/3528641314] conn(0x7f106c0a9eb0 0x7f106c0ac2d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:48.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.763+0000 7f1080ff9700 1 -- 172.21.15.82:0/185963745 <== osd.1 v2:172.21.15.98:6800/3528641314 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f106c0ac980 con 0x7f106c0a9eb0 2023-09-09T23:12:48.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.780+0000 7f1089eb6700 1 -- 172.21.15.82:0/185963745 --> [v2:172.21.15.98:6800/3528641314,v1:172.21.15.98:6801/3528641314] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f108404e300 con 0x7f106c0a9eb0 2023-09-09T23:12:48.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.781+0000 7f1080ff9700 1 -- 172.21.15.82:0/185963745 <== osd.1 v2:172.21.15.98:6800/3528641314 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f108404e300 con 0x7f106c0a9eb0 2023-09-09T23:12:48.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.782+0000 7f1089eb6700 1 -- 172.21.15.82:0/185963745 >> [v2:172.21.15.98:6800/3528641314,v1:172.21.15.98:6801/3528641314] conn(0x7f106c0a9eb0 msgr2=0x7f106c0ac2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.782+0000 7f1089eb6700 1 --2- 172.21.15.82:0/185963745 >> [v2:172.21.15.98:6800/3528641314,v1:172.21.15.98:6801/3528641314] conn(0x7f106c0a9eb0 0x7f106c0ac2d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.782+0000 7f1089eb6700 1 -- 172.21.15.82:0/185963745 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f106c0a06d0 msgr2=0x7f106c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.782+0000 7f1089eb6700 1 --2- 172.21.15.82:0/185963745 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f106c0a06d0 0x7f106c0a2b90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1084100da0 tx=0x7f107800a380 comp rx=0 tx=0).stop 2023-09-09T23:12:48.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.782+0000 7f1089eb6700 1 -- 172.21.15.82:0/185963745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1084068120 msgr2=0x7f10840ff5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.782+0000 7f1089eb6700 1 --2- 172.21.15.82:0/185963745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1084068120 0x7f10840ff5b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f107400c9b0 tx=0x7f107400ccc0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.782+0000 7f1083fff700 1 -- 172.21.15.82:0/185963745 reap_dead start 2023-09-09T23:12:48.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.782+0000 7f1089eb6700 1 -- 172.21.15.82:0/185963745 shutdown_connections 2023-09-09T23:12:48.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.782+0000 7f1089eb6700 1 -- 172.21.15.82:0/185963745 >> 172.21.15.82:0/185963745 conn(0x7f1084076600 msgr2=0x7f1084107410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:48.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.782+0000 7f1089eb6700 1 -- 172.21.15.82:0/185963745 shutdown_connections 2023-09-09T23:12:48.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.783+0000 7f1089eb6700 1 -- 172.21.15.82:0/185963745 wait complete. 2023-09-09T23:12:48.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.920+0000 7effcc688700 1 -- 172.21.15.82:0/4275485270 --> [v2:172.21.15.170:6808/775005168,v1:172.21.15.170:6809/775005168] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7effa8002d70 con 0x7effb00a9eb0 2023-09-09T23:12:48.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.921+0000 7effbb7fe700 1 -- 172.21.15.82:0/4275485270 <== osd.5 v2:172.21.15.170:6808/775005168 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7effa8002d70 con 0x7effb00a9eb0 2023-09-09T23:12:48.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.921+0000 7effcc688700 1 -- 172.21.15.82:0/4275485270 >> [v2:172.21.15.170:6808/775005168,v1:172.21.15.170:6809/775005168] conn(0x7effb00a9eb0 msgr2=0x7effb00ac2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.921+0000 7effcc688700 1 --2- 172.21.15.82:0/4275485270 >> [v2:172.21.15.170:6808/775005168,v1:172.21.15.170:6809/775005168] conn(0x7effb00a9eb0 0x7effb00ac2d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.921+0000 7effcc688700 1 -- 172.21.15.82:0/4275485270 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7effb00a06d0 msgr2=0x7effb00a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.921+0000 7effcc688700 1 --2- 172.21.15.82:0/4275485270 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7effb00a06d0 0x7effb00a2b90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7effc0005950 tx=0x7effc000a300 comp rx=0 tx=0).stop 2023-09-09T23:12:48.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.921+0000 7effcc688700 1 -- 172.21.15.82:0/4275485270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effc41048e0 msgr2=0x7effc4199ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:48.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.921+0000 7effcc688700 1 --2- 172.21.15.82:0/4275485270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effc41048e0 0x7effc4199ef0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7effbc007ee0 tx=0x7effbc00d3b0 comp rx=0 tx=0).stop 2023-09-09T23:12:48.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.921+0000 7effcac25700 1 -- 172.21.15.82:0/4275485270 reap_dead start 2023-09-09T23:12:48.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.921+0000 7effcc688700 1 -- 172.21.15.82:0/4275485270 shutdown_connections 2023-09-09T23:12:48.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.921+0000 7effcc688700 1 -- 172.21.15.82:0/4275485270 >> 172.21.15.82:0/4275485270 conn(0x7effc4075a70 msgr2=0x7effc4103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:48.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.921+0000 7effcc688700 1 -- 172.21.15.82:0/4275485270 shutdown_connections 2023-09-09T23:12:48.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:48.921+0000 7effcc688700 1 -- 172.21.15.82:0/4275485270 wait complete. 2023-09-09T23:12:49.067 INFO:teuthology.orchestra.run.smithi082.stdout:34359738430 2023-09-09T23:12:49.068 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.0 2023-09-09T23:12:49.798 INFO:teuthology.orchestra.run.smithi082.stdout:158913789993 2023-09-09T23:12:49.799 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.5 2023-09-09T23:12:50.139 INFO:teuthology.orchestra.run.smithi082.stdout:326417514512 2023-09-09T23:12:50.139 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.11 2023-09-09T23:12:50.173 INFO:teuthology.orchestra.run.smithi082.stdout:244813135902 2023-09-09T23:12:50.174 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.8 2023-09-09T23:12:50.316 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:50 smithi082 ceph-mon[98694]: pgmap v271: 97 pgs: 97 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-09T23:12:50.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.205+0000 7f2dbff76700 1 -- 172.21.15.82:0/1568338293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2db8068ae0 msgr2=0x7f2db810d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:50.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.205+0000 7f2dbff76700 1 --2- 172.21.15.82:0/1568338293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2db8068ae0 0x7f2db810d440 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2db000b0a0 tx=0x7f2db000b3b0 comp rx=0 tx=0).stop 2023-09-09T23:12:50.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.206+0000 7f2dbff76700 1 -- 172.21.15.82:0/1568338293 shutdown_connections 2023-09-09T23:12:50.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.206+0000 7f2dbff76700 1 --2- 172.21.15.82:0/1568338293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2db8106770 0x7f2db8106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:50.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.206+0000 7f2dbff76700 1 --2- 172.21.15.82:0/1568338293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2db8068ae0 0x7f2db810d440 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:50.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.206+0000 7f2dbff76700 1 --2- 172.21.15.82:0/1568338293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2db8068120 0x7f2db80685a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:50.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.206+0000 7f2dbff76700 1 -- 172.21.15.82:0/1568338293 >> 172.21.15.82:0/1568338293 conn(0x7f2db8076600 msgr2=0x7f2db8076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:50.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.206+0000 7f2dbff76700 1 -- 172.21.15.82:0/1568338293 shutdown_connections 2023-09-09T23:12:50.340 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.206+0000 7f2dbff76700 1 -- 172.21.15.82:0/1568338293 wait complete. 2023-09-09T23:12:50.340 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.206+0000 7f2dbff76700 1 Processor -- start 2023-09-09T23:12:50.340 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.206+0000 7f2dbff76700 1 -- start start 2023-09-09T23:12:50.340 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.206+0000 7f2dbff76700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2db8068120 0x7f2db819c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:50.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.206+0000 7f2dbff76700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2db8068ae0 0x7f2db819caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:50.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.207+0000 7f2dbff76700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2db8106770 0x7f2db8196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:50.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.207+0000 7f2dbff76700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db8196ca0 con 0x7f2db8068ae0 2023-09-09T23:12:50.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.207+0000 7f2dbff76700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db8196e10 con 0x7f2db8106770 2023-09-09T23:12:50.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.207+0000 7f2dbff76700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db8196fd0 con 0x7f2db8068120 2023-09-09T23:12:50.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.207+0000 7f2dbd511700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2db8068ae0 0x7f2db819caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:50.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.207+0000 7f2dbd511700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2db8068ae0 0x7f2db819caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54888/0 (socket says 172.21.15.82:54888) 2023-09-09T23:12:50.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.207+0000 7f2dbd511700 1 -- 172.21.15.82:0/3467324780 learned_addr learned my addr 172.21.15.82:0/3467324780 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:50.344 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.207+0000 7f2dbe513700 1 --2- 172.21.15.82:0/3467324780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2db8106770 0x7f2db8196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:50.344 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.207+0000 7f2dbdd12700 1 --2- 172.21.15.82:0/3467324780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2db8068120 0x7f2db819c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:50.344 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.207+0000 7f2dbd511700 1 -- 172.21.15.82:0/3467324780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2db8068120 msgr2=0x7f2db819c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:50.344 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.207+0000 7f2dbd511700 1 --2- 172.21.15.82:0/3467324780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2db8068120 0x7f2db819c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:50.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.207+0000 7f2dbd511700 1 -- 172.21.15.82:0/3467324780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2db8106770 msgr2=0x7f2db8196630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:50.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.207+0000 7f2dbd511700 1 --2- 172.21.15.82:0/3467324780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2db8106770 0x7f2db8196630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:50.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.207+0000 7f2dbd511700 1 -- 172.21.15.82:0/3467324780 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db0009d00 con 0x7f2db8068ae0 2023-09-09T23:12:50.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.207+0000 7f2dbdd12700 1 --2- 172.21.15.82:0/3467324780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2db8068120 0x7f2db819c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:12:50.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.207+0000 7f2dbe513700 1 --2- 172.21.15.82:0/3467324780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2db8106770 0x7f2db8196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:12:50.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.207+0000 7f2dbd511700 1 --2- 172.21.15.82:0/3467324780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2db8068ae0 0x7f2db819caf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2da800c9b0 tx=0x7f2da800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:50.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.207+0000 7f2daeffd700 1 -- 172.21.15.82:0/3467324780 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da8007b00 con 0x7f2db8068ae0 2023-09-09T23:12:50.347 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.207+0000 7f2dbff76700 1 -- 172.21.15.82:0/3467324780 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db81972b0 con 0x7f2db8068ae0 2023-09-09T23:12:50.347 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.207+0000 7f2daeffd700 1 -- 172.21.15.82:0/3467324780 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2da8007c60 con 0x7f2db8068ae0 2023-09-09T23:12:50.347 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.207+0000 7f2daeffd700 1 -- 172.21.15.82:0/3467324780 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da8018660 con 0x7f2db8068ae0 2023-09-09T23:12:50.347 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.207+0000 7f2dbff76700 1 -- 172.21.15.82:0/3467324780 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db81a35a0 con 0x7f2db8068ae0 2023-09-09T23:12:50.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.208+0000 7f2dbff76700 1 -- 172.21.15.82:0/3467324780 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2db8110930 con 0x7f2db8068ae0 2023-09-09T23:12:50.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.209+0000 7f2daeffd700 1 -- 172.21.15.82:0/3467324780 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f2da8018880 con 0x7f2db8068ae0 2023-09-09T23:12:50.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.210+0000 7f2daeffd700 1 --2- 172.21.15.82:0/3467324780 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f2da40a06d0 0x7f2da40a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:50.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.210+0000 7f2dbdd12700 1 --2- 172.21.15.82:0/3467324780 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f2da40a06d0 0x7f2da40a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:50.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.210+0000 7f2daeffd700 1 -- 172.21.15.82:0/3467324780 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2da80cbc00 con 0x7f2db8068ae0 2023-09-09T23:12:50.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.210+0000 7f2daeffd700 1 --2- 172.21.15.82:0/3467324780 >> [v2:172.21.15.82:6810/3647629241,v1:172.21.15.82:6811/3647629241] conn(0x7f2da40a9eb0 0x7f2da40ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:50.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.210+0000 7f2daeffd700 1 -- 172.21.15.82:0/3467324780 --> [v2:172.21.15.82:6810/3647629241,v1:172.21.15.82:6811/3647629241] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2da40ac980 con 0x7f2da40a9eb0 2023-09-09T23:12:50.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.210+0000 7f2daeffd700 1 -- 172.21.15.82:0/3467324780 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2da80ce310 con 0x7f2db8068ae0 2023-09-09T23:12:50.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.210+0000 7f2dbe513700 1 --2- 172.21.15.82:0/3467324780 >> [v2:172.21.15.82:6810/3647629241,v1:172.21.15.82:6811/3647629241] conn(0x7f2da40a9eb0 0x7f2da40ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:50.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.210+0000 7f2dbdd12700 1 --2- 172.21.15.82:0/3467324780 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f2da40a06d0 0x7f2da40a2b90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2db4009b50 tx=0x7f2db4009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:50.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.210+0000 7f2dbe513700 1 --2- 172.21.15.82:0/3467324780 >> [v2:172.21.15.82:6810/3647629241,v1:172.21.15.82:6811/3647629241] conn(0x7f2da40a9eb0 0x7f2da40ac2d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:50.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.211+0000 7f2daeffd700 1 -- 172.21.15.82:0/3467324780 <== osd.3 v2:172.21.15.82:6810/3647629241 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f2da40ac980 con 0x7f2da40a9eb0 2023-09-09T23:12:50.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.237+0000 7f2dbff76700 1 -- 172.21.15.82:0/3467324780 --> [v2:172.21.15.82:6810/3647629241,v1:172.21.15.82:6811/3647629241] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2db804e300 con 0x7f2da40a9eb0 2023-09-09T23:12:50.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.237+0000 7f2daeffd700 1 -- 172.21.15.82:0/3467324780 <== osd.3 v2:172.21.15.82:6810/3647629241 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2db804e300 con 0x7f2da40a9eb0 2023-09-09T23:12:50.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.237+0000 7f2dbff76700 1 -- 172.21.15.82:0/3467324780 >> [v2:172.21.15.82:6810/3647629241,v1:172.21.15.82:6811/3647629241] conn(0x7f2da40a9eb0 msgr2=0x7f2da40ac2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:50.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.237+0000 7f2dbff76700 1 --2- 172.21.15.82:0/3467324780 >> [v2:172.21.15.82:6810/3647629241,v1:172.21.15.82:6811/3647629241] conn(0x7f2da40a9eb0 0x7f2da40ac2d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:50.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.238+0000 7f2dbff76700 1 -- 172.21.15.82:0/3467324780 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f2da40a06d0 msgr2=0x7f2da40a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:50.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.238+0000 7f2dbff76700 1 --2- 172.21.15.82:0/3467324780 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f2da40a06d0 0x7f2da40a2b90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2db4009b50 tx=0x7f2db4009550 comp rx=0 tx=0).stop 2023-09-09T23:12:50.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.238+0000 7f2dbff76700 1 -- 172.21.15.82:0/3467324780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2db8068ae0 msgr2=0x7f2db819caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:50.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.238+0000 7f2dbff76700 1 --2- 172.21.15.82:0/3467324780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2db8068ae0 0x7f2db819caf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2da800c9b0 tx=0x7f2da800cd70 comp rx=0 tx=0).stop 2023-09-09T23:12:50.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.238+0000 7f2dbe513700 1 -- 172.21.15.82:0/3467324780 reap_dead start 2023-09-09T23:12:50.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.238+0000 7f2dbff76700 1 -- 172.21.15.82:0/3467324780 shutdown_connections 2023-09-09T23:12:50.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.238+0000 7f2dbff76700 1 -- 172.21.15.82:0/3467324780 >> 172.21.15.82:0/3467324780 conn(0x7f2db8076600 msgr2=0x7f2db80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:50.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.238+0000 7f2dbff76700 1 -- 172.21.15.82:0/3467324780 shutdown_connections 2023-09-09T23:12:50.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:50.238+0000 7f2dbff76700 1 -- 172.21.15.82:0/3467324780 wait complete. 2023-09-09T23:12:50.531 INFO:teuthology.orchestra.run.smithi082.stdout:274877906968 2023-09-09T23:12:50.532 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.9 2023-09-09T23:12:50.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:50 smithi170 ceph-mon[102832]: pgmap v271: 97 pgs: 97 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-09T23:12:50.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:50 smithi098 ceph-mon[103124]: pgmap v271: 97 pgs: 97 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-09T23:12:50.924 INFO:teuthology.orchestra.run.smithi082.stdout:55834574907 2023-09-09T23:12:50.925 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.1 2023-09-09T23:12:50.965 INFO:teuthology.orchestra.run.smithi082.stdout:107374182450 2023-09-09T23:12:50.965 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.3 2023-09-09T23:12:50.971 INFO:teuthology.orchestra.run.smithi082.stdout:214748364833 2023-09-09T23:12:50.971 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.7 2023-09-09T23:12:50.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.386+0000 7f7b99019700 1 -- 172.21.15.82:0/2723742417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b940feef0 msgr2=0x7f7b940ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:50.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.386+0000 7f7b99019700 1 --2- 172.21.15.82:0/2723742417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b940feef0 0x7f7b940ff2d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009990 tx=0x7f7b84009ca0 comp rx=0 tx=0).stop 2023-09-09T23:12:50.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.386+0000 7f7b99019700 1 -- 172.21.15.82:0/2723742417 shutdown_connections 2023-09-09T23:12:50.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.386+0000 7f7b99019700 1 --2- 172.21.15.82:0/2723742417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b94103fd0 0x7f7b9410bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:50.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.386+0000 7f7b99019700 1 --2- 172.21.15.82:0/2723742417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b940feef0 0x7f7b940ff2d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:50.974 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.386+0000 7f7b99019700 1 --2- 172.21.15.82:0/2723742417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b940ff8a0 0x7f7b941038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:50.974 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.386+0000 7f7b99019700 1 -- 172.21.15.82:0/2723742417 >> 172.21.15.82:0/2723742417 conn(0x7f7b94075a70 msgr2=0x7f7b940fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:50.974 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.386+0000 7f7b99019700 1 -- 172.21.15.82:0/2723742417 shutdown_connections 2023-09-09T23:12:50.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.386+0000 7f7b99019700 1 -- 172.21.15.82:0/2723742417 wait complete. 2023-09-09T23:12:50.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.386+0000 7f7b99019700 1 Processor -- start 2023-09-09T23:12:50.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.387+0000 7f7b99019700 1 -- start start 2023-09-09T23:12:50.976 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.387+0000 7f7b99019700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b940feef0 0x7f7b94196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:50.976 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.387+0000 7f7b99019700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b940ff8a0 0x7f7b94196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:50.976 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.387+0000 7f7b99019700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b94103fd0 0x7f7b9419b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:50.976 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.387+0000 7f7b99019700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9419b630 con 0x7f7b940feef0 2023-09-09T23:12:50.977 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.387+0000 7f7b99019700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9419b7a0 con 0x7f7b94103fd0 2023-09-09T23:12:50.977 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.387+0000 7f7b99019700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9419b960 con 0x7f7b940ff8a0 2023-09-09T23:12:50.977 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.387+0000 7f7b92d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b940feef0 0x7f7b94196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:50.977 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.387+0000 7f7b92d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b940feef0 0x7f7b94196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54872/0 (socket says 172.21.15.82:54872) 2023-09-09T23:12:50.977 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.387+0000 7f7b92d9d700 1 -- 172.21.15.82:0/514287375 learned_addr learned my addr 172.21.15.82:0/514287375 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:50.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.387+0000 7f7b9359e700 1 --2- 172.21.15.82:0/514287375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b94103fd0 0x7f7b9419b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:50.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.387+0000 7f7b9259c700 1 --2- 172.21.15.82:0/514287375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b940ff8a0 0x7f7b94196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:50.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.387+0000 7f7b92d9d700 1 -- 172.21.15.82:0/514287375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b940ff8a0 msgr2=0x7f7b94196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:50.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.387+0000 7f7b92d9d700 1 --2- 172.21.15.82:0/514287375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b940ff8a0 0x7f7b94196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:50.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.387+0000 7f7b92d9d700 1 -- 172.21.15.82:0/514287375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b94103fd0 msgr2=0x7f7b9419b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:50.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.387+0000 7f7b92d9d700 1 --2- 172.21.15.82:0/514287375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b94103fd0 0x7f7b9419b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:50.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.387+0000 7f7b92d9d700 1 -- 172.21.15.82:0/514287375 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b840095f0 con 0x7f7b940feef0 2023-09-09T23:12:50.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.387+0000 7f7b9359e700 1 --2- 172.21.15.82:0/514287375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b94103fd0 0x7f7b9419b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:12:50.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.387+0000 7f7b9259c700 1 --2- 172.21.15.82:0/514287375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b940ff8a0 0x7f7b94196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:12:50.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.387+0000 7f7b92d9d700 1 --2- 172.21.15.82:0/514287375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b940feef0 0x7f7b94196670 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7b840095c0 tx=0x7f7b8400f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:50.980 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.387+0000 7f7b8bfff700 1 -- 172.21.15.82:0/514287375 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8401d070 con 0x7f7b940feef0 2023-09-09T23:12:50.980 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.388+0000 7f7b99019700 1 -- 172.21.15.82:0/514287375 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9419bbe0 con 0x7f7b940feef0 2023-09-09T23:12:50.980 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.388+0000 7f7b8bfff700 1 -- 172.21.15.82:0/514287375 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b8400fc00 con 0x7f7b940feef0 2023-09-09T23:12:50.980 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.388+0000 7f7b8bfff700 1 -- 172.21.15.82:0/514287375 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b840173f0 con 0x7f7b940feef0 2023-09-09T23:12:50.980 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.388+0000 7f7b99019700 1 -- 172.21.15.82:0/514287375 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b941a34a0 con 0x7f7b940feef0 2023-09-09T23:12:50.980 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.388+0000 7f7b99019700 1 -- 172.21.15.82:0/514287375 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7b9404e300 con 0x7f7b940feef0 2023-09-09T23:12:50.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.389+0000 7f7b8bfff700 1 -- 172.21.15.82:0/514287375 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f7b84022470 con 0x7f7b940feef0 2023-09-09T23:12:50.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.390+0000 7f7b8bfff700 1 --2- 172.21.15.82:0/514287375 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f7b800a06d0 0x7f7b800a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:50.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.390+0000 7f7b9259c700 1 --2- 172.21.15.82:0/514287375 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f7b800a06d0 0x7f7b800a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:50.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.390+0000 7f7b8bfff700 1 -- 172.21.15.82:0/514287375 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7b840cd530 con 0x7f7b940feef0 2023-09-09T23:12:50.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.390+0000 7f7b8bfff700 1 --2- 172.21.15.82:0/514287375 >> [v2:172.21.15.98:6824/2069114959,v1:172.21.15.98:6825/2069114959] conn(0x7f7b800a9eb0 0x7f7b800ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:50.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.390+0000 7f7b8bfff700 1 -- 172.21.15.82:0/514287375 --> [v2:172.21.15.98:6824/2069114959,v1:172.21.15.98:6825/2069114959] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7b800ac980 con 0x7f7b800a9eb0 2023-09-09T23:12:50.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.390+0000 7f7b8bfff700 1 -- 172.21.15.82:0/514287375 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b840cd9d0 con 0x7f7b940feef0 2023-09-09T23:12:50.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.390+0000 7f7b9259c700 1 --2- 172.21.15.82:0/514287375 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f7b800a06d0 0x7f7b800a2b90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c005fd0 tx=0x7f7b7c005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:50.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.391+0000 7f7b9359e700 1 --2- 172.21.15.82:0/514287375 >> [v2:172.21.15.98:6824/2069114959,v1:172.21.15.98:6825/2069114959] conn(0x7f7b800a9eb0 0x7f7b800ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:50.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.391+0000 7f7b9359e700 1 --2- 172.21.15.82:0/514287375 >> [v2:172.21.15.98:6824/2069114959,v1:172.21.15.98:6825/2069114959] conn(0x7f7b800a9eb0 0x7f7b800ac2d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:50.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.392+0000 7f7b8bfff700 1 -- 172.21.15.82:0/514287375 <== osd.10 v2:172.21.15.98:6824/2069114959 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f7b800ac980 con 0x7f7b800a9eb0 2023-09-09T23:12:50.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.408+0000 7f7b99019700 1 -- 172.21.15.82:0/514287375 --> [v2:172.21.15.98:6824/2069114959,v1:172.21.15.98:6825/2069114959] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7b94110900 con 0x7f7b800a9eb0 2023-09-09T23:12:50.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.408+0000 7f7b8bfff700 1 -- 172.21.15.82:0/514287375 <== osd.10 v2:172.21.15.98:6824/2069114959 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7b8c009d90 con 0x7f7b800a9eb0 2023-09-09T23:12:50.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.409+0000 7f7b99019700 1 -- 172.21.15.82:0/514287375 >> [v2:172.21.15.98:6824/2069114959,v1:172.21.15.98:6825/2069114959] conn(0x7f7b800a9eb0 msgr2=0x7f7b800ac2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:50.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.409+0000 7f7b99019700 1 --2- 172.21.15.82:0/514287375 >> [v2:172.21.15.98:6824/2069114959,v1:172.21.15.98:6825/2069114959] conn(0x7f7b800a9eb0 0x7f7b800ac2d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:50.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.409+0000 7f7b99019700 1 -- 172.21.15.82:0/514287375 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f7b800a06d0 msgr2=0x7f7b800a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:50.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.409+0000 7f7b99019700 1 --2- 172.21.15.82:0/514287375 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f7b800a06d0 0x7f7b800a2b90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c005fd0 tx=0x7f7b7c005e20 comp rx=0 tx=0).stop 2023-09-09T23:12:50.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.409+0000 7f7b99019700 1 -- 172.21.15.82:0/514287375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b940feef0 msgr2=0x7f7b94196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:50.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.409+0000 7f7b99019700 1 --2- 172.21.15.82:0/514287375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b940feef0 0x7f7b94196670 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7b840095c0 tx=0x7f7b8400f710 comp rx=0 tx=0).stop 2023-09-09T23:12:50.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.409+0000 7f7b9359e700 1 -- 172.21.15.82:0/514287375 reap_dead start 2023-09-09T23:12:50.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.409+0000 7f7b99019700 1 -- 172.21.15.82:0/514287375 shutdown_connections 2023-09-09T23:12:50.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.409+0000 7f7b99019700 1 -- 172.21.15.82:0/514287375 >> 172.21.15.82:0/514287375 conn(0x7f7b94075a70 msgr2=0x7f7b94102860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:50.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.409+0000 7f7b99019700 1 -- 172.21.15.82:0/514287375 shutdown_connections 2023-09-09T23:12:50.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:49.409+0000 7f7b99019700 1 -- 172.21.15.82:0/514287375 wait complete. 2023-09-09T23:12:51.016 INFO:teuthology.orchestra.run.smithi082.stdout:137438953518 2023-09-09T23:12:51.017 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.4 2023-09-09T23:12:51.280 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:51.458 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:51.460 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:51.598 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:51.811 INFO:teuthology.orchestra.run.smithi082.stdout:296352743445 2023-09-09T23:12:51.812 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph osd last-stat-seq osd.10 2023-09-09T23:12:52.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:52 smithi082 ceph-mon[98694]: pgmap v272: 97 pgs: 97 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2023-09-09T23:12:52.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:52 smithi170 ceph-mon[102832]: pgmap v272: 97 pgs: 97 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2023-09-09T23:12:52.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:52 smithi098 ceph-mon[103124]: pgmap v272: 97 pgs: 97 active+clean; 637 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2023-09-09T23:12:52.784 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:52.784 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:52.785 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:53.010 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:54.246 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:54.402 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:54.444 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:54.454 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:54 smithi082 ceph-mon[98694]: pgmap v273: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 11 op/s 2023-09-09T23:12:54.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:54 smithi170 ceph-mon[102832]: pgmap v273: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 11 op/s 2023-09-09T23:12:54.730 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.732+0000 7fb3d66a5700 1 -- 172.21.15.82:0/3790107558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3d0101250 msgr2=0x7fb3d010d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:54.730 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.732+0000 7fb3d66a5700 1 --2- 172.21.15.82:0/3790107558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3d0101250 0x7fb3d010d4f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800b0a0 tx=0x7fb3c800b3b0 comp rx=0 tx=0).stop 2023-09-09T23:12:54.730 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.732+0000 7fb3d66a5700 1 -- 172.21.15.82:0/3790107558 shutdown_connections 2023-09-09T23:12:54.731 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.733+0000 7fb3d66a5700 1 --2- 172.21.15.82:0/3790107558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3d0101250 0x7fb3d010d4f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:54.731 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.733+0000 7fb3d66a5700 1 --2- 172.21.15.82:0/3790107558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3d0106760 0x7fb3d0106b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:54.731 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.733+0000 7fb3d66a5700 1 --2- 172.21.15.82:0/3790107558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3d0100890 0x7fb3d0100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:54.731 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.733+0000 7fb3d66a5700 1 -- 172.21.15.82:0/3790107558 >> 172.21.15.82:0/3790107558 conn(0x7fb3d0078620 msgr2=0x7fb3d0078a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:54.731 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.733+0000 7fb3d66a5700 1 -- 172.21.15.82:0/3790107558 shutdown_connections 2023-09-09T23:12:54.732 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.733+0000 7fb3d66a5700 1 -- 172.21.15.82:0/3790107558 wait complete. 2023-09-09T23:12:54.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.733+0000 7fb3d66a5700 1 Processor -- start 2023-09-09T23:12:54.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.733+0000 7fb3d66a5700 1 -- start start 2023-09-09T23:12:54.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.734+0000 7fb3d66a5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3d0100890 0x7fb3d019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:54.735 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.734+0000 7fb3d66a5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3d0101250 0x7fb3d019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:54.735 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.734+0000 7fb3d66a5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3d0106760 0x7fb3d0196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:54.735 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.734+0000 7fb3d66a5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d0196bf0 con 0x7fb3d0106760 2023-09-09T23:12:54.735 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.734+0000 7fb3d66a5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d0196d60 con 0x7fb3d0100890 2023-09-09T23:12:54.735 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.734+0000 7fb3d66a5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d0196f20 con 0x7fb3d0101250 2023-09-09T23:12:54.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.734+0000 7fb3d4c42700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3d0106760 0x7fb3d0196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:54.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.734+0000 7fb3d4c42700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3d0106760 0x7fb3d0196580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54904/0 (socket says 172.21.15.82:54904) 2023-09-09T23:12:54.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.734+0000 7fb3d4c42700 1 -- 172.21.15.82:0/4209856865 learned_addr learned my addr 172.21.15.82:0/4209856865 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:54.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.734+0000 7fb3cffff700 1 --2- 172.21.15.82:0/4209856865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3d0100890 0x7fb3d019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:54.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.734+0000 7fb3cf7fe700 1 --2- 172.21.15.82:0/4209856865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3d0101250 0x7fb3d019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:54.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.734+0000 7fb3d4c42700 1 -- 172.21.15.82:0/4209856865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3d0101250 msgr2=0x7fb3d019ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:54.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.734+0000 7fb3d4c42700 1 --2- 172.21.15.82:0/4209856865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3d0101250 0x7fb3d019ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:54.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.734+0000 7fb3d4c42700 1 -- 172.21.15.82:0/4209856865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3d0100890 msgr2=0x7fb3d019c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:54.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.734+0000 7fb3d4c42700 1 --2- 172.21.15.82:0/4209856865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3d0100890 0x7fb3d019c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:54.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.734+0000 7fb3d4c42700 1 -- 172.21.15.82:0/4209856865 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c8009d00 con 0x7fb3d0106760 2023-09-09T23:12:54.738 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.734+0000 7fb3cf7fe700 1 --2- 172.21.15.82:0/4209856865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3d0101250 0x7fb3d019ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:12:54.738 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.734+0000 7fb3d4c42700 1 --2- 172.21.15.82:0/4209856865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3d0106760 0x7fb3d0196580 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800b830 tx=0x7fb3c8007a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:54.738 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.734+0000 7fb3cd7fa700 1 -- 172.21.15.82:0/4209856865 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c800e040 con 0x7fb3d0106760 2023-09-09T23:12:54.738 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.734+0000 7fb3cd7fa700 1 -- 172.21.15.82:0/4209856865 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3c8004e10 con 0x7fb3d0106760 2023-09-09T23:12:54.739 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.734+0000 7fb3d66a5700 1 -- 172.21.15.82:0/4209856865 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d01971a0 con 0x7fb3d0106760 2023-09-09T23:12:54.739 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.734+0000 7fb3cd7fa700 1 -- 172.21.15.82:0/4209856865 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c801d480 con 0x7fb3d0106760 2023-09-09T23:12:54.739 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.734+0000 7fb3d66a5700 1 -- 172.21.15.82:0/4209856865 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d01a34f0 con 0x7fb3d0106760 2023-09-09T23:12:54.739 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.735+0000 7fb3d66a5700 1 -- 172.21.15.82:0/4209856865 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3d004e300 con 0x7fb3d0106760 2023-09-09T23:12:54.739 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.736+0000 7fb3cd7fa700 1 -- 172.21.15.82:0/4209856865 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fb3c8004020 con 0x7fb3d0106760 2023-09-09T23:12:54.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.737+0000 7fb3cd7fa700 1 --2- 172.21.15.82:0/4209856865 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb3b80a06d0 0x7fb3b80a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:54.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.737+0000 7fb3cffff700 1 --2- 172.21.15.82:0/4209856865 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb3b80a06d0 0x7fb3b80a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:54.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.737+0000 7fb3cd7fa700 1 -- 172.21.15.82:0/4209856865 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb3c80cd3a0 con 0x7fb3d0106760 2023-09-09T23:12:54.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.737+0000 7fb3cffff700 1 --2- 172.21.15.82:0/4209856865 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb3b80a06d0 0x7fb3b80a2b90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0005950 tx=0x7fb3c00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:54.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.741+0000 7fb3cd7fa700 1 -- 172.21.15.82:0/4209856865 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb3c8096fb0 con 0x7fb3d0106760 2023-09-09T23:12:54.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:54 smithi098 ceph-mon[103124]: pgmap v273: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 11 op/s 2023-09-09T23:12:54.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.947+0000 7fb3d66a5700 1 -- 172.21.15.82:0/4209856865 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb3d0110650 con 0x7fb3d0106760 2023-09-09T23:12:54.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.948+0000 7fb3cd7fa700 1 -- 172.21.15.82:0/4209856865 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb3c809a5d0 con 0x7fb3d0106760 2023-09-09T23:12:54.946 INFO:teuthology.orchestra.run.smithi082.stdout:158913789994 2023-09-09T23:12:54.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.950+0000 7fb3d66a5700 1 -- 172.21.15.82:0/4209856865 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb3b80a06d0 msgr2=0x7fb3b80a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:54.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.950+0000 7fb3d66a5700 1 --2- 172.21.15.82:0/4209856865 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb3b80a06d0 0x7fb3b80a2b90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0005950 tx=0x7fb3c00058e0 comp rx=0 tx=0).stop 2023-09-09T23:12:54.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.950+0000 7fb3d66a5700 1 -- 172.21.15.82:0/4209856865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3d0106760 msgr2=0x7fb3d0196580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:54.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.950+0000 7fb3d66a5700 1 --2- 172.21.15.82:0/4209856865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3d0106760 0x7fb3d0196580 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800b830 tx=0x7fb3c8007a60 comp rx=0 tx=0).stop 2023-09-09T23:12:54.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.951+0000 7fb3d66a5700 1 -- 172.21.15.82:0/4209856865 shutdown_connections 2023-09-09T23:12:54.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.951+0000 7fb3d66a5700 1 --2- 172.21.15.82:0/4209856865 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb3b80a06d0 0x7fb3b80a2b90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:54.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.951+0000 7fb3d66a5700 1 --2- 172.21.15.82:0/4209856865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3d0100890 0x7fb3d019c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:54.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.951+0000 7fb3d66a5700 1 --2- 172.21.15.82:0/4209856865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3d0106760 0x7fb3d0196580 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:54.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.951+0000 7fb3d66a5700 1 --2- 172.21.15.82:0/4209856865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3d0101250 0x7fb3d019ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:54.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.951+0000 7fb3d66a5700 1 -- 172.21.15.82:0/4209856865 >> 172.21.15.82:0/4209856865 conn(0x7fb3d0078620 msgr2=0x7fb3d0107530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:54.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.951+0000 7fb3d66a5700 1 -- 172.21.15.82:0/4209856865 shutdown_connections 2023-09-09T23:12:54.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.951+0000 7fb3d66a5700 1 -- 172.21.15.82:0/4209856865 wait complete. 2023-09-09T23:12:55.133 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:12:55.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.994+0000 7f3c360e2700 1 -- 172.21.15.82:0/968667832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c300ff570 msgr2=0x7f3c300ff9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:55.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.994+0000 7f3c360e2700 1 --2- 172.21.15.82:0/968667832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c300ff570 0x7f3c300ff9f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3c240099c0 tx=0x7f3c24009cd0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.995+0000 7f3c360e2700 1 -- 172.21.15.82:0/968667832 shutdown_connections 2023-09-09T23:12:55.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.995+0000 7f3c360e2700 1 --2- 172.21.15.82:0/968667832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c30105610 0x7f3c301059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.995+0000 7f3c360e2700 1 --2- 172.21.15.82:0/968667832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c300fff30 0x7f3c3010c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.995+0000 7f3c360e2700 1 --2- 172.21.15.82:0/968667832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c300ff570 0x7f3c300ff9f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.995+0000 7f3c360e2700 1 -- 172.21.15.82:0/968667832 >> 172.21.15.82:0/968667832 conn(0x7f3c30076680 msgr2=0x7f3c30076a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:55.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.995+0000 7f3c360e2700 1 -- 172.21.15.82:0/968667832 shutdown_connections 2023-09-09T23:12:55.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.995+0000 7f3c360e2700 1 -- 172.21.15.82:0/968667832 wait complete. 2023-09-09T23:12:55.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.995+0000 7f3c360e2700 1 Processor -- start 2023-09-09T23:12:55.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.995+0000 7f3c360e2700 1 -- start start 2023-09-09T23:12:55.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.995+0000 7f3c360e2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c300ff570 0x7f3c3019c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:55.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.995+0000 7f3c360e2700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c300fff30 0x7f3c3019cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:55.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.995+0000 7f3c360e2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c30105610 0x7f3c30196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:55.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.996+0000 7f3c360e2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c30196ce0 con 0x7f3c30105610 2023-09-09T23:12:55.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.996+0000 7f3c360e2700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c30196e50 con 0x7f3c300fff30 2023-09-09T23:12:55.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.996+0000 7f3c360e2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c30197010 con 0x7f3c300ff570 2023-09-09T23:12:55.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.996+0000 7f3c2ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c30105610 0x7f3c30196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:55.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.996+0000 7f3c2ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c30105610 0x7f3c30196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54916/0 (socket says 172.21.15.82:54916) 2023-09-09T23:12:55.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.996+0000 7f3c2ffff700 1 -- 172.21.15.82:0/2933320783 learned_addr learned my addr 172.21.15.82:0/2933320783 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:55.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.996+0000 7f3c2effd700 1 --2- 172.21.15.82:0/2933320783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c300fff30 0x7f3c3019cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:55.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.996+0000 7f3c2f7fe700 1 --2- 172.21.15.82:0/2933320783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c300ff570 0x7f3c3019c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:55.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.996+0000 7f3c2ffff700 1 -- 172.21.15.82:0/2933320783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c300ff570 msgr2=0x7f3c3019c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:55.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.996+0000 7f3c2ffff700 1 --2- 172.21.15.82:0/2933320783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c300ff570 0x7f3c3019c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.996+0000 7f3c2ffff700 1 -- 172.21.15.82:0/2933320783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c300fff30 msgr2=0x7f3c3019cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:55.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.996+0000 7f3c2ffff700 1 --2- 172.21.15.82:0/2933320783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c300fff30 0x7f3c3019cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.996+0000 7f3c2ffff700 1 -- 172.21.15.82:0/2933320783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c240095f0 con 0x7f3c30105610 2023-09-09T23:12:55.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.996+0000 7f3c2f7fe700 1 --2- 172.21.15.82:0/2933320783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c300ff570 0x7f3c3019c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:12:55.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.996+0000 7f3c2ffff700 1 --2- 172.21.15.82:0/2933320783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c30105610 0x7f3c30196670 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800b970 tx=0x7f3c2800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:55.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.996+0000 7f3c2cff9700 1 -- 172.21.15.82:0/2933320783 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2800c9b0 con 0x7f3c30105610 2023-09-09T23:12:55.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.996+0000 7f3c360e2700 1 -- 172.21.15.82:0/2933320783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c301972f0 con 0x7f3c30105610 2023-09-09T23:12:55.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.996+0000 7f3c2cff9700 1 -- 172.21.15.82:0/2933320783 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c28014440 con 0x7f3c30105610 2023-09-09T23:12:55.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.996+0000 7f3c2cff9700 1 -- 172.21.15.82:0/2933320783 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c280075d0 con 0x7f3c30105610 2023-09-09T23:12:55.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.996+0000 7f3c360e2700 1 -- 172.21.15.82:0/2933320783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c301a35e0 con 0x7f3c30105610 2023-09-09T23:12:55.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.997+0000 7f3c360e2700 1 -- 172.21.15.82:0/2933320783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c3004e300 con 0x7f3c30105610 2023-09-09T23:12:55.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.999+0000 7f3c2cff9700 1 -- 172.21.15.82:0/2933320783 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f3c280145b0 con 0x7f3c30105610 2023-09-09T23:12:55.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:54.999+0000 7f3c2cff9700 1 --2- 172.21.15.82:0/2933320783 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3c180a06d0 0x7f3c180a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:55.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.000+0000 7f3c2f7fe700 1 --2- 172.21.15.82:0/2933320783 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3c180a06d0 0x7f3c180a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:55.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.000+0000 7f3c2cff9700 1 -- 172.21.15.82:0/2933320783 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3c28006e40 con 0x7f3c30105610 2023-09-09T23:12:55.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.000+0000 7f3c2f7fe700 1 --2- 172.21.15.82:0/2933320783 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3c180a06d0 0x7f3c180a2b90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3c20005fd0 tx=0x7f3c20005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:55.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.004+0000 7f3c2cff9700 1 -- 172.21.15.82:0/2933320783 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3c2804ee10 con 0x7f3c30105610 2023-09-09T23:12:55.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.209+0000 7f3c360e2700 1 -- 172.21.15.82:0/2933320783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3c3010f520 con 0x7f3c30105610 2023-09-09T23:12:55.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.210+0000 7f3c2cff9700 1 -- 172.21.15.82:0/2933320783 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3c28094e70 con 0x7f3c30105610 2023-09-09T23:12:55.208 INFO:teuthology.orchestra.run.smithi082.stdout:77309411384 2023-09-09T23:12:55.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.212+0000 7f3c360e2700 1 -- 172.21.15.82:0/2933320783 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3c180a06d0 msgr2=0x7f3c180a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:55.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.212+0000 7f3c360e2700 1 --2- 172.21.15.82:0/2933320783 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3c180a06d0 0x7f3c180a2b90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3c20005fd0 tx=0x7f3c20005ee0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.212+0000 7f3c360e2700 1 -- 172.21.15.82:0/2933320783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c30105610 msgr2=0x7f3c30196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:55.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.212+0000 7f3c360e2700 1 --2- 172.21.15.82:0/2933320783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c30105610 0x7f3c30196670 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800b970 tx=0x7f3c2800bd30 comp rx=0 tx=0).stop 2023-09-09T23:12:55.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.213+0000 7f3c360e2700 1 -- 172.21.15.82:0/2933320783 shutdown_connections 2023-09-09T23:12:55.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.213+0000 7f3c360e2700 1 --2- 172.21.15.82:0/2933320783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c30105610 0x7f3c30196670 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.213+0000 7f3c360e2700 1 --2- 172.21.15.82:0/2933320783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c300ff570 0x7f3c3019c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.213+0000 7f3c360e2700 1 --2- 172.21.15.82:0/2933320783 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3c180a06d0 0x7f3c180a2b90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.213+0000 7f3c360e2700 1 --2- 172.21.15.82:0/2933320783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c300fff30 0x7f3c3019cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.213+0000 7f3c360e2700 1 -- 172.21.15.82:0/2933320783 >> 172.21.15.82:0/2933320783 conn(0x7f3c30076680 msgr2=0x7f3c30068b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:55.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.214+0000 7f3c360e2700 1 -- 172.21.15.82:0/2933320783 shutdown_connections 2023-09-09T23:12:55.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.214+0000 7f3c360e2700 1 -- 172.21.15.82:0/2933320783 wait complete. 2023-09-09T23:12:55.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.453+0000 7ff220e13700 1 -- 172.21.15.82:0/961494002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff21c106700 msgr2=0x7ff21c106ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:55.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.453+0000 7ff220e13700 1 --2- 172.21.15.82:0/961494002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff21c106700 0x7ff21c106ae0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff20c009990 tx=0x7ff20c009ca0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.454+0000 7ff220e13700 1 -- 172.21.15.82:0/961494002 shutdown_connections 2023-09-09T23:12:55.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.454+0000 7ff220e13700 1 --2- 172.21.15.82:0/961494002 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff21c069440 0x7ff21c10d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.454+0000 7ff220e13700 1 --2- 172.21.15.82:0/961494002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff21c106700 0x7ff21c106ae0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.454+0000 7ff220e13700 1 --2- 172.21.15.82:0/961494002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff21c068a80 0x7ff21c068f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.454+0000 7ff220e13700 1 -- 172.21.15.82:0/961494002 >> 172.21.15.82:0/961494002 conn(0x7ff21c076680 msgr2=0x7ff21c076a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:55.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.454+0000 7ff220e13700 1 -- 172.21.15.82:0/961494002 shutdown_connections 2023-09-09T23:12:55.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.454+0000 7ff220e13700 1 -- 172.21.15.82:0/961494002 wait complete. 2023-09-09T23:12:55.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.455+0000 7ff220e13700 1 Processor -- start 2023-09-09T23:12:55.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.455+0000 7ff220e13700 1 -- start start 2023-09-09T23:12:55.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.455+0000 7ff220e13700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff21c068a80 0x7ff21c1966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:55.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.455+0000 7ff220e13700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff21c069440 0x7ff21c196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:55.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.455+0000 7ff220e13700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff21c106700 0x7ff21c199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:55.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.455+0000 7ff220e13700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c19a5a0 con 0x7ff21c106700 2023-09-09T23:12:55.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.455+0000 7ff220e13700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c19a710 con 0x7ff21c069440 2023-09-09T23:12:55.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.455+0000 7ff220e13700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c19a8d0 con 0x7ff21c068a80 2023-09-09T23:12:55.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.455+0000 7ff21ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff21c106700 0x7ff21c199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:55.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.455+0000 7ff21ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff21c106700 0x7ff21c199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54928/0 (socket says 172.21.15.82:54928) 2023-09-09T23:12:55.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.455+0000 7ff21ad9d700 1 -- 172.21.15.82:0/793181732 learned_addr learned my addr 172.21.15.82:0/793181732 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:55.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.455+0000 7ff21a59c700 1 --2- 172.21.15.82:0/793181732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff21c068a80 0x7ff21c1966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:55.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.455+0000 7ff219d9b700 1 --2- 172.21.15.82:0/793181732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff21c069440 0x7ff21c196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:55.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.455+0000 7ff21ad9d700 1 -- 172.21.15.82:0/793181732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff21c068a80 msgr2=0x7ff21c1966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:55.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.455+0000 7ff21ad9d700 1 --2- 172.21.15.82:0/793181732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff21c068a80 0x7ff21c1966a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.455+0000 7ff21ad9d700 1 -- 172.21.15.82:0/793181732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff21c069440 msgr2=0x7ff21c196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:55.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.455+0000 7ff21ad9d700 1 --2- 172.21.15.82:0/793181732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff21c069440 0x7ff21c196be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.456+0000 7ff21ad9d700 1 -- 172.21.15.82:0/793181732 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff20c0095f0 con 0x7ff21c106700 2023-09-09T23:12:55.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.456+0000 7ff21ad9d700 1 --2- 172.21.15.82:0/793181732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff21c106700 0x7ff21c199fa0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff214007ee0 tx=0x7ff21400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:55.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.456+0000 7ff20b7fe700 1 -- 172.21.15.82:0/793181732 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff214017070 con 0x7ff21c106700 2023-09-09T23:12:55.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.456+0000 7ff220e13700 1 -- 172.21.15.82:0/793181732 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff21c19abb0 con 0x7ff21c106700 2023-09-09T23:12:55.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.456+0000 7ff220e13700 1 -- 172.21.15.82:0/793181732 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff21c1a3500 con 0x7ff21c106700 2023-09-09T23:12:55.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.456+0000 7ff20b7fe700 1 -- 172.21.15.82:0/793181732 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff21400f040 con 0x7ff21c106700 2023-09-09T23:12:55.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.456+0000 7ff20b7fe700 1 -- 172.21.15.82:0/793181732 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff214005740 con 0x7ff21c106700 2023-09-09T23:12:55.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.457+0000 7ff220e13700 1 -- 172.21.15.82:0/793181732 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1fc005320 con 0x7ff21c106700 2023-09-09T23:12:55.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.458+0000 7ff20b7fe700 1 -- 172.21.15.82:0/793181732 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7ff2140058a0 con 0x7ff21c106700 2023-09-09T23:12:55.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.459+0000 7ff20b7fe700 1 --2- 172.21.15.82:0/793181732 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff2040a4ae0 0x7ff2040a6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:55.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.459+0000 7ff21a59c700 1 --2- 172.21.15.82:0/793181732 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff2040a4ae0 0x7ff2040a6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:55.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.459+0000 7ff20b7fe700 1 -- 172.21.15.82:0/793181732 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff2140cb2c0 con 0x7ff21c106700 2023-09-09T23:12:55.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.459+0000 7ff21a59c700 1 --2- 172.21.15.82:0/793181732 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff2040a4ae0 0x7ff2040a6fa0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff20c006010 tx=0x7ff20c00b580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:55.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.463+0000 7ff20b7fe700 1 -- 172.21.15.82:0/793181732 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff214094ed0 con 0x7ff21c106700 2023-09-09T23:12:55.542 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:55 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/4209856865' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-09-09T23:12:55.542 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:55 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2933320783' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-09-09T23:12:55.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.538+0000 7f11ef76f700 1 -- 172.21.15.82:0/1073321699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11e80feb90 msgr2=0x7f11e80ff010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:55.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.538+0000 7f11ef76f700 1 --2- 172.21.15.82:0/1073321699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11e80feb90 0x7f11e80ff010 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f11d80099c0 tx=0x7f11d8009cd0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.538+0000 7f11ef76f700 1 -- 172.21.15.82:0/1073321699 shutdown_connections 2023-09-09T23:12:55.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.538+0000 7f11ef76f700 1 --2- 172.21.15.82:0/1073321699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11e8106610 0x7f11e81069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.538+0000 7f11ef76f700 1 --2- 172.21.15.82:0/1073321699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11e80ff550 0x7f11e810d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.538+0000 7f11ef76f700 1 --2- 172.21.15.82:0/1073321699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11e80feb90 0x7f11e80ff010 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.538+0000 7f11ef76f700 1 -- 172.21.15.82:0/1073321699 >> 172.21.15.82:0/1073321699 conn(0x7f11e8075a70 msgr2=0x7f11e80fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:55.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.539+0000 7f11ef76f700 1 -- 172.21.15.82:0/1073321699 shutdown_connections 2023-09-09T23:12:55.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.539+0000 7f11ef76f700 1 -- 172.21.15.82:0/1073321699 wait complete. 2023-09-09T23:12:55.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.539+0000 7f11ef76f700 1 Processor -- start 2023-09-09T23:12:55.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.539+0000 7f11ef76f700 1 -- start start 2023-09-09T23:12:55.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.539+0000 7f11ef76f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11e80feb90 0x7f11e81966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:55.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.539+0000 7f11ef76f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11e80ff550 0x7f11e8196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:55.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.539+0000 7f11ef76f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11e8106610 0x7f11e8199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:55.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.539+0000 7f11ef76f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e819a5d0 con 0x7f11e80ff550 2023-09-09T23:12:55.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.539+0000 7f11ef76f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e819a740 con 0x7f11e8106610 2023-09-09T23:12:55.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.539+0000 7f11ef76f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e819a900 con 0x7f11e80feb90 2023-09-09T23:12:55.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.540+0000 7f11ecd0a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11e80ff550 0x7f11e8196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:55.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.540+0000 7f11ecd0a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11e80ff550 0x7f11e8196c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54940/0 (socket says 172.21.15.82:54940) 2023-09-09T23:12:55.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.540+0000 7f11ecd0a700 1 -- 172.21.15.82:0/2368875162 learned_addr learned my addr 172.21.15.82:0/2368875162 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:55.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.540+0000 7f11edd0c700 1 --2- 172.21.15.82:0/2368875162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11e8106610 0x7f11e8199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:55.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.540+0000 7f11ed50b700 1 --2- 172.21.15.82:0/2368875162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11e80feb90 0x7f11e81966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:55.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.540+0000 7f11ecd0a700 1 -- 172.21.15.82:0/2368875162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11e80feb90 msgr2=0x7f11e81966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:55.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.540+0000 7f11ecd0a700 1 --2- 172.21.15.82:0/2368875162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11e80feb90 0x7f11e81966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.540+0000 7f11ecd0a700 1 -- 172.21.15.82:0/2368875162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11e8106610 msgr2=0x7f11e8199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:55.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.540+0000 7f11ecd0a700 1 --2- 172.21.15.82:0/2368875162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11e8106610 0x7f11e8199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.540+0000 7f11ecd0a700 1 -- 172.21.15.82:0/2368875162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d80095f0 con 0x7f11e80ff550 2023-09-09T23:12:55.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.540+0000 7f11ecd0a700 1 --2- 172.21.15.82:0/2368875162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11e80ff550 0x7f11e8196c10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009990 tx=0x7f11d8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:55.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.540+0000 7f11de7fc700 1 -- 172.21.15.82:0/2368875162 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d801d070 con 0x7f11e80ff550 2023-09-09T23:12:55.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.540+0000 7f11ef76f700 1 -- 172.21.15.82:0/2368875162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e819ab80 con 0x7f11e80ff550 2023-09-09T23:12:55.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.540+0000 7f11de7fc700 1 -- 172.21.15.82:0/2368875162 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11d8004cb0 con 0x7f11e80ff550 2023-09-09T23:12:55.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.540+0000 7f11de7fc700 1 -- 172.21.15.82:0/2368875162 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d8003b30 con 0x7f11e80ff550 2023-09-09T23:12:55.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.540+0000 7f11ef76f700 1 -- 172.21.15.82:0/2368875162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e819ae50 con 0x7f11e80ff550 2023-09-09T23:12:55.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.542+0000 7f11ef76f700 1 -- 172.21.15.82:0/2368875162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e804e300 con 0x7f11e80ff550 2023-09-09T23:12:55.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.542+0000 7f11de7fc700 1 -- 172.21.15.82:0/2368875162 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f11d80047f0 con 0x7f11e80ff550 2023-09-09T23:12:55.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.543+0000 7f11de7fc700 1 --2- 172.21.15.82:0/2368875162 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f11d40a06d0 0x7f11d40a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:55.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.543+0000 7f11de7fc700 1 -- 172.21.15.82:0/2368875162 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f11d80cc980 con 0x7f11e80ff550 2023-09-09T23:12:55.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.543+0000 7f11ed50b700 1 --2- 172.21.15.82:0/2368875162 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f11d40a06d0 0x7f11d40a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:55.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.543+0000 7f11ed50b700 1 --2- 172.21.15.82:0/2368875162 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f11d40a06d0 0x7f11d40a2b90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f11e4005950 tx=0x7f11e400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:55.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.548+0000 7f11de7fc700 1 -- 172.21.15.82:0/2368875162 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f11d80966b0 con 0x7f11e80ff550 2023-09-09T23:12:55.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.666+0000 7ff220e13700 1 -- 172.21.15.82:0/793181732 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7ff1fc0058a0 con 0x7ff21c106700 2023-09-09T23:12:55.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.666+0000 7ff20b7fe700 1 -- 172.21.15.82:0/793181732 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff2140984f0 con 0x7ff21c106700 2023-09-09T23:12:55.664 INFO:teuthology.orchestra.run.smithi082.stdout:274877906969 2023-09-09T23:12:55.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.668+0000 7ff220e13700 1 -- 172.21.15.82:0/793181732 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff2040a4ae0 msgr2=0x7ff2040a6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:55.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.668+0000 7ff220e13700 1 --2- 172.21.15.82:0/793181732 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff2040a4ae0 0x7ff2040a6fa0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff20c006010 tx=0x7ff20c00b580 comp rx=0 tx=0).stop 2023-09-09T23:12:55.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.668+0000 7ff220e13700 1 -- 172.21.15.82:0/793181732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff21c106700 msgr2=0x7ff21c199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:55.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.668+0000 7ff220e13700 1 --2- 172.21.15.82:0/793181732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff21c106700 0x7ff21c199fa0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff214007ee0 tx=0x7ff21400d3b0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.668+0000 7ff220e13700 1 -- 172.21.15.82:0/793181732 shutdown_connections 2023-09-09T23:12:55.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.668+0000 7ff220e13700 1 --2- 172.21.15.82:0/793181732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff21c106700 0x7ff21c199fa0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.668+0000 7ff220e13700 1 --2- 172.21.15.82:0/793181732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff21c068a80 0x7ff21c1966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.668+0000 7ff220e13700 1 --2- 172.21.15.82:0/793181732 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff2040a4ae0 0x7ff2040a6fa0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.668+0000 7ff220e13700 1 --2- 172.21.15.82:0/793181732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff21c069440 0x7ff21c196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.668+0000 7ff220e13700 1 -- 172.21.15.82:0/793181732 >> 172.21.15.82:0/793181732 conn(0x7ff21c076680 msgr2=0x7ff21c0fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:55.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.668+0000 7ff220e13700 1 -- 172.21.15.82:0/793181732 shutdown_connections 2023-09-09T23:12:55.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.668+0000 7ff220e13700 1 -- 172.21.15.82:0/793181732 wait complete. 2023-09-09T23:12:55.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:55 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/4209856865' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-09-09T23:12:55.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:55 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2933320783' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-09-09T23:12:55.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:55 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/4209856865' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-09-09T23:12:55.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:55 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2933320783' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-09-09T23:12:55.762 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.764+0000 7f11ef76f700 1 -- 172.21.15.82:0/2368875162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f11e8110550 con 0x7f11e80ff550 2023-09-09T23:12:55.762 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.765+0000 7f11de7fc700 1 -- 172.21.15.82:0/2368875162 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f11d8026020 con 0x7f11e80ff550 2023-09-09T23:12:55.763 INFO:teuthology.orchestra.run.smithi082.stdout:34359738432 2023-09-09T23:12:55.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.768+0000 7f11ef76f700 1 -- 172.21.15.82:0/2368875162 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f11d40a06d0 msgr2=0x7f11d40a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:55.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.768+0000 7f11ef76f700 1 --2- 172.21.15.82:0/2368875162 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f11d40a06d0 0x7f11d40a2b90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f11e4005950 tx=0x7f11e400b410 comp rx=0 tx=0).stop 2023-09-09T23:12:55.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.768+0000 7f11ef76f700 1 -- 172.21.15.82:0/2368875162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11e80ff550 msgr2=0x7f11e8196c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:55.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.768+0000 7f11ef76f700 1 --2- 172.21.15.82:0/2368875162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11e80ff550 0x7f11e8196c10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009990 tx=0x7f11d8009580 comp rx=0 tx=0).stop 2023-09-09T23:12:55.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.768+0000 7f11ef76f700 1 -- 172.21.15.82:0/2368875162 shutdown_connections 2023-09-09T23:12:55.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.768+0000 7f11ef76f700 1 --2- 172.21.15.82:0/2368875162 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f11d40a06d0 0x7f11d40a2b90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.768+0000 7f11ef76f700 1 --2- 172.21.15.82:0/2368875162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11e8106610 0x7f11e8199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.768+0000 7f11ef76f700 1 --2- 172.21.15.82:0/2368875162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11e80feb90 0x7f11e81966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.768+0000 7f11ef76f700 1 --2- 172.21.15.82:0/2368875162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11e80ff550 0x7f11e8196c10 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:55.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.768+0000 7f11ef76f700 1 -- 172.21.15.82:0/2368875162 >> 172.21.15.82:0/2368875162 conn(0x7f11e8075a70 msgr2=0x7f11e80fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:55.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.768+0000 7f11ef76f700 1 -- 172.21.15.82:0/2368875162 shutdown_connections 2023-09-09T23:12:55.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:55.768+0000 7f11ef76f700 1 -- 172.21.15.82:0/2368875162 wait complete. 2023-09-09T23:12:55.782 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411382 got 77309411384 for osd.2 2023-09-09T23:12:55.782 DEBUG:teuthology.parallel:result is None 2023-09-09T23:12:56.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.245+0000 7f2567383700 1 -- 172.21.15.82:0/3152330494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25600ff880 msgr2=0x7f25600ffc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:56.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.245+0000 7f2567383700 1 --2- 172.21.15.82:0/3152330494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25600ff880 0x7f25600ffc60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2550009990 tx=0x7f2550009ca0 comp rx=0 tx=0).stop 2023-09-09T23:12:56.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.245+0000 7f2567383700 1 -- 172.21.15.82:0/3152330494 shutdown_connections 2023-09-09T23:12:56.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.245+0000 7f2567383700 1 --2- 172.21.15.82:0/3152330494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2560101840 0x7f256010c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:56.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.245+0000 7f2567383700 1 --2- 172.21.15.82:0/3152330494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25600ff880 0x7f25600ffc60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:56.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.245+0000 7f2567383700 1 --2- 172.21.15.82:0/3152330494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25601001a0 0x7f2560101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:56.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.245+0000 7f2567383700 1 -- 172.21.15.82:0/3152330494 >> 172.21.15.82:0/3152330494 conn(0x7f25600765f0 msgr2=0x7f2560076a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:56.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.245+0000 7f2567383700 1 -- 172.21.15.82:0/3152330494 shutdown_connections 2023-09-09T23:12:56.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.245+0000 7f2567383700 1 -- 172.21.15.82:0/3152330494 wait complete. 2023-09-09T23:12:56.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.246+0000 7f2567383700 1 Processor -- start 2023-09-09T23:12:56.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.246+0000 7f2567383700 1 -- start start 2023-09-09T23:12:56.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.246+0000 7f2567383700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25600ff880 0x7f25601953d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:56.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.246+0000 7f2567383700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25601001a0 0x7f2560195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:56.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.246+0000 7f2567383700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2560101840 0x7f2560195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:56.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.246+0000 7f2567383700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25601a4c10 con 0x7f25600ff880 2023-09-09T23:12:56.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.246+0000 7f2567383700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25601a4d80 con 0x7f25601001a0 2023-09-09T23:12:56.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.246+0000 7f2567383700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25601a4f40 con 0x7f2560101840 2023-09-09T23:12:56.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.246+0000 7f256511f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25600ff880 0x7f25601953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:56.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.246+0000 7f256511f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25600ff880 0x7f25601953d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54952/0 (socket says 172.21.15.82:54952) 2023-09-09T23:12:56.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.246+0000 7f256511f700 1 -- 172.21.15.82:0/3305526407 learned_addr learned my addr 172.21.15.82:0/3305526407 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:56.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.246+0000 7f2565920700 1 --2- 172.21.15.82:0/3305526407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2560101840 0x7f2560195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:56.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.246+0000 7f256491e700 1 --2- 172.21.15.82:0/3305526407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25601001a0 0x7f2560195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:56.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.246+0000 7f256511f700 1 -- 172.21.15.82:0/3305526407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2560101840 msgr2=0x7f2560195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:56.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.246+0000 7f256511f700 1 --2- 172.21.15.82:0/3305526407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2560101840 0x7f2560195e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:56.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.246+0000 7f256511f700 1 -- 172.21.15.82:0/3305526407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25601001a0 msgr2=0x7f2560195910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:56.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.246+0000 7f256511f700 1 --2- 172.21.15.82:0/3305526407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25601001a0 0x7f2560195910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:56.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.246+0000 7f256511f700 1 -- 172.21.15.82:0/3305526407 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25500095f0 con 0x7f25600ff880 2023-09-09T23:12:56.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.246+0000 7f256511f700 1 --2- 172.21.15.82:0/3305526407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25600ff880 0x7f25601953d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2550000c00 tx=0x7f255000f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:56.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.247+0000 7f25567fc700 1 -- 172.21.15.82:0/3305526407 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255001d070 con 0x7f25600ff880 2023-09-09T23:12:56.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.247+0000 7f25567fc700 1 -- 172.21.15.82:0/3305526407 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f255000fc00 con 0x7f25600ff880 2023-09-09T23:12:56.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.247+0000 7f2567383700 1 -- 172.21.15.82:0/3305526407 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25601a51c0 con 0x7f25600ff880 2023-09-09T23:12:56.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.247+0000 7f25567fc700 1 -- 172.21.15.82:0/3305526407 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2550017650 con 0x7f25600ff880 2023-09-09T23:12:56.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.247+0000 7f2567383700 1 -- 172.21.15.82:0/3305526407 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25601a5660 con 0x7f25600ff880 2023-09-09T23:12:56.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.247+0000 7f2567383700 1 -- 172.21.15.82:0/3305526407 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f256004e300 con 0x7f25600ff880 2023-09-09T23:12:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.248+0000 7f25567fc700 1 -- 172.21.15.82:0/3305526407 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f2550003960 con 0x7f25600ff880 2023-09-09T23:12:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.249+0000 7f25567fc700 1 --2- 172.21.15.82:0/3305526407 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f254c0a06d0 0x7f254c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.249+0000 7f256491e700 1 --2- 172.21.15.82:0/3305526407 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f254c0a06d0 0x7f254c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.249+0000 7f25567fc700 1 -- 172.21.15.82:0/3305526407 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f25500cd200 con 0x7f25600ff880 2023-09-09T23:12:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.250+0000 7f256491e700 1 --2- 172.21.15.82:0/3305526407 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f254c0a06d0 0x7f254c0a2b90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f25600692b0 tx=0x7f255c00b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:56.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.253+0000 7f25567fc700 1 -- 172.21.15.82:0/3305526407 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2550096d10 con 0x7f25600ff880 2023-09-09T23:12:56.451 INFO:teuthology.orchestra.run.smithi082.stdout:193273528359 2023-09-09T23:12:56.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.453+0000 7f2567383700 1 -- 172.21.15.82:0/3305526407 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f2560068f40 con 0x7f25600ff880 2023-09-09T23:12:56.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.453+0000 7f25567fc700 1 -- 172.21.15.82:0/3305526407 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f255009a330 con 0x7f25600ff880 2023-09-09T23:12:56.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.455+0000 7f2567383700 1 -- 172.21.15.82:0/3305526407 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f254c0a06d0 msgr2=0x7f254c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:56.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.455+0000 7f2567383700 1 --2- 172.21.15.82:0/3305526407 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f254c0a06d0 0x7f254c0a2b90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f25600692b0 tx=0x7f255c00b5b0 comp rx=0 tx=0).stop 2023-09-09T23:12:56.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.455+0000 7f2567383700 1 -- 172.21.15.82:0/3305526407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25600ff880 msgr2=0x7f25601953d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:56.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.455+0000 7f2567383700 1 --2- 172.21.15.82:0/3305526407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25600ff880 0x7f25601953d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2550000c00 tx=0x7f255000f710 comp rx=0 tx=0).stop 2023-09-09T23:12:56.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.455+0000 7f2567383700 1 -- 172.21.15.82:0/3305526407 shutdown_connections 2023-09-09T23:12:56.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.455+0000 7f2567383700 1 --2- 172.21.15.82:0/3305526407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2560101840 0x7f2560195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:56.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.455+0000 7f2567383700 1 --2- 172.21.15.82:0/3305526407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25600ff880 0x7f25601953d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:56.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.455+0000 7f2567383700 1 --2- 172.21.15.82:0/3305526407 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f254c0a06d0 0x7f254c0a2b90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:56.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.455+0000 7f2567383700 1 --2- 172.21.15.82:0/3305526407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25601001a0 0x7f2560195910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:56.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.455+0000 7f2567383700 1 -- 172.21.15.82:0/3305526407 >> 172.21.15.82:0/3305526407 conn(0x7f25600765f0 msgr2=0x7f25600fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:56.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.456+0000 7f2567383700 1 -- 172.21.15.82:0/3305526407 shutdown_connections 2023-09-09T23:12:56.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:56.456+0000 7f2567383700 1 -- 172.21.15.82:0/3305526407 wait complete. 2023-09-09T23:12:56.473 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:56 smithi082 ceph-mon[98694]: pgmap v274: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-09T23:12:56.473 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:56 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/793181732' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-09-09T23:12:56.473 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:56 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2368875162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-09-09T23:12:56.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:56 smithi170 ceph-mon[102832]: pgmap v274: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-09T23:12:56.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:56 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/793181732' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-09-09T23:12:56.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:56 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2368875162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-09-09T23:12:56.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:56 smithi098 ceph-mon[103124]: pgmap v274: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-09T23:12:56.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:56 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/793181732' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-09-09T23:12:56.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:56 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2368875162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-09-09T23:12:56.846 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789993 got 158913789994 for osd.5 2023-09-09T23:12:56.846 DEBUG:teuthology.parallel:result is None 2023-09-09T23:12:56.984 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906968 got 274877906969 for osd.9 2023-09-09T23:12:56.984 DEBUG:teuthology.parallel:result is None 2023-09-09T23:12:57.649 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:57 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3305526407' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-09-09T23:12:57.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:57 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3305526407' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-09-09T23:12:57.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:57 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3305526407' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-09-09T23:12:57.811 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738430 got 34359738432 for osd.0 2023-09-09T23:12:57.812 DEBUG:teuthology.parallel:result is None 2023-09-09T23:12:58.648 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.650+0000 7fa2765a1700 1 -- 172.21.15.82:0/892360734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2700ffe20 msgr2=0x7fa270100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:58.648 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.650+0000 7fa2765a1700 1 --2- 172.21.15.82:0/892360734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2700ffe20 0x7fa270100200 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa26c009990 tx=0x7fa26c009ca0 comp rx=0 tx=0).stop 2023-09-09T23:12:58.648 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.650+0000 7fa2765a1700 1 -- 172.21.15.82:0/892360734 shutdown_connections 2023-09-09T23:12:58.649 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.650+0000 7fa2765a1700 1 --2- 172.21.15.82:0/892360734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2701046e0 0x7fa27010c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:58.649 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.650+0000 7fa2765a1700 1 --2- 172.21.15.82:0/892360734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2700ffe20 0x7fa270100200 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:58.649 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.650+0000 7fa2765a1700 1 --2- 172.21.15.82:0/892360734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2701007d0 0x7fa270104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:58.649 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.650+0000 7fa2765a1700 1 -- 172.21.15.82:0/892360734 >> 172.21.15.82:0/892360734 conn(0x7fa2700fac00 msgr2=0x7fa2700fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:58.649 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.650+0000 7fa2765a1700 1 -- 172.21.15.82:0/892360734 shutdown_connections 2023-09-09T23:12:58.650 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.650+0000 7fa2765a1700 1 -- 172.21.15.82:0/892360734 wait complete. 2023-09-09T23:12:58.650 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.651+0000 7fa2765a1700 1 Processor -- start 2023-09-09T23:12:58.650 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.651+0000 7fa2765a1700 1 -- start start 2023-09-09T23:12:58.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.651+0000 7fa2765a1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2700ffe20 0x7fa2700fb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:58.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.651+0000 7fa2765a1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2701007d0 0x7fa2700fb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:58.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.651+0000 7fa2765a1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2701046e0 0x7fa2700fbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:58.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.651+0000 7fa2765a1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27019a4d0 con 0x7fa2700ffe20 2023-09-09T23:12:58.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.651+0000 7fa2765a1700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2701a8ea0 con 0x7fa2701046e0 2023-09-09T23:12:58.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.651+0000 7fa2765a1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2701a9060 con 0x7fa2701007d0 2023-09-09T23:12:58.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.651+0000 7fa27559f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2700ffe20 0x7fa2700fb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:58.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.651+0000 7fa27559f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2700ffe20 0x7fa2700fb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54964/0 (socket says 172.21.15.82:54964) 2023-09-09T23:12:58.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.651+0000 7fa27559f700 1 -- 172.21.15.82:0/2994954594 learned_addr learned my addr 172.21.15.82:0/2994954594 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:58.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.652+0000 7fa274d9e700 1 --2- 172.21.15.82:0/2994954594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2701007d0 0x7fa2700fb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:58.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.652+0000 7fa275da0700 1 --2- 172.21.15.82:0/2994954594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2701046e0 0x7fa2700fbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:58.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.652+0000 7fa27559f700 1 -- 172.21.15.82:0/2994954594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2701007d0 msgr2=0x7fa2700fb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:58.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.652+0000 7fa27559f700 1 --2- 172.21.15.82:0/2994954594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2701007d0 0x7fa2700fb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:58.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.652+0000 7fa27559f700 1 -- 172.21.15.82:0/2994954594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2701046e0 msgr2=0x7fa2700fbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:58.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.652+0000 7fa27559f700 1 --2- 172.21.15.82:0/2994954594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2701046e0 0x7fa2700fbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:58.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.652+0000 7fa27559f700 1 -- 172.21.15.82:0/2994954594 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa26c0095f0 con 0x7fa2700ffe20 2023-09-09T23:12:58.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.652+0000 7fa274d9e700 1 --2- 172.21.15.82:0/2994954594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2701007d0 0x7fa2700fb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:12:58.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.652+0000 7fa275da0700 1 --2- 172.21.15.82:0/2994954594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2701046e0 0x7fa2700fbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:12:58.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.652+0000 7fa27559f700 1 --2- 172.21.15.82:0/2994954594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2700ffe20 0x7fa2700fb0b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa26c0095c0 tx=0x7fa26c00f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:58.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.652+0000 7fa2667fc700 1 -- 172.21.15.82:0/2994954594 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c02e070 con 0x7fa2700ffe20 2023-09-09T23:12:58.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.652+0000 7fa2667fc700 1 -- 172.21.15.82:0/2994954594 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa26c00fc00 con 0x7fa2700ffe20 2023-09-09T23:12:58.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.652+0000 7fa2765a1700 1 -- 172.21.15.82:0/2994954594 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27019e680 con 0x7fa2700ffe20 2023-09-09T23:12:58.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.652+0000 7fa2667fc700 1 -- 172.21.15.82:0/2994954594 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c028400 con 0x7fa2700ffe20 2023-09-09T23:12:58.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.652+0000 7fa2765a1700 1 -- 172.21.15.82:0/2994954594 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27019eb20 con 0x7fa2700ffe20 2023-09-09T23:12:58.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.654+0000 7fa2765a1700 1 -- 172.21.15.82:0/2994954594 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27004eb50 con 0x7fa2700ffe20 2023-09-09T23:12:58.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.654+0000 7fa2667fc700 1 -- 172.21.15.82:0/2994954594 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fa26c033470 con 0x7fa2700ffe20 2023-09-09T23:12:58.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.655+0000 7fa2667fc700 1 --2- 172.21.15.82:0/2994954594 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa25c0c26e0 0x7fa25c0c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:58.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.655+0000 7fa2667fc700 1 -- 172.21.15.82:0/2994954594 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa26c0cc820 con 0x7fa2700ffe20 2023-09-09T23:12:58.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.655+0000 7fa274d9e700 1 --2- 172.21.15.82:0/2994954594 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa25c0c26e0 0x7fa25c0c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:58.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.655+0000 7fa274d9e700 1 --2- 172.21.15.82:0/2994954594 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa25c0c26e0 0x7fa25c0c4ba0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa260005fd0 tx=0x7fa260005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:58.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.659+0000 7fa2667fc700 1 -- 172.21.15.82:0/2994954594 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa26c096550 con 0x7fa2700ffe20 2023-09-09T23:12:58.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:58 smithi170 ceph-mon[102832]: pgmap v275: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 10 op/s 2023-09-09T23:12:58.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:58 smithi082 ceph-mon[98694]: pgmap v275: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 10 op/s 2023-09-09T23:12:58.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:58 smithi098 ceph-mon[103124]: pgmap v275: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 10 op/s 2023-09-09T23:12:58.767 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528357 got 193273528359 for osd.6 2023-09-09T23:12:58.767 DEBUG:teuthology.parallel:result is None 2023-09-09T23:12:58.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.895+0000 7fa2765a1700 1 -- 172.21.15.82:0/2994954594 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fa27010f380 con 0x7fa2700ffe20 2023-09-09T23:12:58.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.895+0000 7fa2667fc700 1 -- 172.21.15.82:0/2994954594 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fa26c01c020 con 0x7fa2700ffe20 2023-09-09T23:12:58.893 INFO:teuthology.orchestra.run.smithi082.stdout:296352743447 2023-09-09T23:12:58.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.897+0000 7fa2765a1700 1 -- 172.21.15.82:0/2994954594 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa25c0c26e0 msgr2=0x7fa25c0c4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:58.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.897+0000 7fa2765a1700 1 --2- 172.21.15.82:0/2994954594 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa25c0c26e0 0x7fa25c0c4ba0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa260005fd0 tx=0x7fa260005dc0 comp rx=0 tx=0).stop 2023-09-09T23:12:58.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.897+0000 7fa2765a1700 1 -- 172.21.15.82:0/2994954594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2700ffe20 msgr2=0x7fa2700fb0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:58.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.897+0000 7fa2765a1700 1 --2- 172.21.15.82:0/2994954594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2700ffe20 0x7fa2700fb0b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa26c0095c0 tx=0x7fa26c00f710 comp rx=0 tx=0).stop 2023-09-09T23:12:58.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.897+0000 7fa2765a1700 1 -- 172.21.15.82:0/2994954594 shutdown_connections 2023-09-09T23:12:58.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.897+0000 7fa2765a1700 1 --2- 172.21.15.82:0/2994954594 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa25c0c26e0 0x7fa25c0c4ba0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:58.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.897+0000 7fa2765a1700 1 --2- 172.21.15.82:0/2994954594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2701046e0 0x7fa2700fbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:58.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.897+0000 7fa2765a1700 1 --2- 172.21.15.82:0/2994954594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2700ffe20 0x7fa2700fb0b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:58.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.897+0000 7fa2765a1700 1 --2- 172.21.15.82:0/2994954594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2701007d0 0x7fa2700fb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:58.897 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.897+0000 7fa2765a1700 1 -- 172.21.15.82:0/2994954594 >> 172.21.15.82:0/2994954594 conn(0x7fa2700fac00 msgr2=0x7fa2700feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:58.897 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.897+0000 7fa2765a1700 1 -- 172.21.15.82:0/2994954594 shutdown_connections 2023-09-09T23:12:58.897 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:58.897+0000 7fa2765a1700 1 -- 172.21.15.82:0/2994954594 wait complete. 2023-09-09T23:12:59.651 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:12:59 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2994954594' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-09-09T23:12:59.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:12:59 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2994954594' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-09-09T23:12:59.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:12:59 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2994954594' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-09-09T23:12:59.801 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.803+0000 7fcdebc0f700 1 -- 172.21.15.82:0/2767908848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde40ffee0 msgr2=0x7fcde410c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:59.801 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.803+0000 7fcdebc0f700 1 --2- 172.21.15.82:0/2767908848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde40ffee0 0x7fcde410c3f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00b0a0 tx=0x7fcddc00b3b0 comp rx=0 tx=0).stop 2023-09-09T23:12:59.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.803+0000 7fcdebc0f700 1 -- 172.21.15.82:0/2767908848 shutdown_connections 2023-09-09T23:12:59.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.803+0000 7fcdebc0f700 1 --2- 172.21.15.82:0/2767908848 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcde4104530 0x7fcde4104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:59.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.803+0000 7fcdebc0f700 1 --2- 172.21.15.82:0/2767908848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde40ffee0 0x7fcde410c3f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:59.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.803+0000 7fcdebc0f700 1 --2- 172.21.15.82:0/2767908848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcde40ff520 0x7fcde40ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:59.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.803+0000 7fcdebc0f700 1 -- 172.21.15.82:0/2767908848 >> 172.21.15.82:0/2767908848 conn(0x7fcde4076600 msgr2=0x7fcde4076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:59.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.804+0000 7fcdebc0f700 1 -- 172.21.15.82:0/2767908848 shutdown_connections 2023-09-09T23:12:59.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.804+0000 7fcdebc0f700 1 -- 172.21.15.82:0/2767908848 wait complete. 2023-09-09T23:12:59.804 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.804+0000 7fcdebc0f700 1 Processor -- start 2023-09-09T23:12:59.805 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.804+0000 7fcdebc0f700 1 -- start start 2023-09-09T23:12:59.805 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.805+0000 7fcdebc0f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde40ff520 0x7fcde419b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:59.805 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.805+0000 7fcdebc0f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcde40ffee0 0x7fcde419bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:59.805 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.805+0000 7fcdebc0f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcde4104530 0x7fcde4195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:59.805 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.805+0000 7fcdebc0f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde4195cb0 con 0x7fcde40ff520 2023-09-09T23:12:59.806 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.805+0000 7fcdebc0f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde4195e20 con 0x7fcde40ffee0 2023-09-09T23:12:59.806 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.805+0000 7fcdebc0f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde4195fe0 con 0x7fcde4104530 2023-09-09T23:12:59.806 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.805+0000 7fcde99ab700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde40ff520 0x7fcde419b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:59.806 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.805+0000 7fcde99ab700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde40ff520 0x7fcde419b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54976/0 (socket says 172.21.15.82:54976) 2023-09-09T23:12:59.806 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.805+0000 7fcde99ab700 1 -- 172.21.15.82:0/2933229283 learned_addr learned my addr 172.21.15.82:0/2933229283 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:59.806 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.805+0000 7fcde91aa700 1 --2- 172.21.15.82:0/2933229283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcde40ffee0 0x7fcde419bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:59.807 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.805+0000 7fcdea1ac700 1 --2- 172.21.15.82:0/2933229283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcde4104530 0x7fcde4195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:59.807 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.805+0000 7fcde99ab700 1 -- 172.21.15.82:0/2933229283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcde4104530 msgr2=0x7fcde4195640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:59.807 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.805+0000 7fcde99ab700 1 --2- 172.21.15.82:0/2933229283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcde4104530 0x7fcde4195640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:59.807 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.805+0000 7fcde99ab700 1 -- 172.21.15.82:0/2933229283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcde40ffee0 msgr2=0x7fcde419bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:59.807 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.805+0000 7fcde99ab700 1 --2- 172.21.15.82:0/2933229283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcde40ffee0 0x7fcde419bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:59.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.805+0000 7fcde99ab700 1 -- 172.21.15.82:0/2933229283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc009d00 con 0x7fcde40ff520 2023-09-09T23:12:59.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.805+0000 7fcde99ab700 1 --2- 172.21.15.82:0/2933229283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde40ff520 0x7fcde419b5c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcde000c970 tx=0x7fcde000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:59.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.805+0000 7fcddaffd700 1 -- 172.21.15.82:0/2933229283 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde0004220 con 0x7fcde40ff520 2023-09-09T23:12:59.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.805+0000 7fcddaffd700 1 -- 172.21.15.82:0/2933229283 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcde0007d60 con 0x7fcde40ff520 2023-09-09T23:12:59.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.805+0000 7fcddaffd700 1 -- 172.21.15.82:0/2933229283 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde0003c90 con 0x7fcde40ff520 2023-09-09T23:12:59.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.805+0000 7fcdebc0f700 1 -- 172.21.15.82:0/2933229283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde41962c0 con 0x7fcde40ff520 2023-09-09T23:12:59.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.805+0000 7fcdebc0f700 1 -- 172.21.15.82:0/2933229283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde4068290 con 0x7fcde40ff520 2023-09-09T23:12:59.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.806+0000 7fcdebc0f700 1 -- 172.21.15.82:0/2933229283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde404e300 con 0x7fcde40ff520 2023-09-09T23:12:59.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.807+0000 7fcddaffd700 1 -- 172.21.15.82:0/2933229283 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fcde0004380 con 0x7fcde40ff520 2023-09-09T23:12:59.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.808+0000 7fcddaffd700 1 --2- 172.21.15.82:0/2933229283 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcdd00a0750 0x7fcdd00a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:59.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.808+0000 7fcddaffd700 1 -- 172.21.15.82:0/2933229283 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcde00cb650 con 0x7fcde40ff520 2023-09-09T23:12:59.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.808+0000 7fcde91aa700 1 --2- 172.21.15.82:0/2933229283 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcdd00a0750 0x7fcdd00a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:59.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.808+0000 7fcde91aa700 1 --2- 172.21.15.82:0/2933229283 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcdd00a0750 0x7fcdd00a2c10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcde4076ea0 tx=0x7fcdd4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:59.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.812+0000 7fcddaffd700 1 -- 172.21.15.82:0/2933229283 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fcde00951d0 con 0x7fcde40ff520 2023-09-09T23:12:59.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.943+0000 7f0ef641c700 1 -- 172.21.15.82:0/3546580553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ef0068120 msgr2=0x7f0ef00685a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:59.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.943+0000 7f0ef641c700 1 --2- 172.21.15.82:0/3546580553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ef0068120 0x7f0ef00685a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009990 tx=0x7f0ee4009ca0 comp rx=0 tx=0).stop 2023-09-09T23:12:59.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.943+0000 7f0ef641c700 1 -- 172.21.15.82:0/3546580553 shutdown_connections 2023-09-09T23:12:59.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.943+0000 7f0ef641c700 1 --2- 172.21.15.82:0/3546580553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ef0068ae0 0x7f0ef010d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:59.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.943+0000 7f0ef641c700 1 --2- 172.21.15.82:0/3546580553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ef0106770 0x7f0ef0106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:59.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.943+0000 7f0ef641c700 1 --2- 172.21.15.82:0/3546580553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ef0068120 0x7f0ef00685a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:59.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.943+0000 7f0ef641c700 1 -- 172.21.15.82:0/3546580553 >> 172.21.15.82:0/3546580553 conn(0x7f0ef0076600 msgr2=0x7f0ef0076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:12:59.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.943+0000 7f0ef641c700 1 -- 172.21.15.82:0/3546580553 shutdown_connections 2023-09-09T23:12:59.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.943+0000 7f0ef641c700 1 -- 172.21.15.82:0/3546580553 wait complete. 2023-09-09T23:12:59.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.943+0000 7f0ef641c700 1 Processor -- start 2023-09-09T23:12:59.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.943+0000 7f0ef641c700 1 -- start start 2023-09-09T23:12:59.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.944+0000 7f0ef641c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ef0068120 0x7f0ef0196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:59.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.944+0000 7f0ef641c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ef0068ae0 0x7f0ef0196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:59.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.944+0000 7f0ef641c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ef0106770 0x7f0ef019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:59.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.944+0000 7f0ef641c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef019a610 con 0x7f0ef0068ae0 2023-09-09T23:12:59.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.944+0000 7f0ef641c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef019a780 con 0x7f0ef0106770 2023-09-09T23:12:59.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.944+0000 7f0ef641c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef019a940 con 0x7f0ef0068120 2023-09-09T23:12:59.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.944+0000 7f0eef7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ef0068ae0 0x7f0ef0196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:59.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.944+0000 7f0eef7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ef0068ae0 0x7f0ef0196c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54992/0 (socket says 172.21.15.82:54992) 2023-09-09T23:12:59.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.944+0000 7f0eef7fe700 1 -- 172.21.15.82:0/1896725627 learned_addr learned my addr 172.21.15.82:0/1896725627 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:12:59.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.944+0000 7f0ef49b9700 1 --2- 172.21.15.82:0/1896725627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ef0106770 0x7f0ef019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:59.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.944+0000 7f0eeffff700 1 --2- 172.21.15.82:0/1896725627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ef0068120 0x7f0ef0196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:59.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.944+0000 7f0eef7fe700 1 -- 172.21.15.82:0/1896725627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ef0068120 msgr2=0x7f0ef0196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:59.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.944+0000 7f0eef7fe700 1 --2- 172.21.15.82:0/1896725627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ef0068120 0x7f0ef0196710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:59.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.944+0000 7f0eef7fe700 1 -- 172.21.15.82:0/1896725627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ef0106770 msgr2=0x7f0ef019a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:12:59.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.944+0000 7f0eef7fe700 1 --2- 172.21.15.82:0/1896725627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ef0106770 0x7f0ef019a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:12:59.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.944+0000 7f0eef7fe700 1 -- 172.21.15.82:0/1896725627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee40095f0 con 0x7f0ef0068ae0 2023-09-09T23:12:59.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.944+0000 7f0ef49b9700 1 --2- 172.21.15.82:0/1896725627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ef0106770 0x7f0ef019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:12:59.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.944+0000 7f0eef7fe700 1 --2- 172.21.15.82:0/1896725627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ef0068ae0 0x7f0ef0196c50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4006010 tx=0x7f0ee400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:59.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.945+0000 7f0eed7fa700 1 -- 172.21.15.82:0/1896725627 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee400fbf0 con 0x7f0ef0068ae0 2023-09-09T23:12:59.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.945+0000 7f0eed7fa700 1 -- 172.21.15.82:0/1896725627 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ee4023070 con 0x7f0ef0068ae0 2023-09-09T23:12:59.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.945+0000 7f0ef641c700 1 -- 172.21.15.82:0/1896725627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef019abc0 con 0x7f0ef0068ae0 2023-09-09T23:12:59.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.945+0000 7f0eed7fa700 1 -- 172.21.15.82:0/1896725627 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4017650 con 0x7f0ef0068ae0 2023-09-09T23:12:59.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.945+0000 7f0ef641c700 1 -- 172.21.15.82:0/1896725627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef00ff6b0 con 0x7f0ef0068ae0 2023-09-09T23:12:59.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.946+0000 7f0ef641c700 1 -- 172.21.15.82:0/1896725627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ef019ae60 con 0x7f0ef0068ae0 2023-09-09T23:12:59.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.947+0000 7f0eed7fa700 1 -- 172.21.15.82:0/1896725627 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f0ee4025030 con 0x7f0ef0068ae0 2023-09-09T23:12:59.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.948+0000 7f0eed7fa700 1 --2- 172.21.15.82:0/1896725627 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0ed80a06d0 0x7f0ed80a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:12:59.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.948+0000 7f0eeffff700 1 --2- 172.21.15.82:0/1896725627 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0ed80a06d0 0x7f0ed80a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:12:59.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.948+0000 7f0eed7fa700 1 -- 172.21.15.82:0/1896725627 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0ee40d0fb0 con 0x7f0ef0068ae0 2023-09-09T23:12:59.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.949+0000 7f0eeffff700 1 --2- 172.21.15.82:0/1896725627 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0ed80a06d0 0x7f0ed80a2b90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0005fd0 tx=0x7f0ee000af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:12:59.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:12:59.953+0000 7f0eed7fa700 1 -- 172.21.15.82:0/1896725627 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0ee409acd0 con 0x7f0ef0068ae0 2023-09-09T23:13:00.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.022+0000 7fcdebc0f700 1 -- 172.21.15.82:0/2933229283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fcde4196b50 con 0x7fcde40ff520 2023-09-09T23:13:00.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.023+0000 7fcddaffd700 1 -- 172.21.15.82:0/2933229283 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fcde00987f0 con 0x7fcde40ff520 2023-09-09T23:13:00.021 INFO:teuthology.orchestra.run.smithi082.stdout:326417514515 2023-09-09T23:13:00.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.025+0000 7fcdebc0f700 1 -- 172.21.15.82:0/2933229283 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcdd00a0750 msgr2=0x7fcdd00a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:00.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.025+0000 7fcdebc0f700 1 --2- 172.21.15.82:0/2933229283 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcdd00a0750 0x7fcdd00a2c10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcde4076ea0 tx=0x7fcdd4009190 comp rx=0 tx=0).stop 2023-09-09T23:13:00.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.025+0000 7fcdebc0f700 1 -- 172.21.15.82:0/2933229283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde40ff520 msgr2=0x7fcde419b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:00.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.025+0000 7fcdebc0f700 1 --2- 172.21.15.82:0/2933229283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde40ff520 0x7fcde419b5c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcde000c970 tx=0x7fcde000cc80 comp rx=0 tx=0).stop 2023-09-09T23:13:00.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.026+0000 7fcdebc0f700 1 -- 172.21.15.82:0/2933229283 shutdown_connections 2023-09-09T23:13:00.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.026+0000 7fcdebc0f700 1 --2- 172.21.15.82:0/2933229283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcde4104530 0x7fcde4195640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:00.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.026+0000 7fcdebc0f700 1 --2- 172.21.15.82:0/2933229283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde40ff520 0x7fcde419b5c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:00.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.026+0000 7fcdebc0f700 1 --2- 172.21.15.82:0/2933229283 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcdd00a0750 0x7fcdd00a2c10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:00.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.026+0000 7fcdebc0f700 1 --2- 172.21.15.82:0/2933229283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcde40ffee0 0x7fcde419bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:00.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.026+0000 7fcdebc0f700 1 -- 172.21.15.82:0/2933229283 >> 172.21.15.82:0/2933229283 conn(0x7fcde4076600 msgr2=0x7fcde40fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:00.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.026+0000 7fcdebc0f700 1 -- 172.21.15.82:0/2933229283 shutdown_connections 2023-09-09T23:13:00.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.026+0000 7fcdebc0f700 1 -- 172.21.15.82:0/2933229283 wait complete. 2023-09-09T23:13:00.167 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.169+0000 7f0ef641c700 1 -- 172.21.15.82:0/1896725627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0ef0197b10 con 0x7f0ef0068ae0 2023-09-09T23:13:00.167 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.169+0000 7f0eed7fa700 1 -- 172.21.15.82:0/1896725627 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0ee409e2f0 con 0x7f0ef0068ae0 2023-09-09T23:13:00.167 INFO:teuthology.orchestra.run.smithi082.stdout:107374182452 2023-09-09T23:13:00.169 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.171+0000 7f0ef641c700 1 -- 172.21.15.82:0/1896725627 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0ed80a06d0 msgr2=0x7f0ed80a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:00.169 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.171+0000 7f0ef641c700 1 --2- 172.21.15.82:0/1896725627 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0ed80a06d0 0x7f0ed80a2b90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0005fd0 tx=0x7f0ee000af20 comp rx=0 tx=0).stop 2023-09-09T23:13:00.169 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.171+0000 7f0ef641c700 1 -- 172.21.15.82:0/1896725627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ef0068ae0 msgr2=0x7f0ef0196c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:00.169 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.171+0000 7f0ef641c700 1 --2- 172.21.15.82:0/1896725627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ef0068ae0 0x7f0ef0196c50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4006010 tx=0x7f0ee400f690 comp rx=0 tx=0).stop 2023-09-09T23:13:00.169 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.171+0000 7f0ef641c700 1 -- 172.21.15.82:0/1896725627 shutdown_connections 2023-09-09T23:13:00.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.171+0000 7f0ef641c700 1 --2- 172.21.15.82:0/1896725627 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0ed80a06d0 0x7f0ed80a2b90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:00.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.171+0000 7f0ef641c700 1 --2- 172.21.15.82:0/1896725627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ef0106770 0x7f0ef019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:00.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.171+0000 7f0ef641c700 1 --2- 172.21.15.82:0/1896725627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ef0068120 0x7f0ef0196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:00.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.171+0000 7f0ef641c700 1 --2- 172.21.15.82:0/1896725627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ef0068ae0 0x7f0ef0196c50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:00.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.171+0000 7f0ef641c700 1 -- 172.21.15.82:0/1896725627 >> 172.21.15.82:0/1896725627 conn(0x7f0ef0076600 msgr2=0x7f0ef00fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:00.171 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.171+0000 7f0ef641c700 1 -- 172.21.15.82:0/1896725627 shutdown_connections 2023-09-09T23:13:00.171 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.171+0000 7f0ef641c700 1 -- 172.21.15.82:0/1896725627 wait complete. 2023-09-09T23:13:00.339 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743445 got 296352743447 for osd.10 2023-09-09T23:13:00.339 DEBUG:teuthology.parallel:result is None 2023-09-09T23:13:00.619 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:00 smithi082 ceph-mon[98694]: pgmap v276: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-09T23:13:00.620 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:00 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2933229283' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-09-09T23:13:00.620 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:00 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1896725627' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-09-09T23:13:00.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:00 smithi170 ceph-mon[102832]: pgmap v276: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-09T23:13:00.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:00 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2933229283' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-09-09T23:13:00.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:00 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1896725627' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-09-09T23:13:00.684 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514512 got 326417514515 for osd.11 2023-09-09T23:13:00.685 DEBUG:teuthology.parallel:result is None 2023-09-09T23:13:00.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:00 smithi098 ceph-mon[103124]: pgmap v276: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-09T23:13:00.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:00 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2933229283' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-09-09T23:13:00.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:00 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1896725627' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-09-09T23:13:00.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.611+0000 7ff7adfe9700 1 -- 172.21.15.82:0/2824367861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a80ff520 msgr2=0x7ff7a80ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:00.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.611+0000 7ff7adfe9700 1 --2- 172.21.15.82:0/2824367861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a80ff520 0x7ff7a80ff9a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff798009990 tx=0x7ff798009ca0 comp rx=0 tx=0).stop 2023-09-09T23:13:00.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.611+0000 7ff7adfe9700 1 -- 172.21.15.82:0/2824367861 shutdown_connections 2023-09-09T23:13:00.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.611+0000 7ff7adfe9700 1 --2- 172.21.15.82:0/2824367861 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7a8104530 0x7ff7a8104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:00.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.611+0000 7ff7adfe9700 1 --2- 172.21.15.82:0/2824367861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7a80ffee0 0x7ff7a810c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:00.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.611+0000 7ff7adfe9700 1 --2- 172.21.15.82:0/2824367861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a80ff520 0x7ff7a80ff9a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:00.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.611+0000 7ff7adfe9700 1 -- 172.21.15.82:0/2824367861 >> 172.21.15.82:0/2824367861 conn(0x7ff7a8076600 msgr2=0x7ff7a8076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:00.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.611+0000 7ff7adfe9700 1 -- 172.21.15.82:0/2824367861 shutdown_connections 2023-09-09T23:13:00.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.611+0000 7ff7adfe9700 1 -- 172.21.15.82:0/2824367861 wait complete. 2023-09-09T23:13:00.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.611+0000 7ff7adfe9700 1 Processor -- start 2023-09-09T23:13:00.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.611+0000 7ff7adfe9700 1 -- start start 2023-09-09T23:13:00.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.612+0000 7ff7adfe9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7a80ff520 0x7ff7a8197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:00.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.612+0000 7ff7adfe9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7a80ffee0 0x7ff7a8197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:00.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.612+0000 7ff7adfe9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a8104530 0x7ff7a81a26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:00.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.612+0000 7ff7adfe9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a8198480 con 0x7ff7a8104530 2023-09-09T23:13:00.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.612+0000 7ff7adfe9700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a81a2bf0 con 0x7ff7a80ffee0 2023-09-09T23:13:00.989 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.612+0000 7ff7adfe9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a81a2d60 con 0x7ff7a80ff520 2023-09-09T23:13:00.989 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.612+0000 7ff7a7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a8104530 0x7ff7a81a26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:00.989 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.612+0000 7ff7a7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a8104530 0x7ff7a81a26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55008/0 (socket says 172.21.15.82:55008) 2023-09-09T23:13:00.989 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.612+0000 7ff7a7fff700 1 -- 172.21.15.82:0/684971190 learned_addr learned my addr 172.21.15.82:0/684971190 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:00.989 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.612+0000 7ff7a6ffd700 1 --2- 172.21.15.82:0/684971190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7a80ffee0 0x7ff7a8197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:00.990 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.612+0000 7ff7a77fe700 1 --2- 172.21.15.82:0/684971190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7a80ff520 0x7ff7a8197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:00.990 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.612+0000 7ff7a7fff700 1 -- 172.21.15.82:0/684971190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7a80ff520 msgr2=0x7ff7a8197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:00.990 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.612+0000 7ff7a7fff700 1 --2- 172.21.15.82:0/684971190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7a80ff520 0x7ff7a8197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:00.990 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.612+0000 7ff7a7fff700 1 -- 172.21.15.82:0/684971190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7a80ffee0 msgr2=0x7ff7a8197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:00.990 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.612+0000 7ff7a7fff700 1 --2- 172.21.15.82:0/684971190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7a80ffee0 0x7ff7a8197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:00.991 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.612+0000 7ff7a7fff700 1 -- 172.21.15.82:0/684971190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7980095f0 con 0x7ff7a8104530 2023-09-09T23:13:00.991 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.612+0000 7ff7a7fff700 1 --2- 172.21.15.82:0/684971190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a8104530 0x7ff7a81a26b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000b9e0 tx=0x7ff7a000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:00.991 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.612+0000 7ff7a4ff9700 1 -- 172.21.15.82:0/684971190 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a000c950 con 0x7ff7a8104530 2023-09-09T23:13:00.991 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.612+0000 7ff7adfe9700 1 -- 172.21.15.82:0/684971190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a81a3040 con 0x7ff7a8104530 2023-09-09T23:13:00.991 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.612+0000 7ff7a4ff9700 1 -- 172.21.15.82:0/684971190 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7a0014440 con 0x7ff7a8104530 2023-09-09T23:13:00.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.612+0000 7ff7a4ff9700 1 -- 172.21.15.82:0/684971190 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a00075d0 con 0x7ff7a8104530 2023-09-09T23:13:00.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.612+0000 7ff7adfe9700 1 -- 172.21.15.82:0/684971190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a81a3510 con 0x7ff7a8104530 2023-09-09T23:13:00.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.613+0000 7ff7adfe9700 1 -- 172.21.15.82:0/684971190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a804e300 con 0x7ff7a8104530 2023-09-09T23:13:00.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.615+0000 7ff7a4ff9700 1 -- 172.21.15.82:0/684971190 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7ff7a00145b0 con 0x7ff7a8104530 2023-09-09T23:13:00.993 INFO:teuthology.orchestra.run.smithi082.stdout:137438953521 2023-09-09T23:13:00.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.615+0000 7ff7a4ff9700 1 --2- 172.21.15.82:0/684971190 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff7900a06d0 0x7ff7900a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:00.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.615+0000 7ff7a77fe700 1 --2- 172.21.15.82:0/684971190 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff7900a06d0 0x7ff7900a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:00.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.616+0000 7ff7a4ff9700 1 -- 172.21.15.82:0/684971190 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff7a0006e40 con 0x7ff7a8104530 2023-09-09T23:13:00.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.616+0000 7ff7a77fe700 1 --2- 172.21.15.82:0/684971190 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff7900a06d0 0x7ff7900a2b90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff79c006fd0 tx=0x7ff79c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:00.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.619+0000 7ff7a4ff9700 1 -- 172.21.15.82:0/684971190 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff7a0095210 con 0x7ff7a8104530 2023-09-09T23:13:00.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.824+0000 7ff7adfe9700 1 -- 172.21.15.82:0/684971190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7ff7a810f5b0 con 0x7ff7a8104530 2023-09-09T23:13:00.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.824+0000 7ff7a4ff9700 1 -- 172.21.15.82:0/684971190 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff7a0023090 con 0x7ff7a8104530 2023-09-09T23:13:00.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.826+0000 7ff7adfe9700 1 -- 172.21.15.82:0/684971190 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff7900a06d0 msgr2=0x7ff7900a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:00.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.826+0000 7ff7adfe9700 1 --2- 172.21.15.82:0/684971190 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff7900a06d0 0x7ff7900a2b90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff79c006fd0 tx=0x7ff79c008040 comp rx=0 tx=0).stop 2023-09-09T23:13:00.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.826+0000 7ff7adfe9700 1 -- 172.21.15.82:0/684971190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a8104530 msgr2=0x7ff7a81a26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:00.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.826+0000 7ff7adfe9700 1 --2- 172.21.15.82:0/684971190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a8104530 0x7ff7a81a26b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000b9e0 tx=0x7ff7a000bda0 comp rx=0 tx=0).stop 2023-09-09T23:13:00.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.826+0000 7ff7adfe9700 1 -- 172.21.15.82:0/684971190 shutdown_connections 2023-09-09T23:13:00.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.826+0000 7ff7adfe9700 1 --2- 172.21.15.82:0/684971190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a8104530 0x7ff7a81a26b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:00.996 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.826+0000 7ff7adfe9700 1 --2- 172.21.15.82:0/684971190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7a80ff520 0x7ff7a8197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:00.996 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.826+0000 7ff7adfe9700 1 --2- 172.21.15.82:0/684971190 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff7900a06d0 0x7ff7900a2b90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:00.996 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.826+0000 7ff7adfe9700 1 --2- 172.21.15.82:0/684971190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7a80ffee0 0x7ff7a8197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:00.996 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.826+0000 7ff7adfe9700 1 -- 172.21.15.82:0/684971190 >> 172.21.15.82:0/684971190 conn(0x7ff7a8076600 msgr2=0x7ff7a80fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:00.996 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.826+0000 7ff7adfe9700 1 -- 172.21.15.82:0/684971190 shutdown_connections 2023-09-09T23:13:00.997 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:00.826+0000 7ff7adfe9700 1 -- 172.21.15.82:0/684971190 wait complete. 2023-09-09T23:13:00.997 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182450 got 107374182452 for osd.3 2023-09-09T23:13:00.998 DEBUG:teuthology.parallel:result is None 2023-09-09T23:13:01.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.036+0000 7fb887978700 1 -- 172.21.15.82:0/3756888584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8801005c0 msgr2=0x7fb88010c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:01.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.036+0000 7fb887978700 1 --2- 172.21.15.82:0/3756888584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8801005c0 0x7fb88010c3a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb87800b0a0 tx=0x7fb87800b3b0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.036+0000 7fb887978700 1 -- 172.21.15.82:0/3756888584 shutdown_connections 2023-09-09T23:13:01.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.036+0000 7fb887978700 1 --2- 172.21.15.82:0/3756888584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb880105540 0x7fb880105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.036+0000 7fb887978700 1 --2- 172.21.15.82:0/3756888584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8801005c0 0x7fb88010c3a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.036+0000 7fb887978700 1 --2- 172.21.15.82:0/3756888584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8800ffc00 0x7fb880100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.036+0000 7fb887978700 1 -- 172.21.15.82:0/3756888584 >> 172.21.15.82:0/3756888584 conn(0x7fb880075a70 msgr2=0x7fb8800fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:01.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.036+0000 7fb887978700 1 -- 172.21.15.82:0/3756888584 shutdown_connections 2023-09-09T23:13:01.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.036+0000 7fb887978700 1 -- 172.21.15.82:0/3756888584 wait complete. 2023-09-09T23:13:01.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.036+0000 7fb887978700 1 Processor -- start 2023-09-09T23:13:01.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.036+0000 7fb887978700 1 -- start start 2023-09-09T23:13:01.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.037+0000 7fb887978700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8800ffc00 0x7fb880197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:01.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.037+0000 7fb887978700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8801005c0 0x7fb880197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:01.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.037+0000 7fb887978700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb880105540 0x7fb8801a26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:01.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.037+0000 7fb887978700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8801984e0 con 0x7fb880105540 2023-09-09T23:13:01.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.037+0000 7fb887978700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8801a2c20 con 0x7fb8800ffc00 2023-09-09T23:13:01.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.037+0000 7fb887978700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8801a2de0 con 0x7fb8801005c0 2023-09-09T23:13:01.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.037+0000 7fb885f15700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb880105540 0x7fb8801a26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:01.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.037+0000 7fb885f15700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb880105540 0x7fb8801a26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55022/0 (socket says 172.21.15.82:55022) 2023-09-09T23:13:01.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.037+0000 7fb885f15700 1 -- 172.21.15.82:0/2216047001 learned_addr learned my addr 172.21.15.82:0/2216047001 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:01.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.037+0000 7fb884f13700 1 --2- 172.21.15.82:0/2216047001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8801005c0 0x7fb880197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:01.041 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.037+0000 7fb885714700 1 --2- 172.21.15.82:0/2216047001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8800ffc00 0x7fb880197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:01.041 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.037+0000 7fb885f15700 1 -- 172.21.15.82:0/2216047001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8801005c0 msgr2=0x7fb880197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:01.041 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.037+0000 7fb885f15700 1 --2- 172.21.15.82:0/2216047001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8801005c0 0x7fb880197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.041 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.037+0000 7fb885f15700 1 -- 172.21.15.82:0/2216047001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8800ffc00 msgr2=0x7fb880197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:01.041 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.037+0000 7fb885f15700 1 --2- 172.21.15.82:0/2216047001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8800ffc00 0x7fb880197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.042 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.037+0000 7fb885f15700 1 -- 172.21.15.82:0/2216047001 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb878009d00 con 0x7fb880105540 2023-09-09T23:13:01.042 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.037+0000 7fb885f15700 1 --2- 172.21.15.82:0/2216047001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb880105540 0x7fb8801a26e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb8780062a0 tx=0x7fb87800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:01.042 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.037+0000 7fb8767fc700 1 -- 172.21.15.82:0/2216047001 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87800e040 con 0x7fb880105540 2023-09-09T23:13:01.042 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.037+0000 7fb8767fc700 1 -- 172.21.15.82:0/2216047001 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb878004020 con 0x7fb880105540 2023-09-09T23:13:01.042 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.037+0000 7fb887978700 1 -- 172.21.15.82:0/2216047001 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8801a3060 con 0x7fb880105540 2023-09-09T23:13:01.043 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.038+0000 7fb8767fc700 1 -- 172.21.15.82:0/2216047001 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87801b690 con 0x7fb880105540 2023-09-09T23:13:01.043 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.038+0000 7fb887978700 1 -- 172.21.15.82:0/2216047001 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8801a3560 con 0x7fb880105540 2023-09-09T23:13:01.043 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.038+0000 7fb887978700 1 -- 172.21.15.82:0/2216047001 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb88004e300 con 0x7fb880105540 2023-09-09T23:13:01.043 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.040+0000 7fb8767fc700 1 -- 172.21.15.82:0/2216047001 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fb878004400 con 0x7fb880105540 2023-09-09T23:13:01.044 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.040+0000 7fb8767fc700 1 --2- 172.21.15.82:0/2216047001 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb86c0a06d0 0x7fb86c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:01.044 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.040+0000 7fb885714700 1 --2- 172.21.15.82:0/2216047001 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb86c0a06d0 0x7fb86c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:01.044 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.040+0000 7fb8767fc700 1 -- 172.21.15.82:0/2216047001 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb8780cd850 con 0x7fb880105540 2023-09-09T23:13:01.044 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.040+0000 7fb885714700 1 --2- 172.21.15.82:0/2216047001 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb86c0a06d0 0x7fb86c0a2b90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb870005ea0 tx=0x7fb870005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:01.045 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.043+0000 7fb8767fc700 1 -- 172.21.15.82:0/2216047001 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb878097460 con 0x7fb880105540 2023-09-09T23:13:01.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.240+0000 7fb887978700 1 -- 172.21.15.82:0/2216047001 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb88010f6f0 con 0x7fb880105540 2023-09-09T23:13:01.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.240+0000 7fb8767fc700 1 -- 172.21.15.82:0/2216047001 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb87809aa80 con 0x7fb880105540 2023-09-09T23:13:01.238 INFO:teuthology.orchestra.run.smithi082.stdout:55834574909 2023-09-09T23:13:01.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.242+0000 7fb887978700 1 -- 172.21.15.82:0/2216047001 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb86c0a06d0 msgr2=0x7fb86c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:01.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.242+0000 7fb887978700 1 --2- 172.21.15.82:0/2216047001 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb86c0a06d0 0x7fb86c0a2b90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb870005ea0 tx=0x7fb870005e30 comp rx=0 tx=0).stop 2023-09-09T23:13:01.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.242+0000 7fb887978700 1 -- 172.21.15.82:0/2216047001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb880105540 msgr2=0x7fb8801a26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:01.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.242+0000 7fb887978700 1 --2- 172.21.15.82:0/2216047001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb880105540 0x7fb8801a26e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb8780062a0 tx=0x7fb87800cf70 comp rx=0 tx=0).stop 2023-09-09T23:13:01.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.243+0000 7fb887978700 1 -- 172.21.15.82:0/2216047001 shutdown_connections 2023-09-09T23:13:01.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.243+0000 7fb887978700 1 --2- 172.21.15.82:0/2216047001 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb86c0a06d0 0x7fb86c0a2b90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.243+0000 7fb887978700 1 --2- 172.21.15.82:0/2216047001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8800ffc00 0x7fb880197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.243+0000 7fb887978700 1 --2- 172.21.15.82:0/2216047001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb880105540 0x7fb8801a26e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.243+0000 7fb887978700 1 --2- 172.21.15.82:0/2216047001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8801005c0 0x7fb880197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.243+0000 7fb887978700 1 -- 172.21.15.82:0/2216047001 >> 172.21.15.82:0/2216047001 conn(0x7fb880075a70 msgr2=0x7fb8800fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:01.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.243+0000 7fb887978700 1 -- 172.21.15.82:0/2216047001 shutdown_connections 2023-09-09T23:13:01.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.243+0000 7fb887978700 1 -- 172.21.15.82:0/2216047001 wait complete. 2023-09-09T23:13:01.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.334+0000 7f24630b2700 1 -- 172.21.15.82:0/4234003281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f245c104530 msgr2=0x7f245c104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:01.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.334+0000 7f24630b2700 1 --2- 172.21.15.82:0/4234003281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f245c104530 0x7f245c104910 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2450009990 tx=0x7f2450009ca0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.334+0000 7f24630b2700 1 -- 172.21.15.82:0/4234003281 shutdown_connections 2023-09-09T23:13:01.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.334+0000 7f24630b2700 1 --2- 172.21.15.82:0/4234003281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f245c0ffee0 0x7f245c10c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.334+0000 7f24630b2700 1 --2- 172.21.15.82:0/4234003281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f245c104530 0x7f245c104910 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.334+0000 7f24630b2700 1 --2- 172.21.15.82:0/4234003281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f245c0ff520 0x7f245c0ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.334+0000 7f24630b2700 1 -- 172.21.15.82:0/4234003281 >> 172.21.15.82:0/4234003281 conn(0x7f245c076600 msgr2=0x7f245c076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:01.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.334+0000 7f24630b2700 1 -- 172.21.15.82:0/4234003281 shutdown_connections 2023-09-09T23:13:01.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.334+0000 7f24630b2700 1 -- 172.21.15.82:0/4234003281 wait complete. 2023-09-09T23:13:01.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.334+0000 7f24630b2700 1 Processor -- start 2023-09-09T23:13:01.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.334+0000 7f24630b2700 1 -- start start 2023-09-09T23:13:01.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.335+0000 7f24630b2700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f245c0ff520 0x7f245c1975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:01.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.335+0000 7f24630b2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f245c0ffee0 0x7f245c197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:01.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.335+0000 7f24630b2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f245c104530 0x7f245c1a2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:01.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.335+0000 7f24630b2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245c1a2b90 con 0x7f245c0ffee0 2023-09-09T23:13:01.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.335+0000 7f24630b2700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245c1a2cd0 con 0x7f245c0ff520 2023-09-09T23:13:01.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.335+0000 7f24630b2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245c1a2e10 con 0x7f245c104530 2023-09-09T23:13:01.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.335+0000 7f245bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f245c0ffee0 0x7f245c197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:01.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.335+0000 7f245bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f245c0ffee0 0x7f245c197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55034/0 (socket says 172.21.15.82:55034) 2023-09-09T23:13:01.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.335+0000 7f245bfff700 1 -- 172.21.15.82:0/3906752901 learned_addr learned my addr 172.21.15.82:0/3906752901 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:01.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.335+0000 7f246164f700 1 --2- 172.21.15.82:0/3906752901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f245c104530 0x7f245c1a2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:01.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.335+0000 7f2460e4e700 1 --2- 172.21.15.82:0/3906752901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f245c0ff520 0x7f245c1975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:01.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.335+0000 7f245bfff700 1 -- 172.21.15.82:0/3906752901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f245c104530 msgr2=0x7f245c1a2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:01.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.335+0000 7f245bfff700 1 --2- 172.21.15.82:0/3906752901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f245c104530 0x7f245c1a2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.335+0000 7f245bfff700 1 -- 172.21.15.82:0/3906752901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f245c0ff520 msgr2=0x7f245c1975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:01.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.335+0000 7f245bfff700 1 --2- 172.21.15.82:0/3906752901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f245c0ff520 0x7f245c1975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.335+0000 7f245bfff700 1 -- 172.21.15.82:0/3906752901 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24500095f0 con 0x7f245c0ffee0 2023-09-09T23:13:01.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.335+0000 7f2460e4e700 1 --2- 172.21.15.82:0/3906752901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f245c0ff520 0x7f245c1975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:01.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.335+0000 7f245bfff700 1 --2- 172.21.15.82:0/3906752901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f245c0ffee0 0x7f245c197b10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f244800c920 tx=0x7f244800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:01.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.335+0000 7f2459ffb700 1 -- 172.21.15.82:0/3906752901 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2448007b00 con 0x7f245c0ffee0 2023-09-09T23:13:01.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.336+0000 7f2459ffb700 1 -- 172.21.15.82:0/3906752901 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2448004cf0 con 0x7f245c0ffee0 2023-09-09T23:13:01.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.336+0000 7f24630b2700 1 -- 172.21.15.82:0/3906752901 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f245c1a3070 con 0x7f245c0ffee0 2023-09-09T23:13:01.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.336+0000 7f2459ffb700 1 -- 172.21.15.82:0/3906752901 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2448005610 con 0x7f245c0ffee0 2023-09-09T23:13:01.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.336+0000 7f24630b2700 1 -- 172.21.15.82:0/3906752901 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f245c1a3540 con 0x7f245c0ffee0 2023-09-09T23:13:01.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.337+0000 7f24630b2700 1 -- 172.21.15.82:0/3906752901 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f245c04e300 con 0x7f245c0ffee0 2023-09-09T23:13:01.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.339+0000 7f2459ffb700 1 -- 172.21.15.82:0/3906752901 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f2448004750 con 0x7f245c0ffee0 2023-09-09T23:13:01.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.340+0000 7f2459ffb700 1 --2- 172.21.15.82:0/3906752901 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f244c0a0750 0x7f244c0a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:01.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.340+0000 7f2460e4e700 1 --2- 172.21.15.82:0/3906752901 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f244c0a0750 0x7f244c0a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:01.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.340+0000 7f2459ffb700 1 -- 172.21.15.82:0/3906752901 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f24480cbc60 con 0x7f245c0ffee0 2023-09-09T23:13:01.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.340+0000 7f2460e4e700 1 --2- 172.21.15.82:0/3906752901 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f244c0a0750 0x7f244c0a2c10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f24500095c0 tx=0x7f2450009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:01.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.344+0000 7f2459ffb700 1 -- 172.21.15.82:0/3906752901 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f24480956e0 con 0x7f245c0ffee0 2023-09-09T23:13:01.468 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.469+0000 7f7098301700 1 -- 172.21.15.82:0/4107917984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7090068120 msgr2=0x7f70900685a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:01.468 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.469+0000 7f7098301700 1 --2- 172.21.15.82:0/4107917984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7090068120 0x7f70900685a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7080009990 tx=0x7f7080009ca0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.468 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.469+0000 7f7098301700 1 -- 172.21.15.82:0/4107917984 shutdown_connections 2023-09-09T23:13:01.468 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.469+0000 7f7098301700 1 --2- 172.21.15.82:0/4107917984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7090106770 0x7f7090106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.469+0000 7f7098301700 1 --2- 172.21.15.82:0/4107917984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7090068ae0 0x7f709010d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.469+0000 7f7098301700 1 --2- 172.21.15.82:0/4107917984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7090068120 0x7f70900685a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.469+0000 7f7098301700 1 -- 172.21.15.82:0/4107917984 >> 172.21.15.82:0/4107917984 conn(0x7f7090076600 msgr2=0x7f7090076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.469+0000 7f7098301700 1 -- 172.21.15.82:0/4107917984 shutdown_connections 2023-09-09T23:13:01.469 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.469+0000 7f7098301700 1 -- 172.21.15.82:0/4107917984 wait complete. 2023-09-09T23:13:01.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.469+0000 7f7098301700 1 Processor -- start 2023-09-09T23:13:01.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.469+0000 7f7098301700 1 -- start start 2023-09-09T23:13:01.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.470+0000 7f7098301700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7090068120 0x7f709019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:01.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.470+0000 7f7098301700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7090068ae0 0x7f709019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:01.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.470+0000 7f7098301700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7090106770 0x7f7090196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:01.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.470+0000 7f7098301700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7090196bf0 con 0x7f7090068120 2023-09-09T23:13:01.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.470+0000 7f7098301700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7090196d60 con 0x7f7090106770 2023-09-09T23:13:01.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.470+0000 7f7098301700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7090196f20 con 0x7f7090068ae0 2023-09-09T23:13:01.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.470+0000 7f709609d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7090068120 0x7f709019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:01.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.470+0000 7f709609d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7090068120 0x7f709019c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55046/0 (socket says 172.21.15.82:55046) 2023-09-09T23:13:01.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.470+0000 7f709609d700 1 -- 172.21.15.82:0/3927039058 learned_addr learned my addr 172.21.15.82:0/3927039058 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:01.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.470+0000 7f709689e700 1 --2- 172.21.15.82:0/3927039058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7090106770 0x7f7090196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:01.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.470+0000 7f709589c700 1 --2- 172.21.15.82:0/3927039058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7090068ae0 0x7f709019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:01.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.470+0000 7f709609d700 1 -- 172.21.15.82:0/3927039058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7090068ae0 msgr2=0x7f709019ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:01.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.470+0000 7f709609d700 1 --2- 172.21.15.82:0/3927039058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7090068ae0 0x7f709019ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.470+0000 7f709609d700 1 -- 172.21.15.82:0/3927039058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7090106770 msgr2=0x7f7090196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:01.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.470+0000 7f709609d700 1 --2- 172.21.15.82:0/3927039058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7090106770 0x7f7090196580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.470+0000 7f709609d700 1 -- 172.21.15.82:0/3927039058 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70800095f0 con 0x7f7090068120 2023-09-09T23:13:01.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.470+0000 7f709609d700 1 --2- 172.21.15.82:0/3927039058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7090068120 0x7f709019c500 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f708c00d950 tx=0x7f708c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:01.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.470+0000 7f70877fe700 1 -- 172.21.15.82:0/3927039058 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c00f890 con 0x7f7090068120 2023-09-09T23:13:01.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.470+0000 7f70877fe700 1 -- 172.21.15.82:0/3927039058 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f708c010460 con 0x7f7090068120 2023-09-09T23:13:01.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.471+0000 7f7098301700 1 -- 172.21.15.82:0/3927039058 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7090197200 con 0x7f7090068120 2023-09-09T23:13:01.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.471+0000 7f70877fe700 1 -- 172.21.15.82:0/3927039058 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c00e610 con 0x7f7090068120 2023-09-09T23:13:01.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.471+0000 7f7098301700 1 -- 172.21.15.82:0/3927039058 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70901a34f0 con 0x7f7090068120 2023-09-09T23:13:01.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.471+0000 7f7098301700 1 -- 172.21.15.82:0/3927039058 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f709004e300 con 0x7f7090068120 2023-09-09T23:13:01.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.473+0000 7f70877fe700 1 -- 172.21.15.82:0/3927039058 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f708c0108d0 con 0x7f7090068120 2023-09-09T23:13:01.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.473+0000 7f70877fe700 1 --2- 172.21.15.82:0/3927039058 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f707c0a0750 0x7f707c0a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:01.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.473+0000 7f709589c700 1 --2- 172.21.15.82:0/3927039058 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f707c0a0750 0x7f707c0a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:01.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.473+0000 7f70877fe700 1 -- 172.21.15.82:0/3927039058 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f708c0cb8f0 con 0x7f7090068120 2023-09-09T23:13:01.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.474+0000 7f709589c700 1 --2- 172.21.15.82:0/3927039058 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f707c0a0750 0x7f707c0a2c10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f70900fdd20 tx=0x7f7080005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:01.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.481+0000 7f70877fe700 1 -- 172.21.15.82:0/3927039058 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f708c095470 con 0x7f7090068120 2023-09-09T23:13:01.545 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.548+0000 7f24630b2700 1 -- 172.21.15.82:0/3906752901 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f245c068710 con 0x7f245c0ffee0 2023-09-09T23:13:01.546 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.548+0000 7f2459ffb700 1 -- 172.21.15.82:0/3906752901 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f244801d020 con 0x7f245c0ffee0 2023-09-09T23:13:01.546 INFO:teuthology.orchestra.run.smithi082.stdout:244813135904 2023-09-09T23:13:01.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.550+0000 7f24630b2700 1 -- 172.21.15.82:0/3906752901 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f244c0a0750 msgr2=0x7f244c0a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:01.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.550+0000 7f24630b2700 1 --2- 172.21.15.82:0/3906752901 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f244c0a0750 0x7f244c0a2c10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f24500095c0 tx=0x7f2450009510 comp rx=0 tx=0).stop 2023-09-09T23:13:01.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.550+0000 7f24630b2700 1 -- 172.21.15.82:0/3906752901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f245c0ffee0 msgr2=0x7f245c197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:01.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.550+0000 7f24630b2700 1 --2- 172.21.15.82:0/3906752901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f245c0ffee0 0x7f245c197b10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f244800c920 tx=0x7f244800cce0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.550+0000 7f24630b2700 1 -- 172.21.15.82:0/3906752901 shutdown_connections 2023-09-09T23:13:01.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.550+0000 7f24630b2700 1 --2- 172.21.15.82:0/3906752901 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f244c0a0750 0x7f244c0a2c10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.550+0000 7f24630b2700 1 --2- 172.21.15.82:0/3906752901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f245c0ff520 0x7f245c1975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.550+0000 7f24630b2700 1 --2- 172.21.15.82:0/3906752901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f245c104530 0x7f245c1a2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.550+0000 7f24630b2700 1 --2- 172.21.15.82:0/3906752901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f245c0ffee0 0x7f245c197b10 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.550+0000 7f24630b2700 1 -- 172.21.15.82:0/3906752901 >> 172.21.15.82:0/3906752901 conn(0x7f245c076600 msgr2=0x7f245c0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:01.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.551+0000 7f24630b2700 1 -- 172.21.15.82:0/3906752901 shutdown_connections 2023-09-09T23:13:01.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.551+0000 7f24630b2700 1 -- 172.21.15.82:0/3906752901 wait complete. 2023-09-09T23:13:01.570 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:01 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/684971190' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-09-09T23:13:01.570 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:01 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2216047001' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-09-09T23:13:01.634 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953518 got 137438953521 for osd.4 2023-09-09T23:13:01.634 DEBUG:teuthology.parallel:result is None 2023-09-09T23:13:01.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:01 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/684971190' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-09-09T23:13:01.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:01 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2216047001' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-09-09T23:13:01.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.694+0000 7f7098301700 1 -- 172.21.15.82:0/3927039058 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f7090197a90 con 0x7f7090068120 2023-09-09T23:13:01.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.694+0000 7f70877fe700 1 -- 172.21.15.82:0/3927039058 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f708c020090 con 0x7f7090068120 2023-09-09T23:13:01.692 INFO:teuthology.orchestra.run.smithi082.stdout:214748364836 2023-09-09T23:13:01.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.697+0000 7f7098301700 1 -- 172.21.15.82:0/3927039058 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f707c0a0750 msgr2=0x7f707c0a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:01.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.697+0000 7f7098301700 1 --2- 172.21.15.82:0/3927039058 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f707c0a0750 0x7f707c0a2c10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f70900fdd20 tx=0x7f7080005fd0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.697+0000 7f7098301700 1 -- 172.21.15.82:0/3927039058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7090068120 msgr2=0x7f709019c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:01.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.697+0000 7f7098301700 1 --2- 172.21.15.82:0/3927039058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7090068120 0x7f709019c500 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f708c00d950 tx=0x7f708c00dc60 comp rx=0 tx=0).stop 2023-09-09T23:13:01.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.697+0000 7f7098301700 1 -- 172.21.15.82:0/3927039058 shutdown_connections 2023-09-09T23:13:01.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.697+0000 7f7098301700 1 --2- 172.21.15.82:0/3927039058 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f707c0a0750 0x7f707c0a2c10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.697+0000 7f7098301700 1 --2- 172.21.15.82:0/3927039058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7090106770 0x7f7090196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.696 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.697+0000 7f7098301700 1 --2- 172.21.15.82:0/3927039058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7090068120 0x7f709019c500 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.696 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.697+0000 7f7098301700 1 --2- 172.21.15.82:0/3927039058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7090068ae0 0x7f709019ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:01.696 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.697+0000 7f7098301700 1 -- 172.21.15.82:0/3927039058 >> 172.21.15.82:0/3927039058 conn(0x7f7090076600 msgr2=0x7f70900fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:01.696 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.697+0000 7f7098301700 1 -- 172.21.15.82:0/3927039058 shutdown_connections 2023-09-09T23:13:01.696 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:01.697+0000 7f7098301700 1 -- 172.21.15.82:0/3927039058 wait complete. 2023-09-09T23:13:01.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:01 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/684971190' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-09-09T23:13:01.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:01 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2216047001' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-09-09T23:13:02.236 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135902 got 244813135904 for osd.8 2023-09-09T23:13:02.236 DEBUG:teuthology.parallel:result is None 2023-09-09T23:13:02.269 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364833 got 214748364836 for osd.7 2023-09-09T23:13:02.270 DEBUG:teuthology.parallel:result is None 2023-09-09T23:13:02.307 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574907 got 55834574909 for osd.1 2023-09-09T23:13:02.308 DEBUG:teuthology.parallel:result is None 2023-09-09T23:13:02.308 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-09-09T23:13:02.308 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph pg dump --format=json 2023-09-09T23:13:02.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:02 smithi170 ceph-mon[102832]: pgmap v277: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-09T23:13:02.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:02 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3906752901' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-09-09T23:13:02.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:02 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3927039058' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-09-09T23:13:02.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:02 smithi082 ceph-mon[98694]: pgmap v277: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-09T23:13:02.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:02 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3906752901' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-09-09T23:13:02.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:02 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3927039058' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-09-09T23:13:02.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:02 smithi098 ceph-mon[103124]: pgmap v277: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-09T23:13:02.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:02 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3906752901' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-09-09T23:13:02.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:02 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3927039058' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-09-09T23:13:02.900 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:13:03.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.957+0000 7f5651cad700 1 -- 172.21.15.82:0/1811077563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f564c1066a0 msgr2=0x7f564c106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:03.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.957+0000 7f5651cad700 1 --2- 172.21.15.82:0/1811077563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f564c1066a0 0x7f564c106a80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5640009990 tx=0x7f5640009ca0 comp rx=0 tx=0).stop 2023-09-09T23:13:03.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.957+0000 7f5651cad700 1 -- 172.21.15.82:0/1811077563 shutdown_connections 2023-09-09T23:13:03.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.957+0000 7f5651cad700 1 --2- 172.21.15.82:0/1811077563 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f564c100fd0 0x7f564c10d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:03.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.957+0000 7f5651cad700 1 --2- 172.21.15.82:0/1811077563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f564c1066a0 0x7f564c106a80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:03.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.957+0000 7f5651cad700 1 --2- 172.21.15.82:0/1811077563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f564c100610 0x7f564c100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:03.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.957+0000 7f5651cad700 1 -- 172.21.15.82:0/1811077563 >> 172.21.15.82:0/1811077563 conn(0x7f564c0fc0a0 msgr2=0x7f564c0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:03.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.957+0000 7f5651cad700 1 -- 172.21.15.82:0/1811077563 shutdown_connections 2023-09-09T23:13:03.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.957+0000 7f5651cad700 1 -- 172.21.15.82:0/1811077563 wait complete. 2023-09-09T23:13:03.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.958+0000 7f5651cad700 1 Processor -- start 2023-09-09T23:13:03.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.958+0000 7f5651cad700 1 -- start start 2023-09-09T23:13:03.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.958+0000 7f5651cad700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f564c100610 0x7f564c196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:03.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.958+0000 7f5651cad700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f564c100fd0 0x7f564c196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:03.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.958+0000 7f5651cad700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f564c1066a0 0x7f564c19b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:03.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.958+0000 7f5651cad700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564c19b620 con 0x7f564c1066a0 2023-09-09T23:13:03.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.958+0000 7f5651cad700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564c19b790 con 0x7f564c100fd0 2023-09-09T23:13:03.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.958+0000 7f5651cad700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564c19b950 con 0x7f564c100610 2023-09-09T23:13:03.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.958+0000 7f564bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f564c1066a0 0x7f564c19b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:03.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.958+0000 7f564bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f564c1066a0 0x7f564c19b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55062/0 (socket says 172.21.15.82:55062) 2023-09-09T23:13:03.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.958+0000 7f564bfff700 1 -- 172.21.15.82:0/1296509154 learned_addr learned my addr 172.21.15.82:0/1296509154 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:03.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.958+0000 7f564b7fe700 1 --2- 172.21.15.82:0/1296509154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f564c100610 0x7f564c196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:03.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.958+0000 7f564affd700 1 --2- 172.21.15.82:0/1296509154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f564c100fd0 0x7f564c196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:03.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.958+0000 7f564bfff700 1 -- 172.21.15.82:0/1296509154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f564c100610 msgr2=0x7f564c196660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:03.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.958+0000 7f564bfff700 1 --2- 172.21.15.82:0/1296509154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f564c100610 0x7f564c196660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:03.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.958+0000 7f564bfff700 1 -- 172.21.15.82:0/1296509154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f564c100fd0 msgr2=0x7f564c196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:03.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.958+0000 7f564bfff700 1 --2- 172.21.15.82:0/1296509154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f564c100fd0 0x7f564c196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:03.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.958+0000 7f564bfff700 1 -- 172.21.15.82:0/1296509154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56400095f0 con 0x7f564c1066a0 2023-09-09T23:13:03.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.958+0000 7f564affd700 1 --2- 172.21.15.82:0/1296509154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f564c100fd0 0x7f564c196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:03.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.959+0000 7f564bfff700 1 --2- 172.21.15.82:0/1296509154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f564c1066a0 0x7f564c19b010 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f564400b970 tx=0x7f564400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:03.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.959+0000 7f5648ff9700 1 -- 172.21.15.82:0/1296509154 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564400c930 con 0x7f564c1066a0 2023-09-09T23:13:03.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.959+0000 7f5648ff9700 1 -- 172.21.15.82:0/1296509154 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5644014440 con 0x7f564c1066a0 2023-09-09T23:13:03.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.959+0000 7f5648ff9700 1 -- 172.21.15.82:0/1296509154 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56440075d0 con 0x7f564c1066a0 2023-09-09T23:13:03.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.959+0000 7f5651cad700 1 -- 172.21.15.82:0/1296509154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f564c19bc30 con 0x7f564c1066a0 2023-09-09T23:13:03.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.959+0000 7f5651cad700 1 -- 172.21.15.82:0/1296509154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f564c1a3490 con 0x7f564c1066a0 2023-09-09T23:13:03.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.960+0000 7f5651cad700 1 -- 172.21.15.82:0/1296509154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f564c04e300 con 0x7f564c1066a0 2023-09-09T23:13:03.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.961+0000 7f5648ff9700 1 -- 172.21.15.82:0/1296509154 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f56440145b0 con 0x7f564c1066a0 2023-09-09T23:13:03.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.961+0000 7f5648ff9700 1 --2- 172.21.15.82:0/1296509154 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f56340a06d0 0x7f56340a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:03.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.962+0000 7f564b7fe700 1 --2- 172.21.15.82:0/1296509154 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f56340a06d0 0x7f56340a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:03.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.962+0000 7f5648ff9700 1 -- 172.21.15.82:0/1296509154 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5644006e40 con 0x7f564c1066a0 2023-09-09T23:13:03.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.962+0000 7f564b7fe700 1 --2- 172.21.15.82:0/1296509154 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f56340a06d0 0x7f56340a2b90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f564000b5c0 tx=0x7f5640005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:03.966 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:03.965+0000 7f5648ff9700 1 -- 172.21.15.82:0/1296509154 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5644095240 con 0x7f564c1066a0 2023-09-09T23:13:04.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:04.151+0000 7f5651cad700 1 -- 172.21.15.82:0/1296509154 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f564c110880 con 0x7f56340a06d0 2023-09-09T23:13:04.158 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:04.161+0000 7f5648ff9700 1 -- 172.21.15.82:0/1296509154 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340377 (secure 0 0 0) 0x7f564c110880 con 0x7f56340a06d0 2023-09-09T23:13:04.162 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:04.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:04.164+0000 7f5651cad700 1 -- 172.21.15.82:0/1296509154 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f56340a06d0 msgr2=0x7f56340a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:04.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:04.164+0000 7f5651cad700 1 --2- 172.21.15.82:0/1296509154 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f56340a06d0 0x7f56340a2b90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f564000b5c0 tx=0x7f5640005fd0 comp rx=0 tx=0).stop 2023-09-09T23:13:04.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:04.164+0000 7f5651cad700 1 -- 172.21.15.82:0/1296509154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f564c1066a0 msgr2=0x7f564c19b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:04.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:04.164+0000 7f5651cad700 1 --2- 172.21.15.82:0/1296509154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f564c1066a0 0x7f564c19b010 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f564400b970 tx=0x7f564400bc80 comp rx=0 tx=0).stop 2023-09-09T23:13:04.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:04.164+0000 7f5651cad700 1 -- 172.21.15.82:0/1296509154 shutdown_connections 2023-09-09T23:13:04.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:04.164+0000 7f5651cad700 1 --2- 172.21.15.82:0/1296509154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f564c1066a0 0x7f564c19b010 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:04.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:04.164+0000 7f5651cad700 1 --2- 172.21.15.82:0/1296509154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f564c100610 0x7f564c196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:04.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:04.164+0000 7f5651cad700 1 --2- 172.21.15.82:0/1296509154 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f56340a06d0 0x7f56340a2b90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:04.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:04.164+0000 7f5651cad700 1 --2- 172.21.15.82:0/1296509154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f564c100fd0 0x7f564c196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:04.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:04.164+0000 7f5651cad700 1 -- 172.21.15.82:0/1296509154 >> 172.21.15.82:0/1296509154 conn(0x7f564c0fc0a0 msgr2=0x7f564c107500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:04.167 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:04.164+0000 7f5651cad700 1 -- 172.21.15.82:0/1296509154 shutdown_connections 2023-09-09T23:13:04.167 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:04.164+0000 7f5651cad700 1 -- 172.21.15.82:0/1296509154 wait complete. 2023-09-09T23:13:04.167 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-09-09T23:13:04.454 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:04 smithi082 ceph-mon[98694]: pgmap v278: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-09T23:13:04.544 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2023-09-09T23:13:02.989000+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654105,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":547,"num_read_kb":397,"num_write":413,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336664,"kb_used_data":12472,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124605160,"statfs":{"total":1151940427776,"available":1151595683840,"internally_reserved":0,"allocated":12771328,"data_stored":6954073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":60,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003124"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.988649+0000","last_change":"2023-09-09T23:12:17.903811+0000","last_active":"2023-09-09T23:12:18.988649+0000","last_peered":"2023-09-09T23:12:18.988649+0000","last_clean":"2023-09-09T23:12:18.988649+0000","last_became_active":"2023-09-09T23:12:17.903386+0000","last_became_peered":"2023-09-09T23:12:17.903386+0000","last_unstale":"2023-09-09T23:12:18.988649+0000","last_undegraded":"2023-09-09T23:12:18.988649+0000","last_fullsized":"2023-09-09T23:12:18.988649+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:36:45.122068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.485007+0000","last_change":"2023-09-09T23:12:17.897915+0000","last_active":"2023-09-09T23:12:19.485007+0000","last_peered":"2023-09-09T23:12:19.485007+0000","last_clean":"2023-09-09T23:12:19.485007+0000","last_became_active":"2023-09-09T23:12:17.896727+0000","last_became_peered":"2023-09-09T23:12:17.896727+0000","last_unstale":"2023-09-09T23:12:19.485007+0000","last_undegraded":"2023-09-09T23:12:19.485007+0000","last_fullsized":"2023-09-09T23:12:19.485007+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:36:03.053572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.485083+0000","last_change":"2023-09-09T23:12:17.907957+0000","last_active":"2023-09-09T23:12:19.485083+0000","last_peered":"2023-09-09T23:12:19.485083+0000","last_clean":"2023-09-09T23:12:19.485083+0000","last_became_active":"2023-09-09T23:12:17.907625+0000","last_became_peered":"2023-09-09T23:12:17.907625+0000","last_unstale":"2023-09-09T23:12:19.485083+0000","last_undegraded":"2023-09-09T23:12:19.485083+0000","last_fullsized":"2023-09-09T23:12:19.485083+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:26:56.520987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.045364+0000","last_change":"2023-09-09T23:12:17.903489+0000","last_active":"2023-09-09T23:12:20.045364+0000","last_peered":"2023-09-09T23:12:20.045364+0000","last_clean":"2023-09-09T23:12:20.045364+0000","last_became_active":"2023-09-09T23:12:17.901184+0000","last_became_peered":"2023-09-09T23:12:17.901184+0000","last_unstale":"2023-09-09T23:12:20.045364+0000","last_undegraded":"2023-09-09T23:12:20.045364+0000","last_fullsized":"2023-09-09T23:12:20.045364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:42:01.899998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897810+0000","last_change":"2023-09-09T23:12:17.899487+0000","last_active":"2023-09-09T23:12:18.897810+0000","last_peered":"2023-09-09T23:12:18.897810+0000","last_clean":"2023-09-09T23:12:18.897810+0000","last_became_active":"2023-09-09T23:12:17.899083+0000","last_became_peered":"2023-09-09T23:12:17.899083+0000","last_unstale":"2023-09-09T23:12:18.897810+0000","last_undegraded":"2023-09-09T23:12:18.897810+0000","last_fullsized":"2023-09-09T23:12:18.897810+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:15:20.109849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690024+0000","last_change":"2023-09-09T23:12:17.901554+0000","last_active":"2023-09-09T23:12:20.690024+0000","last_peered":"2023-09-09T23:12:20.690024+0000","last_clean":"2023-09-09T23:12:20.690024+0000","last_became_active":"2023-09-09T23:12:17.901246+0000","last_became_peered":"2023-09-09T23:12:17.901246+0000","last_unstale":"2023-09-09T23:12:20.690024+0000","last_undegraded":"2023-09-09T23:12:20.690024+0000","last_fullsized":"2023-09-09T23:12:20.690024+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:19:08.795927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689639+0000","last_change":"2023-09-09T23:12:17.906989+0000","last_active":"2023-09-09T23:12:20.689639+0000","last_peered":"2023-09-09T23:12:20.689639+0000","last_clean":"2023-09-09T23:12:20.689639+0000","last_became_active":"2023-09-09T23:12:17.906704+0000","last_became_peered":"2023-09-09T23:12:17.906704+0000","last_unstale":"2023-09-09T23:12:20.689639+0000","last_undegraded":"2023-09-09T23:12:20.689639+0000","last_fullsized":"2023-09-09T23:12:20.689639+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:41:33.612168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897513+0000","last_change":"2023-09-09T23:12:17.906269+0000","last_active":"2023-09-09T23:12:18.897513+0000","last_peered":"2023-09-09T23:12:18.897513+0000","last_clean":"2023-09-09T23:12:18.897513+0000","last_became_active":"2023-09-09T23:12:17.906017+0000","last_became_peered":"2023-09-09T23:12:17.906017+0000","last_unstale":"2023-09-09T23:12:18.897513+0000","last_undegraded":"2023-09-09T23:12:18.897513+0000","last_fullsized":"2023-09-09T23:12:18.897513+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:50:21.893506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.520143+0000","last_change":"2023-09-09T23:12:17.905191+0000","last_active":"2023-09-09T23:12:19.520143+0000","last_peered":"2023-09-09T23:12:19.520143+0000","last_clean":"2023-09-09T23:12:19.520143+0000","last_became_active":"2023-09-09T23:12:17.904813+0000","last_became_peered":"2023-09-09T23:12:17.904813+0000","last_unstale":"2023-09-09T23:12:19.520143+0000","last_undegraded":"2023-09-09T23:12:19.520143+0000","last_fullsized":"2023-09-09T23:12:19.520143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:29:54.722210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690667+0000","last_change":"2023-09-09T23:12:17.899980+0000","last_active":"2023-09-09T23:12:20.690667+0000","last_peered":"2023-09-09T23:12:20.690667+0000","last_clean":"2023-09-09T23:12:20.690667+0000","last_became_active":"2023-09-09T23:12:17.899692+0000","last_became_peered":"2023-09-09T23:12:17.899692+0000","last_unstale":"2023-09-09T23:12:20.690667+0000","last_undegraded":"2023-09-09T23:12:20.690667+0000","last_fullsized":"2023-09-09T23:12:20.690667+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:25:28.056684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690090+0000","last_change":"2023-09-09T23:12:17.907010+0000","last_active":"2023-09-09T23:12:20.690090+0000","last_peered":"2023-09-09T23:12:20.690090+0000","last_clean":"2023-09-09T23:12:20.690090+0000","last_became_active":"2023-09-09T23:12:17.906565+0000","last_became_peered":"2023-09-09T23:12:17.906565+0000","last_unstale":"2023-09-09T23:12:20.690090+0000","last_undegraded":"2023-09-09T23:12:20.690090+0000","last_fullsized":"2023-09-09T23:12:20.690090+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:08:03.733608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.045231+0000","last_change":"2023-09-09T23:12:17.902889+0000","last_active":"2023-09-09T23:12:20.045231+0000","last_peered":"2023-09-09T23:12:20.045231+0000","last_clean":"2023-09-09T23:12:20.045231+0000","last_became_active":"2023-09-09T23:12:17.900747+0000","last_became_peered":"2023-09-09T23:12:17.900747+0000","last_unstale":"2023-09-09T23:12:20.045231+0000","last_undegraded":"2023-09-09T23:12:20.045231+0000","last_fullsized":"2023-09-09T23:12:20.045231+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:01:29.195275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689191+0000","last_change":"2023-09-09T23:12:17.897844+0000","last_active":"2023-09-09T23:12:20.689191+0000","last_peered":"2023-09-09T23:12:20.689191+0000","last_clean":"2023-09-09T23:12:20.689191+0000","last_became_active":"2023-09-09T23:12:17.897529+0000","last_became_peered":"2023-09-09T23:12:17.897529+0000","last_unstale":"2023-09-09T23:12:20.689191+0000","last_undegraded":"2023-09-09T23:12:20.689191+0000","last_fullsized":"2023-09-09T23:12:20.689191+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:37:37.425752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.484862+0000","last_change":"2023-09-09T23:12:17.899255+0000","last_active":"2023-09-09T23:12:19.484862+0000","last_peered":"2023-09-09T23:12:19.484862+0000","last_clean":"2023-09-09T23:12:19.484862+0000","last_became_active":"2023-09-09T23:12:17.898932+0000","last_became_peered":"2023-09-09T23:12:17.898932+0000","last_unstale":"2023-09-09T23:12:19.484862+0000","last_undegraded":"2023-09-09T23:12:19.484862+0000","last_fullsized":"2023-09-09T23:12:19.484862+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:10:56.864588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.484825+0000","last_change":"2023-09-09T23:12:17.901083+0000","last_active":"2023-09-09T23:12:19.484825+0000","last_peered":"2023-09-09T23:12:19.484825+0000","last_clean":"2023-09-09T23:12:19.484825+0000","last_became_active":"2023-09-09T23:12:17.900744+0000","last_became_peered":"2023-09-09T23:12:17.900744+0000","last_unstale":"2023-09-09T23:12:19.484825+0000","last_undegraded":"2023-09-09T23:12:19.484825+0000","last_fullsized":"2023-09-09T23:12:19.484825+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:37:11.073786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.519605+0000","last_change":"2023-09-09T23:12:17.901962+0000","last_active":"2023-09-09T23:12:19.519605+0000","last_peered":"2023-09-09T23:12:19.519605+0000","last_clean":"2023-09-09T23:12:19.519605+0000","last_became_active":"2023-09-09T23:12:17.901638+0000","last_became_peered":"2023-09-09T23:12:17.901638+0000","last_unstale":"2023-09-09T23:12:19.519605+0000","last_undegraded":"2023-09-09T23:12:19.519605+0000","last_fullsized":"2023-09-09T23:12:19.519605+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:34:52.083262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.988721+0000","last_change":"2023-09-09T23:12:17.904424+0000","last_active":"2023-09-09T23:12:18.988721+0000","last_peered":"2023-09-09T23:12:18.988721+0000","last_clean":"2023-09-09T23:12:18.988721+0000","last_became_active":"2023-09-09T23:12:17.903879+0000","last_became_peered":"2023-09-09T23:12:17.903879+0000","last_unstale":"2023-09-09T23:12:18.988721+0000","last_undegraded":"2023-09-09T23:12:18.988721+0000","last_fullsized":"2023-09-09T23:12:18.988721+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:56:42.468630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898791+0000","last_change":"2023-09-09T23:12:17.905065+0000","last_active":"2023-09-09T23:12:18.898791+0000","last_peered":"2023-09-09T23:12:18.898791+0000","last_clean":"2023-09-09T23:12:18.898791+0000","last_became_active":"2023-09-09T23:12:17.904646+0000","last_became_peered":"2023-09-09T23:12:17.904646+0000","last_unstale":"2023-09-09T23:12:18.898791+0000","last_undegraded":"2023-09-09T23:12:18.898791+0000","last_fullsized":"2023-09-09T23:12:18.898791+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:11:14.705884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690263+0000","last_change":"2023-09-09T23:12:17.899751+0000","last_active":"2023-09-09T23:12:20.690263+0000","last_peered":"2023-09-09T23:12:20.690263+0000","last_clean":"2023-09-09T23:12:20.690263+0000","last_became_active":"2023-09-09T23:12:17.899410+0000","last_became_peered":"2023-09-09T23:12:17.899410+0000","last_unstale":"2023-09-09T23:12:20.690263+0000","last_undegraded":"2023-09-09T23:12:20.690263+0000","last_fullsized":"2023-09-09T23:12:20.690263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:23:06.098276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":63,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:59.350604+0000","last_change":"2023-09-09T23:12:17.175671+0000","last_active":"2023-09-09T23:12:59.350604+0000","last_peered":"2023-09-09T23:12:59.350604+0000","last_clean":"2023-09-09T23:12:59.350604+0000","last_became_active":"2023-09-09T23:12:17.175327+0000","last_became_peered":"2023-09-09T23:12:17.175327+0000","last_unstale":"2023-09-09T23:12:59.350604+0000","last_undegraded":"2023-09-09T23:12:59.350604+0000","last_fullsized":"2023-09-09T23:12:59.350604+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:38:57.724260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898994+0000","last_change":"2023-09-09T23:12:17.906338+0000","last_active":"2023-09-09T23:12:18.898994+0000","last_peered":"2023-09-09T23:12:18.898994+0000","last_clean":"2023-09-09T23:12:18.898994+0000","last_became_active":"2023-09-09T23:12:17.906094+0000","last_became_peered":"2023-09-09T23:12:17.906094+0000","last_unstale":"2023-09-09T23:12:18.898994+0000","last_undegraded":"2023-09-09T23:12:18.898994+0000","last_fullsized":"2023-09-09T23:12:18.898994+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:43:18.183144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.899063+0000","last_change":"2023-09-09T23:12:17.167763+0000","last_active":"2023-09-09T23:12:18.899063+0000","last_peered":"2023-09-09T23:12:18.899063+0000","last_clean":"2023-09-09T23:12:18.899063+0000","last_became_active":"2023-09-09T23:12:17.167455+0000","last_became_peered":"2023-09-09T23:12:17.167455+0000","last_unstale":"2023-09-09T23:12:18.899063+0000","last_undegraded":"2023-09-09T23:12:18.899063+0000","last_fullsized":"2023-09-09T23:12:18.899063+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:15:23.415792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.485136+0000","last_change":"2023-09-09T23:12:17.907936+0000","last_active":"2023-09-09T23:12:19.485136+0000","last_peered":"2023-09-09T23:12:19.485136+0000","last_clean":"2023-09-09T23:12:19.485136+0000","last_became_active":"2023-09-09T23:12:17.907588+0000","last_became_peered":"2023-09-09T23:12:17.907588+0000","last_unstale":"2023-09-09T23:12:19.485136+0000","last_undegraded":"2023-09-09T23:12:19.485136+0000","last_fullsized":"2023-09-09T23:12:19.485136+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:57:30.910812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:25.294384+0000","last_change":"2023-09-09T23:12:17.172481+0000","last_active":"2023-09-09T23:12:25.294384+0000","last_peered":"2023-09-09T23:12:25.294384+0000","last_clean":"2023-09-09T23:12:25.294384+0000","last_became_active":"2023-09-09T23:12:17.172178+0000","last_became_peered":"2023-09-09T23:12:17.172178+0000","last_unstale":"2023-09-09T23:12:25.294384+0000","last_undegraded":"2023-09-09T23:12:25.294384+0000","last_fullsized":"2023-09-09T23:12:25.294384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:08:55.328423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.519862+0000","last_change":"2023-09-09T23:12:17.900155+0000","last_active":"2023-09-09T23:12:19.519862+0000","last_peered":"2023-09-09T23:12:19.519862+0000","last_clean":"2023-09-09T23:12:19.519862+0000","last_became_active":"2023-09-09T23:12:17.899862+0000","last_became_peered":"2023-09-09T23:12:17.899862+0000","last_unstale":"2023-09-09T23:12:19.519862+0000","last_undegraded":"2023-09-09T23:12:19.519862+0000","last_fullsized":"2023-09-09T23:12:19.519862+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:57:14.949174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:24.279082+0000","last_change":"2023-09-09T23:12:17.171890+0000","last_active":"2023-09-09T23:12:24.279082+0000","last_peered":"2023-09-09T23:12:24.279082+0000","last_clean":"2023-09-09T23:12:24.279082+0000","last_became_active":"2023-09-09T23:12:17.171496+0000","last_became_peered":"2023-09-09T23:12:17.171496+0000","last_unstale":"2023-09-09T23:12:24.279082+0000","last_undegraded":"2023-09-09T23:12:24.279082+0000","last_fullsized":"2023-09-09T23:12:24.279082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:38:25.704989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689810+0000","last_change":"2023-09-09T23:12:17.901067+0000","last_active":"2023-09-09T23:12:20.689810+0000","last_peered":"2023-09-09T23:12:20.689810+0000","last_clean":"2023-09-09T23:12:20.689810+0000","last_became_active":"2023-09-09T23:12:17.900069+0000","last_became_peered":"2023-09-09T23:12:17.900069+0000","last_unstale":"2023-09-09T23:12:20.689810+0000","last_undegraded":"2023-09-09T23:12:20.689810+0000","last_fullsized":"2023-09-09T23:12:20.689810+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:59:55.356235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":58,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:57.319161+0000","last_change":"2023-09-09T23:12:17.169474+0000","last_active":"2023-09-09T23:12:57.319161+0000","last_peered":"2023-09-09T23:12:57.319161+0000","last_clean":"2023-09-09T23:12:57.319161+0000","last_became_active":"2023-09-09T23:12:17.169186+0000","last_became_peered":"2023-09-09T23:12:17.169186+0000","last_unstale":"2023-09-09T23:12:57.319161+0000","last_undegraded":"2023-09-09T23:12:57.319161+0000","last_fullsized":"2023-09-09T23:12:57.319161+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:47:13.027949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897942+0000","last_change":"2023-09-09T23:12:17.901605+0000","last_active":"2023-09-09T23:12:18.897942+0000","last_peered":"2023-09-09T23:12:18.897942+0000","last_clean":"2023-09-09T23:12:18.897942+0000","last_became_active":"2023-09-09T23:12:17.901287+0000","last_became_peered":"2023-09-09T23:12:17.901287+0000","last_unstale":"2023-09-09T23:12:18.897942+0000","last_undegraded":"2023-09-09T23:12:18.897942+0000","last_fullsized":"2023-09-09T23:12:18.897942+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:16:41.570750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:23.271624+0000","last_change":"2023-09-09T23:12:17.171797+0000","last_active":"2023-09-09T23:12:23.271624+0000","last_peered":"2023-09-09T23:12:23.271624+0000","last_clean":"2023-09-09T23:12:23.271624+0000","last_became_active":"2023-09-09T23:12:17.171521+0000","last_became_peered":"2023-09-09T23:12:17.171521+0000","last_unstale":"2023-09-09T23:12:23.271624+0000","last_undegraded":"2023-09-09T23:12:23.271624+0000","last_fullsized":"2023-09-09T23:12:23.271624+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:07:51.951393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690229+0000","last_change":"2023-09-09T23:12:17.900284+0000","last_active":"2023-09-09T23:12:20.690229+0000","last_peered":"2023-09-09T23:12:20.690229+0000","last_clean":"2023-09-09T23:12:20.690229+0000","last_became_active":"2023-09-09T23:12:17.899975+0000","last_became_peered":"2023-09-09T23:12:17.899975+0000","last_unstale":"2023-09-09T23:12:20.690229+0000","last_undegraded":"2023-09-09T23:12:20.690229+0000","last_fullsized":"2023-09-09T23:12:20.690229+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:10:56.723338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:22.583122+0000","last_change":"2023-09-09T23:12:17.175328+0000","last_active":"2023-09-09T23:12:22.583122+0000","last_peered":"2023-09-09T23:12:22.583122+0000","last_clean":"2023-09-09T23:12:22.583122+0000","last_became_active":"2023-09-09T23:12:17.174977+0000","last_became_peered":"2023-09-09T23:12:17.174977+0000","last_unstale":"2023-09-09T23:12:22.583122+0000","last_undegraded":"2023-09-09T23:12:22.583122+0000","last_fullsized":"2023-09-09T23:12:22.583122+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:01:36.272333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897568+0000","last_change":"2023-09-09T23:12:17.902982+0000","last_active":"2023-09-09T23:12:18.897568+0000","last_peered":"2023-09-09T23:12:18.897568+0000","last_clean":"2023-09-09T23:12:18.897568+0000","last_became_active":"2023-09-09T23:12:17.902677+0000","last_became_peered":"2023-09-09T23:12:17.902677+0000","last_unstale":"2023-09-09T23:12:18.897568+0000","last_undegraded":"2023-09-09T23:12:18.897568+0000","last_fullsized":"2023-09-09T23:12:18.897568+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:08:31.507899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:24.280201+0000","last_change":"2023-09-09T23:12:17.170099+0000","last_active":"2023-09-09T23:12:24.280201+0000","last_peered":"2023-09-09T23:12:24.280201+0000","last_clean":"2023-09-09T23:12:24.280201+0000","last_became_active":"2023-09-09T23:12:17.169827+0000","last_became_peered":"2023-09-09T23:12:17.169827+0000","last_unstale":"2023-09-09T23:12:24.280201+0000","last_undegraded":"2023-09-09T23:12:24.280201+0000","last_fullsized":"2023-09-09T23:12:24.280201+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:16:34.693568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898646+0000","last_change":"2023-09-09T23:12:17.903735+0000","last_active":"2023-09-09T23:12:18.898646+0000","last_peered":"2023-09-09T23:12:18.898646+0000","last_clean":"2023-09-09T23:12:18.898646+0000","last_became_active":"2023-09-09T23:12:17.903410+0000","last_became_peered":"2023-09-09T23:12:17.903410+0000","last_unstale":"2023-09-09T23:12:18.898646+0000","last_undegraded":"2023-09-09T23:12:18.898646+0000","last_fullsized":"2023-09-09T23:12:18.898646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:00:47.664613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":93,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:13:01.325026+0000","last_change":"2023-09-09T23:12:17.176004+0000","last_active":"2023-09-09T23:13:01.325026+0000","last_peered":"2023-09-09T23:13:01.325026+0000","last_clean":"2023-09-09T23:13:01.325026+0000","last_became_active":"2023-09-09T23:12:17.175514+0000","last_became_peered":"2023-09-09T23:12:17.175514+0000","last_unstale":"2023-09-09T23:13:01.325026+0000","last_undegraded":"2023-09-09T23:13:01.325026+0000","last_fullsized":"2023-09-09T23:13:01.325026+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:43:54.587724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.520135+0000","last_change":"2023-09-09T23:12:17.900943+0000","last_active":"2023-09-09T23:12:19.520135+0000","last_peered":"2023-09-09T23:12:19.520135+0000","last_clean":"2023-09-09T23:12:19.520135+0000","last_became_active":"2023-09-09T23:12:17.900662+0000","last_became_peered":"2023-09-09T23:12:17.900662+0000","last_unstale":"2023-09-09T23:12:19.520135+0000","last_undegraded":"2023-09-09T23:12:19.520135+0000","last_fullsized":"2023-09-09T23:12:19.520135+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:39:41.480033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:24.280105+0000","last_change":"2023-09-09T23:12:17.172458+0000","last_active":"2023-09-09T23:12:24.280105+0000","last_peered":"2023-09-09T23:12:24.280105+0000","last_clean":"2023-09-09T23:12:24.280105+0000","last_became_active":"2023-09-09T23:12:17.172119+0000","last_became_peered":"2023-09-09T23:12:17.172119+0000","last_unstale":"2023-09-09T23:12:24.280105+0000","last_undegraded":"2023-09-09T23:12:24.280105+0000","last_fullsized":"2023-09-09T23:12:24.280105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:31:27.821245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.485149+0000","last_change":"2023-09-09T23:12:17.903252+0000","last_active":"2023-09-09T23:12:19.485149+0000","last_peered":"2023-09-09T23:12:19.485149+0000","last_clean":"2023-09-09T23:12:19.485149+0000","last_became_active":"2023-09-09T23:12:17.902466+0000","last_became_peered":"2023-09-09T23:12:17.902466+0000","last_unstale":"2023-09-09T23:12:19.485149+0000","last_undegraded":"2023-09-09T23:12:19.485149+0000","last_fullsized":"2023-09-09T23:12:19.485149+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:28:07.535937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:23.264739+0000","last_change":"2023-09-09T23:12:17.165106+0000","last_active":"2023-09-09T23:12:23.264739+0000","last_peered":"2023-09-09T23:12:23.264739+0000","last_clean":"2023-09-09T23:12:23.264739+0000","last_became_active":"2023-09-09T23:12:17.164783+0000","last_became_peered":"2023-09-09T23:12:17.164783+0000","last_unstale":"2023-09-09T23:12:23.264739+0000","last_undegraded":"2023-09-09T23:12:23.264739+0000","last_fullsized":"2023-09-09T23:12:23.264739+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:02:14.984870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.519952+0000","last_change":"2023-09-09T23:12:17.899473+0000","last_active":"2023-09-09T23:12:19.519952+0000","last_peered":"2023-09-09T23:12:19.519952+0000","last_clean":"2023-09-09T23:12:19.519952+0000","last_became_active":"2023-09-09T23:12:17.899142+0000","last_became_peered":"2023-09-09T23:12:17.899142+0000","last_unstale":"2023-09-09T23:12:19.519952+0000","last_undegraded":"2023-09-09T23:12:19.519952+0000","last_fullsized":"2023-09-09T23:12:19.519952+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:50:30.261873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'29","reported_seq":128,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:13:01.329027+0000","last_change":"2023-09-09T23:12:17.172560+0000","last_active":"2023-09-09T23:13:01.329027+0000","last_peered":"2023-09-09T23:13:01.329027+0000","last_clean":"2023-09-09T23:13:01.329027+0000","last_became_active":"2023-09-09T23:12:17.172213+0000","last_became_peered":"2023-09-09T23:12:17.172213+0000","last_unstale":"2023-09-09T23:13:01.329027+0000","last_undegraded":"2023-09-09T23:13:01.329027+0000","last_fullsized":"2023-09-09T23:13:01.329027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:30:00.206901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56289,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":86,"num_read_kb":97,"num_write":31,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898030+0000","last_change":"2023-09-09T23:12:17.898279+0000","last_active":"2023-09-09T23:12:18.898030+0000","last_peered":"2023-09-09T23:12:18.898030+0000","last_clean":"2023-09-09T23:12:18.898030+0000","last_became_active":"2023-09-09T23:12:17.898041+0000","last_became_peered":"2023-09-09T23:12:17.898041+0000","last_unstale":"2023-09-09T23:12:18.898030+0000","last_undegraded":"2023-09-09T23:12:18.898030+0000","last_fullsized":"2023-09-09T23:12:18.898030+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:01:10.315445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:24.277670+0000","last_change":"2023-09-09T23:12:17.172079+0000","last_active":"2023-09-09T23:12:24.277670+0000","last_peered":"2023-09-09T23:12:24.277670+0000","last_clean":"2023-09-09T23:12:24.277670+0000","last_became_active":"2023-09-09T23:12:17.171779+0000","last_became_peered":"2023-09-09T23:12:17.171779+0000","last_unstale":"2023-09-09T23:12:24.277670+0000","last_undegraded":"2023-09-09T23:12:24.277670+0000","last_fullsized":"2023-09-09T23:12:24.277670+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:49:50.287124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.688927+0000","last_change":"2023-09-09T23:12:17.904087+0000","last_active":"2023-09-09T23:12:20.688927+0000","last_peered":"2023-09-09T23:12:20.688927+0000","last_clean":"2023-09-09T23:12:20.688927+0000","last_became_active":"2023-09-09T23:12:17.903670+0000","last_became_peered":"2023-09-09T23:12:17.903670+0000","last_unstale":"2023-09-09T23:12:20.688927+0000","last_undegraded":"2023-09-09T23:12:20.688927+0000","last_fullsized":"2023-09-09T23:12:20.688927+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:24:04.403871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:24.280209+0000","last_change":"2023-09-09T23:12:17.170594+0000","last_active":"2023-09-09T23:12:24.280209+0000","last_peered":"2023-09-09T23:12:24.280209+0000","last_clean":"2023-09-09T23:12:24.280209+0000","last_became_active":"2023-09-09T23:12:17.170223+0000","last_became_peered":"2023-09-09T23:12:17.170223+0000","last_unstale":"2023-09-09T23:12:24.280209+0000","last_undegraded":"2023-09-09T23:12:24.280209+0000","last_fullsized":"2023-09-09T23:12:24.280209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:30:09.222947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689977+0000","last_change":"2023-09-09T23:12:17.901047+0000","last_active":"2023-09-09T23:12:20.689977+0000","last_peered":"2023-09-09T23:12:20.689977+0000","last_clean":"2023-09-09T23:12:20.689977+0000","last_became_active":"2023-09-09T23:12:17.900129+0000","last_became_peered":"2023-09-09T23:12:17.900129+0000","last_unstale":"2023-09-09T23:12:20.689977+0000","last_undegraded":"2023-09-09T23:12:20.689977+0000","last_fullsized":"2023-09-09T23:12:20.689977+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:59:36.338392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'6","reported_seq":58,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:57.318732+0000","last_change":"2023-09-09T23:12:17.169495+0000","last_active":"2023-09-09T23:12:57.318732+0000","last_peered":"2023-09-09T23:12:57.318732+0000","last_clean":"2023-09-09T23:12:57.318732+0000","last_became_active":"2023-09-09T23:12:17.169170+0000","last_became_peered":"2023-09-09T23:12:17.169170+0000","last_unstale":"2023-09-09T23:12:57.318732+0000","last_undegraded":"2023-09-09T23:12:57.318732+0000","last_fullsized":"2023-09-09T23:12:57.318732+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:36:23.765574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689415+0000","last_change":"2023-09-09T23:12:17.907826+0000","last_active":"2023-09-09T23:12:20.689415+0000","last_peered":"2023-09-09T23:12:20.689415+0000","last_clean":"2023-09-09T23:12:20.689415+0000","last_became_active":"2023-09-09T23:12:17.907526+0000","last_became_peered":"2023-09-09T23:12:17.907526+0000","last_unstale":"2023-09-09T23:12:20.689415+0000","last_undegraded":"2023-09-09T23:12:20.689415+0000","last_fullsized":"2023-09-09T23:12:20.689415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:17:20.587253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:23.270744+0000","last_change":"2023-09-09T23:12:17.170139+0000","last_active":"2023-09-09T23:12:23.270744+0000","last_peered":"2023-09-09T23:12:23.270744+0000","last_clean":"2023-09-09T23:12:23.270744+0000","last_became_active":"2023-09-09T23:12:17.169797+0000","last_became_peered":"2023-09-09T23:12:17.169797+0000","last_unstale":"2023-09-09T23:12:23.270744+0000","last_undegraded":"2023-09-09T23:12:23.270744+0000","last_fullsized":"2023-09-09T23:12:23.270744+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:59:44.668114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689753+0000","last_change":"2023-09-09T23:12:17.902567+0000","last_active":"2023-09-09T23:12:20.689753+0000","last_peered":"2023-09-09T23:12:20.689753+0000","last_clean":"2023-09-09T23:12:20.689753+0000","last_became_active":"2023-09-09T23:12:17.902229+0000","last_became_peered":"2023-09-09T23:12:17.902229+0000","last_unstale":"2023-09-09T23:12:20.689753+0000","last_undegraded":"2023-09-09T23:12:20.689753+0000","last_fullsized":"2023-09-09T23:12:20.689753+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:49:50.049925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:23.260243+0000","last_change":"2023-09-09T23:12:17.171143+0000","last_active":"2023-09-09T23:12:23.260243+0000","last_peered":"2023-09-09T23:12:23.260243+0000","last_clean":"2023-09-09T23:12:23.260243+0000","last_became_active":"2023-09-09T23:12:17.170847+0000","last_became_peered":"2023-09-09T23:12:17.170847+0000","last_unstale":"2023-09-09T23:12:23.260243+0000","last_undegraded":"2023-09-09T23:12:23.260243+0000","last_fullsized":"2023-09-09T23:12:23.260243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:35:55.570394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.988514+0000","last_change":"2023-09-09T23:12:17.900284+0000","last_active":"2023-09-09T23:12:18.988514+0000","last_peered":"2023-09-09T23:12:18.988514+0000","last_clean":"2023-09-09T23:12:18.988514+0000","last_became_active":"2023-09-09T23:12:17.899923+0000","last_became_peered":"2023-09-09T23:12:17.899923+0000","last_unstale":"2023-09-09T23:12:18.988514+0000","last_undegraded":"2023-09-09T23:12:18.988514+0000","last_fullsized":"2023-09-09T23:12:18.988514+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:32:32.230968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898542+0000","last_change":"2023-09-09T23:12:17.176313+0000","last_active":"2023-09-09T23:12:18.898542+0000","last_peered":"2023-09-09T23:12:18.898542+0000","last_clean":"2023-09-09T23:12:18.898542+0000","last_became_active":"2023-09-09T23:12:17.175934+0000","last_became_peered":"2023-09-09T23:12:17.175934+0000","last_unstale":"2023-09-09T23:12:18.898542+0000","last_undegraded":"2023-09-09T23:12:18.898542+0000","last_fullsized":"2023-09-09T23:12:18.898542+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:33:15.044986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:21.932198+0000","last_change":"2023-09-09T23:12:17.900284+0000","last_active":"2023-09-09T23:12:21.932198+0000","last_peered":"2023-09-09T23:12:21.932198+0000","last_clean":"2023-09-09T23:12:21.932198+0000","last_became_active":"2023-09-09T23:12:17.899970+0000","last_became_peered":"2023-09-09T23:12:17.899970+0000","last_unstale":"2023-09-09T23:12:21.932198+0000","last_undegraded":"2023-09-09T23:12:21.932198+0000","last_fullsized":"2023-09-09T23:12:21.932198+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:01:37.415195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":54,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:58.349691+0000","last_change":"2023-09-09T23:12:17.170123+0000","last_active":"2023-09-09T23:12:58.349691+0000","last_peered":"2023-09-09T23:12:58.349691+0000","last_clean":"2023-09-09T23:12:58.349691+0000","last_became_active":"2023-09-09T23:12:17.169775+0000","last_became_peered":"2023-09-09T23:12:17.169775+0000","last_unstale":"2023-09-09T23:12:58.349691+0000","last_undegraded":"2023-09-09T23:12:58.349691+0000","last_fullsized":"2023-09-09T23:12:58.349691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:22:44.520489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690417+0000","last_change":"2023-09-09T23:12:17.894881+0000","last_active":"2023-09-09T23:12:20.690417+0000","last_peered":"2023-09-09T23:12:20.690417+0000","last_clean":"2023-09-09T23:12:20.690417+0000","last_became_active":"2023-09-09T23:12:17.894567+0000","last_became_peered":"2023-09-09T23:12:17.894567+0000","last_unstale":"2023-09-09T23:12:20.690417+0000","last_undegraded":"2023-09-09T23:12:20.690417+0000","last_fullsized":"2023-09-09T23:12:20.690417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:40:37.098780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689780+0000","last_change":"2023-09-09T23:12:17.903319+0000","last_active":"2023-09-09T23:12:20.689780+0000","last_peered":"2023-09-09T23:12:20.689780+0000","last_clean":"2023-09-09T23:12:20.689780+0000","last_became_active":"2023-09-09T23:12:17.902642+0000","last_became_peered":"2023-09-09T23:12:17.902642+0000","last_unstale":"2023-09-09T23:12:20.689780+0000","last_undegraded":"2023-09-09T23:12:20.689780+0000","last_fullsized":"2023-09-09T23:12:20.689780+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:03:29.562813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":56,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:58.319787+0000","last_change":"2023-09-09T23:12:17.172639+0000","last_active":"2023-09-09T23:12:58.319787+0000","last_peered":"2023-09-09T23:12:58.319787+0000","last_clean":"2023-09-09T23:12:58.319787+0000","last_became_active":"2023-09-09T23:12:17.172309+0000","last_became_peered":"2023-09-09T23:12:17.172309+0000","last_unstale":"2023-09-09T23:12:58.319787+0000","last_undegraded":"2023-09-09T23:12:58.319787+0000","last_fullsized":"2023-09-09T23:12:58.319787+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:05:56.160984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":38,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898148+0000","last_change":"2023-09-09T23:12:17.906676+0000","last_active":"2023-09-09T23:12:18.898148+0000","last_peered":"2023-09-09T23:12:18.898148+0000","last_clean":"2023-09-09T23:12:18.898148+0000","last_became_active":"2023-09-09T23:12:17.906381+0000","last_became_peered":"2023-09-09T23:12:17.906381+0000","last_unstale":"2023-09-09T23:12:18.898148+0000","last_undegraded":"2023-09-09T23:12:18.898148+0000","last_fullsized":"2023-09-09T23:12:18.898148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:42:28.033255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:24.279852+0000","last_change":"2023-09-09T23:12:17.166623+0000","last_active":"2023-09-09T23:12:24.279852+0000","last_peered":"2023-09-09T23:12:24.279852+0000","last_clean":"2023-09-09T23:12:24.279852+0000","last_became_active":"2023-09-09T23:12:17.166293+0000","last_became_peered":"2023-09-09T23:12:17.166293+0000","last_unstale":"2023-09-09T23:12:24.279852+0000","last_undegraded":"2023-09-09T23:12:24.279852+0000","last_fullsized":"2023-09-09T23:12:24.279852+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:52:51.267501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":317,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.963859+0000","last_change":"2023-09-09T23:11:20.948461+0000","last_active":"2023-09-09T23:12:19.963859+0000","last_peered":"2023-09-09T23:12:19.963859+0000","last_clean":"2023-09-09T23:12:19.963859+0000","last_became_active":"2023-09-09T23:11:20.947856+0000","last_became_peered":"2023-09-09T23:11:20.947856+0000","last_unstale":"2023-09-09T23:12:19.963859+0000","last_undegraded":"2023-09-09T23:12:19.963859+0000","last_fullsized":"2023-09-09T23:12:19.963859+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:08:31.737051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:08:31.737051+0000","last_clean_scrub_stamp":"2023-09-09T23:08:31.737051+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:20:16.293707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.688806+0000","last_change":"2023-09-09T23:12:17.907468+0000","last_active":"2023-09-09T23:12:20.688806+0000","last_peered":"2023-09-09T23:12:20.688806+0000","last_clean":"2023-09-09T23:12:20.688806+0000","last_became_active":"2023-09-09T23:12:17.907117+0000","last_became_peered":"2023-09-09T23:12:17.907117+0000","last_unstale":"2023-09-09T23:12:20.688806+0000","last_undegraded":"2023-09-09T23:12:20.688806+0000","last_fullsized":"2023-09-09T23:12:20.688806+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:27:16.623116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:22.583028+0000","last_change":"2023-09-09T23:12:17.171505+0000","last_active":"2023-09-09T23:12:22.583028+0000","last_peered":"2023-09-09T23:12:22.583028+0000","last_clean":"2023-09-09T23:12:22.583028+0000","last_became_active":"2023-09-09T23:12:17.171185+0000","last_became_peered":"2023-09-09T23:12:17.171185+0000","last_unstale":"2023-09-09T23:12:22.583028+0000","last_undegraded":"2023-09-09T23:12:22.583028+0000","last_fullsized":"2023-09-09T23:12:22.583028+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:07:54.271364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:23.271034+0000","last_change":"2023-09-09T23:12:17.176450+0000","last_active":"2023-09-09T23:12:23.271034+0000","last_peered":"2023-09-09T23:12:23.271034+0000","last_clean":"2023-09-09T23:12:23.271034+0000","last_became_active":"2023-09-09T23:12:17.176217+0000","last_became_peered":"2023-09-09T23:12:17.176217+0000","last_unstale":"2023-09-09T23:12:23.271034+0000","last_undegraded":"2023-09-09T23:12:23.271034+0000","last_fullsized":"2023-09-09T23:12:23.271034+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:57:26.604261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689035+0000","last_change":"2023-09-09T23:12:17.897612+0000","last_active":"2023-09-09T23:12:20.689035+0000","last_peered":"2023-09-09T23:12:20.689035+0000","last_clean":"2023-09-09T23:12:20.689035+0000","last_became_active":"2023-09-09T23:12:17.897292+0000","last_became_peered":"2023-09-09T23:12:17.897292+0000","last_unstale":"2023-09-09T23:12:20.689035+0000","last_undegraded":"2023-09-09T23:12:20.689035+0000","last_fullsized":"2023-09-09T23:12:20.689035+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:25:46.374556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'9","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:26.680531+0000","last_change":"2023-09-09T23:12:17.175682+0000","last_active":"2023-09-09T23:12:26.680531+0000","last_peered":"2023-09-09T23:12:26.680531+0000","last_clean":"2023-09-09T23:12:26.680531+0000","last_became_active":"2023-09-09T23:12:17.175291+0000","last_became_peered":"2023-09-09T23:12:17.175291+0000","last_unstale":"2023-09-09T23:12:26.680531+0000","last_undegraded":"2023-09-09T23:12:26.680531+0000","last_fullsized":"2023-09-09T23:12:26.680531+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:29:02.349319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.899164+0000","last_change":"2023-09-09T23:12:17.904153+0000","last_active":"2023-09-09T23:12:18.899164+0000","last_peered":"2023-09-09T23:12:18.899164+0000","last_clean":"2023-09-09T23:12:18.899164+0000","last_became_active":"2023-09-09T23:12:17.903804+0000","last_became_peered":"2023-09-09T23:12:17.903804+0000","last_unstale":"2023-09-09T23:12:18.899164+0000","last_undegraded":"2023-09-09T23:12:18.899164+0000","last_fullsized":"2023-09-09T23:12:18.899164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:39:34.046551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:24.280288+0000","last_change":"2023-09-09T23:12:17.169052+0000","last_active":"2023-09-09T23:12:24.280288+0000","last_peered":"2023-09-09T23:12:24.280288+0000","last_clean":"2023-09-09T23:12:24.280288+0000","last_became_active":"2023-09-09T23:12:17.168719+0000","last_became_peered":"2023-09-09T23:12:17.168719+0000","last_unstale":"2023-09-09T23:12:24.280288+0000","last_undegraded":"2023-09-09T23:12:24.280288+0000","last_fullsized":"2023-09-09T23:12:24.280288+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:56:44.427954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.485050+0000","last_change":"2023-09-09T23:12:17.900945+0000","last_active":"2023-09-09T23:12:19.485050+0000","last_peered":"2023-09-09T23:12:19.485050+0000","last_clean":"2023-09-09T23:12:19.485050+0000","last_became_active":"2023-09-09T23:12:17.900602+0000","last_became_peered":"2023-09-09T23:12:17.900602+0000","last_unstale":"2023-09-09T23:12:19.485050+0000","last_undegraded":"2023-09-09T23:12:19.485050+0000","last_fullsized":"2023-09-09T23:12:19.485050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:22:10.336285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:21.574908+0000","last_change":"2023-09-09T23:12:17.166619+0000","last_active":"2023-09-09T23:12:21.574908+0000","last_peered":"2023-09-09T23:12:21.574908+0000","last_clean":"2023-09-09T23:12:21.574908+0000","last_became_active":"2023-09-09T23:12:17.166286+0000","last_became_peered":"2023-09-09T23:12:17.166286+0000","last_unstale":"2023-09-09T23:12:21.574908+0000","last_undegraded":"2023-09-09T23:12:21.574908+0000","last_fullsized":"2023-09-09T23:12:21.574908+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:41:40.713296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897742+0000","last_change":"2023-09-09T23:12:17.897131+0000","last_active":"2023-09-09T23:12:18.897742+0000","last_peered":"2023-09-09T23:12:18.897742+0000","last_clean":"2023-09-09T23:12:18.897742+0000","last_became_active":"2023-09-09T23:12:17.896873+0000","last_became_peered":"2023-09-09T23:12:17.896873+0000","last_unstale":"2023-09-09T23:12:18.897742+0000","last_undegraded":"2023-09-09T23:12:18.897742+0000","last_fullsized":"2023-09-09T23:12:18.897742+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:54:21.091418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:24.277166+0000","last_change":"2023-09-09T23:12:17.168868+0000","last_active":"2023-09-09T23:12:24.277166+0000","last_peered":"2023-09-09T23:12:24.277166+0000","last_clean":"2023-09-09T23:12:24.277166+0000","last_became_active":"2023-09-09T23:12:17.168572+0000","last_became_peered":"2023-09-09T23:12:17.168572+0000","last_unstale":"2023-09-09T23:12:24.277166+0000","last_undegraded":"2023-09-09T23:12:24.277166+0000","last_fullsized":"2023-09-09T23:12:24.277166+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:54:08.108023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.988326+0000","last_change":"2023-09-09T23:12:17.902887+0000","last_active":"2023-09-09T23:12:18.988326+0000","last_peered":"2023-09-09T23:12:18.988326+0000","last_clean":"2023-09-09T23:12:18.988326+0000","last_became_active":"2023-09-09T23:12:17.902507+0000","last_became_peered":"2023-09-09T23:12:17.902507+0000","last_unstale":"2023-09-09T23:12:18.988326+0000","last_undegraded":"2023-09-09T23:12:18.988326+0000","last_fullsized":"2023-09-09T23:12:18.988326+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:18:08.000184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:23.271048+0000","last_change":"2023-09-09T23:12:17.171379+0000","last_active":"2023-09-09T23:12:23.271048+0000","last_peered":"2023-09-09T23:12:23.271048+0000","last_clean":"2023-09-09T23:12:23.271048+0000","last_became_active":"2023-09-09T23:12:17.171024+0000","last_became_peered":"2023-09-09T23:12:17.171024+0000","last_unstale":"2023-09-09T23:12:23.271048+0000","last_undegraded":"2023-09-09T23:12:23.271048+0000","last_fullsized":"2023-09-09T23:12:23.271048+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:17:12.206768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898049+0000","last_change":"2023-09-09T23:12:17.905510+0000","last_active":"2023-09-09T23:12:18.898049+0000","last_peered":"2023-09-09T23:12:18.898049+0000","last_clean":"2023-09-09T23:12:18.898049+0000","last_became_active":"2023-09-09T23:12:17.905207+0000","last_became_peered":"2023-09-09T23:12:17.905207+0000","last_unstale":"2023-09-09T23:12:18.898049+0000","last_undegraded":"2023-09-09T23:12:18.898049+0000","last_fullsized":"2023-09-09T23:12:18.898049+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:48:20.167421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:21.574555+0000","last_change":"2023-09-09T23:12:17.173764+0000","last_active":"2023-09-09T23:12:21.574555+0000","last_peered":"2023-09-09T23:12:21.574555+0000","last_clean":"2023-09-09T23:12:21.574555+0000","last_became_active":"2023-09-09T23:12:17.173470+0000","last_became_peered":"2023-09-09T23:12:17.173470+0000","last_unstale":"2023-09-09T23:12:21.574555+0000","last_undegraded":"2023-09-09T23:12:21.574555+0000","last_fullsized":"2023-09-09T23:12:21.574555+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:01:54.091664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690237+0000","last_change":"2023-09-09T23:12:17.906292+0000","last_active":"2023-09-09T23:12:20.690237+0000","last_peered":"2023-09-09T23:12:20.690237+0000","last_clean":"2023-09-09T23:12:20.690237+0000","last_became_active":"2023-09-09T23:12:17.905995+0000","last_became_peered":"2023-09-09T23:12:17.905995+0000","last_unstale":"2023-09-09T23:12:20.690237+0000","last_undegraded":"2023-09-09T23:12:20.690237+0000","last_fullsized":"2023-09-09T23:12:20.690237+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:52:29.972831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'5","reported_seq":57,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:13:00.358958+0000","last_change":"2023-09-09T23:12:17.173818+0000","last_active":"2023-09-09T23:13:00.358958+0000","last_peered":"2023-09-09T23:13:00.358958+0000","last_clean":"2023-09-09T23:13:00.358958+0000","last_became_active":"2023-09-09T23:12:17.173413+0000","last_became_peered":"2023-09-09T23:12:17.173413+0000","last_unstale":"2023-09-09T23:13:00.358958+0000","last_undegraded":"2023-09-09T23:13:00.358958+0000","last_fullsized":"2023-09-09T23:13:00.358958+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:55:28.856863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690266+0000","last_change":"2023-09-09T23:12:17.906833+0000","last_active":"2023-09-09T23:12:20.690266+0000","last_peered":"2023-09-09T23:12:20.690266+0000","last_clean":"2023-09-09T23:12:20.690266+0000","last_became_active":"2023-09-09T23:12:17.906138+0000","last_became_peered":"2023-09-09T23:12:17.906138+0000","last_unstale":"2023-09-09T23:12:20.690266+0000","last_undegraded":"2023-09-09T23:12:20.690266+0000","last_fullsized":"2023-09-09T23:12:20.690266+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:38:07.779167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:23.271410+0000","last_change":"2023-09-09T23:12:17.171338+0000","last_active":"2023-09-09T23:12:23.271410+0000","last_peered":"2023-09-09T23:12:23.271410+0000","last_clean":"2023-09-09T23:12:23.271410+0000","last_became_active":"2023-09-09T23:12:17.170613+0000","last_became_peered":"2023-09-09T23:12:17.170613+0000","last_unstale":"2023-09-09T23:12:23.271410+0000","last_undegraded":"2023-09-09T23:12:23.271410+0000","last_fullsized":"2023-09-09T23:12:23.271410+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:24:07.794464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689777+0000","last_change":"2023-09-09T23:12:17.906470+0000","last_active":"2023-09-09T23:12:20.689777+0000","last_peered":"2023-09-09T23:12:20.689777+0000","last_clean":"2023-09-09T23:12:20.689777+0000","last_became_active":"2023-09-09T23:12:17.906170+0000","last_became_peered":"2023-09-09T23:12:17.906170+0000","last_unstale":"2023-09-09T23:12:20.689777+0000","last_undegraded":"2023-09-09T23:12:20.689777+0000","last_fullsized":"2023-09-09T23:12:20.689777+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:44:37.483541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:24.275625+0000","last_change":"2023-09-09T23:12:17.172142+0000","last_active":"2023-09-09T23:12:24.275625+0000","last_peered":"2023-09-09T23:12:24.275625+0000","last_clean":"2023-09-09T23:12:24.275625+0000","last_became_active":"2023-09-09T23:12:17.171703+0000","last_became_peered":"2023-09-09T23:12:17.171703+0000","last_unstale":"2023-09-09T23:12:24.275625+0000","last_undegraded":"2023-09-09T23:12:24.275625+0000","last_fullsized":"2023-09-09T23:12:24.275625+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:49:31.414385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689874+0000","last_change":"2023-09-09T23:12:17.900699+0000","last_active":"2023-09-09T23:12:20.689874+0000","last_peered":"2023-09-09T23:12:20.689874+0000","last_clean":"2023-09-09T23:12:20.689874+0000","last_became_active":"2023-09-09T23:12:17.900416+0000","last_became_peered":"2023-09-09T23:12:17.900416+0000","last_unstale":"2023-09-09T23:12:20.689874+0000","last_undegraded":"2023-09-09T23:12:20.689874+0000","last_fullsized":"2023-09-09T23:12:20.689874+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:54:29.328413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.988389+0000","last_change":"2023-09-09T23:12:17.900480+0000","last_active":"2023-09-09T23:12:18.988389+0000","last_peered":"2023-09-09T23:12:18.988389+0000","last_clean":"2023-09-09T23:12:18.988389+0000","last_became_active":"2023-09-09T23:12:17.900135+0000","last_became_peered":"2023-09-09T23:12:17.900135+0000","last_unstale":"2023-09-09T23:12:18.988389+0000","last_undegraded":"2023-09-09T23:12:18.988389+0000","last_fullsized":"2023-09-09T23:12:18.988389+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:25:44.254310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690446+0000","last_change":"2023-09-09T23:12:17.901302+0000","last_active":"2023-09-09T23:12:20.690446+0000","last_peered":"2023-09-09T23:12:20.690446+0000","last_clean":"2023-09-09T23:12:20.690446+0000","last_became_active":"2023-09-09T23:12:17.900982+0000","last_became_peered":"2023-09-09T23:12:17.900982+0000","last_unstale":"2023-09-09T23:12:20.690446+0000","last_undegraded":"2023-09-09T23:12:20.690446+0000","last_fullsized":"2023-09-09T23:12:20.690446+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:41:41.119614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689782+0000","last_change":"2023-09-09T23:12:17.904328+0000","last_active":"2023-09-09T23:12:20.689782+0000","last_peered":"2023-09-09T23:12:20.689782+0000","last_clean":"2023-09-09T23:12:20.689782+0000","last_became_active":"2023-09-09T23:12:17.904040+0000","last_became_peered":"2023-09-09T23:12:17.904040+0000","last_unstale":"2023-09-09T23:12:20.689782+0000","last_undegraded":"2023-09-09T23:12:20.689782+0000","last_fullsized":"2023-09-09T23:12:20.689782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:01:05.986097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.485015+0000","last_change":"2023-09-09T23:12:17.901737+0000","last_active":"2023-09-09T23:12:19.485015+0000","last_peered":"2023-09-09T23:12:19.485015+0000","last_clean":"2023-09-09T23:12:19.485015+0000","last_became_active":"2023-09-09T23:12:17.901452+0000","last_became_peered":"2023-09-09T23:12:17.901452+0000","last_unstale":"2023-09-09T23:12:19.485015+0000","last_undegraded":"2023-09-09T23:12:19.485015+0000","last_fullsized":"2023-09-09T23:12:19.485015+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:35:28.698304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.688950+0000","last_change":"2023-09-09T23:12:17.907433+0000","last_active":"2023-09-09T23:12:20.688950+0000","last_peered":"2023-09-09T23:12:20.688950+0000","last_clean":"2023-09-09T23:12:20.688950+0000","last_became_active":"2023-09-09T23:12:17.907127+0000","last_became_peered":"2023-09-09T23:12:17.907127+0000","last_unstale":"2023-09-09T23:12:20.688950+0000","last_undegraded":"2023-09-09T23:12:20.688950+0000","last_fullsized":"2023-09-09T23:12:20.688950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:00:32.291141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.988586+0000","last_change":"2023-09-09T23:12:17.901373+0000","last_active":"2023-09-09T23:12:18.988586+0000","last_peered":"2023-09-09T23:12:18.988586+0000","last_clean":"2023-09-09T23:12:18.988586+0000","last_became_active":"2023-09-09T23:12:17.901080+0000","last_became_peered":"2023-09-09T23:12:17.901080+0000","last_unstale":"2023-09-09T23:12:18.988586+0000","last_undegraded":"2023-09-09T23:12:18.988586+0000","last_fullsized":"2023-09-09T23:12:18.988586+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:23:12.316659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690576+0000","last_change":"2023-09-09T23:12:17.896579+0000","last_active":"2023-09-09T23:12:20.690576+0000","last_peered":"2023-09-09T23:12:20.690576+0000","last_clean":"2023-09-09T23:12:20.690576+0000","last_became_active":"2023-09-09T23:12:17.896253+0000","last_became_peered":"2023-09-09T23:12:17.896253+0000","last_unstale":"2023-09-09T23:12:20.690576+0000","last_undegraded":"2023-09-09T23:12:20.690576+0000","last_fullsized":"2023-09-09T23:12:20.690576+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:30:20.249103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.899186+0000","last_change":"2023-09-09T23:12:17.903569+0000","last_active":"2023-09-09T23:12:18.899186+0000","last_peered":"2023-09-09T23:12:18.899186+0000","last_clean":"2023-09-09T23:12:18.899186+0000","last_became_active":"2023-09-09T23:12:17.903027+0000","last_became_peered":"2023-09-09T23:12:17.903027+0000","last_unstale":"2023-09-09T23:12:18.899186+0000","last_undegraded":"2023-09-09T23:12:18.899186+0000","last_fullsized":"2023-09-09T23:12:18.899186+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:38:46.392973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689604+0000","last_change":"2023-09-09T23:12:17.902944+0000","last_active":"2023-09-09T23:12:20.689604+0000","last_peered":"2023-09-09T23:12:20.689604+0000","last_clean":"2023-09-09T23:12:20.689604+0000","last_became_active":"2023-09-09T23:12:17.901812+0000","last_became_peered":"2023-09-09T23:12:17.901812+0000","last_unstale":"2023-09-09T23:12:20.689604+0000","last_undegraded":"2023-09-09T23:12:20.689604+0000","last_fullsized":"2023-09-09T23:12:20.689604+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:49:50.981693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897608+0000","last_change":"2023-09-09T23:12:17.897612+0000","last_active":"2023-09-09T23:12:18.897608+0000","last_peered":"2023-09-09T23:12:18.897608+0000","last_clean":"2023-09-09T23:12:18.897608+0000","last_became_active":"2023-09-09T23:12:17.897318+0000","last_became_peered":"2023-09-09T23:12:17.897318+0000","last_unstale":"2023-09-09T23:12:18.897608+0000","last_undegraded":"2023-09-09T23:12:18.897608+0000","last_fullsized":"2023-09-09T23:12:18.897608+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:17:38.687470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.519770+0000","last_change":"2023-09-09T23:12:17.903849+0000","last_active":"2023-09-09T23:12:19.519770+0000","last_peered":"2023-09-09T23:12:19.519770+0000","last_clean":"2023-09-09T23:12:19.519770+0000","last_became_active":"2023-09-09T23:12:17.903534+0000","last_became_peered":"2023-09-09T23:12:17.903534+0000","last_unstale":"2023-09-09T23:12:19.519770+0000","last_undegraded":"2023-09-09T23:12:19.519770+0000","last_fullsized":"2023-09-09T23:12:19.519770+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:06:18.073811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689155+0000","last_change":"2023-09-09T23:12:17.899782+0000","last_active":"2023-09-09T23:12:20.689155+0000","last_peered":"2023-09-09T23:12:20.689155+0000","last_clean":"2023-09-09T23:12:20.689155+0000","last_became_active":"2023-09-09T23:12:17.899445+0000","last_became_peered":"2023-09-09T23:12:17.899445+0000","last_unstale":"2023-09-09T23:12:20.689155+0000","last_undegraded":"2023-09-09T23:12:20.689155+0000","last_fullsized":"2023-09-09T23:12:20.689155+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:49:20.276927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.988408+0000","last_change":"2023-09-09T23:12:17.898693+0000","last_active":"2023-09-09T23:12:18.988408+0000","last_peered":"2023-09-09T23:12:18.988408+0000","last_clean":"2023-09-09T23:12:18.988408+0000","last_became_active":"2023-09-09T23:12:17.898343+0000","last_became_peered":"2023-09-09T23:12:17.898343+0000","last_unstale":"2023-09-09T23:12:18.988408+0000","last_undegraded":"2023-09-09T23:12:18.988408+0000","last_fullsized":"2023-09-09T23:12:18.988408+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:37:43.229953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63491,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":411,"num_read_kb":278,"num_write":230,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":190473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514515,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":880640,"data_stored":417227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.502},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.57699999999999996}]},{"osd":1,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.56699999999999995}]},{"osd":2,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.61399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":4,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.75},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.52700000000000002}]},{"osd":5,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.54600000000000004}]},{"osd":6,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.442},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.70599999999999996}]},{"osd":7,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.627}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":9,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.80900000000000005}]},{"osd":10,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.79300000000000004}]}]},{"osd":10,"up_from":69,"seq":296352743448,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1576960,"data_stored":1064941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":1,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.80900000000000005}]},{"osd":2,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.76400000000000001}]},{"osd":3,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.64500000000000002}]},{"osd":4,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.79500000000000004}]},{"osd":5,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.82999999999999996}]},{"osd":6,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.85199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":8,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.94099999999999995}]},{"osd":9,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.878}]},{"osd":11,"last update":"Sat Sep 9 23:12:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":1.004}]}]},{"osd":9,"up_from":64,"seq":274877906971,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":417157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.435,"15min":0.443},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.73999999999999999,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.29499999999999998}]},{"osd":1,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56799999999999995,"15min":0.56899999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.41899999999999998}]},{"osd":2,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.626,"15min":0.63},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.625,"15min":0.63200000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.44500000000000001}]},{"osd":3,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.51300000000000001,"15min":0.52300000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.75}]},{"osd":4,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.629,"15min":0.63500000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96799999999999997,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.67400000000000004,"15min":0.68799999999999994},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81999999999999995,"5min":1.028,"15min":1.028},"last":0.56799999999999995}]},{"osd":5,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.73399999999999999,"15min":0.71499999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":5.6699999999999999,"5min":5.6699999999999999,"15min":5.6699999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.68200000000000005,"15min":0.69399999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.66400000000000003}]},{"osd":6,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.64300000000000002,"15min":0.65500000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.65600000000000003,"15min":0.67000000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.69299999999999995}]},{"osd":7,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.745,"15min":0.76100000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.82499999999999996,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.746,"15min":0.75800000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.71399999999999997}]},{"osd":8,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.73799999999999999,"15min":0.747},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.74099999999999999,"15min":0.755},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.85899999999999999,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.64900000000000002}]},{"osd":10,"last update":"Sat Sep 9 23:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.68100000000000005}]}]},{"osd":8,"up_from":57,"seq":244813135905,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":417535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.38,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80700000000000005,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.87},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.435,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81799999999999995,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.78500000000000003}]},{"osd":1,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.60899999999999999,"15min":0.59399999999999997},"min":{"1min":0.45600000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.59399999999999997,"15min":0.58499999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82599999999999996,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.64200000000000002}]},{"osd":2,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48199999999999998,"15min":0.48299999999999998},"min":{"1min":0.29399999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.47999999999999998,"15min":0.48699999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.60899999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.52800000000000002}]},{"osd":3,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.57099999999999995,"15min":0.57799999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.91300000000000003,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.755},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.63600000000000001,"15min":0.63900000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85699999999999998,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.66400000000000003}]},{"osd":4,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64100000000000001,"15min":0.63700000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94399999999999995,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.62},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.68999999999999995,"15min":0.68200000000000005},"min":{"1min":0.41899999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.77200000000000002}]},{"osd":5,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54700000000000004,"15min":0.55400000000000005},"min":{"1min":0.34100000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68500000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.442},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55500000000000005}]},{"osd":6,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71899999999999997,"15min":0.72499999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.036,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.68400000000000005,"15min":0.69499999999999995},"min":{"1min":0.41199999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86699999999999999,"5min":0.996,"15min":0.996},"last":0.79700000000000004}]},{"osd":7,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71999999999999997,"15min":0.72399999999999998},"min":{"1min":0.51400000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.91200000000000003,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72599999999999998,"15min":0.72699999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.93999999999999995,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.68999999999999995}]},{"osd":9,"last update":"Sat Sep 9 23:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.91600000000000004}]},{"osd":10,"last update":"Sat Sep 9 23:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.84099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]}]},{"osd":7,"up_from":50,"seq":214748364836,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28476,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716676,"statfs":{"total":95995035648,"available":95965876224,"internally_reserved":0,"allocated":1495040,"data_stored":1010117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.439},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.33100000000000002}]},{"osd":1,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.435,"15min":0.432},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.60999999999999999,"5min":0.625,"15min":0.625},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.42699999999999999,"15min":0.41699999999999998},"min":{"1min":0.26200000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.51100000000000001}]},{"osd":2,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.77200000000000002,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.66400000000000003}]},{"osd":3,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.40600000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.626,"15min":0.625},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.79800000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.72299999999999998}]},{"osd":4,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52900000000000003,"15min":0.52300000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.76400000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.314,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.35799999999999998}]},{"osd":5,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.69399999999999995,"15min":0.69599999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86399999999999999,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.66900000000000004,"15min":0.66800000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.53100000000000003}]},{"osd":6,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.746,"15min":0.753},"min":{"1min":0.48899999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90900000000000003,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.72599999999999998,"15min":0.72899999999999998},"min":{"1min":0.46800000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92600000000000005,"5min":1.254,"15min":1.254},"last":0.49299999999999999}]},{"osd":8,"last update":"Sat Sep 9 23:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73399999999999999,"15min":0.73899999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.86399999999999999,"5min":1.008,"15min":1.008},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.76400000000000001,"15min":0.77000000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.88,"5min":1.387,"15min":1.387},"last":0.752}]},{"osd":9,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.75,"15min":0.76000000000000001},"min":{"1min":0.51700000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.89900000000000002,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.92400000000000004,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.627}]},{"osd":10,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.58499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]}]},{"osd":6,"up_from":45,"seq":193273528361,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":888832,"data_stored":416880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.377,"15min":0.36899999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.40400000000000003,"15min":0.40200000000000002},"min":{"1min":0.26800000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.53700000000000003,"5min":0.753,"15min":0.753},"last":0.31}]},{"osd":1,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78100000000000003,"5min":0.877,"15min":0.877},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77200000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.44600000000000001}]},{"osd":2,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.41699999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79600000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76800000000000002,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.56100000000000005}]},{"osd":3,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.43099999999999999,"15min":0.433},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70199999999999996,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.501,"15min":0.51200000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.59399999999999997,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.38200000000000001}]},{"osd":4,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60699999999999998,"15min":0.60299999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.78100000000000003,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.77900000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.54400000000000004}]},{"osd":5,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.63200000000000001,"15min":0.64000000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.75600000000000001,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.495},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.65900000000000003,"15min":0.65700000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82199999999999995,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59499999999999997}]},{"osd":7,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93400000000000005,"5min":1.032,"15min":1.032},"last":1.073},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.91500000000000004}]},{"osd":8,"last update":"Sat Sep 9 23:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.76700000000000002,"15min":0.78800000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85099999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.74399999999999999,"15min":0.76400000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.80100000000000005,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.68799999999999994}]},{"osd":9,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.64500000000000002,"15min":0.66100000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70799999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.72199999999999998,"15min":0.752},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73099999999999998,"5min":1.534,"15min":1.534},"last":0.57899999999999996}]},{"osd":10,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.67400000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":474076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.51600000000000001,"15min":0.498},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.51400000000000001,"15min":0.48999999999999999},"min":{"1min":0.35099999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.46000000000000002}]},{"osd":2,"last update":"Sat Sep 9 23:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.54300000000000004,"15min":0.50900000000000001},"min":{"1min":0.442,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.55600000000000005,"15min":0.52300000000000002},"min":{"1min":0.45200000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.90400000000000003,"5min":1.133,"15min":1.133},"last":0.746}]},{"osd":3,"last update":"Sat Sep 9 23:11:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.47299999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.57499999999999996}]},{"osd":4,"last update":"Sat Sep 9 23:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.438,"15min":0.41399999999999998},"min":{"1min":0.30499999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.48599999999999999,"15min":0.44800000000000001},"min":{"1min":0.28499999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.68500000000000005}]},{"osd":5,"last update":"Sat Sep 9 23:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.58199999999999996,"15min":0.56100000000000005},"min":{"1min":0.40200000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.83299999999999996,"5min":1.022,"15min":1.022},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.60199999999999998,"15min":0.57399999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89400000000000002,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.85199999999999998}]},{"osd":6,"last update":"Sat Sep 9 23:11:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.58499999999999996,"15min":0.56499999999999995},"min":{"1min":0.45300000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.628},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.40699999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.65000000000000002}]},{"osd":7,"last update":"Sat Sep 9 23:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.53100000000000003,"15min":0.51400000000000001},"min":{"1min":0.376,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52800000000000002,"15min":0.52000000000000002},"min":{"1min":0.33300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.80200000000000005}]},{"osd":8,"last update":"Sat Sep 9 23:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.66200000000000003,"15min":0.65600000000000003},"min":{"1min":0.441,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.69899999999999995,"15min":0.69499999999999995},"min":{"1min":0.502,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.95499999999999996,"5min":1.218,"15min":1.218},"last":0.76800000000000002}]},{"osd":9,"last update":"Sat Sep 9 23:11:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.70499999999999996}]},{"osd":10,"last update":"Sat Sep 9 23:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.874}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738433,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":418213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 9 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.53100000000000003,"15min":0.51100000000000001},"min":{"1min":0.39700000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.78000000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51400000000000001,"15min":0.5},"min":{"1min":0.374,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71799999999999997,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.49199999999999999}]},{"osd":2,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53700000000000003,"15min":0.52900000000000003},"min":{"1min":0.39000000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78200000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.501},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.40000000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76800000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.55200000000000005}]},{"osd":3,"last update":"Sat Sep 9 23:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.39300000000000002,"15min":0.38900000000000001},"min":{"1min":0.27900000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.41599999999999998,"15min":0.40999999999999998},"min":{"1min":0.33500000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.312}]},{"osd":4,"last update":"Sat Sep 9 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.54900000000000004,"15min":0.53300000000000003},"min":{"1min":0.42899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82999999999999996,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.439},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.375,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.45200000000000001}]},{"osd":5,"last update":"Sat Sep 9 23:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71099999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56799999999999995,"15min":0.56499999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.752,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.46800000000000003}]},{"osd":6,"last update":"Sat Sep 9 23:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.495,"15min":0.497},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.55300000000000005,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.504,"15min":0.501},"min":{"1min":0.33000000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67300000000000004,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.38200000000000001}]},{"osd":7,"last update":"Sat Sep 9 23:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.47599999999999998}]},{"osd":8,"last update":"Sat Sep 9 23:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.68700000000000006,"15min":0.69899999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.78000000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.68200000000000005,"15min":0.68999999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85699999999999998,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.57599999999999996}]},{"osd":9,"last update":"Sat Sep 9 23:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.628,"15min":0.65400000000000003},"min":{"1min":0.36799999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.60999999999999999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.67300000000000004,"15min":0.69799999999999995},"min":{"1min":0.34699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70599999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.47999999999999998}]},{"osd":10,"last update":"Sat Sep 9 23:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.53800000000000003}]},{"osd":11,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.59599999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":474686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52500000000000002,"15min":0.53300000000000003},"min":{"1min":0.42299999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84199999999999997,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.33800000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.751,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.52500000000000002}]},{"osd":1,"last update":"Sat Sep 9 23:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.56499999999999995,"15min":0.53900000000000003},"min":{"1min":0.47799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84199999999999997,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.55800000000000005,"15min":0.51800000000000002},"min":{"1min":0.47499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.68999999999999995}]},{"osd":3,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59999999999999998,"15min":0.60699999999999998},"min":{"1min":0.36199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0009999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61099999999999999,"15min":0.61599999999999999},"min":{"1min":0.46300000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90700000000000003,"5min":1.478,"15min":1.478},"last":0.64500000000000002}]},{"osd":4,"last update":"Sat Sep 9 23:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57299999999999995,"15min":0.57199999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.78000000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.62},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.85899999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.59799999999999998}]},{"osd":5,"last update":"Sat Sep 9 23:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.42999999999999999,"15min":0.40600000000000003},"min":{"1min":0.33800000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.443,"15min":0.42299999999999999},"min":{"1min":0.33100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.495}]},{"osd":6,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.56999999999999995,"15min":0.53200000000000003},"min":{"1min":0.47799999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.749,"5min":0.63300000000000001,"15min":0.59899999999999998},"min":{"1min":0.48799999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.78900000000000003}]},{"osd":7,"last update":"Sat Sep 9 23:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.65700000000000003}]},{"osd":8,"last update":"Sat Sep 9 23:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.59899999999999998,"15min":0.58599999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.753}]},{"osd":9,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.70499999999999996,"15min":0.69099999999999995},"min":{"1min":0.52100000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.752,"15min":0.73699999999999999},"min":{"1min":0.59799999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.76300000000000001}]},{"osd":10,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.77600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":416380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.32700000000000001,"15min":0.313},"min":{"1min":0.22800000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.501,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.314},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35199999999999998,"15min":0.33400000000000002},"min":{"1min":0.249,"5min":0.222,"15min":0.222},"max":{"1min":0.53800000000000003,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.51500000000000001}]},{"osd":1,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54500000000000004,"15min":0.51900000000000002},"min":{"1min":0.32900000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75900000000000001,"5min":1.091,"15min":1.091},"last":0.44},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.51000000000000001,"15min":0.51400000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.60799999999999998,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.47899999999999998}]},{"osd":2,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54400000000000004,"15min":0.52000000000000002},"min":{"1min":0.36099999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65100000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53800000000000003,"15min":0.51800000000000002},"min":{"1min":0.40300000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.66200000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.55500000000000005}]},{"osd":4,"last update":"Sat Sep 9 23:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.53400000000000003,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.71799999999999997,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.54300000000000004,"15min":0.51900000000000002},"min":{"1min":0.36499999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76300000000000001,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.503}]},{"osd":5,"last update":"Sat Sep 9 23:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.57699999999999996,"15min":0.58299999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.70099999999999996,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61899999999999999,"15min":0.60599999999999998},"min":{"1min":0.49399999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.79200000000000004,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.68400000000000005}]},{"osd":6,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.49099999999999999,"15min":0.49199999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73399999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.50900000000000001,"15min":0.50600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64800000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.60299999999999998}]},{"osd":7,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66600000000000004,"15min":0.66400000000000003},"min":{"1min":0.45400000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69299999999999995,"15min":0.69799999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.623}]},{"osd":8,"last update":"Sat Sep 9 23:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.49199999999999999}]},{"osd":9,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.66500000000000004,"15min":0.68899999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77500000000000002,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.67700000000000005,"15min":0.69899999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.746,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.58499999999999996}]},{"osd":10,"last update":"Sat Sep 9 23:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.69999999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]}]},{"osd":4,"up_from":32,"seq":137438953521,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":888832,"data_stored":417125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.503,"15min":0.47099999999999997},"min":{"1min":0.44900000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.52200000000000002,"15min":0.48799999999999999},"min":{"1min":0.41999999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.33300000000000002}]},{"osd":1,"last update":"Sat Sep 9 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.35899999999999999,"15min":0.33800000000000002},"min":{"1min":0.26100000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.52300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.38100000000000001,"15min":0.35699999999999998},"min":{"1min":0.27400000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.68999999999999995,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.51800000000000002}]},{"osd":2,"last update":"Sat Sep 9 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53000000000000003,"15min":0.51400000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.69399999999999995,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.54700000000000004,"15min":0.52200000000000002},"min":{"1min":0.377,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79700000000000004,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.83199999999999996}]},{"osd":3,"last update":"Sat Sep 9 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.52300000000000002,"15min":0.48599999999999999},"min":{"1min":0.47199999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.621},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.51700000000000002,"15min":0.49299999999999999},"min":{"1min":0.39200000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.55000000000000004}]},{"osd":5,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59299999999999997,"15min":0.57699999999999996},"min":{"1min":0.40899999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.59599999999999997,"15min":0.56000000000000005},"min":{"1min":0.42599999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.86099999999999999}]},{"osd":6,"last update":"Sat Sep 9 23:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.63800000000000001,"15min":0.61499999999999999},"min":{"1min":0.56200000000000006,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.749},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.63500000000000001,"15min":0.61299999999999999},"min":{"1min":0.48999999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.60299999999999998}]},{"osd":7,"last update":"Sat Sep 9 23:12:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.83799999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.36699999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72399999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.65300000000000002}]},{"osd":8,"last update":"Sat Sep 9 23:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69199999999999995,"15min":0.68899999999999995},"min":{"1min":0.45700000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.878},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.749,"15min":0.751},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.89500000000000002,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.78600000000000003}]},{"osd":9,"last update":"Sat Sep 9 23:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.79500000000000004}]},{"osd":10,"last update":"Sat Sep 9 23:12:20 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.76500000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]}]},{"osd":5,"up_from":37,"seq":158913789996,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1511424,"data_stored":1009736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.48399999999999999,"15min":0.46300000000000002},"min":{"1min":0.40500000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.502,"15min":0.49299999999999999},"min":{"1min":0.377,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.70199999999999996}]},{"osd":1,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.52600000000000002,"15min":0.50800000000000001},"min":{"1min":0.39000000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.53300000000000003,"15min":0.51500000000000001},"min":{"1min":0.36899999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.54100000000000004}]},{"osd":2,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.38400000000000001,"15min":0.372},"min":{"1min":0.27700000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.66500000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.377},{"interface":"front","average":{"1min":0.432,"5min":0.39400000000000002,"15min":0.38800000000000001},"min":{"1min":0.309,"5min":0.248,"15min":0.248},"max":{"1min":0.64100000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.41899999999999998}]},{"osd":3,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.38600000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.75,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.54600000000000004,"15min":0.52400000000000002},"min":{"1min":0.41799999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86599999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.59799999999999998}]},{"osd":4,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.55900000000000005,"15min":0.53300000000000003},"min":{"1min":0.40799999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.46700000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.67400000000000004}]},{"osd":6,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56000000000000005,"15min":0.54600000000000004},"min":{"1min":0.38300000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88300000000000001,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.437,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92200000000000004,"5min":0.998,"15min":0.998},"last":0.77800000000000002}]},{"osd":7,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.59599999999999997,"15min":0.58799999999999997},"min":{"1min":0.498,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79200000000000004,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82199999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.81000000000000005}]},{"osd":8,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57199999999999995,"15min":0.56399999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83799999999999997,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55200000000000005,"15min":0.54600000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.75600000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.57799999999999996}]},{"osd":9,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.53700000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.82099999999999995,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.70599999999999996,"15min":0.70299999999999996},"min":{"1min":0.56999999999999995,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93899999999999995,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.69099999999999995}]},{"osd":10,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.749},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.83299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":58415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-09T23:13:04.552 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph pg dump --format=json 2023-09-09T23:13:04.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:04 smithi170 ceph-mon[102832]: pgmap v278: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-09T23:13:04.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:04 smithi098 ceph-mon[103124]: pgmap v278: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-09T23:13:05.104 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:13:05.602 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:05 smithi082 ceph-mon[98694]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-09T23:13:05.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:05 smithi170 ceph-mon[102832]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-09T23:13:05.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:05 smithi098 ceph-mon[103124]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-09T23:13:06.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.265+0000 7f36ee618700 1 -- 172.21.15.82:0/3809126845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36e80ffca0 msgr2=0x7f36e8100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:06.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.265+0000 7f36ee618700 1 --2- 172.21.15.82:0/3809126845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36e80ffca0 0x7f36e8100120 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f36d80099c0 tx=0x7f36d8009cd0 comp rx=0 tx=0).stop 2023-09-09T23:13:06.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.265+0000 7f36ee618700 1 -- 172.21.15.82:0/3809126845 shutdown_connections 2023-09-09T23:13:06.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.265+0000 7f36ee618700 1 --2- 172.21.15.82:0/3809126845 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36e81055e0 0x7f36e81059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:06.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.265+0000 7f36ee618700 1 --2- 172.21.15.82:0/3809126845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36e8100660 0x7f36e810c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:06.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.265+0000 7f36ee618700 1 --2- 172.21.15.82:0/3809126845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36e80ffca0 0x7f36e8100120 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:06.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.265+0000 7f36ee618700 1 -- 172.21.15.82:0/3809126845 >> 172.21.15.82:0/3809126845 conn(0x7f36e8075a70 msgr2=0x7f36e80fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:06.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.265+0000 7f36ee618700 1 -- 172.21.15.82:0/3809126845 shutdown_connections 2023-09-09T23:13:06.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.265+0000 7f36ee618700 1 -- 172.21.15.82:0/3809126845 wait complete. 2023-09-09T23:13:06.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.266+0000 7f36ee618700 1 Processor -- start 2023-09-09T23:13:06.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.266+0000 7f36ee618700 1 -- start start 2023-09-09T23:13:06.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.266+0000 7f36ee618700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36e80ffca0 0x7f36e8197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:06.268 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.266+0000 7f36ee618700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36e8100660 0x7f36e8197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:06.268 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.266+0000 7f36ee618700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36e81055e0 0x7f36e81a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:06.268 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.266+0000 7f36ee618700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e8198390 con 0x7f36e8100660 2023-09-09T23:13:06.268 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.266+0000 7f36ee618700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e81a2ad0 con 0x7f36e81055e0 2023-09-09T23:13:06.268 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.266+0000 7f36ee618700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e81a2c90 con 0x7f36e80ffca0 2023-09-09T23:13:06.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.267+0000 7f36e77fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36e8100660 0x7f36e8197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:06.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.267+0000 7f36e77fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36e8100660 0x7f36e8197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55070/0 (socket says 172.21.15.82:55070) 2023-09-09T23:13:06.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.267+0000 7f36e77fe700 1 -- 172.21.15.82:0/2216446144 learned_addr learned my addr 172.21.15.82:0/2216446144 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:06.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.267+0000 7f36ecbb5700 1 --2- 172.21.15.82:0/2216446144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36e81055e0 0x7f36e81a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:06.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.267+0000 7f36e7fff700 1 --2- 172.21.15.82:0/2216446144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36e80ffca0 0x7f36e8197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:06.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.267+0000 7f36e77fe700 1 -- 172.21.15.82:0/2216446144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36e80ffca0 msgr2=0x7f36e8197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:06.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.267+0000 7f36e77fe700 1 --2- 172.21.15.82:0/2216446144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36e80ffca0 0x7f36e8197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:06.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.267+0000 7f36e77fe700 1 -- 172.21.15.82:0/2216446144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36e81055e0 msgr2=0x7f36e81a2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:06.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.267+0000 7f36e77fe700 1 --2- 172.21.15.82:0/2216446144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36e81055e0 0x7f36e81a2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:06.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.267+0000 7f36e77fe700 1 -- 172.21.15.82:0/2216446144 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d80095f0 con 0x7f36e8100660 2023-09-09T23:13:06.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.267+0000 7f36ecbb5700 1 --2- 172.21.15.82:0/2216446144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36e81055e0 0x7f36e81a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:06.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.267+0000 7f36e77fe700 1 --2- 172.21.15.82:0/2216446144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36e8100660 0x7f36e8197a50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009990 tx=0x7f36d8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:06.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.267+0000 7f36e57fa700 1 -- 172.21.15.82:0/2216446144 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d801d070 con 0x7f36e8100660 2023-09-09T23:13:06.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.267+0000 7f36e57fa700 1 -- 172.21.15.82:0/2216446144 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36d8004010 con 0x7f36e8100660 2023-09-09T23:13:06.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.267+0000 7f36e57fa700 1 -- 172.21.15.82:0/2216446144 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8022d40 con 0x7f36e8100660 2023-09-09T23:13:06.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.267+0000 7f36ee618700 1 -- 172.21.15.82:0/2216446144 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e81a2f10 con 0x7f36e8100660 2023-09-09T23:13:06.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.267+0000 7f36ee618700 1 -- 172.21.15.82:0/2216446144 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e81a32c0 con 0x7f36e8100660 2023-09-09T23:13:06.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.268+0000 7f36ee618700 1 -- 172.21.15.82:0/2216446144 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e804e300 con 0x7f36e8100660 2023-09-09T23:13:06.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.269+0000 7f36e57fa700 1 -- 172.21.15.82:0/2216446144 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f36d8022470 con 0x7f36e8100660 2023-09-09T23:13:06.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.270+0000 7f36e57fa700 1 --2- 172.21.15.82:0/2216446144 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f36d40a0750 0x7f36d40a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:06.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.270+0000 7f36e7fff700 1 --2- 172.21.15.82:0/2216446144 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f36d40a0750 0x7f36d40a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:06.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.270+0000 7f36e57fa700 1 -- 172.21.15.82:0/2216446144 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f36d80ccf20 con 0x7f36e8100660 2023-09-09T23:13:06.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.270+0000 7f36e7fff700 1 --2- 172.21.15.82:0/2216446144 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f36d40a0750 0x7f36d40a2c10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f36d0005950 tx=0x7f36d00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:06.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.274+0000 7f36e57fa700 1 -- 172.21.15.82:0/2216446144 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f36d8096b30 con 0x7f36e8100660 2023-09-09T23:13:06.454 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:06 smithi082 ceph-mon[98694]: pgmap v279: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2023-09-09T23:13:06.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.463+0000 7f36ee618700 1 -- 172.21.15.82:0/2216446144 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f36e810f7b0 con 0x7f36d40a0750 2023-09-09T23:13:06.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.473+0000 7f36e57fa700 1 -- 172.21.15.82:0/2216446144 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340433 (secure 0 0 0) 0x7f36e810f7b0 con 0x7f36d40a0750 2023-09-09T23:13:06.475 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:06.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.476+0000 7f36ee618700 1 -- 172.21.15.82:0/2216446144 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f36d40a0750 msgr2=0x7f36d40a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:06.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.476+0000 7f36ee618700 1 --2- 172.21.15.82:0/2216446144 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f36d40a0750 0x7f36d40a2c10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f36d0005950 tx=0x7f36d00058e0 comp rx=0 tx=0).stop 2023-09-09T23:13:06.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.476+0000 7f36ee618700 1 -- 172.21.15.82:0/2216446144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36e8100660 msgr2=0x7f36e8197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:06.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.476+0000 7f36ee618700 1 --2- 172.21.15.82:0/2216446144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36e8100660 0x7f36e8197a50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009990 tx=0x7f36d8009580 comp rx=0 tx=0).stop 2023-09-09T23:13:06.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.477+0000 7f36ee618700 1 -- 172.21.15.82:0/2216446144 shutdown_connections 2023-09-09T23:13:06.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.477+0000 7f36ee618700 1 --2- 172.21.15.82:0/2216446144 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f36d40a0750 0x7f36d40a2c10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:06.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.477+0000 7f36ee618700 1 --2- 172.21.15.82:0/2216446144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36e81055e0 0x7f36e81a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:06.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.477+0000 7f36ee618700 1 --2- 172.21.15.82:0/2216446144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36e80ffca0 0x7f36e8197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:06.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.477+0000 7f36ee618700 1 --2- 172.21.15.82:0/2216446144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36e8100660 0x7f36e8197a50 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:06.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.477+0000 7f36ee618700 1 -- 172.21.15.82:0/2216446144 >> 172.21.15.82:0/2216446144 conn(0x7f36e8075a70 msgr2=0x7f36e80fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:06.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.477+0000 7f36ee618700 1 -- 172.21.15.82:0/2216446144 shutdown_connections 2023-09-09T23:13:06.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:06.477+0000 7f36ee618700 1 -- 172.21.15.82:0/2216446144 wait complete. 2023-09-09T23:13:06.480 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-09-09T23:13:06.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:06 smithi170 ceph-mon[102832]: pgmap v279: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2023-09-09T23:13:06.747 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":279,"stamp":"2023-09-09T23:13:04.989435+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654105,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":562,"num_read_kb":397,"num_write":413,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336664,"kb_used_data":12472,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124605160,"statfs":{"total":1151940427776,"available":1151595683840,"internally_reserved":0,"allocated":12771328,"data_stored":6954073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":52,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003017"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.988649+0000","last_change":"2023-09-09T23:12:17.903811+0000","last_active":"2023-09-09T23:12:18.988649+0000","last_peered":"2023-09-09T23:12:18.988649+0000","last_clean":"2023-09-09T23:12:18.988649+0000","last_became_active":"2023-09-09T23:12:17.903386+0000","last_became_peered":"2023-09-09T23:12:17.903386+0000","last_unstale":"2023-09-09T23:12:18.988649+0000","last_undegraded":"2023-09-09T23:12:18.988649+0000","last_fullsized":"2023-09-09T23:12:18.988649+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:36:45.122068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.485007+0000","last_change":"2023-09-09T23:12:17.897915+0000","last_active":"2023-09-09T23:12:19.485007+0000","last_peered":"2023-09-09T23:12:19.485007+0000","last_clean":"2023-09-09T23:12:19.485007+0000","last_became_active":"2023-09-09T23:12:17.896727+0000","last_became_peered":"2023-09-09T23:12:17.896727+0000","last_unstale":"2023-09-09T23:12:19.485007+0000","last_undegraded":"2023-09-09T23:12:19.485007+0000","last_fullsized":"2023-09-09T23:12:19.485007+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:36:03.053572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.485083+0000","last_change":"2023-09-09T23:12:17.907957+0000","last_active":"2023-09-09T23:12:19.485083+0000","last_peered":"2023-09-09T23:12:19.485083+0000","last_clean":"2023-09-09T23:12:19.485083+0000","last_became_active":"2023-09-09T23:12:17.907625+0000","last_became_peered":"2023-09-09T23:12:17.907625+0000","last_unstale":"2023-09-09T23:12:19.485083+0000","last_undegraded":"2023-09-09T23:12:19.485083+0000","last_fullsized":"2023-09-09T23:12:19.485083+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:26:56.520987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.045364+0000","last_change":"2023-09-09T23:12:17.903489+0000","last_active":"2023-09-09T23:12:20.045364+0000","last_peered":"2023-09-09T23:12:20.045364+0000","last_clean":"2023-09-09T23:12:20.045364+0000","last_became_active":"2023-09-09T23:12:17.901184+0000","last_became_peered":"2023-09-09T23:12:17.901184+0000","last_unstale":"2023-09-09T23:12:20.045364+0000","last_undegraded":"2023-09-09T23:12:20.045364+0000","last_fullsized":"2023-09-09T23:12:20.045364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:42:01.899998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897810+0000","last_change":"2023-09-09T23:12:17.899487+0000","last_active":"2023-09-09T23:12:18.897810+0000","last_peered":"2023-09-09T23:12:18.897810+0000","last_clean":"2023-09-09T23:12:18.897810+0000","last_became_active":"2023-09-09T23:12:17.899083+0000","last_became_peered":"2023-09-09T23:12:17.899083+0000","last_unstale":"2023-09-09T23:12:18.897810+0000","last_undegraded":"2023-09-09T23:12:18.897810+0000","last_fullsized":"2023-09-09T23:12:18.897810+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:15:20.109849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690024+0000","last_change":"2023-09-09T23:12:17.901554+0000","last_active":"2023-09-09T23:12:20.690024+0000","last_peered":"2023-09-09T23:12:20.690024+0000","last_clean":"2023-09-09T23:12:20.690024+0000","last_became_active":"2023-09-09T23:12:17.901246+0000","last_became_peered":"2023-09-09T23:12:17.901246+0000","last_unstale":"2023-09-09T23:12:20.690024+0000","last_undegraded":"2023-09-09T23:12:20.690024+0000","last_fullsized":"2023-09-09T23:12:20.690024+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:19:08.795927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689639+0000","last_change":"2023-09-09T23:12:17.906989+0000","last_active":"2023-09-09T23:12:20.689639+0000","last_peered":"2023-09-09T23:12:20.689639+0000","last_clean":"2023-09-09T23:12:20.689639+0000","last_became_active":"2023-09-09T23:12:17.906704+0000","last_became_peered":"2023-09-09T23:12:17.906704+0000","last_unstale":"2023-09-09T23:12:20.689639+0000","last_undegraded":"2023-09-09T23:12:20.689639+0000","last_fullsized":"2023-09-09T23:12:20.689639+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:41:33.612168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897513+0000","last_change":"2023-09-09T23:12:17.906269+0000","last_active":"2023-09-09T23:12:18.897513+0000","last_peered":"2023-09-09T23:12:18.897513+0000","last_clean":"2023-09-09T23:12:18.897513+0000","last_became_active":"2023-09-09T23:12:17.906017+0000","last_became_peered":"2023-09-09T23:12:17.906017+0000","last_unstale":"2023-09-09T23:12:18.897513+0000","last_undegraded":"2023-09-09T23:12:18.897513+0000","last_fullsized":"2023-09-09T23:12:18.897513+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:50:21.893506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.520143+0000","last_change":"2023-09-09T23:12:17.905191+0000","last_active":"2023-09-09T23:12:19.520143+0000","last_peered":"2023-09-09T23:12:19.520143+0000","last_clean":"2023-09-09T23:12:19.520143+0000","last_became_active":"2023-09-09T23:12:17.904813+0000","last_became_peered":"2023-09-09T23:12:17.904813+0000","last_unstale":"2023-09-09T23:12:19.520143+0000","last_undegraded":"2023-09-09T23:12:19.520143+0000","last_fullsized":"2023-09-09T23:12:19.520143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:29:54.722210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690667+0000","last_change":"2023-09-09T23:12:17.899980+0000","last_active":"2023-09-09T23:12:20.690667+0000","last_peered":"2023-09-09T23:12:20.690667+0000","last_clean":"2023-09-09T23:12:20.690667+0000","last_became_active":"2023-09-09T23:12:17.899692+0000","last_became_peered":"2023-09-09T23:12:17.899692+0000","last_unstale":"2023-09-09T23:12:20.690667+0000","last_undegraded":"2023-09-09T23:12:20.690667+0000","last_fullsized":"2023-09-09T23:12:20.690667+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:25:28.056684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690090+0000","last_change":"2023-09-09T23:12:17.907010+0000","last_active":"2023-09-09T23:12:20.690090+0000","last_peered":"2023-09-09T23:12:20.690090+0000","last_clean":"2023-09-09T23:12:20.690090+0000","last_became_active":"2023-09-09T23:12:17.906565+0000","last_became_peered":"2023-09-09T23:12:17.906565+0000","last_unstale":"2023-09-09T23:12:20.690090+0000","last_undegraded":"2023-09-09T23:12:20.690090+0000","last_fullsized":"2023-09-09T23:12:20.690090+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:08:03.733608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.045231+0000","last_change":"2023-09-09T23:12:17.902889+0000","last_active":"2023-09-09T23:12:20.045231+0000","last_peered":"2023-09-09T23:12:20.045231+0000","last_clean":"2023-09-09T23:12:20.045231+0000","last_became_active":"2023-09-09T23:12:17.900747+0000","last_became_peered":"2023-09-09T23:12:17.900747+0000","last_unstale":"2023-09-09T23:12:20.045231+0000","last_undegraded":"2023-09-09T23:12:20.045231+0000","last_fullsized":"2023-09-09T23:12:20.045231+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:01:29.195275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689191+0000","last_change":"2023-09-09T23:12:17.897844+0000","last_active":"2023-09-09T23:12:20.689191+0000","last_peered":"2023-09-09T23:12:20.689191+0000","last_clean":"2023-09-09T23:12:20.689191+0000","last_became_active":"2023-09-09T23:12:17.897529+0000","last_became_peered":"2023-09-09T23:12:17.897529+0000","last_unstale":"2023-09-09T23:12:20.689191+0000","last_undegraded":"2023-09-09T23:12:20.689191+0000","last_fullsized":"2023-09-09T23:12:20.689191+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:37:37.425752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.484862+0000","last_change":"2023-09-09T23:12:17.899255+0000","last_active":"2023-09-09T23:12:19.484862+0000","last_peered":"2023-09-09T23:12:19.484862+0000","last_clean":"2023-09-09T23:12:19.484862+0000","last_became_active":"2023-09-09T23:12:17.898932+0000","last_became_peered":"2023-09-09T23:12:17.898932+0000","last_unstale":"2023-09-09T23:12:19.484862+0000","last_undegraded":"2023-09-09T23:12:19.484862+0000","last_fullsized":"2023-09-09T23:12:19.484862+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:10:56.864588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.484825+0000","last_change":"2023-09-09T23:12:17.901083+0000","last_active":"2023-09-09T23:12:19.484825+0000","last_peered":"2023-09-09T23:12:19.484825+0000","last_clean":"2023-09-09T23:12:19.484825+0000","last_became_active":"2023-09-09T23:12:17.900744+0000","last_became_peered":"2023-09-09T23:12:17.900744+0000","last_unstale":"2023-09-09T23:12:19.484825+0000","last_undegraded":"2023-09-09T23:12:19.484825+0000","last_fullsized":"2023-09-09T23:12:19.484825+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:37:11.073786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.519605+0000","last_change":"2023-09-09T23:12:17.901962+0000","last_active":"2023-09-09T23:12:19.519605+0000","last_peered":"2023-09-09T23:12:19.519605+0000","last_clean":"2023-09-09T23:12:19.519605+0000","last_became_active":"2023-09-09T23:12:17.901638+0000","last_became_peered":"2023-09-09T23:12:17.901638+0000","last_unstale":"2023-09-09T23:12:19.519605+0000","last_undegraded":"2023-09-09T23:12:19.519605+0000","last_fullsized":"2023-09-09T23:12:19.519605+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:34:52.083262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.988721+0000","last_change":"2023-09-09T23:12:17.904424+0000","last_active":"2023-09-09T23:12:18.988721+0000","last_peered":"2023-09-09T23:12:18.988721+0000","last_clean":"2023-09-09T23:12:18.988721+0000","last_became_active":"2023-09-09T23:12:17.903879+0000","last_became_peered":"2023-09-09T23:12:17.903879+0000","last_unstale":"2023-09-09T23:12:18.988721+0000","last_undegraded":"2023-09-09T23:12:18.988721+0000","last_fullsized":"2023-09-09T23:12:18.988721+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:56:42.468630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898791+0000","last_change":"2023-09-09T23:12:17.905065+0000","last_active":"2023-09-09T23:12:18.898791+0000","last_peered":"2023-09-09T23:12:18.898791+0000","last_clean":"2023-09-09T23:12:18.898791+0000","last_became_active":"2023-09-09T23:12:17.904646+0000","last_became_peered":"2023-09-09T23:12:17.904646+0000","last_unstale":"2023-09-09T23:12:18.898791+0000","last_undegraded":"2023-09-09T23:12:18.898791+0000","last_fullsized":"2023-09-09T23:12:18.898791+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:11:14.705884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690263+0000","last_change":"2023-09-09T23:12:17.899751+0000","last_active":"2023-09-09T23:12:20.690263+0000","last_peered":"2023-09-09T23:12:20.690263+0000","last_clean":"2023-09-09T23:12:20.690263+0000","last_became_active":"2023-09-09T23:12:17.899410+0000","last_became_peered":"2023-09-09T23:12:17.899410+0000","last_unstale":"2023-09-09T23:12:20.690263+0000","last_undegraded":"2023-09-09T23:12:20.690263+0000","last_fullsized":"2023-09-09T23:12:20.690263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:23:06.098276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":63,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:59.350604+0000","last_change":"2023-09-09T23:12:17.175671+0000","last_active":"2023-09-09T23:12:59.350604+0000","last_peered":"2023-09-09T23:12:59.350604+0000","last_clean":"2023-09-09T23:12:59.350604+0000","last_became_active":"2023-09-09T23:12:17.175327+0000","last_became_peered":"2023-09-09T23:12:17.175327+0000","last_unstale":"2023-09-09T23:12:59.350604+0000","last_undegraded":"2023-09-09T23:12:59.350604+0000","last_fullsized":"2023-09-09T23:12:59.350604+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:38:57.724260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898994+0000","last_change":"2023-09-09T23:12:17.906338+0000","last_active":"2023-09-09T23:12:18.898994+0000","last_peered":"2023-09-09T23:12:18.898994+0000","last_clean":"2023-09-09T23:12:18.898994+0000","last_became_active":"2023-09-09T23:12:17.906094+0000","last_became_peered":"2023-09-09T23:12:17.906094+0000","last_unstale":"2023-09-09T23:12:18.898994+0000","last_undegraded":"2023-09-09T23:12:18.898994+0000","last_fullsized":"2023-09-09T23:12:18.898994+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:43:18.183144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.899063+0000","last_change":"2023-09-09T23:12:17.167763+0000","last_active":"2023-09-09T23:12:18.899063+0000","last_peered":"2023-09-09T23:12:18.899063+0000","last_clean":"2023-09-09T23:12:18.899063+0000","last_became_active":"2023-09-09T23:12:17.167455+0000","last_became_peered":"2023-09-09T23:12:17.167455+0000","last_unstale":"2023-09-09T23:12:18.899063+0000","last_undegraded":"2023-09-09T23:12:18.899063+0000","last_fullsized":"2023-09-09T23:12:18.899063+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:15:23.415792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.485136+0000","last_change":"2023-09-09T23:12:17.907936+0000","last_active":"2023-09-09T23:12:19.485136+0000","last_peered":"2023-09-09T23:12:19.485136+0000","last_clean":"2023-09-09T23:12:19.485136+0000","last_became_active":"2023-09-09T23:12:17.907588+0000","last_became_peered":"2023-09-09T23:12:17.907588+0000","last_unstale":"2023-09-09T23:12:19.485136+0000","last_undegraded":"2023-09-09T23:12:19.485136+0000","last_fullsized":"2023-09-09T23:12:19.485136+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:57:30.910812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:25.294384+0000","last_change":"2023-09-09T23:12:17.172481+0000","last_active":"2023-09-09T23:12:25.294384+0000","last_peered":"2023-09-09T23:12:25.294384+0000","last_clean":"2023-09-09T23:12:25.294384+0000","last_became_active":"2023-09-09T23:12:17.172178+0000","last_became_peered":"2023-09-09T23:12:17.172178+0000","last_unstale":"2023-09-09T23:12:25.294384+0000","last_undegraded":"2023-09-09T23:12:25.294384+0000","last_fullsized":"2023-09-09T23:12:25.294384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:08:55.328423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.519862+0000","last_change":"2023-09-09T23:12:17.900155+0000","last_active":"2023-09-09T23:12:19.519862+0000","last_peered":"2023-09-09T23:12:19.519862+0000","last_clean":"2023-09-09T23:12:19.519862+0000","last_became_active":"2023-09-09T23:12:17.899862+0000","last_became_peered":"2023-09-09T23:12:17.899862+0000","last_unstale":"2023-09-09T23:12:19.519862+0000","last_undegraded":"2023-09-09T23:12:19.519862+0000","last_fullsized":"2023-09-09T23:12:19.519862+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:57:14.949174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:24.279082+0000","last_change":"2023-09-09T23:12:17.171890+0000","last_active":"2023-09-09T23:12:24.279082+0000","last_peered":"2023-09-09T23:12:24.279082+0000","last_clean":"2023-09-09T23:12:24.279082+0000","last_became_active":"2023-09-09T23:12:17.171496+0000","last_became_peered":"2023-09-09T23:12:17.171496+0000","last_unstale":"2023-09-09T23:12:24.279082+0000","last_undegraded":"2023-09-09T23:12:24.279082+0000","last_fullsized":"2023-09-09T23:12:24.279082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:38:25.704989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689810+0000","last_change":"2023-09-09T23:12:17.901067+0000","last_active":"2023-09-09T23:12:20.689810+0000","last_peered":"2023-09-09T23:12:20.689810+0000","last_clean":"2023-09-09T23:12:20.689810+0000","last_became_active":"2023-09-09T23:12:17.900069+0000","last_became_peered":"2023-09-09T23:12:17.900069+0000","last_unstale":"2023-09-09T23:12:20.689810+0000","last_undegraded":"2023-09-09T23:12:20.689810+0000","last_fullsized":"2023-09-09T23:12:20.689810+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:59:55.356235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":63,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:13:02.327247+0000","last_change":"2023-09-09T23:12:17.169474+0000","last_active":"2023-09-09T23:13:02.327247+0000","last_peered":"2023-09-09T23:13:02.327247+0000","last_clean":"2023-09-09T23:13:02.327247+0000","last_became_active":"2023-09-09T23:12:17.169186+0000","last_became_peered":"2023-09-09T23:12:17.169186+0000","last_unstale":"2023-09-09T23:13:02.327247+0000","last_undegraded":"2023-09-09T23:13:02.327247+0000","last_fullsized":"2023-09-09T23:13:02.327247+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:47:13.027949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897942+0000","last_change":"2023-09-09T23:12:17.901605+0000","last_active":"2023-09-09T23:12:18.897942+0000","last_peered":"2023-09-09T23:12:18.897942+0000","last_clean":"2023-09-09T23:12:18.897942+0000","last_became_active":"2023-09-09T23:12:17.901287+0000","last_became_peered":"2023-09-09T23:12:17.901287+0000","last_unstale":"2023-09-09T23:12:18.897942+0000","last_undegraded":"2023-09-09T23:12:18.897942+0000","last_fullsized":"2023-09-09T23:12:18.897942+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:16:41.570750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:23.271624+0000","last_change":"2023-09-09T23:12:17.171797+0000","last_active":"2023-09-09T23:12:23.271624+0000","last_peered":"2023-09-09T23:12:23.271624+0000","last_clean":"2023-09-09T23:12:23.271624+0000","last_became_active":"2023-09-09T23:12:17.171521+0000","last_became_peered":"2023-09-09T23:12:17.171521+0000","last_unstale":"2023-09-09T23:12:23.271624+0000","last_undegraded":"2023-09-09T23:12:23.271624+0000","last_fullsized":"2023-09-09T23:12:23.271624+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:07:51.951393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690229+0000","last_change":"2023-09-09T23:12:17.900284+0000","last_active":"2023-09-09T23:12:20.690229+0000","last_peered":"2023-09-09T23:12:20.690229+0000","last_clean":"2023-09-09T23:12:20.690229+0000","last_became_active":"2023-09-09T23:12:17.899975+0000","last_became_peered":"2023-09-09T23:12:17.899975+0000","last_unstale":"2023-09-09T23:12:20.690229+0000","last_undegraded":"2023-09-09T23:12:20.690229+0000","last_fullsized":"2023-09-09T23:12:20.690229+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:10:56.723338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:22.583122+0000","last_change":"2023-09-09T23:12:17.175328+0000","last_active":"2023-09-09T23:12:22.583122+0000","last_peered":"2023-09-09T23:12:22.583122+0000","last_clean":"2023-09-09T23:12:22.583122+0000","last_became_active":"2023-09-09T23:12:17.174977+0000","last_became_peered":"2023-09-09T23:12:17.174977+0000","last_unstale":"2023-09-09T23:12:22.583122+0000","last_undegraded":"2023-09-09T23:12:22.583122+0000","last_fullsized":"2023-09-09T23:12:22.583122+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:01:36.272333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897568+0000","last_change":"2023-09-09T23:12:17.902982+0000","last_active":"2023-09-09T23:12:18.897568+0000","last_peered":"2023-09-09T23:12:18.897568+0000","last_clean":"2023-09-09T23:12:18.897568+0000","last_became_active":"2023-09-09T23:12:17.902677+0000","last_became_peered":"2023-09-09T23:12:17.902677+0000","last_unstale":"2023-09-09T23:12:18.897568+0000","last_undegraded":"2023-09-09T23:12:18.897568+0000","last_fullsized":"2023-09-09T23:12:18.897568+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:08:31.507899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:24.280201+0000","last_change":"2023-09-09T23:12:17.170099+0000","last_active":"2023-09-09T23:12:24.280201+0000","last_peered":"2023-09-09T23:12:24.280201+0000","last_clean":"2023-09-09T23:12:24.280201+0000","last_became_active":"2023-09-09T23:12:17.169827+0000","last_became_peered":"2023-09-09T23:12:17.169827+0000","last_unstale":"2023-09-09T23:12:24.280201+0000","last_undegraded":"2023-09-09T23:12:24.280201+0000","last_fullsized":"2023-09-09T23:12:24.280201+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:16:34.693568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898646+0000","last_change":"2023-09-09T23:12:17.903735+0000","last_active":"2023-09-09T23:12:18.898646+0000","last_peered":"2023-09-09T23:12:18.898646+0000","last_clean":"2023-09-09T23:12:18.898646+0000","last_became_active":"2023-09-09T23:12:17.903410+0000","last_became_peered":"2023-09-09T23:12:17.903410+0000","last_unstale":"2023-09-09T23:12:18.898646+0000","last_undegraded":"2023-09-09T23:12:18.898646+0000","last_fullsized":"2023-09-09T23:12:18.898646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:00:47.664613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":93,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:13:01.325026+0000","last_change":"2023-09-09T23:12:17.176004+0000","last_active":"2023-09-09T23:13:01.325026+0000","last_peered":"2023-09-09T23:13:01.325026+0000","last_clean":"2023-09-09T23:13:01.325026+0000","last_became_active":"2023-09-09T23:12:17.175514+0000","last_became_peered":"2023-09-09T23:12:17.175514+0000","last_unstale":"2023-09-09T23:13:01.325026+0000","last_undegraded":"2023-09-09T23:13:01.325026+0000","last_fullsized":"2023-09-09T23:13:01.325026+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:43:54.587724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.520135+0000","last_change":"2023-09-09T23:12:17.900943+0000","last_active":"2023-09-09T23:12:19.520135+0000","last_peered":"2023-09-09T23:12:19.520135+0000","last_clean":"2023-09-09T23:12:19.520135+0000","last_became_active":"2023-09-09T23:12:17.900662+0000","last_became_peered":"2023-09-09T23:12:17.900662+0000","last_unstale":"2023-09-09T23:12:19.520135+0000","last_undegraded":"2023-09-09T23:12:19.520135+0000","last_fullsized":"2023-09-09T23:12:19.520135+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:39:41.480033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:24.280105+0000","last_change":"2023-09-09T23:12:17.172458+0000","last_active":"2023-09-09T23:12:24.280105+0000","last_peered":"2023-09-09T23:12:24.280105+0000","last_clean":"2023-09-09T23:12:24.280105+0000","last_became_active":"2023-09-09T23:12:17.172119+0000","last_became_peered":"2023-09-09T23:12:17.172119+0000","last_unstale":"2023-09-09T23:12:24.280105+0000","last_undegraded":"2023-09-09T23:12:24.280105+0000","last_fullsized":"2023-09-09T23:12:24.280105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:31:27.821245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.485149+0000","last_change":"2023-09-09T23:12:17.903252+0000","last_active":"2023-09-09T23:12:19.485149+0000","last_peered":"2023-09-09T23:12:19.485149+0000","last_clean":"2023-09-09T23:12:19.485149+0000","last_became_active":"2023-09-09T23:12:17.902466+0000","last_became_peered":"2023-09-09T23:12:17.902466+0000","last_unstale":"2023-09-09T23:12:19.485149+0000","last_undegraded":"2023-09-09T23:12:19.485149+0000","last_fullsized":"2023-09-09T23:12:19.485149+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:28:07.535937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:23.264739+0000","last_change":"2023-09-09T23:12:17.165106+0000","last_active":"2023-09-09T23:12:23.264739+0000","last_peered":"2023-09-09T23:12:23.264739+0000","last_clean":"2023-09-09T23:12:23.264739+0000","last_became_active":"2023-09-09T23:12:17.164783+0000","last_became_peered":"2023-09-09T23:12:17.164783+0000","last_unstale":"2023-09-09T23:12:23.264739+0000","last_undegraded":"2023-09-09T23:12:23.264739+0000","last_fullsized":"2023-09-09T23:12:23.264739+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:02:14.984870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.519952+0000","last_change":"2023-09-09T23:12:17.899473+0000","last_active":"2023-09-09T23:12:19.519952+0000","last_peered":"2023-09-09T23:12:19.519952+0000","last_clean":"2023-09-09T23:12:19.519952+0000","last_became_active":"2023-09-09T23:12:17.899142+0000","last_became_peered":"2023-09-09T23:12:17.899142+0000","last_unstale":"2023-09-09T23:12:19.519952+0000","last_undegraded":"2023-09-09T23:12:19.519952+0000","last_fullsized":"2023-09-09T23:12:19.519952+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:50:30.261873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'29","reported_seq":128,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:13:01.329027+0000","last_change":"2023-09-09T23:12:17.172560+0000","last_active":"2023-09-09T23:13:01.329027+0000","last_peered":"2023-09-09T23:13:01.329027+0000","last_clean":"2023-09-09T23:13:01.329027+0000","last_became_active":"2023-09-09T23:12:17.172213+0000","last_became_peered":"2023-09-09T23:12:17.172213+0000","last_unstale":"2023-09-09T23:13:01.329027+0000","last_undegraded":"2023-09-09T23:13:01.329027+0000","last_fullsized":"2023-09-09T23:13:01.329027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:30:00.206901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56289,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":86,"num_read_kb":97,"num_write":31,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898030+0000","last_change":"2023-09-09T23:12:17.898279+0000","last_active":"2023-09-09T23:12:18.898030+0000","last_peered":"2023-09-09T23:12:18.898030+0000","last_clean":"2023-09-09T23:12:18.898030+0000","last_became_active":"2023-09-09T23:12:17.898041+0000","last_became_peered":"2023-09-09T23:12:17.898041+0000","last_unstale":"2023-09-09T23:12:18.898030+0000","last_undegraded":"2023-09-09T23:12:18.898030+0000","last_fullsized":"2023-09-09T23:12:18.898030+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:01:10.315445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:24.277670+0000","last_change":"2023-09-09T23:12:17.172079+0000","last_active":"2023-09-09T23:12:24.277670+0000","last_peered":"2023-09-09T23:12:24.277670+0000","last_clean":"2023-09-09T23:12:24.277670+0000","last_became_active":"2023-09-09T23:12:17.171779+0000","last_became_peered":"2023-09-09T23:12:17.171779+0000","last_unstale":"2023-09-09T23:12:24.277670+0000","last_undegraded":"2023-09-09T23:12:24.277670+0000","last_fullsized":"2023-09-09T23:12:24.277670+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:49:50.287124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.688927+0000","last_change":"2023-09-09T23:12:17.904087+0000","last_active":"2023-09-09T23:12:20.688927+0000","last_peered":"2023-09-09T23:12:20.688927+0000","last_clean":"2023-09-09T23:12:20.688927+0000","last_became_active":"2023-09-09T23:12:17.903670+0000","last_became_peered":"2023-09-09T23:12:17.903670+0000","last_unstale":"2023-09-09T23:12:20.688927+0000","last_undegraded":"2023-09-09T23:12:20.688927+0000","last_fullsized":"2023-09-09T23:12:20.688927+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:24:04.403871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:24.280209+0000","last_change":"2023-09-09T23:12:17.170594+0000","last_active":"2023-09-09T23:12:24.280209+0000","last_peered":"2023-09-09T23:12:24.280209+0000","last_clean":"2023-09-09T23:12:24.280209+0000","last_became_active":"2023-09-09T23:12:17.170223+0000","last_became_peered":"2023-09-09T23:12:17.170223+0000","last_unstale":"2023-09-09T23:12:24.280209+0000","last_undegraded":"2023-09-09T23:12:24.280209+0000","last_fullsized":"2023-09-09T23:12:24.280209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:30:09.222947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689977+0000","last_change":"2023-09-09T23:12:17.901047+0000","last_active":"2023-09-09T23:12:20.689977+0000","last_peered":"2023-09-09T23:12:20.689977+0000","last_clean":"2023-09-09T23:12:20.689977+0000","last_became_active":"2023-09-09T23:12:17.900129+0000","last_became_peered":"2023-09-09T23:12:17.900129+0000","last_unstale":"2023-09-09T23:12:20.689977+0000","last_undegraded":"2023-09-09T23:12:20.689977+0000","last_fullsized":"2023-09-09T23:12:20.689977+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:59:36.338392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'6","reported_seq":63,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:13:02.326695+0000","last_change":"2023-09-09T23:12:17.169495+0000","last_active":"2023-09-09T23:13:02.326695+0000","last_peered":"2023-09-09T23:13:02.326695+0000","last_clean":"2023-09-09T23:13:02.326695+0000","last_became_active":"2023-09-09T23:12:17.169170+0000","last_became_peered":"2023-09-09T23:12:17.169170+0000","last_unstale":"2023-09-09T23:13:02.326695+0000","last_undegraded":"2023-09-09T23:13:02.326695+0000","last_fullsized":"2023-09-09T23:13:02.326695+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:36:23.765574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":45,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689415+0000","last_change":"2023-09-09T23:12:17.907826+0000","last_active":"2023-09-09T23:12:20.689415+0000","last_peered":"2023-09-09T23:12:20.689415+0000","last_clean":"2023-09-09T23:12:20.689415+0000","last_became_active":"2023-09-09T23:12:17.907526+0000","last_became_peered":"2023-09-09T23:12:17.907526+0000","last_unstale":"2023-09-09T23:12:20.689415+0000","last_undegraded":"2023-09-09T23:12:20.689415+0000","last_fullsized":"2023-09-09T23:12:20.689415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:17:20.587253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:23.270744+0000","last_change":"2023-09-09T23:12:17.170139+0000","last_active":"2023-09-09T23:12:23.270744+0000","last_peered":"2023-09-09T23:12:23.270744+0000","last_clean":"2023-09-09T23:12:23.270744+0000","last_became_active":"2023-09-09T23:12:17.169797+0000","last_became_peered":"2023-09-09T23:12:17.169797+0000","last_unstale":"2023-09-09T23:12:23.270744+0000","last_undegraded":"2023-09-09T23:12:23.270744+0000","last_fullsized":"2023-09-09T23:12:23.270744+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:59:44.668114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689753+0000","last_change":"2023-09-09T23:12:17.902567+0000","last_active":"2023-09-09T23:12:20.689753+0000","last_peered":"2023-09-09T23:12:20.689753+0000","last_clean":"2023-09-09T23:12:20.689753+0000","last_became_active":"2023-09-09T23:12:17.902229+0000","last_became_peered":"2023-09-09T23:12:17.902229+0000","last_unstale":"2023-09-09T23:12:20.689753+0000","last_undegraded":"2023-09-09T23:12:20.689753+0000","last_fullsized":"2023-09-09T23:12:20.689753+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:49:50.049925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:23.260243+0000","last_change":"2023-09-09T23:12:17.171143+0000","last_active":"2023-09-09T23:12:23.260243+0000","last_peered":"2023-09-09T23:12:23.260243+0000","last_clean":"2023-09-09T23:12:23.260243+0000","last_became_active":"2023-09-09T23:12:17.170847+0000","last_became_peered":"2023-09-09T23:12:17.170847+0000","last_unstale":"2023-09-09T23:12:23.260243+0000","last_undegraded":"2023-09-09T23:12:23.260243+0000","last_fullsized":"2023-09-09T23:12:23.260243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:35:55.570394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.988514+0000","last_change":"2023-09-09T23:12:17.900284+0000","last_active":"2023-09-09T23:12:18.988514+0000","last_peered":"2023-09-09T23:12:18.988514+0000","last_clean":"2023-09-09T23:12:18.988514+0000","last_became_active":"2023-09-09T23:12:17.899923+0000","last_became_peered":"2023-09-09T23:12:17.899923+0000","last_unstale":"2023-09-09T23:12:18.988514+0000","last_undegraded":"2023-09-09T23:12:18.988514+0000","last_fullsized":"2023-09-09T23:12:18.988514+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:32:32.230968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898542+0000","last_change":"2023-09-09T23:12:17.176313+0000","last_active":"2023-09-09T23:12:18.898542+0000","last_peered":"2023-09-09T23:12:18.898542+0000","last_clean":"2023-09-09T23:12:18.898542+0000","last_became_active":"2023-09-09T23:12:17.175934+0000","last_became_peered":"2023-09-09T23:12:17.175934+0000","last_unstale":"2023-09-09T23:12:18.898542+0000","last_undegraded":"2023-09-09T23:12:18.898542+0000","last_fullsized":"2023-09-09T23:12:18.898542+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:33:15.044986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:21.932198+0000","last_change":"2023-09-09T23:12:17.900284+0000","last_active":"2023-09-09T23:12:21.932198+0000","last_peered":"2023-09-09T23:12:21.932198+0000","last_clean":"2023-09-09T23:12:21.932198+0000","last_became_active":"2023-09-09T23:12:17.899970+0000","last_became_peered":"2023-09-09T23:12:17.899970+0000","last_unstale":"2023-09-09T23:12:21.932198+0000","last_undegraded":"2023-09-09T23:12:21.932198+0000","last_fullsized":"2023-09-09T23:12:21.932198+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:01:37.415195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":59,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:13:03.356617+0000","last_change":"2023-09-09T23:12:17.170123+0000","last_active":"2023-09-09T23:13:03.356617+0000","last_peered":"2023-09-09T23:13:03.356617+0000","last_clean":"2023-09-09T23:13:03.356617+0000","last_became_active":"2023-09-09T23:12:17.169775+0000","last_became_peered":"2023-09-09T23:12:17.169775+0000","last_unstale":"2023-09-09T23:13:03.356617+0000","last_undegraded":"2023-09-09T23:13:03.356617+0000","last_fullsized":"2023-09-09T23:13:03.356617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:22:44.520489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690417+0000","last_change":"2023-09-09T23:12:17.894881+0000","last_active":"2023-09-09T23:12:20.690417+0000","last_peered":"2023-09-09T23:12:20.690417+0000","last_clean":"2023-09-09T23:12:20.690417+0000","last_became_active":"2023-09-09T23:12:17.894567+0000","last_became_peered":"2023-09-09T23:12:17.894567+0000","last_unstale":"2023-09-09T23:12:20.690417+0000","last_undegraded":"2023-09-09T23:12:20.690417+0000","last_fullsized":"2023-09-09T23:12:20.690417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:40:37.098780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689780+0000","last_change":"2023-09-09T23:12:17.903319+0000","last_active":"2023-09-09T23:12:20.689780+0000","last_peered":"2023-09-09T23:12:20.689780+0000","last_clean":"2023-09-09T23:12:20.689780+0000","last_became_active":"2023-09-09T23:12:17.902642+0000","last_became_peered":"2023-09-09T23:12:17.902642+0000","last_unstale":"2023-09-09T23:12:20.689780+0000","last_undegraded":"2023-09-09T23:12:20.689780+0000","last_fullsized":"2023-09-09T23:12:20.689780+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:03:29.562813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":56,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:58.319787+0000","last_change":"2023-09-09T23:12:17.172639+0000","last_active":"2023-09-09T23:12:58.319787+0000","last_peered":"2023-09-09T23:12:58.319787+0000","last_clean":"2023-09-09T23:12:58.319787+0000","last_became_active":"2023-09-09T23:12:17.172309+0000","last_became_peered":"2023-09-09T23:12:17.172309+0000","last_unstale":"2023-09-09T23:12:58.319787+0000","last_undegraded":"2023-09-09T23:12:58.319787+0000","last_fullsized":"2023-09-09T23:12:58.319787+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:05:56.160984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":38,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898148+0000","last_change":"2023-09-09T23:12:17.906676+0000","last_active":"2023-09-09T23:12:18.898148+0000","last_peered":"2023-09-09T23:12:18.898148+0000","last_clean":"2023-09-09T23:12:18.898148+0000","last_became_active":"2023-09-09T23:12:17.906381+0000","last_became_peered":"2023-09-09T23:12:17.906381+0000","last_unstale":"2023-09-09T23:12:18.898148+0000","last_undegraded":"2023-09-09T23:12:18.898148+0000","last_fullsized":"2023-09-09T23:12:18.898148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:42:28.033255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:24.279852+0000","last_change":"2023-09-09T23:12:17.166623+0000","last_active":"2023-09-09T23:12:24.279852+0000","last_peered":"2023-09-09T23:12:24.279852+0000","last_clean":"2023-09-09T23:12:24.279852+0000","last_became_active":"2023-09-09T23:12:17.166293+0000","last_became_peered":"2023-09-09T23:12:17.166293+0000","last_unstale":"2023-09-09T23:12:24.279852+0000","last_undegraded":"2023-09-09T23:12:24.279852+0000","last_fullsized":"2023-09-09T23:12:24.279852+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:52:51.267501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":317,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.963859+0000","last_change":"2023-09-09T23:11:20.948461+0000","last_active":"2023-09-09T23:12:19.963859+0000","last_peered":"2023-09-09T23:12:19.963859+0000","last_clean":"2023-09-09T23:12:19.963859+0000","last_became_active":"2023-09-09T23:11:20.947856+0000","last_became_peered":"2023-09-09T23:11:20.947856+0000","last_unstale":"2023-09-09T23:12:19.963859+0000","last_undegraded":"2023-09-09T23:12:19.963859+0000","last_fullsized":"2023-09-09T23:12:19.963859+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:08:31.737051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:08:31.737051+0000","last_clean_scrub_stamp":"2023-09-09T23:08:31.737051+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:20:16.293707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.688806+0000","last_change":"2023-09-09T23:12:17.907468+0000","last_active":"2023-09-09T23:12:20.688806+0000","last_peered":"2023-09-09T23:12:20.688806+0000","last_clean":"2023-09-09T23:12:20.688806+0000","last_became_active":"2023-09-09T23:12:17.907117+0000","last_became_peered":"2023-09-09T23:12:17.907117+0000","last_unstale":"2023-09-09T23:12:20.688806+0000","last_undegraded":"2023-09-09T23:12:20.688806+0000","last_fullsized":"2023-09-09T23:12:20.688806+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:27:16.623116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:22.583028+0000","last_change":"2023-09-09T23:12:17.171505+0000","last_active":"2023-09-09T23:12:22.583028+0000","last_peered":"2023-09-09T23:12:22.583028+0000","last_clean":"2023-09-09T23:12:22.583028+0000","last_became_active":"2023-09-09T23:12:17.171185+0000","last_became_peered":"2023-09-09T23:12:17.171185+0000","last_unstale":"2023-09-09T23:12:22.583028+0000","last_undegraded":"2023-09-09T23:12:22.583028+0000","last_fullsized":"2023-09-09T23:12:22.583028+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:07:54.271364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:23.271034+0000","last_change":"2023-09-09T23:12:17.176450+0000","last_active":"2023-09-09T23:12:23.271034+0000","last_peered":"2023-09-09T23:12:23.271034+0000","last_clean":"2023-09-09T23:12:23.271034+0000","last_became_active":"2023-09-09T23:12:17.176217+0000","last_became_peered":"2023-09-09T23:12:17.176217+0000","last_unstale":"2023-09-09T23:12:23.271034+0000","last_undegraded":"2023-09-09T23:12:23.271034+0000","last_fullsized":"2023-09-09T23:12:23.271034+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:57:26.604261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689035+0000","last_change":"2023-09-09T23:12:17.897612+0000","last_active":"2023-09-09T23:12:20.689035+0000","last_peered":"2023-09-09T23:12:20.689035+0000","last_clean":"2023-09-09T23:12:20.689035+0000","last_became_active":"2023-09-09T23:12:17.897292+0000","last_became_peered":"2023-09-09T23:12:17.897292+0000","last_unstale":"2023-09-09T23:12:20.689035+0000","last_undegraded":"2023-09-09T23:12:20.689035+0000","last_fullsized":"2023-09-09T23:12:20.689035+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:25:46.374556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'9","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:26.680531+0000","last_change":"2023-09-09T23:12:17.175682+0000","last_active":"2023-09-09T23:12:26.680531+0000","last_peered":"2023-09-09T23:12:26.680531+0000","last_clean":"2023-09-09T23:12:26.680531+0000","last_became_active":"2023-09-09T23:12:17.175291+0000","last_became_peered":"2023-09-09T23:12:17.175291+0000","last_unstale":"2023-09-09T23:12:26.680531+0000","last_undegraded":"2023-09-09T23:12:26.680531+0000","last_fullsized":"2023-09-09T23:12:26.680531+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:29:02.349319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.899164+0000","last_change":"2023-09-09T23:12:17.904153+0000","last_active":"2023-09-09T23:12:18.899164+0000","last_peered":"2023-09-09T23:12:18.899164+0000","last_clean":"2023-09-09T23:12:18.899164+0000","last_became_active":"2023-09-09T23:12:17.903804+0000","last_became_peered":"2023-09-09T23:12:17.903804+0000","last_unstale":"2023-09-09T23:12:18.899164+0000","last_undegraded":"2023-09-09T23:12:18.899164+0000","last_fullsized":"2023-09-09T23:12:18.899164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:39:34.046551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:24.280288+0000","last_change":"2023-09-09T23:12:17.169052+0000","last_active":"2023-09-09T23:12:24.280288+0000","last_peered":"2023-09-09T23:12:24.280288+0000","last_clean":"2023-09-09T23:12:24.280288+0000","last_became_active":"2023-09-09T23:12:17.168719+0000","last_became_peered":"2023-09-09T23:12:17.168719+0000","last_unstale":"2023-09-09T23:12:24.280288+0000","last_undegraded":"2023-09-09T23:12:24.280288+0000","last_fullsized":"2023-09-09T23:12:24.280288+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:56:44.427954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.485050+0000","last_change":"2023-09-09T23:12:17.900945+0000","last_active":"2023-09-09T23:12:19.485050+0000","last_peered":"2023-09-09T23:12:19.485050+0000","last_clean":"2023-09-09T23:12:19.485050+0000","last_became_active":"2023-09-09T23:12:17.900602+0000","last_became_peered":"2023-09-09T23:12:17.900602+0000","last_unstale":"2023-09-09T23:12:19.485050+0000","last_undegraded":"2023-09-09T23:12:19.485050+0000","last_fullsized":"2023-09-09T23:12:19.485050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:22:10.336285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:21.574908+0000","last_change":"2023-09-09T23:12:17.166619+0000","last_active":"2023-09-09T23:12:21.574908+0000","last_peered":"2023-09-09T23:12:21.574908+0000","last_clean":"2023-09-09T23:12:21.574908+0000","last_became_active":"2023-09-09T23:12:17.166286+0000","last_became_peered":"2023-09-09T23:12:17.166286+0000","last_unstale":"2023-09-09T23:12:21.574908+0000","last_undegraded":"2023-09-09T23:12:21.574908+0000","last_fullsized":"2023-09-09T23:12:21.574908+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:41:40.713296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897742+0000","last_change":"2023-09-09T23:12:17.897131+0000","last_active":"2023-09-09T23:12:18.897742+0000","last_peered":"2023-09-09T23:12:18.897742+0000","last_clean":"2023-09-09T23:12:18.897742+0000","last_became_active":"2023-09-09T23:12:17.896873+0000","last_became_peered":"2023-09-09T23:12:17.896873+0000","last_unstale":"2023-09-09T23:12:18.897742+0000","last_undegraded":"2023-09-09T23:12:18.897742+0000","last_fullsized":"2023-09-09T23:12:18.897742+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:54:21.091418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:24.277166+0000","last_change":"2023-09-09T23:12:17.168868+0000","last_active":"2023-09-09T23:12:24.277166+0000","last_peered":"2023-09-09T23:12:24.277166+0000","last_clean":"2023-09-09T23:12:24.277166+0000","last_became_active":"2023-09-09T23:12:17.168572+0000","last_became_peered":"2023-09-09T23:12:17.168572+0000","last_unstale":"2023-09-09T23:12:24.277166+0000","last_undegraded":"2023-09-09T23:12:24.277166+0000","last_fullsized":"2023-09-09T23:12:24.277166+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:54:08.108023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.988326+0000","last_change":"2023-09-09T23:12:17.902887+0000","last_active":"2023-09-09T23:12:18.988326+0000","last_peered":"2023-09-09T23:12:18.988326+0000","last_clean":"2023-09-09T23:12:18.988326+0000","last_became_active":"2023-09-09T23:12:17.902507+0000","last_became_peered":"2023-09-09T23:12:17.902507+0000","last_unstale":"2023-09-09T23:12:18.988326+0000","last_undegraded":"2023-09-09T23:12:18.988326+0000","last_fullsized":"2023-09-09T23:12:18.988326+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:18:08.000184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:23.271048+0000","last_change":"2023-09-09T23:12:17.171379+0000","last_active":"2023-09-09T23:12:23.271048+0000","last_peered":"2023-09-09T23:12:23.271048+0000","last_clean":"2023-09-09T23:12:23.271048+0000","last_became_active":"2023-09-09T23:12:17.171024+0000","last_became_peered":"2023-09-09T23:12:17.171024+0000","last_unstale":"2023-09-09T23:12:23.271048+0000","last_undegraded":"2023-09-09T23:12:23.271048+0000","last_fullsized":"2023-09-09T23:12:23.271048+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:17:12.206768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.898049+0000","last_change":"2023-09-09T23:12:17.905510+0000","last_active":"2023-09-09T23:12:18.898049+0000","last_peered":"2023-09-09T23:12:18.898049+0000","last_clean":"2023-09-09T23:12:18.898049+0000","last_became_active":"2023-09-09T23:12:17.905207+0000","last_became_peered":"2023-09-09T23:12:17.905207+0000","last_unstale":"2023-09-09T23:12:18.898049+0000","last_undegraded":"2023-09-09T23:12:18.898049+0000","last_fullsized":"2023-09-09T23:12:18.898049+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:48:20.167421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:21.574555+0000","last_change":"2023-09-09T23:12:17.173764+0000","last_active":"2023-09-09T23:12:21.574555+0000","last_peered":"2023-09-09T23:12:21.574555+0000","last_clean":"2023-09-09T23:12:21.574555+0000","last_became_active":"2023-09-09T23:12:17.173470+0000","last_became_peered":"2023-09-09T23:12:17.173470+0000","last_unstale":"2023-09-09T23:12:21.574555+0000","last_undegraded":"2023-09-09T23:12:21.574555+0000","last_fullsized":"2023-09-09T23:12:21.574555+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:01:54.091664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690237+0000","last_change":"2023-09-09T23:12:17.906292+0000","last_active":"2023-09-09T23:12:20.690237+0000","last_peered":"2023-09-09T23:12:20.690237+0000","last_clean":"2023-09-09T23:12:20.690237+0000","last_became_active":"2023-09-09T23:12:17.905995+0000","last_became_peered":"2023-09-09T23:12:17.905995+0000","last_unstale":"2023-09-09T23:12:20.690237+0000","last_undegraded":"2023-09-09T23:12:20.690237+0000","last_fullsized":"2023-09-09T23:12:20.690237+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:52:29.972831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'5","reported_seq":57,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:13:00.358958+0000","last_change":"2023-09-09T23:12:17.173818+0000","last_active":"2023-09-09T23:13:00.358958+0000","last_peered":"2023-09-09T23:13:00.358958+0000","last_clean":"2023-09-09T23:13:00.358958+0000","last_became_active":"2023-09-09T23:12:17.173413+0000","last_became_peered":"2023-09-09T23:12:17.173413+0000","last_unstale":"2023-09-09T23:13:00.358958+0000","last_undegraded":"2023-09-09T23:13:00.358958+0000","last_fullsized":"2023-09-09T23:13:00.358958+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:55:28.856863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690266+0000","last_change":"2023-09-09T23:12:17.906833+0000","last_active":"2023-09-09T23:12:20.690266+0000","last_peered":"2023-09-09T23:12:20.690266+0000","last_clean":"2023-09-09T23:12:20.690266+0000","last_became_active":"2023-09-09T23:12:17.906138+0000","last_became_peered":"2023-09-09T23:12:17.906138+0000","last_unstale":"2023-09-09T23:12:20.690266+0000","last_undegraded":"2023-09-09T23:12:20.690266+0000","last_fullsized":"2023-09-09T23:12:20.690266+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:38:07.779167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:23.271410+0000","last_change":"2023-09-09T23:12:17.171338+0000","last_active":"2023-09-09T23:12:23.271410+0000","last_peered":"2023-09-09T23:12:23.271410+0000","last_clean":"2023-09-09T23:12:23.271410+0000","last_became_active":"2023-09-09T23:12:17.170613+0000","last_became_peered":"2023-09-09T23:12:17.170613+0000","last_unstale":"2023-09-09T23:12:23.271410+0000","last_undegraded":"2023-09-09T23:12:23.271410+0000","last_fullsized":"2023-09-09T23:12:23.271410+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:24:07.794464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689777+0000","last_change":"2023-09-09T23:12:17.906470+0000","last_active":"2023-09-09T23:12:20.689777+0000","last_peered":"2023-09-09T23:12:20.689777+0000","last_clean":"2023-09-09T23:12:20.689777+0000","last_became_active":"2023-09-09T23:12:17.906170+0000","last_became_peered":"2023-09-09T23:12:17.906170+0000","last_unstale":"2023-09-09T23:12:20.689777+0000","last_undegraded":"2023-09-09T23:12:20.689777+0000","last_fullsized":"2023-09-09T23:12:20.689777+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:44:37.483541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:24.275625+0000","last_change":"2023-09-09T23:12:17.172142+0000","last_active":"2023-09-09T23:12:24.275625+0000","last_peered":"2023-09-09T23:12:24.275625+0000","last_clean":"2023-09-09T23:12:24.275625+0000","last_became_active":"2023-09-09T23:12:17.171703+0000","last_became_peered":"2023-09-09T23:12:17.171703+0000","last_unstale":"2023-09-09T23:12:24.275625+0000","last_undegraded":"2023-09-09T23:12:24.275625+0000","last_fullsized":"2023-09-09T23:12:24.275625+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:49:31.414385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689874+0000","last_change":"2023-09-09T23:12:17.900699+0000","last_active":"2023-09-09T23:12:20.689874+0000","last_peered":"2023-09-09T23:12:20.689874+0000","last_clean":"2023-09-09T23:12:20.689874+0000","last_became_active":"2023-09-09T23:12:17.900416+0000","last_became_peered":"2023-09-09T23:12:17.900416+0000","last_unstale":"2023-09-09T23:12:20.689874+0000","last_undegraded":"2023-09-09T23:12:20.689874+0000","last_fullsized":"2023-09-09T23:12:20.689874+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:54:29.328413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.988389+0000","last_change":"2023-09-09T23:12:17.900480+0000","last_active":"2023-09-09T23:12:18.988389+0000","last_peered":"2023-09-09T23:12:18.988389+0000","last_clean":"2023-09-09T23:12:18.988389+0000","last_became_active":"2023-09-09T23:12:17.900135+0000","last_became_peered":"2023-09-09T23:12:17.900135+0000","last_unstale":"2023-09-09T23:12:18.988389+0000","last_undegraded":"2023-09-09T23:12:18.988389+0000","last_fullsized":"2023-09-09T23:12:18.988389+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:25:44.254310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690446+0000","last_change":"2023-09-09T23:12:17.901302+0000","last_active":"2023-09-09T23:12:20.690446+0000","last_peered":"2023-09-09T23:12:20.690446+0000","last_clean":"2023-09-09T23:12:20.690446+0000","last_became_active":"2023-09-09T23:12:17.900982+0000","last_became_peered":"2023-09-09T23:12:17.900982+0000","last_unstale":"2023-09-09T23:12:20.690446+0000","last_undegraded":"2023-09-09T23:12:20.690446+0000","last_fullsized":"2023-09-09T23:12:20.690446+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:41:41.119614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689782+0000","last_change":"2023-09-09T23:12:17.904328+0000","last_active":"2023-09-09T23:12:20.689782+0000","last_peered":"2023-09-09T23:12:20.689782+0000","last_clean":"2023-09-09T23:12:20.689782+0000","last_became_active":"2023-09-09T23:12:17.904040+0000","last_became_peered":"2023-09-09T23:12:17.904040+0000","last_unstale":"2023-09-09T23:12:20.689782+0000","last_undegraded":"2023-09-09T23:12:20.689782+0000","last_fullsized":"2023-09-09T23:12:20.689782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:01:05.986097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.485015+0000","last_change":"2023-09-09T23:12:17.901737+0000","last_active":"2023-09-09T23:12:19.485015+0000","last_peered":"2023-09-09T23:12:19.485015+0000","last_clean":"2023-09-09T23:12:19.485015+0000","last_became_active":"2023-09-09T23:12:17.901452+0000","last_became_peered":"2023-09-09T23:12:17.901452+0000","last_unstale":"2023-09-09T23:12:19.485015+0000","last_undegraded":"2023-09-09T23:12:19.485015+0000","last_fullsized":"2023-09-09T23:12:19.485015+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:35:28.698304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.688950+0000","last_change":"2023-09-09T23:12:17.907433+0000","last_active":"2023-09-09T23:12:20.688950+0000","last_peered":"2023-09-09T23:12:20.688950+0000","last_clean":"2023-09-09T23:12:20.688950+0000","last_became_active":"2023-09-09T23:12:17.907127+0000","last_became_peered":"2023-09-09T23:12:17.907127+0000","last_unstale":"2023-09-09T23:12:20.688950+0000","last_undegraded":"2023-09-09T23:12:20.688950+0000","last_fullsized":"2023-09-09T23:12:20.688950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:00:32.291141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.988586+0000","last_change":"2023-09-09T23:12:17.901373+0000","last_active":"2023-09-09T23:12:18.988586+0000","last_peered":"2023-09-09T23:12:18.988586+0000","last_clean":"2023-09-09T23:12:18.988586+0000","last_became_active":"2023-09-09T23:12:17.901080+0000","last_became_peered":"2023-09-09T23:12:17.901080+0000","last_unstale":"2023-09-09T23:12:18.988586+0000","last_undegraded":"2023-09-09T23:12:18.988586+0000","last_fullsized":"2023-09-09T23:12:18.988586+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:23:12.316659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.690576+0000","last_change":"2023-09-09T23:12:17.896579+0000","last_active":"2023-09-09T23:12:20.690576+0000","last_peered":"2023-09-09T23:12:20.690576+0000","last_clean":"2023-09-09T23:12:20.690576+0000","last_became_active":"2023-09-09T23:12:17.896253+0000","last_became_peered":"2023-09-09T23:12:17.896253+0000","last_unstale":"2023-09-09T23:12:20.690576+0000","last_undegraded":"2023-09-09T23:12:20.690576+0000","last_fullsized":"2023-09-09T23:12:20.690576+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:30:20.249103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.899186+0000","last_change":"2023-09-09T23:12:17.903569+0000","last_active":"2023-09-09T23:12:18.899186+0000","last_peered":"2023-09-09T23:12:18.899186+0000","last_clean":"2023-09-09T23:12:18.899186+0000","last_became_active":"2023-09-09T23:12:17.903027+0000","last_became_peered":"2023-09-09T23:12:17.903027+0000","last_unstale":"2023-09-09T23:12:18.899186+0000","last_undegraded":"2023-09-09T23:12:18.899186+0000","last_fullsized":"2023-09-09T23:12:18.899186+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:38:46.392973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689604+0000","last_change":"2023-09-09T23:12:17.902944+0000","last_active":"2023-09-09T23:12:20.689604+0000","last_peered":"2023-09-09T23:12:20.689604+0000","last_clean":"2023-09-09T23:12:20.689604+0000","last_became_active":"2023-09-09T23:12:17.901812+0000","last_became_peered":"2023-09-09T23:12:17.901812+0000","last_unstale":"2023-09-09T23:12:20.689604+0000","last_undegraded":"2023-09-09T23:12:20.689604+0000","last_fullsized":"2023-09-09T23:12:20.689604+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:49:50.981693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.897608+0000","last_change":"2023-09-09T23:12:17.897612+0000","last_active":"2023-09-09T23:12:18.897608+0000","last_peered":"2023-09-09T23:12:18.897608+0000","last_clean":"2023-09-09T23:12:18.897608+0000","last_became_active":"2023-09-09T23:12:17.897318+0000","last_became_peered":"2023-09-09T23:12:17.897318+0000","last_unstale":"2023-09-09T23:12:18.897608+0000","last_undegraded":"2023-09-09T23:12:18.897608+0000","last_fullsized":"2023-09-09T23:12:18.897608+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:17:38.687470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:19.519770+0000","last_change":"2023-09-09T23:12:17.903849+0000","last_active":"2023-09-09T23:12:19.519770+0000","last_peered":"2023-09-09T23:12:19.519770+0000","last_clean":"2023-09-09T23:12:19.519770+0000","last_became_active":"2023-09-09T23:12:17.903534+0000","last_became_peered":"2023-09-09T23:12:17.903534+0000","last_unstale":"2023-09-09T23:12:19.519770+0000","last_undegraded":"2023-09-09T23:12:19.519770+0000","last_fullsized":"2023-09-09T23:12:19.519770+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:06:18.073811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:20.689155+0000","last_change":"2023-09-09T23:12:17.899782+0000","last_active":"2023-09-09T23:12:20.689155+0000","last_peered":"2023-09-09T23:12:20.689155+0000","last_clean":"2023-09-09T23:12:20.689155+0000","last_became_active":"2023-09-09T23:12:17.899445+0000","last_became_peered":"2023-09-09T23:12:17.899445+0000","last_unstale":"2023-09-09T23:12:20.689155+0000","last_undegraded":"2023-09-09T23:12:20.689155+0000","last_fullsized":"2023-09-09T23:12:20.689155+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:49:20.276927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-09-09T23:12:18.988408+0000","last_change":"2023-09-09T23:12:17.898693+0000","last_active":"2023-09-09T23:12:18.988408+0000","last_peered":"2023-09-09T23:12:18.988408+0000","last_clean":"2023-09-09T23:12:18.988408+0000","last_became_active":"2023-09-09T23:12:17.898343+0000","last_became_peered":"2023-09-09T23:12:17.898343+0000","last_unstale":"2023-09-09T23:12:18.988408+0000","last_undegraded":"2023-09-09T23:12:18.988408+0000","last_fullsized":"2023-09-09T23:12:18.988408+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:37:43.229953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63491,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":426,"num_read_kb":278,"num_write":230,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":190473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514516,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":880640,"data_stored":417227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.61599999999999999}]},{"osd":1,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.51200000000000001}]},{"osd":2,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.63500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":4,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.59599999999999997}]},{"osd":5,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.73899999999999999}]},{"osd":6,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.65100000000000002}]},{"osd":7,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.76500000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":9,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.85299999999999998}]},{"osd":10,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.75600000000000001}]}]},{"osd":10,"up_from":69,"seq":296352743448,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1576960,"data_stored":1064941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":1,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.80900000000000005}]},{"osd":2,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.76400000000000001}]},{"osd":3,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.64500000000000002}]},{"osd":4,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.79500000000000004}]},{"osd":5,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.82999999999999996}]},{"osd":6,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.85199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":8,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.94099999999999995}]},{"osd":9,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.878}]},{"osd":11,"last update":"Sat Sep 9 23:12:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":1.004}]}]},{"osd":9,"up_from":64,"seq":274877906971,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":417157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.435,"15min":0.443},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.73999999999999999,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.29499999999999998}]},{"osd":1,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56799999999999995,"15min":0.56899999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.41899999999999998}]},{"osd":2,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.626,"15min":0.63},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.625,"15min":0.63200000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.44500000000000001}]},{"osd":3,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.51300000000000001,"15min":0.52300000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.75}]},{"osd":4,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.629,"15min":0.63500000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96799999999999997,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.67400000000000004,"15min":0.68799999999999994},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81999999999999995,"5min":1.028,"15min":1.028},"last":0.56799999999999995}]},{"osd":5,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.73399999999999999,"15min":0.71499999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":5.6699999999999999,"5min":5.6699999999999999,"15min":5.6699999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.68200000000000005,"15min":0.69399999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.66400000000000003}]},{"osd":6,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.64300000000000002,"15min":0.65500000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.65600000000000003,"15min":0.67000000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.69299999999999995}]},{"osd":7,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.745,"15min":0.76100000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.82499999999999996,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.746,"15min":0.75800000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.71399999999999997}]},{"osd":8,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.73799999999999999,"15min":0.747},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.74099999999999999,"15min":0.755},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.85899999999999999,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.64900000000000002}]},{"osd":10,"last update":"Sat Sep 9 23:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.68100000000000005}]}]},{"osd":8,"up_from":57,"seq":244813135905,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":417535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.38,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80700000000000005,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.87},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.435,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81799999999999995,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.78500000000000003}]},{"osd":1,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.60899999999999999,"15min":0.59399999999999997},"min":{"1min":0.45600000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.59399999999999997,"15min":0.58499999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82599999999999996,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.64200000000000002}]},{"osd":2,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48199999999999998,"15min":0.48299999999999998},"min":{"1min":0.29399999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.47999999999999998,"15min":0.48699999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.60899999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.52800000000000002}]},{"osd":3,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.57099999999999995,"15min":0.57799999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.91300000000000003,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.755},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.63600000000000001,"15min":0.63900000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85699999999999998,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.66400000000000003}]},{"osd":4,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64100000000000001,"15min":0.63700000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94399999999999995,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.62},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.68999999999999995,"15min":0.68200000000000005},"min":{"1min":0.41899999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.77200000000000002}]},{"osd":5,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54700000000000004,"15min":0.55400000000000005},"min":{"1min":0.34100000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68500000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.442},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55500000000000005}]},{"osd":6,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71899999999999997,"15min":0.72499999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.036,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.68400000000000005,"15min":0.69499999999999995},"min":{"1min":0.41199999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86699999999999999,"5min":0.996,"15min":0.996},"last":0.79700000000000004}]},{"osd":7,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71999999999999997,"15min":0.72399999999999998},"min":{"1min":0.51400000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.91200000000000003,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72599999999999998,"15min":0.72699999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.93999999999999995,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.68999999999999995}]},{"osd":9,"last update":"Sat Sep 9 23:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.91600000000000004}]},{"osd":10,"last update":"Sat Sep 9 23:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.84099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]}]},{"osd":7,"up_from":50,"seq":214748364837,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28476,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716676,"statfs":{"total":95995035648,"available":95965876224,"internally_reserved":0,"allocated":1495040,"data_stored":1010117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.439},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.33100000000000002}]},{"osd":1,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.435,"15min":0.432},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.60999999999999999,"5min":0.625,"15min":0.625},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.42699999999999999,"15min":0.41699999999999998},"min":{"1min":0.26200000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.56599999999999995}]},{"osd":2,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.77200000000000002,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.61299999999999999}]},{"osd":3,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.40600000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.626,"15min":0.625},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.79800000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.47999999999999998}]},{"osd":4,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52900000000000003,"15min":0.52300000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.76400000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.372},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.314,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.45000000000000001}]},{"osd":5,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.69399999999999995,"15min":0.69599999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86399999999999999,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.66900000000000004,"15min":0.66800000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.71499999999999997}]},{"osd":6,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.746,"15min":0.753},"min":{"1min":0.48899999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90900000000000003,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.627},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.72599999999999998,"15min":0.72899999999999998},"min":{"1min":0.46800000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92600000000000005,"5min":1.254,"15min":1.254},"last":0.58499999999999996}]},{"osd":8,"last update":"Sat Sep 9 23:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73399999999999999,"15min":0.73899999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.86399999999999999,"5min":1.008,"15min":1.008},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.76400000000000001,"15min":0.77000000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.88,"5min":1.387,"15min":1.387},"last":0.73699999999999999}]},{"osd":9,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.75,"15min":0.76000000000000001},"min":{"1min":0.51700000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.89900000000000002,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.92400000000000004,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.751}]},{"osd":10,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69599999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]}]},{"osd":6,"up_from":45,"seq":193273528361,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":888832,"data_stored":416880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.377,"15min":0.36899999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.40400000000000003,"15min":0.40200000000000002},"min":{"1min":0.26800000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.53700000000000003,"5min":0.753,"15min":0.753},"last":0.31}]},{"osd":1,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78100000000000003,"5min":0.877,"15min":0.877},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77200000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.44600000000000001}]},{"osd":2,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.41699999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79600000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76800000000000002,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.56100000000000005}]},{"osd":3,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.43099999999999999,"15min":0.433},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70199999999999996,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.501,"15min":0.51200000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.59399999999999997,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.38200000000000001}]},{"osd":4,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60699999999999998,"15min":0.60299999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.78100000000000003,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.77900000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.54400000000000004}]},{"osd":5,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.63200000000000001,"15min":0.64000000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.75600000000000001,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.495},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.65900000000000003,"15min":0.65700000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82199999999999995,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59499999999999997}]},{"osd":7,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93400000000000005,"5min":1.032,"15min":1.032},"last":1.073},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.91500000000000004}]},{"osd":8,"last update":"Sat Sep 9 23:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.76700000000000002,"15min":0.78800000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85099999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.74399999999999999,"15min":0.76400000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.80100000000000005,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.68799999999999994}]},{"osd":9,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.64500000000000002,"15min":0.66100000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70799999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.72199999999999998,"15min":0.752},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73099999999999998,"5min":1.534,"15min":1.534},"last":0.57899999999999996}]},{"osd":10,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.67400000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":474076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.51600000000000001,"15min":0.498},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.51400000000000001,"15min":0.48999999999999999},"min":{"1min":0.35099999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.46000000000000002}]},{"osd":2,"last update":"Sat Sep 9 23:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.54300000000000004,"15min":0.50900000000000001},"min":{"1min":0.442,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.55600000000000005,"15min":0.52300000000000002},"min":{"1min":0.45200000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.90400000000000003,"5min":1.133,"15min":1.133},"last":0.746}]},{"osd":3,"last update":"Sat Sep 9 23:11:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.47299999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.57499999999999996}]},{"osd":4,"last update":"Sat Sep 9 23:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.438,"15min":0.41399999999999998},"min":{"1min":0.30499999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.48599999999999999,"15min":0.44800000000000001},"min":{"1min":0.28499999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.68500000000000005}]},{"osd":5,"last update":"Sat Sep 9 23:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.58199999999999996,"15min":0.56100000000000005},"min":{"1min":0.40200000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.83299999999999996,"5min":1.022,"15min":1.022},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.60199999999999998,"15min":0.57399999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89400000000000002,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.85199999999999998}]},{"osd":6,"last update":"Sat Sep 9 23:11:59 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.58499999999999996,"15min":0.56499999999999995},"min":{"1min":0.45300000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.628},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"min":{"1min":0.40699999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.65000000000000002}]},{"osd":7,"last update":"Sat Sep 9 23:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.53100000000000003,"15min":0.51400000000000001},"min":{"1min":0.376,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52800000000000002,"15min":0.52000000000000002},"min":{"1min":0.33300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.80200000000000005}]},{"osd":8,"last update":"Sat Sep 9 23:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.66200000000000003,"15min":0.65600000000000003},"min":{"1min":0.441,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.69899999999999995,"15min":0.69499999999999995},"min":{"1min":0.502,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.95499999999999996,"5min":1.218,"15min":1.218},"last":0.76800000000000002}]},{"osd":9,"last update":"Sat Sep 9 23:11:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.70499999999999996}]},{"osd":10,"last update":"Sat Sep 9 23:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.874}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738434,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":418213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 9 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.53100000000000003,"15min":0.51100000000000001},"min":{"1min":0.39700000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.78000000000000003,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51400000000000001,"15min":0.5},"min":{"1min":0.374,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71799999999999997,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.33400000000000002}]},{"osd":2,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53700000000000003,"15min":0.52900000000000003},"min":{"1min":0.39000000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78200000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.40000000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76800000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.49299999999999999}]},{"osd":3,"last update":"Sat Sep 9 23:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.39300000000000002,"15min":0.38900000000000001},"min":{"1min":0.27900000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.41599999999999998,"15min":0.40999999999999998},"min":{"1min":0.33500000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.47899999999999998}]},{"osd":4,"last update":"Sat Sep 9 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.54900000000000004,"15min":0.53300000000000003},"min":{"1min":0.42899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82999999999999996,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.375,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.55800000000000005}]},{"osd":5,"last update":"Sat Sep 9 23:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71099999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56799999999999995,"15min":0.56499999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.752,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.57699999999999996}]},{"osd":6,"last update":"Sat Sep 9 23:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.495,"15min":0.497},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.55300000000000005,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.504,"15min":0.501},"min":{"1min":0.33000000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67300000000000004,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.68400000000000005}]},{"osd":7,"last update":"Sat Sep 9 23:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.47599999999999998}]},{"osd":8,"last update":"Sat Sep 9 23:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.68700000000000006,"15min":0.69899999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.78000000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.68200000000000005,"15min":0.68999999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85699999999999998,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.71699999999999997}]},{"osd":9,"last update":"Sat Sep 9 23:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.628,"15min":0.65400000000000003},"min":{"1min":0.36799999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.60999999999999999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.67300000000000004,"15min":0.69799999999999995},"min":{"1min":0.34699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70599999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.54400000000000004}]},{"osd":10,"last update":"Sat Sep 9 23:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.65500000000000003}]},{"osd":11,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.751}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":474686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52500000000000002,"15min":0.53300000000000003},"min":{"1min":0.42299999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84199999999999997,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.33800000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.751,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.52500000000000002}]},{"osd":1,"last update":"Sat Sep 9 23:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.56499999999999995,"15min":0.53900000000000003},"min":{"1min":0.47799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84199999999999997,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.55800000000000005,"15min":0.51800000000000002},"min":{"1min":0.47499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.68999999999999995}]},{"osd":3,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59999999999999998,"15min":0.60699999999999998},"min":{"1min":0.36199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0009999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61099999999999999,"15min":0.61599999999999999},"min":{"1min":0.46300000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90700000000000003,"5min":1.478,"15min":1.478},"last":0.64500000000000002}]},{"osd":4,"last update":"Sat Sep 9 23:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57299999999999995,"15min":0.57199999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.78000000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.62},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.85899999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.59799999999999998}]},{"osd":5,"last update":"Sat Sep 9 23:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.42999999999999999,"15min":0.40600000000000003},"min":{"1min":0.33800000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.443,"15min":0.42299999999999999},"min":{"1min":0.33100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.495}]},{"osd":6,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.56999999999999995,"15min":0.53200000000000003},"min":{"1min":0.47799999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.749,"5min":0.63300000000000001,"15min":0.59899999999999998},"min":{"1min":0.48799999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.78900000000000003}]},{"osd":7,"last update":"Sat Sep 9 23:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.65700000000000003}]},{"osd":8,"last update":"Sat Sep 9 23:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.59899999999999998,"15min":0.58599999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.753}]},{"osd":9,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.70499999999999996,"15min":0.69099999999999995},"min":{"1min":0.52100000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.752,"15min":0.73699999999999999},"min":{"1min":0.59799999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.76300000000000001}]},{"osd":10,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.77600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":416380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.32700000000000001,"15min":0.313},"min":{"1min":0.22800000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.501,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.314},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35199999999999998,"15min":0.33400000000000002},"min":{"1min":0.249,"5min":0.222,"15min":0.222},"max":{"1min":0.53800000000000003,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.51500000000000001}]},{"osd":1,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54500000000000004,"15min":0.51900000000000002},"min":{"1min":0.32900000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75900000000000001,"5min":1.091,"15min":1.091},"last":0.44},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.51000000000000001,"15min":0.51400000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.60799999999999998,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.47899999999999998}]},{"osd":2,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54400000000000004,"15min":0.52000000000000002},"min":{"1min":0.36099999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65100000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53800000000000003,"15min":0.51800000000000002},"min":{"1min":0.40300000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.66200000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.55500000000000005}]},{"osd":4,"last update":"Sat Sep 9 23:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.53400000000000003,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.71799999999999997,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.54300000000000004,"15min":0.51900000000000002},"min":{"1min":0.36499999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76300000000000001,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.503}]},{"osd":5,"last update":"Sat Sep 9 23:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.57699999999999996,"15min":0.58299999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.70099999999999996,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61899999999999999,"15min":0.60599999999999998},"min":{"1min":0.49399999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.79200000000000004,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.68400000000000005}]},{"osd":6,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.49099999999999999,"15min":0.49199999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73399999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.50900000000000001,"15min":0.50600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64800000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.60299999999999998}]},{"osd":7,"last update":"Sat Sep 9 23:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66600000000000004,"15min":0.66400000000000003},"min":{"1min":0.45400000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69299999999999995,"15min":0.69799999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.623}]},{"osd":8,"last update":"Sat Sep 9 23:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.49199999999999999}]},{"osd":9,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.66500000000000004,"15min":0.68899999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77500000000000002,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.67700000000000005,"15min":0.69899999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.746,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.58499999999999996}]},{"osd":10,"last update":"Sat Sep 9 23:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.69999999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]}]},{"osd":4,"up_from":32,"seq":137438953522,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":888832,"data_stored":417125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.503,"15min":0.47099999999999997},"min":{"1min":0.44900000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.52200000000000002,"15min":0.48799999999999999},"min":{"1min":0.41999999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.33300000000000002}]},{"osd":1,"last update":"Sat Sep 9 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.35899999999999999,"15min":0.33800000000000002},"min":{"1min":0.26100000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.52300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.38100000000000001,"15min":0.35699999999999998},"min":{"1min":0.27400000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.68999999999999995,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.58899999999999997}]},{"osd":2,"last update":"Sat Sep 9 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53000000000000003,"15min":0.51400000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.69399999999999995,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.54700000000000004,"15min":0.52200000000000002},"min":{"1min":0.377,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79700000000000004,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.71799999999999997}]},{"osd":3,"last update":"Sat Sep 9 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.52300000000000002,"15min":0.48599999999999999},"min":{"1min":0.47199999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.51700000000000002,"15min":0.49299999999999999},"min":{"1min":0.39200000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.66300000000000003}]},{"osd":5,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59299999999999997,"15min":0.57699999999999996},"min":{"1min":0.40899999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.59599999999999997,"15min":0.56000000000000005},"min":{"1min":0.42599999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.76000000000000001}]},{"osd":6,"last update":"Sat Sep 9 23:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64800000000000002,"15min":0.61799999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.85899999999999999,"5min":1.226,"15min":1.226},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.65600000000000003,"15min":0.62},"min":{"1min":0.40000000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.5219999999999998,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":0.84299999999999997}]},{"osd":7,"last update":"Sat Sep 9 23:12:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.83799999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.36699999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72399999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.748}]},{"osd":8,"last update":"Sat Sep 9 23:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69199999999999995,"15min":0.68899999999999995},"min":{"1min":0.45700000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.749,"15min":0.751},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.89500000000000002,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.98099999999999998}]},{"osd":9,"last update":"Sat Sep 9 23:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80700000000000005,"15min":0.81200000000000006},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.75800000000000001,"15min":0.76600000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.92800000000000005,"5min":1.127,"15min":1.127},"last":0.92800000000000005}]},{"osd":10,"last update":"Sat Sep 9 23:12:20 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.79900000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]}]},{"osd":5,"up_from":37,"seq":158913789997,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1511424,"data_stored":1009736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.48399999999999999,"15min":0.46300000000000002},"min":{"1min":0.40500000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.502,"15min":0.49299999999999999},"min":{"1min":0.377,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.48099999999999998}]},{"osd":1,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.52600000000000002,"15min":0.50800000000000001},"min":{"1min":0.39000000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.53300000000000003,"15min":0.51500000000000001},"min":{"1min":0.36899999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.57099999999999995}]},{"osd":2,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.38400000000000001,"15min":0.372},"min":{"1min":0.27700000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.66500000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.432,"5min":0.39400000000000002,"15min":0.38800000000000001},"min":{"1min":0.309,"5min":0.248,"15min":0.248},"max":{"1min":0.64100000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.60799999999999998}]},{"osd":3,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.38600000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.75,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.54600000000000004,"15min":0.52400000000000002},"min":{"1min":0.41799999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86599999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.50700000000000001}]},{"osd":4,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.55900000000000005,"15min":0.53300000000000003},"min":{"1min":0.40799999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.46700000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.66800000000000004}]},{"osd":6,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56000000000000005,"15min":0.54600000000000004},"min":{"1min":0.38300000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88300000000000001,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.437,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92200000000000004,"5min":0.998,"15min":0.998},"last":0.627}]},{"osd":7,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.59599999999999997,"15min":0.58799999999999997},"min":{"1min":0.498,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79200000000000004,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82199999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.72099999999999997}]},{"osd":8,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57199999999999995,"15min":0.56399999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83799999999999997,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55200000000000005,"15min":0.54600000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.75600000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.70099999999999996}]},{"osd":9,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.53700000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.82099999999999995,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.755},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.70599999999999996,"15min":0.70299999999999996},"min":{"1min":0.56999999999999995,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93899999999999995,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.73599999999999999}]},{"osd":10,"last update":"Sat Sep 9 23:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.77000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":58415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-09T23:13:06.755 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-09-09T23:13:06.756 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-09-09T23:13:06.756 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-09-09T23:13:06.756 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph health --format=json 2023-09-09T23:13:06.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:06 smithi098 ceph-mon[103124]: pgmap v279: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2023-09-09T23:13:07.310 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:13:08.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.471+0000 7ff702f20700 1 -- 172.21.15.82:0/203670141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6fc0ffee0 msgr2=0x7ff6fc10c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:08.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.471+0000 7ff702f20700 1 --2- 172.21.15.82:0/203670141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6fc0ffee0 0x7ff6fc10c3f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400b0a0 tx=0x7ff6f400b3b0 comp rx=0 tx=0).stop 2023-09-09T23:13:08.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.471+0000 7ff702f20700 1 -- 172.21.15.82:0/203670141 shutdown_connections 2023-09-09T23:13:08.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.471+0000 7ff702f20700 1 --2- 172.21.15.82:0/203670141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6fc104530 0x7ff6fc104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:08.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.471+0000 7ff702f20700 1 --2- 172.21.15.82:0/203670141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6fc0ffee0 0x7ff6fc10c3f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:08.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.471+0000 7ff702f20700 1 --2- 172.21.15.82:0/203670141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6fc0ff520 0x7ff6fc0ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:08.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.471+0000 7ff702f20700 1 -- 172.21.15.82:0/203670141 >> 172.21.15.82:0/203670141 conn(0x7ff6fc076600 msgr2=0x7ff6fc076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:08.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.471+0000 7ff702f20700 1 -- 172.21.15.82:0/203670141 shutdown_connections 2023-09-09T23:13:08.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.471+0000 7ff702f20700 1 -- 172.21.15.82:0/203670141 wait complete. 2023-09-09T23:13:08.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.472+0000 7ff702f20700 1 Processor -- start 2023-09-09T23:13:08.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.472+0000 7ff702f20700 1 -- start start 2023-09-09T23:13:08.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.472+0000 7ff702f20700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6fc0ff520 0x7ff6fc194c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:08.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.472+0000 7ff702f20700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6fc0ffee0 0x7ff6fc1951b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:08.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.472+0000 7ff702f20700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6fc104530 0x7ff6fc19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:08.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.472+0000 7ff702f20700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc19a780 con 0x7ff6fc0ff520 2023-09-09T23:13:08.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.472+0000 7ff702f20700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc19a8f0 con 0x7ff6fc0ffee0 2023-09-09T23:13:08.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.472+0000 7ff702f20700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc19aab0 con 0x7ff6fc104530 2023-09-09T23:13:08.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.472+0000 7ff700cbc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6fc0ff520 0x7ff6fc194c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:08.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.472+0000 7ff700cbc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6fc0ff520 0x7ff6fc194c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55082/0 (socket says 172.21.15.82:55082) 2023-09-09T23:13:08.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.472+0000 7ff700cbc700 1 -- 172.21.15.82:0/2004328828 learned_addr learned my addr 172.21.15.82:0/2004328828 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:08.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.472+0000 7ff700cbc700 1 -- 172.21.15.82:0/2004328828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6fc104530 msgr2=0x7ff6fc19a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:08.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.472+0000 7ff700cbc700 1 --2- 172.21.15.82:0/2004328828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6fc104530 0x7ff6fc19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:08.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.472+0000 7ff700cbc700 1 -- 172.21.15.82:0/2004328828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6fc0ffee0 msgr2=0x7ff6fc1951b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:08.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.472+0000 7ff6fbfff700 1 --2- 172.21.15.82:0/2004328828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6fc0ffee0 0x7ff6fc1951b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:08.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.473+0000 7ff700cbc700 1 --2- 172.21.15.82:0/2004328828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6fc0ffee0 0x7ff6fc1951b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:08.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.473+0000 7ff700cbc700 1 -- 172.21.15.82:0/2004328828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f4009d00 con 0x7ff6fc0ff520 2023-09-09T23:13:08.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.473+0000 7ff6fbfff700 1 --2- 172.21.15.82:0/2004328828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6fc0ffee0 0x7ff6fc1951b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:08.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.473+0000 7ff700cbc700 1 --2- 172.21.15.82:0/2004328828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6fc0ff520 0x7ff6fc194c70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff6f000c970 tx=0x7ff6f000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:08.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.473+0000 7ff6f9ffb700 1 -- 172.21.15.82:0/2004328828 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0004220 con 0x7ff6fc0ff520 2023-09-09T23:13:08.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.473+0000 7ff6f9ffb700 1 -- 172.21.15.82:0/2004328828 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6f0007d60 con 0x7ff6fc0ff520 2023-09-09T23:13:08.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.473+0000 7ff702f20700 1 -- 172.21.15.82:0/2004328828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6fc19ad90 con 0x7ff6fc0ff520 2023-09-09T23:13:08.477 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.473+0000 7ff6f9ffb700 1 -- 172.21.15.82:0/2004328828 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0003d00 con 0x7ff6fc0ff520 2023-09-09T23:13:08.477 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.473+0000 7ff702f20700 1 -- 172.21.15.82:0/2004328828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6fc1b24b0 con 0x7ff6fc0ff520 2023-09-09T23:13:08.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.474+0000 7ff702f20700 1 -- 172.21.15.82:0/2004328828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6fc04e300 con 0x7ff6fc0ff520 2023-09-09T23:13:08.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.475+0000 7ff6f9ffb700 1 -- 172.21.15.82:0/2004328828 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7ff6f0004380 con 0x7ff6fc0ff520 2023-09-09T23:13:08.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.475+0000 7ff6f9ffb700 1 --2- 172.21.15.82:0/2004328828 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff6ec0a0820 0x7ff6ec0a2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:08.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.476+0000 7ff6fbfff700 1 --2- 172.21.15.82:0/2004328828 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff6ec0a0820 0x7ff6ec0a2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:08.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.476+0000 7ff6f9ffb700 1 -- 172.21.15.82:0/2004328828 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff6f00cb9b0 con 0x7ff6fc0ff520 2023-09-09T23:13:08.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.476+0000 7ff6fbfff700 1 --2- 172.21.15.82:0/2004328828 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff6ec0a0820 0x7ff6ec0a2ce0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8005fd0 tx=0x7ff6e8005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:08.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.480+0000 7ff6f9ffb700 1 -- 172.21.15.82:0/2004328828 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff6f0095530 con 0x7ff6fc0ff520 2023-09-09T23:13:08.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:08 smithi170 ceph-mon[102832]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-09T23:13:08.783 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:08 smithi170 ceph-mon[102832]: pgmap v280: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s 2023-09-09T23:13:08.784 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:08 smithi098 ceph-mon[103124]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-09T23:13:08.785 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:08 smithi098 ceph-mon[103124]: pgmap v280: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s 2023-09-09T23:13:08.785 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:08 smithi082 ceph-mon[98694]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-09T23:13:08.785 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:08 smithi082 ceph-mon[98694]: pgmap v280: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s 2023-09-09T23:13:08.786 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.710+0000 7ff702f20700 1 -- 172.21.15.82:0/2004328828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff6fc195ab0 con 0x7ff6fc0ff520 2023-09-09T23:13:08.786 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.711+0000 7ff6f9ffb700 1 -- 172.21.15.82:0/2004328828 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7ff6f001d080 con 0x7ff6fc0ff520 2023-09-09T23:13:08.786 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:08.787 INFO:teuthology.orchestra.run.smithi082.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-09-09T23:13:08.787 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.713+0000 7ff702f20700 1 -- 172.21.15.82:0/2004328828 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff6ec0a0820 msgr2=0x7ff6ec0a2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:08.787 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.713+0000 7ff702f20700 1 --2- 172.21.15.82:0/2004328828 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff6ec0a0820 0x7ff6ec0a2ce0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8005fd0 tx=0x7ff6e8005dc0 comp rx=0 tx=0).stop 2023-09-09T23:13:08.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.713+0000 7ff702f20700 1 -- 172.21.15.82:0/2004328828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6fc0ff520 msgr2=0x7ff6fc194c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:08.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.713+0000 7ff702f20700 1 --2- 172.21.15.82:0/2004328828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6fc0ff520 0x7ff6fc194c70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff6f000c970 tx=0x7ff6f000cd30 comp rx=0 tx=0).stop 2023-09-09T23:13:08.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.713+0000 7ff702f20700 1 -- 172.21.15.82:0/2004328828 shutdown_connections 2023-09-09T23:13:08.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.713+0000 7ff702f20700 1 --2- 172.21.15.82:0/2004328828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6fc104530 0x7ff6fc19a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:08.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.713+0000 7ff702f20700 1 --2- 172.21.15.82:0/2004328828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6fc0ff520 0x7ff6fc194c70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:08.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.713+0000 7ff702f20700 1 --2- 172.21.15.82:0/2004328828 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff6ec0a0820 0x7ff6ec0a2ce0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:08.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.713+0000 7ff702f20700 1 --2- 172.21.15.82:0/2004328828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6fc0ffee0 0x7ff6fc1951b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:08.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.713+0000 7ff702f20700 1 -- 172.21.15.82:0/2004328828 >> 172.21.15.82:0/2004328828 conn(0x7ff6fc076600 msgr2=0x7ff6fc0680c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:08.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.713+0000 7ff702f20700 1 -- 172.21.15.82:0/2004328828 shutdown_connections 2023-09-09T23:13:08.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:08.713+0000 7ff702f20700 1 -- 172.21.15.82:0/2004328828 wait complete. 2023-09-09T23:13:09.063 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-09-09T23:13:09.064 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-09T23:13:09.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.192+0000 7f3291a9f700 1 -- 172.21.15.82:0/390091801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f328c1039f0 msgr2=0x7f328c105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:09.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.192+0000 7f3291a9f700 1 --2- 172.21.15.82:0/390091801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f328c1039f0 0x7f328c105de0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f327c009990 tx=0x7f327c009ca0 comp rx=0 tx=0).stop 2023-09-09T23:13:09.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.192+0000 7f3291a9f700 1 -- 172.21.15.82:0/390091801 shutdown_connections 2023-09-09T23:13:09.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.192+0000 7f3291a9f700 1 --2- 172.21.15.82:0/390091801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f328c1010c0 0x7f328c1034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:09.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.192+0000 7f3291a9f700 1 --2- 172.21.15.82:0/390091801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f328c106320 0x7f328c108730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:09.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.192+0000 7f3291a9f700 1 --2- 172.21.15.82:0/390091801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f328c1039f0 0x7f328c105de0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:09.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.192+0000 7f3291a9f700 1 -- 172.21.15.82:0/390091801 >> 172.21.15.82:0/390091801 conn(0x7f328c0faa30 msgr2=0x7f328c0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:09.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.192+0000 7f3291a9f700 1 -- 172.21.15.82:0/390091801 shutdown_connections 2023-09-09T23:13:09.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.192+0000 7f3291a9f700 1 -- 172.21.15.82:0/390091801 wait complete. 2023-09-09T23:13:09.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.193+0000 7f3291a9f700 1 Processor -- start 2023-09-09T23:13:09.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.193+0000 7f3291a9f700 1 -- start start 2023-09-09T23:13:09.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.193+0000 7f3291a9f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f328c1010c0 0x7f328c100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:09.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.193+0000 7f3291a9f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f328c1039f0 0x7f328c0ff070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:09.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.193+0000 7f3291a9f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f328c106320 0x7f328c0ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:09.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.193+0000 7f3291a9f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c0ffd50 con 0x7f328c1039f0 2023-09-09T23:13:09.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.193+0000 7f3291a9f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c0ffec0 con 0x7f328c106320 2023-09-09T23:13:09.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.193+0000 7f3291a9f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c1a0350 con 0x7f328c1010c0 2023-09-09T23:13:09.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.193+0000 7f328affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f328c1039f0 0x7f328c0ff070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:09.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.193+0000 7f328affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f328c1039f0 0x7f328c0ff070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55094/0 (socket says 172.21.15.82:55094) 2023-09-09T23:13:09.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.193+0000 7f328affd700 1 -- 172.21.15.82:0/2485447324 learned_addr learned my addr 172.21.15.82:0/2485447324 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:09.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.193+0000 7f328b7fe700 1 --2- 172.21.15.82:0/2485447324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f328c1010c0 0x7f328c100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:09.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.193+0000 7f328affd700 1 -- 172.21.15.82:0/2485447324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f328c1010c0 msgr2=0x7f328c100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:09.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.193+0000 7f328affd700 1 --2- 172.21.15.82:0/2485447324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f328c1010c0 0x7f328c100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:09.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.193+0000 7f328affd700 1 -- 172.21.15.82:0/2485447324 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f328c106320 msgr2=0x7f328c0ff5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:09.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.193+0000 7f328affd700 1 --2- 172.21.15.82:0/2485447324 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f328c106320 0x7f328c0ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:09.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.193+0000 7f328affd700 1 -- 172.21.15.82:0/2485447324 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327c0095f0 con 0x7f328c1039f0 2023-09-09T23:13:09.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.193+0000 7f328affd700 1 --2- 172.21.15.82:0/2485447324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f328c1039f0 0x7f328c0ff070 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f327c006010 tx=0x7f327c00f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:09.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.194+0000 7f3288ff9700 1 -- 172.21.15.82:0/2485447324 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c01d070 con 0x7f328c1039f0 2023-09-09T23:13:09.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.194+0000 7f3291a9f700 1 -- 172.21.15.82:0/2485447324 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f328c1a05d0 con 0x7f328c1039f0 2023-09-09T23:13:09.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.194+0000 7f3288ff9700 1 -- 172.21.15.82:0/2485447324 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f327c00fca0 con 0x7f328c1039f0 2023-09-09T23:13:09.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.194+0000 7f3288ff9700 1 -- 172.21.15.82:0/2485447324 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c0176e0 con 0x7f328c1039f0 2023-09-09T23:13:09.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.194+0000 7f3291a9f700 1 -- 172.21.15.82:0/2485447324 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f328c1a09f0 con 0x7f328c1039f0 2023-09-09T23:13:09.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.195+0000 7f3291a9f700 1 -- 172.21.15.82:0/2485447324 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f328c0fc5b0 con 0x7f328c1039f0 2023-09-09T23:13:09.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.196+0000 7f3288ff9700 1 -- 172.21.15.82:0/2485447324 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f327c017840 con 0x7f328c1039f0 2023-09-09T23:13:09.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.197+0000 7f3288ff9700 1 --2- 172.21.15.82:0/2485447324 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f32740a0710 0x7f32740a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:09.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.197+0000 7f328b7fe700 1 --2- 172.21.15.82:0/2485447324 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f32740a0710 0x7f32740a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:09.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.197+0000 7f3288ff9700 1 -- 172.21.15.82:0/2485447324 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f327c0cd280 con 0x7f328c1039f0 2023-09-09T23:13:09.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.197+0000 7f328b7fe700 1 --2- 172.21.15.82:0/2485447324 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f32740a0710 0x7f32740a2bd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3280005fd0 tx=0x7f328000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:09.200 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.200+0000 7f3288ff9700 1 -- 172.21.15.82:0/2485447324 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f327c096e90 con 0x7f328c1039f0 2023-09-09T23:13:09.395 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.398+0000 7f3291a9f700 1 -- 172.21.15.82:0/2485447324 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f328c02d020 con 0x7f328c1039f0 2023-09-09T23:13:09.396 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.399+0000 7f3288ff9700 1 -- 172.21.15.82:0/2485447324 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22411 (secure 0 0 0) 0x7f327c09a4b0 con 0x7f328c1039f0 2023-09-09T23:13:09.397 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:09.397 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":82,"fsid":"3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da","created":"2023-09-09T23:05:14.222190+0000","modified":"2023-09-09T23:13:09.319273+0000","last_up_change":"2023-09-09T23:11:37.253759+0000","last_in_change":"2023-09-09T23:11:19.933812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-09T23:08:31.234958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-09T23:12:15.524887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-09T23:12:16.785403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1277eb17-a2e7-4e10-9832-f12a5e93ffde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6803","nonce":3197434929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6805","nonce":3197434929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6809","nonce":3197434929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6807","nonce":3197434929}]},"public_addr":"172.21.15.82:6803/3197434929","cluster_addr":"172.21.15.82:6805/3197434929","heartbeat_back_addr":"172.21.15.82:6809/3197434929","heartbeat_front_addr":"172.21.15.82:6807/3197434929","state":["exists","up"]},{"osd":1,"uuid":"f68723fd-0908-4e21-ae8e-1de1c3a9a65c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6801","nonce":3528641314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6803","nonce":3528641314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6807","nonce":3528641314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6805","nonce":3528641314}]},"public_addr":"172.21.15.98:6801/3528641314","cluster_addr":"172.21.15.98:6803/3528641314","heartbeat_back_addr":"172.21.15.98:6807/3528641314","heartbeat_front_addr":"172.21.15.98:6805/3528641314","state":["exists","up"]},{"osd":2,"uuid":"06f89c53-1419-4f52-acad-6e92916b476f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6801","nonce":4006414551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6803","nonce":4006414551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6807","nonce":4006414551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6805","nonce":4006414551}]},"public_addr":"172.21.15.170:6801/4006414551","cluster_addr":"172.21.15.170:6803/4006414551","heartbeat_back_addr":"172.21.15.170:6807/4006414551","heartbeat_front_addr":"172.21.15.170:6805/4006414551","state":["exists","up"]},{"osd":3,"uuid":"fedbd4c9-6852-42b5-8f3a-ba8d9bfe5d59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6811","nonce":3647629241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6813","nonce":3647629241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6817","nonce":3647629241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6815","nonce":3647629241}]},"public_addr":"172.21.15.82:6811/3647629241","cluster_addr":"172.21.15.82:6813/3647629241","heartbeat_back_addr":"172.21.15.82:6817/3647629241","heartbeat_front_addr":"172.21.15.82:6815/3647629241","state":["exists","up"]},{"osd":4,"uuid":"a71f3084-6919-4161-87bf-345f3355d8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6809","nonce":2507035643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6811","nonce":2507035643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6815","nonce":2507035643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6813","nonce":2507035643}]},"public_addr":"172.21.15.98:6809/2507035643","cluster_addr":"172.21.15.98:6811/2507035643","heartbeat_back_addr":"172.21.15.98:6815/2507035643","heartbeat_front_addr":"172.21.15.98:6813/2507035643","state":["exists","up"]},{"osd":5,"uuid":"f76058e9-a90e-476e-87f0-3c6723795e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6809","nonce":775005168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6811","nonce":775005168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6815","nonce":775005168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6813","nonce":775005168}]},"public_addr":"172.21.15.170:6809/775005168","cluster_addr":"172.21.15.170:6811/775005168","heartbeat_back_addr":"172.21.15.170:6815/775005168","heartbeat_front_addr":"172.21.15.170:6813/775005168","state":["exists","up"]},{"osd":6,"uuid":"25940be0-c1bd-4c72-95bf-fd8367aa2f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6819","nonce":3008405982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6821","nonce":3008405982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6825","nonce":3008405982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6823","nonce":3008405982}]},"public_addr":"172.21.15.82:6819/3008405982","cluster_addr":"172.21.15.82:6821/3008405982","heartbeat_back_addr":"172.21.15.82:6825/3008405982","heartbeat_front_addr":"172.21.15.82:6823/3008405982","state":["exists","up"]},{"osd":7,"uuid":"f651bb8c-0b25-42d5-b028-58563f4c27f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6817","nonce":3547615284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6819","nonce":3547615284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6823","nonce":3547615284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6821","nonce":3547615284}]},"public_addr":"172.21.15.98:6817/3547615284","cluster_addr":"172.21.15.98:6819/3547615284","heartbeat_back_addr":"172.21.15.98:6823/3547615284","heartbeat_front_addr":"172.21.15.98:6821/3547615284","state":["exists","up"]},{"osd":8,"uuid":"23b27c86-8c98-441e-b7ab-d57bc584e58f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6817","nonce":632038848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6819","nonce":632038848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6823","nonce":632038848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6821","nonce":632038848}]},"public_addr":"172.21.15.170:6817/632038848","cluster_addr":"172.21.15.170:6819/632038848","heartbeat_back_addr":"172.21.15.170:6823/632038848","heartbeat_front_addr":"172.21.15.170:6821/632038848","state":["exists","up"]},{"osd":9,"uuid":"383b8989-4922-4eee-9c77-f198258ac243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6827","nonce":3354242311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6829","nonce":3354242311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6833","nonce":3354242311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6831","nonce":3354242311}]},"public_addr":"172.21.15.82:6827/3354242311","cluster_addr":"172.21.15.82:6829/3354242311","heartbeat_back_addr":"172.21.15.82:6833/3354242311","heartbeat_front_addr":"172.21.15.82:6831/3354242311","state":["exists","up"]},{"osd":10,"uuid":"33a59389-7ced-42d5-bf54-2071317132d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6825","nonce":2069114959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6827","nonce":2069114959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6831","nonce":2069114959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6829","nonce":2069114959}]},"public_addr":"172.21.15.98:6825/2069114959","cluster_addr":"172.21.15.98:6827/2069114959","heartbeat_back_addr":"172.21.15.98:6831/2069114959","heartbeat_front_addr":"172.21.15.98:6829/2069114959","state":["exists","up"]},{"osd":11,"uuid":"b7a71f5c-85e2-4fec-be4b-95be0788b302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6825","nonce":146964512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6827","nonce":146964512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6831","nonce":146964512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6829","nonce":146964512}]},"public_addr":"172.21.15.170:6825/146964512","cluster_addr":"172.21.15.170:6827/146964512","heartbeat_back_addr":"172.21.15.170:6831/146964512","heartbeat_front_addr":"172.21.15.170:6829/146964512","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:07:47.563043+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:08.359260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:29.070374+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:49.897775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:11.434248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:31.538464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:52.377109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:13.452968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:33.806897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:54.083177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:14.980087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:35.690089+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4013747788":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6800/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/2823341464":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/75116806":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/1037625632":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/3142088817":"2023-09-10T23:05:46.550354+0000","172.21.15.82:6800/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/1679622963":"2023-09-10T23:05:46.550354+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-09T23:13:09.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.401+0000 7f3291a9f700 1 -- 172.21.15.82:0/2485447324 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f32740a0710 msgr2=0x7f32740a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:09.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.401+0000 7f3291a9f700 1 --2- 172.21.15.82:0/2485447324 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f32740a0710 0x7f32740a2bd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3280005fd0 tx=0x7f328000a560 comp rx=0 tx=0).stop 2023-09-09T23:13:09.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.401+0000 7f3291a9f700 1 -- 172.21.15.82:0/2485447324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f328c1039f0 msgr2=0x7f328c0ff070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:09.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.401+0000 7f3291a9f700 1 --2- 172.21.15.82:0/2485447324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f328c1039f0 0x7f328c0ff070 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f327c006010 tx=0x7f327c00f7b0 comp rx=0 tx=0).stop 2023-09-09T23:13:09.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.401+0000 7f3291a9f700 1 -- 172.21.15.82:0/2485447324 shutdown_connections 2023-09-09T23:13:09.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.401+0000 7f3291a9f700 1 --2- 172.21.15.82:0/2485447324 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f32740a0710 0x7f32740a2bd0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:09.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.401+0000 7f3291a9f700 1 --2- 172.21.15.82:0/2485447324 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f328c106320 0x7f328c0ff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:09.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.401+0000 7f3291a9f700 1 --2- 172.21.15.82:0/2485447324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f328c1010c0 0x7f328c100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:09.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.401+0000 7f3291a9f700 1 --2- 172.21.15.82:0/2485447324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f328c1039f0 0x7f328c0ff070 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:09.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.401+0000 7f3291a9f700 1 -- 172.21.15.82:0/2485447324 >> 172.21.15.82:0/2485447324 conn(0x7f328c0faa30 msgr2=0x7f328c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:09.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.401+0000 7f3291a9f700 1 -- 172.21.15.82:0/2485447324 shutdown_connections 2023-09-09T23:13:09.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.402+0000 7f3291a9f700 1 -- 172.21.15.82:0/2485447324 wait complete. 2023-09-09T23:13:09.411 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-09T23:08:31.234958+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-09T23:12:15.524887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-09T23:12:16.785403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-09T23:13:09.412 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-09T23:13:09.539 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.539+0000 7f6b74ad9700 1 -- 172.21.15.82:0/89324816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b701006e0 msgr2=0x7f6b70100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:09.539 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.539+0000 7f6b74ad9700 1 --2- 172.21.15.82:0/89324816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b701006e0 0x7f6b70100b40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6b600099c0 tx=0x7f6b60009cd0 comp rx=0 tx=0).stop 2023-09-09T23:13:09.540 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.539+0000 7f6b74ad9700 1 -- 172.21.15.82:0/89324816 shutdown_connections 2023-09-09T23:13:09.540 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.539+0000 7f6b74ad9700 1 --2- 172.21.15.82:0/89324816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b70101080 0x7f6b70107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:09.540 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.539+0000 7f6b74ad9700 1 --2- 172.21.15.82:0/89324816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b700ff4f0 0x7f6b700ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:09.540 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.539+0000 7f6b74ad9700 1 --2- 172.21.15.82:0/89324816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b701006e0 0x7f6b70100b40 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:09.540 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.539+0000 7f6b74ad9700 1 -- 172.21.15.82:0/89324816 >> 172.21.15.82:0/89324816 conn(0x7f6b700faa60 msgr2=0x7f6b700fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:09.541 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.539+0000 7f6b74ad9700 1 -- 172.21.15.82:0/89324816 shutdown_connections 2023-09-09T23:13:09.541 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.539+0000 7f6b74ad9700 1 -- 172.21.15.82:0/89324816 wait complete. 2023-09-09T23:13:09.541 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.540+0000 7f6b74ad9700 1 Processor -- start 2023-09-09T23:13:09.541 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.540+0000 7f6b74ad9700 1 -- start start 2023-09-09T23:13:09.542 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.540+0000 7f6b74ad9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b700ff4f0 0x7f6b70195240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:09.542 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.540+0000 7f6b74ad9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b701006e0 0x7f6b70195780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:09.542 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.540+0000 7f6b74ad9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b70101080 0x7f6b7019c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:09.542 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.540+0000 7f6b74ad9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7019cdc0 con 0x7f6b701006e0 2023-09-09T23:13:09.542 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.540+0000 7f6b74ad9700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7019cf00 con 0x7f6b70101080 2023-09-09T23:13:09.543 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.540+0000 7f6b74ad9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7019d090 con 0x7f6b700ff4f0 2023-09-09T23:13:09.543 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.540+0000 7f6b6dd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b701006e0 0x7f6b70195780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:09.543 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.540+0000 7f6b6dd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b701006e0 0x7f6b70195780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55110/0 (socket says 172.21.15.82:55110) 2023-09-09T23:13:09.543 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.540+0000 7f6b6dd9b700 1 -- 172.21.15.82:0/3841492077 learned_addr learned my addr 172.21.15.82:0/3841492077 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:09.543 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.541+0000 7f6b6e59c700 1 --2- 172.21.15.82:0/3841492077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b700ff4f0 0x7f6b70195240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:09.544 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.541+0000 7f6b6ed9d700 1 --2- 172.21.15.82:0/3841492077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b70101080 0x7f6b7019c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:09.544 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.541+0000 7f6b6dd9b700 1 -- 172.21.15.82:0/3841492077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b700ff4f0 msgr2=0x7f6b70195240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:09.544 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.541+0000 7f6b6dd9b700 1 --2- 172.21.15.82:0/3841492077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b700ff4f0 0x7f6b70195240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:09.544 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.541+0000 7f6b6dd9b700 1 -- 172.21.15.82:0/3841492077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b70101080 msgr2=0x7f6b7019c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:09.544 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.541+0000 7f6b6dd9b700 1 --2- 172.21.15.82:0/3841492077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b70101080 0x7f6b7019c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:09.545 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.541+0000 7f6b6dd9b700 1 -- 172.21.15.82:0/3841492077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b600095f0 con 0x7f6b701006e0 2023-09-09T23:13:09.545 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.541+0000 7f6b6e59c700 1 --2- 172.21.15.82:0/3841492077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b700ff4f0 0x7f6b70195240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:09.545 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.541+0000 7f6b6dd9b700 1 --2- 172.21.15.82:0/3841492077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b701006e0 0x7f6b70195780 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009990 tx=0x7f6b60003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:09.545 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.541+0000 7f6b677fe700 1 -- 172.21.15.82:0/3841492077 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6001d070 con 0x7f6b701006e0 2023-09-09T23:13:09.545 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.541+0000 7f6b677fe700 1 -- 172.21.15.82:0/3841492077 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b60004640 con 0x7f6b701006e0 2023-09-09T23:13:09.546 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.541+0000 7f6b74ad9700 1 -- 172.21.15.82:0/3841492077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b7019d310 con 0x7f6b701006e0 2023-09-09T23:13:09.546 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.541+0000 7f6b677fe700 1 -- 172.21.15.82:0/3841492077 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6000f460 con 0x7f6b701006e0 2023-09-09T23:13:09.546 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.541+0000 7f6b74ad9700 1 -- 172.21.15.82:0/3841492077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b7019d7c0 con 0x7f6b701006e0 2023-09-09T23:13:09.546 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.542+0000 7f6b74ad9700 1 -- 172.21.15.82:0/3841492077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b70106a90 con 0x7f6b701006e0 2023-09-09T23:13:09.547 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.543+0000 7f6b677fe700 1 -- 172.21.15.82:0/3841492077 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f6b60022470 con 0x7f6b701006e0 2023-09-09T23:13:09.547 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.544+0000 7f6b677fe700 1 --2- 172.21.15.82:0/3841492077 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f6b5c0a06d0 0x7f6b5c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:09.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.544+0000 7f6b6e59c700 1 --2- 172.21.15.82:0/3841492077 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f6b5c0a06d0 0x7f6b5c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:09.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.544+0000 7f6b677fe700 1 -- 172.21.15.82:0/3841492077 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f6b600cd9e0 con 0x7f6b701006e0 2023-09-09T23:13:09.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.544+0000 7f6b6e59c700 1 --2- 172.21.15.82:0/3841492077 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f6b5c0a06d0 0x7f6b5c0a2b90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6b58005fd0 tx=0x7f6b58005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:09.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.547+0000 7f6b677fe700 1 -- 172.21.15.82:0/3841492077 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6b60097440 con 0x7f6b701006e0 2023-09-09T23:13:09.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:09 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2004328828' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-09-09T23:13:09.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:09 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-09-09T23:13:09.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:09 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-09-09T23:13:09.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:09 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2023-09-09T23:13:09.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:09 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2023-09-09T23:13:09.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:09 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2023-09-09T23:13:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:09 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2023-09-09T23:13:09.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:09 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2004328828' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-09-09T23:13:09.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-09-09T23:13:09.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-09-09T23:13:09.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2023-09-09T23:13:09.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2023-09-09T23:13:09.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2023-09-09T23:13:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:09 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2023-09-09T23:13:09.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.738+0000 7f6b74ad9700 1 -- 172.21.15.82:0/3841492077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6b7010af10 con 0x7f6b701006e0 2023-09-09T23:13:09.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.739+0000 7f6b677fe700 1 -- 172.21.15.82:0/3841492077 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6b6009aa60 con 0x7f6b701006e0 2023-09-09T23:13:09.737 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2023-09-09T23:13:09.738 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.741+0000 7f6b74ad9700 1 -- 172.21.15.82:0/3841492077 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f6b5c0a06d0 msgr2=0x7f6b5c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:09.739 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.741+0000 7f6b74ad9700 1 --2- 172.21.15.82:0/3841492077 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f6b5c0a06d0 0x7f6b5c0a2b90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6b58005fd0 tx=0x7f6b58005e40 comp rx=0 tx=0).stop 2023-09-09T23:13:09.739 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.741+0000 7f6b74ad9700 1 -- 172.21.15.82:0/3841492077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b701006e0 msgr2=0x7f6b70195780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:09.739 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.741+0000 7f6b74ad9700 1 --2- 172.21.15.82:0/3841492077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b701006e0 0x7f6b70195780 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009990 tx=0x7f6b60003ba0 comp rx=0 tx=0).stop 2023-09-09T23:13:09.739 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.741+0000 7f6b74ad9700 1 -- 172.21.15.82:0/3841492077 shutdown_connections 2023-09-09T23:13:09.739 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.741+0000 7f6b74ad9700 1 --2- 172.21.15.82:0/3841492077 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f6b5c0a06d0 0x7f6b5c0a2b90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:09.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.741+0000 7f6b74ad9700 1 --2- 172.21.15.82:0/3841492077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b70101080 0x7f6b7019c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:09.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.741+0000 7f6b74ad9700 1 --2- 172.21.15.82:0/3841492077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b700ff4f0 0x7f6b70195240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:09.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.741+0000 7f6b74ad9700 1 --2- 172.21.15.82:0/3841492077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b701006e0 0x7f6b70195780 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:09.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.741+0000 7f6b74ad9700 1 -- 172.21.15.82:0/3841492077 >> 172.21.15.82:0/3841492077 conn(0x7f6b700faa60 msgr2=0x7f6b70103bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:09.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.741+0000 7f6b74ad9700 1 -- 172.21.15.82:0/3841492077 shutdown_connections 2023-09-09T23:13:09.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.741+0000 7f6b74ad9700 1 -- 172.21.15.82:0/3841492077 wait complete. 2023-09-09T23:13:09.750 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-09T23:13:09.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:09 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2004328828' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-09-09T23:13:09.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:09 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-09-09T23:13:09.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:09 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-09-09T23:13:09.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:09 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2023-09-09T23:13:09.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:09 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2023-09-09T23:13:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:09 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2023-09-09T23:13:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:09 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2023-09-09T23:13:09.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.879+0000 7f54886f8700 1 -- 172.21.15.82:0/2311253658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5480101080 msgr2=0x7f5480107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:09.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.879+0000 7f54886f8700 1 --2- 172.21.15.82:0/2311253658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5480101080 0x7f5480107930 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f547800b0d0 tx=0x7f547800b3e0 comp rx=0 tx=0).stop 2023-09-09T23:13:09.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.879+0000 7f54886f8700 1 -- 172.21.15.82:0/2311253658 shutdown_connections 2023-09-09T23:13:09.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.879+0000 7f54886f8700 1 --2- 172.21.15.82:0/2311253658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5480101080 0x7f5480107930 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:09.880 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.879+0000 7f54886f8700 1 --2- 172.21.15.82:0/2311253658 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54800ff4f0 0x7f54800ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:09.880 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.879+0000 7f54886f8700 1 --2- 172.21.15.82:0/2311253658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54801006e0 0x7f5480100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:09.880 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.879+0000 7f54886f8700 1 -- 172.21.15.82:0/2311253658 >> 172.21.15.82:0/2311253658 conn(0x7f54800faa60 msgr2=0x7f54800fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:09.880 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.879+0000 7f54886f8700 1 -- 172.21.15.82:0/2311253658 shutdown_connections 2023-09-09T23:13:09.881 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.879+0000 7f54886f8700 1 -- 172.21.15.82:0/2311253658 wait complete. 2023-09-09T23:13:09.881 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.880+0000 7f54886f8700 1 Processor -- start 2023-09-09T23:13:09.881 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.880+0000 7f54886f8700 1 -- start start 2023-09-09T23:13:09.881 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.880+0000 7f54886f8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54800ff4f0 0x7f5480196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:09.881 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.880+0000 7f54886f8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54801006e0 0x7f5480196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:09.882 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.880+0000 7f54886f8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5480101080 0x7f548019d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:09.882 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.880+0000 7f54886f8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548019ddd0 con 0x7f54800ff4f0 2023-09-09T23:13:09.882 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.880+0000 7f54886f8700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548019df10 con 0x7f5480101080 2023-09-09T23:13:09.882 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.880+0000 7f54886f8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548019e0a0 con 0x7f54801006e0 2023-09-09T23:13:09.882 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.880+0000 7f5486494700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54800ff4f0 0x7f5480196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:09.883 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.880+0000 7f5486494700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54800ff4f0 0x7f5480196250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55122/0 (socket says 172.21.15.82:55122) 2023-09-09T23:13:09.883 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.880+0000 7f5486494700 1 -- 172.21.15.82:0/749610583 learned_addr learned my addr 172.21.15.82:0/749610583 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:09.883 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.880+0000 7f5486c95700 1 --2- 172.21.15.82:0/749610583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5480101080 0x7f548019d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:09.883 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.880+0000 7f5486494700 1 -- 172.21.15.82:0/749610583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54801006e0 msgr2=0x7f5480196790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:09.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.880+0000 7f5486494700 1 --2- 172.21.15.82:0/749610583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54801006e0 0x7f5480196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:09.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.880+0000 7f5486494700 1 -- 172.21.15.82:0/749610583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5480101080 msgr2=0x7f548019d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:09.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.880+0000 7f5486494700 1 --2- 172.21.15.82:0/749610583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5480101080 0x7f548019d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:09.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.880+0000 7f5486494700 1 -- 172.21.15.82:0/749610583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5478009d00 con 0x7f54800ff4f0 2023-09-09T23:13:09.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.880+0000 7f5486c95700 1 --2- 172.21.15.82:0/749610583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5480101080 0x7f548019d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:09.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.880+0000 7f5486494700 1 --2- 172.21.15.82:0/749610583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54800ff4f0 0x7f5480196250 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f547000b550 tx=0x7f547000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:09.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.880+0000 7f54777fe700 1 -- 172.21.15.82:0/749610583 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5470011830 con 0x7f54800ff4f0 2023-09-09T23:13:09.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.880+0000 7f54777fe700 1 -- 172.21.15.82:0/749610583 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5470011e50 con 0x7f54800ff4f0 2023-09-09T23:13:09.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.880+0000 7f54886f8700 1 -- 172.21.15.82:0/749610583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f548019e380 con 0x7f54800ff4f0 2023-09-09T23:13:09.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.880+0000 7f54777fe700 1 -- 172.21.15.82:0/749610583 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547000f2e0 con 0x7f54800ff4f0 2023-09-09T23:13:09.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.880+0000 7f54886f8700 1 -- 172.21.15.82:0/749610583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548019e910 con 0x7f54800ff4f0 2023-09-09T23:13:09.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.881+0000 7f54886f8700 1 -- 172.21.15.82:0/749610583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5480107580 con 0x7f54800ff4f0 2023-09-09T23:13:09.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.883+0000 7f54777fe700 1 -- 172.21.15.82:0/749610583 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f5470010580 con 0x7f54800ff4f0 2023-09-09T23:13:09.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.883+0000 7f54777fe700 1 --2- 172.21.15.82:0/749610583 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f546c0a06d0 0x7f546c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:09.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.884+0000 7f5485c93700 1 --2- 172.21.15.82:0/749610583 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f546c0a06d0 0x7f546c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:09.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.884+0000 7f54777fe700 1 -- 172.21.15.82:0/749610583 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f54700cbcb0 con 0x7f54800ff4f0 2023-09-09T23:13:09.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.884+0000 7f5485c93700 1 --2- 172.21.15.82:0/749610583 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f546c0a06d0 0x7f546c0a2b90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f547c005fd0 tx=0x7f547c005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:09.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:09.887+0000 7f54777fe700 1 -- 172.21.15.82:0/749610583 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5470095790 con 0x7f54800ff4f0 2023-09-09T23:13:10.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.079+0000 7f54886f8700 1 -- 172.21.15.82:0/749610583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f548010af30 con 0x7f54800ff4f0 2023-09-09T23:13:10.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.079+0000 7f54777fe700 1 -- 172.21.15.82:0/749610583 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5470098db0 con 0x7f54800ff4f0 2023-09-09T23:13:10.077 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2023-09-09T23:13:10.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.081+0000 7f54886f8700 1 -- 172.21.15.82:0/749610583 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f546c0a06d0 msgr2=0x7f546c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:10.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.081+0000 7f54886f8700 1 --2- 172.21.15.82:0/749610583 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f546c0a06d0 0x7f546c0a2b90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f547c005fd0 tx=0x7f547c005d00 comp rx=0 tx=0).stop 2023-09-09T23:13:10.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.081+0000 7f54886f8700 1 -- 172.21.15.82:0/749610583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54800ff4f0 msgr2=0x7f5480196250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:10.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.081+0000 7f54886f8700 1 --2- 172.21.15.82:0/749610583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54800ff4f0 0x7f5480196250 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f547000b550 tx=0x7f547000b910 comp rx=0 tx=0).stop 2023-09-09T23:13:10.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.081+0000 7f54886f8700 1 -- 172.21.15.82:0/749610583 shutdown_connections 2023-09-09T23:13:10.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.081+0000 7f54886f8700 1 --2- 172.21.15.82:0/749610583 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f546c0a06d0 0x7f546c0a2b90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.081+0000 7f54886f8700 1 --2- 172.21.15.82:0/749610583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5480101080 0x7f548019d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.081+0000 7f54886f8700 1 --2- 172.21.15.82:0/749610583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54800ff4f0 0x7f5480196250 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.081+0000 7f54886f8700 1 --2- 172.21.15.82:0/749610583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54801006e0 0x7f5480196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.081+0000 7f54886f8700 1 -- 172.21.15.82:0/749610583 >> 172.21.15.82:0/749610583 conn(0x7f54800faa60 msgr2=0x7f54800fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:10.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.081+0000 7f54886f8700 1 -- 172.21.15.82:0/749610583 shutdown_connections 2023-09-09T23:13:10.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.082+0000 7f54886f8700 1 -- 172.21.15.82:0/749610583 wait complete. 2023-09-09T23:13:10.091 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-09T23:13:10.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.218+0000 7fedabf4c700 1 -- 172.21.15.82:0/252837403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feda40ff4f0 msgr2=0x7feda40ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:10.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.218+0000 7fedabf4c700 1 --2- 172.21.15.82:0/252837403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feda40ff4f0 0x7feda40ff910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fed940099c0 tx=0x7fed94009cd0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.218+0000 7fedabf4c700 1 -- 172.21.15.82:0/252837403 shutdown_connections 2023-09-09T23:13:10.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.218+0000 7fedabf4c700 1 --2- 172.21.15.82:0/252837403 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feda4101080 0x7feda4107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.218+0000 7fedabf4c700 1 --2- 172.21.15.82:0/252837403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feda40ff4f0 0x7feda40ff910 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.218+0000 7fedabf4c700 1 --2- 172.21.15.82:0/252837403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feda41006e0 0x7feda4100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.218+0000 7fedabf4c700 1 -- 172.21.15.82:0/252837403 >> 172.21.15.82:0/252837403 conn(0x7feda40faa60 msgr2=0x7feda40fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:10.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.218+0000 7fedabf4c700 1 -- 172.21.15.82:0/252837403 shutdown_connections 2023-09-09T23:13:10.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.218+0000 7fedabf4c700 1 -- 172.21.15.82:0/252837403 wait complete. 2023-09-09T23:13:10.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.219+0000 7fedabf4c700 1 Processor -- start 2023-09-09T23:13:10.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.219+0000 7fedabf4c700 1 -- start start 2023-09-09T23:13:10.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.219+0000 7fedabf4c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feda40ff4f0 0x7feda41951a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:10.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.219+0000 7fedabf4c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feda41006e0 0x7feda41956e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:10.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.219+0000 7fedabf4c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feda4101080 0x7feda419c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:10.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.219+0000 7fedabf4c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda419cd20 con 0x7feda4101080 2023-09-09T23:13:10.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.219+0000 7fedabf4c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda419ce60 con 0x7feda41006e0 2023-09-09T23:13:10.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.219+0000 7fedabf4c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda419cff0 con 0x7feda40ff4f0 2023-09-09T23:13:10.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.219+0000 7fedaa4e9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feda4101080 0x7feda419c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:10.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.219+0000 7fedaa4e9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feda4101080 0x7feda419c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55140/0 (socket says 172.21.15.82:55140) 2023-09-09T23:13:10.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.219+0000 7fedaa4e9700 1 -- 172.21.15.82:0/1671377827 learned_addr learned my addr 172.21.15.82:0/1671377827 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:10.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.219+0000 7feda94e7700 1 --2- 172.21.15.82:0/1671377827 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feda41006e0 0x7feda41956e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:10.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.219+0000 7feda9ce8700 1 --2- 172.21.15.82:0/1671377827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feda40ff4f0 0x7feda41951a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:10.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.219+0000 7fedaa4e9700 1 -- 172.21.15.82:0/1671377827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feda40ff4f0 msgr2=0x7feda41951a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:10.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.219+0000 7fedaa4e9700 1 --2- 172.21.15.82:0/1671377827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feda40ff4f0 0x7feda41951a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.219+0000 7fedaa4e9700 1 -- 172.21.15.82:0/1671377827 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feda41006e0 msgr2=0x7feda41956e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:10.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.219+0000 7fedaa4e9700 1 --2- 172.21.15.82:0/1671377827 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feda41006e0 0x7feda41956e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.219+0000 7fedaa4e9700 1 -- 172.21.15.82:0/1671377827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed940095f0 con 0x7feda4101080 2023-09-09T23:13:10.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.220+0000 7fedaa4e9700 1 --2- 172.21.15.82:0/1671377827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feda4101080 0x7feda419c750 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fed9c007d70 tx=0x7fed9c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:10.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.220+0000 7fed9affd700 1 -- 172.21.15.82:0/1671377827 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c00ad40 con 0x7feda4101080 2023-09-09T23:13:10.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.220+0000 7fed9affd700 1 -- 172.21.15.82:0/1671377827 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed9c0128e0 con 0x7feda4101080 2023-09-09T23:13:10.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.220+0000 7fedabf4c700 1 -- 172.21.15.82:0/1671377827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda419d2d0 con 0x7feda4101080 2023-09-09T23:13:10.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.220+0000 7fed9affd700 1 -- 172.21.15.82:0/1671377827 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c011780 con 0x7feda4101080 2023-09-09T23:13:10.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.220+0000 7fedabf4c700 1 -- 172.21.15.82:0/1671377827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda419d7e0 con 0x7feda4101080 2023-09-09T23:13:10.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.221+0000 7fedabf4c700 1 -- 172.21.15.82:0/1671377827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feda4106470 con 0x7feda4101080 2023-09-09T23:13:10.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.222+0000 7fed9affd700 1 -- 172.21.15.82:0/1671377827 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fed9c012a50 con 0x7feda4101080 2023-09-09T23:13:10.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.222+0000 7fed9affd700 1 --2- 172.21.15.82:0/1671377827 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fed900a0750 0x7fed900a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:10.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.222+0000 7feda9ce8700 1 --2- 172.21.15.82:0/1671377827 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fed900a0750 0x7fed900a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:10.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.222+0000 7fed9affd700 1 -- 172.21.15.82:0/1671377827 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fed9c0cc260 con 0x7feda4101080 2023-09-09T23:13:10.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.223+0000 7feda9ce8700 1 --2- 172.21.15.82:0/1671377827 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fed900a0750 0x7fed900a2c10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fed940098e0 tx=0x7fed9400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:10.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.225+0000 7fed9affd700 1 -- 172.21.15.82:0/1671377827 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fed9c095e40 con 0x7feda4101080 2023-09-09T23:13:10.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.427+0000 7fedabf4c700 1 -- 172.21.15.82:0/1671377827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feda410aea0 con 0x7feda4101080 2023-09-09T23:13:10.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.428+0000 7fed9affd700 1 -- 172.21.15.82:0/1671377827 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7fed9c099460 con 0x7feda4101080 2023-09-09T23:13:10.426 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2023-09-09T23:13:10.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.430+0000 7fedabf4c700 1 -- 172.21.15.82:0/1671377827 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fed900a0750 msgr2=0x7fed900a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:10.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.430+0000 7fedabf4c700 1 --2- 172.21.15.82:0/1671377827 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fed900a0750 0x7fed900a2c10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fed940098e0 tx=0x7fed9400b540 comp rx=0 tx=0).stop 2023-09-09T23:13:10.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.430+0000 7fedabf4c700 1 -- 172.21.15.82:0/1671377827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feda4101080 msgr2=0x7feda419c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:10.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.430+0000 7fedabf4c700 1 --2- 172.21.15.82:0/1671377827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feda4101080 0x7feda419c750 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fed9c007d70 tx=0x7fed9c00a340 comp rx=0 tx=0).stop 2023-09-09T23:13:10.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.430+0000 7fedabf4c700 1 -- 172.21.15.82:0/1671377827 shutdown_connections 2023-09-09T23:13:10.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.430+0000 7fedabf4c700 1 --2- 172.21.15.82:0/1671377827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feda4101080 0x7feda419c750 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.430+0000 7fedabf4c700 1 --2- 172.21.15.82:0/1671377827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feda40ff4f0 0x7feda41951a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.430+0000 7fedabf4c700 1 --2- 172.21.15.82:0/1671377827 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fed900a0750 0x7fed900a2c10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.430+0000 7fedabf4c700 1 --2- 172.21.15.82:0/1671377827 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feda41006e0 0x7feda41956e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.430+0000 7fedabf4c700 1 -- 172.21.15.82:0/1671377827 >> 172.21.15.82:0/1671377827 conn(0x7feda40faa60 msgr2=0x7feda40fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:10.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.430+0000 7fedabf4c700 1 -- 172.21.15.82:0/1671377827 shutdown_connections 2023-09-09T23:13:10.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.430+0000 7fedabf4c700 1 -- 172.21.15.82:0/1671377827 wait complete. 2023-09-09T23:13:10.439 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:10 smithi082 ceph-mon[98694]: pgmap v281: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:13:10.439 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:10 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-09-09T23:13:10.439 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:10 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-09-09T23:13:10.440 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:10 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2023-09-09T23:13:10.440 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:10 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2023-09-09T23:13:10.440 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:10 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2023-09-09T23:13:10.440 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:10 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2023-09-09T23:13:10.440 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:10 smithi082 ceph-mon[98694]: osdmap e82: 12 total, 12 up, 12 in 2023-09-09T23:13:10.441 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:10 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2485447324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:10.441 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:10 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3841492077' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:13:10.441 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:10 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/749610583' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-09T23:13:10.442 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-09T23:13:10.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.567+0000 7f8f252f2700 1 -- 172.21.15.82:0/3757515003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f20101940 msgr2=0x7f8f20101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:10.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.567+0000 7f8f252f2700 1 --2- 172.21.15.82:0/3757515003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f20101940 0x7f8f20101da0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8f14009990 tx=0x7f8f14009ca0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.567+0000 7f8f252f2700 1 -- 172.21.15.82:0/3757515003 shutdown_connections 2023-09-09T23:13:10.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.567+0000 7f8f252f2700 1 --2- 172.21.15.82:0/3757515003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f200fe8d0 0x7f8f200fecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.567+0000 7f8f252f2700 1 --2- 172.21.15.82:0/3757515003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f201022e0 0x7f8f20108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.567+0000 7f8f252f2700 1 --2- 172.21.15.82:0/3757515003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f20101940 0x7f8f20101da0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.567+0000 7f8f252f2700 1 -- 172.21.15.82:0/3757515003 >> 172.21.15.82:0/3757515003 conn(0x7f8f200fa1c0 msgr2=0x7f8f200fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:10.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.567+0000 7f8f252f2700 1 -- 172.21.15.82:0/3757515003 shutdown_connections 2023-09-09T23:13:10.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.567+0000 7f8f252f2700 1 -- 172.21.15.82:0/3757515003 wait complete. 2023-09-09T23:13:10.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.567+0000 7f8f252f2700 1 Processor -- start 2023-09-09T23:13:10.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.567+0000 7f8f252f2700 1 -- start start 2023-09-09T23:13:10.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.568+0000 7f8f252f2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f200fe8d0 0x7f8f20196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:10.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.568+0000 7f8f252f2700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f20101940 0x7f8f201967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:10.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.568+0000 7f8f252f2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f201022e0 0x7f8f2019d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:10.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.568+0000 7f8f252f2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2019de00 con 0x7f8f200fe8d0 2023-09-09T23:13:10.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.568+0000 7f8f252f2700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2019df40 con 0x7f8f20101940 2023-09-09T23:13:10.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.568+0000 7f8f252f2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2019e0d0 con 0x7f8f201022e0 2023-09-09T23:13:10.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.568+0000 7f8f1effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f200fe8d0 0x7f8f20196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:10.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.568+0000 7f8f1effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f200fe8d0 0x7f8f20196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55150/0 (socket says 172.21.15.82:55150) 2023-09-09T23:13:10.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.568+0000 7f8f1effd700 1 -- 172.21.15.82:0/4162753226 learned_addr learned my addr 172.21.15.82:0/4162753226 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:10.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.568+0000 7f8f1f7fe700 1 --2- 172.21.15.82:0/4162753226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f201022e0 0x7f8f2019d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:10.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.568+0000 7f8f1effd700 1 -- 172.21.15.82:0/4162753226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f201022e0 msgr2=0x7f8f2019d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:10.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.568+0000 7f8f1effd700 1 --2- 172.21.15.82:0/4162753226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f201022e0 0x7f8f2019d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.568+0000 7f8f1e7fc700 1 --2- 172.21.15.82:0/4162753226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f20101940 0x7f8f201967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:10.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.568+0000 7f8f1effd700 1 -- 172.21.15.82:0/4162753226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f20101940 msgr2=0x7f8f201967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:10.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.568+0000 7f8f1effd700 1 --2- 172.21.15.82:0/4162753226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f20101940 0x7f8f201967c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.568+0000 7f8f1effd700 1 -- 172.21.15.82:0/4162753226 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f140095f0 con 0x7f8f200fe8d0 2023-09-09T23:13:10.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.568+0000 7f8f1effd700 1 --2- 172.21.15.82:0/4162753226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f200fe8d0 0x7f8f20196280 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8f18007ee0 tx=0x7f8f1800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:10.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.568+0000 7f8f07fff700 1 -- 172.21.15.82:0/4162753226 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1800de90 con 0x7f8f200fe8d0 2023-09-09T23:13:10.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.568+0000 7f8f07fff700 1 -- 172.21.15.82:0/4162753226 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f1800f040 con 0x7f8f200fe8d0 2023-09-09T23:13:10.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.568+0000 7f8f07fff700 1 -- 172.21.15.82:0/4162753226 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f18014760 con 0x7f8f200fe8d0 2023-09-09T23:13:10.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.568+0000 7f8f252f2700 1 -- 172.21.15.82:0/4162753226 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f2019e3b0 con 0x7f8f200fe8d0 2023-09-09T23:13:10.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.568+0000 7f8f252f2700 1 -- 172.21.15.82:0/4162753226 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f2019e870 con 0x7f8f200fe8d0 2023-09-09T23:13:10.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.569+0000 7f8f252f2700 1 -- 172.21.15.82:0/4162753226 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f20107cf0 con 0x7f8f200fe8d0 2023-09-09T23:13:10.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.571+0000 7f8f07fff700 1 -- 172.21.15.82:0/4162753226 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f8f18015400 con 0x7f8f200fe8d0 2023-09-09T23:13:10.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.571+0000 7f8f07fff700 1 --2- 172.21.15.82:0/4162753226 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f8f080a0710 0x7f8f080a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:10.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.571+0000 7f8f07fff700 1 -- 172.21.15.82:0/4162753226 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f8f180cbe10 con 0x7f8f200fe8d0 2023-09-09T23:13:10.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.571+0000 7f8f1e7fc700 1 --2- 172.21.15.82:0/4162753226 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f8f080a0710 0x7f8f080a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:10.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.572+0000 7f8f1e7fc700 1 --2- 172.21.15.82:0/4162753226 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f8f080a0710 0x7f8f080a2bd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8f140095c0 tx=0x7f8f14009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:10.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.575+0000 7f8f07fff700 1 -- 172.21.15.82:0/4162753226 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8f18095860 con 0x7f8f200fe8d0 2023-09-09T23:13:10.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:10 smithi170 ceph-mon[102832]: pgmap v281: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:13:10.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:10 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-09-09T23:13:10.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:10 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-09-09T23:13:10.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:10 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2023-09-09T23:13:10.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:10 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2023-09-09T23:13:10.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:10 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2023-09-09T23:13:10.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:10 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2023-09-09T23:13:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:10 smithi170 ceph-mon[102832]: osdmap e82: 12 total, 12 up, 12 in 2023-09-09T23:13:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:10 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2485447324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:10 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3841492077' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:13:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:10 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/749610583' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-09T23:13:10.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:10 smithi098 ceph-mon[103124]: pgmap v281: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:13:10.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:10 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-09-09T23:13:10.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:10 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-09-09T23:13:10.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:10 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2023-09-09T23:13:10.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:10 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2023-09-09T23:13:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:10 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2023-09-09T23:13:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:10 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2023-09-09T23:13:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:10 smithi098 ceph-mon[103124]: osdmap e82: 12 total, 12 up, 12 in 2023-09-09T23:13:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:10 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2485447324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:10 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3841492077' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:13:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:10 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/749610583' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-09T23:13:10.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.813+0000 7f8f252f2700 1 -- 172.21.15.82:0/4162753226 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f2010c040 con 0x7f8f200fe8d0 2023-09-09T23:13:10.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.814+0000 7f8f07fff700 1 -- 172.21.15.82:0/4162753226 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11368 (secure 0 0 0) 0x7f8f18098e80 con 0x7f8f200fe8d0 2023-09-09T23:13:10.811 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:10.811 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2471432684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2471432684},{"type":"v1","addr":"172.21.15.82:6835","nonce":2471432684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3579387137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3579387137},{"type":"v1","addr":"172.21.15.82:6837","nonce":3579387137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-09T23:12:17.586036+0000","modified":"2023-09-09T23:12:25.285958+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24542,"mds_2":24523,"mds_3":24491,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14712":{"gid":14712,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/2127301345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2127301345},{"type":"v1","addr":"172.21.15.82:6839","nonce":2127301345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14724":{"gid":14724,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/4143604488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":4143604488},{"type":"v1","addr":"172.21.15.82:6841","nonce":4143604488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.170:6833/1931961940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1931961940},{"type":"v1","addr":"172.21.15.170:6833","nonce":1931961940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24491":{"gid":24491,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.98:6833/3855146755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3855146755},{"type":"v1","addr":"172.21.15.98:6833","nonce":3855146755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2313861511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2313861511},{"type":"v1","addr":"172.21.15.170:6835","nonce":2313861511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/4006360308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":4006360308},{"type":"v1","addr":"172.21.15.170:6837","nonce":4006360308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1356474828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1356474828},{"type":"v1","addr":"172.21.15.98:6835","nonce":1356474828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/3858369735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3858369735},{"type":"v1","addr":"172.21.15.170:6839","nonce":3858369735}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/2019449840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2019449840},{"type":"v1","addr":"172.21.15.98:6837","nonce":2019449840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2785086371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2785086371},{"type":"v1","addr":"172.21.15.98:6839","nonce":2785086371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-09T23:13:10.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.816+0000 7f8f252f2700 1 -- 172.21.15.82:0/4162753226 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f8f080a0710 msgr2=0x7f8f080a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:10.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.816+0000 7f8f252f2700 1 --2- 172.21.15.82:0/4162753226 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f8f080a0710 0x7f8f080a2bd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8f140095c0 tx=0x7f8f14009530 comp rx=0 tx=0).stop 2023-09-09T23:13:10.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.816+0000 7f8f252f2700 1 -- 172.21.15.82:0/4162753226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f200fe8d0 msgr2=0x7f8f20196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:10.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.816+0000 7f8f252f2700 1 --2- 172.21.15.82:0/4162753226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f200fe8d0 0x7f8f20196280 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8f18007ee0 tx=0x7f8f1800d3b0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.816+0000 7f8f252f2700 1 -- 172.21.15.82:0/4162753226 shutdown_connections 2023-09-09T23:13:10.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.816+0000 7f8f252f2700 1 --2- 172.21.15.82:0/4162753226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f201022e0 0x7f8f2019d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.816+0000 7f8f252f2700 1 --2- 172.21.15.82:0/4162753226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f200fe8d0 0x7f8f20196280 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.816+0000 7f8f252f2700 1 --2- 172.21.15.82:0/4162753226 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f8f080a0710 0x7f8f080a2bd0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.816+0000 7f8f252f2700 1 --2- 172.21.15.82:0/4162753226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f20101940 0x7f8f201967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.816+0000 7f8f252f2700 1 -- 172.21.15.82:0/4162753226 >> 172.21.15.82:0/4162753226 conn(0x7f8f200fa1c0 msgr2=0x7f8f20104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:10.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.816+0000 7f8f252f2700 1 -- 172.21.15.82:0/4162753226 shutdown_connections 2023-09-09T23:13:10.816 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.816+0000 7f8f252f2700 1 -- 172.21.15.82:0/4162753226 wait complete. 2023-09-09T23:13:10.816 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 27 2023-09-09T23:13:10.826 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-09T23:13:10.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.949+0000 7fc0aa8d0700 1 -- 172.21.15.82:0/690847385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0a41000d0 msgr2=0x7fc0a4107970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:10.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.949+0000 7fc0aa8d0700 1 --2- 172.21.15.82:0/690847385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0a41000d0 0x7fc0a4107970 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00b0a0 tx=0x7fc09c00b3b0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.949+0000 7fc0aa8d0700 1 -- 172.21.15.82:0/690847385 shutdown_connections 2023-09-09T23:13:10.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.949+0000 7fc0aa8d0700 1 --2- 172.21.15.82:0/690847385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0a41000d0 0x7fc0a4107970 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.949+0000 7fc0aa8d0700 1 --2- 172.21.15.82:0/690847385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0a4100ac0 0x7fc0a4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.949+0000 7fc0aa8d0700 1 --2- 172.21.15.82:0/690847385 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0a40ff710 0x7fc0a40ffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.949+0000 7fc0aa8d0700 1 -- 172.21.15.82:0/690847385 >> 172.21.15.82:0/690847385 conn(0x7fc0a40fa1c0 msgr2=0x7fc0a40fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:10.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.949+0000 7fc0aa8d0700 1 -- 172.21.15.82:0/690847385 shutdown_connections 2023-09-09T23:13:10.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.949+0000 7fc0aa8d0700 1 -- 172.21.15.82:0/690847385 wait complete. 2023-09-09T23:13:10.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.950+0000 7fc0aa8d0700 1 Processor -- start 2023-09-09T23:13:10.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.950+0000 7fc0aa8d0700 1 -- start start 2023-09-09T23:13:10.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.950+0000 7fc0aa8d0700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0a40ff710 0x7fc0a41962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:10.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.950+0000 7fc0aa8d0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0a41000d0 0x7fc0a4196830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:10.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.950+0000 7fc0aa8d0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0a4100ac0 0x7fc0a419d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:10.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.950+0000 7fc0aa8d0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a419de70 con 0x7fc0a4100ac0 2023-09-09T23:13:10.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.950+0000 7fc0aa8d0700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a419dfb0 con 0x7fc0a40ff710 2023-09-09T23:13:10.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.950+0000 7fc0aa8d0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a419e140 con 0x7fc0a41000d0 2023-09-09T23:13:10.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.950+0000 7fc0a8e6d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0a4100ac0 0x7fc0a419d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:10.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.950+0000 7fc0a8e6d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0a4100ac0 0x7fc0a419d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55166/0 (socket says 172.21.15.82:55166) 2023-09-09T23:13:10.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.950+0000 7fc0a8e6d700 1 -- 172.21.15.82:0/3047946149 learned_addr learned my addr 172.21.15.82:0/3047946149 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:10.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.951+0000 7fc0a3fff700 1 --2- 172.21.15.82:0/3047946149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0a40ff710 0x7fc0a41962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:10.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.951+0000 7fc0a37fe700 1 --2- 172.21.15.82:0/3047946149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0a41000d0 0x7fc0a4196830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:10.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.951+0000 7fc0a8e6d700 1 -- 172.21.15.82:0/3047946149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0a41000d0 msgr2=0x7fc0a4196830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:10.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.951+0000 7fc0a8e6d700 1 --2- 172.21.15.82:0/3047946149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0a41000d0 0x7fc0a4196830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.951+0000 7fc0a8e6d700 1 -- 172.21.15.82:0/3047946149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0a40ff710 msgr2=0x7fc0a41962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:10.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.951+0000 7fc0a8e6d700 1 --2- 172.21.15.82:0/3047946149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0a40ff710 0x7fc0a41962f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:10.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.951+0000 7fc0a8e6d700 1 -- 172.21.15.82:0/3047946149 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc09c009d00 con 0x7fc0a4100ac0 2023-09-09T23:13:10.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.951+0000 7fc0a3fff700 1 --2- 172.21.15.82:0/3047946149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0a40ff710 0x7fc0a41962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:10.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.951+0000 7fc0a37fe700 1 --2- 172.21.15.82:0/3047946149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0a41000d0 0x7fc0a4196830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:10.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.951+0000 7fc0a8e6d700 1 --2- 172.21.15.82:0/3047946149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0a4100ac0 0x7fc0a419d8a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00b830 tx=0x7fc09c004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:10.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.951+0000 7fc0a17fa700 1 -- 172.21.15.82:0/3047946149 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c00e040 con 0x7fc0a4100ac0 2023-09-09T23:13:10.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.951+0000 7fc0a17fa700 1 -- 172.21.15.82:0/3047946149 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc09c01db50 con 0x7fc0a4100ac0 2023-09-09T23:13:10.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.951+0000 7fc0aa8d0700 1 -- 172.21.15.82:0/3047946149 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a419e3c0 con 0x7fc0a4100ac0 2023-09-09T23:13:10.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.951+0000 7fc0a17fa700 1 -- 172.21.15.82:0/3047946149 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c01bb50 con 0x7fc0a4100ac0 2023-09-09T23:13:10.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.951+0000 7fc0aa8d0700 1 -- 172.21.15.82:0/3047946149 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a419e840 con 0x7fc0a4100ac0 2023-09-09T23:13:10.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.952+0000 7fc0aa8d0700 1 -- 172.21.15.82:0/3047946149 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a410ad30 con 0x7fc0a4100ac0 2023-09-09T23:13:10.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.953+0000 7fc0a17fa700 1 -- 172.21.15.82:0/3047946149 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fc09c01dcc0 con 0x7fc0a4100ac0 2023-09-09T23:13:10.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.953+0000 7fc0a17fa700 1 --2- 172.21.15.82:0/3047946149 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fc08c0a0710 0x7fc08c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:10.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.954+0000 7fc0a17fa700 1 -- 172.21.15.82:0/3047946149 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fc09c0cdf20 con 0x7fc0a4100ac0 2023-09-09T23:13:10.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.954+0000 7fc0a3fff700 1 --2- 172.21.15.82:0/3047946149 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fc08c0a0710 0x7fc08c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:10.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.954+0000 7fc0a3fff700 1 --2- 172.21.15.82:0/3047946149 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fc08c0a0710 0x7fc08c0a2bd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4067230 tx=0x7fc09400a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:10.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:10.957+0000 7fc0a17fa700 1 -- 172.21.15.82:0/3047946149 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc09c097a80 con 0x7fc0a4100ac0 2023-09-09T23:13:11.146 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.149+0000 7fc0aa8d0700 1 -- 172.21.15.82:0/3047946149 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0a419ea60 con 0x7fc0a4100ac0 2023-09-09T23:13:11.147 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.150+0000 7fc0a17fa700 1 -- 172.21.15.82:0/3047946149 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22411 (secure 0 0 0) 0x7fc09c09b0a0 con 0x7fc0a4100ac0 2023-09-09T23:13:11.148 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:11.148 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":83,"fsid":"3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da","created":"2023-09-09T23:05:14.222190+0000","modified":"2023-09-09T23:13:10.324596+0000","last_up_change":"2023-09-09T23:11:37.253759+0000","last_in_change":"2023-09-09T23:11:19.933812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-09T23:08:31.234958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-09T23:12:15.524887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-09T23:12:16.785403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1277eb17-a2e7-4e10-9832-f12a5e93ffde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6803","nonce":3197434929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6805","nonce":3197434929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6809","nonce":3197434929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6807","nonce":3197434929}]},"public_addr":"172.21.15.82:6803/3197434929","cluster_addr":"172.21.15.82:6805/3197434929","heartbeat_back_addr":"172.21.15.82:6809/3197434929","heartbeat_front_addr":"172.21.15.82:6807/3197434929","state":["exists","up"]},{"osd":1,"uuid":"f68723fd-0908-4e21-ae8e-1de1c3a9a65c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6801","nonce":3528641314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6803","nonce":3528641314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6807","nonce":3528641314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6805","nonce":3528641314}]},"public_addr":"172.21.15.98:6801/3528641314","cluster_addr":"172.21.15.98:6803/3528641314","heartbeat_back_addr":"172.21.15.98:6807/3528641314","heartbeat_front_addr":"172.21.15.98:6805/3528641314","state":["exists","up"]},{"osd":2,"uuid":"06f89c53-1419-4f52-acad-6e92916b476f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6801","nonce":4006414551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6803","nonce":4006414551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6807","nonce":4006414551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6805","nonce":4006414551}]},"public_addr":"172.21.15.170:6801/4006414551","cluster_addr":"172.21.15.170:6803/4006414551","heartbeat_back_addr":"172.21.15.170:6807/4006414551","heartbeat_front_addr":"172.21.15.170:6805/4006414551","state":["exists","up"]},{"osd":3,"uuid":"fedbd4c9-6852-42b5-8f3a-ba8d9bfe5d59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6811","nonce":3647629241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6813","nonce":3647629241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6817","nonce":3647629241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6815","nonce":3647629241}]},"public_addr":"172.21.15.82:6811/3647629241","cluster_addr":"172.21.15.82:6813/3647629241","heartbeat_back_addr":"172.21.15.82:6817/3647629241","heartbeat_front_addr":"172.21.15.82:6815/3647629241","state":["exists","up"]},{"osd":4,"uuid":"a71f3084-6919-4161-87bf-345f3355d8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6809","nonce":2507035643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6811","nonce":2507035643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6815","nonce":2507035643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6813","nonce":2507035643}]},"public_addr":"172.21.15.98:6809/2507035643","cluster_addr":"172.21.15.98:6811/2507035643","heartbeat_back_addr":"172.21.15.98:6815/2507035643","heartbeat_front_addr":"172.21.15.98:6813/2507035643","state":["exists","up"]},{"osd":5,"uuid":"f76058e9-a90e-476e-87f0-3c6723795e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6809","nonce":775005168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6811","nonce":775005168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6815","nonce":775005168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6813","nonce":775005168}]},"public_addr":"172.21.15.170:6809/775005168","cluster_addr":"172.21.15.170:6811/775005168","heartbeat_back_addr":"172.21.15.170:6815/775005168","heartbeat_front_addr":"172.21.15.170:6813/775005168","state":["exists","up"]},{"osd":6,"uuid":"25940be0-c1bd-4c72-95bf-fd8367aa2f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6819","nonce":3008405982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6821","nonce":3008405982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6825","nonce":3008405982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6823","nonce":3008405982}]},"public_addr":"172.21.15.82:6819/3008405982","cluster_addr":"172.21.15.82:6821/3008405982","heartbeat_back_addr":"172.21.15.82:6825/3008405982","heartbeat_front_addr":"172.21.15.82:6823/3008405982","state":["exists","up"]},{"osd":7,"uuid":"f651bb8c-0b25-42d5-b028-58563f4c27f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6817","nonce":3547615284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6819","nonce":3547615284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6823","nonce":3547615284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6821","nonce":3547615284}]},"public_addr":"172.21.15.98:6817/3547615284","cluster_addr":"172.21.15.98:6819/3547615284","heartbeat_back_addr":"172.21.15.98:6823/3547615284","heartbeat_front_addr":"172.21.15.98:6821/3547615284","state":["exists","up"]},{"osd":8,"uuid":"23b27c86-8c98-441e-b7ab-d57bc584e58f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6817","nonce":632038848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6819","nonce":632038848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6823","nonce":632038848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6821","nonce":632038848}]},"public_addr":"172.21.15.170:6817/632038848","cluster_addr":"172.21.15.170:6819/632038848","heartbeat_back_addr":"172.21.15.170:6823/632038848","heartbeat_front_addr":"172.21.15.170:6821/632038848","state":["exists","up"]},{"osd":9,"uuid":"383b8989-4922-4eee-9c77-f198258ac243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6827","nonce":3354242311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6829","nonce":3354242311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6833","nonce":3354242311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6831","nonce":3354242311}]},"public_addr":"172.21.15.82:6827/3354242311","cluster_addr":"172.21.15.82:6829/3354242311","heartbeat_back_addr":"172.21.15.82:6833/3354242311","heartbeat_front_addr":"172.21.15.82:6831/3354242311","state":["exists","up"]},{"osd":10,"uuid":"33a59389-7ced-42d5-bf54-2071317132d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6825","nonce":2069114959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6827","nonce":2069114959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6831","nonce":2069114959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6829","nonce":2069114959}]},"public_addr":"172.21.15.98:6825/2069114959","cluster_addr":"172.21.15.98:6827/2069114959","heartbeat_back_addr":"172.21.15.98:6831/2069114959","heartbeat_front_addr":"172.21.15.98:6829/2069114959","state":["exists","up"]},{"osd":11,"uuid":"b7a71f5c-85e2-4fec-be4b-95be0788b302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6825","nonce":146964512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6827","nonce":146964512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6831","nonce":146964512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6829","nonce":146964512}]},"public_addr":"172.21.15.170:6825/146964512","cluster_addr":"172.21.15.170:6827/146964512","heartbeat_back_addr":"172.21.15.170:6831/146964512","heartbeat_front_addr":"172.21.15.170:6829/146964512","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:07:47.563043+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:08.359260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:29.070374+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:49.897775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:11.434248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:31.538464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:52.377109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:13.452968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:33.806897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:54.083177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:14.980087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:35.690089+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4013747788":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6800/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/2823341464":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/75116806":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/1037625632":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/3142088817":"2023-09-10T23:05:46.550354+0000","172.21.15.82:6800/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/1679622963":"2023-09-10T23:05:46.550354+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-09T23:13:11.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.152+0000 7fc0aa8d0700 1 -- 172.21.15.82:0/3047946149 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fc08c0a0710 msgr2=0x7fc08c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:11.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.152+0000 7fc0aa8d0700 1 --2- 172.21.15.82:0/3047946149 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fc08c0a0710 0x7fc08c0a2bd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4067230 tx=0x7fc09400a680 comp rx=0 tx=0).stop 2023-09-09T23:13:11.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.152+0000 7fc0aa8d0700 1 -- 172.21.15.82:0/3047946149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0a4100ac0 msgr2=0x7fc0a419d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:11.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.152+0000 7fc0aa8d0700 1 --2- 172.21.15.82:0/3047946149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0a4100ac0 0x7fc0a419d8a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00b830 tx=0x7fc09c004db0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.152+0000 7fc0aa8d0700 1 -- 172.21.15.82:0/3047946149 shutdown_connections 2023-09-09T23:13:11.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.152+0000 7fc0aa8d0700 1 --2- 172.21.15.82:0/3047946149 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fc08c0a0710 0x7fc08c0a2bd0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.152+0000 7fc0aa8d0700 1 --2- 172.21.15.82:0/3047946149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0a40ff710 0x7fc0a41962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.152+0000 7fc0aa8d0700 1 --2- 172.21.15.82:0/3047946149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0a4100ac0 0x7fc0a419d8a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.152+0000 7fc0aa8d0700 1 --2- 172.21.15.82:0/3047946149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0a41000d0 0x7fc0a4196830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.152+0000 7fc0aa8d0700 1 -- 172.21.15.82:0/3047946149 >> 172.21.15.82:0/3047946149 conn(0x7fc0a40fa1c0 msgr2=0x7fc0a40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:11.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.152+0000 7fc0aa8d0700 1 -- 172.21.15.82:0/3047946149 shutdown_connections 2023-09-09T23:13:11.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.152+0000 7fc0aa8d0700 1 -- 172.21.15.82:0/3047946149 wait complete. 2023-09-09T23:13:11.162 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-09T23:08:31.234958+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-09T23:12:15.524887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-09T23:12:16.785403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-09T23:13:11.163 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-09T23:13:11.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.291+0000 7f28b1f49700 1 -- 172.21.15.82:0/121197247 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28ac101080 msgr2=0x7f28ac107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:11.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.291+0000 7f28b1f49700 1 --2- 172.21.15.82:0/121197247 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28ac101080 0x7f28ac107930 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f28a400b0d0 tx=0x7f28a400b3e0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.291+0000 7f28b1f49700 1 -- 172.21.15.82:0/121197247 shutdown_connections 2023-09-09T23:13:11.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.291+0000 7f28b1f49700 1 --2- 172.21.15.82:0/121197247 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28ac101080 0x7f28ac107930 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.292 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.291+0000 7f28b1f49700 1 --2- 172.21.15.82:0/121197247 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28ac0ff4f0 0x7f28ac0ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.292 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.291+0000 7f28b1f49700 1 --2- 172.21.15.82:0/121197247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28ac1006e0 0x7f28ac100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.292 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.291+0000 7f28b1f49700 1 -- 172.21.15.82:0/121197247 >> 172.21.15.82:0/121197247 conn(0x7f28ac0faa60 msgr2=0x7f28ac0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:11.292 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.291+0000 7f28b1f49700 1 -- 172.21.15.82:0/121197247 shutdown_connections 2023-09-09T23:13:11.292 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.291+0000 7f28b1f49700 1 -- 172.21.15.82:0/121197247 wait complete. 2023-09-09T23:13:11.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.291+0000 7f28b1f49700 1 Processor -- start 2023-09-09T23:13:11.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.292+0000 7f28b1f49700 1 -- start start 2023-09-09T23:13:11.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.292+0000 7f28b1f49700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28ac0ff4f0 0x7f28ac103480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:11.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.292+0000 7f28b1f49700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28ac1006e0 0x7f28ac101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:11.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.292+0000 7f28b1f49700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28ac101080 0x7f28ac102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:11.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.292+0000 7f28b1f49700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac1027a0 con 0x7f28ac0ff4f0 2023-09-09T23:13:11.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.292+0000 7f28b1f49700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac102910 con 0x7f28ac1006e0 2023-09-09T23:13:11.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.292+0000 7f28b1f49700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac1a03d0 con 0x7f28ac101080 2023-09-09T23:13:11.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.292+0000 7f28ab7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28ac0ff4f0 0x7f28ac103480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:11.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.292+0000 7f28ab7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28ac0ff4f0 0x7f28ac103480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55176/0 (socket says 172.21.15.82:55176) 2023-09-09T23:13:11.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.292+0000 7f28ab7fe700 1 -- 172.21.15.82:0/671159092 learned_addr learned my addr 172.21.15.82:0/671159092 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:11.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.292+0000 7f28abfff700 1 --2- 172.21.15.82:0/671159092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28ac101080 0x7f28ac102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:11.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.292+0000 7f28aaffd700 1 --2- 172.21.15.82:0/671159092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28ac1006e0 0x7f28ac101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:11.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.292+0000 7f28ab7fe700 1 -- 172.21.15.82:0/671159092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28ac101080 msgr2=0x7f28ac102000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:11.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.292+0000 7f28ab7fe700 1 --2- 172.21.15.82:0/671159092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28ac101080 0x7f28ac102000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.292+0000 7f28ab7fe700 1 -- 172.21.15.82:0/671159092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28ac1006e0 msgr2=0x7f28ac101ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:11.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.292+0000 7f28ab7fe700 1 --2- 172.21.15.82:0/671159092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28ac1006e0 0x7f28ac101ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.292+0000 7f28ab7fe700 1 -- 172.21.15.82:0/671159092 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a4009d00 con 0x7f28ac0ff4f0 2023-09-09T23:13:11.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.292+0000 7f28aaffd700 1 --2- 172.21.15.82:0/671159092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28ac1006e0 0x7f28ac101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:11.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.292+0000 7f28abfff700 1 --2- 172.21.15.82:0/671159092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28ac101080 0x7f28ac102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:11.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.292+0000 7f28ab7fe700 1 --2- 172.21.15.82:0/671159092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28ac0ff4f0 0x7f28ac103480 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f28a000c990 tx=0x7f28a000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:11.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.292+0000 7f28a8ff9700 1 -- 172.21.15.82:0/671159092 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a0007b00 con 0x7f28ac0ff4f0 2023-09-09T23:13:11.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.292+0000 7f28b1f49700 1 -- 172.21.15.82:0/671159092 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28ac1a06b0 con 0x7f28ac0ff4f0 2023-09-09T23:13:11.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.293+0000 7f28a8ff9700 1 -- 172.21.15.82:0/671159092 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28a0007c60 con 0x7f28ac0ff4f0 2023-09-09T23:13:11.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.293+0000 7f28a8ff9700 1 -- 172.21.15.82:0/671159092 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a0018660 con 0x7f28ac0ff4f0 2023-09-09T23:13:11.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.293+0000 7f28b1f49700 1 -- 172.21.15.82:0/671159092 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28ac1a0c10 con 0x7f28ac0ff4f0 2023-09-09T23:13:11.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.294+0000 7f28b1f49700 1 -- 172.21.15.82:0/671159092 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28ac10ad60 con 0x7f28ac0ff4f0 2023-09-09T23:13:11.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.295+0000 7f28a8ff9700 1 -- 172.21.15.82:0/671159092 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f28a000f450 con 0x7f28ac0ff4f0 2023-09-09T23:13:11.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.295+0000 7f28a8ff9700 1 --2- 172.21.15.82:0/671159092 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f28940a06d0 0x7f28940a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:11.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.295+0000 7f28aaffd700 1 --2- 172.21.15.82:0/671159092 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f28940a06d0 0x7f28940a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:11.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.295+0000 7f28a8ff9700 1 -- 172.21.15.82:0/671159092 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f28a00cbd60 con 0x7f28ac0ff4f0 2023-09-09T23:13:11.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.296+0000 7f28aaffd700 1 --2- 172.21.15.82:0/671159092 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f28940a06d0 0x7f28940a2b90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f289c009be0 tx=0x7f289c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:11.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.299+0000 7f28a8ff9700 1 -- 172.21.15.82:0/671159092 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f28a00958c0 con 0x7f28ac0ff4f0 2023-09-09T23:13:11.454 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:11 smithi082 ceph-mon[98694]: osdmap e83: 12 total, 12 up, 12 in 2023-09-09T23:13:11.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:11 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1671377827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-09T23:13:11.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:11 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/4162753226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:13:11.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:11 smithi082 ceph-mon[98694]: pgmap v284: 97 pgs: 2 peering, 95 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-09-09T23:13:11.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:11 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3047946149' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:11.490 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.493+0000 7f28b1f49700 1 -- 172.21.15.82:0/671159092 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f28ac1a0e90 con 0x7f28ac0ff4f0 2023-09-09T23:13:11.491 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.493+0000 7f28a8ff9700 1 -- 172.21.15.82:0/671159092 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f28a0098ee0 con 0x7f28ac0ff4f0 2023-09-09T23:13:11.491 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2023-09-09T23:13:11.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.496+0000 7f28b1f49700 1 -- 172.21.15.82:0/671159092 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f28940a06d0 msgr2=0x7f28940a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:11.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.496+0000 7f28b1f49700 1 --2- 172.21.15.82:0/671159092 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f28940a06d0 0x7f28940a2b90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f289c009be0 tx=0x7f289c009260 comp rx=0 tx=0).stop 2023-09-09T23:13:11.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.496+0000 7f28b1f49700 1 -- 172.21.15.82:0/671159092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28ac0ff4f0 msgr2=0x7f28ac103480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:11.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.496+0000 7f28b1f49700 1 --2- 172.21.15.82:0/671159092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28ac0ff4f0 0x7f28ac103480 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f28a000c990 tx=0x7f28a000cd50 comp rx=0 tx=0).stop 2023-09-09T23:13:11.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.496+0000 7f28b1f49700 1 -- 172.21.15.82:0/671159092 shutdown_connections 2023-09-09T23:13:11.495 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.496+0000 7f28b1f49700 1 --2- 172.21.15.82:0/671159092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28ac101080 0x7f28ac102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.495 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.496+0000 7f28b1f49700 1 --2- 172.21.15.82:0/671159092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28ac0ff4f0 0x7f28ac103480 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.495 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.496+0000 7f28b1f49700 1 --2- 172.21.15.82:0/671159092 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f28940a06d0 0x7f28940a2b90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.495 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.496+0000 7f28b1f49700 1 --2- 172.21.15.82:0/671159092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28ac1006e0 0x7f28ac101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.495 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.496+0000 7f28b1f49700 1 -- 172.21.15.82:0/671159092 >> 172.21.15.82:0/671159092 conn(0x7f28ac0faa60 msgr2=0x7f28ac067780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:11.496 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.496+0000 7f28b1f49700 1 -- 172.21.15.82:0/671159092 shutdown_connections 2023-09-09T23:13:11.496 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.496+0000 7f28b1f49700 1 -- 172.21.15.82:0/671159092 wait complete. 2023-09-09T23:13:11.506 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-09T23:13:11.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.633+0000 7f13489fe700 1 -- 172.21.15.82:0/2160091768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1344101800 msgr2=0x7f1344101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:11.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.633+0000 7f13489fe700 1 --2- 172.21.15.82:0/2160091768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1344101800 0x7f1344101c60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f13340099c0 tx=0x7f1334009cd0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.633+0000 7f13489fe700 1 -- 172.21.15.82:0/2160091768 shutdown_connections 2023-09-09T23:13:11.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.633+0000 7f13489fe700 1 --2- 172.21.15.82:0/2160091768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1344100610 0x7f1344100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.633+0000 7f13489fe700 1 --2- 172.21.15.82:0/2160091768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13441021a0 0x7f1344108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.633+0000 7f13489fe700 1 --2- 172.21.15.82:0/2160091768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1344101800 0x7f1344101c60 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.633+0000 7f13489fe700 1 -- 172.21.15.82:0/2160091768 >> 172.21.15.82:0/2160091768 conn(0x7f13440fbb80 msgr2=0x7f13440fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:11.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.633+0000 7f13489fe700 1 -- 172.21.15.82:0/2160091768 shutdown_connections 2023-09-09T23:13:11.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.633+0000 7f13489fe700 1 -- 172.21.15.82:0/2160091768 wait complete. 2023-09-09T23:13:11.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.634+0000 7f13489fe700 1 Processor -- start 2023-09-09T23:13:11.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.634+0000 7f13489fe700 1 -- start start 2023-09-09T23:13:11.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.634+0000 7f13489fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1344100610 0x7f13440743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:11.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.634+0000 7f13489fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1344101800 0x7f13440729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:11.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.634+0000 7f13489fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13441021a0 0x7f1344072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:11.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.634+0000 7f13489fe700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13440736d0 con 0x7f1344100610 2023-09-09T23:13:11.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.634+0000 7f13489fe700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1344073840 con 0x7f1344101800 2023-09-09T23:13:11.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.634+0000 7f13489fe700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13441a0330 con 0x7f13441021a0 2023-09-09T23:13:11.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.634+0000 7f134259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1344100610 0x7f13440743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:11.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.634+0000 7f134259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1344100610 0x7f13440743b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55190/0 (socket says 172.21.15.82:55190) 2023-09-09T23:13:11.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.634+0000 7f134259c700 1 -- 172.21.15.82:0/1614917270 learned_addr learned my addr 172.21.15.82:0/1614917270 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:11.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.635+0000 7f1342d9d700 1 --2- 172.21.15.82:0/1614917270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13441021a0 0x7f1344072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:11.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.635+0000 7f1341d9b700 1 --2- 172.21.15.82:0/1614917270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1344101800 0x7f13440729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:11.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.635+0000 7f134259c700 1 -- 172.21.15.82:0/1614917270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13441021a0 msgr2=0x7f1344072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:11.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.635+0000 7f134259c700 1 --2- 172.21.15.82:0/1614917270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13441021a0 0x7f1344072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.635+0000 7f134259c700 1 -- 172.21.15.82:0/1614917270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1344101800 msgr2=0x7f13440729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:11.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.635+0000 7f134259c700 1 --2- 172.21.15.82:0/1614917270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1344101800 0x7f13440729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.635+0000 7f134259c700 1 -- 172.21.15.82:0/1614917270 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13340095f0 con 0x7f1344100610 2023-09-09T23:13:11.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.635+0000 7f1341d9b700 1 --2- 172.21.15.82:0/1614917270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1344101800 0x7f13440729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:11.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.635+0000 7f134259c700 1 --2- 172.21.15.82:0/1614917270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1344100610 0x7f13440743b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f133800c970 tx=0x7f133800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:11.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.635+0000 7f13337fe700 1 -- 172.21.15.82:0/1614917270 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1338004220 con 0x7f1344100610 2023-09-09T23:13:11.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.635+0000 7f13337fe700 1 -- 172.21.15.82:0/1614917270 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1338007d60 con 0x7f1344100610 2023-09-09T23:13:11.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.635+0000 7f13489fe700 1 -- 172.21.15.82:0/1614917270 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13441a0610 con 0x7f1344100610 2023-09-09T23:13:11.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.635+0000 7f13337fe700 1 -- 172.21.15.82:0/1614917270 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1338003c90 con 0x7f1344100610 2023-09-09T23:13:11.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.635+0000 7f13489fe700 1 -- 172.21.15.82:0/1614917270 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13441a0b70 con 0x7f1344100610 2023-09-09T23:13:11.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.636+0000 7f13489fe700 1 -- 172.21.15.82:0/1614917270 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f134410be30 con 0x7f1344100610 2023-09-09T23:13:11.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.637+0000 7f13337fe700 1 -- 172.21.15.82:0/1614917270 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f1338004380 con 0x7f1344100610 2023-09-09T23:13:11.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.638+0000 7f13337fe700 1 --2- 172.21.15.82:0/1614917270 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f132c0a07d0 0x7f132c0a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:11.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.638+0000 7f1341d9b700 1 --2- 172.21.15.82:0/1614917270 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f132c0a07d0 0x7f132c0a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:11.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.638+0000 7f13337fe700 1 -- 172.21.15.82:0/1614917270 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f13380cbc20 con 0x7f1344100610 2023-09-09T23:13:11.642 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.638+0000 7f1341d9b700 1 --2- 172.21.15.82:0/1614917270 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f132c0a07d0 0x7f132c0a2c90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1334000c00 tx=0x7f1334005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:11.642 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.641+0000 7f13337fe700 1 -- 172.21.15.82:0/1614917270 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1338095670 con 0x7f1344100610 2023-09-09T23:13:11.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:11 smithi170 ceph-mon[102832]: osdmap e83: 12 total, 12 up, 12 in 2023-09-09T23:13:11.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:11 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1671377827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-09T23:13:11.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:11 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/4162753226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:13:11.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:11 smithi170 ceph-mon[102832]: pgmap v284: 97 pgs: 2 peering, 95 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-09-09T23:13:11.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:11 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3047946149' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:11.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:11 smithi098 ceph-mon[103124]: osdmap e83: 12 total, 12 up, 12 in 2023-09-09T23:13:11.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:11 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1671377827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-09T23:13:11.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:11 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/4162753226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:13:11.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:11 smithi098 ceph-mon[103124]: pgmap v284: 97 pgs: 2 peering, 95 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-09-09T23:13:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:11 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3047946149' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:11.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.830+0000 7f13489fe700 1 -- 172.21.15.82:0/1614917270 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1344107550 con 0x7f1344100610 2023-09-09T23:13:11.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.830+0000 7f13337fe700 1 -- 172.21.15.82:0/1614917270 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f133801d020 con 0x7f1344100610 2023-09-09T23:13:11.828 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2023-09-09T23:13:11.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.832+0000 7f13489fe700 1 -- 172.21.15.82:0/1614917270 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f132c0a07d0 msgr2=0x7f132c0a2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:11.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.832+0000 7f13489fe700 1 --2- 172.21.15.82:0/1614917270 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f132c0a07d0 0x7f132c0a2c90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1334000c00 tx=0x7f1334005fb0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.832+0000 7f13489fe700 1 -- 172.21.15.82:0/1614917270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1344100610 msgr2=0x7f13440743b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:11.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.832+0000 7f13489fe700 1 --2- 172.21.15.82:0/1614917270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1344100610 0x7f13440743b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f133800c970 tx=0x7f133800cc80 comp rx=0 tx=0).stop 2023-09-09T23:13:11.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.832+0000 7f13489fe700 1 -- 172.21.15.82:0/1614917270 shutdown_connections 2023-09-09T23:13:11.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.832+0000 7f13489fe700 1 --2- 172.21.15.82:0/1614917270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13441021a0 0x7f1344072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.832+0000 7f13489fe700 1 --2- 172.21.15.82:0/1614917270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1344100610 0x7f13440743b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.832+0000 7f13489fe700 1 --2- 172.21.15.82:0/1614917270 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f132c0a07d0 0x7f132c0a2c90 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.832+0000 7f13489fe700 1 --2- 172.21.15.82:0/1614917270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1344101800 0x7f13440729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.832+0000 7f13489fe700 1 -- 172.21.15.82:0/1614917270 >> 172.21.15.82:0/1614917270 conn(0x7f13440fbb80 msgr2=0x7f13440fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:11.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.832+0000 7f13489fe700 1 -- 172.21.15.82:0/1614917270 shutdown_connections 2023-09-09T23:13:11.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.832+0000 7f13489fe700 1 -- 172.21.15.82:0/1614917270 wait complete. 2023-09-09T23:13:11.842 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-09T23:13:11.968 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.971+0000 7fd5281ac700 1 -- 172.21.15.82:0/187478391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5201022f0 msgr2=0x7fd520108ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:11.969 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.971+0000 7fd5281ac700 1 --2- 172.21.15.82:0/187478391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5201022f0 0x7fd520108ba0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd51800b0a0 tx=0x7fd51800b3b0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.969 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.971+0000 7fd5281ac700 1 -- 172.21.15.82:0/187478391 shutdown_connections 2023-09-09T23:13:11.969 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.971+0000 7fd5281ac700 1 --2- 172.21.15.82:0/187478391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5200fe8e0 0x7fd5200fed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.970 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.971+0000 7fd5281ac700 1 --2- 172.21.15.82:0/187478391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5201022f0 0x7fd520108ba0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.970 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.971+0000 7fd5281ac700 1 --2- 172.21.15.82:0/187478391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd520101950 0x7fd520101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.970 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.971+0000 7fd5281ac700 1 -- 172.21.15.82:0/187478391 >> 172.21.15.82:0/187478391 conn(0x7fd5200fa1f0 msgr2=0x7fd5200fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:11.970 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.971+0000 7fd5281ac700 1 -- 172.21.15.82:0/187478391 shutdown_connections 2023-09-09T23:13:11.970 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.971+0000 7fd5281ac700 1 -- 172.21.15.82:0/187478391 wait complete. 2023-09-09T23:13:11.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.972+0000 7fd5281ac700 1 Processor -- start 2023-09-09T23:13:11.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.972+0000 7fd5281ac700 1 -- start start 2023-09-09T23:13:11.974 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.972+0000 7fd5281ac700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5200fe8e0 0x7fd520196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:11.974 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.972+0000 7fd5281ac700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd520101950 0x7fd520196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:11.974 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.972+0000 7fd5281ac700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5201022f0 0x7fd52019d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:11.974 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.972+0000 7fd5281ac700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52019df80 con 0x7fd5201022f0 2023-09-09T23:13:11.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.972+0000 7fd5281ac700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52019e0c0 con 0x7fd520101950 2023-09-09T23:13:11.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.972+0000 7fd5281ac700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52019e250 con 0x7fd5200fe8e0 2023-09-09T23:13:11.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.972+0000 7fd526749700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5201022f0 0x7fd52019d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:11.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.972+0000 7fd526749700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5201022f0 0x7fd52019d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55208/0 (socket says 172.21.15.82:55208) 2023-09-09T23:13:11.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.972+0000 7fd526749700 1 -- 172.21.15.82:0/3830493004 learned_addr learned my addr 172.21.15.82:0/3830493004 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:11.976 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.972+0000 7fd525f48700 1 --2- 172.21.15.82:0/3830493004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5200fe8e0 0x7fd520196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:11.976 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.972+0000 7fd525747700 1 --2- 172.21.15.82:0/3830493004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd520101950 0x7fd520196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:11.976 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.972+0000 7fd526749700 1 -- 172.21.15.82:0/3830493004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5200fe8e0 msgr2=0x7fd520196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:11.976 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.972+0000 7fd526749700 1 --2- 172.21.15.82:0/3830493004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5200fe8e0 0x7fd520196400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.976 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.972+0000 7fd526749700 1 -- 172.21.15.82:0/3830493004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd520101950 msgr2=0x7fd520196940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:11.977 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.972+0000 7fd526749700 1 --2- 172.21.15.82:0/3830493004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd520101950 0x7fd520196940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:11.977 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.972+0000 7fd526749700 1 -- 172.21.15.82:0/3830493004 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd518009d00 con 0x7fd5201022f0 2023-09-09T23:13:11.977 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.972+0000 7fd525f48700 1 --2- 172.21.15.82:0/3830493004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5200fe8e0 0x7fd520196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:11.977 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.972+0000 7fd526749700 1 --2- 172.21.15.82:0/3830493004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5201022f0 0x7fd52019d9b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd5180062a0 tx=0x7fd518004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:11.977 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.973+0000 7fd516ffd700 1 -- 172.21.15.82:0/3830493004 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51800e040 con 0x7fd5201022f0 2023-09-09T23:13:11.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.973+0000 7fd516ffd700 1 -- 172.21.15.82:0/3830493004 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd51801db50 con 0x7fd5201022f0 2023-09-09T23:13:11.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.973+0000 7fd5281ac700 1 -- 172.21.15.82:0/3830493004 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd52019e4d0 con 0x7fd5201022f0 2023-09-09T23:13:11.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.973+0000 7fd516ffd700 1 -- 172.21.15.82:0/3830493004 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51801bc40 con 0x7fd5201022f0 2023-09-09T23:13:11.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.973+0000 7fd5281ac700 1 -- 172.21.15.82:0/3830493004 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd52019e9d0 con 0x7fd5201022f0 2023-09-09T23:13:11.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.974+0000 7fd5281ac700 1 -- 172.21.15.82:0/3830493004 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd52010be90 con 0x7fd5201022f0 2023-09-09T23:13:11.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.975+0000 7fd516ffd700 1 -- 172.21.15.82:0/3830493004 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fd51801d690 con 0x7fd5201022f0 2023-09-09T23:13:11.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.976+0000 7fd516ffd700 1 --2- 172.21.15.82:0/3830493004 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd50c0a06d0 0x7fd50c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:11.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.976+0000 7fd525f48700 1 --2- 172.21.15.82:0/3830493004 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd50c0a06d0 0x7fd50c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:11.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.976+0000 7fd516ffd700 1 -- 172.21.15.82:0/3830493004 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fd5180cdfd0 con 0x7fd5201022f0 2023-09-09T23:13:11.980 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.976+0000 7fd525f48700 1 --2- 172.21.15.82:0/3830493004 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd50c0a06d0 0x7fd50c0a2b90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd510005950 tx=0x7fd5100058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:11.980 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:11.979+0000 7fd516ffd700 1 -- 172.21.15.82:0/3830493004 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd518097b30 con 0x7fd5201022f0 2023-09-09T23:13:12.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.169+0000 7fd5281ac700 1 -- 172.21.15.82:0/3830493004 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd5201075b0 con 0x7fd5201022f0 2023-09-09T23:13:12.167 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.170+0000 7fd516ffd700 1 -- 172.21.15.82:0/3830493004 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd518021020 con 0x7fd5201022f0 2023-09-09T23:13:12.167 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2023-09-09T23:13:12.169 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.172+0000 7fd5281ac700 1 -- 172.21.15.82:0/3830493004 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd50c0a06d0 msgr2=0x7fd50c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:12.169 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.172+0000 7fd5281ac700 1 --2- 172.21.15.82:0/3830493004 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd50c0a06d0 0x7fd50c0a2b90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd510005950 tx=0x7fd5100058e0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.172+0000 7fd5281ac700 1 -- 172.21.15.82:0/3830493004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5201022f0 msgr2=0x7fd52019d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:12.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.172+0000 7fd5281ac700 1 --2- 172.21.15.82:0/3830493004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5201022f0 0x7fd52019d9b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd5180062a0 tx=0x7fd518004db0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.172+0000 7fd5281ac700 1 -- 172.21.15.82:0/3830493004 shutdown_connections 2023-09-09T23:13:12.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.172+0000 7fd5281ac700 1 --2- 172.21.15.82:0/3830493004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5201022f0 0x7fd52019d9b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.172+0000 7fd5281ac700 1 --2- 172.21.15.82:0/3830493004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5200fe8e0 0x7fd520196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.171 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.172+0000 7fd5281ac700 1 --2- 172.21.15.82:0/3830493004 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd50c0a06d0 0x7fd50c0a2b90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.171 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.172+0000 7fd5281ac700 1 --2- 172.21.15.82:0/3830493004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd520101950 0x7fd520196940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.171 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.172+0000 7fd5281ac700 1 -- 172.21.15.82:0/3830493004 >> 172.21.15.82:0/3830493004 conn(0x7fd5200fa1f0 msgr2=0x7fd5200fc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:12.172 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.172+0000 7fd5281ac700 1 -- 172.21.15.82:0/3830493004 shutdown_connections 2023-09-09T23:13:12.172 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.172+0000 7fd5281ac700 1 -- 172.21.15.82:0/3830493004 wait complete. 2023-09-09T23:13:12.181 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-09T23:13:12.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.305+0000 7fcf4fcd2700 1 -- 172.21.15.82:0/2119672136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf48102190 msgr2=0x7fcf48108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:12.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.305+0000 7fcf4fcd2700 1 --2- 172.21.15.82:0/2119672136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf48102190 0x7fcf48108a40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000b0d0 tx=0x7fcf4000b3e0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.305+0000 7fcf4fcd2700 1 -- 172.21.15.82:0/2119672136 shutdown_connections 2023-09-09T23:13:12.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.305+0000 7fcf4fcd2700 1 --2- 172.21.15.82:0/2119672136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf48102190 0x7fcf48108a40 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.306 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.305+0000 7fcf4fcd2700 1 --2- 172.21.15.82:0/2119672136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf48100600 0x7fcf48100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.306 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.305+0000 7fcf4fcd2700 1 --2- 172.21.15.82:0/2119672136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf481017f0 0x7fcf48101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.306 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.305+0000 7fcf4fcd2700 1 -- 172.21.15.82:0/2119672136 >> 172.21.15.82:0/2119672136 conn(0x7fcf480fbb50 msgr2=0x7fcf480fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:12.306 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.306+0000 7fcf4fcd2700 1 -- 172.21.15.82:0/2119672136 shutdown_connections 2023-09-09T23:13:12.306 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.306+0000 7fcf4fcd2700 1 -- 172.21.15.82:0/2119672136 wait complete. 2023-09-09T23:13:12.307 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.306+0000 7fcf4fcd2700 1 Processor -- start 2023-09-09T23:13:12.307 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.306+0000 7fcf4fcd2700 1 -- start start 2023-09-09T23:13:12.307 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.306+0000 7fcf4fcd2700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf48100600 0x7fcf480743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:12.307 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.306+0000 7fcf4fcd2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf481017f0 0x7fcf480729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:12.307 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.306+0000 7fcf4fcd2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf48102190 0x7fcf48072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:12.308 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.306+0000 7fcf4fcd2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf480736d0 con 0x7fcf481017f0 2023-09-09T23:13:12.308 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.306+0000 7fcf4fcd2700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf48073840 con 0x7fcf48100600 2023-09-09T23:13:12.308 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.306+0000 7fcf4fcd2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf481a0390 con 0x7fcf48102190 2023-09-09T23:13:12.308 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.306+0000 7fcf4d26d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf481017f0 0x7fcf480729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:12.308 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.306+0000 7fcf4d26d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf481017f0 0x7fcf480729f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55220/0 (socket says 172.21.15.82:55220) 2023-09-09T23:13:12.309 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.306+0000 7fcf4d26d700 1 -- 172.21.15.82:0/1629733169 learned_addr learned my addr 172.21.15.82:0/1629733169 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:12.309 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.307+0000 7fcf4e26f700 1 --2- 172.21.15.82:0/1629733169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf48102190 0x7fcf48072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:12.309 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.307+0000 7fcf4da6e700 1 --2- 172.21.15.82:0/1629733169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf48100600 0x7fcf480743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:12.309 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.307+0000 7fcf4d26d700 1 -- 172.21.15.82:0/1629733169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf48102190 msgr2=0x7fcf48072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:12.309 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.307+0000 7fcf4d26d700 1 --2- 172.21.15.82:0/1629733169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf48102190 0x7fcf48072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.310 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.307+0000 7fcf4d26d700 1 -- 172.21.15.82:0/1629733169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf48100600 msgr2=0x7fcf480743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:12.310 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.307+0000 7fcf4d26d700 1 --2- 172.21.15.82:0/1629733169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf48100600 0x7fcf480743b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.310 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.307+0000 7fcf4d26d700 1 -- 172.21.15.82:0/1629733169 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf40009d00 con 0x7fcf481017f0 2023-09-09T23:13:12.310 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.307+0000 7fcf4e26f700 1 --2- 172.21.15.82:0/1629733169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf48102190 0x7fcf48072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:12.310 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.307+0000 7fcf4d26d700 1 --2- 172.21.15.82:0/1629733169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf481017f0 0x7fcf480729f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800c9b0 tx=0x7fcf3800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:12.311 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.307+0000 7fcf3effd700 1 -- 172.21.15.82:0/1629733169 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf38007b00 con 0x7fcf481017f0 2023-09-09T23:13:12.311 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.307+0000 7fcf3effd700 1 -- 172.21.15.82:0/1629733169 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf38007c60 con 0x7fcf481017f0 2023-09-09T23:13:12.311 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.307+0000 7fcf4fcd2700 1 -- 172.21.15.82:0/1629733169 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf481a0670 con 0x7fcf481017f0 2023-09-09T23:13:12.311 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.307+0000 7fcf3effd700 1 -- 172.21.15.82:0/1629733169 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf38018660 con 0x7fcf481017f0 2023-09-09T23:13:12.311 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.307+0000 7fcf4fcd2700 1 -- 172.21.15.82:0/1629733169 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf481a0b80 con 0x7fcf481017f0 2023-09-09T23:13:12.312 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.308+0000 7fcf4fcd2700 1 -- 172.21.15.82:0/1629733169 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf48107a50 con 0x7fcf481017f0 2023-09-09T23:13:12.313 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.309+0000 7fcf3effd700 1 -- 172.21.15.82:0/1629733169 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fcf38018880 con 0x7fcf481017f0 2023-09-09T23:13:12.313 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.310+0000 7fcf3effd700 1 --2- 172.21.15.82:0/1629733169 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcf340a0710 0x7fcf340a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:12.313 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.310+0000 7fcf4da6e700 1 --2- 172.21.15.82:0/1629733169 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcf340a0710 0x7fcf340a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:12.313 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.310+0000 7fcf3effd700 1 -- 172.21.15.82:0/1629733169 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fcf380cbfe0 con 0x7fcf481017f0 2023-09-09T23:13:12.313 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.310+0000 7fcf4da6e700 1 --2- 172.21.15.82:0/1629733169 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcf340a0710 0x7fcf340a2bd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcf44005950 tx=0x7fcf440058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:12.314 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.313+0000 7fcf3effd700 1 -- 172.21.15.82:0/1629733169 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fcf38095a80 con 0x7fcf481017f0 2023-09-09T23:13:12.454 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:12 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/671159092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:13:12.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:12 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1614917270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-09T23:13:12.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:12 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3830493004' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-09T23:13:12.546 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.549+0000 7fcf4fcd2700 1 -- 172.21.15.82:0/1629733169 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf4802cfa0 con 0x7fcf481017f0 2023-09-09T23:13:12.547 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.550+0000 7fcf3effd700 1 -- 172.21.15.82:0/1629733169 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11368 (secure 0 0 0) 0x7fcf380990a0 con 0x7fcf481017f0 2023-09-09T23:13:12.547 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:12.548 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2471432684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2471432684},{"type":"v1","addr":"172.21.15.82:6835","nonce":2471432684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3579387137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3579387137},{"type":"v1","addr":"172.21.15.82:6837","nonce":3579387137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-09T23:12:17.586036+0000","modified":"2023-09-09T23:12:25.285958+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24542,"mds_2":24523,"mds_3":24491,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14712":{"gid":14712,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/2127301345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2127301345},{"type":"v1","addr":"172.21.15.82:6839","nonce":2127301345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14724":{"gid":14724,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/4143604488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":4143604488},{"type":"v1","addr":"172.21.15.82:6841","nonce":4143604488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.170:6833/1931961940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1931961940},{"type":"v1","addr":"172.21.15.170:6833","nonce":1931961940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24491":{"gid":24491,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.98:6833/3855146755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3855146755},{"type":"v1","addr":"172.21.15.98:6833","nonce":3855146755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2313861511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2313861511},{"type":"v1","addr":"172.21.15.170:6835","nonce":2313861511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/4006360308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":4006360308},{"type":"v1","addr":"172.21.15.170:6837","nonce":4006360308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1356474828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1356474828},{"type":"v1","addr":"172.21.15.98:6835","nonce":1356474828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/3858369735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3858369735},{"type":"v1","addr":"172.21.15.170:6839","nonce":3858369735}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/2019449840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2019449840},{"type":"v1","addr":"172.21.15.98:6837","nonce":2019449840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2785086371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2785086371},{"type":"v1","addr":"172.21.15.98:6839","nonce":2785086371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-09T23:13:12.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.552+0000 7fcf4fcd2700 1 -- 172.21.15.82:0/1629733169 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcf340a0710 msgr2=0x7fcf340a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:12.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.552+0000 7fcf4fcd2700 1 --2- 172.21.15.82:0/1629733169 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcf340a0710 0x7fcf340a2bd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcf44005950 tx=0x7fcf440058e0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.552+0000 7fcf4fcd2700 1 -- 172.21.15.82:0/1629733169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf481017f0 msgr2=0x7fcf480729f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:12.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.552+0000 7fcf4fcd2700 1 --2- 172.21.15.82:0/1629733169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf481017f0 0x7fcf480729f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800c9b0 tx=0x7fcf3800cd70 comp rx=0 tx=0).stop 2023-09-09T23:13:12.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.552+0000 7fcf4fcd2700 1 -- 172.21.15.82:0/1629733169 shutdown_connections 2023-09-09T23:13:12.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.552+0000 7fcf4fcd2700 1 --2- 172.21.15.82:0/1629733169 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcf340a0710 0x7fcf340a2bd0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.552+0000 7fcf4fcd2700 1 --2- 172.21.15.82:0/1629733169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf48100600 0x7fcf480743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.552+0000 7fcf4fcd2700 1 --2- 172.21.15.82:0/1629733169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf48102190 0x7fcf48072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.552+0000 7fcf4fcd2700 1 --2- 172.21.15.82:0/1629733169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf481017f0 0x7fcf480729f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.552+0000 7fcf4fcd2700 1 -- 172.21.15.82:0/1629733169 >> 172.21.15.82:0/1629733169 conn(0x7fcf480fbb50 msgr2=0x7fcf48104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:12.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.552+0000 7fcf4fcd2700 1 -- 172.21.15.82:0/1629733169 shutdown_connections 2023-09-09T23:13:12.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.552+0000 7fcf4fcd2700 1 -- 172.21.15.82:0/1629733169 wait complete. 2023-09-09T23:13:12.552 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 27 2023-09-09T23:13:12.562 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-09T23:13:12.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:12 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/671159092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:13:12.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:12 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1614917270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-09T23:13:12.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:12 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3830493004' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-09T23:13:12.684 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.686+0000 7f3cc691a700 1 -- 172.21.15.82:0/125559490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cc0101070 msgr2=0x7f3cc0103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:12.684 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.686+0000 7f3cc691a700 1 --2- 172.21.15.82:0/125559490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cc0101070 0x7f3cc0103460 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0009990 tx=0x7f3cb0009ca0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.684 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.686+0000 7f3cc691a700 1 -- 172.21.15.82:0/125559490 shutdown_connections 2023-09-09T23:13:12.685 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.686+0000 7f3cc691a700 1 --2- 172.21.15.82:0/125559490 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cc01062d0 0x7f3cc0108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.685 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.686+0000 7f3cc691a700 1 --2- 172.21.15.82:0/125559490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cc0101070 0x7f3cc0103460 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.685 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.686+0000 7f3cc691a700 1 --2- 172.21.15.82:0/125559490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cc01039a0 0x7f3cc0105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.685 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.686+0000 7f3cc691a700 1 -- 172.21.15.82:0/125559490 >> 172.21.15.82:0/125559490 conn(0x7f3cc00faa20 msgr2=0x7f3cc00fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:12.685 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.686+0000 7f3cc691a700 1 -- 172.21.15.82:0/125559490 shutdown_connections 2023-09-09T23:13:12.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.686+0000 7f3cc691a700 1 -- 172.21.15.82:0/125559490 wait complete. 2023-09-09T23:13:12.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.687+0000 7f3cc691a700 1 Processor -- start 2023-09-09T23:13:12.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.687+0000 7f3cc691a700 1 -- start start 2023-09-09T23:13:12.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.687+0000 7f3cc691a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cc0101070 0x7f3cc0196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:12.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.687+0000 7f3cc691a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cc01039a0 0x7f3cc0196770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:12.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.687+0000 7f3cc691a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cc01062d0 0x7f3cc019d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:12.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.687+0000 7f3cc691a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc019ddb0 con 0x7f3cc01062d0 2023-09-09T23:13:12.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.687+0000 7f3cc691a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc019def0 con 0x7f3cc0101070 2023-09-09T23:13:12.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.687+0000 7f3cc691a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc019e080 con 0x7f3cc01039a0 2023-09-09T23:13:12.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.687+0000 7f3cc4eb7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cc01062d0 0x7f3cc019d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:12.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.687+0000 7f3cc4eb7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cc01062d0 0x7f3cc019d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55234/0 (socket says 172.21.15.82:55234) 2023-09-09T23:13:12.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.687+0000 7f3cc4eb7700 1 -- 172.21.15.82:0/1280719835 learned_addr learned my addr 172.21.15.82:0/1280719835 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:12.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.688+0000 7f3cbffff700 1 --2- 172.21.15.82:0/1280719835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cc0101070 0x7f3cc0196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:12.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.688+0000 7f3cc4eb7700 1 -- 172.21.15.82:0/1280719835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cc01039a0 msgr2=0x7f3cc0196770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:12.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.688+0000 7f3cbf7fe700 1 --2- 172.21.15.82:0/1280719835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cc01039a0 0x7f3cc0196770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:12.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.688+0000 7f3cc4eb7700 1 --2- 172.21.15.82:0/1280719835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cc01039a0 0x7f3cc0196770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.688+0000 7f3cc4eb7700 1 -- 172.21.15.82:0/1280719835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cc0101070 msgr2=0x7f3cc0196230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:12.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.688+0000 7f3cc4eb7700 1 --2- 172.21.15.82:0/1280719835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cc0101070 0x7f3cc0196230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.688+0000 7f3cc4eb7700 1 -- 172.21.15.82:0/1280719835 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb00095f0 con 0x7f3cc01062d0 2023-09-09T23:13:12.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.688+0000 7f3cbffff700 1 --2- 172.21.15.82:0/1280719835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cc0101070 0x7f3cc0196230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:12.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.688+0000 7f3cc4eb7700 1 --2- 172.21.15.82:0/1280719835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cc01062d0 0x7f3cc019d7e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8007ee0 tx=0x7f3cb800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:12.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.688+0000 7f3cbd7fa700 1 -- 172.21.15.82:0/1280719835 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8017070 con 0x7f3cc01062d0 2023-09-09T23:13:12.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.688+0000 7f3cc691a700 1 -- 172.21.15.82:0/1280719835 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc019e360 con 0x7f3cc01062d0 2023-09-09T23:13:12.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.688+0000 7f3cbd7fa700 1 -- 172.21.15.82:0/1280719835 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cb800f040 con 0x7f3cc01062d0 2023-09-09T23:13:12.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.688+0000 7f3cbd7fa700 1 -- 172.21.15.82:0/1280719835 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8005740 con 0x7f3cc01062d0 2023-09-09T23:13:12.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.688+0000 7f3cc691a700 1 -- 172.21.15.82:0/1280719835 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc019e820 con 0x7f3cc01062d0 2023-09-09T23:13:12.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.689+0000 7f3cc691a700 1 -- 172.21.15.82:0/1280719835 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc00fc5a0 con 0x7f3cc01062d0 2023-09-09T23:13:12.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.690+0000 7f3cbd7fa700 1 -- 172.21.15.82:0/1280719835 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f3cb80058a0 con 0x7f3cc01062d0 2023-09-09T23:13:12.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.690+0000 7f3cbd7fa700 1 --2- 172.21.15.82:0/1280719835 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3ca80a0710 0x7f3ca80a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:12.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.690+0000 7f3cbd7fa700 1 -- 172.21.15.82:0/1280719835 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f3cb80cb530 con 0x7f3cc01062d0 2023-09-09T23:13:12.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.690+0000 7f3cbffff700 1 --2- 172.21.15.82:0/1280719835 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3ca80a0710 0x7f3ca80a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:12.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.691+0000 7f3cbffff700 1 --2- 172.21.15.82:0/1280719835 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3ca80a0710 0x7f3ca80a2bd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000b5c0 tx=0x7f3cb0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:12.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.695+0000 7f3cbd7fa700 1 -- 172.21.15.82:0/1280719835 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3cb8095000 con 0x7f3cc01062d0 2023-09-09T23:13:12.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:12 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/671159092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:13:12.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:12 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1614917270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-09T23:13:12.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:12 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3830493004' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-09T23:13:12.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.885+0000 7f3cc691a700 1 -- 172.21.15.82:0/1280719835 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3cc010c010 con 0x7f3cc01062d0 2023-09-09T23:13:12.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.887+0000 7f3cbd7fa700 1 -- 172.21.15.82:0/1280719835 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22411 (secure 0 0 0) 0x7f3cb8098620 con 0x7f3cc01062d0 2023-09-09T23:13:12.886 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:12.886 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":83,"fsid":"3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da","created":"2023-09-09T23:05:14.222190+0000","modified":"2023-09-09T23:13:10.324596+0000","last_up_change":"2023-09-09T23:11:37.253759+0000","last_in_change":"2023-09-09T23:11:19.933812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-09T23:08:31.234958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-09T23:12:15.524887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-09T23:12:16.785403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1277eb17-a2e7-4e10-9832-f12a5e93ffde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6803","nonce":3197434929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6805","nonce":3197434929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6809","nonce":3197434929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6807","nonce":3197434929}]},"public_addr":"172.21.15.82:6803/3197434929","cluster_addr":"172.21.15.82:6805/3197434929","heartbeat_back_addr":"172.21.15.82:6809/3197434929","heartbeat_front_addr":"172.21.15.82:6807/3197434929","state":["exists","up"]},{"osd":1,"uuid":"f68723fd-0908-4e21-ae8e-1de1c3a9a65c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6801","nonce":3528641314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6803","nonce":3528641314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6807","nonce":3528641314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6805","nonce":3528641314}]},"public_addr":"172.21.15.98:6801/3528641314","cluster_addr":"172.21.15.98:6803/3528641314","heartbeat_back_addr":"172.21.15.98:6807/3528641314","heartbeat_front_addr":"172.21.15.98:6805/3528641314","state":["exists","up"]},{"osd":2,"uuid":"06f89c53-1419-4f52-acad-6e92916b476f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6801","nonce":4006414551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6803","nonce":4006414551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6807","nonce":4006414551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6805","nonce":4006414551}]},"public_addr":"172.21.15.170:6801/4006414551","cluster_addr":"172.21.15.170:6803/4006414551","heartbeat_back_addr":"172.21.15.170:6807/4006414551","heartbeat_front_addr":"172.21.15.170:6805/4006414551","state":["exists","up"]},{"osd":3,"uuid":"fedbd4c9-6852-42b5-8f3a-ba8d9bfe5d59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6811","nonce":3647629241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6813","nonce":3647629241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6817","nonce":3647629241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6815","nonce":3647629241}]},"public_addr":"172.21.15.82:6811/3647629241","cluster_addr":"172.21.15.82:6813/3647629241","heartbeat_back_addr":"172.21.15.82:6817/3647629241","heartbeat_front_addr":"172.21.15.82:6815/3647629241","state":["exists","up"]},{"osd":4,"uuid":"a71f3084-6919-4161-87bf-345f3355d8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6809","nonce":2507035643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6811","nonce":2507035643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6815","nonce":2507035643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6813","nonce":2507035643}]},"public_addr":"172.21.15.98:6809/2507035643","cluster_addr":"172.21.15.98:6811/2507035643","heartbeat_back_addr":"172.21.15.98:6815/2507035643","heartbeat_front_addr":"172.21.15.98:6813/2507035643","state":["exists","up"]},{"osd":5,"uuid":"f76058e9-a90e-476e-87f0-3c6723795e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6809","nonce":775005168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6811","nonce":775005168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6815","nonce":775005168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6813","nonce":775005168}]},"public_addr":"172.21.15.170:6809/775005168","cluster_addr":"172.21.15.170:6811/775005168","heartbeat_back_addr":"172.21.15.170:6815/775005168","heartbeat_front_addr":"172.21.15.170:6813/775005168","state":["exists","up"]},{"osd":6,"uuid":"25940be0-c1bd-4c72-95bf-fd8367aa2f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6819","nonce":3008405982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6821","nonce":3008405982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6825","nonce":3008405982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6823","nonce":3008405982}]},"public_addr":"172.21.15.82:6819/3008405982","cluster_addr":"172.21.15.82:6821/3008405982","heartbeat_back_addr":"172.21.15.82:6825/3008405982","heartbeat_front_addr":"172.21.15.82:6823/3008405982","state":["exists","up"]},{"osd":7,"uuid":"f651bb8c-0b25-42d5-b028-58563f4c27f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6817","nonce":3547615284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6819","nonce":3547615284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6823","nonce":3547615284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6821","nonce":3547615284}]},"public_addr":"172.21.15.98:6817/3547615284","cluster_addr":"172.21.15.98:6819/3547615284","heartbeat_back_addr":"172.21.15.98:6823/3547615284","heartbeat_front_addr":"172.21.15.98:6821/3547615284","state":["exists","up"]},{"osd":8,"uuid":"23b27c86-8c98-441e-b7ab-d57bc584e58f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6817","nonce":632038848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6819","nonce":632038848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6823","nonce":632038848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6821","nonce":632038848}]},"public_addr":"172.21.15.170:6817/632038848","cluster_addr":"172.21.15.170:6819/632038848","heartbeat_back_addr":"172.21.15.170:6823/632038848","heartbeat_front_addr":"172.21.15.170:6821/632038848","state":["exists","up"]},{"osd":9,"uuid":"383b8989-4922-4eee-9c77-f198258ac243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6827","nonce":3354242311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6829","nonce":3354242311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6833","nonce":3354242311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6831","nonce":3354242311}]},"public_addr":"172.21.15.82:6827/3354242311","cluster_addr":"172.21.15.82:6829/3354242311","heartbeat_back_addr":"172.21.15.82:6833/3354242311","heartbeat_front_addr":"172.21.15.82:6831/3354242311","state":["exists","up"]},{"osd":10,"uuid":"33a59389-7ced-42d5-bf54-2071317132d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6825","nonce":2069114959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6827","nonce":2069114959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6831","nonce":2069114959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6829","nonce":2069114959}]},"public_addr":"172.21.15.98:6825/2069114959","cluster_addr":"172.21.15.98:6827/2069114959","heartbeat_back_addr":"172.21.15.98:6831/2069114959","heartbeat_front_addr":"172.21.15.98:6829/2069114959","state":["exists","up"]},{"osd":11,"uuid":"b7a71f5c-85e2-4fec-be4b-95be0788b302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6825","nonce":146964512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6827","nonce":146964512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6831","nonce":146964512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6829","nonce":146964512}]},"public_addr":"172.21.15.170:6825/146964512","cluster_addr":"172.21.15.170:6827/146964512","heartbeat_back_addr":"172.21.15.170:6831/146964512","heartbeat_front_addr":"172.21.15.170:6829/146964512","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:07:47.563043+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:08.359260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:29.070374+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:49.897775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:11.434248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:31.538464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:52.377109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:13.452968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:33.806897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:54.083177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:14.980087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:35.690089+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4013747788":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6800/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/2823341464":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/75116806":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/1037625632":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/3142088817":"2023-09-10T23:05:46.550354+0000","172.21.15.82:6800/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/1679622963":"2023-09-10T23:05:46.550354+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-09T23:13:12.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.889+0000 7f3cc691a700 1 -- 172.21.15.82:0/1280719835 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3ca80a0710 msgr2=0x7f3ca80a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:12.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.889+0000 7f3cc691a700 1 --2- 172.21.15.82:0/1280719835 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3ca80a0710 0x7f3ca80a2bd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000b5c0 tx=0x7f3cb0005fb0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.889+0000 7f3cc691a700 1 -- 172.21.15.82:0/1280719835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cc01062d0 msgr2=0x7f3cc019d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:12.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.889+0000 7f3cc691a700 1 --2- 172.21.15.82:0/1280719835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cc01062d0 0x7f3cc019d7e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8007ee0 tx=0x7f3cb800d3b0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.889+0000 7f3cc691a700 1 -- 172.21.15.82:0/1280719835 shutdown_connections 2023-09-09T23:13:12.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.889+0000 7f3cc691a700 1 --2- 172.21.15.82:0/1280719835 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3ca80a0710 0x7f3ca80a2bd0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.889+0000 7f3cc691a700 1 --2- 172.21.15.82:0/1280719835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cc0101070 0x7f3cc0196230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.889+0000 7f3cc691a700 1 --2- 172.21.15.82:0/1280719835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cc01062d0 0x7f3cc019d7e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.889+0000 7f3cc691a700 1 --2- 172.21.15.82:0/1280719835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cc01039a0 0x7f3cc0196770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:12.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.889+0000 7f3cc691a700 1 -- 172.21.15.82:0/1280719835 >> 172.21.15.82:0/1280719835 conn(0x7f3cc00faa20 msgr2=0x7f3cc0101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:12.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.889+0000 7f3cc691a700 1 -- 172.21.15.82:0/1280719835 shutdown_connections 2023-09-09T23:13:12.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:12.889+0000 7f3cc691a700 1 -- 172.21.15.82:0/1280719835 wait complete. 2023-09-09T23:13:12.900 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-09T23:13:13.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.023+0000 7fd519ecd700 1 -- 172.21.15.82:0/2626059916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd514100610 msgr2=0x7fd514100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.023+0000 7fd519ecd700 1 --2- 172.21.15.82:0/2626059916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd514100610 0x7fd514100a30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd5040099c0 tx=0x7fd504009cd0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.023+0000 7fd519ecd700 1 -- 172.21.15.82:0/2626059916 shutdown_connections 2023-09-09T23:13:13.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.023+0000 7fd519ecd700 1 --2- 172.21.15.82:0/2626059916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5141021a0 0x7fd514108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.023+0000 7fd519ecd700 1 --2- 172.21.15.82:0/2626059916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd514100610 0x7fd514100a30 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.023+0000 7fd519ecd700 1 --2- 172.21.15.82:0/2626059916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd514101800 0x7fd514101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.023+0000 7fd519ecd700 1 -- 172.21.15.82:0/2626059916 >> 172.21.15.82:0/2626059916 conn(0x7fd5140fbb80 msgr2=0x7fd5140fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:13.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.023+0000 7fd519ecd700 1 -- 172.21.15.82:0/2626059916 shutdown_connections 2023-09-09T23:13:13.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.023+0000 7fd519ecd700 1 -- 172.21.15.82:0/2626059916 wait complete. 2023-09-09T23:13:13.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.024+0000 7fd519ecd700 1 Processor -- start 2023-09-09T23:13:13.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.024+0000 7fd519ecd700 1 -- start start 2023-09-09T23:13:13.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.024+0000 7fd519ecd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd514100610 0x7fd5141962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.024+0000 7fd519ecd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd514101800 0x7fd514196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.024+0000 7fd519ecd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5141021a0 0x7fd51419d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.024+0000 7fd519ecd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51419de40 con 0x7fd514101800 2023-09-09T23:13:13.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.024+0000 7fd519ecd700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51419df80 con 0x7fd5141021a0 2023-09-09T23:13:13.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.024+0000 7fd519ecd700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51419e110 con 0x7fd514100610 2023-09-09T23:13:13.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.024+0000 7fd512ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd514101800 0x7fd514196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.024+0000 7fd512ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd514101800 0x7fd514196800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55246/0 (socket says 172.21.15.82:55246) 2023-09-09T23:13:13.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.024+0000 7fd512ffd700 1 -- 172.21.15.82:0/1381700950 learned_addr learned my addr 172.21.15.82:0/1381700950 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:13.029 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.024+0000 7fd513fff700 1 --2- 172.21.15.82:0/1381700950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5141021a0 0x7fd51419d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.029 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.025+0000 7fd5137fe700 1 --2- 172.21.15.82:0/1381700950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd514100610 0x7fd5141962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.029 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.025+0000 7fd512ffd700 1 -- 172.21.15.82:0/1381700950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd514100610 msgr2=0x7fd5141962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.029 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.025+0000 7fd512ffd700 1 --2- 172.21.15.82:0/1381700950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd514100610 0x7fd5141962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.029 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.025+0000 7fd512ffd700 1 -- 172.21.15.82:0/1381700950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5141021a0 msgr2=0x7fd51419d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.029 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.025+0000 7fd512ffd700 1 --2- 172.21.15.82:0/1381700950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5141021a0 0x7fd51419d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.030 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.025+0000 7fd512ffd700 1 -- 172.21.15.82:0/1381700950 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5040095f0 con 0x7fd514101800 2023-09-09T23:13:13.030 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.025+0000 7fd5137fe700 1 --2- 172.21.15.82:0/1381700950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd514100610 0x7fd5141962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:13.030 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.025+0000 7fd512ffd700 1 --2- 172.21.15.82:0/1381700950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd514101800 0x7fd514196800 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd50800d950 tx=0x7fd50800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:13.030 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.025+0000 7fd510ff9700 1 -- 172.21.15.82:0/1381700950 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd508009940 con 0x7fd514101800 2023-09-09T23:13:13.030 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.025+0000 7fd519ecd700 1 -- 172.21.15.82:0/1381700950 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd51419e3f0 con 0x7fd514101800 2023-09-09T23:13:13.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.025+0000 7fd510ff9700 1 -- 172.21.15.82:0/1381700950 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd508010460 con 0x7fd514101800 2023-09-09T23:13:13.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.025+0000 7fd510ff9700 1 -- 172.21.15.82:0/1381700950 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50800f620 con 0x7fd514101800 2023-09-09T23:13:13.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.025+0000 7fd519ecd700 1 -- 172.21.15.82:0/1381700950 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd51419e950 con 0x7fd514101800 2023-09-09T23:13:13.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.026+0000 7fd519ecd700 1 -- 172.21.15.82:0/1381700950 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd514107b50 con 0x7fd514101800 2023-09-09T23:13:13.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.027+0000 7fd510ff9700 1 -- 172.21.15.82:0/1381700950 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fd5080108d0 con 0x7fd514101800 2023-09-09T23:13:13.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.027+0000 7fd510ff9700 1 --2- 172.21.15.82:0/1381700950 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd4fc0a0710 0x7fd4fc0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.028+0000 7fd5137fe700 1 --2- 172.21.15.82:0/1381700950 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd4fc0a0710 0x7fd4fc0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.028+0000 7fd510ff9700 1 -- 172.21.15.82:0/1381700950 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fd5080cbaa0 con 0x7fd514101800 2023-09-09T23:13:13.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.028+0000 7fd5137fe700 1 --2- 172.21.15.82:0/1381700950 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd4fc0a0710 0x7fd4fc0a2bd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd50400b580 tx=0x7fd504005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:13.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.031+0000 7fd510ff9700 1 -- 172.21.15.82:0/1381700950 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd508095570 con 0x7fd514101800 2023-09-09T23:13:13.268 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.270+0000 7fd519ecd700 1 -- 172.21.15.82:0/1381700950 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd51410be40 con 0x7fd514101800 2023-09-09T23:13:13.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.270+0000 7fd510ff9700 1 -- 172.21.15.82:0/1381700950 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11368 (secure 0 0 0) 0x7fd508098b90 con 0x7fd514101800 2023-09-09T23:13:13.269 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:13.269 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2471432684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2471432684},{"type":"v1","addr":"172.21.15.82:6835","nonce":2471432684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3579387137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3579387137},{"type":"v1","addr":"172.21.15.82:6837","nonce":3579387137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-09T23:12:17.586036+0000","modified":"2023-09-09T23:12:25.285958+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24542,"mds_2":24523,"mds_3":24491,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14712":{"gid":14712,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/2127301345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2127301345},{"type":"v1","addr":"172.21.15.82:6839","nonce":2127301345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14724":{"gid":14724,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/4143604488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":4143604488},{"type":"v1","addr":"172.21.15.82:6841","nonce":4143604488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.170:6833/1931961940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1931961940},{"type":"v1","addr":"172.21.15.170:6833","nonce":1931961940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24491":{"gid":24491,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.98:6833/3855146755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3855146755},{"type":"v1","addr":"172.21.15.98:6833","nonce":3855146755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2313861511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2313861511},{"type":"v1","addr":"172.21.15.170:6835","nonce":2313861511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/4006360308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":4006360308},{"type":"v1","addr":"172.21.15.170:6837","nonce":4006360308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1356474828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1356474828},{"type":"v1","addr":"172.21.15.98:6835","nonce":1356474828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/3858369735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3858369735},{"type":"v1","addr":"172.21.15.170:6839","nonce":3858369735}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/2019449840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2019449840},{"type":"v1","addr":"172.21.15.98:6837","nonce":2019449840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2785086371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2785086371},{"type":"v1","addr":"172.21.15.98:6839","nonce":2785086371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-09T23:13:13.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.272+0000 7fd519ecd700 1 -- 172.21.15.82:0/1381700950 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd4fc0a0710 msgr2=0x7fd4fc0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.272+0000 7fd519ecd700 1 --2- 172.21.15.82:0/1381700950 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd4fc0a0710 0x7fd4fc0a2bd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd50400b580 tx=0x7fd504005fb0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.272+0000 7fd519ecd700 1 -- 172.21.15.82:0/1381700950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd514101800 msgr2=0x7fd514196800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.272+0000 7fd519ecd700 1 --2- 172.21.15.82:0/1381700950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd514101800 0x7fd514196800 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd50800d950 tx=0x7fd50800dc60 comp rx=0 tx=0).stop 2023-09-09T23:13:13.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.272+0000 7fd519ecd700 1 -- 172.21.15.82:0/1381700950 shutdown_connections 2023-09-09T23:13:13.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.272+0000 7fd519ecd700 1 --2- 172.21.15.82:0/1381700950 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd4fc0a0710 0x7fd4fc0a2bd0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.273+0000 7fd519ecd700 1 --2- 172.21.15.82:0/1381700950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5141021a0 0x7fd51419d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.273+0000 7fd519ecd700 1 --2- 172.21.15.82:0/1381700950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd514100610 0x7fd5141962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.273+0000 7fd519ecd700 1 --2- 172.21.15.82:0/1381700950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd514101800 0x7fd514196800 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.273+0000 7fd519ecd700 1 -- 172.21.15.82:0/1381700950 >> 172.21.15.82:0/1381700950 conn(0x7fd5140fbb80 msgr2=0x7fd514104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:13.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.273+0000 7fd519ecd700 1 -- 172.21.15.82:0/1381700950 shutdown_connections 2023-09-09T23:13:13.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.273+0000 7fd519ecd700 1 -- 172.21.15.82:0/1381700950 wait complete. 2023-09-09T23:13:13.274 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 27 2023-09-09T23:13:13.284 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-09-09T23:12:17.586036+0000', 'modified': '2023-09-09T23:12:25.285958+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24566, 'mds_1': 24542, 'mds_2': 24523, 'mds_3': 24491, 'mds_4': 24460}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14712': {'gid': 14712, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.82:6839/2127301345', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6838', 'nonce': 2127301345}, {'type': 'v1', 'addr': '172.21.15.82:6839', 'nonce': 2127301345}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14724': {'gid': 14724, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.82:6841/4143604488', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6840', 'nonce': 4143604488}, {'type': 'v1', 'addr': '172.21.15.82:6841', 'nonce': 4143604488}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24460': {'gid': 24460, 'name': 'l', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.170:6833/1931961940', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6832', 'nonce': 1931961940}, {'type': 'v1', 'addr': '172.21.15.170:6833', 'nonce': 1931961940}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24491': {'gid': 24491, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.98:6833/3855146755', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6832', 'nonce': 3855146755}, {'type': 'v1', 'addr': '172.21.15.98:6833', 'nonce': 3855146755}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24493': {'gid': 24493, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6835/2313861511', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 2313861511}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 2313861511}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24502': {'gid': 24502, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6837/4006360308', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6836', 'nonce': 4006360308}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 4006360308}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24518': {'gid': 24518, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.98:6835/1356474828', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6834', 'nonce': 1356474828}, {'type': 'v1', 'addr': '172.21.15.98:6835', 'nonce': 1356474828}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24523': {'gid': 24523, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.170:6839/3858369735', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6838', 'nonce': 3858369735}, {'type': 'v1', 'addr': '172.21.15.170:6839', 'nonce': 3858369735}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24542': {'gid': 24542, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.98:6837/2019449840', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6836', 'nonce': 2019449840}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 2019449840}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24566': {'gid': 24566, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.98:6839/2785086371', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6838', 'nonce': 2785086371}, {'type': 'v1', 'addr': '172.21.15.98:6839', 'nonce': 2785086371}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-09T23:13:13.284 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-09-09T23:13:13.285 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2023-09-09T23:13:13.410 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.410+0000 7f48e5a39700 1 -- 172.21.15.82:0/2119215911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48e00ff760 msgr2=0x7f48e00ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.411 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.410+0000 7f48e5a39700 1 --2- 172.21.15.82:0/2119215911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48e00ff760 0x7f48e00ffbe0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f48d0009990 tx=0x7f48d0009ca0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.411 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.410+0000 7f48e5a39700 1 -- 172.21.15.82:0/2119215911 shutdown_connections 2023-09-09T23:13:13.411 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.410+0000 7f48e5a39700 1 --2- 172.21.15.82:0/2119215911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48e0100120 0x7f48e01079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.411 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.410+0000 7f48e5a39700 1 --2- 172.21.15.82:0/2119215911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48e0100ba0 0x7f48e0100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.410+0000 7f48e5a39700 1 --2- 172.21.15.82:0/2119215911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48e00ff760 0x7f48e00ffbe0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.410+0000 7f48e5a39700 1 -- 172.21.15.82:0/2119215911 >> 172.21.15.82:0/2119215911 conn(0x7f48e00fa1f0 msgr2=0x7f48e00fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:13.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.410+0000 7f48e5a39700 1 -- 172.21.15.82:0/2119215911 shutdown_connections 2023-09-09T23:13:13.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.410+0000 7f48e5a39700 1 -- 172.21.15.82:0/2119215911 wait complete. 2023-09-09T23:13:13.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.410+0000 7f48e5a39700 1 Processor -- start 2023-09-09T23:13:13.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.411+0000 7f48e5a39700 1 -- start start 2023-09-09T23:13:13.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.411+0000 7f48e5a39700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48e00ff760 0x7f48e0196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.411+0000 7f48e5a39700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48e0100120 0x7f48e01968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.411+0000 7f48e5a39700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48e0100ba0 0x7f48e019d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.411+0000 7f48e5a39700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e019def0 con 0x7f48e0100ba0 2023-09-09T23:13:13.414 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.411+0000 7f48e5a39700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e019e030 con 0x7f48e00ff760 2023-09-09T23:13:13.414 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.411+0000 7f48e5a39700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e019e1c0 con 0x7f48e0100120 2023-09-09T23:13:13.414 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.411+0000 7f48df7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48e0100ba0 0x7f48e019d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.414 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.411+0000 7f48df7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48e0100ba0 0x7f48e019d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55262/0 (socket says 172.21.15.82:55262) 2023-09-09T23:13:13.414 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.411+0000 7f48df7fe700 1 -- 172.21.15.82:0/4172042338 learned_addr learned my addr 172.21.15.82:0/4172042338 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:13.415 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.411+0000 7f48deffd700 1 --2- 172.21.15.82:0/4172042338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48e00ff760 0x7f48e0196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.415 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.411+0000 7f48de7fc700 1 --2- 172.21.15.82:0/4172042338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48e0100120 0x7f48e01968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.415 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.411+0000 7f48df7fe700 1 -- 172.21.15.82:0/4172042338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48e0100120 msgr2=0x7f48e01968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.415 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.411+0000 7f48df7fe700 1 --2- 172.21.15.82:0/4172042338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48e0100120 0x7f48e01968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.415 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.411+0000 7f48df7fe700 1 -- 172.21.15.82:0/4172042338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48e00ff760 msgr2=0x7f48e0196370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.411+0000 7f48df7fe700 1 --2- 172.21.15.82:0/4172042338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48e00ff760 0x7f48e0196370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.411+0000 7f48df7fe700 1 -- 172.21.15.82:0/4172042338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d00095f0 con 0x7f48e0100ba0 2023-09-09T23:13:13.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.411+0000 7f48de7fc700 1 --2- 172.21.15.82:0/4172042338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48e0100120 0x7f48e01968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:13.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.411+0000 7f48df7fe700 1 --2- 172.21.15.82:0/4172042338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48e0100ba0 0x7f48e019d920 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f48d800b970 tx=0x7f48d800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:13.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.411+0000 7f48e4a37700 1 -- 172.21.15.82:0/4172042338 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d800c9b0 con 0x7f48e0100ba0 2023-09-09T23:13:13.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.411+0000 7f48e5a39700 1 -- 172.21.15.82:0/4172042338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e019e4a0 con 0x7f48e0100ba0 2023-09-09T23:13:13.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.411+0000 7f48e4a37700 1 -- 172.21.15.82:0/4172042338 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48d8014440 con 0x7f48e0100ba0 2023-09-09T23:13:13.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.411+0000 7f48e4a37700 1 -- 172.21.15.82:0/4172042338 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d80075d0 con 0x7f48e0100ba0 2023-09-09T23:13:13.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.412+0000 7f48e5a39700 1 -- 172.21.15.82:0/4172042338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e019e9b0 con 0x7f48e0100ba0 2023-09-09T23:13:13.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.414+0000 7f48e4a37700 1 -- 172.21.15.82:0/4172042338 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f48d80145b0 con 0x7f48e0100ba0 2023-09-09T23:13:13.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.414+0000 7f48e4a37700 1 --2- 172.21.15.82:0/4172042338 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f48cc0a06d0 0x7f48cc0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.414+0000 7f48deffd700 1 --2- 172.21.15.82:0/4172042338 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f48cc0a06d0 0x7f48cc0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.414+0000 7f48e4a37700 1 -- 172.21.15.82:0/4172042338 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f48d8006e40 con 0x7f48e0100ba0 2023-09-09T23:13:13.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.415+0000 7f48deffd700 1 --2- 172.21.15.82:0/4172042338 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f48cc0a06d0 0x7f48cc0a2b90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f48c8005fd0 tx=0x7f48c800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:13.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.415+0000 7f48f1bffb80 1 Processor -- start 2023-09-09T23:13:13.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.415+0000 7f48f1bffb80 1 -- start start 2023-09-09T23:13:13.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.415+0000 7f48f1bffb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595b9536020 0x5595b9536440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.415+0000 7f48f1bffb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595b9536980 0x5595b9534c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.415+0000 7f48f1bffb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595b95351e0 0x5595b9535660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.415+0000 7f48f1bffb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b94896e0 con 0x5595b9536020 2023-09-09T23:13:13.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.415+0000 7f48f1bffb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b9487000 con 0x5595b9536980 2023-09-09T23:13:13.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.415+0000 7f48f1bffb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b94828f0 con 0x5595b95351e0 2023-09-09T23:13:13.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.415+0000 7f48deffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595b9536020 0x5595b9536440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.415+0000 7f48deffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595b9536020 0x5595b9536440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55266/0 (socket says 172.21.15.82:55266) 2023-09-09T23:13:13.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.415+0000 7f48deffd700 1 -- 172.21.15.82:0/1250144125 learned_addr learned my addr 172.21.15.82:0/1250144125 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:13.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.416+0000 7f48de7fc700 1 --2- 172.21.15.82:0/1250144125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595b9536980 0x5595b9534c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.416+0000 7f48deffd700 1 -- 172.21.15.82:0/1250144125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595b95351e0 msgr2=0x5595b9535660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:13.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.416+0000 7f48deffd700 1 --2- 172.21.15.82:0/1250144125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595b95351e0 0x5595b9535660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.416+0000 7f48deffd700 1 -- 172.21.15.82:0/1250144125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595b9536980 msgr2=0x5595b9534c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:13.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.416+0000 7f48deffd700 1 --2- 172.21.15.82:0/1250144125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595b9536980 0x5595b9534c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.416+0000 7f48deffd700 1 -- 172.21.15.82:0/1250144125 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595b92aed90 con 0x5595b9536020 2023-09-09T23:13:13.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.416+0000 7f48de7fc700 1 --2- 172.21.15.82:0/1250144125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595b9536980 0x5595b9534c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:13.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.416+0000 7f48deffd700 1 --2- 172.21.15.82:0/1250144125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595b9536020 0x5595b9536440 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f48c800ec80 tx=0x7f48c8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80fa8a157d16b75 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:13.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.416+0000 7f48d5ffb700 1 -- 172.21.15.82:0/1250144125 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c8010cd0 con 0x5595b9536020 2023-09-09T23:13:13.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.416+0000 7f48d5ffb700 1 -- 172.21.15.82:0/1250144125 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48c8010e30 con 0x5595b9536020 2023-09-09T23:13:13.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.416+0000 7f48d5ffb700 1 -- 172.21.15.82:0/1250144125 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c800f7b0 con 0x5595b9536020 2023-09-09T23:13:13.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.417+0000 7f48f1bffb80 1 -- 172.21.15.82:0/1250144125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595b9536020 msgr2=0x5595b9536440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.417+0000 7f48f1bffb80 1 --2- 172.21.15.82:0/1250144125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595b9536020 0x5595b9536440 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f48c800ec80 tx=0x7f48c8011000 comp rx=0 tx=0).stop 2023-09-09T23:13:13.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.417+0000 7f48f1bffb80 1 -- 172.21.15.82:0/1250144125 shutdown_connections 2023-09-09T23:13:13.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.417+0000 7f48f1bffb80 1 --2- 172.21.15.82:0/1250144125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595b95351e0 0x5595b9535660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.417+0000 7f48f1bffb80 1 --2- 172.21.15.82:0/1250144125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595b9536020 0x5595b9536440 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.417+0000 7f48f1bffb80 1 --2- 172.21.15.82:0/1250144125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595b9536980 0x5595b9534c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.417+0000 7f48f1bffb80 1 -- 172.21.15.82:0/1250144125 >> 172.21.15.82:0/1250144125 conn(0x5595b94a5040 msgr2=0x5595b952b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:13.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.417+0000 7f48f1bffb80 1 -- 172.21.15.82:0/1250144125 shutdown_connections 2023-09-09T23:13:13.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.417+0000 7f48f1bffb80 1 -- 172.21.15.82:0/1250144125 wait complete. 2023-09-09T23:13:13.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.417+0000 7f48f1bffb80 1 -- start start 2023-09-09T23:13:13.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.417+0000 7f48f1bffb80 1 Processor -- start 2023-09-09T23:13:13.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.417+0000 7f48f1bffb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595b95351e0 0x5595b953e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.417+0000 7f48f1bffb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595b9536020 0x5595b953ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.417+0000 7f48f1bffb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595b9536980 0x5595b9545f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.417+0000 7f48f1bffb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b9420b20 con 0x5595b9536980 2023-09-09T23:13:13.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.417+0000 7f48f1bffb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b9420820 con 0x5595b9536020 2023-09-09T23:13:13.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.417+0000 7f48f1bffb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b93048d0 con 0x5595b95351e0 2023-09-09T23:13:13.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.417+0000 7f48f1bffb80 10 client.0 _ll_register_callbacks cb 0x5595b936aa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:13.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.418+0000 7f48df7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595b9536980 0x5595b9545f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.418+0000 7f48df7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595b9536980 0x5595b9545f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55276/0 (socket says 172.21.15.82:55276) 2023-09-09T23:13:13.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.418+0000 7f48df7fe700 1 -- 172.21.15.82:0/3972076494 learned_addr learned my addr 172.21.15.82:0/3972076494 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:13.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.418+0000 7f48de7fc700 1 --2- 172.21.15.82:0/3972076494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595b9536020 0x5595b953ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.418+0000 7f48deffd700 1 --2- 172.21.15.82:0/3972076494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595b95351e0 0x5595b953e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.418+0000 7f48df7fe700 1 -- 172.21.15.82:0/3972076494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595b95351e0 msgr2=0x5595b953e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:13.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.418+0000 7f48df7fe700 1 --2- 172.21.15.82:0/3972076494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595b95351e0 0x5595b953e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.418+0000 7f48df7fe700 1 -- 172.21.15.82:0/3972076494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595b9536020 msgr2=0x5595b953ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:13.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.418+0000 7f48df7fe700 1 --2- 172.21.15.82:0/3972076494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595b9536020 0x5595b953ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.418+0000 7f48df7fe700 1 -- 172.21.15.82:0/3972076494 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c800e930 con 0x5595b9536980 2023-09-09T23:13:13.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.418+0000 7f48df7fe700 1 --2- 172.21.15.82:0/3972076494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595b9536980 0x5595b9545f70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f48d8014ce0 tx=0x7f48d8099000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e14aecda7905cf server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:13.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.418+0000 7f48beffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:13.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.418+0000 7f48beffd700 1 -- 172.21.15.82:0/3972076494 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d809c8f0 con 0x5595b9536980 2023-09-09T23:13:13.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.418+0000 7f48beffd700 1 -- 172.21.15.82:0/3972076494 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f48d80a0b30 con 0x5595b9536980 2023-09-09T23:13:13.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.418+0000 7f48f1bffb80 1 -- 172.21.15.82:0/3972076494 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5595b93da4a0 con 0x5595b9536980 2023-09-09T23:13:13.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.418+0000 7f48beffd700 1 -- 172.21.15.82:0/3972076494 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48d80a06a0 con 0x5595b9536980 2023-09-09T23:13:13.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.418+0000 7f48f1bffb80 10 client.15144 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:13.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.418+0000 7f48f1bffb80 1 -- 172.21.15.82:0/3972076494 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5595b9297c10 con 0x5595b9536980 2023-09-09T23:13:13.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.418+0000 7f48beffd700 1 -- 172.21.15.82:0/3972076494 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48d80b3020 con 0x5595b9536980 2023-09-09T23:13:13.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.419+0000 7f48beffd700 1 -- 172.21.15.82:0/3972076494 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f48d8094a90 con 0x5595b9536980 2023-09-09T23:13:13.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.419+0000 7f48f1bffb80 10 client.15144 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:13.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.419+0000 7f48f1bffb80 10 client.15144 resolve_mds: resolved name 'l' to daemon mds.l 2023-09-09T23:13:13.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.419+0000 7f48f1bffb80 20 client.15144 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:13.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.419+0000 7f48f1bffb80 1 --2- 172.21.15.82:0/3972076494 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x5595b9549fb0 0x5595b9548fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.419+0000 7f48f1bffb80 4 client.15144 mds_command: new command op to 24460 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:13:13.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.419+0000 7f48f1bffb80 1 -- 172.21.15.82:0/3972076494 --> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5595b9497850 con 0x5595b9549fb0 2023-09-09T23:13:13.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.419+0000 7f48deffd700 1 --2- 172.21.15.82:0/3972076494 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x5595b9549fb0 0x5595b9548fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.419+0000 7f48beffd700 10 client.15144 ms_handle_connect on v2:172.21.15.170:6832/1931961940 2023-09-09T23:13:13.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.419+0000 7f48deffd700 1 --2- 172.21.15.82:0/3972076494 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x5595b9549fb0 0x5595b9548fe0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fc1c429cd6456d1f server_cookie=c9cc10134139bdf9 in_seq=0 out_seq=0 2023-09-09T23:13:13.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.420+0000 7f48beffd700 1 -- 172.21.15.82:0/3972076494 <== mds.4 v2:172.21.15.170:6832/1931961940 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5595b9497850 con 0x5595b9549fb0 2023-09-09T23:13:13.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.420+0000 7f48beffd700 10 client.15144 handle_command_reply: tid=0 2023-09-09T23:13:13.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.420+0000 7f48f1bffb80 1 client.15144 shutdown 2023-09-09T23:13:13.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.420+0000 7f48f1bffb80 1 -- 172.21.15.82:0/3972076494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595b9536980 msgr2=0x5595b9545f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.420+0000 7f48f1bffb80 1 --2- 172.21.15.82:0/3972076494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595b9536980 0x5595b9545f70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f48d8014ce0 tx=0x7f48d8099000 comp rx=0 tx=0).stop 2023-09-09T23:13:13.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.420+0000 7f48f1bffb80 1 -- 172.21.15.82:0/3972076494 shutdown_connections 2023-09-09T23:13:13.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.420+0000 7f48f1bffb80 1 --2- 172.21.15.82:0/3972076494 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x5595b9549fb0 0x5595b9548fe0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.420+0000 7f48f1bffb80 1 --2- 172.21.15.82:0/3972076494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595b9536980 0x5595b9545f70 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.420+0000 7f48beffd700 0 client.15144 ms_handle_reset on v2:172.21.15.170:6832/1931961940 2023-09-09T23:13:13.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.420+0000 7f48f1bffb80 1 --2- 172.21.15.82:0/3972076494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595b95351e0 0x5595b953e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.420+0000 7f48f1bffb80 1 --2- 172.21.15.82:0/3972076494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595b9536020 0x5595b953ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.420+0000 7f48f1bffb80 1 -- 172.21.15.82:0/3972076494 >> 172.21.15.82:0/3972076494 conn(0x5595b94a5040 msgr2=0x5595b952ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:13.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.420+0000 7f48f1bffb80 1 -- 172.21.15.82:0/3972076494 shutdown_connections 2023-09-09T23:13:13.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.420+0000 7f48f1bffb80 1 -- 172.21.15.82:0/3972076494 wait complete. 2023-09-09T23:13:13.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.420+0000 7f48f1bffb80 20 client.15144 trim_cache size 0 max 16384 2023-09-09T23:13:13.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.438+0000 7f48f1bffb80 1 Processor -- start 2023-09-09T23:13:13.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.438+0000 7f48f1bffb80 1 -- start start 2023-09-09T23:13:13.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.438+0000 7f48f1bffb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595b95351e0 0x5595b953d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.438+0000 7f48f1bffb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595b9536020 0x5595b9545f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.438+0000 7f48f1bffb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595b9536980 0x5595b952b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.438+0000 7f48f1bffb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b94f5800 con 0x5595b9536980 2023-09-09T23:13:13.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.438+0000 7f48f1bffb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b953d4f0 con 0x5595b9536020 2023-09-09T23:13:13.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.438+0000 7f48f1bffb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b944bf70 con 0x5595b95351e0 2023-09-09T23:13:13.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.438+0000 7f48df7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595b9536980 0x5595b952b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.438+0000 7f48df7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595b9536980 0x5595b952b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55284/0 (socket says 172.21.15.82:55284) 2023-09-09T23:13:13.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.438+0000 7f48df7fe700 1 -- 172.21.15.82:0/4219642377 learned_addr learned my addr 172.21.15.82:0/4219642377 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:13.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.438+0000 7f48df7fe700 1 -- 172.21.15.82:0/4219642377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595b95351e0 msgr2=0x5595b953d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:13.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.438+0000 7f48df7fe700 1 --2- 172.21.15.82:0/4219642377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595b95351e0 0x5595b953d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.438+0000 7f48df7fe700 1 -- 172.21.15.82:0/4219642377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595b9536020 msgr2=0x5595b9545f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:13.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.438+0000 7f48df7fe700 1 --2- 172.21.15.82:0/4219642377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595b9536020 0x5595b9545f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.438+0000 7f48df7fe700 1 -- 172.21.15.82:0/4219642377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595b92d5200 con 0x5595b9536980 2023-09-09T23:13:13.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.439+0000 7f48df7fe700 1 --2- 172.21.15.82:0/4219642377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595b9536980 0x5595b952b640 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f48d8098a50 tx=0x7f48d8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e96b88627fe3958a server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:13.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.439+0000 7f48d67fc700 1 -- 172.21.15.82:0/4219642377 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d80a0b30 con 0x5595b9536980 2023-09-09T23:13:13.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.439+0000 7f48d67fc700 1 -- 172.21.15.82:0/4219642377 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48d80b3070 con 0x5595b9536980 2023-09-09T23:13:13.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.439+0000 7f48d67fc700 1 -- 172.21.15.82:0/4219642377 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d80a0900 con 0x5595b9536980 2023-09-09T23:13:13.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.439+0000 7f48f1bffb80 1 -- 172.21.15.82:0/4219642377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595b9536980 msgr2=0x5595b952b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.439+0000 7f48f1bffb80 1 --2- 172.21.15.82:0/4219642377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595b9536980 0x5595b952b640 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f48d8098a50 tx=0x7f48d8004ad0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.439+0000 7f48f1bffb80 1 -- 172.21.15.82:0/4219642377 shutdown_connections 2023-09-09T23:13:13.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.439+0000 7f48f1bffb80 1 --2- 172.21.15.82:0/4219642377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595b9536980 0x5595b952b640 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.439+0000 7f48f1bffb80 1 --2- 172.21.15.82:0/4219642377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595b95351e0 0x5595b953d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.439+0000 7f48f1bffb80 1 --2- 172.21.15.82:0/4219642377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595b9536020 0x5595b9545f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.439+0000 7f48f1bffb80 1 -- 172.21.15.82:0/4219642377 >> 172.21.15.82:0/4219642377 conn(0x5595b94a5040 msgr2=0x5595b952c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:13.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.439+0000 7f48f1bffb80 1 -- 172.21.15.82:0/4219642377 shutdown_connections 2023-09-09T23:13:13.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.439+0000 7f48f1bffb80 1 -- 172.21.15.82:0/4219642377 wait complete. 2023-09-09T23:13:13.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.440+0000 7f48f1bffb80 1 -- start start 2023-09-09T23:13:13.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.440+0000 7f48f1bffb80 1 Processor -- start 2023-09-09T23:13:13.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.440+0000 7f48f1bffb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595b95351e0 0x5595b95411a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.440+0000 7f48f1bffb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595b9536020 0x5595b95416e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.440+0000 7f48f1bffb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595b9536980 0x5595b959e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.440+0000 7f48f1bffb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b9541dc0 con 0x5595b95351e0 2023-09-09T23:13:13.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.440+0000 7f48f1bffb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b959e540 con 0x5595b9536020 2023-09-09T23:13:13.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.440+0000 7f48f1bffb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b959e680 con 0x5595b9536980 2023-09-09T23:13:13.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.440+0000 7f48f1bffb80 10 client.0 _ll_register_callbacks cb 0x7f48d8098690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:13.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.440+0000 7f48deffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595b95351e0 0x5595b95411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.440+0000 7f48deffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595b95351e0 0x5595b95411a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55286/0 (socket says 172.21.15.82:55286) 2023-09-09T23:13:13.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.440+0000 7f48deffd700 1 -- 172.21.15.82:0/1141738908 learned_addr learned my addr 172.21.15.82:0/1141738908 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:13.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.440+0000 7f48de7fc700 1 --2- 172.21.15.82:0/1141738908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595b9536020 0x5595b95416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.440+0000 7f48deffd700 1 -- 172.21.15.82:0/1141738908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595b9536980 msgr2=0x5595b959e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:13.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.440+0000 7f48df7fe700 1 --2- 172.21.15.82:0/1141738908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595b9536980 0x5595b959e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.441+0000 7f48deffd700 1 --2- 172.21.15.82:0/1141738908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595b9536980 0x5595b959e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.441+0000 7f48deffd700 1 -- 172.21.15.82:0/1141738908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595b9536020 msgr2=0x5595b95416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:13.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.441+0000 7f48deffd700 1 --2- 172.21.15.82:0/1141738908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595b9536020 0x5595b95416e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.441+0000 7f48deffd700 1 -- 172.21.15.82:0/1141738908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595b9297c10 con 0x5595b95351e0 2023-09-09T23:13:13.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.441+0000 7f48de7fc700 1 --2- 172.21.15.82:0/1141738908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595b9536020 0x5595b95416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:13.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.441+0000 7f48df7fe700 1 --2- 172.21.15.82:0/1141738908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595b9536980 0x5595b959e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:13.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.441+0000 7f48deffd700 1 --2- 172.21.15.82:0/1141738908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595b95351e0 0x5595b95411a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f48c8010710 tx=0x7f48c8010bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe46ebb5cb3c021 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:13.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.441+0000 7f48beffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:13.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.441+0000 7f48beffd700 1 -- 172.21.15.82:0/1141738908 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c8035070 con 0x5595b95351e0 2023-09-09T23:13:13.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.441+0000 7f48f1bffb80 1 -- 172.21.15.82:0/1141738908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48c8010e30 con 0x5595b95351e0 2023-09-09T23:13:13.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.441+0000 7f48beffd700 1 -- 172.21.15.82:0/1141738908 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f48c80351d0 con 0x5595b95351e0 2023-09-09T23:13:13.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.441+0000 7f48beffd700 1 -- 172.21.15.82:0/1141738908 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48c8035390 con 0x5595b95351e0 2023-09-09T23:13:13.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.441+0000 7f48f1bffb80 10 client.15150 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:13.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.441+0000 7f48f1bffb80 1 -- 172.21.15.82:0/1141738908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5595b92ad260 con 0x5595b95351e0 2023-09-09T23:13:13.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.441+0000 7f48beffd700 1 -- 172.21.15.82:0/1141738908 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48c8010e30 con 0x5595b95351e0 2023-09-09T23:13:13.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.441+0000 7f48beffd700 1 -- 172.21.15.82:0/1141738908 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f48c8004920 con 0x5595b95351e0 2023-09-09T23:13:13.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.441+0000 7f48f1bffb80 10 client.15150 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:13.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.441+0000 7f48f1bffb80 10 client.15150 resolve_mds: resolved name 'l' to daemon mds.l 2023-09-09T23:13:13.466 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.441+0000 7f48f1bffb80 20 client.15150 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:13.466 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.441+0000 7f48f1bffb80 1 --2- 172.21.15.82:0/1141738908 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x5595b95a3890 0x5595b959fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.466 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.441+0000 7f48f1bffb80 4 client.15150 mds_command: new command op to 24460 tid=0[{"prefix": "status"}] 2023-09-09T23:13:13.466 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.441+0000 7f48f1bffb80 1 -- 172.21.15.82:0/1141738908 --> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5595b947b8d0 con 0x5595b95a3890 2023-09-09T23:13:13.466 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.442+0000 7f48de7fc700 1 --2- 172.21.15.82:0/1141738908 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x5595b95a3890 0x5595b959fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.467 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.442+0000 7f48beffd700 10 client.15150 ms_handle_connect on v2:172.21.15.170:6832/1931961940 2023-09-09T23:13:13.467 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.442+0000 7f48de7fc700 1 --2- 172.21.15.82:0/1141738908 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x5595b95a3890 0x5595b959fd40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=59e59bcc7b1a116c server_cookie=26c62aa29d1c747 in_seq=0 out_seq=0 2023-09-09T23:13:13.467 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.442+0000 7f48beffd700 1 -- 172.21.15.82:0/1141738908 <== mds.4 v2:172.21.15.170:6832/1931961940 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5595b947b8d0 con 0x5595b95a3890 2023-09-09T23:13:13.467 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.442+0000 7f48beffd700 10 client.15150 handle_command_reply: tid=0 2023-09-09T23:13:13.467 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.442+0000 7f48f1bffb80 1 client.15150 shutdown 2023-09-09T23:13:13.468 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.442+0000 7f48f1bffb80 1 -- 172.21.15.82:0/1141738908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595b95351e0 msgr2=0x5595b95411a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.468 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.442+0000 7f48f1bffb80 1 --2- 172.21.15.82:0/1141738908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595b95351e0 0x5595b95411a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f48c8010710 tx=0x7f48c8010bf0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.468 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.442+0000 7f48f1bffb80 1 -- 172.21.15.82:0/1141738908 shutdown_connections 2023-09-09T23:13:13.468 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.443+0000 7f48f1bffb80 1 --2- 172.21.15.82:0/1141738908 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x5595b95a3890 0x5595b959fd40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.468 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.443+0000 7f48f1bffb80 1 --2- 172.21.15.82:0/1141738908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595b9536980 0x5595b959e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.468 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.443+0000 7f48f1bffb80 1 --2- 172.21.15.82:0/1141738908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595b95351e0 0x5595b95411a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.469 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.443+0000 7f48beffd700 0 client.15150 ms_handle_reset on v2:172.21.15.170:6832/1931961940 2023-09-09T23:13:13.469 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.443+0000 7f48f1bffb80 1 --2- 172.21.15.82:0/1141738908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595b9536020 0x5595b95416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.469 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.443+0000 7f48f1bffb80 1 -- 172.21.15.82:0/1141738908 >> 172.21.15.82:0/1141738908 conn(0x5595b94a5040 msgr2=0x5595b953d700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:13.469 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.443+0000 7f48f1bffb80 1 -- 172.21.15.82:0/1141738908 shutdown_connections 2023-09-09T23:13:13.469 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.443+0000 7f48f1bffb80 1 -- 172.21.15.82:0/1141738908 wait complete. 2023-09-09T23:13:13.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.443+0000 7f48f1bffb80 20 client.15150 trim_cache size 0 max 16384 2023-09-09T23:13:13.470 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:13:13.470 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da", 2023-09-09T23:13:13.471 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 4, 2023-09-09T23:13:13.471 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24460, 2023-09-09T23:13:13.471 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2023-09-09T23:13:13.471 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2023-09-09T23:13:13.471 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2023-09-09T23:13:13.471 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 49.175981966999998, 2023-09-09T23:13:13.472 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 27, 2023-09-09T23:13:13.472 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 83, 2023-09-09T23:13:13.472 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 81, 2023-09-09T23:13:13.472 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 85.694037234000007 2023-09-09T23:13:13.472 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:13:13.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.443+0000 7f48e5a39700 1 -- 172.21.15.82:0/4172042338 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f48cc0a06d0 msgr2=0x7f48cc0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.443+0000 7f48e5a39700 1 --2- 172.21.15.82:0/4172042338 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f48cc0a06d0 0x7f48cc0a2b90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f48c8005fd0 tx=0x7f48c800a560 comp rx=0 tx=0).stop 2023-09-09T23:13:13.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.443+0000 7f48e5a39700 1 -- 172.21.15.82:0/4172042338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48e0100ba0 msgr2=0x7f48e019d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.443+0000 7f48e5a39700 1 --2- 172.21.15.82:0/4172042338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48e0100ba0 0x7f48e019d920 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f48d800b970 tx=0x7f48d800bd30 comp rx=0 tx=0).stop 2023-09-09T23:13:13.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.443+0000 7f48e5a39700 1 -- 172.21.15.82:0/4172042338 shutdown_connections 2023-09-09T23:13:13.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.443+0000 7f48e5a39700 1 --2- 172.21.15.82:0/4172042338 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f48cc0a06d0 0x7f48cc0a2b90 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.443+0000 7f48e5a39700 1 --2- 172.21.15.82:0/4172042338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48e00ff760 0x7f48e0196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.443+0000 7f48e5a39700 1 --2- 172.21.15.82:0/4172042338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48e0100ba0 0x7f48e019d920 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.443+0000 7f48e5a39700 1 --2- 172.21.15.82:0/4172042338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48e0100120 0x7f48e01968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.443+0000 7f48e5a39700 1 -- 172.21.15.82:0/4172042338 >> 172.21.15.82:0/4172042338 conn(0x7f48e00fa1f0 msgr2=0x7f48e00fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:13.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.443+0000 7f48e5a39700 1 -- 172.21.15.82:0/4172042338 shutdown_connections 2023-09-09T23:13:13.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.443+0000 7f48e5a39700 1 -- 172.21.15.82:0/4172042338 wait complete. 2023-09-09T23:13:13.476 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2023-09-09T23:13:13.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.601+0000 7f92a3710700 1 -- 172.21.15.82:0/557111284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f929c1062f0 msgr2=0x7f929c108760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.601+0000 7f92a3710700 1 --2- 172.21.15.82:0/557111284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f929c1062f0 0x7f929c108760 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f929400b0a0 tx=0x7f929400b3b0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.601+0000 7f92a3710700 1 -- 172.21.15.82:0/557111284 shutdown_connections 2023-09-09T23:13:13.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.601+0000 7f92a3710700 1 --2- 172.21.15.82:0/557111284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f929c101090 0x7f929c103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.601+0000 7f92a3710700 1 --2- 172.21.15.82:0/557111284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f929c1062f0 0x7f929c108760 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.601+0000 7f92a3710700 1 --2- 172.21.15.82:0/557111284 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f929c1039c0 0x7f929c105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.601+0000 7f92a3710700 1 -- 172.21.15.82:0/557111284 >> 172.21.15.82:0/557111284 conn(0x7f929c0faac0 msgr2=0x7f929c0fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:13.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.602+0000 7f92a3710700 1 -- 172.21.15.82:0/557111284 shutdown_connections 2023-09-09T23:13:13.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.602+0000 7f92a3710700 1 -- 172.21.15.82:0/557111284 wait complete. 2023-09-09T23:13:13.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.602+0000 7f92a3710700 1 Processor -- start 2023-09-09T23:13:13.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.602+0000 7f92a3710700 1 -- start start 2023-09-09T23:13:13.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.602+0000 7f92a3710700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f929c101090 0x7f929c1963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.602+0000 7f92a3710700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f929c1039c0 0x7f929c196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.602+0000 7f92a3710700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f929c1062f0 0x7f929c19d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.602+0000 7f92a3710700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c19df40 con 0x7f929c1062f0 2023-09-09T23:13:13.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.602+0000 7f92a3710700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c19e080 con 0x7f929c1039c0 2023-09-09T23:13:13.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.602+0000 7f92a3710700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c19e210 con 0x7f929c101090 2023-09-09T23:13:13.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.602+0000 7f92a1cad700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f929c1062f0 0x7f929c19d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.602+0000 7f92a1cad700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f929c1062f0 0x7f929c19d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55304/0 (socket says 172.21.15.82:55304) 2023-09-09T23:13:13.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.602+0000 7f92a1cad700 1 -- 172.21.15.82:0/2101297068 learned_addr learned my addr 172.21.15.82:0/2101297068 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:13.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.603+0000 7f92a0cab700 1 --2- 172.21.15.82:0/2101297068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f929c1039c0 0x7f929c196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.603+0000 7f92a1cad700 1 -- 172.21.15.82:0/2101297068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f929c101090 msgr2=0x7f929c1963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.603+0000 7f92a1cad700 1 --2- 172.21.15.82:0/2101297068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f929c101090 0x7f929c1963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.603+0000 7f92a1cad700 1 -- 172.21.15.82:0/2101297068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f929c1039c0 msgr2=0x7f929c196900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.603+0000 7f92a1cad700 1 --2- 172.21.15.82:0/2101297068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f929c1039c0 0x7f929c196900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.603+0000 7f92a1cad700 1 -- 172.21.15.82:0/2101297068 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9294009d00 con 0x7f929c1062f0 2023-09-09T23:13:13.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.603+0000 7f92a1cad700 1 --2- 172.21.15.82:0/2101297068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f929c1062f0 0x7f929c19d970 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f92940062a0 tx=0x7f9294004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:13.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.603+0000 7f92927fc700 1 -- 172.21.15.82:0/2101297068 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929400e040 con 0x7f929c1062f0 2023-09-09T23:13:13.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.603+0000 7f92927fc700 1 -- 172.21.15.82:0/2101297068 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f929401db50 con 0x7f929c1062f0 2023-09-09T23:13:13.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.603+0000 7f92a3710700 1 -- 172.21.15.82:0/2101297068 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f929c19e490 con 0x7f929c1062f0 2023-09-09T23:13:13.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.603+0000 7f92927fc700 1 -- 172.21.15.82:0/2101297068 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929401bc40 con 0x7f929c1062f0 2023-09-09T23:13:13.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.603+0000 7f92a3710700 1 -- 172.21.15.82:0/2101297068 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f929c19e910 con 0x7f929c1062f0 2023-09-09T23:13:13.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.605+0000 7f92927fc700 1 -- 172.21.15.82:0/2101297068 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f929401d690 con 0x7f929c1062f0 2023-09-09T23:13:13.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.606+0000 7f92927fc700 1 --2- 172.21.15.82:0/2101297068 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f92880a0750 0x7f92880a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.606+0000 7f92a14ac700 1 --2- 172.21.15.82:0/2101297068 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f92880a0750 0x7f92880a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.606+0000 7f92927fc700 1 -- 172.21.15.82:0/2101297068 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f92940cd980 con 0x7f929c1062f0 2023-09-09T23:13:13.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.606+0000 7f92a14ac700 1 --2- 172.21.15.82:0/2101297068 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f92880a0750 0x7f92880a2c10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9298005950 tx=0x7f92980058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:13.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.607+0000 7f92af8d6b80 1 Processor -- start 2023-09-09T23:13:13.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.607+0000 7f92af8d6b80 1 -- start start 2023-09-09T23:13:13.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.607+0000 7f92af8d6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6ffce9bb0 0x55f6ffce9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.607+0000 7f92af8d6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6ffcea550 0x55f6ffce87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.607+0000 7f92af8d6b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ffce8db0 0x55f6ffce9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.607+0000 7f92af8d6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ffc416e0 con 0x55f6ffce9bb0 2023-09-09T23:13:13.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.607+0000 7f92af8d6b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ffc3f000 con 0x55f6ffce8db0 2023-09-09T23:13:13.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.607+0000 7f92af8d6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ffc3a8f0 con 0x55f6ffcea550 2023-09-09T23:13:13.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.607+0000 7f92a14ac700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6ffce9bb0 0x55f6ffce9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.607+0000 7f92a14ac700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6ffce9bb0 0x55f6ffce9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55308/0 (socket says 172.21.15.82:55308) 2023-09-09T23:13:13.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.607+0000 7f92a14ac700 1 -- 172.21.15.82:0/3774609725 learned_addr learned my addr 172.21.15.82:0/3774609725 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:13.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.607+0000 7f92a0cab700 1 --2- 172.21.15.82:0/3774609725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6ffcea550 0x55f6ffce87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.607+0000 7f92a1cad700 1 --2- 172.21.15.82:0/3774609725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ffce8db0 0x55f6ffce9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.607+0000 7f92a14ac700 1 -- 172.21.15.82:0/3774609725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6ffcea550 msgr2=0x55f6ffce87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:13.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.607+0000 7f92a14ac700 1 --2- 172.21.15.82:0/3774609725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6ffcea550 0x55f6ffce87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.607+0000 7f92a14ac700 1 -- 172.21.15.82:0/3774609725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ffce8db0 msgr2=0x55f6ffce9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:13.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.607+0000 7f92a14ac700 1 --2- 172.21.15.82:0/3774609725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ffce8db0 0x55f6ffce9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.607+0000 7f92a14ac700 1 -- 172.21.15.82:0/3774609725 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6ffa66d90 con 0x55f6ffce9bb0 2023-09-09T23:13:13.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.607+0000 7f92a1cad700 1 --2- 172.21.15.82:0/3774609725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ffce8db0 0x55f6ffce9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:13.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.607+0000 7f92a0cab700 1 --2- 172.21.15.82:0/3774609725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6ffcea550 0x55f6ffce87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:13.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.607+0000 7f92a14ac700 1 --2- 172.21.15.82:0/3774609725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6ffce9bb0 0x55f6ffce9fd0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9298009270 tx=0x7f9298010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3371f23d2de5f4b3 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:13.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.608+0000 7f92877fe700 1 -- 172.21.15.82:0/3774609725 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9298010ac0 con 0x55f6ffce9bb0 2023-09-09T23:13:13.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.608+0000 7f92877fe700 1 -- 172.21.15.82:0/3774609725 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9298010c20 con 0x55f6ffce9bb0 2023-09-09T23:13:13.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.608+0000 7f92877fe700 1 -- 172.21.15.82:0/3774609725 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929800f620 con 0x55f6ffce9bb0 2023-09-09T23:13:13.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.608+0000 7f92af8d6b80 1 -- 172.21.15.82:0/3774609725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6ffce9bb0 msgr2=0x55f6ffce9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.608+0000 7f92af8d6b80 1 --2- 172.21.15.82:0/3774609725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6ffce9bb0 0x55f6ffce9fd0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9298009270 tx=0x7f9298010000 comp rx=0 tx=0).stop 2023-09-09T23:13:13.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.608+0000 7f92af8d6b80 1 -- 172.21.15.82:0/3774609725 shutdown_connections 2023-09-09T23:13:13.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.608+0000 7f92af8d6b80 1 --2- 172.21.15.82:0/3774609725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ffce8db0 0x55f6ffce9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.608+0000 7f92af8d6b80 1 --2- 172.21.15.82:0/3774609725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6ffce9bb0 0x55f6ffce9fd0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.608+0000 7f92af8d6b80 1 --2- 172.21.15.82:0/3774609725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6ffcea550 0x55f6ffce87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.608+0000 7f92af8d6b80 1 -- 172.21.15.82:0/3774609725 >> 172.21.15.82:0/3774609725 conn(0x55f6ffc5d040 msgr2=0x55f6ffce36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:13.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.608+0000 7f92af8d6b80 1 -- 172.21.15.82:0/3774609725 shutdown_connections 2023-09-09T23:13:13.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.608+0000 7f92af8d6b80 1 -- 172.21.15.82:0/3774609725 wait complete. 2023-09-09T23:13:13.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.609+0000 7f92af8d6b80 1 -- start start 2023-09-09T23:13:13.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.609+0000 7f92af8d6b80 1 Processor -- start 2023-09-09T23:13:13.627 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.609+0000 7f92af8d6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6ffce8db0 0x55f6ffcf24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.627 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.609+0000 7f92af8d6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6ffce9bb0 0x55f6ffcf2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.627 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.609+0000 7f92af8d6b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ffcea550 0x55f6ffcf9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.627 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.609+0000 7f92af8d6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ffbd8b20 con 0x55f6ffce9bb0 2023-09-09T23:13:13.627 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.609+0000 7f92af8d6b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ffbd8820 con 0x55f6ffcea550 2023-09-09T23:13:13.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.609+0000 7f92af8d6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ffabc8d0 con 0x55f6ffce8db0 2023-09-09T23:13:13.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.609+0000 7f92af8d6b80 10 client.0 _ll_register_callbacks cb 0x55f6ffb22a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:13.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.609+0000 7f92a0cab700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6ffce9bb0 0x55f6ffcf2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.609+0000 7f92a0cab700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6ffce9bb0 0x55f6ffcf2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55316/0 (socket says 172.21.15.82:55316) 2023-09-09T23:13:13.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.609+0000 7f92a0cab700 1 -- 172.21.15.82:0/2426427847 learned_addr learned my addr 172.21.15.82:0/2426427847 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:13.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.609+0000 7f92a14ac700 1 --2- 172.21.15.82:0/2426427847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6ffce8db0 0x55f6ffcf24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.609+0000 7f92a0cab700 1 -- 172.21.15.82:0/2426427847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6ffce8db0 msgr2=0x55f6ffcf24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:13.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.609+0000 7f92a0cab700 1 --2- 172.21.15.82:0/2426427847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6ffce8db0 0x55f6ffcf24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.610+0000 7f92a0cab700 1 -- 172.21.15.82:0/2426427847 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ffcea550 msgr2=0x55f6ffcf9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:13.630 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.610+0000 7f92a0cab700 1 --2- 172.21.15.82:0/2426427847 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ffcea550 0x55f6ffcf9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.630 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.610+0000 7f92a0cab700 1 -- 172.21.15.82:0/2426427847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f929800d960 con 0x55f6ffce9bb0 2023-09-09T23:13:13.630 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.610+0000 7f92a0cab700 1 --2- 172.21.15.82:0/2426427847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6ffce9bb0 0x55f6ffcf2a30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f928c0113e0 tx=0x7f928c0117a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f885cdc94eff69ce server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:13.630 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.610+0000 7f9284ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:13.630 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.610+0000 7f9284ff9700 1 -- 172.21.15.82:0/2426427847 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928c012650 con 0x55f6ffce9bb0 2023-09-09T23:13:13.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.610+0000 7f9284ff9700 1 -- 172.21.15.82:0/2426427847 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f928c0127b0 con 0x55f6ffce9bb0 2023-09-09T23:13:13.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.610+0000 7f92af8d6b80 1 -- 172.21.15.82:0/2426427847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6ffb924a0 con 0x55f6ffce9bb0 2023-09-09T23:13:13.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.610+0000 7f9284ff9700 1 -- 172.21.15.82:0/2426427847 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f928c011ed0 con 0x55f6ffce9bb0 2023-09-09T23:13:13.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.610+0000 7f92af8d6b80 10 client.15162 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:13.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.610+0000 7f92af8d6b80 1 -- 172.21.15.82:0/2426427847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f6ffa4fc10 con 0x55f6ffce9bb0 2023-09-09T23:13:13.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.610+0000 7f9284ff9700 1 -- 172.21.15.82:0/2426427847 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f6ffb924a0 con 0x55f6ffce9bb0 2023-09-09T23:13:13.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.610+0000 7f9284ff9700 1 -- 172.21.15.82:0/2426427847 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f928c01bc60 con 0x55f6ffce9bb0 2023-09-09T23:13:13.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.610+0000 7f92af8d6b80 10 client.15162 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:13.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.610+0000 7f92af8d6b80 10 client.15162 resolve_mds: resolved name 'k' to daemon mds.k 2023-09-09T23:13:13.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.610+0000 7f92af8d6b80 20 client.15162 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:13.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.610+0000 7f92af8d6b80 1 --2- 172.21.15.82:0/2426427847 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55f6ffcfdcf0 0x55f6ffcfcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.610+0000 7f92af8d6b80 4 client.15162 mds_command: new command op to 24491 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:13:13.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.610+0000 7f92af8d6b80 1 -- 172.21.15.82:0/2426427847 --> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6ffc4f850 con 0x55f6ffcfdcf0 2023-09-09T23:13:13.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.611+0000 7f92a14ac700 1 --2- 172.21.15.82:0/2426427847 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55f6ffcfdcf0 0x55f6ffcfcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.611+0000 7f9284ff9700 10 client.15162 ms_handle_connect on v2:172.21.15.98:6832/3855146755 2023-09-09T23:13:13.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.611+0000 7f92a14ac700 1 --2- 172.21.15.82:0/2426427847 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55f6ffcfdcf0 0x55f6ffcfcbd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bb894697f4d0fd7e server_cookie=8d4d3cc05e6c6da1 in_seq=0 out_seq=0 2023-09-09T23:13:13.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.612+0000 7f9284ff9700 1 -- 172.21.15.82:0/2426427847 <== mds.3 v2:172.21.15.98:6832/3855146755 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f6ffc4f850 con 0x55f6ffcfdcf0 2023-09-09T23:13:13.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.612+0000 7f9284ff9700 10 client.15162 handle_command_reply: tid=0 2023-09-09T23:13:13.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.612+0000 7f92af8d6b80 1 client.15162 shutdown 2023-09-09T23:13:13.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.612+0000 7f92af8d6b80 1 -- 172.21.15.82:0/2426427847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6ffce9bb0 msgr2=0x55f6ffcf2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.612+0000 7f92af8d6b80 1 --2- 172.21.15.82:0/2426427847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6ffce9bb0 0x55f6ffcf2a30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f928c0113e0 tx=0x7f928c0117a0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.612+0000 7f92af8d6b80 1 -- 172.21.15.82:0/2426427847 shutdown_connections 2023-09-09T23:13:13.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.612+0000 7f92af8d6b80 1 --2- 172.21.15.82:0/2426427847 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55f6ffcfdcf0 0x55f6ffcfcbd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.612+0000 7f92af8d6b80 1 --2- 172.21.15.82:0/2426427847 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ffcea550 0x55f6ffcf9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.612+0000 7f92af8d6b80 1 --2- 172.21.15.82:0/2426427847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6ffce8db0 0x55f6ffcf24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.612+0000 7f92af8d6b80 1 --2- 172.21.15.82:0/2426427847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6ffce9bb0 0x55f6ffcf2a30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.612+0000 7f9284ff9700 0 client.15162 ms_handle_reset on v2:172.21.15.98:6832/3855146755 2023-09-09T23:13:13.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.612+0000 7f92af8d6b80 1 -- 172.21.15.82:0/2426427847 >> 172.21.15.82:0/2426427847 conn(0x55f6ffc5d040 msgr2=0x55f6ffce2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:13.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.612+0000 7f92af8d6b80 1 -- 172.21.15.82:0/2426427847 shutdown_connections 2023-09-09T23:13:13.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.612+0000 7f92af8d6b80 1 -- 172.21.15.82:0/2426427847 wait complete. 2023-09-09T23:13:13.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.612+0000 7f92af8d6b80 20 client.15162 trim_cache size 0 max 16384 2023-09-09T23:13:13.643 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.630+0000 7f92af8d6b80 1 Processor -- start 2023-09-09T23:13:13.643 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.630+0000 7f92af8d6b80 1 -- start start 2023-09-09T23:13:13.643 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.630+0000 7f92af8d6b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ffce8db0 0x55f6ffce86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.643 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.630+0000 7f92af8d6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6ffce9bb0 0x55f6ffcf9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.644 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.630+0000 7f92af8d6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6ffcea550 0x55f6ffcfa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.644 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.630+0000 7f92af8d6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ffcf20c0 con 0x55f6ffcea550 2023-09-09T23:13:13.644 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.630+0000 7f92af8d6b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ffcff0e0 con 0x55f6ffce8db0 2023-09-09T23:13:13.644 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.630+0000 7f92af8d6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ffac8de0 con 0x55f6ffce9bb0 2023-09-09T23:13:13.645 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.630+0000 7f92a1cad700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6ffcea550 0x55f6ffcfa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.645 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.630+0000 7f92a1cad700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6ffcea550 0x55f6ffcfa2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55326/0 (socket says 172.21.15.82:55326) 2023-09-09T23:13:13.645 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.630+0000 7f92a1cad700 1 -- 172.21.15.82:0/1201264922 learned_addr learned my addr 172.21.15.82:0/1201264922 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:13.645 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.630+0000 7f92a14ac700 1 --2- 172.21.15.82:0/1201264922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ffce8db0 0x55f6ffce86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.646 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.630+0000 7f92a0cab700 1 --2- 172.21.15.82:0/1201264922 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6ffce9bb0 0x55f6ffcf9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.646 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.630+0000 7f92a1cad700 1 -- 172.21.15.82:0/1201264922 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6ffce9bb0 msgr2=0x55f6ffcf9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:13.646 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.630+0000 7f92a1cad700 1 --2- 172.21.15.82:0/1201264922 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6ffce9bb0 0x55f6ffcf9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.646 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.630+0000 7f92a1cad700 1 -- 172.21.15.82:0/1201264922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ffce8db0 msgr2=0x55f6ffce86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:13.646 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.630+0000 7f92a1cad700 1 --2- 172.21.15.82:0/1201264922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ffce8db0 0x55f6ffce86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.647 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.630+0000 7f92a1cad700 1 -- 172.21.15.82:0/1201264922 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6ffa8d200 con 0x55f6ffcea550 2023-09-09T23:13:13.647 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.630+0000 7f92a14ac700 1 --2- 172.21.15.82:0/1201264922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ffce8db0 0x55f6ffce86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:13.647 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.631+0000 7f92a1cad700 1 --2- 172.21.15.82:0/1201264922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6ffcea550 0x55f6ffcfa2a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9294007ab0 tx=0x7f9294004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ff698f98dcc179 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:13.647 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.631+0000 7f9287fff700 1 -- 172.21.15.82:0/1201264922 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929409b0a0 con 0x55f6ffcea550 2023-09-09T23:13:13.647 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.631+0000 7f9287fff700 1 -- 172.21.15.82:0/1201264922 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f929409eed0 con 0x55f6ffcea550 2023-09-09T23:13:13.648 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.631+0000 7f9287fff700 1 -- 172.21.15.82:0/1201264922 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9294099440 con 0x55f6ffcea550 2023-09-09T23:13:13.648 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.631+0000 7f92af8d6b80 1 -- 172.21.15.82:0/1201264922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6ffcea550 msgr2=0x55f6ffcfa2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.649 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.631+0000 7f92af8d6b80 1 --2- 172.21.15.82:0/1201264922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6ffcea550 0x55f6ffcfa2a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9294007ab0 tx=0x7f9294004f90 comp rx=0 tx=0).stop 2023-09-09T23:13:13.649 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.631+0000 7f92af8d6b80 1 -- 172.21.15.82:0/1201264922 shutdown_connections 2023-09-09T23:13:13.649 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.631+0000 7f92af8d6b80 1 --2- 172.21.15.82:0/1201264922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ffce8db0 0x55f6ffce86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.650 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.631+0000 7f92af8d6b80 1 --2- 172.21.15.82:0/1201264922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6ffcea550 0x55f6ffcfa2a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.650 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.631+0000 7f92af8d6b80 1 --2- 172.21.15.82:0/1201264922 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6ffce9bb0 0x55f6ffcf9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.650 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.631+0000 7f92af8d6b80 1 -- 172.21.15.82:0/1201264922 >> 172.21.15.82:0/1201264922 conn(0x55f6ffc5d040 msgr2=0x55f6ffce3680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:13.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.631+0000 7f92af8d6b80 1 -- 172.21.15.82:0/1201264922 shutdown_connections 2023-09-09T23:13:13.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.631+0000 7f92af8d6b80 1 -- 172.21.15.82:0/1201264922 wait complete. 2023-09-09T23:13:13.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.632+0000 7f92af8d6b80 1 -- start start 2023-09-09T23:13:13.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.632+0000 7f92af8d6b80 1 Processor -- start 2023-09-09T23:13:13.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.632+0000 7f92af8d6b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ffce8db0 0x55f6ffcf4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.632+0000 7f92af8d6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6ffce9bb0 0x55f6ffcf51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.632+0000 7f92af8d6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6ffcea550 0x55f6ffd51d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.632+0000 7f92af8d6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ffcf9c80 con 0x55f6ffcea550 2023-09-09T23:13:13.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.632+0000 7f92af8d6b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ffcf9df0 con 0x55f6ffce8db0 2023-09-09T23:13:13.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.632+0000 7f92af8d6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ffd52280 con 0x55f6ffce9bb0 2023-09-09T23:13:13.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.632+0000 7f92af8d6b80 10 client.0 _ll_register_callbacks cb 0x7f928c00ba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:13.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.632+0000 7f92a1cad700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6ffcea550 0x55f6ffd51d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.632+0000 7f92a1cad700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6ffcea550 0x55f6ffd51d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55332/0 (socket says 172.21.15.82:55332) 2023-09-09T23:13:13.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.632+0000 7f92a1cad700 1 -- 172.21.15.82:0/1875501062 learned_addr learned my addr 172.21.15.82:0/1875501062 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:13.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.632+0000 7f92a14ac700 1 --2- 172.21.15.82:0/1875501062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ffce8db0 0x55f6ffcf4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.632+0000 7f92a0cab700 1 --2- 172.21.15.82:0/1875501062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6ffce9bb0 0x55f6ffcf51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.632+0000 7f92a1cad700 1 -- 172.21.15.82:0/1875501062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6ffce9bb0 msgr2=0x55f6ffcf51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:13.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.632+0000 7f92a1cad700 1 --2- 172.21.15.82:0/1875501062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6ffce9bb0 0x55f6ffcf51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.632+0000 7f92a1cad700 1 -- 172.21.15.82:0/1875501062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ffce8db0 msgr2=0x55f6ffcf4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:13.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.632+0000 7f92a1cad700 1 --2- 172.21.15.82:0/1875501062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ffce8db0 0x55f6ffcf4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.632+0000 7f92a1cad700 1 -- 172.21.15.82:0/1875501062 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f929c19e910 con 0x55f6ffcea550 2023-09-09T23:13:13.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.633+0000 7f92a1cad700 1 --2- 172.21.15.82:0/1875501062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6ffcea550 0x55f6ffd51d40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f929401df90 tx=0x7f929409b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730677a38f18fdfe server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:13.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.633+0000 7f9284ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:13.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.633+0000 7f9284ff9700 1 -- 172.21.15.82:0/1875501062 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92940a3080 con 0x55f6ffcea550 2023-09-09T23:13:13.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.633+0000 7f9284ff9700 1 -- 172.21.15.82:0/1875501062 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f9294099440 con 0x55f6ffcea550 2023-09-09T23:13:13.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.633+0000 7f92af8d6b80 1 -- 172.21.15.82:0/1875501062 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9298010c20 con 0x55f6ffcea550 2023-09-09T23:13:13.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.633+0000 7f9284ff9700 1 -- 172.21.15.82:0/1875501062 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f929409b300 con 0x55f6ffcea550 2023-09-09T23:13:13.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.633+0000 7f92af8d6b80 10 client.15168 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:13.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.633+0000 7f92af8d6b80 1 -- 172.21.15.82:0/1875501062 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f6ffa65260 con 0x55f6ffcea550 2023-09-09T23:13:13.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.633+0000 7f9284ff9700 1 -- 172.21.15.82:0/1875501062 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f929407d020 con 0x55f6ffcea550 2023-09-09T23:13:13.659 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.633+0000 7f9284ff9700 1 -- 172.21.15.82:0/1875501062 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f92940a2bf0 con 0x55f6ffcea550 2023-09-09T23:13:13.659 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.633+0000 7f92af8d6b80 10 client.15168 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:13.659 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.633+0000 7f92af8d6b80 10 client.15168 resolve_mds: resolved name 'k' to daemon mds.k 2023-09-09T23:13:13.659 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.633+0000 7f92af8d6b80 20 client.15168 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:13.659 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.633+0000 7f92af8d6b80 1 --2- 172.21.15.82:0/1875501062 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55f6ffd56f90 0x55f6ffd53450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.633+0000 7f92af8d6b80 4 client.15168 mds_command: new command op to 24491 tid=0[{"prefix": "status"}] 2023-09-09T23:13:13.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.633+0000 7f92af8d6b80 1 -- 172.21.15.82:0/1875501062 --> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f6ffc338d0 con 0x55f6ffd56f90 2023-09-09T23:13:13.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.633+0000 7f92a14ac700 1 --2- 172.21.15.82:0/1875501062 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55f6ffd56f90 0x55f6ffd53450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.634+0000 7f9284ff9700 10 client.15168 ms_handle_connect on v2:172.21.15.98:6832/3855146755 2023-09-09T23:13:13.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.634+0000 7f92a14ac700 1 --2- 172.21.15.82:0/1875501062 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55f6ffd56f90 0x55f6ffd53450 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f68b586e928bea78 server_cookie=862ce2efc11c7ce7 in_seq=0 out_seq=0 2023-09-09T23:13:13.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.634+0000 7f9284ff9700 1 -- 172.21.15.82:0/1875501062 <== mds.3 v2:172.21.15.98:6832/3855146755 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55f6ffc338d0 con 0x55f6ffd56f90 2023-09-09T23:13:13.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.634+0000 7f9284ff9700 10 client.15168 handle_command_reply: tid=0 2023-09-09T23:13:13.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.634+0000 7f92af8d6b80 1 client.15168 shutdown 2023-09-09T23:13:13.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.634+0000 7f92af8d6b80 1 -- 172.21.15.82:0/1875501062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6ffcea550 msgr2=0x55f6ffd51d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.634+0000 7f92af8d6b80 1 --2- 172.21.15.82:0/1875501062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6ffcea550 0x55f6ffd51d40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f929401df90 tx=0x7f929409b860 comp rx=0 tx=0).stop 2023-09-09T23:13:13.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.634+0000 7f92af8d6b80 1 -- 172.21.15.82:0/1875501062 shutdown_connections 2023-09-09T23:13:13.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.634+0000 7f92af8d6b80 1 --2- 172.21.15.82:0/1875501062 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55f6ffd56f90 0x55f6ffd53450 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.634+0000 7f92af8d6b80 1 --2- 172.21.15.82:0/1875501062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6ffce8db0 0x55f6ffcf4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.634+0000 7f92af8d6b80 1 --2- 172.21.15.82:0/1875501062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6ffcea550 0x55f6ffd51d40 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.634+0000 7f92af8d6b80 1 --2- 172.21.15.82:0/1875501062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6ffce9bb0 0x55f6ffcf51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.634+0000 7f9284ff9700 0 client.15168 ms_handle_reset on v2:172.21.15.98:6832/3855146755 2023-09-09T23:13:13.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.634+0000 7f92af8d6b80 1 -- 172.21.15.82:0/1875501062 >> 172.21.15.82:0/1875501062 conn(0x55f6ffc5d040 msgr2=0x55f6ffcfcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:13.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.634+0000 7f92af8d6b80 1 -- 172.21.15.82:0/1875501062 shutdown_connections 2023-09-09T23:13:13.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.634+0000 7f92af8d6b80 1 -- 172.21.15.82:0/1875501062 wait complete. 2023-09-09T23:13:13.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.635+0000 7f92af8d6b80 20 client.15168 trim_cache size 0 max 16384 2023-09-09T23:13:13.664 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:13:13.664 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da", 2023-09-09T23:13:13.664 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 3, 2023-09-09T23:13:13.664 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24491, 2023-09-09T23:13:13.665 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2023-09-09T23:13:13.665 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2023-09-09T23:13:13.665 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2023-09-09T23:13:13.665 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 50.374487049999999, 2023-09-09T23:13:13.665 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 27, 2023-09-09T23:13:13.666 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 83, 2023-09-09T23:13:13.666 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 81, 2023-09-09T23:13:13.666 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 83.752315257999996 2023-09-09T23:13:13.666 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:13:13.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.635+0000 7f92a3710700 1 -- 172.21.15.82:0/2101297068 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f92880a0750 msgr2=0x7f92880a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.635+0000 7f92a3710700 1 --2- 172.21.15.82:0/2101297068 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f92880a0750 0x7f92880a2c10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9298005950 tx=0x7f92980058e0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.635+0000 7f92a3710700 1 -- 172.21.15.82:0/2101297068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f929c1062f0 msgr2=0x7f929c19d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.635+0000 7f92a3710700 1 --2- 172.21.15.82:0/2101297068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f929c1062f0 0x7f929c19d970 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f92940062a0 tx=0x7f9294004db0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.635+0000 7f92a3710700 1 -- 172.21.15.82:0/2101297068 shutdown_connections 2023-09-09T23:13:13.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.635+0000 7f92a3710700 1 --2- 172.21.15.82:0/2101297068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f929c1062f0 0x7f929c19d970 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.635+0000 7f92a3710700 1 --2- 172.21.15.82:0/2101297068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f929c101090 0x7f929c1963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.635+0000 7f92a3710700 1 --2- 172.21.15.82:0/2101297068 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f92880a0750 0x7f92880a2c10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.635+0000 7f92a3710700 1 --2- 172.21.15.82:0/2101297068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f929c1039c0 0x7f929c196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.635+0000 7f92a3710700 1 -- 172.21.15.82:0/2101297068 >> 172.21.15.82:0/2101297068 conn(0x7f929c0faac0 msgr2=0x7f929c101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:13.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.635+0000 7f92a3710700 1 -- 172.21.15.82:0/2101297068 shutdown_connections 2023-09-09T23:13:13.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.635+0000 7f92a3710700 1 -- 172.21.15.82:0/2101297068 wait complete. 2023-09-09T23:13:13.670 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:13 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1629733169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:13:13.670 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:13 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1280719835' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:13.670 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:13 smithi082 ceph-mon[98694]: pgmap v285: 97 pgs: 2 peering, 95 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2023-09-09T23:13:13.670 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:13 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1381700950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:13:13.671 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-09-09T23:13:13.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:13 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1629733169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:13:13.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:13 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1280719835' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:13.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:13 smithi170 ceph-mon[102832]: pgmap v285: 97 pgs: 2 peering, 95 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2023-09-09T23:13:13.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:13 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1381700950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:13:13.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:13 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1629733169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:13:13.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:13 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1280719835' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:13.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:13 smithi098 ceph-mon[103124]: pgmap v285: 97 pgs: 2 peering, 95 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2023-09-09T23:13:13.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:13 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1381700950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:13:13.797 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.795+0000 7fb55e344700 1 -- 172.21.15.82:0/376947277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb558102210 msgr2=0x7fb558108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.797 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.795+0000 7fb55e344700 1 --2- 172.21.15.82:0/376947277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb558102210 0x7fb558108ac0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb55000b0a0 tx=0x7fb55000b3b0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.797 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.795+0000 7fb55e344700 1 -- 172.21.15.82:0/376947277 shutdown_connections 2023-09-09T23:13:13.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.795+0000 7fb55e344700 1 --2- 172.21.15.82:0/376947277 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5580fe8a0 0x7fb5580fecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.795+0000 7fb55e344700 1 --2- 172.21.15.82:0/376947277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb558102210 0x7fb558108ac0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.795+0000 7fb55e344700 1 --2- 172.21.15.82:0/376947277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb558101870 0x7fb558101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.795+0000 7fb55e344700 1 -- 172.21.15.82:0/376947277 >> 172.21.15.82:0/376947277 conn(0x7fb5580fa1d0 msgr2=0x7fb5580fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:13.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.795+0000 7fb55e344700 1 -- 172.21.15.82:0/376947277 shutdown_connections 2023-09-09T23:13:13.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.796+0000 7fb55e344700 1 -- 172.21.15.82:0/376947277 wait complete. 2023-09-09T23:13:13.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.796+0000 7fb55e344700 1 Processor -- start 2023-09-09T23:13:13.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.796+0000 7fb55e344700 1 -- start start 2023-09-09T23:13:13.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.796+0000 7fb55e344700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5580fe8a0 0x7fb558196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.796+0000 7fb55e344700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb558101870 0x7fb558196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.796+0000 7fb55e344700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb558102210 0x7fb55819d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.796+0000 7fb55e344700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55819dfc0 con 0x7fb5580fe8a0 2023-09-09T23:13:13.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.796+0000 7fb55e344700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55819e100 con 0x7fb558102210 2023-09-09T23:13:13.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.796+0000 7fb55e344700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55819e290 con 0x7fb558101870 2023-09-09T23:13:13.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.796+0000 7fb557fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5580fe8a0 0x7fb558196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.797+0000 7fb557fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5580fe8a0 0x7fb558196440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55340/0 (socket says 172.21.15.82:55340) 2023-09-09T23:13:13.801 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.797+0000 7fb557fff700 1 -- 172.21.15.82:0/379921632 learned_addr learned my addr 172.21.15.82:0/379921632 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:13.801 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.797+0000 7fb5577fe700 1 --2- 172.21.15.82:0/379921632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb558101870 0x7fb558196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.801 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.797+0000 7fb55c8e1700 1 --2- 172.21.15.82:0/379921632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb558102210 0x7fb55819d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.801 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.797+0000 7fb557fff700 1 -- 172.21.15.82:0/379921632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb558101870 msgr2=0x7fb558196980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.801 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.797+0000 7fb557fff700 1 --2- 172.21.15.82:0/379921632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb558101870 0x7fb558196980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.797+0000 7fb557fff700 1 -- 172.21.15.82:0/379921632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb558102210 msgr2=0x7fb55819d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.797+0000 7fb557fff700 1 --2- 172.21.15.82:0/379921632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb558102210 0x7fb55819d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.797+0000 7fb557fff700 1 -- 172.21.15.82:0/379921632 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb550009d00 con 0x7fb5580fe8a0 2023-09-09T23:13:13.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.797+0000 7fb55c8e1700 1 --2- 172.21.15.82:0/379921632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb558102210 0x7fb55819d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:13.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.797+0000 7fb5577fe700 1 --2- 172.21.15.82:0/379921632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb558101870 0x7fb558196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:13.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.797+0000 7fb557fff700 1 --2- 172.21.15.82:0/379921632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5580fe8a0 0x7fb558196440 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00b500 tx=0x7fb54c00b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:13.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.797+0000 7fb5557fa700 1 -- 172.21.15.82:0/379921632 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c011830 con 0x7fb5580fe8a0 2023-09-09T23:13:13.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.797+0000 7fb5557fa700 1 -- 172.21.15.82:0/379921632 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb54c011e50 con 0x7fb5580fe8a0 2023-09-09T23:13:13.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.797+0000 7fb5557fa700 1 -- 172.21.15.82:0/379921632 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c00f2e0 con 0x7fb5580fe8a0 2023-09-09T23:13:13.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.797+0000 7fb55e344700 1 -- 172.21.15.82:0/379921632 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb55819e570 con 0x7fb5580fe8a0 2023-09-09T23:13:13.804 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.797+0000 7fb55e344700 1 -- 172.21.15.82:0/379921632 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb55819eb00 con 0x7fb5580fe8a0 2023-09-09T23:13:13.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.800+0000 7fb5557fa700 1 -- 172.21.15.82:0/379921632 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fb54c010580 con 0x7fb5580fe8a0 2023-09-09T23:13:13.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.800+0000 7fb5557fa700 1 --2- 172.21.15.82:0/379921632 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb5400a0750 0x7fb5400a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.800+0000 7fb5577fe700 1 --2- 172.21.15.82:0/379921632 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb5400a0750 0x7fb5400a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.800+0000 7fb5557fa700 1 -- 172.21.15.82:0/379921632 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb54c0cb5a0 con 0x7fb5580fe8a0 2023-09-09T23:13:13.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.801+0000 7fb5577fe700 1 --2- 172.21.15.82:0/379921632 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb5400a0750 0x7fb5400a2c10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb548005950 tx=0x7fb5480058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:13.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.801+0000 7fb56a50ab80 1 Processor -- start 2023-09-09T23:13:13.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.801+0000 7fb56a50ab80 1 -- start start 2023-09-09T23:13:13.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.801+0000 7fb56a50ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c398649bb0 0x55c398649fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.801+0000 7fb56a50ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c39864a550 0x55c3986487b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.801+0000 7fb56a50ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c398648db0 0x55c398649260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.801+0000 7fb56a50ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3985a16e0 con 0x55c398649bb0 2023-09-09T23:13:13.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.801+0000 7fb56a50ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39859f000 con 0x55c39864a550 2023-09-09T23:13:13.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.801+0000 7fb56a50ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39859a8f0 con 0x55c398648db0 2023-09-09T23:13:13.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.801+0000 7fb557fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c398649bb0 0x55c398649fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.801+0000 7fb557fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c398649bb0 0x55c398649fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55348/0 (socket says 172.21.15.82:55348) 2023-09-09T23:13:13.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.801+0000 7fb557fff700 1 -- 172.21.15.82:0/1278795287 learned_addr learned my addr 172.21.15.82:0/1278795287 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:13.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.801+0000 7fb5577fe700 1 --2- 172.21.15.82:0/1278795287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c39864a550 0x55c3986487b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.801+0000 7fb557fff700 1 -- 172.21.15.82:0/1278795287 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c398648db0 msgr2=0x55c398649260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:13.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.801+0000 7fb557fff700 1 --2- 172.21.15.82:0/1278795287 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c398648db0 0x55c398649260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.802+0000 7fb557fff700 1 -- 172.21.15.82:0/1278795287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c39864a550 msgr2=0x55c3986487b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:13.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.802+0000 7fb557fff700 1 --2- 172.21.15.82:0/1278795287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c39864a550 0x55c3986487b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.802+0000 7fb557fff700 1 -- 172.21.15.82:0/1278795287 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3983c6d90 con 0x55c398649bb0 2023-09-09T23:13:13.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.802+0000 7fb5577fe700 1 --2- 172.21.15.82:0/1278795287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c39864a550 0x55c3986487b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:13.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.802+0000 7fb557fff700 1 --2- 172.21.15.82:0/1278795287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c398649bb0 0x55c398649fd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb54c095340 tx=0x7fb54c098a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a842f868e884fb3 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:13.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.802+0000 7fb53e7fc700 1 -- 172.21.15.82:0/1278795287 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c0b1210 con 0x55c398649bb0 2023-09-09T23:13:13.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.802+0000 7fb53e7fc700 1 -- 172.21.15.82:0/1278795287 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb54c099540 con 0x55c398649bb0 2023-09-09T23:13:13.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.802+0000 7fb53e7fc700 1 -- 172.21.15.82:0/1278795287 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c099300 con 0x55c398649bb0 2023-09-09T23:13:13.816 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.802+0000 7fb56a50ab80 1 -- 172.21.15.82:0/1278795287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c398649bb0 msgr2=0x55c398649fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.816 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.802+0000 7fb56a50ab80 1 --2- 172.21.15.82:0/1278795287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c398649bb0 0x55c398649fd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb54c095340 tx=0x7fb54c098a70 comp rx=0 tx=0).stop 2023-09-09T23:13:13.816 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.803+0000 7fb56a50ab80 1 -- 172.21.15.82:0/1278795287 shutdown_connections 2023-09-09T23:13:13.816 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.803+0000 7fb56a50ab80 1 --2- 172.21.15.82:0/1278795287 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c398648db0 0x55c398649260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.816 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.803+0000 7fb56a50ab80 1 --2- 172.21.15.82:0/1278795287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c398649bb0 0x55c398649fd0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.803+0000 7fb56a50ab80 1 --2- 172.21.15.82:0/1278795287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c39864a550 0x55c3986487b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.803+0000 7fb56a50ab80 1 -- 172.21.15.82:0/1278795287 >> 172.21.15.82:0/1278795287 conn(0x55c3985bd040 msgr2=0x55c3986436d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:13.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.803+0000 7fb56a50ab80 1 -- 172.21.15.82:0/1278795287 shutdown_connections 2023-09-09T23:13:13.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.803+0000 7fb56a50ab80 1 -- 172.21.15.82:0/1278795287 wait complete. 2023-09-09T23:13:13.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.803+0000 7fb56a50ab80 1 -- start start 2023-09-09T23:13:13.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.803+0000 7fb56a50ab80 1 Processor -- start 2023-09-09T23:13:13.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.803+0000 7fb56a50ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c398648db0 0x55c3986525e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.803+0000 7fb56a50ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c398649bb0 0x55c398652b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.803+0000 7fb56a50ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c39864a550 0x55c398659b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.803+0000 7fb56a50ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c398538b20 con 0x55c398649bb0 2023-09-09T23:13:13.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.803+0000 7fb56a50ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c398538820 con 0x55c398648db0 2023-09-09T23:13:13.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.803+0000 7fb56a50ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39841c8d0 con 0x55c39864a550 2023-09-09T23:13:13.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.803+0000 7fb56a50ab80 10 client.0 _ll_register_callbacks cb 0x55c398482a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:13.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.803+0000 7fb5577fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c398649bb0 0x55c398652b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.803+0000 7fb5577fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c398649bb0 0x55c398652b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55356/0 (socket says 172.21.15.82:55356) 2023-09-09T23:13:13.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.803+0000 7fb5577fe700 1 -- 172.21.15.82:0/2313051160 learned_addr learned my addr 172.21.15.82:0/2313051160 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:13.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.804+0000 7fb557fff700 1 --2- 172.21.15.82:0/2313051160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c398648db0 0x55c3986525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.804+0000 7fb5577fe700 1 -- 172.21.15.82:0/2313051160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c39864a550 msgr2=0x55c398659b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:13.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.804+0000 7fb5577fe700 1 --2- 172.21.15.82:0/2313051160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c39864a550 0x55c398659b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.804+0000 7fb5577fe700 1 -- 172.21.15.82:0/2313051160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c398648db0 msgr2=0x55c3986525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:13.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.804+0000 7fb5577fe700 1 --2- 172.21.15.82:0/2313051160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c398648db0 0x55c3986525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.804+0000 7fb5577fe700 1 -- 172.21.15.82:0/2313051160 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb55819eb00 con 0x55c398649bb0 2023-09-09T23:13:13.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.804+0000 7fb557fff700 1 --2- 172.21.15.82:0/2313051160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c398648db0 0x55c3986525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:13.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.804+0000 7fb5577fe700 1 --2- 172.21.15.82:0/2313051160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c398649bb0 0x55c398652b20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb54800ff30 tx=0x7fb548004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1489a38a623915 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:13.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.804+0000 7fb5377fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:13.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.804+0000 7fb5377fe700 1 -- 172.21.15.82:0/2313051160 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb548017040 con 0x55c398649bb0 2023-09-09T23:13:13.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.804+0000 7fb5377fe700 1 -- 172.21.15.82:0/2313051160 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb548003a90 con 0x55c398649bb0 2023-09-09T23:13:13.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.804+0000 7fb56a50ab80 1 -- 172.21.15.82:0/2313051160 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3984f24a0 con 0x55c398649bb0 2023-09-09T23:13:13.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.804+0000 7fb5377fe700 1 -- 172.21.15.82:0/2313051160 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb548004580 con 0x55c398649bb0 2023-09-09T23:13:13.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.804+0000 7fb56a50ab80 10 client.15180 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:13.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.804+0000 7fb56a50ab80 1 -- 172.21.15.82:0/2313051160 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c3983afc10 con 0x55c398649bb0 2023-09-09T23:13:13.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.804+0000 7fb5377fe700 1 -- 172.21.15.82:0/2313051160 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c3984f24a0 con 0x55c398649bb0 2023-09-09T23:13:13.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.804+0000 7fb5377fe700 1 -- 172.21.15.82:0/2313051160 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb54801f430 con 0x55c398649bb0 2023-09-09T23:13:13.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.804+0000 7fb56a50ab80 10 client.15180 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:13.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.804+0000 7fb56a50ab80 10 client.15180 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-09T23:13:13.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.804+0000 7fb56a50ab80 20 client.15180 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:13.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.804+0000 7fb56a50ab80 1 --2- 172.21.15.82:0/2313051160 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x55c39865dcd0 0x55c39865cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.804+0000 7fb56a50ab80 4 client.15180 mds_command: new command op to 24523 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:13:13.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.805+0000 7fb56a50ab80 1 -- 172.21.15.82:0/2313051160 --> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3985af850 con 0x55c39865dcd0 2023-09-09T23:13:13.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.805+0000 7fb557fff700 1 --2- 172.21.15.82:0/2313051160 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x55c39865dcd0 0x55c39865cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.805+0000 7fb557fff700 1 --2- 172.21.15.82:0/2313051160 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x55c39865dcd0 0x55c39865cbe0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=200eef8168e0e278 server_cookie=f3e7805af79d77a0 in_seq=0 out_seq=0 2023-09-09T23:13:13.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.805+0000 7fb5377fe700 10 client.15180 ms_handle_connect on v2:172.21.15.170:6838/3858369735 2023-09-09T23:13:13.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.806+0000 7fb5377fe700 1 -- 172.21.15.82:0/2313051160 <== mds.2 v2:172.21.15.170:6838/3858369735 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c3985af850 con 0x55c39865dcd0 2023-09-09T23:13:13.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.806+0000 7fb5377fe700 10 client.15180 handle_command_reply: tid=0 2023-09-09T23:13:13.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.806+0000 7fb56a50ab80 1 client.15180 shutdown 2023-09-09T23:13:13.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.806+0000 7fb56a50ab80 1 -- 172.21.15.82:0/2313051160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c398649bb0 msgr2=0x55c398652b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.806+0000 7fb56a50ab80 1 --2- 172.21.15.82:0/2313051160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c398649bb0 0x55c398652b20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb54800ff30 tx=0x7fb548004750 comp rx=0 tx=0).stop 2023-09-09T23:13:13.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.806+0000 7fb56a50ab80 1 -- 172.21.15.82:0/2313051160 shutdown_connections 2023-09-09T23:13:13.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.806+0000 7fb56a50ab80 1 --2- 172.21.15.82:0/2313051160 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x55c39865dcd0 0x55c39865cbe0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.806+0000 7fb56a50ab80 1 --2- 172.21.15.82:0/2313051160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c398648db0 0x55c3986525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.806+0000 7fb5377fe700 0 client.15180 ms_handle_reset on v2:172.21.15.170:6838/3858369735 2023-09-09T23:13:13.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.806+0000 7fb56a50ab80 1 --2- 172.21.15.82:0/2313051160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c39864a550 0x55c398659b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.806+0000 7fb56a50ab80 1 --2- 172.21.15.82:0/2313051160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c398649bb0 0x55c398652b20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.806+0000 7fb56a50ab80 1 -- 172.21.15.82:0/2313051160 >> 172.21.15.82:0/2313051160 conn(0x55c3985bd040 msgr2=0x55c398642d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:13.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.806+0000 7fb56a50ab80 1 -- 172.21.15.82:0/2313051160 shutdown_connections 2023-09-09T23:13:13.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.806+0000 7fb56a50ab80 1 -- 172.21.15.82:0/2313051160 wait complete. 2023-09-09T23:13:13.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.806+0000 7fb56a50ab80 20 client.15180 trim_cache size 0 max 16384 2023-09-09T23:13:13.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.824+0000 7fb56a50ab80 1 Processor -- start 2023-09-09T23:13:13.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.824+0000 7fb56a50ab80 1 -- start start 2023-09-09T23:13:13.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.824+0000 7fb56a50ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c398648db0 0x55c39865a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.824+0000 7fb56a50ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c398649bb0 0x55c398659a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.824+0000 7fb56a50ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c39864a550 0x55c39864cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.824+0000 7fb56a50ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3985af850 con 0x55c398649bb0 2023-09-09T23:13:13.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.824+0000 7fb56a50ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3984f24a0 con 0x55c398648db0 2023-09-09T23:13:13.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.824+0000 7fb56a50ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c398650b40 con 0x55c39864a550 2023-09-09T23:13:13.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.824+0000 7fb5577fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c398649bb0 0x55c398659a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.824+0000 7fb5577fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c398649bb0 0x55c398659a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55364/0 (socket says 172.21.15.82:55364) 2023-09-09T23:13:13.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.824+0000 7fb5577fe700 1 -- 172.21.15.82:0/916805529 learned_addr learned my addr 172.21.15.82:0/916805529 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:13.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.824+0000 7fb557fff700 1 --2- 172.21.15.82:0/916805529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c398648db0 0x55c39865a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.824+0000 7fb55c8e1700 1 --2- 172.21.15.82:0/916805529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c39864a550 0x55c39864cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.824+0000 7fb5577fe700 1 -- 172.21.15.82:0/916805529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c39864a550 msgr2=0x55c39864cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:13.840 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.824+0000 7fb5577fe700 1 --2- 172.21.15.82:0/916805529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c39864a550 0x55c39864cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.840 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.824+0000 7fb5577fe700 1 -- 172.21.15.82:0/916805529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c398648db0 msgr2=0x55c39865a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:13.840 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.824+0000 7fb5577fe700 1 --2- 172.21.15.82:0/916805529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c398648db0 0x55c39865a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.840 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.824+0000 7fb5577fe700 1 -- 172.21.15.82:0/916805529 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3983ed200 con 0x55c398649bb0 2023-09-09T23:13:13.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.824+0000 7fb55c8e1700 1 --2- 172.21.15.82:0/916805529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c39864a550 0x55c39864cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:13.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.824+0000 7fb5577fe700 1 --2- 172.21.15.82:0/916805529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c398649bb0 0x55c398659a80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb54800c010 tx=0x7fb54800bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f24a65241b5b38a server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:13.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.824+0000 7fb53effd700 1 -- 172.21.15.82:0/916805529 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54801f430 con 0x55c398649bb0 2023-09-09T23:13:13.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.825+0000 7fb53effd700 1 -- 172.21.15.82:0/916805529 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb54801f590 con 0x55c398649bb0 2023-09-09T23:13:13.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.825+0000 7fb53effd700 1 -- 172.21.15.82:0/916805529 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5480346e0 con 0x55c398649bb0 2023-09-09T23:13:13.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.825+0000 7fb56a50ab80 1 -- 172.21.15.82:0/916805529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c398649bb0 msgr2=0x55c398659a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.825+0000 7fb56a50ab80 1 --2- 172.21.15.82:0/916805529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c398649bb0 0x55c398659a80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb54800c010 tx=0x7fb54800bb10 comp rx=0 tx=0).stop 2023-09-09T23:13:13.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.825+0000 7fb56a50ab80 1 -- 172.21.15.82:0/916805529 shutdown_connections 2023-09-09T23:13:13.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.825+0000 7fb56a50ab80 1 --2- 172.21.15.82:0/916805529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c398648db0 0x55c39865a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.825+0000 7fb56a50ab80 1 --2- 172.21.15.82:0/916805529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c39864a550 0x55c39864cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.825+0000 7fb56a50ab80 1 --2- 172.21.15.82:0/916805529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c398649bb0 0x55c398659a80 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.825+0000 7fb56a50ab80 1 -- 172.21.15.82:0/916805529 >> 172.21.15.82:0/916805529 conn(0x55c3985bd040 msgr2=0x55c398650430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:13.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.825+0000 7fb56a50ab80 1 -- 172.21.15.82:0/916805529 shutdown_connections 2023-09-09T23:13:13.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.825+0000 7fb56a50ab80 1 -- 172.21.15.82:0/916805529 wait complete. 2023-09-09T23:13:13.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.825+0000 7fb56a50ab80 1 -- start start 2023-09-09T23:13:13.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.826+0000 7fb56a50ab80 1 Processor -- start 2023-09-09T23:13:13.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.826+0000 7fb56a50ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c398648db0 0x55c398656960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.826+0000 7fb56a50ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c398649bb0 0x55c398656ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.826+0000 7fb56a50ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c39864a550 0x55c3986b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.826+0000 7fb56a50ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c398657580 con 0x55c398648db0 2023-09-09T23:13:13.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.826+0000 7fb56a50ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3986b2260 con 0x55c398649bb0 2023-09-09T23:13:13.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.826+0000 7fb56a50ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3986b23a0 con 0x55c39864a550 2023-09-09T23:13:13.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.826+0000 7fb56a50ab80 10 client.0 _ll_register_callbacks cb 0x7fb54800bf00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:13.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.826+0000 7fb557fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c398648db0 0x55c398656960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.826+0000 7fb557fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c398648db0 0x55c398656960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55368/0 (socket says 172.21.15.82:55368) 2023-09-09T23:13:13.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.826+0000 7fb557fff700 1 -- 172.21.15.82:0/4001657350 learned_addr learned my addr 172.21.15.82:0/4001657350 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:13.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.826+0000 7fb55c8e1700 1 --2- 172.21.15.82:0/4001657350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c39864a550 0x55c3986b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.826+0000 7fb557fff700 1 -- 172.21.15.82:0/4001657350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c39864a550 msgr2=0x55c3986b1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:13.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.826+0000 7fb557fff700 1 --2- 172.21.15.82:0/4001657350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c39864a550 0x55c3986b1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.826+0000 7fb557fff700 1 -- 172.21.15.82:0/4001657350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c398649bb0 msgr2=0x55c398656ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:13.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.826+0000 7fb5577fe700 1 --2- 172.21.15.82:0/4001657350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c398649bb0 0x55c398656ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.826+0000 7fb557fff700 1 --2- 172.21.15.82:0/4001657350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c398649bb0 0x55c398656ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.826+0000 7fb557fff700 1 -- 172.21.15.82:0/4001657350 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3983afc10 con 0x55c398648db0 2023-09-09T23:13:13.848 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.826+0000 7fb55c8e1700 1 --2- 172.21.15.82:0/4001657350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c39864a550 0x55c3986b1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:13.848 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.826+0000 7fb5577fe700 1 --2- 172.21.15.82:0/4001657350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c398649bb0 0x55c398656ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:13.848 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.826+0000 7fb557fff700 1 --2- 172.21.15.82:0/4001657350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c398648db0 0x55c398656960 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb54c0958b0 tx=0x7fb54c004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb0cd6e223d5cca server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:13.848 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.826+0000 7fb5377fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:13.849 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.826+0000 7fb5377fe700 1 -- 172.21.15.82:0/4001657350 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c098980 con 0x55c398648db0 2023-09-09T23:13:13.849 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.827+0000 7fb5377fe700 1 -- 172.21.15.82:0/4001657350 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb54c0b1210 con 0x55c398648db0 2023-09-09T23:13:13.849 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.827+0000 7fb56a50ab80 1 -- 172.21.15.82:0/4001657350 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb54c099540 con 0x55c398648db0 2023-09-09T23:13:13.849 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.827+0000 7fb5377fe700 1 -- 172.21.15.82:0/4001657350 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb54c0cb3a0 con 0x55c398648db0 2023-09-09T23:13:13.849 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.827+0000 7fb56a50ab80 10 client.15186 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:13.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.827+0000 7fb56a50ab80 1 -- 172.21.15.82:0/4001657350 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c3983c5260 con 0x55c398648db0 2023-09-09T23:13:13.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.827+0000 7fb5377fe700 1 -- 172.21.15.82:0/4001657350 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb54c099540 con 0x55c398648db0 2023-09-09T23:13:13.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.827+0000 7fb5377fe700 1 -- 172.21.15.82:0/4001657350 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb54c0991a0 con 0x55c398648db0 2023-09-09T23:13:13.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.827+0000 7fb56a50ab80 10 client.15186 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:13.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.827+0000 7fb56a50ab80 10 client.15186 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-09T23:13:13.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.827+0000 7fb56a50ab80 20 client.15186 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:13.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.827+0000 7fb56a50ab80 1 --2- 172.21.15.82:0/4001657350 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x55c39865dcd0 0x55c3986b3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.827+0000 7fb56a50ab80 4 client.15186 mds_command: new command op to 24523 tid=0[{"prefix": "status"}] 2023-09-09T23:13:13.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.827+0000 7fb56a50ab80 1 -- 172.21.15.82:0/4001657350 --> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c3985938d0 con 0x55c39865dcd0 2023-09-09T23:13:13.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.827+0000 7fb5577fe700 1 --2- 172.21.15.82:0/4001657350 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x55c39865dcd0 0x55c3986b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.828+0000 7fb5377fe700 10 client.15186 ms_handle_connect on v2:172.21.15.170:6838/3858369735 2023-09-09T23:13:13.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.828+0000 7fb5577fe700 1 --2- 172.21.15.82:0/4001657350 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x55c39865dcd0 0x55c3986b3860 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1a4f8fb2d1887f62 server_cookie=36c3c66b0320d767 in_seq=0 out_seq=0 2023-09-09T23:13:13.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.828+0000 7fb5377fe700 1 -- 172.21.15.82:0/4001657350 <== mds.2 v2:172.21.15.170:6838/3858369735 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55c3985938d0 con 0x55c39865dcd0 2023-09-09T23:13:13.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.828+0000 7fb5377fe700 10 client.15186 handle_command_reply: tid=0 2023-09-09T23:13:13.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.828+0000 7fb56a50ab80 1 client.15186 shutdown 2023-09-09T23:13:13.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.828+0000 7fb56a50ab80 1 -- 172.21.15.82:0/4001657350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c398648db0 msgr2=0x55c398656960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.828+0000 7fb56a50ab80 1 --2- 172.21.15.82:0/4001657350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c398648db0 0x55c398656960 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb54c0958b0 tx=0x7fb54c004860 comp rx=0 tx=0).stop 2023-09-09T23:13:13.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.828+0000 7fb56a50ab80 1 -- 172.21.15.82:0/4001657350 shutdown_connections 2023-09-09T23:13:13.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.828+0000 7fb56a50ab80 1 --2- 172.21.15.82:0/4001657350 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x55c39865dcd0 0x55c3986b3860 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.828+0000 7fb56a50ab80 1 --2- 172.21.15.82:0/4001657350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c39864a550 0x55c3986b1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.828+0000 7fb5377fe700 0 client.15186 ms_handle_reset on v2:172.21.15.170:6838/3858369735 2023-09-09T23:13:13.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.828+0000 7fb56a50ab80 1 --2- 172.21.15.82:0/4001657350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c398648db0 0x55c398656960 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.828+0000 7fb56a50ab80 1 --2- 172.21.15.82:0/4001657350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c398649bb0 0x55c398656ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.828+0000 7fb56a50ab80 1 -- 172.21.15.82:0/4001657350 >> 172.21.15.82:0/4001657350 conn(0x55c3985bd040 msgr2=0x55c39865a390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:13.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.828+0000 7fb56a50ab80 1 -- 172.21.15.82:0/4001657350 shutdown_connections 2023-09-09T23:13:13.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.828+0000 7fb56a50ab80 1 -- 172.21.15.82:0/4001657350 wait complete. 2023-09-09T23:13:13.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.828+0000 7fb56a50ab80 20 client.15186 trim_cache size 0 max 16384 2023-09-09T23:13:13.856 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:13:13.856 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da", 2023-09-09T23:13:13.856 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 2, 2023-09-09T23:13:13.856 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24523, 2023-09-09T23:13:13.856 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2023-09-09T23:13:13.857 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2023-09-09T23:13:13.857 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2023-09-09T23:13:13.857 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 51.255796367000002, 2023-09-09T23:13:13.857 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 27, 2023-09-09T23:13:13.858 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 83, 2023-09-09T23:13:13.858 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 81, 2023-09-09T23:13:13.858 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 68.697915179999995 2023-09-09T23:13:13.858 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:13:13.860 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.829+0000 7fb55e344700 1 -- 172.21.15.82:0/379921632 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb5400a0750 msgr2=0x7fb5400a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.860 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.829+0000 7fb55e344700 1 --2- 172.21.15.82:0/379921632 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb5400a0750 0x7fb5400a2c10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb548005950 tx=0x7fb5480058e0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.860 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.829+0000 7fb55e344700 1 -- 172.21.15.82:0/379921632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5580fe8a0 msgr2=0x7fb558196440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.829+0000 7fb55e344700 1 --2- 172.21.15.82:0/379921632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5580fe8a0 0x7fb558196440 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00b500 tx=0x7fb54c00b8c0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.829+0000 7fb55e344700 1 -- 172.21.15.82:0/379921632 shutdown_connections 2023-09-09T23:13:13.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.829+0000 7fb55e344700 1 --2- 172.21.15.82:0/379921632 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb5400a0750 0x7fb5400a2c10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.829+0000 7fb55e344700 1 --2- 172.21.15.82:0/379921632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb558102210 0x7fb55819d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.862 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.829+0000 7fb55e344700 1 --2- 172.21.15.82:0/379921632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5580fe8a0 0x7fb558196440 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.862 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.829+0000 7fb55e344700 1 --2- 172.21.15.82:0/379921632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb558101870 0x7fb558196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.862 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.829+0000 7fb55e344700 1 -- 172.21.15.82:0/379921632 >> 172.21.15.82:0/379921632 conn(0x7fb5580fa1d0 msgr2=0x7fb5580fc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:13.862 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.829+0000 7fb55e344700 1 -- 172.21.15.82:0/379921632 shutdown_connections 2023-09-09T23:13:13.862 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.829+0000 7fb55e344700 1 -- 172.21.15.82:0/379921632 wait complete. 2023-09-09T23:13:13.863 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-09-09T23:13:13.990 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.990+0000 7f91bb55f700 1 -- 172.21.15.82:0/2882214804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91b4100620 msgr2=0x7f91b4100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.991 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.990+0000 7f91bb55f700 1 --2- 172.21.15.82:0/2882214804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91b4100620 0x7f91b4100a40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f91a40099c0 tx=0x7f91a4009cd0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.991 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.990+0000 7f91bb55f700 1 -- 172.21.15.82:0/2882214804 shutdown_connections 2023-09-09T23:13:13.991 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.990+0000 7f91bb55f700 1 --2- 172.21.15.82:0/2882214804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91b41021b0 0x7f91b4108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.990+0000 7f91bb55f700 1 --2- 172.21.15.82:0/2882214804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91b4100620 0x7f91b4100a40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.990+0000 7f91bb55f700 1 --2- 172.21.15.82:0/2882214804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91b4101810 0x7f91b4101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.990+0000 7f91bb55f700 1 -- 172.21.15.82:0/2882214804 >> 172.21.15.82:0/2882214804 conn(0x7f91b40fbb70 msgr2=0x7f91b40fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:13.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.990+0000 7f91bb55f700 1 -- 172.21.15.82:0/2882214804 shutdown_connections 2023-09-09T23:13:13.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.990+0000 7f91bb55f700 1 -- 172.21.15.82:0/2882214804 wait complete. 2023-09-09T23:13:13.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.990+0000 7f91bb55f700 1 Processor -- start 2023-09-09T23:13:13.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.991+0000 7f91bb55f700 1 -- start start 2023-09-09T23:13:13.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.991+0000 7f91bb55f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91b4100620 0x7f91b4196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.991+0000 7f91bb55f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91b4101810 0x7f91b4196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.991+0000 7f91bb55f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91b41021b0 0x7f91b419d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:13.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.991+0000 7f91bb55f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b419df90 con 0x7f91b4101810 2023-09-09T23:13:13.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.991+0000 7f91bb55f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b419e0d0 con 0x7f91b4100620 2023-09-09T23:13:13.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.991+0000 7f91bb55f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b419e260 con 0x7f91b41021b0 2023-09-09T23:13:13.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.991+0000 7f91b8afa700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91b4101810 0x7f91b4196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.996 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.991+0000 7f91b8afa700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91b4101810 0x7f91b4196950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55384/0 (socket says 172.21.15.82:55384) 2023-09-09T23:13:13.996 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.991+0000 7f91b8afa700 1 -- 172.21.15.82:0/3268533231 learned_addr learned my addr 172.21.15.82:0/3268533231 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:13.997 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.991+0000 7f91b92fb700 1 --2- 172.21.15.82:0/3268533231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91b4100620 0x7f91b4196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.997 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.991+0000 7f91b9afc700 1 --2- 172.21.15.82:0/3268533231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91b41021b0 0x7f91b419d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:13.997 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.991+0000 7f91b8afa700 1 -- 172.21.15.82:0/3268533231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91b41021b0 msgr2=0x7f91b419d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.998 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.991+0000 7f91b8afa700 1 --2- 172.21.15.82:0/3268533231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91b41021b0 0x7f91b419d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.998 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.991+0000 7f91b8afa700 1 -- 172.21.15.82:0/3268533231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91b4100620 msgr2=0x7f91b4196410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:13.998 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.991+0000 7f91b8afa700 1 --2- 172.21.15.82:0/3268533231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91b4100620 0x7f91b4196410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:13.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.991+0000 7f91b8afa700 1 -- 172.21.15.82:0/3268533231 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a40095f0 con 0x7f91b4101810 2023-09-09T23:13:13.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.991+0000 7f91b8afa700 1 --2- 172.21.15.82:0/3268533231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91b4101810 0x7f91b4196950 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f91b000b530 tx=0x7f91b000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:13.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.991+0000 7f91aa7fc700 1 -- 172.21.15.82:0/3268533231 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b0011830 con 0x7f91b4101810 2023-09-09T23:13:14.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.991+0000 7f91bb55f700 1 -- 172.21.15.82:0/3268533231 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b419e540 con 0x7f91b4101810 2023-09-09T23:13:14.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.991+0000 7f91aa7fc700 1 -- 172.21.15.82:0/3268533231 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91b0011e50 con 0x7f91b4101810 2023-09-09T23:13:14.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.991+0000 7f91aa7fc700 1 -- 172.21.15.82:0/3268533231 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b000f2e0 con 0x7f91b4101810 2023-09-09T23:13:14.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.991+0000 7f91bb55f700 1 -- 172.21.15.82:0/3268533231 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b419ea50 con 0x7f91b4101810 2023-09-09T23:13:14.008 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.993+0000 7f91aa7fc700 1 -- 172.21.15.82:0/3268533231 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f91b0010380 con 0x7f91b4101810 2023-09-09T23:13:14.008 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.994+0000 7f91aa7fc700 1 --2- 172.21.15.82:0/3268533231 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f91a00a06d0 0x7f91a00a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.008 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.994+0000 7f91aa7fc700 1 -- 172.21.15.82:0/3268533231 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f91b00cb6b0 con 0x7f91b4101810 2023-09-09T23:13:14.008 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.994+0000 7f91b92fb700 1 --2- 172.21.15.82:0/3268533231 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f91a00a06d0 0x7f91a00a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.994+0000 7f91b92fb700 1 --2- 172.21.15.82:0/3268533231 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f91a00a06d0 0x7f91a00a2b90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f91a4009f50 tx=0x7f91a400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:14.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.995+0000 7f91c7725b80 1 Processor -- start 2023-09-09T23:13:14.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.995+0000 7f91c7725b80 1 -- start start 2023-09-09T23:13:14.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.995+0000 7f91c7725b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a779a34bb0 0x55a779a34fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.995+0000 7f91c7725b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a779a35550 0x55a779a337b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.010 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.995+0000 7f91c7725b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a779a33db0 0x55a779a34260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.010 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.995+0000 7f91c7725b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77998c6e0 con 0x55a779a33db0 2023-09-09T23:13:14.010 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.995+0000 7f91c7725b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77998a000 con 0x55a779a35550 2023-09-09T23:13:14.011 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.995+0000 7f91c7725b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7799858f0 con 0x55a779a34bb0 2023-09-09T23:13:14.011 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.995+0000 7f91b9afc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a779a33db0 0x55a779a34260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.011 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.995+0000 7f91b9afc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a779a33db0 0x55a779a34260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55394/0 (socket says 172.21.15.82:55394) 2023-09-09T23:13:14.011 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.995+0000 7f91b9afc700 1 -- 172.21.15.82:0/856072937 learned_addr learned my addr 172.21.15.82:0/856072937 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:14.012 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.995+0000 7f91b8afa700 1 --2- 172.21.15.82:0/856072937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a779a35550 0x55a779a337b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.012 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.995+0000 7f91b92fb700 1 --2- 172.21.15.82:0/856072937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a779a34bb0 0x55a779a34fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.012 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.995+0000 7f91b9afc700 1 -- 172.21.15.82:0/856072937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a779a34bb0 msgr2=0x55a779a34fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:14.012 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.995+0000 7f91b9afc700 1 --2- 172.21.15.82:0/856072937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a779a34bb0 0x55a779a34fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.012 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.995+0000 7f91b9afc700 1 -- 172.21.15.82:0/856072937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a779a35550 msgr2=0x55a779a337b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:14.013 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.995+0000 7f91b9afc700 1 --2- 172.21.15.82:0/856072937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a779a35550 0x55a779a337b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.013 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.995+0000 7f91b9afc700 1 -- 172.21.15.82:0/856072937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7797b1d90 con 0x55a779a33db0 2023-09-09T23:13:14.013 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.996+0000 7f91b9afc700 1 --2- 172.21.15.82:0/856072937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a779a33db0 0x55a779a34260 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f91ac008550 tx=0x7f91ac008860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5131f53c0922346 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:14.013 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.996+0000 7f919f7fe700 1 -- 172.21.15.82:0/856072937 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac003c00 con 0x55a779a33db0 2023-09-09T23:13:14.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.996+0000 7f919f7fe700 1 -- 172.21.15.82:0/856072937 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91ac010040 con 0x55a779a33db0 2023-09-09T23:13:14.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.996+0000 7f919f7fe700 1 -- 172.21.15.82:0/856072937 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac012760 con 0x55a779a33db0 2023-09-09T23:13:14.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.996+0000 7f91c7725b80 1 -- 172.21.15.82:0/856072937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a779a33db0 msgr2=0x55a779a34260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:14.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.996+0000 7f91c7725b80 1 --2- 172.21.15.82:0/856072937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a779a33db0 0x55a779a34260 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f91ac008550 tx=0x7f91ac008860 comp rx=0 tx=0).stop 2023-09-09T23:13:14.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.996+0000 7f91c7725b80 1 -- 172.21.15.82:0/856072937 shutdown_connections 2023-09-09T23:13:14.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.996+0000 7f91c7725b80 1 --2- 172.21.15.82:0/856072937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a779a33db0 0x55a779a34260 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.996+0000 7f91c7725b80 1 --2- 172.21.15.82:0/856072937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a779a34bb0 0x55a779a34fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.996+0000 7f91c7725b80 1 --2- 172.21.15.82:0/856072937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a779a35550 0x55a779a337b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.996+0000 7f91c7725b80 1 -- 172.21.15.82:0/856072937 >> 172.21.15.82:0/856072937 conn(0x55a7799a8040 msgr2=0x55a779a2e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:14.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.996+0000 7f91c7725b80 1 -- 172.21.15.82:0/856072937 shutdown_connections 2023-09-09T23:13:14.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.996+0000 7f91c7725b80 1 -- 172.21.15.82:0/856072937 wait complete. 2023-09-09T23:13:14.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.997+0000 7f91c7725b80 1 -- start start 2023-09-09T23:13:14.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.997+0000 7f91c7725b80 1 Processor -- start 2023-09-09T23:13:14.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.997+0000 7f91c7725b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a779a33db0 0x55a779a3d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.997+0000 7f91c7725b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a779a34bb0 0x55a779a3da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.997+0000 7f91c7725b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a779a35550 0x55a779a44a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.997+0000 7f91c7725b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a779923b20 con 0x55a779a33db0 2023-09-09T23:13:14.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.997+0000 7f91c7725b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a779923820 con 0x55a779a35550 2023-09-09T23:13:14.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.997+0000 7f91c7725b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7798078d0 con 0x55a779a34bb0 2023-09-09T23:13:14.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.997+0000 7f91c7725b80 10 client.0 _ll_register_callbacks cb 0x55a77986da30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:14.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.997+0000 7f91b92fb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a779a33db0 0x55a779a3d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.997+0000 7f91b92fb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a779a33db0 0x55a779a3d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55396/0 (socket says 172.21.15.82:55396) 2023-09-09T23:13:14.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.997+0000 7f91b92fb700 1 -- 172.21.15.82:0/1778718103 learned_addr learned my addr 172.21.15.82:0/1778718103 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:14.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.998+0000 7f91b8afa700 1 --2- 172.21.15.82:0/1778718103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a779a34bb0 0x55a779a3da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.998+0000 7f91b9afc700 1 --2- 172.21.15.82:0/1778718103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a779a35550 0x55a779a44a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.998+0000 7f91b92fb700 1 -- 172.21.15.82:0/1778718103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a779a34bb0 msgr2=0x55a779a3da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:14.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.998+0000 7f91b92fb700 1 --2- 172.21.15.82:0/1778718103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a779a34bb0 0x55a779a3da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.998+0000 7f91b92fb700 1 -- 172.21.15.82:0/1778718103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a779a35550 msgr2=0x55a779a44a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:14.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.998+0000 7f91b92fb700 1 --2- 172.21.15.82:0/1778718103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a779a35550 0x55a779a44a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.998+0000 7f91b92fb700 1 -- 172.21.15.82:0/1778718103 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91ac007d90 con 0x55a779a33db0 2023-09-09T23:13:14.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.998+0000 7f91b92fb700 1 --2- 172.21.15.82:0/1778718103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a779a33db0 0x55a779a3d500 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f91b4101670 tx=0x7f91a401a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0bfd0e1df7ab3e server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:14.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.998+0000 7f919cff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:14.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.998+0000 7f919cff9700 1 -- 172.21.15.82:0/1778718103 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a40056b0 con 0x55a779a33db0 2023-09-09T23:13:14.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.998+0000 7f919cff9700 1 -- 172.21.15.82:0/1778718103 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f91a4004500 con 0x55a779a33db0 2023-09-09T23:13:14.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.998+0000 7f91c7725b80 1 -- 172.21.15.82:0/1778718103 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7798dd4a0 con 0x55a779a33db0 2023-09-09T23:13:14.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.998+0000 7f919cff9700 1 -- 172.21.15.82:0/1778718103 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91a40046c0 con 0x55a779a33db0 2023-09-09T23:13:14.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.998+0000 7f91c7725b80 10 client.15198 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:14.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.998+0000 7f91c7725b80 1 -- 172.21.15.82:0/1778718103 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a77979ac10 con 0x55a779a33db0 2023-09-09T23:13:14.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.998+0000 7f919cff9700 1 -- 172.21.15.82:0/1778718103 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a7798dd4a0 con 0x55a779a33db0 2023-09-09T23:13:14.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.998+0000 7f919cff9700 1 -- 172.21.15.82:0/1778718103 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f91a400f890 con 0x55a779a33db0 2023-09-09T23:13:14.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.999+0000 7f91c7725b80 10 client.15198 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:14.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.999+0000 7f91c7725b80 10 client.15198 resolve_mds: resolved name 'h' to daemon mds.h 2023-09-09T23:13:14.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.999+0000 7f91c7725b80 20 client.15198 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:14.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.999+0000 7f91c7725b80 1 --2- 172.21.15.82:0/1778718103 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x55a779a48aa0 0x55a779a47a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.999+0000 7f91c7725b80 4 client.15198 mds_command: new command op to 24542 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:13:14.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.999+0000 7f91c7725b80 1 -- 172.21.15.82:0/1778718103 --> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a77999a850 con 0x55a779a48aa0 2023-09-09T23:13:14.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.999+0000 7f91b8afa700 1 --2- 172.21.15.82:0/1778718103 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x55a779a48aa0 0x55a779a47a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.999+0000 7f919cff9700 10 client.15198 ms_handle_connect on v2:172.21.15.98:6836/2019449840 2023-09-09T23:13:14.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:13.999+0000 7f91b8afa700 1 --2- 172.21.15.82:0/1778718103 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x55a779a48aa0 0x55a779a47a60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=479614987ded4d9c server_cookie=e30ca26218c2d15c in_seq=0 out_seq=0 2023-09-09T23:13:14.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.000+0000 7f919cff9700 1 -- 172.21.15.82:0/1778718103 <== mds.1 v2:172.21.15.98:6836/2019449840 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a77999a850 con 0x55a779a48aa0 2023-09-09T23:13:14.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.000+0000 7f919cff9700 10 client.15198 handle_command_reply: tid=0 2023-09-09T23:13:14.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.000+0000 7f91c7725b80 1 client.15198 shutdown 2023-09-09T23:13:14.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.000+0000 7f91c7725b80 1 -- 172.21.15.82:0/1778718103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a779a33db0 msgr2=0x55a779a3d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:14.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.000+0000 7f91c7725b80 1 --2- 172.21.15.82:0/1778718103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a779a33db0 0x55a779a3d500 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f91b4101670 tx=0x7f91a401a050 comp rx=0 tx=0).stop 2023-09-09T23:13:14.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.000+0000 7f91c7725b80 1 -- 172.21.15.82:0/1778718103 shutdown_connections 2023-09-09T23:13:14.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.000+0000 7f91c7725b80 1 --2- 172.21.15.82:0/1778718103 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x55a779a48aa0 0x55a779a47a60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.000+0000 7f91c7725b80 1 --2- 172.21.15.82:0/1778718103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a779a35550 0x55a779a44a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.000+0000 7f919cff9700 0 client.15198 ms_handle_reset on v2:172.21.15.98:6836/2019449840 2023-09-09T23:13:14.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.000+0000 7f91c7725b80 1 --2- 172.21.15.82:0/1778718103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a779a33db0 0x55a779a3d500 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.000+0000 7f91c7725b80 1 --2- 172.21.15.82:0/1778718103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a779a34bb0 0x55a779a3da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.000+0000 7f91c7725b80 1 -- 172.21.15.82:0/1778718103 >> 172.21.15.82:0/1778718103 conn(0x55a7799a8040 msgr2=0x55a779a2dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:14.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.000+0000 7f91c7725b80 1 -- 172.21.15.82:0/1778718103 shutdown_connections 2023-09-09T23:13:14.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.000+0000 7f91c7725b80 1 -- 172.21.15.82:0/1778718103 wait complete. 2023-09-09T23:13:14.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.001+0000 7f91c7725b80 20 client.15198 trim_cache size 0 max 16384 2023-09-09T23:13:14.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.018+0000 7f91c7725b80 1 Processor -- start 2023-09-09T23:13:14.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.018+0000 7f91c7725b80 1 -- start start 2023-09-09T23:13:14.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.018+0000 7f91c7725b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a779a33db0 0x55a779a336c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.018+0000 7f91c7725b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a779a34bb0 0x55a779a44a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.018+0000 7f91c7725b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a779a35550 0x55a779a47a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.018+0000 7f91c7725b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7798dd4a0 con 0x55a779a35550 2023-09-09T23:13:14.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.018+0000 7f91c7725b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a779a49e90 con 0x55a779a34bb0 2023-09-09T23:13:14.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.018+0000 7f91c7725b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77992d4d0 con 0x55a779a33db0 2023-09-09T23:13:14.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.018+0000 7f91b9afc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a779a35550 0x55a779a47a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.018+0000 7f91b9afc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a779a35550 0x55a779a47a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55408/0 (socket says 172.21.15.82:55408) 2023-09-09T23:13:14.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.018+0000 7f91b9afc700 1 -- 172.21.15.82:0/3293008745 learned_addr learned my addr 172.21.15.82:0/3293008745 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:14.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.018+0000 7f91b8afa700 1 --2- 172.21.15.82:0/3293008745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a779a34bb0 0x55a779a44a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.019+0000 7f91b92fb700 1 --2- 172.21.15.82:0/3293008745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a779a33db0 0x55a779a336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.019+0000 7f91b9afc700 1 -- 172.21.15.82:0/3293008745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a779a33db0 msgr2=0x55a779a336c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.019+0000 7f91b9afc700 1 --2- 172.21.15.82:0/3293008745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a779a33db0 0x55a779a336c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.019+0000 7f91b9afc700 1 -- 172.21.15.82:0/3293008745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a779a34bb0 msgr2=0x55a779a44a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:14.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.019+0000 7f91b9afc700 1 --2- 172.21.15.82:0/3293008745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a779a34bb0 0x55a779a44a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.019+0000 7f91b9afc700 1 -- 172.21.15.82:0/3293008745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a779a3c550 con 0x55a779a35550 2023-09-09T23:13:14.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.019+0000 7f91b92fb700 1 --2- 172.21.15.82:0/3293008745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a779a33db0 0x55a779a336c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:14.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.019+0000 7f91b9afc700 1 --2- 172.21.15.82:0/3293008745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a779a35550 0x55a779a47a60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x55a779a29180 tx=0x7f91ac004930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4db96b6e59ca0ba server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:14.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.019+0000 7f919ffff700 1 -- 172.21.15.82:0/3293008745 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac007ac0 con 0x55a779a35550 2023-09-09T23:13:14.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.019+0000 7f919ffff700 1 -- 172.21.15.82:0/3293008745 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91ac007c20 con 0x55a779a35550 2023-09-09T23:13:14.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.019+0000 7f919ffff700 1 -- 172.21.15.82:0/3293008745 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac01f890 con 0x55a779a35550 2023-09-09T23:13:14.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.020+0000 7f91c7725b80 1 -- 172.21.15.82:0/3293008745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a779a35550 msgr2=0x55a779a47a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:14.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.020+0000 7f91c7725b80 1 --2- 172.21.15.82:0/3293008745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a779a35550 0x55a779a47a60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x55a779a29180 tx=0x7f91ac004930 comp rx=0 tx=0).stop 2023-09-09T23:13:14.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.020+0000 7f91c7725b80 1 -- 172.21.15.82:0/3293008745 shutdown_connections 2023-09-09T23:13:14.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.020+0000 7f91c7725b80 1 --2- 172.21.15.82:0/3293008745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a779a35550 0x55a779a47a60 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.041 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.020+0000 7f91c7725b80 1 --2- 172.21.15.82:0/3293008745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a779a33db0 0x55a779a336c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.041 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.020+0000 7f91c7725b80 1 --2- 172.21.15.82:0/3293008745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a779a34bb0 0x55a779a44a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.041 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.020+0000 7f91c7725b80 1 -- 172.21.15.82:0/3293008745 >> 172.21.15.82:0/3293008745 conn(0x55a7799a8040 msgr2=0x55a779a34190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:14.041 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.020+0000 7f91c7725b80 1 -- 172.21.15.82:0/3293008745 shutdown_connections 2023-09-09T23:13:14.041 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.020+0000 7f91c7725b80 1 -- 172.21.15.82:0/3293008745 wait complete. 2023-09-09T23:13:14.042 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.020+0000 7f91c7725b80 1 -- start start 2023-09-09T23:13:14.042 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.020+0000 7f91c7725b80 1 Processor -- start 2023-09-09T23:13:14.042 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.021+0000 7f91c7725b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a779a33db0 0x55a779a400a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.042 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.021+0000 7f91c7725b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a779a34bb0 0x55a779a405e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.042 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.021+0000 7f91c7725b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a779a35550 0x55a779a43600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.043 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.021+0000 7f91c7725b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a779a43b40 con 0x55a779a33db0 2023-09-09T23:13:14.043 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.021+0000 7f91c7725b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a779a43cb0 con 0x55a779a35550 2023-09-09T23:13:14.043 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.021+0000 7f91c7725b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a779a43e70 con 0x55a779a34bb0 2023-09-09T23:13:14.043 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.021+0000 7f91c7725b80 10 client.0 _ll_register_callbacks cb 0x55a77986da30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:14.043 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.021+0000 7f91b92fb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a779a33db0 0x55a779a400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.043 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.021+0000 7f91b92fb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a779a33db0 0x55a779a400a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55410/0 (socket says 172.21.15.82:55410) 2023-09-09T23:13:14.044 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.021+0000 7f91b92fb700 1 -- 172.21.15.82:0/3103389764 learned_addr learned my addr 172.21.15.82:0/3103389764 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:14.044 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.021+0000 7f91b9afc700 1 --2- 172.21.15.82:0/3103389764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a779a35550 0x55a779a43600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.044 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.021+0000 7f91b8afa700 1 --2- 172.21.15.82:0/3103389764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a779a34bb0 0x55a779a405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.045 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.021+0000 7f91b92fb700 1 -- 172.21.15.82:0/3103389764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a779a34bb0 msgr2=0x55a779a405e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:14.045 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.021+0000 7f91b92fb700 1 --2- 172.21.15.82:0/3103389764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a779a34bb0 0x55a779a405e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.045 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.021+0000 7f91b92fb700 1 -- 172.21.15.82:0/3103389764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a779a35550 msgr2=0x55a779a43600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:14.045 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.021+0000 7f91b92fb700 1 --2- 172.21.15.82:0/3103389764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a779a35550 0x55a779a43600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.045 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.021+0000 7f91b92fb700 1 -- 172.21.15.82:0/3103389764 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7797b1d90 con 0x55a779a33db0 2023-09-09T23:13:14.046 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.021+0000 7f91b9afc700 1 --2- 172.21.15.82:0/3103389764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a779a35550 0x55a779a43600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:14.046 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.021+0000 7f91b92fb700 1 --2- 172.21.15.82:0/3103389764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a779a33db0 0x55a779a400a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f91a40098e0 tx=0x7f91a400f820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f1bbf48b0fd0ee server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:14.046 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.021+0000 7f919cff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:14.047 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.021+0000 7f919cff9700 1 -- 172.21.15.82:0/3103389764 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a4004700 con 0x55a779a33db0 2023-09-09T23:13:14.047 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.021+0000 7f919cff9700 1 -- 172.21.15.82:0/3103389764 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f91a400fe10 con 0x55a779a33db0 2023-09-09T23:13:14.047 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.021+0000 7f91c7725b80 1 -- 172.21.15.82:0/3103389764 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91ac010040 con 0x55a779a33db0 2023-09-09T23:13:14.047 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.021+0000 7f919cff9700 1 -- 172.21.15.82:0/3103389764 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91a4003f30 con 0x55a779a33db0 2023-09-09T23:13:14.048 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.021+0000 7f91c7725b80 10 client.15204 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:14.048 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.022+0000 7f91c7725b80 1 -- 172.21.15.82:0/3103389764 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a7797d8200 con 0x55a779a33db0 2023-09-09T23:13:14.048 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.022+0000 7f919cff9700 1 -- 172.21.15.82:0/3103389764 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91ac010040 con 0x55a779a33db0 2023-09-09T23:13:14.048 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.022+0000 7f919cff9700 1 -- 172.21.15.82:0/3103389764 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f91a403b640 con 0x55a779a33db0 2023-09-09T23:13:14.049 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.022+0000 7f91c7725b80 10 client.15204 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:14.049 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.022+0000 7f91c7725b80 10 client.15204 resolve_mds: resolved name 'h' to daemon mds.h 2023-09-09T23:13:14.049 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.022+0000 7f91c7725b80 20 client.15204 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:14.049 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.022+0000 7f91c7725b80 1 --2- 172.21.15.82:0/3103389764 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x55a779aa2720 0x55a779a9eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.049 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.022+0000 7f91c7725b80 4 client.15204 mds_command: new command op to 24542 tid=0[{"prefix": "status"}] 2023-09-09T23:13:14.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.022+0000 7f91c7725b80 1 -- 172.21.15.82:0/3103389764 --> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a77997e8d0 con 0x55a779aa2720 2023-09-09T23:13:14.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.022+0000 7f91b8afa700 1 --2- 172.21.15.82:0/3103389764 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x55a779aa2720 0x55a779a9eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.023+0000 7f919cff9700 10 client.15204 ms_handle_connect on v2:172.21.15.98:6836/2019449840 2023-09-09T23:13:14.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.023+0000 7f91b8afa700 1 --2- 172.21.15.82:0/3103389764 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x55a779aa2720 0x55a779a9eb80 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c1753992e8625c0 server_cookie=2148dbaee77a7161 in_seq=0 out_seq=0 2023-09-09T23:13:14.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.023+0000 7f919cff9700 1 -- 172.21.15.82:0/3103389764 <== mds.1 v2:172.21.15.98:6836/2019449840 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55a77997e8d0 con 0x55a779aa2720 2023-09-09T23:13:14.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.023+0000 7f919cff9700 10 client.15204 handle_command_reply: tid=0 2023-09-09T23:13:14.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.023+0000 7f91c7725b80 1 client.15204 shutdown 2023-09-09T23:13:14.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.023+0000 7f91c7725b80 1 -- 172.21.15.82:0/3103389764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a779a33db0 msgr2=0x55a779a400a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:14.052 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.023+0000 7f91c7725b80 1 --2- 172.21.15.82:0/3103389764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a779a33db0 0x55a779a400a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f91a40098e0 tx=0x7f91a400f820 comp rx=0 tx=0).stop 2023-09-09T23:13:14.052 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.023+0000 7f91c7725b80 1 -- 172.21.15.82:0/3103389764 shutdown_connections 2023-09-09T23:13:14.052 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.023+0000 7f91c7725b80 1 --2- 172.21.15.82:0/3103389764 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x55a779aa2720 0x55a779a9eb80 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.052 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.023+0000 7f91c7725b80 1 --2- 172.21.15.82:0/3103389764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a779a35550 0x55a779a43600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.052 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.023+0000 7f919cff9700 0 client.15204 ms_handle_reset on v2:172.21.15.98:6836/2019449840 2023-09-09T23:13:14.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.023+0000 7f91c7725b80 1 --2- 172.21.15.82:0/3103389764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a779a33db0 0x55a779a400a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.023+0000 7f91c7725b80 1 --2- 172.21.15.82:0/3103389764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a779a34bb0 0x55a779a405e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.023+0000 7f91c7725b80 1 -- 172.21.15.82:0/3103389764 >> 172.21.15.82:0/3103389764 conn(0x55a7799a8040 msgr2=0x55a779a47a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:14.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.023+0000 7f91c7725b80 1 -- 172.21.15.82:0/3103389764 shutdown_connections 2023-09-09T23:13:14.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.023+0000 7f91c7725b80 1 -- 172.21.15.82:0/3103389764 wait complete. 2023-09-09T23:13:14.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.023+0000 7f91c7725b80 20 client.15204 trim_cache size 0 max 16384 2023-09-09T23:13:14.054 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:13:14.054 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da", 2023-09-09T23:13:14.055 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 1, 2023-09-09T23:13:14.055 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24542, 2023-09-09T23:13:14.055 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2023-09-09T23:13:14.055 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2023-09-09T23:13:14.056 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2023-09-09T23:13:14.056 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 52.459281578000002, 2023-09-09T23:13:14.056 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 27, 2023-09-09T23:13:14.056 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 83, 2023-09-09T23:13:14.056 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 81, 2023-09-09T23:13:14.057 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 71.159520697000005 2023-09-09T23:13:14.057 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:13:14.058 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.024+0000 7f91bb55f700 1 -- 172.21.15.82:0/3268533231 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f91a00a06d0 msgr2=0x7f91a00a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:14.058 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.024+0000 7f91bb55f700 1 --2- 172.21.15.82:0/3268533231 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f91a00a06d0 0x7f91a00a2b90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f91a4009f50 tx=0x7f91a400b540 comp rx=0 tx=0).stop 2023-09-09T23:13:14.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.024+0000 7f91bb55f700 1 -- 172.21.15.82:0/3268533231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91b4101810 msgr2=0x7f91b4196950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:14.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.024+0000 7f91bb55f700 1 --2- 172.21.15.82:0/3268533231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91b4101810 0x7f91b4196950 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f91b000b530 tx=0x7f91b000b840 comp rx=0 tx=0).stop 2023-09-09T23:13:14.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.024+0000 7f91bb55f700 1 -- 172.21.15.82:0/3268533231 shutdown_connections 2023-09-09T23:13:14.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.024+0000 7f91bb55f700 1 --2- 172.21.15.82:0/3268533231 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f91a00a06d0 0x7f91a00a2b90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.024+0000 7f91bb55f700 1 --2- 172.21.15.82:0/3268533231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91b4100620 0x7f91b4196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.024+0000 7f91bb55f700 1 --2- 172.21.15.82:0/3268533231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91b41021b0 0x7f91b419d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.024+0000 7f91bb55f700 1 --2- 172.21.15.82:0/3268533231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91b4101810 0x7f91b4196950 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.024+0000 7f91bb55f700 1 -- 172.21.15.82:0/3268533231 >> 172.21.15.82:0/3268533231 conn(0x7f91b40fbb70 msgr2=0x7f91b4104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:14.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.024+0000 7f91bb55f700 1 -- 172.21.15.82:0/3268533231 shutdown_connections 2023-09-09T23:13:14.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.024+0000 7f91bb55f700 1 -- 172.21.15.82:0/3268533231 wait complete. 2023-09-09T23:13:14.061 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-09-09T23:13:14.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.186+0000 7f1e9afd9700 1 -- 172.21.15.82:0/1615740717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e940ff4f0 msgr2=0x7f1e940ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:14.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.186+0000 7f1e9afd9700 1 --2- 172.21.15.82:0/1615740717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e940ff4f0 0x7f1e940ff910 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1e880099c0 tx=0x7f1e88009cd0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.186+0000 7f1e9afd9700 1 -- 172.21.15.82:0/1615740717 shutdown_connections 2023-09-09T23:13:14.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.186+0000 7f1e9afd9700 1 --2- 172.21.15.82:0/1615740717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e94101080 0x7f1e94107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.186+0000 7f1e9afd9700 1 --2- 172.21.15.82:0/1615740717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e940ff4f0 0x7f1e940ff910 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.186+0000 7f1e9afd9700 1 --2- 172.21.15.82:0/1615740717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e941006e0 0x7f1e94100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.186+0000 7f1e9afd9700 1 -- 172.21.15.82:0/1615740717 >> 172.21.15.82:0/1615740717 conn(0x7f1e940faa60 msgr2=0x7f1e940fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:14.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.186+0000 7f1e9afd9700 1 -- 172.21.15.82:0/1615740717 shutdown_connections 2023-09-09T23:13:14.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.186+0000 7f1e9afd9700 1 -- 172.21.15.82:0/1615740717 wait complete. 2023-09-09T23:13:14.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.186+0000 7f1e9afd9700 1 Processor -- start 2023-09-09T23:13:14.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.186+0000 7f1e9afd9700 1 -- start start 2023-09-09T23:13:14.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.186+0000 7f1e9afd9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e940ff4f0 0x7f1e940682d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.186+0000 7f1e9afd9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e941006e0 0x7f1e9419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.187+0000 7f1e9afd9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e94101080 0x7f1e9419f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.187+0000 7f1e9afd9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e941a00a0 con 0x7f1e94101080 2023-09-09T23:13:14.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.187+0000 7f1e9afd9700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e941a0210 con 0x7f1e941006e0 2023-09-09T23:13:14.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.187+0000 7f1e9afd9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e941a03d0 con 0x7f1e940ff4f0 2023-09-09T23:13:14.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.187+0000 7f1e99576700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e94101080 0x7f1e9419f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.187+0000 7f1e99576700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e94101080 0x7f1e9419f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55428/0 (socket says 172.21.15.82:55428) 2023-09-09T23:13:14.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.187+0000 7f1e99576700 1 -- 172.21.15.82:0/1114086745 learned_addr learned my addr 172.21.15.82:0/1114086745 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:14.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.187+0000 7f1e93fff700 1 --2- 172.21.15.82:0/1114086745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e941006e0 0x7f1e9419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.187+0000 7f1e98d75700 1 --2- 172.21.15.82:0/1114086745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e940ff4f0 0x7f1e940682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.187+0000 7f1e99576700 1 -- 172.21.15.82:0/1114086745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e940ff4f0 msgr2=0x7f1e940682d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:14.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.187+0000 7f1e99576700 1 --2- 172.21.15.82:0/1114086745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e940ff4f0 0x7f1e940682d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.187+0000 7f1e99576700 1 -- 172.21.15.82:0/1114086745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e941006e0 msgr2=0x7f1e9419d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:14.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.187+0000 7f1e99576700 1 --2- 172.21.15.82:0/1114086745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e941006e0 0x7f1e9419d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.187+0000 7f1e99576700 1 -- 172.21.15.82:0/1114086745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e880095f0 con 0x7f1e94101080 2023-09-09T23:13:14.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.187+0000 7f1e98d75700 1 --2- 172.21.15.82:0/1114086745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e940ff4f0 0x7f1e940682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:14.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.187+0000 7f1e99576700 1 --2- 172.21.15.82:0/1114086745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e94101080 0x7f1e9419f9f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c00ada0 tx=0x7f1e8c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:14.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.187+0000 7f1e91ffb700 1 -- 172.21.15.82:0/1114086745 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c00f070 con 0x7f1e94101080 2023-09-09T23:13:14.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.187+0000 7f1e9afd9700 1 -- 172.21.15.82:0/1114086745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e941a06b0 con 0x7f1e94101080 2023-09-09T23:13:14.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.187+0000 7f1e91ffb700 1 -- 172.21.15.82:0/1114086745 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e8c007b30 con 0x7f1e94101080 2023-09-09T23:13:14.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.187+0000 7f1e91ffb700 1 -- 172.21.15.82:0/1114086745 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c005440 con 0x7f1e94101080 2023-09-09T23:13:14.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.187+0000 7f1e9afd9700 1 -- 172.21.15.82:0/1114086745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e941a0bc0 con 0x7f1e94101080 2023-09-09T23:13:14.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.189+0000 7f1e91ffb700 1 -- 172.21.15.82:0/1114086745 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f1e8c004ad0 con 0x7f1e94101080 2023-09-09T23:13:14.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.190+0000 7f1e91ffb700 1 --2- 172.21.15.82:0/1114086745 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1e840a06d0 0x7f1e840a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.190+0000 7f1e91ffb700 1 -- 172.21.15.82:0/1114086745 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f1e8c0cb2d0 con 0x7f1e94101080 2023-09-09T23:13:14.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.190+0000 7f1e98d75700 1 --2- 172.21.15.82:0/1114086745 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1e840a06d0 0x7f1e840a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.190+0000 7f1e98d75700 1 --2- 172.21.15.82:0/1114086745 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1e840a06d0 0x7f1e840a2b90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1e880098e0 tx=0x7f1e88000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:14.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.191+0000 7f1ea719fb80 1 Processor -- start 2023-09-09T23:13:14.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.191+0000 7f1ea719fb80 1 -- start start 2023-09-09T23:13:14.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.191+0000 7f1ea719fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556275bc2bb0 0x556275bc2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.191+0000 7f1ea719fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556275bc3550 0x556275bc17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.191+0000 7f1ea719fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556275bc1db0 0x556275bc2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.191+0000 7f1ea719fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556275b1a6e0 con 0x556275bc1db0 2023-09-09T23:13:14.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.191+0000 7f1ea719fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556275b18000 con 0x556275bc2bb0 2023-09-09T23:13:14.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.191+0000 7f1ea719fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556275b138f0 con 0x556275bc3550 2023-09-09T23:13:14.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.191+0000 7f1e99576700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556275bc1db0 0x556275bc2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.191+0000 7f1e99576700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556275bc1db0 0x556275bc2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55436/0 (socket says 172.21.15.82:55436) 2023-09-09T23:13:14.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.191+0000 7f1e99576700 1 -- 172.21.15.82:0/2384330347 learned_addr learned my addr 172.21.15.82:0/2384330347 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:14.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.191+0000 7f1e93fff700 1 --2- 172.21.15.82:0/2384330347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556275bc3550 0x556275bc17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.191+0000 7f1e98d75700 1 --2- 172.21.15.82:0/2384330347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556275bc2bb0 0x556275bc2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.192+0000 7f1e99576700 1 -- 172.21.15.82:0/2384330347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556275bc3550 msgr2=0x556275bc17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:14.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.192+0000 7f1e99576700 1 --2- 172.21.15.82:0/2384330347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556275bc3550 0x556275bc17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.192+0000 7f1e99576700 1 -- 172.21.15.82:0/2384330347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556275bc2bb0 msgr2=0x556275bc2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:14.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.192+0000 7f1e99576700 1 --2- 172.21.15.82:0/2384330347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556275bc2bb0 0x556275bc2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.192+0000 7f1e99576700 1 -- 172.21.15.82:0/2384330347 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55627593fd90 con 0x556275bc1db0 2023-09-09T23:13:14.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.192+0000 7f1e99576700 1 --2- 172.21.15.82:0/2384330347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556275bc1db0 0x556275bc2260 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c00a0a0 tx=0x7f1e8c098700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3034b9ccea8abbb server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:14.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.192+0000 7f1e7effd700 1 -- 172.21.15.82:0/2384330347 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c098af0 con 0x556275bc1db0 2023-09-09T23:13:14.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.192+0000 7f1e7effd700 1 -- 172.21.15.82:0/2384330347 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e8c09c920 con 0x556275bc1db0 2023-09-09T23:13:14.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.192+0000 7f1e7effd700 1 -- 172.21.15.82:0/2384330347 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c097fe0 con 0x556275bc1db0 2023-09-09T23:13:14.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.193+0000 7f1ea719fb80 1 -- 172.21.15.82:0/2384330347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556275bc1db0 msgr2=0x556275bc2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:14.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.193+0000 7f1ea719fb80 1 --2- 172.21.15.82:0/2384330347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556275bc1db0 0x556275bc2260 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c00a0a0 tx=0x7f1e8c098700 comp rx=0 tx=0).stop 2023-09-09T23:13:14.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.193+0000 7f1ea719fb80 1 -- 172.21.15.82:0/2384330347 shutdown_connections 2023-09-09T23:13:14.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.193+0000 7f1ea719fb80 1 --2- 172.21.15.82:0/2384330347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556275bc2bb0 0x556275bc2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.193+0000 7f1ea719fb80 1 --2- 172.21.15.82:0/2384330347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556275bc1db0 0x556275bc2260 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.193+0000 7f1ea719fb80 1 --2- 172.21.15.82:0/2384330347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556275bc3550 0x556275bc17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.193+0000 7f1ea719fb80 1 -- 172.21.15.82:0/2384330347 >> 172.21.15.82:0/2384330347 conn(0x556275b36040 msgr2=0x556275bbc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:14.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.193+0000 7f1ea719fb80 1 -- 172.21.15.82:0/2384330347 shutdown_connections 2023-09-09T23:13:14.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.193+0000 7f1ea719fb80 1 -- 172.21.15.82:0/2384330347 wait complete. 2023-09-09T23:13:14.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.193+0000 7f1ea719fb80 1 -- start start 2023-09-09T23:13:14.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.193+0000 7f1ea719fb80 1 Processor -- start 2023-09-09T23:13:14.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.193+0000 7f1ea719fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556275bc1db0 0x556275bcb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.193+0000 7f1ea719fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556275bc2bb0 0x556275bcba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.193+0000 7f1ea719fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556275bc3550 0x556275bd2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.193+0000 7f1ea719fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556275ab1b20 con 0x556275bc2bb0 2023-09-09T23:13:14.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.193+0000 7f1ea719fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556275ab1820 con 0x556275bc3550 2023-09-09T23:13:14.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.193+0000 7f1ea719fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562759958d0 con 0x556275bc1db0 2023-09-09T23:13:14.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.194+0000 7f1ea719fb80 10 client.0 _ll_register_callbacks cb 0x5562759fba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:14.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.194+0000 7f1e93fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556275bc2bb0 0x556275bcba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.194+0000 7f1e93fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556275bc2bb0 0x556275bcba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55440/0 (socket says 172.21.15.82:55440) 2023-09-09T23:13:14.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.194+0000 7f1e93fff700 1 -- 172.21.15.82:0/405335852 learned_addr learned my addr 172.21.15.82:0/405335852 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:14.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.194+0000 7f1e99576700 1 --2- 172.21.15.82:0/405335852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556275bc3550 0x556275bd2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.194+0000 7f1e93fff700 1 -- 172.21.15.82:0/405335852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556275bc1db0 msgr2=0x556275bcb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:14.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.194+0000 7f1e98d75700 1 --2- 172.21.15.82:0/405335852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556275bc1db0 0x556275bcb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.194+0000 7f1e93fff700 1 --2- 172.21.15.82:0/405335852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556275bc1db0 0x556275bcb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.194+0000 7f1e93fff700 1 -- 172.21.15.82:0/405335852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556275bc3550 msgr2=0x556275bd2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:14.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.194+0000 7f1e93fff700 1 --2- 172.21.15.82:0/405335852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556275bc3550 0x556275bd2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.194+0000 7f1e93fff700 1 -- 172.21.15.82:0/405335852 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e941a0bc0 con 0x556275bc2bb0 2023-09-09T23:13:14.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.194+0000 7f1e99576700 1 --2- 172.21.15.82:0/405335852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556275bc3550 0x556275bd2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:14.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.194+0000 7f1e93fff700 1 --2- 172.21.15.82:0/405335852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556275bc2bb0 0x556275bcba30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1e80011a60 tx=0x7f1e80011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee8ea45ff38eef6 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:14.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.194+0000 7f1e73fff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:14.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.194+0000 7f1e73fff700 1 -- 172.21.15.82:0/405335852 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8000c800 con 0x556275bc2bb0 2023-09-09T23:13:14.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.194+0000 7f1e73fff700 1 -- 172.21.15.82:0/405335852 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f1e8000fcc0 con 0x556275bc2bb0 2023-09-09T23:13:14.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.194+0000 7f1ea719fb80 1 -- 172.21.15.82:0/405335852 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556275a6b4a0 con 0x556275bc2bb0 2023-09-09T23:13:14.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.194+0000 7f1e73fff700 1 -- 172.21.15.82:0/405335852 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e8000fe80 con 0x556275bc2bb0 2023-09-09T23:13:14.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.194+0000 7f1ea719fb80 10 client.15216 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:14.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.194+0000 7f1ea719fb80 1 -- 172.21.15.82:0/405335852 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556275928c10 con 0x556275bc2bb0 2023-09-09T23:13:14.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.194+0000 7f1e73fff700 1 -- 172.21.15.82:0/405335852 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556275a6b4a0 con 0x556275bc2bb0 2023-09-09T23:13:14.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.195+0000 7f1e73fff700 1 -- 172.21.15.82:0/405335852 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1e80025610 con 0x556275bc2bb0 2023-09-09T23:13:14.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.195+0000 7f1ea719fb80 10 client.15216 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:14.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.195+0000 7f1ea719fb80 10 client.15216 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-09T23:13:14.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.195+0000 7f1ea719fb80 20 client.15216 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:14.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.195+0000 7f1ea719fb80 1 --2- 172.21.15.82:0/405335852 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x556275bd6c90 0x556275bd5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.195+0000 7f1ea719fb80 4 client.15216 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:13:14.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.195+0000 7f1ea719fb80 1 -- 172.21.15.82:0/405335852 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556275b28850 con 0x556275bd6c90 2023-09-09T23:13:14.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.195+0000 7f1e98d75700 1 --2- 172.21.15.82:0/405335852 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x556275bd6c90 0x556275bd5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.195+0000 7f1e73fff700 10 client.15216 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:13:14.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.195+0000 7f1e98d75700 1 --2- 172.21.15.82:0/405335852 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x556275bd6c90 0x556275bd5c20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50015b0b51fc8c7d server_cookie=63de14ba02ceda73 in_seq=0 out_seq=0 2023-09-09T23:13:14.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.196+0000 7f1e73fff700 1 -- 172.21.15.82:0/405335852 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556275b28850 con 0x556275bd6c90 2023-09-09T23:13:14.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.196+0000 7f1e73fff700 10 client.15216 handle_command_reply: tid=0 2023-09-09T23:13:14.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.196+0000 7f1ea719fb80 1 client.15216 shutdown 2023-09-09T23:13:14.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.197+0000 7f1ea719fb80 1 -- 172.21.15.82:0/405335852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556275bc2bb0 msgr2=0x556275bcba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:14.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.197+0000 7f1ea719fb80 1 --2- 172.21.15.82:0/405335852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556275bc2bb0 0x556275bcba30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1e80011a60 tx=0x7f1e80011e20 comp rx=0 tx=0).stop 2023-09-09T23:13:14.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.197+0000 7f1ea719fb80 1 -- 172.21.15.82:0/405335852 shutdown_connections 2023-09-09T23:13:14.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.197+0000 7f1ea719fb80 1 --2- 172.21.15.82:0/405335852 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x556275bd6c90 0x556275bd5c20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.197+0000 7f1ea719fb80 1 --2- 172.21.15.82:0/405335852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556275bc3550 0x556275bd2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.197+0000 7f1ea719fb80 1 --2- 172.21.15.82:0/405335852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556275bc1db0 0x556275bcb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.197+0000 7f1e73fff700 0 client.15216 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:13:14.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.197+0000 7f1ea719fb80 1 --2- 172.21.15.82:0/405335852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556275bc2bb0 0x556275bcba30 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.197+0000 7f1ea719fb80 1 -- 172.21.15.82:0/405335852 >> 172.21.15.82:0/405335852 conn(0x556275b36040 msgr2=0x556275bbbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:14.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.197+0000 7f1ea719fb80 1 -- 172.21.15.82:0/405335852 shutdown_connections 2023-09-09T23:13:14.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.197+0000 7f1ea719fb80 1 -- 172.21.15.82:0/405335852 wait complete. 2023-09-09T23:13:14.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.197+0000 7f1ea719fb80 20 client.15216 trim_cache size 0 max 16384 2023-09-09T23:13:14.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.214+0000 7f1ea719fb80 1 Processor -- start 2023-09-09T23:13:14.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.214+0000 7f1ea719fb80 1 -- start start 2023-09-09T23:13:14.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.214+0000 7f1ea719fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556275bc1db0 0x556275bd5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.214+0000 7f1ea719fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556275bc3550 0x556275bc9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.214+0000 7f1ea719fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556275bd3240 0x556275bd2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.214+0000 7f1ea719fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556275bcb0c0 con 0x556275bc3550 2023-09-09T23:13:14.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.214+0000 7f1ea719fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562759a1de0 con 0x556275bd3240 2023-09-09T23:13:14.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.214+0000 7f1ea719fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556275adcf70 con 0x556275bc1db0 2023-09-09T23:13:14.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.214+0000 7f1e93fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556275bc3550 0x556275bc9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.214+0000 7f1e93fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556275bc3550 0x556275bc9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55448/0 (socket says 172.21.15.82:55448) 2023-09-09T23:13:14.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.214+0000 7f1e93fff700 1 -- 172.21.15.82:0/1364525205 learned_addr learned my addr 172.21.15.82:0/1364525205 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:14.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.214+0000 7f1e98d75700 1 --2- 172.21.15.82:0/1364525205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556275bc1db0 0x556275bd5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.214+0000 7f1e93fff700 1 -- 172.21.15.82:0/1364525205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556275bc1db0 msgr2=0x556275bd5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:14.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.214+0000 7f1e93fff700 1 --2- 172.21.15.82:0/1364525205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556275bc1db0 0x556275bd5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.214+0000 7f1e93fff700 1 -- 172.21.15.82:0/1364525205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556275bd3240 msgr2=0x556275bd2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:14.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.214+0000 7f1e93fff700 1 --2- 172.21.15.82:0/1364525205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556275bd3240 0x556275bd2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.215+0000 7f1e93fff700 1 -- 172.21.15.82:0/1364525205 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556275966200 con 0x556275bc3550 2023-09-09T23:13:14.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.215+0000 7f1e98d75700 1 --2- 172.21.15.82:0/1364525205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556275bc1db0 0x556275bd5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:14.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.215+0000 7f1e93fff700 1 --2- 172.21.15.82:0/1364525205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556275bc3550 0x556275bc9460 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1e80011a30 tx=0x7f1e80005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee2d3aadfe40e8a server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:14.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.215+0000 7f1e7f7fe700 1 -- 172.21.15.82:0/1364525205 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e80025e60 con 0x556275bc3550 2023-09-09T23:13:14.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.215+0000 7f1e7f7fe700 1 -- 172.21.15.82:0/1364525205 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e8000fac0 con 0x556275bc3550 2023-09-09T23:13:14.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.215+0000 7f1e7f7fe700 1 -- 172.21.15.82:0/1364525205 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e80005040 con 0x556275bc3550 2023-09-09T23:13:14.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.215+0000 7f1ea719fb80 1 -- 172.21.15.82:0/1364525205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556275bc3550 msgr2=0x556275bc9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:14.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.215+0000 7f1ea719fb80 1 --2- 172.21.15.82:0/1364525205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556275bc3550 0x556275bc9460 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1e80011a30 tx=0x7f1e80005b20 comp rx=0 tx=0).stop 2023-09-09T23:13:14.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.215+0000 7f1ea719fb80 1 -- 172.21.15.82:0/1364525205 shutdown_connections 2023-09-09T23:13:14.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.215+0000 7f1ea719fb80 1 --2- 172.21.15.82:0/1364525205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556275bd3240 0x556275bd2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.215+0000 7f1ea719fb80 1 --2- 172.21.15.82:0/1364525205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556275bc1db0 0x556275bd5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.215+0000 7f1ea719fb80 1 --2- 172.21.15.82:0/1364525205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556275bc3550 0x556275bc9460 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.215+0000 7f1ea719fb80 1 -- 172.21.15.82:0/1364525205 >> 172.21.15.82:0/1364525205 conn(0x556275b36040 msgr2=0x556275bc21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:14.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.215+0000 7f1ea719fb80 1 -- 172.21.15.82:0/1364525205 shutdown_connections 2023-09-09T23:13:14.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.215+0000 7f1ea719fb80 1 -- 172.21.15.82:0/1364525205 wait complete. 2023-09-09T23:13:14.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.216+0000 7f1ea719fb80 1 -- start start 2023-09-09T23:13:14.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.216+0000 7f1ea719fb80 1 Processor -- start 2023-09-09T23:13:14.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.216+0000 7f1ea719fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556275bc1db0 0x556275bcdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.216+0000 7f1ea719fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556275bc3550 0x556275bce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.216+0000 7f1ea719fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556275bd3240 0x556275bd1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.216+0000 7f1ea719fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556275bd1550 con 0x556275bc1db0 2023-09-09T23:13:14.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.216+0000 7f1ea719fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556275bd16c0 con 0x556275bd3240 2023-09-09T23:13:14.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.216+0000 7f1ea719fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556275bd1880 con 0x556275bc3550 2023-09-09T23:13:14.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.216+0000 7f1ea719fb80 10 client.0 _ll_register_callbacks cb 0x7f1e8000cd00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:14.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.216+0000 7f1e98d75700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556275bc1db0 0x556275bcdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.216+0000 7f1e98d75700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556275bc1db0 0x556275bcdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55452/0 (socket says 172.21.15.82:55452) 2023-09-09T23:13:14.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.216+0000 7f1e98d75700 1 -- 172.21.15.82:0/1933707672 learned_addr learned my addr 172.21.15.82:0/1933707672 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:14.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.216+0000 7f1e99576700 1 --2- 172.21.15.82:0/1933707672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556275bd3240 0x556275bd1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.216+0000 7f1e98d75700 1 -- 172.21.15.82:0/1933707672 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556275bc3550 msgr2=0x556275bce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:14.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.216+0000 7f1e98d75700 1 --2- 172.21.15.82:0/1933707672 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556275bc3550 0x556275bce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.216+0000 7f1e98d75700 1 -- 172.21.15.82:0/1933707672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556275bd3240 msgr2=0x556275bd1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:14.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.216+0000 7f1e98d75700 1 --2- 172.21.15.82:0/1933707672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556275bd3240 0x556275bd1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.216+0000 7f1e98d75700 1 -- 172.21.15.82:0/1933707672 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556275928c10 con 0x556275bc1db0 2023-09-09T23:13:14.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.217+0000 7f1e98d75700 1 --2- 172.21.15.82:0/1933707672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556275bc1db0 0x556275bcdca0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1e94100540 tx=0x7f1e88009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166e3645e1a1dae5 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:14.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.217+0000 7f1e73fff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:14.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.217+0000 7f1e73fff700 1 -- 172.21.15.82:0/1933707672 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e880342f0 con 0x556275bc1db0 2023-09-09T23:13:14.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.217+0000 7f1e73fff700 1 -- 172.21.15.82:0/1933707672 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f1e88034450 con 0x556275bc1db0 2023-09-09T23:13:14.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.217+0000 7f1ea719fb80 1 -- 172.21.15.82:0/1933707672 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e8c09c920 con 0x556275bc1db0 2023-09-09T23:13:14.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.217+0000 7f1e73fff700 1 -- 172.21.15.82:0/1933707672 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e88034e50 con 0x556275bc1db0 2023-09-09T23:13:14.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.217+0000 7f1ea719fb80 10 client.15222 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:14.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.217+0000 7f1ea719fb80 1 -- 172.21.15.82:0/1933707672 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55627593e260 con 0x556275bc1db0 2023-09-09T23:13:14.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.217+0000 7f1e73fff700 1 -- 172.21.15.82:0/1933707672 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e88044020 con 0x556275bc1db0 2023-09-09T23:13:14.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.217+0000 7f1e73fff700 1 -- 172.21.15.82:0/1933707672 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1e88004bf0 con 0x556275bc1db0 2023-09-09T23:13:14.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.217+0000 7f1ea719fb80 10 client.15222 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:14.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.217+0000 7f1ea719fb80 10 client.15222 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-09T23:13:14.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.217+0000 7f1ea719fb80 20 client.15222 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:14.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.217+0000 7f1ea719fb80 1 --2- 172.21.15.82:0/1933707672 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x556275c308d0 0x556275c2ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:14.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.217+0000 7f1ea719fb80 4 client.15222 mds_command: new command op to 24566 tid=0[{"prefix": "status"}] 2023-09-09T23:13:14.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.217+0000 7f1ea719fb80 1 -- 172.21.15.82:0/1933707672 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556275b0c8d0 con 0x556275c308d0 2023-09-09T23:13:14.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.218+0000 7f1e93fff700 1 --2- 172.21.15.82:0/1933707672 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x556275c308d0 0x556275c2ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:14.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.218+0000 7f1e73fff700 10 client.15222 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:13:14.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.218+0000 7f1e93fff700 1 --2- 172.21.15.82:0/1933707672 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x556275c308d0 0x556275c2ccf0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=723354e8474b4218 server_cookie=76e28c6acb591a1f in_seq=0 out_seq=0 2023-09-09T23:13:14.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.218+0000 7f1e73fff700 1 -- 172.21.15.82:0/1933707672 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x556275b0c8d0 con 0x556275c308d0 2023-09-09T23:13:14.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.218+0000 7f1e73fff700 10 client.15222 handle_command_reply: tid=0 2023-09-09T23:13:14.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.218+0000 7f1ea719fb80 1 client.15222 shutdown 2023-09-09T23:13:14.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.218+0000 7f1ea719fb80 1 -- 172.21.15.82:0/1933707672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556275bc1db0 msgr2=0x556275bcdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:14.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.218+0000 7f1ea719fb80 1 --2- 172.21.15.82:0/1933707672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556275bc1db0 0x556275bcdca0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1e94100540 tx=0x7f1e88009cd0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.218+0000 7f1ea719fb80 1 -- 172.21.15.82:0/1933707672 shutdown_connections 2023-09-09T23:13:14.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.218+0000 7f1ea719fb80 1 --2- 172.21.15.82:0/1933707672 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x556275c308d0 0x556275c2ccf0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.218+0000 7f1ea719fb80 1 --2- 172.21.15.82:0/1933707672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556275bd3240 0x556275bd1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.218+0000 7f1ea719fb80 1 --2- 172.21.15.82:0/1933707672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556275bc1db0 0x556275bcdca0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.218+0000 7f1e73fff700 0 client.15222 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:13:14.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.218+0000 7f1ea719fb80 1 --2- 172.21.15.82:0/1933707672 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556275bc3550 0x556275bce1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.218+0000 7f1ea719fb80 1 -- 172.21.15.82:0/1933707672 >> 172.21.15.82:0/1933707672 conn(0x556275b36040 msgr2=0x556275bd5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:14.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.218+0000 7f1ea719fb80 1 -- 172.21.15.82:0/1933707672 shutdown_connections 2023-09-09T23:13:14.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.218+0000 7f1ea719fb80 1 -- 172.21.15.82:0/1933707672 wait complete. 2023-09-09T23:13:14.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.219+0000 7f1ea719fb80 20 client.15222 trim_cache size 0 max 16384 2023-09-09T23:13:14.252 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:13:14.252 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da", 2023-09-09T23:13:14.252 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2023-09-09T23:13:14.252 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24566, 2023-09-09T23:13:14.253 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2023-09-09T23:13:14.253 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2023-09-09T23:13:14.253 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2023-09-09T23:13:14.253 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 55.684427839000001, 2023-09-09T23:13:14.253 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 27, 2023-09-09T23:13:14.254 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 83, 2023-09-09T23:13:14.254 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 81, 2023-09-09T23:13:14.254 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 62.543032339 2023-09-09T23:13:14.254 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:13:14.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.219+0000 7f1e9afd9700 1 -- 172.21.15.82:0/1114086745 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1e840a06d0 msgr2=0x7f1e840a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:14.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.219+0000 7f1e9afd9700 1 --2- 172.21.15.82:0/1114086745 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1e840a06d0 0x7f1e840a2b90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1e880098e0 tx=0x7f1e88000bc0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.219+0000 7f1e9afd9700 1 -- 172.21.15.82:0/1114086745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e94101080 msgr2=0x7f1e9419f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:14.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.219+0000 7f1e9afd9700 1 --2- 172.21.15.82:0/1114086745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e94101080 0x7f1e9419f9f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c00ada0 tx=0x7f1e8c00c330 comp rx=0 tx=0).stop 2023-09-09T23:13:14.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.219+0000 7f1e9afd9700 1 -- 172.21.15.82:0/1114086745 shutdown_connections 2023-09-09T23:13:14.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.219+0000 7f1e9afd9700 1 --2- 172.21.15.82:0/1114086745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e94101080 0x7f1e9419f9f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.219+0000 7f1e9afd9700 1 --2- 172.21.15.82:0/1114086745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e940ff4f0 0x7f1e940682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.219+0000 7f1e9afd9700 1 --2- 172.21.15.82:0/1114086745 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1e840a06d0 0x7f1e840a2b90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.219+0000 7f1e9afd9700 1 --2- 172.21.15.82:0/1114086745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e941006e0 0x7f1e9419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:14.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.219+0000 7f1e9afd9700 1 -- 172.21.15.82:0/1114086745 >> 172.21.15.82:0/1114086745 conn(0x7f1e940faa60 msgr2=0x7f1e940fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:14.258 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.219+0000 7f1e9afd9700 1 -- 172.21.15.82:0/1114086745 shutdown_connections 2023-09-09T23:13:14.258 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:14.219+0000 7f1e9afd9700 1 -- 172.21.15.82:0/1114086745 wait complete. 2023-09-09T23:13:14.258 INFO:tasks.cephadm:Setup complete, yielding 2023-09-09T23:13:14.259 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-09-09T23:13:14.270 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi082.front.sepia.ceph.com 2023-09-09T23:13:14.271 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- bash -c 'ceph orch status' 2023-09-09T23:13:14.820 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:13:16.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.030+0000 7fa80a817700 1 -- 172.21.15.82:0/2820713491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8040fff10 msgr2=0x7fa804100370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:16.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.030+0000 7fa80a817700 1 --2- 172.21.15.82:0/2820713491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8040fff10 0x7fa804100370 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800aad0 tx=0x7fa7f800ade0 comp rx=0 tx=0).stop 2023-09-09T23:13:16.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.030+0000 7fa80a817700 1 -- 172.21.15.82:0/2820713491 shutdown_connections 2023-09-09T23:13:16.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.030+0000 7fa80a817700 1 --2- 172.21.15.82:0/2820713491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8041008b0 0x7fa804107160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:16.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.030+0000 7fa80a817700 1 --2- 172.21.15.82:0/2820713491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa8040fed10 0x7fa8040ff130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:16.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.030+0000 7fa80a817700 1 --2- 172.21.15.82:0/2820713491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8040fff10 0x7fa804100370 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:16.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.030+0000 7fa80a817700 1 -- 172.21.15.82:0/2820713491 >> 172.21.15.82:0/2820713491 conn(0x7fa8040fa2b0 msgr2=0x7fa8040fc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:16.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.030+0000 7fa80a817700 1 -- 172.21.15.82:0/2820713491 shutdown_connections 2023-09-09T23:13:16.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.031+0000 7fa80a817700 1 -- 172.21.15.82:0/2820713491 wait complete. 2023-09-09T23:13:16.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.031+0000 7fa80a817700 1 Processor -- start 2023-09-09T23:13:16.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.031+0000 7fa80a817700 1 -- start start 2023-09-09T23:13:16.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.032+0000 7fa80a817700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa8040fed10 0x7fa8041941e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:16.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.032+0000 7fa80a817700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8040fff10 0x7fa804194720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:16.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.032+0000 7fa80a817700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8041008b0 0x7fa80419b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:16.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.032+0000 7fa80a817700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80419bd70 con 0x7fa8040fff10 2023-09-09T23:13:16.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.032+0000 7fa80a817700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80419beb0 con 0x7fa8041008b0 2023-09-09T23:13:16.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.032+0000 7fa80a817700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80419c040 con 0x7fa8040fed10 2023-09-09T23:13:16.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.032+0000 7fa8037fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8040fff10 0x7fa804194720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:16.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.032+0000 7fa8037fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8040fff10 0x7fa804194720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55468/0 (socket says 172.21.15.82:55468) 2023-09-09T23:13:16.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.032+0000 7fa8037fe700 1 -- 172.21.15.82:0/1975934053 learned_addr learned my addr 172.21.15.82:0/1975934053 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:16.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.032+0000 7fa808db4700 1 --2- 172.21.15.82:0/1975934053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8041008b0 0x7fa80419b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:16.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.032+0000 7fa803fff700 1 --2- 172.21.15.82:0/1975934053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa8040fed10 0x7fa8041941e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:16.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.032+0000 7fa8037fe700 1 -- 172.21.15.82:0/1975934053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa8040fed10 msgr2=0x7fa8041941e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:16.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.032+0000 7fa8037fe700 1 --2- 172.21.15.82:0/1975934053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa8040fed10 0x7fa8041941e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:16.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.032+0000 7fa8037fe700 1 -- 172.21.15.82:0/1975934053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8041008b0 msgr2=0x7fa80419b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:16.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.032+0000 7fa8037fe700 1 --2- 172.21.15.82:0/1975934053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8041008b0 0x7fa80419b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:16.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.032+0000 7fa8037fe700 1 -- 172.21.15.82:0/1975934053 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f800a700 con 0x7fa8040fff10 2023-09-09T23:13:16.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.032+0000 7fa803fff700 1 --2- 172.21.15.82:0/1975934053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa8040fed10 0x7fa8041941e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:16.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.032+0000 7fa8037fe700 1 --2- 172.21.15.82:0/1975934053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8040fff10 0x7fa804194720 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8003bc0 tx=0x7fa7f800ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:16.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.032+0000 7fa8017fa700 1 -- 172.21.15.82:0/1975934053 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8004200 con 0x7fa8040fff10 2023-09-09T23:13:16.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.032+0000 7fa80a817700 1 -- 172.21.15.82:0/1975934053 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa80419c2c0 con 0x7fa8040fff10 2023-09-09T23:13:16.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.032+0000 7fa8017fa700 1 -- 172.21.15.82:0/1975934053 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7f8004360 con 0x7fa8040fff10 2023-09-09T23:13:16.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.032+0000 7fa8017fa700 1 -- 172.21.15.82:0/1975934053 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f80166f0 con 0x7fa8040fff10 2023-09-09T23:13:16.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.032+0000 7fa80a817700 1 -- 172.21.15.82:0/1975934053 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa80419c730 con 0x7fa8040fff10 2023-09-09T23:13:16.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.034+0000 7fa80a817700 1 -- 172.21.15.82:0/1975934053 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa804106300 con 0x7fa8040fff10 2023-09-09T23:13:16.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.034+0000 7fa8017fa700 1 -- 172.21.15.82:0/1975934053 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fa7f8016850 con 0x7fa8040fff10 2023-09-09T23:13:16.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.035+0000 7fa8017fa700 1 --2- 172.21.15.82:0/1975934053 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa7ec0a06d0 0x7fa7ec0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:16.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.035+0000 7fa803fff700 1 --2- 172.21.15.82:0/1975934053 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa7ec0a06d0 0x7fa7ec0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:16.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.035+0000 7fa8017fa700 1 -- 172.21.15.82:0/1975934053 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fa7f8012070 con 0x7fa8040fff10 2023-09-09T23:13:16.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.035+0000 7fa803fff700 1 --2- 172.21.15.82:0/1975934053 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa7ec0a06d0 0x7fa7ec0a2b90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa8040ffd70 tx=0x7fa7f4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:16.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.039+0000 7fa8017fa700 1 -- 172.21.15.82:0/1975934053 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa7f8098720 con 0x7fa8040fff10 2023-09-09T23:13:16.204 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:16 smithi082 ceph-mon[98694]: pgmap v286: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 7 op/s; 11 B/s, 1 keys/s, 0 objects/s recovering 2023-09-09T23:13:16.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.250+0000 7fa80a817700 1 -- 172.21.15.82:0/1975934053 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa80402ce90 con 0x7fa7ec0a06d0 2023-09-09T23:13:16.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.251+0000 7fa8017fa700 1 -- 172.21.15.82:0/1975934053 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fa80402ce90 con 0x7fa7ec0a06d0 2023-09-09T23:13:16.250 INFO:teuthology.orchestra.run.smithi082.stdout:Backend: cephadm 2023-09-09T23:13:16.250 INFO:teuthology.orchestra.run.smithi082.stdout:Available: Yes 2023-09-09T23:13:16.250 INFO:teuthology.orchestra.run.smithi082.stdout:Paused: No 2023-09-09T23:13:16.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.253+0000 7fa80a817700 1 -- 172.21.15.82:0/1975934053 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa7ec0a06d0 msgr2=0x7fa7ec0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:16.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.253+0000 7fa80a817700 1 --2- 172.21.15.82:0/1975934053 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa7ec0a06d0 0x7fa7ec0a2b90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa8040ffd70 tx=0x7fa7f4008040 comp rx=0 tx=0).stop 2023-09-09T23:13:16.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.253+0000 7fa80a817700 1 -- 172.21.15.82:0/1975934053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8040fff10 msgr2=0x7fa804194720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:16.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.253+0000 7fa80a817700 1 --2- 172.21.15.82:0/1975934053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8040fff10 0x7fa804194720 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8003bc0 tx=0x7fa7f800ce20 comp rx=0 tx=0).stop 2023-09-09T23:13:16.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.254+0000 7fa80a817700 1 -- 172.21.15.82:0/1975934053 shutdown_connections 2023-09-09T23:13:16.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.254+0000 7fa80a817700 1 --2- 172.21.15.82:0/1975934053 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa7ec0a06d0 0x7fa7ec0a2b90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:16.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.254+0000 7fa80a817700 1 --2- 172.21.15.82:0/1975934053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8041008b0 0x7fa80419b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:16.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.254+0000 7fa80a817700 1 --2- 172.21.15.82:0/1975934053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa8040fed10 0x7fa8041941e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:16.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.254+0000 7fa80a817700 1 --2- 172.21.15.82:0/1975934053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8040fff10 0x7fa804194720 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:16.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.254+0000 7fa80a817700 1 -- 172.21.15.82:0/1975934053 >> 172.21.15.82:0/1975934053 conn(0x7fa8040fa2b0 msgr2=0x7fa804103440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:16.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.254+0000 7fa80a817700 1 -- 172.21.15.82:0/1975934053 shutdown_connections 2023-09-09T23:13:16.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:16.254+0000 7fa80a817700 1 -- 172.21.15.82:0/1975934053 wait complete. 2023-09-09T23:13:16.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:16 smithi170 ceph-mon[102832]: pgmap v286: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 7 op/s; 11 B/s, 1 keys/s, 0 objects/s recovering 2023-09-09T23:13:16.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:16 smithi098 ceph-mon[103124]: pgmap v286: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 7 op/s; 11 B/s, 1 keys/s, 0 objects/s recovering 2023-09-09T23:13:16.575 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- bash -c 'ceph orch ps' 2023-09-09T23:13:17.129 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:13:17.339 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:17 smithi082 ceph-mon[98694]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:13:17.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:17 smithi170 ceph-mon[102832]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:13:17.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:17 smithi098 ceph-mon[103124]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:13:18.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.317+0000 7f63cafe6700 1 -- 172.21.15.82:0/255230059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63c4100200 msgr2=0x7f63c4107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:18.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.317+0000 7f63cafe6700 1 --2- 172.21.15.82:0/255230059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63c4100200 0x7f63c4107aa0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f63bc00b0d0 tx=0x7f63bc00b3e0 comp rx=0 tx=0).stop 2023-09-09T23:13:18.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.317+0000 7f63cafe6700 1 -- 172.21.15.82:0/255230059 shutdown_connections 2023-09-09T23:13:18.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.317+0000 7f63cafe6700 1 --2- 172.21.15.82:0/255230059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63c4100200 0x7f63c4107aa0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:18.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.317+0000 7f63cafe6700 1 --2- 172.21.15.82:0/255230059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63c4100b90 0x7f63c4100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:18.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.317+0000 7f63cafe6700 1 --2- 172.21.15.82:0/255230059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63c40ff840 0x7f63c40ffcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:18.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.317+0000 7f63cafe6700 1 -- 172.21.15.82:0/255230059 >> 172.21.15.82:0/255230059 conn(0x7f63c40fa2b0 msgr2=0x7f63c40fc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:18.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.317+0000 7f63cafe6700 1 -- 172.21.15.82:0/255230059 shutdown_connections 2023-09-09T23:13:18.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.317+0000 7f63cafe6700 1 -- 172.21.15.82:0/255230059 wait complete. 2023-09-09T23:13:18.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.317+0000 7f63cafe6700 1 Processor -- start 2023-09-09T23:13:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.317+0000 7f63cafe6700 1 -- start start 2023-09-09T23:13:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.318+0000 7f63cafe6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63c40ff840 0x7f63c40ff290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.318+0000 7f63cafe6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63c4100200 0x7f63c419b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.318+0000 7f63cafe6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63c4100b90 0x7f63c419d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:18.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.318+0000 7f63cafe6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c419df90 con 0x7f63c4100200 2023-09-09T23:13:18.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.318+0000 7f63cafe6700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c419e100 con 0x7f63c4100b90 2023-09-09T23:13:18.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.318+0000 7f63cafe6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c419e2c0 con 0x7f63c40ff840 2023-09-09T23:13:18.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.318+0000 7f63c3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63c4100200 0x7f63c419b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:18.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.318+0000 7f63c3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63c4100200 0x7f63c419b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55482/0 (socket says 172.21.15.82:55482) 2023-09-09T23:13:18.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.318+0000 7f63c3fff700 1 -- 172.21.15.82:0/877245183 learned_addr learned my addr 172.21.15.82:0/877245183 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:18.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.318+0000 7f63c8d82700 1 --2- 172.21.15.82:0/877245183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63c40ff840 0x7f63c40ff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:18.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.318+0000 7f63c9583700 1 --2- 172.21.15.82:0/877245183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63c4100b90 0x7f63c419d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:18.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.318+0000 7f63c3fff700 1 -- 172.21.15.82:0/877245183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63c40ff840 msgr2=0x7f63c40ff290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:18.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.318+0000 7f63c3fff700 1 --2- 172.21.15.82:0/877245183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63c40ff840 0x7f63c40ff290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:18.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.318+0000 7f63c3fff700 1 -- 172.21.15.82:0/877245183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63c4100b90 msgr2=0x7f63c419d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:18.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.318+0000 7f63c3fff700 1 --2- 172.21.15.82:0/877245183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63c4100b90 0x7f63c419d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:18.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.318+0000 7f63c3fff700 1 -- 172.21.15.82:0/877245183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63bc009d00 con 0x7f63c4100200 2023-09-09T23:13:18.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.318+0000 7f63c3fff700 1 --2- 172.21.15.82:0/877245183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63c4100200 0x7f63c419b1b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f63b000e970 tx=0x7f63b000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:18.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.319+0000 7f63c1ffb700 1 -- 172.21.15.82:0/877245183 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b0009790 con 0x7f63c4100200 2023-09-09T23:13:18.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.319+0000 7f63c1ffb700 1 -- 172.21.15.82:0/877245183 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63b000c7e0 con 0x7f63c4100200 2023-09-09T23:13:18.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.319+0000 7f63cafe6700 1 -- 172.21.15.82:0/877245183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63c419e5a0 con 0x7f63c4100200 2023-09-09T23:13:18.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.319+0000 7f63c1ffb700 1 -- 172.21.15.82:0/877245183 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b0003c70 con 0x7f63c4100200 2023-09-09T23:13:18.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.319+0000 7f63cafe6700 1 -- 172.21.15.82:0/877245183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c419eaa0 con 0x7f63c4100200 2023-09-09T23:13:18.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.320+0000 7f63cafe6700 1 -- 172.21.15.82:0/877245183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63c4106bf0 con 0x7f63c4100200 2023-09-09T23:13:18.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.321+0000 7f63c1ffb700 1 -- 172.21.15.82:0/877245183 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f63b000cc50 con 0x7f63c4100200 2023-09-09T23:13:18.325 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.322+0000 7f63c1ffb700 1 --2- 172.21.15.82:0/877245183 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f63b40a0820 0x7f63b40a2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:18.325 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.322+0000 7f63c8d82700 1 --2- 172.21.15.82:0/877245183 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f63b40a0820 0x7f63b40a2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:18.325 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.322+0000 7f63c1ffb700 1 -- 172.21.15.82:0/877245183 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f63b0014070 con 0x7f63c4100200 2023-09-09T23:13:18.326 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.322+0000 7f63c8d82700 1 --2- 172.21.15.82:0/877245183 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f63b40a0820 0x7f63b40a2ce0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f63c4067e60 tx=0x7f63b800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:18.326 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.326+0000 7f63c1ffb700 1 -- 172.21.15.82:0/877245183 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f63b00957b0 con 0x7f63c4100200 2023-09-09T23:13:18.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:18 smithi170 ceph-mon[102832]: pgmap v287: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s; 11 B/s, 1 keys/s, 0 objects/s recovering 2023-09-09T23:13:18.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:18 smithi082 ceph-mon[98694]: pgmap v287: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s; 11 B/s, 1 keys/s, 0 objects/s recovering 2023-09-09T23:13:18.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:18 smithi098 ceph-mon[103124]: pgmap v287: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s; 11 B/s, 1 keys/s, 0 objects/s recovering 2023-09-09T23:13:18.534 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.535+0000 7f63cafe6700 1 -- 172.21.15.82:0/877245183 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f63c402d010 con 0x7f63b40a0820 2023-09-09T23:13:18.546 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.548+0000 7f63c1ffb700 1 -- 172.21.15.82:0/877245183 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f63c402d010 con 0x7f63b40a0820 2023-09-09T23:13:18.547 INFO:teuthology.orchestra.run.smithi082.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-09-09T23:13:18.547 INFO:teuthology.orchestra.run.smithi082.stdout:mds.a smithi082 running (69s) - 69s 16.8M - 18.0.0-6025-g9670cca5 20c3861ef6c9 0d3c44e14041 2023-09-09T23:13:18.547 INFO:teuthology.orchestra.run.smithi082.stdout:mds.b smithi098 running (67s) - 66s 14.3M - 18.0.0-6025-g9670cca5 20c3861ef6c9 a777e31dd425 2023-09-09T23:13:18.547 INFO:teuthology.orchestra.run.smithi082.stdout:mds.c smithi170 running (78s) - 77s 16.9M - 18.0.0-6025-g9670cca5 20c3861ef6c9 e96b4b244f2a 2023-09-09T23:13:18.547 INFO:teuthology.orchestra.run.smithi082.stdout:mds.d smithi082 running (84s) - 84s 16.8M - 18.0.0-6025-g9670cca5 20c3861ef6c9 1ded25eb76b9 2023-09-09T23:13:18.548 INFO:teuthology.orchestra.run.smithi082.stdout:mds.e smithi098 running (82s) - 82s 14.4M - 18.0.0-6025-g9670cca5 20c3861ef6c9 07915a74d9ac 2023-09-09T23:13:18.548 INFO:teuthology.orchestra.run.smithi082.stdout:mds.f smithi170 running (80s) - 79s 13.5M - 18.0.0-6025-g9670cca5 20c3861ef6c9 de1f3b85b2e9 2023-09-09T23:13:18.548 INFO:teuthology.orchestra.run.smithi082.stdout:mds.g smithi082 running (71s) - 71s 14.3M - 18.0.0-6025-g9670cca5 20c3861ef6c9 78166051d7ce 2023-09-09T23:13:18.548 INFO:teuthology.orchestra.run.smithi082.stdout:mds.h smithi098 running (76s) - 75s 16.8M - 18.0.0-6025-g9670cca5 20c3861ef6c9 7d1a5706bd5b 2023-09-09T23:13:18.548 INFO:teuthology.orchestra.run.smithi082.stdout:mds.i smithi170 running (73s) - 73s 14.0M - 18.0.0-6025-g9670cca5 20c3861ef6c9 ae36f5b1195c 2023-09-09T23:13:18.549 INFO:teuthology.orchestra.run.smithi082.stdout:mds.j smithi082 running (86s) - 86s 13.5M - 18.0.0-6025-g9670cca5 20c3861ef6c9 ff62a5ecd079 2023-09-09T23:13:18.549 INFO:teuthology.orchestra.run.smithi082.stdout:mds.k smithi098 running (89s) - 88s 13.4M - 18.0.0-6025-g9670cca5 20c3861ef6c9 2c5e8d7e40e0 2023-09-09T23:13:18.549 INFO:teuthology.orchestra.run.smithi082.stdout:mds.l smithi170 running (91s) - 90s 13.4M - 18.0.0-6025-g9670cca5 20c3861ef6c9 d67bbf0f7325 2023-09-09T23:13:18.549 INFO:teuthology.orchestra.run.smithi082.stdout:mgr.x smithi082 *:9283,8765 running (7m) - 7m 515M - 18.0.0-6025-g9670cca5 20c3861ef6c9 aac272d16713 2023-09-09T23:13:18.549 INFO:teuthology.orchestra.run.smithi082.stdout:mgr.y smithi098 *:8443,8765 running (5m) - 5m 451M - 18.0.0-6025-g9670cca5 20c3861ef6c9 ed48cd763390 2023-09-09T23:13:18.549 INFO:teuthology.orchestra.run.smithi082.stdout:mgr.z smithi170 *:8443,8765 running (5m) - 5m 451M - 18.0.0-6025-g9670cca5 20c3861ef6c9 9f7feac68312 2023-09-09T23:13:18.550 INFO:teuthology.orchestra.run.smithi082.stdout:mon.a smithi082 running (8m) - 8m 86.2M 2048M 18.0.0-6025-g9670cca5 20c3861ef6c9 bf092fc33c23 2023-09-09T23:13:18.550 INFO:teuthology.orchestra.run.smithi082.stdout:mon.b smithi098 running (6m) - 6m 59.2M 2048M 18.0.0-6025-g9670cca5 20c3861ef6c9 0ac2ec160985 2023-09-09T23:13:18.550 INFO:teuthology.orchestra.run.smithi082.stdout:mon.c smithi170 running (6m) - 6m 55.7M 2048M 18.0.0-6025-g9670cca5 20c3861ef6c9 50a05b43745b 2023-09-09T23:13:18.550 INFO:teuthology.orchestra.run.smithi082.stdout:osd.0 smithi082 running (5m) - 5m 56.6M 4292M 18.0.0-6025-g9670cca5 20c3861ef6c9 43a4a0bfe32f 2023-09-09T23:13:18.551 INFO:teuthology.orchestra.run.smithi082.stdout:osd.1 smithi098 running (5m) - 5m 60.8M 4292M 18.0.0-6025-g9670cca5 20c3861ef6c9 05e5a1a5ef24 2023-09-09T23:13:18.551 INFO:teuthology.orchestra.run.smithi082.stdout:osd.2 smithi170 running (4m) - 4m 56.1M 4292M 18.0.0-6025-g9670cca5 20c3861ef6c9 64fa5c61f1bb 2023-09-09T23:13:18.551 INFO:teuthology.orchestra.run.smithi082.stdout:osd.3 smithi082 running (4m) - 4m 50.8M 4292M 18.0.0-6025-g9670cca5 20c3861ef6c9 2375d008e4da 2023-09-09T23:13:18.551 INFO:teuthology.orchestra.run.smithi082.stdout:osd.4 smithi098 running (4m) - 4m 47.0M 4292M 18.0.0-6025-g9670cca5 20c3861ef6c9 3df0cb5448f8 2023-09-09T23:13:18.551 INFO:teuthology.orchestra.run.smithi082.stdout:osd.5 smithi170 running (3m) - 3m 53.4M 4292M 18.0.0-6025-g9670cca5 20c3861ef6c9 60e9836864c0 2023-09-09T23:13:18.552 INFO:teuthology.orchestra.run.smithi082.stdout:osd.6 smithi082 running (3m) - 3m 49.9M 4292M 18.0.0-6025-g9670cca5 20c3861ef6c9 43daa2a0359d 2023-09-09T23:13:18.552 INFO:teuthology.orchestra.run.smithi082.stdout:osd.7 smithi098 running (3m) - 3m 54.4M 4292M 18.0.0-6025-g9670cca5 20c3861ef6c9 dee8e4646109 2023-09-09T23:13:18.552 INFO:teuthology.orchestra.run.smithi082.stdout:osd.8 smithi170 running (2m) - 2m 48.6M 4292M 18.0.0-6025-g9670cca5 20c3861ef6c9 fe7e369dc67d 2023-09-09T23:13:18.552 INFO:teuthology.orchestra.run.smithi082.stdout:osd.9 smithi082 running (2m) - 2m 47.0M 4292M 18.0.0-6025-g9670cca5 20c3861ef6c9 fa69515de1e7 2023-09-09T23:13:18.553 INFO:teuthology.orchestra.run.smithi082.stdout:osd.10 smithi098 running (2m) - 2m 51.5M 4292M 18.0.0-6025-g9670cca5 20c3861ef6c9 465d0cb2729c 2023-09-09T23:13:18.553 INFO:teuthology.orchestra.run.smithi082.stdout:osd.11 smithi170 running (108s) - 107s 47.4M 4292M 18.0.0-6025-g9670cca5 20c3861ef6c9 dcd32eb430eb 2023-09-09T23:13:18.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.551+0000 7f63cafe6700 1 -- 172.21.15.82:0/877245183 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f63b40a0820 msgr2=0x7f63b40a2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:18.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.551+0000 7f63cafe6700 1 --2- 172.21.15.82:0/877245183 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f63b40a0820 0x7f63b40a2ce0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f63c4067e60 tx=0x7f63b800b410 comp rx=0 tx=0).stop 2023-09-09T23:13:18.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.551+0000 7f63cafe6700 1 -- 172.21.15.82:0/877245183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63c4100200 msgr2=0x7f63c419b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:18.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.551+0000 7f63cafe6700 1 --2- 172.21.15.82:0/877245183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63c4100200 0x7f63c419b1b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f63b000e970 tx=0x7f63b000ec80 comp rx=0 tx=0).stop 2023-09-09T23:13:18.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.551+0000 7f63cafe6700 1 -- 172.21.15.82:0/877245183 shutdown_connections 2023-09-09T23:13:18.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.551+0000 7f63cafe6700 1 --2- 172.21.15.82:0/877245183 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f63b40a0820 0x7f63b40a2ce0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:18.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.551+0000 7f63cafe6700 1 --2- 172.21.15.82:0/877245183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63c4100b90 0x7f63c419d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:18.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.551+0000 7f63cafe6700 1 --2- 172.21.15.82:0/877245183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63c40ff840 0x7f63c40ff290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:18.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.551+0000 7f63cafe6700 1 --2- 172.21.15.82:0/877245183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63c4100200 0x7f63c419b1b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:18.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.551+0000 7f63cafe6700 1 -- 172.21.15.82:0/877245183 >> 172.21.15.82:0/877245183 conn(0x7f63c40fa2b0 msgr2=0x7f63c4103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:18.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.551+0000 7f63cafe6700 1 -- 172.21.15.82:0/877245183 shutdown_connections 2023-09-09T23:13:18.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:18.551+0000 7f63cafe6700 1 -- 172.21.15.82:0/877245183 wait complete. 2023-09-09T23:13:18.872 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- bash -c 'ceph orch ls' 2023-09-09T23:13:19.204 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:19 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:13:19.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:19 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:13:19.491 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:13:19.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:19 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:13:20.563 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:20 smithi082 ceph-mon[98694]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:13:20.564 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:20 smithi082 ceph-mon[98694]: pgmap v288: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s; 9 B/s, 1 keys/s, 0 objects/s recovering 2023-09-09T23:13:20.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:20 smithi170 ceph-mon[102832]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:13:20.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:20 smithi170 ceph-mon[102832]: pgmap v288: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s; 9 B/s, 1 keys/s, 0 objects/s recovering 2023-09-09T23:13:20.715 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.716+0000 7f48219ab700 1 -- 172.21.15.82:0/123991312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f481c1023f0 msgr2=0x7f481c108ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:20.715 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.716+0000 7f48219ab700 1 --2- 172.21.15.82:0/123991312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f481c1023f0 0x7f481c108ca0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f481400b0a0 tx=0x7f481400b3b0 comp rx=0 tx=0).stop 2023-09-09T23:13:20.715 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.716+0000 7f48219ab700 1 -- 172.21.15.82:0/123991312 shutdown_connections 2023-09-09T23:13:20.715 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.716+0000 7f48219ab700 1 --2- 172.21.15.82:0/123991312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f481c0fe9d0 0x7f481c0fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:20.716 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.716+0000 7f48219ab700 1 --2- 172.21.15.82:0/123991312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f481c1023f0 0x7f481c108ca0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:20.716 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.716+0000 7f48219ab700 1 --2- 172.21.15.82:0/123991312 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f481c101a50 0x7f481c101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:20.716 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.716+0000 7f48219ab700 1 -- 172.21.15.82:0/123991312 >> 172.21.15.82:0/123991312 conn(0x7f481c0fa310 msgr2=0x7f481c0fc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:20.716 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.716+0000 7f48219ab700 1 -- 172.21.15.82:0/123991312 shutdown_connections 2023-09-09T23:13:20.716 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.716+0000 7f48219ab700 1 -- 172.21.15.82:0/123991312 wait complete. 2023-09-09T23:13:20.717 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.716+0000 7f48219ab700 1 Processor -- start 2023-09-09T23:13:20.717 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.717+0000 7f48219ab700 1 -- start start 2023-09-09T23:13:20.718 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.717+0000 7f48219ab700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f481c0fe9d0 0x7f481c1013b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:20.719 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.717+0000 7f48219ab700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f481c101a50 0x7f481c0ffa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:20.719 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.717+0000 7f48219ab700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f481c1023f0 0x7f481c0fff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:20.719 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.717+0000 7f48219ab700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c1006c0 con 0x7f481c1023f0 2023-09-09T23:13:20.719 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.717+0000 7f48219ab700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c100830 con 0x7f481c0fe9d0 2023-09-09T23:13:20.719 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.717+0000 7f48219ab700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c1a0470 con 0x7f481c101a50 2023-09-09T23:13:20.720 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.717+0000 7f481b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f481c1023f0 0x7f481c0fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:20.720 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.717+0000 7f481a7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f481c101a50 0x7f481c0ffa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:20.720 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.717+0000 7f481a7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f481c101a50 0x7f481c0ffa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:54010/0 (socket says 172.21.15.82:54010) 2023-09-09T23:13:20.720 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.717+0000 7f481b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f481c1023f0 0x7f481c0fff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55498/0 (socket says 172.21.15.82:55498) 2023-09-09T23:13:20.720 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.717+0000 7f481a7fc700 1 -- 172.21.15.82:0/3119643034 learned_addr learned my addr 172.21.15.82:0/3119643034 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:20.721 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.717+0000 7f481affd700 1 --2- 172.21.15.82:0/3119643034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f481c0fe9d0 0x7f481c1013b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:20.721 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.717+0000 7f481a7fc700 1 -- 172.21.15.82:0/3119643034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f481c0fe9d0 msgr2=0x7f481c1013b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:20.721 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.717+0000 7f481a7fc700 1 --2- 172.21.15.82:0/3119643034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f481c0fe9d0 0x7f481c1013b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:20.721 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.717+0000 7f481a7fc700 1 -- 172.21.15.82:0/3119643034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f481c1023f0 msgr2=0x7f481c0fff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:20.721 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.717+0000 7f481a7fc700 1 --2- 172.21.15.82:0/3119643034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f481c1023f0 0x7f481c0fff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:20.721 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.717+0000 7f481a7fc700 1 -- 172.21.15.82:0/3119643034 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4814009d00 con 0x7f481c101a50 2023-09-09T23:13:20.722 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.717+0000 7f481b7fe700 1 --2- 172.21.15.82:0/3119643034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f481c1023f0 0x7f481c0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:20.722 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.717+0000 7f481a7fc700 1 --2- 172.21.15.82:0/3119643034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f481c101a50 0x7f481c0ffa00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f481000e9a0 tx=0x7f481000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:20.722 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.718+0000 7f48209a9700 1 -- 172.21.15.82:0/3119643034 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4810009790 con 0x7f481c101a50 2023-09-09T23:13:20.722 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.718+0000 7f48209a9700 1 -- 172.21.15.82:0/3119643034 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f481000cbd0 con 0x7f481c101a50 2023-09-09T23:13:20.722 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.718+0000 7f48219ab700 1 -- 172.21.15.82:0/3119643034 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f481c1a0750 con 0x7f481c101a50 2023-09-09T23:13:20.723 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.718+0000 7f48209a9700 1 -- 172.21.15.82:0/3119643034 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4810018680 con 0x7f481c101a50 2023-09-09T23:13:20.723 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.718+0000 7f48219ab700 1 -- 172.21.15.82:0/3119643034 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f481c1a0c50 con 0x7f481c101a50 2023-09-09T23:13:20.723 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.719+0000 7f48219ab700 1 -- 172.21.15.82:0/3119643034 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f481c107d60 con 0x7f481c101a50 2023-09-09T23:13:20.724 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.722+0000 7f48209a9700 1 -- 172.21.15.82:0/3119643034 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f481000cd40 con 0x7f481c101a50 2023-09-09T23:13:20.724 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.722+0000 7f48209a9700 1 --2- 172.21.15.82:0/3119643034 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f48040a06d0 0x7f48040a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:20.724 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.722+0000 7f481affd700 1 --2- 172.21.15.82:0/3119643034 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f48040a06d0 0x7f48040a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:20.725 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.723+0000 7f48209a9700 1 -- 172.21.15.82:0/3119643034 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f4810014070 con 0x7f481c101a50 2023-09-09T23:13:20.725 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.723+0000 7f481affd700 1 --2- 172.21.15.82:0/3119643034 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f48040a06d0 0x7f48040a2b90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f480c005950 tx=0x7f480c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:20.725 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.725+0000 7f48209a9700 1 -- 172.21.15.82:0/3119643034 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4810095c20 con 0x7f481c101a50 2023-09-09T23:13:20.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:20 smithi098 ceph-mon[103124]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:13:20.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:20 smithi098 ceph-mon[103124]: pgmap v288: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s; 9 B/s, 1 keys/s, 0 objects/s recovering 2023-09-09T23:13:20.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.946+0000 7f48219ab700 1 -- 172.21.15.82:0/3119643034 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f481c02cfa0 con 0x7f48040a06d0 2023-09-09T23:13:20.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.952+0000 7f48209a9700 1 -- 172.21.15.82:0/3119643034 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f481c02cfa0 con 0x7f48040a06d0 2023-09-09T23:13:20.950 INFO:teuthology.orchestra.run.smithi082.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-09-09T23:13:20.951 INFO:teuthology.orchestra.run.smithi082.stdout:mds.all 12/12 - 95s smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2023-09-09T23:13:20.951 INFO:teuthology.orchestra.run.smithi082.stdout:mgr 3/3 - 5m smithi082=x;smithi098=y;smithi170=z;count:3 2023-09-09T23:13:20.951 INFO:teuthology.orchestra.run.smithi082.stdout:mon 3/3 - 6m smithi082:172.21.15.82=a;smithi098:172.21.15.98=b;smithi170:172.21.15.170=c;count:3 2023-09-09T23:13:20.951 INFO:teuthology.orchestra.run.smithi082.stdout:osd 12 - - 2023-09-09T23:13:20.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.954+0000 7f48219ab700 1 -- 172.21.15.82:0/3119643034 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f48040a06d0 msgr2=0x7f48040a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.954+0000 7f48219ab700 1 --2- 172.21.15.82:0/3119643034 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f48040a06d0 0x7f48040a2b90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f480c005950 tx=0x7f480c0058e0 comp rx=0 tx=0).stop 2023-09-09T23:13:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.954+0000 7f48219ab700 1 -- 172.21.15.82:0/3119643034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f481c101a50 msgr2=0x7f481c0ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.954+0000 7f48219ab700 1 --2- 172.21.15.82:0/3119643034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f481c101a50 0x7f481c0ffa00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f481000e9a0 tx=0x7f481000ecb0 comp rx=0 tx=0).stop 2023-09-09T23:13:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.954+0000 7f48219ab700 1 -- 172.21.15.82:0/3119643034 shutdown_connections 2023-09-09T23:13:20.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.954+0000 7f48219ab700 1 --2- 172.21.15.82:0/3119643034 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f48040a06d0 0x7f48040a2b90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:20.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.954+0000 7f48219ab700 1 --2- 172.21.15.82:0/3119643034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f481c0fe9d0 0x7f481c1013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:20.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.954+0000 7f48219ab700 1 --2- 172.21.15.82:0/3119643034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f481c1023f0 0x7f481c0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:20.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.954+0000 7f48219ab700 1 --2- 172.21.15.82:0/3119643034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f481c101a50 0x7f481c0ffa00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:20.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.954+0000 7f48219ab700 1 -- 172.21.15.82:0/3119643034 >> 172.21.15.82:0/3119643034 conn(0x7f481c0fa310 msgr2=0x7f481c104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:20.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.954+0000 7f48219ab700 1 -- 172.21.15.82:0/3119643034 shutdown_connections 2023-09-09T23:13:20.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:20.954+0000 7f48219ab700 1 -- 172.21.15.82:0/3119643034 wait complete. 2023-09-09T23:13:21.246 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- bash -c 'ceph orch host ls' 2023-09-09T23:13:21.805 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:13:22.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:21 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:13:22.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:21 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:13:22.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:21 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:13:22.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:21 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:13:22.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:21 smithi098 ceph-mon[103124]: from='client.24799 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:13:22.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:21 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:13:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:21 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:13:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:21 smithi098 ceph-mon[103124]: pgmap v289: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 8 B/s, 0 keys/s, 0 objects/s recovering 2023-09-09T23:13:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:21 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:13:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:21 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:13:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:21 smithi098 ceph-mon[103124]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:13:22.021 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:21 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:13:22.022 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:21 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:13:22.022 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:21 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:13:22.022 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:21 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:13:22.022 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:21 smithi082 ceph-mon[98694]: from='client.24799 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:13:22.023 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:21 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:13:22.023 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:21 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:13:22.023 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:21 smithi082 ceph-mon[98694]: pgmap v289: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 8 B/s, 0 keys/s, 0 objects/s recovering 2023-09-09T23:13:22.023 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:21 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:13:22.023 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:21 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:13:22.024 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:21 smithi082 ceph-mon[98694]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:13:22.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:21 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:13:22.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:21 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:13:22.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:21 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:13:22.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:21 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:13:22.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:21 smithi170 ceph-mon[102832]: from='client.24799 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:13:22.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:21 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:13:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:21 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:13:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:21 smithi170 ceph-mon[102832]: pgmap v289: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 8 B/s, 0 keys/s, 0 objects/s recovering 2023-09-09T23:13:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:21 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:13:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:21 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:13:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:21 smithi170 ceph-mon[102832]: from='mgr.14152 172.21.15.82:0/834106015' entity='mgr.x' 2023-09-09T23:13:22.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.953+0000 7f0eb4330700 1 -- 172.21.15.82:0/4087615623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eac100790 msgr2=0x7f0eac100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:22.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.953+0000 7f0eb4330700 1 --2- 172.21.15.82:0/4087615623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eac100790 0x7f0eac100bf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80099c0 tx=0x7f0ea8009cd0 comp rx=0 tx=0).stop 2023-09-09T23:13:22.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.953+0000 7f0eb4330700 1 -- 172.21.15.82:0/4087615623 shutdown_connections 2023-09-09T23:13:22.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.953+0000 7f0eb4330700 1 --2- 172.21.15.82:0/4087615623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0eac101130 0x7f0eac1079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:22.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.953+0000 7f0eb4330700 1 --2- 172.21.15.82:0/4087615623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0eac0ff590 0x7f0eac0ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:22.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.953+0000 7f0eb4330700 1 --2- 172.21.15.82:0/4087615623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eac100790 0x7f0eac100bf0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:22.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.953+0000 7f0eb4330700 1 -- 172.21.15.82:0/4087615623 >> 172.21.15.82:0/4087615623 conn(0x7f0eac0fab10 msgr2=0x7f0eac0fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:22.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.953+0000 7f0eb4330700 1 -- 172.21.15.82:0/4087615623 shutdown_connections 2023-09-09T23:13:22.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.953+0000 7f0eb4330700 1 -- 172.21.15.82:0/4087615623 wait complete. 2023-09-09T23:13:22.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.954+0000 7f0eb4330700 1 Processor -- start 2023-09-09T23:13:22.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.954+0000 7f0eb4330700 1 -- start start 2023-09-09T23:13:22.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.954+0000 7f0eb4330700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0eac0ff590 0x7f0eac196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:22.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.954+0000 7f0eb4330700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0eac100790 0x7f0eac196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:22.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.954+0000 7f0eb4330700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eac101130 0x7f0eac19d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:22.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.954+0000 7f0eb4330700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac19dda0 con 0x7f0eac101130 2023-09-09T23:13:22.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.954+0000 7f0eb4330700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac19dee0 con 0x7f0eac0ff590 2023-09-09T23:13:22.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.954+0000 7f0eb4330700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac19e070 con 0x7f0eac100790 2023-09-09T23:13:22.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.954+0000 7f0eb28cd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eac101130 0x7f0eac19d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:22.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.954+0000 7f0eb28cd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eac101130 0x7f0eac19d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55510/0 (socket says 172.21.15.82:55510) 2023-09-09T23:13:22.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.954+0000 7f0eb28cd700 1 -- 172.21.15.82:0/3879203934 learned_addr learned my addr 172.21.15.82:0/3879203934 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:22.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.954+0000 7f0eb20cc700 1 --2- 172.21.15.82:0/3879203934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0eac0ff590 0x7f0eac196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:22.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.954+0000 7f0eb18cb700 1 --2- 172.21.15.82:0/3879203934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0eac100790 0x7f0eac196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:22.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.954+0000 7f0eb28cd700 1 -- 172.21.15.82:0/3879203934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0eac100790 msgr2=0x7f0eac196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:22.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.954+0000 7f0eb28cd700 1 --2- 172.21.15.82:0/3879203934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0eac100790 0x7f0eac196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:22.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.954+0000 7f0eb28cd700 1 -- 172.21.15.82:0/3879203934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0eac0ff590 msgr2=0x7f0eac196210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:22.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.954+0000 7f0eb28cd700 1 --2- 172.21.15.82:0/3879203934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0eac0ff590 0x7f0eac196210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:22.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.954+0000 7f0eb28cd700 1 -- 172.21.15.82:0/3879203934 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea80095f0 con 0x7f0eac101130 2023-09-09T23:13:22.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.954+0000 7f0eb28cd700 1 --2- 172.21.15.82:0/3879203934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eac101130 0x7f0eac19d7d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4007da0 tx=0x7f0ea400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:22.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.955+0000 7f0ea37fe700 1 -- 172.21.15.82:0/3879203934 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea400ac50 con 0x7f0eac101130 2023-09-09T23:13:22.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.955+0000 7f0ea37fe700 1 -- 172.21.15.82:0/3879203934 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ea400adb0 con 0x7f0eac101130 2023-09-09T23:13:22.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.955+0000 7f0ea37fe700 1 -- 172.21.15.82:0/3879203934 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea40115c0 con 0x7f0eac101130 2023-09-09T23:13:22.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.955+0000 7f0eb4330700 1 -- 172.21.15.82:0/3879203934 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eac19e350 con 0x7f0eac101130 2023-09-09T23:13:22.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.955+0000 7f0eb4330700 1 -- 172.21.15.82:0/3879203934 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eac19e850 con 0x7f0eac101130 2023-09-09T23:13:22.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.956+0000 7f0eb4330700 1 -- 172.21.15.82:0/3879203934 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eac10ada0 con 0x7f0eac101130 2023-09-09T23:13:22.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.957+0000 7f0ea37fe700 1 -- 172.21.15.82:0/3879203934 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f0ea4012420 con 0x7f0eac101130 2023-09-09T23:13:22.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.958+0000 7f0ea37fe700 1 --2- 172.21.15.82:0/3879203934 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0e980a0750 0x7f0e980a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:22.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.958+0000 7f0eb20cc700 1 --2- 172.21.15.82:0/3879203934 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0e980a0750 0x7f0e980a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:22.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.958+0000 7f0ea37fe700 1 -- 172.21.15.82:0/3879203934 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f0ea40cc0e0 con 0x7f0eac101130 2023-09-09T23:13:22.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.958+0000 7f0eb20cc700 1 --2- 172.21.15.82:0/3879203934 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0e980a0750 0x7f0e980a2c10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0eac1005f0 tx=0x7f0e9c00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:22.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:22.961+0000 7f0ea37fe700 1 -- 172.21.15.82:0/3879203934 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0ea4095cc0 con 0x7f0eac101130 2023-09-09T23:13:23.157 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:23.158+0000 7f0eb4330700 1 -- 172.21.15.82:0/3879203934 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f0eac0687b0 con 0x7f0e980a0750 2023-09-09T23:13:23.158 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:23.159+0000 7f0ea37fe700 1 -- 172.21.15.82:0/3879203934 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f0eac0687b0 con 0x7f0e980a0750 2023-09-09T23:13:23.158 INFO:teuthology.orchestra.run.smithi082.stdout:HOST ADDR LABELS STATUS 2023-09-09T23:13:23.158 INFO:teuthology.orchestra.run.smithi082.stdout:smithi082 172.21.15.82 2023-09-09T23:13:23.159 INFO:teuthology.orchestra.run.smithi082.stdout:smithi098 172.21.15.98 2023-09-09T23:13:23.159 INFO:teuthology.orchestra.run.smithi082.stdout:smithi170 172.21.15.170 2023-09-09T23:13:23.159 INFO:teuthology.orchestra.run.smithi082.stdout:3 hosts in cluster 2023-09-09T23:13:23.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:23.161+0000 7f0eb4330700 1 -- 172.21.15.82:0/3879203934 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0e980a0750 msgr2=0x7f0e980a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:23.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:23.161+0000 7f0eb4330700 1 --2- 172.21.15.82:0/3879203934 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0e980a0750 0x7f0e980a2c10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0eac1005f0 tx=0x7f0e9c00a3b0 comp rx=0 tx=0).stop 2023-09-09T23:13:23.161 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:23.162+0000 7f0eb4330700 1 -- 172.21.15.82:0/3879203934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eac101130 msgr2=0x7f0eac19d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:23.161 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:23.162+0000 7f0eb4330700 1 --2- 172.21.15.82:0/3879203934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eac101130 0x7f0eac19d7d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4007da0 tx=0x7f0ea400a340 comp rx=0 tx=0).stop 2023-09-09T23:13:23.161 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:23.162+0000 7f0eb4330700 1 -- 172.21.15.82:0/3879203934 shutdown_connections 2023-09-09T23:13:23.161 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:23.162+0000 7f0eb4330700 1 --2- 172.21.15.82:0/3879203934 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0e980a0750 0x7f0e980a2c10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:23.162 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:23.162+0000 7f0eb4330700 1 --2- 172.21.15.82:0/3879203934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0eac0ff590 0x7f0eac196210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:23.162 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:23.162+0000 7f0eb4330700 1 --2- 172.21.15.82:0/3879203934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eac101130 0x7f0eac19d7d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:23.162 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:23.162+0000 7f0eb4330700 1 --2- 172.21.15.82:0/3879203934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0eac100790 0x7f0eac196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:23.162 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:23.162+0000 7f0eb4330700 1 -- 172.21.15.82:0/3879203934 >> 172.21.15.82:0/3879203934 conn(0x7f0eac0fab10 msgr2=0x7f0eac0680a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:23.162 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:23.162+0000 7f0eb4330700 1 -- 172.21.15.82:0/3879203934 shutdown_connections 2023-09-09T23:13:23.163 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:23.162+0000 7f0eb4330700 1 -- 172.21.15.82:0/3879203934 wait complete. 2023-09-09T23:13:23.547 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- bash -c 'ceph orch device ls' 2023-09-09T23:13:24.102 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:13:24.598 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:24 smithi082 ceph-mon[98694]: pgmap v290: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s; 7 B/s, 0 keys/s, 0 objects/s recovering 2023-09-09T23:13:24.599 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:24 smithi082 ceph-mon[98694]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:13:24.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:24 smithi170 ceph-mon[102832]: pgmap v290: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s; 7 B/s, 0 keys/s, 0 objects/s recovering 2023-09-09T23:13:24.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:24 smithi170 ceph-mon[102832]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:13:24.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:24 smithi098 ceph-mon[103124]: pgmap v290: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s; 7 B/s, 0 keys/s, 0 objects/s recovering 2023-09-09T23:13:24.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:24 smithi098 ceph-mon[103124]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:13:25.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.280+0000 7f2a38fc9700 1 -- 172.21.15.82:0/70040363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a340ff840 msgr2=0x7f2a340ffcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:25.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.280+0000 7f2a38fc9700 1 --2- 172.21.15.82:0/70040363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a340ff840 0x7f2a340ffcc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009990 tx=0x7f2a24009ca0 comp rx=0 tx=0).stop 2023-09-09T23:13:25.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.280+0000 7f2a38fc9700 1 -- 172.21.15.82:0/70040363 shutdown_connections 2023-09-09T23:13:25.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.280+0000 7f2a38fc9700 1 --2- 172.21.15.82:0/70040363 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a34100200 0x7f2a34107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:25.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.280+0000 7f2a38fc9700 1 --2- 172.21.15.82:0/70040363 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a34100be0 0x7f2a34101000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:25.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.280+0000 7f2a38fc9700 1 --2- 172.21.15.82:0/70040363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a340ff840 0x7f2a340ffcc0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:25.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.280+0000 7f2a38fc9700 1 -- 172.21.15.82:0/70040363 >> 172.21.15.82:0/70040363 conn(0x7f2a340fa2f0 msgr2=0x7f2a340fc730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:25.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.280+0000 7f2a38fc9700 1 -- 172.21.15.82:0/70040363 shutdown_connections 2023-09-09T23:13:25.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.280+0000 7f2a38fc9700 1 -- 172.21.15.82:0/70040363 wait complete. 2023-09-09T23:13:25.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.281+0000 7f2a38fc9700 1 Processor -- start 2023-09-09T23:13:25.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.281+0000 7f2a38fc9700 1 -- start start 2023-09-09T23:13:25.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.281+0000 7f2a38fc9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a340ff840 0x7f2a34195340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:25.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.281+0000 7f2a38fc9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a34100200 0x7f2a34195880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:25.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.281+0000 7f2a38fc9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a34100be0 0x7f2a3419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:25.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.281+0000 7f2a38fc9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3419ced0 con 0x7f2a34100be0 2023-09-09T23:13:25.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.281+0000 7f2a38fc9700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3419d010 con 0x7f2a34100200 2023-09-09T23:13:25.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.281+0000 7f2a38fc9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3419d1a0 con 0x7f2a340ff840 2023-09-09T23:13:25.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.281+0000 7f2a32d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a34100be0 0x7f2a3419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:25.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.281+0000 7f2a32d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a34100be0 0x7f2a3419c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55524/0 (socket says 172.21.15.82:55524) 2023-09-09T23:13:25.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.281+0000 7f2a32d9d700 1 -- 172.21.15.82:0/1063474414 learned_addr learned my addr 172.21.15.82:0/1063474414 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:25.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.282+0000 7f2a31d9b700 1 --2- 172.21.15.82:0/1063474414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a34100200 0x7f2a34195880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:25.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.282+0000 7f2a3259c700 1 --2- 172.21.15.82:0/1063474414 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a340ff840 0x7f2a34195340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:25.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.282+0000 7f2a32d9d700 1 -- 172.21.15.82:0/1063474414 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a340ff840 msgr2=0x7f2a34195340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:25.286 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.282+0000 7f2a32d9d700 1 --2- 172.21.15.82:0/1063474414 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a340ff840 0x7f2a34195340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:25.286 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.282+0000 7f2a32d9d700 1 -- 172.21.15.82:0/1063474414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a34100200 msgr2=0x7f2a34195880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:25.286 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.282+0000 7f2a32d9d700 1 --2- 172.21.15.82:0/1063474414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a34100200 0x7f2a34195880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:25.286 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.282+0000 7f2a32d9d700 1 -- 172.21.15.82:0/1063474414 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a240095f0 con 0x7f2a34100be0 2023-09-09T23:13:25.286 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.282+0000 7f2a3259c700 1 --2- 172.21.15.82:0/1063474414 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a340ff840 0x7f2a34195340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:25.287 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.282+0000 7f2a32d9d700 1 --2- 172.21.15.82:0/1063474414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a34100be0 0x7f2a3419c900 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00b9a0 tx=0x7f2a2c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:25.287 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.282+0000 7f2a237fe700 1 -- 172.21.15.82:0/1063474414 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c00c9b0 con 0x7f2a34100be0 2023-09-09T23:13:25.287 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.282+0000 7f2a38fc9700 1 -- 172.21.15.82:0/1063474414 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a340fe850 con 0x7f2a34100be0 2023-09-09T23:13:25.287 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.282+0000 7f2a237fe700 1 -- 172.21.15.82:0/1063474414 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a2c014440 con 0x7f2a34100be0 2023-09-09T23:13:25.287 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.282+0000 7f2a237fe700 1 -- 172.21.15.82:0/1063474414 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c0075d0 con 0x7f2a34100be0 2023-09-09T23:13:25.288 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.282+0000 7f2a38fc9700 1 -- 172.21.15.82:0/1063474414 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a340fed80 con 0x7f2a34100be0 2023-09-09T23:13:25.288 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.283+0000 7f2a38fc9700 1 -- 172.21.15.82:0/1063474414 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a3410aee0 con 0x7f2a34100be0 2023-09-09T23:13:25.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.284+0000 7f2a237fe700 1 -- 172.21.15.82:0/1063474414 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f2a2c0145b0 con 0x7f2a34100be0 2023-09-09T23:13:25.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.284+0000 7f2a237fe700 1 --2- 172.21.15.82:0/1063474414 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f2a1c0a06d0 0x7f2a1c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:25.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.285+0000 7f2a237fe700 1 -- 172.21.15.82:0/1063474414 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f2a2c006e40 con 0x7f2a34100be0 2023-09-09T23:13:25.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.285+0000 7f2a3259c700 1 --2- 172.21.15.82:0/1063474414 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f2a1c0a06d0 0x7f2a1c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:25.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.285+0000 7f2a3259c700 1 --2- 172.21.15.82:0/1063474414 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f2a1c0a06d0 0x7f2a1c0a2b90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2a340687c0 tx=0x7f2a28008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:25.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.288+0000 7f2a237fe700 1 -- 172.21.15.82:0/1063474414 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2a2c095380 con 0x7f2a34100be0 2023-09-09T23:13:25.486 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.487+0000 7f2a38fc9700 1 -- 172.21.15.82:0/1063474414 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a34106b90 con 0x7f2a1c0a06d0 2023-09-09T23:13:25.487 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.489+0000 7f2a237fe700 1 -- 172.21.15.82:0/1063474414 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f2a34106b90 con 0x7f2a1c0a06d0 2023-09-09T23:13:25.488 INFO:teuthology.orchestra.run.smithi082.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-09-09T23:13:25.488 INFO:teuthology.orchestra.run.smithi082.stdout:smithi082 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400SD400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-09-09T23:13:25.488 INFO:teuthology.orchestra.run.smithi082.stdout:smithi098 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204010H400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-09-09T23:13:25.489 INFO:teuthology.orchestra.run.smithi082.stdout:smithi170 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002R480DGN 447G No 109s ago Has a FileSystem, LVM detected 2023-09-09T23:13:25.490 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.491+0000 7f2a38fc9700 1 -- 172.21.15.82:0/1063474414 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f2a1c0a06d0 msgr2=0x7f2a1c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:25.490 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.491+0000 7f2a38fc9700 1 --2- 172.21.15.82:0/1063474414 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f2a1c0a06d0 0x7f2a1c0a2b90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2a340687c0 tx=0x7f2a28008040 comp rx=0 tx=0).stop 2023-09-09T23:13:25.490 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.491+0000 7f2a38fc9700 1 -- 172.21.15.82:0/1063474414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a34100be0 msgr2=0x7f2a3419c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:25.490 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.491+0000 7f2a38fc9700 1 --2- 172.21.15.82:0/1063474414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a34100be0 0x7f2a3419c900 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00b9a0 tx=0x7f2a2c00bd60 comp rx=0 tx=0).stop 2023-09-09T23:13:25.491 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.491+0000 7f2a38fc9700 1 -- 172.21.15.82:0/1063474414 shutdown_connections 2023-09-09T23:13:25.491 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.491+0000 7f2a38fc9700 1 --2- 172.21.15.82:0/1063474414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a34100be0 0x7f2a3419c900 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:25.491 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.491+0000 7f2a38fc9700 1 --2- 172.21.15.82:0/1063474414 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a340ff840 0x7f2a34195340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:25.491 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.491+0000 7f2a38fc9700 1 --2- 172.21.15.82:0/1063474414 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f2a1c0a06d0 0x7f2a1c0a2b90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:25.491 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.491+0000 7f2a38fc9700 1 --2- 172.21.15.82:0/1063474414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a34100200 0x7f2a34195880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:25.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.491+0000 7f2a38fc9700 1 -- 172.21.15.82:0/1063474414 >> 172.21.15.82:0/1063474414 conn(0x7f2a340fa2f0 msgr2=0x7f2a34103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:25.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.491+0000 7f2a38fc9700 1 -- 172.21.15.82:0/1063474414 shutdown_connections 2023-09-09T23:13:25.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:25.491+0000 7f2a38fc9700 1 -- 172.21.15.82:0/1063474414 wait complete. 2023-09-09T23:13:25.957 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-09-09T23:13:25.966 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi082.front.sepia.ceph.com 2023-09-09T23:13:25.967 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- bash -c 'ceph fs dump' 2023-09-09T23:13:26.519 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:13:26.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:26 smithi170 ceph-mon[102832]: pgmap v291: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s; 7 B/s, 0 keys/s, 0 objects/s recovering 2023-09-09T23:13:26.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:26 smithi082 ceph-mon[98694]: pgmap v291: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s; 7 B/s, 0 keys/s, 0 objects/s recovering 2023-09-09T23:13:26.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:26 smithi098 ceph-mon[103124]: pgmap v291: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s; 7 B/s, 0 keys/s, 0 objects/s recovering 2023-09-09T23:13:27.637 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:27 smithi082 ceph-mon[98694]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:13:27.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:27 smithi170 ceph-mon[102832]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:13:27.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:27 smithi098 ceph-mon[103124]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:13:27.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.818+0000 7f2163826700 1 -- 172.21.15.82:0/2723476827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f215c101190 msgr2=0x7f215c107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:27.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.818+0000 7f2163826700 1 --2- 172.21.15.82:0/2723476827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f215c101190 0x7f215c107a40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f215400b0d0 tx=0x7f215400b3e0 comp rx=0 tx=0).stop 2023-09-09T23:13:27.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.818+0000 7f2163826700 1 -- 172.21.15.82:0/2723476827 shutdown_connections 2023-09-09T23:13:27.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.818+0000 7f2163826700 1 --2- 172.21.15.82:0/2723476827 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f215c0ff5f0 0x7f215c0ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:27.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.818+0000 7f2163826700 1 --2- 172.21.15.82:0/2723476827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f215c101190 0x7f215c107a40 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:27.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.818+0000 7f2163826700 1 --2- 172.21.15.82:0/2723476827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f215c1007f0 0x7f215c100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:27.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.818+0000 7f2163826700 1 -- 172.21.15.82:0/2723476827 >> 172.21.15.82:0/2723476827 conn(0x7f215c0fab90 msgr2=0x7f215c0fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:27.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.818+0000 7f2163826700 1 -- 172.21.15.82:0/2723476827 shutdown_connections 2023-09-09T23:13:27.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.818+0000 7f2163826700 1 -- 172.21.15.82:0/2723476827 wait complete. 2023-09-09T23:13:27.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.818+0000 7f2163826700 1 Processor -- start 2023-09-09T23:13:27.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.818+0000 7f2163826700 1 -- start start 2023-09-09T23:13:27.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.819+0000 7f2163826700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f215c0ff5f0 0x7f215c196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:27.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.819+0000 7f2163826700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f215c1007f0 0x7f215c196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:27.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.819+0000 7f2163826700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f215c101190 0x7f215c19da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:27.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.819+0000 7f2163826700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215c19e060 con 0x7f215c1007f0 2023-09-09T23:13:27.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.819+0000 7f2163826700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215c19e1a0 con 0x7f215c0ff5f0 2023-09-09T23:13:27.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.819+0000 7f2163826700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215c19e330 con 0x7f215c101190 2023-09-09T23:13:27.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.819+0000 7f2160dc1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f215c1007f0 0x7f215c196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:27.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.819+0000 7f2160dc1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f215c1007f0 0x7f215c196a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55534/0 (socket says 172.21.15.82:55534) 2023-09-09T23:13:27.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.819+0000 7f2160dc1700 1 -- 172.21.15.82:0/3323187274 learned_addr learned my addr 172.21.15.82:0/3323187274 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:27.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.819+0000 7f2161dc3700 1 --2- 172.21.15.82:0/3323187274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f215c101190 0x7f215c19da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:27.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.819+0000 7f21615c2700 1 --2- 172.21.15.82:0/3323187274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f215c0ff5f0 0x7f215c196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:27.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.819+0000 7f2160dc1700 1 -- 172.21.15.82:0/3323187274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f215c101190 msgr2=0x7f215c19da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:27.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.819+0000 7f2160dc1700 1 --2- 172.21.15.82:0/3323187274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f215c101190 0x7f215c19da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:27.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.819+0000 7f2160dc1700 1 -- 172.21.15.82:0/3323187274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f215c0ff5f0 msgr2=0x7f215c196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:27.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.819+0000 7f2160dc1700 1 --2- 172.21.15.82:0/3323187274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f215c0ff5f0 0x7f215c196520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:27.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.819+0000 7f2160dc1700 1 -- 172.21.15.82:0/3323187274 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2154009d00 con 0x7f215c1007f0 2023-09-09T23:13:27.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.819+0000 7f2161dc3700 1 --2- 172.21.15.82:0/3323187274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f215c101190 0x7f215c19da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:27.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.819+0000 7f2160dc1700 1 --2- 172.21.15.82:0/3323187274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f215c1007f0 0x7f215c196a60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f215800d980 tx=0x7f215800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:27.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.819+0000 7f21527fc700 1 -- 172.21.15.82:0/3323187274 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2158009b40 con 0x7f215c1007f0 2023-09-09T23:13:27.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.819+0000 7f2163826700 1 -- 172.21.15.82:0/3323187274 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f215c19e610 con 0x7f215c1007f0 2023-09-09T23:13:27.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.819+0000 7f21527fc700 1 -- 172.21.15.82:0/3323187274 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f215800b6e0 con 0x7f215c1007f0 2023-09-09T23:13:27.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.819+0000 7f21527fc700 1 -- 172.21.15.82:0/3323187274 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215800f620 con 0x7f215c1007f0 2023-09-09T23:13:27.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.819+0000 7f2163826700 1 -- 172.21.15.82:0/3323187274 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f215c19eb90 con 0x7f215c1007f0 2023-09-09T23:13:27.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.820+0000 7f2163826700 1 -- 172.21.15.82:0/3323187274 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f215c10aea0 con 0x7f215c1007f0 2023-09-09T23:13:27.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.821+0000 7f21527fc700 1 -- 172.21.15.82:0/3323187274 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f2158010460 con 0x7f215c1007f0 2023-09-09T23:13:27.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.822+0000 7f21527fc700 1 --2- 172.21.15.82:0/3323187274 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f21480a06d0 0x7f21480a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:27.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.822+0000 7f21527fc700 1 -- 172.21.15.82:0/3323187274 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f21580cbab0 con 0x7f215c1007f0 2023-09-09T23:13:27.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.822+0000 7f21615c2700 1 --2- 172.21.15.82:0/3323187274 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f21480a06d0 0x7f21480a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:27.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.822+0000 7f21615c2700 1 --2- 172.21.15.82:0/3323187274 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f21480a06d0 0x7f21480a2b90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f214c006fd0 tx=0x7f214c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:27.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:27.825+0000 7f21527fc700 1 -- 172.21.15.82:0/3323187274 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2158095580 con 0x7f215c1007f0 2023-09-09T23:13:28.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:28.058+0000 7f2163826700 1 -- 172.21.15.82:0/3323187274 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f215c10b0b0 con 0x7f215c1007f0 2023-09-09T23:13:28.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:28.058+0000 7f21527fc700 1 -- 172.21.15.82:0/3323187274 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+2971 (secure 0 0 0) 0x7f2158016bb0 con 0x7f215c1007f0 2023-09-09T23:13:28.057 INFO:teuthology.orchestra.run.smithi082.stdout:e27 2023-09-09T23:13:28.057 INFO:teuthology.orchestra.run.smithi082.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-09-09T23:13:28.058 INFO:teuthology.orchestra.run.smithi082.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-09T23:13:28.058 INFO:teuthology.orchestra.run.smithi082.stdout:legacy client fscid: 1 2023-09-09T23:13:28.058 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:28.058 INFO:teuthology.orchestra.run.smithi082.stdout:Filesystem 'cephfs' (1) 2023-09-09T23:13:28.059 INFO:teuthology.orchestra.run.smithi082.stdout:fs_name cephfs 2023-09-09T23:13:28.059 INFO:teuthology.orchestra.run.smithi082.stdout:epoch 27 2023-09-09T23:13:28.059 INFO:teuthology.orchestra.run.smithi082.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-09-09T23:13:28.059 INFO:teuthology.orchestra.run.smithi082.stdout:created 2023-09-09T23:12:17.586036+0000 2023-09-09T23:13:28.059 INFO:teuthology.orchestra.run.smithi082.stdout:modified 2023-09-09T23:12:25.285958+0000 2023-09-09T23:13:28.060 INFO:teuthology.orchestra.run.smithi082.stdout:tableserver 0 2023-09-09T23:13:28.060 INFO:teuthology.orchestra.run.smithi082.stdout:root 0 2023-09-09T23:13:28.060 INFO:teuthology.orchestra.run.smithi082.stdout:session_timeout 300 2023-09-09T23:13:28.060 INFO:teuthology.orchestra.run.smithi082.stdout:session_autoclose 300 2023-09-09T23:13:28.060 INFO:teuthology.orchestra.run.smithi082.stdout:max_file_size 1099511627776 2023-09-09T23:13:28.060 INFO:teuthology.orchestra.run.smithi082.stdout:max_xattr_size 65536 2023-09-09T23:13:28.061 INFO:teuthology.orchestra.run.smithi082.stdout:required_client_features {} 2023-09-09T23:13:28.061 INFO:teuthology.orchestra.run.smithi082.stdout:last_failure 0 2023-09-09T23:13:28.061 INFO:teuthology.orchestra.run.smithi082.stdout:last_failure_osd_epoch 0 2023-09-09T23:13:28.061 INFO:teuthology.orchestra.run.smithi082.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-09T23:13:28.061 INFO:teuthology.orchestra.run.smithi082.stdout:max_mds 5 2023-09-09T23:13:28.062 INFO:teuthology.orchestra.run.smithi082.stdout:in 0,1,2,3,4 2023-09-09T23:13:28.062 INFO:teuthology.orchestra.run.smithi082.stdout:up {0=24566,1=24542,2=24523,3=24491,4=24460} 2023-09-09T23:13:28.062 INFO:teuthology.orchestra.run.smithi082.stdout:failed 2023-09-09T23:13:28.062 INFO:teuthology.orchestra.run.smithi082.stdout:damaged 2023-09-09T23:13:28.062 INFO:teuthology.orchestra.run.smithi082.stdout:stopped 2023-09-09T23:13:28.063 INFO:teuthology.orchestra.run.smithi082.stdout:data_pools [3] 2023-09-09T23:13:28.063 INFO:teuthology.orchestra.run.smithi082.stdout:metadata_pool 2 2023-09-09T23:13:28.063 INFO:teuthology.orchestra.run.smithi082.stdout:inline_data disabled 2023-09-09T23:13:28.063 INFO:teuthology.orchestra.run.smithi082.stdout:balancer 2023-09-09T23:13:28.063 INFO:teuthology.orchestra.run.smithi082.stdout:bal_rank_mask -1 2023-09-09T23:13:28.064 INFO:teuthology.orchestra.run.smithi082.stdout:standby_count_wanted 0 2023-09-09T23:13:28.064 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.b{0:24566} state up:active seq 3 addr [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] compat {c=[1],r=[1],i=[fff]}] 2023-09-09T23:13:28.064 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.e{0:24518} state up:standby-replay seq 1 addr [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] compat {c=[1],r=[1],i=[fff]}] 2023-09-09T23:13:28.064 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.h{1:24542} state up:active seq 6 addr [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] compat {c=[1],r=[1],i=[fff]}] 2023-09-09T23:13:28.064 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.c{1:24502} state up:standby-replay seq 1 addr [v2:172.21.15.170:6836/4006360308,v1:172.21.15.170:6837/4006360308] compat {c=[1],r=[1],i=[fff]}] 2023-09-09T23:13:28.064 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.i{2:24523} state up:active seq 6 addr [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] compat {c=[1],r=[1],i=[fff]}] 2023-09-09T23:13:28.065 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.f{2:24493} state up:standby-replay seq 1 addr [v2:172.21.15.170:6834/2313861511,v1:172.21.15.170:6835/2313861511] compat {c=[1],r=[1],i=[fff]}] 2023-09-09T23:13:28.065 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.k{3:24491} state up:active seq 10 addr [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] compat {c=[1],r=[1],i=[fff]}] 2023-09-09T23:13:28.065 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.a{3:14724} state up:standby-replay seq 1 addr [v2:172.21.15.82:6840/4143604488,v1:172.21.15.82:6841/4143604488] compat {c=[1],r=[1],i=[fff]}] 2023-09-09T23:13:28.065 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.l{4:24460} state up:active seq 11 addr [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] compat {c=[1],r=[1],i=[fff]}] 2023-09-09T23:13:28.065 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.g{4:14712} state up:standby-replay seq 1 addr [v2:172.21.15.82:6838/2127301345,v1:172.21.15.82:6839/2127301345] compat {c=[1],r=[1],i=[fff]}] 2023-09-09T23:13:28.066 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:28.066 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:28.066 INFO:teuthology.orchestra.run.smithi082.stdout:Standby daemons: 2023-09-09T23:13:28.066 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:28.066 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.j{-1:14640} state up:standby seq 1 addr [v2:172.21.15.82:6834/2471432684,v1:172.21.15.82:6835/2471432684] compat {c=[1],r=[1],i=[fff]}] 2023-09-09T23:13:28.067 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.d{-1:14652} state up:standby seq 1 addr [v2:172.21.15.82:6836/3579387137,v1:172.21.15.82:6837/3579387137] compat {c=[1],r=[1],i=[fff]}] 2023-09-09T23:13:28.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:28.060+0000 7f2163826700 1 -- 172.21.15.82:0/3323187274 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f21480a06d0 msgr2=0x7f21480a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:28.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:28.060+0000 7f2163826700 1 --2- 172.21.15.82:0/3323187274 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f21480a06d0 0x7f21480a2b90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f214c006fd0 tx=0x7f214c008040 comp rx=0 tx=0).stop 2023-09-09T23:13:28.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:28.060+0000 7f2163826700 1 -- 172.21.15.82:0/3323187274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f215c1007f0 msgr2=0x7f215c196a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:28.070 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:28.060+0000 7f2163826700 1 --2- 172.21.15.82:0/3323187274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f215c1007f0 0x7f215c196a60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f215800d980 tx=0x7f215800dd40 comp rx=0 tx=0).stop 2023-09-09T23:13:28.070 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:28.060+0000 7f2163826700 1 -- 172.21.15.82:0/3323187274 shutdown_connections 2023-09-09T23:13:28.070 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:28.060+0000 7f2163826700 1 --2- 172.21.15.82:0/3323187274 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f21480a06d0 0x7f21480a2b90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:28.070 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:28.060+0000 7f2163826700 1 --2- 172.21.15.82:0/3323187274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f215c0ff5f0 0x7f215c196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:28.070 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:28.060+0000 7f2163826700 1 --2- 172.21.15.82:0/3323187274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f215c101190 0x7f215c19da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:28.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:28.060+0000 7f2163826700 1 --2- 172.21.15.82:0/3323187274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f215c1007f0 0x7f215c196a60 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:28.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:28.060+0000 7f2163826700 1 -- 172.21.15.82:0/3323187274 >> 172.21.15.82:0/3323187274 conn(0x7f215c0fab90 msgr2=0x7f215c068a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:28.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:28.061+0000 7f2163826700 1 -- 172.21.15.82:0/3323187274 shutdown_connections 2023-09-09T23:13:28.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:28.061+0000 7f2163826700 1 -- 172.21.15.82:0/3323187274 wait complete. 2023-09-09T23:13:28.071 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 27 2023-09-09T23:13:28.369 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:28 smithi082 ceph-mon[98694]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-09-09T23:13:28.370 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:28 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3323187274' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-09-09T23:13:28.376 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- bash -c 'ceph osd dump' 2023-09-09T23:13:28.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:28 smithi170 ceph-mon[102832]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-09-09T23:13:28.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:28 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3323187274' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-09-09T23:13:28.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:28 smithi098 ceph-mon[103124]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-09-09T23:13:28.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:28 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3323187274' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-09-09T23:13:28.930 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:13:30.146 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.146+0000 7f87382f7700 1 -- 172.21.15.82:0/3447393914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8730101160 msgr2=0x7f8730107a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:30.147 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.146+0000 7f87382f7700 1 --2- 172.21.15.82:0/3447393914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8730101160 0x7f8730107a10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f872800b0d0 tx=0x7f872800b3e0 comp rx=0 tx=0).stop 2023-09-09T23:13:30.147 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.146+0000 7f87382f7700 1 -- 172.21.15.82:0/3447393914 shutdown_connections 2023-09-09T23:13:30.147 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.146+0000 7f87382f7700 1 --2- 172.21.15.82:0/3447393914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87300ff5c0 0x7f87300ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:30.147 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.146+0000 7f87382f7700 1 --2- 172.21.15.82:0/3447393914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8730101160 0x7f8730107a10 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:30.147 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.146+0000 7f87382f7700 1 --2- 172.21.15.82:0/3447393914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87301007c0 0x7f8730100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.146+0000 7f87382f7700 1 -- 172.21.15.82:0/3447393914 >> 172.21.15.82:0/3447393914 conn(0x7f87300fab80 msgr2=0x7f87300fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.146+0000 7f87382f7700 1 -- 172.21.15.82:0/3447393914 shutdown_connections 2023-09-09T23:13:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.146+0000 7f87382f7700 1 -- 172.21.15.82:0/3447393914 wait complete. 2023-09-09T23:13:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.147+0000 7f87382f7700 1 Processor -- start 2023-09-09T23:13:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.147+0000 7f87382f7700 1 -- start start 2023-09-09T23:13:30.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.147+0000 7f87382f7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87300ff5c0 0x7f8730196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:30.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.147+0000 7f87382f7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87301007c0 0x7f87301969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:30.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.147+0000 7f87382f7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8730101160 0x7f873019da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:30.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.147+0000 7f87382f7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873019e020 con 0x7f8730101160 2023-09-09T23:13:30.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.147+0000 7f87382f7700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873019e160 con 0x7f87301007c0 2023-09-09T23:13:30.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.147+0000 7f87382f7700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873019e2f0 con 0x7f87300ff5c0 2023-09-09T23:13:30.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.147+0000 7f8736894700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8730101160 0x7f873019da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:30.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.147+0000 7f8736894700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8730101160 0x7f873019da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55550/0 (socket says 172.21.15.82:55550) 2023-09-09T23:13:30.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.147+0000 7f8736894700 1 -- 172.21.15.82:0/1261721243 learned_addr learned my addr 172.21.15.82:0/1261721243 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:30.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.147+0000 7f8735892700 1 --2- 172.21.15.82:0/1261721243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87301007c0 0x7f87301969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:30.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.147+0000 7f8736894700 1 -- 172.21.15.82:0/1261721243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87300ff5c0 msgr2=0x7f8730196490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:30.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.147+0000 7f8736093700 1 --2- 172.21.15.82:0/1261721243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87300ff5c0 0x7f8730196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:30.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.147+0000 7f8736894700 1 --2- 172.21.15.82:0/1261721243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87300ff5c0 0x7f8730196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:30.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.147+0000 7f8736894700 1 -- 172.21.15.82:0/1261721243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87301007c0 msgr2=0x7f87301969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:30.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.147+0000 7f8736894700 1 --2- 172.21.15.82:0/1261721243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87301007c0 0x7f87301969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.147+0000 7f8736894700 1 -- 172.21.15.82:0/1261721243 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8728009d00 con 0x7f8730101160 2023-09-09T23:13:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.147+0000 7f8735892700 1 --2- 172.21.15.82:0/1261721243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87301007c0 0x7f87301969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.147+0000 7f8736894700 1 --2- 172.21.15.82:0/1261721243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8730101160 0x7f873019da50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f87280094c0 tx=0x7f87280095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.148+0000 7f87277fe700 1 -- 172.21.15.82:0/1261721243 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872800e040 con 0x7f8730101160 2023-09-09T23:13:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.148+0000 7f87277fe700 1 -- 172.21.15.82:0/1261721243 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8728003e10 con 0x7f8730101160 2023-09-09T23:13:30.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.148+0000 7f87382f7700 1 -- 172.21.15.82:0/1261721243 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f873019e570 con 0x7f8730101160 2023-09-09T23:13:30.153 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.148+0000 7f87277fe700 1 -- 172.21.15.82:0/1261721243 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87280055b0 con 0x7f8730101160 2023-09-09T23:13:30.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.148+0000 7f87382f7700 1 -- 172.21.15.82:0/1261721243 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f873019ea00 con 0x7f8730101160 2023-09-09T23:13:30.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.149+0000 7f87382f7700 1 -- 172.21.15.82:0/1261721243 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f873010ae40 con 0x7f8730101160 2023-09-09T23:13:30.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.150+0000 7f87277fe700 1 -- 172.21.15.82:0/1261721243 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f8728005710 con 0x7f8730101160 2023-09-09T23:13:30.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.150+0000 7f87277fe700 1 --2- 172.21.15.82:0/1261721243 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f871c0a06d0 0x7f871c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:30.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.150+0000 7f8736093700 1 --2- 172.21.15.82:0/1261721243 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f871c0a06d0 0x7f871c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:30.155 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.150+0000 7f87277fe700 1 -- 172.21.15.82:0/1261721243 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f87280cd3b0 con 0x7f8730101160 2023-09-09T23:13:30.155 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.151+0000 7f8736093700 1 --2- 172.21.15.82:0/1261721243 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f871c0a06d0 0x7f871c0a2b90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8720005950 tx=0x7f87200058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:30.155 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.155+0000 7f87277fe700 1 -- 172.21.15.82:0/1261721243 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8728096f10 con 0x7f8730101160 2023-09-09T23:13:30.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.349+0000 7f87382f7700 1 -- 172.21.15.82:0/1261721243 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f873019ec20 con 0x7f8730101160 2023-09-09T23:13:30.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.350+0000 7f87277fe700 1 -- 172.21.15.82:0/1261721243 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v83) v1 ==== 56+0+5215 (secure 0 0 0) 0x7f872809a530 con 0x7f8730101160 2023-09-09T23:13:30.349 INFO:teuthology.orchestra.run.smithi082.stdout:epoch 83 2023-09-09T23:13:30.349 INFO:teuthology.orchestra.run.smithi082.stdout:fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da 2023-09-09T23:13:30.350 INFO:teuthology.orchestra.run.smithi082.stdout:created 2023-09-09T23:05:14.222190+0000 2023-09-09T23:13:30.350 INFO:teuthology.orchestra.run.smithi082.stdout:modified 2023-09-09T23:13:10.324596+0000 2023-09-09T23:13:30.350 INFO:teuthology.orchestra.run.smithi082.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-09-09T23:13:30.350 INFO:teuthology.orchestra.run.smithi082.stdout:crush_version 26 2023-09-09T23:13:30.350 INFO:teuthology.orchestra.run.smithi082.stdout:full_ratio 0.95 2023-09-09T23:13:30.351 INFO:teuthology.orchestra.run.smithi082.stdout:backfillfull_ratio 0.9 2023-09-09T23:13:30.351 INFO:teuthology.orchestra.run.smithi082.stdout:nearfull_ratio 0.85 2023-09-09T23:13:30.351 INFO:teuthology.orchestra.run.smithi082.stdout:require_min_compat_client luminous 2023-09-09T23:13:30.351 INFO:teuthology.orchestra.run.smithi082.stdout:min_compat_client luminous 2023-09-09T23:13:30.352 INFO:teuthology.orchestra.run.smithi082.stdout:require_osd_release reef 2023-09-09T23:13:30.352 INFO:teuthology.orchestra.run.smithi082.stdout:stretch_mode_enabled false 2023-09-09T23:13:30.352 INFO:teuthology.orchestra.run.smithi082.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-09-09T23:13:30.352 INFO:teuthology.orchestra.run.smithi082.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-09-09T23:13:30.352 INFO:teuthology.orchestra.run.smithi082.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2023-09-09T23:13:30.353 INFO:teuthology.orchestra.run.smithi082.stdout:max_osd 12 2023-09-09T23:13:30.353 INFO:teuthology.orchestra.run.smithi082.stdout:osd.0 up in weight 1 up_from 8 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6802/3197434929,v1:172.21.15.82:6803/3197434929] [v2:172.21.15.82:6804/3197434929,v1:172.21.15.82:6805/3197434929] exists,up 1277eb17-a2e7-4e10-9832-f12a5e93ffde 2023-09-09T23:13:30.353 INFO:teuthology.orchestra.run.smithi082.stdout:osd.1 up in weight 1 up_from 13 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6800/3528641314,v1:172.21.15.98:6801/3528641314] [v2:172.21.15.98:6802/3528641314,v1:172.21.15.98:6803/3528641314] exists,up f68723fd-0908-4e21-ae8e-1de1c3a9a65c 2023-09-09T23:13:30.353 INFO:teuthology.orchestra.run.smithi082.stdout:osd.2 up in weight 1 up_from 18 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6800/4006414551,v1:172.21.15.170:6801/4006414551] [v2:172.21.15.170:6802/4006414551,v1:172.21.15.170:6803/4006414551] exists,up 06f89c53-1419-4f52-acad-6e92916b476f 2023-09-09T23:13:30.354 INFO:teuthology.orchestra.run.smithi082.stdout:osd.3 up in weight 1 up_from 25 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6810/3647629241,v1:172.21.15.82:6811/3647629241] [v2:172.21.15.82:6812/3647629241,v1:172.21.15.82:6813/3647629241] exists,up fedbd4c9-6852-42b5-8f3a-ba8d9bfe5d59 2023-09-09T23:13:30.354 INFO:teuthology.orchestra.run.smithi082.stdout:osd.4 up in weight 1 up_from 32 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6808/2507035643,v1:172.21.15.98:6809/2507035643] [v2:172.21.15.98:6810/2507035643,v1:172.21.15.98:6811/2507035643] exists,up a71f3084-6919-4161-87bf-345f3355d8c2 2023-09-09T23:13:30.354 INFO:teuthology.orchestra.run.smithi082.stdout:osd.5 up in weight 1 up_from 37 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6808/775005168,v1:172.21.15.170:6809/775005168] [v2:172.21.15.170:6810/775005168,v1:172.21.15.170:6811/775005168] exists,up f76058e9-a90e-476e-87f0-3c6723795e26 2023-09-09T23:13:30.354 INFO:teuthology.orchestra.run.smithi082.stdout:osd.6 up in weight 1 up_from 45 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6818/3008405982,v1:172.21.15.82:6819/3008405982] [v2:172.21.15.82:6820/3008405982,v1:172.21.15.82:6821/3008405982] exists,up 25940be0-c1bd-4c72-95bf-fd8367aa2f08 2023-09-09T23:13:30.355 INFO:teuthology.orchestra.run.smithi082.stdout:osd.7 up in weight 1 up_from 50 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6816/3547615284,v1:172.21.15.98:6817/3547615284] [v2:172.21.15.98:6818/3547615284,v1:172.21.15.98:6819/3547615284] exists,up f651bb8c-0b25-42d5-b028-58563f4c27f8 2023-09-09T23:13:30.355 INFO:teuthology.orchestra.run.smithi082.stdout:osd.8 up in weight 1 up_from 57 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6816/632038848,v1:172.21.15.170:6817/632038848] [v2:172.21.15.170:6818/632038848,v1:172.21.15.170:6819/632038848] exists,up 23b27c86-8c98-441e-b7ab-d57bc584e58f 2023-09-09T23:13:30.355 INFO:teuthology.orchestra.run.smithi082.stdout:osd.9 up in weight 1 up_from 64 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6826/3354242311,v1:172.21.15.82:6827/3354242311] [v2:172.21.15.82:6828/3354242311,v1:172.21.15.82:6829/3354242311] exists,up 383b8989-4922-4eee-9c77-f198258ac243 2023-09-09T23:13:30.355 INFO:teuthology.orchestra.run.smithi082.stdout:osd.10 up in weight 1 up_from 69 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6824/2069114959,v1:172.21.15.98:6825/2069114959] [v2:172.21.15.98:6826/2069114959,v1:172.21.15.98:6827/2069114959] exists,up 33a59389-7ced-42d5-bf54-2071317132d8 2023-09-09T23:13:30.356 INFO:teuthology.orchestra.run.smithi082.stdout:osd.11 up in weight 1 up_from 76 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6824/146964512,v1:172.21.15.170:6825/146964512] [v2:172.21.15.170:6826/146964512,v1:172.21.15.170:6827/146964512] exists,up b7a71f5c-85e2-4fec-be4b-95be0788b302 2023-09-09T23:13:30.356 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 2.f [8,11] 2023-09-09T23:13:30.356 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 2.13 [2,11] 2023-09-09T23:13:30.356 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 3.2 [3,9] 2023-09-09T23:13:30.356 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 3.23 [4,8] 2023-09-09T23:13:30.357 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 3.2c [4,9] 2023-09-09T23:13:30.357 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 3.33 [5,9] 2023-09-09T23:13:30.357 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/4013747788 expires 2023-09-10T23:06:08.903527+0000 2023-09-09T23:13:30.357 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:6801/1192991521 expires 2023-09-10T23:06:08.903527+0000 2023-09-09T23:13:30.357 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:6800/1192991521 expires 2023-09-10T23:06:08.903527+0000 2023-09-09T23:13:30.358 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/2823341464 expires 2023-09-10T23:06:08.903527+0000 2023-09-09T23:13:30.358 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:6801/3263999756 expires 2023-09-10T23:05:46.550354+0000 2023-09-09T23:13:30.358 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/75116806 expires 2023-09-10T23:06:08.903527+0000 2023-09-09T23:13:30.358 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/1037625632 expires 2023-09-10T23:05:46.550354+0000 2023-09-09T23:13:30.359 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/3142088817 expires 2023-09-10T23:05:46.550354+0000 2023-09-09T23:13:30.360 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:6800/3263999756 expires 2023-09-10T23:05:46.550354+0000 2023-09-09T23:13:30.360 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/1679622963 expires 2023-09-10T23:05:46.550354+0000 2023-09-09T23:13:30.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.352+0000 7f87382f7700 1 -- 172.21.15.82:0/1261721243 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f871c0a06d0 msgr2=0x7f871c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:30.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.352+0000 7f87382f7700 1 --2- 172.21.15.82:0/1261721243 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f871c0a06d0 0x7f871c0a2b90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8720005950 tx=0x7f87200058e0 comp rx=0 tx=0).stop 2023-09-09T23:13:30.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.352+0000 7f87382f7700 1 -- 172.21.15.82:0/1261721243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8730101160 msgr2=0x7f873019da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:30.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.352+0000 7f87382f7700 1 --2- 172.21.15.82:0/1261721243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8730101160 0x7f873019da50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f87280094c0 tx=0x7f87280095a0 comp rx=0 tx=0).stop 2023-09-09T23:13:30.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.352+0000 7f87382f7700 1 -- 172.21.15.82:0/1261721243 shutdown_connections 2023-09-09T23:13:30.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.352+0000 7f87382f7700 1 --2- 172.21.15.82:0/1261721243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8730101160 0x7f873019da50 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:30.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.352+0000 7f87382f7700 1 --2- 172.21.15.82:0/1261721243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87300ff5c0 0x7f8730196490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:30.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.352+0000 7f87382f7700 1 --2- 172.21.15.82:0/1261721243 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f871c0a06d0 0x7f871c0a2b90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:30.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.352+0000 7f87382f7700 1 --2- 172.21.15.82:0/1261721243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87301007c0 0x7f87301969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:30.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.353+0000 7f87382f7700 1 -- 172.21.15.82:0/1261721243 >> 172.21.15.82:0/1261721243 conn(0x7f87300fab80 msgr2=0x7f8730068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:30.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.353+0000 7f87382f7700 1 -- 172.21.15.82:0/1261721243 shutdown_connections 2023-09-09T23:13:30.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.353+0000 7f87382f7700 1 -- 172.21.15.82:0/1261721243 wait complete. 2023-09-09T23:13:30.643 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:30 smithi082 ceph-mon[98694]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:13:30.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:30 smithi170 ceph-mon[102832]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:13:30.685 INFO:teuthology.run_tasks:Running task fs.ready... 2023-09-09T23:13:30.697 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-09T23:13:30.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:30 smithi098 ceph-mon[103124]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:13:30.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.824+0000 7f6000dea700 1 -- 172.21.15.82:0/1921994378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ffc1006e0 msgr2=0x7f5ffc100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:30.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.824+0000 7f6000dea700 1 --2- 172.21.15.82:0/1921994378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ffc1006e0 0x7f5ffc100b40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0099c0 tx=0x7f5fec009cd0 comp rx=0 tx=0).stop 2023-09-09T23:13:30.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.824+0000 7f6000dea700 1 -- 172.21.15.82:0/1921994378 shutdown_connections 2023-09-09T23:13:30.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.824+0000 7f6000dea700 1 --2- 172.21.15.82:0/1921994378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ffc0ff4f0 0x7f5ffc0ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:30.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.824+0000 7f6000dea700 1 --2- 172.21.15.82:0/1921994378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ffc101080 0x7f5ffc107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:30.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.824+0000 7f6000dea700 1 --2- 172.21.15.82:0/1921994378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ffc1006e0 0x7f5ffc100b40 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:30.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.824+0000 7f6000dea700 1 -- 172.21.15.82:0/1921994378 >> 172.21.15.82:0/1921994378 conn(0x7f5ffc0faa60 msgr2=0x7f5ffc0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:30.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.824+0000 7f6000dea700 1 -- 172.21.15.82:0/1921994378 shutdown_connections 2023-09-09T23:13:30.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.824+0000 7f6000dea700 1 -- 172.21.15.82:0/1921994378 wait complete. 2023-09-09T23:13:30.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.825+0000 7f6000dea700 1 Processor -- start 2023-09-09T23:13:30.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.825+0000 7f6000dea700 1 -- start start 2023-09-09T23:13:30.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.825+0000 7f6000dea700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ffc0ff4f0 0x7f5ffc0694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:30.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.825+0000 7f6000dea700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ffc1006e0 0x7f5ffc19aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:30.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.825+0000 7f6000dea700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ffc101080 0x7f5ffc19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:30.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.825+0000 7f6000dea700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ffc19ddd0 con 0x7f5ffc101080 2023-09-09T23:13:30.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.825+0000 7f6000dea700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ffc19df40 con 0x7f5ffc1006e0 2023-09-09T23:13:30.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.825+0000 7f6000dea700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ffc19e100 con 0x7f5ffc0ff4f0 2023-09-09T23:13:30.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.825+0000 7f5ffad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ffc101080 0x7f5ffc19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:30.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.825+0000 7f5ffad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ffc101080 0x7f5ffc19d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55562/0 (socket says 172.21.15.82:55562) 2023-09-09T23:13:30.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.825+0000 7f5ffad9d700 1 -- 172.21.15.82:0/3628558592 learned_addr learned my addr 172.21.15.82:0/3628558592 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:30.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.825+0000 7f5ff9d9b700 1 --2- 172.21.15.82:0/3628558592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ffc1006e0 0x7f5ffc19aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:30.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.825+0000 7f5ffa59c700 1 --2- 172.21.15.82:0/3628558592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ffc0ff4f0 0x7f5ffc0694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:30.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.825+0000 7f5ffad9d700 1 -- 172.21.15.82:0/3628558592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ffc0ff4f0 msgr2=0x7f5ffc0694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:30.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.825+0000 7f5ffad9d700 1 --2- 172.21.15.82:0/3628558592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ffc0ff4f0 0x7f5ffc0694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:30.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.825+0000 7f5ffad9d700 1 -- 172.21.15.82:0/3628558592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ffc1006e0 msgr2=0x7f5ffc19aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:30.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.825+0000 7f5ffad9d700 1 --2- 172.21.15.82:0/3628558592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ffc1006e0 0x7f5ffc19aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:30.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.825+0000 7f5ffad9d700 1 -- 172.21.15.82:0/3628558592 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fec0095f0 con 0x7f5ffc101080 2023-09-09T23:13:30.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.826+0000 7f5ffad9d700 1 --2- 172.21.15.82:0/3628558592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ffc101080 0x7f5ffc19d630 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400b970 tx=0x7f5ff400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:30.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.826+0000 7f5feb7fe700 1 -- 172.21.15.82:0/3628558592 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff400c9b0 con 0x7f5ffc101080 2023-09-09T23:13:30.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.826+0000 7f6000dea700 1 -- 172.21.15.82:0/3628558592 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ffc19e3e0 con 0x7f5ffc101080 2023-09-09T23:13:30.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.826+0000 7f5feb7fe700 1 -- 172.21.15.82:0/3628558592 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ff4014440 con 0x7f5ffc101080 2023-09-09T23:13:30.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.826+0000 7f5feb7fe700 1 -- 172.21.15.82:0/3628558592 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff40075d0 con 0x7f5ffc101080 2023-09-09T23:13:30.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.826+0000 7f6000dea700 1 -- 172.21.15.82:0/3628558592 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ffc19e8f0 con 0x7f5ffc101080 2023-09-09T23:13:30.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.827+0000 7f6000dea700 1 -- 172.21.15.82:0/3628558592 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ffc106a40 con 0x7f5ffc101080 2023-09-09T23:13:30.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.828+0000 7f5feb7fe700 1 -- 172.21.15.82:0/3628558592 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f5ff40145b0 con 0x7f5ffc101080 2023-09-09T23:13:30.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.828+0000 7f5feb7fe700 1 --2- 172.21.15.82:0/3628558592 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5fe40a0710 0x7f5fe40a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:30.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.829+0000 7f5ffa59c700 1 --2- 172.21.15.82:0/3628558592 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5fe40a0710 0x7f5fe40a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:30.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.829+0000 7f5feb7fe700 1 -- 172.21.15.82:0/3628558592 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f5ff4006e40 con 0x7f5ffc101080 2023-09-09T23:13:30.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.829+0000 7f5ffa59c700 1 --2- 172.21.15.82:0/3628558592 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5fe40a0710 0x7f5fe40a2bd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0005fd0 tx=0x7f5ff0005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:30.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:30.832+0000 7f5feb7fe700 1 -- 172.21.15.82:0/3628558592 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5ff4095300 con 0x7f5ffc101080 2023-09-09T23:13:31.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.024+0000 7f6000dea700 1 -- 172.21.15.82:0/3628558592 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ffc10ad80 con 0x7f5ffc101080 2023-09-09T23:13:31.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.025+0000 7f5feb7fe700 1 -- 172.21.15.82:0/3628558592 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22411 (secure 0 0 0) 0x7f5ff4023090 con 0x7f5ffc101080 2023-09-09T23:13:31.024 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:31.024 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":83,"fsid":"3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da","created":"2023-09-09T23:05:14.222190+0000","modified":"2023-09-09T23:13:10.324596+0000","last_up_change":"2023-09-09T23:11:37.253759+0000","last_in_change":"2023-09-09T23:11:19.933812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-09T23:08:31.234958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-09T23:12:15.524887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-09T23:12:16.785403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1277eb17-a2e7-4e10-9832-f12a5e93ffde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6803","nonce":3197434929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6805","nonce":3197434929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6809","nonce":3197434929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6807","nonce":3197434929}]},"public_addr":"172.21.15.82:6803/3197434929","cluster_addr":"172.21.15.82:6805/3197434929","heartbeat_back_addr":"172.21.15.82:6809/3197434929","heartbeat_front_addr":"172.21.15.82:6807/3197434929","state":["exists","up"]},{"osd":1,"uuid":"f68723fd-0908-4e21-ae8e-1de1c3a9a65c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6801","nonce":3528641314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6803","nonce":3528641314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6807","nonce":3528641314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6805","nonce":3528641314}]},"public_addr":"172.21.15.98:6801/3528641314","cluster_addr":"172.21.15.98:6803/3528641314","heartbeat_back_addr":"172.21.15.98:6807/3528641314","heartbeat_front_addr":"172.21.15.98:6805/3528641314","state":["exists","up"]},{"osd":2,"uuid":"06f89c53-1419-4f52-acad-6e92916b476f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6801","nonce":4006414551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6803","nonce":4006414551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6807","nonce":4006414551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6805","nonce":4006414551}]},"public_addr":"172.21.15.170:6801/4006414551","cluster_addr":"172.21.15.170:6803/4006414551","heartbeat_back_addr":"172.21.15.170:6807/4006414551","heartbeat_front_addr":"172.21.15.170:6805/4006414551","state":["exists","up"]},{"osd":3,"uuid":"fedbd4c9-6852-42b5-8f3a-ba8d9bfe5d59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6811","nonce":3647629241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6813","nonce":3647629241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6817","nonce":3647629241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6815","nonce":3647629241}]},"public_addr":"172.21.15.82:6811/3647629241","cluster_addr":"172.21.15.82:6813/3647629241","heartbeat_back_addr":"172.21.15.82:6817/3647629241","heartbeat_front_addr":"172.21.15.82:6815/3647629241","state":["exists","up"]},{"osd":4,"uuid":"a71f3084-6919-4161-87bf-345f3355d8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6809","nonce":2507035643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6811","nonce":2507035643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6815","nonce":2507035643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6813","nonce":2507035643}]},"public_addr":"172.21.15.98:6809/2507035643","cluster_addr":"172.21.15.98:6811/2507035643","heartbeat_back_addr":"172.21.15.98:6815/2507035643","heartbeat_front_addr":"172.21.15.98:6813/2507035643","state":["exists","up"]},{"osd":5,"uuid":"f76058e9-a90e-476e-87f0-3c6723795e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6809","nonce":775005168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6811","nonce":775005168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6815","nonce":775005168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6813","nonce":775005168}]},"public_addr":"172.21.15.170:6809/775005168","cluster_addr":"172.21.15.170:6811/775005168","heartbeat_back_addr":"172.21.15.170:6815/775005168","heartbeat_front_addr":"172.21.15.170:6813/775005168","state":["exists","up"]},{"osd":6,"uuid":"25940be0-c1bd-4c72-95bf-fd8367aa2f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6819","nonce":3008405982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6821","nonce":3008405982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6825","nonce":3008405982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6823","nonce":3008405982}]},"public_addr":"172.21.15.82:6819/3008405982","cluster_addr":"172.21.15.82:6821/3008405982","heartbeat_back_addr":"172.21.15.82:6825/3008405982","heartbeat_front_addr":"172.21.15.82:6823/3008405982","state":["exists","up"]},{"osd":7,"uuid":"f651bb8c-0b25-42d5-b028-58563f4c27f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6817","nonce":3547615284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6819","nonce":3547615284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6823","nonce":3547615284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6821","nonce":3547615284}]},"public_addr":"172.21.15.98:6817/3547615284","cluster_addr":"172.21.15.98:6819/3547615284","heartbeat_back_addr":"172.21.15.98:6823/3547615284","heartbeat_front_addr":"172.21.15.98:6821/3547615284","state":["exists","up"]},{"osd":8,"uuid":"23b27c86-8c98-441e-b7ab-d57bc584e58f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6817","nonce":632038848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6819","nonce":632038848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6823","nonce":632038848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6821","nonce":632038848}]},"public_addr":"172.21.15.170:6817/632038848","cluster_addr":"172.21.15.170:6819/632038848","heartbeat_back_addr":"172.21.15.170:6823/632038848","heartbeat_front_addr":"172.21.15.170:6821/632038848","state":["exists","up"]},{"osd":9,"uuid":"383b8989-4922-4eee-9c77-f198258ac243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6827","nonce":3354242311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6829","nonce":3354242311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6833","nonce":3354242311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6831","nonce":3354242311}]},"public_addr":"172.21.15.82:6827/3354242311","cluster_addr":"172.21.15.82:6829/3354242311","heartbeat_back_addr":"172.21.15.82:6833/3354242311","heartbeat_front_addr":"172.21.15.82:6831/3354242311","state":["exists","up"]},{"osd":10,"uuid":"33a59389-7ced-42d5-bf54-2071317132d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6825","nonce":2069114959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6827","nonce":2069114959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6831","nonce":2069114959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6829","nonce":2069114959}]},"public_addr":"172.21.15.98:6825/2069114959","cluster_addr":"172.21.15.98:6827/2069114959","heartbeat_back_addr":"172.21.15.98:6831/2069114959","heartbeat_front_addr":"172.21.15.98:6829/2069114959","state":["exists","up"]},{"osd":11,"uuid":"b7a71f5c-85e2-4fec-be4b-95be0788b302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6825","nonce":146964512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6827","nonce":146964512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6831","nonce":146964512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6829","nonce":146964512}]},"public_addr":"172.21.15.170:6825/146964512","cluster_addr":"172.21.15.170:6827/146964512","heartbeat_back_addr":"172.21.15.170:6831/146964512","heartbeat_front_addr":"172.21.15.170:6829/146964512","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:07:47.563043+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:08.359260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:29.070374+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:49.897775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:11.434248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:31.538464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:52.377109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:13.452968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:33.806897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:54.083177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:14.980087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:35.690089+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4013747788":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6800/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/2823341464":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/75116806":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/1037625632":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/3142088817":"2023-09-10T23:05:46.550354+0000","172.21.15.82:6800/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/1679622963":"2023-09-10T23:05:46.550354+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-09T23:13:31.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.027+0000 7f6000dea700 1 -- 172.21.15.82:0/3628558592 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5fe40a0710 msgr2=0x7f5fe40a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:31.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.027+0000 7f6000dea700 1 --2- 172.21.15.82:0/3628558592 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5fe40a0710 0x7f5fe40a2bd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0005fd0 tx=0x7f5ff0005dd0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.027+0000 7f6000dea700 1 -- 172.21.15.82:0/3628558592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ffc101080 msgr2=0x7f5ffc19d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:31.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.027+0000 7f6000dea700 1 --2- 172.21.15.82:0/3628558592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ffc101080 0x7f5ffc19d630 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400b970 tx=0x7f5ff400bd30 comp rx=0 tx=0).stop 2023-09-09T23:13:31.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.027+0000 7f6000dea700 1 -- 172.21.15.82:0/3628558592 shutdown_connections 2023-09-09T23:13:31.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.027+0000 7f6000dea700 1 --2- 172.21.15.82:0/3628558592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ffc101080 0x7f5ffc19d630 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.027+0000 7f6000dea700 1 --2- 172.21.15.82:0/3628558592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ffc0ff4f0 0x7f5ffc0694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.027+0000 7f6000dea700 1 --2- 172.21.15.82:0/3628558592 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5fe40a0710 0x7f5fe40a2bd0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.027+0000 7f6000dea700 1 --2- 172.21.15.82:0/3628558592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ffc1006e0 0x7f5ffc19aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.027+0000 7f6000dea700 1 -- 172.21.15.82:0/3628558592 >> 172.21.15.82:0/3628558592 conn(0x7f5ffc0faa60 msgr2=0x7f5ffc103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:31.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.028+0000 7f6000dea700 1 -- 172.21.15.82:0/3628558592 shutdown_connections 2023-09-09T23:13:31.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.028+0000 7f6000dea700 1 -- 172.21.15.82:0/3628558592 wait complete. 2023-09-09T23:13:31.039 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-09T23:08:31.234958+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-09T23:12:15.524887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-09T23:12:16.785403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-09T23:13:31.039 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-09T23:13:31.163 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.165+0000 7f0b191c8700 1 -- 172.21.15.82:0/1920902508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b140ff4f0 msgr2=0x7f0b140ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:31.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.165+0000 7f0b191c8700 1 --2- 172.21.15.82:0/1920902508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b140ff4f0 0x7f0b140ff910 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0b080099c0 tx=0x7f0b08009cd0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.165+0000 7f0b191c8700 1 -- 172.21.15.82:0/1920902508 shutdown_connections 2023-09-09T23:13:31.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.165+0000 7f0b191c8700 1 --2- 172.21.15.82:0/1920902508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b14101080 0x7f0b14107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.165+0000 7f0b191c8700 1 --2- 172.21.15.82:0/1920902508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b140ff4f0 0x7f0b140ff910 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.165+0000 7f0b191c8700 1 --2- 172.21.15.82:0/1920902508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b141006e0 0x7f0b14100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.165+0000 7f0b191c8700 1 -- 172.21.15.82:0/1920902508 >> 172.21.15.82:0/1920902508 conn(0x7f0b140faa60 msgr2=0x7f0b140fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:31.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.165+0000 7f0b191c8700 1 -- 172.21.15.82:0/1920902508 shutdown_connections 2023-09-09T23:13:31.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.165+0000 7f0b191c8700 1 -- 172.21.15.82:0/1920902508 wait complete. 2023-09-09T23:13:31.168 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.165+0000 7f0b191c8700 1 Processor -- start 2023-09-09T23:13:31.168 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.166+0000 7f0b191c8700 1 -- start start 2023-09-09T23:13:31.168 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.166+0000 7f0b191c8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b140ff4f0 0x7f0b14196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:31.168 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.166+0000 7f0b191c8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b141006e0 0x7f0b141968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:31.169 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.166+0000 7f0b191c8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b14101080 0x7f0b1419d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:31.169 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.166+0000 7f0b191c8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1419dee0 con 0x7f0b140ff4f0 2023-09-09T23:13:31.169 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.166+0000 7f0b191c8700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1419e020 con 0x7f0b141006e0 2023-09-09T23:13:31.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.166+0000 7f0b191c8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1419e1b0 con 0x7f0b14101080 2023-09-09T23:13:31.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.166+0000 7f0b12d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b140ff4f0 0x7f0b14196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:31.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.166+0000 7f0b12d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b140ff4f0 0x7f0b14196360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55572/0 (socket says 172.21.15.82:55572) 2023-09-09T23:13:31.171 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.166+0000 7f0b12d9d700 1 -- 172.21.15.82:0/2423451542 learned_addr learned my addr 172.21.15.82:0/2423451542 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:31.171 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.166+0000 7f0b1259c700 1 --2- 172.21.15.82:0/2423451542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b141006e0 0x7f0b141968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:31.171 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.166+0000 7f0b1359e700 1 --2- 172.21.15.82:0/2423451542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b14101080 0x7f0b1419d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:31.172 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.166+0000 7f0b12d9d700 1 -- 172.21.15.82:0/2423451542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b14101080 msgr2=0x7f0b1419d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:31.172 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.166+0000 7f0b12d9d700 1 --2- 172.21.15.82:0/2423451542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b14101080 0x7f0b1419d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.172 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.166+0000 7f0b12d9d700 1 -- 172.21.15.82:0/2423451542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b141006e0 msgr2=0x7f0b141968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:31.173 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.166+0000 7f0b12d9d700 1 --2- 172.21.15.82:0/2423451542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b141006e0 0x7f0b141968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.173 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.166+0000 7f0b12d9d700 1 -- 172.21.15.82:0/2423451542 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b080095f0 con 0x7f0b140ff4f0 2023-09-09T23:13:31.173 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.166+0000 7f0b12d9d700 1 --2- 172.21.15.82:0/2423451542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b140ff4f0 0x7f0b14196360 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009990 tx=0x7f0b0800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:31.174 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.167+0000 7f0afbfff700 1 -- 172.21.15.82:0/2423451542 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0801d070 con 0x7f0b140ff4f0 2023-09-09T23:13:31.174 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.167+0000 7f0b191c8700 1 -- 172.21.15.82:0/2423451542 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b1419e430 con 0x7f0b140ff4f0 2023-09-09T23:13:31.174 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.167+0000 7f0afbfff700 1 -- 172.21.15.82:0/2423451542 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b080054f0 con 0x7f0b140ff4f0 2023-09-09T23:13:31.174 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.167+0000 7f0afbfff700 1 -- 172.21.15.82:0/2423451542 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b08004c70 con 0x7f0b140ff4f0 2023-09-09T23:13:31.174 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.167+0000 7f0b191c8700 1 -- 172.21.15.82:0/2423451542 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b1419e850 con 0x7f0b140ff4f0 2023-09-09T23:13:31.175 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.168+0000 7f0b191c8700 1 -- 172.21.15.82:0/2423451542 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b14107670 con 0x7f0b140ff4f0 2023-09-09T23:13:31.175 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.169+0000 7f0afbfff700 1 -- 172.21.15.82:0/2423451542 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f0b0800fc30 con 0x7f0b140ff4f0 2023-09-09T23:13:31.175 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.169+0000 7f0afbfff700 1 --2- 172.21.15.82:0/2423451542 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0afc0a06d0 0x7f0afc0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:31.175 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.169+0000 7f0b1259c700 1 --2- 172.21.15.82:0/2423451542 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0afc0a06d0 0x7f0afc0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:31.175 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.169+0000 7f0afbfff700 1 -- 172.21.15.82:0/2423451542 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f0b080ccf60 con 0x7f0b140ff4f0 2023-09-09T23:13:31.176 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.170+0000 7f0b1259c700 1 --2- 172.21.15.82:0/2423451542 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0afc0a06d0 0x7f0afc0a2b90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0b04005950 tx=0x7f0b040058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:31.176 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.173+0000 7f0afbfff700 1 -- 172.21.15.82:0/2423451542 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0b08096ac0 con 0x7f0b140ff4f0 2023-09-09T23:13:31.288 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:31 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1261721243' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-09-09T23:13:31.288 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:31 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3628558592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:31.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.364+0000 7f0b191c8700 1 -- 172.21.15.82:0/2423451542 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0b1410aef0 con 0x7f0b140ff4f0 2023-09-09T23:13:31.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.364+0000 7f0afbfff700 1 -- 172.21.15.82:0/2423451542 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0b08026020 con 0x7f0b140ff4f0 2023-09-09T23:13:31.364 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2023-09-09T23:13:31.366 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.366+0000 7f0b191c8700 1 -- 172.21.15.82:0/2423451542 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0afc0a06d0 msgr2=0x7f0afc0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:31.366 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.366+0000 7f0b191c8700 1 --2- 172.21.15.82:0/2423451542 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0afc0a06d0 0x7f0afc0a2b90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0b04005950 tx=0x7f0b040058e0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.367 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.366+0000 7f0b191c8700 1 -- 172.21.15.82:0/2423451542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b140ff4f0 msgr2=0x7f0b14196360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:31.367 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.366+0000 7f0b191c8700 1 --2- 172.21.15.82:0/2423451542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b140ff4f0 0x7f0b14196360 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009990 tx=0x7f0b0800f740 comp rx=0 tx=0).stop 2023-09-09T23:13:31.367 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.367+0000 7f0b191c8700 1 -- 172.21.15.82:0/2423451542 shutdown_connections 2023-09-09T23:13:31.368 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.367+0000 7f0b191c8700 1 --2- 172.21.15.82:0/2423451542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b14101080 0x7f0b1419d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.368 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.367+0000 7f0b191c8700 1 --2- 172.21.15.82:0/2423451542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b140ff4f0 0x7f0b14196360 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.368 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.367+0000 7f0b191c8700 1 --2- 172.21.15.82:0/2423451542 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0afc0a06d0 0x7f0afc0a2b90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.367+0000 7f0b191c8700 1 --2- 172.21.15.82:0/2423451542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b141006e0 0x7f0b141968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.367+0000 7f0b191c8700 1 -- 172.21.15.82:0/2423451542 >> 172.21.15.82:0/2423451542 conn(0x7f0b140faa60 msgr2=0x7f0b140fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:31.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.367+0000 7f0b191c8700 1 -- 172.21.15.82:0/2423451542 shutdown_connections 2023-09-09T23:13:31.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.367+0000 7f0b191c8700 1 -- 172.21.15.82:0/2423451542 wait complete. 2023-09-09T23:13:31.377 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-09T23:13:31.502 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.503+0000 7fbaf0fc3700 1 -- 172.21.15.82:0/1519176620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaec101870 msgr2=0x7fbaec101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:31.503 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.503+0000 7fbaf0fc3700 1 --2- 172.21.15.82:0/1519176620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaec101870 0x7fbaec101cd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbad4009990 tx=0x7fbad4009ca0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.503 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.503+0000 7fbaf0fc3700 1 -- 172.21.15.82:0/1519176620 shutdown_connections 2023-09-09T23:13:31.503 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.503+0000 7fbaf0fc3700 1 --2- 172.21.15.82:0/1519176620 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbaec102210 0x7fbaec108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.503 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.503+0000 7fbaf0fc3700 1 --2- 172.21.15.82:0/1519176620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbaec0fe8a0 0x7fbaec0fecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.503 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.503+0000 7fbaf0fc3700 1 --2- 172.21.15.82:0/1519176620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaec101870 0x7fbaec101cd0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.503+0000 7fbaf0fc3700 1 -- 172.21.15.82:0/1519176620 >> 172.21.15.82:0/1519176620 conn(0x7fbaec0fa1d0 msgr2=0x7fbaec0fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.503+0000 7fbaf0fc3700 1 -- 172.21.15.82:0/1519176620 shutdown_connections 2023-09-09T23:13:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.503+0000 7fbaf0fc3700 1 -- 172.21.15.82:0/1519176620 wait complete. 2023-09-09T23:13:31.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.504+0000 7fbaf0fc3700 1 Processor -- start 2023-09-09T23:13:31.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.504+0000 7fbaf0fc3700 1 -- start start 2023-09-09T23:13:31.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.505+0000 7fbaf0fc3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaec0fe8a0 0x7fbaec1962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:31.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.505+0000 7fbaf0fc3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbaec101870 0x7fbaec196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:31.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.505+0000 7fbaf0fc3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbaec102210 0x7fbaec19d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:31.508 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.505+0000 7fbaf0fc3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaec19de40 con 0x7fbaec0fe8a0 2023-09-09T23:13:31.508 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.505+0000 7fbaf0fc3700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaec19df80 con 0x7fbaec102210 2023-09-09T23:13:31.508 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.505+0000 7fbaf0fc3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaec19e110 con 0x7fbaec101870 2023-09-09T23:13:31.508 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.505+0000 7fbaea59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaec0fe8a0 0x7fbaec1962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:31.508 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.505+0000 7fbaea59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaec0fe8a0 0x7fbaec1962c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55586/0 (socket says 172.21.15.82:55586) 2023-09-09T23:13:31.509 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.505+0000 7fbaea59c700 1 -- 172.21.15.82:0/2237544642 learned_addr learned my addr 172.21.15.82:0/2237544642 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:31.509 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.505+0000 7fbae9d9b700 1 --2- 172.21.15.82:0/2237544642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbaec101870 0x7fbaec196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:31.509 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.505+0000 7fbaead9d700 1 --2- 172.21.15.82:0/2237544642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbaec102210 0x7fbaec19d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:31.509 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.505+0000 7fbaea59c700 1 -- 172.21.15.82:0/2237544642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbaec101870 msgr2=0x7fbaec196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:31.509 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.505+0000 7fbaea59c700 1 --2- 172.21.15.82:0/2237544642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbaec101870 0x7fbaec196800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.509 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.505+0000 7fbaea59c700 1 -- 172.21.15.82:0/2237544642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbaec102210 msgr2=0x7fbaec19d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:31.510 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.505+0000 7fbaea59c700 1 --2- 172.21.15.82:0/2237544642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbaec102210 0x7fbaec19d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.510 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.505+0000 7fbaea59c700 1 -- 172.21.15.82:0/2237544642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad40095f0 con 0x7fbaec0fe8a0 2023-09-09T23:13:31.510 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.505+0000 7fbaea59c700 1 --2- 172.21.15.82:0/2237544642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaec0fe8a0 0x7fbaec1962c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbadc00d950 tx=0x7fbadc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:31.510 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.505+0000 7fbae37fe700 1 -- 172.21.15.82:0/2237544642 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc004a30 con 0x7fbaec0fe8a0 2023-09-09T23:13:31.510 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.505+0000 7fbae37fe700 1 -- 172.21.15.82:0/2237544642 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbadc0054e0 con 0x7fbaec0fe8a0 2023-09-09T23:13:31.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.505+0000 7fbaf0fc3700 1 -- 172.21.15.82:0/2237544642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaec19e3f0 con 0x7fbaec0fe8a0 2023-09-09T23:13:31.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.505+0000 7fbae37fe700 1 -- 172.21.15.82:0/2237544642 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc00fbc0 con 0x7fbaec0fe8a0 2023-09-09T23:13:31.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.505+0000 7fbaf0fc3700 1 -- 172.21.15.82:0/2237544642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaec19e8b0 con 0x7fbaec0fe8a0 2023-09-09T23:13:31.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.506+0000 7fbaf0fc3700 1 -- 172.21.15.82:0/2237544642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaec10be50 con 0x7fbaec0fe8a0 2023-09-09T23:13:31.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.507+0000 7fbae37fe700 1 -- 172.21.15.82:0/2237544642 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fbadc004c20 con 0x7fbaec0fe8a0 2023-09-09T23:13:31.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.508+0000 7fbae37fe700 1 --2- 172.21.15.82:0/2237544642 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fbad80a06d0 0x7fbad80a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:31.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.508+0000 7fbae9d9b700 1 --2- 172.21.15.82:0/2237544642 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fbad80a06d0 0x7fbad80a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:31.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.508+0000 7fbae37fe700 1 -- 172.21.15.82:0/2237544642 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fbadc0cba10 con 0x7fbaec0fe8a0 2023-09-09T23:13:31.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.508+0000 7fbae9d9b700 1 --2- 172.21.15.82:0/2237544642 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fbad80a06d0 0x7fbad80a2b90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbad400b5c0 tx=0x7fbad4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:31.513 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.512+0000 7fbae37fe700 1 -- 172.21.15.82:0/2237544642 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fbadc0954e0 con 0x7fbaec0fe8a0 2023-09-09T23:13:31.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:31 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1261721243' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-09-09T23:13:31.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:31 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3628558592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:31.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.702+0000 7fbaf0fc3700 1 -- 172.21.15.82:0/2237544642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbaec107560 con 0x7fbaec0fe8a0 2023-09-09T23:13:31.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.702+0000 7fbae37fe700 1 -- 172.21.15.82:0/2237544642 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbadc098b00 con 0x7fbaec0fe8a0 2023-09-09T23:13:31.701 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2023-09-09T23:13:31.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.705+0000 7fbaf0fc3700 1 -- 172.21.15.82:0/2237544642 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fbad80a06d0 msgr2=0x7fbad80a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:31.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.705+0000 7fbaf0fc3700 1 --2- 172.21.15.82:0/2237544642 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fbad80a06d0 0x7fbad80a2b90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbad400b5c0 tx=0x7fbad4005fb0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.705+0000 7fbaf0fc3700 1 -- 172.21.15.82:0/2237544642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaec0fe8a0 msgr2=0x7fbaec1962c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:31.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.705+0000 7fbaf0fc3700 1 --2- 172.21.15.82:0/2237544642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaec0fe8a0 0x7fbaec1962c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbadc00d950 tx=0x7fbadc00dc60 comp rx=0 tx=0).stop 2023-09-09T23:13:31.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.705+0000 7fbaf0fc3700 1 -- 172.21.15.82:0/2237544642 shutdown_connections 2023-09-09T23:13:31.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.705+0000 7fbaf0fc3700 1 --2- 172.21.15.82:0/2237544642 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fbad80a06d0 0x7fbad80a2b90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.705+0000 7fbaf0fc3700 1 --2- 172.21.15.82:0/2237544642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbaec102210 0x7fbaec19d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.705+0000 7fbaf0fc3700 1 --2- 172.21.15.82:0/2237544642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaec0fe8a0 0x7fbaec1962c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.705+0000 7fbaf0fc3700 1 --2- 172.21.15.82:0/2237544642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbaec101870 0x7fbaec196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.705+0000 7fbaf0fc3700 1 -- 172.21.15.82:0/2237544642 >> 172.21.15.82:0/2237544642 conn(0x7fbaec0fa1d0 msgr2=0x7fbaec0fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:31.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.705+0000 7fbaf0fc3700 1 -- 172.21.15.82:0/2237544642 shutdown_connections 2023-09-09T23:13:31.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.705+0000 7fbaf0fc3700 1 -- 172.21.15.82:0/2237544642 wait complete. 2023-09-09T23:13:31.716 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-09T23:13:31.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:31 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1261721243' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-09-09T23:13:31.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:31 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3628558592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:31.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.842+0000 7f158c3a2700 1 -- 172.21.15.82:0/3999008368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15840ff4f0 msgr2=0x7f15840ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:31.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.842+0000 7f158c3a2700 1 --2- 172.21.15.82:0/3999008368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15840ff4f0 0x7f15840ff910 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f15740099c0 tx=0x7f1574009cd0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.842+0000 7f158c3a2700 1 -- 172.21.15.82:0/3999008368 shutdown_connections 2023-09-09T23:13:31.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.842+0000 7f158c3a2700 1 --2- 172.21.15.82:0/3999008368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1584101080 0x7f1584107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.842+0000 7f158c3a2700 1 --2- 172.21.15.82:0/3999008368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15840ff4f0 0x7f15840ff910 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.842+0000 7f158c3a2700 1 --2- 172.21.15.82:0/3999008368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15841006e0 0x7f1584100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.842+0000 7f158c3a2700 1 -- 172.21.15.82:0/3999008368 >> 172.21.15.82:0/3999008368 conn(0x7f15840faa60 msgr2=0x7f15840fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:31.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.842+0000 7f158c3a2700 1 -- 172.21.15.82:0/3999008368 shutdown_connections 2023-09-09T23:13:31.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.842+0000 7f158c3a2700 1 -- 172.21.15.82:0/3999008368 wait complete. 2023-09-09T23:13:31.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.843+0000 7f158c3a2700 1 Processor -- start 2023-09-09T23:13:31.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.843+0000 7f158c3a2700 1 -- start start 2023-09-09T23:13:31.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.843+0000 7f158c3a2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15840ff4f0 0x7f1584069360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:31.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.843+0000 7f158c3a2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15841006e0 0x7f158419b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:31.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.843+0000 7f158c3a2700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1584101080 0x7f158419d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:31.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.843+0000 7f158c3a2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158419de70 con 0x7f15841006e0 2023-09-09T23:13:31.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.843+0000 7f158c3a2700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158419dfe0 con 0x7f1584101080 2023-09-09T23:13:31.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.843+0000 7f158c3a2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158419e1a0 con 0x7f15840ff4f0 2023-09-09T23:13:31.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.843+0000 7f158993d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15841006e0 0x7f158419b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:31.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.843+0000 7f158993d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15841006e0 0x7f158419b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55602/0 (socket says 172.21.15.82:55602) 2023-09-09T23:13:31.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.843+0000 7f158993d700 1 -- 172.21.15.82:0/683826282 learned_addr learned my addr 172.21.15.82:0/683826282 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:31.848 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.843+0000 7f158a13e700 1 --2- 172.21.15.82:0/683826282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15840ff4f0 0x7f1584069360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:31.848 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.843+0000 7f158a93f700 1 --2- 172.21.15.82:0/683826282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1584101080 0x7f158419d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:31.848 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.843+0000 7f158993d700 1 -- 172.21.15.82:0/683826282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15840ff4f0 msgr2=0x7f1584069360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:31.848 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.843+0000 7f158993d700 1 --2- 172.21.15.82:0/683826282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15840ff4f0 0x7f1584069360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.848 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.843+0000 7f158993d700 1 -- 172.21.15.82:0/683826282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1584101080 msgr2=0x7f158419d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:31.849 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.843+0000 7f158993d700 1 --2- 172.21.15.82:0/683826282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1584101080 0x7f158419d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:31.849 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.843+0000 7f158993d700 1 -- 172.21.15.82:0/683826282 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15740095f0 con 0x7f15841006e0 2023-09-09T23:13:31.849 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.843+0000 7f158a13e700 1 --2- 172.21.15.82:0/683826282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15840ff4f0 0x7f1584069360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:31.849 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.843+0000 7f158a93f700 1 --2- 172.21.15.82:0/683826282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1584101080 0x7f158419d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:31.849 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.844+0000 7f158993d700 1 --2- 172.21.15.82:0/683826282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15841006e0 0x7f158419b040 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f158000d950 tx=0x7f158000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:31.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.844+0000 7f157b7fe700 1 -- 172.21.15.82:0/683826282 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1580009b40 con 0x7f15841006e0 2023-09-09T23:13:31.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.844+0000 7f158c3a2700 1 -- 172.21.15.82:0/683826282 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f158419e480 con 0x7f15841006e0 2023-09-09T23:13:31.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.844+0000 7f157b7fe700 1 -- 172.21.15.82:0/683826282 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1580009ca0 con 0x7f15841006e0 2023-09-09T23:13:31.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.844+0000 7f157b7fe700 1 -- 172.21.15.82:0/683826282 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158000f620 con 0x7f15841006e0 2023-09-09T23:13:31.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.844+0000 7f158c3a2700 1 -- 172.21.15.82:0/683826282 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f158419e990 con 0x7f15841006e0 2023-09-09T23:13:31.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.845+0000 7f158c3a2700 1 -- 172.21.15.82:0/683826282 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15841064a0 con 0x7f15841006e0 2023-09-09T23:13:31.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.846+0000 7f157b7fe700 1 -- 172.21.15.82:0/683826282 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f1580010460 con 0x7f15841006e0 2023-09-09T23:13:31.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.846+0000 7f157b7fe700 1 --2- 172.21.15.82:0/683826282 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f15700a06d0 0x7f15700a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:31.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.846+0000 7f157b7fe700 1 -- 172.21.15.82:0/683826282 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f15800cc030 con 0x7f15841006e0 2023-09-09T23:13:31.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.846+0000 7f158a13e700 1 --2- 172.21.15.82:0/683826282 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f15700a06d0 0x7f15700a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:31.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.847+0000 7f158a13e700 1 --2- 172.21.15.82:0/683826282 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f15700a06d0 0x7f15700a2b90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f157400b580 tx=0x7f1574005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:31.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:31.850+0000 7f157b7fe700 1 -- 172.21.15.82:0/683826282 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1580095b00 con 0x7f15841006e0 2023-09-09T23:13:32.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.041+0000 7f158c3a2700 1 -- 172.21.15.82:0/683826282 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f158410aef0 con 0x7f15841006e0 2023-09-09T23:13:32.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.042+0000 7f157b7fe700 1 -- 172.21.15.82:0/683826282 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f15800200a0 con 0x7f15841006e0 2023-09-09T23:13:32.041 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2023-09-09T23:13:32.043 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.044+0000 7f158c3a2700 1 -- 172.21.15.82:0/683826282 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f15700a06d0 msgr2=0x7f15700a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:32.043 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.044+0000 7f158c3a2700 1 --2- 172.21.15.82:0/683826282 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f15700a06d0 0x7f15700a2b90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f157400b580 tx=0x7f1574005fb0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.043 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.044+0000 7f158c3a2700 1 -- 172.21.15.82:0/683826282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15841006e0 msgr2=0x7f158419b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:32.043 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.044+0000 7f158c3a2700 1 --2- 172.21.15.82:0/683826282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15841006e0 0x7f158419b040 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f158000d950 tx=0x7f158000dd10 comp rx=0 tx=0).stop 2023-09-09T23:13:32.044 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.044+0000 7f158c3a2700 1 -- 172.21.15.82:0/683826282 shutdown_connections 2023-09-09T23:13:32.044 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.044+0000 7f158c3a2700 1 --2- 172.21.15.82:0/683826282 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f15700a06d0 0x7f15700a2b90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.044 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.044+0000 7f158c3a2700 1 --2- 172.21.15.82:0/683826282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1584101080 0x7f158419d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.044 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.044+0000 7f158c3a2700 1 --2- 172.21.15.82:0/683826282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15840ff4f0 0x7f1584069360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.044 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.044+0000 7f158c3a2700 1 --2- 172.21.15.82:0/683826282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15841006e0 0x7f158419b040 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.045 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.044+0000 7f158c3a2700 1 -- 172.21.15.82:0/683826282 >> 172.21.15.82:0/683826282 conn(0x7f15840faa60 msgr2=0x7f15840fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:32.045 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.044+0000 7f158c3a2700 1 -- 172.21.15.82:0/683826282 shutdown_connections 2023-09-09T23:13:32.045 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.044+0000 7f158c3a2700 1 -- 172.21.15.82:0/683826282 wait complete. 2023-09-09T23:13:32.055 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-09T23:13:32.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.178+0000 7fd2fbbe8700 1 -- 172.21.15.82:0/558979447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2f41062c0 msgr2=0x7fd2f4108730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:32.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.178+0000 7fd2fbbe8700 1 --2- 172.21.15.82:0/558979447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2f41062c0 0x7fd2f4108730 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00b0a0 tx=0x7fd2ec00b3b0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.178+0000 7fd2fbbe8700 1 -- 172.21.15.82:0/558979447 shutdown_connections 2023-09-09T23:13:32.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.178+0000 7fd2fbbe8700 1 --2- 172.21.15.82:0/558979447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2f41062c0 0x7fd2f4108730 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.178+0000 7fd2fbbe8700 1 --2- 172.21.15.82:0/558979447 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2f4101060 0x7fd2f4103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.184 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.178+0000 7fd2fbbe8700 1 --2- 172.21.15.82:0/558979447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2f4103990 0x7fd2f4105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.184 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.178+0000 7fd2fbbe8700 1 -- 172.21.15.82:0/558979447 >> 172.21.15.82:0/558979447 conn(0x7fd2f40faa30 msgr2=0x7fd2f40fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:32.184 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.178+0000 7fd2fbbe8700 1 -- 172.21.15.82:0/558979447 shutdown_connections 2023-09-09T23:13:32.184 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.178+0000 7fd2fbbe8700 1 -- 172.21.15.82:0/558979447 wait complete. 2023-09-09T23:13:32.185 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.178+0000 7fd2fbbe8700 1 Processor -- start 2023-09-09T23:13:32.185 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.178+0000 7fd2fbbe8700 1 -- start start 2023-09-09T23:13:32.185 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.178+0000 7fd2fbbe8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2f4101060 0x7fd2f41009d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:32.186 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.179+0000 7fd2fbbe8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2f4103990 0x7fd2f40ff010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:32.186 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.179+0000 7fd2fbbe8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2f41062c0 0x7fd2f40ff550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:32.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.179+0000 7fd2fbbe8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f40ffcf0 con 0x7fd2f41062c0 2023-09-09T23:13:32.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.179+0000 7fd2fbbe8700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f40ffe60 con 0x7fd2f4103990 2023-09-09T23:13:32.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.179+0000 7fd2fbbe8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f41a0400 con 0x7fd2f4101060 2023-09-09T23:13:32.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.179+0000 7fd2fa185700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2f41062c0 0x7fd2f40ff550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:32.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.179+0000 7fd2fa185700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2f41062c0 0x7fd2f40ff550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55618/0 (socket says 172.21.15.82:55618) 2023-09-09T23:13:32.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.179+0000 7fd2fa185700 1 -- 172.21.15.82:0/3472920368 learned_addr learned my addr 172.21.15.82:0/3472920368 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:32.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.179+0000 7fd2f9984700 1 --2- 172.21.15.82:0/3472920368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2f4101060 0x7fd2f41009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:32.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.179+0000 7fd2f9183700 1 --2- 172.21.15.82:0/3472920368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2f4103990 0x7fd2f40ff010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:32.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.179+0000 7fd2fa185700 1 -- 172.21.15.82:0/3472920368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2f4101060 msgr2=0x7fd2f41009d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:32.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.179+0000 7fd2fa185700 1 --2- 172.21.15.82:0/3472920368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2f4101060 0x7fd2f41009d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.179+0000 7fd2fa185700 1 -- 172.21.15.82:0/3472920368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2f4103990 msgr2=0x7fd2f40ff010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:32.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.179+0000 7fd2fa185700 1 --2- 172.21.15.82:0/3472920368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2f4103990 0x7fd2f40ff010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.179+0000 7fd2fa185700 1 -- 172.21.15.82:0/3472920368 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ec009d00 con 0x7fd2f41062c0 2023-09-09T23:13:32.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.179+0000 7fd2f9183700 1 --2- 172.21.15.82:0/3472920368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2f4103990 0x7fd2f40ff010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:32.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.179+0000 7fd2fa185700 1 --2- 172.21.15.82:0/3472920368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2f41062c0 0x7fd2f40ff550 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00b070 tx=0x7fd2ec004b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:32.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.179+0000 7fd2eaffd700 1 -- 172.21.15.82:0/3472920368 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec00e040 con 0x7fd2f41062c0 2023-09-09T23:13:32.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.179+0000 7fd2eaffd700 1 -- 172.21.15.82:0/3472920368 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2ec004020 con 0x7fd2f41062c0 2023-09-09T23:13:32.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.179+0000 7fd2eaffd700 1 -- 172.21.15.82:0/3472920368 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec01b6b0 con 0x7fd2f41062c0 2023-09-09T23:13:32.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.179+0000 7fd2fbbe8700 1 -- 172.21.15.82:0/3472920368 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f41a0680 con 0x7fd2f41062c0 2023-09-09T23:13:32.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.179+0000 7fd2fbbe8700 1 -- 172.21.15.82:0/3472920368 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f41a0ab0 con 0x7fd2f41062c0 2023-09-09T23:13:32.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.180+0000 7fd2fbbe8700 1 -- 172.21.15.82:0/3472920368 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2f40fc5b0 con 0x7fd2f41062c0 2023-09-09T23:13:32.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.181+0000 7fd2eaffd700 1 -- 172.21.15.82:0/3472920368 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fd2ec004190 con 0x7fd2f41062c0 2023-09-09T23:13:32.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.182+0000 7fd2eaffd700 1 --2- 172.21.15.82:0/3472920368 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd2e00a0790 0x7fd2e00a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:32.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.182+0000 7fd2f9984700 1 --2- 172.21.15.82:0/3472920368 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd2e00a0790 0x7fd2e00a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:32.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.182+0000 7fd2eaffd700 1 -- 172.21.15.82:0/3472920368 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fd2ec0cd320 con 0x7fd2f41062c0 2023-09-09T23:13:32.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.182+0000 7fd2f9984700 1 --2- 172.21.15.82:0/3472920368 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd2e00a0790 0x7fd2e00a2c50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4005950 tx=0x7fd2e40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:32.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.185+0000 7fd2eaffd700 1 -- 172.21.15.82:0/3472920368 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd2ec096d80 con 0x7fd2f41062c0 2023-09-09T23:13:32.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.420+0000 7fd2fbbe8700 1 -- 172.21.15.82:0/3472920368 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2f410c0e0 con 0x7fd2f41062c0 2023-09-09T23:13:32.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.421+0000 7fd2eaffd700 1 -- 172.21.15.82:0/3472920368 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11368 (secure 0 0 0) 0x7fd2ec09a3a0 con 0x7fd2f41062c0 2023-09-09T23:13:32.420 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:32.420 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2471432684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2471432684},{"type":"v1","addr":"172.21.15.82:6835","nonce":2471432684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3579387137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3579387137},{"type":"v1","addr":"172.21.15.82:6837","nonce":3579387137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-09T23:12:17.586036+0000","modified":"2023-09-09T23:12:25.285958+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24542,"mds_2":24523,"mds_3":24491,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14712":{"gid":14712,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/2127301345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2127301345},{"type":"v1","addr":"172.21.15.82:6839","nonce":2127301345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14724":{"gid":14724,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/4143604488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":4143604488},{"type":"v1","addr":"172.21.15.82:6841","nonce":4143604488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.170:6833/1931961940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1931961940},{"type":"v1","addr":"172.21.15.170:6833","nonce":1931961940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24491":{"gid":24491,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.98:6833/3855146755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3855146755},{"type":"v1","addr":"172.21.15.98:6833","nonce":3855146755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2313861511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2313861511},{"type":"v1","addr":"172.21.15.170:6835","nonce":2313861511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/4006360308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":4006360308},{"type":"v1","addr":"172.21.15.170:6837","nonce":4006360308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1356474828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1356474828},{"type":"v1","addr":"172.21.15.98:6835","nonce":1356474828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/3858369735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3858369735},{"type":"v1","addr":"172.21.15.170:6839","nonce":3858369735}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/2019449840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2019449840},{"type":"v1","addr":"172.21.15.98:6837","nonce":2019449840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2785086371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2785086371},{"type":"v1","addr":"172.21.15.98:6839","nonce":2785086371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-09T23:13:32.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.423+0000 7fd2fbbe8700 1 -- 172.21.15.82:0/3472920368 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd2e00a0790 msgr2=0x7fd2e00a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:32.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.423+0000 7fd2fbbe8700 1 --2- 172.21.15.82:0/3472920368 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd2e00a0790 0x7fd2e00a2c50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4005950 tx=0x7fd2e40058e0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.423+0000 7fd2fbbe8700 1 -- 172.21.15.82:0/3472920368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2f41062c0 msgr2=0x7fd2f40ff550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:32.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.423+0000 7fd2fbbe8700 1 --2- 172.21.15.82:0/3472920368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2f41062c0 0x7fd2f40ff550 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00b070 tx=0x7fd2ec004b00 comp rx=0 tx=0).stop 2023-09-09T23:13:32.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.423+0000 7fd2fbbe8700 1 -- 172.21.15.82:0/3472920368 shutdown_connections 2023-09-09T23:13:32.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.423+0000 7fd2fbbe8700 1 --2- 172.21.15.82:0/3472920368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2f41062c0 0x7fd2f40ff550 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.423+0000 7fd2fbbe8700 1 --2- 172.21.15.82:0/3472920368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2f4101060 0x7fd2f41009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.423+0000 7fd2fbbe8700 1 --2- 172.21.15.82:0/3472920368 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd2e00a0790 0x7fd2e00a2c50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.423+0000 7fd2fbbe8700 1 --2- 172.21.15.82:0/3472920368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2f4103990 0x7fd2f40ff010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.423+0000 7fd2fbbe8700 1 -- 172.21.15.82:0/3472920368 >> 172.21.15.82:0/3472920368 conn(0x7fd2f40faa30 msgr2=0x7fd2f4101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:32.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.423+0000 7fd2fbbe8700 1 -- 172.21.15.82:0/3472920368 shutdown_connections 2023-09-09T23:13:32.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.423+0000 7fd2fbbe8700 1 -- 172.21.15.82:0/3472920368 wait complete. 2023-09-09T23:13:32.424 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 27 2023-09-09T23:13:32.433 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:32 smithi082 ceph-mon[98694]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-09T23:13:32.433 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:32 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2423451542' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:13:32.433 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:32 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2237544642' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-09T23:13:32.433 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:32 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/683826282' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-09T23:13:32.435 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-09T23:13:32.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.558+0000 7fec6caaf700 1 -- 172.21.15.82:0/3800947666 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec68100190 msgr2=0x7fec68107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:32.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.558+0000 7fec6caaf700 1 --2- 172.21.15.82:0/3800947666 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec68100190 0x7fec68107a50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fec6000b0a0 tx=0x7fec6000b3b0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.558+0000 7fec6caaf700 1 -- 172.21.15.82:0/3800947666 shutdown_connections 2023-09-09T23:13:32.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.558+0000 7fec6caaf700 1 --2- 172.21.15.82:0/3800947666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec68100ac0 0x7fec68100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.558+0000 7fec6caaf700 1 --2- 172.21.15.82:0/3800947666 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec68100190 0x7fec68107a50 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.558+0000 7fec6caaf700 1 --2- 172.21.15.82:0/3800947666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec680ff7f0 0x7fec680ffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.558+0000 7fec6caaf700 1 -- 172.21.15.82:0/3800947666 >> 172.21.15.82:0/3800947666 conn(0x7fec680fa200 msgr2=0x7fec680fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:32.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.558+0000 7fec6caaf700 1 -- 172.21.15.82:0/3800947666 shutdown_connections 2023-09-09T23:13:32.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.558+0000 7fec6caaf700 1 -- 172.21.15.82:0/3800947666 wait complete. 2023-09-09T23:13:32.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.559+0000 7fec6caaf700 1 Processor -- start 2023-09-09T23:13:32.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.559+0000 7fec6caaf700 1 -- start start 2023-09-09T23:13:32.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.559+0000 7fec6caaf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec680ff7f0 0x7fec680ff250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:32.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.559+0000 7fec6caaf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec68100190 0x7fec6819b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:32.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.559+0000 7fec6caaf700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec68100ac0 0x7fec6819d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:32.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.559+0000 7fec6caaf700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6819de50 con 0x7fec680ff7f0 2023-09-09T23:13:32.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.559+0000 7fec6caaf700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6819dfc0 con 0x7fec68100ac0 2023-09-09T23:13:32.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.559+0000 7fec6caaf700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6819e180 con 0x7fec68100190 2023-09-09T23:13:32.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.559+0000 7fec6659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec680ff7f0 0x7fec680ff250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:32.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.559+0000 7fec6659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec680ff7f0 0x7fec680ff250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55628/0 (socket says 172.21.15.82:55628) 2023-09-09T23:13:32.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.559+0000 7fec6659c700 1 -- 172.21.15.82:0/1031457255 learned_addr learned my addr 172.21.15.82:0/1031457255 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:32.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.559+0000 7fec65d9b700 1 --2- 172.21.15.82:0/1031457255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec68100190 0x7fec6819b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:32.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.559+0000 7fec66d9d700 1 --2- 172.21.15.82:0/1031457255 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec68100ac0 0x7fec6819d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:32.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.559+0000 7fec6659c700 1 -- 172.21.15.82:0/1031457255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec68100190 msgr2=0x7fec6819b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:32.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.559+0000 7fec6659c700 1 --2- 172.21.15.82:0/1031457255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec68100190 0x7fec6819b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.559+0000 7fec6659c700 1 -- 172.21.15.82:0/1031457255 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec68100ac0 msgr2=0x7fec6819d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:32.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.559+0000 7fec6659c700 1 --2- 172.21.15.82:0/1031457255 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec68100ac0 0x7fec6819d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.559+0000 7fec6659c700 1 -- 172.21.15.82:0/1031457255 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec60009d00 con 0x7fec680ff7f0 2023-09-09T23:13:32.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.559+0000 7fec66d9d700 1 --2- 172.21.15.82:0/1031457255 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec68100ac0 0x7fec6819d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:32.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.559+0000 7fec6659c700 1 --2- 172.21.15.82:0/1031457255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec680ff7f0 0x7fec680ff250 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00d980 tx=0x7fec5c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:32.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.559+0000 7fec577fe700 1 -- 172.21.15.82:0/1031457255 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c004020 con 0x7fec680ff7f0 2023-09-09T23:13:32.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.560+0000 7fec6caaf700 1 -- 172.21.15.82:0/1031457255 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec6819e460 con 0x7fec680ff7f0 2023-09-09T23:13:32.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.560+0000 7fec577fe700 1 -- 172.21.15.82:0/1031457255 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec5c009730 con 0x7fec680ff7f0 2023-09-09T23:13:32.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.560+0000 7fec577fe700 1 -- 172.21.15.82:0/1031457255 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c003a30 con 0x7fec680ff7f0 2023-09-09T23:13:32.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.560+0000 7fec6caaf700 1 -- 172.21.15.82:0/1031457255 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec6819e970 con 0x7fec680ff7f0 2023-09-09T23:13:32.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.560+0000 7fec6caaf700 1 -- 172.21.15.82:0/1031457255 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec6810ad40 con 0x7fec680ff7f0 2023-09-09T23:13:32.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.561+0000 7fec577fe700 1 -- 172.21.15.82:0/1031457255 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fec5c0098a0 con 0x7fec680ff7f0 2023-09-09T23:13:32.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.562+0000 7fec577fe700 1 --2- 172.21.15.82:0/1031457255 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fec500a0790 0x7fec500a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:32.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.562+0000 7fec65d9b700 1 --2- 172.21.15.82:0/1031457255 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fec500a0790 0x7fec500a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:32.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.562+0000 7fec577fe700 1 -- 172.21.15.82:0/1031457255 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fec5c0cba10 con 0x7fec680ff7f0 2023-09-09T23:13:32.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.562+0000 7fec65d9b700 1 --2- 172.21.15.82:0/1031457255 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fec500a0790 0x7fec500a2c50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fec58005950 tx=0x7fec5800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:32.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.566+0000 7fec577fe700 1 -- 172.21.15.82:0/1031457255 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fec5c0953e0 con 0x7fec680ff7f0 2023-09-09T23:13:32.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:32 smithi170 ceph-mon[102832]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-09T23:13:32.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:32 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2423451542' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:13:32.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:32 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2237544642' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-09T23:13:32.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:32 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/683826282' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-09T23:13:32.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:32 smithi098 ceph-mon[103124]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-09T23:13:32.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:32 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2423451542' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:13:32.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:32 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2237544642' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-09T23:13:32.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:32 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/683826282' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-09T23:13:32.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.762+0000 7fec6caaf700 1 -- 172.21.15.82:0/1031457255 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec6810af50 con 0x7fec680ff7f0 2023-09-09T23:13:32.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.763+0000 7fec577fe700 1 -- 172.21.15.82:0/1031457255 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22411 (secure 0 0 0) 0x7fec5c098a00 con 0x7fec680ff7f0 2023-09-09T23:13:32.762 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:32.762 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":83,"fsid":"3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da","created":"2023-09-09T23:05:14.222190+0000","modified":"2023-09-09T23:13:10.324596+0000","last_up_change":"2023-09-09T23:11:37.253759+0000","last_in_change":"2023-09-09T23:11:19.933812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-09T23:08:31.234958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-09T23:12:15.524887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-09T23:12:16.785403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1277eb17-a2e7-4e10-9832-f12a5e93ffde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6803","nonce":3197434929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6805","nonce":3197434929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6809","nonce":3197434929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6807","nonce":3197434929}]},"public_addr":"172.21.15.82:6803/3197434929","cluster_addr":"172.21.15.82:6805/3197434929","heartbeat_back_addr":"172.21.15.82:6809/3197434929","heartbeat_front_addr":"172.21.15.82:6807/3197434929","state":["exists","up"]},{"osd":1,"uuid":"f68723fd-0908-4e21-ae8e-1de1c3a9a65c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6801","nonce":3528641314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6803","nonce":3528641314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6807","nonce":3528641314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6805","nonce":3528641314}]},"public_addr":"172.21.15.98:6801/3528641314","cluster_addr":"172.21.15.98:6803/3528641314","heartbeat_back_addr":"172.21.15.98:6807/3528641314","heartbeat_front_addr":"172.21.15.98:6805/3528641314","state":["exists","up"]},{"osd":2,"uuid":"06f89c53-1419-4f52-acad-6e92916b476f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6801","nonce":4006414551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6803","nonce":4006414551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6807","nonce":4006414551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6805","nonce":4006414551}]},"public_addr":"172.21.15.170:6801/4006414551","cluster_addr":"172.21.15.170:6803/4006414551","heartbeat_back_addr":"172.21.15.170:6807/4006414551","heartbeat_front_addr":"172.21.15.170:6805/4006414551","state":["exists","up"]},{"osd":3,"uuid":"fedbd4c9-6852-42b5-8f3a-ba8d9bfe5d59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6811","nonce":3647629241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6813","nonce":3647629241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6817","nonce":3647629241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6815","nonce":3647629241}]},"public_addr":"172.21.15.82:6811/3647629241","cluster_addr":"172.21.15.82:6813/3647629241","heartbeat_back_addr":"172.21.15.82:6817/3647629241","heartbeat_front_addr":"172.21.15.82:6815/3647629241","state":["exists","up"]},{"osd":4,"uuid":"a71f3084-6919-4161-87bf-345f3355d8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6809","nonce":2507035643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6811","nonce":2507035643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6815","nonce":2507035643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6813","nonce":2507035643}]},"public_addr":"172.21.15.98:6809/2507035643","cluster_addr":"172.21.15.98:6811/2507035643","heartbeat_back_addr":"172.21.15.98:6815/2507035643","heartbeat_front_addr":"172.21.15.98:6813/2507035643","state":["exists","up"]},{"osd":5,"uuid":"f76058e9-a90e-476e-87f0-3c6723795e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6809","nonce":775005168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6811","nonce":775005168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6815","nonce":775005168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6813","nonce":775005168}]},"public_addr":"172.21.15.170:6809/775005168","cluster_addr":"172.21.15.170:6811/775005168","heartbeat_back_addr":"172.21.15.170:6815/775005168","heartbeat_front_addr":"172.21.15.170:6813/775005168","state":["exists","up"]},{"osd":6,"uuid":"25940be0-c1bd-4c72-95bf-fd8367aa2f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6819","nonce":3008405982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6821","nonce":3008405982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6825","nonce":3008405982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6823","nonce":3008405982}]},"public_addr":"172.21.15.82:6819/3008405982","cluster_addr":"172.21.15.82:6821/3008405982","heartbeat_back_addr":"172.21.15.82:6825/3008405982","heartbeat_front_addr":"172.21.15.82:6823/3008405982","state":["exists","up"]},{"osd":7,"uuid":"f651bb8c-0b25-42d5-b028-58563f4c27f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6817","nonce":3547615284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6819","nonce":3547615284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6823","nonce":3547615284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6821","nonce":3547615284}]},"public_addr":"172.21.15.98:6817/3547615284","cluster_addr":"172.21.15.98:6819/3547615284","heartbeat_back_addr":"172.21.15.98:6823/3547615284","heartbeat_front_addr":"172.21.15.98:6821/3547615284","state":["exists","up"]},{"osd":8,"uuid":"23b27c86-8c98-441e-b7ab-d57bc584e58f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6817","nonce":632038848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6819","nonce":632038848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6823","nonce":632038848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6821","nonce":632038848}]},"public_addr":"172.21.15.170:6817/632038848","cluster_addr":"172.21.15.170:6819/632038848","heartbeat_back_addr":"172.21.15.170:6823/632038848","heartbeat_front_addr":"172.21.15.170:6821/632038848","state":["exists","up"]},{"osd":9,"uuid":"383b8989-4922-4eee-9c77-f198258ac243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6827","nonce":3354242311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6829","nonce":3354242311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6833","nonce":3354242311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6831","nonce":3354242311}]},"public_addr":"172.21.15.82:6827/3354242311","cluster_addr":"172.21.15.82:6829/3354242311","heartbeat_back_addr":"172.21.15.82:6833/3354242311","heartbeat_front_addr":"172.21.15.82:6831/3354242311","state":["exists","up"]},{"osd":10,"uuid":"33a59389-7ced-42d5-bf54-2071317132d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6825","nonce":2069114959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6827","nonce":2069114959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6831","nonce":2069114959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6829","nonce":2069114959}]},"public_addr":"172.21.15.98:6825/2069114959","cluster_addr":"172.21.15.98:6827/2069114959","heartbeat_back_addr":"172.21.15.98:6831/2069114959","heartbeat_front_addr":"172.21.15.98:6829/2069114959","state":["exists","up"]},{"osd":11,"uuid":"b7a71f5c-85e2-4fec-be4b-95be0788b302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6825","nonce":146964512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6827","nonce":146964512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6831","nonce":146964512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6829","nonce":146964512}]},"public_addr":"172.21.15.170:6825/146964512","cluster_addr":"172.21.15.170:6827/146964512","heartbeat_back_addr":"172.21.15.170:6831/146964512","heartbeat_front_addr":"172.21.15.170:6829/146964512","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:07:47.563043+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:08.359260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:29.070374+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:49.897775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:11.434248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:31.538464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:52.377109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:13.452968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:33.806897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:54.083177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:14.980087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:35.690089+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4013747788":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6800/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/2823341464":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/75116806":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/1037625632":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/3142088817":"2023-09-10T23:05:46.550354+0000","172.21.15.82:6800/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/1679622963":"2023-09-10T23:05:46.550354+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-09T23:13:32.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.765+0000 7fec6caaf700 1 -- 172.21.15.82:0/1031457255 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fec500a0790 msgr2=0x7fec500a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:32.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.765+0000 7fec6caaf700 1 --2- 172.21.15.82:0/1031457255 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fec500a0790 0x7fec500a2c50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fec58005950 tx=0x7fec5800b410 comp rx=0 tx=0).stop 2023-09-09T23:13:32.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.765+0000 7fec6caaf700 1 -- 172.21.15.82:0/1031457255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec680ff7f0 msgr2=0x7fec680ff250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:32.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.765+0000 7fec6caaf700 1 --2- 172.21.15.82:0/1031457255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec680ff7f0 0x7fec680ff250 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00d980 tx=0x7fec5c00dc90 comp rx=0 tx=0).stop 2023-09-09T23:13:32.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.765+0000 7fec6caaf700 1 -- 172.21.15.82:0/1031457255 shutdown_connections 2023-09-09T23:13:32.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.765+0000 7fec6caaf700 1 --2- 172.21.15.82:0/1031457255 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fec500a0790 0x7fec500a2c50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.765+0000 7fec6caaf700 1 --2- 172.21.15.82:0/1031457255 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec68100ac0 0x7fec6819d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.765+0000 7fec6caaf700 1 --2- 172.21.15.82:0/1031457255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec680ff7f0 0x7fec680ff250 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.765+0000 7fec6caaf700 1 --2- 172.21.15.82:0/1031457255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec68100190 0x7fec6819b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.765+0000 7fec6caaf700 1 -- 172.21.15.82:0/1031457255 >> 172.21.15.82:0/1031457255 conn(0x7fec680fa200 msgr2=0x7fec680fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:32.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.765+0000 7fec6caaf700 1 -- 172.21.15.82:0/1031457255 shutdown_connections 2023-09-09T23:13:32.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.765+0000 7fec6caaf700 1 -- 172.21.15.82:0/1031457255 wait complete. 2023-09-09T23:13:32.776 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-09T23:08:31.234958+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-09T23:12:15.524887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-09T23:12:16.785403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-09T23:13:32.777 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-09T23:13:32.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.903+0000 7f1e47419700 1 -- 172.21.15.82:0/1186237803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e40103a40 msgr2=0x7f1e40105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:32.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.903+0000 7f1e47419700 1 --2- 172.21.15.82:0/1186237803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e40103a40 0x7f1e40105e30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1e30009990 tx=0x7f1e30009ca0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.904+0000 7f1e47419700 1 -- 172.21.15.82:0/1186237803 shutdown_connections 2023-09-09T23:13:32.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.904+0000 7f1e47419700 1 --2- 172.21.15.82:0/1186237803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e40106370 0x7f1e40108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.904+0000 7f1e47419700 1 --2- 172.21.15.82:0/1186237803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e40101110 0x7f1e40103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.904+0000 7f1e47419700 1 --2- 172.21.15.82:0/1186237803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e40103a40 0x7f1e40105e30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.904+0000 7f1e47419700 1 -- 172.21.15.82:0/1186237803 >> 172.21.15.82:0/1186237803 conn(0x7f1e400faa60 msgr2=0x7f1e400fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:32.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.904+0000 7f1e47419700 1 -- 172.21.15.82:0/1186237803 shutdown_connections 2023-09-09T23:13:32.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.904+0000 7f1e47419700 1 -- 172.21.15.82:0/1186237803 wait complete. 2023-09-09T23:13:32.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.904+0000 7f1e47419700 1 Processor -- start 2023-09-09T23:13:32.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.904+0000 7f1e47419700 1 -- start start 2023-09-09T23:13:32.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.904+0000 7f1e47419700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e40101110 0x7f1e40100a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:32.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.904+0000 7f1e47419700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e40103a40 0x7f1e400ff0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:32.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.904+0000 7f1e47419700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e40106370 0x7f1e400ff600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:32.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.904+0000 7f1e47419700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e400ffda0 con 0x7f1e40106370 2023-09-09T23:13:32.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.904+0000 7f1e47419700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e400fff10 con 0x7f1e40103a40 2023-09-09T23:13:32.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.904+0000 7f1e47419700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e401a0340 con 0x7f1e40101110 2023-09-09T23:13:32.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.904+0000 7f1e459b6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e40106370 0x7f1e400ff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:32.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.904+0000 7f1e459b6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e40106370 0x7f1e400ff600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55646/0 (socket says 172.21.15.82:55646) 2023-09-09T23:13:32.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.905+0000 7f1e459b6700 1 -- 172.21.15.82:0/148020440 learned_addr learned my addr 172.21.15.82:0/148020440 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:32.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.905+0000 7f1e451b5700 1 --2- 172.21.15.82:0/148020440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e40101110 0x7f1e40100a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:32.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.905+0000 7f1e449b4700 1 --2- 172.21.15.82:0/148020440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e40103a40 0x7f1e400ff0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:32.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.905+0000 7f1e459b6700 1 -- 172.21.15.82:0/148020440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e40101110 msgr2=0x7f1e40100a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:32.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.905+0000 7f1e459b6700 1 --2- 172.21.15.82:0/148020440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e40101110 0x7f1e40100a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.905+0000 7f1e459b6700 1 -- 172.21.15.82:0/148020440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e40103a40 msgr2=0x7f1e400ff0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:32.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.905+0000 7f1e459b6700 1 --2- 172.21.15.82:0/148020440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e40103a40 0x7f1e400ff0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:32.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.905+0000 7f1e459b6700 1 -- 172.21.15.82:0/148020440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e300095f0 con 0x7f1e40106370 2023-09-09T23:13:32.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.905+0000 7f1e459b6700 1 --2- 172.21.15.82:0/148020440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e40106370 0x7f1e400ff600 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800ef30 tx=0x7f1e3800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:32.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.905+0000 7f1e367fc700 1 -- 172.21.15.82:0/148020440 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e38018070 con 0x7f1e40106370 2023-09-09T23:13:32.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.905+0000 7f1e47419700 1 -- 172.21.15.82:0/148020440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e401a05c0 con 0x7f1e40106370 2023-09-09T23:13:32.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.905+0000 7f1e367fc700 1 -- 172.21.15.82:0/148020440 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e3800f040 con 0x7f1e40106370 2023-09-09T23:13:32.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.905+0000 7f1e367fc700 1 -- 172.21.15.82:0/148020440 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3800cdb0 con 0x7f1e40106370 2023-09-09T23:13:32.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.905+0000 7f1e47419700 1 -- 172.21.15.82:0/148020440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e401a0a80 con 0x7f1e40106370 2023-09-09T23:13:32.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.906+0000 7f1e47419700 1 -- 172.21.15.82:0/148020440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4010be40 con 0x7f1e40106370 2023-09-09T23:13:32.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.907+0000 7f1e367fc700 1 -- 172.21.15.82:0/148020440 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f1e38003a00 con 0x7f1e40106370 2023-09-09T23:13:32.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.908+0000 7f1e367fc700 1 --2- 172.21.15.82:0/148020440 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1e2c0a06d0 0x7f1e2c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:32.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.908+0000 7f1e451b5700 1 --2- 172.21.15.82:0/148020440 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1e2c0a06d0 0x7f1e2c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:32.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.908+0000 7f1e367fc700 1 -- 172.21.15.82:0/148020440 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f1e380cc210 con 0x7f1e40106370 2023-09-09T23:13:32.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.908+0000 7f1e451b5700 1 --2- 172.21.15.82:0/148020440 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1e2c0a06d0 0x7f1e2c0a2b90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c005950 tx=0x7f1e3c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:32.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:32.911+0000 7f1e367fc700 1 -- 172.21.15.82:0/148020440 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1e38095d70 con 0x7f1e40106370 2023-09-09T23:13:33.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.100+0000 7f1e47419700 1 -- 172.21.15.82:0/148020440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e401a0ca0 con 0x7f1e40106370 2023-09-09T23:13:33.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.101+0000 7f1e367fc700 1 -- 172.21.15.82:0/148020440 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e38099390 con 0x7f1e40106370 2023-09-09T23:13:33.100 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2023-09-09T23:13:33.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.103+0000 7f1e47419700 1 -- 172.21.15.82:0/148020440 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1e2c0a06d0 msgr2=0x7f1e2c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:33.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.103+0000 7f1e47419700 1 --2- 172.21.15.82:0/148020440 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1e2c0a06d0 0x7f1e2c0a2b90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c005950 tx=0x7f1e3c00a300 comp rx=0 tx=0).stop 2023-09-09T23:13:33.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.103+0000 7f1e47419700 1 -- 172.21.15.82:0/148020440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e40106370 msgr2=0x7f1e400ff600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:33.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.103+0000 7f1e47419700 1 --2- 172.21.15.82:0/148020440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e40106370 0x7f1e400ff600 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800ef30 tx=0x7f1e3800c3a0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.103+0000 7f1e47419700 1 -- 172.21.15.82:0/148020440 shutdown_connections 2023-09-09T23:13:33.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.103+0000 7f1e47419700 1 --2- 172.21.15.82:0/148020440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e40106370 0x7f1e400ff600 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.103+0000 7f1e47419700 1 --2- 172.21.15.82:0/148020440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e40101110 0x7f1e40100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.103+0000 7f1e47419700 1 --2- 172.21.15.82:0/148020440 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f1e2c0a06d0 0x7f1e2c0a2b90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.103+0000 7f1e47419700 1 --2- 172.21.15.82:0/148020440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e40103a40 0x7f1e400ff0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.103+0000 7f1e47419700 1 -- 172.21.15.82:0/148020440 >> 172.21.15.82:0/148020440 conn(0x7f1e400faa60 msgr2=0x7f1e401025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:33.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.103+0000 7f1e47419700 1 -- 172.21.15.82:0/148020440 shutdown_connections 2023-09-09T23:13:33.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.103+0000 7f1e47419700 1 -- 172.21.15.82:0/148020440 wait complete. 2023-09-09T23:13:33.114 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-09T23:13:33.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.240+0000 7ff7cb5f9700 1 -- 172.21.15.82:0/2235362187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7c4101080 msgr2=0x7ff7c4107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:33.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.240+0000 7ff7cb5f9700 1 --2- 172.21.15.82:0/2235362187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7c4101080 0x7ff7c4107930 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc00b0d0 tx=0x7ff7bc00b3e0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.240+0000 7ff7cb5f9700 1 -- 172.21.15.82:0/2235362187 shutdown_connections 2023-09-09T23:13:33.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.240+0000 7ff7cb5f9700 1 --2- 172.21.15.82:0/2235362187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7c40ff4f0 0x7ff7c40ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.240+0000 7ff7cb5f9700 1 --2- 172.21.15.82:0/2235362187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7c4101080 0x7ff7c4107930 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.240+0000 7ff7cb5f9700 1 --2- 172.21.15.82:0/2235362187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7c41006e0 0x7ff7c4100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.240+0000 7ff7cb5f9700 1 -- 172.21.15.82:0/2235362187 >> 172.21.15.82:0/2235362187 conn(0x7ff7c40faa60 msgr2=0x7ff7c40fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:33.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.240+0000 7ff7cb5f9700 1 -- 172.21.15.82:0/2235362187 shutdown_connections 2023-09-09T23:13:33.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.240+0000 7ff7cb5f9700 1 -- 172.21.15.82:0/2235362187 wait complete. 2023-09-09T23:13:33.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.240+0000 7ff7cb5f9700 1 Processor -- start 2023-09-09T23:13:33.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.240+0000 7ff7cb5f9700 1 -- start start 2023-09-09T23:13:33.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.240+0000 7ff7cb5f9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7c40ff4f0 0x7ff7c40682f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:33.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.240+0000 7ff7cb5f9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7c41006e0 0x7ff7c419b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:33.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.240+0000 7ff7cb5f9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7c4101080 0x7ff7c419d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:33.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.240+0000 7ff7cb5f9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c419df00 con 0x7ff7c40ff4f0 2023-09-09T23:13:33.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.240+0000 7ff7cb5f9700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c419e070 con 0x7ff7c41006e0 2023-09-09T23:13:33.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.241+0000 7ff7cb5f9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c419e230 con 0x7ff7c4101080 2023-09-09T23:13:33.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.241+0000 7ff7c9395700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7c40ff4f0 0x7ff7c40682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:33.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.241+0000 7ff7c9395700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7c40ff4f0 0x7ff7c40682f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55654/0 (socket says 172.21.15.82:55654) 2023-09-09T23:13:33.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.241+0000 7ff7c9395700 1 -- 172.21.15.82:0/41915599 learned_addr learned my addr 172.21.15.82:0/41915599 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:33.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.241+0000 7ff7c9b96700 1 --2- 172.21.15.82:0/41915599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7c4101080 0x7ff7c419d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:33.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.241+0000 7ff7c8b94700 1 --2- 172.21.15.82:0/41915599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7c41006e0 0x7ff7c419b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:33.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.241+0000 7ff7c9395700 1 -- 172.21.15.82:0/41915599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7c4101080 msgr2=0x7ff7c419d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:33.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.241+0000 7ff7c9395700 1 --2- 172.21.15.82:0/41915599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7c4101080 0x7ff7c419d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.241+0000 7ff7c9395700 1 -- 172.21.15.82:0/41915599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7c41006e0 msgr2=0x7ff7c419b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:33.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.241+0000 7ff7c9395700 1 --2- 172.21.15.82:0/41915599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7c41006e0 0x7ff7c419b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.241+0000 7ff7c9395700 1 -- 172.21.15.82:0/41915599 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7bc009d00 con 0x7ff7c40ff4f0 2023-09-09T23:13:33.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.241+0000 7ff7c9b96700 1 --2- 172.21.15.82:0/41915599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7c4101080 0x7ff7c419d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:33.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.241+0000 7ff7c9395700 1 --2- 172.21.15.82:0/41915599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7c40ff4f0 0x7ff7c40682f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400c970 tx=0x7ff7b400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:33.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.241+0000 7ff7ba7fc700 1 -- 172.21.15.82:0/41915599 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4004220 con 0x7ff7c40ff4f0 2023-09-09T23:13:33.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.241+0000 7ff7ba7fc700 1 -- 172.21.15.82:0/41915599 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7b4007d60 con 0x7ff7c40ff4f0 2023-09-09T23:13:33.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.241+0000 7ff7cb5f9700 1 -- 172.21.15.82:0/41915599 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c419e510 con 0x7ff7c40ff4f0 2023-09-09T23:13:33.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.241+0000 7ff7ba7fc700 1 -- 172.21.15.82:0/41915599 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4003d00 con 0x7ff7c40ff4f0 2023-09-09T23:13:33.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.241+0000 7ff7cb5f9700 1 -- 172.21.15.82:0/41915599 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c419ea70 con 0x7ff7c40ff4f0 2023-09-09T23:13:33.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.242+0000 7ff7cb5f9700 1 -- 172.21.15.82:0/41915599 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c4106420 con 0x7ff7c40ff4f0 2023-09-09T23:13:33.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.243+0000 7ff7ba7fc700 1 -- 172.21.15.82:0/41915599 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7ff7b4004380 con 0x7ff7c40ff4f0 2023-09-09T23:13:33.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.244+0000 7ff7ba7fc700 1 --2- 172.21.15.82:0/41915599 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff7b00a0750 0x7ff7b00a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:33.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.244+0000 7ff7c8b94700 1 --2- 172.21.15.82:0/41915599 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff7b00a0750 0x7ff7b00a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:33.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.244+0000 7ff7ba7fc700 1 -- 172.21.15.82:0/41915599 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff7b40cbc40 con 0x7ff7c40ff4f0 2023-09-09T23:13:33.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.244+0000 7ff7c8b94700 1 --2- 172.21.15.82:0/41915599 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff7b00a0750 0x7ff7b00a2c10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0005950 tx=0x7ff7c00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:33.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.248+0000 7ff7ba7fc700 1 -- 172.21.15.82:0/41915599 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff7b4095610 con 0x7ff7c40ff4f0 2023-09-09T23:13:33.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.441+0000 7ff7cb5f9700 1 -- 172.21.15.82:0/41915599 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff7c410af70 con 0x7ff7c40ff4f0 2023-09-09T23:13:33.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.441+0000 7ff7ba7fc700 1 -- 172.21.15.82:0/41915599 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff7b4098c30 con 0x7ff7c40ff4f0 2023-09-09T23:13:33.440 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2023-09-09T23:13:33.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.444+0000 7ff7cb5f9700 1 -- 172.21.15.82:0/41915599 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff7b00a0750 msgr2=0x7ff7b00a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:33.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.444+0000 7ff7cb5f9700 1 --2- 172.21.15.82:0/41915599 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff7b00a0750 0x7ff7b00a2c10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0005950 tx=0x7ff7c00058e0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.444+0000 7ff7cb5f9700 1 -- 172.21.15.82:0/41915599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7c40ff4f0 msgr2=0x7ff7c40682f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:33.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.444+0000 7ff7cb5f9700 1 --2- 172.21.15.82:0/41915599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7c40ff4f0 0x7ff7c40682f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400c970 tx=0x7ff7b400cd30 comp rx=0 tx=0).stop 2023-09-09T23:13:33.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.444+0000 7ff7cb5f9700 1 -- 172.21.15.82:0/41915599 shutdown_connections 2023-09-09T23:13:33.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.444+0000 7ff7cb5f9700 1 --2- 172.21.15.82:0/41915599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7c4101080 0x7ff7c419d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.444+0000 7ff7cb5f9700 1 --2- 172.21.15.82:0/41915599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7c40ff4f0 0x7ff7c40682f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.444+0000 7ff7cb5f9700 1 --2- 172.21.15.82:0/41915599 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff7b00a0750 0x7ff7b00a2c10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.444+0000 7ff7cb5f9700 1 --2- 172.21.15.82:0/41915599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7c41006e0 0x7ff7c419b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.444+0000 7ff7cb5f9700 1 -- 172.21.15.82:0/41915599 >> 172.21.15.82:0/41915599 conn(0x7ff7c40faa60 msgr2=0x7ff7c40fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:33.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.444+0000 7ff7cb5f9700 1 -- 172.21.15.82:0/41915599 shutdown_connections 2023-09-09T23:13:33.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.444+0000 7ff7cb5f9700 1 -- 172.21.15.82:0/41915599 wait complete. 2023-09-09T23:13:33.453 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:33 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3472920368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:13:33.453 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:33 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1031457255' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:33.454 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:33 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/148020440' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:13:33.455 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-09T23:13:33.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.581+0000 7f5a47828700 1 -- 172.21.15.82:0/164468448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a40101080 msgr2=0x7f5a40107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:33.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.581+0000 7f5a47828700 1 --2- 172.21.15.82:0/164468448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a40101080 0x7f5a40107930 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800b0d0 tx=0x7f5a3800b3e0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.581+0000 7f5a47828700 1 -- 172.21.15.82:0/164468448 shutdown_connections 2023-09-09T23:13:33.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.581+0000 7f5a47828700 1 --2- 172.21.15.82:0/164468448 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a400ff4f0 0x7f5a400ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.581+0000 7f5a47828700 1 --2- 172.21.15.82:0/164468448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a40101080 0x7f5a40107930 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.581+0000 7f5a47828700 1 --2- 172.21.15.82:0/164468448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a401006e0 0x7f5a40100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.581+0000 7f5a47828700 1 -- 172.21.15.82:0/164468448 >> 172.21.15.82:0/164468448 conn(0x7f5a400faa60 msgr2=0x7f5a400fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:33.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.581+0000 7f5a47828700 1 -- 172.21.15.82:0/164468448 shutdown_connections 2023-09-09T23:13:33.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.581+0000 7f5a47828700 1 -- 172.21.15.82:0/164468448 wait complete. 2023-09-09T23:13:33.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.582+0000 7f5a47828700 1 Processor -- start 2023-09-09T23:13:33.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.582+0000 7f5a47828700 1 -- start start 2023-09-09T23:13:33.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.582+0000 7f5a47828700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a400ff4f0 0x7f5a40196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:33.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.582+0000 7f5a47828700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a401006e0 0x7f5a40196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:33.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.582+0000 7f5a47828700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a40101080 0x7f5a4019d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:33.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.582+0000 7f5a47828700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4019dfb0 con 0x7f5a401006e0 2023-09-09T23:13:33.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.582+0000 7f5a47828700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4019e0f0 con 0x7f5a40101080 2023-09-09T23:13:33.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.582+0000 7f5a47828700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4019e280 con 0x7f5a400ff4f0 2023-09-09T23:13:33.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.582+0000 7f5a44dc3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a401006e0 0x7f5a40196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:33.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.582+0000 7f5a44dc3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a401006e0 0x7f5a40196970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55670/0 (socket says 172.21.15.82:55670) 2023-09-09T23:13:33.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.582+0000 7f5a44dc3700 1 -- 172.21.15.82:0/1807127184 learned_addr learned my addr 172.21.15.82:0/1807127184 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:33.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.582+0000 7f5a455c4700 1 --2- 172.21.15.82:0/1807127184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a400ff4f0 0x7f5a40196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:33.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.582+0000 7f5a44dc3700 1 -- 172.21.15.82:0/1807127184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a400ff4f0 msgr2=0x7f5a40196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:33.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.582+0000 7f5a44dc3700 1 --2- 172.21.15.82:0/1807127184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a400ff4f0 0x7f5a40196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.582+0000 7f5a44dc3700 1 -- 172.21.15.82:0/1807127184 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a40101080 msgr2=0x7f5a4019d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:33.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.582+0000 7f5a44dc3700 1 --2- 172.21.15.82:0/1807127184 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a40101080 0x7f5a4019d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.582+0000 7f5a44dc3700 1 -- 172.21.15.82:0/1807127184 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a38009d00 con 0x7f5a401006e0 2023-09-09T23:13:33.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.582+0000 7f5a44dc3700 1 --2- 172.21.15.82:0/1807127184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a401006e0 0x7f5a40196970 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c00b530 tx=0x7f5a3c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:33.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.582+0000 7f5a367fc700 1 -- 172.21.15.82:0/1807127184 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c011830 con 0x7f5a401006e0 2023-09-09T23:13:33.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.582+0000 7f5a367fc700 1 -- 172.21.15.82:0/1807127184 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a3c011e50 con 0x7f5a401006e0 2023-09-09T23:13:33.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.582+0000 7f5a47828700 1 -- 172.21.15.82:0/1807127184 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a4019e560 con 0x7f5a401006e0 2023-09-09T23:13:33.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.583+0000 7f5a367fc700 1 -- 172.21.15.82:0/1807127184 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c00f2e0 con 0x7f5a401006e0 2023-09-09T23:13:33.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.583+0000 7f5a47828700 1 -- 172.21.15.82:0/1807127184 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a4019eaf0 con 0x7f5a401006e0 2023-09-09T23:13:33.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.583+0000 7f5a47828700 1 -- 172.21.15.82:0/1807127184 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a4010ad80 con 0x7f5a401006e0 2023-09-09T23:13:33.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.585+0000 7f5a367fc700 1 -- 172.21.15.82:0/1807127184 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f5a3c010380 con 0x7f5a401006e0 2023-09-09T23:13:33.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.585+0000 7f5a367fc700 1 --2- 172.21.15.82:0/1807127184 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5a2c0a06d0 0x7f5a2c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:33.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.585+0000 7f5a455c4700 1 --2- 172.21.15.82:0/1807127184 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5a2c0a06d0 0x7f5a2c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:33.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.586+0000 7f5a367fc700 1 -- 172.21.15.82:0/1807127184 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f5a3c0cbc00 con 0x7f5a401006e0 2023-09-09T23:13:33.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.586+0000 7f5a455c4700 1 --2- 172.21.15.82:0/1807127184 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5a2c0a06d0 0x7f5a2c0a2b90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5a40100540 tx=0x7f5a3000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:33.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.589+0000 7f5a367fc700 1 -- 172.21.15.82:0/1807127184 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5a3c0957e0 con 0x7f5a401006e0 2023-09-09T23:13:33.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:33 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3472920368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:13:33.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:33 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1031457255' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:33.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:33 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/148020440' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:13:33.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:33 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3472920368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:13:33.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:33 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1031457255' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:33.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:33 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/148020440' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:13:33.778 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.780+0000 7f5a47828700 1 -- 172.21.15.82:0/1807127184 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5a4019ed70 con 0x7f5a401006e0 2023-09-09T23:13:33.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.780+0000 7f5a367fc700 1 -- 172.21.15.82:0/1807127184 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5a3c015020 con 0x7f5a401006e0 2023-09-09T23:13:33.779 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2023-09-09T23:13:33.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.782+0000 7f5a47828700 1 -- 172.21.15.82:0/1807127184 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5a2c0a06d0 msgr2=0x7f5a2c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:33.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.782+0000 7f5a47828700 1 --2- 172.21.15.82:0/1807127184 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5a2c0a06d0 0x7f5a2c0a2b90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5a40100540 tx=0x7f5a3000b410 comp rx=0 tx=0).stop 2023-09-09T23:13:33.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.782+0000 7f5a47828700 1 -- 172.21.15.82:0/1807127184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a401006e0 msgr2=0x7f5a40196970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:33.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.782+0000 7f5a47828700 1 --2- 172.21.15.82:0/1807127184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a401006e0 0x7f5a40196970 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c00b530 tx=0x7f5a3c00b840 comp rx=0 tx=0).stop 2023-09-09T23:13:33.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.783+0000 7f5a47828700 1 -- 172.21.15.82:0/1807127184 shutdown_connections 2023-09-09T23:13:33.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.783+0000 7f5a47828700 1 --2- 172.21.15.82:0/1807127184 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5a2c0a06d0 0x7f5a2c0a2b90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.783+0000 7f5a47828700 1 --2- 172.21.15.82:0/1807127184 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a40101080 0x7f5a4019d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.783+0000 7f5a47828700 1 --2- 172.21.15.82:0/1807127184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a400ff4f0 0x7f5a40196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.783+0000 7f5a47828700 1 --2- 172.21.15.82:0/1807127184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a401006e0 0x7f5a40196970 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.783+0000 7f5a47828700 1 -- 172.21.15.82:0/1807127184 >> 172.21.15.82:0/1807127184 conn(0x7f5a400faa60 msgr2=0x7f5a40067720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:33.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.783+0000 7f5a47828700 1 -- 172.21.15.82:0/1807127184 shutdown_connections 2023-09-09T23:13:33.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.783+0000 7f5a47828700 1 -- 172.21.15.82:0/1807127184 wait complete. 2023-09-09T23:13:33.794 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-09T23:13:33.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.916+0000 7fad52835700 1 -- 172.21.15.82:0/1483650790 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad4c101070 msgr2=0x7fad4c103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:33.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.916+0000 7fad52835700 1 --2- 172.21.15.82:0/1483650790 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad4c101070 0x7fad4c103460 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fad40009990 tx=0x7fad40009ca0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.916+0000 7fad52835700 1 -- 172.21.15.82:0/1483650790 shutdown_connections 2023-09-09T23:13:33.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.916+0000 7fad52835700 1 --2- 172.21.15.82:0/1483650790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad4c1062d0 0x7fad4c108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.916+0000 7fad52835700 1 --2- 172.21.15.82:0/1483650790 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad4c101070 0x7fad4c103460 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.916+0000 7fad52835700 1 --2- 172.21.15.82:0/1483650790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad4c1039a0 0x7fad4c105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.916+0000 7fad52835700 1 -- 172.21.15.82:0/1483650790 >> 172.21.15.82:0/1483650790 conn(0x7fad4c0faa20 msgr2=0x7fad4c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:33.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.916+0000 7fad52835700 1 -- 172.21.15.82:0/1483650790 shutdown_connections 2023-09-09T23:13:33.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.916+0000 7fad52835700 1 -- 172.21.15.82:0/1483650790 wait complete. 2023-09-09T23:13:33.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.916+0000 7fad52835700 1 Processor -- start 2023-09-09T23:13:33.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.916+0000 7fad52835700 1 -- start start 2023-09-09T23:13:33.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.917+0000 7fad52835700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad4c101070 0x7fad4c1962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:33.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.917+0000 7fad52835700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad4c1039a0 0x7fad4c1967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:33.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.917+0000 7fad52835700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad4c1062d0 0x7fad4c19d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:33.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.917+0000 7fad52835700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c19de20 con 0x7fad4c101070 2023-09-09T23:13:33.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.917+0000 7fad52835700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c19df60 con 0x7fad4c1039a0 2023-09-09T23:13:33.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.917+0000 7fad52835700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c19e0f0 con 0x7fad4c1062d0 2023-09-09T23:13:33.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.917+0000 7fad4bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad4c101070 0x7fad4c1962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:33.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.917+0000 7fad4bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad4c101070 0x7fad4c1962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55682/0 (socket says 172.21.15.82:55682) 2023-09-09T23:13:33.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.917+0000 7fad4bfff700 1 -- 172.21.15.82:0/3584651974 learned_addr learned my addr 172.21.15.82:0/3584651974 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:33.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.917+0000 7fad4bfff700 1 -- 172.21.15.82:0/3584651974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad4c1062d0 msgr2=0x7fad4c19d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:33.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.917+0000 7fad4b7fe700 1 --2- 172.21.15.82:0/3584651974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad4c1039a0 0x7fad4c1967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:33.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.917+0000 7fad50dd2700 1 --2- 172.21.15.82:0/3584651974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad4c1062d0 0x7fad4c19d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:33.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.917+0000 7fad4bfff700 1 --2- 172.21.15.82:0/3584651974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad4c1062d0 0x7fad4c19d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.917+0000 7fad4bfff700 1 -- 172.21.15.82:0/3584651974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad4c1039a0 msgr2=0x7fad4c1967e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:33.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.917+0000 7fad4bfff700 1 --2- 172.21.15.82:0/3584651974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad4c1039a0 0x7fad4c1967e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:33.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.917+0000 7fad4bfff700 1 -- 172.21.15.82:0/3584651974 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad400095f0 con 0x7fad4c101070 2023-09-09T23:13:33.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.917+0000 7fad4bfff700 1 --2- 172.21.15.82:0/3584651974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad4c101070 0x7fad4c1962a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fad40006010 tx=0x7fad4000f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:33.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.917+0000 7fad497fa700 1 -- 172.21.15.82:0/3584651974 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4001d070 con 0x7fad4c101070 2023-09-09T23:13:33.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.917+0000 7fad497fa700 1 -- 172.21.15.82:0/3584651974 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad4000fca0 con 0x7fad4c101070 2023-09-09T23:13:33.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.917+0000 7fad52835700 1 -- 172.21.15.82:0/3584651974 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad4c19e370 con 0x7fad4c101070 2023-09-09T23:13:33.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.917+0000 7fad497fa700 1 -- 172.21.15.82:0/3584651974 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad400176e0 con 0x7fad4c101070 2023-09-09T23:13:33.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.917+0000 7fad52835700 1 -- 172.21.15.82:0/3584651974 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad4c19e7f0 con 0x7fad4c101070 2023-09-09T23:13:33.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.918+0000 7fad52835700 1 -- 172.21.15.82:0/3584651974 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad4c0fc5a0 con 0x7fad4c101070 2023-09-09T23:13:33.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.919+0000 7fad497fa700 1 -- 172.21.15.82:0/3584651974 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fad40017840 con 0x7fad4c101070 2023-09-09T23:13:33.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.920+0000 7fad497fa700 1 --2- 172.21.15.82:0/3584651974 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fad340a0810 0x7fad340a2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:33.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.920+0000 7fad4b7fe700 1 --2- 172.21.15.82:0/3584651974 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fad340a0810 0x7fad340a2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:33.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.920+0000 7fad497fa700 1 -- 172.21.15.82:0/3584651974 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fad400cd370 con 0x7fad4c101070 2023-09-09T23:13:33.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.920+0000 7fad4b7fe700 1 --2- 172.21.15.82:0/3584651974 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fad340a0810 0x7fad340a2cd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fad3c006fd0 tx=0x7fad3c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:33.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:33.923+0000 7fad497fa700 1 -- 172.21.15.82:0/3584651974 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fad40096dd0 con 0x7fad4c101070 2023-09-09T23:13:34.156 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.157+0000 7fad52835700 1 -- 172.21.15.82:0/3584651974 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad4c10bea0 con 0x7fad4c101070 2023-09-09T23:13:34.156 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.158+0000 7fad497fa700 1 -- 172.21.15.82:0/3584651974 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11368 (secure 0 0 0) 0x7fad4009a3f0 con 0x7fad4c101070 2023-09-09T23:13:34.157 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:34.157 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2471432684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2471432684},{"type":"v1","addr":"172.21.15.82:6835","nonce":2471432684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3579387137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3579387137},{"type":"v1","addr":"172.21.15.82:6837","nonce":3579387137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-09T23:12:17.586036+0000","modified":"2023-09-09T23:12:25.285958+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24542,"mds_2":24523,"mds_3":24491,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14712":{"gid":14712,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/2127301345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2127301345},{"type":"v1","addr":"172.21.15.82:6839","nonce":2127301345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14724":{"gid":14724,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/4143604488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":4143604488},{"type":"v1","addr":"172.21.15.82:6841","nonce":4143604488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.170:6833/1931961940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1931961940},{"type":"v1","addr":"172.21.15.170:6833","nonce":1931961940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24491":{"gid":24491,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.98:6833/3855146755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3855146755},{"type":"v1","addr":"172.21.15.98:6833","nonce":3855146755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2313861511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2313861511},{"type":"v1","addr":"172.21.15.170:6835","nonce":2313861511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/4006360308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":4006360308},{"type":"v1","addr":"172.21.15.170:6837","nonce":4006360308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1356474828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1356474828},{"type":"v1","addr":"172.21.15.98:6835","nonce":1356474828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/3858369735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3858369735},{"type":"v1","addr":"172.21.15.170:6839","nonce":3858369735}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/2019449840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2019449840},{"type":"v1","addr":"172.21.15.98:6837","nonce":2019449840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2785086371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2785086371},{"type":"v1","addr":"172.21.15.98:6839","nonce":2785086371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-09T23:13:34.159 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.160+0000 7fad52835700 1 -- 172.21.15.82:0/3584651974 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fad340a0810 msgr2=0x7fad340a2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:34.159 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.160+0000 7fad52835700 1 --2- 172.21.15.82:0/3584651974 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fad340a0810 0x7fad340a2cd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fad3c006fd0 tx=0x7fad3c008040 comp rx=0 tx=0).stop 2023-09-09T23:13:34.159 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.160+0000 7fad52835700 1 -- 172.21.15.82:0/3584651974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad4c101070 msgr2=0x7fad4c1962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:34.159 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.160+0000 7fad52835700 1 --2- 172.21.15.82:0/3584651974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad4c101070 0x7fad4c1962a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fad40006010 tx=0x7fad4000f7b0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.160+0000 7fad52835700 1 -- 172.21.15.82:0/3584651974 shutdown_connections 2023-09-09T23:13:34.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.160+0000 7fad52835700 1 --2- 172.21.15.82:0/3584651974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad4c1062d0 0x7fad4c19d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.160+0000 7fad52835700 1 --2- 172.21.15.82:0/3584651974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad4c101070 0x7fad4c1962a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.160+0000 7fad52835700 1 --2- 172.21.15.82:0/3584651974 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fad340a0810 0x7fad340a2cd0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.160+0000 7fad52835700 1 --2- 172.21.15.82:0/3584651974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad4c1039a0 0x7fad4c1967e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.161 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.160+0000 7fad52835700 1 -- 172.21.15.82:0/3584651974 >> 172.21.15.82:0/3584651974 conn(0x7fad4c0faa20 msgr2=0x7fad4c101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:34.161 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.160+0000 7fad52835700 1 -- 172.21.15.82:0/3584651974 shutdown_connections 2023-09-09T23:13:34.161 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.160+0000 7fad52835700 1 -- 172.21.15.82:0/3584651974 wait complete. 2023-09-09T23:13:34.161 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 27 2023-09-09T23:13:34.171 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-09T23:13:34.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.294+0000 7f063d2f0700 1 -- 172.21.15.82:0/485575918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0638100720 msgr2=0x7f0638100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:34.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.294+0000 7f063d2f0700 1 --2- 172.21.15.82:0/485575918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0638100720 0x7f0638100b80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f062c0099c0 tx=0x7f062c009cd0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.294+0000 7f063d2f0700 1 -- 172.21.15.82:0/485575918 shutdown_connections 2023-09-09T23:13:34.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.294+0000 7f063d2f0700 1 --2- 172.21.15.82:0/485575918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06381010c0 0x7f0638107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.294+0000 7f063d2f0700 1 --2- 172.21.15.82:0/485575918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06380ff530 0x7f06380ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.294+0000 7f063d2f0700 1 --2- 172.21.15.82:0/485575918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0638100720 0x7f0638100b80 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.294+0000 7f063d2f0700 1 -- 172.21.15.82:0/485575918 >> 172.21.15.82:0/485575918 conn(0x7f06380faa80 msgr2=0x7f06380fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:34.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.294+0000 7f063d2f0700 1 -- 172.21.15.82:0/485575918 shutdown_connections 2023-09-09T23:13:34.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.294+0000 7f063d2f0700 1 -- 172.21.15.82:0/485575918 wait complete. 2023-09-09T23:13:34.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.295+0000 7f063d2f0700 1 Processor -- start 2023-09-09T23:13:34.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.295+0000 7f063d2f0700 1 -- start start 2023-09-09T23:13:34.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.295+0000 7f063d2f0700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06380ff530 0x7f06381951a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.295+0000 7f063d2f0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0638100720 0x7f06381956e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.295+0000 7f063d2f0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06381010c0 0x7f063819c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.295+0000 7f063d2f0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063819cd20 con 0x7f0638100720 2023-09-09T23:13:34.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.295+0000 7f063d2f0700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063819ce60 con 0x7f06380ff530 2023-09-09T23:13:34.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.295+0000 7f063d2f0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063819cff0 con 0x7f06381010c0 2023-09-09T23:13:34.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.295+0000 7f06367fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0638100720 0x7f06381956e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.295+0000 7f06367fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0638100720 0x7f06381956e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55698/0 (socket says 172.21.15.82:55698) 2023-09-09T23:13:34.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.295+0000 7f06367fc700 1 -- 172.21.15.82:0/841952155 learned_addr learned my addr 172.21.15.82:0/841952155 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:34.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.295+0000 7f0636ffd700 1 --2- 172.21.15.82:0/841952155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06380ff530 0x7f06381951a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.295+0000 7f06377fe700 1 --2- 172.21.15.82:0/841952155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06381010c0 0x7f063819c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.296+0000 7f06367fc700 1 -- 172.21.15.82:0/841952155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06381010c0 msgr2=0x7f063819c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:34.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.296+0000 7f06367fc700 1 --2- 172.21.15.82:0/841952155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06381010c0 0x7f063819c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.296+0000 7f06367fc700 1 -- 172.21.15.82:0/841952155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06380ff530 msgr2=0x7f06381951a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:34.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.296+0000 7f06367fc700 1 --2- 172.21.15.82:0/841952155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06380ff530 0x7f06381951a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.296+0000 7f06367fc700 1 -- 172.21.15.82:0/841952155 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062c0095f0 con 0x7f0638100720 2023-09-09T23:13:34.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.296+0000 7f06367fc700 1 --2- 172.21.15.82:0/841952155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0638100720 0x7f06381956e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f062c00c010 tx=0x7f062c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:34.301 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.298+0000 7f061ffff700 1 -- 172.21.15.82:0/841952155 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c01f070 con 0x7f0638100720 2023-09-09T23:13:34.301 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.299+0000 7f063d2f0700 1 -- 172.21.15.82:0/841952155 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06380689c0 con 0x7f0638100720 2023-09-09T23:13:34.301 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.299+0000 7f061ffff700 1 -- 172.21.15.82:0/841952155 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f062c019cd0 con 0x7f0638100720 2023-09-09T23:13:34.301 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.299+0000 7f061ffff700 1 -- 172.21.15.82:0/841952155 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c004230 con 0x7f0638100720 2023-09-09T23:13:34.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.299+0000 7f063d2f0700 1 -- 172.21.15.82:0/841952155 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0638068ec0 con 0x7f0638100720 2023-09-09T23:13:34.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.300+0000 7f063d2f0700 1 -- 172.21.15.82:0/841952155 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06380698b0 con 0x7f0638100720 2023-09-09T23:13:34.303 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.300+0000 7f061ffff700 1 -- 172.21.15.82:0/841952155 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f062c019810 con 0x7f0638100720 2023-09-09T23:13:34.303 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.301+0000 7f061ffff700 1 --2- 172.21.15.82:0/841952155 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f06200a0710 0x7f06200a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.303 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.301+0000 7f0636ffd700 1 --2- 172.21.15.82:0/841952155 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f06200a0710 0x7f06200a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.303 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.301+0000 7f061ffff700 1 -- 172.21.15.82:0/841952155 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f062c0cf030 con 0x7f0638100720 2023-09-09T23:13:34.303 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.301+0000 7f0636ffd700 1 --2- 172.21.15.82:0/841952155 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f06200a0710 0x7f06200a2bd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0638100580 tx=0x7f062800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:34.304 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.305+0000 7f061ffff700 1 -- 172.21.15.82:0/841952155 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f062c098b90 con 0x7f0638100720 2023-09-09T23:13:34.454 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:34 smithi082 ceph-mon[98694]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-09-09T23:13:34.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:34 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/41915599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-09T23:13:34.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:34 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1807127184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-09T23:13:34.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:34 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3584651974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:13:34.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.501+0000 7f063d2f0700 1 -- 172.21.15.82:0/841952155 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f063810af70 con 0x7f0638100720 2023-09-09T23:13:34.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.503+0000 7f061ffff700 1 -- 172.21.15.82:0/841952155 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22411 (secure 0 0 0) 0x7f062c09c1b0 con 0x7f0638100720 2023-09-09T23:13:34.502 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:34.502 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":83,"fsid":"3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da","created":"2023-09-09T23:05:14.222190+0000","modified":"2023-09-09T23:13:10.324596+0000","last_up_change":"2023-09-09T23:11:37.253759+0000","last_in_change":"2023-09-09T23:11:19.933812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-09T23:08:31.234958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-09T23:12:15.524887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-09T23:12:16.785403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1277eb17-a2e7-4e10-9832-f12a5e93ffde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6803","nonce":3197434929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6805","nonce":3197434929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6809","nonce":3197434929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6807","nonce":3197434929}]},"public_addr":"172.21.15.82:6803/3197434929","cluster_addr":"172.21.15.82:6805/3197434929","heartbeat_back_addr":"172.21.15.82:6809/3197434929","heartbeat_front_addr":"172.21.15.82:6807/3197434929","state":["exists","up"]},{"osd":1,"uuid":"f68723fd-0908-4e21-ae8e-1de1c3a9a65c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6801","nonce":3528641314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6803","nonce":3528641314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6807","nonce":3528641314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6805","nonce":3528641314}]},"public_addr":"172.21.15.98:6801/3528641314","cluster_addr":"172.21.15.98:6803/3528641314","heartbeat_back_addr":"172.21.15.98:6807/3528641314","heartbeat_front_addr":"172.21.15.98:6805/3528641314","state":["exists","up"]},{"osd":2,"uuid":"06f89c53-1419-4f52-acad-6e92916b476f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6801","nonce":4006414551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6803","nonce":4006414551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6807","nonce":4006414551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6805","nonce":4006414551}]},"public_addr":"172.21.15.170:6801/4006414551","cluster_addr":"172.21.15.170:6803/4006414551","heartbeat_back_addr":"172.21.15.170:6807/4006414551","heartbeat_front_addr":"172.21.15.170:6805/4006414551","state":["exists","up"]},{"osd":3,"uuid":"fedbd4c9-6852-42b5-8f3a-ba8d9bfe5d59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6811","nonce":3647629241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6813","nonce":3647629241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6817","nonce":3647629241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6815","nonce":3647629241}]},"public_addr":"172.21.15.82:6811/3647629241","cluster_addr":"172.21.15.82:6813/3647629241","heartbeat_back_addr":"172.21.15.82:6817/3647629241","heartbeat_front_addr":"172.21.15.82:6815/3647629241","state":["exists","up"]},{"osd":4,"uuid":"a71f3084-6919-4161-87bf-345f3355d8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6809","nonce":2507035643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6811","nonce":2507035643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6815","nonce":2507035643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6813","nonce":2507035643}]},"public_addr":"172.21.15.98:6809/2507035643","cluster_addr":"172.21.15.98:6811/2507035643","heartbeat_back_addr":"172.21.15.98:6815/2507035643","heartbeat_front_addr":"172.21.15.98:6813/2507035643","state":["exists","up"]},{"osd":5,"uuid":"f76058e9-a90e-476e-87f0-3c6723795e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6809","nonce":775005168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6811","nonce":775005168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6815","nonce":775005168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6813","nonce":775005168}]},"public_addr":"172.21.15.170:6809/775005168","cluster_addr":"172.21.15.170:6811/775005168","heartbeat_back_addr":"172.21.15.170:6815/775005168","heartbeat_front_addr":"172.21.15.170:6813/775005168","state":["exists","up"]},{"osd":6,"uuid":"25940be0-c1bd-4c72-95bf-fd8367aa2f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6819","nonce":3008405982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6821","nonce":3008405982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6825","nonce":3008405982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6823","nonce":3008405982}]},"public_addr":"172.21.15.82:6819/3008405982","cluster_addr":"172.21.15.82:6821/3008405982","heartbeat_back_addr":"172.21.15.82:6825/3008405982","heartbeat_front_addr":"172.21.15.82:6823/3008405982","state":["exists","up"]},{"osd":7,"uuid":"f651bb8c-0b25-42d5-b028-58563f4c27f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6817","nonce":3547615284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6819","nonce":3547615284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6823","nonce":3547615284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6821","nonce":3547615284}]},"public_addr":"172.21.15.98:6817/3547615284","cluster_addr":"172.21.15.98:6819/3547615284","heartbeat_back_addr":"172.21.15.98:6823/3547615284","heartbeat_front_addr":"172.21.15.98:6821/3547615284","state":["exists","up"]},{"osd":8,"uuid":"23b27c86-8c98-441e-b7ab-d57bc584e58f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6817","nonce":632038848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6819","nonce":632038848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6823","nonce":632038848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6821","nonce":632038848}]},"public_addr":"172.21.15.170:6817/632038848","cluster_addr":"172.21.15.170:6819/632038848","heartbeat_back_addr":"172.21.15.170:6823/632038848","heartbeat_front_addr":"172.21.15.170:6821/632038848","state":["exists","up"]},{"osd":9,"uuid":"383b8989-4922-4eee-9c77-f198258ac243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6827","nonce":3354242311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6829","nonce":3354242311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6833","nonce":3354242311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6831","nonce":3354242311}]},"public_addr":"172.21.15.82:6827/3354242311","cluster_addr":"172.21.15.82:6829/3354242311","heartbeat_back_addr":"172.21.15.82:6833/3354242311","heartbeat_front_addr":"172.21.15.82:6831/3354242311","state":["exists","up"]},{"osd":10,"uuid":"33a59389-7ced-42d5-bf54-2071317132d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6825","nonce":2069114959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6827","nonce":2069114959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6831","nonce":2069114959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6829","nonce":2069114959}]},"public_addr":"172.21.15.98:6825/2069114959","cluster_addr":"172.21.15.98:6827/2069114959","heartbeat_back_addr":"172.21.15.98:6831/2069114959","heartbeat_front_addr":"172.21.15.98:6829/2069114959","state":["exists","up"]},{"osd":11,"uuid":"b7a71f5c-85e2-4fec-be4b-95be0788b302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6825","nonce":146964512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6827","nonce":146964512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6831","nonce":146964512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6829","nonce":146964512}]},"public_addr":"172.21.15.170:6825/146964512","cluster_addr":"172.21.15.170:6827/146964512","heartbeat_back_addr":"172.21.15.170:6831/146964512","heartbeat_front_addr":"172.21.15.170:6829/146964512","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:07:47.563043+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:08.359260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:29.070374+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:49.897775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:11.434248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:31.538464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:52.377109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:13.452968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:33.806897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:54.083177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:14.980087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:35.690089+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4013747788":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6800/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/2823341464":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/75116806":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/1037625632":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/3142088817":"2023-09-10T23:05:46.550354+0000","172.21.15.82:6800/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/1679622963":"2023-09-10T23:05:46.550354+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-09T23:13:34.503 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.505+0000 7f063d2f0700 1 -- 172.21.15.82:0/841952155 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f06200a0710 msgr2=0x7f06200a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:34.504 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.505+0000 7f063d2f0700 1 --2- 172.21.15.82:0/841952155 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f06200a0710 0x7f06200a2bd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0638100580 tx=0x7f062800b410 comp rx=0 tx=0).stop 2023-09-09T23:13:34.504 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.505+0000 7f063d2f0700 1 -- 172.21.15.82:0/841952155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0638100720 msgr2=0x7f06381956e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:34.504 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.505+0000 7f063d2f0700 1 --2- 172.21.15.82:0/841952155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0638100720 0x7f06381956e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f062c00c010 tx=0x7f062c009580 comp rx=0 tx=0).stop 2023-09-09T23:13:34.504 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.505+0000 7f063d2f0700 1 -- 172.21.15.82:0/841952155 shutdown_connections 2023-09-09T23:13:34.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.505+0000 7f063d2f0700 1 --2- 172.21.15.82:0/841952155 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f06200a0710 0x7f06200a2bd0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.505+0000 7f063d2f0700 1 --2- 172.21.15.82:0/841952155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06380ff530 0x7f06381951a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.505+0000 7f063d2f0700 1 --2- 172.21.15.82:0/841952155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06381010c0 0x7f063819c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.505+0000 7f063d2f0700 1 --2- 172.21.15.82:0/841952155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0638100720 0x7f06381956e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.505+0000 7f063d2f0700 1 -- 172.21.15.82:0/841952155 >> 172.21.15.82:0/841952155 conn(0x7f06380faa80 msgr2=0x7f0638103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:34.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.505+0000 7f063d2f0700 1 -- 172.21.15.82:0/841952155 shutdown_connections 2023-09-09T23:13:34.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.505+0000 7f063d2f0700 1 -- 172.21.15.82:0/841952155 wait complete. 2023-09-09T23:13:34.516 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-09-09T23:12:17.586036+0000', 'modified': '2023-09-09T23:12:25.285958+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24566, 'mds_1': 24542, 'mds_2': 24523, 'mds_3': 24491, 'mds_4': 24460}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14712': {'gid': 14712, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.82:6839/2127301345', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6838', 'nonce': 2127301345}, {'type': 'v1', 'addr': '172.21.15.82:6839', 'nonce': 2127301345}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14724': {'gid': 14724, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.82:6841/4143604488', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6840', 'nonce': 4143604488}, {'type': 'v1', 'addr': '172.21.15.82:6841', 'nonce': 4143604488}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24460': {'gid': 24460, 'name': 'l', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.170:6833/1931961940', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6832', 'nonce': 1931961940}, {'type': 'v1', 'addr': '172.21.15.170:6833', 'nonce': 1931961940}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24491': {'gid': 24491, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.98:6833/3855146755', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6832', 'nonce': 3855146755}, {'type': 'v1', 'addr': '172.21.15.98:6833', 'nonce': 3855146755}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24493': {'gid': 24493, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6835/2313861511', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 2313861511}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 2313861511}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24502': {'gid': 24502, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6837/4006360308', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6836', 'nonce': 4006360308}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 4006360308}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24518': {'gid': 24518, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.98:6835/1356474828', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6834', 'nonce': 1356474828}, {'type': 'v1', 'addr': '172.21.15.98:6835', 'nonce': 1356474828}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24523': {'gid': 24523, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.170:6839/3858369735', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6838', 'nonce': 3858369735}, {'type': 'v1', 'addr': '172.21.15.170:6839', 'nonce': 3858369735}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24542': {'gid': 24542, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.98:6837/2019449840', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6836', 'nonce': 2019449840}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 2019449840}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24566': {'gid': 24566, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.98:6839/2785086371', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6838', 'nonce': 2785086371}, {'type': 'v1', 'addr': '172.21.15.98:6839', 'nonce': 2785086371}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-09T23:13:34.517 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-09-09T23:13:34.517 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2023-09-09T23:13:34.642 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.641+0000 7fa9d67d9700 1 -- 172.21.15.82:0/1282620324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9d00fe900 msgr2=0x7fa9d00fed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:34.642 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.641+0000 7fa9d67d9700 1 --2- 172.21.15.82:0/1282620324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9d00fe900 0x7fa9d00fed20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0009990 tx=0x7fa9c0009ca0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.642 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.641+0000 7fa9d67d9700 1 -- 172.21.15.82:0/1282620324 shutdown_connections 2023-09-09T23:13:34.643 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.641+0000 7fa9d67d9700 1 --2- 172.21.15.82:0/1282620324 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9d0102310 0x7fa9d0108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.643 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.641+0000 7fa9d67d9700 1 --2- 172.21.15.82:0/1282620324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9d00fe900 0x7fa9d00fed20 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.643 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.641+0000 7fa9d67d9700 1 --2- 172.21.15.82:0/1282620324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9d0101970 0x7fa9d0101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.643 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.641+0000 7fa9d67d9700 1 -- 172.21.15.82:0/1282620324 >> 172.21.15.82:0/1282620324 conn(0x7fa9d00fa1f0 msgr2=0x7fa9d00fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:34.643 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.641+0000 7fa9d67d9700 1 -- 172.21.15.82:0/1282620324 shutdown_connections 2023-09-09T23:13:34.644 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.641+0000 7fa9d67d9700 1 -- 172.21.15.82:0/1282620324 wait complete. 2023-09-09T23:13:34.644 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.642+0000 7fa9d67d9700 1 Processor -- start 2023-09-09T23:13:34.644 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.642+0000 7fa9d67d9700 1 -- start start 2023-09-09T23:13:34.644 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.642+0000 7fa9d67d9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9d00fe900 0x7fa9d01963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.644 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.642+0000 7fa9d67d9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9d0101970 0x7fa9d0196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.645 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.642+0000 7fa9d67d9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9d0102310 0x7fa9d019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.645 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.642+0000 7fa9d67d9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d019df40 con 0x7fa9d00fe900 2023-09-09T23:13:34.645 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.642+0000 7fa9d67d9700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d019e080 con 0x7fa9d0101970 2023-09-09T23:13:34.645 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.642+0000 7fa9d67d9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d019e210 con 0x7fa9d0102310 2023-09-09T23:13:34.645 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.642+0000 7fa9cffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9d00fe900 0x7fa9d01963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.646 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.642+0000 7fa9cffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9d00fe900 0x7fa9d01963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55710/0 (socket says 172.21.15.82:55710) 2023-09-09T23:13:34.646 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.642+0000 7fa9cffff700 1 -- 172.21.15.82:0/1318264391 learned_addr learned my addr 172.21.15.82:0/1318264391 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:34.646 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.642+0000 7fa9d4d76700 1 --2- 172.21.15.82:0/1318264391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9d0102310 0x7fa9d019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.646 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.642+0000 7fa9cf7fe700 1 --2- 172.21.15.82:0/1318264391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9d0101970 0x7fa9d0196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.646 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.643+0000 7fa9cffff700 1 -- 172.21.15.82:0/1318264391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9d0102310 msgr2=0x7fa9d019d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:34.646 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.643+0000 7fa9cffff700 1 --2- 172.21.15.82:0/1318264391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9d0102310 0x7fa9d019d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.647 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.643+0000 7fa9cffff700 1 -- 172.21.15.82:0/1318264391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9d0101970 msgr2=0x7fa9d0196900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:34.647 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.643+0000 7fa9cffff700 1 --2- 172.21.15.82:0/1318264391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9d0101970 0x7fa9d0196900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.647 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.643+0000 7fa9cffff700 1 -- 172.21.15.82:0/1318264391 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c00095f0 con 0x7fa9d00fe900 2023-09-09T23:13:34.647 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.643+0000 7fa9cffff700 1 --2- 172.21.15.82:0/1318264391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9d00fe900 0x7fa9d01963c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0000c00 tx=0x7fa9c000f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:34.647 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.643+0000 7fa9cd7fa700 1 -- 172.21.15.82:0/1318264391 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c001d070 con 0x7fa9d00fe900 2023-09-09T23:13:34.648 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.643+0000 7fa9cd7fa700 1 -- 172.21.15.82:0/1318264391 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9c000fca0 con 0x7fa9d00fe900 2023-09-09T23:13:34.648 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.643+0000 7fa9d67d9700 1 -- 172.21.15.82:0/1318264391 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d019e490 con 0x7fa9d00fe900 2023-09-09T23:13:34.648 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.643+0000 7fa9cd7fa700 1 -- 172.21.15.82:0/1318264391 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0017760 con 0x7fa9d00fe900 2023-09-09T23:13:34.648 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.643+0000 7fa9d67d9700 1 -- 172.21.15.82:0/1318264391 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d019e930 con 0x7fa9d00fe900 2023-09-09T23:13:34.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.645+0000 7fa9cd7fa700 1 -- 172.21.15.82:0/1318264391 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fa9c00178c0 con 0x7fa9d00fe900 2023-09-09T23:13:34.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.646+0000 7fa9cd7fa700 1 --2- 172.21.15.82:0/1318264391 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa9b80a06d0 0x7fa9b80a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.646+0000 7fa9cf7fe700 1 --2- 172.21.15.82:0/1318264391 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa9b80a06d0 0x7fa9b80a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.646+0000 7fa9cd7fa700 1 -- 172.21.15.82:0/1318264391 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fa9c00cd300 con 0x7fa9d00fe900 2023-09-09T23:13:34.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.646+0000 7fa9cf7fe700 1 --2- 172.21.15.82:0/1318264391 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa9b80a06d0 0x7fa9b80a2b90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4006fd0 tx=0x7fa9c4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:34.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.646+0000 7fa9e299fb80 1 Processor -- start 2023-09-09T23:13:34.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.646+0000 7fa9e299fb80 1 -- start start 2023-09-09T23:13:34.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.647+0000 7fa9e299fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610260c2bb0 0x5610260c2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.647+0000 7fa9e299fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5610260c3550 0x5610260c17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.647+0000 7fa9e299fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610260c1db0 0x5610260c2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.647+0000 7fa9e299fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56102601a6e0 con 0x5610260c1db0 2023-09-09T23:13:34.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.647+0000 7fa9e299fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561026018000 con 0x5610260c2bb0 2023-09-09T23:13:34.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.647+0000 7fa9e299fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610260138f0 con 0x5610260c3550 2023-09-09T23:13:34.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.647+0000 7fa9d4d76700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610260c1db0 0x5610260c2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.647+0000 7fa9d4d76700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610260c1db0 0x5610260c2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55722/0 (socket says 172.21.15.82:55722) 2023-09-09T23:13:34.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.647+0000 7fa9d4d76700 1 -- 172.21.15.82:0/3091162130 learned_addr learned my addr 172.21.15.82:0/3091162130 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:34.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.647+0000 7fa9cf7fe700 1 --2- 172.21.15.82:0/3091162130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5610260c3550 0x5610260c17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.659 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.647+0000 7fa9cffff700 1 --2- 172.21.15.82:0/3091162130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610260c2bb0 0x5610260c2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.659 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.647+0000 7fa9d4d76700 1 -- 172.21.15.82:0/3091162130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5610260c3550 msgr2=0x5610260c17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:34.659 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.647+0000 7fa9d4d76700 1 --2- 172.21.15.82:0/3091162130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5610260c3550 0x5610260c17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.659 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.647+0000 7fa9d4d76700 1 -- 172.21.15.82:0/3091162130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610260c2bb0 msgr2=0x5610260c2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:34.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.647+0000 7fa9d4d76700 1 --2- 172.21.15.82:0/3091162130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610260c2bb0 0x5610260c2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.647+0000 7fa9d4d76700 1 -- 172.21.15.82:0/3091162130 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561025e3fd90 con 0x5610260c1db0 2023-09-09T23:13:34.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.647+0000 7fa9cffff700 1 --2- 172.21.15.82:0/3091162130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610260c2bb0 0x5610260c2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:34.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.647+0000 7fa9cf7fe700 1 --2- 172.21.15.82:0/3091162130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5610260c3550 0x5610260c17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:34.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.647+0000 7fa9d4d76700 1 --2- 172.21.15.82:0/3091162130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610260c1db0 0x5610260c2260 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009f20 tx=0x7fa9c8009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53e547a0942447d server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:34.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.647+0000 7fa9b67fc700 1 -- 172.21.15.82:0/3091162130 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8011070 con 0x5610260c1db0 2023-09-09T23:13:34.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.647+0000 7fa9b67fc700 1 -- 172.21.15.82:0/3091162130 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9c8009a50 con 0x5610260c1db0 2023-09-09T23:13:34.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.647+0000 7fa9b67fc700 1 -- 172.21.15.82:0/3091162130 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c800a400 con 0x5610260c1db0 2023-09-09T23:13:34.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.648+0000 7fa9e299fb80 1 -- 172.21.15.82:0/3091162130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610260c1db0 msgr2=0x5610260c2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:34.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.648+0000 7fa9e299fb80 1 --2- 172.21.15.82:0/3091162130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610260c1db0 0x5610260c2260 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009f20 tx=0x7fa9c8009f50 comp rx=0 tx=0).stop 2023-09-09T23:13:34.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.648+0000 7fa9e299fb80 1 -- 172.21.15.82:0/3091162130 shutdown_connections 2023-09-09T23:13:34.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.648+0000 7fa9e299fb80 1 --2- 172.21.15.82:0/3091162130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610260c2bb0 0x5610260c2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.648+0000 7fa9e299fb80 1 --2- 172.21.15.82:0/3091162130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610260c1db0 0x5610260c2260 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.648+0000 7fa9e299fb80 1 --2- 172.21.15.82:0/3091162130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5610260c3550 0x5610260c17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.648+0000 7fa9e299fb80 1 -- 172.21.15.82:0/3091162130 >> 172.21.15.82:0/3091162130 conn(0x561026036040 msgr2=0x5610260bc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:34.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.648+0000 7fa9e299fb80 1 -- 172.21.15.82:0/3091162130 shutdown_connections 2023-09-09T23:13:34.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.648+0000 7fa9e299fb80 1 -- 172.21.15.82:0/3091162130 wait complete. 2023-09-09T23:13:34.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.648+0000 7fa9e299fb80 1 -- start start 2023-09-09T23:13:34.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.649+0000 7fa9e299fb80 1 Processor -- start 2023-09-09T23:13:34.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.649+0000 7fa9e299fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5610260c1db0 0x5610260cb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.649+0000 7fa9e299fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610260c2bb0 0x5610260cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.649+0000 7fa9e299fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610260c3550 0x5610260d2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.649+0000 7fa9e299fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561025fb1b20 con 0x5610260c2bb0 2023-09-09T23:13:34.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.649+0000 7fa9e299fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561025fb1820 con 0x5610260c3550 2023-09-09T23:13:34.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.649+0000 7fa9e299fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561025e958d0 con 0x5610260c1db0 2023-09-09T23:13:34.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.649+0000 7fa9e299fb80 10 client.0 _ll_register_callbacks cb 0x561025efba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:34.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.649+0000 7fa9cf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610260c2bb0 0x5610260cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.649+0000 7fa9cf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610260c2bb0 0x5610260cbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55724/0 (socket says 172.21.15.82:55724) 2023-09-09T23:13:34.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.649+0000 7fa9cf7fe700 1 -- 172.21.15.82:0/4142322488 learned_addr learned my addr 172.21.15.82:0/4142322488 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:34.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.649+0000 7fa9d4d76700 1 --2- 172.21.15.82:0/4142322488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610260c3550 0x5610260d2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.649+0000 7fa9cffff700 1 --2- 172.21.15.82:0/4142322488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5610260c1db0 0x5610260cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.649+0000 7fa9cf7fe700 1 -- 172.21.15.82:0/4142322488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5610260c1db0 msgr2=0x5610260cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:34.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.649+0000 7fa9cf7fe700 1 --2- 172.21.15.82:0/4142322488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5610260c1db0 0x5610260cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.649+0000 7fa9cf7fe700 1 -- 172.21.15.82:0/4142322488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610260c3550 msgr2=0x5610260d2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:34.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.649+0000 7fa9cf7fe700 1 --2- 172.21.15.82:0/4142322488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610260c3550 0x5610260d2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.649+0000 7fa9cf7fe700 1 -- 172.21.15.82:0/4142322488 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9c8009bd0 con 0x5610260c2bb0 2023-09-09T23:13:34.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.650+0000 7fa9cf7fe700 1 --2- 172.21.15.82:0/4142322488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610260c2bb0 0x5610260cbac0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa9c400de80 tx=0x7fa9c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15656cd916396b5 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:34.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.650+0000 7fa9af7fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:34.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.650+0000 7fa9af7fe700 1 -- 172.21.15.82:0/4142322488 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c400deb0 con 0x5610260c2bb0 2023-09-09T23:13:34.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.650+0000 7fa9af7fe700 1 -- 172.21.15.82:0/4142322488 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fa9c4020420 con 0x5610260c2bb0 2023-09-09T23:13:34.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.650+0000 7fa9e299fb80 1 -- 172.21.15.82:0/4142322488 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561026028850 con 0x5610260c2bb0 2023-09-09T23:13:34.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.650+0000 7fa9af7fe700 1 -- 172.21.15.82:0/4142322488 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9c4014e60 con 0x5610260c2bb0 2023-09-09T23:13:34.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.650+0000 7fa9e299fb80 10 client.15342 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:34.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.650+0000 7fa9e299fb80 1 -- 172.21.15.82:0/4142322488 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561025e28c10 con 0x5610260c2bb0 2023-09-09T23:13:34.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.650+0000 7fa9af7fe700 1 -- 172.21.15.82:0/4142322488 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9c402f020 con 0x5610260c2bb0 2023-09-09T23:13:34.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.650+0000 7fa9af7fe700 1 -- 172.21.15.82:0/4142322488 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa9c40296d0 con 0x5610260c2bb0 2023-09-09T23:13:34.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.650+0000 7fa9e299fb80 10 client.15342 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:34.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.650+0000 7fa9e299fb80 10 client.15342 resolve_mds: resolved name 'l' to daemon mds.l 2023-09-09T23:13:34.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.650+0000 7fa9e299fb80 20 client.15342 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:34.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.650+0000 7fa9e299fb80 1 --2- 172.21.15.82:0/4142322488 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x5610260d71b0 0x5610260d5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.650+0000 7fa9e299fb80 4 client.15342 mds_command: new command op to 24460 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:13:34.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.650+0000 7fa9e299fb80 1 -- 172.21.15.82:0/4142322488 --> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56102600c8d0 con 0x5610260d71b0 2023-09-09T23:13:34.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.651+0000 7fa9cffff700 1 --2- 172.21.15.82:0/4142322488 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x5610260d71b0 0x5610260d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.651+0000 7fa9cffff700 1 --2- 172.21.15.82:0/4142322488 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x5610260d71b0 0x5610260d5bc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fe1ffaf9d2b05112 server_cookie=a37f0171c0763434 in_seq=0 out_seq=0 2023-09-09T23:13:34.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.651+0000 7fa9af7fe700 10 client.15342 ms_handle_connect on v2:172.21.15.170:6832/1931961940 2023-09-09T23:13:34.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.652+0000 7fa9af7fe700 1 -- 172.21.15.82:0/4142322488 <== mds.4 v2:172.21.15.170:6832/1931961940 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56102600c8d0 con 0x5610260d71b0 2023-09-09T23:13:34.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.652+0000 7fa9af7fe700 10 client.15342 handle_command_reply: tid=0 2023-09-09T23:13:34.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.652+0000 7fa9e299fb80 1 client.15342 shutdown 2023-09-09T23:13:34.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.652+0000 7fa9e299fb80 1 -- 172.21.15.82:0/4142322488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610260c2bb0 msgr2=0x5610260cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:34.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.652+0000 7fa9e299fb80 1 --2- 172.21.15.82:0/4142322488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610260c2bb0 0x5610260cbac0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa9c400de80 tx=0x7fa9c4004750 comp rx=0 tx=0).stop 2023-09-09T23:13:34.672 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.652+0000 7fa9e299fb80 1 -- 172.21.15.82:0/4142322488 shutdown_connections 2023-09-09T23:13:34.672 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.652+0000 7fa9e299fb80 1 --2- 172.21.15.82:0/4142322488 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x5610260d71b0 0x5610260d5bc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.672 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.652+0000 7fa9e299fb80 1 --2- 172.21.15.82:0/4142322488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610260c3550 0x5610260d2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.672 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.652+0000 7fa9af7fe700 0 client.15342 ms_handle_reset on v2:172.21.15.170:6832/1931961940 2023-09-09T23:13:34.672 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.652+0000 7fa9e299fb80 1 --2- 172.21.15.82:0/4142322488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5610260c1db0 0x5610260cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.652+0000 7fa9e299fb80 1 --2- 172.21.15.82:0/4142322488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610260c2bb0 0x5610260cbac0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.652+0000 7fa9e299fb80 1 -- 172.21.15.82:0/4142322488 >> 172.21.15.82:0/4142322488 conn(0x561026036040 msgr2=0x5610260bbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:34.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.652+0000 7fa9e299fb80 1 -- 172.21.15.82:0/4142322488 shutdown_connections 2023-09-09T23:13:34.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.652+0000 7fa9e299fb80 1 -- 172.21.15.82:0/4142322488 wait complete. 2023-09-09T23:13:34.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.652+0000 7fa9e299fb80 20 client.15342 trim_cache size 0 max 16384 2023-09-09T23:13:34.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.669+0000 7fa9e299fb80 1 Processor -- start 2023-09-09T23:13:34.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.669+0000 7fa9e299fb80 1 -- start start 2023-09-09T23:13:34.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.670+0000 7fa9e299fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610260c1db0 0x5610260c2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.670+0000 7fa9e299fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5610260c2bb0 0x5610260d2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.670+0000 7fa9e299fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610260c3550 0x5610260d3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.670+0000 7fa9e299fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561025fdcf70 con 0x5610260c1db0 2023-09-09T23:13:34.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.670+0000 7fa9e299fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561025fcfed0 con 0x5610260c3550 2023-09-09T23:13:34.681 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.670+0000 7fa9e299fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610260ca0d0 con 0x5610260c2bb0 2023-09-09T23:13:34.681 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.670+0000 7fa9cffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610260c1db0 0x5610260c2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.681 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.670+0000 7fa9cffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610260c1db0 0x5610260c2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55732/0 (socket says 172.21.15.82:55732) 2023-09-09T23:13:34.681 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.670+0000 7fa9cffff700 1 -- 172.21.15.82:0/292334977 learned_addr learned my addr 172.21.15.82:0/292334977 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:34.681 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.670+0000 7fa9d4d76700 1 --2- 172.21.15.82:0/292334977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610260c3550 0x5610260d3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.670+0000 7fa9cf7fe700 1 --2- 172.21.15.82:0/292334977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5610260c2bb0 0x5610260d2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.670+0000 7fa9cffff700 1 -- 172.21.15.82:0/292334977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5610260c2bb0 msgr2=0x5610260d2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:34.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.670+0000 7fa9cffff700 1 --2- 172.21.15.82:0/292334977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5610260c2bb0 0x5610260d2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.670+0000 7fa9cffff700 1 -- 172.21.15.82:0/292334977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610260c3550 msgr2=0x5610260d3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:34.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.670+0000 7fa9cffff700 1 --2- 172.21.15.82:0/292334977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610260c3550 0x5610260d3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.683 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.670+0000 7fa9cffff700 1 -- 172.21.15.82:0/292334977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610260dfde0 con 0x5610260c1db0 2023-09-09T23:13:34.683 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.670+0000 7fa9d4d76700 1 --2- 172.21.15.82:0/292334977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610260c3550 0x5610260d3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:34.683 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.670+0000 7fa9cf7fe700 1 --2- 172.21.15.82:0/292334977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5610260c2bb0 0x5610260d2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:34.683 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.670+0000 7fa9cffff700 1 --2- 172.21.15.82:0/292334977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610260c1db0 0x5610260c2190 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0096fd0 tx=0x7fa9c0022d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505b455ceee1fce8 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:34.683 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.670+0000 7fa9b6ffd700 1 -- 172.21.15.82:0/292334977 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c00b5020 con 0x5610260c1db0 2023-09-09T23:13:34.684 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.670+0000 7fa9b6ffd700 1 -- 172.21.15.82:0/292334977 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9c009e950 con 0x5610260c1db0 2023-09-09T23:13:34.684 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.670+0000 7fa9b6ffd700 1 -- 172.21.15.82:0/292334977 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c009a270 con 0x5610260c1db0 2023-09-09T23:13:34.684 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.671+0000 7fa9e299fb80 1 -- 172.21.15.82:0/292334977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610260c1db0 msgr2=0x5610260c2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:34.684 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.671+0000 7fa9e299fb80 1 --2- 172.21.15.82:0/292334977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610260c1db0 0x5610260c2190 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0096fd0 tx=0x7fa9c0022d30 comp rx=0 tx=0).stop 2023-09-09T23:13:34.684 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.671+0000 7fa9e299fb80 1 -- 172.21.15.82:0/292334977 shutdown_connections 2023-09-09T23:13:34.685 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.671+0000 7fa9e299fb80 1 --2- 172.21.15.82:0/292334977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610260c3550 0x5610260d3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.685 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.671+0000 7fa9e299fb80 1 --2- 172.21.15.82:0/292334977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610260c1db0 0x5610260c2190 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.685 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.671+0000 7fa9e299fb80 1 --2- 172.21.15.82:0/292334977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5610260c2bb0 0x5610260d2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.685 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.671+0000 7fa9e299fb80 1 -- 172.21.15.82:0/292334977 >> 172.21.15.82:0/292334977 conn(0x561026036040 msgr2=0x5610260bd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:34.685 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.671+0000 7fa9e299fb80 1 -- 172.21.15.82:0/292334977 shutdown_connections 2023-09-09T23:13:34.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.671+0000 7fa9e299fb80 1 -- 172.21.15.82:0/292334977 wait complete. 2023-09-09T23:13:34.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.671+0000 7fa9e299fb80 1 -- start start 2023-09-09T23:13:34.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.671+0000 7fa9e299fb80 1 Processor -- start 2023-09-09T23:13:34.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.672+0000 7fa9e299fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610260c1db0 0x5610260cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.672+0000 7fa9e299fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610260c2bb0 0x5610260ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.672+0000 7fa9e299fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5610260c3550 0x56102612aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.672+0000 7fa9e299fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56102612b0e0 con 0x5610260c1db0 2023-09-09T23:13:34.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.672+0000 7fa9e299fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56102612b220 con 0x5610260c2bb0 2023-09-09T23:13:34.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.672+0000 7fa9e299fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56102612b390 con 0x5610260c3550 2023-09-09T23:13:34.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.672+0000 7fa9e299fb80 10 client.0 _ll_register_callbacks cb 0x7fa9c400d7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:34.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.672+0000 7fa9cffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610260c1db0 0x5610260cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.672+0000 7fa9cffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610260c1db0 0x5610260cdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55738/0 (socket says 172.21.15.82:55738) 2023-09-09T23:13:34.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.672+0000 7fa9cffff700 1 -- 172.21.15.82:0/3713766549 learned_addr learned my addr 172.21.15.82:0/3713766549 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:34.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.672+0000 7fa9cf7fe700 1 --2- 172.21.15.82:0/3713766549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610260c2bb0 0x5610260ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.672+0000 7fa9d4d76700 1 --2- 172.21.15.82:0/3713766549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5610260c3550 0x56102612aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.672+0000 7fa9cffff700 1 -- 172.21.15.82:0/3713766549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5610260c3550 msgr2=0x56102612aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:34.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.672+0000 7fa9cffff700 1 --2- 172.21.15.82:0/3713766549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5610260c3550 0x56102612aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.672+0000 7fa9cffff700 1 -- 172.21.15.82:0/3713766549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610260c2bb0 msgr2=0x5610260ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:34.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.672+0000 7fa9cffff700 1 --2- 172.21.15.82:0/3713766549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610260c2bb0 0x5610260ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.672+0000 7fa9cffff700 1 -- 172.21.15.82:0/3713766549 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d019e930 con 0x5610260c1db0 2023-09-09T23:13:34.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.672+0000 7fa9cffff700 1 --2- 172.21.15.82:0/3713766549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610260c1db0 0x5610260cdd30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa9c009aab0 tx=0x7fa9c0005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e87476d0ca80f1f server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:34.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.672+0000 7fa9af7fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:34.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.672+0000 7fa9af7fe700 1 -- 172.21.15.82:0/3713766549 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c00b5040 con 0x5610260c1db0 2023-09-09T23:13:34.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.672+0000 7fa9af7fe700 1 -- 172.21.15.82:0/3713766549 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fa9c009afd0 con 0x5610260c1db0 2023-09-09T23:13:34.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.672+0000 7fa9e299fb80 1 -- 172.21.15.82:0/3713766549 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561025f6b4a0 con 0x5610260c1db0 2023-09-09T23:13:34.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.672+0000 7fa9af7fe700 1 -- 172.21.15.82:0/3713766549 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9c00a2b00 con 0x5610260c1db0 2023-09-09T23:13:34.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.672+0000 7fa9e299fb80 10 client.15348 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:34.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.672+0000 7fa9e299fb80 1 -- 172.21.15.82:0/3713766549 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561025e66200 con 0x5610260c1db0 2023-09-09T23:13:34.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.672+0000 7fa9af7fe700 1 -- 172.21.15.82:0/3713766549 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561025f6b4a0 con 0x5610260c1db0 2023-09-09T23:13:34.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.673+0000 7fa9af7fe700 1 -- 172.21.15.82:0/3713766549 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa9c00b8880 con 0x5610260c1db0 2023-09-09T23:13:34.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.673+0000 7fa9e299fb80 10 client.15348 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:34.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.673+0000 7fa9e299fb80 10 client.15348 resolve_mds: resolved name 'l' to daemon mds.l 2023-09-09T23:13:34.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.673+0000 7fa9e299fb80 20 client.15348 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:34.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.673+0000 7fa9e299fb80 1 --2- 172.21.15.82:0/3713766549 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x5610260d71b0 0x56102612c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.673+0000 7fa9e299fb80 4 client.15348 mds_command: new command op to 24460 tid=0[{"prefix": "status"}] 2023-09-09T23:13:34.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.673+0000 7fa9e299fb80 1 -- 172.21.15.82:0/3713766549 --> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561025fcd060 con 0x5610260d71b0 2023-09-09T23:13:34.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.673+0000 7fa9cf7fe700 1 --2- 172.21.15.82:0/3713766549 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x5610260d71b0 0x56102612c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.673+0000 7fa9af7fe700 10 client.15348 ms_handle_connect on v2:172.21.15.170:6832/1931961940 2023-09-09T23:13:34.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.673+0000 7fa9cf7fe700 1 --2- 172.21.15.82:0/3713766549 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x5610260d71b0 0x56102612c600 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f714b60a05278c78 server_cookie=422b8c0c6941e9f0 in_seq=0 out_seq=0 2023-09-09T23:13:34.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.673+0000 7fa9af7fe700 1 -- 172.21.15.82:0/3713766549 <== mds.4 v2:172.21.15.170:6832/1931961940 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x561025fcd060 con 0x5610260d71b0 2023-09-09T23:13:34.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.674+0000 7fa9af7fe700 10 client.15348 handle_command_reply: tid=0 2023-09-09T23:13:34.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.674+0000 7fa9e299fb80 1 client.15348 shutdown 2023-09-09T23:13:34.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.674+0000 7fa9e299fb80 1 -- 172.21.15.82:0/3713766549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610260c1db0 msgr2=0x5610260cdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:34.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.674+0000 7fa9e299fb80 1 --2- 172.21.15.82:0/3713766549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610260c1db0 0x5610260cdd30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa9c009aab0 tx=0x7fa9c0005c60 comp rx=0 tx=0).stop 2023-09-09T23:13:34.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.674+0000 7fa9e299fb80 1 -- 172.21.15.82:0/3713766549 shutdown_connections 2023-09-09T23:13:34.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.674+0000 7fa9e299fb80 1 --2- 172.21.15.82:0/3713766549 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x5610260d71b0 0x56102612c600 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.674+0000 7fa9e299fb80 1 --2- 172.21.15.82:0/3713766549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5610260c3550 0x56102612aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.674+0000 7fa9af7fe700 0 client.15348 ms_handle_reset on v2:172.21.15.170:6832/1931961940 2023-09-09T23:13:34.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.674+0000 7fa9e299fb80 1 --2- 172.21.15.82:0/3713766549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5610260c1db0 0x5610260cdd30 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.674+0000 7fa9e299fb80 1 --2- 172.21.15.82:0/3713766549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610260c2bb0 0x5610260ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.674+0000 7fa9e299fb80 1 -- 172.21.15.82:0/3713766549 >> 172.21.15.82:0/3713766549 conn(0x561026036040 msgr2=0x5610260c2190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:34.696 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.674+0000 7fa9e299fb80 1 -- 172.21.15.82:0/3713766549 shutdown_connections 2023-09-09T23:13:34.696 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.674+0000 7fa9e299fb80 1 -- 172.21.15.82:0/3713766549 wait complete. 2023-09-09T23:13:34.696 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.674+0000 7fa9e299fb80 20 client.15348 trim_cache size 0 max 16384 2023-09-09T23:13:34.697 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:13:34.697 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da", 2023-09-09T23:13:34.697 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 4, 2023-09-09T23:13:34.697 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24460, 2023-09-09T23:13:34.697 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2023-09-09T23:13:34.698 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2023-09-09T23:13:34.698 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2023-09-09T23:13:34.698 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 70.407173204000003, 2023-09-09T23:13:34.698 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 27, 2023-09-09T23:13:34.698 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 83, 2023-09-09T23:13:34.699 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 81, 2023-09-09T23:13:34.699 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 106.925227161 2023-09-09T23:13:34.699 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:13:34.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.674+0000 7fa9d67d9700 1 -- 172.21.15.82:0/1318264391 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa9b80a06d0 msgr2=0x7fa9b80a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:34.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.674+0000 7fa9d67d9700 1 --2- 172.21.15.82:0/1318264391 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa9b80a06d0 0x7fa9b80a2b90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4006fd0 tx=0x7fa9c4008040 comp rx=0 tx=0).stop 2023-09-09T23:13:34.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.674+0000 7fa9d67d9700 1 -- 172.21.15.82:0/1318264391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9d00fe900 msgr2=0x7fa9d01963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:34.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.674+0000 7fa9d67d9700 1 --2- 172.21.15.82:0/1318264391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9d00fe900 0x7fa9d01963c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0000c00 tx=0x7fa9c000f7b0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.674+0000 7fa9d67d9700 1 -- 172.21.15.82:0/1318264391 shutdown_connections 2023-09-09T23:13:34.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.674+0000 7fa9d67d9700 1 --2- 172.21.15.82:0/1318264391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9d0102310 0x7fa9d019d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.674+0000 7fa9d67d9700 1 --2- 172.21.15.82:0/1318264391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9d00fe900 0x7fa9d01963c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.674+0000 7fa9d67d9700 1 --2- 172.21.15.82:0/1318264391 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa9b80a06d0 0x7fa9b80a2b90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.674+0000 7fa9d67d9700 1 --2- 172.21.15.82:0/1318264391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9d0101970 0x7fa9d0196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.674+0000 7fa9d67d9700 1 -- 172.21.15.82:0/1318264391 >> 172.21.15.82:0/1318264391 conn(0x7fa9d00fa1f0 msgr2=0x7fa9d00fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:34.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.674+0000 7fa9d67d9700 1 -- 172.21.15.82:0/1318264391 shutdown_connections 2023-09-09T23:13:34.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.674+0000 7fa9d67d9700 1 -- 172.21.15.82:0/1318264391 wait complete. 2023-09-09T23:13:34.703 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:34 smithi170 ceph-mon[102832]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-09-09T23:13:34.703 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:34 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/41915599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-09T23:13:34.703 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:34 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1807127184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-09T23:13:34.704 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:34 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3584651974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:13:34.704 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2023-09-09T23:13:34.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:34 smithi098 ceph-mon[103124]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-09-09T23:13:34.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:34 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/41915599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-09T23:13:34.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:34 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1807127184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-09T23:13:34.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:34 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3584651974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:13:34.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.828+0000 7f2b3b571700 1 -- 172.21.15.82:0/1950774677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b34100830 msgr2=0x7f2b341070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:34.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.828+0000 7f2b3b571700 1 --2- 172.21.15.82:0/1950774677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b34100830 0x7f2b341070e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c00b0d0 tx=0x7f2b2c00b3e0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.828+0000 7f2b3b571700 1 -- 172.21.15.82:0/1950774677 shutdown_connections 2023-09-09T23:13:34.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.828+0000 7f2b3b571700 1 --2- 172.21.15.82:0/1950774677 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b340feca0 0x7f2b340ff0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.828+0000 7f2b3b571700 1 --2- 172.21.15.82:0/1950774677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b34100830 0x7f2b341070e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.828+0000 7f2b3b571700 1 --2- 172.21.15.82:0/1950774677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b340ffe90 0x7f2b341002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.828+0000 7f2b3b571700 1 -- 172.21.15.82:0/1950774677 >> 172.21.15.82:0/1950774677 conn(0x7f2b340fa1f0 msgr2=0x7f2b340fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:34.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.828+0000 7f2b3b571700 1 -- 172.21.15.82:0/1950774677 shutdown_connections 2023-09-09T23:13:34.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.828+0000 7f2b3b571700 1 -- 172.21.15.82:0/1950774677 wait complete. 2023-09-09T23:13:34.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.829+0000 7f2b3b571700 1 Processor -- start 2023-09-09T23:13:34.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.829+0000 7f2b3b571700 1 -- start start 2023-09-09T23:13:34.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.829+0000 7f2b3b571700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b340feca0 0x7f2b340fc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.829+0000 7f2b3b571700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b340ffe90 0x7f2b3419b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.829+0000 7f2b3b571700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b34100830 0x7f2b3419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.829+0000 7f2b3b571700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3419deb0 con 0x7f2b34100830 2023-09-09T23:13:34.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.829+0000 7f2b3b571700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3419e020 con 0x7f2b340feca0 2023-09-09T23:13:34.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.829+0000 7f2b3b571700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3419e1e0 con 0x7f2b340ffe90 2023-09-09T23:13:34.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.829+0000 7f2b39b0e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b34100830 0x7f2b3419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.829+0000 7f2b39b0e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b34100830 0x7f2b3419d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55756/0 (socket says 172.21.15.82:55756) 2023-09-09T23:13:34.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.829+0000 7f2b39b0e700 1 -- 172.21.15.82:0/3936952203 learned_addr learned my addr 172.21.15.82:0/3936952203 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:34.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.829+0000 7f2b3930d700 1 --2- 172.21.15.82:0/3936952203 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b340feca0 0x7f2b340fc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.829+0000 7f2b38b0c700 1 --2- 172.21.15.82:0/3936952203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b340ffe90 0x7f2b3419b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.829+0000 7f2b39b0e700 1 -- 172.21.15.82:0/3936952203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b340ffe90 msgr2=0x7f2b3419b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:34.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.829+0000 7f2b39b0e700 1 --2- 172.21.15.82:0/3936952203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b340ffe90 0x7f2b3419b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.829+0000 7f2b39b0e700 1 -- 172.21.15.82:0/3936952203 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b340feca0 msgr2=0x7f2b340fc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:34.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.829+0000 7f2b39b0e700 1 --2- 172.21.15.82:0/3936952203 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b340feca0 0x7f2b340fc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.829+0000 7f2b39b0e700 1 -- 172.21.15.82:0/3936952203 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2c009d00 con 0x7f2b34100830 2023-09-09T23:13:34.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.829+0000 7f2b3930d700 1 --2- 172.21.15.82:0/3936952203 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b340feca0 0x7f2b340fc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:34.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.830+0000 7f2b39b0e700 1 --2- 172.21.15.82:0/3936952203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b34100830 0x7f2b3419d710 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c008f10 tx=0x7f2b2c008ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:34.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.830+0000 7f2b267fc700 1 -- 172.21.15.82:0/3936952203 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c00e040 con 0x7f2b34100830 2023-09-09T23:13:34.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.830+0000 7f2b3b571700 1 -- 172.21.15.82:0/3936952203 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b3419e460 con 0x7f2b34100830 2023-09-09T23:13:34.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.830+0000 7f2b267fc700 1 -- 172.21.15.82:0/3936952203 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b2c003c00 con 0x7f2b34100830 2023-09-09T23:13:34.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.830+0000 7f2b267fc700 1 -- 172.21.15.82:0/3936952203 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c01b7d0 con 0x7f2b34100830 2023-09-09T23:13:34.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.830+0000 7f2b3b571700 1 -- 172.21.15.82:0/3936952203 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b3419e890 con 0x7f2b34100830 2023-09-09T23:13:34.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.832+0000 7f2b267fc700 1 -- 172.21.15.82:0/3936952203 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f2b2c024410 con 0x7f2b34100830 2023-09-09T23:13:34.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.833+0000 7f2b267fc700 1 --2- 172.21.15.82:0/3936952203 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f2b200a06d0 0x7f2b200a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.833+0000 7f2b267fc700 1 -- 172.21.15.82:0/3936952203 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f2b2c0ccfe0 con 0x7f2b34100830 2023-09-09T23:13:34.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.833+0000 7f2b3930d700 1 --2- 172.21.15.82:0/3936952203 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f2b200a06d0 0x7f2b200a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.833+0000 7f2b3930d700 1 --2- 172.21.15.82:0/3936952203 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f2b200a06d0 0x7f2b200a2b90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2b340ffcf0 tx=0x7f2b28009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:34.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.834+0000 7f2b47737b80 1 Processor -- start 2023-09-09T23:13:34.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.834+0000 7f2b47737b80 1 -- start start 2023-09-09T23:13:34.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.834+0000 7f2b47737b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbb98e6020 0x55cbb98e6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.834+0000 7f2b47737b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbb98e6980 0x55cbb98e4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.834+0000 7f2b47737b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbb98e51e0 0x55cbb98e5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.834+0000 7f2b47737b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb98396e0 con 0x55cbb98e6980 2023-09-09T23:13:34.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.834+0000 7f2b47737b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb9837000 con 0x55cbb98e51e0 2023-09-09T23:13:34.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.834+0000 7f2b47737b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb98328f0 con 0x55cbb98e6020 2023-09-09T23:13:34.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.834+0000 7f2b38b0c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbb98e6980 0x55cbb98e4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.834+0000 7f2b38b0c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbb98e6980 0x55cbb98e4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55762/0 (socket says 172.21.15.82:55762) 2023-09-09T23:13:34.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.834+0000 7f2b38b0c700 1 -- 172.21.15.82:0/2827457057 learned_addr learned my addr 172.21.15.82:0/2827457057 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:34.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.835+0000 7f2b3930d700 1 --2- 172.21.15.82:0/2827457057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbb98e6020 0x55cbb98e6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.835+0000 7f2b39b0e700 1 --2- 172.21.15.82:0/2827457057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbb98e51e0 0x55cbb98e5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.835+0000 7f2b38b0c700 1 -- 172.21.15.82:0/2827457057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbb98e6020 msgr2=0x55cbb98e6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:34.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.835+0000 7f2b38b0c700 1 --2- 172.21.15.82:0/2827457057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbb98e6020 0x55cbb98e6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.835+0000 7f2b38b0c700 1 -- 172.21.15.82:0/2827457057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbb98e51e0 msgr2=0x55cbb98e5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:34.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.835+0000 7f2b38b0c700 1 --2- 172.21.15.82:0/2827457057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbb98e51e0 0x55cbb98e5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.835+0000 7f2b38b0c700 1 -- 172.21.15.82:0/2827457057 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbb965ed90 con 0x55cbb98e6980 2023-09-09T23:13:34.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.835+0000 7f2b3930d700 1 --2- 172.21.15.82:0/2827457057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbb98e6020 0x55cbb98e6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:34.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.835+0000 7f2b38b0c700 1 --2- 172.21.15.82:0/2827457057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbb98e6980 0x55cbb98e4c00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2b30005e20 tx=0x7f2b3000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73326ce19a4c9c17 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:34.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.835+0000 7f2b1f7fe700 1 -- 172.21.15.82:0/2827457057 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3000ee20 con 0x55cbb98e6980 2023-09-09T23:13:34.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.835+0000 7f2b1f7fe700 1 -- 172.21.15.82:0/2827457057 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b300108f0 con 0x55cbb98e6980 2023-09-09T23:13:34.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.835+0000 7f2b1f7fe700 1 -- 172.21.15.82:0/2827457057 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3001a930 con 0x55cbb98e6980 2023-09-09T23:13:34.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.836+0000 7f2b47737b80 1 -- 172.21.15.82:0/2827457057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbb98e6980 msgr2=0x55cbb98e4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:34.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.836+0000 7f2b47737b80 1 --2- 172.21.15.82:0/2827457057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbb98e6980 0x55cbb98e4c00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2b30005e20 tx=0x7f2b3000cf90 comp rx=0 tx=0).stop 2023-09-09T23:13:34.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.836+0000 7f2b47737b80 1 -- 172.21.15.82:0/2827457057 shutdown_connections 2023-09-09T23:13:34.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.836+0000 7f2b47737b80 1 --2- 172.21.15.82:0/2827457057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbb98e51e0 0x55cbb98e5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.836+0000 7f2b47737b80 1 --2- 172.21.15.82:0/2827457057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbb98e6020 0x55cbb98e6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.836+0000 7f2b47737b80 1 --2- 172.21.15.82:0/2827457057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbb98e6980 0x55cbb98e4c00 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.836+0000 7f2b47737b80 1 -- 172.21.15.82:0/2827457057 >> 172.21.15.82:0/2827457057 conn(0x55cbb9855040 msgr2=0x55cbb98db6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:34.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.836+0000 7f2b47737b80 1 -- 172.21.15.82:0/2827457057 shutdown_connections 2023-09-09T23:13:34.848 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.836+0000 7f2b47737b80 1 -- 172.21.15.82:0/2827457057 wait complete. 2023-09-09T23:13:34.848 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.836+0000 7f2b47737b80 1 -- start start 2023-09-09T23:13:34.848 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.836+0000 7f2b47737b80 1 Processor -- start 2023-09-09T23:13:34.848 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.836+0000 7f2b47737b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbb98e51e0 0x55cbb98eea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.849 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.836+0000 7f2b47737b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbb98e6020 0x55cbb98eef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.849 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.836+0000 7f2b47737b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbb98e6980 0x55cbb98f5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.849 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.836+0000 7f2b47737b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb97d0b20 con 0x55cbb98e6020 2023-09-09T23:13:34.849 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.836+0000 7f2b47737b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb97d0820 con 0x55cbb98e51e0 2023-09-09T23:13:34.849 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.836+0000 7f2b47737b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb96b48d0 con 0x55cbb98e6980 2023-09-09T23:13:34.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.837+0000 7f2b47737b80 10 client.0 _ll_register_callbacks cb 0x55cbb971aa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:34.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.837+0000 7f2b38b0c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbb98e6020 0x55cbb98eef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.837+0000 7f2b38b0c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbb98e6020 0x55cbb98eef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55768/0 (socket says 172.21.15.82:55768) 2023-09-09T23:13:34.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.837+0000 7f2b38b0c700 1 -- 172.21.15.82:0/1126750613 learned_addr learned my addr 172.21.15.82:0/1126750613 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:34.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.837+0000 7f2b3930d700 1 --2- 172.21.15.82:0/1126750613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbb98e51e0 0x55cbb98eea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.837+0000 7f2b39b0e700 1 --2- 172.21.15.82:0/1126750613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbb98e6980 0x55cbb98f5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.837+0000 7f2b38b0c700 1 -- 172.21.15.82:0/1126750613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbb98e6980 msgr2=0x55cbb98f5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:34.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.837+0000 7f2b38b0c700 1 --2- 172.21.15.82:0/1126750613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbb98e6980 0x55cbb98f5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.837+0000 7f2b38b0c700 1 -- 172.21.15.82:0/1126750613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbb98e51e0 msgr2=0x55cbb98eea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:34.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.837+0000 7f2b38b0c700 1 --2- 172.21.15.82:0/1126750613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbb98e51e0 0x55cbb98eea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.837+0000 7f2b38b0c700 1 -- 172.21.15.82:0/1126750613 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b3000c910 con 0x55cbb98e6020 2023-09-09T23:13:34.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.837+0000 7f2b38b0c700 1 --2- 172.21.15.82:0/1126750613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbb98e6020 0x55cbb98eef50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2b30010c90 tx=0x7f2b30010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e004108fd5a9efe6 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:34.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.837+0000 7f2b1cff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:34.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.837+0000 7f2b1cff9700 1 -- 172.21.15.82:0/1126750613 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3001f070 con 0x55cbb98e6020 2023-09-09T23:13:34.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.837+0000 7f2b1cff9700 1 -- 172.21.15.82:0/1126750613 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f2b300106c0 con 0x55cbb98e6020 2023-09-09T23:13:34.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.837+0000 7f2b47737b80 1 -- 172.21.15.82:0/1126750613 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbb978a4a0 con 0x55cbb98e6020 2023-09-09T23:13:34.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.837+0000 7f2b1cff9700 1 -- 172.21.15.82:0/1126750613 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b30005040 con 0x55cbb98e6020 2023-09-09T23:13:34.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.837+0000 7f2b47737b80 10 client.15360 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:34.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.837+0000 7f2b47737b80 1 -- 172.21.15.82:0/1126750613 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cbb9647c10 con 0x55cbb98e6020 2023-09-09T23:13:34.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.837+0000 7f2b1cff9700 1 -- 172.21.15.82:0/1126750613 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b30032020 con 0x55cbb98e6020 2023-09-09T23:13:34.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.838+0000 7f2b1cff9700 1 -- 172.21.15.82:0/1126750613 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2b3001a6d0 con 0x55cbb98e6020 2023-09-09T23:13:34.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.838+0000 7f2b47737b80 10 client.15360 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:34.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.838+0000 7f2b47737b80 10 client.15360 resolve_mds: resolved name 'k' to daemon mds.k 2023-09-09T23:13:34.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.838+0000 7f2b47737b80 20 client.15360 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:34.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.838+0000 7f2b47737b80 1 --2- 172.21.15.82:0/1126750613 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55cbb98f9f70 0x55cbb98f8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.838+0000 7f2b47737b80 4 client.15360 mds_command: new command op to 24491 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:13:34.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.838+0000 7f2b47737b80 1 -- 172.21.15.82:0/1126750613 --> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cbb9847850 con 0x55cbb98f9f70 2023-09-09T23:13:34.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.838+0000 7f2b3930d700 1 --2- 172.21.15.82:0/1126750613 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55cbb98f9f70 0x55cbb98f8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.838+0000 7f2b1cff9700 10 client.15360 ms_handle_connect on v2:172.21.15.98:6832/3855146755 2023-09-09T23:13:34.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.838+0000 7f2b3930d700 1 --2- 172.21.15.82:0/1126750613 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55cbb98f9f70 0x55cbb98f8f40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a2556792a0b29b8d server_cookie=bd05ea7d2ab6cfdc in_seq=0 out_seq=0 2023-09-09T23:13:34.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.839+0000 7f2b1cff9700 1 -- 172.21.15.82:0/1126750613 <== mds.3 v2:172.21.15.98:6832/3855146755 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cbb9847850 con 0x55cbb98f9f70 2023-09-09T23:13:34.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.839+0000 7f2b1cff9700 10 client.15360 handle_command_reply: tid=0 2023-09-09T23:13:34.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.839+0000 7f2b47737b80 1 client.15360 shutdown 2023-09-09T23:13:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.839+0000 7f2b47737b80 1 -- 172.21.15.82:0/1126750613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbb98e6020 msgr2=0x55cbb98eef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.839+0000 7f2b47737b80 1 --2- 172.21.15.82:0/1126750613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbb98e6020 0x55cbb98eef50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2b30010c90 tx=0x7f2b30010d70 comp rx=0 tx=0).stop 2023-09-09T23:13:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.839+0000 7f2b47737b80 1 -- 172.21.15.82:0/1126750613 shutdown_connections 2023-09-09T23:13:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.839+0000 7f2b47737b80 1 --2- 172.21.15.82:0/1126750613 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55cbb98f9f70 0x55cbb98f8f40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.839+0000 7f2b47737b80 1 --2- 172.21.15.82:0/1126750613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbb98e51e0 0x55cbb98eea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.839+0000 7f2b1cff9700 0 client.15360 ms_handle_reset on v2:172.21.15.98:6832/3855146755 2023-09-09T23:13:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.839+0000 7f2b47737b80 1 --2- 172.21.15.82:0/1126750613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbb98e6980 0x55cbb98f5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.839+0000 7f2b47737b80 1 --2- 172.21.15.82:0/1126750613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbb98e6020 0x55cbb98eef50 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.839+0000 7f2b47737b80 1 -- 172.21.15.82:0/1126750613 >> 172.21.15.82:0/1126750613 conn(0x55cbb9855040 msgr2=0x55cbb98dad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.839+0000 7f2b47737b80 1 -- 172.21.15.82:0/1126750613 shutdown_connections 2023-09-09T23:13:34.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.839+0000 7f2b47737b80 1 -- 172.21.15.82:0/1126750613 wait complete. 2023-09-09T23:13:34.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.840+0000 7f2b47737b80 20 client.15360 trim_cache size 0 max 16384 2023-09-09T23:13:34.863 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.857+0000 7f2b47737b80 1 Processor -- start 2023-09-09T23:13:34.863 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.857+0000 7f2b47737b80 1 -- start start 2023-09-09T23:13:34.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.857+0000 7f2b47737b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbb98e51e0 0x55cbb98e55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.857+0000 7f2b47737b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbb98e6020 0x55cbb98f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.857+0000 7f2b47737b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbb98e6980 0x55cbb98ed6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.857+0000 7f2b47737b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb97fbf70 con 0x55cbb98e6980 2023-09-09T23:13:34.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.857+0000 7f2b47737b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb97eeed0 con 0x55cbb98e51e0 2023-09-09T23:13:34.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.857+0000 7f2b47737b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb97da4d0 con 0x55cbb98e6020 2023-09-09T23:13:34.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.857+0000 7f2b39b0e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbb98e6980 0x55cbb98ed6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.857+0000 7f2b39b0e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbb98e6980 0x55cbb98ed6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55778/0 (socket says 172.21.15.82:55778) 2023-09-09T23:13:34.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.857+0000 7f2b39b0e700 1 -- 172.21.15.82:0/2183750648 learned_addr learned my addr 172.21.15.82:0/2183750648 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:34.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.857+0000 7f2b38b0c700 1 --2- 172.21.15.82:0/2183750648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbb98e6020 0x55cbb98f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.866 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.857+0000 7f2b39b0e700 1 -- 172.21.15.82:0/2183750648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbb98e6020 msgr2=0x55cbb98f5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:34.866 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.857+0000 7f2b39b0e700 1 --2- 172.21.15.82:0/2183750648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbb98e6020 0x55cbb98f5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.866 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.857+0000 7f2b39b0e700 1 -- 172.21.15.82:0/2183750648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbb98e51e0 msgr2=0x55cbb98e55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:34.866 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.857+0000 7f2b39b0e700 1 --2- 172.21.15.82:0/2183750648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbb98e51e0 0x55cbb98e55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.866 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.857+0000 7f2b39b0e700 1 -- 172.21.15.82:0/2183750648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbb98ef960 con 0x55cbb98e6980 2023-09-09T23:13:34.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.857+0000 7f2b39b0e700 1 --2- 172.21.15.82:0/2183750648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbb98e6980 0x55cbb98ed6d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0971b0 tx=0x7f2b2c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397aaeef64fbddfd server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:34.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.858+0000 7f2b1ffff700 1 -- 172.21.15.82:0/2183750648 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c09a0a0 con 0x55cbb98e6980 2023-09-09T23:13:34.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.858+0000 7f2b1ffff700 1 -- 172.21.15.82:0/2183750648 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b2c09ae40 con 0x55cbb98e6980 2023-09-09T23:13:34.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.858+0000 7f2b1ffff700 1 -- 172.21.15.82:0/2183750648 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c098b70 con 0x55cbb98e6980 2023-09-09T23:13:34.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.858+0000 7f2b47737b80 1 -- 172.21.15.82:0/2183750648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbb98e6980 msgr2=0x55cbb98ed6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:34.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.858+0000 7f2b47737b80 1 --2- 172.21.15.82:0/2183750648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbb98e6980 0x55cbb98ed6d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0971b0 tx=0x7f2b2c00bf10 comp rx=0 tx=0).stop 2023-09-09T23:13:34.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.858+0000 7f2b47737b80 1 -- 172.21.15.82:0/2183750648 shutdown_connections 2023-09-09T23:13:34.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.858+0000 7f2b47737b80 1 --2- 172.21.15.82:0/2183750648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbb98e51e0 0x55cbb98e55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.858+0000 7f2b47737b80 1 --2- 172.21.15.82:0/2183750648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbb98e6980 0x55cbb98ed6d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.858+0000 7f2b47737b80 1 --2- 172.21.15.82:0/2183750648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbb98e6020 0x55cbb98f5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.858+0000 7f2b47737b80 1 -- 172.21.15.82:0/2183750648 >> 172.21.15.82:0/2183750648 conn(0x55cbb9855040 msgr2=0x55cbb98db6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:34.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.858+0000 7f2b47737b80 1 -- 172.21.15.82:0/2183750648 shutdown_connections 2023-09-09T23:13:34.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.858+0000 7f2b47737b80 1 -- 172.21.15.82:0/2183750648 wait complete. 2023-09-09T23:13:34.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.858+0000 7f2b47737b80 1 -- start start 2023-09-09T23:13:34.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.858+0000 7f2b47737b80 1 Processor -- start 2023-09-09T23:13:34.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.859+0000 7f2b47737b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbb98e51e0 0x55cbb98f1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.859+0000 7f2b47737b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbb98e6020 0x55cbb98f1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.859+0000 7f2b47737b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbb98e6980 0x55cbb98f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.859+0000 7f2b47737b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb98f4c30 con 0x55cbb98e6020 2023-09-09T23:13:34.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.859+0000 7f2b47737b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb98f4da0 con 0x55cbb98e51e0 2023-09-09T23:13:34.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.859+0000 7f2b47737b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb98f4f60 con 0x55cbb98e6980 2023-09-09T23:13:34.871 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.859+0000 7f2b47737b80 10 client.0 _ll_register_callbacks cb 0x7f2b3000cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:34.871 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.859+0000 7f2b38b0c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbb98e6020 0x55cbb98f1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.871 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.859+0000 7f2b38b0c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbb98e6020 0x55cbb98f1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55782/0 (socket says 172.21.15.82:55782) 2023-09-09T23:13:34.871 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.859+0000 7f2b38b0c700 1 -- 172.21.15.82:0/2301146345 learned_addr learned my addr 172.21.15.82:0/2301146345 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:34.872 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.859+0000 7f2b39b0e700 1 --2- 172.21.15.82:0/2301146345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbb98e6980 0x55cbb98f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.872 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.859+0000 7f2b3930d700 1 --2- 172.21.15.82:0/2301146345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbb98e51e0 0x55cbb98f1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.872 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.859+0000 7f2b38b0c700 1 -- 172.21.15.82:0/2301146345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbb98e6980 msgr2=0x55cbb98f46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:34.872 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.859+0000 7f2b38b0c700 1 --2- 172.21.15.82:0/2301146345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbb98e6980 0x55cbb98f46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.872 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.859+0000 7f2b38b0c700 1 -- 172.21.15.82:0/2301146345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbb98e51e0 msgr2=0x55cbb98f1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:34.873 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.859+0000 7f2b38b0c700 1 --2- 172.21.15.82:0/2301146345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbb98e51e0 0x55cbb98f1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.873 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.859+0000 7f2b38b0c700 1 -- 172.21.15.82:0/2301146345 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b3419e890 con 0x55cbb98e6020 2023-09-09T23:13:34.873 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.859+0000 7f2b39b0e700 1 --2- 172.21.15.82:0/2301146345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbb98e6980 0x55cbb98f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:34.873 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.859+0000 7f2b38b0c700 1 --2- 172.21.15.82:0/2301146345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbb98e6020 0x55cbb98f1740 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2b30000c00 tx=0x7f2b3001aea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d6b919d8cc38dc server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:34.873 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.859+0000 7f2b1cff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:34.873 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.859+0000 7f2b1cff9700 1 -- 172.21.15.82:0/2301146345 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3001f070 con 0x55cbb98e6020 2023-09-09T23:13:34.874 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.860+0000 7f2b1cff9700 1 -- 172.21.15.82:0/2301146345 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f2b30000e40 con 0x55cbb98e6020 2023-09-09T23:13:34.874 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.860+0000 7f2b47737b80 1 -- 172.21.15.82:0/2301146345 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b300108f0 con 0x55cbb98e6020 2023-09-09T23:13:34.874 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.860+0000 7f2b1cff9700 1 -- 172.21.15.82:0/2301146345 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b3001a5c0 con 0x55cbb98e6020 2023-09-09T23:13:34.874 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.860+0000 7f2b47737b80 10 client.15366 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:34.874 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.860+0000 7f2b47737b80 1 -- 172.21.15.82:0/2301146345 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cbb9685200 con 0x55cbb98e6020 2023-09-09T23:13:34.875 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.860+0000 7f2b1cff9700 1 -- 172.21.15.82:0/2301146345 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b300108f0 con 0x55cbb98e6020 2023-09-09T23:13:34.875 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.860+0000 7f2b1cff9700 1 -- 172.21.15.82:0/2301146345 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2b30004780 con 0x55cbb98e6020 2023-09-09T23:13:34.875 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.860+0000 7f2b47737b80 10 client.15366 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:34.875 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.860+0000 7f2b47737b80 10 client.15366 resolve_mds: resolved name 'k' to daemon mds.k 2023-09-09T23:13:34.875 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.860+0000 7f2b47737b80 20 client.15366 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:34.876 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.860+0000 7f2b47737b80 1 --2- 172.21.15.82:0/2301146345 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55cbb98f9f70 0x55cbb994ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:34.876 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.860+0000 7f2b47737b80 4 client.15366 mds_command: new command op to 24491 tid=0[{"prefix": "status"}] 2023-09-09T23:13:34.876 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.860+0000 7f2b47737b80 1 -- 172.21.15.82:0/2301146345 --> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cbb982b8d0 con 0x55cbb98f9f70 2023-09-09T23:13:34.876 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.860+0000 7f2b3930d700 1 --2- 172.21.15.82:0/2301146345 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55cbb98f9f70 0x55cbb994ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:34.876 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.861+0000 7f2b1cff9700 10 client.15366 ms_handle_connect on v2:172.21.15.98:6832/3855146755 2023-09-09T23:13:34.877 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.861+0000 7f2b3930d700 1 --2- 172.21.15.82:0/2301146345 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55cbb98f9f70 0x55cbb994ffd0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=333c6b92ba840959 server_cookie=34f8f75bb68c4e3b in_seq=0 out_seq=0 2023-09-09T23:13:34.877 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.861+0000 7f2b1cff9700 1 -- 172.21.15.82:0/2301146345 <== mds.3 v2:172.21.15.98:6832/3855146755 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55cbb982b8d0 con 0x55cbb98f9f70 2023-09-09T23:13:34.877 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.861+0000 7f2b1cff9700 10 client.15366 handle_command_reply: tid=0 2023-09-09T23:13:34.877 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.861+0000 7f2b47737b80 1 client.15366 shutdown 2023-09-09T23:13:34.877 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.861+0000 7f2b47737b80 1 -- 172.21.15.82:0/2301146345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbb98e6020 msgr2=0x55cbb98f1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:34.878 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.861+0000 7f2b47737b80 1 --2- 172.21.15.82:0/2301146345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbb98e6020 0x55cbb98f1740 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2b30000c00 tx=0x7f2b3001aea0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.878 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.861+0000 7f2b47737b80 1 -- 172.21.15.82:0/2301146345 shutdown_connections 2023-09-09T23:13:34.878 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.861+0000 7f2b47737b80 1 --2- 172.21.15.82:0/2301146345 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55cbb98f9f70 0x55cbb994ffd0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.878 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.861+0000 7f2b47737b80 1 --2- 172.21.15.82:0/2301146345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbb98e51e0 0x55cbb98f1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.878 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.861+0000 7f2b1cff9700 0 client.15366 ms_handle_reset on v2:172.21.15.98:6832/3855146755 2023-09-09T23:13:34.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.861+0000 7f2b47737b80 1 --2- 172.21.15.82:0/2301146345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cbb98e6980 0x55cbb98f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.861+0000 7f2b47737b80 1 --2- 172.21.15.82:0/2301146345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbb98e6020 0x55cbb98f1740 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.861+0000 7f2b47737b80 1 -- 172.21.15.82:0/2301146345 >> 172.21.15.82:0/2301146345 conn(0x55cbb9855040 msgr2=0x55cbb98db620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:34.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.861+0000 7f2b47737b80 1 -- 172.21.15.82:0/2301146345 shutdown_connections 2023-09-09T23:13:34.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.861+0000 7f2b47737b80 1 -- 172.21.15.82:0/2301146345 wait complete. 2023-09-09T23:13:34.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.861+0000 7f2b47737b80 20 client.15366 trim_cache size 0 max 16384 2023-09-09T23:13:34.880 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:13:34.880 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da", 2023-09-09T23:13:34.880 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 3, 2023-09-09T23:13:34.880 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24491, 2023-09-09T23:13:34.881 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2023-09-09T23:13:34.881 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2023-09-09T23:13:34.881 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2023-09-09T23:13:34.881 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 71.601331223000003, 2023-09-09T23:13:34.881 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 27, 2023-09-09T23:13:34.882 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 83, 2023-09-09T23:13:34.882 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 81, 2023-09-09T23:13:34.882 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 104.979158731 2023-09-09T23:13:34.882 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:13:34.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.862+0000 7f2b3b571700 1 -- 172.21.15.82:0/3936952203 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f2b200a06d0 msgr2=0x7f2b200a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:34.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.862+0000 7f2b3b571700 1 --2- 172.21.15.82:0/3936952203 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f2b200a06d0 0x7f2b200a2b90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2b340ffcf0 tx=0x7f2b28009260 comp rx=0 tx=0).stop 2023-09-09T23:13:34.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.862+0000 7f2b3b571700 1 -- 172.21.15.82:0/3936952203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b34100830 msgr2=0x7f2b3419d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:34.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.862+0000 7f2b3b571700 1 --2- 172.21.15.82:0/3936952203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b34100830 0x7f2b3419d710 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c008f10 tx=0x7f2b2c008ff0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.862+0000 7f2b3b571700 1 -- 172.21.15.82:0/3936952203 shutdown_connections 2023-09-09T23:13:34.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.862+0000 7f2b3b571700 1 --2- 172.21.15.82:0/3936952203 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f2b200a06d0 0x7f2b200a2b90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.862+0000 7f2b3b571700 1 --2- 172.21.15.82:0/3936952203 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b340feca0 0x7f2b340fc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.862+0000 7f2b3b571700 1 --2- 172.21.15.82:0/3936952203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b34100830 0x7f2b3419d710 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.862+0000 7f2b3b571700 1 --2- 172.21.15.82:0/3936952203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b340ffe90 0x7f2b3419b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:34.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.862+0000 7f2b3b571700 1 -- 172.21.15.82:0/3936952203 >> 172.21.15.82:0/3936952203 conn(0x7f2b340fa1f0 msgr2=0x7f2b34103950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:34.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.862+0000 7f2b3b571700 1 -- 172.21.15.82:0/3936952203 shutdown_connections 2023-09-09T23:13:34.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:34.862+0000 7f2b3b571700 1 -- 172.21.15.82:0/3936952203 wait complete. 2023-09-09T23:13:34.886 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-09-09T23:13:35.010 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.010+0000 7f5ec56cd700 1 -- 172.21.15.82:0/2948422645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ec0100ad0 msgr2=0x7f5ec0100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:35.011 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.010+0000 7f5ec56cd700 1 --2- 172.21.15.82:0/2948422645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ec0100ad0 0x7f5ec0100ef0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009990 tx=0x7f5ea8009ca0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.011 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.010+0000 7f5ec56cd700 1 -- 172.21.15.82:0/2948422645 shutdown_connections 2023-09-09T23:13:35.011 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.010+0000 7f5ec56cd700 1 --2- 172.21.15.82:0/2948422645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ec01000e0 0x7f5ec0107980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.011 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.010+0000 7f5ec56cd700 1 --2- 172.21.15.82:0/2948422645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ec0100ad0 0x7f5ec0100ef0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.012 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.010+0000 7f5ec56cd700 1 --2- 172.21.15.82:0/2948422645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ec00ff720 0x7f5ec00ffba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.012 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.010+0000 7f5ec56cd700 1 -- 172.21.15.82:0/2948422645 >> 172.21.15.82:0/2948422645 conn(0x7f5ec00fa1f0 msgr2=0x7f5ec00fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:35.012 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.010+0000 7f5ec56cd700 1 -- 172.21.15.82:0/2948422645 shutdown_connections 2023-09-09T23:13:35.012 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.010+0000 7f5ec56cd700 1 -- 172.21.15.82:0/2948422645 wait complete. 2023-09-09T23:13:35.012 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.010+0000 7f5ec56cd700 1 Processor -- start 2023-09-09T23:13:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.011+0000 7f5ec56cd700 1 -- start start 2023-09-09T23:13:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.011+0000 7f5ec56cd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ec00ff720 0x7f5ec00ff180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.011+0000 7f5ec56cd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ec01000e0 0x7f5ec019d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.011+0000 7f5ec56cd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ec0100ad0 0x7f5ec019fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.011+0000 7f5ec56cd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec01a0190 con 0x7f5ec0100ad0 2023-09-09T23:13:35.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.011+0000 7f5ec56cd700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec01a0300 con 0x7f5ec01000e0 2023-09-09T23:13:35.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.011+0000 7f5ec56cd700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec01a04c0 con 0x7f5ec00ff720 2023-09-09T23:13:35.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.011+0000 7f5ebf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ec0100ad0 0x7f5ec019fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.011+0000 7f5ebf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ec0100ad0 0x7f5ec019fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55798/0 (socket says 172.21.15.82:55798) 2023-09-09T23:13:35.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.011+0000 7f5ebeffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ec00ff720 0x7f5ec00ff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.011+0000 7f5ebf7fe700 1 -- 172.21.15.82:0/1331301973 learned_addr learned my addr 172.21.15.82:0/1331301973 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:35.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.011+0000 7f5ebe7fc700 1 --2- 172.21.15.82:0/1331301973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ec01000e0 0x7f5ec019d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.011+0000 7f5ebf7fe700 1 -- 172.21.15.82:0/1331301973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ec00ff720 msgr2=0x7f5ec00ff180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:35.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.011+0000 7f5ebf7fe700 1 --2- 172.21.15.82:0/1331301973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ec00ff720 0x7f5ec00ff180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.011+0000 7f5ebf7fe700 1 -- 172.21.15.82:0/1331301973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ec01000e0 msgr2=0x7f5ec019d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:35.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.011+0000 7f5ebf7fe700 1 --2- 172.21.15.82:0/1331301973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ec01000e0 0x7f5ec019d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.011+0000 7f5ebf7fe700 1 -- 172.21.15.82:0/1331301973 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea80095f0 con 0x7f5ec0100ad0 2023-09-09T23:13:35.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.011+0000 7f5ebf7fe700 1 --2- 172.21.15.82:0/1331301973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ec0100ad0 0x7f5ec019fae0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800b970 tx=0x7f5eb800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:35.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.012+0000 7f5eb7fff700 1 -- 172.21.15.82:0/1331301973 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb800d860 con 0x7f5ec0100ad0 2023-09-09T23:13:35.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.012+0000 7f5ec56cd700 1 -- 172.21.15.82:0/1331301973 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec01a07a0 con 0x7f5ec0100ad0 2023-09-09T23:13:35.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.012+0000 7f5eb7fff700 1 -- 172.21.15.82:0/1331301973 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5eb800de80 con 0x7f5ec0100ad0 2023-09-09T23:13:35.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.012+0000 7f5eb7fff700 1 -- 172.21.15.82:0/1331301973 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8017400 con 0x7f5ec0100ad0 2023-09-09T23:13:35.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.012+0000 7f5ec56cd700 1 -- 172.21.15.82:0/1331301973 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec01a0d30 con 0x7f5ec0100ad0 2023-09-09T23:13:35.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.014+0000 7f5eb7fff700 1 -- 172.21.15.82:0/1331301973 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f5eb800e630 con 0x7f5ec0100ad0 2023-09-09T23:13:35.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.014+0000 7f5eb7fff700 1 --2- 172.21.15.82:0/1331301973 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5eac0a06d0 0x7f5eac0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.014+0000 7f5ebeffd700 1 --2- 172.21.15.82:0/1331301973 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5eac0a06d0 0x7f5eac0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.014+0000 7f5eb7fff700 1 -- 172.21.15.82:0/1331301973 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f5eb8006e40 con 0x7f5ec0100ad0 2023-09-09T23:13:35.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.015+0000 7f5ebeffd700 1 --2- 172.21.15.82:0/1331301973 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5eac0a06d0 0x7f5eac0a2b90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800b5c0 tx=0x7f5ea8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:35.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.015+0000 7f5ed1893b80 1 Processor -- start 2023-09-09T23:13:35.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.015+0000 7f5ed1893b80 1 -- start start 2023-09-09T23:13:35.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.015+0000 7f5ed1893b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562519d63bb0 0x562519d63fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.015+0000 7f5ed1893b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562519d64550 0x562519d627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.015+0000 7f5ed1893b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562519d62db0 0x562519d63260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.015+0000 7f5ed1893b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562519cbb6e0 con 0x562519d63bb0 2023-09-09T23:13:35.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.016+0000 7f5ed1893b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x562519cb9000 con 0x562519d62db0 2023-09-09T23:13:35.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.016+0000 7f5ed1893b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562519cb48f0 con 0x562519d64550 2023-09-09T23:13:35.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.016+0000 7f5ebeffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562519d63bb0 0x562519d63fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.016+0000 7f5ebeffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562519d63bb0 0x562519d63fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55802/0 (socket says 172.21.15.82:55802) 2023-09-09T23:13:35.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.016+0000 7f5ebeffd700 1 -- 172.21.15.82:0/3659152597 learned_addr learned my addr 172.21.15.82:0/3659152597 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:35.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.016+0000 7f5ebeffd700 1 -- 172.21.15.82:0/3659152597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562519d64550 msgr2=0x562519d627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:35.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.016+0000 7f5ebeffd700 1 --2- 172.21.15.82:0/3659152597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562519d64550 0x562519d627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.016+0000 7f5ebf7fe700 1 --2- 172.21.15.82:0/3659152597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562519d62db0 0x562519d63260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.016+0000 7f5ebeffd700 1 -- 172.21.15.82:0/3659152597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562519d62db0 msgr2=0x562519d63260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:35.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.016+0000 7f5ebeffd700 1 --2- 172.21.15.82:0/3659152597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562519d62db0 0x562519d63260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.016+0000 7f5ebeffd700 1 -- 172.21.15.82:0/3659152597 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562519ae0d90 con 0x562519d63bb0 2023-09-09T23:13:35.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.016+0000 7f5ebeffd700 1 --2- 172.21.15.82:0/3659152597 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562519d63bb0 0x562519d63fd0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5ea80055f0 tx=0x7f5ea8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd84609f25b8ae42 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:35.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.016+0000 7f5eb57fa700 1 -- 172.21.15.82:0/3659152597 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8003890 con 0x562519d63bb0 2023-09-09T23:13:35.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.016+0000 7f5eb57fa700 1 -- 172.21.15.82:0/3659152597 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ea801e040 con 0x562519d63bb0 2023-09-09T23:13:35.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.016+0000 7f5eb57fa700 1 -- 172.21.15.82:0/3659152597 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8004a10 con 0x562519d63bb0 2023-09-09T23:13:35.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.017+0000 7f5ed1893b80 1 -- 172.21.15.82:0/3659152597 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562519d63bb0 msgr2=0x562519d63fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:35.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.017+0000 7f5ed1893b80 1 --2- 172.21.15.82:0/3659152597 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562519d63bb0 0x562519d63fd0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5ea80055f0 tx=0x7f5ea8020000 comp rx=0 tx=0).stop 2023-09-09T23:13:35.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.017+0000 7f5ed1893b80 1 -- 172.21.15.82:0/3659152597 shutdown_connections 2023-09-09T23:13:35.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.017+0000 7f5ed1893b80 1 --2- 172.21.15.82:0/3659152597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562519d62db0 0x562519d63260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.017+0000 7f5ed1893b80 1 --2- 172.21.15.82:0/3659152597 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562519d63bb0 0x562519d63fd0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.017+0000 7f5ed1893b80 1 --2- 172.21.15.82:0/3659152597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562519d64550 0x562519d627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.017+0000 7f5ed1893b80 1 -- 172.21.15.82:0/3659152597 >> 172.21.15.82:0/3659152597 conn(0x562519cd7040 msgr2=0x562519d5d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:35.029 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.017+0000 7f5ed1893b80 1 -- 172.21.15.82:0/3659152597 shutdown_connections 2023-09-09T23:13:35.029 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.017+0000 7f5ed1893b80 1 -- 172.21.15.82:0/3659152597 wait complete. 2023-09-09T23:13:35.029 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.017+0000 7f5ed1893b80 1 -- start start 2023-09-09T23:13:35.029 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.017+0000 7f5ed1893b80 1 Processor -- start 2023-09-09T23:13:35.029 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.018+0000 7f5ed1893b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562519d62db0 0x562519d6c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.030 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.018+0000 7f5ed1893b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562519d63bb0 0x562519d6cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.030 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.018+0000 7f5ed1893b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562519d64550 0x562519d73b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.030 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.018+0000 7f5ed1893b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562519c52b20 con 0x562519d62db0 2023-09-09T23:13:35.030 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.018+0000 7f5ed1893b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x562519c52820 con 0x562519d63bb0 2023-09-09T23:13:35.030 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.018+0000 7f5ed1893b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562519b368d0 con 0x562519d64550 2023-09-09T23:13:35.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.018+0000 7f5ed1893b80 10 client.0 _ll_register_callbacks cb 0x562519b9ca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:35.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.018+0000 7f5ebeffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562519d62db0 0x562519d6c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.018+0000 7f5ebeffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562519d62db0 0x562519d6c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55808/0 (socket says 172.21.15.82:55808) 2023-09-09T23:13:35.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.018+0000 7f5ebeffd700 1 -- 172.21.15.82:0/3811682233 learned_addr learned my addr 172.21.15.82:0/3811682233 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:35.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.018+0000 7f5ebeffd700 1 -- 172.21.15.82:0/3811682233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562519d64550 msgr2=0x562519d73b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:35.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.018+0000 7f5ebeffd700 1 --2- 172.21.15.82:0/3811682233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562519d64550 0x562519d73b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.018+0000 7f5ebeffd700 1 -- 172.21.15.82:0/3811682233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562519d63bb0 msgr2=0x562519d6cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:35.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.018+0000 7f5ebeffd700 1 --2- 172.21.15.82:0/3811682233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562519d63bb0 0x562519d6cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.018+0000 7f5ebeffd700 1 -- 172.21.15.82:0/3811682233 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec01082b0 con 0x562519d62db0 2023-09-09T23:13:35.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.018+0000 7f5ebeffd700 1 --2- 172.21.15.82:0/3811682233 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562519d62db0 0x562519d6c5a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8032040 tx=0x7f5ea8004f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f65682517c4007 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:35.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.018+0000 7f5ea67fc700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:35.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.018+0000 7f5ea67fc700 1 -- 172.21.15.82:0/3811682233 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8003ca0 con 0x562519d62db0 2023-09-09T23:13:35.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.018+0000 7f5ed1893b80 1 -- 172.21.15.82:0/3811682233 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562519c0c4a0 con 0x562519d62db0 2023-09-09T23:13:35.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.018+0000 7f5ea67fc700 1 -- 172.21.15.82:0/3811682233 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f5ea8003e00 con 0x562519d62db0 2023-09-09T23:13:35.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.018+0000 7f5ea67fc700 1 -- 172.21.15.82:0/3811682233 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ea80056e0 con 0x562519d62db0 2023-09-09T23:13:35.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.018+0000 7f5ed1893b80 10 client.15378 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:35.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.018+0000 7f5ed1893b80 1 -- 172.21.15.82:0/3811682233 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562519ac9c10 con 0x562519d62db0 2023-09-09T23:13:35.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.018+0000 7f5ea67fc700 1 -- 172.21.15.82:0/3811682233 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562519c0c4a0 con 0x562519d62db0 2023-09-09T23:13:35.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.019+0000 7f5ea67fc700 1 -- 172.21.15.82:0/3811682233 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5ea8044410 con 0x562519d62db0 2023-09-09T23:13:35.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.019+0000 7f5ed1893b80 10 client.15378 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:35.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.019+0000 7f5ed1893b80 10 client.15378 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-09T23:13:35.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.019+0000 7f5ed1893b80 20 client.15378 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:35.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.019+0000 7f5ed1893b80 1 --2- 172.21.15.82:0/3811682233 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x562519d77870 0x562519d76a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.019+0000 7f5ed1893b80 4 client.15378 mds_command: new command op to 24523 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:13:35.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.019+0000 7f5ed1893b80 1 -- 172.21.15.82:0/3811682233 --> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562519cc9850 con 0x562519d77870 2023-09-09T23:13:35.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.019+0000 7f5ebe7fc700 1 --2- 172.21.15.82:0/3811682233 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x562519d77870 0x562519d76a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.019+0000 7f5ea67fc700 10 client.15378 ms_handle_connect on v2:172.21.15.170:6838/3858369735 2023-09-09T23:13:35.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.019+0000 7f5ebe7fc700 1 --2- 172.21.15.82:0/3811682233 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x562519d77870 0x562519d76a50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=106b97abcdb8f9d server_cookie=75d92c333266e284 in_seq=0 out_seq=0 2023-09-09T23:13:35.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.020+0000 7f5ea67fc700 1 -- 172.21.15.82:0/3811682233 <== mds.2 v2:172.21.15.170:6838/3858369735 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562519cc9850 con 0x562519d77870 2023-09-09T23:13:35.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.020+0000 7f5ea67fc700 10 client.15378 handle_command_reply: tid=0 2023-09-09T23:13:35.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.020+0000 7f5ed1893b80 1 client.15378 shutdown 2023-09-09T23:13:35.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.020+0000 7f5ed1893b80 1 -- 172.21.15.82:0/3811682233 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562519d62db0 msgr2=0x562519d6c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:35.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.020+0000 7f5ed1893b80 1 --2- 172.21.15.82:0/3811682233 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562519d62db0 0x562519d6c5a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8032040 tx=0x7f5ea8004f40 comp rx=0 tx=0).stop 2023-09-09T23:13:35.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.020+0000 7f5ed1893b80 1 -- 172.21.15.82:0/3811682233 shutdown_connections 2023-09-09T23:13:35.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.020+0000 7f5ed1893b80 1 --2- 172.21.15.82:0/3811682233 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x562519d77870 0x562519d76a50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.020+0000 7f5ed1893b80 1 --2- 172.21.15.82:0/3811682233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562519d64550 0x562519d73b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.020+0000 7f5ed1893b80 1 --2- 172.21.15.82:0/3811682233 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562519d62db0 0x562519d6c5a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.020+0000 7f5ea67fc700 0 client.15378 ms_handle_reset on v2:172.21.15.170:6838/3858369735 2023-09-09T23:13:35.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.020+0000 7f5ed1893b80 1 --2- 172.21.15.82:0/3811682233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562519d63bb0 0x562519d6cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.020+0000 7f5ed1893b80 1 -- 172.21.15.82:0/3811682233 >> 172.21.15.82:0/3811682233 conn(0x562519cd7040 msgr2=0x562519d5cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:35.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.020+0000 7f5ed1893b80 1 -- 172.21.15.82:0/3811682233 shutdown_connections 2023-09-09T23:13:35.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.020+0000 7f5ed1893b80 1 -- 172.21.15.82:0/3811682233 wait complete. 2023-09-09T23:13:35.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.020+0000 7f5ed1893b80 20 client.15378 trim_cache size 0 max 16384 2023-09-09T23:13:35.044 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.037+0000 7f5ed1893b80 1 Processor -- start 2023-09-09T23:13:35.044 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.037+0000 7f5ed1893b80 1 -- start start 2023-09-09T23:13:35.044 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.038+0000 7f5ed1893b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562519d62db0 0x562519d74350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.044 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.038+0000 7f5ed1893b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562519d63bb0 0x562519d73ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.045 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.038+0000 7f5ed1893b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562519d64550 0x562519d6b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.045 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.038+0000 7f5ed1893b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562519c0c4a0 con 0x562519d64550 2023-09-09T23:13:35.045 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.038+0000 7f5ed1893b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x562519d6d790 con 0x562519d63bb0 2023-09-09T23:13:35.045 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.038+0000 7f5ed1893b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562519d5ef90 con 0x562519d62db0 2023-09-09T23:13:35.045 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.038+0000 7f5ebf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562519d64550 0x562519d6b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.046 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.038+0000 7f5ebf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562519d64550 0x562519d6b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55820/0 (socket says 172.21.15.82:55820) 2023-09-09T23:13:35.046 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.038+0000 7f5ebf7fe700 1 -- 172.21.15.82:0/3658850112 learned_addr learned my addr 172.21.15.82:0/3658850112 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:35.046 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.038+0000 7f5ebf7fe700 1 -- 172.21.15.82:0/3658850112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562519d62db0 msgr2=0x562519d74350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:35.046 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.038+0000 7f5ebf7fe700 1 --2- 172.21.15.82:0/3658850112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562519d62db0 0x562519d74350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.049 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.038+0000 7f5ebe7fc700 1 --2- 172.21.15.82:0/3658850112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562519d63bb0 0x562519d73ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.049 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.038+0000 7f5ebf7fe700 1 -- 172.21.15.82:0/3658850112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562519d63bb0 msgr2=0x562519d73ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:35.049 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.038+0000 7f5ebf7fe700 1 --2- 172.21.15.82:0/3658850112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562519d63bb0 0x562519d73ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.049 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.038+0000 7f5ebf7fe700 1 -- 172.21.15.82:0/3658850112 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562519b07200 con 0x562519d64550 2023-09-09T23:13:35.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.038+0000 7f5ebf7fe700 1 --2- 172.21.15.82:0/3658850112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562519d64550 0x562519d6b280 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800bfd0 tx=0x7f5eb8098c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca1fd896b9167bd server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:35.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.038+0000 7f5eb5ffb700 1 -- 172.21.15.82:0/3658850112 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb80b0070 con 0x562519d64550 2023-09-09T23:13:35.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.038+0000 7f5eb5ffb700 1 -- 172.21.15.82:0/3658850112 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5eb8099020 con 0x562519d64550 2023-09-09T23:13:35.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.038+0000 7f5eb5ffb700 1 -- 172.21.15.82:0/3658850112 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8099280 con 0x562519d64550 2023-09-09T23:13:35.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.039+0000 7f5ed1893b80 1 -- 172.21.15.82:0/3658850112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562519d64550 msgr2=0x562519d6b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:35.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.039+0000 7f5ed1893b80 1 --2- 172.21.15.82:0/3658850112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562519d64550 0x562519d6b280 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800bfd0 tx=0x7f5eb8098c70 comp rx=0 tx=0).stop 2023-09-09T23:13:35.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.039+0000 7f5ed1893b80 1 -- 172.21.15.82:0/3658850112 shutdown_connections 2023-09-09T23:13:35.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.039+0000 7f5ed1893b80 1 --2- 172.21.15.82:0/3658850112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562519d64550 0x562519d6b280 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.039+0000 7f5ed1893b80 1 --2- 172.21.15.82:0/3658850112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562519d62db0 0x562519d74350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.039+0000 7f5ed1893b80 1 --2- 172.21.15.82:0/3658850112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562519d63bb0 0x562519d73ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.052 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.039+0000 7f5ed1893b80 1 -- 172.21.15.82:0/3658850112 >> 172.21.15.82:0/3658850112 conn(0x562519cd7040 msgr2=0x562519d5d680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:35.052 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.039+0000 7f5ed1893b80 1 -- 172.21.15.82:0/3658850112 shutdown_connections 2023-09-09T23:13:35.052 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.039+0000 7f5ed1893b80 1 -- 172.21.15.82:0/3658850112 wait complete. 2023-09-09T23:13:35.052 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.039+0000 7f5ed1893b80 1 -- start start 2023-09-09T23:13:35.052 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.039+0000 7f5ed1893b80 1 Processor -- start 2023-09-09T23:13:35.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.040+0000 7f5ed1893b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562519d62db0 0x562519d70ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.040+0000 7f5ed1893b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562519d63bb0 0x562519d711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.040+0000 7f5ed1893b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562519d64550 0x562519d721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.040+0000 7f5ed1893b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562519d72730 con 0x562519d62db0 2023-09-09T23:13:35.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.040+0000 7f5ed1893b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x562519d728a0 con 0x562519d64550 2023-09-09T23:13:35.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.040+0000 7f5ed1893b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562519d72a60 con 0x562519d63bb0 2023-09-09T23:13:35.054 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.040+0000 7f5ed1893b80 10 client.0 _ll_register_callbacks cb 0x7f5ea80054f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:35.054 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.040+0000 7f5ebeffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562519d62db0 0x562519d70ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.054 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.040+0000 7f5ebeffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562519d62db0 0x562519d70ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55822/0 (socket says 172.21.15.82:55822) 2023-09-09T23:13:35.054 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.040+0000 7f5ebeffd700 1 -- 172.21.15.82:0/1398888584 learned_addr learned my addr 172.21.15.82:0/1398888584 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:35.054 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.040+0000 7f5ebf7fe700 1 --2- 172.21.15.82:0/1398888584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562519d64550 0x562519d721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.040+0000 7f5ebeffd700 1 -- 172.21.15.82:0/1398888584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562519d63bb0 msgr2=0x562519d711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:35.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.040+0000 7f5ebeffd700 1 --2- 172.21.15.82:0/1398888584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562519d63bb0 0x562519d711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.040+0000 7f5ebeffd700 1 -- 172.21.15.82:0/1398888584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562519d64550 msgr2=0x562519d721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:35.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.040+0000 7f5ebeffd700 1 --2- 172.21.15.82:0/1398888584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562519d64550 0x562519d721f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.040+0000 7f5ebeffd700 1 -- 172.21.15.82:0/1398888584 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec01a0d30 con 0x562519d62db0 2023-09-09T23:13:35.056 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.040+0000 7f5ebf7fe700 1 --2- 172.21.15.82:0/1398888584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562519d64550 0x562519d721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:35.056 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.040+0000 7f5ebeffd700 1 --2- 172.21.15.82:0/1398888584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562519d62db0 0x562519d70ca0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009990 tx=0x7f5ea8005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d48885994e2003 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:35.056 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.040+0000 7f5ea67fc700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:35.056 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.040+0000 7f5ea67fc700 1 -- 172.21.15.82:0/1398888584 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8044b10 con 0x562519d62db0 2023-09-09T23:13:35.056 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.040+0000 7f5ea67fc700 1 -- 172.21.15.82:0/1398888584 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f5ea8003aa0 con 0x562519d62db0 2023-09-09T23:13:35.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.040+0000 7f5ed1893b80 1 -- 172.21.15.82:0/1398888584 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ea801e040 con 0x562519d62db0 2023-09-09T23:13:35.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.040+0000 7f5ea67fc700 1 -- 172.21.15.82:0/1398888584 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ea8003c80 con 0x562519d62db0 2023-09-09T23:13:35.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.040+0000 7f5ed1893b80 10 client.15384 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:35.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.040+0000 7f5ed1893b80 1 -- 172.21.15.82:0/1398888584 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562519adf260 con 0x562519d62db0 2023-09-09T23:13:35.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.041+0000 7f5ea67fc700 1 -- 172.21.15.82:0/1398888584 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ea8053020 con 0x562519d62db0 2023-09-09T23:13:35.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.041+0000 7f5ea67fc700 1 -- 172.21.15.82:0/1398888584 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5ea800fa20 con 0x562519d62db0 2023-09-09T23:13:35.058 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.041+0000 7f5ed1893b80 10 client.15384 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:35.058 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.041+0000 7f5ed1893b80 10 client.15384 resolve_mds: resolved name 'i' to daemon mds.i 2023-09-09T23:13:35.058 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.041+0000 7f5ed1893b80 20 client.15384 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:35.058 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.041+0000 7f5ed1893b80 1 --2- 172.21.15.82:0/1398888584 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x562519dd1140 0x562519dcd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.058 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.041+0000 7f5ed1893b80 4 client.15384 mds_command: new command op to 24523 tid=0[{"prefix": "status"}] 2023-09-09T23:13:35.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.041+0000 7f5ed1893b80 1 -- 172.21.15.82:0/1398888584 --> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562519cad8d0 con 0x562519dd1140 2023-09-09T23:13:35.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.041+0000 7f5ebe7fc700 1 --2- 172.21.15.82:0/1398888584 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x562519dd1140 0x562519dcd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.041+0000 7f5ea67fc700 10 client.15384 ms_handle_connect on v2:172.21.15.170:6838/3858369735 2023-09-09T23:13:35.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.041+0000 7f5ebe7fc700 1 --2- 172.21.15.82:0/1398888584 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x562519dd1140 0x562519dcd8d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5a0215df95cd1407 server_cookie=270be8fe61ac822 in_seq=0 out_seq=0 2023-09-09T23:13:35.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.042+0000 7f5ea67fc700 1 -- 172.21.15.82:0/1398888584 <== mds.2 v2:172.21.15.170:6838/3858369735 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x562519cad8d0 con 0x562519dd1140 2023-09-09T23:13:35.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.042+0000 7f5ea67fc700 10 client.15384 handle_command_reply: tid=0 2023-09-09T23:13:35.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.042+0000 7f5ed1893b80 1 client.15384 shutdown 2023-09-09T23:13:35.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.042+0000 7f5ed1893b80 1 -- 172.21.15.82:0/1398888584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562519d62db0 msgr2=0x562519d70ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:35.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.042+0000 7f5ed1893b80 1 --2- 172.21.15.82:0/1398888584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562519d62db0 0x562519d70ca0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009990 tx=0x7f5ea8005790 comp rx=0 tx=0).stop 2023-09-09T23:13:35.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.042+0000 7f5ed1893b80 1 -- 172.21.15.82:0/1398888584 shutdown_connections 2023-09-09T23:13:35.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.042+0000 7f5ed1893b80 1 --2- 172.21.15.82:0/1398888584 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x562519dd1140 0x562519dcd8d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.042+0000 7f5ed1893b80 1 --2- 172.21.15.82:0/1398888584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562519d64550 0x562519d721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.042+0000 7f5ed1893b80 1 --2- 172.21.15.82:0/1398888584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562519d62db0 0x562519d70ca0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.042+0000 7f5ea67fc700 0 client.15384 ms_handle_reset on v2:172.21.15.170:6838/3858369735 2023-09-09T23:13:35.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.042+0000 7f5ed1893b80 1 --2- 172.21.15.82:0/1398888584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562519d63bb0 0x562519d711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.042+0000 7f5ed1893b80 1 -- 172.21.15.82:0/1398888584 >> 172.21.15.82:0/1398888584 conn(0x562519cd7040 msgr2=0x562519d6b280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:35.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.042+0000 7f5ed1893b80 1 -- 172.21.15.82:0/1398888584 shutdown_connections 2023-09-09T23:13:35.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.042+0000 7f5ed1893b80 1 -- 172.21.15.82:0/1398888584 wait complete. 2023-09-09T23:13:35.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.042+0000 7f5ed1893b80 20 client.15384 trim_cache size 0 max 16384 2023-09-09T23:13:35.063 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:13:35.063 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da", 2023-09-09T23:13:35.063 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 2, 2023-09-09T23:13:35.063 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24523, 2023-09-09T23:13:35.063 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2023-09-09T23:13:35.064 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2023-09-09T23:13:35.064 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2023-09-09T23:13:35.064 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 72.469519950999995, 2023-09-09T23:13:35.064 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 27, 2023-09-09T23:13:35.064 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 83, 2023-09-09T23:13:35.065 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 81, 2023-09-09T23:13:35.065 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 89.911634804000002 2023-09-09T23:13:35.065 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:13:35.066 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.042+0000 7f5ec56cd700 1 -- 172.21.15.82:0/1331301973 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5eac0a06d0 msgr2=0x7f5eac0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:35.066 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.042+0000 7f5ec56cd700 1 --2- 172.21.15.82:0/1331301973 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5eac0a06d0 0x7f5eac0a2b90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800b5c0 tx=0x7f5ea8005960 comp rx=0 tx=0).stop 2023-09-09T23:13:35.067 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.042+0000 7f5ec56cd700 1 -- 172.21.15.82:0/1331301973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ec0100ad0 msgr2=0x7f5ec019fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:35.067 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.042+0000 7f5ec56cd700 1 --2- 172.21.15.82:0/1331301973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ec0100ad0 0x7f5ec019fae0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800b970 tx=0x7f5eb800bc80 comp rx=0 tx=0).stop 2023-09-09T23:13:35.067 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.042+0000 7f5ec56cd700 1 -- 172.21.15.82:0/1331301973 shutdown_connections 2023-09-09T23:13:35.067 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.042+0000 7f5ec56cd700 1 --2- 172.21.15.82:0/1331301973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ec0100ad0 0x7f5ec019fae0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.042+0000 7f5ec56cd700 1 --2- 172.21.15.82:0/1331301973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ec00ff720 0x7f5ec00ff180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.042+0000 7f5ec56cd700 1 --2- 172.21.15.82:0/1331301973 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5eac0a06d0 0x7f5eac0a2b90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.042+0000 7f5ec56cd700 1 --2- 172.21.15.82:0/1331301973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ec01000e0 0x7f5ec019d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.042+0000 7f5ec56cd700 1 -- 172.21.15.82:0/1331301973 >> 172.21.15.82:0/1331301973 conn(0x7f5ec00fa1f0 msgr2=0x7f5ec00fc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:35.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.043+0000 7f5ec56cd700 1 -- 172.21.15.82:0/1331301973 shutdown_connections 2023-09-09T23:13:35.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.043+0000 7f5ec56cd700 1 -- 172.21.15.82:0/1331301973 wait complete. 2023-09-09T23:13:35.069 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-09-09T23:13:35.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.192+0000 7fb357c81700 1 -- 172.21.15.82:0/2878809836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3501000c0 msgr2=0x7fb350107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:35.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.192+0000 7fb357c81700 1 --2- 172.21.15.82:0/2878809836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3501000c0 0x7fb350107960 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb34800b0a0 tx=0x7fb34800b3b0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.192+0000 7fb357c81700 1 -- 172.21.15.82:0/2878809836 shutdown_connections 2023-09-09T23:13:35.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.192+0000 7fb357c81700 1 --2- 172.21.15.82:0/2878809836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3501000c0 0x7fb350107960 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.192+0000 7fb357c81700 1 --2- 172.21.15.82:0/2878809836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb350100ab0 0x7fb350100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.192+0000 7fb357c81700 1 --2- 172.21.15.82:0/2878809836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3500ff700 0x7fb3500ffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.192+0000 7fb357c81700 1 -- 172.21.15.82:0/2878809836 >> 172.21.15.82:0/2878809836 conn(0x7fb3500fa1d0 msgr2=0x7fb3500fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:35.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.192+0000 7fb357c81700 1 -- 172.21.15.82:0/2878809836 shutdown_connections 2023-09-09T23:13:35.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.192+0000 7fb357c81700 1 -- 172.21.15.82:0/2878809836 wait complete. 2023-09-09T23:13:35.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.192+0000 7fb357c81700 1 Processor -- start 2023-09-09T23:13:35.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.192+0000 7fb357c81700 1 -- start start 2023-09-09T23:13:35.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.192+0000 7fb357c81700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3500ff700 0x7fb3501963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.192+0000 7fb357c81700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3501000c0 0x7fb350196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.192+0000 7fb357c81700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb350100ab0 0x7fb35019d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.192+0000 7fb357c81700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35019df60 con 0x7fb350100ab0 2023-09-09T23:13:35.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.192+0000 7fb357c81700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35019e0a0 con 0x7fb3501000c0 2023-09-09T23:13:35.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.192+0000 7fb357c81700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35019e230 con 0x7fb3500ff700 2023-09-09T23:13:35.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.193+0000 7fb35621e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb350100ab0 0x7fb35019d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.193+0000 7fb35621e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb350100ab0 0x7fb35019d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55838/0 (socket says 172.21.15.82:55838) 2023-09-09T23:13:35.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.193+0000 7fb35621e700 1 -- 172.21.15.82:0/290723837 learned_addr learned my addr 172.21.15.82:0/290723837 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:35.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.193+0000 7fb35521c700 1 --2- 172.21.15.82:0/290723837 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3501000c0 0x7fb350196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.193+0000 7fb35621e700 1 -- 172.21.15.82:0/290723837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3500ff700 msgr2=0x7fb3501963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:35.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.193+0000 7fb35621e700 1 --2- 172.21.15.82:0/290723837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3500ff700 0x7fb3501963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.193+0000 7fb35621e700 1 -- 172.21.15.82:0/290723837 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3501000c0 msgr2=0x7fb350196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:35.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.193+0000 7fb35621e700 1 --2- 172.21.15.82:0/290723837 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3501000c0 0x7fb350196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.193+0000 7fb35621e700 1 -- 172.21.15.82:0/290723837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb348009d00 con 0x7fb350100ab0 2023-09-09T23:13:35.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.193+0000 7fb35621e700 1 --2- 172.21.15.82:0/290723837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb350100ab0 0x7fb35019d990 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb34800b830 tx=0x7fb348004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:35.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.193+0000 7fb346ffd700 1 -- 172.21.15.82:0/290723837 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34800e040 con 0x7fb350100ab0 2023-09-09T23:13:35.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.193+0000 7fb357c81700 1 -- 172.21.15.82:0/290723837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb35019e4b0 con 0x7fb350100ab0 2023-09-09T23:13:35.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.193+0000 7fb346ffd700 1 -- 172.21.15.82:0/290723837 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3480094c0 con 0x7fb350100ab0 2023-09-09T23:13:35.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.193+0000 7fb346ffd700 1 -- 172.21.15.82:0/290723837 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34801bc70 con 0x7fb350100ab0 2023-09-09T23:13:35.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.193+0000 7fb357c81700 1 -- 172.21.15.82:0/290723837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb35019e930 con 0x7fb350100ab0 2023-09-09T23:13:35.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.195+0000 7fb346ffd700 1 -- 172.21.15.82:0/290723837 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7fb34801d890 con 0x7fb350100ab0 2023-09-09T23:13:35.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.196+0000 7fb346ffd700 1 --2- 172.21.15.82:0/290723837 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb33c0a06d0 0x7fb33c0a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.196+0000 7fb355a1d700 1 --2- 172.21.15.82:0/290723837 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb33c0a06d0 0x7fb33c0a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.196+0000 7fb346ffd700 1 -- 172.21.15.82:0/290723837 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb3480cdc30 con 0x7fb350100ab0 2023-09-09T23:13:35.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.196+0000 7fb355a1d700 1 --2- 172.21.15.82:0/290723837 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb33c0a06d0 0x7fb33c0a2b90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb340005950 tx=0x7fb3400058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:35.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.197+0000 7fb363e47b80 1 Processor -- start 2023-09-09T23:13:35.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.197+0000 7fb363e47b80 1 -- start start 2023-09-09T23:13:35.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.197+0000 7fb363e47b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562eeb43a020 0x562eeb43a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.197+0000 7fb363e47b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562eeb43a980 0x562eeb438c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.197+0000 7fb363e47b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562eeb4391e0 0x562eeb439660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.197+0000 7fb363e47b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eeb38d6e0 con 0x562eeb4391e0 2023-09-09T23:13:35.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.197+0000 7fb363e47b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eeb38b000 con 0x562eeb43a980 2023-09-09T23:13:35.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.197+0000 7fb363e47b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eeb3868f0 con 0x562eeb43a020 2023-09-09T23:13:35.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.197+0000 7fb35621e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562eeb4391e0 0x562eeb439660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.197+0000 7fb35621e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562eeb4391e0 0x562eeb439660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55846/0 (socket says 172.21.15.82:55846) 2023-09-09T23:13:35.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.197+0000 7fb35621e700 1 -- 172.21.15.82:0/1267170955 learned_addr learned my addr 172.21.15.82:0/1267170955 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:35.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.197+0000 7fb355a1d700 1 --2- 172.21.15.82:0/1267170955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562eeb43a020 0x562eeb43a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.197+0000 7fb35521c700 1 --2- 172.21.15.82:0/1267170955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562eeb43a980 0x562eeb438c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.197+0000 7fb35621e700 1 -- 172.21.15.82:0/1267170955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562eeb43a020 msgr2=0x562eeb43a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:35.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.197+0000 7fb35621e700 1 --2- 172.21.15.82:0/1267170955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562eeb43a020 0x562eeb43a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.197+0000 7fb35621e700 1 -- 172.21.15.82:0/1267170955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562eeb43a980 msgr2=0x562eeb438c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:35.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.197+0000 7fb35621e700 1 --2- 172.21.15.82:0/1267170955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562eeb43a980 0x562eeb438c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.197+0000 7fb35621e700 1 -- 172.21.15.82:0/1267170955 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562eeb1b2d90 con 0x562eeb4391e0 2023-09-09T23:13:35.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.197+0000 7fb35521c700 1 --2- 172.21.15.82:0/1267170955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562eeb43a980 0x562eeb438c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:35.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.197+0000 7fb35621e700 1 --2- 172.21.15.82:0/1267170955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562eeb4391e0 0x562eeb439660 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb348097850 tx=0x7fb34801b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60613e861f1c4fd7 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:35.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.198+0000 7fb33bfff700 1 -- 172.21.15.82:0/1267170955 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb348086070 con 0x562eeb4391e0 2023-09-09T23:13:35.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.198+0000 7fb33bfff700 1 -- 172.21.15.82:0/1267170955 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb34809b350 con 0x562eeb4391e0 2023-09-09T23:13:35.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.198+0000 7fb33bfff700 1 -- 172.21.15.82:0/1267170955 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34801db40 con 0x562eeb4391e0 2023-09-09T23:13:35.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.198+0000 7fb363e47b80 1 -- 172.21.15.82:0/1267170955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562eeb4391e0 msgr2=0x562eeb439660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:35.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.198+0000 7fb363e47b80 1 --2- 172.21.15.82:0/1267170955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562eeb4391e0 0x562eeb439660 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb348097850 tx=0x7fb34801b710 comp rx=0 tx=0).stop 2023-09-09T23:13:35.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.198+0000 7fb363e47b80 1 -- 172.21.15.82:0/1267170955 shutdown_connections 2023-09-09T23:13:35.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.198+0000 7fb363e47b80 1 --2- 172.21.15.82:0/1267170955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562eeb4391e0 0x562eeb439660 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.198+0000 7fb363e47b80 1 --2- 172.21.15.82:0/1267170955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562eeb43a020 0x562eeb43a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.198+0000 7fb363e47b80 1 --2- 172.21.15.82:0/1267170955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562eeb43a980 0x562eeb438c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.198+0000 7fb363e47b80 1 -- 172.21.15.82:0/1267170955 >> 172.21.15.82:0/1267170955 conn(0x562eeb3a9040 msgr2=0x562eeb42f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:35.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.198+0000 7fb363e47b80 1 -- 172.21.15.82:0/1267170955 shutdown_connections 2023-09-09T23:13:35.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.198+0000 7fb363e47b80 1 -- 172.21.15.82:0/1267170955 wait complete. 2023-09-09T23:13:35.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.199+0000 7fb363e47b80 1 -- start start 2023-09-09T23:13:35.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.199+0000 7fb363e47b80 1 Processor -- start 2023-09-09T23:13:35.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.199+0000 7fb363e47b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562eeb4391e0 0x562eeb442600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.199+0000 7fb363e47b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562eeb43a020 0x562eeb442b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.199+0000 7fb363e47b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562eeb43a980 0x562eeb449b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.199+0000 7fb363e47b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eeb324b20 con 0x562eeb43a020 2023-09-09T23:13:35.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.199+0000 7fb363e47b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eeb324820 con 0x562eeb4391e0 2023-09-09T23:13:35.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.199+0000 7fb363e47b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eeb2088d0 con 0x562eeb43a980 2023-09-09T23:13:35.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.200+0000 7fb35521c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562eeb43a020 0x562eeb442b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.200+0000 7fb363e47b80 10 client.0 _ll_register_callbacks cb 0x562eeb26ea30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:35.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.200+0000 7fb35521c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562eeb43a020 0x562eeb442b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55850/0 (socket says 172.21.15.82:55850) 2023-09-09T23:13:35.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.200+0000 7fb35521c700 1 -- 172.21.15.82:0/323171474 learned_addr learned my addr 172.21.15.82:0/323171474 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:35.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.200+0000 7fb35621e700 1 --2- 172.21.15.82:0/323171474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562eeb43a980 0x562eeb449b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.200+0000 7fb355a1d700 1 --2- 172.21.15.82:0/323171474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562eeb4391e0 0x562eeb442600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.200+0000 7fb35521c700 1 -- 172.21.15.82:0/323171474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562eeb43a980 msgr2=0x562eeb449b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:35.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.200+0000 7fb35521c700 1 --2- 172.21.15.82:0/323171474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562eeb43a980 0x562eeb449b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.200+0000 7fb35521c700 1 -- 172.21.15.82:0/323171474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562eeb4391e0 msgr2=0x562eeb442600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:35.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.200+0000 7fb35521c700 1 --2- 172.21.15.82:0/323171474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562eeb4391e0 0x562eeb442600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.200+0000 7fb35521c700 1 -- 172.21.15.82:0/323171474 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb35019e930 con 0x562eeb43a020 2023-09-09T23:13:35.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.200+0000 7fb35521c700 1 --2- 172.21.15.82:0/323171474 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562eeb43a020 0x562eeb442b40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00f360 tx=0x7fb34c00f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba56f9c05f06b49 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:35.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.200+0000 7fb3397fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:35.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.200+0000 7fb3397fa700 1 -- 172.21.15.82:0/323171474 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c011850 con 0x562eeb43a020 2023-09-09T23:13:35.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.200+0000 7fb3397fa700 1 -- 172.21.15.82:0/323171474 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb34c015070 con 0x562eeb43a020 2023-09-09T23:13:35.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.200+0000 7fb363e47b80 1 -- 172.21.15.82:0/323171474 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562eeb2de4a0 con 0x562eeb43a020 2023-09-09T23:13:35.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.200+0000 7fb3397fa700 1 -- 172.21.15.82:0/323171474 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb34c00fe50 con 0x562eeb43a020 2023-09-09T23:13:35.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.200+0000 7fb363e47b80 10 client.15396 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:35.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.200+0000 7fb363e47b80 1 -- 172.21.15.82:0/323171474 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562eeb19bc10 con 0x562eeb43a020 2023-09-09T23:13:35.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.200+0000 7fb3397fa700 1 -- 172.21.15.82:0/323171474 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562eeb2de4a0 con 0x562eeb43a020 2023-09-09T23:13:35.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.201+0000 7fb3397fa700 1 -- 172.21.15.82:0/323171474 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb34c01aba0 con 0x562eeb43a020 2023-09-09T23:13:35.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.201+0000 7fb363e47b80 10 client.15396 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:35.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.201+0000 7fb363e47b80 10 client.15396 resolve_mds: resolved name 'h' to daemon mds.h 2023-09-09T23:13:35.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.201+0000 7fb363e47b80 20 client.15396 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:35.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.201+0000 7fb363e47b80 1 --2- 172.21.15.82:0/323171474 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x562eeb44dcc0 0x562eeb44cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.201+0000 7fb363e47b80 4 client.15396 mds_command: new command op to 24542 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:13:35.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.201+0000 7fb363e47b80 1 -- 172.21.15.82:0/323171474 --> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562eeb39b850 con 0x562eeb44dcc0 2023-09-09T23:13:35.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.201+0000 7fb355a1d700 1 --2- 172.21.15.82:0/323171474 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x562eeb44dcc0 0x562eeb44cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.201+0000 7fb3397fa700 10 client.15396 ms_handle_connect on v2:172.21.15.98:6836/2019449840 2023-09-09T23:13:35.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.201+0000 7fb355a1d700 1 --2- 172.21.15.82:0/323171474 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x562eeb44dcc0 0x562eeb44cca0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c8d01f408c2a689e server_cookie=385d25eef5570133 in_seq=0 out_seq=0 2023-09-09T23:13:35.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.202+0000 7fb3397fa700 1 -- 172.21.15.82:0/323171474 <== mds.1 v2:172.21.15.98:6836/2019449840 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562eeb39b850 con 0x562eeb44dcc0 2023-09-09T23:13:35.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.202+0000 7fb3397fa700 10 client.15396 handle_command_reply: tid=0 2023-09-09T23:13:35.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.202+0000 7fb363e47b80 1 client.15396 shutdown 2023-09-09T23:13:35.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.202+0000 7fb363e47b80 1 -- 172.21.15.82:0/323171474 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562eeb43a020 msgr2=0x562eeb442b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:35.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.202+0000 7fb363e47b80 1 --2- 172.21.15.82:0/323171474 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562eeb43a020 0x562eeb442b40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00f360 tx=0x7fb34c00f720 comp rx=0 tx=0).stop 2023-09-09T23:13:35.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.202+0000 7fb363e47b80 1 -- 172.21.15.82:0/323171474 shutdown_connections 2023-09-09T23:13:35.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.202+0000 7fb363e47b80 1 --2- 172.21.15.82:0/323171474 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x562eeb44dcc0 0x562eeb44cca0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.202+0000 7fb363e47b80 1 --2- 172.21.15.82:0/323171474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562eeb4391e0 0x562eeb442600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.202+0000 7fb3397fa700 0 client.15396 ms_handle_reset on v2:172.21.15.98:6836/2019449840 2023-09-09T23:13:35.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.202+0000 7fb363e47b80 1 --2- 172.21.15.82:0/323171474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562eeb43a980 0x562eeb449b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.202+0000 7fb363e47b80 1 --2- 172.21.15.82:0/323171474 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562eeb43a020 0x562eeb442b40 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.202+0000 7fb363e47b80 1 -- 172.21.15.82:0/323171474 >> 172.21.15.82:0/323171474 conn(0x562eeb3a9040 msgr2=0x562eeb42ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:35.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.202+0000 7fb363e47b80 1 -- 172.21.15.82:0/323171474 shutdown_connections 2023-09-09T23:13:35.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.202+0000 7fb363e47b80 1 -- 172.21.15.82:0/323171474 wait complete. 2023-09-09T23:13:35.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.202+0000 7fb363e47b80 20 client.15396 trim_cache size 0 max 16384 2023-09-09T23:13:35.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.220+0000 7fb363e47b80 1 Processor -- start 2023-09-09T23:13:35.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.220+0000 7fb363e47b80 1 -- start start 2023-09-09T23:13:35.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.220+0000 7fb363e47b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562eeb4391e0 0x562eeb44cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.220+0000 7fb363e47b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562eeb43a020 0x562eeb449b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.220+0000 7fb363e47b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562eeb43a980 0x562eeb440770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.220+0000 7fb363e47b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eeb430620 con 0x562eeb4391e0 2023-09-09T23:13:35.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.220+0000 7fb363e47b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eeb39b880 con 0x562eeb43a980 2023-09-09T23:13:35.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.220+0000 7fb363e47b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eeb214de0 con 0x562eeb43a020 2023-09-09T23:13:35.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.220+0000 7fb355a1d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562eeb4391e0 0x562eeb44cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.220+0000 7fb355a1d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562eeb4391e0 0x562eeb44cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55856/0 (socket says 172.21.15.82:55856) 2023-09-09T23:13:35.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.220+0000 7fb355a1d700 1 -- 172.21.15.82:0/386462416 learned_addr learned my addr 172.21.15.82:0/386462416 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:35.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.220+0000 7fb35521c700 1 --2- 172.21.15.82:0/386462416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562eeb43a020 0x562eeb449b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.220+0000 7fb355a1d700 1 -- 172.21.15.82:0/386462416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562eeb43a020 msgr2=0x562eeb449b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:35.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.220+0000 7fb355a1d700 1 --2- 172.21.15.82:0/386462416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562eeb43a020 0x562eeb449b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.220+0000 7fb35621e700 1 --2- 172.21.15.82:0/386462416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562eeb43a980 0x562eeb440770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.220+0000 7fb355a1d700 1 -- 172.21.15.82:0/386462416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562eeb43a980 msgr2=0x562eeb440770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:35.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.220+0000 7fb355a1d700 1 --2- 172.21.15.82:0/386462416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562eeb43a980 0x562eeb440770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.220+0000 7fb355a1d700 1 -- 172.21.15.82:0/386462416 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562eeb1d9200 con 0x562eeb4391e0 2023-09-09T23:13:35.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.220+0000 7fb35621e700 1 --2- 172.21.15.82:0/386462416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562eeb43a980 0x562eeb440770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:35.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.221+0000 7fb355a1d700 1 --2- 172.21.15.82:0/386462416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562eeb4391e0 0x562eeb44cca0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb340005d90 tx=0x7fb340013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed2e51c54363e32 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:35.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.221+0000 7fb344d89700 1 -- 172.21.15.82:0/386462416 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb340024070 con 0x562eeb4391e0 2023-09-09T23:13:35.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.221+0000 7fb344d89700 1 -- 172.21.15.82:0/386462416 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb340025e90 con 0x562eeb4391e0 2023-09-09T23:13:35.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.221+0000 7fb344d89700 1 -- 172.21.15.82:0/386462416 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb340010d60 con 0x562eeb4391e0 2023-09-09T23:13:35.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.221+0000 7fb363e47b80 1 -- 172.21.15.82:0/386462416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562eeb4391e0 msgr2=0x562eeb44cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:35.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.221+0000 7fb363e47b80 1 --2- 172.21.15.82:0/386462416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562eeb4391e0 0x562eeb44cca0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb340005d90 tx=0x7fb340013000 comp rx=0 tx=0).stop 2023-09-09T23:13:35.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.221+0000 7fb363e47b80 1 -- 172.21.15.82:0/386462416 shutdown_connections 2023-09-09T23:13:35.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.221+0000 7fb363e47b80 1 --2- 172.21.15.82:0/386462416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562eeb43a980 0x562eeb440770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.221+0000 7fb363e47b80 1 --2- 172.21.15.82:0/386462416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562eeb4391e0 0x562eeb44cca0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.221+0000 7fb363e47b80 1 --2- 172.21.15.82:0/386462416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562eeb43a020 0x562eeb449b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.221+0000 7fb363e47b80 1 -- 172.21.15.82:0/386462416 >> 172.21.15.82:0/386462416 conn(0x562eeb3a9040 msgr2=0x562eeb4395c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:35.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.221+0000 7fb363e47b80 1 -- 172.21.15.82:0/386462416 shutdown_connections 2023-09-09T23:13:35.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.221+0000 7fb363e47b80 1 -- 172.21.15.82:0/386462416 wait complete. 2023-09-09T23:13:35.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.222+0000 7fb363e47b80 1 -- start start 2023-09-09T23:13:35.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.222+0000 7fb363e47b80 1 Processor -- start 2023-09-09T23:13:35.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.222+0000 7fb363e47b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562eeb4391e0 0x562eeb444db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.222+0000 7fb363e47b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562eeb43a020 0x562eeb4452f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.222+0000 7fb363e47b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562eeb43a980 0x562eeb448310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.222+0000 7fb363e47b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eeb448850 con 0x562eeb43a980 2023-09-09T23:13:35.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.222+0000 7fb363e47b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eeb4489c0 con 0x562eeb4391e0 2023-09-09T23:13:35.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.222+0000 7fb363e47b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eeb448b80 con 0x562eeb43a020 2023-09-09T23:13:35.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.222+0000 7fb363e47b80 10 client.0 _ll_register_callbacks cb 0x7fb34c00a840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:35.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.222+0000 7fb35621e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562eeb43a980 0x562eeb448310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.222+0000 7fb35621e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562eeb43a980 0x562eeb448310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55866/0 (socket says 172.21.15.82:55866) 2023-09-09T23:13:35.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.222+0000 7fb35621e700 1 -- 172.21.15.82:0/3869545946 learned_addr learned my addr 172.21.15.82:0/3869545946 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:35.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.222+0000 7fb35521c700 1 --2- 172.21.15.82:0/3869545946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562eeb43a020 0x562eeb4452f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.222+0000 7fb355a1d700 1 --2- 172.21.15.82:0/3869545946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562eeb4391e0 0x562eeb444db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.222+0000 7fb35621e700 1 -- 172.21.15.82:0/3869545946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562eeb43a020 msgr2=0x562eeb4452f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:35.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.222+0000 7fb35621e700 1 --2- 172.21.15.82:0/3869545946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562eeb43a020 0x562eeb4452f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.222+0000 7fb35621e700 1 -- 172.21.15.82:0/3869545946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562eeb4391e0 msgr2=0x562eeb444db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:35.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.222+0000 7fb35621e700 1 --2- 172.21.15.82:0/3869545946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562eeb4391e0 0x562eeb444db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.222+0000 7fb35621e700 1 -- 172.21.15.82:0/3869545946 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34000dcb0 con 0x562eeb43a980 2023-09-09T23:13:35.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.222+0000 7fb355a1d700 1 --2- 172.21.15.82:0/3869545946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562eeb4391e0 0x562eeb444db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:35.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.222+0000 7fb35521c700 1 --2- 172.21.15.82:0/3869545946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562eeb43a020 0x562eeb4452f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:35.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.223+0000 7fb35621e700 1 --2- 172.21.15.82:0/3869545946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562eeb43a980 0x562eeb448310 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb3480a2e00 tx=0x7fb34800b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958686988fc55680 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:35.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.223+0000 7fb3397fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:35.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.223+0000 7fb3397fa700 1 -- 172.21.15.82:0/3869545946 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3480a3330 con 0x562eeb43a980 2023-09-09T23:13:35.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.223+0000 7fb3397fa700 1 -- 172.21.15.82:0/3869545946 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb34809f180 con 0x562eeb43a980 2023-09-09T23:13:35.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.223+0000 7fb363e47b80 1 -- 172.21.15.82:0/3869545946 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb34809b350 con 0x562eeb43a980 2023-09-09T23:13:35.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.223+0000 7fb3397fa700 1 -- 172.21.15.82:0/3869545946 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb34801b450 con 0x562eeb43a980 2023-09-09T23:13:35.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.223+0000 7fb363e47b80 10 client.15402 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:35.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.223+0000 7fb363e47b80 1 -- 172.21.15.82:0/3869545946 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562eeb1b1260 con 0x562eeb43a980 2023-09-09T23:13:35.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.223+0000 7fb3397fa700 1 -- 172.21.15.82:0/3869545946 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb34809b350 con 0x562eeb43a980 2023-09-09T23:13:35.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.223+0000 7fb3397fa700 1 -- 172.21.15.82:0/3869545946 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb3480aec70 con 0x562eeb43a980 2023-09-09T23:13:35.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.223+0000 7fb363e47b80 10 client.15402 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:35.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.223+0000 7fb363e47b80 10 client.15402 resolve_mds: resolved name 'h' to daemon mds.h 2023-09-09T23:13:35.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.223+0000 7fb363e47b80 20 client.15402 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:35.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.223+0000 7fb363e47b80 1 --2- 172.21.15.82:0/3869545946 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x562eeb4a7680 0x562eeb4a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.223+0000 7fb363e47b80 4 client.15402 mds_command: new command op to 24542 tid=0[{"prefix": "status"}] 2023-09-09T23:13:35.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.223+0000 7fb363e47b80 1 -- 172.21.15.82:0/3869545946 --> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562eeb37f8d0 con 0x562eeb4a7680 2023-09-09T23:13:35.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.223+0000 7fb355a1d700 1 --2- 172.21.15.82:0/3869545946 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x562eeb4a7680 0x562eeb4a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.224+0000 7fb3397fa700 10 client.15402 ms_handle_connect on v2:172.21.15.98:6836/2019449840 2023-09-09T23:13:35.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.224+0000 7fb355a1d700 1 --2- 172.21.15.82:0/3869545946 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x562eeb4a7680 0x562eeb4a3d20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d497cbeea13f68ff server_cookie=e59926263cce36a1 in_seq=0 out_seq=0 2023-09-09T23:13:35.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.224+0000 7fb3397fa700 1 -- 172.21.15.82:0/3869545946 <== mds.1 v2:172.21.15.98:6836/2019449840 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x562eeb37f8d0 con 0x562eeb4a7680 2023-09-09T23:13:35.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.224+0000 7fb3397fa700 10 client.15402 handle_command_reply: tid=0 2023-09-09T23:13:35.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.224+0000 7fb363e47b80 1 client.15402 shutdown 2023-09-09T23:13:35.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.224+0000 7fb363e47b80 1 -- 172.21.15.82:0/3869545946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562eeb43a980 msgr2=0x562eeb448310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:35.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.224+0000 7fb363e47b80 1 --2- 172.21.15.82:0/3869545946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562eeb43a980 0x562eeb448310 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb3480a2e00 tx=0x7fb34800b3b0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.224+0000 7fb363e47b80 1 -- 172.21.15.82:0/3869545946 shutdown_connections 2023-09-09T23:13:35.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.224+0000 7fb363e47b80 1 --2- 172.21.15.82:0/3869545946 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x562eeb4a7680 0x562eeb4a3d20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.224+0000 7fb363e47b80 1 --2- 172.21.15.82:0/3869545946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562eeb4391e0 0x562eeb444db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.224+0000 7fb363e47b80 1 --2- 172.21.15.82:0/3869545946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562eeb43a980 0x562eeb448310 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.224+0000 7fb3397fa700 0 client.15402 ms_handle_reset on v2:172.21.15.98:6836/2019449840 2023-09-09T23:13:35.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.224+0000 7fb363e47b80 1 --2- 172.21.15.82:0/3869545946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562eeb43a020 0x562eeb4452f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.224+0000 7fb363e47b80 1 -- 172.21.15.82:0/3869545946 >> 172.21.15.82:0/3869545946 conn(0x562eeb3a9040 msgr2=0x562eeb44cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:35.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.224+0000 7fb363e47b80 1 -- 172.21.15.82:0/3869545946 shutdown_connections 2023-09-09T23:13:35.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.224+0000 7fb363e47b80 1 -- 172.21.15.82:0/3869545946 wait complete. 2023-09-09T23:13:35.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.224+0000 7fb363e47b80 20 client.15402 trim_cache size 0 max 16384 2023-09-09T23:13:35.250 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:13:35.250 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da", 2023-09-09T23:13:35.250 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 1, 2023-09-09T23:13:35.251 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24542, 2023-09-09T23:13:35.251 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2023-09-09T23:13:35.251 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2023-09-09T23:13:35.251 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2023-09-09T23:13:35.251 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 73.660538596999999, 2023-09-09T23:13:35.251 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 27, 2023-09-09T23:13:35.252 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 83, 2023-09-09T23:13:35.252 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 81, 2023-09-09T23:13:35.252 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 92.360778448999994 2023-09-09T23:13:35.252 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:13:35.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.225+0000 7fb357c81700 1 -- 172.21.15.82:0/290723837 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb33c0a06d0 msgr2=0x7fb33c0a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:35.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.225+0000 7fb357c81700 1 --2- 172.21.15.82:0/290723837 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb33c0a06d0 0x7fb33c0a2b90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb340005950 tx=0x7fb3400058e0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.225+0000 7fb357c81700 1 -- 172.21.15.82:0/290723837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb350100ab0 msgr2=0x7fb35019d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:35.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.225+0000 7fb357c81700 1 --2- 172.21.15.82:0/290723837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb350100ab0 0x7fb35019d990 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb34800b830 tx=0x7fb348004d30 comp rx=0 tx=0).stop 2023-09-09T23:13:35.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.225+0000 7fb357c81700 1 -- 172.21.15.82:0/290723837 shutdown_connections 2023-09-09T23:13:35.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.225+0000 7fb357c81700 1 --2- 172.21.15.82:0/290723837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb350100ab0 0x7fb35019d990 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.225+0000 7fb357c81700 1 --2- 172.21.15.82:0/290723837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3500ff700 0x7fb3501963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.225+0000 7fb357c81700 1 --2- 172.21.15.82:0/290723837 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb33c0a06d0 0x7fb33c0a2b90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.225+0000 7fb357c81700 1 --2- 172.21.15.82:0/290723837 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3501000c0 0x7fb350196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.225+0000 7fb357c81700 1 -- 172.21.15.82:0/290723837 >> 172.21.15.82:0/290723837 conn(0x7fb3500fa1d0 msgr2=0x7fb3500fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:35.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.225+0000 7fb357c81700 1 -- 172.21.15.82:0/290723837 shutdown_connections 2023-09-09T23:13:35.258 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.225+0000 7fb357c81700 1 -- 172.21.15.82:0/290723837 wait complete. 2023-09-09T23:13:35.258 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-09-09T23:13:35.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.386+0000 7ff62f306700 1 -- 172.21.15.82:0/3299888319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff628100a70 msgr2=0x7ff628100e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:35.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.386+0000 7ff62f306700 1 --2- 172.21.15.82:0/3299888319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff628100a70 0x7ff628100e90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff624009990 tx=0x7ff624009ca0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.386+0000 7ff62f306700 1 -- 172.21.15.82:0/3299888319 shutdown_connections 2023-09-09T23:13:35.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.386+0000 7ff62f306700 1 --2- 172.21.15.82:0/3299888319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff628100140 0x7ff628107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.388 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.386+0000 7ff62f306700 1 --2- 172.21.15.82:0/3299888319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff628100a70 0x7ff628100e90 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.388 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.386+0000 7ff62f306700 1 --2- 172.21.15.82:0/3299888319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6280ff7a0 0x7ff6280ffc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.388 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.386+0000 7ff62f306700 1 -- 172.21.15.82:0/3299888319 >> 172.21.15.82:0/3299888319 conn(0x7ff6280fa1d0 msgr2=0x7ff6280fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:35.388 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.386+0000 7ff62f306700 1 -- 172.21.15.82:0/3299888319 shutdown_connections 2023-09-09T23:13:35.388 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.386+0000 7ff62f306700 1 -- 172.21.15.82:0/3299888319 wait complete. 2023-09-09T23:13:35.389 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.387+0000 7ff62f306700 1 Processor -- start 2023-09-09T23:13:35.389 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.387+0000 7ff62f306700 1 -- start start 2023-09-09T23:13:35.389 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.387+0000 7ff62f306700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6280ff7a0 0x7ff6280ff200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.389 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.387+0000 7ff62f306700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff628100140 0x7ff62819b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.389 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.387+0000 7ff62f306700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff628100a70 0x7ff62819d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.390 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.387+0000 7ff62f306700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62819de70 con 0x7ff628100140 2023-09-09T23:13:35.390 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.387+0000 7ff62f306700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62819dfe0 con 0x7ff6280ff7a0 2023-09-09T23:13:35.390 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.387+0000 7ff62f306700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62819e1a0 con 0x7ff628100a70 2023-09-09T23:13:35.390 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.387+0000 7ff62c8a1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff628100140 0x7ff62819b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.390 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.387+0000 7ff62c8a1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff628100140 0x7ff62819b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55878/0 (socket says 172.21.15.82:55878) 2023-09-09T23:13:35.391 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.387+0000 7ff62c8a1700 1 -- 172.21.15.82:0/2617365152 learned_addr learned my addr 172.21.15.82:0/2617365152 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:35.391 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.387+0000 7ff62d8a3700 1 --2- 172.21.15.82:0/2617365152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff628100a70 0x7ff62819d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.391 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.387+0000 7ff62d0a2700 1 --2- 172.21.15.82:0/2617365152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6280ff7a0 0x7ff6280ff200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.391 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.387+0000 7ff62c8a1700 1 -- 172.21.15.82:0/2617365152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff628100a70 msgr2=0x7ff62819d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:35.391 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.387+0000 7ff62c8a1700 1 --2- 172.21.15.82:0/2617365152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff628100a70 0x7ff62819d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.392 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.387+0000 7ff62c8a1700 1 -- 172.21.15.82:0/2617365152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6280ff7a0 msgr2=0x7ff6280ff200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:35.392 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.387+0000 7ff62c8a1700 1 --2- 172.21.15.82:0/2617365152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6280ff7a0 0x7ff6280ff200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.392 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.387+0000 7ff62c8a1700 1 -- 172.21.15.82:0/2617365152 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6240095f0 con 0x7ff628100140 2023-09-09T23:13:35.392 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.387+0000 7ff62c8a1700 1 --2- 172.21.15.82:0/2617365152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff628100140 0x7ff62819b090 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00d950 tx=0x7ff61c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:35.392 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.388+0000 7ff61a7fc700 1 -- 172.21.15.82:0/2617365152 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c009940 con 0x7ff628100140 2023-09-09T23:13:35.393 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.388+0000 7ff62f306700 1 -- 172.21.15.82:0/2617365152 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff62819e480 con 0x7ff628100140 2023-09-09T23:13:35.393 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.388+0000 7ff61a7fc700 1 -- 172.21.15.82:0/2617365152 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff61c010460 con 0x7ff628100140 2023-09-09T23:13:35.393 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.388+0000 7ff61a7fc700 1 -- 172.21.15.82:0/2617365152 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c00f620 con 0x7ff628100140 2023-09-09T23:13:35.393 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.388+0000 7ff62f306700 1 -- 172.21.15.82:0/2617365152 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff62819e990 con 0x7ff628100140 2023-09-09T23:13:35.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.389+0000 7ff61a7fc700 1 -- 172.21.15.82:0/2617365152 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7ff61c0108d0 con 0x7ff628100140 2023-09-09T23:13:35.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.390+0000 7ff61a7fc700 1 --2- 172.21.15.82:0/2617365152 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff6140a06d0 0x7ff6140a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.390+0000 7ff62d0a2700 1 --2- 172.21.15.82:0/2617365152 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff6140a06d0 0x7ff6140a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.390+0000 7ff61a7fc700 1 -- 172.21.15.82:0/2617365152 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff61c0cba80 con 0x7ff628100140 2023-09-09T23:13:35.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.390+0000 7ff62d0a2700 1 --2- 172.21.15.82:0/2617365152 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff6140a06d0 0x7ff6140a2b90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff624006010 tx=0x7ff62400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:35.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.391+0000 7ff63b4ccb80 1 Processor -- start 2023-09-09T23:13:35.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.391+0000 7ff63b4ccb80 1 -- start start 2023-09-09T23:13:35.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.391+0000 7ff63b4ccb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560008d95020 0x560008d95440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.391+0000 7ff63b4ccb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560008d95980 0x560008d93c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.391+0000 7ff63b4ccb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560008d941e0 0x560008d94660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.391+0000 7ff63b4ccb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560008ce86e0 con 0x560008d95020 2023-09-09T23:13:35.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.391+0000 7ff63b4ccb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x560008ce6000 con 0x560008d95980 2023-09-09T23:13:35.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.391+0000 7ff63b4ccb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560008ce18f0 con 0x560008d941e0 2023-09-09T23:13:35.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.391+0000 7ff62d0a2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560008d95020 0x560008d95440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.392+0000 7ff62d0a2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560008d95020 0x560008d95440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55884/0 (socket says 172.21.15.82:55884) 2023-09-09T23:13:35.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.392+0000 7ff62d0a2700 1 -- 172.21.15.82:0/1439265631 learned_addr learned my addr 172.21.15.82:0/1439265631 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:35.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.392+0000 7ff62c8a1700 1 --2- 172.21.15.82:0/1439265631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560008d95980 0x560008d93c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.392+0000 7ff62d8a3700 1 --2- 172.21.15.82:0/1439265631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560008d941e0 0x560008d94660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.392+0000 7ff62d0a2700 1 -- 172.21.15.82:0/1439265631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560008d941e0 msgr2=0x560008d94660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:35.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.392+0000 7ff62d0a2700 1 --2- 172.21.15.82:0/1439265631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560008d941e0 0x560008d94660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.392+0000 7ff62d0a2700 1 -- 172.21.15.82:0/1439265631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560008d95980 msgr2=0x560008d93c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:35.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.392+0000 7ff62d0a2700 1 --2- 172.21.15.82:0/1439265631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560008d95980 0x560008d93c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.392+0000 7ff62d0a2700 1 -- 172.21.15.82:0/1439265631 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560008b0dd90 con 0x560008d95020 2023-09-09T23:13:35.407 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.392+0000 7ff62d0a2700 1 --2- 172.21.15.82:0/1439265631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560008d95020 0x560008d95440 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff624015000 tx=0x7ff62401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4feb2efe76ce25 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:35.407 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.392+0000 7ff6137fe700 1 -- 172.21.15.82:0/1439265631 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62402b070 con 0x560008d95020 2023-09-09T23:13:35.407 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.392+0000 7ff6137fe700 1 -- 172.21.15.82:0/1439265631 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff62401e040 con 0x560008d95020 2023-09-09T23:13:35.408 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.392+0000 7ff6137fe700 1 -- 172.21.15.82:0/1439265631 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6240247d0 con 0x560008d95020 2023-09-09T23:13:35.408 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.393+0000 7ff63b4ccb80 1 -- 172.21.15.82:0/1439265631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560008d95020 msgr2=0x560008d95440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:35.408 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.393+0000 7ff63b4ccb80 1 --2- 172.21.15.82:0/1439265631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560008d95020 0x560008d95440 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff624015000 tx=0x7ff62401f000 comp rx=0 tx=0).stop 2023-09-09T23:13:35.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.393+0000 7ff63b4ccb80 1 -- 172.21.15.82:0/1439265631 shutdown_connections 2023-09-09T23:13:35.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.393+0000 7ff63b4ccb80 1 --2- 172.21.15.82:0/1439265631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560008d941e0 0x560008d94660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.410 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.393+0000 7ff63b4ccb80 1 --2- 172.21.15.82:0/1439265631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560008d95020 0x560008d95440 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.410 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.393+0000 7ff63b4ccb80 1 --2- 172.21.15.82:0/1439265631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560008d95980 0x560008d93c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.410 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.393+0000 7ff63b4ccb80 1 -- 172.21.15.82:0/1439265631 >> 172.21.15.82:0/1439265631 conn(0x560008d04040 msgr2=0x560008d8a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:35.411 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.393+0000 7ff63b4ccb80 1 -- 172.21.15.82:0/1439265631 shutdown_connections 2023-09-09T23:13:35.411 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.393+0000 7ff63b4ccb80 1 -- 172.21.15.82:0/1439265631 wait complete. 2023-09-09T23:13:35.411 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.393+0000 7ff63b4ccb80 1 -- start start 2023-09-09T23:13:35.411 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.393+0000 7ff63b4ccb80 1 Processor -- start 2023-09-09T23:13:35.411 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.394+0000 7ff63b4ccb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560008d941e0 0x560008d9d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.394+0000 7ff63b4ccb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560008d95020 0x560008d9de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.394+0000 7ff63b4ccb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560008d95980 0x560008da4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.394+0000 7ff63b4ccb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560008c7fb20 con 0x560008d95020 2023-09-09T23:13:35.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.394+0000 7ff63b4ccb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x560008c7f820 con 0x560008d95980 2023-09-09T23:13:35.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.394+0000 7ff63b4ccb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560008b638d0 con 0x560008d941e0 2023-09-09T23:13:35.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.394+0000 7ff63b4ccb80 10 client.0 _ll_register_callbacks cb 0x560008bc9a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:35.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.394+0000 7ff62c8a1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560008d95020 0x560008d9de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.394+0000 7ff62c8a1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560008d95020 0x560008d9de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55892/0 (socket says 172.21.15.82:55892) 2023-09-09T23:13:35.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.394+0000 7ff62c8a1700 1 -- 172.21.15.82:0/3321632258 learned_addr learned my addr 172.21.15.82:0/3321632258 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:35.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.394+0000 7ff62d0a2700 1 --2- 172.21.15.82:0/3321632258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560008d941e0 0x560008d9d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.414 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.394+0000 7ff62d8a3700 1 --2- 172.21.15.82:0/3321632258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560008d95980 0x560008da4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.414 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.394+0000 7ff62c8a1700 1 -- 172.21.15.82:0/3321632258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560008d941e0 msgr2=0x560008d9d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:35.414 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.394+0000 7ff62c8a1700 1 --2- 172.21.15.82:0/3321632258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560008d941e0 0x560008d9d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.414 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.394+0000 7ff62c8a1700 1 -- 172.21.15.82:0/3321632258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560008d95980 msgr2=0x560008da4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:35.414 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.394+0000 7ff62c8a1700 1 --2- 172.21.15.82:0/3321632258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560008d95980 0x560008da4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.415 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.394+0000 7ff62c8a1700 1 -- 172.21.15.82:0/3321632258 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff628108270 con 0x560008d95020 2023-09-09T23:13:35.415 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.394+0000 7ff62c8a1700 1 --2- 172.21.15.82:0/3321632258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560008d95020 0x560008d9de20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff61c095710 tx=0x7ff61c099330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0fbaa8891182d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:35.415 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.394+0000 7ff610ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:35.415 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.394+0000 7ff610ff9700 1 -- 172.21.15.82:0/3321632258 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c098700 con 0x560008d95020 2023-09-09T23:13:35.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.394+0000 7ff610ff9700 1 -- 172.21.15.82:0/3321632258 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff61c098860 con 0x560008d95020 2023-09-09T23:13:35.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.394+0000 7ff63b4ccb80 1 -- 172.21.15.82:0/3321632258 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560008c394a0 con 0x560008d95020 2023-09-09T23:13:35.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.394+0000 7ff610ff9700 1 -- 172.21.15.82:0/3321632258 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff61c0a0df0 con 0x560008d95020 2023-09-09T23:13:35.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.394+0000 7ff63b4ccb80 10 client.15414 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:35.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.394+0000 7ff63b4ccb80 1 -- 172.21.15.82:0/3321632258 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560008af6c10 con 0x560008d95020 2023-09-09T23:13:35.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.395+0000 7ff610ff9700 1 -- 172.21.15.82:0/3321632258 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560008c394a0 con 0x560008d95020 2023-09-09T23:13:35.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.395+0000 7ff610ff9700 1 -- 172.21.15.82:0/3321632258 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff61c098170 con 0x560008d95020 2023-09-09T23:13:35.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.395+0000 7ff63b4ccb80 10 client.15414 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:35.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.395+0000 7ff63b4ccb80 10 client.15414 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-09T23:13:35.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.395+0000 7ff63b4ccb80 20 client.15414 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:35.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.395+0000 7ff63b4ccb80 1 --2- 172.21.15.82:0/3321632258 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x560008da8f60 0x560008da7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.418 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.395+0000 7ff63b4ccb80 4 client.15414 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:13:35.418 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.395+0000 7ff63b4ccb80 1 -- 172.21.15.82:0/3321632258 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560008cf6850 con 0x560008da8f60 2023-09-09T23:13:35.418 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.395+0000 7ff62d0a2700 1 --2- 172.21.15.82:0/3321632258 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x560008da8f60 0x560008da7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.418 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.396+0000 7ff610ff9700 10 client.15414 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:13:35.418 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.396+0000 7ff62d0a2700 1 --2- 172.21.15.82:0/3321632258 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x560008da8f60 0x560008da7f60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=209d9ea015bc5f01 server_cookie=87d9c92bb6e8435 in_seq=0 out_seq=0 2023-09-09T23:13:35.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.396+0000 7ff610ff9700 1 -- 172.21.15.82:0/3321632258 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560008cf6850 con 0x560008da8f60 2023-09-09T23:13:35.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.396+0000 7ff610ff9700 10 client.15414 handle_command_reply: tid=0 2023-09-09T23:13:35.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.396+0000 7ff63b4ccb80 1 client.15414 shutdown 2023-09-09T23:13:35.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.396+0000 7ff63b4ccb80 1 -- 172.21.15.82:0/3321632258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560008d95020 msgr2=0x560008d9de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:35.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.396+0000 7ff63b4ccb80 1 --2- 172.21.15.82:0/3321632258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560008d95020 0x560008d9de20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff61c095710 tx=0x7ff61c099330 comp rx=0 tx=0).stop 2023-09-09T23:13:35.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.396+0000 7ff63b4ccb80 1 -- 172.21.15.82:0/3321632258 shutdown_connections 2023-09-09T23:13:35.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.396+0000 7ff63b4ccb80 1 --2- 172.21.15.82:0/3321632258 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x560008da8f60 0x560008da7f60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.396+0000 7ff63b4ccb80 1 --2- 172.21.15.82:0/3321632258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560008d95980 0x560008da4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.396+0000 7ff610ff9700 0 client.15414 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:13:35.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.396+0000 7ff63b4ccb80 1 --2- 172.21.15.82:0/3321632258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560008d941e0 0x560008d9d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.396+0000 7ff63b4ccb80 1 --2- 172.21.15.82:0/3321632258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560008d95020 0x560008d9de20 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.396+0000 7ff63b4ccb80 1 -- 172.21.15.82:0/3321632258 >> 172.21.15.82:0/3321632258 conn(0x560008d04040 msgr2=0x560008d89d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:35.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.397+0000 7ff63b4ccb80 1 -- 172.21.15.82:0/3321632258 shutdown_connections 2023-09-09T23:13:35.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.397+0000 7ff63b4ccb80 1 -- 172.21.15.82:0/3321632258 wait complete. 2023-09-09T23:13:35.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.397+0000 7ff63b4ccb80 20 client.15414 trim_cache size 0 max 16384 2023-09-09T23:13:35.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.414+0000 7ff63b4ccb80 1 Processor -- start 2023-09-09T23:13:35.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.414+0000 7ff63b4ccb80 1 -- start start 2023-09-09T23:13:35.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.414+0000 7ff63b4ccb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560008d941e0 0x560008da7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.415+0000 7ff63b4ccb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560008d95020 0x560008da4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.415+0000 7ff63b4ccb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560008d95980 0x560008da5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.415+0000 7ff63b4ccb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560008c394a0 con 0x560008d95020 2023-09-09T23:13:35.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.415+0000 7ff63b4ccb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x560008d9ead0 con 0x560008d941e0 2023-09-09T23:13:35.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.415+0000 7ff63b4ccb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560008b6fde0 con 0x560008d95980 2023-09-09T23:13:35.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.415+0000 7ff62c8a1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560008d95020 0x560008da4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.415+0000 7ff62c8a1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560008d95020 0x560008da4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55900/0 (socket says 172.21.15.82:55900) 2023-09-09T23:13:35.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.415+0000 7ff62c8a1700 1 -- 172.21.15.82:0/4168332816 learned_addr learned my addr 172.21.15.82:0/4168332816 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:35.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.415+0000 7ff62d0a2700 1 --2- 172.21.15.82:0/4168332816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560008d941e0 0x560008da7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.415+0000 7ff62c8a1700 1 -- 172.21.15.82:0/4168332816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560008d95980 msgr2=0x560008da5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:35.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.415+0000 7ff62c8a1700 1 --2- 172.21.15.82:0/4168332816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560008d95980 0x560008da5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.415+0000 7ff62c8a1700 1 -- 172.21.15.82:0/4168332816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560008d941e0 msgr2=0x560008da7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:35.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.415+0000 7ff62c8a1700 1 --2- 172.21.15.82:0/4168332816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560008d941e0 0x560008da7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.415+0000 7ff62c8a1700 1 -- 172.21.15.82:0/4168332816 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560008b34200 con 0x560008d95020 2023-09-09T23:13:35.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.415+0000 7ff62c8a1700 1 --2- 172.21.15.82:0/4168332816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560008d95020 0x560008da4d80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff61c095c80 tx=0x7ff61c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e589c67ddb131a server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:35.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.415+0000 7ff613fff700 1 -- 172.21.15.82:0/4168332816 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c079020 con 0x560008d95020 2023-09-09T23:13:35.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.415+0000 7ff613fff700 1 -- 172.21.15.82:0/4168332816 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff61c099680 con 0x560008d95020 2023-09-09T23:13:35.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.415+0000 7ff613fff700 1 -- 172.21.15.82:0/4168332816 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c014070 con 0x560008d95020 2023-09-09T23:13:35.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.416+0000 7ff63b4ccb80 1 -- 172.21.15.82:0/4168332816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560008d95020 msgr2=0x560008da4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:35.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.416+0000 7ff63b4ccb80 1 --2- 172.21.15.82:0/4168332816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560008d95020 0x560008da4d80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff61c095c80 tx=0x7ff61c000bc0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.416+0000 7ff63b4ccb80 1 -- 172.21.15.82:0/4168332816 shutdown_connections 2023-09-09T23:13:35.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.416+0000 7ff63b4ccb80 1 --2- 172.21.15.82:0/4168332816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560008d941e0 0x560008da7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.416+0000 7ff63b4ccb80 1 --2- 172.21.15.82:0/4168332816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560008d95980 0x560008da5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.416+0000 7ff63b4ccb80 1 --2- 172.21.15.82:0/4168332816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560008d95020 0x560008da4d80 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.416+0000 7ff63b4ccb80 1 -- 172.21.15.82:0/4168332816 >> 172.21.15.82:0/4168332816 conn(0x560008d04040 msgr2=0x560008d9c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:35.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.416+0000 7ff63b4ccb80 1 -- 172.21.15.82:0/4168332816 shutdown_connections 2023-09-09T23:13:35.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.416+0000 7ff63b4ccb80 1 -- 172.21.15.82:0/4168332816 wait complete. 2023-09-09T23:13:35.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.416+0000 7ff63b4ccb80 1 -- start start 2023-09-09T23:13:35.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.416+0000 7ff63b4ccb80 1 Processor -- start 2023-09-09T23:13:35.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.417+0000 7ff63b4ccb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560008d941e0 0x560008d9fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.417+0000 7ff63b4ccb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560008d95020 0x560008da0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.417+0000 7ff63b4ccb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560008d95980 0x560008dfcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.417+0000 7ff63b4ccb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560008da0ab0 con 0x560008d95020 2023-09-09T23:13:35.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.417+0000 7ff63b4ccb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x560008da0bf0 con 0x560008d941e0 2023-09-09T23:13:35.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.417+0000 7ff63b4ccb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560008dfd4f0 con 0x560008d95980 2023-09-09T23:13:35.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.417+0000 7ff63b4ccb80 10 client.0 _ll_register_callbacks cb 0x7ff61c099910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:35.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.417+0000 7ff62c8a1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560008d95020 0x560008da0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.417+0000 7ff62c8a1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560008d95020 0x560008da0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55906/0 (socket says 172.21.15.82:55906) 2023-09-09T23:13:35.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.417+0000 7ff62c8a1700 1 -- 172.21.15.82:0/917810599 learned_addr learned my addr 172.21.15.82:0/917810599 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:35.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.417+0000 7ff62d8a3700 1 --2- 172.21.15.82:0/917810599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560008d95980 0x560008dfcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.417+0000 7ff62d0a2700 1 --2- 172.21.15.82:0/917810599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560008d941e0 0x560008d9fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.417+0000 7ff62c8a1700 1 -- 172.21.15.82:0/917810599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560008d95980 msgr2=0x560008dfcfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:35.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.417+0000 7ff62c8a1700 1 --2- 172.21.15.82:0/917810599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560008d95980 0x560008dfcfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.417+0000 7ff62c8a1700 1 -- 172.21.15.82:0/917810599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560008d941e0 msgr2=0x560008d9fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:35.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.417+0000 7ff62c8a1700 1 --2- 172.21.15.82:0/917810599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560008d941e0 0x560008d9fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.417+0000 7ff62c8a1700 1 -- 172.21.15.82:0/917810599 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560008af6c10 con 0x560008d95020 2023-09-09T23:13:35.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.417+0000 7ff62c8a1700 1 --2- 172.21.15.82:0/917810599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560008d95020 0x560008da0530 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff61c098ec0 tx=0x7ff61c0a1170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bdc9571ce3d9bb8 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:35.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.417+0000 7ff610ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:35.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.417+0000 7ff610ff9700 1 -- 172.21.15.82:0/917810599 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c079070 con 0x560008d95020 2023-09-09T23:13:35.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.417+0000 7ff610ff9700 1 -- 172.21.15.82:0/917810599 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff61c0acb30 con 0x560008d95020 2023-09-09T23:13:35.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.417+0000 7ff63b4ccb80 1 -- 172.21.15.82:0/917810599 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff62401e040 con 0x560008d95020 2023-09-09T23:13:35.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.417+0000 7ff610ff9700 1 -- 172.21.15.82:0/917810599 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff61c014030 con 0x560008d95020 2023-09-09T23:13:35.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.417+0000 7ff63b4ccb80 10 client.15420 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:35.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.417+0000 7ff63b4ccb80 1 -- 172.21.15.82:0/917810599 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560008b0c260 con 0x560008d95020 2023-09-09T23:13:35.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.418+0000 7ff610ff9700 1 -- 172.21.15.82:0/917810599 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff62401e040 con 0x560008d95020 2023-09-09T23:13:35.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.418+0000 7ff610ff9700 1 -- 172.21.15.82:0/917810599 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff61c009bc0 con 0x560008d95020 2023-09-09T23:13:35.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.418+0000 7ff63b4ccb80 10 client.15420 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:35.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.418+0000 7ff63b4ccb80 10 client.15420 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-09T23:13:35.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.418+0000 7ff63b4ccb80 20 client.15420 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:35.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.418+0000 7ff63b4ccb80 1 --2- 172.21.15.82:0/917810599 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x560008da8f60 0x560008dfe810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:35.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.418+0000 7ff63b4ccb80 4 client.15420 mds_command: new command op to 24566 tid=0[{"prefix": "status"}] 2023-09-09T23:13:35.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.418+0000 7ff63b4ccb80 1 -- 172.21.15.82:0/917810599 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560008cda8d0 con 0x560008da8f60 2023-09-09T23:13:35.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.418+0000 7ff62d0a2700 1 --2- 172.21.15.82:0/917810599 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x560008da8f60 0x560008dfe810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:35.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.418+0000 7ff610ff9700 10 client.15420 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:13:35.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.418+0000 7ff62d0a2700 1 --2- 172.21.15.82:0/917810599 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x560008da8f60 0x560008dfe810 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69eba6d2b47da53d server_cookie=2b2be114eedd7ea7 in_seq=0 out_seq=0 2023-09-09T23:13:35.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.419+0000 7ff610ff9700 1 -- 172.21.15.82:0/917810599 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x560008cda8d0 con 0x560008da8f60 2023-09-09T23:13:35.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.419+0000 7ff610ff9700 10 client.15420 handle_command_reply: tid=0 2023-09-09T23:13:35.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.419+0000 7ff63b4ccb80 1 client.15420 shutdown 2023-09-09T23:13:35.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.419+0000 7ff63b4ccb80 1 -- 172.21.15.82:0/917810599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560008d95020 msgr2=0x560008da0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:35.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.419+0000 7ff63b4ccb80 1 --2- 172.21.15.82:0/917810599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560008d95020 0x560008da0530 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff61c098ec0 tx=0x7ff61c0a1170 comp rx=0 tx=0).stop 2023-09-09T23:13:35.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.419+0000 7ff63b4ccb80 1 -- 172.21.15.82:0/917810599 shutdown_connections 2023-09-09T23:13:35.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.419+0000 7ff63b4ccb80 1 --2- 172.21.15.82:0/917810599 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x560008da8f60 0x560008dfe810 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.419+0000 7ff63b4ccb80 1 --2- 172.21.15.82:0/917810599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560008d941e0 0x560008d9fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.419+0000 7ff63b4ccb80 1 --2- 172.21.15.82:0/917810599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560008d95980 0x560008dfcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.419+0000 7ff610ff9700 0 client.15420 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:13:35.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.419+0000 7ff63b4ccb80 1 --2- 172.21.15.82:0/917810599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560008d95020 0x560008da0530 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.419+0000 7ff63b4ccb80 1 -- 172.21.15.82:0/917810599 >> 172.21.15.82:0/917810599 conn(0x560008d04040 msgr2=0x560008da7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:35.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.419+0000 7ff63b4ccb80 1 -- 172.21.15.82:0/917810599 shutdown_connections 2023-09-09T23:13:35.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.419+0000 7ff63b4ccb80 1 -- 172.21.15.82:0/917810599 wait complete. 2023-09-09T23:13:35.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.419+0000 7ff63b4ccb80 20 client.15420 trim_cache size 0 max 16384 2023-09-09T23:13:35.447 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:13:35.447 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da", 2023-09-09T23:13:35.447 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2023-09-09T23:13:35.447 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24566, 2023-09-09T23:13:35.447 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2023-09-09T23:13:35.448 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2023-09-09T23:13:35.448 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2023-09-09T23:13:35.448 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 76.884895890999999, 2023-09-09T23:13:35.448 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 27, 2023-09-09T23:13:35.448 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 83, 2023-09-09T23:13:35.449 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 81, 2023-09-09T23:13:35.449 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 83.743501334000001 2023-09-09T23:13:35.449 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:13:35.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.419+0000 7ff62f306700 1 -- 172.21.15.82:0/2617365152 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff6140a06d0 msgr2=0x7ff6140a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:35.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.419+0000 7ff62f306700 1 --2- 172.21.15.82:0/2617365152 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff6140a06d0 0x7ff6140a2b90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff624006010 tx=0x7ff62400b540 comp rx=0 tx=0).stop 2023-09-09T23:13:35.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.419+0000 7ff62f306700 1 -- 172.21.15.82:0/2617365152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff628100140 msgr2=0x7ff62819b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:35.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.419+0000 7ff62f306700 1 --2- 172.21.15.82:0/2617365152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff628100140 0x7ff62819b090 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00d950 tx=0x7ff61c00dc60 comp rx=0 tx=0).stop 2023-09-09T23:13:35.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.420+0000 7ff62f306700 1 -- 172.21.15.82:0/2617365152 shutdown_connections 2023-09-09T23:13:35.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.420+0000 7ff62f306700 1 --2- 172.21.15.82:0/2617365152 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff6140a06d0 0x7ff6140a2b90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.420+0000 7ff62f306700 1 --2- 172.21.15.82:0/2617365152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6280ff7a0 0x7ff6280ff200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.420+0000 7ff62f306700 1 --2- 172.21.15.82:0/2617365152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff628100a70 0x7ff62819d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.420+0000 7ff62f306700 1 --2- 172.21.15.82:0/2617365152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff628100140 0x7ff62819b090 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:35.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.420+0000 7ff62f306700 1 -- 172.21.15.82:0/2617365152 >> 172.21.15.82:0/2617365152 conn(0x7ff6280fa1d0 msgr2=0x7ff6280fc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:35.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.420+0000 7ff62f306700 1 -- 172.21.15.82:0/2617365152 shutdown_connections 2023-09-09T23:13:35.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:35.420+0000 7ff62f306700 1 -- 172.21.15.82:0/2617365152 wait complete. 2023-09-09T23:13:35.453 INFO:teuthology.run_tasks:Running task kclient... 2023-09-09T23:13:35.479 INFO:tasks.kclient:Mounting kernel clients... 2023-09-09T23:13:35.480 INFO:tasks.kclient:config is {'client.0': {}} 2023-09-09T23:13:35.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-09T23:13:35.481 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2023-09-09T23:13:35.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-09T23:13:35.499 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link delete ceph-brx 2023-09-09T23:13:35.561 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:35 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/841952155' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:35.567 INFO:teuthology.orchestra.run.smithi082.stderr:Cannot find device "ceph-brx" 2023-09-09T23:13:35.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-09T23:13:35.569 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v2', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2023-09-09T23:13:35.570 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-09-09T23:13:35.570 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2023-09-09T23:13:35.571 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2023-09-09T23:13:35.571 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9670cca5bc8e17916865dc84939edfc10b1f2c00 shell --fsid 3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da -- ceph fs ls 2023-09-09T23:13:35.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:35 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/841952155' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:35.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:35 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/841952155' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:36.166 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/mon.a/config 2023-09-09T23:13:36.565 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:36 smithi082 ceph-mon[98694]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-09T23:13:36.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:36 smithi170 ceph-mon[102832]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-09T23:13:36.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:36 smithi098 ceph-mon[103124]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-09T23:13:37.366 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.367+0000 7f35c1762700 1 -- 172.21.15.82:0/691051498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35bc0ff540 msgr2=0x7f35bc0ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:37.366 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.367+0000 7f35c1762700 1 --2- 172.21.15.82:0/691051498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35bc0ff540 0x7f35bc0ff9c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0099c0 tx=0x7f35ac009cd0 comp rx=0 tx=0).stop 2023-09-09T23:13:37.366 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.367+0000 7f35c1762700 1 -- 172.21.15.82:0/691051498 shutdown_connections 2023-09-09T23:13:37.366 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.367+0000 7f35c1762700 1 --2- 172.21.15.82:0/691051498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35bc1055f0 0x7f35bc1059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:37.367 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.367+0000 7f35c1762700 1 --2- 172.21.15.82:0/691051498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35bc0fff00 0x7f35bc10c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:37.367 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.367+0000 7f35c1762700 1 --2- 172.21.15.82:0/691051498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35bc0ff540 0x7f35bc0ff9c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:37.367 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.367+0000 7f35c1762700 1 -- 172.21.15.82:0/691051498 >> 172.21.15.82:0/691051498 conn(0x7f35bc0fb070 msgr2=0x7f35bc0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:37.367 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.367+0000 7f35c1762700 1 -- 172.21.15.82:0/691051498 shutdown_connections 2023-09-09T23:13:37.367 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.367+0000 7f35c1762700 1 -- 172.21.15.82:0/691051498 wait complete. 2023-09-09T23:13:37.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.368+0000 7f35c1762700 1 Processor -- start 2023-09-09T23:13:37.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.368+0000 7f35c1762700 1 -- start start 2023-09-09T23:13:37.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.368+0000 7f35c1762700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35bc0ff540 0x7f35bc1011b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:37.370 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.368+0000 7f35c1762700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35bc0fff00 0x7f35bc1016f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:37.370 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.368+0000 7f35c1762700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35bc1055f0 0x7f35bc101c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:37.370 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.368+0000 7f35c1762700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc1a4af0 con 0x7f35bc0fff00 2023-09-09T23:13:37.370 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.368+0000 7f35c1762700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc1a4c60 con 0x7f35bc1055f0 2023-09-09T23:13:37.370 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.368+0000 7f35c1762700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc1a4e20 con 0x7f35bc0ff540 2023-09-09T23:13:37.371 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.368+0000 7f35ba7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35bc0fff00 0x7f35bc1016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:37.371 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.368+0000 7f35ba7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35bc0fff00 0x7f35bc1016f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55920/0 (socket says 172.21.15.82:55920) 2023-09-09T23:13:37.371 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.368+0000 7f35ba7fc700 1 -- 172.21.15.82:0/2766903225 learned_addr learned my addr 172.21.15.82:0/2766903225 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:37.371 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.369+0000 7f35baffd700 1 --2- 172.21.15.82:0/2766903225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35bc0ff540 0x7f35bc1011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:37.371 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.369+0000 7f35bb7fe700 1 --2- 172.21.15.82:0/2766903225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35bc1055f0 0x7f35bc101c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:37.372 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.369+0000 7f35ba7fc700 1 -- 172.21.15.82:0/2766903225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35bc0ff540 msgr2=0x7f35bc1011b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:37.372 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.369+0000 7f35ba7fc700 1 --2- 172.21.15.82:0/2766903225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35bc0ff540 0x7f35bc1011b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:37.372 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.369+0000 7f35ba7fc700 1 -- 172.21.15.82:0/2766903225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35bc1055f0 msgr2=0x7f35bc101c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:37.372 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.369+0000 7f35ba7fc700 1 --2- 172.21.15.82:0/2766903225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35bc1055f0 0x7f35bc101c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:37.372 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.369+0000 7f35ba7fc700 1 -- 172.21.15.82:0/2766903225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ac0095f0 con 0x7f35bc0fff00 2023-09-09T23:13:37.373 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.369+0000 7f35bb7fe700 1 --2- 172.21.15.82:0/2766903225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35bc1055f0 0x7f35bc101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:37.373 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.369+0000 7f35ba7fc700 1 --2- 172.21.15.82:0/2766903225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35bc0fff00 0x7f35bc1016f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f35ac004730 tx=0x7f35ac004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:37.373 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.369+0000 7f35b3fff700 1 -- 172.21.15.82:0/2766903225 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac01d070 con 0x7f35bc0fff00 2023-09-09T23:13:37.373 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.369+0000 7f35b3fff700 1 -- 172.21.15.82:0/2766903225 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35ac004c90 con 0x7f35bc0fff00 2023-09-09T23:13:37.373 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.369+0000 7f35b3fff700 1 -- 172.21.15.82:0/2766903225 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac005720 con 0x7f35bc0fff00 2023-09-09T23:13:37.373 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.369+0000 7f35c1762700 1 -- 172.21.15.82:0/2766903225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35bc1a50a0 con 0x7f35bc0fff00 2023-09-09T23:13:37.374 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.369+0000 7f35c1762700 1 -- 172.21.15.82:0/2766903225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35bc1a5400 con 0x7f35bc0fff00 2023-09-09T23:13:37.374 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.370+0000 7f35c1762700 1 -- 172.21.15.82:0/2766903225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35bc04e300 con 0x7f35bc0fff00 2023-09-09T23:13:37.375 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.371+0000 7f35b3fff700 1 -- 172.21.15.82:0/2766903225 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135310+0+0 (secure 0 0 0) 0x7f35ac005000 con 0x7f35bc0fff00 2023-09-09T23:13:37.375 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.372+0000 7f35b3fff700 1 --2- 172.21.15.82:0/2766903225 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f35a80a06d0 0x7f35a80a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:37.375 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.372+0000 7f35baffd700 1 --2- 172.21.15.82:0/2766903225 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f35a80a06d0 0x7f35a80a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:37.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.372+0000 7f35b3fff700 1 -- 172.21.15.82:0/2766903225 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f35ac0cc860 con 0x7f35bc0fff00 2023-09-09T23:13:37.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.372+0000 7f35baffd700 1 --2- 172.21.15.82:0/2766903225 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f35a80a06d0 0x7f35a80a2b90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f35a4005950 tx=0x7f35a400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:37.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.376+0000 7f35b3fff700 1 -- 172.21.15.82:0/2766903225 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f35ac0963c0 con 0x7f35bc0fff00 2023-09-09T23:13:37.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.609+0000 7f35c1762700 1 -- 172.21.15.82:0/2766903225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f35bc10f790 con 0x7f35bc0fff00 2023-09-09T23:13:37.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.609+0000 7f35b3fff700 1 -- 172.21.15.82:0/2766903225 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7f35bc10f790 con 0x7f35bc0fff00 2023-09-09T23:13:37.609 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-09T23:13:37.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.611+0000 7f35c1762700 1 -- 172.21.15.82:0/2766903225 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f35a80a06d0 msgr2=0x7f35a80a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:37.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.611+0000 7f35c1762700 1 --2- 172.21.15.82:0/2766903225 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f35a80a06d0 0x7f35a80a2b90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f35a4005950 tx=0x7f35a400b410 comp rx=0 tx=0).stop 2023-09-09T23:13:37.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.611+0000 7f35c1762700 1 -- 172.21.15.82:0/2766903225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35bc0fff00 msgr2=0x7f35bc1016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:37.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.611+0000 7f35c1762700 1 --2- 172.21.15.82:0/2766903225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35bc0fff00 0x7f35bc1016f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f35ac004730 tx=0x7f35ac004760 comp rx=0 tx=0).stop 2023-09-09T23:13:37.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.612+0000 7f35c1762700 1 -- 172.21.15.82:0/2766903225 shutdown_connections 2023-09-09T23:13:37.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.612+0000 7f35c1762700 1 --2- 172.21.15.82:0/2766903225 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f35a80a06d0 0x7f35a80a2b90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:37.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.612+0000 7f35c1762700 1 --2- 172.21.15.82:0/2766903225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35bc1055f0 0x7f35bc101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:37.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.612+0000 7f35c1762700 1 --2- 172.21.15.82:0/2766903225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35bc0ff540 0x7f35bc1011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:37.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.612+0000 7f35c1762700 1 --2- 172.21.15.82:0/2766903225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35bc0fff00 0x7f35bc1016f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:37.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.612+0000 7f35c1762700 1 -- 172.21.15.82:0/2766903225 >> 172.21.15.82:0/2766903225 conn(0x7f35bc0fb070 msgr2=0x7f35bc0fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:37.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.612+0000 7f35c1762700 1 -- 172.21.15.82:0/2766903225 shutdown_connections 2023-09-09T23:13:37.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:37.612+0000 7f35c1762700 1 -- 172.21.15.82:0/2766903225 wait complete. 2023-09-09T23:13:38.107 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-09T23:13:38.108 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-09T23:13:38.108 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2023-09-09T23:13:38.109 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-09T23:13:38.109 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2023-09-09T23:13:38.109 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-09T23:13:38.110 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-09T23:13:38.110 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-09T23:13:38.110 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2023-09-09T23:13:38.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-09T23:13:38.111 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2023-09-09T23:13:38.130 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-09T23:13:38.130 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-09T23:13:38.130 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2023-09-09T23:13:38.131 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2023-09-09T23:13:38.131 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2023-09-09T23:13:38.131 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2023-09-09T23:13:38.131 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-09T23:13:38.132 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2023-09-09T23:13:38.132 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-09T23:13:38.132 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2023-09-09T23:13:38.132 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-09T23:13:38.132 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2023-09-09T23:13:38.133 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-09T23:13:38.133 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2023-09-09T23:13:38.133 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-09-09T23:13:38.133 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 41883sec preferred_lft 41883sec 2023-09-09T23:13:38.134 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::164c:74f1:387:f0a9/64 scope link noprefixroute 2023-09-09T23:13:38.134 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2023-09-09T23:13:38.134 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-09-09T23:13:38.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-09T23:13:38.135 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2023-09-09T23:13:38.135 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2023-09-09T23:13:38.136 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add name ceph-brx type bridge 2023-09-09T23:13:38.136 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip addr flush dev ceph-brx 2023-09-09T23:13:38.136 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set ceph-brx up 2023-09-09T23:13:38.137 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-09-09T23:13:38.137 DEBUG:teuthology.orchestra.run.smithi082:> ') 2023-09-09T23:13:38.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-09T23:13:38.305 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-09T23:13:38.316 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:38 smithi082 ceph-mon[98694]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-09-09T23:13:38.316 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:38 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2766903225' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-09T23:13:38.334 INFO:teuthology.orchestra.run.smithi082.stdout:1 2023-09-09T23:13:38.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-09T23:13:38.335 DEBUG:teuthology.orchestra.run.smithi082:> route 2023-09-09T23:13:38.409 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2023-09-09T23:13:38.409 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-09T23:13:38.409 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-09T23:13:38.410 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-09-09T23:13:38.410 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-09T23:13:38.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-09T23:13:38.411 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2023-09-09T23:13:38.411 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2023-09-09T23:13:38.411 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-09T23:13:38.412 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-09T23:13:38.412 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-09T23:13:38.412 DEBUG:teuthology.orchestra.run.smithi082:> ') 2023-09-09T23:13:38.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-09T23:13:38.654 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2023-09-09T23:13:38.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-09T23:13:38.672 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list-id 2023-09-09T23:13:38.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:38 smithi170 ceph-mon[102832]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-09-09T23:13:38.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:38 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2766903225' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-09T23:13:38.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-09T23:13:38.728 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2023-09-09T23:13:38.728 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2023-09-09T23:13:38.728 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns add ceph-ns--tmp-cephfs 2023-09-09T23:13:38.729 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns set ceph-ns--tmp-cephfs 0 2023-09-09T23:13:38.729 DEBUG:teuthology.orchestra.run.smithi082:> ') 2023-09-09T23:13:38.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:38 smithi098 ceph-mon[103124]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-09-09T23:13:38.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:38 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2766903225' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-09T23:13:38.823 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2023-09-09T23:13:38.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-09T23:13:38.824 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2023-09-09T23:13:38.824 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2023-09-09T23:13:38.824 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2023-09-09T23:13:38.824 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-09-09T23:13:38.825 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2023-09-09T23:13:38.825 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2023-09-09T23:13:38.825 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2023-09-09T23:13:38.825 DEBUG:teuthology.orchestra.run.smithi082:> ') 2023-09-09T23:13:39.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-09T23:13:39.029 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2023-09-09T23:13:39.030 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2023-09-09T23:13:39.030 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2023-09-09T23:13:39.030 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set dev brx.0 master ceph-brx 2023-09-09T23:13:39.031 DEBUG:teuthology.orchestra.run.smithi082:> ') 2023-09-09T23:13:39.128 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-09T23:13:39.129 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /tmp/cephfs 2023-09-09T23:13:39.146 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/tmp/cephfs' 2023-09-09T23:13:39.146 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-09T23:13:39.146 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /tmp/cephfs 2023-09-09T23:13:39.203 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-09-09T23:13:39.203 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-09-09T23:13:39.203 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/_nogroup/sv_1/0a302eaa-eeda-4440-b80e-72c42f5e6ec4 2023-09-09T23:13:39.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-09T23:13:39.204 DEBUG:teuthology.orchestra.run.smithi082:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/_nogroup/sv_1/0a302eaa-eeda-4440-b80e-72c42f5e6ec4 /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,context=system_u:object_r:postgresql_db_t:s0,nofallback 2023-09-09T23:13:39.329 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2023-09-09T23:13:39.329 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2023-09-09T23:13:39.470 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2023-09-09T23:13:39.470 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2023-09-09T23:13:39.470 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.98:3300,172.21.15.170:3300" 2023-09-09T23:13:39.471 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: trying mount with new device syntax: 0@3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da.cephfs=/volumes/_nogroup/sv_1/0a302eaa-eeda-4440-b80e-72c42f5e6ec4 2023-09-09T23:13:39.471 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",name=0,key=0,mon_addr=172.21.15.82:3300/172.21.15.98:3300/172.21.15.170:3300" will pass to kernel 2023-09-09T23:13:39.471 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: v2 mount support check returned 2 2023-09-09T23:13:39.471 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: kernel does not support v2 syntax 2023-09-09T23:13:39.471 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.82:3300,172.21.15.98:3300,172.21.15.170:3300:/volumes/_nogroup/sv_1/0a302eaa-eeda-4440-b80e-72c42f5e6ec4 2023-09-09T23:13:39.472 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",name=0,key=0,mds_namespace=cephfs,fsid=3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da" will pass to kernel 2023-09-09T23:13:39.474 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-09-09T23:13:39.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-09T23:13:39.475 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /tmp/cephfs 2023-09-09T23:13:39.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-09T23:13:39.509 DEBUG:teuthology.orchestra.run.smithi082:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-09-09T23:13:39.581 INFO:teuthology.orchestra.run.smithi082.stdout:client24913 2023-09-09T23:13:39.581 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-09-09T23:13:39.581 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-09T23:13:39.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-09T23:13:39.640 DEBUG:teuthology.orchestra.run.smithi082:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-09-09T23:13:39.711 INFO:teuthology.orchestra.run.smithi082.stdout:client24913 2023-09-09T23:13:39.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-09T23:13:39.712 DEBUG:teuthology.orchestra.run.smithi082:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-09-09T23:13:39.785 INFO:teuthology.orchestra.run.smithi082.stdout:client24913 2023-09-09T23:13:40.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-09T23:13:40.026 DEBUG:teuthology.orchestra.run.smithi082:> (cd /tmp/cephfs && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da.client24913/status') 2023-09-09T23:13:40.069 INFO:teuthology.orchestra.run.smithi082.stdout:instance: client.24913 (3)172.21.15.82:0/1669103154 2023-09-09T23:13:40.070 INFO:teuthology.orchestra.run.smithi082.stdout:blocklisted: false 2023-09-09T23:13:40.070 INFO:teuthology.orchestra.run.smithi082.stderr:0+1 records in 2023-09-09T23:13:40.071 INFO:teuthology.orchestra.run.smithi082.stderr:0+1 records out 2023-09-09T23:13:40.071 INFO:teuthology.orchestra.run.smithi082.stderr:71 bytes copied, 2.147e-05 s, 3.3 MB/s 2023-09-09T23:13:40.072 INFO:teuthology.run_tasks:Running task check-counter... 2023-09-09T23:13:40.085 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-09-09T23:13:40.099 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-09T23:13:40.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.239+0000 7fa6d8d54700 1 -- 172.21.15.82:0/3503793142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d40ff4f0 msgr2=0x7fa6d40ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:40.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.239+0000 7fa6d8d54700 1 --2- 172.21.15.82:0/3503793142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d40ff4f0 0x7fa6d40ff910 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80099c0 tx=0x7fa6c8009cd0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.239+0000 7fa6d8d54700 1 -- 172.21.15.82:0/3503793142 shutdown_connections 2023-09-09T23:13:40.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.239+0000 7fa6d8d54700 1 --2- 172.21.15.82:0/3503793142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6d4101080 0x7fa6d4107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.239+0000 7fa6d8d54700 1 --2- 172.21.15.82:0/3503793142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d40ff4f0 0x7fa6d40ff910 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.239+0000 7fa6d8d54700 1 --2- 172.21.15.82:0/3503793142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6d41006e0 0x7fa6d4100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.239+0000 7fa6d8d54700 1 -- 172.21.15.82:0/3503793142 >> 172.21.15.82:0/3503793142 conn(0x7fa6d40faa20 msgr2=0x7fa6d40fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:40.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.239+0000 7fa6d8d54700 1 -- 172.21.15.82:0/3503793142 shutdown_connections 2023-09-09T23:13:40.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.239+0000 7fa6d8d54700 1 -- 172.21.15.82:0/3503793142 wait complete. 2023-09-09T23:13:40.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.240+0000 7fa6d8d54700 1 Processor -- start 2023-09-09T23:13:40.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.240+0000 7fa6d8d54700 1 -- start start 2023-09-09T23:13:40.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.240+0000 7fa6d8d54700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6d40ff4f0 0x7fa6d4103480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:40.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.240+0000 7fa6d8d54700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d41006e0 0x7fa6d4101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:40.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.240+0000 7fa6d8d54700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6d4101080 0x7fa6d4102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:40.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.240+0000 7fa6d8d54700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d41027a0 con 0x7fa6d41006e0 2023-09-09T23:13:40.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.240+0000 7fa6d8d54700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d4102910 con 0x7fa6d40ff4f0 2023-09-09T23:13:40.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.240+0000 7fa6d8d54700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d41a0330 con 0x7fa6d4101080 2023-09-09T23:13:40.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.240+0000 7fa6d1d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d41006e0 0x7fa6d4101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:40.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.240+0000 7fa6d1d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d41006e0 0x7fa6d4101ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55962/0 (socket says 172.21.15.82:55962) 2023-09-09T23:13:40.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.240+0000 7fa6d1d9b700 1 -- 172.21.15.82:0/2978953457 learned_addr learned my addr 172.21.15.82:0/2978953457 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:40.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.240+0000 7fa6d259c700 1 --2- 172.21.15.82:0/2978953457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6d40ff4f0 0x7fa6d4103480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:40.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.240+0000 7fa6d2d9d700 1 --2- 172.21.15.82:0/2978953457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6d4101080 0x7fa6d4102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:40.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.241+0000 7fa6d1d9b700 1 -- 172.21.15.82:0/2978953457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6d4101080 msgr2=0x7fa6d4102000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:40.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.241+0000 7fa6d1d9b700 1 --2- 172.21.15.82:0/2978953457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6d4101080 0x7fa6d4102000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.241+0000 7fa6d1d9b700 1 -- 172.21.15.82:0/2978953457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6d40ff4f0 msgr2=0x7fa6d4103480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:40.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.241+0000 7fa6d1d9b700 1 --2- 172.21.15.82:0/2978953457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6d40ff4f0 0x7fa6d4103480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.241+0000 7fa6d1d9b700 1 -- 172.21.15.82:0/2978953457 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c80095f0 con 0x7fa6d41006e0 2023-09-09T23:13:40.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.241+0000 7fa6d259c700 1 --2- 172.21.15.82:0/2978953457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6d40ff4f0 0x7fa6d4103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:40.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.241+0000 7fa6d2d9d700 1 --2- 172.21.15.82:0/2978953457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6d4101080 0x7fa6d4102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:40.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.241+0000 7fa6d1d9b700 1 --2- 172.21.15.82:0/2978953457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d41006e0 0x7fa6d4101ac0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400b530 tx=0x7fa6c400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:40.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.241+0000 7fa6c37fe700 1 -- 172.21.15.82:0/2978953457 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4011830 con 0x7fa6d41006e0 2023-09-09T23:13:40.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.241+0000 7fa6d8d54700 1 -- 172.21.15.82:0/2978953457 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d41a0610 con 0x7fa6d41006e0 2023-09-09T23:13:40.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.241+0000 7fa6c37fe700 1 -- 172.21.15.82:0/2978953457 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6c4011e50 con 0x7fa6d41006e0 2023-09-09T23:13:40.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.241+0000 7fa6c37fe700 1 -- 172.21.15.82:0/2978953457 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c400f2e0 con 0x7fa6d41006e0 2023-09-09T23:13:40.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.241+0000 7fa6d8d54700 1 -- 172.21.15.82:0/2978953457 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d41a0b20 con 0x7fa6d41006e0 2023-09-09T23:13:40.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.242+0000 7fa6d8d54700 1 -- 172.21.15.82:0/2978953457 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d410ad80 con 0x7fa6d41006e0 2023-09-09T23:13:40.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.243+0000 7fa6c37fe700 1 -- 172.21.15.82:0/2978953457 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fa6c4010580 con 0x7fa6d41006e0 2023-09-09T23:13:40.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.243+0000 7fa6c37fe700 1 --2- 172.21.15.82:0/2978953457 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa6bc0a0690 0x7fa6bc0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:40.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.243+0000 7fa6d259c700 1 --2- 172.21.15.82:0/2978953457 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa6bc0a0690 0x7fa6bc0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:40.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.243+0000 7fa6c37fe700 1 -- 172.21.15.82:0/2978953457 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fa6c40cb6c0 con 0x7fa6d41006e0 2023-09-09T23:13:40.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.244+0000 7fa6d259c700 1 --2- 172.21.15.82:0/2978953457 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa6bc0a0690 0x7fa6bc0a2b50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8006010 tx=0x7fa6c8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:40.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.248+0000 7fa6c37fe700 1 -- 172.21.15.82:0/2978953457 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa6c40951f0 con 0x7fa6d41006e0 2023-09-09T23:13:40.344 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:40 smithi082 ceph-mon[98694]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:13:40.344 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:40 smithi082 ceph-mon[98694]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-09-09T23:13:40.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.443+0000 7fa6d8d54700 1 -- 172.21.15.82:0/2978953457 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6d410afc0 con 0x7fa6d41006e0 2023-09-09T23:13:40.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.445+0000 7fa6c37fe700 1 -- 172.21.15.82:0/2978953457 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22411 (secure 0 0 0) 0x7fa6c4098810 con 0x7fa6d41006e0 2023-09-09T23:13:40.444 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:40.444 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":83,"fsid":"3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da","created":"2023-09-09T23:05:14.222190+0000","modified":"2023-09-09T23:13:10.324596+0000","last_up_change":"2023-09-09T23:11:37.253759+0000","last_in_change":"2023-09-09T23:11:19.933812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-09T23:08:31.234958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-09T23:12:15.524887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-09T23:12:16.785403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1277eb17-a2e7-4e10-9832-f12a5e93ffde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6803","nonce":3197434929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6805","nonce":3197434929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6809","nonce":3197434929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6807","nonce":3197434929}]},"public_addr":"172.21.15.82:6803/3197434929","cluster_addr":"172.21.15.82:6805/3197434929","heartbeat_back_addr":"172.21.15.82:6809/3197434929","heartbeat_front_addr":"172.21.15.82:6807/3197434929","state":["exists","up"]},{"osd":1,"uuid":"f68723fd-0908-4e21-ae8e-1de1c3a9a65c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6801","nonce":3528641314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6803","nonce":3528641314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6807","nonce":3528641314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6805","nonce":3528641314}]},"public_addr":"172.21.15.98:6801/3528641314","cluster_addr":"172.21.15.98:6803/3528641314","heartbeat_back_addr":"172.21.15.98:6807/3528641314","heartbeat_front_addr":"172.21.15.98:6805/3528641314","state":["exists","up"]},{"osd":2,"uuid":"06f89c53-1419-4f52-acad-6e92916b476f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6801","nonce":4006414551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6803","nonce":4006414551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6807","nonce":4006414551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6805","nonce":4006414551}]},"public_addr":"172.21.15.170:6801/4006414551","cluster_addr":"172.21.15.170:6803/4006414551","heartbeat_back_addr":"172.21.15.170:6807/4006414551","heartbeat_front_addr":"172.21.15.170:6805/4006414551","state":["exists","up"]},{"osd":3,"uuid":"fedbd4c9-6852-42b5-8f3a-ba8d9bfe5d59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6811","nonce":3647629241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6813","nonce":3647629241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6817","nonce":3647629241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6815","nonce":3647629241}]},"public_addr":"172.21.15.82:6811/3647629241","cluster_addr":"172.21.15.82:6813/3647629241","heartbeat_back_addr":"172.21.15.82:6817/3647629241","heartbeat_front_addr":"172.21.15.82:6815/3647629241","state":["exists","up"]},{"osd":4,"uuid":"a71f3084-6919-4161-87bf-345f3355d8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6809","nonce":2507035643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6811","nonce":2507035643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6815","nonce":2507035643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6813","nonce":2507035643}]},"public_addr":"172.21.15.98:6809/2507035643","cluster_addr":"172.21.15.98:6811/2507035643","heartbeat_back_addr":"172.21.15.98:6815/2507035643","heartbeat_front_addr":"172.21.15.98:6813/2507035643","state":["exists","up"]},{"osd":5,"uuid":"f76058e9-a90e-476e-87f0-3c6723795e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6809","nonce":775005168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6811","nonce":775005168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6815","nonce":775005168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6813","nonce":775005168}]},"public_addr":"172.21.15.170:6809/775005168","cluster_addr":"172.21.15.170:6811/775005168","heartbeat_back_addr":"172.21.15.170:6815/775005168","heartbeat_front_addr":"172.21.15.170:6813/775005168","state":["exists","up"]},{"osd":6,"uuid":"25940be0-c1bd-4c72-95bf-fd8367aa2f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6819","nonce":3008405982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6821","nonce":3008405982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6825","nonce":3008405982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6823","nonce":3008405982}]},"public_addr":"172.21.15.82:6819/3008405982","cluster_addr":"172.21.15.82:6821/3008405982","heartbeat_back_addr":"172.21.15.82:6825/3008405982","heartbeat_front_addr":"172.21.15.82:6823/3008405982","state":["exists","up"]},{"osd":7,"uuid":"f651bb8c-0b25-42d5-b028-58563f4c27f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6817","nonce":3547615284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6819","nonce":3547615284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6823","nonce":3547615284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6821","nonce":3547615284}]},"public_addr":"172.21.15.98:6817/3547615284","cluster_addr":"172.21.15.98:6819/3547615284","heartbeat_back_addr":"172.21.15.98:6823/3547615284","heartbeat_front_addr":"172.21.15.98:6821/3547615284","state":["exists","up"]},{"osd":8,"uuid":"23b27c86-8c98-441e-b7ab-d57bc584e58f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6817","nonce":632038848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6819","nonce":632038848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6823","nonce":632038848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6821","nonce":632038848}]},"public_addr":"172.21.15.170:6817/632038848","cluster_addr":"172.21.15.170:6819/632038848","heartbeat_back_addr":"172.21.15.170:6823/632038848","heartbeat_front_addr":"172.21.15.170:6821/632038848","state":["exists","up"]},{"osd":9,"uuid":"383b8989-4922-4eee-9c77-f198258ac243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6827","nonce":3354242311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6829","nonce":3354242311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6833","nonce":3354242311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6831","nonce":3354242311}]},"public_addr":"172.21.15.82:6827/3354242311","cluster_addr":"172.21.15.82:6829/3354242311","heartbeat_back_addr":"172.21.15.82:6833/3354242311","heartbeat_front_addr":"172.21.15.82:6831/3354242311","state":["exists","up"]},{"osd":10,"uuid":"33a59389-7ced-42d5-bf54-2071317132d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6825","nonce":2069114959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6827","nonce":2069114959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6831","nonce":2069114959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6829","nonce":2069114959}]},"public_addr":"172.21.15.98:6825/2069114959","cluster_addr":"172.21.15.98:6827/2069114959","heartbeat_back_addr":"172.21.15.98:6831/2069114959","heartbeat_front_addr":"172.21.15.98:6829/2069114959","state":["exists","up"]},{"osd":11,"uuid":"b7a71f5c-85e2-4fec-be4b-95be0788b302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6825","nonce":146964512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6827","nonce":146964512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6831","nonce":146964512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6829","nonce":146964512}]},"public_addr":"172.21.15.170:6825/146964512","cluster_addr":"172.21.15.170:6827/146964512","heartbeat_back_addr":"172.21.15.170:6831/146964512","heartbeat_front_addr":"172.21.15.170:6829/146964512","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:07:47.563043+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:08.359260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:29.070374+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:49.897775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:11.434248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:31.538464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:52.377109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:13.452968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:33.806897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:54.083177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:14.980087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:35.690089+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4013747788":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6800/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/2823341464":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/75116806":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/1037625632":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/3142088817":"2023-09-10T23:05:46.550354+0000","172.21.15.82:6800/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/1679622963":"2023-09-10T23:05:46.550354+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-09T23:13:40.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.447+0000 7fa6d8d54700 1 -- 172.21.15.82:0/2978953457 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa6bc0a0690 msgr2=0x7fa6bc0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:40.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.447+0000 7fa6d8d54700 1 --2- 172.21.15.82:0/2978953457 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa6bc0a0690 0x7fa6bc0a2b50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8006010 tx=0x7fa6c8000bc0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.447+0000 7fa6d8d54700 1 -- 172.21.15.82:0/2978953457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d41006e0 msgr2=0x7fa6d4101ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:40.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.447+0000 7fa6d8d54700 1 --2- 172.21.15.82:0/2978953457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d41006e0 0x7fa6d4101ac0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400b530 tx=0x7fa6c400b8f0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.447+0000 7fa6d8d54700 1 -- 172.21.15.82:0/2978953457 shutdown_connections 2023-09-09T23:13:40.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.447+0000 7fa6d8d54700 1 --2- 172.21.15.82:0/2978953457 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fa6bc0a0690 0x7fa6bc0a2b50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.447+0000 7fa6d8d54700 1 --2- 172.21.15.82:0/2978953457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6d40ff4f0 0x7fa6d4103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.447+0000 7fa6d8d54700 1 --2- 172.21.15.82:0/2978953457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6d4101080 0x7fa6d4102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.447+0000 7fa6d8d54700 1 --2- 172.21.15.82:0/2978953457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d41006e0 0x7fa6d4101ac0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.447+0000 7fa6d8d54700 1 -- 172.21.15.82:0/2978953457 >> 172.21.15.82:0/2978953457 conn(0x7fa6d40faa20 msgr2=0x7fa6d40687d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:40.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.447+0000 7fa6d8d54700 1 -- 172.21.15.82:0/2978953457 shutdown_connections 2023-09-09T23:13:40.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.447+0000 7fa6d8d54700 1 -- 172.21.15.82:0/2978953457 wait complete. 2023-09-09T23:13:40.458 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-09T23:08:31.234958+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-09T23:12:15.524887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-09T23:12:16.785403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-09T23:13:40.458 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-09T23:13:40.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.585+0000 7f5576d67700 1 -- 172.21.15.82:0/1881329761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5570101100 msgr2=0x7f55701034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:40.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.585+0000 7f5576d67700 1 --2- 172.21.15.82:0/1881329761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5570101100 0x7f55701034f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5564009990 tx=0x7f5564009ca0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.585+0000 7f5576d67700 1 -- 172.21.15.82:0/1881329761 shutdown_connections 2023-09-09T23:13:40.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.585+0000 7f5576d67700 1 --2- 172.21.15.82:0/1881329761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5570106360 0x7f5570108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.585+0000 7f5576d67700 1 --2- 172.21.15.82:0/1881329761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5570101100 0x7f55701034f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.585+0000 7f5576d67700 1 --2- 172.21.15.82:0/1881329761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5570103a30 0x7f5570105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.585+0000 7f5576d67700 1 -- 172.21.15.82:0/1881329761 >> 172.21.15.82:0/1881329761 conn(0x7f55700faa90 msgr2=0x7f55700fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:40.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.585+0000 7f5576d67700 1 -- 172.21.15.82:0/1881329761 shutdown_connections 2023-09-09T23:13:40.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.585+0000 7f5576d67700 1 -- 172.21.15.82:0/1881329761 wait complete. 2023-09-09T23:13:40.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.586+0000 7f5576d67700 1 Processor -- start 2023-09-09T23:13:40.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.586+0000 7f5576d67700 1 -- start start 2023-09-09T23:13:40.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.586+0000 7f5576d67700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5570101100 0x7f5570196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:40.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.586+0000 7f5576d67700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5570103a30 0x7f5570196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:40.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.586+0000 7f5576d67700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5570106360 0x7f557019d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:40.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.586+0000 7f5576d67700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557019dda0 con 0x7f5570106360 2023-09-09T23:13:40.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.586+0000 7f5576d67700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557019dee0 con 0x7f5570101100 2023-09-09T23:13:40.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.586+0000 7f5576d67700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557019e070 con 0x7f5570103a30 2023-09-09T23:13:40.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.586+0000 7f5575304700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5570106360 0x7f557019d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:40.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.586+0000 7f5575304700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5570106360 0x7f557019d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55978/0 (socket says 172.21.15.82:55978) 2023-09-09T23:13:40.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.586+0000 7f5575304700 1 -- 172.21.15.82:0/1416172349 learned_addr learned my addr 172.21.15.82:0/1416172349 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:40.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.586+0000 7f5574b03700 1 --2- 172.21.15.82:0/1416172349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5570101100 0x7f5570196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:40.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.586+0000 7f556ffff700 1 --2- 172.21.15.82:0/1416172349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5570103a30 0x7f5570196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:40.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.587+0000 7f5575304700 1 -- 172.21.15.82:0/1416172349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5570103a30 msgr2=0x7f5570196760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:40.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.587+0000 7f5575304700 1 --2- 172.21.15.82:0/1416172349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5570103a30 0x7f5570196760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.587+0000 7f5575304700 1 -- 172.21.15.82:0/1416172349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5570101100 msgr2=0x7f5570196220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:40.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.587+0000 7f5575304700 1 --2- 172.21.15.82:0/1416172349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5570101100 0x7f5570196220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.587+0000 7f5575304700 1 -- 172.21.15.82:0/1416172349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55640095f0 con 0x7f5570106360 2023-09-09T23:13:40.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.587+0000 7f5575304700 1 --2- 172.21.15.82:0/1416172349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5570106360 0x7f557019d7d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5568007ee0 tx=0x7f556800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:40.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.587+0000 7f556dffb700 1 -- 172.21.15.82:0/1416172349 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5568017070 con 0x7f5570106360 2023-09-09T23:13:40.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.587+0000 7f5576d67700 1 -- 172.21.15.82:0/1416172349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f557019e350 con 0x7f5570106360 2023-09-09T23:13:40.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.587+0000 7f556dffb700 1 -- 172.21.15.82:0/1416172349 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f556800f040 con 0x7f5570106360 2023-09-09T23:13:40.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.587+0000 7f556dffb700 1 -- 172.21.15.82:0/1416172349 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5568005550 con 0x7f5570106360 2023-09-09T23:13:40.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.587+0000 7f5576d67700 1 -- 172.21.15.82:0/1416172349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f557019e8b0 con 0x7f5570106360 2023-09-09T23:13:40.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.588+0000 7f5576d67700 1 -- 172.21.15.82:0/1416172349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f557010be20 con 0x7f5570106360 2023-09-09T23:13:40.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.589+0000 7f556dffb700 1 -- 172.21.15.82:0/1416172349 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f556800db20 con 0x7f5570106360 2023-09-09T23:13:40.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.589+0000 7f556dffb700 1 --2- 172.21.15.82:0/1416172349 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f55600a0640 0x7f55600a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:40.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.590+0000 7f556dffb700 1 -- 172.21.15.82:0/1416172349 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f55680cb230 con 0x7f5570106360 2023-09-09T23:13:40.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.590+0000 7f5574b03700 1 --2- 172.21.15.82:0/1416172349 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f55600a0640 0x7f55600a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:40.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.590+0000 7f5574b03700 1 --2- 172.21.15.82:0/1416172349 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f55600a0640 0x7f55600a2b00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f556400b5c0 tx=0x7f5564005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:40.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.594+0000 7f556dffb700 1 -- 172.21.15.82:0/1416172349 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5568094e00 con 0x7f5570106360 2023-09-09T23:13:40.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:40 smithi170 ceph-mon[102832]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:13:40.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:40 smithi170 ceph-mon[102832]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-09-09T23:13:40.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:40 smithi098 ceph-mon[103124]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:13:40.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:40 smithi098 ceph-mon[103124]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-09-09T23:13:40.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.781+0000 7f5576d67700 1 -- 172.21.15.82:0/1416172349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f557019ead0 con 0x7f5570106360 2023-09-09T23:13:40.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.781+0000 7f556dffb700 1 -- 172.21.15.82:0/1416172349 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5568098420 con 0x7f5570106360 2023-09-09T23:13:40.780 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2023-09-09T23:13:40.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.783+0000 7f5576d67700 1 -- 172.21.15.82:0/1416172349 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f55600a0640 msgr2=0x7f55600a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:40.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.783+0000 7f5576d67700 1 --2- 172.21.15.82:0/1416172349 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f55600a0640 0x7f55600a2b00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f556400b5c0 tx=0x7f5564005960 comp rx=0 tx=0).stop 2023-09-09T23:13:40.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.783+0000 7f5576d67700 1 -- 172.21.15.82:0/1416172349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5570106360 msgr2=0x7f557019d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:40.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.783+0000 7f5576d67700 1 --2- 172.21.15.82:0/1416172349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5570106360 0x7f557019d7d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5568007ee0 tx=0x7f556800d3b0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.784+0000 7f5576d67700 1 -- 172.21.15.82:0/1416172349 shutdown_connections 2023-09-09T23:13:40.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.784+0000 7f5576d67700 1 --2- 172.21.15.82:0/1416172349 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f55600a0640 0x7f55600a2b00 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.784+0000 7f5576d67700 1 --2- 172.21.15.82:0/1416172349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5570101100 0x7f5570196220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.784+0000 7f5576d67700 1 --2- 172.21.15.82:0/1416172349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5570106360 0x7f557019d7d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.784+0000 7f5576d67700 1 --2- 172.21.15.82:0/1416172349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5570103a30 0x7f5570196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.784+0000 7f5576d67700 1 -- 172.21.15.82:0/1416172349 >> 172.21.15.82:0/1416172349 conn(0x7f55700faa90 msgr2=0x7f5570102550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:40.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.784+0000 7f5576d67700 1 -- 172.21.15.82:0/1416172349 shutdown_connections 2023-09-09T23:13:40.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.784+0000 7f5576d67700 1 -- 172.21.15.82:0/1416172349 wait complete. 2023-09-09T23:13:40.794 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-09T23:13:40.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.920+0000 7ff7411ea700 1 -- 172.21.15.82:0/1945035104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff73c1010f0 msgr2=0x7ff73c1079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:40.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.920+0000 7ff7411ea700 1 --2- 172.21.15.82:0/1945035104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff73c1010f0 0x7ff73c1079a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff73400b0d0 tx=0x7ff73400b3e0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.920+0000 7ff7411ea700 1 -- 172.21.15.82:0/1945035104 shutdown_connections 2023-09-09T23:13:40.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.920+0000 7ff7411ea700 1 --2- 172.21.15.82:0/1945035104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff73c1010f0 0x7ff73c1079a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.920+0000 7ff7411ea700 1 --2- 172.21.15.82:0/1945035104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff73c0ff560 0x7ff73c0ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.920+0000 7ff7411ea700 1 --2- 172.21.15.82:0/1945035104 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff73c100750 0x7ff73c100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.920+0000 7ff7411ea700 1 -- 172.21.15.82:0/1945035104 >> 172.21.15.82:0/1945035104 conn(0x7ff73c0faad0 msgr2=0x7ff73c0fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:40.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.920+0000 7ff7411ea700 1 -- 172.21.15.82:0/1945035104 shutdown_connections 2023-09-09T23:13:40.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.920+0000 7ff7411ea700 1 -- 172.21.15.82:0/1945035104 wait complete. 2023-09-09T23:13:40.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.921+0000 7ff7411ea700 1 Processor -- start 2023-09-09T23:13:40.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.921+0000 7ff7411ea700 1 -- start start 2023-09-09T23:13:40.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.921+0000 7ff7411ea700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff73c0ff560 0x7ff73c1963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:40.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.921+0000 7ff7411ea700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff73c100750 0x7ff73c1968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:40.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.921+0000 7ff7411ea700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff73c1010f0 0x7ff73c19d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:40.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.921+0000 7ff7411ea700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c19df30 con 0x7ff73c0ff560 2023-09-09T23:13:40.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.921+0000 7ff7411ea700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c19e070 con 0x7ff73c1010f0 2023-09-09T23:13:40.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.921+0000 7ff7411ea700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c19e200 con 0x7ff73c100750 2023-09-09T23:13:40.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.921+0000 7ff73ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff73c0ff560 0x7ff73c1963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:40.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.921+0000 7ff73ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff73c0ff560 0x7ff73c1963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55988/0 (socket says 172.21.15.82:55988) 2023-09-09T23:13:40.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.921+0000 7ff73ad9d700 1 -- 172.21.15.82:0/3650934540 learned_addr learned my addr 172.21.15.82:0/3650934540 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:40.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.921+0000 7ff73b59e700 1 --2- 172.21.15.82:0/3650934540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff73c1010f0 0x7ff73c19d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:40.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.921+0000 7ff73a59c700 1 --2- 172.21.15.82:0/3650934540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff73c100750 0x7ff73c1968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:40.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.921+0000 7ff73ad9d700 1 -- 172.21.15.82:0/3650934540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff73c100750 msgr2=0x7ff73c1968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:40.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.921+0000 7ff73ad9d700 1 --2- 172.21.15.82:0/3650934540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff73c100750 0x7ff73c1968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.921+0000 7ff73ad9d700 1 -- 172.21.15.82:0/3650934540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff73c1010f0 msgr2=0x7ff73c19d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:40.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.921+0000 7ff73ad9d700 1 --2- 172.21.15.82:0/3650934540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff73c1010f0 0x7ff73c19d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:40.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.921+0000 7ff73ad9d700 1 -- 172.21.15.82:0/3650934540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff734009d00 con 0x7ff73c0ff560 2023-09-09T23:13:40.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.921+0000 7ff73a59c700 1 --2- 172.21.15.82:0/3650934540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff73c100750 0x7ff73c1968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:40.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.922+0000 7ff73ad9d700 1 --2- 172.21.15.82:0/3650934540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff73c0ff560 0x7ff73c1963b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff73000d980 tx=0x7ff73000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:40.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.922+0000 7ff723fff700 1 -- 172.21.15.82:0/3650934540 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff730004020 con 0x7ff73c0ff560 2023-09-09T23:13:40.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.922+0000 7ff723fff700 1 -- 172.21.15.82:0/3650934540 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff730009730 con 0x7ff73c0ff560 2023-09-09T23:13:40.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.922+0000 7ff7411ea700 1 -- 172.21.15.82:0/3650934540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff73c19e4e0 con 0x7ff73c0ff560 2023-09-09T23:13:40.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.922+0000 7ff723fff700 1 -- 172.21.15.82:0/3650934540 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff730003a80 con 0x7ff73c0ff560 2023-09-09T23:13:40.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.922+0000 7ff7411ea700 1 -- 172.21.15.82:0/3650934540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff73c19e9f0 con 0x7ff73c0ff560 2023-09-09T23:13:40.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.923+0000 7ff7411ea700 1 -- 172.21.15.82:0/3650934540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff73c10ad00 con 0x7ff73c0ff560 2023-09-09T23:13:40.928 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.924+0000 7ff723fff700 1 -- 172.21.15.82:0/3650934540 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7ff7300098a0 con 0x7ff73c0ff560 2023-09-09T23:13:40.928 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.924+0000 7ff723fff700 1 --2- 172.21.15.82:0/3650934540 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff7240a0640 0x7ff7240a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:40.928 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.924+0000 7ff73a59c700 1 --2- 172.21.15.82:0/3650934540 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff7240a0640 0x7ff7240a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:40.928 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.924+0000 7ff723fff700 1 -- 172.21.15.82:0/3650934540 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff7300cbb30 con 0x7ff73c0ff560 2023-09-09T23:13:40.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.925+0000 7ff73a59c700 1 --2- 172.21.15.82:0/3650934540 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff7240a0640 0x7ff7240a2b00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff72c005950 tx=0x7ff72c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:40.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:40.928+0000 7ff723fff700 1 -- 172.21.15.82:0/3650934540 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff730095700 con 0x7ff73c0ff560 2023-09-09T23:13:41.117 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.119+0000 7ff7411ea700 1 -- 172.21.15.82:0/3650934540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff73c19ec70 con 0x7ff73c0ff560 2023-09-09T23:13:41.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.119+0000 7ff723fff700 1 -- 172.21.15.82:0/3650934540 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff730098d20 con 0x7ff73c0ff560 2023-09-09T23:13:41.118 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2023-09-09T23:13:41.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.121+0000 7ff7411ea700 1 -- 172.21.15.82:0/3650934540 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff7240a0640 msgr2=0x7ff7240a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:41.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.121+0000 7ff7411ea700 1 --2- 172.21.15.82:0/3650934540 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff7240a0640 0x7ff7240a2b00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff72c005950 tx=0x7ff72c0058e0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.121+0000 7ff7411ea700 1 -- 172.21.15.82:0/3650934540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff73c0ff560 msgr2=0x7ff73c1963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:41.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.121+0000 7ff7411ea700 1 --2- 172.21.15.82:0/3650934540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff73c0ff560 0x7ff73c1963b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff73000d980 tx=0x7ff73000dc90 comp rx=0 tx=0).stop 2023-09-09T23:13:41.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.121+0000 7ff7411ea700 1 -- 172.21.15.82:0/3650934540 shutdown_connections 2023-09-09T23:13:41.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.121+0000 7ff7411ea700 1 --2- 172.21.15.82:0/3650934540 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff7240a0640 0x7ff7240a2b00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.121+0000 7ff7411ea700 1 --2- 172.21.15.82:0/3650934540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff73c1010f0 0x7ff73c19d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.121+0000 7ff7411ea700 1 --2- 172.21.15.82:0/3650934540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff73c0ff560 0x7ff73c1963b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.121+0000 7ff7411ea700 1 --2- 172.21.15.82:0/3650934540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff73c100750 0x7ff73c1968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.122 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.121+0000 7ff7411ea700 1 -- 172.21.15.82:0/3650934540 >> 172.21.15.82:0/3650934540 conn(0x7ff73c0faad0 msgr2=0x7ff73c0689d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:41.122 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.121+0000 7ff7411ea700 1 -- 172.21.15.82:0/3650934540 shutdown_connections 2023-09-09T23:13:41.122 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.121+0000 7ff7411ea700 1 -- 172.21.15.82:0/3650934540 wait complete. 2023-09-09T23:13:41.132 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-09T23:13:41.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.257+0000 7f5751bf9700 1 -- 172.21.15.82:0/3326327856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f574c1039a0 msgr2=0x7f574c105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:41.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.257+0000 7f5751bf9700 1 --2- 172.21.15.82:0/3326327856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f574c1039a0 0x7f574c105d90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f573c009990 tx=0x7f573c009ca0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.257+0000 7f5751bf9700 1 -- 172.21.15.82:0/3326327856 shutdown_connections 2023-09-09T23:13:41.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.257+0000 7f5751bf9700 1 --2- 172.21.15.82:0/3326327856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f574c101070 0x7f574c103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.257+0000 7f5751bf9700 1 --2- 172.21.15.82:0/3326327856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f574c1062d0 0x7f574c108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.257+0000 7f5751bf9700 1 --2- 172.21.15.82:0/3326327856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f574c1039a0 0x7f574c105d90 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.257+0000 7f5751bf9700 1 -- 172.21.15.82:0/3326327856 >> 172.21.15.82:0/3326327856 conn(0x7f574c0faa60 msgr2=0x7f574c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:41.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.257+0000 7f5751bf9700 1 -- 172.21.15.82:0/3326327856 shutdown_connections 2023-09-09T23:13:41.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.257+0000 7f5751bf9700 1 -- 172.21.15.82:0/3326327856 wait complete. 2023-09-09T23:13:41.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.257+0000 7f5751bf9700 1 Processor -- start 2023-09-09T23:13:41.260 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.258+0000 7f5751bf9700 1 -- start start 2023-09-09T23:13:41.260 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.258+0000 7f5751bf9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f574c101070 0x7f574c196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:41.260 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.258+0000 7f5751bf9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f574c1039a0 0x7f574c196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:41.261 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.258+0000 7f5751bf9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f574c1062d0 0x7f574c19d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:41.261 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.258+0000 7f5751bf9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c19dea0 con 0x7f574c1039a0 2023-09-09T23:13:41.261 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.258+0000 7f5751bf9700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c19dfe0 con 0x7f574c101070 2023-09-09T23:13:41.261 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.258+0000 7f5751bf9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c19e170 con 0x7f574c1062d0 2023-09-09T23:13:41.261 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.258+0000 7f574affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f574c1039a0 0x7f574c196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:41.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.258+0000 7f574affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f574c1039a0 0x7f574c196860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56004/0 (socket says 172.21.15.82:56004) 2023-09-09T23:13:41.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.258+0000 7f574affd700 1 -- 172.21.15.82:0/2249565459 learned_addr learned my addr 172.21.15.82:0/2249565459 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:41.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.258+0000 7f574bfff700 1 --2- 172.21.15.82:0/2249565459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f574c1062d0 0x7f574c19d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:41.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.258+0000 7f574b7fe700 1 --2- 172.21.15.82:0/2249565459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f574c101070 0x7f574c196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:41.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.258+0000 7f574affd700 1 -- 172.21.15.82:0/2249565459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f574c1062d0 msgr2=0x7f574c19d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:41.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.258+0000 7f574affd700 1 --2- 172.21.15.82:0/2249565459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f574c1062d0 0x7f574c19d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.258+0000 7f574affd700 1 -- 172.21.15.82:0/2249565459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f574c101070 msgr2=0x7f574c196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:41.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.258+0000 7f574affd700 1 --2- 172.21.15.82:0/2249565459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f574c101070 0x7f574c196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.258+0000 7f574affd700 1 -- 172.21.15.82:0/2249565459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573c0095f0 con 0x7f574c1039a0 2023-09-09T23:13:41.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.258+0000 7f574bfff700 1 --2- 172.21.15.82:0/2249565459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f574c1062d0 0x7f574c19d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:41.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.258+0000 7f574b7fe700 1 --2- 172.21.15.82:0/2249565459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f574c101070 0x7f574c196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:41.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.258+0000 7f574affd700 1 --2- 172.21.15.82:0/2249565459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f574c1039a0 0x7f574c196860 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f573c004730 tx=0x7f573c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:41.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.258+0000 7f5748ff9700 1 -- 172.21.15.82:0/2249565459 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c01d070 con 0x7f574c1039a0 2023-09-09T23:13:41.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.258+0000 7f5748ff9700 1 -- 172.21.15.82:0/2249565459 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f573c004cd0 con 0x7f574c1039a0 2023-09-09T23:13:41.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.259+0000 7f5751bf9700 1 -- 172.21.15.82:0/2249565459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574c19e3f0 con 0x7f574c1039a0 2023-09-09T23:13:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.259+0000 7f5748ff9700 1 -- 172.21.15.82:0/2249565459 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c005610 con 0x7f574c1039a0 2023-09-09T23:13:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.259+0000 7f5751bf9700 1 -- 172.21.15.82:0/2249565459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574c19e810 con 0x7f574c1039a0 2023-09-09T23:13:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.259+0000 7f5751bf9700 1 -- 172.21.15.82:0/2249565459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f574c0fc5e0 con 0x7f574c1039a0 2023-09-09T23:13:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.260+0000 7f5748ff9700 1 -- 172.21.15.82:0/2249565459 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f573c00fc70 con 0x7f574c1039a0 2023-09-09T23:13:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.261+0000 7f5748ff9700 1 --2- 172.21.15.82:0/2249565459 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f57340a06c0 0x7f57340a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:41.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.261+0000 7f574b7fe700 1 --2- 172.21.15.82:0/2249565459 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f57340a06c0 0x7f57340a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:41.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.261+0000 7f5748ff9700 1 -- 172.21.15.82:0/2249565459 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f573c0cd360 con 0x7f574c1039a0 2023-09-09T23:13:41.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.261+0000 7f574b7fe700 1 --2- 172.21.15.82:0/2249565459 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f57340a06c0 0x7f57340a2b80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5740009be0 tx=0x7f5740009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:41.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.264+0000 7f5748ff9700 1 -- 172.21.15.82:0/2249565459 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f573c096fc0 con 0x7f574c1039a0 2023-09-09T23:13:41.454 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:41 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2978953457' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:41.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:41 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1416172349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:13:41.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:41 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3650934540' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-09T23:13:41.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.458+0000 7f5751bf9700 1 -- 172.21.15.82:0/2249565459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f574c0008d0 con 0x7f574c1039a0 2023-09-09T23:13:41.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.458+0000 7f5748ff9700 1 -- 172.21.15.82:0/2249565459 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f573c09a5e0 con 0x7f574c1039a0 2023-09-09T23:13:41.457 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2023-09-09T23:13:41.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.460+0000 7f5751bf9700 1 -- 172.21.15.82:0/2249565459 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f57340a06c0 msgr2=0x7f57340a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:41.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.460+0000 7f5751bf9700 1 --2- 172.21.15.82:0/2249565459 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f57340a06c0 0x7f57340a2b80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5740009be0 tx=0x7f5740009260 comp rx=0 tx=0).stop 2023-09-09T23:13:41.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.460+0000 7f5751bf9700 1 -- 172.21.15.82:0/2249565459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f574c1039a0 msgr2=0x7f574c196860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:41.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.460+0000 7f5751bf9700 1 --2- 172.21.15.82:0/2249565459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f574c1039a0 0x7f574c196860 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f573c004730 tx=0x7f573c005ef0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.460+0000 7f5751bf9700 1 -- 172.21.15.82:0/2249565459 shutdown_connections 2023-09-09T23:13:41.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.460+0000 7f5751bf9700 1 --2- 172.21.15.82:0/2249565459 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f57340a06c0 0x7f57340a2b80 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.460+0000 7f5751bf9700 1 --2- 172.21.15.82:0/2249565459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f574c101070 0x7f574c196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.460+0000 7f5751bf9700 1 --2- 172.21.15.82:0/2249565459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f574c1062d0 0x7f574c19d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.460+0000 7f5751bf9700 1 --2- 172.21.15.82:0/2249565459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f574c1039a0 0x7f574c196860 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.460+0000 7f5751bf9700 1 -- 172.21.15.82:0/2249565459 >> 172.21.15.82:0/2249565459 conn(0x7f574c0faa60 msgr2=0x7f574c101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:41.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.461+0000 7f5751bf9700 1 -- 172.21.15.82:0/2249565459 shutdown_connections 2023-09-09T23:13:41.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.461+0000 7f5751bf9700 1 -- 172.21.15.82:0/2249565459 wait complete. 2023-09-09T23:13:41.471 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-09T23:13:41.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.594+0000 7f3ae5698700 1 -- 172.21.15.82:0/246111569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ae01006d0 msgr2=0x7f3ae0100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:41.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.594+0000 7f3ae5698700 1 --2- 172.21.15.82:0/246111569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ae01006d0 0x7f3ae0100b30 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3ad00099c0 tx=0x7f3ad0009cd0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.594+0000 7f3ae5698700 1 -- 172.21.15.82:0/246111569 shutdown_connections 2023-09-09T23:13:41.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.594+0000 7f3ae5698700 1 --2- 172.21.15.82:0/246111569 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ae0101070 0x7f3ae0107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.594+0000 7f3ae5698700 1 --2- 172.21.15.82:0/246111569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ae00ff4e0 0x7f3ae00ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.594+0000 7f3ae5698700 1 --2- 172.21.15.82:0/246111569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ae01006d0 0x7f3ae0100b30 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.594+0000 7f3ae5698700 1 -- 172.21.15.82:0/246111569 >> 172.21.15.82:0/246111569 conn(0x7f3ae00faa30 msgr2=0x7f3ae00fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:41.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.594+0000 7f3ae5698700 1 -- 172.21.15.82:0/246111569 shutdown_connections 2023-09-09T23:13:41.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.594+0000 7f3ae5698700 1 -- 172.21.15.82:0/246111569 wait complete. 2023-09-09T23:13:41.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.594+0000 7f3ae5698700 1 Processor -- start 2023-09-09T23:13:41.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.594+0000 7f3ae5698700 1 -- start start 2023-09-09T23:13:41.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.595+0000 7f3ae5698700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ae00ff4e0 0x7f3ae00682f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:41.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.595+0000 7f3ae5698700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ae01006d0 0x7f3ae019aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:41.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.595+0000 7f3ae5698700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ae0101070 0x7f3ae019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:41.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.595+0000 7f3ae5698700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae019de10 con 0x7f3ae0101070 2023-09-09T23:13:41.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.595+0000 7f3ae5698700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae019df80 con 0x7f3ae01006d0 2023-09-09T23:13:41.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.595+0000 7f3ae5698700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae019e140 con 0x7f3ae00ff4e0 2023-09-09T23:13:41.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.595+0000 7f3adf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ae0101070 0x7f3ae019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:41.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.595+0000 7f3adf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ae0101070 0x7f3ae019d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56020/0 (socket says 172.21.15.82:56020) 2023-09-09T23:13:41.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.595+0000 7f3adf7fe700 1 -- 172.21.15.82:0/3510201257 learned_addr learned my addr 172.21.15.82:0/3510201257 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:41.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.595+0000 7f3ade7fc700 1 --2- 172.21.15.82:0/3510201257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ae01006d0 0x7f3ae019aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:41.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.595+0000 7f3adf7fe700 1 -- 172.21.15.82:0/3510201257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ae00ff4e0 msgr2=0x7f3ae00682f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:41.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.595+0000 7f3adeffd700 1 --2- 172.21.15.82:0/3510201257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ae00ff4e0 0x7f3ae00682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:41.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.595+0000 7f3adf7fe700 1 --2- 172.21.15.82:0/3510201257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ae00ff4e0 0x7f3ae00682f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.595+0000 7f3adf7fe700 1 -- 172.21.15.82:0/3510201257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ae01006d0 msgr2=0x7f3ae019aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:41.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.595+0000 7f3adf7fe700 1 --2- 172.21.15.82:0/3510201257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ae01006d0 0x7f3ae019aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.595+0000 7f3adf7fe700 1 -- 172.21.15.82:0/3510201257 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad00095f0 con 0x7f3ae0101070 2023-09-09T23:13:41.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.595+0000 7f3adf7fe700 1 --2- 172.21.15.82:0/3510201257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ae0101070 0x7f3ae019d640 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8007d70 tx=0x7f3ad800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:41.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.595+0000 7f3ac7fff700 1 -- 172.21.15.82:0/3510201257 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad800ad40 con 0x7f3ae0101070 2023-09-09T23:13:41.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.595+0000 7f3ac7fff700 1 -- 172.21.15.82:0/3510201257 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ad80128e0 con 0x7f3ae0101070 2023-09-09T23:13:41.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.595+0000 7f3ae5698700 1 -- 172.21.15.82:0/3510201257 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae019e420 con 0x7f3ae0101070 2023-09-09T23:13:41.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.595+0000 7f3ac7fff700 1 -- 172.21.15.82:0/3510201257 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8011780 con 0x7f3ae0101070 2023-09-09T23:13:41.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.595+0000 7f3ae5698700 1 -- 172.21.15.82:0/3510201257 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae019e950 con 0x7f3ae0101070 2023-09-09T23:13:41.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.596+0000 7f3ae5698700 1 -- 172.21.15.82:0/3510201257 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae0106a20 con 0x7f3ae0101070 2023-09-09T23:13:41.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.597+0000 7f3ac7fff700 1 -- 172.21.15.82:0/3510201257 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f3ad8012a50 con 0x7f3ae0101070 2023-09-09T23:13:41.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.598+0000 7f3ac7fff700 1 --2- 172.21.15.82:0/3510201257 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3ac80a0710 0x7f3ac80a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:41.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.598+0000 7f3ac7fff700 1 -- 172.21.15.82:0/3510201257 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f3ad80cc2a0 con 0x7f3ae0101070 2023-09-09T23:13:41.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.598+0000 7f3adeffd700 1 --2- 172.21.15.82:0/3510201257 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3ac80a0710 0x7f3ac80a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:41.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.598+0000 7f3adeffd700 1 --2- 172.21.15.82:0/3510201257 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3ac80a0710 0x7f3ac80a2bd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4006fd0 tx=0x7f3ad4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:41.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.601+0000 7f3ac7fff700 1 -- 172.21.15.82:0/3510201257 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3ad8095dd0 con 0x7f3ae0101070 2023-09-09T23:13:41.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:41 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2978953457' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:41.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:41 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1416172349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:13:41.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:41 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3650934540' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-09T23:13:41.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:41 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2978953457' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:41.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:41 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1416172349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:13:41.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:41 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3650934540' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-09T23:13:41.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.793+0000 7f3ae5698700 1 -- 172.21.15.82:0/3510201257 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ae010ad90 con 0x7f3ae0101070 2023-09-09T23:13:41.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.794+0000 7f3ac7fff700 1 -- 172.21.15.82:0/3510201257 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22411 (secure 0 0 0) 0x7f3ad80993f0 con 0x7f3ae0101070 2023-09-09T23:13:41.793 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:41.793 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":83,"fsid":"3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da","created":"2023-09-09T23:05:14.222190+0000","modified":"2023-09-09T23:13:10.324596+0000","last_up_change":"2023-09-09T23:11:37.253759+0000","last_in_change":"2023-09-09T23:11:19.933812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-09T23:08:31.234958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-09T23:12:15.524887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-09T23:12:16.785403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1277eb17-a2e7-4e10-9832-f12a5e93ffde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6803","nonce":3197434929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6805","nonce":3197434929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6809","nonce":3197434929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6807","nonce":3197434929}]},"public_addr":"172.21.15.82:6803/3197434929","cluster_addr":"172.21.15.82:6805/3197434929","heartbeat_back_addr":"172.21.15.82:6809/3197434929","heartbeat_front_addr":"172.21.15.82:6807/3197434929","state":["exists","up"]},{"osd":1,"uuid":"f68723fd-0908-4e21-ae8e-1de1c3a9a65c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6801","nonce":3528641314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6803","nonce":3528641314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6807","nonce":3528641314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6805","nonce":3528641314}]},"public_addr":"172.21.15.98:6801/3528641314","cluster_addr":"172.21.15.98:6803/3528641314","heartbeat_back_addr":"172.21.15.98:6807/3528641314","heartbeat_front_addr":"172.21.15.98:6805/3528641314","state":["exists","up"]},{"osd":2,"uuid":"06f89c53-1419-4f52-acad-6e92916b476f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6801","nonce":4006414551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6803","nonce":4006414551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6807","nonce":4006414551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6805","nonce":4006414551}]},"public_addr":"172.21.15.170:6801/4006414551","cluster_addr":"172.21.15.170:6803/4006414551","heartbeat_back_addr":"172.21.15.170:6807/4006414551","heartbeat_front_addr":"172.21.15.170:6805/4006414551","state":["exists","up"]},{"osd":3,"uuid":"fedbd4c9-6852-42b5-8f3a-ba8d9bfe5d59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6811","nonce":3647629241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6813","nonce":3647629241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6817","nonce":3647629241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6815","nonce":3647629241}]},"public_addr":"172.21.15.82:6811/3647629241","cluster_addr":"172.21.15.82:6813/3647629241","heartbeat_back_addr":"172.21.15.82:6817/3647629241","heartbeat_front_addr":"172.21.15.82:6815/3647629241","state":["exists","up"]},{"osd":4,"uuid":"a71f3084-6919-4161-87bf-345f3355d8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6809","nonce":2507035643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6811","nonce":2507035643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6815","nonce":2507035643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6813","nonce":2507035643}]},"public_addr":"172.21.15.98:6809/2507035643","cluster_addr":"172.21.15.98:6811/2507035643","heartbeat_back_addr":"172.21.15.98:6815/2507035643","heartbeat_front_addr":"172.21.15.98:6813/2507035643","state":["exists","up"]},{"osd":5,"uuid":"f76058e9-a90e-476e-87f0-3c6723795e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6809","nonce":775005168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6811","nonce":775005168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6815","nonce":775005168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6813","nonce":775005168}]},"public_addr":"172.21.15.170:6809/775005168","cluster_addr":"172.21.15.170:6811/775005168","heartbeat_back_addr":"172.21.15.170:6815/775005168","heartbeat_front_addr":"172.21.15.170:6813/775005168","state":["exists","up"]},{"osd":6,"uuid":"25940be0-c1bd-4c72-95bf-fd8367aa2f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6819","nonce":3008405982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6821","nonce":3008405982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6825","nonce":3008405982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6823","nonce":3008405982}]},"public_addr":"172.21.15.82:6819/3008405982","cluster_addr":"172.21.15.82:6821/3008405982","heartbeat_back_addr":"172.21.15.82:6825/3008405982","heartbeat_front_addr":"172.21.15.82:6823/3008405982","state":["exists","up"]},{"osd":7,"uuid":"f651bb8c-0b25-42d5-b028-58563f4c27f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6817","nonce":3547615284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6819","nonce":3547615284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6823","nonce":3547615284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6821","nonce":3547615284}]},"public_addr":"172.21.15.98:6817/3547615284","cluster_addr":"172.21.15.98:6819/3547615284","heartbeat_back_addr":"172.21.15.98:6823/3547615284","heartbeat_front_addr":"172.21.15.98:6821/3547615284","state":["exists","up"]},{"osd":8,"uuid":"23b27c86-8c98-441e-b7ab-d57bc584e58f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6817","nonce":632038848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6819","nonce":632038848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6823","nonce":632038848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6821","nonce":632038848}]},"public_addr":"172.21.15.170:6817/632038848","cluster_addr":"172.21.15.170:6819/632038848","heartbeat_back_addr":"172.21.15.170:6823/632038848","heartbeat_front_addr":"172.21.15.170:6821/632038848","state":["exists","up"]},{"osd":9,"uuid":"383b8989-4922-4eee-9c77-f198258ac243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6827","nonce":3354242311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6829","nonce":3354242311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6833","nonce":3354242311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6831","nonce":3354242311}]},"public_addr":"172.21.15.82:6827/3354242311","cluster_addr":"172.21.15.82:6829/3354242311","heartbeat_back_addr":"172.21.15.82:6833/3354242311","heartbeat_front_addr":"172.21.15.82:6831/3354242311","state":["exists","up"]},{"osd":10,"uuid":"33a59389-7ced-42d5-bf54-2071317132d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6825","nonce":2069114959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6827","nonce":2069114959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6831","nonce":2069114959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6829","nonce":2069114959}]},"public_addr":"172.21.15.98:6825/2069114959","cluster_addr":"172.21.15.98:6827/2069114959","heartbeat_back_addr":"172.21.15.98:6831/2069114959","heartbeat_front_addr":"172.21.15.98:6829/2069114959","state":["exists","up"]},{"osd":11,"uuid":"b7a71f5c-85e2-4fec-be4b-95be0788b302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6825","nonce":146964512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6827","nonce":146964512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6831","nonce":146964512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6829","nonce":146964512}]},"public_addr":"172.21.15.170:6825/146964512","cluster_addr":"172.21.15.170:6827/146964512","heartbeat_back_addr":"172.21.15.170:6831/146964512","heartbeat_front_addr":"172.21.15.170:6829/146964512","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:07:47.563043+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:08.359260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:29.070374+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:49.897775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:11.434248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:31.538464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:52.377109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:13.452968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:33.806897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:54.083177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:14.980087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:35.690089+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4013747788":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6800/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/2823341464":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/75116806":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/1037625632":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/3142088817":"2023-09-10T23:05:46.550354+0000","172.21.15.82:6800/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/1679622963":"2023-09-10T23:05:46.550354+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-09T23:13:41.795 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.796+0000 7f3ae5698700 1 -- 172.21.15.82:0/3510201257 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3ac80a0710 msgr2=0x7f3ac80a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:41.795 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.796+0000 7f3ae5698700 1 --2- 172.21.15.82:0/3510201257 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3ac80a0710 0x7f3ac80a2bd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4006fd0 tx=0x7f3ad4008040 comp rx=0 tx=0).stop 2023-09-09T23:13:41.796 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.796+0000 7f3ae5698700 1 -- 172.21.15.82:0/3510201257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ae0101070 msgr2=0x7f3ae019d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:41.796 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.797+0000 7f3ae5698700 1 --2- 172.21.15.82:0/3510201257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ae0101070 0x7f3ae019d640 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8007d70 tx=0x7f3ad800a340 comp rx=0 tx=0).stop 2023-09-09T23:13:41.796 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.797+0000 7f3ae5698700 1 -- 172.21.15.82:0/3510201257 shutdown_connections 2023-09-09T23:13:41.796 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.797+0000 7f3ae5698700 1 --2- 172.21.15.82:0/3510201257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ae0101070 0x7f3ae019d640 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.796 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.797+0000 7f3ae5698700 1 --2- 172.21.15.82:0/3510201257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ae00ff4e0 0x7f3ae00682f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.796 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.797+0000 7f3ae5698700 1 --2- 172.21.15.82:0/3510201257 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3ac80a0710 0x7f3ac80a2bd0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.797 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.797+0000 7f3ae5698700 1 --2- 172.21.15.82:0/3510201257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ae01006d0 0x7f3ae019aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.797 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.797+0000 7f3ae5698700 1 -- 172.21.15.82:0/3510201257 >> 172.21.15.82:0/3510201257 conn(0x7f3ae00faa30 msgr2=0x7f3ae0103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:41.797 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.797+0000 7f3ae5698700 1 -- 172.21.15.82:0/3510201257 shutdown_connections 2023-09-09T23:13:41.797 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.797+0000 7f3ae5698700 1 -- 172.21.15.82:0/3510201257 wait complete. 2023-09-09T23:13:41.808 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-09T23:08:31.234958+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-09T23:12:15.524887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-09T23:12:16.785403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-09T23:13:41.809 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-09T23:13:41.935 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.935+0000 7f98085c6700 1 -- 172.21.15.82:0/1671831008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9800101080 msgr2=0x7f9800107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:41.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.935+0000 7f98085c6700 1 --2- 172.21.15.82:0/1671831008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9800101080 0x7f9800107930 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f97f800b0d0 tx=0x7f97f800b3e0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.935+0000 7f98085c6700 1 -- 172.21.15.82:0/1671831008 shutdown_connections 2023-09-09T23:13:41.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.935+0000 7f98085c6700 1 --2- 172.21.15.82:0/1671831008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9800101080 0x7f9800107930 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.935+0000 7f98085c6700 1 --2- 172.21.15.82:0/1671831008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98000ff4f0 0x7f98000ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.935+0000 7f98085c6700 1 --2- 172.21.15.82:0/1671831008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98001006e0 0x7f9800100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.935+0000 7f98085c6700 1 -- 172.21.15.82:0/1671831008 >> 172.21.15.82:0/1671831008 conn(0x7f98000faa60 msgr2=0x7f98000fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:41.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.935+0000 7f98085c6700 1 -- 172.21.15.82:0/1671831008 shutdown_connections 2023-09-09T23:13:41.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.935+0000 7f98085c6700 1 -- 172.21.15.82:0/1671831008 wait complete. 2023-09-09T23:13:41.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.935+0000 7f98085c6700 1 Processor -- start 2023-09-09T23:13:41.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.935+0000 7f98085c6700 1 -- start start 2023-09-09T23:13:41.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.935+0000 7f98085c6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98000ff4f0 0x7f98001952e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:41.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.935+0000 7f98085c6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98001006e0 0x7f9800195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:41.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.935+0000 7f98085c6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9800101080 0x7f980019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:41.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.935+0000 7f98085c6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980019ce60 con 0x7f98000ff4f0 2023-09-09T23:13:41.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.935+0000 7f98085c6700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980019cfa0 con 0x7f9800101080 2023-09-09T23:13:41.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.935+0000 7f98085c6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980019d130 con 0x7f98001006e0 2023-09-09T23:13:41.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.936+0000 7f9806362700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98000ff4f0 0x7f98001952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:41.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.936+0000 7f9806362700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98000ff4f0 0x7f98001952e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56030/0 (socket says 172.21.15.82:56030) 2023-09-09T23:13:41.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.936+0000 7f9806362700 1 -- 172.21.15.82:0/405168340 learned_addr learned my addr 172.21.15.82:0/405168340 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:41.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.936+0000 7f9806b63700 1 --2- 172.21.15.82:0/405168340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9800101080 0x7f980019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:41.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.936+0000 7f9805b61700 1 --2- 172.21.15.82:0/405168340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98001006e0 0x7f9800195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:41.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.936+0000 7f9806362700 1 -- 172.21.15.82:0/405168340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98001006e0 msgr2=0x7f9800195820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:41.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.936+0000 7f9806362700 1 --2- 172.21.15.82:0/405168340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98001006e0 0x7f9800195820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.936+0000 7f9806362700 1 -- 172.21.15.82:0/405168340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9800101080 msgr2=0x7f980019c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:41.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.936+0000 7f9806362700 1 --2- 172.21.15.82:0/405168340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9800101080 0x7f980019c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:41.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.936+0000 7f9806362700 1 -- 172.21.15.82:0/405168340 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f8009d00 con 0x7f98000ff4f0 2023-09-09T23:13:41.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.936+0000 7f9806362700 1 --2- 172.21.15.82:0/405168340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98000ff4f0 0x7f98001952e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f97fc00c970 tx=0x7f97fc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:41.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.936+0000 7f97f77fe700 1 -- 172.21.15.82:0/405168340 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc004220 con 0x7f98000ff4f0 2023-09-09T23:13:41.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.936+0000 7f98085c6700 1 -- 172.21.15.82:0/405168340 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f980019d410 con 0x7f98000ff4f0 2023-09-09T23:13:41.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.936+0000 7f97f77fe700 1 -- 172.21.15.82:0/405168340 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97fc007d60 con 0x7f98000ff4f0 2023-09-09T23:13:41.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.936+0000 7f97f77fe700 1 -- 172.21.15.82:0/405168340 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc003c90 con 0x7f98000ff4f0 2023-09-09T23:13:41.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.936+0000 7f98085c6700 1 -- 172.21.15.82:0/405168340 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f980019d970 con 0x7f98000ff4f0 2023-09-09T23:13:41.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.937+0000 7f98085c6700 1 -- 172.21.15.82:0/405168340 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f980010ad00 con 0x7f98000ff4f0 2023-09-09T23:13:41.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.938+0000 7f97f77fe700 1 -- 172.21.15.82:0/405168340 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f97fc004380 con 0x7f98000ff4f0 2023-09-09T23:13:41.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.939+0000 7f97f77fe700 1 --2- 172.21.15.82:0/405168340 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f97ec0a0640 0x7f97ec0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:41.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.939+0000 7f9805b61700 1 --2- 172.21.15.82:0/405168340 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f97ec0a0640 0x7f97ec0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:41.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.939+0000 7f97f77fe700 1 -- 172.21.15.82:0/405168340 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f97fc0cb760 con 0x7f98000ff4f0 2023-09-09T23:13:41.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.939+0000 7f9805b61700 1 --2- 172.21.15.82:0/405168340 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f97ec0a0640 0x7f97ec0a2b00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f97f0009b30 tx=0x7f97f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:41.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:41.943+0000 7f97f77fe700 1 -- 172.21.15.82:0/405168340 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f97fc095290 con 0x7f98000ff4f0 2023-09-09T23:13:42.134 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.135+0000 7f98085c6700 1 -- 172.21.15.82:0/405168340 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f98001063f0 con 0x7f98000ff4f0 2023-09-09T23:13:42.134 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.135+0000 7f97f77fe700 1 -- 172.21.15.82:0/405168340 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f97fc0988b0 con 0x7f98000ff4f0 2023-09-09T23:13:42.134 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2023-09-09T23:13:42.136 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.138+0000 7f98085c6700 1 -- 172.21.15.82:0/405168340 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f97ec0a0640 msgr2=0x7f97ec0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:42.137 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.138+0000 7f98085c6700 1 --2- 172.21.15.82:0/405168340 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f97ec0a0640 0x7f97ec0a2b00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f97f0009b30 tx=0x7f97f0009260 comp rx=0 tx=0).stop 2023-09-09T23:13:42.137 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.138+0000 7f98085c6700 1 -- 172.21.15.82:0/405168340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98000ff4f0 msgr2=0x7f98001952e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:42.137 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.138+0000 7f98085c6700 1 --2- 172.21.15.82:0/405168340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98000ff4f0 0x7f98001952e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f97fc00c970 tx=0x7f97fc00cc80 comp rx=0 tx=0).stop 2023-09-09T23:13:42.137 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.138+0000 7f98085c6700 1 -- 172.21.15.82:0/405168340 shutdown_connections 2023-09-09T23:13:42.137 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.138+0000 7f98085c6700 1 --2- 172.21.15.82:0/405168340 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f97ec0a0640 0x7f97ec0a2b00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.138 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.138+0000 7f98085c6700 1 --2- 172.21.15.82:0/405168340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9800101080 0x7f980019c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.138 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.138+0000 7f98085c6700 1 --2- 172.21.15.82:0/405168340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98000ff4f0 0x7f98001952e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.138 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.138+0000 7f98085c6700 1 --2- 172.21.15.82:0/405168340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98001006e0 0x7f9800195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.138 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.138+0000 7f98085c6700 1 -- 172.21.15.82:0/405168340 >> 172.21.15.82:0/405168340 conn(0x7f98000faa60 msgr2=0x7f98000fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:42.138 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.138+0000 7f98085c6700 1 -- 172.21.15.82:0/405168340 shutdown_connections 2023-09-09T23:13:42.139 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.138+0000 7f98085c6700 1 -- 172.21.15.82:0/405168340 wait complete. 2023-09-09T23:13:42.148 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-09T23:13:42.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.274+0000 7fca48ca0700 1 -- 172.21.15.82:0/2484434957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca44100b80 msgr2=0x7fca44100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:42.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.274+0000 7fca48ca0700 1 --2- 172.21.15.82:0/2484434957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca44100b80 0x7fca44100fa0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fca38009990 tx=0x7fca38009ca0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.274+0000 7fca48ca0700 1 -- 172.21.15.82:0/2484434957 shutdown_connections 2023-09-09T23:13:42.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.274+0000 7fca48ca0700 1 --2- 172.21.15.82:0/2484434957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fca44100100 0x7fca441079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.274+0000 7fca48ca0700 1 --2- 172.21.15.82:0/2484434957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca44100b80 0x7fca44100fa0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.274+0000 7fca48ca0700 1 --2- 172.21.15.82:0/2484434957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca440ff740 0x7fca440ffbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.274+0000 7fca48ca0700 1 -- 172.21.15.82:0/2484434957 >> 172.21.15.82:0/2484434957 conn(0x7fca440fa1d0 msgr2=0x7fca440fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:42.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.274+0000 7fca48ca0700 1 -- 172.21.15.82:0/2484434957 shutdown_connections 2023-09-09T23:13:42.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.274+0000 7fca48ca0700 1 -- 172.21.15.82:0/2484434957 wait complete. 2023-09-09T23:13:42.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.275+0000 7fca48ca0700 1 Processor -- start 2023-09-09T23:13:42.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.275+0000 7fca48ca0700 1 -- start start 2023-09-09T23:13:42.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.275+0000 7fca48ca0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca440ff740 0x7fca441962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:42.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.275+0000 7fca48ca0700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fca44100100 0x7fca44196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:42.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.275+0000 7fca48ca0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca44100b80 0x7fca4419d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:42.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.275+0000 7fca48ca0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4419de40 con 0x7fca440ff740 2023-09-09T23:13:42.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.275+0000 7fca48ca0700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4419df80 con 0x7fca44100100 2023-09-09T23:13:42.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.275+0000 7fca48ca0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4419e110 con 0x7fca44100b80 2023-09-09T23:13:42.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.275+0000 7fca4259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca440ff740 0x7fca441962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:42.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.275+0000 7fca4259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca440ff740 0x7fca441962c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56042/0 (socket says 172.21.15.82:56042) 2023-09-09T23:13:42.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.275+0000 7fca4259c700 1 -- 172.21.15.82:0/1241490386 learned_addr learned my addr 172.21.15.82:0/1241490386 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:42.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.275+0000 7fca42d9d700 1 --2- 172.21.15.82:0/1241490386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca44100b80 0x7fca4419d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:42.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.275+0000 7fca41d9b700 1 --2- 172.21.15.82:0/1241490386 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fca44100100 0x7fca44196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:42.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.275+0000 7fca4259c700 1 -- 172.21.15.82:0/1241490386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca44100b80 msgr2=0x7fca4419d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:42.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.275+0000 7fca4259c700 1 --2- 172.21.15.82:0/1241490386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca44100b80 0x7fca4419d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.275+0000 7fca4259c700 1 -- 172.21.15.82:0/1241490386 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fca44100100 msgr2=0x7fca44196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:42.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.275+0000 7fca4259c700 1 --2- 172.21.15.82:0/1241490386 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fca44100100 0x7fca44196800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.275+0000 7fca4259c700 1 -- 172.21.15.82:0/1241490386 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca380095f0 con 0x7fca440ff740 2023-09-09T23:13:42.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.275+0000 7fca4259c700 1 --2- 172.21.15.82:0/1241490386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca440ff740 0x7fca441962c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fca380059d0 tx=0x7fca380047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:42.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.275+0000 7fca337fe700 1 -- 172.21.15.82:0/1241490386 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3801d070 con 0x7fca440ff740 2023-09-09T23:13:42.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.276+0000 7fca337fe700 1 -- 172.21.15.82:0/1241490386 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca3800fb90 con 0x7fca440ff740 2023-09-09T23:13:42.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.276+0000 7fca48ca0700 1 -- 172.21.15.82:0/1241490386 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca4419e390 con 0x7fca440ff740 2023-09-09T23:13:42.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.276+0000 7fca337fe700 1 -- 172.21.15.82:0/1241490386 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca38017650 con 0x7fca440ff740 2023-09-09T23:13:42.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.276+0000 7fca48ca0700 1 -- 172.21.15.82:0/1241490386 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4419e7b0 con 0x7fca440ff740 2023-09-09T23:13:42.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.276+0000 7fca48ca0700 1 -- 172.21.15.82:0/1241490386 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca44106b50 con 0x7fca440ff740 2023-09-09T23:13:42.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.278+0000 7fca337fe700 1 -- 172.21.15.82:0/1241490386 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fca3800fd00 con 0x7fca440ff740 2023-09-09T23:13:42.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.278+0000 7fca337fe700 1 --2- 172.21.15.82:0/1241490386 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fca2c0a0640 0x7fca2c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:42.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.278+0000 7fca337fe700 1 -- 172.21.15.82:0/1241490386 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fca380cd640 con 0x7fca440ff740 2023-09-09T23:13:42.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.278+0000 7fca41d9b700 1 --2- 172.21.15.82:0/1241490386 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fca2c0a0640 0x7fca2c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:42.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.279+0000 7fca41d9b700 1 --2- 172.21.15.82:0/1241490386 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fca2c0a0640 0x7fca2c0a2b00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fca34005950 tx=0x7fca3400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:42.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.282+0000 7fca337fe700 1 -- 172.21.15.82:0/1241490386 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fca380972a0 con 0x7fca440ff740 2023-09-09T23:13:42.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:42 smithi082 ceph-mon[98694]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-09T23:13:42.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:42 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2249565459' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-09T23:13:42.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:42 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3510201257' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:42.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:42 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/405168340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:13:42.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.477+0000 7fca48ca0700 1 -- 172.21.15.82:0/1241490386 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fca4410bf80 con 0x7fca440ff740 2023-09-09T23:13:42.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.477+0000 7fca337fe700 1 -- 172.21.15.82:0/1241490386 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7fca3809a8c0 con 0x7fca440ff740 2023-09-09T23:13:42.476 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2023-09-09T23:13:42.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.479+0000 7fca48ca0700 1 -- 172.21.15.82:0/1241490386 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fca2c0a0640 msgr2=0x7fca2c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:42.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.479+0000 7fca48ca0700 1 --2- 172.21.15.82:0/1241490386 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fca2c0a0640 0x7fca2c0a2b00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fca34005950 tx=0x7fca3400a300 comp rx=0 tx=0).stop 2023-09-09T23:13:42.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.479+0000 7fca48ca0700 1 -- 172.21.15.82:0/1241490386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca440ff740 msgr2=0x7fca441962c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:42.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.479+0000 7fca48ca0700 1 --2- 172.21.15.82:0/1241490386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca440ff740 0x7fca441962c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fca380059d0 tx=0x7fca380047a0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.479+0000 7fca48ca0700 1 -- 172.21.15.82:0/1241490386 shutdown_connections 2023-09-09T23:13:42.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.479+0000 7fca48ca0700 1 --2- 172.21.15.82:0/1241490386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca44100b80 0x7fca4419d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.479+0000 7fca48ca0700 1 --2- 172.21.15.82:0/1241490386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca440ff740 0x7fca441962c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.479+0000 7fca48ca0700 1 --2- 172.21.15.82:0/1241490386 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fca2c0a0640 0x7fca2c0a2b00 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.479+0000 7fca48ca0700 1 --2- 172.21.15.82:0/1241490386 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fca44100100 0x7fca44196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.481 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.479+0000 7fca48ca0700 1 -- 172.21.15.82:0/1241490386 >> 172.21.15.82:0/1241490386 conn(0x7fca440fa1d0 msgr2=0x7fca44103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:42.481 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.480+0000 7fca48ca0700 1 -- 172.21.15.82:0/1241490386 shutdown_connections 2023-09-09T23:13:42.481 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.480+0000 7fca48ca0700 1 -- 172.21.15.82:0/1241490386 wait complete. 2023-09-09T23:13:42.490 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-09T23:13:42.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.616+0000 7f5de607d700 1 -- 172.21.15.82:0/3326085492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5de01000e0 msgr2=0x7f5de0107980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:42.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.616+0000 7f5de607d700 1 --2- 172.21.15.82:0/3326085492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5de01000e0 0x7f5de0107980 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800b0a0 tx=0x7f5dd800b3b0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.616+0000 7f5de607d700 1 -- 172.21.15.82:0/3326085492 shutdown_connections 2023-09-09T23:13:42.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.616+0000 7f5de607d700 1 --2- 172.21.15.82:0/3326085492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5de0100b60 0x7f5de0100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.616+0000 7f5de607d700 1 --2- 172.21.15.82:0/3326085492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5de01000e0 0x7f5de0107980 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.616+0000 7f5de607d700 1 --2- 172.21.15.82:0/3326085492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5de00ff720 0x7f5de00ffba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.616+0000 7f5de607d700 1 -- 172.21.15.82:0/3326085492 >> 172.21.15.82:0/3326085492 conn(0x7f5de00fa1d0 msgr2=0x7f5de00fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:42.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.616+0000 7f5de607d700 1 -- 172.21.15.82:0/3326085492 shutdown_connections 2023-09-09T23:13:42.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.616+0000 7f5de607d700 1 -- 172.21.15.82:0/3326085492 wait complete. 2023-09-09T23:13:42.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.617+0000 7f5de607d700 1 Processor -- start 2023-09-09T23:13:42.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.617+0000 7f5de607d700 1 -- start start 2023-09-09T23:13:42.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.617+0000 7f5de607d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5de00ff720 0x7f5de01951f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:42.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.617+0000 7f5de607d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5de01000e0 0x7f5de0195730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:42.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.617+0000 7f5de607d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5de0100b60 0x7f5de019c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:42.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.617+0000 7f5de607d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de019cd70 con 0x7f5de01000e0 2023-09-09T23:13:42.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.617+0000 7f5de607d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de019ceb0 con 0x7f5de0100b60 2023-09-09T23:13:42.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.618+0000 7f5de607d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de019d040 con 0x7f5de00ff720 2023-09-09T23:13:42.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.618+0000 7f5ddeffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5de01000e0 0x7f5de0195730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:42.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.618+0000 7f5ddeffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5de01000e0 0x7f5de0195730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56058/0 (socket says 172.21.15.82:56058) 2023-09-09T23:13:42.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.618+0000 7f5ddeffd700 1 -- 172.21.15.82:0/3214365944 learned_addr learned my addr 172.21.15.82:0/3214365944 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:42.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.618+0000 7f5ddffff700 1 --2- 172.21.15.82:0/3214365944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5de0100b60 0x7f5de019c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:42.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.618+0000 7f5ddf7fe700 1 --2- 172.21.15.82:0/3214365944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5de00ff720 0x7f5de01951f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:42.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.618+0000 7f5ddeffd700 1 -- 172.21.15.82:0/3214365944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5de00ff720 msgr2=0x7f5de01951f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:42.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.618+0000 7f5ddeffd700 1 --2- 172.21.15.82:0/3214365944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5de00ff720 0x7f5de01951f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.618+0000 7f5ddeffd700 1 -- 172.21.15.82:0/3214365944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5de0100b60 msgr2=0x7f5de019c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:42.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.618+0000 7f5ddeffd700 1 --2- 172.21.15.82:0/3214365944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5de0100b60 0x7f5de019c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.618+0000 7f5ddeffd700 1 -- 172.21.15.82:0/3214365944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd8009d00 con 0x7f5de01000e0 2023-09-09T23:13:42.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.618+0000 7f5ddf7fe700 1 --2- 172.21.15.82:0/3214365944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5de00ff720 0x7f5de01951f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:42.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.618+0000 7f5ddeffd700 1 --2- 172.21.15.82:0/3214365944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5de01000e0 0x7f5de0195730 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400c9b0 tx=0x7f5dd400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:42.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.618+0000 7f5ddcff9700 1 -- 172.21.15.82:0/3214365944 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4007b00 con 0x7f5de01000e0 2023-09-09T23:13:42.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.618+0000 7f5ddcff9700 1 -- 172.21.15.82:0/3214365944 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dd4007c60 con 0x7f5de01000e0 2023-09-09T23:13:42.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.618+0000 7f5ddcff9700 1 -- 172.21.15.82:0/3214365944 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4018660 con 0x7f5de01000e0 2023-09-09T23:13:42.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.618+0000 7f5de607d700 1 -- 172.21.15.82:0/3214365944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de019d320 con 0x7f5de01000e0 2023-09-09T23:13:42.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.618+0000 7f5de607d700 1 -- 172.21.15.82:0/3214365944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de019d7e0 con 0x7f5de01000e0 2023-09-09T23:13:42.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.619+0000 7f5de607d700 1 -- 172.21.15.82:0/3214365944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5de0106380 con 0x7f5de01000e0 2023-09-09T23:13:42.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.620+0000 7f5ddcff9700 1 -- 172.21.15.82:0/3214365944 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f5dd400f450 con 0x7f5de01000e0 2023-09-09T23:13:42.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.621+0000 7f5ddcff9700 1 --2- 172.21.15.82:0/3214365944 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5dc80a0640 0x7f5dc80a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:42.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.621+0000 7f5ddcff9700 1 -- 172.21.15.82:0/3214365944 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f5dd40cbd90 con 0x7f5de01000e0 2023-09-09T23:13:42.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.621+0000 7f5ddf7fe700 1 --2- 172.21.15.82:0/3214365944 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5dc80a0640 0x7f5dc80a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:42.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.621+0000 7f5ddf7fe700 1 --2- 172.21.15.82:0/3214365944 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5dc80a0640 0x7f5dc80a2b00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0005fd0 tx=0x7f5dd0005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:42.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.624+0000 7f5ddcff9700 1 -- 172.21.15.82:0/3214365944 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5dd40959f0 con 0x7f5de01000e0 2023-09-09T23:13:42.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:42 smithi170 ceph-mon[102832]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-09T23:13:42.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:42 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2249565459' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-09T23:13:42.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:42 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3510201257' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:42.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:42 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/405168340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:13:42.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:42 smithi098 ceph-mon[103124]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-09T23:13:42.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:42 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2249565459' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-09T23:13:42.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:42 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3510201257' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:42 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/405168340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:13:42.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.814+0000 7f5de607d700 1 -- 172.21.15.82:0/3214365944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5de010afa0 con 0x7f5de01000e0 2023-09-09T23:13:42.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.815+0000 7f5ddcff9700 1 -- 172.21.15.82:0/3214365944 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5dd4099010 con 0x7f5de01000e0 2023-09-09T23:13:42.814 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2023-09-09T23:13:42.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.817+0000 7f5de607d700 1 -- 172.21.15.82:0/3214365944 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5dc80a0640 msgr2=0x7f5dc80a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:42.816 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.817+0000 7f5de607d700 1 --2- 172.21.15.82:0/3214365944 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5dc80a0640 0x7f5dc80a2b00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0005fd0 tx=0x7f5dd0005e20 comp rx=0 tx=0).stop 2023-09-09T23:13:42.816 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.817+0000 7f5de607d700 1 -- 172.21.15.82:0/3214365944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5de01000e0 msgr2=0x7f5de0195730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:42.816 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.817+0000 7f5de607d700 1 --2- 172.21.15.82:0/3214365944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5de01000e0 0x7f5de0195730 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400c9b0 tx=0x7f5dd400ccc0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.816 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.817+0000 7f5de607d700 1 -- 172.21.15.82:0/3214365944 shutdown_connections 2023-09-09T23:13:42.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.817+0000 7f5de607d700 1 --2- 172.21.15.82:0/3214365944 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5dc80a0640 0x7f5dc80a2b00 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.817+0000 7f5de607d700 1 --2- 172.21.15.82:0/3214365944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5de0100b60 0x7f5de019c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.817+0000 7f5de607d700 1 --2- 172.21.15.82:0/3214365944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5de00ff720 0x7f5de01951f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.817+0000 7f5de607d700 1 --2- 172.21.15.82:0/3214365944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5de01000e0 0x7f5de0195730 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.817+0000 7f5de607d700 1 -- 172.21.15.82:0/3214365944 >> 172.21.15.82:0/3214365944 conn(0x7f5de00fa1d0 msgr2=0x7f5de00fc360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:42.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.817+0000 7f5de607d700 1 -- 172.21.15.82:0/3214365944 shutdown_connections 2023-09-09T23:13:42.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.817+0000 7f5de607d700 1 -- 172.21.15.82:0/3214365944 wait complete. 2023-09-09T23:13:42.828 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-09-09T23:13:42.828 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-09T23:13:42.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.950+0000 7f26dce88700 1 -- 172.21.15.82:0/2284308077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26d80ff4e0 msgr2=0x7f26d80ff900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:42.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.950+0000 7f26dce88700 1 --2- 172.21.15.82:0/2284308077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26d80ff4e0 0x7f26d80ff900 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f26c80099c0 tx=0x7f26c8009cd0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.951+0000 7f26dce88700 1 -- 172.21.15.82:0/2284308077 shutdown_connections 2023-09-09T23:13:42.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.951+0000 7f26dce88700 1 --2- 172.21.15.82:0/2284308077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26d8101070 0x7f26d8107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.951+0000 7f26dce88700 1 --2- 172.21.15.82:0/2284308077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26d80ff4e0 0x7f26d80ff900 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.951+0000 7f26dce88700 1 --2- 172.21.15.82:0/2284308077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26d81006d0 0x7f26d8100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.951+0000 7f26dce88700 1 -- 172.21.15.82:0/2284308077 >> 172.21.15.82:0/2284308077 conn(0x7f26d80faa30 msgr2=0x7f26d80fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:42.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.951+0000 7f26dce88700 1 -- 172.21.15.82:0/2284308077 shutdown_connections 2023-09-09T23:13:42.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.951+0000 7f26dce88700 1 -- 172.21.15.82:0/2284308077 wait complete. 2023-09-09T23:13:42.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.951+0000 7f26dce88700 1 Processor -- start 2023-09-09T23:13:42.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.951+0000 7f26dce88700 1 -- start start 2023-09-09T23:13:42.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.951+0000 7f26dce88700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26d80ff4e0 0x7f26d8194090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:42.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.951+0000 7f26dce88700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26d81006d0 0x7f26d81945d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:42.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.951+0000 7f26dce88700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26d8101070 0x7f26d819b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:42.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.951+0000 7f26dce88700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d819bc10 con 0x7f26d80ff4e0 2023-09-09T23:13:42.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.951+0000 7f26dce88700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d819bd50 con 0x7f26d8101070 2023-09-09T23:13:42.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.952+0000 7f26dce88700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d819bee0 con 0x7f26d81006d0 2023-09-09T23:13:42.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.952+0000 7f26d659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26d80ff4e0 0x7f26d8194090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:42.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.952+0000 7f26d659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26d80ff4e0 0x7f26d8194090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56070/0 (socket says 172.21.15.82:56070) 2023-09-09T23:13:42.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.952+0000 7f26d659c700 1 -- 172.21.15.82:0/2105972440 learned_addr learned my addr 172.21.15.82:0/2105972440 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:42.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.952+0000 7f26d6d9d700 1 --2- 172.21.15.82:0/2105972440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26d8101070 0x7f26d819b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:42.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.952+0000 7f26d659c700 1 -- 172.21.15.82:0/2105972440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26d81006d0 msgr2=0x7f26d81945d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:42.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.952+0000 7f26d5d9b700 1 --2- 172.21.15.82:0/2105972440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26d81006d0 0x7f26d81945d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:42.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.952+0000 7f26d659c700 1 --2- 172.21.15.82:0/2105972440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26d81006d0 0x7f26d81945d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.952+0000 7f26d659c700 1 -- 172.21.15.82:0/2105972440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26d8101070 msgr2=0x7f26d819b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:42.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.952+0000 7f26d659c700 1 --2- 172.21.15.82:0/2105972440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26d8101070 0x7f26d819b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:42.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.952+0000 7f26d659c700 1 -- 172.21.15.82:0/2105972440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c80095f0 con 0x7f26d80ff4e0 2023-09-09T23:13:42.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.952+0000 7f26d6d9d700 1 --2- 172.21.15.82:0/2105972440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26d8101070 0x7f26d819b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:42.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.952+0000 7f26d5d9b700 1 --2- 172.21.15.82:0/2105972440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26d81006d0 0x7f26d81945d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:42.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.952+0000 7f26d659c700 1 --2- 172.21.15.82:0/2105972440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26d80ff4e0 0x7f26d8194090 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009990 tx=0x7f26c8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:42.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.952+0000 7f26c77fe700 1 -- 172.21.15.82:0/2105972440 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c801d070 con 0x7f26d80ff4e0 2023-09-09T23:13:42.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.952+0000 7f26dce88700 1 -- 172.21.15.82:0/2105972440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d819c160 con 0x7f26d80ff4e0 2023-09-09T23:13:42.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.952+0000 7f26c77fe700 1 -- 172.21.15.82:0/2105972440 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26c8004200 con 0x7f26d80ff4e0 2023-09-09T23:13:42.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.952+0000 7f26c77fe700 1 -- 172.21.15.82:0/2105972440 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c8022e30 con 0x7f26d80ff4e0 2023-09-09T23:13:42.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.952+0000 7f26dce88700 1 -- 172.21.15.82:0/2105972440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d819c660 con 0x7f26d80ff4e0 2023-09-09T23:13:42.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.953+0000 7f26dce88700 1 -- 172.21.15.82:0/2105972440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26d8109c60 con 0x7f26d80ff4e0 2023-09-09T23:13:42.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.954+0000 7f26c77fe700 1 -- 172.21.15.82:0/2105972440 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f26c8003d40 con 0x7f26d80ff4e0 2023-09-09T23:13:42.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.955+0000 7f26c77fe700 1 --2- 172.21.15.82:0/2105972440 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f26c00a28a0 0x7f26c00a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:42.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.955+0000 7f26d5d9b700 1 --2- 172.21.15.82:0/2105972440 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f26c00a28a0 0x7f26c00a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:42.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.955+0000 7f26c77fe700 1 -- 172.21.15.82:0/2105972440 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f26c80cd180 con 0x7f26d80ff4e0 2023-09-09T23:13:42.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.955+0000 7f26d5d9b700 1 --2- 172.21.15.82:0/2105972440 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f26c00a28a0 0x7f26c00a4d60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f26cc005fd0 tx=0x7f26cc005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:42.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:42.958+0000 7f26c77fe700 1 -- 172.21.15.82:0/2105972440 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f26c8096de0 con 0x7f26d80ff4e0 2023-09-09T23:13:43.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.195+0000 7f26dce88700 1 -- 172.21.15.82:0/2105972440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26d8106410 con 0x7f26d80ff4e0 2023-09-09T23:13:43.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.196+0000 7f26c77fe700 1 -- 172.21.15.82:0/2105972440 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11368 (secure 0 0 0) 0x7f26c809a400 con 0x7f26d80ff4e0 2023-09-09T23:13:43.195 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:43.195 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2471432684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2471432684},{"type":"v1","addr":"172.21.15.82:6835","nonce":2471432684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3579387137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3579387137},{"type":"v1","addr":"172.21.15.82:6837","nonce":3579387137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-09T23:12:17.586036+0000","modified":"2023-09-09T23:12:25.285958+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24542,"mds_2":24523,"mds_3":24491,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14712":{"gid":14712,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/2127301345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2127301345},{"type":"v1","addr":"172.21.15.82:6839","nonce":2127301345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14724":{"gid":14724,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/4143604488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":4143604488},{"type":"v1","addr":"172.21.15.82:6841","nonce":4143604488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.170:6833/1931961940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1931961940},{"type":"v1","addr":"172.21.15.170:6833","nonce":1931961940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24491":{"gid":24491,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.98:6833/3855146755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3855146755},{"type":"v1","addr":"172.21.15.98:6833","nonce":3855146755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2313861511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2313861511},{"type":"v1","addr":"172.21.15.170:6835","nonce":2313861511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/4006360308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":4006360308},{"type":"v1","addr":"172.21.15.170:6837","nonce":4006360308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1356474828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1356474828},{"type":"v1","addr":"172.21.15.98:6835","nonce":1356474828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/3858369735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3858369735},{"type":"v1","addr":"172.21.15.170:6839","nonce":3858369735}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/2019449840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2019449840},{"type":"v1","addr":"172.21.15.98:6837","nonce":2019449840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2785086371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2785086371},{"type":"v1","addr":"172.21.15.98:6839","nonce":2785086371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-09T23:13:43.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.198+0000 7f26dce88700 1 -- 172.21.15.82:0/2105972440 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f26c00a28a0 msgr2=0x7f26c00a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:43.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.198+0000 7f26dce88700 1 --2- 172.21.15.82:0/2105972440 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f26c00a28a0 0x7f26c00a4d60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f26cc005fd0 tx=0x7f26cc005ee0 comp rx=0 tx=0).stop 2023-09-09T23:13:43.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.198+0000 7f26dce88700 1 -- 172.21.15.82:0/2105972440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26d80ff4e0 msgr2=0x7f26d8194090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:43.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.198+0000 7f26dce88700 1 --2- 172.21.15.82:0/2105972440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26d80ff4e0 0x7f26d8194090 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009990 tx=0x7f26c8005e80 comp rx=0 tx=0).stop 2023-09-09T23:13:43.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.198+0000 7f26dce88700 1 -- 172.21.15.82:0/2105972440 shutdown_connections 2023-09-09T23:13:43.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.199+0000 7f26dce88700 1 --2- 172.21.15.82:0/2105972440 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f26c00a28a0 0x7f26c00a4d60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:43.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.199+0000 7f26dce88700 1 --2- 172.21.15.82:0/2105972440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26d8101070 0x7f26d819b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:43.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.199+0000 7f26dce88700 1 --2- 172.21.15.82:0/2105972440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26d80ff4e0 0x7f26d8194090 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:43.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.199+0000 7f26dce88700 1 --2- 172.21.15.82:0/2105972440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26d81006d0 0x7f26d81945d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:43.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.199+0000 7f26dce88700 1 -- 172.21.15.82:0/2105972440 >> 172.21.15.82:0/2105972440 conn(0x7f26d80faa30 msgr2=0x7f26d80fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:43.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.199+0000 7f26dce88700 1 -- 172.21.15.82:0/2105972440 shutdown_connections 2023-09-09T23:13:43.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.199+0000 7f26dce88700 1 -- 172.21.15.82:0/2105972440 wait complete. 2023-09-09T23:13:43.200 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 27 2023-09-09T23:13:43.209 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-09-09T23:13:43.210 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-09-09T23:13:43.210 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-09T23:13:43.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.338+0000 7ff459764700 1 -- 172.21.15.82:0/3772622771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4541018c0 msgr2=0x7ff454101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:43.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.338+0000 7ff459764700 1 --2- 172.21.15.82:0/3772622771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4541018c0 0x7ff454101d20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff444009990 tx=0x7ff444009ca0 comp rx=0 tx=0).stop 2023-09-09T23:13:43.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.338+0000 7ff459764700 1 -- 172.21.15.82:0/3772622771 shutdown_connections 2023-09-09T23:13:43.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.338+0000 7ff459764700 1 --2- 172.21.15.82:0/3772622771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4540fe8f0 0x7ff4540fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:43.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.338+0000 7ff459764700 1 --2- 172.21.15.82:0/3772622771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff454102260 0x7ff454108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:43.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.338+0000 7ff459764700 1 --2- 172.21.15.82:0/3772622771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4541018c0 0x7ff454101d20 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:43.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.338+0000 7ff459764700 1 -- 172.21.15.82:0/3772622771 >> 172.21.15.82:0/3772622771 conn(0x7ff4540fa200 msgr2=0x7ff4540fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:43.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.338+0000 7ff459764700 1 -- 172.21.15.82:0/3772622771 shutdown_connections 2023-09-09T23:13:43.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.338+0000 7ff459764700 1 -- 172.21.15.82:0/3772622771 wait complete. 2023-09-09T23:13:43.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.338+0000 7ff459764700 1 Processor -- start 2023-09-09T23:13:43.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.339+0000 7ff459764700 1 -- start start 2023-09-09T23:13:43.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.339+0000 7ff459764700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4540fe8f0 0x7ff454101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:43.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.339+0000 7ff459764700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4541018c0 0x7ff4540ff870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:43.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.339+0000 7ff459764700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff454102260 0x7ff4540ffdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:43.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.339+0000 7ff459764700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff454100550 con 0x7ff454102260 2023-09-09T23:13:43.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.339+0000 7ff459764700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4541006c0 con 0x7ff4540fe8f0 2023-09-09T23:13:43.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.339+0000 7ff459764700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4541a0370 con 0x7ff4541018c0 2023-09-09T23:13:43.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.339+0000 7ff4537fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff454102260 0x7ff4540ffdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:43.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.339+0000 7ff4537fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff454102260 0x7ff4540ffdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56088/0 (socket says 172.21.15.82:56088) 2023-09-09T23:13:43.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.339+0000 7ff4537fe700 1 -- 172.21.15.82:0/2341005242 learned_addr learned my addr 172.21.15.82:0/2341005242 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:43.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.339+0000 7ff452ffd700 1 --2- 172.21.15.82:0/2341005242 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4540fe8f0 0x7ff454101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:43.344 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.339+0000 7ff4527fc700 1 --2- 172.21.15.82:0/2341005242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4541018c0 0x7ff4540ff870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:43.344 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.339+0000 7ff4537fe700 1 -- 172.21.15.82:0/2341005242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4541018c0 msgr2=0x7ff4540ff870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:43.344 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.339+0000 7ff4537fe700 1 --2- 172.21.15.82:0/2341005242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4541018c0 0x7ff4540ff870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:43.344 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.339+0000 7ff4537fe700 1 -- 172.21.15.82:0/2341005242 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4540fe8f0 msgr2=0x7ff454101230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:43.344 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.339+0000 7ff4537fe700 1 --2- 172.21.15.82:0/2341005242 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4540fe8f0 0x7ff454101230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:43.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.339+0000 7ff4537fe700 1 -- 172.21.15.82:0/2341005242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4440095f0 con 0x7ff454102260 2023-09-09T23:13:43.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.339+0000 7ff4527fc700 1 --2- 172.21.15.82:0/2341005242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4541018c0 0x7ff4540ff870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:43.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.339+0000 7ff4537fe700 1 --2- 172.21.15.82:0/2341005242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff454102260 0x7ff4540ffdb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff44c00b970 tx=0x7ff44c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:43.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.340+0000 7ff43bfff700 1 -- 172.21.15.82:0/2341005242 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44c00c930 con 0x7ff454102260 2023-09-09T23:13:43.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.340+0000 7ff43bfff700 1 -- 172.21.15.82:0/2341005242 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff44c014440 con 0x7ff454102260 2023-09-09T23:13:43.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.340+0000 7ff459764700 1 -- 172.21.15.82:0/2341005242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4541a0650 con 0x7ff454102260 2023-09-09T23:13:43.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.340+0000 7ff43bfff700 1 -- 172.21.15.82:0/2341005242 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44c0075d0 con 0x7ff454102260 2023-09-09T23:13:43.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.340+0000 7ff459764700 1 -- 172.21.15.82:0/2341005242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4541a0b60 con 0x7ff454102260 2023-09-09T23:13:43.347 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.341+0000 7ff459764700 1 -- 172.21.15.82:0/2341005242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff454107c20 con 0x7ff454102260 2023-09-09T23:13:43.347 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.342+0000 7ff43bfff700 1 -- 172.21.15.82:0/2341005242 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7ff44c0145b0 con 0x7ff454102260 2023-09-09T23:13:43.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.342+0000 7ff43bfff700 1 --2- 172.21.15.82:0/2341005242 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff43c0a0690 0x7ff43c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:43.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.342+0000 7ff452ffd700 1 --2- 172.21.15.82:0/2341005242 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff43c0a0690 0x7ff43c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:43.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.342+0000 7ff43bfff700 1 -- 172.21.15.82:0/2341005242 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff44c006e40 con 0x7ff454102260 2023-09-09T23:13:43.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.343+0000 7ff452ffd700 1 --2- 172.21.15.82:0/2341005242 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff43c0a0690 0x7ff43c0a2b50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff4540687c0 tx=0x7ff448007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:43.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.346+0000 7ff43bfff700 1 -- 172.21.15.82:0/2341005242 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff44c094f70 con 0x7ff454102260 2023-09-09T23:13:43.532 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.534+0000 7ff459764700 1 -- 172.21.15.82:0/2341005242 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff45402cf60 con 0x7ff43c0a0690 2023-09-09T23:13:43.542 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.544+0000 7ff43bfff700 1 -- 172.21.15.82:0/2341005242 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344353 (secure 0 0 0) 0x7ff45402cf60 con 0x7ff43c0a0690 2023-09-09T23:13:43.546 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:43.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.547+0000 7ff459764700 1 -- 172.21.15.82:0/2341005242 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff43c0a0690 msgr2=0x7ff43c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:43.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.547+0000 7ff459764700 1 --2- 172.21.15.82:0/2341005242 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff43c0a0690 0x7ff43c0a2b50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff4540687c0 tx=0x7ff448007400 comp rx=0 tx=0).stop 2023-09-09T23:13:43.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.547+0000 7ff459764700 1 -- 172.21.15.82:0/2341005242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff454102260 msgr2=0x7ff4540ffdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:43.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.547+0000 7ff459764700 1 --2- 172.21.15.82:0/2341005242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff454102260 0x7ff4540ffdb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff44c00b970 tx=0x7ff44c00bc80 comp rx=0 tx=0).stop 2023-09-09T23:13:43.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.547+0000 7ff459764700 1 -- 172.21.15.82:0/2341005242 shutdown_connections 2023-09-09T23:13:43.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.547+0000 7ff459764700 1 --2- 172.21.15.82:0/2341005242 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff43c0a0690 0x7ff43c0a2b50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:43.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.547+0000 7ff459764700 1 --2- 172.21.15.82:0/2341005242 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4540fe8f0 0x7ff454101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:43.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.547+0000 7ff459764700 1 --2- 172.21.15.82:0/2341005242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff454102260 0x7ff4540ffdb0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:43.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.547+0000 7ff459764700 1 --2- 172.21.15.82:0/2341005242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4541018c0 0x7ff4540ff870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:43.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.547+0000 7ff459764700 1 -- 172.21.15.82:0/2341005242 >> 172.21.15.82:0/2341005242 conn(0x7ff4540fa200 msgr2=0x7ff454104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:43.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.547+0000 7ff459764700 1 -- 172.21.15.82:0/2341005242 shutdown_connections 2023-09-09T23:13:43.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.547+0000 7ff459764700 1 -- 172.21.15.82:0/2341005242 wait complete. 2023-09-09T23:13:43.552 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-09-09T23:13:43.557 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:43 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1241490386' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-09T23:13:43.557 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:43 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3214365944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-09T23:13:43.557 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:43 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2105972440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:13:43.558 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2023-09-09T23:13:43.000017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660192,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":948,"num_read_kb":598,"num_write":421,"num_write_kb":3078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180826,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":781,"ondisk_log_size":781,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336984,"kb_used_data":12768,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124604840,"statfs":{"total":1151940427776,"available":1151595356160,"internally_reserved":0,"allocated":13074432,"data_stored":7167208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,8,0,0,1,27],"upper_bound":64},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6087,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":72,"num_write":8,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004132"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.327081+0000","last_change":"2023-09-09T23:12:17.903811+0000","last_active":"2023-09-09T23:13:09.327081+0000","last_peered":"2023-09-09T23:13:09.327081+0000","last_clean":"2023-09-09T23:13:09.327081+0000","last_became_active":"2023-09-09T23:12:17.903386+0000","last_became_peered":"2023-09-09T23:12:17.903386+0000","last_unstale":"2023-09-09T23:13:09.327081+0000","last_undegraded":"2023-09-09T23:13:09.327081+0000","last_fullsized":"2023-09-09T23:13:09.327081+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:36:45.122068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.016086+0000","last_change":"2023-09-09T23:12:17.897915+0000","last_active":"2023-09-09T23:13:39.016086+0000","last_peered":"2023-09-09T23:13:39.016086+0000","last_clean":"2023-09-09T23:13:39.016086+0000","last_became_active":"2023-09-09T23:12:17.896727+0000","last_became_peered":"2023-09-09T23:12:17.896727+0000","last_unstale":"2023-09-09T23:13:39.016086+0000","last_undegraded":"2023-09-09T23:13:39.016086+0000","last_fullsized":"2023-09-09T23:13:39.016086+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:36:03.053572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.020527+0000","last_change":"2023-09-09T23:12:17.907957+0000","last_active":"2023-09-09T23:13:39.020527+0000","last_peered":"2023-09-09T23:13:39.020527+0000","last_clean":"2023-09-09T23:13:39.020527+0000","last_became_active":"2023-09-09T23:12:17.907625+0000","last_became_peered":"2023-09-09T23:12:17.907625+0000","last_unstale":"2023-09-09T23:13:39.020527+0000","last_undegraded":"2023-09-09T23:13:39.020527+0000","last_fullsized":"2023-09-09T23:13:39.020527+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:26:56.520987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.327490+0000","last_change":"2023-09-09T23:12:17.903489+0000","last_active":"2023-09-09T23:13:09.327490+0000","last_peered":"2023-09-09T23:13:09.327490+0000","last_clean":"2023-09-09T23:13:09.327490+0000","last_became_active":"2023-09-09T23:12:17.901184+0000","last_became_peered":"2023-09-09T23:12:17.901184+0000","last_unstale":"2023-09-09T23:13:09.327490+0000","last_undegraded":"2023-09-09T23:13:09.327490+0000","last_fullsized":"2023-09-09T23:13:09.327490+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:42:01.899998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.007034+0000","last_change":"2023-09-09T23:12:17.899487+0000","last_active":"2023-09-09T23:13:39.007034+0000","last_peered":"2023-09-09T23:13:39.007034+0000","last_clean":"2023-09-09T23:13:39.007034+0000","last_became_active":"2023-09-09T23:12:17.899083+0000","last_became_peered":"2023-09-09T23:12:17.899083+0000","last_unstale":"2023-09-09T23:13:39.007034+0000","last_undegraded":"2023-09-09T23:13:39.007034+0000","last_fullsized":"2023-09-09T23:13:39.007034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:15:20.109849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.006501+0000","last_change":"2023-09-09T23:12:17.901554+0000","last_active":"2023-09-09T23:13:39.006501+0000","last_peered":"2023-09-09T23:13:39.006501+0000","last_clean":"2023-09-09T23:13:39.006501+0000","last_became_active":"2023-09-09T23:12:17.901246+0000","last_became_peered":"2023-09-09T23:12:17.901246+0000","last_unstale":"2023-09-09T23:13:39.006501+0000","last_undegraded":"2023-09-09T23:13:39.006501+0000","last_fullsized":"2023-09-09T23:13:39.006501+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:19:08.795927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.016998+0000","last_change":"2023-09-09T23:12:17.906989+0000","last_active":"2023-09-09T23:13:39.016998+0000","last_peered":"2023-09-09T23:13:39.016998+0000","last_clean":"2023-09-09T23:13:39.016998+0000","last_became_active":"2023-09-09T23:12:17.906704+0000","last_became_peered":"2023-09-09T23:12:17.906704+0000","last_unstale":"2023-09-09T23:13:39.016998+0000","last_undegraded":"2023-09-09T23:13:39.016998+0000","last_fullsized":"2023-09-09T23:13:39.016998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:41:33.612168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.015516+0000","last_change":"2023-09-09T23:12:17.906269+0000","last_active":"2023-09-09T23:13:39.015516+0000","last_peered":"2023-09-09T23:13:39.015516+0000","last_clean":"2023-09-09T23:13:39.015516+0000","last_became_active":"2023-09-09T23:12:17.906017+0000","last_became_peered":"2023-09-09T23:12:17.906017+0000","last_unstale":"2023-09-09T23:13:39.015516+0000","last_undegraded":"2023-09-09T23:13:39.015516+0000","last_fullsized":"2023-09-09T23:13:39.015516+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:50:21.893506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.020152+0000","last_change":"2023-09-09T23:12:17.905191+0000","last_active":"2023-09-09T23:13:39.020152+0000","last_peered":"2023-09-09T23:13:39.020152+0000","last_clean":"2023-09-09T23:13:39.020152+0000","last_became_active":"2023-09-09T23:12:17.904813+0000","last_became_peered":"2023-09-09T23:12:17.904813+0000","last_unstale":"2023-09-09T23:13:39.020152+0000","last_undegraded":"2023-09-09T23:13:39.020152+0000","last_fullsized":"2023-09-09T23:13:39.020152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:29:54.722210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.008289+0000","last_change":"2023-09-09T23:12:17.899980+0000","last_active":"2023-09-09T23:13:39.008289+0000","last_peered":"2023-09-09T23:13:39.008289+0000","last_clean":"2023-09-09T23:13:39.008289+0000","last_became_active":"2023-09-09T23:12:17.899692+0000","last_became_peered":"2023-09-09T23:12:17.899692+0000","last_unstale":"2023-09-09T23:13:39.008289+0000","last_undegraded":"2023-09-09T23:13:39.008289+0000","last_fullsized":"2023-09-09T23:13:39.008289+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:25:28.056684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.021313+0000","last_change":"2023-09-09T23:12:17.907010+0000","last_active":"2023-09-09T23:13:39.021313+0000","last_peered":"2023-09-09T23:13:39.021313+0000","last_clean":"2023-09-09T23:13:39.021313+0000","last_became_active":"2023-09-09T23:12:17.906565+0000","last_became_peered":"2023-09-09T23:12:17.906565+0000","last_unstale":"2023-09-09T23:13:39.021313+0000","last_undegraded":"2023-09-09T23:13:39.021313+0000","last_fullsized":"2023-09-09T23:13:39.021313+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:08:03.733608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.327384+0000","last_change":"2023-09-09T23:12:17.902889+0000","last_active":"2023-09-09T23:13:09.327384+0000","last_peered":"2023-09-09T23:13:09.327384+0000","last_clean":"2023-09-09T23:13:09.327384+0000","last_became_active":"2023-09-09T23:12:17.900747+0000","last_became_peered":"2023-09-09T23:12:17.900747+0000","last_unstale":"2023-09-09T23:13:09.327384+0000","last_undegraded":"2023-09-09T23:13:09.327384+0000","last_fullsized":"2023-09-09T23:13:09.327384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:01:29.195275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.006815+0000","last_change":"2023-09-09T23:13:10.338443+0000","last_active":"2023-09-09T23:13:39.006815+0000","last_peered":"2023-09-09T23:13:39.006815+0000","last_clean":"2023-09-09T23:13:39.006815+0000","last_became_active":"2023-09-09T23:13:10.338011+0000","last_became_peered":"2023-09-09T23:13:10.338011+0000","last_unstale":"2023-09-09T23:13:39.006815+0000","last_undegraded":"2023-09-09T23:13:39.006815+0000","last_fullsized":"2023-09-09T23:13:39.006815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:27:24.646344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.013765+0000","last_change":"2023-09-09T23:12:17.899255+0000","last_active":"2023-09-09T23:13:39.013765+0000","last_peered":"2023-09-09T23:13:39.013765+0000","last_clean":"2023-09-09T23:13:39.013765+0000","last_became_active":"2023-09-09T23:12:17.898932+0000","last_became_peered":"2023-09-09T23:12:17.898932+0000","last_unstale":"2023-09-09T23:13:39.013765+0000","last_undegraded":"2023-09-09T23:13:39.013765+0000","last_fullsized":"2023-09-09T23:13:39.013765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:10:56.864588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.015679+0000","last_change":"2023-09-09T23:12:17.901083+0000","last_active":"2023-09-09T23:13:39.015679+0000","last_peered":"2023-09-09T23:13:39.015679+0000","last_clean":"2023-09-09T23:13:39.015679+0000","last_became_active":"2023-09-09T23:12:17.900744+0000","last_became_peered":"2023-09-09T23:12:17.900744+0000","last_unstale":"2023-09-09T23:13:39.015679+0000","last_undegraded":"2023-09-09T23:13:39.015679+0000","last_fullsized":"2023-09-09T23:13:39.015679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:37:11.073786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.022442+0000","last_change":"2023-09-09T23:12:17.901962+0000","last_active":"2023-09-09T23:13:39.022442+0000","last_peered":"2023-09-09T23:13:39.022442+0000","last_clean":"2023-09-09T23:13:39.022442+0000","last_became_active":"2023-09-09T23:12:17.901638+0000","last_became_peered":"2023-09-09T23:12:17.901638+0000","last_unstale":"2023-09-09T23:13:39.022442+0000","last_undegraded":"2023-09-09T23:13:39.022442+0000","last_fullsized":"2023-09-09T23:13:39.022442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:34:52.083262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.327117+0000","last_change":"2023-09-09T23:12:17.904424+0000","last_active":"2023-09-09T23:13:09.327117+0000","last_peered":"2023-09-09T23:13:09.327117+0000","last_clean":"2023-09-09T23:13:09.327117+0000","last_became_active":"2023-09-09T23:12:17.903879+0000","last_became_peered":"2023-09-09T23:12:17.903879+0000","last_unstale":"2023-09-09T23:13:09.327117+0000","last_undegraded":"2023-09-09T23:13:09.327117+0000","last_fullsized":"2023-09-09T23:13:09.327117+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:56:42.468630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.020503+0000","last_change":"2023-09-09T23:12:17.905065+0000","last_active":"2023-09-09T23:13:39.020503+0000","last_peered":"2023-09-09T23:13:39.020503+0000","last_clean":"2023-09-09T23:13:39.020503+0000","last_became_active":"2023-09-09T23:12:17.904646+0000","last_became_peered":"2023-09-09T23:12:17.904646+0000","last_unstale":"2023-09-09T23:13:39.020503+0000","last_undegraded":"2023-09-09T23:13:39.020503+0000","last_fullsized":"2023-09-09T23:13:39.020503+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:11:14.705884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.007659+0000","last_change":"2023-09-09T23:12:17.899751+0000","last_active":"2023-09-09T23:13:39.007659+0000","last_peered":"2023-09-09T23:13:39.007659+0000","last_clean":"2023-09-09T23:13:39.007659+0000","last_became_active":"2023-09-09T23:12:17.899410+0000","last_became_peered":"2023-09-09T23:12:17.899410+0000","last_unstale":"2023-09-09T23:13:39.007659+0000","last_undegraded":"2023-09-09T23:13:39.007659+0000","last_fullsized":"2023-09-09T23:13:39.007659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:23:06.098276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":105,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.421559+0000","last_change":"2023-09-09T23:12:17.175671+0000","last_active":"2023-09-09T23:13:39.421559+0000","last_peered":"2023-09-09T23:13:39.421559+0000","last_clean":"2023-09-09T23:13:39.421559+0000","last_became_active":"2023-09-09T23:12:17.175327+0000","last_became_peered":"2023-09-09T23:12:17.175327+0000","last_unstale":"2023-09-09T23:13:39.421559+0000","last_undegraded":"2023-09-09T23:13:39.421559+0000","last_fullsized":"2023-09-09T23:13:39.421559+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:38:57.724260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.008565+0000","last_change":"2023-09-09T23:12:17.906338+0000","last_active":"2023-09-09T23:13:39.008565+0000","last_peered":"2023-09-09T23:13:39.008565+0000","last_clean":"2023-09-09T23:13:39.008565+0000","last_became_active":"2023-09-09T23:12:17.906094+0000","last_became_peered":"2023-09-09T23:12:17.906094+0000","last_unstale":"2023-09-09T23:13:39.008565+0000","last_undegraded":"2023-09-09T23:13:39.008565+0000","last_fullsized":"2023-09-09T23:13:39.008565+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:43:18.183144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.329391+0000","last_change":"2023-09-09T23:12:17.167763+0000","last_active":"2023-09-09T23:13:09.329391+0000","last_peered":"2023-09-09T23:13:09.329391+0000","last_clean":"2023-09-09T23:13:09.329391+0000","last_became_active":"2023-09-09T23:12:17.167455+0000","last_became_peered":"2023-09-09T23:12:17.167455+0000","last_unstale":"2023-09-09T23:13:09.329391+0000","last_undegraded":"2023-09-09T23:13:09.329391+0000","last_fullsized":"2023-09-09T23:13:09.329391+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:15:23.415792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.020222+0000","last_change":"2023-09-09T23:12:17.907936+0000","last_active":"2023-09-09T23:13:39.020222+0000","last_peered":"2023-09-09T23:13:39.020222+0000","last_clean":"2023-09-09T23:13:39.020222+0000","last_became_active":"2023-09-09T23:12:17.907588+0000","last_became_peered":"2023-09-09T23:12:17.907588+0000","last_unstale":"2023-09-09T23:13:39.020222+0000","last_undegraded":"2023-09-09T23:13:39.020222+0000","last_fullsized":"2023-09-09T23:13:39.020222+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:57:30.910812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.329807+0000","last_change":"2023-09-09T23:12:17.172481+0000","last_active":"2023-09-09T23:13:09.329807+0000","last_peered":"2023-09-09T23:13:09.329807+0000","last_clean":"2023-09-09T23:13:09.329807+0000","last_became_active":"2023-09-09T23:12:17.172178+0000","last_became_peered":"2023-09-09T23:12:17.172178+0000","last_unstale":"2023-09-09T23:13:09.329807+0000","last_undegraded":"2023-09-09T23:13:09.329807+0000","last_fullsized":"2023-09-09T23:13:09.329807+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:08:55.328423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.021598+0000","last_change":"2023-09-09T23:12:17.900155+0000","last_active":"2023-09-09T23:13:39.021598+0000","last_peered":"2023-09-09T23:13:39.021598+0000","last_clean":"2023-09-09T23:13:39.021598+0000","last_became_active":"2023-09-09T23:12:17.899862+0000","last_became_peered":"2023-09-09T23:12:17.899862+0000","last_unstale":"2023-09-09T23:13:39.021598+0000","last_undegraded":"2023-09-09T23:13:39.021598+0000","last_fullsized":"2023-09-09T23:13:39.021598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:57:14.949174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.329618+0000","last_change":"2023-09-09T23:12:17.171890+0000","last_active":"2023-09-09T23:13:09.329618+0000","last_peered":"2023-09-09T23:13:09.329618+0000","last_clean":"2023-09-09T23:13:09.329618+0000","last_became_active":"2023-09-09T23:12:17.171496+0000","last_became_peered":"2023-09-09T23:12:17.171496+0000","last_unstale":"2023-09-09T23:13:09.329618+0000","last_undegraded":"2023-09-09T23:13:09.329618+0000","last_fullsized":"2023-09-09T23:13:09.329618+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:38:25.704989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.020476+0000","last_change":"2023-09-09T23:12:17.901067+0000","last_active":"2023-09-09T23:13:39.020476+0000","last_peered":"2023-09-09T23:13:39.020476+0000","last_clean":"2023-09-09T23:13:39.020476+0000","last_became_active":"2023-09-09T23:12:17.900069+0000","last_became_peered":"2023-09-09T23:12:17.900069+0000","last_unstale":"2023-09-09T23:13:39.020476+0000","last_undegraded":"2023-09-09T23:13:39.020476+0000","last_fullsized":"2023-09-09T23:13:39.020476+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:59:55.356235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'7","reported_seq":101,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:38.379346+0000","last_change":"2023-09-09T23:12:17.169474+0000","last_active":"2023-09-09T23:13:38.379346+0000","last_peered":"2023-09-09T23:13:38.379346+0000","last_clean":"2023-09-09T23:13:38.379346+0000","last_became_active":"2023-09-09T23:12:17.169186+0000","last_became_peered":"2023-09-09T23:12:17.169186+0000","last_unstale":"2023-09-09T23:13:38.379346+0000","last_undegraded":"2023-09-09T23:13:38.379346+0000","last_fullsized":"2023-09-09T23:13:38.379346+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:47:13.027949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.014143+0000","last_change":"2023-09-09T23:12:17.901605+0000","last_active":"2023-09-09T23:13:39.014143+0000","last_peered":"2023-09-09T23:13:39.014143+0000","last_clean":"2023-09-09T23:13:39.014143+0000","last_became_active":"2023-09-09T23:12:17.901287+0000","last_became_peered":"2023-09-09T23:12:17.901287+0000","last_unstale":"2023-09-09T23:13:39.014143+0000","last_undegraded":"2023-09-09T23:13:39.014143+0000","last_fullsized":"2023-09-09T23:13:39.014143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:16:41.570750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.333973+0000","last_change":"2023-09-09T23:12:17.171797+0000","last_active":"2023-09-09T23:13:09.333973+0000","last_peered":"2023-09-09T23:13:09.333973+0000","last_clean":"2023-09-09T23:13:09.333973+0000","last_became_active":"2023-09-09T23:12:17.171521+0000","last_became_peered":"2023-09-09T23:12:17.171521+0000","last_unstale":"2023-09-09T23:13:09.333973+0000","last_undegraded":"2023-09-09T23:13:09.333973+0000","last_fullsized":"2023-09-09T23:13:09.333973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:07:51.951393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.009071+0000","last_change":"2023-09-09T23:12:17.900284+0000","last_active":"2023-09-09T23:13:39.009071+0000","last_peered":"2023-09-09T23:13:39.009071+0000","last_clean":"2023-09-09T23:13:39.009071+0000","last_became_active":"2023-09-09T23:12:17.899975+0000","last_became_peered":"2023-09-09T23:12:17.899975+0000","last_unstale":"2023-09-09T23:13:39.009071+0000","last_undegraded":"2023-09-09T23:13:39.009071+0000","last_fullsized":"2023-09-09T23:13:39.009071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:10:56.723338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.333263+0000","last_change":"2023-09-09T23:12:17.175328+0000","last_active":"2023-09-09T23:13:09.333263+0000","last_peered":"2023-09-09T23:13:09.333263+0000","last_clean":"2023-09-09T23:13:09.333263+0000","last_became_active":"2023-09-09T23:12:17.174977+0000","last_became_peered":"2023-09-09T23:12:17.174977+0000","last_unstale":"2023-09-09T23:13:09.333263+0000","last_undegraded":"2023-09-09T23:13:09.333263+0000","last_fullsized":"2023-09-09T23:13:09.333263+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:01:36.272333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.016297+0000","last_change":"2023-09-09T23:12:17.902982+0000","last_active":"2023-09-09T23:13:39.016297+0000","last_peered":"2023-09-09T23:13:39.016297+0000","last_clean":"2023-09-09T23:13:39.016297+0000","last_became_active":"2023-09-09T23:12:17.902677+0000","last_became_peered":"2023-09-09T23:12:17.902677+0000","last_unstale":"2023-09-09T23:13:39.016297+0000","last_undegraded":"2023-09-09T23:13:39.016297+0000","last_fullsized":"2023-09-09T23:13:39.016297+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:08:31.507899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.330756+0000","last_change":"2023-09-09T23:12:17.170099+0000","last_active":"2023-09-09T23:13:09.330756+0000","last_peered":"2023-09-09T23:13:09.330756+0000","last_clean":"2023-09-09T23:13:09.330756+0000","last_became_active":"2023-09-09T23:12:17.169827+0000","last_became_peered":"2023-09-09T23:12:17.169827+0000","last_unstale":"2023-09-09T23:13:09.330756+0000","last_undegraded":"2023-09-09T23:13:09.330756+0000","last_fullsized":"2023-09-09T23:13:09.330756+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:16:34.693568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.007732+0000","last_change":"2023-09-09T23:12:17.903735+0000","last_active":"2023-09-09T23:13:39.007732+0000","last_peered":"2023-09-09T23:13:39.007732+0000","last_clean":"2023-09-09T23:13:39.007732+0000","last_became_active":"2023-09-09T23:12:17.903410+0000","last_became_peered":"2023-09-09T23:12:17.903410+0000","last_unstale":"2023-09-09T23:13:39.007732+0000","last_undegraded":"2023-09-09T23:13:39.007732+0000","last_fullsized":"2023-09-09T23:13:39.007732+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:00:47.664613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'5","reported_seq":176,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:41.384523+0000","last_change":"2023-09-09T23:12:17.176004+0000","last_active":"2023-09-09T23:13:41.384523+0000","last_peered":"2023-09-09T23:13:41.384523+0000","last_clean":"2023-09-09T23:13:41.384523+0000","last_became_active":"2023-09-09T23:12:17.175514+0000","last_became_peered":"2023-09-09T23:12:17.175514+0000","last_unstale":"2023-09-09T23:13:41.384523+0000","last_undegraded":"2023-09-09T23:13:41.384523+0000","last_fullsized":"2023-09-09T23:13:41.384523+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:43:54.587724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":156,"num_read_kb":156,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.021253+0000","last_change":"2023-09-09T23:12:17.900943+0000","last_active":"2023-09-09T23:13:39.021253+0000","last_peered":"2023-09-09T23:13:39.021253+0000","last_clean":"2023-09-09T23:13:39.021253+0000","last_became_active":"2023-09-09T23:12:17.900662+0000","last_became_peered":"2023-09-09T23:12:17.900662+0000","last_unstale":"2023-09-09T23:13:39.021253+0000","last_undegraded":"2023-09-09T23:13:39.021253+0000","last_fullsized":"2023-09-09T23:13:39.021253+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:39:41.480033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.366461+0000","last_change":"2023-09-09T23:12:17.172458+0000","last_active":"2023-09-09T23:13:09.366461+0000","last_peered":"2023-09-09T23:13:09.366461+0000","last_clean":"2023-09-09T23:13:09.366461+0000","last_became_active":"2023-09-09T23:12:17.172119+0000","last_became_peered":"2023-09-09T23:12:17.172119+0000","last_unstale":"2023-09-09T23:13:09.366461+0000","last_undegraded":"2023-09-09T23:13:09.366461+0000","last_fullsized":"2023-09-09T23:13:09.366461+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:31:27.821245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.008347+0000","last_change":"2023-09-09T23:12:17.903252+0000","last_active":"2023-09-09T23:13:39.008347+0000","last_peered":"2023-09-09T23:13:39.008347+0000","last_clean":"2023-09-09T23:13:39.008347+0000","last_became_active":"2023-09-09T23:12:17.902466+0000","last_became_peered":"2023-09-09T23:12:17.902466+0000","last_unstale":"2023-09-09T23:13:39.008347+0000","last_undegraded":"2023-09-09T23:13:39.008347+0000","last_fullsized":"2023-09-09T23:13:39.008347+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:28:07.535937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.331894+0000","last_change":"2023-09-09T23:12:17.165106+0000","last_active":"2023-09-09T23:13:09.331894+0000","last_peered":"2023-09-09T23:13:09.331894+0000","last_clean":"2023-09-09T23:13:09.331894+0000","last_became_active":"2023-09-09T23:12:17.164783+0000","last_became_peered":"2023-09-09T23:12:17.164783+0000","last_unstale":"2023-09-09T23:13:09.331894+0000","last_undegraded":"2023-09-09T23:13:09.331894+0000","last_fullsized":"2023-09-09T23:13:09.331894+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:02:14.984870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.021467+0000","last_change":"2023-09-09T23:12:17.899473+0000","last_active":"2023-09-09T23:13:39.021467+0000","last_peered":"2023-09-09T23:13:39.021467+0000","last_clean":"2023-09-09T23:13:39.021467+0000","last_became_active":"2023-09-09T23:12:17.899142+0000","last_became_peered":"2023-09-09T23:12:17.899142+0000","last_unstale":"2023-09-09T23:13:39.021467+0000","last_undegraded":"2023-09-09T23:13:39.021467+0000","last_fullsized":"2023-09-09T23:13:39.021467+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:50:30.261873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'34","reported_seq":217,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:41.384767+0000","last_change":"2023-09-09T23:12:17.172560+0000","last_active":"2023-09-09T23:13:41.384767+0000","last_peered":"2023-09-09T23:13:41.384767+0000","last_clean":"2023-09-09T23:13:41.384767+0000","last_became_active":"2023-09-09T23:12:17.172213+0000","last_became_peered":"2023-09-09T23:12:17.172213+0000","last_unstale":"2023-09-09T23:13:41.384767+0000","last_undegraded":"2023-09-09T23:13:41.384767+0000","last_fullsized":"2023-09-09T23:13:41.384767+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:30:00.206901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62235,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":168,"num_read_kb":139,"num_write":36,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.017552+0000","last_change":"2023-09-09T23:12:17.898279+0000","last_active":"2023-09-09T23:13:39.017552+0000","last_peered":"2023-09-09T23:13:39.017552+0000","last_clean":"2023-09-09T23:13:39.017552+0000","last_became_active":"2023-09-09T23:12:17.898041+0000","last_became_peered":"2023-09-09T23:12:17.898041+0000","last_unstale":"2023-09-09T23:13:39.017552+0000","last_undegraded":"2023-09-09T23:13:39.017552+0000","last_fullsized":"2023-09-09T23:13:39.017552+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:01:10.315445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:10.350264+0000","last_change":"2023-09-09T23:13:10.349904+0000","last_active":"2023-09-09T23:13:10.350264+0000","last_peered":"2023-09-09T23:13:10.350264+0000","last_clean":"2023-09-09T23:13:10.350264+0000","last_became_active":"2023-09-09T23:13:10.337554+0000","last_became_peered":"2023-09-09T23:13:10.337554+0000","last_unstale":"2023-09-09T23:13:10.350264+0000","last_undegraded":"2023-09-09T23:13:10.350264+0000","last_fullsized":"2023-09-09T23:13:10.350264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:16:39.275038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.012288+0000","last_change":"2023-09-09T23:12:17.904087+0000","last_active":"2023-09-09T23:13:39.012288+0000","last_peered":"2023-09-09T23:13:39.012288+0000","last_clean":"2023-09-09T23:13:39.012288+0000","last_became_active":"2023-09-09T23:12:17.903670+0000","last_became_peered":"2023-09-09T23:12:17.903670+0000","last_unstale":"2023-09-09T23:13:39.012288+0000","last_undegraded":"2023-09-09T23:13:39.012288+0000","last_fullsized":"2023-09-09T23:13:39.012288+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:24:04.403871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.331421+0000","last_change":"2023-09-09T23:12:17.170594+0000","last_active":"2023-09-09T23:13:09.331421+0000","last_peered":"2023-09-09T23:13:09.331421+0000","last_clean":"2023-09-09T23:13:09.331421+0000","last_became_active":"2023-09-09T23:12:17.170223+0000","last_became_peered":"2023-09-09T23:12:17.170223+0000","last_unstale":"2023-09-09T23:13:09.331421+0000","last_undegraded":"2023-09-09T23:13:09.331421+0000","last_fullsized":"2023-09-09T23:13:09.331421+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:30:09.222947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.019450+0000","last_change":"2023-09-09T23:12:17.901047+0000","last_active":"2023-09-09T23:13:39.019450+0000","last_peered":"2023-09-09T23:13:39.019450+0000","last_clean":"2023-09-09T23:13:39.019450+0000","last_became_active":"2023-09-09T23:12:17.900129+0000","last_became_peered":"2023-09-09T23:12:17.900129+0000","last_unstale":"2023-09-09T23:13:39.019450+0000","last_undegraded":"2023-09-09T23:13:39.019450+0000","last_fullsized":"2023-09-09T23:13:39.019450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:59:36.338392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'6","reported_seq":100,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:37.378029+0000","last_change":"2023-09-09T23:12:17.169495+0000","last_active":"2023-09-09T23:13:37.378029+0000","last_peered":"2023-09-09T23:13:37.378029+0000","last_clean":"2023-09-09T23:13:37.378029+0000","last_became_active":"2023-09-09T23:12:17.169170+0000","last_became_peered":"2023-09-09T23:12:17.169170+0000","last_unstale":"2023-09-09T23:13:37.378029+0000","last_undegraded":"2023-09-09T23:13:37.378029+0000","last_fullsized":"2023-09-09T23:13:37.378029+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:36:23.765574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.018670+0000","last_change":"2023-09-09T23:12:17.907826+0000","last_active":"2023-09-09T23:13:39.018670+0000","last_peered":"2023-09-09T23:13:39.018670+0000","last_clean":"2023-09-09T23:13:39.018670+0000","last_became_active":"2023-09-09T23:12:17.907526+0000","last_became_peered":"2023-09-09T23:12:17.907526+0000","last_unstale":"2023-09-09T23:13:39.018670+0000","last_undegraded":"2023-09-09T23:13:39.018670+0000","last_fullsized":"2023-09-09T23:13:39.018670+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:17:20.587253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.328902+0000","last_change":"2023-09-09T23:12:17.170139+0000","last_active":"2023-09-09T23:13:09.328902+0000","last_peered":"2023-09-09T23:13:09.328902+0000","last_clean":"2023-09-09T23:13:09.328902+0000","last_became_active":"2023-09-09T23:12:17.169797+0000","last_became_peered":"2023-09-09T23:12:17.169797+0000","last_unstale":"2023-09-09T23:13:09.328902+0000","last_undegraded":"2023-09-09T23:13:09.328902+0000","last_fullsized":"2023-09-09T23:13:09.328902+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:59:44.668114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.021950+0000","last_change":"2023-09-09T23:12:17.902567+0000","last_active":"2023-09-09T23:13:39.021950+0000","last_peered":"2023-09-09T23:13:39.021950+0000","last_clean":"2023-09-09T23:13:39.021950+0000","last_became_active":"2023-09-09T23:12:17.902229+0000","last_became_peered":"2023-09-09T23:12:17.902229+0000","last_unstale":"2023-09-09T23:13:39.021950+0000","last_undegraded":"2023-09-09T23:13:39.021950+0000","last_fullsized":"2023-09-09T23:13:39.021950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:49:50.049925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:10.341547+0000","last_change":"2023-09-09T23:13:10.341173+0000","last_active":"2023-09-09T23:13:10.341547+0000","last_peered":"2023-09-09T23:13:10.341547+0000","last_clean":"2023-09-09T23:13:10.341547+0000","last_became_active":"2023-09-09T23:13:10.336085+0000","last_became_peered":"2023-09-09T23:13:10.336085+0000","last_unstale":"2023-09-09T23:13:10.341547+0000","last_undegraded":"2023-09-09T23:13:10.341547+0000","last_fullsized":"2023-09-09T23:13:10.341547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:14:47.365794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.326890+0000","last_change":"2023-09-09T23:12:17.900284+0000","last_active":"2023-09-09T23:13:09.326890+0000","last_peered":"2023-09-09T23:13:09.326890+0000","last_clean":"2023-09-09T23:13:09.326890+0000","last_became_active":"2023-09-09T23:12:17.899923+0000","last_became_peered":"2023-09-09T23:12:17.899923+0000","last_unstale":"2023-09-09T23:13:09.326890+0000","last_undegraded":"2023-09-09T23:13:09.326890+0000","last_fullsized":"2023-09-09T23:13:09.326890+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:32:32.230968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.331153+0000","last_change":"2023-09-09T23:12:17.176313+0000","last_active":"2023-09-09T23:13:09.331153+0000","last_peered":"2023-09-09T23:13:09.331153+0000","last_clean":"2023-09-09T23:13:09.331153+0000","last_became_active":"2023-09-09T23:12:17.175934+0000","last_became_peered":"2023-09-09T23:12:17.175934+0000","last_unstale":"2023-09-09T23:13:09.331153+0000","last_undegraded":"2023-09-09T23:13:09.331153+0000","last_fullsized":"2023-09-09T23:13:09.331153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:33:15.044986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.006607+0000","last_change":"2023-09-09T23:12:17.900284+0000","last_active":"2023-09-09T23:13:39.006607+0000","last_peered":"2023-09-09T23:13:39.006607+0000","last_clean":"2023-09-09T23:13:39.006607+0000","last_became_active":"2023-09-09T23:12:17.899970+0000","last_became_peered":"2023-09-09T23:12:17.899970+0000","last_unstale":"2023-09-09T23:13:39.006607+0000","last_undegraded":"2023-09-09T23:13:39.006607+0000","last_fullsized":"2023-09-09T23:13:39.006607+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:01:37.415195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":96,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:38.405308+0000","last_change":"2023-09-09T23:12:17.170123+0000","last_active":"2023-09-09T23:13:38.405308+0000","last_peered":"2023-09-09T23:13:38.405308+0000","last_clean":"2023-09-09T23:13:38.405308+0000","last_became_active":"2023-09-09T23:12:17.169775+0000","last_became_peered":"2023-09-09T23:12:17.169775+0000","last_unstale":"2023-09-09T23:13:38.405308+0000","last_undegraded":"2023-09-09T23:13:38.405308+0000","last_fullsized":"2023-09-09T23:13:38.405308+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:22:44.520489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.011772+0000","last_change":"2023-09-09T23:12:17.894881+0000","last_active":"2023-09-09T23:13:39.011772+0000","last_peered":"2023-09-09T23:13:39.011772+0000","last_clean":"2023-09-09T23:13:39.011772+0000","last_became_active":"2023-09-09T23:12:17.894567+0000","last_became_peered":"2023-09-09T23:12:17.894567+0000","last_unstale":"2023-09-09T23:13:39.011772+0000","last_undegraded":"2023-09-09T23:13:39.011772+0000","last_fullsized":"2023-09-09T23:13:39.011772+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:40:37.098780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.015183+0000","last_change":"2023-09-09T23:12:17.903319+0000","last_active":"2023-09-09T23:13:39.015183+0000","last_peered":"2023-09-09T23:13:39.015183+0000","last_clean":"2023-09-09T23:13:39.015183+0000","last_became_active":"2023-09-09T23:12:17.902642+0000","last_became_peered":"2023-09-09T23:12:17.902642+0000","last_unstale":"2023-09-09T23:13:39.015183+0000","last_undegraded":"2023-09-09T23:13:39.015183+0000","last_fullsized":"2023-09-09T23:13:39.015183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:03:29.562813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":99,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:38.975476+0000","last_change":"2023-09-09T23:12:17.172639+0000","last_active":"2023-09-09T23:13:38.975476+0000","last_peered":"2023-09-09T23:13:38.975476+0000","last_clean":"2023-09-09T23:13:38.975476+0000","last_became_active":"2023-09-09T23:12:17.172309+0000","last_became_peered":"2023-09-09T23:12:17.172309+0000","last_unstale":"2023-09-09T23:13:38.975476+0000","last_undegraded":"2023-09-09T23:13:38.975476+0000","last_fullsized":"2023-09-09T23:13:38.975476+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:05:56.160984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.019865+0000","last_change":"2023-09-09T23:13:10.336403+0000","last_active":"2023-09-09T23:13:39.019865+0000","last_peered":"2023-09-09T23:13:39.019865+0000","last_clean":"2023-09-09T23:13:39.019865+0000","last_became_active":"2023-09-09T23:13:10.335730+0000","last_became_peered":"2023-09-09T23:13:10.335730+0000","last_unstale":"2023-09-09T23:13:39.019865+0000","last_undegraded":"2023-09-09T23:13:39.019865+0000","last_fullsized":"2023-09-09T23:13:39.019865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:35:00.228469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.327321+0000","last_change":"2023-09-09T23:12:17.166623+0000","last_active":"2023-09-09T23:13:09.327321+0000","last_peered":"2023-09-09T23:13:09.327321+0000","last_clean":"2023-09-09T23:13:09.327321+0000","last_became_active":"2023-09-09T23:12:17.166293+0000","last_became_peered":"2023-09-09T23:12:17.166293+0000","last_unstale":"2023-09-09T23:13:09.327321+0000","last_undegraded":"2023-09-09T23:13:09.327321+0000","last_fullsized":"2023-09-09T23:13:09.327321+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:52:51.267501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":319,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.366308+0000","last_change":"2023-09-09T23:11:20.948461+0000","last_active":"2023-09-09T23:13:09.366308+0000","last_peered":"2023-09-09T23:13:09.366308+0000","last_clean":"2023-09-09T23:13:09.366308+0000","last_became_active":"2023-09-09T23:11:20.947856+0000","last_became_peered":"2023-09-09T23:11:20.947856+0000","last_unstale":"2023-09-09T23:13:09.366308+0000","last_undegraded":"2023-09-09T23:13:09.366308+0000","last_fullsized":"2023-09-09T23:13:09.366308+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:08:31.737051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:08:31.737051+0000","last_clean_scrub_stamp":"2023-09-09T23:08:31.737051+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:20:16.293707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.013785+0000","last_change":"2023-09-09T23:12:17.907468+0000","last_active":"2023-09-09T23:13:39.013785+0000","last_peered":"2023-09-09T23:13:39.013785+0000","last_clean":"2023-09-09T23:13:39.013785+0000","last_became_active":"2023-09-09T23:12:17.907117+0000","last_became_peered":"2023-09-09T23:12:17.907117+0000","last_unstale":"2023-09-09T23:13:39.013785+0000","last_undegraded":"2023-09-09T23:13:39.013785+0000","last_fullsized":"2023-09-09T23:13:39.013785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:27:16.623116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.366405+0000","last_change":"2023-09-09T23:12:17.171505+0000","last_active":"2023-09-09T23:13:09.366405+0000","last_peered":"2023-09-09T23:13:09.366405+0000","last_clean":"2023-09-09T23:13:09.366405+0000","last_became_active":"2023-09-09T23:12:17.171185+0000","last_became_peered":"2023-09-09T23:12:17.171185+0000","last_unstale":"2023-09-09T23:13:09.366405+0000","last_undegraded":"2023-09-09T23:13:09.366405+0000","last_fullsized":"2023-09-09T23:13:09.366405+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:07:54.271364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.329554+0000","last_change":"2023-09-09T23:12:17.176450+0000","last_active":"2023-09-09T23:13:09.329554+0000","last_peered":"2023-09-09T23:13:09.329554+0000","last_clean":"2023-09-09T23:13:09.329554+0000","last_became_active":"2023-09-09T23:12:17.176217+0000","last_became_peered":"2023-09-09T23:12:17.176217+0000","last_unstale":"2023-09-09T23:13:09.329554+0000","last_undegraded":"2023-09-09T23:13:09.329554+0000","last_fullsized":"2023-09-09T23:13:09.329554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:57:26.604261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.008179+0000","last_change":"2023-09-09T23:12:17.897612+0000","last_active":"2023-09-09T23:13:39.008179+0000","last_peered":"2023-09-09T23:13:39.008179+0000","last_clean":"2023-09-09T23:13:39.008179+0000","last_became_active":"2023-09-09T23:12:17.897292+0000","last_became_peered":"2023-09-09T23:12:17.897292+0000","last_unstale":"2023-09-09T23:13:39.008179+0000","last_undegraded":"2023-09-09T23:13:39.008179+0000","last_fullsized":"2023-09-09T23:13:39.008179+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:25:46.374556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'9","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.331260+0000","last_change":"2023-09-09T23:12:17.175682+0000","last_active":"2023-09-09T23:13:09.331260+0000","last_peered":"2023-09-09T23:13:09.331260+0000","last_clean":"2023-09-09T23:13:09.331260+0000","last_became_active":"2023-09-09T23:12:17.175291+0000","last_became_peered":"2023-09-09T23:12:17.175291+0000","last_unstale":"2023-09-09T23:13:09.331260+0000","last_undegraded":"2023-09-09T23:13:09.331260+0000","last_fullsized":"2023-09-09T23:13:09.331260+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:29:02.349319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.016472+0000","last_change":"2023-09-09T23:12:17.904153+0000","last_active":"2023-09-09T23:13:39.016472+0000","last_peered":"2023-09-09T23:13:39.016472+0000","last_clean":"2023-09-09T23:13:39.016472+0000","last_became_active":"2023-09-09T23:12:17.903804+0000","last_became_peered":"2023-09-09T23:12:17.903804+0000","last_unstale":"2023-09-09T23:13:39.016472+0000","last_undegraded":"2023-09-09T23:13:39.016472+0000","last_fullsized":"2023-09-09T23:13:39.016472+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:39:34.046551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.327392+0000","last_change":"2023-09-09T23:12:17.169052+0000","last_active":"2023-09-09T23:13:09.327392+0000","last_peered":"2023-09-09T23:13:09.327392+0000","last_clean":"2023-09-09T23:13:09.327392+0000","last_became_active":"2023-09-09T23:12:17.168719+0000","last_became_peered":"2023-09-09T23:12:17.168719+0000","last_unstale":"2023-09-09T23:13:09.327392+0000","last_undegraded":"2023-09-09T23:13:09.327392+0000","last_fullsized":"2023-09-09T23:13:09.327392+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:56:44.427954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.011144+0000","last_change":"2023-09-09T23:12:17.900945+0000","last_active":"2023-09-09T23:13:39.011144+0000","last_peered":"2023-09-09T23:13:39.011144+0000","last_clean":"2023-09-09T23:13:39.011144+0000","last_became_active":"2023-09-09T23:12:17.900602+0000","last_became_peered":"2023-09-09T23:12:17.900602+0000","last_unstale":"2023-09-09T23:13:39.011144+0000","last_undegraded":"2023-09-09T23:13:39.011144+0000","last_fullsized":"2023-09-09T23:13:39.011144+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:22:10.336285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.331603+0000","last_change":"2023-09-09T23:12:17.166619+0000","last_active":"2023-09-09T23:13:09.331603+0000","last_peered":"2023-09-09T23:13:09.331603+0000","last_clean":"2023-09-09T23:13:09.331603+0000","last_became_active":"2023-09-09T23:12:17.166286+0000","last_became_peered":"2023-09-09T23:12:17.166286+0000","last_unstale":"2023-09-09T23:13:09.331603+0000","last_undegraded":"2023-09-09T23:13:09.331603+0000","last_fullsized":"2023-09-09T23:13:09.331603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:41:40.713296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:38.983854+0000","last_change":"2023-09-09T23:12:17.897131+0000","last_active":"2023-09-09T23:13:38.983854+0000","last_peered":"2023-09-09T23:13:38.983854+0000","last_clean":"2023-09-09T23:13:38.983854+0000","last_became_active":"2023-09-09T23:12:17.896873+0000","last_became_peered":"2023-09-09T23:12:17.896873+0000","last_unstale":"2023-09-09T23:13:38.983854+0000","last_undegraded":"2023-09-09T23:13:38.983854+0000","last_fullsized":"2023-09-09T23:13:38.983854+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:54:21.091418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.328807+0000","last_change":"2023-09-09T23:12:17.168868+0000","last_active":"2023-09-09T23:13:09.328807+0000","last_peered":"2023-09-09T23:13:09.328807+0000","last_clean":"2023-09-09T23:13:09.328807+0000","last_became_active":"2023-09-09T23:12:17.168572+0000","last_became_peered":"2023-09-09T23:12:17.168572+0000","last_unstale":"2023-09-09T23:13:09.328807+0000","last_undegraded":"2023-09-09T23:13:09.328807+0000","last_fullsized":"2023-09-09T23:13:09.328807+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:54:08.108023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.326672+0000","last_change":"2023-09-09T23:12:17.902887+0000","last_active":"2023-09-09T23:13:09.326672+0000","last_peered":"2023-09-09T23:13:09.326672+0000","last_clean":"2023-09-09T23:13:09.326672+0000","last_became_active":"2023-09-09T23:12:17.902507+0000","last_became_peered":"2023-09-09T23:12:17.902507+0000","last_unstale":"2023-09-09T23:13:09.326672+0000","last_undegraded":"2023-09-09T23:13:09.326672+0000","last_fullsized":"2023-09-09T23:13:09.326672+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:18:08.000184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.333618+0000","last_change":"2023-09-09T23:12:17.171379+0000","last_active":"2023-09-09T23:13:09.333618+0000","last_peered":"2023-09-09T23:13:09.333618+0000","last_clean":"2023-09-09T23:13:09.333618+0000","last_became_active":"2023-09-09T23:12:17.171024+0000","last_became_peered":"2023-09-09T23:12:17.171024+0000","last_unstale":"2023-09-09T23:13:09.333618+0000","last_undegraded":"2023-09-09T23:13:09.333618+0000","last_fullsized":"2023-09-09T23:13:09.333618+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:17:12.206768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.009341+0000","last_change":"2023-09-09T23:12:17.905510+0000","last_active":"2023-09-09T23:13:39.009341+0000","last_peered":"2023-09-09T23:13:39.009341+0000","last_clean":"2023-09-09T23:13:39.009341+0000","last_became_active":"2023-09-09T23:12:17.905207+0000","last_became_peered":"2023-09-09T23:12:17.905207+0000","last_unstale":"2023-09-09T23:13:39.009341+0000","last_undegraded":"2023-09-09T23:13:39.009341+0000","last_fullsized":"2023-09-09T23:13:39.009341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:48:20.167421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.329203+0000","last_change":"2023-09-09T23:12:17.173764+0000","last_active":"2023-09-09T23:13:09.329203+0000","last_peered":"2023-09-09T23:13:09.329203+0000","last_clean":"2023-09-09T23:13:09.329203+0000","last_became_active":"2023-09-09T23:12:17.173470+0000","last_became_peered":"2023-09-09T23:12:17.173470+0000","last_unstale":"2023-09-09T23:13:09.329203+0000","last_undegraded":"2023-09-09T23:13:09.329203+0000","last_fullsized":"2023-09-09T23:13:09.329203+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:01:54.091664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.020873+0000","last_change":"2023-09-09T23:12:17.906292+0000","last_active":"2023-09-09T23:13:39.020873+0000","last_peered":"2023-09-09T23:13:39.020873+0000","last_clean":"2023-09-09T23:13:39.020873+0000","last_became_active":"2023-09-09T23:12:17.905995+0000","last_became_peered":"2023-09-09T23:12:17.905995+0000","last_unstale":"2023-09-09T23:13:39.020873+0000","last_undegraded":"2023-09-09T23:13:39.020873+0000","last_fullsized":"2023-09-09T23:13:39.020873+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:52:29.972831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'5","reported_seq":99,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:40.421016+0000","last_change":"2023-09-09T23:12:17.173818+0000","last_active":"2023-09-09T23:13:40.421016+0000","last_peered":"2023-09-09T23:13:40.421016+0000","last_clean":"2023-09-09T23:13:40.421016+0000","last_became_active":"2023-09-09T23:12:17.173413+0000","last_became_peered":"2023-09-09T23:12:17.173413+0000","last_unstale":"2023-09-09T23:13:40.421016+0000","last_undegraded":"2023-09-09T23:13:40.421016+0000","last_fullsized":"2023-09-09T23:13:40.421016+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:55:28.856863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.021529+0000","last_change":"2023-09-09T23:12:17.906833+0000","last_active":"2023-09-09T23:13:39.021529+0000","last_peered":"2023-09-09T23:13:39.021529+0000","last_clean":"2023-09-09T23:13:39.021529+0000","last_became_active":"2023-09-09T23:12:17.906138+0000","last_became_peered":"2023-09-09T23:12:17.906138+0000","last_unstale":"2023-09-09T23:13:39.021529+0000","last_undegraded":"2023-09-09T23:13:39.021529+0000","last_fullsized":"2023-09-09T23:13:39.021529+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:38:07.779167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.331058+0000","last_change":"2023-09-09T23:12:17.171338+0000","last_active":"2023-09-09T23:13:09.331058+0000","last_peered":"2023-09-09T23:13:09.331058+0000","last_clean":"2023-09-09T23:13:09.331058+0000","last_became_active":"2023-09-09T23:12:17.170613+0000","last_became_peered":"2023-09-09T23:12:17.170613+0000","last_unstale":"2023-09-09T23:13:09.331058+0000","last_undegraded":"2023-09-09T23:13:09.331058+0000","last_fullsized":"2023-09-09T23:13:09.331058+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:24:07.794464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.017147+0000","last_change":"2023-09-09T23:12:17.906470+0000","last_active":"2023-09-09T23:13:39.017147+0000","last_peered":"2023-09-09T23:13:39.017147+0000","last_clean":"2023-09-09T23:13:39.017147+0000","last_became_active":"2023-09-09T23:12:17.906170+0000","last_became_peered":"2023-09-09T23:12:17.906170+0000","last_unstale":"2023-09-09T23:13:39.017147+0000","last_undegraded":"2023-09-09T23:13:39.017147+0000","last_fullsized":"2023-09-09T23:13:39.017147+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:44:37.483541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.333478+0000","last_change":"2023-09-09T23:12:17.172142+0000","last_active":"2023-09-09T23:13:09.333478+0000","last_peered":"2023-09-09T23:13:09.333478+0000","last_clean":"2023-09-09T23:13:09.333478+0000","last_became_active":"2023-09-09T23:12:17.171703+0000","last_became_peered":"2023-09-09T23:12:17.171703+0000","last_unstale":"2023-09-09T23:13:09.333478+0000","last_undegraded":"2023-09-09T23:13:09.333478+0000","last_fullsized":"2023-09-09T23:13:09.333478+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:49:31.414385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.018478+0000","last_change":"2023-09-09T23:12:17.900699+0000","last_active":"2023-09-09T23:13:39.018478+0000","last_peered":"2023-09-09T23:13:39.018478+0000","last_clean":"2023-09-09T23:13:39.018478+0000","last_became_active":"2023-09-09T23:12:17.900416+0000","last_became_peered":"2023-09-09T23:12:17.900416+0000","last_unstale":"2023-09-09T23:13:39.018478+0000","last_undegraded":"2023-09-09T23:13:39.018478+0000","last_fullsized":"2023-09-09T23:13:39.018478+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:54:29.328413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.326771+0000","last_change":"2023-09-09T23:12:17.900480+0000","last_active":"2023-09-09T23:13:09.326771+0000","last_peered":"2023-09-09T23:13:09.326771+0000","last_clean":"2023-09-09T23:13:09.326771+0000","last_became_active":"2023-09-09T23:12:17.900135+0000","last_became_peered":"2023-09-09T23:12:17.900135+0000","last_unstale":"2023-09-09T23:13:09.326771+0000","last_undegraded":"2023-09-09T23:13:09.326771+0000","last_fullsized":"2023-09-09T23:13:09.326771+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:25:44.254310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.017893+0000","last_change":"2023-09-09T23:12:17.901302+0000","last_active":"2023-09-09T23:13:39.017893+0000","last_peered":"2023-09-09T23:13:39.017893+0000","last_clean":"2023-09-09T23:13:39.017893+0000","last_became_active":"2023-09-09T23:12:17.900982+0000","last_became_peered":"2023-09-09T23:12:17.900982+0000","last_unstale":"2023-09-09T23:13:39.017893+0000","last_undegraded":"2023-09-09T23:13:39.017893+0000","last_fullsized":"2023-09-09T23:13:39.017893+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:41:41.119614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.012146+0000","last_change":"2023-09-09T23:12:17.904328+0000","last_active":"2023-09-09T23:13:39.012146+0000","last_peered":"2023-09-09T23:13:39.012146+0000","last_clean":"2023-09-09T23:13:39.012146+0000","last_became_active":"2023-09-09T23:12:17.904040+0000","last_became_peered":"2023-09-09T23:12:17.904040+0000","last_unstale":"2023-09-09T23:13:39.012146+0000","last_undegraded":"2023-09-09T23:13:39.012146+0000","last_fullsized":"2023-09-09T23:13:39.012146+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:01:05.986097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.004794+0000","last_change":"2023-09-09T23:13:10.337251+0000","last_active":"2023-09-09T23:13:39.004794+0000","last_peered":"2023-09-09T23:13:39.004794+0000","last_clean":"2023-09-09T23:13:39.004794+0000","last_became_active":"2023-09-09T23:13:10.336914+0000","last_became_peered":"2023-09-09T23:13:10.336914+0000","last_unstale":"2023-09-09T23:13:39.004794+0000","last_undegraded":"2023-09-09T23:13:39.004794+0000","last_fullsized":"2023-09-09T23:13:39.004794+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:00:49.058538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.013756+0000","last_change":"2023-09-09T23:12:17.907433+0000","last_active":"2023-09-09T23:13:39.013756+0000","last_peered":"2023-09-09T23:13:39.013756+0000","last_clean":"2023-09-09T23:13:39.013756+0000","last_became_active":"2023-09-09T23:12:17.907127+0000","last_became_peered":"2023-09-09T23:12:17.907127+0000","last_unstale":"2023-09-09T23:13:39.013756+0000","last_undegraded":"2023-09-09T23:13:39.013756+0000","last_fullsized":"2023-09-09T23:13:39.013756+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:00:32.291141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.326990+0000","last_change":"2023-09-09T23:12:17.901373+0000","last_active":"2023-09-09T23:13:09.326990+0000","last_peered":"2023-09-09T23:13:09.326990+0000","last_clean":"2023-09-09T23:13:09.326990+0000","last_became_active":"2023-09-09T23:12:17.901080+0000","last_became_peered":"2023-09-09T23:12:17.901080+0000","last_unstale":"2023-09-09T23:13:09.326990+0000","last_undegraded":"2023-09-09T23:13:09.326990+0000","last_fullsized":"2023-09-09T23:13:09.326990+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:23:12.316659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.020913+0000","last_change":"2023-09-09T23:12:17.896579+0000","last_active":"2023-09-09T23:13:39.020913+0000","last_peered":"2023-09-09T23:13:39.020913+0000","last_clean":"2023-09-09T23:13:39.020913+0000","last_became_active":"2023-09-09T23:12:17.896253+0000","last_became_peered":"2023-09-09T23:12:17.896253+0000","last_unstale":"2023-09-09T23:13:39.020913+0000","last_undegraded":"2023-09-09T23:13:39.020913+0000","last_fullsized":"2023-09-09T23:13:39.020913+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:30:20.249103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.008502+0000","last_change":"2023-09-09T23:12:17.903569+0000","last_active":"2023-09-09T23:13:39.008502+0000","last_peered":"2023-09-09T23:13:39.008502+0000","last_clean":"2023-09-09T23:13:39.008502+0000","last_became_active":"2023-09-09T23:12:17.903027+0000","last_became_peered":"2023-09-09T23:12:17.903027+0000","last_unstale":"2023-09-09T23:13:39.008502+0000","last_undegraded":"2023-09-09T23:13:39.008502+0000","last_fullsized":"2023-09-09T23:13:39.008502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:38:46.392973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.014057+0000","last_change":"2023-09-09T23:12:17.902944+0000","last_active":"2023-09-09T23:13:39.014057+0000","last_peered":"2023-09-09T23:13:39.014057+0000","last_clean":"2023-09-09T23:13:39.014057+0000","last_became_active":"2023-09-09T23:12:17.901812+0000","last_became_peered":"2023-09-09T23:12:17.901812+0000","last_unstale":"2023-09-09T23:13:39.014057+0000","last_undegraded":"2023-09-09T23:13:39.014057+0000","last_fullsized":"2023-09-09T23:13:39.014057+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:49:50.981693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.015867+0000","last_change":"2023-09-09T23:12:17.897612+0000","last_active":"2023-09-09T23:13:39.015867+0000","last_peered":"2023-09-09T23:13:39.015867+0000","last_clean":"2023-09-09T23:13:39.015867+0000","last_became_active":"2023-09-09T23:12:17.897318+0000","last_became_peered":"2023-09-09T23:12:17.897318+0000","last_unstale":"2023-09-09T23:13:39.015867+0000","last_undegraded":"2023-09-09T23:13:39.015867+0000","last_fullsized":"2023-09-09T23:13:39.015867+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:17:38.687470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.022189+0000","last_change":"2023-09-09T23:12:17.903849+0000","last_active":"2023-09-09T23:13:39.022189+0000","last_peered":"2023-09-09T23:13:39.022189+0000","last_clean":"2023-09-09T23:13:39.022189+0000","last_became_active":"2023-09-09T23:12:17.903534+0000","last_became_peered":"2023-09-09T23:12:17.903534+0000","last_unstale":"2023-09-09T23:13:39.022189+0000","last_undegraded":"2023-09-09T23:13:39.022189+0000","last_fullsized":"2023-09-09T23:13:39.022189+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:06:18.073811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.013153+0000","last_change":"2023-09-09T23:12:17.899782+0000","last_active":"2023-09-09T23:13:39.013153+0000","last_peered":"2023-09-09T23:13:39.013153+0000","last_clean":"2023-09-09T23:13:39.013153+0000","last_became_active":"2023-09-09T23:12:17.899445+0000","last_became_peered":"2023-09-09T23:12:17.899445+0000","last_unstale":"2023-09-09T23:13:39.013153+0000","last_undegraded":"2023-09-09T23:13:39.013153+0000","last_fullsized":"2023-09-09T23:13:39.013153+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:49:20.276927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:10.334992+0000","last_change":"2023-09-09T23:13:10.334992+0000","last_active":"2023-09-09T23:13:10.334992+0000","last_peered":"2023-09-09T23:13:10.334992+0000","last_clean":"2023-09-09T23:13:10.334992+0000","last_became_active":"2023-09-09T23:13:10.334629+0000","last_became_peered":"2023-09-09T23:13:10.334629+0000","last_unstale":"2023-09-09T23:13:10.334992+0000","last_undegraded":"2023-09-09T23:13:10.334992+0000","last_fullsized":"2023-09-09T23:13:10.334992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:24:48.291826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":434,"ondisk_log_size":434,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69578,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":812,"num_read_kb":479,"num_write":238,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":90,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569344,"data_stored":204273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514523,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":434403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.65900000000000003}]},{"osd":1,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.747},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.77300000000000002}]},{"osd":2,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":4,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.88600000000000001}]},{"osd":5,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.79500000000000004}]},{"osd":6,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.74199999999999999}]},{"osd":7,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.92200000000000004}]},{"osd":8,"last update":"Sat Sep 9 23:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.72999999999999998}]},{"osd":9,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.93500000000000005}]},{"osd":10,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.873}]}]},{"osd":10,"up_from":69,"seq":296352743456,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1609728,"data_stored":1088114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.65400000000000003}]},{"osd":1,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.33200000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.68000000000000005,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.50700000000000001,"15min":0.51500000000000001},"min":{"1min":0.33700000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.80900000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.58999999999999997}]},{"osd":2,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.623,"15min":0.61499999999999999},"min":{"1min":0.55100000000000005,"5min":0.379,"15min":0.379},"max":{"1min":0.746,"5min":1.018,"15min":1.018},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.65000000000000002,"15min":0.65300000000000002},"min":{"1min":0.501,"5min":0.436,"15min":0.436},"max":{"1min":0.79400000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71999999999999997}]},{"osd":3,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.622,"15min":0.63700000000000001},"min":{"1min":0.41399999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.67400000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.64400000000000002,"15min":0.65900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.67200000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.746}]},{"osd":4,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60899999999999999,"15min":0.61299999999999999},"min":{"1min":0.42399999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.72199999999999998,"5min":1.024,"15min":1.024},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.36099999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79500000000000004,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.73299999999999998}]},{"osd":5,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.69799999999999995,"15min":0.70099999999999996},"min":{"1min":0.56000000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.85799999999999998,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.88},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71399999999999997,"15min":0.71599999999999997},"min":{"1min":0.47699999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88800000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.68899999999999995}]},{"osd":6,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.73099999999999998,"15min":0.75},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90000000000000002,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.73099999999999998,"15min":0.74099999999999999},"min":{"1min":0.54100000000000004,"5min":0.439,"15min":0.439},"max":{"1min":0.85199999999999998,"5min":1.038,"15min":1.038},"last":0.82599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":8,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.80200000000000005,"15min":0.81200000000000006},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.91500000000000004,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78500000000000003,"15min":0.78900000000000003},"min":{"1min":0.623,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.94099999999999995,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.84299999999999997}]},{"osd":9,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.81299999999999994,"15min":0.82399999999999995},"min":{"1min":0.61399999999999999,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.96799999999999997,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.80500000000000005,"15min":0.81999999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.878,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.79600000000000004}]},{"osd":11,"last update":"Sat Sep 9 23:12:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.95899999999999996}]}]},{"osd":9,"up_from":64,"seq":274877906979,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":434243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.435,"15min":0.443},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.73999999999999999,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.39200000000000002}]},{"osd":1,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.62},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56799999999999995,"15min":0.56899999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.60599999999999998}]},{"osd":2,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.626,"15min":0.63},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.625,"15min":0.63200000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.45700000000000002}]},{"osd":3,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.51300000000000001,"15min":0.52300000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.376}]},{"osd":4,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.629,"15min":0.63500000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96799999999999997,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.497},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.67400000000000004,"15min":0.68799999999999994},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81999999999999995,"5min":1.028,"15min":1.028},"last":0.59399999999999997}]},{"osd":5,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.73399999999999999,"15min":0.71499999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":5.6699999999999999,"5min":5.6699999999999999,"15min":5.6699999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.68200000000000005,"15min":0.69399999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.54300000000000004}]},{"osd":6,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.64300000000000002,"15min":0.65500000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.65600000000000003,"15min":0.67000000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.55400000000000005}]},{"osd":7,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.745,"15min":0.76100000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.82499999999999996,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.746,"15min":0.75800000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.66400000000000003}]},{"osd":8,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.73799999999999999,"15min":0.747},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.74099999999999999,"15min":0.755},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.85899999999999999,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.63300000000000001}]},{"osd":10,"last update":"Sat Sep 9 23:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73599999999999999,"15min":0.74199999999999999},"min":{"1min":0.56000000000000005,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.85899999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.88,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.84599999999999997}]}]},{"osd":8,"up_from":57,"seq":244813135913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":434599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.39400000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.87,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.38200000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78500000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.53900000000000003}]},{"osd":1,"last update":"Sat Sep 9 23:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.63200000000000001,"15min":0.60199999999999998},"min":{"1min":0.45600000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.89300000000000002,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60499999999999998,"15min":0.58899999999999997},"min":{"1min":0.36599999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82199999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.63700000000000001}]},{"osd":2,"last update":"Sat Sep 9 23:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.48599999999999999,"15min":0.48399999999999999},"min":{"1min":0.36299999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.78500000000000003,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.48499999999999999,"15min":0.48899999999999999},"min":{"1min":0.309,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69499999999999995,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.72399999999999998}]},{"osd":3,"last update":"Sat Sep 9 23:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57399999999999995,"15min":0.57899999999999996},"min":{"1min":0.497,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.755,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.53900000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.76100000000000001,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.48799999999999999}]},{"osd":4,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64100000000000001,"15min":0.63700000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94399999999999995,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.68999999999999995,"15min":0.68200000000000005},"min":{"1min":0.41899999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.73699999999999999}]},{"osd":5,"last update":"Sat Sep 9 23:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.34699999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.66900000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.47299999999999998}]},{"osd":6,"last update":"Sat Sep 9 23:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.72299999999999998,"15min":0.72599999999999998},"min":{"1min":0.56299999999999994,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86099999999999999,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68600000000000005,"15min":0.69599999999999995},"min":{"1min":0.56699999999999995,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85799999999999998,"5min":0.996,"15min":0.996},"last":0.498}]},{"osd":7,"last update":"Sat Sep 9 23:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72499999999999998,"15min":0.72599999999999998},"min":{"1min":0.52600000000000002,"5min":0.503,"15min":0.503},"max":{"1min":0.96499999999999997,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.89100000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.60799999999999998}]},{"osd":9,"last update":"Sat Sep 9 23:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71999999999999997,"15min":0.72099999999999997},"min":{"1min":0.48899999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.99399999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.77200000000000002,"15min":0.77000000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1060000000000001,"5min":1.115,"15min":1.115},"last":0.65400000000000003}]},{"osd":10,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.74299999999999999,"15min":0.73299999999999998},"min":{"1min":0.63400000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.97499999999999998,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.77500000000000002,"15min":0.75700000000000001},"min":{"1min":0.70699999999999996,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.621}]},{"osd":11,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.73699999999999999}]}]},{"osd":7,"up_from":50,"seq":214748364844,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1519616,"data_stored":1027485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.439},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.33100000000000002}]},{"osd":1,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.436,"15min":0.432},"min":{"1min":0.27800000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.437,"15min":0.42099999999999999},"min":{"1min":0.33200000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.59699999999999998,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.38700000000000001}]},{"osd":2,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63500000000000001,"15min":0.63900000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84399999999999997,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60299999999999998,"15min":0.60199999999999998},"min":{"1min":0.51200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.76600000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.53300000000000003}]},{"osd":3,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.42799999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82799999999999996,"5min":0.88,"15min":0.88},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.623,"15min":0.624},"min":{"1min":0.437,"5min":0.375,"15min":0.375},"max":{"1min":0.81100000000000005,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.56899999999999995}]},{"osd":4,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.35299999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.72599999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.35799999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.79700000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.42399999999999999}]},{"osd":5,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.69099999999999995,"15min":0.69499999999999995},"min":{"1min":0.54900000000000004,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.93999999999999995,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.503,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.88100000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58499999999999996}]},{"osd":6,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.72499999999999998,"15min":0.746},"min":{"1min":0.51500000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.86499999999999999,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.69999999999999996,"15min":0.71999999999999997},"min":{"1min":0.45900000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.753,"5min":1.254,"15min":1.254},"last":0.60899999999999999}]},{"osd":8,"last update":"Sat Sep 9 23:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73399999999999999,"15min":0.73899999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.86399999999999999,"5min":1.008,"15min":1.008},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.76400000000000001,"15min":0.77000000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.88,"5min":1.387,"15min":1.387},"last":0.73899999999999999}]},{"osd":9,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.75,"15min":0.76000000000000001},"min":{"1min":0.51700000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.89900000000000002,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.92400000000000004,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.753}]},{"osd":10,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.46899999999999997,"5min":0.442,"15min":0.442},"max":{"1min":0.95499999999999996,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.74299999999999999,"15min":0.745},"min":{"1min":0.58499999999999996,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.91500000000000004,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.72299999999999998}]},{"osd":11,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.70299999999999996}]}]},{"osd":6,"up_from":45,"seq":193273528369,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":433966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,8,0,0,1,27],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.377,"15min":0.36899999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":5.6859999999999999},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.40400000000000003,"15min":0.40200000000000002},"min":{"1min":0.26800000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.53700000000000003,"5min":0.753,"15min":0.753},"last":1.925}]},{"osd":1,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78100000000000003,"5min":0.877,"15min":0.877},"last":8.6560000000000006},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77200000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":14.58}]},{"osd":2,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.41699999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79600000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":1.798},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76800000000000002,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":4.298}]},{"osd":3,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.43099999999999999,"15min":0.433},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70199999999999996,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":1.181},{"interface":"front","average":{"1min":0.437,"5min":0.501,"15min":0.51200000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.59399999999999997,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":3.1949999999999998}]},{"osd":4,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60699999999999998,"15min":0.60299999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.78100000000000003,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":2.3620000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.77900000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":2.0139999999999998}]},{"osd":5,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.63200000000000001,"15min":0.64000000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.75600000000000001,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.2150000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.65900000000000003,"15min":0.65700000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82199999999999995,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":8.7859999999999996}]},{"osd":7,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.67700000000000005,"15min":0.68000000000000005},"min":{"1min":0.45000000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":6.766},{"interface":"front","average":{"1min":0.626,"5min":0.65800000000000003,"15min":0.66000000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.91500000000000004,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":9.1010000000000009}]},{"osd":8,"last update":"Sat Sep 9 23:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.76700000000000002,"15min":0.78800000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85099999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":1.405},{"interface":"front","average":{"1min":0.628,"5min":0.74399999999999999,"15min":0.76400000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.80100000000000005,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.556}]},{"osd":9,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.64500000000000002,"15min":0.66100000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70799999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":2.4569999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.72199999999999998,"15min":0.752},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73099999999999998,"5min":1.534,"15min":1.534},"last":1.7809999999999999}]},{"osd":10,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.72099999999999997,"15min":0.73099999999999998},"min":{"1min":0.51300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.95099999999999996,"5min":1.163,"15min":1.163},"last":1.522},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.73999999999999999,"15min":0.749},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.84499999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":1.2549999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":491729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.501},"min":{"1min":0.35999999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.88600000000000001,"5min":1.085,"15min":1.085},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52800000000000002,"15min":0.495},"min":{"1min":0.39900000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.96999999999999997,"5min":1.042,"15min":1.042},"last":0.755}]},{"osd":2,"last update":"Sat Sep 9 23:13:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57799999999999996,"15min":0.52000000000000002},"min":{"1min":0.438,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78200000000000003,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.60399999999999998,"15min":0.53900000000000003},"min":{"1min":0.443,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.86599999999999999,"5min":1.133,"15min":1.133},"last":0.54800000000000004}]},{"osd":3,"last update":"Sat Sep 9 23:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.39600000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69699999999999995,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.56999999999999995},"min":{"1min":0.41099999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79500000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.73099999999999998}]},{"osd":4,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.46700000000000003,"15min":0.42399999999999999},"min":{"1min":0.34300000000000003,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.74099999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.53200000000000003,"15min":0.46400000000000002},"min":{"1min":0.498,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.85499999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.501}]},{"osd":5,"last update":"Sat Sep 9 23:13:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.60899999999999999,"15min":0.56999999999999995},"min":{"1min":0.496,"5min":0.377,"15min":0.377},"max":{"1min":0.872,"5min":1.022,"15min":1.022},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.64200000000000002,"15min":0.58699999999999997},"min":{"1min":0.499,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0129999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.66000000000000003}]},{"osd":6,"last update":"Sat Sep 9 23:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59999999999999998,"15min":0.56999999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89600000000000002,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36099999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.68799999999999994,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.70299999999999996}]},{"osd":7,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.57199999999999995,"15min":0.52800000000000002},"min":{"1min":0.36599999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.55300000000000005,"15min":0.52800000000000002},"min":{"1min":0.42099999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80200000000000005,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.46899999999999997}]},{"osd":8,"last update":"Sat Sep 9 23:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.66200000000000003,"15min":0.65600000000000003},"min":{"1min":0.441,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.69899999999999995,"15min":0.69499999999999995},"min":{"1min":0.502,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.95499999999999996,"5min":1.218,"15min":1.218},"last":0.68899999999999995}]},{"osd":9,"last update":"Sat Sep 9 23:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.76200000000000001,"15min":0.76700000000000002},"min":{"1min":0.53700000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93100000000000005,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.70599999999999996,"15min":0.71199999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.82299999999999995,"5min":1.528,"15min":1.528},"last":0.65200000000000002}]},{"osd":10,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.47299999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.89000000000000001,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.79100000000000004,"15min":0.78300000000000003},"min":{"1min":0.55300000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0009999999999999,"5min":1.355,"15min":1.355},"last":0.622}]},{"osd":11,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.78900000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":435299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 9 23:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53900000000000003,"15min":0.51300000000000001},"min":{"1min":0.39200000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.70299999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.33400000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68200000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.626}]},{"osd":2,"last update":"Sat Sep 9 23:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53800000000000003,"15min":0.52900000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.70599999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55100000000000005,"15min":0.52900000000000003},"min":{"1min":0.42299999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.70499999999999996,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.59299999999999997}]},{"osd":3,"last update":"Sat Sep 9 23:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.45700000000000002}]},{"osd":4,"last update":"Sat Sep 9 23:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55100000000000005,"15min":0.53300000000000003},"min":{"1min":0.375,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73399999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.38300000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.67000000000000004,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.52300000000000002}]},{"osd":5,"last update":"Sat Sep 9 23:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71099999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56799999999999995,"15min":0.56499999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.752,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.67100000000000004}]},{"osd":6,"last update":"Sat Sep 9 23:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.495,"15min":0.497},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.55300000000000005,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.504,"15min":0.501},"min":{"1min":0.33000000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67300000000000004,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.64700000000000002}]},{"osd":7,"last update":"Sat Sep 9 23:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.47599999999999998}]},{"osd":8,"last update":"Sat Sep 9 23:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.68700000000000006,"15min":0.69899999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.78000000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.68200000000000005,"15min":0.68999999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85699999999999998,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.71699999999999997}]},{"osd":9,"last update":"Sat Sep 9 23:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.628,"15min":0.65400000000000003},"min":{"1min":0.36799999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.60999999999999999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.67300000000000004,"15min":0.69799999999999995},"min":{"1min":0.34699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70599999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.63600000000000001}]},{"osd":10,"last update":"Sat Sep 9 23:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68200000000000005,"15min":0.68999999999999995},"min":{"1min":0.46800000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81699999999999995,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.67800000000000005,"15min":0.69499999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.71299999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.60599999999999998}]},{"osd":11,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.73399999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":492730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.44,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.92100000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52300000000000002,"15min":0.51700000000000002},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.63200000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.47999999999999998}]},{"osd":1,"last update":"Sat Sep 9 23:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.59899999999999998,"15min":0.55100000000000005},"min":{"1min":0.38700000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.60499999999999998,"15min":0.53400000000000003},"min":{"1min":0.53300000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.93100000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.86399999999999999}]},{"osd":3,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59999999999999998,"15min":0.60699999999999998},"min":{"1min":0.36199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0009999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61099999999999999,"15min":0.61599999999999999},"min":{"1min":0.46300000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90700000000000003,"5min":1.478,"15min":1.478},"last":0.58099999999999996}]},{"osd":4,"last update":"Sat Sep 9 23:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.89900000000000002}]},{"osd":5,"last update":"Sat Sep 9 23:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.45900000000000002,"15min":0.41599999999999998},"min":{"1min":0.35799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.96899999999999997,"5min":1.014,"15min":1.014},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.46999999999999997,"15min":0.432},"min":{"1min":0.29899999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76900000000000002,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.65800000000000003}]},{"osd":6,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.56999999999999995,"15min":0.53200000000000003},"min":{"1min":0.47799999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.749,"5min":0.63300000000000001,"15min":0.59899999999999998},"min":{"1min":0.48799999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.69699999999999995}]},{"osd":7,"last update":"Sat Sep 9 23:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.65700000000000003}]},{"osd":8,"last update":"Sat Sep 9 23:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.626,"15min":0.59499999999999997},"min":{"1min":0.442,"5min":0.37,"15min":0.37},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.63300000000000001,"15min":0.60999999999999999},"min":{"1min":0.56100000000000005,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.878,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.75700000000000001}]},{"osd":9,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.70499999999999996,"15min":0.69099999999999995},"min":{"1min":0.52100000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.752,"15min":0.73699999999999999},"min":{"1min":0.59799999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.73799999999999999}]},{"osd":10,"last update":"Sat Sep 9 23:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.68100000000000005,"15min":0.66700000000000004},"min":{"1min":0.59099999999999997,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.75900000000000001,"15min":0.74099999999999999},"min":{"1min":0.61099999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.873}]},{"osd":11,"last update":"Sat Sep 9 23:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.92300000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182461,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":433466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.32700000000000001,"15min":0.313},"min":{"1min":0.22800000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.501,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35199999999999998,"15min":0.33400000000000002},"min":{"1min":0.249,"5min":0.222,"15min":0.222},"max":{"1min":0.53800000000000003,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.25700000000000001}]},{"osd":1,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54500000000000004,"15min":0.51900000000000002},"min":{"1min":0.32900000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75900000000000001,"5min":1.091,"15min":1.091},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.51000000000000001,"15min":0.51400000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.60799999999999998,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.72699999999999998}]},{"osd":2,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54400000000000004,"15min":0.52000000000000002},"min":{"1min":0.36099999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65100000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53800000000000003,"15min":0.51800000000000002},"min":{"1min":0.40300000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.66200000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.39200000000000002}]},{"osd":4,"last update":"Sat Sep 9 23:13:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54600000000000004,"15min":0.50800000000000001},"min":{"1min":0.39300000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.73599999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.436},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54800000000000004,"15min":0.52000000000000002},"min":{"1min":0.40600000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.65400000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.61099999999999999}]},{"osd":5,"last update":"Sat Sep 9 23:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56499999999999995,"15min":0.57899999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.64200000000000002,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.61199999999999999,"15min":0.60399999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.69499999999999995,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.47099999999999997}]},{"osd":6,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.49099999999999999,"15min":0.49199999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73399999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.377},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.50900000000000001,"15min":0.50600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64800000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.36099999999999999}]},{"osd":7,"last update":"Sat Sep 9 23:13:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.65000000000000002,"15min":0.65900000000000003},"min":{"1min":0.42699999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79000000000000004,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.67500000000000004,"15min":0.69199999999999995},"min":{"1min":0.44800000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.77800000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.63700000000000001}]},{"osd":8,"last update":"Sat Sep 9 23:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.49199999999999999}]},{"osd":9,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.66500000000000004,"15min":0.68899999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77500000000000002,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.67700000000000005,"15min":0.69899999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.746,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.497}]},{"osd":10,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.42799999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.66200000000000003,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.627},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.71299999999999997,"15min":0.72899999999999998},"min":{"1min":0.505,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.76700000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.68700000000000006}]},{"osd":11,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.55400000000000005}]}]},{"osd":4,"up_from":32,"seq":137438953529,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":434211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.503,"15min":0.47099999999999997},"min":{"1min":0.44900000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.52200000000000002,"15min":0.48799999999999999},"min":{"1min":0.41999999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.33300000000000002}]},{"osd":1,"last update":"Sat Sep 9 23:13:14 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.38,"15min":0.34499999999999997},"min":{"1min":0.313,"5min":0.248,"15min":0.248},"max":{"1min":0.60999999999999999,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.41399999999999998,"15min":0.36799999999999999},"min":{"1min":0.36899999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.46700000000000003}]},{"osd":2,"last update":"Sat Sep 9 23:13:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.55800000000000005,"15min":0.52400000000000002},"min":{"1min":0.433,"5min":0.313,"15min":0.313},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.57799999999999996,"15min":0.53200000000000003},"min":{"1min":0.47799999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.55600000000000005}]},{"osd":3,"last update":"Sat Sep 9 23:13:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.55200000000000005,"15min":0.496},"min":{"1min":0.41999999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78000000000000003,"5min":1.008,"15min":1.008},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53400000000000003,"15min":0.499},"min":{"1min":0.38900000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.91100000000000003,"5min":1.002,"15min":1.002},"last":0.497}]},{"osd":5,"last update":"Sat Sep 9 23:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.60599999999999998,"15min":0.58199999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.874,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.627,"15min":0.57099999999999995},"min":{"1min":0.432,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92600000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.72399999999999998}]},{"osd":6,"last update":"Sat Sep 9 23:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64800000000000002,"15min":0.61799999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.85899999999999999,"5min":1.226,"15min":1.226},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.65600000000000003,"15min":0.62},"min":{"1min":0.40000000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.5219999999999998,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":0.68000000000000005}]},{"osd":7,"last update":"Sat Sep 9 23:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.42299999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.88,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.36399999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.754,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.65900000000000003}]},{"osd":8,"last update":"Sat Sep 9 23:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69199999999999995,"15min":0.68899999999999995},"min":{"1min":0.45700000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.749,"15min":0.751},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.89500000000000002,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.82799999999999996}]},{"osd":9,"last update":"Sat Sep 9 23:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80700000000000005,"15min":0.81200000000000006},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.753},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.75800000000000001,"15min":0.76600000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.92800000000000005,"5min":1.127,"15min":1.127},"last":0.76400000000000001}]},{"osd":10,"last update":"Sat Sep 9 23:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.748,"15min":0.751},"min":{"1min":0.53600000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90200000000000002,"5min":1.208,"15min":1.208},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.66800000000000004,"15min":0.66700000000000004},"min":{"1min":0.48799999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.80900000000000005,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.79100000000000004}]},{"osd":11,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.85699999999999998}]}]},{"osd":5,"up_from":37,"seq":158913790004,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1536000,"data_stored":1026963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.48399999999999999,"15min":0.46300000000000002},"min":{"1min":0.40500000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.502,"15min":0.49299999999999999},"min":{"1min":0.377,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.58499999999999996}]},{"osd":1,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.52600000000000002,"15min":0.50800000000000001},"min":{"1min":0.39000000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.53300000000000003,"15min":0.51500000000000001},"min":{"1min":0.36899999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.76500000000000001}]},{"osd":2,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.38400000000000001,"15min":0.372},"min":{"1min":0.27700000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.66500000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.433},{"interface":"front","average":{"1min":0.432,"5min":0.39400000000000002,"15min":0.38800000000000001},"min":{"1min":0.309,"5min":0.248,"15min":0.248},"max":{"1min":0.64100000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.48199999999999998}]},{"osd":3,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.38600000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.75,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.54600000000000004,"15min":0.52400000000000002},"min":{"1min":0.41799999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86599999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.66900000000000004}]},{"osd":4,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.55900000000000005,"15min":0.53300000000000003},"min":{"1min":0.40799999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.46700000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.74199999999999999}]},{"osd":6,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56000000000000005,"15min":0.54600000000000004},"min":{"1min":0.38300000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88300000000000001,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.437,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92200000000000004,"5min":0.998,"15min":0.998},"last":0.627}]},{"osd":7,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.627,"15min":0.59899999999999998},"min":{"1min":0.57299999999999995,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.96799999999999997,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.63100000000000001,"15min":0.61299999999999999},"min":{"1min":0.55400000000000005,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.79000000000000004}]},{"osd":8,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57199999999999995,"15min":0.56399999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83799999999999997,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55200000000000005,"15min":0.54600000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.75600000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.65400000000000003}]},{"osd":9,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.53700000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.82099999999999995,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.70599999999999996,"15min":0.70299999999999996},"min":{"1min":0.56999999999999995,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93899999999999995,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.84099999999999997}]},{"osd":10,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.71899999999999997,"15min":0.70199999999999996},"min":{"1min":0.72199999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.68999999999999995,"15min":0.67500000000000004},"min":{"1min":0.61199999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.82799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-09T23:13:43.566 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-09T23:13:43.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:43 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1241490386' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-09T23:13:43.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:43 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3214365944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-09T23:13:43.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:43 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2105972440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:13:43.722 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.721+0000 7f4172ebd700 1 -- 172.21.15.82:0/2533060504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f416c100150 msgr2=0x7f416c1079f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:43.723 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.721+0000 7f4172ebd700 1 --2- 172.21.15.82:0/2533060504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f416c100150 0x7f416c1079f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f416400b0a0 tx=0x7f416400b3b0 comp rx=0 tx=0).stop 2023-09-09T23:13:43.723 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.721+0000 7f4172ebd700 1 -- 172.21.15.82:0/2533060504 shutdown_connections 2023-09-09T23:13:43.723 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.721+0000 7f4172ebd700 1 --2- 172.21.15.82:0/2533060504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f416c100af0 0x7f416c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:43.724 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.721+0000 7f4172ebd700 1 --2- 172.21.15.82:0/2533060504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f416c100150 0x7f416c1079f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:43.724 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.721+0000 7f4172ebd700 1 --2- 172.21.15.82:0/2533060504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f416c0ff790 0x7f416c0ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:43.725 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.721+0000 7f4172ebd700 1 -- 172.21.15.82:0/2533060504 >> 172.21.15.82:0/2533060504 conn(0x7f416c0fa1c0 msgr2=0x7f416c0fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:43.725 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.721+0000 7f4172ebd700 1 -- 172.21.15.82:0/2533060504 shutdown_connections 2023-09-09T23:13:43.725 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.721+0000 7f4172ebd700 1 -- 172.21.15.82:0/2533060504 wait complete. 2023-09-09T23:13:43.726 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.721+0000 7f4172ebd700 1 Processor -- start 2023-09-09T23:13:43.726 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.721+0000 7f4172ebd700 1 -- start start 2023-09-09T23:13:43.726 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.721+0000 7f4172ebd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f416c0ff790 0x7f416c195240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:43.727 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.721+0000 7f4172ebd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f416c100150 0x7f416c195780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:43.727 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.721+0000 7f4172ebd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f416c100af0 0x7f416c19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:43.728 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.721+0000 7f4172ebd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c19cdc0 con 0x7f416c0ff790 2023-09-09T23:13:43.728 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.721+0000 7f4172ebd700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c19cf00 con 0x7f416c100150 2023-09-09T23:13:43.728 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.721+0000 7f4172ebd700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c19d090 con 0x7f416c100af0 2023-09-09T23:13:43.729 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.722+0000 7f4170c59700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f416c0ff790 0x7f416c195240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:43.729 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.722+0000 7f4170c59700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f416c0ff790 0x7f416c195240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56098/0 (socket says 172.21.15.82:56098) 2023-09-09T23:13:43.729 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.722+0000 7f4170c59700 1 -- 172.21.15.82:0/4241701115 learned_addr learned my addr 172.21.15.82:0/4241701115 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:43.730 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.722+0000 7f417145a700 1 --2- 172.21.15.82:0/4241701115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f416c100af0 0x7f416c19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:43.730 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.722+0000 7f416bfff700 1 --2- 172.21.15.82:0/4241701115 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f416c100150 0x7f416c195780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:43.730 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.722+0000 7f4170c59700 1 -- 172.21.15.82:0/4241701115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f416c100af0 msgr2=0x7f416c19c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:43.730 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.722+0000 7f4170c59700 1 --2- 172.21.15.82:0/4241701115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f416c100af0 0x7f416c19c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:43.730 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.722+0000 7f4170c59700 1 -- 172.21.15.82:0/4241701115 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f416c100150 msgr2=0x7f416c195780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:43.731 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.722+0000 7f4170c59700 1 --2- 172.21.15.82:0/4241701115 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f416c100150 0x7f416c195780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:43.731 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.722+0000 7f4170c59700 1 -- 172.21.15.82:0/4241701115 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4164009d00 con 0x7f416c0ff790 2023-09-09T23:13:43.731 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.722+0000 7f4170c59700 1 --2- 172.21.15.82:0/4241701115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f416c0ff790 0x7f416c195240 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f416000d980 tx=0x7f416000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:43.731 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.722+0000 7f4169ffb700 1 -- 172.21.15.82:0/4241701115 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416000de60 con 0x7f416c0ff790 2023-09-09T23:13:43.732 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.722+0000 7f4169ffb700 1 -- 172.21.15.82:0/4241701115 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41600044e0 con 0x7f416c0ff790 2023-09-09T23:13:43.732 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.722+0000 7f4172ebd700 1 -- 172.21.15.82:0/4241701115 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f416c19d370 con 0x7f416c0ff790 2023-09-09T23:13:43.732 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.722+0000 7f4169ffb700 1 -- 172.21.15.82:0/4241701115 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4160003be0 con 0x7f416c0ff790 2023-09-09T23:13:43.732 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.722+0000 7f4172ebd700 1 -- 172.21.15.82:0/4241701115 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416c19d880 con 0x7f416c0ff790 2023-09-09T23:13:43.732 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.723+0000 7f4172ebd700 1 -- 172.21.15.82:0/4241701115 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f416c106440 con 0x7f416c0ff790 2023-09-09T23:13:43.733 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.725+0000 7f4169ffb700 1 -- 172.21.15.82:0/4241701115 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f4160009bb0 con 0x7f416c0ff790 2023-09-09T23:13:43.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.725+0000 7f4169ffb700 1 --2- 172.21.15.82:0/4241701115 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f415c0a0690 0x7f415c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:43.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.725+0000 7f4169ffb700 1 -- 172.21.15.82:0/4241701115 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f41600cbba0 con 0x7f416c0ff790 2023-09-09T23:13:43.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.725+0000 7f416bfff700 1 --2- 172.21.15.82:0/4241701115 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f415c0a0690 0x7f415c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:43.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.726+0000 7f416bfff700 1 --2- 172.21.15.82:0/4241701115 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f415c0a0690 0x7f415c0a2b50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f416c100810 tx=0x7f4158009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:43.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.729+0000 7f4169ffb700 1 -- 172.21.15.82:0/4241701115 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f41600cbe60 con 0x7f416c0ff790 2023-09-09T23:13:43.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:43 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1241490386' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-09T23:13:43.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:43 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3214365944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-09T23:13:43.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:43 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2105972440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:13:43.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.916+0000 7f4172ebd700 1 -- 172.21.15.82:0/4241701115 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f416c02d000 con 0x7f415c0a0690 2023-09-09T23:13:43.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.926+0000 7f4169ffb700 1 -- 172.21.15.82:0/4241701115 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344353 (secure 0 0 0) 0x7f416c02d000 con 0x7f415c0a0690 2023-09-09T23:13:43.928 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:43.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.929+0000 7f4172ebd700 1 -- 172.21.15.82:0/4241701115 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f415c0a0690 msgr2=0x7f415c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:43.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.929+0000 7f4172ebd700 1 --2- 172.21.15.82:0/4241701115 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f415c0a0690 0x7f415c0a2b50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f416c100810 tx=0x7f4158009260 comp rx=0 tx=0).stop 2023-09-09T23:13:43.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.929+0000 7f4172ebd700 1 -- 172.21.15.82:0/4241701115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f416c0ff790 msgr2=0x7f416c195240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:43.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.929+0000 7f4172ebd700 1 --2- 172.21.15.82:0/4241701115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f416c0ff790 0x7f416c195240 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f416000d980 tx=0x7f416000dd40 comp rx=0 tx=0).stop 2023-09-09T23:13:43.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.929+0000 7f4172ebd700 1 -- 172.21.15.82:0/4241701115 shutdown_connections 2023-09-09T23:13:43.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.930+0000 7f4172ebd700 1 --2- 172.21.15.82:0/4241701115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f416c100af0 0x7f416c19c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:43.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.930+0000 7f4172ebd700 1 --2- 172.21.15.82:0/4241701115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f416c0ff790 0x7f416c195240 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:43.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.930+0000 7f4172ebd700 1 --2- 172.21.15.82:0/4241701115 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f415c0a0690 0x7f415c0a2b50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:43.933 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.930+0000 7f4172ebd700 1 --2- 172.21.15.82:0/4241701115 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f416c100150 0x7f416c195780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:43.933 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.930+0000 7f4172ebd700 1 -- 172.21.15.82:0/4241701115 >> 172.21.15.82:0/4241701115 conn(0x7f416c0fa1c0 msgr2=0x7f416c0fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:43.933 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.930+0000 7f4172ebd700 1 -- 172.21.15.82:0/4241701115 shutdown_connections 2023-09-09T23:13:43.933 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:43.930+0000 7f4172ebd700 1 -- 172.21.15.82:0/4241701115 wait complete. 2023-09-09T23:13:43.934 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-09-09T23:13:43.940 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2023-09-09T23:13:43.000017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660192,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":948,"num_read_kb":598,"num_write":421,"num_write_kb":3078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180826,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":781,"ondisk_log_size":781,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336984,"kb_used_data":12768,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124604840,"statfs":{"total":1151940427776,"available":1151595356160,"internally_reserved":0,"allocated":13074432,"data_stored":7167208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,8,0,0,1,27],"upper_bound":64},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6087,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":72,"num_write":8,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004132"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.327081+0000","last_change":"2023-09-09T23:12:17.903811+0000","last_active":"2023-09-09T23:13:09.327081+0000","last_peered":"2023-09-09T23:13:09.327081+0000","last_clean":"2023-09-09T23:13:09.327081+0000","last_became_active":"2023-09-09T23:12:17.903386+0000","last_became_peered":"2023-09-09T23:12:17.903386+0000","last_unstale":"2023-09-09T23:13:09.327081+0000","last_undegraded":"2023-09-09T23:13:09.327081+0000","last_fullsized":"2023-09-09T23:13:09.327081+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:36:45.122068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.016086+0000","last_change":"2023-09-09T23:12:17.897915+0000","last_active":"2023-09-09T23:13:39.016086+0000","last_peered":"2023-09-09T23:13:39.016086+0000","last_clean":"2023-09-09T23:13:39.016086+0000","last_became_active":"2023-09-09T23:12:17.896727+0000","last_became_peered":"2023-09-09T23:12:17.896727+0000","last_unstale":"2023-09-09T23:13:39.016086+0000","last_undegraded":"2023-09-09T23:13:39.016086+0000","last_fullsized":"2023-09-09T23:13:39.016086+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:36:03.053572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.020527+0000","last_change":"2023-09-09T23:12:17.907957+0000","last_active":"2023-09-09T23:13:39.020527+0000","last_peered":"2023-09-09T23:13:39.020527+0000","last_clean":"2023-09-09T23:13:39.020527+0000","last_became_active":"2023-09-09T23:12:17.907625+0000","last_became_peered":"2023-09-09T23:12:17.907625+0000","last_unstale":"2023-09-09T23:13:39.020527+0000","last_undegraded":"2023-09-09T23:13:39.020527+0000","last_fullsized":"2023-09-09T23:13:39.020527+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:26:56.520987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.327490+0000","last_change":"2023-09-09T23:12:17.903489+0000","last_active":"2023-09-09T23:13:09.327490+0000","last_peered":"2023-09-09T23:13:09.327490+0000","last_clean":"2023-09-09T23:13:09.327490+0000","last_became_active":"2023-09-09T23:12:17.901184+0000","last_became_peered":"2023-09-09T23:12:17.901184+0000","last_unstale":"2023-09-09T23:13:09.327490+0000","last_undegraded":"2023-09-09T23:13:09.327490+0000","last_fullsized":"2023-09-09T23:13:09.327490+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:42:01.899998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.007034+0000","last_change":"2023-09-09T23:12:17.899487+0000","last_active":"2023-09-09T23:13:39.007034+0000","last_peered":"2023-09-09T23:13:39.007034+0000","last_clean":"2023-09-09T23:13:39.007034+0000","last_became_active":"2023-09-09T23:12:17.899083+0000","last_became_peered":"2023-09-09T23:12:17.899083+0000","last_unstale":"2023-09-09T23:13:39.007034+0000","last_undegraded":"2023-09-09T23:13:39.007034+0000","last_fullsized":"2023-09-09T23:13:39.007034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:15:20.109849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.006501+0000","last_change":"2023-09-09T23:12:17.901554+0000","last_active":"2023-09-09T23:13:39.006501+0000","last_peered":"2023-09-09T23:13:39.006501+0000","last_clean":"2023-09-09T23:13:39.006501+0000","last_became_active":"2023-09-09T23:12:17.901246+0000","last_became_peered":"2023-09-09T23:12:17.901246+0000","last_unstale":"2023-09-09T23:13:39.006501+0000","last_undegraded":"2023-09-09T23:13:39.006501+0000","last_fullsized":"2023-09-09T23:13:39.006501+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:19:08.795927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.016998+0000","last_change":"2023-09-09T23:12:17.906989+0000","last_active":"2023-09-09T23:13:39.016998+0000","last_peered":"2023-09-09T23:13:39.016998+0000","last_clean":"2023-09-09T23:13:39.016998+0000","last_became_active":"2023-09-09T23:12:17.906704+0000","last_became_peered":"2023-09-09T23:12:17.906704+0000","last_unstale":"2023-09-09T23:13:39.016998+0000","last_undegraded":"2023-09-09T23:13:39.016998+0000","last_fullsized":"2023-09-09T23:13:39.016998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:41:33.612168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.015516+0000","last_change":"2023-09-09T23:12:17.906269+0000","last_active":"2023-09-09T23:13:39.015516+0000","last_peered":"2023-09-09T23:13:39.015516+0000","last_clean":"2023-09-09T23:13:39.015516+0000","last_became_active":"2023-09-09T23:12:17.906017+0000","last_became_peered":"2023-09-09T23:12:17.906017+0000","last_unstale":"2023-09-09T23:13:39.015516+0000","last_undegraded":"2023-09-09T23:13:39.015516+0000","last_fullsized":"2023-09-09T23:13:39.015516+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:50:21.893506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.020152+0000","last_change":"2023-09-09T23:12:17.905191+0000","last_active":"2023-09-09T23:13:39.020152+0000","last_peered":"2023-09-09T23:13:39.020152+0000","last_clean":"2023-09-09T23:13:39.020152+0000","last_became_active":"2023-09-09T23:12:17.904813+0000","last_became_peered":"2023-09-09T23:12:17.904813+0000","last_unstale":"2023-09-09T23:13:39.020152+0000","last_undegraded":"2023-09-09T23:13:39.020152+0000","last_fullsized":"2023-09-09T23:13:39.020152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:29:54.722210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.008289+0000","last_change":"2023-09-09T23:12:17.899980+0000","last_active":"2023-09-09T23:13:39.008289+0000","last_peered":"2023-09-09T23:13:39.008289+0000","last_clean":"2023-09-09T23:13:39.008289+0000","last_became_active":"2023-09-09T23:12:17.899692+0000","last_became_peered":"2023-09-09T23:12:17.899692+0000","last_unstale":"2023-09-09T23:13:39.008289+0000","last_undegraded":"2023-09-09T23:13:39.008289+0000","last_fullsized":"2023-09-09T23:13:39.008289+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:25:28.056684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.021313+0000","last_change":"2023-09-09T23:12:17.907010+0000","last_active":"2023-09-09T23:13:39.021313+0000","last_peered":"2023-09-09T23:13:39.021313+0000","last_clean":"2023-09-09T23:13:39.021313+0000","last_became_active":"2023-09-09T23:12:17.906565+0000","last_became_peered":"2023-09-09T23:12:17.906565+0000","last_unstale":"2023-09-09T23:13:39.021313+0000","last_undegraded":"2023-09-09T23:13:39.021313+0000","last_fullsized":"2023-09-09T23:13:39.021313+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:08:03.733608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.327384+0000","last_change":"2023-09-09T23:12:17.902889+0000","last_active":"2023-09-09T23:13:09.327384+0000","last_peered":"2023-09-09T23:13:09.327384+0000","last_clean":"2023-09-09T23:13:09.327384+0000","last_became_active":"2023-09-09T23:12:17.900747+0000","last_became_peered":"2023-09-09T23:12:17.900747+0000","last_unstale":"2023-09-09T23:13:09.327384+0000","last_undegraded":"2023-09-09T23:13:09.327384+0000","last_fullsized":"2023-09-09T23:13:09.327384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:01:29.195275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.006815+0000","last_change":"2023-09-09T23:13:10.338443+0000","last_active":"2023-09-09T23:13:39.006815+0000","last_peered":"2023-09-09T23:13:39.006815+0000","last_clean":"2023-09-09T23:13:39.006815+0000","last_became_active":"2023-09-09T23:13:10.338011+0000","last_became_peered":"2023-09-09T23:13:10.338011+0000","last_unstale":"2023-09-09T23:13:39.006815+0000","last_undegraded":"2023-09-09T23:13:39.006815+0000","last_fullsized":"2023-09-09T23:13:39.006815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:27:24.646344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.013765+0000","last_change":"2023-09-09T23:12:17.899255+0000","last_active":"2023-09-09T23:13:39.013765+0000","last_peered":"2023-09-09T23:13:39.013765+0000","last_clean":"2023-09-09T23:13:39.013765+0000","last_became_active":"2023-09-09T23:12:17.898932+0000","last_became_peered":"2023-09-09T23:12:17.898932+0000","last_unstale":"2023-09-09T23:13:39.013765+0000","last_undegraded":"2023-09-09T23:13:39.013765+0000","last_fullsized":"2023-09-09T23:13:39.013765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:10:56.864588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.015679+0000","last_change":"2023-09-09T23:12:17.901083+0000","last_active":"2023-09-09T23:13:39.015679+0000","last_peered":"2023-09-09T23:13:39.015679+0000","last_clean":"2023-09-09T23:13:39.015679+0000","last_became_active":"2023-09-09T23:12:17.900744+0000","last_became_peered":"2023-09-09T23:12:17.900744+0000","last_unstale":"2023-09-09T23:13:39.015679+0000","last_undegraded":"2023-09-09T23:13:39.015679+0000","last_fullsized":"2023-09-09T23:13:39.015679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:37:11.073786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.022442+0000","last_change":"2023-09-09T23:12:17.901962+0000","last_active":"2023-09-09T23:13:39.022442+0000","last_peered":"2023-09-09T23:13:39.022442+0000","last_clean":"2023-09-09T23:13:39.022442+0000","last_became_active":"2023-09-09T23:12:17.901638+0000","last_became_peered":"2023-09-09T23:12:17.901638+0000","last_unstale":"2023-09-09T23:13:39.022442+0000","last_undegraded":"2023-09-09T23:13:39.022442+0000","last_fullsized":"2023-09-09T23:13:39.022442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:34:52.083262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.327117+0000","last_change":"2023-09-09T23:12:17.904424+0000","last_active":"2023-09-09T23:13:09.327117+0000","last_peered":"2023-09-09T23:13:09.327117+0000","last_clean":"2023-09-09T23:13:09.327117+0000","last_became_active":"2023-09-09T23:12:17.903879+0000","last_became_peered":"2023-09-09T23:12:17.903879+0000","last_unstale":"2023-09-09T23:13:09.327117+0000","last_undegraded":"2023-09-09T23:13:09.327117+0000","last_fullsized":"2023-09-09T23:13:09.327117+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:56:42.468630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.020503+0000","last_change":"2023-09-09T23:12:17.905065+0000","last_active":"2023-09-09T23:13:39.020503+0000","last_peered":"2023-09-09T23:13:39.020503+0000","last_clean":"2023-09-09T23:13:39.020503+0000","last_became_active":"2023-09-09T23:12:17.904646+0000","last_became_peered":"2023-09-09T23:12:17.904646+0000","last_unstale":"2023-09-09T23:13:39.020503+0000","last_undegraded":"2023-09-09T23:13:39.020503+0000","last_fullsized":"2023-09-09T23:13:39.020503+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:11:14.705884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.007659+0000","last_change":"2023-09-09T23:12:17.899751+0000","last_active":"2023-09-09T23:13:39.007659+0000","last_peered":"2023-09-09T23:13:39.007659+0000","last_clean":"2023-09-09T23:13:39.007659+0000","last_became_active":"2023-09-09T23:12:17.899410+0000","last_became_peered":"2023-09-09T23:12:17.899410+0000","last_unstale":"2023-09-09T23:13:39.007659+0000","last_undegraded":"2023-09-09T23:13:39.007659+0000","last_fullsized":"2023-09-09T23:13:39.007659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:23:06.098276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":105,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.421559+0000","last_change":"2023-09-09T23:12:17.175671+0000","last_active":"2023-09-09T23:13:39.421559+0000","last_peered":"2023-09-09T23:13:39.421559+0000","last_clean":"2023-09-09T23:13:39.421559+0000","last_became_active":"2023-09-09T23:12:17.175327+0000","last_became_peered":"2023-09-09T23:12:17.175327+0000","last_unstale":"2023-09-09T23:13:39.421559+0000","last_undegraded":"2023-09-09T23:13:39.421559+0000","last_fullsized":"2023-09-09T23:13:39.421559+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:38:57.724260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.008565+0000","last_change":"2023-09-09T23:12:17.906338+0000","last_active":"2023-09-09T23:13:39.008565+0000","last_peered":"2023-09-09T23:13:39.008565+0000","last_clean":"2023-09-09T23:13:39.008565+0000","last_became_active":"2023-09-09T23:12:17.906094+0000","last_became_peered":"2023-09-09T23:12:17.906094+0000","last_unstale":"2023-09-09T23:13:39.008565+0000","last_undegraded":"2023-09-09T23:13:39.008565+0000","last_fullsized":"2023-09-09T23:13:39.008565+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:43:18.183144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.329391+0000","last_change":"2023-09-09T23:12:17.167763+0000","last_active":"2023-09-09T23:13:09.329391+0000","last_peered":"2023-09-09T23:13:09.329391+0000","last_clean":"2023-09-09T23:13:09.329391+0000","last_became_active":"2023-09-09T23:12:17.167455+0000","last_became_peered":"2023-09-09T23:12:17.167455+0000","last_unstale":"2023-09-09T23:13:09.329391+0000","last_undegraded":"2023-09-09T23:13:09.329391+0000","last_fullsized":"2023-09-09T23:13:09.329391+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:15:23.415792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.020222+0000","last_change":"2023-09-09T23:12:17.907936+0000","last_active":"2023-09-09T23:13:39.020222+0000","last_peered":"2023-09-09T23:13:39.020222+0000","last_clean":"2023-09-09T23:13:39.020222+0000","last_became_active":"2023-09-09T23:12:17.907588+0000","last_became_peered":"2023-09-09T23:12:17.907588+0000","last_unstale":"2023-09-09T23:13:39.020222+0000","last_undegraded":"2023-09-09T23:13:39.020222+0000","last_fullsized":"2023-09-09T23:13:39.020222+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:57:30.910812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.329807+0000","last_change":"2023-09-09T23:12:17.172481+0000","last_active":"2023-09-09T23:13:09.329807+0000","last_peered":"2023-09-09T23:13:09.329807+0000","last_clean":"2023-09-09T23:13:09.329807+0000","last_became_active":"2023-09-09T23:12:17.172178+0000","last_became_peered":"2023-09-09T23:12:17.172178+0000","last_unstale":"2023-09-09T23:13:09.329807+0000","last_undegraded":"2023-09-09T23:13:09.329807+0000","last_fullsized":"2023-09-09T23:13:09.329807+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:08:55.328423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.021598+0000","last_change":"2023-09-09T23:12:17.900155+0000","last_active":"2023-09-09T23:13:39.021598+0000","last_peered":"2023-09-09T23:13:39.021598+0000","last_clean":"2023-09-09T23:13:39.021598+0000","last_became_active":"2023-09-09T23:12:17.899862+0000","last_became_peered":"2023-09-09T23:12:17.899862+0000","last_unstale":"2023-09-09T23:13:39.021598+0000","last_undegraded":"2023-09-09T23:13:39.021598+0000","last_fullsized":"2023-09-09T23:13:39.021598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:57:14.949174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.329618+0000","last_change":"2023-09-09T23:12:17.171890+0000","last_active":"2023-09-09T23:13:09.329618+0000","last_peered":"2023-09-09T23:13:09.329618+0000","last_clean":"2023-09-09T23:13:09.329618+0000","last_became_active":"2023-09-09T23:12:17.171496+0000","last_became_peered":"2023-09-09T23:12:17.171496+0000","last_unstale":"2023-09-09T23:13:09.329618+0000","last_undegraded":"2023-09-09T23:13:09.329618+0000","last_fullsized":"2023-09-09T23:13:09.329618+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:38:25.704989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.020476+0000","last_change":"2023-09-09T23:12:17.901067+0000","last_active":"2023-09-09T23:13:39.020476+0000","last_peered":"2023-09-09T23:13:39.020476+0000","last_clean":"2023-09-09T23:13:39.020476+0000","last_became_active":"2023-09-09T23:12:17.900069+0000","last_became_peered":"2023-09-09T23:12:17.900069+0000","last_unstale":"2023-09-09T23:13:39.020476+0000","last_undegraded":"2023-09-09T23:13:39.020476+0000","last_fullsized":"2023-09-09T23:13:39.020476+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:59:55.356235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'7","reported_seq":101,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:38.379346+0000","last_change":"2023-09-09T23:12:17.169474+0000","last_active":"2023-09-09T23:13:38.379346+0000","last_peered":"2023-09-09T23:13:38.379346+0000","last_clean":"2023-09-09T23:13:38.379346+0000","last_became_active":"2023-09-09T23:12:17.169186+0000","last_became_peered":"2023-09-09T23:12:17.169186+0000","last_unstale":"2023-09-09T23:13:38.379346+0000","last_undegraded":"2023-09-09T23:13:38.379346+0000","last_fullsized":"2023-09-09T23:13:38.379346+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:47:13.027949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.014143+0000","last_change":"2023-09-09T23:12:17.901605+0000","last_active":"2023-09-09T23:13:39.014143+0000","last_peered":"2023-09-09T23:13:39.014143+0000","last_clean":"2023-09-09T23:13:39.014143+0000","last_became_active":"2023-09-09T23:12:17.901287+0000","last_became_peered":"2023-09-09T23:12:17.901287+0000","last_unstale":"2023-09-09T23:13:39.014143+0000","last_undegraded":"2023-09-09T23:13:39.014143+0000","last_fullsized":"2023-09-09T23:13:39.014143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:16:41.570750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.333973+0000","last_change":"2023-09-09T23:12:17.171797+0000","last_active":"2023-09-09T23:13:09.333973+0000","last_peered":"2023-09-09T23:13:09.333973+0000","last_clean":"2023-09-09T23:13:09.333973+0000","last_became_active":"2023-09-09T23:12:17.171521+0000","last_became_peered":"2023-09-09T23:12:17.171521+0000","last_unstale":"2023-09-09T23:13:09.333973+0000","last_undegraded":"2023-09-09T23:13:09.333973+0000","last_fullsized":"2023-09-09T23:13:09.333973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:07:51.951393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.009071+0000","last_change":"2023-09-09T23:12:17.900284+0000","last_active":"2023-09-09T23:13:39.009071+0000","last_peered":"2023-09-09T23:13:39.009071+0000","last_clean":"2023-09-09T23:13:39.009071+0000","last_became_active":"2023-09-09T23:12:17.899975+0000","last_became_peered":"2023-09-09T23:12:17.899975+0000","last_unstale":"2023-09-09T23:13:39.009071+0000","last_undegraded":"2023-09-09T23:13:39.009071+0000","last_fullsized":"2023-09-09T23:13:39.009071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:10:56.723338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.333263+0000","last_change":"2023-09-09T23:12:17.175328+0000","last_active":"2023-09-09T23:13:09.333263+0000","last_peered":"2023-09-09T23:13:09.333263+0000","last_clean":"2023-09-09T23:13:09.333263+0000","last_became_active":"2023-09-09T23:12:17.174977+0000","last_became_peered":"2023-09-09T23:12:17.174977+0000","last_unstale":"2023-09-09T23:13:09.333263+0000","last_undegraded":"2023-09-09T23:13:09.333263+0000","last_fullsized":"2023-09-09T23:13:09.333263+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:01:36.272333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.016297+0000","last_change":"2023-09-09T23:12:17.902982+0000","last_active":"2023-09-09T23:13:39.016297+0000","last_peered":"2023-09-09T23:13:39.016297+0000","last_clean":"2023-09-09T23:13:39.016297+0000","last_became_active":"2023-09-09T23:12:17.902677+0000","last_became_peered":"2023-09-09T23:12:17.902677+0000","last_unstale":"2023-09-09T23:13:39.016297+0000","last_undegraded":"2023-09-09T23:13:39.016297+0000","last_fullsized":"2023-09-09T23:13:39.016297+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:08:31.507899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.330756+0000","last_change":"2023-09-09T23:12:17.170099+0000","last_active":"2023-09-09T23:13:09.330756+0000","last_peered":"2023-09-09T23:13:09.330756+0000","last_clean":"2023-09-09T23:13:09.330756+0000","last_became_active":"2023-09-09T23:12:17.169827+0000","last_became_peered":"2023-09-09T23:12:17.169827+0000","last_unstale":"2023-09-09T23:13:09.330756+0000","last_undegraded":"2023-09-09T23:13:09.330756+0000","last_fullsized":"2023-09-09T23:13:09.330756+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:16:34.693568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.007732+0000","last_change":"2023-09-09T23:12:17.903735+0000","last_active":"2023-09-09T23:13:39.007732+0000","last_peered":"2023-09-09T23:13:39.007732+0000","last_clean":"2023-09-09T23:13:39.007732+0000","last_became_active":"2023-09-09T23:12:17.903410+0000","last_became_peered":"2023-09-09T23:12:17.903410+0000","last_unstale":"2023-09-09T23:13:39.007732+0000","last_undegraded":"2023-09-09T23:13:39.007732+0000","last_fullsized":"2023-09-09T23:13:39.007732+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:00:47.664613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'5","reported_seq":176,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:41.384523+0000","last_change":"2023-09-09T23:12:17.176004+0000","last_active":"2023-09-09T23:13:41.384523+0000","last_peered":"2023-09-09T23:13:41.384523+0000","last_clean":"2023-09-09T23:13:41.384523+0000","last_became_active":"2023-09-09T23:12:17.175514+0000","last_became_peered":"2023-09-09T23:12:17.175514+0000","last_unstale":"2023-09-09T23:13:41.384523+0000","last_undegraded":"2023-09-09T23:13:41.384523+0000","last_fullsized":"2023-09-09T23:13:41.384523+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:43:54.587724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":156,"num_read_kb":156,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.021253+0000","last_change":"2023-09-09T23:12:17.900943+0000","last_active":"2023-09-09T23:13:39.021253+0000","last_peered":"2023-09-09T23:13:39.021253+0000","last_clean":"2023-09-09T23:13:39.021253+0000","last_became_active":"2023-09-09T23:12:17.900662+0000","last_became_peered":"2023-09-09T23:12:17.900662+0000","last_unstale":"2023-09-09T23:13:39.021253+0000","last_undegraded":"2023-09-09T23:13:39.021253+0000","last_fullsized":"2023-09-09T23:13:39.021253+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:39:41.480033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.366461+0000","last_change":"2023-09-09T23:12:17.172458+0000","last_active":"2023-09-09T23:13:09.366461+0000","last_peered":"2023-09-09T23:13:09.366461+0000","last_clean":"2023-09-09T23:13:09.366461+0000","last_became_active":"2023-09-09T23:12:17.172119+0000","last_became_peered":"2023-09-09T23:12:17.172119+0000","last_unstale":"2023-09-09T23:13:09.366461+0000","last_undegraded":"2023-09-09T23:13:09.366461+0000","last_fullsized":"2023-09-09T23:13:09.366461+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:31:27.821245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.008347+0000","last_change":"2023-09-09T23:12:17.903252+0000","last_active":"2023-09-09T23:13:39.008347+0000","last_peered":"2023-09-09T23:13:39.008347+0000","last_clean":"2023-09-09T23:13:39.008347+0000","last_became_active":"2023-09-09T23:12:17.902466+0000","last_became_peered":"2023-09-09T23:12:17.902466+0000","last_unstale":"2023-09-09T23:13:39.008347+0000","last_undegraded":"2023-09-09T23:13:39.008347+0000","last_fullsized":"2023-09-09T23:13:39.008347+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:28:07.535937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.331894+0000","last_change":"2023-09-09T23:12:17.165106+0000","last_active":"2023-09-09T23:13:09.331894+0000","last_peered":"2023-09-09T23:13:09.331894+0000","last_clean":"2023-09-09T23:13:09.331894+0000","last_became_active":"2023-09-09T23:12:17.164783+0000","last_became_peered":"2023-09-09T23:12:17.164783+0000","last_unstale":"2023-09-09T23:13:09.331894+0000","last_undegraded":"2023-09-09T23:13:09.331894+0000","last_fullsized":"2023-09-09T23:13:09.331894+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:02:14.984870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.021467+0000","last_change":"2023-09-09T23:12:17.899473+0000","last_active":"2023-09-09T23:13:39.021467+0000","last_peered":"2023-09-09T23:13:39.021467+0000","last_clean":"2023-09-09T23:13:39.021467+0000","last_became_active":"2023-09-09T23:12:17.899142+0000","last_became_peered":"2023-09-09T23:12:17.899142+0000","last_unstale":"2023-09-09T23:13:39.021467+0000","last_undegraded":"2023-09-09T23:13:39.021467+0000","last_fullsized":"2023-09-09T23:13:39.021467+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:50:30.261873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'34","reported_seq":217,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:41.384767+0000","last_change":"2023-09-09T23:12:17.172560+0000","last_active":"2023-09-09T23:13:41.384767+0000","last_peered":"2023-09-09T23:13:41.384767+0000","last_clean":"2023-09-09T23:13:41.384767+0000","last_became_active":"2023-09-09T23:12:17.172213+0000","last_became_peered":"2023-09-09T23:12:17.172213+0000","last_unstale":"2023-09-09T23:13:41.384767+0000","last_undegraded":"2023-09-09T23:13:41.384767+0000","last_fullsized":"2023-09-09T23:13:41.384767+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:30:00.206901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62235,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":168,"num_read_kb":139,"num_write":36,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.017552+0000","last_change":"2023-09-09T23:12:17.898279+0000","last_active":"2023-09-09T23:13:39.017552+0000","last_peered":"2023-09-09T23:13:39.017552+0000","last_clean":"2023-09-09T23:13:39.017552+0000","last_became_active":"2023-09-09T23:12:17.898041+0000","last_became_peered":"2023-09-09T23:12:17.898041+0000","last_unstale":"2023-09-09T23:13:39.017552+0000","last_undegraded":"2023-09-09T23:13:39.017552+0000","last_fullsized":"2023-09-09T23:13:39.017552+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:01:10.315445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:10.350264+0000","last_change":"2023-09-09T23:13:10.349904+0000","last_active":"2023-09-09T23:13:10.350264+0000","last_peered":"2023-09-09T23:13:10.350264+0000","last_clean":"2023-09-09T23:13:10.350264+0000","last_became_active":"2023-09-09T23:13:10.337554+0000","last_became_peered":"2023-09-09T23:13:10.337554+0000","last_unstale":"2023-09-09T23:13:10.350264+0000","last_undegraded":"2023-09-09T23:13:10.350264+0000","last_fullsized":"2023-09-09T23:13:10.350264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:16:39.275038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.012288+0000","last_change":"2023-09-09T23:12:17.904087+0000","last_active":"2023-09-09T23:13:39.012288+0000","last_peered":"2023-09-09T23:13:39.012288+0000","last_clean":"2023-09-09T23:13:39.012288+0000","last_became_active":"2023-09-09T23:12:17.903670+0000","last_became_peered":"2023-09-09T23:12:17.903670+0000","last_unstale":"2023-09-09T23:13:39.012288+0000","last_undegraded":"2023-09-09T23:13:39.012288+0000","last_fullsized":"2023-09-09T23:13:39.012288+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:24:04.403871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.331421+0000","last_change":"2023-09-09T23:12:17.170594+0000","last_active":"2023-09-09T23:13:09.331421+0000","last_peered":"2023-09-09T23:13:09.331421+0000","last_clean":"2023-09-09T23:13:09.331421+0000","last_became_active":"2023-09-09T23:12:17.170223+0000","last_became_peered":"2023-09-09T23:12:17.170223+0000","last_unstale":"2023-09-09T23:13:09.331421+0000","last_undegraded":"2023-09-09T23:13:09.331421+0000","last_fullsized":"2023-09-09T23:13:09.331421+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:30:09.222947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.019450+0000","last_change":"2023-09-09T23:12:17.901047+0000","last_active":"2023-09-09T23:13:39.019450+0000","last_peered":"2023-09-09T23:13:39.019450+0000","last_clean":"2023-09-09T23:13:39.019450+0000","last_became_active":"2023-09-09T23:12:17.900129+0000","last_became_peered":"2023-09-09T23:12:17.900129+0000","last_unstale":"2023-09-09T23:13:39.019450+0000","last_undegraded":"2023-09-09T23:13:39.019450+0000","last_fullsized":"2023-09-09T23:13:39.019450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:59:36.338392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'6","reported_seq":100,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:37.378029+0000","last_change":"2023-09-09T23:12:17.169495+0000","last_active":"2023-09-09T23:13:37.378029+0000","last_peered":"2023-09-09T23:13:37.378029+0000","last_clean":"2023-09-09T23:13:37.378029+0000","last_became_active":"2023-09-09T23:12:17.169170+0000","last_became_peered":"2023-09-09T23:12:17.169170+0000","last_unstale":"2023-09-09T23:13:37.378029+0000","last_undegraded":"2023-09-09T23:13:37.378029+0000","last_fullsized":"2023-09-09T23:13:37.378029+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:36:23.765574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.018670+0000","last_change":"2023-09-09T23:12:17.907826+0000","last_active":"2023-09-09T23:13:39.018670+0000","last_peered":"2023-09-09T23:13:39.018670+0000","last_clean":"2023-09-09T23:13:39.018670+0000","last_became_active":"2023-09-09T23:12:17.907526+0000","last_became_peered":"2023-09-09T23:12:17.907526+0000","last_unstale":"2023-09-09T23:13:39.018670+0000","last_undegraded":"2023-09-09T23:13:39.018670+0000","last_fullsized":"2023-09-09T23:13:39.018670+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:17:20.587253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.328902+0000","last_change":"2023-09-09T23:12:17.170139+0000","last_active":"2023-09-09T23:13:09.328902+0000","last_peered":"2023-09-09T23:13:09.328902+0000","last_clean":"2023-09-09T23:13:09.328902+0000","last_became_active":"2023-09-09T23:12:17.169797+0000","last_became_peered":"2023-09-09T23:12:17.169797+0000","last_unstale":"2023-09-09T23:13:09.328902+0000","last_undegraded":"2023-09-09T23:13:09.328902+0000","last_fullsized":"2023-09-09T23:13:09.328902+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:59:44.668114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.021950+0000","last_change":"2023-09-09T23:12:17.902567+0000","last_active":"2023-09-09T23:13:39.021950+0000","last_peered":"2023-09-09T23:13:39.021950+0000","last_clean":"2023-09-09T23:13:39.021950+0000","last_became_active":"2023-09-09T23:12:17.902229+0000","last_became_peered":"2023-09-09T23:12:17.902229+0000","last_unstale":"2023-09-09T23:13:39.021950+0000","last_undegraded":"2023-09-09T23:13:39.021950+0000","last_fullsized":"2023-09-09T23:13:39.021950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:49:50.049925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:10.341547+0000","last_change":"2023-09-09T23:13:10.341173+0000","last_active":"2023-09-09T23:13:10.341547+0000","last_peered":"2023-09-09T23:13:10.341547+0000","last_clean":"2023-09-09T23:13:10.341547+0000","last_became_active":"2023-09-09T23:13:10.336085+0000","last_became_peered":"2023-09-09T23:13:10.336085+0000","last_unstale":"2023-09-09T23:13:10.341547+0000","last_undegraded":"2023-09-09T23:13:10.341547+0000","last_fullsized":"2023-09-09T23:13:10.341547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:14:47.365794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.326890+0000","last_change":"2023-09-09T23:12:17.900284+0000","last_active":"2023-09-09T23:13:09.326890+0000","last_peered":"2023-09-09T23:13:09.326890+0000","last_clean":"2023-09-09T23:13:09.326890+0000","last_became_active":"2023-09-09T23:12:17.899923+0000","last_became_peered":"2023-09-09T23:12:17.899923+0000","last_unstale":"2023-09-09T23:13:09.326890+0000","last_undegraded":"2023-09-09T23:13:09.326890+0000","last_fullsized":"2023-09-09T23:13:09.326890+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:32:32.230968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.331153+0000","last_change":"2023-09-09T23:12:17.176313+0000","last_active":"2023-09-09T23:13:09.331153+0000","last_peered":"2023-09-09T23:13:09.331153+0000","last_clean":"2023-09-09T23:13:09.331153+0000","last_became_active":"2023-09-09T23:12:17.175934+0000","last_became_peered":"2023-09-09T23:12:17.175934+0000","last_unstale":"2023-09-09T23:13:09.331153+0000","last_undegraded":"2023-09-09T23:13:09.331153+0000","last_fullsized":"2023-09-09T23:13:09.331153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:33:15.044986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.006607+0000","last_change":"2023-09-09T23:12:17.900284+0000","last_active":"2023-09-09T23:13:39.006607+0000","last_peered":"2023-09-09T23:13:39.006607+0000","last_clean":"2023-09-09T23:13:39.006607+0000","last_became_active":"2023-09-09T23:12:17.899970+0000","last_became_peered":"2023-09-09T23:12:17.899970+0000","last_unstale":"2023-09-09T23:13:39.006607+0000","last_undegraded":"2023-09-09T23:13:39.006607+0000","last_fullsized":"2023-09-09T23:13:39.006607+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:01:37.415195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":96,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:38.405308+0000","last_change":"2023-09-09T23:12:17.170123+0000","last_active":"2023-09-09T23:13:38.405308+0000","last_peered":"2023-09-09T23:13:38.405308+0000","last_clean":"2023-09-09T23:13:38.405308+0000","last_became_active":"2023-09-09T23:12:17.169775+0000","last_became_peered":"2023-09-09T23:12:17.169775+0000","last_unstale":"2023-09-09T23:13:38.405308+0000","last_undegraded":"2023-09-09T23:13:38.405308+0000","last_fullsized":"2023-09-09T23:13:38.405308+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:22:44.520489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.011772+0000","last_change":"2023-09-09T23:12:17.894881+0000","last_active":"2023-09-09T23:13:39.011772+0000","last_peered":"2023-09-09T23:13:39.011772+0000","last_clean":"2023-09-09T23:13:39.011772+0000","last_became_active":"2023-09-09T23:12:17.894567+0000","last_became_peered":"2023-09-09T23:12:17.894567+0000","last_unstale":"2023-09-09T23:13:39.011772+0000","last_undegraded":"2023-09-09T23:13:39.011772+0000","last_fullsized":"2023-09-09T23:13:39.011772+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:40:37.098780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.015183+0000","last_change":"2023-09-09T23:12:17.903319+0000","last_active":"2023-09-09T23:13:39.015183+0000","last_peered":"2023-09-09T23:13:39.015183+0000","last_clean":"2023-09-09T23:13:39.015183+0000","last_became_active":"2023-09-09T23:12:17.902642+0000","last_became_peered":"2023-09-09T23:12:17.902642+0000","last_unstale":"2023-09-09T23:13:39.015183+0000","last_undegraded":"2023-09-09T23:13:39.015183+0000","last_fullsized":"2023-09-09T23:13:39.015183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:03:29.562813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":99,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:38.975476+0000","last_change":"2023-09-09T23:12:17.172639+0000","last_active":"2023-09-09T23:13:38.975476+0000","last_peered":"2023-09-09T23:13:38.975476+0000","last_clean":"2023-09-09T23:13:38.975476+0000","last_became_active":"2023-09-09T23:12:17.172309+0000","last_became_peered":"2023-09-09T23:12:17.172309+0000","last_unstale":"2023-09-09T23:13:38.975476+0000","last_undegraded":"2023-09-09T23:13:38.975476+0000","last_fullsized":"2023-09-09T23:13:38.975476+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:05:56.160984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.019865+0000","last_change":"2023-09-09T23:13:10.336403+0000","last_active":"2023-09-09T23:13:39.019865+0000","last_peered":"2023-09-09T23:13:39.019865+0000","last_clean":"2023-09-09T23:13:39.019865+0000","last_became_active":"2023-09-09T23:13:10.335730+0000","last_became_peered":"2023-09-09T23:13:10.335730+0000","last_unstale":"2023-09-09T23:13:39.019865+0000","last_undegraded":"2023-09-09T23:13:39.019865+0000","last_fullsized":"2023-09-09T23:13:39.019865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:35:00.228469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.327321+0000","last_change":"2023-09-09T23:12:17.166623+0000","last_active":"2023-09-09T23:13:09.327321+0000","last_peered":"2023-09-09T23:13:09.327321+0000","last_clean":"2023-09-09T23:13:09.327321+0000","last_became_active":"2023-09-09T23:12:17.166293+0000","last_became_peered":"2023-09-09T23:12:17.166293+0000","last_unstale":"2023-09-09T23:13:09.327321+0000","last_undegraded":"2023-09-09T23:13:09.327321+0000","last_fullsized":"2023-09-09T23:13:09.327321+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:52:51.267501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":319,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.366308+0000","last_change":"2023-09-09T23:11:20.948461+0000","last_active":"2023-09-09T23:13:09.366308+0000","last_peered":"2023-09-09T23:13:09.366308+0000","last_clean":"2023-09-09T23:13:09.366308+0000","last_became_active":"2023-09-09T23:11:20.947856+0000","last_became_peered":"2023-09-09T23:11:20.947856+0000","last_unstale":"2023-09-09T23:13:09.366308+0000","last_undegraded":"2023-09-09T23:13:09.366308+0000","last_fullsized":"2023-09-09T23:13:09.366308+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:08:31.737051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:08:31.737051+0000","last_clean_scrub_stamp":"2023-09-09T23:08:31.737051+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:20:16.293707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.013785+0000","last_change":"2023-09-09T23:12:17.907468+0000","last_active":"2023-09-09T23:13:39.013785+0000","last_peered":"2023-09-09T23:13:39.013785+0000","last_clean":"2023-09-09T23:13:39.013785+0000","last_became_active":"2023-09-09T23:12:17.907117+0000","last_became_peered":"2023-09-09T23:12:17.907117+0000","last_unstale":"2023-09-09T23:13:39.013785+0000","last_undegraded":"2023-09-09T23:13:39.013785+0000","last_fullsized":"2023-09-09T23:13:39.013785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:27:16.623116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.366405+0000","last_change":"2023-09-09T23:12:17.171505+0000","last_active":"2023-09-09T23:13:09.366405+0000","last_peered":"2023-09-09T23:13:09.366405+0000","last_clean":"2023-09-09T23:13:09.366405+0000","last_became_active":"2023-09-09T23:12:17.171185+0000","last_became_peered":"2023-09-09T23:12:17.171185+0000","last_unstale":"2023-09-09T23:13:09.366405+0000","last_undegraded":"2023-09-09T23:13:09.366405+0000","last_fullsized":"2023-09-09T23:13:09.366405+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:07:54.271364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.329554+0000","last_change":"2023-09-09T23:12:17.176450+0000","last_active":"2023-09-09T23:13:09.329554+0000","last_peered":"2023-09-09T23:13:09.329554+0000","last_clean":"2023-09-09T23:13:09.329554+0000","last_became_active":"2023-09-09T23:12:17.176217+0000","last_became_peered":"2023-09-09T23:12:17.176217+0000","last_unstale":"2023-09-09T23:13:09.329554+0000","last_undegraded":"2023-09-09T23:13:09.329554+0000","last_fullsized":"2023-09-09T23:13:09.329554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:57:26.604261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.008179+0000","last_change":"2023-09-09T23:12:17.897612+0000","last_active":"2023-09-09T23:13:39.008179+0000","last_peered":"2023-09-09T23:13:39.008179+0000","last_clean":"2023-09-09T23:13:39.008179+0000","last_became_active":"2023-09-09T23:12:17.897292+0000","last_became_peered":"2023-09-09T23:12:17.897292+0000","last_unstale":"2023-09-09T23:13:39.008179+0000","last_undegraded":"2023-09-09T23:13:39.008179+0000","last_fullsized":"2023-09-09T23:13:39.008179+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:25:46.374556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'9","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.331260+0000","last_change":"2023-09-09T23:12:17.175682+0000","last_active":"2023-09-09T23:13:09.331260+0000","last_peered":"2023-09-09T23:13:09.331260+0000","last_clean":"2023-09-09T23:13:09.331260+0000","last_became_active":"2023-09-09T23:12:17.175291+0000","last_became_peered":"2023-09-09T23:12:17.175291+0000","last_unstale":"2023-09-09T23:13:09.331260+0000","last_undegraded":"2023-09-09T23:13:09.331260+0000","last_fullsized":"2023-09-09T23:13:09.331260+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:29:02.349319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.016472+0000","last_change":"2023-09-09T23:12:17.904153+0000","last_active":"2023-09-09T23:13:39.016472+0000","last_peered":"2023-09-09T23:13:39.016472+0000","last_clean":"2023-09-09T23:13:39.016472+0000","last_became_active":"2023-09-09T23:12:17.903804+0000","last_became_peered":"2023-09-09T23:12:17.903804+0000","last_unstale":"2023-09-09T23:13:39.016472+0000","last_undegraded":"2023-09-09T23:13:39.016472+0000","last_fullsized":"2023-09-09T23:13:39.016472+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:39:34.046551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.327392+0000","last_change":"2023-09-09T23:12:17.169052+0000","last_active":"2023-09-09T23:13:09.327392+0000","last_peered":"2023-09-09T23:13:09.327392+0000","last_clean":"2023-09-09T23:13:09.327392+0000","last_became_active":"2023-09-09T23:12:17.168719+0000","last_became_peered":"2023-09-09T23:12:17.168719+0000","last_unstale":"2023-09-09T23:13:09.327392+0000","last_undegraded":"2023-09-09T23:13:09.327392+0000","last_fullsized":"2023-09-09T23:13:09.327392+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:56:44.427954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.011144+0000","last_change":"2023-09-09T23:12:17.900945+0000","last_active":"2023-09-09T23:13:39.011144+0000","last_peered":"2023-09-09T23:13:39.011144+0000","last_clean":"2023-09-09T23:13:39.011144+0000","last_became_active":"2023-09-09T23:12:17.900602+0000","last_became_peered":"2023-09-09T23:12:17.900602+0000","last_unstale":"2023-09-09T23:13:39.011144+0000","last_undegraded":"2023-09-09T23:13:39.011144+0000","last_fullsized":"2023-09-09T23:13:39.011144+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:22:10.336285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.331603+0000","last_change":"2023-09-09T23:12:17.166619+0000","last_active":"2023-09-09T23:13:09.331603+0000","last_peered":"2023-09-09T23:13:09.331603+0000","last_clean":"2023-09-09T23:13:09.331603+0000","last_became_active":"2023-09-09T23:12:17.166286+0000","last_became_peered":"2023-09-09T23:12:17.166286+0000","last_unstale":"2023-09-09T23:13:09.331603+0000","last_undegraded":"2023-09-09T23:13:09.331603+0000","last_fullsized":"2023-09-09T23:13:09.331603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:41:40.713296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:38.983854+0000","last_change":"2023-09-09T23:12:17.897131+0000","last_active":"2023-09-09T23:13:38.983854+0000","last_peered":"2023-09-09T23:13:38.983854+0000","last_clean":"2023-09-09T23:13:38.983854+0000","last_became_active":"2023-09-09T23:12:17.896873+0000","last_became_peered":"2023-09-09T23:12:17.896873+0000","last_unstale":"2023-09-09T23:13:38.983854+0000","last_undegraded":"2023-09-09T23:13:38.983854+0000","last_fullsized":"2023-09-09T23:13:38.983854+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:54:21.091418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.328807+0000","last_change":"2023-09-09T23:12:17.168868+0000","last_active":"2023-09-09T23:13:09.328807+0000","last_peered":"2023-09-09T23:13:09.328807+0000","last_clean":"2023-09-09T23:13:09.328807+0000","last_became_active":"2023-09-09T23:12:17.168572+0000","last_became_peered":"2023-09-09T23:12:17.168572+0000","last_unstale":"2023-09-09T23:13:09.328807+0000","last_undegraded":"2023-09-09T23:13:09.328807+0000","last_fullsized":"2023-09-09T23:13:09.328807+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:54:08.108023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.326672+0000","last_change":"2023-09-09T23:12:17.902887+0000","last_active":"2023-09-09T23:13:09.326672+0000","last_peered":"2023-09-09T23:13:09.326672+0000","last_clean":"2023-09-09T23:13:09.326672+0000","last_became_active":"2023-09-09T23:12:17.902507+0000","last_became_peered":"2023-09-09T23:12:17.902507+0000","last_unstale":"2023-09-09T23:13:09.326672+0000","last_undegraded":"2023-09-09T23:13:09.326672+0000","last_fullsized":"2023-09-09T23:13:09.326672+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:18:08.000184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.333618+0000","last_change":"2023-09-09T23:12:17.171379+0000","last_active":"2023-09-09T23:13:09.333618+0000","last_peered":"2023-09-09T23:13:09.333618+0000","last_clean":"2023-09-09T23:13:09.333618+0000","last_became_active":"2023-09-09T23:12:17.171024+0000","last_became_peered":"2023-09-09T23:12:17.171024+0000","last_unstale":"2023-09-09T23:13:09.333618+0000","last_undegraded":"2023-09-09T23:13:09.333618+0000","last_fullsized":"2023-09-09T23:13:09.333618+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:17:12.206768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.009341+0000","last_change":"2023-09-09T23:12:17.905510+0000","last_active":"2023-09-09T23:13:39.009341+0000","last_peered":"2023-09-09T23:13:39.009341+0000","last_clean":"2023-09-09T23:13:39.009341+0000","last_became_active":"2023-09-09T23:12:17.905207+0000","last_became_peered":"2023-09-09T23:12:17.905207+0000","last_unstale":"2023-09-09T23:13:39.009341+0000","last_undegraded":"2023-09-09T23:13:39.009341+0000","last_fullsized":"2023-09-09T23:13:39.009341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:48:20.167421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.329203+0000","last_change":"2023-09-09T23:12:17.173764+0000","last_active":"2023-09-09T23:13:09.329203+0000","last_peered":"2023-09-09T23:13:09.329203+0000","last_clean":"2023-09-09T23:13:09.329203+0000","last_became_active":"2023-09-09T23:12:17.173470+0000","last_became_peered":"2023-09-09T23:12:17.173470+0000","last_unstale":"2023-09-09T23:13:09.329203+0000","last_undegraded":"2023-09-09T23:13:09.329203+0000","last_fullsized":"2023-09-09T23:13:09.329203+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:01:54.091664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.020873+0000","last_change":"2023-09-09T23:12:17.906292+0000","last_active":"2023-09-09T23:13:39.020873+0000","last_peered":"2023-09-09T23:13:39.020873+0000","last_clean":"2023-09-09T23:13:39.020873+0000","last_became_active":"2023-09-09T23:12:17.905995+0000","last_became_peered":"2023-09-09T23:12:17.905995+0000","last_unstale":"2023-09-09T23:13:39.020873+0000","last_undegraded":"2023-09-09T23:13:39.020873+0000","last_fullsized":"2023-09-09T23:13:39.020873+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:52:29.972831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'5","reported_seq":99,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:40.421016+0000","last_change":"2023-09-09T23:12:17.173818+0000","last_active":"2023-09-09T23:13:40.421016+0000","last_peered":"2023-09-09T23:13:40.421016+0000","last_clean":"2023-09-09T23:13:40.421016+0000","last_became_active":"2023-09-09T23:12:17.173413+0000","last_became_peered":"2023-09-09T23:12:17.173413+0000","last_unstale":"2023-09-09T23:13:40.421016+0000","last_undegraded":"2023-09-09T23:13:40.421016+0000","last_fullsized":"2023-09-09T23:13:40.421016+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:55:28.856863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.021529+0000","last_change":"2023-09-09T23:12:17.906833+0000","last_active":"2023-09-09T23:13:39.021529+0000","last_peered":"2023-09-09T23:13:39.021529+0000","last_clean":"2023-09-09T23:13:39.021529+0000","last_became_active":"2023-09-09T23:12:17.906138+0000","last_became_peered":"2023-09-09T23:12:17.906138+0000","last_unstale":"2023-09-09T23:13:39.021529+0000","last_undegraded":"2023-09-09T23:13:39.021529+0000","last_fullsized":"2023-09-09T23:13:39.021529+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:38:07.779167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.331058+0000","last_change":"2023-09-09T23:12:17.171338+0000","last_active":"2023-09-09T23:13:09.331058+0000","last_peered":"2023-09-09T23:13:09.331058+0000","last_clean":"2023-09-09T23:13:09.331058+0000","last_became_active":"2023-09-09T23:12:17.170613+0000","last_became_peered":"2023-09-09T23:12:17.170613+0000","last_unstale":"2023-09-09T23:13:09.331058+0000","last_undegraded":"2023-09-09T23:13:09.331058+0000","last_fullsized":"2023-09-09T23:13:09.331058+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:24:07.794464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.017147+0000","last_change":"2023-09-09T23:12:17.906470+0000","last_active":"2023-09-09T23:13:39.017147+0000","last_peered":"2023-09-09T23:13:39.017147+0000","last_clean":"2023-09-09T23:13:39.017147+0000","last_became_active":"2023-09-09T23:12:17.906170+0000","last_became_peered":"2023-09-09T23:12:17.906170+0000","last_unstale":"2023-09-09T23:13:39.017147+0000","last_undegraded":"2023-09-09T23:13:39.017147+0000","last_fullsized":"2023-09-09T23:13:39.017147+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:44:37.483541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.333478+0000","last_change":"2023-09-09T23:12:17.172142+0000","last_active":"2023-09-09T23:13:09.333478+0000","last_peered":"2023-09-09T23:13:09.333478+0000","last_clean":"2023-09-09T23:13:09.333478+0000","last_became_active":"2023-09-09T23:12:17.171703+0000","last_became_peered":"2023-09-09T23:12:17.171703+0000","last_unstale":"2023-09-09T23:13:09.333478+0000","last_undegraded":"2023-09-09T23:13:09.333478+0000","last_fullsized":"2023-09-09T23:13:09.333478+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:49:31.414385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.018478+0000","last_change":"2023-09-09T23:12:17.900699+0000","last_active":"2023-09-09T23:13:39.018478+0000","last_peered":"2023-09-09T23:13:39.018478+0000","last_clean":"2023-09-09T23:13:39.018478+0000","last_became_active":"2023-09-09T23:12:17.900416+0000","last_became_peered":"2023-09-09T23:12:17.900416+0000","last_unstale":"2023-09-09T23:13:39.018478+0000","last_undegraded":"2023-09-09T23:13:39.018478+0000","last_fullsized":"2023-09-09T23:13:39.018478+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:54:29.328413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.326771+0000","last_change":"2023-09-09T23:12:17.900480+0000","last_active":"2023-09-09T23:13:09.326771+0000","last_peered":"2023-09-09T23:13:09.326771+0000","last_clean":"2023-09-09T23:13:09.326771+0000","last_became_active":"2023-09-09T23:12:17.900135+0000","last_became_peered":"2023-09-09T23:12:17.900135+0000","last_unstale":"2023-09-09T23:13:09.326771+0000","last_undegraded":"2023-09-09T23:13:09.326771+0000","last_fullsized":"2023-09-09T23:13:09.326771+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:25:44.254310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.017893+0000","last_change":"2023-09-09T23:12:17.901302+0000","last_active":"2023-09-09T23:13:39.017893+0000","last_peered":"2023-09-09T23:13:39.017893+0000","last_clean":"2023-09-09T23:13:39.017893+0000","last_became_active":"2023-09-09T23:12:17.900982+0000","last_became_peered":"2023-09-09T23:12:17.900982+0000","last_unstale":"2023-09-09T23:13:39.017893+0000","last_undegraded":"2023-09-09T23:13:39.017893+0000","last_fullsized":"2023-09-09T23:13:39.017893+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:41:41.119614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.012146+0000","last_change":"2023-09-09T23:12:17.904328+0000","last_active":"2023-09-09T23:13:39.012146+0000","last_peered":"2023-09-09T23:13:39.012146+0000","last_clean":"2023-09-09T23:13:39.012146+0000","last_became_active":"2023-09-09T23:12:17.904040+0000","last_became_peered":"2023-09-09T23:12:17.904040+0000","last_unstale":"2023-09-09T23:13:39.012146+0000","last_undegraded":"2023-09-09T23:13:39.012146+0000","last_fullsized":"2023-09-09T23:13:39.012146+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:01:05.986097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.004794+0000","last_change":"2023-09-09T23:13:10.337251+0000","last_active":"2023-09-09T23:13:39.004794+0000","last_peered":"2023-09-09T23:13:39.004794+0000","last_clean":"2023-09-09T23:13:39.004794+0000","last_became_active":"2023-09-09T23:13:10.336914+0000","last_became_peered":"2023-09-09T23:13:10.336914+0000","last_unstale":"2023-09-09T23:13:39.004794+0000","last_undegraded":"2023-09-09T23:13:39.004794+0000","last_fullsized":"2023-09-09T23:13:39.004794+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:00:49.058538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.013756+0000","last_change":"2023-09-09T23:12:17.907433+0000","last_active":"2023-09-09T23:13:39.013756+0000","last_peered":"2023-09-09T23:13:39.013756+0000","last_clean":"2023-09-09T23:13:39.013756+0000","last_became_active":"2023-09-09T23:12:17.907127+0000","last_became_peered":"2023-09-09T23:12:17.907127+0000","last_unstale":"2023-09-09T23:13:39.013756+0000","last_undegraded":"2023-09-09T23:13:39.013756+0000","last_fullsized":"2023-09-09T23:13:39.013756+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:00:32.291141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.326990+0000","last_change":"2023-09-09T23:12:17.901373+0000","last_active":"2023-09-09T23:13:09.326990+0000","last_peered":"2023-09-09T23:13:09.326990+0000","last_clean":"2023-09-09T23:13:09.326990+0000","last_became_active":"2023-09-09T23:12:17.901080+0000","last_became_peered":"2023-09-09T23:12:17.901080+0000","last_unstale":"2023-09-09T23:13:09.326990+0000","last_undegraded":"2023-09-09T23:13:09.326990+0000","last_fullsized":"2023-09-09T23:13:09.326990+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:23:12.316659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.020913+0000","last_change":"2023-09-09T23:12:17.896579+0000","last_active":"2023-09-09T23:13:39.020913+0000","last_peered":"2023-09-09T23:13:39.020913+0000","last_clean":"2023-09-09T23:13:39.020913+0000","last_became_active":"2023-09-09T23:12:17.896253+0000","last_became_peered":"2023-09-09T23:12:17.896253+0000","last_unstale":"2023-09-09T23:13:39.020913+0000","last_undegraded":"2023-09-09T23:13:39.020913+0000","last_fullsized":"2023-09-09T23:13:39.020913+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:30:20.249103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.008502+0000","last_change":"2023-09-09T23:12:17.903569+0000","last_active":"2023-09-09T23:13:39.008502+0000","last_peered":"2023-09-09T23:13:39.008502+0000","last_clean":"2023-09-09T23:13:39.008502+0000","last_became_active":"2023-09-09T23:12:17.903027+0000","last_became_peered":"2023-09-09T23:12:17.903027+0000","last_unstale":"2023-09-09T23:13:39.008502+0000","last_undegraded":"2023-09-09T23:13:39.008502+0000","last_fullsized":"2023-09-09T23:13:39.008502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:38:46.392973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.014057+0000","last_change":"2023-09-09T23:12:17.902944+0000","last_active":"2023-09-09T23:13:39.014057+0000","last_peered":"2023-09-09T23:13:39.014057+0000","last_clean":"2023-09-09T23:13:39.014057+0000","last_became_active":"2023-09-09T23:12:17.901812+0000","last_became_peered":"2023-09-09T23:12:17.901812+0000","last_unstale":"2023-09-09T23:13:39.014057+0000","last_undegraded":"2023-09-09T23:13:39.014057+0000","last_fullsized":"2023-09-09T23:13:39.014057+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:49:50.981693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.015867+0000","last_change":"2023-09-09T23:12:17.897612+0000","last_active":"2023-09-09T23:13:39.015867+0000","last_peered":"2023-09-09T23:13:39.015867+0000","last_clean":"2023-09-09T23:13:39.015867+0000","last_became_active":"2023-09-09T23:12:17.897318+0000","last_became_peered":"2023-09-09T23:12:17.897318+0000","last_unstale":"2023-09-09T23:13:39.015867+0000","last_undegraded":"2023-09-09T23:13:39.015867+0000","last_fullsized":"2023-09-09T23:13:39.015867+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:17:38.687470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.022189+0000","last_change":"2023-09-09T23:12:17.903849+0000","last_active":"2023-09-09T23:13:39.022189+0000","last_peered":"2023-09-09T23:13:39.022189+0000","last_clean":"2023-09-09T23:13:39.022189+0000","last_became_active":"2023-09-09T23:12:17.903534+0000","last_became_peered":"2023-09-09T23:12:17.903534+0000","last_unstale":"2023-09-09T23:13:39.022189+0000","last_undegraded":"2023-09-09T23:13:39.022189+0000","last_fullsized":"2023-09-09T23:13:39.022189+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:06:18.073811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.013153+0000","last_change":"2023-09-09T23:12:17.899782+0000","last_active":"2023-09-09T23:13:39.013153+0000","last_peered":"2023-09-09T23:13:39.013153+0000","last_clean":"2023-09-09T23:13:39.013153+0000","last_became_active":"2023-09-09T23:12:17.899445+0000","last_became_peered":"2023-09-09T23:12:17.899445+0000","last_unstale":"2023-09-09T23:13:39.013153+0000","last_undegraded":"2023-09-09T23:13:39.013153+0000","last_fullsized":"2023-09-09T23:13:39.013153+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:49:20.276927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:10.334992+0000","last_change":"2023-09-09T23:13:10.334992+0000","last_active":"2023-09-09T23:13:10.334992+0000","last_peered":"2023-09-09T23:13:10.334992+0000","last_clean":"2023-09-09T23:13:10.334992+0000","last_became_active":"2023-09-09T23:13:10.334629+0000","last_became_peered":"2023-09-09T23:13:10.334629+0000","last_unstale":"2023-09-09T23:13:10.334992+0000","last_undegraded":"2023-09-09T23:13:10.334992+0000","last_fullsized":"2023-09-09T23:13:10.334992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:24:48.291826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":434,"ondisk_log_size":434,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69578,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":812,"num_read_kb":479,"num_write":238,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":90,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569344,"data_stored":204273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514523,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":434403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.65900000000000003}]},{"osd":1,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.747},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.77300000000000002}]},{"osd":2,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":4,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.88600000000000001}]},{"osd":5,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.79500000000000004}]},{"osd":6,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.74199999999999999}]},{"osd":7,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.92200000000000004}]},{"osd":8,"last update":"Sat Sep 9 23:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.72999999999999998}]},{"osd":9,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.93500000000000005}]},{"osd":10,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.873}]}]},{"osd":10,"up_from":69,"seq":296352743456,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1609728,"data_stored":1088114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.65400000000000003}]},{"osd":1,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.33200000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.68000000000000005,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.50700000000000001,"15min":0.51500000000000001},"min":{"1min":0.33700000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.80900000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.58999999999999997}]},{"osd":2,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.623,"15min":0.61499999999999999},"min":{"1min":0.55100000000000005,"5min":0.379,"15min":0.379},"max":{"1min":0.746,"5min":1.018,"15min":1.018},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.65000000000000002,"15min":0.65300000000000002},"min":{"1min":0.501,"5min":0.436,"15min":0.436},"max":{"1min":0.79400000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71999999999999997}]},{"osd":3,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.622,"15min":0.63700000000000001},"min":{"1min":0.41399999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.67400000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.64400000000000002,"15min":0.65900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.67200000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.746}]},{"osd":4,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60899999999999999,"15min":0.61299999999999999},"min":{"1min":0.42399999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.72199999999999998,"5min":1.024,"15min":1.024},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.36099999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79500000000000004,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.73299999999999998}]},{"osd":5,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.69799999999999995,"15min":0.70099999999999996},"min":{"1min":0.56000000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.85799999999999998,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.88},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71399999999999997,"15min":0.71599999999999997},"min":{"1min":0.47699999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88800000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.68899999999999995}]},{"osd":6,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.73099999999999998,"15min":0.75},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90000000000000002,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.73099999999999998,"15min":0.74099999999999999},"min":{"1min":0.54100000000000004,"5min":0.439,"15min":0.439},"max":{"1min":0.85199999999999998,"5min":1.038,"15min":1.038},"last":0.82599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":8,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.80200000000000005,"15min":0.81200000000000006},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.91500000000000004,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78500000000000003,"15min":0.78900000000000003},"min":{"1min":0.623,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.94099999999999995,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.84299999999999997}]},{"osd":9,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.81299999999999994,"15min":0.82399999999999995},"min":{"1min":0.61399999999999999,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.96799999999999997,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.80500000000000005,"15min":0.81999999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.878,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.79600000000000004}]},{"osd":11,"last update":"Sat Sep 9 23:12:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.95899999999999996}]}]},{"osd":9,"up_from":64,"seq":274877906979,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":434243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.435,"15min":0.443},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.73999999999999999,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.39200000000000002}]},{"osd":1,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.62},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56799999999999995,"15min":0.56899999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.60599999999999998}]},{"osd":2,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.626,"15min":0.63},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.625,"15min":0.63200000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.45700000000000002}]},{"osd":3,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.51300000000000001,"15min":0.52300000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.376}]},{"osd":4,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.629,"15min":0.63500000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96799999999999997,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.497},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.67400000000000004,"15min":0.68799999999999994},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81999999999999995,"5min":1.028,"15min":1.028},"last":0.59399999999999997}]},{"osd":5,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.73399999999999999,"15min":0.71499999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":5.6699999999999999,"5min":5.6699999999999999,"15min":5.6699999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.68200000000000005,"15min":0.69399999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.54300000000000004}]},{"osd":6,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.64300000000000002,"15min":0.65500000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.65600000000000003,"15min":0.67000000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.55400000000000005}]},{"osd":7,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.745,"15min":0.76100000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.82499999999999996,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.746,"15min":0.75800000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.66400000000000003}]},{"osd":8,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.73799999999999999,"15min":0.747},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.74099999999999999,"15min":0.755},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.85899999999999999,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.63300000000000001}]},{"osd":10,"last update":"Sat Sep 9 23:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73599999999999999,"15min":0.74199999999999999},"min":{"1min":0.56000000000000005,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.85899999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.88,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.84599999999999997}]}]},{"osd":8,"up_from":57,"seq":244813135913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":434599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.39400000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.87,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.38200000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78500000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.53900000000000003}]},{"osd":1,"last update":"Sat Sep 9 23:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.63200000000000001,"15min":0.60199999999999998},"min":{"1min":0.45600000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.89300000000000002,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60499999999999998,"15min":0.58899999999999997},"min":{"1min":0.36599999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82199999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.63700000000000001}]},{"osd":2,"last update":"Sat Sep 9 23:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.48599999999999999,"15min":0.48399999999999999},"min":{"1min":0.36299999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.78500000000000003,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.48499999999999999,"15min":0.48899999999999999},"min":{"1min":0.309,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69499999999999995,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.72399999999999998}]},{"osd":3,"last update":"Sat Sep 9 23:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57399999999999995,"15min":0.57899999999999996},"min":{"1min":0.497,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.755,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.53900000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.76100000000000001,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.48799999999999999}]},{"osd":4,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64100000000000001,"15min":0.63700000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94399999999999995,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.68999999999999995,"15min":0.68200000000000005},"min":{"1min":0.41899999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.73699999999999999}]},{"osd":5,"last update":"Sat Sep 9 23:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.34699999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.66900000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.47299999999999998}]},{"osd":6,"last update":"Sat Sep 9 23:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.72299999999999998,"15min":0.72599999999999998},"min":{"1min":0.56299999999999994,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86099999999999999,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68600000000000005,"15min":0.69599999999999995},"min":{"1min":0.56699999999999995,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85799999999999998,"5min":0.996,"15min":0.996},"last":0.498}]},{"osd":7,"last update":"Sat Sep 9 23:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72499999999999998,"15min":0.72599999999999998},"min":{"1min":0.52600000000000002,"5min":0.503,"15min":0.503},"max":{"1min":0.96499999999999997,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.89100000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.60799999999999998}]},{"osd":9,"last update":"Sat Sep 9 23:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71999999999999997,"15min":0.72099999999999997},"min":{"1min":0.48899999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.99399999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.77200000000000002,"15min":0.77000000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1060000000000001,"5min":1.115,"15min":1.115},"last":0.65400000000000003}]},{"osd":10,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.74299999999999999,"15min":0.73299999999999998},"min":{"1min":0.63400000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.97499999999999998,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.77500000000000002,"15min":0.75700000000000001},"min":{"1min":0.70699999999999996,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.621}]},{"osd":11,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.73699999999999999}]}]},{"osd":7,"up_from":50,"seq":214748364844,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1519616,"data_stored":1027485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.439},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.33100000000000002}]},{"osd":1,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.436,"15min":0.432},"min":{"1min":0.27800000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.437,"15min":0.42099999999999999},"min":{"1min":0.33200000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.59699999999999998,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.38700000000000001}]},{"osd":2,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63500000000000001,"15min":0.63900000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84399999999999997,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60299999999999998,"15min":0.60199999999999998},"min":{"1min":0.51200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.76600000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.53300000000000003}]},{"osd":3,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.42799999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82799999999999996,"5min":0.88,"15min":0.88},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.623,"15min":0.624},"min":{"1min":0.437,"5min":0.375,"15min":0.375},"max":{"1min":0.81100000000000005,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.56899999999999995}]},{"osd":4,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.35299999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.72599999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.35799999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.79700000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.42399999999999999}]},{"osd":5,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.69099999999999995,"15min":0.69499999999999995},"min":{"1min":0.54900000000000004,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.93999999999999995,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.503,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.88100000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58499999999999996}]},{"osd":6,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.72499999999999998,"15min":0.746},"min":{"1min":0.51500000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.86499999999999999,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.69999999999999996,"15min":0.71999999999999997},"min":{"1min":0.45900000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.753,"5min":1.254,"15min":1.254},"last":0.60899999999999999}]},{"osd":8,"last update":"Sat Sep 9 23:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73399999999999999,"15min":0.73899999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.86399999999999999,"5min":1.008,"15min":1.008},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.76400000000000001,"15min":0.77000000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.88,"5min":1.387,"15min":1.387},"last":0.73899999999999999}]},{"osd":9,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.75,"15min":0.76000000000000001},"min":{"1min":0.51700000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.89900000000000002,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.92400000000000004,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.753}]},{"osd":10,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.46899999999999997,"5min":0.442,"15min":0.442},"max":{"1min":0.95499999999999996,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.74299999999999999,"15min":0.745},"min":{"1min":0.58499999999999996,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.91500000000000004,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.72299999999999998}]},{"osd":11,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.70299999999999996}]}]},{"osd":6,"up_from":45,"seq":193273528369,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":433966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,8,0,0,1,27],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.377,"15min":0.36899999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":5.6859999999999999},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.40400000000000003,"15min":0.40200000000000002},"min":{"1min":0.26800000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.53700000000000003,"5min":0.753,"15min":0.753},"last":1.925}]},{"osd":1,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78100000000000003,"5min":0.877,"15min":0.877},"last":8.6560000000000006},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77200000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":14.58}]},{"osd":2,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.41699999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79600000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":1.798},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76800000000000002,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":4.298}]},{"osd":3,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.43099999999999999,"15min":0.433},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70199999999999996,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":1.181},{"interface":"front","average":{"1min":0.437,"5min":0.501,"15min":0.51200000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.59399999999999997,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":3.1949999999999998}]},{"osd":4,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60699999999999998,"15min":0.60299999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.78100000000000003,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":2.3620000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.77900000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":2.0139999999999998}]},{"osd":5,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.63200000000000001,"15min":0.64000000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.75600000000000001,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.2150000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.65900000000000003,"15min":0.65700000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82199999999999995,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":8.7859999999999996}]},{"osd":7,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.67700000000000005,"15min":0.68000000000000005},"min":{"1min":0.45000000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":6.766},{"interface":"front","average":{"1min":0.626,"5min":0.65800000000000003,"15min":0.66000000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.91500000000000004,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":9.1010000000000009}]},{"osd":8,"last update":"Sat Sep 9 23:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.76700000000000002,"15min":0.78800000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85099999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":1.405},{"interface":"front","average":{"1min":0.628,"5min":0.74399999999999999,"15min":0.76400000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.80100000000000005,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.556}]},{"osd":9,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.64500000000000002,"15min":0.66100000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70799999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":2.4569999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.72199999999999998,"15min":0.752},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73099999999999998,"5min":1.534,"15min":1.534},"last":1.7809999999999999}]},{"osd":10,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.72099999999999997,"15min":0.73099999999999998},"min":{"1min":0.51300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.95099999999999996,"5min":1.163,"15min":1.163},"last":1.522},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.73999999999999999,"15min":0.749},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.84499999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":1.2549999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":491729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.501},"min":{"1min":0.35999999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.88600000000000001,"5min":1.085,"15min":1.085},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52800000000000002,"15min":0.495},"min":{"1min":0.39900000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.96999999999999997,"5min":1.042,"15min":1.042},"last":0.755}]},{"osd":2,"last update":"Sat Sep 9 23:13:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57799999999999996,"15min":0.52000000000000002},"min":{"1min":0.438,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78200000000000003,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.60399999999999998,"15min":0.53900000000000003},"min":{"1min":0.443,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.86599999999999999,"5min":1.133,"15min":1.133},"last":0.54800000000000004}]},{"osd":3,"last update":"Sat Sep 9 23:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.39600000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69699999999999995,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.56999999999999995},"min":{"1min":0.41099999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79500000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.73099999999999998}]},{"osd":4,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.46700000000000003,"15min":0.42399999999999999},"min":{"1min":0.34300000000000003,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.74099999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.53200000000000003,"15min":0.46400000000000002},"min":{"1min":0.498,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.85499999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.501}]},{"osd":5,"last update":"Sat Sep 9 23:13:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.60899999999999999,"15min":0.56999999999999995},"min":{"1min":0.496,"5min":0.377,"15min":0.377},"max":{"1min":0.872,"5min":1.022,"15min":1.022},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.64200000000000002,"15min":0.58699999999999997},"min":{"1min":0.499,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0129999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.66000000000000003}]},{"osd":6,"last update":"Sat Sep 9 23:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59999999999999998,"15min":0.56999999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89600000000000002,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36099999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.68799999999999994,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.70299999999999996}]},{"osd":7,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.57199999999999995,"15min":0.52800000000000002},"min":{"1min":0.36599999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.55300000000000005,"15min":0.52800000000000002},"min":{"1min":0.42099999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80200000000000005,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.46899999999999997}]},{"osd":8,"last update":"Sat Sep 9 23:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.66200000000000003,"15min":0.65600000000000003},"min":{"1min":0.441,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.69899999999999995,"15min":0.69499999999999995},"min":{"1min":0.502,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.95499999999999996,"5min":1.218,"15min":1.218},"last":0.68899999999999995}]},{"osd":9,"last update":"Sat Sep 9 23:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.76200000000000001,"15min":0.76700000000000002},"min":{"1min":0.53700000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93100000000000005,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.70599999999999996,"15min":0.71199999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.82299999999999995,"5min":1.528,"15min":1.528},"last":0.65200000000000002}]},{"osd":10,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.47299999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.89000000000000001,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.79100000000000004,"15min":0.78300000000000003},"min":{"1min":0.55300000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0009999999999999,"5min":1.355,"15min":1.355},"last":0.622}]},{"osd":11,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.78900000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":435299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 9 23:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53900000000000003,"15min":0.51300000000000001},"min":{"1min":0.39200000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.70299999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.33400000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68200000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.626}]},{"osd":2,"last update":"Sat Sep 9 23:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53800000000000003,"15min":0.52900000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.70599999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55100000000000005,"15min":0.52900000000000003},"min":{"1min":0.42299999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.70499999999999996,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.59299999999999997}]},{"osd":3,"last update":"Sat Sep 9 23:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.45700000000000002}]},{"osd":4,"last update":"Sat Sep 9 23:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55100000000000005,"15min":0.53300000000000003},"min":{"1min":0.375,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73399999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.38300000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.67000000000000004,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.52300000000000002}]},{"osd":5,"last update":"Sat Sep 9 23:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71099999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56799999999999995,"15min":0.56499999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.752,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.67100000000000004}]},{"osd":6,"last update":"Sat Sep 9 23:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.495,"15min":0.497},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.55300000000000005,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.504,"15min":0.501},"min":{"1min":0.33000000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67300000000000004,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.64700000000000002}]},{"osd":7,"last update":"Sat Sep 9 23:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.47599999999999998}]},{"osd":8,"last update":"Sat Sep 9 23:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.68700000000000006,"15min":0.69899999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.78000000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.68200000000000005,"15min":0.68999999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85699999999999998,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.71699999999999997}]},{"osd":9,"last update":"Sat Sep 9 23:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.628,"15min":0.65400000000000003},"min":{"1min":0.36799999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.60999999999999999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.67300000000000004,"15min":0.69799999999999995},"min":{"1min":0.34699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70599999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.63600000000000001}]},{"osd":10,"last update":"Sat Sep 9 23:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68200000000000005,"15min":0.68999999999999995},"min":{"1min":0.46800000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81699999999999995,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.67800000000000005,"15min":0.69499999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.71299999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.60599999999999998}]},{"osd":11,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.73399999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":492730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.44,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.92100000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52300000000000002,"15min":0.51700000000000002},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.63200000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.47999999999999998}]},{"osd":1,"last update":"Sat Sep 9 23:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.59899999999999998,"15min":0.55100000000000005},"min":{"1min":0.38700000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.60499999999999998,"15min":0.53400000000000003},"min":{"1min":0.53300000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.93100000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.86399999999999999}]},{"osd":3,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59999999999999998,"15min":0.60699999999999998},"min":{"1min":0.36199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0009999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61099999999999999,"15min":0.61599999999999999},"min":{"1min":0.46300000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90700000000000003,"5min":1.478,"15min":1.478},"last":0.58099999999999996}]},{"osd":4,"last update":"Sat Sep 9 23:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.89900000000000002}]},{"osd":5,"last update":"Sat Sep 9 23:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.45900000000000002,"15min":0.41599999999999998},"min":{"1min":0.35799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.96899999999999997,"5min":1.014,"15min":1.014},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.46999999999999997,"15min":0.432},"min":{"1min":0.29899999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76900000000000002,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.65800000000000003}]},{"osd":6,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.56999999999999995,"15min":0.53200000000000003},"min":{"1min":0.47799999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.749,"5min":0.63300000000000001,"15min":0.59899999999999998},"min":{"1min":0.48799999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.69699999999999995}]},{"osd":7,"last update":"Sat Sep 9 23:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.65700000000000003}]},{"osd":8,"last update":"Sat Sep 9 23:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.626,"15min":0.59499999999999997},"min":{"1min":0.442,"5min":0.37,"15min":0.37},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.63300000000000001,"15min":0.60999999999999999},"min":{"1min":0.56100000000000005,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.878,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.75700000000000001}]},{"osd":9,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.70499999999999996,"15min":0.69099999999999995},"min":{"1min":0.52100000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.752,"15min":0.73699999999999999},"min":{"1min":0.59799999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.73799999999999999}]},{"osd":10,"last update":"Sat Sep 9 23:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.68100000000000005,"15min":0.66700000000000004},"min":{"1min":0.59099999999999997,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.75900000000000001,"15min":0.74099999999999999},"min":{"1min":0.61099999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.873}]},{"osd":11,"last update":"Sat Sep 9 23:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.92300000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182461,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":433466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.32700000000000001,"15min":0.313},"min":{"1min":0.22800000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.501,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35199999999999998,"15min":0.33400000000000002},"min":{"1min":0.249,"5min":0.222,"15min":0.222},"max":{"1min":0.53800000000000003,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.25700000000000001}]},{"osd":1,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54500000000000004,"15min":0.51900000000000002},"min":{"1min":0.32900000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75900000000000001,"5min":1.091,"15min":1.091},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.51000000000000001,"15min":0.51400000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.60799999999999998,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.72699999999999998}]},{"osd":2,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54400000000000004,"15min":0.52000000000000002},"min":{"1min":0.36099999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65100000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53800000000000003,"15min":0.51800000000000002},"min":{"1min":0.40300000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.66200000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.39200000000000002}]},{"osd":4,"last update":"Sat Sep 9 23:13:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54600000000000004,"15min":0.50800000000000001},"min":{"1min":0.39300000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.73599999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.436},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54800000000000004,"15min":0.52000000000000002},"min":{"1min":0.40600000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.65400000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.61099999999999999}]},{"osd":5,"last update":"Sat Sep 9 23:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56499999999999995,"15min":0.57899999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.64200000000000002,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.61199999999999999,"15min":0.60399999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.69499999999999995,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.47099999999999997}]},{"osd":6,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.49099999999999999,"15min":0.49199999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73399999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.377},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.50900000000000001,"15min":0.50600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64800000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.36099999999999999}]},{"osd":7,"last update":"Sat Sep 9 23:13:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.65000000000000002,"15min":0.65900000000000003},"min":{"1min":0.42699999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79000000000000004,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.67500000000000004,"15min":0.69199999999999995},"min":{"1min":0.44800000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.77800000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.63700000000000001}]},{"osd":8,"last update":"Sat Sep 9 23:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.49199999999999999}]},{"osd":9,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.66500000000000004,"15min":0.68899999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77500000000000002,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.67700000000000005,"15min":0.69899999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.746,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.497}]},{"osd":10,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.42799999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.66200000000000003,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.627},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.71299999999999997,"15min":0.72899999999999998},"min":{"1min":0.505,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.76700000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.68700000000000006}]},{"osd":11,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.55400000000000005}]}]},{"osd":4,"up_from":32,"seq":137438953529,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":434211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.503,"15min":0.47099999999999997},"min":{"1min":0.44900000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.52200000000000002,"15min":0.48799999999999999},"min":{"1min":0.41999999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.33300000000000002}]},{"osd":1,"last update":"Sat Sep 9 23:13:14 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.38,"15min":0.34499999999999997},"min":{"1min":0.313,"5min":0.248,"15min":0.248},"max":{"1min":0.60999999999999999,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.41399999999999998,"15min":0.36799999999999999},"min":{"1min":0.36899999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.46700000000000003}]},{"osd":2,"last update":"Sat Sep 9 23:13:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.55800000000000005,"15min":0.52400000000000002},"min":{"1min":0.433,"5min":0.313,"15min":0.313},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.57799999999999996,"15min":0.53200000000000003},"min":{"1min":0.47799999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.55600000000000005}]},{"osd":3,"last update":"Sat Sep 9 23:13:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.55200000000000005,"15min":0.496},"min":{"1min":0.41999999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78000000000000003,"5min":1.008,"15min":1.008},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53400000000000003,"15min":0.499},"min":{"1min":0.38900000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.91100000000000003,"5min":1.002,"15min":1.002},"last":0.497}]},{"osd":5,"last update":"Sat Sep 9 23:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.60599999999999998,"15min":0.58199999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.874,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.627,"15min":0.57099999999999995},"min":{"1min":0.432,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92600000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.72399999999999998}]},{"osd":6,"last update":"Sat Sep 9 23:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64800000000000002,"15min":0.61799999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.85899999999999999,"5min":1.226,"15min":1.226},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.65600000000000003,"15min":0.62},"min":{"1min":0.40000000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.5219999999999998,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":0.68000000000000005}]},{"osd":7,"last update":"Sat Sep 9 23:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.42299999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.88,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.36399999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.754,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.65900000000000003}]},{"osd":8,"last update":"Sat Sep 9 23:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69199999999999995,"15min":0.68899999999999995},"min":{"1min":0.45700000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.749,"15min":0.751},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.89500000000000002,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.82799999999999996}]},{"osd":9,"last update":"Sat Sep 9 23:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80700000000000005,"15min":0.81200000000000006},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.753},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.75800000000000001,"15min":0.76600000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.92800000000000005,"5min":1.127,"15min":1.127},"last":0.76400000000000001}]},{"osd":10,"last update":"Sat Sep 9 23:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.748,"15min":0.751},"min":{"1min":0.53600000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90200000000000002,"5min":1.208,"15min":1.208},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.66800000000000004,"15min":0.66700000000000004},"min":{"1min":0.48799999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.80900000000000005,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.79100000000000004}]},{"osd":11,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.85699999999999998}]}]},{"osd":5,"up_from":37,"seq":158913790004,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1536000,"data_stored":1026963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.48399999999999999,"15min":0.46300000000000002},"min":{"1min":0.40500000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.502,"15min":0.49299999999999999},"min":{"1min":0.377,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.58499999999999996}]},{"osd":1,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.52600000000000002,"15min":0.50800000000000001},"min":{"1min":0.39000000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.53300000000000003,"15min":0.51500000000000001},"min":{"1min":0.36899999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.76500000000000001}]},{"osd":2,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.38400000000000001,"15min":0.372},"min":{"1min":0.27700000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.66500000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.433},{"interface":"front","average":{"1min":0.432,"5min":0.39400000000000002,"15min":0.38800000000000001},"min":{"1min":0.309,"5min":0.248,"15min":0.248},"max":{"1min":0.64100000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.48199999999999998}]},{"osd":3,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.38600000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.75,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.54600000000000004,"15min":0.52400000000000002},"min":{"1min":0.41799999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86599999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.66900000000000004}]},{"osd":4,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.55900000000000005,"15min":0.53300000000000003},"min":{"1min":0.40799999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.46700000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.74199999999999999}]},{"osd":6,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56000000000000005,"15min":0.54600000000000004},"min":{"1min":0.38300000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88300000000000001,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.437,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92200000000000004,"5min":0.998,"15min":0.998},"last":0.627}]},{"osd":7,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.627,"15min":0.59899999999999998},"min":{"1min":0.57299999999999995,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.96799999999999997,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.63100000000000001,"15min":0.61299999999999999},"min":{"1min":0.55400000000000005,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.79000000000000004}]},{"osd":8,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57199999999999995,"15min":0.56399999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83799999999999997,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55200000000000005,"15min":0.54600000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.75600000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.65400000000000003}]},{"osd":9,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.53700000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.82099999999999995,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.70599999999999996,"15min":0.70299999999999996},"min":{"1min":0.56999999999999995,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93899999999999995,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.84099999999999997}]},{"osd":10,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.71899999999999997,"15min":0.70199999999999996},"min":{"1min":0.72199999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.68999999999999995,"15min":0.67500000000000004},"min":{"1min":0.61199999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.82799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-09T23:13:43.950 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-09-09T23:13:43.951 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-09T23:13:44.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.103+0000 7ff88a654700 1 -- 172.21.15.82:0/3546003465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff884106310 msgr2=0x7ff884108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:44.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.103+0000 7ff88a654700 1 --2- 172.21.15.82:0/3546003465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff884106310 0x7ff884108740 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff87c00b0a0 tx=0x7ff87c00b3b0 comp rx=0 tx=0).stop 2023-09-09T23:13:44.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.103+0000 7ff88a654700 1 -- 172.21.15.82:0/3546003465 shutdown_connections 2023-09-09T23:13:44.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.103+0000 7ff88a654700 1 --2- 172.21.15.82:0/3546003465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff884106310 0x7ff884108740 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:44.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.103+0000 7ff88a654700 1 --2- 172.21.15.82:0/3546003465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8841010b0 0x7ff8841034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:44.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.103+0000 7ff88a654700 1 --2- 172.21.15.82:0/3546003465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8841039e0 0x7ff884105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:44.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.103+0000 7ff88a654700 1 -- 172.21.15.82:0/3546003465 >> 172.21.15.82:0/3546003465 conn(0x7ff8840faa20 msgr2=0x7ff8840fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:44.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.103+0000 7ff88a654700 1 -- 172.21.15.82:0/3546003465 shutdown_connections 2023-09-09T23:13:44.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.103+0000 7ff88a654700 1 -- 172.21.15.82:0/3546003465 wait complete. 2023-09-09T23:13:44.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.104+0000 7ff88a654700 1 Processor -- start 2023-09-09T23:13:44.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.104+0000 7ff88a654700 1 -- start start 2023-09-09T23:13:44.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.104+0000 7ff88a654700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8841010b0 0x7ff884196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:44.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.104+0000 7ff88a654700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8841039e0 0x7ff8841968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:44.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.104+0000 7ff88a654700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff884106310 0x7ff88419d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:44.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.104+0000 7ff88a654700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88419df10 con 0x7ff8841039e0 2023-09-09T23:13:44.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.104+0000 7ff88a654700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88419e050 con 0x7ff8841010b0 2023-09-09T23:13:44.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.104+0000 7ff88a654700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88419e1e0 con 0x7ff884106310 2023-09-09T23:13:44.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.104+0000 7ff87bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8841039e0 0x7ff8841968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:44.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.104+0000 7ff87bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8841039e0 0x7ff8841968d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56114/0 (socket says 172.21.15.82:56114) 2023-09-09T23:13:44.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.104+0000 7ff87bfff700 1 -- 172.21.15.82:0/3454797324 learned_addr learned my addr 172.21.15.82:0/3454797324 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:44.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.104+0000 7ff888bf1700 1 --2- 172.21.15.82:0/3454797324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff884106310 0x7ff88419d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:44.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.104+0000 7ff883fff700 1 --2- 172.21.15.82:0/3454797324 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8841010b0 0x7ff884196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:44.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.104+0000 7ff87bfff700 1 -- 172.21.15.82:0/3454797324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff884106310 msgr2=0x7ff88419d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:44.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.104+0000 7ff87bfff700 1 --2- 172.21.15.82:0/3454797324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff884106310 0x7ff88419d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:44.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.104+0000 7ff87bfff700 1 -- 172.21.15.82:0/3454797324 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8841010b0 msgr2=0x7ff884196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:44.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.104+0000 7ff87bfff700 1 --2- 172.21.15.82:0/3454797324 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8841010b0 0x7ff884196390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:44.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.104+0000 7ff87bfff700 1 -- 172.21.15.82:0/3454797324 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87c009d00 con 0x7ff8841039e0 2023-09-09T23:13:44.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.104+0000 7ff87bfff700 1 --2- 172.21.15.82:0/3454797324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8841039e0 0x7ff8841968d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00e9a0 tx=0x7ff86c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:44.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.105+0000 7ff881ffb700 1 -- 172.21.15.82:0/3454797324 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c009790 con 0x7ff8841039e0 2023-09-09T23:13:44.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.105+0000 7ff881ffb700 1 -- 172.21.15.82:0/3454797324 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff86c00cbd0 con 0x7ff8841039e0 2023-09-09T23:13:44.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.105+0000 7ff88a654700 1 -- 172.21.15.82:0/3454797324 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff88419e4c0 con 0x7ff8841039e0 2023-09-09T23:13:44.110 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.105+0000 7ff881ffb700 1 -- 172.21.15.82:0/3454797324 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c018680 con 0x7ff8841039e0 2023-09-09T23:13:44.110 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.105+0000 7ff88a654700 1 -- 172.21.15.82:0/3454797324 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff88419e9d0 con 0x7ff8841039e0 2023-09-09T23:13:44.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.106+0000 7ff88a654700 1 -- 172.21.15.82:0/3454797324 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8840fc5a0 con 0x7ff8841039e0 2023-09-09T23:13:44.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.107+0000 7ff881ffb700 1 -- 172.21.15.82:0/3454797324 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7ff86c00cd40 con 0x7ff8841039e0 2023-09-09T23:13:44.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.107+0000 7ff881ffb700 1 --2- 172.21.15.82:0/3454797324 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff8700a0690 0x7ff8700a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:44.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.107+0000 7ff883fff700 1 --2- 172.21.15.82:0/3454797324 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff8700a0690 0x7ff8700a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:44.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.107+0000 7ff881ffb700 1 -- 172.21.15.82:0/3454797324 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff86c014070 con 0x7ff8841039e0 2023-09-09T23:13:44.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.108+0000 7ff883fff700 1 --2- 172.21.15.82:0/3454797324 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff8700a0690 0x7ff8700a2b50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff87400bbd0 tx=0x7ff87400b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:44.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.111+0000 7ff881ffb700 1 -- 172.21.15.82:0/3454797324 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff86c095e80 con 0x7ff8841039e0 2023-09-09T23:13:44.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.298+0000 7ff88a654700 1 -- 172.21.15.82:0/3454797324 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff88402cfa0 con 0x7ff8700a0690 2023-09-09T23:13:44.306 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.308+0000 7ff881ffb700 1 -- 172.21.15.82:0/3454797324 <== mgr.14152 v2:172.21.15.82:6800/1453069857 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344353 (secure 0 0 0) 0x7ff88402cfa0 con 0x7ff8700a0690 2023-09-09T23:13:44.314 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:44.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.312+0000 7ff88a654700 1 -- 172.21.15.82:0/3454797324 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff8700a0690 msgr2=0x7ff8700a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:44.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.312+0000 7ff88a654700 1 --2- 172.21.15.82:0/3454797324 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff8700a0690 0x7ff8700a2b50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff87400bbd0 tx=0x7ff87400b5b0 comp rx=0 tx=0).stop 2023-09-09T23:13:44.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.312+0000 7ff88a654700 1 -- 172.21.15.82:0/3454797324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8841039e0 msgr2=0x7ff8841968d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:44.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.312+0000 7ff88a654700 1 --2- 172.21.15.82:0/3454797324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8841039e0 0x7ff8841968d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00e9a0 tx=0x7ff86c00ed60 comp rx=0 tx=0).stop 2023-09-09T23:13:44.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.312+0000 7ff88a654700 1 -- 172.21.15.82:0/3454797324 shutdown_connections 2023-09-09T23:13:44.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.312+0000 7ff88a654700 1 --2- 172.21.15.82:0/3454797324 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff8700a0690 0x7ff8700a2b50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:44.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.312+0000 7ff88a654700 1 --2- 172.21.15.82:0/3454797324 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8841010b0 0x7ff884196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:44.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.312+0000 7ff88a654700 1 --2- 172.21.15.82:0/3454797324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff884106310 0x7ff88419d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:44.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.312+0000 7ff88a654700 1 --2- 172.21.15.82:0/3454797324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8841039e0 0x7ff8841968d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:44.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.312+0000 7ff88a654700 1 -- 172.21.15.82:0/3454797324 >> 172.21.15.82:0/3454797324 conn(0x7ff8840faa20 msgr2=0x7ff884101d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:44.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.312+0000 7ff88a654700 1 -- 172.21.15.82:0/3454797324 shutdown_connections 2023-09-09T23:13:44.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.312+0000 7ff88a654700 1 -- 172.21.15.82:0/3454797324 wait complete. 2023-09-09T23:13:44.323 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-09-09T23:13:44.324 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2023-09-09T23:13:43.000017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660192,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":948,"num_read_kb":598,"num_write":421,"num_write_kb":3078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180826,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":781,"ondisk_log_size":781,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336984,"kb_used_data":12768,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124604840,"statfs":{"total":1151940427776,"available":1151595356160,"internally_reserved":0,"allocated":13074432,"data_stored":7167208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,8,0,0,1,27],"upper_bound":64},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6087,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":72,"num_write":8,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004132"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.327081+0000","last_change":"2023-09-09T23:12:17.903811+0000","last_active":"2023-09-09T23:13:09.327081+0000","last_peered":"2023-09-09T23:13:09.327081+0000","last_clean":"2023-09-09T23:13:09.327081+0000","last_became_active":"2023-09-09T23:12:17.903386+0000","last_became_peered":"2023-09-09T23:12:17.903386+0000","last_unstale":"2023-09-09T23:13:09.327081+0000","last_undegraded":"2023-09-09T23:13:09.327081+0000","last_fullsized":"2023-09-09T23:13:09.327081+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:36:45.122068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.016086+0000","last_change":"2023-09-09T23:12:17.897915+0000","last_active":"2023-09-09T23:13:39.016086+0000","last_peered":"2023-09-09T23:13:39.016086+0000","last_clean":"2023-09-09T23:13:39.016086+0000","last_became_active":"2023-09-09T23:12:17.896727+0000","last_became_peered":"2023-09-09T23:12:17.896727+0000","last_unstale":"2023-09-09T23:13:39.016086+0000","last_undegraded":"2023-09-09T23:13:39.016086+0000","last_fullsized":"2023-09-09T23:13:39.016086+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:36:03.053572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.020527+0000","last_change":"2023-09-09T23:12:17.907957+0000","last_active":"2023-09-09T23:13:39.020527+0000","last_peered":"2023-09-09T23:13:39.020527+0000","last_clean":"2023-09-09T23:13:39.020527+0000","last_became_active":"2023-09-09T23:12:17.907625+0000","last_became_peered":"2023-09-09T23:12:17.907625+0000","last_unstale":"2023-09-09T23:13:39.020527+0000","last_undegraded":"2023-09-09T23:13:39.020527+0000","last_fullsized":"2023-09-09T23:13:39.020527+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:26:56.520987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.327490+0000","last_change":"2023-09-09T23:12:17.903489+0000","last_active":"2023-09-09T23:13:09.327490+0000","last_peered":"2023-09-09T23:13:09.327490+0000","last_clean":"2023-09-09T23:13:09.327490+0000","last_became_active":"2023-09-09T23:12:17.901184+0000","last_became_peered":"2023-09-09T23:12:17.901184+0000","last_unstale":"2023-09-09T23:13:09.327490+0000","last_undegraded":"2023-09-09T23:13:09.327490+0000","last_fullsized":"2023-09-09T23:13:09.327490+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:42:01.899998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.007034+0000","last_change":"2023-09-09T23:12:17.899487+0000","last_active":"2023-09-09T23:13:39.007034+0000","last_peered":"2023-09-09T23:13:39.007034+0000","last_clean":"2023-09-09T23:13:39.007034+0000","last_became_active":"2023-09-09T23:12:17.899083+0000","last_became_peered":"2023-09-09T23:12:17.899083+0000","last_unstale":"2023-09-09T23:13:39.007034+0000","last_undegraded":"2023-09-09T23:13:39.007034+0000","last_fullsized":"2023-09-09T23:13:39.007034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:15:20.109849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.006501+0000","last_change":"2023-09-09T23:12:17.901554+0000","last_active":"2023-09-09T23:13:39.006501+0000","last_peered":"2023-09-09T23:13:39.006501+0000","last_clean":"2023-09-09T23:13:39.006501+0000","last_became_active":"2023-09-09T23:12:17.901246+0000","last_became_peered":"2023-09-09T23:12:17.901246+0000","last_unstale":"2023-09-09T23:13:39.006501+0000","last_undegraded":"2023-09-09T23:13:39.006501+0000","last_fullsized":"2023-09-09T23:13:39.006501+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:19:08.795927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.016998+0000","last_change":"2023-09-09T23:12:17.906989+0000","last_active":"2023-09-09T23:13:39.016998+0000","last_peered":"2023-09-09T23:13:39.016998+0000","last_clean":"2023-09-09T23:13:39.016998+0000","last_became_active":"2023-09-09T23:12:17.906704+0000","last_became_peered":"2023-09-09T23:12:17.906704+0000","last_unstale":"2023-09-09T23:13:39.016998+0000","last_undegraded":"2023-09-09T23:13:39.016998+0000","last_fullsized":"2023-09-09T23:13:39.016998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:41:33.612168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.015516+0000","last_change":"2023-09-09T23:12:17.906269+0000","last_active":"2023-09-09T23:13:39.015516+0000","last_peered":"2023-09-09T23:13:39.015516+0000","last_clean":"2023-09-09T23:13:39.015516+0000","last_became_active":"2023-09-09T23:12:17.906017+0000","last_became_peered":"2023-09-09T23:12:17.906017+0000","last_unstale":"2023-09-09T23:13:39.015516+0000","last_undegraded":"2023-09-09T23:13:39.015516+0000","last_fullsized":"2023-09-09T23:13:39.015516+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:50:21.893506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.020152+0000","last_change":"2023-09-09T23:12:17.905191+0000","last_active":"2023-09-09T23:13:39.020152+0000","last_peered":"2023-09-09T23:13:39.020152+0000","last_clean":"2023-09-09T23:13:39.020152+0000","last_became_active":"2023-09-09T23:12:17.904813+0000","last_became_peered":"2023-09-09T23:12:17.904813+0000","last_unstale":"2023-09-09T23:13:39.020152+0000","last_undegraded":"2023-09-09T23:13:39.020152+0000","last_fullsized":"2023-09-09T23:13:39.020152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:29:54.722210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.008289+0000","last_change":"2023-09-09T23:12:17.899980+0000","last_active":"2023-09-09T23:13:39.008289+0000","last_peered":"2023-09-09T23:13:39.008289+0000","last_clean":"2023-09-09T23:13:39.008289+0000","last_became_active":"2023-09-09T23:12:17.899692+0000","last_became_peered":"2023-09-09T23:12:17.899692+0000","last_unstale":"2023-09-09T23:13:39.008289+0000","last_undegraded":"2023-09-09T23:13:39.008289+0000","last_fullsized":"2023-09-09T23:13:39.008289+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:25:28.056684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.021313+0000","last_change":"2023-09-09T23:12:17.907010+0000","last_active":"2023-09-09T23:13:39.021313+0000","last_peered":"2023-09-09T23:13:39.021313+0000","last_clean":"2023-09-09T23:13:39.021313+0000","last_became_active":"2023-09-09T23:12:17.906565+0000","last_became_peered":"2023-09-09T23:12:17.906565+0000","last_unstale":"2023-09-09T23:13:39.021313+0000","last_undegraded":"2023-09-09T23:13:39.021313+0000","last_fullsized":"2023-09-09T23:13:39.021313+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:08:03.733608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.327384+0000","last_change":"2023-09-09T23:12:17.902889+0000","last_active":"2023-09-09T23:13:09.327384+0000","last_peered":"2023-09-09T23:13:09.327384+0000","last_clean":"2023-09-09T23:13:09.327384+0000","last_became_active":"2023-09-09T23:12:17.900747+0000","last_became_peered":"2023-09-09T23:12:17.900747+0000","last_unstale":"2023-09-09T23:13:09.327384+0000","last_undegraded":"2023-09-09T23:13:09.327384+0000","last_fullsized":"2023-09-09T23:13:09.327384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:01:29.195275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.006815+0000","last_change":"2023-09-09T23:13:10.338443+0000","last_active":"2023-09-09T23:13:39.006815+0000","last_peered":"2023-09-09T23:13:39.006815+0000","last_clean":"2023-09-09T23:13:39.006815+0000","last_became_active":"2023-09-09T23:13:10.338011+0000","last_became_peered":"2023-09-09T23:13:10.338011+0000","last_unstale":"2023-09-09T23:13:39.006815+0000","last_undegraded":"2023-09-09T23:13:39.006815+0000","last_fullsized":"2023-09-09T23:13:39.006815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:27:24.646344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.013765+0000","last_change":"2023-09-09T23:12:17.899255+0000","last_active":"2023-09-09T23:13:39.013765+0000","last_peered":"2023-09-09T23:13:39.013765+0000","last_clean":"2023-09-09T23:13:39.013765+0000","last_became_active":"2023-09-09T23:12:17.898932+0000","last_became_peered":"2023-09-09T23:12:17.898932+0000","last_unstale":"2023-09-09T23:13:39.013765+0000","last_undegraded":"2023-09-09T23:13:39.013765+0000","last_fullsized":"2023-09-09T23:13:39.013765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:10:56.864588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.015679+0000","last_change":"2023-09-09T23:12:17.901083+0000","last_active":"2023-09-09T23:13:39.015679+0000","last_peered":"2023-09-09T23:13:39.015679+0000","last_clean":"2023-09-09T23:13:39.015679+0000","last_became_active":"2023-09-09T23:12:17.900744+0000","last_became_peered":"2023-09-09T23:12:17.900744+0000","last_unstale":"2023-09-09T23:13:39.015679+0000","last_undegraded":"2023-09-09T23:13:39.015679+0000","last_fullsized":"2023-09-09T23:13:39.015679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:37:11.073786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.022442+0000","last_change":"2023-09-09T23:12:17.901962+0000","last_active":"2023-09-09T23:13:39.022442+0000","last_peered":"2023-09-09T23:13:39.022442+0000","last_clean":"2023-09-09T23:13:39.022442+0000","last_became_active":"2023-09-09T23:12:17.901638+0000","last_became_peered":"2023-09-09T23:12:17.901638+0000","last_unstale":"2023-09-09T23:13:39.022442+0000","last_undegraded":"2023-09-09T23:13:39.022442+0000","last_fullsized":"2023-09-09T23:13:39.022442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:34:52.083262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.327117+0000","last_change":"2023-09-09T23:12:17.904424+0000","last_active":"2023-09-09T23:13:09.327117+0000","last_peered":"2023-09-09T23:13:09.327117+0000","last_clean":"2023-09-09T23:13:09.327117+0000","last_became_active":"2023-09-09T23:12:17.903879+0000","last_became_peered":"2023-09-09T23:12:17.903879+0000","last_unstale":"2023-09-09T23:13:09.327117+0000","last_undegraded":"2023-09-09T23:13:09.327117+0000","last_fullsized":"2023-09-09T23:13:09.327117+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:56:42.468630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.020503+0000","last_change":"2023-09-09T23:12:17.905065+0000","last_active":"2023-09-09T23:13:39.020503+0000","last_peered":"2023-09-09T23:13:39.020503+0000","last_clean":"2023-09-09T23:13:39.020503+0000","last_became_active":"2023-09-09T23:12:17.904646+0000","last_became_peered":"2023-09-09T23:12:17.904646+0000","last_unstale":"2023-09-09T23:13:39.020503+0000","last_undegraded":"2023-09-09T23:13:39.020503+0000","last_fullsized":"2023-09-09T23:13:39.020503+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:11:14.705884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.007659+0000","last_change":"2023-09-09T23:12:17.899751+0000","last_active":"2023-09-09T23:13:39.007659+0000","last_peered":"2023-09-09T23:13:39.007659+0000","last_clean":"2023-09-09T23:13:39.007659+0000","last_became_active":"2023-09-09T23:12:17.899410+0000","last_became_peered":"2023-09-09T23:12:17.899410+0000","last_unstale":"2023-09-09T23:13:39.007659+0000","last_undegraded":"2023-09-09T23:13:39.007659+0000","last_fullsized":"2023-09-09T23:13:39.007659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:23:06.098276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":105,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.421559+0000","last_change":"2023-09-09T23:12:17.175671+0000","last_active":"2023-09-09T23:13:39.421559+0000","last_peered":"2023-09-09T23:13:39.421559+0000","last_clean":"2023-09-09T23:13:39.421559+0000","last_became_active":"2023-09-09T23:12:17.175327+0000","last_became_peered":"2023-09-09T23:12:17.175327+0000","last_unstale":"2023-09-09T23:13:39.421559+0000","last_undegraded":"2023-09-09T23:13:39.421559+0000","last_fullsized":"2023-09-09T23:13:39.421559+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:38:57.724260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.008565+0000","last_change":"2023-09-09T23:12:17.906338+0000","last_active":"2023-09-09T23:13:39.008565+0000","last_peered":"2023-09-09T23:13:39.008565+0000","last_clean":"2023-09-09T23:13:39.008565+0000","last_became_active":"2023-09-09T23:12:17.906094+0000","last_became_peered":"2023-09-09T23:12:17.906094+0000","last_unstale":"2023-09-09T23:13:39.008565+0000","last_undegraded":"2023-09-09T23:13:39.008565+0000","last_fullsized":"2023-09-09T23:13:39.008565+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:43:18.183144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.329391+0000","last_change":"2023-09-09T23:12:17.167763+0000","last_active":"2023-09-09T23:13:09.329391+0000","last_peered":"2023-09-09T23:13:09.329391+0000","last_clean":"2023-09-09T23:13:09.329391+0000","last_became_active":"2023-09-09T23:12:17.167455+0000","last_became_peered":"2023-09-09T23:12:17.167455+0000","last_unstale":"2023-09-09T23:13:09.329391+0000","last_undegraded":"2023-09-09T23:13:09.329391+0000","last_fullsized":"2023-09-09T23:13:09.329391+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:15:23.415792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.020222+0000","last_change":"2023-09-09T23:12:17.907936+0000","last_active":"2023-09-09T23:13:39.020222+0000","last_peered":"2023-09-09T23:13:39.020222+0000","last_clean":"2023-09-09T23:13:39.020222+0000","last_became_active":"2023-09-09T23:12:17.907588+0000","last_became_peered":"2023-09-09T23:12:17.907588+0000","last_unstale":"2023-09-09T23:13:39.020222+0000","last_undegraded":"2023-09-09T23:13:39.020222+0000","last_fullsized":"2023-09-09T23:13:39.020222+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:57:30.910812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.329807+0000","last_change":"2023-09-09T23:12:17.172481+0000","last_active":"2023-09-09T23:13:09.329807+0000","last_peered":"2023-09-09T23:13:09.329807+0000","last_clean":"2023-09-09T23:13:09.329807+0000","last_became_active":"2023-09-09T23:12:17.172178+0000","last_became_peered":"2023-09-09T23:12:17.172178+0000","last_unstale":"2023-09-09T23:13:09.329807+0000","last_undegraded":"2023-09-09T23:13:09.329807+0000","last_fullsized":"2023-09-09T23:13:09.329807+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:08:55.328423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.021598+0000","last_change":"2023-09-09T23:12:17.900155+0000","last_active":"2023-09-09T23:13:39.021598+0000","last_peered":"2023-09-09T23:13:39.021598+0000","last_clean":"2023-09-09T23:13:39.021598+0000","last_became_active":"2023-09-09T23:12:17.899862+0000","last_became_peered":"2023-09-09T23:12:17.899862+0000","last_unstale":"2023-09-09T23:13:39.021598+0000","last_undegraded":"2023-09-09T23:13:39.021598+0000","last_fullsized":"2023-09-09T23:13:39.021598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:57:14.949174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.329618+0000","last_change":"2023-09-09T23:12:17.171890+0000","last_active":"2023-09-09T23:13:09.329618+0000","last_peered":"2023-09-09T23:13:09.329618+0000","last_clean":"2023-09-09T23:13:09.329618+0000","last_became_active":"2023-09-09T23:12:17.171496+0000","last_became_peered":"2023-09-09T23:12:17.171496+0000","last_unstale":"2023-09-09T23:13:09.329618+0000","last_undegraded":"2023-09-09T23:13:09.329618+0000","last_fullsized":"2023-09-09T23:13:09.329618+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:38:25.704989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.020476+0000","last_change":"2023-09-09T23:12:17.901067+0000","last_active":"2023-09-09T23:13:39.020476+0000","last_peered":"2023-09-09T23:13:39.020476+0000","last_clean":"2023-09-09T23:13:39.020476+0000","last_became_active":"2023-09-09T23:12:17.900069+0000","last_became_peered":"2023-09-09T23:12:17.900069+0000","last_unstale":"2023-09-09T23:13:39.020476+0000","last_undegraded":"2023-09-09T23:13:39.020476+0000","last_fullsized":"2023-09-09T23:13:39.020476+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:59:55.356235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'7","reported_seq":101,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:38.379346+0000","last_change":"2023-09-09T23:12:17.169474+0000","last_active":"2023-09-09T23:13:38.379346+0000","last_peered":"2023-09-09T23:13:38.379346+0000","last_clean":"2023-09-09T23:13:38.379346+0000","last_became_active":"2023-09-09T23:12:17.169186+0000","last_became_peered":"2023-09-09T23:12:17.169186+0000","last_unstale":"2023-09-09T23:13:38.379346+0000","last_undegraded":"2023-09-09T23:13:38.379346+0000","last_fullsized":"2023-09-09T23:13:38.379346+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:47:13.027949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.014143+0000","last_change":"2023-09-09T23:12:17.901605+0000","last_active":"2023-09-09T23:13:39.014143+0000","last_peered":"2023-09-09T23:13:39.014143+0000","last_clean":"2023-09-09T23:13:39.014143+0000","last_became_active":"2023-09-09T23:12:17.901287+0000","last_became_peered":"2023-09-09T23:12:17.901287+0000","last_unstale":"2023-09-09T23:13:39.014143+0000","last_undegraded":"2023-09-09T23:13:39.014143+0000","last_fullsized":"2023-09-09T23:13:39.014143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:16:41.570750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.333973+0000","last_change":"2023-09-09T23:12:17.171797+0000","last_active":"2023-09-09T23:13:09.333973+0000","last_peered":"2023-09-09T23:13:09.333973+0000","last_clean":"2023-09-09T23:13:09.333973+0000","last_became_active":"2023-09-09T23:12:17.171521+0000","last_became_peered":"2023-09-09T23:12:17.171521+0000","last_unstale":"2023-09-09T23:13:09.333973+0000","last_undegraded":"2023-09-09T23:13:09.333973+0000","last_fullsized":"2023-09-09T23:13:09.333973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:07:51.951393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.009071+0000","last_change":"2023-09-09T23:12:17.900284+0000","last_active":"2023-09-09T23:13:39.009071+0000","last_peered":"2023-09-09T23:13:39.009071+0000","last_clean":"2023-09-09T23:13:39.009071+0000","last_became_active":"2023-09-09T23:12:17.899975+0000","last_became_peered":"2023-09-09T23:12:17.899975+0000","last_unstale":"2023-09-09T23:13:39.009071+0000","last_undegraded":"2023-09-09T23:13:39.009071+0000","last_fullsized":"2023-09-09T23:13:39.009071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:10:56.723338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.333263+0000","last_change":"2023-09-09T23:12:17.175328+0000","last_active":"2023-09-09T23:13:09.333263+0000","last_peered":"2023-09-09T23:13:09.333263+0000","last_clean":"2023-09-09T23:13:09.333263+0000","last_became_active":"2023-09-09T23:12:17.174977+0000","last_became_peered":"2023-09-09T23:12:17.174977+0000","last_unstale":"2023-09-09T23:13:09.333263+0000","last_undegraded":"2023-09-09T23:13:09.333263+0000","last_fullsized":"2023-09-09T23:13:09.333263+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:01:36.272333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.016297+0000","last_change":"2023-09-09T23:12:17.902982+0000","last_active":"2023-09-09T23:13:39.016297+0000","last_peered":"2023-09-09T23:13:39.016297+0000","last_clean":"2023-09-09T23:13:39.016297+0000","last_became_active":"2023-09-09T23:12:17.902677+0000","last_became_peered":"2023-09-09T23:12:17.902677+0000","last_unstale":"2023-09-09T23:13:39.016297+0000","last_undegraded":"2023-09-09T23:13:39.016297+0000","last_fullsized":"2023-09-09T23:13:39.016297+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:08:31.507899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.330756+0000","last_change":"2023-09-09T23:12:17.170099+0000","last_active":"2023-09-09T23:13:09.330756+0000","last_peered":"2023-09-09T23:13:09.330756+0000","last_clean":"2023-09-09T23:13:09.330756+0000","last_became_active":"2023-09-09T23:12:17.169827+0000","last_became_peered":"2023-09-09T23:12:17.169827+0000","last_unstale":"2023-09-09T23:13:09.330756+0000","last_undegraded":"2023-09-09T23:13:09.330756+0000","last_fullsized":"2023-09-09T23:13:09.330756+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:16:34.693568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.007732+0000","last_change":"2023-09-09T23:12:17.903735+0000","last_active":"2023-09-09T23:13:39.007732+0000","last_peered":"2023-09-09T23:13:39.007732+0000","last_clean":"2023-09-09T23:13:39.007732+0000","last_became_active":"2023-09-09T23:12:17.903410+0000","last_became_peered":"2023-09-09T23:12:17.903410+0000","last_unstale":"2023-09-09T23:13:39.007732+0000","last_undegraded":"2023-09-09T23:13:39.007732+0000","last_fullsized":"2023-09-09T23:13:39.007732+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:00:47.664613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'5","reported_seq":176,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:41.384523+0000","last_change":"2023-09-09T23:12:17.176004+0000","last_active":"2023-09-09T23:13:41.384523+0000","last_peered":"2023-09-09T23:13:41.384523+0000","last_clean":"2023-09-09T23:13:41.384523+0000","last_became_active":"2023-09-09T23:12:17.175514+0000","last_became_peered":"2023-09-09T23:12:17.175514+0000","last_unstale":"2023-09-09T23:13:41.384523+0000","last_undegraded":"2023-09-09T23:13:41.384523+0000","last_fullsized":"2023-09-09T23:13:41.384523+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:43:54.587724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":156,"num_read_kb":156,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.021253+0000","last_change":"2023-09-09T23:12:17.900943+0000","last_active":"2023-09-09T23:13:39.021253+0000","last_peered":"2023-09-09T23:13:39.021253+0000","last_clean":"2023-09-09T23:13:39.021253+0000","last_became_active":"2023-09-09T23:12:17.900662+0000","last_became_peered":"2023-09-09T23:12:17.900662+0000","last_unstale":"2023-09-09T23:13:39.021253+0000","last_undegraded":"2023-09-09T23:13:39.021253+0000","last_fullsized":"2023-09-09T23:13:39.021253+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:39:41.480033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.366461+0000","last_change":"2023-09-09T23:12:17.172458+0000","last_active":"2023-09-09T23:13:09.366461+0000","last_peered":"2023-09-09T23:13:09.366461+0000","last_clean":"2023-09-09T23:13:09.366461+0000","last_became_active":"2023-09-09T23:12:17.172119+0000","last_became_peered":"2023-09-09T23:12:17.172119+0000","last_unstale":"2023-09-09T23:13:09.366461+0000","last_undegraded":"2023-09-09T23:13:09.366461+0000","last_fullsized":"2023-09-09T23:13:09.366461+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:31:27.821245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.008347+0000","last_change":"2023-09-09T23:12:17.903252+0000","last_active":"2023-09-09T23:13:39.008347+0000","last_peered":"2023-09-09T23:13:39.008347+0000","last_clean":"2023-09-09T23:13:39.008347+0000","last_became_active":"2023-09-09T23:12:17.902466+0000","last_became_peered":"2023-09-09T23:12:17.902466+0000","last_unstale":"2023-09-09T23:13:39.008347+0000","last_undegraded":"2023-09-09T23:13:39.008347+0000","last_fullsized":"2023-09-09T23:13:39.008347+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:28:07.535937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.331894+0000","last_change":"2023-09-09T23:12:17.165106+0000","last_active":"2023-09-09T23:13:09.331894+0000","last_peered":"2023-09-09T23:13:09.331894+0000","last_clean":"2023-09-09T23:13:09.331894+0000","last_became_active":"2023-09-09T23:12:17.164783+0000","last_became_peered":"2023-09-09T23:12:17.164783+0000","last_unstale":"2023-09-09T23:13:09.331894+0000","last_undegraded":"2023-09-09T23:13:09.331894+0000","last_fullsized":"2023-09-09T23:13:09.331894+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T04:02:14.984870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.021467+0000","last_change":"2023-09-09T23:12:17.899473+0000","last_active":"2023-09-09T23:13:39.021467+0000","last_peered":"2023-09-09T23:13:39.021467+0000","last_clean":"2023-09-09T23:13:39.021467+0000","last_became_active":"2023-09-09T23:12:17.899142+0000","last_became_peered":"2023-09-09T23:12:17.899142+0000","last_unstale":"2023-09-09T23:13:39.021467+0000","last_undegraded":"2023-09-09T23:13:39.021467+0000","last_fullsized":"2023-09-09T23:13:39.021467+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:50:30.261873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'34","reported_seq":217,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:41.384767+0000","last_change":"2023-09-09T23:12:17.172560+0000","last_active":"2023-09-09T23:13:41.384767+0000","last_peered":"2023-09-09T23:13:41.384767+0000","last_clean":"2023-09-09T23:13:41.384767+0000","last_became_active":"2023-09-09T23:12:17.172213+0000","last_became_peered":"2023-09-09T23:12:17.172213+0000","last_unstale":"2023-09-09T23:13:41.384767+0000","last_undegraded":"2023-09-09T23:13:41.384767+0000","last_fullsized":"2023-09-09T23:13:41.384767+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:30:00.206901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62235,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":168,"num_read_kb":139,"num_write":36,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.017552+0000","last_change":"2023-09-09T23:12:17.898279+0000","last_active":"2023-09-09T23:13:39.017552+0000","last_peered":"2023-09-09T23:13:39.017552+0000","last_clean":"2023-09-09T23:13:39.017552+0000","last_became_active":"2023-09-09T23:12:17.898041+0000","last_became_peered":"2023-09-09T23:12:17.898041+0000","last_unstale":"2023-09-09T23:13:39.017552+0000","last_undegraded":"2023-09-09T23:13:39.017552+0000","last_fullsized":"2023-09-09T23:13:39.017552+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:01:10.315445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:10.350264+0000","last_change":"2023-09-09T23:13:10.349904+0000","last_active":"2023-09-09T23:13:10.350264+0000","last_peered":"2023-09-09T23:13:10.350264+0000","last_clean":"2023-09-09T23:13:10.350264+0000","last_became_active":"2023-09-09T23:13:10.337554+0000","last_became_peered":"2023-09-09T23:13:10.337554+0000","last_unstale":"2023-09-09T23:13:10.350264+0000","last_undegraded":"2023-09-09T23:13:10.350264+0000","last_fullsized":"2023-09-09T23:13:10.350264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:16:39.275038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.012288+0000","last_change":"2023-09-09T23:12:17.904087+0000","last_active":"2023-09-09T23:13:39.012288+0000","last_peered":"2023-09-09T23:13:39.012288+0000","last_clean":"2023-09-09T23:13:39.012288+0000","last_became_active":"2023-09-09T23:12:17.903670+0000","last_became_peered":"2023-09-09T23:12:17.903670+0000","last_unstale":"2023-09-09T23:13:39.012288+0000","last_undegraded":"2023-09-09T23:13:39.012288+0000","last_fullsized":"2023-09-09T23:13:39.012288+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:24:04.403871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.331421+0000","last_change":"2023-09-09T23:12:17.170594+0000","last_active":"2023-09-09T23:13:09.331421+0000","last_peered":"2023-09-09T23:13:09.331421+0000","last_clean":"2023-09-09T23:13:09.331421+0000","last_became_active":"2023-09-09T23:12:17.170223+0000","last_became_peered":"2023-09-09T23:12:17.170223+0000","last_unstale":"2023-09-09T23:13:09.331421+0000","last_undegraded":"2023-09-09T23:13:09.331421+0000","last_fullsized":"2023-09-09T23:13:09.331421+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:30:09.222947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.019450+0000","last_change":"2023-09-09T23:12:17.901047+0000","last_active":"2023-09-09T23:13:39.019450+0000","last_peered":"2023-09-09T23:13:39.019450+0000","last_clean":"2023-09-09T23:13:39.019450+0000","last_became_active":"2023-09-09T23:12:17.900129+0000","last_became_peered":"2023-09-09T23:12:17.900129+0000","last_unstale":"2023-09-09T23:13:39.019450+0000","last_undegraded":"2023-09-09T23:13:39.019450+0000","last_fullsized":"2023-09-09T23:13:39.019450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:59:36.338392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'6","reported_seq":100,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:37.378029+0000","last_change":"2023-09-09T23:12:17.169495+0000","last_active":"2023-09-09T23:13:37.378029+0000","last_peered":"2023-09-09T23:13:37.378029+0000","last_clean":"2023-09-09T23:13:37.378029+0000","last_became_active":"2023-09-09T23:12:17.169170+0000","last_became_peered":"2023-09-09T23:12:17.169170+0000","last_unstale":"2023-09-09T23:13:37.378029+0000","last_undegraded":"2023-09-09T23:13:37.378029+0000","last_fullsized":"2023-09-09T23:13:37.378029+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:36:23.765574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.018670+0000","last_change":"2023-09-09T23:12:17.907826+0000","last_active":"2023-09-09T23:13:39.018670+0000","last_peered":"2023-09-09T23:13:39.018670+0000","last_clean":"2023-09-09T23:13:39.018670+0000","last_became_active":"2023-09-09T23:12:17.907526+0000","last_became_peered":"2023-09-09T23:12:17.907526+0000","last_unstale":"2023-09-09T23:13:39.018670+0000","last_undegraded":"2023-09-09T23:13:39.018670+0000","last_fullsized":"2023-09-09T23:13:39.018670+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:17:20.587253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.328902+0000","last_change":"2023-09-09T23:12:17.170139+0000","last_active":"2023-09-09T23:13:09.328902+0000","last_peered":"2023-09-09T23:13:09.328902+0000","last_clean":"2023-09-09T23:13:09.328902+0000","last_became_active":"2023-09-09T23:12:17.169797+0000","last_became_peered":"2023-09-09T23:12:17.169797+0000","last_unstale":"2023-09-09T23:13:09.328902+0000","last_undegraded":"2023-09-09T23:13:09.328902+0000","last_fullsized":"2023-09-09T23:13:09.328902+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:59:44.668114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.021950+0000","last_change":"2023-09-09T23:12:17.902567+0000","last_active":"2023-09-09T23:13:39.021950+0000","last_peered":"2023-09-09T23:13:39.021950+0000","last_clean":"2023-09-09T23:13:39.021950+0000","last_became_active":"2023-09-09T23:12:17.902229+0000","last_became_peered":"2023-09-09T23:12:17.902229+0000","last_unstale":"2023-09-09T23:13:39.021950+0000","last_undegraded":"2023-09-09T23:13:39.021950+0000","last_fullsized":"2023-09-09T23:13:39.021950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:49:50.049925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:10.341547+0000","last_change":"2023-09-09T23:13:10.341173+0000","last_active":"2023-09-09T23:13:10.341547+0000","last_peered":"2023-09-09T23:13:10.341547+0000","last_clean":"2023-09-09T23:13:10.341547+0000","last_became_active":"2023-09-09T23:13:10.336085+0000","last_became_peered":"2023-09-09T23:13:10.336085+0000","last_unstale":"2023-09-09T23:13:10.341547+0000","last_undegraded":"2023-09-09T23:13:10.341547+0000","last_fullsized":"2023-09-09T23:13:10.341547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:14:47.365794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.326890+0000","last_change":"2023-09-09T23:12:17.900284+0000","last_active":"2023-09-09T23:13:09.326890+0000","last_peered":"2023-09-09T23:13:09.326890+0000","last_clean":"2023-09-09T23:13:09.326890+0000","last_became_active":"2023-09-09T23:12:17.899923+0000","last_became_peered":"2023-09-09T23:12:17.899923+0000","last_unstale":"2023-09-09T23:13:09.326890+0000","last_undegraded":"2023-09-09T23:13:09.326890+0000","last_fullsized":"2023-09-09T23:13:09.326890+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T10:32:32.230968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.331153+0000","last_change":"2023-09-09T23:12:17.176313+0000","last_active":"2023-09-09T23:13:09.331153+0000","last_peered":"2023-09-09T23:13:09.331153+0000","last_clean":"2023-09-09T23:13:09.331153+0000","last_became_active":"2023-09-09T23:12:17.175934+0000","last_became_peered":"2023-09-09T23:12:17.175934+0000","last_unstale":"2023-09-09T23:13:09.331153+0000","last_undegraded":"2023-09-09T23:13:09.331153+0000","last_fullsized":"2023-09-09T23:13:09.331153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:33:15.044986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.006607+0000","last_change":"2023-09-09T23:12:17.900284+0000","last_active":"2023-09-09T23:13:39.006607+0000","last_peered":"2023-09-09T23:13:39.006607+0000","last_clean":"2023-09-09T23:13:39.006607+0000","last_became_active":"2023-09-09T23:12:17.899970+0000","last_became_peered":"2023-09-09T23:12:17.899970+0000","last_unstale":"2023-09-09T23:13:39.006607+0000","last_undegraded":"2023-09-09T23:13:39.006607+0000","last_fullsized":"2023-09-09T23:13:39.006607+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:01:37.415195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":96,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:38.405308+0000","last_change":"2023-09-09T23:12:17.170123+0000","last_active":"2023-09-09T23:13:38.405308+0000","last_peered":"2023-09-09T23:13:38.405308+0000","last_clean":"2023-09-09T23:13:38.405308+0000","last_became_active":"2023-09-09T23:12:17.169775+0000","last_became_peered":"2023-09-09T23:12:17.169775+0000","last_unstale":"2023-09-09T23:13:38.405308+0000","last_undegraded":"2023-09-09T23:13:38.405308+0000","last_fullsized":"2023-09-09T23:13:38.405308+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:22:44.520489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.011772+0000","last_change":"2023-09-09T23:12:17.894881+0000","last_active":"2023-09-09T23:13:39.011772+0000","last_peered":"2023-09-09T23:13:39.011772+0000","last_clean":"2023-09-09T23:13:39.011772+0000","last_became_active":"2023-09-09T23:12:17.894567+0000","last_became_peered":"2023-09-09T23:12:17.894567+0000","last_unstale":"2023-09-09T23:13:39.011772+0000","last_undegraded":"2023-09-09T23:13:39.011772+0000","last_fullsized":"2023-09-09T23:13:39.011772+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:40:37.098780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.015183+0000","last_change":"2023-09-09T23:12:17.903319+0000","last_active":"2023-09-09T23:13:39.015183+0000","last_peered":"2023-09-09T23:13:39.015183+0000","last_clean":"2023-09-09T23:13:39.015183+0000","last_became_active":"2023-09-09T23:12:17.902642+0000","last_became_peered":"2023-09-09T23:12:17.902642+0000","last_unstale":"2023-09-09T23:13:39.015183+0000","last_undegraded":"2023-09-09T23:13:39.015183+0000","last_fullsized":"2023-09-09T23:13:39.015183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:03:29.562813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":99,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:38.975476+0000","last_change":"2023-09-09T23:12:17.172639+0000","last_active":"2023-09-09T23:13:38.975476+0000","last_peered":"2023-09-09T23:13:38.975476+0000","last_clean":"2023-09-09T23:13:38.975476+0000","last_became_active":"2023-09-09T23:12:17.172309+0000","last_became_peered":"2023-09-09T23:12:17.172309+0000","last_unstale":"2023-09-09T23:13:38.975476+0000","last_undegraded":"2023-09-09T23:13:38.975476+0000","last_fullsized":"2023-09-09T23:13:38.975476+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:05:56.160984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.019865+0000","last_change":"2023-09-09T23:13:10.336403+0000","last_active":"2023-09-09T23:13:39.019865+0000","last_peered":"2023-09-09T23:13:39.019865+0000","last_clean":"2023-09-09T23:13:39.019865+0000","last_became_active":"2023-09-09T23:13:10.335730+0000","last_became_peered":"2023-09-09T23:13:10.335730+0000","last_unstale":"2023-09-09T23:13:39.019865+0000","last_undegraded":"2023-09-09T23:13:39.019865+0000","last_fullsized":"2023-09-09T23:13:39.019865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:35:00.228469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.327321+0000","last_change":"2023-09-09T23:12:17.166623+0000","last_active":"2023-09-09T23:13:09.327321+0000","last_peered":"2023-09-09T23:13:09.327321+0000","last_clean":"2023-09-09T23:13:09.327321+0000","last_became_active":"2023-09-09T23:12:17.166293+0000","last_became_peered":"2023-09-09T23:12:17.166293+0000","last_unstale":"2023-09-09T23:13:09.327321+0000","last_undegraded":"2023-09-09T23:13:09.327321+0000","last_fullsized":"2023-09-09T23:13:09.327321+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:52:51.267501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":319,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.366308+0000","last_change":"2023-09-09T23:11:20.948461+0000","last_active":"2023-09-09T23:13:09.366308+0000","last_peered":"2023-09-09T23:13:09.366308+0000","last_clean":"2023-09-09T23:13:09.366308+0000","last_became_active":"2023-09-09T23:11:20.947856+0000","last_became_peered":"2023-09-09T23:11:20.947856+0000","last_unstale":"2023-09-09T23:13:09.366308+0000","last_undegraded":"2023-09-09T23:13:09.366308+0000","last_fullsized":"2023-09-09T23:13:09.366308+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:08:31.737051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:08:31.737051+0000","last_clean_scrub_stamp":"2023-09-09T23:08:31.737051+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:20:16.293707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.013785+0000","last_change":"2023-09-09T23:12:17.907468+0000","last_active":"2023-09-09T23:13:39.013785+0000","last_peered":"2023-09-09T23:13:39.013785+0000","last_clean":"2023-09-09T23:13:39.013785+0000","last_became_active":"2023-09-09T23:12:17.907117+0000","last_became_peered":"2023-09-09T23:12:17.907117+0000","last_unstale":"2023-09-09T23:13:39.013785+0000","last_undegraded":"2023-09-09T23:13:39.013785+0000","last_fullsized":"2023-09-09T23:13:39.013785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:27:16.623116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.366405+0000","last_change":"2023-09-09T23:12:17.171505+0000","last_active":"2023-09-09T23:13:09.366405+0000","last_peered":"2023-09-09T23:13:09.366405+0000","last_clean":"2023-09-09T23:13:09.366405+0000","last_became_active":"2023-09-09T23:12:17.171185+0000","last_became_peered":"2023-09-09T23:12:17.171185+0000","last_unstale":"2023-09-09T23:13:09.366405+0000","last_undegraded":"2023-09-09T23:13:09.366405+0000","last_fullsized":"2023-09-09T23:13:09.366405+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:07:54.271364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.329554+0000","last_change":"2023-09-09T23:12:17.176450+0000","last_active":"2023-09-09T23:13:09.329554+0000","last_peered":"2023-09-09T23:13:09.329554+0000","last_clean":"2023-09-09T23:13:09.329554+0000","last_became_active":"2023-09-09T23:12:17.176217+0000","last_became_peered":"2023-09-09T23:12:17.176217+0000","last_unstale":"2023-09-09T23:13:09.329554+0000","last_undegraded":"2023-09-09T23:13:09.329554+0000","last_fullsized":"2023-09-09T23:13:09.329554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T01:57:26.604261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.008179+0000","last_change":"2023-09-09T23:12:17.897612+0000","last_active":"2023-09-09T23:13:39.008179+0000","last_peered":"2023-09-09T23:13:39.008179+0000","last_clean":"2023-09-09T23:13:39.008179+0000","last_became_active":"2023-09-09T23:12:17.897292+0000","last_became_peered":"2023-09-09T23:12:17.897292+0000","last_unstale":"2023-09-09T23:13:39.008179+0000","last_undegraded":"2023-09-09T23:13:39.008179+0000","last_fullsized":"2023-09-09T23:13:39.008179+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:25:46.374556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'9","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.331260+0000","last_change":"2023-09-09T23:12:17.175682+0000","last_active":"2023-09-09T23:13:09.331260+0000","last_peered":"2023-09-09T23:13:09.331260+0000","last_clean":"2023-09-09T23:13:09.331260+0000","last_became_active":"2023-09-09T23:12:17.175291+0000","last_became_peered":"2023-09-09T23:12:17.175291+0000","last_unstale":"2023-09-09T23:13:09.331260+0000","last_undegraded":"2023-09-09T23:13:09.331260+0000","last_fullsized":"2023-09-09T23:13:09.331260+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:29:02.349319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.016472+0000","last_change":"2023-09-09T23:12:17.904153+0000","last_active":"2023-09-09T23:13:39.016472+0000","last_peered":"2023-09-09T23:13:39.016472+0000","last_clean":"2023-09-09T23:13:39.016472+0000","last_became_active":"2023-09-09T23:12:17.903804+0000","last_became_peered":"2023-09-09T23:12:17.903804+0000","last_unstale":"2023-09-09T23:13:39.016472+0000","last_undegraded":"2023-09-09T23:13:39.016472+0000","last_fullsized":"2023-09-09T23:13:39.016472+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:39:34.046551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.327392+0000","last_change":"2023-09-09T23:12:17.169052+0000","last_active":"2023-09-09T23:13:09.327392+0000","last_peered":"2023-09-09T23:13:09.327392+0000","last_clean":"2023-09-09T23:13:09.327392+0000","last_became_active":"2023-09-09T23:12:17.168719+0000","last_became_peered":"2023-09-09T23:12:17.168719+0000","last_unstale":"2023-09-09T23:13:09.327392+0000","last_undegraded":"2023-09-09T23:13:09.327392+0000","last_fullsized":"2023-09-09T23:13:09.327392+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:56:44.427954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.011144+0000","last_change":"2023-09-09T23:12:17.900945+0000","last_active":"2023-09-09T23:13:39.011144+0000","last_peered":"2023-09-09T23:13:39.011144+0000","last_clean":"2023-09-09T23:13:39.011144+0000","last_became_active":"2023-09-09T23:12:17.900602+0000","last_became_peered":"2023-09-09T23:12:17.900602+0000","last_unstale":"2023-09-09T23:13:39.011144+0000","last_undegraded":"2023-09-09T23:13:39.011144+0000","last_fullsized":"2023-09-09T23:13:39.011144+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:22:10.336285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.331603+0000","last_change":"2023-09-09T23:12:17.166619+0000","last_active":"2023-09-09T23:13:09.331603+0000","last_peered":"2023-09-09T23:13:09.331603+0000","last_clean":"2023-09-09T23:13:09.331603+0000","last_became_active":"2023-09-09T23:12:17.166286+0000","last_became_peered":"2023-09-09T23:12:17.166286+0000","last_unstale":"2023-09-09T23:13:09.331603+0000","last_undegraded":"2023-09-09T23:13:09.331603+0000","last_fullsized":"2023-09-09T23:13:09.331603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:41:40.713296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:38.983854+0000","last_change":"2023-09-09T23:12:17.897131+0000","last_active":"2023-09-09T23:13:38.983854+0000","last_peered":"2023-09-09T23:13:38.983854+0000","last_clean":"2023-09-09T23:13:38.983854+0000","last_became_active":"2023-09-09T23:12:17.896873+0000","last_became_peered":"2023-09-09T23:12:17.896873+0000","last_unstale":"2023-09-09T23:13:38.983854+0000","last_undegraded":"2023-09-09T23:13:38.983854+0000","last_fullsized":"2023-09-09T23:13:38.983854+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:54:21.091418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.328807+0000","last_change":"2023-09-09T23:12:17.168868+0000","last_active":"2023-09-09T23:13:09.328807+0000","last_peered":"2023-09-09T23:13:09.328807+0000","last_clean":"2023-09-09T23:13:09.328807+0000","last_became_active":"2023-09-09T23:12:17.168572+0000","last_became_peered":"2023-09-09T23:12:17.168572+0000","last_unstale":"2023-09-09T23:13:09.328807+0000","last_undegraded":"2023-09-09T23:13:09.328807+0000","last_fullsized":"2023-09-09T23:13:09.328807+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:54:08.108023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.326672+0000","last_change":"2023-09-09T23:12:17.902887+0000","last_active":"2023-09-09T23:13:09.326672+0000","last_peered":"2023-09-09T23:13:09.326672+0000","last_clean":"2023-09-09T23:13:09.326672+0000","last_became_active":"2023-09-09T23:12:17.902507+0000","last_became_peered":"2023-09-09T23:12:17.902507+0000","last_unstale":"2023-09-09T23:13:09.326672+0000","last_undegraded":"2023-09-09T23:13:09.326672+0000","last_fullsized":"2023-09-09T23:13:09.326672+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:18:08.000184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.333618+0000","last_change":"2023-09-09T23:12:17.171379+0000","last_active":"2023-09-09T23:13:09.333618+0000","last_peered":"2023-09-09T23:13:09.333618+0000","last_clean":"2023-09-09T23:13:09.333618+0000","last_became_active":"2023-09-09T23:12:17.171024+0000","last_became_peered":"2023-09-09T23:12:17.171024+0000","last_unstale":"2023-09-09T23:13:09.333618+0000","last_undegraded":"2023-09-09T23:13:09.333618+0000","last_fullsized":"2023-09-09T23:13:09.333618+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:17:12.206768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.009341+0000","last_change":"2023-09-09T23:12:17.905510+0000","last_active":"2023-09-09T23:13:39.009341+0000","last_peered":"2023-09-09T23:13:39.009341+0000","last_clean":"2023-09-09T23:13:39.009341+0000","last_became_active":"2023-09-09T23:12:17.905207+0000","last_became_peered":"2023-09-09T23:12:17.905207+0000","last_unstale":"2023-09-09T23:13:39.009341+0000","last_undegraded":"2023-09-09T23:13:39.009341+0000","last_fullsized":"2023-09-09T23:13:39.009341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T02:48:20.167421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.329203+0000","last_change":"2023-09-09T23:12:17.173764+0000","last_active":"2023-09-09T23:13:09.329203+0000","last_peered":"2023-09-09T23:13:09.329203+0000","last_clean":"2023-09-09T23:13:09.329203+0000","last_became_active":"2023-09-09T23:12:17.173470+0000","last_became_peered":"2023-09-09T23:12:17.173470+0000","last_unstale":"2023-09-09T23:13:09.329203+0000","last_undegraded":"2023-09-09T23:13:09.329203+0000","last_fullsized":"2023-09-09T23:13:09.329203+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:01:54.091664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.020873+0000","last_change":"2023-09-09T23:12:17.906292+0000","last_active":"2023-09-09T23:13:39.020873+0000","last_peered":"2023-09-09T23:13:39.020873+0000","last_clean":"2023-09-09T23:13:39.020873+0000","last_became_active":"2023-09-09T23:12:17.905995+0000","last_became_peered":"2023-09-09T23:12:17.905995+0000","last_unstale":"2023-09-09T23:13:39.020873+0000","last_undegraded":"2023-09-09T23:13:39.020873+0000","last_fullsized":"2023-09-09T23:13:39.020873+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:52:29.972831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'5","reported_seq":99,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:40.421016+0000","last_change":"2023-09-09T23:12:17.173818+0000","last_active":"2023-09-09T23:13:40.421016+0000","last_peered":"2023-09-09T23:13:40.421016+0000","last_clean":"2023-09-09T23:13:40.421016+0000","last_became_active":"2023-09-09T23:12:17.173413+0000","last_became_peered":"2023-09-09T23:12:17.173413+0000","last_unstale":"2023-09-09T23:13:40.421016+0000","last_undegraded":"2023-09-09T23:13:40.421016+0000","last_fullsized":"2023-09-09T23:13:40.421016+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:55:28.856863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.021529+0000","last_change":"2023-09-09T23:12:17.906833+0000","last_active":"2023-09-09T23:13:39.021529+0000","last_peered":"2023-09-09T23:13:39.021529+0000","last_clean":"2023-09-09T23:13:39.021529+0000","last_became_active":"2023-09-09T23:12:17.906138+0000","last_became_peered":"2023-09-09T23:12:17.906138+0000","last_unstale":"2023-09-09T23:13:39.021529+0000","last_undegraded":"2023-09-09T23:13:39.021529+0000","last_fullsized":"2023-09-09T23:13:39.021529+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:38:07.779167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.331058+0000","last_change":"2023-09-09T23:12:17.171338+0000","last_active":"2023-09-09T23:13:09.331058+0000","last_peered":"2023-09-09T23:13:09.331058+0000","last_clean":"2023-09-09T23:13:09.331058+0000","last_became_active":"2023-09-09T23:12:17.170613+0000","last_became_peered":"2023-09-09T23:12:17.170613+0000","last_unstale":"2023-09-09T23:13:09.331058+0000","last_undegraded":"2023-09-09T23:13:09.331058+0000","last_fullsized":"2023-09-09T23:13:09.331058+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:24:07.794464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.017147+0000","last_change":"2023-09-09T23:12:17.906470+0000","last_active":"2023-09-09T23:13:39.017147+0000","last_peered":"2023-09-09T23:13:39.017147+0000","last_clean":"2023-09-09T23:13:39.017147+0000","last_became_active":"2023-09-09T23:12:17.906170+0000","last_became_peered":"2023-09-09T23:12:17.906170+0000","last_unstale":"2023-09-09T23:13:39.017147+0000","last_undegraded":"2023-09-09T23:13:39.017147+0000","last_fullsized":"2023-09-09T23:13:39.017147+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:44:37.483541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.333478+0000","last_change":"2023-09-09T23:12:17.172142+0000","last_active":"2023-09-09T23:13:09.333478+0000","last_peered":"2023-09-09T23:13:09.333478+0000","last_clean":"2023-09-09T23:13:09.333478+0000","last_became_active":"2023-09-09T23:12:17.171703+0000","last_became_peered":"2023-09-09T23:12:17.171703+0000","last_unstale":"2023-09-09T23:13:09.333478+0000","last_undegraded":"2023-09-09T23:13:09.333478+0000","last_fullsized":"2023-09-09T23:13:09.333478+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:16.151071+0000","last_clean_scrub_stamp":"2023-09-09T23:12:16.151071+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:49:31.414385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.018478+0000","last_change":"2023-09-09T23:12:17.900699+0000","last_active":"2023-09-09T23:13:39.018478+0000","last_peered":"2023-09-09T23:13:39.018478+0000","last_clean":"2023-09-09T23:13:39.018478+0000","last_became_active":"2023-09-09T23:12:17.900416+0000","last_became_peered":"2023-09-09T23:12:17.900416+0000","last_unstale":"2023-09-09T23:13:39.018478+0000","last_undegraded":"2023-09-09T23:13:39.018478+0000","last_fullsized":"2023-09-09T23:13:39.018478+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T03:54:29.328413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.326771+0000","last_change":"2023-09-09T23:12:17.900480+0000","last_active":"2023-09-09T23:13:09.326771+0000","last_peered":"2023-09-09T23:13:09.326771+0000","last_clean":"2023-09-09T23:13:09.326771+0000","last_became_active":"2023-09-09T23:12:17.900135+0000","last_became_peered":"2023-09-09T23:12:17.900135+0000","last_unstale":"2023-09-09T23:13:09.326771+0000","last_undegraded":"2023-09-09T23:13:09.326771+0000","last_fullsized":"2023-09-09T23:13:09.326771+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:25:44.254310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.017893+0000","last_change":"2023-09-09T23:12:17.901302+0000","last_active":"2023-09-09T23:13:39.017893+0000","last_peered":"2023-09-09T23:13:39.017893+0000","last_clean":"2023-09-09T23:13:39.017893+0000","last_became_active":"2023-09-09T23:12:17.900982+0000","last_became_peered":"2023-09-09T23:12:17.900982+0000","last_unstale":"2023-09-09T23:13:39.017893+0000","last_undegraded":"2023-09-09T23:13:39.017893+0000","last_fullsized":"2023-09-09T23:13:39.017893+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:41:41.119614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.012146+0000","last_change":"2023-09-09T23:12:17.904328+0000","last_active":"2023-09-09T23:13:39.012146+0000","last_peered":"2023-09-09T23:13:39.012146+0000","last_clean":"2023-09-09T23:13:39.012146+0000","last_became_active":"2023-09-09T23:12:17.904040+0000","last_became_peered":"2023-09-09T23:12:17.904040+0000","last_unstale":"2023-09-09T23:13:39.012146+0000","last_undegraded":"2023-09-09T23:13:39.012146+0000","last_fullsized":"2023-09-09T23:13:39.012146+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:01:05.986097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.004794+0000","last_change":"2023-09-09T23:13:10.337251+0000","last_active":"2023-09-09T23:13:39.004794+0000","last_peered":"2023-09-09T23:13:39.004794+0000","last_clean":"2023-09-09T23:13:39.004794+0000","last_became_active":"2023-09-09T23:13:10.336914+0000","last_became_peered":"2023-09-09T23:13:10.336914+0000","last_unstale":"2023-09-09T23:13:39.004794+0000","last_undegraded":"2023-09-09T23:13:39.004794+0000","last_fullsized":"2023-09-09T23:13:39.004794+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T11:00:49.058538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.013756+0000","last_change":"2023-09-09T23:12:17.907433+0000","last_active":"2023-09-09T23:13:39.013756+0000","last_peered":"2023-09-09T23:13:39.013756+0000","last_clean":"2023-09-09T23:13:39.013756+0000","last_became_active":"2023-09-09T23:12:17.907127+0000","last_became_peered":"2023-09-09T23:12:17.907127+0000","last_unstale":"2023-09-09T23:13:39.013756+0000","last_undegraded":"2023-09-09T23:13:39.013756+0000","last_fullsized":"2023-09-09T23:13:39.013756+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:00:32.291141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-09T23:13:09.326990+0000","last_change":"2023-09-09T23:12:17.901373+0000","last_active":"2023-09-09T23:13:09.326990+0000","last_peered":"2023-09-09T23:13:09.326990+0000","last_clean":"2023-09-09T23:13:09.326990+0000","last_became_active":"2023-09-09T23:12:17.901080+0000","last_became_peered":"2023-09-09T23:12:17.901080+0000","last_unstale":"2023-09-09T23:13:09.326990+0000","last_undegraded":"2023-09-09T23:13:09.326990+0000","last_fullsized":"2023-09-09T23:13:09.326990+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T07:23:12.316659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.020913+0000","last_change":"2023-09-09T23:12:17.896579+0000","last_active":"2023-09-09T23:13:39.020913+0000","last_peered":"2023-09-09T23:13:39.020913+0000","last_clean":"2023-09-09T23:13:39.020913+0000","last_became_active":"2023-09-09T23:12:17.896253+0000","last_became_peered":"2023-09-09T23:12:17.896253+0000","last_unstale":"2023-09-09T23:13:39.020913+0000","last_undegraded":"2023-09-09T23:13:39.020913+0000","last_fullsized":"2023-09-09T23:13:39.020913+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T05:30:20.249103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.008502+0000","last_change":"2023-09-09T23:12:17.903569+0000","last_active":"2023-09-09T23:13:39.008502+0000","last_peered":"2023-09-09T23:13:39.008502+0000","last_clean":"2023-09-09T23:13:39.008502+0000","last_became_active":"2023-09-09T23:12:17.903027+0000","last_became_peered":"2023-09-09T23:12:17.903027+0000","last_unstale":"2023-09-09T23:13:39.008502+0000","last_undegraded":"2023-09-09T23:13:39.008502+0000","last_fullsized":"2023-09-09T23:13:39.008502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:38:46.392973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.014057+0000","last_change":"2023-09-09T23:12:17.902944+0000","last_active":"2023-09-09T23:13:39.014057+0000","last_peered":"2023-09-09T23:13:39.014057+0000","last_clean":"2023-09-09T23:13:39.014057+0000","last_became_active":"2023-09-09T23:12:17.901812+0000","last_became_peered":"2023-09-09T23:12:17.901812+0000","last_unstale":"2023-09-09T23:13:39.014057+0000","last_undegraded":"2023-09-09T23:13:39.014057+0000","last_fullsized":"2023-09-09T23:13:39.014057+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T09:49:50.981693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.015867+0000","last_change":"2023-09-09T23:12:17.897612+0000","last_active":"2023-09-09T23:13:39.015867+0000","last_peered":"2023-09-09T23:13:39.015867+0000","last_clean":"2023-09-09T23:13:39.015867+0000","last_became_active":"2023-09-09T23:12:17.897318+0000","last_became_peered":"2023-09-09T23:12:17.897318+0000","last_unstale":"2023-09-09T23:13:39.015867+0000","last_undegraded":"2023-09-09T23:13:39.015867+0000","last_fullsized":"2023-09-09T23:13:39.015867+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T00:17:38.687470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.022189+0000","last_change":"2023-09-09T23:12:17.903849+0000","last_active":"2023-09-09T23:13:39.022189+0000","last_peered":"2023-09-09T23:13:39.022189+0000","last_clean":"2023-09-09T23:13:39.022189+0000","last_became_active":"2023-09-09T23:12:17.903534+0000","last_became_peered":"2023-09-09T23:12:17.903534+0000","last_unstale":"2023-09-09T23:13:39.022189+0000","last_undegraded":"2023-09-09T23:13:39.022189+0000","last_fullsized":"2023-09-09T23:13:39.022189+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T06:06:18.073811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:39.013153+0000","last_change":"2023-09-09T23:12:17.899782+0000","last_active":"2023-09-09T23:13:39.013153+0000","last_peered":"2023-09-09T23:13:39.013153+0000","last_clean":"2023-09-09T23:13:39.013153+0000","last_became_active":"2023-09-09T23:12:17.899445+0000","last_became_peered":"2023-09-09T23:12:17.899445+0000","last_unstale":"2023-09-09T23:13:39.013153+0000","last_undegraded":"2023-09-09T23:13:39.013153+0000","last_fullsized":"2023-09-09T23:13:39.013153+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-10T23:49:20.276927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-09T23:13:10.334992+0000","last_change":"2023-09-09T23:13:10.334992+0000","last_active":"2023-09-09T23:13:10.334992+0000","last_peered":"2023-09-09T23:13:10.334992+0000","last_clean":"2023-09-09T23:13:10.334992+0000","last_became_active":"2023-09-09T23:13:10.334629+0000","last_became_peered":"2023-09-09T23:13:10.334629+0000","last_unstale":"2023-09-09T23:13:10.334992+0000","last_undegraded":"2023-09-09T23:13:10.334992+0000","last_fullsized":"2023-09-09T23:13:10.334992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-09T23:12:17.153056+0000","last_clean_scrub_stamp":"2023-09-09T23:12:17.153056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-11T08:24:48.291826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":434,"ondisk_log_size":434,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69578,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":812,"num_read_kb":479,"num_write":238,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":90,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569344,"data_stored":204273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514523,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":434403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.65900000000000003}]},{"osd":1,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.747},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.77300000000000002}]},{"osd":2,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":4,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.88600000000000001}]},{"osd":5,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.79500000000000004}]},{"osd":6,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.74199999999999999}]},{"osd":7,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.92200000000000004}]},{"osd":8,"last update":"Sat Sep 9 23:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.72999999999999998}]},{"osd":9,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.93500000000000005}]},{"osd":10,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.873}]}]},{"osd":10,"up_from":69,"seq":296352743456,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1609728,"data_stored":1088114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.65400000000000003}]},{"osd":1,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.33200000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.68000000000000005,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.50700000000000001,"15min":0.51500000000000001},"min":{"1min":0.33700000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.80900000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.58999999999999997}]},{"osd":2,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.623,"15min":0.61499999999999999},"min":{"1min":0.55100000000000005,"5min":0.379,"15min":0.379},"max":{"1min":0.746,"5min":1.018,"15min":1.018},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.65000000000000002,"15min":0.65300000000000002},"min":{"1min":0.501,"5min":0.436,"15min":0.436},"max":{"1min":0.79400000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71999999999999997}]},{"osd":3,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.622,"15min":0.63700000000000001},"min":{"1min":0.41399999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.67400000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.64400000000000002,"15min":0.65900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.67200000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.746}]},{"osd":4,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60899999999999999,"15min":0.61299999999999999},"min":{"1min":0.42399999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.72199999999999998,"5min":1.024,"15min":1.024},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.36099999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79500000000000004,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.73299999999999998}]},{"osd":5,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.69799999999999995,"15min":0.70099999999999996},"min":{"1min":0.56000000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.85799999999999998,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.88},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71399999999999997,"15min":0.71599999999999997},"min":{"1min":0.47699999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88800000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.68899999999999995}]},{"osd":6,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.73099999999999998,"15min":0.75},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90000000000000002,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.73099999999999998,"15min":0.74099999999999999},"min":{"1min":0.54100000000000004,"5min":0.439,"15min":0.439},"max":{"1min":0.85199999999999998,"5min":1.038,"15min":1.038},"last":0.82599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":8,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.80200000000000005,"15min":0.81200000000000006},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.91500000000000004,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78500000000000003,"15min":0.78900000000000003},"min":{"1min":0.623,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.94099999999999995,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.84299999999999997}]},{"osd":9,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.81299999999999994,"15min":0.82399999999999995},"min":{"1min":0.61399999999999999,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.96799999999999997,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.80500000000000005,"15min":0.81999999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.878,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.79600000000000004}]},{"osd":11,"last update":"Sat Sep 9 23:12:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.95899999999999996}]}]},{"osd":9,"up_from":64,"seq":274877906979,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":434243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.435,"15min":0.443},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.73999999999999999,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.39200000000000002}]},{"osd":1,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.62},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56799999999999995,"15min":0.56899999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.60599999999999998}]},{"osd":2,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.626,"15min":0.63},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.625,"15min":0.63200000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.45700000000000002}]},{"osd":3,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.51300000000000001,"15min":0.52300000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.376}]},{"osd":4,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.629,"15min":0.63500000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96799999999999997,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.497},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.67400000000000004,"15min":0.68799999999999994},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81999999999999995,"5min":1.028,"15min":1.028},"last":0.59399999999999997}]},{"osd":5,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.73399999999999999,"15min":0.71499999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":5.6699999999999999,"5min":5.6699999999999999,"15min":5.6699999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.68200000000000005,"15min":0.69399999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.54300000000000004}]},{"osd":6,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.64300000000000002,"15min":0.65500000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.65600000000000003,"15min":0.67000000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.55400000000000005}]},{"osd":7,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.745,"15min":0.76100000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.82499999999999996,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.746,"15min":0.75800000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.66400000000000003}]},{"osd":8,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.73799999999999999,"15min":0.747},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.74099999999999999,"15min":0.755},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.85899999999999999,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.63300000000000001}]},{"osd":10,"last update":"Sat Sep 9 23:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73599999999999999,"15min":0.74199999999999999},"min":{"1min":0.56000000000000005,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.85899999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.88,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.84599999999999997}]}]},{"osd":8,"up_from":57,"seq":244813135913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":434599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.39400000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.87,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.38200000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78500000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.53900000000000003}]},{"osd":1,"last update":"Sat Sep 9 23:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.63200000000000001,"15min":0.60199999999999998},"min":{"1min":0.45600000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.89300000000000002,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60499999999999998,"15min":0.58899999999999997},"min":{"1min":0.36599999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82199999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.63700000000000001}]},{"osd":2,"last update":"Sat Sep 9 23:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.48599999999999999,"15min":0.48399999999999999},"min":{"1min":0.36299999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.78500000000000003,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.48499999999999999,"15min":0.48899999999999999},"min":{"1min":0.309,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69499999999999995,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.72399999999999998}]},{"osd":3,"last update":"Sat Sep 9 23:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57399999999999995,"15min":0.57899999999999996},"min":{"1min":0.497,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.755,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.53900000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.76100000000000001,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.48799999999999999}]},{"osd":4,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64100000000000001,"15min":0.63700000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94399999999999995,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.68999999999999995,"15min":0.68200000000000005},"min":{"1min":0.41899999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.73699999999999999}]},{"osd":5,"last update":"Sat Sep 9 23:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.34699999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.66900000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.47299999999999998}]},{"osd":6,"last update":"Sat Sep 9 23:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.72299999999999998,"15min":0.72599999999999998},"min":{"1min":0.56299999999999994,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86099999999999999,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68600000000000005,"15min":0.69599999999999995},"min":{"1min":0.56699999999999995,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85799999999999998,"5min":0.996,"15min":0.996},"last":0.498}]},{"osd":7,"last update":"Sat Sep 9 23:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72499999999999998,"15min":0.72599999999999998},"min":{"1min":0.52600000000000002,"5min":0.503,"15min":0.503},"max":{"1min":0.96499999999999997,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.89100000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.60799999999999998}]},{"osd":9,"last update":"Sat Sep 9 23:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71999999999999997,"15min":0.72099999999999997},"min":{"1min":0.48899999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.99399999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.77200000000000002,"15min":0.77000000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1060000000000001,"5min":1.115,"15min":1.115},"last":0.65400000000000003}]},{"osd":10,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.74299999999999999,"15min":0.73299999999999998},"min":{"1min":0.63400000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.97499999999999998,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.77500000000000002,"15min":0.75700000000000001},"min":{"1min":0.70699999999999996,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.621}]},{"osd":11,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.73699999999999999}]}]},{"osd":7,"up_from":50,"seq":214748364844,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1519616,"data_stored":1027485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.439},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.33100000000000002}]},{"osd":1,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.436,"15min":0.432},"min":{"1min":0.27800000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.437,"15min":0.42099999999999999},"min":{"1min":0.33200000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.59699999999999998,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.38700000000000001}]},{"osd":2,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63500000000000001,"15min":0.63900000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84399999999999997,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60299999999999998,"15min":0.60199999999999998},"min":{"1min":0.51200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.76600000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.53300000000000003}]},{"osd":3,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.42799999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82799999999999996,"5min":0.88,"15min":0.88},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.623,"15min":0.624},"min":{"1min":0.437,"5min":0.375,"15min":0.375},"max":{"1min":0.81100000000000005,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.56899999999999995}]},{"osd":4,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.35299999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.72599999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.35799999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.79700000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.42399999999999999}]},{"osd":5,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.69099999999999995,"15min":0.69499999999999995},"min":{"1min":0.54900000000000004,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.93999999999999995,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.503,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.88100000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58499999999999996}]},{"osd":6,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.72499999999999998,"15min":0.746},"min":{"1min":0.51500000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.86499999999999999,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.69999999999999996,"15min":0.71999999999999997},"min":{"1min":0.45900000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.753,"5min":1.254,"15min":1.254},"last":0.60899999999999999}]},{"osd":8,"last update":"Sat Sep 9 23:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73399999999999999,"15min":0.73899999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.86399999999999999,"5min":1.008,"15min":1.008},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.76400000000000001,"15min":0.77000000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.88,"5min":1.387,"15min":1.387},"last":0.73899999999999999}]},{"osd":9,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.75,"15min":0.76000000000000001},"min":{"1min":0.51700000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.89900000000000002,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.92400000000000004,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.753}]},{"osd":10,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.46899999999999997,"5min":0.442,"15min":0.442},"max":{"1min":0.95499999999999996,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.74299999999999999,"15min":0.745},"min":{"1min":0.58499999999999996,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.91500000000000004,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.72299999999999998}]},{"osd":11,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.70299999999999996}]}]},{"osd":6,"up_from":45,"seq":193273528369,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":433966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,8,0,0,1,27],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.377,"15min":0.36899999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":5.6859999999999999},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.40400000000000003,"15min":0.40200000000000002},"min":{"1min":0.26800000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.53700000000000003,"5min":0.753,"15min":0.753},"last":1.925}]},{"osd":1,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78100000000000003,"5min":0.877,"15min":0.877},"last":8.6560000000000006},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77200000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":14.58}]},{"osd":2,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.41699999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79600000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":1.798},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76800000000000002,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":4.298}]},{"osd":3,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.43099999999999999,"15min":0.433},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70199999999999996,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":1.181},{"interface":"front","average":{"1min":0.437,"5min":0.501,"15min":0.51200000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.59399999999999997,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":3.1949999999999998}]},{"osd":4,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60699999999999998,"15min":0.60299999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.78100000000000003,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":2.3620000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.77900000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":2.0139999999999998}]},{"osd":5,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.63200000000000001,"15min":0.64000000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.75600000000000001,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.2150000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.65900000000000003,"15min":0.65700000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82199999999999995,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":8.7859999999999996}]},{"osd":7,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.67700000000000005,"15min":0.68000000000000005},"min":{"1min":0.45000000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":6.766},{"interface":"front","average":{"1min":0.626,"5min":0.65800000000000003,"15min":0.66000000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.91500000000000004,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":9.1010000000000009}]},{"osd":8,"last update":"Sat Sep 9 23:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.76700000000000002,"15min":0.78800000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85099999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":1.405},{"interface":"front","average":{"1min":0.628,"5min":0.74399999999999999,"15min":0.76400000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.80100000000000005,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.556}]},{"osd":9,"last update":"Sat Sep 9 23:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.64500000000000002,"15min":0.66100000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70799999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":2.4569999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.72199999999999998,"15min":0.752},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73099999999999998,"5min":1.534,"15min":1.534},"last":1.7809999999999999}]},{"osd":10,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.72099999999999997,"15min":0.73099999999999998},"min":{"1min":0.51300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.95099999999999996,"5min":1.163,"15min":1.163},"last":1.522},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.73999999999999999,"15min":0.749},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.84499999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":1.2549999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":491729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.501},"min":{"1min":0.35999999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.88600000000000001,"5min":1.085,"15min":1.085},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52800000000000002,"15min":0.495},"min":{"1min":0.39900000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.96999999999999997,"5min":1.042,"15min":1.042},"last":0.755}]},{"osd":2,"last update":"Sat Sep 9 23:13:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57799999999999996,"15min":0.52000000000000002},"min":{"1min":0.438,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78200000000000003,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.60399999999999998,"15min":0.53900000000000003},"min":{"1min":0.443,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.86599999999999999,"5min":1.133,"15min":1.133},"last":0.54800000000000004}]},{"osd":3,"last update":"Sat Sep 9 23:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.39600000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69699999999999995,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.56999999999999995},"min":{"1min":0.41099999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79500000000000004,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.73099999999999998}]},{"osd":4,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.46700000000000003,"15min":0.42399999999999999},"min":{"1min":0.34300000000000003,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.74099999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.53200000000000003,"15min":0.46400000000000002},"min":{"1min":0.498,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.85499999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.501}]},{"osd":5,"last update":"Sat Sep 9 23:13:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.60899999999999999,"15min":0.56999999999999995},"min":{"1min":0.496,"5min":0.377,"15min":0.377},"max":{"1min":0.872,"5min":1.022,"15min":1.022},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.64200000000000002,"15min":0.58699999999999997},"min":{"1min":0.499,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0129999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.66000000000000003}]},{"osd":6,"last update":"Sat Sep 9 23:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59999999999999998,"15min":0.56999999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89600000000000002,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36099999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.68799999999999994,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.70299999999999996}]},{"osd":7,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.57199999999999995,"15min":0.52800000000000002},"min":{"1min":0.36599999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.55300000000000005,"15min":0.52800000000000002},"min":{"1min":0.42099999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80200000000000005,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.46899999999999997}]},{"osd":8,"last update":"Sat Sep 9 23:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.66200000000000003,"15min":0.65600000000000003},"min":{"1min":0.441,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.69899999999999995,"15min":0.69499999999999995},"min":{"1min":0.502,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.95499999999999996,"5min":1.218,"15min":1.218},"last":0.68899999999999995}]},{"osd":9,"last update":"Sat Sep 9 23:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.76200000000000001,"15min":0.76700000000000002},"min":{"1min":0.53700000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93100000000000005,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.70599999999999996,"15min":0.71199999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.82299999999999995,"5min":1.528,"15min":1.528},"last":0.65200000000000002}]},{"osd":10,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.47299999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.89000000000000001,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.79100000000000004,"15min":0.78300000000000003},"min":{"1min":0.55300000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0009999999999999,"5min":1.355,"15min":1.355},"last":0.622}]},{"osd":11,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.78900000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":435299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 9 23:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53900000000000003,"15min":0.51300000000000001},"min":{"1min":0.39200000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.70299999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.33400000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68200000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.626}]},{"osd":2,"last update":"Sat Sep 9 23:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53800000000000003,"15min":0.52900000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.70599999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55100000000000005,"15min":0.52900000000000003},"min":{"1min":0.42299999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.70499999999999996,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.59299999999999997}]},{"osd":3,"last update":"Sat Sep 9 23:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.45700000000000002}]},{"osd":4,"last update":"Sat Sep 9 23:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55100000000000005,"15min":0.53300000000000003},"min":{"1min":0.375,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73399999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.38300000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.67000000000000004,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.52300000000000002}]},{"osd":5,"last update":"Sat Sep 9 23:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71099999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56799999999999995,"15min":0.56499999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.752,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.67100000000000004}]},{"osd":6,"last update":"Sat Sep 9 23:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.495,"15min":0.497},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.55300000000000005,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.504,"15min":0.501},"min":{"1min":0.33000000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67300000000000004,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.64700000000000002}]},{"osd":7,"last update":"Sat Sep 9 23:11:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.47599999999999998}]},{"osd":8,"last update":"Sat Sep 9 23:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.68700000000000006,"15min":0.69899999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.78000000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.68200000000000005,"15min":0.68999999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85699999999999998,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.71699999999999997}]},{"osd":9,"last update":"Sat Sep 9 23:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.628,"15min":0.65400000000000003},"min":{"1min":0.36799999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.60999999999999999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.67300000000000004,"15min":0.69799999999999995},"min":{"1min":0.34699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70599999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.63600000000000001}]},{"osd":10,"last update":"Sat Sep 9 23:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68200000000000005,"15min":0.68999999999999995},"min":{"1min":0.46800000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81699999999999995,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.67800000000000005,"15min":0.69499999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.71299999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.60599999999999998}]},{"osd":11,"last update":"Sat Sep 9 23:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.73399999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":492730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.44,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.92100000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52300000000000002,"15min":0.51700000000000002},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.63200000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.47999999999999998}]},{"osd":1,"last update":"Sat Sep 9 23:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.59899999999999998,"15min":0.55100000000000005},"min":{"1min":0.38700000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.60499999999999998,"15min":0.53400000000000003},"min":{"1min":0.53300000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.93100000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.86399999999999999}]},{"osd":3,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59999999999999998,"15min":0.60699999999999998},"min":{"1min":0.36199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0009999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61099999999999999,"15min":0.61599999999999999},"min":{"1min":0.46300000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90700000000000003,"5min":1.478,"15min":1.478},"last":0.58099999999999996}]},{"osd":4,"last update":"Sat Sep 9 23:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.89900000000000002}]},{"osd":5,"last update":"Sat Sep 9 23:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.45900000000000002,"15min":0.41599999999999998},"min":{"1min":0.35799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.96899999999999997,"5min":1.014,"15min":1.014},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.46999999999999997,"15min":0.432},"min":{"1min":0.29899999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76900000000000002,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.65800000000000003}]},{"osd":6,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.56999999999999995,"15min":0.53200000000000003},"min":{"1min":0.47799999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.749,"5min":0.63300000000000001,"15min":0.59899999999999998},"min":{"1min":0.48799999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.69699999999999995}]},{"osd":7,"last update":"Sat Sep 9 23:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.65700000000000003}]},{"osd":8,"last update":"Sat Sep 9 23:13:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.626,"15min":0.59499999999999997},"min":{"1min":0.442,"5min":0.37,"15min":0.37},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.63300000000000001,"15min":0.60999999999999999},"min":{"1min":0.56100000000000005,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.878,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.75700000000000001}]},{"osd":9,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.70499999999999996,"15min":0.69099999999999995},"min":{"1min":0.52100000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.752,"15min":0.73699999999999999},"min":{"1min":0.59799999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.73799999999999999}]},{"osd":10,"last update":"Sat Sep 9 23:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.68100000000000005,"15min":0.66700000000000004},"min":{"1min":0.59099999999999997,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.75900000000000001,"15min":0.74099999999999999},"min":{"1min":0.61099999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.873}]},{"osd":11,"last update":"Sat Sep 9 23:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.92300000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182461,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":433466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.32700000000000001,"15min":0.313},"min":{"1min":0.22800000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.501,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35199999999999998,"15min":0.33400000000000002},"min":{"1min":0.249,"5min":0.222,"15min":0.222},"max":{"1min":0.53800000000000003,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.25700000000000001}]},{"osd":1,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54500000000000004,"15min":0.51900000000000002},"min":{"1min":0.32900000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75900000000000001,"5min":1.091,"15min":1.091},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.51000000000000001,"15min":0.51400000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.60799999999999998,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.72699999999999998}]},{"osd":2,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54400000000000004,"15min":0.52000000000000002},"min":{"1min":0.36099999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65100000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53800000000000003,"15min":0.51800000000000002},"min":{"1min":0.40300000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.66200000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.39200000000000002}]},{"osd":4,"last update":"Sat Sep 9 23:13:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54600000000000004,"15min":0.50800000000000001},"min":{"1min":0.39300000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.73599999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.436},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54800000000000004,"15min":0.52000000000000002},"min":{"1min":0.40600000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.65400000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.61099999999999999}]},{"osd":5,"last update":"Sat Sep 9 23:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56499999999999995,"15min":0.57899999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.64200000000000002,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.61199999999999999,"15min":0.60399999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.69499999999999995,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.47099999999999997}]},{"osd":6,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.49099999999999999,"15min":0.49199999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73399999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.377},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.50900000000000001,"15min":0.50600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64800000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.36099999999999999}]},{"osd":7,"last update":"Sat Sep 9 23:13:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.65000000000000002,"15min":0.65900000000000003},"min":{"1min":0.42699999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79000000000000004,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.67500000000000004,"15min":0.69199999999999995},"min":{"1min":0.44800000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.77800000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.63700000000000001}]},{"osd":8,"last update":"Sat Sep 9 23:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.49199999999999999}]},{"osd":9,"last update":"Sat Sep 9 23:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.66500000000000004,"15min":0.68899999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77500000000000002,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.67700000000000005,"15min":0.69899999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.746,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.497}]},{"osd":10,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.42799999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.66200000000000003,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.627},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.71299999999999997,"15min":0.72899999999999998},"min":{"1min":0.505,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.76700000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.68700000000000006}]},{"osd":11,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.55400000000000005}]}]},{"osd":4,"up_from":32,"seq":137438953529,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":434211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.503,"15min":0.47099999999999997},"min":{"1min":0.44900000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.52200000000000002,"15min":0.48799999999999999},"min":{"1min":0.41999999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.33300000000000002}]},{"osd":1,"last update":"Sat Sep 9 23:13:14 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.38,"15min":0.34499999999999997},"min":{"1min":0.313,"5min":0.248,"15min":0.248},"max":{"1min":0.60999999999999999,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.41399999999999998,"15min":0.36799999999999999},"min":{"1min":0.36899999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.46700000000000003}]},{"osd":2,"last update":"Sat Sep 9 23:13:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.55800000000000005,"15min":0.52400000000000002},"min":{"1min":0.433,"5min":0.313,"15min":0.313},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.57799999999999996,"15min":0.53200000000000003},"min":{"1min":0.47799999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.55600000000000005}]},{"osd":3,"last update":"Sat Sep 9 23:13:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.55200000000000005,"15min":0.496},"min":{"1min":0.41999999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78000000000000003,"5min":1.008,"15min":1.008},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53400000000000003,"15min":0.499},"min":{"1min":0.38900000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.91100000000000003,"5min":1.002,"15min":1.002},"last":0.497}]},{"osd":5,"last update":"Sat Sep 9 23:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.60599999999999998,"15min":0.58199999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.874,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.627,"15min":0.57099999999999995},"min":{"1min":0.432,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92600000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.72399999999999998}]},{"osd":6,"last update":"Sat Sep 9 23:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64800000000000002,"15min":0.61799999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.85899999999999999,"5min":1.226,"15min":1.226},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.65600000000000003,"15min":0.62},"min":{"1min":0.40000000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.5219999999999998,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":0.68000000000000005}]},{"osd":7,"last update":"Sat Sep 9 23:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.42299999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.88,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.36399999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.754,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.65900000000000003}]},{"osd":8,"last update":"Sat Sep 9 23:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69199999999999995,"15min":0.68899999999999995},"min":{"1min":0.45700000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.749,"15min":0.751},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.89500000000000002,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.82799999999999996}]},{"osd":9,"last update":"Sat Sep 9 23:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80700000000000005,"15min":0.81200000000000006},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.753},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.75800000000000001,"15min":0.76600000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.92800000000000005,"5min":1.127,"15min":1.127},"last":0.76400000000000001}]},{"osd":10,"last update":"Sat Sep 9 23:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.748,"15min":0.751},"min":{"1min":0.53600000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90200000000000002,"5min":1.208,"15min":1.208},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.66800000000000004,"15min":0.66700000000000004},"min":{"1min":0.48799999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.80900000000000005,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.79100000000000004}]},{"osd":11,"last update":"Sat Sep 9 23:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.85699999999999998}]}]},{"osd":5,"up_from":37,"seq":158913790004,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1536000,"data_stored":1026963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.48399999999999999,"15min":0.46300000000000002},"min":{"1min":0.40500000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.502,"15min":0.49299999999999999},"min":{"1min":0.377,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.58499999999999996}]},{"osd":1,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.52600000000000002,"15min":0.50800000000000001},"min":{"1min":0.39000000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.53300000000000003,"15min":0.51500000000000001},"min":{"1min":0.36899999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.76500000000000001}]},{"osd":2,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.38400000000000001,"15min":0.372},"min":{"1min":0.27700000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.66500000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.433},{"interface":"front","average":{"1min":0.432,"5min":0.39400000000000002,"15min":0.38800000000000001},"min":{"1min":0.309,"5min":0.248,"15min":0.248},"max":{"1min":0.64100000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.48199999999999998}]},{"osd":3,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.38600000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.75,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.54600000000000004,"15min":0.52400000000000002},"min":{"1min":0.41799999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86599999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.66900000000000004}]},{"osd":4,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.55900000000000005,"15min":0.53300000000000003},"min":{"1min":0.40799999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.46700000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.74199999999999999}]},{"osd":6,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56000000000000005,"15min":0.54600000000000004},"min":{"1min":0.38300000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88300000000000001,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.437,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92200000000000004,"5min":0.998,"15min":0.998},"last":0.627}]},{"osd":7,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.627,"15min":0.59899999999999998},"min":{"1min":0.57299999999999995,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.96799999999999997,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.63100000000000001,"15min":0.61299999999999999},"min":{"1min":0.55400000000000005,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.79000000000000004}]},{"osd":8,"last update":"Sat Sep 9 23:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57199999999999995,"15min":0.56399999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83799999999999997,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55200000000000005,"15min":0.54600000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.75600000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.65400000000000003}]},{"osd":9,"last update":"Sat Sep 9 23:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.53700000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.82099999999999995,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.70599999999999996,"15min":0.70299999999999996},"min":{"1min":0.56999999999999995,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93899999999999995,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.84099999999999997}]},{"osd":10,"last update":"Sat Sep 9 23:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.71899999999999997,"15min":0.70199999999999996},"min":{"1min":0.72199999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.68999999999999995,"15min":0.67500000000000004},"min":{"1min":0.61199999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.82799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-09T23:13:44.334 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-09T23:13:44.487 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.488+0000 7f88c7371700 1 -- 172.21.15.82:0/4031957606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88c0101800 msgr2=0x7f88c0101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:44.487 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.488+0000 7f88c7371700 1 --2- 172.21.15.82:0/4031957606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88c0101800 0x7f88c0101c60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0099c0 tx=0x7f88bc009cd0 comp rx=0 tx=0).stop 2023-09-09T23:13:44.487 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.488+0000 7f88c7371700 1 -- 172.21.15.82:0/4031957606 shutdown_connections 2023-09-09T23:13:44.487 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.488+0000 7f88c7371700 1 --2- 172.21.15.82:0/4031957606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88c0100610 0x7f88c0100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:44.488 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.488+0000 7f88c7371700 1 --2- 172.21.15.82:0/4031957606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88c01021a0 0x7f88c0108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:44.488 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.488+0000 7f88c7371700 1 --2- 172.21.15.82:0/4031957606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88c0101800 0x7f88c0101c60 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:44.488 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.488+0000 7f88c7371700 1 -- 172.21.15.82:0/4031957606 >> 172.21.15.82:0/4031957606 conn(0x7f88c00fbb80 msgr2=0x7f88c00fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:44.488 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.488+0000 7f88c7371700 1 -- 172.21.15.82:0/4031957606 shutdown_connections 2023-09-09T23:13:44.488 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.488+0000 7f88c7371700 1 -- 172.21.15.82:0/4031957606 wait complete. 2023-09-09T23:13:44.489 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.489+0000 7f88c7371700 1 Processor -- start 2023-09-09T23:13:44.489 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.489+0000 7f88c7371700 1 -- start start 2023-09-09T23:13:44.491 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.489+0000 7f88c7371700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88c0100610 0x7f88c0196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:44.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.489+0000 7f88c7371700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88c0101800 0x7f88c01967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:44.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.489+0000 7f88c7371700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88c01021a0 0x7f88c019d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:44.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.489+0000 7f88c7371700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c019ddf0 con 0x7f88c01021a0 2023-09-09T23:13:44.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.489+0000 7f88c7371700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c019df30 con 0x7f88c0101800 2023-09-09T23:13:44.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.489+0000 7f88c7371700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c019e0c0 con 0x7f88c0100610 2023-09-09T23:13:44.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.489+0000 7f88c590e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88c01021a0 0x7f88c019d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:44.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.489+0000 7f88c590e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88c01021a0 0x7f88c019d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56130/0 (socket says 172.21.15.82:56130) 2023-09-09T23:13:44.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.489+0000 7f88c590e700 1 -- 172.21.15.82:0/3488082543 learned_addr learned my addr 172.21.15.82:0/3488082543 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:44.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.489+0000 7f88c510d700 1 --2- 172.21.15.82:0/3488082543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88c0100610 0x7f88c0196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:44.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.489+0000 7f88c490c700 1 --2- 172.21.15.82:0/3488082543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88c0101800 0x7f88c01967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:44.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.489+0000 7f88c590e700 1 -- 172.21.15.82:0/3488082543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88c0100610 msgr2=0x7f88c0196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:44.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.489+0000 7f88c590e700 1 --2- 172.21.15.82:0/3488082543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88c0100610 0x7f88c0196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:44.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.489+0000 7f88c590e700 1 -- 172.21.15.82:0/3488082543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88c0101800 msgr2=0x7f88c01967b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:44.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.489+0000 7f88c590e700 1 --2- 172.21.15.82:0/3488082543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88c0101800 0x7f88c01967b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:44.495 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.489+0000 7f88c590e700 1 -- 172.21.15.82:0/3488082543 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88bc0095f0 con 0x7f88c01021a0 2023-09-09T23:13:44.495 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.489+0000 7f88c490c700 1 --2- 172.21.15.82:0/3488082543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88c0101800 0x7f88c01967b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:44.495 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.489+0000 7f88c590e700 1 --2- 172.21.15.82:0/3488082543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88c01021a0 0x7f88c019d820 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f88b8007ee0 tx=0x7f88b800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:44.495 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.490+0000 7f88b67fc700 1 -- 172.21.15.82:0/3488082543 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b8017070 con 0x7f88c01021a0 2023-09-09T23:13:44.495 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.490+0000 7f88b67fc700 1 -- 172.21.15.82:0/3488082543 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88b800f040 con 0x7f88c01021a0 2023-09-09T23:13:44.496 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.490+0000 7f88c7371700 1 -- 172.21.15.82:0/3488082543 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c019e3a0 con 0x7f88c01021a0 2023-09-09T23:13:44.496 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.490+0000 7f88b67fc700 1 -- 172.21.15.82:0/3488082543 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b8005740 con 0x7f88c01021a0 2023-09-09T23:13:44.496 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.490+0000 7f88c7371700 1 -- 172.21.15.82:0/3488082543 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c019e8b0 con 0x7f88c01021a0 2023-09-09T23:13:44.496 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.491+0000 7f88c7371700 1 -- 172.21.15.82:0/3488082543 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c010be30 con 0x7f88c01021a0 2023-09-09T23:13:44.496 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.492+0000 7f88b67fc700 1 -- 172.21.15.82:0/3488082543 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f88b800dc60 con 0x7f88c01021a0 2023-09-09T23:13:44.496 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.492+0000 7f88b67fc700 1 --2- 172.21.15.82:0/3488082543 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f88ac0a0710 0x7f88ac0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:44.497 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.492+0000 7f88c510d700 1 --2- 172.21.15.82:0/3488082543 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f88ac0a0710 0x7f88ac0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:44.497 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.492+0000 7f88b67fc700 1 -- 172.21.15.82:0/3488082543 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f88b80cb270 con 0x7f88c01021a0 2023-09-09T23:13:44.497 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.493+0000 7f88c510d700 1 --2- 172.21.15.82:0/3488082543 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f88ac0a0710 0x7f88ac0a2bd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f88c0101660 tx=0x7f88b000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:44.497 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.496+0000 7f88b67fc700 1 -- 172.21.15.82:0/3488082543 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f88b8094e40 con 0x7f88c01021a0 2023-09-09T23:13:44.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:44 smithi170 ceph-mon[102832]: pgmap v300: 97 pgs: 97 active+clean; 645 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2023-09-09T23:13:44.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:44 smithi170 ceph-mon[102832]: from='client.15486 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-09T23:13:44.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:44 smithi170 ceph-mon[102832]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-09T23:13:44.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:44 smithi170 ceph-mon[102832]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-09T23:13:44.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.689+0000 7f88c7371700 1 -- 172.21.15.82:0/3488082543 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88c01074a0 con 0x7f88c01021a0 2023-09-09T23:13:44.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.691+0000 7f88b67fc700 1 -- 172.21.15.82:0/3488082543 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22411 (secure 0 0 0) 0x7f88b8098460 con 0x7f88c01021a0 2023-09-09T23:13:44.690 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:44.690 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":83,"fsid":"3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da","created":"2023-09-09T23:05:14.222190+0000","modified":"2023-09-09T23:13:10.324596+0000","last_up_change":"2023-09-09T23:11:37.253759+0000","last_in_change":"2023-09-09T23:11:19.933812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-09T23:08:31.234958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-09T23:12:15.524887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-09T23:12:16.785403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1277eb17-a2e7-4e10-9832-f12a5e93ffde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6803","nonce":3197434929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6805","nonce":3197434929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6809","nonce":3197434929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6807","nonce":3197434929}]},"public_addr":"172.21.15.82:6803/3197434929","cluster_addr":"172.21.15.82:6805/3197434929","heartbeat_back_addr":"172.21.15.82:6809/3197434929","heartbeat_front_addr":"172.21.15.82:6807/3197434929","state":["exists","up"]},{"osd":1,"uuid":"f68723fd-0908-4e21-ae8e-1de1c3a9a65c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6801","nonce":3528641314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6803","nonce":3528641314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6807","nonce":3528641314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6805","nonce":3528641314}]},"public_addr":"172.21.15.98:6801/3528641314","cluster_addr":"172.21.15.98:6803/3528641314","heartbeat_back_addr":"172.21.15.98:6807/3528641314","heartbeat_front_addr":"172.21.15.98:6805/3528641314","state":["exists","up"]},{"osd":2,"uuid":"06f89c53-1419-4f52-acad-6e92916b476f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6801","nonce":4006414551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6803","nonce":4006414551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6807","nonce":4006414551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6805","nonce":4006414551}]},"public_addr":"172.21.15.170:6801/4006414551","cluster_addr":"172.21.15.170:6803/4006414551","heartbeat_back_addr":"172.21.15.170:6807/4006414551","heartbeat_front_addr":"172.21.15.170:6805/4006414551","state":["exists","up"]},{"osd":3,"uuid":"fedbd4c9-6852-42b5-8f3a-ba8d9bfe5d59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6811","nonce":3647629241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6813","nonce":3647629241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6817","nonce":3647629241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6815","nonce":3647629241}]},"public_addr":"172.21.15.82:6811/3647629241","cluster_addr":"172.21.15.82:6813/3647629241","heartbeat_back_addr":"172.21.15.82:6817/3647629241","heartbeat_front_addr":"172.21.15.82:6815/3647629241","state":["exists","up"]},{"osd":4,"uuid":"a71f3084-6919-4161-87bf-345f3355d8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6809","nonce":2507035643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6811","nonce":2507035643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6815","nonce":2507035643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6813","nonce":2507035643}]},"public_addr":"172.21.15.98:6809/2507035643","cluster_addr":"172.21.15.98:6811/2507035643","heartbeat_back_addr":"172.21.15.98:6815/2507035643","heartbeat_front_addr":"172.21.15.98:6813/2507035643","state":["exists","up"]},{"osd":5,"uuid":"f76058e9-a90e-476e-87f0-3c6723795e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6809","nonce":775005168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6811","nonce":775005168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6815","nonce":775005168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6813","nonce":775005168}]},"public_addr":"172.21.15.170:6809/775005168","cluster_addr":"172.21.15.170:6811/775005168","heartbeat_back_addr":"172.21.15.170:6815/775005168","heartbeat_front_addr":"172.21.15.170:6813/775005168","state":["exists","up"]},{"osd":6,"uuid":"25940be0-c1bd-4c72-95bf-fd8367aa2f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6819","nonce":3008405982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6821","nonce":3008405982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6825","nonce":3008405982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6823","nonce":3008405982}]},"public_addr":"172.21.15.82:6819/3008405982","cluster_addr":"172.21.15.82:6821/3008405982","heartbeat_back_addr":"172.21.15.82:6825/3008405982","heartbeat_front_addr":"172.21.15.82:6823/3008405982","state":["exists","up"]},{"osd":7,"uuid":"f651bb8c-0b25-42d5-b028-58563f4c27f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6817","nonce":3547615284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6819","nonce":3547615284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6823","nonce":3547615284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6821","nonce":3547615284}]},"public_addr":"172.21.15.98:6817/3547615284","cluster_addr":"172.21.15.98:6819/3547615284","heartbeat_back_addr":"172.21.15.98:6823/3547615284","heartbeat_front_addr":"172.21.15.98:6821/3547615284","state":["exists","up"]},{"osd":8,"uuid":"23b27c86-8c98-441e-b7ab-d57bc584e58f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6817","nonce":632038848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6819","nonce":632038848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6823","nonce":632038848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6821","nonce":632038848}]},"public_addr":"172.21.15.170:6817/632038848","cluster_addr":"172.21.15.170:6819/632038848","heartbeat_back_addr":"172.21.15.170:6823/632038848","heartbeat_front_addr":"172.21.15.170:6821/632038848","state":["exists","up"]},{"osd":9,"uuid":"383b8989-4922-4eee-9c77-f198258ac243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6827","nonce":3354242311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6829","nonce":3354242311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6833","nonce":3354242311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6831","nonce":3354242311}]},"public_addr":"172.21.15.82:6827/3354242311","cluster_addr":"172.21.15.82:6829/3354242311","heartbeat_back_addr":"172.21.15.82:6833/3354242311","heartbeat_front_addr":"172.21.15.82:6831/3354242311","state":["exists","up"]},{"osd":10,"uuid":"33a59389-7ced-42d5-bf54-2071317132d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6825","nonce":2069114959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6827","nonce":2069114959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6831","nonce":2069114959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6829","nonce":2069114959}]},"public_addr":"172.21.15.98:6825/2069114959","cluster_addr":"172.21.15.98:6827/2069114959","heartbeat_back_addr":"172.21.15.98:6831/2069114959","heartbeat_front_addr":"172.21.15.98:6829/2069114959","state":["exists","up"]},{"osd":11,"uuid":"b7a71f5c-85e2-4fec-be4b-95be0788b302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6825","nonce":146964512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6827","nonce":146964512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6831","nonce":146964512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6829","nonce":146964512}]},"public_addr":"172.21.15.170:6825/146964512","cluster_addr":"172.21.15.170:6827/146964512","heartbeat_back_addr":"172.21.15.170:6831/146964512","heartbeat_front_addr":"172.21.15.170:6829/146964512","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:07:47.563043+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:08.359260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:29.070374+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:49.897775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:11.434248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:31.538464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:52.377109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:13.452968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:33.806897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:54.083177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:14.980087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:35.690089+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4013747788":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6800/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/2823341464":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/75116806":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/1037625632":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/3142088817":"2023-09-10T23:05:46.550354+0000","172.21.15.82:6800/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/1679622963":"2023-09-10T23:05:46.550354+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-09T23:13:44.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.693+0000 7f88c7371700 1 -- 172.21.15.82:0/3488082543 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f88ac0a0710 msgr2=0x7f88ac0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:44.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.693+0000 7f88c7371700 1 --2- 172.21.15.82:0/3488082543 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f88ac0a0710 0x7f88ac0a2bd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f88c0101660 tx=0x7f88b000b410 comp rx=0 tx=0).stop 2023-09-09T23:13:44.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.694+0000 7f88c7371700 1 -- 172.21.15.82:0/3488082543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88c01021a0 msgr2=0x7f88c019d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:44.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.694+0000 7f88c7371700 1 --2- 172.21.15.82:0/3488082543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88c01021a0 0x7f88c019d820 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f88b8007ee0 tx=0x7f88b800d3b0 comp rx=0 tx=0).stop 2023-09-09T23:13:44.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.694+0000 7f88c7371700 1 -- 172.21.15.82:0/3488082543 shutdown_connections 2023-09-09T23:13:44.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.694+0000 7f88c7371700 1 --2- 172.21.15.82:0/3488082543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88c01021a0 0x7f88c019d820 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:44.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.694+0000 7f88c7371700 1 --2- 172.21.15.82:0/3488082543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88c0100610 0x7f88c0196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:44.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.694+0000 7f88c7371700 1 --2- 172.21.15.82:0/3488082543 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f88ac0a0710 0x7f88ac0a2bd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:44.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.694+0000 7f88c7371700 1 --2- 172.21.15.82:0/3488082543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88c0101800 0x7f88c01967b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:44.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.694+0000 7f88c7371700 1 -- 172.21.15.82:0/3488082543 >> 172.21.15.82:0/3488082543 conn(0x7f88c00fbb80 msgr2=0x7f88c00fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:44.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.694+0000 7f88c7371700 1 -- 172.21.15.82:0/3488082543 shutdown_connections 2023-09-09T23:13:44.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.694+0000 7f88c7371700 1 -- 172.21.15.82:0/3488082543 wait complete. 2023-09-09T23:13:44.703 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:44 smithi082 ceph-mon[98694]: pgmap v300: 97 pgs: 97 active+clean; 645 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2023-09-09T23:13:44.703 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:44 smithi082 ceph-mon[98694]: from='client.15486 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-09T23:13:44.703 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:44 smithi082 ceph-mon[98694]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-09T23:13:44.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:44 smithi082 ceph-mon[98694]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-09T23:13:44.705 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-09T23:08:31.234958+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-09T23:12:15.524887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-09T23:12:16.785403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-09T23:13:44.706 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-09T23:13:44.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:44 smithi098 ceph-mon[103124]: pgmap v300: 97 pgs: 97 active+clean; 645 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2023-09-09T23:13:44.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:44 smithi098 ceph-mon[103124]: from='client.15486 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-09T23:13:44.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:44 smithi098 ceph-mon[103124]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-09T23:13:44.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:44 smithi098 ceph-mon[103124]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-09T23:13:44.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.832+0000 7f81a21d0700 1 -- 172.21.15.82:0/2152633161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f819c0ff510 msgr2=0x7f819c0ff930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:44.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.832+0000 7f81a21d0700 1 --2- 172.21.15.82:0/2152633161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f819c0ff510 0x7f819c0ff930 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f81900099c0 tx=0x7f8190009cd0 comp rx=0 tx=0).stop 2023-09-09T23:13:44.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.832+0000 7f81a21d0700 1 -- 172.21.15.82:0/2152633161 shutdown_connections 2023-09-09T23:13:44.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.832+0000 7f81a21d0700 1 --2- 172.21.15.82:0/2152633161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f819c1010a0 0x7f819c107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:44.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.832+0000 7f81a21d0700 1 --2- 172.21.15.82:0/2152633161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f819c0ff510 0x7f819c0ff930 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:44.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.832+0000 7f81a21d0700 1 --2- 172.21.15.82:0/2152633161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f819c100700 0x7f819c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:44.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.832+0000 7f81a21d0700 1 -- 172.21.15.82:0/2152633161 >> 172.21.15.82:0/2152633161 conn(0x7f819c0faa80 msgr2=0x7f819c0fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:44.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.832+0000 7f81a21d0700 1 -- 172.21.15.82:0/2152633161 shutdown_connections 2023-09-09T23:13:44.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.832+0000 7f81a21d0700 1 -- 172.21.15.82:0/2152633161 wait complete. 2023-09-09T23:13:44.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.832+0000 7f81a21d0700 1 Processor -- start 2023-09-09T23:13:44.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.832+0000 7f81a21d0700 1 -- start start 2023-09-09T23:13:44.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.833+0000 7f81a21d0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f819c0ff510 0x7f819c069520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:44.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.833+0000 7f81a21d0700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f819c100700 0x7f819c19afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:44.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.833+0000 7f81a21d0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f819c1010a0 0x7f819c19d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:44.836 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.833+0000 7f81a21d0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c19ddd0 con 0x7f819c1010a0 2023-09-09T23:13:44.836 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.833+0000 7f81a21d0700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c19df40 con 0x7f819c100700 2023-09-09T23:13:44.836 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.833+0000 7f81a21d0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c19e100 con 0x7f819c0ff510 2023-09-09T23:13:44.836 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.833+0000 7f819bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f819c1010a0 0x7f819c19d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:44.836 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.833+0000 7f819bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f819c1010a0 0x7f819c19d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56144/0 (socket says 172.21.15.82:56144) 2023-09-09T23:13:44.836 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.833+0000 7f819bfff700 1 -- 172.21.15.82:0/1350905983 learned_addr learned my addr 172.21.15.82:0/1350905983 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:44.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.833+0000 7f819b7fe700 1 --2- 172.21.15.82:0/1350905983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f819c0ff510 0x7f819c069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:44.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.833+0000 7f819affd700 1 --2- 172.21.15.82:0/1350905983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f819c100700 0x7f819c19afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:44.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.833+0000 7f819bfff700 1 -- 172.21.15.82:0/1350905983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f819c0ff510 msgr2=0x7f819c069520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:44.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.833+0000 7f819bfff700 1 --2- 172.21.15.82:0/1350905983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f819c0ff510 0x7f819c069520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:44.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.833+0000 7f819bfff700 1 -- 172.21.15.82:0/1350905983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f819c100700 msgr2=0x7f819c19afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:44.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.833+0000 7f819bfff700 1 --2- 172.21.15.82:0/1350905983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f819c100700 0x7f819c19afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:44.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.833+0000 7f819bfff700 1 -- 172.21.15.82:0/1350905983 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81900095f0 con 0x7f819c1010a0 2023-09-09T23:13:44.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.833+0000 7f819affd700 1 --2- 172.21.15.82:0/1350905983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f819c100700 0x7f819c19afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:44.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.833+0000 7f819bfff700 1 --2- 172.21.15.82:0/1350905983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f819c1010a0 0x7f819c19d600 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f819400ada0 tx=0x7f819400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:44.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.833+0000 7f8198ff9700 1 -- 172.21.15.82:0/1350905983 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819400f070 con 0x7f819c1010a0 2023-09-09T23:13:44.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.833+0000 7f8198ff9700 1 -- 172.21.15.82:0/1350905983 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8194007b30 con 0x7f819c1010a0 2023-09-09T23:13:44.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.833+0000 7f8198ff9700 1 -- 172.21.15.82:0/1350905983 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81940053e0 con 0x7f819c1010a0 2023-09-09T23:13:44.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.833+0000 7f81a21d0700 1 -- 172.21.15.82:0/1350905983 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f819c19e3e0 con 0x7f819c1010a0 2023-09-09T23:13:44.840 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.833+0000 7f81a21d0700 1 -- 172.21.15.82:0/1350905983 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f819c19e8a0 con 0x7f819c1010a0 2023-09-09T23:13:44.840 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.834+0000 7f81a21d0700 1 -- 172.21.15.82:0/1350905983 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f819c10ad20 con 0x7f819c1010a0 2023-09-09T23:13:44.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.835+0000 7f8198ff9700 1 -- 172.21.15.82:0/1350905983 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f819400c980 con 0x7f819c1010a0 2023-09-09T23:13:44.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.836+0000 7f8198ff9700 1 --2- 172.21.15.82:0/1350905983 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f81840a0640 0x7f81840a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:44.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.836+0000 7f8198ff9700 1 -- 172.21.15.82:0/1350905983 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f81940cb120 con 0x7f819c1010a0 2023-09-09T23:13:44.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.836+0000 7f819b7fe700 1 --2- 172.21.15.82:0/1350905983 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f81840a0640 0x7f81840a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:44.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.836+0000 7f819b7fe700 1 --2- 172.21.15.82:0/1350905983 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f81840a0640 0x7f81840a2b00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8190009f50 tx=0x7f819000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:44.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:44.840+0000 7f8198ff9700 1 -- 172.21.15.82:0/1350905983 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8194094d80 con 0x7f819c1010a0 2023-09-09T23:13:45.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.033+0000 7f81a21d0700 1 -- 172.21.15.82:0/1350905983 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f819c106410 con 0x7f819c1010a0 2023-09-09T23:13:45.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.033+0000 7f8198ff9700 1 -- 172.21.15.82:0/1350905983 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f81940983a0 con 0x7f819c1010a0 2023-09-09T23:13:45.032 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2023-09-09T23:13:45.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.035+0000 7f81a21d0700 1 -- 172.21.15.82:0/1350905983 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f81840a0640 msgr2=0x7f81840a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:45.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.035+0000 7f81a21d0700 1 --2- 172.21.15.82:0/1350905983 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f81840a0640 0x7f81840a2b00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8190009f50 tx=0x7f819000b540 comp rx=0 tx=0).stop 2023-09-09T23:13:45.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.035+0000 7f81a21d0700 1 -- 172.21.15.82:0/1350905983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f819c1010a0 msgr2=0x7f819c19d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:45.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.035+0000 7f81a21d0700 1 --2- 172.21.15.82:0/1350905983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f819c1010a0 0x7f819c19d600 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f819400ada0 tx=0x7f819400c330 comp rx=0 tx=0).stop 2023-09-09T23:13:45.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.036+0000 7f81a21d0700 1 -- 172.21.15.82:0/1350905983 shutdown_connections 2023-09-09T23:13:45.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.036+0000 7f81a21d0700 1 --2- 172.21.15.82:0/1350905983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f819c1010a0 0x7f819c19d600 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.036+0000 7f81a21d0700 1 --2- 172.21.15.82:0/1350905983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f819c0ff510 0x7f819c069520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.036+0000 7f81a21d0700 1 --2- 172.21.15.82:0/1350905983 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f81840a0640 0x7f81840a2b00 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.036+0000 7f81a21d0700 1 --2- 172.21.15.82:0/1350905983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f819c100700 0x7f819c19afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.036+0000 7f81a21d0700 1 -- 172.21.15.82:0/1350905983 >> 172.21.15.82:0/1350905983 conn(0x7f819c0faa80 msgr2=0x7f819c0fcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:45.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.036+0000 7f81a21d0700 1 -- 172.21.15.82:0/1350905983 shutdown_connections 2023-09-09T23:13:45.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.036+0000 7f81a21d0700 1 -- 172.21.15.82:0/1350905983 wait complete. 2023-09-09T23:13:45.046 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-09T23:13:45.174 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.173+0000 7f581c7ec700 1 -- 172.21.15.82:0/790536896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58140ff7f0 msgr2=0x7f58140ffc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:45.174 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.173+0000 7f581c7ec700 1 --2- 172.21.15.82:0/790536896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58140ff7f0 0x7f58140ffc50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f58040098f0 tx=0x7f5804009c00 comp rx=0 tx=0).stop 2023-09-09T23:13:45.174 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.174+0000 7f581c7ec700 1 -- 172.21.15.82:0/790536896 shutdown_connections 2023-09-09T23:13:45.175 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.174+0000 7f581c7ec700 1 --2- 172.21.15.82:0/790536896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5814100190 0x7f5814107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.175 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.174+0000 7f581c7ec700 1 --2- 172.21.15.82:0/790536896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5814100b50 0x7f5814100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.175 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.174+0000 7f581c7ec700 1 --2- 172.21.15.82:0/790536896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58140ff7f0 0x7f58140ffc50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.175 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.174+0000 7f581c7ec700 1 -- 172.21.15.82:0/790536896 >> 172.21.15.82:0/790536896 conn(0x7f58140fa260 msgr2=0x7f58140fc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:45.175 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.174+0000 7f581c7ec700 1 -- 172.21.15.82:0/790536896 shutdown_connections 2023-09-09T23:13:45.176 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.174+0000 7f581c7ec700 1 -- 172.21.15.82:0/790536896 wait complete. 2023-09-09T23:13:45.176 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.174+0000 7f581c7ec700 1 Processor -- start 2023-09-09T23:13:45.176 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.174+0000 7f581c7ec700 1 -- start start 2023-09-09T23:13:45.176 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.174+0000 7f581c7ec700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58140ff7f0 0x7f58140ff270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:45.176 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.174+0000 7f581c7ec700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5814100190 0x7f581419b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:45.177 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.174+0000 7f581c7ec700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5814100b50 0x7f581419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:45.177 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.174+0000 7f581c7ec700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581419de80 con 0x7f5814100190 2023-09-09T23:13:45.177 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.174+0000 7f581c7ec700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581419dff0 con 0x7f5814100b50 2023-09-09T23:13:45.177 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.174+0000 7f581c7ec700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581419e1b0 con 0x7f58140ff7f0 2023-09-09T23:13:45.177 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.175+0000 7f581a588700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58140ff7f0 0x7f58140ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:45.178 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.175+0000 7f581a588700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58140ff7f0 0x7f58140ff270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:54670/0 (socket says 172.21.15.82:54670) 2023-09-09T23:13:45.178 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.175+0000 7f581a588700 1 -- 172.21.15.82:0/2613673378 learned_addr learned my addr 172.21.15.82:0/2613673378 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:45.178 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.175+0000 7f5819d87700 1 --2- 172.21.15.82:0/2613673378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5814100190 0x7f581419b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:45.178 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.175+0000 7f581ad89700 1 --2- 172.21.15.82:0/2613673378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5814100b50 0x7f581419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:45.178 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.175+0000 7f581a588700 1 -- 172.21.15.82:0/2613673378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5814100b50 msgr2=0x7f581419d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:45.179 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.175+0000 7f581a588700 1 --2- 172.21.15.82:0/2613673378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5814100b50 0x7f581419d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.179 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.175+0000 7f581a588700 1 -- 172.21.15.82:0/2613673378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5814100190 msgr2=0x7f581419b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:45.179 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.175+0000 7f581a588700 1 --2- 172.21.15.82:0/2613673378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5814100190 0x7f581419b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.179 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.175+0000 7f581a588700 1 -- 172.21.15.82:0/2613673378 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5804009520 con 0x7f58140ff7f0 2023-09-09T23:13:45.179 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.175+0000 7f581ad89700 1 --2- 172.21.15.82:0/2613673378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5814100b50 0x7f581419d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:45.180 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.175+0000 7f581a588700 1 --2- 172.21.15.82:0/2613673378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58140ff7f0 0x7f58140ff270 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f581000e9f0 tx=0x7f581000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:45.180 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.175+0000 7f580b7fe700 1 -- 172.21.15.82:0/2613673378 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5810004220 con 0x7f58140ff7f0 2023-09-09T23:13:45.180 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.175+0000 7f581c7ec700 1 -- 172.21.15.82:0/2613673378 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f581419e490 con 0x7f58140ff7f0 2023-09-09T23:13:45.180 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.175+0000 7f581c7ec700 1 -- 172.21.15.82:0/2613673378 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581419e9c0 con 0x7f58140ff7f0 2023-09-09T23:13:45.180 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.176+0000 7f580b7fe700 1 -- 172.21.15.82:0/2613673378 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5810004380 con 0x7f58140ff7f0 2023-09-09T23:13:45.180 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.176+0000 7f580b7fe700 1 -- 172.21.15.82:0/2613673378 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5810003be0 con 0x7f58140ff7f0 2023-09-09T23:13:45.181 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.176+0000 7f581c7ec700 1 -- 172.21.15.82:0/2613673378 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f8005320 con 0x7f58140ff7f0 2023-09-09T23:13:45.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.178+0000 7f580b7fe700 1 -- 172.21.15.82:0/2613673378 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f5810003d40 con 0x7f58140ff7f0 2023-09-09T23:13:45.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.179+0000 7f580b7fe700 1 --2- 172.21.15.82:0/2613673378 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f58000a06c0 0x7f58000a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:45.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.179+0000 7f5819d87700 1 --2- 172.21.15.82:0/2613673378 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f58000a06c0 0x7f58000a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:45.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.179+0000 7f580b7fe700 1 -- 172.21.15.82:0/2613673378 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f5810014070 con 0x7f58140ff7f0 2023-09-09T23:13:45.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.179+0000 7f5819d87700 1 --2- 172.21.15.82:0/2613673378 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f58000a06c0 0x7f58000a2b80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f580400fa40 tx=0x7f5804019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:45.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.182+0000 7f580b7fe700 1 -- 172.21.15.82:0/2613673378 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5810095880 con 0x7f58140ff7f0 2023-09-09T23:13:45.374 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.376+0000 7f581c7ec700 1 -- 172.21.15.82:0/2613673378 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f57f8005b50 con 0x7f58140ff7f0 2023-09-09T23:13:45.375 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.376+0000 7f580b7fe700 1 -- 172.21.15.82:0/2613673378 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5810098ea0 con 0x7f58140ff7f0 2023-09-09T23:13:45.375 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2023-09-09T23:13:45.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.378+0000 7f581c7ec700 1 -- 172.21.15.82:0/2613673378 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f58000a06c0 msgr2=0x7f58000a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:45.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.378+0000 7f581c7ec700 1 --2- 172.21.15.82:0/2613673378 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f58000a06c0 0x7f58000a2b80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f580400fa40 tx=0x7f5804019040 comp rx=0 tx=0).stop 2023-09-09T23:13:45.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.378+0000 7f581c7ec700 1 -- 172.21.15.82:0/2613673378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58140ff7f0 msgr2=0x7f58140ff270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:45.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.378+0000 7f581c7ec700 1 --2- 172.21.15.82:0/2613673378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58140ff7f0 0x7f58140ff270 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f581000e9f0 tx=0x7f581000ed00 comp rx=0 tx=0).stop 2023-09-09T23:13:45.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.378+0000 7f581c7ec700 1 -- 172.21.15.82:0/2613673378 shutdown_connections 2023-09-09T23:13:45.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.378+0000 7f581c7ec700 1 --2- 172.21.15.82:0/2613673378 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f58000a06c0 0x7f58000a2b80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.378+0000 7f581c7ec700 1 --2- 172.21.15.82:0/2613673378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5814100b50 0x7f581419d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.378+0000 7f581c7ec700 1 --2- 172.21.15.82:0/2613673378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58140ff7f0 0x7f58140ff270 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.378+0000 7f581c7ec700 1 --2- 172.21.15.82:0/2613673378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5814100190 0x7f581419b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.379+0000 7f581c7ec700 1 -- 172.21.15.82:0/2613673378 >> 172.21.15.82:0/2613673378 conn(0x7f58140fa260 msgr2=0x7f58140fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:45.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.379+0000 7f581c7ec700 1 -- 172.21.15.82:0/2613673378 shutdown_connections 2023-09-09T23:13:45.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.379+0000 7f581c7ec700 1 -- 172.21.15.82:0/2613673378 wait complete. 2023-09-09T23:13:45.389 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-09T23:13:45.515 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.515+0000 7facde8cb700 1 -- 172.21.15.82:0/866123657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facd8100750 msgr2=0x7facd8100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:45.516 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.515+0000 7facde8cb700 1 --2- 172.21.15.82:0/866123657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facd8100750 0x7facd8100bd0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7facc8009990 tx=0x7facc8009ca0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.516 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.515+0000 7facde8cb700 1 -- 172.21.15.82:0/866123657 shutdown_connections 2023-09-09T23:13:45.516 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.515+0000 7facde8cb700 1 --2- 172.21.15.82:0/866123657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facd8101110 0x7facd81079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.516 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.515+0000 7facde8cb700 1 --2- 172.21.15.82:0/866123657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facd80ffa40 0x7facd80ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.517 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.515+0000 7facde8cb700 1 --2- 172.21.15.82:0/866123657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facd8100750 0x7facd8100bd0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.517 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.515+0000 7facde8cb700 1 -- 172.21.15.82:0/866123657 >> 172.21.15.82:0/866123657 conn(0x7facd80fa1f0 msgr2=0x7facd80fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:45.517 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.515+0000 7facde8cb700 1 -- 172.21.15.82:0/866123657 shutdown_connections 2023-09-09T23:13:45.517 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.515+0000 7facde8cb700 1 -- 172.21.15.82:0/866123657 wait complete. 2023-09-09T23:13:45.517 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.515+0000 7facde8cb700 1 Processor -- start 2023-09-09T23:13:45.518 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.515+0000 7facde8cb700 1 -- start start 2023-09-09T23:13:45.518 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.516+0000 7facde8cb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facd80ffa40 0x7facd8195200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:45.518 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.516+0000 7facde8cb700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facd8100750 0x7facd8195740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:45.518 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.516+0000 7facde8cb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facd8101110 0x7facd819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:45.518 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.516+0000 7facde8cb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd819cd80 con 0x7facd80ffa40 2023-09-09T23:13:45.519 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.516+0000 7facde8cb700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd819cec0 con 0x7facd8100750 2023-09-09T23:13:45.519 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.516+0000 7facde8cb700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd819d050 con 0x7facd8101110 2023-09-09T23:13:45.519 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.516+0000 7facd7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facd80ffa40 0x7facd8195200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:45.519 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.516+0000 7facd7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facd80ffa40 0x7facd8195200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56170/0 (socket says 172.21.15.82:56170) 2023-09-09T23:13:45.519 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.516+0000 7facd7fff700 1 -- 172.21.15.82:0/279599879 learned_addr learned my addr 172.21.15.82:0/279599879 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:45.520 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.516+0000 7facd77fe700 1 --2- 172.21.15.82:0/279599879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facd8100750 0x7facd8195740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:45.520 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.516+0000 7facdce68700 1 --2- 172.21.15.82:0/279599879 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facd8101110 0x7facd819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:45.520 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.516+0000 7facd7fff700 1 -- 172.21.15.82:0/279599879 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facd8101110 msgr2=0x7facd819c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:45.520 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.516+0000 7facd7fff700 1 --2- 172.21.15.82:0/279599879 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facd8101110 0x7facd819c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.520 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.516+0000 7facd7fff700 1 -- 172.21.15.82:0/279599879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facd8100750 msgr2=0x7facd8195740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:45.521 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.516+0000 7facd7fff700 1 --2- 172.21.15.82:0/279599879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facd8100750 0x7facd8195740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.521 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.516+0000 7facd7fff700 1 -- 172.21.15.82:0/279599879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc80095f0 con 0x7facd80ffa40 2023-09-09T23:13:45.521 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.516+0000 7facd7fff700 1 --2- 172.21.15.82:0/279599879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facd80ffa40 0x7facd8195200 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7faccc00b500 tx=0x7faccc00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:45.521 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.516+0000 7facd57fa700 1 -- 172.21.15.82:0/279599879 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc011830 con 0x7facd80ffa40 2023-09-09T23:13:45.521 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.516+0000 7facd57fa700 1 -- 172.21.15.82:0/279599879 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faccc011e50 con 0x7facd80ffa40 2023-09-09T23:13:45.521 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.516+0000 7facde8cb700 1 -- 172.21.15.82:0/279599879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd819d330 con 0x7facd80ffa40 2023-09-09T23:13:45.522 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.516+0000 7facd57fa700 1 -- 172.21.15.82:0/279599879 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc00f2e0 con 0x7facd80ffa40 2023-09-09T23:13:45.522 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.516+0000 7facde8cb700 1 -- 172.21.15.82:0/279599879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd819d8c0 con 0x7facd80ffa40 2023-09-09T23:13:45.522 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.517+0000 7facde8cb700 1 -- 172.21.15.82:0/279599879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facd810adf0 con 0x7facd80ffa40 2023-09-09T23:13:45.523 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.518+0000 7facd57fa700 1 -- 172.21.15.82:0/279599879 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7faccc010580 con 0x7facd80ffa40 2023-09-09T23:13:45.523 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.519+0000 7facd57fa700 1 --2- 172.21.15.82:0/279599879 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7facc00a0640 0x7facc00a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:45.523 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.519+0000 7facd57fa700 1 -- 172.21.15.82:0/279599879 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7faccc0cbb60 con 0x7facd80ffa40 2023-09-09T23:13:45.524 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.519+0000 7facd77fe700 1 --2- 172.21.15.82:0/279599879 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7facc00a0640 0x7facc00a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:45.524 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.519+0000 7facd77fe700 1 --2- 172.21.15.82:0/279599879 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7facc00a0640 0x7facc00a2b00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7facc8006010 tx=0x7facc8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:45.524 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.522+0000 7facd57fa700 1 -- 172.21.15.82:0/279599879 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7faccc095690 con 0x7facd80ffa40 2023-09-09T23:13:45.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:45 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3488082543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:45.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:45 smithi170 ceph-mon[102832]: pgmap v301: 97 pgs: 97 active+clean; 645 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-09T23:13:45.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:45 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1350905983' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:13:45.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:45 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3488082543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:45.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:45 smithi082 ceph-mon[98694]: pgmap v301: 97 pgs: 97 active+clean; 645 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-09T23:13:45.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:45 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1350905983' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:13:45.715 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.717+0000 7facde8cb700 1 -- 172.21.15.82:0/279599879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7facd819db40 con 0x7facd80ffa40 2023-09-09T23:13:45.715 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.717+0000 7facd57fa700 1 -- 172.21.15.82:0/279599879 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7faccc098cb0 con 0x7facd80ffa40 2023-09-09T23:13:45.716 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2023-09-09T23:13:45.718 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.719+0000 7facde8cb700 1 -- 172.21.15.82:0/279599879 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7facc00a0640 msgr2=0x7facc00a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:45.718 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.719+0000 7facde8cb700 1 --2- 172.21.15.82:0/279599879 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7facc00a0640 0x7facc00a2b00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7facc8006010 tx=0x7facc8005960 comp rx=0 tx=0).stop 2023-09-09T23:13:45.718 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.719+0000 7facde8cb700 1 -- 172.21.15.82:0/279599879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facd80ffa40 msgr2=0x7facd8195200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:45.719 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.719+0000 7facde8cb700 1 --2- 172.21.15.82:0/279599879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facd80ffa40 0x7facd8195200 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7faccc00b500 tx=0x7faccc00b810 comp rx=0 tx=0).stop 2023-09-09T23:13:45.719 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.720+0000 7facde8cb700 1 -- 172.21.15.82:0/279599879 shutdown_connections 2023-09-09T23:13:45.719 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.720+0000 7facde8cb700 1 --2- 172.21.15.82:0/279599879 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facd8101110 0x7facd819c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.719 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.720+0000 7facde8cb700 1 --2- 172.21.15.82:0/279599879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facd80ffa40 0x7facd8195200 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.719 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.720+0000 7facde8cb700 1 --2- 172.21.15.82:0/279599879 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7facc00a0640 0x7facc00a2b00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.720 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.720+0000 7facde8cb700 1 --2- 172.21.15.82:0/279599879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facd8100750 0x7facd8195740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.720 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.720+0000 7facde8cb700 1 -- 172.21.15.82:0/279599879 >> 172.21.15.82:0/279599879 conn(0x7facd80fa1f0 msgr2=0x7facd80fc400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:45.720 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.720+0000 7facde8cb700 1 -- 172.21.15.82:0/279599879 shutdown_connections 2023-09-09T23:13:45.720 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.720+0000 7facde8cb700 1 -- 172.21.15.82:0/279599879 wait complete. 2023-09-09T23:13:45.730 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-09T23:13:45.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:45 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3488082543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:45.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:45 smithi098 ceph-mon[103124]: pgmap v301: 97 pgs: 97 active+clean; 645 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-09T23:13:45.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:45 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1350905983' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-09T23:13:45.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.852+0000 7ffb55923700 1 -- 172.21.15.82:0/3149736217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb50100710 msgr2=0x7ffb50100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:45.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.852+0000 7ffb55923700 1 --2- 172.21.15.82:0/3149736217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb50100710 0x7ffb50100b70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ffb400099c0 tx=0x7ffb40009cd0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.852+0000 7ffb55923700 1 -- 172.21.15.82:0/3149736217 shutdown_connections 2023-09-09T23:13:45.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.852+0000 7ffb55923700 1 --2- 172.21.15.82:0/3149736217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb501010b0 0x7ffb50107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.852+0000 7ffb55923700 1 --2- 172.21.15.82:0/3149736217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb500ff520 0x7ffb500ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.852+0000 7ffb55923700 1 --2- 172.21.15.82:0/3149736217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb50100710 0x7ffb50100b70 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.852+0000 7ffb55923700 1 -- 172.21.15.82:0/3149736217 >> 172.21.15.82:0/3149736217 conn(0x7ffb500faa90 msgr2=0x7ffb500fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:45.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.852+0000 7ffb55923700 1 -- 172.21.15.82:0/3149736217 shutdown_connections 2023-09-09T23:13:45.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.852+0000 7ffb55923700 1 -- 172.21.15.82:0/3149736217 wait complete. 2023-09-09T23:13:45.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.853+0000 7ffb55923700 1 Processor -- start 2023-09-09T23:13:45.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.853+0000 7ffb55923700 1 -- start start 2023-09-09T23:13:45.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.853+0000 7ffb55923700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb500ff520 0x7ffb50198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:45.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.853+0000 7ffb55923700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb50100710 0x7ffb501989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:45.856 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.853+0000 7ffb55923700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb501010b0 0x7ffb5019fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:45.856 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.853+0000 7ffb55923700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb501a0000 con 0x7ffb50100710 2023-09-09T23:13:45.856 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.853+0000 7ffb55923700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb501a0140 con 0x7ffb500ff520 2023-09-09T23:13:45.856 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.853+0000 7ffb55923700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb501a02d0 con 0x7ffb501010b0 2023-09-09T23:13:45.856 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.853+0000 7ffb4e7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb50100710 0x7ffb501989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:45.856 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.853+0000 7ffb4e7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb50100710 0x7ffb501989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56186/0 (socket says 172.21.15.82:56186) 2023-09-09T23:13:45.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.853+0000 7ffb4e7fc700 1 -- 172.21.15.82:0/1163975145 learned_addr learned my addr 172.21.15.82:0/1163975145 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:45.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.853+0000 7ffb4effd700 1 --2- 172.21.15.82:0/1163975145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb500ff520 0x7ffb50198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:45.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.853+0000 7ffb4e7fc700 1 -- 172.21.15.82:0/1163975145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb501010b0 msgr2=0x7ffb5019fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:45.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.853+0000 7ffb4f7fe700 1 --2- 172.21.15.82:0/1163975145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb501010b0 0x7ffb5019fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:45.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.854+0000 7ffb4e7fc700 1 --2- 172.21.15.82:0/1163975145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb501010b0 0x7ffb5019fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.854+0000 7ffb4e7fc700 1 -- 172.21.15.82:0/1163975145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb500ff520 msgr2=0x7ffb50198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:45.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.854+0000 7ffb4e7fc700 1 --2- 172.21.15.82:0/1163975145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb500ff520 0x7ffb50198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:45.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.854+0000 7ffb4e7fc700 1 -- 172.21.15.82:0/1163975145 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb400095f0 con 0x7ffb50100710 2023-09-09T23:13:45.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.854+0000 7ffb4effd700 1 --2- 172.21.15.82:0/1163975145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb500ff520 0x7ffb50198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:45.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.854+0000 7ffb4e7fc700 1 --2- 172.21.15.82:0/1163975145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb50100710 0x7ffb501989c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ffb40009990 tx=0x7ffb40003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.854+0000 7ffb54921700 1 -- 172.21.15.82:0/1163975145 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4001d070 con 0x7ffb50100710 2023-09-09T23:13:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.854+0000 7ffb54921700 1 -- 172.21.15.82:0/1163975145 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb40004640 con 0x7ffb50100710 2023-09-09T23:13:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.854+0000 7ffb55923700 1 -- 172.21.15.82:0/1163975145 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb501a0550 con 0x7ffb50100710 2023-09-09T23:13:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.854+0000 7ffb54921700 1 -- 172.21.15.82:0/1163975145 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4000f460 con 0x7ffb50100710 2023-09-09T23:13:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.854+0000 7ffb55923700 1 -- 172.21.15.82:0/1163975145 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb501a0a50 con 0x7ffb50100710 2023-09-09T23:13:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.855+0000 7ffb55923700 1 -- 172.21.15.82:0/1163975145 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb5010add0 con 0x7ffb50100710 2023-09-09T23:13:45.860 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.856+0000 7ffb54921700 1 -- 172.21.15.82:0/1163975145 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7ffb40022470 con 0x7ffb50100710 2023-09-09T23:13:45.860 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.856+0000 7ffb54921700 1 --2- 172.21.15.82:0/1163975145 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ffb3c0a0710 0x7ffb3c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:45.860 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.856+0000 7ffb4effd700 1 --2- 172.21.15.82:0/1163975145 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ffb3c0a0710 0x7ffb3c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:45.860 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.856+0000 7ffb54921700 1 -- 172.21.15.82:0/1163975145 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ffb400cd0d0 con 0x7ffb50100710 2023-09-09T23:13:45.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.857+0000 7ffb4effd700 1 --2- 172.21.15.82:0/1163975145 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ffb3c0a0710 0x7ffb3c0a2bd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ffb50100570 tx=0x7ffb38008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:45.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:45.861+0000 7ffb54921700 1 -- 172.21.15.82:0/1163975145 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ffb40096e50 con 0x7ffb50100710 2023-09-09T23:13:46.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.099+0000 7ffb55923700 1 -- 172.21.15.82:0/1163975145 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb5010afc0 con 0x7ffb50100710 2023-09-09T23:13:46.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.100+0000 7ffb54921700 1 -- 172.21.15.82:0/1163975145 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11368 (secure 0 0 0) 0x7ffb40027090 con 0x7ffb50100710 2023-09-09T23:13:46.099 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:46.099 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2471432684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2471432684},{"type":"v1","addr":"172.21.15.82:6835","nonce":2471432684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3579387137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3579387137},{"type":"v1","addr":"172.21.15.82:6837","nonce":3579387137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-09T23:12:17.586036+0000","modified":"2023-09-09T23:12:25.285958+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24542,"mds_2":24523,"mds_3":24491,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14712":{"gid":14712,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/2127301345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2127301345},{"type":"v1","addr":"172.21.15.82:6839","nonce":2127301345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14724":{"gid":14724,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/4143604488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":4143604488},{"type":"v1","addr":"172.21.15.82:6841","nonce":4143604488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.170:6833/1931961940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1931961940},{"type":"v1","addr":"172.21.15.170:6833","nonce":1931961940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24491":{"gid":24491,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.98:6833/3855146755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3855146755},{"type":"v1","addr":"172.21.15.98:6833","nonce":3855146755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2313861511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2313861511},{"type":"v1","addr":"172.21.15.170:6835","nonce":2313861511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/4006360308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":4006360308},{"type":"v1","addr":"172.21.15.170:6837","nonce":4006360308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1356474828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1356474828},{"type":"v1","addr":"172.21.15.98:6835","nonce":1356474828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/3858369735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3858369735},{"type":"v1","addr":"172.21.15.170:6839","nonce":3858369735}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/2019449840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2019449840},{"type":"v1","addr":"172.21.15.98:6837","nonce":2019449840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2785086371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2785086371},{"type":"v1","addr":"172.21.15.98:6839","nonce":2785086371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-09T23:13:46.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.103+0000 7ffb55923700 1 -- 172.21.15.82:0/1163975145 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ffb3c0a0710 msgr2=0x7ffb3c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.103+0000 7ffb55923700 1 --2- 172.21.15.82:0/1163975145 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ffb3c0a0710 0x7ffb3c0a2bd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ffb50100570 tx=0x7ffb38008040 comp rx=0 tx=0).stop 2023-09-09T23:13:46.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.103+0000 7ffb55923700 1 -- 172.21.15.82:0/1163975145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb50100710 msgr2=0x7ffb501989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.103+0000 7ffb55923700 1 --2- 172.21.15.82:0/1163975145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb50100710 0x7ffb501989c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ffb40009990 tx=0x7ffb40003ba0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.103+0000 7ffb55923700 1 -- 172.21.15.82:0/1163975145 shutdown_connections 2023-09-09T23:13:46.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.103+0000 7ffb55923700 1 --2- 172.21.15.82:0/1163975145 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ffb3c0a0710 0x7ffb3c0a2bd0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.103+0000 7ffb55923700 1 --2- 172.21.15.82:0/1163975145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb500ff520 0x7ffb50198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.103+0000 7ffb55923700 1 --2- 172.21.15.82:0/1163975145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb501010b0 0x7ffb5019fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.103+0000 7ffb55923700 1 --2- 172.21.15.82:0/1163975145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb50100710 0x7ffb501989c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.103+0000 7ffb55923700 1 -- 172.21.15.82:0/1163975145 >> 172.21.15.82:0/1163975145 conn(0x7ffb500faa90 msgr2=0x7ffb500687b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:46.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.103+0000 7ffb55923700 1 -- 172.21.15.82:0/1163975145 shutdown_connections 2023-09-09T23:13:46.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.103+0000 7ffb55923700 1 -- 172.21.15.82:0/1163975145 wait complete. 2023-09-09T23:13:46.104 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 27 2023-09-09T23:13:46.113 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-09T23:13:46.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.236+0000 7fb2b83dd700 1 -- 172.21.15.82:0/719395350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2b01000e0 msgr2=0x7fb2b0107980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.236+0000 7fb2b83dd700 1 --2- 172.21.15.82:0/719395350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2b01000e0 0x7fb2b0107980 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb2a800b0a0 tx=0x7fb2a800b3b0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.237+0000 7fb2b83dd700 1 -- 172.21.15.82:0/719395350 shutdown_connections 2023-09-09T23:13:46.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.237+0000 7fb2b83dd700 1 --2- 172.21.15.82:0/719395350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2b01000e0 0x7fb2b0107980 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.237+0000 7fb2b83dd700 1 --2- 172.21.15.82:0/719395350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2b0100ad0 0x7fb2b0100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.237+0000 7fb2b83dd700 1 --2- 172.21.15.82:0/719395350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2b00ff720 0x7fb2b00ffba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.237+0000 7fb2b83dd700 1 -- 172.21.15.82:0/719395350 >> 172.21.15.82:0/719395350 conn(0x7fb2b00fa1d0 msgr2=0x7fb2b00fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:46.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.237+0000 7fb2b83dd700 1 -- 172.21.15.82:0/719395350 shutdown_connections 2023-09-09T23:13:46.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.237+0000 7fb2b83dd700 1 -- 172.21.15.82:0/719395350 wait complete. 2023-09-09T23:13:46.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.237+0000 7fb2b83dd700 1 Processor -- start 2023-09-09T23:13:46.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.237+0000 7fb2b83dd700 1 -- start start 2023-09-09T23:13:46.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.237+0000 7fb2b83dd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2b00ff720 0x7fb2b0195260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.237+0000 7fb2b83dd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2b01000e0 0x7fb2b01957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.237+0000 7fb2b83dd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2b0100ad0 0x7fb2b019c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.237+0000 7fb2b83dd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b019cde0 con 0x7fb2b01000e0 2023-09-09T23:13:46.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.237+0000 7fb2b83dd700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b019cf20 con 0x7fb2b0100ad0 2023-09-09T23:13:46.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.237+0000 7fb2b83dd700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b019d0b0 con 0x7fb2b00ff720 2023-09-09T23:13:46.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.238+0000 7fb2b5978700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2b01000e0 0x7fb2b01957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.238+0000 7fb2b5978700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2b01000e0 0x7fb2b01957a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56200/0 (socket says 172.21.15.82:56200) 2023-09-09T23:13:46.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.238+0000 7fb2b5978700 1 -- 172.21.15.82:0/4055047989 learned_addr learned my addr 172.21.15.82:0/4055047989 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:46.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.238+0000 7fb2b697a700 1 --2- 172.21.15.82:0/4055047989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2b0100ad0 0x7fb2b019c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.238+0000 7fb2b6179700 1 --2- 172.21.15.82:0/4055047989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2b00ff720 0x7fb2b0195260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.238+0000 7fb2b5978700 1 -- 172.21.15.82:0/4055047989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2b00ff720 msgr2=0x7fb2b0195260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.238+0000 7fb2b5978700 1 --2- 172.21.15.82:0/4055047989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2b00ff720 0x7fb2b0195260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.238+0000 7fb2b5978700 1 -- 172.21.15.82:0/4055047989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2b0100ad0 msgr2=0x7fb2b019c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.238+0000 7fb2b5978700 1 --2- 172.21.15.82:0/4055047989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2b0100ad0 0x7fb2b019c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.238+0000 7fb2b5978700 1 -- 172.21.15.82:0/4055047989 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a8009d00 con 0x7fb2b01000e0 2023-09-09T23:13:46.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.238+0000 7fb2b5978700 1 --2- 172.21.15.82:0/4055047989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2b01000e0 0x7fb2b01957a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb2a00098f0 tx=0x7fb2a0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:46.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.238+0000 7fb2a77fe700 1 -- 172.21.15.82:0/4055047989 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0012070 con 0x7fb2b01000e0 2023-09-09T23:13:46.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.238+0000 7fb2a77fe700 1 -- 172.21.15.82:0/4055047989 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2a00044e0 con 0x7fb2b01000e0 2023-09-09T23:13:46.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.238+0000 7fb2b83dd700 1 -- 172.21.15.82:0/4055047989 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b00fe670 con 0x7fb2b01000e0 2023-09-09T23:13:46.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.238+0000 7fb2a77fe700 1 -- 172.21.15.82:0/4055047989 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0003df0 con 0x7fb2b01000e0 2023-09-09T23:13:46.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.238+0000 7fb2b83dd700 1 -- 172.21.15.82:0/4055047989 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b00febd0 con 0x7fb2b01000e0 2023-09-09T23:13:46.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.239+0000 7fb2b83dd700 1 -- 172.21.15.82:0/4055047989 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2b010ad40 con 0x7fb2b01000e0 2023-09-09T23:13:46.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.240+0000 7fb2a77fe700 1 -- 172.21.15.82:0/4055047989 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fb2a0016030 con 0x7fb2b01000e0 2023-09-09T23:13:46.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.241+0000 7fb2a77fe700 1 --2- 172.21.15.82:0/4055047989 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb29c0a0710 0x7fb29c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.241+0000 7fb2b6179700 1 --2- 172.21.15.82:0/4055047989 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb29c0a0710 0x7fb29c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.241+0000 7fb2a77fe700 1 -- 172.21.15.82:0/4055047989 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb2a00ccf30 con 0x7fb2b01000e0 2023-09-09T23:13:46.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.241+0000 7fb2b6179700 1 --2- 172.21.15.82:0/4055047989 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb29c0a0710 0x7fb29c0a2bd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0068450 tx=0x7fb2ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:46.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.245+0000 7fb2a77fe700 1 -- 172.21.15.82:0/4055047989 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb2a0096b00 con 0x7fb2b01000e0 2023-09-09T23:13:46.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.441+0000 7fb2b83dd700 1 -- 172.21.15.82:0/4055047989 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2b010af00 con 0x7fb2b01000e0 2023-09-09T23:13:46.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.442+0000 7fb2a77fe700 1 -- 172.21.15.82:0/4055047989 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22411 (secure 0 0 0) 0x7fb2a009a120 con 0x7fb2b01000e0 2023-09-09T23:13:46.441 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:46.441 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":83,"fsid":"3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da","created":"2023-09-09T23:05:14.222190+0000","modified":"2023-09-09T23:13:10.324596+0000","last_up_change":"2023-09-09T23:11:37.253759+0000","last_in_change":"2023-09-09T23:11:19.933812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-09T23:08:31.234958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-09T23:12:15.524887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-09T23:12:16.785403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1277eb17-a2e7-4e10-9832-f12a5e93ffde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6803","nonce":3197434929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6805","nonce":3197434929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6809","nonce":3197434929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3197434929},{"type":"v1","addr":"172.21.15.82:6807","nonce":3197434929}]},"public_addr":"172.21.15.82:6803/3197434929","cluster_addr":"172.21.15.82:6805/3197434929","heartbeat_back_addr":"172.21.15.82:6809/3197434929","heartbeat_front_addr":"172.21.15.82:6807/3197434929","state":["exists","up"]},{"osd":1,"uuid":"f68723fd-0908-4e21-ae8e-1de1c3a9a65c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6801","nonce":3528641314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6803","nonce":3528641314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6807","nonce":3528641314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3528641314},{"type":"v1","addr":"172.21.15.98:6805","nonce":3528641314}]},"public_addr":"172.21.15.98:6801/3528641314","cluster_addr":"172.21.15.98:6803/3528641314","heartbeat_back_addr":"172.21.15.98:6807/3528641314","heartbeat_front_addr":"172.21.15.98:6805/3528641314","state":["exists","up"]},{"osd":2,"uuid":"06f89c53-1419-4f52-acad-6e92916b476f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6801","nonce":4006414551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6803","nonce":4006414551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6807","nonce":4006414551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":4006414551},{"type":"v1","addr":"172.21.15.170:6805","nonce":4006414551}]},"public_addr":"172.21.15.170:6801/4006414551","cluster_addr":"172.21.15.170:6803/4006414551","heartbeat_back_addr":"172.21.15.170:6807/4006414551","heartbeat_front_addr":"172.21.15.170:6805/4006414551","state":["exists","up"]},{"osd":3,"uuid":"fedbd4c9-6852-42b5-8f3a-ba8d9bfe5d59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6811","nonce":3647629241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6813","nonce":3647629241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6817","nonce":3647629241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3647629241},{"type":"v1","addr":"172.21.15.82:6815","nonce":3647629241}]},"public_addr":"172.21.15.82:6811/3647629241","cluster_addr":"172.21.15.82:6813/3647629241","heartbeat_back_addr":"172.21.15.82:6817/3647629241","heartbeat_front_addr":"172.21.15.82:6815/3647629241","state":["exists","up"]},{"osd":4,"uuid":"a71f3084-6919-4161-87bf-345f3355d8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6809","nonce":2507035643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6811","nonce":2507035643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6815","nonce":2507035643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2507035643},{"type":"v1","addr":"172.21.15.98:6813","nonce":2507035643}]},"public_addr":"172.21.15.98:6809/2507035643","cluster_addr":"172.21.15.98:6811/2507035643","heartbeat_back_addr":"172.21.15.98:6815/2507035643","heartbeat_front_addr":"172.21.15.98:6813/2507035643","state":["exists","up"]},{"osd":5,"uuid":"f76058e9-a90e-476e-87f0-3c6723795e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6809","nonce":775005168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6811","nonce":775005168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6815","nonce":775005168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":775005168},{"type":"v1","addr":"172.21.15.170:6813","nonce":775005168}]},"public_addr":"172.21.15.170:6809/775005168","cluster_addr":"172.21.15.170:6811/775005168","heartbeat_back_addr":"172.21.15.170:6815/775005168","heartbeat_front_addr":"172.21.15.170:6813/775005168","state":["exists","up"]},{"osd":6,"uuid":"25940be0-c1bd-4c72-95bf-fd8367aa2f08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6819","nonce":3008405982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6821","nonce":3008405982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6825","nonce":3008405982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":3008405982},{"type":"v1","addr":"172.21.15.82:6823","nonce":3008405982}]},"public_addr":"172.21.15.82:6819/3008405982","cluster_addr":"172.21.15.82:6821/3008405982","heartbeat_back_addr":"172.21.15.82:6825/3008405982","heartbeat_front_addr":"172.21.15.82:6823/3008405982","state":["exists","up"]},{"osd":7,"uuid":"f651bb8c-0b25-42d5-b028-58563f4c27f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6817","nonce":3547615284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6819","nonce":3547615284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6823","nonce":3547615284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3547615284},{"type":"v1","addr":"172.21.15.98:6821","nonce":3547615284}]},"public_addr":"172.21.15.98:6817/3547615284","cluster_addr":"172.21.15.98:6819/3547615284","heartbeat_back_addr":"172.21.15.98:6823/3547615284","heartbeat_front_addr":"172.21.15.98:6821/3547615284","state":["exists","up"]},{"osd":8,"uuid":"23b27c86-8c98-441e-b7ab-d57bc584e58f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6817","nonce":632038848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6819","nonce":632038848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6823","nonce":632038848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":632038848},{"type":"v1","addr":"172.21.15.170:6821","nonce":632038848}]},"public_addr":"172.21.15.170:6817/632038848","cluster_addr":"172.21.15.170:6819/632038848","heartbeat_back_addr":"172.21.15.170:6823/632038848","heartbeat_front_addr":"172.21.15.170:6821/632038848","state":["exists","up"]},{"osd":9,"uuid":"383b8989-4922-4eee-9c77-f198258ac243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6827","nonce":3354242311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6829","nonce":3354242311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6833","nonce":3354242311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3354242311},{"type":"v1","addr":"172.21.15.82:6831","nonce":3354242311}]},"public_addr":"172.21.15.82:6827/3354242311","cluster_addr":"172.21.15.82:6829/3354242311","heartbeat_back_addr":"172.21.15.82:6833/3354242311","heartbeat_front_addr":"172.21.15.82:6831/3354242311","state":["exists","up"]},{"osd":10,"uuid":"33a59389-7ced-42d5-bf54-2071317132d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6825","nonce":2069114959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6827","nonce":2069114959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6831","nonce":2069114959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2069114959},{"type":"v1","addr":"172.21.15.98:6829","nonce":2069114959}]},"public_addr":"172.21.15.98:6825/2069114959","cluster_addr":"172.21.15.98:6827/2069114959","heartbeat_back_addr":"172.21.15.98:6831/2069114959","heartbeat_front_addr":"172.21.15.98:6829/2069114959","state":["exists","up"]},{"osd":11,"uuid":"b7a71f5c-85e2-4fec-be4b-95be0788b302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6825","nonce":146964512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6827","nonce":146964512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6831","nonce":146964512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":146964512},{"type":"v1","addr":"172.21.15.170:6829","nonce":146964512}]},"public_addr":"172.21.15.170:6825/146964512","cluster_addr":"172.21.15.170:6827/146964512","heartbeat_back_addr":"172.21.15.170:6831/146964512","heartbeat_front_addr":"172.21.15.170:6829/146964512","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:07:47.563043+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:08.359260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:29.070374+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:08:49.897775+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:11.434248+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:31.538464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:09:52.377109+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:13.452968+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:33.806897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:10:54.083177+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:14.980087+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-09T23:11:35.690089+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4013747788":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6800/1192991521":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/2823341464":"2023-09-10T23:06:08.903527+0000","172.21.15.82:6801/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/75116806":"2023-09-10T23:06:08.903527+0000","172.21.15.82:0/1037625632":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/3142088817":"2023-09-10T23:05:46.550354+0000","172.21.15.82:6800/3263999756":"2023-09-10T23:05:46.550354+0000","172.21.15.82:0/1679622963":"2023-09-10T23:05:46.550354+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-09T23:13:46.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.444+0000 7fb2b83dd700 1 -- 172.21.15.82:0/4055047989 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb29c0a0710 msgr2=0x7fb29c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.444+0000 7fb2b83dd700 1 --2- 172.21.15.82:0/4055047989 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb29c0a0710 0x7fb29c0a2bd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0068450 tx=0x7fb2ac009260 comp rx=0 tx=0).stop 2023-09-09T23:13:46.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.444+0000 7fb2b83dd700 1 -- 172.21.15.82:0/4055047989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2b01000e0 msgr2=0x7fb2b01957a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.444+0000 7fb2b83dd700 1 --2- 172.21.15.82:0/4055047989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2b01000e0 0x7fb2b01957a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb2a00098f0 tx=0x7fb2a0009c00 comp rx=0 tx=0).stop 2023-09-09T23:13:46.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.444+0000 7fb2b83dd700 1 -- 172.21.15.82:0/4055047989 shutdown_connections 2023-09-09T23:13:46.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.444+0000 7fb2b83dd700 1 --2- 172.21.15.82:0/4055047989 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fb29c0a0710 0x7fb29c0a2bd0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.444+0000 7fb2b83dd700 1 --2- 172.21.15.82:0/4055047989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2b0100ad0 0x7fb2b019c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.444+0000 7fb2b83dd700 1 --2- 172.21.15.82:0/4055047989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2b00ff720 0x7fb2b0195260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.444+0000 7fb2b83dd700 1 --2- 172.21.15.82:0/4055047989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2b01000e0 0x7fb2b01957a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.444+0000 7fb2b83dd700 1 -- 172.21.15.82:0/4055047989 >> 172.21.15.82:0/4055047989 conn(0x7fb2b00fa1d0 msgr2=0x7fb2b00fc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:46.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.444+0000 7fb2b83dd700 1 -- 172.21.15.82:0/4055047989 shutdown_connections 2023-09-09T23:13:46.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.445+0000 7fb2b83dd700 1 -- 172.21.15.82:0/4055047989 wait complete. 2023-09-09T23:13:46.454 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:46 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2613673378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-09T23:13:46.454 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:46 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/279599879' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-09T23:13:46.454 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:46 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/1163975145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:13:46.456 DEBUG:tasks.fwd_scrub:Yielding 2023-09-09T23:13:46.456 INFO:teuthology.run_tasks:Running task exec... 2023-09-09T23:13:46.467 INFO:teuthology.task.exec:Executing custom commands... 2023-09-09T23:13:46.467 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi082.front.sepia.ceph.com 2023-09-09T23:13:46.468 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2023-09-09T23:13:46.468 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-09-09T23:13:46.468 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-09T23:13:46.469 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-09T23:13:46.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.602+0000 7f9d482dc700 1 -- 172.21.15.82:0/2077809046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d401010d0 msgr2=0x7f9d401034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.602+0000 7f9d482dc700 1 --2- 172.21.15.82:0/2077809046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d401010d0 0x7f9d401034c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c009990 tx=0x7f9d3c009ca0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.602+0000 7f9d482dc700 1 -- 172.21.15.82:0/2077809046 shutdown_connections 2023-09-09T23:13:46.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.602+0000 7f9d482dc700 1 --2- 172.21.15.82:0/2077809046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d401010d0 0x7f9d401034c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.602+0000 7f9d482dc700 1 --2- 172.21.15.82:0/2077809046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d40106330 0x7f9d401087a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.602+0000 7f9d482dc700 1 --2- 172.21.15.82:0/2077809046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d40103a00 0x7f9d40105df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.602+0000 7f9d482dc700 1 -- 172.21.15.82:0/2077809046 >> 172.21.15.82:0/2077809046 conn(0x7f9d400faac0 msgr2=0x7f9d400fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:46.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.602+0000 7f9d482dc700 1 -- 172.21.15.82:0/2077809046 shutdown_connections 2023-09-09T23:13:46.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.602+0000 7f9d482dc700 1 -- 172.21.15.82:0/2077809046 wait complete. 2023-09-09T23:13:46.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.603+0000 7f9d482dc700 1 Processor -- start 2023-09-09T23:13:46.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.603+0000 7f9d482dc700 1 -- start start 2023-09-09T23:13:46.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.603+0000 7f9d482dc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d401010d0 0x7f9d401963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.603+0000 7f9d482dc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d40103a00 0x7f9d40196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.603+0000 7f9d482dc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d40106330 0x7f9d40197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.603+0000 7f9d482dc700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d40197df0 con 0x7f9d401010d0 2023-09-09T23:13:46.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.603+0000 7f9d482dc700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d40197f60 con 0x7f9d40106330 2023-09-09T23:13:46.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.603+0000 7f9d482dc700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d40198120 con 0x7f9d40103a00 2023-09-09T23:13:46.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.603+0000 7f9d46879700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d40106330 0x7f9d40197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.603+0000 7f9d46078700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d401010d0 0x7f9d401963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.603+0000 7f9d46879700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d40106330 0x7f9d40197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:41780/0 (socket says 172.21.15.82:41780) 2023-09-09T23:13:46.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.603+0000 7f9d46078700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d401010d0 0x7f9d401963d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56212/0 (socket says 172.21.15.82:56212) 2023-09-09T23:13:46.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.603+0000 7f9d46879700 1 -- 172.21.15.82:0/2666960734 learned_addr learned my addr 172.21.15.82:0/2666960734 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:46.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.603+0000 7f9d45877700 1 --2- 172.21.15.82:0/2666960734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d40103a00 0x7f9d40196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.604+0000 7f9d46879700 1 -- 172.21.15.82:0/2666960734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d40103a00 msgr2=0x7f9d40196910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.604+0000 7f9d46879700 1 --2- 172.21.15.82:0/2666960734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d40103a00 0x7f9d40196910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.604+0000 7f9d46879700 1 -- 172.21.15.82:0/2666960734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d401010d0 msgr2=0x7f9d401963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.604+0000 7f9d46879700 1 --2- 172.21.15.82:0/2666960734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d401010d0 0x7f9d401963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.604+0000 7f9d46879700 1 -- 172.21.15.82:0/2666960734 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d38009c40 con 0x7f9d40106330 2023-09-09T23:13:46.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.604+0000 7f9d46879700 1 --2- 172.21.15.82:0/2666960734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d40106330 0x7f9d40197790 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800c370 tx=0x7f9d3800c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:46.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.604+0000 7f9d377fe700 1 -- 172.21.15.82:0/2666960734 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d38017070 con 0x7f9d40106330 2023-09-09T23:13:46.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.604+0000 7f9d482dc700 1 -- 172.21.15.82:0/2666960734 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d3c0095f0 con 0x7f9d40106330 2023-09-09T23:13:46.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.604+0000 7f9d377fe700 1 -- 172.21.15.82:0/2666960734 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d3800f040 con 0x7f9d40106330 2023-09-09T23:13:46.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.604+0000 7f9d377fe700 1 -- 172.21.15.82:0/2666960734 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3800e050 con 0x7f9d40106330 2023-09-09T23:13:46.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.604+0000 7f9d482dc700 1 -- 172.21.15.82:0/2666960734 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d4019e9e0 con 0x7f9d40106330 2023-09-09T23:13:46.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.605+0000 7fe0fcc07700 1 -- 172.21.15.82:0/980358574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0f80ff500 msgr2=0x7fe0f80ff920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.605+0000 7fe0fcc07700 1 --2- 172.21.15.82:0/980358574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0f80ff500 0x7fe0f80ff920 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80099c0 tx=0x7fe0e8009cd0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.605+0000 7f9d482dc700 1 -- 172.21.15.82:0/2666960734 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d400fc640 con 0x7f9d40106330 2023-09-09T23:13:46.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.605+0000 7fe0fcc07700 1 -- 172.21.15.82:0/980358574 shutdown_connections 2023-09-09T23:13:46.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.605+0000 7fe0fcc07700 1 --2- 172.21.15.82:0/980358574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0f8101090 0x7fe0f8107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.605+0000 7fe0fcc07700 1 --2- 172.21.15.82:0/980358574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0f80ff500 0x7fe0f80ff920 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.605+0000 7fe0fcc07700 1 --2- 172.21.15.82:0/980358574 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0f81006f0 0x7fe0f8100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.605+0000 7fe0fcc07700 1 -- 172.21.15.82:0/980358574 >> 172.21.15.82:0/980358574 conn(0x7fe0f80faa90 msgr2=0x7fe0f80fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:46.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.605+0000 7fe0fcc07700 1 -- 172.21.15.82:0/980358574 shutdown_connections 2023-09-09T23:13:46.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.605+0000 7fe0fcc07700 1 -- 172.21.15.82:0/980358574 wait complete. 2023-09-09T23:13:46.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.606+0000 7fe0fcc07700 1 Processor -- start 2023-09-09T23:13:46.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.606+0000 7fe0fcc07700 1 -- start start 2023-09-09T23:13:46.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.606+0000 7fe0fcc07700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0f80ff500 0x7fe0f81962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.606+0000 7fe0fcc07700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0f81006f0 0x7fe0f8196830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.606+0000 7fe0fcc07700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0f8101090 0x7fe0f819d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.606+0000 7fe0fcc07700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f819de70 con 0x7fe0f80ff500 2023-09-09T23:13:46.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.606+0000 7fe0fcc07700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f819dfb0 con 0x7fe0f8101090 2023-09-09T23:13:46.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.606+0000 7fe0fcc07700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f819e140 con 0x7fe0f81006f0 2023-09-09T23:13:46.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.606+0000 7fe0f659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0f80ff500 0x7fe0f81962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.606+0000 7fe0f659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0f80ff500 0x7fe0f81962f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56224/0 (socket says 172.21.15.82:56224) 2023-09-09T23:13:46.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.606+0000 7fe0f659c700 1 -- 172.21.15.82:0/3899059993 learned_addr learned my addr 172.21.15.82:0/3899059993 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:46.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.606+0000 7fe0f6d9d700 1 --2- 172.21.15.82:0/3899059993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0f8101090 0x7fe0f819d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.606+0000 7fe0f5d9b700 1 --2- 172.21.15.82:0/3899059993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0f81006f0 0x7fe0f8196830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.618 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.607+0000 7fe0f659c700 1 -- 172.21.15.82:0/3899059993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0f81006f0 msgr2=0x7fe0f8196830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.618 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.607+0000 7fe0f659c700 1 --2- 172.21.15.82:0/3899059993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0f81006f0 0x7fe0f8196830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.618 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.607+0000 7fe0f659c700 1 -- 172.21.15.82:0/3899059993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0f8101090 msgr2=0x7fe0f819d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.618 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.607+0000 7fe0f659c700 1 --2- 172.21.15.82:0/3899059993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0f8101090 0x7fe0f819d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.618 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.607+0000 7fe0f659c700 1 -- 172.21.15.82:0/3899059993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e80095f0 con 0x7fe0f80ff500 2023-09-09T23:13:46.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.607+0000 7fe0f659c700 1 --2- 172.21.15.82:0/3899059993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0f80ff500 0x7fe0f81962f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8009fd0 tx=0x7fe0e8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:46.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.607+0000 7f9d377fe700 1 -- 172.21.15.82:0/2666960734 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f9d3800cb60 con 0x7f9d40106330 2023-09-09T23:13:46.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.607+0000 7fe0e77fe700 1 -- 172.21.15.82:0/3899059993 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e801d070 con 0x7fe0f80ff500 2023-09-09T23:13:46.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.607+0000 7fe0fcc07700 1 -- 172.21.15.82:0/3899059993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f819e3c0 con 0x7fe0f80ff500 2023-09-09T23:13:46.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.607+0000 7fe0e77fe700 1 -- 172.21.15.82:0/3899059993 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0e8005030 con 0x7fe0f80ff500 2023-09-09T23:13:46.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.607+0000 7fe0e77fe700 1 -- 172.21.15.82:0/3899059993 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8004b30 con 0x7fe0f80ff500 2023-09-09T23:13:46.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.607+0000 7f9d377fe700 1 --2- 172.21.15.82:0/2666960734 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f9d2c0a0640 0x7f9d2c0a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.608+0000 7f9d46078700 1 --2- 172.21.15.82:0/2666960734 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f9d2c0a0640 0x7f9d2c0a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.607+0000 7fe0fcc07700 1 -- 172.21.15.82:0/3899059993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f819e7e0 con 0x7fe0f80ff500 2023-09-09T23:13:46.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.608+0000 7f9d377fe700 1 -- 172.21.15.82:0/2666960734 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f9d380cbaf0 con 0x7f9d40106330 2023-09-09T23:13:46.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.608+0000 7f9d46078700 1 --2- 172.21.15.82:0/2666960734 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f9d2c0a0640 0x7f9d2c0a2b00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00b2e0 tx=0x7f9d3c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:46.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.609+0000 7fe0e77fe700 1 -- 172.21.15.82:0/3899059993 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7fe0e80051a0 con 0x7fe0f80ff500 2023-09-09T23:13:46.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.609+0000 7fe0e77fe700 1 --2- 172.21.15.82:0/3899059993 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fe0e00a0640 0x7fe0e00a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.610+0000 7fe0f5d9b700 1 --2- 172.21.15.82:0/3899059993 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fe0e00a0640 0x7fe0e00a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.610+0000 7fe0e77fe700 1 -- 172.21.15.82:0/3899059993 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fe0e80cc8c0 con 0x7fe0f80ff500 2023-09-09T23:13:46.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.610+0000 7fe0f5d9b700 1 --2- 172.21.15.82:0/3899059993 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fe0e00a0640 0x7fe0e00a2b00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec009bf0 tx=0x7fe0ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:46.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.611+0000 7fe108dcdb80 1 Processor -- start 2023-09-09T23:13:46.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.611+0000 7f9d377fe700 1 -- 172.21.15.82:0/2666960734 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9d380956c0 con 0x7f9d40106330 2023-09-09T23:13:46.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.611+0000 7fe108dcdb80 1 -- start start 2023-09-09T23:13:46.630 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.611+0000 7fe108dcdb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631fbbd60f0 0x5631fbbd6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.630 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.611+0000 7fe108dcdb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5631fbbd6a50 0x5631fbbd4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.630 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.611+0000 7fe108dcdb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631fbbd52b0 0x5631fbbd5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.630 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.611+0000 7fe108dcdb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631fbb297b0 con 0x5631fbbd52b0 2023-09-09T23:13:46.630 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.611+0000 7fe108dcdb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631fbb270d0 con 0x5631fbbd6a50 2023-09-09T23:13:46.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.611+0000 7fe108dcdb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631fbb229c0 con 0x5631fbbd60f0 2023-09-09T23:13:46.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.611+0000 7fe0f6d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631fbbd52b0 0x5631fbbd5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.611+0000 7fe0f6d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631fbbd52b0 0x5631fbbd5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56238/0 (socket says 172.21.15.82:56238) 2023-09-09T23:13:46.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.611+0000 7fe0f6d9d700 1 -- 172.21.15.82:0/3681565630 learned_addr learned my addr 172.21.15.82:0/3681565630 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:46.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.611+0000 7fe0f5d9b700 1 --2- 172.21.15.82:0/3681565630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5631fbbd6a50 0x5631fbbd4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.611+0000 7fe0f659c700 1 --2- 172.21.15.82:0/3681565630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631fbbd60f0 0x5631fbbd6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.611+0000 7fe0f6d9d700 1 -- 172.21.15.82:0/3681565630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631fbbd60f0 msgr2=0x5631fbbd6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:46.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.611+0000 7fe0f6d9d700 1 --2- 172.21.15.82:0/3681565630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631fbbd60f0 0x5631fbbd6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.611+0000 7fe0f6d9d700 1 -- 172.21.15.82:0/3681565630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5631fbbd6a50 msgr2=0x5631fbbd4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:46.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.611+0000 7fe0f6d9d700 1 --2- 172.21.15.82:0/3681565630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5631fbbd6a50 0x5631fbbd4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.611+0000 7fe0f6d9d700 1 -- 172.21.15.82:0/3681565630 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631fb94ee60 con 0x5631fbbd52b0 2023-09-09T23:13:46.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.612+0000 7fe0f6d9d700 1 --2- 172.21.15.82:0/3681565630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631fbbd52b0 0x5631fbbd5730 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0011680 tx=0x7fe0f0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878b27f763af8a9f server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:46.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.612+0000 7fe0e4ff9700 1 -- 172.21.15.82:0/3681565630 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f000e040 con 0x5631fbbd52b0 2023-09-09T23:13:46.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.612+0000 7fe0e4ff9700 1 -- 172.21.15.82:0/3681565630 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0f000f040 con 0x5631fbbd52b0 2023-09-09T23:13:46.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.612+0000 7fe0e4ff9700 1 -- 172.21.15.82:0/3681565630 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0004880 con 0x5631fbbd52b0 2023-09-09T23:13:46.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.612+0000 7fe108dcdb80 1 -- 172.21.15.82:0/3681565630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631fbbd52b0 msgr2=0x5631fbbd5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.612+0000 7fe108dcdb80 1 --2- 172.21.15.82:0/3681565630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631fbbd52b0 0x5631fbbd5730 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0011680 tx=0x7fe0f0009920 comp rx=0 tx=0).stop 2023-09-09T23:13:46.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.613+0000 7fe108dcdb80 1 -- 172.21.15.82:0/3681565630 shutdown_connections 2023-09-09T23:13:46.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.613+0000 7fe108dcdb80 1 --2- 172.21.15.82:0/3681565630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631fbbd52b0 0x5631fbbd5730 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.613+0000 7fe108dcdb80 1 --2- 172.21.15.82:0/3681565630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631fbbd60f0 0x5631fbbd6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.613+0000 7fe108dcdb80 1 --2- 172.21.15.82:0/3681565630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5631fbbd6a50 0x5631fbbd4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.613+0000 7fe108dcdb80 1 -- 172.21.15.82:0/3681565630 >> 172.21.15.82:0/3681565630 conn(0x5631fbb45110 msgr2=0x5631fbbcb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:46.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.613+0000 7fe108dcdb80 1 -- 172.21.15.82:0/3681565630 shutdown_connections 2023-09-09T23:13:46.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.613+0000 7fe108dcdb80 1 -- 172.21.15.82:0/3681565630 wait complete. 2023-09-09T23:13:46.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.613+0000 7fe108dcdb80 1 -- start start 2023-09-09T23:13:46.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.613+0000 7fe108dcdb80 1 Processor -- start 2023-09-09T23:13:46.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.613+0000 7fe108dcdb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631fbbd52b0 0x5631fbbdea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.613+0000 7fe108dcdb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631fbbd60f0 0x5631fbbdef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.613+0000 7fe108dcdb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5631fbbd6a50 0x5631fbbe5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.613+0000 7fe108dcdb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631fbac0bf0 con 0x5631fbbd52b0 2023-09-09T23:13:46.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.613+0000 7fe108dcdb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631fbac08f0 con 0x5631fbbd6a50 2023-09-09T23:13:46.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.614+0000 7fe108dcdb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631fb9a49a0 con 0x5631fbbd60f0 2023-09-09T23:13:46.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.614+0000 7fe108dcdb80 10 client.0 _ll_register_callbacks cb 0x5631fba0ab00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:46.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.614+0000 7fe0f659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631fbbd52b0 0x5631fbbdea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.614+0000 7fe0f6d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5631fbbd6a50 0x5631fbbe5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.614+0000 7fe0f659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631fbbd52b0 0x5631fbbdea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56240/0 (socket says 172.21.15.82:56240) 2023-09-09T23:13:46.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.614+0000 7fe0f659c700 1 -- 172.21.15.82:0/1035639590 learned_addr learned my addr 172.21.15.82:0/1035639590 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:46.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.614+0000 7fe0f5d9b700 1 --2- 172.21.15.82:0/1035639590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631fbbd60f0 0x5631fbbdef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.614+0000 7fe0f659c700 1 -- 172.21.15.82:0/1035639590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631fbbd60f0 msgr2=0x5631fbbdef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:46.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.614+0000 7fe0f659c700 1 --2- 172.21.15.82:0/1035639590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631fbbd60f0 0x5631fbbdef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.614+0000 7fe0f659c700 1 -- 172.21.15.82:0/1035639590 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5631fbbd6a50 msgr2=0x5631fbbe5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:46.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.614+0000 7fe0f659c700 1 --2- 172.21.15.82:0/1035639590 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5631fbbd6a50 0x5631fbbe5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.614+0000 7fe0f659c700 1 -- 172.21.15.82:0/1035639590 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f0011330 con 0x5631fbbd52b0 2023-09-09T23:13:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.614+0000 7fe0f6d9d700 1 --2- 172.21.15.82:0/1035639590 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5631fbbd6a50 0x5631fbbe5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.614+0000 7fe0f659c700 1 --2- 172.21.15.82:0/1035639590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631fbbd52b0 0x5631fbbdea00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8000c00 tx=0x7fe0e8005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43860a189beac071 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.614+0000 7fe0ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.614+0000 7fe0ddffb700 1 -- 172.21.15.82:0/1035639590 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8098580 con 0x5631fbbd52b0 2023-09-09T23:13:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.614+0000 7fe0ddffb700 1 -- 172.21.15.82:0/1035639590 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fe0e809a440 con 0x5631fbbd52b0 2023-09-09T23:13:46.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.614+0000 7fe108dcdb80 1 -- 172.21.15.82:0/1035639590 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5631fba7a570 con 0x5631fbbd52b0 2023-09-09T23:13:46.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.614+0000 7fe0ddffb700 1 -- 172.21.15.82:0/1035639590 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0e80b8170 con 0x5631fbbd52b0 2023-09-09T23:13:46.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.614+0000 7fe108dcdb80 10 client.15552 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:46.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.614+0000 7fe108dcdb80 1 -- 172.21.15.82:0/1035639590 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5631fb937ce0 con 0x5631fbbd52b0 2023-09-09T23:13:46.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.615+0000 7fe0ddffb700 1 -- 172.21.15.82:0/1035639590 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0e8064080 con 0x5631fbbd52b0 2023-09-09T23:13:46.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.615+0000 7fe0ddffb700 1 -- 172.21.15.82:0/1035639590 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe0e80a1d30 con 0x5631fbbd52b0 2023-09-09T23:13:46.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.615+0000 7fe108dcdb80 10 client.15552 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:46.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.615+0000 7fe108dcdb80 10 client.15552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:13:46.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.615+0000 7fe108dcdb80 20 client.15552 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:46.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.615+0000 7fe108dcdb80 1 --2- 172.21.15.82:0/1035639590 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5631fbbea410 0x5631fbbec800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.642 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.615+0000 7fe108dcdb80 4 client.15552 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:13:46.642 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.615+0000 7fe108dcdb80 1 -- 172.21.15.82:0/1035639590 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5631fbb37920 con 0x5631fbbea410 2023-09-09T23:13:46.642 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.615+0000 7fe0f5d9b700 1 --2- 172.21.15.82:0/1035639590 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5631fbbea410 0x5631fbbec800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.642 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.616+0000 7fe0f5d9b700 1 --2- 172.21.15.82:0/1035639590 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5631fbbea410 0x5631fbbec800 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f9a5315fb219e9c server_cookie=9ac56f79c5201191 in_seq=0 out_seq=0 2023-09-09T23:13:46.642 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.616+0000 7fe0ddffb700 10 client.15552 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:13:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.616+0000 7fe0ddffb700 1 -- 172.21.15.82:0/1035639590 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5631fbb37920 con 0x5631fbbea410 2023-09-09T23:13:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.616+0000 7fe0ddffb700 10 client.15552 handle_command_reply: tid=0 2023-09-09T23:13:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.616+0000 7fe108dcdb80 1 client.15552 shutdown 2023-09-09T23:13:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.616+0000 7fe108dcdb80 1 -- 172.21.15.82:0/1035639590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631fbbd52b0 msgr2=0x5631fbbdea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.616+0000 7fe108dcdb80 1 --2- 172.21.15.82:0/1035639590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631fbbd52b0 0x5631fbbdea00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8000c00 tx=0x7fe0e8005760 comp rx=0 tx=0).stop 2023-09-09T23:13:46.644 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.616+0000 7fe108dcdb80 1 -- 172.21.15.82:0/1035639590 shutdown_connections 2023-09-09T23:13:46.644 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.616+0000 7fe108dcdb80 1 --2- 172.21.15.82:0/1035639590 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5631fbbea410 0x5631fbbec800 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.644 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.616+0000 7fe108dcdb80 1 --2- 172.21.15.82:0/1035639590 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5631fbbd6a50 0x5631fbbe5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.644 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.617+0000 7fe108dcdb80 1 --2- 172.21.15.82:0/1035639590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631fbbd52b0 0x5631fbbdea00 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.644 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.617+0000 7fe0ddffb700 0 client.15552 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:13:46.644 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.617+0000 7fe108dcdb80 1 --2- 172.21.15.82:0/1035639590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631fbbd60f0 0x5631fbbdef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.645 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.617+0000 7fe108dcdb80 1 -- 172.21.15.82:0/1035639590 >> 172.21.15.82:0/1035639590 conn(0x5631fbb45110 msgr2=0x5631fbbcae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:46.645 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.617+0000 7fe108dcdb80 1 -- 172.21.15.82:0/1035639590 shutdown_connections 2023-09-09T23:13:46.645 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.617+0000 7fe108dcdb80 1 -- 172.21.15.82:0/1035639590 wait complete. 2023-09-09T23:13:46.645 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.617+0000 7fe108dcdb80 20 client.15552 trim_cache size 0 max 16384 2023-09-09T23:13:46.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.633+0000 7fe108dcdb80 1 Processor -- start 2023-09-09T23:13:46.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.635+0000 7fe108dcdb80 1 -- start start 2023-09-09T23:13:46.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.635+0000 7fe108dcdb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631fbbd52b0 0x5631fbbde7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.635+0000 7fe108dcdb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631fbbd60f0 0x5631fbbe5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.635+0000 7fe108dcdb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5631fbbd6a50 0x5631fbbdecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.635+0000 7fe108dcdb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631fbbd5dc0 con 0x5631fbbd60f0 2023-09-09T23:13:46.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.635+0000 7fe108dcdb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631fbbde570 con 0x5631fbbd6a50 2023-09-09T23:13:46.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.635+0000 7fe108dcdb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631fbbcaa50 con 0x5631fbbd52b0 2023-09-09T23:13:46.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.635+0000 7fe0f5d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631fbbd60f0 0x5631fbbe5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.635+0000 7fe0f5d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631fbbd60f0 0x5631fbbe5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56250/0 (socket says 172.21.15.82:56250) 2023-09-09T23:13:46.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.635+0000 7fe0f5d9b700 1 -- 172.21.15.82:0/3368696337 learned_addr learned my addr 172.21.15.82:0/3368696337 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:46.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.635+0000 7fe0f6d9d700 1 --2- 172.21.15.82:0/3368696337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5631fbbd6a50 0x5631fbbdecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.635+0000 7fe0f659c700 1 --2- 172.21.15.82:0/3368696337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631fbbd52b0 0x5631fbbde7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.635+0000 7fe0f5d9b700 1 -- 172.21.15.82:0/3368696337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631fbbd52b0 msgr2=0x5631fbbde7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:46.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.635+0000 7fe0f5d9b700 1 --2- 172.21.15.82:0/3368696337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631fbbd52b0 0x5631fbbde7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.635+0000 7fe0f5d9b700 1 -- 172.21.15.82:0/3368696337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5631fbbd6a50 msgr2=0x5631fbbdecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:46.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.635+0000 7fe0f5d9b700 1 --2- 172.21.15.82:0/3368696337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5631fbbd6a50 0x5631fbbdecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.635+0000 7fe0f5d9b700 1 -- 172.21.15.82:0/3368696337 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631fb9752d0 con 0x5631fbbd60f0 2023-09-09T23:13:46.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.636+0000 7fe0f5d9b700 1 --2- 172.21.15.82:0/3368696337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631fbbd60f0 0x5631fbbe5f40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec009ab0 tx=0x7fe0ec014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff516030f1c2b931 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:46.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.636+0000 7fe0e57fa700 1 -- 172.21.15.82:0/3368696337 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec024040 con 0x5631fbbd60f0 2023-09-09T23:13:46.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.636+0000 7fe0e57fa700 1 -- 172.21.15.82:0/3368696337 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0ec016070 con 0x5631fbbd60f0 2023-09-09T23:13:46.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.636+0000 7fe0e57fa700 1 -- 172.21.15.82:0/3368696337 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec029b60 con 0x5631fbbd60f0 2023-09-09T23:13:46.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.636+0000 7fe108dcdb80 1 -- 172.21.15.82:0/3368696337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631fbbd60f0 msgr2=0x5631fbbe5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.636+0000 7fe108dcdb80 1 --2- 172.21.15.82:0/3368696337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631fbbd60f0 0x5631fbbe5f40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec009ab0 tx=0x7fe0ec014000 comp rx=0 tx=0).stop 2023-09-09T23:13:46.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.636+0000 7fe108dcdb80 1 -- 172.21.15.82:0/3368696337 shutdown_connections 2023-09-09T23:13:46.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.636+0000 7fe108dcdb80 1 --2- 172.21.15.82:0/3368696337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5631fbbd6a50 0x5631fbbdecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.636+0000 7fe108dcdb80 1 --2- 172.21.15.82:0/3368696337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631fbbd52b0 0x5631fbbde7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.636+0000 7fe108dcdb80 1 --2- 172.21.15.82:0/3368696337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631fbbd60f0 0x5631fbbe5f40 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.636+0000 7fe108dcdb80 1 -- 172.21.15.82:0/3368696337 >> 172.21.15.82:0/3368696337 conn(0x5631fbb45110 msgr2=0x5631fbbd9370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:46.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.636+0000 7fe108dcdb80 1 -- 172.21.15.82:0/3368696337 shutdown_connections 2023-09-09T23:13:46.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.636+0000 7fe108dcdb80 1 -- 172.21.15.82:0/3368696337 wait complete. 2023-09-09T23:13:46.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.636+0000 7fe108dcdb80 1 -- start start 2023-09-09T23:13:46.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.636+0000 7fe108dcdb80 1 Processor -- start 2023-09-09T23:13:46.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.636+0000 7fe108dcdb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631fbbd52b0 0x5631fbbe3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.636+0000 7fe108dcdb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631fbbd60f0 0x5631fbbe37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.636+0000 7fe108dcdb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5631fbbd6a50 0x5631fbbe47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.636+0000 7fe108dcdb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631fbbe4cf0 con 0x5631fbbd52b0 2023-09-09T23:13:46.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.636+0000 7fe108dcdb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631fbbe4e60 con 0x5631fbbd6a50 2023-09-09T23:13:46.659 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.636+0000 7fe108dcdb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631fbbe5020 con 0x5631fbbd60f0 2023-09-09T23:13:46.659 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.636+0000 7fe108dcdb80 10 client.0 _ll_register_callbacks cb 0x7fe0e809a8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:46.659 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.636+0000 7fe0f659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631fbbd52b0 0x5631fbbe3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.659 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.636+0000 7fe0f659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631fbbd52b0 0x5631fbbe3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56254/0 (socket says 172.21.15.82:56254) 2023-09-09T23:13:46.659 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.636+0000 7fe0f659c700 1 -- 172.21.15.82:0/3259104344 learned_addr learned my addr 172.21.15.82:0/3259104344 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:46.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.636+0000 7fe0f6d9d700 1 --2- 172.21.15.82:0/3259104344 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5631fbbd6a50 0x5631fbbe47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.637+0000 7fe0f5d9b700 1 --2- 172.21.15.82:0/3259104344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631fbbd60f0 0x5631fbbe37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.637+0000 7fe0f659c700 1 -- 172.21.15.82:0/3259104344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631fbbd60f0 msgr2=0x5631fbbe37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:46.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.637+0000 7fe0f659c700 1 --2- 172.21.15.82:0/3259104344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631fbbd60f0 0x5631fbbe37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.637+0000 7fe0f659c700 1 -- 172.21.15.82:0/3259104344 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5631fbbd6a50 msgr2=0x5631fbbe47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:46.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.637+0000 7fe0f659c700 1 --2- 172.21.15.82:0/3259104344 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5631fbbd6a50 0x5631fbbe47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.637+0000 7fe0f659c700 1 -- 172.21.15.82:0/3259104344 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0ec012450 con 0x5631fbbd52b0 2023-09-09T23:13:46.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.637+0000 7fe0f5d9b700 1 --2- 172.21.15.82:0/3259104344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631fbbd60f0 0x5631fbbe37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:46.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.637+0000 7fe0f659c700 1 --2- 172.21.15.82:0/3259104344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631fbbd52b0 0x5631fbbe3260 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80057d0 tx=0x7fe0e80a2280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f841f71c048892e1 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:46.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.637+0000 7fe0ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:46.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.637+0000 7fe0ddffb700 1 -- 172.21.15.82:0/3259104344 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e80b2200 con 0x5631fbbd52b0 2023-09-09T23:13:46.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.637+0000 7fe0ddffb700 1 -- 172.21.15.82:0/3259104344 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fe0e80996d0 con 0x5631fbbd52b0 2023-09-09T23:13:46.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.637+0000 7fe108dcdb80 1 -- 172.21.15.82:0/3259104344 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0f000f040 con 0x5631fbbd52b0 2023-09-09T23:13:46.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.637+0000 7fe0ddffb700 1 -- 172.21.15.82:0/3259104344 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0e8099890 con 0x5631fbbd52b0 2023-09-09T23:13:46.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.637+0000 7fe108dcdb80 10 client.15558 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:46.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.637+0000 7fe108dcdb80 1 -- 172.21.15.82:0/3259104344 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5631fb94d330 con 0x5631fbbd52b0 2023-09-09T23:13:46.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.637+0000 7fe0ddffb700 1 -- 172.21.15.82:0/3259104344 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0f000f040 con 0x5631fbbd52b0 2023-09-09T23:13:46.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.637+0000 7fe0ddffb700 1 -- 172.21.15.82:0/3259104344 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe0e8095fd0 con 0x5631fbbd52b0 2023-09-09T23:13:46.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.637+0000 7fe108dcdb80 10 client.15558 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:46.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.637+0000 7fe108dcdb80 10 client.15558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:13:46.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.637+0000 7fe108dcdb80 20 client.15558 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:46.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.637+0000 7fe108dcdb80 1 --2- 172.21.15.82:0/3259104344 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5631fbc40220 0x5631fbc42610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.637+0000 7fe108dcdb80 4 client.15558 mds_command: new command op to 24566 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-09T23:13:46.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.637+0000 7fe108dcdb80 1 -- 172.21.15.82:0/3259104344 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5631fbb1b9a0 con 0x5631fbc40220 2023-09-09T23:13:46.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.639+0000 7fe0f5d9b700 1 --2- 172.21.15.82:0/3259104344 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5631fbc40220 0x5631fbc42610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.639+0000 7fe0ddffb700 10 client.15558 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:13:46.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.639+0000 7fe0f5d9b700 1 --2- 172.21.15.82:0/3259104344 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5631fbc40220 0x5631fbc42610 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10801379cefd61d8 server_cookie=4fec7e431efa20f2 in_seq=0 out_seq=0 2023-09-09T23:13:46.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.639+0000 7fe0ddffb700 1 -- 172.21.15.82:0/3259104344 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5631fbb1b9a0 con 0x5631fbc40220 2023-09-09T23:13:46.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.639+0000 7fe0ddffb700 10 client.15558 handle_command_reply: tid=0 2023-09-09T23:13:46.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.639+0000 7fe108dcdb80 1 client.15558 shutdown 2023-09-09T23:13:46.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.639+0000 7fe108dcdb80 1 -- 172.21.15.82:0/3259104344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631fbbd52b0 msgr2=0x5631fbbe3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.639+0000 7fe108dcdb80 1 --2- 172.21.15.82:0/3259104344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631fbbd52b0 0x5631fbbe3260 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80057d0 tx=0x7fe0e80a2280 comp rx=0 tx=0).stop 2023-09-09T23:13:46.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.639+0000 7fe108dcdb80 1 -- 172.21.15.82:0/3259104344 shutdown_connections 2023-09-09T23:13:46.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.639+0000 7fe108dcdb80 1 --2- 172.21.15.82:0/3259104344 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5631fbc40220 0x5631fbc42610 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.639+0000 7fe108dcdb80 1 --2- 172.21.15.82:0/3259104344 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5631fbbd6a50 0x5631fbbe47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.639+0000 7fe108dcdb80 1 --2- 172.21.15.82:0/3259104344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631fbbd52b0 0x5631fbbe3260 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.639+0000 7fe0ddffb700 0 client.15558 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:13:46.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.639+0000 7fe108dcdb80 1 --2- 172.21.15.82:0/3259104344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631fbbd60f0 0x5631fbbe37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.639+0000 7fe108dcdb80 1 -- 172.21.15.82:0/3259104344 >> 172.21.15.82:0/3259104344 conn(0x5631fbb45110 msgr2=0x5631fbbde7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:46.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.639+0000 7fe108dcdb80 1 -- 172.21.15.82:0/3259104344 shutdown_connections 2023-09-09T23:13:46.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.639+0000 7fe108dcdb80 1 -- 172.21.15.82:0/3259104344 wait complete. 2023-09-09T23:13:46.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.640+0000 7fe108dcdb80 20 client.15558 trim_cache size 0 max 16384 2023-09-09T23:13:46.668 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:13:46.668 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2023-09-09T23:13:46.668 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "88b37e29-a02d-4e1e-9e1e-7c18aca241d7", 2023-09-09T23:13:46.668 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2023-09-09T23:13:46.669 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:13:46.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.640+0000 7fe0fcc07700 1 -- 172.21.15.82:0/3899059993 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fe0e00a0640 msgr2=0x7fe0e00a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.640+0000 7fe0fcc07700 1 --2- 172.21.15.82:0/3899059993 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fe0e00a0640 0x7fe0e00a2b00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec009bf0 tx=0x7fe0ec009260 comp rx=0 tx=0).stop 2023-09-09T23:13:46.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.640+0000 7fe0fcc07700 1 -- 172.21.15.82:0/3899059993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0f80ff500 msgr2=0x7fe0f81962f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.640+0000 7fe0fcc07700 1 --2- 172.21.15.82:0/3899059993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0f80ff500 0x7fe0f81962f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8009fd0 tx=0x7fe0e8009580 comp rx=0 tx=0).stop 2023-09-09T23:13:46.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.640+0000 7fe0fcc07700 1 -- 172.21.15.82:0/3899059993 shutdown_connections 2023-09-09T23:13:46.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.640+0000 7fe0fcc07700 1 --2- 172.21.15.82:0/3899059993 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fe0e00a0640 0x7fe0e00a2b00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.640+0000 7fe0fcc07700 1 --2- 172.21.15.82:0/3899059993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0f8101090 0x7fe0f819d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.640+0000 7fe0fcc07700 1 --2- 172.21.15.82:0/3899059993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0f80ff500 0x7fe0f81962f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.672 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.640+0000 7fe0fcc07700 1 --2- 172.21.15.82:0/3899059993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0f81006f0 0x7fe0f8196830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.672 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.640+0000 7fe0fcc07700 1 -- 172.21.15.82:0/3899059993 >> 172.21.15.82:0/3899059993 conn(0x7fe0f80faa90 msgr2=0x7fe0f8103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:46.672 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.640+0000 7fe0fcc07700 1 -- 172.21.15.82:0/3899059993 shutdown_connections 2023-09-09T23:13:46.672 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.640+0000 7fe0fcc07700 1 -- 172.21.15.82:0/3899059993 wait complete. 2023-09-09T23:13:46.673 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:13:46.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:46 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2613673378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-09T23:13:46.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:46 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/279599879' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-09T23:13:46.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:46 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/1163975145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:13:46.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:46 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2613673378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-09T23:13:46.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:46 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/279599879' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-09T23:13:46.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:46 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/1163975145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:13:46.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.799+0000 7f99d6f4d700 1 -- 172.21.15.82:0/925930201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99d0100610 msgr2=0x7f99d0100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.799+0000 7f99d6f4d700 1 --2- 172.21.15.82:0/925930201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99d0100610 0x7f99d0100a30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f99c40099c0 tx=0x7f99c4009cd0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.799+0000 7f99d6f4d700 1 -- 172.21.15.82:0/925930201 shutdown_connections 2023-09-09T23:13:46.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.799+0000 7f99d6f4d700 1 --2- 172.21.15.82:0/925930201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99d01021a0 0x7f99d0108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.799+0000 7f99d6f4d700 1 --2- 172.21.15.82:0/925930201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99d0100610 0x7f99d0100a30 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.799+0000 7f99d6f4d700 1 --2- 172.21.15.82:0/925930201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99d0101800 0x7f99d0101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.799+0000 7f99d6f4d700 1 -- 172.21.15.82:0/925930201 >> 172.21.15.82:0/925930201 conn(0x7f99d00fbb80 msgr2=0x7f99d00fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:46.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.799+0000 7f99d6f4d700 1 -- 172.21.15.82:0/925930201 shutdown_connections 2023-09-09T23:13:46.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.799+0000 7f99d6f4d700 1 -- 172.21.15.82:0/925930201 wait complete. 2023-09-09T23:13:46.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.800+0000 7f99d6f4d700 1 Processor -- start 2023-09-09T23:13:46.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.800+0000 7f99d6f4d700 1 -- start start 2023-09-09T23:13:46.801 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.800+0000 7f99d6f4d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99d0100610 0x7f99d0196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.801 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.800+0000 7f99d6f4d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99d0101800 0x7f99d0196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.801 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.800+0000 7f99d6f4d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99d01021a0 0x7f99d019d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.801 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.800+0000 7f99d6f4d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d019dec0 con 0x7f99d0101800 2023-09-09T23:13:46.801 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.800+0000 7f99d6f4d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d019e000 con 0x7f99d0100610 2023-09-09T23:13:46.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.800+0000 7f99d6f4d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d019e190 con 0x7f99d01021a0 2023-09-09T23:13:46.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.800+0000 7f99cffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99d0101800 0x7f99d0196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.800+0000 7f99cffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99d0101800 0x7f99d0196880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56270/0 (socket says 172.21.15.82:56270) 2023-09-09T23:13:46.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.800+0000 7f99cffff700 1 -- 172.21.15.82:0/645670298 learned_addr learned my addr 172.21.15.82:0/645670298 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:46.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.800+0000 7f99d54ea700 1 --2- 172.21.15.82:0/645670298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99d01021a0 0x7f99d019d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.800+0000 7f99d4ce9700 1 --2- 172.21.15.82:0/645670298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99d0100610 0x7f99d0196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.800+0000 7f99cffff700 1 -- 172.21.15.82:0/645670298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99d01021a0 msgr2=0x7f99d019d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.800+0000 7f99cffff700 1 --2- 172.21.15.82:0/645670298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99d01021a0 0x7f99d019d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.800+0000 7f99cffff700 1 -- 172.21.15.82:0/645670298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99d0100610 msgr2=0x7f99d0196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.800+0000 7f99cffff700 1 --2- 172.21.15.82:0/645670298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99d0100610 0x7f99d0196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.800+0000 7f99cffff700 1 -- 172.21.15.82:0/645670298 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c40095f0 con 0x7f99d0101800 2023-09-09T23:13:46.804 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.801+0000 7f99cffff700 1 --2- 172.21.15.82:0/645670298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99d0101800 0x7f99d0196880 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00d980 tx=0x7f99bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:46.804 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.801+0000 7f99cdffb700 1 -- 172.21.15.82:0/645670298 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc004a30 con 0x7f99d0101800 2023-09-09T23:13:46.804 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.801+0000 7f99d6f4d700 1 -- 172.21.15.82:0/645670298 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99d019e470 con 0x7f99d0101800 2023-09-09T23:13:46.804 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.801+0000 7f99cdffb700 1 -- 172.21.15.82:0/645670298 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99bc0054e0 con 0x7f99d0101800 2023-09-09T23:13:46.804 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.801+0000 7f99cdffb700 1 -- 172.21.15.82:0/645670298 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc009c90 con 0x7f99d0101800 2023-09-09T23:13:46.804 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.801+0000 7f99d6f4d700 1 -- 172.21.15.82:0/645670298 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99d019e930 con 0x7f99d0101800 2023-09-09T23:13:46.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.803+0000 7f99cdffb700 1 -- 172.21.15.82:0/645670298 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f99bc004c20 con 0x7f99d0101800 2023-09-09T23:13:46.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.803+0000 7f99cdffb700 1 --2- 172.21.15.82:0/645670298 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f99c00a0640 0x7f99c00a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.804+0000 7f99cdffb700 1 -- 172.21.15.82:0/645670298 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f99bc0cb9b0 con 0x7f99d0101800 2023-09-09T23:13:46.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.804+0000 7f99d4ce9700 1 --2- 172.21.15.82:0/645670298 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f99c00a0640 0x7f99c00a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.804+0000 7f99d4ce9700 1 --2- 172.21.15.82:0/645670298 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f99c00a0640 0x7f99c00a2b00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f99c400b580 tx=0x7f99c4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:46.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.805+0000 7f99e3113b80 1 Processor -- start 2023-09-09T23:13:46.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.805+0000 7f99e3113b80 1 -- start start 2023-09-09T23:13:46.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.805+0000 7f99e3113b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4f69cc00 0x55ae4f69d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.805+0000 7f99e3113b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae4f69d5a0 0x55ae4f69b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.805+0000 7f99e3113b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae4f69be00 0x55ae4f69c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.805+0000 7f99e3113b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae4f5f4730 con 0x55ae4f69d5a0 2023-09-09T23:13:46.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.805+0000 7f99e3113b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae4f5f2050 con 0x55ae4f69be00 2023-09-09T23:13:46.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.805+0000 7f99e3113b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae4f5ed940 con 0x55ae4f69cc00 2023-09-09T23:13:46.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.805+0000 7f99cffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae4f69d5a0 0x55ae4f69b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.805+0000 7f99d4ce9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4f69cc00 0x55ae4f69d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.805+0000 7f99d4ce9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4f69cc00 0x55ae4f69d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:54790/0 (socket says 172.21.15.82:54790) 2023-09-09T23:13:46.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.805+0000 7f99d4ce9700 1 -- 172.21.15.82:0/2109553526 learned_addr learned my addr 172.21.15.82:0/2109553526 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:46.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.805+0000 7f99d54ea700 1 --2- 172.21.15.82:0/2109553526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae4f69be00 0x55ae4f69c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.805+0000 7f99d4ce9700 1 -- 172.21.15.82:0/2109553526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae4f69be00 msgr2=0x55ae4f69c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:46.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.805+0000 7f99d4ce9700 1 --2- 172.21.15.82:0/2109553526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae4f69be00 0x55ae4f69c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.805+0000 7f99d4ce9700 1 -- 172.21.15.82:0/2109553526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae4f69d5a0 msgr2=0x55ae4f69b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:46.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.805+0000 7f99d4ce9700 1 --2- 172.21.15.82:0/2109553526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae4f69d5a0 0x55ae4f69b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.805+0000 7f99d4ce9700 1 -- 172.21.15.82:0/2109553526 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae4f419de0 con 0x55ae4f69cc00 2023-09-09T23:13:46.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.805+0000 7f99cffff700 1 --2- 172.21.15.82:0/2109553526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae4f69d5a0 0x55ae4f69b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:46.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.806+0000 7f99d4ce9700 1 --2- 172.21.15.82:0/2109553526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4f69cc00 0x55ae4f69d020 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f99d0101660 tx=0x7f99c401f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89a97f22870fc039 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:46.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.806+0000 7f99baffd700 1 -- 172.21.15.82:0/2109553526 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c402b070 con 0x55ae4f69cc00 2023-09-09T23:13:46.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.806+0000 7f99baffd700 1 -- 172.21.15.82:0/2109553526 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99c401e040 con 0x55ae4f69cc00 2023-09-09T23:13:46.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.806+0000 7f99baffd700 1 -- 172.21.15.82:0/2109553526 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c4003ed0 con 0x55ae4f69cc00 2023-09-09T23:13:46.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.806+0000 7f99e3113b80 1 -- 172.21.15.82:0/2109553526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4f69cc00 msgr2=0x55ae4f69d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.806+0000 7f99e3113b80 1 --2- 172.21.15.82:0/2109553526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4f69cc00 0x55ae4f69d020 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f99d0101660 tx=0x7f99c401f000 comp rx=0 tx=0).stop 2023-09-09T23:13:46.816 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.806+0000 7f99e3113b80 1 -- 172.21.15.82:0/2109553526 shutdown_connections 2023-09-09T23:13:46.816 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.806+0000 7f99e3113b80 1 --2- 172.21.15.82:0/2109553526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae4f69be00 0x55ae4f69c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.816 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.806+0000 7f99e3113b80 1 --2- 172.21.15.82:0/2109553526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4f69cc00 0x55ae4f69d020 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.816 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.806+0000 7f99e3113b80 1 --2- 172.21.15.82:0/2109553526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae4f69d5a0 0x55ae4f69b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.816 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.806+0000 7f99e3113b80 1 -- 172.21.15.82:0/2109553526 >> 172.21.15.82:0/2109553526 conn(0x55ae4f610090 msgr2=0x55ae4f696720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:46.816 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.806+0000 7f99e3113b80 1 -- 172.21.15.82:0/2109553526 shutdown_connections 2023-09-09T23:13:46.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.807+0000 7f99e3113b80 1 -- 172.21.15.82:0/2109553526 wait complete. 2023-09-09T23:13:46.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.807+0000 7f99e3113b80 1 -- start start 2023-09-09T23:13:46.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.807+0000 7f99e3113b80 1 Processor -- start 2023-09-09T23:13:46.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.807+0000 7f99e3113b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4f69be00 0x55ae4f6a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.807+0000 7f99e3113b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae4f69cc00 0x55ae4f6a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.807+0000 7f99e3113b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae4f69d5a0 0x55ae4f6aca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.807+0000 7f99e3113b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae4f58bb70 con 0x55ae4f69d5a0 2023-09-09T23:13:46.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.807+0000 7f99e3113b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae4f58b870 con 0x55ae4f69cc00 2023-09-09T23:13:46.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.807+0000 7f99e3113b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae4f46f920 con 0x55ae4f69be00 2023-09-09T23:13:46.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.807+0000 7f99e3113b80 10 client.0 _ll_register_callbacks cb 0x55ae4f4d5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:46.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.807+0000 7f99d54ea700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae4f69d5a0 0x55ae4f6aca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.807+0000 7f99d54ea700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae4f69d5a0 0x55ae4f6aca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56286/0 (socket says 172.21.15.82:56286) 2023-09-09T23:13:46.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.807+0000 7f99d54ea700 1 -- 172.21.15.82:0/3201361319 learned_addr learned my addr 172.21.15.82:0/3201361319 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:46.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.807+0000 7f99d4ce9700 1 --2- 172.21.15.82:0/3201361319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4f69be00 0x55ae4f6a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.808+0000 7f99cffff700 1 --2- 172.21.15.82:0/3201361319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae4f69cc00 0x55ae4f6a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.808+0000 7f99d4ce9700 1 -- 172.21.15.82:0/3201361319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae4f69cc00 msgr2=0x55ae4f6a5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:46.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.808+0000 7f99d4ce9700 1 --2- 172.21.15.82:0/3201361319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae4f69cc00 0x55ae4f6a5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.808+0000 7f99d4ce9700 1 -- 172.21.15.82:0/3201361319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae4f69d5a0 msgr2=0x55ae4f6aca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:46.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.808+0000 7f99d4ce9700 1 --2- 172.21.15.82:0/3201361319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae4f69d5a0 0x55ae4f6aca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.808+0000 7f99d4ce9700 1 -- 172.21.15.82:0/3201361319 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99d019e930 con 0x55ae4f69be00 2023-09-09T23:13:46.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.808+0000 7f99cffff700 1 --2- 172.21.15.82:0/3201361319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae4f69cc00 0x55ae4f6a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:46.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.808+0000 7f99d54ea700 1 --2- 172.21.15.82:0/3201361319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae4f69d5a0 0x55ae4f6aca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:46.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.808+0000 7f99d4ce9700 1 --2- 172.21.15.82:0/3201361319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4f69be00 0x55ae4f6a54e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f99c40098e0 tx=0x7f99c400ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a16b31bc8388d83 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:46.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.808+0000 7f99affff700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-09T23:13:46.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.808+0000 7f99affff700 1 -- 172.21.15.82:0/3201361319 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c402b070 con 0x55ae4f69be00 2023-09-09T23:13:46.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.808+0000 7f99e3113b80 1 -- 172.21.15.82:0/3201361319 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae4f5454f0 con 0x55ae4f69be00 2023-09-09T23:13:46.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.808+0000 7f99affff700 1 -- 172.21.15.82:0/3201361319 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f99c4004190 con 0x55ae4f69be00 2023-09-09T23:13:46.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.808+0000 7f99affff700 1 -- 172.21.15.82:0/3201361319 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99c4005960 con 0x55ae4f69be00 2023-09-09T23:13:46.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.808+0000 7f99e3113b80 10 client.25003 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:46.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.808+0000 7f99e3113b80 1 -- 172.21.15.82:0/3201361319 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f99d01093a0 con 0x55ae4f69be00 2023-09-09T23:13:46.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.809+0000 7f99affff700 1 -- 172.21.15.82:0/3201361319 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ae4f5454f0 con 0x55ae4f69be00 2023-09-09T23:13:46.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.809+0000 7f99affff700 1 -- 172.21.15.82:0/3201361319 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f99c400f460 con 0x55ae4f69be00 2023-09-09T23:13:46.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.809+0000 7f99e3113b80 10 client.25003 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:46.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.809+0000 7f99e3113b80 10 client.25003 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:13:46.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.809+0000 7f99e3113b80 20 client.25003 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:46.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.809+0000 7f99e3113b80 1 --2- 172.21.15.82:0/3201361319 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55ae4f6b0d40 0x55ae4f6b3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.809+0000 7f99e3113b80 4 client.25003 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:13:46.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.809+0000 7f99e3113b80 1 -- 172.21.15.82:0/3201361319 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae4f6028a0 con 0x55ae4f6b0d40 2023-09-09T23:13:46.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.809+0000 7f99cffff700 1 --2- 172.21.15.82:0/3201361319 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55ae4f6b0d40 0x55ae4f6b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.809+0000 7f99affff700 10 client.25003 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:13:46.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.809+0000 7f99cffff700 1 --2- 172.21.15.82:0/3201361319 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55ae4f6b0d40 0x55ae4f6b3130 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=794e0b2080429d91 server_cookie=e25203c163a5581 in_seq=0 out_seq=0 2023-09-09T23:13:46.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.810+0000 7f99affff700 1 -- 172.21.15.82:0/3201361319 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ae4f6028a0 con 0x55ae4f6b0d40 2023-09-09T23:13:46.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.810+0000 7f99affff700 10 client.25003 handle_command_reply: tid=0 2023-09-09T23:13:46.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.810+0000 7f99e3113b80 1 client.25003 shutdown 2023-09-09T23:13:46.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.810+0000 7f99e3113b80 1 -- 172.21.15.82:0/3201361319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4f69be00 msgr2=0x55ae4f6a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.810+0000 7f99e3113b80 1 --2- 172.21.15.82:0/3201361319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4f69be00 0x55ae4f6a54e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f99c40098e0 tx=0x7f99c400ba90 comp rx=0 tx=0).stop 2023-09-09T23:13:46.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.810+0000 7f99e3113b80 1 -- 172.21.15.82:0/3201361319 shutdown_connections 2023-09-09T23:13:46.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.810+0000 7f99e3113b80 1 --2- 172.21.15.82:0/3201361319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae4f69d5a0 0x55ae4f6aca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.810+0000 7f99e3113b80 1 --2- 172.21.15.82:0/3201361319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4f69be00 0x55ae4f6a54e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.810+0000 7f99e3113b80 1 --2- 172.21.15.82:0/3201361319 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55ae4f6b0d40 0x55ae4f6b3130 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.810+0000 7f99e3113b80 1 --2- 172.21.15.82:0/3201361319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae4f69cc00 0x55ae4f6a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.810+0000 7f99affff700 0 client.25003 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:13:46.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.810+0000 7f99e3113b80 1 -- 172.21.15.82:0/3201361319 >> 172.21.15.82:0/3201361319 conn(0x55ae4f610090 msgr2=0x55ae4f695dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:46.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.810+0000 7f99e3113b80 1 -- 172.21.15.82:0/3201361319 shutdown_connections 2023-09-09T23:13:46.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.810+0000 7f99e3113b80 1 -- 172.21.15.82:0/3201361319 wait complete. 2023-09-09T23:13:46.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.810+0000 7f99e3113b80 20 client.25003 trim_cache size 0 max 16384 2023-09-09T23:13:46.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.827+0000 7f99e3113b80 1 Processor -- start 2023-09-09T23:13:46.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.827+0000 7f99e3113b80 1 -- start start 2023-09-09T23:13:46.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.827+0000 7f99e3113b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae4f69be00 0x55ae4f6a3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.827+0000 7f99e3113b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4f69cc00 0x55ae4f6a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.827+0000 7f99e3113b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae4f69d5a0 0x55ae4f6aca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.827+0000 7f99e3113b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae4f6a4990 con 0x55ae4f69be00 2023-09-09T23:13:46.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.827+0000 7f99e3113b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae4f6a5050 con 0x55ae4f69d5a0 2023-09-09T23:13:46.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.827+0000 7f99e3113b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae4f660880 con 0x55ae4f69cc00 2023-09-09T23:13:46.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.827+0000 7f99d4ce9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae4f69be00 0x55ae4f6a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.827+0000 7f99d4ce9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae4f69be00 0x55ae4f6a3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56290/0 (socket says 172.21.15.82:56290) 2023-09-09T23:13:46.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.827+0000 7f99d4ce9700 1 -- 172.21.15.82:0/847184241 learned_addr learned my addr 172.21.15.82:0/847184241 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:46.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.827+0000 7f99d54ea700 1 --2- 172.21.15.82:0/847184241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae4f69d5a0 0x55ae4f6aca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.827+0000 7f99cffff700 1 --2- 172.21.15.82:0/847184241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4f69cc00 0x55ae4f6a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.827+0000 7f99d4ce9700 1 -- 172.21.15.82:0/847184241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4f69cc00 msgr2=0x55ae4f6a36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:46.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.827+0000 7f99d4ce9700 1 --2- 172.21.15.82:0/847184241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4f69cc00 0x55ae4f6a36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.827+0000 7f99d4ce9700 1 -- 172.21.15.82:0/847184241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae4f69d5a0 msgr2=0x55ae4f6aca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:46.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.827+0000 7f99d4ce9700 1 --2- 172.21.15.82:0/847184241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae4f69d5a0 0x55ae4f6aca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.827+0000 7f99d4ce9700 1 -- 172.21.15.82:0/847184241 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae4f402c60 con 0x55ae4f69be00 2023-09-09T23:13:46.836 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.827+0000 7f99cffff700 1 --2- 172.21.15.82:0/847184241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4f69cc00 0x55ae4f6a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:13:46.836 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.828+0000 7f99d4ce9700 1 --2- 172.21.15.82:0/847184241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae4f69be00 0x55ae4f6a3170 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f99c4000c00 tx=0x7f99c4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4832210105625df1 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:46.836 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.828+0000 7f99bb7fe700 1 -- 172.21.15.82:0/847184241 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c402b040 con 0x55ae4f69be00 2023-09-09T23:13:46.836 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.828+0000 7f99bb7fe700 1 -- 172.21.15.82:0/847184241 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99c40038c0 con 0x55ae4f69be00 2023-09-09T23:13:46.836 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.828+0000 7f99bb7fe700 1 -- 172.21.15.82:0/847184241 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c4004230 con 0x55ae4f69be00 2023-09-09T23:13:46.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.828+0000 7f99e3113b80 1 -- 172.21.15.82:0/847184241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae4f69be00 msgr2=0x55ae4f6a3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.828+0000 7f99e3113b80 1 --2- 172.21.15.82:0/847184241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae4f69be00 0x55ae4f6a3170 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f99c4000c00 tx=0x7f99c4005740 comp rx=0 tx=0).stop 2023-09-09T23:13:46.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.828+0000 7f99e3113b80 1 -- 172.21.15.82:0/847184241 shutdown_connections 2023-09-09T23:13:46.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.828+0000 7f99e3113b80 1 --2- 172.21.15.82:0/847184241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae4f69d5a0 0x55ae4f6aca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.828+0000 7f99e3113b80 1 --2- 172.21.15.82:0/847184241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae4f69be00 0x55ae4f6a3170 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.828+0000 7f99e3113b80 1 --2- 172.21.15.82:0/847184241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4f69cc00 0x55ae4f6a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.828+0000 7f99e3113b80 1 -- 172.21.15.82:0/847184241 >> 172.21.15.82:0/847184241 conn(0x55ae4f610090 msgr2=0x55ae4f69c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:46.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.828+0000 7f99e3113b80 1 -- 172.21.15.82:0/847184241 shutdown_connections 2023-09-09T23:13:46.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.828+0000 7f99e3113b80 1 -- 172.21.15.82:0/847184241 wait complete. 2023-09-09T23:13:46.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.829+0000 7f99e3113b80 1 -- start start 2023-09-09T23:13:46.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.829+0000 7f99e3113b80 1 Processor -- start 2023-09-09T23:13:46.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.829+0000 7f99e3113b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4f69be00 0x55ae4f6a8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.829+0000 7f99e3113b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae4f69cc00 0x55ae4f6a9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.829+0000 7f99e3113b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae4f69d5a0 0x55ae4f6ac270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.829+0000 7f99e3113b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae4f6ac7b0 con 0x55ae4f69d5a0 2023-09-09T23:13:46.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.829+0000 7f99e3113b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae4f6ac920 con 0x55ae4f69cc00 2023-09-09T23:13:46.840 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.829+0000 7f99e3113b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae4f6acae0 con 0x55ae4f69be00 2023-09-09T23:13:46.840 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.829+0000 7f99e3113b80 10 client.0 _ll_register_callbacks cb 0x7f99bc095580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:46.840 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.830+0000 7f99d54ea700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae4f69d5a0 0x55ae4f6ac270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.840 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.830+0000 7f99d54ea700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae4f69d5a0 0x55ae4f6ac270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56300/0 (socket says 172.21.15.82:56300) 2023-09-09T23:13:46.840 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.830+0000 7f99d54ea700 1 -- 172.21.15.82:0/2389169442 learned_addr learned my addr 172.21.15.82:0/2389169442 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:46.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.830+0000 7f99cffff700 1 --2- 172.21.15.82:0/2389169442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae4f69cc00 0x55ae4f6a9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.830+0000 7f99d4ce9700 1 --2- 172.21.15.82:0/2389169442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4f69be00 0x55ae4f6a8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.830+0000 7f99d54ea700 1 -- 172.21.15.82:0/2389169442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4f69be00 msgr2=0x55ae4f6a8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:46.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.830+0000 7f99d54ea700 1 --2- 172.21.15.82:0/2389169442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4f69be00 0x55ae4f6a8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.830+0000 7f99d54ea700 1 -- 172.21.15.82:0/2389169442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae4f69cc00 msgr2=0x55ae4f6a9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:46.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.830+0000 7f99d54ea700 1 --2- 172.21.15.82:0/2389169442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae4f69cc00 0x55ae4f6a9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.830+0000 7f99d54ea700 1 -- 172.21.15.82:0/2389169442 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99d01093a0 con 0x55ae4f69d5a0 2023-09-09T23:13:46.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.830+0000 7f99d54ea700 1 --2- 172.21.15.82:0/2389169442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae4f69d5a0 0x55ae4f6ac270 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f99c8018f00 tx=0x7f99c8009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e2cb3ced8f5d18 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:46.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.830+0000 7f99affff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:46.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.830+0000 7f99affff700 1 -- 172.21.15.82:0/2389169442 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c8007d40 con 0x55ae4f69d5a0 2023-09-09T23:13:46.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.830+0000 7f99affff700 1 -- 172.21.15.82:0/2389169442 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f99c80203f0 con 0x55ae4f69d5a0 2023-09-09T23:13:46.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.830+0000 7f99e3113b80 1 -- 172.21.15.82:0/2389169442 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99c401e040 con 0x55ae4f69d5a0 2023-09-09T23:13:46.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.830+0000 7f99affff700 1 -- 172.21.15.82:0/2389169442 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99c8007ea0 con 0x55ae4f69d5a0 2023-09-09T23:13:46.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.830+0000 7f99e3113b80 10 client.15576 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:46.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.830+0000 7f99e3113b80 1 -- 172.21.15.82:0/2389169442 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ae4f440250 con 0x55ae4f69d5a0 2023-09-09T23:13:46.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.831+0000 7f99affff700 1 -- 172.21.15.82:0/2389169442 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99c401e040 con 0x55ae4f69d5a0 2023-09-09T23:13:46.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.831+0000 7f99affff700 1 -- 172.21.15.82:0/2389169442 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f99c8016580 con 0x55ae4f69d5a0 2023-09-09T23:13:46.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.831+0000 7f99e3113b80 10 client.15576 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:46.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.831+0000 7f99e3113b80 10 client.15576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:13:46.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.831+0000 7f99e3113b80 20 client.15576 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:46.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.831+0000 7f99e3113b80 1 --2- 172.21.15.82:0/2389169442 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55ae4f705ff0 0x55ae4f708410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.831+0000 7f99e3113b80 4 client.15576 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:13:46.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.831+0000 7f99e3113b80 1 -- 172.21.15.82:0/2389169442 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ae4f5e6920 con 0x55ae4f705ff0 2023-09-09T23:13:46.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.831+0000 7f99d4ce9700 1 --2- 172.21.15.82:0/2389169442 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55ae4f705ff0 0x55ae4f708410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.831+0000 7f99affff700 10 client.15576 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:13:46.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.831+0000 7f99d4ce9700 1 --2- 172.21.15.82:0/2389169442 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55ae4f705ff0 0x55ae4f708410 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=905e3171ee8a0095 server_cookie=7e0355d321e3d324 in_seq=0 out_seq=0 2023-09-09T23:13:46.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.832+0000 7f99affff700 1 -- 172.21.15.82:0/2389169442 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ae4f5e6920 con 0x55ae4f705ff0 2023-09-09T23:13:46.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.832+0000 7f99affff700 10 client.15576 handle_command_reply: tid=0 2023-09-09T23:13:46.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.832+0000 7f99e3113b80 1 client.15576 shutdown 2023-09-09T23:13:46.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.832+0000 7f99e3113b80 1 -- 172.21.15.82:0/2389169442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae4f69d5a0 msgr2=0x55ae4f6ac270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.832+0000 7f99e3113b80 1 --2- 172.21.15.82:0/2389169442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae4f69d5a0 0x55ae4f6ac270 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f99c8018f00 tx=0x7f99c8009f60 comp rx=0 tx=0).stop 2023-09-09T23:13:46.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.832+0000 7f99e3113b80 1 -- 172.21.15.82:0/2389169442 shutdown_connections 2023-09-09T23:13:46.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.832+0000 7f99e3113b80 1 --2- 172.21.15.82:0/2389169442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ae4f69d5a0 0x55ae4f6ac270 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.832+0000 7f99e3113b80 1 --2- 172.21.15.82:0/2389169442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ae4f69be00 0x55ae4f6a8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.832+0000 7f99e3113b80 1 --2- 172.21.15.82:0/2389169442 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55ae4f705ff0 0x55ae4f708410 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.832+0000 7f99e3113b80 1 --2- 172.21.15.82:0/2389169442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae4f69cc00 0x55ae4f6a9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.832+0000 7f99e3113b80 1 -- 172.21.15.82:0/2389169442 >> 172.21.15.82:0/2389169442 conn(0x55ae4f610090 msgr2=0x55ae4f6a3170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:46.848 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.832+0000 7f99affff700 0 client.15576 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:13:46.848 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.832+0000 7f99e3113b80 1 -- 172.21.15.82:0/2389169442 shutdown_connections 2023-09-09T23:13:46.848 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.832+0000 7f99e3113b80 1 -- 172.21.15.82:0/2389169442 wait complete. 2023-09-09T23:13:46.848 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.832+0000 7f99e3113b80 20 client.15576 trim_cache size 0 max 16384 2023-09-09T23:13:46.848 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:13:46.849 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "no active scrubs running", 2023-09-09T23:13:46.849 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": {} 2023-09-09T23:13:46.849 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:13:46.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.832+0000 7f99d6f4d700 1 -- 172.21.15.82:0/645670298 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f99c00a0640 msgr2=0x7f99c00a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.832+0000 7f99d6f4d700 1 --2- 172.21.15.82:0/645670298 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f99c00a0640 0x7f99c00a2b00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f99c400b580 tx=0x7f99c4005fb0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.832+0000 7f99d6f4d700 1 -- 172.21.15.82:0/645670298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99d0101800 msgr2=0x7f99d0196880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.832+0000 7f99d6f4d700 1 --2- 172.21.15.82:0/645670298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99d0101800 0x7f99d0196880 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00d980 tx=0x7f99bc00dc90 comp rx=0 tx=0).stop 2023-09-09T23:13:46.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.833+0000 7f99d6f4d700 1 -- 172.21.15.82:0/645670298 shutdown_connections 2023-09-09T23:13:46.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.833+0000 7f99d6f4d700 1 --2- 172.21.15.82:0/645670298 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f99c00a0640 0x7f99c00a2b00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.833+0000 7f99d6f4d700 1 --2- 172.21.15.82:0/645670298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99d0100610 0x7f99d0196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.833+0000 7f99d6f4d700 1 --2- 172.21.15.82:0/645670298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99d01021a0 0x7f99d019d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.833+0000 7f99d6f4d700 1 --2- 172.21.15.82:0/645670298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99d0101800 0x7f99d0196880 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.833+0000 7f99d6f4d700 1 -- 172.21.15.82:0/645670298 >> 172.21.15.82:0/645670298 conn(0x7f99d00fbb80 msgr2=0x7f99d00fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:46.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.833+0000 7f99d6f4d700 1 -- 172.21.15.82:0/645670298 shutdown_connections 2023-09-09T23:13:46.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.833+0000 7f99d6f4d700 1 -- 172.21.15.82:0/645670298 wait complete. 2023-09-09T23:13:46.853 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-09T23:13:46.853 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-09T23:13:46.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.882+0000 7f9d482dc700 1 -- 172.21.15.82:0/2666960734 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f9d4010bfe0 con 0x7f9d40106330 2023-09-09T23:13:46.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.978+0000 7f5b33e97700 1 -- 172.21.15.82:0/2175074119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b2c103a30 msgr2=0x7f5b2c105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.978+0000 7f5b33e97700 1 --2- 172.21.15.82:0/2175074119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b2c103a30 0x7f5b2c105e20 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5b28009990 tx=0x7f5b28009ca0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.978+0000 7f5b33e97700 1 -- 172.21.15.82:0/2175074119 shutdown_connections 2023-09-09T23:13:46.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.978+0000 7f5b33e97700 1 --2- 172.21.15.82:0/2175074119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b2c106360 0x7f5b2c108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.978+0000 7f5b33e97700 1 --2- 172.21.15.82:0/2175074119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b2c101100 0x7f5b2c1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.978+0000 7f5b33e97700 1 --2- 172.21.15.82:0/2175074119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b2c103a30 0x7f5b2c105e20 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.978+0000 7f5b33e97700 1 -- 172.21.15.82:0/2175074119 >> 172.21.15.82:0/2175074119 conn(0x7f5b2c0faa90 msgr2=0x7f5b2c0fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:46.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.978+0000 7f5b33e97700 1 -- 172.21.15.82:0/2175074119 shutdown_connections 2023-09-09T23:13:46.980 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.978+0000 7f5b33e97700 1 -- 172.21.15.82:0/2175074119 wait complete. 2023-09-09T23:13:46.980 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.979+0000 7f5b33e97700 1 Processor -- start 2023-09-09T23:13:46.980 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.979+0000 7f5b33e97700 1 -- start start 2023-09-09T23:13:46.980 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.979+0000 7f5b33e97700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b2c101100 0x7f5b2c1961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.979+0000 7f5b33e97700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b2c103a30 0x7f5b2c1966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.979+0000 7f5b33e97700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b2c106360 0x7f5b2c19d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.979+0000 7f5b33e97700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c19dd30 con 0x7f5b2c101100 2023-09-09T23:13:46.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.979+0000 7f5b33e97700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c19de70 con 0x7f5b2c103a30 2023-09-09T23:13:46.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.979+0000 7f5b33e97700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c19e000 con 0x7f5b2c106360 2023-09-09T23:13:46.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.979+0000 7f5b31c33700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b2c101100 0x7f5b2c1961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.979+0000 7f5b31c33700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b2c101100 0x7f5b2c1961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56308/0 (socket says 172.21.15.82:56308) 2023-09-09T23:13:46.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.979+0000 7f5b31c33700 1 -- 172.21.15.82:0/2043204235 learned_addr learned my addr 172.21.15.82:0/2043204235 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:46.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.979+0000 7f5b32434700 1 --2- 172.21.15.82:0/2043204235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b2c106360 0x7f5b2c19d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.979+0000 7f5b31c33700 1 -- 172.21.15.82:0/2043204235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b2c106360 msgr2=0x7f5b2c19d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.979+0000 7f5b31c33700 1 --2- 172.21.15.82:0/2043204235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b2c106360 0x7f5b2c19d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.979+0000 7f5b31c33700 1 -- 172.21.15.82:0/2043204235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b2c103a30 msgr2=0x7f5b2c1966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:46.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.979+0000 7f5b31432700 1 --2- 172.21.15.82:0/2043204235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b2c103a30 0x7f5b2c1966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.980+0000 7f5b31c33700 1 --2- 172.21.15.82:0/2043204235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b2c103a30 0x7f5b2c1966f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:46.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.980+0000 7f5b31c33700 1 -- 172.21.15.82:0/2043204235 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b280095f0 con 0x7f5b2c101100 2023-09-09T23:13:46.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.980+0000 7f5b32434700 1 --2- 172.21.15.82:0/2043204235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b2c106360 0x7f5b2c19d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:46.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.980+0000 7f5b31c33700 1 --2- 172.21.15.82:0/2043204235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b2c101100 0x7f5b2c1961b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c00e9a0 tx=0x7f5b1c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:46.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.980+0000 7f5b22ffd700 1 -- 172.21.15.82:0/2043204235 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c009790 con 0x7f5b2c101100 2023-09-09T23:13:46.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.980+0000 7f5b22ffd700 1 -- 172.21.15.82:0/2043204235 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b1c00cbd0 con 0x7f5b2c101100 2023-09-09T23:13:46.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.980+0000 7f5b33e97700 1 -- 172.21.15.82:0/2043204235 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2c19e2e0 con 0x7f5b2c101100 2023-09-09T23:13:46.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.980+0000 7f5b22ffd700 1 -- 172.21.15.82:0/2043204235 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c018680 con 0x7f5b2c101100 2023-09-09T23:13:46.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.980+0000 7f5b33e97700 1 -- 172.21.15.82:0/2043204235 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b2c19e7f0 con 0x7f5b2c101100 2023-09-09T23:13:46.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.981+0000 7f5b33e97700 1 -- 172.21.15.82:0/2043204235 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b2c0fc610 con 0x7f5b2c101100 2023-09-09T23:13:46.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.982+0000 7f5b22ffd700 1 -- 172.21.15.82:0/2043204235 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135259+0+0 (secure 0 0 0) 0x7f5b1c00cd40 con 0x7f5b2c101100 2023-09-09T23:13:46.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.982+0000 7f5b22ffd700 1 --2- 172.21.15.82:0/2043204235 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5b180a0690 0x7f5b180a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:46.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.983+0000 7f5b31432700 1 --2- 172.21.15.82:0/2043204235 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5b180a0690 0x7f5b180a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:46.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.983+0000 7f5b22ffd700 1 -- 172.21.15.82:0/2043204235 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f5b1c014070 con 0x7f5b2c101100 2023-09-09T23:13:46.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.983+0000 7f5b31432700 1 --2- 172.21.15.82:0/2043204235 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5b180a0690 0x7f5b180a2b50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5b280098e0 tx=0x7f5b28000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:46.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:46.986+0000 7f5b22ffd700 1 -- 172.21.15.82:0/2043204235 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5b1c095f00 con 0x7f5b2c101100 2023-09-09T23:13:47.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.222+0000 7f5b33e97700 1 -- 172.21.15.82:0/2043204235 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b2c10be80 con 0x7f5b2c101100 2023-09-09T23:13:47.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.222+0000 7f5b22ffd700 1 -- 172.21.15.82:0/2043204235 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11368 (secure 0 0 0) 0x7f5b1c099520 con 0x7f5b2c101100 2023-09-09T23:13:47.220 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:47.220 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2471432684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2471432684},{"type":"v1","addr":"172.21.15.82:6835","nonce":2471432684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3579387137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3579387137},{"type":"v1","addr":"172.21.15.82:6837","nonce":3579387137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-09T23:12:17.586036+0000","modified":"2023-09-09T23:12:25.285958+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24542,"mds_2":24523,"mds_3":24491,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14712":{"gid":14712,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/2127301345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2127301345},{"type":"v1","addr":"172.21.15.82:6839","nonce":2127301345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14724":{"gid":14724,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/4143604488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":4143604488},{"type":"v1","addr":"172.21.15.82:6841","nonce":4143604488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.170:6833/1931961940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1931961940},{"type":"v1","addr":"172.21.15.170:6833","nonce":1931961940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24491":{"gid":24491,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.98:6833/3855146755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3855146755},{"type":"v1","addr":"172.21.15.98:6833","nonce":3855146755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2313861511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2313861511},{"type":"v1","addr":"172.21.15.170:6835","nonce":2313861511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/4006360308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":4006360308},{"type":"v1","addr":"172.21.15.170:6837","nonce":4006360308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1356474828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1356474828},{"type":"v1","addr":"172.21.15.98:6835","nonce":1356474828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/3858369735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3858369735},{"type":"v1","addr":"172.21.15.170:6839","nonce":3858369735}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/2019449840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2019449840},{"type":"v1","addr":"172.21.15.98:6837","nonce":2019449840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2785086371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2785086371},{"type":"v1","addr":"172.21.15.98:6839","nonce":2785086371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-09T23:13:47.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.224+0000 7f5b33e97700 1 -- 172.21.15.82:0/2043204235 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5b180a0690 msgr2=0x7f5b180a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.224+0000 7f5b33e97700 1 --2- 172.21.15.82:0/2043204235 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5b180a0690 0x7f5b180a2b50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5b280098e0 tx=0x7f5b28000bc0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.224+0000 7f5b33e97700 1 -- 172.21.15.82:0/2043204235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b2c101100 msgr2=0x7f5b2c1961b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.224+0000 7f5b33e97700 1 --2- 172.21.15.82:0/2043204235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b2c101100 0x7f5b2c1961b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c00e9a0 tx=0x7f5b1c00ed60 comp rx=0 tx=0).stop 2023-09-09T23:13:47.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.224+0000 7f5b33e97700 1 -- 172.21.15.82:0/2043204235 shutdown_connections 2023-09-09T23:13:47.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.224+0000 7f5b33e97700 1 --2- 172.21.15.82:0/2043204235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b2c106360 0x7f5b2c19d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.224+0000 7f5b33e97700 1 --2- 172.21.15.82:0/2043204235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b2c101100 0x7f5b2c1961b0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.224+0000 7f5b33e97700 1 --2- 172.21.15.82:0/2043204235 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f5b180a0690 0x7f5b180a2b50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.224+0000 7f5b33e97700 1 --2- 172.21.15.82:0/2043204235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b2c103a30 0x7f5b2c1966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.224+0000 7f5b33e97700 1 -- 172.21.15.82:0/2043204235 >> 172.21.15.82:0/2043204235 conn(0x7f5b2c0faa90 msgr2=0x7f5b2c101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:47.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.225+0000 7f5b33e97700 1 -- 172.21.15.82:0/2043204235 shutdown_connections 2023-09-09T23:13:47.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.225+0000 7f5b33e97700 1 -- 172.21.15.82:0/2043204235 wait complete. 2023-09-09T23:13:47.225 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 27 2023-09-09T23:13:47.235 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-09-09T23:13:47.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.354+0000 7f9d377fe700 1 -- 172.21.15.82:0/2666960734 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mgrmap(e 19) v1 ==== 135276+0+0 (secure 0 0 0) 0x7f9d38095a90 con 0x7f9d40106330 2023-09-09T23:13:47.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.356+0000 7f9d377fe700 1 -- 172.21.15.82:0/2666960734 <== mon.2 v2:172.21.15.98:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f9d38099150 con 0x7f9d40106330 2023-09-09T23:13:47.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.359+0000 7f9d482dc700 1 -- 172.21.15.82:0/2666960734 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f9d2c0a0640 msgr2=0x7f9d2c0a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.359+0000 7f9d482dc700 1 --2- 172.21.15.82:0/2666960734 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f9d2c0a0640 0x7f9d2c0a2b00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00b2e0 tx=0x7f9d3c005fb0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.359+0000 7f9d482dc700 1 -- 172.21.15.82:0/2666960734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d40106330 msgr2=0x7f9d40197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.359+0000 7f9d482dc700 1 --2- 172.21.15.82:0/2666960734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d40106330 0x7f9d40197790 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800c370 tx=0x7f9d3800c680 comp rx=0 tx=0).stop 2023-09-09T23:13:47.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.359+0000 7f9d482dc700 1 -- 172.21.15.82:0/2666960734 shutdown_connections 2023-09-09T23:13:47.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.359+0000 7f9d482dc700 1 --2- 172.21.15.82:0/2666960734 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f9d2c0a0640 0x7f9d2c0a2b00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.359+0000 7f9d482dc700 1 --2- 172.21.15.82:0/2666960734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d40106330 0x7f9d40197790 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.359+0000 7f9d482dc700 1 --2- 172.21.15.82:0/2666960734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d401010d0 0x7f9d401963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.359+0000 7f9d482dc700 1 --2- 172.21.15.82:0/2666960734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d40103a00 0x7f9d40196910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.359+0000 7f9d482dc700 1 -- 172.21.15.82:0/2666960734 >> 172.21.15.82:0/2666960734 conn(0x7f9d400faac0 msgr2=0x7f9d40101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:47.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.359+0000 7f9d482dc700 1 -- 172.21.15.82:0/2666960734 shutdown_connections 2023-09-09T23:13:47.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.359+0000 7f9d482dc700 1 -- 172.21.15.82:0/2666960734 wait complete. 2023-09-09T23:13:47.369 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2023-09-09T23:13:47.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.421+0000 7f103a808700 1 -- 172.21.15.82:0/1087459621 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1034103a70 msgr2=0x7f1034105e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.421+0000 7f103a808700 1 --2- 172.21.15.82:0/1087459621 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1034103a70 0x7f1034105e60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1028009990 tx=0x7f1028009ca0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.422+0000 7f103a808700 1 -- 172.21.15.82:0/1087459621 shutdown_connections 2023-09-09T23:13:47.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.422+0000 7f103a808700 1 --2- 172.21.15.82:0/1087459621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10341063a0 0x7f10341087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.422+0000 7f103a808700 1 --2- 172.21.15.82:0/1087459621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1034101140 0x7f1034103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.422+0000 7f103a808700 1 --2- 172.21.15.82:0/1087459621 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1034103a70 0x7f1034105e60 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.422+0000 7f103a808700 1 -- 172.21.15.82:0/1087459621 >> 172.21.15.82:0/1087459621 conn(0x7f10340faad0 msgr2=0x7f10340fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:47.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.422+0000 7f103a808700 1 -- 172.21.15.82:0/1087459621 shutdown_connections 2023-09-09T23:13:47.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.422+0000 7f103a808700 1 -- 172.21.15.82:0/1087459621 wait complete. 2023-09-09T23:13:47.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.422+0000 7f103a808700 1 Processor -- start 2023-09-09T23:13:47.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.422+0000 7f103a808700 1 -- start start 2023-09-09T23:13:47.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.423+0000 7f103a808700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1034101140 0x7f10341962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.423+0000 7f103a808700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1034103a70 0x7f1034196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.423+0000 7f103a808700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10341063a0 0x7f103419d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.423+0000 7f103a808700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103419de50 con 0x7f1034103a70 2023-09-09T23:13:47.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.423+0000 7f103a808700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103419df90 con 0x7f10341063a0 2023-09-09T23:13:47.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.423+0000 7f103a808700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103419e120 con 0x7f1034101140 2023-09-09T23:13:47.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.423+0000 7f10337fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1034103a70 0x7f1034196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.423+0000 7f1033fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1034101140 0x7f10341962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.423+0000 7f1033fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1034101140 0x7f10341962d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:54862/0 (socket says 172.21.15.82:54862) 2023-09-09T23:13:47.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.423+0000 7f1033fff700 1 -- 172.21.15.82:0/1693469271 learned_addr learned my addr 172.21.15.82:0/1693469271 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:47.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.423+0000 7f10337fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1034103a70 0x7f1034196810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56350/0 (socket says 172.21.15.82:56350) 2023-09-09T23:13:47.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.423+0000 7f1038da5700 1 --2- 172.21.15.82:0/1693469271 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10341063a0 0x7f103419d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.423+0000 7f1033fff700 1 -- 172.21.15.82:0/1693469271 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10341063a0 msgr2=0x7f103419d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.423+0000 7f1033fff700 1 --2- 172.21.15.82:0/1693469271 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10341063a0 0x7f103419d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.423+0000 7f1033fff700 1 -- 172.21.15.82:0/1693469271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1034103a70 msgr2=0x7f1034196810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.423+0000 7f1033fff700 1 --2- 172.21.15.82:0/1693469271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1034103a70 0x7f1034196810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.423+0000 7f1033fff700 1 -- 172.21.15.82:0/1693469271 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10280095f0 con 0x7f1034101140 2023-09-09T23:13:47.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.423+0000 7f10337fe700 1 --2- 172.21.15.82:0/1693469271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1034103a70 0x7f1034196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:47.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.424+0000 7f1033fff700 1 --2- 172.21.15.82:0/1693469271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1034101140 0x7f10341962d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f102400e9a0 tx=0x7f102400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:47.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.424+0000 7f10317fa700 1 -- 172.21.15.82:0/1693469271 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024009790 con 0x7f1034101140 2023-09-09T23:13:47.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.424+0000 7f103a808700 1 -- 172.21.15.82:0/1693469271 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f103419e400 con 0x7f1034101140 2023-09-09T23:13:47.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.424+0000 7f10317fa700 1 -- 172.21.15.82:0/1693469271 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1024010430 con 0x7f1034101140 2023-09-09T23:13:47.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.424+0000 7f10317fa700 1 -- 172.21.15.82:0/1693469271 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024018680 con 0x7f1034101140 2023-09-09T23:13:47.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.424+0000 7f103a808700 1 -- 172.21.15.82:0/1693469271 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f103419e910 con 0x7f1034101140 2023-09-09T23:13:47.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.427+0000 7f10317fa700 1 -- 172.21.15.82:0/1693469271 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 19) v1 ==== 135276+0+0 (secure 0 0 0) 0x7f10240105a0 con 0x7f1034101140 2023-09-09T23:13:47.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.427+0000 7f10317fa700 1 --2- 172.21.15.82:0/1693469271 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f101c0a0690 0x7f101c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.427+0000 7f10469ceb80 1 Processor -- start 2023-09-09T23:13:47.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.428+0000 7f10469ceb80 1 -- start start 2023-09-09T23:13:47.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.428+0000 7f10337fe700 1 -- 172.21.15.82:0/1693469271 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f101c0a0690 msgr2=0x7f101c0a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1453069857 2023-09-09T23:13:47.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.428+0000 7f10317fa700 1 -- 172.21.15.82:0/1693469271 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f1024014070 con 0x7f1034101140 2023-09-09T23:13:47.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.428+0000 7f10337fe700 1 --2- 172.21.15.82:0/1693469271 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f101c0a0690 0x7f101c0a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-09T23:13:47.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.428+0000 7f10469ceb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55faa4db3c00 0x55faa4db4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.428+0000 7f10469ceb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faa4db45a0 0x55faa4db2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.428+0000 7f10469ceb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55faa4db2e00 0x55faa4db32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.428+0000 7f10469ceb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa4d0b730 con 0x55faa4db45a0 2023-09-09T23:13:47.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.428+0000 7f10469ceb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa4d09050 con 0x55faa4db2e00 2023-09-09T23:13:47.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.428+0000 7f10469ceb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa4d04940 con 0x55faa4db3c00 2023-09-09T23:13:47.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.428+0000 7f1033fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55faa4db3c00 0x55faa4db4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.428+0000 7f1033fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55faa4db3c00 0x55faa4db4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:54870/0 (socket says 172.21.15.82:54870) 2023-09-09T23:13:47.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.428+0000 7f1033fff700 1 -- 172.21.15.82:0/1182329753 learned_addr learned my addr 172.21.15.82:0/1182329753 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:47.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.428+0000 7f1038da5700 1 --2- 172.21.15.82:0/1182329753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55faa4db2e00 0x55faa4db32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.428+0000 7f10337fe700 1 --2- 172.21.15.82:0/1182329753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faa4db45a0 0x55faa4db2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.428+0000 7f1033fff700 1 -- 172.21.15.82:0/1182329753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55faa4db2e00 msgr2=0x55faa4db32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:47.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.428+0000 7f1033fff700 1 --2- 172.21.15.82:0/1182329753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55faa4db2e00 0x55faa4db32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.428+0000 7f1033fff700 1 -- 172.21.15.82:0/1182329753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faa4db45a0 msgr2=0x55faa4db2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:47.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.428+0000 7f1033fff700 1 --2- 172.21.15.82:0/1182329753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faa4db45a0 0x55faa4db2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.428+0000 7f1033fff700 1 -- 172.21.15.82:0/1182329753 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55faa4b30de0 con 0x55faa4db3c00 2023-09-09T23:13:47.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.428+0000 7f10337fe700 1 --2- 172.21.15.82:0/1182329753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faa4db45a0 0x55faa4db2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:47.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.428+0000 7f1038da5700 1 --2- 172.21.15.82:0/1182329753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55faa4db2e00 0x55faa4db32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:47.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.429+0000 7f1033fff700 1 --2- 172.21.15.82:0/1182329753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55faa4db3c00 0x55faa4db4020 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1024095a90 tx=0x7f10240991d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fde37dc54d224227 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:47.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.429+0000 7f101a7fc700 1 -- 172.21.15.82:0/1182329753 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024099590 con 0x55faa4db3c00 2023-09-09T23:13:47.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.429+0000 7f101a7fc700 1 -- 172.21.15.82:0/1182329753 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f102409d3c0 con 0x55faa4db3c00 2023-09-09T23:13:47.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.429+0000 7f101a7fc700 1 -- 172.21.15.82:0/1182329753 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024084070 con 0x55faa4db3c00 2023-09-09T23:13:47.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.429+0000 7f10469ceb80 1 -- 172.21.15.82:0/1182329753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55faa4db3c00 msgr2=0x55faa4db4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.429+0000 7f10469ceb80 1 --2- 172.21.15.82:0/1182329753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55faa4db3c00 0x55faa4db4020 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1024095a90 tx=0x7f10240991d0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.430+0000 7f10469ceb80 1 -- 172.21.15.82:0/1182329753 shutdown_connections 2023-09-09T23:13:47.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.430+0000 7f10469ceb80 1 --2- 172.21.15.82:0/1182329753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55faa4db2e00 0x55faa4db32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.430+0000 7f10469ceb80 1 --2- 172.21.15.82:0/1182329753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55faa4db3c00 0x55faa4db4020 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.430+0000 7f10469ceb80 1 --2- 172.21.15.82:0/1182329753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faa4db45a0 0x55faa4db2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.430+0000 7f10469ceb80 1 -- 172.21.15.82:0/1182329753 >> 172.21.15.82:0/1182329753 conn(0x55faa4d27090 msgr2=0x55faa4dad720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:47.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.430+0000 7f10469ceb80 1 -- 172.21.15.82:0/1182329753 shutdown_connections 2023-09-09T23:13:47.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.430+0000 7f10469ceb80 1 -- 172.21.15.82:0/1182329753 wait complete. 2023-09-09T23:13:47.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.430+0000 7f10469ceb80 1 -- start start 2023-09-09T23:13:47.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.430+0000 7f10469ceb80 1 Processor -- start 2023-09-09T23:13:47.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.431+0000 7f10469ceb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faa4db2e00 0x55faa4dbc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.431+0000 7f10469ceb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55faa4db3c00 0x55faa4dbcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.431+0000 7f10469ceb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55faa4db45a0 0x55faa4dc3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.431+0000 7f10469ceb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa4ca2b70 con 0x55faa4db2e00 2023-09-09T23:13:47.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.431+0000 7f10469ceb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa4ca2870 con 0x55faa4db3c00 2023-09-09T23:13:47.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.431+0000 7f10469ceb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa4b86920 con 0x55faa4db45a0 2023-09-09T23:13:47.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.431+0000 7f10469ceb80 10 client.0 _ll_register_callbacks cb 0x55faa4beca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:47.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.431+0000 7f1033fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faa4db2e00 0x55faa4dbc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.431+0000 7f10337fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55faa4db3c00 0x55faa4dbcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.431+0000 7f1033fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faa4db2e00 0x55faa4dbc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56362/0 (socket says 172.21.15.82:56362) 2023-09-09T23:13:47.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.431+0000 7f10337fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55faa4db3c00 0x55faa4dbcb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:41928/0 (socket says 172.21.15.82:41928) 2023-09-09T23:13:47.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.431+0000 7f1033fff700 1 -- 172.21.15.82:0/904342434 learned_addr learned my addr 172.21.15.82:0/904342434 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:47.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.431+0000 7f1038da5700 1 --2- 172.21.15.82:0/904342434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55faa4db45a0 0x55faa4dc3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.431+0000 7f1033fff700 1 -- 172.21.15.82:0/904342434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55faa4db45a0 msgr2=0x55faa4dc3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:47.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.431+0000 7f1033fff700 1 --2- 172.21.15.82:0/904342434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55faa4db45a0 0x55faa4dc3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.431+0000 7f1033fff700 1 -- 172.21.15.82:0/904342434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55faa4db3c00 msgr2=0x55faa4dbcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:47.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.431+0000 7f1033fff700 1 --2- 172.21.15.82:0/904342434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55faa4db3c00 0x55faa4dbcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.431+0000 7f1033fff700 1 -- 172.21.15.82:0/904342434 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f102c011230 con 0x55faa4db2e00 2023-09-09T23:13:47.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.431+0000 7f10337fe700 1 --2- 172.21.15.82:0/904342434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55faa4db3c00 0x55faa4dbcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:47.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.431+0000 7f1038da5700 1 --2- 172.21.15.82:0/904342434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55faa4db45a0 0x55faa4dc3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:47.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.432+0000 7f1033fff700 1 --2- 172.21.15.82:0/904342434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faa4db2e00 0x55faa4dbc600 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1024099bf0 tx=0x7f102400c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713280b79078d7e6 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:47.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.432+0000 7f10137fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:47.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.432+0000 7f10137fe700 1 -- 172.21.15.82:0/904342434 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024098c80 con 0x55faa4db2e00 2023-09-09T23:13:47.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.432+0000 7f10137fe700 1 -- 172.21.15.82:0/904342434 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f1024099590 con 0x55faa4db2e00 2023-09-09T23:13:47.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.432+0000 7f10469ceb80 1 -- 172.21.15.82:0/904342434 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55faa4c5c4f0 con 0x55faa4db2e00 2023-09-09T23:13:47.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.432+0000 7f10137fe700 1 -- 172.21.15.82:0/904342434 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10240651f0 con 0x55faa4db2e00 2023-09-09T23:13:47.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.432+0000 7f10469ceb80 10 client.15609 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:47.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.432+0000 7f10469ceb80 1 -- 172.21.15.82:0/904342434 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f10341093a0 con 0x55faa4db2e00 2023-09-09T23:13:47.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.432+0000 7f10137fe700 1 -- 172.21.15.82:0/904342434 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f102407c020 con 0x55faa4db2e00 2023-09-09T23:13:47.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.432+0000 7f10137fe700 1 -- 172.21.15.82:0/904342434 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f10240b1c20 con 0x55faa4db2e00 2023-09-09T23:13:47.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.432+0000 7f10469ceb80 10 client.15609 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:47.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.432+0000 7f10469ceb80 10 client.15609 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2023-09-09T23:13:47.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.432+0000 7f10469ceb80 20 client.15609 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:47.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.432+0000 7f10469ceb80 1 --2- 172.21.15.82:0/904342434 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x55faa4dc7f70 0x55faa4dca390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.432+0000 7f10469ceb80 4 client.15609 mds_command: new command op to 24460 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:13:47.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.432+0000 7f10469ceb80 1 -- 172.21.15.82:0/904342434 --> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55faa4d198a0 con 0x55faa4dc7f70 2023-09-09T23:13:47.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.433+0000 7f10337fe700 1 --2- 172.21.15.82:0/904342434 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x55faa4dc7f70 0x55faa4dca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.433+0000 7f10137fe700 10 client.15609 ms_handle_connect on v2:172.21.15.170:6832/1931961940 2023-09-09T23:13:47.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.433+0000 7f10337fe700 1 --2- 172.21.15.82:0/904342434 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x55faa4dc7f70 0x55faa4dca390 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3804a95b940383b5 server_cookie=a904bc80c7d6f934 in_seq=0 out_seq=0 2023-09-09T23:13:47.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.434+0000 7f10137fe700 1 -- 172.21.15.82:0/904342434 <== mds.4 v2:172.21.15.170:6832/1931961940 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55faa4d198a0 con 0x55faa4dc7f70 2023-09-09T23:13:47.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.434+0000 7f10137fe700 10 client.15609 handle_command_reply: tid=0 2023-09-09T23:13:47.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.434+0000 7f10469ceb80 1 client.15609 shutdown 2023-09-09T23:13:47.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.434+0000 7f10469ceb80 1 -- 172.21.15.82:0/904342434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faa4db2e00 msgr2=0x55faa4dbc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.434+0000 7f10469ceb80 1 --2- 172.21.15.82:0/904342434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faa4db2e00 0x55faa4dbc600 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1024099bf0 tx=0x7f102400c4a0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.434+0000 7f10469ceb80 1 -- 172.21.15.82:0/904342434 shutdown_connections 2023-09-09T23:13:47.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.434+0000 7f10469ceb80 1 --2- 172.21.15.82:0/904342434 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x55faa4dc7f70 0x55faa4dca390 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.434+0000 7f10469ceb80 1 --2- 172.21.15.82:0/904342434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55faa4db45a0 0x55faa4dc3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.434+0000 7f10469ceb80 1 --2- 172.21.15.82:0/904342434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faa4db2e00 0x55faa4dbc600 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.434+0000 7f10137fe700 0 client.15609 ms_handle_reset on v2:172.21.15.170:6832/1931961940 2023-09-09T23:13:47.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.434+0000 7f10469ceb80 1 --2- 172.21.15.82:0/904342434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55faa4db3c00 0x55faa4dbcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.434+0000 7f10469ceb80 1 -- 172.21.15.82:0/904342434 >> 172.21.15.82:0/904342434 conn(0x55faa4d27090 msgr2=0x55faa4dacdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:47.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.434+0000 7f10469ceb80 1 -- 172.21.15.82:0/904342434 shutdown_connections 2023-09-09T23:13:47.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.434+0000 7f10469ceb80 1 -- 172.21.15.82:0/904342434 wait complete. 2023-09-09T23:13:47.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.434+0000 7f10469ceb80 20 client.15609 trim_cache size 0 max 16384 2023-09-09T23:13:47.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.447+0000 7f10469ceb80 1 Processor -- start 2023-09-09T23:13:47.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.447+0000 7f10469ceb80 1 -- start start 2023-09-09T23:13:47.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.447+0000 7f10469ceb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faa4db2e00 0x55faa4dba1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.447+0000 7f10469ceb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55faa4db3c00 0x55faa4dc3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.447+0000 7f10469ceb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55faa4db45a0 0x55faa4dba720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.447+0000 7f10469ceb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa4d778e0 con 0x55faa4db2e00 2023-09-09T23:13:47.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.447+0000 7f10469ceb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa4dbd750 con 0x55faa4db3c00 2023-09-09T23:13:47.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.447+0000 7f10469ceb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa4dae950 con 0x55faa4db45a0 2023-09-09T23:13:47.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.448+0000 7f1033fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faa4db2e00 0x55faa4dba1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.448+0000 7f10337fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55faa4db3c00 0x55faa4dc3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.448+0000 7f10337fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55faa4db3c00 0x55faa4dc3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:41934/0 (socket says 172.21.15.82:41934) 2023-09-09T23:13:47.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.448+0000 7f10337fe700 1 -- 172.21.15.82:0/4092141820 learned_addr learned my addr 172.21.15.82:0/4092141820 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:47.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.448+0000 7f1038da5700 1 --2- 172.21.15.82:0/4092141820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55faa4db45a0 0x55faa4dba720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.448+0000 7f10337fe700 1 -- 172.21.15.82:0/4092141820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55faa4db45a0 msgr2=0x55faa4dba720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:47.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.448+0000 7f10337fe700 1 --2- 172.21.15.82:0/4092141820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55faa4db45a0 0x55faa4dba720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.448+0000 7f10337fe700 1 -- 172.21.15.82:0/4092141820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faa4db2e00 msgr2=0x55faa4dba1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:47.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.448+0000 7f10337fe700 1 --2- 172.21.15.82:0/4092141820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faa4db2e00 0x55faa4dba1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.448+0000 7f10337fe700 1 -- 172.21.15.82:0/4092141820 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103419e910 con 0x55faa4db3c00 2023-09-09T23:13:47.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.448+0000 7f1033fff700 1 --2- 172.21.15.82:0/4092141820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faa4db2e00 0x55faa4dba1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:47.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.448+0000 7f10337fe700 1 --2- 172.21.15.82:0/4092141820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55faa4db3c00 0x55faa4dc3b40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f10280098e0 tx=0x7f1028005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e4c1a42b3724e49 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:47.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.448+0000 7f101affd700 1 -- 172.21.15.82:0/4092141820 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102801d070 con 0x55faa4db3c00 2023-09-09T23:13:47.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.448+0000 7f101affd700 1 -- 172.21.15.82:0/4092141820 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f102801e040 con 0x55faa4db3c00 2023-09-09T23:13:47.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.448+0000 7f101affd700 1 -- 172.21.15.82:0/4092141820 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1028004960 con 0x55faa4db3c00 2023-09-09T23:13:47.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.449+0000 7f10469ceb80 1 -- 172.21.15.82:0/4092141820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55faa4db3c00 msgr2=0x55faa4dc3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.449+0000 7f10469ceb80 1 --2- 172.21.15.82:0/4092141820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55faa4db3c00 0x55faa4dc3b40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f10280098e0 tx=0x7f1028005fb0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.449+0000 7f10469ceb80 1 -- 172.21.15.82:0/4092141820 shutdown_connections 2023-09-09T23:13:47.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.449+0000 7f10469ceb80 1 --2- 172.21.15.82:0/4092141820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55faa4db45a0 0x55faa4dba720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.449+0000 7f10469ceb80 1 --2- 172.21.15.82:0/4092141820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faa4db2e00 0x55faa4dba1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.449+0000 7f10469ceb80 1 --2- 172.21.15.82:0/4092141820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55faa4db3c00 0x55faa4dc3b40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.449+0000 7f10469ceb80 1 -- 172.21.15.82:0/4092141820 >> 172.21.15.82:0/4092141820 conn(0x55faa4d27090 msgr2=0x55faa4dc4470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:47.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.449+0000 7f10469ceb80 1 -- 172.21.15.82:0/4092141820 shutdown_connections 2023-09-09T23:13:47.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.449+0000 7f10469ceb80 1 -- 172.21.15.82:0/4092141820 wait complete. 2023-09-09T23:13:47.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.449+0000 7f10469ceb80 1 -- start start 2023-09-09T23:13:47.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.450+0000 7f10469ceb80 1 Processor -- start 2023-09-09T23:13:47.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.450+0000 7f10469ceb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55faa4db2e00 0x55faa4dc3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.450+0000 7f10469ceb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55faa4db3c00 0x55faa4dc1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.450+0000 7f10469ceb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faa4db45a0 0x55faa4dc2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.466 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.450+0000 7f10469ceb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa4dc3e40 con 0x55faa4db45a0 2023-09-09T23:13:47.466 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.450+0000 7f10469ceb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa4dc26b0 con 0x55faa4db2e00 2023-09-09T23:13:47.466 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.450+0000 7f10469ceb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faa4dc2870 con 0x55faa4db3c00 2023-09-09T23:13:47.466 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.450+0000 7f10469ceb80 10 client.0 _ll_register_callbacks cb 0x7f102c0097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:47.466 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.450+0000 7f1038da5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faa4db45a0 0x55faa4dc2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.466 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.450+0000 7f10337fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55faa4db3c00 0x55faa4dc1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.467 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.450+0000 7f1033fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55faa4db2e00 0x55faa4dc3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.467 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.450+0000 7f1038da5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faa4db45a0 0x55faa4dc2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56380/0 (socket says 172.21.15.82:56380) 2023-09-09T23:13:47.467 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.450+0000 7f1038da5700 1 -- 172.21.15.82:0/4170208235 learned_addr learned my addr 172.21.15.82:0/4170208235 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:47.467 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.450+0000 7f10337fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55faa4db3c00 0x55faa4dc1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:54892/0 (socket says 172.21.15.82:54892) 2023-09-09T23:13:47.467 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.450+0000 7f10337fe700 1 -- 172.21.15.82:0/4170208235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55faa4db2e00 msgr2=0x55faa4dc3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:47.468 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.450+0000 7f10337fe700 1 --2- 172.21.15.82:0/4170208235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55faa4db2e00 0x55faa4dc3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.468 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.450+0000 7f10337fe700 1 -- 172.21.15.82:0/4170208235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faa4db45a0 msgr2=0x55faa4dc2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:47.468 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.450+0000 7f10337fe700 1 --2- 172.21.15.82:0/4170208235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faa4db45a0 0x55faa4dc2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.468 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.450+0000 7f10337fe700 1 -- 172.21.15.82:0/4170208235 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10341093a0 con 0x55faa4db3c00 2023-09-09T23:13:47.468 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.450+0000 7f1038da5700 1 --2- 172.21.15.82:0/4170208235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faa4db45a0 0x55faa4dc2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:47.469 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.451+0000 7f10337fe700 1 --2- 172.21.15.82:0/4170208235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55faa4db3c00 0x55faa4dc1c30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1028009990 tx=0x7f1028005a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbec986ee468b1a2 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:47.469 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.451+0000 7f10137fe700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-09T23:13:47.469 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.451+0000 7f10137fe700 1 -- 172.21.15.82:0/4170208235 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102801d050 con 0x55faa4db3c00 2023-09-09T23:13:47.469 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.451+0000 7f10137fe700 1 -- 172.21.15.82:0/4170208235 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f1028029360 con 0x55faa4db3c00 2023-09-09T23:13:47.469 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.451+0000 7f10469ceb80 1 -- 172.21.15.82:0/4170208235 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f102409d3c0 con 0x55faa4db3c00 2023-09-09T23:13:47.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.451+0000 7f10137fe700 1 -- 172.21.15.82:0/4170208235 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1028029520 con 0x55faa4db3c00 2023-09-09T23:13:47.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.451+0000 7f10469ceb80 10 client.25042 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:47.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.451+0000 7f10469ceb80 1 -- 172.21.15.82:0/4170208235 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f10280053e0 con 0x55faa4db3c00 2023-09-09T23:13:47.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.451+0000 7f10137fe700 1 -- 172.21.15.82:0/4170208235 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1028043020 con 0x55faa4db3c00 2023-09-09T23:13:47.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.451+0000 7f10137fe700 1 -- 172.21.15.82:0/4170208235 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f102800f600 con 0x55faa4db3c00 2023-09-09T23:13:47.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.451+0000 7f10469ceb80 10 client.25042 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:47.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.451+0000 7f10469ceb80 10 client.25042 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2023-09-09T23:13:47.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.451+0000 7f10469ceb80 20 client.25042 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:47.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.451+0000 7f10469ceb80 1 --2- 172.21.15.82:0/4170208235 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x55faa4e1cf10 0x55faa4e1f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.451+0000 7f10469ceb80 4 client.25042 mds_command: new command op to 24460 tid=0[{"prefix": "damage ls"}] 2023-09-09T23:13:47.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.451+0000 7f10469ceb80 1 -- 172.21.15.82:0/4170208235 --> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55faa4cfd920 con 0x55faa4e1cf10 2023-09-09T23:13:47.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.452+0000 7f1033fff700 1 --2- 172.21.15.82:0/4170208235 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x55faa4e1cf10 0x55faa4e1f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.452+0000 7f10137fe700 10 client.25042 ms_handle_connect on v2:172.21.15.170:6832/1931961940 2023-09-09T23:13:47.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.452+0000 7f1033fff700 1 --2- 172.21.15.82:0/4170208235 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x55faa4e1cf10 0x55faa4e1f330 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1f4cd4d6dd1fece8 server_cookie=a61a68b207381106 in_seq=0 out_seq=0 2023-09-09T23:13:47.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.452+0000 7f10137fe700 1 -- 172.21.15.82:0/4170208235 <== mds.4 v2:172.21.15.170:6832/1931961940 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55faa4cfd920 con 0x55faa4e1cf10 2023-09-09T23:13:47.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.452+0000 7f10137fe700 10 client.25042 handle_command_reply: tid=0 2023-09-09T23:13:47.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.452+0000 7f10469ceb80 1 client.25042 shutdown 2023-09-09T23:13:47.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.452+0000 7f10469ceb80 1 -- 172.21.15.82:0/4170208235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55faa4db3c00 msgr2=0x55faa4dc1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.452+0000 7f10469ceb80 1 --2- 172.21.15.82:0/4170208235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55faa4db3c00 0x55faa4dc1c30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1028009990 tx=0x7f1028005a10 comp rx=0 tx=0).stop 2023-09-09T23:13:47.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.452+0000 7f10469ceb80 1 -- 172.21.15.82:0/4170208235 shutdown_connections 2023-09-09T23:13:47.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.452+0000 7f10469ceb80 1 --2- 172.21.15.82:0/4170208235 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x55faa4e1cf10 0x55faa4e1f330 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.453+0000 7f10469ceb80 1 --2- 172.21.15.82:0/4170208235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55faa4db2e00 0x55faa4dc3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.453+0000 7f10469ceb80 1 --2- 172.21.15.82:0/4170208235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faa4db45a0 0x55faa4dc2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.453+0000 7f10469ceb80 1 --2- 172.21.15.82:0/4170208235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55faa4db3c00 0x55faa4dc1c30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.453+0000 7f10137fe700 0 client.25042 ms_handle_reset on v2:172.21.15.170:6832/1931961940 2023-09-09T23:13:47.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.453+0000 7f10469ceb80 1 -- 172.21.15.82:0/4170208235 >> 172.21.15.82:0/4170208235 conn(0x55faa4d27090 msgr2=0x55faa4dba1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:47.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.453+0000 7f10469ceb80 1 -- 172.21.15.82:0/4170208235 shutdown_connections 2023-09-09T23:13:47.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.453+0000 7f10469ceb80 1 -- 172.21.15.82:0/4170208235 wait complete. 2023-09-09T23:13:47.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.453+0000 7f10469ceb80 20 client.25042 trim_cache size 0 max 16384 2023-09-09T23:13:47.475 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2023-09-09T23:13:47.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.453+0000 7f103a808700 1 -- 172.21.15.82:0/1693469271 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f101c0a0690 msgr2=0x7f101c0a2b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-09T23:13:47.477 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.453+0000 7f103a808700 1 --2- 172.21.15.82:0/1693469271 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f101c0a0690 0x7f101c0a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.477 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.453+0000 7f103a808700 1 -- 172.21.15.82:0/1693469271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1034101140 msgr2=0x7f10341962d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.477 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.453+0000 7f103a808700 1 --2- 172.21.15.82:0/1693469271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1034101140 0x7f10341962d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f102400e9a0 tx=0x7f102400ecb0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.477 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.453+0000 7f103a808700 1 -- 172.21.15.82:0/1693469271 shutdown_connections 2023-09-09T23:13:47.477 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.453+0000 7f103a808700 1 --2- 172.21.15.82:0/1693469271 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f101c0a0690 0x7f101c0a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.453+0000 7f103a808700 1 --2- 172.21.15.82:0/1693469271 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10341063a0 0x7f103419d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.453+0000 7f103a808700 1 --2- 172.21.15.82:0/1693469271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1034101140 0x7f10341962d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.453+0000 7f103a808700 1 --2- 172.21.15.82:0/1693469271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1034103a70 0x7f1034196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.453+0000 7f103a808700 1 -- 172.21.15.82:0/1693469271 >> 172.21.15.82:0/1693469271 conn(0x7f10340faad0 msgr2=0x7f1034101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:47.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.453+0000 7f103a808700 1 -- 172.21.15.82:0/1693469271 shutdown_connections 2023-09-09T23:13:47.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.453+0000 7f103a808700 1 -- 172.21.15.82:0/1693469271 wait complete. 2023-09-09T23:13:47.479 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-09-09T23:13:47.504 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:47 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/4055047989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:47.505 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:47 smithi082 ceph-mon[98694]: scrub queued for path: / 2023-09-09T23:13:47.505 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:47 smithi082 ceph-mon[98694]: scrub summary: idle+waiting paths [/] 2023-09-09T23:13:47.505 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:47 smithi082 ceph-mon[98694]: scrub summary: active paths [/] 2023-09-09T23:13:47.505 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:47 smithi082 ceph-mon[98694]: scrub summary: idle+waiting paths [/] 2023-09-09T23:13:47.505 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:47 smithi082 ceph-mon[98694]: scrub summary: idle 2023-09-09T23:13:47.506 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:47 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2666960734' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-09-09T23:13:47.506 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:47 smithi082 ceph-mon[98694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-09-09T23:13:47.506 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:47 smithi082 ceph-mon[98694]: pgmap v302: 97 pgs: 97 active+clean; 645 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2023-09-09T23:13:47.506 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:47 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/2043204235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:13:47.507 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:47 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: ignoring --setuser ceph since I am not root 2023-09-09T23:13:47.507 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:47 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: ignoring --setgroup ceph since I am not root 2023-09-09T23:13:47.509 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.509+0000 7ff7a8a2d700 1 -- 172.21.15.82:0/3492218713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a4106380 msgr2=0x7ff7a4108790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.509 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.509+0000 7ff7a8a2d700 1 --2- 172.21.15.82:0/3492218713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a4106380 0x7ff7a4108790 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00b0a0 tx=0x7ff79c00b3b0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.509 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.509+0000 7ff7a8a2d700 1 -- 172.21.15.82:0/3492218713 shutdown_connections 2023-09-09T23:13:47.510 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.509+0000 7ff7a8a2d700 1 --2- 172.21.15.82:0/3492218713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7a4101120 0x7ff7a4103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.510 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.509+0000 7ff7a8a2d700 1 --2- 172.21.15.82:0/3492218713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a4106380 0x7ff7a4108790 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.510 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.509+0000 7ff7a8a2d700 1 --2- 172.21.15.82:0/3492218713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7a4103a50 0x7ff7a4105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.510 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.509+0000 7ff7a8a2d700 1 -- 172.21.15.82:0/3492218713 >> 172.21.15.82:0/3492218713 conn(0x7ff7a40faab0 msgr2=0x7ff7a40fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:47.510 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.509+0000 7ff7a8a2d700 1 -- 172.21.15.82:0/3492218713 shutdown_connections 2023-09-09T23:13:47.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.509+0000 7ff7a8a2d700 1 -- 172.21.15.82:0/3492218713 wait complete. 2023-09-09T23:13:47.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.510+0000 7ff7a8a2d700 1 Processor -- start 2023-09-09T23:13:47.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.510+0000 7ff7a8a2d700 1 -- start start 2023-09-09T23:13:47.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.510+0000 7ff7a8a2d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a4101120 0x7ff7a41985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.510+0000 7ff7a8a2d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7a4103a50 0x7ff7a4198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.510+0000 7ff7a8a2d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7a4106380 0x7ff7a4199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.510+0000 7ff7a8a2d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a4199ff0 con 0x7ff7a4101120 2023-09-09T23:13:47.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.511+0000 7ff7a8a2d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a419a160 con 0x7ff7a4106380 2023-09-09T23:13:47.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.511+0000 7ff7a8a2d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a419a320 con 0x7ff7a4103a50 2023-09-09T23:13:47.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.511+0000 7ff7a259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a4101120 0x7ff7a41985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.511+0000 7ff7a259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a4101120 0x7ff7a41985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56390/0 (socket says 172.21.15.82:56390) 2023-09-09T23:13:47.513 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.511+0000 7ff7a259c700 1 -- 172.21.15.82:0/501254384 learned_addr learned my addr 172.21.15.82:0/501254384 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:47.513 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.511+0000 7ff7a1d9b700 1 --2- 172.21.15.82:0/501254384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7a4103a50 0x7ff7a4198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.513 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.511+0000 7ff7a2d9d700 1 --2- 172.21.15.82:0/501254384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7a4106380 0x7ff7a4199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.513 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.511+0000 7ff7a259c700 1 -- 172.21.15.82:0/501254384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7a4103a50 msgr2=0x7ff7a4198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.513 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.511+0000 7ff7a259c700 1 --2- 172.21.15.82:0/501254384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7a4103a50 0x7ff7a4198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.514 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.511+0000 7ff7a259c700 1 -- 172.21.15.82:0/501254384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7a4106380 msgr2=0x7ff7a4199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.514 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.511+0000 7ff7a259c700 1 --2- 172.21.15.82:0/501254384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7a4106380 0x7ff7a4199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.514 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.511+0000 7ff7a259c700 1 -- 172.21.15.82:0/501254384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79c009d00 con 0x7ff7a4101120 2023-09-09T23:13:47.514 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.511+0000 7ff7a259c700 1 --2- 172.21.15.82:0/501254384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a4101120 0x7ff7a41985d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff79400e9a0 tx=0x7ff79400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:47.514 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.511+0000 7ff7937fe700 1 -- 172.21.15.82:0/501254384 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff794009790 con 0x7ff7a4101120 2023-09-09T23:13:47.514 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.511+0000 7ff7a8a2d700 1 -- 172.21.15.82:0/501254384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a419a600 con 0x7ff7a4101120 2023-09-09T23:13:47.515 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.511+0000 7ff7937fe700 1 -- 172.21.15.82:0/501254384 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff794010430 con 0x7ff7a4101120 2023-09-09T23:13:47.515 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.511+0000 7ff7937fe700 1 -- 172.21.15.82:0/501254384 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff794018680 con 0x7ff7a4101120 2023-09-09T23:13:47.515 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.512+0000 7ff7a8a2d700 1 -- 172.21.15.82:0/501254384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a41a0e40 con 0x7ff7a4101120 2023-09-09T23:13:47.516 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.513+0000 7ff7a8a2d700 1 -- 172.21.15.82:0/501254384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a40fc630 con 0x7ff7a4101120 2023-09-09T23:13:47.516 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.515+0000 7ff7937fe700 1 -- 172.21.15.82:0/501254384 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 19) v1 ==== 135276+0+0 (secure 0 0 0) 0x7ff7940105a0 con 0x7ff7a4101120 2023-09-09T23:13:47.516 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.516+0000 7ff7937fe700 1 --2- 172.21.15.82:0/501254384 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff78c0a0690 0x7ff78c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.517 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.516+0000 7ff7a1d9b700 1 -- 172.21.15.82:0/501254384 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff78c0a0690 msgr2=0x7ff78c0a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1453069857 2023-09-09T23:13:47.517 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.516+0000 7ff7937fe700 1 -- 172.21.15.82:0/501254384 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff794014070 con 0x7ff7a4101120 2023-09-09T23:13:47.517 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.516+0000 7ff7a1d9b700 1 --2- 172.21.15.82:0/501254384 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff78c0a0690 0x7ff78c0a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-09T23:13:47.517 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.518+0000 7ff7937fe700 1 -- 172.21.15.82:0/501254384 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff794095ab0 con 0x7ff7a4101120 2023-09-09T23:13:47.602 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:13:47 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: ignoring --setuser ceph since I am not root 2023-09-09T23:13:47.602 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:13:47 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: ignoring --setgroup ceph since I am not root 2023-09-09T23:13:47.603 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:13:47 smithi170 ceph-mgr[104931]: -- 172.21.15.170:0/1168452833 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558702cb51e0 con 0x558702c8b000 2023-09-09T23:13:47.603 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:13:47 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:13:47.524+0000 7fc6bfd29200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-09T23:13:47.604 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:47 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/4055047989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:47.604 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:47 smithi170 ceph-mon[102832]: scrub queued for path: / 2023-09-09T23:13:47.604 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:47 smithi170 ceph-mon[102832]: scrub summary: idle+waiting paths [/] 2023-09-09T23:13:47.605 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:47 smithi170 ceph-mon[102832]: scrub summary: active paths [/] 2023-09-09T23:13:47.605 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:47 smithi170 ceph-mon[102832]: scrub summary: idle+waiting paths [/] 2023-09-09T23:13:47.605 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:47 smithi170 ceph-mon[102832]: scrub summary: idle 2023-09-09T23:13:47.606 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:47 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2666960734' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-09-09T23:13:47.606 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:47 smithi170 ceph-mon[102832]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-09-09T23:13:47.606 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:47 smithi170 ceph-mon[102832]: pgmap v302: 97 pgs: 97 active+clean; 645 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2023-09-09T23:13:47.607 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:47 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/2043204235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:13:47.607 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:13:47 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: ignoring --setuser ceph since I am not root 2023-09-09T23:13:47.607 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:13:47 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: ignoring --setgroup ceph since I am not root 2023-09-09T23:13:47.608 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:13:47 smithi098 ceph-mgr[104628]: -- 172.21.15.98:0/374671014 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5579499c91e0 con 0x55794999f000 2023-09-09T23:13:47.608 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:13:47 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:13:47.525+0000 7f0b0093b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-09T23:13:47.608 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:47 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/4055047989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-09T23:13:47.608 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:47 smithi098 ceph-mon[103124]: scrub queued for path: / 2023-09-09T23:13:47.609 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:47 smithi098 ceph-mon[103124]: scrub summary: idle+waiting paths [/] 2023-09-09T23:13:47.609 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:47 smithi098 ceph-mon[103124]: scrub summary: active paths [/] 2023-09-09T23:13:47.609 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:47 smithi098 ceph-mon[103124]: scrub summary: idle+waiting paths [/] 2023-09-09T23:13:47.610 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:47 smithi098 ceph-mon[103124]: scrub summary: idle 2023-09-09T23:13:47.610 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:47 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2666960734' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-09-09T23:13:47.610 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:47 smithi098 ceph-mon[103124]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-09-09T23:13:47.610 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:47 smithi098 ceph-mon[103124]: pgmap v302: 97 pgs: 97 active+clean; 645 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2023-09-09T23:13:47.610 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:47 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/2043204235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:13:47.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.613+0000 7f12027bf700 1 -- 172.21.15.82:0/4112349638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11fc101030 msgr2=0x7f11fc1078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.613+0000 7f12027bf700 1 --2- 172.21.15.82:0/4112349638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11fc101030 0x7f11fc1078e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f11f400d360 tx=0x7f11f400d670 comp rx=0 tx=0).stop 2023-09-09T23:13:47.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.613+0000 7f12027bf700 1 -- 172.21.15.82:0/4112349638 shutdown_connections 2023-09-09T23:13:47.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.613+0000 7f12027bf700 1 --2- 172.21.15.82:0/4112349638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11fc0ff4a0 0x7f11fc0ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.613+0000 7f12027bf700 1 --2- 172.21.15.82:0/4112349638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11fc101030 0x7f11fc1078e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.613+0000 7f12027bf700 1 --2- 172.21.15.82:0/4112349638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11fc100690 0x7f11fc100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.613+0000 7f12027bf700 1 -- 172.21.15.82:0/4112349638 >> 172.21.15.82:0/4112349638 conn(0x7f11fc0faa10 msgr2=0x7f11fc0fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:47.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.613+0000 7f12027bf700 1 -- 172.21.15.82:0/4112349638 shutdown_connections 2023-09-09T23:13:47.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.613+0000 7f12027bf700 1 -- 172.21.15.82:0/4112349638 wait complete. 2023-09-09T23:13:47.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.613+0000 7f12027bf700 1 Processor -- start 2023-09-09T23:13:47.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.613+0000 7f12027bf700 1 -- start start 2023-09-09T23:13:47.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.614+0000 7f12027bf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11fc0ff4a0 0x7f11fc194150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.614+0000 7f12027bf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11fc100690 0x7f11fc194690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.614+0000 7f12027bf700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11fc101030 0x7f11fc19b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.614+0000 7f12027bf700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc19bcd0 con 0x7f11fc0ff4a0 2023-09-09T23:13:47.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.614+0000 7f12027bf700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc19be10 con 0x7f11fc101030 2023-09-09T23:13:47.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.614+0000 7f12027bf700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc19bfa0 con 0x7f11fc100690 2023-09-09T23:13:47.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.614+0000 7f11fbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11fc0ff4a0 0x7f11fc194150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.614+0000 7f11fbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11fc0ff4a0 0x7f11fc194150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56454/0 (socket says 172.21.15.82:56454) 2023-09-09T23:13:47.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.614+0000 7f11fbfff700 1 -- 172.21.15.82:0/4102511977 learned_addr learned my addr 172.21.15.82:0/4102511977 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:47.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.614+0000 7f11fb7fe700 1 --2- 172.21.15.82:0/4102511977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11fc100690 0x7f11fc194690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.614+0000 7f1200d5c700 1 --2- 172.21.15.82:0/4102511977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11fc101030 0x7f11fc19b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.614+0000 7f11fbfff700 1 -- 172.21.15.82:0/4102511977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11fc100690 msgr2=0x7f11fc194690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.614+0000 7f11fbfff700 1 --2- 172.21.15.82:0/4102511977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11fc100690 0x7f11fc194690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.614+0000 7f11fbfff700 1 -- 172.21.15.82:0/4102511977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11fc101030 msgr2=0x7f11fc19b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.618 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.614+0000 7f11fbfff700 1 --2- 172.21.15.82:0/4102511977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11fc101030 0x7f11fc19b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.618 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.614+0000 7f11fbfff700 1 -- 172.21.15.82:0/4102511977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f4009d00 con 0x7f11fc0ff4a0 2023-09-09T23:13:47.618 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.614+0000 7f11fb7fe700 1 --2- 172.21.15.82:0/4102511977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11fc100690 0x7f11fc194690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:47.618 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.614+0000 7f1200d5c700 1 --2- 172.21.15.82:0/4102511977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11fc101030 0x7f11fc19b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:47.618 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.614+0000 7f11fbfff700 1 --2- 172.21.15.82:0/4102511977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11fc0ff4a0 0x7f11fc194150 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00e970 tx=0x7f11ec00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:47.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.615+0000 7f11f97fa700 1 -- 172.21.15.82:0/4102511977 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec009790 con 0x7f11fc0ff4a0 2023-09-09T23:13:47.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.615+0000 7f11f97fa700 1 -- 172.21.15.82:0/4102511977 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11ec00c7e0 con 0x7f11fc0ff4a0 2023-09-09T23:13:47.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.615+0000 7f12027bf700 1 -- 172.21.15.82:0/4102511977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11fc19c280 con 0x7f11fc0ff4a0 2023-09-09T23:13:47.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.615+0000 7f11f97fa700 1 -- 172.21.15.82:0/4102511977 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec003c70 con 0x7f11fc0ff4a0 2023-09-09T23:13:47.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.615+0000 7f12027bf700 1 -- 172.21.15.82:0/4102511977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11fc19c7e0 con 0x7f11fc0ff4a0 2023-09-09T23:13:47.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.617+0000 7f11f97fa700 1 -- 172.21.15.82:0/4102511977 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 19) v1 ==== 135276+0+0 (secure 0 0 0) 0x7f11ec00cc50 con 0x7f11fc0ff4a0 2023-09-09T23:13:47.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.618+0000 7f11f97fa700 1 --2- 172.21.15.82:0/4102511977 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f11e40a0710 0x7f11e40a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.618+0000 7f11fb7fe700 1 -- 172.21.15.82:0/4102511977 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f11e40a0710 msgr2=0x7f11e40a2bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1453069857 2023-09-09T23:13:47.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.618+0000 7f11f97fa700 1 -- 172.21.15.82:0/4102511977 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f11ec014070 con 0x7f11fc0ff4a0 2023-09-09T23:13:47.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.618+0000 7f11fb7fe700 1 --2- 172.21.15.82:0/4102511977 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f11e40a0710 0x7f11e40a2bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-09T23:13:47.627 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.620+0000 7f120e985b80 1 Processor -- start 2023-09-09T23:13:47.627 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.620+0000 7f120e985b80 1 -- start start 2023-09-09T23:13:47.627 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.620+0000 7f120e985b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55764bf41070 0x55764bf41490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.627 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.620+0000 7f120e985b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55764bf419d0 0x55764bf3fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.627 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.620+0000 7f120e985b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55764bf40230 0x55764bf406b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.620+0000 7f120e985b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55764be94730 con 0x55764bf40230 2023-09-09T23:13:47.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.620+0000 7f120e985b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55764be92050 con 0x55764bf419d0 2023-09-09T23:13:47.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.620+0000 7f120e985b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55764be8d940 con 0x55764bf41070 2023-09-09T23:13:47.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.620+0000 7f11fbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55764bf41070 0x55764bf41490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.620+0000 7f11fbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55764bf41070 0x55764bf41490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:54976/0 (socket says 172.21.15.82:54976) 2023-09-09T23:13:47.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.620+0000 7f11fbfff700 1 -- 172.21.15.82:0/3459638137 learned_addr learned my addr 172.21.15.82:0/3459638137 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:47.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.620+0000 7f11fb7fe700 1 --2- 172.21.15.82:0/3459638137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55764bf419d0 0x55764bf3fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.620+0000 7f1200d5c700 1 --2- 172.21.15.82:0/3459638137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55764bf40230 0x55764bf406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.620+0000 7f11fbfff700 1 -- 172.21.15.82:0/3459638137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55764bf419d0 msgr2=0x55764bf3fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:47.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.620+0000 7f11fbfff700 1 --2- 172.21.15.82:0/3459638137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55764bf419d0 0x55764bf3fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.620+0000 7f11fbfff700 1 -- 172.21.15.82:0/3459638137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55764bf40230 msgr2=0x55764bf406b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:47.630 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.620+0000 7f11fbfff700 1 --2- 172.21.15.82:0/3459638137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55764bf40230 0x55764bf406b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.630 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.620+0000 7f11fbfff700 1 -- 172.21.15.82:0/3459638137 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55764bcb9de0 con 0x55764bf41070 2023-09-09T23:13:47.630 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.620+0000 7f1200d5c700 1 --2- 172.21.15.82:0/3459638137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55764bf40230 0x55764bf406b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:47.630 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.621+0000 7f11fbfff700 1 --2- 172.21.15.82:0/3459638137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55764bf41070 0x55764bf41490 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f11ec0959b0 tx=0x7f11ec000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73b262c93a1f69ab server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:47.630 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.621+0000 7f11e27fc700 1 -- 172.21.15.82:0/3459638137 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec09d2e0 con 0x55764bf41070 2023-09-09T23:13:47.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.621+0000 7f11e27fc700 1 -- 172.21.15.82:0/3459638137 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11ec0a1520 con 0x55764bf41070 2023-09-09T23:13:47.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.621+0000 7f11e27fc700 1 -- 172.21.15.82:0/3459638137 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec0c9e40 con 0x55764bf41070 2023-09-09T23:13:47.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.622+0000 7f120e985b80 1 -- 172.21.15.82:0/3459638137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55764bf41070 msgr2=0x55764bf41490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.622+0000 7f120e985b80 1 --2- 172.21.15.82:0/3459638137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55764bf41070 0x55764bf41490 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f11ec0959b0 tx=0x7f11ec000bc0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.622+0000 7f120e985b80 1 -- 172.21.15.82:0/3459638137 shutdown_connections 2023-09-09T23:13:47.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.622+0000 7f120e985b80 1 --2- 172.21.15.82:0/3459638137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55764bf40230 0x55764bf406b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.622+0000 7f120e985b80 1 --2- 172.21.15.82:0/3459638137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55764bf41070 0x55764bf41490 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.622+0000 7f120e985b80 1 --2- 172.21.15.82:0/3459638137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55764bf419d0 0x55764bf3fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.622+0000 7f120e985b80 1 -- 172.21.15.82:0/3459638137 >> 172.21.15.82:0/3459638137 conn(0x55764beb0090 msgr2=0x55764bf36720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:47.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.622+0000 7f120e985b80 1 -- 172.21.15.82:0/3459638137 shutdown_connections 2023-09-09T23:13:47.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.622+0000 7f120e985b80 1 -- 172.21.15.82:0/3459638137 wait complete. 2023-09-09T23:13:47.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.622+0000 7f120e985b80 1 -- start start 2023-09-09T23:13:47.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.623+0000 7f120e985b80 1 Processor -- start 2023-09-09T23:13:47.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.623+0000 7f120e985b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55764bf40230 0x55764bf49a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.623+0000 7f120e985b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55764bf41070 0x55764bf49f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.623+0000 7f120e985b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55764bf419d0 0x55764bf50f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.623+0000 7f120e985b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55764be2bb70 con 0x55764bf41070 2023-09-09T23:13:47.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.623+0000 7f120e985b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55764be2b870 con 0x55764bf40230 2023-09-09T23:13:47.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.623+0000 7f120e985b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55764bd0f920 con 0x55764bf419d0 2023-09-09T23:13:47.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.623+0000 7f120e985b80 10 client.0 _ll_register_callbacks cb 0x55764bd75a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:47.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.623+0000 7f11fb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55764bf41070 0x55764bf49f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.623+0000 7f11fbfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55764bf40230 0x55764bf49a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.623+0000 7f11fbfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55764bf40230 0x55764bf49a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:42032/0 (socket says 172.21.15.82:42032) 2023-09-09T23:13:47.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.623+0000 7f11fb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55764bf41070 0x55764bf49f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56470/0 (socket says 172.21.15.82:56470) 2023-09-09T23:13:47.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.623+0000 7f11fbfff700 1 -- 172.21.15.82:0/207069728 learned_addr learned my addr 172.21.15.82:0/207069728 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:47.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.623+0000 7f1200d5c700 1 --2- 172.21.15.82:0/207069728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55764bf419d0 0x55764bf50f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.623+0000 7f11fbfff700 1 -- 172.21.15.82:0/207069728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55764bf419d0 msgr2=0x55764bf50f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:47.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.623+0000 7f11fbfff700 1 --2- 172.21.15.82:0/207069728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55764bf419d0 0x55764bf50f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.623+0000 7f11fbfff700 1 -- 172.21.15.82:0/207069728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55764bf41070 msgr2=0x55764bf49f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:47.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.623+0000 7f11fbfff700 1 --2- 172.21.15.82:0/207069728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55764bf41070 0x55764bf49f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.623+0000 7f11fbfff700 1 -- 172.21.15.82:0/207069728 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11fc19c7e0 con 0x55764bf40230 2023-09-09T23:13:47.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.623+0000 7f1200d5c700 1 --2- 172.21.15.82:0/207069728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55764bf419d0 0x55764bf50f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:47.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.624+0000 7f11fbfff700 1 --2- 172.21.15.82:0/207069728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55764bf40230 0x55764bf49a10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00efd0 tx=0x7f11ec0958a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77e12f62e25a672 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:47.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.624+0000 7f11db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-09T23:13:47.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.624+0000 7f11db7fe700 1 -- 172.21.15.82:0/207069728 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec0975f0 con 0x55764bf40230 2023-09-09T23:13:47.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.624+0000 7f120e985b80 1 -- 172.21.15.82:0/207069728 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55764bde54f0 con 0x55764bf40230 2023-09-09T23:13:47.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.624+0000 7f11db7fe700 1 -- 172.21.15.82:0/207069728 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f11ec09d2e0 con 0x55764bf40230 2023-09-09T23:13:47.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.624+0000 7f11db7fe700 1 -- 172.21.15.82:0/207069728 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11ec003dd0 con 0x55764bf40230 2023-09-09T23:13:47.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.624+0000 7f120e985b80 10 client.25154 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:47.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.624+0000 7f120e985b80 1 -- 172.21.15.82:0/207069728 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55764bca2c60 con 0x55764bf40230 2023-09-09T23:13:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.624+0000 7f11db7fe700 1 -- 172.21.15.82:0/207069728 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11ec0a5080 con 0x55764bf40230 2023-09-09T23:13:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.624+0000 7f11db7fe700 1 -- 172.21.15.82:0/207069728 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f11ec0a1090 con 0x55764bf40230 2023-09-09T23:13:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.624+0000 7f120e985b80 10 client.25154 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.624+0000 7f120e985b80 10 client.25154 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2023-09-09T23:13:47.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.624+0000 7f120e985b80 20 client.25154 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:47.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.624+0000 7f120e985b80 1 --2- 172.21.15.82:0/207069728 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55764bf55310 0x55764bf57730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.624+0000 7f120e985b80 4 client.25154 mds_command: new command op to 24491 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:13:47.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.624+0000 7f120e985b80 1 -- 172.21.15.82:0/207069728 --> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55764bea28a0 con 0x55764bf55310 2023-09-09T23:13:47.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.625+0000 7f11fb7fe700 1 --2- 172.21.15.82:0/207069728 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55764bf55310 0x55764bf57730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.625+0000 7f11fb7fe700 1 --2- 172.21.15.82:0/207069728 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55764bf55310 0x55764bf57730 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=672d68590f8766ce server_cookie=88bde6d0b403c193 in_seq=0 out_seq=0 2023-09-09T23:13:47.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.625+0000 7f11db7fe700 10 client.25154 ms_handle_connect on v2:172.21.15.98:6832/3855146755 2023-09-09T23:13:47.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.625+0000 7f11db7fe700 1 -- 172.21.15.82:0/207069728 <== mds.3 v2:172.21.15.98:6832/3855146755 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55764bea28a0 con 0x55764bf55310 2023-09-09T23:13:47.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.625+0000 7f11db7fe700 10 client.25154 handle_command_reply: tid=0 2023-09-09T23:13:47.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.625+0000 7f120e985b80 1 client.25154 shutdown 2023-09-09T23:13:47.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.626+0000 7f120e985b80 1 -- 172.21.15.82:0/207069728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55764bf40230 msgr2=0x55764bf49a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.626+0000 7f120e985b80 1 --2- 172.21.15.82:0/207069728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55764bf40230 0x55764bf49a10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00efd0 tx=0x7f11ec0958a0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.626+0000 7f120e985b80 1 -- 172.21.15.82:0/207069728 shutdown_connections 2023-09-09T23:13:47.642 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.626+0000 7f120e985b80 1 --2- 172.21.15.82:0/207069728 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55764bf55310 0x55764bf57730 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.642 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.626+0000 7f120e985b80 1 --2- 172.21.15.82:0/207069728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55764bf40230 0x55764bf49a10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.642 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.626+0000 7f120e985b80 1 --2- 172.21.15.82:0/207069728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55764bf419d0 0x55764bf50f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.642 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.626+0000 7f11db7fe700 0 client.25154 ms_handle_reset on v2:172.21.15.98:6832/3855146755 2023-09-09T23:13:47.642 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.626+0000 7f120e985b80 1 --2- 172.21.15.82:0/207069728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55764bf41070 0x55764bf49f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.643 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.626+0000 7f120e985b80 1 -- 172.21.15.82:0/207069728 >> 172.21.15.82:0/207069728 conn(0x55764beb0090 msgr2=0x55764bf35dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:47.643 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.626+0000 7f120e985b80 1 -- 172.21.15.82:0/207069728 shutdown_connections 2023-09-09T23:13:47.643 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.626+0000 7f120e985b80 1 -- 172.21.15.82:0/207069728 wait complete. 2023-09-09T23:13:47.643 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.626+0000 7f120e985b80 20 client.25154 trim_cache size 0 max 16384 2023-09-09T23:13:47.648 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.638+0000 7f120e985b80 1 Processor -- start 2023-09-09T23:13:47.648 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.639+0000 7f120e985b80 1 -- start start 2023-09-09T23:13:47.649 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.639+0000 7f120e985b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55764bf40230 0x55764bf47860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.649 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.639+0000 7f120e985b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55764bf41070 0x55764bf50f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.649 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.639+0000 7f120e985b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55764bf419d0 0x55764bf44390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.649 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.639+0000 7f120e985b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55764bf495b0 con 0x55764bf40230 2023-09-09T23:13:47.650 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.639+0000 7f120e985b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55764bf37670 con 0x55764bf419d0 2023-09-09T23:13:47.650 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.639+0000 7f120e985b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55764bf38010 con 0x55764bf41070 2023-09-09T23:13:47.650 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.639+0000 7f11fbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55764bf40230 0x55764bf47860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.650 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.639+0000 7f11fbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55764bf40230 0x55764bf47860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56476/0 (socket says 172.21.15.82:56476) 2023-09-09T23:13:47.650 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.639+0000 7f11fbfff700 1 -- 172.21.15.82:0/3177219200 learned_addr learned my addr 172.21.15.82:0/3177219200 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:47.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.639+0000 7f11fb7fe700 1 --2- 172.21.15.82:0/3177219200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55764bf41070 0x55764bf50f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.639+0000 7f1200d5c700 1 --2- 172.21.15.82:0/3177219200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55764bf419d0 0x55764bf44390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.639+0000 7f11fb7fe700 1 -- 172.21.15.82:0/3177219200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55764bf419d0 msgr2=0x55764bf44390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:47.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.639+0000 7f11fb7fe700 1 --2- 172.21.15.82:0/3177219200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55764bf419d0 0x55764bf44390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.639+0000 7f11fb7fe700 1 -- 172.21.15.82:0/3177219200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55764bf40230 msgr2=0x55764bf47860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:47.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.639+0000 7f11fb7fe700 1 --2- 172.21.15.82:0/3177219200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55764bf40230 0x55764bf47860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.639+0000 7f11fb7fe700 1 -- 172.21.15.82:0/3177219200 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55764bce0250 con 0x55764bf41070 2023-09-09T23:13:47.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.639+0000 7f11fbfff700 1 --2- 172.21.15.82:0/3177219200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55764bf40230 0x55764bf47860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:47.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.640+0000 7f11fb7fe700 1 --2- 172.21.15.82:0/3177219200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55764bf41070 0x55764bf50f50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x55764bf57fe0 tx=0x7f11f0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5113e7c91f5e34dd server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:47.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.640+0000 7f11e2ffd700 1 -- 172.21.15.82:0/3177219200 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f0016040 con 0x55764bf41070 2023-09-09T23:13:47.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.640+0000 7f11e2ffd700 1 -- 172.21.15.82:0/3177219200 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11f0021d30 con 0x55764bf41070 2023-09-09T23:13:47.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.640+0000 7f11e2ffd700 1 -- 172.21.15.82:0/3177219200 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f000fe70 con 0x55764bf41070 2023-09-09T23:13:47.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.640+0000 7f120e985b80 1 -- 172.21.15.82:0/3177219200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55764bf41070 msgr2=0x55764bf50f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.640+0000 7f120e985b80 1 --2- 172.21.15.82:0/3177219200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55764bf41070 0x55764bf50f50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x55764bf57fe0 tx=0x7f11f0009260 comp rx=0 tx=0).stop 2023-09-09T23:13:47.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.640+0000 7f120e985b80 1 -- 172.21.15.82:0/3177219200 shutdown_connections 2023-09-09T23:13:47.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.640+0000 7f120e985b80 1 --2- 172.21.15.82:0/3177219200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55764bf419d0 0x55764bf44390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.640+0000 7f120e985b80 1 --2- 172.21.15.82:0/3177219200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55764bf40230 0x55764bf47860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.640+0000 7f120e985b80 1 --2- 172.21.15.82:0/3177219200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55764bf41070 0x55764bf50f50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.640+0000 7f120e985b80 1 -- 172.21.15.82:0/3177219200 >> 172.21.15.82:0/3177219200 conn(0x55764beb0090 msgr2=0x55764bf40610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:47.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.640+0000 7f120e985b80 1 -- 172.21.15.82:0/3177219200 shutdown_connections 2023-09-09T23:13:47.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.640+0000 7f120e985b80 1 -- 172.21.15.82:0/3177219200 wait complete. 2023-09-09T23:13:47.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.641+0000 7f120e985b80 1 -- start start 2023-09-09T23:13:47.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.641+0000 7f120e985b80 1 Processor -- start 2023-09-09T23:13:47.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.641+0000 7f120e985b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55764bf40230 0x55764bf50b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.641+0000 7f120e985b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55764bf41070 0x55764bf51050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.641+0000 7f120e985b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55764bf419d0 0x55764bf4d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.641+0000 7f120e985b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55764bf4ddd0 con 0x55764bf419d0 2023-09-09T23:13:47.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.641+0000 7f120e985b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55764bf4df40 con 0x55764bf40230 2023-09-09T23:13:47.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.641+0000 7f120e985b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55764bf4e100 con 0x55764bf41070 2023-09-09T23:13:47.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.641+0000 7f120e985b80 10 client.0 _ll_register_callbacks cb 0x7f11f0009c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:47.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.641+0000 7f11fbfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55764bf40230 0x55764bf50b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.641+0000 7f1200d5c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55764bf419d0 0x55764bf4d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.641+0000 7f11fbfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55764bf40230 0x55764bf50b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:42046/0 (socket says 172.21.15.82:42046) 2023-09-09T23:13:47.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.641+0000 7f11fbfff700 1 -- 172.21.15.82:0/3793774221 learned_addr learned my addr 172.21.15.82:0/3793774221 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:47.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.641+0000 7f11fb7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55764bf41070 0x55764bf51050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.642+0000 7f11fbfff700 1 -- 172.21.15.82:0/3793774221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55764bf41070 msgr2=0x55764bf51050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:47.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.642+0000 7f11fbfff700 1 --2- 172.21.15.82:0/3793774221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55764bf41070 0x55764bf51050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.642+0000 7f11fbfff700 1 -- 172.21.15.82:0/3793774221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55764bf419d0 msgr2=0x55764bf4d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:47.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.642+0000 7f11fbfff700 1 --2- 172.21.15.82:0/3793774221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55764bf419d0 0x55764bf4d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.642+0000 7f11fbfff700 1 -- 172.21.15.82:0/3793774221 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55764bca2c60 con 0x55764bf40230 2023-09-09T23:13:47.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.642+0000 7f11fb7fe700 1 --2- 172.21.15.82:0/3793774221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55764bf41070 0x55764bf51050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:47.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.642+0000 7f1200d5c700 1 --2- 172.21.15.82:0/3793774221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55764bf419d0 0x55764bf4d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:47.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.642+0000 7f11fbfff700 1 --2- 172.21.15.82:0/3793774221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55764bf40230 0x55764bf50b10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f11ec0a1420 tx=0x7f11ec097950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef8ca8d6391217b6 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:47.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.642+0000 7f11db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-09T23:13:47.659 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.642+0000 7f11db7fe700 1 -- 172.21.15.82:0/3793774221 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec0c9df0 con 0x55764bf40230 2023-09-09T23:13:47.659 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.642+0000 7f11db7fe700 1 -- 172.21.15.82:0/3793774221 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f11ec099bb0 con 0x55764bf40230 2023-09-09T23:13:47.659 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.642+0000 7f120e985b80 1 -- 172.21.15.82:0/3793774221 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11ec0a1520 con 0x55764bf40230 2023-09-09T23:13:47.659 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.642+0000 7f11db7fe700 1 -- 172.21.15.82:0/3793774221 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11ec09d2e0 con 0x55764bf40230 2023-09-09T23:13:47.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.642+0000 7f120e985b80 10 client.25160 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:47.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.642+0000 7f120e985b80 1 -- 172.21.15.82:0/3793774221 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55764bcb82b0 con 0x55764bf40230 2023-09-09T23:13:47.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.643+0000 7f11db7fe700 1 -- 172.21.15.82:0/3793774221 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11ec0a1520 con 0x55764bf40230 2023-09-09T23:13:47.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.643+0000 7f11db7fe700 1 -- 172.21.15.82:0/3793774221 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f11ec0b72c0 con 0x55764bf40230 2023-09-09T23:13:47.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.643+0000 7f120e985b80 10 client.25160 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:47.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.643+0000 7f120e985b80 10 client.25160 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2023-09-09T23:13:47.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.643+0000 7f120e985b80 20 client.25160 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:47.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.643+0000 7f120e985b80 1 --2- 172.21.15.82:0/3793774221 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55764bf55310 0x55764bfac4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.643+0000 7f120e985b80 4 client.25160 mds_command: new command op to 24491 tid=0[{"prefix": "damage ls"}] 2023-09-09T23:13:47.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.643+0000 7f120e985b80 1 -- 172.21.15.82:0/3793774221 --> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55764be86920 con 0x55764bf55310 2023-09-09T23:13:47.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.643+0000 7f11fb7fe700 1 --2- 172.21.15.82:0/3793774221 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55764bf55310 0x55764bfac4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.643+0000 7f11db7fe700 10 client.25160 ms_handle_connect on v2:172.21.15.98:6832/3855146755 2023-09-09T23:13:47.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.643+0000 7f11fb7fe700 1 --2- 172.21.15.82:0/3793774221 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55764bf55310 0x55764bfac4d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=68fc24d9dfbfe8f3 server_cookie=28edbef97d738de7 in_seq=0 out_seq=0 2023-09-09T23:13:47.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.644+0000 7f11db7fe700 1 -- 172.21.15.82:0/3793774221 <== mds.3 v2:172.21.15.98:6832/3855146755 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55764be86920 con 0x55764bf55310 2023-09-09T23:13:47.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.644+0000 7f11db7fe700 10 client.25160 handle_command_reply: tid=0 2023-09-09T23:13:47.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.644+0000 7f120e985b80 1 client.25160 shutdown 2023-09-09T23:13:47.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.644+0000 7f120e985b80 1 -- 172.21.15.82:0/3793774221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55764bf40230 msgr2=0x55764bf50b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.644+0000 7f120e985b80 1 --2- 172.21.15.82:0/3793774221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55764bf40230 0x55764bf50b10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f11ec0a1420 tx=0x7f11ec097950 comp rx=0 tx=0).stop 2023-09-09T23:13:47.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.644+0000 7f120e985b80 1 -- 172.21.15.82:0/3793774221 shutdown_connections 2023-09-09T23:13:47.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.644+0000 7f120e985b80 1 --2- 172.21.15.82:0/3793774221 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55764bf55310 0x55764bfac4d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.644+0000 7f120e985b80 1 --2- 172.21.15.82:0/3793774221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55764bf40230 0x55764bf50b10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.644+0000 7f11db7fe700 0 client.25160 ms_handle_reset on v2:172.21.15.98:6832/3855146755 2023-09-09T23:13:47.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.644+0000 7f120e985b80 1 --2- 172.21.15.82:0/3793774221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55764bf419d0 0x55764bf4d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.644+0000 7f120e985b80 1 --2- 172.21.15.82:0/3793774221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55764bf41070 0x55764bf51050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.644+0000 7f120e985b80 1 -- 172.21.15.82:0/3793774221 >> 172.21.15.82:0/3793774221 conn(0x55764beb0090 msgr2=0x55764bf47860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:47.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.644+0000 7f120e985b80 1 -- 172.21.15.82:0/3793774221 shutdown_connections 2023-09-09T23:13:47.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.644+0000 7f120e985b80 1 -- 172.21.15.82:0/3793774221 wait complete. 2023-09-09T23:13:47.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.644+0000 7f120e985b80 20 client.25160 trim_cache size 0 max 16384 2023-09-09T23:13:47.665 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2023-09-09T23:13:47.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.644+0000 7f12027bf700 1 -- 172.21.15.82:0/4102511977 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f11e40a0710 msgr2=0x7f11e40a2bd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-09T23:13:47.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.644+0000 7f12027bf700 1 --2- 172.21.15.82:0/4102511977 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f11e40a0710 0x7f11e40a2bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.644+0000 7f12027bf700 1 -- 172.21.15.82:0/4102511977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11fc0ff4a0 msgr2=0x7f11fc194150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.644+0000 7f12027bf700 1 --2- 172.21.15.82:0/4102511977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11fc0ff4a0 0x7f11fc194150 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00e970 tx=0x7f11ec00ec80 comp rx=0 tx=0).stop 2023-09-09T23:13:47.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.645+0000 7f12027bf700 1 -- 172.21.15.82:0/4102511977 shutdown_connections 2023-09-09T23:13:47.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.645+0000 7f12027bf700 1 --2- 172.21.15.82:0/4102511977 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f11e40a0710 0x7f11e40a2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.645+0000 7f12027bf700 1 --2- 172.21.15.82:0/4102511977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11fc101030 0x7f11fc19b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.645+0000 7f12027bf700 1 --2- 172.21.15.82:0/4102511977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11fc0ff4a0 0x7f11fc194150 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.645+0000 7f12027bf700 1 --2- 172.21.15.82:0/4102511977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11fc100690 0x7f11fc194690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.645+0000 7f12027bf700 1 -- 172.21.15.82:0/4102511977 >> 172.21.15.82:0/4102511977 conn(0x7f11fc0faa10 msgr2=0x7f11fc0fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:47.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.645+0000 7f12027bf700 1 -- 172.21.15.82:0/4102511977 shutdown_connections 2023-09-09T23:13:47.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.645+0000 7f12027bf700 1 -- 172.21.15.82:0/4102511977 wait complete. 2023-09-09T23:13:47.672 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-09T23:13:47.714 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.717+0000 7ff7a1d9b700 1 -- 172.21.15.82:0/501254384 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff78c0a0690 msgr2=0x7ff78c0a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1453069857 2023-09-09T23:13:47.714 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.717+0000 7ff7a1d9b700 1 --2- 172.21.15.82:0/501254384 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff78c0a0690 0x7ff78c0a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-09T23:13:47.723 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.726+0000 7ff7a8a2d700 1 -- 172.21.15.82:0/501254384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7ff7a404e3e0 con 0x7ff7a4101120 2023-09-09T23:13:47.730 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.733+0000 7ff7937fe700 1 -- 172.21.15.82:0/501254384 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v44)=0 v44) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff7940260d0 con 0x7ff7a4101120 2023-09-09T23:13:47.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.736+0000 7ff7a8a2d700 1 -- 172.21.15.82:0/501254384 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff78c0a0690 msgr2=0x7ff78c0a2b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-09T23:13:47.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.736+0000 7ff7a8a2d700 1 --2- 172.21.15.82:0/501254384 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff78c0a0690 0x7ff78c0a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.736+0000 7ff7a8a2d700 1 -- 172.21.15.82:0/501254384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a4101120 msgr2=0x7ff7a41985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.736+0000 7ff7a8a2d700 1 --2- 172.21.15.82:0/501254384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a4101120 0x7ff7a41985d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff79400e9a0 tx=0x7ff79400ecb0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.736+0000 7ff7a8a2d700 1 -- 172.21.15.82:0/501254384 shutdown_connections 2023-09-09T23:13:47.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.736+0000 7ff7a8a2d700 1 --2- 172.21.15.82:0/501254384 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7ff78c0a0690 0x7ff78c0a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.736+0000 7ff7a8a2d700 1 --2- 172.21.15.82:0/501254384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7a4106380 0x7ff7a4199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.736+0000 7ff7a8a2d700 1 --2- 172.21.15.82:0/501254384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7a4101120 0x7ff7a41985d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.738 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.736+0000 7ff7a8a2d700 1 --2- 172.21.15.82:0/501254384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7a4103a50 0x7ff7a4198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.738 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.736+0000 7ff7a8a2d700 1 -- 172.21.15.82:0/501254384 >> 172.21.15.82:0/501254384 conn(0x7ff7a40faab0 msgr2=0x7ff7a4101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:47.738 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.736+0000 7ff7a8a2d700 1 -- 172.21.15.82:0/501254384 shutdown_connections 2023-09-09T23:13:47.738 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.736+0000 7ff7a8a2d700 1 -- 172.21.15.82:0/501254384 wait complete. 2023-09-09T23:13:47.747 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2023-09-09T23:13:47.811 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:47 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:47.545+0000 7f0a735f8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-09T23:13:47.811 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:47 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:47.631+0000 7f0a735f8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-09T23:13:47.811 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:47 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:47.728+0000 7f0a735f8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-09T23:13:47.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.840+0000 7f99a09f8700 1 -- 172.21.15.82:0/984303327 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f999c1018b0 msgr2=0x7f999c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.840 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.840+0000 7f99a09f8700 1 --2- 172.21.15.82:0/984303327 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f999c1018b0 0x7f999c101d10 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9990009990 tx=0x7f9990009ca0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.840 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.840+0000 7f99a09f8700 1 -- 172.21.15.82:0/984303327 shutdown_connections 2023-09-09T23:13:47.840 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.840+0000 7f99a09f8700 1 --2- 172.21.15.82:0/984303327 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f999c0fe8e0 0x7f999c0fed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.840 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.840+0000 7f99a09f8700 1 --2- 172.21.15.82:0/984303327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f999c102250 0x7f999c108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.840 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.840+0000 7f99a09f8700 1 --2- 172.21.15.82:0/984303327 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f999c1018b0 0x7f999c101d10 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.840+0000 7f99a09f8700 1 -- 172.21.15.82:0/984303327 >> 172.21.15.82:0/984303327 conn(0x7f999c0fa1f0 msgr2=0x7f999c0fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:47.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.840+0000 7f99a09f8700 1 -- 172.21.15.82:0/984303327 shutdown_connections 2023-09-09T23:13:47.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.840+0000 7f99a09f8700 1 -- 172.21.15.82:0/984303327 wait complete. 2023-09-09T23:13:47.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.841+0000 7f99a09f8700 1 Processor -- start 2023-09-09T23:13:47.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.841+0000 7f99a09f8700 1 -- start start 2023-09-09T23:13:47.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.841+0000 7f99a09f8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f999c0fe8e0 0x7f999c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.841+0000 7f99a09f8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f999c1018b0 0x7f999c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.841+0000 7f99a09f8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f999c102250 0x7f999c19fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.841+0000 7f99a09f8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999c1a00a0 con 0x7f999c0fe8e0 2023-09-09T23:13:47.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.841+0000 7f99a09f8700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999c1a01e0 con 0x7f999c1018b0 2023-09-09T23:13:47.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.841+0000 7f99a09f8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999c1a0370 con 0x7f999c102250 2023-09-09T23:13:47.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.841+0000 7f999a59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f999c0fe8e0 0x7f999c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.841+0000 7f999a59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f999c0fe8e0 0x7f999c198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56496/0 (socket says 172.21.15.82:56496) 2023-09-09T23:13:47.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.841+0000 7f999a59c700 1 -- 172.21.15.82:0/4250782671 learned_addr learned my addr 172.21.15.82:0/4250782671 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:47.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.841+0000 7f9999d9b700 1 --2- 172.21.15.82:0/4250782671 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f999c1018b0 0x7f999c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.841+0000 7f999ad9d700 1 --2- 172.21.15.82:0/4250782671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f999c102250 0x7f999c19fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.841+0000 7f999a59c700 1 -- 172.21.15.82:0/4250782671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f999c102250 msgr2=0x7f999c19fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.841+0000 7f999a59c700 1 --2- 172.21.15.82:0/4250782671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f999c102250 0x7f999c19fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.841+0000 7f999a59c700 1 -- 172.21.15.82:0/4250782671 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f999c1018b0 msgr2=0x7f999c198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.841+0000 7f999a59c700 1 --2- 172.21.15.82:0/4250782671 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f999c1018b0 0x7f999c198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.841+0000 7f999a59c700 1 -- 172.21.15.82:0/4250782671 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998c009520 con 0x7f999c0fe8e0 2023-09-09T23:13:47.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.841+0000 7f9999d9b700 1 --2- 172.21.15.82:0/4250782671 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f999c1018b0 0x7f999c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:47.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.841+0000 7f999ad9d700 1 --2- 172.21.15.82:0/4250782671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f999c102250 0x7f999c19fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:47.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.842+0000 7f999a59c700 1 --2- 172.21.15.82:0/4250782671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f999c0fe8e0 0x7f999c198520 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f998c00ea50 tx=0x7f998c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:47.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.842+0000 7f998b7fe700 1 -- 172.21.15.82:0/4250782671 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c00cbd0 con 0x7f999c0fe8e0 2023-09-09T23:13:47.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.842+0000 7f998b7fe700 1 -- 172.21.15.82:0/4250782671 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f998c00cd30 con 0x7f999c0fe8e0 2023-09-09T23:13:47.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.842+0000 7f998b7fe700 1 -- 172.21.15.82:0/4250782671 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c0186c0 con 0x7f999c0fe8e0 2023-09-09T23:13:47.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.842+0000 7f99a09f8700 1 -- 172.21.15.82:0/4250782671 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99900095f0 con 0x7f999c0fe8e0 2023-09-09T23:13:47.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.842+0000 7f99a09f8700 1 -- 172.21.15.82:0/4250782671 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f999c1a0980 con 0x7f999c0fe8e0 2023-09-09T23:13:47.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.845+0000 7f998b7fe700 1 -- 172.21.15.82:0/4250782671 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 19) v1 ==== 135276+0+0 (secure 0 0 0) 0x7f998c018820 con 0x7f999c0fe8e0 2023-09-09T23:13:47.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.845+0000 7f998b7fe700 1 --2- 172.21.15.82:0/4250782671 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f99840a0690 0x7f99840a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.845+0000 7f9999d9b700 1 -- 172.21.15.82:0/4250782671 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f99840a0690 msgr2=0x7f99840a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1453069857 2023-09-09T23:13:47.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.845+0000 7f998b7fe700 1 -- 172.21.15.82:0/4250782671 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f998c014070 con 0x7f999c0fe8e0 2023-09-09T23:13:47.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.845+0000 7f9999d9b700 1 --2- 172.21.15.82:0/4250782671 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f99840a0690 0x7f99840a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-09T23:13:47.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.847+0000 7f99acbbeb80 1 Processor -- start 2023-09-09T23:13:47.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.847+0000 7f99acbbeb80 1 -- start start 2023-09-09T23:13:47.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.847+0000 7f99acbbeb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603c978dc00 0x5603c978e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.847+0000 7f99acbbeb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c978e5a0 0x5603c978c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.847+0000 7f99acbbeb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603c978ce00 0x5603c978d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.847+0000 7f99acbbeb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c96e5730 con 0x5603c978e5a0 2023-09-09T23:13:47.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.847+0000 7f99acbbeb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c96e3050 con 0x5603c978dc00 2023-09-09T23:13:47.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.847+0000 7f99acbbeb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c96de940 con 0x5603c978ce00 2023-09-09T23:13:47.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.847+0000 7f9999d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c978e5a0 0x5603c978c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.848+0000 7f999a59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603c978dc00 0x5603c978e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.848+0000 7f9999d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c978e5a0 0x5603c978c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56504/0 (socket says 172.21.15.82:56504) 2023-09-09T23:13:47.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.848+0000 7f9999d9b700 1 -- 172.21.15.82:0/3357649959 learned_addr learned my addr 172.21.15.82:0/3357649959 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:47.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.848+0000 7f999ad9d700 1 --2- 172.21.15.82:0/3357649959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603c978ce00 0x5603c978d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.848+0000 7f999a59c700 1 -- 172.21.15.82:0/3357649959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603c978ce00 msgr2=0x5603c978d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:47.856 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.848+0000 7f999a59c700 1 --2- 172.21.15.82:0/3357649959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603c978ce00 0x5603c978d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.856 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.848+0000 7f999a59c700 1 -- 172.21.15.82:0/3357649959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c978e5a0 msgr2=0x5603c978c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:47.856 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.848+0000 7f999a59c700 1 --2- 172.21.15.82:0/3357649959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c978e5a0 0x5603c978c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.856 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.848+0000 7f999a59c700 1 -- 172.21.15.82:0/3357649959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603c950ade0 con 0x5603c978dc00 2023-09-09T23:13:47.856 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.848+0000 7f999ad9d700 1 --2- 172.21.15.82:0/3357649959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603c978ce00 0x5603c978d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:47.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.848+0000 7f9999d9b700 1 --2- 172.21.15.82:0/3357649959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c978e5a0 0x5603c978c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:47.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.848+0000 7f999a59c700 1 --2- 172.21.15.82:0/3357649959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603c978dc00 0x5603c978e020 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f998c0c9cc0 tx=0x7f998c099860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dea7abc5a9616e1d server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:47.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.848+0000 7f9988ff9700 1 -- 172.21.15.82:0/3357649959 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c01e2c0 con 0x5603c978dc00 2023-09-09T23:13:47.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.848+0000 7f9988ff9700 1 -- 172.21.15.82:0/3357649959 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f998c09a060 con 0x5603c978dc00 2023-09-09T23:13:47.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.848+0000 7f9988ff9700 1 -- 172.21.15.82:0/3357649959 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c099e20 con 0x5603c978dc00 2023-09-09T23:13:47.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.849+0000 7f99acbbeb80 1 -- 172.21.15.82:0/3357649959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603c978dc00 msgr2=0x5603c978e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.849+0000 7f99acbbeb80 1 --2- 172.21.15.82:0/3357649959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603c978dc00 0x5603c978e020 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f998c0c9cc0 tx=0x7f998c099860 comp rx=0 tx=0).stop 2023-09-09T23:13:47.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.849+0000 7f99acbbeb80 1 -- 172.21.15.82:0/3357649959 shutdown_connections 2023-09-09T23:13:47.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.849+0000 7f99acbbeb80 1 --2- 172.21.15.82:0/3357649959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603c978dc00 0x5603c978e020 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.849+0000 7f99acbbeb80 1 --2- 172.21.15.82:0/3357649959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603c978ce00 0x5603c978d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.849+0000 7f99acbbeb80 1 --2- 172.21.15.82:0/3357649959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c978e5a0 0x5603c978c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.849+0000 7f99acbbeb80 1 -- 172.21.15.82:0/3357649959 >> 172.21.15.82:0/3357649959 conn(0x5603c9701090 msgr2=0x5603c9787720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:47.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.849+0000 7f99acbbeb80 1 -- 172.21.15.82:0/3357649959 shutdown_connections 2023-09-09T23:13:47.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.849+0000 7f99acbbeb80 1 -- 172.21.15.82:0/3357649959 wait complete. 2023-09-09T23:13:47.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.850+0000 7f99acbbeb80 1 -- start start 2023-09-09T23:13:47.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.850+0000 7f99acbbeb80 1 Processor -- start 2023-09-09T23:13:47.860 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.850+0000 7f99acbbeb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c978ce00 0x5603c97965c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.860 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.850+0000 7f99acbbeb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603c978dc00 0x5603c9796b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.860 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.850+0000 7f99acbbeb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603c978e5a0 0x5603c979db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.860 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.850+0000 7f99acbbeb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c967cb70 con 0x5603c978ce00 2023-09-09T23:13:47.860 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.850+0000 7f99acbbeb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c967c870 con 0x5603c978dc00 2023-09-09T23:13:47.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.850+0000 7f99acbbeb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c9560920 con 0x5603c978e5a0 2023-09-09T23:13:47.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.850+0000 7f99acbbeb80 10 client.0 _ll_register_callbacks cb 0x5603c95c6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:47.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.850+0000 7f999a59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c978ce00 0x5603c97965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.850+0000 7f999a59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c978ce00 0x5603c97965c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56510/0 (socket says 172.21.15.82:56510) 2023-09-09T23:13:47.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.850+0000 7f999a59c700 1 -- 172.21.15.82:0/3135984118 learned_addr learned my addr 172.21.15.82:0/3135984118 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:47.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.850+0000 7f9999d9b700 1 --2- 172.21.15.82:0/3135984118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603c978dc00 0x5603c9796b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.862 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.851+0000 7f999ad9d700 1 --2- 172.21.15.82:0/3135984118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603c978e5a0 0x5603c979db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.862 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.851+0000 7f999a59c700 1 -- 172.21.15.82:0/3135984118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603c978e5a0 msgr2=0x5603c979db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:47.862 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.851+0000 7f999a59c700 1 --2- 172.21.15.82:0/3135984118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603c978e5a0 0x5603c979db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.862 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.851+0000 7f999a59c700 1 -- 172.21.15.82:0/3135984118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603c978dc00 msgr2=0x5603c9796b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:47.862 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.851+0000 7f999a59c700 1 --2- 172.21.15.82:0/3135984118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603c978dc00 0x5603c9796b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.863 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.851+0000 7f999a59c700 1 -- 172.21.15.82:0/3135984118 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f999c109400 con 0x5603c978ce00 2023-09-09T23:13:47.863 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.851+0000 7f9999d9b700 1 --2- 172.21.15.82:0/3135984118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603c978dc00 0x5603c9796b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:47.863 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.851+0000 7f999ad9d700 1 --2- 172.21.15.82:0/3135984118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603c978e5a0 0x5603c979db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:47.863 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.851+0000 7f999a59c700 1 --2- 172.21.15.82:0/3135984118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c978ce00 0x5603c97965c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f998c009230 tx=0x7f998c0a1a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6cbae92f1396e5e server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:47.863 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.851+0000 7f9981ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:47.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.851+0000 7f9981ffb700 1 -- 172.21.15.82:0/3135984118 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c099960 con 0x5603c978ce00 2023-09-09T23:13:47.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.851+0000 7f9981ffb700 1 -- 172.21.15.82:0/3135984118 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f998c09d790 con 0x5603c978ce00 2023-09-09T23:13:47.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.851+0000 7f99acbbeb80 1 -- 172.21.15.82:0/3135984118 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5603c96364f0 con 0x5603c978ce00 2023-09-09T23:13:47.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.851+0000 7f9981ffb700 1 -- 172.21.15.82:0/3135984118 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f998c099bc0 con 0x5603c978ce00 2023-09-09T23:13:47.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.851+0000 7f99acbbeb80 10 client.15651 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:47.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.851+0000 7f99acbbeb80 1 -- 172.21.15.82:0/3135984118 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f998c084040 con 0x5603c978ce00 2023-09-09T23:13:47.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.852+0000 7f9981ffb700 1 -- 172.21.15.82:0/3135984118 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f998c0a5080 con 0x5603c978ce00 2023-09-09T23:13:47.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.852+0000 7f9981ffb700 1 -- 172.21.15.82:0/3135984118 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f998c0b7bb0 con 0x5603c978ce00 2023-09-09T23:13:47.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.852+0000 7f99acbbeb80 10 client.15651 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:47.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.852+0000 7f99acbbeb80 10 client.15651 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-09-09T23:13:47.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.852+0000 7f99acbbeb80 20 client.15651 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:47.866 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.852+0000 7f99acbbeb80 1 --2- 172.21.15.82:0/3135984118 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x5603c97a1e70 0x5603c97a4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.866 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.852+0000 7f99acbbeb80 4 client.15651 mds_command: new command op to 24523 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:13:47.866 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.852+0000 7f99acbbeb80 1 -- 172.21.15.82:0/3135984118 --> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5603c96f38a0 con 0x5603c97a1e70 2023-09-09T23:13:47.866 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.852+0000 7f9999d9b700 1 --2- 172.21.15.82:0/3135984118 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x5603c97a1e70 0x5603c97a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.866 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.853+0000 7f9981ffb700 10 client.15651 ms_handle_connect on v2:172.21.15.170:6838/3858369735 2023-09-09T23:13:47.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.853+0000 7f9999d9b700 1 --2- 172.21.15.82:0/3135984118 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x5603c97a1e70 0x5603c97a4290 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d31abab4b347beb4 server_cookie=22b6c63110410412 in_seq=0 out_seq=0 2023-09-09T23:13:47.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.853+0000 7f9981ffb700 1 -- 172.21.15.82:0/3135984118 <== mds.2 v2:172.21.15.170:6838/3858369735 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5603c96f38a0 con 0x5603c97a1e70 2023-09-09T23:13:47.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.853+0000 7f9981ffb700 10 client.15651 handle_command_reply: tid=0 2023-09-09T23:13:47.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.853+0000 7f99acbbeb80 1 client.15651 shutdown 2023-09-09T23:13:47.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.853+0000 7f99acbbeb80 1 -- 172.21.15.82:0/3135984118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c978ce00 msgr2=0x5603c97965c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.853+0000 7f99acbbeb80 1 --2- 172.21.15.82:0/3135984118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c978ce00 0x5603c97965c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f998c009230 tx=0x7f998c0a1a60 comp rx=0 tx=0).stop 2023-09-09T23:13:47.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.853+0000 7f99acbbeb80 1 -- 172.21.15.82:0/3135984118 shutdown_connections 2023-09-09T23:13:47.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.853+0000 7f99acbbeb80 1 --2- 172.21.15.82:0/3135984118 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x5603c97a1e70 0x5603c97a4290 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.853+0000 7f99acbbeb80 1 --2- 172.21.15.82:0/3135984118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603c978e5a0 0x5603c979db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.853+0000 7f9981ffb700 0 client.15651 ms_handle_reset on v2:172.21.15.170:6838/3858369735 2023-09-09T23:13:47.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.853+0000 7f99acbbeb80 1 --2- 172.21.15.82:0/3135984118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c978ce00 0x5603c97965c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.853+0000 7f99acbbeb80 1 --2- 172.21.15.82:0/3135984118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603c978dc00 0x5603c9796b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.853+0000 7f99acbbeb80 1 -- 172.21.15.82:0/3135984118 >> 172.21.15.82:0/3135984118 conn(0x5603c9701090 msgr2=0x5603c9786dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:47.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.854+0000 7f99acbbeb80 1 -- 172.21.15.82:0/3135984118 shutdown_connections 2023-09-09T23:13:47.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.854+0000 7f99acbbeb80 1 -- 172.21.15.82:0/3135984118 wait complete. 2023-09-09T23:13:47.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.854+0000 7f99acbbeb80 20 client.15651 trim_cache size 0 max 16384 2023-09-09T23:13:47.875 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.866+0000 7f99acbbeb80 1 Processor -- start 2023-09-09T23:13:47.875 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.866+0000 7f99acbbeb80 1 -- start start 2023-09-09T23:13:47.876 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.867+0000 7f99acbbeb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603c978ce00 0x5603c978c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.876 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.867+0000 7f99acbbeb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603c978dc00 0x5603c979db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.876 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.867+0000 7f99acbbeb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c978e5a0 0x5603c9790f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.876 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.867+0000 7f99acbbeb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c9796130 con 0x5603c978e5a0 2023-09-09T23:13:47.876 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.867+0000 7f99acbbeb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c97843d0 con 0x5603c978dc00 2023-09-09T23:13:47.877 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.867+0000 7f99acbbeb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c978cc50 con 0x5603c978ce00 2023-09-09T23:13:47.877 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.867+0000 7f999a59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603c978ce00 0x5603c978c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.877 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.867+0000 7f999ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c978e5a0 0x5603c9790f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.877 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.867+0000 7f999ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c978e5a0 0x5603c9790f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56522/0 (socket says 172.21.15.82:56522) 2023-09-09T23:13:47.877 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.867+0000 7f999ad9d700 1 -- 172.21.15.82:0/1316581720 learned_addr learned my addr 172.21.15.82:0/1316581720 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:47.878 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.867+0000 7f9999d9b700 1 --2- 172.21.15.82:0/1316581720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603c978dc00 0x5603c979db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.878 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.867+0000 7f999a59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603c978ce00 0x5603c978c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:55032/0 (socket says 172.21.15.82:55032) 2023-09-09T23:13:47.878 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.867+0000 7f999ad9d700 1 -- 172.21.15.82:0/1316581720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603c978ce00 msgr2=0x5603c978c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:47.878 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.867+0000 7f999ad9d700 1 --2- 172.21.15.82:0/1316581720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603c978ce00 0x5603c978c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.878 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.867+0000 7f999ad9d700 1 -- 172.21.15.82:0/1316581720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603c978dc00 msgr2=0x5603c979db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:47.878 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.867+0000 7f999ad9d700 1 --2- 172.21.15.82:0/1316581720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603c978dc00 0x5603c979db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.867+0000 7f999ad9d700 1 -- 172.21.15.82:0/1316581720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603c94f3c60 con 0x5603c978e5a0 2023-09-09T23:13:47.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.867+0000 7f9999d9b700 1 --2- 172.21.15.82:0/1316581720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603c978dc00 0x5603c979db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:47.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.867+0000 7f999ad9d700 1 --2- 172.21.15.82:0/1316581720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c978e5a0 0x5603c9790f70 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f99940084d0 tx=0x7f999400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7511c937f14c81 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:47.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.868+0000 7f99897fa700 1 -- 172.21.15.82:0/1316581720 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999400e030 con 0x5603c978e5a0 2023-09-09T23:13:47.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.868+0000 7f99897fa700 1 -- 172.21.15.82:0/1316581720 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f999400f040 con 0x5603c978e5a0 2023-09-09T23:13:47.880 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.868+0000 7f99897fa700 1 -- 172.21.15.82:0/1316581720 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9994016430 con 0x5603c978e5a0 2023-09-09T23:13:47.880 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.868+0000 7f99acbbeb80 1 -- 172.21.15.82:0/1316581720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c978e5a0 msgr2=0x5603c9790f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.880 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.868+0000 7f99acbbeb80 1 --2- 172.21.15.82:0/1316581720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c978e5a0 0x5603c9790f70 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f99940084d0 tx=0x7f999400b040 comp rx=0 tx=0).stop 2023-09-09T23:13:47.880 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.868+0000 7f99acbbeb80 1 -- 172.21.15.82:0/1316581720 shutdown_connections 2023-09-09T23:13:47.880 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.868+0000 7f99acbbeb80 1 --2- 172.21.15.82:0/1316581720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c978e5a0 0x5603c9790f70 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.881 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.868+0000 7f99acbbeb80 1 --2- 172.21.15.82:0/1316581720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603c978ce00 0x5603c978c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.881 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.868+0000 7f99acbbeb80 1 --2- 172.21.15.82:0/1316581720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603c978dc00 0x5603c979db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.881 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.868+0000 7f99acbbeb80 1 -- 172.21.15.82:0/1316581720 >> 172.21.15.82:0/1316581720 conn(0x5603c9701090 msgr2=0x5603c979e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:47.881 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.868+0000 7f99acbbeb80 1 -- 172.21.15.82:0/1316581720 shutdown_connections 2023-09-09T23:13:47.881 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.868+0000 7f99acbbeb80 1 -- 172.21.15.82:0/1316581720 wait complete. 2023-09-09T23:13:47.881 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.869+0000 7f99acbbeb80 1 -- start start 2023-09-09T23:13:47.882 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.869+0000 7f99acbbeb80 1 Processor -- start 2023-09-09T23:13:47.882 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.869+0000 7f99acbbeb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603c978ce00 0x5603c979d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.882 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.869+0000 7f99acbbeb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603c978dc00 0x5603c979c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.882 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.869+0000 7f99acbbeb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c978e5a0 0x5603c979c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.882 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.869+0000 7f99acbbeb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c979cc70 con 0x5603c978e5a0 2023-09-09T23:13:47.883 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.869+0000 7f99acbbeb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c979cde0 con 0x5603c978ce00 2023-09-09T23:13:47.883 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.869+0000 7f99acbbeb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c97f5c90 con 0x5603c978dc00 2023-09-09T23:13:47.883 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.869+0000 7f99acbbeb80 10 client.0 _ll_register_callbacks cb 0x5603c95c6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:47.883 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.869+0000 7f999ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c978e5a0 0x5603c979c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.883 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.869+0000 7f999a59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603c978ce00 0x5603c979d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.869+0000 7f999ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c978e5a0 0x5603c979c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56528/0 (socket says 172.21.15.82:56528) 2023-09-09T23:13:47.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.869+0000 7f999a59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603c978ce00 0x5603c979d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:42088/0 (socket says 172.21.15.82:42088) 2023-09-09T23:13:47.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.869+0000 7f999ad9d700 1 -- 172.21.15.82:0/339767380 learned_addr learned my addr 172.21.15.82:0/339767380 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:47.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.870+0000 7f9999d9b700 1 --2- 172.21.15.82:0/339767380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603c978dc00 0x5603c979c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.870+0000 7f999ad9d700 1 -- 172.21.15.82:0/339767380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603c978dc00 msgr2=0x5603c979c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:47.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.870+0000 7f999ad9d700 1 --2- 172.21.15.82:0/339767380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603c978dc00 0x5603c979c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.870+0000 7f999ad9d700 1 -- 172.21.15.82:0/339767380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603c978ce00 msgr2=0x5603c979d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:47.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.870+0000 7f999ad9d700 1 --2- 172.21.15.82:0/339767380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603c978ce00 0x5603c979d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.870+0000 7f999ad9d700 1 -- 172.21.15.82:0/339767380 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9990005410 con 0x5603c978e5a0 2023-09-09T23:13:47.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.870+0000 7f999a59c700 1 --2- 172.21.15.82:0/339767380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603c978ce00 0x5603c979d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:47.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.870+0000 7f999ad9d700 1 --2- 172.21.15.82:0/339767380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c978e5a0 0x5603c979c560 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f999400b010 tx=0x7f9994004020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddce63b03aaf9687 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:47.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.870+0000 7f9981ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:47.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.870+0000 7f9981ffb700 1 -- 172.21.15.82:0/339767380 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999400e050 con 0x5603c978e5a0 2023-09-09T23:13:47.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.870+0000 7f99acbbeb80 1 -- 172.21.15.82:0/339767380 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f998c09a060 con 0x5603c978e5a0 2023-09-09T23:13:47.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.870+0000 7f9981ffb700 1 -- 172.21.15.82:0/339767380 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f9994003a00 con 0x5603c978e5a0 2023-09-09T23:13:47.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.870+0000 7f9981ffb700 1 -- 172.21.15.82:0/339767380 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9994023070 con 0x5603c978e5a0 2023-09-09T23:13:47.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.870+0000 7f99acbbeb80 10 client.15657 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:47.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.870+0000 7f99acbbeb80 1 -- 172.21.15.82:0/339767380 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f9994009230 con 0x5603c978e5a0 2023-09-09T23:13:47.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.871+0000 7f9981ffb700 1 -- 172.21.15.82:0/339767380 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f998c09a060 con 0x5603c978e5a0 2023-09-09T23:13:47.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.871+0000 7f9981ffb700 1 -- 172.21.15.82:0/339767380 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f999401e400 con 0x5603c978e5a0 2023-09-09T23:13:47.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.871+0000 7f99acbbeb80 10 client.15657 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:47.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.871+0000 7f99acbbeb80 10 client.15657 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-09-09T23:13:47.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.871+0000 7f99acbbeb80 20 client.15657 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:47.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.871+0000 7f99acbbeb80 1 --2- 172.21.15.82:0/339767380 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x5603c97f7590 0x5603c97f99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.871+0000 7f99acbbeb80 4 client.15657 mds_command: new command op to 24523 tid=0[{"prefix": "damage ls"}] 2023-09-09T23:13:47.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.871+0000 7f99acbbeb80 1 -- 172.21.15.82:0/339767380 --> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5603c96d7920 con 0x5603c97f7590 2023-09-09T23:13:47.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.871+0000 7f999a59c700 1 --2- 172.21.15.82:0/339767380 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x5603c97f7590 0x5603c97f99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.871+0000 7f999a59c700 1 --2- 172.21.15.82:0/339767380 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x5603c97f7590 0x5603c97f99b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aee8fee549cbdf72 server_cookie=c2107d3108d79ab7 in_seq=0 out_seq=0 2023-09-09T23:13:47.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.871+0000 7f9981ffb700 10 client.15657 ms_handle_connect on v2:172.21.15.170:6838/3858369735 2023-09-09T23:13:47.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.872+0000 7f9981ffb700 1 -- 172.21.15.82:0/339767380 <== mds.2 v2:172.21.15.170:6838/3858369735 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5603c96d7920 con 0x5603c97f7590 2023-09-09T23:13:47.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.872+0000 7f9981ffb700 10 client.15657 handle_command_reply: tid=0 2023-09-09T23:13:47.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.872+0000 7f99acbbeb80 1 client.15657 shutdown 2023-09-09T23:13:47.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.872+0000 7f99acbbeb80 1 -- 172.21.15.82:0/339767380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c978e5a0 msgr2=0x5603c979c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.872+0000 7f99acbbeb80 1 --2- 172.21.15.82:0/339767380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c978e5a0 0x5603c979c560 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f999400b010 tx=0x7f9994004020 comp rx=0 tx=0).stop 2023-09-09T23:13:47.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.872+0000 7f99acbbeb80 1 -- 172.21.15.82:0/339767380 shutdown_connections 2023-09-09T23:13:47.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.872+0000 7f99acbbeb80 1 --2- 172.21.15.82:0/339767380 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x5603c97f7590 0x5603c97f99b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.872+0000 7f99acbbeb80 1 --2- 172.21.15.82:0/339767380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603c978ce00 0x5603c979d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.872+0000 7f99acbbeb80 1 --2- 172.21.15.82:0/339767380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c978e5a0 0x5603c979c560 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.872+0000 7f99acbbeb80 1 --2- 172.21.15.82:0/339767380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603c978dc00 0x5603c979c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.872+0000 7f99acbbeb80 1 -- 172.21.15.82:0/339767380 >> 172.21.15.82:0/339767380 conn(0x5603c9701090 msgr2=0x5603c9790f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:47.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.872+0000 7f9981ffb700 0 client.15657 ms_handle_reset on v2:172.21.15.170:6838/3858369735 2023-09-09T23:13:47.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.872+0000 7f99acbbeb80 1 -- 172.21.15.82:0/339767380 shutdown_connections 2023-09-09T23:13:47.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.872+0000 7f99acbbeb80 1 -- 172.21.15.82:0/339767380 wait complete. 2023-09-09T23:13:47.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.872+0000 7f99acbbeb80 20 client.15657 trim_cache size 0 max 16384 2023-09-09T23:13:47.892 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2023-09-09T23:13:47.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.873+0000 7f99a09f8700 1 -- 172.21.15.82:0/4250782671 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f99840a0690 msgr2=0x7f99840a2b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-09T23:13:47.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.873+0000 7f99a09f8700 1 --2- 172.21.15.82:0/4250782671 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f99840a0690 0x7f99840a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.873+0000 7f99a09f8700 1 -- 172.21.15.82:0/4250782671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f999c0fe8e0 msgr2=0x7f999c198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.873+0000 7f99a09f8700 1 --2- 172.21.15.82:0/4250782671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f999c0fe8e0 0x7f999c198520 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f998c00ea50 tx=0x7f998c00ed60 comp rx=0 tx=0).stop 2023-09-09T23:13:47.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.873+0000 7f99a09f8700 1 -- 172.21.15.82:0/4250782671 shutdown_connections 2023-09-09T23:13:47.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.873+0000 7f99a09f8700 1 --2- 172.21.15.82:0/4250782671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f999c102250 0x7f999c19fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.873+0000 7f99a09f8700 1 --2- 172.21.15.82:0/4250782671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f999c0fe8e0 0x7f999c198520 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.873+0000 7f99a09f8700 1 --2- 172.21.15.82:0/4250782671 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f99840a0690 0x7f99840a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.873+0000 7f99a09f8700 1 --2- 172.21.15.82:0/4250782671 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f999c1018b0 0x7f999c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.873+0000 7f99a09f8700 1 -- 172.21.15.82:0/4250782671 >> 172.21.15.82:0/4250782671 conn(0x7f999c0fa1f0 msgr2=0x7f999c0fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:47.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.873+0000 7f99a09f8700 1 -- 172.21.15.82:0/4250782671 shutdown_connections 2023-09-09T23:13:47.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.873+0000 7f99a09f8700 1 -- 172.21.15.82:0/4250782671 wait complete. 2023-09-09T23:13:47.896 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-09T23:13:47.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.921+0000 7f3b080d3700 1 -- 172.21.15.82:0/1278374080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b000fe8e0 msgr2=0x7f3b000fed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.921+0000 7f3b080d3700 1 --2- 172.21.15.82:0/1278374080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b000fe8e0 0x7f3b000fed00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3afc009990 tx=0x7f3afc009ca0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.921+0000 7f3b080d3700 1 -- 172.21.15.82:0/1278374080 shutdown_connections 2023-09-09T23:13:47.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.921+0000 7f3b080d3700 1 --2- 172.21.15.82:0/1278374080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b001022f0 0x7f3b00108ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.921+0000 7f3b080d3700 1 --2- 172.21.15.82:0/1278374080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b000fe8e0 0x7f3b000fed00 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.921+0000 7f3b080d3700 1 --2- 172.21.15.82:0/1278374080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b00101950 0x7f3b00101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.921+0000 7f3b080d3700 1 -- 172.21.15.82:0/1278374080 >> 172.21.15.82:0/1278374080 conn(0x7f3b000fa1d0 msgr2=0x7f3b000fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:47.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.921+0000 7f3b080d3700 1 -- 172.21.15.82:0/1278374080 shutdown_connections 2023-09-09T23:13:47.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.921+0000 7f3b080d3700 1 -- 172.21.15.82:0/1278374080 wait complete. 2023-09-09T23:13:47.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.922+0000 7f3b080d3700 1 Processor -- start 2023-09-09T23:13:47.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.922+0000 7f3b080d3700 1 -- start start 2023-09-09T23:13:47.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.922+0000 7f3b080d3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b000fe8e0 0x7f3b00196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.922+0000 7f3b080d3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b00101950 0x7f3b00196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.922+0000 7f3b080d3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b001022f0 0x7f3b001977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.922+0000 7f3b080d3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b00197e50 con 0x7f3b000fe8e0 2023-09-09T23:13:47.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.922+0000 7f3b080d3700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b00197fc0 con 0x7f3b00101950 2023-09-09T23:13:47.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.922+0000 7f3b080d3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b00198180 con 0x7f3b001022f0 2023-09-09T23:13:47.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.922+0000 7f3b05e6f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b000fe8e0 0x7f3b00196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.922+0000 7f3b05e6f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b000fe8e0 0x7f3b00196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56538/0 (socket says 172.21.15.82:56538) 2023-09-09T23:13:47.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.922+0000 7f3b05e6f700 1 -- 172.21.15.82:0/676332639 learned_addr learned my addr 172.21.15.82:0/676332639 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:47.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.922+0000 7f3b0566e700 1 --2- 172.21.15.82:0/676332639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b00101950 0x7f3b00196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.922+0000 7f3b06670700 1 --2- 172.21.15.82:0/676332639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b001022f0 0x7f3b001977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:47.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.923+0000 7f3b05e6f700 1 -- 172.21.15.82:0/676332639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b001022f0 msgr2=0x7f3b001977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.923+0000 7f3b05e6f700 1 --2- 172.21.15.82:0/676332639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b001022f0 0x7f3b001977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.923+0000 7f3b05e6f700 1 -- 172.21.15.82:0/676332639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b00101950 msgr2=0x7f3b00196970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:47.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.923+0000 7f3b05e6f700 1 --2- 172.21.15.82:0/676332639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b00101950 0x7f3b00196970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:47.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.923+0000 7f3b05e6f700 1 -- 172.21.15.82:0/676332639 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3afc0095f0 con 0x7f3b000fe8e0 2023-09-09T23:13:47.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.923+0000 7f3b05e6f700 1 --2- 172.21.15.82:0/676332639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b000fe8e0 0x7f3b00196430 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3afc0095c0 tx=0x7f3afc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:47.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.923+0000 7f3af6ffd700 1 -- 172.21.15.82:0/676332639 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc01d070 con 0x7f3b000fe8e0 2023-09-09T23:13:47.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.923+0000 7f3af6ffd700 1 -- 172.21.15.82:0/676332639 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3afc005030 con 0x7f3b000fe8e0 2023-09-09T23:13:47.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.923+0000 7f3af6ffd700 1 -- 172.21.15.82:0/676332639 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc004af0 con 0x7f3b000fe8e0 2023-09-09T23:13:47.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.924+0000 7f3b080d3700 1 -- 172.21.15.82:0/676332639 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b00198400 con 0x7f3b000fe8e0 2023-09-09T23:13:47.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.924+0000 7f3b080d3700 1 -- 172.21.15.82:0/676332639 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b00198650 con 0x7f3b000fe8e0 2023-09-09T23:13:47.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.925+0000 7f3b080d3700 1 -- 172.21.15.82:0/676332639 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b00107da0 con 0x7f3b000fe8e0 2023-09-09T23:13:47.928 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:13:47 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:13:47.604+0000 7fc6bfd29200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-09T23:13:47.928 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:13:47 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:13:47.695+0000 7fc6bfd29200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-09T23:13:47.928 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:13:47 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:13:47.788+0000 7fc6bfd29200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-09T23:13:47.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.927+0000 7f3af6ffd700 1 -- 172.21.15.82:0/676332639 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 19) v1 ==== 135276+0+0 (secure 0 0 0) 0x7f3afc004c50 con 0x7f3b000fe8e0 2023-09-09T23:13:47.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.927+0000 7f3af6ffd700 1 --2- 172.21.15.82:0/676332639 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3aec0a0690 0x7f3aec0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:47.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.927+0000 7f3b0566e700 1 -- 172.21.15.82:0/676332639 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3aec0a0690 msgr2=0x7f3aec0a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1453069857 2023-09-09T23:13:47.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.927+0000 7f3af6ffd700 1 -- 172.21.15.82:0/676332639 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f3afc0ccf30 con 0x7f3b000fe8e0 2023-09-09T23:13:47.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.927+0000 7f3b0566e700 1 --2- 172.21.15.82:0/676332639 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3aec0a0690 0x7f3aec0a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-09T23:13:47.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:47.930+0000 7f3af6ffd700 1 -- 172.21.15.82:0/676332639 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3afc096b90 con 0x7f3b000fe8e0 2023-09-09T23:13:47.960 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:13:47 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:13:47.605+0000 7f0b0093b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-09T23:13:47.961 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:13:47 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:13:47.697+0000 7f0b0093b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-09T23:13:47.961 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:13:47 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:13:47.792+0000 7f0b0093b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-09T23:13:48.046 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.048+0000 7fe6fd20e700 1 -- 172.21.15.82:0/836954941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6f80feee0 msgr2=0x7fe6f80ff300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:48.046 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.048+0000 7fe6fd20e700 1 --2- 172.21.15.82:0/836954941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6f80feee0 0x7fe6f80ff300 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8009990 tx=0x7fe6e8009ca0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.046 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.048+0000 7fe6fd20e700 1 -- 172.21.15.82:0/836954941 shutdown_connections 2023-09-09T23:13:48.046 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.048+0000 7fe6fd20e700 1 --2- 172.21.15.82:0/836954941 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f8100200 0x7fe6f8100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.047 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.048+0000 7fe6fd20e700 1 --2- 172.21.15.82:0/836954941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6f80feee0 0x7fe6f80ff300 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.047 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.048+0000 7fe6fd20e700 1 --2- 172.21.15.82:0/836954941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6f80ff840 0x7fe6f80ffcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.047 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.048+0000 7fe6fd20e700 1 -- 172.21.15.82:0/836954941 >> 172.21.15.82:0/836954941 conn(0x7fe6f80faa60 msgr2=0x7fe6f80fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:48.047 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.048+0000 7fe6fd20e700 1 -- 172.21.15.82:0/836954941 shutdown_connections 2023-09-09T23:13:48.047 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.048+0000 7fe6fd20e700 1 -- 172.21.15.82:0/836954941 wait complete. 2023-09-09T23:13:48.049 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.049+0000 7fe6fd20e700 1 Processor -- start 2023-09-09T23:13:48.049 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.049+0000 7fe6fd20e700 1 -- start start 2023-09-09T23:13:48.049 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.049+0000 7fe6fd20e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f80feee0 0x7fe6f81984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.049+0000 7fe6fd20e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6f80ff840 0x7fe6f8198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.049+0000 7fe6fd20e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6f8100200 0x7fe6f819faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.049+0000 7fe6fd20e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f81a0070 con 0x7fe6f80ff840 2023-09-09T23:13:48.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.049+0000 7fe6fd20e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f81a01b0 con 0x7fe6f80feee0 2023-09-09T23:13:48.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.049+0000 7fe6fd20e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f81a0340 con 0x7fe6f8100200 2023-09-09T23:13:48.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.050+0000 7fe6f659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6f80ff840 0x7fe6f8198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.050+0000 7fe6f6d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f80feee0 0x7fe6f81984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.050+0000 7fe6f659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6f80ff840 0x7fe6f8198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56592/0 (socket says 172.21.15.82:56592) 2023-09-09T23:13:48.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.050+0000 7fe6f659c700 1 -- 172.21.15.82:0/2220843487 learned_addr learned my addr 172.21.15.82:0/2220843487 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:48.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.050+0000 7fe6f6d9d700 1 -- 172.21.15.82:0/2220843487 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6f8100200 msgr2=0x7fe6f819faa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-09T23:13:48.052 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.050+0000 7fe6f6d9d700 1 --2- 172.21.15.82:0/2220843487 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6f8100200 0x7fe6f819faa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.052 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.050+0000 7fe6f6d9d700 1 -- 172.21.15.82:0/2220843487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6f80ff840 msgr2=0x7fe6f8198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:48.052 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.050+0000 7fe6f6d9d700 1 --2- 172.21.15.82:0/2220843487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6f80ff840 0x7fe6f8198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.052 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.050+0000 7fe6f6d9d700 1 -- 172.21.15.82:0/2220843487 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e80095f0 con 0x7fe6f80feee0 2023-09-09T23:13:48.052 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.050+0000 7fe6f659c700 1 --2- 172.21.15.82:0/2220843487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6f80ff840 0x7fe6f8198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:48.052 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.050+0000 7fe6f6d9d700 1 --2- 172.21.15.82:0/2220843487 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f80feee0 0x7fe6f81984f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80059d0 tx=0x7fe6e8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:48.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.050+0000 7fe6dffff700 1 -- 172.21.15.82:0/2220843487 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e801d070 con 0x7fe6f80feee0 2023-09-09T23:13:48.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.050+0000 7fe6fd20e700 1 -- 172.21.15.82:0/2220843487 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f81a05c0 con 0x7fe6f80feee0 2023-09-09T23:13:48.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.050+0000 7fe6dffff700 1 -- 172.21.15.82:0/2220843487 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6e8022470 con 0x7fe6f80feee0 2023-09-09T23:13:48.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.050+0000 7fe6dffff700 1 -- 172.21.15.82:0/2220843487 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8017650 con 0x7fe6f80feee0 2023-09-09T23:13:48.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.050+0000 7fe6fd20e700 1 -- 172.21.15.82:0/2220843487 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f81a0a40 con 0x7fe6f80feee0 2023-09-09T23:13:48.058 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.053+0000 7fe6dffff700 1 -- 172.21.15.82:0/2220843487 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) v1 ==== 135276+0+0 (secure 0 0 0) 0x7fe6e80225e0 con 0x7fe6f80feee0 2023-09-09T23:13:48.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.054+0000 7fe6dffff700 1 --2- 172.21.15.82:0/2220843487 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fe6e00a0690 0x7fe6e00a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.054+0000 7fe6f659c700 1 -- 172.21.15.82:0/2220843487 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fe6e00a0690 msgr2=0x7fe6e00a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1453069857 2023-09-09T23:13:48.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.054+0000 7fe7093d4b80 1 Processor -- start 2023-09-09T23:13:48.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.054+0000 7fe7093d4b80 1 -- start start 2023-09-09T23:13:48.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.054+0000 7fe6dffff700 1 -- 172.21.15.82:0/2220843487 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fe6e80ccd90 con 0x7fe6f80feee0 2023-09-09T23:13:48.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.054+0000 7fe6f659c700 1 --2- 172.21.15.82:0/2220843487 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fe6e00a0690 0x7fe6e00a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-09T23:13:48.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.054+0000 7fe7093d4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x565315d60c00 0x565315d61020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.054+0000 7fe7093d4b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565315d615a0 0x565315d5f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.054+0000 7fe7093d4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565315d5fe00 0x565315d602b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.054+0000 7fe7093d4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x565315cb8730 con 0x565315d5fe00 2023-09-09T23:13:48.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.054+0000 7fe7093d4b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x565315cb6050 con 0x565315d615a0 2023-09-09T23:13:48.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.054+0000 7fe7093d4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x565315cb1940 con 0x565315d60c00 2023-09-09T23:13:48.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.054+0000 7fe6f759e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565315d5fe00 0x565315d602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.054+0000 7fe6f6d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x565315d60c00 0x565315d61020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.054+0000 7fe6f759e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565315d5fe00 0x565315d602b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56602/0 (socket says 172.21.15.82:56602) 2023-09-09T23:13:48.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.054+0000 7fe6f759e700 1 -- 172.21.15.82:0/901905579 learned_addr learned my addr 172.21.15.82:0/901905579 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:48.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.054+0000 7fe6f659c700 1 --2- 172.21.15.82:0/901905579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565315d615a0 0x565315d5f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.055+0000 7fe6f6d9d700 1 -- 172.21.15.82:0/901905579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565315d615a0 msgr2=0x565315d5f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:48.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.055+0000 7fe6f6d9d700 1 --2- 172.21.15.82:0/901905579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565315d615a0 0x565315d5f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.055+0000 7fe6f6d9d700 1 -- 172.21.15.82:0/901905579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565315d5fe00 msgr2=0x565315d602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:48.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.055+0000 7fe6f6d9d700 1 --2- 172.21.15.82:0/901905579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565315d5fe00 0x565315d602b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.063 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.055+0000 7fe6f6d9d700 1 -- 172.21.15.82:0/901905579 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565315addde0 con 0x565315d60c00 2023-09-09T23:13:48.063 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.055+0000 7fe6f759e700 1 --2- 172.21.15.82:0/901905579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565315d5fe00 0x565315d602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:48.063 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.055+0000 7fe6f659c700 1 --2- 172.21.15.82:0/901905579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565315d615a0 0x565315d5f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:48.063 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.055+0000 7fe6f6d9d700 1 --2- 172.21.15.82:0/901905579 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x565315d60c00 0x565315d61020 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80ca980 tx=0x7fe6e809a1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e64f445222e37414 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:48.063 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.055+0000 7fe6dd7fa700 1 -- 172.21.15.82:0/901905579 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e809a6b0 con 0x565315d60c00 2023-09-09T23:13:48.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.055+0000 7fe6dd7fa700 1 -- 172.21.15.82:0/901905579 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6e809e4e0 con 0x565315d60c00 2023-09-09T23:13:48.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.055+0000 7fe6dd7fa700 1 -- 172.21.15.82:0/901905579 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e809ab70 con 0x565315d60c00 2023-09-09T23:13:48.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.056+0000 7fe7093d4b80 1 -- 172.21.15.82:0/901905579 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x565315d60c00 msgr2=0x565315d61020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:48.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.056+0000 7fe7093d4b80 1 --2- 172.21.15.82:0/901905579 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x565315d60c00 0x565315d61020 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80ca980 tx=0x7fe6e809a1f0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.056+0000 7fe7093d4b80 1 -- 172.21.15.82:0/901905579 shutdown_connections 2023-09-09T23:13:48.065 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.056+0000 7fe7093d4b80 1 --2- 172.21.15.82:0/901905579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565315d5fe00 0x565315d602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.065 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.056+0000 7fe7093d4b80 1 --2- 172.21.15.82:0/901905579 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x565315d60c00 0x565315d61020 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.065 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.056+0000 7fe7093d4b80 1 --2- 172.21.15.82:0/901905579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565315d615a0 0x565315d5f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.065 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.056+0000 7fe7093d4b80 1 -- 172.21.15.82:0/901905579 >> 172.21.15.82:0/901905579 conn(0x565315cd4090 msgr2=0x565315d5a720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:48.065 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.056+0000 7fe7093d4b80 1 -- 172.21.15.82:0/901905579 shutdown_connections 2023-09-09T23:13:48.065 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.056+0000 7fe7093d4b80 1 -- 172.21.15.82:0/901905579 wait complete. 2023-09-09T23:13:48.066 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.056+0000 7fe7093d4b80 1 -- start start 2023-09-09T23:13:48.066 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.056+0000 7fe7093d4b80 1 Processor -- start 2023-09-09T23:13:48.066 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.057+0000 7fe7093d4b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565315d5fe00 0x565315d69260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.066 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.057+0000 7fe7093d4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565315d60c00 0x565315d697a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.066 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.057+0000 7fe7093d4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x565315d615a0 0x565315d707e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.067 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.057+0000 7fe7093d4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x565315c4fb70 con 0x565315d60c00 2023-09-09T23:13:48.067 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.057+0000 7fe7093d4b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x565315c4f870 con 0x565315d5fe00 2023-09-09T23:13:48.067 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.057+0000 7fe7093d4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x565315b33920 con 0x565315d615a0 2023-09-09T23:13:48.067 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.057+0000 7fe7093d4b80 10 client.0 _ll_register_callbacks cb 0x565315b99a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:48.067 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.057+0000 7fe6f659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565315d60c00 0x565315d697a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.057+0000 7fe6f6d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565315d5fe00 0x565315d69260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.057+0000 7fe6f659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565315d60c00 0x565315d697a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56606/0 (socket says 172.21.15.82:56606) 2023-09-09T23:13:48.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.057+0000 7fe6f659c700 1 -- 172.21.15.82:0/864055846 learned_addr learned my addr 172.21.15.82:0/864055846 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:48.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.057+0000 7fe6f759e700 1 --2- 172.21.15.82:0/864055846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x565315d615a0 0x565315d707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.057+0000 7fe6f759e700 1 -- 172.21.15.82:0/864055846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565315d5fe00 msgr2=0x565315d69260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:48.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.057+0000 7fe6f759e700 1 --2- 172.21.15.82:0/864055846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565315d5fe00 0x565315d69260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.057+0000 7fe6f759e700 1 -- 172.21.15.82:0/864055846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565315d60c00 msgr2=0x565315d697a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:48.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.057+0000 7fe6f759e700 1 --2- 172.21.15.82:0/864055846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565315d60c00 0x565315d697a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.057+0000 7fe6f759e700 1 -- 172.21.15.82:0/864055846 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f000e140 con 0x565315d615a0 2023-09-09T23:13:48.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.057+0000 7fe6f659c700 1 --2- 172.21.15.82:0/864055846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565315d60c00 0x565315d697a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:48.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.057+0000 7fe6f759e700 1 --2- 172.21.15.82:0/864055846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x565315d615a0 0x565315d707e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0009c10 tx=0x7fe6f0007dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc37a9a3e3e9ca41 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:48.070 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.057+0000 7fe6daff5700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-09T23:13:48.070 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.058+0000 7fe6daff5700 1 -- 172.21.15.82:0/864055846 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f001b070 con 0x565315d615a0 2023-09-09T23:13:48.070 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.058+0000 7fe6daff5700 1 -- 172.21.15.82:0/864055846 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fe6f0004bb0 con 0x565315d615a0 2023-09-09T23:13:48.070 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.058+0000 7fe7093d4b80 1 -- 172.21.15.82:0/864055846 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565315c094f0 con 0x565315d615a0 2023-09-09T23:13:48.070 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.058+0000 7fe6daff5700 1 -- 172.21.15.82:0/864055846 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6f0004d70 con 0x565315d615a0 2023-09-09T23:13:48.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.058+0000 7fe7093d4b80 10 client.25096 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:48.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.058+0000 7fe7093d4b80 1 -- 172.21.15.82:0/864055846 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fe6ec011440 con 0x565315d615a0 2023-09-09T23:13:48.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.058+0000 7fe6daff5700 1 -- 172.21.15.82:0/864055846 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6f0026020 con 0x565315d615a0 2023-09-09T23:13:48.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.058+0000 7fe6daff5700 1 -- 172.21.15.82:0/864055846 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe6f001e020 con 0x565315d615a0 2023-09-09T23:13:48.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.058+0000 7fe7093d4b80 10 client.25096 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:48.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.058+0000 7fe7093d4b80 10 client.25096 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-09-09T23:13:48.072 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.058+0000 7fe7093d4b80 20 client.25096 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:48.072 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.058+0000 7fe7093d4b80 1 --2- 172.21.15.82:0/864055846 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x565315d75160 0x565315d77580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.072 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.058+0000 7fe7093d4b80 4 client.25096 mds_command: new command op to 24542 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:13:48.072 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.058+0000 7fe7093d4b80 1 -- 172.21.15.82:0/864055846 --> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565315cc68a0 con 0x565315d75160 2023-09-09T23:13:48.072 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.061+0000 7fe6f6d9d700 1 --2- 172.21.15.82:0/864055846 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x565315d75160 0x565315d77580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.073 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.061+0000 7fe6f6d9d700 1 --2- 172.21.15.82:0/864055846 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x565315d75160 0x565315d77580 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6b8e51c26afbe04b server_cookie=4b0b687e87c643a4 in_seq=0 out_seq=0 2023-09-09T23:13:48.073 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.061+0000 7fe6daff5700 10 client.25096 ms_handle_connect on v2:172.21.15.98:6836/2019449840 2023-09-09T23:13:48.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.062+0000 7fe6daff5700 1 -- 172.21.15.82:0/864055846 <== mds.1 v2:172.21.15.98:6836/2019449840 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x565315cc68a0 con 0x565315d75160 2023-09-09T23:13:48.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.062+0000 7fe6daff5700 10 client.25096 handle_command_reply: tid=0 2023-09-09T23:13:48.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.062+0000 7fe7093d4b80 1 client.25096 shutdown 2023-09-09T23:13:48.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.062+0000 7fe7093d4b80 1 -- 172.21.15.82:0/864055846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x565315d615a0 msgr2=0x565315d707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:48.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.062+0000 7fe7093d4b80 1 --2- 172.21.15.82:0/864055846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x565315d615a0 0x565315d707e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0009c10 tx=0x7fe6f0007dd0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.062+0000 7fe7093d4b80 1 -- 172.21.15.82:0/864055846 shutdown_connections 2023-09-09T23:13:48.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.062+0000 7fe7093d4b80 1 --2- 172.21.15.82:0/864055846 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x565315d75160 0x565315d77580 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.062+0000 7fe7093d4b80 1 --2- 172.21.15.82:0/864055846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565315d5fe00 0x565315d69260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.062+0000 7fe6daff5700 0 client.25096 ms_handle_reset on v2:172.21.15.98:6836/2019449840 2023-09-09T23:13:48.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.062+0000 7fe7093d4b80 1 --2- 172.21.15.82:0/864055846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x565315d615a0 0x565315d707e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.062+0000 7fe7093d4b80 1 --2- 172.21.15.82:0/864055846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565315d60c00 0x565315d697a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.062+0000 7fe7093d4b80 1 -- 172.21.15.82:0/864055846 >> 172.21.15.82:0/864055846 conn(0x565315cd4090 msgr2=0x565315d59dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:48.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.062+0000 7fe7093d4b80 1 -- 172.21.15.82:0/864055846 shutdown_connections 2023-09-09T23:13:48.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.062+0000 7fe7093d4b80 1 -- 172.21.15.82:0/864055846 wait complete. 2023-09-09T23:13:48.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.062+0000 7fe7093d4b80 20 client.25096 trim_cache size 0 max 16384 2023-09-09T23:13:48.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.075+0000 7fe7093d4b80 1 Processor -- start 2023-09-09T23:13:48.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.075+0000 7fe7093d4b80 1 -- start start 2023-09-09T23:13:48.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.075+0000 7fe7093d4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x565315d5fe00 0x565315d5f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.075+0000 7fe7093d4b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565315d60c00 0x565315d707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.075+0000 7fe7093d4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565315d615a0 0x565315d71010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.075+0000 7fe7093d4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x565315d5b8c0 con 0x565315d615a0 2023-09-09T23:13:48.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.075+0000 7fe7093d4b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x565315cc68d0 con 0x565315d60c00 2023-09-09T23:13:48.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.075+0000 7fe7093d4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x565315d6a3e0 con 0x565315d5fe00 2023-09-09T23:13:48.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.075+0000 7fe6f759e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565315d615a0 0x565315d71010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.075+0000 7fe6f659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565315d60c00 0x565315d707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.075+0000 7fe6f6d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x565315d5fe00 0x565315d5f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.076+0000 7fe6f759e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565315d615a0 0x565315d71010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56616/0 (socket says 172.21.15.82:56616) 2023-09-09T23:13:48.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.076+0000 7fe6f659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565315d60c00 0x565315d707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:42176/0 (socket says 172.21.15.82:42176) 2023-09-09T23:13:48.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.076+0000 7fe6f759e700 1 -- 172.21.15.82:0/189972473 learned_addr learned my addr 172.21.15.82:0/189972473 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:48.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.076+0000 7fe6f659c700 1 -- 172.21.15.82:0/189972473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x565315d5fe00 msgr2=0x565315d5f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:48.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.076+0000 7fe6f659c700 1 --2- 172.21.15.82:0/189972473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x565315d5fe00 0x565315d5f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.076+0000 7fe6f659c700 1 -- 172.21.15.82:0/189972473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565315d615a0 msgr2=0x565315d71010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:48.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.076+0000 7fe6f659c700 1 --2- 172.21.15.82:0/189972473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565315d615a0 0x565315d71010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.085 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.076+0000 7fe6f659c700 1 -- 172.21.15.82:0/189972473 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f81a0a40 con 0x565315d60c00 2023-09-09T23:13:48.085 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.076+0000 7fe6f759e700 1 --2- 172.21.15.82:0/189972473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565315d615a0 0x565315d71010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:48.085 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.076+0000 7fe6f6d9d700 1 --2- 172.21.15.82:0/189972473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x565315d5fe00 0x565315d5f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:48.085 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.076+0000 7fe6f659c700 1 --2- 172.21.15.82:0/189972473 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565315d60c00 0x565315d707a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec009fd0 tx=0x7fe6ec011be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dc41a3ef41e3365 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:48.085 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.076+0000 7fe6ddffb700 1 -- 172.21.15.82:0/189972473 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec021070 con 0x565315d60c00 2023-09-09T23:13:48.085 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.076+0000 7fe6ddffb700 1 -- 172.21.15.82:0/189972473 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6ec017040 con 0x565315d60c00 2023-09-09T23:13:48.086 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.076+0000 7fe6ddffb700 1 -- 172.21.15.82:0/189972473 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec005280 con 0x565315d60c00 2023-09-09T23:13:48.086 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.077+0000 7fe7093d4b80 1 -- 172.21.15.82:0/189972473 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565315d60c00 msgr2=0x565315d707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:48.086 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.077+0000 7fe7093d4b80 1 --2- 172.21.15.82:0/189972473 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565315d60c00 0x565315d707a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec009fd0 tx=0x7fe6ec011be0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.086 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.077+0000 7fe7093d4b80 1 -- 172.21.15.82:0/189972473 shutdown_connections 2023-09-09T23:13:48.086 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.077+0000 7fe7093d4b80 1 --2- 172.21.15.82:0/189972473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565315d615a0 0x565315d71010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.077+0000 7fe7093d4b80 1 --2- 172.21.15.82:0/189972473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x565315d5fe00 0x565315d5f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.077+0000 7fe7093d4b80 1 --2- 172.21.15.82:0/189972473 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565315d60c00 0x565315d707a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.077+0000 7fe7093d4b80 1 -- 172.21.15.82:0/189972473 >> 172.21.15.82:0/189972473 conn(0x565315cd4090 msgr2=0x565315d601e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:48.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.077+0000 7fe7093d4b80 1 -- 172.21.15.82:0/189972473 shutdown_connections 2023-09-09T23:13:48.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.077+0000 7fe7093d4b80 1 -- 172.21.15.82:0/189972473 wait complete. 2023-09-09T23:13:48.088 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.077+0000 7fe7093d4b80 1 -- start start 2023-09-09T23:13:48.088 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.077+0000 7fe7093d4b80 1 Processor -- start 2023-09-09T23:13:48.088 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.078+0000 7fe7093d4b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565315d5fe00 0x565315d6e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.088 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.078+0000 7fe7093d4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565315d60c00 0x565315d6ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.088 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.078+0000 7fe7093d4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x565315d615a0 0x565315d6fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.078+0000 7fe7093d4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x565315d70300 con 0x565315d60c00 2023-09-09T23:13:48.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.078+0000 7fe7093d4b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x565315d70470 con 0x565315d5fe00 2023-09-09T23:13:48.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.078+0000 7fe7093d4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x565315d70630 con 0x565315d615a0 2023-09-09T23:13:48.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.078+0000 7fe7093d4b80 10 client.0 _ll_register_callbacks cb 0x7fe6f0009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:48.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.078+0000 7fe6f659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565315d60c00 0x565315d6ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.078+0000 7fe6f659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565315d60c00 0x565315d6ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56620/0 (socket says 172.21.15.82:56620) 2023-09-09T23:13:48.090 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.078+0000 7fe6f659c700 1 -- 172.21.15.82:0/3066530382 learned_addr learned my addr 172.21.15.82:0/3066530382 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:48.090 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.078+0000 7fe6f6d9d700 1 --2- 172.21.15.82:0/3066530382 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565315d5fe00 0x565315d6e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.090 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.078+0000 7fe6f759e700 1 --2- 172.21.15.82:0/3066530382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x565315d615a0 0x565315d6fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.090 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.078+0000 7fe6f659c700 1 -- 172.21.15.82:0/3066530382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x565315d615a0 msgr2=0x565315d6fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:48.090 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.078+0000 7fe6f659c700 1 --2- 172.21.15.82:0/3066530382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x565315d615a0 0x565315d6fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.091 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.078+0000 7fe6f659c700 1 -- 172.21.15.82:0/3066530382 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565315d5fe00 msgr2=0x565315d6e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:48.091 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.078+0000 7fe6f659c700 1 --2- 172.21.15.82:0/3066530382 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565315d5fe00 0x565315d6e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.091 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.078+0000 7fe6f659c700 1 -- 172.21.15.82:0/3066530382 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565315addde0 con 0x565315d60c00 2023-09-09T23:13:48.091 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.078+0000 7fe6f659c700 1 --2- 172.21.15.82:0/3066530382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565315d60c00 0x565315d6ee90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec003c70 tx=0x7fe6ec0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1590126af5126bb3 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:48.091 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.078+0000 7fe6daff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:48.092 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.078+0000 7fe6daff5700 1 -- 172.21.15.82:0/3066530382 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec021040 con 0x565315d60c00 2023-09-09T23:13:48.092 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.078+0000 7fe6daff5700 1 -- 172.21.15.82:0/3066530382 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fe6ec004950 con 0x565315d60c00 2023-09-09T23:13:48.092 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.079+0000 7fe7093d4b80 1 -- 172.21.15.82:0/3066530382 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6e809e4e0 con 0x565315d60c00 2023-09-09T23:13:48.092 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.079+0000 7fe6daff5700 1 -- 172.21.15.82:0/3066530382 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6ec0056f0 con 0x565315d60c00 2023-09-09T23:13:48.092 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.079+0000 7fe7093d4b80 10 client.15681 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:48.092 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.079+0000 7fe7093d4b80 1 -- 172.21.15.82:0/3066530382 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fe6ec011440 con 0x565315d60c00 2023-09-09T23:13:48.093 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.079+0000 7fe6daff5700 1 -- 172.21.15.82:0/3066530382 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6ec035020 con 0x565315d60c00 2023-09-09T23:13:48.093 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.079+0000 7fe6daff5700 1 -- 172.21.15.82:0/3066530382 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe6ec02d060 con 0x565315d60c00 2023-09-09T23:13:48.093 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.079+0000 7fe7093d4b80 10 client.15681 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:48.093 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.079+0000 7fe7093d4b80 10 client.15681 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-09-09T23:13:48.093 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.079+0000 7fe7093d4b80 20 client.15681 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:48.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.079+0000 7fe7093d4b80 1 --2- 172.21.15.82:0/3066530382 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x565315dca240 0x565315dcc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.079+0000 7fe7093d4b80 4 client.15681 mds_command: new command op to 24542 tid=0[{"prefix": "damage ls"}] 2023-09-09T23:13:48.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.079+0000 7fe7093d4b80 1 -- 172.21.15.82:0/3066530382 --> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x565315caa920 con 0x565315dca240 2023-09-09T23:13:48.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.079+0000 7fe6f6d9d700 1 --2- 172.21.15.82:0/3066530382 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x565315dca240 0x565315dcc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.080+0000 7fe6f6d9d700 1 --2- 172.21.15.82:0/3066530382 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x565315dca240 0x565315dcc660 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f385a26946c0baed server_cookie=e371a8ec684f53a7 in_seq=0 out_seq=0 2023-09-09T23:13:48.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.080+0000 7fe6daff5700 10 client.15681 ms_handle_connect on v2:172.21.15.98:6836/2019449840 2023-09-09T23:13:48.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.080+0000 7fe6daff5700 1 -- 172.21.15.82:0/3066530382 <== mds.1 v2:172.21.15.98:6836/2019449840 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x565315caa920 con 0x565315dca240 2023-09-09T23:13:48.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.080+0000 7fe6daff5700 10 client.15681 handle_command_reply: tid=0 2023-09-09T23:13:48.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.080+0000 7fe7093d4b80 1 client.15681 shutdown 2023-09-09T23:13:48.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.080+0000 7fe7093d4b80 1 -- 172.21.15.82:0/3066530382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565315d60c00 msgr2=0x565315d6ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:48.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.080+0000 7fe7093d4b80 1 --2- 172.21.15.82:0/3066530382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565315d60c00 0x565315d6ee90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec003c70 tx=0x7fe6ec0052f0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.080+0000 7fe7093d4b80 1 -- 172.21.15.82:0/3066530382 shutdown_connections 2023-09-09T23:13:48.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.080+0000 7fe7093d4b80 1 --2- 172.21.15.82:0/3066530382 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x565315dca240 0x565315dcc660 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.080+0000 7fe7093d4b80 1 --2- 172.21.15.82:0/3066530382 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565315d5fe00 0x565315d6e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.080+0000 7fe6daff5700 0 client.15681 ms_handle_reset on v2:172.21.15.98:6836/2019449840 2023-09-09T23:13:48.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.080+0000 7fe7093d4b80 1 --2- 172.21.15.82:0/3066530382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x565315d615a0 0x565315d6fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.080+0000 7fe7093d4b80 1 --2- 172.21.15.82:0/3066530382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565315d60c00 0x565315d6ee90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.080+0000 7fe7093d4b80 1 -- 172.21.15.82:0/3066530382 >> 172.21.15.82:0/3066530382 conn(0x565315cd4090 msgr2=0x565315d71010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:48.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.080+0000 7fe7093d4b80 1 -- 172.21.15.82:0/3066530382 shutdown_connections 2023-09-09T23:13:48.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.080+0000 7fe7093d4b80 1 -- 172.21.15.82:0/3066530382 wait complete. 2023-09-09T23:13:48.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.080+0000 7fe7093d4b80 20 client.15681 trim_cache size 0 max 16384 2023-09-09T23:13:48.098 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2023-09-09T23:13:48.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.081+0000 7fe6fd20e700 1 -- 172.21.15.82:0/2220843487 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fe6e00a0690 msgr2=0x7fe6e00a2b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-09T23:13:48.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.081+0000 7fe6fd20e700 1 --2- 172.21.15.82:0/2220843487 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fe6e00a0690 0x7fe6e00a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.081+0000 7fe6fd20e700 1 -- 172.21.15.82:0/2220843487 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f80feee0 msgr2=0x7fe6f81984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:48.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.081+0000 7fe6fd20e700 1 --2- 172.21.15.82:0/2220843487 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f80feee0 0x7fe6f81984f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80059d0 tx=0x7fe6e8004730 comp rx=0 tx=0).stop 2023-09-09T23:13:48.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.081+0000 7fe6fd20e700 1 -- 172.21.15.82:0/2220843487 shutdown_connections 2023-09-09T23:13:48.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.081+0000 7fe6fd20e700 1 --2- 172.21.15.82:0/2220843487 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fe6e00a0690 0x7fe6e00a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.081+0000 7fe6fd20e700 1 --2- 172.21.15.82:0/2220843487 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f80feee0 0x7fe6f81984f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.081+0000 7fe6fd20e700 1 --2- 172.21.15.82:0/2220843487 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6f8100200 0x7fe6f819faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.081+0000 7fe6fd20e700 1 --2- 172.21.15.82:0/2220843487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6f80ff840 0x7fe6f8198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.081+0000 7fe6fd20e700 1 -- 172.21.15.82:0/2220843487 >> 172.21.15.82:0/2220843487 conn(0x7fe6f80faa60 msgr2=0x7fe6f80fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:48.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.081+0000 7fe6fd20e700 1 -- 172.21.15.82:0/2220843487 shutdown_connections 2023-09-09T23:13:48.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.081+0000 7fe6fd20e700 1 -- 172.21.15.82:0/2220843487 wait complete. 2023-09-09T23:13:48.101 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:47 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:47.832+0000 7f0a735f8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-09T23:13:48.101 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:48 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:48.028+0000 7f0a735f8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-09T23:13:48.102 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-09T23:13:48.125 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.128+0000 7f3b0566e700 1 -- 172.21.15.82:0/676332639 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3aec0a0690 msgr2=0x7f3aec0a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1453069857 2023-09-09T23:13:48.125 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.128+0000 7f3b0566e700 1 --2- 172.21.15.82:0/676332639 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3aec0a0690 0x7f3aec0a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-09T23:13:48.136 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.139+0000 7f3b080d3700 1 -- 172.21.15.82:0/676332639 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f3b0004e3e0 con 0x7f3b000fe8e0 2023-09-09T23:13:48.142 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.145+0000 7f3af6ffd700 1 -- 172.21.15.82:0/676332639 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v45)=0 v45) v1 ==== 133+0+0 (secure 0 0 0) 0x7f3afc09a1b0 con 0x7f3b000fe8e0 2023-09-09T23:13:48.146 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.149+0000 7f3b080d3700 1 -- 172.21.15.82:0/676332639 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3aec0a0690 msgr2=0x7f3aec0a2b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-09T23:13:48.146 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.149+0000 7f3b080d3700 1 --2- 172.21.15.82:0/676332639 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3aec0a0690 0x7f3aec0a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.147 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.149+0000 7f3b080d3700 1 -- 172.21.15.82:0/676332639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b000fe8e0 msgr2=0x7f3b00196430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:48.147 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.149+0000 7f3b080d3700 1 --2- 172.21.15.82:0/676332639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b000fe8e0 0x7f3b00196430 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3afc0095c0 tx=0x7f3afc00f690 comp rx=0 tx=0).stop 2023-09-09T23:13:48.147 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.149+0000 7f3b080d3700 1 -- 172.21.15.82:0/676332639 shutdown_connections 2023-09-09T23:13:48.147 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.149+0000 7f3b080d3700 1 --2- 172.21.15.82:0/676332639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b001022f0 0x7f3b001977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.147 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.149+0000 7f3b080d3700 1 --2- 172.21.15.82:0/676332639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b000fe8e0 0x7f3b00196430 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.149+0000 7f3b080d3700 1 --2- 172.21.15.82:0/676332639 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f3aec0a0690 0x7f3aec0a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.149+0000 7f3b080d3700 1 --2- 172.21.15.82:0/676332639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b00101950 0x7f3b00196970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.149+0000 7f3b080d3700 1 -- 172.21.15.82:0/676332639 >> 172.21.15.82:0/676332639 conn(0x7f3b000fa1d0 msgr2=0x7f3b00104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:48.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.149+0000 7f3b080d3700 1 -- 172.21.15.82:0/676332639 shutdown_connections 2023-09-09T23:13:48.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.149+0000 7f3b080d3700 1 -- 172.21.15.82:0/676332639 wait complete. 2023-09-09T23:13:48.160 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M' 2023-09-09T23:13:48.251 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:13:47 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:13:47.963+0000 7f0b0093b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-09T23:13:48.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.258+0000 7fd5a9389700 1 -- 172.21.15.82:0/1843966828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5a4101080 msgr2=0x7fd5a4107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:48.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.258+0000 7fd5a9389700 1 --2- 172.21.15.82:0/1843966828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5a4101080 0x7fd5a4107930 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00b0d0 tx=0x7fd59c00b3e0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.258+0000 7fd5a9389700 1 -- 172.21.15.82:0/1843966828 shutdown_connections 2023-09-09T23:13:48.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.258+0000 7fd5a9389700 1 --2- 172.21.15.82:0/1843966828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5a40ff4f0 0x7fd5a40ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.258+0000 7fd5a9389700 1 --2- 172.21.15.82:0/1843966828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5a4101080 0x7fd5a4107930 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.258+0000 7fd5a9389700 1 --2- 172.21.15.82:0/1843966828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5a41006e0 0x7fd5a4100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.258+0000 7fd5a9389700 1 -- 172.21.15.82:0/1843966828 >> 172.21.15.82:0/1843966828 conn(0x7fd5a40faa60 msgr2=0x7fd5a40fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:48.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.258+0000 7fd5a9389700 1 -- 172.21.15.82:0/1843966828 shutdown_connections 2023-09-09T23:13:48.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.258+0000 7fd5a9389700 1 -- 172.21.15.82:0/1843966828 wait complete. 2023-09-09T23:13:48.259 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.258+0000 7fd5a9389700 1 Processor -- start 2023-09-09T23:13:48.259 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.259+0000 7fd5a9389700 1 -- start start 2023-09-09T23:13:48.259 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.259+0000 7fd5a9389700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5a40ff4f0 0x7fd5a41952b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.259 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.259+0000 7fd5a9389700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5a41006e0 0x7fd5a41957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.259 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.259+0000 7fd5a9389700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5a4101080 0x7fd5a419c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.260 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.259+0000 7fd5a9389700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a419ce30 con 0x7fd5a41006e0 2023-09-09T23:13:48.260 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.259+0000 7fd5a9389700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a419cf70 con 0x7fd5a4101080 2023-09-09T23:13:48.260 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.259+0000 7fd5a9389700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a419d100 con 0x7fd5a40ff4f0 2023-09-09T23:13:48.260 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.259+0000 7fd5a27fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5a41006e0 0x7fd5a41957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.260 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.259+0000 7fd5a27fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5a41006e0 0x7fd5a41957f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56634/0 (socket says 172.21.15.82:56634) 2023-09-09T23:13:48.261 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.259+0000 7fd5a27fc700 1 -- 172.21.15.82:0/1977998774 learned_addr learned my addr 172.21.15.82:0/1977998774 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:48.261 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.259+0000 7fd5a2ffd700 1 --2- 172.21.15.82:0/1977998774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5a40ff4f0 0x7fd5a41952b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.261 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.259+0000 7fd5a37fe700 1 --2- 172.21.15.82:0/1977998774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5a4101080 0x7fd5a419c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.261 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.259+0000 7fd5a2ffd700 1 -- 172.21.15.82:0/1977998774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5a4101080 msgr2=0x7fd5a419c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:48.261 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.259+0000 7fd5a2ffd700 1 --2- 172.21.15.82:0/1977998774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5a4101080 0x7fd5a419c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.261 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.259+0000 7fd5a2ffd700 1 -- 172.21.15.82:0/1977998774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5a41006e0 msgr2=0x7fd5a41957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:48.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.259+0000 7fd5a2ffd700 1 --2- 172.21.15.82:0/1977998774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5a41006e0 0x7fd5a41957f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.259+0000 7fd5a2ffd700 1 -- 172.21.15.82:0/1977998774 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd594009520 con 0x7fd5a40ff4f0 2023-09-09T23:13:48.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.259+0000 7fd5a37fe700 1 --2- 172.21.15.82:0/1977998774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5a4101080 0x7fd5a419c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:48.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.260+0000 7fd5a2ffd700 1 --2- 172.21.15.82:0/1977998774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5a40ff4f0 0x7fd5a41952b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd59400e9c0 tx=0x7fd59400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:48.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.260+0000 7fd58bfff700 1 -- 172.21.15.82:0/1977998774 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59400cb70 con 0x7fd5a40ff4f0 2023-09-09T23:13:48.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.260+0000 7fd5a9389700 1 -- 172.21.15.82:0/1977998774 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd59c009d00 con 0x7fd5a40ff4f0 2023-09-09T23:13:48.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.260+0000 7fd58bfff700 1 -- 172.21.15.82:0/1977998774 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd59400ee60 con 0x7fd5a40ff4f0 2023-09-09T23:13:48.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.260+0000 7fd58bfff700 1 -- 172.21.15.82:0/1977998774 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd594018680 con 0x7fd5a40ff4f0 2023-09-09T23:13:48.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.260+0000 7fd5a9389700 1 -- 172.21.15.82:0/1977998774 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a4067b90 con 0x7fd5a40ff4f0 2023-09-09T23:13:48.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.262+0000 7fd58bfff700 1 -- 172.21.15.82:0/1977998774 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 19) v1 ==== 135276+0+0 (secure 0 0 0) 0x7fd594010430 con 0x7fd5a40ff4f0 2023-09-09T23:13:48.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.262+0000 7fd58bfff700 1 --2- 172.21.15.82:0/1977998774 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd58c0a0690 0x7fd58c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.263+0000 7fd5a27fc700 1 -- 172.21.15.82:0/1977998774 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd58c0a0690 msgr2=0x7fd58c0a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1453069857 2023-09-09T23:13:48.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.263+0000 7fd58bfff700 1 -- 172.21.15.82:0/1977998774 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fd594014070 con 0x7fd5a40ff4f0 2023-09-09T23:13:48.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.263+0000 7fd5a27fc700 1 --2- 172.21.15.82:0/1977998774 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd58c0a0690 0x7fd58c0a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-09T23:13:48.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.265+0000 7fd5b554fb80 1 Processor -- start 2023-09-09T23:13:48.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.265+0000 7fd5b554fb80 1 -- start start 2023-09-09T23:13:48.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.265+0000 7fd5b554fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3a4907c00 0x55c3a4908020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.265+0000 7fd5b554fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3a49085a0 0x55c3a4906800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.265+0000 7fd5b554fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3a4906e00 0x55c3a49072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.265+0000 7fd5b554fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a485f730 con 0x55c3a4907c00 2023-09-09T23:13:48.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.265+0000 7fd5b554fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a485d050 con 0x55c3a4906e00 2023-09-09T23:13:48.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.265+0000 7fd5b554fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a4858940 con 0x55c3a49085a0 2023-09-09T23:13:48.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.265+0000 7fd5a2ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3a4907c00 0x55c3a4908020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.265+0000 7fd5a2ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3a4907c00 0x55c3a4908020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56642/0 (socket says 172.21.15.82:56642) 2023-09-09T23:13:48.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.265+0000 7fd5a2ffd700 1 -- 172.21.15.82:0/2217730557 learned_addr learned my addr 172.21.15.82:0/2217730557 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:48.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.265+0000 7fd5a27fc700 1 --2- 172.21.15.82:0/2217730557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3a49085a0 0x55c3a4906800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.265+0000 7fd5a37fe700 1 --2- 172.21.15.82:0/2217730557 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3a4906e00 0x55c3a49072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.265+0000 7fd5a2ffd700 1 -- 172.21.15.82:0/2217730557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3a49085a0 msgr2=0x55c3a4906800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:48.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.265+0000 7fd5a2ffd700 1 --2- 172.21.15.82:0/2217730557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3a49085a0 0x55c3a4906800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.265+0000 7fd5a2ffd700 1 -- 172.21.15.82:0/2217730557 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3a4906e00 msgr2=0x55c3a49072b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:48.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.265+0000 7fd5a2ffd700 1 --2- 172.21.15.82:0/2217730557 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3a4906e00 0x55c3a49072b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.265+0000 7fd5a2ffd700 1 -- 172.21.15.82:0/2217730557 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3a4684de0 con 0x55c3a4907c00 2023-09-09T23:13:48.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.265+0000 7fd5a37fe700 1 --2- 172.21.15.82:0/2217730557 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3a4906e00 0x55c3a49072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:48.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.266+0000 7fd5a2ffd700 1 --2- 172.21.15.82:0/2217730557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3a4907c00 0x55c3a4908020 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd5940961c0 tx=0x7fd594099520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529862b2da8f7c10 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:48.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.266+0000 7fd5897fa700 1 -- 172.21.15.82:0/2217730557 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5940990b0 con 0x55c3a4907c00 2023-09-09T23:13:48.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.266+0000 7fd5897fa700 1 -- 172.21.15.82:0/2217730557 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd594099e50 con 0x55c3a4907c00 2023-09-09T23:13:48.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.266+0000 7fd5897fa700 1 -- 172.21.15.82:0/2217730557 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59409d380 con 0x55c3a4907c00 2023-09-09T23:13:48.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.267+0000 7fd5b554fb80 1 -- 172.21.15.82:0/2217730557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3a4907c00 msgr2=0x55c3a4908020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:48.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.267+0000 7fd5b554fb80 1 --2- 172.21.15.82:0/2217730557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3a4907c00 0x55c3a4908020 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd5940961c0 tx=0x7fd594099520 comp rx=0 tx=0).stop 2023-09-09T23:13:48.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.267+0000 7fd5b554fb80 1 -- 172.21.15.82:0/2217730557 shutdown_connections 2023-09-09T23:13:48.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.267+0000 7fd5b554fb80 1 --2- 172.21.15.82:0/2217730557 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3a4906e00 0x55c3a49072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.267+0000 7fd5b554fb80 1 --2- 172.21.15.82:0/2217730557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3a4907c00 0x55c3a4908020 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.267+0000 7fd5b554fb80 1 --2- 172.21.15.82:0/2217730557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3a49085a0 0x55c3a4906800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.267+0000 7fd5b554fb80 1 -- 172.21.15.82:0/2217730557 >> 172.21.15.82:0/2217730557 conn(0x55c3a487b090 msgr2=0x55c3a4901720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:48.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.267+0000 7fd5b554fb80 1 -- 172.21.15.82:0/2217730557 shutdown_connections 2023-09-09T23:13:48.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.267+0000 7fd5b554fb80 1 -- 172.21.15.82:0/2217730557 wait complete. 2023-09-09T23:13:48.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.267+0000 7fd5b554fb80 1 -- start start 2023-09-09T23:13:48.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.267+0000 7fd5b554fb80 1 Processor -- start 2023-09-09T23:13:48.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.267+0000 7fd5b554fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3a4906e00 0x55c3a4910650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.267+0000 7fd5b554fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3a4907c00 0x55c3a4910b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.267+0000 7fd5b554fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3a49085a0 0x55c3a4917bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.267+0000 7fd5b554fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a47f6b70 con 0x55c3a49085a0 2023-09-09T23:13:48.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.267+0000 7fd5b554fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a47f6870 con 0x55c3a4907c00 2023-09-09T23:13:48.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.267+0000 7fd5b554fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a46da920 con 0x55c3a4906e00 2023-09-09T23:13:48.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.268+0000 7fd5b554fb80 10 client.0 _ll_register_callbacks cb 0x55c3a4740a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:48.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.268+0000 7fd5a2ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3a4906e00 0x55c3a4910650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.268+0000 7fd5a27fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3a4907c00 0x55c3a4910b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.268+0000 7fd5a27fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3a4907c00 0x55c3a4910b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:42214/0 (socket says 172.21.15.82:42214) 2023-09-09T23:13:48.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.268+0000 7fd5a2ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3a4906e00 0x55c3a4910650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:55162/0 (socket says 172.21.15.82:55162) 2023-09-09T23:13:48.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.268+0000 7fd5a37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3a49085a0 0x55c3a4917bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.268+0000 7fd5a27fc700 1 -- 172.21.15.82:0/1663265275 learned_addr learned my addr 172.21.15.82:0/1663265275 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:48.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.268+0000 7fd5a27fc700 1 -- 172.21.15.82:0/1663265275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3a4906e00 msgr2=0x55c3a4910650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:48.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.268+0000 7fd5a27fc700 1 --2- 172.21.15.82:0/1663265275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3a4906e00 0x55c3a4910650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.268+0000 7fd5a27fc700 1 -- 172.21.15.82:0/1663265275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3a49085a0 msgr2=0x55c3a4917bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:48.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.268+0000 7fd5a27fc700 1 --2- 172.21.15.82:0/1663265275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3a49085a0 0x55c3a4917bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.268+0000 7fd5a27fc700 1 -- 172.21.15.82:0/1663265275 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a4067b90 con 0x55c3a4907c00 2023-09-09T23:13:48.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.268+0000 7fd5a2ffd700 1 --2- 172.21.15.82:0/1663265275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3a4906e00 0x55c3a4910650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:48.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.268+0000 7fd5a27fc700 1 --2- 172.21.15.82:0/1663265275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3a4907c00 0x55c3a4910b90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd59800e680 tx=0x7fd59800e990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44d4e84d586bda75 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:48.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.268+0000 7fd586ff5700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-09T23:13:48.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.268+0000 7fd586ff5700 1 -- 172.21.15.82:0/1663265275 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598013040 con 0x55c3a4907c00 2023-09-09T23:13:48.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.268+0000 7fd586ff5700 1 -- 172.21.15.82:0/1663265275 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fd598004440 con 0x55c3a4907c00 2023-09-09T23:13:48.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.268+0000 7fd5b554fb80 1 -- 172.21.15.82:0/1663265275 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3a47b04f0 con 0x55c3a4907c00 2023-09-09T23:13:48.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.268+0000 7fd586ff5700 1 -- 172.21.15.82:0/1663265275 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd598003680 con 0x55c3a4907c00 2023-09-09T23:13:48.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.269+0000 7fd5b554fb80 10 client.25214 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:48.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.269+0000 7fd5b554fb80 1 -- 172.21.15.82:0/1663265275 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c3a466dc60 con 0x55c3a4907c00 2023-09-09T23:13:48.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.269+0000 7fd586ff5700 1 -- 172.21.15.82:0/1663265275 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd598028020 con 0x55c3a4907c00 2023-09-09T23:13:48.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.269+0000 7fd586ff5700 1 -- 172.21.15.82:0/1663265275 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd598019840 con 0x55c3a4907c00 2023-09-09T23:13:48.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.269+0000 7fd5b554fb80 10 client.25214 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:48.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.269+0000 7fd5b554fb80 10 client.25214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:13:48.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.269+0000 7fd5b554fb80 20 client.25214 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:48.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.269+0000 7fd5b554fb80 1 --2- 172.21.15.82:0/1663265275 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c3a491c580 0x55c3a491e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.269+0000 7fd5b554fb80 4 client.25214 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:13:48.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.269+0000 7fd5b554fb80 1 -- 172.21.15.82:0/1663265275 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3a486d8a0 con 0x55c3a491c580 2023-09-09T23:13:48.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.269+0000 7fd5a2ffd700 1 --2- 172.21.15.82:0/1663265275 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c3a491c580 0x55c3a491e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.269+0000 7fd586ff5700 10 client.25214 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:13:48.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.269+0000 7fd5a2ffd700 1 --2- 172.21.15.82:0/1663265275 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c3a491c580 0x55c3a491e9a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=396f014f67c2cdb1 server_cookie=49b0ddb3ce944292 in_seq=0 out_seq=0 2023-09-09T23:13:48.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.270+0000 7fd586ff5700 1 -- 172.21.15.82:0/1663265275 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c3a486d8a0 con 0x55c3a491c580 2023-09-09T23:13:48.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.270+0000 7fd586ff5700 10 client.25214 handle_command_reply: tid=0 2023-09-09T23:13:48.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.270+0000 7fd5b554fb80 1 client.25214 shutdown 2023-09-09T23:13:48.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.270+0000 7fd5b554fb80 1 -- 172.21.15.82:0/1663265275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3a4907c00 msgr2=0x55c3a4910b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:48.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.270+0000 7fd5b554fb80 1 --2- 172.21.15.82:0/1663265275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3a4907c00 0x55c3a4910b90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd59800e680 tx=0x7fd59800e990 comp rx=0 tx=0).stop 2023-09-09T23:13:48.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.270+0000 7fd5b554fb80 1 -- 172.21.15.82:0/1663265275 shutdown_connections 2023-09-09T23:13:48.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.270+0000 7fd5b554fb80 1 --2- 172.21.15.82:0/1663265275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3a49085a0 0x55c3a4917bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.270+0000 7fd5b554fb80 1 --2- 172.21.15.82:0/1663265275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3a4906e00 0x55c3a4910650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.270+0000 7fd5b554fb80 1 --2- 172.21.15.82:0/1663265275 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c3a491c580 0x55c3a491e9a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.270+0000 7fd5b554fb80 1 --2- 172.21.15.82:0/1663265275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3a4907c00 0x55c3a4910b90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.270+0000 7fd586ff5700 0 client.25214 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:13:48.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.270+0000 7fd5b554fb80 1 -- 172.21.15.82:0/1663265275 >> 172.21.15.82:0/1663265275 conn(0x55c3a487b090 msgr2=0x55c3a4900dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:48.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.270+0000 7fd5b554fb80 1 -- 172.21.15.82:0/1663265275 shutdown_connections 2023-09-09T23:13:48.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.270+0000 7fd5b554fb80 1 -- 172.21.15.82:0/1663265275 wait complete. 2023-09-09T23:13:48.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.271+0000 7fd5b554fb80 20 client.25214 trim_cache size 0 max 16384 2023-09-09T23:13:48.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.295+0000 7fd5b554fb80 1 Processor -- start 2023-09-09T23:13:48.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.296+0000 7fd5b554fb80 1 -- start start 2023-09-09T23:13:48.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.296+0000 7fd5b554fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3a4906e00 0x55c3a490afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.296+0000 7fd5b554fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3a4907c00 0x55c3a490b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.296+0000 7fd5b554fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3a49085a0 0x55c3a4917af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.296+0000 7fd5b554fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a490f950 con 0x55c3a49085a0 2023-09-09T23:13:48.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.296+0000 7fd5b554fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a490fb00 con 0x55c3a4907c00 2023-09-09T23:13:48.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.296+0000 7fd5b554fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a49101c0 con 0x55c3a4906e00 2023-09-09T23:13:48.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.296+0000 7fd5a37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3a49085a0 0x55c3a4917af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.296+0000 7fd5a2ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3a4906e00 0x55c3a490afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.296+0000 7fd5a37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3a49085a0 0x55c3a4917af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56658/0 (socket says 172.21.15.82:56658) 2023-09-09T23:13:48.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.296+0000 7fd5a37fe700 1 -- 172.21.15.82:0/3700003279 learned_addr learned my addr 172.21.15.82:0/3700003279 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:48.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.296+0000 7fd5a37fe700 1 -- 172.21.15.82:0/3700003279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3a4906e00 msgr2=0x55c3a490afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:48.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.296+0000 7fd5a37fe700 1 --2- 172.21.15.82:0/3700003279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3a4906e00 0x55c3a490afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.296+0000 7fd5a37fe700 1 -- 172.21.15.82:0/3700003279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3a4907c00 msgr2=0x55c3a490b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:48.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.296+0000 7fd5a37fe700 1 --2- 172.21.15.82:0/3700003279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3a4907c00 0x55c3a490b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.296+0000 7fd5a37fe700 1 -- 172.21.15.82:0/3700003279 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3a46ab250 con 0x55c3a49085a0 2023-09-09T23:13:48.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.296+0000 7fd5a2ffd700 1 --2- 172.21.15.82:0/3700003279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3a4906e00 0x55c3a490afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:48.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.296+0000 7fd5a37fe700 1 --2- 172.21.15.82:0/3700003279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3a49085a0 0x55c3a4917af0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd59c015000 tx=0x7fd59c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9fcaa8c0e51bc6f server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:48.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.297+0000 7fd589ffb700 1 -- 172.21.15.82:0/3700003279 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c00e040 con 0x55c3a49085a0 2023-09-09T23:13:48.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.297+0000 7fd589ffb700 1 -- 172.21.15.82:0/3700003279 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd59c00bca0 con 0x55c3a49085a0 2023-09-09T23:13:48.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.297+0000 7fd589ffb700 1 -- 172.21.15.82:0/3700003279 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c005760 con 0x55c3a49085a0 2023-09-09T23:13:48.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.297+0000 7fd5b554fb80 1 -- 172.21.15.82:0/3700003279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3a49085a0 msgr2=0x55c3a4917af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:48.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.297+0000 7fd5b554fb80 1 --2- 172.21.15.82:0/3700003279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3a49085a0 0x55c3a4917af0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd59c015000 tx=0x7fd59c009c30 comp rx=0 tx=0).stop 2023-09-09T23:13:48.303 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.297+0000 7fd5b554fb80 1 -- 172.21.15.82:0/3700003279 shutdown_connections 2023-09-09T23:13:48.303 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.297+0000 7fd5b554fb80 1 --2- 172.21.15.82:0/3700003279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3a49085a0 0x55c3a4917af0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.303 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.297+0000 7fd5b554fb80 1 --2- 172.21.15.82:0/3700003279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3a4906e00 0x55c3a490afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.303 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.297+0000 7fd5b554fb80 1 --2- 172.21.15.82:0/3700003279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3a4907c00 0x55c3a490b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.303 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.297+0000 7fd5b554fb80 1 -- 172.21.15.82:0/3700003279 >> 172.21.15.82:0/3700003279 conn(0x55c3a487b090 msgr2=0x55c3a4918340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:48.304 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.297+0000 7fd5b554fb80 1 -- 172.21.15.82:0/3700003279 shutdown_connections 2023-09-09T23:13:48.304 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.298+0000 7fd5b554fb80 1 -- 172.21.15.82:0/3700003279 wait complete. 2023-09-09T23:13:48.304 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.298+0000 7fd5b554fb80 1 -- start start 2023-09-09T23:13:48.304 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.298+0000 7fd5b554fb80 1 Processor -- start 2023-09-09T23:13:48.304 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.298+0000 7fd5b554fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3a4906e00 0x55c3a49179d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.298+0000 7fd5b554fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3a4907c00 0x55c3a4916050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.298+0000 7fd5b554fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3a49085a0 0x55c3a4916590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.298+0000 7fd5b554fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a4916c20 con 0x55c3a4906e00 2023-09-09T23:13:48.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.298+0000 7fd5b554fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a4916d90 con 0x55c3a4907c00 2023-09-09T23:13:48.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.298+0000 7fd5b554fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a49703a0 con 0x55c3a49085a0 2023-09-09T23:13:48.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.298+0000 7fd5b554fb80 10 client.0 _ll_register_callbacks cb 0x7fd598009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:48.306 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.298+0000 7fd5a2ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3a4906e00 0x55c3a49179d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.306 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.298+0000 7fd5a2ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3a4906e00 0x55c3a49179d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56660/0 (socket says 172.21.15.82:56660) 2023-09-09T23:13:48.306 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.298+0000 7fd5a2ffd700 1 -- 172.21.15.82:0/3713113322 learned_addr learned my addr 172.21.15.82:0/3713113322 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:48.306 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.298+0000 7fd5a27fc700 1 --2- 172.21.15.82:0/3713113322 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3a4907c00 0x55c3a4916050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.306 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.298+0000 7fd5a37fe700 1 --2- 172.21.15.82:0/3713113322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3a49085a0 0x55c3a4916590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.307 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.299+0000 7fd5a2ffd700 1 -- 172.21.15.82:0/3713113322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3a49085a0 msgr2=0x55c3a4916590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:48.307 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.299+0000 7fd5a2ffd700 1 --2- 172.21.15.82:0/3713113322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3a49085a0 0x55c3a4916590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.307 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.299+0000 7fd5a2ffd700 1 -- 172.21.15.82:0/3713113322 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3a4907c00 msgr2=0x55c3a4916050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:48.307 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.299+0000 7fd5a2ffd700 1 --2- 172.21.15.82:0/3713113322 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3a4907c00 0x55c3a4916050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.307 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.299+0000 7fd5a2ffd700 1 -- 172.21.15.82:0/3713113322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5940b7040 con 0x55c3a4906e00 2023-09-09T23:13:48.308 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.299+0000 7fd5a27fc700 1 --2- 172.21.15.82:0/3713113322 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3a4907c00 0x55c3a4916050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:48.308 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.299+0000 7fd5a2ffd700 1 --2- 172.21.15.82:0/3713113322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3a4906e00 0x55c3a49179d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd5940c9a60 tx=0x7fd59400efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab852648b719b9c server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:48.308 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.299+0000 7fd586ff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:48.308 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.299+0000 7fd586ff5700 1 -- 172.21.15.82:0/3713113322 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd594099750 con 0x55c3a4906e00 2023-09-09T23:13:48.308 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.299+0000 7fd586ff5700 1 -- 172.21.15.82:0/3713113322 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fd594097af0 con 0x55c3a4906e00 2023-09-09T23:13:48.308 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.299+0000 7fd5b554fb80 1 -- 172.21.15.82:0/3713113322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd594099e50 con 0x55c3a4906e00 2023-09-09T23:13:48.309 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.299+0000 7fd586ff5700 1 -- 172.21.15.82:0/3713113322 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5940990b0 con 0x55c3a4906e00 2023-09-09T23:13:48.309 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.299+0000 7fd5b554fb80 10 client.15699 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:48.309 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.299+0000 7fd5b554fb80 1 -- 172.21.15.82:0/3713113322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fd5a4108280 con 0x55c3a4906e00 2023-09-09T23:13:48.309 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.299+0000 7fd586ff5700 1 -- 172.21.15.82:0/3713113322 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd594099e50 con 0x55c3a4906e00 2023-09-09T23:13:48.309 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.300+0000 7fd586ff5700 1 -- 172.21.15.82:0/3713113322 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd594098c40 con 0x55c3a4906e00 2023-09-09T23:13:48.310 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.300+0000 7fd5b554fb80 10 client.15699 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:48.310 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.300+0000 7fd5b554fb80 10 client.15699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:13:48.310 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.300+0000 7fd5b554fb80 20 client.15699 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:48.310 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.300+0000 7fd5b554fb80 1 --2- 172.21.15.82:0/3713113322 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c3a4971a10 0x55c3a4973e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.310 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.300+0000 7fd5b554fb80 4 client.15699 mds_command: new command op to 24566 tid=0[{"prefix": "damage ls"}] 2023-09-09T23:13:48.311 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.300+0000 7fd5b554fb80 1 -- 172.21.15.82:0/3713113322 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c3a4851920 con 0x55c3a4971a10 2023-09-09T23:13:48.311 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.300+0000 7fd5a27fc700 1 --2- 172.21.15.82:0/3713113322 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c3a4971a10 0x55c3a4973e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.311 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.300+0000 7fd586ff5700 10 client.15699 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:13:48.311 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.300+0000 7fd5a27fc700 1 --2- 172.21.15.82:0/3713113322 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c3a4971a10 0x55c3a4973e30 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ece7bbbf1024fc7 server_cookie=d04ec57b05d66cb7 in_seq=0 out_seq=0 2023-09-09T23:13:48.311 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.301+0000 7fd586ff5700 1 -- 172.21.15.82:0/3713113322 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c3a4851920 con 0x55c3a4971a10 2023-09-09T23:13:48.312 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.301+0000 7fd586ff5700 10 client.15699 handle_command_reply: tid=0 2023-09-09T23:13:48.312 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.301+0000 7fd5b554fb80 1 client.15699 shutdown 2023-09-09T23:13:48.312 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.301+0000 7fd5b554fb80 1 -- 172.21.15.82:0/3713113322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3a4906e00 msgr2=0x55c3a49179d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:48.312 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.301+0000 7fd5b554fb80 1 --2- 172.21.15.82:0/3713113322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3a4906e00 0x55c3a49179d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd5940c9a60 tx=0x7fd59400efd0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.312 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.301+0000 7fd5b554fb80 1 -- 172.21.15.82:0/3713113322 shutdown_connections 2023-09-09T23:13:48.312 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.301+0000 7fd5b554fb80 1 --2- 172.21.15.82:0/3713113322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c3a49085a0 0x55c3a4916590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.313 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.301+0000 7fd5b554fb80 1 --2- 172.21.15.82:0/3713113322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3a4906e00 0x55c3a49179d0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.313 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.301+0000 7fd5b554fb80 1 --2- 172.21.15.82:0/3713113322 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c3a4971a10 0x55c3a4973e30 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.313 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.301+0000 7fd5b554fb80 1 --2- 172.21.15.82:0/3713113322 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3a4907c00 0x55c3a4916050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.313 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.301+0000 7fd5b554fb80 1 -- 172.21.15.82:0/3713113322 >> 172.21.15.82:0/3713113322 conn(0x55c3a487b090 msgr2=0x55c3a490afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:48.313 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.301+0000 7fd586ff5700 0 client.15699 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:13:48.314 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.301+0000 7fd5b554fb80 1 -- 172.21.15.82:0/3713113322 shutdown_connections 2023-09-09T23:13:48.314 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.301+0000 7fd5b554fb80 1 -- 172.21.15.82:0/3713113322 wait complete. 2023-09-09T23:13:48.314 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.301+0000 7fd5b554fb80 20 client.15699 trim_cache size 0 max 16384 2023-09-09T23:13:48.314 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2023-09-09T23:13:48.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.302+0000 7fd5a9389700 1 -- 172.21.15.82:0/1977998774 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd58c0a0690 msgr2=0x7fd58c0a2b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-09T23:13:48.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.302+0000 7fd5a9389700 1 --2- 172.21.15.82:0/1977998774 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd58c0a0690 0x7fd58c0a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.302+0000 7fd5a9389700 1 -- 172.21.15.82:0/1977998774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5a40ff4f0 msgr2=0x7fd5a41952b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:48.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.302+0000 7fd5a9389700 1 --2- 172.21.15.82:0/1977998774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5a40ff4f0 0x7fd5a41952b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd59400e9c0 tx=0x7fd59400ecd0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.302+0000 7fd5a9389700 1 -- 172.21.15.82:0/1977998774 shutdown_connections 2023-09-09T23:13:48.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.302+0000 7fd5a9389700 1 --2- 172.21.15.82:0/1977998774 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fd58c0a0690 0x7fd58c0a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.302+0000 7fd5a9389700 1 --2- 172.21.15.82:0/1977998774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5a4101080 0x7fd5a419c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.302+0000 7fd5a9389700 1 --2- 172.21.15.82:0/1977998774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5a40ff4f0 0x7fd5a41952b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.302+0000 7fd5a9389700 1 --2- 172.21.15.82:0/1977998774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5a41006e0 0x7fd5a41957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.302+0000 7fd5a9389700 1 -- 172.21.15.82:0/1977998774 >> 172.21.15.82:0/1977998774 conn(0x7fd5a40faa60 msgr2=0x7fd5a40fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:48.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.302+0000 7fd5a9389700 1 -- 172.21.15.82:0/1977998774 shutdown_connections 2023-09-09T23:13:48.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.302+0000 7fd5a9389700 1 -- 172.21.15.82:0/1977998774 wait complete. 2023-09-09T23:13:48.327 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:13:47 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:13:47.960+0000 7fc6bfd29200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-09T23:13:48.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.351+0000 7fcfaa16a700 1 -- 172.21.15.82:0/1954411106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfa41006f0 msgr2=0x7fcfa4100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:48.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.351+0000 7fcfaa16a700 1 --2- 172.21.15.82:0/1954411106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfa41006f0 0x7fcfa4100b50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fcf980099c0 tx=0x7fcf98009cd0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.351+0000 7fcfaa16a700 1 -- 172.21.15.82:0/1954411106 shutdown_connections 2023-09-09T23:13:48.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.351+0000 7fcfaa16a700 1 --2- 172.21.15.82:0/1954411106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfa4101090 0x7fcfa4107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.351+0000 7fcfaa16a700 1 --2- 172.21.15.82:0/1954411106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfa40ff500 0x7fcfa40ff920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.351+0000 7fcfaa16a700 1 --2- 172.21.15.82:0/1954411106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfa41006f0 0x7fcfa4100b50 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.351+0000 7fcfaa16a700 1 -- 172.21.15.82:0/1954411106 >> 172.21.15.82:0/1954411106 conn(0x7fcfa40faa50 msgr2=0x7fcfa40fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:48.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.351+0000 7fcfaa16a700 1 -- 172.21.15.82:0/1954411106 shutdown_connections 2023-09-09T23:13:48.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.351+0000 7fcfaa16a700 1 -- 172.21.15.82:0/1954411106 wait complete. 2023-09-09T23:13:48.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.351+0000 7fcfaa16a700 1 Processor -- start 2023-09-09T23:13:48.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.352+0000 7fcfaa16a700 1 -- start start 2023-09-09T23:13:48.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.352+0000 7fcfaa16a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfa40ff500 0x7fcfa419b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.352+0000 7fcfaa16a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfa41006f0 0x7fcfa419d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.352+0000 7fcfaa16a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfa4101090 0x7fcfa419ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.352+0000 7fcfaa16a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa4068940 con 0x7fcfa40ff500 2023-09-09T23:13:48.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.352+0000 7fcfaa16a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa41a0440 con 0x7fcfa41006f0 2023-09-09T23:13:48.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.352+0000 7fcfaa16a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa41a05b0 con 0x7fcfa4101090 2023-09-09T23:13:48.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.352+0000 7fcfa37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfa40ff500 0x7fcfa419b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.352+0000 7fcfa37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfa40ff500 0x7fcfa419b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56674/0 (socket says 172.21.15.82:56674) 2023-09-09T23:13:48.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.352+0000 7fcfa37fe700 1 -- 172.21.15.82:0/394847667 learned_addr learned my addr 172.21.15.82:0/394847667 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:48.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.352+0000 7fcfa2ffd700 1 --2- 172.21.15.82:0/394847667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfa41006f0 0x7fcfa419d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.352+0000 7fcfa3fff700 1 --2- 172.21.15.82:0/394847667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfa4101090 0x7fcfa419ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:48.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.352+0000 7fcfa37fe700 1 -- 172.21.15.82:0/394847667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfa4101090 msgr2=0x7fcfa419ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:48.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.352+0000 7fcfa37fe700 1 --2- 172.21.15.82:0/394847667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfa4101090 0x7fcfa419ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.352+0000 7fcfa37fe700 1 -- 172.21.15.82:0/394847667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfa41006f0 msgr2=0x7fcfa419d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:48.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.352+0000 7fcfa37fe700 1 --2- 172.21.15.82:0/394847667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfa41006f0 0x7fcfa419d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:48.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.352+0000 7fcfa37fe700 1 -- 172.21.15.82:0/394847667 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf980095f0 con 0x7fcfa40ff500 2023-09-09T23:13:48.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.352+0000 7fcfa2ffd700 1 --2- 172.21.15.82:0/394847667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfa41006f0 0x7fcfa419d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:48.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.352+0000 7fcfa37fe700 1 --2- 172.21.15.82:0/394847667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfa40ff500 0x7fcfa419b460 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400e970 tx=0x7fcf9400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:48.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.352+0000 7fcfa0ff9700 1 -- 172.21.15.82:0/394847667 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf94009790 con 0x7fcfa40ff500 2023-09-09T23:13:48.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.353+0000 7fcfa0ff9700 1 -- 172.21.15.82:0/394847667 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf9400c7e0 con 0x7fcfa40ff500 2023-09-09T23:13:48.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.353+0000 7fcfa0ff9700 1 -- 172.21.15.82:0/394847667 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf94003c70 con 0x7fcfa40ff500 2023-09-09T23:13:48.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.353+0000 7fcfaa16a700 1 -- 172.21.15.82:0/394847667 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa41a0890 con 0x7fcfa40ff500 2023-09-09T23:13:48.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.353+0000 7fcfaa16a700 1 -- 172.21.15.82:0/394847667 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa41a0d50 con 0x7fcfa40ff500 2023-09-09T23:13:48.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.354+0000 7fcfaa16a700 1 -- 172.21.15.82:0/394847667 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa4106430 con 0x7fcfa40ff500 2023-09-09T23:13:48.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.355+0000 7fcfa0ff9700 1 -- 172.21.15.82:0/394847667 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 19) v1 ==== 135276+0+0 (secure 0 0 0) 0x7fcf9400cc50 con 0x7fcfa40ff500 2023-09-09T23:13:48.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.356+0000 7fcfa0ff9700 1 --2- 172.21.15.82:0/394847667 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcf8c0a0690 0x7fcf8c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:48.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.356+0000 7fcfa2ffd700 1 -- 172.21.15.82:0/394847667 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcf8c0a0690 msgr2=0x7fcf8c0a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1453069857 2023-09-09T23:13:48.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.356+0000 7fcfa0ff9700 1 -- 172.21.15.82:0/394847667 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fcf94014070 con 0x7fcfa40ff500 2023-09-09T23:13:48.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.356+0000 7fcfa2ffd700 1 --2- 172.21.15.82:0/394847667 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcf8c0a0690 0x7fcf8c0a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-09T23:13:48.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.360+0000 7fcfa0ff9700 1 -- 172.21.15.82:0/394847667 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fcf94095a40 con 0x7fcfa40ff500 2023-09-09T23:13:48.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.557+0000 7fcfa2ffd700 1 -- 172.21.15.82:0/394847667 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcf8c0a0690 msgr2=0x7fcf8c0a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1453069857 2023-09-09T23:13:48.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.557+0000 7fcfa2ffd700 1 --2- 172.21.15.82:0/394847667 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcf8c0a0690 0x7fcf8c0a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-09T23:13:48.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.582+0000 7fcfaa16a700 1 -- 172.21.15.82:0/394847667 --> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfa402ce60 con 0x7fcf8c0a0690 2023-09-09T23:13:48.643 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:13:48 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:13:48.329+0000 7fc6bfd29200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-09T23:13:48.644 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:13:48 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:13:48.462+0000 7fc6bfd29200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-09T23:13:48.644 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:13:48 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:13:48.550+0000 7fc6bfd29200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-09T23:13:48.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:48 smithi170 ceph-mon[102832]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-09-09T23:13:48.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:48 smithi170 ceph-mon[102832]: mgrmap e19: x(active, since 7m), standbys: z, y 2023-09-09T23:13:48.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:48 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/501254384' entity='client.admin' 2023-09-09T23:13:48.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:48 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/676332639' entity='client.admin' 2023-09-09T23:13:48.645 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:48 smithi098 ceph-mon[103124]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-09-09T23:13:48.646 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:48 smithi098 ceph-mon[103124]: mgrmap e19: x(active, since 7m), standbys: z, y 2023-09-09T23:13:48.646 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:48 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/501254384' entity='client.admin' 2023-09-09T23:13:48.646 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:48 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/676332639' entity='client.admin' 2023-09-09T23:13:48.646 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:13:48 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:13:48.331+0000 7f0b0093b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-09T23:13:48.647 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:13:48 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:13:48.462+0000 7f0b0093b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-09T23:13:48.647 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:13:48 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:13:48.550+0000 7f0b0093b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-09T23:13:48.653 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:48 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:48.433+0000 7f0a735f8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-09T23:13:48.653 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:48 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:48.566+0000 7f0a735f8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-09T23:13:48.653 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:48 smithi082 ceph-mon[98694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-09-09T23:13:48.653 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:48 smithi082 ceph-mon[98694]: mgrmap e19: x(active, since 7m), standbys: z, y 2023-09-09T23:13:48.654 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:48 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/501254384' entity='client.admin' 2023-09-09T23:13:48.654 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:48 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/676332639' entity='client.admin' 2023-09-09T23:13:48.954 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:48 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:48.655+0000 7f0a735f8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-09T23:13:48.955 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:48 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:48.750+0000 7f0a735f8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-09T23:13:48.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.958+0000 7fcfa2ffd700 1 -- 172.21.15.82:0/394847667 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcf8c0a0690 msgr2=0x7fcf8c0a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1453069857 2023-09-09T23:13:48.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:48.958+0000 7fcfa2ffd700 1 --2- 172.21.15.82:0/394847667 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcf8c0a0690 0x7fcf8c0a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-09T23:13:49.001 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:13:48 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:13:48.646+0000 7f0b0093b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-09T23:13:49.319 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-09T23:13:49.320 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-09T23:13:49.425 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:13:48 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:13:48.646+0000 7fc6bfd29200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-09T23:13:49.425 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:13:49 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:13:49.066+0000 7fc6bfd29200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-09T23:13:49.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.450+0000 7f0bd6999700 1 -- 172.21.15.82:0/3014854571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bd0101820 msgr2=0x7f0bd0101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:49.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.450+0000 7f0bd6999700 1 --2- 172.21.15.82:0/3014854571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bd0101820 0x7f0bd0101c80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40099c0 tx=0x7f0bc4009cd0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.451+0000 7f0bd6999700 1 -- 172.21.15.82:0/3014854571 shutdown_connections 2023-09-09T23:13:49.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.451+0000 7f0bd6999700 1 --2- 172.21.15.82:0/3014854571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0bd01021c0 0x7f0bd0108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.451+0000 7f0bd6999700 1 --2- 172.21.15.82:0/3014854571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bd0100630 0x7f0bd0100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.451+0000 7f0bd6999700 1 --2- 172.21.15.82:0/3014854571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bd0101820 0x7f0bd0101c80 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.451+0000 7f0bd6999700 1 -- 172.21.15.82:0/3014854571 >> 172.21.15.82:0/3014854571 conn(0x7f0bd00fbb80 msgr2=0x7f0bd00fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:49.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.451+0000 7f0bd6999700 1 -- 172.21.15.82:0/3014854571 shutdown_connections 2023-09-09T23:13:49.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.451+0000 7f0bd6999700 1 -- 172.21.15.82:0/3014854571 wait complete. 2023-09-09T23:13:49.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.451+0000 7f0bd6999700 1 Processor -- start 2023-09-09T23:13:49.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.451+0000 7f0bd6999700 1 -- start start 2023-09-09T23:13:49.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.451+0000 7f0bd6999700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bd0100630 0x7f0bd01962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.452+0000 7f0bd6999700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0bd0101820 0x7f0bd0196820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.452+0000 7f0bd6999700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bd01021c0 0x7f0bd019d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.452+0000 7f0bd6999700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd019de60 con 0x7f0bd0100630 2023-09-09T23:13:49.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.452+0000 7f0bd6999700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd019dfa0 con 0x7f0bd0101820 2023-09-09T23:13:49.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.452+0000 7f0bd6999700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd019e130 con 0x7f0bd01021c0 2023-09-09T23:13:49.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.452+0000 7f0bcffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bd0100630 0x7f0bd01962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.452+0000 7f0bcffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bd0100630 0x7f0bd01962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56722/0 (socket says 172.21.15.82:56722) 2023-09-09T23:13:49.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.452+0000 7f0bcffff700 1 -- 172.21.15.82:0/338322842 learned_addr learned my addr 172.21.15.82:0/338322842 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:49.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.452+0000 7f0bcf7fe700 1 --2- 172.21.15.82:0/338322842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0bd0101820 0x7f0bd0196820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.452+0000 7f0bd4f36700 1 --2- 172.21.15.82:0/338322842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bd01021c0 0x7f0bd019d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.452+0000 7f0bcffff700 1 -- 172.21.15.82:0/338322842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bd01021c0 msgr2=0x7f0bd019d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:49.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.452+0000 7f0bcffff700 1 --2- 172.21.15.82:0/338322842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bd01021c0 0x7f0bd019d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.452+0000 7f0bcffff700 1 -- 172.21.15.82:0/338322842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0bd0101820 msgr2=0x7f0bd0196820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:49.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.452+0000 7f0bcffff700 1 --2- 172.21.15.82:0/338322842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0bd0101820 0x7f0bd0196820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.452+0000 7f0bcffff700 1 -- 172.21.15.82:0/338322842 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc40095f0 con 0x7f0bd0100630 2023-09-09T23:13:49.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.452+0000 7f0bcffff700 1 --2- 172.21.15.82:0/338322842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bd0100630 0x7f0bd01962e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0bc000e970 tx=0x7f0bc000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:49.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.452+0000 7f0bcd7fa700 1 -- 172.21.15.82:0/338322842 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0009790 con 0x7f0bd0100630 2023-09-09T23:13:49.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.452+0000 7f0bd6999700 1 -- 172.21.15.82:0/338322842 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd019e410 con 0x7f0bd0100630 2023-09-09T23:13:49.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.452+0000 7f0bcd7fa700 1 -- 172.21.15.82:0/338322842 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bc000c7e0 con 0x7f0bd0100630 2023-09-09T23:13:49.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.452+0000 7f0bcd7fa700 1 -- 172.21.15.82:0/338322842 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0003c70 con 0x7f0bd0100630 2023-09-09T23:13:49.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.452+0000 7f0bd6999700 1 -- 172.21.15.82:0/338322842 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd019e970 con 0x7f0bd0100630 2023-09-09T23:13:49.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.455+0000 7f0bcd7fa700 1 -- 172.21.15.82:0/338322842 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 19) v1 ==== 135276+0+0 (secure 0 0 0) 0x7f0bc000cc50 con 0x7f0bd0100630 2023-09-09T23:13:49.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.455+0000 7f0bcd7fa700 1 --2- 172.21.15.82:0/338322842 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0bb80a0690 0x7f0bb80a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.456+0000 7f0bcf7fe700 1 -- 172.21.15.82:0/338322842 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0bb80a0690 msgr2=0x7f0bb80a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1453069857 2023-09-09T23:13:49.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.456+0000 7f0bcd7fa700 1 -- 172.21.15.82:0/338322842 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f0bc0014070 con 0x7f0bd0100630 2023-09-09T23:13:49.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.456+0000 7f0bcf7fe700 1 --2- 172.21.15.82:0/338322842 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0bb80a0690 0x7f0bb80a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-09T23:13:49.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.456+0000 7f0be2b5fb80 1 Processor -- start 2023-09-09T23:13:49.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.456+0000 7f0be2b5fb80 1 -- start start 2023-09-09T23:13:49.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.456+0000 7f0be2b5fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563500c700f0 0x563500c70510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.456+0000 7f0be2b5fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563500c70a50 0x563500c6ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.456+0000 7f0be2b5fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563500c6f2b0 0x563500c6f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.456+0000 7f0be2b5fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563500bc37b0 con 0x563500c70a50 2023-09-09T23:13:49.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.456+0000 7f0be2b5fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563500bc10d0 con 0x563500c700f0 2023-09-09T23:13:49.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.456+0000 7f0be2b5fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563500bbc9c0 con 0x563500c6f2b0 2023-09-09T23:13:49.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.456+0000 7f0bcf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563500c70a50 0x563500c6ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.456+0000 7f0bcffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563500c700f0 0x563500c70510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.456+0000 7f0bcffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563500c700f0 0x563500c70510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:42294/0 (socket says 172.21.15.82:42294) 2023-09-09T23:13:49.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.456+0000 7f0bcffff700 1 -- 172.21.15.82:0/924451959 learned_addr learned my addr 172.21.15.82:0/924451959 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:49.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.456+0000 7f0bcf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563500c70a50 0x563500c6ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56732/0 (socket says 172.21.15.82:56732) 2023-09-09T23:13:49.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.456+0000 7f0bd4f36700 1 --2- 172.21.15.82:0/924451959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563500c6f2b0 0x563500c6f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.457+0000 7f0bd4f36700 1 -- 172.21.15.82:0/924451959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563500c700f0 msgr2=0x563500c70510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:49.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.457+0000 7f0bd4f36700 1 --2- 172.21.15.82:0/924451959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563500c700f0 0x563500c70510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.457+0000 7f0bd4f36700 1 -- 172.21.15.82:0/924451959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563500c70a50 msgr2=0x563500c6ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:49.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.457+0000 7f0bd4f36700 1 --2- 172.21.15.82:0/924451959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563500c70a50 0x563500c6ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.457+0000 7f0bd4f36700 1 -- 172.21.15.82:0/924451959 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635009e8e60 con 0x563500c6f2b0 2023-09-09T23:13:49.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.457+0000 7f0bd4f36700 1 --2- 172.21.15.82:0/924451959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563500c6f2b0 0x563500c6f730 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8011580 tx=0x7f0bc8009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc2fb5bf6fd0b56 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:49.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.457+0000 7f0bb67fc700 1 -- 172.21.15.82:0/924451959 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc800e040 con 0x563500c6f2b0 2023-09-09T23:13:49.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.457+0000 7f0bb67fc700 1 -- 172.21.15.82:0/924451959 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bc800f040 con 0x563500c6f2b0 2023-09-09T23:13:49.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.457+0000 7f0bb67fc700 1 -- 172.21.15.82:0/924451959 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8004490 con 0x563500c6f2b0 2023-09-09T23:13:49.465 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:49 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:49.171+0000 7f0a735f8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-09T23:13:49.468 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.458+0000 7f0be2b5fb80 1 -- 172.21.15.82:0/924451959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563500c6f2b0 msgr2=0x563500c6f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:49.468 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.458+0000 7f0be2b5fb80 1 --2- 172.21.15.82:0/924451959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563500c6f2b0 0x563500c6f730 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8011580 tx=0x7f0bc8009920 comp rx=0 tx=0).stop 2023-09-09T23:13:49.469 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.458+0000 7f0be2b5fb80 1 -- 172.21.15.82:0/924451959 shutdown_connections 2023-09-09T23:13:49.469 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.458+0000 7f0be2b5fb80 1 --2- 172.21.15.82:0/924451959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563500c700f0 0x563500c70510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.469 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.458+0000 7f0be2b5fb80 1 --2- 172.21.15.82:0/924451959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563500c6f2b0 0x563500c6f730 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.469 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.458+0000 7f0be2b5fb80 1 --2- 172.21.15.82:0/924451959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563500c70a50 0x563500c6ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.469 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.458+0000 7f0be2b5fb80 1 -- 172.21.15.82:0/924451959 >> 172.21.15.82:0/924451959 conn(0x563500bdf110 msgr2=0x563500c657a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:49.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.458+0000 7f0be2b5fb80 1 -- 172.21.15.82:0/924451959 shutdown_connections 2023-09-09T23:13:49.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.458+0000 7f0be2b5fb80 1 -- 172.21.15.82:0/924451959 wait complete. 2023-09-09T23:13:49.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.458+0000 7f0be2b5fb80 1 -- start start 2023-09-09T23:13:49.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.458+0000 7f0be2b5fb80 1 Processor -- start 2023-09-09T23:13:49.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.459+0000 7f0be2b5fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563500c6f2b0 0x563500c789d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.459+0000 7f0be2b5fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563500c700f0 0x563500c78f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.459+0000 7f0be2b5fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563500c70a50 0x563500c7ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.459+0000 7f0be2b5fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563500b5abf0 con 0x563500c700f0 2023-09-09T23:13:49.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.459+0000 7f0be2b5fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563500b5a8f0 con 0x563500c6f2b0 2023-09-09T23:13:49.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.459+0000 7f0be2b5fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563500a3e9a0 con 0x563500c70a50 2023-09-09T23:13:49.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.459+0000 7f0be2b5fb80 10 client.0 _ll_register_callbacks cb 0x563500aa4b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:49.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.459+0000 7f0bcffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563500c6f2b0 0x563500c789d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.459+0000 7f0bcf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563500c700f0 0x563500c78f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.459+0000 7f0bcffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563500c6f2b0 0x563500c789d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:42300/0 (socket says 172.21.15.82:42300) 2023-09-09T23:13:49.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.459+0000 7f0bd4f36700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563500c70a50 0x563500c7ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.459+0000 7f0bcffff700 1 -- 172.21.15.82:0/2686853054 learned_addr learned my addr 172.21.15.82:0/2686853054 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:49.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.459+0000 7f0bcf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563500c700f0 0x563500c78f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56738/0 (socket says 172.21.15.82:56738) 2023-09-09T23:13:49.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.459+0000 7f0bcf7fe700 1 -- 172.21.15.82:0/2686853054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563500c70a50 msgr2=0x563500c7ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:49.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.459+0000 7f0bcf7fe700 1 --2- 172.21.15.82:0/2686853054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563500c70a50 0x563500c7ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.459+0000 7f0bcf7fe700 1 -- 172.21.15.82:0/2686853054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563500c6f2b0 msgr2=0x563500c789d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:49.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.459+0000 7f0bcf7fe700 1 --2- 172.21.15.82:0/2686853054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563500c6f2b0 0x563500c789d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.459+0000 7f0bcf7fe700 1 -- 172.21.15.82:0/2686853054 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc8011230 con 0x563500c700f0 2023-09-09T23:13:49.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.459+0000 7f0bcffff700 1 --2- 172.21.15.82:0/2686853054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563500c6f2b0 0x563500c789d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:49.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.459+0000 7f0bcf7fe700 1 --2- 172.21.15.82:0/2686853054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563500c700f0 0x563500c78f10 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009fd0 tx=0x7f0bc4005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36779a687296d6e0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:49.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.459+0000 7f0baf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:49.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.459+0000 7f0baf7fe700 1 -- 172.21.15.82:0/2686853054 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4003e20 con 0x563500c700f0 2023-09-09T23:13:49.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.459+0000 7f0baf7fe700 1 -- 172.21.15.82:0/2686853054 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f0bc4005040 con 0x563500c700f0 2023-09-09T23:13:49.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.459+0000 7f0be2b5fb80 1 -- 172.21.15.82:0/2686853054 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563500b14570 con 0x563500c700f0 2023-09-09T23:13:49.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.460+0000 7f0baf7fe700 1 -- 172.21.15.82:0/2686853054 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bc4005200 con 0x563500c700f0 2023-09-09T23:13:49.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.460+0000 7f0be2b5fb80 10 client.15717 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:49.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.460+0000 7f0be2b5fb80 1 -- 172.21.15.82:0/2686853054 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5635009d1ce0 con 0x563500c700f0 2023-09-09T23:13:49.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.460+0000 7f0baf7fe700 1 -- 172.21.15.82:0/2686853054 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563500b14570 con 0x563500c700f0 2023-09-09T23:13:49.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.460+0000 7f0baf7fe700 1 -- 172.21.15.82:0/2686853054 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0bc4024bf0 con 0x563500c700f0 2023-09-09T23:13:49.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.460+0000 7f0be2b5fb80 10 client.15717 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:49.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.460+0000 7f0be2b5fb80 10 client.15717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:13:49.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.460+0000 7f0be2b5fb80 20 client.15717 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:49.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.460+0000 7f0be2b5fb80 1 --2- 172.21.15.82:0/2686853054 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x563500c84780 0x563500c86ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.477 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.460+0000 7f0be2b5fb80 4 client.15717 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:13:49.477 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.460+0000 7f0be2b5fb80 1 -- 172.21.15.82:0/2686853054 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563500bd1920 con 0x563500c84780 2023-09-09T23:13:49.477 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.460+0000 7f0bcffff700 1 --2- 172.21.15.82:0/2686853054 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x563500c84780 0x563500c86ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.477 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.461+0000 7f0baf7fe700 10 client.15717 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:13:49.477 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.461+0000 7f0bcffff700 1 --2- 172.21.15.82:0/2686853054 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x563500c84780 0x563500c86ba0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84d0de84c6407394 server_cookie=cb5d678414fbd93f in_seq=0 out_seq=0 2023-09-09T23:13:49.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.461+0000 7f0baf7fe700 1 -- 172.21.15.82:0/2686853054 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563500bd1920 con 0x563500c84780 2023-09-09T23:13:49.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.461+0000 7f0baf7fe700 10 client.15717 handle_command_reply: tid=0 2023-09-09T23:13:49.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.461+0000 7f0be2b5fb80 1 client.15717 shutdown 2023-09-09T23:13:49.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.462+0000 7f0be2b5fb80 1 -- 172.21.15.82:0/2686853054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563500c700f0 msgr2=0x563500c78f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:49.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.462+0000 7f0be2b5fb80 1 --2- 172.21.15.82:0/2686853054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563500c700f0 0x563500c78f10 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009fd0 tx=0x7f0bc4005c90 comp rx=0 tx=0).stop 2023-09-09T23:13:49.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.462+0000 7f0be2b5fb80 1 -- 172.21.15.82:0/2686853054 shutdown_connections 2023-09-09T23:13:49.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.462+0000 7f0be2b5fb80 1 --2- 172.21.15.82:0/2686853054 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x563500c84780 0x563500c86ba0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.462+0000 7f0be2b5fb80 1 --2- 172.21.15.82:0/2686853054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563500c6f2b0 0x563500c789d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.462+0000 7f0be2b5fb80 1 --2- 172.21.15.82:0/2686853054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563500c70a50 0x563500c7ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.462+0000 7f0baf7fe700 0 client.15717 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:13:49.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.462+0000 7f0be2b5fb80 1 --2- 172.21.15.82:0/2686853054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563500c700f0 0x563500c78f10 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.462+0000 7f0be2b5fb80 1 -- 172.21.15.82:0/2686853054 >> 172.21.15.82:0/2686853054 conn(0x563500bdf110 msgr2=0x563500c64e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:49.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.462+0000 7f0be2b5fb80 1 -- 172.21.15.82:0/2686853054 shutdown_connections 2023-09-09T23:13:49.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.462+0000 7f0be2b5fb80 1 -- 172.21.15.82:0/2686853054 wait complete. 2023-09-09T23:13:49.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.462+0000 7f0be2b5fb80 20 client.15717 trim_cache size 0 max 16384 2023-09-09T23:13:49.485 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.478+0000 7f0be2b5fb80 1 Processor -- start 2023-09-09T23:13:49.486 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.478+0000 7f0be2b5fb80 1 -- start start 2023-09-09T23:13:49.486 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.479+0000 7f0be2b5fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563500c6f2b0 0x563500c80780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.486 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.479+0000 7f0be2b5fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563500c700f0 0x563500c7ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.486 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.479+0000 7f0be2b5fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563500c70a50 0x563500c73330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.487 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.479+0000 7f0be2b5fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563500bd1920 con 0x563500c700f0 2023-09-09T23:13:49.487 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.479+0000 7f0be2b5fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563500b14570 con 0x563500c70a50 2023-09-09T23:13:49.487 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.479+0000 7f0be2b5fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563500c77e80 con 0x563500c6f2b0 2023-09-09T23:13:49.487 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.479+0000 7f0bcf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563500c700f0 0x563500c7ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.487 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.479+0000 7f0bcf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563500c700f0 0x563500c7ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56746/0 (socket says 172.21.15.82:56746) 2023-09-09T23:13:49.488 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.479+0000 7f0bcf7fe700 1 -- 172.21.15.82:0/1817835152 learned_addr learned my addr 172.21.15.82:0/1817835152 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:49.488 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.479+0000 7f0bcffff700 1 --2- 172.21.15.82:0/1817835152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563500c6f2b0 0x563500c80780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.488 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.479+0000 7f0bcf7fe700 1 -- 172.21.15.82:0/1817835152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563500c6f2b0 msgr2=0x563500c80780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:49.488 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.479+0000 7f0bcf7fe700 1 --2- 172.21.15.82:0/1817835152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563500c6f2b0 0x563500c80780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.488 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.479+0000 7f0bcf7fe700 1 -- 172.21.15.82:0/1817835152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563500c70a50 msgr2=0x563500c73330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:49.488 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.479+0000 7f0bcf7fe700 1 --2- 172.21.15.82:0/1817835152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563500c70a50 0x563500c73330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.489 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.479+0000 7f0bcf7fe700 1 -- 172.21.15.82:0/1817835152 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563500c8b810 con 0x563500c700f0 2023-09-09T23:13:49.489 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.479+0000 7f0bcf7fe700 1 --2- 172.21.15.82:0/1817835152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563500c700f0 0x563500c7ff10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009d10 tx=0x7f0bc4009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b548de42b0e55d5 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:49.489 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.479+0000 7f0bb6ffd700 1 -- 172.21.15.82:0/1817835152 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc400f5a0 con 0x563500c700f0 2023-09-09T23:13:49.489 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.480+0000 7f0bb6ffd700 1 -- 172.21.15.82:0/1817835152 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bc401e040 con 0x563500c700f0 2023-09-09T23:13:49.490 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.480+0000 7f0bb6ffd700 1 -- 172.21.15.82:0/1817835152 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4005700 con 0x563500c700f0 2023-09-09T23:13:49.490 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.480+0000 7f0be2b5fb80 1 -- 172.21.15.82:0/1817835152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563500c700f0 msgr2=0x563500c7ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:49.490 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.480+0000 7f0be2b5fb80 1 --2- 172.21.15.82:0/1817835152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563500c700f0 0x563500c7ff10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009d10 tx=0x7f0bc4009940 comp rx=0 tx=0).stop 2023-09-09T23:13:49.490 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.480+0000 7f0be2b5fb80 1 -- 172.21.15.82:0/1817835152 shutdown_connections 2023-09-09T23:13:49.490 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.480+0000 7f0be2b5fb80 1 --2- 172.21.15.82:0/1817835152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563500c70a50 0x563500c73330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.491 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.480+0000 7f0be2b5fb80 1 --2- 172.21.15.82:0/1817835152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563500c6f2b0 0x563500c80780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.491 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.480+0000 7f0be2b5fb80 1 --2- 172.21.15.82:0/1817835152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563500c700f0 0x563500c7ff10 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.491 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.480+0000 7f0be2b5fb80 1 -- 172.21.15.82:0/1817835152 >> 172.21.15.82:0/1817835152 conn(0x563500bdf110 msgr2=0x563500c6f690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:49.491 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.480+0000 7f0be2b5fb80 1 -- 172.21.15.82:0/1817835152 shutdown_connections 2023-09-09T23:13:49.491 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.480+0000 7f0be2b5fb80 1 -- 172.21.15.82:0/1817835152 wait complete. 2023-09-09T23:13:49.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.481+0000 7f0be2b5fb80 1 -- start start 2023-09-09T23:13:49.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.481+0000 7f0be2b5fb80 1 Processor -- start 2023-09-09T23:13:49.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.481+0000 7f0be2b5fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563500c6f2b0 0x563500c7fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.481+0000 7f0be2b5fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563500c700f0 0x563500c7c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.481+0000 7f0be2b5fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563500c70a50 0x563500c7cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.481+0000 7f0be2b5fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563500c7d260 con 0x563500c6f2b0 2023-09-09T23:13:49.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.481+0000 7f0be2b5fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563500c7d3d0 con 0x563500c70a50 2023-09-09T23:13:49.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.481+0000 7f0be2b5fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563500cd8610 con 0x563500c700f0 2023-09-09T23:13:49.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.481+0000 7f0be2b5fb80 10 client.0 _ll_register_callbacks cb 0x7f0bc400baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:49.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.481+0000 7f0bcffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563500c6f2b0 0x563500c7fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.481+0000 7f0bcffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563500c6f2b0 0x563500c7fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56750/0 (socket says 172.21.15.82:56750) 2023-09-09T23:13:49.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.481+0000 7f0bcffff700 1 -- 172.21.15.82:0/886392469 learned_addr learned my addr 172.21.15.82:0/886392469 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:49.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.481+0000 7f0bcf7fe700 1 --2- 172.21.15.82:0/886392469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563500c700f0 0x563500c7c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.481+0000 7f0bcffff700 1 -- 172.21.15.82:0/886392469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563500c700f0 msgr2=0x563500c7c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:49.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.481+0000 7f0bcffff700 1 --2- 172.21.15.82:0/886392469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563500c700f0 0x563500c7c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.481+0000 7f0bcffff700 1 -- 172.21.15.82:0/886392469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563500c70a50 msgr2=0x563500c7cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:49.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.481+0000 7f0bcffff700 1 --2- 172.21.15.82:0/886392469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563500c70a50 0x563500c7cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.495 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.481+0000 7f0bcffff700 1 -- 172.21.15.82:0/886392469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635009d1ce0 con 0x563500c6f2b0 2023-09-09T23:13:49.495 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.482+0000 7f0bcffff700 1 --2- 172.21.15.82:0/886392469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563500c6f2b0 0x563500c7fc70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0095fd0 tx=0x7f0bc0084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a4cda2cae15837 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:49.495 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.482+0000 7f0baf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:49.495 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.482+0000 7f0baf7fe700 1 -- 172.21.15.82:0/886392469 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc00a1620 con 0x563500c6f2b0 2023-09-09T23:13:49.496 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.482+0000 7f0baf7fe700 1 -- 172.21.15.82:0/886392469 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f0bc009d0b0 con 0x563500c6f2b0 2023-09-09T23:13:49.496 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.482+0000 7f0be2b5fb80 1 -- 172.21.15.82:0/886392469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0bc800f040 con 0x563500c6f2b0 2023-09-09T23:13:49.496 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.482+0000 7f0baf7fe700 1 -- 172.21.15.82:0/886392469 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bc00651a0 con 0x563500c6f2b0 2023-09-09T23:13:49.496 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.482+0000 7f0be2b5fb80 10 client.15723 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:49.496 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.482+0000 7f0be2b5fb80 1 -- 172.21.15.82:0/886392469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563500c6ee70 con 0x563500c6f2b0 2023-09-09T23:13:49.496 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.482+0000 7f0baf7fe700 1 -- 172.21.15.82:0/886392469 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bc800f040 con 0x563500c6f2b0 2023-09-09T23:13:49.497 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.482+0000 7f0baf7fe700 1 -- 172.21.15.82:0/886392469 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0bc0099930 con 0x563500c6f2b0 2023-09-09T23:13:49.497 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.482+0000 7f0be2b5fb80 10 client.15723 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:49.497 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.482+0000 7f0be2b5fb80 10 client.15723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:13:49.497 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.482+0000 7f0be2b5fb80 20 client.15723 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:49.497 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.482+0000 7f0be2b5fb80 1 --2- 172.21.15.82:0/886392469 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x563500cda020 0x563500cdc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.498 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.482+0000 7f0be2b5fb80 4 client.15723 mds_command: new command op to 24566 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-09T23:13:49.498 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.482+0000 7f0be2b5fb80 1 -- 172.21.15.82:0/886392469 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563500bb59a0 con 0x563500cda020 2023-09-09T23:13:49.498 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.482+0000 7f0bcf7fe700 1 --2- 172.21.15.82:0/886392469 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x563500cda020 0x563500cdc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.498 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.483+0000 7f0baf7fe700 10 client.15723 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:13:49.498 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.483+0000 7f0bcf7fe700 1 --2- 172.21.15.82:0/886392469 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x563500cda020 0x563500cdc440 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f312a0e9cbbe0d1c server_cookie=6055aeee314a05db in_seq=0 out_seq=0 2023-09-09T23:13:49.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.484+0000 7f0baf7fe700 1 -- 172.21.15.82:0/886392469 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563500bb59a0 con 0x563500cda020 2023-09-09T23:13:49.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.484+0000 7f0baf7fe700 10 client.15723 handle_command_reply: tid=0 2023-09-09T23:13:49.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.484+0000 7f0be2b5fb80 1 client.15723 shutdown 2023-09-09T23:13:49.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.484+0000 7f0be2b5fb80 1 -- 172.21.15.82:0/886392469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563500c6f2b0 msgr2=0x563500c7fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:49.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.484+0000 7f0be2b5fb80 1 --2- 172.21.15.82:0/886392469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563500c6f2b0 0x563500c7fc70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0095fd0 tx=0x7f0bc0084040 comp rx=0 tx=0).stop 2023-09-09T23:13:49.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.484+0000 7f0be2b5fb80 1 -- 172.21.15.82:0/886392469 shutdown_connections 2023-09-09T23:13:49.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.484+0000 7f0be2b5fb80 1 --2- 172.21.15.82:0/886392469 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x563500cda020 0x563500cdc440 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.484+0000 7f0be2b5fb80 1 --2- 172.21.15.82:0/886392469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563500c70a50 0x563500c7cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.484+0000 7f0baf7fe700 0 client.15723 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:13:49.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.484+0000 7f0be2b5fb80 1 --2- 172.21.15.82:0/886392469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563500c6f2b0 0x563500c7fc70 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.484+0000 7f0be2b5fb80 1 --2- 172.21.15.82:0/886392469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563500c700f0 0x563500c7c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.484+0000 7f0be2b5fb80 1 -- 172.21.15.82:0/886392469 >> 172.21.15.82:0/886392469 conn(0x563500bdf110 msgr2=0x563500c80780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:49.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.484+0000 7f0be2b5fb80 1 -- 172.21.15.82:0/886392469 shutdown_connections 2023-09-09T23:13:49.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.485+0000 7f0be2b5fb80 1 -- 172.21.15.82:0/886392469 wait complete. 2023-09-09T23:13:49.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.485+0000 7f0be2b5fb80 20 client.15723 trim_cache size 0 max 16384 2023-09-09T23:13:49.501 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:13:49.502 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2023-09-09T23:13:49.502 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "86192229-b7c8-4fe3-8787-9b91e68b875f", 2023-09-09T23:13:49.502 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2023-09-09T23:13:49.502 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:13:49.504 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.485+0000 7f0bd6999700 1 -- 172.21.15.82:0/338322842 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0bb80a0690 msgr2=0x7f0bb80a2b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-09T23:13:49.504 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.485+0000 7f0bd6999700 1 --2- 172.21.15.82:0/338322842 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0bb80a0690 0x7f0bb80a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.504 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.485+0000 7f0bd6999700 1 -- 172.21.15.82:0/338322842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bd0100630 msgr2=0x7f0bd01962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:49.504 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.485+0000 7f0bd6999700 1 --2- 172.21.15.82:0/338322842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bd0100630 0x7f0bd01962e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0bc000e970 tx=0x7f0bc000ec80 comp rx=0 tx=0).stop 2023-09-09T23:13:49.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.485+0000 7f0bd6999700 1 -- 172.21.15.82:0/338322842 shutdown_connections 2023-09-09T23:13:49.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.485+0000 7f0bd6999700 1 --2- 172.21.15.82:0/338322842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bd01021c0 0x7f0bd019d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.485+0000 7f0bd6999700 1 --2- 172.21.15.82:0/338322842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bd0100630 0x7f0bd01962e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.485+0000 7f0bd6999700 1 --2- 172.21.15.82:0/338322842 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f0bb80a0690 0x7f0bb80a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.506 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.485+0000 7f0bd6999700 1 --2- 172.21.15.82:0/338322842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0bd0101820 0x7f0bd0196820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.506 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.485+0000 7f0bd6999700 1 -- 172.21.15.82:0/338322842 >> 172.21.15.82:0/338322842 conn(0x7f0bd00fbb80 msgr2=0x7f0bd00fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:49.506 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.485+0000 7f0bd6999700 1 -- 172.21.15.82:0/338322842 shutdown_connections 2023-09-09T23:13:49.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.485+0000 7f0bd6999700 1 -- 172.21.15.82:0/338322842 wait complete. 2023-09-09T23:13:49.507 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:13:49.508 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:13:49 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:13:49.066+0000 7f0b0093b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-09T23:13:49.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.664+0000 7f3744942700 1 -- 172.21.15.82:0/255707401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37401021a0 msgr2=0x7f3740108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:49.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.664+0000 7f3744942700 1 --2- 172.21.15.82:0/255707401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37401021a0 0x7f3740108a50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f373800b0d0 tx=0x7f373800b3e0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.664+0000 7f3744942700 1 -- 172.21.15.82:0/255707401 shutdown_connections 2023-09-09T23:13:49.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.664+0000 7f3744942700 1 --2- 172.21.15.82:0/255707401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3740100610 0x7f3740100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.664+0000 7f3744942700 1 --2- 172.21.15.82:0/255707401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37401021a0 0x7f3740108a50 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.664+0000 7f3744942700 1 --2- 172.21.15.82:0/255707401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3740101800 0x7f3740101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.664+0000 7f3744942700 1 -- 172.21.15.82:0/255707401 >> 172.21.15.82:0/255707401 conn(0x7f37400fbb80 msgr2=0x7f37400fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:49.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.664+0000 7f3744942700 1 -- 172.21.15.82:0/255707401 shutdown_connections 2023-09-09T23:13:49.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.664+0000 7f3744942700 1 -- 172.21.15.82:0/255707401 wait complete. 2023-09-09T23:13:49.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.665+0000 7f3744942700 1 Processor -- start 2023-09-09T23:13:49.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.665+0000 7f3744942700 1 -- start start 2023-09-09T23:13:49.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.665+0000 7f3744942700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3740100610 0x7f37401963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.665+0000 7f3744942700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3740101800 0x7f37401968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.665+0000 7f3744942700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37401021a0 0x7f374019d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.665+0000 7f3744942700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374019df30 con 0x7f3740101800 2023-09-09T23:13:49.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.665+0000 7f3744942700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374019e070 con 0x7f3740100610 2023-09-09T23:13:49.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.665+0000 7f3744942700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374019e200 con 0x7f37401021a0 2023-09-09T23:13:49.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.665+0000 7f373dd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3740101800 0x7f37401968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.665+0000 7f373dd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3740101800 0x7f37401968f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56766/0 (socket says 172.21.15.82:56766) 2023-09-09T23:13:49.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.665+0000 7f373dd9b700 1 -- 172.21.15.82:0/1259773254 learned_addr learned my addr 172.21.15.82:0/1259773254 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:49.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.665+0000 7f373e59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3740100610 0x7f37401963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.665+0000 7f373ed9d700 1 --2- 172.21.15.82:0/1259773254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37401021a0 0x7f374019d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.666+0000 7f373dd9b700 1 -- 172.21.15.82:0/1259773254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37401021a0 msgr2=0x7f374019d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:49.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.666+0000 7f373dd9b700 1 --2- 172.21.15.82:0/1259773254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37401021a0 0x7f374019d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.666+0000 7f373dd9b700 1 -- 172.21.15.82:0/1259773254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3740100610 msgr2=0x7f37401963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:49.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.666+0000 7f373dd9b700 1 --2- 172.21.15.82:0/1259773254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3740100610 0x7f37401963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.666+0000 7f373dd9b700 1 -- 172.21.15.82:0/1259773254 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3738009d00 con 0x7f3740101800 2023-09-09T23:13:49.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.666+0000 7f373e59c700 1 --2- 172.21.15.82:0/1259773254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3740100610 0x7f37401963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:49.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.666+0000 7f373dd9b700 1 --2- 172.21.15.82:0/1259773254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3740101800 0x7f37401968f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f373400d980 tx=0x7f373400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:49.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.666+0000 7f372f7fe700 1 -- 172.21.15.82:0/1259773254 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3734004220 con 0x7f3740101800 2023-09-09T23:13:49.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.666+0000 7f3744942700 1 -- 172.21.15.82:0/1259773254 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f374019e4e0 con 0x7f3740101800 2023-09-09T23:13:49.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.666+0000 7f372f7fe700 1 -- 172.21.15.82:0/1259773254 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3734007640 con 0x7f3740101800 2023-09-09T23:13:49.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.666+0000 7f372f7fe700 1 -- 172.21.15.82:0/1259773254 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3734005690 con 0x7f3740101800 2023-09-09T23:13:49.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.667+0000 7f3744942700 1 -- 172.21.15.82:0/1259773254 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f374019ea40 con 0x7f3740101800 2023-09-09T23:13:49.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.668+0000 7f372f7fe700 1 -- 172.21.15.82:0/1259773254 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 19) v1 ==== 135276+0+0 (secure 0 0 0) 0x7f3734004750 con 0x7f3740101800 2023-09-09T23:13:49.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.669+0000 7f372f7fe700 1 --2- 172.21.15.82:0/1259773254 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f37280a0690 0x7f37280a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.669+0000 7f373e59c700 1 -- 172.21.15.82:0/1259773254 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f37280a0690 msgr2=0x7f37280a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1453069857 2023-09-09T23:13:49.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.669+0000 7f372f7fe700 1 -- 172.21.15.82:0/1259773254 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f37340cb6e0 con 0x7f3740101800 2023-09-09T23:13:49.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.669+0000 7f373e59c700 1 --2- 172.21.15.82:0/1259773254 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f37280a0690 0x7f37280a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-09T23:13:49.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.670+0000 7f3750b08b80 1 Processor -- start 2023-09-09T23:13:49.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.670+0000 7f3750b08b80 1 -- start start 2023-09-09T23:13:49.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.670+0000 7f3750b08b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561852f65c00 0x561852f66020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.670+0000 7f3750b08b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561852f665a0 0x561852f64800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.670+0000 7f3750b08b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561852f64e00 0x561852f652b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.670+0000 7f3750b08b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561852ebd730 con 0x561852f65c00 2023-09-09T23:13:49.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.670+0000 7f3750b08b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561852ebb050 con 0x561852f64e00 2023-09-09T23:13:49.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.670+0000 7f3750b08b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561852eb6940 con 0x561852f665a0 2023-09-09T23:13:49.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.671+0000 7f373e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561852f65c00 0x561852f66020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.671+0000 7f373e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561852f65c00 0x561852f66020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56772/0 (socket says 172.21.15.82:56772) 2023-09-09T23:13:49.678 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.671+0000 7f373e59c700 1 -- 172.21.15.82:0/3212321974 learned_addr learned my addr 172.21.15.82:0/3212321974 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:49.678 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.671+0000 7f373ed9d700 1 --2- 172.21.15.82:0/3212321974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561852f64e00 0x561852f652b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.678 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.671+0000 7f373dd9b700 1 --2- 172.21.15.82:0/3212321974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561852f665a0 0x561852f64800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.678 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.671+0000 7f373e59c700 1 -- 172.21.15.82:0/3212321974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561852f665a0 msgr2=0x561852f64800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:49.678 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.671+0000 7f373e59c700 1 --2- 172.21.15.82:0/3212321974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561852f665a0 0x561852f64800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.671+0000 7f373e59c700 1 -- 172.21.15.82:0/3212321974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561852f64e00 msgr2=0x561852f652b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:49.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.671+0000 7f373e59c700 1 --2- 172.21.15.82:0/3212321974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561852f64e00 0x561852f652b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.671+0000 7f373e59c700 1 -- 172.21.15.82:0/3212321974 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561852ce2de0 con 0x561852f65c00 2023-09-09T23:13:49.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.671+0000 7f373ed9d700 1 --2- 172.21.15.82:0/3212321974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561852f64e00 0x561852f652b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:49.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.671+0000 7f373e59c700 1 --2- 172.21.15.82:0/3212321974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561852f65c00 0x561852f66020 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f37300117e0 tx=0x7f3730011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b64b1b79227a0d server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:49.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.671+0000 7f372cff9700 1 -- 172.21.15.82:0/3212321974 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3730004830 con 0x561852f65c00 2023-09-09T23:13:49.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.671+0000 7f372cff9700 1 -- 172.21.15.82:0/3212321974 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3730004e50 con 0x561852f65c00 2023-09-09T23:13:49.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.671+0000 7f372cff9700 1 -- 172.21.15.82:0/3212321974 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373000f770 con 0x561852f65c00 2023-09-09T23:13:49.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.672+0000 7f3750b08b80 1 -- 172.21.15.82:0/3212321974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561852f65c00 msgr2=0x561852f66020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:49.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.672+0000 7f3750b08b80 1 --2- 172.21.15.82:0/3212321974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561852f65c00 0x561852f66020 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f37300117e0 tx=0x7f3730011af0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.672+0000 7f3750b08b80 1 -- 172.21.15.82:0/3212321974 shutdown_connections 2023-09-09T23:13:49.681 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.672+0000 7f3750b08b80 1 --2- 172.21.15.82:0/3212321974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561852f64e00 0x561852f652b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.681 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.672+0000 7f3750b08b80 1 --2- 172.21.15.82:0/3212321974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561852f65c00 0x561852f66020 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.681 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.672+0000 7f3750b08b80 1 --2- 172.21.15.82:0/3212321974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561852f665a0 0x561852f64800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.681 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.672+0000 7f3750b08b80 1 -- 172.21.15.82:0/3212321974 >> 172.21.15.82:0/3212321974 conn(0x561852ed9090 msgr2=0x561852f5f720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:49.681 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.672+0000 7f3750b08b80 1 -- 172.21.15.82:0/3212321974 shutdown_connections 2023-09-09T23:13:49.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.672+0000 7f3750b08b80 1 -- 172.21.15.82:0/3212321974 wait complete. 2023-09-09T23:13:49.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.672+0000 7f3750b08b80 1 -- start start 2023-09-09T23:13:49.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.673+0000 7f3750b08b80 1 Processor -- start 2023-09-09T23:13:49.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.673+0000 7f3750b08b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561852f64e00 0x561852f6e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.673+0000 7f3750b08b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561852f65c00 0x561852f6eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.683 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.673+0000 7f3750b08b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561852f665a0 0x561852f75b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.683 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.673+0000 7f3750b08b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561852e54b70 con 0x561852f64e00 2023-09-09T23:13:49.683 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.673+0000 7f3750b08b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561852e54870 con 0x561852f665a0 2023-09-09T23:13:49.683 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.673+0000 7f3750b08b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561852d38920 con 0x561852f65c00 2023-09-09T23:13:49.683 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.673+0000 7f3750b08b80 10 client.0 _ll_register_callbacks cb 0x561852d9ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:49.683 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.673+0000 7f373e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561852f64e00 0x561852f6e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.684 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.673+0000 7f373e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561852f64e00 0x561852f6e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56778/0 (socket says 172.21.15.82:56778) 2023-09-09T23:13:49.684 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.673+0000 7f373e59c700 1 -- 172.21.15.82:0/1245498943 learned_addr learned my addr 172.21.15.82:0/1245498943 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:49.684 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.673+0000 7f373dd9b700 1 --2- 172.21.15.82:0/1245498943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561852f65c00 0x561852f6eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.684 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.673+0000 7f373ed9d700 1 --2- 172.21.15.82:0/1245498943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561852f665a0 0x561852f75b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.684 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.673+0000 7f373e59c700 1 -- 172.21.15.82:0/1245498943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561852f65c00 msgr2=0x561852f6eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:49.685 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.673+0000 7f373e59c700 1 --2- 172.21.15.82:0/1245498943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561852f65c00 0x561852f6eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.685 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.673+0000 7f373e59c700 1 -- 172.21.15.82:0/1245498943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561852f665a0 msgr2=0x561852f75b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:49.685 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.673+0000 7f373e59c700 1 --2- 172.21.15.82:0/1245498943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561852f665a0 0x561852f75b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.685 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.673+0000 7f373e59c700 1 -- 172.21.15.82:0/1245498943 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3730011440 con 0x561852f64e00 2023-09-09T23:13:49.685 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.674+0000 7f373e59c700 1 --2- 172.21.15.82:0/1245498943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561852f64e00 0x561852f6e5d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f373001b040 tx=0x7f37300038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3a26dee8b40d34 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:49.685 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.674+0000 7f3725ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:49.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.674+0000 7f3725ffb700 1 -- 172.21.15.82:0/1245498943 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3730003ce0 con 0x561852f64e00 2023-09-09T23:13:49.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.674+0000 7f3725ffb700 1 -- 172.21.15.82:0/1245498943 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f3730003e40 con 0x561852f64e00 2023-09-09T23:13:49.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.674+0000 7f3750b08b80 1 -- 172.21.15.82:0/1245498943 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561852e0e4f0 con 0x561852f64e00 2023-09-09T23:13:49.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.674+0000 7f3725ffb700 1 -- 172.21.15.82:0/1245498943 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f373000f4f0 con 0x561852f64e00 2023-09-09T23:13:49.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.674+0000 7f3750b08b80 10 client.15735 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:49.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.674+0000 7f3750b08b80 1 -- 172.21.15.82:0/1245498943 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561852ccbc60 con 0x561852f64e00 2023-09-09T23:13:49.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.674+0000 7f3725ffb700 1 -- 172.21.15.82:0/1245498943 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561852e0e4f0 con 0x561852f64e00 2023-09-09T23:13:49.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.674+0000 7f3725ffb700 1 -- 172.21.15.82:0/1245498943 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f37300227b0 con 0x561852f64e00 2023-09-09T23:13:49.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.674+0000 7f3750b08b80 10 client.15735 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:49.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.674+0000 7f3750b08b80 10 client.15735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:13:49.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.674+0000 7f3750b08b80 20 client.15735 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:49.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.674+0000 7f3750b08b80 1 --2- 172.21.15.82:0/1245498943 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x561852f7a210 0x561852f7c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.674+0000 7f3750b08b80 4 client.15735 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:13:49.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.674+0000 7f3750b08b80 1 -- 172.21.15.82:0/1245498943 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561852ecb8a0 con 0x561852f7a210 2023-09-09T23:13:49.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.675+0000 7f373dd9b700 1 --2- 172.21.15.82:0/1245498943 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x561852f7a210 0x561852f7c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.675+0000 7f373dd9b700 1 --2- 172.21.15.82:0/1245498943 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x561852f7a210 0x561852f7c630 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af78dd3592251325 server_cookie=c0e0dfa1d3739f84 in_seq=0 out_seq=0 2023-09-09T23:13:49.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.675+0000 7f3725ffb700 10 client.15735 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:13:49.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.675+0000 7f3725ffb700 1 -- 172.21.15.82:0/1245498943 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561852ecb8a0 con 0x561852f7a210 2023-09-09T23:13:49.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.675+0000 7f3725ffb700 10 client.15735 handle_command_reply: tid=0 2023-09-09T23:13:49.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.675+0000 7f3750b08b80 1 client.15735 shutdown 2023-09-09T23:13:49.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.676+0000 7f3750b08b80 1 -- 172.21.15.82:0/1245498943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561852f64e00 msgr2=0x561852f6e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:49.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.676+0000 7f3750b08b80 1 --2- 172.21.15.82:0/1245498943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561852f64e00 0x561852f6e5d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f373001b040 tx=0x7f37300038b0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.676+0000 7f3750b08b80 1 -- 172.21.15.82:0/1245498943 shutdown_connections 2023-09-09T23:13:49.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.676+0000 7f3750b08b80 1 --2- 172.21.15.82:0/1245498943 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x561852f7a210 0x561852f7c630 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.676+0000 7f3750b08b80 1 --2- 172.21.15.82:0/1245498943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561852f665a0 0x561852f75b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.676+0000 7f3750b08b80 1 --2- 172.21.15.82:0/1245498943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561852f64e00 0x561852f6e5d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.676+0000 7f3750b08b80 1 --2- 172.21.15.82:0/1245498943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561852f65c00 0x561852f6eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.676+0000 7f3725ffb700 0 client.15735 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:13:49.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.676+0000 7f3750b08b80 1 -- 172.21.15.82:0/1245498943 >> 172.21.15.82:0/1245498943 conn(0x561852ed9090 msgr2=0x561852f5edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:49.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.676+0000 7f3750b08b80 1 -- 172.21.15.82:0/1245498943 shutdown_connections 2023-09-09T23:13:49.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.676+0000 7f3750b08b80 1 -- 172.21.15.82:0/1245498943 wait complete. 2023-09-09T23:13:49.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.676+0000 7f3750b08b80 20 client.15735 trim_cache size 0 max 16384 2023-09-09T23:13:49.696 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.692+0000 7f3750b08b80 1 Processor -- start 2023-09-09T23:13:49.697 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.692+0000 7f3750b08b80 1 -- start start 2023-09-09T23:13:49.697 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.692+0000 7f3750b08b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561852f64e00 0x561852f76440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.697 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.692+0000 7f3750b08b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561852f65c00 0x561852f75b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.697 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.692+0000 7f3750b08b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561852f665a0 0x561852f6c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.697 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.692+0000 7f3750b08b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561852f29850 con 0x561852f64e00 2023-09-09T23:13:49.698 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.692+0000 7f3750b08b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561852f6f720 con 0x561852f665a0 2023-09-09T23:13:49.698 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.692+0000 7f3750b08b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561852f79f30 con 0x561852f65c00 2023-09-09T23:13:49.698 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.692+0000 7f373e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561852f64e00 0x561852f76440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.698 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.692+0000 7f373e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561852f64e00 0x561852f76440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56786/0 (socket says 172.21.15.82:56786) 2023-09-09T23:13:49.698 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.693+0000 7f373e59c700 1 -- 172.21.15.82:0/1257258041 learned_addr learned my addr 172.21.15.82:0/1257258041 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:49.699 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.693+0000 7f373ed9d700 1 --2- 172.21.15.82:0/1257258041 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561852f665a0 0x561852f6c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.699 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.693+0000 7f373dd9b700 1 --2- 172.21.15.82:0/1257258041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561852f65c00 0x561852f75b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.699 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.693+0000 7f373e59c700 1 -- 172.21.15.82:0/1257258041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561852f65c00 msgr2=0x561852f75b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:49.699 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.693+0000 7f373e59c700 1 --2- 172.21.15.82:0/1257258041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561852f65c00 0x561852f75b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.699 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.693+0000 7f373e59c700 1 -- 172.21.15.82:0/1257258041 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561852f665a0 msgr2=0x561852f6c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:49.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.693+0000 7f373e59c700 1 --2- 172.21.15.82:0/1257258041 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561852f665a0 0x561852f6c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.693+0000 7f373e59c700 1 -- 172.21.15.82:0/1257258041 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561852d09250 con 0x561852f64e00 2023-09-09T23:13:49.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.693+0000 7f373dd9b700 1 --2- 172.21.15.82:0/1257258041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561852f65c00 0x561852f75b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:13:49.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.693+0000 7f373e59c700 1 --2- 172.21.15.82:0/1257258041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561852f64e00 0x561852f76440 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f373002f040 tx=0x7f3730005c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e205ef0c5f22bd0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:49.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.693+0000 7f372d7fa700 1 -- 172.21.15.82:0/1257258041 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3730022450 con 0x561852f64e00 2023-09-09T23:13:49.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.693+0000 7f372d7fa700 1 -- 172.21.15.82:0/1257258041 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f373000f480 con 0x561852f64e00 2023-09-09T23:13:49.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.693+0000 7f372d7fa700 1 -- 172.21.15.82:0/1257258041 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373002d690 con 0x561852f64e00 2023-09-09T23:13:49.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.694+0000 7f3750b08b80 1 -- 172.21.15.82:0/1257258041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561852f64e00 msgr2=0x561852f76440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:49.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.694+0000 7f3750b08b80 1 --2- 172.21.15.82:0/1257258041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561852f64e00 0x561852f76440 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f373002f040 tx=0x7f3730005c20 comp rx=0 tx=0).stop 2023-09-09T23:13:49.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.694+0000 7f3750b08b80 1 -- 172.21.15.82:0/1257258041 shutdown_connections 2023-09-09T23:13:49.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.694+0000 7f3750b08b80 1 --2- 172.21.15.82:0/1257258041 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561852f665a0 0x561852f6c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.694+0000 7f3750b08b80 1 --2- 172.21.15.82:0/1257258041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561852f64e00 0x561852f76440 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.694+0000 7f3750b08b80 1 --2- 172.21.15.82:0/1257258041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561852f65c00 0x561852f75b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.694+0000 7f3750b08b80 1 -- 172.21.15.82:0/1257258041 >> 172.21.15.82:0/1257258041 conn(0x561852ed9090 msgr2=0x561852f651e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:49.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.694+0000 7f3750b08b80 1 -- 172.21.15.82:0/1257258041 shutdown_connections 2023-09-09T23:13:49.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.694+0000 7f3750b08b80 1 -- 172.21.15.82:0/1257258041 wait complete. 2023-09-09T23:13:49.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.694+0000 7f3750b08b80 1 -- start start 2023-09-09T23:13:49.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.694+0000 7f3750b08b80 1 Processor -- start 2023-09-09T23:13:49.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.694+0000 7f3750b08b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561852f64e00 0x561852f75620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.694+0000 7f3750b08b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561852f65c00 0x561852f73c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.694+0000 7f3750b08b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561852f665a0 0x561852f74170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.694+0000 7f3750b08b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561852f75e40 con 0x561852f65c00 2023-09-09T23:13:49.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.694+0000 7f3750b08b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561852f746b0 con 0x561852f665a0 2023-09-09T23:13:49.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.694+0000 7f3750b08b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561852f74870 con 0x561852f64e00 2023-09-09T23:13:49.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.695+0000 7f3750b08b80 10 client.0 _ll_register_callbacks cb 0x7f3730011660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:13:49.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.695+0000 7f373dd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561852f65c00 0x561852f73c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.695+0000 7f373ed9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561852f665a0 0x561852f74170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.695+0000 7f373dd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561852f65c00 0x561852f73c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56794/0 (socket says 172.21.15.82:56794) 2023-09-09T23:13:49.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.695+0000 7f373e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561852f64e00 0x561852f75620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.695+0000 7f373dd9b700 1 -- 172.21.15.82:0/1756687715 learned_addr learned my addr 172.21.15.82:0/1756687715 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:49.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.695+0000 7f373dd9b700 1 -- 172.21.15.82:0/1756687715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561852f64e00 msgr2=0x561852f75620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:49.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.695+0000 7f373dd9b700 1 --2- 172.21.15.82:0/1756687715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561852f64e00 0x561852f75620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.695+0000 7f373dd9b700 1 -- 172.21.15.82:0/1756687715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561852f665a0 msgr2=0x561852f74170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:13:49.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.695+0000 7f373dd9b700 1 --2- 172.21.15.82:0/1756687715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561852f665a0 0x561852f74170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.695+0000 7f373dd9b700 1 -- 172.21.15.82:0/1756687715 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561852ccbc60 con 0x561852f65c00 2023-09-09T23:13:49.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.695+0000 7f373ed9d700 1 --2- 172.21.15.82:0/1756687715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561852f665a0 0x561852f74170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:13:49.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.695+0000 7f373dd9b700 1 --2- 172.21.15.82:0/1756687715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561852f65c00 0x561852f73c30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f37340c9280 tx=0x7f3734000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641351ead083e62 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:49.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.695+0000 7f3725ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:13:49.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.695+0000 7f3725ffb700 1 -- 172.21.15.82:0/1756687715 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37340b1290 con 0x561852f65c00 2023-09-09T23:13:49.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.695+0000 7f3725ffb700 1 -- 172.21.15.82:0/1756687715 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f3734004380 con 0x561852f65c00 2023-09-09T23:13:49.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.695+0000 7f3750b08b80 1 -- 172.21.15.82:0/1756687715 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3730004e50 con 0x561852f65c00 2023-09-09T23:13:49.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.695+0000 7f3725ffb700 1 -- 172.21.15.82:0/1756687715 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37340991d0 con 0x561852f65c00 2023-09-09T23:13:49.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.695+0000 7f3750b08b80 10 client.15741 fetch_fsmap learned FSMap version 27 2023-09-09T23:13:49.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.695+0000 7f3750b08b80 1 -- 172.21.15.82:0/1756687715 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561852ce12b0 con 0x561852f65c00 2023-09-09T23:13:49.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.696+0000 7f3725ffb700 1 -- 172.21.15.82:0/1756687715 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3730004e50 con 0x561852f65c00 2023-09-09T23:13:49.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.696+0000 7f3725ffb700 1 -- 172.21.15.82:0/1756687715 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f37340a0b50 con 0x561852f65c00 2023-09-09T23:13:49.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.696+0000 7f3750b08b80 10 client.15741 fetch_fsmap finished waiting for FSMap version 27 2023-09-09T23:13:49.709 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.696+0000 7f3750b08b80 10 client.15741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:13:49.709 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.696+0000 7f3750b08b80 20 client.15741 populate_metadata read hostname 'smithi082' 2023-09-09T23:13:49.709 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.696+0000 7f3750b08b80 1 --2- 172.21.15.82:0/1756687715 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x561852fcf5e0 0x561852fd1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:49.709 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.696+0000 7f3750b08b80 4 client.15741 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:13:49.709 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.696+0000 7f3750b08b80 1 -- 172.21.15.82:0/1756687715 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561852eaf920 con 0x561852fcf5e0 2023-09-09T23:13:49.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.696+0000 7f373e59c700 1 --2- 172.21.15.82:0/1756687715 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x561852fcf5e0 0x561852fd1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:49.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.696+0000 7f3725ffb700 10 client.15741 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:13:49.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.696+0000 7f373e59c700 1 --2- 172.21.15.82:0/1756687715 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x561852fcf5e0 0x561852fd1a00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc733afe7082178f server_cookie=a1b2797411c02a95 in_seq=0 out_seq=0 2023-09-09T23:13:49.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.697+0000 7f3725ffb700 1 -- 172.21.15.82:0/1756687715 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561852eaf920 con 0x561852fcf5e0 2023-09-09T23:13:49.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.697+0000 7f3725ffb700 10 client.15741 handle_command_reply: tid=0 2023-09-09T23:13:49.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.697+0000 7f3750b08b80 1 client.15741 shutdown 2023-09-09T23:13:49.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.697+0000 7f3750b08b80 1 -- 172.21.15.82:0/1756687715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561852f65c00 msgr2=0x561852f73c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:49.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.697+0000 7f3750b08b80 1 --2- 172.21.15.82:0/1756687715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561852f65c00 0x561852f73c30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f37340c9280 tx=0x7f3734000bc0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.697+0000 7f3750b08b80 1 -- 172.21.15.82:0/1756687715 shutdown_connections 2023-09-09T23:13:49.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.697+0000 7f3750b08b80 1 --2- 172.21.15.82:0/1756687715 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x561852fcf5e0 0x561852fd1a00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.697+0000 7f3750b08b80 1 --2- 172.21.15.82:0/1756687715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561852f665a0 0x561852f74170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.697+0000 7f3725ffb700 0 client.15741 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:13:49.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.697+0000 7f3750b08b80 1 --2- 172.21.15.82:0/1756687715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561852f64e00 0x561852f75620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.697+0000 7f3750b08b80 1 --2- 172.21.15.82:0/1756687715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561852f65c00 0x561852f73c30 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.697+0000 7f3750b08b80 1 -- 172.21.15.82:0/1756687715 >> 172.21.15.82:0/1756687715 conn(0x561852ed9090 msgr2=0x561852f76440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:49.713 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.697+0000 7f3750b08b80 1 -- 172.21.15.82:0/1756687715 shutdown_connections 2023-09-09T23:13:49.713 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.697+0000 7f3750b08b80 1 -- 172.21.15.82:0/1756687715 wait complete. 2023-09-09T23:13:49.713 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.697+0000 7f3750b08b80 20 client.15741 trim_cache size 0 max 16384 2023-09-09T23:13:49.713 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:13:49.714 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-09T23:13:49.714 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:13:49.714 INFO:teuthology.orchestra.run.smithi082.stdout: "86192229-b7c8-4fe3-8787-9b91e68b875f": { 2023-09-09T23:13:49.715 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:13:49.715 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "86192229-b7c8-4fe3-8787-9b91e68b875f", 2023-09-09T23:13:49.715 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:13:49.715 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:13:49.715 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:13:49.715 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:13:49.717 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.697+0000 7f3744942700 1 -- 172.21.15.82:0/1259773254 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f37280a0690 msgr2=0x7f37280a2b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-09T23:13:49.717 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.697+0000 7f3744942700 1 --2- 172.21.15.82:0/1259773254 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f37280a0690 0x7f37280a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.717 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.697+0000 7f3744942700 1 -- 172.21.15.82:0/1259773254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3740101800 msgr2=0x7f37401968f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:49.717 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.697+0000 7f3744942700 1 --2- 172.21.15.82:0/1259773254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3740101800 0x7f37401968f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f373400d980 tx=0x7f373400dc90 comp rx=0 tx=0).stop 2023-09-09T23:13:49.717 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.697+0000 7f3744942700 1 -- 172.21.15.82:0/1259773254 shutdown_connections 2023-09-09T23:13:49.718 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.697+0000 7f3744942700 1 --2- 172.21.15.82:0/1259773254 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7f37280a0690 0x7f37280a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.718 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.697+0000 7f3744942700 1 --2- 172.21.15.82:0/1259773254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3740100610 0x7f37401963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.718 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.697+0000 7f3744942700 1 --2- 172.21.15.82:0/1259773254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37401021a0 0x7f374019d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.718 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.697+0000 7f3744942700 1 --2- 172.21.15.82:0/1259773254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3740101800 0x7f37401968f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:49.718 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.697+0000 7f3744942700 1 -- 172.21.15.82:0/1259773254 >> 172.21.15.82:0/1259773254 conn(0x7f37400fbb80 msgr2=0x7f37400fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:49.719 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.698+0000 7f3744942700 1 -- 172.21.15.82:0/1259773254 shutdown_connections 2023-09-09T23:13:49.719 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.698+0000 7f3744942700 1 -- 172.21.15.82:0/1259773254 wait complete. 2023-09-09T23:13:49.719 INFO:tasks.cephfs.filesystem:scrub status for tag:86192229-b7c8-4fe3-8787-9b91e68b875f - {'path': '/', 'tag': '86192229-b7c8-4fe3-8787-9b91e68b875f', 'options': 'recursive,force'} 2023-09-09T23:13:49.756 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.759+0000 7fcfa2ffd700 1 -- 172.21.15.82:0/394847667 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcf8c0a0690 msgr2=0x7fcf8c0a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1453069857 2023-09-09T23:13:49.756 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:49.759+0000 7fcfa2ffd700 1 --2- 172.21.15.82:0/394847667 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcf8c0a0690 0x7fcf8c0a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-09T23:13:49.856 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:49 smithi082 ceph-mon[98694]: scrub queued for path: / 2023-09-09T23:13:49.857 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:49 smithi082 ceph-mon[98694]: scrub summary: idle+waiting paths [/] 2023-09-09T23:13:49.857 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:49 smithi082 ceph-mon[98694]: scrub summary: active paths [/] 2023-09-09T23:13:49.857 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:49 smithi082 ceph-mon[98694]: scrub summary: idle+waiting paths [/] 2023-09-09T23:13:50.001 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:13:49 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:13:49.719+0000 7f0b0093b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-09T23:13:50.001 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:13:49 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:13:49.803+0000 7f0b0093b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-09T23:13:50.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:49 smithi098 ceph-mon[103124]: scrub queued for path: / 2023-09-09T23:13:50.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:49 smithi098 ceph-mon[103124]: scrub summary: idle+waiting paths [/] 2023-09-09T23:13:50.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:49 smithi098 ceph-mon[103124]: scrub summary: active paths [/] 2023-09-09T23:13:50.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:49 smithi098 ceph-mon[103124]: scrub summary: idle+waiting paths [/] 2023-09-09T23:13:50.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:49 smithi170 ceph-mon[102832]: scrub queued for path: / 2023-09-09T23:13:50.175 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:49 smithi170 ceph-mon[102832]: scrub summary: idle+waiting paths [/] 2023-09-09T23:13:50.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:49 smithi170 ceph-mon[102832]: scrub summary: active paths [/] 2023-09-09T23:13:50.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:49 smithi170 ceph-mon[102832]: scrub summary: idle+waiting paths [/] 2023-09-09T23:13:50.176 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:13:49 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:13:49.727+0000 7fc6bfd29200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-09T23:13:50.176 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:13:49 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:13:49.811+0000 7fc6bfd29200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-09T23:13:50.204 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:49 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:49.859+0000 7f0a735f8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-09T23:13:50.205 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:49 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:49.942+0000 7f0a735f8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-09T23:13:50.834 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:13:50 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:13:50.574+0000 7f0b0093b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-09T23:13:50.835 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:13:50 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:13:50.664+0000 7f0b0093b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-09T23:13:50.851 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:13:50 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:13:50.588+0000 7fc6bfd29200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-09T23:13:50.851 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:13:50 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:13:50.681+0000 7fc6bfd29200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-09T23:13:50.982 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:50 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:50.718+0000 7f0a735f8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-09T23:13:50.983 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:50 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:50.811+0000 7f0a735f8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-09T23:13:51.112 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:13:50 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:13:50.837+0000 7f0b0093b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-09T23:13:51.130 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:13:50 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:13:50.853+0000 7fc6bfd29200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-09T23:13:51.264 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:50 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:50.985+0000 7f0a735f8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-09T23:13:51.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:51.360+0000 7fcfa2ffd700 1 -- 172.21.15.82:0/394847667 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcf8c0a0690 msgr2=0x7fcf8c0a2b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1453069857 2023-09-09T23:13:51.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:51.360+0000 7fcfa2ffd700 1 --2- 172.21.15.82:0/394847667 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcf8c0a0690 0x7fcf8c0a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-09-09T23:13:51.425 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:13:51 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:13:51.132+0000 7fc6bfd29200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-09T23:13:51.425 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:13:51 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:13:51.361+0000 7fc6bfd29200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-09T23:13:51.485 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:13:51 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:13:51.115+0000 7f0b0093b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-09T23:13:51.485 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:13:51 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:13:51.340+0000 7f0b0093b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-09T23:13:51.640 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:51 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:51.267+0000 7f0a735f8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-09T23:13:51.641 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:51 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:51.494+0000 7f0a735f8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-09T23:13:51.740 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:13:51 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:13:51.488+0000 7f0b0093b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-09T23:13:51.768 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:13:51 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:13:51.508+0000 7fc6bfd29200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-09T23:13:51.769 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:51 smithi170 ceph-mon[102832]: scrub summary: idle 2023-09-09T23:13:51.911 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:51 smithi082 ceph-mon[98694]: scrub summary: idle 2023-09-09T23:13:51.912 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:51 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:51.643+0000 7f0a735f8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-09T23:13:52.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:51 smithi098 ceph-mon[103124]: scrub summary: idle 2023-09-09T23:13:52.002 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:13:51 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:13:51.753+0000 7f0b0093b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-09T23:13:52.175 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:13:51 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:13:51.771+0000 7fc6bfd29200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-09T23:13:52.204 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:51 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:51.914+0000 7f0a735f8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-09T23:13:52.751 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:13:52 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:13:52.450+0000 7f0b0093b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-09T23:13:52.751 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:13:52 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:13:52.582+0000 7f0b0093b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-09T23:13:52.796 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:13:52 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:13:52.474+0000 7fc6bfd29200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-09T23:13:52.797 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:13:52 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:13:52.606+0000 7fc6bfd29200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-09T23:13:52.949 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:52 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:52.623+0000 7f0a735f8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-09T23:13:52.950 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:52 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:52.755+0000 7f0a735f8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-09T23:13:53.103 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:13:52 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:13:52.777+0000 7f0b0093b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-09T23:13:53.103 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:13:52 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:13:52.890+0000 7f0b0093b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-09T23:13:53.125 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:13:52 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:13:52.798+0000 7fc6bfd29200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-09T23:13:53.125 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:13:52 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:13:52.911+0000 7fc6bfd29200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-09T23:13:53.204 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:52 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:52.951+0000 7f0a735f8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-09T23:13:53.205 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:53 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:53.066+0000 7f0a735f8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-09T23:13:53.425 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:13:53 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:13:53.127+0000 7fc6bfd29200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-09T23:13:53.501 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:13:53 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:13:53.105+0000 7f0b0093b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-09T23:13:53.704 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:53 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:53.289+0000 7f0a735f8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-09T23:13:54.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:54.185+0000 7fcfa0ff9700 1 -- 172.21.15.82:0/394847667 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 20) v1 ==== 134944+0+0 (secure 0 0 0) 0x7fcf9400c950 con 0x7fcfa40ff500 2023-09-09T23:13:54.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:54.185+0000 7fcfa0ff9700 1 -- 172.21.15.82:0/394847667 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcf8c0a0690 msgr2=0x7fcf8c0a2b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-09T23:13:54.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:54.185+0000 7fcfa0ff9700 1 --2- 172.21.15.82:0/394847667 >> [v2:172.21.15.82:6800/1453069857,v1:172.21.15.82:6801/1453069857] conn(0x7fcf8c0a0690 0x7fcf8c0a2b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:54.251 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 09 23:13:53 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: 2023-09-09T23:13:53.971+0000 7f0b0093b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-09T23:13:54.277 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Sep 09 23:13:53 smithi170 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-z[104846]: 2023-09-09T23:13:53.996+0000 7fc6bfd29200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-09T23:13:54.435 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:54.161+0000 7f0a735f8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-09T23:13:54.436 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:54.364+0000 7f09d6733700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-09T23:13:54.436 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:54.364+0000 7f09d6733700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-09T23:13:54.436 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:54.364+0000 7f09d6733700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-09T23:13:54.436 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:54.364+0000 7f09d6733700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-09T23:13:54.436 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:54.364+0000 7f09d6733700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-09T23:13:54.437 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:54.365+0000 7f09d372d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-09T23:13:54.437 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:54.365+0000 7f09d372d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-09T23:13:54.437 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:54.365+0000 7f09d372d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-09T23:13:54.437 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:54.365+0000 7f09d372d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-09T23:13:54.437 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:13:54.365+0000 7f09d372d700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-09T23:13:54.438 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: Standby manager daemon y restarted 2023-09-09T23:13:54.438 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: Standby manager daemon y started 2023-09-09T23:13:54.438 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.? 172.21.15.98:0/3639032558' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-09T23:13:54.438 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.? 172.21.15.98:0/3639032558' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-09T23:13:54.439 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.? 172.21.15.98:0/3639032558' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-09T23:13:54.439 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.? 172.21.15.98:0/3639032558' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-09T23:13:54.439 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: Standby manager daemon z restarted 2023-09-09T23:13:54.439 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: Standby manager daemon z started 2023-09-09T23:13:54.439 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.? 172.21.15.170:0/3421068220' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-09T23:13:54.440 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.? 172.21.15.170:0/3421068220' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-09T23:13:54.440 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.? 172.21.15.170:0/3421068220' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-09T23:13:54.440 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.? 172.21.15.170:0/3421068220' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-09T23:13:54.440 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: Active manager daemon x restarted 2023-09-09T23:13:54.440 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: Activating manager daemon x 2023-09-09T23:13:54.441 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: osdmap e84: 12 total, 12 up, 12 in 2023-09-09T23:13:54.441 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: mgrmap e20: x(active, starting, since 0.0115841s), standbys: y, z 2023-09-09T23:13:54.441 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-09T23:13:54.441 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:13:54.441 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:13:54.442 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-09T23:13:54.442 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-09T23:13:54.442 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-09T23:13:54.442 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-09T23:13:54.442 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-09T23:13:54.443 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-09T23:13:54.443 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-09T23:13:54.443 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-09T23:13:54.443 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-09T23:13:54.444 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-09T23:13:54.444 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-09T23:13:54.444 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-09T23:13:54.444 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-09T23:13:54.444 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-09T23:13:54.445 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-09T23:13:54.445 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-09T23:13:54.445 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-09T23:13:54.445 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-09T23:13:54.445 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-09T23:13:54.446 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-09T23:13:54.446 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-09T23:13:54.446 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-09T23:13:54.446 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-09T23:13:54.446 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-09T23:13:54.447 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-09T23:13:54.447 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-09T23:13:54.447 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-09T23:13:54.447 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-09T23:13:54.447 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-09T23:13:54.448 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-09T23:13:54.448 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: Manager daemon x is now available 2023-09-09T23:13:54.448 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:13:54.448 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:13:54.448 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-09T23:13:54.449 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-09T23:13:54.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: Standby manager daemon y restarted 2023-09-09T23:13:54.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: Standby manager daemon y started 2023-09-09T23:13:54.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.? 172.21.15.98:0/3639032558' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-09T23:13:54.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.? 172.21.15.98:0/3639032558' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-09T23:13:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.? 172.21.15.98:0/3639032558' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-09T23:13:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.? 172.21.15.98:0/3639032558' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-09T23:13:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: Standby manager daemon z restarted 2023-09-09T23:13:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: Standby manager daemon z started 2023-09-09T23:13:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.? 172.21.15.170:0/3421068220' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-09T23:13:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.? 172.21.15.170:0/3421068220' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-09T23:13:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.? 172.21.15.170:0/3421068220' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-09T23:13:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.? 172.21.15.170:0/3421068220' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-09T23:13:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: Active manager daemon x restarted 2023-09-09T23:13:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: Activating manager daemon x 2023-09-09T23:13:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: osdmap e84: 12 total, 12 up, 12 in 2023-09-09T23:13:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: mgrmap e20: x(active, starting, since 0.0115841s), standbys: y, z 2023-09-09T23:13:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-09T23:13:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:13:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:13:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-09T23:13:54.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-09T23:13:54.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-09T23:13:54.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-09T23:13:54.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-09T23:13:54.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-09T23:13:54.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-09T23:13:54.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-09T23:13:54.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-09T23:13:54.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-09T23:13:54.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-09T23:13:54.682 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-09T23:13:54.682 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-09T23:13:54.682 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-09T23:13:54.682 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-09T23:13:54.682 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-09T23:13:54.683 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-09T23:13:54.683 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-09T23:13:54.683 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-09T23:13:54.683 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-09T23:13:54.683 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-09T23:13:54.683 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-09T23:13:54.684 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-09T23:13:54.684 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-09T23:13:54.684 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-09T23:13:54.684 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-09T23:13:54.684 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-09T23:13:54.685 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-09T23:13:54.685 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-09T23:13:54.685 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-09T23:13:54.685 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: Manager daemon x is now available 2023-09-09T23:13:54.685 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:13:54.686 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:13:54.686 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-09T23:13:54.686 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-09T23:13:54.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: Standby manager daemon y restarted 2023-09-09T23:13:54.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: Standby manager daemon y started 2023-09-09T23:13:54.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.? 172.21.15.98:0/3639032558' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-09T23:13:54.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.? 172.21.15.98:0/3639032558' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-09T23:13:54.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.? 172.21.15.98:0/3639032558' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-09T23:13:54.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.? 172.21.15.98:0/3639032558' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-09T23:13:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: Standby manager daemon z restarted 2023-09-09T23:13:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: Standby manager daemon z started 2023-09-09T23:13:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.? 172.21.15.170:0/3421068220' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-09T23:13:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.? 172.21.15.170:0/3421068220' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-09T23:13:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.? 172.21.15.170:0/3421068220' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-09T23:13:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.? 172.21.15.170:0/3421068220' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-09T23:13:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: Active manager daemon x restarted 2023-09-09T23:13:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: Activating manager daemon x 2023-09-09T23:13:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: osdmap e84: 12 total, 12 up, 12 in 2023-09-09T23:13:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: mgrmap e20: x(active, starting, since 0.0115841s), standbys: y, z 2023-09-09T23:13:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-09T23:13:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-09T23:13:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-09T23:13:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-09T23:13:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-09T23:13:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-09T23:13:54.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-09T23:13:54.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-09T23:13:54.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-09T23:13:54.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-09T23:13:54.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-09T23:13:54.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-09T23:13:54.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-09T23:13:54.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-09T23:13:54.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-09T23:13:54.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-09T23:13:54.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-09T23:13:54.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-09T23:13:54.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-09T23:13:54.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-09T23:13:54.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-09T23:13:54.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-09T23:13:54.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-09T23:13:54.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-09T23:13:54.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-09T23:13:54.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-09T23:13:54.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-09T23:13:54.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-09T23:13:54.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-09T23:13:54.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-09T23:13:54.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-09T23:13:54.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-09T23:13:54.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-09T23:13:54.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: Manager daemon x is now available 2023-09-09T23:13:54.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:13:54.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:13:54.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-09T23:13:54.762 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-09T23:13:55.186 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.188+0000 7fcfa0ff9700 1 -- 172.21.15.82:0/394847667 <== mon.0 v2:172.21.15.82:3300/0 8 ==== mgrmap(e 21) v1 ==== 135071+0+0 (secure 0 0 0) 0x7fcf940c9a00 con 0x7fcfa40ff500 2023-09-09T23:13:55.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.189+0000 7fcfa0ff9700 1 --2- 172.21.15.82:0/394847667 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fcf8c050140 0x7fcf8c050520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:55.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.189+0000 7fcfa0ff9700 1 -- 172.21.15.82:0/394847667 --> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfa402ce60 con 0x7fcf8c050140 2023-09-09T23:13:55.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.192+0000 7fcfa2ffd700 1 --2- 172.21.15.82:0/394847667 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fcf8c050140 0x7fcf8c050520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:55.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.196+0000 7fcfa2ffd700 1 --2- 172.21.15.82:0/394847667 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fcf8c050140 0x7fcf8c050520 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4067ea0 tx=0x7fcf98005fb0 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:55.259 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.262+0000 7fcfa0ff9700 1 -- 172.21.15.82:0/394847667 <== mgr.15597 v2:172.21.15.82:6800/4121269095 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fcfa402ce60 con 0x7fcf8c050140 2023-09-09T23:13:55.260 INFO:teuthology.orchestra.run.smithi082.stdout:Schedule set for path / 2023-09-09T23:13:55.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.264+0000 7fcfaa16a700 1 -- 172.21.15.82:0/394847667 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fcf8c050140 msgr2=0x7fcf8c050520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:55.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.264+0000 7fcfaa16a700 1 --2- 172.21.15.82:0/394847667 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fcf8c050140 0x7fcf8c050520 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4067ea0 tx=0x7fcf98005fb0 comp rx=0 tx=0).stop 2023-09-09T23:13:55.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.264+0000 7fcfaa16a700 1 -- 172.21.15.82:0/394847667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfa40ff500 msgr2=0x7fcfa419b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:55.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.264+0000 7fcfaa16a700 1 --2- 172.21.15.82:0/394847667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfa40ff500 0x7fcfa419b460 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400e970 tx=0x7fcf9400ec80 comp rx=0 tx=0).stop 2023-09-09T23:13:55.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.264+0000 7fcfa3fff700 1 -- 172.21.15.82:0/394847667 reap_dead start 2023-09-09T23:13:55.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.264+0000 7fcfaa16a700 1 -- 172.21.15.82:0/394847667 shutdown_connections 2023-09-09T23:13:55.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.264+0000 7fcfaa16a700 1 -- 172.21.15.82:0/394847667 >> 172.21.15.82:0/394847667 conn(0x7fcfa40faa50 msgr2=0x7fcfa40fcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:55.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.265+0000 7fcfaa16a700 1 -- 172.21.15.82:0/394847667 shutdown_connections 2023-09-09T23:13:55.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.265+0000 7fcfaa16a700 1 -- 172.21.15.82:0/394847667 wait complete. 2023-09-09T23:13:55.275 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h' 2023-09-09T23:13:55.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.449+0000 7fcc650cc700 1 -- 172.21.15.82:0/3155763085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc600ff510 msgr2=0x7fcc600ff930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:55.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.449+0000 7fcc650cc700 1 --2- 172.21.15.82:0/3155763085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc600ff510 0x7fcc600ff930 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800b0d0 tx=0x7fcc5800b3e0 comp rx=0 tx=0).stop 2023-09-09T23:13:55.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.449+0000 7fcc650cc700 1 -- 172.21.15.82:0/3155763085 shutdown_connections 2023-09-09T23:13:55.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.449+0000 7fcc650cc700 1 --2- 172.21.15.82:0/3155763085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc601010a0 0x7fcc60107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:55.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.449+0000 7fcc650cc700 1 --2- 172.21.15.82:0/3155763085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc600ff510 0x7fcc600ff930 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:55.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.449+0000 7fcc650cc700 1 --2- 172.21.15.82:0/3155763085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc60100700 0x7fcc60100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:55.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.449+0000 7fcc650cc700 1 -- 172.21.15.82:0/3155763085 >> 172.21.15.82:0/3155763085 conn(0x7fcc600faa60 msgr2=0x7fcc600fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:55.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.449+0000 7fcc650cc700 1 -- 172.21.15.82:0/3155763085 shutdown_connections 2023-09-09T23:13:55.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.449+0000 7fcc650cc700 1 -- 172.21.15.82:0/3155763085 wait complete. 2023-09-09T23:13:55.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.450+0000 7fcc650cc700 1 Processor -- start 2023-09-09T23:13:55.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.450+0000 7fcc650cc700 1 -- start start 2023-09-09T23:13:55.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.450+0000 7fcc650cc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc600ff510 0x7fcc60199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:55.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.450+0000 7fcc650cc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc60100700 0x7fcc6019b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:55.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.450+0000 7fcc650cc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc601010a0 0x7fcc6019dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:55.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.450+0000 7fcc650cc700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6019e270 con 0x7fcc600ff510 2023-09-09T23:13:55.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.450+0000 7fcc650cc700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6019e3b0 con 0x7fcc60100700 2023-09-09T23:13:55.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.450+0000 7fcc650cc700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6019e570 con 0x7fcc601010a0 2023-09-09T23:13:55.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.450+0000 7fcc5ed9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc600ff510 0x7fcc60199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:55.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.450+0000 7fcc5ed9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc600ff510 0x7fcc60199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57054/0 (socket says 172.21.15.82:57054) 2023-09-09T23:13:55.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.450+0000 7fcc5ed9d700 1 -- 172.21.15.82:0/3136650401 learned_addr learned my addr 172.21.15.82:0/3136650401 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:55.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.450+0000 7fcc5e59c700 1 --2- 172.21.15.82:0/3136650401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc60100700 0x7fcc6019b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:55.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.450+0000 7fcc5f59e700 1 --2- 172.21.15.82:0/3136650401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc601010a0 0x7fcc6019dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:55.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.451+0000 7fcc5ed9d700 1 -- 172.21.15.82:0/3136650401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc601010a0 msgr2=0x7fcc6019dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:55.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.451+0000 7fcc5ed9d700 1 --2- 172.21.15.82:0/3136650401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc601010a0 0x7fcc6019dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:55.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.451+0000 7fcc5ed9d700 1 -- 172.21.15.82:0/3136650401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc60100700 msgr2=0x7fcc6019b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:55.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.451+0000 7fcc5ed9d700 1 --2- 172.21.15.82:0/3136650401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc60100700 0x7fcc6019b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:55.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.451+0000 7fcc5ed9d700 1 -- 172.21.15.82:0/3136650401 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc58009d00 con 0x7fcc600ff510 2023-09-09T23:13:55.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.451+0000 7fcc5ed9d700 1 --2- 172.21.15.82:0/3136650401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc600ff510 0x7fcc60199200 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800b820 tx=0x7fcc580094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:55.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.451+0000 7fcc47fff700 1 -- 172.21.15.82:0/3136650401 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5800e040 con 0x7fcc600ff510 2023-09-09T23:13:55.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.451+0000 7fcc650cc700 1 -- 172.21.15.82:0/3136650401 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc6019e7f0 con 0x7fcc600ff510 2023-09-09T23:13:55.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.451+0000 7fcc47fff700 1 -- 172.21.15.82:0/3136650401 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc58003e10 con 0x7fcc600ff510 2023-09-09T23:13:55.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.451+0000 7fcc47fff700 1 -- 172.21.15.82:0/3136650401 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5801da20 con 0x7fcc600ff510 2023-09-09T23:13:55.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.451+0000 7fcc650cc700 1 -- 172.21.15.82:0/3136650401 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6019ec10 con 0x7fcc600ff510 2023-09-09T23:13:55.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.452+0000 7fcc650cc700 1 -- 172.21.15.82:0/3136650401 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc6004e3e0 con 0x7fcc600ff510 2023-09-09T23:13:55.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.453+0000 7fcc47fff700 1 -- 172.21.15.82:0/3136650401 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 21) v1 ==== 135071+0+0 (secure 0 0 0) 0x7fcc580056c0 con 0x7fcc600ff510 2023-09-09T23:13:55.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.454+0000 7fcc47fff700 1 --2- 172.21.15.82:0/3136650401 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fcc480a0440 0x7fcc480a2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:55.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.454+0000 7fcc5e59c700 1 --2- 172.21.15.82:0/3136650401 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fcc480a0440 0x7fcc480a2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:55.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.454+0000 7fcc47fff700 1 -- 172.21.15.82:0/3136650401 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fcc580ccd20 con 0x7fcc600ff510 2023-09-09T23:13:55.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.454+0000 7fcc5e59c700 1 --2- 172.21.15.82:0/3136650401 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fcc480a0440 0x7fcc480a2900 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcc60068a20 tx=0x7fcc5400b410 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:55.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.458+0000 7fcc47fff700 1 -- 172.21.15.82:0/3136650401 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fcc58096b20 con 0x7fcc600ff510 2023-09-09T23:13:55.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.689+0000 7fcc650cc700 1 -- 172.21.15.82:0/3136650401 --> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc60060ae0 con 0x7fcc480a0440 2023-09-09T23:13:55.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.744+0000 7fcc47fff700 1 -- 172.21.15.82:0/3136650401 <== mgr.15597 v2:172.21.15.82:6800/4121269095 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fcc60060ae0 con 0x7fcc480a0440 2023-09-09T23:13:55.743 INFO:teuthology.orchestra.run.smithi082.stdout:Retention added to path / 2023-09-09T23:13:55.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.746+0000 7fcc650cc700 1 -- 172.21.15.82:0/3136650401 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fcc480a0440 msgr2=0x7fcc480a2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:55.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.746+0000 7fcc650cc700 1 --2- 172.21.15.82:0/3136650401 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fcc480a0440 0x7fcc480a2900 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcc60068a20 tx=0x7fcc5400b410 comp rx=0 tx=0).stop 2023-09-09T23:13:55.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.746+0000 7fcc650cc700 1 -- 172.21.15.82:0/3136650401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc600ff510 msgr2=0x7fcc60199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:55.745 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.746+0000 7fcc650cc700 1 --2- 172.21.15.82:0/3136650401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc600ff510 0x7fcc60199200 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800b820 tx=0x7fcc580094c0 comp rx=0 tx=0).stop 2023-09-09T23:13:55.745 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.746+0000 7fcc650cc700 1 -- 172.21.15.82:0/3136650401 shutdown_connections 2023-09-09T23:13:55.745 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.746+0000 7fcc650cc700 1 --2- 172.21.15.82:0/3136650401 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fcc480a0440 0x7fcc480a2900 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:55.745 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.746+0000 7fcc650cc700 1 --2- 172.21.15.82:0/3136650401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc601010a0 0x7fcc6019dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:55.745 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.746+0000 7fcc650cc700 1 --2- 172.21.15.82:0/3136650401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc600ff510 0x7fcc60199200 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:55.746 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.746+0000 7fcc650cc700 1 --2- 172.21.15.82:0/3136650401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc60100700 0x7fcc6019b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:55.746 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.746+0000 7fcc650cc700 1 -- 172.21.15.82:0/3136650401 >> 172.21.15.82:0/3136650401 conn(0x7fcc600faa60 msgr2=0x7fcc600fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:55.746 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.746+0000 7fcc650cc700 1 -- 172.21.15.82:0/3136650401 shutdown_connections 2023-09-09T23:13:55.746 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.746+0000 7fcc650cc700 1 -- 172.21.15.82:0/3136650401 wait complete. 2023-09-09T23:13:55.757 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-09-09T23:13:55.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.887+0000 7f62d5c13700 1 -- 172.21.15.82:0/691192859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62d0100710 msgr2=0x7f62d0100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:55.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.887+0000 7f62d5c13700 1 --2- 172.21.15.82:0/691192859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62d0100710 0x7f62d0100b70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f62b80099c0 tx=0x7f62b8009cd0 comp rx=0 tx=0).stop 2023-09-09T23:13:55.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.887+0000 7f62d5c13700 1 -- 172.21.15.82:0/691192859 shutdown_connections 2023-09-09T23:13:55.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.887+0000 7f62d5c13700 1 --2- 172.21.15.82:0/691192859 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62d00ff520 0x7f62d00ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:55.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.887+0000 7f62d5c13700 1 --2- 172.21.15.82:0/691192859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62d01010b0 0x7f62d0107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:55.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.887+0000 7f62d5c13700 1 --2- 172.21.15.82:0/691192859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62d0100710 0x7f62d0100b70 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:55.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.887+0000 7f62d5c13700 1 -- 172.21.15.82:0/691192859 >> 172.21.15.82:0/691192859 conn(0x7f62d00faa90 msgr2=0x7f62d00fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:55.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.887+0000 7f62d5c13700 1 -- 172.21.15.82:0/691192859 shutdown_connections 2023-09-09T23:13:55.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.887+0000 7f62d5c13700 1 -- 172.21.15.82:0/691192859 wait complete. 2023-09-09T23:13:55.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.887+0000 7f62d5c13700 1 Processor -- start 2023-09-09T23:13:55.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.888+0000 7f62d5c13700 1 -- start start 2023-09-09T23:13:55.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.888+0000 7f62d5c13700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62d00ff520 0x7f62d0199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:55.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.888+0000 7f62d5c13700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62d0100710 0x7f62d019b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:55.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.888+0000 7f62d5c13700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62d01010b0 0x7f62d019db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:55.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.888+0000 7f62d5c13700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d019e0d0 con 0x7f62d00ff520 2023-09-09T23:13:55.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.888+0000 7f62d5c13700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d019e210 con 0x7f62d0100710 2023-09-09T23:13:55.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.888+0000 7f62d5c13700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d019e3d0 con 0x7f62d01010b0 2023-09-09T23:13:55.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.888+0000 7f62cf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62d00ff520 0x7f62d0199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:55.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.888+0000 7f62cf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62d00ff520 0x7f62d0199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57068/0 (socket says 172.21.15.82:57068) 2023-09-09T23:13:55.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.888+0000 7f62cf7fe700 1 -- 172.21.15.82:0/1314136788 learned_addr learned my addr 172.21.15.82:0/1314136788 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:55.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.888+0000 7f62ceffd700 1 --2- 172.21.15.82:0/1314136788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62d0100710 0x7f62d019b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:55.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.888+0000 7f62cffff700 1 --2- 172.21.15.82:0/1314136788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62d01010b0 0x7f62d019db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:55.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.888+0000 7f62cf7fe700 1 -- 172.21.15.82:0/1314136788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62d01010b0 msgr2=0x7f62d019db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:55.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.888+0000 7f62cf7fe700 1 --2- 172.21.15.82:0/1314136788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62d01010b0 0x7f62d019db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:55.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.888+0000 7f62cf7fe700 1 -- 172.21.15.82:0/1314136788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62d0100710 msgr2=0x7f62d019b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:55.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.888+0000 7f62cf7fe700 1 --2- 172.21.15.82:0/1314136788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62d0100710 0x7f62d019b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:55.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.888+0000 7f62cf7fe700 1 -- 172.21.15.82:0/1314136788 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b80095f0 con 0x7f62d00ff520 2023-09-09T23:13:55.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.888+0000 7f62cf7fe700 1 --2- 172.21.15.82:0/1314136788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62d00ff520 0x7f62d0199060 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f62c000d980 tx=0x7f62c000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:55.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.888+0000 7f62ccff9700 1 -- 172.21.15.82:0/1314136788 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c0004020 con 0x7f62d00ff520 2023-09-09T23:13:55.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.888+0000 7f62d5c13700 1 -- 172.21.15.82:0/1314136788 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d019e6b0 con 0x7f62d00ff520 2023-09-09T23:13:55.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.888+0000 7f62ccff9700 1 -- 172.21.15.82:0/1314136788 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62c000f690 con 0x7f62d00ff520 2023-09-09T23:13:55.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.889+0000 7f62ccff9700 1 -- 172.21.15.82:0/1314136788 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c0003a60 con 0x7f62d00ff520 2023-09-09T23:13:55.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.889+0000 7f62d5c13700 1 -- 172.21.15.82:0/1314136788 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d019ebc0 con 0x7f62d00ff520 2023-09-09T23:13:55.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.889+0000 7f62d5c13700 1 -- 172.21.15.82:0/1314136788 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d004e3e0 con 0x7f62d00ff520 2023-09-09T23:13:55.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.891+0000 7f62ccff9700 1 -- 172.21.15.82:0/1314136788 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 21) v1 ==== 135071+0+0 (secure 0 0 0) 0x7f62c000fb00 con 0x7f62d00ff520 2023-09-09T23:13:55.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.891+0000 7f62ccff9700 1 --2- 172.21.15.82:0/1314136788 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f62bc0a0440 0x7f62bc0a2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:55.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.891+0000 7f62ccff9700 1 -- 172.21.15.82:0/1314136788 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f62c00cb790 con 0x7f62d00ff520 2023-09-09T23:13:55.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.891+0000 7f62ceffd700 1 --2- 172.21.15.82:0/1314136788 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f62bc0a0440 0x7f62bc0a2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:55.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.892+0000 7f62ceffd700 1 --2- 172.21.15.82:0/1314136788 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f62bc0a0440 0x7f62bc0a2900 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f62d0068d70 tx=0x7f62b800b540 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:55.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:55.894+0000 7f62ccff9700 1 -- 172.21.15.82:0/1314136788 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f62c0095500 con 0x7f62d00ff520 2023-09-09T23:13:56.128 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.131+0000 7f62d5c13700 1 -- 172.21.15.82:0/1314136788 --> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f62d0060a40 con 0x7f62bc0a0440 2023-09-09T23:13:56.136 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.139+0000 7f62ccff9700 1 -- 172.21.15.82:0/1314136788 <== mgr.15597 v2:172.21.15.82:6800/4121269095 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7f62d0060a40 con 0x7f62bc0a0440 2023-09-09T23:13:56.136 INFO:teuthology.orchestra.run.smithi082.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-09-09T00:00:00", "created": "2023-09-09T23:13:55", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-09-09T23:13:56.142 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.141+0000 7f62d5c13700 1 -- 172.21.15.82:0/1314136788 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f62bc0a0440 msgr2=0x7f62bc0a2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:56.142 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.141+0000 7f62d5c13700 1 --2- 172.21.15.82:0/1314136788 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f62bc0a0440 0x7f62bc0a2900 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f62d0068d70 tx=0x7f62b800b540 comp rx=0 tx=0).stop 2023-09-09T23:13:56.143 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.141+0000 7f62d5c13700 1 -- 172.21.15.82:0/1314136788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62d00ff520 msgr2=0x7f62d0199060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:56.143 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.141+0000 7f62d5c13700 1 --2- 172.21.15.82:0/1314136788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62d00ff520 0x7f62d0199060 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f62c000d980 tx=0x7f62c000dc90 comp rx=0 tx=0).stop 2023-09-09T23:13:56.143 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.141+0000 7f62d5c13700 1 -- 172.21.15.82:0/1314136788 shutdown_connections 2023-09-09T23:13:56.144 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.141+0000 7f62d5c13700 1 --2- 172.21.15.82:0/1314136788 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f62bc0a0440 0x7f62bc0a2900 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:56.144 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.141+0000 7f62d5c13700 1 --2- 172.21.15.82:0/1314136788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62d01010b0 0x7f62d019db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:56.144 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.141+0000 7f62d5c13700 1 --2- 172.21.15.82:0/1314136788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62d00ff520 0x7f62d0199060 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:56.145 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.141+0000 7f62d5c13700 1 --2- 172.21.15.82:0/1314136788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62d0100710 0x7f62d019b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:56.145 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.141+0000 7f62d5c13700 1 -- 172.21.15.82:0/1314136788 >> 172.21.15.82:0/1314136788 conn(0x7f62d00faa90 msgr2=0x7f62d00fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:56.145 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.141+0000 7f62d5c13700 1 -- 172.21.15.82:0/1314136788 shutdown_connections 2023-09-09T23:13:56.146 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.141+0000 7f62d5c13700 1 -- 172.21.15.82:0/1314136788 wait complete. 2023-09-09T23:13:56.151 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-09-09T23:13:56.288 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.288+0000 7fecf6383700 1 -- 172.21.15.82:0/2029666571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecf0102230 msgr2=0x7fecf0108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:56.288 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.288+0000 7fecf6383700 1 --2- 172.21.15.82:0/2029666571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecf0102230 0x7fecf0108ae0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fece8009f90 tx=0x7fece800b330 comp rx=0 tx=0).stop 2023-09-09T23:13:56.288 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.288+0000 7fecf6383700 1 -- 172.21.15.82:0/2029666571 shutdown_connections 2023-09-09T23:13:56.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.288+0000 7fecf6383700 1 --2- 172.21.15.82:0/2029666571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecf00fe8c0 0x7fecf00fece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:56.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.288+0000 7fecf6383700 1 --2- 172.21.15.82:0/2029666571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecf0102230 0x7fecf0108ae0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:56.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.288+0000 7fecf6383700 1 --2- 172.21.15.82:0/2029666571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf0101890 0x7fecf0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:56.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.288+0000 7fecf6383700 1 -- 172.21.15.82:0/2029666571 >> 172.21.15.82:0/2029666571 conn(0x7fecf00fa1d0 msgr2=0x7fecf00fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:56.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.288+0000 7fecf6383700 1 -- 172.21.15.82:0/2029666571 shutdown_connections 2023-09-09T23:13:56.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.288+0000 7fecf6383700 1 -- 172.21.15.82:0/2029666571 wait complete. 2023-09-09T23:13:56.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.289+0000 7fecf6383700 1 Processor -- start 2023-09-09T23:13:56.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.289+0000 7fecf6383700 1 -- start start 2023-09-09T23:13:56.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.289+0000 7fecf6383700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf00fe8c0 0x7fecf01964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:56.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.289+0000 7fecf6383700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecf0101890 0x7fecf0196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:56.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.289+0000 7fecf6383700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecf0102230 0x7fecf0197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:56.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.289+0000 7fecf6383700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf0197ef0 con 0x7fecf00fe8c0 2023-09-09T23:13:56.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.289+0000 7fecf6383700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf0198060 con 0x7fecf0102230 2023-09-09T23:13:56.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.289+0000 7fecf6383700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf0198220 con 0x7fecf0101890 2023-09-09T23:13:56.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.289+0000 7feceffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf00fe8c0 0x7fecf01964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:56.292 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.290+0000 7feceffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf00fe8c0 0x7fecf01964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57098/0 (socket says 172.21.15.82:57098) 2023-09-09T23:13:56.292 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.290+0000 7feceffff700 1 -- 172.21.15.82:0/2232751707 learned_addr learned my addr 172.21.15.82:0/2232751707 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:13:56.292 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.290+0000 7fecf4920700 1 --2- 172.21.15.82:0/2232751707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecf0102230 0x7fecf0197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:56.292 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.290+0000 7fecef7fe700 1 --2- 172.21.15.82:0/2232751707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecf0101890 0x7fecf0196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:56.292 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.290+0000 7feceffff700 1 -- 172.21.15.82:0/2232751707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecf0101890 msgr2=0x7fecf0196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:56.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.290+0000 7feceffff700 1 --2- 172.21.15.82:0/2232751707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecf0101890 0x7fecf0196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:56.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.290+0000 7feceffff700 1 -- 172.21.15.82:0/2232751707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecf0102230 msgr2=0x7fecf0197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:56.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.290+0000 7feceffff700 1 --2- 172.21.15.82:0/2232751707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecf0102230 0x7fecf0197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:56.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.290+0000 7feceffff700 1 -- 172.21.15.82:0/2232751707 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece40095f0 con 0x7fecf00fe8c0 2023-09-09T23:13:56.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.290+0000 7feceffff700 1 --2- 172.21.15.82:0/2232751707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf00fe8c0 0x7fecf01964d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fece000e970 tx=0x7fece000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:56.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.290+0000 7feced7fa700 1 -- 172.21.15.82:0/2232751707 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece0009790 con 0x7fecf00fe8c0 2023-09-09T23:13:56.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.290+0000 7fecf6383700 1 -- 172.21.15.82:0/2232751707 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece8009c40 con 0x7fecf00fe8c0 2023-09-09T23:13:56.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.290+0000 7feced7fa700 1 -- 172.21.15.82:0/2232751707 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fece000eec0 con 0x7fecf00fe8c0 2023-09-09T23:13:56.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.290+0000 7feced7fa700 1 -- 172.21.15.82:0/2232751707 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece0003cf0 con 0x7fecf00fe8c0 2023-09-09T23:13:56.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.290+0000 7fecf6383700 1 -- 172.21.15.82:0/2232751707 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf01986f0 con 0x7fecf00fe8c0 2023-09-09T23:13:56.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.291+0000 7fecf6383700 1 -- 172.21.15.82:0/2232751707 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecf004e3e0 con 0x7fecf00fe8c0 2023-09-09T23:13:56.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.292+0000 7feced7fa700 1 -- 172.21.15.82:0/2232751707 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 21) v1 ==== 135071+0+0 (secure 0 0 0) 0x7fece000ccd0 con 0x7fecf00fe8c0 2023-09-09T23:13:56.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.293+0000 7feced7fa700 1 --2- 172.21.15.82:0/2232751707 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fecd80a0440 0x7fecd80a2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:13:56.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.293+0000 7feced7fa700 1 -- 172.21.15.82:0/2232751707 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fece0014070 con 0x7fecf00fe8c0 2023-09-09T23:13:56.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.293+0000 7fecef7fe700 1 --2- 172.21.15.82:0/2232751707 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fecd80a0440 0x7fecd80a2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:13:56.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.293+0000 7fecef7fe700 1 --2- 172.21.15.82:0/2232751707 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fecd80a0440 0x7fecd80a2900 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fece4005fd0 tx=0x7fece4009310 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:13:56.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.297+0000 7feced7fa700 1 -- 172.21.15.82:0/2232751707 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fece0095830 con 0x7fecf00fe8c0 2023-09-09T23:13:56.454 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:56 smithi082 ceph-mon[98694]: [09/Sep/2023:23:13:55] ENGINE Bus STARTING 2023-09-09T23:13:56.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:56 smithi082 ceph-mon[98694]: mgrmap e21: x(active, since 1.01775s), standbys: y, z 2023-09-09T23:13:56.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:56 smithi082 ceph-mon[98694]: from='client.15705 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:13:56.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:56 smithi082 ceph-mon[98694]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:13:56.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:56 smithi082 ceph-mon[98694]: [09/Sep/2023:23:13:55] ENGINE Serving on http://172.21.15.82:8765 2023-09-09T23:13:56.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:56 smithi082 ceph-mon[98694]: [09/Sep/2023:23:13:55] ENGINE Serving on https://172.21.15.82:7150 2023-09-09T23:13:56.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:56 smithi082 ceph-mon[98694]: [09/Sep/2023:23:13:55] ENGINE Bus STARTED 2023-09-09T23:13:56.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:56 smithi098 ceph-mon[103124]: [09/Sep/2023:23:13:55] ENGINE Bus STARTING 2023-09-09T23:13:56.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:56 smithi098 ceph-mon[103124]: mgrmap e21: x(active, since 1.01775s), standbys: y, z 2023-09-09T23:13:56.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:56 smithi098 ceph-mon[103124]: from='client.15705 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:13:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:56 smithi098 ceph-mon[103124]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:13:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:56 smithi098 ceph-mon[103124]: [09/Sep/2023:23:13:55] ENGINE Serving on http://172.21.15.82:8765 2023-09-09T23:13:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:56 smithi098 ceph-mon[103124]: [09/Sep/2023:23:13:55] ENGINE Serving on https://172.21.15.82:7150 2023-09-09T23:13:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:56 smithi098 ceph-mon[103124]: [09/Sep/2023:23:13:55] ENGINE Bus STARTED 2023-09-09T23:13:56.521 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.524+0000 7fecf6383700 1 -- 172.21.15.82:0/2232751707 --> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fecf0060a40 con 0x7fecd80a0440 2023-09-09T23:13:56.528 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.531+0000 7feced7fa700 1 -- 172.21.15.82:0/2232751707 <== mgr.15597 v2:172.21.15.82:6800/4121269095 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fecf0060a40 con 0x7fecd80a0440 2023-09-09T23:13:56.528 INFO:teuthology.orchestra.run.smithi082.stdout:/ 1M 6M3h 2023-09-09T23:13:56.531 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.533+0000 7fecf6383700 1 -- 172.21.15.82:0/2232751707 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fecd80a0440 msgr2=0x7fecd80a2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:56.531 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.533+0000 7fecf6383700 1 --2- 172.21.15.82:0/2232751707 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fecd80a0440 0x7fecd80a2900 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fece4005fd0 tx=0x7fece4009310 comp rx=0 tx=0).stop 2023-09-09T23:13:56.532 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.533+0000 7fecf6383700 1 -- 172.21.15.82:0/2232751707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf00fe8c0 msgr2=0x7fecf01964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:13:56.532 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.533+0000 7fecf6383700 1 --2- 172.21.15.82:0/2232751707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf00fe8c0 0x7fecf01964d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fece000e970 tx=0x7fece000ec80 comp rx=0 tx=0).stop 2023-09-09T23:13:56.532 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.533+0000 7fecf6383700 1 -- 172.21.15.82:0/2232751707 shutdown_connections 2023-09-09T23:13:56.532 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.533+0000 7fecf6383700 1 --2- 172.21.15.82:0/2232751707 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fecd80a0440 0x7fecd80a2900 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:56.533 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.533+0000 7fecf6383700 1 --2- 172.21.15.82:0/2232751707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecf0102230 0x7fecf0197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:56.533 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.533+0000 7fecf6383700 1 --2- 172.21.15.82:0/2232751707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf00fe8c0 0x7fecf01964d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:56.533 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.533+0000 7fecf6383700 1 --2- 172.21.15.82:0/2232751707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecf0101890 0x7fecf0196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:13:56.533 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.533+0000 7fecf6383700 1 -- 172.21.15.82:0/2232751707 >> 172.21.15.82:0/2232751707 conn(0x7fecf00fa1d0 msgr2=0x7fecf00fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:13:56.533 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.533+0000 7fecf6383700 1 -- 172.21.15.82:0/2232751707 shutdown_connections 2023-09-09T23:13:56.534 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:13:56.533+0000 7fecf6383700 1 -- 172.21.15.82:0/2232751707 wait complete. 2023-09-09T23:13:56.545 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-09-09T23:13:56.574 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-09-09T23:13:56.586 INFO:teuthology.run_tasks:Running task exec... 2023-09-09T23:13:56.655 INFO:teuthology.task.exec:Executing custom commands... 2023-09-09T23:13:56.655 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi082.front.sepia.ceph.com 2023-09-09T23:13:56.655 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2023-09-09T23:13:56.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:56 smithi170 ceph-mon[102832]: [09/Sep/2023:23:13:55] ENGINE Bus STARTING 2023-09-09T23:13:56.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:56 smithi170 ceph-mon[102832]: mgrmap e21: x(active, since 1.01775s), standbys: y, z 2023-09-09T23:13:56.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:56 smithi170 ceph-mon[102832]: from='client.15705 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:13:56.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:56 smithi170 ceph-mon[102832]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:13:56.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:56 smithi170 ceph-mon[102832]: [09/Sep/2023:23:13:55] ENGINE Serving on http://172.21.15.82:8765 2023-09-09T23:13:56.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:56 smithi170 ceph-mon[102832]: [09/Sep/2023:23:13:55] ENGINE Serving on https://172.21.15.82:7150 2023-09-09T23:13:56.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:56 smithi170 ceph-mon[102832]: [09/Sep/2023:23:13:55] ENGINE Bus STARTED 2023-09-09T23:13:56.695 INFO:teuthology.orchestra.run.smithi082.stdout:total 0 2023-09-09T23:13:56.757 INFO:teuthology.orchestra.run.smithi082.stdout:Filesystem Size Used Avail Use% Mounted on 2023-09-09T23:13:56.757 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:3300,172.21.15.98:3300,172.21.15.170:3300:/volumes/_nogroup/sv_1/0a302eaa-eeda-4440-b80e-72c42f5e6ec4 340G 0 340G 0% /tmp/cephfs 2023-09-09T23:13:56.759 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2023-09-09T23:13:56.856 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2023-09-09T23:13:56.897 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2023-09-09T23:13:57.454 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:57 smithi082 ceph-mon[98694]: from='client.15780 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:13:57.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:57 smithi082 ceph-mon[98694]: from='client.15786 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:13:57.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:57 smithi082 ceph-mon[98694]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:13:57.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:57.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:57.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:57.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:57 smithi098 ceph-mon[103124]: from='client.15780 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:13:57.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:57 smithi098 ceph-mon[103124]: from='client.15786 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:13:57.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:57 smithi098 ceph-mon[103124]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:13:57.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:57.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:57.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:57.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:57 smithi170 ceph-mon[102832]: from='client.15780 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:13:57.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:57 smithi170 ceph-mon[102832]: from='client.15786 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:13:57.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:57 smithi170 ceph-mon[102832]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:13:57.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:57.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:57.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:58.454 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:58 smithi082 ceph-mon[98694]: from='client.15798 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:13:58.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:58.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:58 smithi082 ceph-mon[98694]: mgrmap e22: x(active, since 3s), standbys: y, z 2023-09-09T23:13:58.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:58.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:58.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:58.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:58.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:58.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:58.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:58.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:58 smithi098 ceph-mon[103124]: from='client.15798 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:13:58.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:58.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:58 smithi098 ceph-mon[103124]: mgrmap e22: x(active, since 3s), standbys: y, z 2023-09-09T23:13:58.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:58.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:58.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:58.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:58.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:58.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:58.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:58 smithi170 ceph-mon[102832]: from='client.15798 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-09T23:13:58.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:58.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:58 smithi170 ceph-mon[102832]: mgrmap e22: x(active, since 3s), standbys: y, z 2023-09-09T23:13:58.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:58.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:58.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:58.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:59.452 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2023-09-09T23:13:59.480 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:59 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-09T23:13:59.480 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:59 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-09T23:13:59.480 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:59 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-09T23:13:59.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:59 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-09T23:13:59.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:59 smithi082 ceph-mon[98694]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:13:59.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:59.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:59.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:59.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:59.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:59.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:59.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:59 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-09T23:13:59.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:59 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-09T23:13:59.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:13:59.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:13:59.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:59 smithi082 ceph-mon[98694]: Updating smithi082:/etc/ceph/ceph.conf 2023-09-09T23:13:59.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:59 smithi082 ceph-mon[98694]: Updating smithi098:/etc/ceph/ceph.conf 2023-09-09T23:13:59.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:59 smithi082 ceph-mon[98694]: Updating smithi170:/etc/ceph/ceph.conf 2023-09-09T23:13:59.484 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:13:59 smithi082 ceph-mon[98694]: mgrmap e23: x(active, since 4s), standbys: y, z 2023-09-09T23:13:59.493 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2023-09-09T23:13:59.495 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2023-09-09T23:13:59.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:59 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-09T23:13:59.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:59 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-09T23:13:59.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:59 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-09T23:13:59.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:59 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-09T23:13:59.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:59 smithi098 ceph-mon[103124]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:13:59.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:59.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:59.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:59 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-09T23:13:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:59 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-09T23:13:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:13:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:13:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:59 smithi098 ceph-mon[103124]: Updating smithi082:/etc/ceph/ceph.conf 2023-09-09T23:13:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:59 smithi098 ceph-mon[103124]: Updating smithi098:/etc/ceph/ceph.conf 2023-09-09T23:13:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:59 smithi098 ceph-mon[103124]: Updating smithi170:/etc/ceph/ceph.conf 2023-09-09T23:13:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:13:59 smithi098 ceph-mon[103124]: mgrmap e23: x(active, since 4s), standbys: y, z 2023-09-09T23:13:59.577 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-09-09T23:13:59.656 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/: 2023-09-09T23:13:59.656 INFO:teuthology.orchestra.run.smithi082.stdout:total 0 2023-09-09T23:13:59.657 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 9 23:13 . 2023-09-09T23:13:59.657 INFO:teuthology.orchestra.run.smithi082.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 9 23:13 .. 2023-09-09T23:13:59.657 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:13 data 2023-09-09T23:13:59.657 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:13:59.657 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/data: 2023-09-09T23:13:59.658 INFO:teuthology.orchestra.run.smithi082.stdout:total 0 2023-09-09T23:13:59.658 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:13 . 2023-09-09T23:13:59.658 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 9 23:13 .. 2023-09-09T23:13:59.659 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2023-09-09T23:13:59.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:59 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-09T23:13:59.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:59 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-09T23:13:59.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:59 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-09T23:13:59.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:59 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-09T23:13:59.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:59 smithi170 ceph-mon[102832]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:13:59.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:59.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:13:59.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:13:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:59 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-09T23:13:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:59 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-09T23:13:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:13:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:13:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:59 smithi170 ceph-mon[102832]: Updating smithi082:/etc/ceph/ceph.conf 2023-09-09T23:13:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:59 smithi170 ceph-mon[102832]: Updating smithi098:/etc/ceph/ceph.conf 2023-09-09T23:13:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:59 smithi170 ceph-mon[102832]: Updating smithi170:/etc/ceph/ceph.conf 2023-09-09T23:13:59.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:13:59 smithi170 ceph-mon[102832]: mgrmap e23: x(active, since 4s), standbys: y, z 2023-09-09T23:13:59.746 INFO:teuthology.orchestra.run.smithi082.stdout:[Service] 2023-09-09T23:13:59.747 INFO:teuthology.orchestra.run.smithi082.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2023-09-09T23:13:59.747 INFO:teuthology.orchestra.run.smithi082.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2023-09-09T23:13:59.749 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2023-09-09T23:13:59.837 INFO:teuthology.orchestra.run.smithi082.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2023-09-09T23:14:00.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:00 smithi170 ceph-mon[102832]: Updating smithi170:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-09T23:14:00.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:00 smithi170 ceph-mon[102832]: Updating smithi098:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-09T23:14:00.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:00 smithi170 ceph-mon[102832]: Updating smithi082:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-09T23:14:00.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:00 smithi170 ceph-mon[102832]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2023-09-09T23:14:00.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:00 smithi170 ceph-mon[102832]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2023-09-09T23:14:00.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:00 smithi170 ceph-mon[102832]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2023-09-09T23:14:00.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:00 smithi170 ceph-mon[102832]: Updating smithi170:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-09T23:14:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:00 smithi170 ceph-mon[102832]: Updating smithi098:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-09T23:14:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:00 smithi170 ceph-mon[102832]: Updating smithi082:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-09T23:14:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:14:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:14:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:14:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:14:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:14:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:14:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:14:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:14:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:14:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:14:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:14:00.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:00 smithi082 ceph-mon[98694]: Updating smithi170:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-09T23:14:00.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:00 smithi082 ceph-mon[98694]: Updating smithi098:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-09T23:14:00.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:00 smithi082 ceph-mon[98694]: Updating smithi082:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-09T23:14:00.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:00 smithi082 ceph-mon[98694]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2023-09-09T23:14:00.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:00 smithi082 ceph-mon[98694]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2023-09-09T23:14:00.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:00 smithi082 ceph-mon[98694]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2023-09-09T23:14:00.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:00 smithi082 ceph-mon[98694]: Updating smithi170:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-09T23:14:00.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:00 smithi082 ceph-mon[98694]: Updating smithi098:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-09T23:14:00.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:00 smithi082 ceph-mon[98694]: Updating smithi082:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-09T23:14:00.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:14:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:14:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:14:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:14:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:14:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:14:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:14:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:14:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:14:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:14:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:14:00.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:00 smithi098 ceph-mon[103124]: Updating smithi170:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-09T23:14:00.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:00 smithi098 ceph-mon[103124]: Updating smithi098:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-09T23:14:00.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:00 smithi098 ceph-mon[103124]: Updating smithi082:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.conf 2023-09-09T23:14:00.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:00 smithi098 ceph-mon[103124]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2023-09-09T23:14:00.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:00 smithi098 ceph-mon[103124]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2023-09-09T23:14:00.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:00 smithi098 ceph-mon[103124]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2023-09-09T23:14:00.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:00 smithi098 ceph-mon[103124]: Updating smithi170:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-09T23:14:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:00 smithi098 ceph-mon[103124]: Updating smithi098:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-09T23:14:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:00 smithi098 ceph-mon[103124]: Updating smithi082:/var/lib/ceph/3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-09T23:14:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:14:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:14:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:14:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:14:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:14:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:14:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:14:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:14:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:14:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:14:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:14:01.204 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:14:00 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:14:00.753+0000 7f09b432f700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-09T23:14:01.205 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:14:00 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:14:00.753+0000 7f09b432f700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-09T23:14:01.205 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:14:00 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:14:00.753+0000 7f09b432f700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-09T23:14:01.205 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:14:00 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:14:00.753+0000 7f09b432f700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-09T23:14:01.205 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Sep 09 23:14:00 smithi082 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-x[98938]: 2023-09-09T23:14:00.753+0000 7f09b432f700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-09T23:14:01.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:01 smithi170 ceph-mon[102832]: pgmap v6: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:14:01.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:01 smithi082 ceph-mon[98694]: pgmap v6: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:14:01.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:01 smithi098 ceph-mon[103124]: pgmap v6: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-09T23:14:03.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:03 smithi170 ceph-mon[102832]: pgmap v7: 97 pgs: 97 active+clean; 1.4 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 152 KiB/s wr, 73 op/s 2023-09-09T23:14:03.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:03 smithi170 ceph-mon[102832]: mgrmap e24: x(active, since 8s), standbys: y, z 2023-09-09T23:14:03.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:03 smithi082 ceph-mon[98694]: pgmap v7: 97 pgs: 97 active+clean; 1.4 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 152 KiB/s wr, 73 op/s 2023-09-09T23:14:03.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:03 smithi082 ceph-mon[98694]: mgrmap e24: x(active, since 8s), standbys: y, z 2023-09-09T23:14:03.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:03 smithi098 ceph-mon[103124]: pgmap v7: 97 pgs: 97 active+clean; 1.4 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 152 KiB/s wr, 73 op/s 2023-09-09T23:14:03.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:03 smithi098 ceph-mon[103124]: mgrmap e24: x(active, since 8s), standbys: y, z 2023-09-09T23:14:05.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:05 smithi170 ceph-mon[102832]: pgmap v8: 97 pgs: 97 active+clean; 1.4 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 118 KiB/s wr, 57 op/s 2023-09-09T23:14:05.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:05 smithi082 ceph-mon[98694]: pgmap v8: 97 pgs: 97 active+clean; 1.4 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 118 KiB/s wr, 57 op/s 2023-09-09T23:14:05.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:05 smithi098 ceph-mon[103124]: pgmap v8: 97 pgs: 97 active+clean; 1.4 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 118 KiB/s wr, 57 op/s 2023-09-09T23:14:07.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:07 smithi170 ceph-mon[102832]: pgmap v9: 97 pgs: 97 active+clean; 9.9 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 817 KiB/s wr, 214 op/s 2023-09-09T23:14:07.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:07 smithi082 ceph-mon[98694]: pgmap v9: 97 pgs: 97 active+clean; 9.9 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 817 KiB/s wr, 214 op/s 2023-09-09T23:14:07.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:07 smithi098 ceph-mon[103124]: pgmap v9: 97 pgs: 97 active+clean; 9.9 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 817 KiB/s wr, 214 op/s 2023-09-09T23:14:09.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:09 smithi170 ceph-mon[102832]: pgmap v10: 97 pgs: 97 active+clean; 9.9 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 748 KiB/s wr, 196 op/s 2023-09-09T23:14:09.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:09 smithi082 ceph-mon[98694]: pgmap v10: 97 pgs: 97 active+clean; 9.9 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 748 KiB/s wr, 196 op/s 2023-09-09T23:14:09.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:09 smithi098 ceph-mon[103124]: pgmap v10: 97 pgs: 97 active+clean; 9.9 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 748 KiB/s wr, 196 op/s 2023-09-09T23:14:11.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:11 smithi170 ceph-mon[102832]: pgmap v11: 97 pgs: 97 active+clean; 9.9 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 748 KiB/s wr, 196 op/s 2023-09-09T23:14:11.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:11 smithi082 ceph-mon[98694]: pgmap v11: 97 pgs: 97 active+clean; 9.9 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 748 KiB/s wr, 196 op/s 2023-09-09T23:14:11.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:11 smithi098 ceph-mon[103124]: pgmap v11: 97 pgs: 97 active+clean; 9.9 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 748 KiB/s wr, 196 op/s 2023-09-09T23:14:12.814 INFO:teuthology.orchestra.run.smithi082.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2023-09-09T23:14:12.816 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-09-09T23:14:12.860 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/: 2023-09-09T23:14:12.860 INFO:teuthology.orchestra.run.smithi082.stdout:total 0 2023-09-09T23:14:12.860 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 9 23:13 . 2023-09-09T23:14:12.860 INFO:teuthology.orchestra.run.smithi082.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 9 23:13 .. 2023-09-09T23:14:12.861 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 9 23:14 data 2023-09-09T23:14:12.861 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:14:12.861 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/data: 2023-09-09T23:14:12.861 INFO:teuthology.orchestra.run.smithi082.stdout:total 30 2023-09-09T23:14:12.861 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 9 23:14 . 2023-09-09T23:14:12.862 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 9 23:13 .. 2023-09-09T23:14:12.862 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 9 23:14 base 2023-09-09T23:14:12.862 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Sep 9 23:14 global 2023-09-09T23:14:12.862 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 log 2023-09-09T23:14:12.862 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:13 pg_commit_ts 2023-09-09T23:14:12.863 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:13 pg_dynshmem 2023-09-09T23:14:12.863 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Sep 9 23:13 pg_hba.conf 2023-09-09T23:14:12.863 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 9 23:13 pg_ident.conf 2023-09-09T23:14:12.863 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 9 23:14 pg_logical 2023-09-09T23:14:12.864 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 9 23:13 pg_multixact 2023-09-09T23:14:12.864 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 9 23:14 pg_notify 2023-09-09T23:14:12.864 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:13 pg_replslot 2023-09-09T23:14:12.864 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:13 pg_serial 2023-09-09T23:14:12.864 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:13 pg_snapshots 2023-09-09T23:14:12.864 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:13 pg_stat 2023-09-09T23:14:12.865 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:13 pg_stat_tmp 2023-09-09T23:14:12.865 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 9 23:13 pg_subtrans 2023-09-09T23:14:12.865 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:13 pg_tblspc 2023-09-09T23:14:12.865 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:13 pg_twophase 2023-09-09T23:14:12.865 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 9 23:13 PG_VERSION 2023-09-09T23:14:12.866 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 9 23:13 pg_wal 2023-09-09T23:14:12.866 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 9 23:13 pg_xact 2023-09-09T23:14:12.866 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 9 23:13 postgresql.auto.conf 2023-09-09T23:14:12.866 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Sep 9 23:13 postgresql.conf 2023-09-09T23:14:12.866 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:14:12.867 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/data/base: 2023-09-09T23:14:12.867 INFO:teuthology.orchestra.run.smithi082.stdout:total 0 2023-09-09T23:14:12.867 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 9 23:14 . 2023-09-09T23:14:12.867 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 9 23:14 .. 2023-09-09T23:14:12.867 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 9 23:14 1 2023-09-09T23:14:12.868 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 9 23:14 12346 2023-09-09T23:14:12.868 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 9 23:14 12347 2023-09-09T23:14:12.868 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:14:12.868 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/data/base/1: 2023-09-09T23:14:12.868 INFO:teuthology.orchestra.run.smithi082.stdout:total 7185 2023-09-09T23:14:12.869 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 9 23:14 . 2023-09-09T23:14:12.869 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 9 23:14 .. 2023-09-09T23:14:12.869 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 112 2023-09-09T23:14:12.869 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 113 2023-09-09T23:14:12.869 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 9 23:14 12182 2023-09-09T23:14:12.869 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 12182_fsm 2023-09-09T23:14:12.870 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12182_vm 2023-09-09T23:14:12.870 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 12184 2023-09-09T23:14:12.870 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12186 2023-09-09T23:14:12.870 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12187 2023-09-09T23:14:12.870 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 12187_fsm 2023-09-09T23:14:12.871 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12187_vm 2023-09-09T23:14:12.871 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 12189 2023-09-09T23:14:12.871 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12191 2023-09-09T23:14:12.871 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12192 2023-09-09T23:14:12.871 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 12192_fsm 2023-09-09T23:14:12.872 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12192_vm 2023-09-09T23:14:12.872 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 12194 2023-09-09T23:14:12.872 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12196 2023-09-09T23:14:12.872 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12197 2023-09-09T23:14:12.872 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 12197_fsm 2023-09-09T23:14:12.873 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12197_vm 2023-09-09T23:14:12.873 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 12199 2023-09-09T23:14:12.873 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12201 2023-09-09T23:14:12.873 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12202 2023-09-09T23:14:12.873 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 12202_fsm 2023-09-09T23:14:12.873 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12202_vm 2023-09-09T23:14:12.874 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 12204 2023-09-09T23:14:12.874 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12206 2023-09-09T23:14:12.874 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12207 2023-09-09T23:14:12.874 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 12207_fsm 2023-09-09T23:14:12.874 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12207_vm 2023-09-09T23:14:12.875 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 12209 2023-09-09T23:14:12.875 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12211 2023-09-09T23:14:12.875 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 12212 2023-09-09T23:14:12.875 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 12214 2023-09-09T23:14:12.875 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12216 2023-09-09T23:14:12.876 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Sep 9 23:14 1247 2023-09-09T23:14:12.876 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 1247_fsm 2023-09-09T23:14:12.876 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 1247_vm 2023-09-09T23:14:12.876 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Sep 9 23:14 1249 2023-09-09T23:14:12.876 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 1249_fsm 2023-09-09T23:14:12.877 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 1249_vm 2023-09-09T23:14:12.877 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Sep 9 23:14 1255 2023-09-09T23:14:12.877 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 1255_fsm 2023-09-09T23:14:12.877 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 1255_vm 2023-09-09T23:14:12.877 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 9 23:14 1259 2023-09-09T23:14:12.877 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 1259_fsm 2023-09-09T23:14:12.878 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 1259_vm 2023-09-09T23:14:12.878 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 1417 2023-09-09T23:14:12.878 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 1417_vm 2023-09-09T23:14:12.878 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 1418 2023-09-09T23:14:12.878 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 1418_vm 2023-09-09T23:14:12.879 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 174 2023-09-09T23:14:12.879 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 175 2023-09-09T23:14:12.879 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2187 2023-09-09T23:14:12.879 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2224 2023-09-09T23:14:12.879 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2224_vm 2023-09-09T23:14:12.880 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2328 2023-09-09T23:14:12.880 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2328_vm 2023-09-09T23:14:12.880 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2336 2023-09-09T23:14:12.880 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2336_vm 2023-09-09T23:14:12.880 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2337 2023-09-09T23:14:12.881 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2600 2023-09-09T23:14:12.881 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2600_fsm 2023-09-09T23:14:12.881 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2600_vm 2023-09-09T23:14:12.881 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2601 2023-09-09T23:14:12.881 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2601_fsm 2023-09-09T23:14:12.882 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2601_vm 2023-09-09T23:14:12.882 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 9 23:14 2602 2023-09-09T23:14:12.882 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2602_fsm 2023-09-09T23:14:12.882 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2602_vm 2023-09-09T23:14:12.882 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 9 23:14 2603 2023-09-09T23:14:12.883 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2603_fsm 2023-09-09T23:14:12.883 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2603_vm 2023-09-09T23:14:12.883 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2604 2023-09-09T23:14:12.883 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2604_vm 2023-09-09T23:14:12.883 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2605 2023-09-09T23:14:12.883 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2605_fsm 2023-09-09T23:14:12.884 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2605_vm 2023-09-09T23:14:12.884 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2606 2023-09-09T23:14:12.884 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2606_fsm 2023-09-09T23:14:12.884 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2606_vm 2023-09-09T23:14:12.884 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2607 2023-09-09T23:14:12.885 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2607_fsm 2023-09-09T23:14:12.885 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2607_vm 2023-09-09T23:14:12.885 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Sep 9 23:14 2608 2023-09-09T23:14:12.885 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2608_fsm 2023-09-09T23:14:12.885 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2608_vm 2023-09-09T23:14:12.886 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Sep 9 23:14 2609 2023-09-09T23:14:12.886 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2609_fsm 2023-09-09T23:14:12.886 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2609_vm 2023-09-09T23:14:12.886 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2610 2023-09-09T23:14:12.886 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2610_fsm 2023-09-09T23:14:12.887 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2610_vm 2023-09-09T23:14:12.887 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2611 2023-09-09T23:14:12.887 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2611_vm 2023-09-09T23:14:12.887 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2612 2023-09-09T23:14:12.887 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2612_fsm 2023-09-09T23:14:12.887 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2612_vm 2023-09-09T23:14:12.888 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2613 2023-09-09T23:14:12.888 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2613_vm 2023-09-09T23:14:12.888 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2615 2023-09-09T23:14:12.888 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2615_fsm 2023-09-09T23:14:12.888 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2615_vm 2023-09-09T23:14:12.889 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2616 2023-09-09T23:14:12.889 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2616_fsm 2023-09-09T23:14:12.889 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2616_vm 2023-09-09T23:14:12.889 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 9 23:14 2617 2023-09-09T23:14:12.889 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2617_fsm 2023-09-09T23:14:12.889 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2617_vm 2023-09-09T23:14:12.890 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 9 23:14 2618 2023-09-09T23:14:12.890 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2618_fsm 2023-09-09T23:14:12.890 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2618_vm 2023-09-09T23:14:12.890 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 9 23:14 2619 2023-09-09T23:14:12.890 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2619_fsm 2023-09-09T23:14:12.891 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2619_vm 2023-09-09T23:14:12.891 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2620 2023-09-09T23:14:12.891 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2620_vm 2023-09-09T23:14:12.891 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2650 2023-09-09T23:14:12.891 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2651 2023-09-09T23:14:12.892 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2652 2023-09-09T23:14:12.892 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 9 23:14 2653 2023-09-09T23:14:12.892 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 9 23:14 2654 2023-09-09T23:14:12.892 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 9 23:14 2655 2023-09-09T23:14:12.892 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2656 2023-09-09T23:14:12.893 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2657 2023-09-09T23:14:12.893 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 9 23:14 2658 2023-09-09T23:14:12.893 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 9 23:14 2659 2023-09-09T23:14:12.893 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2660 2023-09-09T23:14:12.893 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2661 2023-09-09T23:14:12.894 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 9 23:14 2662 2023-09-09T23:14:12.894 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 9 23:14 2663 2023-09-09T23:14:12.894 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2664 2023-09-09T23:14:12.894 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2665 2023-09-09T23:14:12.894 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2666 2023-09-09T23:14:12.895 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2667 2023-09-09T23:14:12.895 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2668 2023-09-09T23:14:12.895 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2669 2023-09-09T23:14:12.895 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2670 2023-09-09T23:14:12.895 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 311296 Sep 9 23:14 2673 2023-09-09T23:14:12.895 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Sep 9 23:14 2674 2023-09-09T23:14:12.896 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 9 23:14 2675 2023-09-09T23:14:12.896 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2678 2023-09-09T23:14:12.896 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2679 2023-09-09T23:14:12.896 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2680 2023-09-09T23:14:12.896 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2681 2023-09-09T23:14:12.897 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2682 2023-09-09T23:14:12.897 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2683 2023-09-09T23:14:12.897 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2684 2023-09-09T23:14:12.897 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2685 2023-09-09T23:14:12.897 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2686 2023-09-09T23:14:12.898 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2687 2023-09-09T23:14:12.898 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 9 23:14 2688 2023-09-09T23:14:12.898 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 9 23:14 2689 2023-09-09T23:14:12.898 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 9 23:14 2690 2023-09-09T23:14:12.898 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Sep 9 23:14 2691 2023-09-09T23:14:12.899 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2692 2023-09-09T23:14:12.899 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2693 2023-09-09T23:14:12.899 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2696 2023-09-09T23:14:12.899 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2699 2023-09-09T23:14:12.899 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2701 2023-09-09T23:14:12.899 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2702 2023-09-09T23:14:12.900 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2703 2023-09-09T23:14:12.900 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 9 23:14 2704 2023-09-09T23:14:12.900 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2753 2023-09-09T23:14:12.900 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2753_fsm 2023-09-09T23:14:12.900 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2753_vm 2023-09-09T23:14:12.901 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2754 2023-09-09T23:14:12.901 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2755 2023-09-09T23:14:12.901 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 9 23:14 2756 2023-09-09T23:14:12.901 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 9 23:14 2757 2023-09-09T23:14:12.901 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2830 2023-09-09T23:14:12.902 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2830_vm 2023-09-09T23:14:12.902 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2831 2023-09-09T23:14:12.902 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2832 2023-09-09T23:14:12.902 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2832_vm 2023-09-09T23:14:12.902 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2833 2023-09-09T23:14:12.903 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2834 2023-09-09T23:14:12.903 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2834_vm 2023-09-09T23:14:12.903 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2835 2023-09-09T23:14:12.903 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2836 2023-09-09T23:14:12.903 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2836_vm 2023-09-09T23:14:12.904 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2837 2023-09-09T23:14:12.904 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Sep 9 23:14 2838 2023-09-09T23:14:12.904 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2838_fsm 2023-09-09T23:14:12.904 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2838_vm 2023-09-09T23:14:12.904 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2839 2023-09-09T23:14:12.904 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2840 2023-09-09T23:14:12.905 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2840_fsm 2023-09-09T23:14:12.905 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2840_vm 2023-09-09T23:14:12.905 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2841 2023-09-09T23:14:12.905 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2995 2023-09-09T23:14:12.905 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2995_vm 2023-09-09T23:14:12.906 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2996 2023-09-09T23:14:12.906 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3079 2023-09-09T23:14:12.906 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 3079_fsm 2023-09-09T23:14:12.906 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3079_vm 2023-09-09T23:14:12.906 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3080 2023-09-09T23:14:12.907 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3081 2023-09-09T23:14:12.907 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3085 2023-09-09T23:14:12.907 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3118 2023-09-09T23:14:12.907 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3118_vm 2023-09-09T23:14:12.907 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3119 2023-09-09T23:14:12.907 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3164 2023-09-09T23:14:12.908 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3256 2023-09-09T23:14:12.908 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3256_vm 2023-09-09T23:14:12.908 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3257 2023-09-09T23:14:12.908 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3258 2023-09-09T23:14:12.908 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3350 2023-09-09T23:14:12.909 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3350_vm 2023-09-09T23:14:12.909 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3351 2023-09-09T23:14:12.909 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3379 2023-09-09T23:14:12.909 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3380 2023-09-09T23:14:12.909 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3381 2023-09-09T23:14:12.910 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3381_vm 2023-09-09T23:14:12.910 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3394 2023-09-09T23:14:12.910 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 3394_fsm 2023-09-09T23:14:12.910 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3394_vm 2023-09-09T23:14:12.910 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3395 2023-09-09T23:14:12.911 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3439 2023-09-09T23:14:12.911 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3439_vm 2023-09-09T23:14:12.911 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3440 2023-09-09T23:14:12.911 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 9 23:14 3455 2023-09-09T23:14:12.911 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3456 2023-09-09T23:14:12.912 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 3456_fsm 2023-09-09T23:14:12.912 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3456_vm 2023-09-09T23:14:12.912 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3466 2023-09-09T23:14:12.912 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3466_vm 2023-09-09T23:14:12.912 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3467 2023-09-09T23:14:12.913 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3468 2023-09-09T23:14:12.913 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3501 2023-09-09T23:14:12.913 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3501_vm 2023-09-09T23:14:12.913 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3502 2023-09-09T23:14:12.913 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3503 2023-09-09T23:14:12.913 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3534 2023-09-09T23:14:12.914 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3541 2023-09-09T23:14:12.914 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 3541_fsm 2023-09-09T23:14:12.914 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3541_vm 2023-09-09T23:14:12.914 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3542 2023-09-09T23:14:12.914 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3574 2023-09-09T23:14:12.915 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3575 2023-09-09T23:14:12.915 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3576 2023-09-09T23:14:12.915 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3576_vm 2023-09-09T23:14:12.915 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3596 2023-09-09T23:14:12.915 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3596_vm 2023-09-09T23:14:12.915 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3597 2023-09-09T23:14:12.916 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3598 2023-09-09T23:14:12.916 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3598_vm 2023-09-09T23:14:12.916 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3599 2023-09-09T23:14:12.916 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3600 2023-09-09T23:14:12.916 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 3600_fsm 2023-09-09T23:14:12.916 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3600_vm 2023-09-09T23:14:12.917 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3601 2023-09-09T23:14:12.917 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 3601_fsm 2023-09-09T23:14:12.917 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3601_vm 2023-09-09T23:14:12.917 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3602 2023-09-09T23:14:12.917 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 3602_fsm 2023-09-09T23:14:12.917 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3602_vm 2023-09-09T23:14:12.918 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3603 2023-09-09T23:14:12.918 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 3603_fsm 2023-09-09T23:14:12.918 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3603_vm 2023-09-09T23:14:12.918 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3604 2023-09-09T23:14:12.918 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3605 2023-09-09T23:14:12.919 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3606 2023-09-09T23:14:12.922 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3607 2023-09-09T23:14:12.922 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3608 2023-09-09T23:14:12.922 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 9 23:14 3609 2023-09-09T23:14:12.922 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3712 2023-09-09T23:14:12.923 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3764 2023-09-09T23:14:12.923 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 3764_fsm 2023-09-09T23:14:12.923 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3764_vm 2023-09-09T23:14:12.923 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3766 2023-09-09T23:14:12.923 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3767 2023-09-09T23:14:12.923 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3997 2023-09-09T23:14:12.924 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 5002 2023-09-09T23:14:12.924 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 548 2023-09-09T23:14:12.924 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 549 2023-09-09T23:14:12.924 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 6102 2023-09-09T23:14:12.924 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 6102_vm 2023-09-09T23:14:12.925 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 6104 2023-09-09T23:14:12.925 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 6104_vm 2023-09-09T23:14:12.925 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 6106 2023-09-09T23:14:12.925 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 6106_vm 2023-09-09T23:14:12.925 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 6110 2023-09-09T23:14:12.925 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 6111 2023-09-09T23:14:12.926 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 6112 2023-09-09T23:14:12.926 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 6113 2023-09-09T23:14:12.926 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 6117 2023-09-09T23:14:12.926 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 826 2023-09-09T23:14:12.926 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 826_vm 2023-09-09T23:14:12.927 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 827 2023-09-09T23:14:12.927 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 828 2023-09-09T23:14:12.927 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 9 23:14 pg_filenode.map 2023-09-09T23:14:12.927 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 9 23:14 PG_VERSION 2023-09-09T23:14:12.927 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:14:12.927 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-09-09T23:14:12.928 INFO:teuthology.orchestra.run.smithi082.stdout:total 7185 2023-09-09T23:14:12.928 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 9 23:14 . 2023-09-09T23:14:12.928 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 9 23:14 .. 2023-09-09T23:14:12.928 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 112 2023-09-09T23:14:12.928 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 113 2023-09-09T23:14:12.929 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 9 23:14 12182 2023-09-09T23:14:12.929 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 12182_fsm 2023-09-09T23:14:12.929 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12182_vm 2023-09-09T23:14:12.929 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 12184 2023-09-09T23:14:12.929 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12186 2023-09-09T23:14:12.929 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12187 2023-09-09T23:14:12.930 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 12187_fsm 2023-09-09T23:14:12.930 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12187_vm 2023-09-09T23:14:12.930 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 12189 2023-09-09T23:14:12.930 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12191 2023-09-09T23:14:12.930 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12192 2023-09-09T23:14:12.931 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 12192_fsm 2023-09-09T23:14:12.931 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12192_vm 2023-09-09T23:14:12.931 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 12194 2023-09-09T23:14:12.931 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12196 2023-09-09T23:14:12.931 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12197 2023-09-09T23:14:12.931 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 12197_fsm 2023-09-09T23:14:12.932 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12197_vm 2023-09-09T23:14:12.932 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 12199 2023-09-09T23:14:12.932 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12201 2023-09-09T23:14:12.932 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12202 2023-09-09T23:14:12.932 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 12202_fsm 2023-09-09T23:14:12.933 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12202_vm 2023-09-09T23:14:12.933 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 12204 2023-09-09T23:14:12.933 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12206 2023-09-09T23:14:12.933 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12207 2023-09-09T23:14:12.933 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 12207_fsm 2023-09-09T23:14:12.934 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12207_vm 2023-09-09T23:14:12.934 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 12209 2023-09-09T23:14:12.934 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12211 2023-09-09T23:14:12.934 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 12212 2023-09-09T23:14:12.935 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 12214 2023-09-09T23:14:12.935 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12216 2023-09-09T23:14:12.935 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Sep 9 23:14 1247 2023-09-09T23:14:12.935 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 1247_fsm 2023-09-09T23:14:12.935 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 1247_vm 2023-09-09T23:14:12.935 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Sep 9 23:14 1249 2023-09-09T23:14:12.936 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 1249_fsm 2023-09-09T23:14:12.936 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 1249_vm 2023-09-09T23:14:12.936 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Sep 9 23:14 1255 2023-09-09T23:14:12.936 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 1255_fsm 2023-09-09T23:14:12.936 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 1255_vm 2023-09-09T23:14:12.937 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 9 23:14 1259 2023-09-09T23:14:12.937 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 1259_fsm 2023-09-09T23:14:12.937 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 1259_vm 2023-09-09T23:14:12.937 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 1417 2023-09-09T23:14:12.937 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 1417_vm 2023-09-09T23:14:12.938 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 1418 2023-09-09T23:14:12.938 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 1418_vm 2023-09-09T23:14:12.938 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 174 2023-09-09T23:14:12.938 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 175 2023-09-09T23:14:12.938 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2187 2023-09-09T23:14:12.939 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2224 2023-09-09T23:14:12.939 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2224_vm 2023-09-09T23:14:12.939 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2328 2023-09-09T23:14:12.939 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2328_vm 2023-09-09T23:14:12.939 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2336 2023-09-09T23:14:12.939 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2336_vm 2023-09-09T23:14:12.940 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2337 2023-09-09T23:14:12.940 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2600 2023-09-09T23:14:12.940 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2600_fsm 2023-09-09T23:14:12.940 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2600_vm 2023-09-09T23:14:12.940 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2601 2023-09-09T23:14:12.941 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2601_fsm 2023-09-09T23:14:12.941 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2601_vm 2023-09-09T23:14:12.941 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 9 23:14 2602 2023-09-09T23:14:12.941 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2602_fsm 2023-09-09T23:14:12.941 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2602_vm 2023-09-09T23:14:12.941 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 9 23:14 2603 2023-09-09T23:14:12.942 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2603_fsm 2023-09-09T23:14:12.942 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2603_vm 2023-09-09T23:14:12.942 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2604 2023-09-09T23:14:12.942 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2604_vm 2023-09-09T23:14:12.942 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2605 2023-09-09T23:14:12.943 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2605_fsm 2023-09-09T23:14:12.943 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2605_vm 2023-09-09T23:14:12.943 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2606 2023-09-09T23:14:12.943 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2606_fsm 2023-09-09T23:14:12.943 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2606_vm 2023-09-09T23:14:12.944 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2607 2023-09-09T23:14:12.944 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2607_fsm 2023-09-09T23:14:12.944 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2607_vm 2023-09-09T23:14:12.944 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Sep 9 23:14 2608 2023-09-09T23:14:12.944 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2608_fsm 2023-09-09T23:14:12.945 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2608_vm 2023-09-09T23:14:12.945 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Sep 9 23:14 2609 2023-09-09T23:14:12.945 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2609_fsm 2023-09-09T23:14:12.945 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2609_vm 2023-09-09T23:14:12.945 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2610 2023-09-09T23:14:12.946 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2610_fsm 2023-09-09T23:14:12.946 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2610_vm 2023-09-09T23:14:12.946 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2611 2023-09-09T23:14:12.946 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2611_vm 2023-09-09T23:14:12.946 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2612 2023-09-09T23:14:12.947 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2612_fsm 2023-09-09T23:14:12.947 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2612_vm 2023-09-09T23:14:12.947 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2613 2023-09-09T23:14:12.947 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2613_vm 2023-09-09T23:14:12.947 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2615 2023-09-09T23:14:12.948 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2615_fsm 2023-09-09T23:14:12.948 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2615_vm 2023-09-09T23:14:12.948 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2616 2023-09-09T23:14:12.948 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2616_fsm 2023-09-09T23:14:12.948 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2616_vm 2023-09-09T23:14:12.948 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 9 23:14 2617 2023-09-09T23:14:12.949 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2617_fsm 2023-09-09T23:14:12.949 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2617_vm 2023-09-09T23:14:12.949 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 9 23:14 2618 2023-09-09T23:14:12.949 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2618_fsm 2023-09-09T23:14:12.949 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2618_vm 2023-09-09T23:14:12.950 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 9 23:14 2619 2023-09-09T23:14:12.950 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2619_fsm 2023-09-09T23:14:12.950 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2619_vm 2023-09-09T23:14:12.950 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2620 2023-09-09T23:14:12.950 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2620_vm 2023-09-09T23:14:12.951 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2650 2023-09-09T23:14:12.951 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2651 2023-09-09T23:14:12.951 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2652 2023-09-09T23:14:12.951 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 9 23:14 2653 2023-09-09T23:14:12.951 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 9 23:14 2654 2023-09-09T23:14:12.951 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 9 23:14 2655 2023-09-09T23:14:12.952 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2656 2023-09-09T23:14:12.952 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2657 2023-09-09T23:14:12.952 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 9 23:14 2658 2023-09-09T23:14:12.952 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 9 23:14 2659 2023-09-09T23:14:12.952 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2660 2023-09-09T23:14:12.953 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2661 2023-09-09T23:14:12.953 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 9 23:14 2662 2023-09-09T23:14:12.953 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 9 23:14 2663 2023-09-09T23:14:12.953 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2664 2023-09-09T23:14:12.953 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2665 2023-09-09T23:14:12.954 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2666 2023-09-09T23:14:12.954 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2667 2023-09-09T23:14:12.954 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2668 2023-09-09T23:14:12.954 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2669 2023-09-09T23:14:12.954 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2670 2023-09-09T23:14:12.954 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 311296 Sep 9 23:14 2673 2023-09-09T23:14:12.955 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Sep 9 23:14 2674 2023-09-09T23:14:12.955 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 9 23:14 2675 2023-09-09T23:14:12.955 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2678 2023-09-09T23:14:12.955 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2679 2023-09-09T23:14:12.955 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2680 2023-09-09T23:14:12.956 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2681 2023-09-09T23:14:12.956 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2682 2023-09-09T23:14:12.956 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2683 2023-09-09T23:14:12.956 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2684 2023-09-09T23:14:12.956 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2685 2023-09-09T23:14:12.957 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2686 2023-09-09T23:14:12.957 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2687 2023-09-09T23:14:12.957 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 9 23:14 2688 2023-09-09T23:14:12.957 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 9 23:14 2689 2023-09-09T23:14:12.957 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 9 23:14 2690 2023-09-09T23:14:12.957 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Sep 9 23:14 2691 2023-09-09T23:14:12.958 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2692 2023-09-09T23:14:12.958 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2693 2023-09-09T23:14:12.958 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2696 2023-09-09T23:14:12.958 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2699 2023-09-09T23:14:12.958 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2701 2023-09-09T23:14:12.959 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2702 2023-09-09T23:14:12.959 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2703 2023-09-09T23:14:12.959 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 9 23:14 2704 2023-09-09T23:14:12.959 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2753 2023-09-09T23:14:12.959 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2753_fsm 2023-09-09T23:14:12.959 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2753_vm 2023-09-09T23:14:12.960 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2754 2023-09-09T23:14:12.960 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2755 2023-09-09T23:14:12.960 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 9 23:14 2756 2023-09-09T23:14:12.960 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 9 23:14 2757 2023-09-09T23:14:12.960 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2830 2023-09-09T23:14:12.961 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2830_vm 2023-09-09T23:14:12.961 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2831 2023-09-09T23:14:12.961 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2832 2023-09-09T23:14:12.961 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2832_vm 2023-09-09T23:14:12.961 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2833 2023-09-09T23:14:12.961 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2834 2023-09-09T23:14:12.962 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2834_vm 2023-09-09T23:14:12.962 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2835 2023-09-09T23:14:12.962 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2836 2023-09-09T23:14:12.962 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2836_vm 2023-09-09T23:14:12.962 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2837 2023-09-09T23:14:12.963 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Sep 9 23:14 2838 2023-09-09T23:14:12.963 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2838_fsm 2023-09-09T23:14:12.963 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2838_vm 2023-09-09T23:14:12.963 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2839 2023-09-09T23:14:12.963 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2840 2023-09-09T23:14:12.963 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2840_fsm 2023-09-09T23:14:12.964 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2840_vm 2023-09-09T23:14:12.964 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2841 2023-09-09T23:14:12.964 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2995 2023-09-09T23:14:12.964 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2995_vm 2023-09-09T23:14:12.964 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2996 2023-09-09T23:14:12.964 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3079 2023-09-09T23:14:12.965 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 3079_fsm 2023-09-09T23:14:12.965 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3079_vm 2023-09-09T23:14:12.965 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3080 2023-09-09T23:14:12.965 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3081 2023-09-09T23:14:12.965 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3085 2023-09-09T23:14:12.966 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3118 2023-09-09T23:14:12.966 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3118_vm 2023-09-09T23:14:12.966 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3119 2023-09-09T23:14:12.966 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3164 2023-09-09T23:14:12.966 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3256 2023-09-09T23:14:12.967 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3256_vm 2023-09-09T23:14:12.967 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3257 2023-09-09T23:14:12.967 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3258 2023-09-09T23:14:12.967 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3350 2023-09-09T23:14:12.967 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3350_vm 2023-09-09T23:14:12.967 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3351 2023-09-09T23:14:12.968 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3379 2023-09-09T23:14:12.968 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3380 2023-09-09T23:14:12.968 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3381 2023-09-09T23:14:12.968 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3381_vm 2023-09-09T23:14:12.968 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3394 2023-09-09T23:14:12.968 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 3394_fsm 2023-09-09T23:14:12.969 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3394_vm 2023-09-09T23:14:12.969 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3395 2023-09-09T23:14:12.969 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3439 2023-09-09T23:14:12.969 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3439_vm 2023-09-09T23:14:12.969 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3440 2023-09-09T23:14:12.969 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 9 23:14 3455 2023-09-09T23:14:12.970 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3456 2023-09-09T23:14:12.970 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 3456_fsm 2023-09-09T23:14:12.970 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3456_vm 2023-09-09T23:14:12.970 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3466 2023-09-09T23:14:12.970 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3466_vm 2023-09-09T23:14:12.971 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3467 2023-09-09T23:14:12.971 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3468 2023-09-09T23:14:12.971 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3501 2023-09-09T23:14:12.971 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3501_vm 2023-09-09T23:14:12.971 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3502 2023-09-09T23:14:12.971 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3503 2023-09-09T23:14:12.972 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3534 2023-09-09T23:14:12.972 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3541 2023-09-09T23:14:12.972 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 3541_fsm 2023-09-09T23:14:12.972 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3541_vm 2023-09-09T23:14:12.972 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3542 2023-09-09T23:14:12.973 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3574 2023-09-09T23:14:12.973 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3575 2023-09-09T23:14:12.973 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3576 2023-09-09T23:14:12.973 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3576_vm 2023-09-09T23:14:12.973 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3596 2023-09-09T23:14:12.973 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3596_vm 2023-09-09T23:14:12.974 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3597 2023-09-09T23:14:12.974 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3598 2023-09-09T23:14:12.974 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3598_vm 2023-09-09T23:14:12.974 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3599 2023-09-09T23:14:12.974 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3600 2023-09-09T23:14:12.974 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 3600_fsm 2023-09-09T23:14:12.975 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3600_vm 2023-09-09T23:14:12.975 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3601 2023-09-09T23:14:12.975 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 3601_fsm 2023-09-09T23:14:12.975 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3601_vm 2023-09-09T23:14:12.975 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3602 2023-09-09T23:14:12.975 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 3602_fsm 2023-09-09T23:14:12.976 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3602_vm 2023-09-09T23:14:12.976 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3603 2023-09-09T23:14:12.976 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 3603_fsm 2023-09-09T23:14:12.976 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3603_vm 2023-09-09T23:14:12.976 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3604 2023-09-09T23:14:12.976 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3605 2023-09-09T23:14:12.977 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3606 2023-09-09T23:14:12.977 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3607 2023-09-09T23:14:12.977 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3608 2023-09-09T23:14:12.977 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 9 23:14 3609 2023-09-09T23:14:12.977 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3712 2023-09-09T23:14:12.978 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3764 2023-09-09T23:14:12.978 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 3764_fsm 2023-09-09T23:14:12.978 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3764_vm 2023-09-09T23:14:12.978 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3766 2023-09-09T23:14:12.978 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3767 2023-09-09T23:14:12.978 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3997 2023-09-09T23:14:12.979 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 5002 2023-09-09T23:14:12.979 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 548 2023-09-09T23:14:12.979 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 549 2023-09-09T23:14:12.979 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 6102 2023-09-09T23:14:12.979 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 6102_vm 2023-09-09T23:14:12.979 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 6104 2023-09-09T23:14:12.980 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 6104_vm 2023-09-09T23:14:12.980 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 6106 2023-09-09T23:14:12.980 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 6106_vm 2023-09-09T23:14:12.980 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 6110 2023-09-09T23:14:12.980 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 6111 2023-09-09T23:14:12.980 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 6112 2023-09-09T23:14:12.981 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 6113 2023-09-09T23:14:12.981 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 6117 2023-09-09T23:14:12.981 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 826 2023-09-09T23:14:12.981 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 826_vm 2023-09-09T23:14:12.981 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 827 2023-09-09T23:14:12.981 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 828 2023-09-09T23:14:12.982 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 9 23:14 pg_filenode.map 2023-09-09T23:14:12.982 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 9 23:14 PG_VERSION 2023-09-09T23:14:12.982 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:14:12.982 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-09-09T23:14:12.982 INFO:teuthology.orchestra.run.smithi082.stdout:total 7185 2023-09-09T23:14:12.983 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 9 23:14 . 2023-09-09T23:14:12.983 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 9 23:14 .. 2023-09-09T23:14:12.983 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 112 2023-09-09T23:14:12.983 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 113 2023-09-09T23:14:12.983 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 9 23:14 12182 2023-09-09T23:14:12.983 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 12182_fsm 2023-09-09T23:14:12.984 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12182_vm 2023-09-09T23:14:12.984 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 12184 2023-09-09T23:14:12.984 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12186 2023-09-09T23:14:12.984 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12187 2023-09-09T23:14:12.984 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 12187_fsm 2023-09-09T23:14:12.985 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12187_vm 2023-09-09T23:14:12.985 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 12189 2023-09-09T23:14:12.985 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12191 2023-09-09T23:14:12.985 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12192 2023-09-09T23:14:12.985 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 12192_fsm 2023-09-09T23:14:12.985 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12192_vm 2023-09-09T23:14:12.986 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 12194 2023-09-09T23:14:12.986 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12196 2023-09-09T23:14:12.986 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12197 2023-09-09T23:14:12.986 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 12197_fsm 2023-09-09T23:14:12.986 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12197_vm 2023-09-09T23:14:12.986 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 12199 2023-09-09T23:14:12.987 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12201 2023-09-09T23:14:12.987 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12202 2023-09-09T23:14:12.987 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 12202_fsm 2023-09-09T23:14:12.987 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12202_vm 2023-09-09T23:14:12.987 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 12204 2023-09-09T23:14:12.987 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12206 2023-09-09T23:14:12.988 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12207 2023-09-09T23:14:12.988 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 12207_fsm 2023-09-09T23:14:12.988 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12207_vm 2023-09-09T23:14:12.988 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 12209 2023-09-09T23:14:12.988 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12211 2023-09-09T23:14:12.988 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 12212 2023-09-09T23:14:12.989 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 12214 2023-09-09T23:14:12.989 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 12216 2023-09-09T23:14:12.989 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Sep 9 23:14 1247 2023-09-09T23:14:12.989 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 1247_fsm 2023-09-09T23:14:12.989 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 1247_vm 2023-09-09T23:14:12.989 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Sep 9 23:14 1249 2023-09-09T23:14:12.990 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 1249_fsm 2023-09-09T23:14:12.990 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 1249_vm 2023-09-09T23:14:12.990 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Sep 9 23:14 1255 2023-09-09T23:14:12.990 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 1255_fsm 2023-09-09T23:14:12.990 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 1255_vm 2023-09-09T23:14:12.991 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 9 23:14 1259 2023-09-09T23:14:12.991 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 1259_fsm 2023-09-09T23:14:12.991 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 1259_vm 2023-09-09T23:14:12.991 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 1417 2023-09-09T23:14:12.991 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 1417_vm 2023-09-09T23:14:12.991 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 1418 2023-09-09T23:14:12.992 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 1418_vm 2023-09-09T23:14:12.992 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 174 2023-09-09T23:14:12.992 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 175 2023-09-09T23:14:12.992 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2187 2023-09-09T23:14:12.992 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2224 2023-09-09T23:14:12.993 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2224_vm 2023-09-09T23:14:12.993 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2328 2023-09-09T23:14:12.993 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2328_vm 2023-09-09T23:14:12.993 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2336 2023-09-09T23:14:12.993 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2336_vm 2023-09-09T23:14:12.993 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2337 2023-09-09T23:14:12.994 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2600 2023-09-09T23:14:12.994 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2600_fsm 2023-09-09T23:14:12.994 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2600_vm 2023-09-09T23:14:12.994 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2601 2023-09-09T23:14:12.994 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2601_fsm 2023-09-09T23:14:12.995 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2601_vm 2023-09-09T23:14:12.995 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 9 23:14 2602 2023-09-09T23:14:12.995 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2602_fsm 2023-09-09T23:14:12.995 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2602_vm 2023-09-09T23:14:12.995 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 9 23:14 2603 2023-09-09T23:14:12.995 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2603_fsm 2023-09-09T23:14:12.996 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2603_vm 2023-09-09T23:14:12.996 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2604 2023-09-09T23:14:12.996 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2604_vm 2023-09-09T23:14:12.996 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2605 2023-09-09T23:14:12.996 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2605_fsm 2023-09-09T23:14:12.996 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2605_vm 2023-09-09T23:14:12.997 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2606 2023-09-09T23:14:12.997 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2606_fsm 2023-09-09T23:14:12.997 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2606_vm 2023-09-09T23:14:12.997 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2607 2023-09-09T23:14:12.997 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2607_fsm 2023-09-09T23:14:12.997 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2607_vm 2023-09-09T23:14:12.998 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Sep 9 23:14 2608 2023-09-09T23:14:12.998 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2608_fsm 2023-09-09T23:14:12.998 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2608_vm 2023-09-09T23:14:12.998 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Sep 9 23:14 2609 2023-09-09T23:14:12.998 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2609_fsm 2023-09-09T23:14:12.998 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2609_vm 2023-09-09T23:14:12.999 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2610 2023-09-09T23:14:12.999 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2610_fsm 2023-09-09T23:14:12.999 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2610_vm 2023-09-09T23:14:12.999 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2611 2023-09-09T23:14:12.999 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2611_vm 2023-09-09T23:14:13.000 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2612 2023-09-09T23:14:13.000 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2612_fsm 2023-09-09T23:14:13.000 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2612_vm 2023-09-09T23:14:13.000 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2613 2023-09-09T23:14:13.000 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2613_vm 2023-09-09T23:14:13.000 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2615 2023-09-09T23:14:13.001 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2615_fsm 2023-09-09T23:14:13.001 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2615_vm 2023-09-09T23:14:13.001 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2616 2023-09-09T23:14:13.001 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2616_fsm 2023-09-09T23:14:13.001 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2616_vm 2023-09-09T23:14:13.001 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 9 23:14 2617 2023-09-09T23:14:13.002 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2617_fsm 2023-09-09T23:14:13.002 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2617_vm 2023-09-09T23:14:13.002 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 9 23:14 2618 2023-09-09T23:14:13.002 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2618_fsm 2023-09-09T23:14:13.002 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2618_vm 2023-09-09T23:14:13.003 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 9 23:14 2619 2023-09-09T23:14:13.003 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2619_fsm 2023-09-09T23:14:13.003 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2619_vm 2023-09-09T23:14:13.003 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2620 2023-09-09T23:14:13.003 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2620_vm 2023-09-09T23:14:13.003 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2650 2023-09-09T23:14:13.004 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2651 2023-09-09T23:14:13.004 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2652 2023-09-09T23:14:13.004 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 9 23:14 2653 2023-09-09T23:14:13.004 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 9 23:14 2654 2023-09-09T23:14:13.004 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 9 23:14 2655 2023-09-09T23:14:13.004 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2656 2023-09-09T23:14:13.005 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2657 2023-09-09T23:14:13.005 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 9 23:14 2658 2023-09-09T23:14:13.005 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 9 23:14 2659 2023-09-09T23:14:13.005 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2660 2023-09-09T23:14:13.005 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2661 2023-09-09T23:14:13.005 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 9 23:14 2662 2023-09-09T23:14:13.006 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 9 23:14 2663 2023-09-09T23:14:13.006 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2664 2023-09-09T23:14:13.006 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2665 2023-09-09T23:14:13.006 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2666 2023-09-09T23:14:13.006 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2667 2023-09-09T23:14:13.007 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2668 2023-09-09T23:14:13.007 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2669 2023-09-09T23:14:13.007 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2670 2023-09-09T23:14:13.007 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 311296 Sep 9 23:14 2673 2023-09-09T23:14:13.007 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Sep 9 23:14 2674 2023-09-09T23:14:13.007 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 9 23:14 2675 2023-09-09T23:14:13.008 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2678 2023-09-09T23:14:13.008 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2679 2023-09-09T23:14:13.008 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2680 2023-09-09T23:14:13.008 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2681 2023-09-09T23:14:13.008 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2682 2023-09-09T23:14:13.008 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2683 2023-09-09T23:14:13.009 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2684 2023-09-09T23:14:13.009 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2685 2023-09-09T23:14:13.009 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2686 2023-09-09T23:14:13.009 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2687 2023-09-09T23:14:13.009 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 9 23:14 2688 2023-09-09T23:14:13.010 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 9 23:14 2689 2023-09-09T23:14:13.010 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 9 23:14 2690 2023-09-09T23:14:13.010 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Sep 9 23:14 2691 2023-09-09T23:14:13.010 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2692 2023-09-09T23:14:13.010 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2693 2023-09-09T23:14:13.011 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2696 2023-09-09T23:14:13.011 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2699 2023-09-09T23:14:13.011 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2701 2023-09-09T23:14:13.011 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2702 2023-09-09T23:14:13.011 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2703 2023-09-09T23:14:13.011 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 9 23:14 2704 2023-09-09T23:14:13.012 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2753 2023-09-09T23:14:13.012 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2753_fsm 2023-09-09T23:14:13.012 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2753_vm 2023-09-09T23:14:13.012 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2754 2023-09-09T23:14:13.012 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2755 2023-09-09T23:14:13.013 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 9 23:14 2756 2023-09-09T23:14:13.013 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 9 23:14 2757 2023-09-09T23:14:13.013 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2830 2023-09-09T23:14:13.013 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2830_vm 2023-09-09T23:14:13.013 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2831 2023-09-09T23:14:13.014 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2832 2023-09-09T23:14:13.014 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2832_vm 2023-09-09T23:14:13.014 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2833 2023-09-09T23:14:13.014 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2834 2023-09-09T23:14:13.014 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2834_vm 2023-09-09T23:14:13.014 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2835 2023-09-09T23:14:13.015 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2836 2023-09-09T23:14:13.015 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2836_vm 2023-09-09T23:14:13.015 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2837 2023-09-09T23:14:13.015 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Sep 9 23:14 2838 2023-09-09T23:14:13.015 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2838_fsm 2023-09-09T23:14:13.015 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2838_vm 2023-09-09T23:14:13.016 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2839 2023-09-09T23:14:13.016 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2840 2023-09-09T23:14:13.016 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2840_fsm 2023-09-09T23:14:13.016 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2840_vm 2023-09-09T23:14:13.016 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2841 2023-09-09T23:14:13.017 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2995 2023-09-09T23:14:13.017 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2995_vm 2023-09-09T23:14:13.017 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2996 2023-09-09T23:14:13.017 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3079 2023-09-09T23:14:13.017 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 3079_fsm 2023-09-09T23:14:13.017 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3079_vm 2023-09-09T23:14:13.018 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3080 2023-09-09T23:14:13.018 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3081 2023-09-09T23:14:13.018 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3085 2023-09-09T23:14:13.018 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3118 2023-09-09T23:14:13.018 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3118_vm 2023-09-09T23:14:13.019 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3119 2023-09-09T23:14:13.019 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3164 2023-09-09T23:14:13.019 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3256 2023-09-09T23:14:13.019 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3256_vm 2023-09-09T23:14:13.019 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3257 2023-09-09T23:14:13.020 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3258 2023-09-09T23:14:13.020 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3350 2023-09-09T23:14:13.020 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3350_vm 2023-09-09T23:14:13.020 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3351 2023-09-09T23:14:13.020 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3379 2023-09-09T23:14:13.021 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3380 2023-09-09T23:14:13.021 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3381 2023-09-09T23:14:13.021 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3381_vm 2023-09-09T23:14:13.021 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3394 2023-09-09T23:14:13.021 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 3394_fsm 2023-09-09T23:14:13.022 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3394_vm 2023-09-09T23:14:13.022 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3395 2023-09-09T23:14:13.022 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3439 2023-09-09T23:14:13.022 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3439_vm 2023-09-09T23:14:13.022 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3440 2023-09-09T23:14:13.023 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 9 23:14 3455 2023-09-09T23:14:13.023 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3456 2023-09-09T23:14:13.023 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 3456_fsm 2023-09-09T23:14:13.023 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3456_vm 2023-09-09T23:14:13.023 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3466 2023-09-09T23:14:13.023 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3466_vm 2023-09-09T23:14:13.024 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3467 2023-09-09T23:14:13.024 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3468 2023-09-09T23:14:13.024 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3501 2023-09-09T23:14:13.024 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3501_vm 2023-09-09T23:14:13.024 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3502 2023-09-09T23:14:13.025 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3503 2023-09-09T23:14:13.025 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3534 2023-09-09T23:14:13.025 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3541 2023-09-09T23:14:13.025 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 3541_fsm 2023-09-09T23:14:13.025 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3541_vm 2023-09-09T23:14:13.025 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3542 2023-09-09T23:14:13.026 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3574 2023-09-09T23:14:13.026 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3575 2023-09-09T23:14:13.026 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3576 2023-09-09T23:14:13.026 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3576_vm 2023-09-09T23:14:13.026 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3596 2023-09-09T23:14:13.027 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3596_vm 2023-09-09T23:14:13.027 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3597 2023-09-09T23:14:13.027 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3598 2023-09-09T23:14:13.027 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3598_vm 2023-09-09T23:14:13.027 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3599 2023-09-09T23:14:13.027 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3600 2023-09-09T23:14:13.028 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 3600_fsm 2023-09-09T23:14:13.028 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3600_vm 2023-09-09T23:14:13.028 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3601 2023-09-09T23:14:13.028 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 3601_fsm 2023-09-09T23:14:13.028 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3601_vm 2023-09-09T23:14:13.029 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3602 2023-09-09T23:14:13.029 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 3602_fsm 2023-09-09T23:14:13.029 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3602_vm 2023-09-09T23:14:13.029 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3603 2023-09-09T23:14:13.029 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 3603_fsm 2023-09-09T23:14:13.029 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3603_vm 2023-09-09T23:14:13.030 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3604 2023-09-09T23:14:13.030 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3605 2023-09-09T23:14:13.030 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3606 2023-09-09T23:14:13.030 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3607 2023-09-09T23:14:13.030 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3608 2023-09-09T23:14:13.030 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 9 23:14 3609 2023-09-09T23:14:13.031 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3712 2023-09-09T23:14:13.031 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3764 2023-09-09T23:14:13.031 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 3764_fsm 2023-09-09T23:14:13.031 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3764_vm 2023-09-09T23:14:13.031 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3766 2023-09-09T23:14:13.031 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 3767 2023-09-09T23:14:13.032 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3997 2023-09-09T23:14:13.032 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 5002 2023-09-09T23:14:13.032 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 548 2023-09-09T23:14:13.032 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 549 2023-09-09T23:14:13.032 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 6102 2023-09-09T23:14:13.033 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 6102_vm 2023-09-09T23:14:13.033 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 6104 2023-09-09T23:14:13.033 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 6104_vm 2023-09-09T23:14:13.033 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 6106 2023-09-09T23:14:13.033 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 6106_vm 2023-09-09T23:14:13.033 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 6110 2023-09-09T23:14:13.034 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 6111 2023-09-09T23:14:13.034 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 6112 2023-09-09T23:14:13.034 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 6113 2023-09-09T23:14:13.034 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 6117 2023-09-09T23:14:13.034 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 826 2023-09-09T23:14:13.035 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 826_vm 2023-09-09T23:14:13.035 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 827 2023-09-09T23:14:13.035 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 828 2023-09-09T23:14:13.035 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 9 23:14 pg_filenode.map 2023-09-09T23:14:13.035 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 9 23:14 PG_VERSION 2023-09-09T23:14:13.035 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:14:13.036 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/data/global: 2023-09-09T23:14:13.036 INFO:teuthology.orchestra.run.smithi082.stdout:total 553 2023-09-09T23:14:13.036 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Sep 9 23:14 . 2023-09-09T23:14:13.036 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 9 23:14 .. 2023-09-09T23:14:13.036 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 1136 2023-09-09T23:14:13.037 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 1136_fsm 2023-09-09T23:14:13.037 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 1136_vm 2023-09-09T23:14:13.037 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 1137 2023-09-09T23:14:13.037 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 1213 2023-09-09T23:14:13.037 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 1213_fsm 2023-09-09T23:14:13.037 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 1213_vm 2023-09-09T23:14:13.038 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 1214 2023-09-09T23:14:13.038 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 1214_fsm 2023-09-09T23:14:13.038 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 1214_vm 2023-09-09T23:14:13.038 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 1232 2023-09-09T23:14:13.038 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 1233 2023-09-09T23:14:13.039 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 1260 2023-09-09T23:14:13.039 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 1260_fsm 2023-09-09T23:14:13.039 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 1260_vm 2023-09-09T23:14:13.039 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 1261 2023-09-09T23:14:13.039 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 1261_fsm 2023-09-09T23:14:13.040 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 1261_vm 2023-09-09T23:14:13.040 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 1262 2023-09-09T23:14:13.040 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 1262_fsm 2023-09-09T23:14:13.040 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 1262_vm 2023-09-09T23:14:13.040 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2396 2023-09-09T23:14:13.040 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 9 23:14 2396_fsm 2023-09-09T23:14:13.041 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2396_vm 2023-09-09T23:14:13.041 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2397 2023-09-09T23:14:13.041 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2671 2023-09-09T23:14:13.041 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2672 2023-09-09T23:14:13.041 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2676 2023-09-09T23:14:13.041 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2677 2023-09-09T23:14:13.042 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2694 2023-09-09T23:14:13.042 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2695 2023-09-09T23:14:13.042 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2697 2023-09-09T23:14:13.042 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 9 23:14 2698 2023-09-09T23:14:13.042 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2846 2023-09-09T23:14:13.043 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2846_vm 2023-09-09T23:14:13.043 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2847 2023-09-09T23:14:13.043 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2964 2023-09-09T23:14:13.043 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2964_vm 2023-09-09T23:14:13.043 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2965 2023-09-09T23:14:13.043 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2966 2023-09-09T23:14:13.044 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 2966_vm 2023-09-09T23:14:13.044 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 2967 2023-09-09T23:14:13.044 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3592 2023-09-09T23:14:13.044 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 3592_vm 2023-09-09T23:14:13.044 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 3593 2023-09-09T23:14:13.045 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 4060 2023-09-09T23:14:13.045 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 4060_vm 2023-09-09T23:14:13.045 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 4061 2023-09-09T23:14:13.045 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 6000 2023-09-09T23:14:13.045 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 6000_vm 2023-09-09T23:14:13.045 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 6001 2023-09-09T23:14:13.046 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 6002 2023-09-09T23:14:13.046 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 6100 2023-09-09T23:14:13.046 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 6100_vm 2023-09-09T23:14:13.046 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 6114 2023-09-09T23:14:13.046 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 6115 2023-09-09T23:14:13.047 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 pg_control 2023-09-09T23:14:13.047 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 9 23:14 pg_filenode.map 2023-09-09T23:14:13.047 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:14:13.047 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/data/log: 2023-09-09T23:14:13.047 INFO:teuthology.orchestra.run.smithi082.stdout:total 0 2023-09-09T23:14:13.047 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:14 . 2023-09-09T23:14:13.048 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 9 23:14 .. 2023-09-09T23:14:13.048 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:14:13.048 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-09-09T23:14:13.048 INFO:teuthology.orchestra.run.smithi082.stdout:total 0 2023-09-09T23:14:13.048 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:13 . 2023-09-09T23:14:13.048 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 9 23:14 .. 2023-09-09T23:14:13.049 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:14:13.049 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-09-09T23:14:13.049 INFO:teuthology.orchestra.run.smithi082.stdout:total 0 2023-09-09T23:14:13.049 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:13 . 2023-09-09T23:14:13.049 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 9 23:14 .. 2023-09-09T23:14:13.049 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:14:13.050 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-09-09T23:14:13.050 INFO:teuthology.orchestra.run.smithi082.stdout:total 1 2023-09-09T23:14:13.050 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 9 23:14 . 2023-09-09T23:14:13.050 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 9 23:14 .. 2023-09-09T23:14:13.050 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:13 mappings 2023-09-09T23:14:13.051 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 9 23:14 replorigin_checkpoint 2023-09-09T23:14:13.051 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:13 snapshots 2023-09-09T23:14:13.051 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:14:13.051 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-09-09T23:14:13.051 INFO:teuthology.orchestra.run.smithi082.stdout:total 0 2023-09-09T23:14:13.052 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:13 . 2023-09-09T23:14:13.052 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 9 23:14 .. 2023-09-09T23:14:13.052 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:14:13.052 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-09-09T23:14:13.052 INFO:teuthology.orchestra.run.smithi082.stdout:total 0 2023-09-09T23:14:13.052 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:13 . 2023-09-09T23:14:13.053 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 9 23:14 .. 2023-09-09T23:14:13.053 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:14:13.053 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-09-09T23:14:13.053 INFO:teuthology.orchestra.run.smithi082.stdout:total 0 2023-09-09T23:14:13.053 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 9 23:13 . 2023-09-09T23:14:13.054 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 9 23:14 .. 2023-09-09T23:14:13.054 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 9 23:13 members 2023-09-09T23:14:13.054 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 9 23:13 offsets 2023-09-09T23:14:13.054 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:14:13.054 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-09-09T23:14:13.054 INFO:teuthology.orchestra.run.smithi082.stdout:total 8 2023-09-09T23:14:13.055 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 9 23:13 . 2023-09-09T23:14:13.055 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 9 23:13 .. 2023-09-09T23:14:13.055 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 0000 2023-09-09T23:14:13.055 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:14:13.055 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-09-09T23:14:13.056 INFO:teuthology.orchestra.run.smithi082.stdout:total 8 2023-09-09T23:14:13.056 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 9 23:13 . 2023-09-09T23:14:13.056 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 9 23:13 .. 2023-09-09T23:14:13.056 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 0000 2023-09-09T23:14:13.056 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:14:13.056 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-09-09T23:14:13.057 INFO:teuthology.orchestra.run.smithi082.stdout:total 8 2023-09-09T23:14:13.057 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 9 23:14 . 2023-09-09T23:14:13.057 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 9 23:14 .. 2023-09-09T23:14:13.057 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 0000 2023-09-09T23:14:13.057 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:14:13.057 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-09-09T23:14:13.058 INFO:teuthology.orchestra.run.smithi082.stdout:total 0 2023-09-09T23:14:13.058 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:13 . 2023-09-09T23:14:13.058 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 9 23:14 .. 2023-09-09T23:14:13.058 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:14:13.058 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-09-09T23:14:13.059 INFO:teuthology.orchestra.run.smithi082.stdout:total 0 2023-09-09T23:14:13.059 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:13 . 2023-09-09T23:14:13.059 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 9 23:14 .. 2023-09-09T23:14:13.059 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:14:13.059 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-09-09T23:14:13.059 INFO:teuthology.orchestra.run.smithi082.stdout:total 0 2023-09-09T23:14:13.060 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:13 . 2023-09-09T23:14:13.060 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 9 23:14 .. 2023-09-09T23:14:13.060 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:14:13.060 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-09-09T23:14:13.060 INFO:teuthology.orchestra.run.smithi082.stdout:total 0 2023-09-09T23:14:13.061 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:13 . 2023-09-09T23:14:13.061 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 9 23:14 .. 2023-09-09T23:14:13.061 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:14:13.061 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-09-09T23:14:13.061 INFO:teuthology.orchestra.run.smithi082.stdout:total 0 2023-09-09T23:14:13.062 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:13 . 2023-09-09T23:14:13.062 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 9 23:14 .. 2023-09-09T23:14:13.062 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:14:13.062 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-09-09T23:14:13.062 INFO:teuthology.orchestra.run.smithi082.stdout:total 8 2023-09-09T23:14:13.063 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 9 23:13 . 2023-09-09T23:14:13.063 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 9 23:14 .. 2023-09-09T23:14:13.063 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 0000 2023-09-09T23:14:13.063 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:14:13.063 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-09-09T23:14:13.063 INFO:teuthology.orchestra.run.smithi082.stdout:total 0 2023-09-09T23:14:13.064 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:13 . 2023-09-09T23:14:13.064 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 9 23:14 .. 2023-09-09T23:14:13.064 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:14:13.064 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-09-09T23:14:13.064 INFO:teuthology.orchestra.run.smithi082.stdout:total 0 2023-09-09T23:14:13.064 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:13 . 2023-09-09T23:14:13.065 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 9 23:14 .. 2023-09-09T23:14:13.065 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:14:13.065 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-09-09T23:14:13.065 INFO:teuthology.orchestra.run.smithi082.stdout:total 16384 2023-09-09T23:14:13.065 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 9 23:13 . 2023-09-09T23:14:13.066 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 9 23:14 .. 2023-09-09T23:14:13.066 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 9 23:14 000000010000000000000001 2023-09-09T23:14:13.066 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:13 archive_status 2023-09-09T23:14:13.066 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:14:13.066 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-09-09T23:14:13.067 INFO:teuthology.orchestra.run.smithi082.stdout:total 0 2023-09-09T23:14:13.067 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 9 23:13 . 2023-09-09T23:14:13.067 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 9 23:13 .. 2023-09-09T23:14:13.067 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:14:13.067 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-09-09T23:14:13.068 INFO:teuthology.orchestra.run.smithi082.stdout:total 8 2023-09-09T23:14:13.068 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 9 23:13 . 2023-09-09T23:14:13.068 INFO:teuthology.orchestra.run.smithi082.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 9 23:14 .. 2023-09-09T23:14:13.068 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 9 23:14 0000 2023-09-09T23:14:13.068 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2023-09-09T23:14:13.179 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 500 -i' 2023-09-09T23:14:13.228 INFO:teuthology.orchestra.run.smithi082.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2023-09-09T23:14:13.235 INFO:teuthology.orchestra.run.smithi082.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2023-09-09T23:14:13.238 INFO:teuthology.orchestra.run.smithi082.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2023-09-09T23:14:13.242 INFO:teuthology.orchestra.run.smithi082.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2023-09-09T23:14:13.414 INFO:teuthology.orchestra.run.smithi082.stderr:creating tables... 2023-09-09T23:14:13.454 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:13 smithi082 ceph-mon[98694]: pgmap v12: 97 pgs: 97 active+clean; 11 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 813 KiB/s wr, 204 op/s 2023-09-09T23:14:13.473 INFO:teuthology.orchestra.run.smithi082.stderr:100000 of 50000000 tuples (0%) done (elapsed 0.06 s, remaining 28.89 s) 2023-09-09T23:14:13.653 INFO:teuthology.orchestra.run.smithi082.stderr:200000 of 50000000 tuples (0%) done (elapsed 0.24 s, remaining 59.00 s) 2023-09-09T23:14:13.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:13 smithi170 ceph-mon[102832]: pgmap v12: 97 pgs: 97 active+clean; 11 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 813 KiB/s wr, 204 op/s 2023-09-09T23:14:13.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:13 smithi098 ceph-mon[103124]: pgmap v12: 97 pgs: 97 active+clean; 11 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 813 KiB/s wr, 204 op/s 2023-09-09T23:14:13.833 INFO:teuthology.orchestra.run.smithi082.stderr:300000 of 50000000 tuples (0%) done (elapsed 0.42 s, remaining 69.16 s) 2023-09-09T23:14:13.892 INFO:teuthology.orchestra.run.smithi082.stderr:400000 of 50000000 tuples (0%) done (elapsed 0.48 s, remaining 59.11 s) 2023-09-09T23:14:14.072 INFO:teuthology.orchestra.run.smithi082.stderr:500000 of 50000000 tuples (1%) done (elapsed 0.66 s, remaining 65.01 s) 2023-09-09T23:14:14.244 INFO:teuthology.orchestra.run.smithi082.stderr:600000 of 50000000 tuples (1%) done (elapsed 0.83 s, remaining 68.21 s) 2023-09-09T23:14:14.318 INFO:teuthology.orchestra.run.smithi082.stderr:700000 of 50000000 tuples (1%) done (elapsed 0.90 s, remaining 63.56 s) 2023-09-09T23:14:14.500 INFO:teuthology.orchestra.run.smithi082.stderr:800000 of 50000000 tuples (1%) done (elapsed 1.08 s, remaining 66.71 s) 2023-09-09T23:14:14.673 INFO:teuthology.orchestra.run.smithi082.stderr:900000 of 50000000 tuples (1%) done (elapsed 1.26 s, remaining 68.59 s) 2023-09-09T23:14:14.740 INFO:teuthology.orchestra.run.smithi082.stderr:1000000 of 50000000 tuples (2%) done (elapsed 1.32 s, remaining 64.90 s) 2023-09-09T23:14:14.931 INFO:teuthology.orchestra.run.smithi082.stderr:1100000 of 50000000 tuples (2%) done (elapsed 1.52 s, remaining 67.35 s) 2023-09-09T23:14:14.998 INFO:teuthology.orchestra.run.smithi082.stderr:1200000 of 50000000 tuples (2%) done (elapsed 1.58 s, remaining 64.35 s) 2023-09-09T23:14:15.169 INFO:teuthology.orchestra.run.smithi082.stderr:1300000 of 50000000 tuples (2%) done (elapsed 1.75 s, remaining 65.69 s) 2023-09-09T23:14:15.355 INFO:teuthology.orchestra.run.smithi082.stderr:1400000 of 50000000 tuples (2%) done (elapsed 1.94 s, remaining 67.31 s) 2023-09-09T23:14:15.460 INFO:teuthology.orchestra.run.smithi082.stderr:1500000 of 50000000 tuples (3%) done (elapsed 2.04 s, remaining 66.10 s) 2023-09-09T23:14:15.615 INFO:teuthology.orchestra.run.smithi082.stderr:1600000 of 50000000 tuples (3%) done (elapsed 2.20 s, remaining 66.54 s) 2023-09-09T23:14:15.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:15 smithi170 ceph-mon[102832]: pgmap v13: 97 pgs: 97 active+clean; 11 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 725 KiB/s wr, 161 op/s 2023-09-09T23:14:15.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:15 smithi082 ceph-mon[98694]: pgmap v13: 97 pgs: 97 active+clean; 11 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 725 KiB/s wr, 161 op/s 2023-09-09T23:14:15.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:15 smithi098 ceph-mon[103124]: pgmap v13: 97 pgs: 97 active+clean; 11 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 725 KiB/s wr, 161 op/s 2023-09-09T23:14:15.780 INFO:teuthology.orchestra.run.smithi082.stderr:1700000 of 50000000 tuples (3%) done (elapsed 2.36 s, remaining 67.19 s) 2023-09-09T23:14:15.853 INFO:teuthology.orchestra.run.smithi082.stderr:1800000 of 50000000 tuples (3%) done (elapsed 2.44 s, remaining 65.26 s) 2023-09-09T23:14:16.037 INFO:teuthology.orchestra.run.smithi082.stderr:1900000 of 50000000 tuples (3%) done (elapsed 2.62 s, remaining 66.37 s) 2023-09-09T23:14:16.205 INFO:teuthology.orchestra.run.smithi082.stderr:2000000 of 50000000 tuples (4%) done (elapsed 2.79 s, remaining 66.94 s) 2023-09-09T23:14:16.290 INFO:teuthology.orchestra.run.smithi082.stderr:2100000 of 50000000 tuples (4%) done (elapsed 2.87 s, remaining 65.56 s) 2023-09-09T23:14:16.472 INFO:teuthology.orchestra.run.smithi082.stderr:2200000 of 50000000 tuples (4%) done (elapsed 3.06 s, remaining 66.41 s) 2023-09-09T23:14:16.577 INFO:teuthology.orchestra.run.smithi082.stderr:2300000 of 50000000 tuples (4%) done (elapsed 3.16 s, remaining 65.56 s) 2023-09-09T23:14:16.741 INFO:teuthology.orchestra.run.smithi082.stderr:2400000 of 50000000 tuples (4%) done (elapsed 3.33 s, remaining 65.96 s) 2023-09-09T23:14:16.937 INFO:teuthology.orchestra.run.smithi082.stderr:2500000 of 50000000 tuples (5%) done (elapsed 3.52 s, remaining 66.90 s) 2023-09-09T23:14:17.033 INFO:teuthology.orchestra.run.smithi082.stderr:2600000 of 50000000 tuples (5%) done (elapsed 3.62 s, remaining 65.94 s) 2023-09-09T23:14:17.202 INFO:teuthology.orchestra.run.smithi082.stderr:2700000 of 50000000 tuples (5%) done (elapsed 3.79 s, remaining 66.34 s) 2023-09-09T23:14:17.379 INFO:teuthology.orchestra.run.smithi082.stderr:2800000 of 50000000 tuples (5%) done (elapsed 3.96 s, remaining 66.81 s) 2023-09-09T23:14:17.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:17 smithi170 ceph-mon[102832]: pgmap v14: 97 pgs: 97 active+clean; 188 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 25 MiB/s wr, 311 op/s 2023-09-09T23:14:17.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:17 smithi082 ceph-mon[98694]: pgmap v14: 97 pgs: 97 active+clean; 188 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 25 MiB/s wr, 311 op/s 2023-09-09T23:14:17.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:17 smithi098 ceph-mon[103124]: pgmap v14: 97 pgs: 97 active+clean; 188 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 25 MiB/s wr, 311 op/s 2023-09-09T23:14:18.020 INFO:teuthology.orchestra.run.smithi082.stderr:2900000 of 50000000 tuples (5%) done (elapsed 4.60 s, remaining 74.79 s) 2023-09-09T23:14:18.349 INFO:teuthology.orchestra.run.smithi082.stderr:3000000 of 50000000 tuples (6%) done (elapsed 4.93 s, remaining 77.29 s) 2023-09-09T23:14:18.451 INFO:teuthology.orchestra.run.smithi082.stderr:3100000 of 50000000 tuples (6%) done (elapsed 5.04 s, remaining 76.19 s) 2023-09-09T23:14:18.621 INFO:teuthology.orchestra.run.smithi082.stderr:3200000 of 50000000 tuples (6%) done (elapsed 5.21 s, remaining 76.13 s) 2023-09-09T23:14:18.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:14:18.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:14:18.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:14:18.764 INFO:teuthology.orchestra.run.smithi082.stderr:3300000 of 50000000 tuples (6%) done (elapsed 5.35 s, remaining 75.69 s) 2023-09-09T23:14:18.882 INFO:teuthology.orchestra.run.smithi082.stderr:3400000 of 50000000 tuples (6%) done (elapsed 5.47 s, remaining 74.92 s) 2023-09-09T23:14:19.003 INFO:teuthology.orchestra.run.smithi082.stderr:3500000 of 50000000 tuples (7%) done (elapsed 5.59 s, remaining 74.23 s) 2023-09-09T23:14:19.190 INFO:teuthology.orchestra.run.smithi082.stderr:3600000 of 50000000 tuples (7%) done (elapsed 5.77 s, remaining 74.43 s) 2023-09-09T23:14:19.288 INFO:teuthology.orchestra.run.smithi082.stderr:3700000 of 50000000 tuples (7%) done (elapsed 5.87 s, remaining 73.49 s) 2023-09-09T23:14:19.444 INFO:teuthology.orchestra.run.smithi082.stderr:3800000 of 50000000 tuples (7%) done (elapsed 6.03 s, remaining 73.29 s) 2023-09-09T23:14:19.633 INFO:teuthology.orchestra.run.smithi082.stderr:3900000 of 50000000 tuples (7%) done (elapsed 6.22 s, remaining 73.49 s) 2023-09-09T23:14:19.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:19 smithi170 ceph-mon[102832]: pgmap v15: 97 pgs: 97 active+clean; 188 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 25 MiB/s wr, 157 op/s 2023-09-09T23:14:19.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:19 smithi082 ceph-mon[98694]: pgmap v15: 97 pgs: 97 active+clean; 188 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 25 MiB/s wr, 157 op/s 2023-09-09T23:14:19.720 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:14:19.734 INFO:teuthology.orchestra.run.smithi082.stderr:4000000 of 50000000 tuples (8%) done (elapsed 6.32 s, remaining 72.67 s) 2023-09-09T23:14:19.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:19 smithi098 ceph-mon[103124]: pgmap v15: 97 pgs: 97 active+clean; 188 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 25 MiB/s wr, 157 op/s 2023-09-09T23:14:19.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.864+0000 7f78c4fc6700 1 -- 172.21.15.82:0/3114042098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78c0101710 msgr2=0x7f78c0101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:19.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.864+0000 7f78c4fc6700 1 --2- 172.21.15.82:0/3114042098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78c0101710 0x7f78c0101af0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009990 tx=0x7f78b0009ca0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.866 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.864+0000 7f78c4fc6700 1 -- 172.21.15.82:0/3114042098 shutdown_connections 2023-09-09T23:14:19.866 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.864+0000 7f78c4fc6700 1 --2- 172.21.15.82:0/3114042098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78c010ab30 0x7f78c010cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.866 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.864+0000 7f78c4fc6700 1 --2- 172.21.15.82:0/3114042098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78c0101710 0x7f78c0101af0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.866 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.864+0000 7f78c4fc6700 1 --2- 172.21.15.82:0/3114042098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78c0102030 0x7f78c010a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.864+0000 7f78c4fc6700 1 -- 172.21.15.82:0/3114042098 >> 172.21.15.82:0/3114042098 conn(0x7f78c00faf50 msgr2=0x7f78c00fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:19.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.864+0000 7f78c4fc6700 1 -- 172.21.15.82:0/3114042098 shutdown_connections 2023-09-09T23:14:19.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.864+0000 7f78c4fc6700 1 -- 172.21.15.82:0/3114042098 wait complete. 2023-09-09T23:14:19.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.865+0000 7f78c4fc6700 1 Processor -- start 2023-09-09T23:14:19.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.865+0000 7f78c4fc6700 1 -- start start 2023-09-09T23:14:19.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.865+0000 7f78c4fc6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78c0101710 0x7f78c00ff360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:19.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.865+0000 7f78c4fc6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78c0102030 0x7f78c00ff8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:19.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.865+0000 7f78c4fc6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78c010ab30 0x7f78c01a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:19.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.865+0000 7f78c4fc6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c01000d0 con 0x7f78c0102030 2023-09-09T23:14:19.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.865+0000 7f78c4fc6700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c01a4e20 con 0x7f78c010ab30 2023-09-09T23:14:19.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.865+0000 7f78c4fc6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c01a4f60 con 0x7f78c0101710 2023-09-09T23:14:19.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.865+0000 7f78bdd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78c0102030 0x7f78c00ff8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:19.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.865+0000 7f78bdd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78c0102030 0x7f78c00ff8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57160/0 (socket says 172.21.15.82:57160) 2023-09-09T23:14:19.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.865+0000 7f78bdd9b700 1 -- 172.21.15.82:0/3647505459 learned_addr learned my addr 172.21.15.82:0/3647505459 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:19.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.865+0000 7f78be59c700 1 --2- 172.21.15.82:0/3647505459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78c0101710 0x7f78c00ff360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:19.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.865+0000 7f78bed9d700 1 --2- 172.21.15.82:0/3647505459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78c010ab30 0x7f78c01a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:19.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.866+0000 7f78bdd9b700 1 -- 172.21.15.82:0/3647505459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78c0101710 msgr2=0x7f78c00ff360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:19.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.866+0000 7f78bdd9b700 1 --2- 172.21.15.82:0/3647505459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78c0101710 0x7f78c00ff360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.866+0000 7f78bdd9b700 1 -- 172.21.15.82:0/3647505459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78c010ab30 msgr2=0x7f78c01a48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:19.871 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.866+0000 7f78bdd9b700 1 --2- 172.21.15.82:0/3647505459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78c010ab30 0x7f78c01a48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.871 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.866+0000 7f78bdd9b700 1 -- 172.21.15.82:0/3647505459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b00095f0 con 0x7f78c0102030 2023-09-09T23:14:19.871 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.866+0000 7f78bed9d700 1 --2- 172.21.15.82:0/3647505459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78c010ab30 0x7f78c01a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:14:19.872 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.866+0000 7f78bdd9b700 1 --2- 172.21.15.82:0/3647505459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78c0102030 0x7f78c00ff8a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f78b400d950 tx=0x7f78b400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:19.872 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.866+0000 7f78af7fe700 1 -- 172.21.15.82:0/3647505459 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b4009930 con 0x7f78c0102030 2023-09-09T23:14:19.872 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.866+0000 7f78c4fc6700 1 -- 172.21.15.82:0/3647505459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78c01a51c0 con 0x7f78c0102030 2023-09-09T23:14:19.872 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.866+0000 7f78af7fe700 1 -- 172.21.15.82:0/3647505459 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78b4010460 con 0x7f78c0102030 2023-09-09T23:14:19.873 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.866+0000 7f78af7fe700 1 -- 172.21.15.82:0/3647505459 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b400f620 con 0x7f78c0102030 2023-09-09T23:14:19.873 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.866+0000 7f78c4fc6700 1 -- 172.21.15.82:0/3647505459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c01a56f0 con 0x7f78c0102030 2023-09-09T23:14:19.880 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.868+0000 7f78af7fe700 1 -- 172.21.15.82:0/3647505459 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135492+0+0 (secure 0 0 0) 0x7f78b4010860 con 0x7f78c0102030 2023-09-09T23:14:19.881 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.868+0000 7f78af7fe700 1 --2- 172.21.15.82:0/3647505459 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f78a80a08d0 0x7f78a80a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:19.881 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.869+0000 7f78af7fe700 1 -- 172.21.15.82:0/3647505459 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f78b40cbf70 con 0x7f78c0102030 2023-09-09T23:14:19.881 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.869+0000 7f78be59c700 1 --2- 172.21.15.82:0/3647505459 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f78a80a08d0 0x7f78a80a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:19.881 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.869+0000 7f78be59c700 1 --2- 172.21.15.82:0/3647505459 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f78a80a08d0 0x7f78a80a2d90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f78b000b5c0 tx=0x7f78b0005fb0 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:19.881 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.871+0000 7f78d118cb80 1 Processor -- start 2023-09-09T23:14:19.882 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.871+0000 7f78d118cb80 1 -- start start 2023-09-09T23:14:19.882 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.871+0000 7f78d118cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611987ba710 0x5611987baaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:19.882 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.871+0000 7f78d118cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611987bb030 0x5611987c4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:19.882 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.871+0000 7f78d118cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611987c4e40 0x5611987c7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:19.882 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.871+0000 7f78d118cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561198713730 con 0x5611987bb030 2023-09-09T23:14:19.883 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.871+0000 7f78d118cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x561198711050 con 0x5611987ba710 2023-09-09T23:14:19.883 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.871+0000 7f78d118cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56119870c940 con 0x5611987c4e40 2023-09-09T23:14:19.883 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.872+0000 7f78bdd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611987bb030 0x5611987c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:19.883 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.872+0000 7f78be59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611987ba710 0x5611987baaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:19.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.872+0000 7f78be59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611987ba710 0x5611987baaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:42730/0 (socket says 172.21.15.82:42730) 2023-09-09T23:14:19.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.872+0000 7f78be59c700 1 -- 172.21.15.82:0/1218817876 learned_addr learned my addr 172.21.15.82:0/1218817876 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:19.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.872+0000 7f78bdd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611987bb030 0x5611987c4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57168/0 (socket says 172.21.15.82:57168) 2023-09-09T23:14:19.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.872+0000 7f78bed9d700 1 --2- 172.21.15.82:0/1218817876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611987c4e40 0x5611987c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:19.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.872+0000 7f78be59c700 1 -- 172.21.15.82:0/1218817876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611987c4e40 msgr2=0x5611987c7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:19.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.872+0000 7f78be59c700 1 --2- 172.21.15.82:0/1218817876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611987c4e40 0x5611987c7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.872+0000 7f78be59c700 1 -- 172.21.15.82:0/1218817876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611987bb030 msgr2=0x5611987c4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:19.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.872+0000 7f78be59c700 1 --2- 172.21.15.82:0/1218817876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611987bb030 0x5611987c4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.872+0000 7f78be59c700 1 -- 172.21.15.82:0/1218817876 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561198538de0 con 0x5611987ba710 2023-09-09T23:14:19.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.872+0000 7f78bdd9b700 1 --2- 172.21.15.82:0/1218817876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611987bb030 0x5611987c4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:14:19.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.872+0000 7f78be59c700 1 --2- 172.21.15.82:0/1218817876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611987ba710 0x5611987baaf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f78b000b800 tx=0x7f78b0020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c5374a164d20f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:19.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.872+0000 7f78acff9700 1 -- 172.21.15.82:0/1218817876 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b002a070 con 0x5611987ba710 2023-09-09T23:14:19.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.872+0000 7f78acff9700 1 -- 172.21.15.82:0/1218817876 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78b001e040 con 0x5611987ba710 2023-09-09T23:14:19.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.872+0000 7f78acff9700 1 -- 172.21.15.82:0/1218817876 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b00237e0 con 0x5611987ba710 2023-09-09T23:14:19.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.873+0000 7f78d118cb80 1 -- 172.21.15.82:0/1218817876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611987ba710 msgr2=0x5611987baaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:19.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.873+0000 7f78d118cb80 1 --2- 172.21.15.82:0/1218817876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611987ba710 0x5611987baaf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f78b000b800 tx=0x7f78b0020000 comp rx=0 tx=0).stop 2023-09-09T23:14:19.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.873+0000 7f78d118cb80 1 -- 172.21.15.82:0/1218817876 shutdown_connections 2023-09-09T23:14:19.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.873+0000 7f78d118cb80 1 --2- 172.21.15.82:0/1218817876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611987ba710 0x5611987baaf0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.873+0000 7f78d118cb80 1 --2- 172.21.15.82:0/1218817876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611987c4e40 0x5611987c7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.873+0000 7f78d118cb80 1 --2- 172.21.15.82:0/1218817876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611987bb030 0x5611987c4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.873+0000 7f78d118cb80 1 -- 172.21.15.82:0/1218817876 >> 172.21.15.82:0/1218817876 conn(0x56119872f090 msgr2=0x5611987b5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:19.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.873+0000 7f78d118cb80 1 -- 172.21.15.82:0/1218817876 shutdown_connections 2023-09-09T23:14:19.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.873+0000 7f78d118cb80 1 -- 172.21.15.82:0/1218817876 wait complete. 2023-09-09T23:14:19.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.874+0000 7f78d118cb80 1 -- start start 2023-09-09T23:14:19.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.874+0000 7f78d118cb80 1 Processor -- start 2023-09-09T23:14:19.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.874+0000 7f78d118cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611987ba710 0x5611987c4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:19.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.874+0000 7f78d118cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611987bb030 0x5611987c1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:19.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.874+0000 7f78d118cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611987c4e40 0x5611987c2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:19.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.874+0000 7f78d118cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611986aab70 con 0x5611987bb030 2023-09-09T23:14:19.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.874+0000 7f78d118cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611986aa870 con 0x5611987ba710 2023-09-09T23:14:19.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.874+0000 7f78d118cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56119858e920 con 0x5611987c4e40 2023-09-09T23:14:19.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.874+0000 7f78d118cb80 10 client.0 _ll_register_callbacks cb 0x5611985f4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:14:19.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.874+0000 7f78bdd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611987bb030 0x5611987c1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:19.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.874+0000 7f78bdd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611987bb030 0x5611987c1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57174/0 (socket says 172.21.15.82:57174) 2023-09-09T23:14:19.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.874+0000 7f78bdd9b700 1 -- 172.21.15.82:0/817837832 learned_addr learned my addr 172.21.15.82:0/817837832 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:19.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.874+0000 7f78be59c700 1 --2- 172.21.15.82:0/817837832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611987ba710 0x5611987c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:19.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.874+0000 7f78bed9d700 1 --2- 172.21.15.82:0/817837832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611987c4e40 0x5611987c2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:19.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.874+0000 7f78bdd9b700 1 -- 172.21.15.82:0/817837832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611987c4e40 msgr2=0x5611987c2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:19.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.874+0000 7f78bdd9b700 1 --2- 172.21.15.82:0/817837832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611987c4e40 0x5611987c2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.874+0000 7f78bdd9b700 1 -- 172.21.15.82:0/817837832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611987ba710 msgr2=0x5611987c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:19.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.874+0000 7f78bdd9b700 1 --2- 172.21.15.82:0/817837832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611987ba710 0x5611987c4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.874+0000 7f78bdd9b700 1 -- 172.21.15.82:0/817837832 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c01a56f0 con 0x5611987bb030 2023-09-09T23:14:19.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.874+0000 7f78bdd9b700 1 --2- 172.21.15.82:0/817837832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611987bb030 0x5611987c1a60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f78b400b2f0 tx=0x7f78b4099a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1196a925ae1b6908 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:19.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.874+0000 7f78a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:14:19.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.875+0000 7f78a5ffb700 1 -- 172.21.15.82:0/817837832 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b4098930 con 0x5611987bb030 2023-09-09T23:14:19.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.875+0000 7f78a5ffb700 1 -- 172.21.15.82:0/817837832 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f78b40a12b0 con 0x5611987bb030 2023-09-09T23:14:19.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.875+0000 7f78d118cb80 1 -- 172.21.15.82:0/817837832 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5611986644f0 con 0x5611987bb030 2023-09-09T23:14:19.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.875+0000 7f78a5ffb700 1 -- 172.21.15.82:0/817837832 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78b40b71a0 con 0x5611987bb030 2023-09-09T23:14:19.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.875+0000 7f78d118cb80 10 client.15816 fetch_fsmap learned FSMap version 28 2023-09-09T23:14:19.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.875+0000 7f78d118cb80 1 -- 172.21.15.82:0/817837832 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f78c010dcb0 con 0x5611987bb030 2023-09-09T23:14:19.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.875+0000 7f78a5ffb700 1 -- 172.21.15.82:0/817837832 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5611986644f0 con 0x5611987bb030 2023-09-09T23:14:19.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.875+0000 7f78a5ffb700 1 -- 172.21.15.82:0/817837832 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f78b40c9920 con 0x5611987bb030 2023-09-09T23:14:19.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.875+0000 7f78d118cb80 10 client.15816 fetch_fsmap finished waiting for FSMap version 28 2023-09-09T23:14:19.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.875+0000 7f78d118cb80 10 client.15816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:14:19.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.875+0000 7f78d118cb80 20 client.15816 populate_metadata read hostname 'smithi082' 2023-09-09T23:14:19.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.875+0000 7f78d118cb80 1 --2- 172.21.15.82:0/817837832 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5611987d4700 0x5611987d6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:19.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.875+0000 7f78d118cb80 4 client.15816 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:14:19.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.875+0000 7f78d118cb80 1 -- 172.21.15.82:0/817837832 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611987218a0 con 0x5611987d4700 2023-09-09T23:14:19.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.875+0000 7f78be59c700 1 --2- 172.21.15.82:0/817837832 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5611987d4700 0x5611987d6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:19.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.876+0000 7f78a5ffb700 10 client.15816 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:14:19.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.876+0000 7f78be59c700 1 --2- 172.21.15.82:0/817837832 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5611987d4700 0x5611987d6b20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=692dced65816fffd server_cookie=a0c1f08736274e1c in_seq=0 out_seq=0 2023-09-09T23:14:19.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.876+0000 7f78a5ffb700 1 -- 172.21.15.82:0/817837832 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5611987218a0 con 0x5611987d4700 2023-09-09T23:14:19.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.876+0000 7f78a5ffb700 10 client.15816 handle_command_reply: tid=0 2023-09-09T23:14:19.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.876+0000 7f78d118cb80 1 client.15816 shutdown 2023-09-09T23:14:19.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.876+0000 7f78d118cb80 1 -- 172.21.15.82:0/817837832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611987bb030 msgr2=0x5611987c1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:19.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.876+0000 7f78d118cb80 1 --2- 172.21.15.82:0/817837832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611987bb030 0x5611987c1a60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f78b400b2f0 tx=0x7f78b4099a80 comp rx=0 tx=0).stop 2023-09-09T23:14:19.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.876+0000 7f78d118cb80 1 -- 172.21.15.82:0/817837832 shutdown_connections 2023-09-09T23:14:19.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.876+0000 7f78d118cb80 1 --2- 172.21.15.82:0/817837832 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5611987d4700 0x5611987d6b20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.897 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.876+0000 7f78d118cb80 1 --2- 172.21.15.82:0/817837832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611987ba710 0x5611987c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.897 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.876+0000 7f78d118cb80 1 --2- 172.21.15.82:0/817837832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611987c4e40 0x5611987c2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.897 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.876+0000 7f78a5ffb700 0 client.15816 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:14:19.897 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.876+0000 7f78d118cb80 1 --2- 172.21.15.82:0/817837832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611987bb030 0x5611987c1a60 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.897 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.876+0000 7f78d118cb80 1 -- 172.21.15.82:0/817837832 >> 172.21.15.82:0/817837832 conn(0x56119872f090 msgr2=0x5611987b57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:19.898 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.876+0000 7f78d118cb80 1 -- 172.21.15.82:0/817837832 shutdown_connections 2023-09-09T23:14:19.898 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.876+0000 7f78d118cb80 1 -- 172.21.15.82:0/817837832 wait complete. 2023-09-09T23:14:19.898 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.877+0000 7f78d118cb80 20 client.15816 trim_cache size 0 max 16384 2023-09-09T23:14:19.898 INFO:teuthology.orchestra.run.smithi082.stderr:4100000 of 50000000 tuples (8%) done (elapsed 6.47 s, remaining 72.42 s) 2023-09-09T23:14:19.903 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.893+0000 7f78d118cb80 1 Processor -- start 2023-09-09T23:14:19.903 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.893+0000 7f78d118cb80 1 -- start start 2023-09-09T23:14:19.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.893+0000 7f78d118cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611987ba710 0x5611987cb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:19.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.893+0000 7f78d118cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611987bb030 0x5611987c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:19.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.893+0000 7f78d118cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611987c4e40 0x5611987bcd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:19.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.893+0000 7f78d118cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611987b70b0 con 0x5611987bb030 2023-09-09T23:14:19.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.893+0000 7f78d118cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611987bb9f0 con 0x5611987c4e40 2023-09-09T23:14:19.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.893+0000 7f78d118cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611987b23d0 con 0x5611987ba710 2023-09-09T23:14:19.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.893+0000 7f78bdd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611987bb030 0x5611987c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:19.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.894+0000 7f78bdd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611987bb030 0x5611987c4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57182/0 (socket says 172.21.15.82:57182) 2023-09-09T23:14:19.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.894+0000 7f78bdd9b700 1 -- 172.21.15.82:0/4237356644 learned_addr learned my addr 172.21.15.82:0/4237356644 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:19.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.894+0000 7f78bed9d700 1 --2- 172.21.15.82:0/4237356644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611987c4e40 0x5611987bcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:19.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.894+0000 7f78be59c700 1 --2- 172.21.15.82:0/4237356644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611987ba710 0x5611987cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:19.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.894+0000 7f78bdd9b700 1 -- 172.21.15.82:0/4237356644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611987ba710 msgr2=0x5611987cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:19.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.894+0000 7f78bdd9b700 1 --2- 172.21.15.82:0/4237356644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611987ba710 0x5611987cb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.894+0000 7f78bdd9b700 1 -- 172.21.15.82:0/4237356644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611987c4e40 msgr2=0x5611987bcd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:19.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.894+0000 7f78bdd9b700 1 --2- 172.21.15.82:0/4237356644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611987c4e40 0x5611987bcd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.894+0000 7f78bdd9b700 1 -- 172.21.15.82:0/4237356644 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561198521c60 con 0x5611987bb030 2023-09-09T23:14:19.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.894+0000 7f78bdd9b700 1 --2- 172.21.15.82:0/4237356644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611987bb030 0x5611987c4380 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f78b40991d0 tx=0x7f78b40977f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5cdc3ecb8e5532 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:19.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.894+0000 7f78ad7fa700 1 -- 172.21.15.82:0/4237356644 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b4014050 con 0x5611987bb030 2023-09-09T23:14:19.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.894+0000 7f78ad7fa700 1 -- 172.21.15.82:0/4237356644 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78b40acb60 con 0x5611987bb030 2023-09-09T23:14:19.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.894+0000 7f78ad7fa700 1 -- 172.21.15.82:0/4237356644 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b40a1080 con 0x5611987bb030 2023-09-09T23:14:19.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.895+0000 7f78d118cb80 1 -- 172.21.15.82:0/4237356644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611987bb030 msgr2=0x5611987c4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:19.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.895+0000 7f78d118cb80 1 --2- 172.21.15.82:0/4237356644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611987bb030 0x5611987c4380 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f78b40991d0 tx=0x7f78b40977f0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.895+0000 7f78d118cb80 1 -- 172.21.15.82:0/4237356644 shutdown_connections 2023-09-09T23:14:19.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.895+0000 7f78d118cb80 1 --2- 172.21.15.82:0/4237356644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611987c4e40 0x5611987bcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.895+0000 7f78d118cb80 1 --2- 172.21.15.82:0/4237356644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611987ba710 0x5611987cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.895+0000 7f78d118cb80 1 --2- 172.21.15.82:0/4237356644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611987bb030 0x5611987c4380 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.895+0000 7f78d118cb80 1 -- 172.21.15.82:0/4237356644 >> 172.21.15.82:0/4237356644 conn(0x56119872f090 msgr2=0x5611987d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:19.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.895+0000 7f78d118cb80 1 -- 172.21.15.82:0/4237356644 shutdown_connections 2023-09-09T23:14:19.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.895+0000 7f78d118cb80 1 -- 172.21.15.82:0/4237356644 wait complete. 2023-09-09T23:14:19.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.896+0000 7f78d118cb80 1 -- start start 2023-09-09T23:14:19.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.896+0000 7f78d118cb80 1 Processor -- start 2023-09-09T23:14:19.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.896+0000 7f78d118cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611987ba710 0x5611987cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:19.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.896+0000 7f78d118cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611987bb030 0x5611987c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:19.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.896+0000 7f78d118cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611987c4e40 0x5611987ca280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:19.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.896+0000 7f78d118cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611987ca990 con 0x5611987bb030 2023-09-09T23:14:19.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.896+0000 7f78d118cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611987cab00 con 0x5611987c4e40 2023-09-09T23:14:19.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.896+0000 7f78d118cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561198828580 con 0x5611987ba710 2023-09-09T23:14:19.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.896+0000 7f78d118cb80 10 client.0 _ll_register_callbacks cb 0x5611985f4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:14:19.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.896+0000 7f78bdd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611987bb030 0x5611987c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:19.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.896+0000 7f78bdd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611987bb030 0x5611987c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57188/0 (socket says 172.21.15.82:57188) 2023-09-09T23:14:19.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.896+0000 7f78bdd9b700 1 -- 172.21.15.82:0/381437869 learned_addr learned my addr 172.21.15.82:0/381437869 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:19.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.896+0000 7f78be59c700 1 --2- 172.21.15.82:0/381437869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611987ba710 0x5611987cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:19.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.896+0000 7f78bed9d700 1 --2- 172.21.15.82:0/381437869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611987c4e40 0x5611987ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:19.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.896+0000 7f78bdd9b700 1 -- 172.21.15.82:0/381437869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611987ba710 msgr2=0x5611987cb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:19.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.896+0000 7f78bdd9b700 1 --2- 172.21.15.82:0/381437869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611987ba710 0x5611987cb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.896+0000 7f78bdd9b700 1 -- 172.21.15.82:0/381437869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611987c4e40 msgr2=0x5611987ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:19.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.896+0000 7f78bdd9b700 1 --2- 172.21.15.82:0/381437869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611987c4e40 0x5611987ca280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.896+0000 7f78bdd9b700 1 -- 172.21.15.82:0/381437869 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c010dcb0 con 0x5611987bb030 2023-09-09T23:14:19.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.897+0000 7f78bdd9b700 1 --2- 172.21.15.82:0/381437869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611987bb030 0x5611987c9d20 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f78b40a0d80 tx=0x7f78b400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ddf22f5f164af76 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:19.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.897+0000 7f78a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:14:19.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.897+0000 7f78a5ffb700 1 -- 172.21.15.82:0/381437869 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b4014040 con 0x5611987bb030 2023-09-09T23:14:19.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.897+0000 7f78a5ffb700 1 -- 172.21.15.82:0/381437869 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f78b40a12b0 con 0x5611987bb030 2023-09-09T23:14:19.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.897+0000 7f78d118cb80 1 -- 172.21.15.82:0/381437869 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f78b001e040 con 0x5611987bb030 2023-09-09T23:14:19.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.897+0000 7f78a5ffb700 1 -- 172.21.15.82:0/381437869 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78b4098730 con 0x5611987bb030 2023-09-09T23:14:19.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.897+0000 7f78d118cb80 10 client.15822 fetch_fsmap learned FSMap version 28 2023-09-09T23:14:19.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.897+0000 7f78d118cb80 1 -- 172.21.15.82:0/381437869 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56119855f250 con 0x5611987bb030 2023-09-09T23:14:19.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.897+0000 7f78a5ffb700 1 -- 172.21.15.82:0/381437869 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78b001e040 con 0x5611987bb030 2023-09-09T23:14:19.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.897+0000 7f78a5ffb700 1 -- 172.21.15.82:0/381437869 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f78b409ddb0 con 0x5611987bb030 2023-09-09T23:14:19.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.897+0000 7f78d118cb80 10 client.15822 fetch_fsmap finished waiting for FSMap version 28 2023-09-09T23:14:19.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.897+0000 7f78d118cb80 10 client.15822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:14:19.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.897+0000 7f78d118cb80 20 client.15822 populate_metadata read hostname 'smithi082' 2023-09-09T23:14:19.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.897+0000 7f78d118cb80 1 --2- 172.21.15.82:0/381437869 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x561198829810 0x56119882bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:19.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.897+0000 7f78d118cb80 4 client.15822 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:14:19.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.897+0000 7f78d118cb80 1 -- 172.21.15.82:0/381437869 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561198705920 con 0x561198829810 2023-09-09T23:14:19.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.897+0000 7f78be59c700 1 --2- 172.21.15.82:0/381437869 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x561198829810 0x56119882bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:19.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.898+0000 7f78be59c700 1 --2- 172.21.15.82:0/381437869 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x561198829810 0x56119882bc30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=857476f9ac8e650e server_cookie=9ed76c9226d6c853 in_seq=0 out_seq=0 2023-09-09T23:14:19.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.898+0000 7f78a5ffb700 10 client.15822 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:14:19.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.898+0000 7f78a5ffb700 1 -- 172.21.15.82:0/381437869 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561198705920 con 0x561198829810 2023-09-09T23:14:19.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.898+0000 7f78a5ffb700 10 client.15822 handle_command_reply: tid=0 2023-09-09T23:14:19.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.898+0000 7f78d118cb80 1 client.15822 shutdown 2023-09-09T23:14:19.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.898+0000 7f78d118cb80 1 -- 172.21.15.82:0/381437869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611987bb030 msgr2=0x5611987c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:19.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.898+0000 7f78d118cb80 1 --2- 172.21.15.82:0/381437869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611987bb030 0x5611987c9d20 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f78b40a0d80 tx=0x7f78b400bf90 comp rx=0 tx=0).stop 2023-09-09T23:14:19.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.898+0000 7f78d118cb80 1 -- 172.21.15.82:0/381437869 shutdown_connections 2023-09-09T23:14:19.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.898+0000 7f78d118cb80 1 --2- 172.21.15.82:0/381437869 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x561198829810 0x56119882bc30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.898+0000 7f78d118cb80 1 --2- 172.21.15.82:0/381437869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5611987c4e40 0x5611987ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.898+0000 7f78a5ffb700 0 client.15822 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:14:19.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.898+0000 7f78d118cb80 1 --2- 172.21.15.82:0/381437869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5611987ba710 0x5611987cb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.898+0000 7f78d118cb80 1 --2- 172.21.15.82:0/381437869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611987bb030 0x5611987c9d20 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.898+0000 7f78d118cb80 1 -- 172.21.15.82:0/381437869 >> 172.21.15.82:0/381437869 conn(0x56119872f090 msgr2=0x5611987c1300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:19.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.898+0000 7f78d118cb80 1 -- 172.21.15.82:0/381437869 shutdown_connections 2023-09-09T23:14:19.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.898+0000 7f78d118cb80 1 -- 172.21.15.82:0/381437869 wait complete. 2023-09-09T23:14:19.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.898+0000 7f78d118cb80 20 client.15822 trim_cache size 0 max 16384 2023-09-09T23:14:19.919 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:14:19.919 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "no active scrubs running", 2023-09-09T23:14:19.919 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": {} 2023-09-09T23:14:19.919 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:14:19.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.899+0000 7f78c4fc6700 1 -- 172.21.15.82:0/3647505459 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f78a80a08d0 msgr2=0x7f78a80a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:19.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.899+0000 7f78c4fc6700 1 --2- 172.21.15.82:0/3647505459 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f78a80a08d0 0x7f78a80a2d90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f78b000b5c0 tx=0x7f78b0005fb0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.899+0000 7f78c4fc6700 1 -- 172.21.15.82:0/3647505459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78c0102030 msgr2=0x7f78c00ff8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:19.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.899+0000 7f78c4fc6700 1 --2- 172.21.15.82:0/3647505459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78c0102030 0x7f78c00ff8a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f78b400d950 tx=0x7f78b400dc60 comp rx=0 tx=0).stop 2023-09-09T23:14:19.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.899+0000 7f78c4fc6700 1 -- 172.21.15.82:0/3647505459 shutdown_connections 2023-09-09T23:14:19.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.899+0000 7f78c4fc6700 1 --2- 172.21.15.82:0/3647505459 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f78a80a08d0 0x7f78a80a2d90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.899+0000 7f78c4fc6700 1 --2- 172.21.15.82:0/3647505459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78c010ab30 0x7f78c01a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.899+0000 7f78c4fc6700 1 --2- 172.21.15.82:0/3647505459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78c0101710 0x7f78c00ff360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.899+0000 7f78c4fc6700 1 --2- 172.21.15.82:0/3647505459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78c0102030 0x7f78c00ff8a0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:19.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.899+0000 7f78c4fc6700 1 -- 172.21.15.82:0/3647505459 >> 172.21.15.82:0/3647505459 conn(0x7f78c00faf50 msgr2=0x7f78c010cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:19.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.899+0000 7f78c4fc6700 1 -- 172.21.15.82:0/3647505459 shutdown_connections 2023-09-09T23:14:19.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:19.899+0000 7f78c4fc6700 1 -- 172.21.15.82:0/3647505459 wait complete. 2023-09-09T23:14:19.923 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-09T23:14:19.924 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-09T23:14:20.085 INFO:teuthology.orchestra.run.smithi082.stderr:4200000 of 50000000 tuples (8%) done (elapsed 6.67 s, remaining 72.73 s) 2023-09-09T23:14:20.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.099+0000 7f32b419c700 1 -- 172.21.15.82:0/3090838235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32ac0ffe20 msgr2=0x7f32ac10c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.099+0000 7f32b419c700 1 --2- 172.21.15.82:0/3090838235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32ac0ffe20 0x7f32ac10c330 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f32a400b0a0 tx=0x7f32a400b3b0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.100+0000 7f32b419c700 1 -- 172.21.15.82:0/3090838235 shutdown_connections 2023-09-09T23:14:20.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.100+0000 7f32b419c700 1 --2- 172.21.15.82:0/3090838235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32ac105500 0x7f32ac1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.100+0000 7f32b419c700 1 --2- 172.21.15.82:0/3090838235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32ac0ffe20 0x7f32ac10c330 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.100+0000 7f32b419c700 1 --2- 172.21.15.82:0/3090838235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32ac0ff460 0x7f32ac0ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.100+0000 7f32b419c700 1 -- 172.21.15.82:0/3090838235 >> 172.21.15.82:0/3090838235 conn(0x7f32ac0faf10 msgr2=0x7f32ac0fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:20.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.100+0000 7f32b419c700 1 -- 172.21.15.82:0/3090838235 shutdown_connections 2023-09-09T23:14:20.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.100+0000 7f32b419c700 1 -- 172.21.15.82:0/3090838235 wait complete. 2023-09-09T23:14:20.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.100+0000 7f32b419c700 1 Processor -- start 2023-09-09T23:14:20.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.100+0000 7f32b419c700 1 -- start start 2023-09-09T23:14:20.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.100+0000 7f32b419c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32ac0ff460 0x7f32ac1973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.100+0000 7f32b419c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32ac0ffe20 0x7f32ac1978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.100+0000 7f32b419c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32ac105500 0x7f32ac1a2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.100+0000 7f32b419c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac1981f0 con 0x7f32ac0ffe20 2023-09-09T23:14:20.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.100+0000 7f32b419c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac1a2960 con 0x7f32ac0ff460 2023-09-09T23:14:20.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.100+0000 7f32b419c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac1a2ad0 con 0x7f32ac105500 2023-09-09T23:14:20.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.101+0000 7f32b1737700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32ac0ffe20 0x7f32ac1978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.101+0000 7f32b1737700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32ac0ffe20 0x7f32ac1978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57202/0 (socket says 172.21.15.82:57202) 2023-09-09T23:14:20.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.101+0000 7f32b1737700 1 -- 172.21.15.82:0/3932962369 learned_addr learned my addr 172.21.15.82:0/3932962369 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:20.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.101+0000 7f32b1f38700 1 --2- 172.21.15.82:0/3932962369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32ac0ff460 0x7f32ac1973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.101+0000 7f32b2739700 1 --2- 172.21.15.82:0/3932962369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32ac105500 0x7f32ac1a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.101+0000 7f32b1737700 1 -- 172.21.15.82:0/3932962369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32ac105500 msgr2=0x7f32ac1a2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.101+0000 7f32b1737700 1 --2- 172.21.15.82:0/3932962369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32ac105500 0x7f32ac1a2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.101+0000 7f32b1737700 1 -- 172.21.15.82:0/3932962369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32ac0ff460 msgr2=0x7f32ac1973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.101+0000 7f32b1737700 1 --2- 172.21.15.82:0/3932962369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32ac0ff460 0x7f32ac1973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.101+0000 7f32b1737700 1 -- 172.21.15.82:0/3932962369 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a4009d00 con 0x7f32ac0ffe20 2023-09-09T23:14:20.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.101+0000 7f32b1737700 1 --2- 172.21.15.82:0/3932962369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32ac0ffe20 0x7f32ac1978e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f329c00c9b0 tx=0x7f329c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:20.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.101+0000 7f32a2ffd700 1 -- 172.21.15.82:0/3932962369 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c007b00 con 0x7f32ac0ffe20 2023-09-09T23:14:20.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.101+0000 7f32b419c700 1 -- 172.21.15.82:0/3932962369 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32ac1a2db0 con 0x7f32ac0ffe20 2023-09-09T23:14:20.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.101+0000 7f32a2ffd700 1 -- 172.21.15.82:0/3932962369 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f329c007c60 con 0x7f32ac0ffe20 2023-09-09T23:14:20.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.101+0000 7f32a2ffd700 1 -- 172.21.15.82:0/3932962369 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c018660 con 0x7f32ac0ffe20 2023-09-09T23:14:20.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.101+0000 7f32b419c700 1 -- 172.21.15.82:0/3932962369 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32ac1a3220 con 0x7f32ac0ffe20 2023-09-09T23:14:20.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.102+0000 7f32b419c700 1 -- 172.21.15.82:0/3932962369 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32ac066b60 con 0x7f32ac0ffe20 2023-09-09T23:14:20.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.104+0000 7f32a2ffd700 1 -- 172.21.15.82:0/3932962369 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135492+0+0 (secure 0 0 0) 0x7f329c00f450 con 0x7f32ac0ffe20 2023-09-09T23:14:20.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.105+0000 7f32a2ffd700 1 --2- 172.21.15.82:0/3932962369 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f32980a0920 0x7f32980a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.105+0000 7f32a2ffd700 1 -- 172.21.15.82:0/3932962369 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f329c0cc480 con 0x7f32ac0ffe20 2023-09-09T23:14:20.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.105+0000 7f32b1f38700 1 --2- 172.21.15.82:0/3932962369 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f32980a0920 0x7f32980a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.105+0000 7f32b1f38700 1 --2- 172.21.15.82:0/3932962369 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f32980a0920 0x7f32980a2de0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009a10 tx=0x7f32a8009190 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:20.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.108+0000 7f32a2ffd700 1 -- 172.21.15.82:0/3932962369 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f329c095ac0 con 0x7f32ac0ffe20 2023-09-09T23:14:20.183 INFO:teuthology.orchestra.run.smithi082.stderr:4300000 of 50000000 tuples (8%) done (elapsed 6.77 s, remaining 71.93 s) 2023-09-09T23:14:20.327 INFO:teuthology.orchestra.run.smithi082.stderr:4400000 of 50000000 tuples (8%) done (elapsed 6.91 s, remaining 71.62 s) 2023-09-09T23:14:20.395 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.397+0000 7f32b419c700 1 -- 172.21.15.82:0/3932962369 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32ac068d80 con 0x7f32ac0ffe20 2023-09-09T23:14:20.396 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.397+0000 7f32a2ffd700 1 -- 172.21.15.82:0/3932962369 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11369 (secure 0 0 0) 0x7f329c0990e0 con 0x7f32ac0ffe20 2023-09-09T23:14:20.396 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-09T23:14:20.396 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14640,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2471432684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2471432684},{"type":"v1","addr":"172.21.15.82:6835","nonce":2471432684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3579387137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3579387137},{"type":"v1","addr":"172.21.15.82:6837","nonce":3579387137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-09T23:12:17.586036+0000","modified":"2023-09-09T23:14:16.683847+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24542,"mds_2":24523,"mds_3":24491,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14712":{"gid":14712,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/2127301345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2127301345},{"type":"v1","addr":"172.21.15.82:6839","nonce":2127301345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14724":{"gid":14724,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/4143604488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":4143604488},{"type":"v1","addr":"172.21.15.82:6841","nonce":4143604488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.170:6833/1931961940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1931961940},{"type":"v1","addr":"172.21.15.170:6833","nonce":1931961940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24491":{"gid":24491,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.98:6833/3855146755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3855146755},{"type":"v1","addr":"172.21.15.98:6833","nonce":3855146755}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2313861511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2313861511},{"type":"v1","addr":"172.21.15.170:6835","nonce":2313861511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/4006360308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":4006360308},{"type":"v1","addr":"172.21.15.170:6837","nonce":4006360308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1356474828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1356474828},{"type":"v1","addr":"172.21.15.98:6835","nonce":1356474828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24523":{"gid":24523,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.170:6839/3858369735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3858369735},{"type":"v1","addr":"172.21.15.170:6839","nonce":3858369735}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.98:6837/2019449840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2019449840},{"type":"v1","addr":"172.21.15.98:6837","nonce":2019449840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/2785086371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":2785086371},{"type":"v1","addr":"172.21.15.98:6839","nonce":2785086371}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-09T23:14:20.398 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.399+0000 7f32b419c700 1 -- 172.21.15.82:0/3932962369 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f32980a0920 msgr2=0x7f32980a2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.398 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.399+0000 7f32b419c700 1 --2- 172.21.15.82:0/3932962369 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f32980a0920 0x7f32980a2de0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009a10 tx=0x7f32a8009190 comp rx=0 tx=0).stop 2023-09-09T23:14:20.398 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.399+0000 7f32b419c700 1 -- 172.21.15.82:0/3932962369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32ac0ffe20 msgr2=0x7f32ac1978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.399+0000 7f32b419c700 1 --2- 172.21.15.82:0/3932962369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32ac0ffe20 0x7f32ac1978e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f329c00c9b0 tx=0x7f329c00ccc0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.399+0000 7f32b419c700 1 -- 172.21.15.82:0/3932962369 shutdown_connections 2023-09-09T23:14:20.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.399+0000 7f32b419c700 1 --2- 172.21.15.82:0/3932962369 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f32980a0920 0x7f32980a2de0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.399+0000 7f32b419c700 1 --2- 172.21.15.82:0/3932962369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32ac0ff460 0x7f32ac1973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.399+0000 7f32b419c700 1 --2- 172.21.15.82:0/3932962369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32ac105500 0x7f32ac1a2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.399+0000 7f32b419c700 1 --2- 172.21.15.82:0/3932962369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32ac0ffe20 0x7f32ac1978e0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.399+0000 7f32b419c700 1 -- 172.21.15.82:0/3932962369 >> 172.21.15.82:0/3932962369 conn(0x7f32ac0faf10 msgr2=0x7f32ac106350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:20.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.400+0000 7f32b419c700 1 -- 172.21.15.82:0/3932962369 shutdown_connections 2023-09-09T23:14:20.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.400+0000 7f32b419c700 1 -- 172.21.15.82:0/3932962369 wait complete. 2023-09-09T23:14:20.401 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 28 2023-09-09T23:14:20.410 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-09-09T23:14:20.426 INFO:teuthology.orchestra.run.smithi082.stderr:4500000 of 50000000 tuples (9%) done (elapsed 7.01 s, remaining 70.88 s) 2023-09-09T23:14:20.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.550+0000 7f6d7950f700 1 -- 172.21.15.82:0/311857514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d7410ab40 msgr2=0x7f6d7410cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.550+0000 7f6d7950f700 1 --2- 172.21.15.82:0/311857514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d7410ab40 0x7f6d7410cfe0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00b0a0 tx=0x7f6d6c00b3b0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.551+0000 7f6d7950f700 1 -- 172.21.15.82:0/311857514 shutdown_connections 2023-09-09T23:14:20.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.551+0000 7f6d7950f700 1 --2- 172.21.15.82:0/311857514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d7410ab40 0x7f6d7410cfe0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.551+0000 7f6d7950f700 1 --2- 172.21.15.82:0/311857514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d74101770 0x7f6d74101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.551+0000 7f6d7950f700 1 --2- 172.21.15.82:0/311857514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d74102090 0x7f6d7410a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.551+0000 7f6d7950f700 1 -- 172.21.15.82:0/311857514 >> 172.21.15.82:0/311857514 conn(0x7f6d740faf50 msgr2=0x7f6d740fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:20.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.551+0000 7f6d7950f700 1 -- 172.21.15.82:0/311857514 shutdown_connections 2023-09-09T23:14:20.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.551+0000 7f6d7950f700 1 -- 172.21.15.82:0/311857514 wait complete. 2023-09-09T23:14:20.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.551+0000 7f6d7950f700 1 Processor -- start 2023-09-09T23:14:20.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.552+0000 7f6d7950f700 1 -- start start 2023-09-09T23:14:20.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.552+0000 7f6d7950f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d74101770 0x7f6d7419c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.552+0000 7f6d7950f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d74102090 0x7f6d7419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.552+0000 7f6d7950f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d7410ab40 0x7f6d741965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.552+0000 7f6d7950f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d74196c10 con 0x7f6d74102090 2023-09-09T23:14:20.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.552+0000 7f6d7950f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d74196d80 con 0x7f6d7410ab40 2023-09-09T23:14:20.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.552+0000 7f6d7950f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d74196f40 con 0x7f6d74101770 2023-09-09T23:14:20.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.552+0000 7f6d727fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d74102090 0x7f6d7419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.552+0000 7f6d727fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d74102090 0x7f6d7419ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57216/0 (socket says 172.21.15.82:57216) 2023-09-09T23:14:20.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.552+0000 7f6d727fc700 1 -- 172.21.15.82:0/1169227942 learned_addr learned my addr 172.21.15.82:0/1169227942 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:20.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.552+0000 7f6d72ffd700 1 --2- 172.21.15.82:0/1169227942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d74101770 0x7f6d7419c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.552+0000 7f6d737fe700 1 --2- 172.21.15.82:0/1169227942 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d7410ab40 0x7f6d741965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.552+0000 7f6d727fc700 1 -- 172.21.15.82:0/1169227942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d74101770 msgr2=0x7f6d7419c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.552+0000 7f6d727fc700 1 --2- 172.21.15.82:0/1169227942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d74101770 0x7f6d7419c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.552+0000 7f6d727fc700 1 -- 172.21.15.82:0/1169227942 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d7410ab40 msgr2=0x7f6d741965a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.552+0000 7f6d727fc700 1 --2- 172.21.15.82:0/1169227942 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d7410ab40 0x7f6d741965a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.552+0000 7f6d727fc700 1 -- 172.21.15.82:0/1169227942 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d6c009d00 con 0x7f6d74102090 2023-09-09T23:14:20.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.552+0000 7f6d737fe700 1 --2- 172.21.15.82:0/1169227942 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d7410ab40 0x7f6d741965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:14:20.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.553+0000 7f6d727fc700 1 --2- 172.21.15.82:0/1169227942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d74102090 0x7f6d7419ca70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6d6800d980 tx=0x7f6d6800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:20.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.553+0000 7f6d5bfff700 1 -- 172.21.15.82:0/1169227942 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6800bb60 con 0x7f6d74102090 2023-09-09T23:14:20.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.553+0000 7f6d5bfff700 1 -- 172.21.15.82:0/1169227942 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d68007640 con 0x7f6d74102090 2023-09-09T23:14:20.559 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.553+0000 7f6d7950f700 1 -- 172.21.15.82:0/1169227942 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d74197220 con 0x7f6d74102090 2023-09-09T23:14:20.559 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.553+0000 7f6d5bfff700 1 -- 172.21.15.82:0/1169227942 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d68004e90 con 0x7f6d74102090 2023-09-09T23:14:20.559 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.553+0000 7f6d7950f700 1 -- 172.21.15.82:0/1169227942 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d741a3520 con 0x7f6d74102090 2023-09-09T23:14:20.559 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.555+0000 7f6d5bfff700 1 -- 172.21.15.82:0/1169227942 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135492+0+0 (secure 0 0 0) 0x7f6d6800bcc0 con 0x7f6d74102090 2023-09-09T23:14:20.559 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.555+0000 7f6d5bfff700 1 --2- 172.21.15.82:0/1169227942 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f6d5c0a08d0 0x7f6d5c0a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.559 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.555+0000 7f6d5bfff700 1 -- 172.21.15.82:0/1169227942 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f6d680cc000 con 0x7f6d74102090 2023-09-09T23:14:20.560 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.556+0000 7f6d72ffd700 1 --2- 172.21.15.82:0/1169227942 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f6d5c0a08d0 0x7f6d5c0a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.560 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.556+0000 7f6d72ffd700 1 --2- 172.21.15.82:0/1169227942 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f6d5c0a08d0 0x7f6d5c0a2d90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6d64009bf0 tx=0x7f6d64009260 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:20.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.558+0000 7f6d856d5b80 1 Processor -- start 2023-09-09T23:14:20.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.558+0000 7f6d856d5b80 1 -- start start 2023-09-09T23:14:20.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.558+0000 7f6d856d5b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615d4000710 0x5615d4000af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.558+0000 7f6d856d5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615d4001030 0x5615d400a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.558+0000 7f6d856d5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615d400ae40 0x5615d400d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.558+0000 7f6d856d5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615d3f59730 con 0x5615d4001030 2023-09-09T23:14:20.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.558+0000 7f6d856d5b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615d3f57050 con 0x5615d4000710 2023-09-09T23:14:20.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.558+0000 7f6d856d5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615d3f52940 con 0x5615d400ae40 2023-09-09T23:14:20.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.558+0000 7f6d727fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615d4001030 0x5615d400a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.558+0000 7f6d727fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615d4001030 0x5615d400a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57224/0 (socket says 172.21.15.82:57224) 2023-09-09T23:14:20.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.558+0000 7f6d727fc700 1 -- 172.21.15.82:0/1132707302 learned_addr learned my addr 172.21.15.82:0/1132707302 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:20.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.558+0000 7f6d72ffd700 1 --2- 172.21.15.82:0/1132707302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615d4000710 0x5615d4000af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.558+0000 7f6d737fe700 1 --2- 172.21.15.82:0/1132707302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615d400ae40 0x5615d400d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.558+0000 7f6d727fc700 1 -- 172.21.15.82:0/1132707302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615d400ae40 msgr2=0x5615d400d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:20.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.558+0000 7f6d727fc700 1 --2- 172.21.15.82:0/1132707302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615d400ae40 0x5615d400d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.558+0000 7f6d727fc700 1 -- 172.21.15.82:0/1132707302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615d4000710 msgr2=0x5615d4000af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:20.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.558+0000 7f6d727fc700 1 --2- 172.21.15.82:0/1132707302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615d4000710 0x5615d4000af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.558+0000 7f6d727fc700 1 -- 172.21.15.82:0/1132707302 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615d3d7ede0 con 0x5615d4001030 2023-09-09T23:14:20.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.559+0000 7f6d727fc700 1 --2- 172.21.15.82:0/1132707302 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615d4001030 0x5615d400a900 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6d68098fd0 tx=0x7f6d680047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b0c4be45c8ce04 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:20.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.559+0000 7f6d597fa700 1 -- 172.21.15.82:0/1132707302 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d680999d0 con 0x5615d4001030 2023-09-09T23:14:20.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.559+0000 7f6d597fa700 1 -- 172.21.15.82:0/1132707302 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d680992d0 con 0x5615d4001030 2023-09-09T23:14:20.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.559+0000 7f6d597fa700 1 -- 172.21.15.82:0/1132707302 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d680a1110 con 0x5615d4001030 2023-09-09T23:14:20.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.560+0000 7f6d856d5b80 1 -- 172.21.15.82:0/1132707302 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615d4001030 msgr2=0x5615d400a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.560+0000 7f6d856d5b80 1 --2- 172.21.15.82:0/1132707302 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615d4001030 0x5615d400a900 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6d68098fd0 tx=0x7f6d680047d0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.560+0000 7f6d856d5b80 1 -- 172.21.15.82:0/1132707302 shutdown_connections 2023-09-09T23:14:20.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.560+0000 7f6d856d5b80 1 --2- 172.21.15.82:0/1132707302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615d4000710 0x5615d4000af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.560+0000 7f6d856d5b80 1 --2- 172.21.15.82:0/1132707302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615d400ae40 0x5615d400d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.560+0000 7f6d856d5b80 1 --2- 172.21.15.82:0/1132707302 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615d4001030 0x5615d400a900 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.560+0000 7f6d856d5b80 1 -- 172.21.15.82:0/1132707302 >> 172.21.15.82:0/1132707302 conn(0x5615d3f75090 msgr2=0x5615d3ffbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:20.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.560+0000 7f6d856d5b80 1 -- 172.21.15.82:0/1132707302 shutdown_connections 2023-09-09T23:14:20.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.560+0000 7f6d856d5b80 1 -- 172.21.15.82:0/1132707302 wait complete. 2023-09-09T23:14:20.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.560+0000 7f6d856d5b80 1 -- start start 2023-09-09T23:14:20.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.560+0000 7f6d856d5b80 1 Processor -- start 2023-09-09T23:14:20.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.561+0000 7f6d856d5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615d4000710 0x5615d4007b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.561+0000 7f6d856d5b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615d4001030 0x5615d4008060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.561+0000 7f6d856d5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615d400ae40 0x5615d40085a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.561+0000 7f6d856d5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615d3ef0b70 con 0x5615d4000710 2023-09-09T23:14:20.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.561+0000 7f6d856d5b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615d3ef0870 con 0x5615d4001030 2023-09-09T23:14:20.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.561+0000 7f6d856d5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615d3dd4920 con 0x5615d400ae40 2023-09-09T23:14:20.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.561+0000 7f6d856d5b80 10 client.0 _ll_register_callbacks cb 0x5615d3e3aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:14:20.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.561+0000 7f6d72ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615d4000710 0x5615d4007b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.561+0000 7f6d72ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615d4000710 0x5615d4007b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57228/0 (socket says 172.21.15.82:57228) 2023-09-09T23:14:20.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.561+0000 7f6d72ffd700 1 -- 172.21.15.82:0/267944724 learned_addr learned my addr 172.21.15.82:0/267944724 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:20.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.561+0000 7f6d737fe700 1 --2- 172.21.15.82:0/267944724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615d400ae40 0x5615d40085a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.561+0000 7f6d727fc700 1 --2- 172.21.15.82:0/267944724 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615d4001030 0x5615d4008060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.561+0000 7f6d72ffd700 1 -- 172.21.15.82:0/267944724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615d400ae40 msgr2=0x5615d40085a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:20.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.561+0000 7f6d72ffd700 1 --2- 172.21.15.82:0/267944724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615d400ae40 0x5615d40085a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.561+0000 7f6d72ffd700 1 -- 172.21.15.82:0/267944724 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615d4001030 msgr2=0x5615d4008060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:20.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.561+0000 7f6d72ffd700 1 --2- 172.21.15.82:0/267944724 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615d4001030 0x5615d4008060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.561+0000 7f6d72ffd700 1 -- 172.21.15.82:0/267944724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d741a3520 con 0x5615d4000710 2023-09-09T23:14:20.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.561+0000 7f6d72ffd700 1 --2- 172.21.15.82:0/267944724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615d4000710 0x5615d4007b20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6d64012870 tx=0x7f6d640047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e51a7f92c92bfa server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:20.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.561+0000 7f6d56ff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:14:20.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.562+0000 7f6d56ff5700 1 -- 172.21.15.82:0/267944724 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64004220 con 0x5615d4000710 2023-09-09T23:14:20.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.562+0000 7f6d56ff5700 1 -- 172.21.15.82:0/267944724 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f6d64018070 con 0x5615d4000710 2023-09-09T23:14:20.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.562+0000 7f6d856d5b80 1 -- 172.21.15.82:0/267944724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615d3eaa4f0 con 0x5615d4000710 2023-09-09T23:14:20.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.562+0000 7f6d56ff5700 1 -- 172.21.15.82:0/267944724 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d64004380 con 0x5615d4000710 2023-09-09T23:14:20.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.562+0000 7f6d856d5b80 10 client.15840 fetch_fsmap learned FSMap version 28 2023-09-09T23:14:20.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.562+0000 7f6d856d5b80 1 -- 172.21.15.82:0/267944724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5615d3d67c60 con 0x5615d4000710 2023-09-09T23:14:20.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.562+0000 7f6d56ff5700 1 -- 172.21.15.82:0/267944724 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5615d3eaa4f0 con 0x5615d4000710 2023-09-09T23:14:20.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.562+0000 7f6d56ff5700 1 -- 172.21.15.82:0/267944724 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f6d640208c0 con 0x5615d4000710 2023-09-09T23:14:20.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.562+0000 7f6d856d5b80 10 client.15840 fetch_fsmap finished waiting for FSMap version 28 2023-09-09T23:14:20.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.562+0000 7f6d856d5b80 10 client.15840 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2023-09-09T23:14:20.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.562+0000 7f6d856d5b80 20 client.15840 populate_metadata read hostname 'smithi082' 2023-09-09T23:14:20.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.562+0000 7f6d856d5b80 1 --2- 172.21.15.82:0/267944724 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x5615d401a930 0x5615d401cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.562+0000 7f6d856d5b80 4 client.15840 mds_command: new command op to 24460 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:14:20.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.562+0000 7f6d856d5b80 1 -- 172.21.15.82:0/267944724 --> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615d3f678a0 con 0x5615d401a930 2023-09-09T23:14:20.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.562+0000 7f6d727fc700 1 --2- 172.21.15.82:0/267944724 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x5615d401a930 0x5615d401cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.563+0000 7f6d56ff5700 10 client.15840 ms_handle_connect on v2:172.21.15.170:6832/1931961940 2023-09-09T23:14:20.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.563+0000 7f6d727fc700 1 --2- 172.21.15.82:0/267944724 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x5615d401a930 0x5615d401cd50 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=676697c56353a1e6 server_cookie=e331aaad96b5d9d5 in_seq=0 out_seq=0 2023-09-09T23:14:20.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.563+0000 7f6d56ff5700 1 -- 172.21.15.82:0/267944724 <== mds.4 v2:172.21.15.170:6832/1931961940 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5615d3f678a0 con 0x5615d401a930 2023-09-09T23:14:20.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.563+0000 7f6d56ff5700 10 client.15840 handle_command_reply: tid=0 2023-09-09T23:14:20.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.563+0000 7f6d856d5b80 1 client.15840 shutdown 2023-09-09T23:14:20.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.564+0000 7f6d856d5b80 1 -- 172.21.15.82:0/267944724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615d4000710 msgr2=0x5615d4007b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.564+0000 7f6d856d5b80 1 --2- 172.21.15.82:0/267944724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615d4000710 0x5615d4007b20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6d64012870 tx=0x7f6d640047c0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.564+0000 7f6d856d5b80 1 -- 172.21.15.82:0/267944724 shutdown_connections 2023-09-09T23:14:20.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.564+0000 7f6d856d5b80 1 --2- 172.21.15.82:0/267944724 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x5615d401a930 0x5615d401cd50 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.564+0000 7f6d856d5b80 1 --2- 172.21.15.82:0/267944724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615d400ae40 0x5615d40085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.564+0000 7f6d56ff5700 0 client.15840 ms_handle_reset on v2:172.21.15.170:6832/1931961940 2023-09-09T23:14:20.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.564+0000 7f6d856d5b80 1 --2- 172.21.15.82:0/267944724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615d4000710 0x5615d4007b20 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.564+0000 7f6d856d5b80 1 --2- 172.21.15.82:0/267944724 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615d4001030 0x5615d4008060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.564+0000 7f6d856d5b80 1 -- 172.21.15.82:0/267944724 >> 172.21.15.82:0/267944724 conn(0x5615d3f75090 msgr2=0x5615d3ffb780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:20.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.564+0000 7f6d856d5b80 1 -- 172.21.15.82:0/267944724 shutdown_connections 2023-09-09T23:14:20.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.564+0000 7f6d856d5b80 1 -- 172.21.15.82:0/267944724 wait complete. 2023-09-09T23:14:20.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.564+0000 7f6d856d5b80 20 client.15840 trim_cache size 0 max 16384 2023-09-09T23:14:20.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.577+0000 7f6d856d5b80 1 Processor -- start 2023-09-09T23:14:20.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.577+0000 7f6d856d5b80 1 -- start start 2023-09-09T23:14:20.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.577+0000 7f6d856d5b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615d4000710 0x5615d40083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.577+0000 7f6d856d5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615d4001030 0x5615d3ffb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.577+0000 7f6d856d5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615d400ae40 0x5615d3ffbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.577+0000 7f6d856d5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615d3ffc1b0 con 0x5615d400ae40 2023-09-09T23:14:20.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.577+0000 7f6d856d5b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615d4000e80 con 0x5615d4000710 2023-09-09T23:14:20.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.577+0000 7f6d856d5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615d3ff3430 con 0x5615d4001030 2023-09-09T23:14:20.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.577+0000 7f6d737fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615d400ae40 0x5615d3ffbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.577+0000 7f6d737fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615d400ae40 0x5615d3ffbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57240/0 (socket says 172.21.15.82:57240) 2023-09-09T23:14:20.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.577+0000 7f6d737fe700 1 -- 172.21.15.82:0/2093468679 learned_addr learned my addr 172.21.15.82:0/2093468679 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:20.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.577+0000 7f6d72ffd700 1 --2- 172.21.15.82:0/2093468679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615d4000710 0x5615d40083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.577+0000 7f6d727fc700 1 --2- 172.21.15.82:0/2093468679 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615d4001030 0x5615d3ffb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.577+0000 7f6d737fe700 1 -- 172.21.15.82:0/2093468679 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615d4001030 msgr2=0x5615d3ffb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:20.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.577+0000 7f6d737fe700 1 --2- 172.21.15.82:0/2093468679 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615d4001030 0x5615d3ffb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.577+0000 7f6d737fe700 1 -- 172.21.15.82:0/2093468679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615d4000710 msgr2=0x5615d40083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:20.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.577+0000 7f6d737fe700 1 --2- 172.21.15.82:0/2093468679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615d4000710 0x5615d40083d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.577+0000 7f6d737fe700 1 -- 172.21.15.82:0/2093468679 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615d3da5250 con 0x5615d400ae40 2023-09-09T23:14:20.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.577+0000 7f6d72ffd700 1 --2- 172.21.15.82:0/2093468679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615d4000710 0x5615d40083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:14:20.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.578+0000 7f6d737fe700 1 --2- 172.21.15.82:0/2093468679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615d400ae40 0x5615d3ffbc30 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c015000 tx=0x7f6d6c009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ac5de19fe13919 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:20.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.578+0000 7f6d59ffb700 1 -- 172.21.15.82:0/2093468679 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c00e040 con 0x5615d400ae40 2023-09-09T23:14:20.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.578+0000 7f6d59ffb700 1 -- 172.21.15.82:0/2093468679 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d6c00bcb0 con 0x5615d400ae40 2023-09-09T23:14:20.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.578+0000 7f6d59ffb700 1 -- 172.21.15.82:0/2093468679 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c0052a0 con 0x5615d400ae40 2023-09-09T23:14:20.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.579+0000 7f6d856d5b80 1 -- 172.21.15.82:0/2093468679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615d400ae40 msgr2=0x5615d3ffbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.579+0000 7f6d856d5b80 1 --2- 172.21.15.82:0/2093468679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615d400ae40 0x5615d3ffbc30 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c015000 tx=0x7f6d6c009390 comp rx=0 tx=0).stop 2023-09-09T23:14:20.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.579+0000 7f6d856d5b80 1 -- 172.21.15.82:0/2093468679 shutdown_connections 2023-09-09T23:14:20.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.579+0000 7f6d856d5b80 1 --2- 172.21.15.82:0/2093468679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615d4000710 0x5615d40083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.579+0000 7f6d856d5b80 1 --2- 172.21.15.82:0/2093468679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615d400ae40 0x5615d3ffbc30 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.579+0000 7f6d856d5b80 1 --2- 172.21.15.82:0/2093468679 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615d4001030 0x5615d3ffb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.579+0000 7f6d856d5b80 1 -- 172.21.15.82:0/2093468679 >> 172.21.15.82:0/2093468679 conn(0x5615d3f75090 msgr2=0x5615d4002e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:20.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.579+0000 7f6d856d5b80 1 -- 172.21.15.82:0/2093468679 shutdown_connections 2023-09-09T23:14:20.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.579+0000 7f6d856d5b80 1 -- 172.21.15.82:0/2093468679 wait complete. 2023-09-09T23:14:20.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.579+0000 7f6d856d5b80 1 -- start start 2023-09-09T23:14:20.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.579+0000 7f6d856d5b80 1 Processor -- start 2023-09-09T23:14:20.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.580+0000 7f6d856d5b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615d4000710 0x5615d4011680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.580+0000 7f6d856d5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615d4001030 0x5615d400f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.580+0000 7f6d856d5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615d400ae40 0x5615d400fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.580+0000 7f6d856d5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615d4010380 con 0x5615d4001030 2023-09-09T23:14:20.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.580+0000 7f6d856d5b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615d40104f0 con 0x5615d4000710 2023-09-09T23:14:20.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.580+0000 7f6d856d5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615d40106b0 con 0x5615d400ae40 2023-09-09T23:14:20.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.580+0000 7f6d856d5b80 10 client.0 _ll_register_callbacks cb 0x7f6d64012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:14:20.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.580+0000 7f6d727fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615d4001030 0x5615d400f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.580+0000 7f6d727fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615d4001030 0x5615d400f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57244/0 (socket says 172.21.15.82:57244) 2023-09-09T23:14:20.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.580+0000 7f6d727fc700 1 -- 172.21.15.82:0/932415131 learned_addr learned my addr 172.21.15.82:0/932415131 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:20.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.580+0000 7f6d737fe700 1 --2- 172.21.15.82:0/932415131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615d400ae40 0x5615d400fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.580+0000 7f6d72ffd700 1 --2- 172.21.15.82:0/932415131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615d4000710 0x5615d4011680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.580+0000 7f6d727fc700 1 -- 172.21.15.82:0/932415131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615d400ae40 msgr2=0x5615d400fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:20.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.580+0000 7f6d727fc700 1 --2- 172.21.15.82:0/932415131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615d400ae40 0x5615d400fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.580+0000 7f6d727fc700 1 -- 172.21.15.82:0/932415131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615d4000710 msgr2=0x5615d4011680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:20.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.580+0000 7f6d727fc700 1 --2- 172.21.15.82:0/932415131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615d4000710 0x5615d4011680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.580+0000 7f6d727fc700 1 -- 172.21.15.82:0/932415131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5615d3d67c60 con 0x5615d4001030 2023-09-09T23:14:20.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.580+0000 7f6d727fc700 1 --2- 172.21.15.82:0/932415131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615d4001030 0x5615d400f900 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6d680c9dc0 tx=0x7f6d680840c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e065c12eef7892 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:20.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.580+0000 7f6d56ff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:14:20.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.580+0000 7f6d56ff5700 1 -- 172.21.15.82:0/932415131 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d68014040 con 0x5615d4001030 2023-09-09T23:14:20.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.580+0000 7f6d56ff5700 1 -- 172.21.15.82:0/932415131 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f6d680a1340 con 0x5615d4001030 2023-09-09T23:14:20.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.581+0000 7f6d856d5b80 1 -- 172.21.15.82:0/932415131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d680992d0 con 0x5615d4001030 2023-09-09T23:14:20.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.581+0000 7f6d56ff5700 1 -- 172.21.15.82:0/932415131 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d680999d0 con 0x5615d4001030 2023-09-09T23:14:20.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.581+0000 7f6d856d5b80 10 client.15846 fetch_fsmap learned FSMap version 28 2023-09-09T23:14:20.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.581+0000 7f6d856d5b80 1 -- 172.21.15.82:0/932415131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f6d7410dca0 con 0x5615d4001030 2023-09-09T23:14:20.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.581+0000 7f6d56ff5700 1 -- 172.21.15.82:0/932415131 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d680992d0 con 0x5615d4001030 2023-09-09T23:14:20.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.581+0000 7f6d56ff5700 1 -- 172.21.15.82:0/932415131 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f6d680a0eb0 con 0x5615d4001030 2023-09-09T23:14:20.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.581+0000 7f6d856d5b80 10 client.15846 fetch_fsmap finished waiting for FSMap version 28 2023-09-09T23:14:20.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.581+0000 7f6d856d5b80 10 client.15846 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2023-09-09T23:14:20.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.581+0000 7f6d856d5b80 20 client.15846 populate_metadata read hostname 'smithi082' 2023-09-09T23:14:20.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.581+0000 7f6d856d5b80 1 --2- 172.21.15.82:0/932415131 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x5615d406f7b0 0x5615d4071bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.581+0000 7f6d856d5b80 4 client.15846 mds_command: new command op to 24460 tid=0[{"prefix": "damage ls"}] 2023-09-09T23:14:20.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.581+0000 7f6d856d5b80 1 -- 172.21.15.82:0/932415131 --> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5615d3f4b920 con 0x5615d406f7b0 2023-09-09T23:14:20.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.581+0000 7f6d72ffd700 1 --2- 172.21.15.82:0/932415131 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x5615d406f7b0 0x5615d4071bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.582+0000 7f6d56ff5700 10 client.15846 ms_handle_connect on v2:172.21.15.170:6832/1931961940 2023-09-09T23:14:20.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.582+0000 7f6d72ffd700 1 --2- 172.21.15.82:0/932415131 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x5615d406f7b0 0x5615d4071bd0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bb4add4a72e3e457 server_cookie=856a398bec299add in_seq=0 out_seq=0 2023-09-09T23:14:20.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.582+0000 7f6d56ff5700 1 -- 172.21.15.82:0/932415131 <== mds.4 v2:172.21.15.170:6832/1931961940 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5615d3f4b920 con 0x5615d406f7b0 2023-09-09T23:14:20.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.582+0000 7f6d56ff5700 10 client.15846 handle_command_reply: tid=0 2023-09-09T23:14:20.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.582+0000 7f6d856d5b80 1 client.15846 shutdown 2023-09-09T23:14:20.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.582+0000 7f6d856d5b80 1 -- 172.21.15.82:0/932415131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615d4001030 msgr2=0x5615d400f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.582+0000 7f6d856d5b80 1 --2- 172.21.15.82:0/932415131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615d4001030 0x5615d400f900 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6d680c9dc0 tx=0x7f6d680840c0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.582+0000 7f6d856d5b80 1 -- 172.21.15.82:0/932415131 shutdown_connections 2023-09-09T23:14:20.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.582+0000 7f6d856d5b80 1 --2- 172.21.15.82:0/932415131 >> [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] conn(0x5615d406f7b0 0x5615d4071bd0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.582+0000 7f6d856d5b80 1 --2- 172.21.15.82:0/932415131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615d4000710 0x5615d4011680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.582+0000 7f6d856d5b80 1 --2- 172.21.15.82:0/932415131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615d400ae40 0x5615d400fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.582+0000 7f6d856d5b80 1 --2- 172.21.15.82:0/932415131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615d4001030 0x5615d400f900 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.582+0000 7f6d56ff5700 0 client.15846 ms_handle_reset on v2:172.21.15.170:6832/1931961940 2023-09-09T23:14:20.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.582+0000 7f6d856d5b80 1 -- 172.21.15.82:0/932415131 >> 172.21.15.82:0/932415131 conn(0x5615d3f75090 msgr2=0x5615d4002e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:20.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.582+0000 7f6d856d5b80 1 -- 172.21.15.82:0/932415131 shutdown_connections 2023-09-09T23:14:20.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.582+0000 7f6d856d5b80 1 -- 172.21.15.82:0/932415131 wait complete. 2023-09-09T23:14:20.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.582+0000 7f6d856d5b80 20 client.15846 trim_cache size 0 max 16384 2023-09-09T23:14:20.602 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2023-09-09T23:14:20.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.583+0000 7f6d7950f700 1 -- 172.21.15.82:0/1169227942 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f6d5c0a08d0 msgr2=0x7f6d5c0a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.583+0000 7f6d7950f700 1 --2- 172.21.15.82:0/1169227942 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f6d5c0a08d0 0x7f6d5c0a2d90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6d64009bf0 tx=0x7f6d64009260 comp rx=0 tx=0).stop 2023-09-09T23:14:20.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.583+0000 7f6d7950f700 1 -- 172.21.15.82:0/1169227942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d74102090 msgr2=0x7f6d7419ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.583+0000 7f6d7950f700 1 --2- 172.21.15.82:0/1169227942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d74102090 0x7f6d7419ca70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6d6800d980 tx=0x7f6d6800dd40 comp rx=0 tx=0).stop 2023-09-09T23:14:20.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.583+0000 7f6d7950f700 1 -- 172.21.15.82:0/1169227942 shutdown_connections 2023-09-09T23:14:20.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.583+0000 7f6d7950f700 1 --2- 172.21.15.82:0/1169227942 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f6d5c0a08d0 0x7f6d5c0a2d90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.583+0000 7f6d7950f700 1 --2- 172.21.15.82:0/1169227942 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d7410ab40 0x7f6d741965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.583+0000 7f6d7950f700 1 --2- 172.21.15.82:0/1169227942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d74101770 0x7f6d7419c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.583+0000 7f6d7950f700 1 --2- 172.21.15.82:0/1169227942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d74102090 0x7f6d7419ca70 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.583+0000 7f6d7950f700 1 -- 172.21.15.82:0/1169227942 >> 172.21.15.82:0/1169227942 conn(0x7f6d740faf50 msgr2=0x7f6d74102510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:20.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.583+0000 7f6d7950f700 1 -- 172.21.15.82:0/1169227942 shutdown_connections 2023-09-09T23:14:20.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.583+0000 7f6d7950f700 1 -- 172.21.15.82:0/1169227942 wait complete. 2023-09-09T23:14:20.606 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-09-09T23:14:20.613 INFO:teuthology.orchestra.run.smithi082.stderr:4600000 of 50000000 tuples (9%) done (elapsed 7.20 s, remaining 71.03 s) 2023-09-09T23:14:20.746 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.747+0000 7f8259a68700 1 -- 172.21.15.82:0/3690969831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82540ffe00 msgr2=0x7f825410c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.746 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.747+0000 7f8259a68700 1 --2- 172.21.15.82:0/3690969831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82540ffe00 0x7f825410c310 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f824c00b0a0 tx=0x7f824c00b3b0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.747 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.747+0000 7f8259a68700 1 -- 172.21.15.82:0/3690969831 shutdown_connections 2023-09-09T23:14:20.747 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.747+0000 7f8259a68700 1 --2- 172.21.15.82:0/3690969831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82540ffe00 0x7f825410c310 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.747 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.747+0000 7f8259a68700 1 --2- 172.21.15.82:0/3690969831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82541054e0 0x7f82541058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.747 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.747+0000 7f8259a68700 1 --2- 172.21.15.82:0/3690969831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82540ff440 0x7f82540ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.748 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.747+0000 7f8259a68700 1 -- 172.21.15.82:0/3690969831 >> 172.21.15.82:0/3690969831 conn(0x7f82540faf50 msgr2=0x7f82540fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:20.748 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.747+0000 7f8259a68700 1 -- 172.21.15.82:0/3690969831 shutdown_connections 2023-09-09T23:14:20.748 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.747+0000 7f8259a68700 1 -- 172.21.15.82:0/3690969831 wait complete. 2023-09-09T23:14:20.748 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.748+0000 7f8259a68700 1 Processor -- start 2023-09-09T23:14:20.748 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.748+0000 7f8259a68700 1 -- start start 2023-09-09T23:14:20.752 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.748+0000 7f8259a68700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82540ff440 0x7f82541954d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.752 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.749+0000 7f8259a68700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82540ffe00 0x7f8254195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.749+0000 7f8259a68700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82541054e0 0x7f8254199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.749+0000 7f8259a68700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825419a4b0 con 0x7f82541054e0 2023-09-09T23:14:20.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.749+0000 7f8259a68700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825419a620 con 0x7f82540ff440 2023-09-09T23:14:20.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.749+0000 7f8259a68700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825419a7e0 con 0x7f82540ffe00 2023-09-09T23:14:20.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.749+0000 7f82537fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82540ff440 0x7f82541954d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.749+0000 7f8253fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82541054e0 0x7f8254199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.749+0000 7f82537fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82540ff440 0x7f82541954d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:42820/0 (socket says 172.21.15.82:42820) 2023-09-09T23:14:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.749+0000 7f82537fe700 1 -- 172.21.15.82:0/2213674482 learned_addr learned my addr 172.21.15.82:0/2213674482 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.749+0000 7f824affd700 1 --2- 172.21.15.82:0/2213674482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82540ffe00 0x7f8254195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.749+0000 7f82537fe700 1 -- 172.21.15.82:0/2213674482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82540ffe00 msgr2=0x7f8254195a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.749+0000 7f82537fe700 1 --2- 172.21.15.82:0/2213674482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82540ffe00 0x7f8254195a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.749+0000 7f82537fe700 1 -- 172.21.15.82:0/2213674482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82541054e0 msgr2=0x7f8254199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.749+0000 7f82537fe700 1 --2- 172.21.15.82:0/2213674482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82541054e0 0x7f8254199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.749+0000 7f82537fe700 1 -- 172.21.15.82:0/2213674482 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f824c009d00 con 0x7f82540ff440 2023-09-09T23:14:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.749+0000 7f8253fff700 1 --2- 172.21.15.82:0/2213674482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82541054e0 0x7f8254199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:14:20.756 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.749+0000 7f82537fe700 1 --2- 172.21.15.82:0/2213674482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82540ff440 0x7f82541954d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f824400e9a0 tx=0x7f824400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:20.756 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.749+0000 7f82517fa700 1 -- 172.21.15.82:0/2213674482 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8244009790 con 0x7f82540ff440 2023-09-09T23:14:20.756 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.749+0000 7f8259a68700 1 -- 172.21.15.82:0/2213674482 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f825419aac0 con 0x7f82540ff440 2023-09-09T23:14:20.756 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.749+0000 7f82517fa700 1 -- 172.21.15.82:0/2213674482 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8244010430 con 0x7f82540ff440 2023-09-09T23:14:20.756 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.749+0000 7f82517fa700 1 -- 172.21.15.82:0/2213674482 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8244018680 con 0x7f82540ff440 2023-09-09T23:14:20.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.749+0000 7f8259a68700 1 -- 172.21.15.82:0/2213674482 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82541a2300 con 0x7f82540ff440 2023-09-09T23:14:20.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.752+0000 7f82517fa700 1 -- 172.21.15.82:0/2213674482 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) v1 ==== 135492+0+0 (secure 0 0 0) 0x7f82440105a0 con 0x7f82540ff440 2023-09-09T23:14:20.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.752+0000 7f82517fa700 1 --2- 172.21.15.82:0/2213674482 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f82400a08d0 0x7f82400a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.752+0000 7f824affd700 1 --2- 172.21.15.82:0/2213674482 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f82400a08d0 0x7f82400a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.752+0000 7f82517fa700 1 -- 172.21.15.82:0/2213674482 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f8244014070 con 0x7f82540ff440 2023-09-09T23:14:20.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.753+0000 7f824affd700 1 --2- 172.21.15.82:0/2213674482 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f82400a08d0 0x7f82400a2d90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8254196ae0 tx=0x7f823c009260 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:20.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.753+0000 7f8265c2eb80 1 Processor -- start 2023-09-09T23:14:20.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.753+0000 7f8265c2eb80 1 -- start start 2023-09-09T23:14:20.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.753+0000 7f8265c2eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7469cb710 0x55e7469cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.753+0000 7f8265c2eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e7469cc030 0x55e7469d5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.753+0000 7f8265c2eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7469d5e40 0x55e7469d8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.753+0000 7f8265c2eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e746924730 con 0x55e7469cb710 2023-09-09T23:14:20.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.753+0000 7f8265c2eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e746922050 con 0x55e7469cc030 2023-09-09T23:14:20.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.753+0000 7f8265c2eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e74691d940 con 0x55e7469d5e40 2023-09-09T23:14:20.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.753+0000 7f82537fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7469cb710 0x55e7469cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.753+0000 7f82537fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7469cb710 0x55e7469cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57264/0 (socket says 172.21.15.82:57264) 2023-09-09T23:14:20.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.753+0000 7f82537fe700 1 -- 172.21.15.82:0/302368606 learned_addr learned my addr 172.21.15.82:0/302368606 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:20.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.753+0000 7f82537fe700 1 -- 172.21.15.82:0/302368606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7469d5e40 msgr2=0x55e7469d8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:20.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.753+0000 7f82537fe700 1 --2- 172.21.15.82:0/302368606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7469d5e40 0x55e7469d8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.753+0000 7f82537fe700 1 -- 172.21.15.82:0/302368606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e7469cc030 msgr2=0x55e7469d5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:20.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.753+0000 7f82537fe700 1 --2- 172.21.15.82:0/302368606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e7469cc030 0x55e7469d5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.753+0000 7f82537fe700 1 -- 172.21.15.82:0/302368606 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e746749de0 con 0x55e7469cb710 2023-09-09T23:14:20.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.754+0000 7f82537fe700 1 --2- 172.21.15.82:0/302368606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7469cb710 0x55e7469cbaf0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f82440992d0 tx=0x7f8244098fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d483870dbc249e86 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:20.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.754+0000 7f8249ffb700 1 -- 172.21.15.82:0/302368606 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824409d400 con 0x55e7469cb710 2023-09-09T23:14:20.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.754+0000 7f8249ffb700 1 -- 172.21.15.82:0/302368606 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82440a1640 con 0x55e7469cb710 2023-09-09T23:14:20.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.754+0000 7f8249ffb700 1 -- 172.21.15.82:0/302368606 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8244098ac0 con 0x55e7469cb710 2023-09-09T23:14:20.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.754+0000 7f8265c2eb80 1 -- 172.21.15.82:0/302368606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7469cb710 msgr2=0x55e7469cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.754+0000 7f8265c2eb80 1 --2- 172.21.15.82:0/302368606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7469cb710 0x55e7469cbaf0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f82440992d0 tx=0x7f8244098fb0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.754+0000 7f8265c2eb80 1 -- 172.21.15.82:0/302368606 shutdown_connections 2023-09-09T23:14:20.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.754+0000 7f8265c2eb80 1 --2- 172.21.15.82:0/302368606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7469d5e40 0x55e7469d8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.754+0000 7f8265c2eb80 1 --2- 172.21.15.82:0/302368606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7469cb710 0x55e7469cbaf0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.754+0000 7f8265c2eb80 1 --2- 172.21.15.82:0/302368606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e7469cc030 0x55e7469d5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.754+0000 7f8265c2eb80 1 -- 172.21.15.82:0/302368606 >> 172.21.15.82:0/302368606 conn(0x55e746940090 msgr2=0x55e7469c6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:20.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.758+0000 7f8265c2eb80 1 -- 172.21.15.82:0/302368606 shutdown_connections 2023-09-09T23:14:20.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.758+0000 7f8265c2eb80 1 -- 172.21.15.82:0/302368606 wait complete. 2023-09-09T23:14:20.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.758+0000 7f8265c2eb80 1 -- start start 2023-09-09T23:14:20.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.758+0000 7f8265c2eb80 1 Processor -- start 2023-09-09T23:14:20.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.758+0000 7f8265c2eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7469cb710 0x55e7469d2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.758+0000 7f8265c2eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e7469d5e40 0x55e7469d3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.758+0000 7f8265c2eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7469d37b0 0x55e7469e1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.758+0000 7f8265c2eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7468bbb70 con 0x55e7469cb710 2023-09-09T23:14:20.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.758+0000 7f8265c2eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7468bb870 con 0x55e7469d5e40 2023-09-09T23:14:20.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.758+0000 7f8265c2eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e74679f920 con 0x55e7469d37b0 2023-09-09T23:14:20.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.759+0000 7f8265c2eb80 10 client.0 _ll_register_callbacks cb 0x55e746805a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:14:20.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.759+0000 7f82537fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7469cb710 0x55e7469d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.759+0000 7f82537fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7469cb710 0x55e7469d2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57270/0 (socket says 172.21.15.82:57270) 2023-09-09T23:14:20.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.759+0000 7f82537fe700 1 -- 172.21.15.82:0/2089103391 learned_addr learned my addr 172.21.15.82:0/2089103391 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:20.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.759+0000 7f824affd700 1 --2- 172.21.15.82:0/2089103391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e7469d5e40 0x55e7469d3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.759+0000 7f8253fff700 1 --2- 172.21.15.82:0/2089103391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7469d37b0 0x55e7469e1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.759+0000 7f82537fe700 1 -- 172.21.15.82:0/2089103391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7469d37b0 msgr2=0x55e7469e1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:20.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.759+0000 7f82537fe700 1 --2- 172.21.15.82:0/2089103391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7469d37b0 0x55e7469e1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.759+0000 7f82537fe700 1 -- 172.21.15.82:0/2089103391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e7469d5e40 msgr2=0x55e7469d3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:20.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.759+0000 7f82537fe700 1 --2- 172.21.15.82:0/2089103391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e7469d5e40 0x55e7469d3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.759+0000 7f82537fe700 1 -- 172.21.15.82:0/2089103391 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82541a2300 con 0x55e7469cb710 2023-09-09T23:14:20.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.759+0000 7f824affd700 1 --2- 172.21.15.82:0/2089103391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e7469d5e40 0x55e7469d3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:14:20.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.759+0000 7f82537fe700 1 --2- 172.21.15.82:0/2089103391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7469cb710 0x55e7469d2d30 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8244095ad0 tx=0x7f824400c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c1dbc973825e29 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:20.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.759+0000 7f8232ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:14:20.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.759+0000 7f8232ffd700 1 -- 172.21.15.82:0/2089103391 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8244098ac0 con 0x55e7469cb710 2023-09-09T23:14:20.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.759+0000 7f8232ffd700 1 -- 172.21.15.82:0/2089103391 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f8244098c20 con 0x55e7469cb710 2023-09-09T23:14:20.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.759+0000 7f8265c2eb80 1 -- 172.21.15.82:0/2089103391 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e7468754f0 con 0x55e7469cb710 2023-09-09T23:14:20.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.759+0000 7f8232ffd700 1 -- 172.21.15.82:0/2089103391 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82440651a0 con 0x55e7469cb710 2023-09-09T23:14:20.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.759+0000 7f8265c2eb80 10 client.15858 fetch_fsmap learned FSMap version 28 2023-09-09T23:14:20.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.759+0000 7f8265c2eb80 1 -- 172.21.15.82:0/2089103391 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e746732c60 con 0x55e7469cb710 2023-09-09T23:14:20.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.760+0000 7f8232ffd700 1 -- 172.21.15.82:0/2089103391 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e7468754f0 con 0x55e7469cb710 2023-09-09T23:14:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.760+0000 7f8232ffd700 1 -- 172.21.15.82:0/2089103391 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f82440c9c20 con 0x55e7469cb710 2023-09-09T23:14:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.760+0000 7f8265c2eb80 10 client.15858 fetch_fsmap finished waiting for FSMap version 28 2023-09-09T23:14:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.760+0000 7f8265c2eb80 10 client.15858 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2023-09-09T23:14:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.760+0000 7f8265c2eb80 20 client.15858 populate_metadata read hostname 'smithi082' 2023-09-09T23:14:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.760+0000 7f8265c2eb80 1 --2- 172.21.15.82:0/2089103391 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55e7469e5b60 0x55e7469e7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.760+0000 7f8265c2eb80 4 client.15858 mds_command: new command op to 24491 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:14:20.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.760+0000 7f8265c2eb80 1 -- 172.21.15.82:0/2089103391 --> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7469328a0 con 0x55e7469e5b60 2023-09-09T23:14:20.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.760+0000 7f824affd700 1 --2- 172.21.15.82:0/2089103391 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55e7469e5b60 0x55e7469e7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.760+0000 7f8232ffd700 10 client.15858 ms_handle_connect on v2:172.21.15.98:6832/3855146755 2023-09-09T23:14:20.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.760+0000 7f824affd700 1 --2- 172.21.15.82:0/2089103391 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55e7469e5b60 0x55e7469e7f80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7528d03bc05ca5c4 server_cookie=500796d5846232e7 in_seq=0 out_seq=0 2023-09-09T23:14:20.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.761+0000 7f8232ffd700 1 -- 172.21.15.82:0/2089103391 <== mds.3 v2:172.21.15.98:6832/3855146755 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e7469328a0 con 0x55e7469e5b60 2023-09-09T23:14:20.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.761+0000 7f8232ffd700 10 client.15858 handle_command_reply: tid=0 2023-09-09T23:14:20.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.761+0000 7f8265c2eb80 1 client.15858 shutdown 2023-09-09T23:14:20.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.761+0000 7f8265c2eb80 1 -- 172.21.15.82:0/2089103391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7469cb710 msgr2=0x55e7469d2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.761+0000 7f8265c2eb80 1 --2- 172.21.15.82:0/2089103391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7469cb710 0x55e7469d2d30 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8244095ad0 tx=0x7f824400c470 comp rx=0 tx=0).stop 2023-09-09T23:14:20.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.761+0000 7f8265c2eb80 1 -- 172.21.15.82:0/2089103391 shutdown_connections 2023-09-09T23:14:20.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.761+0000 7f8265c2eb80 1 --2- 172.21.15.82:0/2089103391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7469d37b0 0x55e7469e1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.761+0000 7f8265c2eb80 1 --2- 172.21.15.82:0/2089103391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7469cb710 0x55e7469d2d30 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.761+0000 7f8265c2eb80 1 --2- 172.21.15.82:0/2089103391 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55e7469e5b60 0x55e7469e7f80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.761+0000 7f8265c2eb80 1 --2- 172.21.15.82:0/2089103391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e7469d5e40 0x55e7469d3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.761+0000 7f8232ffd700 0 client.15858 ms_handle_reset on v2:172.21.15.98:6832/3855146755 2023-09-09T23:14:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.761+0000 7f8265c2eb80 1 -- 172.21.15.82:0/2089103391 >> 172.21.15.82:0/2089103391 conn(0x55e746940090 msgr2=0x55e7469c67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.761+0000 7f8265c2eb80 1 -- 172.21.15.82:0/2089103391 shutdown_connections 2023-09-09T23:14:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.761+0000 7f8265c2eb80 1 -- 172.21.15.82:0/2089103391 wait complete. 2023-09-09T23:14:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.761+0000 7f8265c2eb80 20 client.15858 trim_cache size 0 max 16384 2023-09-09T23:14:20.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.774+0000 7f8265c2eb80 1 Processor -- start 2023-09-09T23:14:20.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.774+0000 7f8265c2eb80 1 -- start start 2023-09-09T23:14:20.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.774+0000 7f8265c2eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7469cb710 0x55e7469c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.774+0000 7f8265c2eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7469d37b0 0x55e7469c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.774+0000 7f8265c2eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e7469d5e40 0x55e7469e1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.774+0000 7f8265c2eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7469c71a0 con 0x55e7469d37b0 2023-09-09T23:14:20.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.774+0000 7f8265c2eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7469d3d00 con 0x55e7469d5e40 2023-09-09T23:14:20.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.774+0000 7f8265c2eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7469c3420 con 0x55e7469cb710 2023-09-09T23:14:20.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.775+0000 7f824affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7469d37b0 0x55e7469c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.775+0000 7f8253fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e7469d5e40 0x55e7469e1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.775+0000 7f824affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7469d37b0 0x55e7469c6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57280/0 (socket says 172.21.15.82:57280) 2023-09-09T23:14:20.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.775+0000 7f824affd700 1 -- 172.21.15.82:0/594459913 learned_addr learned my addr 172.21.15.82:0/594459913 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:20.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.775+0000 7f82537fe700 1 --2- 172.21.15.82:0/594459913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7469cb710 0x55e7469c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.775+0000 7f824affd700 1 -- 172.21.15.82:0/594459913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7469cb710 msgr2=0x55e7469c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:20.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.775+0000 7f824affd700 1 --2- 172.21.15.82:0/594459913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7469cb710 0x55e7469c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.775+0000 7f824affd700 1 -- 172.21.15.82:0/594459913 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e7469d5e40 msgr2=0x55e7469e1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:20.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.775+0000 7f824affd700 1 --2- 172.21.15.82:0/594459913 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e7469d5e40 0x55e7469e1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.775+0000 7f824affd700 1 -- 172.21.15.82:0/594459913 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e746770250 con 0x55e7469d37b0 2023-09-09T23:14:20.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.775+0000 7f8253fff700 1 --2- 172.21.15.82:0/594459913 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e7469d5e40 0x55e7469e1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:14:20.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.775+0000 7f824affd700 1 --2- 172.21.15.82:0/594459913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7469d37b0 0x55e7469c6c30 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x55e7469cc7b0 tx=0x7f823c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a561d872e034986 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:20.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.775+0000 7f824a7fc700 1 -- 172.21.15.82:0/594459913 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823c025250 con 0x55e7469d37b0 2023-09-09T23:14:20.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.775+0000 7f824a7fc700 1 -- 172.21.15.82:0/594459913 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f823c024040 con 0x55e7469d37b0 2023-09-09T23:14:20.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.775+0000 7f824a7fc700 1 -- 172.21.15.82:0/594459913 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823c0039b0 con 0x55e7469d37b0 2023-09-09T23:14:20.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.776+0000 7f8265c2eb80 1 -- 172.21.15.82:0/594459913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7469d37b0 msgr2=0x55e7469c6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.776+0000 7f8265c2eb80 1 --2- 172.21.15.82:0/594459913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7469d37b0 0x55e7469c6c30 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x55e7469cc7b0 tx=0x7f823c014000 comp rx=0 tx=0).stop 2023-09-09T23:14:20.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.776+0000 7f8265c2eb80 1 -- 172.21.15.82:0/594459913 shutdown_connections 2023-09-09T23:14:20.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.776+0000 7f8265c2eb80 1 --2- 172.21.15.82:0/594459913 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e7469d5e40 0x55e7469e1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.776+0000 7f8265c2eb80 1 --2- 172.21.15.82:0/594459913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7469cb710 0x55e7469c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.776+0000 7f8265c2eb80 1 --2- 172.21.15.82:0/594459913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7469d37b0 0x55e7469c6c30 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.776+0000 7f8265c2eb80 1 -- 172.21.15.82:0/594459913 >> 172.21.15.82:0/594459913 conn(0x55e746940090 msgr2=0x55e7469e22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:20.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.776+0000 7f8265c2eb80 1 -- 172.21.15.82:0/594459913 shutdown_connections 2023-09-09T23:14:20.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.776+0000 7f8265c2eb80 1 -- 172.21.15.82:0/594459913 wait complete. 2023-09-09T23:14:20.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.777+0000 7f8265c2eb80 1 -- start start 2023-09-09T23:14:20.786 INFO:teuthology.orchestra.run.smithi082.stderr:4700000 of 50000000 tuples (9%) done (elapsed 7.36 s, remaining 70.93 s) 2023-09-09T23:14:20.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.777+0000 7f8265c2eb80 1 Processor -- start 2023-09-09T23:14:20.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.777+0000 7f8265c2eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7469cb710 0x55e7469daff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.777+0000 7f8265c2eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7469d37b0 0x55e7469db530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.777+0000 7f8265c2eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e7469d5e40 0x55e7469dba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.777+0000 7f8265c2eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e746a33970 con 0x55e7469cb710 2023-09-09T23:14:20.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.777+0000 7f8265c2eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e746a33ae0 con 0x55e7469d5e40 2023-09-09T23:14:20.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.777+0000 7f8265c2eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e746a33ca0 con 0x55e7469d37b0 2023-09-09T23:14:20.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.777+0000 7f8265c2eb80 10 client.0 _ll_register_callbacks cb 0x7f8244099150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:14:20.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.777+0000 7f82537fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7469cb710 0x55e7469daff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.777+0000 7f82537fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7469cb710 0x55e7469daff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57284/0 (socket says 172.21.15.82:57284) 2023-09-09T23:14:20.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.777+0000 7f82537fe700 1 -- 172.21.15.82:0/3399865573 learned_addr learned my addr 172.21.15.82:0/3399865573 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:20.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.777+0000 7f824affd700 1 --2- 172.21.15.82:0/3399865573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7469d37b0 0x55e7469db530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.777+0000 7f8253fff700 1 --2- 172.21.15.82:0/3399865573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e7469d5e40 0x55e7469dba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.777+0000 7f82537fe700 1 -- 172.21.15.82:0/3399865573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7469d37b0 msgr2=0x55e7469db530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:20.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.777+0000 7f82537fe700 1 --2- 172.21.15.82:0/3399865573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7469d37b0 0x55e7469db530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.778+0000 7f82537fe700 1 -- 172.21.15.82:0/3399865573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e7469d5e40 msgr2=0x55e7469dba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:20.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.778+0000 7f82537fe700 1 --2- 172.21.15.82:0/3399865573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e7469d5e40 0x55e7469dba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.778+0000 7f82537fe700 1 -- 172.21.15.82:0/3399865573 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f825410cc10 con 0x55e7469cb710 2023-09-09T23:14:20.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.778+0000 7f82537fe700 1 --2- 172.21.15.82:0/3399865573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7469cb710 0x55e7469daff0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8244097920 tx=0x7f82440843c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f224aea643fd7a25 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:20.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.778+0000 7f8232ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:14:20.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.778+0000 7f8232ffd700 1 -- 172.21.15.82:0/3399865573 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82440b7460 con 0x55e7469cb710 2023-09-09T23:14:20.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.778+0000 7f8265c2eb80 1 -- 172.21.15.82:0/3399865573 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82440a1640 con 0x55e7469cb710 2023-09-09T23:14:20.794 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.778+0000 7f8232ffd700 1 -- 172.21.15.82:0/3399865573 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f8244098ac0 con 0x55e7469cb710 2023-09-09T23:14:20.794 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.778+0000 7f8232ffd700 1 -- 172.21.15.82:0/3399865573 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82440c9e20 con 0x55e7469cb710 2023-09-09T23:14:20.794 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.778+0000 7f8265c2eb80 10 client.15864 fetch_fsmap learned FSMap version 28 2023-09-09T23:14:20.794 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.778+0000 7f8265c2eb80 1 -- 172.21.15.82:0/3399865573 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f823c017070 con 0x55e7469cb710 2023-09-09T23:14:20.794 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.778+0000 7f8232ffd700 1 -- 172.21.15.82:0/3399865573 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82440a1640 con 0x55e7469cb710 2023-09-09T23:14:20.795 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.778+0000 7f8232ffd700 1 -- 172.21.15.82:0/3399865573 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f82440b1bd0 con 0x55e7469cb710 2023-09-09T23:14:20.795 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.778+0000 7f8265c2eb80 10 client.15864 fetch_fsmap finished waiting for FSMap version 28 2023-09-09T23:14:20.795 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.778+0000 7f8265c2eb80 10 client.15864 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2023-09-09T23:14:20.795 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.778+0000 7f8265c2eb80 20 client.15864 populate_metadata read hostname 'smithi082' 2023-09-09T23:14:20.795 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.778+0000 7f8265c2eb80 1 --2- 172.21.15.82:0/3399865573 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55e746a34ef0 0x55e746a352d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.796 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.778+0000 7f8265c2eb80 4 client.15864 mds_command: new command op to 24491 tid=0[{"prefix": "damage ls"}] 2023-09-09T23:14:20.796 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.778+0000 7f8265c2eb80 1 -- 172.21.15.82:0/3399865573 --> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e746916920 con 0x55e746a34ef0 2023-09-09T23:14:20.796 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.779+0000 7f824affd700 1 --2- 172.21.15.82:0/3399865573 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55e746a34ef0 0x55e746a352d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.796 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.779+0000 7f8232ffd700 10 client.15864 ms_handle_connect on v2:172.21.15.98:6832/3855146755 2023-09-09T23:14:20.796 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.779+0000 7f824affd700 1 --2- 172.21.15.82:0/3399865573 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55e746a34ef0 0x55e746a352d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6f7c8ce0761954ff server_cookie=d04caeb273933d40 in_seq=0 out_seq=0 2023-09-09T23:14:20.797 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.779+0000 7f8232ffd700 1 -- 172.21.15.82:0/3399865573 <== mds.3 v2:172.21.15.98:6832/3855146755 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e746916920 con 0x55e746a34ef0 2023-09-09T23:14:20.797 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.779+0000 7f8232ffd700 10 client.15864 handle_command_reply: tid=0 2023-09-09T23:14:20.797 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.779+0000 7f8265c2eb80 1 client.15864 shutdown 2023-09-09T23:14:20.797 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.779+0000 7f8265c2eb80 1 -- 172.21.15.82:0/3399865573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7469cb710 msgr2=0x55e7469daff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.797 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.779+0000 7f8265c2eb80 1 --2- 172.21.15.82:0/3399865573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7469cb710 0x55e7469daff0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8244097920 tx=0x7f82440843c0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.779+0000 7f8265c2eb80 1 -- 172.21.15.82:0/3399865573 shutdown_connections 2023-09-09T23:14:20.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.779+0000 7f8265c2eb80 1 --2- 172.21.15.82:0/3399865573 >> [v2:172.21.15.98:6832/3855146755,v1:172.21.15.98:6833/3855146755] conn(0x55e746a34ef0 0x55e746a352d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.779+0000 7f8265c2eb80 1 --2- 172.21.15.82:0/3399865573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e7469d5e40 0x55e7469dba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.779+0000 7f8232ffd700 0 client.15864 ms_handle_reset on v2:172.21.15.98:6832/3855146755 2023-09-09T23:14:20.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.779+0000 7f8265c2eb80 1 --2- 172.21.15.82:0/3399865573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7469cb710 0x55e7469daff0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.779+0000 7f8265c2eb80 1 --2- 172.21.15.82:0/3399865573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e7469d37b0 0x55e7469db530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.779+0000 7f8265c2eb80 1 -- 172.21.15.82:0/3399865573 >> 172.21.15.82:0/3399865573 conn(0x55e746940090 msgr2=0x55e7469c6670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:20.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.779+0000 7f8265c2eb80 1 -- 172.21.15.82:0/3399865573 shutdown_connections 2023-09-09T23:14:20.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.779+0000 7f8265c2eb80 1 -- 172.21.15.82:0/3399865573 wait complete. 2023-09-09T23:14:20.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.779+0000 7f8265c2eb80 20 client.15864 trim_cache size 0 max 16384 2023-09-09T23:14:20.800 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2023-09-09T23:14:20.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.780+0000 7f8259a68700 1 -- 172.21.15.82:0/2213674482 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f82400a08d0 msgr2=0x7f82400a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.780+0000 7f8259a68700 1 --2- 172.21.15.82:0/2213674482 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f82400a08d0 0x7f82400a2d90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8254196ae0 tx=0x7f823c009260 comp rx=0 tx=0).stop 2023-09-09T23:14:20.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.780+0000 7f8259a68700 1 -- 172.21.15.82:0/2213674482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82540ff440 msgr2=0x7f82541954d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.780+0000 7f8259a68700 1 --2- 172.21.15.82:0/2213674482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82540ff440 0x7f82541954d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f824400e9a0 tx=0x7f824400ecb0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.780+0000 7f8259a68700 1 -- 172.21.15.82:0/2213674482 shutdown_connections 2023-09-09T23:14:20.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.780+0000 7f8259a68700 1 --2- 172.21.15.82:0/2213674482 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f82400a08d0 0x7f82400a2d90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.780+0000 7f8259a68700 1 --2- 172.21.15.82:0/2213674482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82540ff440 0x7f82541954d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.780+0000 7f8259a68700 1 --2- 172.21.15.82:0/2213674482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82541054e0 0x7f8254199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.780+0000 7f8259a68700 1 --2- 172.21.15.82:0/2213674482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82540ffe00 0x7f8254195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.804 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.780+0000 7f8259a68700 1 -- 172.21.15.82:0/2213674482 >> 172.21.15.82:0/2213674482 conn(0x7f82540faf50 msgr2=0x7f82540fb530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:20.804 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.780+0000 7f8259a68700 1 -- 172.21.15.82:0/2213674482 shutdown_connections 2023-09-09T23:14:20.804 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.780+0000 7f8259a68700 1 -- 172.21.15.82:0/2213674482 wait complete. 2023-09-09T23:14:20.804 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-09T23:14:20.889 INFO:teuthology.orchestra.run.smithi082.stderr:4800000 of 50000000 tuples (9%) done (elapsed 7.47 s, remaining 70.37 s) 2023-09-09T23:14:20.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.950+0000 7f58fc9ea700 1 -- 172.21.15.82:0/3277162751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58f81020d0 msgr2=0x7f58f810a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.950+0000 7f58fc9ea700 1 --2- 172.21.15.82:0/3277162751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58f81020d0 0x7f58f810a5b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f58ec009990 tx=0x7f58ec009ca0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.951+0000 7f58fc9ea700 1 -- 172.21.15.82:0/3277162751 shutdown_connections 2023-09-09T23:14:20.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.951+0000 7f58fc9ea700 1 --2- 172.21.15.82:0/3277162751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58f810ab80 0x7f58f810cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.951+0000 7f58fc9ea700 1 --2- 172.21.15.82:0/3277162751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58f81017b0 0x7f58f8101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.951+0000 7f58fc9ea700 1 --2- 172.21.15.82:0/3277162751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58f81020d0 0x7f58f810a5b0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.951+0000 7f58fc9ea700 1 -- 172.21.15.82:0/3277162751 >> 172.21.15.82:0/3277162751 conn(0x7f58f80faf50 msgr2=0x7f58f80fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:20.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.951+0000 7f58fc9ea700 1 -- 172.21.15.82:0/3277162751 shutdown_connections 2023-09-09T23:14:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.951+0000 7f58fc9ea700 1 -- 172.21.15.82:0/3277162751 wait complete. 2023-09-09T23:14:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.952+0000 7f58fc9ea700 1 Processor -- start 2023-09-09T23:14:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.952+0000 7f58fc9ea700 1 -- start start 2023-09-09T23:14:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.952+0000 7f58fc9ea700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58f81017b0 0x7f58f80ff400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.952+0000 7f58fc9ea700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58f81020d0 0x7f58f80ff940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.952+0000 7f58fc9ea700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58f810ab80 0x7f58f81a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.952+0000 7f58fc9ea700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f8100170 con 0x7f58f810ab80 2023-09-09T23:14:20.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.952+0000 7f58fc9ea700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f81a4de0 con 0x7f58f81020d0 2023-09-09T23:14:20.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.952+0000 7f58fc9ea700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f81a4f20 con 0x7f58f81017b0 2023-09-09T23:14:20.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.952+0000 7f58f6d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58f810ab80 0x7f58f81a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.952+0000 7f58f6d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58f810ab80 0x7f58f81a48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57298/0 (socket says 172.21.15.82:57298) 2023-09-09T23:14:20.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.952+0000 7f58f6d9d700 1 -- 172.21.15.82:0/3911178717 learned_addr learned my addr 172.21.15.82:0/3911178717 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:20.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.953+0000 7f58f6d9d700 1 -- 172.21.15.82:0/3911178717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58f81017b0 msgr2=0x7f58f80ff400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.953+0000 7f58f6d9d700 1 --2- 172.21.15.82:0/3911178717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58f81017b0 0x7f58f80ff400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.953+0000 7f58f6d9d700 1 -- 172.21.15.82:0/3911178717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58f81020d0 msgr2=0x7f58f80ff940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.953+0000 7f58f6d9d700 1 --2- 172.21.15.82:0/3911178717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58f81020d0 0x7f58f80ff940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.953+0000 7f58f6d9d700 1 -- 172.21.15.82:0/3911178717 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ec0095f0 con 0x7f58f810ab80 2023-09-09T23:14:20.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.953+0000 7f58f6d9d700 1 --2- 172.21.15.82:0/3911178717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58f810ab80 0x7f58f81a48a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f58f000ef30 tx=0x7f58f000c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:20.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.953+0000 7f58e77fe700 1 -- 172.21.15.82:0/3911178717 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0018070 con 0x7f58f810ab80 2023-09-09T23:14:20.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.953+0000 7f58e77fe700 1 -- 172.21.15.82:0/3911178717 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58f000f040 con 0x7f58f810ab80 2023-09-09T23:14:20.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.953+0000 7f58e77fe700 1 -- 172.21.15.82:0/3911178717 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0003dc0 con 0x7f58f810ab80 2023-09-09T23:14:20.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.953+0000 7f58fc9ea700 1 -- 172.21.15.82:0/3911178717 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f81a5120 con 0x7f58f810ab80 2023-09-09T23:14:20.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.953+0000 7f58fc9ea700 1 -- 172.21.15.82:0/3911178717 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f81a55b0 con 0x7f58f810ab80 2023-09-09T23:14:20.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.955+0000 7f58e77fe700 1 -- 172.21.15.82:0/3911178717 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135492+0+0 (secure 0 0 0) 0x7f58f000caa0 con 0x7f58f810ab80 2023-09-09T23:14:20.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.956+0000 7f58e77fe700 1 --2- 172.21.15.82:0/3911178717 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f58e00a08d0 0x7f58e00a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.956+0000 7f58e77fe700 1 -- 172.21.15.82:0/3911178717 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f58f00cbe60 con 0x7f58f810ab80 2023-09-09T23:14:20.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.956+0000 7f58f659c700 1 --2- 172.21.15.82:0/3911178717 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f58e00a08d0 0x7f58e00a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.957+0000 7f58f659c700 1 --2- 172.21.15.82:0/3911178717 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f58e00a08d0 0x7f58e00a2d90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f58e8005950 tx=0x7f58e80058e0 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:20.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.959+0000 7f5908bb0b80 1 Processor -- start 2023-09-09T23:14:20.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.959+0000 7f5908bb0b80 1 -- start start 2023-09-09T23:14:20.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.960+0000 7f5908bb0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f3b129710 0x563f3b129af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.960+0000 7f5908bb0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f3b12a030 0x563f3b133900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.960+0000 7f5908bb0b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f3b133e40 0x563f3b136230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.960+0000 7f5908bb0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3b082730 con 0x563f3b12a030 2023-09-09T23:14:20.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.960+0000 7f5908bb0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3b080050 con 0x563f3b129710 2023-09-09T23:14:20.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.960+0000 7f5908bb0b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3b07b940 con 0x563f3b133e40 2023-09-09T23:14:20.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.960+0000 7f58f6d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f3b133e40 0x563f3b136230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.960+0000 7f58f6d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f3b133e40 0x563f3b136230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:55818/0 (socket says 172.21.15.82:55818) 2023-09-09T23:14:20.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.960+0000 7f58f6d9d700 1 -- 172.21.15.82:0/3365558381 learned_addr learned my addr 172.21.15.82:0/3365558381 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:20.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.961+0000 7f58f6d9d700 1 -- 172.21.15.82:0/3365558381 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f3b129710 msgr2=0x563f3b129af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:20.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.961+0000 7f58f6d9d700 1 --2- 172.21.15.82:0/3365558381 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f3b129710 0x563f3b129af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.961+0000 7f58f6d9d700 1 -- 172.21.15.82:0/3365558381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f3b12a030 msgr2=0x563f3b133900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-09T23:14:20.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.961+0000 7f58f6d9d700 1 --2- 172.21.15.82:0/3365558381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f3b12a030 0x563f3b133900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.961+0000 7f58f6d9d700 1 -- 172.21.15.82:0/3365558381 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f3aea7de0 con 0x563f3b133e40 2023-09-09T23:14:20.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.961+0000 7f58f6d9d700 1 --2- 172.21.15.82:0/3365558381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f3b133e40 0x563f3b136230 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f58f0098ec0 tx=0x7f58f0095630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6781d9a2dde8c2e3 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:20.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.961+0000 7f58e4ff9700 1 -- 172.21.15.82:0/3365558381 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f009f0d0 con 0x563f3b133e40 2023-09-09T23:14:20.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.961+0000 7f58e4ff9700 1 -- 172.21.15.82:0/3365558381 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58f00991c0 con 0x563f3b133e40 2023-09-09T23:14:20.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.961+0000 7f58e4ff9700 1 -- 172.21.15.82:0/3365558381 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0097680 con 0x563f3b133e40 2023-09-09T23:14:20.968 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.962+0000 7f5908bb0b80 1 -- 172.21.15.82:0/3365558381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f3b133e40 msgr2=0x563f3b136230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.969 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.962+0000 7f5908bb0b80 1 --2- 172.21.15.82:0/3365558381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f3b133e40 0x563f3b136230 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f58f0098ec0 tx=0x7f58f0095630 comp rx=0 tx=0).stop 2023-09-09T23:14:20.969 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.962+0000 7f5908bb0b80 1 -- 172.21.15.82:0/3365558381 shutdown_connections 2023-09-09T23:14:20.969 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.962+0000 7f5908bb0b80 1 --2- 172.21.15.82:0/3365558381 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f3b129710 0x563f3b129af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.969 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.962+0000 7f5908bb0b80 1 --2- 172.21.15.82:0/3365558381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f3b133e40 0x563f3b136230 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.969 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.962+0000 7f5908bb0b80 1 --2- 172.21.15.82:0/3365558381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f3b12a030 0x563f3b133900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.970 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.962+0000 7f5908bb0b80 1 -- 172.21.15.82:0/3365558381 >> 172.21.15.82:0/3365558381 conn(0x563f3b09e090 msgr2=0x563f3b124bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:20.970 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.962+0000 7f5908bb0b80 1 -- 172.21.15.82:0/3365558381 shutdown_connections 2023-09-09T23:14:20.970 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.962+0000 7f5908bb0b80 1 -- 172.21.15.82:0/3365558381 wait complete. 2023-09-09T23:14:20.970 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.963+0000 7f5908bb0b80 1 -- start start 2023-09-09T23:14:20.970 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.963+0000 7f5908bb0b80 1 Processor -- start 2023-09-09T23:14:20.970 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.963+0000 7f5908bb0b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f3b129710 0x563f3b130b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.963+0000 7f5908bb0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f3b12a030 0x563f3b1310b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.963+0000 7f5908bb0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f3b133e40 0x563f3b1315f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.963+0000 7f5908bb0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3b019b70 con 0x563f3b12a030 2023-09-09T23:14:20.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.963+0000 7f5908bb0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3b019870 con 0x563f3b133e40 2023-09-09T23:14:20.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.963+0000 7f5908bb0b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3aefd920 con 0x563f3b129710 2023-09-09T23:14:20.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.963+0000 7f5908bb0b80 10 client.0 _ll_register_callbacks cb 0x563f3af63a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:14:20.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.963+0000 7f58f5d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f3b12a030 0x563f3b1310b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.963+0000 7f58f5d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f3b12a030 0x563f3b1310b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57308/0 (socket says 172.21.15.82:57308) 2023-09-09T23:14:20.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.963+0000 7f58f5d9b700 1 -- 172.21.15.82:0/3285737707 learned_addr learned my addr 172.21.15.82:0/3285737707 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:20.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.964+0000 7f58f659c700 1 --2- 172.21.15.82:0/3285737707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f3b129710 0x563f3b130b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.964+0000 7f58f5d9b700 1 -- 172.21.15.82:0/3285737707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f3b129710 msgr2=0x563f3b130b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:20.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.964+0000 7f58f5d9b700 1 --2- 172.21.15.82:0/3285737707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f3b129710 0x563f3b130b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.964+0000 7f58f5d9b700 1 -- 172.21.15.82:0/3285737707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f3b133e40 msgr2=0x563f3b1315f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-09T23:14:20.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.964+0000 7f58f5d9b700 1 --2- 172.21.15.82:0/3285737707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f3b133e40 0x563f3b1315f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.964+0000 7f58f5d9b700 1 -- 172.21.15.82:0/3285737707 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f81a55b0 con 0x563f3b12a030 2023-09-09T23:14:20.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.964+0000 7f58f5d9b700 1 --2- 172.21.15.82:0/3285737707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f3b12a030 0x563f3b1310b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f58ec006010 tx=0x7f58ec005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1602f64960e325fa server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:20.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.964+0000 7f58ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:14:20.974 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.964+0000 7f58ddffb700 1 -- 172.21.15.82:0/3285737707 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec01d070 con 0x563f3b12a030 2023-09-09T23:14:20.974 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.964+0000 7f58ddffb700 1 -- 172.21.15.82:0/3285737707 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f58ec004830 con 0x563f3b12a030 2023-09-09T23:14:20.974 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.964+0000 7f5908bb0b80 1 -- 172.21.15.82:0/3285737707 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f3afd34f0 con 0x563f3b12a030 2023-09-09T23:14:20.974 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.964+0000 7f58ddffb700 1 -- 172.21.15.82:0/3285737707 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58ec0049f0 con 0x563f3b12a030 2023-09-09T23:14:20.974 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.964+0000 7f5908bb0b80 10 client.15873 fetch_fsmap learned FSMap version 28 2023-09-09T23:14:20.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.964+0000 7f5908bb0b80 1 -- 172.21.15.82:0/3285737707 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563f3ae90c60 con 0x563f3b12a030 2023-09-09T23:14:20.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.964+0000 7f58ddffb700 1 -- 172.21.15.82:0/3285737707 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563f3afd34f0 con 0x563f3b12a030 2023-09-09T23:14:20.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.965+0000 7f58ddffb700 1 -- 172.21.15.82:0/3285737707 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f58ec022620 con 0x563f3b12a030 2023-09-09T23:14:20.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.965+0000 7f5908bb0b80 10 client.15873 fetch_fsmap finished waiting for FSMap version 28 2023-09-09T23:14:20.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.965+0000 7f5908bb0b80 10 client.15873 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-09-09T23:14:20.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.965+0000 7f5908bb0b80 20 client.15873 populate_metadata read hostname 'smithi082' 2023-09-09T23:14:20.976 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.965+0000 7f5908bb0b80 1 --2- 172.21.15.82:0/3285737707 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x563f3b143ae0 0x563f3b145f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.976 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.965+0000 7f5908bb0b80 4 client.15873 mds_command: new command op to 24523 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:14:20.976 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.965+0000 7f5908bb0b80 1 -- 172.21.15.82:0/3285737707 --> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f3b0908a0 con 0x563f3b143ae0 2023-09-09T23:14:20.976 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.965+0000 7f58f659c700 1 --2- 172.21.15.82:0/3285737707 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x563f3b143ae0 0x563f3b145f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.976 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.966+0000 7f58ddffb700 10 client.15873 ms_handle_connect on v2:172.21.15.170:6838/3858369735 2023-09-09T23:14:20.977 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.966+0000 7f58f659c700 1 --2- 172.21.15.82:0/3285737707 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x563f3b143ae0 0x563f3b145f00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9954befab03cd36 server_cookie=62444ba15a72ec94 in_seq=0 out_seq=0 2023-09-09T23:14:20.977 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.967+0000 7f58ddffb700 1 -- 172.21.15.82:0/3285737707 <== mds.2 v2:172.21.15.170:6838/3858369735 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563f3b0908a0 con 0x563f3b143ae0 2023-09-09T23:14:20.977 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.967+0000 7f58ddffb700 10 client.15873 handle_command_reply: tid=0 2023-09-09T23:14:20.977 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.967+0000 7f5908bb0b80 1 client.15873 shutdown 2023-09-09T23:14:20.977 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.967+0000 7f5908bb0b80 1 -- 172.21.15.82:0/3285737707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f3b12a030 msgr2=0x563f3b1310b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.967+0000 7f5908bb0b80 1 --2- 172.21.15.82:0/3285737707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f3b12a030 0x563f3b1310b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f58ec006010 tx=0x7f58ec005c70 comp rx=0 tx=0).stop 2023-09-09T23:14:20.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.967+0000 7f5908bb0b80 1 -- 172.21.15.82:0/3285737707 shutdown_connections 2023-09-09T23:14:20.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.967+0000 7f5908bb0b80 1 --2- 172.21.15.82:0/3285737707 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x563f3b143ae0 0x563f3b145f00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.967+0000 7f5908bb0b80 1 --2- 172.21.15.82:0/3285737707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f3b133e40 0x563f3b1315f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.967+0000 7f5908bb0b80 1 --2- 172.21.15.82:0/3285737707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f3b129710 0x563f3b130b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.967+0000 7f5908bb0b80 1 --2- 172.21.15.82:0/3285737707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f3b12a030 0x563f3b1310b0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.967+0000 7f5908bb0b80 1 -- 172.21.15.82:0/3285737707 >> 172.21.15.82:0/3285737707 conn(0x563f3b09e090 msgr2=0x563f3b124780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:20.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.967+0000 7f58ddffb700 0 client.15873 ms_handle_reset on v2:172.21.15.170:6838/3858369735 2023-09-09T23:14:20.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.967+0000 7f5908bb0b80 1 -- 172.21.15.82:0/3285737707 shutdown_connections 2023-09-09T23:14:20.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.967+0000 7f5908bb0b80 1 -- 172.21.15.82:0/3285737707 wait complete. 2023-09-09T23:14:20.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.967+0000 7f5908bb0b80 20 client.15873 trim_cache size 0 max 16384 2023-09-09T23:14:20.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.981+0000 7f5908bb0b80 1 Processor -- start 2023-09-09T23:14:20.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.982+0000 7f5908bb0b80 1 -- start start 2023-09-09T23:14:20.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.982+0000 7f5908bb0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f3b129710 0x563f3b13a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.982+0000 7f5908bb0b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f3b12a030 0x563f3b12de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.982+0000 7f5908bb0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f3b133e40 0x563f3b12e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.982+0000 7f5908bb0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3b1260b0 con 0x563f3b133e40 2023-09-09T23:14:20.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.982+0000 7f5908bb0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3b129eb0 con 0x563f3b129710 2023-09-09T23:14:20.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.982+0000 7f5908bb0b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3b12a9f0 con 0x563f3b12a030 2023-09-09T23:14:20.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.982+0000 7f58f6d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f3b133e40 0x563f3b12e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.982+0000 7f58f5d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f3b12a030 0x563f3b12de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.982+0000 7f58f6d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f3b133e40 0x563f3b12e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57318/0 (socket says 172.21.15.82:57318) 2023-09-09T23:14:20.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.982+0000 7f58f6d9d700 1 -- 172.21.15.82:0/2651914767 learned_addr learned my addr 172.21.15.82:0/2651914767 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:20.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.982+0000 7f58f659c700 1 --2- 172.21.15.82:0/2651914767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f3b129710 0x563f3b13a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.982+0000 7f58f6d9d700 1 -- 172.21.15.82:0/2651914767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f3b12a030 msgr2=0x563f3b12de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:20.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.982+0000 7f58f6d9d700 1 --2- 172.21.15.82:0/2651914767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f3b12a030 0x563f3b12de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.982+0000 7f58f6d9d700 1 -- 172.21.15.82:0/2651914767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f3b129710 msgr2=0x563f3b13a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:20.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.982+0000 7f58f6d9d700 1 --2- 172.21.15.82:0/2651914767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f3b129710 0x563f3b13a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.982+0000 7f58f6d9d700 1 -- 172.21.15.82:0/2651914767 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f3aece250 con 0x563f3b133e40 2023-09-09T23:14:20.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.982+0000 7f58f659c700 1 --2- 172.21.15.82:0/2651914767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f3b129710 0x563f3b13a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:14:20.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.983+0000 7f58f6d9d700 1 --2- 172.21.15.82:0/2651914767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f3b133e40 0x563f3b12e340 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f58f00c95b0 tx=0x7f58f0098e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d507d82497f85f3 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:20.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.983+0000 7f58e57fa700 1 -- 172.21.15.82:0/2651914767 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f009cff0 con 0x563f3b133e40 2023-09-09T23:14:20.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.983+0000 7f58e57fa700 1 -- 172.21.15.82:0/2651914767 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58f00a11a0 con 0x563f3b133e40 2023-09-09T23:14:20.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.983+0000 7f58e57fa700 1 -- 172.21.15.82:0/2651914767 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f00986b0 con 0x563f3b133e40 2023-09-09T23:14:20.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.983+0000 7f5908bb0b80 1 -- 172.21.15.82:0/2651914767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f3b133e40 msgr2=0x563f3b12e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:20.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.983+0000 7f5908bb0b80 1 --2- 172.21.15.82:0/2651914767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f3b133e40 0x563f3b12e340 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f58f00c95b0 tx=0x7f58f0098e30 comp rx=0 tx=0).stop 2023-09-09T23:14:20.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.983+0000 7f5908bb0b80 1 -- 172.21.15.82:0/2651914767 shutdown_connections 2023-09-09T23:14:20.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.983+0000 7f5908bb0b80 1 --2- 172.21.15.82:0/2651914767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f3b129710 0x563f3b13a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.983+0000 7f5908bb0b80 1 --2- 172.21.15.82:0/2651914767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f3b133e40 0x563f3b12e340 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.983+0000 7f5908bb0b80 1 --2- 172.21.15.82:0/2651914767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f3b12a030 0x563f3b12de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.983+0000 7f5908bb0b80 1 -- 172.21.15.82:0/2651914767 >> 172.21.15.82:0/2651914767 conn(0x563f3b09e090 msgr2=0x563f3b12bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:20.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.984+0000 7f5908bb0b80 1 -- 172.21.15.82:0/2651914767 shutdown_connections 2023-09-09T23:14:20.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.984+0000 7f5908bb0b80 1 -- 172.21.15.82:0/2651914767 wait complete. 2023-09-09T23:14:20.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.984+0000 7f5908bb0b80 1 -- start start 2023-09-09T23:14:20.991 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.984+0000 7f5908bb0b80 1 Processor -- start 2023-09-09T23:14:20.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.984+0000 7f5908bb0b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f3b129710 0x563f3b18fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.984+0000 7f5908bb0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f3b12a030 0x563f3b1900e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.984+0000 7f5908bb0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f3b133e40 0x563f3b1389d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.984+0000 7f5908bb0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3b138fd0 con 0x563f3b133e40 2023-09-09T23:14:20.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.984+0000 7f5908bb0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3b139140 con 0x563f3b12a030 2023-09-09T23:14:20.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.984+0000 7f5908bb0b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3b139300 con 0x563f3b129710 2023-09-09T23:14:20.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.984+0000 7f5908bb0b80 10 client.0 _ll_register_callbacks cb 0x7f58f00097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:14:20.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.984+0000 7f58f6d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f3b133e40 0x563f3b1389d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.984+0000 7f58f5d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f3b12a030 0x563f3b1900e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.984+0000 7f58f5d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f3b12a030 0x563f3b1900e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:42886/0 (socket says 172.21.15.82:42886) 2023-09-09T23:14:20.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.984+0000 7f58f6d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f3b133e40 0x563f3b1389d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57324/0 (socket says 172.21.15.82:57324) 2023-09-09T23:14:20.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.984+0000 7f58f5d9b700 1 -- 172.21.15.82:0/2951344684 learned_addr learned my addr 172.21.15.82:0/2951344684 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:20.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.984+0000 7f58f659c700 1 --2- 172.21.15.82:0/2951344684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f3b129710 0x563f3b18fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.985+0000 7f58f6d9d700 1 -- 172.21.15.82:0/2951344684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f3b129710 msgr2=0x563f3b18fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:20.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.985+0000 7f58f6d9d700 1 --2- 172.21.15.82:0/2951344684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f3b129710 0x563f3b18fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.985+0000 7f58f6d9d700 1 -- 172.21.15.82:0/2951344684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f3b12a030 msgr2=0x563f3b1900e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:20.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.985+0000 7f58f6d9d700 1 --2- 172.21.15.82:0/2951344684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f3b12a030 0x563f3b1900e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:20.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.985+0000 7f58f6d9d700 1 -- 172.21.15.82:0/2951344684 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e800e920 con 0x563f3b133e40 2023-09-09T23:14:20.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.985+0000 7f58f5d9b700 1 --2- 172.21.15.82:0/2951344684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f3b12a030 0x563f3b1900e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:14:20.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.985+0000 7f58f6d9d700 1 --2- 172.21.15.82:0/2951344684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f3b133e40 0x563f3b1389d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f58f000ef00 tx=0x7f58f0009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49baba56dd55b460 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:20.996 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.985+0000 7f58ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:14:20.996 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.985+0000 7f58ddffb700 1 -- 172.21.15.82:0/2951344684 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f009cff0 con 0x563f3b133e40 2023-09-09T23:14:20.996 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.985+0000 7f58ddffb700 1 -- 172.21.15.82:0/2951344684 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f58f000cd50 con 0x563f3b133e40 2023-09-09T23:14:20.996 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.985+0000 7f5908bb0b80 1 -- 172.21.15.82:0/2951344684 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58f00991c0 con 0x563f3b133e40 2023-09-09T23:14:20.996 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.985+0000 7f58ddffb700 1 -- 172.21.15.82:0/2951344684 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58f00a0d10 con 0x563f3b133e40 2023-09-09T23:14:20.997 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.985+0000 7f5908bb0b80 10 client.15879 fetch_fsmap learned FSMap version 28 2023-09-09T23:14:20.997 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.985+0000 7f5908bb0b80 1 -- 172.21.15.82:0/2951344684 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563f3aea7de0 con 0x563f3b133e40 2023-09-09T23:14:20.997 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.985+0000 7f58ddffb700 1 -- 172.21.15.82:0/2951344684 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58f00991c0 con 0x563f3b133e40 2023-09-09T23:14:20.997 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.985+0000 7f58ddffb700 1 -- 172.21.15.82:0/2951344684 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f58f0094fb0 con 0x563f3b133e40 2023-09-09T23:14:20.997 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.985+0000 7f5908bb0b80 10 client.15879 fetch_fsmap finished waiting for FSMap version 28 2023-09-09T23:14:20.997 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.985+0000 7f5908bb0b80 10 client.15879 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-09-09T23:14:20.998 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.985+0000 7f5908bb0b80 20 client.15879 populate_metadata read hostname 'smithi082' 2023-09-09T23:14:20.998 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.985+0000 7f5908bb0b80 1 --2- 172.21.15.82:0/2951344684 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x563f3b1a95d0 0x563f3b1ab9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:20.998 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.985+0000 7f5908bb0b80 4 client.15879 mds_command: new command op to 24523 tid=0[{"prefix": "damage ls"}] 2023-09-09T23:14:20.998 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.985+0000 7f5908bb0b80 1 -- 172.21.15.82:0/2951344684 --> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563f3b074920 con 0x563f3b1a95d0 2023-09-09T23:14:20.998 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.986+0000 7f58f659c700 1 --2- 172.21.15.82:0/2951344684 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x563f3b1a95d0 0x563f3b1ab9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:20.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.986+0000 7f58f659c700 1 --2- 172.21.15.82:0/2951344684 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x563f3b1a95d0 0x563f3b1ab9c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=665673ca523f879b server_cookie=15663cbc00a219f2 in_seq=0 out_seq=0 2023-09-09T23:14:20.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.986+0000 7f58ddffb700 10 client.15879 ms_handle_connect on v2:172.21.15.170:6838/3858369735 2023-09-09T23:14:20.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.986+0000 7f58ddffb700 1 -- 172.21.15.82:0/2951344684 <== mds.2 v2:172.21.15.170:6838/3858369735 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563f3b074920 con 0x563f3b1a95d0 2023-09-09T23:14:20.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.986+0000 7f58ddffb700 10 client.15879 handle_command_reply: tid=0 2023-09-09T23:14:20.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.986+0000 7f5908bb0b80 1 client.15879 shutdown 2023-09-09T23:14:21.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.986+0000 7f5908bb0b80 1 -- 172.21.15.82:0/2951344684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f3b133e40 msgr2=0x563f3b1389d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:21.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.986+0000 7f5908bb0b80 1 --2- 172.21.15.82:0/2951344684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f3b133e40 0x563f3b1389d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f58f000ef00 tx=0x7f58f0009ed0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.986+0000 7f5908bb0b80 1 -- 172.21.15.82:0/2951344684 shutdown_connections 2023-09-09T23:14:21.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.986+0000 7f5908bb0b80 1 --2- 172.21.15.82:0/2951344684 >> [v2:172.21.15.170:6838/3858369735,v1:172.21.15.170:6839/3858369735] conn(0x563f3b1a95d0 0x563f3b1ab9c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.986+0000 7f5908bb0b80 1 --2- 172.21.15.82:0/2951344684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f3b133e40 0x563f3b1389d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.001 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.986+0000 7f5908bb0b80 1 --2- 172.21.15.82:0/2951344684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f3b129710 0x563f3b18fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.001 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.986+0000 7f58ddffb700 0 client.15879 ms_handle_reset on v2:172.21.15.170:6838/3858369735 2023-09-09T23:14:21.001 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.986+0000 7f5908bb0b80 1 --2- 172.21.15.82:0/2951344684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f3b12a030 0x563f3b1900e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.001 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.986+0000 7f5908bb0b80 1 -- 172.21.15.82:0/2951344684 >> 172.21.15.82:0/2951344684 conn(0x563f3b09e090 msgr2=0x563f3b12f320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:21.001 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.986+0000 7f5908bb0b80 1 -- 172.21.15.82:0/2951344684 shutdown_connections 2023-09-09T23:14:21.002 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.986+0000 7f5908bb0b80 1 -- 172.21.15.82:0/2951344684 wait complete. 2023-09-09T23:14:21.002 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.987+0000 7f5908bb0b80 20 client.15879 trim_cache size 0 max 16384 2023-09-09T23:14:21.002 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2023-09-09T23:14:21.003 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.987+0000 7f58fc9ea700 1 -- 172.21.15.82:0/3911178717 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f58e00a08d0 msgr2=0x7f58e00a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:21.004 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.987+0000 7f58fc9ea700 1 --2- 172.21.15.82:0/3911178717 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f58e00a08d0 0x7f58e00a2d90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f58e8005950 tx=0x7f58e80058e0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.004 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.987+0000 7f58fc9ea700 1 -- 172.21.15.82:0/3911178717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58f810ab80 msgr2=0x7f58f81a48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:21.004 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.987+0000 7f58fc9ea700 1 --2- 172.21.15.82:0/3911178717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58f810ab80 0x7f58f81a48a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f58f000ef30 tx=0x7f58f000c450 comp rx=0 tx=0).stop 2023-09-09T23:14:21.004 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.987+0000 7f58fc9ea700 1 -- 172.21.15.82:0/3911178717 shutdown_connections 2023-09-09T23:14:21.004 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.987+0000 7f58fc9ea700 1 --2- 172.21.15.82:0/3911178717 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f58e00a08d0 0x7f58e00a2d90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.005 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.987+0000 7f58fc9ea700 1 --2- 172.21.15.82:0/3911178717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58f810ab80 0x7f58f81a48a0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.005 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.987+0000 7f58fc9ea700 1 --2- 172.21.15.82:0/3911178717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58f81017b0 0x7f58f80ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.005 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.987+0000 7f58fc9ea700 1 --2- 172.21.15.82:0/3911178717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58f81020d0 0x7f58f80ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.005 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.987+0000 7f58fc9ea700 1 -- 172.21.15.82:0/3911178717 >> 172.21.15.82:0/3911178717 conn(0x7f58f80faf50 msgr2=0x7f58f80fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:21.005 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.987+0000 7f58fc9ea700 1 -- 172.21.15.82:0/3911178717 shutdown_connections 2023-09-09T23:14:21.005 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:20.987+0000 7f58fc9ea700 1 -- 172.21.15.82:0/3911178717 wait complete. 2023-09-09T23:14:21.006 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-09T23:14:21.068 INFO:teuthology.orchestra.run.smithi082.stderr:4900000 of 50000000 tuples (9%) done (elapsed 7.65 s, remaining 70.43 s) 2023-09-09T23:14:21.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.148+0000 7eff9c6ed700 1 -- 172.21.15.82:0/3758346343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff940ffe70 msgr2=0x7eff9410c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:21.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.148+0000 7eff9c6ed700 1 --2- 172.21.15.82:0/3758346343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff940ffe70 0x7eff9410c330 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00b0d0 tx=0x7eff8c00b3e0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.148+0000 7eff9c6ed700 1 -- 172.21.15.82:0/3758346343 shutdown_connections 2023-09-09T23:14:21.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.148+0000 7eff9c6ed700 1 --2- 172.21.15.82:0/3758346343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff940ffe70 0x7eff9410c330 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.148+0000 7eff9c6ed700 1 --2- 172.21.15.82:0/3758346343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff94105570 0x7eff94105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.148+0000 7eff9c6ed700 1 --2- 172.21.15.82:0/3758346343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff940ff4b0 0x7eff940ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.148+0000 7eff9c6ed700 1 -- 172.21.15.82:0/3758346343 >> 172.21.15.82:0/3758346343 conn(0x7eff940fafa0 msgr2=0x7eff940fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:21.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.148+0000 7eff9c6ed700 1 -- 172.21.15.82:0/3758346343 shutdown_connections 2023-09-09T23:14:21.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.148+0000 7eff9c6ed700 1 -- 172.21.15.82:0/3758346343 wait complete. 2023-09-09T23:14:21.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.148+0000 7eff9c6ed700 1 Processor -- start 2023-09-09T23:14:21.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.148+0000 7eff9c6ed700 1 -- start start 2023-09-09T23:14:21.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.148+0000 7eff9c6ed700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff940ff4b0 0x7eff94195320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.148+0000 7eff9c6ed700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff940ffe70 0x7eff94195860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.148+0000 7eff9c6ed700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff94105570 0x7eff94195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.148+0000 7eff9c6ed700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff941a4b60 con 0x7eff940ffe70 2023-09-09T23:14:21.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.148+0000 7eff9c6ed700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff941a4cd0 con 0x7eff940ff4b0 2023-09-09T23:14:21.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.148+0000 7eff9c6ed700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff941a4e90 con 0x7eff94105570 2023-09-09T23:14:21.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.149+0000 7eff9a489700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff940ff4b0 0x7eff94195320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.149+0000 7eff9a489700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff940ff4b0 0x7eff94195320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:42898/0 (socket says 172.21.15.82:42898) 2023-09-09T23:14:21.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.149+0000 7eff9a489700 1 -- 172.21.15.82:0/812172498 learned_addr learned my addr 172.21.15.82:0/812172498 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:21.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.149+0000 7eff99c88700 1 --2- 172.21.15.82:0/812172498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff940ffe70 0x7eff94195860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.149+0000 7eff9ac8a700 1 --2- 172.21.15.82:0/812172498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff94105570 0x7eff94195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.149+0000 7eff9a489700 1 -- 172.21.15.82:0/812172498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff94105570 msgr2=0x7eff94195da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:21.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.149+0000 7eff9a489700 1 --2- 172.21.15.82:0/812172498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff94105570 0x7eff94195da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.153 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.149+0000 7eff9a489700 1 -- 172.21.15.82:0/812172498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff940ffe70 msgr2=0x7eff94195860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:21.153 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.149+0000 7eff9a489700 1 --2- 172.21.15.82:0/812172498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff940ffe70 0x7eff94195860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.153 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.149+0000 7eff9a489700 1 -- 172.21.15.82:0/812172498 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff8c009d00 con 0x7eff940ff4b0 2023-09-09T23:14:21.153 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.149+0000 7eff9a489700 1 --2- 172.21.15.82:0/812172498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff940ff4b0 0x7eff94195320 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7eff9000d950 tx=0x7eff9000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:21.153 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.149+0000 7eff8b7fe700 1 -- 172.21.15.82:0/812172498 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9000f890 con 0x7eff940ff4b0 2023-09-09T23:14:21.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.149+0000 7eff9c6ed700 1 -- 172.21.15.82:0/812172498 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff941a5170 con 0x7eff940ff4b0 2023-09-09T23:14:21.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.149+0000 7eff8b7fe700 1 -- 172.21.15.82:0/812172498 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff90010460 con 0x7eff940ff4b0 2023-09-09T23:14:21.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.149+0000 7eff8b7fe700 1 -- 172.21.15.82:0/812172498 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9000e610 con 0x7eff940ff4b0 2023-09-09T23:14:21.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.149+0000 7eff9c6ed700 1 -- 172.21.15.82:0/812172498 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff941a5680 con 0x7eff940ff4b0 2023-09-09T23:14:21.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.152+0000 7eff8b7fe700 1 -- 172.21.15.82:0/812172498 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) v1 ==== 135492+0+0 (secure 0 0 0) 0x7eff900108d0 con 0x7eff940ff4b0 2023-09-09T23:14:21.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.153+0000 7eff8b7fe700 1 --2- 172.21.15.82:0/812172498 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7eff800a08d0 0x7eff800a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.153+0000 7eff99c88700 1 --2- 172.21.15.82:0/812172498 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7eff800a08d0 0x7eff800a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.161 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.153+0000 7eff8b7fe700 1 -- 172.21.15.82:0/812172498 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7eff900cbfb0 con 0x7eff940ff4b0 2023-09-09T23:14:21.161 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.153+0000 7eff99c88700 1 --2- 172.21.15.82:0/812172498 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7eff800a08d0 0x7eff800a2d90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7eff94069050 tx=0x7eff84009260 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:21.161 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.153+0000 7effa88b3b80 1 Processor -- start 2023-09-09T23:14:21.161 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.154+0000 7effa88b3b80 1 -- start start 2023-09-09T23:14:21.161 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.154+0000 7effa88b3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e3deb5710 0x555e3deb5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.162 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.154+0000 7effa88b3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e3deb6030 0x555e3debf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.162 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.154+0000 7effa88b3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e3debfe40 0x555e3dec2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.162 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.154+0000 7effa88b3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3de0e730 con 0x555e3deb5710 2023-09-09T23:14:21.162 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.154+0000 7effa88b3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3de0c050 con 0x555e3debfe40 2023-09-09T23:14:21.162 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.154+0000 7effa88b3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3de07940 con 0x555e3deb6030 2023-09-09T23:14:21.163 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.154+0000 7eff9a489700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e3deb5710 0x555e3deb5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.163 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.154+0000 7eff9a489700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e3deb5710 0x555e3deb5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57344/0 (socket says 172.21.15.82:57344) 2023-09-09T23:14:21.163 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.154+0000 7eff9ac8a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e3debfe40 0x555e3dec2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.163 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.154+0000 7eff9a489700 1 -- 172.21.15.82:0/3301091663 learned_addr learned my addr 172.21.15.82:0/3301091663 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:21.163 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.154+0000 7eff99c88700 1 --2- 172.21.15.82:0/3301091663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e3deb6030 0x555e3debf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.163 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.154+0000 7eff9ac8a700 1 -- 172.21.15.82:0/3301091663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e3deb6030 msgr2=0x555e3debf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:21.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.154+0000 7eff9ac8a700 1 --2- 172.21.15.82:0/3301091663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e3deb6030 0x555e3debf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.154+0000 7eff9ac8a700 1 -- 172.21.15.82:0/3301091663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e3deb5710 msgr2=0x555e3deb5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:21.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.154+0000 7eff9ac8a700 1 --2- 172.21.15.82:0/3301091663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e3deb5710 0x555e3deb5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.154+0000 7eff9ac8a700 1 -- 172.21.15.82:0/3301091663 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e3dc33de0 con 0x555e3debfe40 2023-09-09T23:14:21.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.154+0000 7eff9a489700 1 --2- 172.21.15.82:0/3301091663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e3deb5710 0x555e3deb5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:14:21.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.154+0000 7eff9ac8a700 1 --2- 172.21.15.82:0/3301091663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e3debfe40 0x555e3dec2230 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00b770 tx=0x7eff8c009c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b0fa1ad7241f92 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:21.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.155+0000 7eff88ff9700 1 -- 172.21.15.82:0/3301091663 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c00e040 con 0x555e3debfe40 2023-09-09T23:14:21.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.155+0000 7eff88ff9700 1 -- 172.21.15.82:0/3301091663 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff8c0074e0 con 0x555e3debfe40 2023-09-09T23:14:21.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.155+0000 7eff88ff9700 1 -- 172.21.15.82:0/3301091663 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c005070 con 0x555e3debfe40 2023-09-09T23:14:21.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.155+0000 7effa88b3b80 1 -- 172.21.15.82:0/3301091663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e3debfe40 msgr2=0x555e3dec2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:21.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.155+0000 7effa88b3b80 1 --2- 172.21.15.82:0/3301091663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e3debfe40 0x555e3dec2230 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00b770 tx=0x7eff8c009c90 comp rx=0 tx=0).stop 2023-09-09T23:14:21.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.155+0000 7effa88b3b80 1 -- 172.21.15.82:0/3301091663 shutdown_connections 2023-09-09T23:14:21.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.155+0000 7effa88b3b80 1 --2- 172.21.15.82:0/3301091663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e3debfe40 0x555e3dec2230 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.155+0000 7effa88b3b80 1 --2- 172.21.15.82:0/3301091663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e3deb5710 0x555e3deb5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.155+0000 7effa88b3b80 1 --2- 172.21.15.82:0/3301091663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e3deb6030 0x555e3debf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.155+0000 7effa88b3b80 1 -- 172.21.15.82:0/3301091663 >> 172.21.15.82:0/3301091663 conn(0x555e3de2a090 msgr2=0x555e3deb0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:21.167 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.155+0000 7effa88b3b80 1 -- 172.21.15.82:0/3301091663 shutdown_connections 2023-09-09T23:14:21.167 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.155+0000 7effa88b3b80 1 -- 172.21.15.82:0/3301091663 wait complete. 2023-09-09T23:14:21.167 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.156+0000 7effa88b3b80 1 -- start start 2023-09-09T23:14:21.167 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.156+0000 7effa88b3b80 1 Processor -- start 2023-09-09T23:14:21.167 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.156+0000 7effa88b3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e3deb5710 0x555e3debf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.168 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.156+0000 7effa88b3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e3deb6030 0x555e3debca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.168 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.156+0000 7effa88b3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e3debfe40 0x555e3debd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.168 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.156+0000 7effa88b3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3dda5b70 con 0x555e3deb5710 2023-09-09T23:14:21.168 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.156+0000 7effa88b3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3dda5870 con 0x555e3deb6030 2023-09-09T23:14:21.168 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.156+0000 7effa88b3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3dc89920 con 0x555e3debfe40 2023-09-09T23:14:21.169 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.156+0000 7eff9a489700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e3deb5710 0x555e3debf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.169 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.156+0000 7effa88b3b80 10 client.0 _ll_register_callbacks cb 0x555e3dcefa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:14:21.169 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.156+0000 7eff9a489700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e3deb5710 0x555e3debf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57348/0 (socket says 172.21.15.82:57348) 2023-09-09T23:14:21.169 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.156+0000 7eff9a489700 1 -- 172.21.15.82:0/1179374793 learned_addr learned my addr 172.21.15.82:0/1179374793 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:21.169 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.156+0000 7eff99c88700 1 --2- 172.21.15.82:0/1179374793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e3deb6030 0x555e3debca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.169 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.156+0000 7eff9ac8a700 1 --2- 172.21.15.82:0/1179374793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e3debfe40 0x555e3debd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.157+0000 7eff9a489700 1 -- 172.21.15.82:0/1179374793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e3debfe40 msgr2=0x555e3debd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:21.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.157+0000 7eff9a489700 1 --2- 172.21.15.82:0/1179374793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e3debfe40 0x555e3debd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.157+0000 7eff9a489700 1 -- 172.21.15.82:0/1179374793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e3deb6030 msgr2=0x555e3debca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:21.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.157+0000 7eff9a489700 1 --2- 172.21.15.82:0/1179374793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e3deb6030 0x555e3debca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.157+0000 7eff9a489700 1 -- 172.21.15.82:0/1179374793 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff941a5680 con 0x555e3deb5710 2023-09-09T23:14:21.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.157+0000 7eff9ac8a700 1 --2- 172.21.15.82:0/1179374793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e3debfe40 0x555e3debd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:14:21.171 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.157+0000 7eff9a489700 1 --2- 172.21.15.82:0/1179374793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e3deb5710 0x555e3debf7f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7eff900c9700 tx=0x7eff90099280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2dee92ca5c3e09e server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:21.171 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.157+0000 7eff7dffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:14:21.171 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.157+0000 7eff7dffb700 1 -- 172.21.15.82:0/1179374793 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff900a12f0 con 0x555e3deb5710 2023-09-09T23:14:21.171 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.157+0000 7eff7dffb700 1 -- 172.21.15.82:0/1179374793 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7eff90098770 con 0x555e3deb5710 2023-09-09T23:14:21.171 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.157+0000 7effa88b3b80 1 -- 172.21.15.82:0/1179374793 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e3dd5f4f0 con 0x555e3deb5710 2023-09-09T23:14:21.172 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.157+0000 7eff7dffb700 1 -- 172.21.15.82:0/1179374793 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff90098930 con 0x555e3deb5710 2023-09-09T23:14:21.172 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.157+0000 7effa88b3b80 10 client.15891 fetch_fsmap learned FSMap version 28 2023-09-09T23:14:21.172 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.157+0000 7effa88b3b80 1 -- 172.21.15.82:0/1179374793 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7eff9410cc30 con 0x555e3deb5710 2023-09-09T23:14:21.172 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.157+0000 7eff7dffb700 1 -- 172.21.15.82:0/1179374793 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff90063080 con 0x555e3deb5710 2023-09-09T23:14:21.172 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.157+0000 7eff7dffb700 1 -- 172.21.15.82:0/1179374793 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7eff900c98d0 con 0x555e3deb5710 2023-09-09T23:14:21.172 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.157+0000 7effa88b3b80 10 client.15891 fetch_fsmap finished waiting for FSMap version 28 2023-09-09T23:14:21.173 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.157+0000 7effa88b3b80 10 client.15891 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-09-09T23:14:21.173 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.157+0000 7effa88b3b80 20 client.15891 populate_metadata read hostname 'smithi082' 2023-09-09T23:14:21.173 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.157+0000 7effa88b3b80 1 --2- 172.21.15.82:0/1179374793 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x555e3decf720 0x555e3ded1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.173 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.158+0000 7effa88b3b80 4 client.15891 mds_command: new command op to 24542 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:14:21.173 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.158+0000 7effa88b3b80 1 -- 172.21.15.82:0/1179374793 --> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e3de1c8a0 con 0x555e3decf720 2023-09-09T23:14:21.174 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.158+0000 7eff99c88700 1 --2- 172.21.15.82:0/1179374793 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x555e3decf720 0x555e3ded1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.174 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.158+0000 7eff99c88700 1 --2- 172.21.15.82:0/1179374793 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x555e3decf720 0x555e3ded1b40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=30f061332739e73d server_cookie=6958950ac7e128d6 in_seq=0 out_seq=0 2023-09-09T23:14:21.174 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.158+0000 7eff7dffb700 10 client.15891 ms_handle_connect on v2:172.21.15.98:6836/2019449840 2023-09-09T23:14:21.174 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.159+0000 7eff7dffb700 1 -- 172.21.15.82:0/1179374793 <== mds.1 v2:172.21.15.98:6836/2019449840 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555e3de1c8a0 con 0x555e3decf720 2023-09-09T23:14:21.174 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.159+0000 7eff7dffb700 10 client.15891 handle_command_reply: tid=0 2023-09-09T23:14:21.174 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.159+0000 7effa88b3b80 1 client.15891 shutdown 2023-09-09T23:14:21.175 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.159+0000 7effa88b3b80 1 -- 172.21.15.82:0/1179374793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e3deb5710 msgr2=0x555e3debf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:21.175 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.159+0000 7effa88b3b80 1 --2- 172.21.15.82:0/1179374793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e3deb5710 0x555e3debf7f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7eff900c9700 tx=0x7eff90099280 comp rx=0 tx=0).stop 2023-09-09T23:14:21.175 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.159+0000 7effa88b3b80 1 -- 172.21.15.82:0/1179374793 shutdown_connections 2023-09-09T23:14:21.175 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.159+0000 7effa88b3b80 1 --2- 172.21.15.82:0/1179374793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e3debfe40 0x555e3debd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.175 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.159+0000 7effa88b3b80 1 --2- 172.21.15.82:0/1179374793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e3deb5710 0x555e3debf7f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.175 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.159+0000 7effa88b3b80 1 --2- 172.21.15.82:0/1179374793 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x555e3decf720 0x555e3ded1b40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.176 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.159+0000 7effa88b3b80 1 --2- 172.21.15.82:0/1179374793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e3deb6030 0x555e3debca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.176 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.159+0000 7eff7dffb700 0 client.15891 ms_handle_reset on v2:172.21.15.98:6836/2019449840 2023-09-09T23:14:21.176 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.159+0000 7effa88b3b80 1 -- 172.21.15.82:0/1179374793 >> 172.21.15.82:0/1179374793 conn(0x555e3de2a090 msgr2=0x555e3deb07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:21.176 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.159+0000 7effa88b3b80 1 -- 172.21.15.82:0/1179374793 shutdown_connections 2023-09-09T23:14:21.176 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.159+0000 7effa88b3b80 1 -- 172.21.15.82:0/1179374793 wait complete. 2023-09-09T23:14:21.176 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.159+0000 7effa88b3b80 20 client.15891 trim_cache size 0 max 16384 2023-09-09T23:14:21.181 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.174+0000 7effa88b3b80 1 Processor -- start 2023-09-09T23:14:21.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.174+0000 7effa88b3b80 1 -- start start 2023-09-09T23:14:21.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.175+0000 7effa88b3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e3deb5710 0x555e3ded1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.175+0000 7effa88b3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e3deb6030 0x555e3debf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.175+0000 7effa88b3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e3debfe40 0x555e3debc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.175+0000 7effa88b3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3debe050 con 0x555e3deb5710 2023-09-09T23:14:21.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.175+0000 7effa88b3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3deb2190 con 0x555e3deb6030 2023-09-09T23:14:21.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.175+0000 7effa88b3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3deb8650 con 0x555e3debfe40 2023-09-09T23:14:21.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.175+0000 7eff9a489700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e3deb5710 0x555e3ded1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.175+0000 7eff9a489700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e3deb5710 0x555e3ded1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57356/0 (socket says 172.21.15.82:57356) 2023-09-09T23:14:21.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.175+0000 7eff9a489700 1 -- 172.21.15.82:0/4074662108 learned_addr learned my addr 172.21.15.82:0/4074662108 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:21.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.175+0000 7eff9a489700 1 -- 172.21.15.82:0/4074662108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e3debfe40 msgr2=0x555e3debc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:21.184 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.175+0000 7eff9a489700 1 --2- 172.21.15.82:0/4074662108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e3debfe40 0x555e3debc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.184 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.175+0000 7eff9a489700 1 -- 172.21.15.82:0/4074662108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e3deb6030 msgr2=0x555e3debf390 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-09T23:14:21.184 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.175+0000 7eff9a489700 1 --2- 172.21.15.82:0/4074662108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e3deb6030 0x555e3debf390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.184 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.175+0000 7eff9a489700 1 -- 172.21.15.82:0/4074662108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e3dc1cc60 con 0x555e3deb5710 2023-09-09T23:14:21.184 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.175+0000 7eff9a489700 1 --2- 172.21.15.82:0/4074662108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e3deb5710 0x555e3ded1b10 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7eff9000b2f0 tx=0x7eff90010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f985f6b42dfb7fc3 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:21.184 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.175+0000 7eff897fa700 1 -- 172.21.15.82:0/4074662108 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff90014050 con 0x555e3deb5710 2023-09-09T23:14:21.185 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.175+0000 7eff897fa700 1 -- 172.21.15.82:0/4074662108 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555e3dd5f4f0 con 0x555e3deb5710 2023-09-09T23:14:21.185 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.175+0000 7eff897fa700 1 -- 172.21.15.82:0/4074662108 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff900a0e60 con 0x555e3deb5710 2023-09-09T23:14:21.185 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.176+0000 7effa88b3b80 1 -- 172.21.15.82:0/4074662108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e3deb5710 msgr2=0x555e3ded1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:21.185 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.176+0000 7effa88b3b80 1 --2- 172.21.15.82:0/4074662108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e3deb5710 0x555e3ded1b10 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7eff9000b2f0 tx=0x7eff90010cd0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.186 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.177+0000 7effa88b3b80 1 -- 172.21.15.82:0/4074662108 shutdown_connections 2023-09-09T23:14:21.186 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.177+0000 7effa88b3b80 1 --2- 172.21.15.82:0/4074662108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e3debfe40 0x555e3debc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.186 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.177+0000 7effa88b3b80 1 --2- 172.21.15.82:0/4074662108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e3deb5710 0x555e3ded1b10 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.186 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.177+0000 7effa88b3b80 1 --2- 172.21.15.82:0/4074662108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e3deb6030 0x555e3debf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.186 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.177+0000 7effa88b3b80 1 -- 172.21.15.82:0/4074662108 >> 172.21.15.82:0/4074662108 conn(0x555e3de2a090 msgr2=0x555e3decbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:21.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.177+0000 7effa88b3b80 1 -- 172.21.15.82:0/4074662108 shutdown_connections 2023-09-09T23:14:21.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.177+0000 7effa88b3b80 1 -- 172.21.15.82:0/4074662108 wait complete. 2023-09-09T23:14:21.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.177+0000 7effa88b3b80 1 -- start start 2023-09-09T23:14:21.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.177+0000 7effa88b3b80 1 Processor -- start 2023-09-09T23:14:21.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.178+0000 7effa88b3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e3deb5710 0x555e3dec69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.178+0000 7effa88b3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e3deb6030 0x555e3dec4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.178+0000 7effa88b3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e3debfe40 0x555e3dec52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.178+0000 7effa88b3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3dec59c0 con 0x555e3debfe40 2023-09-09T23:14:21.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.178+0000 7effa88b3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3dec5b30 con 0x555e3deb5710 2023-09-09T23:14:21.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.178+0000 7effa88b3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3df23580 con 0x555e3deb6030 2023-09-09T23:14:21.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.178+0000 7effa88b3b80 10 client.0 _ll_register_callbacks cb 0x555e3dcefa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:14:21.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.178+0000 7eff9ac8a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e3debfe40 0x555e3dec52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.178+0000 7eff9ac8a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e3debfe40 0x555e3dec52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57366/0 (socket says 172.21.15.82:57366) 2023-09-09T23:14:21.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.178+0000 7eff9ac8a700 1 -- 172.21.15.82:0/1850160509 learned_addr learned my addr 172.21.15.82:0/1850160509 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:21.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.178+0000 7eff9ac8a700 1 -- 172.21.15.82:0/1850160509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e3deb6030 msgr2=0x555e3dec4d10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-09T23:14:21.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.178+0000 7eff9ac8a700 1 --2- 172.21.15.82:0/1850160509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e3deb6030 0x555e3dec4d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.179+0000 7eff9ac8a700 1 -- 172.21.15.82:0/1850160509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e3deb5710 msgr2=0x555e3dec69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:21.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.179+0000 7eff9ac8a700 1 --2- 172.21.15.82:0/1850160509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e3deb5710 0x555e3dec69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.179+0000 7eff9ac8a700 1 -- 172.21.15.82:0/1850160509 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff9410cc30 con 0x555e3debfe40 2023-09-09T23:14:21.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.179+0000 7eff9ac8a700 1 --2- 172.21.15.82:0/1850160509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e3debfe40 0x555e3dec52b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0095c0 tx=0x7eff8c0079a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0773eac678dda19 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:21.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.179+0000 7eff7dffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:14:21.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.179+0000 7eff7dffb700 1 -- 172.21.15.82:0/1850160509 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c00e040 con 0x555e3debfe40 2023-09-09T23:14:21.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.179+0000 7eff7dffb700 1 -- 172.21.15.82:0/1850160509 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7eff8c003a00 con 0x555e3debfe40 2023-09-09T23:14:21.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.179+0000 7effa88b3b80 1 -- 172.21.15.82:0/1850160509 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff8c0074e0 con 0x555e3debfe40 2023-09-09T23:14:21.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.179+0000 7eff7dffb700 1 -- 172.21.15.82:0/1850160509 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff8c003bc0 con 0x555e3debfe40 2023-09-09T23:14:21.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.179+0000 7effa88b3b80 10 client.15897 fetch_fsmap learned FSMap version 28 2023-09-09T23:14:21.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.179+0000 7effa88b3b80 1 -- 172.21.15.82:0/1850160509 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555e3dc5a250 con 0x555e3debfe40 2023-09-09T23:14:21.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.179+0000 7eff7dffb700 1 -- 172.21.15.82:0/1850160509 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff8c0074e0 con 0x555e3debfe40 2023-09-09T23:14:21.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.179+0000 7eff7dffb700 1 -- 172.21.15.82:0/1850160509 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7eff8c00b100 con 0x555e3debfe40 2023-09-09T23:14:21.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.179+0000 7effa88b3b80 10 client.15897 fetch_fsmap finished waiting for FSMap version 28 2023-09-09T23:14:21.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.179+0000 7effa88b3b80 10 client.15897 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-09-09T23:14:21.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.179+0000 7effa88b3b80 20 client.15897 populate_metadata read hostname 'smithi082' 2023-09-09T23:14:21.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.179+0000 7effa88b3b80 1 --2- 172.21.15.82:0/1850160509 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x555e3df247f0 0x555e3df26c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.179+0000 7effa88b3b80 4 client.15897 mds_command: new command op to 24542 tid=0[{"prefix": "damage ls"}] 2023-09-09T23:14:21.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.179+0000 7effa88b3b80 1 -- 172.21.15.82:0/1850160509 --> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555e3de00920 con 0x555e3df247f0 2023-09-09T23:14:21.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.180+0000 7eff9a489700 1 --2- 172.21.15.82:0/1850160509 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x555e3df247f0 0x555e3df26c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.181+0000 7eff7dffb700 10 client.15897 ms_handle_connect on v2:172.21.15.98:6836/2019449840 2023-09-09T23:14:21.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.181+0000 7eff9a489700 1 --2- 172.21.15.82:0/1850160509 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x555e3df247f0 0x555e3df26c10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1fd3a32fffaf1e2c server_cookie=b6730e979dbc86a1 in_seq=0 out_seq=0 2023-09-09T23:14:21.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.181+0000 7eff7dffb700 1 -- 172.21.15.82:0/1850160509 <== mds.1 v2:172.21.15.98:6836/2019449840 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7eff90063020 con 0x555e3df247f0 2023-09-09T23:14:21.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.181+0000 7eff7dffb700 10 client.15897 handle_command_reply: tid=0 2023-09-09T23:14:21.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.181+0000 7effa88b3b80 1 client.15897 shutdown 2023-09-09T23:14:21.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.181+0000 7effa88b3b80 1 -- 172.21.15.82:0/1850160509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e3debfe40 msgr2=0x555e3dec52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:21.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.181+0000 7effa88b3b80 1 --2- 172.21.15.82:0/1850160509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e3debfe40 0x555e3dec52b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0095c0 tx=0x7eff8c0079a0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.181+0000 7effa88b3b80 1 -- 172.21.15.82:0/1850160509 shutdown_connections 2023-09-09T23:14:21.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.181+0000 7effa88b3b80 1 --2- 172.21.15.82:0/1850160509 >> [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] conn(0x555e3df247f0 0x555e3df26c10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.181+0000 7effa88b3b80 1 --2- 172.21.15.82:0/1850160509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555e3deb5710 0x555e3dec69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.181+0000 7eff7dffb700 0 client.15897 ms_handle_reset on v2:172.21.15.98:6836/2019449840 2023-09-09T23:14:21.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.181+0000 7effa88b3b80 1 --2- 172.21.15.82:0/1850160509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e3debfe40 0x555e3dec52b0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.181+0000 7effa88b3b80 1 --2- 172.21.15.82:0/1850160509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555e3deb6030 0x555e3dec4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.181+0000 7effa88b3b80 1 -- 172.21.15.82:0/1850160509 >> 172.21.15.82:0/1850160509 conn(0x555e3de2a090 msgr2=0x555e3deb4700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:21.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.181+0000 7effa88b3b80 1 -- 172.21.15.82:0/1850160509 shutdown_connections 2023-09-09T23:14:21.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.181+0000 7effa88b3b80 1 -- 172.21.15.82:0/1850160509 wait complete. 2023-09-09T23:14:21.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.182+0000 7effa88b3b80 20 client.15897 trim_cache size 0 max 16384 2023-09-09T23:14:21.196 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2023-09-09T23:14:21.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.182+0000 7eff9c6ed700 1 -- 172.21.15.82:0/812172498 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7eff800a08d0 msgr2=0x7eff800a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:21.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.182+0000 7eff9c6ed700 1 --2- 172.21.15.82:0/812172498 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7eff800a08d0 0x7eff800a2d90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7eff94069050 tx=0x7eff84009260 comp rx=0 tx=0).stop 2023-09-09T23:14:21.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.182+0000 7eff9c6ed700 1 -- 172.21.15.82:0/812172498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff940ff4b0 msgr2=0x7eff94195320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:21.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.182+0000 7eff9c6ed700 1 --2- 172.21.15.82:0/812172498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff940ff4b0 0x7eff94195320 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7eff9000d950 tx=0x7eff9000dc60 comp rx=0 tx=0).stop 2023-09-09T23:14:21.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.182+0000 7eff9c6ed700 1 -- 172.21.15.82:0/812172498 shutdown_connections 2023-09-09T23:14:21.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.182+0000 7eff9c6ed700 1 --2- 172.21.15.82:0/812172498 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7eff800a08d0 0x7eff800a2d90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.182+0000 7eff9c6ed700 1 --2- 172.21.15.82:0/812172498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff940ff4b0 0x7eff94195320 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.182+0000 7eff9c6ed700 1 --2- 172.21.15.82:0/812172498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff94105570 0x7eff94195da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.182+0000 7eff9c6ed700 1 --2- 172.21.15.82:0/812172498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff940ffe70 0x7eff94195860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.182+0000 7eff9c6ed700 1 -- 172.21.15.82:0/812172498 >> 172.21.15.82:0/812172498 conn(0x7eff940fafa0 msgr2=0x7eff940fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:21.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.182+0000 7eff9c6ed700 1 -- 172.21.15.82:0/812172498 shutdown_connections 2023-09-09T23:14:21.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.182+0000 7eff9c6ed700 1 -- 172.21.15.82:0/812172498 wait complete. 2023-09-09T23:14:21.199 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-09T23:14:21.224 INFO:teuthology.orchestra.run.smithi082.stderr:5000000 of 50000000 tuples (10%) done (elapsed 7.81 s, remaining 70.28 s) 2023-09-09T23:14:21.328 INFO:teuthology.orchestra.run.smithi082.stderr:5100000 of 50000000 tuples (10%) done (elapsed 7.91 s, remaining 69.66 s) 2023-09-09T23:14:21.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.335+0000 7fb68b751700 1 -- 172.21.15.82:0/221158713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb684100590 msgr2=0x7fb684100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:21.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.335+0000 7fb68b751700 1 --2- 172.21.15.82:0/221158713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb684100590 0x7fb684100a10 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb6740099c0 tx=0x7fb674009cd0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.334 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.335+0000 7fb68b751700 1 -- 172.21.15.82:0/221158713 shutdown_connections 2023-09-09T23:14:21.334 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.335+0000 7fb68b751700 1 --2- 172.21.15.82:0/221158713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb684100f50 0x7fb68410d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.334 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.335+0000 7fb68b751700 1 --2- 172.21.15.82:0/221158713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb684106650 0x7fb684106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.334 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.335+0000 7fb68b751700 1 --2- 172.21.15.82:0/221158713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb684100590 0x7fb684100a10 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.334 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.335+0000 7fb68b751700 1 -- 172.21.15.82:0/221158713 >> 172.21.15.82:0/221158713 conn(0x7fb6840fc060 msgr2=0x7fb6840fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:21.335 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.335+0000 7fb68b751700 1 -- 172.21.15.82:0/221158713 shutdown_connections 2023-09-09T23:14:21.335 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.335+0000 7fb68b751700 1 -- 172.21.15.82:0/221158713 wait complete. 2023-09-09T23:14:21.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.335+0000 7fb68b751700 1 Processor -- start 2023-09-09T23:14:21.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.336+0000 7fb68b751700 1 -- start start 2023-09-09T23:14:21.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.336+0000 7fb68b751700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb684100590 0x7fb6841965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.336+0000 7fb68b751700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb684100f50 0x7fb684196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.336+0000 7fb68b751700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb684106650 0x7fb68419af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.336+0000 7fb68b751700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68419b5d0 con 0x7fb684100f50 2023-09-09T23:14:21.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.336+0000 7fb68b751700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68419b740 con 0x7fb684106650 2023-09-09T23:14:21.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.336+0000 7fb68b751700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68419b900 con 0x7fb684100590 2023-09-09T23:14:21.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.336+0000 7fb688cec700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb684100f50 0x7fb684196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.336+0000 7fb688cec700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb684100f50 0x7fb684196b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57376/0 (socket says 172.21.15.82:57376) 2023-09-09T23:14:21.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.336+0000 7fb688cec700 1 -- 172.21.15.82:0/4060471980 learned_addr learned my addr 172.21.15.82:0/4060471980 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:21.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.336+0000 7fb6894ed700 1 --2- 172.21.15.82:0/4060471980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb684100590 0x7fb6841965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.336+0000 7fb689cee700 1 --2- 172.21.15.82:0/4060471980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb684106650 0x7fb68419af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.336+0000 7fb688cec700 1 -- 172.21.15.82:0/4060471980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb684100590 msgr2=0x7fb6841965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:21.340 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.336+0000 7fb688cec700 1 --2- 172.21.15.82:0/4060471980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb684100590 0x7fb6841965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.340 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.336+0000 7fb688cec700 1 -- 172.21.15.82:0/4060471980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb684106650 msgr2=0x7fb68419af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:21.340 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.336+0000 7fb688cec700 1 --2- 172.21.15.82:0/4060471980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb684106650 0x7fb68419af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.340 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.336+0000 7fb688cec700 1 -- 172.21.15.82:0/4060471980 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6740095f0 con 0x7fb684100f50 2023-09-09T23:14:21.340 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.336+0000 7fb689cee700 1 --2- 172.21.15.82:0/4060471980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb684106650 0x7fb68419af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:14:21.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.336+0000 7fb688cec700 1 --2- 172.21.15.82:0/4060471980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb684100f50 0x7fb684196b30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb67400b580 tx=0x7fb674005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:21.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.337+0000 7fb67a7fc700 1 -- 172.21.15.82:0/4060471980 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67401d070 con 0x7fb684100f50 2023-09-09T23:14:21.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.337+0000 7fb67a7fc700 1 -- 172.21.15.82:0/4060471980 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb674003f90 con 0x7fb684100f50 2023-09-09T23:14:21.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.337+0000 7fb68b751700 1 -- 172.21.15.82:0/4060471980 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb68419bb80 con 0x7fb684100f50 2023-09-09T23:14:21.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.337+0000 7fb67a7fc700 1 -- 172.21.15.82:0/4060471980 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb674022d20 con 0x7fb684100f50 2023-09-09T23:14:21.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.337+0000 7fb68b751700 1 -- 172.21.15.82:0/4060471980 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6841a3420 con 0x7fb684100f50 2023-09-09T23:14:21.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.339+0000 7fb67a7fc700 1 -- 172.21.15.82:0/4060471980 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135492+0+0 (secure 0 0 0) 0x7fb674022470 con 0x7fb684100f50 2023-09-09T23:14:21.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.339+0000 7fb67a7fc700 1 --2- 172.21.15.82:0/4060471980 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fb6700a08d0 0x7fb6700a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.339+0000 7fb67a7fc700 1 -- 172.21.15.82:0/4060471980 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fb6740cd5b0 con 0x7fb684100f50 2023-09-09T23:14:21.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.340+0000 7fb6894ed700 1 --2- 172.21.15.82:0/4060471980 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fb6700a08d0 0x7fb6700a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.340+0000 7fb6894ed700 1 --2- 172.21.15.82:0/4060471980 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fb6700a08d0 0x7fb6700a2d90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb68000abd0 tx=0x7fb68000a380 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:21.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.341+0000 7fb697917b80 1 Processor -- start 2023-09-09T23:14:21.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.341+0000 7fb697917b80 1 -- start start 2023-09-09T23:14:21.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.341+0000 7fb697917b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460c317710 0x56460c317af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.341+0000 7fb697917b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56460c318030 0x56460c325d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.341+0000 7fb697917b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460c326250 0x56460c328640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.341+0000 7fb697917b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56460c270730 con 0x56460c326250 2023-09-09T23:14:21.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.341+0000 7fb697917b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56460c26e050 con 0x56460c318030 2023-09-09T23:14:21.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.341+0000 7fb697917b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56460c269940 con 0x56460c317710 2023-09-09T23:14:21.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.341+0000 7fb6894ed700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460c317710 0x56460c317af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.341+0000 7fb689cee700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460c326250 0x56460c328640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.341+0000 7fb6894ed700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460c317710 0x56460c317af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:55898/0 (socket says 172.21.15.82:55898) 2023-09-09T23:14:21.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.341+0000 7fb6894ed700 1 -- 172.21.15.82:0/4255224141 learned_addr learned my addr 172.21.15.82:0/4255224141 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:21.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.341+0000 7fb689cee700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460c326250 0x56460c328640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57388/0 (socket says 172.21.15.82:57388) 2023-09-09T23:14:21.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.341+0000 7fb688cec700 1 --2- 172.21.15.82:0/4255224141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56460c318030 0x56460c325d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.341+0000 7fb6894ed700 1 -- 172.21.15.82:0/4255224141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56460c318030 msgr2=0x56460c325d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:21.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.341+0000 7fb6894ed700 1 --2- 172.21.15.82:0/4255224141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56460c318030 0x56460c325d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.341+0000 7fb6894ed700 1 -- 172.21.15.82:0/4255224141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460c326250 msgr2=0x56460c328640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:21.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.341+0000 7fb6894ed700 1 --2- 172.21.15.82:0/4255224141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460c326250 0x56460c328640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.341+0000 7fb6894ed700 1 -- 172.21.15.82:0/4255224141 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56460c095de0 con 0x56460c317710 2023-09-09T23:14:21.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.341+0000 7fb689cee700 1 --2- 172.21.15.82:0/4255224141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460c326250 0x56460c328640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:14:21.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.342+0000 7fb6894ed700 1 --2- 172.21.15.82:0/4255224141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460c317710 0x56460c317af0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb680004530 tx=0x7fb680012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24ab7c9e5a578895 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:21.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.342+0000 7fb66f7fe700 1 -- 172.21.15.82:0/4255224141 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb680003a90 con 0x56460c317710 2023-09-09T23:14:21.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.342+0000 7fb66f7fe700 1 -- 172.21.15.82:0/4255224141 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb680003bf0 con 0x56460c317710 2023-09-09T23:14:21.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.342+0000 7fb66f7fe700 1 -- 172.21.15.82:0/4255224141 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68001d660 con 0x56460c317710 2023-09-09T23:14:21.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.342+0000 7fb697917b80 1 -- 172.21.15.82:0/4255224141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460c317710 msgr2=0x56460c317af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:21.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.342+0000 7fb697917b80 1 --2- 172.21.15.82:0/4255224141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460c317710 0x56460c317af0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb680004530 tx=0x7fb680012000 comp rx=0 tx=0).stop 2023-09-09T23:14:21.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.342+0000 7fb697917b80 1 -- 172.21.15.82:0/4255224141 shutdown_connections 2023-09-09T23:14:21.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.342+0000 7fb697917b80 1 --2- 172.21.15.82:0/4255224141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460c326250 0x56460c328640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.342+0000 7fb697917b80 1 --2- 172.21.15.82:0/4255224141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460c317710 0x56460c317af0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.342+0000 7fb697917b80 1 --2- 172.21.15.82:0/4255224141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56460c318030 0x56460c325d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.342+0000 7fb697917b80 1 -- 172.21.15.82:0/4255224141 >> 172.21.15.82:0/4255224141 conn(0x56460c28c090 msgr2=0x56460c312bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:21.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.343+0000 7fb697917b80 1 -- 172.21.15.82:0/4255224141 shutdown_connections 2023-09-09T23:14:21.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.343+0000 7fb697917b80 1 -- 172.21.15.82:0/4255224141 wait complete. 2023-09-09T23:14:21.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.343+0000 7fb697917b80 1 -- start start 2023-09-09T23:14:21.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.343+0000 7fb697917b80 1 Processor -- start 2023-09-09T23:14:21.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.343+0000 7fb697917b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56460c317710 0x56460c325c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.343+0000 7fb697917b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460c318030 0x56460c322f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.343+0000 7fb697917b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460c326250 0x56460c323580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.343+0000 7fb697917b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56460c207b70 con 0x56460c318030 2023-09-09T23:14:21.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.343+0000 7fb697917b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56460c207870 con 0x56460c317710 2023-09-09T23:14:21.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.343+0000 7fb697917b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56460c0eb920 con 0x56460c326250 2023-09-09T23:14:21.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.343+0000 7fb697917b80 10 client.0 _ll_register_callbacks cb 0x56460c151a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:14:21.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.343+0000 7fb688cec700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460c318030 0x56460c322f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.343+0000 7fb688cec700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460c318030 0x56460c322f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57392/0 (socket says 172.21.15.82:57392) 2023-09-09T23:14:21.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.343+0000 7fb688cec700 1 -- 172.21.15.82:0/3534468261 learned_addr learned my addr 172.21.15.82:0/3534468261 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:21.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.344+0000 7fb6894ed700 1 --2- 172.21.15.82:0/3534468261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56460c317710 0x56460c325c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.344+0000 7fb689cee700 1 --2- 172.21.15.82:0/3534468261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460c326250 0x56460c323580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.344+0000 7fb688cec700 1 -- 172.21.15.82:0/3534468261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460c326250 msgr2=0x56460c323580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:21.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.344+0000 7fb688cec700 1 --2- 172.21.15.82:0/3534468261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460c326250 0x56460c323580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.344+0000 7fb688cec700 1 -- 172.21.15.82:0/3534468261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56460c317710 msgr2=0x56460c325c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:21.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.344+0000 7fb688cec700 1 --2- 172.21.15.82:0/3534468261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56460c317710 0x56460c325c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.344+0000 7fb688cec700 1 -- 172.21.15.82:0/3534468261 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb67c0098c0 con 0x56460c318030 2023-09-09T23:14:21.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.344+0000 7fb688cec700 1 --2- 172.21.15.82:0/3534468261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460c318030 0x56460c322f20 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb674097380 tx=0x7fb67409ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be5d2628b188c6c server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:21.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.344+0000 7fb66cff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:14:21.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.344+0000 7fb66cff9700 1 -- 172.21.15.82:0/3534468261 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6740a2660 con 0x56460c318030 2023-09-09T23:14:21.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.344+0000 7fb66cff9700 1 -- 172.21.15.82:0/3534468261 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fb67409b010 con 0x56460c318030 2023-09-09T23:14:21.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.344+0000 7fb697917b80 1 -- 172.21.15.82:0/3534468261 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56460c1c14f0 con 0x56460c318030 2023-09-09T23:14:21.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.344+0000 7fb66cff9700 1 -- 172.21.15.82:0/3534468261 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb67409a910 con 0x56460c318030 2023-09-09T23:14:21.360 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.344+0000 7fb697917b80 10 client.15909 fetch_fsmap learned FSMap version 28 2023-09-09T23:14:21.360 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.344+0000 7fb697917b80 1 -- 172.21.15.82:0/3534468261 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fb6800041e0 con 0x56460c318030 2023-09-09T23:14:21.360 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.344+0000 7fb66cff9700 1 -- 172.21.15.82:0/3534468261 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6740b8020 con 0x56460c318030 2023-09-09T23:14:21.360 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.345+0000 7fb66cff9700 1 -- 172.21.15.82:0/3534468261 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fb674060b60 con 0x56460c318030 2023-09-09T23:14:21.360 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.345+0000 7fb697917b80 10 client.15909 fetch_fsmap finished waiting for FSMap version 28 2023-09-09T23:14:21.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.345+0000 7fb697917b80 10 client.15909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:14:21.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.345+0000 7fb697917b80 20 client.15909 populate_metadata read hostname 'smithi082' 2023-09-09T23:14:21.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.345+0000 7fb697917b80 1 --2- 172.21.15.82:0/3534468261 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x56460c335a60 0x56460c337e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.345+0000 7fb697917b80 4 client.15909 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:14:21.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.345+0000 7fb697917b80 1 -- 172.21.15.82:0/3534468261 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56460c27e8a0 con 0x56460c335a60 2023-09-09T23:14:21.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.345+0000 7fb6894ed700 1 --2- 172.21.15.82:0/3534468261 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x56460c335a60 0x56460c337e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.346+0000 7fb6894ed700 1 --2- 172.21.15.82:0/3534468261 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x56460c335a60 0x56460c337e80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85da3c2ddde3027e server_cookie=78cf53100370f6ca in_seq=0 out_seq=0 2023-09-09T23:14:21.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.346+0000 7fb66cff9700 10 client.15909 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:14:21.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.346+0000 7fb66cff9700 1 -- 172.21.15.82:0/3534468261 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56460c27e8a0 con 0x56460c335a60 2023-09-09T23:14:21.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.346+0000 7fb66cff9700 10 client.15909 handle_command_reply: tid=0 2023-09-09T23:14:21.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.346+0000 7fb697917b80 1 client.15909 shutdown 2023-09-09T23:14:21.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.346+0000 7fb697917b80 1 -- 172.21.15.82:0/3534468261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460c318030 msgr2=0x56460c322f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:21.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.346+0000 7fb697917b80 1 --2- 172.21.15.82:0/3534468261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460c318030 0x56460c322f20 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb674097380 tx=0x7fb67409ae50 comp rx=0 tx=0).stop 2023-09-09T23:14:21.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.347+0000 7fb697917b80 1 -- 172.21.15.82:0/3534468261 shutdown_connections 2023-09-09T23:14:21.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.347+0000 7fb697917b80 1 --2- 172.21.15.82:0/3534468261 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x56460c335a60 0x56460c337e80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.347+0000 7fb697917b80 1 --2- 172.21.15.82:0/3534468261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56460c317710 0x56460c325c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.347+0000 7fb66cff9700 0 client.15909 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:14:21.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.347+0000 7fb697917b80 1 --2- 172.21.15.82:0/3534468261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460c326250 0x56460c323580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.347+0000 7fb697917b80 1 --2- 172.21.15.82:0/3534468261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460c318030 0x56460c322f20 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.347+0000 7fb697917b80 1 -- 172.21.15.82:0/3534468261 >> 172.21.15.82:0/3534468261 conn(0x56460c28c090 msgr2=0x56460c3127e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:21.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.347+0000 7fb697917b80 1 -- 172.21.15.82:0/3534468261 shutdown_connections 2023-09-09T23:14:21.365 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.347+0000 7fb697917b80 1 -- 172.21.15.82:0/3534468261 wait complete. 2023-09-09T23:14:21.365 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.347+0000 7fb697917b80 20 client.15909 trim_cache size 0 max 16384 2023-09-09T23:14:21.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.360+0000 7fb697917b80 1 Processor -- start 2023-09-09T23:14:21.370 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.360+0000 7fb697917b80 1 -- start start 2023-09-09T23:14:21.370 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.360+0000 7fb697917b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460c317710 0x56460c322730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.370 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.360+0000 7fb697917b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56460c318030 0x56460c331550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.370 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.360+0000 7fb697917b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460c326250 0x56460c31e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.370 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.361+0000 7fb697917b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56460c328b60 con 0x56460c317710 2023-09-09T23:14:21.371 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.361+0000 7fb697917b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56460c324500 con 0x56460c318030 2023-09-09T23:14:21.371 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.361+0000 7fb697917b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56460c314280 con 0x56460c326250 2023-09-09T23:14:21.371 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.361+0000 7fb6894ed700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460c317710 0x56460c322730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.371 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.361+0000 7fb6894ed700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460c317710 0x56460c322730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57398/0 (socket says 172.21.15.82:57398) 2023-09-09T23:14:21.371 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.361+0000 7fb6894ed700 1 -- 172.21.15.82:0/3928531959 learned_addr learned my addr 172.21.15.82:0/3928531959 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:21.371 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.361+0000 7fb6894ed700 1 -- 172.21.15.82:0/3928531959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460c326250 msgr2=0x56460c31e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:21.372 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.361+0000 7fb6894ed700 1 --2- 172.21.15.82:0/3928531959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460c326250 0x56460c31e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.372 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.361+0000 7fb6894ed700 1 -- 172.21.15.82:0/3928531959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56460c318030 msgr2=0x56460c331550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:21.372 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.361+0000 7fb6894ed700 1 --2- 172.21.15.82:0/3928531959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56460c318030 0x56460c331550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.372 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.361+0000 7fb6894ed700 1 -- 172.21.15.82:0/3928531959 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56460c07ec60 con 0x56460c317710 2023-09-09T23:14:21.372 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.361+0000 7fb6894ed700 1 --2- 172.21.15.82:0/3928531959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460c317710 0x56460c322730 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb680010da0 tx=0x7fb680010d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35a875fcb76249 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:21.373 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.361+0000 7fb66ffff700 1 -- 172.21.15.82:0/3928531959 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68002b040 con 0x56460c317710 2023-09-09T23:14:21.373 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.361+0000 7fb66ffff700 1 -- 172.21.15.82:0/3928531959 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb68002c040 con 0x56460c317710 2023-09-09T23:14:21.373 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.361+0000 7fb66ffff700 1 -- 172.21.15.82:0/3928531959 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb680010430 con 0x56460c317710 2023-09-09T23:14:21.373 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.362+0000 7fb697917b80 1 -- 172.21.15.82:0/3928531959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460c317710 msgr2=0x56460c322730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:21.373 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.362+0000 7fb697917b80 1 --2- 172.21.15.82:0/3928531959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460c317710 0x56460c322730 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb680010da0 tx=0x7fb680010d30 comp rx=0 tx=0).stop 2023-09-09T23:14:21.373 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.362+0000 7fb697917b80 1 -- 172.21.15.82:0/3928531959 shutdown_connections 2023-09-09T23:14:21.374 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.362+0000 7fb697917b80 1 --2- 172.21.15.82:0/3928531959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460c326250 0x56460c31e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.374 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.362+0000 7fb697917b80 1 --2- 172.21.15.82:0/3928531959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460c317710 0x56460c322730 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.374 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.362+0000 7fb697917b80 1 --2- 172.21.15.82:0/3928531959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56460c318030 0x56460c331550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.374 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.362+0000 7fb697917b80 1 -- 172.21.15.82:0/3928531959 >> 172.21.15.82:0/3928531959 conn(0x56460c28c090 msgr2=0x56460c311320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:21.374 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.362+0000 7fb697917b80 1 -- 172.21.15.82:0/3928531959 shutdown_connections 2023-09-09T23:14:21.375 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.362+0000 7fb697917b80 1 -- 172.21.15.82:0/3928531959 wait complete. 2023-09-09T23:14:21.375 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.362+0000 7fb697917b80 1 -- start start 2023-09-09T23:14:21.375 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.363+0000 7fb697917b80 1 Processor -- start 2023-09-09T23:14:21.375 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.363+0000 7fb697917b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56460c317710 0x56460c32cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.375 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.363+0000 7fb697917b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460c318030 0x56460c32b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.375 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.363+0000 7fb697917b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460c326250 0x56460c32b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.363+0000 7fb697917b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56460c32c090 con 0x56460c318030 2023-09-09T23:14:21.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.363+0000 7fb697917b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56460c3898a0 con 0x56460c317710 2023-09-09T23:14:21.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.363+0000 7fb697917b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56460c3899e0 con 0x56460c326250 2023-09-09T23:14:21.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.363+0000 7fb697917b80 10 client.0 _ll_register_callbacks cb 0x56460c151a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:14:21.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.363+0000 7fb688cec700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460c318030 0x56460c32b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.363+0000 7fb688cec700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460c318030 0x56460c32b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57406/0 (socket says 172.21.15.82:57406) 2023-09-09T23:14:21.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.363+0000 7fb688cec700 1 -- 172.21.15.82:0/263001077 learned_addr learned my addr 172.21.15.82:0/263001077 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:21.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.363+0000 7fb6894ed700 1 --2- 172.21.15.82:0/263001077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56460c317710 0x56460c32cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.363+0000 7fb688cec700 1 -- 172.21.15.82:0/263001077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460c326250 msgr2=0x56460c32b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:21.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.363+0000 7fb688cec700 1 --2- 172.21.15.82:0/263001077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460c326250 0x56460c32b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.363+0000 7fb688cec700 1 -- 172.21.15.82:0/263001077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56460c317710 msgr2=0x56460c32cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:21.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.363+0000 7fb688cec700 1 --2- 172.21.15.82:0/263001077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56460c317710 0x56460c32cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.363+0000 7fb688cec700 1 -- 172.21.15.82:0/263001077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56460c095de0 con 0x56460c318030 2023-09-09T23:14:21.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.364+0000 7fb6894ed700 1 --2- 172.21.15.82:0/263001077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56460c317710 0x56460c32cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:14:21.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.364+0000 7fb688cec700 1 --2- 172.21.15.82:0/263001077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460c318030 0x56460c32b280 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb6740cb370 tx=0x7fb6740a27f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d396d0e356905db server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:21.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.364+0000 7fb66cff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:14:21.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.364+0000 7fb66cff9700 1 -- 172.21.15.82:0/263001077 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6740cab30 con 0x56460c318030 2023-09-09T23:14:21.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.364+0000 7fb697917b80 1 -- 172.21.15.82:0/263001077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb680003bf0 con 0x56460c318030 2023-09-09T23:14:21.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.364+0000 7fb66cff9700 1 -- 172.21.15.82:0/263001077 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fb67409b010 con 0x56460c318030 2023-09-09T23:14:21.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.364+0000 7fb66cff9700 1 -- 172.21.15.82:0/263001077 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb67409a910 con 0x56460c318030 2023-09-09T23:14:21.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.364+0000 7fb697917b80 10 client.15915 fetch_fsmap learned FSMap version 28 2023-09-09T23:14:21.380 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.364+0000 7fb697917b80 1 -- 172.21.15.82:0/263001077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56460c0bc250 con 0x56460c318030 2023-09-09T23:14:21.380 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.364+0000 7fb66cff9700 1 -- 172.21.15.82:0/263001077 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb680003bf0 con 0x56460c318030 2023-09-09T23:14:21.380 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.364+0000 7fb66cff9700 1 -- 172.21.15.82:0/263001077 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fb674096700 con 0x56460c318030 2023-09-09T23:14:21.380 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.364+0000 7fb697917b80 10 client.15915 fetch_fsmap finished waiting for FSMap version 28 2023-09-09T23:14:21.380 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.364+0000 7fb697917b80 10 client.15915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:14:21.380 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.364+0000 7fb697917b80 20 client.15915 populate_metadata read hostname 'smithi082' 2023-09-09T23:14:21.381 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.364+0000 7fb697917b80 1 --2- 172.21.15.82:0/263001077 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x56460c38ac90 0x56460c38d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:21.381 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.364+0000 7fb697917b80 4 client.15915 mds_command: new command op to 24566 tid=0[{"prefix": "damage ls"}] 2023-09-09T23:14:21.381 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.364+0000 7fb697917b80 1 -- 172.21.15.82:0/263001077 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56460c262920 con 0x56460c38ac90 2023-09-09T23:14:21.381 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.365+0000 7fb6894ed700 1 --2- 172.21.15.82:0/263001077 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x56460c38ac90 0x56460c38d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:21.381 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.365+0000 7fb66cff9700 10 client.15915 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:14:21.382 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.365+0000 7fb6894ed700 1 --2- 172.21.15.82:0/263001077 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x56460c38ac90 0x56460c38d0b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c9aadfacbbfd5ad server_cookie=a1103284588ad88e in_seq=0 out_seq=0 2023-09-09T23:14:21.382 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.365+0000 7fb66cff9700 1 -- 172.21.15.82:0/263001077 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56460c262920 con 0x56460c38ac90 2023-09-09T23:14:21.382 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.365+0000 7fb66cff9700 10 client.15915 handle_command_reply: tid=0 2023-09-09T23:14:21.382 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.365+0000 7fb697917b80 1 client.15915 shutdown 2023-09-09T23:14:21.382 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.365+0000 7fb697917b80 1 -- 172.21.15.82:0/263001077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460c318030 msgr2=0x56460c32b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:21.383 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.365+0000 7fb697917b80 1 --2- 172.21.15.82:0/263001077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460c318030 0x56460c32b280 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb6740cb370 tx=0x7fb6740a27f0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.383 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.365+0000 7fb697917b80 1 -- 172.21.15.82:0/263001077 shutdown_connections 2023-09-09T23:14:21.383 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.365+0000 7fb697917b80 1 --2- 172.21.15.82:0/263001077 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x56460c38ac90 0x56460c38d0b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.383 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.365+0000 7fb697917b80 1 --2- 172.21.15.82:0/263001077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56460c317710 0x56460c32cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.383 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.365+0000 7fb697917b80 1 --2- 172.21.15.82:0/263001077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460c326250 0x56460c32b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.384 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.365+0000 7fb66cff9700 0 client.15915 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:14:21.384 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.365+0000 7fb697917b80 1 --2- 172.21.15.82:0/263001077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460c318030 0x56460c32b280 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.384 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.365+0000 7fb697917b80 1 -- 172.21.15.82:0/263001077 >> 172.21.15.82:0/263001077 conn(0x56460c28c090 msgr2=0x56460c331550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:21.384 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.365+0000 7fb697917b80 1 -- 172.21.15.82:0/263001077 shutdown_connections 2023-09-09T23:14:21.384 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.366+0000 7fb697917b80 1 -- 172.21.15.82:0/263001077 wait complete. 2023-09-09T23:14:21.384 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.366+0000 7fb697917b80 20 client.15915 trim_cache size 0 max 16384 2023-09-09T23:14:21.385 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2023-09-09T23:14:21.386 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.366+0000 7fb68b751700 1 -- 172.21.15.82:0/4060471980 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fb6700a08d0 msgr2=0x7fb6700a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:21.386 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.366+0000 7fb68b751700 1 --2- 172.21.15.82:0/4060471980 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fb6700a08d0 0x7fb6700a2d90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb68000abd0 tx=0x7fb68000a380 comp rx=0 tx=0).stop 2023-09-09T23:14:21.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.366+0000 7fb68b751700 1 -- 172.21.15.82:0/4060471980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb684100f50 msgr2=0x7fb684196b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:21.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.366+0000 7fb68b751700 1 --2- 172.21.15.82:0/4060471980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb684100f50 0x7fb684196b30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb67400b580 tx=0x7fb674005740 comp rx=0 tx=0).stop 2023-09-09T23:14:21.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.366+0000 7fb68b751700 1 -- 172.21.15.82:0/4060471980 shutdown_connections 2023-09-09T23:14:21.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.366+0000 7fb68b751700 1 --2- 172.21.15.82:0/4060471980 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fb6700a08d0 0x7fb6700a2d90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.388 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.366+0000 7fb68b751700 1 --2- 172.21.15.82:0/4060471980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb684106650 0x7fb68419af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.388 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.366+0000 7fb68b751700 1 --2- 172.21.15.82:0/4060471980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb684100590 0x7fb6841965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.388 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.366+0000 7fb68b751700 1 --2- 172.21.15.82:0/4060471980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb684100f50 0x7fb684196b30 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:21.388 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.366+0000 7fb68b751700 1 -- 172.21.15.82:0/4060471980 >> 172.21.15.82:0/4060471980 conn(0x7fb6840fc060 msgr2=0x7fb6840fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:21.388 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.366+0000 7fb68b751700 1 -- 172.21.15.82:0/4060471980 shutdown_connections 2023-09-09T23:14:21.389 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:21.366+0000 7fb68b751700 1 -- 172.21.15.82:0/4060471980 wait complete. 2023-09-09T23:14:21.454 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:21 smithi082 ceph-mon[98694]: pgmap v16: 97 pgs: 97 active+clean; 236 MiB data, 929 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 32 MiB/s wr, 159 op/s 2023-09-09T23:14:21.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:21 smithi082 ceph-mon[98694]: from='client.? 172.21.15.82:0/3932962369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:14:21.493 INFO:teuthology.orchestra.run.smithi082.stderr:5200000 of 50000000 tuples (10%) done (elapsed 8.08 s, remaining 69.59 s) 2023-09-09T23:14:21.582 INFO:teuthology.orchestra.run.smithi082.stderr:5300000 of 50000000 tuples (10%) done (elapsed 8.17 s, remaining 68.88 s) 2023-09-09T23:14:21.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:21 smithi170 ceph-mon[102832]: pgmap v16: 97 pgs: 97 active+clean; 236 MiB data, 929 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 32 MiB/s wr, 159 op/s 2023-09-09T23:14:21.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:21 smithi170 ceph-mon[102832]: from='client.? 172.21.15.82:0/3932962369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:14:21.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:21 smithi098 ceph-mon[103124]: pgmap v16: 97 pgs: 97 active+clean; 236 MiB data, 929 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 32 MiB/s wr, 159 op/s 2023-09-09T23:14:21.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:21 smithi098 ceph-mon[103124]: from='client.? 172.21.15.82:0/3932962369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-09T23:14:21.780 INFO:teuthology.orchestra.run.smithi082.stderr:5400000 of 50000000 tuples (10%) done (elapsed 8.36 s, remaining 69.08 s) 2023-09-09T23:14:21.924 INFO:teuthology.orchestra.run.smithi082.stderr:5500000 of 50000000 tuples (11%) done (elapsed 8.51 s, remaining 68.84 s) 2023-09-09T23:14:22.020 INFO:teuthology.orchestra.run.smithi082.stderr:5600000 of 50000000 tuples (11%) done (elapsed 8.60 s, remaining 68.22 s) 2023-09-09T23:14:22.194 INFO:teuthology.orchestra.run.smithi082.stderr:5700000 of 50000000 tuples (11%) done (elapsed 8.78 s, remaining 68.22 s) 2023-09-09T23:14:22.329 INFO:teuthology.orchestra.run.smithi082.stderr:5800000 of 50000000 tuples (11%) done (elapsed 8.91 s, remaining 67.93 s) 2023-09-09T23:14:22.389 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-09T23:14:22.390 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-09T23:14:22.425 INFO:teuthology.orchestra.run.smithi082.stderr:5900000 of 50000000 tuples (11%) done (elapsed 9.01 s, remaining 67.34 s) 2023-09-09T23:14:22.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.652+0000 7fe37581b700 1 -- 172.21.15.82:0/1665383158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe370119de0 msgr2=0x7fe37011c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:22.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.652+0000 7fe37581b700 1 --2- 172.21.15.82:0/1665383158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe370119de0 0x7fe37011c1d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe36401a9a0 tx=0x7fe36401acb0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.652+0000 7fe37581b700 1 -- 172.21.15.82:0/1665383158 shutdown_connections 2023-09-09T23:14:22.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.652+0000 7fe37581b700 1 --2- 172.21.15.82:0/1665383158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe370119de0 0x7fe37011c1d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.652+0000 7fe37581b700 1 --2- 172.21.15.82:0/1665383158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe37010ecb0 0x7fe37010c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.652+0000 7fe37581b700 1 --2- 172.21.15.82:0/1665383158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe37010f120 0x7fe37010cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.652+0000 7fe37581b700 1 -- 172.21.15.82:0/1665383158 >> 172.21.15.82:0/1665383158 conn(0x7fe37006c8c0 msgr2=0x7fe37006ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:22.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.664+0000 7fe37581b700 1 -- 172.21.15.82:0/1665383158 shutdown_connections 2023-09-09T23:14:22.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.666+0000 7fe37581b700 1 -- 172.21.15.82:0/1665383158 wait complete. 2023-09-09T23:14:22.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.667+0000 7fe37581b700 1 Processor -- start 2023-09-09T23:14:22.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.667+0000 7fe37581b700 1 -- start start 2023-09-09T23:14:22.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.667+0000 7fe37581b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe37010ecb0 0x7fe3701151d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:22.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.667+0000 7fe37581b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe37010f120 0x7fe370115710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:22.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.667+0000 7fe37581b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe370119de0 0x7fe3701caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:22.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.667+0000 7fe37581b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3701cb080 con 0x7fe37010ecb0 2023-09-09T23:14:22.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.667+0000 7fe37581b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3701cb1f0 con 0x7fe370119de0 2023-09-09T23:14:22.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.667+0000 7fe37581b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3701cb3b0 con 0x7fe37010f120 2023-09-09T23:14:22.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.668+0000 7fe36f7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe370119de0 0x7fe3701caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:22.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.668+0000 7fe36f7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe370119de0 0x7fe3701caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:42986/0 (socket says 172.21.15.82:42986) 2023-09-09T23:14:22.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.668+0000 7fe36f7fe700 1 -- 172.21.15.82:0/4081819501 learned_addr learned my addr 172.21.15.82:0/4081819501 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:22.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.669+0000 7fe36e7fc700 1 --2- 172.21.15.82:0/4081819501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe37010f120 0x7fe370115710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:22.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.670+0000 7fe36effd700 1 --2- 172.21.15.82:0/4081819501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe37010ecb0 0x7fe3701151d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:22.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.672+0000 7fe36e7fc700 1 -- 172.21.15.82:0/4081819501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe370119de0 msgr2=0x7fe3701caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:22.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.672+0000 7fe36e7fc700 1 --2- 172.21.15.82:0/4081819501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe370119de0 0x7fe3701caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.672+0000 7fe36e7fc700 1 -- 172.21.15.82:0/4081819501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe37010ecb0 msgr2=0x7fe3701151d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:22.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.672+0000 7fe36e7fc700 1 --2- 172.21.15.82:0/4081819501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe37010ecb0 0x7fe3701151d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.672+0000 7fe36e7fc700 1 -- 172.21.15.82:0/4081819501 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36401a600 con 0x7fe37010f120 2023-09-09T23:14:22.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.674+0000 7fe36e7fc700 1 --2- 172.21.15.82:0/4081819501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe37010f120 0x7fe370115710 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe36800bb80 tx=0x7fe36800bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:22.678 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.676+0000 7fe374819700 1 -- 172.21.15.82:0/4081819501 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36800cb60 con 0x7fe37010f120 2023-09-09T23:14:22.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.676+0000 7fe374819700 1 -- 172.21.15.82:0/4081819501 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe36800ccc0 con 0x7fe37010f120 2023-09-09T23:14:22.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.676+0000 7fe374819700 1 -- 172.21.15.82:0/4081819501 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe368012650 con 0x7fe37010f120 2023-09-09T23:14:22.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.676+0000 7fe37581b700 1 -- 172.21.15.82:0/4081819501 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3701cb690 con 0x7fe37010f120 2023-09-09T23:14:22.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.676+0000 7fe37581b700 1 -- 172.21.15.82:0/4081819501 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3701cbb50 con 0x7fe37010f120 2023-09-09T23:14:22.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.692+0000 7fe3819e1b80 1 Processor -- start 2023-09-09T23:14:22.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.693+0000 7fe3819e1b80 1 -- start start 2023-09-09T23:14:22.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.694+0000 7fe3819e1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f3d93c790 0x559f3d93cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:22.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.694+0000 7fe3819e1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f3d93d0b0 0x559f3d946980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:22.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.694+0000 7fe3819e1b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f3d946ec0 0x559f3d9492b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:22.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.694+0000 7fe3819e1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3d8957b0 con 0x559f3d93d0b0 2023-09-09T23:14:22.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.694+0000 7fe3819e1b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3d8930d0 con 0x559f3d946ec0 2023-09-09T23:14:22.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.694+0000 7fe3819e1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3d88e9c0 con 0x559f3d93c790 2023-09-09T23:14:22.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.694+0000 7fe374819700 1 -- 172.21.15.82:0/4081819501 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 24) v1 ==== 135492+0+0 (secure 0 0 0) 0x7fe368014440 con 0x7fe37010f120 2023-09-09T23:14:22.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.695+0000 7fe374819700 1 --2- 172.21.15.82:0/4081819501 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fe3580a09d0 0x7fe3580a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:22.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.695+0000 7fe36e7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f3d946ec0 0x559f3d9492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:22.696 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.695+0000 7fe36f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f3d93c790 0x559f3d93cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:22.696 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.695+0000 7fe36e7fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f3d946ec0 0x559f3d9492b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:42992/0 (socket says 172.21.15.82:42992) 2023-09-09T23:14:22.696 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.695+0000 7fe36e7fc700 1 -- 172.21.15.82:0/855915518 learned_addr learned my addr 172.21.15.82:0/855915518 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:22.696 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.695+0000 7fe36effd700 1 --2- 172.21.15.82:0/855915518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f3d93d0b0 0x559f3d946980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:22.697 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.696+0000 7fe374819700 1 -- 172.21.15.82:0/4081819501 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fe3680cb8b0 con 0x7fe37010f120 2023-09-09T23:14:22.698 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.696+0000 7fe36f7fe700 1 --2- 172.21.15.82:0/4081819501 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fe3580a09d0 0x7fe3580a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:22.698 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.696+0000 7fe36effd700 1 -- 172.21.15.82:0/855915518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f3d93c790 msgr2=0x559f3d93cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:22.698 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.697+0000 7fe36effd700 1 --2- 172.21.15.82:0/855915518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f3d93c790 0x559f3d93cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.698 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.697+0000 7fe36effd700 1 -- 172.21.15.82:0/855915518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f3d946ec0 msgr2=0x559f3d9492b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:22.698 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.697+0000 7fe36effd700 1 --2- 172.21.15.82:0/855915518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f3d946ec0 0x559f3d9492b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.699 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.697+0000 7fe36effd700 1 -- 172.21.15.82:0/855915518 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f3d6bae60 con 0x559f3d93d0b0 2023-09-09T23:14:22.699 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.697+0000 7fe36effd700 1 --2- 172.21.15.82:0/855915518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f3d93d0b0 0x559f3d946980 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe360007750 tx=0x7fe360007a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522caf911ec34dcd server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:22.699 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.697+0000 7fe36f7fe700 1 --2- 172.21.15.82:0/4081819501 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fe3580a09d0 0x7fe3580a2e90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe364005cd0 tx=0x7fe3640057f0 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:22.699 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.699+0000 7fe355ffb700 1 -- 172.21.15.82:0/855915518 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe360011590 con 0x559f3d93d0b0 2023-09-09T23:14:22.699 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.699+0000 7fe355ffb700 1 -- 172.21.15.82:0/855915518 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe360011bb0 con 0x559f3d93d0b0 2023-09-09T23:14:22.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.699+0000 7fe355ffb700 1 -- 172.21.15.82:0/855915518 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe360012bc0 con 0x559f3d93d0b0 2023-09-09T23:14:22.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.700+0000 7fe3819e1b80 1 -- 172.21.15.82:0/855915518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f3d93d0b0 msgr2=0x559f3d946980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:22.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.700+0000 7fe3819e1b80 1 --2- 172.21.15.82:0/855915518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f3d93d0b0 0x559f3d946980 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe360007750 tx=0x7fe360007a60 comp rx=0 tx=0).stop 2023-09-09T23:14:22.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.700+0000 7fe3819e1b80 1 -- 172.21.15.82:0/855915518 shutdown_connections 2023-09-09T23:14:22.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.700+0000 7fe3819e1b80 1 --2- 172.21.15.82:0/855915518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f3d946ec0 0x559f3d9492b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.700+0000 7fe3819e1b80 1 --2- 172.21.15.82:0/855915518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f3d93c790 0x559f3d93cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.700+0000 7fe3819e1b80 1 --2- 172.21.15.82:0/855915518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f3d93d0b0 0x559f3d946980 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.700+0000 7fe3819e1b80 1 -- 172.21.15.82:0/855915518 >> 172.21.15.82:0/855915518 conn(0x559f3d8b1110 msgr2=0x559f3d937c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:22.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.701+0000 7fe3819e1b80 1 -- 172.21.15.82:0/855915518 shutdown_connections 2023-09-09T23:14:22.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.701+0000 7fe3819e1b80 1 -- 172.21.15.82:0/855915518 wait complete. 2023-09-09T23:14:22.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.701+0000 7fe3819e1b80 1 -- start start 2023-09-09T23:14:22.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.702+0000 7fe3819e1b80 1 Processor -- start 2023-09-09T23:14:22.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.703+0000 7fe3819e1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f3d93c790 0x559f3d9468c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:22.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.703+0000 7fe3819e1b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f3d93d0b0 0x559f3d943ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:22.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.703+0000 7fe3819e1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f3d946ec0 0x559f3d9440e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:22.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.703+0000 7fe3819e1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3d82cbf0 con 0x559f3d946ec0 2023-09-09T23:14:22.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.703+0000 7fe3819e1b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3d82c8f0 con 0x559f3d93d0b0 2023-09-09T23:14:22.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.703+0000 7fe3819e1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3d7109a0 con 0x559f3d93c790 2023-09-09T23:14:22.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.703+0000 7fe3819e1b80 10 client.0 _ll_register_callbacks cb 0x559f3d776b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:14:22.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.703+0000 7fe36f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f3d946ec0 0x559f3d9440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:22.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.703+0000 7fe36f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f3d946ec0 0x559f3d9440e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57434/0 (socket says 172.21.15.82:57434) 2023-09-09T23:14:22.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.703+0000 7fe36f7fe700 1 -- 172.21.15.82:0/1037884854 learned_addr learned my addr 172.21.15.82:0/1037884854 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:22.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.704+0000 7fe36e7fc700 1 --2- 172.21.15.82:0/1037884854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f3d93d0b0 0x559f3d943ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:22.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.704+0000 7fe36f7fe700 1 -- 172.21.15.82:0/1037884854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f3d93c790 msgr2=0x559f3d9468c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-09T23:14:22.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.704+0000 7fe36f7fe700 1 --2- 172.21.15.82:0/1037884854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f3d93c790 0x559f3d9468c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.704+0000 7fe36f7fe700 1 -- 172.21.15.82:0/1037884854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f3d93d0b0 msgr2=0x559f3d943ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:22.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.704+0000 7fe36f7fe700 1 --2- 172.21.15.82:0/1037884854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f3d93d0b0 0x559f3d943ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.704+0000 7fe36f7fe700 1 -- 172.21.15.82:0/1037884854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36000e040 con 0x559f3d946ec0 2023-09-09T23:14:22.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.704+0000 7fe36f7fe700 1 --2- 172.21.15.82:0/1037884854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f3d946ec0 0x559f3d9440e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe364005bf0 tx=0x7fe364005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0ff4ba04d94376 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:22.709 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.705+0000 7fe34effd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:14:22.709 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.705+0000 7fe34effd700 1 -- 172.21.15.82:0/1037884854 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe364019070 con 0x559f3d946ec0 2023-09-09T23:14:22.709 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.705+0000 7fe3819e1b80 1 -- 172.21.15.82:0/1037884854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559f3d7e6570 con 0x559f3d946ec0 2023-09-09T23:14:22.709 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.705+0000 7fe34effd700 1 -- 172.21.15.82:0/1037884854 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fe364011820 con 0x559f3d946ec0 2023-09-09T23:14:22.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.705+0000 7fe34effd700 1 -- 172.21.15.82:0/1037884854 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3640119e0 con 0x559f3d946ec0 2023-09-09T23:14:22.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.706+0000 7fe3819e1b80 10 client.15924 fetch_fsmap learned FSMap version 29 2023-09-09T23:14:22.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.706+0000 7fe3819e1b80 1 -- 172.21.15.82:0/1037884854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559f3d6a3ce0 con 0x559f3d946ec0 2023-09-09T23:14:22.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.709+0000 7fe34effd700 1 -- 172.21.15.82:0/1037884854 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559f3d7e6570 con 0x559f3d946ec0 2023-09-09T23:14:22.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.711+0000 7fe34effd700 1 -- 172.21.15.82:0/1037884854 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe364013400 con 0x559f3d946ec0 2023-09-09T23:14:22.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.711+0000 7fe3819e1b80 10 client.15924 fetch_fsmap finished waiting for FSMap version 29 2023-09-09T23:14:22.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.711+0000 7fe3819e1b80 10 client.15924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:14:22.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.711+0000 7fe3819e1b80 20 client.15924 populate_metadata read hostname 'smithi082' 2023-09-09T23:14:22.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.711+0000 7fe3819e1b80 1 --2- 172.21.15.82:0/1037884854 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x559f3d9569c0 0x559f3d958de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:22.713 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.711+0000 7fe3819e1b80 4 client.15924 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:14:22.713 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.711+0000 7fe3819e1b80 1 -- 172.21.15.82:0/1037884854 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f3d8a3920 con 0x559f3d9569c0 2023-09-09T23:14:22.713 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.713+0000 7fe36effd700 1 --2- 172.21.15.82:0/1037884854 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x559f3d9569c0 0x559f3d958de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:22.717 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.718+0000 7fe36effd700 1 --2- 172.21.15.82:0/1037884854 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x559f3d9569c0 0x559f3d958de0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff1de5cf0b41744e server_cookie=819979d17fb5bb8a in_seq=0 out_seq=0 2023-09-09T23:14:22.718 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.718+0000 7fe34effd700 10 client.15924 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:14:22.719 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.721+0000 7fe34effd700 1 -- 172.21.15.82:0/1037884854 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559f3d8a3920 con 0x559f3d9569c0 2023-09-09T23:14:22.720 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.721+0000 7fe34effd700 10 client.15924 handle_command_reply: tid=0 2023-09-09T23:14:22.720 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.721+0000 7fe3819e1b80 1 client.15924 shutdown 2023-09-09T23:14:22.721 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.721+0000 7fe3819e1b80 1 -- 172.21.15.82:0/1037884854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f3d946ec0 msgr2=0x559f3d9440e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:22.721 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.721+0000 7fe3819e1b80 1 --2- 172.21.15.82:0/1037884854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f3d946ec0 0x559f3d9440e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe364005bf0 tx=0x7fe364005c60 comp rx=0 tx=0).stop 2023-09-09T23:14:22.722 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.721+0000 7fe3819e1b80 1 -- 172.21.15.82:0/1037884854 shutdown_connections 2023-09-09T23:14:22.722 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.721+0000 7fe3819e1b80 1 --2- 172.21.15.82:0/1037884854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f3d946ec0 0x559f3d9440e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.723 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.721+0000 7fe3819e1b80 1 --2- 172.21.15.82:0/1037884854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f3d93c790 0x559f3d9468c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.723 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.721+0000 7fe3819e1b80 1 --2- 172.21.15.82:0/1037884854 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x559f3d9569c0 0x559f3d958de0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.723 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.721+0000 7fe3819e1b80 1 --2- 172.21.15.82:0/1037884854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f3d93d0b0 0x559f3d943ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.724 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.721+0000 7fe3819e1b80 1 -- 172.21.15.82:0/1037884854 >> 172.21.15.82:0/1037884854 conn(0x559f3d8b1110 msgr2=0x559f3d937860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:22.724 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.721+0000 7fe34effd700 0 client.15924 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:14:22.726 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.723+0000 7fe3819e1b80 1 -- 172.21.15.82:0/1037884854 shutdown_connections 2023-09-09T23:14:22.726 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.723+0000 7fe3819e1b80 1 -- 172.21.15.82:0/1037884854 wait complete. 2023-09-09T23:14:22.726 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.723+0000 7fe3819e1b80 20 client.15924 trim_cache size 0 max 16384 2023-09-09T23:14:22.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.758+0000 7fe3819e1b80 1 Processor -- start 2023-09-09T23:14:22.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.758+0000 7fe3819e1b80 1 -- start start 2023-09-09T23:14:22.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.758+0000 7fe3819e1b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f3d93c790 0x559f3d94da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:22.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.758+0000 7fe3819e1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f3d93d0b0 0x559f3d93be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:22.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.758+0000 7fe3819e1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f3d946ec0 0x559f3d940d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:22.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.758+0000 7fe3819e1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3d94dff0 con 0x559f3d946ec0 2023-09-09T23:14:22.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.758+0000 7fe3819e1b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3d93d490 con 0x559f3d93c790 2023-09-09T23:14:22.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.758+0000 7fe3819e1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3d942030 con 0x559f3d93d0b0 2023-09-09T23:14:22.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.758+0000 7fe36f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f3d946ec0 0x559f3d940d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:22.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.758+0000 7fe36f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f3d946ec0 0x559f3d940d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57444/0 (socket says 172.21.15.82:57444) 2023-09-09T23:14:22.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.758+0000 7fe36f7fe700 1 -- 172.21.15.82:0/1567638187 learned_addr learned my addr 172.21.15.82:0/1567638187 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:22.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.759+0000 7fe36e7fc700 1 --2- 172.21.15.82:0/1567638187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f3d93d0b0 0x559f3d93be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:22.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.759+0000 7fe36f7fe700 1 -- 172.21.15.82:0/1567638187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f3d93d0b0 msgr2=0x559f3d93be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:22.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.759+0000 7fe36f7fe700 1 --2- 172.21.15.82:0/1567638187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f3d93d0b0 0x559f3d93be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.759+0000 7fe36f7fe700 1 -- 172.21.15.82:0/1567638187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f3d93c790 msgr2=0x559f3d94da70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-09T23:14:22.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.759+0000 7fe36f7fe700 1 --2- 172.21.15.82:0/1567638187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f3d93c790 0x559f3d94da70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.759+0000 7fe36f7fe700 1 -- 172.21.15.82:0/1567638187 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f3d95da50 con 0x559f3d946ec0 2023-09-09T23:14:22.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.759+0000 7fe36f7fe700 1 --2- 172.21.15.82:0/1567638187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f3d946ec0 0x559f3d940d40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe36401c650 tx=0x7fe36401a0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38970449a2d9d8bd server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:22.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.759+0000 7fe3567fc700 1 -- 172.21.15.82:0/1567638187 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe364011d80 con 0x559f3d946ec0 2023-09-09T23:14:22.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.759+0000 7fe3567fc700 1 -- 172.21.15.82:0/1567638187 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe364011ae0 con 0x559f3d946ec0 2023-09-09T23:14:22.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.759+0000 7fe3567fc700 1 -- 172.21.15.82:0/1567638187 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe364013e80 con 0x559f3d946ec0 2023-09-09T23:14:22.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.760+0000 7fe3819e1b80 1 -- 172.21.15.82:0/1567638187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f3d946ec0 msgr2=0x559f3d940d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:22.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.760+0000 7fe3819e1b80 1 --2- 172.21.15.82:0/1567638187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f3d946ec0 0x559f3d940d40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe36401c650 tx=0x7fe36401a0e0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.760+0000 7fe3819e1b80 1 -- 172.21.15.82:0/1567638187 shutdown_connections 2023-09-09T23:14:22.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.760+0000 7fe3819e1b80 1 --2- 172.21.15.82:0/1567638187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f3d93c790 0x559f3d94da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.760+0000 7fe3819e1b80 1 --2- 172.21.15.82:0/1567638187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f3d946ec0 0x559f3d940d40 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.760+0000 7fe3819e1b80 1 --2- 172.21.15.82:0/1567638187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f3d93d0b0 0x559f3d93be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.760+0000 7fe3819e1b80 1 -- 172.21.15.82:0/1567638187 >> 172.21.15.82:0/1567638187 conn(0x559f3d8b1110 msgr2=0x559f3d93b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:22.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.760+0000 7fe3819e1b80 1 -- 172.21.15.82:0/1567638187 shutdown_connections 2023-09-09T23:14:22.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.760+0000 7fe3819e1b80 1 -- 172.21.15.82:0/1567638187 wait complete. 2023-09-09T23:14:22.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.761+0000 7fe3819e1b80 1 -- start start 2023-09-09T23:14:22.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.761+0000 7fe3819e1b80 1 Processor -- start 2023-09-09T23:14:22.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.762+0000 7fe3819e1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f3d93c790 0x559f3d9a4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:22.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.762+0000 7fe3819e1b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f3d93d0b0 0x559f3d9a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:22.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.762+0000 7fe3819e1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f3d946ec0 0x559f3d99ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:22.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.762+0000 7fe3819e1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3d99efe0 con 0x559f3d946ec0 2023-09-09T23:14:22.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.762+0000 7fe3819e1b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3d99f150 con 0x559f3d93d0b0 2023-09-09T23:14:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.762+0000 7fe3819e1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f3d99f310 con 0x559f3d93c790 2023-09-09T23:14:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.762+0000 7fe3819e1b80 10 client.0 _ll_register_callbacks cb 0x559f3d776b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:14:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.762+0000 7fe36f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f3d946ec0 0x559f3d99ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.762+0000 7fe36f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f3d946ec0 0x559f3d99ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57448/0 (socket says 172.21.15.82:57448) 2023-09-09T23:14:22.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.762+0000 7fe36f7fe700 1 -- 172.21.15.82:0/339953043 learned_addr learned my addr 172.21.15.82:0/339953043 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:22.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.762+0000 7fe36f7fe700 1 -- 172.21.15.82:0/339953043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f3d93c790 msgr2=0x559f3d9a4660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-09T23:14:22.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.762+0000 7fe36f7fe700 1 --2- 172.21.15.82:0/339953043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f3d93c790 0x559f3d9a4660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.762+0000 7fe36f7fe700 1 -- 172.21.15.82:0/339953043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f3d93d0b0 msgr2=0x559f3d9a4ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-09T23:14:22.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.762+0000 7fe36f7fe700 1 --2- 172.21.15.82:0/339953043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f3d93d0b0 0x559f3d9a4ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.762+0000 7fe36f7fe700 1 -- 172.21.15.82:0/339953043 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f3d6a3ce0 con 0x559f3d946ec0 2023-09-09T23:14:22.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.763+0000 7fe36f7fe700 1 --2- 172.21.15.82:0/339953043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f3d946ec0 0x559f3d99ea00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe36401d010 tx=0x7fe3640058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c9495314b65fbf server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:22.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.763+0000 7fe34effd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:14:22.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.763+0000 7fe34effd700 1 -- 172.21.15.82:0/339953043 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe364019040 con 0x559f3d946ec0 2023-09-09T23:14:22.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.763+0000 7fe34effd700 1 -- 172.21.15.82:0/339953043 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fe364013a60 con 0x559f3d946ec0 2023-09-09T23:14:22.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.763+0000 7fe3819e1b80 1 -- 172.21.15.82:0/339953043 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe360011bb0 con 0x559f3d946ec0 2023-09-09T23:14:22.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.763+0000 7fe34effd700 1 -- 172.21.15.82:0/339953043 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe364013820 con 0x559f3d946ec0 2023-09-09T23:14:22.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.763+0000 7fe3819e1b80 10 client.15930 fetch_fsmap learned FSMap version 29 2023-09-09T23:14:22.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.763+0000 7fe3819e1b80 1 -- 172.21.15.82:0/339953043 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559f3d6e12d0 con 0x559f3d946ec0 2023-09-09T23:14:22.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.763+0000 7fe34effd700 1 -- 172.21.15.82:0/339953043 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe364052020 con 0x559f3d946ec0 2023-09-09T23:14:22.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.764+0000 7fe34effd700 1 -- 172.21.15.82:0/339953043 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fe36404c070 con 0x559f3d946ec0 2023-09-09T23:14:22.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.764+0000 7fe3819e1b80 10 client.15930 fetch_fsmap finished waiting for FSMap version 29 2023-09-09T23:14:22.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.764+0000 7fe3819e1b80 10 client.15930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:14:22.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.764+0000 7fe3819e1b80 20 client.15930 populate_metadata read hostname 'smithi082' 2023-09-09T23:14:22.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.764+0000 7fe3819e1b80 1 --2- 172.21.15.82:0/339953043 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x559f3d9ac130 0x559f3d9ae550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:22.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.764+0000 7fe3819e1b80 4 client.15930 mds_command: new command op to 24566 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-09T23:14:22.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.764+0000 7fe3819e1b80 1 -- 172.21.15.82:0/339953043 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559f3d8879a0 con 0x559f3d9ac130 2023-09-09T23:14:22.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.764+0000 7fe36effd700 1 --2- 172.21.15.82:0/339953043 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x559f3d9ac130 0x559f3d9ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:22.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.765+0000 7fe36effd700 1 --2- 172.21.15.82:0/339953043 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x559f3d9ac130 0x559f3d9ae550 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ca6f2d65686e85 server_cookie=1223755ae34dd190 in_seq=0 out_seq=0 2023-09-09T23:14:22.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.765+0000 7fe34effd700 10 client.15930 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:14:22.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.766+0000 7fe34effd700 1 -- 172.21.15.82:0/339953043 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559f3d8879a0 con 0x559f3d9ac130 2023-09-09T23:14:22.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.766+0000 7fe34effd700 10 client.15930 handle_command_reply: tid=0 2023-09-09T23:14:22.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.767+0000 7fe3819e1b80 1 client.15930 shutdown 2023-09-09T23:14:22.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.767+0000 7fe3819e1b80 1 -- 172.21.15.82:0/339953043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f3d946ec0 msgr2=0x559f3d99ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:22.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.767+0000 7fe3819e1b80 1 --2- 172.21.15.82:0/339953043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f3d946ec0 0x559f3d99ea00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe36401d010 tx=0x7fe3640058a0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.767+0000 7fe3819e1b80 1 -- 172.21.15.82:0/339953043 shutdown_connections 2023-09-09T23:14:22.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.767+0000 7fe3819e1b80 1 --2- 172.21.15.82:0/339953043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f3d946ec0 0x559f3d99ea00 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.767+0000 7fe3819e1b80 1 --2- 172.21.15.82:0/339953043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559f3d93c790 0x559f3d9a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.767+0000 7fe3819e1b80 1 --2- 172.21.15.82:0/339953043 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x559f3d9ac130 0x559f3d9ae550 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.767+0000 7fe3819e1b80 1 --2- 172.21.15.82:0/339953043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f3d93d0b0 0x559f3d9a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.767+0000 7fe34effd700 0 client.15930 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:14:22.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.767+0000 7fe3819e1b80 1 -- 172.21.15.82:0/339953043 >> 172.21.15.82:0/339953043 conn(0x559f3d8b1110 msgr2=0x559f3d93c010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:22.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.767+0000 7fe3819e1b80 1 -- 172.21.15.82:0/339953043 shutdown_connections 2023-09-09T23:14:22.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.767+0000 7fe3819e1b80 1 -- 172.21.15.82:0/339953043 wait complete. 2023-09-09T23:14:22.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.767+0000 7fe3819e1b80 20 client.15930 trim_cache size 0 max 16384 2023-09-09T23:14:22.777 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:14:22.777 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2023-09-09T23:14:22.777 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:14:22.777 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2023-09-09T23:14:22.778 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:14:22.778 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.768+0000 7fe37581b700 1 -- 172.21.15.82:0/4081819501 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fe3580a09d0 msgr2=0x7fe3580a2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:22.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.768+0000 7fe37581b700 1 --2- 172.21.15.82:0/4081819501 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fe3580a09d0 0x7fe3580a2e90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe364005cd0 tx=0x7fe3640057f0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.768+0000 7fe37581b700 1 -- 172.21.15.82:0/4081819501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe37010f120 msgr2=0x7fe370115710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:22.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.768+0000 7fe37581b700 1 --2- 172.21.15.82:0/4081819501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe37010f120 0x7fe370115710 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe36800bb80 tx=0x7fe36800bf40 comp rx=0 tx=0).stop 2023-09-09T23:14:22.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.768+0000 7fe37581b700 1 -- 172.21.15.82:0/4081819501 shutdown_connections 2023-09-09T23:14:22.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.768+0000 7fe37581b700 1 --2- 172.21.15.82:0/4081819501 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fe3580a09d0 0x7fe3580a2e90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.768+0000 7fe37581b700 1 --2- 172.21.15.82:0/4081819501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe370119de0 0x7fe3701caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.768+0000 7fe37581b700 1 --2- 172.21.15.82:0/4081819501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe37010ecb0 0x7fe3701151d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:22.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.768+0000 7fe37581b700 1 --2- 172.21.15.82:0/4081819501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe37010f120 0x7fe370115710 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe36800bb80 tx=0x7fe36800bf40 comp rx=0 tx=0).stop 2023-09-09T23:14:22.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.768+0000 7fe37581b700 1 -- 172.21.15.82:0/4081819501 >> 172.21.15.82:0/4081819501 conn(0x7fe37006c8c0 msgr2=0x7fe37011b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:22.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.768+0000 7fe37581b700 1 -- 172.21.15.82:0/4081819501 shutdown_connections 2023-09-09T23:14:22.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:22.768+0000 7fe37581b700 1 -- 172.21.15.82:0/4081819501 wait complete. 2023-09-09T23:14:22.781 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:14:23.002 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.003+0000 7fd647a11700 1 -- 172.21.15.82:0/1067342499 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6400ff830 msgr2=0x7fd640066ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:23.003 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.003+0000 7fd647a11700 1 --2- 172.21.15.82:0/1067342499 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6400ff830 0x7fd640066ed0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd6300099c0 tx=0x7fd630009cd0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.003 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.003+0000 7fd647a11700 1 -- 172.21.15.82:0/1067342499 shutdown_connections 2023-09-09T23:14:23.003 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.003+0000 7fd647a11700 1 --2- 172.21.15.82:0/1067342499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd640103870 0x7fd640103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.004 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.003+0000 7fd647a11700 1 --2- 172.21.15.82:0/1067342499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6400fee80 0x7fd6400ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.004 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.003+0000 7fd647a11700 1 --2- 172.21.15.82:0/1067342499 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6400ff830 0x7fd640066ed0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.004 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.003+0000 7fd647a11700 1 -- 172.21.15.82:0/1067342499 >> 172.21.15.82:0/1067342499 conn(0x7fd640076a70 msgr2=0x7fd6400fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:23.004 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.004+0000 7fd647a11700 1 -- 172.21.15.82:0/1067342499 shutdown_connections 2023-09-09T23:14:23.004 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.004+0000 7fd647a11700 1 -- 172.21.15.82:0/1067342499 wait complete. 2023-09-09T23:14:23.005 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.004+0000 7fd647a11700 1 Processor -- start 2023-09-09T23:14:23.005 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.004+0000 7fd647a11700 1 -- start start 2023-09-09T23:14:23.006 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.005+0000 7fd647a11700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6400fee80 0x7fd6401989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:23.007 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.005+0000 7fd647a11700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd640103870 0x7fd640198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:23.007 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.005+0000 7fd647a11700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd64019d380 0x7fd64019d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:23.007 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.005+0000 7fd647a11700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64019dd70 con 0x7fd64019d380 2023-09-09T23:14:23.007 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.005+0000 7fd647a11700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64019dee0 con 0x7fd640103870 2023-09-09T23:14:23.008 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.005+0000 7fd647a11700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64019e0a0 con 0x7fd6400fee80 2023-09-09T23:14:23.008 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.005+0000 7fd645fae700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd64019d380 0x7fd64019d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:23.008 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.005+0000 7fd645fae700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd64019d380 0x7fd64019d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57464/0 (socket says 172.21.15.82:57464) 2023-09-09T23:14:23.008 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.005+0000 7fd645fae700 1 -- 172.21.15.82:0/2777327096 learned_addr learned my addr 172.21.15.82:0/2777327096 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:23.008 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.005+0000 7fd645fae700 1 -- 172.21.15.82:0/2777327096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6400fee80 msgr2=0x7fd6401989b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-09T23:14:23.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.005+0000 7fd645fae700 1 --2- 172.21.15.82:0/2777327096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6400fee80 0x7fd6401989b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.005+0000 7fd645fae700 1 -- 172.21.15.82:0/2777327096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd640103870 msgr2=0x7fd640198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:23.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.005+0000 7fd645fae700 1 --2- 172.21.15.82:0/2777327096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd640103870 0x7fd640198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.005+0000 7fd645fae700 1 -- 172.21.15.82:0/2777327096 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6300095f0 con 0x7fd64019d380 2023-09-09T23:14:23.010 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.005+0000 7fd645fae700 1 --2- 172.21.15.82:0/2777327096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd64019d380 0x7fd64019d830 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fd63800b970 tx=0x7fd63800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:23.010 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.006+0000 7fd6367fc700 1 -- 172.21.15.82:0/2777327096 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63800c9b0 con 0x7fd64019d380 2023-09-09T23:14:23.010 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.006+0000 7fd6367fc700 1 -- 172.21.15.82:0/2777327096 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd638014440 con 0x7fd64019d380 2023-09-09T23:14:23.013 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.006+0000 7fd647a11700 1 -- 172.21.15.82:0/2777327096 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6401a5630 con 0x7fd64019d380 2023-09-09T23:14:23.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.006+0000 7fd6367fc700 1 -- 172.21.15.82:0/2777327096 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6380075d0 con 0x7fd64019d380 2023-09-09T23:14:23.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.006+0000 7fd647a11700 1 -- 172.21.15.82:0/2777327096 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6401a5ac0 con 0x7fd64019d380 2023-09-09T23:14:23.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.008+0000 7fd6367fc700 1 -- 172.21.15.82:0/2777327096 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135492+0+0 (secure 0 0 0) 0x7fd6380145b0 con 0x7fd64019d380 2023-09-09T23:14:23.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.009+0000 7fd6367fc700 1 --2- 172.21.15.82:0/2777327096 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fd62c0a08d0 0x7fd62c0a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:23.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.009+0000 7fd6457ad700 1 --2- 172.21.15.82:0/2777327096 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fd62c0a08d0 0x7fd62c0a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:23.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.009+0000 7fd6367fc700 1 -- 172.21.15.82:0/2777327096 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fd638006e40 con 0x7fd64019d380 2023-09-09T23:14:23.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.010+0000 7fd6457ad700 1 --2- 172.21.15.82:0/2777327096 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fd62c0a08d0 0x7fd62c0a2d90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd63c009710 tx=0x7fd63c006c60 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:23.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.010+0000 7fd653bd7b80 1 Processor -- start 2023-09-09T23:14:23.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.010+0000 7fd653bd7b80 1 -- start start 2023-09-09T23:14:23.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.010+0000 7fd653bd7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e08c86c710 0x55e08c86caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:23.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.010+0000 7fd653bd7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e08c86d030 0x55e08c876900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:23.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.010+0000 7fd653bd7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e08c876e40 0x55e08c879230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:23.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.010+0000 7fd653bd7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e08c7c5730 con 0x55e08c876e40 2023-09-09T23:14:23.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.010+0000 7fd653bd7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e08c7c3050 con 0x55e08c86d030 2023-09-09T23:14:23.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.010+0000 7fd653bd7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e08c7be940 con 0x55e08c86c710 2023-09-09T23:14:23.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.010+0000 7fd645fae700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e08c876e40 0x55e08c879230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:23.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.010+0000 7fd645fae700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e08c876e40 0x55e08c879230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57472/0 (socket says 172.21.15.82:57472) 2023-09-09T23:14:23.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.010+0000 7fd645fae700 1 -- 172.21.15.82:0/244102107 learned_addr learned my addr 172.21.15.82:0/244102107 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:23.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.011+0000 7fd645fae700 1 -- 172.21.15.82:0/244102107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e08c86c710 msgr2=0x55e08c86caf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-09T23:14:23.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.011+0000 7fd645fae700 1 --2- 172.21.15.82:0/244102107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e08c86c710 0x55e08c86caf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.011+0000 7fd645fae700 1 -- 172.21.15.82:0/244102107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e08c86d030 msgr2=0x55e08c876900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:23.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.011+0000 7fd645fae700 1 --2- 172.21.15.82:0/244102107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e08c86d030 0x55e08c876900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.011+0000 7fd645fae700 1 -- 172.21.15.82:0/244102107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e08c5eade0 con 0x55e08c876e40 2023-09-09T23:14:23.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.011+0000 7fd645fae700 1 --2- 172.21.15.82:0/244102107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e08c876e40 0x55e08c879230 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd63800be50 tx=0x7fd638099380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db308efd35387f1a server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:23.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.011+0000 7fd62b7fe700 1 -- 172.21.15.82:0/244102107 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63809ca30 con 0x55e08c876e40 2023-09-09T23:14:23.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.011+0000 7fd62b7fe700 1 -- 172.21.15.82:0/244102107 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6380a0c70 con 0x55e08c876e40 2023-09-09T23:14:23.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.011+0000 7fd62b7fe700 1 -- 172.21.15.82:0/244102107 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6380982f0 con 0x55e08c876e40 2023-09-09T23:14:23.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.012+0000 7fd653bd7b80 1 -- 172.21.15.82:0/244102107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e08c876e40 msgr2=0x55e08c879230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:23.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.012+0000 7fd653bd7b80 1 --2- 172.21.15.82:0/244102107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e08c876e40 0x55e08c879230 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd63800be50 tx=0x7fd638099380 comp rx=0 tx=0).stop 2023-09-09T23:14:23.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.012+0000 7fd653bd7b80 1 -- 172.21.15.82:0/244102107 shutdown_connections 2023-09-09T23:14:23.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.012+0000 7fd653bd7b80 1 --2- 172.21.15.82:0/244102107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e08c876e40 0x55e08c879230 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.012+0000 7fd653bd7b80 1 --2- 172.21.15.82:0/244102107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e08c86c710 0x55e08c86caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.012+0000 7fd653bd7b80 1 --2- 172.21.15.82:0/244102107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e08c86d030 0x55e08c876900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.012+0000 7fd653bd7b80 1 -- 172.21.15.82:0/244102107 >> 172.21.15.82:0/244102107 conn(0x55e08c7e1090 msgr2=0x55e08c867bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:23.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.012+0000 7fd653bd7b80 1 -- 172.21.15.82:0/244102107 shutdown_connections 2023-09-09T23:14:23.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.012+0000 7fd653bd7b80 1 -- 172.21.15.82:0/244102107 wait complete. 2023-09-09T23:14:23.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.012+0000 7fd653bd7b80 1 -- start start 2023-09-09T23:14:23.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.012+0000 7fd653bd7b80 1 Processor -- start 2023-09-09T23:14:23.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.013+0000 7fd653bd7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e08c86c710 0x55e08c873c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:23.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.013+0000 7fd653bd7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e08c86d030 0x55e08c874140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:23.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.013+0000 7fd653bd7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e08c876e40 0x55e08c874680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:23.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.013+0000 7fd653bd7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e08c75cb70 con 0x55e08c86d030 2023-09-09T23:14:23.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.013+0000 7fd653bd7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e08c75c870 con 0x55e08c876e40 2023-09-09T23:14:23.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.013+0000 7fd653bd7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e08c640920 con 0x55e08c86c710 2023-09-09T23:14:23.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.013+0000 7fd653bd7b80 10 client.0 _ll_register_callbacks cb 0x55e08c6a6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:14:23.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.013+0000 7fd644fac700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e08c86d030 0x55e08c874140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:23.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.013+0000 7fd644fac700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e08c86d030 0x55e08c874140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57476/0 (socket says 172.21.15.82:57476) 2023-09-09T23:14:23.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.013+0000 7fd644fac700 1 -- 172.21.15.82:0/1400520671 learned_addr learned my addr 172.21.15.82:0/1400520671 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:23.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.013+0000 7fd644fac700 1 -- 172.21.15.82:0/1400520671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e08c86c710 msgr2=0x55e08c873c00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-09T23:14:23.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.013+0000 7fd644fac700 1 --2- 172.21.15.82:0/1400520671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e08c86c710 0x55e08c873c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.014+0000 7fd644fac700 1 -- 172.21.15.82:0/1400520671 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e08c876e40 msgr2=0x55e08c874680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-09T23:14:23.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.014+0000 7fd644fac700 1 --2- 172.21.15.82:0/1400520671 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e08c876e40 0x55e08c874680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.014+0000 7fd644fac700 1 -- 172.21.15.82:0/1400520671 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6401a5ac0 con 0x55e08c86d030 2023-09-09T23:14:23.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.014+0000 7fd644fac700 1 --2- 172.21.15.82:0/1400520671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e08c86d030 0x55e08c874140 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd6300099c0 tx=0x7fd63000ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50479adc58f2f22f server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:23.029 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.014+0000 7fd628ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:14:23.029 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.014+0000 7fd628ff9700 1 -- 172.21.15.82:0/1400520671 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63001d070 con 0x55e08c86d030 2023-09-09T23:14:23.029 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.014+0000 7fd628ff9700 1 -- 172.21.15.82:0/1400520671 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fd63000be20 con 0x55e08c86d030 2023-09-09T23:14:23.029 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.014+0000 7fd653bd7b80 1 -- 172.21.15.82:0/1400520671 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e08c7164f0 con 0x55e08c86d030 2023-09-09T23:14:23.029 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.014+0000 7fd628ff9700 1 -- 172.21.15.82:0/1400520671 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6300038d0 con 0x55e08c86d030 2023-09-09T23:14:23.030 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.014+0000 7fd653bd7b80 10 client.15942 fetch_fsmap learned FSMap version 29 2023-09-09T23:14:23.030 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.014+0000 7fd653bd7b80 1 -- 172.21.15.82:0/1400520671 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e08c5d3c60 con 0x55e08c86d030 2023-09-09T23:14:23.030 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.015+0000 7fd628ff9700 1 -- 172.21.15.82:0/1400520671 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e08c7164f0 con 0x55e08c86d030 2023-09-09T23:14:23.030 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.015+0000 7fd628ff9700 1 -- 172.21.15.82:0/1400520671 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd63000f980 con 0x55e08c86d030 2023-09-09T23:14:23.030 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.015+0000 7fd653bd7b80 10 client.15942 fetch_fsmap finished waiting for FSMap version 29 2023-09-09T23:14:23.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.015+0000 7fd653bd7b80 10 client.15942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:14:23.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.015+0000 7fd653bd7b80 20 client.15942 populate_metadata read hostname 'smithi082' 2023-09-09T23:14:23.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.015+0000 7fd653bd7b80 1 --2- 172.21.15.82:0/1400520671 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55e08c897ae0 0x55e08c899f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:23.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.015+0000 7fd653bd7b80 4 client.15942 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:14:23.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.015+0000 7fd653bd7b80 1 -- 172.21.15.82:0/1400520671 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e08c7d38a0 con 0x55e08c897ae0 2023-09-09T23:14:23.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.016+0000 7fd6457ad700 1 --2- 172.21.15.82:0/1400520671 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55e08c897ae0 0x55e08c899f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:23.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.020+0000 7fd628ff9700 10 client.15942 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:14:23.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.020+0000 7fd6457ad700 1 --2- 172.21.15.82:0/1400520671 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55e08c897ae0 0x55e08c899f00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aee8e87f3e4ef250 server_cookie=f50513bcec953919 in_seq=0 out_seq=0 2023-09-09T23:14:23.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.023+0000 7fd628ff9700 1 -- 172.21.15.82:0/1400520671 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e08c7d38a0 con 0x55e08c897ae0 2023-09-09T23:14:23.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.023+0000 7fd628ff9700 10 client.15942 handle_command_reply: tid=0 2023-09-09T23:14:23.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.023+0000 7fd653bd7b80 1 client.15942 shutdown 2023-09-09T23:14:23.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.023+0000 7fd653bd7b80 1 -- 172.21.15.82:0/1400520671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e08c86d030 msgr2=0x55e08c874140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:23.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.023+0000 7fd653bd7b80 1 --2- 172.21.15.82:0/1400520671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e08c86d030 0x55e08c874140 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd6300099c0 tx=0x7fd63000ba70 comp rx=0 tx=0).stop 2023-09-09T23:14:23.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.023+0000 7fd653bd7b80 1 -- 172.21.15.82:0/1400520671 shutdown_connections 2023-09-09T23:14:23.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.023+0000 7fd653bd7b80 1 --2- 172.21.15.82:0/1400520671 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55e08c897ae0 0x55e08c899f00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.023+0000 7fd653bd7b80 1 --2- 172.21.15.82:0/1400520671 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e08c876e40 0x55e08c874680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.023+0000 7fd653bd7b80 1 --2- 172.21.15.82:0/1400520671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e08c86c710 0x55e08c873c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.023+0000 7fd653bd7b80 1 --2- 172.21.15.82:0/1400520671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e08c86d030 0x55e08c874140 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.023+0000 7fd653bd7b80 1 -- 172.21.15.82:0/1400520671 >> 172.21.15.82:0/1400520671 conn(0x55e08c7e1090 msgr2=0x55e08c867780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:23.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.023+0000 7fd628ff9700 0 client.15942 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:14:23.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.023+0000 7fd653bd7b80 1 -- 172.21.15.82:0/1400520671 shutdown_connections 2023-09-09T23:14:23.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.023+0000 7fd653bd7b80 1 -- 172.21.15.82:0/1400520671 wait complete. 2023-09-09T23:14:23.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.023+0000 7fd653bd7b80 20 client.15942 trim_cache size 0 max 16384 2023-09-09T23:14:23.054 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.053+0000 7fd653bd7b80 1 Processor -- start 2023-09-09T23:14:23.054 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.053+0000 7fd653bd7b80 1 -- start start 2023-09-09T23:14:23.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.053+0000 7fd653bd7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e08c86c710 0x55e08c88e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:23.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.053+0000 7fd653bd7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e08c86d030 0x55e08c8744b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:23.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.053+0000 7fd653bd7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e08c876e40 0x55e08c870e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:23.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.053+0000 7fd653bd7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e08c88efa0 con 0x55e08c86c710 2023-09-09T23:14:23.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.053+0000 7fd653bd7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e08c86cd40 con 0x55e08c86d030 2023-09-09T23:14:23.056 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.053+0000 7fd653bd7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e08c868e60 con 0x55e08c876e40 2023-09-09T23:14:23.056 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.053+0000 7fd6457ad700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e08c86c710 0x55e08c88e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:23.056 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.054+0000 7fd6457ad700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e08c86c710 0x55e08c88e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57482/0 (socket says 172.21.15.82:57482) 2023-09-09T23:14:23.056 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.054+0000 7fd6457ad700 1 -- 172.21.15.82:0/3307340834 learned_addr learned my addr 172.21.15.82:0/3307340834 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:23.058 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.054+0000 7fd644fac700 1 --2- 172.21.15.82:0/3307340834 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e08c86d030 0x55e08c8744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:23.058 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.054+0000 7fd6457ad700 1 -- 172.21.15.82:0/3307340834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e08c876e40 msgr2=0x55e08c870e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:23.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.054+0000 7fd6457ad700 1 --2- 172.21.15.82:0/3307340834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e08c876e40 0x55e08c870e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.054+0000 7fd6457ad700 1 -- 172.21.15.82:0/3307340834 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e08c86d030 msgr2=0x55e08c8744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:23.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.054+0000 7fd6457ad700 1 --2- 172.21.15.82:0/3307340834 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e08c86d030 0x55e08c8744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.054+0000 7fd6457ad700 1 -- 172.21.15.82:0/3307340834 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e08c8669b0 con 0x55e08c86c710 2023-09-09T23:14:23.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.054+0000 7fd6457ad700 1 --2- 172.21.15.82:0/3307340834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e08c86c710 0x55e08c88e9f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd63c013760 tx=0x7fd63c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f12fc38328de5666 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:23.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.055+0000 7fd62bfff700 1 -- 172.21.15.82:0/3307340834 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c01a040 con 0x55e08c86c710 2023-09-09T23:14:23.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.055+0000 7fd62bfff700 1 -- 172.21.15.82:0/3307340834 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd63c01f5a0 con 0x55e08c86c710 2023-09-09T23:14:23.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.055+0000 7fd62bfff700 1 -- 172.21.15.82:0/3307340834 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c012de0 con 0x55e08c86c710 2023-09-09T23:14:23.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.056+0000 7fd653bd7b80 1 -- 172.21.15.82:0/3307340834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e08c86c710 msgr2=0x55e08c88e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:23.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.056+0000 7fd653bd7b80 1 --2- 172.21.15.82:0/3307340834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e08c86c710 0x55e08c88e9f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd63c013760 tx=0x7fd63c009f90 comp rx=0 tx=0).stop 2023-09-09T23:14:23.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.056+0000 7fd653bd7b80 1 -- 172.21.15.82:0/3307340834 shutdown_connections 2023-09-09T23:14:23.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.056+0000 7fd653bd7b80 1 --2- 172.21.15.82:0/3307340834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e08c876e40 0x55e08c870e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.056+0000 7fd653bd7b80 1 --2- 172.21.15.82:0/3307340834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e08c86c710 0x55e08c88e9f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.056+0000 7fd653bd7b80 1 --2- 172.21.15.82:0/3307340834 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e08c86d030 0x55e08c8744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.056+0000 7fd653bd7b80 1 -- 172.21.15.82:0/3307340834 >> 172.21.15.82:0/3307340834 conn(0x55e08c7e1090 msgr2=0x55e08c86eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:23.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.056+0000 7fd653bd7b80 1 -- 172.21.15.82:0/3307340834 shutdown_connections 2023-09-09T23:14:23.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.056+0000 7fd653bd7b80 1 -- 172.21.15.82:0/3307340834 wait complete. 2023-09-09T23:14:23.063 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.057+0000 7fd653bd7b80 1 -- start start 2023-09-09T23:14:23.063 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.057+0000 7fd653bd7b80 1 Processor -- start 2023-09-09T23:14:23.063 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.058+0000 7fd653bd7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e08c86c710 0x55e08c88e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:23.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.058+0000 7fd653bd7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e08c86d030 0x55e08c88ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:23.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.058+0000 7fd653bd7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e08c876e40 0x55e08c88cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:23.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.058+0000 7fd653bd7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e08c88d500 con 0x55e08c876e40 2023-09-09T23:14:23.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.058+0000 7fd653bd7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e08c88d640 con 0x55e08c86d030 2023-09-09T23:14:23.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.058+0000 7fd653bd7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e08c88d800 con 0x55e08c86c710 2023-09-09T23:14:23.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.059+0000 7fd645fae700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e08c876e40 0x55e08c88cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:23.070 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.059+0000 7fd653bd7b80 10 client.0 _ll_register_callbacks cb 0x7fd630009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:14:23.070 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.059+0000 7fd645fae700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e08c876e40 0x55e08c88cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57490/0 (socket says 172.21.15.82:57490) 2023-09-09T23:14:23.070 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.059+0000 7fd645fae700 1 -- 172.21.15.82:0/2468060169 learned_addr learned my addr 172.21.15.82:0/2468060169 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:23.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.059+0000 7fd645fae700 1 -- 172.21.15.82:0/2468060169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e08c86c710 msgr2=0x55e08c88e710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-09T23:14:23.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.059+0000 7fd645fae700 1 --2- 172.21.15.82:0/2468060169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e08c86c710 0x55e08c88e710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.059+0000 7fd645fae700 1 -- 172.21.15.82:0/2468060169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e08c86d030 msgr2=0x55e08c88ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:23.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.059+0000 7fd645fae700 1 --2- 172.21.15.82:0/2468060169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e08c86d030 0x55e08c88ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.059+0000 7fd645fae700 1 -- 172.21.15.82:0/2468060169 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63c0134a0 con 0x55e08c876e40 2023-09-09T23:14:23.072 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.059+0000 7fd645fae700 1 --2- 172.21.15.82:0/2468060169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e08c876e40 0x55e08c88cf80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd638098b90 tx=0x7fd638099370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c537276a3bd403 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:23.072 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.059+0000 7fd628ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:14:23.072 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.060+0000 7fd628ff9700 1 -- 172.21.15.82:0/2468060169 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6380982f0 con 0x55e08c876e40 2023-09-09T23:14:23.072 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.060+0000 7fd653bd7b80 1 -- 172.21.15.82:0/2468060169 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd6380a0c70 con 0x55e08c876e40 2023-09-09T23:14:23.072 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.060+0000 7fd628ff9700 1 -- 172.21.15.82:0/2468060169 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fd6380a07e0 con 0x55e08c876e40 2023-09-09T23:14:23.073 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.060+0000 7fd628ff9700 1 -- 172.21.15.82:0/2468060169 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6380a09a0 con 0x55e08c876e40 2023-09-09T23:14:23.073 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.060+0000 7fd653bd7b80 10 client.15948 fetch_fsmap learned FSMap version 29 2023-09-09T23:14:23.073 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.060+0000 7fd653bd7b80 1 -- 172.21.15.82:0/2468060169 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e08c611250 con 0x55e08c876e40 2023-09-09T23:14:23.073 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.060+0000 7fd628ff9700 1 -- 172.21.15.82:0/2468060169 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6380a0c70 con 0x55e08c876e40 2023-09-09T23:14:23.073 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.060+0000 7fd628ff9700 1 -- 172.21.15.82:0/2468060169 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd638094b80 con 0x55e08c876e40 2023-09-09T23:14:23.074 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.060+0000 7fd653bd7b80 10 client.15948 fetch_fsmap finished waiting for FSMap version 29 2023-09-09T23:14:23.074 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.061+0000 7fd653bd7b80 10 client.15948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:14:23.074 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.061+0000 7fd653bd7b80 20 client.15948 populate_metadata read hostname 'smithi082' 2023-09-09T23:14:23.074 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.061+0000 7fd653bd7b80 1 --2- 172.21.15.82:0/2468060169 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55e08c8ecc30 0x55e08c8ef050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:23.074 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.061+0000 7fd653bd7b80 4 client.15948 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:14:23.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.061+0000 7fd653bd7b80 1 -- 172.21.15.82:0/2468060169 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e08c7b7920 con 0x55e08c8ecc30 2023-09-09T23:14:23.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.062+0000 7fd6457ad700 1 --2- 172.21.15.82:0/2468060169 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55e08c8ecc30 0x55e08c8ef050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:23.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.067+0000 7fd628ff9700 10 client.15948 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:14:23.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.067+0000 7fd6457ad700 1 --2- 172.21.15.82:0/2468060169 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55e08c8ecc30 0x55e08c8ef050 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cea76cbbe90d06cf server_cookie=a3c154cd444a1f6b in_seq=0 out_seq=0 2023-09-09T23:14:23.076 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.070+0000 7fd628ff9700 1 -- 172.21.15.82:0/2468060169 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55e08c7b7920 con 0x55e08c8ecc30 2023-09-09T23:14:23.076 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.070+0000 7fd628ff9700 10 client.15948 handle_command_reply: tid=0 2023-09-09T23:14:23.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.070+0000 7fd653bd7b80 1 client.15948 shutdown 2023-09-09T23:14:23.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.070+0000 7fd653bd7b80 1 -- 172.21.15.82:0/2468060169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e08c876e40 msgr2=0x55e08c88cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:23.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.070+0000 7fd653bd7b80 1 --2- 172.21.15.82:0/2468060169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e08c876e40 0x55e08c88cf80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd638098b90 tx=0x7fd638099370 comp rx=0 tx=0).stop 2023-09-09T23:14:23.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.070+0000 7fd653bd7b80 1 -- 172.21.15.82:0/2468060169 shutdown_connections 2023-09-09T23:14:23.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.070+0000 7fd653bd7b80 1 --2- 172.21.15.82:0/2468060169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e08c876e40 0x55e08c88cf80 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.070+0000 7fd653bd7b80 1 --2- 172.21.15.82:0/2468060169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e08c86c710 0x55e08c88e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.070+0000 7fd653bd7b80 1 --2- 172.21.15.82:0/2468060169 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55e08c8ecc30 0x55e08c8ef050 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.070+0000 7fd653bd7b80 1 --2- 172.21.15.82:0/2468060169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e08c86d030 0x55e08c88ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.070+0000 7fd628ff9700 0 client.15948 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:14:23.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.070+0000 7fd653bd7b80 1 -- 172.21.15.82:0/2468060169 >> 172.21.15.82:0/2468060169 conn(0x55e08c7e1090 msgr2=0x55e08c86b700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:23.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.071+0000 7fd653bd7b80 1 -- 172.21.15.82:0/2468060169 shutdown_connections 2023-09-09T23:14:23.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.071+0000 7fd653bd7b80 1 -- 172.21.15.82:0/2468060169 wait complete. 2023-09-09T23:14:23.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.071+0000 7fd653bd7b80 20 client.15948 trim_cache size 0 max 16384 2023-09-09T23:14:23.079 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:14:23.080 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (2 inodes in the stack)", 2023-09-09T23:14:23.080 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:14:23.080 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:14:23.080 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:14:23.080 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:14:23.081 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:14:23.081 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:14:23.081 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:14:23.081 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:14:23.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.071+0000 7fd647a11700 1 -- 172.21.15.82:0/2777327096 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fd62c0a08d0 msgr2=0x7fd62c0a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:23.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.071+0000 7fd647a11700 1 --2- 172.21.15.82:0/2777327096 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fd62c0a08d0 0x7fd62c0a2d90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd63c009710 tx=0x7fd63c006c60 comp rx=0 tx=0).stop 2023-09-09T23:14:23.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.071+0000 7fd647a11700 1 -- 172.21.15.82:0/2777327096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd64019d380 msgr2=0x7fd64019d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:23.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.071+0000 7fd647a11700 1 --2- 172.21.15.82:0/2777327096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd64019d380 0x7fd64019d830 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fd63800b970 tx=0x7fd63800bd30 comp rx=0 tx=0).stop 2023-09-09T23:14:23.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.072+0000 7fd647a11700 1 -- 172.21.15.82:0/2777327096 shutdown_connections 2023-09-09T23:14:23.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.072+0000 7fd647a11700 1 --2- 172.21.15.82:0/2777327096 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fd62c0a08d0 0x7fd62c0a2d90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.072+0000 7fd647a11700 1 --2- 172.21.15.82:0/2777327096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd64019d380 0x7fd64019d830 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.072+0000 7fd647a11700 1 --2- 172.21.15.82:0/2777327096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6400fee80 0x7fd6401989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.072+0000 7fd647a11700 1 --2- 172.21.15.82:0/2777327096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd640103870 0x7fd640198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:23.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.072+0000 7fd647a11700 1 -- 172.21.15.82:0/2777327096 >> 172.21.15.82:0/2777327096 conn(0x7fd640076a70 msgr2=0x7fd640106960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:23.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.072+0000 7fd647a11700 1 -- 172.21.15.82:0/2777327096 shutdown_connections 2023-09-09T23:14:23.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:23.072+0000 7fd647a11700 1 -- 172.21.15.82:0/2777327096 wait complete. 2023-09-09T23:14:23.091 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:14:23.558 INFO:teuthology.orchestra.run.smithi082.stderr:6000000 of 50000000 tuples (12%) done (elapsed 10.14 s, remaining 74.38 s) 2023-09-09T23:14:23.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:23 smithi170 ceph-mon[102832]: pgmap v17: 97 pgs: 97 active+clean; 493 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 76 MiB/s wr, 244 op/s 2023-09-09T23:14:23.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:14:23.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:23 smithi170 ceph-mon[102832]: scrub summary: active paths [/] 2023-09-09T23:14:23.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:23 smithi170 ceph-mon[102832]: scrub summary: idle+waiting paths [/] 2023-09-09T23:14:23.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:23 smithi170 ceph-mon[102832]: scrub summary: active paths [/] 2023-09-09T23:14:23.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:23 smithi170 ceph-mon[102832]: scrub summary: idle+waiting paths [/] 2023-09-09T23:14:23.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:23 smithi170 ceph-mon[102832]: scrub summary: active paths [/] 2023-09-09T23:14:23.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:23 smithi170 ceph-mon[102832]: scrub summary: idle+waiting paths [/] 2023-09-09T23:14:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:23 smithi170 ceph-mon[102832]: scrub summary: active paths [/] 2023-09-09T23:14:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:23 smithi170 ceph-mon[102832]: scrub summary: active paths [/] 2023-09-09T23:14:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:23 smithi170 ceph-mon[102832]: scrub summary: active paths [/] 2023-09-09T23:14:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:23 smithi170 ceph-mon[102832]: scrub summary: idle+waiting paths [/] 2023-09-09T23:14:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:23 smithi170 ceph-mon[102832]: scrub summary: idle+waiting paths [/] 2023-09-09T23:14:23.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:23 smithi082 ceph-mon[98694]: pgmap v17: 97 pgs: 97 active+clean; 493 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 76 MiB/s wr, 244 op/s 2023-09-09T23:14:23.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:14:23.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:23 smithi082 ceph-mon[98694]: scrub summary: active paths [/] 2023-09-09T23:14:23.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:23 smithi082 ceph-mon[98694]: scrub summary: idle+waiting paths [/] 2023-09-09T23:14:23.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:23 smithi082 ceph-mon[98694]: scrub summary: active paths [/] 2023-09-09T23:14:23.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:23 smithi082 ceph-mon[98694]: scrub summary: idle+waiting paths [/] 2023-09-09T23:14:23.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:23 smithi082 ceph-mon[98694]: scrub summary: active paths [/] 2023-09-09T23:14:23.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:23 smithi082 ceph-mon[98694]: scrub summary: idle+waiting paths [/] 2023-09-09T23:14:23.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:23 smithi082 ceph-mon[98694]: scrub summary: active paths [/] 2023-09-09T23:14:23.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:23 smithi082 ceph-mon[98694]: scrub summary: active paths [/] 2023-09-09T23:14:23.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:23 smithi082 ceph-mon[98694]: scrub summary: active paths [/] 2023-09-09T23:14:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:23 smithi082 ceph-mon[98694]: scrub summary: idle+waiting paths [/] 2023-09-09T23:14:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:23 smithi082 ceph-mon[98694]: scrub summary: idle+waiting paths [/] 2023-09-09T23:14:23.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:23 smithi098 ceph-mon[103124]: pgmap v17: 97 pgs: 97 active+clean; 493 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 76 MiB/s wr, 244 op/s 2023-09-09T23:14:23.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:14:23.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:23 smithi098 ceph-mon[103124]: scrub summary: active paths [/] 2023-09-09T23:14:23.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:23 smithi098 ceph-mon[103124]: scrub summary: idle+waiting paths [/] 2023-09-09T23:14:23.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:23 smithi098 ceph-mon[103124]: scrub summary: active paths [/] 2023-09-09T23:14:23.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:23 smithi098 ceph-mon[103124]: scrub summary: idle+waiting paths [/] 2023-09-09T23:14:23.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:23 smithi098 ceph-mon[103124]: scrub summary: active paths [/] 2023-09-09T23:14:23.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:23 smithi098 ceph-mon[103124]: scrub summary: idle+waiting paths [/] 2023-09-09T23:14:23.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:23 smithi098 ceph-mon[103124]: scrub summary: active paths [/] 2023-09-09T23:14:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:23 smithi098 ceph-mon[103124]: scrub summary: active paths [/] 2023-09-09T23:14:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:23 smithi098 ceph-mon[103124]: scrub summary: active paths [/] 2023-09-09T23:14:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:23 smithi098 ceph-mon[103124]: scrub summary: idle+waiting paths [/] 2023-09-09T23:14:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:23 smithi098 ceph-mon[103124]: scrub summary: idle+waiting paths [/] 2023-09-09T23:14:24.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:24 smithi170 ceph-mon[102832]: scrub queued for path: / 2023-09-09T23:14:24.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:24 smithi170 ceph-mon[102832]: scrub summary: idle+waiting paths [/] 2023-09-09T23:14:24.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:24 smithi170 ceph-mon[102832]: scrub summary: active paths [/] 2023-09-09T23:14:24.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:24 smithi170 ceph-mon[102832]: scrub summary: idle+waiting paths [/] 2023-09-09T23:14:24.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:24 smithi082 ceph-mon[98694]: scrub queued for path: / 2023-09-09T23:14:24.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:24 smithi082 ceph-mon[98694]: scrub summary: idle+waiting paths [/] 2023-09-09T23:14:24.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:24 smithi082 ceph-mon[98694]: scrub summary: active paths [/] 2023-09-09T23:14:24.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:24 smithi082 ceph-mon[98694]: scrub summary: idle+waiting paths [/] 2023-09-09T23:14:24.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:24 smithi098 ceph-mon[103124]: scrub queued for path: / 2023-09-09T23:14:24.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:24 smithi098 ceph-mon[103124]: scrub summary: idle+waiting paths [/] 2023-09-09T23:14:24.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:24 smithi098 ceph-mon[103124]: scrub summary: active paths [/] 2023-09-09T23:14:24.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:24 smithi098 ceph-mon[103124]: scrub summary: idle+waiting paths [/] 2023-09-09T23:14:25.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:25 smithi170 ceph-mon[102832]: pgmap v18: 97 pgs: 97 active+clean; 493 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 76 MiB/s wr, 237 op/s 2023-09-09T23:14:25.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:25 smithi082 ceph-mon[98694]: pgmap v18: 97 pgs: 97 active+clean; 493 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 76 MiB/s wr, 237 op/s 2023-09-09T23:14:25.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:25 smithi098 ceph-mon[103124]: pgmap v18: 97 pgs: 97 active+clean; 493 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 76 MiB/s wr, 237 op/s 2023-09-09T23:14:26.931 INFO:teuthology.orchestra.run.smithi082.stderr:6100000 of 50000000 tuples (12%) done (elapsed 13.52 s, remaining 97.27 s) 2023-09-09T23:14:27.034 INFO:teuthology.orchestra.run.smithi082.stderr:6200000 of 50000000 tuples (12%) done (elapsed 13.62 s, remaining 96.21 s) 2023-09-09T23:14:27.196 INFO:teuthology.orchestra.run.smithi082.stderr:6300000 of 50000000 tuples (12%) done (elapsed 13.78 s, remaining 95.59 s) 2023-09-09T23:14:27.374 INFO:teuthology.orchestra.run.smithi082.stderr:6400000 of 50000000 tuples (12%) done (elapsed 13.96 s, remaining 95.09 s) 2023-09-09T23:14:27.444 INFO:teuthology.orchestra.run.smithi082.stderr:6500000 of 50000000 tuples (13%) done (elapsed 14.03 s, remaining 93.88 s) 2023-09-09T23:14:27.621 INFO:teuthology.orchestra.run.smithi082.stderr:6600000 of 50000000 tuples (13%) done (elapsed 14.21 s, remaining 93.41 s) 2023-09-09T23:14:27.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:27 smithi170 ceph-mon[102832]: pgmap v19: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 170 MiB/s wr, 278 op/s 2023-09-09T23:14:27.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:27 smithi170 ceph-mon[102832]: scrub summary: active paths [/] 2023-09-09T23:14:27.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:27 smithi170 ceph-mon[102832]: scrub summary: idle+waiting paths [/] 2023-09-09T23:14:27.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:27 smithi082 ceph-mon[98694]: pgmap v19: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 170 MiB/s wr, 278 op/s 2023-09-09T23:14:27.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:27 smithi082 ceph-mon[98694]: scrub summary: active paths [/] 2023-09-09T23:14:27.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:27 smithi082 ceph-mon[98694]: scrub summary: idle+waiting paths [/] 2023-09-09T23:14:27.722 INFO:teuthology.orchestra.run.smithi082.stderr:6700000 of 50000000 tuples (13%) done (elapsed 14.31 s, remaining 92.46 s) 2023-09-09T23:14:27.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:27 smithi098 ceph-mon[103124]: pgmap v19: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 170 MiB/s wr, 278 op/s 2023-09-09T23:14:27.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:27 smithi098 ceph-mon[103124]: scrub summary: active paths [/] 2023-09-09T23:14:27.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:27 smithi098 ceph-mon[103124]: scrub summary: idle+waiting paths [/] 2023-09-09T23:14:27.871 INFO:teuthology.orchestra.run.smithi082.stderr:6800000 of 50000000 tuples (13%) done (elapsed 14.46 s, remaining 91.84 s) 2023-09-09T23:14:28.043 INFO:teuthology.orchestra.run.smithi082.stderr:6900000 of 50000000 tuples (13%) done (elapsed 14.63 s, remaining 91.37 s) 2023-09-09T23:14:28.218 INFO:teuthology.orchestra.run.smithi082.stderr:7000000 of 50000000 tuples (14%) done (elapsed 14.80 s, remaining 90.93 s) 2023-09-09T23:14:28.286 INFO:teuthology.orchestra.run.smithi082.stderr:7100000 of 50000000 tuples (14%) done (elapsed 14.87 s, remaining 89.85 s) 2023-09-09T23:14:28.482 INFO:teuthology.orchestra.run.smithi082.stderr:7200000 of 50000000 tuples (14%) done (elapsed 15.07 s, remaining 89.57 s) 2023-09-09T23:14:28.663 INFO:teuthology.orchestra.run.smithi082.stderr:7300000 of 50000000 tuples (14%) done (elapsed 15.25 s, remaining 89.19 s) 2023-09-09T23:14:28.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:28 smithi170 ceph-mon[102832]: scrub summary: active paths [/] 2023-09-09T23:14:28.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:28 smithi170 ceph-mon[102832]: scrub summary: idle+waiting paths [/] 2023-09-09T23:14:28.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:28 smithi082 ceph-mon[98694]: scrub summary: active paths [/] 2023-09-09T23:14:28.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:28 smithi082 ceph-mon[98694]: scrub summary: idle+waiting paths [/] 2023-09-09T23:14:28.726 INFO:teuthology.orchestra.run.smithi082.stderr:7400000 of 50000000 tuples (14%) done (elapsed 15.31 s, remaining 88.14 s) 2023-09-09T23:14:28.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:28 smithi098 ceph-mon[103124]: scrub summary: active paths [/] 2023-09-09T23:14:28.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:28 smithi098 ceph-mon[103124]: scrub summary: idle+waiting paths [/] 2023-09-09T23:14:28.912 INFO:teuthology.orchestra.run.smithi082.stderr:7500000 of 50000000 tuples (15%) done (elapsed 15.50 s, remaining 87.81 s) 2023-09-09T23:14:28.973 INFO:teuthology.orchestra.run.smithi082.stderr:7600000 of 50000000 tuples (15%) done (elapsed 15.56 s, remaining 86.80 s) 2023-09-09T23:14:29.148 INFO:teuthology.orchestra.run.smithi082.stderr:7700000 of 50000000 tuples (15%) done (elapsed 15.73 s, remaining 86.43 s) 2023-09-09T23:14:29.308 INFO:teuthology.orchestra.run.smithi082.stderr:7800000 of 50000000 tuples (15%) done (elapsed 15.89 s, remaining 85.98 s) 2023-09-09T23:14:29.394 INFO:teuthology.orchestra.run.smithi082.stderr:7900000 of 50000000 tuples (15%) done (elapsed 15.98 s, remaining 85.15 s) 2023-09-09T23:14:29.578 INFO:teuthology.orchestra.run.smithi082.stderr:8000000 of 50000000 tuples (16%) done (elapsed 16.16 s, remaining 84.85 s) 2023-09-09T23:14:29.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:29 smithi170 ceph-mon[102832]: pgmap v20: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 146 MiB/s wr, 128 op/s 2023-09-09T23:14:29.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:29 smithi082 ceph-mon[98694]: pgmap v20: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 146 MiB/s wr, 128 op/s 2023-09-09T23:14:29.738 INFO:teuthology.orchestra.run.smithi082.stderr:8100000 of 50000000 tuples (16%) done (elapsed 16.32 s, remaining 84.43 s) 2023-09-09T23:14:29.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:29 smithi098 ceph-mon[103124]: pgmap v20: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 146 MiB/s wr, 128 op/s 2023-09-09T23:14:29.824 INFO:teuthology.orchestra.run.smithi082.stderr:8200000 of 50000000 tuples (16%) done (elapsed 16.41 s, remaining 83.64 s) 2023-09-09T23:14:30.004 INFO:teuthology.orchestra.run.smithi082.stderr:8300000 of 50000000 tuples (16%) done (elapsed 16.59 s, remaining 83.34 s) 2023-09-09T23:14:30.172 INFO:teuthology.orchestra.run.smithi082.stderr:8400000 of 50000000 tuples (16%) done (elapsed 16.76 s, remaining 82.98 s) 2023-09-09T23:14:30.254 INFO:teuthology.orchestra.run.smithi082.stderr:8500000 of 50000000 tuples (17%) done (elapsed 16.84 s, remaining 82.21 s) 2023-09-09T23:14:30.430 INFO:teuthology.orchestra.run.smithi082.stderr:8600000 of 50000000 tuples (17%) done (elapsed 17.01 s, remaining 81.91 s) 2023-09-09T23:14:30.498 INFO:teuthology.orchestra.run.smithi082.stderr:8700000 of 50000000 tuples (17%) done (elapsed 17.08 s, remaining 81.09 s) 2023-09-09T23:14:30.672 INFO:teuthology.orchestra.run.smithi082.stderr:8800000 of 50000000 tuples (17%) done (elapsed 17.26 s, remaining 80.79 s) 2023-09-09T23:14:30.856 INFO:teuthology.orchestra.run.smithi082.stderr:8900000 of 50000000 tuples (17%) done (elapsed 17.44 s, remaining 80.54 s) 2023-09-09T23:14:30.918 INFO:teuthology.orchestra.run.smithi082.stderr:9000000 of 50000000 tuples (18%) done (elapsed 17.50 s, remaining 79.73 s) 2023-09-09T23:14:31.097 INFO:teuthology.orchestra.run.smithi082.stderr:9100000 of 50000000 tuples (18%) done (elapsed 17.68 s, remaining 79.47 s) 2023-09-09T23:14:31.258 INFO:teuthology.orchestra.run.smithi082.stderr:9200000 of 50000000 tuples (18%) done (elapsed 17.84 s, remaining 79.13 s) 2023-09-09T23:14:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:9300000 of 50000000 tuples (18%) done (elapsed 17.93 s, remaining 78.46 s) 2023-09-09T23:14:31.510 INFO:teuthology.orchestra.run.smithi082.stderr:9400000 of 50000000 tuples (18%) done (elapsed 18.09 s, remaining 78.15 s) 2023-09-09T23:14:31.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:31 smithi170 ceph-mon[102832]: pgmap v21: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 154 MiB/s wr, 144 op/s 2023-09-09T23:14:31.687 INFO:teuthology.orchestra.run.smithi082.stderr:9500000 of 50000000 tuples (19%) done (elapsed 18.27 s, remaining 77.89 s) 2023-09-09T23:14:31.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:31 smithi082 ceph-mon[98694]: pgmap v21: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 154 MiB/s wr, 144 op/s 2023-09-09T23:14:31.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:31 smithi098 ceph-mon[103124]: pgmap v21: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 154 MiB/s wr, 144 op/s 2023-09-09T23:14:31.765 INFO:teuthology.orchestra.run.smithi082.stderr:9600000 of 50000000 tuples (19%) done (elapsed 18.35 s, remaining 77.22 s) 2023-09-09T23:14:31.942 INFO:teuthology.orchestra.run.smithi082.stderr:9700000 of 50000000 tuples (19%) done (elapsed 18.53 s, remaining 76.97 s) 2023-09-09T23:14:32.800 INFO:teuthology.orchestra.run.smithi082.stderr:9800000 of 50000000 tuples (19%) done (elapsed 19.38 s, remaining 79.51 s) 2023-09-09T23:14:32.866 INFO:teuthology.orchestra.run.smithi082.stderr:9900000 of 50000000 tuples (19%) done (elapsed 19.45 s, remaining 78.78 s) 2023-09-09T23:14:32.974 INFO:teuthology.orchestra.run.smithi082.stderr:10000000 of 50000000 tuples (20%) done (elapsed 19.56 s, remaining 78.23 s) 2023-09-09T23:14:33.065 INFO:teuthology.orchestra.run.smithi082.stderr:10100000 of 50000000 tuples (20%) done (elapsed 19.65 s, remaining 77.63 s) 2023-09-09T23:14:33.169 INFO:teuthology.orchestra.run.smithi082.stderr:10200000 of 50000000 tuples (20%) done (elapsed 19.75 s, remaining 77.08 s) 2023-09-09T23:14:33.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:33 smithi170 ceph-mon[102832]: pgmap v22: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 178 MiB/s wr, 159 op/s 2023-09-09T23:14:33.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:33 smithi082 ceph-mon[98694]: pgmap v22: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 178 MiB/s wr, 159 op/s 2023-09-09T23:14:33.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:33 smithi098 ceph-mon[103124]: pgmap v22: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 178 MiB/s wr, 159 op/s 2023-09-09T23:14:34.615 INFO:teuthology.orchestra.run.smithi082.stderr:10300000 of 50000000 tuples (20%) done (elapsed 21.20 s, remaining 81.71 s) 2023-09-09T23:14:35.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:35 smithi170 ceph-mon[102832]: pgmap v23: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 134 MiB/s wr, 75 op/s 2023-09-09T23:14:35.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:35 smithi082 ceph-mon[98694]: pgmap v23: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 134 MiB/s wr, 75 op/s 2023-09-09T23:14:35.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:35 smithi098 ceph-mon[103124]: pgmap v23: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 134 MiB/s wr, 75 op/s 2023-09-09T23:14:37.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:37 smithi170 ceph-mon[102832]: pgmap v24: 97 pgs: 97 active+clean; 2.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 206 MiB/s wr, 232 op/s 2023-09-09T23:14:37.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:37 smithi082 ceph-mon[98694]: pgmap v24: 97 pgs: 97 active+clean; 2.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 206 MiB/s wr, 232 op/s 2023-09-09T23:14:37.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:37 smithi098 ceph-mon[103124]: pgmap v24: 97 pgs: 97 active+clean; 2.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 206 MiB/s wr, 232 op/s 2023-09-09T23:14:38.837 INFO:teuthology.orchestra.run.smithi082.stderr:10400000 of 50000000 tuples (20%) done (elapsed 25.42 s, remaining 96.80 s) 2023-09-09T23:14:38.914 INFO:teuthology.orchestra.run.smithi082.stderr:10500000 of 50000000 tuples (21%) done (elapsed 25.50 s, remaining 95.92 s) 2023-09-09T23:14:39.016 INFO:teuthology.orchestra.run.smithi082.stderr:10600000 of 50000000 tuples (21%) done (elapsed 25.60 s, remaining 95.16 s) 2023-09-09T23:14:39.129 INFO:teuthology.orchestra.run.smithi082.stderr:10700000 of 50000000 tuples (21%) done (elapsed 25.71 s, remaining 94.44 s) 2023-09-09T23:14:39.227 INFO:teuthology.orchestra.run.smithi082.stderr:10800000 of 50000000 tuples (21%) done (elapsed 25.81 s, remaining 93.69 s) 2023-09-09T23:14:39.334 INFO:teuthology.orchestra.run.smithi082.stderr:10900000 of 50000000 tuples (21%) done (elapsed 25.92 s, remaining 92.97 s) 2023-09-09T23:14:39.399 INFO:teuthology.orchestra.run.smithi082.stderr:11000000 of 50000000 tuples (22%) done (elapsed 25.98 s, remaining 92.12 s) 2023-09-09T23:14:39.515 INFO:teuthology.orchestra.run.smithi082.stderr:11100000 of 50000000 tuples (22%) done (elapsed 26.10 s, remaining 91.46 s) 2023-09-09T23:14:39.618 INFO:teuthology.orchestra.run.smithi082.stderr:11200000 of 50000000 tuples (22%) done (elapsed 26.20 s, remaining 90.77 s) 2023-09-09T23:14:39.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:39 smithi170 ceph-mon[102832]: pgmap v25: 97 pgs: 97 active+clean; 2.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 112 MiB/s wr, 190 op/s 2023-09-09T23:14:39.700 INFO:teuthology.orchestra.run.smithi082.stderr:11300000 of 50000000 tuples (22%) done (elapsed 26.28 s, remaining 90.02 s) 2023-09-09T23:14:39.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:39 smithi082 ceph-mon[98694]: pgmap v25: 97 pgs: 97 active+clean; 2.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 112 MiB/s wr, 190 op/s 2023-09-09T23:14:39.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:39 smithi098 ceph-mon[103124]: pgmap v25: 97 pgs: 97 active+clean; 2.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 112 MiB/s wr, 190 op/s 2023-09-09T23:14:39.813 INFO:teuthology.orchestra.run.smithi082.stderr:11400000 of 50000000 tuples (22%) done (elapsed 26.40 s, remaining 89.38 s) 2023-09-09T23:14:39.912 INFO:teuthology.orchestra.run.smithi082.stderr:11500000 of 50000000 tuples (23%) done (elapsed 26.50 s, remaining 88.70 s) 2023-09-09T23:14:40.004 INFO:teuthology.orchestra.run.smithi082.stderr:11600000 of 50000000 tuples (23%) done (elapsed 26.59 s, remaining 88.02 s) 2023-09-09T23:14:40.109 INFO:teuthology.orchestra.run.smithi082.stderr:11700000 of 50000000 tuples (23%) done (elapsed 26.69 s, remaining 87.38 s) 2023-09-09T23:14:40.223 INFO:teuthology.orchestra.run.smithi082.stderr:11800000 of 50000000 tuples (23%) done (elapsed 26.81 s, remaining 86.79 s) 2023-09-09T23:14:40.321 INFO:teuthology.orchestra.run.smithi082.stderr:11900000 of 50000000 tuples (23%) done (elapsed 26.91 s, remaining 86.14 s) 2023-09-09T23:14:40.423 INFO:teuthology.orchestra.run.smithi082.stderr:12000000 of 50000000 tuples (24%) done (elapsed 27.01 s, remaining 85.52 s) 2023-09-09T23:14:40.489 INFO:teuthology.orchestra.run.smithi082.stderr:12100000 of 50000000 tuples (24%) done (elapsed 27.07 s, remaining 84.80 s) 2023-09-09T23:14:40.612 INFO:teuthology.orchestra.run.smithi082.stderr:12200000 of 50000000 tuples (24%) done (elapsed 27.20 s, remaining 84.26 s) 2023-09-09T23:14:40.716 INFO:teuthology.orchestra.run.smithi082.stderr:12300000 of 50000000 tuples (24%) done (elapsed 27.30 s, remaining 83.68 s) 2023-09-09T23:14:40.795 INFO:teuthology.orchestra.run.smithi082.stderr:12400000 of 50000000 tuples (24%) done (elapsed 27.38 s, remaining 83.02 s) 2023-09-09T23:14:40.907 INFO:teuthology.orchestra.run.smithi082.stderr:12500000 of 50000000 tuples (25%) done (elapsed 27.49 s, remaining 82.47 s) 2023-09-09T23:14:40.996 INFO:teuthology.orchestra.run.smithi082.stderr:12600000 of 50000000 tuples (25%) done (elapsed 27.58 s, remaining 81.87 s) 2023-09-09T23:14:41.084 INFO:teuthology.orchestra.run.smithi082.stderr:12700000 of 50000000 tuples (25%) done (elapsed 27.67 s, remaining 81.26 s) 2023-09-09T23:14:41.215 INFO:teuthology.orchestra.run.smithi082.stderr:12800000 of 50000000 tuples (25%) done (elapsed 27.80 s, remaining 80.79 s) 2023-09-09T23:14:41.334 INFO:teuthology.orchestra.run.smithi082.stderr:12900000 of 50000000 tuples (25%) done (elapsed 27.92 s, remaining 80.29 s) 2023-09-09T23:14:41.434 INFO:teuthology.orchestra.run.smithi082.stderr:13000000 of 50000000 tuples (26%) done (elapsed 28.02 s, remaining 79.74 s) 2023-09-09T23:14:41.543 INFO:teuthology.orchestra.run.smithi082.stderr:13100000 of 50000000 tuples (26%) done (elapsed 28.13 s, remaining 79.23 s) 2023-09-09T23:14:41.649 INFO:teuthology.orchestra.run.smithi082.stderr:13200000 of 50000000 tuples (26%) done (elapsed 28.23 s, remaining 78.71 s) 2023-09-09T23:14:41.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:41 smithi170 ceph-mon[102832]: pgmap v26: 97 pgs: 97 active+clean; 2.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 114 MiB/s wr, 194 op/s 2023-09-09T23:14:41.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:41 smithi082 ceph-mon[98694]: pgmap v26: 97 pgs: 97 active+clean; 2.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 114 MiB/s wr, 194 op/s 2023-09-09T23:14:41.735 INFO:teuthology.orchestra.run.smithi082.stderr:13300000 of 50000000 tuples (26%) done (elapsed 28.32 s, remaining 78.14 s) 2023-09-09T23:14:41.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:41 smithi098 ceph-mon[103124]: pgmap v26: 97 pgs: 97 active+clean; 2.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 114 MiB/s wr, 194 op/s 2023-09-09T23:14:41.844 INFO:teuthology.orchestra.run.smithi082.stderr:13400000 of 50000000 tuples (26%) done (elapsed 28.43 s, remaining 77.65 s) 2023-09-09T23:14:42.365 INFO:teuthology.orchestra.run.smithi082.stderr:13500000 of 50000000 tuples (27%) done (elapsed 28.95 s, remaining 78.27 s) 2023-09-09T23:14:43.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:43 smithi170 ceph-mon[102832]: pgmap v27: 97 pgs: 97 active+clean; 2.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 116 MiB/s wr, 414 op/s 2023-09-09T23:14:43.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:43 smithi082 ceph-mon[98694]: pgmap v27: 97 pgs: 97 active+clean; 2.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 116 MiB/s wr, 414 op/s 2023-09-09T23:14:43.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:43 smithi098 ceph-mon[103124]: pgmap v27: 97 pgs: 97 active+clean; 2.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 116 MiB/s wr, 414 op/s 2023-09-09T23:14:44.089 INFO:teuthology.orchestra.run.smithi082.stderr:13600000 of 50000000 tuples (27%) done (elapsed 30.67 s, remaining 82.10 s) 2023-09-09T23:14:45.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:45 smithi170 ceph-mon[102832]: pgmap v28: 97 pgs: 97 active+clean; 2.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 84 MiB/s wr, 396 op/s 2023-09-09T23:14:45.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:45 smithi082 ceph-mon[98694]: pgmap v28: 97 pgs: 97 active+clean; 2.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 84 MiB/s wr, 396 op/s 2023-09-09T23:14:45.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:45 smithi098 ceph-mon[103124]: pgmap v28: 97 pgs: 97 active+clean; 2.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 84 MiB/s wr, 396 op/s 2023-09-09T23:14:46.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:46 smithi098 ceph-mon[103124]: pgmap v29: 97 pgs: 97 active+clean; 2.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 101 MiB/s wr, 603 op/s 2023-09-09T23:14:46.900 INFO:teuthology.orchestra.run.smithi082.stderr:13700000 of 50000000 tuples (27%) done (elapsed 33.48 s, remaining 88.72 s) 2023-09-09T23:14:46.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:46 smithi170 ceph-mon[102832]: pgmap v29: 97 pgs: 97 active+clean; 2.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 101 MiB/s wr, 603 op/s 2023-09-09T23:14:46.954 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:46 smithi082 ceph-mon[98694]: pgmap v29: 97 pgs: 97 active+clean; 2.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 101 MiB/s wr, 603 op/s 2023-09-09T23:14:47.123 INFO:teuthology.orchestra.run.smithi082.stderr:13800000 of 50000000 tuples (27%) done (elapsed 33.71 s, remaining 88.42 s) 2023-09-09T23:14:47.232 INFO:teuthology.orchestra.run.smithi082.stderr:13900000 of 50000000 tuples (27%) done (elapsed 33.82 s, remaining 87.82 s) 2023-09-09T23:14:47.336 INFO:teuthology.orchestra.run.smithi082.stderr:14000000 of 50000000 tuples (28%) done (elapsed 33.92 s, remaining 87.22 s) 2023-09-09T23:14:47.428 INFO:teuthology.orchestra.run.smithi082.stderr:14100000 of 50000000 tuples (28%) done (elapsed 34.01 s, remaining 86.60 s) 2023-09-09T23:14:47.539 INFO:teuthology.orchestra.run.smithi082.stderr:14200000 of 50000000 tuples (28%) done (elapsed 34.12 s, remaining 86.03 s) 2023-09-09T23:14:47.647 INFO:teuthology.orchestra.run.smithi082.stderr:14300000 of 50000000 tuples (28%) done (elapsed 34.23 s, remaining 85.46 s) 2023-09-09T23:14:47.730 INFO:teuthology.orchestra.run.smithi082.stderr:14400000 of 50000000 tuples (28%) done (elapsed 34.31 s, remaining 84.83 s) 2023-09-09T23:14:47.820 INFO:teuthology.orchestra.run.smithi082.stderr:14500000 of 50000000 tuples (29%) done (elapsed 34.40 s, remaining 84.23 s) 2023-09-09T23:14:47.955 INFO:teuthology.orchestra.run.smithi082.stderr:14600000 of 50000000 tuples (29%) done (elapsed 34.54 s, remaining 83.75 s) 2023-09-09T23:14:48.030 INFO:teuthology.orchestra.run.smithi082.stderr:14700000 of 50000000 tuples (29%) done (elapsed 34.61 s, remaining 83.12 s) 2023-09-09T23:14:48.127 INFO:teuthology.orchestra.run.smithi082.stderr:14800000 of 50000000 tuples (29%) done (elapsed 34.71 s, remaining 82.56 s) 2023-09-09T23:14:48.229 INFO:teuthology.orchestra.run.smithi082.stderr:14900000 of 50000000 tuples (29%) done (elapsed 34.81 s, remaining 82.01 s) 2023-09-09T23:14:48.335 INFO:teuthology.orchestra.run.smithi082.stderr:15000000 of 50000000 tuples (30%) done (elapsed 34.92 s, remaining 81.48 s) 2023-09-09T23:14:48.435 INFO:teuthology.orchestra.run.smithi082.stderr:15100000 of 50000000 tuples (30%) done (elapsed 35.02 s, remaining 80.94 s) 2023-09-09T23:14:48.527 INFO:teuthology.orchestra.run.smithi082.stderr:15200000 of 50000000 tuples (30%) done (elapsed 35.11 s, remaining 80.39 s) 2023-09-09T23:14:48.649 INFO:teuthology.orchestra.run.smithi082.stderr:15300000 of 50000000 tuples (30%) done (elapsed 35.23 s, remaining 79.91 s) 2023-09-09T23:14:48.742 INFO:teuthology.orchestra.run.smithi082.stderr:15400000 of 50000000 tuples (30%) done (elapsed 35.33 s, remaining 79.37 s) 2023-09-09T23:14:48.822 INFO:teuthology.orchestra.run.smithi082.stderr:15500000 of 50000000 tuples (31%) done (elapsed 35.41 s, remaining 78.81 s) 2023-09-09T23:14:48.913 INFO:teuthology.orchestra.run.smithi082.stderr:15600000 of 50000000 tuples (31%) done (elapsed 35.50 s, remaining 78.28 s) 2023-09-09T23:14:49.044 INFO:teuthology.orchestra.run.smithi082.stderr:15700000 of 50000000 tuples (31%) done (elapsed 35.63 s, remaining 77.84 s) 2023-09-09T23:14:49.123 INFO:teuthology.orchestra.run.smithi082.stderr:15800000 of 50000000 tuples (31%) done (elapsed 35.71 s, remaining 77.29 s) 2023-09-09T23:14:49.221 INFO:teuthology.orchestra.run.smithi082.stderr:15900000 of 50000000 tuples (31%) done (elapsed 35.81 s, remaining 76.79 s) 2023-09-09T23:14:49.334 INFO:teuthology.orchestra.run.smithi082.stderr:16000000 of 50000000 tuples (32%) done (elapsed 35.92 s, remaining 76.33 s) 2023-09-09T23:14:49.425 INFO:teuthology.orchestra.run.smithi082.stderr:16100000 of 50000000 tuples (32%) done (elapsed 36.01 s, remaining 75.82 s) 2023-09-09T23:14:49.529 INFO:teuthology.orchestra.run.smithi082.stderr:16200000 of 50000000 tuples (32%) done (elapsed 36.11 s, remaining 75.35 s) 2023-09-09T23:14:49.616 INFO:teuthology.orchestra.run.smithi082.stderr:16300000 of 50000000 tuples (32%) done (elapsed 36.20 s, remaining 74.84 s) 2023-09-09T23:14:49.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:49 smithi170 ceph-mon[102832]: pgmap v30: 97 pgs: 97 active+clean; 2.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 446 op/s 2023-09-09T23:14:49.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:49 smithi082 ceph-mon[98694]: pgmap v30: 97 pgs: 97 active+clean; 2.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 446 op/s 2023-09-09T23:14:49.728 INFO:teuthology.orchestra.run.smithi082.stderr:16400000 of 50000000 tuples (32%) done (elapsed 36.31 s, remaining 74.40 s) 2023-09-09T23:14:49.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:49 smithi098 ceph-mon[103124]: pgmap v30: 97 pgs: 97 active+clean; 2.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 446 op/s 2023-09-09T23:14:49.822 INFO:teuthology.orchestra.run.smithi082.stderr:16500000 of 50000000 tuples (33%) done (elapsed 36.41 s, remaining 73.92 s) 2023-09-09T23:14:49.916 INFO:teuthology.orchestra.run.smithi082.stderr:16600000 of 50000000 tuples (33%) done (elapsed 36.50 s, remaining 73.44 s) 2023-09-09T23:14:50.033 INFO:teuthology.orchestra.run.smithi082.stderr:16700000 of 50000000 tuples (33%) done (elapsed 36.62 s, remaining 73.02 s) 2023-09-09T23:14:50.132 INFO:teuthology.orchestra.run.smithi082.stderr:16800000 of 50000000 tuples (33%) done (elapsed 36.72 s, remaining 72.56 s) 2023-09-09T23:14:50.212 INFO:teuthology.orchestra.run.smithi082.stderr:16900000 of 50000000 tuples (33%) done (elapsed 36.80 s, remaining 72.07 s) 2023-09-09T23:14:50.314 INFO:teuthology.orchestra.run.smithi082.stderr:17000000 of 50000000 tuples (34%) done (elapsed 36.90 s, remaining 71.63 s) 2023-09-09T23:14:50.415 INFO:teuthology.orchestra.run.smithi082.stderr:17100000 of 50000000 tuples (34%) done (elapsed 37.00 s, remaining 71.19 s) 2023-09-09T23:14:50.526 INFO:teuthology.orchestra.run.smithi082.stderr:17200000 of 50000000 tuples (34%) done (elapsed 37.11 s, remaining 70.77 s) 2023-09-09T23:14:50.617 INFO:teuthology.orchestra.run.smithi082.stderr:17300000 of 50000000 tuples (34%) done (elapsed 37.20 s, remaining 70.32 s) 2023-09-09T23:14:50.712 INFO:teuthology.orchestra.run.smithi082.stderr:17400000 of 50000000 tuples (34%) done (elapsed 37.30 s, remaining 69.88 s) 2023-09-09T23:14:50.825 INFO:teuthology.orchestra.run.smithi082.stderr:17500000 of 50000000 tuples (35%) done (elapsed 37.41 s, remaining 69.47 s) 2023-09-09T23:14:50.925 INFO:teuthology.orchestra.run.smithi082.stderr:17600000 of 50000000 tuples (35%) done (elapsed 37.51 s, remaining 69.05 s) 2023-09-09T23:14:51.259 INFO:teuthology.orchestra.run.smithi082.stderr:17700000 of 50000000 tuples (35%) done (elapsed 37.84 s, remaining 69.06 s) 2023-09-09T23:14:51.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:51 smithi170 ceph-mon[102832]: pgmap v31: 97 pgs: 97 active+clean; 2.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 69 MiB/s wr, 468 op/s 2023-09-09T23:14:51.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:51 smithi082 ceph-mon[98694]: pgmap v31: 97 pgs: 97 active+clean; 2.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 69 MiB/s wr, 468 op/s 2023-09-09T23:14:51.711 INFO:teuthology.orchestra.run.smithi082.stderr:17800000 of 50000000 tuples (35%) done (elapsed 38.30 s, remaining 69.28 s) 2023-09-09T23:14:51.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:51 smithi098 ceph-mon[103124]: pgmap v31: 97 pgs: 97 active+clean; 2.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 69 MiB/s wr, 468 op/s 2023-09-09T23:14:51.818 INFO:teuthology.orchestra.run.smithi082.stderr:17900000 of 50000000 tuples (35%) done (elapsed 38.40 s, remaining 68.87 s) 2023-09-09T23:14:51.920 INFO:teuthology.orchestra.run.smithi082.stderr:18000000 of 50000000 tuples (36%) done (elapsed 38.50 s, remaining 68.45 s) 2023-09-09T23:14:51.988 INFO:teuthology.orchestra.run.smithi082.stderr:18100000 of 50000000 tuples (36%) done (elapsed 38.57 s, remaining 67.98 s) 2023-09-09T23:14:52.131 INFO:teuthology.orchestra.run.smithi082.stderr:18200000 of 50000000 tuples (36%) done (elapsed 38.72 s, remaining 67.65 s) 2023-09-09T23:14:52.213 INFO:teuthology.orchestra.run.smithi082.stderr:18300000 of 50000000 tuples (36%) done (elapsed 38.80 s, remaining 67.21 s) 2023-09-09T23:14:52.319 INFO:teuthology.orchestra.run.smithi082.stderr:18400000 of 50000000 tuples (36%) done (elapsed 38.90 s, remaining 66.81 s) 2023-09-09T23:14:52.403 INFO:teuthology.orchestra.run.smithi082.stderr:18500000 of 50000000 tuples (37%) done (elapsed 38.99 s, remaining 66.38 s) 2023-09-09T23:14:52.523 INFO:teuthology.orchestra.run.smithi082.stderr:18600000 of 50000000 tuples (37%) done (elapsed 39.11 s, remaining 66.02 s) 2023-09-09T23:14:52.618 INFO:teuthology.orchestra.run.smithi082.stderr:18700000 of 50000000 tuples (37%) done (elapsed 39.20 s, remaining 65.62 s) 2023-09-09T23:14:52.711 INFO:teuthology.orchestra.run.smithi082.stderr:18800000 of 50000000 tuples (37%) done (elapsed 39.30 s, remaining 65.21 s) 2023-09-09T23:14:52.829 INFO:teuthology.orchestra.run.smithi082.stderr:18900000 of 50000000 tuples (37%) done (elapsed 39.41 s, remaining 64.85 s) 2023-09-09T23:14:52.932 INFO:teuthology.orchestra.run.smithi082.stderr:19000000 of 50000000 tuples (38%) done (elapsed 39.52 s, remaining 64.47 s) 2023-09-09T23:14:53.031 INFO:teuthology.orchestra.run.smithi082.stderr:19100000 of 50000000 tuples (38%) done (elapsed 39.62 s, remaining 64.09 s) 2023-09-09T23:14:53.092 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:14:53.143 INFO:teuthology.orchestra.run.smithi082.stderr:19200000 of 50000000 tuples (38%) done (elapsed 39.73 s, remaining 63.73 s) 2023-09-09T23:14:53.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.242+0000 7f01df342700 1 -- 172.21.15.82:0/3850693856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01d8106640 msgr2=0x7f01d8106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:53.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.242+0000 7f01df342700 1 --2- 172.21.15.82:0/3850693856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01d8106640 0x7f01d8106a20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f01d40099c0 tx=0x7f01d4009cd0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.242+0000 7f01df342700 1 -- 172.21.15.82:0/3850693856 shutdown_connections 2023-09-09T23:14:53.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.242+0000 7f01df342700 1 --2- 172.21.15.82:0/3850693856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01d8100f40 0x7f01d810d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.243+0000 7f01df342700 1 --2- 172.21.15.82:0/3850693856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01d8106640 0x7f01d8106a20 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.243+0000 7f01df342700 1 --2- 172.21.15.82:0/3850693856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01d8100580 0x7f01d8100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.243+0000 7f01df342700 1 -- 172.21.15.82:0/3850693856 >> 172.21.15.82:0/3850693856 conn(0x7f01d80fc070 msgr2=0x7f01d80fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:53.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.243+0000 7f01df342700 1 -- 172.21.15.82:0/3850693856 shutdown_connections 2023-09-09T23:14:53.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.243+0000 7f01df342700 1 -- 172.21.15.82:0/3850693856 wait complete. 2023-09-09T23:14:53.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.243+0000 7f01df342700 1 Processor -- start 2023-09-09T23:14:53.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.243+0000 7f01df342700 1 -- start start 2023-09-09T23:14:53.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.243+0000 7f01df342700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01d8100580 0x7f01d8072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:53.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.243+0000 7f01df342700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01d8100f40 0x7f01d8072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:53.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.243+0000 7f01df342700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01d8106640 0x7f01d81a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:53.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.243+0000 7f01df342700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d8073670 con 0x7f01d8106640 2023-09-09T23:14:53.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.243+0000 7f01df342700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d81a4cc0 con 0x7f01d8100f40 2023-09-09T23:14:53.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.243+0000 7f01df342700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d81a4e00 con 0x7f01d8100580 2023-09-09T23:14:53.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.244+0000 7f01dd0de700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01d8100580 0x7f01d8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:53.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.244+0000 7f01dd8df700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01d8106640 0x7f01d81a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:53.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.244+0000 7f01dd0de700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01d8100580 0x7f01d8072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:56012/0 (socket says 172.21.15.82:56012) 2023-09-09T23:14:53.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.244+0000 7f01dd0de700 1 -- 172.21.15.82:0/389256464 learned_addr learned my addr 172.21.15.82:0/389256464 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:53.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.244+0000 7f01dc8dd700 1 --2- 172.21.15.82:0/389256464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01d8100f40 0x7f01d8072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:53.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.244+0000 7f01dd0de700 1 -- 172.21.15.82:0/389256464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01d8100f40 msgr2=0x7f01d8072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:53.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.244+0000 7f01dd0de700 1 --2- 172.21.15.82:0/389256464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01d8100f40 0x7f01d8072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.244+0000 7f01dd0de700 1 -- 172.21.15.82:0/389256464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01d8106640 msgr2=0x7f01d81a4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:53.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.244+0000 7f01dd0de700 1 --2- 172.21.15.82:0/389256464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01d8106640 0x7f01d81a4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.244+0000 7f01dd0de700 1 -- 172.21.15.82:0/389256464 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d40095f0 con 0x7f01d8100580 2023-09-09T23:14:53.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.244+0000 7f01dd8df700 1 --2- 172.21.15.82:0/389256464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01d8106640 0x7f01d81a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:14:53.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.244+0000 7f01dd0de700 1 --2- 172.21.15.82:0/389256464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01d8100580 0x7f01d8072900 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f01d400b580 tx=0x7f01d400bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:53.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.244+0000 7f01ce7fc700 1 -- 172.21.15.82:0/389256464 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d401d070 con 0x7f01d8100580 2023-09-09T23:14:53.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.244+0000 7f01df342700 1 -- 172.21.15.82:0/389256464 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d81a5000 con 0x7f01d8100580 2023-09-09T23:14:53.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.244+0000 7f01ce7fc700 1 -- 172.21.15.82:0/389256464 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01d4022470 con 0x7f01d8100580 2023-09-09T23:14:53.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.244+0000 7f01ce7fc700 1 -- 172.21.15.82:0/389256464 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d4017650 con 0x7f01d8100580 2023-09-09T23:14:53.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.244+0000 7f01df342700 1 -- 172.21.15.82:0/389256464 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d81a5480 con 0x7f01d8100580 2023-09-09T23:14:53.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.247+0000 7f01ce7fc700 1 -- 172.21.15.82:0/389256464 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 24) v1 ==== 135492+0+0 (secure 0 0 0) 0x7f01d40225e0 con 0x7f01d8100580 2023-09-09T23:14:53.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.247+0000 7f01ce7fc700 1 --2- 172.21.15.82:0/389256464 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f01c40a08d0 0x7f01c40a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:53.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.247+0000 7f01ce7fc700 1 -- 172.21.15.82:0/389256464 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f01d40cd090 con 0x7f01d8100580 2023-09-09T23:14:53.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.247+0000 7f01dc8dd700 1 --2- 172.21.15.82:0/389256464 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f01c40a08d0 0x7f01c40a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:53.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.248+0000 7f01dc8dd700 1 --2- 172.21.15.82:0/389256464 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f01c40a08d0 0x7f01c40a2d90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f01d80740f0 tx=0x7f01c8009260 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:53.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.248+0000 7f01eb508b80 1 Processor -- start 2023-09-09T23:14:53.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.248+0000 7f01eb508b80 1 -- start start 2023-09-09T23:14:53.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.248+0000 7f01eb508b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8ff59e710 0x55b8ff59eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:53.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.248+0000 7f01eb508b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b8ff59f030 0x55b8ff5a8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:53.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.248+0000 7f01eb508b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b8ff5a8e40 0x55b8ff5ab230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:53.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.248+0000 7f01eb508b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8ff4f7730 con 0x55b8ff5a8e40 2023-09-09T23:14:53.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.248+0000 7f01eb508b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8ff4f5050 con 0x55b8ff59e710 2023-09-09T23:14:53.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.248+0000 7f01eb508b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8ff4f0940 con 0x55b8ff59f030 2023-09-09T23:14:53.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.248+0000 7f01dd8df700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b8ff5a8e40 0x55b8ff5ab230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:53.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.248+0000 7f01dd8df700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b8ff5a8e40 0x55b8ff5ab230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57510/0 (socket says 172.21.15.82:57510) 2023-09-09T23:14:53.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.248+0000 7f01dc8dd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b8ff59f030 0x55b8ff5a8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:53.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.248+0000 7f01dd8df700 1 -- 172.21.15.82:0/367978840 learned_addr learned my addr 172.21.15.82:0/367978840 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:53.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.248+0000 7f01dd0de700 1 --2- 172.21.15.82:0/367978840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8ff59e710 0x55b8ff59eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:53.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.249+0000 7f01dd8df700 1 -- 172.21.15.82:0/367978840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b8ff59f030 msgr2=0x55b8ff5a8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:53.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.249+0000 7f01dd8df700 1 --2- 172.21.15.82:0/367978840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b8ff59f030 0x55b8ff5a8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.249+0000 7f01dd8df700 1 -- 172.21.15.82:0/367978840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8ff59e710 msgr2=0x55b8ff59eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:53.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.249+0000 7f01dd8df700 1 --2- 172.21.15.82:0/367978840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8ff59e710 0x55b8ff59eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.249+0000 7f01dd8df700 1 -- 172.21.15.82:0/367978840 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8ff31cde0 con 0x55b8ff5a8e40 2023-09-09T23:14:53.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.249+0000 7f01dd0de700 1 --2- 172.21.15.82:0/367978840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8ff59e710 0x55b8ff59eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:14:53.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.249+0000 7f01dd8df700 1 --2- 172.21.15.82:0/367978840 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b8ff5a8e40 0x55b8ff5ab230 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f01d0003e40 tx=0x7f01d0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b77c62f3b45f7d server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:53.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.249+0000 7f01c37fe700 1 -- 172.21.15.82:0/367978840 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d00092d0 con 0x55b8ff5a8e40 2023-09-09T23:14:53.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.249+0000 7f01c37fe700 1 -- 172.21.15.82:0/367978840 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01d000f040 con 0x55b8ff5a8e40 2023-09-09T23:14:53.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.249+0000 7f01c37fe700 1 -- 172.21.15.82:0/367978840 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d000e750 con 0x55b8ff5a8e40 2023-09-09T23:14:53.257 INFO:teuthology.orchestra.run.smithi082.stderr:19300000 of 50000000 tuples (38%) done (elapsed 39.83 s, remaining 63.36 s) 2023-09-09T23:14:53.260 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.250+0000 7f01eb508b80 1 -- 172.21.15.82:0/367978840 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b8ff5a8e40 msgr2=0x55b8ff5ab230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:53.261 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.250+0000 7f01eb508b80 1 --2- 172.21.15.82:0/367978840 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b8ff5a8e40 0x55b8ff5ab230 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f01d0003e40 tx=0x7f01d0009920 comp rx=0 tx=0).stop 2023-09-09T23:14:53.261 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.250+0000 7f01eb508b80 1 -- 172.21.15.82:0/367978840 shutdown_connections 2023-09-09T23:14:53.261 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.250+0000 7f01eb508b80 1 --2- 172.21.15.82:0/367978840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8ff59e710 0x55b8ff59eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.261 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.250+0000 7f01eb508b80 1 --2- 172.21.15.82:0/367978840 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b8ff5a8e40 0x55b8ff5ab230 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.250+0000 7f01eb508b80 1 --2- 172.21.15.82:0/367978840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b8ff59f030 0x55b8ff5a8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.250+0000 7f01eb508b80 1 -- 172.21.15.82:0/367978840 >> 172.21.15.82:0/367978840 conn(0x55b8ff513090 msgr2=0x55b8ff599bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:53.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.250+0000 7f01eb508b80 1 -- 172.21.15.82:0/367978840 shutdown_connections 2023-09-09T23:14:53.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.250+0000 7f01eb508b80 1 -- 172.21.15.82:0/367978840 wait complete. 2023-09-09T23:14:53.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.250+0000 7f01eb508b80 1 -- start start 2023-09-09T23:14:53.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.250+0000 7f01eb508b80 1 Processor -- start 2023-09-09T23:14:53.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.251+0000 7f01eb508b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b8ff59e710 0x55b8ff5a8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:53.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.251+0000 7f01eb508b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8ff59f030 0x55b8ff5a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:53.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.251+0000 7f01eb508b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b8ff5a8e40 0x55b8ff5a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:53.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.251+0000 7f01eb508b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8ff48eb70 con 0x55b8ff5a8e40 2023-09-09T23:14:53.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.251+0000 7f01eb508b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8ff48e870 con 0x55b8ff59f030 2023-09-09T23:14:53.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.251+0000 7f01eb508b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8ff372920 con 0x55b8ff59e710 2023-09-09T23:14:53.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.251+0000 7f01eb508b80 10 client.0 _ll_register_callbacks cb 0x55b8ff3d8a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:14:53.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.251+0000 7f01dd0de700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b8ff59e710 0x55b8ff5a8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:53.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.251+0000 7f01dc8dd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8ff59f030 0x55b8ff5a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:53.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.251+0000 7f01dd8df700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b8ff5a8e40 0x55b8ff5a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:53.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.251+0000 7f01dd0de700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b8ff59e710 0x55b8ff5a8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:56026/0 (socket says 172.21.15.82:56026) 2023-09-09T23:14:53.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.251+0000 7f01dd0de700 1 -- 172.21.15.82:0/2584602357 learned_addr learned my addr 172.21.15.82:0/2584602357 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:53.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.251+0000 7f01dc8dd700 1 -- 172.21.15.82:0/2584602357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b8ff59e710 msgr2=0x55b8ff5a8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:53.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.251+0000 7f01dc8dd700 1 --2- 172.21.15.82:0/2584602357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b8ff59e710 0x55b8ff5a8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.251+0000 7f01dc8dd700 1 -- 172.21.15.82:0/2584602357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b8ff5a8e40 msgr2=0x55b8ff5a5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:53.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.251+0000 7f01dc8dd700 1 --2- 172.21.15.82:0/2584602357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b8ff5a8e40 0x55b8ff5a5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.251+0000 7f01dc8dd700 1 -- 172.21.15.82:0/2584602357 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d81a5480 con 0x55b8ff59f030 2023-09-09T23:14:53.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.251+0000 7f01dc8dd700 1 --2- 172.21.15.82:0/2584602357 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8ff59f030 0x55b8ff5a5a20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x55b8ff594280 tx=0x7f01c80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dee43489b4dd4303 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:53.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.251+0000 7f01c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-09T23:14:53.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.252+0000 7f01c0ff9700 1 -- 172.21.15.82:0/2584602357 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c8022070 con 0x55b8ff59f030 2023-09-09T23:14:53.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.252+0000 7f01eb508b80 1 -- 172.21.15.82:0/2584602357 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b8ff4484f0 con 0x55b8ff59f030 2023-09-09T23:14:53.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.252+0000 7f01c0ff9700 1 -- 172.21.15.82:0/2584602357 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f01c80044f0 con 0x55b8ff59f030 2023-09-09T23:14:53.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.252+0000 7f01c0ff9700 1 -- 172.21.15.82:0/2584602357 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01c8003cc0 con 0x55b8ff59f030 2023-09-09T23:14:53.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.252+0000 7f01eb508b80 10 client.25418 fetch_fsmap learned FSMap version 29 2023-09-09T23:14:53.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.252+0000 7f01eb508b80 1 -- 172.21.15.82:0/2584602357 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f01d0003b20 con 0x55b8ff59f030 2023-09-09T23:14:53.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.252+0000 7f01c0ff9700 1 -- 172.21.15.82:0/2584602357 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b8ff4484f0 con 0x55b8ff59f030 2023-09-09T23:14:53.268 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.252+0000 7f01c0ff9700 1 -- 172.21.15.82:0/2584602357 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f01c801d410 con 0x55b8ff59f030 2023-09-09T23:14:53.268 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.252+0000 7f01eb508b80 10 client.25418 fetch_fsmap finished waiting for FSMap version 29 2023-09-09T23:14:53.268 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.252+0000 7f01eb508b80 10 client.25418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:14:53.268 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.252+0000 7f01eb508b80 20 client.25418 populate_metadata read hostname 'smithi082' 2023-09-09T23:14:53.268 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.252+0000 7f01eb508b80 1 --2- 172.21.15.82:0/2584602357 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55b8ff5b8920 0x55b8ff5bad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:53.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.252+0000 7f01eb508b80 4 client.25418 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:14:53.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.252+0000 7f01eb508b80 1 -- 172.21.15.82:0/2584602357 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b8ff5058a0 con 0x55b8ff5b8920 2023-09-09T23:14:53.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.252+0000 7f01dd0de700 1 --2- 172.21.15.82:0/2584602357 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55b8ff5b8920 0x55b8ff5bad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:53.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.253+0000 7f01c0ff9700 10 client.25418 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:14:53.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.253+0000 7f01dd0de700 1 --2- 172.21.15.82:0/2584602357 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55b8ff5b8920 0x55b8ff5bad40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ae93b32c0b7aa01 server_cookie=31004c4dd122ae1c in_seq=0 out_seq=0 2023-09-09T23:14:53.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.254+0000 7f01c0ff9700 1 -- 172.21.15.82:0/2584602357 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b8ff5058a0 con 0x55b8ff5b8920 2023-09-09T23:14:53.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.254+0000 7f01c0ff9700 10 client.25418 handle_command_reply: tid=0 2023-09-09T23:14:53.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.254+0000 7f01eb508b80 1 client.25418 shutdown 2023-09-09T23:14:53.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.254+0000 7f01eb508b80 1 -- 172.21.15.82:0/2584602357 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8ff59f030 msgr2=0x55b8ff5a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:53.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.254+0000 7f01eb508b80 1 --2- 172.21.15.82:0/2584602357 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8ff59f030 0x55b8ff5a5a20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x55b8ff594280 tx=0x7f01c80047c0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.254+0000 7f01eb508b80 1 -- 172.21.15.82:0/2584602357 shutdown_connections 2023-09-09T23:14:53.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.254+0000 7f01eb508b80 1 --2- 172.21.15.82:0/2584602357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b8ff5a8e40 0x55b8ff5a5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.254+0000 7f01eb508b80 1 --2- 172.21.15.82:0/2584602357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b8ff59e710 0x55b8ff5a8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.254+0000 7f01eb508b80 1 --2- 172.21.15.82:0/2584602357 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55b8ff5b8920 0x55b8ff5bad40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.254+0000 7f01eb508b80 1 --2- 172.21.15.82:0/2584602357 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8ff59f030 0x55b8ff5a5a20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.254+0000 7f01c0ff9700 0 client.25418 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:14:53.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.254+0000 7f01eb508b80 1 -- 172.21.15.82:0/2584602357 >> 172.21.15.82:0/2584602357 conn(0x55b8ff513090 msgr2=0x55b8ff5997e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:53.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.254+0000 7f01eb508b80 1 -- 172.21.15.82:0/2584602357 shutdown_connections 2023-09-09T23:14:53.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.254+0000 7f01eb508b80 1 -- 172.21.15.82:0/2584602357 wait complete. 2023-09-09T23:14:53.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.254+0000 7f01eb508b80 20 client.25418 trim_cache size 0 max 16384 2023-09-09T23:14:53.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.271+0000 7f01eb508b80 1 Processor -- start 2023-09-09T23:14:53.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.271+0000 7f01eb508b80 1 -- start start 2023-09-09T23:14:53.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.271+0000 7f01eb508b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b8ff59e710 0x55b8ff5afa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:53.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.271+0000 7f01eb508b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8ff59f030 0x55b8ff5a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:53.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.271+0000 7f01eb508b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b8ff5a8e40 0x55b8ff5a1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:53.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.271+0000 7f01eb508b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8ff5b0020 con 0x55b8ff5a8e40 2023-09-09T23:14:53.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.271+0000 7f01eb508b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8ff5a3fb0 con 0x55b8ff59f030 2023-09-09T23:14:53.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.271+0000 7f01eb508b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8ff5989a0 con 0x55b8ff59e710 2023-09-09T23:14:53.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.271+0000 7f01dc8dd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8ff59f030 0x55b8ff5a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:53.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.271+0000 7f01dd0de700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b8ff59e710 0x55b8ff5afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:53.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.271+0000 7f01dd8df700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b8ff5a8e40 0x55b8ff5a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:53.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.271+0000 7f01dc8dd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8ff59f030 0x55b8ff5a0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:43086/0 (socket says 172.21.15.82:43086) 2023-09-09T23:14:53.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.271+0000 7f01dd0de700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b8ff59e710 0x55b8ff5afa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:56034/0 (socket says 172.21.15.82:56034) 2023-09-09T23:14:53.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.271+0000 7f01dc8dd700 1 -- 172.21.15.82:0/2519847349 learned_addr learned my addr 172.21.15.82:0/2519847349 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:53.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.271+0000 7f01dd0de700 1 -- 172.21.15.82:0/2519847349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8ff59f030 msgr2=0x55b8ff5a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:53.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.271+0000 7f01dd0de700 1 --2- 172.21.15.82:0/2519847349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8ff59f030 0x55b8ff5a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.272+0000 7f01dd0de700 1 -- 172.21.15.82:0/2519847349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b8ff5a8e40 msgr2=0x55b8ff5a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:53.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.272+0000 7f01dd0de700 1 --2- 172.21.15.82:0/2519847349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b8ff5a8e40 0x55b8ff5a1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.272+0000 7f01dd0de700 1 -- 172.21.15.82:0/2519847349 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8ff305c60 con 0x55b8ff59e710 2023-09-09T23:14:53.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.272+0000 7f01dc8dd700 1 --2- 172.21.15.82:0/2519847349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8ff59f030 0x55b8ff5a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:14:53.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.272+0000 7f01dd8df700 1 --2- 172.21.15.82:0/2519847349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b8ff5a8e40 0x55b8ff5a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:14:53.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.272+0000 7f01dd0de700 1 --2- 172.21.15.82:0/2519847349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b8ff59e710 0x55b8ff5afa30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009fd0 tx=0x7f01d4022c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9482201d84353c3 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:53.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.272+0000 7f01c3fff700 1 -- 172.21.15.82:0/2519847349 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d409acb0 con 0x55b8ff59e710 2023-09-09T23:14:53.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.272+0000 7f01c3fff700 1 -- 172.21.15.82:0/2519847349 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01d409a5b0 con 0x55b8ff59e710 2023-09-09T23:14:53.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.272+0000 7f01c3fff700 1 -- 172.21.15.82:0/2519847349 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d409aa70 con 0x55b8ff59e710 2023-09-09T23:14:53.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.273+0000 7f01eb508b80 1 -- 172.21.15.82:0/2519847349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b8ff59e710 msgr2=0x55b8ff5afa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:53.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.273+0000 7f01eb508b80 1 --2- 172.21.15.82:0/2519847349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b8ff59e710 0x55b8ff5afa30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009fd0 tx=0x7f01d4022c60 comp rx=0 tx=0).stop 2023-09-09T23:14:53.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.273+0000 7f01eb508b80 1 -- 172.21.15.82:0/2519847349 shutdown_connections 2023-09-09T23:14:53.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.273+0000 7f01eb508b80 1 --2- 172.21.15.82:0/2519847349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b8ff5a8e40 0x55b8ff5a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.273+0000 7f01eb508b80 1 --2- 172.21.15.82:0/2519847349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b8ff59e710 0x55b8ff5afa30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.273+0000 7f01eb508b80 1 --2- 172.21.15.82:0/2519847349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8ff59f030 0x55b8ff5a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.273+0000 7f01eb508b80 1 -- 172.21.15.82:0/2519847349 >> 172.21.15.82:0/2519847349 conn(0x55b8ff513090 msgr2=0x55b8ff5b4180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:53.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.273+0000 7f01eb508b80 1 -- 172.21.15.82:0/2519847349 shutdown_connections 2023-09-09T23:14:53.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.273+0000 7f01eb508b80 1 -- 172.21.15.82:0/2519847349 wait complete. 2023-09-09T23:14:53.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.273+0000 7f01eb508b80 1 -- start start 2023-09-09T23:14:53.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.273+0000 7f01eb508b80 1 Processor -- start 2023-09-09T23:14:53.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.273+0000 7f01eb508b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b8ff59e710 0x55b8ff604ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:53.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.273+0000 7f01eb508b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b8ff59f030 0x55b8ff605000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:53.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.273+0000 7f01eb508b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8ff5a8e40 0x55b8ff6006e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:53.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.273+0000 7f01eb508b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8ff600df0 con 0x55b8ff59e710 2023-09-09T23:14:53.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.273+0000 7f01eb508b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8ff600f60 con 0x55b8ff5a8e40 2023-09-09T23:14:53.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.273+0000 7f01eb508b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8ff601120 con 0x55b8ff59f030 2023-09-09T23:14:53.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.273+0000 7f01eb508b80 10 client.0 _ll_register_callbacks cb 0x55b8ff3d8a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:14:53.286 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.274+0000 7f01dd0de700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b8ff59e710 0x55b8ff604ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:53.286 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.274+0000 7f01dc8dd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b8ff59f030 0x55b8ff605000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:53.286 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.274+0000 7f01dd0de700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b8ff59e710 0x55b8ff604ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57526/0 (socket says 172.21.15.82:57526) 2023-09-09T23:14:53.286 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.274+0000 7f01dd0de700 1 -- 172.21.15.82:0/1624849205 learned_addr learned my addr 172.21.15.82:0/1624849205 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:14:53.286 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.274+0000 7f01dd8df700 1 --2- 172.21.15.82:0/1624849205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8ff5a8e40 0x55b8ff6006e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:53.287 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.274+0000 7f01dd0de700 1 -- 172.21.15.82:0/1624849205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b8ff59f030 msgr2=0x55b8ff605000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:53.287 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.274+0000 7f01dd0de700 1 --2- 172.21.15.82:0/1624849205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b8ff59f030 0x55b8ff605000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.287 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.274+0000 7f01dd0de700 1 -- 172.21.15.82:0/1624849205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8ff5a8e40 msgr2=0x55b8ff6006e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:14:53.287 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.274+0000 7f01dd0de700 1 --2- 172.21.15.82:0/1624849205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8ff5a8e40 0x55b8ff6006e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.287 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.274+0000 7f01dd0de700 1 -- 172.21.15.82:0/1624849205 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d0003b20 con 0x55b8ff59e710 2023-09-09T23:14:53.288 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.274+0000 7f01dd0de700 1 --2- 172.21.15.82:0/1624849205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b8ff59e710 0x55b8ff604ac0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f01d409a2b0 tx=0x7f01d409e450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8851b70f3878c20 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:14:53.288 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.274+0000 7f01c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:14:53.288 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.274+0000 7f01c0ff9700 1 -- 172.21.15.82:0/1624849205 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d40986f0 con 0x55b8ff59e710 2023-09-09T23:14:53.288 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.274+0000 7f01eb508b80 1 -- 172.21.15.82:0/1624849205 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01d000f040 con 0x55b8ff59e710 2023-09-09T23:14:53.288 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.274+0000 7f01c0ff9700 1 -- 172.21.15.82:0/1624849205 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f01d40ade40 con 0x55b8ff59e710 2023-09-09T23:14:53.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.274+0000 7f01c0ff9700 1 -- 172.21.15.82:0/1624849205 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01d409acb0 con 0x55b8ff59e710 2023-09-09T23:14:53.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.274+0000 7f01eb508b80 10 client.15966 fetch_fsmap learned FSMap version 29 2023-09-09T23:14:53.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.274+0000 7f01eb508b80 1 -- 172.21.15.82:0/1624849205 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b8ff31cde0 con 0x55b8ff59e710 2023-09-09T23:14:53.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.275+0000 7f01c0ff9700 1 -- 172.21.15.82:0/1624849205 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01d000f040 con 0x55b8ff59e710 2023-09-09T23:14:53.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.275+0000 7f01c0ff9700 1 -- 172.21.15.82:0/1624849205 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f01d40caa50 con 0x55b8ff59e710 2023-09-09T23:14:53.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.275+0000 7f01eb508b80 10 client.15966 fetch_fsmap finished waiting for FSMap version 29 2023-09-09T23:14:53.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.275+0000 7f01eb508b80 10 client.15966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:14:53.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.275+0000 7f01eb508b80 20 client.15966 populate_metadata read hostname 'smithi082' 2023-09-09T23:14:53.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.275+0000 7f01eb508b80 1 --2- 172.21.15.82:0/1624849205 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55b8ff60d760 0x55b8ff60fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:14:53.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.275+0000 7f01eb508b80 4 client.15966 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:14:53.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.275+0000 7f01eb508b80 1 -- 172.21.15.82:0/1624849205 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b8ff4e9920 con 0x55b8ff60d760 2023-09-09T23:14:53.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.275+0000 7f01dc8dd700 1 --2- 172.21.15.82:0/1624849205 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55b8ff60d760 0x55b8ff60fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:14:53.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.275+0000 7f01c0ff9700 10 client.15966 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:14:53.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.275+0000 7f01dc8dd700 1 --2- 172.21.15.82:0/1624849205 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55b8ff60d760 0x55b8ff60fb80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4085cd441eb86a70 server_cookie=ce054cc4c3d4221d in_seq=0 out_seq=0 2023-09-09T23:14:53.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.275+0000 7f01c0ff9700 1 -- 172.21.15.82:0/1624849205 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55b8ff4e9920 con 0x55b8ff60d760 2023-09-09T23:14:53.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.275+0000 7f01c0ff9700 10 client.15966 handle_command_reply: tid=0 2023-09-09T23:14:53.292 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.275+0000 7f01eb508b80 1 client.15966 shutdown 2023-09-09T23:14:53.292 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.276+0000 7f01eb508b80 1 -- 172.21.15.82:0/1624849205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b8ff59e710 msgr2=0x55b8ff604ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:53.292 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.276+0000 7f01eb508b80 1 --2- 172.21.15.82:0/1624849205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b8ff59e710 0x55b8ff604ac0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f01d409a2b0 tx=0x7f01d409e450 comp rx=0 tx=0).stop 2023-09-09T23:14:53.292 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.276+0000 7f01eb508b80 1 -- 172.21.15.82:0/1624849205 shutdown_connections 2023-09-09T23:14:53.292 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.276+0000 7f01eb508b80 1 --2- 172.21.15.82:0/1624849205 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55b8ff60d760 0x55b8ff60fb80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.276+0000 7f01eb508b80 1 --2- 172.21.15.82:0/1624849205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8ff5a8e40 0x55b8ff6006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.276+0000 7f01c0ff9700 0 client.15966 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:14:53.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.276+0000 7f01eb508b80 1 --2- 172.21.15.82:0/1624849205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b8ff59e710 0x55b8ff604ac0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.276+0000 7f01eb508b80 1 --2- 172.21.15.82:0/1624849205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b8ff59f030 0x55b8ff605000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.276+0000 7f01eb508b80 1 -- 172.21.15.82:0/1624849205 >> 172.21.15.82:0/1624849205 conn(0x55b8ff513090 msgr2=0x55b8ff5a0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:53.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.276+0000 7f01eb508b80 1 -- 172.21.15.82:0/1624849205 shutdown_connections 2023-09-09T23:14:53.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.276+0000 7f01eb508b80 1 -- 172.21.15.82:0/1624849205 wait complete. 2023-09-09T23:14:53.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.276+0000 7f01eb508b80 20 client.15966 trim_cache size 0 max 16384 2023-09-09T23:14:53.294 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:14:53.295 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:14:53.295 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:14:53.295 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:14:53.295 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:14:53.295 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:14:53.295 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:14:53.296 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:14:53.296 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:14:53.296 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:14:53.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.276+0000 7f01df342700 1 -- 172.21.15.82:0/389256464 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f01c40a08d0 msgr2=0x7f01c40a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:53.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.276+0000 7f01df342700 1 --2- 172.21.15.82:0/389256464 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f01c40a08d0 0x7f01c40a2d90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f01d80740f0 tx=0x7f01c8009260 comp rx=0 tx=0).stop 2023-09-09T23:14:53.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.276+0000 7f01df342700 1 -- 172.21.15.82:0/389256464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01d8100580 msgr2=0x7f01d8072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:14:53.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.276+0000 7f01df342700 1 --2- 172.21.15.82:0/389256464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01d8100580 0x7f01d8072900 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f01d400b580 tx=0x7f01d400bfd0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.276+0000 7f01df342700 1 -- 172.21.15.82:0/389256464 shutdown_connections 2023-09-09T23:14:53.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.276+0000 7f01df342700 1 --2- 172.21.15.82:0/389256464 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f01c40a08d0 0x7f01c40a2d90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.276+0000 7f01df342700 1 --2- 172.21.15.82:0/389256464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01d8106640 0x7f01d81a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.276+0000 7f01df342700 1 --2- 172.21.15.82:0/389256464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01d8100580 0x7f01d8072900 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.276+0000 7f01df342700 1 --2- 172.21.15.82:0/389256464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01d8100f40 0x7f01d8072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:14:53.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.276+0000 7f01df342700 1 -- 172.21.15.82:0/389256464 >> 172.21.15.82:0/389256464 conn(0x7f01d80fc070 msgr2=0x7f01d80fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:14:53.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.276+0000 7f01df342700 1 -- 172.21.15.82:0/389256464 shutdown_connections 2023-09-09T23:14:53.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:14:53.276+0000 7f01df342700 1 -- 172.21.15.82:0/389256464 wait complete. 2023-09-09T23:14:53.300 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:14:53.333 INFO:teuthology.orchestra.run.smithi082.stderr:19400000 of 50000000 tuples (38%) done (elapsed 39.92 s, remaining 62.96 s) 2023-09-09T23:14:53.454 INFO:teuthology.orchestra.run.smithi082.stderr:19500000 of 50000000 tuples (39%) done (elapsed 40.04 s, remaining 62.62 s) 2023-09-09T23:14:53.454 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:53 smithi082 ceph-mon[98694]: pgmap v32: 97 pgs: 97 active+clean; 2.8 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 99 MiB/s wr, 544 op/s 2023-09-09T23:14:53.555 INFO:teuthology.orchestra.run.smithi082.stderr:19600000 of 50000000 tuples (39%) done (elapsed 40.14 s, remaining 62.26 s) 2023-09-09T23:14:53.635 INFO:teuthology.orchestra.run.smithi082.stderr:19700000 of 50000000 tuples (39%) done (elapsed 40.22 s, remaining 61.86 s) 2023-09-09T23:14:53.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:53 smithi170 ceph-mon[102832]: pgmap v32: 97 pgs: 97 active+clean; 2.8 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 99 MiB/s wr, 544 op/s 2023-09-09T23:14:53.729 INFO:teuthology.orchestra.run.smithi082.stderr:19800000 of 50000000 tuples (39%) done (elapsed 40.31 s, remaining 61.49 s) 2023-09-09T23:14:53.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:53 smithi098 ceph-mon[103124]: pgmap v32: 97 pgs: 97 active+clean; 2.8 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 99 MiB/s wr, 544 op/s 2023-09-09T23:14:53.824 INFO:teuthology.orchestra.run.smithi082.stderr:19900000 of 50000000 tuples (39%) done (elapsed 40.41 s, remaining 61.12 s) 2023-09-09T23:14:53.947 INFO:teuthology.orchestra.run.smithi082.stderr:20000000 of 50000000 tuples (40%) done (elapsed 40.53 s, remaining 60.80 s) 2023-09-09T23:14:54.043 INFO:teuthology.orchestra.run.smithi082.stderr:20100000 of 50000000 tuples (40%) done (elapsed 40.63 s, remaining 60.44 s) 2023-09-09T23:14:54.139 INFO:teuthology.orchestra.run.smithi082.stderr:20200000 of 50000000 tuples (40%) done (elapsed 40.72 s, remaining 60.08 s) 2023-09-09T23:14:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:20300000 of 50000000 tuples (40%) done (elapsed 40.83 s, remaining 59.74 s) 2023-09-09T23:14:54.349 INFO:teuthology.orchestra.run.smithi082.stderr:20400000 of 50000000 tuples (40%) done (elapsed 40.93 s, remaining 59.39 s) 2023-09-09T23:14:54.436 INFO:teuthology.orchestra.run.smithi082.stderr:20500000 of 50000000 tuples (41%) done (elapsed 41.02 s, remaining 59.03 s) 2023-09-09T23:14:54.531 INFO:teuthology.orchestra.run.smithi082.stderr:20600000 of 50000000 tuples (41%) done (elapsed 41.12 s, remaining 58.68 s) 2023-09-09T23:14:54.635 INFO:teuthology.orchestra.run.smithi082.stderr:20700000 of 50000000 tuples (41%) done (elapsed 41.22 s, remaining 58.34 s) 2023-09-09T23:14:54.747 INFO:teuthology.orchestra.run.smithi082.stderr:20800000 of 50000000 tuples (41%) done (elapsed 41.33 s, remaining 58.02 s) 2023-09-09T23:14:54.842 INFO:teuthology.orchestra.run.smithi082.stderr:20900000 of 50000000 tuples (41%) done (elapsed 41.43 s, remaining 57.68 s) 2023-09-09T23:14:54.931 INFO:teuthology.orchestra.run.smithi082.stderr:21000000 of 50000000 tuples (42%) done (elapsed 41.52 s, remaining 57.33 s) 2023-09-09T23:14:55.042 INFO:teuthology.orchestra.run.smithi082.stderr:21100000 of 50000000 tuples (42%) done (elapsed 41.63 s, remaining 57.01 s) 2023-09-09T23:14:55.139 INFO:teuthology.orchestra.run.smithi082.stderr:21200000 of 50000000 tuples (42%) done (elapsed 41.72 s, remaining 56.68 s) 2023-09-09T23:14:55.243 INFO:teuthology.orchestra.run.smithi082.stderr:21300000 of 50000000 tuples (42%) done (elapsed 41.83 s, remaining 56.36 s) 2023-09-09T23:14:55.333 INFO:teuthology.orchestra.run.smithi082.stderr:21400000 of 50000000 tuples (42%) done (elapsed 41.92 s, remaining 56.02 s) 2023-09-09T23:14:55.443 INFO:teuthology.orchestra.run.smithi082.stderr:21500000 of 50000000 tuples (43%) done (elapsed 42.03 s, remaining 55.71 s) 2023-09-09T23:14:55.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:55 smithi170 ceph-mon[102832]: pgmap v33: 97 pgs: 97 active+clean; 2.8 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 88 MiB/s wr, 308 op/s 2023-09-09T23:14:55.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:55 smithi082 ceph-mon[98694]: pgmap v33: 97 pgs: 97 active+clean; 2.8 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 88 MiB/s wr, 308 op/s 2023-09-09T23:14:55.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:55 smithi098 ceph-mon[103124]: pgmap v33: 97 pgs: 97 active+clean; 2.8 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 88 MiB/s wr, 308 op/s 2023-09-09T23:14:56.104 INFO:teuthology.orchestra.run.smithi082.stderr:21600000 of 50000000 tuples (43%) done (elapsed 42.69 s, remaining 56.13 s) 2023-09-09T23:14:56.220 INFO:teuthology.orchestra.run.smithi082.stderr:21700000 of 50000000 tuples (43%) done (elapsed 42.80 s, remaining 55.82 s) 2023-09-09T23:14:56.288 INFO:teuthology.orchestra.run.smithi082.stderr:21800000 of 50000000 tuples (43%) done (elapsed 42.87 s, remaining 55.46 s) 2023-09-09T23:14:56.414 INFO:teuthology.orchestra.run.smithi082.stderr:21900000 of 50000000 tuples (43%) done (elapsed 43.00 s, remaining 55.17 s) 2023-09-09T23:14:57.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:57 smithi170 ceph-mon[102832]: pgmap v34: 97 pgs: 97 active+clean; 3.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 171 MiB/s wr, 346 op/s 2023-09-09T23:14:57.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:57 smithi082 ceph-mon[98694]: pgmap v34: 97 pgs: 97 active+clean; 3.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 171 MiB/s wr, 346 op/s 2023-09-09T23:14:57.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:57 smithi098 ceph-mon[103124]: pgmap v34: 97 pgs: 97 active+clean; 3.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 171 MiB/s wr, 346 op/s 2023-09-09T23:14:59.346 INFO:teuthology.orchestra.run.smithi082.stderr:22000000 of 50000000 tuples (44%) done (elapsed 45.93 s, remaining 58.46 s) 2023-09-09T23:14:59.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:59 smithi170 ceph-mon[102832]: pgmap v35: 97 pgs: 97 active+clean; 3.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 154 MiB/s wr, 140 op/s 2023-09-09T23:14:59.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:14:59 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:14:59.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:59 smithi082 ceph-mon[98694]: pgmap v35: 97 pgs: 97 active+clean; 3.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 154 MiB/s wr, 140 op/s 2023-09-09T23:14:59.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:14:59 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:14:59.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:59 smithi098 ceph-mon[103124]: pgmap v35: 97 pgs: 97 active+clean; 3.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 154 MiB/s wr, 140 op/s 2023-09-09T23:14:59.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:14:59 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:15:00.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:15:00.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:15:00.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:15:01.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:01 smithi170 ceph-mon[102832]: pgmap v36: 97 pgs: 97 active+clean; 3.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 185 MiB/s wr, 229 op/s 2023-09-09T23:15:01.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:15:01.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:15:01.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:15:01.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:01 smithi082 ceph-mon[98694]: pgmap v36: 97 pgs: 97 active+clean; 3.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 185 MiB/s wr, 229 op/s 2023-09-09T23:15:01.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:15:01.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:15:01.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:15:01.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:01 smithi098 ceph-mon[103124]: pgmap v36: 97 pgs: 97 active+clean; 3.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 185 MiB/s wr, 229 op/s 2023-09-09T23:15:01.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:15:01.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:15:01.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:15:03.508 INFO:teuthology.orchestra.run.smithi082.stderr:22100000 of 50000000 tuples (44%) done (elapsed 50.09 s, remaining 63.24 s) 2023-09-09T23:15:03.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:03 smithi170 ceph-mon[102832]: pgmap v37: 97 pgs: 97 active+clean; 3.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 164 MiB/s wr, 375 op/s 2023-09-09T23:15:03.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:03 smithi082 ceph-mon[98694]: pgmap v37: 97 pgs: 97 active+clean; 3.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 164 MiB/s wr, 375 op/s 2023-09-09T23:15:03.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:03 smithi098 ceph-mon[103124]: pgmap v37: 97 pgs: 97 active+clean; 3.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 164 MiB/s wr, 375 op/s 2023-09-09T23:15:04.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:15:04.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:15:04.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:15:05.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:05 smithi170 ceph-mon[102832]: pgmap v38: 97 pgs: 97 active+clean; 3.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 132 MiB/s wr, 295 op/s 2023-09-09T23:15:05.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:05 smithi082 ceph-mon[98694]: pgmap v38: 97 pgs: 97 active+clean; 3.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 132 MiB/s wr, 295 op/s 2023-09-09T23:15:05.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:05 smithi098 ceph-mon[103124]: pgmap v38: 97 pgs: 97 active+clean; 3.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 132 MiB/s wr, 295 op/s 2023-09-09T23:15:06.444 INFO:teuthology.orchestra.run.smithi082.stderr:22200000 of 50000000 tuples (44%) done (elapsed 53.03 s, remaining 66.41 s) 2023-09-09T23:15:06.551 INFO:teuthology.orchestra.run.smithi082.stderr:22300000 of 50000000 tuples (44%) done (elapsed 53.14 s, remaining 66.00 s) 2023-09-09T23:15:06.622 INFO:teuthology.orchestra.run.smithi082.stderr:22400000 of 50000000 tuples (44%) done (elapsed 53.21 s, remaining 65.56 s) 2023-09-09T23:15:06.767 INFO:teuthology.orchestra.run.smithi082.stderr:22500000 of 50000000 tuples (45%) done (elapsed 53.35 s, remaining 65.21 s) 2023-09-09T23:15:06.833 INFO:teuthology.orchestra.run.smithi082.stderr:22600000 of 50000000 tuples (45%) done (elapsed 53.42 s, remaining 64.76 s) 2023-09-09T23:15:06.948 INFO:teuthology.orchestra.run.smithi082.stderr:22700000 of 50000000 tuples (45%) done (elapsed 53.53 s, remaining 64.38 s) 2023-09-09T23:15:07.076 INFO:teuthology.orchestra.run.smithi082.stderr:22800000 of 50000000 tuples (45%) done (elapsed 53.66 s, remaining 64.01 s) 2023-09-09T23:15:07.173 INFO:teuthology.orchestra.run.smithi082.stderr:22900000 of 50000000 tuples (45%) done (elapsed 53.76 s, remaining 63.62 s) 2023-09-09T23:15:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:23000000 of 50000000 tuples (46%) done (elapsed 53.84 s, remaining 63.21 s) 2023-09-09T23:15:07.379 INFO:teuthology.orchestra.run.smithi082.stderr:23100000 of 50000000 tuples (46%) done (elapsed 53.96 s, remaining 62.84 s) 2023-09-09T23:15:07.470 INFO:teuthology.orchestra.run.smithi082.stderr:23200000 of 50000000 tuples (46%) done (elapsed 54.05 s, remaining 62.44 s) 2023-09-09T23:15:07.565 INFO:teuthology.orchestra.run.smithi082.stderr:23300000 of 50000000 tuples (46%) done (elapsed 54.15 s, remaining 62.05 s) 2023-09-09T23:15:07.674 INFO:teuthology.orchestra.run.smithi082.stderr:23400000 of 50000000 tuples (46%) done (elapsed 54.26 s, remaining 61.68 s) 2023-09-09T23:15:07.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:07 smithi170 ceph-mon[102832]: pgmap v39: 97 pgs: 97 active+clean; 3.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 135 MiB/s wr, 629 op/s 2023-09-09T23:15:07.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:07 smithi082 ceph-mon[98694]: pgmap v39: 97 pgs: 97 active+clean; 3.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 135 MiB/s wr, 629 op/s 2023-09-09T23:15:07.749 INFO:teuthology.orchestra.run.smithi082.stderr:23500000 of 50000000 tuples (47%) done (elapsed 54.33 s, remaining 61.27 s) 2023-09-09T23:15:07.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:07 smithi098 ceph-mon[103124]: pgmap v39: 97 pgs: 97 active+clean; 3.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 135 MiB/s wr, 629 op/s 2023-09-09T23:15:07.892 INFO:teuthology.orchestra.run.smithi082.stderr:23600000 of 50000000 tuples (47%) done (elapsed 54.48 s, remaining 60.94 s) 2023-09-09T23:15:08.009 INFO:teuthology.orchestra.run.smithi082.stderr:23700000 of 50000000 tuples (47%) done (elapsed 54.59 s, remaining 60.58 s) 2023-09-09T23:15:08.074 INFO:teuthology.orchestra.run.smithi082.stderr:23800000 of 50000000 tuples (47%) done (elapsed 54.66 s, remaining 60.17 s) 2023-09-09T23:15:08.193 INFO:teuthology.orchestra.run.smithi082.stderr:23900000 of 50000000 tuples (47%) done (elapsed 54.78 s, remaining 59.82 s) 2023-09-09T23:15:08.321 INFO:teuthology.orchestra.run.smithi082.stderr:24000000 of 50000000 tuples (48%) done (elapsed 54.91 s, remaining 59.48 s) 2023-09-09T23:15:08.389 INFO:teuthology.orchestra.run.smithi082.stderr:24100000 of 50000000 tuples (48%) done (elapsed 54.97 s, remaining 59.08 s) 2023-09-09T23:15:08.536 INFO:teuthology.orchestra.run.smithi082.stderr:24200000 of 50000000 tuples (48%) done (elapsed 55.12 s, remaining 58.76 s) 2023-09-09T23:15:08.604 INFO:teuthology.orchestra.run.smithi082.stderr:24300000 of 50000000 tuples (48%) done (elapsed 55.19 s, remaining 58.37 s) 2023-09-09T23:15:08.717 INFO:teuthology.orchestra.run.smithi082.stderr:24400000 of 50000000 tuples (48%) done (elapsed 55.30 s, remaining 58.02 s) 2023-09-09T23:15:08.811 INFO:teuthology.orchestra.run.smithi082.stderr:24500000 of 50000000 tuples (49%) done (elapsed 55.40 s, remaining 57.66 s) 2023-09-09T23:15:08.940 INFO:teuthology.orchestra.run.smithi082.stderr:24600000 of 50000000 tuples (49%) done (elapsed 55.52 s, remaining 57.33 s) 2023-09-09T23:15:09.016 INFO:teuthology.orchestra.run.smithi082.stderr:24700000 of 50000000 tuples (49%) done (elapsed 55.60 s, remaining 56.95 s) 2023-09-09T23:15:09.117 INFO:teuthology.orchestra.run.smithi082.stderr:24800000 of 50000000 tuples (49%) done (elapsed 55.70 s, remaining 56.60 s) 2023-09-09T23:15:09.223 INFO:teuthology.orchestra.run.smithi082.stderr:24900000 of 50000000 tuples (49%) done (elapsed 55.81 s, remaining 56.26 s) 2023-09-09T23:15:09.313 INFO:teuthology.orchestra.run.smithi082.stderr:25000000 of 50000000 tuples (50%) done (elapsed 55.90 s, remaining 55.90 s) 2023-09-09T23:15:09.424 INFO:teuthology.orchestra.run.smithi082.stderr:25100000 of 50000000 tuples (50%) done (elapsed 56.01 s, remaining 55.56 s) 2023-09-09T23:15:09.502 INFO:teuthology.orchestra.run.smithi082.stderr:25200000 of 50000000 tuples (50%) done (elapsed 56.09 s, remaining 55.20 s) 2023-09-09T23:15:09.648 INFO:teuthology.orchestra.run.smithi082.stderr:25300000 of 50000000 tuples (50%) done (elapsed 56.23 s, remaining 54.90 s) 2023-09-09T23:15:09.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:09 smithi170 ceph-mon[102832]: pgmap v40: 97 pgs: 97 active+clean; 3.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 52 MiB/s wr, 590 op/s 2023-09-09T23:15:09.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:09 smithi082 ceph-mon[98694]: pgmap v40: 97 pgs: 97 active+clean; 3.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 52 MiB/s wr, 590 op/s 2023-09-09T23:15:09.724 INFO:teuthology.orchestra.run.smithi082.stderr:25400000 of 50000000 tuples (50%) done (elapsed 56.31 s, remaining 54.53 s) 2023-09-09T23:15:09.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:09 smithi098 ceph-mon[103124]: pgmap v40: 97 pgs: 97 active+clean; 3.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 52 MiB/s wr, 590 op/s 2023-09-09T23:15:09.959 INFO:teuthology.orchestra.run.smithi082.stderr:25500000 of 50000000 tuples (51%) done (elapsed 56.54 s, remaining 54.33 s) 2023-09-09T23:15:10.581 INFO:teuthology.orchestra.run.smithi082.stderr:25600000 of 50000000 tuples (51%) done (elapsed 57.17 s, remaining 54.49 s) 2023-09-09T23:15:11.003 INFO:teuthology.orchestra.run.smithi082.stderr:25700000 of 50000000 tuples (51%) done (elapsed 57.59 s, remaining 54.45 s) 2023-09-09T23:15:11.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:11 smithi170 ceph-mon[102832]: pgmap v41: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 105 MiB/s wr, 682 op/s 2023-09-09T23:15:11.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:11 smithi082 ceph-mon[98694]: pgmap v41: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 105 MiB/s wr, 682 op/s 2023-09-09T23:15:11.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:11 smithi098 ceph-mon[103124]: pgmap v41: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 105 MiB/s wr, 682 op/s 2023-09-09T23:15:13.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:13 smithi170 ceph-mon[102832]: pgmap v42: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 75 MiB/s wr, 593 op/s 2023-09-09T23:15:13.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:13 smithi082 ceph-mon[98694]: pgmap v42: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 75 MiB/s wr, 593 op/s 2023-09-09T23:15:13.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:13 smithi098 ceph-mon[103124]: pgmap v42: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 75 MiB/s wr, 593 op/s 2023-09-09T23:15:15.392 INFO:teuthology.orchestra.run.smithi082.stderr:25800000 of 50000000 tuples (51%) done (elapsed 61.98 s, remaining 58.13 s) 2023-09-09T23:15:15.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:15 smithi170 ceph-mon[102832]: pgmap v43: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 56 MiB/s wr, 424 op/s 2023-09-09T23:15:15.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:15 smithi082 ceph-mon[98694]: pgmap v43: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 56 MiB/s wr, 424 op/s 2023-09-09T23:15:15.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:15 smithi098 ceph-mon[103124]: pgmap v43: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 56 MiB/s wr, 424 op/s 2023-09-09T23:15:16.835 INFO:teuthology.orchestra.run.smithi082.stderr:25900000 of 50000000 tuples (51%) done (elapsed 63.42 s, remaining 59.01 s) 2023-09-09T23:15:16.920 INFO:teuthology.orchestra.run.smithi082.stderr:26000000 of 50000000 tuples (52%) done (elapsed 63.50 s, remaining 58.62 s) 2023-09-09T23:15:17.028 INFO:teuthology.orchestra.run.smithi082.stderr:26100000 of 50000000 tuples (52%) done (elapsed 63.61 s, remaining 58.25 s) 2023-09-09T23:15:17.129 INFO:teuthology.orchestra.run.smithi082.stderr:26200000 of 50000000 tuples (52%) done (elapsed 63.71 s, remaining 57.88 s) 2023-09-09T23:15:17.226 INFO:teuthology.orchestra.run.smithi082.stderr:26300000 of 50000000 tuples (52%) done (elapsed 63.81 s, remaining 57.50 s) 2023-09-09T23:15:17.337 INFO:teuthology.orchestra.run.smithi082.stderr:26400000 of 50000000 tuples (52%) done (elapsed 63.92 s, remaining 57.14 s) 2023-09-09T23:15:17.438 INFO:teuthology.orchestra.run.smithi082.stderr:26500000 of 50000000 tuples (53%) done (elapsed 64.02 s, remaining 56.77 s) 2023-09-09T23:15:17.513 INFO:teuthology.orchestra.run.smithi082.stderr:26600000 of 50000000 tuples (53%) done (elapsed 64.10 s, remaining 56.39 s) 2023-09-09T23:15:17.655 INFO:teuthology.orchestra.run.smithi082.stderr:26700000 of 50000000 tuples (53%) done (elapsed 64.24 s, remaining 56.06 s) 2023-09-09T23:15:17.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:17 smithi170 ceph-mon[102832]: pgmap v44: 97 pgs: 97 active+clean; 4.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 81 MiB/s wr, 724 op/s 2023-09-09T23:15:17.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:17 smithi082 ceph-mon[98694]: pgmap v44: 97 pgs: 97 active+clean; 4.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 81 MiB/s wr, 724 op/s 2023-09-09T23:15:17.720 INFO:teuthology.orchestra.run.smithi082.stderr:26800000 of 50000000 tuples (53%) done (elapsed 64.30 s, remaining 55.67 s) 2023-09-09T23:15:17.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:17 smithi098 ceph-mon[103124]: pgmap v44: 97 pgs: 97 active+clean; 4.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 81 MiB/s wr, 724 op/s 2023-09-09T23:15:17.828 INFO:teuthology.orchestra.run.smithi082.stderr:26900000 of 50000000 tuples (53%) done (elapsed 64.41 s, remaining 55.31 s) 2023-09-09T23:15:17.938 INFO:teuthology.orchestra.run.smithi082.stderr:27000000 of 50000000 tuples (54%) done (elapsed 64.52 s, remaining 54.96 s) 2023-09-09T23:15:18.050 INFO:teuthology.orchestra.run.smithi082.stderr:27100000 of 50000000 tuples (54%) done (elapsed 64.63 s, remaining 54.62 s) 2023-09-09T23:15:18.133 INFO:teuthology.orchestra.run.smithi082.stderr:27200000 of 50000000 tuples (54%) done (elapsed 64.72 s, remaining 54.25 s) 2023-09-09T23:15:18.237 INFO:teuthology.orchestra.run.smithi082.stderr:27300000 of 50000000 tuples (54%) done (elapsed 64.82 s, remaining 53.90 s) 2023-09-09T23:15:18.346 INFO:teuthology.orchestra.run.smithi082.stderr:27400000 of 50000000 tuples (54%) done (elapsed 64.93 s, remaining 53.56 s) 2023-09-09T23:15:18.433 INFO:teuthology.orchestra.run.smithi082.stderr:27500000 of 50000000 tuples (55%) done (elapsed 65.02 s, remaining 53.20 s) 2023-09-09T23:15:18.553 INFO:teuthology.orchestra.run.smithi082.stderr:27600000 of 50000000 tuples (55%) done (elapsed 65.14 s, remaining 52.87 s) 2023-09-09T23:15:18.625 INFO:teuthology.orchestra.run.smithi082.stderr:27700000 of 50000000 tuples (55%) done (elapsed 65.21 s, remaining 52.50 s) 2023-09-09T23:15:18.740 INFO:teuthology.orchestra.run.smithi082.stderr:27800000 of 50000000 tuples (55%) done (elapsed 65.32 s, remaining 52.17 s) 2023-09-09T23:15:18.830 INFO:teuthology.orchestra.run.smithi082.stderr:27900000 of 50000000 tuples (55%) done (elapsed 65.41 s, remaining 51.82 s) 2023-09-09T23:15:18.945 INFO:teuthology.orchestra.run.smithi082.stderr:28000000 of 50000000 tuples (56%) done (elapsed 65.53 s, remaining 51.49 s) 2023-09-09T23:15:19.040 INFO:teuthology.orchestra.run.smithi082.stderr:28100000 of 50000000 tuples (56%) done (elapsed 65.62 s, remaining 51.14 s) 2023-09-09T23:15:19.167 INFO:teuthology.orchestra.run.smithi082.stderr:28200000 of 50000000 tuples (56%) done (elapsed 65.75 s, remaining 50.83 s) 2023-09-09T23:15:19.247 INFO:teuthology.orchestra.run.smithi082.stderr:28300000 of 50000000 tuples (56%) done (elapsed 65.83 s, remaining 50.48 s) 2023-09-09T23:15:19.345 INFO:teuthology.orchestra.run.smithi082.stderr:28400000 of 50000000 tuples (56%) done (elapsed 65.93 s, remaining 50.14 s) 2023-09-09T23:15:19.453 INFO:teuthology.orchestra.run.smithi082.stderr:28500000 of 50000000 tuples (57%) done (elapsed 66.04 s, remaining 49.82 s) 2023-09-09T23:15:19.552 INFO:teuthology.orchestra.run.smithi082.stderr:28600000 of 50000000 tuples (57%) done (elapsed 66.14 s, remaining 49.49 s) 2023-09-09T23:15:19.663 INFO:teuthology.orchestra.run.smithi082.stderr:28700000 of 50000000 tuples (57%) done (elapsed 66.25 s, remaining 49.17 s) 2023-09-09T23:15:19.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:19 smithi170 ceph-mon[102832]: pgmap v45: 97 pgs: 97 active+clean; 4.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 78 MiB/s wr, 391 op/s 2023-09-09T23:15:19.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:19 smithi082 ceph-mon[98694]: pgmap v45: 97 pgs: 97 active+clean; 4.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 78 MiB/s wr, 391 op/s 2023-09-09T23:15:19.732 INFO:teuthology.orchestra.run.smithi082.stderr:28800000 of 50000000 tuples (57%) done (elapsed 66.32 s, remaining 48.82 s) 2023-09-09T23:15:19.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:19 smithi098 ceph-mon[103124]: pgmap v45: 97 pgs: 97 active+clean; 4.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 78 MiB/s wr, 391 op/s 2023-09-09T23:15:19.844 INFO:teuthology.orchestra.run.smithi082.stderr:28900000 of 50000000 tuples (57%) done (elapsed 66.43 s, remaining 48.50 s) 2023-09-09T23:15:19.940 INFO:teuthology.orchestra.run.smithi082.stderr:29000000 of 50000000 tuples (58%) done (elapsed 66.52 s, remaining 48.17 s) 2023-09-09T23:15:20.048 INFO:teuthology.orchestra.run.smithi082.stderr:29100000 of 50000000 tuples (58%) done (elapsed 66.63 s, remaining 47.86 s) 2023-09-09T23:15:20.159 INFO:teuthology.orchestra.run.smithi082.stderr:29200000 of 50000000 tuples (58%) done (elapsed 66.74 s, remaining 47.54 s) 2023-09-09T23:15:20.269 INFO:teuthology.orchestra.run.smithi082.stderr:29300000 of 50000000 tuples (58%) done (elapsed 66.85 s, remaining 47.23 s) 2023-09-09T23:15:20.359 INFO:teuthology.orchestra.run.smithi082.stderr:29400000 of 50000000 tuples (58%) done (elapsed 66.94 s, remaining 46.91 s) 2023-09-09T23:15:20.464 INFO:teuthology.orchestra.run.smithi082.stderr:29500000 of 50000000 tuples (59%) done (elapsed 67.05 s, remaining 46.59 s) 2023-09-09T23:15:20.636 INFO:teuthology.orchestra.run.smithi082.stderr:29600000 of 50000000 tuples (59%) done (elapsed 67.22 s, remaining 46.33 s) 2023-09-09T23:15:21.480 INFO:teuthology.orchestra.run.smithi082.stderr:29700000 of 50000000 tuples (59%) done (elapsed 68.06 s, remaining 46.52 s) 2023-09-09T23:15:21.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:21 smithi082 ceph-mon[98694]: pgmap v46: 97 pgs: 97 active+clean; 5.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 107 MiB/s wr, 493 op/s 2023-09-09T23:15:21.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:21 smithi098 ceph-mon[103124]: pgmap v46: 97 pgs: 97 active+clean; 5.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 107 MiB/s wr, 493 op/s 2023-09-09T23:15:21.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:21 smithi170 ceph-mon[102832]: pgmap v46: 97 pgs: 97 active+clean; 5.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 107 MiB/s wr, 493 op/s 2023-09-09T23:15:21.962 INFO:teuthology.orchestra.run.smithi082.stderr:29800000 of 50000000 tuples (59%) done (elapsed 68.55 s, remaining 46.46 s) 2023-09-09T23:15:22.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:22 smithi082 ceph-mon[98694]: pgmap v47: 97 pgs: 97 active+clean; 5.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 54 MiB/s wr, 402 op/s 2023-09-09T23:15:22.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:22 smithi098 ceph-mon[103124]: pgmap v47: 97 pgs: 97 active+clean; 5.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 54 MiB/s wr, 402 op/s 2023-09-09T23:15:22.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:22 smithi170 ceph-mon[102832]: pgmap v47: 97 pgs: 97 active+clean; 5.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 54 MiB/s wr, 402 op/s 2023-09-09T23:15:23.300 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:15:23.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.431+0000 7fbee500a700 1 -- 172.21.15.82:0/2416499106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbee01004d0 msgr2=0x7fbee010c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:15:23.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.431+0000 7fbee500a700 1 --2- 172.21.15.82:0/2416499106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbee01004d0 0x7fbee010c330 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fbed800b0d0 tx=0x7fbed800b3e0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.431+0000 7fbee500a700 1 -- 172.21.15.82:0/2416499106 shutdown_connections 2023-09-09T23:15:23.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.431+0000 7fbee500a700 1 --2- 172.21.15.82:0/2416499106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbee0105570 0x7fbee0105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.431+0000 7fbee500a700 1 --2- 172.21.15.82:0/2416499106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbee01004d0 0x7fbee010c330 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.431+0000 7fbee500a700 1 --2- 172.21.15.82:0/2416499106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbee00ffb10 0x7fbee00fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.431+0000 7fbee500a700 1 -- 172.21.15.82:0/2416499106 >> 172.21.15.82:0/2416499106 conn(0x7fbee0076a70 msgr2=0x7fbee00fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:15:23.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.431+0000 7fbee500a700 1 -- 172.21.15.82:0/2416499106 shutdown_connections 2023-09-09T23:15:23.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.431+0000 7fbee500a700 1 -- 172.21.15.82:0/2416499106 wait complete. 2023-09-09T23:15:23.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.431+0000 7fbee500a700 1 Processor -- start 2023-09-09T23:15:23.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.432+0000 7fbee500a700 1 -- start start 2023-09-09T23:15:23.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.432+0000 7fbee500a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbee00ffb10 0x7fbee0195650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:23.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.432+0000 7fbee500a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbee01004d0 0x7fbee0195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:23.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.432+0000 7fbee500a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbee0105570 0x7fbee0199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:23.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.432+0000 7fbee500a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee019a630 con 0x7fbee0105570 2023-09-09T23:15:23.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.432+0000 7fbee500a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee019a7a0 con 0x7fbee00ffb10 2023-09-09T23:15:23.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.432+0000 7fbee500a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee019a960 con 0x7fbee01004d0 2023-09-09T23:15:23.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.432+0000 7fbeded9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbee00ffb10 0x7fbee0195650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:23.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.432+0000 7fbedf59e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbee0105570 0x7fbee0199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:23.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.432+0000 7fbede59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbee01004d0 0x7fbee0195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:23.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.432+0000 7fbeded9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbee00ffb10 0x7fbee0195650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:43104/0 (socket says 172.21.15.82:43104) 2023-09-09T23:15:23.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.432+0000 7fbeded9d700 1 -- 172.21.15.82:0/2232326054 learned_addr learned my addr 172.21.15.82:0/2232326054 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:15:23.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.432+0000 7fbeded9d700 1 -- 172.21.15.82:0/2232326054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbee01004d0 msgr2=0x7fbee0195b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:15:23.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.432+0000 7fbeded9d700 1 --2- 172.21.15.82:0/2232326054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbee01004d0 0x7fbee0195b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.432+0000 7fbeded9d700 1 -- 172.21.15.82:0/2232326054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbee0105570 msgr2=0x7fbee0199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:15:23.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.432+0000 7fbeded9d700 1 --2- 172.21.15.82:0/2232326054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbee0105570 0x7fbee0199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.432+0000 7fbeded9d700 1 -- 172.21.15.82:0/2232326054 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed8009d00 con 0x7fbee00ffb10 2023-09-09T23:15:23.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.432+0000 7fbede59c700 1 --2- 172.21.15.82:0/2232326054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbee01004d0 0x7fbee0195b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:15:23.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.432+0000 7fbedf59e700 1 --2- 172.21.15.82:0/2232326054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbee0105570 0x7fbee0199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:15:23.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.433+0000 7fbeded9d700 1 --2- 172.21.15.82:0/2232326054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbee00ffb10 0x7fbee0195650 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbed400e9a0 tx=0x7fbed400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:15:23.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.433+0000 7fbec7fff700 1 -- 172.21.15.82:0/2232326054 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed4009790 con 0x7fbee00ffb10 2023-09-09T23:15:23.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.433+0000 7fbee500a700 1 -- 172.21.15.82:0/2232326054 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee019ac40 con 0x7fbee00ffb10 2023-09-09T23:15:23.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.433+0000 7fbec7fff700 1 -- 172.21.15.82:0/2232326054 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbed4010430 con 0x7fbee00ffb10 2023-09-09T23:15:23.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.433+0000 7fbec7fff700 1 -- 172.21.15.82:0/2232326054 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed4018680 con 0x7fbee00ffb10 2023-09-09T23:15:23.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.433+0000 7fbee500a700 1 -- 172.21.15.82:0/2232326054 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee01a2480 con 0x7fbee00ffb10 2023-09-09T23:15:23.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.435+0000 7fbec7fff700 1 -- 172.21.15.82:0/2232326054 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) v1 ==== 135492+0+0 (secure 0 0 0) 0x7fbed40105a0 con 0x7fbee00ffb10 2023-09-09T23:15:23.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.436+0000 7fbec7fff700 1 --2- 172.21.15.82:0/2232326054 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fbec80a08d0 0x7fbec80a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:23.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.436+0000 7fbec7fff700 1 -- 172.21.15.82:0/2232326054 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fbed4014070 con 0x7fbee00ffb10 2023-09-09T23:15:23.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.436+0000 7fbede59c700 1 --2- 172.21.15.82:0/2232326054 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fbec80a08d0 0x7fbec80a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:23.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.436+0000 7fbef11d0b80 1 Processor -- start 2023-09-09T23:15:23.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.436+0000 7fbef11d0b80 1 -- start start 2023-09-09T23:15:23.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.436+0000 7fbede59c700 1 --2- 172.21.15.82:0/2232326054 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fbec80a08d0 0x7fbec80a2d90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbee0196c60 tx=0x7fbed0009260 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:15:23.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.436+0000 7fbef11d0b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3f1eee710 0x55d3f1eeeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:23.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.436+0000 7fbef11d0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3f1eef030 0x55d3f1efcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:23.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.436+0000 7fbef11d0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d3f1efd440 0x55d3f1eff830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:23.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.436+0000 7fbef11d0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3f1e47730 con 0x55d3f1efd440 2023-09-09T23:15:23.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.436+0000 7fbef11d0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3f1e45050 con 0x55d3f1eef030 2023-09-09T23:15:23.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.436+0000 7fbef11d0b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3f1e40940 con 0x55d3f1eee710 2023-09-09T23:15:23.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.437+0000 7fbeded9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3f1eee710 0x55d3f1eeeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:23.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.437+0000 7fbedf59e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d3f1efd440 0x55d3f1eff830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:23.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.437+0000 7fbede59c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3f1eef030 0x55d3f1efcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:23.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.437+0000 7fbeded9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3f1eee710 0x55d3f1eeeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:56062/0 (socket says 172.21.15.82:56062) 2023-09-09T23:15:23.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.437+0000 7fbeded9d700 1 -- 172.21.15.82:0/4272142884 learned_addr learned my addr 172.21.15.82:0/4272142884 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:15:23.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.437+0000 7fbeded9d700 1 -- 172.21.15.82:0/4272142884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3f1eef030 msgr2=0x55d3f1efcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:15:23.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.437+0000 7fbeded9d700 1 --2- 172.21.15.82:0/4272142884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3f1eef030 0x55d3f1efcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.437+0000 7fbeded9d700 1 -- 172.21.15.82:0/4272142884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d3f1efd440 msgr2=0x55d3f1eff830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:15:23.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.437+0000 7fbeded9d700 1 --2- 172.21.15.82:0/4272142884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d3f1efd440 0x55d3f1eff830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.437+0000 7fbeded9d700 1 -- 172.21.15.82:0/4272142884 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3f1c6cde0 con 0x55d3f1eee710 2023-09-09T23:15:23.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.437+0000 7fbedf59e700 1 --2- 172.21.15.82:0/4272142884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d3f1efd440 0x55d3f1eff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:15:23.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.437+0000 7fbeded9d700 1 --2- 172.21.15.82:0/4272142884 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3f1eee710 0x55d3f1eeeaf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbed4000c00 tx=0x7fbed4099030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81893fbe2d4cf756 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:15:23.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.437+0000 7fbec57fa700 1 -- 172.21.15.82:0/4272142884 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed409d480 con 0x55d3f1eee710 2023-09-09T23:15:23.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.437+0000 7fbec57fa700 1 -- 172.21.15.82:0/4272142884 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbed40a16c0 con 0x55d3f1eee710 2023-09-09T23:15:23.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.437+0000 7fbec57fa700 1 -- 172.21.15.82:0/4272142884 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed4098b40 con 0x55d3f1eee710 2023-09-09T23:15:23.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.438+0000 7fbef11d0b80 1 -- 172.21.15.82:0/4272142884 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3f1eee710 msgr2=0x55d3f1eeeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:15:23.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.438+0000 7fbef11d0b80 1 --2- 172.21.15.82:0/4272142884 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3f1eee710 0x55d3f1eeeaf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbed4000c00 tx=0x7fbed4099030 comp rx=0 tx=0).stop 2023-09-09T23:15:23.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.438+0000 7fbef11d0b80 1 -- 172.21.15.82:0/4272142884 shutdown_connections 2023-09-09T23:15:23.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.438+0000 7fbef11d0b80 1 --2- 172.21.15.82:0/4272142884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d3f1efd440 0x55d3f1eff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.438+0000 7fbef11d0b80 1 --2- 172.21.15.82:0/4272142884 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3f1eee710 0x55d3f1eeeaf0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.438+0000 7fbef11d0b80 1 --2- 172.21.15.82:0/4272142884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3f1eef030 0x55d3f1efcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.438+0000 7fbef11d0b80 1 -- 172.21.15.82:0/4272142884 >> 172.21.15.82:0/4272142884 conn(0x55d3f1e63090 msgr2=0x55d3f1ee9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:15:23.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.438+0000 7fbef11d0b80 1 -- 172.21.15.82:0/4272142884 shutdown_connections 2023-09-09T23:15:23.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.438+0000 7fbef11d0b80 1 -- 172.21.15.82:0/4272142884 wait complete. 2023-09-09T23:15:23.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.438+0000 7fbef11d0b80 1 -- start start 2023-09-09T23:15:23.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.438+0000 7fbef11d0b80 1 Processor -- start 2023-09-09T23:15:23.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.439+0000 7fbef11d0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3f1eee710 0x55d3f1efcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:23.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.439+0000 7fbef11d0b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3f1eef030 0x55d3f1efa140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:23.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.439+0000 7fbef11d0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d3f1efd440 0x55d3f1efa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:23.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.439+0000 7fbef11d0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3f1ddeb70 con 0x55d3f1efd440 2023-09-09T23:15:23.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.439+0000 7fbef11d0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3f1dde870 con 0x55d3f1eee710 2023-09-09T23:15:23.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.439+0000 7fbef11d0b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3f1cc2920 con 0x55d3f1eef030 2023-09-09T23:15:23.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.439+0000 7fbef11d0b80 10 client.0 _ll_register_callbacks cb 0x55d3f1d28a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:15:23.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.439+0000 7fbedf59e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d3f1efd440 0x55d3f1efa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:23.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.439+0000 7fbedf59e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d3f1efd440 0x55d3f1efa830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57558/0 (socket says 172.21.15.82:57558) 2023-09-09T23:15:23.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.439+0000 7fbeded9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3f1eee710 0x55d3f1efcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:23.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.439+0000 7fbedf59e700 1 -- 172.21.15.82:0/2134509912 learned_addr learned my addr 172.21.15.82:0/2134509912 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:15:23.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.439+0000 7fbede59c700 1 --2- 172.21.15.82:0/2134509912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3f1eef030 0x55d3f1efa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:23.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.439+0000 7fbedf59e700 1 -- 172.21.15.82:0/2134509912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3f1eef030 msgr2=0x55d3f1efa140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:15:23.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.439+0000 7fbedf59e700 1 --2- 172.21.15.82:0/2134509912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3f1eef030 0x55d3f1efa140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.439+0000 7fbedf59e700 1 -- 172.21.15.82:0/2134509912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3f1eee710 msgr2=0x55d3f1efcec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:15:23.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.439+0000 7fbedf59e700 1 --2- 172.21.15.82:0/2134509912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3f1eee710 0x55d3f1efcec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.439+0000 7fbedf59e700 1 -- 172.21.15.82:0/2134509912 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee010cc30 con 0x55d3f1efd440 2023-09-09T23:15:23.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.439+0000 7fbeded9d700 1 --2- 172.21.15.82:0/2134509912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3f1eee710 0x55d3f1efcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:15:23.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.439+0000 7fbedf59e700 1 --2- 172.21.15.82:0/2134509912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d3f1efd440 0x55d3f1efa830 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbed800b0d0 tx=0x7fbed800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e30366925c87a16 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:15:23.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.439+0000 7fbec2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:15:23.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.440+0000 7fbec2ff5700 1 -- 172.21.15.82:0/2134509912 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed800e040 con 0x55d3f1efd440 2023-09-09T23:15:23.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.440+0000 7fbec2ff5700 1 -- 172.21.15.82:0/2134509912 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fbed80076f0 con 0x55d3f1efd440 2023-09-09T23:15:23.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.440+0000 7fbef11d0b80 1 -- 172.21.15.82:0/2134509912 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3f1d984f0 con 0x55d3f1efd440 2023-09-09T23:15:23.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.440+0000 7fbec2ff5700 1 -- 172.21.15.82:0/2134509912 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbed80078b0 con 0x55d3f1efd440 2023-09-09T23:15:23.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.440+0000 7fbef11d0b80 10 client.15978 fetch_fsmap learned FSMap version 31 2023-09-09T23:15:23.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.440+0000 7fbef11d0b80 1 -- 172.21.15.82:0/2134509912 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fbee01a2480 con 0x55d3f1efd440 2023-09-09T23:15:23.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.440+0000 7fbec2ff5700 1 -- 172.21.15.82:0/2134509912 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d3f1d984f0 con 0x55d3f1efd440 2023-09-09T23:15:23.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.440+0000 7fbec2ff5700 1 -- 172.21.15.82:0/2134509912 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fbed800b100 con 0x55d3f1efd440 2023-09-09T23:15:23.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.440+0000 7fbef11d0b80 10 client.15978 fetch_fsmap finished waiting for FSMap version 31 2023-09-09T23:15:23.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.440+0000 7fbef11d0b80 10 client.15978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:15:23.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.440+0000 7fbef11d0b80 20 client.15978 populate_metadata read hostname 'smithi082' 2023-09-09T23:15:23.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.440+0000 7fbef11d0b80 1 --2- 172.21.15.82:0/2134509912 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55d3f1f0ce90 0x55d3f1f0f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:23.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.440+0000 7fbef11d0b80 4 client.15978 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:15:23.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.440+0000 7fbef11d0b80 1 -- 172.21.15.82:0/2134509912 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3f1e558a0 con 0x55d3f1f0ce90 2023-09-09T23:15:23.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.440+0000 7fbeded9d700 1 --2- 172.21.15.82:0/2134509912 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55d3f1f0ce90 0x55d3f1f0f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:23.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.441+0000 7fbeded9d700 1 --2- 172.21.15.82:0/2134509912 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55d3f1f0ce90 0x55d3f1f0f2b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b586041e8cdb93b4 server_cookie=1cb942db1ac6125 in_seq=0 out_seq=0 2023-09-09T23:15:23.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.441+0000 7fbec2ff5700 10 client.15978 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:15:23.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.441+0000 7fbec2ff5700 1 -- 172.21.15.82:0/2134509912 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d3f1e558a0 con 0x55d3f1f0ce90 2023-09-09T23:15:23.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.441+0000 7fbec2ff5700 10 client.15978 handle_command_reply: tid=0 2023-09-09T23:15:23.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.442+0000 7fbef11d0b80 1 client.15978 shutdown 2023-09-09T23:15:23.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.442+0000 7fbef11d0b80 1 -- 172.21.15.82:0/2134509912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d3f1efd440 msgr2=0x55d3f1efa830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:15:23.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.442+0000 7fbef11d0b80 1 --2- 172.21.15.82:0/2134509912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d3f1efd440 0x55d3f1efa830 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbed800b0d0 tx=0x7fbed800be10 comp rx=0 tx=0).stop 2023-09-09T23:15:23.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.442+0000 7fbef11d0b80 1 -- 172.21.15.82:0/2134509912 shutdown_connections 2023-09-09T23:15:23.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.442+0000 7fbef11d0b80 1 --2- 172.21.15.82:0/2134509912 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55d3f1f0ce90 0x55d3f1f0f2b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.442+0000 7fbef11d0b80 1 --2- 172.21.15.82:0/2134509912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3f1eee710 0x55d3f1efcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.442+0000 7fbef11d0b80 1 --2- 172.21.15.82:0/2134509912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d3f1efd440 0x55d3f1efa830 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.442+0000 7fbef11d0b80 1 --2- 172.21.15.82:0/2134509912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3f1eef030 0x55d3f1efa140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.442+0000 7fbec2ff5700 0 client.15978 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:15:23.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.442+0000 7fbef11d0b80 1 -- 172.21.15.82:0/2134509912 >> 172.21.15.82:0/2134509912 conn(0x55d3f1e63090 msgr2=0x55d3f1ee9780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:15:23.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.442+0000 7fbef11d0b80 1 -- 172.21.15.82:0/2134509912 shutdown_connections 2023-09-09T23:15:23.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.442+0000 7fbef11d0b80 1 -- 172.21.15.82:0/2134509912 wait complete. 2023-09-09T23:15:23.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.442+0000 7fbef11d0b80 20 client.15978 trim_cache size 0 max 16384 2023-09-09T23:15:23.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.459+0000 7fbef11d0b80 1 Processor -- start 2023-09-09T23:15:23.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.459+0000 7fbef11d0b80 1 -- start start 2023-09-09T23:15:23.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.459+0000 7fbef11d0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3f1eee710 0x55d3f1efa740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:23.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.459+0000 7fbef11d0b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3f1eef030 0x55d3f1ef9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:23.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.459+0000 7fbef11d0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d3f1efd440 0x55d3f1f07fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:23.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.459+0000 7fbef11d0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3f1ee8940 con 0x55d3f1efd440 2023-09-09T23:15:23.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.459+0000 7fbef11d0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3f1efae10 con 0x55d3f1eee710 2023-09-09T23:15:23.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.459+0000 7fbef11d0b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3f1f0a3f0 con 0x55d3f1eef030 2023-09-09T23:15:23.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.459+0000 7fbedf59e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d3f1efd440 0x55d3f1f07fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:23.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.459+0000 7fbedf59e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d3f1efd440 0x55d3f1f07fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57566/0 (socket says 172.21.15.82:57566) 2023-09-09T23:15:23.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.459+0000 7fbede59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3f1eef030 0x55d3f1ef9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:23.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.459+0000 7fbedf59e700 1 -- 172.21.15.82:0/3164517908 learned_addr learned my addr 172.21.15.82:0/3164517908 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:15:23.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.459+0000 7fbeded9d700 1 --2- 172.21.15.82:0/3164517908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3f1eee710 0x55d3f1efa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:23.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.460+0000 7fbedf59e700 1 -- 172.21.15.82:0/3164517908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3f1eef030 msgr2=0x55d3f1ef9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:15:23.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.460+0000 7fbedf59e700 1 --2- 172.21.15.82:0/3164517908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3f1eef030 0x55d3f1ef9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.460+0000 7fbedf59e700 1 -- 172.21.15.82:0/3164517908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3f1eee710 msgr2=0x55d3f1efa740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:15:23.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.460+0000 7fbedf59e700 1 --2- 172.21.15.82:0/3164517908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3f1eee710 0x55d3f1efa740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.460+0000 7fbedf59e700 1 -- 172.21.15.82:0/3164517908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3f1c55c60 con 0x55d3f1efd440 2023-09-09T23:15:23.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.460+0000 7fbede59c700 1 --2- 172.21.15.82:0/3164517908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3f1eef030 0x55d3f1ef9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:15:23.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.460+0000 7fbedf59e700 1 --2- 172.21.15.82:0/3164517908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d3f1efd440 0x55d3f1f07fb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbed8005bb0 tx=0x7fbed800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1e14a4194492c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:15:23.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.460+0000 7fbec5ffb700 1 -- 172.21.15.82:0/3164517908 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed800e070 con 0x55d3f1efd440 2023-09-09T23:15:23.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.460+0000 7fbec5ffb700 1 -- 172.21.15.82:0/3164517908 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbed800b100 con 0x55d3f1efd440 2023-09-09T23:15:23.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.460+0000 7fbec5ffb700 1 -- 172.21.15.82:0/3164517908 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed8003a00 con 0x55d3f1efd440 2023-09-09T23:15:23.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.461+0000 7fbef11d0b80 1 -- 172.21.15.82:0/3164517908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d3f1efd440 msgr2=0x55d3f1f07fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:15:23.477 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.461+0000 7fbef11d0b80 1 --2- 172.21.15.82:0/3164517908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d3f1efd440 0x55d3f1f07fb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbed8005bb0 tx=0x7fbed800be80 comp rx=0 tx=0).stop 2023-09-09T23:15:23.477 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.461+0000 7fbef11d0b80 1 -- 172.21.15.82:0/3164517908 shutdown_connections 2023-09-09T23:15:23.477 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.461+0000 7fbef11d0b80 1 --2- 172.21.15.82:0/3164517908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3f1eee710 0x55d3f1efa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.477 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.461+0000 7fbef11d0b80 1 --2- 172.21.15.82:0/3164517908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d3f1efd440 0x55d3f1f07fb0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.477 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.461+0000 7fbef11d0b80 1 --2- 172.21.15.82:0/3164517908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3f1eef030 0x55d3f1ef9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.461+0000 7fbef11d0b80 1 -- 172.21.15.82:0/3164517908 >> 172.21.15.82:0/3164517908 conn(0x55d3f1e63090 msgr2=0x55d3f1eedbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:15:23.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.461+0000 7fbef11d0b80 1 -- 172.21.15.82:0/3164517908 shutdown_connections 2023-09-09T23:15:23.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.461+0000 7fbef11d0b80 1 -- 172.21.15.82:0/3164517908 wait complete. 2023-09-09T23:15:23.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.461+0000 7fbef11d0b80 1 -- start start 2023-09-09T23:15:23.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.461+0000 7fbef11d0b80 1 Processor -- start 2023-09-09T23:15:23.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.461+0000 7fbef11d0b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3f1eee710 0x55d3f1f01660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:23.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.461+0000 7fbef11d0b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3f1eef030 0x55d3f1eff990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:23.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.461+0000 7fbef11d0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d3f1efd440 0x55d3f1effed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:23.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.461+0000 7fbef11d0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3f1f00410 con 0x55d3f1efd440 2023-09-09T23:15:23.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.461+0000 7fbef11d0b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3f1f00580 con 0x55d3f1eee710 2023-09-09T23:15:23.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.461+0000 7fbef11d0b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3f1f00740 con 0x55d3f1eef030 2023-09-09T23:15:23.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.461+0000 7fbef11d0b80 10 client.0 _ll_register_callbacks cb 0x7fbed800baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:15:23.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.462+0000 7fbedf59e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d3f1efd440 0x55d3f1effed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:23.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.462+0000 7fbeded9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3f1eee710 0x55d3f1f01660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:23.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.462+0000 7fbedf59e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d3f1efd440 0x55d3f1effed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57572/0 (socket says 172.21.15.82:57572) 2023-09-09T23:15:23.481 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.462+0000 7fbeded9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3f1eee710 0x55d3f1f01660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:43132/0 (socket says 172.21.15.82:43132) 2023-09-09T23:15:23.481 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.462+0000 7fbedf59e700 1 -- 172.21.15.82:0/1277728106 learned_addr learned my addr 172.21.15.82:0/1277728106 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:15:23.481 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.462+0000 7fbede59c700 1 --2- 172.21.15.82:0/1277728106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3f1eef030 0x55d3f1eff990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:23.481 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.462+0000 7fbedf59e700 1 -- 172.21.15.82:0/1277728106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3f1eef030 msgr2=0x55d3f1eff990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:15:23.481 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.462+0000 7fbedf59e700 1 --2- 172.21.15.82:0/1277728106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3f1eef030 0x55d3f1eff990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.462+0000 7fbedf59e700 1 -- 172.21.15.82:0/1277728106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3f1eee710 msgr2=0x55d3f1f01660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:15:23.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.462+0000 7fbedf59e700 1 --2- 172.21.15.82:0/1277728106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3f1eee710 0x55d3f1f01660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.462+0000 7fbedf59e700 1 -- 172.21.15.82:0/1277728106 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed0017070 con 0x55d3f1efd440 2023-09-09T23:15:23.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.462+0000 7fbeded9d700 1 --2- 172.21.15.82:0/1277728106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3f1eee710 0x55d3f1f01660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:15:23.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.462+0000 7fbedf59e700 1 --2- 172.21.15.82:0/1277728106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d3f1efd440 0x55d3f1effed0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fbed801e040 tx=0x7fbed8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec9699297413c53 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:15:23.483 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.462+0000 7fbec2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:15:23.483 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.462+0000 7fbec2ff5700 1 -- 172.21.15.82:0/1277728106 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed800e050 con 0x55d3f1efd440 2023-09-09T23:15:23.483 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.462+0000 7fbec2ff5700 1 -- 172.21.15.82:0/1277728106 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fbed800b270 con 0x55d3f1efd440 2023-09-09T23:15:23.483 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.462+0000 7fbef11d0b80 1 -- 172.21.15.82:0/1277728106 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbed40a16c0 con 0x55d3f1efd440 2023-09-09T23:15:23.483 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.462+0000 7fbec2ff5700 1 -- 172.21.15.82:0/1277728106 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbed80052d0 con 0x55d3f1efd440 2023-09-09T23:15:23.483 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.462+0000 7fbef11d0b80 10 client.15984 fetch_fsmap learned FSMap version 32 2023-09-09T23:15:23.484 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.462+0000 7fbef11d0b80 1 -- 172.21.15.82:0/1277728106 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7fbee01a2480 con 0x55d3f1efd440 2023-09-09T23:15:23.484 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.462+0000 7fbec2ff5700 1 -- 172.21.15.82:0/1277728106 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbed40a16c0 con 0x55d3f1efd440 2023-09-09T23:15:23.484 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.463+0000 7fbec2ff5700 1 -- 172.21.15.82:0/1277728106 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fbed8025cb0 con 0x55d3f1efd440 2023-09-09T23:15:23.484 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.463+0000 7fbef11d0b80 10 client.15984 fetch_fsmap finished waiting for FSMap version 32 2023-09-09T23:15:23.484 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.463+0000 7fbef11d0b80 10 client.15984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:15:23.485 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.463+0000 7fbef11d0b80 20 client.15984 populate_metadata read hostname 'smithi082' 2023-09-09T23:15:23.485 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.463+0000 7fbef11d0b80 1 --2- 172.21.15.82:0/1277728106 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55d3f1f61ef0 0x55d3f1f64310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:23.485 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.463+0000 7fbef11d0b80 4 client.15984 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:15:23.485 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.463+0000 7fbef11d0b80 1 -- 172.21.15.82:0/1277728106 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d3f1e39920 con 0x55d3f1f61ef0 2023-09-09T23:15:23.485 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.463+0000 7fbeded9d700 1 --2- 172.21.15.82:0/1277728106 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55d3f1f61ef0 0x55d3f1f64310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:23.486 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.463+0000 7fbec2ff5700 10 client.15984 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:15:23.486 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.463+0000 7fbeded9d700 1 --2- 172.21.15.82:0/1277728106 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55d3f1f61ef0 0x55d3f1f64310 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=151b4980908057f2 server_cookie=7b5cc59721ef6b14 in_seq=0 out_seq=0 2023-09-09T23:15:23.486 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.463+0000 7fbec2ff5700 1 -- 172.21.15.82:0/1277728106 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55d3f1e39920 con 0x55d3f1f61ef0 2023-09-09T23:15:23.486 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.463+0000 7fbec2ff5700 10 client.15984 handle_command_reply: tid=0 2023-09-09T23:15:23.486 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.463+0000 7fbef11d0b80 1 client.15984 shutdown 2023-09-09T23:15:23.486 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.464+0000 7fbef11d0b80 1 -- 172.21.15.82:0/1277728106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d3f1efd440 msgr2=0x55d3f1effed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:15:23.487 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.464+0000 7fbef11d0b80 1 --2- 172.21.15.82:0/1277728106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d3f1efd440 0x55d3f1effed0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fbed801e040 tx=0x7fbed8003c10 comp rx=0 tx=0).stop 2023-09-09T23:15:23.487 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.464+0000 7fbef11d0b80 1 -- 172.21.15.82:0/1277728106 shutdown_connections 2023-09-09T23:15:23.487 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.464+0000 7fbef11d0b80 1 --2- 172.21.15.82:0/1277728106 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55d3f1f61ef0 0x55d3f1f64310 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.487 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.464+0000 7fbef11d0b80 1 --2- 172.21.15.82:0/1277728106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d3f1eee710 0x55d3f1f01660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.487 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.464+0000 7fbec2ff5700 0 client.15984 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:15:23.488 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.464+0000 7fbef11d0b80 1 --2- 172.21.15.82:0/1277728106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d3f1efd440 0x55d3f1effed0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.488 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.464+0000 7fbef11d0b80 1 --2- 172.21.15.82:0/1277728106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3f1eef030 0x55d3f1eff990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.488 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.464+0000 7fbef11d0b80 1 -- 172.21.15.82:0/1277728106 >> 172.21.15.82:0/1277728106 conn(0x55d3f1e63090 msgr2=0x55d3f1f07fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:15:23.488 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.464+0000 7fbef11d0b80 1 -- 172.21.15.82:0/1277728106 shutdown_connections 2023-09-09T23:15:23.488 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.464+0000 7fbef11d0b80 1 -- 172.21.15.82:0/1277728106 wait complete. 2023-09-09T23:15:23.488 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.464+0000 7fbef11d0b80 20 client.15984 trim_cache size 0 max 16384 2023-09-09T23:15:23.489 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:15:23.489 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:15:23.489 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:15:23.489 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:15:23.490 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:15:23.490 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:15:23.490 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:15:23.490 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:15:23.490 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:15:23.491 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:15:23.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.464+0000 7fbee500a700 1 -- 172.21.15.82:0/2232326054 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fbec80a08d0 msgr2=0x7fbec80a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:15:23.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.464+0000 7fbee500a700 1 --2- 172.21.15.82:0/2232326054 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fbec80a08d0 0x7fbec80a2d90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbee0196c60 tx=0x7fbed0009260 comp rx=0 tx=0).stop 2023-09-09T23:15:23.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.464+0000 7fbee500a700 1 -- 172.21.15.82:0/2232326054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbee00ffb10 msgr2=0x7fbee0195650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:15:23.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.464+0000 7fbee500a700 1 --2- 172.21.15.82:0/2232326054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbee00ffb10 0x7fbee0195650 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbed400e9a0 tx=0x7fbed400ecb0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.464+0000 7fbee500a700 1 -- 172.21.15.82:0/2232326054 shutdown_connections 2023-09-09T23:15:23.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.464+0000 7fbee500a700 1 --2- 172.21.15.82:0/2232326054 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fbec80a08d0 0x7fbec80a2d90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.464+0000 7fbee500a700 1 --2- 172.21.15.82:0/2232326054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbee00ffb10 0x7fbee0195650 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.464+0000 7fbee500a700 1 --2- 172.21.15.82:0/2232326054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbee0105570 0x7fbee0199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.464+0000 7fbee500a700 1 --2- 172.21.15.82:0/2232326054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbee01004d0 0x7fbee0195b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:23.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.464+0000 7fbee500a700 1 -- 172.21.15.82:0/2232326054 >> 172.21.15.82:0/2232326054 conn(0x7fbee0076a70 msgr2=0x7fbee00fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:15:23.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.465+0000 7fbee500a700 1 -- 172.21.15.82:0/2232326054 shutdown_connections 2023-09-09T23:15:23.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:23.465+0000 7fbee500a700 1 -- 172.21.15.82:0/2232326054 wait complete. 2023-09-09T23:15:23.495 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:15:24.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:15:24.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:24 smithi082 ceph-mon[98694]: pgmap v48: 97 pgs: 97 active+clean; 5.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 54 MiB/s wr, 402 op/s 2023-09-09T23:15:24.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:15:24.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:24 smithi098 ceph-mon[103124]: pgmap v48: 97 pgs: 97 active+clean; 5.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 54 MiB/s wr, 402 op/s 2023-09-09T23:15:24.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:15:24.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:24 smithi170 ceph-mon[102832]: pgmap v48: 97 pgs: 97 active+clean; 5.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 54 MiB/s wr, 402 op/s 2023-09-09T23:15:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:29900000 of 50000000 tuples (59%) done (elapsed 73.24 s, remaining 49.23 s) 2023-09-09T23:15:26.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:26 smithi098 ceph-mon[103124]: mgrmap e25: x(active, since 91s), standbys: y, z 2023-09-09T23:15:26.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:26 smithi098 ceph-mon[103124]: pgmap v49: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 108 MiB/s wr, 641 op/s 2023-09-09T23:15:26.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:26 smithi170 ceph-mon[102832]: mgrmap e25: x(active, since 91s), standbys: y, z 2023-09-09T23:15:26.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:26 smithi170 ceph-mon[102832]: pgmap v49: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 108 MiB/s wr, 641 op/s 2023-09-09T23:15:26.954 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:26 smithi082 ceph-mon[98694]: mgrmap e25: x(active, since 91s), standbys: y, z 2023-09-09T23:15:26.954 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:26 smithi082 ceph-mon[98694]: pgmap v49: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 108 MiB/s wr, 641 op/s 2023-09-09T23:15:28.117 INFO:teuthology.orchestra.run.smithi082.stderr:30000000 of 50000000 tuples (60%) done (elapsed 74.70 s, remaining 49.80 s) 2023-09-09T23:15:28.183 INFO:teuthology.orchestra.run.smithi082.stderr:30100000 of 50000000 tuples (60%) done (elapsed 74.77 s, remaining 49.43 s) 2023-09-09T23:15:28.300 INFO:teuthology.orchestra.run.smithi082.stderr:30200000 of 50000000 tuples (60%) done (elapsed 74.88 s, remaining 49.10 s) 2023-09-09T23:15:28.413 INFO:teuthology.orchestra.run.smithi082.stderr:30300000 of 50000000 tuples (60%) done (elapsed 75.00 s, remaining 48.76 s) 2023-09-09T23:15:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:30400000 of 50000000 tuples (60%) done (elapsed 75.10 s, remaining 48.42 s) 2023-09-09T23:15:28.601 INFO:teuthology.orchestra.run.smithi082.stderr:30500000 of 50000000 tuples (61%) done (elapsed 75.19 s, remaining 48.07 s) 2023-09-09T23:15:28.702 INFO:teuthology.orchestra.run.smithi082.stderr:30600000 of 50000000 tuples (61%) done (elapsed 75.29 s, remaining 47.73 s) 2023-09-09T23:15:28.813 INFO:teuthology.orchestra.run.smithi082.stderr:30700000 of 50000000 tuples (61%) done (elapsed 75.40 s, remaining 47.40 s) 2023-09-09T23:15:28.906 INFO:teuthology.orchestra.run.smithi082.stderr:30800000 of 50000000 tuples (61%) done (elapsed 75.49 s, remaining 47.06 s) 2023-09-09T23:15:29.007 INFO:teuthology.orchestra.run.smithi082.stderr:30900000 of 50000000 tuples (61%) done (elapsed 75.59 s, remaining 46.72 s) 2023-09-09T23:15:29.123 INFO:teuthology.orchestra.run.smithi082.stderr:31000000 of 50000000 tuples (62%) done (elapsed 75.71 s, remaining 46.40 s) 2023-09-09T23:15:29.231 INFO:teuthology.orchestra.run.smithi082.stderr:31100000 of 50000000 tuples (62%) done (elapsed 75.82 s, remaining 46.07 s) 2023-09-09T23:15:29.339 INFO:teuthology.orchestra.run.smithi082.stderr:31200000 of 50000000 tuples (62%) done (elapsed 75.92 s, remaining 45.75 s) 2023-09-09T23:15:29.405 INFO:teuthology.orchestra.run.smithi082.stderr:31300000 of 50000000 tuples (62%) done (elapsed 75.99 s, remaining 45.40 s) 2023-09-09T23:15:29.523 INFO:teuthology.orchestra.run.smithi082.stderr:31400000 of 50000000 tuples (62%) done (elapsed 76.11 s, remaining 45.08 s) 2023-09-09T23:15:29.629 INFO:teuthology.orchestra.run.smithi082.stderr:31500000 of 50000000 tuples (63%) done (elapsed 76.21 s, remaining 44.76 s) 2023-09-09T23:15:29.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:29 smithi170 ceph-mon[102832]: pgmap v50: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 84 MiB/s wr, 341 op/s 2023-09-09T23:15:29.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:29 smithi082 ceph-mon[98694]: pgmap v50: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 84 MiB/s wr, 341 op/s 2023-09-09T23:15:29.713 INFO:teuthology.orchestra.run.smithi082.stderr:31600000 of 50000000 tuples (63%) done (elapsed 76.30 s, remaining 44.43 s) 2023-09-09T23:15:29.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:29 smithi098 ceph-mon[103124]: pgmap v50: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 84 MiB/s wr, 341 op/s 2023-09-09T23:15:29.830 INFO:teuthology.orchestra.run.smithi082.stderr:31700000 of 50000000 tuples (63%) done (elapsed 76.42 s, remaining 44.11 s) 2023-09-09T23:15:29.930 INFO:teuthology.orchestra.run.smithi082.stderr:31800000 of 50000000 tuples (63%) done (elapsed 76.51 s, remaining 43.79 s) 2023-09-09T23:15:30.033 INFO:teuthology.orchestra.run.smithi082.stderr:31900000 of 50000000 tuples (63%) done (elapsed 76.62 s, remaining 43.47 s) 2023-09-09T23:15:30.153 INFO:teuthology.orchestra.run.smithi082.stderr:32000000 of 50000000 tuples (64%) done (elapsed 76.74 s, remaining 43.16 s) 2023-09-09T23:15:30.275 INFO:teuthology.orchestra.run.smithi082.stderr:32100000 of 50000000 tuples (64%) done (elapsed 76.86 s, remaining 42.86 s) 2023-09-09T23:15:30.354 INFO:teuthology.orchestra.run.smithi082.stderr:32200000 of 50000000 tuples (64%) done (elapsed 76.94 s, remaining 42.53 s) 2023-09-09T23:15:30.462 INFO:teuthology.orchestra.run.smithi082.stderr:32300000 of 50000000 tuples (64%) done (elapsed 77.05 s, remaining 42.22 s) 2023-09-09T23:15:30.568 INFO:teuthology.orchestra.run.smithi082.stderr:32400000 of 50000000 tuples (64%) done (elapsed 77.15 s, remaining 41.91 s) 2023-09-09T23:15:30.666 INFO:teuthology.orchestra.run.smithi082.stderr:32500000 of 50000000 tuples (65%) done (elapsed 77.25 s, remaining 41.60 s) 2023-09-09T23:15:30.769 INFO:teuthology.orchestra.run.smithi082.stderr:32600000 of 50000000 tuples (65%) done (elapsed 77.35 s, remaining 41.29 s) 2023-09-09T23:15:30.851 INFO:teuthology.orchestra.run.smithi082.stderr:32700000 of 50000000 tuples (65%) done (elapsed 77.44 s, remaining 40.97 s) 2023-09-09T23:15:30.978 INFO:teuthology.orchestra.run.smithi082.stderr:32800000 of 50000000 tuples (65%) done (elapsed 77.56 s, remaining 40.67 s) 2023-09-09T23:15:31.081 INFO:teuthology.orchestra.run.smithi082.stderr:32900000 of 50000000 tuples (65%) done (elapsed 77.67 s, remaining 40.37 s) 2023-09-09T23:15:31.161 INFO:teuthology.orchestra.run.smithi082.stderr:33000000 of 50000000 tuples (66%) done (elapsed 77.75 s, remaining 40.05 s) 2023-09-09T23:15:31.262 INFO:teuthology.orchestra.run.smithi082.stderr:33100000 of 50000000 tuples (66%) done (elapsed 77.85 s, remaining 39.75 s) 2023-09-09T23:15:31.412 INFO:teuthology.orchestra.run.smithi082.stderr:33200000 of 50000000 tuples (66%) done (elapsed 78.00 s, remaining 39.47 s) 2023-09-09T23:15:31.499 INFO:teuthology.orchestra.run.smithi082.stderr:33300000 of 50000000 tuples (66%) done (elapsed 78.08 s, remaining 39.16 s) 2023-09-09T23:15:31.596 INFO:teuthology.orchestra.run.smithi082.stderr:33400000 of 50000000 tuples (66%) done (elapsed 78.18 s, remaining 38.86 s) 2023-09-09T23:15:31.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:31 smithi170 ceph-mon[102832]: pgmap v51: 97 pgs: 97 active+clean; 5.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 102 MiB/s wr, 507 op/s 2023-09-09T23:15:31.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:31 smithi082 ceph-mon[98694]: pgmap v51: 97 pgs: 97 active+clean; 5.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 102 MiB/s wr, 507 op/s 2023-09-09T23:15:31.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:31 smithi098 ceph-mon[103124]: pgmap v51: 97 pgs: 97 active+clean; 5.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 102 MiB/s wr, 507 op/s 2023-09-09T23:15:33.039 INFO:teuthology.orchestra.run.smithi082.stderr:33500000 of 50000000 tuples (67%) done (elapsed 79.62 s, remaining 39.22 s) 2023-09-09T23:15:33.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:33 smithi170 ceph-mon[102832]: pgmap v52: 97 pgs: 97 active+clean; 5.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 73 MiB/s wr, 405 op/s 2023-09-09T23:15:33.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:33 smithi082 ceph-mon[98694]: pgmap v52: 97 pgs: 97 active+clean; 5.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 73 MiB/s wr, 405 op/s 2023-09-09T23:15:33.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:33 smithi098 ceph-mon[103124]: pgmap v52: 97 pgs: 97 active+clean; 5.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 73 MiB/s wr, 405 op/s 2023-09-09T23:15:35.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:35 smithi170 ceph-mon[102832]: pgmap v53: 97 pgs: 97 active+clean; 5.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 73 MiB/s wr, 405 op/s 2023-09-09T23:15:35.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:35 smithi082 ceph-mon[98694]: pgmap v53: 97 pgs: 97 active+clean; 5.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 73 MiB/s wr, 405 op/s 2023-09-09T23:15:35.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:35 smithi098 ceph-mon[103124]: pgmap v53: 97 pgs: 97 active+clean; 5.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 73 MiB/s wr, 405 op/s 2023-09-09T23:15:36.268 INFO:teuthology.orchestra.run.smithi082.stderr:33600000 of 50000000 tuples (67%) done (elapsed 82.85 s, remaining 40.44 s) 2023-09-09T23:15:37.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:37 smithi170 ceph-mon[102832]: pgmap v54: 97 pgs: 97 active+clean; 6.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 130 MiB/s wr, 560 op/s 2023-09-09T23:15:37.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:37 smithi082 ceph-mon[98694]: pgmap v54: 97 pgs: 97 active+clean; 6.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 130 MiB/s wr, 560 op/s 2023-09-09T23:15:37.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:37 smithi098 ceph-mon[103124]: pgmap v54: 97 pgs: 97 active+clean; 6.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 130 MiB/s wr, 560 op/s 2023-09-09T23:15:39.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:39 smithi170 ceph-mon[102832]: pgmap v55: 97 pgs: 97 active+clean; 6.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 75 MiB/s wr, 321 op/s 2023-09-09T23:15:39.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:39 smithi082 ceph-mon[98694]: pgmap v55: 97 pgs: 97 active+clean; 6.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 75 MiB/s wr, 321 op/s 2023-09-09T23:15:39.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:39 smithi098 ceph-mon[103124]: pgmap v55: 97 pgs: 97 active+clean; 6.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 75 MiB/s wr, 321 op/s 2023-09-09T23:15:39.847 INFO:teuthology.orchestra.run.smithi082.stderr:33700000 of 50000000 tuples (67%) done (elapsed 86.43 s, remaining 41.81 s) 2023-09-09T23:15:39.922 INFO:teuthology.orchestra.run.smithi082.stderr:33800000 of 50000000 tuples (67%) done (elapsed 86.51 s, remaining 41.46 s) 2023-09-09T23:15:40.033 INFO:teuthology.orchestra.run.smithi082.stderr:33900000 of 50000000 tuples (67%) done (elapsed 86.62 s, remaining 41.14 s) 2023-09-09T23:15:40.134 INFO:teuthology.orchestra.run.smithi082.stderr:34000000 of 50000000 tuples (68%) done (elapsed 86.72 s, remaining 40.81 s) 2023-09-09T23:15:40.241 INFO:teuthology.orchestra.run.smithi082.stderr:34100000 of 50000000 tuples (68%) done (elapsed 86.83 s, remaining 40.48 s) 2023-09-09T23:15:40.337 INFO:teuthology.orchestra.run.smithi082.stderr:34200000 of 50000000 tuples (68%) done (elapsed 86.92 s, remaining 40.16 s) 2023-09-09T23:15:40.442 INFO:teuthology.orchestra.run.smithi082.stderr:34300000 of 50000000 tuples (68%) done (elapsed 87.03 s, remaining 39.83 s) 2023-09-09T23:15:40.519 INFO:teuthology.orchestra.run.smithi082.stderr:34400000 of 50000000 tuples (68%) done (elapsed 87.10 s, remaining 39.50 s) 2023-09-09T23:15:40.626 INFO:teuthology.orchestra.run.smithi082.stderr:34500000 of 50000000 tuples (69%) done (elapsed 87.21 s, remaining 39.18 s) 2023-09-09T23:15:40.730 INFO:teuthology.orchestra.run.smithi082.stderr:34600000 of 50000000 tuples (69%) done (elapsed 87.31 s, remaining 38.86 s) 2023-09-09T23:15:40.838 INFO:teuthology.orchestra.run.smithi082.stderr:34700000 of 50000000 tuples (69%) done (elapsed 87.42 s, remaining 38.55 s) 2023-09-09T23:15:40.931 INFO:teuthology.orchestra.run.smithi082.stderr:34800000 of 50000000 tuples (69%) done (elapsed 87.52 s, remaining 38.23 s) 2023-09-09T23:15:41.015 INFO:teuthology.orchestra.run.smithi082.stderr:34900000 of 50000000 tuples (69%) done (elapsed 87.60 s, remaining 37.90 s) 2023-09-09T23:15:41.133 INFO:teuthology.orchestra.run.smithi082.stderr:35000000 of 50000000 tuples (70%) done (elapsed 87.72 s, remaining 37.59 s) 2023-09-09T23:15:41.234 INFO:teuthology.orchestra.run.smithi082.stderr:35100000 of 50000000 tuples (70%) done (elapsed 87.82 s, remaining 37.28 s) 2023-09-09T23:15:41.348 INFO:teuthology.orchestra.run.smithi082.stderr:35200000 of 50000000 tuples (70%) done (elapsed 87.93 s, remaining 36.97 s) 2023-09-09T23:15:41.444 INFO:teuthology.orchestra.run.smithi082.stderr:35300000 of 50000000 tuples (70%) done (elapsed 88.03 s, remaining 36.66 s) 2023-09-09T23:15:41.545 INFO:teuthology.orchestra.run.smithi082.stderr:35400000 of 50000000 tuples (70%) done (elapsed 88.13 s, remaining 36.35 s) 2023-09-09T23:15:41.619 INFO:teuthology.orchestra.run.smithi082.stderr:35500000 of 50000000 tuples (71%) done (elapsed 88.20 s, remaining 36.03 s) 2023-09-09T23:15:41.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:41 smithi170 ceph-mon[102832]: pgmap v56: 97 pgs: 97 active+clean; 6.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 80 MiB/s wr, 572 op/s 2023-09-09T23:15:41.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:41 smithi082 ceph-mon[98694]: pgmap v56: 97 pgs: 97 active+clean; 6.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 80 MiB/s wr, 572 op/s 2023-09-09T23:15:41.737 INFO:teuthology.orchestra.run.smithi082.stderr:35600000 of 50000000 tuples (71%) done (elapsed 88.32 s, remaining 35.73 s) 2023-09-09T23:15:41.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:41 smithi098 ceph-mon[103124]: pgmap v56: 97 pgs: 97 active+clean; 6.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 80 MiB/s wr, 572 op/s 2023-09-09T23:15:41.822 INFO:teuthology.orchestra.run.smithi082.stderr:35700000 of 50000000 tuples (71%) done (elapsed 88.41 s, remaining 35.41 s) 2023-09-09T23:15:41.931 INFO:teuthology.orchestra.run.smithi082.stderr:35800000 of 50000000 tuples (71%) done (elapsed 88.52 s, remaining 35.11 s) 2023-09-09T23:15:42.026 INFO:teuthology.orchestra.run.smithi082.stderr:35900000 of 50000000 tuples (71%) done (elapsed 88.61 s, remaining 34.80 s) 2023-09-09T23:15:42.140 INFO:teuthology.orchestra.run.smithi082.stderr:36000000 of 50000000 tuples (72%) done (elapsed 88.72 s, remaining 34.50 s) 2023-09-09T23:15:42.243 INFO:teuthology.orchestra.run.smithi082.stderr:36100000 of 50000000 tuples (72%) done (elapsed 88.83 s, remaining 34.20 s) 2023-09-09T23:15:42.350 INFO:teuthology.orchestra.run.smithi082.stderr:36200000 of 50000000 tuples (72%) done (elapsed 88.93 s, remaining 33.90 s) 2023-09-09T23:15:42.465 INFO:teuthology.orchestra.run.smithi082.stderr:36300000 of 50000000 tuples (72%) done (elapsed 89.05 s, remaining 33.61 s) 2023-09-09T23:15:42.564 INFO:teuthology.orchestra.run.smithi082.stderr:36400000 of 50000000 tuples (72%) done (elapsed 89.15 s, remaining 33.31 s) 2023-09-09T23:15:42.667 INFO:teuthology.orchestra.run.smithi082.stderr:36500000 of 50000000 tuples (73%) done (elapsed 89.25 s, remaining 33.01 s) 2023-09-09T23:15:43.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:43 smithi170 ceph-mon[102832]: pgmap v57: 97 pgs: 97 active+clean; 6.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 61 MiB/s wr, 406 op/s 2023-09-09T23:15:43.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:43 smithi082 ceph-mon[98694]: pgmap v57: 97 pgs: 97 active+clean; 6.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 61 MiB/s wr, 406 op/s 2023-09-09T23:15:43.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:43 smithi098 ceph-mon[103124]: pgmap v57: 97 pgs: 97 active+clean; 6.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 61 MiB/s wr, 406 op/s 2023-09-09T23:15:45.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:45 smithi170 ceph-mon[102832]: pgmap v58: 97 pgs: 97 active+clean; 6.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 61 MiB/s wr, 406 op/s 2023-09-09T23:15:45.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:45 smithi082 ceph-mon[98694]: pgmap v58: 97 pgs: 97 active+clean; 6.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 61 MiB/s wr, 406 op/s 2023-09-09T23:15:45.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:45 smithi098 ceph-mon[103124]: pgmap v58: 97 pgs: 97 active+clean; 6.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 61 MiB/s wr, 406 op/s 2023-09-09T23:15:47.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:47 smithi170 ceph-mon[102832]: pgmap v59: 97 pgs: 97 active+clean; 6.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 83 MiB/s wr, 420 op/s 2023-09-09T23:15:47.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:47 smithi082 ceph-mon[98694]: pgmap v59: 97 pgs: 97 active+clean; 6.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 83 MiB/s wr, 420 op/s 2023-09-09T23:15:47.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:47 smithi098 ceph-mon[103124]: pgmap v59: 97 pgs: 97 active+clean; 6.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 83 MiB/s wr, 420 op/s 2023-09-09T23:15:48.175 INFO:teuthology.orchestra.run.smithi082.stderr:36600000 of 50000000 tuples (73%) done (elapsed 94.76 s, remaining 34.69 s) 2023-09-09T23:15:48.627 INFO:teuthology.orchestra.run.smithi082.stderr:36700000 of 50000000 tuples (73%) done (elapsed 95.21 s, remaining 34.50 s) 2023-09-09T23:15:48.725 INFO:teuthology.orchestra.run.smithi082.stderr:36800000 of 50000000 tuples (73%) done (elapsed 95.31 s, remaining 34.19 s) 2023-09-09T23:15:48.828 INFO:teuthology.orchestra.run.smithi082.stderr:36900000 of 50000000 tuples (73%) done (elapsed 95.41 s, remaining 33.87 s) 2023-09-09T23:15:48.934 INFO:teuthology.orchestra.run.smithi082.stderr:37000000 of 50000000 tuples (74%) done (elapsed 95.52 s, remaining 33.56 s) 2023-09-09T23:15:48.999 INFO:teuthology.orchestra.run.smithi082.stderr:37100000 of 50000000 tuples (74%) done (elapsed 95.58 s, remaining 33.24 s) 2023-09-09T23:15:49.121 INFO:teuthology.orchestra.run.smithi082.stderr:37200000 of 50000000 tuples (74%) done (elapsed 95.71 s, remaining 32.93 s) 2023-09-09T23:15:49.227 INFO:teuthology.orchestra.run.smithi082.stderr:37300000 of 50000000 tuples (74%) done (elapsed 95.81 s, remaining 32.62 s) 2023-09-09T23:15:49.409 INFO:teuthology.orchestra.run.smithi082.stderr:37400000 of 50000000 tuples (74%) done (elapsed 95.99 s, remaining 32.34 s) 2023-09-09T23:15:49.577 INFO:teuthology.orchestra.run.smithi082.stderr:37500000 of 50000000 tuples (75%) done (elapsed 96.16 s, remaining 32.05 s) 2023-09-09T23:15:49.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:49 smithi170 ceph-mon[102832]: pgmap v60: 97 pgs: 97 active+clean; 6.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 MiB/s wr, 264 op/s 2023-09-09T23:15:49.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:49 smithi082 ceph-mon[98694]: pgmap v60: 97 pgs: 97 active+clean; 6.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 MiB/s wr, 264 op/s 2023-09-09T23:15:49.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:49 smithi098 ceph-mon[103124]: pgmap v60: 97 pgs: 97 active+clean; 6.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 MiB/s wr, 264 op/s 2023-09-09T23:15:50.286 INFO:teuthology.orchestra.run.smithi082.stderr:37600000 of 50000000 tuples (75%) done (elapsed 96.87 s, remaining 31.95 s) 2023-09-09T23:15:51.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:51 smithi170 ceph-mon[102832]: pgmap v61: 97 pgs: 97 active+clean; 6.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 70 MiB/s wr, 324 op/s 2023-09-09T23:15:51.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:51 smithi082 ceph-mon[98694]: pgmap v61: 97 pgs: 97 active+clean; 6.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 70 MiB/s wr, 324 op/s 2023-09-09T23:15:51.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:51 smithi098 ceph-mon[103124]: pgmap v61: 97 pgs: 97 active+clean; 6.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 70 MiB/s wr, 324 op/s 2023-09-09T23:15:51.974 INFO:teuthology.orchestra.run.smithi082.stderr:37700000 of 50000000 tuples (75%) done (elapsed 98.56 s, remaining 32.16 s) 2023-09-09T23:15:53.495 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:15:53.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.625+0000 7fa68198f700 1 -- 172.21.15.82:0/1139345401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa67c0ffe00 msgr2=0x7fa67c10c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:15:53.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.625+0000 7fa68198f700 1 --2- 172.21.15.82:0/1139345401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa67c0ffe00 0x7fa67c10c310 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa67400b0a0 tx=0x7fa67400b3b0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.625+0000 7fa68198f700 1 -- 172.21.15.82:0/1139345401 shutdown_connections 2023-09-09T23:15:53.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.625+0000 7fa68198f700 1 --2- 172.21.15.82:0/1139345401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa67c0ffe00 0x7fa67c10c310 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.625+0000 7fa68198f700 1 --2- 172.21.15.82:0/1139345401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa67c1054e0 0x7fa67c1058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.625+0000 7fa68198f700 1 --2- 172.21.15.82:0/1139345401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa67c0ff440 0x7fa67c0ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.625+0000 7fa68198f700 1 -- 172.21.15.82:0/1139345401 >> 172.21.15.82:0/1139345401 conn(0x7fa67c0faf50 msgr2=0x7fa67c0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:15:53.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.625+0000 7fa68198f700 1 -- 172.21.15.82:0/1139345401 shutdown_connections 2023-09-09T23:15:53.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.625+0000 7fa68198f700 1 -- 172.21.15.82:0/1139345401 wait complete. 2023-09-09T23:15:53.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.626+0000 7fa68198f700 1 Processor -- start 2023-09-09T23:15:53.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.626+0000 7fa68198f700 1 -- start start 2023-09-09T23:15:53.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.626+0000 7fa68198f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa67c0ff440 0x7fa67c19e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:53.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.626+0000 7fa68198f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa67c0ffe00 0x7fa67c19eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:53.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.626+0000 7fa68198f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa67c1054e0 0x7fa67c1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:53.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.626+0000 7fa68198f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c198c50 con 0x7fa67c0ff440 2023-09-09T23:15:53.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.626+0000 7fa68198f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c198dc0 con 0x7fa67c0ffe00 2023-09-09T23:15:53.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.626+0000 7fa68198f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c198f80 con 0x7fa67c1054e0 2023-09-09T23:15:53.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.626+0000 7fa67affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa67c0ff440 0x7fa67c19e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:53.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.626+0000 7fa67affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa67c0ff440 0x7fa67c19e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57582/0 (socket says 172.21.15.82:57582) 2023-09-09T23:15:53.630 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.626+0000 7fa67affd700 1 -- 172.21.15.82:0/2653344519 learned_addr learned my addr 172.21.15.82:0/2653344519 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:15:53.630 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.626+0000 7fa67a7fc700 1 --2- 172.21.15.82:0/2653344519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa67c0ffe00 0x7fa67c19eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:53.630 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.626+0000 7fa67b7fe700 1 --2- 172.21.15.82:0/2653344519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa67c1054e0 0x7fa67c1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:53.630 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.627+0000 7fa67affd700 1 -- 172.21.15.82:0/2653344519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa67c1054e0 msgr2=0x7fa67c1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:15:53.630 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.627+0000 7fa67affd700 1 --2- 172.21.15.82:0/2653344519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa67c1054e0 0x7fa67c1985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.627+0000 7fa67affd700 1 -- 172.21.15.82:0/2653344519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa67c0ffe00 msgr2=0x7fa67c19eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:15:53.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.627+0000 7fa67affd700 1 --2- 172.21.15.82:0/2653344519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa67c0ffe00 0x7fa67c19eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.627+0000 7fa67affd700 1 -- 172.21.15.82:0/2653344519 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa674009d00 con 0x7fa67c0ff440 2023-09-09T23:15:53.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.627+0000 7fa67affd700 1 --2- 172.21.15.82:0/2653344519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa67c0ff440 0x7fa67c19e570 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa67000e9a0 tx=0x7fa67000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:15:53.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.627+0000 7fa68098d700 1 -- 172.21.15.82:0/2653344519 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670009790 con 0x7fa67c0ff440 2023-09-09T23:15:53.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.627+0000 7fa68098d700 1 -- 172.21.15.82:0/2653344519 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa670010430 con 0x7fa67c0ff440 2023-09-09T23:15:53.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.627+0000 7fa68198f700 1 -- 172.21.15.82:0/2653344519 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa67c199260 con 0x7fa67c0ff440 2023-09-09T23:15:53.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.627+0000 7fa68098d700 1 -- 172.21.15.82:0/2653344519 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670018680 con 0x7fa67c0ff440 2023-09-09T23:15:53.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.627+0000 7fa68198f700 1 -- 172.21.15.82:0/2653344519 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa67c1a5560 con 0x7fa67c0ff440 2023-09-09T23:15:53.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.629+0000 7fa68098d700 1 -- 172.21.15.82:0/2653344519 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fa6700105a0 con 0x7fa67c0ff440 2023-09-09T23:15:53.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.630+0000 7fa68098d700 1 --2- 172.21.15.82:0/2653344519 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fa6640a07b0 0x7fa6640a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:53.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.630+0000 7fa67a7fc700 1 --2- 172.21.15.82:0/2653344519 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fa6640a07b0 0x7fa6640a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:53.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.630+0000 7fa68098d700 1 -- 172.21.15.82:0/2653344519 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fa670014070 con 0x7fa67c0ff440 2023-09-09T23:15:53.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.630+0000 7fa67a7fc700 1 --2- 172.21.15.82:0/2653344519 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fa6640a07b0 0x7fa6640a2c70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa67c104cb0 tx=0x7fa66c009260 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:15:53.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.631+0000 7fa68db55b80 1 Processor -- start 2023-09-09T23:15:53.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.631+0000 7fa68db55b80 1 -- start start 2023-09-09T23:15:53.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.631+0000 7fa68db55b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56078164c710 0x56078164caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:53.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.631+0000 7fa68db55b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56078164d030 0x560781656900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:53.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.631+0000 7fa68db55b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560781656e40 0x560781659230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:53.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.631+0000 7fa68db55b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607815a5730 con 0x56078164c710 2023-09-09T23:15:53.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.631+0000 7fa68db55b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607815a3050 con 0x56078164d030 2023-09-09T23:15:53.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.631+0000 7fa68db55b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56078159e940 con 0x560781656e40 2023-09-09T23:15:53.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.631+0000 7fa67affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56078164c710 0x56078164caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:53.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.631+0000 7fa67affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56078164c710 0x56078164caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57590/0 (socket says 172.21.15.82:57590) 2023-09-09T23:15:53.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.631+0000 7fa67affd700 1 -- 172.21.15.82:0/1131300167 learned_addr learned my addr 172.21.15.82:0/1131300167 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:15:53.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.631+0000 7fa67a7fc700 1 --2- 172.21.15.82:0/1131300167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56078164d030 0x560781656900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:53.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.631+0000 7fa67b7fe700 1 --2- 172.21.15.82:0/1131300167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560781656e40 0x560781659230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:53.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.631+0000 7fa67affd700 1 -- 172.21.15.82:0/1131300167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560781656e40 msgr2=0x560781659230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:15:53.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.631+0000 7fa67affd700 1 --2- 172.21.15.82:0/1131300167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560781656e40 0x560781659230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.631+0000 7fa67affd700 1 -- 172.21.15.82:0/1131300167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56078164d030 msgr2=0x560781656900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:15:53.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.631+0000 7fa67affd700 1 --2- 172.21.15.82:0/1131300167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56078164d030 0x560781656900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.642 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.631+0000 7fa67affd700 1 -- 172.21.15.82:0/1131300167 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607813cade0 con 0x56078164c710 2023-09-09T23:15:53.642 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.631+0000 7fa67affd700 1 --2- 172.21.15.82:0/1131300167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56078164c710 0x56078164caf0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa6700a10a0 tx=0x7fa6700991b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff21e26a16c154a server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:15:53.642 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.631+0000 7fa661ffb700 1 -- 172.21.15.82:0/1131300167 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67009d390 con 0x56078164c710 2023-09-09T23:15:53.642 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.632+0000 7fa661ffb700 1 -- 172.21.15.82:0/1131300167 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6700a15d0 con 0x56078164c710 2023-09-09T23:15:53.643 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.632+0000 7fa661ffb700 1 -- 172.21.15.82:0/1131300167 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670099a20 con 0x56078164c710 2023-09-09T23:15:53.643 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.632+0000 7fa68db55b80 1 -- 172.21.15.82:0/1131300167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56078164c710 msgr2=0x56078164caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:15:53.643 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.632+0000 7fa68db55b80 1 --2- 172.21.15.82:0/1131300167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56078164c710 0x56078164caf0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa6700a10a0 tx=0x7fa6700991b0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.643 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.632+0000 7fa68db55b80 1 -- 172.21.15.82:0/1131300167 shutdown_connections 2023-09-09T23:15:53.643 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.632+0000 7fa68db55b80 1 --2- 172.21.15.82:0/1131300167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560781656e40 0x560781659230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.644 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.632+0000 7fa68db55b80 1 --2- 172.21.15.82:0/1131300167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56078164c710 0x56078164caf0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.644 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.632+0000 7fa68db55b80 1 --2- 172.21.15.82:0/1131300167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56078164d030 0x560781656900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.645 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.632+0000 7fa68db55b80 1 -- 172.21.15.82:0/1131300167 >> 172.21.15.82:0/1131300167 conn(0x5607815c1090 msgr2=0x560781647bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:15:53.646 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.632+0000 7fa68db55b80 1 -- 172.21.15.82:0/1131300167 shutdown_connections 2023-09-09T23:15:53.646 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.632+0000 7fa68db55b80 1 -- 172.21.15.82:0/1131300167 wait complete. 2023-09-09T23:15:53.646 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.633+0000 7fa68db55b80 1 -- start start 2023-09-09T23:15:53.646 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.633+0000 7fa68db55b80 1 Processor -- start 2023-09-09T23:15:53.646 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.633+0000 7fa68db55b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56078164c710 0x560781653ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:53.647 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.633+0000 7fa68db55b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56078164d030 0x5607816540e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:53.647 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.633+0000 7fa68db55b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560781656e40 0x560781654620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:53.647 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.633+0000 7fa68db55b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56078153cb70 con 0x56078164d030 2023-09-09T23:15:53.647 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.633+0000 7fa68db55b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56078153c870 con 0x560781656e40 2023-09-09T23:15:53.647 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.633+0000 7fa68db55b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560781420920 con 0x56078164c710 2023-09-09T23:15:53.647 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.633+0000 7fa67a7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56078164d030 0x5607816540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:53.648 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.633+0000 7fa68db55b80 10 client.0 _ll_register_callbacks cb 0x560781486a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:15:53.648 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.633+0000 7fa67a7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56078164d030 0x5607816540e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57598/0 (socket says 172.21.15.82:57598) 2023-09-09T23:15:53.648 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.633+0000 7fa67a7fc700 1 -- 172.21.15.82:0/1447660085 learned_addr learned my addr 172.21.15.82:0/1447660085 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:15:53.648 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.633+0000 7fa67affd700 1 --2- 172.21.15.82:0/1447660085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56078164c710 0x560781653ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:53.648 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.633+0000 7fa67b7fe700 1 --2- 172.21.15.82:0/1447660085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560781656e40 0x560781654620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:53.649 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.634+0000 7fa67a7fc700 1 -- 172.21.15.82:0/1447660085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56078164c710 msgr2=0x560781653ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:15:53.649 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.634+0000 7fa67a7fc700 1 --2- 172.21.15.82:0/1447660085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56078164c710 0x560781653ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.649 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.634+0000 7fa67a7fc700 1 -- 172.21.15.82:0/1447660085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560781656e40 msgr2=0x560781654620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:15:53.649 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.634+0000 7fa67a7fc700 1 --2- 172.21.15.82:0/1447660085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560781656e40 0x560781654620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.650 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.634+0000 7fa67a7fc700 1 -- 172.21.15.82:0/1447660085 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa67c1a5560 con 0x56078164d030 2023-09-09T23:15:53.650 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.634+0000 7fa67a7fc700 1 --2- 172.21.15.82:0/1447660085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56078164d030 0x5607816540e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa66c005910 tx=0x7fa66c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43dbad43072aaa92 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:15:53.650 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.634+0000 7fa65affd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:15:53.650 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.634+0000 7fa65affd700 1 -- 172.21.15.82:0/1447660085 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c022070 con 0x56078164d030 2023-09-09T23:15:53.650 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.634+0000 7fa65affd700 1 -- 172.21.15.82:0/1447660085 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fa66c004510 con 0x56078164d030 2023-09-09T23:15:53.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.634+0000 7fa68db55b80 1 -- 172.21.15.82:0/1447660085 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607814f64f0 con 0x56078164d030 2023-09-09T23:15:53.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.634+0000 7fa65affd700 1 -- 172.21.15.82:0/1447660085 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa66c027420 con 0x56078164d030 2023-09-09T23:15:53.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.634+0000 7fa68db55b80 10 client.15996 fetch_fsmap learned FSMap version 32 2023-09-09T23:15:53.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.634+0000 7fa68db55b80 1 -- 172.21.15.82:0/1447660085 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5607813b3c60 con 0x56078164d030 2023-09-09T23:15:53.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.634+0000 7fa65affd700 1 -- 172.21.15.82:0/1447660085 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5607814f64f0 con 0x56078164d030 2023-09-09T23:15:53.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.634+0000 7fa65affd700 1 -- 172.21.15.82:0/1447660085 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fa66c030690 con 0x56078164d030 2023-09-09T23:15:53.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.634+0000 7fa68db55b80 10 client.15996 fetch_fsmap finished waiting for FSMap version 32 2023-09-09T23:15:53.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.634+0000 7fa68db55b80 10 client.15996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:15:53.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.635+0000 7fa68db55b80 20 client.15996 populate_metadata read hostname 'smithi082' 2023-09-09T23:15:53.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.635+0000 7fa68db55b80 1 --2- 172.21.15.82:0/1447660085 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x560781666b30 0x560781668f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:53.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.635+0000 7fa68db55b80 4 client.15996 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:15:53.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.635+0000 7fa68db55b80 1 -- 172.21.15.82:0/1447660085 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5607815b38a0 con 0x560781666b30 2023-09-09T23:15:53.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.635+0000 7fa67affd700 1 --2- 172.21.15.82:0/1447660085 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x560781666b30 0x560781668f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:53.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.635+0000 7fa65affd700 10 client.15996 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:15:53.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.635+0000 7fa67affd700 1 --2- 172.21.15.82:0/1447660085 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x560781666b30 0x560781668f50 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8ce93f1816f193f server_cookie=4b390f63f160f95 in_seq=0 out_seq=0 2023-09-09T23:15:53.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.636+0000 7fa65affd700 1 -- 172.21.15.82:0/1447660085 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5607815b38a0 con 0x560781666b30 2023-09-09T23:15:53.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.636+0000 7fa65affd700 10 client.15996 handle_command_reply: tid=0 2023-09-09T23:15:53.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.636+0000 7fa68db55b80 1 client.15996 shutdown 2023-09-09T23:15:53.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.636+0000 7fa68db55b80 1 -- 172.21.15.82:0/1447660085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56078164d030 msgr2=0x5607816540e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:15:53.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.636+0000 7fa68db55b80 1 --2- 172.21.15.82:0/1447660085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56078164d030 0x5607816540e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa66c005910 tx=0x7fa66c0047c0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.636+0000 7fa68db55b80 1 -- 172.21.15.82:0/1447660085 shutdown_connections 2023-09-09T23:15:53.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.636+0000 7fa68db55b80 1 --2- 172.21.15.82:0/1447660085 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x560781666b30 0x560781668f50 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.636+0000 7fa68db55b80 1 --2- 172.21.15.82:0/1447660085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560781656e40 0x560781654620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.636+0000 7fa68db55b80 1 --2- 172.21.15.82:0/1447660085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56078164c710 0x560781653ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.636+0000 7fa65affd700 0 client.15996 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:15:53.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.636+0000 7fa68db55b80 1 --2- 172.21.15.82:0/1447660085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56078164d030 0x5607816540e0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.636+0000 7fa68db55b80 1 -- 172.21.15.82:0/1447660085 >> 172.21.15.82:0/1447660085 conn(0x5607815c1090 msgr2=0x5607816477e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:15:53.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.636+0000 7fa68db55b80 1 -- 172.21.15.82:0/1447660085 shutdown_connections 2023-09-09T23:15:53.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.636+0000 7fa68db55b80 1 -- 172.21.15.82:0/1447660085 wait complete. 2023-09-09T23:15:53.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.636+0000 7fa68db55b80 20 client.15996 trim_cache size 0 max 16384 2023-09-09T23:15:53.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.653+0000 7fa68db55b80 1 Processor -- start 2023-09-09T23:15:53.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.653+0000 7fa68db55b80 1 -- start start 2023-09-09T23:15:53.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.653+0000 7fa68db55b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56078164c710 0x560781654450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:53.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.653+0000 7fa68db55b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56078164d030 0x5607816476f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:53.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.653+0000 7fa68db55b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560781656e40 0x560781647c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:53.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.653+0000 7fa68db55b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607816633e0 con 0x56078164c710 2023-09-09T23:15:53.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.653+0000 7fa68db55b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56078163f400 con 0x56078164d030 2023-09-09T23:15:53.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.653+0000 7fa68db55b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607816640e0 con 0x560781656e40 2023-09-09T23:15:53.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.653+0000 7fa67affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56078164c710 0x560781654450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:53.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.653+0000 7fa67affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56078164c710 0x560781654450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57604/0 (socket says 172.21.15.82:57604) 2023-09-09T23:15:53.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.653+0000 7fa67affd700 1 -- 172.21.15.82:0/1334090629 learned_addr learned my addr 172.21.15.82:0/1334090629 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:15:53.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.653+0000 7fa67a7fc700 1 --2- 172.21.15.82:0/1334090629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56078164d030 0x5607816476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:53.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.653+0000 7fa67b7fe700 1 --2- 172.21.15.82:0/1334090629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560781656e40 0x560781647c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:53.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.653+0000 7fa67affd700 1 -- 172.21.15.82:0/1334090629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560781656e40 msgr2=0x560781647c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:15:53.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.653+0000 7fa67affd700 1 --2- 172.21.15.82:0/1334090629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560781656e40 0x560781647c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.653+0000 7fa67affd700 1 -- 172.21.15.82:0/1334090629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56078164d030 msgr2=0x5607816476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:15:53.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.653+0000 7fa67affd700 1 --2- 172.21.15.82:0/1334090629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56078164d030 0x5607816476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.653+0000 7fa67affd700 1 -- 172.21.15.82:0/1334090629 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607813f1250 con 0x56078164c710 2023-09-09T23:15:53.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.653+0000 7fa67affd700 1 --2- 172.21.15.82:0/1334090629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56078164c710 0x560781654450 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa6700952e0 tx=0x7fa670084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f548bb551ee6734e server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:15:53.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.654+0000 7fa6627fc700 1 -- 172.21.15.82:0/1334090629 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670097900 con 0x56078164c710 2023-09-09T23:15:53.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.654+0000 7fa6627fc700 1 -- 172.21.15.82:0/1334090629 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa670010880 con 0x56078164c710 2023-09-09T23:15:53.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.654+0000 7fa6627fc700 1 -- 172.21.15.82:0/1334090629 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670099a20 con 0x56078164c710 2023-09-09T23:15:53.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.654+0000 7fa68db55b80 1 -- 172.21.15.82:0/1334090629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56078164c710 msgr2=0x560781654450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:15:53.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.654+0000 7fa68db55b80 1 --2- 172.21.15.82:0/1334090629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56078164c710 0x560781654450 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa6700952e0 tx=0x7fa670084040 comp rx=0 tx=0).stop 2023-09-09T23:15:53.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.654+0000 7fa68db55b80 1 -- 172.21.15.82:0/1334090629 shutdown_connections 2023-09-09T23:15:53.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.654+0000 7fa68db55b80 1 --2- 172.21.15.82:0/1334090629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560781656e40 0x560781647c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.654+0000 7fa68db55b80 1 --2- 172.21.15.82:0/1334090629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56078164c710 0x560781654450 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.654+0000 7fa68db55b80 1 --2- 172.21.15.82:0/1334090629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56078164d030 0x5607816476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.654+0000 7fa68db55b80 1 -- 172.21.15.82:0/1334090629 >> 172.21.15.82:0/1334090629 conn(0x5607815c1090 msgr2=0x56078164eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:15:53.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.654+0000 7fa68db55b80 1 -- 172.21.15.82:0/1334090629 shutdown_connections 2023-09-09T23:15:53.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.654+0000 7fa68db55b80 1 -- 172.21.15.82:0/1334090629 wait complete. 2023-09-09T23:15:53.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.655+0000 7fa68db55b80 1 -- start start 2023-09-09T23:15:53.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.655+0000 7fa68db55b80 1 Processor -- start 2023-09-09T23:15:53.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.655+0000 7fa68db55b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56078164c710 0x56078165da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:53.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.655+0000 7fa68db55b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56078164d030 0x56078165be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:53.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.655+0000 7fa68db55b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560781656e40 0x56078165c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:53.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.655+0000 7fa68db55b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56078165ca50 con 0x56078164c710 2023-09-09T23:15:53.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.655+0000 7fa68db55b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56078165cbc0 con 0x56078164d030 2023-09-09T23:15:53.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.655+0000 7fa68db55b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607816ba950 con 0x560781656e40 2023-09-09T23:15:53.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.655+0000 7fa68db55b80 10 client.0 _ll_register_callbacks cb 0x560781486a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:15:53.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.655+0000 7fa67affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56078164c710 0x56078165da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:53.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.655+0000 7fa67affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56078164c710 0x56078165da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57610/0 (socket says 172.21.15.82:57610) 2023-09-09T23:15:53.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.655+0000 7fa67affd700 1 -- 172.21.15.82:0/3311679194 learned_addr learned my addr 172.21.15.82:0/3311679194 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:15:53.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.655+0000 7fa67a7fc700 1 --2- 172.21.15.82:0/3311679194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56078164d030 0x56078165be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:53.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.655+0000 7fa67b7fe700 1 --2- 172.21.15.82:0/3311679194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560781656e40 0x56078165c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:53.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.655+0000 7fa67affd700 1 -- 172.21.15.82:0/3311679194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560781656e40 msgr2=0x56078165c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:15:53.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.655+0000 7fa67affd700 1 --2- 172.21.15.82:0/3311679194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560781656e40 0x56078165c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.655+0000 7fa67affd700 1 -- 172.21.15.82:0/3311679194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56078164d030 msgr2=0x56078165be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:15:53.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.655+0000 7fa67affd700 1 --2- 172.21.15.82:0/3311679194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56078164d030 0x56078165be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.655+0000 7fa67affd700 1 -- 172.21.15.82:0/3311679194 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5607813cade0 con 0x56078164c710 2023-09-09T23:15:53.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.655+0000 7fa67affd700 1 --2- 172.21.15.82:0/3311679194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56078164c710 0x56078165da20 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa670099260 tx=0x7fa670005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9238576d9908afae server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:15:53.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.656+0000 7fa65affd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:15:53.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.656+0000 7fa65affd700 1 -- 172.21.15.82:0/3311679194 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6700109f0 con 0x56078164c710 2023-09-09T23:15:53.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.656+0000 7fa65affd700 1 -- 172.21.15.82:0/3311679194 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fa670098bd0 con 0x56078164c710 2023-09-09T23:15:53.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.656+0000 7fa68db55b80 1 -- 172.21.15.82:0/3311679194 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6700a15d0 con 0x56078164c710 2023-09-09T23:15:53.672 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.656+0000 7fa65affd700 1 -- 172.21.15.82:0/3311679194 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa670065250 con 0x56078164c710 2023-09-09T23:15:53.672 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.656+0000 7fa68db55b80 10 client.16002 fetch_fsmap learned FSMap version 32 2023-09-09T23:15:53.672 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.656+0000 7fa68db55b80 1 -- 172.21.15.82:0/3311679194 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5607813c92b0 con 0x56078164c710 2023-09-09T23:15:53.672 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.656+0000 7fa65affd700 1 -- 172.21.15.82:0/3311679194 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6700a15d0 con 0x56078164c710 2023-09-09T23:15:53.672 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.656+0000 7fa65affd700 1 -- 172.21.15.82:0/3311679194 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fa6700a1140 con 0x56078164c710 2023-09-09T23:15:53.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.656+0000 7fa68db55b80 10 client.16002 fetch_fsmap finished waiting for FSMap version 32 2023-09-09T23:15:53.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.656+0000 7fa68db55b80 10 client.16002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:15:53.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.656+0000 7fa68db55b80 20 client.16002 populate_metadata read hostname 'smithi082' 2023-09-09T23:15:53.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.656+0000 7fa68db55b80 1 --2- 172.21.15.82:0/3311679194 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5607816bbc50 0x5607816be070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:15:53.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.656+0000 7fa68db55b80 4 client.16002 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:15:53.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.656+0000 7fa68db55b80 1 -- 172.21.15.82:0/3311679194 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560781597920 con 0x5607816bbc50 2023-09-09T23:15:53.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.656+0000 7fa67a7fc700 1 --2- 172.21.15.82:0/3311679194 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5607816bbc50 0x5607816be070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:15:53.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.657+0000 7fa65affd700 10 client.16002 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:15:53.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.657+0000 7fa67a7fc700 1 --2- 172.21.15.82:0/3311679194 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5607816bbc50 0x5607816be070 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebe5888ba53c9eee server_cookie=f55e6ab63dea0a14 in_seq=0 out_seq=0 2023-09-09T23:15:53.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.657+0000 7fa65affd700 1 -- 172.21.15.82:0/3311679194 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x560781597920 con 0x5607816bbc50 2023-09-09T23:15:53.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.657+0000 7fa65affd700 10 client.16002 handle_command_reply: tid=0 2023-09-09T23:15:53.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.657+0000 7fa68db55b80 1 client.16002 shutdown 2023-09-09T23:15:53.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.657+0000 7fa68db55b80 1 -- 172.21.15.82:0/3311679194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56078164c710 msgr2=0x56078165da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:15:53.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.657+0000 7fa68db55b80 1 --2- 172.21.15.82:0/3311679194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56078164c710 0x56078165da20 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa670099260 tx=0x7fa670005b40 comp rx=0 tx=0).stop 2023-09-09T23:15:53.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.657+0000 7fa68db55b80 1 -- 172.21.15.82:0/3311679194 shutdown_connections 2023-09-09T23:15:53.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.657+0000 7fa68db55b80 1 --2- 172.21.15.82:0/3311679194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560781656e40 0x56078165c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.657+0000 7fa68db55b80 1 --2- 172.21.15.82:0/3311679194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56078164c710 0x56078165da20 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.657+0000 7fa68db55b80 1 --2- 172.21.15.82:0/3311679194 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5607816bbc50 0x5607816be070 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.657+0000 7fa68db55b80 1 --2- 172.21.15.82:0/3311679194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56078164d030 0x56078165be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.657+0000 7fa65affd700 0 client.16002 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:15:53.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.657+0000 7fa68db55b80 1 -- 172.21.15.82:0/3311679194 >> 172.21.15.82:0/3311679194 conn(0x5607815c1090 msgr2=0x560781647670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:15:53.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.657+0000 7fa68db55b80 1 -- 172.21.15.82:0/3311679194 shutdown_connections 2023-09-09T23:15:53.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.657+0000 7fa68db55b80 1 -- 172.21.15.82:0/3311679194 wait complete. 2023-09-09T23:15:53.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.657+0000 7fa68db55b80 20 client.16002 trim_cache size 0 max 16384 2023-09-09T23:15:53.677 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:15:53.678 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:15:53.678 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:15:53.678 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:15:53.678 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:15:53.678 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:15:53.679 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:15:53.679 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:15:53.679 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:15:53.679 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:15:53.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.658+0000 7fa68198f700 1 -- 172.21.15.82:0/2653344519 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fa6640a07b0 msgr2=0x7fa6640a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:15:53.681 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.658+0000 7fa68198f700 1 --2- 172.21.15.82:0/2653344519 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fa6640a07b0 0x7fa6640a2c70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa67c104cb0 tx=0x7fa66c009260 comp rx=0 tx=0).stop 2023-09-09T23:15:53.681 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.658+0000 7fa68198f700 1 -- 172.21.15.82:0/2653344519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa67c0ff440 msgr2=0x7fa67c19e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:15:53.681 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.658+0000 7fa68198f700 1 --2- 172.21.15.82:0/2653344519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa67c0ff440 0x7fa67c19e570 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa67000e9a0 tx=0x7fa67000ecb0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.681 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.658+0000 7fa68198f700 1 -- 172.21.15.82:0/2653344519 shutdown_connections 2023-09-09T23:15:53.681 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.658+0000 7fa68198f700 1 --2- 172.21.15.82:0/2653344519 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fa6640a07b0 0x7fa6640a2c70 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.658+0000 7fa68198f700 1 --2- 172.21.15.82:0/2653344519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa67c1054e0 0x7fa67c1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.658+0000 7fa68198f700 1 --2- 172.21.15.82:0/2653344519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa67c0ff440 0x7fa67c19e570 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.658+0000 7fa68198f700 1 --2- 172.21.15.82:0/2653344519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa67c0ffe00 0x7fa67c19eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:15:53.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.658+0000 7fa68198f700 1 -- 172.21.15.82:0/2653344519 >> 172.21.15.82:0/2653344519 conn(0x7fa67c0faf50 msgr2=0x7fa67c10abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:15:53.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.658+0000 7fa68198f700 1 -- 172.21.15.82:0/2653344519 shutdown_connections 2023-09-09T23:15:53.683 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:15:53.658+0000 7fa68198f700 1 -- 172.21.15.82:0/2653344519 wait complete. 2023-09-09T23:15:53.683 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:15:53.683 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:53 smithi082 ceph-mon[98694]: pgmap v62: 97 pgs: 97 active+clean; 6.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 70 MiB/s wr, 75 op/s 2023-09-09T23:15:53.684 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:53 smithi170 ceph-mon[102832]: pgmap v62: 97 pgs: 97 active+clean; 6.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 70 MiB/s wr, 75 op/s 2023-09-09T23:15:53.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:53 smithi098 ceph-mon[103124]: pgmap v62: 97 pgs: 97 active+clean; 6.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 70 MiB/s wr, 75 op/s 2023-09-09T23:15:55.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:55 smithi170 ceph-mon[102832]: pgmap v63: 97 pgs: 97 active+clean; 6.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 70 MiB/s wr, 75 op/s 2023-09-09T23:15:55.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:55 smithi082 ceph-mon[98694]: pgmap v63: 97 pgs: 97 active+clean; 6.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 70 MiB/s wr, 75 op/s 2023-09-09T23:15:55.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:55 smithi098 ceph-mon[103124]: pgmap v63: 97 pgs: 97 active+clean; 6.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 70 MiB/s wr, 75 op/s 2023-09-09T23:15:56.652 INFO:teuthology.orchestra.run.smithi082.stderr:37800000 of 50000000 tuples (75%) done (elapsed 103.24 s, remaining 33.32 s) 2023-09-09T23:15:56.719 INFO:teuthology.orchestra.run.smithi082.stderr:37900000 of 50000000 tuples (75%) done (elapsed 103.30 s, remaining 32.98 s) 2023-09-09T23:15:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:38000000 of 50000000 tuples (76%) done (elapsed 103.41 s, remaining 32.66 s) 2023-09-09T23:15:56.923 INFO:teuthology.orchestra.run.smithi082.stderr:38100000 of 50000000 tuples (76%) done (elapsed 103.51 s, remaining 32.33 s) 2023-09-09T23:15:57.028 INFO:teuthology.orchestra.run.smithi082.stderr:38200000 of 50000000 tuples (76%) done (elapsed 103.61 s, remaining 32.01 s) 2023-09-09T23:15:57.123 INFO:teuthology.orchestra.run.smithi082.stderr:38300000 of 50000000 tuples (76%) done (elapsed 103.71 s, remaining 31.68 s) 2023-09-09T23:15:57.229 INFO:teuthology.orchestra.run.smithi082.stderr:38400000 of 50000000 tuples (76%) done (elapsed 103.81 s, remaining 31.36 s) 2023-09-09T23:15:57.341 INFO:teuthology.orchestra.run.smithi082.stderr:38500000 of 50000000 tuples (77%) done (elapsed 103.93 s, remaining 31.04 s) 2023-09-09T23:15:57.444 INFO:teuthology.orchestra.run.smithi082.stderr:38600000 of 50000000 tuples (77%) done (elapsed 104.03 s, remaining 30.72 s) 2023-09-09T23:15:57.541 INFO:teuthology.orchestra.run.smithi082.stderr:38700000 of 50000000 tuples (77%) done (elapsed 104.13 s, remaining 30.40 s) 2023-09-09T23:15:57.611 INFO:teuthology.orchestra.run.smithi082.stderr:38800000 of 50000000 tuples (77%) done (elapsed 104.20 s, remaining 30.08 s) 2023-09-09T23:15:57.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:57 smithi170 ceph-mon[102832]: pgmap v64: 97 pgs: 97 active+clean; 6.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 82 MiB/s wr, 367 op/s 2023-09-09T23:15:57.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:57 smithi082 ceph-mon[98694]: pgmap v64: 97 pgs: 97 active+clean; 6.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 82 MiB/s wr, 367 op/s 2023-09-09T23:15:57.729 INFO:teuthology.orchestra.run.smithi082.stderr:38900000 of 50000000 tuples (77%) done (elapsed 104.31 s, remaining 29.77 s) 2023-09-09T23:15:57.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:57 smithi098 ceph-mon[103124]: pgmap v64: 97 pgs: 97 active+clean; 6.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 82 MiB/s wr, 367 op/s 2023-09-09T23:15:57.827 INFO:teuthology.orchestra.run.smithi082.stderr:39000000 of 50000000 tuples (78%) done (elapsed 104.41 s, remaining 29.45 s) 2023-09-09T23:15:57.933 INFO:teuthology.orchestra.run.smithi082.stderr:39100000 of 50000000 tuples (78%) done (elapsed 104.52 s, remaining 29.14 s) 2023-09-09T23:15:58.036 INFO:teuthology.orchestra.run.smithi082.stderr:39200000 of 50000000 tuples (78%) done (elapsed 104.62 s, remaining 28.82 s) 2023-09-09T23:15:58.139 INFO:teuthology.orchestra.run.smithi082.stderr:39300000 of 50000000 tuples (78%) done (elapsed 104.72 s, remaining 28.51 s) 2023-09-09T23:15:58.223 INFO:teuthology.orchestra.run.smithi082.stderr:39400000 of 50000000 tuples (78%) done (elapsed 104.81 s, remaining 28.20 s) 2023-09-09T23:15:58.323 INFO:teuthology.orchestra.run.smithi082.stderr:39500000 of 50000000 tuples (79%) done (elapsed 104.91 s, remaining 27.89 s) 2023-09-09T23:15:58.449 INFO:teuthology.orchestra.run.smithi082.stderr:39600000 of 50000000 tuples (79%) done (elapsed 105.03 s, remaining 27.58 s) 2023-09-09T23:15:58.529 INFO:teuthology.orchestra.run.smithi082.stderr:39700000 of 50000000 tuples (79%) done (elapsed 105.11 s, remaining 27.27 s) 2023-09-09T23:15:58.630 INFO:teuthology.orchestra.run.smithi082.stderr:39800000 of 50000000 tuples (79%) done (elapsed 105.21 s, remaining 26.96 s) 2023-09-09T23:15:58.726 INFO:teuthology.orchestra.run.smithi082.stderr:39900000 of 50000000 tuples (79%) done (elapsed 105.31 s, remaining 26.66 s) 2023-09-09T23:15:58.841 INFO:teuthology.orchestra.run.smithi082.stderr:40000000 of 50000000 tuples (80%) done (elapsed 105.43 s, remaining 26.36 s) 2023-09-09T23:15:58.940 INFO:teuthology.orchestra.run.smithi082.stderr:40100000 of 50000000 tuples (80%) done (elapsed 105.52 s, remaining 26.05 s) 2023-09-09T23:15:59.046 INFO:teuthology.orchestra.run.smithi082.stderr:40200000 of 50000000 tuples (80%) done (elapsed 105.63 s, remaining 25.75 s) 2023-09-09T23:15:59.145 INFO:teuthology.orchestra.run.smithi082.stderr:40300000 of 50000000 tuples (80%) done (elapsed 105.73 s, remaining 25.45 s) 2023-09-09T23:15:59.240 INFO:teuthology.orchestra.run.smithi082.stderr:40400000 of 50000000 tuples (80%) done (elapsed 105.82 s, remaining 25.15 s) 2023-09-09T23:15:59.320 INFO:teuthology.orchestra.run.smithi082.stderr:40500000 of 50000000 tuples (81%) done (elapsed 105.90 s, remaining 24.84 s) 2023-09-09T23:15:59.453 INFO:teuthology.orchestra.run.smithi082.stderr:40600000 of 50000000 tuples (81%) done (elapsed 106.04 s, remaining 24.55 s) 2023-09-09T23:15:59.518 INFO:teuthology.orchestra.run.smithi082.stderr:40700000 of 50000000 tuples (81%) done (elapsed 106.10 s, remaining 24.24 s) 2023-09-09T23:15:59.621 INFO:teuthology.orchestra.run.smithi082.stderr:40800000 of 50000000 tuples (81%) done (elapsed 106.21 s, remaining 23.95 s) 2023-09-09T23:15:59.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:15:59 smithi170 ceph-mon[102832]: pgmap v65: 97 pgs: 97 active+clean; 6.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 61 MiB/s wr, 354 op/s 2023-09-09T23:15:59.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:15:59 smithi082 ceph-mon[98694]: pgmap v65: 97 pgs: 97 active+clean; 6.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 61 MiB/s wr, 354 op/s 2023-09-09T23:15:59.720 INFO:teuthology.orchestra.run.smithi082.stderr:40900000 of 50000000 tuples (81%) done (elapsed 106.30 s, remaining 23.65 s) 2023-09-09T23:15:59.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:15:59 smithi098 ceph-mon[103124]: pgmap v65: 97 pgs: 97 active+clean; 6.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 61 MiB/s wr, 354 op/s 2023-09-09T23:15:59.855 INFO:teuthology.orchestra.run.smithi082.stderr:41000000 of 50000000 tuples (82%) done (elapsed 106.44 s, remaining 23.36 s) 2023-09-09T23:15:59.952 INFO:teuthology.orchestra.run.smithi082.stderr:41100000 of 50000000 tuples (82%) done (elapsed 106.54 s, remaining 23.07 s) 2023-09-09T23:16:00.054 INFO:teuthology.orchestra.run.smithi082.stderr:41200000 of 50000000 tuples (82%) done (elapsed 106.64 s, remaining 22.78 s) 2023-09-09T23:16:00.325 INFO:teuthology.orchestra.run.smithi082.stderr:41300000 of 50000000 tuples (82%) done (elapsed 106.91 s, remaining 22.52 s) 2023-09-09T23:16:00.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:00 smithi170 ceph-mon[102832]: pgmap v66: 97 pgs: 97 active+clean; 6.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 91 MiB/s wr, 473 op/s 2023-09-09T23:16:00.954 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:00 smithi082 ceph-mon[98694]: pgmap v66: 97 pgs: 97 active+clean; 6.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 91 MiB/s wr, 473 op/s 2023-09-09T23:16:00.959 INFO:teuthology.orchestra.run.smithi082.stderr:41400000 of 50000000 tuples (82%) done (elapsed 107.54 s, remaining 22.34 s) 2023-09-09T23:16:01.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:00 smithi098 ceph-mon[103124]: pgmap v66: 97 pgs: 97 active+clean; 6.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 91 MiB/s wr, 473 op/s 2023-09-09T23:16:01.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:16:01.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:16:01.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:16:01.954 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:16:01.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:16:01.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:16:02.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:16:02.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:16:02.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:16:02.117 INFO:teuthology.orchestra.run.smithi082.stderr:41500000 of 50000000 tuples (83%) done (elapsed 108.70 s, remaining 22.26 s) 2023-09-09T23:16:02.194 INFO:teuthology.orchestra.run.smithi082.stderr:41600000 of 50000000 tuples (83%) done (elapsed 108.78 s, remaining 21.96 s) 2023-09-09T23:16:02.314 INFO:teuthology.orchestra.run.smithi082.stderr:41700000 of 50000000 tuples (83%) done (elapsed 108.90 s, remaining 21.68 s) 2023-09-09T23:16:02.415 INFO:teuthology.orchestra.run.smithi082.stderr:41800000 of 50000000 tuples (83%) done (elapsed 109.00 s, remaining 21.38 s) 2023-09-09T23:16:02.855 INFO:teuthology.orchestra.run.smithi082.stderr:41900000 of 50000000 tuples (83%) done (elapsed 109.44 s, remaining 21.16 s) 2023-09-09T23:16:02.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:16:02.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:16:02.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:16:02.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:02 smithi170 ceph-mon[102832]: pgmap v67: 97 pgs: 97 active+clean; 6.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 49 MiB/s wr, 414 op/s 2023-09-09T23:16:02.954 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:16:02.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:16:02.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:16:02.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:02 smithi082 ceph-mon[98694]: pgmap v67: 97 pgs: 97 active+clean; 6.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 49 MiB/s wr, 414 op/s 2023-09-09T23:16:03.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:16:03.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:16:03.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:16:03.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:02 smithi098 ceph-mon[103124]: pgmap v67: 97 pgs: 97 active+clean; 6.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 49 MiB/s wr, 414 op/s 2023-09-09T23:16:03.316 INFO:teuthology.orchestra.run.smithi082.stderr:42000000 of 50000000 tuples (84%) done (elapsed 109.90 s, remaining 20.93 s) 2023-09-09T23:16:03.419 INFO:teuthology.orchestra.run.smithi082.stderr:42100000 of 50000000 tuples (84%) done (elapsed 110.00 s, remaining 20.64 s) 2023-09-09T23:16:03.627 INFO:teuthology.orchestra.run.smithi082.stderr:42200000 of 50000000 tuples (84%) done (elapsed 110.21 s, remaining 20.37 s) 2023-09-09T23:16:04.167 INFO:teuthology.orchestra.run.smithi082.stderr:42300000 of 50000000 tuples (84%) done (elapsed 110.75 s, remaining 20.16 s) 2023-09-09T23:16:04.233 INFO:teuthology.orchestra.run.smithi082.stderr:42400000 of 50000000 tuples (84%) done (elapsed 110.82 s, remaining 19.86 s) 2023-09-09T23:16:04.348 INFO:teuthology.orchestra.run.smithi082.stderr:42500000 of 50000000 tuples (85%) done (elapsed 110.93 s, remaining 19.58 s) 2023-09-09T23:16:04.444 INFO:teuthology.orchestra.run.smithi082.stderr:42600000 of 50000000 tuples (85%) done (elapsed 111.03 s, remaining 19.29 s) 2023-09-09T23:16:04.532 INFO:teuthology.orchestra.run.smithi082.stderr:42700000 of 50000000 tuples (85%) done (elapsed 111.12 s, remaining 19.00 s) 2023-09-09T23:16:04.638 INFO:teuthology.orchestra.run.smithi082.stderr:42800000 of 50000000 tuples (85%) done (elapsed 111.22 s, remaining 18.71 s) 2023-09-09T23:16:04.737 INFO:teuthology.orchestra.run.smithi082.stderr:42900000 of 50000000 tuples (85%) done (elapsed 111.32 s, remaining 18.42 s) 2023-09-09T23:16:05.086 INFO:teuthology.orchestra.run.smithi082.stderr:43000000 of 50000000 tuples (86%) done (elapsed 111.67 s, remaining 18.18 s) 2023-09-09T23:16:05.533 INFO:teuthology.orchestra.run.smithi082.stderr:43100000 of 50000000 tuples (86%) done (elapsed 112.12 s, remaining 17.95 s) 2023-09-09T23:16:05.645 INFO:teuthology.orchestra.run.smithi082.stderr:43200000 of 50000000 tuples (86%) done (elapsed 112.23 s, remaining 17.67 s) 2023-09-09T23:16:05.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:05 smithi170 ceph-mon[102832]: pgmap v68: 97 pgs: 97 active+clean; 6.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 MiB/s wr, 412 op/s 2023-09-09T23:16:05.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:05 smithi082 ceph-mon[98694]: pgmap v68: 97 pgs: 97 active+clean; 6.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 MiB/s wr, 412 op/s 2023-09-09T23:16:05.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:05 smithi098 ceph-mon[103124]: pgmap v68: 97 pgs: 97 active+clean; 6.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 MiB/s wr, 412 op/s 2023-09-09T23:16:05.764 INFO:teuthology.orchestra.run.smithi082.stderr:43300000 of 50000000 tuples (86%) done (elapsed 112.35 s, remaining 17.38 s) 2023-09-09T23:16:05.876 INFO:teuthology.orchestra.run.smithi082.stderr:43400000 of 50000000 tuples (86%) done (elapsed 112.46 s, remaining 17.10 s) 2023-09-09T23:16:05.963 INFO:teuthology.orchestra.run.smithi082.stderr:43500000 of 50000000 tuples (87%) done (elapsed 112.55 s, remaining 16.82 s) 2023-09-09T23:16:06.063 INFO:teuthology.orchestra.run.smithi082.stderr:43600000 of 50000000 tuples (87%) done (elapsed 112.65 s, remaining 16.54 s) 2023-09-09T23:16:06.171 INFO:teuthology.orchestra.run.smithi082.stderr:43700000 of 50000000 tuples (87%) done (elapsed 112.76 s, remaining 16.26 s) 2023-09-09T23:16:06.240 INFO:teuthology.orchestra.run.smithi082.stderr:43800000 of 50000000 tuples (87%) done (elapsed 112.82 s, remaining 15.97 s) 2023-09-09T23:16:06.361 INFO:teuthology.orchestra.run.smithi082.stderr:43900000 of 50000000 tuples (87%) done (elapsed 112.95 s, remaining 15.69 s) 2023-09-09T23:16:06.447 INFO:teuthology.orchestra.run.smithi082.stderr:44000000 of 50000000 tuples (88%) done (elapsed 113.03 s, remaining 15.41 s) 2023-09-09T23:16:06.553 INFO:teuthology.orchestra.run.smithi082.stderr:44100000 of 50000000 tuples (88%) done (elapsed 113.14 s, remaining 15.14 s) 2023-09-09T23:16:06.666 INFO:teuthology.orchestra.run.smithi082.stderr:44200000 of 50000000 tuples (88%) done (elapsed 113.25 s, remaining 14.86 s) 2023-09-09T23:16:06.764 INFO:teuthology.orchestra.run.smithi082.stderr:44300000 of 50000000 tuples (88%) done (elapsed 113.35 s, remaining 14.58 s) 2023-09-09T23:16:06.877 INFO:teuthology.orchestra.run.smithi082.stderr:44400000 of 50000000 tuples (88%) done (elapsed 113.46 s, remaining 14.31 s) 2023-09-09T23:16:06.985 INFO:teuthology.orchestra.run.smithi082.stderr:44500000 of 50000000 tuples (89%) done (elapsed 113.57 s, remaining 14.04 s) 2023-09-09T23:16:07.083 INFO:teuthology.orchestra.run.smithi082.stderr:44600000 of 50000000 tuples (89%) done (elapsed 113.67 s, remaining 13.76 s) 2023-09-09T23:16:07.165 INFO:teuthology.orchestra.run.smithi082.stderr:44700000 of 50000000 tuples (89%) done (elapsed 113.75 s, remaining 13.49 s) 2023-09-09T23:16:07.263 INFO:teuthology.orchestra.run.smithi082.stderr:44800000 of 50000000 tuples (89%) done (elapsed 113.85 s, remaining 13.21 s) 2023-09-09T23:16:07.343 INFO:teuthology.orchestra.run.smithi082.stderr:44900000 of 50000000 tuples (89%) done (elapsed 113.93 s, remaining 12.94 s) 2023-09-09T23:16:07.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:07 smithi170 ceph-mon[102832]: pgmap v69: 97 pgs: 97 active+clean; 7.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 105 MiB/s wr, 617 op/s 2023-09-09T23:16:07.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:07 smithi082 ceph-mon[98694]: pgmap v69: 97 pgs: 97 active+clean; 7.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 105 MiB/s wr, 617 op/s 2023-09-09T23:16:07.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:07 smithi098 ceph-mon[103124]: pgmap v69: 97 pgs: 97 active+clean; 7.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 105 MiB/s wr, 617 op/s 2023-09-09T23:16:08.055 INFO:teuthology.orchestra.run.smithi082.stderr:45000000 of 50000000 tuples (90%) done (elapsed 114.64 s, remaining 12.74 s) 2023-09-09T23:16:08.621 INFO:teuthology.orchestra.run.smithi082.stderr:45100000 of 50000000 tuples (90%) done (elapsed 115.21 s, remaining 12.52 s) 2023-09-09T23:16:09.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:09 smithi170 ceph-mon[102832]: pgmap v70: 97 pgs: 97 active+clean; 7.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 93 MiB/s wr, 325 op/s 2023-09-09T23:16:09.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:09 smithi082 ceph-mon[98694]: pgmap v70: 97 pgs: 97 active+clean; 7.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 93 MiB/s wr, 325 op/s 2023-09-09T23:16:09.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:09 smithi098 ceph-mon[103124]: pgmap v70: 97 pgs: 97 active+clean; 7.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 93 MiB/s wr, 325 op/s 2023-09-09T23:16:11.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:11 smithi170 ceph-mon[102832]: pgmap v71: 97 pgs: 97 active+clean; 8.0 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 143 MiB/s wr, 493 op/s 2023-09-09T23:16:11.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:11 smithi082 ceph-mon[98694]: pgmap v71: 97 pgs: 97 active+clean; 8.0 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 143 MiB/s wr, 493 op/s 2023-09-09T23:16:11.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:11 smithi098 ceph-mon[103124]: pgmap v71: 97 pgs: 97 active+clean; 8.0 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 143 MiB/s wr, 493 op/s 2023-09-09T23:16:13.276 INFO:teuthology.orchestra.run.smithi082.stderr:45200000 of 50000000 tuples (90%) done (elapsed 119.86 s, remaining 12.73 s) 2023-09-09T23:16:13.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:13 smithi170 ceph-mon[102832]: pgmap v72: 97 pgs: 97 active+clean; 8.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 120 MiB/s wr, 379 op/s 2023-09-09T23:16:13.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:13 smithi082 ceph-mon[98694]: pgmap v72: 97 pgs: 97 active+clean; 8.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 120 MiB/s wr, 379 op/s 2023-09-09T23:16:13.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:13 smithi098 ceph-mon[103124]: pgmap v72: 97 pgs: 97 active+clean; 8.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 120 MiB/s wr, 379 op/s 2023-09-09T23:16:14.566 INFO:teuthology.orchestra.run.smithi082.stderr:45300000 of 50000000 tuples (90%) done (elapsed 121.15 s, remaining 12.57 s) 2023-09-09T23:16:14.677 INFO:teuthology.orchestra.run.smithi082.stderr:45400000 of 50000000 tuples (90%) done (elapsed 121.26 s, remaining 12.29 s) 2023-09-09T23:16:14.769 INFO:teuthology.orchestra.run.smithi082.stderr:45500000 of 50000000 tuples (91%) done (elapsed 121.35 s, remaining 12.00 s) 2023-09-09T23:16:14.968 INFO:teuthology.orchestra.run.smithi082.stderr:45600000 of 50000000 tuples (91%) done (elapsed 121.55 s, remaining 11.73 s) 2023-09-09T23:16:15.432 INFO:teuthology.orchestra.run.smithi082.stderr:45700000 of 50000000 tuples (91%) done (elapsed 122.02 s, remaining 11.48 s) 2023-09-09T23:16:15.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:15 smithi170 ceph-mon[102832]: pgmap v73: 97 pgs: 97 active+clean; 8.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 117 MiB/s wr, 378 op/s 2023-09-09T23:16:15.688 INFO:teuthology.orchestra.run.smithi082.stderr:45800000 of 50000000 tuples (91%) done (elapsed 122.27 s, remaining 11.21 s) 2023-09-09T23:16:15.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:15 smithi082 ceph-mon[98694]: pgmap v73: 97 pgs: 97 active+clean; 8.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 117 MiB/s wr, 378 op/s 2023-09-09T23:16:15.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:15 smithi098 ceph-mon[103124]: pgmap v73: 97 pgs: 97 active+clean; 8.1 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 117 MiB/s wr, 378 op/s 2023-09-09T23:16:15.861 INFO:teuthology.orchestra.run.smithi082.stderr:45900000 of 50000000 tuples (91%) done (elapsed 122.45 s, remaining 10.94 s) 2023-09-09T23:16:15.993 INFO:teuthology.orchestra.run.smithi082.stderr:46000000 of 50000000 tuples (92%) done (elapsed 122.58 s, remaining 10.66 s) 2023-09-09T23:16:16.097 INFO:teuthology.orchestra.run.smithi082.stderr:46100000 of 50000000 tuples (92%) done (elapsed 122.68 s, remaining 10.38 s) 2023-09-09T23:16:16.723 INFO:teuthology.orchestra.run.smithi082.stderr:46200000 of 50000000 tuples (92%) done (elapsed 123.31 s, remaining 10.14 s) 2023-09-09T23:16:16.787 INFO:teuthology.orchestra.run.smithi082.stderr:46300000 of 50000000 tuples (92%) done (elapsed 123.37 s, remaining 9.86 s) 2023-09-09T23:16:16.908 INFO:teuthology.orchestra.run.smithi082.stderr:46400000 of 50000000 tuples (92%) done (elapsed 123.49 s, remaining 9.58 s) 2023-09-09T23:16:16.995 INFO:teuthology.orchestra.run.smithi082.stderr:46500000 of 50000000 tuples (93%) done (elapsed 123.58 s, remaining 9.30 s) 2023-09-09T23:16:17.105 INFO:teuthology.orchestra.run.smithi082.stderr:46600000 of 50000000 tuples (93%) done (elapsed 123.69 s, remaining 9.02 s) 2023-09-09T23:16:17.221 INFO:teuthology.orchestra.run.smithi082.stderr:46700000 of 50000000 tuples (93%) done (elapsed 123.81 s, remaining 8.75 s) 2023-09-09T23:16:17.322 INFO:teuthology.orchestra.run.smithi082.stderr:46800000 of 50000000 tuples (93%) done (elapsed 123.91 s, remaining 8.47 s) 2023-09-09T23:16:17.424 INFO:teuthology.orchestra.run.smithi082.stderr:46900000 of 50000000 tuples (93%) done (elapsed 124.01 s, remaining 8.20 s) 2023-09-09T23:16:17.529 INFO:teuthology.orchestra.run.smithi082.stderr:47000000 of 50000000 tuples (94%) done (elapsed 124.11 s, remaining 7.92 s) 2023-09-09T23:16:17.629 INFO:teuthology.orchestra.run.smithi082.stderr:47100000 of 50000000 tuples (94%) done (elapsed 124.21 s, remaining 7.65 s) 2023-09-09T23:16:17.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:17 smithi170 ceph-mon[102832]: pgmap v74: 97 pgs: 97 active+clean; 8.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 122 MiB/s wr, 652 op/s 2023-09-09T23:16:17.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:17 smithi082 ceph-mon[98694]: pgmap v74: 97 pgs: 97 active+clean; 8.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 122 MiB/s wr, 652 op/s 2023-09-09T23:16:17.713 INFO:teuthology.orchestra.run.smithi082.stderr:47200000 of 50000000 tuples (94%) done (elapsed 124.30 s, remaining 7.37 s) 2023-09-09T23:16:17.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:17 smithi098 ceph-mon[103124]: pgmap v74: 97 pgs: 97 active+clean; 8.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 122 MiB/s wr, 652 op/s 2023-09-09T23:16:17.816 INFO:teuthology.orchestra.run.smithi082.stderr:47300000 of 50000000 tuples (94%) done (elapsed 124.40 s, remaining 7.10 s) 2023-09-09T23:16:17.896 INFO:teuthology.orchestra.run.smithi082.stderr:47400000 of 50000000 tuples (94%) done (elapsed 124.48 s, remaining 6.83 s) 2023-09-09T23:16:18.014 INFO:teuthology.orchestra.run.smithi082.stderr:47500000 of 50000000 tuples (95%) done (elapsed 124.60 s, remaining 6.56 s) 2023-09-09T23:16:18.104 INFO:teuthology.orchestra.run.smithi082.stderr:47600000 of 50000000 tuples (95%) done (elapsed 124.69 s, remaining 6.29 s) 2023-09-09T23:16:18.206 INFO:teuthology.orchestra.run.smithi082.stderr:47700000 of 50000000 tuples (95%) done (elapsed 124.79 s, remaining 6.02 s) 2023-09-09T23:16:18.321 INFO:teuthology.orchestra.run.smithi082.stderr:47800000 of 50000000 tuples (95%) done (elapsed 124.91 s, remaining 5.75 s) 2023-09-09T23:16:18.418 INFO:teuthology.orchestra.run.smithi082.stderr:47900000 of 50000000 tuples (95%) done (elapsed 125.00 s, remaining 5.48 s) 2023-09-09T23:16:18.723 INFO:teuthology.orchestra.run.smithi082.stderr:48000000 of 50000000 tuples (96%) done (elapsed 125.31 s, remaining 5.22 s) 2023-09-09T23:16:18.816 INFO:teuthology.orchestra.run.smithi082.stderr:48100000 of 50000000 tuples (96%) done (elapsed 125.40 s, remaining 4.95 s) 2023-09-09T23:16:18.918 INFO:teuthology.orchestra.run.smithi082.stderr:48200000 of 50000000 tuples (96%) done (elapsed 125.50 s, remaining 4.69 s) 2023-09-09T23:16:19.004 INFO:teuthology.orchestra.run.smithi082.stderr:48300000 of 50000000 tuples (96%) done (elapsed 125.59 s, remaining 4.42 s) 2023-09-09T23:16:19.107 INFO:teuthology.orchestra.run.smithi082.stderr:48400000 of 50000000 tuples (96%) done (elapsed 125.69 s, remaining 4.16 s) 2023-09-09T23:16:19.212 INFO:teuthology.orchestra.run.smithi082.stderr:48500000 of 50000000 tuples (97%) done (elapsed 125.80 s, remaining 3.89 s) 2023-09-09T23:16:19.332 INFO:teuthology.orchestra.run.smithi082.stderr:48600000 of 50000000 tuples (97%) done (elapsed 125.92 s, remaining 3.63 s) 2023-09-09T23:16:19.435 INFO:teuthology.orchestra.run.smithi082.stderr:48700000 of 50000000 tuples (97%) done (elapsed 126.02 s, remaining 3.36 s) 2023-09-09T23:16:19.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:19 smithi170 ceph-mon[102832]: pgmap v75: 97 pgs: 97 active+clean; 8.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 61 MiB/s wr, 447 op/s 2023-09-09T23:16:19.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:19 smithi082 ceph-mon[98694]: pgmap v75: 97 pgs: 97 active+clean; 8.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 61 MiB/s wr, 447 op/s 2023-09-09T23:16:19.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:19 smithi098 ceph-mon[103124]: pgmap v75: 97 pgs: 97 active+clean; 8.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 61 MiB/s wr, 447 op/s 2023-09-09T23:16:21.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:21 smithi170 ceph-mon[102832]: pgmap v76: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 85 MiB/s wr, 515 op/s 2023-09-09T23:16:21.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:21 smithi082 ceph-mon[98694]: pgmap v76: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 85 MiB/s wr, 515 op/s 2023-09-09T23:16:21.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:21 smithi098 ceph-mon[103124]: pgmap v76: 97 pgs: 97 active+clean; 8.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 85 MiB/s wr, 515 op/s 2023-09-09T23:16:23.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:23 smithi170 ceph-mon[102832]: pgmap v77: 97 pgs: 97 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 39 MiB/s wr, 366 op/s 2023-09-09T23:16:23.684 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:16:23.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:23 smithi082 ceph-mon[98694]: pgmap v77: 97 pgs: 97 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 39 MiB/s wr, 366 op/s 2023-09-09T23:16:23.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:23 smithi098 ceph-mon[103124]: pgmap v77: 97 pgs: 97 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 39 MiB/s wr, 366 op/s 2023-09-09T23:16:23.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.838+0000 7f0b57ebe700 1 -- 172.21.15.82:0/2755681956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b50106650 msgr2=0x7f0b50106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:16:23.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.838+0000 7f0b57ebe700 1 --2- 172.21.15.82:0/2755681956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b50106650 0x7f0b50106a30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c0099c0 tx=0x7f0b4c009cd0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.839+0000 7f0b57ebe700 1 -- 172.21.15.82:0/2755681956 shutdown_connections 2023-09-09T23:16:23.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.839+0000 7f0b57ebe700 1 --2- 172.21.15.82:0/2755681956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b50100f50 0x7f0b5010d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.839+0000 7f0b57ebe700 1 --2- 172.21.15.82:0/2755681956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b50106650 0x7f0b50106a30 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.839+0000 7f0b57ebe700 1 --2- 172.21.15.82:0/2755681956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b50100590 0x7f0b50100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.839+0000 7f0b57ebe700 1 -- 172.21.15.82:0/2755681956 >> 172.21.15.82:0/2755681956 conn(0x7f0b500fc060 msgr2=0x7f0b500fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:16:23.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.839+0000 7f0b57ebe700 1 -- 172.21.15.82:0/2755681956 shutdown_connections 2023-09-09T23:16:23.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.839+0000 7f0b57ebe700 1 -- 172.21.15.82:0/2755681956 wait complete. 2023-09-09T23:16:23.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.840+0000 7f0b57ebe700 1 Processor -- start 2023-09-09T23:16:23.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.840+0000 7f0b57ebe700 1 -- start start 2023-09-09T23:16:23.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.840+0000 7f0b57ebe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b50100590 0x7f0b501966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:23.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.840+0000 7f0b57ebe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b50100f50 0x7f0b50196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:23.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.840+0000 7f0b57ebe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b50106650 0x7f0b5019b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:23.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.840+0000 7f0b57ebe700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5019b660 con 0x7f0b50106650 2023-09-09T23:16:23.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.840+0000 7f0b57ebe700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5019b7d0 con 0x7f0b50100f50 2023-09-09T23:16:23.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.840+0000 7f0b57ebe700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5019b990 con 0x7f0b50100590 2023-09-09T23:16:23.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.840+0000 7f0b55459700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b50100f50 0x7f0b50196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:23.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.840+0000 7f0b55459700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b50100f50 0x7f0b50196c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:43186/0 (socket says 172.21.15.82:43186) 2023-09-09T23:16:23.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.840+0000 7f0b55459700 1 -- 172.21.15.82:0/4168656878 learned_addr learned my addr 172.21.15.82:0/4168656878 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:16:23.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.840+0000 7f0b55c5a700 1 --2- 172.21.15.82:0/4168656878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b50100590 0x7f0b501966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:23.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.840+0000 7f0b55c5a700 1 -- 172.21.15.82:0/4168656878 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b50100f50 msgr2=0x7f0b50196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:16:23.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.840+0000 7f0b55c5a700 1 --2- 172.21.15.82:0/4168656878 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b50100f50 0x7f0b50196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.840+0000 7f0b55c5a700 1 -- 172.21.15.82:0/4168656878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b50106650 msgr2=0x7f0b5019b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:16:23.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.840+0000 7f0b55c5a700 1 --2- 172.21.15.82:0/4168656878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b50106650 0x7f0b5019b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.840+0000 7f0b55c5a700 1 -- 172.21.15.82:0/4168656878 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4c0095f0 con 0x7f0b50100590 2023-09-09T23:16:23.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.840+0000 7f0b55459700 1 --2- 172.21.15.82:0/4168656878 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b50100f50 0x7f0b50196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:16:23.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.841+0000 7f0b55c5a700 1 --2- 172.21.15.82:0/4168656878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b50100590 0x7f0b501966d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00b580 tx=0x7f0b4c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:16:23.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.841+0000 7f0b46ffd700 1 -- 172.21.15.82:0/4168656878 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c01d070 con 0x7f0b50100590 2023-09-09T23:16:23.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.841+0000 7f0b46ffd700 1 -- 172.21.15.82:0/4168656878 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b4c022470 con 0x7f0b50100590 2023-09-09T23:16:23.848 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.841+0000 7f0b57ebe700 1 -- 172.21.15.82:0/4168656878 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b5019bc10 con 0x7f0b50100590 2023-09-09T23:16:23.848 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.841+0000 7f0b46ffd700 1 -- 172.21.15.82:0/4168656878 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c017650 con 0x7f0b50100590 2023-09-09T23:16:23.849 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.841+0000 7f0b57ebe700 1 -- 172.21.15.82:0/4168656878 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b501a34b0 con 0x7f0b50100590 2023-09-09T23:16:23.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.843+0000 7f0b46ffd700 1 -- 172.21.15.82:0/4168656878 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f0b4c0225e0 con 0x7f0b50100590 2023-09-09T23:16:23.856 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.844+0000 7f0b46ffd700 1 --2- 172.21.15.82:0/4168656878 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f0b3c0a07b0 0x7f0b3c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:23.856 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.844+0000 7f0b55459700 1 --2- 172.21.15.82:0/4168656878 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f0b3c0a07b0 0x7f0b3c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:23.856 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.844+0000 7f0b46ffd700 1 -- 172.21.15.82:0/4168656878 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f0b4c0cd060 con 0x7f0b50100590 2023-09-09T23:16:23.856 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.844+0000 7f0b55459700 1 --2- 172.21.15.82:0/4168656878 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f0b3c0a07b0 0x7f0b3c0a2c70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0b50197c90 tx=0x7f0b40009260 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:16:23.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.845+0000 7f0b64084b80 1 Processor -- start 2023-09-09T23:16:23.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.845+0000 7f0b64084b80 1 -- start start 2023-09-09T23:16:23.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.846+0000 7f0b64084b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b351b4710 0x560b351b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:23.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.846+0000 7f0b64084b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b351b5030 0x560b351c2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:23.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.846+0000 7f0b64084b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b351c3250 0x560b351c5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:23.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.846+0000 7f0b64084b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b3510d730 con 0x560b351c3250 2023-09-09T23:16:23.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.846+0000 7f0b64084b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b3510b050 con 0x560b351b5030 2023-09-09T23:16:23.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.846+0000 7f0b64084b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b35106940 con 0x560b351b4710 2023-09-09T23:16:23.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.846+0000 7f0b55459700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b351b5030 0x560b351c2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:23.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.846+0000 7f0b55c5a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b351b4710 0x560b351b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:23.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.846+0000 7f0b5645b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b351c3250 0x560b351c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:23.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.846+0000 7f0b55459700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b351b5030 0x560b351c2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:43192/0 (socket says 172.21.15.82:43192) 2023-09-09T23:16:23.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.846+0000 7f0b55c5a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b351b4710 0x560b351b4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:56142/0 (socket says 172.21.15.82:56142) 2023-09-09T23:16:23.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.846+0000 7f0b55459700 1 -- 172.21.15.82:0/1688727726 learned_addr learned my addr 172.21.15.82:0/1688727726 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:16:23.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.846+0000 7f0b55459700 1 -- 172.21.15.82:0/1688727726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b351b4710 msgr2=0x560b351b4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:16:23.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.846+0000 7f0b55459700 1 --2- 172.21.15.82:0/1688727726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b351b4710 0x560b351b4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.860 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.846+0000 7f0b55459700 1 -- 172.21.15.82:0/1688727726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b351c3250 msgr2=0x560b351c5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:16:23.860 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.846+0000 7f0b55459700 1 --2- 172.21.15.82:0/1688727726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b351c3250 0x560b351c5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.860 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.846+0000 7f0b55459700 1 -- 172.21.15.82:0/1688727726 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b34f32de0 con 0x560b351b5030 2023-09-09T23:16:23.860 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.846+0000 7f0b5645b700 1 --2- 172.21.15.82:0/1688727726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b351c3250 0x560b351c5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:16:23.860 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.846+0000 7f0b55459700 1 --2- 172.21.15.82:0/1688727726 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b351b5030 0x560b351c2d10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009880 tx=0x7f0b40014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cef7f5f3b68c17ac server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:16:23.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.847+0000 7f0b3bfff700 1 -- 172.21.15.82:0/1688727726 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b40026630 con 0x560b351b5030 2023-09-09T23:16:23.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.847+0000 7f0b3bfff700 1 -- 172.21.15.82:0/1688727726 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b40021070 con 0x560b351b5030 2023-09-09T23:16:23.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.847+0000 7f0b3bfff700 1 -- 172.21.15.82:0/1688727726 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4001d670 con 0x560b351b5030 2023-09-09T23:16:23.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.848+0000 7f0b64084b80 1 -- 172.21.15.82:0/1688727726 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b351b5030 msgr2=0x560b351c2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:16:23.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.848+0000 7f0b64084b80 1 --2- 172.21.15.82:0/1688727726 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b351b5030 0x560b351c2d10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009880 tx=0x7f0b40014000 comp rx=0 tx=0).stop 2023-09-09T23:16:23.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.848+0000 7f0b64084b80 1 -- 172.21.15.82:0/1688727726 shutdown_connections 2023-09-09T23:16:23.862 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.848+0000 7f0b64084b80 1 --2- 172.21.15.82:0/1688727726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b351c3250 0x560b351c5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.862 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.848+0000 7f0b64084b80 1 --2- 172.21.15.82:0/1688727726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b351b4710 0x560b351b4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.862 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.848+0000 7f0b64084b80 1 --2- 172.21.15.82:0/1688727726 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b351b5030 0x560b351c2d10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.862 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.848+0000 7f0b64084b80 1 -- 172.21.15.82:0/1688727726 >> 172.21.15.82:0/1688727726 conn(0x560b35129090 msgr2=0x560b351afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:16:23.863 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.848+0000 7f0b64084b80 1 -- 172.21.15.82:0/1688727726 shutdown_connections 2023-09-09T23:16:23.863 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.848+0000 7f0b64084b80 1 -- 172.21.15.82:0/1688727726 wait complete. 2023-09-09T23:16:23.863 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.848+0000 7f0b64084b80 1 -- start start 2023-09-09T23:16:23.863 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.848+0000 7f0b64084b80 1 Processor -- start 2023-09-09T23:16:23.863 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.849+0000 7f0b64084b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b351b4710 0x560b351c2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:23.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.849+0000 7f0b64084b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b351b5030 0x560b351bff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:23.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.849+0000 7f0b64084b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b351c3250 0x560b351c0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:23.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.849+0000 7f0b64084b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b350a4b70 con 0x560b351b5030 2023-09-09T23:16:23.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.849+0000 7f0b64084b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b350a4870 con 0x560b351b4710 2023-09-09T23:16:23.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.849+0000 7f0b64084b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b34f88920 con 0x560b351c3250 2023-09-09T23:16:23.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.849+0000 7f0b64084b80 10 client.0 _ll_register_callbacks cb 0x560b34feea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:16:23.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.849+0000 7f0b55c5a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b351b4710 0x560b351c2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:23.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.849+0000 7f0b5645b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b351c3250 0x560b351c0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:23.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.849+0000 7f0b55c5a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b351b4710 0x560b351c2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:43196/0 (socket says 172.21.15.82:43196) 2023-09-09T23:16:23.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.849+0000 7f0b5645b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b351c3250 0x560b351c0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:56148/0 (socket says 172.21.15.82:56148) 2023-09-09T23:16:23.866 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.849+0000 7f0b55c5a700 1 -- 172.21.15.82:0/4183003016 learned_addr learned my addr 172.21.15.82:0/4183003016 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:16:23.866 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.849+0000 7f0b55459700 1 --2- 172.21.15.82:0/4183003016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b351b5030 0x560b351bff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:23.866 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.849+0000 7f0b5645b700 1 -- 172.21.15.82:0/4183003016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b351b4710 msgr2=0x560b351c2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:16:23.866 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.849+0000 7f0b5645b700 1 --2- 172.21.15.82:0/4183003016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b351b4710 0x560b351c2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.849+0000 7f0b5645b700 1 -- 172.21.15.82:0/4183003016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b351b5030 msgr2=0x560b351bff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:16:23.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.849+0000 7f0b5645b700 1 --2- 172.21.15.82:0/4183003016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b351b5030 0x560b351bff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.849+0000 7f0b5645b700 1 -- 172.21.15.82:0/4183003016 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b4800e0c0 con 0x560b351c3250 2023-09-09T23:16:23.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.850+0000 7f0b5645b700 1 --2- 172.21.15.82:0/4183003016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b351c3250 0x560b351c0630 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0b48004660 tx=0x7f0b48004970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aada67f4a656e69e server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:16:23.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.850+0000 7f0b397fa700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-09T23:16:23.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.850+0000 7f0b397fa700 1 -- 172.21.15.82:0/4183003016 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b48010040 con 0x560b351c3250 2023-09-09T23:16:23.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.850+0000 7f0b64084b80 1 -- 172.21.15.82:0/4183003016 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b3505e4f0 con 0x560b351c3250 2023-09-09T23:16:23.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.850+0000 7f0b397fa700 1 -- 172.21.15.82:0/4183003016 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f0b4800e6c0 con 0x560b351c3250 2023-09-09T23:16:23.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.850+0000 7f0b397fa700 1 -- 172.21.15.82:0/4183003016 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b4800e880 con 0x560b351c3250 2023-09-09T23:16:23.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.850+0000 7f0b64084b80 10 client.25378 fetch_fsmap learned FSMap version 32 2023-09-09T23:16:23.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.850+0000 7f0b64084b80 1 -- 172.21.15.82:0/4183003016 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f0b40017070 con 0x560b351c3250 2023-09-09T23:16:23.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.850+0000 7f0b397fa700 1 -- 172.21.15.82:0/4183003016 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560b3505e4f0 con 0x560b351c3250 2023-09-09T23:16:23.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.850+0000 7f0b397fa700 1 -- 172.21.15.82:0/4183003016 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f0b48007bb0 con 0x560b351c3250 2023-09-09T23:16:23.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.850+0000 7f0b64084b80 10 client.25378 fetch_fsmap finished waiting for FSMap version 32 2023-09-09T23:16:23.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.850+0000 7f0b64084b80 10 client.25378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:16:23.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.850+0000 7f0b64084b80 20 client.25378 populate_metadata read hostname 'smithi082' 2023-09-09T23:16:23.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.850+0000 7f0b64084b80 1 --2- 172.21.15.82:0/4183003016 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x560b351d2e30 0x560b351d5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:23.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.850+0000 7f0b64084b80 4 client.25378 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:16:23.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.850+0000 7f0b64084b80 1 -- 172.21.15.82:0/4183003016 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b3511b8a0 con 0x560b351d2e30 2023-09-09T23:16:23.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.851+0000 7f0b55c5a700 1 --2- 172.21.15.82:0/4183003016 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x560b351d2e30 0x560b351d5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:23.871 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.851+0000 7f0b397fa700 10 client.25378 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:16:23.871 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.851+0000 7f0b55c5a700 1 --2- 172.21.15.82:0/4183003016 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x560b351d2e30 0x560b351d5250 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e79da0329ca3705 server_cookie=f8f40e1059c253c5 in_seq=0 out_seq=0 2023-09-09T23:16:23.871 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.852+0000 7f0b397fa700 1 -- 172.21.15.82:0/4183003016 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560b3511b8a0 con 0x560b351d2e30 2023-09-09T23:16:23.871 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.852+0000 7f0b397fa700 10 client.25378 handle_command_reply: tid=0 2023-09-09T23:16:23.871 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.852+0000 7f0b64084b80 1 client.25378 shutdown 2023-09-09T23:16:23.872 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.852+0000 7f0b64084b80 1 -- 172.21.15.82:0/4183003016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b351c3250 msgr2=0x560b351c0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:16:23.872 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.852+0000 7f0b64084b80 1 --2- 172.21.15.82:0/4183003016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b351c3250 0x560b351c0630 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0b48004660 tx=0x7f0b48004970 comp rx=0 tx=0).stop 2023-09-09T23:16:23.872 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.852+0000 7f0b64084b80 1 -- 172.21.15.82:0/4183003016 shutdown_connections 2023-09-09T23:16:23.872 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.852+0000 7f0b64084b80 1 --2- 172.21.15.82:0/4183003016 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x560b351d2e30 0x560b351d5250 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.873 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.852+0000 7f0b64084b80 1 --2- 172.21.15.82:0/4183003016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b351b4710 0x560b351c2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.873 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.852+0000 7f0b397fa700 0 client.25378 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:16:23.873 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.852+0000 7f0b64084b80 1 --2- 172.21.15.82:0/4183003016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b351c3250 0x560b351c0630 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.873 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.852+0000 7f0b64084b80 1 --2- 172.21.15.82:0/4183003016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b351b5030 0x560b351bff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.873 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.852+0000 7f0b64084b80 1 -- 172.21.15.82:0/4183003016 >> 172.21.15.82:0/4183003016 conn(0x560b35129090 msgr2=0x560b351af7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:16:23.874 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.852+0000 7f0b64084b80 1 -- 172.21.15.82:0/4183003016 shutdown_connections 2023-09-09T23:16:23.874 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.852+0000 7f0b64084b80 1 -- 172.21.15.82:0/4183003016 wait complete. 2023-09-09T23:16:23.874 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.852+0000 7f0b64084b80 20 client.25378 trim_cache size 0 max 16384 2023-09-09T23:16:23.877 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.876+0000 7f0b64084b80 1 Processor -- start 2023-09-09T23:16:23.877 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.876+0000 7f0b64084b80 1 -- start start 2023-09-09T23:16:23.877 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.876+0000 7f0b64084b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b351b4710 0x560b351ae320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:23.877 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.876+0000 7f0b64084b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b351b5030 0x560b351c27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:23.877 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.876+0000 7f0b64084b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b351c3250 0x560b351bb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:23.878 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.876+0000 7f0b64084b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b351d5270 con 0x560b351b4710 2023-09-09T23:16:23.878 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.876+0000 7f0b64084b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b351ae860 con 0x560b351c3250 2023-09-09T23:16:23.878 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.876+0000 7f0b64084b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b351aea20 con 0x560b351b5030 2023-09-09T23:16:23.878 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.876+0000 7f0b55c5a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b351b4710 0x560b351ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:23.878 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.876+0000 7f0b55c5a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b351b4710 0x560b351ae320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57640/0 (socket says 172.21.15.82:57640) 2023-09-09T23:16:23.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.876+0000 7f0b55c5a700 1 -- 172.21.15.82:0/1342294196 learned_addr learned my addr 172.21.15.82:0/1342294196 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:16:23.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.876+0000 7f0b55459700 1 --2- 172.21.15.82:0/1342294196 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b351b5030 0x560b351c27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:23.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.877+0000 7f0b5645b700 1 --2- 172.21.15.82:0/1342294196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b351c3250 0x560b351bb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:23.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.877+0000 7f0b55c5a700 1 -- 172.21.15.82:0/1342294196 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b351b5030 msgr2=0x560b351c27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:16:23.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.877+0000 7f0b55c5a700 1 --2- 172.21.15.82:0/1342294196 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b351b5030 0x560b351c27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.877+0000 7f0b55c5a700 1 -- 172.21.15.82:0/1342294196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b351c3250 msgr2=0x560b351bb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:16:23.880 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.877+0000 7f0b55c5a700 1 --2- 172.21.15.82:0/1342294196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b351c3250 0x560b351bb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.880 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.877+0000 7f0b55c5a700 1 -- 172.21.15.82:0/1342294196 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b34f1bc60 con 0x560b351b4710 2023-09-09T23:16:23.880 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.877+0000 7f0b55c5a700 1 --2- 172.21.15.82:0/1342294196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b351b4710 0x560b351ae320 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c09a5a0 tx=0x7f0b4c09d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46301a02e760e0f server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:16:23.880 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.877+0000 7f0b44ff9700 1 -- 172.21.15.82:0/1342294196 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c0adea0 con 0x560b351b4710 2023-09-09T23:16:23.880 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.877+0000 7f0b44ff9700 1 -- 172.21.15.82:0/1342294196 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b4c09a610 con 0x560b351b4710 2023-09-09T23:16:23.880 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.877+0000 7f0b44ff9700 1 -- 172.21.15.82:0/1342294196 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c0850b0 con 0x560b351b4710 2023-09-09T23:16:23.881 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.878+0000 7f0b64084b80 1 -- 172.21.15.82:0/1342294196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b351b4710 msgr2=0x560b351ae320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:16:23.881 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.878+0000 7f0b64084b80 1 --2- 172.21.15.82:0/1342294196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b351b4710 0x560b351ae320 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c09a5a0 tx=0x7f0b4c09d040 comp rx=0 tx=0).stop 2023-09-09T23:16:23.881 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.878+0000 7f0b64084b80 1 -- 172.21.15.82:0/1342294196 shutdown_connections 2023-09-09T23:16:23.881 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.878+0000 7f0b64084b80 1 --2- 172.21.15.82:0/1342294196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b351c3250 0x560b351bb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.881 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.878+0000 7f0b64084b80 1 --2- 172.21.15.82:0/1342294196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b351b4710 0x560b351ae320 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.882 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.878+0000 7f0b64084b80 1 --2- 172.21.15.82:0/1342294196 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b351b5030 0x560b351c27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.882 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.878+0000 7f0b64084b80 1 -- 172.21.15.82:0/1342294196 >> 172.21.15.82:0/1342294196 conn(0x560b35129090 msgr2=0x560b351d9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:16:23.882 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.878+0000 7f0b64084b80 1 -- 172.21.15.82:0/1342294196 shutdown_connections 2023-09-09T23:16:23.882 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.878+0000 7f0b64084b80 1 -- 172.21.15.82:0/1342294196 wait complete. 2023-09-09T23:16:23.883 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.878+0000 7f0b64084b80 1 -- start start 2023-09-09T23:16:23.883 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.878+0000 7f0b64084b80 1 Processor -- start 2023-09-09T23:16:23.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.879+0000 7f0b64084b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b351b4710 0x560b351c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:23.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.879+0000 7f0b64084b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b351b5030 0x560b351c8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:23.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.879+0000 7f0b64084b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b351c3250 0x560b35225c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:23.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.879+0000 7f0b64084b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b352261a0 con 0x560b351b4710 2023-09-09T23:16:23.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.879+0000 7f0b64084b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b352262e0 con 0x560b351c3250 2023-09-09T23:16:23.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.879+0000 7f0b64084b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b352264a0 con 0x560b351b5030 2023-09-09T23:16:23.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.879+0000 7f0b64084b80 10 client.0 _ll_register_callbacks cb 0x7f0b48003dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:16:23.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.879+0000 7f0b55459700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b351b5030 0x560b351c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:23.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.879+0000 7f0b55459700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b351b5030 0x560b351c8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:56162/0 (socket says 172.21.15.82:56162) 2023-09-09T23:16:23.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.879+0000 7f0b55459700 1 -- 172.21.15.82:0/4113442969 learned_addr learned my addr 172.21.15.82:0/4113442969 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:16:23.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.879+0000 7f0b5645b700 1 --2- 172.21.15.82:0/4113442969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b351c3250 0x560b35225c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:23.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.879+0000 7f0b55c5a700 1 --2- 172.21.15.82:0/4113442969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b351b4710 0x560b351c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:23.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.879+0000 7f0b55459700 1 -- 172.21.15.82:0/4113442969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b351c3250 msgr2=0x560b35225c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:16:23.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.879+0000 7f0b55459700 1 --2- 172.21.15.82:0/4113442969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b351c3250 0x560b35225c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.879+0000 7f0b55459700 1 -- 172.21.15.82:0/4113442969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b351b4710 msgr2=0x560b351c7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:16:23.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.879+0000 7f0b55459700 1 --2- 172.21.15.82:0/4113442969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b351b4710 0x560b351c7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.880+0000 7f0b55459700 1 -- 172.21.15.82:0/4113442969 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b501a34b0 con 0x560b351b5030 2023-09-09T23:16:23.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.880+0000 7f0b55c5a700 1 --2- 172.21.15.82:0/4113442969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b351b4710 0x560b351c7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:16:23.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.880+0000 7f0b55459700 1 --2- 172.21.15.82:0/4113442969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b351b5030 0x560b351c8170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009340 tx=0x7f0b4000ef50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6d9a615baea0d1c server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:16:23.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.880+0000 7f0b397fa700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-09T23:16:23.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.880+0000 7f0b397fa700 1 -- 172.21.15.82:0/4113442969 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b40005250 con 0x560b351b5030 2023-09-09T23:16:23.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.880+0000 7f0b397fa700 1 -- 172.21.15.82:0/4113442969 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f0b400053b0 con 0x560b351b5030 2023-09-09T23:16:23.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.880+0000 7f0b64084b80 1 -- 172.21.15.82:0/4113442969 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b40021070 con 0x560b351b5030 2023-09-09T23:16:23.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.880+0000 7f0b397fa700 1 -- 172.21.15.82:0/4113442969 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b40005570 con 0x560b351b5030 2023-09-09T23:16:23.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.880+0000 7f0b64084b80 10 client.25384 fetch_fsmap learned FSMap version 32 2023-09-09T23:16:23.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.880+0000 7f0b64084b80 1 -- 172.21.15.82:0/4113442969 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560b34f59250 con 0x560b351b5030 2023-09-09T23:16:23.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.880+0000 7f0b397fa700 1 -- 172.21.15.82:0/4113442969 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b40021070 con 0x560b351b5030 2023-09-09T23:16:23.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.880+0000 7f0b397fa700 1 -- 172.21.15.82:0/4113442969 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f0b4002d070 con 0x560b351b5030 2023-09-09T23:16:23.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.880+0000 7f0b64084b80 10 client.25384 fetch_fsmap finished waiting for FSMap version 32 2023-09-09T23:16:23.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.880+0000 7f0b64084b80 10 client.25384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:16:23.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.880+0000 7f0b64084b80 20 client.25384 populate_metadata read hostname 'smithi082' 2023-09-09T23:16:23.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.881+0000 7f0b64084b80 1 --2- 172.21.15.82:0/4113442969 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x560b35227cd0 0x560b3522a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:23.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.881+0000 7f0b64084b80 4 client.25384 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:16:23.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.881+0000 7f0b64084b80 1 -- 172.21.15.82:0/4113442969 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560b350ff920 con 0x560b35227cd0 2023-09-09T23:16:23.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.881+0000 7f0b55c5a700 1 --2- 172.21.15.82:0/4113442969 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x560b35227cd0 0x560b3522a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:23.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.881+0000 7f0b397fa700 10 client.25384 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:16:23.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.881+0000 7f0b55c5a700 1 --2- 172.21.15.82:0/4113442969 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x560b35227cd0 0x560b3522a0f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8578b8c3db73c776 server_cookie=ce2d004ba02c1c4c in_seq=0 out_seq=0 2023-09-09T23:16:23.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.881+0000 7f0b397fa700 1 -- 172.21.15.82:0/4113442969 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x560b350ff920 con 0x560b35227cd0 2023-09-09T23:16:23.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.881+0000 7f0b397fa700 10 client.25384 handle_command_reply: tid=0 2023-09-09T23:16:23.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.881+0000 7f0b64084b80 1 client.25384 shutdown 2023-09-09T23:16:23.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.882+0000 7f0b64084b80 1 -- 172.21.15.82:0/4113442969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b351b5030 msgr2=0x560b351c8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:16:23.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.882+0000 7f0b64084b80 1 --2- 172.21.15.82:0/4113442969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b351b5030 0x560b351c8170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009340 tx=0x7f0b4000ef50 comp rx=0 tx=0).stop 2023-09-09T23:16:23.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.882+0000 7f0b64084b80 1 -- 172.21.15.82:0/4113442969 shutdown_connections 2023-09-09T23:16:23.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.882+0000 7f0b64084b80 1 --2- 172.21.15.82:0/4113442969 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x560b35227cd0 0x560b3522a0f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.882+0000 7f0b64084b80 1 --2- 172.21.15.82:0/4113442969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b351c3250 0x560b35225c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.882+0000 7f0b397fa700 0 client.25384 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:16:23.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.882+0000 7f0b64084b80 1 --2- 172.21.15.82:0/4113442969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b351b4710 0x560b351c7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.882+0000 7f0b64084b80 1 --2- 172.21.15.82:0/4113442969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b351b5030 0x560b351c8170 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.882+0000 7f0b64084b80 1 -- 172.21.15.82:0/4113442969 >> 172.21.15.82:0/4113442969 conn(0x560b35129090 msgr2=0x560b351bb250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:16:23.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.882+0000 7f0b64084b80 1 -- 172.21.15.82:0/4113442969 shutdown_connections 2023-09-09T23:16:23.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.882+0000 7f0b64084b80 1 -- 172.21.15.82:0/4113442969 wait complete. 2023-09-09T23:16:23.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.882+0000 7f0b64084b80 20 client.25384 trim_cache size 0 max 16384 2023-09-09T23:16:23.896 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:16:23.897 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:16:23.897 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:16:23.897 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:16:23.897 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:16:23.897 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:16:23.897 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:16:23.898 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:16:23.898 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:16:23.898 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:16:23.899 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.882+0000 7f0b57ebe700 1 -- 172.21.15.82:0/4168656878 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f0b3c0a07b0 msgr2=0x7f0b3c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:16:23.899 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.882+0000 7f0b57ebe700 1 --2- 172.21.15.82:0/4168656878 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f0b3c0a07b0 0x7f0b3c0a2c70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0b50197c90 tx=0x7f0b40009260 comp rx=0 tx=0).stop 2023-09-09T23:16:23.899 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.882+0000 7f0b57ebe700 1 -- 172.21.15.82:0/4168656878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b50100590 msgr2=0x7f0b501966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:16:23.900 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.882+0000 7f0b57ebe700 1 --2- 172.21.15.82:0/4168656878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b50100590 0x7f0b501966d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00b580 tx=0x7f0b4c00f690 comp rx=0 tx=0).stop 2023-09-09T23:16:23.900 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.882+0000 7f0b57ebe700 1 -- 172.21.15.82:0/4168656878 shutdown_connections 2023-09-09T23:16:23.900 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.882+0000 7f0b57ebe700 1 --2- 172.21.15.82:0/4168656878 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f0b3c0a07b0 0x7f0b3c0a2c70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.900 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.882+0000 7f0b57ebe700 1 --2- 172.21.15.82:0/4168656878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b50106650 0x7f0b5019b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.900 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.882+0000 7f0b57ebe700 1 --2- 172.21.15.82:0/4168656878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b50100590 0x7f0b501966d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.901 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.882+0000 7f0b57ebe700 1 --2- 172.21.15.82:0/4168656878 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b50100f50 0x7f0b50196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:23.901 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.882+0000 7f0b57ebe700 1 -- 172.21.15.82:0/4168656878 >> 172.21.15.82:0/4168656878 conn(0x7f0b500fc060 msgr2=0x7f0b500fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:16:23.901 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.883+0000 7f0b57ebe700 1 -- 172.21.15.82:0/4168656878 shutdown_connections 2023-09-09T23:16:23.901 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:23.883+0000 7f0b57ebe700 1 -- 172.21.15.82:0/4168656878 wait complete. 2023-09-09T23:16:23.902 INFO:teuthology.orchestra.run.smithi082.stderr:48800000 of 50000000 tuples (97%) done (elapsed 130.48 s, remaining 3.21 s) 2023-09-09T23:16:23.903 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:16:25.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:25 smithi170 ceph-mon[102832]: pgmap v78: 97 pgs: 97 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 361 op/s 2023-09-09T23:16:25.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:25 smithi082 ceph-mon[98694]: pgmap v78: 97 pgs: 97 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 361 op/s 2023-09-09T23:16:25.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:25 smithi098 ceph-mon[103124]: pgmap v78: 97 pgs: 97 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 361 op/s 2023-09-09T23:16:26.201 INFO:teuthology.orchestra.run.smithi082.stderr:48900000 of 50000000 tuples (97%) done (elapsed 132.79 s, remaining 2.99 s) 2023-09-09T23:16:26.297 INFO:teuthology.orchestra.run.smithi082.stderr:49000000 of 50000000 tuples (98%) done (elapsed 132.88 s, remaining 2.71 s) 2023-09-09T23:16:26.847 INFO:teuthology.orchestra.run.smithi082.stderr:49100000 of 50000000 tuples (98%) done (elapsed 133.43 s, remaining 2.45 s) 2023-09-09T23:16:27.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:27 smithi170 ceph-mon[102832]: pgmap v79: 97 pgs: 97 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 634 op/s 2023-09-09T23:16:27.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:27 smithi082 ceph-mon[98694]: pgmap v79: 97 pgs: 97 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 634 op/s 2023-09-09T23:16:27.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:27 smithi098 ceph-mon[103124]: pgmap v79: 97 pgs: 97 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 634 op/s 2023-09-09T23:16:29.499 INFO:teuthology.orchestra.run.smithi082.stderr:49200000 of 50000000 tuples (98%) done (elapsed 136.08 s, remaining 2.21 s) 2023-09-09T23:16:29.608 INFO:teuthology.orchestra.run.smithi082.stderr:49300000 of 50000000 tuples (98%) done (elapsed 136.19 s, remaining 1.93 s) 2023-09-09T23:16:29.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:29 smithi170 ceph-mon[102832]: pgmap v80: 97 pgs: 97 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 MiB/s wr, 360 op/s 2023-09-09T23:16:29.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:29 smithi082 ceph-mon[98694]: pgmap v80: 97 pgs: 97 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 MiB/s wr, 360 op/s 2023-09-09T23:16:29.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:29 smithi098 ceph-mon[103124]: pgmap v80: 97 pgs: 97 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 MiB/s wr, 360 op/s 2023-09-09T23:16:31.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:31 smithi170 ceph-mon[102832]: pgmap v81: 97 pgs: 97 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 MiB/s wr, 422 op/s 2023-09-09T23:16:31.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:31 smithi082 ceph-mon[98694]: pgmap v81: 97 pgs: 97 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 MiB/s wr, 422 op/s 2023-09-09T23:16:31.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:31 smithi098 ceph-mon[103124]: pgmap v81: 97 pgs: 97 active+clean; 8.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 MiB/s wr, 422 op/s 2023-09-09T23:16:33.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:33 smithi170 ceph-mon[102832]: pgmap v82: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 361 op/s 2023-09-09T23:16:33.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:33 smithi082 ceph-mon[98694]: pgmap v82: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 361 op/s 2023-09-09T23:16:33.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:33 smithi098 ceph-mon[103124]: pgmap v82: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 MiB/s wr, 361 op/s 2023-09-09T23:16:33.936 INFO:teuthology.orchestra.run.smithi082.stderr:49400000 of 50000000 tuples (98%) done (elapsed 140.52 s, remaining 1.71 s) 2023-09-09T23:16:34.596 INFO:teuthology.orchestra.run.smithi082.stderr:49500000 of 50000000 tuples (99%) done (elapsed 141.18 s, remaining 1.43 s) 2023-09-09T23:16:34.704 INFO:teuthology.orchestra.run.smithi082.stderr:49600000 of 50000000 tuples (99%) done (elapsed 141.29 s, remaining 1.14 s) 2023-09-09T23:16:35.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:35 smithi170 ceph-mon[102832]: pgmap v83: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 342 op/s 2023-09-09T23:16:35.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:35 smithi082 ceph-mon[98694]: pgmap v83: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 342 op/s 2023-09-09T23:16:35.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:35 smithi098 ceph-mon[103124]: pgmap v83: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 342 op/s 2023-09-09T23:16:37.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:37 smithi170 ceph-mon[102832]: pgmap v84: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 17 MiB/s wr, 354 op/s 2023-09-09T23:16:37.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:37 smithi082 ceph-mon[98694]: pgmap v84: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 17 MiB/s wr, 354 op/s 2023-09-09T23:16:37.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:37 smithi098 ceph-mon[103124]: pgmap v84: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 17 MiB/s wr, 354 op/s 2023-09-09T23:16:39.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:39 smithi170 ceph-mon[102832]: pgmap v85: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 15 MiB/s wr, 81 op/s 2023-09-09T23:16:39.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:39 smithi082 ceph-mon[98694]: pgmap v85: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 15 MiB/s wr, 81 op/s 2023-09-09T23:16:39.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:39 smithi098 ceph-mon[103124]: pgmap v85: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 15 MiB/s wr, 81 op/s 2023-09-09T23:16:40.404 INFO:teuthology.orchestra.run.smithi082.stderr:49700000 of 50000000 tuples (99%) done (elapsed 146.99 s, remaining 0.89 s) 2023-09-09T23:16:41.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:41 smithi170 ceph-mon[102832]: pgmap v86: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 87 op/s 2023-09-09T23:16:41.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:41 smithi082 ceph-mon[98694]: pgmap v86: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 87 op/s 2023-09-09T23:16:41.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:41 smithi098 ceph-mon[103124]: pgmap v86: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 15 MiB/s wr, 87 op/s 2023-09-09T23:16:43.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:43 smithi170 ceph-mon[102832]: pgmap v87: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.6 MiB/s wr, 25 op/s 2023-09-09T23:16:43.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:43 smithi082 ceph-mon[98694]: pgmap v87: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.6 MiB/s wr, 25 op/s 2023-09-09T23:16:43.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:43 smithi098 ceph-mon[103124]: pgmap v87: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.6 MiB/s wr, 25 op/s 2023-09-09T23:16:44.730 INFO:teuthology.orchestra.run.smithi082.stderr:49800000 of 50000000 tuples (99%) done (elapsed 151.31 s, remaining 0.61 s) 2023-09-09T23:16:45.366 INFO:teuthology.orchestra.run.smithi082.stderr:49900000 of 50000000 tuples (99%) done (elapsed 151.95 s, remaining 0.30 s) 2023-09-09T23:16:45.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:45 smithi170 ceph-mon[102832]: pgmap v88: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.8 MiB/s wr, 18 op/s 2023-09-09T23:16:45.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:45 smithi082 ceph-mon[98694]: pgmap v88: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.8 MiB/s wr, 18 op/s 2023-09-09T23:16:45.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:45 smithi098 ceph-mon[103124]: pgmap v88: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.8 MiB/s wr, 18 op/s 2023-09-09T23:16:47.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:47 smithi170 ceph-mon[102832]: pgmap v89: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 44 op/s 2023-09-09T23:16:47.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:47 smithi082 ceph-mon[98694]: pgmap v89: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 44 op/s 2023-09-09T23:16:47.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:47 smithi098 ceph-mon[103124]: pgmap v89: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 44 op/s 2023-09-09T23:16:49.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:49 smithi170 ceph-mon[102832]: pgmap v90: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.4 MiB/s wr, 32 op/s 2023-09-09T23:16:49.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:49 smithi082 ceph-mon[98694]: pgmap v90: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.4 MiB/s wr, 32 op/s 2023-09-09T23:16:49.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:49 smithi098 ceph-mon[103124]: pgmap v90: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.4 MiB/s wr, 32 op/s 2023-09-09T23:16:51.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:51 smithi170 ceph-mon[102832]: pgmap v91: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.0 MiB/s wr, 52 op/s 2023-09-09T23:16:51.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:51 smithi082 ceph-mon[98694]: pgmap v91: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.0 MiB/s wr, 52 op/s 2023-09-09T23:16:51.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:51 smithi098 ceph-mon[103124]: pgmap v91: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.0 MiB/s wr, 52 op/s 2023-09-09T23:16:53.017 INFO:teuthology.orchestra.run.smithi082.stderr:50000000 of 50000000 tuples (100%) done (elapsed 159.60 s, remaining 0.00 s) 2023-09-09T23:16:53.043 INFO:teuthology.orchestra.run.smithi082.stderr:vacuum... 2023-09-09T23:16:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:53 smithi170 ceph-mon[102832]: pgmap v92: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.4 MiB/s wr, 47 op/s 2023-09-09T23:16:53.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:53 smithi082 ceph-mon[98694]: pgmap v92: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.4 MiB/s wr, 47 op/s 2023-09-09T23:16:53.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:53 smithi098 ceph-mon[103124]: pgmap v92: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.4 MiB/s wr, 47 op/s 2023-09-09T23:16:53.904 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:16:54.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.031+0000 7fc0d279d700 1 -- 172.21.15.82:0/2114572337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0cc100f40 msgr2=0x7fc0cc10d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:16:54.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.031+0000 7fc0d279d700 1 --2- 172.21.15.82:0/2114572337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0cc100f40 0x7fc0cc10d400 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400b0d0 tx=0x7fc0c400b3e0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.031+0000 7fc0d279d700 1 -- 172.21.15.82:0/2114572337 shutdown_connections 2023-09-09T23:16:54.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.031+0000 7fc0d279d700 1 --2- 172.21.15.82:0/2114572337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0cc106640 0x7fc0cc106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.031+0000 7fc0d279d700 1 --2- 172.21.15.82:0/2114572337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0cc100f40 0x7fc0cc10d400 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.031+0000 7fc0d279d700 1 --2- 172.21.15.82:0/2114572337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0cc100580 0x7fc0cc100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.031+0000 7fc0d279d700 1 -- 172.21.15.82:0/2114572337 >> 172.21.15.82:0/2114572337 conn(0x7fc0cc0fc030 msgr2=0x7fc0cc0fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:16:54.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.031+0000 7fc0d279d700 1 -- 172.21.15.82:0/2114572337 shutdown_connections 2023-09-09T23:16:54.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.031+0000 7fc0d279d700 1 -- 172.21.15.82:0/2114572337 wait complete. 2023-09-09T23:16:54.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.032+0000 7fc0d279d700 1 Processor -- start 2023-09-09T23:16:54.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.032+0000 7fc0d279d700 1 -- start start 2023-09-09T23:16:54.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.032+0000 7fc0d279d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0cc100580 0x7fc0cc196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:54.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.032+0000 7fc0d279d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0cc100f40 0x7fc0cc196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:54.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.032+0000 7fc0d279d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0cc106640 0x7fc0cc199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:54.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.032+0000 7fc0d279d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc19a580 con 0x7fc0cc100f40 2023-09-09T23:16:54.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.032+0000 7fc0d279d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc19a6f0 con 0x7fc0cc106640 2023-09-09T23:16:54.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.032+0000 7fc0d279d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc19a8b0 con 0x7fc0cc100580 2023-09-09T23:16:54.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.032+0000 7fc0cb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0cc100f40 0x7fc0cc196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:54.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.032+0000 7fc0d0d3a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0cc106640 0x7fc0cc199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:54.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.032+0000 7fc0cb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0cc100f40 0x7fc0cc196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57664/0 (socket says 172.21.15.82:57664) 2023-09-09T23:16:54.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.032+0000 7fc0cbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0cc100580 0x7fc0cc196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:54.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.032+0000 7fc0cb7fe700 1 -- 172.21.15.82:0/961076596 learned_addr learned my addr 172.21.15.82:0/961076596 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:16:54.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.032+0000 7fc0cb7fe700 1 -- 172.21.15.82:0/961076596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0cc100580 msgr2=0x7fc0cc196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:16:54.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.032+0000 7fc0cb7fe700 1 --2- 172.21.15.82:0/961076596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0cc100580 0x7fc0cc196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.032+0000 7fc0cb7fe700 1 -- 172.21.15.82:0/961076596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0cc106640 msgr2=0x7fc0cc199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:16:54.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.032+0000 7fc0cb7fe700 1 --2- 172.21.15.82:0/961076596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0cc106640 0x7fc0cc199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.032+0000 7fc0cb7fe700 1 -- 172.21.15.82:0/961076596 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c4009d00 con 0x7fc0cc100f40 2023-09-09T23:16:54.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.033+0000 7fc0cb7fe700 1 --2- 172.21.15.82:0/961076596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0cc100f40 0x7fc0cc196bd0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000d950 tx=0x7fc0c000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:16:54.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.033+0000 7fc0c97fa700 1 -- 172.21.15.82:0/961076596 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c000f890 con 0x7fc0cc100f40 2023-09-09T23:16:54.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.033+0000 7fc0c97fa700 1 -- 172.21.15.82:0/961076596 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0c0010460 con 0x7fc0cc100f40 2023-09-09T23:16:54.041 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.033+0000 7fc0d279d700 1 -- 172.21.15.82:0/961076596 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0cc19ab90 con 0x7fc0cc100f40 2023-09-09T23:16:54.041 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.033+0000 7fc0c97fa700 1 -- 172.21.15.82:0/961076596 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c000e610 con 0x7fc0cc100f40 2023-09-09T23:16:54.041 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.033+0000 7fc0d279d700 1 -- 172.21.15.82:0/961076596 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0cc1a34f0 con 0x7fc0cc100f40 2023-09-09T23:16:54.047 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.035+0000 7fc0c97fa700 1 -- 172.21.15.82:0/961076596 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fc0c00108d0 con 0x7fc0cc100f40 2023-09-09T23:16:54.047 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.035+0000 7fc0c97fa700 1 --2- 172.21.15.82:0/961076596 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fc0b40a0830 0x7fc0b40a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:54.047 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.036+0000 7fc0cbfff700 1 --2- 172.21.15.82:0/961076596 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fc0b40a0830 0x7fc0b40a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:54.047 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.036+0000 7fc0c97fa700 1 -- 172.21.15.82:0/961076596 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fc0c00cbca0 con 0x7fc0cc100f40 2023-09-09T23:16:54.047 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.036+0000 7fc0cbfff700 1 --2- 172.21.15.82:0/961076596 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fc0b40a0830 0x7fc0b40a2cf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009c60 tx=0x7fc0bc009260 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:16:54.048 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.036+0000 7fc0de963b80 1 Processor -- start 2023-09-09T23:16:54.048 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.036+0000 7fc0de963b80 1 -- start start 2023-09-09T23:16:54.048 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.037+0000 7fc0de963b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d03f50710 0x556d03f50af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:54.048 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.037+0000 7fc0de963b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d03f51030 0x556d03f5a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:54.049 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.037+0000 7fc0de963b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d03f5ae40 0x556d03f5d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:54.049 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.037+0000 7fc0de963b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d03ea9730 con 0x556d03f50710 2023-09-09T23:16:54.049 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.037+0000 7fc0de963b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d03ea7050 con 0x556d03f5ae40 2023-09-09T23:16:54.049 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.037+0000 7fc0de963b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d03ea2940 con 0x556d03f51030 2023-09-09T23:16:54.049 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.037+0000 7fc0cbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d03f50710 0x556d03f50af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:54.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.037+0000 7fc0cbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d03f50710 0x556d03f50af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57668/0 (socket says 172.21.15.82:57668) 2023-09-09T23:16:54.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.037+0000 7fc0cbfff700 1 -- 172.21.15.82:0/3311237163 learned_addr learned my addr 172.21.15.82:0/3311237163 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:16:54.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.037+0000 7fc0d0d3a700 1 --2- 172.21.15.82:0/3311237163 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d03f5ae40 0x556d03f5d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:54.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.037+0000 7fc0cb7fe700 1 --2- 172.21.15.82:0/3311237163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d03f51030 0x556d03f5a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:54.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.037+0000 7fc0cbfff700 1 -- 172.21.15.82:0/3311237163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d03f51030 msgr2=0x556d03f5a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:16:54.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.037+0000 7fc0cbfff700 1 --2- 172.21.15.82:0/3311237163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d03f51030 0x556d03f5a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.037+0000 7fc0cbfff700 1 -- 172.21.15.82:0/3311237163 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d03f5ae40 msgr2=0x556d03f5d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:16:54.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.037+0000 7fc0cbfff700 1 --2- 172.21.15.82:0/3311237163 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d03f5ae40 0x556d03f5d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.037+0000 7fc0cbfff700 1 -- 172.21.15.82:0/3311237163 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d03ccede0 con 0x556d03f50710 2023-09-09T23:16:54.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.037+0000 7fc0cbfff700 1 --2- 172.21.15.82:0/3311237163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d03f50710 0x556d03f50af0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc01e860 tx=0x7fc0bc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388935600dfa4b0a server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:16:54.052 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.037+0000 7fc0b27fc700 1 -- 172.21.15.82:0/3311237163 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc023070 con 0x556d03f50710 2023-09-09T23:16:54.052 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.037+0000 7fc0b27fc700 1 -- 172.21.15.82:0/3311237163 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0bc0298e0 con 0x556d03f50710 2023-09-09T23:16:54.052 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.037+0000 7fc0b27fc700 1 -- 172.21.15.82:0/3311237163 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc00e850 con 0x556d03f50710 2023-09-09T23:16:54.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.038+0000 7fc0de963b80 1 -- 172.21.15.82:0/3311237163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d03f50710 msgr2=0x556d03f50af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:16:54.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.038+0000 7fc0de963b80 1 --2- 172.21.15.82:0/3311237163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d03f50710 0x556d03f50af0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc01e860 tx=0x7fc0bc014000 comp rx=0 tx=0).stop 2023-09-09T23:16:54.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.038+0000 7fc0de963b80 1 -- 172.21.15.82:0/3311237163 shutdown_connections 2023-09-09T23:16:54.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.038+0000 7fc0de963b80 1 --2- 172.21.15.82:0/3311237163 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d03f5ae40 0x556d03f5d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.054 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.038+0000 7fc0de963b80 1 --2- 172.21.15.82:0/3311237163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d03f50710 0x556d03f50af0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.054 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.038+0000 7fc0de963b80 1 --2- 172.21.15.82:0/3311237163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d03f51030 0x556d03f5a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.054 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.038+0000 7fc0de963b80 1 -- 172.21.15.82:0/3311237163 >> 172.21.15.82:0/3311237163 conn(0x556d03ec5090 msgr2=0x556d03f4bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:16:54.054 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.038+0000 7fc0de963b80 1 -- 172.21.15.82:0/3311237163 shutdown_connections 2023-09-09T23:16:54.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.038+0000 7fc0de963b80 1 -- 172.21.15.82:0/3311237163 wait complete. 2023-09-09T23:16:54.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.038+0000 7fc0de963b80 1 -- start start 2023-09-09T23:16:54.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.039+0000 7fc0de963b80 1 Processor -- start 2023-09-09T23:16:54.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.039+0000 7fc0de963b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d03f50710 0x556d03f5a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:54.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.039+0000 7fc0de963b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d03f51030 0x556d03f579f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:54.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.039+0000 7fc0de963b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d03f5ae40 0x556d03f57fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:54.056 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.039+0000 7fc0de963b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d03e40b70 con 0x556d03f51030 2023-09-09T23:16:54.056 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.039+0000 7fc0de963b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d03e40870 con 0x556d03f50710 2023-09-09T23:16:54.056 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.039+0000 7fc0de963b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d03d24920 con 0x556d03f5ae40 2023-09-09T23:16:54.056 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.039+0000 7fc0de963b80 10 client.0 _ll_register_callbacks cb 0x556d03d8aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:16:54.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.039+0000 7fc0cb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d03f51030 0x556d03f579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:54.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.039+0000 7fc0cb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d03f51030 0x556d03f579f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57676/0 (socket says 172.21.15.82:57676) 2023-09-09T23:16:54.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.039+0000 7fc0cb7fe700 1 -- 172.21.15.82:0/2841330464 learned_addr learned my addr 172.21.15.82:0/2841330464 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:16:54.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.039+0000 7fc0cbfff700 1 --2- 172.21.15.82:0/2841330464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d03f50710 0x556d03f5a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:54.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.039+0000 7fc0d0d3a700 1 --2- 172.21.15.82:0/2841330464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d03f5ae40 0x556d03f57fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:54.058 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.039+0000 7fc0cb7fe700 1 -- 172.21.15.82:0/2841330464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d03f5ae40 msgr2=0x556d03f57fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:16:54.058 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.039+0000 7fc0cb7fe700 1 --2- 172.21.15.82:0/2841330464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d03f5ae40 0x556d03f57fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.058 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.039+0000 7fc0cb7fe700 1 -- 172.21.15.82:0/2841330464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d03f50710 msgr2=0x556d03f5a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:16:54.058 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.039+0000 7fc0cb7fe700 1 --2- 172.21.15.82:0/2841330464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d03f50710 0x556d03f5a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.058 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.039+0000 7fc0cb7fe700 1 -- 172.21.15.82:0/2841330464 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0bc017070 con 0x556d03f51030 2023-09-09T23:16:54.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.039+0000 7fc0cb7fe700 1 --2- 172.21.15.82:0/2841330464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d03f51030 0x556d03f579f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0010fd0 tx=0x7fc0c0099300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fcbdf0a841389a3 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:16:54.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.039+0000 7fc0ab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:16:54.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.040+0000 7fc0ab7fe700 1 -- 172.21.15.82:0/2841330464 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0099420 con 0x556d03f51030 2023-09-09T23:16:54.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.040+0000 7fc0de963b80 1 -- 172.21.15.82:0/2841330464 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d03dfa4f0 con 0x556d03f51030 2023-09-09T23:16:54.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.040+0000 7fc0ab7fe700 1 -- 172.21.15.82:0/2841330464 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fc0c0099580 con 0x556d03f51030 2023-09-09T23:16:54.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.040+0000 7fc0ab7fe700 1 -- 172.21.15.82:0/2841330464 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0c00b71a0 con 0x556d03f51030 2023-09-09T23:16:54.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.040+0000 7fc0de963b80 10 client.16029 fetch_fsmap learned FSMap version 32 2023-09-09T23:16:54.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.040+0000 7fc0de963b80 1 -- 172.21.15.82:0/2841330464 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x556d03cb7c60 con 0x556d03f51030 2023-09-09T23:16:54.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.040+0000 7fc0ab7fe700 1 -- 172.21.15.82:0/2841330464 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556d03dfa4f0 con 0x556d03f51030 2023-09-09T23:16:54.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.040+0000 7fc0ab7fe700 1 -- 172.21.15.82:0/2841330464 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fc0c00a0ec0 con 0x556d03f51030 2023-09-09T23:16:54.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.040+0000 7fc0de963b80 10 client.16029 fetch_fsmap finished waiting for FSMap version 32 2023-09-09T23:16:54.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.040+0000 7fc0de963b80 10 client.16029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:16:54.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.040+0000 7fc0de963b80 20 client.16029 populate_metadata read hostname 'smithi082' 2023-09-09T23:16:54.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.040+0000 7fc0de963b80 1 --2- 172.21.15.82:0/2841330464 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x556d03f6a8f0 0x556d03f6cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:54.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.040+0000 7fc0de963b80 4 client.16029 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:16:54.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.040+0000 7fc0de963b80 1 -- 172.21.15.82:0/2841330464 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d03eb78a0 con 0x556d03f6a8f0 2023-09-09T23:16:54.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.040+0000 7fc0cbfff700 1 --2- 172.21.15.82:0/2841330464 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x556d03f6a8f0 0x556d03f6cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:54.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.041+0000 7fc0ab7fe700 10 client.16029 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:16:54.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.041+0000 7fc0cbfff700 1 --2- 172.21.15.82:0/2841330464 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x556d03f6a8f0 0x556d03f6cd10 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab4d588098a8138c server_cookie=c5b07533f0fed84d in_seq=0 out_seq=0 2023-09-09T23:16:54.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.041+0000 7fc0ab7fe700 1 -- 172.21.15.82:0/2841330464 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556d03eb78a0 con 0x556d03f6a8f0 2023-09-09T23:16:54.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.041+0000 7fc0ab7fe700 10 client.16029 handle_command_reply: tid=0 2023-09-09T23:16:54.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.041+0000 7fc0de963b80 1 client.16029 shutdown 2023-09-09T23:16:54.063 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.042+0000 7fc0de963b80 1 -- 172.21.15.82:0/2841330464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d03f51030 msgr2=0x556d03f579f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:16:54.063 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.042+0000 7fc0de963b80 1 --2- 172.21.15.82:0/2841330464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d03f51030 0x556d03f579f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0010fd0 tx=0x7fc0c0099300 comp rx=0 tx=0).stop 2023-09-09T23:16:54.063 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.042+0000 7fc0de963b80 1 -- 172.21.15.82:0/2841330464 shutdown_connections 2023-09-09T23:16:54.063 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.042+0000 7fc0de963b80 1 --2- 172.21.15.82:0/2841330464 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x556d03f6a8f0 0x556d03f6cd10 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.063 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.042+0000 7fc0de963b80 1 --2- 172.21.15.82:0/2841330464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d03f50710 0x556d03f5a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.042+0000 7fc0ab7fe700 0 client.16029 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:16:54.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.042+0000 7fc0de963b80 1 --2- 172.21.15.82:0/2841330464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d03f5ae40 0x556d03f57fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.042+0000 7fc0de963b80 1 --2- 172.21.15.82:0/2841330464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d03f51030 0x556d03f579f0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.042+0000 7fc0de963b80 1 -- 172.21.15.82:0/2841330464 >> 172.21.15.82:0/2841330464 conn(0x556d03ec5090 msgr2=0x556d03f4b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:16:54.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.042+0000 7fc0de963b80 1 -- 172.21.15.82:0/2841330464 shutdown_connections 2023-09-09T23:16:54.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.042+0000 7fc0de963b80 1 -- 172.21.15.82:0/2841330464 wait complete. 2023-09-09T23:16:54.065 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.042+0000 7fc0de963b80 20 client.16029 trim_cache size 0 max 16384 2023-09-09T23:16:54.070 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.058+0000 7fc0de963b80 1 Processor -- start 2023-09-09T23:16:54.070 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.058+0000 7fc0de963b80 1 -- start start 2023-09-09T23:16:54.070 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.058+0000 7fc0de963b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d03f50710 0x556d03f619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:54.070 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.058+0000 7fc0de963b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d03f51030 0x556d03f5a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:54.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.058+0000 7fc0de963b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d03f5ae40 0x556d03f52d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:54.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.058+0000 7fc0de963b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d03f48420 con 0x556d03f5ae40 2023-09-09T23:16:54.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.058+0000 7fc0de963b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d03f61f60 con 0x556d03f51030 2023-09-09T23:16:54.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.058+0000 7fc0de963b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d03f51410 con 0x556d03f50710 2023-09-09T23:16:54.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.058+0000 7fc0d0d3a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d03f5ae40 0x556d03f52d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:54.072 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.058+0000 7fc0cbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d03f50710 0x556d03f619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:54.072 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.058+0000 7fc0d0d3a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d03f5ae40 0x556d03f52d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57686/0 (socket says 172.21.15.82:57686) 2023-09-09T23:16:54.072 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.058+0000 7fc0d0d3a700 1 -- 172.21.15.82:0/958781214 learned_addr learned my addr 172.21.15.82:0/958781214 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:16:54.072 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.058+0000 7fc0cbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d03f50710 0x556d03f619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:56196/0 (socket says 172.21.15.82:56196) 2023-09-09T23:16:54.072 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.058+0000 7fc0cb7fe700 1 --2- 172.21.15.82:0/958781214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d03f51030 0x556d03f5a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:54.072 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.059+0000 7fc0cbfff700 1 -- 172.21.15.82:0/958781214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d03f51030 msgr2=0x556d03f5a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:16:54.073 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.059+0000 7fc0cbfff700 1 --2- 172.21.15.82:0/958781214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d03f51030 0x556d03f5a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.073 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.059+0000 7fc0cbfff700 1 -- 172.21.15.82:0/958781214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d03f5ae40 msgr2=0x556d03f52d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:16:54.073 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.059+0000 7fc0cbfff700 1 --2- 172.21.15.82:0/958781214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d03f5ae40 0x556d03f52d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.073 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.059+0000 7fc0cbfff700 1 -- 172.21.15.82:0/958781214 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d03cf5250 con 0x556d03f50710 2023-09-09T23:16:54.073 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.059+0000 7fc0d0d3a700 1 --2- 172.21.15.82:0/958781214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d03f5ae40 0x556d03f52d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:16:54.074 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.059+0000 7fc0cbfff700 1 --2- 172.21.15.82:0/958781214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d03f50710 0x556d03f619f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc011ab0 tx=0x7fc0bc00e450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=286b144b7a572d15 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:16:54.074 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.059+0000 7fc0b2ffd700 1 -- 172.21.15.82:0/958781214 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc023040 con 0x556d03f50710 2023-09-09T23:16:54.074 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.059+0000 7fc0b2ffd700 1 -- 172.21.15.82:0/958781214 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0bc029620 con 0x556d03f50710 2023-09-09T23:16:54.074 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.059+0000 7fc0b2ffd700 1 -- 172.21.15.82:0/958781214 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc0050b0 con 0x556d03f50710 2023-09-09T23:16:54.074 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.060+0000 7fc0de963b80 1 -- 172.21.15.82:0/958781214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d03f50710 msgr2=0x556d03f619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:16:54.074 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.060+0000 7fc0de963b80 1 --2- 172.21.15.82:0/958781214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d03f50710 0x556d03f619f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc011ab0 tx=0x7fc0bc00e450 comp rx=0 tx=0).stop 2023-09-09T23:16:54.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.060+0000 7fc0de963b80 1 -- 172.21.15.82:0/958781214 shutdown_connections 2023-09-09T23:16:54.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.060+0000 7fc0de963b80 1 --2- 172.21.15.82:0/958781214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d03f5ae40 0x556d03f52d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.060+0000 7fc0de963b80 1 --2- 172.21.15.82:0/958781214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d03f50710 0x556d03f619f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.060+0000 7fc0de963b80 1 --2- 172.21.15.82:0/958781214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d03f51030 0x556d03f5a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.060+0000 7fc0de963b80 1 -- 172.21.15.82:0/958781214 >> 172.21.15.82:0/958781214 conn(0x556d03ec5090 msgr2=0x556d03f66140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:16:54.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.060+0000 7fc0de963b80 1 -- 172.21.15.82:0/958781214 shutdown_connections 2023-09-09T23:16:54.076 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.060+0000 7fc0de963b80 1 -- 172.21.15.82:0/958781214 wait complete. 2023-09-09T23:16:54.076 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.060+0000 7fc0de963b80 1 -- start start 2023-09-09T23:16:54.076 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.060+0000 7fc0de963b80 1 Processor -- start 2023-09-09T23:16:54.076 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.061+0000 7fc0de963b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d03f50710 0x556d03f61880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:54.076 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.061+0000 7fc0de963b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d03f51030 0x556d03f5fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:54.076 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.061+0000 7fc0de963b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d03f5ae40 0x556d03f600f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:54.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.061+0000 7fc0de963b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d03f607e0 con 0x556d03f50710 2023-09-09T23:16:54.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.061+0000 7fc0de963b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d03f60950 con 0x556d03f51030 2023-09-09T23:16:54.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.061+0000 7fc0de963b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d03fbe710 con 0x556d03f5ae40 2023-09-09T23:16:54.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.061+0000 7fc0de963b80 10 client.0 _ll_register_callbacks cb 0x556d03d8aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:16:54.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.061+0000 7fc0cbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d03f50710 0x556d03f61880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:54.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.061+0000 7fc0cbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d03f50710 0x556d03f61880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57688/0 (socket says 172.21.15.82:57688) 2023-09-09T23:16:54.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.061+0000 7fc0cb7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d03f51030 0x556d03f5fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:54.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.061+0000 7fc0cbfff700 1 -- 172.21.15.82:0/1781934359 learned_addr learned my addr 172.21.15.82:0/1781934359 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:16:54.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.061+0000 7fc0d0d3a700 1 --2- 172.21.15.82:0/1781934359 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d03f5ae40 0x556d03f600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:54.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.061+0000 7fc0cbfff700 1 -- 172.21.15.82:0/1781934359 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d03f5ae40 msgr2=0x556d03f600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:16:54.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.061+0000 7fc0cbfff700 1 --2- 172.21.15.82:0/1781934359 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d03f5ae40 0x556d03f600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.061+0000 7fc0cbfff700 1 -- 172.21.15.82:0/1781934359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d03f51030 msgr2=0x556d03f5fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:16:54.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.061+0000 7fc0cbfff700 1 --2- 172.21.15.82:0/1781934359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d03f51030 0x556d03f5fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.061+0000 7fc0cbfff700 1 -- 172.21.15.82:0/1781934359 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0cc10dd00 con 0x556d03f50710 2023-09-09T23:16:54.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.061+0000 7fc0cbfff700 1 --2- 172.21.15.82:0/1781934359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d03f50710 0x556d03f61880 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009e90 tx=0x7fc0bc0096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a395974a55ab4e60 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:16:54.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.061+0000 7fc0ab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:16:54.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.062+0000 7fc0ab7fe700 1 -- 172.21.15.82:0/1781934359 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc023070 con 0x556d03f50710 2023-09-09T23:16:54.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.062+0000 7fc0ab7fe700 1 -- 172.21.15.82:0/1781934359 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7fc0bc005b40 con 0x556d03f50710 2023-09-09T23:16:54.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.062+0000 7fc0de963b80 1 -- 172.21.15.82:0/1781934359 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc0bc0298e0 con 0x556d03f50710 2023-09-09T23:16:54.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.062+0000 7fc0ab7fe700 1 -- 172.21.15.82:0/1781934359 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0bc035280 con 0x556d03f50710 2023-09-09T23:16:54.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.062+0000 7fc0de963b80 10 client.16035 fetch_fsmap learned FSMap version 32 2023-09-09T23:16:54.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.062+0000 7fc0de963b80 1 -- 172.21.15.82:0/1781934359 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x556d03ccede0 con 0x556d03f50710 2023-09-09T23:16:54.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.062+0000 7fc0ab7fe700 1 -- 172.21.15.82:0/1781934359 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0bc0298e0 con 0x556d03f50710 2023-09-09T23:16:54.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.062+0000 7fc0ab7fe700 1 -- 172.21.15.82:0/1781934359 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fc0bc03c020 con 0x556d03f50710 2023-09-09T23:16:54.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.062+0000 7fc0de963b80 10 client.16035 fetch_fsmap finished waiting for FSMap version 32 2023-09-09T23:16:54.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.062+0000 7fc0de963b80 10 client.16035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:16:54.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.062+0000 7fc0de963b80 20 client.16035 populate_metadata read hostname 'smithi082' 2023-09-09T23:16:54.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.062+0000 7fc0de963b80 1 --2- 172.21.15.82:0/1781934359 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x556d03fbfb00 0x556d03fc1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:16:54.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.062+0000 7fc0de963b80 4 client.16035 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:16:54.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.062+0000 7fc0de963b80 1 -- 172.21.15.82:0/1781934359 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556d03e9b920 con 0x556d03fbfb00 2023-09-09T23:16:54.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.062+0000 7fc0cb7fe700 1 --2- 172.21.15.82:0/1781934359 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x556d03fbfb00 0x556d03fc1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:16:54.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.063+0000 7fc0ab7fe700 10 client.16035 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:16:54.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.063+0000 7fc0cb7fe700 1 --2- 172.21.15.82:0/1781934359 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x556d03fbfb00 0x556d03fc1f20 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69f1c33d56cc9d4 server_cookie=84a2cd2ca60f67f5 in_seq=0 out_seq=0 2023-09-09T23:16:54.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.063+0000 7fc0ab7fe700 1 -- 172.21.15.82:0/1781934359 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x556d03e9b920 con 0x556d03fbfb00 2023-09-09T23:16:54.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.063+0000 7fc0ab7fe700 10 client.16035 handle_command_reply: tid=0 2023-09-09T23:16:54.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.063+0000 7fc0de963b80 1 client.16035 shutdown 2023-09-09T23:16:54.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.063+0000 7fc0de963b80 1 -- 172.21.15.82:0/1781934359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d03f50710 msgr2=0x556d03f61880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:16:54.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.063+0000 7fc0de963b80 1 --2- 172.21.15.82:0/1781934359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d03f50710 0x556d03f61880 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009e90 tx=0x7fc0bc0096d0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.063+0000 7fc0de963b80 1 -- 172.21.15.82:0/1781934359 shutdown_connections 2023-09-09T23:16:54.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.063+0000 7fc0de963b80 1 --2- 172.21.15.82:0/1781934359 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d03f5ae40 0x556d03f600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.063+0000 7fc0de963b80 1 --2- 172.21.15.82:0/1781934359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d03f50710 0x556d03f61880 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.063+0000 7fc0de963b80 1 --2- 172.21.15.82:0/1781934359 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x556d03fbfb00 0x556d03fc1f20 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.063+0000 7fc0de963b80 1 --2- 172.21.15.82:0/1781934359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d03f51030 0x556d03f5fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.063+0000 7fc0ab7fe700 0 client.16035 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:16:54.085 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.063+0000 7fc0de963b80 1 -- 172.21.15.82:0/1781934359 >> 172.21.15.82:0/1781934359 conn(0x556d03ec5090 msgr2=0x556d03f52d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:16:54.085 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.063+0000 7fc0de963b80 1 -- 172.21.15.82:0/1781934359 shutdown_connections 2023-09-09T23:16:54.085 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.063+0000 7fc0de963b80 1 -- 172.21.15.82:0/1781934359 wait complete. 2023-09-09T23:16:54.085 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.063+0000 7fc0de963b80 20 client.16035 trim_cache size 0 max 16384 2023-09-09T23:16:54.085 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:16:54.086 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:16:54.086 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:16:54.086 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:16:54.086 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:16:54.086 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:16:54.087 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:16:54.087 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:16:54.087 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:16:54.087 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:16:54.088 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.064+0000 7fc0d279d700 1 -- 172.21.15.82:0/961076596 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fc0b40a0830 msgr2=0x7fc0b40a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:16:54.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.064+0000 7fc0d279d700 1 --2- 172.21.15.82:0/961076596 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fc0b40a0830 0x7fc0b40a2cf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009c60 tx=0x7fc0bc009260 comp rx=0 tx=0).stop 2023-09-09T23:16:54.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.064+0000 7fc0d279d700 1 -- 172.21.15.82:0/961076596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0cc100f40 msgr2=0x7fc0cc196bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:16:54.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.064+0000 7fc0d279d700 1 --2- 172.21.15.82:0/961076596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0cc100f40 0x7fc0cc196bd0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000d950 tx=0x7fc0c000dc60 comp rx=0 tx=0).stop 2023-09-09T23:16:54.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.064+0000 7fc0d279d700 1 -- 172.21.15.82:0/961076596 shutdown_connections 2023-09-09T23:16:54.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.064+0000 7fc0d279d700 1 --2- 172.21.15.82:0/961076596 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fc0b40a0830 0x7fc0b40a2cf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.090 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.064+0000 7fc0d279d700 1 --2- 172.21.15.82:0/961076596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0cc106640 0x7fc0cc199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.090 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.064+0000 7fc0d279d700 1 --2- 172.21.15.82:0/961076596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0cc100580 0x7fc0cc196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.090 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.064+0000 7fc0d279d700 1 --2- 172.21.15.82:0/961076596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0cc100f40 0x7fc0cc196bd0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:16:54.090 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.064+0000 7fc0d279d700 1 -- 172.21.15.82:0/961076596 >> 172.21.15.82:0/961076596 conn(0x7fc0cc0fc030 msgr2=0x7fc0cc104280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:16:54.090 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.064+0000 7fc0d279d700 1 -- 172.21.15.82:0/961076596 shutdown_connections 2023-09-09T23:16:54.090 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:16:54.064+0000 7fc0d279d700 1 -- 172.21.15.82:0/961076596 wait complete. 2023-09-09T23:16:54.091 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:16:55.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:55 smithi170 ceph-mon[102832]: pgmap v93: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.4 MiB/s wr, 46 op/s 2023-09-09T23:16:55.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:55 smithi082 ceph-mon[98694]: pgmap v93: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.4 MiB/s wr, 46 op/s 2023-09-09T23:16:55.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:55 smithi098 ceph-mon[103124]: pgmap v93: 97 pgs: 97 active+clean; 8.6 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.4 MiB/s wr, 46 op/s 2023-09-09T23:16:57.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:57 smithi170 ceph-mon[102832]: pgmap v94: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 13 MiB/s wr, 60 op/s 2023-09-09T23:16:57.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:57 smithi082 ceph-mon[98694]: pgmap v94: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 13 MiB/s wr, 60 op/s 2023-09-09T23:16:57.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:57 smithi098 ceph-mon[103124]: pgmap v94: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 13 MiB/s wr, 60 op/s 2023-09-09T23:16:59.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:16:59 smithi170 ceph-mon[102832]: pgmap v95: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 9.2 MiB/s wr, 34 op/s 2023-09-09T23:16:59.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:16:59 smithi082 ceph-mon[98694]: pgmap v95: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 9.2 MiB/s wr, 34 op/s 2023-09-09T23:16:59.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:16:59 smithi098 ceph-mon[103124]: pgmap v95: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 9.2 MiB/s wr, 34 op/s 2023-09-09T23:17:01.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:01 smithi170 ceph-mon[102832]: pgmap v96: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 9.3 MiB/s wr, 42 op/s 2023-09-09T23:17:01.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:01 smithi082 ceph-mon[98694]: pgmap v96: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 9.3 MiB/s wr, 42 op/s 2023-09-09T23:17:01.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:01 smithi098 ceph-mon[103124]: pgmap v96: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 9.3 MiB/s wr, 42 op/s 2023-09-09T23:17:02.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:17:02.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:17:02.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:17:02.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:17:02.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:17:02.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:17:02.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:17:02.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:17:02.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:17:03.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:03 smithi170 ceph-mon[102832]: pgmap v97: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 7.6 MiB/s wr, 23 op/s 2023-09-09T23:17:03.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:17:03.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:17:03.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:17:03.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:17:03.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:17:03.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:03 smithi082 ceph-mon[98694]: pgmap v97: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 7.6 MiB/s wr, 23 op/s 2023-09-09T23:17:03.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:17:03.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:17:03.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:17:03.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:17:03.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:17:03.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:03 smithi098 ceph-mon[103124]: pgmap v97: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 7.6 MiB/s wr, 23 op/s 2023-09-09T23:17:03.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:17:03.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:17:03.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:17:03.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:17:03.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:17:05.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:05 smithi170 ceph-mon[102832]: pgmap v98: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 7.3 MiB/s wr, 22 op/s 2023-09-09T23:17:05.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:05 smithi082 ceph-mon[98694]: pgmap v98: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 7.3 MiB/s wr, 22 op/s 2023-09-09T23:17:05.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:05 smithi098 ceph-mon[103124]: pgmap v98: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 7.3 MiB/s wr, 22 op/s 2023-09-09T23:17:07.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:07 smithi170 ceph-mon[102832]: pgmap v99: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 7.3 MiB/s wr, 42 op/s 2023-09-09T23:17:07.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:07 smithi082 ceph-mon[98694]: pgmap v99: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 7.3 MiB/s wr, 42 op/s 2023-09-09T23:17:07.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:07 smithi098 ceph-mon[103124]: pgmap v99: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 7.3 MiB/s wr, 42 op/s 2023-09-09T23:17:09.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:09 smithi170 ceph-mon[102832]: pgmap v100: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 81 KiB/s wr, 28 op/s 2023-09-09T23:17:09.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:09 smithi082 ceph-mon[98694]: pgmap v100: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 81 KiB/s wr, 28 op/s 2023-09-09T23:17:09.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:09 smithi098 ceph-mon[103124]: pgmap v100: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 81 KiB/s wr, 28 op/s 2023-09-09T23:17:11.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:11 smithi170 ceph-mon[102832]: pgmap v101: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 113 MiB/s rd, 439 KiB/s wr, 78 op/s 2023-09-09T23:17:11.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:11 smithi082 ceph-mon[98694]: pgmap v101: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 113 MiB/s rd, 439 KiB/s wr, 78 op/s 2023-09-09T23:17:11.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:11 smithi098 ceph-mon[103124]: pgmap v101: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 113 MiB/s rd, 439 KiB/s wr, 78 op/s 2023-09-09T23:17:13.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:13 smithi170 ceph-mon[102832]: pgmap v102: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 129 MiB/s rd, 762 KiB/s wr, 77 op/s 2023-09-09T23:17:13.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:17:13.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:13 smithi082 ceph-mon[98694]: pgmap v102: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 129 MiB/s rd, 762 KiB/s wr, 77 op/s 2023-09-09T23:17:13.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:17:13.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:13 smithi098 ceph-mon[103124]: pgmap v102: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 129 MiB/s rd, 762 KiB/s wr, 77 op/s 2023-09-09T23:17:13.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:17:15.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:15 smithi170 ceph-mon[102832]: pgmap v103: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 129 MiB/s rd, 762 KiB/s wr, 76 op/s 2023-09-09T23:17:15.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:15 smithi082 ceph-mon[98694]: pgmap v103: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 129 MiB/s rd, 762 KiB/s wr, 76 op/s 2023-09-09T23:17:15.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:15 smithi098 ceph-mon[103124]: pgmap v103: 97 pgs: 97 active+clean; 8.7 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 129 MiB/s rd, 762 KiB/s wr, 76 op/s 2023-09-09T23:17:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:17 smithi082 ceph-mon[98694]: pgmap v104: 97 pgs: 97 active+clean; 9.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 250 MiB/s rd, 60 MiB/s wr, 149 op/s 2023-09-09T23:17:17.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:17 smithi098 ceph-mon[103124]: pgmap v104: 97 pgs: 97 active+clean; 9.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 250 MiB/s rd, 60 MiB/s wr, 149 op/s 2023-09-09T23:17:17.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:17 smithi170 ceph-mon[102832]: pgmap v104: 97 pgs: 97 active+clean; 9.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 250 MiB/s rd, 60 MiB/s wr, 149 op/s 2023-09-09T23:17:18.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:17:18.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:18 smithi170 ceph-mon[102832]: pgmap v105: 97 pgs: 97 active+clean; 9.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 250 MiB/s rd, 60 MiB/s wr, 129 op/s 2023-09-09T23:17:18.954 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:17:18.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:18 smithi082 ceph-mon[98694]: pgmap v105: 97 pgs: 97 active+clean; 9.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 250 MiB/s rd, 60 MiB/s wr, 129 op/s 2023-09-09T23:17:19.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:17:19.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:18 smithi098 ceph-mon[103124]: pgmap v105: 97 pgs: 97 active+clean; 9.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 250 MiB/s rd, 60 MiB/s wr, 129 op/s 2023-09-09T23:17:21.454 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:21 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:17:21.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:21 smithi082 ceph-mon[98694]: pgmap v106: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 294 MiB/s rd, 280 MiB/s wr, 208 op/s 2023-09-09T23:17:21.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:21 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:17:21.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:21 smithi098 ceph-mon[103124]: pgmap v106: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 294 MiB/s rd, 280 MiB/s wr, 208 op/s 2023-09-09T23:17:21.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:21 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:17:21.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:21 smithi170 ceph-mon[102832]: pgmap v106: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 294 MiB/s rd, 280 MiB/s wr, 208 op/s 2023-09-09T23:17:23.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:23 smithi170 ceph-mon[102832]: pgmap v107: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 186 MiB/s rd, 307 MiB/s wr, 169 op/s 2023-09-09T23:17:23.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:23 smithi082 ceph-mon[98694]: pgmap v107: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 186 MiB/s rd, 307 MiB/s wr, 169 op/s 2023-09-09T23:17:23.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:23 smithi098 ceph-mon[103124]: pgmap v107: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 186 MiB/s rd, 307 MiB/s wr, 169 op/s 2023-09-09T23:17:24.092 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:17:24.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.233+0000 7ff916132700 1 -- 172.21.15.82:0/3262130299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9100ff4b0 msgr2=0x7ff9100ff930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:17:24.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.233+0000 7ff916132700 1 --2- 172.21.15.82:0/3262130299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9100ff4b0 0x7ff9100ff930 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff9040099c0 tx=0x7ff904009cd0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.233+0000 7ff916132700 1 -- 172.21.15.82:0/3262130299 shutdown_connections 2023-09-09T23:17:24.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.233+0000 7ff916132700 1 --2- 172.21.15.82:0/3262130299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9100ffe70 0x7ff91010c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.233+0000 7ff916132700 1 --2- 172.21.15.82:0/3262130299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff910105570 0x7ff910105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.233+0000 7ff916132700 1 --2- 172.21.15.82:0/3262130299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9100ff4b0 0x7ff9100ff930 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.233+0000 7ff916132700 1 -- 172.21.15.82:0/3262130299 >> 172.21.15.82:0/3262130299 conn(0x7ff9100fafa0 msgr2=0x7ff9100fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:17:24.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.233+0000 7ff916132700 1 -- 172.21.15.82:0/3262130299 shutdown_connections 2023-09-09T23:17:24.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.233+0000 7ff916132700 1 -- 172.21.15.82:0/3262130299 wait complete. 2023-09-09T23:17:24.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.233+0000 7ff916132700 1 Processor -- start 2023-09-09T23:17:24.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.234+0000 7ff916132700 1 -- start start 2023-09-09T23:17:24.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.234+0000 7ff916132700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9100ff4b0 0x7ff9101965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:24.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.234+0000 7ff916132700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9100ffe70 0x7ff910196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:24.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.234+0000 7ff916132700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff910105570 0x7ff91019af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:24.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.234+0000 7ff916132700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91019b5d0 con 0x7ff9100ffe70 2023-09-09T23:17:24.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.234+0000 7ff916132700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91019b740 con 0x7ff910105570 2023-09-09T23:17:24.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.234+0000 7ff916132700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91019b900 con 0x7ff9100ff4b0 2023-09-09T23:17:24.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.234+0000 7ff90effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9100ffe70 0x7ff910196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:24.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.234+0000 7ff90effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9100ffe70 0x7ff910196b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57700/0 (socket says 172.21.15.82:57700) 2023-09-09T23:17:24.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.234+0000 7ff90effd700 1 -- 172.21.15.82:0/2901806860 learned_addr learned my addr 172.21.15.82:0/2901806860 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:17:24.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.234+0000 7ff90f7fe700 1 --2- 172.21.15.82:0/2901806860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9100ff4b0 0x7ff9101965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:24.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.234+0000 7ff90ffff700 1 --2- 172.21.15.82:0/2901806860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff910105570 0x7ff91019af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:24.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.234+0000 7ff90effd700 1 -- 172.21.15.82:0/2901806860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9100ff4b0 msgr2=0x7ff9101965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:17:24.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.234+0000 7ff90effd700 1 --2- 172.21.15.82:0/2901806860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9100ff4b0 0x7ff9101965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.234+0000 7ff90effd700 1 -- 172.21.15.82:0/2901806860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff910105570 msgr2=0x7ff91019af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:17:24.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.234+0000 7ff90effd700 1 --2- 172.21.15.82:0/2901806860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff910105570 0x7ff91019af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.234+0000 7ff90effd700 1 -- 172.21.15.82:0/2901806860 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9040095f0 con 0x7ff9100ffe70 2023-09-09T23:17:24.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.234+0000 7ff90effd700 1 --2- 172.21.15.82:0/2901806860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9100ffe70 0x7ff910196b30 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff90400b580 tx=0x7ff90400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:17:24.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.235+0000 7ff90cff9700 1 -- 172.21.15.82:0/2901806860 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90401d070 con 0x7ff9100ffe70 2023-09-09T23:17:24.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.235+0000 7ff90cff9700 1 -- 172.21.15.82:0/2901806860 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff904022470 con 0x7ff9100ffe70 2023-09-09T23:17:24.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.235+0000 7ff916132700 1 -- 172.21.15.82:0/2901806860 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff91019bb80 con 0x7ff9100ffe70 2023-09-09T23:17:24.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.235+0000 7ff90cff9700 1 -- 172.21.15.82:0/2901806860 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff904017650 con 0x7ff9100ffe70 2023-09-09T23:17:24.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.235+0000 7ff916132700 1 -- 172.21.15.82:0/2901806860 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9101a3420 con 0x7ff9100ffe70 2023-09-09T23:17:24.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.237+0000 7ff90cff9700 1 -- 172.21.15.82:0/2901806860 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) v1 ==== 135390+0+0 (secure 0 0 0) 0x7ff9040225e0 con 0x7ff9100ffe70 2023-09-09T23:17:24.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.238+0000 7ff90cff9700 1 --2- 172.21.15.82:0/2901806860 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7ff8f80a07b0 0x7ff8f80a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:24.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.238+0000 7ff90f7fe700 1 --2- 172.21.15.82:0/2901806860 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7ff8f80a07b0 0x7ff8f80a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:24.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.238+0000 7ff90cff9700 1 -- 172.21.15.82:0/2901806860 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7ff9040cd0b0 con 0x7ff9100ffe70 2023-09-09T23:17:24.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.238+0000 7ff90f7fe700 1 --2- 172.21.15.82:0/2901806860 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7ff8f80a07b0 0x7ff8f80a2c70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff900009c60 tx=0x7ff900009260 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:17:24.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.239+0000 7ff9222f8b80 1 Processor -- start 2023-09-09T23:17:24.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.239+0000 7ff9222f8b80 1 -- start start 2023-09-09T23:17:24.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.239+0000 7ff9222f8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db07b19710 0x55db07b19af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:24.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.239+0000 7ff9222f8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db07b1a030 0x55db07b23900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:24.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.239+0000 7ff9222f8b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db07b23e40 0x55db07b26230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:24.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.239+0000 7ff9222f8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db07a72730 con 0x55db07b19710 2023-09-09T23:17:24.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.239+0000 7ff9222f8b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db07a70050 con 0x55db07b23e40 2023-09-09T23:17:24.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.239+0000 7ff9222f8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db07a6b940 con 0x55db07b1a030 2023-09-09T23:17:24.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.239+0000 7ff90f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db07b19710 0x55db07b19af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:24.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.239+0000 7ff90f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db07b19710 0x55db07b19af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57706/0 (socket says 172.21.15.82:57706) 2023-09-09T23:17:24.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.239+0000 7ff90f7fe700 1 -- 172.21.15.82:0/2198384375 learned_addr learned my addr 172.21.15.82:0/2198384375 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:17:24.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.239+0000 7ff90effd700 1 --2- 172.21.15.82:0/2198384375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db07b1a030 0x55db07b23900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:24.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.239+0000 7ff90ffff700 1 --2- 172.21.15.82:0/2198384375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db07b23e40 0x55db07b26230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:24.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.239+0000 7ff90f7fe700 1 -- 172.21.15.82:0/2198384375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db07b1a030 msgr2=0x55db07b23900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:17:24.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.239+0000 7ff90f7fe700 1 --2- 172.21.15.82:0/2198384375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db07b1a030 0x55db07b23900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.239+0000 7ff90f7fe700 1 -- 172.21.15.82:0/2198384375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db07b23e40 msgr2=0x55db07b26230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:17:24.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.239+0000 7ff90f7fe700 1 --2- 172.21.15.82:0/2198384375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db07b23e40 0x55db07b26230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.239+0000 7ff90f7fe700 1 -- 172.21.15.82:0/2198384375 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db07897de0 con 0x55db07b19710 2023-09-09T23:17:24.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.239+0000 7ff90f7fe700 1 --2- 172.21.15.82:0/2198384375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db07b19710 0x55db07b19af0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff90000e4e0 tx=0x7ff900014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ecfb5693209dca server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:17:24.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.239+0000 7ff8f5ffb700 1 -- 172.21.15.82:0/2198384375 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90000ee00 con 0x55db07b19710 2023-09-09T23:17:24.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.240+0000 7ff8f5ffb700 1 -- 172.21.15.82:0/2198384375 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff900021070 con 0x55db07b19710 2023-09-09T23:17:24.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.240+0000 7ff8f5ffb700 1 -- 172.21.15.82:0/2198384375 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90001d7b0 con 0x55db07b19710 2023-09-09T23:17:24.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.240+0000 7ff9222f8b80 1 -- 172.21.15.82:0/2198384375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db07b19710 msgr2=0x55db07b19af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:17:24.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.240+0000 7ff9222f8b80 1 --2- 172.21.15.82:0/2198384375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db07b19710 0x55db07b19af0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff90000e4e0 tx=0x7ff900014000 comp rx=0 tx=0).stop 2023-09-09T23:17:24.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.240+0000 7ff9222f8b80 1 -- 172.21.15.82:0/2198384375 shutdown_connections 2023-09-09T23:17:24.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.240+0000 7ff9222f8b80 1 --2- 172.21.15.82:0/2198384375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db07b23e40 0x55db07b26230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.240+0000 7ff9222f8b80 1 --2- 172.21.15.82:0/2198384375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db07b19710 0x55db07b19af0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.240+0000 7ff9222f8b80 1 --2- 172.21.15.82:0/2198384375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db07b1a030 0x55db07b23900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.240+0000 7ff9222f8b80 1 -- 172.21.15.82:0/2198384375 >> 172.21.15.82:0/2198384375 conn(0x55db07a8e090 msgr2=0x55db07b14bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:17:24.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.240+0000 7ff9222f8b80 1 -- 172.21.15.82:0/2198384375 shutdown_connections 2023-09-09T23:17:24.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.240+0000 7ff9222f8b80 1 -- 172.21.15.82:0/2198384375 wait complete. 2023-09-09T23:17:24.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.241+0000 7ff9222f8b80 1 -- start start 2023-09-09T23:17:24.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.241+0000 7ff9222f8b80 1 Processor -- start 2023-09-09T23:17:24.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.241+0000 7ff9222f8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db07b19710 0x55db07b23770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:24.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.241+0000 7ff9222f8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db07b1a030 0x55db07b20a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:24.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.241+0000 7ff9222f8b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db07b23e40 0x55db07b21020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:24.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.241+0000 7ff9222f8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db07a09b70 con 0x55db07b19710 2023-09-09T23:17:24.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.241+0000 7ff9222f8b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db07a09870 con 0x55db07b23e40 2023-09-09T23:17:24.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.241+0000 7ff9222f8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db078ed920 con 0x55db07b1a030 2023-09-09T23:17:24.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.241+0000 7ff9222f8b80 10 client.0 _ll_register_callbacks cb 0x55db07953a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:17:24.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.241+0000 7ff90f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db07b19710 0x55db07b23770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:24.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.241+0000 7ff90f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db07b19710 0x55db07b23770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57712/0 (socket says 172.21.15.82:57712) 2023-09-09T23:17:24.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.241+0000 7ff90ffff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db07b23e40 0x55db07b21020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:24.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.241+0000 7ff90f7fe700 1 -- 172.21.15.82:0/911281536 learned_addr learned my addr 172.21.15.82:0/911281536 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:17:24.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.241+0000 7ff90effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db07b1a030 0x55db07b20a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:24.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.241+0000 7ff90f7fe700 1 -- 172.21.15.82:0/911281536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db07b1a030 msgr2=0x55db07b20a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:17:24.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.242+0000 7ff90f7fe700 1 --2- 172.21.15.82:0/911281536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db07b1a030 0x55db07b20a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.242+0000 7ff90f7fe700 1 -- 172.21.15.82:0/911281536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db07b23e40 msgr2=0x55db07b21020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:17:24.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.242+0000 7ff90f7fe700 1 --2- 172.21.15.82:0/911281536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db07b23e40 0x55db07b21020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.242+0000 7ff90f7fe700 1 -- 172.21.15.82:0/911281536 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff900017070 con 0x55db07b19710 2023-09-09T23:17:24.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.242+0000 7ff90f7fe700 1 --2- 172.21.15.82:0/911281536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db07b19710 0x55db07b23770 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff90000e4b0 tx=0x7ff90000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad909c4f2bf5e62 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:17:24.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.242+0000 7ff8eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:17:24.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.242+0000 7ff8eeffd700 1 -- 172.21.15.82:0/911281536 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff900005250 con 0x55db07b19710 2023-09-09T23:17:24.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.242+0000 7ff8eeffd700 1 -- 172.21.15.82:0/911281536 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7ff9000053b0 con 0x55db07b19710 2023-09-09T23:17:24.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.242+0000 7ff9222f8b80 1 -- 172.21.15.82:0/911281536 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db079c34f0 con 0x55db07b19710 2023-09-09T23:17:24.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.242+0000 7ff8eeffd700 1 -- 172.21.15.82:0/911281536 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff900005570 con 0x55db07b19710 2023-09-09T23:17:24.258 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.242+0000 7ff9222f8b80 10 client.16047 fetch_fsmap learned FSMap version 35 2023-09-09T23:17:24.258 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.242+0000 7ff9222f8b80 1 -- 172.21.15.82:0/911281536 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55db07880c60 con 0x55db07b19710 2023-09-09T23:17:24.258 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.242+0000 7ff8eeffd700 1 -- 172.21.15.82:0/911281536 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55db079c34f0 con 0x55db07b19710 2023-09-09T23:17:24.258 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.242+0000 7ff8eeffd700 1 -- 172.21.15.82:0/911281536 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 35) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff90001c620 con 0x55db07b19710 2023-09-09T23:17:24.258 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.242+0000 7ff9222f8b80 10 client.16047 fetch_fsmap finished waiting for FSMap version 35 2023-09-09T23:17:24.259 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.242+0000 7ff9222f8b80 10 client.16047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:17:24.259 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.242+0000 7ff9222f8b80 20 client.16047 populate_metadata read hostname 'smithi082' 2023-09-09T23:17:24.259 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.242+0000 7ff9222f8b80 1 --2- 172.21.15.82:0/911281536 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55db07b33760 0x55db07b35b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:24.260 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.242+0000 7ff9222f8b80 4 client.16047 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:17:24.260 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.242+0000 7ff9222f8b80 1 -- 172.21.15.82:0/911281536 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db07a808a0 con 0x55db07b33760 2023-09-09T23:17:24.260 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.243+0000 7ff90effd700 1 --2- 172.21.15.82:0/911281536 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55db07b33760 0x55db07b35b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:24.260 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.243+0000 7ff8eeffd700 10 client.16047 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:17:24.260 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.243+0000 7ff90effd700 1 --2- 172.21.15.82:0/911281536 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55db07b33760 0x55db07b35b80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c29806e60bd2b0 server_cookie=61649b6ea2a02596 in_seq=0 out_seq=0 2023-09-09T23:17:24.261 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.244+0000 7ff8eeffd700 1 -- 172.21.15.82:0/911281536 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55db07a808a0 con 0x55db07b33760 2023-09-09T23:17:24.261 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.244+0000 7ff8eeffd700 10 client.16047 handle_command_reply: tid=0 2023-09-09T23:17:24.261 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.244+0000 7ff9222f8b80 1 client.16047 shutdown 2023-09-09T23:17:24.261 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.244+0000 7ff9222f8b80 1 -- 172.21.15.82:0/911281536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db07b19710 msgr2=0x55db07b23770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:17:24.261 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.244+0000 7ff9222f8b80 1 --2- 172.21.15.82:0/911281536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db07b19710 0x55db07b23770 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff90000e4b0 tx=0x7ff90000ecb0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.244+0000 7ff9222f8b80 1 -- 172.21.15.82:0/911281536 shutdown_connections 2023-09-09T23:17:24.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.244+0000 7ff9222f8b80 1 --2- 172.21.15.82:0/911281536 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55db07b33760 0x55db07b35b80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.244+0000 7ff9222f8b80 1 --2- 172.21.15.82:0/911281536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db07b23e40 0x55db07b21020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.244+0000 7ff8eeffd700 0 client.16047 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:17:24.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.244+0000 7ff9222f8b80 1 --2- 172.21.15.82:0/911281536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db07b19710 0x55db07b23770 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.244+0000 7ff9222f8b80 1 --2- 172.21.15.82:0/911281536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db07b1a030 0x55db07b20a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.244+0000 7ff9222f8b80 1 -- 172.21.15.82:0/911281536 >> 172.21.15.82:0/911281536 conn(0x55db07a8e090 msgr2=0x55db07b147e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:17:24.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.244+0000 7ff9222f8b80 1 -- 172.21.15.82:0/911281536 shutdown_connections 2023-09-09T23:17:24.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.244+0000 7ff9222f8b80 1 -- 172.21.15.82:0/911281536 wait complete. 2023-09-09T23:17:24.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.244+0000 7ff9222f8b80 20 client.16047 trim_cache size 0 max 16384 2023-09-09T23:17:24.268 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.261+0000 7ff9222f8b80 1 Processor -- start 2023-09-09T23:17:24.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.261+0000 7ff9222f8b80 1 -- start start 2023-09-09T23:17:24.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.261+0000 7ff9222f8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db07b19710 0x55db07b23310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:24.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.261+0000 7ff9222f8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db07b1a030 0x55db07b20220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:24.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.261+0000 7ff9222f8b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db07b23e40 0x55db07b20760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:24.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.261+0000 7ff9222f8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db07b30220 con 0x55db07b19710 2023-09-09T23:17:24.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.261+0000 7ff9222f8b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db07b1efc0 con 0x55db07b23e40 2023-09-09T23:17:24.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.261+0000 7ff9222f8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db07b156e0 con 0x55db07b1a030 2023-09-09T23:17:24.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.261+0000 7ff90f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db07b19710 0x55db07b23310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:24.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.261+0000 7ff90f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db07b19710 0x55db07b23310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57720/0 (socket says 172.21.15.82:57720) 2023-09-09T23:17:24.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.261+0000 7ff90f7fe700 1 -- 172.21.15.82:0/1711105497 learned_addr learned my addr 172.21.15.82:0/1711105497 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:17:24.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.261+0000 7ff90ffff700 1 --2- 172.21.15.82:0/1711105497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db07b23e40 0x55db07b20760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:24.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.261+0000 7ff90effd700 1 --2- 172.21.15.82:0/1711105497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db07b1a030 0x55db07b20220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:24.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.261+0000 7ff90f7fe700 1 -- 172.21.15.82:0/1711105497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db07b1a030 msgr2=0x55db07b20220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:17:24.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.261+0000 7ff90f7fe700 1 --2- 172.21.15.82:0/1711105497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db07b1a030 0x55db07b20220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.261+0000 7ff90f7fe700 1 -- 172.21.15.82:0/1711105497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db07b23e40 msgr2=0x55db07b20760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:17:24.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.261+0000 7ff90f7fe700 1 --2- 172.21.15.82:0/1711105497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db07b23e40 0x55db07b20760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.261+0000 7ff90f7fe700 1 -- 172.21.15.82:0/1711105497 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db078be250 con 0x55db07b19710 2023-09-09T23:17:24.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.261+0000 7ff90f7fe700 1 --2- 172.21.15.82:0/1711105497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db07b19710 0x55db07b23310 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff90000e480 tx=0x7ff90000eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28036f5c52faf0ef server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:17:24.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.261+0000 7ff8f67fc700 1 -- 172.21.15.82:0/1711105497 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90001cba0 con 0x55db07b19710 2023-09-09T23:17:24.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.261+0000 7ff8f67fc700 1 -- 172.21.15.82:0/1711105497 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff90001cd00 con 0x55db07b19710 2023-09-09T23:17:24.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.261+0000 7ff8f67fc700 1 -- 172.21.15.82:0/1711105497 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9000038e0 con 0x55db07b19710 2023-09-09T23:17:24.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.262+0000 7ff9222f8b80 1 -- 172.21.15.82:0/1711105497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db07b19710 msgr2=0x55db07b23310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:17:24.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.262+0000 7ff9222f8b80 1 --2- 172.21.15.82:0/1711105497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db07b19710 0x55db07b23310 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff90000e480 tx=0x7ff90000eb70 comp rx=0 tx=0).stop 2023-09-09T23:17:24.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.262+0000 7ff9222f8b80 1 -- 172.21.15.82:0/1711105497 shutdown_connections 2023-09-09T23:17:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.262+0000 7ff9222f8b80 1 --2- 172.21.15.82:0/1711105497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db07b23e40 0x55db07b20760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.262+0000 7ff9222f8b80 1 --2- 172.21.15.82:0/1711105497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db07b19710 0x55db07b23310 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.262+0000 7ff9222f8b80 1 --2- 172.21.15.82:0/1711105497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db07b1a030 0x55db07b20220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.262+0000 7ff9222f8b80 1 -- 172.21.15.82:0/1711105497 >> 172.21.15.82:0/1711105497 conn(0x55db07a8e090 msgr2=0x55db07b1bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:17:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.262+0000 7ff9222f8b80 1 -- 172.21.15.82:0/1711105497 shutdown_connections 2023-09-09T23:17:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.262+0000 7ff9222f8b80 1 -- 172.21.15.82:0/1711105497 wait complete. 2023-09-09T23:17:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.262+0000 7ff9222f8b80 1 -- start start 2023-09-09T23:17:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.263+0000 7ff9222f8b80 1 Processor -- start 2023-09-09T23:17:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.263+0000 7ff9222f8b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db07b19710 0x55db07b2a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.263+0000 7ff9222f8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db07b1a030 0x55db07b28a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.263+0000 7ff9222f8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db07b23e40 0x55db07b28f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:24.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.263+0000 7ff9222f8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db07b294d0 con 0x55db07b23e40 2023-09-09T23:17:24.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.263+0000 7ff9222f8b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db07b29640 con 0x55db07b19710 2023-09-09T23:17:24.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.263+0000 7ff9222f8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db07b29800 con 0x55db07b1a030 2023-09-09T23:17:24.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.263+0000 7ff9222f8b80 10 client.0 _ll_register_callbacks cb 0x7ff908009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:17:24.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.263+0000 7ff90ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db07b23e40 0x55db07b28f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:24.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.263+0000 7ff90ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db07b23e40 0x55db07b28f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57730/0 (socket says 172.21.15.82:57730) 2023-09-09T23:17:24.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.263+0000 7ff90ffff700 1 -- 172.21.15.82:0/212011370 learned_addr learned my addr 172.21.15.82:0/212011370 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:17:24.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.263+0000 7ff90effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db07b1a030 0x55db07b28a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:24.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.263+0000 7ff90f7fe700 1 --2- 172.21.15.82:0/212011370 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db07b19710 0x55db07b2a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:24.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.263+0000 7ff90ffff700 1 -- 172.21.15.82:0/212011370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db07b1a030 msgr2=0x55db07b28a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:17:24.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.263+0000 7ff90ffff700 1 --2- 172.21.15.82:0/212011370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db07b1a030 0x55db07b28a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.263+0000 7ff90ffff700 1 -- 172.21.15.82:0/212011370 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db07b19710 msgr2=0x55db07b2a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:17:24.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.263+0000 7ff90ffff700 1 --2- 172.21.15.82:0/212011370 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db07b19710 0x55db07b2a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.263+0000 7ff90ffff700 1 -- 172.21.15.82:0/212011370 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55db07880c60 con 0x55db07b23e40 2023-09-09T23:17:24.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.263+0000 7ff90ffff700 1 --2- 172.21.15.82:0/212011370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db07b23e40 0x55db07b28f90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff908007bc0 tx=0x7ff908007ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170621d64b16e4c6 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:17:24.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.263+0000 7ff8eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:17:24.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.263+0000 7ff8eeffd700 1 -- 172.21.15.82:0/212011370 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90801c070 con 0x55db07b23e40 2023-09-09T23:17:24.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.264+0000 7ff8eeffd700 1 -- 172.21.15.82:0/212011370 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7ff908011b90 con 0x55db07b23e40 2023-09-09T23:17:24.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.264+0000 7ff9222f8b80 1 -- 172.21.15.82:0/212011370 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff900021070 con 0x55db07b23e40 2023-09-09T23:17:24.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.264+0000 7ff8eeffd700 1 -- 172.21.15.82:0/212011370 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff90800e040 con 0x55db07b23e40 2023-09-09T23:17:24.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.264+0000 7ff9222f8b80 10 client.16053 fetch_fsmap learned FSMap version 35 2023-09-09T23:17:24.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.264+0000 7ff9222f8b80 1 -- 172.21.15.82:0/212011370 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55db078962b0 con 0x55db07b23e40 2023-09-09T23:17:24.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.264+0000 7ff8eeffd700 1 -- 172.21.15.82:0/212011370 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff900021070 con 0x55db07b23e40 2023-09-09T23:17:24.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.264+0000 7ff8eeffd700 1 -- 172.21.15.82:0/212011370 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 35) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff9080044c0 con 0x55db07b23e40 2023-09-09T23:17:24.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.264+0000 7ff9222f8b80 10 client.16053 fetch_fsmap finished waiting for FSMap version 35 2023-09-09T23:17:24.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.264+0000 7ff9222f8b80 10 client.16053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:17:24.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.264+0000 7ff9222f8b80 20 client.16053 populate_metadata read hostname 'smithi082' 2023-09-09T23:17:24.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.264+0000 7ff9222f8b80 1 --2- 172.21.15.82:0/212011370 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55db07b33760 0x55db07b8aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:24.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.264+0000 7ff9222f8b80 4 client.16053 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:17:24.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.264+0000 7ff9222f8b80 1 -- 172.21.15.82:0/212011370 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55db07a64920 con 0x55db07b33760 2023-09-09T23:17:24.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.264+0000 7ff90f7fe700 1 --2- 172.21.15.82:0/212011370 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55db07b33760 0x55db07b8aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:24.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.265+0000 7ff8eeffd700 10 client.16053 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:17:24.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.265+0000 7ff90f7fe700 1 --2- 172.21.15.82:0/212011370 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55db07b33760 0x55db07b8aa20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8075178d8ce886b6 server_cookie=2e15e4b7865eb61d in_seq=0 out_seq=0 2023-09-09T23:17:24.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.265+0000 7ff8eeffd700 1 -- 172.21.15.82:0/212011370 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55db07a64920 con 0x55db07b33760 2023-09-09T23:17:24.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.265+0000 7ff8eeffd700 10 client.16053 handle_command_reply: tid=0 2023-09-09T23:17:24.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.265+0000 7ff9222f8b80 1 client.16053 shutdown 2023-09-09T23:17:24.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.265+0000 7ff9222f8b80 1 -- 172.21.15.82:0/212011370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db07b23e40 msgr2=0x55db07b28f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:17:24.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.265+0000 7ff9222f8b80 1 --2- 172.21.15.82:0/212011370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db07b23e40 0x55db07b28f90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff908007bc0 tx=0x7ff908007ed0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.265+0000 7ff9222f8b80 1 -- 172.21.15.82:0/212011370 shutdown_connections 2023-09-09T23:17:24.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.265+0000 7ff9222f8b80 1 --2- 172.21.15.82:0/212011370 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55db07b33760 0x55db07b8aa20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.265+0000 7ff9222f8b80 1 --2- 172.21.15.82:0/212011370 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db07b19710 0x55db07b2a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.265+0000 7ff9222f8b80 1 --2- 172.21.15.82:0/212011370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db07b23e40 0x55db07b28f90 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.265+0000 7ff9222f8b80 1 --2- 172.21.15.82:0/212011370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db07b1a030 0x55db07b28a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.265+0000 7ff9222f8b80 1 -- 172.21.15.82:0/212011370 >> 172.21.15.82:0/212011370 conn(0x55db07a8e090 msgr2=0x55db07b18700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:17:24.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.265+0000 7ff8eeffd700 0 client.16053 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:17:24.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.265+0000 7ff9222f8b80 1 -- 172.21.15.82:0/212011370 shutdown_connections 2023-09-09T23:17:24.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.265+0000 7ff9222f8b80 1 -- 172.21.15.82:0/212011370 wait complete. 2023-09-09T23:17:24.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.265+0000 7ff9222f8b80 20 client.16053 trim_cache size 0 max 16384 2023-09-09T23:17:24.285 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:17:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:17:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:17:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:17:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:17:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:17:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:17:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:17:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:17:24.287 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:17:24.288 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.266+0000 7ff916132700 1 -- 172.21.15.82:0/2901806860 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7ff8f80a07b0 msgr2=0x7ff8f80a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:17:24.288 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.266+0000 7ff916132700 1 --2- 172.21.15.82:0/2901806860 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7ff8f80a07b0 0x7ff8f80a2c70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff900009c60 tx=0x7ff900009260 comp rx=0 tx=0).stop 2023-09-09T23:17:24.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.266+0000 7ff916132700 1 -- 172.21.15.82:0/2901806860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9100ffe70 msgr2=0x7ff910196b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:17:24.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.266+0000 7ff916132700 1 --2- 172.21.15.82:0/2901806860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9100ffe70 0x7ff910196b30 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff90400b580 tx=0x7ff90400bfd0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.266+0000 7ff916132700 1 -- 172.21.15.82:0/2901806860 shutdown_connections 2023-09-09T23:17:24.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.266+0000 7ff916132700 1 --2- 172.21.15.82:0/2901806860 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7ff8f80a07b0 0x7ff8f80a2c70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.266+0000 7ff916132700 1 --2- 172.21.15.82:0/2901806860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff910105570 0x7ff91019af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.266+0000 7ff916132700 1 --2- 172.21.15.82:0/2901806860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9100ff4b0 0x7ff9101965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.266+0000 7ff916132700 1 --2- 172.21.15.82:0/2901806860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9100ffe70 0x7ff910196b30 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:24.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.266+0000 7ff916132700 1 -- 172.21.15.82:0/2901806860 >> 172.21.15.82:0/2901806860 conn(0x7ff9100fafa0 msgr2=0x7ff9100fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:17:24.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.266+0000 7ff916132700 1 -- 172.21.15.82:0/2901806860 shutdown_connections 2023-09-09T23:17:24.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:24.266+0000 7ff916132700 1 -- 172.21.15.82:0/2901806860 wait complete. 2023-09-09T23:17:24.291 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:17:25.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:25 smithi170 ceph-mon[102832]: pgmap v108: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 169 MiB/s rd, 307 MiB/s wr, 161 op/s 2023-09-09T23:17:25.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:25 smithi082 ceph-mon[98694]: pgmap v108: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 169 MiB/s rd, 307 MiB/s wr, 161 op/s 2023-09-09T23:17:25.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:25 smithi098 ceph-mon[103124]: pgmap v108: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 169 MiB/s rd, 307 MiB/s wr, 161 op/s 2023-09-09T23:17:27.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:27 smithi170 ceph-mon[102832]: pgmap v109: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 174 MiB/s rd, 307 MiB/s wr, 171 op/s 2023-09-09T23:17:27.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:27 smithi082 ceph-mon[98694]: pgmap v109: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 174 MiB/s rd, 307 MiB/s wr, 171 op/s 2023-09-09T23:17:27.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:27 smithi098 ceph-mon[103124]: pgmap v109: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 174 MiB/s rd, 307 MiB/s wr, 171 op/s 2023-09-09T23:17:29.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:29 smithi170 ceph-mon[102832]: pgmap v110: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 53 MiB/s rd, 248 MiB/s wr, 98 op/s 2023-09-09T23:17:29.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:29 smithi082 ceph-mon[98694]: pgmap v110: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 53 MiB/s rd, 248 MiB/s wr, 98 op/s 2023-09-09T23:17:29.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:29 smithi098 ceph-mon[103124]: pgmap v110: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 53 MiB/s rd, 248 MiB/s wr, 98 op/s 2023-09-09T23:17:31.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:31 smithi170 ceph-mon[102832]: pgmap v111: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 116 MiB/s rd, 250 MiB/s wr, 127 op/s 2023-09-09T23:17:31.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:31 smithi082 ceph-mon[98694]: pgmap v111: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 116 MiB/s rd, 250 MiB/s wr, 127 op/s 2023-09-09T23:17:31.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:31 smithi098 ceph-mon[103124]: pgmap v111: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 116 MiB/s rd, 250 MiB/s wr, 127 op/s 2023-09-09T23:17:33.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:33 smithi170 ceph-mon[102832]: pgmap v112: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 133 MiB/s rd, 44 MiB/s wr, 77 op/s 2023-09-09T23:17:33.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:33 smithi082 ceph-mon[98694]: pgmap v112: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 133 MiB/s rd, 44 MiB/s wr, 77 op/s 2023-09-09T23:17:33.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:33 smithi098 ceph-mon[103124]: pgmap v112: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 133 MiB/s rd, 44 MiB/s wr, 77 op/s 2023-09-09T23:17:35.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:35 smithi170 ceph-mon[102832]: pgmap v113: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 128 MiB/s rd, 16 MiB/s wr, 67 op/s 2023-09-09T23:17:35.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:35 smithi082 ceph-mon[98694]: pgmap v113: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 128 MiB/s rd, 16 MiB/s wr, 67 op/s 2023-09-09T23:17:35.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:35 smithi098 ceph-mon[103124]: pgmap v113: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 128 MiB/s rd, 16 MiB/s wr, 67 op/s 2023-09-09T23:17:36.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:17:36.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:17:36.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:17:37.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:37 smithi170 ceph-mon[102832]: pgmap v114: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 128 MiB/s rd, 56 MiB/s wr, 84 op/s 2023-09-09T23:17:37.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:37 smithi082 ceph-mon[98694]: pgmap v114: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 128 MiB/s rd, 56 MiB/s wr, 84 op/s 2023-09-09T23:17:37.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:37 smithi098 ceph-mon[103124]: pgmap v114: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 128 MiB/s rd, 56 MiB/s wr, 84 op/s 2023-09-09T23:17:39.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:39 smithi170 ceph-mon[102832]: pgmap v115: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 123 MiB/s rd, 56 MiB/s wr, 75 op/s 2023-09-09T23:17:39.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:39 smithi082 ceph-mon[98694]: pgmap v115: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 123 MiB/s rd, 56 MiB/s wr, 75 op/s 2023-09-09T23:17:39.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:39 smithi098 ceph-mon[103124]: pgmap v115: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 123 MiB/s rd, 56 MiB/s wr, 75 op/s 2023-09-09T23:17:41.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:41 smithi170 ceph-mon[102832]: pgmap v116: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 123 MiB/s rd, 56 MiB/s wr, 190 op/s 2023-09-09T23:17:41.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:41 smithi082 ceph-mon[98694]: pgmap v116: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 123 MiB/s rd, 56 MiB/s wr, 190 op/s 2023-09-09T23:17:41.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:41 smithi098 ceph-mon[103124]: pgmap v116: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 123 MiB/s rd, 56 MiB/s wr, 190 op/s 2023-09-09T23:17:43.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:43 smithi170 ceph-mon[102832]: pgmap v117: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s rd, 55 MiB/s wr, 261 op/s 2023-09-09T23:17:43.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:43 smithi082 ceph-mon[98694]: pgmap v117: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s rd, 55 MiB/s wr, 261 op/s 2023-09-09T23:17:43.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:43 smithi098 ceph-mon[103124]: pgmap v117: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 60 MiB/s rd, 55 MiB/s wr, 261 op/s 2023-09-09T23:17:45.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:45 smithi170 ceph-mon[102832]: pgmap v118: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 41 MiB/s wr, 232 op/s 2023-09-09T23:17:45.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:45 smithi082 ceph-mon[98694]: pgmap v118: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 41 MiB/s wr, 232 op/s 2023-09-09T23:17:45.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:45 smithi098 ceph-mon[103124]: pgmap v118: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 41 MiB/s wr, 232 op/s 2023-09-09T23:17:47.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:47 smithi082 ceph-mon[98694]: pgmap v119: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 42 MiB/s wr, 469 op/s 2023-09-09T23:17:47.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:47 smithi098 ceph-mon[103124]: pgmap v119: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 42 MiB/s wr, 469 op/s 2023-09-09T23:17:47.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:47 smithi170 ceph-mon[102832]: pgmap v119: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 42 MiB/s wr, 469 op/s 2023-09-09T23:17:49.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:49 smithi082 ceph-mon[98694]: pgmap v120: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.2 MiB/s wr, 452 op/s 2023-09-09T23:17:49.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:49 smithi098 ceph-mon[103124]: pgmap v120: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.2 MiB/s wr, 452 op/s 2023-09-09T23:17:49.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:49 smithi170 ceph-mon[102832]: pgmap v120: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.2 MiB/s wr, 452 op/s 2023-09-09T23:17:51.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:51 smithi082 ceph-mon[98694]: pgmap v121: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 25 MiB/s wr, 650 op/s 2023-09-09T23:17:51.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:51 smithi098 ceph-mon[103124]: pgmap v121: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 25 MiB/s wr, 650 op/s 2023-09-09T23:17:51.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:51 smithi170 ceph-mon[102832]: pgmap v121: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 25 MiB/s wr, 650 op/s 2023-09-09T23:17:53.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:53 smithi098 ceph-mon[103124]: pgmap v122: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 42 MiB/s wr, 699 op/s 2023-09-09T23:17:53.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:53 smithi170 ceph-mon[102832]: pgmap v122: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 42 MiB/s wr, 699 op/s 2023-09-09T23:17:53.954 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:53 smithi082 ceph-mon[98694]: pgmap v122: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 42 MiB/s wr, 699 op/s 2023-09-09T23:17:54.291 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:17:54.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.425+0000 7f2062d8f700 1 -- 172.21.15.82:0/1207579212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f205c103160 msgr2=0x7f205c10f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:17:54.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.425+0000 7f2062d8f700 1 --2- 172.21.15.82:0/1207579212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f205c103160 0x7f205c10f670 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f205400b0a0 tx=0x7f205400b3b0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.425+0000 7f2062d8f700 1 -- 172.21.15.82:0/1207579212 shutdown_connections 2023-09-09T23:17:54.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.425+0000 7f2062d8f700 1 --2- 172.21.15.82:0/1207579212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f205c1087b0 0x7f205c108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.425+0000 7f2062d8f700 1 --2- 172.21.15.82:0/1207579212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f205c103160 0x7f205c10f670 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.425+0000 7f2062d8f700 1 --2- 172.21.15.82:0/1207579212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f205c1027a0 0x7f205c102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.425+0000 7f2062d8f700 1 -- 172.21.15.82:0/1207579212 >> 172.21.15.82:0/1207579212 conn(0x7f205c0fe250 msgr2=0x7f205c100670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:17:54.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.425+0000 7f2062d8f700 1 -- 172.21.15.82:0/1207579212 shutdown_connections 2023-09-09T23:17:54.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.425+0000 7f2062d8f700 1 -- 172.21.15.82:0/1207579212 wait complete. 2023-09-09T23:17:54.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.426+0000 7f2062d8f700 1 Processor -- start 2023-09-09T23:17:54.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.426+0000 7f2062d8f700 1 -- start start 2023-09-09T23:17:54.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.426+0000 7f2062d8f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f205c1027a0 0x7f205c19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:54.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.426+0000 7f2062d8f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f205c103160 0x7f205c19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:54.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.426+0000 7f2062d8f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f205c1087b0 0x7f205c198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:54.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.426+0000 7f2062d8f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c198ce0 con 0x7f205c1087b0 2023-09-09T23:17:54.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.426+0000 7f2062d8f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c198e50 con 0x7f205c1027a0 2023-09-09T23:17:54.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.426+0000 7f2062d8f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c199010 con 0x7f205c103160 2023-09-09T23:17:54.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.426+0000 7f206132c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f205c1087b0 0x7f205c198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:54.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.426+0000 7f2060b2b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f205c1027a0 0x7f205c19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:54.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.426+0000 7f205bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f205c103160 0x7f205c19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:54.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.426+0000 7f2060b2b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f205c1027a0 0x7f205c19e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:43304/0 (socket says 172.21.15.82:43304) 2023-09-09T23:17:54.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.426+0000 7f2060b2b700 1 -- 172.21.15.82:0/2202571762 learned_addr learned my addr 172.21.15.82:0/2202571762 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:17:54.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.426+0000 7f206132c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f205c1087b0 0x7f205c198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57744/0 (socket says 172.21.15.82:57744) 2023-09-09T23:17:54.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.427+0000 7f206132c700 1 -- 172.21.15.82:0/2202571762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f205c103160 msgr2=0x7f205c19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:17:54.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.427+0000 7f206132c700 1 --2- 172.21.15.82:0/2202571762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f205c103160 0x7f205c19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.427+0000 7f206132c700 1 -- 172.21.15.82:0/2202571762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f205c1027a0 msgr2=0x7f205c19e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:17:54.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.427+0000 7f206132c700 1 --2- 172.21.15.82:0/2202571762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f205c1027a0 0x7f205c19e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.427+0000 7f206132c700 1 -- 172.21.15.82:0/2202571762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2054009d00 con 0x7f205c1087b0 2023-09-09T23:17:54.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.427+0000 7f2060b2b700 1 --2- 172.21.15.82:0/2202571762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f205c1027a0 0x7f205c19e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:17:54.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.427+0000 7f206132c700 1 --2- 172.21.15.82:0/2202571762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f205c1087b0 0x7f205c198670 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f205400b0a0 tx=0x7f2054007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:17:54.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.427+0000 7f2059ffb700 1 -- 172.21.15.82:0/2202571762 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205400e040 con 0x7f205c1087b0 2023-09-09T23:17:54.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.427+0000 7f2062d8f700 1 -- 172.21.15.82:0/2202571762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f205c199290 con 0x7f205c1087b0 2023-09-09T23:17:54.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.427+0000 7f2059ffb700 1 -- 172.21.15.82:0/2202571762 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2054004e60 con 0x7f205c1087b0 2023-09-09T23:17:54.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.427+0000 7f2059ffb700 1 -- 172.21.15.82:0/2202571762 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2054005040 con 0x7f205c1087b0 2023-09-09T23:17:54.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.427+0000 7f2062d8f700 1 -- 172.21.15.82:0/2202571762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f205c1a55a0 con 0x7f205c1087b0 2023-09-09T23:17:54.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.429+0000 7f2059ffb700 1 -- 172.21.15.82:0/2202571762 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f205401d470 con 0x7f205c1087b0 2023-09-09T23:17:54.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.430+0000 7f2059ffb700 1 --2- 172.21.15.82:0/2202571762 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f204c0a0830 0x7f204c0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:54.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.430+0000 7f2060b2b700 1 --2- 172.21.15.82:0/2202571762 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f204c0a0830 0x7f204c0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:54.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.430+0000 7f2059ffb700 1 -- 172.21.15.82:0/2202571762 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f20540cd320 con 0x7f205c1087b0 2023-09-09T23:17:54.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.431+0000 7f2060b2b700 1 --2- 172.21.15.82:0/2202571762 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f204c0a0830 0x7f204c0a2cf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2050011550 tx=0x7f2050009260 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:17:54.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.431+0000 7f206ef55b80 1 Processor -- start 2023-09-09T23:17:54.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.431+0000 7f206ef55b80 1 -- start start 2023-09-09T23:17:54.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.432+0000 7f206ef55b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1702bd710 0x55c1702bdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:54.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.432+0000 7f206ef55b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1702be030 0x55c1702c7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:54.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.432+0000 7f206ef55b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c1702c7e40 0x55c1702ca230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:54.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.432+0000 7f206ef55b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c170216730 con 0x55c1702be030 2023-09-09T23:17:54.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.432+0000 7f206ef55b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c170214050 con 0x55c1702c7e40 2023-09-09T23:17:54.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.432+0000 7f206ef55b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17020f940 con 0x55c1702bd710 2023-09-09T23:17:54.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.432+0000 7f205bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1702be030 0x55c1702c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:54.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.432+0000 7f2060b2b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1702bd710 0x55c1702bdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:54.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.432+0000 7f205bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1702be030 0x55c1702c7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57750/0 (socket says 172.21.15.82:57750) 2023-09-09T23:17:54.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.432+0000 7f2060b2b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1702bd710 0x55c1702bdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:56262/0 (socket says 172.21.15.82:56262) 2023-09-09T23:17:54.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.432+0000 7f205bfff700 1 -- 172.21.15.82:0/3624795185 learned_addr learned my addr 172.21.15.82:0/3624795185 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:17:54.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.432+0000 7f206132c700 1 --2- 172.21.15.82:0/3624795185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c1702c7e40 0x55c1702ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:54.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.432+0000 7f205bfff700 1 -- 172.21.15.82:0/3624795185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1702bd710 msgr2=0x55c1702bdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:17:54.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.432+0000 7f205bfff700 1 --2- 172.21.15.82:0/3624795185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1702bd710 0x55c1702bdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.432+0000 7f205bfff700 1 -- 172.21.15.82:0/3624795185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c1702c7e40 msgr2=0x55c1702ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:17:54.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.432+0000 7f205bfff700 1 --2- 172.21.15.82:0/3624795185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c1702c7e40 0x55c1702ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.432+0000 7f205bfff700 1 -- 172.21.15.82:0/3624795185 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c17003bde0 con 0x55c1702be030 2023-09-09T23:17:54.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.432+0000 7f205bfff700 1 --2- 172.21.15.82:0/3624795185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1702be030 0x55c1702c7900 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2048009d20 tx=0x7f2048011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955cbe01220bd40a server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:17:54.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.432+0000 7f2046ffd700 1 -- 172.21.15.82:0/3624795185 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2048019630 con 0x55c1702be030 2023-09-09T23:17:54.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.432+0000 7f2046ffd700 1 -- 172.21.15.82:0/3624795185 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2048019c50 con 0x55c1702be030 2023-09-09T23:17:54.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.432+0000 7f2046ffd700 1 -- 172.21.15.82:0/3624795185 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204800ebe0 con 0x55c1702be030 2023-09-09T23:17:54.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.433+0000 7f206ef55b80 1 -- 172.21.15.82:0/3624795185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1702be030 msgr2=0x55c1702c7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:17:54.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.433+0000 7f206ef55b80 1 --2- 172.21.15.82:0/3624795185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1702be030 0x55c1702c7900 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2048009d20 tx=0x7f2048011720 comp rx=0 tx=0).stop 2023-09-09T23:17:54.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.433+0000 7f206ef55b80 1 -- 172.21.15.82:0/3624795185 shutdown_connections 2023-09-09T23:17:54.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.433+0000 7f206ef55b80 1 --2- 172.21.15.82:0/3624795185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c1702c7e40 0x55c1702ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.433+0000 7f206ef55b80 1 --2- 172.21.15.82:0/3624795185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1702bd710 0x55c1702bdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.433+0000 7f206ef55b80 1 --2- 172.21.15.82:0/3624795185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1702be030 0x55c1702c7900 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.433+0000 7f206ef55b80 1 -- 172.21.15.82:0/3624795185 >> 172.21.15.82:0/3624795185 conn(0x55c170232090 msgr2=0x55c1702b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:17:54.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.433+0000 7f206ef55b80 1 -- 172.21.15.82:0/3624795185 shutdown_connections 2023-09-09T23:17:54.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.433+0000 7f206ef55b80 1 -- 172.21.15.82:0/3624795185 wait complete. 2023-09-09T23:17:54.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.433+0000 7f206ef55b80 1 -- start start 2023-09-09T23:17:54.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.433+0000 7f206ef55b80 1 Processor -- start 2023-09-09T23:17:54.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.434+0000 7f206ef55b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1702bd710 0x55c1702c7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:54.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.434+0000 7f206ef55b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c1702be030 0x55c1702c4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:54.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.434+0000 7f206ef55b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1702c7e40 0x55c1702c5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:54.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.434+0000 7f206ef55b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1701adb70 con 0x55c1702c7e40 2023-09-09T23:17:54.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.434+0000 7f206ef55b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1701ad870 con 0x55c1702be030 2023-09-09T23:17:54.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.434+0000 7f206ef55b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c170091920 con 0x55c1702bd710 2023-09-09T23:17:54.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.434+0000 7f206ef55b80 10 client.0 _ll_register_callbacks cb 0x55c1700f7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:17:54.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.434+0000 7f206132c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1702c7e40 0x55c1702c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:54.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.434+0000 7f206132c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1702c7e40 0x55c1702c5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57758/0 (socket says 172.21.15.82:57758) 2023-09-09T23:17:54.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.434+0000 7f206132c700 1 -- 172.21.15.82:0/684725677 learned_addr learned my addr 172.21.15.82:0/684725677 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:17:54.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.434+0000 7f205bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c1702be030 0x55c1702c4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:54.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.434+0000 7f2060b2b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1702bd710 0x55c1702c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:54.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.434+0000 7f206132c700 1 -- 172.21.15.82:0/684725677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1702bd710 msgr2=0x55c1702c7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:17:54.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.434+0000 7f206132c700 1 --2- 172.21.15.82:0/684725677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1702bd710 0x55c1702c7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.434+0000 7f206132c700 1 -- 172.21.15.82:0/684725677 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c1702be030 msgr2=0x55c1702c4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:17:54.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.434+0000 7f206132c700 1 --2- 172.21.15.82:0/684725677 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c1702be030 0x55c1702c4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.434+0000 7f206132c700 1 -- 172.21.15.82:0/684725677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2048009980 con 0x55c1702c7e40 2023-09-09T23:17:54.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.434+0000 7f2060b2b700 1 --2- 172.21.15.82:0/684725677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1702bd710 0x55c1702c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:17:54.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.435+0000 7f206132c700 1 --2- 172.21.15.82:0/684725677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1702c7e40 0x55c1702c5050 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f205409a5d0 tx=0x7f205409a770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a1f7b9b096e16d server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:17:54.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.435+0000 7f203bfff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:17:54.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.435+0000 7f203bfff700 1 -- 172.21.15.82:0/684725677 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205409ad30 con 0x55c1702c7e40 2023-09-09T23:17:54.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.435+0000 7f203bfff700 1 -- 172.21.15.82:0/684725677 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f2054009450 con 0x55c1702c7e40 2023-09-09T23:17:54.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.435+0000 7f206ef55b80 1 -- 172.21.15.82:0/684725677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c1701674f0 con 0x55c1702c7e40 2023-09-09T23:17:54.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.435+0000 7f203bfff700 1 -- 172.21.15.82:0/684725677 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f205409afd0 con 0x55c1702c7e40 2023-09-09T23:17:54.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.435+0000 7f206ef55b80 10 client.16065 fetch_fsmap learned FSMap version 36 2023-09-09T23:17:54.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.435+0000 7f206ef55b80 1 -- 172.21.15.82:0/684725677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55c170024c60 con 0x55c1702c7e40 2023-09-09T23:17:54.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.435+0000 7f203bfff700 1 -- 172.21.15.82:0/684725677 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c1701674f0 con 0x55c1702c7e40 2023-09-09T23:17:54.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.435+0000 7f203bfff700 1 -- 172.21.15.82:0/684725677 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f20540a2420 con 0x55c1702c7e40 2023-09-09T23:17:54.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.435+0000 7f206ef55b80 10 client.16065 fetch_fsmap finished waiting for FSMap version 36 2023-09-09T23:17:54.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.435+0000 7f206ef55b80 10 client.16065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:17:54.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.435+0000 7f206ef55b80 20 client.16065 populate_metadata read hostname 'smithi082' 2023-09-09T23:17:54.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.435+0000 7f206ef55b80 1 --2- 172.21.15.82:0/684725677 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c1702d7630 0x55c1702d9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:54.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.435+0000 7f206ef55b80 4 client.16065 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:17:54.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.435+0000 7f206ef55b80 1 -- 172.21.15.82:0/684725677 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1702248a0 con 0x55c1702d7630 2023-09-09T23:17:54.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.435+0000 7f2060b2b700 1 --2- 172.21.15.82:0/684725677 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c1702d7630 0x55c1702d9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:54.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.436+0000 7f2060b2b700 1 --2- 172.21.15.82:0/684725677 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c1702d7630 0x55c1702d9a20 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d01e99055129720f server_cookie=58f0e374fa0800a6 in_seq=0 out_seq=0 2023-09-09T23:17:54.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.436+0000 7f203bfff700 10 client.16065 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:17:54.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.437+0000 7f203bfff700 1 -- 172.21.15.82:0/684725677 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c1702248a0 con 0x55c1702d7630 2023-09-09T23:17:54.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.437+0000 7f203bfff700 10 client.16065 handle_command_reply: tid=0 2023-09-09T23:17:54.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.437+0000 7f206ef55b80 1 client.16065 shutdown 2023-09-09T23:17:54.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.437+0000 7f206ef55b80 1 -- 172.21.15.82:0/684725677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1702c7e40 msgr2=0x55c1702c5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:17:54.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.437+0000 7f206ef55b80 1 --2- 172.21.15.82:0/684725677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1702c7e40 0x55c1702c5050 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f205409a5d0 tx=0x7f205409a770 comp rx=0 tx=0).stop 2023-09-09T23:17:54.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.437+0000 7f206ef55b80 1 -- 172.21.15.82:0/684725677 shutdown_connections 2023-09-09T23:17:54.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.437+0000 7f206ef55b80 1 --2- 172.21.15.82:0/684725677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1702c7e40 0x55c1702c5050 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.437+0000 7f206ef55b80 1 --2- 172.21.15.82:0/684725677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1702bd710 0x55c1702c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.437+0000 7f206ef55b80 1 --2- 172.21.15.82:0/684725677 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c1702d7630 0x55c1702d9a20 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.437+0000 7f206ef55b80 1 --2- 172.21.15.82:0/684725677 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c1702be030 0x55c1702c4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.437+0000 7f206ef55b80 1 -- 172.21.15.82:0/684725677 >> 172.21.15.82:0/684725677 conn(0x55c170232090 msgr2=0x55c1702b87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:17:54.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.437+0000 7f203bfff700 0 client.16065 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:17:54.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.437+0000 7f206ef55b80 1 -- 172.21.15.82:0/684725677 shutdown_connections 2023-09-09T23:17:54.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.437+0000 7f206ef55b80 1 -- 172.21.15.82:0/684725677 wait complete. 2023-09-09T23:17:54.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.437+0000 7f206ef55b80 20 client.16065 trim_cache size 0 max 16384 2023-09-09T23:17:54.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.454+0000 7f206ef55b80 1 Processor -- start 2023-09-09T23:17:54.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.454+0000 7f206ef55b80 1 -- start start 2023-09-09T23:17:54.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.454+0000 7f206ef55b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1702bd710 0x55c1702ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:54.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.454+0000 7f206ef55b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1702be030 0x55c1702bfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:54.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.454+0000 7f206ef55b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c1702c7e40 0x55c1702c0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:54.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.454+0000 7f206ef55b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1702c5650 con 0x55c1702be030 2023-09-09T23:17:54.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.454+0000 7f206ef55b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1702ca6d0 con 0x55c1702c7e40 2023-09-09T23:17:54.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.454+0000 7f206ef55b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1702d7410 con 0x55c1702bd710 2023-09-09T23:17:54.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.454+0000 7f205bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1702be030 0x55c1702bfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:54.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.454+0000 7f205bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1702be030 0x55c1702bfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57764/0 (socket says 172.21.15.82:57764) 2023-09-09T23:17:54.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.454+0000 7f205bfff700 1 -- 172.21.15.82:0/2210172854 learned_addr learned my addr 172.21.15.82:0/2210172854 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:17:54.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.454+0000 7f2060b2b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1702bd710 0x55c1702ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:54.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.454+0000 7f206132c700 1 --2- 172.21.15.82:0/2210172854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c1702c7e40 0x55c1702c0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:54.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.454+0000 7f205bfff700 1 -- 172.21.15.82:0/2210172854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1702bd710 msgr2=0x55c1702ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:17:54.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.454+0000 7f205bfff700 1 --2- 172.21.15.82:0/2210172854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1702bd710 0x55c1702ce9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.454+0000 7f205bfff700 1 -- 172.21.15.82:0/2210172854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c1702c7e40 msgr2=0x55c1702c0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:17:54.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.454+0000 7f205bfff700 1 --2- 172.21.15.82:0/2210172854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c1702c7e40 0x55c1702c0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.454+0000 7f205bfff700 1 -- 172.21.15.82:0/2210172854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c170062250 con 0x55c1702be030 2023-09-09T23:17:54.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.454+0000 7f2060b2b700 1 --2- 172.21.15.82:0/2210172854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1702bd710 0x55c1702ce9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:17:54.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.454+0000 7f205bfff700 1 --2- 172.21.15.82:0/2210172854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1702be030 0x55c1702bfd20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x55c1702b3280 tx=0x7f204800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e4238d68251d7d server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:17:54.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.455+0000 7f20477fe700 1 -- 172.21.15.82:0/2210172854 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204800e5b0 con 0x55c1702be030 2023-09-09T23:17:54.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.455+0000 7f20477fe700 1 -- 172.21.15.82:0/2210172854 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20480198f0 con 0x55c1702be030 2023-09-09T23:17:54.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.455+0000 7f20477fe700 1 -- 172.21.15.82:0/2210172854 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204800e790 con 0x55c1702be030 2023-09-09T23:17:54.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.455+0000 7f206ef55b80 1 -- 172.21.15.82:0/2210172854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1702be030 msgr2=0x55c1702bfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:17:54.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.455+0000 7f206ef55b80 1 --2- 172.21.15.82:0/2210172854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1702be030 0x55c1702bfd20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x55c1702b3280 tx=0x7f204800e720 comp rx=0 tx=0).stop 2023-09-09T23:17:54.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.455+0000 7f206ef55b80 1 -- 172.21.15.82:0/2210172854 shutdown_connections 2023-09-09T23:17:54.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.455+0000 7f206ef55b80 1 --2- 172.21.15.82:0/2210172854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c1702c7e40 0x55c1702c0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.455+0000 7f206ef55b80 1 --2- 172.21.15.82:0/2210172854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1702bd710 0x55c1702ce9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.455+0000 7f206ef55b80 1 --2- 172.21.15.82:0/2210172854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1702be030 0x55c1702bfd20 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.455+0000 7f206ef55b80 1 -- 172.21.15.82:0/2210172854 >> 172.21.15.82:0/2210172854 conn(0x55c170232090 msgr2=0x55c1702d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:17:54.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.456+0000 7f206ef55b80 1 -- 172.21.15.82:0/2210172854 shutdown_connections 2023-09-09T23:17:54.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.456+0000 7f206ef55b80 1 -- 172.21.15.82:0/2210172854 wait complete. 2023-09-09T23:17:54.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.456+0000 7f206ef55b80 1 -- start start 2023-09-09T23:17:54.477 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.456+0000 7f206ef55b80 1 Processor -- start 2023-09-09T23:17:54.477 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.456+0000 7f206ef55b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1702bd710 0x55c1703237b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:54.477 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.456+0000 7f206ef55b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c1702be030 0x55c170323cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:54.477 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.456+0000 7f206ef55b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1702c7e40 0x55c17031f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:54.477 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.456+0000 7f206ef55b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17031fa70 con 0x55c1702bd710 2023-09-09T23:17:54.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.456+0000 7f206ef55b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17031fbe0 con 0x55c1702be030 2023-09-09T23:17:54.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.456+0000 7f206ef55b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17031fda0 con 0x55c1702c7e40 2023-09-09T23:17:54.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.457+0000 7f206ef55b80 10 client.0 _ll_register_callbacks cb 0x7f2054096cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:17:54.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.457+0000 7f2060b2b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1702bd710 0x55c1703237b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:54.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.457+0000 7f206132c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1702c7e40 0x55c17031f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:54.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.457+0000 7f205bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c1702be030 0x55c170323cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:54.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.457+0000 7f2060b2b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1702bd710 0x55c1703237b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57768/0 (socket says 172.21.15.82:57768) 2023-09-09T23:17:54.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.457+0000 7f2060b2b700 1 -- 172.21.15.82:0/4026342054 learned_addr learned my addr 172.21.15.82:0/4026342054 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:17:54.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.457+0000 7f206132c700 1 -- 172.21.15.82:0/4026342054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c1702be030 msgr2=0x55c170323cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:17:54.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.457+0000 7f206132c700 1 --2- 172.21.15.82:0/4026342054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c1702be030 0x55c170323cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.457+0000 7f206132c700 1 -- 172.21.15.82:0/4026342054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1702bd710 msgr2=0x55c1703237b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:17:54.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.457+0000 7f206132c700 1 --2- 172.21.15.82:0/4026342054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1702bd710 0x55c1703237b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.457+0000 7f206132c700 1 -- 172.21.15.82:0/4026342054 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2050017070 con 0x55c1702c7e40 2023-09-09T23:17:54.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.457+0000 7f2060b2b700 1 --2- 172.21.15.82:0/4026342054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1702bd710 0x55c1703237b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:17:54.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.457+0000 7f206132c700 1 --2- 172.21.15.82:0/4026342054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1702c7e40 0x55c17031f360 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f20540a2380 tx=0x7f205409aea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7cba81a17e7535b server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:17:54.481 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.457+0000 7f203bfff700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-09T23:17:54.481 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.457+0000 7f203bfff700 1 -- 172.21.15.82:0/4026342054 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205409a410 con 0x55c1702c7e40 2023-09-09T23:17:54.481 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.457+0000 7f203bfff700 1 -- 172.21.15.82:0/4026342054 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f20540b2e40 con 0x55c1702c7e40 2023-09-09T23:17:54.481 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.457+0000 7f206ef55b80 1 -- 172.21.15.82:0/4026342054 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2048019c50 con 0x55c1702c7e40 2023-09-09T23:17:54.481 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.457+0000 7f203bfff700 1 -- 172.21.15.82:0/4026342054 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f205409a040 con 0x55c1702c7e40 2023-09-09T23:17:54.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.457+0000 7f206ef55b80 10 client.25438 fetch_fsmap learned FSMap version 36 2023-09-09T23:17:54.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.457+0000 7f206ef55b80 1 -- 172.21.15.82:0/4026342054 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55c17003bde0 con 0x55c1702c7e40 2023-09-09T23:17:54.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.458+0000 7f203bfff700 1 -- 172.21.15.82:0/4026342054 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20540a6080 con 0x55c1702c7e40 2023-09-09T23:17:54.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.458+0000 7f203bfff700 1 -- 172.21.15.82:0/4026342054 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 36) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f20540b8a90 con 0x55c1702c7e40 2023-09-09T23:17:54.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.458+0000 7f206ef55b80 10 client.25438 fetch_fsmap finished waiting for FSMap version 36 2023-09-09T23:17:54.483 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.458+0000 7f206ef55b80 10 client.25438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:17:54.483 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.458+0000 7f206ef55b80 20 client.25438 populate_metadata read hostname 'smithi082' 2023-09-09T23:17:54.483 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.458+0000 7f206ef55b80 1 --2- 172.21.15.82:0/4026342054 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c17032c2b0 0x55c17032e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:17:54.483 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.458+0000 7f206ef55b80 4 client.25438 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:17:54.483 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.458+0000 7f206ef55b80 1 -- 172.21.15.82:0/4026342054 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c170208920 con 0x55c17032c2b0 2023-09-09T23:17:54.483 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.458+0000 7f2060b2b700 1 --2- 172.21.15.82:0/4026342054 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c17032c2b0 0x55c17032e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:17:54.484 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.458+0000 7f203bfff700 10 client.25438 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:17:54.484 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.458+0000 7f2060b2b700 1 --2- 172.21.15.82:0/4026342054 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c17032c2b0 0x55c17032e6a0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f727100c6e8e142 server_cookie=fa4a0ef0bf117789 in_seq=0 out_seq=0 2023-09-09T23:17:54.484 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.459+0000 7f203bfff700 1 -- 172.21.15.82:0/4026342054 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55c170208920 con 0x55c17032c2b0 2023-09-09T23:17:54.484 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.459+0000 7f203bfff700 10 client.25438 handle_command_reply: tid=0 2023-09-09T23:17:54.484 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.459+0000 7f206ef55b80 1 client.25438 shutdown 2023-09-09T23:17:54.485 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.459+0000 7f206ef55b80 1 -- 172.21.15.82:0/4026342054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1702c7e40 msgr2=0x55c17031f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:17:54.485 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.459+0000 7f206ef55b80 1 --2- 172.21.15.82:0/4026342054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1702c7e40 0x55c17031f360 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f20540a2380 tx=0x7f205409aea0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.485 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.459+0000 7f206ef55b80 1 -- 172.21.15.82:0/4026342054 shutdown_connections 2023-09-09T23:17:54.485 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.459+0000 7f206ef55b80 1 --2- 172.21.15.82:0/4026342054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1702c7e40 0x55c17031f360 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.485 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.459+0000 7f206ef55b80 1 --2- 172.21.15.82:0/4026342054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1702bd710 0x55c1703237b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.486 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.459+0000 7f206ef55b80 1 --2- 172.21.15.82:0/4026342054 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c17032c2b0 0x55c17032e6a0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.486 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.459+0000 7f206ef55b80 1 --2- 172.21.15.82:0/4026342054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c1702be030 0x55c170323cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.486 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.459+0000 7f203bfff700 0 client.25438 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:17:54.486 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.459+0000 7f206ef55b80 1 -- 172.21.15.82:0/4026342054 >> 172.21.15.82:0/4026342054 conn(0x55c170232090 msgr2=0x55c1702bfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:17:54.486 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.459+0000 7f206ef55b80 1 -- 172.21.15.82:0/4026342054 shutdown_connections 2023-09-09T23:17:54.487 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.459+0000 7f206ef55b80 1 -- 172.21.15.82:0/4026342054 wait complete. 2023-09-09T23:17:54.487 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.459+0000 7f206ef55b80 20 client.25438 trim_cache size 0 max 16384 2023-09-09T23:17:54.487 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:17:54.487 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:17:54.488 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:17:54.488 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:17:54.488 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:17:54.488 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:17:54.489 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:17:54.489 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:17:54.489 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:17:54.489 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:17:54.490 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.460+0000 7f2062d8f700 1 -- 172.21.15.82:0/2202571762 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f204c0a0830 msgr2=0x7f204c0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:17:54.491 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.460+0000 7f2062d8f700 1 --2- 172.21.15.82:0/2202571762 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f204c0a0830 0x7f204c0a2cf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2050011550 tx=0x7f2050009260 comp rx=0 tx=0).stop 2023-09-09T23:17:54.491 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.460+0000 7f2062d8f700 1 -- 172.21.15.82:0/2202571762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f205c1087b0 msgr2=0x7f205c198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:17:54.491 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.460+0000 7f2062d8f700 1 --2- 172.21.15.82:0/2202571762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f205c1087b0 0x7f205c198670 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f205400b0a0 tx=0x7f2054007870 comp rx=0 tx=0).stop 2023-09-09T23:17:54.491 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.460+0000 7f2062d8f700 1 -- 172.21.15.82:0/2202571762 shutdown_connections 2023-09-09T23:17:54.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.460+0000 7f2062d8f700 1 --2- 172.21.15.82:0/2202571762 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f204c0a0830 0x7f204c0a2cf0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.460+0000 7f2062d8f700 1 --2- 172.21.15.82:0/2202571762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f205c1027a0 0x7f205c19e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.460+0000 7f2062d8f700 1 --2- 172.21.15.82:0/2202571762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f205c1087b0 0x7f205c198670 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.460+0000 7f2062d8f700 1 --2- 172.21.15.82:0/2202571762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f205c103160 0x7f205c19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:17:54.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.460+0000 7f2062d8f700 1 -- 172.21.15.82:0/2202571762 >> 172.21.15.82:0/2202571762 conn(0x7f205c0fe250 msgr2=0x7f205c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:17:54.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.460+0000 7f2062d8f700 1 -- 172.21.15.82:0/2202571762 shutdown_connections 2023-09-09T23:17:54.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:17:54.460+0000 7f2062d8f700 1 -- 172.21.15.82:0/2202571762 wait complete. 2023-09-09T23:17:54.493 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:17:54.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:54 smithi082 ceph-mon[98694]: pgmap v123: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 42 MiB/s wr, 599 op/s 2023-09-09T23:17:54.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:54 smithi098 ceph-mon[103124]: pgmap v123: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 42 MiB/s wr, 599 op/s 2023-09-09T23:17:54.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:54 smithi170 ceph-mon[102832]: pgmap v123: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 42 MiB/s wr, 599 op/s 2023-09-09T23:17:57.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:57 smithi170 ceph-mon[102832]: pgmap v124: 97 pgs: 97 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 144 MiB/s wr, 1.27k op/s 2023-09-09T23:17:57.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:57 smithi082 ceph-mon[98694]: pgmap v124: 97 pgs: 97 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 144 MiB/s wr, 1.27k op/s 2023-09-09T23:17:57.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:57 smithi098 ceph-mon[103124]: pgmap v124: 97 pgs: 97 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 144 MiB/s wr, 1.27k op/s 2023-09-09T23:17:59.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:17:59 smithi170 ceph-mon[102832]: pgmap v125: 97 pgs: 97 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 143 MiB/s wr, 1.03k op/s 2023-09-09T23:17:59.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:17:59 smithi082 ceph-mon[98694]: pgmap v125: 97 pgs: 97 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 143 MiB/s wr, 1.03k op/s 2023-09-09T23:17:59.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:17:59 smithi098 ceph-mon[103124]: pgmap v125: 97 pgs: 97 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 143 MiB/s wr, 1.03k op/s 2023-09-09T23:18:01.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:01 smithi170 ceph-mon[102832]: pgmap v126: 97 pgs: 97 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 198 MiB/s wr, 1.40k op/s 2023-09-09T23:18:01.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:01 smithi082 ceph-mon[98694]: pgmap v126: 97 pgs: 97 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 198 MiB/s wr, 1.40k op/s 2023-09-09T23:18:01.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:01 smithi098 ceph-mon[103124]: pgmap v126: 97 pgs: 97 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 198 MiB/s wr, 1.40k op/s 2023-09-09T23:18:03.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:03 smithi170 ceph-mon[102832]: pgmap v127: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 199 MiB/s wr, 1.51k op/s 2023-09-09T23:18:03.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:03 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:18:03.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:18:03.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:03 smithi082 ceph-mon[98694]: pgmap v127: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 199 MiB/s wr, 1.51k op/s 2023-09-09T23:18:03.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:03 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:18:03.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:18:03.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:03 smithi098 ceph-mon[103124]: pgmap v127: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 199 MiB/s wr, 1.51k op/s 2023-09-09T23:18:03.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:03 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:18:03.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:18:04.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:18:04.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:18:04.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:18:04.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:18:04.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:18:04.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:18:04.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:18:04.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:18:04.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:18:05.616 INFO:teuthology.orchestra.run.smithi082.stderr:set primary keys... 2023-09-09T23:18:05.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:05 smithi170 ceph-mon[102832]: pgmap v128: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 181 MiB/s wr, 1.34k op/s 2023-09-09T23:18:05.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:05 smithi082 ceph-mon[98694]: pgmap v128: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 181 MiB/s wr, 1.34k op/s 2023-09-09T23:18:05.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:05 smithi098 ceph-mon[103124]: pgmap v128: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 181 MiB/s wr, 1.34k op/s 2023-09-09T23:18:07.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:07 smithi170 ceph-mon[102832]: pgmap v129: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 181 MiB/s wr, 1.40k op/s 2023-09-09T23:18:07.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:07 smithi082 ceph-mon[98694]: pgmap v129: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 181 MiB/s wr, 1.40k op/s 2023-09-09T23:18:07.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:07 smithi098 ceph-mon[103124]: pgmap v129: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 181 MiB/s wr, 1.40k op/s 2023-09-09T23:18:08.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:08 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:18:08.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:08 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:18:08.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:08 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:18:09.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:09 smithi170 ceph-mon[102832]: pgmap v130: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 79 MiB/s wr, 731 op/s 2023-09-09T23:18:09.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:09 smithi082 ceph-mon[98694]: pgmap v130: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 79 MiB/s wr, 731 op/s 2023-09-09T23:18:09.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:09 smithi098 ceph-mon[103124]: pgmap v130: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 79 MiB/s wr, 731 op/s 2023-09-09T23:18:11.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:11 smithi170 ceph-mon[102832]: pgmap v131: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 79 MiB/s wr, 815 op/s 2023-09-09T23:18:11.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:11 smithi082 ceph-mon[98694]: pgmap v131: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 79 MiB/s wr, 815 op/s 2023-09-09T23:18:11.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:11 smithi098 ceph-mon[103124]: pgmap v131: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 79 MiB/s wr, 815 op/s 2023-09-09T23:18:13.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:13 smithi170 ceph-mon[102832]: pgmap v132: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 40 MiB/s rd, 24 MiB/s wr, 576 op/s 2023-09-09T23:18:13.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:13 smithi082 ceph-mon[98694]: pgmap v132: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 40 MiB/s rd, 24 MiB/s wr, 576 op/s 2023-09-09T23:18:13.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:13 smithi098 ceph-mon[103124]: pgmap v132: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 40 MiB/s rd, 24 MiB/s wr, 576 op/s 2023-09-09T23:18:15.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:15 smithi170 ceph-mon[102832]: pgmap v133: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 85 KiB/s wr, 267 op/s 2023-09-09T23:18:15.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:15 smithi082 ceph-mon[98694]: pgmap v133: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 85 KiB/s wr, 267 op/s 2023-09-09T23:18:15.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:15 smithi098 ceph-mon[103124]: pgmap v133: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 85 KiB/s wr, 267 op/s 2023-09-09T23:18:17.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:17 smithi170 ceph-mon[102832]: pgmap v134: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 240 MiB/s rd, 105 KiB/s wr, 1.32k op/s 2023-09-09T23:18:17.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:17 smithi082 ceph-mon[98694]: pgmap v134: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 240 MiB/s rd, 105 KiB/s wr, 1.32k op/s 2023-09-09T23:18:17.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:17 smithi098 ceph-mon[103124]: pgmap v134: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 240 MiB/s rd, 105 KiB/s wr, 1.32k op/s 2023-09-09T23:18:18.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:18:18.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:18:18.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:18:19.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:19 smithi170 ceph-mon[102832]: pgmap v135: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 239 MiB/s rd, 76 KiB/s wr, 1.26k op/s 2023-09-09T23:18:19.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:19 smithi082 ceph-mon[98694]: pgmap v135: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 239 MiB/s rd, 76 KiB/s wr, 1.26k op/s 2023-09-09T23:18:19.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:19 smithi098 ceph-mon[103124]: pgmap v135: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 239 MiB/s rd, 76 KiB/s wr, 1.26k op/s 2023-09-09T23:18:21.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:21 smithi170 ceph-mon[102832]: pgmap v136: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 287 MiB/s rd, 94 KiB/s wr, 1.51k op/s 2023-09-09T23:18:21.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:21 smithi082 ceph-mon[98694]: pgmap v136: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 287 MiB/s rd, 94 KiB/s wr, 1.51k op/s 2023-09-09T23:18:21.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:21 smithi098 ceph-mon[103124]: pgmap v136: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 287 MiB/s rd, 94 KiB/s wr, 1.51k op/s 2023-09-09T23:18:23.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:23 smithi170 ceph-mon[102832]: pgmap v137: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 312 MiB/s rd, 55 KiB/s wr, 1.63k op/s 2023-09-09T23:18:23.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:18:23.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:23 smithi082 ceph-mon[98694]: pgmap v137: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 312 MiB/s rd, 55 KiB/s wr, 1.63k op/s 2023-09-09T23:18:23.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:18:23.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:23 smithi098 ceph-mon[103124]: pgmap v137: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 312 MiB/s rd, 55 KiB/s wr, 1.63k op/s 2023-09-09T23:18:23.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:18:24.494 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:18:24.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.635+0000 7f29f4038700 1 -- 172.21.15.82:0/1149234752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29ec0ff440 msgr2=0x7f29ec0ff8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:18:24.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.635+0000 7f29f4038700 1 --2- 172.21.15.82:0/1149234752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29ec0ff440 0x7f29ec0ff8c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009990 tx=0x7f29e8009ca0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.635+0000 7f29f4038700 1 -- 172.21.15.82:0/1149234752 shutdown_connections 2023-09-09T23:18:24.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.635+0000 7f29f4038700 1 --2- 172.21.15.82:0/1149234752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29ec1054e0 0x7f29ec1058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.635+0000 7f29f4038700 1 --2- 172.21.15.82:0/1149234752 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29ec0ffe00 0x7f29ec10c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.635+0000 7f29f4038700 1 --2- 172.21.15.82:0/1149234752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29ec0ff440 0x7f29ec0ff8c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.635+0000 7f29f4038700 1 -- 172.21.15.82:0/1149234752 >> 172.21.15.82:0/1149234752 conn(0x7f29ec0faf50 msgr2=0x7f29ec0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:18:24.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.635+0000 7f29f4038700 1 -- 172.21.15.82:0/1149234752 shutdown_connections 2023-09-09T23:18:24.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.635+0000 7f29f4038700 1 -- 172.21.15.82:0/1149234752 wait complete. 2023-09-09T23:18:24.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.636+0000 7f29f4038700 1 Processor -- start 2023-09-09T23:18:24.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.636+0000 7f29f4038700 1 -- start start 2023-09-09T23:18:24.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.636+0000 7f29f4038700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29ec0ff440 0x7f29ec195520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:24.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.636+0000 7f29f4038700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29ec0ffe00 0x7f29ec195a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:24.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.636+0000 7f29f4038700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29ec1054e0 0x7f29ec199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:24.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.636+0000 7f29f4038700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec19a500 con 0x7f29ec1054e0 2023-09-09T23:18:24.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.636+0000 7f29f4038700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec19a670 con 0x7f29ec0ffe00 2023-09-09T23:18:24.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.636+0000 7f29f4038700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec19a830 con 0x7f29ec0ff440 2023-09-09T23:18:24.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.636+0000 7f29f25d5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29ec1054e0 0x7f29ec199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:24.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.636+0000 7f29f15d3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29ec0ffe00 0x7f29ec195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:24.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.636+0000 7f29f25d5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29ec1054e0 0x7f29ec199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57786/0 (socket says 172.21.15.82:57786) 2023-09-09T23:18:24.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.636+0000 7f29f1dd4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29ec0ff440 0x7f29ec195520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:24.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.636+0000 7f29f25d5700 1 -- 172.21.15.82:0/2363648937 learned_addr learned my addr 172.21.15.82:0/2363648937 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:18:24.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.637+0000 7f29f25d5700 1 -- 172.21.15.82:0/2363648937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29ec0ff440 msgr2=0x7f29ec195520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:18:24.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.637+0000 7f29f25d5700 1 --2- 172.21.15.82:0/2363648937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29ec0ff440 0x7f29ec195520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.637+0000 7f29f25d5700 1 -- 172.21.15.82:0/2363648937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29ec0ffe00 msgr2=0x7f29ec195a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:18:24.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.637+0000 7f29f25d5700 1 --2- 172.21.15.82:0/2363648937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29ec0ffe00 0x7f29ec195a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.637+0000 7f29f25d5700 1 -- 172.21.15.82:0/2363648937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e80095f0 con 0x7f29ec1054e0 2023-09-09T23:18:24.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.637+0000 7f29f25d5700 1 --2- 172.21.15.82:0/2363648937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29ec1054e0 0x7f29ec199ec0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f29e400ef30 tx=0x7f29e400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:18:24.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.637+0000 7f29e2ffd700 1 -- 172.21.15.82:0/2363648937 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e4018070 con 0x7f29ec1054e0 2023-09-09T23:18:24.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.637+0000 7f29e2ffd700 1 -- 172.21.15.82:0/2363648937 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29e400f040 con 0x7f29ec1054e0 2023-09-09T23:18:24.642 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.637+0000 7f29f4038700 1 -- 172.21.15.82:0/2363648937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29ec19aab0 con 0x7f29ec1054e0 2023-09-09T23:18:24.642 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.637+0000 7f29e2ffd700 1 -- 172.21.15.82:0/2363648937 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e400cd00 con 0x7f29ec1054e0 2023-09-09T23:18:24.642 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.637+0000 7f29f4038700 1 -- 172.21.15.82:0/2363648937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29ec1a2350 con 0x7f29ec1054e0 2023-09-09T23:18:24.650 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.639+0000 7f29e2ffd700 1 -- 172.21.15.82:0/2363648937 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f29e4003a00 con 0x7f29ec1054e0 2023-09-09T23:18:24.650 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.640+0000 7f29e2ffd700 1 --2- 172.21.15.82:0/2363648937 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f29d80a07b0 0x7f29d80a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:24.650 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.640+0000 7f29f1dd4700 1 --2- 172.21.15.82:0/2363648937 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f29d80a07b0 0x7f29d80a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:24.650 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.640+0000 7f29e2ffd700 1 -- 172.21.15.82:0/2363648937 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f29e40cbac0 con 0x7f29ec1054e0 2023-09-09T23:18:24.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.640+0000 7f29f1dd4700 1 --2- 172.21.15.82:0/2363648937 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f29d80a07b0 0x7f29d80a2c70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009be0 tx=0x7f29dc009260 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:18:24.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.641+0000 7f2a001feb80 1 Processor -- start 2023-09-09T23:18:24.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.641+0000 7f2a001feb80 1 -- start start 2023-09-09T23:18:24.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.641+0000 7f2a001feb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f004e3a710 0x55f004e3aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:24.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.641+0000 7f2a001feb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f004e3b030 0x55f004e46b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:24.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.641+0000 7f2a001feb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f004e47050 0x55f004e49440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:24.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.641+0000 7f2a001feb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f004d93730 con 0x55f004e47050 2023-09-09T23:18:24.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.641+0000 7f2a001feb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f004d91050 con 0x55f004e3a710 2023-09-09T23:18:24.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.641+0000 7f2a001feb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f004d8c940 con 0x55f004e3b030 2023-09-09T23:18:24.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.641+0000 7f29f25d5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f004e47050 0x55f004e49440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:24.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.641+0000 7f29f1dd4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f004e3a710 0x55f004e3aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:24.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.641+0000 7f29f25d5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f004e47050 0x55f004e49440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57794/0 (socket says 172.21.15.82:57794) 2023-09-09T23:18:24.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.641+0000 7f29f25d5700 1 -- 172.21.15.82:0/3446543346 learned_addr learned my addr 172.21.15.82:0/3446543346 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:18:24.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.641+0000 7f29f15d3700 1 --2- 172.21.15.82:0/3446543346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f004e3b030 0x55f004e46b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:24.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.642+0000 7f29f25d5700 1 -- 172.21.15.82:0/3446543346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f004e3b030 msgr2=0x55f004e46b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:18:24.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.642+0000 7f29f25d5700 1 --2- 172.21.15.82:0/3446543346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f004e3b030 0x55f004e46b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.642+0000 7f29f25d5700 1 -- 172.21.15.82:0/3446543346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f004e3a710 msgr2=0x55f004e3aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:18:24.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.642+0000 7f29f25d5700 1 --2- 172.21.15.82:0/3446543346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f004e3a710 0x55f004e3aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.642+0000 7f29f25d5700 1 -- 172.21.15.82:0/3446543346 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f004bb8de0 con 0x55f004e47050 2023-09-09T23:18:24.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.642+0000 7f29f1dd4700 1 --2- 172.21.15.82:0/3446543346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f004e3a710 0x55f004e3aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:18:24.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.642+0000 7f29f25d5700 1 --2- 172.21.15.82:0/3446543346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f004e47050 0x55f004e49440 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f29e40a0ac0 tx=0x7f29e4098c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf2d32ef645dcbf server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:18:24.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.642+0000 7f29d7fff700 1 -- 172.21.15.82:0/3446543346 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e4004ad0 con 0x55f004e47050 2023-09-09T23:18:24.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.642+0000 7f29d7fff700 1 -- 172.21.15.82:0/3446543346 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29e401b320 con 0x55f004e47050 2023-09-09T23:18:24.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.642+0000 7f29d7fff700 1 -- 172.21.15.82:0/3446543346 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e40a0ff0 con 0x55f004e47050 2023-09-09T23:18:24.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.643+0000 7f2a001feb80 1 -- 172.21.15.82:0/3446543346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f004e47050 msgr2=0x55f004e49440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:18:24.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.643+0000 7f2a001feb80 1 --2- 172.21.15.82:0/3446543346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f004e47050 0x55f004e49440 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f29e40a0ac0 tx=0x7f29e4098c00 comp rx=0 tx=0).stop 2023-09-09T23:18:24.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.643+0000 7f2a001feb80 1 -- 172.21.15.82:0/3446543346 shutdown_connections 2023-09-09T23:18:24.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.643+0000 7f2a001feb80 1 --2- 172.21.15.82:0/3446543346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f004e3a710 0x55f004e3aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.643+0000 7f2a001feb80 1 --2- 172.21.15.82:0/3446543346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f004e47050 0x55f004e49440 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.643+0000 7f2a001feb80 1 --2- 172.21.15.82:0/3446543346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f004e3b030 0x55f004e46b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.643+0000 7f2a001feb80 1 -- 172.21.15.82:0/3446543346 >> 172.21.15.82:0/3446543346 conn(0x55f004daf090 msgr2=0x55f004e35bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:18:24.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.643+0000 7f2a001feb80 1 -- 172.21.15.82:0/3446543346 shutdown_connections 2023-09-09T23:18:24.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.643+0000 7f2a001feb80 1 -- 172.21.15.82:0/3446543346 wait complete. 2023-09-09T23:18:24.659 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.643+0000 7f2a001feb80 1 -- start start 2023-09-09T23:18:24.659 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.643+0000 7f2a001feb80 1 Processor -- start 2023-09-09T23:18:24.659 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.644+0000 7f2a001feb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f004e3a710 0x55f004e43e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:24.659 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.644+0000 7f2a001feb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f004e3b030 0x55f004e44340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:24.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.644+0000 7f2a001feb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f004e47050 0x55f004e44880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:24.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.644+0000 7f2a001feb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f004d2ab70 con 0x55f004e47050 2023-09-09T23:18:24.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.644+0000 7f2a001feb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f004d2a870 con 0x55f004e3a710 2023-09-09T23:18:24.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.644+0000 7f2a001feb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f004c0e920 con 0x55f004e3b030 2023-09-09T23:18:24.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.644+0000 7f2a001feb80 10 client.0 _ll_register_callbacks cb 0x55f004c74a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:18:24.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.644+0000 7f29f25d5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f004e47050 0x55f004e44880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:24.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.644+0000 7f29f25d5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f004e47050 0x55f004e44880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57800/0 (socket says 172.21.15.82:57800) 2023-09-09T23:18:24.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.644+0000 7f29f25d5700 1 -- 172.21.15.82:0/3600271881 learned_addr learned my addr 172.21.15.82:0/3600271881 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:18:24.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.644+0000 7f29f1dd4700 1 --2- 172.21.15.82:0/3600271881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f004e3a710 0x55f004e43e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:24.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.644+0000 7f29f15d3700 1 --2- 172.21.15.82:0/3600271881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f004e3b030 0x55f004e44340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:24.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.644+0000 7f29f25d5700 1 -- 172.21.15.82:0/3600271881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f004e3b030 msgr2=0x55f004e44340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:18:24.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.644+0000 7f29f25d5700 1 --2- 172.21.15.82:0/3600271881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f004e3b030 0x55f004e44340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.644+0000 7f29f25d5700 1 -- 172.21.15.82:0/3600271881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f004e3a710 msgr2=0x55f004e43e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:18:24.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.644+0000 7f29f25d5700 1 --2- 172.21.15.82:0/3600271881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f004e3a710 0x55f004e43e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.644+0000 7f29f25d5700 1 -- 172.21.15.82:0/3600271881 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29dc017070 con 0x55f004e47050 2023-09-09T23:18:24.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.644+0000 7f29f25d5700 1 --2- 172.21.15.82:0/3600271881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f004e47050 0x55f004e44880 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f29e4003f90 tx=0x7f29e4009450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27476e3ce810c6e server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:18:24.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.644+0000 7f29d57fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:18:24.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.644+0000 7f29d57fa700 1 -- 172.21.15.82:0/3600271881 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e4099710 con 0x55f004e47050 2023-09-09T23:18:24.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.644+0000 7f29d57fa700 1 -- 172.21.15.82:0/3600271881 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f29e400ce60 con 0x55f004e47050 2023-09-09T23:18:24.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.644+0000 7f2a001feb80 1 -- 172.21.15.82:0/3600271881 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f004ce44f0 con 0x55f004e47050 2023-09-09T23:18:24.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.644+0000 7f29d57fa700 1 -- 172.21.15.82:0/3600271881 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29e40a0ff0 con 0x55f004e47050 2023-09-09T23:18:24.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.644+0000 7f2a001feb80 10 client.16083 fetch_fsmap learned FSMap version 40 2023-09-09T23:18:24.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.644+0000 7f2a001feb80 1 -- 172.21.15.82:0/3600271881 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f29ec1a2350 con 0x55f004e47050 2023-09-09T23:18:24.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.644+0000 7f29d57fa700 1 -- 172.21.15.82:0/3600271881 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f004ce44f0 con 0x55f004e47050 2023-09-09T23:18:24.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.645+0000 7f29d57fa700 1 -- 172.21.15.82:0/3600271881 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f29e4098500 con 0x55f004e47050 2023-09-09T23:18:24.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.645+0000 7f2a001feb80 10 client.16083 fetch_fsmap finished waiting for FSMap version 40 2023-09-09T23:18:24.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.645+0000 7f2a001feb80 10 client.16083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:18:24.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.645+0000 7f2a001feb80 20 client.16083 populate_metadata read hostname 'smithi082' 2023-09-09T23:18:24.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.645+0000 7f2a001feb80 1 --2- 172.21.15.82:0/3600271881 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55f004e567e0 0x55f004e58c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:24.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.645+0000 7f2a001feb80 4 client.16083 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:18:24.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.645+0000 7f2a001feb80 1 -- 172.21.15.82:0/3600271881 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f004da18a0 con 0x55f004e567e0 2023-09-09T23:18:24.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.645+0000 7f29f1dd4700 1 --2- 172.21.15.82:0/3600271881 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55f004e567e0 0x55f004e58c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:24.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.645+0000 7f29d57fa700 10 client.16083 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:18:24.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.645+0000 7f29f1dd4700 1 --2- 172.21.15.82:0/3600271881 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55f004e567e0 0x55f004e58c00 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbe9596bba39f1c1 server_cookie=ca288003a93a2b5d in_seq=0 out_seq=0 2023-09-09T23:18:24.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.646+0000 7f29d57fa700 1 -- 172.21.15.82:0/3600271881 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f004da18a0 con 0x55f004e567e0 2023-09-09T23:18:24.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.646+0000 7f29d57fa700 10 client.16083 handle_command_reply: tid=0 2023-09-09T23:18:24.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.646+0000 7f2a001feb80 1 client.16083 shutdown 2023-09-09T23:18:24.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.646+0000 7f2a001feb80 1 -- 172.21.15.82:0/3600271881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f004e47050 msgr2=0x55f004e44880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:18:24.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.646+0000 7f2a001feb80 1 --2- 172.21.15.82:0/3600271881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f004e47050 0x55f004e44880 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f29e4003f90 tx=0x7f29e4009450 comp rx=0 tx=0).stop 2023-09-09T23:18:24.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.646+0000 7f2a001feb80 1 -- 172.21.15.82:0/3600271881 shutdown_connections 2023-09-09T23:18:24.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.646+0000 7f2a001feb80 1 --2- 172.21.15.82:0/3600271881 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55f004e567e0 0x55f004e58c00 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.646+0000 7f2a001feb80 1 --2- 172.21.15.82:0/3600271881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f004e3a710 0x55f004e43e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.646+0000 7f29d57fa700 0 client.16083 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:18:24.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.646+0000 7f2a001feb80 1 --2- 172.21.15.82:0/3600271881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f004e47050 0x55f004e44880 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.646+0000 7f2a001feb80 1 --2- 172.21.15.82:0/3600271881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f004e3b030 0x55f004e44340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.646+0000 7f2a001feb80 1 -- 172.21.15.82:0/3600271881 >> 172.21.15.82:0/3600271881 conn(0x55f004daf090 msgr2=0x55f004e357e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:18:24.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.646+0000 7f2a001feb80 1 -- 172.21.15.82:0/3600271881 shutdown_connections 2023-09-09T23:18:24.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.646+0000 7f2a001feb80 1 -- 172.21.15.82:0/3600271881 wait complete. 2023-09-09T23:18:24.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.647+0000 7f2a001feb80 20 client.16083 trim_cache size 0 max 16384 2023-09-09T23:18:24.678 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.663+0000 7f2a001feb80 1 Processor -- start 2023-09-09T23:18:24.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.663+0000 7f2a001feb80 1 -- start start 2023-09-09T23:18:24.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.663+0000 7f2a001feb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f004e3a710 0x55f004e4dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:24.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.663+0000 7f2a001feb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f004e3b030 0x55f004e3f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:24.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.663+0000 7f2a001feb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f004e47050 0x55f004e3f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:24.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.663+0000 7f2a001feb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f004e3ed50 con 0x55f004e3a710 2023-09-09T23:18:24.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.663+0000 7f2a001feb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f004e4e160 con 0x55f004e3b030 2023-09-09T23:18:24.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.663+0000 7f2a001feb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f004e349a0 con 0x55f004e47050 2023-09-09T23:18:24.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.664+0000 7f29f1dd4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f004e3a710 0x55f004e4dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:24.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.664+0000 7f29f1dd4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f004e3a710 0x55f004e4dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57804/0 (socket says 172.21.15.82:57804) 2023-09-09T23:18:24.681 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.664+0000 7f29f1dd4700 1 -- 172.21.15.82:0/1310613646 learned_addr learned my addr 172.21.15.82:0/1310613646 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:18:24.681 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.664+0000 7f29f15d3700 1 --2- 172.21.15.82:0/1310613646 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f004e3b030 0x55f004e3f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:24.681 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.664+0000 7f29f25d5700 1 --2- 172.21.15.82:0/1310613646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f004e47050 0x55f004e3f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:24.681 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.664+0000 7f29f1dd4700 1 -- 172.21.15.82:0/1310613646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f004e47050 msgr2=0x55f004e3f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:18:24.681 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.664+0000 7f29f1dd4700 1 --2- 172.21.15.82:0/1310613646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f004e47050 0x55f004e3f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.664+0000 7f29f1dd4700 1 -- 172.21.15.82:0/1310613646 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f004e3b030 msgr2=0x55f004e3f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:18:24.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.664+0000 7f29f1dd4700 1 --2- 172.21.15.82:0/1310613646 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f004e3b030 0x55f004e3f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.664+0000 7f29f1dd4700 1 -- 172.21.15.82:0/1310613646 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f004ba1c60 con 0x55f004e3a710 2023-09-09T23:18:24.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.664+0000 7f29f1dd4700 1 --2- 172.21.15.82:0/1310613646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f004e3a710 0x55f004e4dbf0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f29dc005950 tx=0x7f29dc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4a10bee7fb5767 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:18:24.683 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.664+0000 7f29e0ff9700 1 -- 172.21.15.82:0/1310613646 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc024040 con 0x55f004e3a710 2023-09-09T23:18:24.683 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.664+0000 7f29e0ff9700 1 -- 172.21.15.82:0/1310613646 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29dc026040 con 0x55f004e3a710 2023-09-09T23:18:24.683 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.664+0000 7f29e0ff9700 1 -- 172.21.15.82:0/1310613646 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc003d80 con 0x55f004e3a710 2023-09-09T23:18:24.683 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.665+0000 7f2a001feb80 1 -- 172.21.15.82:0/1310613646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f004e3a710 msgr2=0x55f004e4dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:18:24.683 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.665+0000 7f2a001feb80 1 --2- 172.21.15.82:0/1310613646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f004e3a710 0x55f004e4dbf0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f29dc005950 tx=0x7f29dc014000 comp rx=0 tx=0).stop 2023-09-09T23:18:24.684 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.665+0000 7f2a001feb80 1 -- 172.21.15.82:0/1310613646 shutdown_connections 2023-09-09T23:18:24.684 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.665+0000 7f2a001feb80 1 --2- 172.21.15.82:0/1310613646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f004e47050 0x55f004e3f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.684 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.665+0000 7f2a001feb80 1 --2- 172.21.15.82:0/1310613646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f004e3a710 0x55f004e4dbf0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.684 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.665+0000 7f2a001feb80 1 --2- 172.21.15.82:0/1310613646 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f004e3b030 0x55f004e3f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.684 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.665+0000 7f2a001feb80 1 -- 172.21.15.82:0/1310613646 >> 172.21.15.82:0/1310613646 conn(0x55f004daf090 msgr2=0x55f004e52350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:18:24.685 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.665+0000 7f2a001feb80 1 -- 172.21.15.82:0/1310613646 shutdown_connections 2023-09-09T23:18:24.685 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.665+0000 7f2a001feb80 1 -- 172.21.15.82:0/1310613646 wait complete. 2023-09-09T23:18:24.685 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.665+0000 7f2a001feb80 1 -- start start 2023-09-09T23:18:24.685 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.665+0000 7f2a001feb80 1 Processor -- start 2023-09-09T23:18:24.685 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.665+0000 7f2a001feb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f004e3a710 0x55f004e4a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:24.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.665+0000 7f2a001feb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f004e3b030 0x55f004e4a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:24.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.665+0000 7f2a001feb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f004e47050 0x55f004e4ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:24.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.665+0000 7f2a001feb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f004eac9d0 con 0x55f004e3a710 2023-09-09T23:18:24.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.665+0000 7f2a001feb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f004eacb10 con 0x55f004e3b030 2023-09-09T23:18:24.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.665+0000 7f2a001feb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f004eacc50 con 0x55f004e47050 2023-09-09T23:18:24.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.665+0000 7f2a001feb80 10 client.0 _ll_register_callbacks cb 0x7f29dc005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:18:24.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.666+0000 7f29f1dd4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f004e3a710 0x55f004e4a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:24.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.666+0000 7f29f1dd4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f004e3a710 0x55f004e4a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57810/0 (socket says 172.21.15.82:57810) 2023-09-09T23:18:24.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.666+0000 7f29f1dd4700 1 -- 172.21.15.82:0/3753428993 learned_addr learned my addr 172.21.15.82:0/3753428993 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:18:24.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.666+0000 7f29f15d3700 1 --2- 172.21.15.82:0/3753428993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f004e3b030 0x55f004e4a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:24.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.666+0000 7f29f25d5700 1 --2- 172.21.15.82:0/3753428993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f004e47050 0x55f004e4ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:24.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.666+0000 7f29f1dd4700 1 -- 172.21.15.82:0/3753428993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f004e47050 msgr2=0x55f004e4ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:18:24.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.666+0000 7f29f1dd4700 1 --2- 172.21.15.82:0/3753428993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f004e47050 0x55f004e4ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.666+0000 7f29f1dd4700 1 -- 172.21.15.82:0/3753428993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f004e3b030 msgr2=0x55f004e4a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:18:24.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.666+0000 7f29f1dd4700 1 --2- 172.21.15.82:0/3753428993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f004e3b030 0x55f004e4a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.666+0000 7f29f1dd4700 1 -- 172.21.15.82:0/3753428993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29dc011b10 con 0x55f004e3a710 2023-09-09T23:18:24.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.666+0000 7f29f25d5700 1 --2- 172.21.15.82:0/3753428993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f004e47050 0x55f004e4ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:18:24.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.666+0000 7f29f15d3700 1 --2- 172.21.15.82:0/3753428993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f004e3b030 0x55f004e4a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:18:24.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.666+0000 7f29f1dd4700 1 --2- 172.21.15.82:0/3753428993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f004e3a710 0x55f004e4a210 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f29dc0117d0 tx=0x7f29dc02b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cde5381bd011fd9 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:18:24.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.666+0000 7f29d57fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:18:24.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.666+0000 7f29d57fa700 1 -- 172.21.15.82:0/3753428993 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc024040 con 0x55f004e3a710 2023-09-09T23:18:24.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.666+0000 7f29d57fa700 1 -- 172.21.15.82:0/3753428993 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f29dc003d30 con 0x55f004e3a710 2023-09-09T23:18:24.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.666+0000 7f2a001feb80 1 -- 172.21.15.82:0/3753428993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29e401b320 con 0x55f004e3a710 2023-09-09T23:18:24.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.666+0000 7f29d57fa700 1 -- 172.21.15.82:0/3753428993 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29dc02b910 con 0x55f004e3a710 2023-09-09T23:18:24.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.666+0000 7f2a001feb80 10 client.16089 fetch_fsmap learned FSMap version 40 2023-09-09T23:18:24.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.666+0000 7f2a001feb80 1 -- 172.21.15.82:0/3753428993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55f004bdf250 con 0x55f004e3a710 2023-09-09T23:18:24.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.666+0000 7f29d57fa700 1 -- 172.21.15.82:0/3753428993 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29e401b320 con 0x55f004e3a710 2023-09-09T23:18:24.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.667+0000 7f29d57fa700 1 -- 172.21.15.82:0/3753428993 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f29dc032070 con 0x55f004e3a710 2023-09-09T23:18:24.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.667+0000 7f2a001feb80 10 client.16089 fetch_fsmap finished waiting for FSMap version 40 2023-09-09T23:18:24.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.667+0000 7f2a001feb80 10 client.16089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:18:24.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.667+0000 7f2a001feb80 20 client.16089 populate_metadata read hostname 'smithi082' 2023-09-09T23:18:24.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.667+0000 7f2a001feb80 1 --2- 172.21.15.82:0/3753428993 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55f004eae3e0 0x55f004eb0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:24.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.667+0000 7f2a001feb80 4 client.16089 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:18:24.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.667+0000 7f2a001feb80 1 -- 172.21.15.82:0/3753428993 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f004d85920 con 0x55f004eae3e0 2023-09-09T23:18:24.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.667+0000 7f29f15d3700 1 --2- 172.21.15.82:0/3753428993 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55f004eae3e0 0x55f004eb0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:24.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.667+0000 7f29f15d3700 1 --2- 172.21.15.82:0/3753428993 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55f004eae3e0 0x55f004eb0800 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3ceb9a762c5789c server_cookie=e12e34193c079357 in_seq=0 out_seq=0 2023-09-09T23:18:24.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.667+0000 7f29d57fa700 10 client.16089 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:18:24.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.667+0000 7f29d57fa700 1 -- 172.21.15.82:0/3753428993 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55f004d85920 con 0x55f004eae3e0 2023-09-09T23:18:24.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.667+0000 7f29d57fa700 10 client.16089 handle_command_reply: tid=0 2023-09-09T23:18:24.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.667+0000 7f2a001feb80 1 client.16089 shutdown 2023-09-09T23:18:24.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.667+0000 7f2a001feb80 1 -- 172.21.15.82:0/3753428993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f004e3a710 msgr2=0x55f004e4a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:18:24.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.668+0000 7f2a001feb80 1 --2- 172.21.15.82:0/3753428993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f004e3a710 0x55f004e4a210 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f29dc0117d0 tx=0x7f29dc02b530 comp rx=0 tx=0).stop 2023-09-09T23:18:24.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.668+0000 7f2a001feb80 1 -- 172.21.15.82:0/3753428993 shutdown_connections 2023-09-09T23:18:24.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.668+0000 7f2a001feb80 1 --2- 172.21.15.82:0/3753428993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f004e47050 0x55f004e4ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.668+0000 7f2a001feb80 1 --2- 172.21.15.82:0/3753428993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f004e3a710 0x55f004e4a210 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.668+0000 7f2a001feb80 1 --2- 172.21.15.82:0/3753428993 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55f004eae3e0 0x55f004eb0800 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.668+0000 7f2a001feb80 1 --2- 172.21.15.82:0/3753428993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f004e3b030 0x55f004e4a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.668+0000 7f29d57fa700 0 client.16089 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:18:24.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.668+0000 7f2a001feb80 1 -- 172.21.15.82:0/3753428993 >> 172.21.15.82:0/3753428993 conn(0x55f004daf090 msgr2=0x55f004e3f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:18:24.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.668+0000 7f2a001feb80 1 -- 172.21.15.82:0/3753428993 shutdown_connections 2023-09-09T23:18:24.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.668+0000 7f2a001feb80 1 -- 172.21.15.82:0/3753428993 wait complete. 2023-09-09T23:18:24.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.668+0000 7f2a001feb80 20 client.16089 trim_cache size 0 max 16384 2023-09-09T23:18:24.696 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:18:24.696 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:18:24.696 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:18:24.697 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:18:24.697 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:18:24.697 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:18:24.697 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:18:24.697 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:18:24.698 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:18:24.698 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:18:24.699 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.668+0000 7f29f4038700 1 -- 172.21.15.82:0/2363648937 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f29d80a07b0 msgr2=0x7f29d80a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:18:24.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.668+0000 7f29f4038700 1 --2- 172.21.15.82:0/2363648937 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f29d80a07b0 0x7f29d80a2c70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009be0 tx=0x7f29dc009260 comp rx=0 tx=0).stop 2023-09-09T23:18:24.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.668+0000 7f29f4038700 1 -- 172.21.15.82:0/2363648937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29ec1054e0 msgr2=0x7f29ec199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:18:24.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.668+0000 7f29f4038700 1 --2- 172.21.15.82:0/2363648937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29ec1054e0 0x7f29ec199ec0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f29e400ef30 tx=0x7f29e400c3a0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.668+0000 7f29f4038700 1 -- 172.21.15.82:0/2363648937 shutdown_connections 2023-09-09T23:18:24.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.668+0000 7f29f4038700 1 --2- 172.21.15.82:0/2363648937 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f29d80a07b0 0x7f29d80a2c70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.668+0000 7f29f4038700 1 --2- 172.21.15.82:0/2363648937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29ec1054e0 0x7f29ec199ec0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.668+0000 7f29f4038700 1 --2- 172.21.15.82:0/2363648937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29ec0ff440 0x7f29ec195520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.668+0000 7f29f4038700 1 --2- 172.21.15.82:0/2363648937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29ec0ffe00 0x7f29ec195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:24.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.668+0000 7f29f4038700 1 -- 172.21.15.82:0/2363648937 >> 172.21.15.82:0/2363648937 conn(0x7f29ec0faf50 msgr2=0x7f29ec0fb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:18:24.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.668+0000 7f29f4038700 1 -- 172.21.15.82:0/2363648937 shutdown_connections 2023-09-09T23:18:24.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:24.668+0000 7f29f4038700 1 -- 172.21.15.82:0/2363648937 wait complete. 2023-09-09T23:18:24.702 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:18:25.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:25 smithi170 ceph-mon[102832]: pgmap v138: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 290 MiB/s rd, 42 KiB/s wr, 1.50k op/s 2023-09-09T23:18:25.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:25 smithi082 ceph-mon[98694]: pgmap v138: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 290 MiB/s rd, 42 KiB/s wr, 1.50k op/s 2023-09-09T23:18:25.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:25 smithi098 ceph-mon[103124]: pgmap v138: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 290 MiB/s rd, 42 KiB/s wr, 1.50k op/s 2023-09-09T23:18:27.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:27 smithi170 ceph-mon[102832]: pgmap v139: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 290 MiB/s rd, 4.5 MiB/s wr, 1.52k op/s 2023-09-09T23:18:27.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:27 smithi082 ceph-mon[98694]: pgmap v139: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 290 MiB/s rd, 4.5 MiB/s wr, 1.52k op/s 2023-09-09T23:18:27.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:27 smithi098 ceph-mon[103124]: pgmap v139: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 290 MiB/s rd, 4.5 MiB/s wr, 1.52k op/s 2023-09-09T23:18:29.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:29 smithi170 ceph-mon[102832]: pgmap v140: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s rd, 4.5 MiB/s wr, 468 op/s 2023-09-09T23:18:29.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:29 smithi082 ceph-mon[98694]: pgmap v140: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s rd, 4.5 MiB/s wr, 468 op/s 2023-09-09T23:18:29.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:29 smithi098 ceph-mon[103124]: pgmap v140: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s rd, 4.5 MiB/s wr, 468 op/s 2023-09-09T23:18:31.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:31 smithi170 ceph-mon[102832]: pgmap v141: 97 pgs: 97 active+clean; 16 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s rd, 16 MiB/s wr, 489 op/s 2023-09-09T23:18:31.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:31 smithi082 ceph-mon[98694]: pgmap v141: 97 pgs: 97 active+clean; 16 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s rd, 16 MiB/s wr, 489 op/s 2023-09-09T23:18:31.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:31 smithi098 ceph-mon[103124]: pgmap v141: 97 pgs: 97 active+clean; 16 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s rd, 16 MiB/s wr, 489 op/s 2023-09-09T23:18:33.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:33 smithi170 ceph-mon[102832]: pgmap v142: 97 pgs: 97 active+clean; 16 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 40 MiB/s rd, 25 MiB/s wr, 266 op/s 2023-09-09T23:18:33.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:33 smithi082 ceph-mon[98694]: pgmap v142: 97 pgs: 97 active+clean; 16 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 40 MiB/s rd, 25 MiB/s wr, 266 op/s 2023-09-09T23:18:33.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:33 smithi098 ceph-mon[103124]: pgmap v142: 97 pgs: 97 active+clean; 16 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 40 MiB/s rd, 25 MiB/s wr, 266 op/s 2023-09-09T23:18:35.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:35 smithi170 ceph-mon[102832]: pgmap v143: 97 pgs: 97 active+clean; 16 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 MiB/s wr, 61 op/s 2023-09-09T23:18:35.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:35 smithi082 ceph-mon[98694]: pgmap v143: 97 pgs: 97 active+clean; 16 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 MiB/s wr, 61 op/s 2023-09-09T23:18:35.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:35 smithi098 ceph-mon[103124]: pgmap v143: 97 pgs: 97 active+clean; 16 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 MiB/s wr, 61 op/s 2023-09-09T23:18:37.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:37 smithi170 ceph-mon[102832]: pgmap v144: 97 pgs: 97 active+clean; 16 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 25 MiB/s wr, 68 op/s 2023-09-09T23:18:37.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:37 smithi082 ceph-mon[98694]: pgmap v144: 97 pgs: 97 active+clean; 16 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 25 MiB/s wr, 68 op/s 2023-09-09T23:18:37.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:37 smithi098 ceph-mon[103124]: pgmap v144: 97 pgs: 97 active+clean; 16 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 25 MiB/s wr, 68 op/s 2023-09-09T23:18:39.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:39 smithi170 ceph-mon[102832]: pgmap v145: 97 pgs: 97 active+clean; 16 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 MiB/s wr, 53 op/s 2023-09-09T23:18:39.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:39 smithi082 ceph-mon[98694]: pgmap v145: 97 pgs: 97 active+clean; 16 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 MiB/s wr, 53 op/s 2023-09-09T23:18:39.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:39 smithi098 ceph-mon[103124]: pgmap v145: 97 pgs: 97 active+clean; 16 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 MiB/s wr, 53 op/s 2023-09-09T23:18:40.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:40 smithi170 ceph-mon[102832]: pgmap v146: 97 pgs: 97 active+clean; 16 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 29 MiB/s wr, 63 op/s 2023-09-09T23:18:40.954 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:40 smithi082 ceph-mon[98694]: pgmap v146: 97 pgs: 97 active+clean; 16 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 29 MiB/s wr, 63 op/s 2023-09-09T23:18:41.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:40 smithi098 ceph-mon[103124]: pgmap v146: 97 pgs: 97 active+clean; 16 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 29 MiB/s wr, 63 op/s 2023-09-09T23:18:43.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:43 smithi170 ceph-mon[102832]: pgmap v147: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 22 KiB/s rd, 41 MiB/s wr, 51 op/s 2023-09-09T23:18:43.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:43 smithi082 ceph-mon[98694]: pgmap v147: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 22 KiB/s rd, 41 MiB/s wr, 51 op/s 2023-09-09T23:18:43.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:43 smithi098 ceph-mon[103124]: pgmap v147: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 22 KiB/s rd, 41 MiB/s wr, 51 op/s 2023-09-09T23:18:45.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:45 smithi170 ceph-mon[102832]: pgmap v148: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 24 op/s 2023-09-09T23:18:45.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:45 smithi082 ceph-mon[98694]: pgmap v148: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 24 op/s 2023-09-09T23:18:45.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:45 smithi098 ceph-mon[103124]: pgmap v148: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 24 op/s 2023-09-09T23:18:47.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:47 smithi170 ceph-mon[102832]: pgmap v149: 97 pgs: 97 active+clean; 16 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 30 KiB/s rd, 56 MiB/s wr, 39 op/s 2023-09-09T23:18:47.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:47 smithi082 ceph-mon[98694]: pgmap v149: 97 pgs: 97 active+clean; 16 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 30 KiB/s rd, 56 MiB/s wr, 39 op/s 2023-09-09T23:18:47.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:47 smithi098 ceph-mon[103124]: pgmap v149: 97 pgs: 97 active+clean; 16 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 30 KiB/s rd, 56 MiB/s wr, 39 op/s 2023-09-09T23:18:49.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:49 smithi170 ceph-mon[102832]: pgmap v150: 97 pgs: 97 active+clean; 16 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 22 KiB/s rd, 56 MiB/s wr, 32 op/s 2023-09-09T23:18:49.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:49 smithi082 ceph-mon[98694]: pgmap v150: 97 pgs: 97 active+clean; 16 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 22 KiB/s rd, 56 MiB/s wr, 32 op/s 2023-09-09T23:18:49.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:49 smithi098 ceph-mon[103124]: pgmap v150: 97 pgs: 97 active+clean; 16 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 22 KiB/s rd, 56 MiB/s wr, 32 op/s 2023-09-09T23:18:51.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:51 smithi170 ceph-mon[102832]: pgmap v151: 97 pgs: 97 active+clean; 16 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 29 KiB/s rd, 59 MiB/s wr, 38 op/s 2023-09-09T23:18:51.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:51 smithi082 ceph-mon[98694]: pgmap v151: 97 pgs: 97 active+clean; 16 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 29 KiB/s rd, 59 MiB/s wr, 38 op/s 2023-09-09T23:18:51.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:51 smithi098 ceph-mon[103124]: pgmap v151: 97 pgs: 97 active+clean; 16 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 29 KiB/s rd, 59 MiB/s wr, 38 op/s 2023-09-09T23:18:53.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:53 smithi170 ceph-mon[102832]: pgmap v152: 97 pgs: 97 active+clean; 16 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 20 KiB/s rd, 53 MiB/s wr, 32 op/s 2023-09-09T23:18:53.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:53 smithi082 ceph-mon[98694]: pgmap v152: 97 pgs: 97 active+clean; 16 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 20 KiB/s rd, 53 MiB/s wr, 32 op/s 2023-09-09T23:18:53.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:53 smithi098 ceph-mon[103124]: pgmap v152: 97 pgs: 97 active+clean; 16 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 20 KiB/s rd, 53 MiB/s wr, 32 op/s 2023-09-09T23:18:54.704 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:18:54.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.842+0000 7f9102833700 1 -- 172.21.15.82:0/1292751102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90fc0ffe00 msgr2=0x7f90fc10c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:18:54.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.842+0000 7f9102833700 1 --2- 172.21.15.82:0/1292751102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90fc0ffe00 0x7f90fc10c310 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f90f400b0a0 tx=0x7f90f400b3b0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.842+0000 7f9102833700 1 -- 172.21.15.82:0/1292751102 shutdown_connections 2023-09-09T23:18:54.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.842+0000 7f9102833700 1 --2- 172.21.15.82:0/1292751102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90fc1054e0 0x7f90fc1058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.842+0000 7f9102833700 1 --2- 172.21.15.82:0/1292751102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90fc0ffe00 0x7f90fc10c310 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.842+0000 7f9102833700 1 --2- 172.21.15.82:0/1292751102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90fc0ff440 0x7f90fc0ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.842+0000 7f9102833700 1 -- 172.21.15.82:0/1292751102 >> 172.21.15.82:0/1292751102 conn(0x7f90fc0faf50 msgr2=0x7f90fc0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:18:54.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.842+0000 7f9102833700 1 -- 172.21.15.82:0/1292751102 shutdown_connections 2023-09-09T23:18:54.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.842+0000 7f9102833700 1 -- 172.21.15.82:0/1292751102 wait complete. 2023-09-09T23:18:54.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.843+0000 7f9102833700 1 Processor -- start 2023-09-09T23:18:54.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.843+0000 7f9102833700 1 -- start start 2023-09-09T23:18:54.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.843+0000 7f9102833700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90fc0ff440 0x7f90fc195560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:54.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.843+0000 7f9102833700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90fc0ffe00 0x7f90fc195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:54.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.843+0000 7f9102833700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90fc1054e0 0x7f90fc199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:54.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.843+0000 7f9102833700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc19a4f0 con 0x7f90fc0ffe00 2023-09-09T23:18:54.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.843+0000 7f9102833700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc19a660 con 0x7f90fc1054e0 2023-09-09T23:18:54.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.843+0000 7f9102833700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc19a820 con 0x7f90fc0ff440 2023-09-09T23:18:54.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.844+0000 7f90fb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90fc0ffe00 0x7f90fc195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:54.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.844+0000 7f90fb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90fc0ffe00 0x7f90fc195aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57826/0 (socket says 172.21.15.82:57826) 2023-09-09T23:18:54.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.844+0000 7f90fb7fe700 1 -- 172.21.15.82:0/1455732567 learned_addr learned my addr 172.21.15.82:0/1455732567 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:18:54.848 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.844+0000 7f90fbfff700 1 --2- 172.21.15.82:0/1455732567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90fc0ff440 0x7f90fc195560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:54.848 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.844+0000 7f9100dd0700 1 --2- 172.21.15.82:0/1455732567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90fc1054e0 0x7f90fc199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:54.848 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.844+0000 7f90fb7fe700 1 -- 172.21.15.82:0/1455732567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90fc0ff440 msgr2=0x7f90fc195560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:18:54.848 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.844+0000 7f90fb7fe700 1 --2- 172.21.15.82:0/1455732567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90fc0ff440 0x7f90fc195560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.849 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.844+0000 7f90fb7fe700 1 -- 172.21.15.82:0/1455732567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90fc1054e0 msgr2=0x7f90fc199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:18:54.849 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.844+0000 7f90fb7fe700 1 --2- 172.21.15.82:0/1455732567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90fc1054e0 0x7f90fc199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.849 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.844+0000 7f90fb7fe700 1 -- 172.21.15.82:0/1455732567 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f4009d00 con 0x7f90fc0ffe00 2023-09-09T23:18:54.849 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.844+0000 7f90fb7fe700 1 --2- 172.21.15.82:0/1455732567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90fc0ffe00 0x7f90fc195aa0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f90f000e9a0 tx=0x7f90f000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:18:54.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.844+0000 7f90f97fa700 1 -- 172.21.15.82:0/1455732567 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f0009790 con 0x7f90fc0ffe00 2023-09-09T23:18:54.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.844+0000 7f9102833700 1 -- 172.21.15.82:0/1455732567 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90fc19ab00 con 0x7f90fc0ffe00 2023-09-09T23:18:54.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.844+0000 7f90f97fa700 1 -- 172.21.15.82:0/1455732567 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90f0010430 con 0x7f90fc0ffe00 2023-09-09T23:18:54.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.844+0000 7f90f97fa700 1 -- 172.21.15.82:0/1455732567 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f0018680 con 0x7f90fc0ffe00 2023-09-09T23:18:54.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.844+0000 7f9102833700 1 -- 172.21.15.82:0/1455732567 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90fc1a2340 con 0x7f90fc0ffe00 2023-09-09T23:18:54.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.846+0000 7f90f97fa700 1 -- 172.21.15.82:0/1455732567 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f90f00105a0 con 0x7f90fc0ffe00 2023-09-09T23:18:54.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.847+0000 7f90f97fa700 1 --2- 172.21.15.82:0/1455732567 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f90e40a07b0 0x7f90e40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:54.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.847+0000 7f90f97fa700 1 -- 172.21.15.82:0/1455732567 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f90f0014070 con 0x7f90fc0ffe00 2023-09-09T23:18:54.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.847+0000 7f90fbfff700 1 --2- 172.21.15.82:0/1455732567 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f90e40a07b0 0x7f90e40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:54.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.847+0000 7f90fbfff700 1 --2- 172.21.15.82:0/1455732567 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f90e40a07b0 0x7f90e40a2c70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f90ec009be0 tx=0x7f90ec009260 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:18:54.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.848+0000 7f910e9f9b80 1 Processor -- start 2023-09-09T23:18:54.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.848+0000 7f910e9f9b80 1 -- start start 2023-09-09T23:18:54.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.848+0000 7f910e9f9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55817f726710 0x55817f726af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:54.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.848+0000 7f910e9f9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817f727030 0x55817f730900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:54.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.848+0000 7f910e9f9b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55817f730e40 0x55817f733230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:54.860 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.848+0000 7f910e9f9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817f67f730 con 0x55817f727030 2023-09-09T23:18:54.860 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.848+0000 7f910e9f9b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817f67d050 con 0x55817f730e40 2023-09-09T23:18:54.860 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.848+0000 7f910e9f9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817f678940 con 0x55817f726710 2023-09-09T23:18:54.860 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.848+0000 7f90fb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817f727030 0x55817f730900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:54.860 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.848+0000 7f90fb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817f727030 0x55817f730900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57834/0 (socket says 172.21.15.82:57834) 2023-09-09T23:18:54.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.848+0000 7f90fb7fe700 1 -- 172.21.15.82:0/2504697166 learned_addr learned my addr 172.21.15.82:0/2504697166 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:18:54.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.848+0000 7f90fbfff700 1 --2- 172.21.15.82:0/2504697166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55817f726710 0x55817f726af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:54.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.848+0000 7f9100dd0700 1 --2- 172.21.15.82:0/2504697166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55817f730e40 0x55817f733230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:54.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.848+0000 7f90fb7fe700 1 -- 172.21.15.82:0/2504697166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55817f726710 msgr2=0x55817f726af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:18:54.862 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.848+0000 7f90fb7fe700 1 --2- 172.21.15.82:0/2504697166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55817f726710 0x55817f726af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.862 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.848+0000 7f90fb7fe700 1 -- 172.21.15.82:0/2504697166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55817f730e40 msgr2=0x55817f733230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:18:54.862 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.848+0000 7f90fb7fe700 1 --2- 172.21.15.82:0/2504697166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55817f730e40 0x55817f733230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.862 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.848+0000 7f90fb7fe700 1 -- 172.21.15.82:0/2504697166 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55817f4a4de0 con 0x55817f727030 2023-09-09T23:18:54.862 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.849+0000 7f90fb7fe700 1 --2- 172.21.15.82:0/2504697166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817f727030 0x55817f730900 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f90f00a1140 tx=0x7f90f0098f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9b7354dcf48493 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:18:54.863 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.849+0000 7f90e27fc700 1 -- 172.21.15.82:0/2504697166 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f0099600 con 0x55817f727030 2023-09-09T23:18:54.863 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.849+0000 7f90e27fc700 1 -- 172.21.15.82:0/2504697166 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90f009d430 con 0x55817f727030 2023-09-09T23:18:54.863 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.849+0000 7f90e27fc700 1 -- 172.21.15.82:0/2504697166 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f0084070 con 0x55817f727030 2023-09-09T23:18:54.863 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.849+0000 7f910e9f9b80 1 -- 172.21.15.82:0/2504697166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817f727030 msgr2=0x55817f730900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:18:54.863 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.849+0000 7f910e9f9b80 1 --2- 172.21.15.82:0/2504697166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817f727030 0x55817f730900 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f90f00a1140 tx=0x7f90f0098f60 comp rx=0 tx=0).stop 2023-09-09T23:18:54.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.849+0000 7f910e9f9b80 1 -- 172.21.15.82:0/2504697166 shutdown_connections 2023-09-09T23:18:54.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.849+0000 7f910e9f9b80 1 --2- 172.21.15.82:0/2504697166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55817f730e40 0x55817f733230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.849+0000 7f910e9f9b80 1 --2- 172.21.15.82:0/2504697166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55817f726710 0x55817f726af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.849+0000 7f910e9f9b80 1 --2- 172.21.15.82:0/2504697166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817f727030 0x55817f730900 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.849+0000 7f910e9f9b80 1 -- 172.21.15.82:0/2504697166 >> 172.21.15.82:0/2504697166 conn(0x55817f69b090 msgr2=0x55817f721bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:18:54.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.849+0000 7f910e9f9b80 1 -- 172.21.15.82:0/2504697166 shutdown_connections 2023-09-09T23:18:54.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.849+0000 7f910e9f9b80 1 -- 172.21.15.82:0/2504697166 wait complete. 2023-09-09T23:18:54.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.850+0000 7f910e9f9b80 1 -- start start 2023-09-09T23:18:54.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.850+0000 7f910e9f9b80 1 Processor -- start 2023-09-09T23:18:54.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.850+0000 7f910e9f9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55817f726710 0x55817f72db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:54.866 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.850+0000 7f910e9f9b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55817f727030 0x55817f72e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:54.866 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.850+0000 7f910e9f9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817f730e40 0x55817f72e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:54.866 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.850+0000 7f910e9f9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817f616b70 con 0x55817f730e40 2023-09-09T23:18:54.866 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.850+0000 7f910e9f9b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817f616870 con 0x55817f727030 2023-09-09T23:18:54.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.850+0000 7f910e9f9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817f4fa920 con 0x55817f726710 2023-09-09T23:18:54.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.850+0000 7f910e9f9b80 10 client.0 _ll_register_callbacks cb 0x55817f560a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:18:54.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.850+0000 7f9100dd0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817f730e40 0x55817f72e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:54.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.850+0000 7f90fbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55817f726710 0x55817f72db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:54.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.850+0000 7f9100dd0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817f730e40 0x55817f72e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57842/0 (socket says 172.21.15.82:57842) 2023-09-09T23:18:54.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.850+0000 7f9100dd0700 1 -- 172.21.15.82:0/3302705553 learned_addr learned my addr 172.21.15.82:0/3302705553 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:18:54.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.850+0000 7f90fb7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55817f727030 0x55817f72e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:54.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.851+0000 7f9100dd0700 1 -- 172.21.15.82:0/3302705553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55817f726710 msgr2=0x55817f72db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:18:54.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.851+0000 7f9100dd0700 1 --2- 172.21.15.82:0/3302705553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55817f726710 0x55817f72db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.851+0000 7f9100dd0700 1 -- 172.21.15.82:0/3302705553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55817f727030 msgr2=0x55817f72e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:18:54.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.851+0000 7f9100dd0700 1 --2- 172.21.15.82:0/3302705553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55817f727030 0x55817f72e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.851+0000 7f9100dd0700 1 -- 172.21.15.82:0/3302705553 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90fc1a2340 con 0x55817f730e40 2023-09-09T23:18:54.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.851+0000 7f90fbfff700 1 --2- 172.21.15.82:0/3302705553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55817f726710 0x55817f72db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:18:54.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.851+0000 7f9100dd0700 1 --2- 172.21.15.82:0/3302705553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817f730e40 0x55817f72e5e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f90f4015000 tx=0x7f90f4004c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e720e72925d08c11 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:18:54.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.851+0000 7f90db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:18:54.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.851+0000 7f90db7fe700 1 -- 172.21.15.82:0/3302705553 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f400e040 con 0x55817f730e40 2023-09-09T23:18:54.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.851+0000 7f90db7fe700 1 -- 172.21.15.82:0/3302705553 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f90f40076e0 con 0x55817f730e40 2023-09-09T23:18:54.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.851+0000 7f910e9f9b80 1 -- 172.21.15.82:0/3302705553 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55817f5d04f0 con 0x55817f730e40 2023-09-09T23:18:54.871 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.851+0000 7f90db7fe700 1 -- 172.21.15.82:0/3302705553 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90f40078a0 con 0x55817f730e40 2023-09-09T23:18:54.871 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.851+0000 7f910e9f9b80 10 client.16101 fetch_fsmap learned FSMap version 40 2023-09-09T23:18:54.871 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.851+0000 7f910e9f9b80 1 -- 172.21.15.82:0/3302705553 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55817f48dc60 con 0x55817f730e40 2023-09-09T23:18:54.871 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.851+0000 7f90db7fe700 1 -- 172.21.15.82:0/3302705553 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55817f5d04f0 con 0x55817f730e40 2023-09-09T23:18:54.871 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.851+0000 7f90db7fe700 1 -- 172.21.15.82:0/3302705553 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f90f4025030 con 0x55817f730e40 2023-09-09T23:18:54.871 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.851+0000 7f910e9f9b80 10 client.16101 fetch_fsmap finished waiting for FSMap version 40 2023-09-09T23:18:54.872 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.851+0000 7f910e9f9b80 10 client.16101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:18:54.872 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.852+0000 7f910e9f9b80 20 client.16101 populate_metadata read hostname 'smithi082' 2023-09-09T23:18:54.872 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.852+0000 7f910e9f9b80 1 --2- 172.21.15.82:0/3302705553 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55817f740a20 0x55817f742e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:54.872 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.852+0000 7f910e9f9b80 4 client.16101 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:18:54.873 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.852+0000 7f910e9f9b80 1 -- 172.21.15.82:0/3302705553 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55817f68d8a0 con 0x55817f740a20 2023-09-09T23:18:54.873 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.852+0000 7f90fbfff700 1 --2- 172.21.15.82:0/3302705553 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55817f740a20 0x55817f742e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:54.873 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.852+0000 7f90db7fe700 10 client.16101 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:18:54.873 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.852+0000 7f90fbfff700 1 --2- 172.21.15.82:0/3302705553 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55817f740a20 0x55817f742e40 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6de67aa8b59f5bc server_cookie=721f3146f340a65 in_seq=0 out_seq=0 2023-09-09T23:18:54.873 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.853+0000 7f90db7fe700 1 -- 172.21.15.82:0/3302705553 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55817f68d8a0 con 0x55817f740a20 2023-09-09T23:18:54.874 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.853+0000 7f90db7fe700 10 client.16101 handle_command_reply: tid=0 2023-09-09T23:18:54.874 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.853+0000 7f910e9f9b80 1 client.16101 shutdown 2023-09-09T23:18:54.874 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.853+0000 7f910e9f9b80 1 -- 172.21.15.82:0/3302705553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817f730e40 msgr2=0x55817f72e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:18:54.874 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.853+0000 7f910e9f9b80 1 --2- 172.21.15.82:0/3302705553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817f730e40 0x55817f72e5e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f90f4015000 tx=0x7f90f4004c40 comp rx=0 tx=0).stop 2023-09-09T23:18:54.875 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.853+0000 7f910e9f9b80 1 -- 172.21.15.82:0/3302705553 shutdown_connections 2023-09-09T23:18:54.875 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.853+0000 7f910e9f9b80 1 --2- 172.21.15.82:0/3302705553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817f730e40 0x55817f72e5e0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.875 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.853+0000 7f910e9f9b80 1 --2- 172.21.15.82:0/3302705553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55817f726710 0x55817f72db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.875 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.853+0000 7f910e9f9b80 1 --2- 172.21.15.82:0/3302705553 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55817f740a20 0x55817f742e40 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.875 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.853+0000 7f910e9f9b80 1 --2- 172.21.15.82:0/3302705553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55817f727030 0x55817f72e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.875 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.853+0000 7f910e9f9b80 1 -- 172.21.15.82:0/3302705553 >> 172.21.15.82:0/3302705553 conn(0x55817f69b090 msgr2=0x55817f721780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:18:54.876 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.853+0000 7f90db7fe700 0 client.16101 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:18:54.876 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.853+0000 7f910e9f9b80 1 -- 172.21.15.82:0/3302705553 shutdown_connections 2023-09-09T23:18:54.876 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.853+0000 7f910e9f9b80 1 -- 172.21.15.82:0/3302705553 wait complete. 2023-09-09T23:18:54.876 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.853+0000 7f910e9f9b80 20 client.16101 trim_cache size 0 max 16384 2023-09-09T23:18:54.882 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.870+0000 7f910e9f9b80 1 Processor -- start 2023-09-09T23:18:54.882 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.870+0000 7f910e9f9b80 1 -- start start 2023-09-09T23:18:54.882 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.870+0000 7f910e9f9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817f726710 0x55817f737990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:54.882 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.870+0000 7f910e9f9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55817f727030 0x55817f72add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:54.883 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.870+0000 7f910e9f9b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55817f730e40 0x55817f72b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:54.883 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.870+0000 7f910e9f9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817f7209a0 con 0x55817f726710 2023-09-09T23:18:54.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.870+0000 7f910e9f9b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817f726e20 con 0x55817f730e40 2023-09-09T23:18:54.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.870+0000 7f910e9f9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817f7434a0 con 0x55817f727030 2023-09-09T23:18:54.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.870+0000 7f90fbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817f726710 0x55817f737990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:54.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.870+0000 7f90fbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817f726710 0x55817f737990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57846/0 (socket says 172.21.15.82:57846) 2023-09-09T23:18:54.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.870+0000 7f90fbfff700 1 -- 172.21.15.82:0/2004385737 learned_addr learned my addr 172.21.15.82:0/2004385737 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:18:54.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.870+0000 7f90fb7fe700 1 --2- 172.21.15.82:0/2004385737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55817f727030 0x55817f72add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:54.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.870+0000 7f9100dd0700 1 --2- 172.21.15.82:0/2004385737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55817f730e40 0x55817f72b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:54.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.870+0000 7f90fbfff700 1 -- 172.21.15.82:0/2004385737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55817f727030 msgr2=0x55817f72add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:18:54.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.870+0000 7f90fbfff700 1 --2- 172.21.15.82:0/2004385737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55817f727030 0x55817f72add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.870+0000 7f90fbfff700 1 -- 172.21.15.82:0/2004385737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55817f730e40 msgr2=0x55817f72b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:18:54.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.870+0000 7f90fbfff700 1 --2- 172.21.15.82:0/2004385737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55817f730e40 0x55817f72b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.870+0000 7f90fbfff700 1 -- 172.21.15.82:0/2004385737 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55817f4cb250 con 0x55817f726710 2023-09-09T23:18:54.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.870+0000 7f9100dd0700 1 --2- 172.21.15.82:0/2004385737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55817f730e40 0x55817f72b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:18:54.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.870+0000 7f90fbfff700 1 --2- 172.21.15.82:0/2004385737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817f726710 0x55817f737990 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f90ec005bf0 tx=0x7f90ec014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b9e5362a078a87 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:18:54.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.870+0000 7f90e2ffd700 1 -- 172.21.15.82:0/2004385737 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec024040 con 0x55817f726710 2023-09-09T23:18:54.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.870+0000 7f90e2ffd700 1 -- 172.21.15.82:0/2004385737 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90ec0293b0 con 0x55817f726710 2023-09-09T23:18:54.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.870+0000 7f90e2ffd700 1 -- 172.21.15.82:0/2004385737 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec01e680 con 0x55817f726710 2023-09-09T23:18:54.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.871+0000 7f910e9f9b80 1 -- 172.21.15.82:0/2004385737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817f726710 msgr2=0x55817f737990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:18:54.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.871+0000 7f910e9f9b80 1 --2- 172.21.15.82:0/2004385737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817f726710 0x55817f737990 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f90ec005bf0 tx=0x7f90ec014000 comp rx=0 tx=0).stop 2023-09-09T23:18:54.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.871+0000 7f910e9f9b80 1 -- 172.21.15.82:0/2004385737 shutdown_connections 2023-09-09T23:18:54.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.871+0000 7f910e9f9b80 1 --2- 172.21.15.82:0/2004385737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55817f730e40 0x55817f72b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.871+0000 7f910e9f9b80 1 --2- 172.21.15.82:0/2004385737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817f726710 0x55817f737990 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.871+0000 7f910e9f9b80 1 --2- 172.21.15.82:0/2004385737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55817f727030 0x55817f72add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.871+0000 7f910e9f9b80 1 -- 172.21.15.82:0/2004385737 >> 172.21.15.82:0/2004385737 conn(0x55817f69b090 msgr2=0x55817f72e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:18:54.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.871+0000 7f910e9f9b80 1 -- 172.21.15.82:0/2004385737 shutdown_connections 2023-09-09T23:18:54.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.871+0000 7f910e9f9b80 1 -- 172.21.15.82:0/2004385737 wait complete. 2023-09-09T23:18:54.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.871+0000 7f910e9f9b80 1 -- start start 2023-09-09T23:18:54.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.871+0000 7f910e9f9b80 1 Processor -- start 2023-09-09T23:18:54.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.872+0000 7f910e9f9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817f726710 0x55817f78d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:54.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.872+0000 7f910e9f9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55817f727030 0x55817f78d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:54.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.872+0000 7f910e9f9b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55817f730e40 0x55817f7887c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:54.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.872+0000 7f910e9f9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817f788ed0 con 0x55817f726710 2023-09-09T23:18:54.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.872+0000 7f910e9f9b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817f789040 con 0x55817f730e40 2023-09-09T23:18:54.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.872+0000 7f910e9f9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817f789200 con 0x55817f727030 2023-09-09T23:18:54.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.872+0000 7f910e9f9b80 10 client.0 _ll_register_callbacks cb 0x55817f560a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:18:54.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.872+0000 7f90fbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817f726710 0x55817f78d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:54.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.872+0000 7f90fb7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55817f727030 0x55817f78d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:54.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.872+0000 7f90fbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817f726710 0x55817f78d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57852/0 (socket says 172.21.15.82:57852) 2023-09-09T23:18:54.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.872+0000 7f90fbfff700 1 -- 172.21.15.82:0/590416915 learned_addr learned my addr 172.21.15.82:0/590416915 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:18:54.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.872+0000 7f9100dd0700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55817f730e40 0x55817f7887c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:54.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.872+0000 7f90fbfff700 1 -- 172.21.15.82:0/590416915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55817f727030 msgr2=0x55817f78d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:18:54.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.872+0000 7f90fbfff700 1 --2- 172.21.15.82:0/590416915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55817f727030 0x55817f78d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.872+0000 7f90fbfff700 1 -- 172.21.15.82:0/590416915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55817f730e40 msgr2=0x55817f7887c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:18:54.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.872+0000 7f90fbfff700 1 --2- 172.21.15.82:0/590416915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55817f730e40 0x55817f7887c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.872+0000 7f90fbfff700 1 -- 172.21.15.82:0/590416915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90ec017070 con 0x55817f726710 2023-09-09T23:18:54.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.872+0000 7f90fb7fe700 1 --2- 172.21.15.82:0/590416915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55817f727030 0x55817f78d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:18:54.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.872+0000 7f90fbfff700 1 --2- 172.21.15.82:0/590416915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817f726710 0x55817f78d120 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f90ec005950 tx=0x7f90ec009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59506f2afa2485a4 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:18:54.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.872+0000 7f90db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:18:54.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.872+0000 7f90db7fe700 1 -- 172.21.15.82:0/590416915 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec024030 con 0x55817f726710 2023-09-09T23:18:54.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.872+0000 7f90db7fe700 1 -- 172.21.15.82:0/590416915 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f90ec0299c0 con 0x55817f726710 2023-09-09T23:18:54.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.872+0000 7f910e9f9b80 1 -- 172.21.15.82:0/590416915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f90f009d430 con 0x55817f726710 2023-09-09T23:18:54.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.873+0000 7f90db7fe700 1 -- 172.21.15.82:0/590416915 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90ec005040 con 0x55817f726710 2023-09-09T23:18:54.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.873+0000 7f910e9f9b80 10 client.16107 fetch_fsmap learned FSMap version 40 2023-09-09T23:18:54.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.873+0000 7f910e9f9b80 1 -- 172.21.15.82:0/590416915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55817f4a32b0 con 0x55817f726710 2023-09-09T23:18:54.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.873+0000 7f90db7fe700 1 -- 172.21.15.82:0/590416915 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90f009d430 con 0x55817f726710 2023-09-09T23:18:54.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.873+0000 7f90db7fe700 1 -- 172.21.15.82:0/590416915 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f90ec043410 con 0x55817f726710 2023-09-09T23:18:54.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.873+0000 7f910e9f9b80 10 client.16107 fetch_fsmap finished waiting for FSMap version 40 2023-09-09T23:18:54.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.873+0000 7f910e9f9b80 10 client.16107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:18:54.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.873+0000 7f910e9f9b80 20 client.16107 populate_metadata read hostname 'smithi082' 2023-09-09T23:18:54.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.873+0000 7f910e9f9b80 1 --2- 172.21.15.82:0/590416915 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55817f796020 0x55817f798440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:18:54.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.873+0000 7f910e9f9b80 4 client.16107 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:18:54.897 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.873+0000 7f910e9f9b80 1 -- 172.21.15.82:0/590416915 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55817f671920 con 0x55817f796020 2023-09-09T23:18:54.897 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.873+0000 7f90fb7fe700 1 --2- 172.21.15.82:0/590416915 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55817f796020 0x55817f798440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:18:54.897 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.873+0000 7f90db7fe700 10 client.16107 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:18:54.897 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.873+0000 7f90fb7fe700 1 --2- 172.21.15.82:0/590416915 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55817f796020 0x55817f798440 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=346ef5b1efcdd489 server_cookie=c47fdb41ba2b3f7b in_seq=0 out_seq=0 2023-09-09T23:18:54.897 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.874+0000 7f90db7fe700 1 -- 172.21.15.82:0/590416915 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55817f671920 con 0x55817f796020 2023-09-09T23:18:54.898 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.874+0000 7f90db7fe700 10 client.16107 handle_command_reply: tid=0 2023-09-09T23:18:54.898 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.874+0000 7f910e9f9b80 1 client.16107 shutdown 2023-09-09T23:18:54.898 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.874+0000 7f910e9f9b80 1 -- 172.21.15.82:0/590416915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817f726710 msgr2=0x55817f78d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:18:54.898 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.874+0000 7f910e9f9b80 1 --2- 172.21.15.82:0/590416915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817f726710 0x55817f78d120 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f90ec005950 tx=0x7f90ec009040 comp rx=0 tx=0).stop 2023-09-09T23:18:54.898 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.874+0000 7f910e9f9b80 1 -- 172.21.15.82:0/590416915 shutdown_connections 2023-09-09T23:18:54.899 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.874+0000 7f910e9f9b80 1 --2- 172.21.15.82:0/590416915 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55817f796020 0x55817f798440 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.899 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.874+0000 7f910e9f9b80 1 --2- 172.21.15.82:0/590416915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55817f730e40 0x55817f7887c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.899 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.874+0000 7f90db7fe700 0 client.16107 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:18:54.899 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.874+0000 7f910e9f9b80 1 --2- 172.21.15.82:0/590416915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817f726710 0x55817f78d120 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.899 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.874+0000 7f910e9f9b80 1 --2- 172.21.15.82:0/590416915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55817f727030 0x55817f78d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.900 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.874+0000 7f910e9f9b80 1 -- 172.21.15.82:0/590416915 >> 172.21.15.82:0/590416915 conn(0x55817f69b090 msgr2=0x55817f725700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:18:54.900 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.874+0000 7f910e9f9b80 1 -- 172.21.15.82:0/590416915 shutdown_connections 2023-09-09T23:18:54.900 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.874+0000 7f910e9f9b80 1 -- 172.21.15.82:0/590416915 wait complete. 2023-09-09T23:18:54.900 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.874+0000 7f910e9f9b80 20 client.16107 trim_cache size 0 max 16384 2023-09-09T23:18:54.901 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:18:54.901 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:18:54.901 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:18:54.901 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:18:54.902 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:18:54.902 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:18:54.902 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:18:54.902 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:18:54.902 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:18:54.903 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:18:54.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.874+0000 7f9102833700 1 -- 172.21.15.82:0/1455732567 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f90e40a07b0 msgr2=0x7f90e40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:18:54.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.874+0000 7f9102833700 1 --2- 172.21.15.82:0/1455732567 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f90e40a07b0 0x7f90e40a2c70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f90ec009be0 tx=0x7f90ec009260 comp rx=0 tx=0).stop 2023-09-09T23:18:54.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.874+0000 7f9102833700 1 -- 172.21.15.82:0/1455732567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90fc0ffe00 msgr2=0x7f90fc195aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:18:54.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.874+0000 7f9102833700 1 --2- 172.21.15.82:0/1455732567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90fc0ffe00 0x7f90fc195aa0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f90f000e9a0 tx=0x7f90f000ecb0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.875+0000 7f9102833700 1 -- 172.21.15.82:0/1455732567 shutdown_connections 2023-09-09T23:18:54.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.875+0000 7f9102833700 1 --2- 172.21.15.82:0/1455732567 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f90e40a07b0 0x7f90e40a2c70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.875+0000 7f9102833700 1 --2- 172.21.15.82:0/1455732567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90fc1054e0 0x7f90fc199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.875+0000 7f9102833700 1 --2- 172.21.15.82:0/1455732567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90fc0ff440 0x7f90fc195560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.875+0000 7f9102833700 1 --2- 172.21.15.82:0/1455732567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90fc0ffe00 0x7f90fc195aa0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:18:54.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.875+0000 7f9102833700 1 -- 172.21.15.82:0/1455732567 >> 172.21.15.82:0/1455732567 conn(0x7f90fc0faf50 msgr2=0x7f90fc0fb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:18:54.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.875+0000 7f9102833700 1 -- 172.21.15.82:0/1455732567 shutdown_connections 2023-09-09T23:18:54.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:18:54.875+0000 7f9102833700 1 -- 172.21.15.82:0/1455732567 wait complete. 2023-09-09T23:18:54.906 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:18:55.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:55 smithi170 ceph-mon[102832]: pgmap v153: 97 pgs: 97 active+clean; 16 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 19 KiB/s rd, 30 MiB/s wr, 24 op/s 2023-09-09T23:18:55.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:55 smithi082 ceph-mon[98694]: pgmap v153: 97 pgs: 97 active+clean; 16 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 19 KiB/s rd, 30 MiB/s wr, 24 op/s 2023-09-09T23:18:55.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:55 smithi098 ceph-mon[103124]: pgmap v153: 97 pgs: 97 active+clean; 16 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 19 KiB/s rd, 30 MiB/s wr, 24 op/s 2023-09-09T23:18:57.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:57 smithi170 ceph-mon[102832]: pgmap v154: 97 pgs: 97 active+clean; 17 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 31 KiB/s rd, 32 MiB/s wr, 31 op/s 2023-09-09T23:18:57.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:57 smithi082 ceph-mon[98694]: pgmap v154: 97 pgs: 97 active+clean; 17 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 31 KiB/s rd, 32 MiB/s wr, 31 op/s 2023-09-09T23:18:57.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:57 smithi098 ceph-mon[103124]: pgmap v154: 97 pgs: 97 active+clean; 17 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 31 KiB/s rd, 32 MiB/s wr, 31 op/s 2023-09-09T23:18:59.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:18:59 smithi170 ceph-mon[102832]: pgmap v155: 97 pgs: 97 active+clean; 17 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 22 KiB/s rd, 7.4 MiB/s wr, 17 op/s 2023-09-09T23:18:59.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:18:59 smithi082 ceph-mon[98694]: pgmap v155: 97 pgs: 97 active+clean; 17 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 22 KiB/s rd, 7.4 MiB/s wr, 17 op/s 2023-09-09T23:18:59.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:18:59 smithi098 ceph-mon[103124]: pgmap v155: 97 pgs: 97 active+clean; 17 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 22 KiB/s rd, 7.4 MiB/s wr, 17 op/s 2023-09-09T23:19:01.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:01 smithi170 ceph-mon[102832]: pgmap v156: 97 pgs: 97 active+clean; 17 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 27 KiB/s rd, 8.7 MiB/s wr, 23 op/s 2023-09-09T23:19:01.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:01 smithi082 ceph-mon[98694]: pgmap v156: 97 pgs: 97 active+clean; 17 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 27 KiB/s rd, 8.7 MiB/s wr, 23 op/s 2023-09-09T23:19:01.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:01 smithi098 ceph-mon[103124]: pgmap v156: 97 pgs: 97 active+clean; 17 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 27 KiB/s rd, 8.7 MiB/s wr, 23 op/s 2023-09-09T23:19:03.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:03 smithi170 ceph-mon[102832]: pgmap v157: 97 pgs: 97 active+clean; 17 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 22 KiB/s rd, 6.7 MiB/s wr, 19 op/s 2023-09-09T23:19:03.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:03 smithi082 ceph-mon[98694]: pgmap v157: 97 pgs: 97 active+clean; 17 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 22 KiB/s rd, 6.7 MiB/s wr, 19 op/s 2023-09-09T23:19:03.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:03 smithi098 ceph-mon[103124]: pgmap v157: 97 pgs: 97 active+clean; 17 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 22 KiB/s rd, 6.7 MiB/s wr, 19 op/s 2023-09-09T23:19:04.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:19:04.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:19:04.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:19:05.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:05 smithi170 ceph-mon[102832]: pgmap v158: 97 pgs: 97 active+clean; 17 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 19 KiB/s rd, 4.0 MiB/s wr, 16 op/s 2023-09-09T23:19:05.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:19:05.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:19:05.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:19:05.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:05 smithi082 ceph-mon[98694]: pgmap v158: 97 pgs: 97 active+clean; 17 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 19 KiB/s rd, 4.0 MiB/s wr, 16 op/s 2023-09-09T23:19:05.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:19:05.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:19:05.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:19:05.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:05 smithi098 ceph-mon[103124]: pgmap v158: 97 pgs: 97 active+clean; 17 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 19 KiB/s rd, 4.0 MiB/s wr, 16 op/s 2023-09-09T23:19:05.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:19:05.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:19:05.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:19:07.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:07 smithi170 ceph-mon[102832]: pgmap v159: 97 pgs: 97 active+clean; 17 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 37 KiB/s rd, 4.1 MiB/s wr, 33 op/s 2023-09-09T23:19:07.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:07 smithi082 ceph-mon[98694]: pgmap v159: 97 pgs: 97 active+clean; 17 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 37 KiB/s rd, 4.1 MiB/s wr, 33 op/s 2023-09-09T23:19:07.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:07 smithi098 ceph-mon[103124]: pgmap v159: 97 pgs: 97 active+clean; 17 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 37 KiB/s rd, 4.1 MiB/s wr, 33 op/s 2023-09-09T23:19:09.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:09 smithi170 ceph-mon[102832]: pgmap v160: 97 pgs: 97 active+clean; 17 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 24 KiB/s rd, 2.0 MiB/s wr, 26 op/s 2023-09-09T23:19:09.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:09 smithi082 ceph-mon[98694]: pgmap v160: 97 pgs: 97 active+clean; 17 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 24 KiB/s rd, 2.0 MiB/s wr, 26 op/s 2023-09-09T23:19:09.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:09 smithi098 ceph-mon[103124]: pgmap v160: 97 pgs: 97 active+clean; 17 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 24 KiB/s rd, 2.0 MiB/s wr, 26 op/s 2023-09-09T23:19:11.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:11 smithi170 ceph-mon[102832]: pgmap v161: 97 pgs: 97 active+clean; 17 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 34 KiB/s rd, 4.0 MiB/s wr, 41 op/s 2023-09-09T23:19:11.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:11 smithi082 ceph-mon[98694]: pgmap v161: 97 pgs: 97 active+clean; 17 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 34 KiB/s rd, 4.0 MiB/s wr, 41 op/s 2023-09-09T23:19:11.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:11 smithi098 ceph-mon[103124]: pgmap v161: 97 pgs: 97 active+clean; 17 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 34 KiB/s rd, 4.0 MiB/s wr, 41 op/s 2023-09-09T23:19:13.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:13 smithi170 ceph-mon[102832]: pgmap v162: 97 pgs: 97 active+clean; 17 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 31 KiB/s rd, 8.0 MiB/s wr, 57 op/s 2023-09-09T23:19:13.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:13 smithi082 ceph-mon[98694]: pgmap v162: 97 pgs: 97 active+clean; 17 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 31 KiB/s rd, 8.0 MiB/s wr, 57 op/s 2023-09-09T23:19:13.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:13 smithi098 ceph-mon[103124]: pgmap v162: 97 pgs: 97 active+clean; 17 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 31 KiB/s rd, 8.0 MiB/s wr, 57 op/s 2023-09-09T23:19:14.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:14 smithi170 ceph-mon[102832]: 6 slow requests, 5 included below; oldest blocked for > 180.464866 secs 2023-09-09T23:19:14.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:14 smithi170 ceph-mon[102832]: slow request 180.422979 seconds old, received at 2023-09-09T23:16:13.394392+0000: client_request(mds.1:1281 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:14.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:14 smithi170 ceph-mon[102832]: slow request 180.363439 seconds old, received at 2023-09-09T23:16:13.453931+0000: client_request(mds.1:1313 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:14.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:14 smithi170 ceph-mon[102832]: slow request 180.305082 seconds old, received at 2023-09-09T23:16:13.512289+0000: client_request(mds.1:1345 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:14.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:14 smithi170 ceph-mon[102832]: slow request 180.245117 seconds old, received at 2023-09-09T23:16:13.572254+0000: client_request(mds.1:1377 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:14.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:14 smithi170 ceph-mon[102832]: slow request 180.188742 seconds old, received at 2023-09-09T23:16:13.628629+0000: client_request(mds.1:1409 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:14.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:14 smithi082 ceph-mon[98694]: 6 slow requests, 5 included below; oldest blocked for > 180.464866 secs 2023-09-09T23:19:14.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:14 smithi082 ceph-mon[98694]: slow request 180.422979 seconds old, received at 2023-09-09T23:16:13.394392+0000: client_request(mds.1:1281 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:14.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:14 smithi082 ceph-mon[98694]: slow request 180.363439 seconds old, received at 2023-09-09T23:16:13.453931+0000: client_request(mds.1:1313 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:14.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:14 smithi082 ceph-mon[98694]: slow request 180.305082 seconds old, received at 2023-09-09T23:16:13.512289+0000: client_request(mds.1:1345 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:14.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:14 smithi082 ceph-mon[98694]: slow request 180.245117 seconds old, received at 2023-09-09T23:16:13.572254+0000: client_request(mds.1:1377 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:14.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:14 smithi082 ceph-mon[98694]: slow request 180.188742 seconds old, received at 2023-09-09T23:16:13.628629+0000: client_request(mds.1:1409 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:14.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:14 smithi098 ceph-mon[103124]: 6 slow requests, 5 included below; oldest blocked for > 180.464866 secs 2023-09-09T23:19:14.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:14 smithi098 ceph-mon[103124]: slow request 180.422979 seconds old, received at 2023-09-09T23:16:13.394392+0000: client_request(mds.1:1281 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:14.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:14 smithi098 ceph-mon[103124]: slow request 180.363439 seconds old, received at 2023-09-09T23:16:13.453931+0000: client_request(mds.1:1313 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:14.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:14 smithi098 ceph-mon[103124]: slow request 180.305082 seconds old, received at 2023-09-09T23:16:13.512289+0000: client_request(mds.1:1345 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:14.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:14 smithi098 ceph-mon[103124]: slow request 180.245117 seconds old, received at 2023-09-09T23:16:13.572254+0000: client_request(mds.1:1377 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:14.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:14 smithi098 ceph-mon[103124]: slow request 180.188742 seconds old, received at 2023-09-09T23:16:13.628629+0000: client_request(mds.1:1409 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:15.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:15 smithi170 ceph-mon[102832]: pgmap v163: 97 pgs: 97 active+clean; 17 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 30 KiB/s rd, 7.4 MiB/s wr, 54 op/s 2023-09-09T23:19:15.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:15 smithi082 ceph-mon[98694]: pgmap v163: 97 pgs: 97 active+clean; 17 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 30 KiB/s rd, 7.4 MiB/s wr, 54 op/s 2023-09-09T23:19:15.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:15 smithi098 ceph-mon[103124]: pgmap v163: 97 pgs: 97 active+clean; 17 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 30 KiB/s rd, 7.4 MiB/s wr, 54 op/s 2023-09-09T23:19:17.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:17 smithi170 ceph-mon[102832]: pgmap v164: 97 pgs: 97 active+clean; 17 GiB data, 51 GiB used, 1021 GiB / 1.0 TiB avail; 41 KiB/s rd, 37 MiB/s wr, 73 op/s 2023-09-09T23:19:17.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:17 smithi082 ceph-mon[98694]: pgmap v164: 97 pgs: 97 active+clean; 17 GiB data, 51 GiB used, 1021 GiB / 1.0 TiB avail; 41 KiB/s rd, 37 MiB/s wr, 73 op/s 2023-09-09T23:19:17.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:17 smithi098 ceph-mon[103124]: pgmap v164: 97 pgs: 97 active+clean; 17 GiB data, 51 GiB used, 1021 GiB / 1.0 TiB avail; 41 KiB/s rd, 37 MiB/s wr, 73 op/s 2023-09-09T23:19:19.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:19 smithi170 ceph-mon[102832]: pgmap v165: 97 pgs: 97 active+clean; 17 GiB data, 51 GiB used, 1021 GiB / 1.0 TiB avail; 23 KiB/s rd, 37 MiB/s wr, 55 op/s 2023-09-09T23:19:19.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:19 smithi170 ceph-mon[102832]: Health check failed: 1 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:19:19.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:19 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:19:19.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:19 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:19.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:19 smithi082 ceph-mon[98694]: pgmap v165: 97 pgs: 97 active+clean; 17 GiB data, 51 GiB used, 1021 GiB / 1.0 TiB avail; 23 KiB/s rd, 37 MiB/s wr, 55 op/s 2023-09-09T23:19:19.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:19 smithi082 ceph-mon[98694]: Health check failed: 1 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:19:19.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:19 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:19:19.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:19 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:19.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:19 smithi098 ceph-mon[103124]: pgmap v165: 97 pgs: 97 active+clean; 17 GiB data, 51 GiB used, 1021 GiB / 1.0 TiB avail; 23 KiB/s rd, 37 MiB/s wr, 55 op/s 2023-09-09T23:19:19.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:19 smithi098 ceph-mon[103124]: Health check failed: 1 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:19:19.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:19 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:19:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:19 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:20.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:20 smithi170 ceph-mon[102832]: 295 slow requests, 5 included below; oldest blocked for > 185.559258 secs 2023-09-09T23:19:20.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:20 smithi170 ceph-mon[102832]: slow request 185.238423 seconds old, received at 2023-09-09T23:16:13.687666+0000: client_request(mds.1:1441 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:19:20.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:20 smithi170 ceph-mon[102832]: slow request 185.074699 seconds old, received at 2023-09-09T23:16:13.851390+0000: client_request(mds.1:1473 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:19:20.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:20 smithi170 ceph-mon[102832]: slow request 185.014588 seconds old, received at 2023-09-09T23:16:13.911501+0000: client_request(mds.1:1505 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:19:20.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:20 smithi170 ceph-mon[102832]: slow request 184.959068 seconds old, received at 2023-09-09T23:16:13.967021+0000: client_request(mds.1:1537 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:19:20.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:20 smithi170 ceph-mon[102832]: slow request 184.898565 seconds old, received at 2023-09-09T23:16:14.027524+0000: client_request(mds.1:1569 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:19:20.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:20 smithi082 ceph-mon[98694]: 295 slow requests, 5 included below; oldest blocked for > 185.559258 secs 2023-09-09T23:19:20.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:20 smithi082 ceph-mon[98694]: slow request 185.238423 seconds old, received at 2023-09-09T23:16:13.687666+0000: client_request(mds.1:1441 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:19:20.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:20 smithi082 ceph-mon[98694]: slow request 185.074699 seconds old, received at 2023-09-09T23:16:13.851390+0000: client_request(mds.1:1473 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:19:20.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:20 smithi082 ceph-mon[98694]: slow request 185.014588 seconds old, received at 2023-09-09T23:16:13.911501+0000: client_request(mds.1:1505 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:19:20.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:20 smithi082 ceph-mon[98694]: slow request 184.959068 seconds old, received at 2023-09-09T23:16:13.967021+0000: client_request(mds.1:1537 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:19:20.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:20 smithi082 ceph-mon[98694]: slow request 184.898565 seconds old, received at 2023-09-09T23:16:14.027524+0000: client_request(mds.1:1569 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:19:20.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:20 smithi098 ceph-mon[103124]: 295 slow requests, 5 included below; oldest blocked for > 185.559258 secs 2023-09-09T23:19:20.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:20 smithi098 ceph-mon[103124]: slow request 185.238423 seconds old, received at 2023-09-09T23:16:13.687666+0000: client_request(mds.1:1441 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:19:20.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:20 smithi098 ceph-mon[103124]: slow request 185.074699 seconds old, received at 2023-09-09T23:16:13.851390+0000: client_request(mds.1:1473 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:19:20.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:20 smithi098 ceph-mon[103124]: slow request 185.014588 seconds old, received at 2023-09-09T23:16:13.911501+0000: client_request(mds.1:1505 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:19:20.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:20 smithi098 ceph-mon[103124]: slow request 184.959068 seconds old, received at 2023-09-09T23:16:13.967021+0000: client_request(mds.1:1537 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:19:20.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:20 smithi098 ceph-mon[103124]: slow request 184.898565 seconds old, received at 2023-09-09T23:16:14.027524+0000: client_request(mds.1:1569 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:19:21.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:21 smithi170 ceph-mon[102832]: pgmap v166: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 34 KiB/s rd, 37 MiB/s wr, 61 op/s 2023-09-09T23:19:21.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:21 smithi082 ceph-mon[98694]: pgmap v166: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 34 KiB/s rd, 37 MiB/s wr, 61 op/s 2023-09-09T23:19:21.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:21 smithi098 ceph-mon[103124]: pgmap v166: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 34 KiB/s rd, 37 MiB/s wr, 61 op/s 2023-09-09T23:19:23.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:23 smithi170 ceph-mon[102832]: pgmap v167: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 27 KiB/s rd, 36 MiB/s wr, 49 op/s 2023-09-09T23:19:23.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:19:23.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:23.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:23 smithi082 ceph-mon[98694]: pgmap v167: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 27 KiB/s rd, 36 MiB/s wr, 49 op/s 2023-09-09T23:19:23.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:19:23.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:23.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:23 smithi098 ceph-mon[103124]: pgmap v167: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 27 KiB/s rd, 36 MiB/s wr, 49 op/s 2023-09-09T23:19:23.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:19:23.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:24.908 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:19:25.063 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.064+0000 7f5c78897700 1 -- 172.21.15.82:0/2553393011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7410ab90 msgr2=0x7f5c7410d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:19:25.063 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.064+0000 7f5c78897700 1 --2- 172.21.15.82:0/2553393011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7410ab90 0x7f5c7410d000 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00b0a0 tx=0x7f5c6c00b3b0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.063 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.064+0000 7f5c78897700 1 -- 172.21.15.82:0/2553393011 shutdown_connections 2023-09-09T23:19:25.063 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.064+0000 7f5c78897700 1 --2- 172.21.15.82:0/2553393011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c741017c0 0x7f5c74101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.064+0000 7f5c78897700 1 --2- 172.21.15.82:0/2553393011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7410ab90 0x7f5c7410d000 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.064+0000 7f5c78897700 1 --2- 172.21.15.82:0/2553393011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c741020e0 0x7f5c7410a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.064+0000 7f5c78897700 1 -- 172.21.15.82:0/2553393011 >> 172.21.15.82:0/2553393011 conn(0x7f5c740faf60 msgr2=0x7f5c740fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:19:25.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.064+0000 7f5c78897700 1 -- 172.21.15.82:0/2553393011 shutdown_connections 2023-09-09T23:19:25.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.064+0000 7f5c78897700 1 -- 172.21.15.82:0/2553393011 wait complete. 2023-09-09T23:19:25.066 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.065+0000 7f5c78897700 1 Processor -- start 2023-09-09T23:19:25.067 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.065+0000 7f5c78897700 1 -- start start 2023-09-09T23:19:25.067 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.065+0000 7f5c78897700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c741017c0 0x7f5c7419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:25.067 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.065+0000 7f5c78897700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c741020e0 0x7f5c7419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:25.067 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.065+0000 7f5c78897700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7410ab90 0x7f5c741966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:25.067 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.065+0000 7f5c78897700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c74196d20 con 0x7f5c7410ab90 2023-09-09T23:19:25.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.065+0000 7f5c78897700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c74196e90 con 0x7f5c741020e0 2023-09-09T23:19:25.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.065+0000 7f5c78897700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c74197050 con 0x7f5c741017c0 2023-09-09T23:19:25.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.065+0000 7f5c72d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7410ab90 0x7f5c741966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:25.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.065+0000 7f5c7259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c741017c0 0x7f5c7419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:25.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.066+0000 7f5c72d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7410ab90 0x7f5c741966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57870/0 (socket says 172.21.15.82:57870) 2023-09-09T23:19:25.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.066+0000 7f5c71d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c741020e0 0x7f5c7419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.066+0000 7f5c72d9d700 1 -- 172.21.15.82:0/2552770757 learned_addr learned my addr 172.21.15.82:0/2552770757 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:19:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.066+0000 7f5c72d9d700 1 -- 172.21.15.82:0/2552770757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c741017c0 msgr2=0x7f5c7419c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:19:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.066+0000 7f5c72d9d700 1 --2- 172.21.15.82:0/2552770757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c741017c0 0x7f5c7419c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.066+0000 7f5c72d9d700 1 -- 172.21.15.82:0/2552770757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c741020e0 msgr2=0x7f5c7419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:19:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.066+0000 7f5c72d9d700 1 --2- 172.21.15.82:0/2552770757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c741020e0 0x7f5c7419cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.070 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.066+0000 7f5c72d9d700 1 -- 172.21.15.82:0/2552770757 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6c009d00 con 0x7f5c7410ab90 2023-09-09T23:19:25.070 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.066+0000 7f5c7259c700 1 --2- 172.21.15.82:0/2552770757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c741017c0 0x7f5c7419c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:19:25.070 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.066+0000 7f5c71d9b700 1 --2- 172.21.15.82:0/2552770757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c741020e0 0x7f5c7419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:19:25.070 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.066+0000 7f5c72d9d700 1 --2- 172.21.15.82:0/2552770757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7410ab90 0x7f5c741966b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009fd0 tx=0x7f5c6c00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:19:25.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.066+0000 7f5c637fe700 1 -- 172.21.15.82:0/2552770757 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c00e040 con 0x7f5c7410ab90 2023-09-09T23:19:25.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.066+0000 7f5c78897700 1 -- 172.21.15.82:0/2552770757 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c741972d0 con 0x7f5c7410ab90 2023-09-09T23:19:25.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.066+0000 7f5c637fe700 1 -- 172.21.15.82:0/2552770757 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c6c0094c0 con 0x7f5c7410ab90 2023-09-09T23:19:25.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.066+0000 7f5c637fe700 1 -- 172.21.15.82:0/2552770757 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c0052a0 con 0x7f5c7410ab90 2023-09-09T23:19:25.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.066+0000 7f5c78897700 1 -- 172.21.15.82:0/2552770757 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c741a3630 con 0x7f5c7410ab90 2023-09-09T23:19:25.076 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.069+0000 7f5c637fe700 1 -- 172.21.15.82:0/2552770757 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f5c6c01d470 con 0x7f5c7410ab90 2023-09-09T23:19:25.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.069+0000 7f5c637fe700 1 --2- 172.21.15.82:0/2552770757 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f5c5c0a07b0 0x7f5c5c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:25.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.070+0000 7f5c7259c700 1 --2- 172.21.15.82:0/2552770757 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f5c5c0a07b0 0x7f5c5c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:25.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.070+0000 7f5c637fe700 1 -- 172.21.15.82:0/2552770757 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f5c6c0cd3a0 con 0x7f5c7410ab90 2023-09-09T23:19:25.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.070+0000 7f5c7259c700 1 --2- 172.21.15.82:0/2552770757 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f5c5c0a07b0 0x7f5c5c0a2c70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5c64011490 tx=0x7f5c64009260 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:19:25.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.070+0000 7f5c84a5db80 1 Processor -- start 2023-09-09T23:19:25.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.071+0000 7f5c84a5db80 1 -- start start 2023-09-09T23:19:25.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.071+0000 7f5c84a5db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6d4e0c710 0x55d6d4e0caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:25.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.071+0000 7f5c84a5db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6d4e0d030 0x55d6d4e16900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:25.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.071+0000 7f5c84a5db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6d4e16e40 0x55d6d4e19230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:25.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.071+0000 7f5c84a5db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6d4d65730 con 0x55d6d4e0d030 2023-09-09T23:19:25.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.071+0000 7f5c84a5db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6d4d63050 con 0x55d6d4e0c710 2023-09-09T23:19:25.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.071+0000 7f5c84a5db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6d4d5e940 con 0x55d6d4e16e40 2023-09-09T23:19:25.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.071+0000 7f5c71d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6d4e0d030 0x55d6d4e16900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:25.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.071+0000 7f5c7259c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6d4e0c710 0x55d6d4e0caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:25.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.071+0000 7f5c71d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6d4e0d030 0x55d6d4e16900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57876/0 (socket says 172.21.15.82:57876) 2023-09-09T23:19:25.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.071+0000 7f5c71d9b700 1 -- 172.21.15.82:0/1994773746 learned_addr learned my addr 172.21.15.82:0/1994773746 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:19:25.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.071+0000 7f5c72d9d700 1 --2- 172.21.15.82:0/1994773746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6d4e16e40 0x55d6d4e19230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:25.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.071+0000 7f5c71d9b700 1 -- 172.21.15.82:0/1994773746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6d4e16e40 msgr2=0x55d6d4e19230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:19:25.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.071+0000 7f5c71d9b700 1 --2- 172.21.15.82:0/1994773746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6d4e16e40 0x55d6d4e19230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.071+0000 7f5c71d9b700 1 -- 172.21.15.82:0/1994773746 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6d4e0c710 msgr2=0x55d6d4e0caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:19:25.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.071+0000 7f5c71d9b700 1 --2- 172.21.15.82:0/1994773746 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6d4e0c710 0x55d6d4e0caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.071+0000 7f5c71d9b700 1 -- 172.21.15.82:0/1994773746 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6d4b8ade0 con 0x55d6d4e0d030 2023-09-09T23:19:25.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.071+0000 7f5c7259c700 1 --2- 172.21.15.82:0/1994773746 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6d4e0c710 0x55d6d4e0caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:19:25.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.071+0000 7f5c72d9d700 1 --2- 172.21.15.82:0/1994773746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6d4e16e40 0x55d6d4e19230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:19:25.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.072+0000 7f5c71d9b700 1 --2- 172.21.15.82:0/1994773746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6d4e0d030 0x55d6d4e16900 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f5c680114f0 tx=0x7f5c68011800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4270298d99b0b626 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:19:25.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.074+0000 7f5c60ff9700 1 -- 172.21.15.82:0/1994773746 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c68004230 con 0x55d6d4e0d030 2023-09-09T23:19:25.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.074+0000 7f5c60ff9700 1 -- 172.21.15.82:0/1994773746 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c68004390 con 0x55d6d4e0d030 2023-09-09T23:19:25.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.074+0000 7f5c60ff9700 1 -- 172.21.15.82:0/1994773746 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6800e910 con 0x55d6d4e0d030 2023-09-09T23:19:25.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.074+0000 7f5c84a5db80 1 -- 172.21.15.82:0/1994773746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6d4e0d030 msgr2=0x55d6d4e16900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:19:25.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.074+0000 7f5c84a5db80 1 --2- 172.21.15.82:0/1994773746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6d4e0d030 0x55d6d4e16900 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f5c680114f0 tx=0x7f5c68011800 comp rx=0 tx=0).stop 2023-09-09T23:19:25.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.075+0000 7f5c84a5db80 1 -- 172.21.15.82:0/1994773746 shutdown_connections 2023-09-09T23:19:25.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.075+0000 7f5c84a5db80 1 --2- 172.21.15.82:0/1994773746 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6d4e0c710 0x55d6d4e0caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.075+0000 7f5c84a5db80 1 --2- 172.21.15.82:0/1994773746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6d4e16e40 0x55d6d4e19230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.075+0000 7f5c84a5db80 1 --2- 172.21.15.82:0/1994773746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6d4e0d030 0x55d6d4e16900 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.075+0000 7f5c84a5db80 1 -- 172.21.15.82:0/1994773746 >> 172.21.15.82:0/1994773746 conn(0x55d6d4d81090 msgr2=0x55d6d4e07bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:19:25.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.075+0000 7f5c84a5db80 1 -- 172.21.15.82:0/1994773746 shutdown_connections 2023-09-09T23:19:25.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.075+0000 7f5c84a5db80 1 -- 172.21.15.82:0/1994773746 wait complete. 2023-09-09T23:19:25.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.075+0000 7f5c84a5db80 1 -- start start 2023-09-09T23:19:25.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.075+0000 7f5c84a5db80 1 Processor -- start 2023-09-09T23:19:25.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.076+0000 7f5c84a5db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6d4e0c710 0x55d6d4e16810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:25.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.076+0000 7f5c84a5db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6d4e0d030 0x55d6d4e13a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:25.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.076+0000 7f5c84a5db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6d4e16e40 0x55d6d4e14030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:25.085 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.076+0000 7f5c84a5db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6d4cfcb70 con 0x55d6d4e0c710 2023-09-09T23:19:25.085 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.076+0000 7f5c84a5db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6d4cfc870 con 0x55d6d4e0d030 2023-09-09T23:19:25.085 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.076+0000 7f5c84a5db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6d4be0920 con 0x55d6d4e16e40 2023-09-09T23:19:25.085 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.076+0000 7f5c84a5db80 10 client.0 _ll_register_callbacks cb 0x55d6d4c46a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:19:25.085 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.076+0000 7f5c7259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6d4e0c710 0x55d6d4e16810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:25.085 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.076+0000 7f5c7259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6d4e0c710 0x55d6d4e16810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57880/0 (socket says 172.21.15.82:57880) 2023-09-09T23:19:25.086 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.076+0000 7f5c7259c700 1 -- 172.21.15.82:0/3834777457 learned_addr learned my addr 172.21.15.82:0/3834777457 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:19:25.086 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.076+0000 7f5c72d9d700 1 --2- 172.21.15.82:0/3834777457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6d4e16e40 0x55d6d4e14030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:25.086 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.076+0000 7f5c71d9b700 1 --2- 172.21.15.82:0/3834777457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6d4e0d030 0x55d6d4e13a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:25.086 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.076+0000 7f5c7259c700 1 -- 172.21.15.82:0/3834777457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6d4e16e40 msgr2=0x55d6d4e14030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:19:25.086 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.076+0000 7f5c7259c700 1 --2- 172.21.15.82:0/3834777457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6d4e16e40 0x55d6d4e14030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.076+0000 7f5c7259c700 1 -- 172.21.15.82:0/3834777457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6d4e0d030 msgr2=0x55d6d4e13a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:19:25.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.076+0000 7f5c7259c700 1 --2- 172.21.15.82:0/3834777457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6d4e0d030 0x55d6d4e13a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.076+0000 7f5c7259c700 1 -- 172.21.15.82:0/3834777457 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c64017070 con 0x55d6d4e0c710 2023-09-09T23:19:25.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.076+0000 7f5c7259c700 1 --2- 172.21.15.82:0/3834777457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6d4e0c710 0x55d6d4e16810 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f5c64003ab0 tx=0x7f5c640047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab398b65f135b8c server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:19:25.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.076+0000 7f5c59ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:19:25.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.076+0000 7f5c59ffb700 1 -- 172.21.15.82:0/3834777457 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6401fca0 con 0x55d6d4e0c710 2023-09-09T23:19:25.088 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.076+0000 7f5c59ffb700 1 -- 172.21.15.82:0/3834777457 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f5c6401fe00 con 0x55d6d4e0c710 2023-09-09T23:19:25.088 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.076+0000 7f5c84a5db80 1 -- 172.21.15.82:0/3834777457 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d6d4cb64f0 con 0x55d6d4e0c710 2023-09-09T23:19:25.088 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.077+0000 7f5c59ffb700 1 -- 172.21.15.82:0/3834777457 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c64003e80 con 0x55d6d4e0c710 2023-09-09T23:19:25.088 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.077+0000 7f5c84a5db80 10 client.16119 fetch_fsmap learned FSMap version 42 2023-09-09T23:19:25.088 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.077+0000 7f5c84a5db80 1 -- 172.21.15.82:0/3834777457 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f5c68009cd0 con 0x55d6d4e0c710 2023-09-09T23:19:25.088 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.077+0000 7f5c59ffb700 1 -- 172.21.15.82:0/3834777457 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d6d4cb64f0 con 0x55d6d4e0c710 2023-09-09T23:19:25.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.077+0000 7f5c59ffb700 1 -- 172.21.15.82:0/3834777457 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 42) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f5c64027500 con 0x55d6d4e0c710 2023-09-09T23:19:25.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.077+0000 7f5c84a5db80 10 client.16119 fetch_fsmap finished waiting for FSMap version 42 2023-09-09T23:19:25.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.077+0000 7f5c84a5db80 10 client.16119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:19:25.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.077+0000 7f5c84a5db80 20 client.16119 populate_metadata read hostname 'smithi082' 2023-09-09T23:19:25.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.077+0000 7f5c84a5db80 1 --2- 172.21.15.82:0/3834777457 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55d6d4e269f0 0x55d6d4e28e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:25.090 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.077+0000 7f5c84a5db80 4 client.16119 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:19:25.090 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.077+0000 7f5c84a5db80 1 -- 172.21.15.82:0/3834777457 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d6d4d738a0 con 0x55d6d4e269f0 2023-09-09T23:19:25.090 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.077+0000 7f5c71d9b700 1 --2- 172.21.15.82:0/3834777457 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55d6d4e269f0 0x55d6d4e28e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:25.090 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.078+0000 7f5c59ffb700 10 client.16119 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:19:25.090 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.078+0000 7f5c71d9b700 1 --2- 172.21.15.82:0/3834777457 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55d6d4e269f0 0x55d6d4e28e10 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eea26bba1827c3e server_cookie=3b93995b4024e75c in_seq=0 out_seq=0 2023-09-09T23:19:25.091 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.079+0000 7f5c59ffb700 1 -- 172.21.15.82:0/3834777457 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d6d4d738a0 con 0x55d6d4e269f0 2023-09-09T23:19:25.092 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.079+0000 7f5c59ffb700 10 client.16119 handle_command_reply: tid=0 2023-09-09T23:19:25.092 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.079+0000 7f5c84a5db80 1 client.16119 shutdown 2023-09-09T23:19:25.092 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.079+0000 7f5c84a5db80 1 -- 172.21.15.82:0/3834777457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6d4e0c710 msgr2=0x55d6d4e16810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:19:25.092 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.079+0000 7f5c84a5db80 1 --2- 172.21.15.82:0/3834777457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6d4e0c710 0x55d6d4e16810 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f5c64003ab0 tx=0x7f5c640047c0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.092 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.079+0000 7f5c84a5db80 1 -- 172.21.15.82:0/3834777457 shutdown_connections 2023-09-09T23:19:25.093 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.079+0000 7f5c84a5db80 1 --2- 172.21.15.82:0/3834777457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6d4e16e40 0x55d6d4e14030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.093 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.079+0000 7f5c84a5db80 1 --2- 172.21.15.82:0/3834777457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6d4e0c710 0x55d6d4e16810 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.093 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.079+0000 7f5c84a5db80 1 --2- 172.21.15.82:0/3834777457 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55d6d4e269f0 0x55d6d4e28e10 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.093 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.079+0000 7f5c84a5db80 1 --2- 172.21.15.82:0/3834777457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6d4e0d030 0x55d6d4e13a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.093 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.079+0000 7f5c84a5db80 1 -- 172.21.15.82:0/3834777457 >> 172.21.15.82:0/3834777457 conn(0x55d6d4d81090 msgr2=0x55d6d4e07780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:19:25.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.079+0000 7f5c59ffb700 0 client.16119 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:19:25.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.079+0000 7f5c84a5db80 1 -- 172.21.15.82:0/3834777457 shutdown_connections 2023-09-09T23:19:25.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.079+0000 7f5c84a5db80 1 -- 172.21.15.82:0/3834777457 wait complete. 2023-09-09T23:19:25.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.079+0000 7f5c84a5db80 20 client.16119 trim_cache size 0 max 16384 2023-09-09T23:19:25.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.095+0000 7f5c84a5db80 1 Processor -- start 2023-09-09T23:19:25.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.095+0000 7f5c84a5db80 1 -- start start 2023-09-09T23:19:25.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.096+0000 7f5c84a5db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6d4e0c710 0x55d6d4e0ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:25.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.096+0000 7f5c84a5db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6d4e0d030 0x55d6d4e16340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:25.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.096+0000 7f5c84a5db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6d4e16e40 0x55d6d4e131e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:25.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.096+0000 7f5c84a5db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6d4e19750 con 0x55d6d4e0d030 2023-09-09T23:19:25.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.096+0000 7f5c84a5db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6d4e0d410 con 0x55d6d4e16e40 2023-09-09T23:19:25.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.096+0000 7f5c84a5db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6d4e23f50 con 0x55d6d4e0c710 2023-09-09T23:19:25.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.096+0000 7f5c71d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6d4e0d030 0x55d6d4e16340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:25.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.096+0000 7f5c71d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6d4e0d030 0x55d6d4e16340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57890/0 (socket says 172.21.15.82:57890) 2023-09-09T23:19:25.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.096+0000 7f5c71d9b700 1 -- 172.21.15.82:0/3599009109 learned_addr learned my addr 172.21.15.82:0/3599009109 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:19:25.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.096+0000 7f5c7259c700 1 --2- 172.21.15.82:0/3599009109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6d4e0c710 0x55d6d4e0ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:25.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.096+0000 7f5c72d9d700 1 --2- 172.21.15.82:0/3599009109 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6d4e16e40 0x55d6d4e131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:25.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.096+0000 7f5c71d9b700 1 -- 172.21.15.82:0/3599009109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6d4e0c710 msgr2=0x55d6d4e0ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:19:25.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.096+0000 7f5c71d9b700 1 --2- 172.21.15.82:0/3599009109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6d4e0c710 0x55d6d4e0ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.096+0000 7f5c71d9b700 1 -- 172.21.15.82:0/3599009109 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6d4e16e40 msgr2=0x55d6d4e131e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:19:25.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.096+0000 7f5c71d9b700 1 --2- 172.21.15.82:0/3599009109 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6d4e16e40 0x55d6d4e131e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.096+0000 7f5c71d9b700 1 -- 172.21.15.82:0/3599009109 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6d4b73c60 con 0x55d6d4e0d030 2023-09-09T23:19:25.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.096+0000 7f5c72d9d700 1 --2- 172.21.15.82:0/3599009109 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6d4e16e40 0x55d6d4e131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:19:25.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.096+0000 7f5c71d9b700 1 --2- 172.21.15.82:0/3599009109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6d4e0d030 0x55d6d4e16340 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x55d6d4e02280 tx=0x7f5c68011800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa4155bdc5f125c server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:19:25.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.096+0000 7f5c617fa700 1 -- 172.21.15.82:0/3599009109 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c68004230 con 0x55d6d4e0d030 2023-09-09T23:19:25.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.096+0000 7f5c617fa700 1 -- 172.21.15.82:0/3599009109 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c68005500 con 0x55d6d4e0d030 2023-09-09T23:19:25.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.096+0000 7f5c617fa700 1 -- 172.21.15.82:0/3599009109 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6802a7a0 con 0x55d6d4e0d030 2023-09-09T23:19:25.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.097+0000 7f5c84a5db80 1 -- 172.21.15.82:0/3599009109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6d4e0d030 msgr2=0x55d6d4e16340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:19:25.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.097+0000 7f5c84a5db80 1 --2- 172.21.15.82:0/3599009109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6d4e0d030 0x55d6d4e16340 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x55d6d4e02280 tx=0x7f5c68011800 comp rx=0 tx=0).stop 2023-09-09T23:19:25.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.097+0000 7f5c84a5db80 1 -- 172.21.15.82:0/3599009109 shutdown_connections 2023-09-09T23:19:25.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.097+0000 7f5c84a5db80 1 --2- 172.21.15.82:0/3599009109 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6d4e16e40 0x55d6d4e131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.097+0000 7f5c84a5db80 1 --2- 172.21.15.82:0/3599009109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6d4e0c710 0x55d6d4e0ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.097+0000 7f5c84a5db80 1 --2- 172.21.15.82:0/3599009109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6d4e0d030 0x55d6d4e16340 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.097+0000 7f5c84a5db80 1 -- 172.21.15.82:0/3599009109 >> 172.21.15.82:0/3599009109 conn(0x55d6d4d81090 msgr2=0x55d6d4e22140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:19:25.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.097+0000 7f5c84a5db80 1 -- 172.21.15.82:0/3599009109 shutdown_connections 2023-09-09T23:19:25.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.097+0000 7f5c84a5db80 1 -- 172.21.15.82:0/3599009109 wait complete. 2023-09-09T23:19:25.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.097+0000 7f5c84a5db80 1 -- start start 2023-09-09T23:19:25.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.097+0000 7f5c84a5db80 1 Processor -- start 2023-09-09T23:19:25.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.098+0000 7f5c84a5db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6d4e0c710 0x55d6d4e1d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:25.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.098+0000 7f5c84a5db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6d4e0d030 0x55d6d4e1db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:25.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.098+0000 7f5c84a5db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6d4e16e40 0x55d6d4e1bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:25.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.098+0000 7f5c84a5db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6d4e1c270 con 0x55d6d4e0c710 2023-09-09T23:19:25.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.098+0000 7f5c84a5db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6d4e1c3e0 con 0x55d6d4e16e40 2023-09-09T23:19:25.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.098+0000 7f5c84a5db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6d4e1c5a0 con 0x55d6d4e0d030 2023-09-09T23:19:25.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.098+0000 7f5c84a5db80 10 client.0 _ll_register_callbacks cb 0x7f5c64003680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:19:25.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.098+0000 7f5c7259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6d4e0c710 0x55d6d4e1d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:25.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.098+0000 7f5c7259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6d4e0c710 0x55d6d4e1d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57894/0 (socket says 172.21.15.82:57894) 2023-09-09T23:19:25.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.098+0000 7f5c7259c700 1 -- 172.21.15.82:0/3355922101 learned_addr learned my addr 172.21.15.82:0/3355922101 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:19:25.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.098+0000 7f5c71d9b700 1 --2- 172.21.15.82:0/3355922101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6d4e0d030 0x55d6d4e1db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:25.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.098+0000 7f5c72d9d700 1 --2- 172.21.15.82:0/3355922101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6d4e16e40 0x55d6d4e1bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:25.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.098+0000 7f5c7259c700 1 -- 172.21.15.82:0/3355922101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6d4e0d030 msgr2=0x55d6d4e1db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:19:25.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.098+0000 7f5c7259c700 1 --2- 172.21.15.82:0/3355922101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6d4e0d030 0x55d6d4e1db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.098+0000 7f5c7259c700 1 -- 172.21.15.82:0/3355922101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6d4e16e40 msgr2=0x55d6d4e1bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:19:25.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.098+0000 7f5c7259c700 1 --2- 172.21.15.82:0/3355922101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6d4e16e40 0x55d6d4e1bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.098+0000 7f5c7259c700 1 -- 172.21.15.82:0/3355922101 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d6d4b8ade0 con 0x55d6d4e0c710 2023-09-09T23:19:25.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.098+0000 7f5c7259c700 1 --2- 172.21.15.82:0/3355922101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6d4e0c710 0x55d6d4e1d630 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009f10 tx=0x7f5c64027da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22cc7557af2658c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:19:25.110 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.098+0000 7f5c59ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:19:25.110 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.098+0000 7f5c59ffb700 1 -- 172.21.15.82:0/3355922101 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c640092d0 con 0x55d6d4e0c710 2023-09-09T23:19:25.110 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.098+0000 7f5c59ffb700 1 -- 172.21.15.82:0/3355922101 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f5c64039470 con 0x55d6d4e0c710 2023-09-09T23:19:25.110 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.098+0000 7f5c84a5db80 1 -- 172.21.15.82:0/3355922101 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c68004390 con 0x55d6d4e0c710 2023-09-09T23:19:25.110 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.098+0000 7f5c59ffb700 1 -- 172.21.15.82:0/3355922101 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c64003e80 con 0x55d6d4e0c710 2023-09-09T23:19:25.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.098+0000 7f5c84a5db80 10 client.16125 fetch_fsmap learned FSMap version 42 2023-09-09T23:19:25.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.098+0000 7f5c84a5db80 1 -- 172.21.15.82:0/3355922101 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55d6d4bb1250 con 0x55d6d4e0c710 2023-09-09T23:19:25.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.099+0000 7f5c59ffb700 1 -- 172.21.15.82:0/3355922101 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c64047020 con 0x55d6d4e0c710 2023-09-09T23:19:25.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.099+0000 7f5c59ffb700 1 -- 172.21.15.82:0/3355922101 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 42) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f5c64043440 con 0x55d6d4e0c710 2023-09-09T23:19:25.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.099+0000 7f5c84a5db80 10 client.16125 fetch_fsmap finished waiting for FSMap version 42 2023-09-09T23:19:25.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.099+0000 7f5c84a5db80 10 client.16125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:19:25.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.099+0000 7f5c84a5db80 20 client.16125 populate_metadata read hostname 'smithi082' 2023-09-09T23:19:25.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.099+0000 7f5c84a5db80 1 --2- 172.21.15.82:0/3355922101 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55d6d4e7ff70 0x55d6d4e82390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:25.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.099+0000 7f5c84a5db80 4 client.16125 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:19:25.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.099+0000 7f5c84a5db80 1 -- 172.21.15.82:0/3355922101 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d6d4d57920 con 0x55d6d4e7ff70 2023-09-09T23:19:25.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.099+0000 7f5c71d9b700 1 --2- 172.21.15.82:0/3355922101 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55d6d4e7ff70 0x55d6d4e82390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:25.113 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.099+0000 7f5c59ffb700 10 client.16125 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:19:25.113 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.099+0000 7f5c71d9b700 1 --2- 172.21.15.82:0/3355922101 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55d6d4e7ff70 0x55d6d4e82390 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c9a9ae861f82ce3 server_cookie=294ad5e5bb16f27e in_seq=0 out_seq=0 2023-09-09T23:19:25.113 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.100+0000 7f5c59ffb700 1 -- 172.21.15.82:0/3355922101 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55d6d4d57920 con 0x55d6d4e7ff70 2023-09-09T23:19:25.113 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.100+0000 7f5c59ffb700 10 client.16125 handle_command_reply: tid=0 2023-09-09T23:19:25.113 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.100+0000 7f5c84a5db80 1 client.16125 shutdown 2023-09-09T23:19:25.113 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.100+0000 7f5c84a5db80 1 -- 172.21.15.82:0/3355922101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6d4e0c710 msgr2=0x55d6d4e1d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:19:25.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.100+0000 7f5c84a5db80 1 --2- 172.21.15.82:0/3355922101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6d4e0c710 0x55d6d4e1d630 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009f10 tx=0x7f5c64027da0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.100+0000 7f5c84a5db80 1 -- 172.21.15.82:0/3355922101 shutdown_connections 2023-09-09T23:19:25.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.100+0000 7f5c84a5db80 1 --2- 172.21.15.82:0/3355922101 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55d6d4e7ff70 0x55d6d4e82390 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.100+0000 7f5c84a5db80 1 --2- 172.21.15.82:0/3355922101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6d4e16e40 0x55d6d4e1bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.100+0000 7f5c84a5db80 1 --2- 172.21.15.82:0/3355922101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6d4e0c710 0x55d6d4e1d630 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.100+0000 7f5c84a5db80 1 --2- 172.21.15.82:0/3355922101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6d4e0d030 0x55d6d4e1db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.100+0000 7f5c59ffb700 0 client.16125 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:19:25.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.100+0000 7f5c84a5db80 1 -- 172.21.15.82:0/3355922101 >> 172.21.15.82:0/3355922101 conn(0x55d6d4d81090 msgr2=0x55d6d4e0ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:19:25.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.100+0000 7f5c84a5db80 1 -- 172.21.15.82:0/3355922101 shutdown_connections 2023-09-09T23:19:25.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.100+0000 7f5c84a5db80 1 -- 172.21.15.82:0/3355922101 wait complete. 2023-09-09T23:19:25.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.100+0000 7f5c84a5db80 20 client.16125 trim_cache size 0 max 16384 2023-09-09T23:19:25.116 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:19:25.116 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:19:25.116 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:19:25.116 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:19:25.117 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:19:25.117 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:19:25.117 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:19:25.117 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:19:25.117 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:19:25.118 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:19:25.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.100+0000 7f5c78897700 1 -- 172.21.15.82:0/2552770757 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f5c5c0a07b0 msgr2=0x7f5c5c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:19:25.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.100+0000 7f5c78897700 1 --2- 172.21.15.82:0/2552770757 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f5c5c0a07b0 0x7f5c5c0a2c70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5c64011490 tx=0x7f5c64009260 comp rx=0 tx=0).stop 2023-09-09T23:19:25.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.100+0000 7f5c78897700 1 -- 172.21.15.82:0/2552770757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7410ab90 msgr2=0x7f5c741966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:19:25.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.100+0000 7f5c78897700 1 --2- 172.21.15.82:0/2552770757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7410ab90 0x7f5c741966b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009fd0 tx=0x7f5c6c00bce0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.100+0000 7f5c78897700 1 -- 172.21.15.82:0/2552770757 shutdown_connections 2023-09-09T23:19:25.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.100+0000 7f5c78897700 1 --2- 172.21.15.82:0/2552770757 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f5c5c0a07b0 0x7f5c5c0a2c70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.100+0000 7f5c78897700 1 --2- 172.21.15.82:0/2552770757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7410ab90 0x7f5c741966b0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.100+0000 7f5c78897700 1 --2- 172.21.15.82:0/2552770757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c741017c0 0x7f5c7419c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.100+0000 7f5c78897700 1 --2- 172.21.15.82:0/2552770757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c741020e0 0x7f5c7419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:25.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.100+0000 7f5c78897700 1 -- 172.21.15.82:0/2552770757 >> 172.21.15.82:0/2552770757 conn(0x7f5c740faf60 msgr2=0x7f5c7410cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:19:25.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.101+0000 7f5c78897700 1 -- 172.21.15.82:0/2552770757 shutdown_connections 2023-09-09T23:19:25.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:25.101+0000 7f5c78897700 1 -- 172.21.15.82:0/2552770757 wait complete. 2023-09-09T23:19:25.122 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:19:25.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:25 smithi170 ceph-mon[102832]: 391 slow requests, 5 included below; oldest blocked for > 190.624688 secs 2023-09-09T23:19:25.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:25 smithi170 ceph-mon[102832]: slow request 189.923765 seconds old, received at 2023-09-09T23:16:14.083361+0000: client_request(mds.1:1601 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:25.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:25 smithi170 ceph-mon[102832]: slow request 189.866268 seconds old, received at 2023-09-09T23:16:14.140858+0000: client_request(mds.1:1633 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:25.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:25 smithi170 ceph-mon[102832]: slow request 189.809820 seconds old, received at 2023-09-09T23:16:14.197306+0000: client_request(mds.1:1665 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:25.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:25 smithi170 ceph-mon[102832]: slow request 189.752867 seconds old, received at 2023-09-09T23:16:14.254259+0000: client_request(mds.1:1697 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:25.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:25 smithi170 ceph-mon[102832]: slow request 189.693437 seconds old, received at 2023-09-09T23:16:14.313689+0000: client_request(mds.1:1729 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:25.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:25 smithi170 ceph-mon[102832]: pgmap v168: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 25 KiB/s rd, 31 MiB/s wr, 27 op/s 2023-09-09T23:19:25.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:25 smithi082 ceph-mon[98694]: 391 slow requests, 5 included below; oldest blocked for > 190.624688 secs 2023-09-09T23:19:25.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:25 smithi082 ceph-mon[98694]: slow request 189.923765 seconds old, received at 2023-09-09T23:16:14.083361+0000: client_request(mds.1:1601 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:25.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:25 smithi082 ceph-mon[98694]: slow request 189.866268 seconds old, received at 2023-09-09T23:16:14.140858+0000: client_request(mds.1:1633 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:25.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:25 smithi082 ceph-mon[98694]: slow request 189.809820 seconds old, received at 2023-09-09T23:16:14.197306+0000: client_request(mds.1:1665 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:25.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:25 smithi082 ceph-mon[98694]: slow request 189.752867 seconds old, received at 2023-09-09T23:16:14.254259+0000: client_request(mds.1:1697 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:25.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:25 smithi082 ceph-mon[98694]: slow request 189.693437 seconds old, received at 2023-09-09T23:16:14.313689+0000: client_request(mds.1:1729 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:25.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:25 smithi082 ceph-mon[98694]: pgmap v168: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 25 KiB/s rd, 31 MiB/s wr, 27 op/s 2023-09-09T23:19:25.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:25 smithi098 ceph-mon[103124]: 391 slow requests, 5 included below; oldest blocked for > 190.624688 secs 2023-09-09T23:19:25.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:25 smithi098 ceph-mon[103124]: slow request 189.923765 seconds old, received at 2023-09-09T23:16:14.083361+0000: client_request(mds.1:1601 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:25.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:25 smithi098 ceph-mon[103124]: slow request 189.866268 seconds old, received at 2023-09-09T23:16:14.140858+0000: client_request(mds.1:1633 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:25.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:25 smithi098 ceph-mon[103124]: slow request 189.809820 seconds old, received at 2023-09-09T23:16:14.197306+0000: client_request(mds.1:1665 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:25.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:25 smithi098 ceph-mon[103124]: slow request 189.752867 seconds old, received at 2023-09-09T23:16:14.254259+0000: client_request(mds.1:1697 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:25.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:25 smithi098 ceph-mon[103124]: slow request 189.693437 seconds old, received at 2023-09-09T23:16:14.313689+0000: client_request(mds.1:1729 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:25.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:25 smithi098 ceph-mon[103124]: pgmap v168: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 25 KiB/s rd, 31 MiB/s wr, 27 op/s 2023-09-09T23:19:26.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:26.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:26.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:27.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:27 smithi170 ceph-mon[102832]: pgmap v169: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 44 KiB/s rd, 35 MiB/s wr, 38 op/s 2023-09-09T23:19:27.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:27 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:19:27.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:27 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:27.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:27 smithi170 ceph-mon[102832]: 48 slow requests, 5 included below; oldest blocked for > 180.914337 secs 2023-09-09T23:19:27.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:27 smithi170 ceph-mon[102832]: slow request 180.911499 seconds old, received at 2023-09-09T23:16:25.775616+0000: peer_request:mds.1:1291 currently dispatched 2023-09-09T23:19:27.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:27 smithi170 ceph-mon[102832]: slow request 180.900700 seconds old, received at 2023-09-09T23:16:25.786414+0000: peer_request:mds.1:1323 currently dispatched 2023-09-09T23:19:27.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:27 smithi170 ceph-mon[102832]: slow request 180.889969 seconds old, received at 2023-09-09T23:16:25.797146+0000: peer_request:mds.1:1355 currently dispatched 2023-09-09T23:19:27.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:27 smithi170 ceph-mon[102832]: slow request 180.878326 seconds old, received at 2023-09-09T23:16:25.808789+0000: peer_request:mds.1:1387 currently dispatched 2023-09-09T23:19:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:27 smithi170 ceph-mon[102832]: slow request 180.867021 seconds old, received at 2023-09-09T23:16:25.820094+0000: peer_request:mds.1:1419 currently dispatched 2023-09-09T23:19:27.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:27 smithi082 ceph-mon[98694]: pgmap v169: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 44 KiB/s rd, 35 MiB/s wr, 38 op/s 2023-09-09T23:19:27.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:27 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:19:27.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:27 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:27.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:27 smithi082 ceph-mon[98694]: 48 slow requests, 5 included below; oldest blocked for > 180.914337 secs 2023-09-09T23:19:27.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:27 smithi082 ceph-mon[98694]: slow request 180.911499 seconds old, received at 2023-09-09T23:16:25.775616+0000: peer_request:mds.1:1291 currently dispatched 2023-09-09T23:19:27.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:27 smithi082 ceph-mon[98694]: slow request 180.900700 seconds old, received at 2023-09-09T23:16:25.786414+0000: peer_request:mds.1:1323 currently dispatched 2023-09-09T23:19:27.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:27 smithi082 ceph-mon[98694]: slow request 180.889969 seconds old, received at 2023-09-09T23:16:25.797146+0000: peer_request:mds.1:1355 currently dispatched 2023-09-09T23:19:27.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:27 smithi082 ceph-mon[98694]: slow request 180.878326 seconds old, received at 2023-09-09T23:16:25.808789+0000: peer_request:mds.1:1387 currently dispatched 2023-09-09T23:19:27.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:27 smithi082 ceph-mon[98694]: slow request 180.867021 seconds old, received at 2023-09-09T23:16:25.820094+0000: peer_request:mds.1:1419 currently dispatched 2023-09-09T23:19:27.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:27 smithi098 ceph-mon[103124]: pgmap v169: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 44 KiB/s rd, 35 MiB/s wr, 38 op/s 2023-09-09T23:19:27.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:27 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:19:27.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:27 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:27.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:27 smithi098 ceph-mon[103124]: 48 slow requests, 5 included below; oldest blocked for > 180.914337 secs 2023-09-09T23:19:27.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:27 smithi098 ceph-mon[103124]: slow request 180.911499 seconds old, received at 2023-09-09T23:16:25.775616+0000: peer_request:mds.1:1291 currently dispatched 2023-09-09T23:19:27.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:27 smithi098 ceph-mon[103124]: slow request 180.900700 seconds old, received at 2023-09-09T23:16:25.786414+0000: peer_request:mds.1:1323 currently dispatched 2023-09-09T23:19:27.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:27 smithi098 ceph-mon[103124]: slow request 180.889969 seconds old, received at 2023-09-09T23:16:25.797146+0000: peer_request:mds.1:1355 currently dispatched 2023-09-09T23:19:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:27 smithi098 ceph-mon[103124]: slow request 180.878326 seconds old, received at 2023-09-09T23:16:25.808789+0000: peer_request:mds.1:1387 currently dispatched 2023-09-09T23:19:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:27 smithi098 ceph-mon[103124]: slow request 180.867021 seconds old, received at 2023-09-09T23:16:25.820094+0000: peer_request:mds.1:1419 currently dispatched 2023-09-09T23:19:29.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:29 smithi170 ceph-mon[102832]: pgmap v170: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 32 KiB/s rd, 5.4 MiB/s wr, 19 op/s 2023-09-09T23:19:29.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:29 smithi082 ceph-mon[98694]: pgmap v170: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 32 KiB/s rd, 5.4 MiB/s wr, 19 op/s 2023-09-09T23:19:29.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:29 smithi098 ceph-mon[103124]: pgmap v170: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 32 KiB/s rd, 5.4 MiB/s wr, 19 op/s 2023-09-09T23:19:30.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:30 smithi170 ceph-mon[102832]: 341 slow requests, 5 included below; oldest blocked for > 195.667048 secs 2023-09-09T23:19:30.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:30 smithi170 ceph-mon[102832]: slow request 194.692997 seconds old, received at 2023-09-09T23:16:14.371980+0000: client_request(mds.1:1761 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:30.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:30 smithi170 ceph-mon[102832]: slow request 194.635167 seconds old, received at 2023-09-09T23:16:14.429810+0000: client_request(mds.1:1793 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:30.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:30 smithi170 ceph-mon[102832]: slow request 194.576073 seconds old, received at 2023-09-09T23:16:14.488904+0000: client_request(mds.1:1825 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:30.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:30 smithi170 ceph-mon[102832]: slow request 194.521649 seconds old, received at 2023-09-09T23:16:14.543328+0000: client_request(mds.1:1857 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:30.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:30 smithi170 ceph-mon[102832]: slow request 194.468877 seconds old, received at 2023-09-09T23:16:14.596100+0000: client_request(mds.1:1889 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:30.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:30 smithi082 ceph-mon[98694]: 341 slow requests, 5 included below; oldest blocked for > 195.667048 secs 2023-09-09T23:19:30.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:30 smithi082 ceph-mon[98694]: slow request 194.692997 seconds old, received at 2023-09-09T23:16:14.371980+0000: client_request(mds.1:1761 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:30.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:30 smithi082 ceph-mon[98694]: slow request 194.635167 seconds old, received at 2023-09-09T23:16:14.429810+0000: client_request(mds.1:1793 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:30.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:30 smithi082 ceph-mon[98694]: slow request 194.576073 seconds old, received at 2023-09-09T23:16:14.488904+0000: client_request(mds.1:1825 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:30.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:30 smithi082 ceph-mon[98694]: slow request 194.521649 seconds old, received at 2023-09-09T23:16:14.543328+0000: client_request(mds.1:1857 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:30.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:30 smithi082 ceph-mon[98694]: slow request 194.468877 seconds old, received at 2023-09-09T23:16:14.596100+0000: client_request(mds.1:1889 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:30.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:30 smithi098 ceph-mon[103124]: 341 slow requests, 5 included below; oldest blocked for > 195.667048 secs 2023-09-09T23:19:30.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:30 smithi098 ceph-mon[103124]: slow request 194.692997 seconds old, received at 2023-09-09T23:16:14.371980+0000: client_request(mds.1:1761 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:30.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:30 smithi098 ceph-mon[103124]: slow request 194.635167 seconds old, received at 2023-09-09T23:16:14.429810+0000: client_request(mds.1:1793 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:30.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:30 smithi098 ceph-mon[103124]: slow request 194.576073 seconds old, received at 2023-09-09T23:16:14.488904+0000: client_request(mds.1:1825 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:30.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:30 smithi098 ceph-mon[103124]: slow request 194.521649 seconds old, received at 2023-09-09T23:16:14.543328+0000: client_request(mds.1:1857 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:30.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:30 smithi098 ceph-mon[103124]: slow request 194.468877 seconds old, received at 2023-09-09T23:16:14.596100+0000: client_request(mds.1:1889 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:31.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:31 smithi170 ceph-mon[102832]: pgmap v171: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 45 KiB/s rd, 10 MiB/s wr, 27 op/s 2023-09-09T23:19:31.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:19:31.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:31.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:31 smithi082 ceph-mon[98694]: pgmap v171: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 45 KiB/s rd, 10 MiB/s wr, 27 op/s 2023-09-09T23:19:31.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:19:31.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:31.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:31 smithi098 ceph-mon[103124]: pgmap v171: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 45 KiB/s rd, 10 MiB/s wr, 27 op/s 2023-09-09T23:19:31.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:19:31.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:32.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:19:32.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:19:32.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:32.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:32 smithi170 ceph-mon[102832]: 47 slow requests, 5 included below; oldest blocked for > 185.911162 secs 2023-09-09T23:19:32.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:32 smithi170 ceph-mon[102832]: slow request 185.855299 seconds old, received at 2023-09-09T23:16:25.831881+0000: peer_request:mds.1:1451 currently dispatched 2023-09-09T23:19:32.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:32 smithi170 ceph-mon[102832]: slow request 185.844135 seconds old, received at 2023-09-09T23:16:25.843045+0000: peer_request:mds.1:1483 currently dispatched 2023-09-09T23:19:32.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:32 smithi170 ceph-mon[102832]: slow request 185.832806 seconds old, received at 2023-09-09T23:16:25.854375+0000: peer_request:mds.1:1515 currently dispatched 2023-09-09T23:19:32.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:32 smithi170 ceph-mon[102832]: slow request 185.820839 seconds old, received at 2023-09-09T23:16:25.866341+0000: peer_request:mds.1:1547 currently dispatched 2023-09-09T23:19:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:32 smithi170 ceph-mon[102832]: slow request 185.809668 seconds old, received at 2023-09-09T23:16:25.877512+0000: peer_request:mds.1:1579 currently dispatched 2023-09-09T23:19:32.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:19:32.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:19:32.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:32.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:32 smithi082 ceph-mon[98694]: 47 slow requests, 5 included below; oldest blocked for > 185.911162 secs 2023-09-09T23:19:32.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:32 smithi082 ceph-mon[98694]: slow request 185.855299 seconds old, received at 2023-09-09T23:16:25.831881+0000: peer_request:mds.1:1451 currently dispatched 2023-09-09T23:19:32.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:32 smithi082 ceph-mon[98694]: slow request 185.844135 seconds old, received at 2023-09-09T23:16:25.843045+0000: peer_request:mds.1:1483 currently dispatched 2023-09-09T23:19:32.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:32 smithi082 ceph-mon[98694]: slow request 185.832806 seconds old, received at 2023-09-09T23:16:25.854375+0000: peer_request:mds.1:1515 currently dispatched 2023-09-09T23:19:32.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:32 smithi082 ceph-mon[98694]: slow request 185.820839 seconds old, received at 2023-09-09T23:16:25.866341+0000: peer_request:mds.1:1547 currently dispatched 2023-09-09T23:19:32.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:32 smithi082 ceph-mon[98694]: slow request 185.809668 seconds old, received at 2023-09-09T23:16:25.877512+0000: peer_request:mds.1:1579 currently dispatched 2023-09-09T23:19:32.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:19:32.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:19:32.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:32.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:32 smithi098 ceph-mon[103124]: 47 slow requests, 5 included below; oldest blocked for > 185.911162 secs 2023-09-09T23:19:32.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:32 smithi098 ceph-mon[103124]: slow request 185.855299 seconds old, received at 2023-09-09T23:16:25.831881+0000: peer_request:mds.1:1451 currently dispatched 2023-09-09T23:19:32.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:32 smithi098 ceph-mon[103124]: slow request 185.844135 seconds old, received at 2023-09-09T23:16:25.843045+0000: peer_request:mds.1:1483 currently dispatched 2023-09-09T23:19:32.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:32 smithi098 ceph-mon[103124]: slow request 185.832806 seconds old, received at 2023-09-09T23:16:25.854375+0000: peer_request:mds.1:1515 currently dispatched 2023-09-09T23:19:32.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:32 smithi098 ceph-mon[103124]: slow request 185.820839 seconds old, received at 2023-09-09T23:16:25.866341+0000: peer_request:mds.1:1547 currently dispatched 2023-09-09T23:19:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:32 smithi098 ceph-mon[103124]: slow request 185.809668 seconds old, received at 2023-09-09T23:16:25.877512+0000: peer_request:mds.1:1579 currently dispatched 2023-09-09T23:19:33.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:33 smithi170 ceph-mon[102832]: pgmap v172: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 38 KiB/s rd, 12 MiB/s wr, 25 op/s 2023-09-09T23:19:33.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:33 smithi082 ceph-mon[98694]: pgmap v172: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 38 KiB/s rd, 12 MiB/s wr, 25 op/s 2023-09-09T23:19:33.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:33 smithi098 ceph-mon[103124]: pgmap v172: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 38 KiB/s rd, 12 MiB/s wr, 25 op/s 2023-09-09T23:19:35.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:35 smithi170 ceph-mon[102832]: 329 slow requests, 5 included below; oldest blocked for > 200.743778 secs 2023-09-09T23:19:35.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:35 smithi170 ceph-mon[102832]: slow request 199.516277 seconds old, received at 2023-09-09T23:16:14.649713+0000: client_request(mds.1:1921 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:35.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:35 smithi170 ceph-mon[102832]: slow request 199.461076 seconds old, received at 2023-09-09T23:16:14.704914+0000: client_request(mds.1:1953 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:35.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:35 smithi170 ceph-mon[102832]: slow request 199.402864 seconds old, received at 2023-09-09T23:16:14.763126+0000: client_request(mds.1:1985 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:35.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:35 smithi170 ceph-mon[102832]: slow request 199.342746 seconds old, received at 2023-09-09T23:16:14.823245+0000: client_request(mds.1:2017 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:35.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:35 smithi170 ceph-mon[102832]: slow request 199.289119 seconds old, received at 2023-09-09T23:16:14.876871+0000: client_request(mds.1:2049 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:35.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:35 smithi170 ceph-mon[102832]: pgmap v173: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 22 op/s 2023-09-09T23:19:35.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:35 smithi082 ceph-mon[98694]: 329 slow requests, 5 included below; oldest blocked for > 200.743778 secs 2023-09-09T23:19:35.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:35 smithi082 ceph-mon[98694]: slow request 199.516277 seconds old, received at 2023-09-09T23:16:14.649713+0000: client_request(mds.1:1921 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:35.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:35 smithi082 ceph-mon[98694]: slow request 199.461076 seconds old, received at 2023-09-09T23:16:14.704914+0000: client_request(mds.1:1953 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:35.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:35 smithi082 ceph-mon[98694]: slow request 199.402864 seconds old, received at 2023-09-09T23:16:14.763126+0000: client_request(mds.1:1985 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:35.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:35 smithi082 ceph-mon[98694]: slow request 199.342746 seconds old, received at 2023-09-09T23:16:14.823245+0000: client_request(mds.1:2017 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:35.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:35 smithi082 ceph-mon[98694]: slow request 199.289119 seconds old, received at 2023-09-09T23:16:14.876871+0000: client_request(mds.1:2049 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:35.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:35 smithi082 ceph-mon[98694]: pgmap v173: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 22 op/s 2023-09-09T23:19:35.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:35 smithi098 ceph-mon[103124]: 329 slow requests, 5 included below; oldest blocked for > 200.743778 secs 2023-09-09T23:19:35.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:35 smithi098 ceph-mon[103124]: slow request 199.516277 seconds old, received at 2023-09-09T23:16:14.649713+0000: client_request(mds.1:1921 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:35.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:35 smithi098 ceph-mon[103124]: slow request 199.461076 seconds old, received at 2023-09-09T23:16:14.704914+0000: client_request(mds.1:1953 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:35.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:35 smithi098 ceph-mon[103124]: slow request 199.402864 seconds old, received at 2023-09-09T23:16:14.763126+0000: client_request(mds.1:1985 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:35.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:35 smithi098 ceph-mon[103124]: slow request 199.342746 seconds old, received at 2023-09-09T23:16:14.823245+0000: client_request(mds.1:2017 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:35.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:35 smithi098 ceph-mon[103124]: slow request 199.289119 seconds old, received at 2023-09-09T23:16:14.876871+0000: client_request(mds.1:2049 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:35.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:35 smithi098 ceph-mon[103124]: pgmap v173: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 22 op/s 2023-09-09T23:19:36.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:19:36.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:36.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:19:36.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:36.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:19:36.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:37.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:37 smithi082 ceph-mon[98694]: pgmap v174: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 43 KiB/s rd, 10 MiB/s wr, 28 op/s 2023-09-09T23:19:37.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:37 smithi082 ceph-mon[98694]: 668 slow requests, 5 included below; oldest blocked for > 190.908429 secs 2023-09-09T23:19:37.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:37 smithi082 ceph-mon[98694]: slow request 190.798525 seconds old, received at 2023-09-09T23:16:25.888716+0000: peer_request:mds.1:1611 currently dispatched 2023-09-09T23:19:37.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:37 smithi082 ceph-mon[98694]: slow request 190.787737 seconds old, received at 2023-09-09T23:16:25.899505+0000: peer_request:mds.1:1643 currently dispatched 2023-09-09T23:19:37.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:37 smithi082 ceph-mon[98694]: slow request 190.776421 seconds old, received at 2023-09-09T23:16:25.910820+0000: peer_request:mds.1:1675 currently dispatched 2023-09-09T23:19:37.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:37 smithi082 ceph-mon[98694]: slow request 190.764624 seconds old, received at 2023-09-09T23:16:25.922617+0000: peer_request:mds.1:1707 currently dispatched 2023-09-09T23:19:37.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:37 smithi082 ceph-mon[98694]: slow request 190.752519 seconds old, received at 2023-09-09T23:16:25.934723+0000: peer_request:mds.1:1739 currently dispatched 2023-09-09T23:19:37.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:37 smithi098 ceph-mon[103124]: pgmap v174: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 43 KiB/s rd, 10 MiB/s wr, 28 op/s 2023-09-09T23:19:37.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:37 smithi098 ceph-mon[103124]: 668 slow requests, 5 included below; oldest blocked for > 190.908429 secs 2023-09-09T23:19:37.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:37 smithi098 ceph-mon[103124]: slow request 190.798525 seconds old, received at 2023-09-09T23:16:25.888716+0000: peer_request:mds.1:1611 currently dispatched 2023-09-09T23:19:37.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:37 smithi098 ceph-mon[103124]: slow request 190.787737 seconds old, received at 2023-09-09T23:16:25.899505+0000: peer_request:mds.1:1643 currently dispatched 2023-09-09T23:19:37.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:37 smithi098 ceph-mon[103124]: slow request 190.776421 seconds old, received at 2023-09-09T23:16:25.910820+0000: peer_request:mds.1:1675 currently dispatched 2023-09-09T23:19:37.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:37 smithi098 ceph-mon[103124]: slow request 190.764624 seconds old, received at 2023-09-09T23:16:25.922617+0000: peer_request:mds.1:1707 currently dispatched 2023-09-09T23:19:37.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:37 smithi098 ceph-mon[103124]: slow request 190.752519 seconds old, received at 2023-09-09T23:16:25.934723+0000: peer_request:mds.1:1739 currently dispatched 2023-09-09T23:19:37.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:37 smithi170 ceph-mon[102832]: pgmap v174: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 43 KiB/s rd, 10 MiB/s wr, 28 op/s 2023-09-09T23:19:37.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:37 smithi170 ceph-mon[102832]: 668 slow requests, 5 included below; oldest blocked for > 190.908429 secs 2023-09-09T23:19:37.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:37 smithi170 ceph-mon[102832]: slow request 190.798525 seconds old, received at 2023-09-09T23:16:25.888716+0000: peer_request:mds.1:1611 currently dispatched 2023-09-09T23:19:37.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:37 smithi170 ceph-mon[102832]: slow request 190.787737 seconds old, received at 2023-09-09T23:16:25.899505+0000: peer_request:mds.1:1643 currently dispatched 2023-09-09T23:19:37.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:37 smithi170 ceph-mon[102832]: slow request 190.776421 seconds old, received at 2023-09-09T23:16:25.910820+0000: peer_request:mds.1:1675 currently dispatched 2023-09-09T23:19:37.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:37 smithi170 ceph-mon[102832]: slow request 190.764624 seconds old, received at 2023-09-09T23:16:25.922617+0000: peer_request:mds.1:1707 currently dispatched 2023-09-09T23:19:37.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:37 smithi170 ceph-mon[102832]: slow request 190.752519 seconds old, received at 2023-09-09T23:16:25.934723+0000: peer_request:mds.1:1739 currently dispatched 2023-09-09T23:19:39.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:39 smithi082 ceph-mon[98694]: pgmap v175: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 24 KiB/s rd, 6.3 MiB/s wr, 17 op/s 2023-09-09T23:19:39.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:39 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:19:39.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:39 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:39.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:39 smithi098 ceph-mon[103124]: pgmap v175: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 24 KiB/s rd, 6.3 MiB/s wr, 17 op/s 2023-09-09T23:19:39.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:39 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:19:39.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:39 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:39.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:39 smithi170 ceph-mon[102832]: pgmap v175: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 24 KiB/s rd, 6.3 MiB/s wr, 17 op/s 2023-09-09T23:19:39.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:39 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:19:39.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:39 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:40.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:19:40.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:40.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:40 smithi082 ceph-mon[98694]: 329 slow requests, 5 included below; oldest blocked for > 206.074931 secs 2023-09-09T23:19:40.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:40 smithi082 ceph-mon[98694]: slow request 206.058127 seconds old, received at 2023-09-09T23:16:13.455964+0000: client_request(mds.1:1314 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:40.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:40 smithi082 ceph-mon[98694]: slow request 205.999712 seconds old, received at 2023-09-09T23:16:13.514380+0000: client_request(mds.1:1346 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:40.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:40 smithi082 ceph-mon[98694]: slow request 205.940106 seconds old, received at 2023-09-09T23:16:13.573986+0000: client_request(mds.1:1378 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:40.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:40 smithi082 ceph-mon[98694]: slow request 205.883544 seconds old, received at 2023-09-09T23:16:13.630548+0000: client_request(mds.1:1410 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:40.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:40 smithi082 ceph-mon[98694]: slow request 205.824796 seconds old, received at 2023-09-09T23:16:13.689296+0000: client_request(mds.1:1442 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:40.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:19:40.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:40.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:40 smithi098 ceph-mon[103124]: 329 slow requests, 5 included below; oldest blocked for > 206.074931 secs 2023-09-09T23:19:40.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:40 smithi098 ceph-mon[103124]: slow request 206.058127 seconds old, received at 2023-09-09T23:16:13.455964+0000: client_request(mds.1:1314 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:40.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:40 smithi098 ceph-mon[103124]: slow request 205.999712 seconds old, received at 2023-09-09T23:16:13.514380+0000: client_request(mds.1:1346 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:40 smithi098 ceph-mon[103124]: slow request 205.940106 seconds old, received at 2023-09-09T23:16:13.573986+0000: client_request(mds.1:1378 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:40 smithi098 ceph-mon[103124]: slow request 205.883544 seconds old, received at 2023-09-09T23:16:13.630548+0000: client_request(mds.1:1410 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:40 smithi098 ceph-mon[103124]: slow request 205.824796 seconds old, received at 2023-09-09T23:16:13.689296+0000: client_request(mds.1:1442 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:40.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:19:40.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:40.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:40 smithi170 ceph-mon[102832]: 329 slow requests, 5 included below; oldest blocked for > 206.074931 secs 2023-09-09T23:19:40.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:40 smithi170 ceph-mon[102832]: slow request 206.058127 seconds old, received at 2023-09-09T23:16:13.455964+0000: client_request(mds.1:1314 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:40.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:40 smithi170 ceph-mon[102832]: slow request 205.999712 seconds old, received at 2023-09-09T23:16:13.514380+0000: client_request(mds.1:1346 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:40.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:40 smithi170 ceph-mon[102832]: slow request 205.940106 seconds old, received at 2023-09-09T23:16:13.573986+0000: client_request(mds.1:1378 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:40.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:40 smithi170 ceph-mon[102832]: slow request 205.883544 seconds old, received at 2023-09-09T23:16:13.630548+0000: client_request(mds.1:1410 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:40 smithi170 ceph-mon[102832]: slow request 205.824796 seconds old, received at 2023-09-09T23:16:13.689296+0000: client_request(mds.1:1442 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:41.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:41 smithi082 ceph-mon[98694]: pgmap v176: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 31 KiB/s rd, 6.3 MiB/s wr, 22 op/s 2023-09-09T23:19:41.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:41 smithi098 ceph-mon[103124]: pgmap v176: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 31 KiB/s rd, 6.3 MiB/s wr, 22 op/s 2023-09-09T23:19:41.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:41 smithi170 ceph-mon[102832]: pgmap v176: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 31 KiB/s rd, 6.3 MiB/s wr, 22 op/s 2023-09-09T23:19:42.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:42 smithi082 ceph-mon[98694]: 659 slow requests, 5 included below; oldest blocked for > 195.905793 secs 2023-09-09T23:19:42.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:42 smithi082 ceph-mon[98694]: slow request 195.741220 seconds old, received at 2023-09-09T23:16:25.946070+0000: peer_request:mds.1:1771 currently dispatched 2023-09-09T23:19:42.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:42 smithi082 ceph-mon[98694]: slow request 195.730989 seconds old, received at 2023-09-09T23:16:25.956301+0000: peer_request:mds.1:1803 currently dispatched 2023-09-09T23:19:42.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:42 smithi082 ceph-mon[98694]: slow request 195.719545 seconds old, received at 2023-09-09T23:16:25.967745+0000: peer_request:mds.1:1835 currently dispatched 2023-09-09T23:19:42.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:42 smithi082 ceph-mon[98694]: slow request 195.708024 seconds old, received at 2023-09-09T23:16:25.979266+0000: peer_request:mds.1:1867 currently dispatched 2023-09-09T23:19:42.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:42 smithi082 ceph-mon[98694]: slow request 195.697103 seconds old, received at 2023-09-09T23:16:25.990188+0000: peer_request:mds.1:1899 currently dispatched 2023-09-09T23:19:42.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:42 smithi098 ceph-mon[103124]: 659 slow requests, 5 included below; oldest blocked for > 195.905793 secs 2023-09-09T23:19:42.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:42 smithi098 ceph-mon[103124]: slow request 195.741220 seconds old, received at 2023-09-09T23:16:25.946070+0000: peer_request:mds.1:1771 currently dispatched 2023-09-09T23:19:42.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:42 smithi098 ceph-mon[103124]: slow request 195.730989 seconds old, received at 2023-09-09T23:16:25.956301+0000: peer_request:mds.1:1803 currently dispatched 2023-09-09T23:19:42.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:42 smithi098 ceph-mon[103124]: slow request 195.719545 seconds old, received at 2023-09-09T23:16:25.967745+0000: peer_request:mds.1:1835 currently dispatched 2023-09-09T23:19:42.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:42 smithi098 ceph-mon[103124]: slow request 195.708024 seconds old, received at 2023-09-09T23:16:25.979266+0000: peer_request:mds.1:1867 currently dispatched 2023-09-09T23:19:42.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:42 smithi098 ceph-mon[103124]: slow request 195.697103 seconds old, received at 2023-09-09T23:16:25.990188+0000: peer_request:mds.1:1899 currently dispatched 2023-09-09T23:19:42.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:42 smithi170 ceph-mon[102832]: 659 slow requests, 5 included below; oldest blocked for > 195.905793 secs 2023-09-09T23:19:42.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:42 smithi170 ceph-mon[102832]: slow request 195.741220 seconds old, received at 2023-09-09T23:16:25.946070+0000: peer_request:mds.1:1771 currently dispatched 2023-09-09T23:19:42.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:42 smithi170 ceph-mon[102832]: slow request 195.730989 seconds old, received at 2023-09-09T23:16:25.956301+0000: peer_request:mds.1:1803 currently dispatched 2023-09-09T23:19:42.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:42 smithi170 ceph-mon[102832]: slow request 195.719545 seconds old, received at 2023-09-09T23:16:25.967745+0000: peer_request:mds.1:1835 currently dispatched 2023-09-09T23:19:42.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:42 smithi170 ceph-mon[102832]: slow request 195.708024 seconds old, received at 2023-09-09T23:16:25.979266+0000: peer_request:mds.1:1867 currently dispatched 2023-09-09T23:19:42.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:42 smithi170 ceph-mon[102832]: slow request 195.697103 seconds old, received at 2023-09-09T23:16:25.990188+0000: peer_request:mds.1:1899 currently dispatched 2023-09-09T23:19:43.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:43 smithi098 ceph-mon[103124]: pgmap v177: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 19 KiB/s rd, 1.4 MiB/s wr, 15 op/s 2023-09-09T23:19:43.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:43 smithi170 ceph-mon[102832]: pgmap v177: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 19 KiB/s rd, 1.4 MiB/s wr, 15 op/s 2023-09-09T23:19:43.954 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:43 smithi082 ceph-mon[98694]: pgmap v177: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 19 KiB/s rd, 1.4 MiB/s wr, 15 op/s 2023-09-09T23:19:44.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:19:44.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:44.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:19:44.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:44.954 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:19:44.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:45.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:45 smithi098 ceph-mon[103124]: pgmap v178: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 11 op/s 2023-09-09T23:19:45.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:45 smithi098 ceph-mon[103124]: 327 slow requests, 5 included below; oldest blocked for > 211.142726 secs 2023-09-09T23:19:45.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:45 smithi098 ceph-mon[103124]: slow request 210.747539 seconds old, received at 2023-09-09T23:16:13.853187+0000: client_request(mds.1:1474 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:45.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:45 smithi098 ceph-mon[103124]: slow request 210.687287 seconds old, received at 2023-09-09T23:16:13.913439+0000: client_request(mds.1:1506 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:45.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:45 smithi098 ceph-mon[103124]: slow request 210.631808 seconds old, received at 2023-09-09T23:16:13.968918+0000: client_request(mds.1:1538 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:45.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:45 smithi098 ceph-mon[103124]: slow request 210.571458 seconds old, received at 2023-09-09T23:16:14.029268+0000: client_request(mds.1:1570 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:45.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:45 smithi098 ceph-mon[103124]: slow request 210.515645 seconds old, received at 2023-09-09T23:16:14.085081+0000: client_request(mds.1:1602 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:45.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:45 smithi170 ceph-mon[102832]: pgmap v178: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 11 op/s 2023-09-09T23:19:45.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:45 smithi170 ceph-mon[102832]: 327 slow requests, 5 included below; oldest blocked for > 211.142726 secs 2023-09-09T23:19:45.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:45 smithi170 ceph-mon[102832]: slow request 210.747539 seconds old, received at 2023-09-09T23:16:13.853187+0000: client_request(mds.1:1474 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:45.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:45 smithi170 ceph-mon[102832]: slow request 210.687287 seconds old, received at 2023-09-09T23:16:13.913439+0000: client_request(mds.1:1506 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:45 smithi170 ceph-mon[102832]: slow request 210.631808 seconds old, received at 2023-09-09T23:16:13.968918+0000: client_request(mds.1:1538 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:45 smithi170 ceph-mon[102832]: slow request 210.571458 seconds old, received at 2023-09-09T23:16:14.029268+0000: client_request(mds.1:1570 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:45 smithi170 ceph-mon[102832]: slow request 210.515645 seconds old, received at 2023-09-09T23:16:14.085081+0000: client_request(mds.1:1602 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:45.954 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:45 smithi082 ceph-mon[98694]: pgmap v178: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 11 op/s 2023-09-09T23:19:45.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:45 smithi082 ceph-mon[98694]: 327 slow requests, 5 included below; oldest blocked for > 211.142726 secs 2023-09-09T23:19:45.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:45 smithi082 ceph-mon[98694]: slow request 210.747539 seconds old, received at 2023-09-09T23:16:13.853187+0000: client_request(mds.1:1474 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:45.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:45 smithi082 ceph-mon[98694]: slow request 210.687287 seconds old, received at 2023-09-09T23:16:13.913439+0000: client_request(mds.1:1506 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:45.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:45 smithi082 ceph-mon[98694]: slow request 210.631808 seconds old, received at 2023-09-09T23:16:13.968918+0000: client_request(mds.1:1538 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:45.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:45 smithi082 ceph-mon[98694]: slow request 210.571458 seconds old, received at 2023-09-09T23:16:14.029268+0000: client_request(mds.1:1570 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:45.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:45 smithi082 ceph-mon[98694]: slow request 210.515645 seconds old, received at 2023-09-09T23:16:14.085081+0000: client_request(mds.1:1602 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:46.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:46 smithi098 ceph-mon[103124]: pgmap v179: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 15 op/s 2023-09-09T23:19:46.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:46 smithi170 ceph-mon[102832]: pgmap v179: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 15 op/s 2023-09-09T23:19:46.954 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:46 smithi082 ceph-mon[98694]: pgmap v179: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 15 op/s 2023-09-09T23:19:47.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:47 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:19:47.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:47 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:47.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:47 smithi098 ceph-mon[103124]: 649 slow requests, 5 included below; oldest blocked for > 200.902559 secs 2023-09-09T23:19:47.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:47 smithi098 ceph-mon[103124]: slow request 200.687742 seconds old, received at 2023-09-09T23:16:25.999649+0000: peer_request:mds.1:1931 currently dispatched 2023-09-09T23:19:47.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:47 smithi098 ceph-mon[103124]: slow request 200.675291 seconds old, received at 2023-09-09T23:16:26.012101+0000: peer_request:mds.1:1963 currently dispatched 2023-09-09T23:19:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:47 smithi098 ceph-mon[103124]: slow request 200.663356 seconds old, received at 2023-09-09T23:16:26.024036+0000: peer_request:mds.1:1995 currently dispatched 2023-09-09T23:19:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:47 smithi098 ceph-mon[103124]: slow request 200.651668 seconds old, received at 2023-09-09T23:16:26.035724+0000: peer_request:mds.1:2027 currently dispatched 2023-09-09T23:19:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:47 smithi098 ceph-mon[103124]: slow request 200.900685 seconds old, received at 2023-09-09T23:16:25.786707+0000: peer_request:mds.1:1324 currently dispatched 2023-09-09T23:19:47.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:47 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:19:47.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:47 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:47.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:47 smithi170 ceph-mon[102832]: 649 slow requests, 5 included below; oldest blocked for > 200.902559 secs 2023-09-09T23:19:47.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:47 smithi170 ceph-mon[102832]: slow request 200.687742 seconds old, received at 2023-09-09T23:16:25.999649+0000: peer_request:mds.1:1931 currently dispatched 2023-09-09T23:19:47.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:47 smithi170 ceph-mon[102832]: slow request 200.675291 seconds old, received at 2023-09-09T23:16:26.012101+0000: peer_request:mds.1:1963 currently dispatched 2023-09-09T23:19:47.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:47 smithi170 ceph-mon[102832]: slow request 200.663356 seconds old, received at 2023-09-09T23:16:26.024036+0000: peer_request:mds.1:1995 currently dispatched 2023-09-09T23:19:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:47 smithi170 ceph-mon[102832]: slow request 200.651668 seconds old, received at 2023-09-09T23:16:26.035724+0000: peer_request:mds.1:2027 currently dispatched 2023-09-09T23:19:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:47 smithi170 ceph-mon[102832]: slow request 200.900685 seconds old, received at 2023-09-09T23:16:25.786707+0000: peer_request:mds.1:1324 currently dispatched 2023-09-09T23:19:47.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:47 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:19:47.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:47 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:47.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:47 smithi082 ceph-mon[98694]: 649 slow requests, 5 included below; oldest blocked for > 200.902559 secs 2023-09-09T23:19:47.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:47 smithi082 ceph-mon[98694]: slow request 200.687742 seconds old, received at 2023-09-09T23:16:25.999649+0000: peer_request:mds.1:1931 currently dispatched 2023-09-09T23:19:47.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:47 smithi082 ceph-mon[98694]: slow request 200.675291 seconds old, received at 2023-09-09T23:16:26.012101+0000: peer_request:mds.1:1963 currently dispatched 2023-09-09T23:19:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:47 smithi082 ceph-mon[98694]: slow request 200.663356 seconds old, received at 2023-09-09T23:16:26.024036+0000: peer_request:mds.1:1995 currently dispatched 2023-09-09T23:19:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:47 smithi082 ceph-mon[98694]: slow request 200.651668 seconds old, received at 2023-09-09T23:16:26.035724+0000: peer_request:mds.1:2027 currently dispatched 2023-09-09T23:19:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:47 smithi082 ceph-mon[98694]: slow request 200.900685 seconds old, received at 2023-09-09T23:16:25.786707+0000: peer_request:mds.1:1324 currently dispatched 2023-09-09T23:19:49.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:49 smithi170 ceph-mon[102832]: pgmap v180: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 KiB/s wr, 10 op/s 2023-09-09T23:19:49.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:49 smithi082 ceph-mon[98694]: pgmap v180: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 KiB/s wr, 10 op/s 2023-09-09T23:19:49.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:49 smithi098 ceph-mon[103124]: pgmap v180: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 KiB/s wr, 10 op/s 2023-09-09T23:19:50.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:50 smithi170 ceph-mon[102832]: 317 slow requests, 5 included below; oldest blocked for > 216.161285 secs 2023-09-09T23:19:50.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:50 smithi170 ceph-mon[102832]: slow request 215.495011 seconds old, received at 2023-09-09T23:16:14.142898+0000: client_request(mds.1:1634 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:50.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:50 smithi170 ceph-mon[102832]: slow request 215.438607 seconds old, received at 2023-09-09T23:16:14.199302+0000: client_request(mds.1:1666 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:50 smithi170 ceph-mon[102832]: slow request 215.381645 seconds old, received at 2023-09-09T23:16:14.256264+0000: client_request(mds.1:1698 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:50 smithi170 ceph-mon[102832]: slow request 215.322264 seconds old, received at 2023-09-09T23:16:14.315645+0000: client_request(mds.1:1730 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:50 smithi170 ceph-mon[102832]: slow request 215.264128 seconds old, received at 2023-09-09T23:16:14.373781+0000: client_request(mds.1:1762 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:50.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:50 smithi082 ceph-mon[98694]: 317 slow requests, 5 included below; oldest blocked for > 216.161285 secs 2023-09-09T23:19:50.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:50 smithi082 ceph-mon[98694]: slow request 215.495011 seconds old, received at 2023-09-09T23:16:14.142898+0000: client_request(mds.1:1634 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:50.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:50 smithi082 ceph-mon[98694]: slow request 215.438607 seconds old, received at 2023-09-09T23:16:14.199302+0000: client_request(mds.1:1666 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:50.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:50 smithi082 ceph-mon[98694]: slow request 215.381645 seconds old, received at 2023-09-09T23:16:14.256264+0000: client_request(mds.1:1698 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:50.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:50 smithi082 ceph-mon[98694]: slow request 215.322264 seconds old, received at 2023-09-09T23:16:14.315645+0000: client_request(mds.1:1730 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:50 smithi082 ceph-mon[98694]: slow request 215.264128 seconds old, received at 2023-09-09T23:16:14.373781+0000: client_request(mds.1:1762 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:50.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:50 smithi098 ceph-mon[103124]: 317 slow requests, 5 included below; oldest blocked for > 216.161285 secs 2023-09-09T23:19:50.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:50 smithi098 ceph-mon[103124]: slow request 215.495011 seconds old, received at 2023-09-09T23:16:14.142898+0000: client_request(mds.1:1634 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:50.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:50 smithi098 ceph-mon[103124]: slow request 215.438607 seconds old, received at 2023-09-09T23:16:14.199302+0000: client_request(mds.1:1666 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:50.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:50 smithi098 ceph-mon[103124]: slow request 215.381645 seconds old, received at 2023-09-09T23:16:14.256264+0000: client_request(mds.1:1698 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:50 smithi098 ceph-mon[103124]: slow request 215.322264 seconds old, received at 2023-09-09T23:16:14.315645+0000: client_request(mds.1:1730 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:50 smithi098 ceph-mon[103124]: slow request 215.264128 seconds old, received at 2023-09-09T23:16:14.373781+0000: client_request(mds.1:1762 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:51.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:51 smithi170 ceph-mon[102832]: pgmap v181: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 KiB/s wr, 14 op/s 2023-09-09T23:19:51.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:51.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:51 smithi082 ceph-mon[98694]: pgmap v181: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 KiB/s wr, 14 op/s 2023-09-09T23:19:51.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:51.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:51 smithi098 ceph-mon[103124]: pgmap v181: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 KiB/s wr, 14 op/s 2023-09-09T23:19:51.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:52.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:19:52.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:52.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:52 smithi170 ceph-mon[102832]: 643 slow requests, 5 included below; oldest blocked for > 205.899468 secs 2023-09-09T23:19:52.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:52 smithi170 ceph-mon[102832]: slow request 205.889949 seconds old, received at 2023-09-09T23:16:25.797503+0000: peer_request:mds.1:1356 currently dispatched 2023-09-09T23:19:52.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:52 smithi170 ceph-mon[102832]: slow request 205.878316 seconds old, received at 2023-09-09T23:16:25.809136+0000: peer_request:mds.1:1388 currently dispatched 2023-09-09T23:19:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:52 smithi170 ceph-mon[102832]: slow request 205.866884 seconds old, received at 2023-09-09T23:16:25.820568+0000: peer_request:mds.1:1420 currently dispatched 2023-09-09T23:19:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:52 smithi170 ceph-mon[102832]: slow request 205.855110 seconds old, received at 2023-09-09T23:16:25.832342+0000: peer_request:mds.1:1452 currently dispatched 2023-09-09T23:19:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:52 smithi170 ceph-mon[102832]: slow request 205.844188 seconds old, received at 2023-09-09T23:16:25.843264+0000: peer_request:mds.1:1484 currently dispatched 2023-09-09T23:19:52.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:19:52.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:52.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:52 smithi082 ceph-mon[98694]: 643 slow requests, 5 included below; oldest blocked for > 205.899468 secs 2023-09-09T23:19:52.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:52 smithi082 ceph-mon[98694]: slow request 205.889949 seconds old, received at 2023-09-09T23:16:25.797503+0000: peer_request:mds.1:1356 currently dispatched 2023-09-09T23:19:52.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:52 smithi082 ceph-mon[98694]: slow request 205.878316 seconds old, received at 2023-09-09T23:16:25.809136+0000: peer_request:mds.1:1388 currently dispatched 2023-09-09T23:19:52.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:52 smithi082 ceph-mon[98694]: slow request 205.866884 seconds old, received at 2023-09-09T23:16:25.820568+0000: peer_request:mds.1:1420 currently dispatched 2023-09-09T23:19:52.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:52 smithi082 ceph-mon[98694]: slow request 205.855110 seconds old, received at 2023-09-09T23:16:25.832342+0000: peer_request:mds.1:1452 currently dispatched 2023-09-09T23:19:52.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:52 smithi082 ceph-mon[98694]: slow request 205.844188 seconds old, received at 2023-09-09T23:16:25.843264+0000: peer_request:mds.1:1484 currently dispatched 2023-09-09T23:19:52.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:19:52.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:52.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:52 smithi098 ceph-mon[103124]: 643 slow requests, 5 included below; oldest blocked for > 205.899468 secs 2023-09-09T23:19:52.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:52 smithi098 ceph-mon[103124]: slow request 205.889949 seconds old, received at 2023-09-09T23:16:25.797503+0000: peer_request:mds.1:1356 currently dispatched 2023-09-09T23:19:52.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:52 smithi098 ceph-mon[103124]: slow request 205.878316 seconds old, received at 2023-09-09T23:16:25.809136+0000: peer_request:mds.1:1388 currently dispatched 2023-09-09T23:19:52.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:52 smithi098 ceph-mon[103124]: slow request 205.866884 seconds old, received at 2023-09-09T23:16:25.820568+0000: peer_request:mds.1:1420 currently dispatched 2023-09-09T23:19:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:52 smithi098 ceph-mon[103124]: slow request 205.855110 seconds old, received at 2023-09-09T23:16:25.832342+0000: peer_request:mds.1:1452 currently dispatched 2023-09-09T23:19:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:52 smithi098 ceph-mon[103124]: slow request 205.844188 seconds old, received at 2023-09-09T23:16:25.843264+0000: peer_request:mds.1:1484 currently dispatched 2023-09-09T23:19:53.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:53 smithi170 ceph-mon[102832]: pgmap v182: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2023-09-09T23:19:53.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:53 smithi082 ceph-mon[98694]: pgmap v182: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2023-09-09T23:19:53.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:53 smithi098 ceph-mon[103124]: pgmap v182: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2023-09-09T23:19:55.122 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:19:55.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.276+0000 7f82b9256700 1 -- 172.21.15.82:0/633878621 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82b41054f0 msgr2=0x7f82b41058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:19:55.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.276+0000 7f82b9256700 1 --2- 172.21.15.82:0/633878621 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82b41054f0 0x7f82b41058d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f82a8009990 tx=0x7f82a8009ca0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.276+0000 7f82b9256700 1 -- 172.21.15.82:0/633878621 shutdown_connections 2023-09-09T23:19:55.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.276+0000 7f82b9256700 1 --2- 172.21.15.82:0/633878621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82b40ffe10 0x7f82b410c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.276+0000 7f82b9256700 1 --2- 172.21.15.82:0/633878621 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82b41054f0 0x7f82b41058d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.276+0000 7f82b9256700 1 --2- 172.21.15.82:0/633878621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82b40ff450 0x7f82b40ff8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.276+0000 7f82b9256700 1 -- 172.21.15.82:0/633878621 >> 172.21.15.82:0/633878621 conn(0x7f82b40faf20 msgr2=0x7f82b40fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:19:55.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.276+0000 7f82b9256700 1 -- 172.21.15.82:0/633878621 shutdown_connections 2023-09-09T23:19:55.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.276+0000 7f82b9256700 1 -- 172.21.15.82:0/633878621 wait complete. 2023-09-09T23:19:55.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.277+0000 7f82b9256700 1 Processor -- start 2023-09-09T23:19:55.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.277+0000 7f82b9256700 1 -- start start 2023-09-09T23:19:55.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.277+0000 7f82b9256700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82b40ff450 0x7f82b4197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:55.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.277+0000 7f82b9256700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82b40ffe10 0x7f82b41979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:55.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.277+0000 7f82b9256700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82b41054f0 0x7f82b41a2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:55.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.277+0000 7f82b9256700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b4198310 con 0x7f82b40ffe10 2023-09-09T23:19:55.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.277+0000 7f82b9256700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b41a2a50 con 0x7f82b41054f0 2023-09-09T23:19:55.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.277+0000 7f82b9256700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b41a2c10 con 0x7f82b40ff450 2023-09-09T23:19:55.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.277+0000 7f82b259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82b40ffe10 0x7f82b41979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:55.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.278+0000 7f82b359e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82b41054f0 0x7f82b41a2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:55.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.278+0000 7f82b2d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82b40ff450 0x7f82b4197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:55.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.278+0000 7f82b259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82b40ffe10 0x7f82b41979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57908/0 (socket says 172.21.15.82:57908) 2023-09-09T23:19:55.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.278+0000 7f82b259c700 1 -- 172.21.15.82:0/2356208609 learned_addr learned my addr 172.21.15.82:0/2356208609 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:19:55.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.278+0000 7f82b259c700 1 -- 172.21.15.82:0/2356208609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82b40ff450 msgr2=0x7f82b4197490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:19:55.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.278+0000 7f82b259c700 1 --2- 172.21.15.82:0/2356208609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82b40ff450 0x7f82b4197490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.278+0000 7f82b259c700 1 -- 172.21.15.82:0/2356208609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82b41054f0 msgr2=0x7f82b41a2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:19:55.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.278+0000 7f82b259c700 1 --2- 172.21.15.82:0/2356208609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82b41054f0 0x7f82b41a2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.278+0000 7f82b259c700 1 -- 172.21.15.82:0/2356208609 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a80095f0 con 0x7f82b40ffe10 2023-09-09T23:19:55.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.278+0000 7f82b359e700 1 --2- 172.21.15.82:0/2356208609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82b41054f0 0x7f82b41a2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:19:55.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.278+0000 7f82b2d9d700 1 --2- 172.21.15.82:0/2356208609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82b40ff450 0x7f82b4197490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:19:55.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.278+0000 7f82b259c700 1 --2- 172.21.15.82:0/2356208609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82b40ffe10 0x7f82b41979d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f82a400e9a0 tx=0x7f82a400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:19:55.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.278+0000 7f829bfff700 1 -- 172.21.15.82:0/2356208609 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a4009790 con 0x7f82b40ffe10 2023-09-09T23:19:55.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.278+0000 7f829bfff700 1 -- 172.21.15.82:0/2356208609 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82a4010430 con 0x7f82b40ffe10 2023-09-09T23:19:55.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.278+0000 7f82b9256700 1 -- 172.21.15.82:0/2356208609 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b41a2ef0 con 0x7f82b40ffe10 2023-09-09T23:19:55.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.278+0000 7f829bfff700 1 -- 172.21.15.82:0/2356208609 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a4018680 con 0x7f82b40ffe10 2023-09-09T23:19:55.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.278+0000 7f82b9256700 1 -- 172.21.15.82:0/2356208609 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b41a3370 con 0x7f82b40ffe10 2023-09-09T23:19:55.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.280+0000 7f829bfff700 1 -- 172.21.15.82:0/2356208609 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f82a40105a0 con 0x7f82b40ffe10 2023-09-09T23:19:55.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.281+0000 7f829bfff700 1 --2- 172.21.15.82:0/2356208609 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f829c0a07b0 0x7f829c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:55.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.281+0000 7f829bfff700 1 -- 172.21.15.82:0/2356208609 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f82a4014070 con 0x7f82b40ffe10 2023-09-09T23:19:55.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.281+0000 7f82b2d9d700 1 --2- 172.21.15.82:0/2356208609 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f829c0a07b0 0x7f829c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:55.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.281+0000 7f82b2d9d700 1 --2- 172.21.15.82:0/2356208609 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f829c0a07b0 0x7f829c0a2c70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f82a800b2e0 tx=0x7f82a8005fd0 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:19:55.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.282+0000 7f82c541cb80 1 Processor -- start 2023-09-09T23:19:55.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.282+0000 7f82c541cb80 1 -- start start 2023-09-09T23:19:55.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.282+0000 7f82c541cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a788266710 0x55a788266af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:55.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.282+0000 7f82c541cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a788267030 0x55a788270900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:55.292 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.282+0000 7f82c541cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a788270e40 0x55a788273230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:55.292 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.282+0000 7f82c541cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7881bf730 con 0x55a788270e40 2023-09-09T23:19:55.292 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.282+0000 7f82c541cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7881bd050 con 0x55a788267030 2023-09-09T23:19:55.292 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.282+0000 7f82c541cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7881b8940 con 0x55a788266710 2023-09-09T23:19:55.292 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.282+0000 7f82b359e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a788270e40 0x55a788273230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:55.292 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.282+0000 7f82b259c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a788267030 0x55a788270900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:55.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.282+0000 7f82b2d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a788266710 0x55a788266af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:55.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.282+0000 7f82b359e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a788270e40 0x55a788273230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57918/0 (socket says 172.21.15.82:57918) 2023-09-09T23:19:55.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.282+0000 7f82b359e700 1 -- 172.21.15.82:0/1603021124 learned_addr learned my addr 172.21.15.82:0/1603021124 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:19:55.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.282+0000 7f82b259c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a788267030 0x55a788270900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:43480/0 (socket says 172.21.15.82:43480) 2023-09-09T23:19:55.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.283+0000 7f82b259c700 1 -- 172.21.15.82:0/1603021124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a788266710 msgr2=0x55a788266af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:19:55.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.283+0000 7f82b259c700 1 --2- 172.21.15.82:0/1603021124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a788266710 0x55a788266af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.283+0000 7f82b259c700 1 -- 172.21.15.82:0/1603021124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a788270e40 msgr2=0x55a788273230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:19:55.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.283+0000 7f82b259c700 1 --2- 172.21.15.82:0/1603021124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a788270e40 0x55a788273230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.283+0000 7f82b259c700 1 -- 172.21.15.82:0/1603021124 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a787fe4de0 con 0x55a788267030 2023-09-09T23:19:55.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.283+0000 7f82b2d9d700 1 --2- 172.21.15.82:0/1603021124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a788266710 0x55a788266af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:19:55.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.283+0000 7f82b359e700 1 --2- 172.21.15.82:0/1603021124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a788270e40 0x55a788273230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:19:55.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.283+0000 7f82b259c700 1 --2- 172.21.15.82:0/1603021124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a788267030 0x55a788270900 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f82a40c9870 tx=0x7f82a4098f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8367dc4da1c3a93c server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:19:55.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.283+0000 7f82997fa700 1 -- 172.21.15.82:0/1603021124 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a409d400 con 0x55a788267030 2023-09-09T23:19:55.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.283+0000 7f82997fa700 1 -- 172.21.15.82:0/1603021124 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82a40a1640 con 0x55a788267030 2023-09-09T23:19:55.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.283+0000 7f82997fa700 1 -- 172.21.15.82:0/1603021124 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a4099830 con 0x55a788267030 2023-09-09T23:19:55.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.284+0000 7f82c541cb80 1 -- 172.21.15.82:0/1603021124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a788267030 msgr2=0x55a788270900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:19:55.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.284+0000 7f82c541cb80 1 --2- 172.21.15.82:0/1603021124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a788267030 0x55a788270900 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f82a40c9870 tx=0x7f82a4098f30 comp rx=0 tx=0).stop 2023-09-09T23:19:55.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.284+0000 7f82c541cb80 1 -- 172.21.15.82:0/1603021124 shutdown_connections 2023-09-09T23:19:55.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.284+0000 7f82c541cb80 1 --2- 172.21.15.82:0/1603021124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a788270e40 0x55a788273230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.284+0000 7f82c541cb80 1 --2- 172.21.15.82:0/1603021124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a788266710 0x55a788266af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.284+0000 7f82c541cb80 1 --2- 172.21.15.82:0/1603021124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a788267030 0x55a788270900 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.284+0000 7f82c541cb80 1 -- 172.21.15.82:0/1603021124 >> 172.21.15.82:0/1603021124 conn(0x55a7881db090 msgr2=0x55a788261bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:19:55.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.284+0000 7f82c541cb80 1 -- 172.21.15.82:0/1603021124 shutdown_connections 2023-09-09T23:19:55.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.284+0000 7f82c541cb80 1 -- 172.21.15.82:0/1603021124 wait complete. 2023-09-09T23:19:55.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.284+0000 7f82c541cb80 1 -- start start 2023-09-09T23:19:55.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.284+0000 7f82c541cb80 1 Processor -- start 2023-09-09T23:19:55.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.284+0000 7f82c541cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a788266710 0x55a788270810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:55.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.284+0000 7f82c541cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a788267030 0x55a78826da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:55.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.284+0000 7f82c541cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a788270e40 0x55a78826e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:55.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.284+0000 7f82c541cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a788156b70 con 0x55a788267030 2023-09-09T23:19:55.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.284+0000 7f82c541cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a788156870 con 0x55a788266710 2023-09-09T23:19:55.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.284+0000 7f82c541cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78803a920 con 0x55a788270e40 2023-09-09T23:19:55.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.284+0000 7f82c541cb80 10 client.0 _ll_register_callbacks cb 0x55a7880a0a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:19:55.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.284+0000 7f82b259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a788267030 0x55a78826da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:55.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.284+0000 7f82b259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a788267030 0x55a78826da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57922/0 (socket says 172.21.15.82:57922) 2023-09-09T23:19:55.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.284+0000 7f82b259c700 1 -- 172.21.15.82:0/230720718 learned_addr learned my addr 172.21.15.82:0/230720718 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:19:55.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.285+0000 7f82b2d9d700 1 --2- 172.21.15.82:0/230720718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a788266710 0x55a788270810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:55.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.285+0000 7f82b359e700 1 --2- 172.21.15.82:0/230720718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a788270e40 0x55a78826e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:55.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.285+0000 7f82b259c700 1 -- 172.21.15.82:0/230720718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a788270e40 msgr2=0x55a78826e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:19:55.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.285+0000 7f82b259c700 1 --2- 172.21.15.82:0/230720718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a788270e40 0x55a78826e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.301 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.285+0000 7f82b259c700 1 -- 172.21.15.82:0/230720718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a788266710 msgr2=0x55a788270810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:19:55.301 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.285+0000 7f82b259c700 1 --2- 172.21.15.82:0/230720718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a788266710 0x55a788270810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.301 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.285+0000 7f82b259c700 1 -- 172.21.15.82:0/230720718 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b410cc20 con 0x55a788267030 2023-09-09T23:19:55.301 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.285+0000 7f82b359e700 1 --2- 172.21.15.82:0/230720718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a788270e40 0x55a78826e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:19:55.301 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.285+0000 7f82b259c700 1 --2- 172.21.15.82:0/230720718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a788267030 0x55a78826da90 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f82a40c98d0 tx=0x7f82a4010b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6cdaeaf5815f04 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:19:55.301 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.285+0000 7f8296ff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:19:55.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.285+0000 7f8296ff5700 1 -- 172.21.15.82:0/230720718 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a409d120 con 0x55a788267030 2023-09-09T23:19:55.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.285+0000 7f8296ff5700 1 -- 172.21.15.82:0/230720718 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f82a4010880 con 0x55a788267030 2023-09-09T23:19:55.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.285+0000 7f82c541cb80 1 -- 172.21.15.82:0/230720718 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7881104f0 con 0x55a788267030 2023-09-09T23:19:55.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.285+0000 7f8296ff5700 1 -- 172.21.15.82:0/230720718 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82a40840b0 con 0x55a788267030 2023-09-09T23:19:55.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.285+0000 7f82c541cb80 10 client.16137 fetch_fsmap learned FSMap version 54 2023-09-09T23:19:55.303 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.285+0000 7f82c541cb80 1 -- 172.21.15.82:0/230720718 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x7f82ac003b90 con 0x55a788267030 2023-09-09T23:19:55.303 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.285+0000 7f8296ff5700 1 -- 172.21.15.82:0/230720718 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82a4065020 con 0x55a788267030 2023-09-09T23:19:55.303 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.285+0000 7f8296ff5700 1 -- 172.21.15.82:0/230720718 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 54) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f82a4099830 con 0x55a788267030 2023-09-09T23:19:55.303 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.286+0000 7f82c541cb80 10 client.16137 fetch_fsmap finished waiting for FSMap version 54 2023-09-09T23:19:55.303 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.286+0000 7f82c541cb80 10 client.16137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:19:55.304 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.286+0000 7f82c541cb80 20 client.16137 populate_metadata read hostname 'smithi082' 2023-09-09T23:19:55.304 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.286+0000 7f82c541cb80 1 --2- 172.21.15.82:0/230720718 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55a7882805b0 0x55a7882829d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:55.304 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.286+0000 7f82c541cb80 4 client.16137 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:19:55.304 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.286+0000 7f82c541cb80 1 -- 172.21.15.82:0/230720718 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7881cd8a0 con 0x55a7882805b0 2023-09-09T23:19:55.304 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.286+0000 7f82b2d9d700 1 --2- 172.21.15.82:0/230720718 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55a7882805b0 0x55a7882829d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.286+0000 7f8296ff5700 10 client.16137 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:19:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.286+0000 7f82b2d9d700 1 --2- 172.21.15.82:0/230720718 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55a7882805b0 0x55a7882829d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c418e52f6c7ca638 server_cookie=7f1f0f8c23f12ec1 in_seq=0 out_seq=0 2023-09-09T23:19:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.287+0000 7f8296ff5700 1 -- 172.21.15.82:0/230720718 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a7881cd8a0 con 0x55a7882805b0 2023-09-09T23:19:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.287+0000 7f8296ff5700 10 client.16137 handle_command_reply: tid=0 2023-09-09T23:19:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.287+0000 7f82c541cb80 1 client.16137 shutdown 2023-09-09T23:19:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.287+0000 7f82c541cb80 1 -- 172.21.15.82:0/230720718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a788267030 msgr2=0x55a78826da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:19:55.306 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.287+0000 7f82c541cb80 1 --2- 172.21.15.82:0/230720718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a788267030 0x55a78826da90 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f82a40c98d0 tx=0x7f82a4010b80 comp rx=0 tx=0).stop 2023-09-09T23:19:55.306 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.287+0000 7f82c541cb80 1 -- 172.21.15.82:0/230720718 shutdown_connections 2023-09-09T23:19:55.306 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.287+0000 7f82c541cb80 1 --2- 172.21.15.82:0/230720718 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55a7882805b0 0x55a7882829d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.306 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.287+0000 7f82c541cb80 1 --2- 172.21.15.82:0/230720718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a788266710 0x55a788270810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.307 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.287+0000 7f8296ff5700 0 client.16137 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:19:55.307 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.287+0000 7f82c541cb80 1 --2- 172.21.15.82:0/230720718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a788270e40 0x55a78826e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.307 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.287+0000 7f82c541cb80 1 --2- 172.21.15.82:0/230720718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a788267030 0x55a78826da90 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.307 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.287+0000 7f82c541cb80 1 -- 172.21.15.82:0/230720718 >> 172.21.15.82:0/230720718 conn(0x55a7881db090 msgr2=0x55a7882617e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:19:55.307 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.288+0000 7f82c541cb80 1 -- 172.21.15.82:0/230720718 shutdown_connections 2023-09-09T23:19:55.307 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.288+0000 7f82c541cb80 1 -- 172.21.15.82:0/230720718 wait complete. 2023-09-09T23:19:55.308 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.288+0000 7f82c541cb80 20 client.16137 trim_cache size 0 max 16384 2023-09-09T23:19:55.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.304+0000 7f82c541cb80 1 Processor -- start 2023-09-09T23:19:55.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.304+0000 7f82c541cb80 1 -- start start 2023-09-09T23:19:55.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.304+0000 7f82c541cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a788266710 0x55a78826d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:55.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.304+0000 7f82c541cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a788267030 0x55a788260320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:55.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.304+0000 7f82c541cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a788270e40 0x55a788270340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:55.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.304+0000 7f82c541cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a788283030 con 0x55a788267030 2023-09-09T23:19:55.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.304+0000 7f82c541cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78826e080 con 0x55a788270e40 2023-09-09T23:19:55.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.304+0000 7f82c541cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a788260860 con 0x55a788266710 2023-09-09T23:19:55.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.305+0000 7f82b259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a788267030 0x55a788260320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:55.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.305+0000 7f82b2d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a788266710 0x55a78826d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:55.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.305+0000 7f82b259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a788267030 0x55a788260320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57930/0 (socket says 172.21.15.82:57930) 2023-09-09T23:19:55.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.305+0000 7f82b259c700 1 -- 172.21.15.82:0/2327594870 learned_addr learned my addr 172.21.15.82:0/2327594870 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:19:55.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.305+0000 7f82b359e700 1 --2- 172.21.15.82:0/2327594870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a788270e40 0x55a788270340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:55.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.305+0000 7f82b259c700 1 -- 172.21.15.82:0/2327594870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a788266710 msgr2=0x55a78826d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:19:55.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.305+0000 7f82b259c700 1 --2- 172.21.15.82:0/2327594870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a788266710 0x55a78826d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.305+0000 7f82b259c700 1 -- 172.21.15.82:0/2327594870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a788270e40 msgr2=0x55a788270340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:19:55.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.305+0000 7f82b259c700 1 --2- 172.21.15.82:0/2327594870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a788270e40 0x55a788270340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.305+0000 7f82b259c700 1 -- 172.21.15.82:0/2327594870 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b41a3370 con 0x55a788267030 2023-09-09T23:19:55.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.305+0000 7f82b2d9d700 1 --2- 172.21.15.82:0/2327594870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a788266710 0x55a78826d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:19:55.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.305+0000 7f82b359e700 1 --2- 172.21.15.82:0/2327594870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a788270e40 0x55a788270340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:19:55.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.305+0000 7f82b259c700 1 --2- 172.21.15.82:0/2327594870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a788267030 0x55a788260320 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f82a4095ad0 tx=0x7f82a4010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569c3b09fb608e54 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:19:55.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.305+0000 7f8299ffb700 1 -- 172.21.15.82:0/2327594870 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a40651b0 con 0x55a788267030 2023-09-09T23:19:55.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.305+0000 7f8299ffb700 1 -- 172.21.15.82:0/2327594870 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82a4065040 con 0x55a788267030 2023-09-09T23:19:55.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.305+0000 7f8299ffb700 1 -- 172.21.15.82:0/2327594870 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a40840b0 con 0x55a788267030 2023-09-09T23:19:55.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.306+0000 7f82c541cb80 1 -- 172.21.15.82:0/2327594870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a788267030 msgr2=0x55a788260320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:19:55.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.306+0000 7f82c541cb80 1 --2- 172.21.15.82:0/2327594870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a788267030 0x55a788260320 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f82a4095ad0 tx=0x7f82a4010c80 comp rx=0 tx=0).stop 2023-09-09T23:19:55.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.306+0000 7f82c541cb80 1 -- 172.21.15.82:0/2327594870 shutdown_connections 2023-09-09T23:19:55.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.306+0000 7f82c541cb80 1 --2- 172.21.15.82:0/2327594870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a788270e40 0x55a788270340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.306+0000 7f82c541cb80 1 --2- 172.21.15.82:0/2327594870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a788266710 0x55a78826d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.306+0000 7f82c541cb80 1 --2- 172.21.15.82:0/2327594870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a788267030 0x55a788260320 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.306+0000 7f82c541cb80 1 -- 172.21.15.82:0/2327594870 >> 172.21.15.82:0/2327594870 conn(0x55a7881db090 msgr2=0x55a7882872f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:19:55.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.306+0000 7f82c541cb80 1 -- 172.21.15.82:0/2327594870 shutdown_connections 2023-09-09T23:19:55.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.306+0000 7f82c541cb80 1 -- 172.21.15.82:0/2327594870 wait complete. 2023-09-09T23:19:55.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.306+0000 7f82c541cb80 1 -- start start 2023-09-09T23:19:55.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.306+0000 7f82c541cb80 1 Processor -- start 2023-09-09T23:19:55.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.306+0000 7f82c541cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a788266710 0x55a788277750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:55.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.306+0000 7f82c541cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a788267030 0x55a788275a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:55.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.306+0000 7f82c541cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a788270e40 0x55a788275f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:55.325 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.306+0000 7f82c541cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a788276490 con 0x55a788270e40 2023-09-09T23:19:55.325 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.306+0000 7f82c541cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a788276600 con 0x55a788266710 2023-09-09T23:19:55.325 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.306+0000 7f82c541cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7882767c0 con 0x55a788267030 2023-09-09T23:19:55.325 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.306+0000 7f82c541cb80 10 client.0 _ll_register_callbacks cb 0x55a7880a0a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:19:55.325 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.306+0000 7f82b359e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a788270e40 0x55a788275f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:55.326 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.306+0000 7f82b359e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a788270e40 0x55a788275f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57938/0 (socket says 172.21.15.82:57938) 2023-09-09T23:19:55.326 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.306+0000 7f82b259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a788267030 0x55a788275a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:55.326 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.306+0000 7f82b359e700 1 -- 172.21.15.82:0/2999216958 learned_addr learned my addr 172.21.15.82:0/2999216958 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:19:55.326 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.306+0000 7f82b2d9d700 1 --2- 172.21.15.82:0/2999216958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a788266710 0x55a788277750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:55.326 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.307+0000 7f82b359e700 1 -- 172.21.15.82:0/2999216958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a788267030 msgr2=0x55a788275a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:19:55.327 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.307+0000 7f82b359e700 1 --2- 172.21.15.82:0/2999216958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a788267030 0x55a788275a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.327 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.307+0000 7f82b359e700 1 -- 172.21.15.82:0/2999216958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a788266710 msgr2=0x55a788277750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:19:55.327 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.307+0000 7f82b359e700 1 --2- 172.21.15.82:0/2999216958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a788266710 0x55a788277750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.327 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.307+0000 7f82b359e700 1 -- 172.21.15.82:0/2999216958 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a800f570 con 0x55a788270e40 2023-09-09T23:19:55.327 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.307+0000 7f82b259c700 1 --2- 172.21.15.82:0/2999216958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a788267030 0x55a788275a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:19:55.328 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.307+0000 7f82b359e700 1 --2- 172.21.15.82:0/2999216958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a788270e40 0x55a788275f50 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f82ac007ba0 tx=0x7f82ac007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba94dc621c080b0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:19:55.328 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.307+0000 7f8296ff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:19:55.328 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.307+0000 7f8296ff5700 1 -- 172.21.15.82:0/2999216958 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ac00ee30 con 0x55a788270e40 2023-09-09T23:19:55.328 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.307+0000 7f8296ff5700 1 -- 172.21.15.82:0/2999216958 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8411+0+0 (secure 0 0 0) 0x7f82ac021430 con 0x55a788270e40 2023-09-09T23:19:55.328 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.307+0000 7f82c541cb80 1 -- 172.21.15.82:0/2999216958 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82a40a1640 con 0x55a788270e40 2023-09-09T23:19:55.328 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.307+0000 7f8296ff5700 1 -- 172.21.15.82:0/2999216958 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82ac0215f0 con 0x55a788270e40 2023-09-09T23:19:55.329 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.307+0000 7f82c541cb80 10 client.16143 fetch_fsmap learned FSMap version 54 2023-09-09T23:19:55.329 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.307+0000 7f82c541cb80 1 -- 172.21.15.82:0/2999216958 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x7f82ac003b90 con 0x55a788270e40 2023-09-09T23:19:55.329 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.307+0000 7f8296ff5700 1 -- 172.21.15.82:0/2999216958 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82a40a1640 con 0x55a788270e40 2023-09-09T23:19:55.329 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.308+0000 7f8296ff5700 1 -- 172.21.15.82:0/2999216958 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 54) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f82ac02a930 con 0x55a788270e40 2023-09-09T23:19:55.329 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.308+0000 7f82c541cb80 10 client.16143 fetch_fsmap finished waiting for FSMap version 54 2023-09-09T23:19:55.330 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.308+0000 7f82c541cb80 10 client.16143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:19:55.330 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.308+0000 7f82c541cb80 20 client.16143 populate_metadata read hostname 'smithi082' 2023-09-09T23:19:55.330 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.308+0000 7f82c541cb80 1 --2- 172.21.15.82:0/2999216958 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55a7882d5910 0x55a7882d7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:19:55.330 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.308+0000 7f82c541cb80 4 client.16143 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:19:55.330 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.308+0000 7f82c541cb80 1 -- 172.21.15.82:0/2999216958 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a7881b1920 con 0x55a7882d5910 2023-09-09T23:19:55.331 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.308+0000 7f82b2d9d700 1 --2- 172.21.15.82:0/2999216958 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55a7882d5910 0x55a7882d7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:19:55.331 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.308+0000 7f8296ff5700 10 client.16143 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:19:55.331 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.308+0000 7f82b2d9d700 1 --2- 172.21.15.82:0/2999216958 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55a7882d5910 0x55a7882d7d30 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c9234bfb341eedf server_cookie=fbfdfffe516f0faf in_seq=0 out_seq=0 2023-09-09T23:19:55.331 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.308+0000 7f8296ff5700 1 -- 172.21.15.82:0/2999216958 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55a7881b1920 con 0x55a7882d5910 2023-09-09T23:19:55.331 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.308+0000 7f8296ff5700 10 client.16143 handle_command_reply: tid=0 2023-09-09T23:19:55.332 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.308+0000 7f82c541cb80 1 client.16143 shutdown 2023-09-09T23:19:55.332 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.308+0000 7f82c541cb80 1 -- 172.21.15.82:0/2999216958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a788270e40 msgr2=0x55a788275f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:19:55.332 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.308+0000 7f82c541cb80 1 --2- 172.21.15.82:0/2999216958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a788270e40 0x55a788275f50 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f82ac007ba0 tx=0x7f82ac007eb0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.332 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.309+0000 7f82c541cb80 1 -- 172.21.15.82:0/2999216958 shutdown_connections 2023-09-09T23:19:55.332 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.309+0000 7f82c541cb80 1 --2- 172.21.15.82:0/2999216958 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55a7882d5910 0x55a7882d7d30 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.309+0000 7f82c541cb80 1 --2- 172.21.15.82:0/2999216958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a788266710 0x55a788277750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.309+0000 7f8296ff5700 0 client.16143 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:19:55.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.309+0000 7f82c541cb80 1 --2- 172.21.15.82:0/2999216958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a788270e40 0x55a788275f50 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.309+0000 7f82c541cb80 1 --2- 172.21.15.82:0/2999216958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a788267030 0x55a788275a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.309+0000 7f82c541cb80 1 -- 172.21.15.82:0/2999216958 >> 172.21.15.82:0/2999216958 conn(0x55a7881db090 msgr2=0x55a788268e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:19:55.334 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.309+0000 7f82c541cb80 1 -- 172.21.15.82:0/2999216958 shutdown_connections 2023-09-09T23:19:55.334 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.309+0000 7f82c541cb80 1 -- 172.21.15.82:0/2999216958 wait complete. 2023-09-09T23:19:55.334 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.309+0000 7f82c541cb80 20 client.16143 trim_cache size 0 max 16384 2023-09-09T23:19:55.334 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:19:55.335 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:19:55.335 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:19:55.335 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:19:55.335 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:19:55.336 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:19:55.336 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:19:55.336 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:19:55.336 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:19:55.336 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:19:55.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.309+0000 7f82b9256700 1 -- 172.21.15.82:0/2356208609 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f829c0a07b0 msgr2=0x7f829c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:19:55.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.309+0000 7f82b9256700 1 --2- 172.21.15.82:0/2356208609 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f829c0a07b0 0x7f829c0a2c70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f82a800b2e0 tx=0x7f82a8005fd0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.309+0000 7f82b9256700 1 -- 172.21.15.82:0/2356208609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82b40ffe10 msgr2=0x7f82b41979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:19:55.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.309+0000 7f82b9256700 1 --2- 172.21.15.82:0/2356208609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82b40ffe10 0x7f82b41979d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f82a400e9a0 tx=0x7f82a400ecb0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.309+0000 7f82b9256700 1 -- 172.21.15.82:0/2356208609 shutdown_connections 2023-09-09T23:19:55.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.309+0000 7f82b9256700 1 --2- 172.21.15.82:0/2356208609 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f829c0a07b0 0x7f829c0a2c70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.309+0000 7f82b9256700 1 --2- 172.21.15.82:0/2356208609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82b41054f0 0x7f82b41a2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.309+0000 7f82b9256700 1 --2- 172.21.15.82:0/2356208609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82b40ff450 0x7f82b4197490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.309+0000 7f82b9256700 1 --2- 172.21.15.82:0/2356208609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82b40ffe10 0x7f82b41979d0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:19:55.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.309+0000 7f82b9256700 1 -- 172.21.15.82:0/2356208609 >> 172.21.15.82:0/2356208609 conn(0x7f82b40faf20 msgr2=0x7f82b4102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:19:55.340 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.309+0000 7f82b9256700 1 -- 172.21.15.82:0/2356208609 shutdown_connections 2023-09-09T23:19:55.340 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:19:55.309+0000 7f82b9256700 1 -- 172.21.15.82:0/2356208609 wait complete. 2023-09-09T23:19:55.340 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:19:55.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:55 smithi082 ceph-mon[98694]: pgmap v183: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2023-09-09T23:19:55.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:19:55.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:55.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:55 smithi082 ceph-mon[98694]: 315 slow requests, 5 included below; oldest blocked for > 221.157351 secs 2023-09-09T23:19:55.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:55 smithi082 ceph-mon[98694]: slow request 220.220443 seconds old, received at 2023-09-09T23:16:14.431555+0000: client_request(mds.1:1794 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:55.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:55 smithi082 ceph-mon[98694]: slow request 220.161322 seconds old, received at 2023-09-09T23:16:14.490676+0000: client_request(mds.1:1826 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:55.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:55 smithi082 ceph-mon[98694]: slow request 220.107161 seconds old, received at 2023-09-09T23:16:14.544837+0000: client_request(mds.1:1858 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:55.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:55 smithi082 ceph-mon[98694]: slow request 220.054308 seconds old, received at 2023-09-09T23:16:14.597690+0000: client_request(mds.1:1890 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:55.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:55 smithi082 ceph-mon[98694]: slow request 220.000558 seconds old, received at 2023-09-09T23:16:14.651440+0000: client_request(mds.1:1922 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:55.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:55 smithi170 ceph-mon[102832]: pgmap v183: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2023-09-09T23:19:55.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:19:55.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:55.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:55 smithi170 ceph-mon[102832]: 315 slow requests, 5 included below; oldest blocked for > 221.157351 secs 2023-09-09T23:19:55.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:55 smithi170 ceph-mon[102832]: slow request 220.220443 seconds old, received at 2023-09-09T23:16:14.431555+0000: client_request(mds.1:1794 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:55.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:55 smithi170 ceph-mon[102832]: slow request 220.161322 seconds old, received at 2023-09-09T23:16:14.490676+0000: client_request(mds.1:1826 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:55 smithi170 ceph-mon[102832]: slow request 220.107161 seconds old, received at 2023-09-09T23:16:14.544837+0000: client_request(mds.1:1858 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:55 smithi170 ceph-mon[102832]: slow request 220.054308 seconds old, received at 2023-09-09T23:16:14.597690+0000: client_request(mds.1:1890 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:55 smithi170 ceph-mon[102832]: slow request 220.000558 seconds old, received at 2023-09-09T23:16:14.651440+0000: client_request(mds.1:1922 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:55.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:55 smithi098 ceph-mon[103124]: pgmap v183: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2023-09-09T23:19:55.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:19:55.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:55.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:55 smithi098 ceph-mon[103124]: 315 slow requests, 5 included below; oldest blocked for > 221.157351 secs 2023-09-09T23:19:55.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:55 smithi098 ceph-mon[103124]: slow request 220.220443 seconds old, received at 2023-09-09T23:16:14.431555+0000: client_request(mds.1:1794 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:55.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:55 smithi098 ceph-mon[103124]: slow request 220.161322 seconds old, received at 2023-09-09T23:16:14.490676+0000: client_request(mds.1:1826 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:55 smithi098 ceph-mon[103124]: slow request 220.107161 seconds old, received at 2023-09-09T23:16:14.544837+0000: client_request(mds.1:1858 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:55 smithi098 ceph-mon[103124]: slow request 220.054308 seconds old, received at 2023-09-09T23:16:14.597690+0000: client_request(mds.1:1890 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:55 smithi098 ceph-mon[103124]: slow request 220.000558 seconds old, received at 2023-09-09T23:16:14.651440+0000: client_request(mds.1:1922 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:19:56.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:19:56.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:56.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:19:56.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:56.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:19:56.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:57.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:57 smithi170 ceph-mon[102832]: pgmap v184: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2023-09-09T23:19:57.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:57 smithi170 ceph-mon[102832]: 633 slow requests, 5 included below; oldest blocked for > 210.896408 secs 2023-09-09T23:19:57.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:57 smithi170 ceph-mon[102832]: slow request 210.832754 seconds old, received at 2023-09-09T23:16:25.854761+0000: peer_request:mds.1:1516 currently dispatched 2023-09-09T23:19:57.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:57 smithi170 ceph-mon[102832]: slow request 210.820864 seconds old, received at 2023-09-09T23:16:25.866651+0000: peer_request:mds.1:1548 currently dispatched 2023-09-09T23:19:57.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:57 smithi170 ceph-mon[102832]: slow request 210.809628 seconds old, received at 2023-09-09T23:16:25.877887+0000: peer_request:mds.1:1580 currently dispatched 2023-09-09T23:19:57.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:57 smithi170 ceph-mon[102832]: slow request 210.798443 seconds old, received at 2023-09-09T23:16:25.889072+0000: peer_request:mds.1:1612 currently dispatched 2023-09-09T23:19:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:57 smithi170 ceph-mon[102832]: slow request 210.787710 seconds old, received at 2023-09-09T23:16:25.899805+0000: peer_request:mds.1:1644 currently dispatched 2023-09-09T23:19:57.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:57 smithi082 ceph-mon[98694]: pgmap v184: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2023-09-09T23:19:57.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:57 smithi082 ceph-mon[98694]: 633 slow requests, 5 included below; oldest blocked for > 210.896408 secs 2023-09-09T23:19:57.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:57 smithi082 ceph-mon[98694]: slow request 210.832754 seconds old, received at 2023-09-09T23:16:25.854761+0000: peer_request:mds.1:1516 currently dispatched 2023-09-09T23:19:57.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:57 smithi082 ceph-mon[98694]: slow request 210.820864 seconds old, received at 2023-09-09T23:16:25.866651+0000: peer_request:mds.1:1548 currently dispatched 2023-09-09T23:19:57.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:57 smithi082 ceph-mon[98694]: slow request 210.809628 seconds old, received at 2023-09-09T23:16:25.877887+0000: peer_request:mds.1:1580 currently dispatched 2023-09-09T23:19:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:57 smithi082 ceph-mon[98694]: slow request 210.798443 seconds old, received at 2023-09-09T23:16:25.889072+0000: peer_request:mds.1:1612 currently dispatched 2023-09-09T23:19:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:57 smithi082 ceph-mon[98694]: slow request 210.787710 seconds old, received at 2023-09-09T23:16:25.899805+0000: peer_request:mds.1:1644 currently dispatched 2023-09-09T23:19:57.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:57 smithi098 ceph-mon[103124]: pgmap v184: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2023-09-09T23:19:57.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:57 smithi098 ceph-mon[103124]: 633 slow requests, 5 included below; oldest blocked for > 210.896408 secs 2023-09-09T23:19:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:57 smithi098 ceph-mon[103124]: slow request 210.832754 seconds old, received at 2023-09-09T23:16:25.854761+0000: peer_request:mds.1:1516 currently dispatched 2023-09-09T23:19:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:57 smithi098 ceph-mon[103124]: slow request 210.820864 seconds old, received at 2023-09-09T23:16:25.866651+0000: peer_request:mds.1:1548 currently dispatched 2023-09-09T23:19:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:57 smithi098 ceph-mon[103124]: slow request 210.809628 seconds old, received at 2023-09-09T23:16:25.877887+0000: peer_request:mds.1:1580 currently dispatched 2023-09-09T23:19:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:57 smithi098 ceph-mon[103124]: slow request 210.798443 seconds old, received at 2023-09-09T23:16:25.889072+0000: peer_request:mds.1:1612 currently dispatched 2023-09-09T23:19:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:57 smithi098 ceph-mon[103124]: slow request 210.787710 seconds old, received at 2023-09-09T23:16:25.899805+0000: peer_request:mds.1:1644 currently dispatched 2023-09-09T23:19:59.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:59 smithi170 ceph-mon[102832]: pgmap v185: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2023-09-09T23:19:59.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:59 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:19:59.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:19:59 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:59.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:59 smithi082 ceph-mon[98694]: pgmap v185: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2023-09-09T23:19:59.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:59 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:19:59.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:19:59 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:19:59.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:59 smithi098 ceph-mon[103124]: pgmap v185: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2023-09-09T23:19:59.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:59 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:19:59.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:19:59 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:00.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:00.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:00.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:00 smithi082 ceph-mon[98694]: 315 slow requests, 5 included below; oldest blocked for > 226.312224 secs 2023-09-09T23:20:00.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:00 smithi082 ceph-mon[98694]: slow request 225.117535 seconds old, received at 2023-09-09T23:16:14.706977+0000: client_request(mds.1:1954 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:00.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:00 smithi082 ceph-mon[98694]: slow request 225.059516 seconds old, received at 2023-09-09T23:16:14.764997+0000: client_request(mds.1:1986 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:00.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:00 smithi082 ceph-mon[98694]: slow request 224.999786 seconds old, received at 2023-09-09T23:16:14.824727+0000: client_request(mds.1:2018 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:00.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:00 smithi082 ceph-mon[98694]: slow request 224.945930 seconds old, received at 2023-09-09T23:16:14.878583+0000: client_request(mds.1:2050 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:00.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:00 smithi082 ceph-mon[98694]: slow request 226.307889 seconds old, received at 2023-09-09T23:16:13.516624+0000: client_request(mds.1:1347 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:00 smithi082 ceph-mon[98694]: Health detail: HEALTH_WARN 2 MDSs report slow requests 2023-09-09T23:20:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:00 smithi082 ceph-mon[98694]: [WRN] MDS_SLOW_REQUEST: 2 MDSs report slow requests 2023-09-09T23:20:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:00 smithi082 ceph-mon[98694]: mds.h(mds.1): 315 slow requests are blocked > 180 secs 2023-09-09T23:20:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:00 smithi082 ceph-mon[98694]: mds.b(mds.0): 633 slow requests are blocked > 180 secs 2023-09-09T23:20:00.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:00.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:00.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:00 smithi098 ceph-mon[103124]: 315 slow requests, 5 included below; oldest blocked for > 226.312224 secs 2023-09-09T23:20:00.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:00 smithi098 ceph-mon[103124]: slow request 225.117535 seconds old, received at 2023-09-09T23:16:14.706977+0000: client_request(mds.1:1954 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:00.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:00 smithi098 ceph-mon[103124]: slow request 225.059516 seconds old, received at 2023-09-09T23:16:14.764997+0000: client_request(mds.1:1986 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:00.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:00 smithi098 ceph-mon[103124]: slow request 224.999786 seconds old, received at 2023-09-09T23:16:14.824727+0000: client_request(mds.1:2018 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:00 smithi098 ceph-mon[103124]: slow request 224.945930 seconds old, received at 2023-09-09T23:16:14.878583+0000: client_request(mds.1:2050 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:00 smithi098 ceph-mon[103124]: slow request 226.307889 seconds old, received at 2023-09-09T23:16:13.516624+0000: client_request(mds.1:1347 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:00 smithi098 ceph-mon[103124]: Health detail: HEALTH_WARN 2 MDSs report slow requests 2023-09-09T23:20:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:00 smithi098 ceph-mon[103124]: [WRN] MDS_SLOW_REQUEST: 2 MDSs report slow requests 2023-09-09T23:20:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:00 smithi098 ceph-mon[103124]: mds.h(mds.1): 315 slow requests are blocked > 180 secs 2023-09-09T23:20:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:00 smithi098 ceph-mon[103124]: mds.b(mds.0): 633 slow requests are blocked > 180 secs 2023-09-09T23:20:00.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:00.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:00.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:00 smithi170 ceph-mon[102832]: 315 slow requests, 5 included below; oldest blocked for > 226.312224 secs 2023-09-09T23:20:00.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:00 smithi170 ceph-mon[102832]: slow request 225.117535 seconds old, received at 2023-09-09T23:16:14.706977+0000: client_request(mds.1:1954 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:00.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:00 smithi170 ceph-mon[102832]: slow request 225.059516 seconds old, received at 2023-09-09T23:16:14.764997+0000: client_request(mds.1:1986 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:00.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:00 smithi170 ceph-mon[102832]: slow request 224.999786 seconds old, received at 2023-09-09T23:16:14.824727+0000: client_request(mds.1:2018 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:00 smithi170 ceph-mon[102832]: slow request 224.945930 seconds old, received at 2023-09-09T23:16:14.878583+0000: client_request(mds.1:2050 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:00 smithi170 ceph-mon[102832]: slow request 226.307889 seconds old, received at 2023-09-09T23:16:13.516624+0000: client_request(mds.1:1347 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:00 smithi170 ceph-mon[102832]: Health detail: HEALTH_WARN 2 MDSs report slow requests 2023-09-09T23:20:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:00 smithi170 ceph-mon[102832]: [WRN] MDS_SLOW_REQUEST: 2 MDSs report slow requests 2023-09-09T23:20:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:00 smithi170 ceph-mon[102832]: mds.h(mds.1): 315 slow requests are blocked > 180 secs 2023-09-09T23:20:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:00 smithi170 ceph-mon[102832]: mds.b(mds.0): 633 slow requests are blocked > 180 secs 2023-09-09T23:20:01.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:01 smithi082 ceph-mon[98694]: pgmap v186: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 11 KiB/s rd, 825 KiB/s wr, 14 op/s 2023-09-09T23:20:01.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:01 smithi098 ceph-mon[103124]: pgmap v186: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 11 KiB/s rd, 825 KiB/s wr, 14 op/s 2023-09-09T23:20:01.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:01 smithi170 ceph-mon[102832]: pgmap v186: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 11 KiB/s rd, 825 KiB/s wr, 14 op/s 2023-09-09T23:20:02.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:02 smithi082 ceph-mon[98694]: 623 slow requests, 5 included below; oldest blocked for > 215.892756 secs 2023-09-09T23:20:02.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:02 smithi082 ceph-mon[98694]: slow request 215.776422 seconds old, received at 2023-09-09T23:16:25.911162+0000: peer_request:mds.1:1676 currently dispatched 2023-09-09T23:20:02.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:02 smithi082 ceph-mon[98694]: slow request 215.764798 seconds old, received at 2023-09-09T23:16:25.922786+0000: peer_request:mds.1:1708 currently dispatched 2023-09-09T23:20:02.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:02 smithi082 ceph-mon[98694]: slow request 215.752500 seconds old, received at 2023-09-09T23:16:25.935084+0000: peer_request:mds.1:1740 currently dispatched 2023-09-09T23:20:02.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:02 smithi082 ceph-mon[98694]: slow request 215.741156 seconds old, received at 2023-09-09T23:16:25.946427+0000: peer_request:mds.1:1772 currently dispatched 2023-09-09T23:20:02.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:02 smithi082 ceph-mon[98694]: slow request 215.730942 seconds old, received at 2023-09-09T23:16:25.956641+0000: peer_request:mds.1:1804 currently dispatched 2023-09-09T23:20:02.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:02 smithi098 ceph-mon[103124]: 623 slow requests, 5 included below; oldest blocked for > 215.892756 secs 2023-09-09T23:20:02.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:02 smithi098 ceph-mon[103124]: slow request 215.776422 seconds old, received at 2023-09-09T23:16:25.911162+0000: peer_request:mds.1:1676 currently dispatched 2023-09-09T23:20:02.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:02 smithi098 ceph-mon[103124]: slow request 215.764798 seconds old, received at 2023-09-09T23:16:25.922786+0000: peer_request:mds.1:1708 currently dispatched 2023-09-09T23:20:02.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:02 smithi098 ceph-mon[103124]: slow request 215.752500 seconds old, received at 2023-09-09T23:16:25.935084+0000: peer_request:mds.1:1740 currently dispatched 2023-09-09T23:20:02.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:02 smithi098 ceph-mon[103124]: slow request 215.741156 seconds old, received at 2023-09-09T23:16:25.946427+0000: peer_request:mds.1:1772 currently dispatched 2023-09-09T23:20:02.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:02 smithi098 ceph-mon[103124]: slow request 215.730942 seconds old, received at 2023-09-09T23:16:25.956641+0000: peer_request:mds.1:1804 currently dispatched 2023-09-09T23:20:02.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:02 smithi170 ceph-mon[102832]: 623 slow requests, 5 included below; oldest blocked for > 215.892756 secs 2023-09-09T23:20:02.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:02 smithi170 ceph-mon[102832]: slow request 215.776422 seconds old, received at 2023-09-09T23:16:25.911162+0000: peer_request:mds.1:1676 currently dispatched 2023-09-09T23:20:02.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:02 smithi170 ceph-mon[102832]: slow request 215.764798 seconds old, received at 2023-09-09T23:16:25.922786+0000: peer_request:mds.1:1708 currently dispatched 2023-09-09T23:20:02.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:02 smithi170 ceph-mon[102832]: slow request 215.752500 seconds old, received at 2023-09-09T23:16:25.935084+0000: peer_request:mds.1:1740 currently dispatched 2023-09-09T23:20:02.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:02 smithi170 ceph-mon[102832]: slow request 215.741156 seconds old, received at 2023-09-09T23:16:25.946427+0000: peer_request:mds.1:1772 currently dispatched 2023-09-09T23:20:02.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:02 smithi170 ceph-mon[102832]: slow request 215.730942 seconds old, received at 2023-09-09T23:16:25.956641+0000: peer_request:mds.1:1804 currently dispatched 2023-09-09T23:20:03.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:03 smithi082 ceph-mon[98694]: pgmap v187: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 8.8 KiB/s rd, 1.5 MiB/s wr, 12 op/s 2023-09-09T23:20:03.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:03 smithi098 ceph-mon[103124]: pgmap v187: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 8.8 KiB/s rd, 1.5 MiB/s wr, 12 op/s 2023-09-09T23:20:03.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:03 smithi170 ceph-mon[102832]: pgmap v187: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 8.8 KiB/s rd, 1.5 MiB/s wr, 12 op/s 2023-09-09T23:20:04.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:04.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:04.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:04.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:04.817 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:04.818 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:05.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:05 smithi082 ceph-mon[98694]: pgmap v188: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 8.0 KiB/s rd, 1.5 MiB/s wr, 10 op/s 2023-09-09T23:20:05.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:20:05.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:05 smithi082 ceph-mon[98694]: 305 slow requests, 5 included below; oldest blocked for > 231.379654 secs 2023-09-09T23:20:05.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:05 smithi082 ceph-mon[98694]: slow request 231.335471 seconds old, received at 2023-09-09T23:16:13.575900+0000: client_request(mds.1:1379 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:05.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:05 smithi082 ceph-mon[98694]: slow request 231.279176 seconds old, received at 2023-09-09T23:16:13.632196+0000: client_request(mds.1:1411 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:05.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:05 smithi082 ceph-mon[98694]: slow request 231.220335 seconds old, received at 2023-09-09T23:16:13.691037+0000: client_request(mds.1:1443 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:05.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:05 smithi082 ceph-mon[98694]: slow request 231.056248 seconds old, received at 2023-09-09T23:16:13.855123+0000: client_request(mds.1:1475 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:05.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:05 smithi082 ceph-mon[98694]: slow request 230.996104 seconds old, received at 2023-09-09T23:16:13.915267+0000: client_request(mds.1:1507 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:20:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:20:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:20:05.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:05 smithi098 ceph-mon[103124]: pgmap v188: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 8.0 KiB/s rd, 1.5 MiB/s wr, 10 op/s 2023-09-09T23:20:05.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:20:05.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:05 smithi098 ceph-mon[103124]: 305 slow requests, 5 included below; oldest blocked for > 231.379654 secs 2023-09-09T23:20:05.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:05 smithi098 ceph-mon[103124]: slow request 231.335471 seconds old, received at 2023-09-09T23:16:13.575900+0000: client_request(mds.1:1379 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:05.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:05 smithi098 ceph-mon[103124]: slow request 231.279176 seconds old, received at 2023-09-09T23:16:13.632196+0000: client_request(mds.1:1411 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:05.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:05 smithi098 ceph-mon[103124]: slow request 231.220335 seconds old, received at 2023-09-09T23:16:13.691037+0000: client_request(mds.1:1443 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:05 smithi098 ceph-mon[103124]: slow request 231.056248 seconds old, received at 2023-09-09T23:16:13.855123+0000: client_request(mds.1:1475 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:05 smithi098 ceph-mon[103124]: slow request 230.996104 seconds old, received at 2023-09-09T23:16:13.915267+0000: client_request(mds.1:1507 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:20:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:20:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:20:05.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:05 smithi170 ceph-mon[102832]: pgmap v188: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 8.0 KiB/s rd, 1.5 MiB/s wr, 10 op/s 2023-09-09T23:20:05.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:20:05.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:05 smithi170 ceph-mon[102832]: 305 slow requests, 5 included below; oldest blocked for > 231.379654 secs 2023-09-09T23:20:05.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:05 smithi170 ceph-mon[102832]: slow request 231.335471 seconds old, received at 2023-09-09T23:16:13.575900+0000: client_request(mds.1:1379 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:05.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:05 smithi170 ceph-mon[102832]: slow request 231.279176 seconds old, received at 2023-09-09T23:16:13.632196+0000: client_request(mds.1:1411 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:05.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:05 smithi170 ceph-mon[102832]: slow request 231.220335 seconds old, received at 2023-09-09T23:16:13.691037+0000: client_request(mds.1:1443 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:05 smithi170 ceph-mon[102832]: slow request 231.056248 seconds old, received at 2023-09-09T23:16:13.855123+0000: client_request(mds.1:1475 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:05 smithi170 ceph-mon[102832]: slow request 230.996104 seconds old, received at 2023-09-09T23:16:13.915267+0000: client_request(mds.1:1507 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:20:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:20:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:20:07.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:07 smithi098 ceph-mon[103124]: pgmap v189: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 20 KiB/s rd, 1.5 MiB/s wr, 30 op/s 2023-09-09T23:20:07.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:07 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:20:07.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:07 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:07.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:07 smithi098 ceph-mon[103124]: 617 slow requests, 5 included below; oldest blocked for > 220.889030 secs 2023-09-09T23:20:07.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:07 smithi098 ceph-mon[103124]: slow request 220.719534 seconds old, received at 2023-09-09T23:16:25.968110+0000: peer_request:mds.1:1836 currently dispatched 2023-09-09T23:20:07.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:07 smithi098 ceph-mon[103124]: slow request 220.708109 seconds old, received at 2023-09-09T23:16:25.979535+0000: peer_request:mds.1:1868 currently dispatched 2023-09-09T23:20:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:07 smithi098 ceph-mon[103124]: slow request 220.693246 seconds old, received at 2023-09-09T23:16:25.994398+0000: peer_request:mds.1:1900 currently dispatched 2023-09-09T23:20:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:07 smithi098 ceph-mon[103124]: slow request 220.687580 seconds old, received at 2023-09-09T23:16:26.000064+0000: peer_request:mds.1:1932 currently dispatched 2023-09-09T23:20:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:07 smithi098 ceph-mon[103124]: slow request 220.675221 seconds old, received at 2023-09-09T23:16:26.012423+0000: peer_request:mds.1:1964 currently dispatched 2023-09-09T23:20:07.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:07 smithi170 ceph-mon[102832]: pgmap v189: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 20 KiB/s rd, 1.5 MiB/s wr, 30 op/s 2023-09-09T23:20:07.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:07 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:20:07.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:07 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:07.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:07 smithi170 ceph-mon[102832]: 617 slow requests, 5 included below; oldest blocked for > 220.889030 secs 2023-09-09T23:20:07.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:07 smithi170 ceph-mon[102832]: slow request 220.719534 seconds old, received at 2023-09-09T23:16:25.968110+0000: peer_request:mds.1:1836 currently dispatched 2023-09-09T23:20:07.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:07 smithi170 ceph-mon[102832]: slow request 220.708109 seconds old, received at 2023-09-09T23:16:25.979535+0000: peer_request:mds.1:1868 currently dispatched 2023-09-09T23:20:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:07 smithi170 ceph-mon[102832]: slow request 220.693246 seconds old, received at 2023-09-09T23:16:25.994398+0000: peer_request:mds.1:1900 currently dispatched 2023-09-09T23:20:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:07 smithi170 ceph-mon[102832]: slow request 220.687580 seconds old, received at 2023-09-09T23:16:26.000064+0000: peer_request:mds.1:1932 currently dispatched 2023-09-09T23:20:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:07 smithi170 ceph-mon[102832]: slow request 220.675221 seconds old, received at 2023-09-09T23:16:26.012423+0000: peer_request:mds.1:1964 currently dispatched 2023-09-09T23:20:07.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:07 smithi082 ceph-mon[98694]: pgmap v189: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 20 KiB/s rd, 1.5 MiB/s wr, 30 op/s 2023-09-09T23:20:07.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:07 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:20:07.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:07 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:07.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:07 smithi082 ceph-mon[98694]: 617 slow requests, 5 included below; oldest blocked for > 220.889030 secs 2023-09-09T23:20:07.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:07 smithi082 ceph-mon[98694]: slow request 220.719534 seconds old, received at 2023-09-09T23:16:25.968110+0000: peer_request:mds.1:1836 currently dispatched 2023-09-09T23:20:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:07 smithi082 ceph-mon[98694]: slow request 220.708109 seconds old, received at 2023-09-09T23:16:25.979535+0000: peer_request:mds.1:1868 currently dispatched 2023-09-09T23:20:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:07 smithi082 ceph-mon[98694]: slow request 220.693246 seconds old, received at 2023-09-09T23:16:25.994398+0000: peer_request:mds.1:1900 currently dispatched 2023-09-09T23:20:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:07 smithi082 ceph-mon[98694]: slow request 220.687580 seconds old, received at 2023-09-09T23:16:26.000064+0000: peer_request:mds.1:1932 currently dispatched 2023-09-09T23:20:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:07 smithi082 ceph-mon[98694]: slow request 220.675221 seconds old, received at 2023-09-09T23:16:26.012423+0000: peer_request:mds.1:1964 currently dispatched 2023-09-09T23:20:08.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:08 smithi098 ceph-mon[103124]: osdmap e85: 12 total, 12 up, 12 in 2023-09-09T23:20:08.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:08 smithi170 ceph-mon[102832]: osdmap e85: 12 total, 12 up, 12 in 2023-09-09T23:20:08.954 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:08 smithi082 ceph-mon[98694]: osdmap e85: 12 total, 12 up, 12 in 2023-09-09T23:20:09.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:09 smithi098 ceph-mon[103124]: pgmap v191: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 22 KiB/s rd, 1.8 MiB/s wr, 31 op/s 2023-09-09T23:20:09.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:09 smithi170 ceph-mon[102832]: pgmap v191: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 22 KiB/s rd, 1.8 MiB/s wr, 31 op/s 2023-09-09T23:20:09.954 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:09 smithi082 ceph-mon[98694]: pgmap v191: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 22 KiB/s rd, 1.8 MiB/s wr, 31 op/s 2023-09-09T23:20:10.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:10 smithi170 ceph-mon[102832]: pgmap v192: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 23 KiB/s rd, 885 KiB/s wr, 31 op/s 2023-09-09T23:20:10.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:10 smithi082 ceph-mon[98694]: pgmap v192: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 23 KiB/s rd, 885 KiB/s wr, 31 op/s 2023-09-09T23:20:11.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:10 smithi098 ceph-mon[103124]: pgmap v192: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 23 KiB/s rd, 885 KiB/s wr, 31 op/s 2023-09-09T23:20:11.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:11 smithi170 ceph-mon[102832]: 301 slow requests, 5 included below; oldest blocked for > 236.404958 secs 2023-09-09T23:20:11.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:11 smithi170 ceph-mon[102832]: slow request 235.984513 seconds old, received at 2023-09-09T23:16:13.970679+0000: client_request(mds.1:1539 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:11.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:11 smithi170 ceph-mon[102832]: slow request 235.924149 seconds old, received at 2023-09-09T23:16:14.031043+0000: client_request(mds.1:1571 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:11.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:11 smithi170 ceph-mon[102832]: slow request 235.868389 seconds old, received at 2023-09-09T23:16:14.086804+0000: client_request(mds.1:1603 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:11.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:11 smithi170 ceph-mon[102832]: slow request 235.810332 seconds old, received at 2023-09-09T23:16:14.144861+0000: client_request(mds.1:1635 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:11.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:11 smithi170 ceph-mon[102832]: slow request 235.754087 seconds old, received at 2023-09-09T23:16:14.201105+0000: client_request(mds.1:1667 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:11.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:11 smithi082 ceph-mon[98694]: 301 slow requests, 5 included below; oldest blocked for > 236.404958 secs 2023-09-09T23:20:11.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:11 smithi082 ceph-mon[98694]: slow request 235.984513 seconds old, received at 2023-09-09T23:16:13.970679+0000: client_request(mds.1:1539 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:11 smithi082 ceph-mon[98694]: slow request 235.924149 seconds old, received at 2023-09-09T23:16:14.031043+0000: client_request(mds.1:1571 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:11 smithi082 ceph-mon[98694]: slow request 235.868389 seconds old, received at 2023-09-09T23:16:14.086804+0000: client_request(mds.1:1603 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:11 smithi082 ceph-mon[98694]: slow request 235.810332 seconds old, received at 2023-09-09T23:16:14.144861+0000: client_request(mds.1:1635 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:11 smithi082 ceph-mon[98694]: slow request 235.754087 seconds old, received at 2023-09-09T23:16:14.201105+0000: client_request(mds.1:1667 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:12.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:11 smithi098 ceph-mon[103124]: 301 slow requests, 5 included below; oldest blocked for > 236.404958 secs 2023-09-09T23:20:12.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:11 smithi098 ceph-mon[103124]: slow request 235.984513 seconds old, received at 2023-09-09T23:16:13.970679+0000: client_request(mds.1:1539 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:12.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:11 smithi098 ceph-mon[103124]: slow request 235.924149 seconds old, received at 2023-09-09T23:16:14.031043+0000: client_request(mds.1:1571 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:12.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:11 smithi098 ceph-mon[103124]: slow request 235.868389 seconds old, received at 2023-09-09T23:16:14.086804+0000: client_request(mds.1:1603 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:11 smithi098 ceph-mon[103124]: slow request 235.810332 seconds old, received at 2023-09-09T23:16:14.144861+0000: client_request(mds.1:1635 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:11 smithi098 ceph-mon[103124]: slow request 235.754087 seconds old, received at 2023-09-09T23:16:14.201105+0000: client_request(mds.1:1667 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:12.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:12.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:12.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:12 smithi170 ceph-mon[102832]: 607 slow requests, 5 included below; oldest blocked for > 225.885551 secs 2023-09-09T23:20:12.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:12 smithi170 ceph-mon[102832]: slow request 225.663353 seconds old, received at 2023-09-09T23:16:26.024354+0000: peer_request:mds.1:1996 currently dispatched 2023-09-09T23:20:12.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:12 smithi170 ceph-mon[102832]: slow request 225.651636 seconds old, received at 2023-09-09T23:16:26.036070+0000: peer_request:mds.1:2028 currently dispatched 2023-09-09T23:20:12.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:12 smithi170 ceph-mon[102832]: slow request 217.827492 seconds old, received at 2023-09-09T23:16:33.860214+0000: peer_request:mds.1:2060 currently dispatched 2023-09-09T23:20:12.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:12 smithi170 ceph-mon[102832]: slow request 225.878260 seconds old, received at 2023-09-09T23:16:25.809446+0000: peer_request:mds.1:1389 currently dispatched 2023-09-09T23:20:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:12 smithi170 ceph-mon[102832]: slow request 225.866860 seconds old, received at 2023-09-09T23:16:25.820846+0000: peer_request:mds.1:1421 currently dispatched 2023-09-09T23:20:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:12 smithi170 ceph-mon[102832]: pgmap v193: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 22 KiB/s rd, 66 KiB/s wr, 30 op/s 2023-09-09T23:20:12.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:12.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:12.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:12 smithi082 ceph-mon[98694]: 607 slow requests, 5 included below; oldest blocked for > 225.885551 secs 2023-09-09T23:20:12.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:12 smithi082 ceph-mon[98694]: slow request 225.663353 seconds old, received at 2023-09-09T23:16:26.024354+0000: peer_request:mds.1:1996 currently dispatched 2023-09-09T23:20:12.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:12 smithi082 ceph-mon[98694]: slow request 225.651636 seconds old, received at 2023-09-09T23:16:26.036070+0000: peer_request:mds.1:2028 currently dispatched 2023-09-09T23:20:12.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:12 smithi082 ceph-mon[98694]: slow request 217.827492 seconds old, received at 2023-09-09T23:16:33.860214+0000: peer_request:mds.1:2060 currently dispatched 2023-09-09T23:20:12.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:12 smithi082 ceph-mon[98694]: slow request 225.878260 seconds old, received at 2023-09-09T23:16:25.809446+0000: peer_request:mds.1:1389 currently dispatched 2023-09-09T23:20:12.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:12 smithi082 ceph-mon[98694]: slow request 225.866860 seconds old, received at 2023-09-09T23:16:25.820846+0000: peer_request:mds.1:1421 currently dispatched 2023-09-09T23:20:12.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:12 smithi082 ceph-mon[98694]: pgmap v193: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 22 KiB/s rd, 66 KiB/s wr, 30 op/s 2023-09-09T23:20:13.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:13.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:13.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:12 smithi098 ceph-mon[103124]: 607 slow requests, 5 included below; oldest blocked for > 225.885551 secs 2023-09-09T23:20:13.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:12 smithi098 ceph-mon[103124]: slow request 225.663353 seconds old, received at 2023-09-09T23:16:26.024354+0000: peer_request:mds.1:1996 currently dispatched 2023-09-09T23:20:13.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:12 smithi098 ceph-mon[103124]: slow request 225.651636 seconds old, received at 2023-09-09T23:16:26.036070+0000: peer_request:mds.1:2028 currently dispatched 2023-09-09T23:20:13.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:12 smithi098 ceph-mon[103124]: slow request 217.827492 seconds old, received at 2023-09-09T23:16:33.860214+0000: peer_request:mds.1:2060 currently dispatched 2023-09-09T23:20:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:12 smithi098 ceph-mon[103124]: slow request 225.878260 seconds old, received at 2023-09-09T23:16:25.809446+0000: peer_request:mds.1:1389 currently dispatched 2023-09-09T23:20:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:12 smithi098 ceph-mon[103124]: slow request 225.866860 seconds old, received at 2023-09-09T23:16:25.820846+0000: peer_request:mds.1:1421 currently dispatched 2023-09-09T23:20:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:12 smithi098 ceph-mon[103124]: pgmap v193: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 22 KiB/s rd, 66 KiB/s wr, 30 op/s 2023-09-09T23:20:14.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:14 smithi170 ceph-mon[102832]: osdmap e86: 12 total, 12 up, 12 in 2023-09-09T23:20:14.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:14 smithi082 ceph-mon[98694]: osdmap e86: 12 total, 12 up, 12 in 2023-09-09T23:20:14.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:14 smithi098 ceph-mon[103124]: osdmap e86: 12 total, 12 up, 12 in 2023-09-09T23:20:15.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:15 smithi170 ceph-mon[102832]: pgmap v195: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 8.5 KiB/s rd, 12 KiB/s wr, 8 op/s 2023-09-09T23:20:15.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:15 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:20:15.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:15 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:15.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:15 smithi082 ceph-mon[98694]: pgmap v195: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 8.5 KiB/s rd, 12 KiB/s wr, 8 op/s 2023-09-09T23:20:15.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:15 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:20:15.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:15 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:15.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:15 smithi098 ceph-mon[103124]: pgmap v195: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 8.5 KiB/s rd, 12 KiB/s wr, 8 op/s 2023-09-09T23:20:15.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:15 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:20:15.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:15 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:16.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:16 smithi170 ceph-mon[102832]: 301 slow requests, 5 included below; oldest blocked for > 241.633069 secs 2023-09-09T23:20:16.687 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:16 smithi170 ceph-mon[102832]: slow request 240.945295 seconds old, received at 2023-09-09T23:16:14.258282+0000: client_request(mds.1:1699 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:16.688 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:16 smithi170 ceph-mon[102832]: slow request 240.886277 seconds old, received at 2023-09-09T23:16:14.317300+0000: client_request(mds.1:1731 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:16.688 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:16 smithi170 ceph-mon[102832]: slow request 240.827742 seconds old, received at 2023-09-09T23:16:14.375835+0000: client_request(mds.1:1763 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:16.688 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:16 smithi170 ceph-mon[102832]: slow request 240.769800 seconds old, received at 2023-09-09T23:16:14.433777+0000: client_request(mds.1:1795 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:16.688 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:16 smithi170 ceph-mon[102832]: slow request 240.711186 seconds old, received at 2023-09-09T23:16:14.492391+0000: client_request(mds.1:1827 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:16.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:16.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:16.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:16 smithi082 ceph-mon[98694]: 301 slow requests, 5 included below; oldest blocked for > 241.633069 secs 2023-09-09T23:20:16.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:16 smithi082 ceph-mon[98694]: slow request 240.945295 seconds old, received at 2023-09-09T23:16:14.258282+0000: client_request(mds.1:1699 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:16.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:16 smithi082 ceph-mon[98694]: slow request 240.886277 seconds old, received at 2023-09-09T23:16:14.317300+0000: client_request(mds.1:1731 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:16 smithi082 ceph-mon[98694]: slow request 240.827742 seconds old, received at 2023-09-09T23:16:14.375835+0000: client_request(mds.1:1763 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:16 smithi082 ceph-mon[98694]: slow request 240.769800 seconds old, received at 2023-09-09T23:16:14.433777+0000: client_request(mds.1:1795 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:16 smithi082 ceph-mon[98694]: slow request 240.711186 seconds old, received at 2023-09-09T23:16:14.492391+0000: client_request(mds.1:1827 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:16.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:16 smithi098 ceph-mon[103124]: 301 slow requests, 5 included below; oldest blocked for > 241.633069 secs 2023-09-09T23:20:16.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:16 smithi098 ceph-mon[103124]: slow request 240.945295 seconds old, received at 2023-09-09T23:16:14.258282+0000: client_request(mds.1:1699 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:16 smithi098 ceph-mon[103124]: slow request 240.886277 seconds old, received at 2023-09-09T23:16:14.317300+0000: client_request(mds.1:1731 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:16 smithi098 ceph-mon[103124]: slow request 240.827742 seconds old, received at 2023-09-09T23:16:14.375835+0000: client_request(mds.1:1763 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:16 smithi098 ceph-mon[103124]: slow request 240.769800 seconds old, received at 2023-09-09T23:16:14.433777+0000: client_request(mds.1:1795 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:16 smithi098 ceph-mon[103124]: slow request 240.711186 seconds old, received at 2023-09-09T23:16:14.492391+0000: client_request(mds.1:1827 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:17.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:17 smithi170 ceph-mon[102832]: pgmap v196: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 22 KiB/s rd, 4.0 MiB/s wr, 16 op/s 2023-09-09T23:20:17.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:17 smithi170 ceph-mon[102832]: 597 slow requests, 5 included below; oldest blocked for > 230.882044 secs 2023-09-09T23:20:17.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:17 smithi170 ceph-mon[102832]: slow request 230.855171 seconds old, received at 2023-09-09T23:16:25.832651+0000: peer_request:mds.1:1453 currently dispatched 2023-09-09T23:20:17.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:17 smithi170 ceph-mon[102832]: slow request 230.844207 seconds old, received at 2023-09-09T23:16:25.843616+0000: peer_request:mds.1:1485 currently dispatched 2023-09-09T23:20:17.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:17 smithi170 ceph-mon[102832]: slow request 230.832662 seconds old, received at 2023-09-09T23:16:25.855160+0000: peer_request:mds.1:1517 currently dispatched 2023-09-09T23:20:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:17 smithi170 ceph-mon[102832]: slow request 230.820794 seconds old, received at 2023-09-09T23:16:25.867028+0000: peer_request:mds.1:1549 currently dispatched 2023-09-09T23:20:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:17 smithi170 ceph-mon[102832]: slow request 230.809634 seconds old, received at 2023-09-09T23:16:25.878188+0000: peer_request:mds.1:1581 currently dispatched 2023-09-09T23:20:17.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:17 smithi082 ceph-mon[98694]: pgmap v196: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 22 KiB/s rd, 4.0 MiB/s wr, 16 op/s 2023-09-09T23:20:17.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:17 smithi082 ceph-mon[98694]: 597 slow requests, 5 included below; oldest blocked for > 230.882044 secs 2023-09-09T23:20:17.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:17 smithi082 ceph-mon[98694]: slow request 230.855171 seconds old, received at 2023-09-09T23:16:25.832651+0000: peer_request:mds.1:1453 currently dispatched 2023-09-09T23:20:17.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:17 smithi082 ceph-mon[98694]: slow request 230.844207 seconds old, received at 2023-09-09T23:16:25.843616+0000: peer_request:mds.1:1485 currently dispatched 2023-09-09T23:20:17.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:17 smithi082 ceph-mon[98694]: slow request 230.832662 seconds old, received at 2023-09-09T23:16:25.855160+0000: peer_request:mds.1:1517 currently dispatched 2023-09-09T23:20:17.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:17 smithi082 ceph-mon[98694]: slow request 230.820794 seconds old, received at 2023-09-09T23:16:25.867028+0000: peer_request:mds.1:1549 currently dispatched 2023-09-09T23:20:17.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:17 smithi082 ceph-mon[98694]: slow request 230.809634 seconds old, received at 2023-09-09T23:16:25.878188+0000: peer_request:mds.1:1581 currently dispatched 2023-09-09T23:20:17.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:17 smithi098 ceph-mon[103124]: pgmap v196: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 22 KiB/s rd, 4.0 MiB/s wr, 16 op/s 2023-09-09T23:20:17.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:17 smithi098 ceph-mon[103124]: 597 slow requests, 5 included below; oldest blocked for > 230.882044 secs 2023-09-09T23:20:17.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:17 smithi098 ceph-mon[103124]: slow request 230.855171 seconds old, received at 2023-09-09T23:16:25.832651+0000: peer_request:mds.1:1453 currently dispatched 2023-09-09T23:20:17.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:17 smithi098 ceph-mon[103124]: slow request 230.844207 seconds old, received at 2023-09-09T23:16:25.843616+0000: peer_request:mds.1:1485 currently dispatched 2023-09-09T23:20:17.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:17 smithi098 ceph-mon[103124]: slow request 230.832662 seconds old, received at 2023-09-09T23:16:25.855160+0000: peer_request:mds.1:1517 currently dispatched 2023-09-09T23:20:17.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:17 smithi098 ceph-mon[103124]: slow request 230.820794 seconds old, received at 2023-09-09T23:16:25.867028+0000: peer_request:mds.1:1549 currently dispatched 2023-09-09T23:20:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:17 smithi098 ceph-mon[103124]: slow request 230.809634 seconds old, received at 2023-09-09T23:16:25.878188+0000: peer_request:mds.1:1581 currently dispatched 2023-09-09T23:20:19.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:19 smithi170 ceph-mon[102832]: pgmap v197: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 19 KiB/s rd, 3.5 MiB/s wr, 14 op/s 2023-09-09T23:20:19.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:19 smithi082 ceph-mon[98694]: pgmap v197: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 19 KiB/s rd, 3.5 MiB/s wr, 14 op/s 2023-09-09T23:20:19.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:19 smithi098 ceph-mon[103124]: pgmap v197: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 19 KiB/s rd, 3.5 MiB/s wr, 14 op/s 2023-09-09T23:20:20.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:20.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:20.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:20.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:20.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:20.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:21.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:21 smithi170 ceph-mon[102832]: 292 slow requests, 5 included below; oldest blocked for > 246.635395 secs 2023-09-09T23:20:21.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:21 smithi170 ceph-mon[102832]: slow request 245.676963 seconds old, received at 2023-09-09T23:16:14.546526+0000: client_request(mds.1:1859 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:21.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:21 smithi170 ceph-mon[102832]: slow request 245.624363 seconds old, received at 2023-09-09T23:16:14.599126+0000: client_request(mds.1:1891 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:21.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:21 smithi170 ceph-mon[102832]: slow request 245.570251 seconds old, received at 2023-09-09T23:16:14.653238+0000: client_request(mds.1:1923 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:21.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:21 smithi170 ceph-mon[102832]: slow request 245.514920 seconds old, received at 2023-09-09T23:16:14.708569+0000: client_request(mds.1:1955 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:21 smithi170 ceph-mon[102832]: slow request 245.456591 seconds old, received at 2023-09-09T23:16:14.766898+0000: client_request(mds.1:1987 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:21 smithi170 ceph-mon[102832]: pgmap v198: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 17 KiB/s rd, 3.5 MiB/s wr, 14 op/s 2023-09-09T23:20:21.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:21 smithi082 ceph-mon[98694]: 292 slow requests, 5 included below; oldest blocked for > 246.635395 secs 2023-09-09T23:20:21.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:21 smithi082 ceph-mon[98694]: slow request 245.676963 seconds old, received at 2023-09-09T23:16:14.546526+0000: client_request(mds.1:1859 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:21.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:21 smithi082 ceph-mon[98694]: slow request 245.624363 seconds old, received at 2023-09-09T23:16:14.599126+0000: client_request(mds.1:1891 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:21.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:21 smithi082 ceph-mon[98694]: slow request 245.570251 seconds old, received at 2023-09-09T23:16:14.653238+0000: client_request(mds.1:1923 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:21.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:21 smithi082 ceph-mon[98694]: slow request 245.514920 seconds old, received at 2023-09-09T23:16:14.708569+0000: client_request(mds.1:1955 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:21.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:21 smithi082 ceph-mon[98694]: slow request 245.456591 seconds old, received at 2023-09-09T23:16:14.766898+0000: client_request(mds.1:1987 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:21.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:21 smithi082 ceph-mon[98694]: pgmap v198: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 17 KiB/s rd, 3.5 MiB/s wr, 14 op/s 2023-09-09T23:20:21.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:21 smithi098 ceph-mon[103124]: 292 slow requests, 5 included below; oldest blocked for > 246.635395 secs 2023-09-09T23:20:21.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:21 smithi098 ceph-mon[103124]: slow request 245.676963 seconds old, received at 2023-09-09T23:16:14.546526+0000: client_request(mds.1:1859 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:21.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:21 smithi098 ceph-mon[103124]: slow request 245.624363 seconds old, received at 2023-09-09T23:16:14.599126+0000: client_request(mds.1:1891 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:21.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:21 smithi098 ceph-mon[103124]: slow request 245.570251 seconds old, received at 2023-09-09T23:16:14.653238+0000: client_request(mds.1:1923 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:21.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:21 smithi098 ceph-mon[103124]: slow request 245.514920 seconds old, received at 2023-09-09T23:16:14.708569+0000: client_request(mds.1:1955 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:21 smithi098 ceph-mon[103124]: slow request 245.456591 seconds old, received at 2023-09-09T23:16:14.766898+0000: client_request(mds.1:1987 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:21 smithi098 ceph-mon[103124]: pgmap v198: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 17 KiB/s rd, 3.5 MiB/s wr, 14 op/s 2023-09-09T23:20:22.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:22 smithi170 ceph-mon[102832]: 590 slow requests, 5 included below; oldest blocked for > 235.878118 secs 2023-09-09T23:20:22.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:22 smithi170 ceph-mon[102832]: slow request 235.798480 seconds old, received at 2023-09-09T23:16:25.889416+0000: peer_request:mds.1:1613 currently dispatched 2023-09-09T23:20:22.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:22 smithi170 ceph-mon[102832]: slow request 235.787821 seconds old, received at 2023-09-09T23:16:25.900074+0000: peer_request:mds.1:1645 currently dispatched 2023-09-09T23:20:22.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:22 smithi170 ceph-mon[102832]: slow request 235.775672 seconds old, received at 2023-09-09T23:16:25.912223+0000: peer_request:mds.1:1677 currently dispatched 2023-09-09T23:20:22.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:22 smithi170 ceph-mon[102832]: slow request 235.764820 seconds old, received at 2023-09-09T23:16:25.923075+0000: peer_request:mds.1:1709 currently dispatched 2023-09-09T23:20:22.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:22 smithi170 ceph-mon[102832]: slow request 235.752484 seconds old, received at 2023-09-09T23:16:25.935411+0000: peer_request:mds.1:1741 currently dispatched 2023-09-09T23:20:22.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:22 smithi082 ceph-mon[98694]: 590 slow requests, 5 included below; oldest blocked for > 235.878118 secs 2023-09-09T23:20:22.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:22 smithi082 ceph-mon[98694]: slow request 235.798480 seconds old, received at 2023-09-09T23:16:25.889416+0000: peer_request:mds.1:1613 currently dispatched 2023-09-09T23:20:22.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:22 smithi082 ceph-mon[98694]: slow request 235.787821 seconds old, received at 2023-09-09T23:16:25.900074+0000: peer_request:mds.1:1645 currently dispatched 2023-09-09T23:20:22.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:22 smithi082 ceph-mon[98694]: slow request 235.775672 seconds old, received at 2023-09-09T23:16:25.912223+0000: peer_request:mds.1:1677 currently dispatched 2023-09-09T23:20:22.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:22 smithi082 ceph-mon[98694]: slow request 235.764820 seconds old, received at 2023-09-09T23:16:25.923075+0000: peer_request:mds.1:1709 currently dispatched 2023-09-09T23:20:22.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:22 smithi082 ceph-mon[98694]: slow request 235.752484 seconds old, received at 2023-09-09T23:16:25.935411+0000: peer_request:mds.1:1741 currently dispatched 2023-09-09T23:20:22.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:22 smithi098 ceph-mon[103124]: 590 slow requests, 5 included below; oldest blocked for > 235.878118 secs 2023-09-09T23:20:22.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:22 smithi098 ceph-mon[103124]: slow request 235.798480 seconds old, received at 2023-09-09T23:16:25.889416+0000: peer_request:mds.1:1613 currently dispatched 2023-09-09T23:20:22.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:22 smithi098 ceph-mon[103124]: slow request 235.787821 seconds old, received at 2023-09-09T23:16:25.900074+0000: peer_request:mds.1:1645 currently dispatched 2023-09-09T23:20:22.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:22 smithi098 ceph-mon[103124]: slow request 235.775672 seconds old, received at 2023-09-09T23:16:25.912223+0000: peer_request:mds.1:1677 currently dispatched 2023-09-09T23:20:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:22 smithi098 ceph-mon[103124]: slow request 235.764820 seconds old, received at 2023-09-09T23:16:25.923075+0000: peer_request:mds.1:1709 currently dispatched 2023-09-09T23:20:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:22 smithi098 ceph-mon[103124]: slow request 235.752484 seconds old, received at 2023-09-09T23:16:25.935411+0000: peer_request:mds.1:1741 currently dispatched 2023-09-09T23:20:23.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:23 smithi170 ceph-mon[102832]: pgmap v199: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 17 KiB/s rd, 3.5 MiB/s wr, 14 op/s 2023-09-09T23:20:23.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:20:23.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:23.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:23 smithi082 ceph-mon[98694]: pgmap v199: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 17 KiB/s rd, 3.5 MiB/s wr, 14 op/s 2023-09-09T23:20:23.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:20:23.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:23.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:23 smithi098 ceph-mon[103124]: pgmap v199: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 17 KiB/s rd, 3.5 MiB/s wr, 14 op/s 2023-09-09T23:20:23.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:20:23.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:24.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:24.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:24.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:24.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:24.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:24.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:25.341 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:20:25.481 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.482+0000 7f5b8c330700 1 -- 172.21.15.82:0/1964471627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b84102040 msgr2=0x7f5b8410a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:20:25.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.482+0000 7f5b8c330700 1 --2- 172.21.15.82:0/1964471627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b84102040 0x7f5b8410a520 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5b80009990 tx=0x7f5b80009ca0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.482+0000 7f5b8c330700 1 -- 172.21.15.82:0/1964471627 shutdown_connections 2023-09-09T23:20:25.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.482+0000 7f5b8c330700 1 --2- 172.21.15.82:0/1964471627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b8410ab40 0x7f5b8410d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.483 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.482+0000 7f5b8c330700 1 --2- 172.21.15.82:0/1964471627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b84101720 0x7f5b84101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.483 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.482+0000 7f5b8c330700 1 --2- 172.21.15.82:0/1964471627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b84102040 0x7f5b8410a520 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.483 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.482+0000 7f5b8c330700 1 -- 172.21.15.82:0/1964471627 >> 172.21.15.82:0/1964471627 conn(0x7f5b840faf20 msgr2=0x7f5b840fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:20:25.483 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.482+0000 7f5b8c330700 1 -- 172.21.15.82:0/1964471627 shutdown_connections 2023-09-09T23:20:25.484 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.482+0000 7f5b8c330700 1 -- 172.21.15.82:0/1964471627 wait complete. 2023-09-09T23:20:25.484 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.483+0000 7f5b8c330700 1 Processor -- start 2023-09-09T23:20:25.484 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.483+0000 7f5b8c330700 1 -- start start 2023-09-09T23:20:25.488 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.484+0000 7f5b8c330700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b84101720 0x7f5b84196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:25.489 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.484+0000 7f5b8c330700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b84102040 0x7f5b84196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:25.489 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.484+0000 7f5b8c330700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b8410ab40 0x7f5b8419b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:25.489 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.484+0000 7f5b8c330700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8419b640 con 0x7f5b8410ab40 2023-09-09T23:20:25.489 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.484+0000 7f5b8c330700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8419b7b0 con 0x7f5b84101720 2023-09-09T23:20:25.489 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.484+0000 7f5b8c330700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8419b970 con 0x7f5b84102040 2023-09-09T23:20:25.490 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.484+0000 7f5b8a8cd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b8410ab40 0x7f5b8419b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:25.490 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.484+0000 7f5b8a0cc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b84101720 0x7f5b84196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:25.490 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.484+0000 7f5b8a8cd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b8410ab40 0x7f5b8419b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57950/0 (socket says 172.21.15.82:57950) 2023-09-09T23:20:25.490 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.484+0000 7f5b8a8cd700 1 -- 172.21.15.82:0/849373103 learned_addr learned my addr 172.21.15.82:0/849373103 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:20:25.491 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.484+0000 7f5b898cb700 1 --2- 172.21.15.82:0/849373103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b84102040 0x7f5b84196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:25.491 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.484+0000 7f5b8a8cd700 1 -- 172.21.15.82:0/849373103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b84102040 msgr2=0x7f5b84196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:20:25.491 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.484+0000 7f5b8a8cd700 1 --2- 172.21.15.82:0/849373103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b84102040 0x7f5b84196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.484+0000 7f5b8a8cd700 1 -- 172.21.15.82:0/849373103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b84101720 msgr2=0x7f5b84196660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:20:25.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.484+0000 7f5b8a8cd700 1 --2- 172.21.15.82:0/849373103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b84101720 0x7f5b84196660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.484+0000 7f5b8a8cd700 1 -- 172.21.15.82:0/849373103 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b800095f0 con 0x7f5b8410ab40 2023-09-09T23:20:25.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.484+0000 7f5b8a8cd700 1 --2- 172.21.15.82:0/849373103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b8410ab40 0x7f5b8419b000 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c00ef30 tx=0x7f5b7c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:20:25.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.484+0000 7f5b7b7fe700 1 -- 172.21.15.82:0/849373103 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c018070 con 0x7f5b8410ab40 2023-09-09T23:20:25.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.484+0000 7f5b7b7fe700 1 -- 172.21.15.82:0/849373103 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b7c00f040 con 0x7f5b8410ab40 2023-09-09T23:20:25.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.484+0000 7f5b8c330700 1 -- 172.21.15.82:0/849373103 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b8419bbf0 con 0x7f5b8410ab40 2023-09-09T23:20:25.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.484+0000 7f5b7b7fe700 1 -- 172.21.15.82:0/849373103 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c00cd00 con 0x7f5b8410ab40 2023-09-09T23:20:25.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.484+0000 7f5b8c330700 1 -- 172.21.15.82:0/849373103 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b841a3490 con 0x7f5b8410ab40 2023-09-09T23:20:25.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.486+0000 7f5b7b7fe700 1 -- 172.21.15.82:0/849373103 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f5b7c003a00 con 0x7f5b8410ab40 2023-09-09T23:20:25.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.487+0000 7f5b7b7fe700 1 --2- 172.21.15.82:0/849373103 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f5b700a0830 0x7f5b700a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:25.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.487+0000 7f5b7b7fe700 1 -- 172.21.15.82:0/849373103 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f5b7c0cb9d0 con 0x7f5b8410ab40 2023-09-09T23:20:25.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.487+0000 7f5b8a0cc700 1 --2- 172.21.15.82:0/849373103 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f5b700a0830 0x7f5b700a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:25.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.487+0000 7f5b8a0cc700 1 --2- 172.21.15.82:0/849373103 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f5b700a0830 0x7f5b700a2cf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5b74009c60 tx=0x7f5b74009260 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:20:25.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.488+0000 7f5b984f6b80 1 Processor -- start 2023-09-09T23:20:25.495 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.488+0000 7f5b984f6b80 1 -- start start 2023-09-09T23:20:25.495 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.488+0000 7f5b984f6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564fbebcb710 0x564fbebcbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:25.495 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.488+0000 7f5b984f6b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fbebcc030 0x564fbebd9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:25.495 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.488+0000 7f5b984f6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564fbebda250 0x564fbebdc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:25.495 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.488+0000 7f5b984f6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fbeb24730 con 0x564fbebcb710 2023-09-09T23:20:25.496 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.488+0000 7f5b984f6b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fbeb22050 con 0x564fbebcc030 2023-09-09T23:20:25.496 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.488+0000 7f5b984f6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fbeb1d940 con 0x564fbebda250 2023-09-09T23:20:25.496 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.488+0000 7f5b8a0cc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564fbebcb710 0x564fbebcbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:25.496 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.488+0000 7f5b8a0cc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564fbebcb710 0x564fbebcbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57954/0 (socket says 172.21.15.82:57954) 2023-09-09T23:20:25.496 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.488+0000 7f5b8a0cc700 1 -- 172.21.15.82:0/2651828322 learned_addr learned my addr 172.21.15.82:0/2651828322 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:20:25.497 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.488+0000 7f5b898cb700 1 --2- 172.21.15.82:0/2651828322 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fbebcc030 0x564fbebd9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:25.497 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.488+0000 7f5b8a8cd700 1 --2- 172.21.15.82:0/2651828322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564fbebda250 0x564fbebdc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:25.497 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.489+0000 7f5b8a0cc700 1 -- 172.21.15.82:0/2651828322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564fbebda250 msgr2=0x564fbebdc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:20:25.497 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.489+0000 7f5b8a0cc700 1 --2- 172.21.15.82:0/2651828322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564fbebda250 0x564fbebdc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.497 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.489+0000 7f5b8a0cc700 1 -- 172.21.15.82:0/2651828322 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fbebcc030 msgr2=0x564fbebd9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:20:25.498 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.489+0000 7f5b8a0cc700 1 --2- 172.21.15.82:0/2651828322 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fbebcc030 0x564fbebd9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.498 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.489+0000 7f5b8a0cc700 1 -- 172.21.15.82:0/2651828322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564fbe949de0 con 0x564fbebcb710 2023-09-09T23:20:25.498 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.489+0000 7f5b8a0cc700 1 --2- 172.21.15.82:0/2651828322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564fbebcb710 0x564fbebcbaf0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400e4e0 tx=0x7f5b74014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21541d33ad4270df server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:20:25.498 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.489+0000 7f5b78d89700 1 -- 172.21.15.82:0/2651828322 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7400ee00 con 0x564fbebcb710 2023-09-09T23:20:25.498 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.489+0000 7f5b78d89700 1 -- 172.21.15.82:0/2651828322 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b74021070 con 0x564fbebcb710 2023-09-09T23:20:25.498 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.489+0000 7f5b78d89700 1 -- 172.21.15.82:0/2651828322 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7401d7b0 con 0x564fbebcb710 2023-09-09T23:20:25.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.490+0000 7f5b984f6b80 1 -- 172.21.15.82:0/2651828322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564fbebcb710 msgr2=0x564fbebcbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:20:25.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.490+0000 7f5b984f6b80 1 --2- 172.21.15.82:0/2651828322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564fbebcb710 0x564fbebcbaf0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400e4e0 tx=0x7f5b74014000 comp rx=0 tx=0).stop 2023-09-09T23:20:25.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.490+0000 7f5b984f6b80 1 -- 172.21.15.82:0/2651828322 shutdown_connections 2023-09-09T23:20:25.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.490+0000 7f5b984f6b80 1 --2- 172.21.15.82:0/2651828322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564fbebda250 0x564fbebdc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.499 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.490+0000 7f5b984f6b80 1 --2- 172.21.15.82:0/2651828322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564fbebcb710 0x564fbebcbaf0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.490+0000 7f5b984f6b80 1 --2- 172.21.15.82:0/2651828322 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fbebcc030 0x564fbebd9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.490+0000 7f5b984f6b80 1 -- 172.21.15.82:0/2651828322 >> 172.21.15.82:0/2651828322 conn(0x564fbeb40090 msgr2=0x564fbebc6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:20:25.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.490+0000 7f5b984f6b80 1 -- 172.21.15.82:0/2651828322 shutdown_connections 2023-09-09T23:20:25.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.490+0000 7f5b984f6b80 1 -- 172.21.15.82:0/2651828322 wait complete. 2023-09-09T23:20:25.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.490+0000 7f5b984f6b80 1 -- start start 2023-09-09T23:20:25.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.490+0000 7f5b984f6b80 1 Processor -- start 2023-09-09T23:20:25.504 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.490+0000 7f5b984f6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564fbebcb710 0x564fbebd9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:25.504 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.490+0000 7f5b984f6b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fbebcc030 0x564fbebd6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:25.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.490+0000 7f5b984f6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564fbebda250 0x564fbebd73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:25.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.490+0000 7f5b984f6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fbeabbb70 con 0x564fbebda250 2023-09-09T23:20:25.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.490+0000 7f5b984f6b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fbeabb870 con 0x564fbebcc030 2023-09-09T23:20:25.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.490+0000 7f5b984f6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fbe99f920 con 0x564fbebcb710 2023-09-09T23:20:25.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.490+0000 7f5b984f6b80 10 client.0 _ll_register_callbacks cb 0x564fbea05a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:20:25.506 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.491+0000 7f5b8a8cd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564fbebda250 0x564fbebd73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:25.506 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.491+0000 7f5b898cb700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fbebcc030 0x564fbebd6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:25.506 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.491+0000 7f5b8a0cc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564fbebcb710 0x564fbebd9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:25.506 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.491+0000 7f5b8a8cd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564fbebda250 0x564fbebd73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57964/0 (socket says 172.21.15.82:57964) 2023-09-09T23:20:25.506 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.491+0000 7f5b8a8cd700 1 -- 172.21.15.82:0/2875621871 learned_addr learned my addr 172.21.15.82:0/2875621871 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:20:25.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.491+0000 7f5b8a8cd700 1 -- 172.21.15.82:0/2875621871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564fbebcb710 msgr2=0x564fbebd9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:20:25.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.491+0000 7f5b8a8cd700 1 --2- 172.21.15.82:0/2875621871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564fbebcb710 0x564fbebd9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.491+0000 7f5b8a8cd700 1 -- 172.21.15.82:0/2875621871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fbebcc030 msgr2=0x564fbebd6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:20:25.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.491+0000 7f5b8a8cd700 1 --2- 172.21.15.82:0/2875621871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fbebcc030 0x564fbebd6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.491+0000 7f5b8a8cd700 1 -- 172.21.15.82:0/2875621871 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b74017070 con 0x564fbebda250 2023-09-09T23:20:25.508 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.491+0000 7f5b8a8cd700 1 --2- 172.21.15.82:0/2875621871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564fbebda250 0x564fbebd73f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c098c50 tx=0x7f5b7c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8abe7669940e7c server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:20:25.508 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.491+0000 7f5b6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:20:25.508 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.491+0000 7f5b6dffb700 1 -- 172.21.15.82:0/2875621871 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c0b0070 con 0x564fbebda250 2023-09-09T23:20:25.508 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.491+0000 7f5b6dffb700 1 -- 172.21.15.82:0/2875621871 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f5b7c099650 con 0x564fbebda250 2023-09-09T23:20:25.509 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.491+0000 7f5b984f6b80 1 -- 172.21.15.82:0/2875621871 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564fbea754f0 con 0x564fbebda250 2023-09-09T23:20:25.509 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.491+0000 7f5b6dffb700 1 -- 172.21.15.82:0/2875621871 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b7c00ce60 con 0x564fbebda250 2023-09-09T23:20:25.509 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.491+0000 7f5b984f6b80 10 client.16155 fetch_fsmap learned FSMap version 65 2023-09-09T23:20:25.509 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.491+0000 7f5b984f6b80 1 -- 172.21.15.82:0/2875621871 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x564fbe932c60 con 0x564fbebda250 2023-09-09T23:20:25.509 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.491+0000 7f5b6dffb700 1 -- 172.21.15.82:0/2875621871 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b7c07b020 con 0x564fbebda250 2023-09-09T23:20:25.510 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.492+0000 7f5b6dffb700 1 -- 172.21.15.82:0/2875621871 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 65) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f5b7c098440 con 0x564fbebda250 2023-09-09T23:20:25.510 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.492+0000 7f5b984f6b80 10 client.16155 fetch_fsmap finished waiting for FSMap version 65 2023-09-09T23:20:25.510 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.492+0000 7f5b984f6b80 10 client.16155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:20:25.510 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.492+0000 7f5b984f6b80 20 client.16155 populate_metadata read hostname 'smithi082' 2023-09-09T23:20:25.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.492+0000 7f5b984f6b80 1 --2- 172.21.15.82:0/2875621871 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x564fbebe9b90 0x564fbebebfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:25.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.492+0000 7f5b984f6b80 4 client.16155 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:20:25.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.492+0000 7f5b984f6b80 1 -- 172.21.15.82:0/2875621871 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564fbeb328a0 con 0x564fbebe9b90 2023-09-09T23:20:25.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.492+0000 7f5b8a0cc700 1 --2- 172.21.15.82:0/2875621871 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x564fbebe9b90 0x564fbebebfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:25.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.492+0000 7f5b6dffb700 10 client.16155 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:20:25.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.492+0000 7f5b8a0cc700 1 --2- 172.21.15.82:0/2875621871 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x564fbebe9b90 0x564fbebebfb0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df1cb5492f147918 server_cookie=c394ccceadd4384c in_seq=0 out_seq=0 2023-09-09T23:20:25.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.493+0000 7f5b6dffb700 1 -- 172.21.15.82:0/2875621871 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564fbeb328a0 con 0x564fbebe9b90 2023-09-09T23:20:25.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.493+0000 7f5b6dffb700 10 client.16155 handle_command_reply: tid=0 2023-09-09T23:20:25.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.493+0000 7f5b984f6b80 1 client.16155 shutdown 2023-09-09T23:20:25.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.493+0000 7f5b984f6b80 1 -- 172.21.15.82:0/2875621871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564fbebda250 msgr2=0x564fbebd73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:20:25.513 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.493+0000 7f5b984f6b80 1 --2- 172.21.15.82:0/2875621871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564fbebda250 0x564fbebd73f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c098c50 tx=0x7f5b7c004b40 comp rx=0 tx=0).stop 2023-09-09T23:20:25.513 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.493+0000 7f5b984f6b80 1 -- 172.21.15.82:0/2875621871 shutdown_connections 2023-09-09T23:20:25.513 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.493+0000 7f5b984f6b80 1 --2- 172.21.15.82:0/2875621871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564fbebda250 0x564fbebd73f0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.513 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.493+0000 7f5b984f6b80 1 --2- 172.21.15.82:0/2875621871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564fbebcb710 0x564fbebd9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.513 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.493+0000 7f5b984f6b80 1 --2- 172.21.15.82:0/2875621871 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x564fbebe9b90 0x564fbebebfb0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.514 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.493+0000 7f5b984f6b80 1 --2- 172.21.15.82:0/2875621871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fbebcc030 0x564fbebd6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.514 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.493+0000 7f5b6dffb700 0 client.16155 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:20:25.514 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.493+0000 7f5b984f6b80 1 -- 172.21.15.82:0/2875621871 >> 172.21.15.82:0/2875621871 conn(0x564fbeb40090 msgr2=0x564fbebc67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:20:25.514 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.493+0000 7f5b984f6b80 1 -- 172.21.15.82:0/2875621871 shutdown_connections 2023-09-09T23:20:25.514 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.493+0000 7f5b984f6b80 1 -- 172.21.15.82:0/2875621871 wait complete. 2023-09-09T23:20:25.515 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.493+0000 7f5b984f6b80 20 client.16155 trim_cache size 0 max 16384 2023-09-09T23:20:25.520 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.510+0000 7f5b984f6b80 1 Processor -- start 2023-09-09T23:20:25.520 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.510+0000 7f5b984f6b80 1 -- start start 2023-09-09T23:20:25.520 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.510+0000 7f5b984f6b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fbebcb710 0x564fbebe0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:25.520 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.510+0000 7f5b984f6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564fbebcc030 0x564fbebd40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:25.521 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.510+0000 7f5b984f6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564fbebda250 0x564fbebd4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:25.521 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.510+0000 7f5b984f6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fbebd1cb0 con 0x564fbebcc030 2023-09-09T23:20:25.521 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.510+0000 7f5b984f6b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fbebc8240 con 0x564fbebcb710 2023-09-09T23:20:25.521 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.510+0000 7f5b984f6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fbebcc9f0 con 0x564fbebda250 2023-09-09T23:20:25.521 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.510+0000 7f5b898cb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564fbebcc030 0x564fbebd40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:25.521 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.510+0000 7f5b898cb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564fbebcc030 0x564fbebd40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57970/0 (socket says 172.21.15.82:57970) 2023-09-09T23:20:25.522 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.510+0000 7f5b898cb700 1 -- 172.21.15.82:0/554811263 learned_addr learned my addr 172.21.15.82:0/554811263 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:20:25.522 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.510+0000 7f5b8a0cc700 1 --2- 172.21.15.82:0/554811263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fbebcb710 0x564fbebe0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:25.522 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.510+0000 7f5b8a8cd700 1 --2- 172.21.15.82:0/554811263 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564fbebda250 0x564fbebd4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:25.522 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.510+0000 7f5b898cb700 1 -- 172.21.15.82:0/554811263 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564fbebda250 msgr2=0x564fbebd4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:20:25.522 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.510+0000 7f5b898cb700 1 --2- 172.21.15.82:0/554811263 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564fbebda250 0x564fbebd4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.523 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.510+0000 7f5b898cb700 1 -- 172.21.15.82:0/554811263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fbebcb710 msgr2=0x564fbebe0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:20:25.523 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.510+0000 7f5b898cb700 1 --2- 172.21.15.82:0/554811263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fbebcb710 0x564fbebe0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.523 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.510+0000 7f5b898cb700 1 -- 172.21.15.82:0/554811263 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564fbe970250 con 0x564fbebcc030 2023-09-09T23:20:25.523 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.511+0000 7f5b898cb700 1 --2- 172.21.15.82:0/554811263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564fbebcc030 0x564fbebd40c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f5b80009990 tx=0x7f5b80005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27afd223715b94c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:20:25.523 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.511+0000 7f5b7958a700 1 -- 172.21.15.82:0/554811263 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8001d070 con 0x564fbebcc030 2023-09-09T23:20:25.524 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.511+0000 7f5b7958a700 1 -- 172.21.15.82:0/554811263 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b8001e040 con 0x564fbebcc030 2023-09-09T23:20:25.524 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.511+0000 7f5b7958a700 1 -- 172.21.15.82:0/554811263 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b80004760 con 0x564fbebcc030 2023-09-09T23:20:25.524 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.511+0000 7f5b984f6b80 1 -- 172.21.15.82:0/554811263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564fbebcc030 msgr2=0x564fbebd40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:20:25.524 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.511+0000 7f5b984f6b80 1 --2- 172.21.15.82:0/554811263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564fbebcc030 0x564fbebd40c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f5b80009990 tx=0x7f5b80005fd0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.524 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.511+0000 7f5b984f6b80 1 -- 172.21.15.82:0/554811263 shutdown_connections 2023-09-09T23:20:25.525 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.511+0000 7f5b984f6b80 1 --2- 172.21.15.82:0/554811263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fbebcb710 0x564fbebe0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.525 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.511+0000 7f5b984f6b80 1 --2- 172.21.15.82:0/554811263 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564fbebda250 0x564fbebd4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.525 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.511+0000 7f5b984f6b80 1 --2- 172.21.15.82:0/554811263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564fbebcc030 0x564fbebd40c0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.525 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.511+0000 7f5b984f6b80 1 -- 172.21.15.82:0/554811263 >> 172.21.15.82:0/554811263 conn(0x564fbeb40090 msgr2=0x564fbebd2130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:20:25.525 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.511+0000 7f5b984f6b80 1 -- 172.21.15.82:0/554811263 shutdown_connections 2023-09-09T23:20:25.525 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.511+0000 7f5b984f6b80 1 -- 172.21.15.82:0/554811263 wait complete. 2023-09-09T23:20:25.526 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.512+0000 7f5b984f6b80 1 -- start start 2023-09-09T23:20:25.526 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.512+0000 7f5b984f6b80 1 Processor -- start 2023-09-09T23:20:25.526 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.512+0000 7f5b984f6b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fbebcb710 0x564fbec37710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:25.526 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.512+0000 7f5b984f6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564fbebda250 0x564fbec37c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:25.526 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.512+0000 7f5b984f6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564fbec31a10 0x564fbec31e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:25.527 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.512+0000 7f5b984f6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fbec323d0 con 0x564fbec31a10 2023-09-09T23:20:25.527 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.512+0000 7f5b984f6b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fbec32540 con 0x564fbebcb710 2023-09-09T23:20:25.527 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.512+0000 7f5b984f6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fbec32700 con 0x564fbebda250 2023-09-09T23:20:25.527 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.512+0000 7f5b984f6b80 10 client.0 _ll_register_callbacks cb 0x7f5b7c098b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:20:25.527 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.512+0000 7f5b8a8cd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564fbec31a10 0x564fbec31e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:25.528 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.512+0000 7f5b8a0cc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fbebcb710 0x564fbec37710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:25.528 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.512+0000 7f5b8a8cd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564fbec31a10 0x564fbec31e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57978/0 (socket says 172.21.15.82:57978) 2023-09-09T23:20:25.528 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.512+0000 7f5b8a8cd700 1 -- 172.21.15.82:0/3038528596 learned_addr learned my addr 172.21.15.82:0/3038528596 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:20:25.528 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.512+0000 7f5b898cb700 1 --2- 172.21.15.82:0/3038528596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564fbebda250 0x564fbec37c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:25.528 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.512+0000 7f5b8a8cd700 1 -- 172.21.15.82:0/3038528596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564fbebda250 msgr2=0x564fbec37c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:20:25.529 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.512+0000 7f5b8a8cd700 1 --2- 172.21.15.82:0/3038528596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564fbebda250 0x564fbec37c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.529 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.512+0000 7f5b8a8cd700 1 -- 172.21.15.82:0/3038528596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fbebcb710 msgr2=0x564fbec37710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:20:25.529 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.512+0000 7f5b8a8cd700 1 --2- 172.21.15.82:0/3038528596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fbebcb710 0x564fbec37710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.529 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.512+0000 7f5b8a8cd700 1 -- 172.21.15.82:0/3038528596 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b8410dcc0 con 0x564fbec31a10 2023-09-09T23:20:25.529 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.513+0000 7f5b8a8cd700 1 --2- 172.21.15.82:0/3038528596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564fbec31a10 0x564fbec31e90 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c00c6f0 tx=0x7f5b7c097500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15348fc606c438a3 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:20:25.530 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.513+0000 7f5b6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:20:25.530 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.513+0000 7f5b6dffb700 1 -- 172.21.15.82:0/3038528596 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c0b0070 con 0x564fbec31a10 2023-09-09T23:20:25.530 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.513+0000 7f5b6dffb700 1 -- 172.21.15.82:0/3038528596 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f5b7c07c080 con 0x564fbec31a10 2023-09-09T23:20:25.530 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.513+0000 7f5b984f6b80 1 -- 172.21.15.82:0/3038528596 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b74021070 con 0x564fbec31a10 2023-09-09T23:20:25.530 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.513+0000 7f5b6dffb700 1 -- 172.21.15.82:0/3038528596 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b7c099650 con 0x564fbec31a10 2023-09-09T23:20:25.531 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.513+0000 7f5b984f6b80 10 client.16161 fetch_fsmap learned FSMap version 65 2023-09-09T23:20:25.531 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.513+0000 7f5b984f6b80 1 -- 172.21.15.82:0/3038528596 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x564fbe9482b0 con 0x564fbec31a10 2023-09-09T23:20:25.531 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.513+0000 7f5b6dffb700 1 -- 172.21.15.82:0/3038528596 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b74021070 con 0x564fbec31a10 2023-09-09T23:20:25.531 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.513+0000 7f5b6dffb700 1 -- 172.21.15.82:0/3038528596 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 65) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f5b7c098440 con 0x564fbec31a10 2023-09-09T23:20:25.531 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.513+0000 7f5b984f6b80 10 client.16161 fetch_fsmap finished waiting for FSMap version 65 2023-09-09T23:20:25.532 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.513+0000 7f5b984f6b80 10 client.16161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:20:25.532 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.513+0000 7f5b984f6b80 20 client.16161 populate_metadata read hostname 'smithi082' 2023-09-09T23:20:25.532 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.513+0000 7f5b984f6b80 1 --2- 172.21.15.82:0/3038528596 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x564fbec3f610 0x564fbec41a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:25.532 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.513+0000 7f5b984f6b80 4 client.16161 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:20:25.532 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.513+0000 7f5b984f6b80 1 -- 172.21.15.82:0/3038528596 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564fbeb16920 con 0x564fbec3f610 2023-09-09T23:20:25.532 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.513+0000 7f5b8a0cc700 1 --2- 172.21.15.82:0/3038528596 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x564fbec3f610 0x564fbec41a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:25.533 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.514+0000 7f5b6dffb700 10 client.16161 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:20:25.533 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.514+0000 7f5b8a0cc700 1 --2- 172.21.15.82:0/3038528596 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x564fbec3f610 0x564fbec41a30 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39df89bd1e1a87f1 server_cookie=91b879bb05a04212 in_seq=0 out_seq=0 2023-09-09T23:20:25.533 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.514+0000 7f5b6dffb700 1 -- 172.21.15.82:0/3038528596 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x564fbeb16920 con 0x564fbec3f610 2023-09-09T23:20:25.533 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.514+0000 7f5b6dffb700 10 client.16161 handle_command_reply: tid=0 2023-09-09T23:20:25.533 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.514+0000 7f5b984f6b80 1 client.16161 shutdown 2023-09-09T23:20:25.534 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.514+0000 7f5b984f6b80 1 -- 172.21.15.82:0/3038528596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564fbec31a10 msgr2=0x564fbec31e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:20:25.534 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.514+0000 7f5b984f6b80 1 --2- 172.21.15.82:0/3038528596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564fbec31a10 0x564fbec31e90 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c00c6f0 tx=0x7f5b7c097500 comp rx=0 tx=0).stop 2023-09-09T23:20:25.537 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.514+0000 7f5b984f6b80 1 -- 172.21.15.82:0/3038528596 shutdown_connections 2023-09-09T23:20:25.537 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.514+0000 7f5b984f6b80 1 --2- 172.21.15.82:0/3038528596 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x564fbec3f610 0x564fbec41a30 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.537 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.514+0000 7f5b984f6b80 1 --2- 172.21.15.82:0/3038528596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564fbebcb710 0x564fbec37710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.537 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.514+0000 7f5b984f6b80 1 --2- 172.21.15.82:0/3038528596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564fbec31a10 0x564fbec31e90 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.537 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.514+0000 7f5b984f6b80 1 --2- 172.21.15.82:0/3038528596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564fbebda250 0x564fbec37c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.538 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.514+0000 7f5b6dffb700 0 client.16161 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:20:25.538 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.514+0000 7f5b984f6b80 1 -- 172.21.15.82:0/3038528596 >> 172.21.15.82:0/3038528596 conn(0x564fbeb40090 msgr2=0x564fbebd86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:20:25.538 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.514+0000 7f5b984f6b80 1 -- 172.21.15.82:0/3038528596 shutdown_connections 2023-09-09T23:20:25.538 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.514+0000 7f5b984f6b80 1 -- 172.21.15.82:0/3038528596 wait complete. 2023-09-09T23:20:25.538 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.514+0000 7f5b984f6b80 20 client.16161 trim_cache size 0 max 16384 2023-09-09T23:20:25.539 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:20:25.539 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:20:25.539 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:20:25.539 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:20:25.540 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:20:25.540 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:20:25.540 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:20:25.540 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:20:25.540 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:20:25.541 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:20:25.542 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.515+0000 7f5b8c330700 1 -- 172.21.15.82:0/849373103 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f5b700a0830 msgr2=0x7f5b700a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:20:25.542 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.515+0000 7f5b8c330700 1 --2- 172.21.15.82:0/849373103 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f5b700a0830 0x7f5b700a2cf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5b74009c60 tx=0x7f5b74009260 comp rx=0 tx=0).stop 2023-09-09T23:20:25.542 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.515+0000 7f5b8c330700 1 -- 172.21.15.82:0/849373103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b8410ab40 msgr2=0x7f5b8419b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:20:25.543 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.515+0000 7f5b8c330700 1 --2- 172.21.15.82:0/849373103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b8410ab40 0x7f5b8419b000 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c00ef30 tx=0x7f5b7c00c3a0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.543 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.515+0000 7f5b8c330700 1 -- 172.21.15.82:0/849373103 shutdown_connections 2023-09-09T23:20:25.543 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.515+0000 7f5b8c330700 1 --2- 172.21.15.82:0/849373103 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f5b700a0830 0x7f5b700a2cf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.543 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.515+0000 7f5b8c330700 1 --2- 172.21.15.82:0/849373103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b84101720 0x7f5b84196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.543 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.515+0000 7f5b8c330700 1 --2- 172.21.15.82:0/849373103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b8410ab40 0x7f5b8419b000 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.544 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.515+0000 7f5b8c330700 1 --2- 172.21.15.82:0/849373103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b84102040 0x7f5b84196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:25.544 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.515+0000 7f5b8c330700 1 -- 172.21.15.82:0/849373103 >> 172.21.15.82:0/849373103 conn(0x7f5b840faf20 msgr2=0x7f5b8410cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:20:25.544 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.515+0000 7f5b8c330700 1 -- 172.21.15.82:0/849373103 shutdown_connections 2023-09-09T23:20:25.544 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:25.515+0000 7f5b8c330700 1 -- 172.21.15.82:0/849373103 wait complete. 2023-09-09T23:20:25.544 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:20:25.704 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:25 smithi082 ceph-mon[98694]: pgmap v200: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 16 KiB/s rd, 3.2 MiB/s wr, 13 op/s 2023-09-09T23:20:25.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:25 smithi098 ceph-mon[103124]: pgmap v200: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 16 KiB/s rd, 3.2 MiB/s wr, 13 op/s 2023-09-09T23:20:25.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:25 smithi170 ceph-mon[102832]: pgmap v200: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 16 KiB/s rd, 3.2 MiB/s wr, 13 op/s 2023-09-09T23:20:26.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:26 smithi082 ceph-mon[98694]: 287 slow requests, 5 included below; oldest blocked for > 251.802550 secs 2023-09-09T23:20:26.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:26 smithi082 ceph-mon[98694]: slow request 250.583829 seconds old, received at 2023-09-09T23:16:14.826501+0000: client_request(mds.1:2019 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:26.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:26 smithi082 ceph-mon[98694]: slow request 250.530234 seconds old, received at 2023-09-09T23:16:14.880096+0000: client_request(mds.1:2051 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:26.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:26 smithi082 ceph-mon[98694]: slow request 251.776385 seconds old, received at 2023-09-09T23:16:13.633945+0000: client_request(mds.1:1412 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:26.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:26 smithi082 ceph-mon[98694]: slow request 251.717407 seconds old, received at 2023-09-09T23:16:13.692923+0000: client_request(mds.1:1444 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:26.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:26 smithi082 ceph-mon[98694]: slow request 251.553186 seconds old, received at 2023-09-09T23:16:13.857144+0000: client_request(mds.1:1476 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:26.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:26 smithi098 ceph-mon[103124]: 287 slow requests, 5 included below; oldest blocked for > 251.802550 secs 2023-09-09T23:20:26.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:26 smithi098 ceph-mon[103124]: slow request 250.583829 seconds old, received at 2023-09-09T23:16:14.826501+0000: client_request(mds.1:2019 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:26.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:26 smithi098 ceph-mon[103124]: slow request 250.530234 seconds old, received at 2023-09-09T23:16:14.880096+0000: client_request(mds.1:2051 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:26.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:26 smithi098 ceph-mon[103124]: slow request 251.776385 seconds old, received at 2023-09-09T23:16:13.633945+0000: client_request(mds.1:1412 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:26.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:26 smithi098 ceph-mon[103124]: slow request 251.717407 seconds old, received at 2023-09-09T23:16:13.692923+0000: client_request(mds.1:1444 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:26.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:26 smithi098 ceph-mon[103124]: slow request 251.553186 seconds old, received at 2023-09-09T23:16:13.857144+0000: client_request(mds.1:1476 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:26.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:26 smithi170 ceph-mon[102832]: 287 slow requests, 5 included below; oldest blocked for > 251.802550 secs 2023-09-09T23:20:26.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:26 smithi170 ceph-mon[102832]: slow request 250.583829 seconds old, received at 2023-09-09T23:16:14.826501+0000: client_request(mds.1:2019 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:26.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:26 smithi170 ceph-mon[102832]: slow request 250.530234 seconds old, received at 2023-09-09T23:16:14.880096+0000: client_request(mds.1:2051 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:26 smithi170 ceph-mon[102832]: slow request 251.776385 seconds old, received at 2023-09-09T23:16:13.633945+0000: client_request(mds.1:1412 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:26 smithi170 ceph-mon[102832]: slow request 251.717407 seconds old, received at 2023-09-09T23:16:13.692923+0000: client_request(mds.1:1444 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:26 smithi170 ceph-mon[102832]: slow request 251.553186 seconds old, received at 2023-09-09T23:16:13.857144+0000: client_request(mds.1:1476 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:27.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:27 smithi082 ceph-mon[98694]: pgmap v201: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 22 KiB/s rd, 2.9 MiB/s wr, 17 op/s 2023-09-09T23:20:27.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:27 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:20:27.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:27 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:27.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:27 smithi082 ceph-mon[98694]: 580 slow requests, 5 included below; oldest blocked for > 240.875146 secs 2023-09-09T23:20:27.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:27 smithi082 ceph-mon[98694]: slow request 240.741262 seconds old, received at 2023-09-09T23:16:25.946715+0000: peer_request:mds.1:1773 currently dispatched 2023-09-09T23:20:27.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:27 smithi082 ceph-mon[98694]: slow request 240.730897 seconds old, received at 2023-09-09T23:16:25.957079+0000: peer_request:mds.1:1805 currently dispatched 2023-09-09T23:20:27.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:27 smithi082 ceph-mon[98694]: slow request 240.719414 seconds old, received at 2023-09-09T23:16:25.968563+0000: peer_request:mds.1:1837 currently dispatched 2023-09-09T23:20:27.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:27 smithi082 ceph-mon[98694]: slow request 240.708155 seconds old, received at 2023-09-09T23:16:25.979821+0000: peer_request:mds.1:1869 currently dispatched 2023-09-09T23:20:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:27 smithi082 ceph-mon[98694]: slow request 240.693558 seconds old, received at 2023-09-09T23:16:25.994419+0000: peer_request:mds.1:1901 currently dispatched 2023-09-09T23:20:27.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:27 smithi098 ceph-mon[103124]: pgmap v201: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 22 KiB/s rd, 2.9 MiB/s wr, 17 op/s 2023-09-09T23:20:27.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:27 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:20:27.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:27 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:27.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:27 smithi098 ceph-mon[103124]: 580 slow requests, 5 included below; oldest blocked for > 240.875146 secs 2023-09-09T23:20:27.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:27 smithi098 ceph-mon[103124]: slow request 240.741262 seconds old, received at 2023-09-09T23:16:25.946715+0000: peer_request:mds.1:1773 currently dispatched 2023-09-09T23:20:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:27 smithi098 ceph-mon[103124]: slow request 240.730897 seconds old, received at 2023-09-09T23:16:25.957079+0000: peer_request:mds.1:1805 currently dispatched 2023-09-09T23:20:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:27 smithi098 ceph-mon[103124]: slow request 240.719414 seconds old, received at 2023-09-09T23:16:25.968563+0000: peer_request:mds.1:1837 currently dispatched 2023-09-09T23:20:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:27 smithi098 ceph-mon[103124]: slow request 240.708155 seconds old, received at 2023-09-09T23:16:25.979821+0000: peer_request:mds.1:1869 currently dispatched 2023-09-09T23:20:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:27 smithi098 ceph-mon[103124]: slow request 240.693558 seconds old, received at 2023-09-09T23:16:25.994419+0000: peer_request:mds.1:1901 currently dispatched 2023-09-09T23:20:27.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:27 smithi170 ceph-mon[102832]: pgmap v201: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 22 KiB/s rd, 2.9 MiB/s wr, 17 op/s 2023-09-09T23:20:27.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:27 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:20:27.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:27 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:27.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:27 smithi170 ceph-mon[102832]: 580 slow requests, 5 included below; oldest blocked for > 240.875146 secs 2023-09-09T23:20:27.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:27 smithi170 ceph-mon[102832]: slow request 240.741262 seconds old, received at 2023-09-09T23:16:25.946715+0000: peer_request:mds.1:1773 currently dispatched 2023-09-09T23:20:27.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:27 smithi170 ceph-mon[102832]: slow request 240.730897 seconds old, received at 2023-09-09T23:16:25.957079+0000: peer_request:mds.1:1805 currently dispatched 2023-09-09T23:20:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:27 smithi170 ceph-mon[102832]: slow request 240.719414 seconds old, received at 2023-09-09T23:16:25.968563+0000: peer_request:mds.1:1837 currently dispatched 2023-09-09T23:20:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:27 smithi170 ceph-mon[102832]: slow request 240.708155 seconds old, received at 2023-09-09T23:16:25.979821+0000: peer_request:mds.1:1869 currently dispatched 2023-09-09T23:20:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:27 smithi170 ceph-mon[102832]: slow request 240.693558 seconds old, received at 2023-09-09T23:16:25.994419+0000: peer_request:mds.1:1901 currently dispatched 2023-09-09T23:20:29.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:29 smithi082 ceph-mon[98694]: pgmap v202: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 KiB/s wr, 10 op/s 2023-09-09T23:20:29.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:29 smithi098 ceph-mon[103124]: pgmap v202: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 KiB/s wr, 10 op/s 2023-09-09T23:20:29.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:29 smithi170 ceph-mon[102832]: pgmap v202: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 KiB/s wr, 10 op/s 2023-09-09T23:20:31.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:31 smithi098 ceph-mon[103124]: pgmap v203: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 19 KiB/s rd, 12 KiB/s wr, 14 op/s 2023-09-09T23:20:31.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:31 smithi098 ceph-mon[103124]: 287 slow requests, 5 included below; oldest blocked for > 256.890701 secs 2023-09-09T23:20:31.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:31 smithi098 ceph-mon[103124]: slow request 256.600261 seconds old, received at 2023-09-09T23:16:13.917357+0000: client_request(mds.1:1508 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:31.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:31 smithi098 ceph-mon[103124]: slow request 256.545136 seconds old, received at 2023-09-09T23:16:13.972482+0000: client_request(mds.1:1540 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:31.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:31 smithi098 ceph-mon[103124]: slow request 256.485268 seconds old, received at 2023-09-09T23:16:14.032351+0000: client_request(mds.1:1572 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:31.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:31 smithi098 ceph-mon[103124]: slow request 256.428918 seconds old, received at 2023-09-09T23:16:14.088700+0000: client_request(mds.1:1604 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:31 smithi098 ceph-mon[103124]: slow request 256.371282 seconds old, received at 2023-09-09T23:16:14.146336+0000: client_request(mds.1:1636 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:31.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:31 smithi170 ceph-mon[102832]: pgmap v203: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 19 KiB/s rd, 12 KiB/s wr, 14 op/s 2023-09-09T23:20:31.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:31 smithi170 ceph-mon[102832]: 287 slow requests, 5 included below; oldest blocked for > 256.890701 secs 2023-09-09T23:20:31.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:31 smithi170 ceph-mon[102832]: slow request 256.600261 seconds old, received at 2023-09-09T23:16:13.917357+0000: client_request(mds.1:1508 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:31.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:31 smithi170 ceph-mon[102832]: slow request 256.545136 seconds old, received at 2023-09-09T23:16:13.972482+0000: client_request(mds.1:1540 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:31.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:31 smithi170 ceph-mon[102832]: slow request 256.485268 seconds old, received at 2023-09-09T23:16:14.032351+0000: client_request(mds.1:1572 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:31.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:31 smithi170 ceph-mon[102832]: slow request 256.428918 seconds old, received at 2023-09-09T23:16:14.088700+0000: client_request(mds.1:1604 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:31 smithi170 ceph-mon[102832]: slow request 256.371282 seconds old, received at 2023-09-09T23:16:14.146336+0000: client_request(mds.1:1636 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:31.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:31 smithi082 ceph-mon[98694]: pgmap v203: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 19 KiB/s rd, 12 KiB/s wr, 14 op/s 2023-09-09T23:20:31.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:31 smithi082 ceph-mon[98694]: 287 slow requests, 5 included below; oldest blocked for > 256.890701 secs 2023-09-09T23:20:31.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:31 smithi082 ceph-mon[98694]: slow request 256.600261 seconds old, received at 2023-09-09T23:16:13.917357+0000: client_request(mds.1:1508 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:31 smithi082 ceph-mon[98694]: slow request 256.545136 seconds old, received at 2023-09-09T23:16:13.972482+0000: client_request(mds.1:1540 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:31 smithi082 ceph-mon[98694]: slow request 256.485268 seconds old, received at 2023-09-09T23:16:14.032351+0000: client_request(mds.1:1572 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:31 smithi082 ceph-mon[98694]: slow request 256.428918 seconds old, received at 2023-09-09T23:16:14.088700+0000: client_request(mds.1:1604 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:31 smithi082 ceph-mon[98694]: slow request 256.371282 seconds old, received at 2023-09-09T23:16:14.146336+0000: client_request(mds.1:1636 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:32.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:32.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:32.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:32 smithi098 ceph-mon[103124]: 569 slow requests, 5 included below; oldest blocked for > 245.871005 secs 2023-09-09T23:20:32.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:32 smithi098 ceph-mon[103124]: slow request 245.687595 seconds old, received at 2023-09-09T23:16:26.000478+0000: peer_request:mds.1:1933 currently dispatched 2023-09-09T23:20:32.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:32 smithi098 ceph-mon[103124]: slow request 245.675225 seconds old, received at 2023-09-09T23:16:26.012848+0000: peer_request:mds.1:1965 currently dispatched 2023-09-09T23:20:32.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:32 smithi098 ceph-mon[103124]: slow request 245.663290 seconds old, received at 2023-09-09T23:16:26.024783+0000: peer_request:mds.1:1997 currently dispatched 2023-09-09T23:20:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:32 smithi098 ceph-mon[103124]: slow request 245.651723 seconds old, received at 2023-09-09T23:16:26.036350+0000: peer_request:mds.1:2029 currently dispatched 2023-09-09T23:20:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:32 smithi098 ceph-mon[103124]: slow request 237.827562 seconds old, received at 2023-09-09T23:16:33.860511+0000: peer_request:mds.1:2061 currently dispatched 2023-09-09T23:20:32.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:32.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:32.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:32 smithi170 ceph-mon[102832]: 569 slow requests, 5 included below; oldest blocked for > 245.871005 secs 2023-09-09T23:20:32.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:32 smithi170 ceph-mon[102832]: slow request 245.687595 seconds old, received at 2023-09-09T23:16:26.000478+0000: peer_request:mds.1:1933 currently dispatched 2023-09-09T23:20:32.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:32 smithi170 ceph-mon[102832]: slow request 245.675225 seconds old, received at 2023-09-09T23:16:26.012848+0000: peer_request:mds.1:1965 currently dispatched 2023-09-09T23:20:32.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:32 smithi170 ceph-mon[102832]: slow request 245.663290 seconds old, received at 2023-09-09T23:16:26.024783+0000: peer_request:mds.1:1997 currently dispatched 2023-09-09T23:20:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:32 smithi170 ceph-mon[102832]: slow request 245.651723 seconds old, received at 2023-09-09T23:16:26.036350+0000: peer_request:mds.1:2029 currently dispatched 2023-09-09T23:20:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:32 smithi170 ceph-mon[102832]: slow request 237.827562 seconds old, received at 2023-09-09T23:16:33.860511+0000: peer_request:mds.1:2061 currently dispatched 2023-09-09T23:20:32.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:32.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:32.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:32 smithi082 ceph-mon[98694]: 569 slow requests, 5 included below; oldest blocked for > 245.871005 secs 2023-09-09T23:20:32.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:32 smithi082 ceph-mon[98694]: slow request 245.687595 seconds old, received at 2023-09-09T23:16:26.000478+0000: peer_request:mds.1:1933 currently dispatched 2023-09-09T23:20:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:32 smithi082 ceph-mon[98694]: slow request 245.675225 seconds old, received at 2023-09-09T23:16:26.012848+0000: peer_request:mds.1:1965 currently dispatched 2023-09-09T23:20:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:32 smithi082 ceph-mon[98694]: slow request 245.663290 seconds old, received at 2023-09-09T23:16:26.024783+0000: peer_request:mds.1:1997 currently dispatched 2023-09-09T23:20:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:32 smithi082 ceph-mon[98694]: slow request 245.651723 seconds old, received at 2023-09-09T23:16:26.036350+0000: peer_request:mds.1:2029 currently dispatched 2023-09-09T23:20:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:32 smithi082 ceph-mon[98694]: slow request 237.827562 seconds old, received at 2023-09-09T23:16:33.860511+0000: peer_request:mds.1:2061 currently dispatched 2023-09-09T23:20:33.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:33 smithi098 ceph-mon[103124]: pgmap v204: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 11 op/s 2023-09-09T23:20:33.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:33 smithi170 ceph-mon[102832]: pgmap v204: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 11 op/s 2023-09-09T23:20:33.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:33 smithi082 ceph-mon[98694]: pgmap v204: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 11 op/s 2023-09-09T23:20:35.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:35 smithi098 ceph-mon[103124]: pgmap v205: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 10 op/s 2023-09-09T23:20:35.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:35 smithi170 ceph-mon[102832]: pgmap v205: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 10 op/s 2023-09-09T23:20:35.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:35 smithi082 ceph-mon[98694]: pgmap v205: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 10 op/s 2023-09-09T23:20:36.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:36.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:36.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:36 smithi098 ceph-mon[103124]: 276 slow requests, 5 included below; oldest blocked for > 262.150793 secs 2023-09-09T23:20:36.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:36 smithi098 ceph-mon[103124]: slow request 261.596205 seconds old, received at 2023-09-09T23:16:14.203154+0000: client_request(mds.1:1668 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:20:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:36 smithi098 ceph-mon[103124]: slow request 261.538995 seconds old, received at 2023-09-09T23:16:14.260364+0000: client_request(mds.1:1700 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:20:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:36 smithi098 ceph-mon[103124]: slow request 261.479911 seconds old, received at 2023-09-09T23:16:14.319448+0000: client_request(mds.1:1732 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:20:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:36 smithi098 ceph-mon[103124]: slow request 261.421927 seconds old, received at 2023-09-09T23:16:14.377432+0000: client_request(mds.1:1764 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:20:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:36 smithi098 ceph-mon[103124]: slow request 261.363539 seconds old, received at 2023-09-09T23:16:14.435820+0000: client_request(mds.1:1796 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:20:36.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:36.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:36.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:36 smithi170 ceph-mon[102832]: 276 slow requests, 5 included below; oldest blocked for > 262.150793 secs 2023-09-09T23:20:36.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:36 smithi170 ceph-mon[102832]: slow request 261.596205 seconds old, received at 2023-09-09T23:16:14.203154+0000: client_request(mds.1:1668 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:20:36.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:36 smithi170 ceph-mon[102832]: slow request 261.538995 seconds old, received at 2023-09-09T23:16:14.260364+0000: client_request(mds.1:1700 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:20:36.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:36 smithi170 ceph-mon[102832]: slow request 261.479911 seconds old, received at 2023-09-09T23:16:14.319448+0000: client_request(mds.1:1732 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:20:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:36 smithi170 ceph-mon[102832]: slow request 261.421927 seconds old, received at 2023-09-09T23:16:14.377432+0000: client_request(mds.1:1764 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:20:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:36 smithi170 ceph-mon[102832]: slow request 261.363539 seconds old, received at 2023-09-09T23:16:14.435820+0000: client_request(mds.1:1796 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:20:36.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:36.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:36.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:36 smithi082 ceph-mon[98694]: 276 slow requests, 5 included below; oldest blocked for > 262.150793 secs 2023-09-09T23:20:36.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:36 smithi082 ceph-mon[98694]: slow request 261.596205 seconds old, received at 2023-09-09T23:16:14.203154+0000: client_request(mds.1:1668 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:20:36.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:36 smithi082 ceph-mon[98694]: slow request 261.538995 seconds old, received at 2023-09-09T23:16:14.260364+0000: client_request(mds.1:1700 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:20:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:36 smithi082 ceph-mon[98694]: slow request 261.479911 seconds old, received at 2023-09-09T23:16:14.319448+0000: client_request(mds.1:1732 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:20:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:36 smithi082 ceph-mon[98694]: slow request 261.421927 seconds old, received at 2023-09-09T23:16:14.377432+0000: client_request(mds.1:1764 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:20:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:36 smithi082 ceph-mon[98694]: slow request 261.363539 seconds old, received at 2023-09-09T23:16:14.435820+0000: client_request(mds.1:1796 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to wrlock, waiting 2023-09-09T23:20:37.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:37 smithi098 ceph-mon[103124]: pgmap v206: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 18 KiB/s rd, 10 KiB/s wr, 14 op/s 2023-09-09T23:20:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:37 smithi098 ceph-mon[103124]: 562 slow requests, 5 included below; oldest blocked for > 250.866999 secs 2023-09-09T23:20:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:37 smithi098 ceph-mon[103124]: slow request 250.866998 seconds old, received at 2023-09-09T23:16:25.821195+0000: peer_request:mds.1:1422 currently dispatched 2023-09-09T23:20:37.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:37 smithi098 ceph-mon[103124]: slow request 250.855144 seconds old, received at 2023-09-09T23:16:25.833050+0000: peer_request:mds.1:1454 currently dispatched 2023-09-09T23:20:37.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:37 smithi098 ceph-mon[103124]: slow request 250.844250 seconds old, received at 2023-09-09T23:16:25.843943+0000: peer_request:mds.1:1486 currently dispatched 2023-09-09T23:20:37.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:37 smithi098 ceph-mon[103124]: slow request 250.832658 seconds old, received at 2023-09-09T23:16:25.855535+0000: peer_request:mds.1:1518 currently dispatched 2023-09-09T23:20:37.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:37 smithi098 ceph-mon[103124]: slow request 250.820846 seconds old, received at 2023-09-09T23:16:25.867347+0000: peer_request:mds.1:1550 currently dispatched 2023-09-09T23:20:37.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:37 smithi170 ceph-mon[102832]: pgmap v206: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 18 KiB/s rd, 10 KiB/s wr, 14 op/s 2023-09-09T23:20:37.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:37 smithi170 ceph-mon[102832]: 562 slow requests, 5 included below; oldest blocked for > 250.866999 secs 2023-09-09T23:20:37.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:37 smithi170 ceph-mon[102832]: slow request 250.866998 seconds old, received at 2023-09-09T23:16:25.821195+0000: peer_request:mds.1:1422 currently dispatched 2023-09-09T23:20:37.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:37 smithi170 ceph-mon[102832]: slow request 250.855144 seconds old, received at 2023-09-09T23:16:25.833050+0000: peer_request:mds.1:1454 currently dispatched 2023-09-09T23:20:37.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:37 smithi170 ceph-mon[102832]: slow request 250.844250 seconds old, received at 2023-09-09T23:16:25.843943+0000: peer_request:mds.1:1486 currently dispatched 2023-09-09T23:20:37.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:37 smithi170 ceph-mon[102832]: slow request 250.832658 seconds old, received at 2023-09-09T23:16:25.855535+0000: peer_request:mds.1:1518 currently dispatched 2023-09-09T23:20:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:37 smithi170 ceph-mon[102832]: slow request 250.820846 seconds old, received at 2023-09-09T23:16:25.867347+0000: peer_request:mds.1:1550 currently dispatched 2023-09-09T23:20:37.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:37 smithi082 ceph-mon[98694]: pgmap v206: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 18 KiB/s rd, 10 KiB/s wr, 14 op/s 2023-09-09T23:20:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:37 smithi082 ceph-mon[98694]: 562 slow requests, 5 included below; oldest blocked for > 250.866999 secs 2023-09-09T23:20:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:37 smithi082 ceph-mon[98694]: slow request 250.866998 seconds old, received at 2023-09-09T23:16:25.821195+0000: peer_request:mds.1:1422 currently dispatched 2023-09-09T23:20:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:37 smithi082 ceph-mon[98694]: slow request 250.855144 seconds old, received at 2023-09-09T23:16:25.833050+0000: peer_request:mds.1:1454 currently dispatched 2023-09-09T23:20:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:37 smithi082 ceph-mon[98694]: slow request 250.844250 seconds old, received at 2023-09-09T23:16:25.843943+0000: peer_request:mds.1:1486 currently dispatched 2023-09-09T23:20:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:37 smithi082 ceph-mon[98694]: slow request 250.832658 seconds old, received at 2023-09-09T23:16:25.855535+0000: peer_request:mds.1:1518 currently dispatched 2023-09-09T23:20:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:37 smithi082 ceph-mon[98694]: slow request 250.820846 seconds old, received at 2023-09-09T23:16:25.867347+0000: peer_request:mds.1:1550 currently dispatched 2023-09-09T23:20:38.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:38.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:38 smithi098 ceph-mon[103124]: pgmap v207: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 10 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2023-09-09T23:20:38.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:38.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:38 smithi170 ceph-mon[102832]: pgmap v207: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 10 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2023-09-09T23:20:38.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:38.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:38 smithi082 ceph-mon[98694]: pgmap v207: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 10 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2023-09-09T23:20:40.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:40.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:40 smithi170 ceph-mon[102832]: pgmap v208: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 15 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2023-09-09T23:20:40.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:40.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:40.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:40 smithi082 ceph-mon[98694]: pgmap v208: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 15 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2023-09-09T23:20:41.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:41.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:41.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:40 smithi098 ceph-mon[103124]: pgmap v208: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 15 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2023-09-09T23:20:41.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:41 smithi170 ceph-mon[102832]: 273 slow requests, 5 included below; oldest blocked for > 267.163161 secs 2023-09-09T23:20:41.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:41 smithi170 ceph-mon[102832]: slow request 266.337204 seconds old, received at 2023-09-09T23:16:14.494004+0000: client_request(mds.1:1828 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:41.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:41 smithi170 ceph-mon[102832]: slow request 266.283115 seconds old, received at 2023-09-09T23:16:14.548094+0000: client_request(mds.1:1860 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:41.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:41 smithi170 ceph-mon[102832]: slow request 266.230193 seconds old, received at 2023-09-09T23:16:14.601016+0000: client_request(mds.1:1892 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:41.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:41 smithi170 ceph-mon[102832]: slow request 266.176081 seconds old, received at 2023-09-09T23:16:14.655128+0000: client_request(mds.1:1924 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:41 smithi170 ceph-mon[102832]: slow request 266.121151 seconds old, received at 2023-09-09T23:16:14.710057+0000: client_request(mds.1:1956 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:41.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:41 smithi082 ceph-mon[98694]: 273 slow requests, 5 included below; oldest blocked for > 267.163161 secs 2023-09-09T23:20:41.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:41 smithi082 ceph-mon[98694]: slow request 266.337204 seconds old, received at 2023-09-09T23:16:14.494004+0000: client_request(mds.1:1828 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:41.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:41 smithi082 ceph-mon[98694]: slow request 266.283115 seconds old, received at 2023-09-09T23:16:14.548094+0000: client_request(mds.1:1860 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:41.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:41 smithi082 ceph-mon[98694]: slow request 266.230193 seconds old, received at 2023-09-09T23:16:14.601016+0000: client_request(mds.1:1892 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:41.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:41 smithi082 ceph-mon[98694]: slow request 266.176081 seconds old, received at 2023-09-09T23:16:14.655128+0000: client_request(mds.1:1924 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:41.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:41 smithi082 ceph-mon[98694]: slow request 266.121151 seconds old, received at 2023-09-09T23:16:14.710057+0000: client_request(mds.1:1956 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:42.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:41 smithi098 ceph-mon[103124]: 273 slow requests, 5 included below; oldest blocked for > 267.163161 secs 2023-09-09T23:20:42.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:41 smithi098 ceph-mon[103124]: slow request 266.337204 seconds old, received at 2023-09-09T23:16:14.494004+0000: client_request(mds.1:1828 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:42.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:41 smithi098 ceph-mon[103124]: slow request 266.283115 seconds old, received at 2023-09-09T23:16:14.548094+0000: client_request(mds.1:1860 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:42.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:41 smithi098 ceph-mon[103124]: slow request 266.230193 seconds old, received at 2023-09-09T23:16:14.601016+0000: client_request(mds.1:1892 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:42.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:41 smithi098 ceph-mon[103124]: slow request 266.176081 seconds old, received at 2023-09-09T23:16:14.655128+0000: client_request(mds.1:1924 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:41 smithi098 ceph-mon[103124]: slow request 266.121151 seconds old, received at 2023-09-09T23:16:14.710057+0000: client_request(mds.1:1956 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:42.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:42 smithi170 ceph-mon[102832]: 550 slow requests, 5 included below; oldest blocked for > 255.862563 secs 2023-09-09T23:20:42.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:42 smithi170 ceph-mon[102832]: slow request 255.809720 seconds old, received at 2023-09-09T23:16:25.878534+0000: peer_request:mds.1:1582 currently dispatched 2023-09-09T23:20:42.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:42 smithi170 ceph-mon[102832]: slow request 255.798574 seconds old, received at 2023-09-09T23:16:25.889680+0000: peer_request:mds.1:1614 currently dispatched 2023-09-09T23:20:42.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:42 smithi170 ceph-mon[102832]: slow request 255.787780 seconds old, received at 2023-09-09T23:16:25.900475+0000: peer_request:mds.1:1646 currently dispatched 2023-09-09T23:20:42.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:42 smithi170 ceph-mon[102832]: slow request 255.776019 seconds old, received at 2023-09-09T23:16:25.912235+0000: peer_request:mds.1:1678 currently dispatched 2023-09-09T23:20:42.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:42 smithi170 ceph-mon[102832]: slow request 255.764905 seconds old, received at 2023-09-09T23:16:25.923350+0000: peer_request:mds.1:1710 currently dispatched 2023-09-09T23:20:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:42 smithi170 ceph-mon[102832]: pgmap v209: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2023-09-09T23:20:42.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:42 smithi082 ceph-mon[98694]: 550 slow requests, 5 included below; oldest blocked for > 255.862563 secs 2023-09-09T23:20:42.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:42 smithi082 ceph-mon[98694]: slow request 255.809720 seconds old, received at 2023-09-09T23:16:25.878534+0000: peer_request:mds.1:1582 currently dispatched 2023-09-09T23:20:42.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:42 smithi082 ceph-mon[98694]: slow request 255.798574 seconds old, received at 2023-09-09T23:16:25.889680+0000: peer_request:mds.1:1614 currently dispatched 2023-09-09T23:20:42.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:42 smithi082 ceph-mon[98694]: slow request 255.787780 seconds old, received at 2023-09-09T23:16:25.900475+0000: peer_request:mds.1:1646 currently dispatched 2023-09-09T23:20:42.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:42 smithi082 ceph-mon[98694]: slow request 255.776019 seconds old, received at 2023-09-09T23:16:25.912235+0000: peer_request:mds.1:1678 currently dispatched 2023-09-09T23:20:42.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:42 smithi082 ceph-mon[98694]: slow request 255.764905 seconds old, received at 2023-09-09T23:16:25.923350+0000: peer_request:mds.1:1710 currently dispatched 2023-09-09T23:20:42.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:42 smithi082 ceph-mon[98694]: pgmap v209: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2023-09-09T23:20:43.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:42 smithi098 ceph-mon[103124]: 550 slow requests, 5 included below; oldest blocked for > 255.862563 secs 2023-09-09T23:20:43.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:42 smithi098 ceph-mon[103124]: slow request 255.809720 seconds old, received at 2023-09-09T23:16:25.878534+0000: peer_request:mds.1:1582 currently dispatched 2023-09-09T23:20:43.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:42 smithi098 ceph-mon[103124]: slow request 255.798574 seconds old, received at 2023-09-09T23:16:25.889680+0000: peer_request:mds.1:1614 currently dispatched 2023-09-09T23:20:43.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:42 smithi098 ceph-mon[103124]: slow request 255.787780 seconds old, received at 2023-09-09T23:16:25.900475+0000: peer_request:mds.1:1646 currently dispatched 2023-09-09T23:20:43.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:42 smithi098 ceph-mon[103124]: slow request 255.776019 seconds old, received at 2023-09-09T23:16:25.912235+0000: peer_request:mds.1:1678 currently dispatched 2023-09-09T23:20:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:42 smithi098 ceph-mon[103124]: slow request 255.764905 seconds old, received at 2023-09-09T23:16:25.923350+0000: peer_request:mds.1:1710 currently dispatched 2023-09-09T23:20:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:42 smithi098 ceph-mon[103124]: pgmap v209: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2023-09-09T23:20:43.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:20:43.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:43.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:20:43.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:44.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:20:44.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:44.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:44.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:44.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:44 smithi170 ceph-mon[102832]: pgmap v210: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2023-09-09T23:20:44.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:44.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:44.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:44 smithi082 ceph-mon[98694]: pgmap v210: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2023-09-09T23:20:45.001 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:45.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:45.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:44 smithi098 ceph-mon[103124]: pgmap v210: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2023-09-09T23:20:46.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:46 smithi170 ceph-mon[102832]: 271 slow requests, 5 included below; oldest blocked for > 272.224480 secs 2023-09-09T23:20:46.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:46 smithi170 ceph-mon[102832]: slow request 271.146762 seconds old, received at 2023-09-09T23:16:14.768755+0000: client_request(mds.1:1988 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:46.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:46 smithi170 ceph-mon[102832]: slow request 271.087502 seconds old, received at 2023-09-09T23:16:14.828015+0000: client_request(mds.1:2020 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:46.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:46 smithi170 ceph-mon[102832]: slow request 271.033843 seconds old, received at 2023-09-09T23:16:14.881674+0000: client_request(mds.1:2052 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:46.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:46 smithi170 ceph-mon[102832]: slow request 272.220615 seconds old, received at 2023-09-09T23:16:13.694901+0000: client_request(mds.1:1445 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:46 smithi170 ceph-mon[102832]: slow request 272.056161 seconds old, received at 2023-09-09T23:16:13.859356+0000: client_request(mds.1:1477 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:46.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:46 smithi082 ceph-mon[98694]: 271 slow requests, 5 included below; oldest blocked for > 272.224480 secs 2023-09-09T23:20:46.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:46 smithi082 ceph-mon[98694]: slow request 271.146762 seconds old, received at 2023-09-09T23:16:14.768755+0000: client_request(mds.1:1988 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:46.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:46 smithi082 ceph-mon[98694]: slow request 271.087502 seconds old, received at 2023-09-09T23:16:14.828015+0000: client_request(mds.1:2020 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:46.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:46 smithi082 ceph-mon[98694]: slow request 271.033843 seconds old, received at 2023-09-09T23:16:14.881674+0000: client_request(mds.1:2052 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:46.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:46 smithi082 ceph-mon[98694]: slow request 272.220615 seconds old, received at 2023-09-09T23:16:13.694901+0000: client_request(mds.1:1445 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:46.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:46 smithi082 ceph-mon[98694]: slow request 272.056161 seconds old, received at 2023-09-09T23:16:13.859356+0000: client_request(mds.1:1477 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:46.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:46 smithi098 ceph-mon[103124]: 271 slow requests, 5 included below; oldest blocked for > 272.224480 secs 2023-09-09T23:20:46.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:46 smithi098 ceph-mon[103124]: slow request 271.146762 seconds old, received at 2023-09-09T23:16:14.768755+0000: client_request(mds.1:1988 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:46.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:46 smithi098 ceph-mon[103124]: slow request 271.087502 seconds old, received at 2023-09-09T23:16:14.828015+0000: client_request(mds.1:2020 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:46.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:46 smithi098 ceph-mon[103124]: slow request 271.033843 seconds old, received at 2023-09-09T23:16:14.881674+0000: client_request(mds.1:2052 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:46.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:46 smithi098 ceph-mon[103124]: slow request 272.220615 seconds old, received at 2023-09-09T23:16:13.694901+0000: client_request(mds.1:1445 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:46 smithi098 ceph-mon[103124]: slow request 272.056161 seconds old, received at 2023-09-09T23:16:13.859356+0000: client_request(mds.1:1477 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:47.425 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:47 smithi170 ceph-mon[102832]: pgmap v211: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2023-09-09T23:20:47.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:47 smithi170 ceph-mon[102832]: 539 slow requests, 5 included below; oldest blocked for > 260.858360 secs 2023-09-09T23:20:47.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:47 smithi170 ceph-mon[102832]: slow request 260.752471 seconds old, received at 2023-09-09T23:16:25.935817+0000: peer_request:mds.1:1742 currently dispatched 2023-09-09T23:20:47.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:47 smithi170 ceph-mon[102832]: slow request 260.741192 seconds old, received at 2023-09-09T23:16:25.947095+0000: peer_request:mds.1:1774 currently dispatched 2023-09-09T23:20:47.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:47 smithi170 ceph-mon[102832]: slow request 260.730878 seconds old, received at 2023-09-09T23:16:25.957409+0000: peer_request:mds.1:1806 currently dispatched 2023-09-09T23:20:47.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:47 smithi170 ceph-mon[102832]: slow request 260.719341 seconds old, received at 2023-09-09T23:16:25.968947+0000: peer_request:mds.1:1838 currently dispatched 2023-09-09T23:20:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:47 smithi170 ceph-mon[102832]: slow request 260.708214 seconds old, received at 2023-09-09T23:16:25.980073+0000: peer_request:mds.1:1870 currently dispatched 2023-09-09T23:20:47.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:47 smithi082 ceph-mon[98694]: pgmap v211: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2023-09-09T23:20:47.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:47 smithi082 ceph-mon[98694]: 539 slow requests, 5 included below; oldest blocked for > 260.858360 secs 2023-09-09T23:20:47.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:47 smithi082 ceph-mon[98694]: slow request 260.752471 seconds old, received at 2023-09-09T23:16:25.935817+0000: peer_request:mds.1:1742 currently dispatched 2023-09-09T23:20:47.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:47 smithi082 ceph-mon[98694]: slow request 260.741192 seconds old, received at 2023-09-09T23:16:25.947095+0000: peer_request:mds.1:1774 currently dispatched 2023-09-09T23:20:47.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:47 smithi082 ceph-mon[98694]: slow request 260.730878 seconds old, received at 2023-09-09T23:16:25.957409+0000: peer_request:mds.1:1806 currently dispatched 2023-09-09T23:20:47.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:47 smithi082 ceph-mon[98694]: slow request 260.719341 seconds old, received at 2023-09-09T23:16:25.968947+0000: peer_request:mds.1:1838 currently dispatched 2023-09-09T23:20:47.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:47 smithi082 ceph-mon[98694]: slow request 260.708214 seconds old, received at 2023-09-09T23:16:25.980073+0000: peer_request:mds.1:1870 currently dispatched 2023-09-09T23:20:47.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:47 smithi098 ceph-mon[103124]: pgmap v211: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2023-09-09T23:20:47.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:47 smithi098 ceph-mon[103124]: 539 slow requests, 5 included below; oldest blocked for > 260.858360 secs 2023-09-09T23:20:47.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:47 smithi098 ceph-mon[103124]: slow request 260.752471 seconds old, received at 2023-09-09T23:16:25.935817+0000: peer_request:mds.1:1742 currently dispatched 2023-09-09T23:20:47.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:47 smithi098 ceph-mon[103124]: slow request 260.741192 seconds old, received at 2023-09-09T23:16:25.947095+0000: peer_request:mds.1:1774 currently dispatched 2023-09-09T23:20:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:47 smithi098 ceph-mon[103124]: slow request 260.730878 seconds old, received at 2023-09-09T23:16:25.957409+0000: peer_request:mds.1:1806 currently dispatched 2023-09-09T23:20:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:47 smithi098 ceph-mon[103124]: slow request 260.719341 seconds old, received at 2023-09-09T23:16:25.968947+0000: peer_request:mds.1:1838 currently dispatched 2023-09-09T23:20:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:47 smithi098 ceph-mon[103124]: slow request 260.708214 seconds old, received at 2023-09-09T23:16:25.980073+0000: peer_request:mds.1:1870 currently dispatched 2023-09-09T23:20:49.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:49 smithi170 ceph-mon[102832]: pgmap v212: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2023-09-09T23:20:49.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:49 smithi082 ceph-mon[98694]: pgmap v212: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2023-09-09T23:20:49.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:49 smithi098 ceph-mon[103124]: pgmap v212: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2023-09-09T23:20:51.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:51 smithi170 ceph-mon[102832]: pgmap v213: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2023-09-09T23:20:51.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:20:51.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:51.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:51 smithi082 ceph-mon[98694]: pgmap v213: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2023-09-09T23:20:51.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:20:51.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:51 smithi098 ceph-mon[103124]: pgmap v213: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2023-09-09T23:20:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:20:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:52.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:52 smithi170 ceph-mon[102832]: 259 slow requests, 5 included below; oldest blocked for > 277.195844 secs 2023-09-09T23:20:52.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:52 smithi170 ceph-mon[102832]: slow request 277.092786 seconds old, received at 2023-09-09T23:16:13.919254+0000: client_request(mds.1:1509 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:52.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:52 smithi170 ceph-mon[102832]: slow request 277.037740 seconds old, received at 2023-09-09T23:16:13.974301+0000: client_request(mds.1:1541 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:52.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:52 smithi170 ceph-mon[102832]: slow request 276.978250 seconds old, received at 2023-09-09T23:16:14.033791+0000: client_request(mds.1:1573 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:52 smithi170 ceph-mon[102832]: slow request 276.921358 seconds old, received at 2023-09-09T23:16:14.090683+0000: client_request(mds.1:1605 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:52 smithi170 ceph-mon[102832]: slow request 276.864055 seconds old, received at 2023-09-09T23:16:14.147985+0000: client_request(mds.1:1637 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:52 smithi170 ceph-mon[102832]: 531 slow requests, 5 included below; oldest blocked for > 265.854121 secs 2023-09-09T23:20:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:52 smithi170 ceph-mon[102832]: slow request 265.693916 seconds old, received at 2023-09-09T23:16:25.994439+0000: peer_request:mds.1:1902 currently dispatched 2023-09-09T23:20:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:52 smithi170 ceph-mon[102832]: slow request 265.687498 seconds old, received at 2023-09-09T23:16:26.000857+0000: peer_request:mds.1:1934 currently dispatched 2023-09-09T23:20:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:52 smithi170 ceph-mon[102832]: slow request 265.675220 seconds old, received at 2023-09-09T23:16:26.013136+0000: peer_request:mds.1:1966 currently dispatched 2023-09-09T23:20:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:52 smithi170 ceph-mon[102832]: slow request 265.663164 seconds old, received at 2023-09-09T23:16:26.025191+0000: peer_request:mds.1:1998 currently dispatched 2023-09-09T23:20:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:52 smithi170 ceph-mon[102832]: slow request 265.651703 seconds old, received at 2023-09-09T23:16:26.036652+0000: peer_request:mds.1:2030 currently dispatched 2023-09-09T23:20:52.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:52 smithi082 ceph-mon[98694]: 259 slow requests, 5 included below; oldest blocked for > 277.195844 secs 2023-09-09T23:20:52.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:52 smithi082 ceph-mon[98694]: slow request 277.092786 seconds old, received at 2023-09-09T23:16:13.919254+0000: client_request(mds.1:1509 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:52.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:52 smithi082 ceph-mon[98694]: slow request 277.037740 seconds old, received at 2023-09-09T23:16:13.974301+0000: client_request(mds.1:1541 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:52.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:52 smithi082 ceph-mon[98694]: slow request 276.978250 seconds old, received at 2023-09-09T23:16:14.033791+0000: client_request(mds.1:1573 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:52.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:52 smithi082 ceph-mon[98694]: slow request 276.921358 seconds old, received at 2023-09-09T23:16:14.090683+0000: client_request(mds.1:1605 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:52.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:52 smithi082 ceph-mon[98694]: slow request 276.864055 seconds old, received at 2023-09-09T23:16:14.147985+0000: client_request(mds.1:1637 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:52.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:52 smithi082 ceph-mon[98694]: 531 slow requests, 5 included below; oldest blocked for > 265.854121 secs 2023-09-09T23:20:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:52 smithi082 ceph-mon[98694]: slow request 265.693916 seconds old, received at 2023-09-09T23:16:25.994439+0000: peer_request:mds.1:1902 currently dispatched 2023-09-09T23:20:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:52 smithi082 ceph-mon[98694]: slow request 265.687498 seconds old, received at 2023-09-09T23:16:26.000857+0000: peer_request:mds.1:1934 currently dispatched 2023-09-09T23:20:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:52 smithi082 ceph-mon[98694]: slow request 265.675220 seconds old, received at 2023-09-09T23:16:26.013136+0000: peer_request:mds.1:1966 currently dispatched 2023-09-09T23:20:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:52 smithi082 ceph-mon[98694]: slow request 265.663164 seconds old, received at 2023-09-09T23:16:26.025191+0000: peer_request:mds.1:1998 currently dispatched 2023-09-09T23:20:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:52 smithi082 ceph-mon[98694]: slow request 265.651703 seconds old, received at 2023-09-09T23:16:26.036652+0000: peer_request:mds.1:2030 currently dispatched 2023-09-09T23:20:52.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:52 smithi098 ceph-mon[103124]: 259 slow requests, 5 included below; oldest blocked for > 277.195844 secs 2023-09-09T23:20:52.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:52 smithi098 ceph-mon[103124]: slow request 277.092786 seconds old, received at 2023-09-09T23:16:13.919254+0000: client_request(mds.1:1509 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:52.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:52 smithi098 ceph-mon[103124]: slow request 277.037740 seconds old, received at 2023-09-09T23:16:13.974301+0000: client_request(mds.1:1541 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:52 smithi098 ceph-mon[103124]: slow request 276.978250 seconds old, received at 2023-09-09T23:16:14.033791+0000: client_request(mds.1:1573 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:52 smithi098 ceph-mon[103124]: slow request 276.921358 seconds old, received at 2023-09-09T23:16:14.090683+0000: client_request(mds.1:1605 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:52 smithi098 ceph-mon[103124]: slow request 276.864055 seconds old, received at 2023-09-09T23:16:14.147985+0000: client_request(mds.1:1637 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:52 smithi098 ceph-mon[103124]: 531 slow requests, 5 included below; oldest blocked for > 265.854121 secs 2023-09-09T23:20:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:52 smithi098 ceph-mon[103124]: slow request 265.693916 seconds old, received at 2023-09-09T23:16:25.994439+0000: peer_request:mds.1:1902 currently dispatched 2023-09-09T23:20:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:52 smithi098 ceph-mon[103124]: slow request 265.687498 seconds old, received at 2023-09-09T23:16:26.000857+0000: peer_request:mds.1:1934 currently dispatched 2023-09-09T23:20:52.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:52 smithi098 ceph-mon[103124]: slow request 265.675220 seconds old, received at 2023-09-09T23:16:26.013136+0000: peer_request:mds.1:1966 currently dispatched 2023-09-09T23:20:52.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:52 smithi098 ceph-mon[103124]: slow request 265.663164 seconds old, received at 2023-09-09T23:16:26.025191+0000: peer_request:mds.1:1998 currently dispatched 2023-09-09T23:20:52.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:52 smithi098 ceph-mon[103124]: slow request 265.651703 seconds old, received at 2023-09-09T23:16:26.036652+0000: peer_request:mds.1:2030 currently dispatched 2023-09-09T23:20:53.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:53 smithi170 ceph-mon[102832]: pgmap v214: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2023-09-09T23:20:53.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:53 smithi082 ceph-mon[98694]: pgmap v214: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2023-09-09T23:20:53.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:53 smithi098 ceph-mon[103124]: pgmap v214: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2023-09-09T23:20:55.546 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:20:55.685 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.687+0000 7f058d8d9700 1 -- 172.21.15.82:0/2839677728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05880ffef0 msgr2=0x7f0588100370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:20:55.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.687+0000 7f058d8d9700 1 --2- 172.21.15.82:0/2839677728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05880ffef0 0x7f0588100370 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f057c0099c0 tx=0x7f057c009cd0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.687+0000 7f058d8d9700 1 -- 172.21.15.82:0/2839677728 shutdown_connections 2023-09-09T23:20:55.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.687+0000 7f058d8d9700 1 --2- 172.21.15.82:0/2839677728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0588105560 0x7f0588105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.687+0000 7f058d8d9700 1 --2- 172.21.15.82:0/2839677728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05881008b0 0x7f058810c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.687+0000 7f058d8d9700 1 --2- 172.21.15.82:0/2839677728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05880ffef0 0x7f0588100370 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.687+0000 7f058d8d9700 1 -- 172.21.15.82:0/2839677728 >> 172.21.15.82:0/2839677728 conn(0x7f05880747a0 msgr2=0x7f0588074bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:20:55.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.687+0000 7f058d8d9700 1 -- 172.21.15.82:0/2839677728 shutdown_connections 2023-09-09T23:20:55.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.687+0000 7f058d8d9700 1 -- 172.21.15.82:0/2839677728 wait complete. 2023-09-09T23:20:55.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.688+0000 7f058d8d9700 1 Processor -- start 2023-09-09T23:20:55.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.688+0000 7f058d8d9700 1 -- start start 2023-09-09T23:20:55.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.689+0000 7f058d8d9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05880ffef0 0x7f0588195570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:55.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.689+0000 7f058d8d9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05881008b0 0x7f0588195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:55.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.689+0000 7f058d8d9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0588105560 0x7f0588199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:55.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.689+0000 7f058d8d9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058819a550 con 0x7f0588105560 2023-09-09T23:20:55.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.689+0000 7f058d8d9700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058819a6c0 con 0x7f05881008b0 2023-09-09T23:20:55.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.689+0000 7f058d8d9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058819a880 con 0x7f05880ffef0 2023-09-09T23:20:55.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.689+0000 7f05877fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0588105560 0x7f0588199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:55.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.689+0000 7f0586ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05880ffef0 0x7f0588195570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:55.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.689+0000 7f0586ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05880ffef0 0x7f0588195570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:56504/0 (socket says 172.21.15.82:56504) 2023-09-09T23:20:55.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.689+0000 7f05877fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0588105560 0x7f0588199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57994/0 (socket says 172.21.15.82:57994) 2023-09-09T23:20:55.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.689+0000 7f0586ffd700 1 -- 172.21.15.82:0/916944776 learned_addr learned my addr 172.21.15.82:0/916944776 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:20:55.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.689+0000 7f05867fc700 1 --2- 172.21.15.82:0/916944776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05881008b0 0x7f0588195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:55.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.689+0000 7f0586ffd700 1 -- 172.21.15.82:0/916944776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05881008b0 msgr2=0x7f0588195ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:20:55.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.689+0000 7f0586ffd700 1 --2- 172.21.15.82:0/916944776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05881008b0 0x7f0588195ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.689+0000 7f0586ffd700 1 -- 172.21.15.82:0/916944776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0588105560 msgr2=0x7f0588199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:20:55.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.689+0000 7f0586ffd700 1 --2- 172.21.15.82:0/916944776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0588105560 0x7f0588199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.689+0000 7f0586ffd700 1 -- 172.21.15.82:0/916944776 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057c0095f0 con 0x7f05880ffef0 2023-09-09T23:20:55.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.689+0000 7f05877fe700 1 --2- 172.21.15.82:0/916944776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0588105560 0x7f0588199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:20:55.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.689+0000 7f0586ffd700 1 --2- 172.21.15.82:0/916944776 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05880ffef0 0x7f0588195570 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f057800e9a0 tx=0x7f057800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:20:55.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.689+0000 7f058c8d7700 1 -- 172.21.15.82:0/916944776 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0578009790 con 0x7f05880ffef0 2023-09-09T23:20:55.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.690+0000 7f058d8d9700 1 -- 172.21.15.82:0/916944776 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f058819ab60 con 0x7f05880ffef0 2023-09-09T23:20:55.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.690+0000 7f058c8d7700 1 -- 172.21.15.82:0/916944776 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0578010430 con 0x7f05880ffef0 2023-09-09T23:20:55.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.690+0000 7f058c8d7700 1 -- 172.21.15.82:0/916944776 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0578018680 con 0x7f05880ffef0 2023-09-09T23:20:55.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.690+0000 7f058d8d9700 1 -- 172.21.15.82:0/916944776 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05881a23a0 con 0x7f05880ffef0 2023-09-09T23:20:55.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.692+0000 7f058c8d7700 1 -- 172.21.15.82:0/916944776 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f05780105a0 con 0x7f05880ffef0 2023-09-09T23:20:55.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.692+0000 7f058c8d7700 1 --2- 172.21.15.82:0/916944776 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f05700a07b0 0x7f05700a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:55.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.692+0000 7f05867fc700 1 --2- 172.21.15.82:0/916944776 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f05700a07b0 0x7f05700a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:55.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.692+0000 7f058c8d7700 1 -- 172.21.15.82:0/916944776 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f0578014070 con 0x7f05880ffef0 2023-09-09T23:20:55.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.693+0000 7f05867fc700 1 --2- 172.21.15.82:0/916944776 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f05700a07b0 0x7f05700a2c70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f057c009fd0 tx=0x7f057c005fb0 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:20:55.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.693+0000 7f0599a9fb80 1 Processor -- start 2023-09-09T23:20:55.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.693+0000 7f0599a9fb80 1 -- start start 2023-09-09T23:20:55.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.693+0000 7f0599a9fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b666885710 0x55b666885af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:55.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.693+0000 7f0599a9fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b666886030 0x55b666891b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:55.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.693+0000 7f0599a9fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b666892050 0x55b666894440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:55.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.693+0000 7f0599a9fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6667de730 con 0x55b666892050 2023-09-09T23:20:55.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.693+0000 7f0599a9fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6667dc050 con 0x55b666885710 2023-09-09T23:20:55.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.693+0000 7f0599a9fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6667d7940 con 0x55b666886030 2023-09-09T23:20:55.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.693+0000 7f05877fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b666892050 0x55b666894440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:55.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.693+0000 7f0586ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b666885710 0x55b666885af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:55.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.693+0000 7f05877fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b666892050 0x55b666894440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58002/0 (socket says 172.21.15.82:58002) 2023-09-09T23:20:55.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.693+0000 7f05877fe700 1 -- 172.21.15.82:0/200326040 learned_addr learned my addr 172.21.15.82:0/200326040 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:20:55.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.693+0000 7f05867fc700 1 --2- 172.21.15.82:0/200326040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b666886030 0x55b666891b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:55.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.693+0000 7f0586ffd700 1 -- 172.21.15.82:0/200326040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b666886030 msgr2=0x55b666891b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:20:55.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.693+0000 7f0586ffd700 1 --2- 172.21.15.82:0/200326040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b666886030 0x55b666891b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.694+0000 7f0586ffd700 1 -- 172.21.15.82:0/200326040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b666892050 msgr2=0x55b666894440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:20:55.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.694+0000 7f0586ffd700 1 --2- 172.21.15.82:0/200326040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b666892050 0x55b666894440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.694+0000 7f0586ffd700 1 -- 172.21.15.82:0/200326040 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b666603de0 con 0x55b666885710 2023-09-09T23:20:55.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.694+0000 7f05877fe700 1 --2- 172.21.15.82:0/200326040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b666892050 0x55b666894440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:20:55.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.694+0000 7f0586ffd700 1 --2- 172.21.15.82:0/200326040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b666885710 0x55b666885af0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f05780c9800 tx=0x7f0578098f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e409e444fab2dfc server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:20:55.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.694+0000 7f056dffb700 1 -- 172.21.15.82:0/200326040 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057809d390 con 0x55b666885710 2023-09-09T23:20:55.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.694+0000 7f056dffb700 1 -- 172.21.15.82:0/200326040 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05780a15d0 con 0x55b666885710 2023-09-09T23:20:55.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.694+0000 7f056dffb700 1 -- 172.21.15.82:0/200326040 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0578099a20 con 0x55b666885710 2023-09-09T23:20:55.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.695+0000 7f0599a9fb80 1 -- 172.21.15.82:0/200326040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b666885710 msgr2=0x55b666885af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:20:55.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.695+0000 7f0599a9fb80 1 --2- 172.21.15.82:0/200326040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b666885710 0x55b666885af0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f05780c9800 tx=0x7f0578098f40 comp rx=0 tx=0).stop 2023-09-09T23:20:55.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.695+0000 7f0599a9fb80 1 -- 172.21.15.82:0/200326040 shutdown_connections 2023-09-09T23:20:55.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.695+0000 7f0599a9fb80 1 --2- 172.21.15.82:0/200326040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b666885710 0x55b666885af0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.695+0000 7f0599a9fb80 1 --2- 172.21.15.82:0/200326040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b666892050 0x55b666894440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.709 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.695+0000 7f0599a9fb80 1 --2- 172.21.15.82:0/200326040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b666886030 0x55b666891b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.709 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.695+0000 7f0599a9fb80 1 -- 172.21.15.82:0/200326040 >> 172.21.15.82:0/200326040 conn(0x55b6667fa090 msgr2=0x55b666880bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:20:55.709 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.695+0000 7f0599a9fb80 1 -- 172.21.15.82:0/200326040 shutdown_connections 2023-09-09T23:20:55.709 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.695+0000 7f0599a9fb80 1 -- 172.21.15.82:0/200326040 wait complete. 2023-09-09T23:20:55.709 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.695+0000 7f0599a9fb80 1 -- start start 2023-09-09T23:20:55.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.695+0000 7f0599a9fb80 1 Processor -- start 2023-09-09T23:20:55.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.696+0000 7f0599a9fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b666885710 0x55b666891b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:55.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.696+0000 7f0599a9fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b666886030 0x55b66688ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:55.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.696+0000 7f0599a9fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b666892050 0x55b66688f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:55.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.696+0000 7f0599a9fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b666775b70 con 0x55b666886030 2023-09-09T23:20:55.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.696+0000 7f0599a9fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b666775870 con 0x55b666885710 2023-09-09T23:20:55.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.696+0000 7f0599a9fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b666659920 con 0x55b666892050 2023-09-09T23:20:55.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.696+0000 7f0599a9fb80 10 client.0 _ll_register_callbacks cb 0x55b6666bfa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:20:55.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.696+0000 7f05867fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b666886030 0x55b66688ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:55.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.696+0000 7f0586ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b666885710 0x55b666891b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:55.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.696+0000 7f05867fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b666886030 0x55b66688ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58006/0 (socket says 172.21.15.82:58006) 2023-09-09T23:20:55.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.696+0000 7f05867fc700 1 -- 172.21.15.82:0/376638421 learned_addr learned my addr 172.21.15.82:0/376638421 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:20:55.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.696+0000 7f05877fe700 1 --2- 172.21.15.82:0/376638421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b666892050 0x55b66688f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:55.713 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.696+0000 7f05867fc700 1 -- 172.21.15.82:0/376638421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b666892050 msgr2=0x55b66688f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:20:55.713 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.696+0000 7f05867fc700 1 --2- 172.21.15.82:0/376638421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b666892050 0x55b66688f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.713 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.696+0000 7f05867fc700 1 -- 172.21.15.82:0/376638421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b666885710 msgr2=0x55b666891b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:20:55.713 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.696+0000 7f05867fc700 1 --2- 172.21.15.82:0/376638421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b666885710 0x55b666891b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.713 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.696+0000 7f05867fc700 1 -- 172.21.15.82:0/376638421 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0580003b90 con 0x55b666886030 2023-09-09T23:20:55.714 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.696+0000 7f05867fc700 1 --2- 172.21.15.82:0/376638421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b666886030 0x55b66688ed80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f057c0099c0 tx=0x7f057c009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fedc776cfcdb2411 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:20:55.714 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.696+0000 7f0566ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:20:55.714 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.696+0000 7f0566ffd700 1 -- 172.21.15.82:0/376638421 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c02c040 con 0x55b666886030 2023-09-09T23:20:55.714 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.696+0000 7f0566ffd700 1 -- 172.21.15.82:0/376638421 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f057c004460 con 0x55b666886030 2023-09-09T23:20:55.714 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.696+0000 7f0599a9fb80 1 -- 172.21.15.82:0/376638421 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b66672f4f0 con 0x55b666886030 2023-09-09T23:20:55.715 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.697+0000 7f0566ffd700 1 -- 172.21.15.82:0/376638421 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f057c004620 con 0x55b666886030 2023-09-09T23:20:55.715 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.697+0000 7f0599a9fb80 10 client.16173 fetch_fsmap learned FSMap version 76 2023-09-09T23:20:55.715 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.697+0000 7f0599a9fb80 1 -- 172.21.15.82:0/376638421 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x7f05881a23a0 con 0x55b666886030 2023-09-09T23:20:55.715 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.697+0000 7f0566ffd700 1 -- 172.21.15.82:0/376638421 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f057c03e020 con 0x55b666886030 2023-09-09T23:20:55.715 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.697+0000 7f0566ffd700 1 -- 172.21.15.82:0/376638421 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 76) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f057c00f460 con 0x55b666886030 2023-09-09T23:20:55.716 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.697+0000 7f0599a9fb80 10 client.16173 fetch_fsmap finished waiting for FSMap version 76 2023-09-09T23:20:55.716 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.697+0000 7f0599a9fb80 10 client.16173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:20:55.716 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.697+0000 7f0599a9fb80 20 client.16173 populate_metadata read hostname 'smithi082' 2023-09-09T23:20:55.716 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.697+0000 7f0599a9fb80 1 --2- 172.21.15.82:0/376638421 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55b6668a1c20 0x55b6668a4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:55.716 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.697+0000 7f0599a9fb80 4 client.16173 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:20:55.717 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.697+0000 7f0599a9fb80 1 -- 172.21.15.82:0/376638421 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b6667ec8a0 con 0x55b6668a1c20 2023-09-09T23:20:55.717 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.697+0000 7f0586ffd700 1 --2- 172.21.15.82:0/376638421 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55b6668a1c20 0x55b6668a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:55.717 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.698+0000 7f0566ffd700 10 client.16173 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:20:55.717 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.698+0000 7f0586ffd700 1 --2- 172.21.15.82:0/376638421 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55b6668a1c20 0x55b6668a4040 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ff001c10c47ef71 server_cookie=53ce1a70311c9295 in_seq=0 out_seq=0 2023-09-09T23:20:55.717 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.698+0000 7f0566ffd700 1 -- 172.21.15.82:0/376638421 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b6667ec8a0 con 0x55b6668a1c20 2023-09-09T23:20:55.718 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.698+0000 7f0566ffd700 10 client.16173 handle_command_reply: tid=0 2023-09-09T23:20:55.718 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.698+0000 7f0599a9fb80 1 client.16173 shutdown 2023-09-09T23:20:55.718 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.699+0000 7f0599a9fb80 1 -- 172.21.15.82:0/376638421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b666886030 msgr2=0x55b66688ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:20:55.718 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.699+0000 7f0599a9fb80 1 --2- 172.21.15.82:0/376638421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b666886030 0x55b66688ed80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f057c0099c0 tx=0x7f057c009cd0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.719 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.699+0000 7f0599a9fb80 1 -- 172.21.15.82:0/376638421 shutdown_connections 2023-09-09T23:20:55.719 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.699+0000 7f0599a9fb80 1 --2- 172.21.15.82:0/376638421 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55b6668a1c20 0x55b6668a4040 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.719 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.699+0000 7f0599a9fb80 1 --2- 172.21.15.82:0/376638421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b666885710 0x55b666891b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.719 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.699+0000 7f0599a9fb80 1 --2- 172.21.15.82:0/376638421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b666892050 0x55b66688f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.719 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.699+0000 7f0599a9fb80 1 --2- 172.21.15.82:0/376638421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b666886030 0x55b66688ed80 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.720 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.699+0000 7f0566ffd700 0 client.16173 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:20:55.720 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.699+0000 7f0599a9fb80 1 -- 172.21.15.82:0/376638421 >> 172.21.15.82:0/376638421 conn(0x55b6667fa090 msgr2=0x55b6668807e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:20:55.720 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.699+0000 7f0599a9fb80 1 -- 172.21.15.82:0/376638421 shutdown_connections 2023-09-09T23:20:55.720 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.699+0000 7f0599a9fb80 1 -- 172.21.15.82:0/376638421 wait complete. 2023-09-09T23:20:55.720 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.699+0000 7f0599a9fb80 20 client.16173 trim_cache size 0 max 16384 2023-09-09T23:20:55.721 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:55 smithi082 ceph-mon[98694]: pgmap v215: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2023-09-09T23:20:55.721 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:20:55.722 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:55.727 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.716+0000 7f0599a9fb80 1 Processor -- start 2023-09-09T23:20:55.728 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.716+0000 7f0599a9fb80 1 -- start start 2023-09-09T23:20:55.728 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.716+0000 7f0599a9fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b666885710 0x55b666898bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:55.728 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.716+0000 7f0599a9fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b666886030 0x55b6668916a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:55.728 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.716+0000 7f0599a9fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b666892050 0x55b66688a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:55.729 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.716+0000 7f0599a9fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b666886710 con 0x55b666885710 2023-09-09T23:20:55.729 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.716+0000 7f0599a9fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6668821e0 con 0x55b666886030 2023-09-09T23:20:55.729 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.716+0000 7f0599a9fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b666899130 con 0x55b666892050 2023-09-09T23:20:55.729 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.716+0000 7f0586ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b666885710 0x55b666898bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:55.729 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.716+0000 7f0586ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b666885710 0x55b666898bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58012/0 (socket says 172.21.15.82:58012) 2023-09-09T23:20:55.730 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.716+0000 7f0586ffd700 1 -- 172.21.15.82:0/4020645725 learned_addr learned my addr 172.21.15.82:0/4020645725 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:20:55.730 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.716+0000 7f05867fc700 1 --2- 172.21.15.82:0/4020645725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b666886030 0x55b6668916a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:55.730 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.716+0000 7f05877fe700 1 --2- 172.21.15.82:0/4020645725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b666892050 0x55b66688a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:55.730 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.716+0000 7f0586ffd700 1 -- 172.21.15.82:0/4020645725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b666892050 msgr2=0x55b66688a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:20:55.730 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.716+0000 7f0586ffd700 1 --2- 172.21.15.82:0/4020645725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b666892050 0x55b66688a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.731 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.716+0000 7f0586ffd700 1 -- 172.21.15.82:0/4020645725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b666886030 msgr2=0x55b6668916a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:20:55.731 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.716+0000 7f0586ffd700 1 --2- 172.21.15.82:0/4020645725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b666886030 0x55b6668916a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.731 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.716+0000 7f0586ffd700 1 -- 172.21.15.82:0/4020645725 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6665ecc60 con 0x55b666885710 2023-09-09T23:20:55.731 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.716+0000 7f0586ffd700 1 --2- 172.21.15.82:0/4020645725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b666885710 0x55b666898bf0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f05780c9860 tx=0x7f0578010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16987f59113fb38e server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:20:55.732 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.716+0000 7f056e7fc700 1 -- 172.21.15.82:0/4020645725 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0578010880 con 0x55b666885710 2023-09-09T23:20:55.732 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.716+0000 7f056e7fc700 1 -- 172.21.15.82:0/4020645725 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05780109e0 con 0x55b666885710 2023-09-09T23:20:55.732 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.716+0000 7f056e7fc700 1 -- 172.21.15.82:0/4020645725 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05780997c0 con 0x55b666885710 2023-09-09T23:20:55.732 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.717+0000 7f0599a9fb80 1 -- 172.21.15.82:0/4020645725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b666885710 msgr2=0x55b666898bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:20:55.732 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.717+0000 7f0599a9fb80 1 --2- 172.21.15.82:0/4020645725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b666885710 0x55b666898bf0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f05780c9860 tx=0x7f0578010c80 comp rx=0 tx=0).stop 2023-09-09T23:20:55.732 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.717+0000 7f0599a9fb80 1 -- 172.21.15.82:0/4020645725 shutdown_connections 2023-09-09T23:20:55.733 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.717+0000 7f0599a9fb80 1 --2- 172.21.15.82:0/4020645725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b666892050 0x55b66688a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.733 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.717+0000 7f0599a9fb80 1 --2- 172.21.15.82:0/4020645725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b666885710 0x55b666898bf0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.733 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.717+0000 7f0599a9fb80 1 --2- 172.21.15.82:0/4020645725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b666886030 0x55b6668916a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.733 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.717+0000 7f0599a9fb80 1 -- 172.21.15.82:0/4020645725 >> 172.21.15.82:0/4020645725 conn(0x55b6667fa090 msgr2=0x55b66689d350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:20:55.733 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.717+0000 7f0599a9fb80 1 -- 172.21.15.82:0/4020645725 shutdown_connections 2023-09-09T23:20:55.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.717+0000 7f0599a9fb80 1 -- 172.21.15.82:0/4020645725 wait complete. 2023-09-09T23:20:55.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.717+0000 7f0599a9fb80 1 -- start start 2023-09-09T23:20:55.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.718+0000 7f0599a9fb80 1 Processor -- start 2023-09-09T23:20:55.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.718+0000 7f0599a9fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b666885710 0x55b666898970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:55.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.718+0000 7f0599a9fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b666886030 0x55b666894cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:55.735 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.718+0000 7f0599a9fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b666892050 0x55b666895230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:55.735 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.718+0000 7f0599a9fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b666895970 con 0x55b666886030 2023-09-09T23:20:55.735 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.718+0000 7f0599a9fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b666895ae0 con 0x55b666885710 2023-09-09T23:20:55.735 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.718+0000 7f0599a9fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6668f7e50 con 0x55b666892050 2023-09-09T23:20:55.735 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.718+0000 7f0599a9fb80 10 client.0 _ll_register_callbacks cb 0x7f057c003f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:20:55.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.718+0000 7f05867fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b666886030 0x55b666894cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:55.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.718+0000 7f0586ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b666885710 0x55b666898970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:55.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.718+0000 7f05867fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b666886030 0x55b666894cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58020/0 (socket says 172.21.15.82:58020) 2023-09-09T23:20:55.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.718+0000 7f05877fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b666892050 0x55b666895230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:55.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.718+0000 7f05867fc700 1 -- 172.21.15.82:0/3285198321 learned_addr learned my addr 172.21.15.82:0/3285198321 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:20:55.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.718+0000 7f05867fc700 1 -- 172.21.15.82:0/3285198321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b666892050 msgr2=0x55b666895230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:20:55.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.718+0000 7f05867fc700 1 --2- 172.21.15.82:0/3285198321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b666892050 0x55b666895230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.718+0000 7f05867fc700 1 -- 172.21.15.82:0/3285198321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b666885710 msgr2=0x55b666898970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:20:55.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.718+0000 7f05867fc700 1 --2- 172.21.15.82:0/3285198321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b666885710 0x55b666898970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.718+0000 7f05867fc700 1 -- 172.21.15.82:0/3285198321 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b666603de0 con 0x55b666886030 2023-09-09T23:20:55.738 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.718+0000 7f0586ffd700 1 --2- 172.21.15.82:0/3285198321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b666885710 0x55b666898970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:20:55.738 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.718+0000 7f05867fc700 1 --2- 172.21.15.82:0/3285198321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b666886030 0x55b666894cf0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f057c03d040 tx=0x7f057c005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3711d5940ba0da0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:20:55.738 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.718+0000 7f0566ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:20:55.738 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.719+0000 7f0566ffd700 1 -- 172.21.15.82:0/3285198321 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c02c070 con 0x55b666886030 2023-09-09T23:20:55.739 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.719+0000 7f0566ffd700 1 -- 172.21.15.82:0/3285198321 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f057c03f080 con 0x55b666886030 2023-09-09T23:20:55.739 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.719+0000 7f0599a9fb80 1 -- 172.21.15.82:0/3285198321 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f05780a15d0 con 0x55b666886030 2023-09-09T23:20:55.739 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.719+0000 7f0566ffd700 1 -- 172.21.15.82:0/3285198321 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f057c00f9b0 con 0x55b666886030 2023-09-09T23:20:55.739 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.719+0000 7f0599a9fb80 10 client.16179 fetch_fsmap learned FSMap version 76 2023-09-09T23:20:55.739 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.719+0000 7f0599a9fb80 1 -- 172.21.15.82:0/3285198321 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x55b66662a250 con 0x55b666886030 2023-09-09T23:20:55.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.719+0000 7f0566ffd700 1 -- 172.21.15.82:0/3285198321 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f057c04f020 con 0x55b666886030 2023-09-09T23:20:55.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.719+0000 7f0566ffd700 1 -- 172.21.15.82:0/3285198321 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 76) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f057c049070 con 0x55b666886030 2023-09-09T23:20:55.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.719+0000 7f0599a9fb80 10 client.16179 fetch_fsmap finished waiting for FSMap version 76 2023-09-09T23:20:55.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.719+0000 7f0599a9fb80 10 client.16179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:20:55.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.719+0000 7f0599a9fb80 20 client.16179 populate_metadata read hostname 'smithi082' 2023-09-09T23:20:55.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.719+0000 7f0599a9fb80 1 --2- 172.21.15.82:0/3285198321 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55b6668f90b0 0x55b6668fb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:20:55.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.719+0000 7f0599a9fb80 4 client.16179 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:20:55.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.719+0000 7f0599a9fb80 1 -- 172.21.15.82:0/3285198321 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b6667d0920 con 0x55b6668f90b0 2023-09-09T23:20:55.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.719+0000 7f0586ffd700 1 --2- 172.21.15.82:0/3285198321 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55b6668f90b0 0x55b6668fb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:20:55.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.720+0000 7f0566ffd700 10 client.16179 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:20:55.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.720+0000 7f0586ffd700 1 --2- 172.21.15.82:0/3285198321 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55b6668f90b0 0x55b6668fb4a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cea1468641b8592c server_cookie=9aa5aeb4fc4d897e in_seq=0 out_seq=0 2023-09-09T23:20:55.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.720+0000 7f0566ffd700 1 -- 172.21.15.82:0/3285198321 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55b6667d0920 con 0x55b6668f90b0 2023-09-09T23:20:55.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.720+0000 7f0566ffd700 10 client.16179 handle_command_reply: tid=0 2023-09-09T23:20:55.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.720+0000 7f0599a9fb80 1 client.16179 shutdown 2023-09-09T23:20:55.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.720+0000 7f0599a9fb80 1 -- 172.21.15.82:0/3285198321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b666886030 msgr2=0x55b666894cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:20:55.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.720+0000 7f0599a9fb80 1 --2- 172.21.15.82:0/3285198321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b666886030 0x55b666894cf0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f057c03d040 tx=0x7f057c005d40 comp rx=0 tx=0).stop 2023-09-09T23:20:55.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.720+0000 7f0599a9fb80 1 -- 172.21.15.82:0/3285198321 shutdown_connections 2023-09-09T23:20:55.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.720+0000 7f0599a9fb80 1 --2- 172.21.15.82:0/3285198321 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55b6668f90b0 0x55b6668fb4a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.720+0000 7f0599a9fb80 1 --2- 172.21.15.82:0/3285198321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b666885710 0x55b666898970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.720+0000 7f0599a9fb80 1 --2- 172.21.15.82:0/3285198321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b666892050 0x55b666895230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.720+0000 7f0599a9fb80 1 --2- 172.21.15.82:0/3285198321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b666886030 0x55b666894cf0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.720+0000 7f0566ffd700 0 client.16179 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:20:55.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.720+0000 7f0599a9fb80 1 -- 172.21.15.82:0/3285198321 >> 172.21.15.82:0/3285198321 conn(0x55b6667fa090 msgr2=0x55b66688a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:20:55.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.720+0000 7f0599a9fb80 1 -- 172.21.15.82:0/3285198321 shutdown_connections 2023-09-09T23:20:55.745 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.720+0000 7f0599a9fb80 1 -- 172.21.15.82:0/3285198321 wait complete. 2023-09-09T23:20:55.745 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.720+0000 7f0599a9fb80 20 client.16179 trim_cache size 0 max 16384 2023-09-09T23:20:55.745 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:20:55.746 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:20:55.746 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:20:55.746 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:20:55.746 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:20:55.746 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:20:55.747 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:20:55.747 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:20:55.747 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:20:55.747 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:20:55.749 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.721+0000 7f058d8d9700 1 -- 172.21.15.82:0/916944776 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f05700a07b0 msgr2=0x7f05700a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:20:55.749 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.721+0000 7f058d8d9700 1 --2- 172.21.15.82:0/916944776 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f05700a07b0 0x7f05700a2c70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f057c009fd0 tx=0x7f057c005fb0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.749 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.721+0000 7f058d8d9700 1 -- 172.21.15.82:0/916944776 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05880ffef0 msgr2=0x7f0588195570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:20:55.749 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.721+0000 7f058d8d9700 1 --2- 172.21.15.82:0/916944776 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05880ffef0 0x7f0588195570 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f057800e9a0 tx=0x7f057800ecb0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.749 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.721+0000 7f058d8d9700 1 -- 172.21.15.82:0/916944776 shutdown_connections 2023-09-09T23:20:55.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.721+0000 7f058d8d9700 1 --2- 172.21.15.82:0/916944776 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f05700a07b0 0x7f05700a2c70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.721+0000 7f058d8d9700 1 --2- 172.21.15.82:0/916944776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0588105560 0x7f0588199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.721+0000 7f058d8d9700 1 --2- 172.21.15.82:0/916944776 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05880ffef0 0x7f0588195570 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.721+0000 7f058d8d9700 1 --2- 172.21.15.82:0/916944776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05881008b0 0x7f0588195ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:20:55.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.721+0000 7f058d8d9700 1 -- 172.21.15.82:0/916944776 >> 172.21.15.82:0/916944776 conn(0x7f05880747a0 msgr2=0x7f05880fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:20:55.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.721+0000 7f058d8d9700 1 -- 172.21.15.82:0/916944776 shutdown_connections 2023-09-09T23:20:55.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:20:55.721+0000 7f058d8d9700 1 -- 172.21.15.82:0/916944776 wait complete. 2023-09-09T23:20:55.751 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:20:55.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:55 smithi098 ceph-mon[103124]: pgmap v215: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2023-09-09T23:20:55.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:20:55.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:55.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:55 smithi170 ceph-mon[102832]: pgmap v215: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2023-09-09T23:20:55.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:20:55.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:56.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:56.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:56.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:56.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:56.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:20:56.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:20:57.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:57 smithi082 ceph-mon[98694]: pgmap v216: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2023-09-09T23:20:57.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:57 smithi082 ceph-mon[98694]: 259 slow requests, 5 included below; oldest blocked for > 282.474715 secs 2023-09-09T23:20:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:57 smithi082 ceph-mon[98694]: slow request 282.111089 seconds old, received at 2023-09-09T23:16:14.205198+0000: client_request(mds.1:1669 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:57 smithi082 ceph-mon[98694]: slow request 282.054305 seconds old, received at 2023-09-09T23:16:14.261982+0000: client_request(mds.1:1701 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:57 smithi082 ceph-mon[98694]: slow request 281.994675 seconds old, received at 2023-09-09T23:16:14.321613+0000: client_request(mds.1:1733 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:57 smithi082 ceph-mon[98694]: slow request 281.936947 seconds old, received at 2023-09-09T23:16:14.379340+0000: client_request(mds.1:1765 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:57 smithi082 ceph-mon[98694]: slow request 281.878508 seconds old, received at 2023-09-09T23:16:14.437779+0000: client_request(mds.1:1797 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:57 smithi082 ceph-mon[98694]: 519 slow requests, 5 included below; oldest blocked for > 270.850287 secs 2023-09-09T23:20:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:57 smithi082 ceph-mon[98694]: slow request 262.827651 seconds old, received at 2023-09-09T23:16:33.860791+0000: peer_request:mds.1:2062 currently dispatched 2023-09-09T23:20:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:57 smithi082 ceph-mon[98694]: slow request 270.844099 seconds old, received at 2023-09-09T23:16:25.844343+0000: peer_request:mds.1:1487 currently dispatched 2023-09-09T23:20:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:57 smithi082 ceph-mon[98694]: slow request 270.832493 seconds old, received at 2023-09-09T23:16:25.855949+0000: peer_request:mds.1:1519 currently dispatched 2023-09-09T23:20:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:57 smithi082 ceph-mon[98694]: slow request 270.820792 seconds old, received at 2023-09-09T23:16:25.867650+0000: peer_request:mds.1:1551 currently dispatched 2023-09-09T23:20:57.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:57 smithi082 ceph-mon[98694]: slow request 270.809554 seconds old, received at 2023-09-09T23:16:25.878888+0000: peer_request:mds.1:1583 currently dispatched 2023-09-09T23:20:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:57 smithi098 ceph-mon[103124]: pgmap v216: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2023-09-09T23:20:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:57 smithi098 ceph-mon[103124]: 259 slow requests, 5 included below; oldest blocked for > 282.474715 secs 2023-09-09T23:20:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:57 smithi098 ceph-mon[103124]: slow request 282.111089 seconds old, received at 2023-09-09T23:16:14.205198+0000: client_request(mds.1:1669 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:57 smithi098 ceph-mon[103124]: slow request 282.054305 seconds old, received at 2023-09-09T23:16:14.261982+0000: client_request(mds.1:1701 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:57 smithi098 ceph-mon[103124]: slow request 281.994675 seconds old, received at 2023-09-09T23:16:14.321613+0000: client_request(mds.1:1733 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:57 smithi098 ceph-mon[103124]: slow request 281.936947 seconds old, received at 2023-09-09T23:16:14.379340+0000: client_request(mds.1:1765 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:57 smithi098 ceph-mon[103124]: slow request 281.878508 seconds old, received at 2023-09-09T23:16:14.437779+0000: client_request(mds.1:1797 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:57 smithi098 ceph-mon[103124]: 519 slow requests, 5 included below; oldest blocked for > 270.850287 secs 2023-09-09T23:20:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:57 smithi098 ceph-mon[103124]: slow request 262.827651 seconds old, received at 2023-09-09T23:16:33.860791+0000: peer_request:mds.1:2062 currently dispatched 2023-09-09T23:20:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:57 smithi098 ceph-mon[103124]: slow request 270.844099 seconds old, received at 2023-09-09T23:16:25.844343+0000: peer_request:mds.1:1487 currently dispatched 2023-09-09T23:20:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:57 smithi098 ceph-mon[103124]: slow request 270.832493 seconds old, received at 2023-09-09T23:16:25.855949+0000: peer_request:mds.1:1519 currently dispatched 2023-09-09T23:20:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:57 smithi098 ceph-mon[103124]: slow request 270.820792 seconds old, received at 2023-09-09T23:16:25.867650+0000: peer_request:mds.1:1551 currently dispatched 2023-09-09T23:20:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:57 smithi098 ceph-mon[103124]: slow request 270.809554 seconds old, received at 2023-09-09T23:16:25.878888+0000: peer_request:mds.1:1583 currently dispatched 2023-09-09T23:20:57.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:57 smithi170 ceph-mon[102832]: pgmap v216: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2023-09-09T23:20:57.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:57 smithi170 ceph-mon[102832]: 259 slow requests, 5 included below; oldest blocked for > 282.474715 secs 2023-09-09T23:20:57.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:57 smithi170 ceph-mon[102832]: slow request 282.111089 seconds old, received at 2023-09-09T23:16:14.205198+0000: client_request(mds.1:1669 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:57.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:57 smithi170 ceph-mon[102832]: slow request 282.054305 seconds old, received at 2023-09-09T23:16:14.261982+0000: client_request(mds.1:1701 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:57.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:57 smithi170 ceph-mon[102832]: slow request 281.994675 seconds old, received at 2023-09-09T23:16:14.321613+0000: client_request(mds.1:1733 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:57 smithi170 ceph-mon[102832]: slow request 281.936947 seconds old, received at 2023-09-09T23:16:14.379340+0000: client_request(mds.1:1765 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:57 smithi170 ceph-mon[102832]: slow request 281.878508 seconds old, received at 2023-09-09T23:16:14.437779+0000: client_request(mds.1:1797 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:20:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:57 smithi170 ceph-mon[102832]: 519 slow requests, 5 included below; oldest blocked for > 270.850287 secs 2023-09-09T23:20:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:57 smithi170 ceph-mon[102832]: slow request 262.827651 seconds old, received at 2023-09-09T23:16:33.860791+0000: peer_request:mds.1:2062 currently dispatched 2023-09-09T23:20:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:57 smithi170 ceph-mon[102832]: slow request 270.844099 seconds old, received at 2023-09-09T23:16:25.844343+0000: peer_request:mds.1:1487 currently dispatched 2023-09-09T23:20:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:57 smithi170 ceph-mon[102832]: slow request 270.832493 seconds old, received at 2023-09-09T23:16:25.855949+0000: peer_request:mds.1:1519 currently dispatched 2023-09-09T23:20:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:57 smithi170 ceph-mon[102832]: slow request 270.820792 seconds old, received at 2023-09-09T23:16:25.867650+0000: peer_request:mds.1:1551 currently dispatched 2023-09-09T23:20:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:57 smithi170 ceph-mon[102832]: slow request 270.809554 seconds old, received at 2023-09-09T23:16:25.878888+0000: peer_request:mds.1:1583 currently dispatched 2023-09-09T23:20:59.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:20:59 smithi082 ceph-mon[98694]: pgmap v217: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2023-09-09T23:20:59.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:20:59 smithi098 ceph-mon[103124]: pgmap v217: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2023-09-09T23:20:59.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:20:59 smithi170 ceph-mon[102832]: pgmap v217: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2023-09-09T23:21:00.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:00.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:00.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:00.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:00.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:00.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:01.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:01 smithi082 ceph-mon[98694]: pgmap v218: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.6 KiB/s wr, 12 op/s 2023-09-09T23:21:01.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:01 smithi098 ceph-mon[103124]: pgmap v218: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.6 KiB/s wr, 12 op/s 2023-09-09T23:21:01.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:01 smithi170 ceph-mon[102832]: pgmap v218: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.6 KiB/s wr, 12 op/s 2023-09-09T23:21:02.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:02 smithi098 ceph-mon[103124]: 251 slow requests, 5 included below; oldest blocked for > 287.684319 secs 2023-09-09T23:21:02.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:02 smithi098 ceph-mon[103124]: slow request 287.055445 seconds old, received at 2023-09-09T23:16:14.495668+0000: client_request(mds.1:1829 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:02.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:02 smithi098 ceph-mon[103124]: slow request 287.001306 seconds old, received at 2023-09-09T23:16:14.549808+0000: client_request(mds.1:1861 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:02.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:02 smithi098 ceph-mon[103124]: slow request 286.948192 seconds old, received at 2023-09-09T23:16:14.602921+0000: client_request(mds.1:1893 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:02.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:02 smithi098 ceph-mon[103124]: slow request 286.894336 seconds old, received at 2023-09-09T23:16:14.656777+0000: client_request(mds.1:1925 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:02 smithi098 ceph-mon[103124]: slow request 286.839770 seconds old, received at 2023-09-09T23:16:14.711344+0000: client_request(mds.1:1957 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:02 smithi098 ceph-mon[103124]: 509 slow requests, 5 included below; oldest blocked for > 275.845784 secs 2023-09-09T23:21:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:02 smithi098 ceph-mon[103124]: slow request 275.798595 seconds old, received at 2023-09-09T23:16:25.889932+0000: peer_request:mds.1:1615 currently dispatched 2023-09-09T23:21:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:02 smithi098 ceph-mon[103124]: slow request 275.787716 seconds old, received at 2023-09-09T23:16:25.900811+0000: peer_request:mds.1:1647 currently dispatched 2023-09-09T23:21:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:02 smithi098 ceph-mon[103124]: slow request 275.776287 seconds old, received at 2023-09-09T23:16:25.912240+0000: peer_request:mds.1:1679 currently dispatched 2023-09-09T23:21:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:02 smithi098 ceph-mon[103124]: slow request 275.764814 seconds old, received at 2023-09-09T23:16:25.923713+0000: peer_request:mds.1:1711 currently dispatched 2023-09-09T23:21:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:02 smithi098 ceph-mon[103124]: slow request 275.752392 seconds old, received at 2023-09-09T23:16:25.936135+0000: peer_request:mds.1:1743 currently dispatched 2023-09-09T23:21:02.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:02 smithi170 ceph-mon[102832]: 251 slow requests, 5 included below; oldest blocked for > 287.684319 secs 2023-09-09T23:21:02.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:02 smithi170 ceph-mon[102832]: slow request 287.055445 seconds old, received at 2023-09-09T23:16:14.495668+0000: client_request(mds.1:1829 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:02.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:02 smithi170 ceph-mon[102832]: slow request 287.001306 seconds old, received at 2023-09-09T23:16:14.549808+0000: client_request(mds.1:1861 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:02.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:02 smithi170 ceph-mon[102832]: slow request 286.948192 seconds old, received at 2023-09-09T23:16:14.602921+0000: client_request(mds.1:1893 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:02.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:02 smithi170 ceph-mon[102832]: slow request 286.894336 seconds old, received at 2023-09-09T23:16:14.656777+0000: client_request(mds.1:1925 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:02 smithi170 ceph-mon[102832]: slow request 286.839770 seconds old, received at 2023-09-09T23:16:14.711344+0000: client_request(mds.1:1957 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:02 smithi170 ceph-mon[102832]: 509 slow requests, 5 included below; oldest blocked for > 275.845784 secs 2023-09-09T23:21:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:02 smithi170 ceph-mon[102832]: slow request 275.798595 seconds old, received at 2023-09-09T23:16:25.889932+0000: peer_request:mds.1:1615 currently dispatched 2023-09-09T23:21:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:02 smithi170 ceph-mon[102832]: slow request 275.787716 seconds old, received at 2023-09-09T23:16:25.900811+0000: peer_request:mds.1:1647 currently dispatched 2023-09-09T23:21:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:02 smithi170 ceph-mon[102832]: slow request 275.776287 seconds old, received at 2023-09-09T23:16:25.912240+0000: peer_request:mds.1:1679 currently dispatched 2023-09-09T23:21:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:02 smithi170 ceph-mon[102832]: slow request 275.764814 seconds old, received at 2023-09-09T23:16:25.923713+0000: peer_request:mds.1:1711 currently dispatched 2023-09-09T23:21:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:02 smithi170 ceph-mon[102832]: slow request 275.752392 seconds old, received at 2023-09-09T23:16:25.936135+0000: peer_request:mds.1:1743 currently dispatched 2023-09-09T23:21:02.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:02 smithi082 ceph-mon[98694]: 251 slow requests, 5 included below; oldest blocked for > 287.684319 secs 2023-09-09T23:21:02.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:02 smithi082 ceph-mon[98694]: slow request 287.055445 seconds old, received at 2023-09-09T23:16:14.495668+0000: client_request(mds.1:1829 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:02.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:02 smithi082 ceph-mon[98694]: slow request 287.001306 seconds old, received at 2023-09-09T23:16:14.549808+0000: client_request(mds.1:1861 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:02.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:02 smithi082 ceph-mon[98694]: slow request 286.948192 seconds old, received at 2023-09-09T23:16:14.602921+0000: client_request(mds.1:1893 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:02.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:02 smithi082 ceph-mon[98694]: slow request 286.894336 seconds old, received at 2023-09-09T23:16:14.656777+0000: client_request(mds.1:1925 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:02.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:02 smithi082 ceph-mon[98694]: slow request 286.839770 seconds old, received at 2023-09-09T23:16:14.711344+0000: client_request(mds.1:1957 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:02.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:02 smithi082 ceph-mon[98694]: 509 slow requests, 5 included below; oldest blocked for > 275.845784 secs 2023-09-09T23:21:02.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:02 smithi082 ceph-mon[98694]: slow request 275.798595 seconds old, received at 2023-09-09T23:16:25.889932+0000: peer_request:mds.1:1615 currently dispatched 2023-09-09T23:21:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:02 smithi082 ceph-mon[98694]: slow request 275.787716 seconds old, received at 2023-09-09T23:16:25.900811+0000: peer_request:mds.1:1647 currently dispatched 2023-09-09T23:21:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:02 smithi082 ceph-mon[98694]: slow request 275.776287 seconds old, received at 2023-09-09T23:16:25.912240+0000: peer_request:mds.1:1679 currently dispatched 2023-09-09T23:21:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:02 smithi082 ceph-mon[98694]: slow request 275.764814 seconds old, received at 2023-09-09T23:16:25.923713+0000: peer_request:mds.1:1711 currently dispatched 2023-09-09T23:21:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:02 smithi082 ceph-mon[98694]: slow request 275.752392 seconds old, received at 2023-09-09T23:16:25.936135+0000: peer_request:mds.1:1743 currently dispatched 2023-09-09T23:21:03.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:03 smithi098 ceph-mon[103124]: pgmap v219: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.6 KiB/s wr, 12 op/s 2023-09-09T23:21:03.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:03 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:03.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:03 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:03.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:03 smithi170 ceph-mon[102832]: pgmap v219: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.6 KiB/s wr, 12 op/s 2023-09-09T23:21:03.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:03 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:03 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:03.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:03 smithi082 ceph-mon[98694]: pgmap v219: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.6 KiB/s wr, 12 op/s 2023-09-09T23:21:03.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:03 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:03.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:03 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:04.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:04.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:04.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:04.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:04.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:04.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:05.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:05 smithi098 ceph-mon[103124]: pgmap v220: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.6 KiB/s wr, 11 op/s 2023-09-09T23:21:05.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:21:05.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:05 smithi170 ceph-mon[102832]: pgmap v220: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.6 KiB/s wr, 11 op/s 2023-09-09T23:21:05.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:21:05.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:05 smithi082 ceph-mon[98694]: pgmap v220: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.6 KiB/s wr, 11 op/s 2023-09-09T23:21:05.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:21:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:07 smithi098 ceph-mon[103124]: pgmap v221: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 19 KiB/s rd, 49 KiB/s wr, 28 op/s 2023-09-09T23:21:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:07 smithi098 ceph-mon[103124]: 245 slow requests, 5 included below; oldest blocked for > 292.729196 secs 2023-09-09T23:21:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:07 smithi098 ceph-mon[103124]: slow request 291.850271 seconds old, received at 2023-09-09T23:16:14.770289+0000: client_request(mds.1:1989 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:07.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:07 smithi098 ceph-mon[103124]: slow request 291.790907 seconds old, received at 2023-09-09T23:16:14.829653+0000: client_request(mds.1:2021 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:07.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:07 smithi098 ceph-mon[103124]: slow request 291.737236 seconds old, received at 2023-09-09T23:16:14.883324+0000: client_request(mds.1:2053 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:07.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:07 smithi098 ceph-mon[103124]: slow request 292.699382 seconds old, received at 2023-09-09T23:16:13.921178+0000: client_request(mds.1:1510 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:07.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:07 smithi098 ceph-mon[103124]: slow request 292.644573 seconds old, received at 2023-09-09T23:16:13.975986+0000: client_request(mds.1:1542 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:07.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:07 smithi098 ceph-mon[103124]: 498 slow requests, 5 included below; oldest blocked for > 280.841016 secs 2023-09-09T23:21:07.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:07 smithi098 ceph-mon[103124]: slow request 280.741180 seconds old, received at 2023-09-09T23:16:25.947406+0000: peer_request:mds.1:1775 currently dispatched 2023-09-09T23:21:07.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:07 smithi098 ceph-mon[103124]: slow request 280.730805 seconds old, received at 2023-09-09T23:16:25.957781+0000: peer_request:mds.1:1807 currently dispatched 2023-09-09T23:21:07.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:07 smithi098 ceph-mon[103124]: slow request 280.719329 seconds old, received at 2023-09-09T23:16:25.969256+0000: peer_request:mds.1:1839 currently dispatched 2023-09-09T23:21:07.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:07 smithi098 ceph-mon[103124]: slow request 280.708170 seconds old, received at 2023-09-09T23:16:25.980415+0000: peer_request:mds.1:1871 currently dispatched 2023-09-09T23:21:07.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:07 smithi098 ceph-mon[103124]: slow request 280.694141 seconds old, received at 2023-09-09T23:16:25.994445+0000: peer_request:mds.1:1903 currently dispatched 2023-09-09T23:21:07.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:07 smithi170 ceph-mon[102832]: pgmap v221: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 19 KiB/s rd, 49 KiB/s wr, 28 op/s 2023-09-09T23:21:07.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:07 smithi170 ceph-mon[102832]: 245 slow requests, 5 included below; oldest blocked for > 292.729196 secs 2023-09-09T23:21:07.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:07 smithi170 ceph-mon[102832]: slow request 291.850271 seconds old, received at 2023-09-09T23:16:14.770289+0000: client_request(mds.1:1989 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:07.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:07 smithi170 ceph-mon[102832]: slow request 291.790907 seconds old, received at 2023-09-09T23:16:14.829653+0000: client_request(mds.1:2021 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:07.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:07 smithi170 ceph-mon[102832]: slow request 291.737236 seconds old, received at 2023-09-09T23:16:14.883324+0000: client_request(mds.1:2053 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:07 smithi170 ceph-mon[102832]: slow request 292.699382 seconds old, received at 2023-09-09T23:16:13.921178+0000: client_request(mds.1:1510 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:07 smithi170 ceph-mon[102832]: slow request 292.644573 seconds old, received at 2023-09-09T23:16:13.975986+0000: client_request(mds.1:1542 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:07 smithi170 ceph-mon[102832]: 498 slow requests, 5 included below; oldest blocked for > 280.841016 secs 2023-09-09T23:21:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:07 smithi170 ceph-mon[102832]: slow request 280.741180 seconds old, received at 2023-09-09T23:16:25.947406+0000: peer_request:mds.1:1775 currently dispatched 2023-09-09T23:21:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:07 smithi170 ceph-mon[102832]: slow request 280.730805 seconds old, received at 2023-09-09T23:16:25.957781+0000: peer_request:mds.1:1807 currently dispatched 2023-09-09T23:21:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:07 smithi170 ceph-mon[102832]: slow request 280.719329 seconds old, received at 2023-09-09T23:16:25.969256+0000: peer_request:mds.1:1839 currently dispatched 2023-09-09T23:21:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:07 smithi170 ceph-mon[102832]: slow request 280.708170 seconds old, received at 2023-09-09T23:16:25.980415+0000: peer_request:mds.1:1871 currently dispatched 2023-09-09T23:21:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:07 smithi170 ceph-mon[102832]: slow request 280.694141 seconds old, received at 2023-09-09T23:16:25.994445+0000: peer_request:mds.1:1903 currently dispatched 2023-09-09T23:21:07.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:07 smithi082 ceph-mon[98694]: pgmap v221: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 19 KiB/s rd, 49 KiB/s wr, 28 op/s 2023-09-09T23:21:07.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:07 smithi082 ceph-mon[98694]: 245 slow requests, 5 included below; oldest blocked for > 292.729196 secs 2023-09-09T23:21:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:07 smithi082 ceph-mon[98694]: slow request 291.850271 seconds old, received at 2023-09-09T23:16:14.770289+0000: client_request(mds.1:1989 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:07 smithi082 ceph-mon[98694]: slow request 291.790907 seconds old, received at 2023-09-09T23:16:14.829653+0000: client_request(mds.1:2021 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:07 smithi082 ceph-mon[98694]: slow request 291.737236 seconds old, received at 2023-09-09T23:16:14.883324+0000: client_request(mds.1:2053 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:07 smithi082 ceph-mon[98694]: slow request 292.699382 seconds old, received at 2023-09-09T23:16:13.921178+0000: client_request(mds.1:1510 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:07 smithi082 ceph-mon[98694]: slow request 292.644573 seconds old, received at 2023-09-09T23:16:13.975986+0000: client_request(mds.1:1542 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:07 smithi082 ceph-mon[98694]: 498 slow requests, 5 included below; oldest blocked for > 280.841016 secs 2023-09-09T23:21:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:07 smithi082 ceph-mon[98694]: slow request 280.741180 seconds old, received at 2023-09-09T23:16:25.947406+0000: peer_request:mds.1:1775 currently dispatched 2023-09-09T23:21:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:07 smithi082 ceph-mon[98694]: slow request 280.730805 seconds old, received at 2023-09-09T23:16:25.957781+0000: peer_request:mds.1:1807 currently dispatched 2023-09-09T23:21:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:07 smithi082 ceph-mon[98694]: slow request 280.719329 seconds old, received at 2023-09-09T23:16:25.969256+0000: peer_request:mds.1:1839 currently dispatched 2023-09-09T23:21:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:07 smithi082 ceph-mon[98694]: slow request 280.708170 seconds old, received at 2023-09-09T23:16:25.980415+0000: peer_request:mds.1:1871 currently dispatched 2023-09-09T23:21:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:07 smithi082 ceph-mon[98694]: slow request 280.694141 seconds old, received at 2023-09-09T23:16:25.994445+0000: peer_request:mds.1:1903 currently dispatched 2023-09-09T23:21:08.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:08 smithi098 ceph-mon[103124]: osdmap e87: 12 total, 12 up, 12 in 2023-09-09T23:21:08.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:08 smithi098 ceph-mon[103124]: pgmap v223: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:21:08.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:08 smithi170 ceph-mon[102832]: osdmap e87: 12 total, 12 up, 12 in 2023-09-09T23:21:08.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:08 smithi170 ceph-mon[102832]: pgmap v223: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:21:08.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:08 smithi082 ceph-mon[98694]: osdmap e87: 12 total, 12 up, 12 in 2023-09-09T23:21:08.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:08 smithi082 ceph-mon[98694]: pgmap v223: 97 pgs: 97 active+clean; 17 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:21:09.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:21:09.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:21:09.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:21:09.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:21:09.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:21:09.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:21:09.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:21:09.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:21:09.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:21:10.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:10 smithi098 ceph-mon[103124]: pgmap v224: 97 pgs: 97 active+clean; 17 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:21:10.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:10 smithi170 ceph-mon[102832]: pgmap v224: 97 pgs: 97 active+clean; 17 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:21:10.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:10 smithi082 ceph-mon[98694]: pgmap v224: 97 pgs: 97 active+clean; 17 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:21:11.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:11 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:11.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:11 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:11.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:11 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:11.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:11 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:11.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:11 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:11.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:11 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:12.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:12.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:12.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:12 smithi170 ceph-mon[102832]: 244 slow requests, 5 included below; oldest blocked for > 297.741447 secs 2023-09-09T23:21:12.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:12 smithi170 ceph-mon[102832]: slow request 297.619470 seconds old, received at 2023-09-09T23:16:14.035415+0000: client_request(mds.1:1574 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:12 smithi170 ceph-mon[102832]: slow request 297.562021 seconds old, received at 2023-09-09T23:16:14.092864+0000: client_request(mds.1:1606 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:12 smithi170 ceph-mon[102832]: slow request 297.505151 seconds old, received at 2023-09-09T23:16:14.149734+0000: client_request(mds.1:1638 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:12 smithi170 ceph-mon[102832]: slow request 297.447951 seconds old, received at 2023-09-09T23:16:14.206934+0000: client_request(mds.1:1670 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:12 smithi170 ceph-mon[102832]: slow request 297.391393 seconds old, received at 2023-09-09T23:16:14.263492+0000: client_request(mds.1:1702 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:12 smithi170 ceph-mon[102832]: 486 slow requests, 5 included below; oldest blocked for > 285.837466 secs 2023-09-09T23:21:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:12 smithi170 ceph-mon[102832]: slow request 285.687393 seconds old, received at 2023-09-09T23:16:26.001252+0000: peer_request:mds.1:1935 currently dispatched 2023-09-09T23:21:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:12 smithi170 ceph-mon[102832]: slow request 285.675125 seconds old, received at 2023-09-09T23:16:26.013520+0000: peer_request:mds.1:1967 currently dispatched 2023-09-09T23:21:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:12 smithi170 ceph-mon[102832]: slow request 285.663108 seconds old, received at 2023-09-09T23:16:26.025536+0000: peer_request:mds.1:1999 currently dispatched 2023-09-09T23:21:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:12 smithi170 ceph-mon[102832]: slow request 285.651717 seconds old, received at 2023-09-09T23:16:26.036927+0000: peer_request:mds.1:2031 currently dispatched 2023-09-09T23:21:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:12 smithi170 ceph-mon[102832]: slow request 277.827529 seconds old, received at 2023-09-09T23:16:33.861115+0000: peer_request:mds.1:2063 currently dispatched 2023-09-09T23:21:12.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:12 smithi170 ceph-mon[102832]: pgmap v225: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 15 KiB/s rd, 54 KiB/s wr, 26 op/s 2023-09-09T23:21:12.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:12.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:12.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:12 smithi082 ceph-mon[98694]: 244 slow requests, 5 included below; oldest blocked for > 297.741447 secs 2023-09-09T23:21:12.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:12 smithi082 ceph-mon[98694]: slow request 297.619470 seconds old, received at 2023-09-09T23:16:14.035415+0000: client_request(mds.1:1574 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:12.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:12 smithi082 ceph-mon[98694]: slow request 297.562021 seconds old, received at 2023-09-09T23:16:14.092864+0000: client_request(mds.1:1606 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:12.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:12 smithi082 ceph-mon[98694]: slow request 297.505151 seconds old, received at 2023-09-09T23:16:14.149734+0000: client_request(mds.1:1638 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:12.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:12 smithi082 ceph-mon[98694]: slow request 297.447951 seconds old, received at 2023-09-09T23:16:14.206934+0000: client_request(mds.1:1670 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:12 smithi082 ceph-mon[98694]: slow request 297.391393 seconds old, received at 2023-09-09T23:16:14.263492+0000: client_request(mds.1:1702 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:12 smithi082 ceph-mon[98694]: 486 slow requests, 5 included below; oldest blocked for > 285.837466 secs 2023-09-09T23:21:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:12 smithi082 ceph-mon[98694]: slow request 285.687393 seconds old, received at 2023-09-09T23:16:26.001252+0000: peer_request:mds.1:1935 currently dispatched 2023-09-09T23:21:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:12 smithi082 ceph-mon[98694]: slow request 285.675125 seconds old, received at 2023-09-09T23:16:26.013520+0000: peer_request:mds.1:1967 currently dispatched 2023-09-09T23:21:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:12 smithi082 ceph-mon[98694]: slow request 285.663108 seconds old, received at 2023-09-09T23:16:26.025536+0000: peer_request:mds.1:1999 currently dispatched 2023-09-09T23:21:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:12 smithi082 ceph-mon[98694]: slow request 285.651717 seconds old, received at 2023-09-09T23:16:26.036927+0000: peer_request:mds.1:2031 currently dispatched 2023-09-09T23:21:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:12 smithi082 ceph-mon[98694]: slow request 277.827529 seconds old, received at 2023-09-09T23:16:33.861115+0000: peer_request:mds.1:2063 currently dispatched 2023-09-09T23:21:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:12 smithi082 ceph-mon[98694]: pgmap v225: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 15 KiB/s rd, 54 KiB/s wr, 26 op/s 2023-09-09T23:21:13.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:13.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:13.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:12 smithi098 ceph-mon[103124]: 244 slow requests, 5 included below; oldest blocked for > 297.741447 secs 2023-09-09T23:21:13.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:12 smithi098 ceph-mon[103124]: slow request 297.619470 seconds old, received at 2023-09-09T23:16:14.035415+0000: client_request(mds.1:1574 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:12 smithi098 ceph-mon[103124]: slow request 297.562021 seconds old, received at 2023-09-09T23:16:14.092864+0000: client_request(mds.1:1606 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:12 smithi098 ceph-mon[103124]: slow request 297.505151 seconds old, received at 2023-09-09T23:16:14.149734+0000: client_request(mds.1:1638 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:12 smithi098 ceph-mon[103124]: slow request 297.447951 seconds old, received at 2023-09-09T23:16:14.206934+0000: client_request(mds.1:1670 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:12 smithi098 ceph-mon[103124]: slow request 297.391393 seconds old, received at 2023-09-09T23:16:14.263492+0000: client_request(mds.1:1702 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:12 smithi098 ceph-mon[103124]: 486 slow requests, 5 included below; oldest blocked for > 285.837466 secs 2023-09-09T23:21:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:12 smithi098 ceph-mon[103124]: slow request 285.687393 seconds old, received at 2023-09-09T23:16:26.001252+0000: peer_request:mds.1:1935 currently dispatched 2023-09-09T23:21:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:12 smithi098 ceph-mon[103124]: slow request 285.675125 seconds old, received at 2023-09-09T23:16:26.013520+0000: peer_request:mds.1:1967 currently dispatched 2023-09-09T23:21:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:12 smithi098 ceph-mon[103124]: slow request 285.663108 seconds old, received at 2023-09-09T23:16:26.025536+0000: peer_request:mds.1:1999 currently dispatched 2023-09-09T23:21:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:12 smithi098 ceph-mon[103124]: slow request 285.651717 seconds old, received at 2023-09-09T23:16:26.036927+0000: peer_request:mds.1:2031 currently dispatched 2023-09-09T23:21:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:12 smithi098 ceph-mon[103124]: slow request 277.827529 seconds old, received at 2023-09-09T23:16:33.861115+0000: peer_request:mds.1:2063 currently dispatched 2023-09-09T23:21:13.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:12 smithi098 ceph-mon[103124]: pgmap v225: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 15 KiB/s rd, 54 KiB/s wr, 26 op/s 2023-09-09T23:21:14.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:14 smithi170 ceph-mon[102832]: osdmap e88: 12 total, 12 up, 12 in 2023-09-09T23:21:14.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:14 smithi082 ceph-mon[98694]: osdmap e88: 12 total, 12 up, 12 in 2023-09-09T23:21:14.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:14 smithi098 ceph-mon[103124]: osdmap e88: 12 total, 12 up, 12 in 2023-09-09T23:21:15.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:15 smithi170 ceph-mon[102832]: pgmap v227: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.5 KiB/s rd, 383 B/s wr, 6 op/s 2023-09-09T23:21:15.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:15 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:15.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:15 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:15.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:15 smithi082 ceph-mon[98694]: pgmap v227: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.5 KiB/s rd, 383 B/s wr, 6 op/s 2023-09-09T23:21:15.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:15 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:15.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:15 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:15.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:15 smithi098 ceph-mon[103124]: pgmap v227: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.5 KiB/s rd, 383 B/s wr, 6 op/s 2023-09-09T23:21:15.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:15 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:15.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:15 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:16.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:16.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:16.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:16.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:16.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:16.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:17.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:17 smithi170 ceph-mon[102832]: pgmap v228: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2023-09-09T23:21:17.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:17 smithi170 ceph-mon[102832]: 477 slow requests, 5 included below; oldest blocked for > 290.832770 secs 2023-09-09T23:21:17.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:17 smithi170 ceph-mon[102832]: slow request 290.832355 seconds old, received at 2023-09-09T23:16:25.856363+0000: peer_request:mds.1:1520 currently dispatched 2023-09-09T23:21:17.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:17 smithi170 ceph-mon[102832]: slow request 290.820754 seconds old, received at 2023-09-09T23:16:25.867964+0000: peer_request:mds.1:1552 currently dispatched 2023-09-09T23:21:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:17 smithi170 ceph-mon[102832]: slow request 290.809528 seconds old, received at 2023-09-09T23:16:25.879191+0000: peer_request:mds.1:1584 currently dispatched 2023-09-09T23:21:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:17 smithi170 ceph-mon[102832]: slow request 290.798535 seconds old, received at 2023-09-09T23:16:25.890184+0000: peer_request:mds.1:1616 currently dispatched 2023-09-09T23:21:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:17 smithi170 ceph-mon[102832]: slow request 290.787535 seconds old, received at 2023-09-09T23:16:25.901183+0000: peer_request:mds.1:1648 currently dispatched 2023-09-09T23:21:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:17 smithi170 ceph-mon[102832]: 231 slow requests, 5 included below; oldest blocked for > 302.754408 secs 2023-09-09T23:21:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:17 smithi170 ceph-mon[102832]: slow request 302.367694 seconds old, received at 2023-09-09T23:16:14.323541+0000: client_request(mds.1:1734 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:17 smithi170 ceph-mon[102832]: slow request 302.310087 seconds old, received at 2023-09-09T23:16:14.381149+0000: client_request(mds.1:1766 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:17 smithi170 ceph-mon[102832]: slow request 302.251502 seconds old, received at 2023-09-09T23:16:14.439734+0000: client_request(mds.1:1798 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:17 smithi170 ceph-mon[102832]: slow request 302.193892 seconds old, received at 2023-09-09T23:16:14.497343+0000: client_request(mds.1:1830 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:17 smithi170 ceph-mon[102832]: slow request 302.139833 seconds old, received at 2023-09-09T23:16:14.551403+0000: client_request(mds.1:1862 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:17.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:17 smithi082 ceph-mon[98694]: pgmap v228: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2023-09-09T23:21:17.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:17 smithi082 ceph-mon[98694]: 477 slow requests, 5 included below; oldest blocked for > 290.832770 secs 2023-09-09T23:21:17.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:17 smithi082 ceph-mon[98694]: slow request 290.832355 seconds old, received at 2023-09-09T23:16:25.856363+0000: peer_request:mds.1:1520 currently dispatched 2023-09-09T23:21:17.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:17 smithi082 ceph-mon[98694]: slow request 290.820754 seconds old, received at 2023-09-09T23:16:25.867964+0000: peer_request:mds.1:1552 currently dispatched 2023-09-09T23:21:17.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:17 smithi082 ceph-mon[98694]: slow request 290.809528 seconds old, received at 2023-09-09T23:16:25.879191+0000: peer_request:mds.1:1584 currently dispatched 2023-09-09T23:21:17.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:17 smithi082 ceph-mon[98694]: slow request 290.798535 seconds old, received at 2023-09-09T23:16:25.890184+0000: peer_request:mds.1:1616 currently dispatched 2023-09-09T23:21:17.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:17 smithi082 ceph-mon[98694]: slow request 290.787535 seconds old, received at 2023-09-09T23:16:25.901183+0000: peer_request:mds.1:1648 currently dispatched 2023-09-09T23:21:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:17 smithi082 ceph-mon[98694]: 231 slow requests, 5 included below; oldest blocked for > 302.754408 secs 2023-09-09T23:21:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:17 smithi082 ceph-mon[98694]: slow request 302.367694 seconds old, received at 2023-09-09T23:16:14.323541+0000: client_request(mds.1:1734 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:17 smithi082 ceph-mon[98694]: slow request 302.310087 seconds old, received at 2023-09-09T23:16:14.381149+0000: client_request(mds.1:1766 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:17 smithi082 ceph-mon[98694]: slow request 302.251502 seconds old, received at 2023-09-09T23:16:14.439734+0000: client_request(mds.1:1798 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:17 smithi082 ceph-mon[98694]: slow request 302.193892 seconds old, received at 2023-09-09T23:16:14.497343+0000: client_request(mds.1:1830 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:17.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:17 smithi082 ceph-mon[98694]: slow request 302.139833 seconds old, received at 2023-09-09T23:16:14.551403+0000: client_request(mds.1:1862 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:17.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:17 smithi098 ceph-mon[103124]: pgmap v228: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2023-09-09T23:21:17.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:17 smithi098 ceph-mon[103124]: 477 slow requests, 5 included below; oldest blocked for > 290.832770 secs 2023-09-09T23:21:17.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:17 smithi098 ceph-mon[103124]: slow request 290.832355 seconds old, received at 2023-09-09T23:16:25.856363+0000: peer_request:mds.1:1520 currently dispatched 2023-09-09T23:21:17.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:17 smithi098 ceph-mon[103124]: slow request 290.820754 seconds old, received at 2023-09-09T23:16:25.867964+0000: peer_request:mds.1:1552 currently dispatched 2023-09-09T23:21:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:17 smithi098 ceph-mon[103124]: slow request 290.809528 seconds old, received at 2023-09-09T23:16:25.879191+0000: peer_request:mds.1:1584 currently dispatched 2023-09-09T23:21:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:17 smithi098 ceph-mon[103124]: slow request 290.798535 seconds old, received at 2023-09-09T23:16:25.890184+0000: peer_request:mds.1:1616 currently dispatched 2023-09-09T23:21:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:17 smithi098 ceph-mon[103124]: slow request 290.787535 seconds old, received at 2023-09-09T23:16:25.901183+0000: peer_request:mds.1:1648 currently dispatched 2023-09-09T23:21:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:17 smithi098 ceph-mon[103124]: 231 slow requests, 5 included below; oldest blocked for > 302.754408 secs 2023-09-09T23:21:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:17 smithi098 ceph-mon[103124]: slow request 302.367694 seconds old, received at 2023-09-09T23:16:14.323541+0000: client_request(mds.1:1734 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:17 smithi098 ceph-mon[103124]: slow request 302.310087 seconds old, received at 2023-09-09T23:16:14.381149+0000: client_request(mds.1:1766 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:17 smithi098 ceph-mon[103124]: slow request 302.251502 seconds old, received at 2023-09-09T23:16:14.439734+0000: client_request(mds.1:1798 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:17 smithi098 ceph-mon[103124]: slow request 302.193892 seconds old, received at 2023-09-09T23:16:14.497343+0000: client_request(mds.1:1830 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:17 smithi098 ceph-mon[103124]: slow request 302.139833 seconds old, received at 2023-09-09T23:16:14.551403+0000: client_request(mds.1:1862 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:19.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:19 smithi082 ceph-mon[98694]: pgmap v229: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 12 op/s 2023-09-09T23:21:19.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:19 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:19.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:19 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:19.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:19 smithi098 ceph-mon[103124]: pgmap v229: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 12 op/s 2023-09-09T23:21:19.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:19 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:19.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:19 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:19.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:19 smithi170 ceph-mon[102832]: pgmap v229: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 12 op/s 2023-09-09T23:21:19.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:19 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:19.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:19 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:20.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:20.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:20.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:20.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:20.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:20.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:21.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:21 smithi082 ceph-mon[98694]: pgmap v230: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-09-09T23:21:21.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:21 smithi098 ceph-mon[103124]: pgmap v230: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-09-09T23:21:21.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:21 smithi170 ceph-mon[102832]: pgmap v230: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-09-09T23:21:22.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:22 smithi082 ceph-mon[98694]: 463 slow requests, 5 included below; oldest blocked for > 295.827435 secs 2023-09-09T23:21:22.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:22 smithi082 ceph-mon[98694]: slow request 295.775931 seconds old, received at 2023-09-09T23:16:25.912846+0000: peer_request:mds.1:1680 currently dispatched 2023-09-09T23:21:22.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:22 smithi082 ceph-mon[98694]: slow request 295.764742 seconds old, received at 2023-09-09T23:16:25.924036+0000: peer_request:mds.1:1712 currently dispatched 2023-09-09T23:21:22.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:22 smithi082 ceph-mon[98694]: slow request 295.752225 seconds old, received at 2023-09-09T23:16:25.936552+0000: peer_request:mds.1:1744 currently dispatched 2023-09-09T23:21:22.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:22 smithi082 ceph-mon[98694]: slow request 295.741035 seconds old, received at 2023-09-09T23:16:25.947743+0000: peer_request:mds.1:1776 currently dispatched 2023-09-09T23:21:22.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:22 smithi082 ceph-mon[98694]: slow request 295.730616 seconds old, received at 2023-09-09T23:16:25.958161+0000: peer_request:mds.1:1808 currently dispatched 2023-09-09T23:21:22.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:22 smithi082 ceph-mon[98694]: 231 slow requests, 5 included below; oldest blocked for > 307.959034 secs 2023-09-09T23:21:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:22 smithi082 ceph-mon[98694]: slow request 307.315313 seconds old, received at 2023-09-09T23:16:14.604440+0000: client_request(mds.1:1894 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:22 smithi082 ceph-mon[98694]: slow request 307.261461 seconds old, received at 2023-09-09T23:16:14.658291+0000: client_request(mds.1:1926 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:22 smithi082 ceph-mon[98694]: slow request 307.206951 seconds old, received at 2023-09-09T23:16:14.712801+0000: client_request(mds.1:1958 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:22 smithi082 ceph-mon[98694]: slow request 307.148033 seconds old, received at 2023-09-09T23:16:14.771720+0000: client_request(mds.1:1990 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:22 smithi082 ceph-mon[98694]: slow request 307.088456 seconds old, received at 2023-09-09T23:16:14.831296+0000: client_request(mds.1:2022 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:22.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:22 smithi098 ceph-mon[103124]: 463 slow requests, 5 included below; oldest blocked for > 295.827435 secs 2023-09-09T23:21:22.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:22 smithi098 ceph-mon[103124]: slow request 295.775931 seconds old, received at 2023-09-09T23:16:25.912846+0000: peer_request:mds.1:1680 currently dispatched 2023-09-09T23:21:22.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:22 smithi098 ceph-mon[103124]: slow request 295.764742 seconds old, received at 2023-09-09T23:16:25.924036+0000: peer_request:mds.1:1712 currently dispatched 2023-09-09T23:21:22.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:22 smithi098 ceph-mon[103124]: slow request 295.752225 seconds old, received at 2023-09-09T23:16:25.936552+0000: peer_request:mds.1:1744 currently dispatched 2023-09-09T23:21:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:22 smithi098 ceph-mon[103124]: slow request 295.741035 seconds old, received at 2023-09-09T23:16:25.947743+0000: peer_request:mds.1:1776 currently dispatched 2023-09-09T23:21:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:22 smithi098 ceph-mon[103124]: slow request 295.730616 seconds old, received at 2023-09-09T23:16:25.958161+0000: peer_request:mds.1:1808 currently dispatched 2023-09-09T23:21:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:22 smithi098 ceph-mon[103124]: 231 slow requests, 5 included below; oldest blocked for > 307.959034 secs 2023-09-09T23:21:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:22 smithi098 ceph-mon[103124]: slow request 307.315313 seconds old, received at 2023-09-09T23:16:14.604440+0000: client_request(mds.1:1894 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:22 smithi098 ceph-mon[103124]: slow request 307.261461 seconds old, received at 2023-09-09T23:16:14.658291+0000: client_request(mds.1:1926 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:22 smithi098 ceph-mon[103124]: slow request 307.206951 seconds old, received at 2023-09-09T23:16:14.712801+0000: client_request(mds.1:1958 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:22 smithi098 ceph-mon[103124]: slow request 307.148033 seconds old, received at 2023-09-09T23:16:14.771720+0000: client_request(mds.1:1990 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:22 smithi098 ceph-mon[103124]: slow request 307.088456 seconds old, received at 2023-09-09T23:16:14.831296+0000: client_request(mds.1:2022 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:22.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:22 smithi170 ceph-mon[102832]: 463 slow requests, 5 included below; oldest blocked for > 295.827435 secs 2023-09-09T23:21:22.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:22 smithi170 ceph-mon[102832]: slow request 295.775931 seconds old, received at 2023-09-09T23:16:25.912846+0000: peer_request:mds.1:1680 currently dispatched 2023-09-09T23:21:22.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:22 smithi170 ceph-mon[102832]: slow request 295.764742 seconds old, received at 2023-09-09T23:16:25.924036+0000: peer_request:mds.1:1712 currently dispatched 2023-09-09T23:21:22.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:22 smithi170 ceph-mon[102832]: slow request 295.752225 seconds old, received at 2023-09-09T23:16:25.936552+0000: peer_request:mds.1:1744 currently dispatched 2023-09-09T23:21:22.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:22 smithi170 ceph-mon[102832]: slow request 295.741035 seconds old, received at 2023-09-09T23:16:25.947743+0000: peer_request:mds.1:1776 currently dispatched 2023-09-09T23:21:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:22 smithi170 ceph-mon[102832]: slow request 295.730616 seconds old, received at 2023-09-09T23:16:25.958161+0000: peer_request:mds.1:1808 currently dispatched 2023-09-09T23:21:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:22 smithi170 ceph-mon[102832]: 231 slow requests, 5 included below; oldest blocked for > 307.959034 secs 2023-09-09T23:21:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:22 smithi170 ceph-mon[102832]: slow request 307.315313 seconds old, received at 2023-09-09T23:16:14.604440+0000: client_request(mds.1:1894 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:22 smithi170 ceph-mon[102832]: slow request 307.261461 seconds old, received at 2023-09-09T23:16:14.658291+0000: client_request(mds.1:1926 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:22 smithi170 ceph-mon[102832]: slow request 307.206951 seconds old, received at 2023-09-09T23:16:14.712801+0000: client_request(mds.1:1958 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:22 smithi170 ceph-mon[102832]: slow request 307.148033 seconds old, received at 2023-09-09T23:16:14.771720+0000: client_request(mds.1:1990 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:22 smithi170 ceph-mon[102832]: slow request 307.088456 seconds old, received at 2023-09-09T23:16:14.831296+0000: client_request(mds.1:2022 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:23.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:23 smithi082 ceph-mon[98694]: pgmap v231: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 15 op/s 2023-09-09T23:21:23.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:23 smithi098 ceph-mon[103124]: pgmap v231: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 15 op/s 2023-09-09T23:21:23.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:23 smithi170 ceph-mon[102832]: pgmap v231: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 15 op/s 2023-09-09T23:21:24.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:24.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:24.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:24.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:24.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:24.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:25.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:25 smithi098 ceph-mon[103124]: pgmap v232: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 14 op/s 2023-09-09T23:21:25.752 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:21:25.782 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:25 smithi082 ceph-mon[98694]: pgmap v232: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 14 op/s 2023-09-09T23:21:25.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.891+0000 7fae0c381700 1 -- 172.21.15.82:0/2272677057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae041054f0 msgr2=0x7fae041058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:21:25.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.891+0000 7fae0c381700 1 --2- 172.21.15.82:0/2272677057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae041054f0 0x7fae041058d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fadf4009990 tx=0x7fadf4009ca0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.891+0000 7fae0c381700 1 -- 172.21.15.82:0/2272677057 shutdown_connections 2023-09-09T23:21:25.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.891+0000 7fae0c381700 1 --2- 172.21.15.82:0/2272677057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae040ffe10 0x7fae0410c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.891+0000 7fae0c381700 1 --2- 172.21.15.82:0/2272677057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae041054f0 0x7fae041058d0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.891+0000 7fae0c381700 1 --2- 172.21.15.82:0/2272677057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae040ff450 0x7fae040ff8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.891+0000 7fae0c381700 1 -- 172.21.15.82:0/2272677057 >> 172.21.15.82:0/2272677057 conn(0x7fae040faf60 msgr2=0x7fae040fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:21:25.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.891+0000 7fae0c381700 1 -- 172.21.15.82:0/2272677057 shutdown_connections 2023-09-09T23:21:25.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.891+0000 7fae0c381700 1 -- 172.21.15.82:0/2272677057 wait complete. 2023-09-09T23:21:25.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.891+0000 7fae0c381700 1 Processor -- start 2023-09-09T23:21:25.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.891+0000 7fae0c381700 1 -- start start 2023-09-09T23:21:25.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.892+0000 7fae0c381700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae040ff450 0x7fae04195550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:25.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.892+0000 7fae0c381700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae040ffe10 0x7fae04195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:25.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.892+0000 7fae0c381700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae041054f0 0x7fae04199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:25.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.892+0000 7fae0c381700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0419a4e0 con 0x7fae041054f0 2023-09-09T23:21:25.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.892+0000 7fae0c381700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0419a650 con 0x7fae040ff450 2023-09-09T23:21:25.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.892+0000 7fae0c381700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0419a810 con 0x7fae040ffe10 2023-09-09T23:21:25.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.892+0000 7fae0a91e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae041054f0 0x7fae04199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:25.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.892+0000 7fae0a11d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae040ff450 0x7fae04195550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:25.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.892+0000 7fae0a91e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae041054f0 0x7fae04199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58040/0 (socket says 172.21.15.82:58040) 2023-09-09T23:21:25.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.892+0000 7fae0991c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae040ffe10 0x7fae04195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:25.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.892+0000 7fae0a91e700 1 -- 172.21.15.82:0/2569250474 learned_addr learned my addr 172.21.15.82:0/2569250474 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:21:25.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.892+0000 7fae0a91e700 1 -- 172.21.15.82:0/2569250474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae040ffe10 msgr2=0x7fae04195a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:21:25.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.892+0000 7fae0a91e700 1 --2- 172.21.15.82:0/2569250474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae040ffe10 0x7fae04195a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.897 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.892+0000 7fae0a91e700 1 -- 172.21.15.82:0/2569250474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae040ff450 msgr2=0x7fae04195550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:21:25.897 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.892+0000 7fae0a91e700 1 --2- 172.21.15.82:0/2569250474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae040ff450 0x7fae04195550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.897 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.892+0000 7fae0a91e700 1 -- 172.21.15.82:0/2569250474 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf40095f0 con 0x7fae041054f0 2023-09-09T23:21:25.897 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.892+0000 7fae0a91e700 1 --2- 172.21.15.82:0/2569250474 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae041054f0 0x7fae04199ea0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fadfc00ef30 tx=0x7fadfc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:21:25.897 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.892+0000 7fadfb7fe700 1 -- 172.21.15.82:0/2569250474 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadfc018070 con 0x7fae041054f0 2023-09-09T23:21:25.898 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.892+0000 7fae0c381700 1 -- 172.21.15.82:0/2569250474 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae0419aa90 con 0x7fae041054f0 2023-09-09T23:21:25.898 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.892+0000 7fadfb7fe700 1 -- 172.21.15.82:0/2569250474 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadfc00f040 con 0x7fae041054f0 2023-09-09T23:21:25.898 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.892+0000 7fadfb7fe700 1 -- 172.21.15.82:0/2569250474 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadfc00cd00 con 0x7fae041054f0 2023-09-09T23:21:25.898 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.892+0000 7fae0c381700 1 -- 172.21.15.82:0/2569250474 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae041a2330 con 0x7fae041054f0 2023-09-09T23:21:25.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.895+0000 7fadfb7fe700 1 -- 172.21.15.82:0/2569250474 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fadfc003a00 con 0x7fae041054f0 2023-09-09T23:21:25.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.896+0000 7fadfb7fe700 1 --2- 172.21.15.82:0/2569250474 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fadf00a07b0 0x7fadf00a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:25.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.896+0000 7fadfb7fe700 1 -- 172.21.15.82:0/2569250474 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fadfc0cbad0 con 0x7fae041054f0 2023-09-09T23:21:25.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.896+0000 7fae0a11d700 1 --2- 172.21.15.82:0/2569250474 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fadf00a07b0 0x7fadf00a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:25.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.896+0000 7fae18547b80 1 Processor -- start 2023-09-09T23:21:25.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.896+0000 7fae0a11d700 1 --2- 172.21.15.82:0/2569250474 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fadf00a07b0 0x7fadf00a2c70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fadf400b2e0 tx=0x7fadf4005960 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:21:25.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.897+0000 7fae18547b80 1 -- start start 2023-09-09T23:21:25.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.897+0000 7fae18547b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec6adae710 0x55ec6adaeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:25.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.897+0000 7fae18547b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec6adaf030 0x55ec6adb8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:25.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.897+0000 7fae18547b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec6adb8e40 0x55ec6adbb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:25.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.897+0000 7fae18547b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6ad07730 con 0x55ec6adae710 2023-09-09T23:21:25.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.897+0000 7fae18547b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6ad05050 con 0x55ec6adb8e40 2023-09-09T23:21:25.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.897+0000 7fae18547b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6ad00940 con 0x55ec6adaf030 2023-09-09T23:21:25.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.897+0000 7fae0a11d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec6adae710 0x55ec6adaeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:25.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.897+0000 7fae0a11d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec6adae710 0x55ec6adaeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58044/0 (socket says 172.21.15.82:58044) 2023-09-09T23:21:25.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.897+0000 7fae0a11d700 1 -- 172.21.15.82:0/2645239688 learned_addr learned my addr 172.21.15.82:0/2645239688 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:21:25.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.897+0000 7fae0991c700 1 --2- 172.21.15.82:0/2645239688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec6adaf030 0x55ec6adb8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:25.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.897+0000 7fae0a91e700 1 --2- 172.21.15.82:0/2645239688 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec6adb8e40 0x55ec6adbb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:25.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.897+0000 7fae0a11d700 1 -- 172.21.15.82:0/2645239688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec6adaf030 msgr2=0x55ec6adb8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:21:25.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.897+0000 7fae0a11d700 1 --2- 172.21.15.82:0/2645239688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec6adaf030 0x55ec6adb8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.897+0000 7fae0a11d700 1 -- 172.21.15.82:0/2645239688 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec6adb8e40 msgr2=0x55ec6adbb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:21:25.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.897+0000 7fae0a11d700 1 --2- 172.21.15.82:0/2645239688 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec6adb8e40 0x55ec6adbb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.897+0000 7fae0a11d700 1 -- 172.21.15.82:0/2645239688 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec6ab2cde0 con 0x55ec6adae710 2023-09-09T23:21:25.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.897+0000 7fae0a91e700 1 --2- 172.21.15.82:0/2645239688 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec6adb8e40 0x55ec6adbb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:21:25.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.897+0000 7fae0991c700 1 --2- 172.21.15.82:0/2645239688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec6adaf030 0x55ec6adb8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:21:25.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.897+0000 7fae0a11d700 1 --2- 172.21.15.82:0/2645239688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec6adae710 0x55ec6adaeaf0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fadf400b8c0 tx=0x7fadf4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4994210891ba3807 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:21:25.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.898+0000 7fadf8d89700 1 -- 172.21.15.82:0/2645239688 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf402b070 con 0x55ec6adae710 2023-09-09T23:21:25.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.898+0000 7fadf8d89700 1 -- 172.21.15.82:0/2645239688 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadf401e040 con 0x55ec6adae710 2023-09-09T23:21:25.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.898+0000 7fadf8d89700 1 -- 172.21.15.82:0/2645239688 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf400f6c0 con 0x55ec6adae710 2023-09-09T23:21:25.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.898+0000 7fae18547b80 1 -- 172.21.15.82:0/2645239688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec6adae710 msgr2=0x55ec6adaeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:21:25.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.898+0000 7fae18547b80 1 --2- 172.21.15.82:0/2645239688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec6adae710 0x55ec6adaeaf0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fadf400b8c0 tx=0x7fadf4020000 comp rx=0 tx=0).stop 2023-09-09T23:21:25.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.898+0000 7fae18547b80 1 -- 172.21.15.82:0/2645239688 shutdown_connections 2023-09-09T23:21:25.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.898+0000 7fae18547b80 1 --2- 172.21.15.82:0/2645239688 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec6adb8e40 0x55ec6adbb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.898+0000 7fae18547b80 1 --2- 172.21.15.82:0/2645239688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec6adae710 0x55ec6adaeaf0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.898+0000 7fae18547b80 1 --2- 172.21.15.82:0/2645239688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec6adaf030 0x55ec6adb8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.898+0000 7fae18547b80 1 -- 172.21.15.82:0/2645239688 >> 172.21.15.82:0/2645239688 conn(0x55ec6ad23090 msgr2=0x55ec6ada9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:21:25.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.898+0000 7fae18547b80 1 -- 172.21.15.82:0/2645239688 shutdown_connections 2023-09-09T23:21:25.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.898+0000 7fae18547b80 1 -- 172.21.15.82:0/2645239688 wait complete. 2023-09-09T23:21:25.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.899+0000 7fae18547b80 1 -- start start 2023-09-09T23:21:25.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.899+0000 7fae18547b80 1 Processor -- start 2023-09-09T23:21:25.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.899+0000 7fae18547b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec6adae710 0x55ec6adb8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:25.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.899+0000 7fae18547b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec6adaf030 0x55ec6adb5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:25.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.899+0000 7fae18547b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec6adb8e40 0x55ec6adb6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:25.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.899+0000 7fae18547b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6ac9eb70 con 0x55ec6adaf030 2023-09-09T23:21:25.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.899+0000 7fae18547b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6ac9e870 con 0x55ec6adb8e40 2023-09-09T23:21:25.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.899+0000 7fae18547b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6ab82920 con 0x55ec6adae710 2023-09-09T23:21:25.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.899+0000 7fae18547b80 10 client.0 _ll_register_callbacks cb 0x55ec6abe8a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:21:25.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.899+0000 7fae0991c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec6adaf030 0x55ec6adb5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:25.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.899+0000 7fae0991c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec6adaf030 0x55ec6adb5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58052/0 (socket says 172.21.15.82:58052) 2023-09-09T23:21:25.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.899+0000 7fae0991c700 1 -- 172.21.15.82:0/1003976959 learned_addr learned my addr 172.21.15.82:0/1003976959 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:21:25.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.899+0000 7fae0a11d700 1 --2- 172.21.15.82:0/1003976959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec6adae710 0x55ec6adb8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:25.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.899+0000 7fae0a91e700 1 --2- 172.21.15.82:0/1003976959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec6adb8e40 0x55ec6adb6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:25.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.900+0000 7fae0991c700 1 -- 172.21.15.82:0/1003976959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec6adae710 msgr2=0x55ec6adb8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:21:25.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.900+0000 7fae0991c700 1 --2- 172.21.15.82:0/1003976959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec6adae710 0x55ec6adb8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.900+0000 7fae0991c700 1 -- 172.21.15.82:0/1003976959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec6adb8e40 msgr2=0x55ec6adb6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:21:25.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.900+0000 7fae0991c700 1 --2- 172.21.15.82:0/1003976959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec6adb8e40 0x55ec6adb6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.900+0000 7fae0991c700 1 -- 172.21.15.82:0/1003976959 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae000098e0 con 0x55ec6adaf030 2023-09-09T23:21:25.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.900+0000 7fae0991c700 1 --2- 172.21.15.82:0/1003976959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec6adaf030 0x55ec6adb5a40 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fae00012880 tx=0x7fae00012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161b0614b7ecd966 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:21:25.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.900+0000 7fadedffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:21:25.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.900+0000 7fadedffb700 1 -- 172.21.15.82:0/1003976959 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae00017070 con 0x55ec6adaf030 2023-09-09T23:21:25.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.900+0000 7fadedffb700 1 -- 172.21.15.82:0/1003976959 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fae0000f560 con 0x55ec6adaf030 2023-09-09T23:21:25.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.900+0000 7fae18547b80 1 -- 172.21.15.82:0/1003976959 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec6ac584f0 con 0x55ec6adaf030 2023-09-09T23:21:25.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.900+0000 7fadedffb700 1 -- 172.21.15.82:0/1003976959 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae0000f720 con 0x55ec6adaf030 2023-09-09T23:21:25.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.900+0000 7fae18547b80 10 client.16191 fetch_fsmap learned FSMap version 86 2023-09-09T23:21:25.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.900+0000 7fae18547b80 1 -- 172.21.15.82:0/1003976959 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x7fae041a2330 con 0x55ec6adaf030 2023-09-09T23:21:25.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.900+0000 7fadedffb700 1 -- 172.21.15.82:0/1003976959 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ec6ac584f0 con 0x55ec6adaf030 2023-09-09T23:21:25.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.900+0000 7fadedffb700 1 -- 172.21.15.82:0/1003976959 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 86) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fae00010940 con 0x55ec6adaf030 2023-09-09T23:21:25.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.900+0000 7fae18547b80 10 client.16191 fetch_fsmap finished waiting for FSMap version 86 2023-09-09T23:21:25.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.900+0000 7fae18547b80 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:21:25.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.900+0000 7fae18547b80 20 client.16191 populate_metadata read hostname 'smithi082' 2023-09-09T23:21:25.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.901+0000 7fae18547b80 1 --2- 172.21.15.82:0/1003976959 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55ec6adc89c0 0x55ec6adcade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:25.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.901+0000 7fae18547b80 4 client.16191 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:21:25.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.901+0000 7fae18547b80 1 -- 172.21.15.82:0/1003976959 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec6ad158a0 con 0x55ec6adc89c0 2023-09-09T23:21:25.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.901+0000 7fae0a11d700 1 --2- 172.21.15.82:0/1003976959 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55ec6adc89c0 0x55ec6adcade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:25.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.901+0000 7fadedffb700 10 client.16191 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:21:25.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.901+0000 7fae0a11d700 1 --2- 172.21.15.82:0/1003976959 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55ec6adc89c0 0x55ec6adcade0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c3819225d144a12 server_cookie=3292bd2335344c4a in_seq=0 out_seq=0 2023-09-09T23:21:25.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.902+0000 7fadedffb700 1 -- 172.21.15.82:0/1003976959 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ec6ad158a0 con 0x55ec6adc89c0 2023-09-09T23:21:25.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.902+0000 7fadedffb700 10 client.16191 handle_command_reply: tid=0 2023-09-09T23:21:25.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.902+0000 7fae18547b80 1 client.16191 shutdown 2023-09-09T23:21:25.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.902+0000 7fae18547b80 1 -- 172.21.15.82:0/1003976959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec6adaf030 msgr2=0x55ec6adb5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:21:25.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.902+0000 7fae18547b80 1 --2- 172.21.15.82:0/1003976959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec6adaf030 0x55ec6adb5a40 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fae00012880 tx=0x7fae00012b90 comp rx=0 tx=0).stop 2023-09-09T23:21:25.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.902+0000 7fae18547b80 1 -- 172.21.15.82:0/1003976959 shutdown_connections 2023-09-09T23:21:25.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.902+0000 7fae18547b80 1 --2- 172.21.15.82:0/1003976959 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55ec6adc89c0 0x55ec6adcade0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.902+0000 7fae18547b80 1 --2- 172.21.15.82:0/1003976959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec6adb8e40 0x55ec6adb6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.902+0000 7fadedffb700 0 client.16191 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:21:25.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.902+0000 7fae18547b80 1 --2- 172.21.15.82:0/1003976959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec6adae710 0x55ec6adb8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.902+0000 7fae18547b80 1 --2- 172.21.15.82:0/1003976959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec6adaf030 0x55ec6adb5a40 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.902+0000 7fae18547b80 1 -- 172.21.15.82:0/1003976959 >> 172.21.15.82:0/1003976959 conn(0x55ec6ad23090 msgr2=0x55ec6ada97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:21:25.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.902+0000 7fae18547b80 1 -- 172.21.15.82:0/1003976959 shutdown_connections 2023-09-09T23:21:25.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.902+0000 7fae18547b80 1 -- 172.21.15.82:0/1003976959 wait complete. 2023-09-09T23:21:25.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.902+0000 7fae18547b80 20 client.16191 trim_cache size 0 max 16384 2023-09-09T23:21:25.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.919+0000 7fae18547b80 1 Processor -- start 2023-09-09T23:21:25.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.919+0000 7fae18547b80 1 -- start start 2023-09-09T23:21:25.928 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.919+0000 7fae18547b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec6adae710 0x55ec6adb8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:25.928 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.919+0000 7fae18547b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec6adaf030 0x55ec6adb2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:25.928 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.919+0000 7fae18547b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec6adb8e40 0x55ec6adb31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:25.928 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.919+0000 7fae18547b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6adaa780 con 0x55ec6adb8e40 2023-09-09T23:21:25.928 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.919+0000 7fae18547b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6adaf9f0 con 0x55ec6adae710 2023-09-09T23:21:25.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.919+0000 7fae18547b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6ada63d0 con 0x55ec6adaf030 2023-09-09T23:21:25.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.919+0000 7fae0a91e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec6adb8e40 0x55ec6adb31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:25.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.919+0000 7fae0991c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec6adaf030 0x55ec6adb2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:25.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.919+0000 7fae0a11d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec6adae710 0x55ec6adb8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:25.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.919+0000 7fae0a91e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec6adb8e40 0x55ec6adb31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58062/0 (socket says 172.21.15.82:58062) 2023-09-09T23:21:25.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.919+0000 7fae0991c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec6adaf030 0x55ec6adb2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:56574/0 (socket says 172.21.15.82:56574) 2023-09-09T23:21:25.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.919+0000 7fae0a91e700 1 -- 172.21.15.82:0/2298610714 learned_addr learned my addr 172.21.15.82:0/2298610714 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:21:25.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.919+0000 7fae0a91e700 1 -- 172.21.15.82:0/2298610714 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec6adaf030 msgr2=0x55ec6adb2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:21:25.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.919+0000 7fae0a91e700 1 --2- 172.21.15.82:0/2298610714 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec6adaf030 0x55ec6adb2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.919+0000 7fae0a91e700 1 -- 172.21.15.82:0/2298610714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec6adae710 msgr2=0x55ec6adb8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:21:25.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.919+0000 7fae0a91e700 1 --2- 172.21.15.82:0/2298610714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec6adae710 0x55ec6adb8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.919+0000 7fae0a91e700 1 -- 172.21.15.82:0/2298610714 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0410cc20 con 0x55ec6adb8e40 2023-09-09T23:21:25.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.920+0000 7fae0a91e700 1 --2- 172.21.15.82:0/2298610714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec6adb8e40 0x55ec6adb31f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fadfc00ef00 tx=0x7fadfc0cb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91bb0ec9d8a6056b server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:21:25.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.920+0000 7fadf958a700 1 -- 172.21.15.82:0/2298610714 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadfc0996f0 con 0x55ec6adb8e40 2023-09-09T23:21:25.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.920+0000 7fadf958a700 1 -- 172.21.15.82:0/2298610714 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadfc00ce60 con 0x55ec6adb8e40 2023-09-09T23:21:25.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.920+0000 7fadf958a700 1 -- 172.21.15.82:0/2298610714 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadfc0c9480 con 0x55ec6adb8e40 2023-09-09T23:21:25.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.920+0000 7fae18547b80 1 -- 172.21.15.82:0/2298610714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec6adb8e40 msgr2=0x55ec6adb31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:21:25.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.920+0000 7fae18547b80 1 --2- 172.21.15.82:0/2298610714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec6adb8e40 0x55ec6adb31f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fadfc00ef00 tx=0x7fadfc0cb040 comp rx=0 tx=0).stop 2023-09-09T23:21:25.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.920+0000 7fae18547b80 1 -- 172.21.15.82:0/2298610714 shutdown_connections 2023-09-09T23:21:25.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.920+0000 7fae18547b80 1 --2- 172.21.15.82:0/2298610714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec6adae710 0x55ec6adb8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.933 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.920+0000 7fae18547b80 1 --2- 172.21.15.82:0/2298610714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec6adb8e40 0x55ec6adb31f0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.933 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.920+0000 7fae18547b80 1 --2- 172.21.15.82:0/2298610714 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec6adaf030 0x55ec6adb2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.933 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.920+0000 7fae18547b80 1 -- 172.21.15.82:0/2298610714 >> 172.21.15.82:0/2298610714 conn(0x55ec6ad23090 msgr2=0x55ec6adb0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:21:25.933 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.920+0000 7fae18547b80 1 -- 172.21.15.82:0/2298610714 shutdown_connections 2023-09-09T23:21:25.933 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.920+0000 7fae18547b80 1 -- 172.21.15.82:0/2298610714 wait complete. 2023-09-09T23:21:25.934 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.921+0000 7fae18547b80 1 -- start start 2023-09-09T23:21:25.934 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.921+0000 7fae18547b80 1 Processor -- start 2023-09-09T23:21:25.934 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.921+0000 7fae18547b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec6adae710 0x55ec6adbe180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:25.934 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.921+0000 7fae18547b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec6adaf030 0x55ec6adbe6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:25.934 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.921+0000 7fae18547b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec6adb8e40 0x55ec6adbec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:25.935 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.921+0000 7fae18547b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6ae1c8a0 con 0x55ec6adb8e40 2023-09-09T23:21:25.935 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.921+0000 7fae18547b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6ae1c9e0 con 0x55ec6adae710 2023-09-09T23:21:25.935 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.921+0000 7fae18547b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6ae1cba0 con 0x55ec6adaf030 2023-09-09T23:21:25.935 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.921+0000 7fae18547b80 10 client.0 _ll_register_callbacks cb 0x7fae00009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:21:25.935 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.921+0000 7fae0a91e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec6adb8e40 0x55ec6adbec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:25.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.921+0000 7fae0a11d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec6adae710 0x55ec6adbe180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:25.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.921+0000 7fae0a91e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec6adb8e40 0x55ec6adbec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58068/0 (socket says 172.21.15.82:58068) 2023-09-09T23:21:25.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.921+0000 7fae0a91e700 1 -- 172.21.15.82:0/3556493700 learned_addr learned my addr 172.21.15.82:0/3556493700 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:21:25.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.921+0000 7fae0991c700 1 --2- 172.21.15.82:0/3556493700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec6adaf030 0x55ec6adbe6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:25.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.921+0000 7fae0a91e700 1 -- 172.21.15.82:0/3556493700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec6adaf030 msgr2=0x55ec6adbe6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:21:25.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.921+0000 7fae0a91e700 1 --2- 172.21.15.82:0/3556493700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec6adaf030 0x55ec6adbe6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.921+0000 7fae0a91e700 1 -- 172.21.15.82:0/3556493700 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec6adae710 msgr2=0x55ec6adbe180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:21:25.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.921+0000 7fae0a91e700 1 --2- 172.21.15.82:0/3556493700 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec6adae710 0x55ec6adbe180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.921+0000 7fae0a91e700 1 -- 172.21.15.82:0/3556493700 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae0419aa90 con 0x55ec6adb8e40 2023-09-09T23:21:25.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.921+0000 7fae0a91e700 1 --2- 172.21.15.82:0/3556493700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec6adb8e40 0x55ec6adbec00 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fadfc098f30 tx=0x7fadfc0c91a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d778489d55f10389 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:21:25.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.921+0000 7fadedffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:21:25.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.921+0000 7fadedffb700 1 -- 172.21.15.82:0/3556493700 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadfc098ff0 con 0x55ec6adb8e40 2023-09-09T23:21:25.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.922+0000 7fadedffb700 1 -- 172.21.15.82:0/3556493700 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fadfc0996f0 con 0x55ec6adb8e40 2023-09-09T23:21:25.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.922+0000 7fae18547b80 1 -- 172.21.15.82:0/3556493700 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fadf401e040 con 0x55ec6adb8e40 2023-09-09T23:21:25.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.922+0000 7fadedffb700 1 -- 172.21.15.82:0/3556493700 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadfc0a0fd0 con 0x55ec6adb8e40 2023-09-09T23:21:25.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.922+0000 7fae18547b80 10 client.16197 fetch_fsmap learned FSMap version 86 2023-09-09T23:21:25.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.922+0000 7fae18547b80 1 -- 172.21.15.82:0/3556493700 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x55ec6ab15c60 con 0x55ec6adb8e40 2023-09-09T23:21:25.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.922+0000 7fadedffb700 1 -- 172.21.15.82:0/3556493700 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadf401e040 con 0x55ec6adb8e40 2023-09-09T23:21:25.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.922+0000 7fadedffb700 1 -- 172.21.15.82:0/3556493700 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 86) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fadfc098600 con 0x55ec6adb8e40 2023-09-09T23:21:25.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.922+0000 7fae18547b80 10 client.16197 fetch_fsmap finished waiting for FSMap version 86 2023-09-09T23:21:25.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.922+0000 7fae18547b80 10 client.16197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:21:25.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.922+0000 7fae18547b80 20 client.16197 populate_metadata read hostname 'smithi082' 2023-09-09T23:21:25.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.922+0000 7fae18547b80 1 --2- 172.21.15.82:0/3556493700 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55ec6ae1ded0 0x55ec6ae202c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:25.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.922+0000 7fae18547b80 4 client.16197 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:21:25.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.922+0000 7fae18547b80 1 -- 172.21.15.82:0/3556493700 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ec6acf9920 con 0x55ec6ae1ded0 2023-09-09T23:21:25.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.922+0000 7fae0a11d700 1 --2- 172.21.15.82:0/3556493700 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55ec6ae1ded0 0x55ec6ae202c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:25.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.923+0000 7fadedffb700 10 client.16197 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:21:25.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.923+0000 7fae0a11d700 1 --2- 172.21.15.82:0/3556493700 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55ec6ae1ded0 0x55ec6ae202c0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=179c2ecd3f1909c6 server_cookie=f28e61697904efef in_seq=0 out_seq=0 2023-09-09T23:21:25.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.923+0000 7fadedffb700 1 -- 172.21.15.82:0/3556493700 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55ec6acf9920 con 0x55ec6ae1ded0 2023-09-09T23:21:25.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.923+0000 7fadedffb700 10 client.16197 handle_command_reply: tid=0 2023-09-09T23:21:25.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.923+0000 7fae18547b80 1 client.16197 shutdown 2023-09-09T23:21:25.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.923+0000 7fae18547b80 1 -- 172.21.15.82:0/3556493700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec6adb8e40 msgr2=0x55ec6adbec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:21:25.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.923+0000 7fae18547b80 1 --2- 172.21.15.82:0/3556493700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec6adb8e40 0x55ec6adbec00 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fadfc098f30 tx=0x7fadfc0c91a0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.923+0000 7fae18547b80 1 -- 172.21.15.82:0/3556493700 shutdown_connections 2023-09-09T23:21:25.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.923+0000 7fae18547b80 1 --2- 172.21.15.82:0/3556493700 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55ec6ae1ded0 0x55ec6ae202c0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.923+0000 7fae18547b80 1 --2- 172.21.15.82:0/3556493700 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec6adae710 0x55ec6adbe180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.923+0000 7fadedffb700 0 client.16197 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:21:25.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.923+0000 7fae18547b80 1 --2- 172.21.15.82:0/3556493700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec6adb8e40 0x55ec6adbec00 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.923+0000 7fae18547b80 1 --2- 172.21.15.82:0/3556493700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec6adaf030 0x55ec6adbe6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.923+0000 7fae18547b80 1 -- 172.21.15.82:0/3556493700 >> 172.21.15.82:0/3556493700 conn(0x55ec6ad23090 msgr2=0x55ec6adb41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:21:25.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.923+0000 7fae18547b80 1 -- 172.21.15.82:0/3556493700 shutdown_connections 2023-09-09T23:21:25.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.923+0000 7fae18547b80 1 -- 172.21.15.82:0/3556493700 wait complete. 2023-09-09T23:21:25.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.923+0000 7fae18547b80 20 client.16197 trim_cache size 0 max 16384 2023-09-09T23:21:25.944 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:21:25.945 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:21:25.945 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:21:25.945 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:21:25.945 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:21:25.945 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:21:25.945 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:21:25.946 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:21:25.946 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:21:25.946 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:21:25.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.924+0000 7fae0c381700 1 -- 172.21.15.82:0/2569250474 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fadf00a07b0 msgr2=0x7fadf00a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:21:25.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.924+0000 7fae0c381700 1 --2- 172.21.15.82:0/2569250474 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fadf00a07b0 0x7fadf00a2c70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fadf400b2e0 tx=0x7fadf4005960 comp rx=0 tx=0).stop 2023-09-09T23:21:25.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.924+0000 7fae0c381700 1 -- 172.21.15.82:0/2569250474 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae041054f0 msgr2=0x7fae04199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:21:25.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.924+0000 7fae0c381700 1 --2- 172.21.15.82:0/2569250474 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae041054f0 0x7fae04199ea0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fadfc00ef30 tx=0x7fadfc00c3a0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.924+0000 7fae0c381700 1 -- 172.21.15.82:0/2569250474 shutdown_connections 2023-09-09T23:21:25.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.924+0000 7fae0c381700 1 --2- 172.21.15.82:0/2569250474 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fadf00a07b0 0x7fadf00a2c70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.924+0000 7fae0c381700 1 --2- 172.21.15.82:0/2569250474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae040ff450 0x7fae04195550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.924+0000 7fae0c381700 1 --2- 172.21.15.82:0/2569250474 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae041054f0 0x7fae04199ea0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.924+0000 7fae0c381700 1 --2- 172.21.15.82:0/2569250474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae040ffe10 0x7fae04195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:25.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.924+0000 7fae0c381700 1 -- 172.21.15.82:0/2569250474 >> 172.21.15.82:0/2569250474 conn(0x7fae040faf60 msgr2=0x7fae040fb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:21:25.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.924+0000 7fae0c381700 1 -- 172.21.15.82:0/2569250474 shutdown_connections 2023-09-09T23:21:25.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:25.924+0000 7fae0c381700 1 -- 172.21.15.82:0/2569250474 wait complete. 2023-09-09T23:21:25.950 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:21:25.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:25 smithi170 ceph-mon[102832]: pgmap v232: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 14 op/s 2023-09-09T23:21:27.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:27 smithi098 ceph-mon[103124]: pgmap v233: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 17 op/s 2023-09-09T23:21:27.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:27 smithi098 ceph-mon[103124]: 453 slow requests, 5 included below; oldest blocked for > 300.822525 secs 2023-09-09T23:21:27.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:27 smithi098 ceph-mon[103124]: slow request 300.719251 seconds old, received at 2023-09-09T23:16:25.969613+0000: peer_request:mds.1:1840 currently dispatched 2023-09-09T23:21:27.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:27 smithi098 ceph-mon[103124]: slow request 300.708248 seconds old, received at 2023-09-09T23:16:25.980616+0000: peer_request:mds.1:1872 currently dispatched 2023-09-09T23:21:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:27 smithi098 ceph-mon[103124]: slow request 300.694417 seconds old, received at 2023-09-09T23:16:25.994448+0000: peer_request:mds.1:1904 currently dispatched 2023-09-09T23:21:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:27 smithi098 ceph-mon[103124]: slow request 300.687289 seconds old, received at 2023-09-09T23:16:26.001575+0000: peer_request:mds.1:1936 currently dispatched 2023-09-09T23:21:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:27 smithi098 ceph-mon[103124]: slow request 300.674971 seconds old, received at 2023-09-09T23:16:26.013894+0000: peer_request:mds.1:1968 currently dispatched 2023-09-09T23:21:27.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:27 smithi170 ceph-mon[102832]: pgmap v233: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 17 op/s 2023-09-09T23:21:27.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:27 smithi170 ceph-mon[102832]: 453 slow requests, 5 included below; oldest blocked for > 300.822525 secs 2023-09-09T23:21:27.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:27 smithi170 ceph-mon[102832]: slow request 300.719251 seconds old, received at 2023-09-09T23:16:25.969613+0000: peer_request:mds.1:1840 currently dispatched 2023-09-09T23:21:27.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:27 smithi170 ceph-mon[102832]: slow request 300.708248 seconds old, received at 2023-09-09T23:16:25.980616+0000: peer_request:mds.1:1872 currently dispatched 2023-09-09T23:21:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:27 smithi170 ceph-mon[102832]: slow request 300.694417 seconds old, received at 2023-09-09T23:16:25.994448+0000: peer_request:mds.1:1904 currently dispatched 2023-09-09T23:21:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:27 smithi170 ceph-mon[102832]: slow request 300.687289 seconds old, received at 2023-09-09T23:16:26.001575+0000: peer_request:mds.1:1936 currently dispatched 2023-09-09T23:21:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:27 smithi170 ceph-mon[102832]: slow request 300.674971 seconds old, received at 2023-09-09T23:16:26.013894+0000: peer_request:mds.1:1968 currently dispatched 2023-09-09T23:21:27.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:27 smithi082 ceph-mon[98694]: pgmap v233: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 17 op/s 2023-09-09T23:21:27.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:27 smithi082 ceph-mon[98694]: 453 slow requests, 5 included below; oldest blocked for > 300.822525 secs 2023-09-09T23:21:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:27 smithi082 ceph-mon[98694]: slow request 300.719251 seconds old, received at 2023-09-09T23:16:25.969613+0000: peer_request:mds.1:1840 currently dispatched 2023-09-09T23:21:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:27 smithi082 ceph-mon[98694]: slow request 300.708248 seconds old, received at 2023-09-09T23:16:25.980616+0000: peer_request:mds.1:1872 currently dispatched 2023-09-09T23:21:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:27 smithi082 ceph-mon[98694]: slow request 300.694417 seconds old, received at 2023-09-09T23:16:25.994448+0000: peer_request:mds.1:1904 currently dispatched 2023-09-09T23:21:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:27 smithi082 ceph-mon[98694]: slow request 300.687289 seconds old, received at 2023-09-09T23:16:26.001575+0000: peer_request:mds.1:1936 currently dispatched 2023-09-09T23:21:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:27 smithi082 ceph-mon[98694]: slow request 300.674971 seconds old, received at 2023-09-09T23:16:26.013894+0000: peer_request:mds.1:1968 currently dispatched 2023-09-09T23:21:28.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:28 smithi098 ceph-mon[103124]: 220 slow requests, 5 included below; oldest blocked for > 313.187137 secs 2023-09-09T23:21:28.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:28 smithi098 ceph-mon[103124]: slow request 312.288381 seconds old, received at 2023-09-09T23:16:14.885243+0000: client_request(mds.1:2054 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:28.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:28 smithi098 ceph-mon[103124]: slow request 313.136301 seconds old, received at 2023-09-09T23:16:14.037323+0000: client_request(mds.1:1575 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:28.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:28 smithi098 ceph-mon[103124]: slow request 313.079135 seconds old, received at 2023-09-09T23:16:14.094489+0000: client_request(mds.1:1607 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:28 smithi098 ceph-mon[103124]: slow request 313.022282 seconds old, received at 2023-09-09T23:16:14.151342+0000: client_request(mds.1:1639 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:28 smithi098 ceph-mon[103124]: slow request 312.964980 seconds old, received at 2023-09-09T23:16:14.208645+0000: client_request(mds.1:1671 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:28.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:28 smithi170 ceph-mon[102832]: 220 slow requests, 5 included below; oldest blocked for > 313.187137 secs 2023-09-09T23:21:28.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:28 smithi170 ceph-mon[102832]: slow request 312.288381 seconds old, received at 2023-09-09T23:16:14.885243+0000: client_request(mds.1:2054 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:28.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:28 smithi170 ceph-mon[102832]: slow request 313.136301 seconds old, received at 2023-09-09T23:16:14.037323+0000: client_request(mds.1:1575 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:28.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:28 smithi170 ceph-mon[102832]: slow request 313.079135 seconds old, received at 2023-09-09T23:16:14.094489+0000: client_request(mds.1:1607 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:28.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:28 smithi170 ceph-mon[102832]: slow request 313.022282 seconds old, received at 2023-09-09T23:16:14.151342+0000: client_request(mds.1:1639 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:28 smithi170 ceph-mon[102832]: slow request 312.964980 seconds old, received at 2023-09-09T23:16:14.208645+0000: client_request(mds.1:1671 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:28.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:28 smithi082 ceph-mon[98694]: 220 slow requests, 5 included below; oldest blocked for > 313.187137 secs 2023-09-09T23:21:28.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:28 smithi082 ceph-mon[98694]: slow request 312.288381 seconds old, received at 2023-09-09T23:16:14.885243+0000: client_request(mds.1:2054 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:28.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:28 smithi082 ceph-mon[98694]: slow request 313.136301 seconds old, received at 2023-09-09T23:16:14.037323+0000: client_request(mds.1:1575 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:28.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:28 smithi082 ceph-mon[98694]: slow request 313.079135 seconds old, received at 2023-09-09T23:16:14.094489+0000: client_request(mds.1:1607 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:28.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:28 smithi082 ceph-mon[98694]: slow request 313.022282 seconds old, received at 2023-09-09T23:16:14.151342+0000: client_request(mds.1:1639 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:28.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:28 smithi082 ceph-mon[98694]: slow request 312.964980 seconds old, received at 2023-09-09T23:16:14.208645+0000: client_request(mds.1:1671 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:29.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:29 smithi098 ceph-mon[103124]: pgmap v234: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 12 op/s 2023-09-09T23:21:29.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:29 smithi170 ceph-mon[102832]: pgmap v234: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 12 op/s 2023-09-09T23:21:29.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:29 smithi082 ceph-mon[98694]: pgmap v234: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 12 op/s 2023-09-09T23:21:31.751 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:31 smithi098 ceph-mon[103124]: pgmap v235: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 26 KiB/s rd, 18 KiB/s wr, 16 op/s 2023-09-09T23:21:31.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:31.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:31.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:31 smithi170 ceph-mon[102832]: pgmap v235: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 26 KiB/s rd, 18 KiB/s wr, 16 op/s 2023-09-09T23:21:31.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:31.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:31.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:31 smithi082 ceph-mon[98694]: pgmap v235: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 26 KiB/s rd, 18 KiB/s wr, 16 op/s 2023-09-09T23:21:31.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:32.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:32.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:32.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:32 smithi098 ceph-mon[103124]: 439 slow requests, 5 included below; oldest blocked for > 305.818170 secs 2023-09-09T23:21:32.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:32 smithi098 ceph-mon[103124]: slow request 305.663003 seconds old, received at 2023-09-09T23:16:26.025924+0000: peer_request:mds.1:2000 currently dispatched 2023-09-09T23:21:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:32 smithi098 ceph-mon[103124]: slow request 305.651570 seconds old, received at 2023-09-09T23:16:26.037357+0000: peer_request:mds.1:2032 currently dispatched 2023-09-09T23:21:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:32 smithi098 ceph-mon[103124]: slow request 297.827477 seconds old, received at 2023-09-09T23:16:33.861450+0000: peer_request:mds.1:2064 currently dispatched 2023-09-09T23:21:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:32 smithi098 ceph-mon[103124]: slow request 305.809378 seconds old, received at 2023-09-09T23:16:25.879549+0000: peer_request:mds.1:1585 currently dispatched 2023-09-09T23:21:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:32 smithi098 ceph-mon[103124]: slow request 305.798325 seconds old, received at 2023-09-09T23:16:25.890602+0000: peer_request:mds.1:1617 currently dispatched 2023-09-09T23:21:32.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:32.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:32.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:32 smithi170 ceph-mon[102832]: 439 slow requests, 5 included below; oldest blocked for > 305.818170 secs 2023-09-09T23:21:32.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:32 smithi170 ceph-mon[102832]: slow request 305.663003 seconds old, received at 2023-09-09T23:16:26.025924+0000: peer_request:mds.1:2000 currently dispatched 2023-09-09T23:21:32.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:32 smithi170 ceph-mon[102832]: slow request 305.651570 seconds old, received at 2023-09-09T23:16:26.037357+0000: peer_request:mds.1:2032 currently dispatched 2023-09-09T23:21:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:32 smithi170 ceph-mon[102832]: slow request 297.827477 seconds old, received at 2023-09-09T23:16:33.861450+0000: peer_request:mds.1:2064 currently dispatched 2023-09-09T23:21:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:32 smithi170 ceph-mon[102832]: slow request 305.809378 seconds old, received at 2023-09-09T23:16:25.879549+0000: peer_request:mds.1:1585 currently dispatched 2023-09-09T23:21:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:32 smithi170 ceph-mon[102832]: slow request 305.798325 seconds old, received at 2023-09-09T23:16:25.890602+0000: peer_request:mds.1:1617 currently dispatched 2023-09-09T23:21:32.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:32.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:32 smithi082 ceph-mon[98694]: 439 slow requests, 5 included below; oldest blocked for > 305.818170 secs 2023-09-09T23:21:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:32 smithi082 ceph-mon[98694]: slow request 305.663003 seconds old, received at 2023-09-09T23:16:26.025924+0000: peer_request:mds.1:2000 currently dispatched 2023-09-09T23:21:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:32 smithi082 ceph-mon[98694]: slow request 305.651570 seconds old, received at 2023-09-09T23:16:26.037357+0000: peer_request:mds.1:2032 currently dispatched 2023-09-09T23:21:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:32 smithi082 ceph-mon[98694]: slow request 297.827477 seconds old, received at 2023-09-09T23:16:33.861450+0000: peer_request:mds.1:2064 currently dispatched 2023-09-09T23:21:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:32 smithi082 ceph-mon[98694]: slow request 305.809378 seconds old, received at 2023-09-09T23:16:25.879549+0000: peer_request:mds.1:1585 currently dispatched 2023-09-09T23:21:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:32 smithi082 ceph-mon[98694]: slow request 305.798325 seconds old, received at 2023-09-09T23:16:25.890602+0000: peer_request:mds.1:1617 currently dispatched 2023-09-09T23:21:33.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:33 smithi098 ceph-mon[103124]: 217 slow requests, 5 included below; oldest blocked for > 318.196301 secs 2023-09-09T23:21:33.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:33 smithi098 ceph-mon[103124]: slow request 317.945015 seconds old, received at 2023-09-09T23:16:14.265298+0000: client_request(mds.1:1703 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:33 smithi098 ceph-mon[103124]: slow request 317.884873 seconds old, received at 2023-09-09T23:16:14.325440+0000: client_request(mds.1:1735 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:33 smithi098 ceph-mon[103124]: slow request 317.826968 seconds old, received at 2023-09-09T23:16:14.383345+0000: client_request(mds.1:1767 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:33 smithi098 ceph-mon[103124]: slow request 317.768753 seconds old, received at 2023-09-09T23:16:14.441561+0000: client_request(mds.1:1799 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:33 smithi098 ceph-mon[103124]: slow request 317.711054 seconds old, received at 2023-09-09T23:16:14.499260+0000: client_request(mds.1:1831 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:33 smithi098 ceph-mon[103124]: pgmap v236: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-09-09T23:21:33.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:33 smithi170 ceph-mon[102832]: 217 slow requests, 5 included below; oldest blocked for > 318.196301 secs 2023-09-09T23:21:33.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:33 smithi170 ceph-mon[102832]: slow request 317.945015 seconds old, received at 2023-09-09T23:16:14.265298+0000: client_request(mds.1:1703 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:33.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:33 smithi170 ceph-mon[102832]: slow request 317.884873 seconds old, received at 2023-09-09T23:16:14.325440+0000: client_request(mds.1:1735 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:33 smithi170 ceph-mon[102832]: slow request 317.826968 seconds old, received at 2023-09-09T23:16:14.383345+0000: client_request(mds.1:1767 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:33 smithi170 ceph-mon[102832]: slow request 317.768753 seconds old, received at 2023-09-09T23:16:14.441561+0000: client_request(mds.1:1799 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:33 smithi170 ceph-mon[102832]: slow request 317.711054 seconds old, received at 2023-09-09T23:16:14.499260+0000: client_request(mds.1:1831 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:33 smithi170 ceph-mon[102832]: pgmap v236: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-09-09T23:21:33.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:33 smithi082 ceph-mon[98694]: 217 slow requests, 5 included below; oldest blocked for > 318.196301 secs 2023-09-09T23:21:33.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:33 smithi082 ceph-mon[98694]: slow request 317.945015 seconds old, received at 2023-09-09T23:16:14.265298+0000: client_request(mds.1:1703 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:33.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:33 smithi082 ceph-mon[98694]: slow request 317.884873 seconds old, received at 2023-09-09T23:16:14.325440+0000: client_request(mds.1:1735 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:33.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:33 smithi082 ceph-mon[98694]: slow request 317.826968 seconds old, received at 2023-09-09T23:16:14.383345+0000: client_request(mds.1:1767 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:33.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:33 smithi082 ceph-mon[98694]: slow request 317.768753 seconds old, received at 2023-09-09T23:16:14.441561+0000: client_request(mds.1:1799 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:33.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:33 smithi082 ceph-mon[98694]: slow request 317.711054 seconds old, received at 2023-09-09T23:16:14.499260+0000: client_request(mds.1:1831 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:33.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:33 smithi082 ceph-mon[98694]: pgmap v236: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-09-09T23:21:34.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:34 smithi098 ceph-mon[103124]: pgmap v237: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 10 op/s 2023-09-09T23:21:34.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:34 smithi170 ceph-mon[102832]: pgmap v237: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 10 op/s 2023-09-09T23:21:34.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:34 smithi082 ceph-mon[98694]: pgmap v237: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 10 op/s 2023-09-09T23:21:35.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:35 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:35.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:35 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:35.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:35 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:35.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:35 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:36.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:35 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:36.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:35 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:36.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:36.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:36.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:36 smithi170 ceph-mon[102832]: pgmap v238: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 20 KiB/s rd, 11 KiB/s wr, 14 op/s 2023-09-09T23:21:36.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:36.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:36.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:36 smithi082 ceph-mon[98694]: pgmap v238: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 20 KiB/s rd, 11 KiB/s wr, 14 op/s 2023-09-09T23:21:37.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:37.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:37.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:36 smithi098 ceph-mon[103124]: pgmap v238: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 20 KiB/s rd, 11 KiB/s wr, 14 op/s 2023-09-09T23:21:37.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:37 smithi170 ceph-mon[102832]: 425 slow requests, 5 included below; oldest blocked for > 310.812612 secs 2023-09-09T23:21:37.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:37 smithi170 ceph-mon[102832]: slow request 310.787515 seconds old, received at 2023-09-09T23:16:25.901470+0000: peer_request:mds.1:1649 currently dispatched 2023-09-09T23:21:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:37 smithi170 ceph-mon[102832]: slow request 310.776007 seconds old, received at 2023-09-09T23:16:25.912978+0000: peer_request:mds.1:1681 currently dispatched 2023-09-09T23:21:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:37 smithi170 ceph-mon[102832]: slow request 310.764534 seconds old, received at 2023-09-09T23:16:25.924451+0000: peer_request:mds.1:1713 currently dispatched 2023-09-09T23:21:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:37 smithi170 ceph-mon[102832]: slow request 310.752011 seconds old, received at 2023-09-09T23:16:25.936974+0000: peer_request:mds.1:1745 currently dispatched 2023-09-09T23:21:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:37 smithi170 ceph-mon[102832]: slow request 310.740896 seconds old, received at 2023-09-09T23:16:25.948089+0000: peer_request:mds.1:1777 currently dispatched 2023-09-09T23:21:37.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:37 smithi082 ceph-mon[98694]: 425 slow requests, 5 included below; oldest blocked for > 310.812612 secs 2023-09-09T23:21:37.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:37 smithi082 ceph-mon[98694]: slow request 310.787515 seconds old, received at 2023-09-09T23:16:25.901470+0000: peer_request:mds.1:1649 currently dispatched 2023-09-09T23:21:37.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:37 smithi082 ceph-mon[98694]: slow request 310.776007 seconds old, received at 2023-09-09T23:16:25.912978+0000: peer_request:mds.1:1681 currently dispatched 2023-09-09T23:21:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:37 smithi082 ceph-mon[98694]: slow request 310.764534 seconds old, received at 2023-09-09T23:16:25.924451+0000: peer_request:mds.1:1713 currently dispatched 2023-09-09T23:21:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:37 smithi082 ceph-mon[98694]: slow request 310.752011 seconds old, received at 2023-09-09T23:16:25.936974+0000: peer_request:mds.1:1745 currently dispatched 2023-09-09T23:21:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:37 smithi082 ceph-mon[98694]: slow request 310.740896 seconds old, received at 2023-09-09T23:16:25.948089+0000: peer_request:mds.1:1777 currently dispatched 2023-09-09T23:21:38.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:37 smithi098 ceph-mon[103124]: 425 slow requests, 5 included below; oldest blocked for > 310.812612 secs 2023-09-09T23:21:38.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:37 smithi098 ceph-mon[103124]: slow request 310.787515 seconds old, received at 2023-09-09T23:16:25.901470+0000: peer_request:mds.1:1649 currently dispatched 2023-09-09T23:21:38.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:37 smithi098 ceph-mon[103124]: slow request 310.776007 seconds old, received at 2023-09-09T23:16:25.912978+0000: peer_request:mds.1:1681 currently dispatched 2023-09-09T23:21:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:37 smithi098 ceph-mon[103124]: slow request 310.764534 seconds old, received at 2023-09-09T23:16:25.924451+0000: peer_request:mds.1:1713 currently dispatched 2023-09-09T23:21:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:37 smithi098 ceph-mon[103124]: slow request 310.752011 seconds old, received at 2023-09-09T23:16:25.936974+0000: peer_request:mds.1:1745 currently dispatched 2023-09-09T23:21:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:37 smithi098 ceph-mon[103124]: slow request 310.740896 seconds old, received at 2023-09-09T23:16:25.948089+0000: peer_request:mds.1:1777 currently dispatched 2023-09-09T23:21:38.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:38 smithi170 ceph-mon[102832]: 209 slow requests, 5 included below; oldest blocked for > 323.268942 secs 2023-09-09T23:21:38.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:38 smithi170 ceph-mon[102832]: slow request 322.756614 seconds old, received at 2023-09-09T23:16:14.553047+0000: client_request(mds.1:1863 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:38.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:38 smithi170 ceph-mon[102832]: slow request 322.703314 seconds old, received at 2023-09-09T23:16:14.606347+0000: client_request(mds.1:1895 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:38.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:38 smithi170 ceph-mon[102832]: slow request 322.649542 seconds old, received at 2023-09-09T23:16:14.660119+0000: client_request(mds.1:1927 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:38 smithi170 ceph-mon[102832]: slow request 322.595011 seconds old, received at 2023-09-09T23:16:14.714650+0000: client_request(mds.1:1959 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:38 smithi170 ceph-mon[102832]: slow request 322.536050 seconds old, received at 2023-09-09T23:16:14.773611+0000: client_request(mds.1:1991 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:38 smithi170 ceph-mon[102832]: pgmap v239: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 11 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2023-09-09T23:21:38.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:38 smithi082 ceph-mon[98694]: 209 slow requests, 5 included below; oldest blocked for > 323.268942 secs 2023-09-09T23:21:38.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:38 smithi082 ceph-mon[98694]: slow request 322.756614 seconds old, received at 2023-09-09T23:16:14.553047+0000: client_request(mds.1:1863 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:38.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:38 smithi082 ceph-mon[98694]: slow request 322.703314 seconds old, received at 2023-09-09T23:16:14.606347+0000: client_request(mds.1:1895 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:38.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:38 smithi082 ceph-mon[98694]: slow request 322.649542 seconds old, received at 2023-09-09T23:16:14.660119+0000: client_request(mds.1:1927 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:38.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:38 smithi082 ceph-mon[98694]: slow request 322.595011 seconds old, received at 2023-09-09T23:16:14.714650+0000: client_request(mds.1:1959 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:38.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:38 smithi082 ceph-mon[98694]: slow request 322.536050 seconds old, received at 2023-09-09T23:16:14.773611+0000: client_request(mds.1:1991 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:38.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:38 smithi082 ceph-mon[98694]: pgmap v239: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 11 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2023-09-09T23:21:39.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:38 smithi098 ceph-mon[103124]: 209 slow requests, 5 included below; oldest blocked for > 323.268942 secs 2023-09-09T23:21:39.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:38 smithi098 ceph-mon[103124]: slow request 322.756614 seconds old, received at 2023-09-09T23:16:14.553047+0000: client_request(mds.1:1863 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:39.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:38 smithi098 ceph-mon[103124]: slow request 322.703314 seconds old, received at 2023-09-09T23:16:14.606347+0000: client_request(mds.1:1895 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:38 smithi098 ceph-mon[103124]: slow request 322.649542 seconds old, received at 2023-09-09T23:16:14.660119+0000: client_request(mds.1:1927 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:38 smithi098 ceph-mon[103124]: slow request 322.595011 seconds old, received at 2023-09-09T23:16:14.714650+0000: client_request(mds.1:1959 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:38 smithi098 ceph-mon[103124]: slow request 322.536050 seconds old, received at 2023-09-09T23:16:14.773611+0000: client_request(mds.1:1991 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:38 smithi098 ceph-mon[103124]: pgmap v239: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 11 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2023-09-09T23:21:39.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:39 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:39.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:39 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:39.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:39 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:39.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:39 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:40.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:39 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:40.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:39 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:40.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:40.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:40.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:40 smithi170 ceph-mon[102832]: pgmap v240: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 12 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2023-09-09T23:21:40.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:40.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:40.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:40 smithi082 ceph-mon[98694]: pgmap v240: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 12 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2023-09-09T23:21:41.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:41.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:41.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:40 smithi098 ceph-mon[103124]: pgmap v240: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 12 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2023-09-09T23:21:42.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:41 smithi170 ceph-mon[102832]: 413 slow requests, 5 included below; oldest blocked for > 315.807224 secs 2023-09-09T23:21:42.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:41 smithi170 ceph-mon[102832]: slow request 315.730429 seconds old, received at 2023-09-09T23:16:25.958617+0000: peer_request:mds.1:1809 currently dispatched 2023-09-09T23:21:42.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:41 smithi170 ceph-mon[102832]: slow request 315.719028 seconds old, received at 2023-09-09T23:16:25.970019+0000: peer_request:mds.1:1841 currently dispatched 2023-09-09T23:21:42.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:41 smithi170 ceph-mon[102832]: slow request 315.708207 seconds old, received at 2023-09-09T23:16:25.980839+0000: peer_request:mds.1:1873 currently dispatched 2023-09-09T23:21:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:41 smithi170 ceph-mon[102832]: slow request 315.694588 seconds old, received at 2023-09-09T23:16:25.994459+0000: peer_request:mds.1:1905 currently dispatched 2023-09-09T23:21:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:41 smithi170 ceph-mon[102832]: slow request 315.687153 seconds old, received at 2023-09-09T23:16:26.001893+0000: peer_request:mds.1:1937 currently dispatched 2023-09-09T23:21:42.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:41 smithi082 ceph-mon[98694]: 413 slow requests, 5 included below; oldest blocked for > 315.807224 secs 2023-09-09T23:21:42.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:41 smithi082 ceph-mon[98694]: slow request 315.730429 seconds old, received at 2023-09-09T23:16:25.958617+0000: peer_request:mds.1:1809 currently dispatched 2023-09-09T23:21:42.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:41 smithi082 ceph-mon[98694]: slow request 315.719028 seconds old, received at 2023-09-09T23:16:25.970019+0000: peer_request:mds.1:1841 currently dispatched 2023-09-09T23:21:42.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:41 smithi082 ceph-mon[98694]: slow request 315.708207 seconds old, received at 2023-09-09T23:16:25.980839+0000: peer_request:mds.1:1873 currently dispatched 2023-09-09T23:21:42.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:41 smithi082 ceph-mon[98694]: slow request 315.694588 seconds old, received at 2023-09-09T23:16:25.994459+0000: peer_request:mds.1:1905 currently dispatched 2023-09-09T23:21:42.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:41 smithi082 ceph-mon[98694]: slow request 315.687153 seconds old, received at 2023-09-09T23:16:26.001893+0000: peer_request:mds.1:1937 currently dispatched 2023-09-09T23:21:42.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:41 smithi098 ceph-mon[103124]: 413 slow requests, 5 included below; oldest blocked for > 315.807224 secs 2023-09-09T23:21:42.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:41 smithi098 ceph-mon[103124]: slow request 315.730429 seconds old, received at 2023-09-09T23:16:25.958617+0000: peer_request:mds.1:1809 currently dispatched 2023-09-09T23:21:42.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:41 smithi098 ceph-mon[103124]: slow request 315.719028 seconds old, received at 2023-09-09T23:16:25.970019+0000: peer_request:mds.1:1841 currently dispatched 2023-09-09T23:21:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:41 smithi098 ceph-mon[103124]: slow request 315.708207 seconds old, received at 2023-09-09T23:16:25.980839+0000: peer_request:mds.1:1873 currently dispatched 2023-09-09T23:21:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:41 smithi098 ceph-mon[103124]: slow request 315.694588 seconds old, received at 2023-09-09T23:16:25.994459+0000: peer_request:mds.1:1905 currently dispatched 2023-09-09T23:21:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:41 smithi098 ceph-mon[103124]: slow request 315.687153 seconds old, received at 2023-09-09T23:16:26.001893+0000: peer_request:mds.1:1937 currently dispatched 2023-09-09T23:21:43.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:42 smithi170 ceph-mon[102832]: pgmap v241: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-09T23:21:43.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:42 smithi082 ceph-mon[98694]: pgmap v241: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-09T23:21:43.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:42 smithi098 ceph-mon[103124]: pgmap v241: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-09T23:21:44.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:43 smithi170 ceph-mon[102832]: 203 slow requests, 5 included below; oldest blocked for > 328.456050 secs 2023-09-09T23:21:44.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:43 smithi170 ceph-mon[102832]: slow request 327.692293 seconds old, received at 2023-09-09T23:16:14.832948+0000: client_request(mds.1:2023 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:44.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:43 smithi170 ceph-mon[102832]: slow request 327.638165 seconds old, received at 2023-09-09T23:16:14.887076+0000: client_request(mds.1:2055 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:44.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:43 smithi170 ceph-mon[102832]: slow request 328.428927 seconds old, received at 2023-09-09T23:16:14.096313+0000: client_request(mds.1:1608 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:43 smithi170 ceph-mon[102832]: slow request 328.372164 seconds old, received at 2023-09-09T23:16:14.153077+0000: client_request(mds.1:1640 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:43 smithi170 ceph-mon[102832]: slow request 328.314634 seconds old, received at 2023-09-09T23:16:14.210607+0000: client_request(mds.1:1672 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:44.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:43 smithi082 ceph-mon[98694]: 203 slow requests, 5 included below; oldest blocked for > 328.456050 secs 2023-09-09T23:21:44.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:43 smithi082 ceph-mon[98694]: slow request 327.692293 seconds old, received at 2023-09-09T23:16:14.832948+0000: client_request(mds.1:2023 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:44.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:43 smithi082 ceph-mon[98694]: slow request 327.638165 seconds old, received at 2023-09-09T23:16:14.887076+0000: client_request(mds.1:2055 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:44.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:43 smithi082 ceph-mon[98694]: slow request 328.428927 seconds old, received at 2023-09-09T23:16:14.096313+0000: client_request(mds.1:1608 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:44.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:43 smithi082 ceph-mon[98694]: slow request 328.372164 seconds old, received at 2023-09-09T23:16:14.153077+0000: client_request(mds.1:1640 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:44.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:43 smithi082 ceph-mon[98694]: slow request 328.314634 seconds old, received at 2023-09-09T23:16:14.210607+0000: client_request(mds.1:1672 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:44.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:43 smithi098 ceph-mon[103124]: 203 slow requests, 5 included below; oldest blocked for > 328.456050 secs 2023-09-09T23:21:44.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:43 smithi098 ceph-mon[103124]: slow request 327.692293 seconds old, received at 2023-09-09T23:16:14.832948+0000: client_request(mds.1:2023 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:44.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:43 smithi098 ceph-mon[103124]: slow request 327.638165 seconds old, received at 2023-09-09T23:16:14.887076+0000: client_request(mds.1:2055 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:43 smithi098 ceph-mon[103124]: slow request 328.428927 seconds old, received at 2023-09-09T23:16:14.096313+0000: client_request(mds.1:1608 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:43 smithi098 ceph-mon[103124]: slow request 328.372164 seconds old, received at 2023-09-09T23:16:14.153077+0000: client_request(mds.1:1640 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:43 smithi098 ceph-mon[103124]: slow request 328.314634 seconds old, received at 2023-09-09T23:16:14.210607+0000: client_request(mds.1:1672 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:45.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:44 smithi170 ceph-mon[102832]: pgmap v242: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2023-09-09T23:21:45.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:44 smithi082 ceph-mon[98694]: pgmap v242: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2023-09-09T23:21:45.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:44 smithi098 ceph-mon[103124]: pgmap v242: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2023-09-09T23:21:47.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:47.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:47.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:46 smithi170 ceph-mon[102832]: pgmap v243: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 6.5 KiB/s rd, 12 op/s 2023-09-09T23:21:47.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:46 smithi170 ceph-mon[102832]: 398 slow requests, 5 included below; oldest blocked for > 320.802091 secs 2023-09-09T23:21:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:46 smithi170 ceph-mon[102832]: slow request 320.674741 seconds old, received at 2023-09-09T23:16:26.014385+0000: peer_request:mds.1:1969 currently dispatched 2023-09-09T23:21:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:46 smithi170 ceph-mon[102832]: slow request 320.662850 seconds old, received at 2023-09-09T23:16:26.026275+0000: peer_request:mds.1:2001 currently dispatched 2023-09-09T23:21:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:46 smithi170 ceph-mon[102832]: slow request 320.651382 seconds old, received at 2023-09-09T23:16:26.037743+0000: peer_request:mds.1:2033 currently dispatched 2023-09-09T23:21:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:46 smithi170 ceph-mon[102832]: slow request 320.798136 seconds old, received at 2023-09-09T23:16:25.890989+0000: peer_request:mds.1:1618 currently dispatched 2023-09-09T23:21:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:46 smithi170 ceph-mon[102832]: slow request 320.787277 seconds old, received at 2023-09-09T23:16:25.901849+0000: peer_request:mds.1:1650 currently dispatched 2023-09-09T23:21:47.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:47.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:47.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:46 smithi082 ceph-mon[98694]: pgmap v243: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 6.5 KiB/s rd, 12 op/s 2023-09-09T23:21:47.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:46 smithi082 ceph-mon[98694]: 398 slow requests, 5 included below; oldest blocked for > 320.802091 secs 2023-09-09T23:21:47.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:46 smithi082 ceph-mon[98694]: slow request 320.674741 seconds old, received at 2023-09-09T23:16:26.014385+0000: peer_request:mds.1:1969 currently dispatched 2023-09-09T23:21:47.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:46 smithi082 ceph-mon[98694]: slow request 320.662850 seconds old, received at 2023-09-09T23:16:26.026275+0000: peer_request:mds.1:2001 currently dispatched 2023-09-09T23:21:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:46 smithi082 ceph-mon[98694]: slow request 320.651382 seconds old, received at 2023-09-09T23:16:26.037743+0000: peer_request:mds.1:2033 currently dispatched 2023-09-09T23:21:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:46 smithi082 ceph-mon[98694]: slow request 320.798136 seconds old, received at 2023-09-09T23:16:25.890989+0000: peer_request:mds.1:1618 currently dispatched 2023-09-09T23:21:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:46 smithi082 ceph-mon[98694]: slow request 320.787277 seconds old, received at 2023-09-09T23:16:25.901849+0000: peer_request:mds.1:1650 currently dispatched 2023-09-09T23:21:47.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:47.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:47.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:46 smithi098 ceph-mon[103124]: pgmap v243: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 6.5 KiB/s rd, 12 op/s 2023-09-09T23:21:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:46 smithi098 ceph-mon[103124]: 398 slow requests, 5 included below; oldest blocked for > 320.802091 secs 2023-09-09T23:21:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:46 smithi098 ceph-mon[103124]: slow request 320.674741 seconds old, received at 2023-09-09T23:16:26.014385+0000: peer_request:mds.1:1969 currently dispatched 2023-09-09T23:21:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:46 smithi098 ceph-mon[103124]: slow request 320.662850 seconds old, received at 2023-09-09T23:16:26.026275+0000: peer_request:mds.1:2001 currently dispatched 2023-09-09T23:21:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:46 smithi098 ceph-mon[103124]: slow request 320.651382 seconds old, received at 2023-09-09T23:16:26.037743+0000: peer_request:mds.1:2033 currently dispatched 2023-09-09T23:21:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:46 smithi098 ceph-mon[103124]: slow request 320.798136 seconds old, received at 2023-09-09T23:16:25.890989+0000: peer_request:mds.1:1618 currently dispatched 2023-09-09T23:21:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:46 smithi098 ceph-mon[103124]: slow request 320.787277 seconds old, received at 2023-09-09T23:16:25.901849+0000: peer_request:mds.1:1650 currently dispatched 2023-09-09T23:21:48.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:48 smithi170 ceph-mon[102832]: 195 slow requests, 5 included below; oldest blocked for > 333.482956 secs 2023-09-09T23:21:48.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:48 smithi170 ceph-mon[102832]: slow request 333.313676 seconds old, received at 2023-09-09T23:16:14.267279+0000: client_request(mds.1:1704 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:48.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:48 smithi170 ceph-mon[102832]: slow request 333.253658 seconds old, received at 2023-09-09T23:16:14.327298+0000: client_request(mds.1:1736 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:48.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:48 smithi170 ceph-mon[102832]: slow request 333.195788 seconds old, received at 2023-09-09T23:16:14.385167+0000: client_request(mds.1:1768 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:48 smithi170 ceph-mon[102832]: slow request 333.137376 seconds old, received at 2023-09-09T23:16:14.443579+0000: client_request(mds.1:1800 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:48 smithi170 ceph-mon[102832]: slow request 333.079728 seconds old, received at 2023-09-09T23:16:14.501227+0000: client_request(mds.1:1832 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:48.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:48 smithi082 ceph-mon[98694]: 195 slow requests, 5 included below; oldest blocked for > 333.482956 secs 2023-09-09T23:21:48.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:48 smithi082 ceph-mon[98694]: slow request 333.313676 seconds old, received at 2023-09-09T23:16:14.267279+0000: client_request(mds.1:1704 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:48.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:48 smithi082 ceph-mon[98694]: slow request 333.253658 seconds old, received at 2023-09-09T23:16:14.327298+0000: client_request(mds.1:1736 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:48.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:48 smithi082 ceph-mon[98694]: slow request 333.195788 seconds old, received at 2023-09-09T23:16:14.385167+0000: client_request(mds.1:1768 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:48.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:48 smithi082 ceph-mon[98694]: slow request 333.137376 seconds old, received at 2023-09-09T23:16:14.443579+0000: client_request(mds.1:1800 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:48.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:48 smithi082 ceph-mon[98694]: slow request 333.079728 seconds old, received at 2023-09-09T23:16:14.501227+0000: client_request(mds.1:1832 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:48.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:48 smithi098 ceph-mon[103124]: 195 slow requests, 5 included below; oldest blocked for > 333.482956 secs 2023-09-09T23:21:48.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:48 smithi098 ceph-mon[103124]: slow request 333.313676 seconds old, received at 2023-09-09T23:16:14.267279+0000: client_request(mds.1:1704 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:48.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:48 smithi098 ceph-mon[103124]: slow request 333.253658 seconds old, received at 2023-09-09T23:16:14.327298+0000: client_request(mds.1:1736 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:48 smithi098 ceph-mon[103124]: slow request 333.195788 seconds old, received at 2023-09-09T23:16:14.385167+0000: client_request(mds.1:1768 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:48 smithi098 ceph-mon[103124]: slow request 333.137376 seconds old, received at 2023-09-09T23:16:14.443579+0000: client_request(mds.1:1800 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:48 smithi098 ceph-mon[103124]: slow request 333.079728 seconds old, received at 2023-09-09T23:16:14.501227+0000: client_request(mds.1:1832 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:49.675 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:49 smithi170 ceph-mon[102832]: pgmap v244: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:21:49.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:49 smithi082 ceph-mon[98694]: pgmap v244: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:21:49.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:49 smithi098 ceph-mon[103124]: pgmap v244: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:21:51.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:51 smithi170 ceph-mon[102832]: pgmap v245: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-09-09T23:21:51.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:51.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:51.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:51 smithi082 ceph-mon[98694]: pgmap v245: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-09-09T23:21:51.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:51.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:51.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:51 smithi098 ceph-mon[103124]: pgmap v245: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-09-09T23:21:51.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:51.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:52.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:52.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:52.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:52 smithi082 ceph-mon[98694]: 385 slow requests, 5 included below; oldest blocked for > 325.796321 secs 2023-09-09T23:21:52.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:52 smithi082 ceph-mon[98694]: slow request 325.775904 seconds old, received at 2023-09-09T23:16:25.913283+0000: peer_request:mds.1:1682 currently dispatched 2023-09-09T23:21:52.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:52 smithi082 ceph-mon[98694]: slow request 325.764372 seconds old, received at 2023-09-09T23:16:25.924816+0000: peer_request:mds.1:1714 currently dispatched 2023-09-09T23:21:52.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:52 smithi082 ceph-mon[98694]: slow request 325.751776 seconds old, received at 2023-09-09T23:16:25.937411+0000: peer_request:mds.1:1746 currently dispatched 2023-09-09T23:21:52.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:52 smithi082 ceph-mon[98694]: slow request 325.740828 seconds old, received at 2023-09-09T23:16:25.948360+0000: peer_request:mds.1:1778 currently dispatched 2023-09-09T23:21:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:52 smithi082 ceph-mon[98694]: slow request 325.730186 seconds old, received at 2023-09-09T23:16:25.959002+0000: peer_request:mds.1:1810 currently dispatched 2023-09-09T23:21:52.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:52.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:52 smithi098 ceph-mon[103124]: 385 slow requests, 5 included below; oldest blocked for > 325.796321 secs 2023-09-09T23:21:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:52 smithi098 ceph-mon[103124]: slow request 325.775904 seconds old, received at 2023-09-09T23:16:25.913283+0000: peer_request:mds.1:1682 currently dispatched 2023-09-09T23:21:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:52 smithi098 ceph-mon[103124]: slow request 325.764372 seconds old, received at 2023-09-09T23:16:25.924816+0000: peer_request:mds.1:1714 currently dispatched 2023-09-09T23:21:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:52 smithi098 ceph-mon[103124]: slow request 325.751776 seconds old, received at 2023-09-09T23:16:25.937411+0000: peer_request:mds.1:1746 currently dispatched 2023-09-09T23:21:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:52 smithi098 ceph-mon[103124]: slow request 325.740828 seconds old, received at 2023-09-09T23:16:25.948360+0000: peer_request:mds.1:1778 currently dispatched 2023-09-09T23:21:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:52 smithi098 ceph-mon[103124]: slow request 325.730186 seconds old, received at 2023-09-09T23:16:25.959002+0000: peer_request:mds.1:1810 currently dispatched 2023-09-09T23:21:52.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:52.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:52.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:52 smithi170 ceph-mon[102832]: 385 slow requests, 5 included below; oldest blocked for > 325.796321 secs 2023-09-09T23:21:52.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:52 smithi170 ceph-mon[102832]: slow request 325.775904 seconds old, received at 2023-09-09T23:16:25.913283+0000: peer_request:mds.1:1682 currently dispatched 2023-09-09T23:21:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:52 smithi170 ceph-mon[102832]: slow request 325.764372 seconds old, received at 2023-09-09T23:16:25.924816+0000: peer_request:mds.1:1714 currently dispatched 2023-09-09T23:21:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:52 smithi170 ceph-mon[102832]: slow request 325.751776 seconds old, received at 2023-09-09T23:16:25.937411+0000: peer_request:mds.1:1746 currently dispatched 2023-09-09T23:21:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:52 smithi170 ceph-mon[102832]: slow request 325.740828 seconds old, received at 2023-09-09T23:16:25.948360+0000: peer_request:mds.1:1778 currently dispatched 2023-09-09T23:21:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:52 smithi170 ceph-mon[102832]: slow request 325.730186 seconds old, received at 2023-09-09T23:16:25.959002+0000: peer_request:mds.1:1810 currently dispatched 2023-09-09T23:21:53.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:53 smithi082 ceph-mon[98694]: pgmap v246: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:21:53.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:53 smithi082 ceph-mon[98694]: 189 slow requests, 5 included below; oldest blocked for > 338.487698 secs 2023-09-09T23:21:53.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:53 smithi082 ceph-mon[98694]: slow request 338.059948 seconds old, received at 2023-09-09T23:16:14.554650+0000: client_request(mds.1:1864 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:53.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:53 smithi082 ceph-mon[98694]: slow request 338.006449 seconds old, received at 2023-09-09T23:16:14.608149+0000: client_request(mds.1:1896 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:53.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:53 smithi082 ceph-mon[98694]: slow request 337.952677 seconds old, received at 2023-09-09T23:16:14.661921+0000: client_request(mds.1:1928 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:53 smithi082 ceph-mon[98694]: slow request 337.897357 seconds old, received at 2023-09-09T23:16:14.717241+0000: client_request(mds.1:1960 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:53 smithi082 ceph-mon[98694]: slow request 337.839209 seconds old, received at 2023-09-09T23:16:14.775389+0000: client_request(mds.1:1992 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:53.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:53 smithi098 ceph-mon[103124]: pgmap v246: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:21:53.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:53 smithi098 ceph-mon[103124]: 189 slow requests, 5 included below; oldest blocked for > 338.487698 secs 2023-09-09T23:21:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:53 smithi098 ceph-mon[103124]: slow request 338.059948 seconds old, received at 2023-09-09T23:16:14.554650+0000: client_request(mds.1:1864 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:53 smithi098 ceph-mon[103124]: slow request 338.006449 seconds old, received at 2023-09-09T23:16:14.608149+0000: client_request(mds.1:1896 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:53 smithi098 ceph-mon[103124]: slow request 337.952677 seconds old, received at 2023-09-09T23:16:14.661921+0000: client_request(mds.1:1928 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:53 smithi098 ceph-mon[103124]: slow request 337.897357 seconds old, received at 2023-09-09T23:16:14.717241+0000: client_request(mds.1:1960 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:53 smithi098 ceph-mon[103124]: slow request 337.839209 seconds old, received at 2023-09-09T23:16:14.775389+0000: client_request(mds.1:1992 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:53.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:53 smithi170 ceph-mon[102832]: pgmap v246: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:21:53.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:53 smithi170 ceph-mon[102832]: 189 slow requests, 5 included below; oldest blocked for > 338.487698 secs 2023-09-09T23:21:53.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:53 smithi170 ceph-mon[102832]: slow request 338.059948 seconds old, received at 2023-09-09T23:16:14.554650+0000: client_request(mds.1:1864 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:53.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:53 smithi170 ceph-mon[102832]: slow request 338.006449 seconds old, received at 2023-09-09T23:16:14.608149+0000: client_request(mds.1:1896 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:53 smithi170 ceph-mon[102832]: slow request 337.952677 seconds old, received at 2023-09-09T23:16:14.661921+0000: client_request(mds.1:1928 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:53 smithi170 ceph-mon[102832]: slow request 337.897357 seconds old, received at 2023-09-09T23:16:14.717241+0000: client_request(mds.1:1960 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:53 smithi170 ceph-mon[102832]: slow request 337.839209 seconds old, received at 2023-09-09T23:16:14.775389+0000: client_request(mds.1:1992 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:55.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:55 smithi082 ceph-mon[98694]: pgmap v247: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:21:55.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:55.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:55.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:55 smithi098 ceph-mon[103124]: pgmap v247: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:21:55.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:55.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:55.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:55 smithi170 ceph-mon[102832]: pgmap v247: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:21:55.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:21:55.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:55.952 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:21:56.092 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.094+0000 7f3dca430700 1 -- 172.21.15.82:0/1030310927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dc4105600 msgr2=0x7f3dc41059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:21:56.092 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.094+0000 7f3dca430700 1 --2- 172.21.15.82:0/1030310927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dc4105600 0x7f3dc41059e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3db80099c0 tx=0x7f3db8009cd0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.093 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.094+0000 7f3dca430700 1 -- 172.21.15.82:0/1030310927 shutdown_connections 2023-09-09T23:21:56.093 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.094+0000 7f3dca430700 1 --2- 172.21.15.82:0/1030310927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dc41005b0 0x7f3dc410c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.093 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.094+0000 7f3dca430700 1 --2- 172.21.15.82:0/1030310927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dc4105600 0x7f3dc41059e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.093 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.094+0000 7f3dca430700 1 --2- 172.21.15.82:0/1030310927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3dc40ffbf0 0x7f3dc4100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.094+0000 7f3dca430700 1 -- 172.21.15.82:0/1030310927 >> 172.21.15.82:0/1030310927 conn(0x7f3dc4076a70 msgr2=0x7f3dc40fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:21:56.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.094+0000 7f3dca430700 1 -- 172.21.15.82:0/1030310927 shutdown_connections 2023-09-09T23:21:56.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.094+0000 7f3dca430700 1 -- 172.21.15.82:0/1030310927 wait complete. 2023-09-09T23:21:56.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.095+0000 7f3dca430700 1 Processor -- start 2023-09-09T23:21:56.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.095+0000 7f3dca430700 1 -- start start 2023-09-09T23:21:56.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.095+0000 7f3dca430700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dc40ffbf0 0x7f3dc41955e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:56.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.095+0000 7f3dca430700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dc41005b0 0x7f3dc4195b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:56.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.095+0000 7f3dca430700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3dc4105600 0x7f3dc4199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:56.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.095+0000 7f3dca430700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc419a5c0 con 0x7f3dc41005b0 2023-09-09T23:21:56.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.095+0000 7f3dca430700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc419a730 con 0x7f3dc4105600 2023-09-09T23:21:56.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.095+0000 7f3dca430700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc419a8f0 con 0x7f3dc40ffbf0 2023-09-09T23:21:56.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.095+0000 7f3dc37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dc41005b0 0x7f3dc4195b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:56.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.095+0000 7f3dc3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dc40ffbf0 0x7f3dc41955e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:56.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.095+0000 7f3dc37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dc41005b0 0x7f3dc4195b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58076/0 (socket says 172.21.15.82:58076) 2023-09-09T23:21:56.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.095+0000 7f3dc37fe700 1 -- 172.21.15.82:0/957107089 learned_addr learned my addr 172.21.15.82:0/957107089 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:21:56.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.095+0000 7f3dc89cd700 1 --2- 172.21.15.82:0/957107089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3dc4105600 0x7f3dc4199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:56.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.096+0000 7f3dc37fe700 1 -- 172.21.15.82:0/957107089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dc40ffbf0 msgr2=0x7f3dc41955e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:21:56.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.096+0000 7f3dc37fe700 1 --2- 172.21.15.82:0/957107089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dc40ffbf0 0x7f3dc41955e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.096+0000 7f3dc37fe700 1 -- 172.21.15.82:0/957107089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3dc4105600 msgr2=0x7f3dc4199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:21:56.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.096+0000 7f3dc37fe700 1 --2- 172.21.15.82:0/957107089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3dc4105600 0x7f3dc4199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.096+0000 7f3dc37fe700 1 -- 172.21.15.82:0/957107089 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dbc009c40 con 0x7f3dc41005b0 2023-09-09T23:21:56.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.096+0000 7f3dc37fe700 1 --2- 172.21.15.82:0/957107089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dc41005b0 0x7f3dc4195b20 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3db400e9a0 tx=0x7f3db400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:21:56.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.096+0000 7f3dc17fa700 1 -- 172.21.15.82:0/957107089 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db4009790 con 0x7f3dc41005b0 2023-09-09T23:21:56.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.096+0000 7f3dca430700 1 -- 172.21.15.82:0/957107089 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db80095f0 con 0x7f3dc41005b0 2023-09-09T23:21:56.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.096+0000 7f3dc17fa700 1 -- 172.21.15.82:0/957107089 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3db400ce80 con 0x7f3dc41005b0 2023-09-09T23:21:56.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.096+0000 7f3dc17fa700 1 -- 172.21.15.82:0/957107089 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db4018680 con 0x7f3dc41005b0 2023-09-09T23:21:56.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.096+0000 7f3dca430700 1 -- 172.21.15.82:0/957107089 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc419adc0 con 0x7f3dc41005b0 2023-09-09T23:21:56.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.098+0000 7f3dc17fa700 1 -- 172.21.15.82:0/957107089 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f3db4010430 con 0x7f3dc41005b0 2023-09-09T23:21:56.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.098+0000 7f3dc17fa700 1 --2- 172.21.15.82:0/957107089 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f3dac0a07b0 0x7f3dac0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:56.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.099+0000 7f3dc17fa700 1 -- 172.21.15.82:0/957107089 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f3db4014070 con 0x7f3dc41005b0 2023-09-09T23:21:56.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.099+0000 7f3dc3fff700 1 --2- 172.21.15.82:0/957107089 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f3dac0a07b0 0x7f3dac0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:56.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.099+0000 7f3dc3fff700 1 --2- 172.21.15.82:0/957107089 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f3dac0a07b0 0x7f3dac0a2c70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009fd0 tx=0x7f3db8005fb0 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:21:56.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.100+0000 7f3dd65f6b80 1 Processor -- start 2023-09-09T23:21:56.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.100+0000 7f3dd65f6b80 1 -- start start 2023-09-09T23:21:56.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.100+0000 7f3dd65f6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a88e749710 0x55a88e749af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:56.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.100+0000 7f3dd65f6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a88e74a030 0x55a88e753900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:56.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.100+0000 7f3dd65f6b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a88e753e40 0x55a88e756230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:56.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.100+0000 7f3dd65f6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a88e6a2730 con 0x55a88e74a030 2023-09-09T23:21:56.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.100+0000 7f3dd65f6b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a88e6a0050 con 0x55a88e753e40 2023-09-09T23:21:56.110 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.100+0000 7f3dd65f6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a88e69b940 con 0x55a88e749710 2023-09-09T23:21:56.110 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.100+0000 7f3dc37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a88e74a030 0x55a88e753900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:56.110 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.100+0000 7f3dc37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a88e74a030 0x55a88e753900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58084/0 (socket says 172.21.15.82:58084) 2023-09-09T23:21:56.110 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.100+0000 7f3dc89cd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a88e753e40 0x55a88e756230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:56.110 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.100+0000 7f3dc37fe700 1 -- 172.21.15.82:0/1158564737 learned_addr learned my addr 172.21.15.82:0/1158564737 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:21:56.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.100+0000 7f3dc3fff700 1 --2- 172.21.15.82:0/1158564737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a88e749710 0x55a88e749af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:56.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.100+0000 7f3dc37fe700 1 -- 172.21.15.82:0/1158564737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a88e749710 msgr2=0x55a88e749af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:21:56.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.100+0000 7f3dc37fe700 1 --2- 172.21.15.82:0/1158564737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a88e749710 0x55a88e749af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.100+0000 7f3dc37fe700 1 -- 172.21.15.82:0/1158564737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a88e753e40 msgr2=0x55a88e756230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:21:56.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.100+0000 7f3dc37fe700 1 --2- 172.21.15.82:0/1158564737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a88e753e40 0x55a88e756230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.100+0000 7f3dc37fe700 1 -- 172.21.15.82:0/1158564737 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a88e4c7de0 con 0x55a88e74a030 2023-09-09T23:21:56.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.100+0000 7f3dc37fe700 1 --2- 172.21.15.82:0/1158564737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a88e74a030 0x55a88e753900 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f3db400c5b0 tx=0x7f3db4099da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4239ad89a87ad0c6 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:21:56.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.101+0000 7f3daa7fc700 1 -- 172.21.15.82:0/1158564737 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db409d450 con 0x55a88e74a030 2023-09-09T23:21:56.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.101+0000 7f3daa7fc700 1 -- 172.21.15.82:0/1158564737 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3db40a1690 con 0x55a88e74a030 2023-09-09T23:21:56.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.101+0000 7f3daa7fc700 1 -- 172.21.15.82:0/1158564737 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db4099880 con 0x55a88e74a030 2023-09-09T23:21:56.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.101+0000 7f3dd65f6b80 1 -- 172.21.15.82:0/1158564737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a88e74a030 msgr2=0x55a88e753900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:21:56.113 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.101+0000 7f3dd65f6b80 1 --2- 172.21.15.82:0/1158564737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a88e74a030 0x55a88e753900 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f3db400c5b0 tx=0x7f3db4099da0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.113 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.101+0000 7f3dd65f6b80 1 -- 172.21.15.82:0/1158564737 shutdown_connections 2023-09-09T23:21:56.113 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.101+0000 7f3dd65f6b80 1 --2- 172.21.15.82:0/1158564737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a88e753e40 0x55a88e756230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.113 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.101+0000 7f3dd65f6b80 1 --2- 172.21.15.82:0/1158564737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a88e749710 0x55a88e749af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.113 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.101+0000 7f3dd65f6b80 1 --2- 172.21.15.82:0/1158564737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a88e74a030 0x55a88e753900 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.101+0000 7f3dd65f6b80 1 -- 172.21.15.82:0/1158564737 >> 172.21.15.82:0/1158564737 conn(0x55a88e6be090 msgr2=0x55a88e744bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:21:56.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.101+0000 7f3dd65f6b80 1 -- 172.21.15.82:0/1158564737 shutdown_connections 2023-09-09T23:21:56.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.101+0000 7f3dd65f6b80 1 -- 172.21.15.82:0/1158564737 wait complete. 2023-09-09T23:21:56.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.102+0000 7f3dd65f6b80 1 -- start start 2023-09-09T23:21:56.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.102+0000 7f3dd65f6b80 1 Processor -- start 2023-09-09T23:21:56.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.102+0000 7f3dd65f6b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a88e749710 0x55a88e7538a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:56.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.102+0000 7f3dd65f6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a88e74a030 0x55a88e750b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:56.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.102+0000 7f3dd65f6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a88e753e40 0x55a88e751260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:56.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.102+0000 7f3dd65f6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a88e639b70 con 0x55a88e74a030 2023-09-09T23:21:56.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.102+0000 7f3dd65f6b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a88e639870 con 0x55a88e749710 2023-09-09T23:21:56.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.102+0000 7f3dd65f6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a88e51d920 con 0x55a88e753e40 2023-09-09T23:21:56.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.102+0000 7f3dd65f6b80 10 client.0 _ll_register_callbacks cb 0x55a88e583a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:21:56.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.102+0000 7f3dc37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a88e74a030 0x55a88e750b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:56.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.102+0000 7f3dc3fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a88e749710 0x55a88e7538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:56.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.102+0000 7f3dc37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a88e74a030 0x55a88e750b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58090/0 (socket says 172.21.15.82:58090) 2023-09-09T23:21:56.117 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.102+0000 7f3dc37fe700 1 -- 172.21.15.82:0/1350946769 learned_addr learned my addr 172.21.15.82:0/1350946769 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:21:56.117 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.102+0000 7f3dc89cd700 1 --2- 172.21.15.82:0/1350946769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a88e753e40 0x55a88e751260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:56.117 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.103+0000 7f3dc37fe700 1 -- 172.21.15.82:0/1350946769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a88e753e40 msgr2=0x55a88e751260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:21:56.117 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.103+0000 7f3dc37fe700 1 --2- 172.21.15.82:0/1350946769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a88e753e40 0x55a88e751260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.117 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.103+0000 7f3dc37fe700 1 -- 172.21.15.82:0/1350946769 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a88e749710 msgr2=0x55a88e7538a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:21:56.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.103+0000 7f3dc37fe700 1 --2- 172.21.15.82:0/1350946769 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a88e749710 0x55a88e7538a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.103+0000 7f3dc37fe700 1 -- 172.21.15.82:0/1350946769 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc419adc0 con 0x55a88e74a030 2023-09-09T23:21:56.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.103+0000 7f3dc3fff700 1 --2- 172.21.15.82:0/1350946769 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a88e749710 0x55a88e7538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:21:56.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.103+0000 7f3dc37fe700 1 --2- 172.21.15.82:0/1350946769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a88e74a030 0x55a88e750b20 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3db4096090 tx=0x7f3db4010940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2acb55e36bbaed75 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:21:56.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.103+0000 7f3da37fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:21:56.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.103+0000 7f3da37fe700 1 -- 172.21.15.82:0/1350946769 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db409d450 con 0x55a88e74a030 2023-09-09T23:21:56.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.103+0000 7f3da37fe700 1 -- 172.21.15.82:0/1350946769 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f3db40979c0 con 0x55a88e74a030 2023-09-09T23:21:56.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.103+0000 7f3dd65f6b80 1 -- 172.21.15.82:0/1350946769 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a88e5f34f0 con 0x55a88e74a030 2023-09-09T23:21:56.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.103+0000 7f3da37fe700 1 -- 172.21.15.82:0/1350946769 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3db40651a0 con 0x55a88e74a030 2023-09-09T23:21:56.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.103+0000 7f3dd65f6b80 10 client.16209 fetch_fsmap learned FSMap version 98 2023-09-09T23:21:56.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.103+0000 7f3dd65f6b80 1 -- 172.21.15.82:0/1350946769 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=4+}) v3 -- 0x55a88e4b0c60 con 0x55a88e74a030 2023-09-09T23:21:56.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.103+0000 7f3da37fe700 1 -- 172.21.15.82:0/1350946769 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a88e5f34f0 con 0x55a88e74a030 2023-09-09T23:21:56.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.103+0000 7f3da37fe700 1 -- 172.21.15.82:0/1350946769 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 98) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3db40a1270 con 0x55a88e74a030 2023-09-09T23:21:56.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.103+0000 7f3dd65f6b80 10 client.16209 fetch_fsmap finished waiting for FSMap version 98 2023-09-09T23:21:56.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.103+0000 7f3dd65f6b80 10 client.16209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:21:56.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.103+0000 7f3dd65f6b80 20 client.16209 populate_metadata read hostname 'smithi082' 2023-09-09T23:21:56.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.103+0000 7f3dd65f6b80 1 --2- 172.21.15.82:0/1350946769 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55a88e763540 0x55a88e765960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:56.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.104+0000 7f3dd65f6b80 4 client.16209 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:21:56.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.104+0000 7f3dd65f6b80 1 -- 172.21.15.82:0/1350946769 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a88e6b08a0 con 0x55a88e763540 2023-09-09T23:21:56.122 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.104+0000 7f3dc3fff700 1 --2- 172.21.15.82:0/1350946769 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55a88e763540 0x55a88e765960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:56.122 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.104+0000 7f3da37fe700 10 client.16209 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:21:56.122 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.104+0000 7f3dc3fff700 1 --2- 172.21.15.82:0/1350946769 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55a88e763540 0x55a88e765960 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10a113dcbbb9e5f2 server_cookie=81c703415a338473 in_seq=0 out_seq=0 2023-09-09T23:21:56.122 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.105+0000 7f3da37fe700 1 -- 172.21.15.82:0/1350946769 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a88e6b08a0 con 0x55a88e763540 2023-09-09T23:21:56.123 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.105+0000 7f3da37fe700 10 client.16209 handle_command_reply: tid=0 2023-09-09T23:21:56.123 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.105+0000 7f3dd65f6b80 1 client.16209 shutdown 2023-09-09T23:21:56.123 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.105+0000 7f3dd65f6b80 1 -- 172.21.15.82:0/1350946769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a88e74a030 msgr2=0x55a88e750b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:21:56.123 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.105+0000 7f3dd65f6b80 1 --2- 172.21.15.82:0/1350946769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a88e74a030 0x55a88e750b20 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3db4096090 tx=0x7f3db4010940 comp rx=0 tx=0).stop 2023-09-09T23:21:56.123 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.105+0000 7f3dd65f6b80 1 -- 172.21.15.82:0/1350946769 shutdown_connections 2023-09-09T23:21:56.124 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.105+0000 7f3dd65f6b80 1 --2- 172.21.15.82:0/1350946769 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55a88e763540 0x55a88e765960 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.124 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.105+0000 7f3dd65f6b80 1 --2- 172.21.15.82:0/1350946769 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a88e749710 0x55a88e7538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.124 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.105+0000 7f3dd65f6b80 1 --2- 172.21.15.82:0/1350946769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a88e753e40 0x55a88e751260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.124 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.105+0000 7f3dd65f6b80 1 --2- 172.21.15.82:0/1350946769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a88e74a030 0x55a88e750b20 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.124 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.105+0000 7f3da37fe700 0 client.16209 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:21:56.125 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.105+0000 7f3dd65f6b80 1 -- 172.21.15.82:0/1350946769 >> 172.21.15.82:0/1350946769 conn(0x55a88e6be090 msgr2=0x55a88e744780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:21:56.125 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.105+0000 7f3dd65f6b80 1 -- 172.21.15.82:0/1350946769 shutdown_connections 2023-09-09T23:21:56.125 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.105+0000 7f3dd65f6b80 1 -- 172.21.15.82:0/1350946769 wait complete. 2023-09-09T23:21:56.125 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.105+0000 7f3dd65f6b80 20 client.16209 trim_cache size 0 max 16384 2023-09-09T23:21:56.131 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.122+0000 7f3dd65f6b80 1 Processor -- start 2023-09-09T23:21:56.131 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.122+0000 7f3dd65f6b80 1 -- start start 2023-09-09T23:21:56.132 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.122+0000 7f3dd65f6b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a88e749710 0x55a88e74be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:56.132 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.122+0000 7f3dd65f6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a88e74a030 0x55a88e74c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:56.132 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.122+0000 7f3dd65f6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a88e753e40 0x55a88e750300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:56.132 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.122+0000 7f3dd65f6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a88e760380 con 0x55a88e753e40 2023-09-09T23:21:56.132 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.122+0000 7f3dd65f6b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a88e74a9c0 con 0x55a88e749710 2023-09-09T23:21:56.133 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.122+0000 7f3dd65f6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a88e73c400 con 0x55a88e74a030 2023-09-09T23:21:56.133 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.122+0000 7f3dc89cd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a88e753e40 0x55a88e750300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:56.133 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.122+0000 7f3dc3fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a88e749710 0x55a88e74be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:56.133 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.122+0000 7f3dc89cd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a88e753e40 0x55a88e750300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58100/0 (socket says 172.21.15.82:58100) 2023-09-09T23:21:56.133 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.122+0000 7f3dc37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a88e74a030 0x55a88e74c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:56.134 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.122+0000 7f3dc89cd700 1 -- 172.21.15.82:0/2634447958 learned_addr learned my addr 172.21.15.82:0/2634447958 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:21:56.134 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.123+0000 7f3dc3fff700 1 -- 172.21.15.82:0/2634447958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a88e74a030 msgr2=0x55a88e74c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:21:56.134 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.123+0000 7f3dc3fff700 1 --2- 172.21.15.82:0/2634447958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a88e74a030 0x55a88e74c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.134 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.123+0000 7f3dc3fff700 1 -- 172.21.15.82:0/2634447958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a88e753e40 msgr2=0x55a88e750300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:21:56.134 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.123+0000 7f3dc3fff700 1 --2- 172.21.15.82:0/2634447958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a88e753e40 0x55a88e750300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.135 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.123+0000 7f3dc3fff700 1 -- 172.21.15.82:0/2634447958 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a88e4ee250 con 0x55a88e749710 2023-09-09T23:21:56.135 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.123+0000 7f3dc37fe700 1 --2- 172.21.15.82:0/2634447958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a88e74a030 0x55a88e74c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:21:56.135 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.123+0000 7f3dc89cd700 1 --2- 172.21.15.82:0/2634447958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a88e753e40 0x55a88e750300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:21:56.135 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.123+0000 7f3dc3fff700 1 --2- 172.21.15.82:0/2634447958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a88e749710 0x55a88e74be40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3db800b580 tx=0x7f3db801f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f93d6f527bde0a3 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:21:56.135 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.123+0000 7f3daaffd700 1 -- 172.21.15.82:0/2634447958 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db802b070 con 0x55a88e749710 2023-09-09T23:21:56.136 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.123+0000 7f3daaffd700 1 -- 172.21.15.82:0/2634447958 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3db801e040 con 0x55a88e749710 2023-09-09T23:21:56.136 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.123+0000 7f3daaffd700 1 -- 172.21.15.82:0/2634447958 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db8025910 con 0x55a88e749710 2023-09-09T23:21:56.136 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.123+0000 7f3dd65f6b80 1 -- 172.21.15.82:0/2634447958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a88e749710 msgr2=0x55a88e74be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:21:56.136 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.123+0000 7f3dd65f6b80 1 --2- 172.21.15.82:0/2634447958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a88e749710 0x55a88e74be40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3db800b580 tx=0x7f3db801f000 comp rx=0 tx=0).stop 2023-09-09T23:21:56.136 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.124+0000 7f3dd65f6b80 1 -- 172.21.15.82:0/2634447958 shutdown_connections 2023-09-09T23:21:56.136 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.124+0000 7f3dd65f6b80 1 --2- 172.21.15.82:0/2634447958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a88e749710 0x55a88e74be40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.137 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.124+0000 7f3dd65f6b80 1 --2- 172.21.15.82:0/2634447958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a88e753e40 0x55a88e750300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.137 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.124+0000 7f3dd65f6b80 1 --2- 172.21.15.82:0/2634447958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a88e74a030 0x55a88e74c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.137 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.124+0000 7f3dd65f6b80 1 -- 172.21.15.82:0/2634447958 >> 172.21.15.82:0/2634447958 conn(0x55a88e6be090 msgr2=0x55a88e75f140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:21:56.137 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.124+0000 7f3dd65f6b80 1 -- 172.21.15.82:0/2634447958 shutdown_connections 2023-09-09T23:21:56.137 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.124+0000 7f3dd65f6b80 1 -- 172.21.15.82:0/2634447958 wait complete. 2023-09-09T23:21:56.138 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.124+0000 7f3dd65f6b80 1 -- start start 2023-09-09T23:21:56.138 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.124+0000 7f3dd65f6b80 1 Processor -- start 2023-09-09T23:21:56.138 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.124+0000 7f3dd65f6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a88e749710 0x55a88e75ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:56.138 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.124+0000 7f3dd65f6b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a88e74a030 0x55a88e758fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:56.138 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.124+0000 7f3dd65f6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a88e753e40 0x55a88e759660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:56.139 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.124+0000 7f3dd65f6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a88e759d30 con 0x55a88e753e40 2023-09-09T23:21:56.139 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.124+0000 7f3dd65f6b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a88e759ea0 con 0x55a88e74a030 2023-09-09T23:21:56.139 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.124+0000 7f3dd65f6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a88e7b73c0 con 0x55a88e749710 2023-09-09T23:21:56.139 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.124+0000 7f3dd65f6b80 10 client.0 _ll_register_callbacks cb 0x7f3dbc00e280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:21:56.139 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.124+0000 7f3dc89cd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a88e753e40 0x55a88e759660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:56.140 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.124+0000 7f3dc3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a88e749710 0x55a88e75ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:56.140 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.124+0000 7f3dc37fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a88e74a030 0x55a88e758fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:56.140 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.124+0000 7f3dc89cd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a88e753e40 0x55a88e759660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58106/0 (socket says 172.21.15.82:58106) 2023-09-09T23:21:56.140 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.124+0000 7f3dc89cd700 1 -- 172.21.15.82:0/3959576308 learned_addr learned my addr 172.21.15.82:0/3959576308 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:21:56.140 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.124+0000 7f3dc3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a88e749710 0x55a88e75ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:56616/0 (socket says 172.21.15.82:56616) 2023-09-09T23:21:56.141 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.125+0000 7f3dc89cd700 1 -- 172.21.15.82:0/3959576308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a88e749710 msgr2=0x55a88e75ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:21:56.141 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.125+0000 7f3dc89cd700 1 --2- 172.21.15.82:0/3959576308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a88e749710 0x55a88e75ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.141 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.125+0000 7f3dc89cd700 1 -- 172.21.15.82:0/3959576308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a88e74a030 msgr2=0x55a88e758fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:21:56.141 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.125+0000 7f3dc89cd700 1 --2- 172.21.15.82:0/3959576308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a88e74a030 0x55a88e758fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.141 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.125+0000 7f3dc89cd700 1 -- 172.21.15.82:0/3959576308 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a88e4b0c60 con 0x55a88e753e40 2023-09-09T23:21:56.142 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.125+0000 7f3dc3fff700 1 --2- 172.21.15.82:0/3959576308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a88e749710 0x55a88e75ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:21:56.142 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.125+0000 7f3dc89cd700 1 --2- 172.21.15.82:0/3959576308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a88e753e40 0x55a88e759660 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009550 tx=0x7f3dbc0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a743cbe20e5c4e server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:21:56.142 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.125+0000 7f3da37fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:21:56.142 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.125+0000 7f3da37fe700 1 -- 172.21.15.82:0/3959576308 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc01c070 con 0x55a88e753e40 2023-09-09T23:21:56.142 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.125+0000 7f3da37fe700 1 -- 172.21.15.82:0/3959576308 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f3dbc003a00 con 0x55a88e753e40 2023-09-09T23:21:56.142 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.125+0000 7f3dd65f6b80 1 -- 172.21.15.82:0/3959576308 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3db40a1690 con 0x55a88e753e40 2023-09-09T23:21:56.143 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.125+0000 7f3da37fe700 1 -- 172.21.15.82:0/3959576308 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3dbc00e040 con 0x55a88e753e40 2023-09-09T23:21:56.143 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.125+0000 7f3dd65f6b80 10 client.16215 fetch_fsmap learned FSMap version 98 2023-09-09T23:21:56.143 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.125+0000 7f3dd65f6b80 1 -- 172.21.15.82:0/3959576308 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=4+}) v3 -- 0x7f3dbc015580 con 0x55a88e753e40 2023-09-09T23:21:56.143 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.125+0000 7f3da37fe700 1 -- 172.21.15.82:0/3959576308 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3db40a1690 con 0x55a88e753e40 2023-09-09T23:21:56.143 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.125+0000 7f3da37fe700 1 -- 172.21.15.82:0/3959576308 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 98) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3dbc016490 con 0x55a88e753e40 2023-09-09T23:21:56.144 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.125+0000 7f3dd65f6b80 10 client.16215 fetch_fsmap finished waiting for FSMap version 98 2023-09-09T23:21:56.144 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.125+0000 7f3dd65f6b80 10 client.16215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:21:56.144 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.125+0000 7f3dd65f6b80 20 client.16215 populate_metadata read hostname 'smithi082' 2023-09-09T23:21:56.144 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.126+0000 7f3dd65f6b80 1 --2- 172.21.15.82:0/3959576308 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55a88e7b8bf0 0x55a88e7bafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:21:56.144 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.126+0000 7f3dd65f6b80 4 client.16215 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:21:56.145 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.126+0000 7f3dd65f6b80 1 -- 172.21.15.82:0/3959576308 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a88e694920 con 0x55a88e7b8bf0 2023-09-09T23:21:56.145 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.126+0000 7f3dc3fff700 1 --2- 172.21.15.82:0/3959576308 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55a88e7b8bf0 0x55a88e7bafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:21:56.145 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.126+0000 7f3da37fe700 10 client.16215 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:21:56.145 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.126+0000 7f3dc3fff700 1 --2- 172.21.15.82:0/3959576308 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55a88e7b8bf0 0x55a88e7bafe0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d000c52c0521e891 server_cookie=5feac747ddd8dbcc in_seq=0 out_seq=0 2023-09-09T23:21:56.145 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.126+0000 7f3da37fe700 1 -- 172.21.15.82:0/3959576308 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55a88e694920 con 0x55a88e7b8bf0 2023-09-09T23:21:56.146 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.126+0000 7f3da37fe700 10 client.16215 handle_command_reply: tid=0 2023-09-09T23:21:56.146 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.126+0000 7f3dd65f6b80 1 client.16215 shutdown 2023-09-09T23:21:56.146 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.126+0000 7f3dd65f6b80 1 -- 172.21.15.82:0/3959576308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a88e753e40 msgr2=0x55a88e759660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:21:56.146 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.126+0000 7f3dd65f6b80 1 --2- 172.21.15.82:0/3959576308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a88e753e40 0x55a88e759660 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009550 tx=0x7f3dbc0077e0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.146 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.126+0000 7f3dd65f6b80 1 -- 172.21.15.82:0/3959576308 shutdown_connections 2023-09-09T23:21:56.147 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.126+0000 7f3dd65f6b80 1 --2- 172.21.15.82:0/3959576308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a88e753e40 0x55a88e759660 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.147 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.126+0000 7f3dd65f6b80 1 --2- 172.21.15.82:0/3959576308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a88e749710 0x55a88e75ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.147 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.126+0000 7f3dd65f6b80 1 --2- 172.21.15.82:0/3959576308 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55a88e7b8bf0 0x55a88e7bafe0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.147 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.126+0000 7f3dd65f6b80 1 --2- 172.21.15.82:0/3959576308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a88e74a030 0x55a88e758fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.147 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.126+0000 7f3da37fe700 0 client.16215 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:21:56.147 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.126+0000 7f3dd65f6b80 1 -- 172.21.15.82:0/3959576308 >> 172.21.15.82:0/3959576308 conn(0x55a88e6be090 msgr2=0x55a88e74be40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:21:56.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.127+0000 7f3dd65f6b80 1 -- 172.21.15.82:0/3959576308 shutdown_connections 2023-09-09T23:21:56.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.127+0000 7f3dd65f6b80 1 -- 172.21.15.82:0/3959576308 wait complete. 2023-09-09T23:21:56.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.127+0000 7f3dd65f6b80 20 client.16215 trim_cache size 0 max 16384 2023-09-09T23:21:56.148 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:21:56.149 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:21:56.149 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:21:56.149 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:21:56.149 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:21:56.150 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:21:56.150 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:21:56.150 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:21:56.150 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:21:56.150 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:21:56.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.127+0000 7f3dca430700 1 -- 172.21.15.82:0/957107089 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f3dac0a07b0 msgr2=0x7f3dac0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:21:56.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.127+0000 7f3dca430700 1 --2- 172.21.15.82:0/957107089 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f3dac0a07b0 0x7f3dac0a2c70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009fd0 tx=0x7f3db8005fb0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.127+0000 7f3dca430700 1 -- 172.21.15.82:0/957107089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dc41005b0 msgr2=0x7f3dc4195b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:21:56.153 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.127+0000 7f3dca430700 1 --2- 172.21.15.82:0/957107089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dc41005b0 0x7f3dc4195b20 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3db400e9a0 tx=0x7f3db400ecb0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.153 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.127+0000 7f3dca430700 1 -- 172.21.15.82:0/957107089 shutdown_connections 2023-09-09T23:21:56.153 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.127+0000 7f3dca430700 1 --2- 172.21.15.82:0/957107089 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f3dac0a07b0 0x7f3dac0a2c70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.153 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.127+0000 7f3dca430700 1 --2- 172.21.15.82:0/957107089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3dc4105600 0x7f3dc4199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.153 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.127+0000 7f3dca430700 1 --2- 172.21.15.82:0/957107089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dc40ffbf0 0x7f3dc41955e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.127+0000 7f3dca430700 1 --2- 172.21.15.82:0/957107089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dc41005b0 0x7f3dc4195b20 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:21:56.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.127+0000 7f3dca430700 1 -- 172.21.15.82:0/957107089 >> 172.21.15.82:0/957107089 conn(0x7f3dc4076a70 msgr2=0x7f3dc40fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:21:56.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.127+0000 7f3dca430700 1 -- 172.21.15.82:0/957107089 shutdown_connections 2023-09-09T23:21:56.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:21:56.127+0000 7f3dca430700 1 -- 172.21.15.82:0/957107089 wait complete. 2023-09-09T23:21:56.155 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:21:56.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:56.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:56.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:56.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:56.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:21:56.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:21:57.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:57 smithi082 ceph-mon[98694]: pgmap v248: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.0 KiB/s wr, 13 op/s 2023-09-09T23:21:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:57 smithi082 ceph-mon[98694]: 369 slow requests, 5 included below; oldest blocked for > 330.790680 secs 2023-09-09T23:21:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:57 smithi082 ceph-mon[98694]: slow request 330.718974 seconds old, received at 2023-09-09T23:16:25.970315+0000: peer_request:mds.1:1842 currently dispatched 2023-09-09T23:21:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:57 smithi082 ceph-mon[98694]: slow request 330.708131 seconds old, received at 2023-09-09T23:16:25.981158+0000: peer_request:mds.1:1874 currently dispatched 2023-09-09T23:21:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:57 smithi082 ceph-mon[98694]: slow request 330.694827 seconds old, received at 2023-09-09T23:16:25.994462+0000: peer_request:mds.1:1906 currently dispatched 2023-09-09T23:21:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:57 smithi082 ceph-mon[98694]: slow request 330.687047 seconds old, received at 2023-09-09T23:16:26.002241+0000: peer_request:mds.1:1938 currently dispatched 2023-09-09T23:21:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:57 smithi082 ceph-mon[98694]: slow request 330.674624 seconds old, received at 2023-09-09T23:16:26.014665+0000: peer_request:mds.1:1970 currently dispatched 2023-09-09T23:21:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:57 smithi098 ceph-mon[103124]: pgmap v248: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.0 KiB/s wr, 13 op/s 2023-09-09T23:21:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:57 smithi098 ceph-mon[103124]: 369 slow requests, 5 included below; oldest blocked for > 330.790680 secs 2023-09-09T23:21:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:57 smithi098 ceph-mon[103124]: slow request 330.718974 seconds old, received at 2023-09-09T23:16:25.970315+0000: peer_request:mds.1:1842 currently dispatched 2023-09-09T23:21:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:57 smithi098 ceph-mon[103124]: slow request 330.708131 seconds old, received at 2023-09-09T23:16:25.981158+0000: peer_request:mds.1:1874 currently dispatched 2023-09-09T23:21:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:57 smithi098 ceph-mon[103124]: slow request 330.694827 seconds old, received at 2023-09-09T23:16:25.994462+0000: peer_request:mds.1:1906 currently dispatched 2023-09-09T23:21:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:57 smithi098 ceph-mon[103124]: slow request 330.687047 seconds old, received at 2023-09-09T23:16:26.002241+0000: peer_request:mds.1:1938 currently dispatched 2023-09-09T23:21:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:57 smithi098 ceph-mon[103124]: slow request 330.674624 seconds old, received at 2023-09-09T23:16:26.014665+0000: peer_request:mds.1:1970 currently dispatched 2023-09-09T23:21:57.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:57 smithi170 ceph-mon[102832]: pgmap v248: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.0 KiB/s wr, 13 op/s 2023-09-09T23:21:57.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:57 smithi170 ceph-mon[102832]: 369 slow requests, 5 included below; oldest blocked for > 330.790680 secs 2023-09-09T23:21:57.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:57 smithi170 ceph-mon[102832]: slow request 330.718974 seconds old, received at 2023-09-09T23:16:25.970315+0000: peer_request:mds.1:1842 currently dispatched 2023-09-09T23:21:57.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:57 smithi170 ceph-mon[102832]: slow request 330.708131 seconds old, received at 2023-09-09T23:16:25.981158+0000: peer_request:mds.1:1874 currently dispatched 2023-09-09T23:21:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:57 smithi170 ceph-mon[102832]: slow request 330.694827 seconds old, received at 2023-09-09T23:16:25.994462+0000: peer_request:mds.1:1906 currently dispatched 2023-09-09T23:21:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:57 smithi170 ceph-mon[102832]: slow request 330.687047 seconds old, received at 2023-09-09T23:16:26.002241+0000: peer_request:mds.1:1938 currently dispatched 2023-09-09T23:21:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:57 smithi170 ceph-mon[102832]: slow request 330.674624 seconds old, received at 2023-09-09T23:16:26.014665+0000: peer_request:mds.1:1970 currently dispatched 2023-09-09T23:21:58.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:58 smithi082 ceph-mon[98694]: 179 slow requests, 5 included below; oldest blocked for > 343.571342 secs 2023-09-09T23:21:58.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:58 smithi082 ceph-mon[98694]: slow request 342.894995 seconds old, received at 2023-09-09T23:16:14.834461+0000: client_request(mds.1:2024 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:58.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:58 smithi082 ceph-mon[98694]: slow request 343.517137 seconds old, received at 2023-09-09T23:16:14.212318+0000: client_request(mds.1:1673 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:58.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:58 smithi082 ceph-mon[98694]: slow request 343.459965 seconds old, received at 2023-09-09T23:16:14.269490+0000: client_request(mds.1:1705 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:58.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:58 smithi082 ceph-mon[98694]: slow request 343.400393 seconds old, received at 2023-09-09T23:16:14.329062+0000: client_request(mds.1:1737 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:58.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:58 smithi082 ceph-mon[98694]: slow request 343.342478 seconds old, received at 2023-09-09T23:16:14.386977+0000: client_request(mds.1:1769 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:58.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:58 smithi098 ceph-mon[103124]: 179 slow requests, 5 included below; oldest blocked for > 343.571342 secs 2023-09-09T23:21:58.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:58 smithi098 ceph-mon[103124]: slow request 342.894995 seconds old, received at 2023-09-09T23:16:14.834461+0000: client_request(mds.1:2024 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:58.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:58 smithi098 ceph-mon[103124]: slow request 343.517137 seconds old, received at 2023-09-09T23:16:14.212318+0000: client_request(mds.1:1673 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:58 smithi098 ceph-mon[103124]: slow request 343.459965 seconds old, received at 2023-09-09T23:16:14.269490+0000: client_request(mds.1:1705 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:58 smithi098 ceph-mon[103124]: slow request 343.400393 seconds old, received at 2023-09-09T23:16:14.329062+0000: client_request(mds.1:1737 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:58 smithi098 ceph-mon[103124]: slow request 343.342478 seconds old, received at 2023-09-09T23:16:14.386977+0000: client_request(mds.1:1769 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:58.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:58 smithi170 ceph-mon[102832]: 179 slow requests, 5 included below; oldest blocked for > 343.571342 secs 2023-09-09T23:21:58.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:58 smithi170 ceph-mon[102832]: slow request 342.894995 seconds old, received at 2023-09-09T23:16:14.834461+0000: client_request(mds.1:2024 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:58.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:58 smithi170 ceph-mon[102832]: slow request 343.517137 seconds old, received at 2023-09-09T23:16:14.212318+0000: client_request(mds.1:1673 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:58.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:58 smithi170 ceph-mon[102832]: slow request 343.459965 seconds old, received at 2023-09-09T23:16:14.269490+0000: client_request(mds.1:1705 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:58 smithi170 ceph-mon[102832]: slow request 343.400393 seconds old, received at 2023-09-09T23:16:14.329062+0000: client_request(mds.1:1737 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:58 smithi170 ceph-mon[102832]: slow request 343.342478 seconds old, received at 2023-09-09T23:16:14.386977+0000: client_request(mds.1:1769 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:21:59.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:21:59 smithi082 ceph-mon[98694]: pgmap v249: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2023-09-09T23:21:59.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:21:59 smithi098 ceph-mon[103124]: pgmap v249: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2023-09-09T23:21:59.925 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:21:59 smithi170 ceph-mon[102832]: pgmap v249: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2023-09-09T23:22:00.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:00.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:00.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:00.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:00.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:01.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:01 smithi098 ceph-mon[103124]: pgmap v250: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.0 KiB/s wr, 12 op/s 2023-09-09T23:22:01.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:01 smithi170 ceph-mon[102832]: pgmap v250: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.0 KiB/s wr, 12 op/s 2023-09-09T23:22:01.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:01 smithi082 ceph-mon[98694]: pgmap v250: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.0 KiB/s wr, 12 op/s 2023-09-09T23:22:02.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:02 smithi098 ceph-mon[103124]: 355 slow requests, 5 included below; oldest blocked for > 335.785114 secs 2023-09-09T23:22:02.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:02 smithi098 ceph-mon[103124]: slow request 335.662660 seconds old, received at 2023-09-09T23:16:26.026686+0000: peer_request:mds.1:2002 currently dispatched 2023-09-09T23:22:02.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:02 smithi098 ceph-mon[103124]: slow request 335.651211 seconds old, received at 2023-09-09T23:16:26.038135+0000: peer_request:mds.1:2034 currently dispatched 2023-09-09T23:22:02.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:02 smithi098 ceph-mon[103124]: slow request 335.775701 seconds old, received at 2023-09-09T23:16:25.913646+0000: peer_request:mds.1:1683 currently dispatched 2023-09-09T23:22:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:02 smithi098 ceph-mon[103124]: slow request 335.764158 seconds old, received at 2023-09-09T23:16:25.925189+0000: peer_request:mds.1:1715 currently dispatched 2023-09-09T23:22:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:02 smithi098 ceph-mon[103124]: slow request 335.751590 seconds old, received at 2023-09-09T23:16:25.937756+0000: peer_request:mds.1:1747 currently dispatched 2023-09-09T23:22:02.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:02 smithi170 ceph-mon[102832]: 355 slow requests, 5 included below; oldest blocked for > 335.785114 secs 2023-09-09T23:22:02.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:02 smithi170 ceph-mon[102832]: slow request 335.662660 seconds old, received at 2023-09-09T23:16:26.026686+0000: peer_request:mds.1:2002 currently dispatched 2023-09-09T23:22:02.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:02 smithi170 ceph-mon[102832]: slow request 335.651211 seconds old, received at 2023-09-09T23:16:26.038135+0000: peer_request:mds.1:2034 currently dispatched 2023-09-09T23:22:02.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:02 smithi170 ceph-mon[102832]: slow request 335.775701 seconds old, received at 2023-09-09T23:16:25.913646+0000: peer_request:mds.1:1683 currently dispatched 2023-09-09T23:22:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:02 smithi170 ceph-mon[102832]: slow request 335.764158 seconds old, received at 2023-09-09T23:16:25.925189+0000: peer_request:mds.1:1715 currently dispatched 2023-09-09T23:22:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:02 smithi170 ceph-mon[102832]: slow request 335.751590 seconds old, received at 2023-09-09T23:16:25.937756+0000: peer_request:mds.1:1747 currently dispatched 2023-09-09T23:22:02.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:02 smithi082 ceph-mon[98694]: 355 slow requests, 5 included below; oldest blocked for > 335.785114 secs 2023-09-09T23:22:02.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:02 smithi082 ceph-mon[98694]: slow request 335.662660 seconds old, received at 2023-09-09T23:16:26.026686+0000: peer_request:mds.1:2002 currently dispatched 2023-09-09T23:22:02.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:02 smithi082 ceph-mon[98694]: slow request 335.651211 seconds old, received at 2023-09-09T23:16:26.038135+0000: peer_request:mds.1:2034 currently dispatched 2023-09-09T23:22:02.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:02 smithi082 ceph-mon[98694]: slow request 335.775701 seconds old, received at 2023-09-09T23:16:25.913646+0000: peer_request:mds.1:1683 currently dispatched 2023-09-09T23:22:02.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:02 smithi082 ceph-mon[98694]: slow request 335.764158 seconds old, received at 2023-09-09T23:16:25.925189+0000: peer_request:mds.1:1715 currently dispatched 2023-09-09T23:22:02.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:02 smithi082 ceph-mon[98694]: slow request 335.751590 seconds old, received at 2023-09-09T23:16:25.937756+0000: peer_request:mds.1:1747 currently dispatched 2023-09-09T23:22:03.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:03 smithi098 ceph-mon[103124]: pgmap v251: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2023-09-09T23:22:03.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:03 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:03.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:03 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:03 smithi098 ceph-mon[103124]: 175 slow requests, 5 included below; oldest blocked for > 348.674355 secs 2023-09-09T23:22:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:03 smithi098 ceph-mon[103124]: slow request 348.418002 seconds old, received at 2023-09-09T23:16:14.445750+0000: client_request(mds.1:1801 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:03 smithi098 ceph-mon[103124]: slow request 348.360690 seconds old, received at 2023-09-09T23:16:14.503063+0000: client_request(mds.1:1833 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:03 smithi098 ceph-mon[103124]: slow request 348.307407 seconds old, received at 2023-09-09T23:16:14.556346+0000: client_request(mds.1:1865 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:03 smithi098 ceph-mon[103124]: slow request 348.254058 seconds old, received at 2023-09-09T23:16:14.609694+0000: client_request(mds.1:1897 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:03 smithi098 ceph-mon[103124]: slow request 348.200065 seconds old, received at 2023-09-09T23:16:14.663687+0000: client_request(mds.1:1929 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:03.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:03 smithi170 ceph-mon[102832]: pgmap v251: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2023-09-09T23:22:03.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:03 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:03.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:03 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:03.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:03 smithi170 ceph-mon[102832]: 175 slow requests, 5 included below; oldest blocked for > 348.674355 secs 2023-09-09T23:22:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:03 smithi170 ceph-mon[102832]: slow request 348.418002 seconds old, received at 2023-09-09T23:16:14.445750+0000: client_request(mds.1:1801 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:03 smithi170 ceph-mon[102832]: slow request 348.360690 seconds old, received at 2023-09-09T23:16:14.503063+0000: client_request(mds.1:1833 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:03 smithi170 ceph-mon[102832]: slow request 348.307407 seconds old, received at 2023-09-09T23:16:14.556346+0000: client_request(mds.1:1865 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:03 smithi170 ceph-mon[102832]: slow request 348.254058 seconds old, received at 2023-09-09T23:16:14.609694+0000: client_request(mds.1:1897 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:03 smithi170 ceph-mon[102832]: slow request 348.200065 seconds old, received at 2023-09-09T23:16:14.663687+0000: client_request(mds.1:1929 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:03.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:03 smithi082 ceph-mon[98694]: pgmap v251: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2023-09-09T23:22:03.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:03 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:03.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:03 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:03.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:03 smithi082 ceph-mon[98694]: 175 slow requests, 5 included below; oldest blocked for > 348.674355 secs 2023-09-09T23:22:03.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:03 smithi082 ceph-mon[98694]: slow request 348.418002 seconds old, received at 2023-09-09T23:16:14.445750+0000: client_request(mds.1:1801 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:03.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:03 smithi082 ceph-mon[98694]: slow request 348.360690 seconds old, received at 2023-09-09T23:16:14.503063+0000: client_request(mds.1:1833 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:03 smithi082 ceph-mon[98694]: slow request 348.307407 seconds old, received at 2023-09-09T23:16:14.556346+0000: client_request(mds.1:1865 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:03 smithi082 ceph-mon[98694]: slow request 348.254058 seconds old, received at 2023-09-09T23:16:14.609694+0000: client_request(mds.1:1897 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:03 smithi082 ceph-mon[98694]: slow request 348.200065 seconds old, received at 2023-09-09T23:16:14.663687+0000: client_request(mds.1:1929 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:04.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:04.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:04.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:04.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:04.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:04.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:05.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:05 smithi098 ceph-mon[103124]: pgmap v252: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2023-09-09T23:22:05.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:05 smithi170 ceph-mon[102832]: pgmap v252: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2023-09-09T23:22:05.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:05 smithi082 ceph-mon[98694]: pgmap v252: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2023-09-09T23:22:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:07 smithi098 ceph-mon[103124]: pgmap v253: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 22 KiB/s rd, 52 KiB/s wr, 28 op/s 2023-09-09T23:22:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:07 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:07 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:07 smithi098 ceph-mon[103124]: 341 slow requests, 5 included below; oldest blocked for > 340.778639 secs 2023-09-09T23:22:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:07 smithi098 ceph-mon[103124]: slow request 340.740728 seconds old, received at 2023-09-09T23:16:25.948732+0000: peer_request:mds.1:1779 currently dispatched 2023-09-09T23:22:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:07 smithi098 ceph-mon[103124]: slow request 340.730114 seconds old, received at 2023-09-09T23:16:25.959345+0000: peer_request:mds.1:1811 currently dispatched 2023-09-09T23:22:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:07 smithi098 ceph-mon[103124]: slow request 340.718730 seconds old, received at 2023-09-09T23:16:25.970730+0000: peer_request:mds.1:1843 currently dispatched 2023-09-09T23:22:07.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:07 smithi098 ceph-mon[103124]: slow request 340.708152 seconds old, received at 2023-09-09T23:16:25.981307+0000: peer_request:mds.1:1875 currently dispatched 2023-09-09T23:22:07.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:07 smithi098 ceph-mon[103124]: slow request 340.694994 seconds old, received at 2023-09-09T23:16:25.994465+0000: peer_request:mds.1:1907 currently dispatched 2023-09-09T23:22:07.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:07 smithi170 ceph-mon[102832]: pgmap v253: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 22 KiB/s rd, 52 KiB/s wr, 28 op/s 2023-09-09T23:22:07.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:07 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:07.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:07 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:07.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:07 smithi170 ceph-mon[102832]: 341 slow requests, 5 included below; oldest blocked for > 340.778639 secs 2023-09-09T23:22:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:07 smithi170 ceph-mon[102832]: slow request 340.740728 seconds old, received at 2023-09-09T23:16:25.948732+0000: peer_request:mds.1:1779 currently dispatched 2023-09-09T23:22:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:07 smithi170 ceph-mon[102832]: slow request 340.730114 seconds old, received at 2023-09-09T23:16:25.959345+0000: peer_request:mds.1:1811 currently dispatched 2023-09-09T23:22:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:07 smithi170 ceph-mon[102832]: slow request 340.718730 seconds old, received at 2023-09-09T23:16:25.970730+0000: peer_request:mds.1:1843 currently dispatched 2023-09-09T23:22:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:07 smithi170 ceph-mon[102832]: slow request 340.708152 seconds old, received at 2023-09-09T23:16:25.981307+0000: peer_request:mds.1:1875 currently dispatched 2023-09-09T23:22:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:07 smithi170 ceph-mon[102832]: slow request 340.694994 seconds old, received at 2023-09-09T23:16:25.994465+0000: peer_request:mds.1:1907 currently dispatched 2023-09-09T23:22:07.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:07 smithi082 ceph-mon[98694]: pgmap v253: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 22 KiB/s rd, 52 KiB/s wr, 28 op/s 2023-09-09T23:22:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:07 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:07 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:07 smithi082 ceph-mon[98694]: 341 slow requests, 5 included below; oldest blocked for > 340.778639 secs 2023-09-09T23:22:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:07 smithi082 ceph-mon[98694]: slow request 340.740728 seconds old, received at 2023-09-09T23:16:25.948732+0000: peer_request:mds.1:1779 currently dispatched 2023-09-09T23:22:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:07 smithi082 ceph-mon[98694]: slow request 340.730114 seconds old, received at 2023-09-09T23:16:25.959345+0000: peer_request:mds.1:1811 currently dispatched 2023-09-09T23:22:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:07 smithi082 ceph-mon[98694]: slow request 340.718730 seconds old, received at 2023-09-09T23:16:25.970730+0000: peer_request:mds.1:1843 currently dispatched 2023-09-09T23:22:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:07 smithi082 ceph-mon[98694]: slow request 340.708152 seconds old, received at 2023-09-09T23:16:25.981307+0000: peer_request:mds.1:1875 currently dispatched 2023-09-09T23:22:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:07 smithi082 ceph-mon[98694]: slow request 340.694994 seconds old, received at 2023-09-09T23:16:25.994465+0000: peer_request:mds.1:1907 currently dispatched 2023-09-09T23:22:08.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:08 smithi170 ceph-mon[102832]: osdmap e89: 12 total, 12 up, 12 in 2023-09-09T23:22:08.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:08 smithi170 ceph-mon[102832]: pgmap v255: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 28 op/s 2023-09-09T23:22:08.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:08 smithi082 ceph-mon[98694]: osdmap e89: 12 total, 12 up, 12 in 2023-09-09T23:22:08.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:08 smithi082 ceph-mon[98694]: pgmap v255: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 28 op/s 2023-09-09T23:22:09.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:08 smithi098 ceph-mon[103124]: osdmap e89: 12 total, 12 up, 12 in 2023-09-09T23:22:09.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:08 smithi098 ceph-mon[103124]: pgmap v255: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 28 op/s 2023-09-09T23:22:09.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:09 smithi170 ceph-mon[102832]: 161 slow requests, 5 included below; oldest blocked for > 353.788682 secs 2023-09-09T23:22:09.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:09 smithi170 ceph-mon[102832]: slow request 353.291184 seconds old, received at 2023-09-09T23:16:14.719462+0000: client_request(mds.1:1961 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:09.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:09 smithi170 ceph-mon[102832]: slow request 353.233325 seconds old, received at 2023-09-09T23:16:14.777321+0000: client_request(mds.1:1993 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:09.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:09 smithi170 ceph-mon[102832]: slow request 353.174392 seconds old, received at 2023-09-09T23:16:14.836254+0000: client_request(mds.1:2025 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:09 smithi170 ceph-mon[102832]: slow request 353.739258 seconds old, received at 2023-09-09T23:16:14.271388+0000: client_request(mds.1:1706 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:09 smithi170 ceph-mon[102832]: slow request 353.680027 seconds old, received at 2023-09-09T23:16:14.330619+0000: client_request(mds.1:1738 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:09.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:09 smithi082 ceph-mon[98694]: 161 slow requests, 5 included below; oldest blocked for > 353.788682 secs 2023-09-09T23:22:09.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:09 smithi082 ceph-mon[98694]: slow request 353.291184 seconds old, received at 2023-09-09T23:16:14.719462+0000: client_request(mds.1:1961 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:09 smithi082 ceph-mon[98694]: slow request 353.233325 seconds old, received at 2023-09-09T23:16:14.777321+0000: client_request(mds.1:1993 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:09 smithi082 ceph-mon[98694]: slow request 353.174392 seconds old, received at 2023-09-09T23:16:14.836254+0000: client_request(mds.1:2025 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:09 smithi082 ceph-mon[98694]: slow request 353.739258 seconds old, received at 2023-09-09T23:16:14.271388+0000: client_request(mds.1:1706 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:09 smithi082 ceph-mon[98694]: slow request 353.680027 seconds old, received at 2023-09-09T23:16:14.330619+0000: client_request(mds.1:1738 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:10.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:09 smithi098 ceph-mon[103124]: 161 slow requests, 5 included below; oldest blocked for > 353.788682 secs 2023-09-09T23:22:10.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:09 smithi098 ceph-mon[103124]: slow request 353.291184 seconds old, received at 2023-09-09T23:16:14.719462+0000: client_request(mds.1:1961 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:10.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:09 smithi098 ceph-mon[103124]: slow request 353.233325 seconds old, received at 2023-09-09T23:16:14.777321+0000: client_request(mds.1:1993 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:09 smithi098 ceph-mon[103124]: slow request 353.174392 seconds old, received at 2023-09-09T23:16:14.836254+0000: client_request(mds.1:2025 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:09 smithi098 ceph-mon[103124]: slow request 353.739258 seconds old, received at 2023-09-09T23:16:14.271388+0000: client_request(mds.1:1706 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:09 smithi098 ceph-mon[103124]: slow request 353.680027 seconds old, received at 2023-09-09T23:16:14.330619+0000: client_request(mds.1:1738 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:10.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:22:10.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:10 smithi170 ceph-mon[102832]: pgmap v256: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 28 op/s 2023-09-09T23:22:10.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:22:10.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:10 smithi082 ceph-mon[98694]: pgmap v256: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 28 op/s 2023-09-09T23:22:11.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:22:11.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:10 smithi098 ceph-mon[103124]: pgmap v256: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 28 op/s 2023-09-09T23:22:11.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:11 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:11.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:11 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:11.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:22:11.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:22:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:22:11.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:11 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:11.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:11 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:22:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:22:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:22:12.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:11 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:12.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:11 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:12.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:22:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:22:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:22:12.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:12.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:12.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:12 smithi170 ceph-mon[102832]: 323 slow requests, 5 included below; oldest blocked for > 345.772163 secs 2023-09-09T23:22:12.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:12 smithi170 ceph-mon[102832]: slow request 345.686911 seconds old, received at 2023-09-09T23:16:26.002605+0000: peer_request:mds.1:1939 currently dispatched 2023-09-09T23:22:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:12 smithi170 ceph-mon[102832]: slow request 345.674480 seconds old, received at 2023-09-09T23:16:26.015036+0000: peer_request:mds.1:1971 currently dispatched 2023-09-09T23:22:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:12 smithi170 ceph-mon[102832]: slow request 345.662417 seconds old, received at 2023-09-09T23:16:26.027098+0000: peer_request:mds.1:2003 currently dispatched 2023-09-09T23:22:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:12 smithi170 ceph-mon[102832]: slow request 345.651039 seconds old, received at 2023-09-09T23:16:26.038477+0000: peer_request:mds.1:2035 currently dispatched 2023-09-09T23:22:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:12 smithi170 ceph-mon[102832]: slow request 345.763920 seconds old, received at 2023-09-09T23:16:25.925596+0000: peer_request:mds.1:1716 currently dispatched 2023-09-09T23:22:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:12 smithi170 ceph-mon[102832]: pgmap v257: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 28 op/s 2023-09-09T23:22:12.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:12.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:12.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:12 smithi082 ceph-mon[98694]: 323 slow requests, 5 included below; oldest blocked for > 345.772163 secs 2023-09-09T23:22:12.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:12 smithi082 ceph-mon[98694]: slow request 345.686911 seconds old, received at 2023-09-09T23:16:26.002605+0000: peer_request:mds.1:1939 currently dispatched 2023-09-09T23:22:12.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:12 smithi082 ceph-mon[98694]: slow request 345.674480 seconds old, received at 2023-09-09T23:16:26.015036+0000: peer_request:mds.1:1971 currently dispatched 2023-09-09T23:22:12.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:12 smithi082 ceph-mon[98694]: slow request 345.662417 seconds old, received at 2023-09-09T23:16:26.027098+0000: peer_request:mds.1:2003 currently dispatched 2023-09-09T23:22:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:12 smithi082 ceph-mon[98694]: slow request 345.651039 seconds old, received at 2023-09-09T23:16:26.038477+0000: peer_request:mds.1:2035 currently dispatched 2023-09-09T23:22:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:12 smithi082 ceph-mon[98694]: slow request 345.763920 seconds old, received at 2023-09-09T23:16:25.925596+0000: peer_request:mds.1:1716 currently dispatched 2023-09-09T23:22:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:12 smithi082 ceph-mon[98694]: pgmap v257: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 28 op/s 2023-09-09T23:22:13.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:13.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:12 smithi098 ceph-mon[103124]: 323 slow requests, 5 included below; oldest blocked for > 345.772163 secs 2023-09-09T23:22:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:12 smithi098 ceph-mon[103124]: slow request 345.686911 seconds old, received at 2023-09-09T23:16:26.002605+0000: peer_request:mds.1:1939 currently dispatched 2023-09-09T23:22:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:12 smithi098 ceph-mon[103124]: slow request 345.674480 seconds old, received at 2023-09-09T23:16:26.015036+0000: peer_request:mds.1:1971 currently dispatched 2023-09-09T23:22:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:12 smithi098 ceph-mon[103124]: slow request 345.662417 seconds old, received at 2023-09-09T23:16:26.027098+0000: peer_request:mds.1:2003 currently dispatched 2023-09-09T23:22:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:12 smithi098 ceph-mon[103124]: slow request 345.651039 seconds old, received at 2023-09-09T23:16:26.038477+0000: peer_request:mds.1:2035 currently dispatched 2023-09-09T23:22:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:12 smithi098 ceph-mon[103124]: slow request 345.763920 seconds old, received at 2023-09-09T23:16:25.925596+0000: peer_request:mds.1:1716 currently dispatched 2023-09-09T23:22:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:12 smithi098 ceph-mon[103124]: pgmap v257: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 28 op/s 2023-09-09T23:22:14.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:14 smithi170 ceph-mon[102832]: 160 slow requests, 5 included below; oldest blocked for > 358.793710 secs 2023-09-09T23:22:14.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:14 smithi170 ceph-mon[102832]: slow request 358.661002 seconds old, received at 2023-09-09T23:16:14.388972+0000: client_request(mds.1:1770 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:14.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:14 smithi170 ceph-mon[102832]: slow request 358.602260 seconds old, received at 2023-09-09T23:16:14.447713+0000: client_request(mds.1:1802 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:14.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:14 smithi170 ceph-mon[102832]: slow request 358.545204 seconds old, received at 2023-09-09T23:16:14.504770+0000: client_request(mds.1:1834 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:14 smithi170 ceph-mon[102832]: slow request 358.491980 seconds old, received at 2023-09-09T23:16:14.557994+0000: client_request(mds.1:1866 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:14 smithi170 ceph-mon[102832]: slow request 358.438521 seconds old, received at 2023-09-09T23:16:14.611452+0000: client_request(mds.1:1898 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:14 smithi170 ceph-mon[102832]: osdmap e90: 12 total, 12 up, 12 in 2023-09-09T23:22:14.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:14 smithi082 ceph-mon[98694]: 160 slow requests, 5 included below; oldest blocked for > 358.793710 secs 2023-09-09T23:22:14.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:14 smithi082 ceph-mon[98694]: slow request 358.661002 seconds old, received at 2023-09-09T23:16:14.388972+0000: client_request(mds.1:1770 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:14.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:14 smithi082 ceph-mon[98694]: slow request 358.602260 seconds old, received at 2023-09-09T23:16:14.447713+0000: client_request(mds.1:1802 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:14.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:14 smithi082 ceph-mon[98694]: slow request 358.545204 seconds old, received at 2023-09-09T23:16:14.504770+0000: client_request(mds.1:1834 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:14.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:14 smithi082 ceph-mon[98694]: slow request 358.491980 seconds old, received at 2023-09-09T23:16:14.557994+0000: client_request(mds.1:1866 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:14.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:14 smithi082 ceph-mon[98694]: slow request 358.438521 seconds old, received at 2023-09-09T23:16:14.611452+0000: client_request(mds.1:1898 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:14 smithi082 ceph-mon[98694]: osdmap e90: 12 total, 12 up, 12 in 2023-09-09T23:22:14.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:14 smithi098 ceph-mon[103124]: 160 slow requests, 5 included below; oldest blocked for > 358.793710 secs 2023-09-09T23:22:14.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:14 smithi098 ceph-mon[103124]: slow request 358.661002 seconds old, received at 2023-09-09T23:16:14.388972+0000: client_request(mds.1:1770 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:14.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:14 smithi098 ceph-mon[103124]: slow request 358.602260 seconds old, received at 2023-09-09T23:16:14.447713+0000: client_request(mds.1:1802 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:14 smithi098 ceph-mon[103124]: slow request 358.545204 seconds old, received at 2023-09-09T23:16:14.504770+0000: client_request(mds.1:1834 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:14 smithi098 ceph-mon[103124]: slow request 358.491980 seconds old, received at 2023-09-09T23:16:14.557994+0000: client_request(mds.1:1866 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:14 smithi098 ceph-mon[103124]: slow request 358.438521 seconds old, received at 2023-09-09T23:16:14.611452+0000: client_request(mds.1:1898 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:14 smithi098 ceph-mon[103124]: osdmap e90: 12 total, 12 up, 12 in 2023-09-09T23:22:15.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:15 smithi082 ceph-mon[98694]: pgmap v259: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-09T23:22:15.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:15 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:15.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:15 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:15.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:15 smithi098 ceph-mon[103124]: pgmap v259: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-09T23:22:15.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:15 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:15.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:15 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:15.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:15 smithi170 ceph-mon[102832]: pgmap v259: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-09T23:22:15.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:15 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:15.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:15 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:16.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:16.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:16.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:16.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:16.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:16.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:17.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:17 smithi082 ceph-mon[98694]: pgmap v260: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.9 KiB/s wr, 13 op/s 2023-09-09T23:22:17.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:17 smithi082 ceph-mon[98694]: 307 slow requests, 5 included below; oldest blocked for > 350.765126 secs 2023-09-09T23:22:17.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:17 smithi082 ceph-mon[98694]: slow request 350.751411 seconds old, received at 2023-09-09T23:16:25.938166+0000: peer_request:mds.1:1748 currently dispatched 2023-09-09T23:22:17.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:17 smithi082 ceph-mon[98694]: slow request 350.740474 seconds old, received at 2023-09-09T23:16:25.949103+0000: peer_request:mds.1:1780 currently dispatched 2023-09-09T23:22:17.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:17 smithi082 ceph-mon[98694]: slow request 350.729889 seconds old, received at 2023-09-09T23:16:25.959687+0000: peer_request:mds.1:1812 currently dispatched 2023-09-09T23:22:17.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:17 smithi082 ceph-mon[98694]: slow request 350.718623 seconds old, received at 2023-09-09T23:16:25.970954+0000: peer_request:mds.1:1844 currently dispatched 2023-09-09T23:22:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:17 smithi082 ceph-mon[98694]: slow request 350.707395 seconds old, received at 2023-09-09T23:16:25.982182+0000: peer_request:mds.1:1876 currently dispatched 2023-09-09T23:22:17.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:17 smithi098 ceph-mon[103124]: pgmap v260: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.9 KiB/s wr, 13 op/s 2023-09-09T23:22:17.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:17 smithi098 ceph-mon[103124]: 307 slow requests, 5 included below; oldest blocked for > 350.765126 secs 2023-09-09T23:22:17.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:17 smithi098 ceph-mon[103124]: slow request 350.751411 seconds old, received at 2023-09-09T23:16:25.938166+0000: peer_request:mds.1:1748 currently dispatched 2023-09-09T23:22:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:17 smithi098 ceph-mon[103124]: slow request 350.740474 seconds old, received at 2023-09-09T23:16:25.949103+0000: peer_request:mds.1:1780 currently dispatched 2023-09-09T23:22:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:17 smithi098 ceph-mon[103124]: slow request 350.729889 seconds old, received at 2023-09-09T23:16:25.959687+0000: peer_request:mds.1:1812 currently dispatched 2023-09-09T23:22:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:17 smithi098 ceph-mon[103124]: slow request 350.718623 seconds old, received at 2023-09-09T23:16:25.970954+0000: peer_request:mds.1:1844 currently dispatched 2023-09-09T23:22:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:17 smithi098 ceph-mon[103124]: slow request 350.707395 seconds old, received at 2023-09-09T23:16:25.982182+0000: peer_request:mds.1:1876 currently dispatched 2023-09-09T23:22:17.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:17 smithi170 ceph-mon[102832]: pgmap v260: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.9 KiB/s wr, 13 op/s 2023-09-09T23:22:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:17 smithi170 ceph-mon[102832]: 307 slow requests, 5 included below; oldest blocked for > 350.765126 secs 2023-09-09T23:22:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:17 smithi170 ceph-mon[102832]: slow request 350.751411 seconds old, received at 2023-09-09T23:16:25.938166+0000: peer_request:mds.1:1748 currently dispatched 2023-09-09T23:22:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:17 smithi170 ceph-mon[102832]: slow request 350.740474 seconds old, received at 2023-09-09T23:16:25.949103+0000: peer_request:mds.1:1780 currently dispatched 2023-09-09T23:22:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:17 smithi170 ceph-mon[102832]: slow request 350.729889 seconds old, received at 2023-09-09T23:16:25.959687+0000: peer_request:mds.1:1812 currently dispatched 2023-09-09T23:22:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:17 smithi170 ceph-mon[102832]: slow request 350.718623 seconds old, received at 2023-09-09T23:16:25.970954+0000: peer_request:mds.1:1844 currently dispatched 2023-09-09T23:22:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:17 smithi170 ceph-mon[102832]: slow request 350.707395 seconds old, received at 2023-09-09T23:16:25.982182+0000: peer_request:mds.1:1876 currently dispatched 2023-09-09T23:22:19.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:19 smithi082 ceph-mon[98694]: 147 slow requests, 5 included below; oldest blocked for > 363.805095 secs 2023-09-09T23:22:19.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:19 smithi082 ceph-mon[98694]: slow request 363.786404 seconds old, received at 2023-09-09T23:16:14.313689+0000: client_request(mds.1:1729 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:19.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:19 smithi082 ceph-mon[98694]: slow request 363.728113 seconds old, received at 2023-09-09T23:16:14.371980+0000: client_request(mds.1:1761 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:19.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:19 smithi082 ceph-mon[98694]: slow request 363.670283 seconds old, received at 2023-09-09T23:16:14.429810+0000: client_request(mds.1:1793 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:19.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:19 smithi082 ceph-mon[98694]: slow request 363.611190 seconds old, received at 2023-09-09T23:16:14.488904+0000: client_request(mds.1:1825 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:19 smithi082 ceph-mon[98694]: slow request 363.556765 seconds old, received at 2023-09-09T23:16:14.543328+0000: client_request(mds.1:1857 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:19 smithi082 ceph-mon[98694]: pgmap v261: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2023-09-09T23:22:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:19 smithi098 ceph-mon[103124]: 147 slow requests, 5 included below; oldest blocked for > 363.805095 secs 2023-09-09T23:22:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:19 smithi098 ceph-mon[103124]: slow request 363.786404 seconds old, received at 2023-09-09T23:16:14.313689+0000: client_request(mds.1:1729 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:19 smithi098 ceph-mon[103124]: slow request 363.728113 seconds old, received at 2023-09-09T23:16:14.371980+0000: client_request(mds.1:1761 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:19.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:19 smithi098 ceph-mon[103124]: slow request 363.670283 seconds old, received at 2023-09-09T23:16:14.429810+0000: client_request(mds.1:1793 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:19.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:19 smithi098 ceph-mon[103124]: slow request 363.611190 seconds old, received at 2023-09-09T23:16:14.488904+0000: client_request(mds.1:1825 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:19.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:19 smithi098 ceph-mon[103124]: slow request 363.556765 seconds old, received at 2023-09-09T23:16:14.543328+0000: client_request(mds.1:1857 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:19.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:19 smithi098 ceph-mon[103124]: pgmap v261: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2023-09-09T23:22:19.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:19 smithi170 ceph-mon[102832]: 147 slow requests, 5 included below; oldest blocked for > 363.805095 secs 2023-09-09T23:22:19.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:19 smithi170 ceph-mon[102832]: slow request 363.786404 seconds old, received at 2023-09-09T23:16:14.313689+0000: client_request(mds.1:1729 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:19 smithi170 ceph-mon[102832]: slow request 363.728113 seconds old, received at 2023-09-09T23:16:14.371980+0000: client_request(mds.1:1761 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:19 smithi170 ceph-mon[102832]: slow request 363.670283 seconds old, received at 2023-09-09T23:16:14.429810+0000: client_request(mds.1:1793 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:19 smithi170 ceph-mon[102832]: slow request 363.611190 seconds old, received at 2023-09-09T23:16:14.488904+0000: client_request(mds.1:1825 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:19 smithi170 ceph-mon[102832]: slow request 363.556765 seconds old, received at 2023-09-09T23:16:14.543328+0000: client_request(mds.1:1857 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:19 smithi170 ceph-mon[102832]: pgmap v261: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2023-09-09T23:22:20.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:20.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:20.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:20.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:20.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:20.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:21.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:21 smithi082 ceph-mon[98694]: pgmap v262: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.6 KiB/s wr, 13 op/s 2023-09-09T23:22:21.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:21 smithi098 ceph-mon[103124]: pgmap v262: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.6 KiB/s wr, 13 op/s 2023-09-09T23:22:21.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:21 smithi170 ceph-mon[102832]: pgmap v262: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.6 KiB/s wr, 13 op/s 2023-09-09T23:22:22.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:22 smithi098 ceph-mon[103124]: 285 slow requests, 5 included below; oldest blocked for > 355.756389 secs 2023-09-09T23:22:22.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:22 smithi098 ceph-mon[103124]: slow request 355.695165 seconds old, received at 2023-09-09T23:16:25.994472+0000: peer_request:mds.1:1908 currently dispatched 2023-09-09T23:22:22.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:22 smithi098 ceph-mon[103124]: slow request 355.686716 seconds old, received at 2023-09-09T23:16:26.002921+0000: peer_request:mds.1:1940 currently dispatched 2023-09-09T23:22:22.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:22 smithi098 ceph-mon[103124]: slow request 355.674255 seconds old, received at 2023-09-09T23:16:26.015382+0000: peer_request:mds.1:1972 currently dispatched 2023-09-09T23:22:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:22 smithi098 ceph-mon[103124]: slow request 355.662071 seconds old, received at 2023-09-09T23:16:26.027567+0000: peer_request:mds.1:2004 currently dispatched 2023-09-09T23:22:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:22 smithi098 ceph-mon[103124]: slow request 355.650862 seconds old, received at 2023-09-09T23:16:26.038776+0000: peer_request:mds.1:2036 currently dispatched 2023-09-09T23:22:22.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:22 smithi170 ceph-mon[102832]: 285 slow requests, 5 included below; oldest blocked for > 355.756389 secs 2023-09-09T23:22:22.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:22 smithi170 ceph-mon[102832]: slow request 355.695165 seconds old, received at 2023-09-09T23:16:25.994472+0000: peer_request:mds.1:1908 currently dispatched 2023-09-09T23:22:22.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:22 smithi170 ceph-mon[102832]: slow request 355.686716 seconds old, received at 2023-09-09T23:16:26.002921+0000: peer_request:mds.1:1940 currently dispatched 2023-09-09T23:22:22.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:22 smithi170 ceph-mon[102832]: slow request 355.674255 seconds old, received at 2023-09-09T23:16:26.015382+0000: peer_request:mds.1:1972 currently dispatched 2023-09-09T23:22:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:22 smithi170 ceph-mon[102832]: slow request 355.662071 seconds old, received at 2023-09-09T23:16:26.027567+0000: peer_request:mds.1:2004 currently dispatched 2023-09-09T23:22:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:22 smithi170 ceph-mon[102832]: slow request 355.650862 seconds old, received at 2023-09-09T23:16:26.038776+0000: peer_request:mds.1:2036 currently dispatched 2023-09-09T23:22:22.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:22 smithi082 ceph-mon[98694]: 285 slow requests, 5 included below; oldest blocked for > 355.756389 secs 2023-09-09T23:22:22.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:22 smithi082 ceph-mon[98694]: slow request 355.695165 seconds old, received at 2023-09-09T23:16:25.994472+0000: peer_request:mds.1:1908 currently dispatched 2023-09-09T23:22:22.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:22 smithi082 ceph-mon[98694]: slow request 355.686716 seconds old, received at 2023-09-09T23:16:26.002921+0000: peer_request:mds.1:1940 currently dispatched 2023-09-09T23:22:22.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:22 smithi082 ceph-mon[98694]: slow request 355.674255 seconds old, received at 2023-09-09T23:16:26.015382+0000: peer_request:mds.1:1972 currently dispatched 2023-09-09T23:22:22.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:22 smithi082 ceph-mon[98694]: slow request 355.662071 seconds old, received at 2023-09-09T23:16:26.027567+0000: peer_request:mds.1:2004 currently dispatched 2023-09-09T23:22:22.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:22 smithi082 ceph-mon[98694]: slow request 355.650862 seconds old, received at 2023-09-09T23:16:26.038776+0000: peer_request:mds.1:2036 currently dispatched 2023-09-09T23:22:23.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:23 smithi098 ceph-mon[103124]: pgmap v263: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 13 op/s 2023-09-09T23:22:23.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:23.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:23.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:23 smithi170 ceph-mon[102832]: pgmap v263: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 13 op/s 2023-09-09T23:22:23.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:23.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:23.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:23 smithi082 ceph-mon[98694]: pgmap v263: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 13 op/s 2023-09-09T23:22:23.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:23.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:24.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:24 smithi098 ceph-mon[103124]: 135 slow requests, 5 included below; oldest blocked for > 368.792689 secs 2023-09-09T23:22:24.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:24 smithi098 ceph-mon[103124]: slow request 368.532871 seconds old, received at 2023-09-09T23:16:14.596100+0000: client_request(mds.1:1889 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:24.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:24 smithi098 ceph-mon[103124]: slow request 368.479258 seconds old, received at 2023-09-09T23:16:14.649713+0000: client_request(mds.1:1921 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:24 smithi098 ceph-mon[103124]: slow request 368.424057 seconds old, received at 2023-09-09T23:16:14.704914+0000: client_request(mds.1:1953 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:24 smithi098 ceph-mon[103124]: slow request 368.365845 seconds old, received at 2023-09-09T23:16:14.763126+0000: client_request(mds.1:1985 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:24 smithi098 ceph-mon[103124]: slow request 368.305727 seconds old, received at 2023-09-09T23:16:14.823245+0000: client_request(mds.1:2017 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:24.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:24 smithi170 ceph-mon[102832]: 135 slow requests, 5 included below; oldest blocked for > 368.792689 secs 2023-09-09T23:22:24.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:24 smithi170 ceph-mon[102832]: slow request 368.532871 seconds old, received at 2023-09-09T23:16:14.596100+0000: client_request(mds.1:1889 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:24.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:24 smithi170 ceph-mon[102832]: slow request 368.479258 seconds old, received at 2023-09-09T23:16:14.649713+0000: client_request(mds.1:1921 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:24 smithi170 ceph-mon[102832]: slow request 368.424057 seconds old, received at 2023-09-09T23:16:14.704914+0000: client_request(mds.1:1953 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:24 smithi170 ceph-mon[102832]: slow request 368.365845 seconds old, received at 2023-09-09T23:16:14.763126+0000: client_request(mds.1:1985 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:24 smithi170 ceph-mon[102832]: slow request 368.305727 seconds old, received at 2023-09-09T23:16:14.823245+0000: client_request(mds.1:2017 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:24.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:24 smithi082 ceph-mon[98694]: 135 slow requests, 5 included below; oldest blocked for > 368.792689 secs 2023-09-09T23:22:24.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:24 smithi082 ceph-mon[98694]: slow request 368.532871 seconds old, received at 2023-09-09T23:16:14.596100+0000: client_request(mds.1:1889 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:24.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:24 smithi082 ceph-mon[98694]: slow request 368.479258 seconds old, received at 2023-09-09T23:16:14.649713+0000: client_request(mds.1:1921 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:24.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:24 smithi082 ceph-mon[98694]: slow request 368.424057 seconds old, received at 2023-09-09T23:16:14.704914+0000: client_request(mds.1:1953 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:24.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:24 smithi082 ceph-mon[98694]: slow request 368.365845 seconds old, received at 2023-09-09T23:16:14.763126+0000: client_request(mds.1:1985 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:24.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:24 smithi082 ceph-mon[98694]: slow request 368.305727 seconds old, received at 2023-09-09T23:16:14.823245+0000: client_request(mds.1:2017 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:25.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:25 smithi098 ceph-mon[103124]: pgmap v264: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.3 KiB/s wr, 12 op/s 2023-09-09T23:22:25.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:25 smithi170 ceph-mon[102832]: pgmap v264: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.3 KiB/s wr, 12 op/s 2023-09-09T23:22:25.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:25 smithi082 ceph-mon[98694]: pgmap v264: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.3 KiB/s wr, 12 op/s 2023-09-09T23:22:26.156 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:22:26.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.294+0000 7f482ec01700 1 -- 172.21.15.82:0/2570691907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4828105920 msgr2=0x7f482810d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:22:26.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.294+0000 7f482ec01700 1 --2- 172.21.15.82:0/2570691907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4828105920 0x7f482810d4d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f482000b0a0 tx=0x7f482000b3b0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.295+0000 7f482ec01700 1 -- 172.21.15.82:0/2570691907 shutdown_connections 2023-09-09T23:22:26.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.295+0000 7f482ec01700 1 --2- 172.21.15.82:0/2570691907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4828105920 0x7f482810d4d0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.295+0000 7f482ec01700 1 --2- 172.21.15.82:0/2570691907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48280ff050 0x7f48280ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.295+0000 7f482ec01700 1 --2- 172.21.15.82:0/2570691907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48280ffa00 0x7f4828105240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.295+0000 7f482ec01700 1 -- 172.21.15.82:0/2570691907 >> 172.21.15.82:0/2570691907 conn(0x7f48280747a0 msgr2=0x7f4828074bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:22:26.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.295+0000 7f482ec01700 1 -- 172.21.15.82:0/2570691907 shutdown_connections 2023-09-09T23:22:26.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.295+0000 7f482ec01700 1 -- 172.21.15.82:0/2570691907 wait complete. 2023-09-09T23:22:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.295+0000 7f482ec01700 1 Processor -- start 2023-09-09T23:22:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.295+0000 7f482ec01700 1 -- start start 2023-09-09T23:22:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.296+0000 7f482ec01700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48280ff050 0x7f482819c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.296+0000 7f482ec01700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4828105920 0x7f482819cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.296+0000 7f482ec01700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4828196660 0x7f4828196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:26.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.296+0000 7f482ec01700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48281970e0 con 0x7f4828105920 2023-09-09T23:22:26.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.296+0000 7f482ec01700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4828197250 con 0x7f4828196660 2023-09-09T23:22:26.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.296+0000 7f482ec01700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4828197410 con 0x7f48280ff050 2023-09-09T23:22:26.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.296+0000 7f4827fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4828105920 0x7f482819cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:26.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.296+0000 7f482c99d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48280ff050 0x7f482819c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:26.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.296+0000 7f4827fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4828105920 0x7f482819cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58124/0 (socket says 172.21.15.82:58124) 2023-09-09T23:22:26.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.296+0000 7f482d19e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4828196660 0x7f4828196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:26.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.296+0000 7f4827fff700 1 -- 172.21.15.82:0/2832143444 learned_addr learned my addr 172.21.15.82:0/2832143444 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:22:26.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.296+0000 7f482c99d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48280ff050 0x7f482819c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:56636/0 (socket says 172.21.15.82:56636) 2023-09-09T23:22:26.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.296+0000 7f482c99d700 1 -- 172.21.15.82:0/2832143444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4828196660 msgr2=0x7f4828196b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:22:26.301 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.296+0000 7f482c99d700 1 --2- 172.21.15.82:0/2832143444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4828196660 0x7f4828196b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.301 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.296+0000 7f482c99d700 1 -- 172.21.15.82:0/2832143444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4828105920 msgr2=0x7f482819cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:22:26.301 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.296+0000 7f482c99d700 1 --2- 172.21.15.82:0/2832143444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4828105920 0x7f482819cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.301 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.296+0000 7f482c99d700 1 -- 172.21.15.82:0/2832143444 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4820009d00 con 0x7f48280ff050 2023-09-09T23:22:26.301 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.296+0000 7f482c99d700 1 --2- 172.21.15.82:0/2832143444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48280ff050 0x7f482819c5f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f481c00e9f0 tx=0x7f481c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:22:26.301 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.296+0000 7f4825ffb700 1 -- 172.21.15.82:0/2832143444 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c00c9c0 con 0x7f48280ff050 2023-09-09T23:22:26.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.296+0000 7f482ec01700 1 -- 172.21.15.82:0/2832143444 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48281a34c0 con 0x7f48280ff050 2023-09-09T23:22:26.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.296+0000 7f4825ffb700 1 -- 172.21.15.82:0/2832143444 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f481c010430 con 0x7f48280ff050 2023-09-09T23:22:26.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.296+0000 7f4825ffb700 1 -- 172.21.15.82:0/2832143444 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c018680 con 0x7f48280ff050 2023-09-09T23:22:26.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.297+0000 7f482ec01700 1 -- 172.21.15.82:0/2832143444 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48281a39a0 con 0x7f48280ff050 2023-09-09T23:22:26.308 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.299+0000 7f4825ffb700 1 -- 172.21.15.82:0/2832143444 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f481c010860 con 0x7f48280ff050 2023-09-09T23:22:26.308 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.299+0000 7f4825ffb700 1 --2- 172.21.15.82:0/2832143444 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f48180a07b0 0x7f48180a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:26.308 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.299+0000 7f4825ffb700 1 -- 172.21.15.82:0/2832143444 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f481c014070 con 0x7f48280ff050 2023-09-09T23:22:26.309 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.299+0000 7f4827fff700 1 --2- 172.21.15.82:0/2832143444 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f48180a07b0 0x7f48180a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:26.309 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.300+0000 7f4827fff700 1 --2- 172.21.15.82:0/2832143444 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f48180a07b0 0x7f48180a2c70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4828197e30 tx=0x7f4814009260 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:22:26.309 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.300+0000 7f483adc7b80 1 Processor -- start 2023-09-09T23:22:26.309 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.300+0000 7f483adc7b80 1 -- start start 2023-09-09T23:22:26.309 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.300+0000 7f483adc7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648950b1710 0x5648950b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:26.310 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.300+0000 7f483adc7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5648950b2030 0x5648950bb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:26.310 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.300+0000 7f483adc7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648950bbe40 0x5648950be230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:26.310 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.300+0000 7f483adc7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56489500a730 con 0x5648950b1710 2023-09-09T23:22:26.310 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.300+0000 7f483adc7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564895008050 con 0x5648950b2030 2023-09-09T23:22:26.310 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.300+0000 7f483adc7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564895003940 con 0x5648950bbe40 2023-09-09T23:22:26.311 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.300+0000 7f482c99d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648950b1710 0x5648950b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:26.311 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.300+0000 7f482c99d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648950b1710 0x5648950b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58130/0 (socket says 172.21.15.82:58130) 2023-09-09T23:22:26.311 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.300+0000 7f482c99d700 1 -- 172.21.15.82:0/1045063571 learned_addr learned my addr 172.21.15.82:0/1045063571 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:22:26.311 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.300+0000 7f4827fff700 1 --2- 172.21.15.82:0/1045063571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5648950b2030 0x5648950bb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:26.311 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.301+0000 7f482d19e700 1 --2- 172.21.15.82:0/1045063571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648950bbe40 0x5648950be230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:26.312 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.301+0000 7f482c99d700 1 -- 172.21.15.82:0/1045063571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648950bbe40 msgr2=0x5648950be230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:22:26.312 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.301+0000 7f482c99d700 1 --2- 172.21.15.82:0/1045063571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648950bbe40 0x5648950be230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.312 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.301+0000 7f482c99d700 1 -- 172.21.15.82:0/1045063571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5648950b2030 msgr2=0x5648950bb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:22:26.312 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.301+0000 7f482c99d700 1 --2- 172.21.15.82:0/1045063571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5648950b2030 0x5648950bb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.312 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.301+0000 7f482c99d700 1 -- 172.21.15.82:0/1045063571 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564894e2fde0 con 0x5648950b1710 2023-09-09T23:22:26.313 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.301+0000 7f482c99d700 1 --2- 172.21.15.82:0/1045063571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648950b1710 0x5648950b1af0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f481c0a1050 tx=0x7f481c098e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80a12319a3c04e6 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:22:26.313 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.301+0000 7f4812ffd700 1 -- 172.21.15.82:0/1045063571 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c09d120 con 0x5648950b1710 2023-09-09T23:22:26.313 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.301+0000 7f4812ffd700 1 -- 172.21.15.82:0/1045063571 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f481c099510 con 0x5648950b1710 2023-09-09T23:22:26.313 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.301+0000 7f4812ffd700 1 -- 172.21.15.82:0/1045063571 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c099770 con 0x5648950b1710 2023-09-09T23:22:26.313 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.302+0000 7f483adc7b80 1 -- 172.21.15.82:0/1045063571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648950b1710 msgr2=0x5648950b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:22:26.314 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.302+0000 7f483adc7b80 1 --2- 172.21.15.82:0/1045063571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648950b1710 0x5648950b1af0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f481c0a1050 tx=0x7f481c098e70 comp rx=0 tx=0).stop 2023-09-09T23:22:26.314 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.302+0000 7f483adc7b80 1 -- 172.21.15.82:0/1045063571 shutdown_connections 2023-09-09T23:22:26.314 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.302+0000 7f483adc7b80 1 --2- 172.21.15.82:0/1045063571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648950bbe40 0x5648950be230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.314 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.302+0000 7f483adc7b80 1 --2- 172.21.15.82:0/1045063571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648950b1710 0x5648950b1af0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.302+0000 7f483adc7b80 1 --2- 172.21.15.82:0/1045063571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5648950b2030 0x5648950bb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.302+0000 7f483adc7b80 1 -- 172.21.15.82:0/1045063571 >> 172.21.15.82:0/1045063571 conn(0x564895026090 msgr2=0x5648950acbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:22:26.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.302+0000 7f483adc7b80 1 -- 172.21.15.82:0/1045063571 shutdown_connections 2023-09-09T23:22:26.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.302+0000 7f483adc7b80 1 -- 172.21.15.82:0/1045063571 wait complete. 2023-09-09T23:22:26.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.302+0000 7f483adc7b80 1 -- start start 2023-09-09T23:22:26.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.302+0000 7f483adc7b80 1 Processor -- start 2023-09-09T23:22:26.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.303+0000 7f483adc7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5648950b1710 0x5648950bb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:26.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.303+0000 7f483adc7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648950b2030 0x5648950b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:26.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.303+0000 7f483adc7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648950bbe40 0x5648950b9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:26.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.303+0000 7f483adc7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564894fa1b70 con 0x5648950bbe40 2023-09-09T23:22:26.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.303+0000 7f483adc7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x564894fa1870 con 0x5648950b1710 2023-09-09T23:22:26.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.303+0000 7f483adc7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564894e85920 con 0x5648950b2030 2023-09-09T23:22:26.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.303+0000 7f483adc7b80 10 client.0 _ll_register_callbacks cb 0x564894eeba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:22:26.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.303+0000 7f482d19e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648950bbe40 0x5648950b9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:26.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.303+0000 7f4827fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648950b2030 0x5648950b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:26.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.303+0000 7f482d19e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648950bbe40 0x5648950b9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58140/0 (socket says 172.21.15.82:58140) 2023-09-09T23:22:26.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.303+0000 7f4827fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648950b2030 0x5648950b8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:56652/0 (socket says 172.21.15.82:56652) 2023-09-09T23:22:26.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.303+0000 7f482d19e700 1 -- 172.21.15.82:0/1602269882 learned_addr learned my addr 172.21.15.82:0/1602269882 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:22:26.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.303+0000 7f482c99d700 1 --2- 172.21.15.82:0/1602269882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5648950b1710 0x5648950bb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:26.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.303+0000 7f482d19e700 1 -- 172.21.15.82:0/1602269882 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648950b2030 msgr2=0x5648950b8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:22:26.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.303+0000 7f482d19e700 1 --2- 172.21.15.82:0/1602269882 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648950b2030 0x5648950b8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.303+0000 7f482d19e700 1 -- 172.21.15.82:0/1602269882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5648950b1710 msgr2=0x5648950bb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:22:26.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.303+0000 7f482d19e700 1 --2- 172.21.15.82:0/1602269882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5648950b1710 0x5648950bb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.303+0000 7f482d19e700 1 -- 172.21.15.82:0/1602269882 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48281a39a0 con 0x5648950bbe40 2023-09-09T23:22:26.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.303+0000 7f482d19e700 1 --2- 172.21.15.82:0/1602269882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648950bbe40 0x5648950b9030 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f482000bf90 tx=0x7f4820009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a53ac998d873a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:22:26.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.303+0000 7f4807fff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:22:26.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.303+0000 7f4807fff700 1 -- 172.21.15.82:0/1602269882 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482000e040 con 0x5648950bbe40 2023-09-09T23:22:26.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.303+0000 7f4807fff700 1 -- 172.21.15.82:0/1602269882 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f48200074e0 con 0x5648950bbe40 2023-09-09T23:22:26.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.303+0000 7f483adc7b80 1 -- 172.21.15.82:0/1602269882 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564894f5b4f0 con 0x5648950bbe40 2023-09-09T23:22:26.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.303+0000 7f4807fff700 1 -- 172.21.15.82:0/1602269882 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48200076a0 con 0x5648950bbe40 2023-09-09T23:22:26.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.303+0000 7f483adc7b80 10 client.16227 fetch_fsmap learned FSMap version 109 2023-09-09T23:22:26.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.303+0000 7f483adc7b80 1 -- 172.21.15.82:0/1602269882 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=4+}) v3 -- 0x564894e18c60 con 0x5648950bbe40 2023-09-09T23:22:26.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.304+0000 7f4807fff700 1 -- 172.21.15.82:0/1602269882 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564894f5b4f0 con 0x5648950bbe40 2023-09-09T23:22:26.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.304+0000 7f4807fff700 1 -- 172.21.15.82:0/1602269882 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 109) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4820025030 con 0x5648950bbe40 2023-09-09T23:22:26.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.304+0000 7f483adc7b80 10 client.16227 fetch_fsmap finished waiting for FSMap version 109 2023-09-09T23:22:26.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.304+0000 7f483adc7b80 10 client.16227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:22:26.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.304+0000 7f483adc7b80 20 client.16227 populate_metadata read hostname 'smithi082' 2023-09-09T23:22:26.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.304+0000 7f483adc7b80 1 --2- 172.21.15.82:0/1602269882 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5648950cb8d0 0x5648950cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:26.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.304+0000 7f483adc7b80 4 client.16227 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:22:26.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.304+0000 7f483adc7b80 1 -- 172.21.15.82:0/1602269882 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5648950188a0 con 0x5648950cb8d0 2023-09-09T23:22:26.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.304+0000 7f482c99d700 1 --2- 172.21.15.82:0/1602269882 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5648950cb8d0 0x5648950cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:26.325 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.304+0000 7f4807fff700 10 client.16227 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:22:26.325 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.304+0000 7f482c99d700 1 --2- 172.21.15.82:0/1602269882 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5648950cb8d0 0x5648950cdcf0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6df569485dd2417e server_cookie=a485e608136f598 in_seq=0 out_seq=0 2023-09-09T23:22:26.325 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.305+0000 7f4807fff700 1 -- 172.21.15.82:0/1602269882 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5648950188a0 con 0x5648950cb8d0 2023-09-09T23:22:26.325 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.305+0000 7f4807fff700 10 client.16227 handle_command_reply: tid=0 2023-09-09T23:22:26.325 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.305+0000 7f483adc7b80 1 client.16227 shutdown 2023-09-09T23:22:26.326 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.305+0000 7f483adc7b80 1 -- 172.21.15.82:0/1602269882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648950bbe40 msgr2=0x5648950b9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:22:26.326 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.305+0000 7f483adc7b80 1 --2- 172.21.15.82:0/1602269882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648950bbe40 0x5648950b9030 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f482000bf90 tx=0x7f4820009c60 comp rx=0 tx=0).stop 2023-09-09T23:22:26.326 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.305+0000 7f483adc7b80 1 -- 172.21.15.82:0/1602269882 shutdown_connections 2023-09-09T23:22:26.326 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.305+0000 7f483adc7b80 1 --2- 172.21.15.82:0/1602269882 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5648950cb8d0 0x5648950cdcf0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.326 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.305+0000 7f483adc7b80 1 --2- 172.21.15.82:0/1602269882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5648950b1710 0x5648950bb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.327 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.305+0000 7f483adc7b80 1 --2- 172.21.15.82:0/1602269882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648950bbe40 0x5648950b9030 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.327 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.305+0000 7f483adc7b80 1 --2- 172.21.15.82:0/1602269882 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648950b2030 0x5648950b8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.327 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.305+0000 7f4807fff700 0 client.16227 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:22:26.327 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.305+0000 7f483adc7b80 1 -- 172.21.15.82:0/1602269882 >> 172.21.15.82:0/1602269882 conn(0x564895026090 msgr2=0x5648950ac7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:22:26.327 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.306+0000 7f483adc7b80 1 -- 172.21.15.82:0/1602269882 shutdown_connections 2023-09-09T23:22:26.327 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.306+0000 7f483adc7b80 1 -- 172.21.15.82:0/1602269882 wait complete. 2023-09-09T23:22:26.328 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.306+0000 7f483adc7b80 20 client.16227 trim_cache size 0 max 16384 2023-09-09T23:22:26.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.322+0000 7f483adc7b80 1 Processor -- start 2023-09-09T23:22:26.547 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.322+0000 7f483adc7b80 1 -- start start 2023-09-09T23:22:26.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.322+0000 7f483adc7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648950b1710 0x5648950b81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:26.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.322+0000 7f483adc7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648950b2030 0x5648950bb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:26.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.322+0000 7f483adc7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5648950bbe40 0x5648950b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:26.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.322+0000 7f483adc7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648950ad180 con 0x5648950b2030 2023-09-09T23:22:26.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.322+0000 7f483adc7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648950b6f80 con 0x5648950bbe40 2023-09-09T23:22:26.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.322+0000 7f483adc7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648950ab9a0 con 0x5648950b1710 2023-09-09T23:22:26.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.322+0000 7f4827fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648950b2030 0x5648950bb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:26.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.322+0000 7f4827fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648950b2030 0x5648950bb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58146/0 (socket says 172.21.15.82:58146) 2023-09-09T23:22:26.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.322+0000 7f4827fff700 1 -- 172.21.15.82:0/1832948946 learned_addr learned my addr 172.21.15.82:0/1832948946 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:22:26.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.323+0000 7f482c99d700 1 --2- 172.21.15.82:0/1832948946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648950b1710 0x5648950b81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:26.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.323+0000 7f482d19e700 1 --2- 172.21.15.82:0/1832948946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5648950bbe40 0x5648950b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:26.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.323+0000 7f4827fff700 1 -- 172.21.15.82:0/1832948946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648950b1710 msgr2=0x5648950b81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:22:26.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.323+0000 7f4827fff700 1 --2- 172.21.15.82:0/1832948946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648950b1710 0x5648950b81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.323+0000 7f4827fff700 1 -- 172.21.15.82:0/1832948946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5648950bbe40 msgr2=0x5648950b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:22:26.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.323+0000 7f4827fff700 1 --2- 172.21.15.82:0/1832948946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5648950bbe40 0x5648950b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.323+0000 7f4827fff700 1 -- 172.21.15.82:0/1832948946 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564894e56250 con 0x5648950b2030 2023-09-09T23:22:26.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.323+0000 7f482d19e700 1 --2- 172.21.15.82:0/1832948946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5648950bbe40 0x5648950b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:22:26.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.323+0000 7f4827fff700 1 --2- 172.21.15.82:0/1832948946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648950b2030 0x5648950bb340 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4814009820 tx=0x7f4814014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14b8770e626bcb7 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:22:26.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.323+0000 7f48137fe700 1 -- 172.21.15.82:0/1832948946 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4814024070 con 0x5648950b2030 2023-09-09T23:22:26.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.323+0000 7f48137fe700 1 -- 172.21.15.82:0/1832948946 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4814004020 con 0x5648950b2030 2023-09-09T23:22:26.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.323+0000 7f48137fe700 1 -- 172.21.15.82:0/1832948946 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4814004d80 con 0x5648950b2030 2023-09-09T23:22:26.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.323+0000 7f483adc7b80 1 -- 172.21.15.82:0/1832948946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648950b2030 msgr2=0x5648950bb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:22:26.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.323+0000 7f483adc7b80 1 --2- 172.21.15.82:0/1832948946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648950b2030 0x5648950bb340 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4814009820 tx=0x7f4814014000 comp rx=0 tx=0).stop 2023-09-09T23:22:26.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.324+0000 7f483adc7b80 1 -- 172.21.15.82:0/1832948946 shutdown_connections 2023-09-09T23:22:26.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.324+0000 7f483adc7b80 1 --2- 172.21.15.82:0/1832948946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5648950bbe40 0x5648950b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.324+0000 7f483adc7b80 1 --2- 172.21.15.82:0/1832948946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648950b1710 0x5648950b81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.324+0000 7f483adc7b80 1 --2- 172.21.15.82:0/1832948946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648950b2030 0x5648950bb340 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.324+0000 7f483adc7b80 1 -- 172.21.15.82:0/1832948946 >> 172.21.15.82:0/1832948946 conn(0x564895026090 msgr2=0x5648950b3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:22:26.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.324+0000 7f483adc7b80 1 -- 172.21.15.82:0/1832948946 shutdown_connections 2023-09-09T23:22:26.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.324+0000 7f483adc7b80 1 -- 172.21.15.82:0/1832948946 wait complete. 2023-09-09T23:22:26.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.324+0000 7f483adc7b80 1 -- start start 2023-09-09T23:22:26.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.324+0000 7f483adc7b80 1 Processor -- start 2023-09-09T23:22:26.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.324+0000 7f483adc7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648950b1710 0x5648950c13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:26.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.324+0000 7f483adc7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648950b2030 0x5648950c1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:26.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.324+0000 7f483adc7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5648950bbe40 0x56489511f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:26.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.324+0000 7f483adc7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648950c2220 con 0x5648950b2030 2023-09-09T23:22:26.559 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.324+0000 7f483adc7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x56489511fc30 con 0x5648950bbe40 2023-09-09T23:22:26.559 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.324+0000 7f483adc7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56489511fd70 con 0x5648950b1710 2023-09-09T23:22:26.559 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.324+0000 7f483adc7b80 10 client.0 _ll_register_callbacks cb 0x7f482000bab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:22:26.560 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.324+0000 7f4827fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648950b2030 0x5648950c1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:26.560 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.324+0000 7f4827fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648950b2030 0x5648950c1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58152/0 (socket says 172.21.15.82:58152) 2023-09-09T23:22:26.560 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.324+0000 7f4827fff700 1 -- 172.21.15.82:0/2762649198 learned_addr learned my addr 172.21.15.82:0/2762649198 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:22:26.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.324+0000 7f482d19e700 1 --2- 172.21.15.82:0/2762649198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5648950bbe40 0x56489511f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:26.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.324+0000 7f482c99d700 1 --2- 172.21.15.82:0/2762649198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648950b1710 0x5648950c13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:26.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.325+0000 7f4827fff700 1 -- 172.21.15.82:0/2762649198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648950b1710 msgr2=0x5648950c13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:22:26.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.325+0000 7f4827fff700 1 --2- 172.21.15.82:0/2762649198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648950b1710 0x5648950c13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.325+0000 7f4827fff700 1 -- 172.21.15.82:0/2762649198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5648950bbe40 msgr2=0x56489511f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:22:26.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.325+0000 7f4827fff700 1 --2- 172.21.15.82:0/2762649198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5648950bbe40 0x56489511f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.325+0000 7f4827fff700 1 -- 172.21.15.82:0/2762649198 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4814017070 con 0x5648950b2030 2023-09-09T23:22:26.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.325+0000 7f4827fff700 1 --2- 172.21.15.82:0/2762649198 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648950b2030 0x5648950c1910 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4814005950 tx=0x7f4814004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3eb7998814cf553 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:22:26.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.325+0000 7f4807fff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:22:26.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.325+0000 7f4807fff700 1 -- 172.21.15.82:0/2762649198 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4814024040 con 0x5648950b2030 2023-09-09T23:22:26.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.325+0000 7f483adc7b80 1 -- 172.21.15.82:0/2762649198 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f481c099510 con 0x5648950b2030 2023-09-09T23:22:26.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.325+0000 7f4807fff700 1 -- 172.21.15.82:0/2762649198 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f4814003680 con 0x5648950b2030 2023-09-09T23:22:26.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.325+0000 7f4807fff700 1 -- 172.21.15.82:0/2762649198 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4814003860 con 0x5648950b2030 2023-09-09T23:22:26.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.325+0000 7f483adc7b80 10 client.16233 fetch_fsmap learned FSMap version 109 2023-09-09T23:22:26.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.325+0000 7f483adc7b80 1 -- 172.21.15.82:0/2762649198 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=4+}) v3 -- 0x564894e2e2b0 con 0x5648950b2030 2023-09-09T23:22:26.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.325+0000 7f4807fff700 1 -- 172.21.15.82:0/2762649198 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f481403e020 con 0x5648950b2030 2023-09-09T23:22:26.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.325+0000 7f4807fff700 1 -- 172.21.15.82:0/2762649198 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 109) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f481403aa50 con 0x5648950b2030 2023-09-09T23:22:26.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.325+0000 7f483adc7b80 10 client.16233 fetch_fsmap finished waiting for FSMap version 109 2023-09-09T23:22:26.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.326+0000 7f483adc7b80 10 client.16233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:22:26.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.326+0000 7f483adc7b80 20 client.16233 populate_metadata read hostname 'smithi082' 2023-09-09T23:22:26.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.326+0000 7f483adc7b80 1 --2- 172.21.15.82:0/2762649198 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x564895121480 0x5648951238a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:26.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.326+0000 7f483adc7b80 4 client.16233 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:22:26.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.326+0000 7f483adc7b80 1 -- 172.21.15.82:0/2762649198 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564894ffc920 con 0x564895121480 2023-09-09T23:22:26.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.326+0000 7f482c99d700 1 --2- 172.21.15.82:0/2762649198 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x564895121480 0x5648951238a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:26.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.326+0000 7f4807fff700 10 client.16233 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:22:26.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.326+0000 7f482c99d700 1 --2- 172.21.15.82:0/2762649198 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x564895121480 0x5648951238a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4721ed92ec94fb8 server_cookie=35dd3459c3bbea91 in_seq=0 out_seq=0 2023-09-09T23:22:26.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.326+0000 7f4807fff700 1 -- 172.21.15.82:0/2762649198 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x564894ffc920 con 0x564895121480 2023-09-09T23:22:26.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.326+0000 7f4807fff700 10 client.16233 handle_command_reply: tid=0 2023-09-09T23:22:26.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.326+0000 7f483adc7b80 1 client.16233 shutdown 2023-09-09T23:22:26.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.326+0000 7f483adc7b80 1 -- 172.21.15.82:0/2762649198 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648950b2030 msgr2=0x5648950c1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:22:26.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.326+0000 7f483adc7b80 1 --2- 172.21.15.82:0/2762649198 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648950b2030 0x5648950c1910 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4814005950 tx=0x7f4814004980 comp rx=0 tx=0).stop 2023-09-09T23:22:26.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.326+0000 7f483adc7b80 1 -- 172.21.15.82:0/2762649198 shutdown_connections 2023-09-09T23:22:26.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.326+0000 7f483adc7b80 1 --2- 172.21.15.82:0/2762649198 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x564895121480 0x5648951238a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.326+0000 7f483adc7b80 1 --2- 172.21.15.82:0/2762649198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5648950bbe40 0x56489511f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.326+0000 7f4807fff700 0 client.16233 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:22:26.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.326+0000 7f483adc7b80 1 --2- 172.21.15.82:0/2762649198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648950b1710 0x5648950c13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.326+0000 7f483adc7b80 1 --2- 172.21.15.82:0/2762649198 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5648950b2030 0x5648950c1910 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.326+0000 7f483adc7b80 1 -- 172.21.15.82:0/2762649198 >> 172.21.15.82:0/2762649198 conn(0x564895026090 msgr2=0x5648950b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:22:26.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.327+0000 7f483adc7b80 1 -- 172.21.15.82:0/2762649198 shutdown_connections 2023-09-09T23:22:26.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.327+0000 7f483adc7b80 1 -- 172.21.15.82:0/2762649198 wait complete. 2023-09-09T23:22:26.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.327+0000 7f483adc7b80 20 client.16233 trim_cache size 0 max 16384 2023-09-09T23:22:26.574 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:22:26.575 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:22:26.575 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:22:26.575 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:22:26.576 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:22:26.576 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:22:26.576 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:22:26.577 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:22:26.577 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:22:26.577 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:22:26.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.327+0000 7f482ec01700 1 -- 172.21.15.82:0/2832143444 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f48180a07b0 msgr2=0x7f48180a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:22:26.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.327+0000 7f482ec01700 1 --2- 172.21.15.82:0/2832143444 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f48180a07b0 0x7f48180a2c70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4828197e30 tx=0x7f4814009260 comp rx=0 tx=0).stop 2023-09-09T23:22:26.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.327+0000 7f482ec01700 1 -- 172.21.15.82:0/2832143444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48280ff050 msgr2=0x7f482819c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:22:26.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.327+0000 7f482ec01700 1 --2- 172.21.15.82:0/2832143444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48280ff050 0x7f482819c5f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f481c00e9f0 tx=0x7f481c00ed00 comp rx=0 tx=0).stop 2023-09-09T23:22:26.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.327+0000 7f482ec01700 1 -- 172.21.15.82:0/2832143444 shutdown_connections 2023-09-09T23:22:26.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.327+0000 7f482ec01700 1 --2- 172.21.15.82:0/2832143444 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f48180a07b0 0x7f48180a2c70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.327+0000 7f482ec01700 1 --2- 172.21.15.82:0/2832143444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4828196660 0x7f4828196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.327+0000 7f482ec01700 1 --2- 172.21.15.82:0/2832143444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48280ff050 0x7f482819c5f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.327+0000 7f482ec01700 1 --2- 172.21.15.82:0/2832143444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4828105920 0x7f482819cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:26.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.327+0000 7f482ec01700 1 -- 172.21.15.82:0/2832143444 >> 172.21.15.82:0/2832143444 conn(0x7f48280747a0 msgr2=0x7f48280fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:22:26.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.327+0000 7f482ec01700 1 -- 172.21.15.82:0/2832143444 shutdown_connections 2023-09-09T23:22:26.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:26.327+0000 7f482ec01700 1 -- 172.21.15.82:0/2832143444 wait complete. 2023-09-09T23:22:26.583 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:22:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:27 smithi098 ceph-mon[103124]: pgmap v265: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 KiB/s wr, 16 op/s 2023-09-09T23:22:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:27 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:27 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:27 smithi098 ceph-mon[103124]: 266 slow requests, 5 included below; oldest blocked for > 360.747954 secs 2023-09-09T23:22:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:27 smithi098 ceph-mon[103124]: slow request 360.743687 seconds old, received at 2023-09-09T23:16:25.946070+0000: peer_request:mds.1:1771 currently dispatched 2023-09-09T23:22:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:27 smithi098 ceph-mon[103124]: slow request 360.733456 seconds old, received at 2023-09-09T23:16:25.956301+0000: peer_request:mds.1:1803 currently dispatched 2023-09-09T23:22:27.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:27 smithi098 ceph-mon[103124]: slow request 360.722011 seconds old, received at 2023-09-09T23:16:25.967745+0000: peer_request:mds.1:1835 currently dispatched 2023-09-09T23:22:27.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:27 smithi098 ceph-mon[103124]: slow request 360.710491 seconds old, received at 2023-09-09T23:16:25.979266+0000: peer_request:mds.1:1867 currently dispatched 2023-09-09T23:22:27.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:27 smithi098 ceph-mon[103124]: slow request 360.699569 seconds old, received at 2023-09-09T23:16:25.990188+0000: peer_request:mds.1:1899 currently dispatched 2023-09-09T23:22:27.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:27 smithi170 ceph-mon[102832]: pgmap v265: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 KiB/s wr, 16 op/s 2023-09-09T23:22:27.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:27 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:27.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:27 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:27 smithi170 ceph-mon[102832]: 266 slow requests, 5 included below; oldest blocked for > 360.747954 secs 2023-09-09T23:22:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:27 smithi170 ceph-mon[102832]: slow request 360.743687 seconds old, received at 2023-09-09T23:16:25.946070+0000: peer_request:mds.1:1771 currently dispatched 2023-09-09T23:22:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:27 smithi170 ceph-mon[102832]: slow request 360.733456 seconds old, received at 2023-09-09T23:16:25.956301+0000: peer_request:mds.1:1803 currently dispatched 2023-09-09T23:22:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:27 smithi170 ceph-mon[102832]: slow request 360.722011 seconds old, received at 2023-09-09T23:16:25.967745+0000: peer_request:mds.1:1835 currently dispatched 2023-09-09T23:22:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:27 smithi170 ceph-mon[102832]: slow request 360.710491 seconds old, received at 2023-09-09T23:16:25.979266+0000: peer_request:mds.1:1867 currently dispatched 2023-09-09T23:22:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:27 smithi170 ceph-mon[102832]: slow request 360.699569 seconds old, received at 2023-09-09T23:16:25.990188+0000: peer_request:mds.1:1899 currently dispatched 2023-09-09T23:22:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:27 smithi082 ceph-mon[98694]: pgmap v265: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 KiB/s wr, 16 op/s 2023-09-09T23:22:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:27 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:27 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:27 smithi082 ceph-mon[98694]: 266 slow requests, 5 included below; oldest blocked for > 360.747954 secs 2023-09-09T23:22:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:27 smithi082 ceph-mon[98694]: slow request 360.743687 seconds old, received at 2023-09-09T23:16:25.946070+0000: peer_request:mds.1:1771 currently dispatched 2023-09-09T23:22:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:27 smithi082 ceph-mon[98694]: slow request 360.733456 seconds old, received at 2023-09-09T23:16:25.956301+0000: peer_request:mds.1:1803 currently dispatched 2023-09-09T23:22:27.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:27 smithi082 ceph-mon[98694]: slow request 360.722011 seconds old, received at 2023-09-09T23:16:25.967745+0000: peer_request:mds.1:1835 currently dispatched 2023-09-09T23:22:27.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:27 smithi082 ceph-mon[98694]: slow request 360.710491 seconds old, received at 2023-09-09T23:16:25.979266+0000: peer_request:mds.1:1867 currently dispatched 2023-09-09T23:22:27.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:27 smithi082 ceph-mon[98694]: slow request 360.699569 seconds old, received at 2023-09-09T23:16:25.990188+0000: peer_request:mds.1:1899 currently dispatched 2023-09-09T23:22:28.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:28 smithi098 ceph-mon[103124]: 3 slow requests, 3 included below; oldest blocked for > 180.020915 secs 2023-09-09T23:22:28.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:28 smithi098 ceph-mon[103124]: slow request 180.020915 seconds old, received at 2023-09-09T23:19:27.740042+0000: client_request(mds.4:47 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently acquired locks 2023-09-09T23:22:28.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:28 smithi098 ceph-mon[103124]: slow request 180.020535 seconds old, received at 2023-09-09T23:19:27.740422+0000: client_request(mds.4:48 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently requesting remote authpins 2023-09-09T23:22:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:28 smithi098 ceph-mon[103124]: slow request 180.018313 seconds old, received at 2023-09-09T23:19:27.742643+0000: client_request(client.24913:3869 unlink #0x10000000007/00000001000000010000000B 2023-09-09T23:19:27.741863+0000 caller_uid=26, caller_gid=26{26,}) currently requesting remote authpins 2023-09-09T23:22:28.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:28 smithi170 ceph-mon[102832]: 3 slow requests, 3 included below; oldest blocked for > 180.020915 secs 2023-09-09T23:22:28.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:28 smithi170 ceph-mon[102832]: slow request 180.020915 seconds old, received at 2023-09-09T23:19:27.740042+0000: client_request(mds.4:47 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently acquired locks 2023-09-09T23:22:28.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:28 smithi170 ceph-mon[102832]: slow request 180.020535 seconds old, received at 2023-09-09T23:19:27.740422+0000: client_request(mds.4:48 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently requesting remote authpins 2023-09-09T23:22:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:28 smithi170 ceph-mon[102832]: slow request 180.018313 seconds old, received at 2023-09-09T23:19:27.742643+0000: client_request(client.24913:3869 unlink #0x10000000007/00000001000000010000000B 2023-09-09T23:19:27.741863+0000 caller_uid=26, caller_gid=26{26,}) currently requesting remote authpins 2023-09-09T23:22:28.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:28 smithi082 ceph-mon[98694]: 3 slow requests, 3 included below; oldest blocked for > 180.020915 secs 2023-09-09T23:22:28.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:28 smithi082 ceph-mon[98694]: slow request 180.020915 seconds old, received at 2023-09-09T23:19:27.740042+0000: client_request(mds.4:47 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently acquired locks 2023-09-09T23:22:28.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:28 smithi082 ceph-mon[98694]: slow request 180.020535 seconds old, received at 2023-09-09T23:19:27.740422+0000: client_request(mds.4:48 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently requesting remote authpins 2023-09-09T23:22:28.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:28 smithi082 ceph-mon[98694]: slow request 180.018313 seconds old, received at 2023-09-09T23:19:27.742643+0000: client_request(client.24913:3869 unlink #0x10000000007/00000001000000010000000B 2023-09-09T23:19:27.741863+0000 caller_uid=26, caller_gid=26{26,}) currently requesting remote authpins 2023-09-09T23:22:29.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:29 smithi098 ceph-mon[103124]: 300 slow requests, 5 included below; oldest blocked for > 373.876008 secs 2023-09-09T23:22:29.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:29 smithi098 ceph-mon[103124]: slow request 373.378476 seconds old, received at 2023-09-09T23:16:14.876871+0000: client_request(mds.1:2049 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:29.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:29 smithi098 ceph-mon[103124]: slow request 373.823792 seconds old, received at 2023-09-09T23:16:14.431555+0000: client_request(mds.1:1794 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:29 smithi098 ceph-mon[103124]: slow request 373.764671 seconds old, received at 2023-09-09T23:16:14.490676+0000: client_request(mds.1:1826 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:29 smithi098 ceph-mon[103124]: slow request 373.710511 seconds old, received at 2023-09-09T23:16:14.544837+0000: client_request(mds.1:1858 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:29 smithi098 ceph-mon[103124]: slow request 373.657658 seconds old, received at 2023-09-09T23:16:14.597690+0000: client_request(mds.1:1890 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:29 smithi098 ceph-mon[103124]: pgmap v266: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 11 op/s 2023-09-09T23:22:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:29 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:22:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:29 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] up:active 2023-09-09T23:22:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:29 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:29.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:29 smithi170 ceph-mon[102832]: 300 slow requests, 5 included below; oldest blocked for > 373.876008 secs 2023-09-09T23:22:29.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:29 smithi170 ceph-mon[102832]: slow request 373.378476 seconds old, received at 2023-09-09T23:16:14.876871+0000: client_request(mds.1:2049 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:29.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:29 smithi170 ceph-mon[102832]: slow request 373.823792 seconds old, received at 2023-09-09T23:16:14.431555+0000: client_request(mds.1:1794 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:29 smithi170 ceph-mon[102832]: slow request 373.764671 seconds old, received at 2023-09-09T23:16:14.490676+0000: client_request(mds.1:1826 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:29 smithi170 ceph-mon[102832]: slow request 373.710511 seconds old, received at 2023-09-09T23:16:14.544837+0000: client_request(mds.1:1858 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:29 smithi170 ceph-mon[102832]: slow request 373.657658 seconds old, received at 2023-09-09T23:16:14.597690+0000: client_request(mds.1:1890 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:29 smithi170 ceph-mon[102832]: pgmap v266: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 11 op/s 2023-09-09T23:22:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:29 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:22:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:29 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] up:active 2023-09-09T23:22:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:29 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:29.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:29 smithi082 ceph-mon[98694]: 300 slow requests, 5 included below; oldest blocked for > 373.876008 secs 2023-09-09T23:22:29.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:29 smithi082 ceph-mon[98694]: slow request 373.378476 seconds old, received at 2023-09-09T23:16:14.876871+0000: client_request(mds.1:2049 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:29 smithi082 ceph-mon[98694]: slow request 373.823792 seconds old, received at 2023-09-09T23:16:14.431555+0000: client_request(mds.1:1794 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:29 smithi082 ceph-mon[98694]: slow request 373.764671 seconds old, received at 2023-09-09T23:16:14.490676+0000: client_request(mds.1:1826 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:29 smithi082 ceph-mon[98694]: slow request 373.710511 seconds old, received at 2023-09-09T23:16:14.544837+0000: client_request(mds.1:1858 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:29 smithi082 ceph-mon[98694]: slow request 373.657658 seconds old, received at 2023-09-09T23:16:14.597690+0000: client_request(mds.1:1890 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:29 smithi082 ceph-mon[98694]: pgmap v266: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 11 op/s 2023-09-09T23:22:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:29 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:22:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:29 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.170:6832/1931961940,v1:172.21.15.170:6833/1931961940] up:active 2023-09-09T23:22:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:29 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:31.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:31 smithi098 ceph-mon[103124]: pgmap v267: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2023-09-09T23:22:31.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:31.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:31.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:31 smithi170 ceph-mon[102832]: pgmap v267: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2023-09-09T23:22:31.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:31.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:31.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:31 smithi082 ceph-mon[98694]: pgmap v267: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2023-09-09T23:22:31.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:31.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:32.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:32.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:32.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:32 smithi170 ceph-mon[102832]: 281 slow requests, 5 included below; oldest blocked for > 365.740026 secs 2023-09-09T23:22:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:32 smithi170 ceph-mon[102832]: slow request 365.690200 seconds old, received at 2023-09-09T23:16:25.999649+0000: peer_request:mds.1:1931 currently dispatched 2023-09-09T23:22:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:32 smithi170 ceph-mon[102832]: slow request 365.677749 seconds old, received at 2023-09-09T23:16:26.012101+0000: peer_request:mds.1:1963 currently dispatched 2023-09-09T23:22:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:32 smithi170 ceph-mon[102832]: slow request 365.665814 seconds old, received at 2023-09-09T23:16:26.024036+0000: peer_request:mds.1:1995 currently dispatched 2023-09-09T23:22:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:32 smithi170 ceph-mon[102832]: slow request 365.654126 seconds old, received at 2023-09-09T23:16:26.035724+0000: peer_request:mds.1:2027 currently dispatched 2023-09-09T23:22:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:32 smithi170 ceph-mon[102832]: slow request 365.733208 seconds old, received at 2023-09-09T23:16:25.956641+0000: peer_request:mds.1:1804 currently dispatched 2023-09-09T23:22:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:32 smithi170 ceph-mon[102832]: pgmap v268: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 KiB/s wr, 10 op/s 2023-09-09T23:22:32.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:32 smithi082 ceph-mon[98694]: 281 slow requests, 5 included below; oldest blocked for > 365.740026 secs 2023-09-09T23:22:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:32 smithi082 ceph-mon[98694]: slow request 365.690200 seconds old, received at 2023-09-09T23:16:25.999649+0000: peer_request:mds.1:1931 currently dispatched 2023-09-09T23:22:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:32 smithi082 ceph-mon[98694]: slow request 365.677749 seconds old, received at 2023-09-09T23:16:26.012101+0000: peer_request:mds.1:1963 currently dispatched 2023-09-09T23:22:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:32 smithi082 ceph-mon[98694]: slow request 365.665814 seconds old, received at 2023-09-09T23:16:26.024036+0000: peer_request:mds.1:1995 currently dispatched 2023-09-09T23:22:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:32 smithi082 ceph-mon[98694]: slow request 365.654126 seconds old, received at 2023-09-09T23:16:26.035724+0000: peer_request:mds.1:2027 currently dispatched 2023-09-09T23:22:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:32 smithi082 ceph-mon[98694]: slow request 365.733208 seconds old, received at 2023-09-09T23:16:25.956641+0000: peer_request:mds.1:1804 currently dispatched 2023-09-09T23:22:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:32 smithi082 ceph-mon[98694]: pgmap v268: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 KiB/s wr, 10 op/s 2023-09-09T23:22:33.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:33.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:33.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:32 smithi098 ceph-mon[103124]: 281 slow requests, 5 included below; oldest blocked for > 365.740026 secs 2023-09-09T23:22:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:32 smithi098 ceph-mon[103124]: slow request 365.690200 seconds old, received at 2023-09-09T23:16:25.999649+0000: peer_request:mds.1:1931 currently dispatched 2023-09-09T23:22:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:32 smithi098 ceph-mon[103124]: slow request 365.677749 seconds old, received at 2023-09-09T23:16:26.012101+0000: peer_request:mds.1:1963 currently dispatched 2023-09-09T23:22:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:32 smithi098 ceph-mon[103124]: slow request 365.665814 seconds old, received at 2023-09-09T23:16:26.024036+0000: peer_request:mds.1:1995 currently dispatched 2023-09-09T23:22:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:32 smithi098 ceph-mon[103124]: slow request 365.654126 seconds old, received at 2023-09-09T23:16:26.035724+0000: peer_request:mds.1:2027 currently dispatched 2023-09-09T23:22:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:32 smithi098 ceph-mon[103124]: slow request 365.733208 seconds old, received at 2023-09-09T23:16:25.956641+0000: peer_request:mds.1:1804 currently dispatched 2023-09-09T23:22:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:32 smithi098 ceph-mon[103124]: pgmap v268: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 KiB/s wr, 10 op/s 2023-09-09T23:22:33.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 185.020977 secs 2023-09-09T23:22:33.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 185.020977 secs 2023-09-09T23:22:34.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 185.020977 secs 2023-09-09T23:22:34.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:34 smithi170 ceph-mon[102832]: 275 slow requests, 5 included below; oldest blocked for > 378.928011 secs 2023-09-09T23:22:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:34 smithi170 ceph-mon[102832]: slow request 378.701405 seconds old, received at 2023-09-09T23:16:14.651440+0000: client_request(mds.1:1922 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:34 smithi170 ceph-mon[102832]: slow request 378.645868 seconds old, received at 2023-09-09T23:16:14.706977+0000: client_request(mds.1:1954 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:34 smithi170 ceph-mon[102832]: slow request 378.587849 seconds old, received at 2023-09-09T23:16:14.764997+0000: client_request(mds.1:1986 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:34 smithi170 ceph-mon[102832]: slow request 378.528119 seconds old, received at 2023-09-09T23:16:14.824727+0000: client_request(mds.1:2018 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:34 smithi170 ceph-mon[102832]: slow request 378.474263 seconds old, received at 2023-09-09T23:16:14.878583+0000: client_request(mds.1:2050 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:34 smithi170 ceph-mon[102832]: pgmap v269: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 KiB/s wr, 9 op/s 2023-09-09T23:22:34.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:34 smithi082 ceph-mon[98694]: 275 slow requests, 5 included below; oldest blocked for > 378.928011 secs 2023-09-09T23:22:34.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:34 smithi082 ceph-mon[98694]: slow request 378.701405 seconds old, received at 2023-09-09T23:16:14.651440+0000: client_request(mds.1:1922 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:34.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:34 smithi082 ceph-mon[98694]: slow request 378.645868 seconds old, received at 2023-09-09T23:16:14.706977+0000: client_request(mds.1:1954 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:34.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:34 smithi082 ceph-mon[98694]: slow request 378.587849 seconds old, received at 2023-09-09T23:16:14.764997+0000: client_request(mds.1:1986 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:34.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:34 smithi082 ceph-mon[98694]: slow request 378.528119 seconds old, received at 2023-09-09T23:16:14.824727+0000: client_request(mds.1:2018 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:34.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:34 smithi082 ceph-mon[98694]: slow request 378.474263 seconds old, received at 2023-09-09T23:16:14.878583+0000: client_request(mds.1:2050 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:34.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:34 smithi082 ceph-mon[98694]: pgmap v269: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 KiB/s wr, 9 op/s 2023-09-09T23:22:35.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:34 smithi098 ceph-mon[103124]: 275 slow requests, 5 included below; oldest blocked for > 378.928011 secs 2023-09-09T23:22:35.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:34 smithi098 ceph-mon[103124]: slow request 378.701405 seconds old, received at 2023-09-09T23:16:14.651440+0000: client_request(mds.1:1922 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:34 smithi098 ceph-mon[103124]: slow request 378.645868 seconds old, received at 2023-09-09T23:16:14.706977+0000: client_request(mds.1:1954 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:34 smithi098 ceph-mon[103124]: slow request 378.587849 seconds old, received at 2023-09-09T23:16:14.764997+0000: client_request(mds.1:1986 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:34 smithi098 ceph-mon[103124]: slow request 378.528119 seconds old, received at 2023-09-09T23:16:14.824727+0000: client_request(mds.1:2018 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:34 smithi098 ceph-mon[103124]: slow request 378.474263 seconds old, received at 2023-09-09T23:16:14.878583+0000: client_request(mds.1:2050 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:34 smithi098 ceph-mon[103124]: pgmap v269: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 KiB/s wr, 9 op/s 2023-09-09T23:22:35.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:35 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:35.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:35 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:35.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:35 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:35.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:35 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:36.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:35 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:36.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:35 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:36.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:36.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:36 smithi170 ceph-mon[102832]: pgmap v270: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.7 KiB/s wr, 13 op/s 2023-09-09T23:22:36.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:36.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:36.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:36 smithi082 ceph-mon[98694]: pgmap v270: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.7 KiB/s wr, 13 op/s 2023-09-09T23:22:37.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:37.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:37.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:36 smithi098 ceph-mon[103124]: pgmap v270: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.7 KiB/s wr, 13 op/s 2023-09-09T23:22:37.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:37 smithi170 ceph-mon[102832]: 254 slow requests, 5 included below; oldest blocked for > 370.731294 secs 2023-09-09T23:22:37.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:37 smithi170 ceph-mon[102832]: slow request 370.721801 seconds old, received at 2023-09-09T23:16:25.968110+0000: peer_request:mds.1:1836 currently dispatched 2023-09-09T23:22:37.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:37 smithi170 ceph-mon[102832]: slow request 370.710376 seconds old, received at 2023-09-09T23:16:25.979535+0000: peer_request:mds.1:1868 currently dispatched 2023-09-09T23:22:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:37 smithi170 ceph-mon[102832]: slow request 370.695513 seconds old, received at 2023-09-09T23:16:25.994398+0000: peer_request:mds.1:1900 currently dispatched 2023-09-09T23:22:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:37 smithi170 ceph-mon[102832]: slow request 370.689847 seconds old, received at 2023-09-09T23:16:26.000064+0000: peer_request:mds.1:1932 currently dispatched 2023-09-09T23:22:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:37 smithi170 ceph-mon[102832]: slow request 370.677488 seconds old, received at 2023-09-09T23:16:26.012423+0000: peer_request:mds.1:1964 currently dispatched 2023-09-09T23:22:37.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:37 smithi082 ceph-mon[98694]: 254 slow requests, 5 included below; oldest blocked for > 370.731294 secs 2023-09-09T23:22:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:37 smithi082 ceph-mon[98694]: slow request 370.721801 seconds old, received at 2023-09-09T23:16:25.968110+0000: peer_request:mds.1:1836 currently dispatched 2023-09-09T23:22:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:37 smithi082 ceph-mon[98694]: slow request 370.710376 seconds old, received at 2023-09-09T23:16:25.979535+0000: peer_request:mds.1:1868 currently dispatched 2023-09-09T23:22:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:37 smithi082 ceph-mon[98694]: slow request 370.695513 seconds old, received at 2023-09-09T23:16:25.994398+0000: peer_request:mds.1:1900 currently dispatched 2023-09-09T23:22:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:37 smithi082 ceph-mon[98694]: slow request 370.689847 seconds old, received at 2023-09-09T23:16:26.000064+0000: peer_request:mds.1:1932 currently dispatched 2023-09-09T23:22:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:37 smithi082 ceph-mon[98694]: slow request 370.677488 seconds old, received at 2023-09-09T23:16:26.012423+0000: peer_request:mds.1:1964 currently dispatched 2023-09-09T23:22:38.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:37 smithi098 ceph-mon[103124]: 254 slow requests, 5 included below; oldest blocked for > 370.731294 secs 2023-09-09T23:22:38.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:37 smithi098 ceph-mon[103124]: slow request 370.721801 seconds old, received at 2023-09-09T23:16:25.968110+0000: peer_request:mds.1:1836 currently dispatched 2023-09-09T23:22:38.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:37 smithi098 ceph-mon[103124]: slow request 370.710376 seconds old, received at 2023-09-09T23:16:25.979535+0000: peer_request:mds.1:1868 currently dispatched 2023-09-09T23:22:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:37 smithi098 ceph-mon[103124]: slow request 370.695513 seconds old, received at 2023-09-09T23:16:25.994398+0000: peer_request:mds.1:1900 currently dispatched 2023-09-09T23:22:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:37 smithi098 ceph-mon[103124]: slow request 370.689847 seconds old, received at 2023-09-09T23:16:26.000064+0000: peer_request:mds.1:1932 currently dispatched 2023-09-09T23:22:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:37 smithi098 ceph-mon[103124]: slow request 370.677488 seconds old, received at 2023-09-09T23:16:26.012423+0000: peer_request:mds.1:1964 currently dispatched 2023-09-09T23:22:38.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 190.021034 secs 2023-09-09T23:22:38.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:38 smithi170 ceph-mon[102832]: pgmap v271: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2023-09-09T23:22:38.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 190.021034 secs 2023-09-09T23:22:38.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:38 smithi082 ceph-mon[98694]: pgmap v271: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2023-09-09T23:22:39.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 190.021034 secs 2023-09-09T23:22:39.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:38 smithi098 ceph-mon[103124]: pgmap v271: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2023-09-09T23:22:39.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:39 smithi170 ceph-mon[102832]: 247 slow requests, 5 included below; oldest blocked for > 383.912412 secs 2023-09-09T23:22:39.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:39 smithi170 ceph-mon[102832]: slow request 383.896242 seconds old, received at 2023-09-09T23:16:14.492391+0000: client_request(mds.1:1827 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:39.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:39 smithi170 ceph-mon[102832]: slow request 383.842107 seconds old, received at 2023-09-09T23:16:14.546526+0000: client_request(mds.1:1859 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:39 smithi170 ceph-mon[102832]: slow request 383.789507 seconds old, received at 2023-09-09T23:16:14.599126+0000: client_request(mds.1:1891 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:39 smithi170 ceph-mon[102832]: slow request 383.735395 seconds old, received at 2023-09-09T23:16:14.653238+0000: client_request(mds.1:1923 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:39 smithi170 ceph-mon[102832]: slow request 383.680064 seconds old, received at 2023-09-09T23:16:14.708569+0000: client_request(mds.1:1955 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:39.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:39 smithi082 ceph-mon[98694]: 247 slow requests, 5 included below; oldest blocked for > 383.912412 secs 2023-09-09T23:22:39.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:39 smithi082 ceph-mon[98694]: slow request 383.896242 seconds old, received at 2023-09-09T23:16:14.492391+0000: client_request(mds.1:1827 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:39 smithi082 ceph-mon[98694]: slow request 383.842107 seconds old, received at 2023-09-09T23:16:14.546526+0000: client_request(mds.1:1859 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:39 smithi082 ceph-mon[98694]: slow request 383.789507 seconds old, received at 2023-09-09T23:16:14.599126+0000: client_request(mds.1:1891 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:39 smithi082 ceph-mon[98694]: slow request 383.735395 seconds old, received at 2023-09-09T23:16:14.653238+0000: client_request(mds.1:1923 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:39 smithi082 ceph-mon[98694]: slow request 383.680064 seconds old, received at 2023-09-09T23:16:14.708569+0000: client_request(mds.1:1955 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:40.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:39 smithi098 ceph-mon[103124]: 247 slow requests, 5 included below; oldest blocked for > 383.912412 secs 2023-09-09T23:22:40.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:39 smithi098 ceph-mon[103124]: slow request 383.896242 seconds old, received at 2023-09-09T23:16:14.492391+0000: client_request(mds.1:1827 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:40.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:39 smithi098 ceph-mon[103124]: slow request 383.842107 seconds old, received at 2023-09-09T23:16:14.546526+0000: client_request(mds.1:1859 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:39 smithi098 ceph-mon[103124]: slow request 383.789507 seconds old, received at 2023-09-09T23:16:14.599126+0000: client_request(mds.1:1891 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:39 smithi098 ceph-mon[103124]: slow request 383.735395 seconds old, received at 2023-09-09T23:16:14.653238+0000: client_request(mds.1:1923 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:39 smithi098 ceph-mon[103124]: slow request 383.680064 seconds old, received at 2023-09-09T23:16:14.708569+0000: client_request(mds.1:1955 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:40.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:40.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:40.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:40 smithi170 ceph-mon[102832]: pgmap v272: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2023-09-09T23:22:40.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:40.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:40.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:40 smithi082 ceph-mon[98694]: pgmap v272: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2023-09-09T23:22:41.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:41.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:41.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:40 smithi098 ceph-mon[103124]: pgmap v272: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2023-09-09T23:22:42.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:42 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:42.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:42 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:42.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:42 smithi170 ceph-mon[102832]: 224 slow requests, 5 included below; oldest blocked for > 375.720770 secs 2023-09-09T23:22:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:42 smithi170 ceph-mon[102832]: slow request 375.665672 seconds old, received at 2023-09-09T23:16:26.024354+0000: peer_request:mds.1:1996 currently dispatched 2023-09-09T23:22:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:42 smithi170 ceph-mon[102832]: slow request 375.653956 seconds old, received at 2023-09-09T23:16:26.036070+0000: peer_request:mds.1:2028 currently dispatched 2023-09-09T23:22:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:42 smithi170 ceph-mon[102832]: slow request 367.829812 seconds old, received at 2023-09-09T23:16:33.860214+0000: peer_request:mds.1:2060 currently dispatched 2023-09-09T23:22:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:42 smithi170 ceph-mon[102832]: slow request 375.710205 seconds old, received at 2023-09-09T23:16:25.979821+0000: peer_request:mds.1:1869 currently dispatched 2023-09-09T23:22:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:42 smithi170 ceph-mon[102832]: slow request 375.695607 seconds old, received at 2023-09-09T23:16:25.994419+0000: peer_request:mds.1:1901 currently dispatched 2023-09-09T23:22:42.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:42 smithi170 ceph-mon[102832]: pgmap v273: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:22:42.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:42 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:42.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:42 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:42.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:42 smithi082 ceph-mon[98694]: 224 slow requests, 5 included below; oldest blocked for > 375.720770 secs 2023-09-09T23:22:42.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:42 smithi082 ceph-mon[98694]: slow request 375.665672 seconds old, received at 2023-09-09T23:16:26.024354+0000: peer_request:mds.1:1996 currently dispatched 2023-09-09T23:22:42.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:42 smithi082 ceph-mon[98694]: slow request 375.653956 seconds old, received at 2023-09-09T23:16:26.036070+0000: peer_request:mds.1:2028 currently dispatched 2023-09-09T23:22:42.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:42 smithi082 ceph-mon[98694]: slow request 367.829812 seconds old, received at 2023-09-09T23:16:33.860214+0000: peer_request:mds.1:2060 currently dispatched 2023-09-09T23:22:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:42 smithi082 ceph-mon[98694]: slow request 375.710205 seconds old, received at 2023-09-09T23:16:25.979821+0000: peer_request:mds.1:1869 currently dispatched 2023-09-09T23:22:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:42 smithi082 ceph-mon[98694]: slow request 375.695607 seconds old, received at 2023-09-09T23:16:25.994419+0000: peer_request:mds.1:1901 currently dispatched 2023-09-09T23:22:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:42 smithi082 ceph-mon[98694]: pgmap v273: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:22:43.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:42 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:43.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:42 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:42 smithi098 ceph-mon[103124]: 224 slow requests, 5 included below; oldest blocked for > 375.720770 secs 2023-09-09T23:22:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:42 smithi098 ceph-mon[103124]: slow request 375.665672 seconds old, received at 2023-09-09T23:16:26.024354+0000: peer_request:mds.1:1996 currently dispatched 2023-09-09T23:22:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:42 smithi098 ceph-mon[103124]: slow request 375.653956 seconds old, received at 2023-09-09T23:16:26.036070+0000: peer_request:mds.1:2028 currently dispatched 2023-09-09T23:22:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:42 smithi098 ceph-mon[103124]: slow request 367.829812 seconds old, received at 2023-09-09T23:16:33.860214+0000: peer_request:mds.1:2060 currently dispatched 2023-09-09T23:22:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:42 smithi098 ceph-mon[103124]: slow request 375.710205 seconds old, received at 2023-09-09T23:16:25.979821+0000: peer_request:mds.1:1869 currently dispatched 2023-09-09T23:22:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:42 smithi098 ceph-mon[103124]: slow request 375.695607 seconds old, received at 2023-09-09T23:16:25.994419+0000: peer_request:mds.1:1901 currently dispatched 2023-09-09T23:22:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:42 smithi098 ceph-mon[103124]: pgmap v273: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:22:43.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:43.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:43.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 195.021093 secs 2023-09-09T23:22:43.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:43.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:43.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 195.021093 secs 2023-09-09T23:22:44.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:44.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:44.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 195.021093 secs 2023-09-09T23:22:44.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:44 smithi170 ceph-mon[102832]: 214 slow requests, 5 included below; oldest blocked for > 388.890806 secs 2023-09-09T23:22:44.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:44 smithi170 ceph-mon[102832]: slow request 388.657740 seconds old, received at 2023-09-09T23:16:14.766898+0000: client_request(mds.1:1987 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:44.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:44 smithi170 ceph-mon[102832]: slow request 388.598137 seconds old, received at 2023-09-09T23:16:14.826501+0000: client_request(mds.1:2019 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:44 smithi170 ceph-mon[102832]: slow request 388.544542 seconds old, received at 2023-09-09T23:16:14.880096+0000: client_request(mds.1:2051 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:44 smithi170 ceph-mon[102832]: slow request 388.876545 seconds old, received at 2023-09-09T23:16:14.548094+0000: client_request(mds.1:1860 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:44 smithi170 ceph-mon[102832]: slow request 388.823622 seconds old, received at 2023-09-09T23:16:14.601016+0000: client_request(mds.1:1892 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:44 smithi170 ceph-mon[102832]: pgmap v274: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:22:44.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:44 smithi082 ceph-mon[98694]: 214 slow requests, 5 included below; oldest blocked for > 388.890806 secs 2023-09-09T23:22:44.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:44 smithi082 ceph-mon[98694]: slow request 388.657740 seconds old, received at 2023-09-09T23:16:14.766898+0000: client_request(mds.1:1987 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:44.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:44 smithi082 ceph-mon[98694]: slow request 388.598137 seconds old, received at 2023-09-09T23:16:14.826501+0000: client_request(mds.1:2019 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:44.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:44 smithi082 ceph-mon[98694]: slow request 388.544542 seconds old, received at 2023-09-09T23:16:14.880096+0000: client_request(mds.1:2051 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:44.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:44 smithi082 ceph-mon[98694]: slow request 388.876545 seconds old, received at 2023-09-09T23:16:14.548094+0000: client_request(mds.1:1860 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:44.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:44 smithi082 ceph-mon[98694]: slow request 388.823622 seconds old, received at 2023-09-09T23:16:14.601016+0000: client_request(mds.1:1892 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:44 smithi082 ceph-mon[98694]: pgmap v274: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:22:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:44 smithi098 ceph-mon[103124]: 214 slow requests, 5 included below; oldest blocked for > 388.890806 secs 2023-09-09T23:22:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:44 smithi098 ceph-mon[103124]: slow request 388.657740 seconds old, received at 2023-09-09T23:16:14.766898+0000: client_request(mds.1:1987 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:44 smithi098 ceph-mon[103124]: slow request 388.598137 seconds old, received at 2023-09-09T23:16:14.826501+0000: client_request(mds.1:2019 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:44 smithi098 ceph-mon[103124]: slow request 388.544542 seconds old, received at 2023-09-09T23:16:14.880096+0000: client_request(mds.1:2051 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:44 smithi098 ceph-mon[103124]: slow request 388.876545 seconds old, received at 2023-09-09T23:16:14.548094+0000: client_request(mds.1:1860 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:44 smithi098 ceph-mon[103124]: slow request 388.823622 seconds old, received at 2023-09-09T23:16:14.601016+0000: client_request(mds.1:1892 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:44 smithi098 ceph-mon[103124]: pgmap v274: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:22:46.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:46.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:46.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:46 smithi170 ceph-mon[102832]: pgmap v275: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:22:46.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:46.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:46.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:46 smithi082 ceph-mon[98694]: pgmap v275: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:22:47.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:47.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:46 smithi098 ceph-mon[103124]: pgmap v275: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:22:47.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:47 smithi170 ceph-mon[102832]: 188 slow requests, 5 included below; oldest blocked for > 380.708811 secs 2023-09-09T23:22:47.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:47 smithi170 ceph-mon[102832]: slow request 380.689639 seconds old, received at 2023-09-09T23:16:26.000478+0000: peer_request:mds.1:1933 currently dispatched 2023-09-09T23:22:47.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:47 smithi170 ceph-mon[102832]: slow request 380.677269 seconds old, received at 2023-09-09T23:16:26.012848+0000: peer_request:mds.1:1965 currently dispatched 2023-09-09T23:22:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:47 smithi170 ceph-mon[102832]: slow request 380.665334 seconds old, received at 2023-09-09T23:16:26.024783+0000: peer_request:mds.1:1997 currently dispatched 2023-09-09T23:22:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:47 smithi170 ceph-mon[102832]: slow request 380.653767 seconds old, received at 2023-09-09T23:16:26.036350+0000: peer_request:mds.1:2029 currently dispatched 2023-09-09T23:22:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:47 smithi170 ceph-mon[102832]: slow request 372.829606 seconds old, received at 2023-09-09T23:16:33.860511+0000: peer_request:mds.1:2061 currently dispatched 2023-09-09T23:22:47.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:47 smithi082 ceph-mon[98694]: 188 slow requests, 5 included below; oldest blocked for > 380.708811 secs 2023-09-09T23:22:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:47 smithi082 ceph-mon[98694]: slow request 380.689639 seconds old, received at 2023-09-09T23:16:26.000478+0000: peer_request:mds.1:1933 currently dispatched 2023-09-09T23:22:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:47 smithi082 ceph-mon[98694]: slow request 380.677269 seconds old, received at 2023-09-09T23:16:26.012848+0000: peer_request:mds.1:1965 currently dispatched 2023-09-09T23:22:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:47 smithi082 ceph-mon[98694]: slow request 380.665334 seconds old, received at 2023-09-09T23:16:26.024783+0000: peer_request:mds.1:1997 currently dispatched 2023-09-09T23:22:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:47 smithi082 ceph-mon[98694]: slow request 380.653767 seconds old, received at 2023-09-09T23:16:26.036350+0000: peer_request:mds.1:2029 currently dispatched 2023-09-09T23:22:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:47 smithi082 ceph-mon[98694]: slow request 372.829606 seconds old, received at 2023-09-09T23:16:33.860511+0000: peer_request:mds.1:2061 currently dispatched 2023-09-09T23:22:48.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:47 smithi098 ceph-mon[103124]: 188 slow requests, 5 included below; oldest blocked for > 380.708811 secs 2023-09-09T23:22:48.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:47 smithi098 ceph-mon[103124]: slow request 380.689639 seconds old, received at 2023-09-09T23:16:26.000478+0000: peer_request:mds.1:1933 currently dispatched 2023-09-09T23:22:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:47 smithi098 ceph-mon[103124]: slow request 380.677269 seconds old, received at 2023-09-09T23:16:26.012848+0000: peer_request:mds.1:1965 currently dispatched 2023-09-09T23:22:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:47 smithi098 ceph-mon[103124]: slow request 380.665334 seconds old, received at 2023-09-09T23:16:26.024783+0000: peer_request:mds.1:1997 currently dispatched 2023-09-09T23:22:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:47 smithi098 ceph-mon[103124]: slow request 380.653767 seconds old, received at 2023-09-09T23:16:26.036350+0000: peer_request:mds.1:2029 currently dispatched 2023-09-09T23:22:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:47 smithi098 ceph-mon[103124]: slow request 372.829606 seconds old, received at 2023-09-09T23:16:33.860511+0000: peer_request:mds.1:2061 currently dispatched 2023-09-09T23:22:48.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 200.021177 secs 2023-09-09T23:22:48.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:48 smithi082 ceph-mon[98694]: pgmap v276: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:22:49.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 200.021177 secs 2023-09-09T23:22:49.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:48 smithi098 ceph-mon[103124]: pgmap v276: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:22:49.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 200.021177 secs 2023-09-09T23:22:49.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:48 smithi170 ceph-mon[102832]: pgmap v276: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:22:49.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:49 smithi082 ceph-mon[98694]: 176 slow requests, 5 included below; oldest blocked for > 393.857962 secs 2023-09-09T23:22:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:49 smithi082 ceph-mon[98694]: slow request 393.798934 seconds old, received at 2023-09-09T23:16:14.655128+0000: client_request(mds.1:1924 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:49 smithi082 ceph-mon[98694]: slow request 393.744005 seconds old, received at 2023-09-09T23:16:14.710057+0000: client_request(mds.1:1956 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:49 smithi082 ceph-mon[98694]: slow request 393.685307 seconds old, received at 2023-09-09T23:16:14.768755+0000: client_request(mds.1:1988 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:49 smithi082 ceph-mon[98694]: slow request 393.626047 seconds old, received at 2023-09-09T23:16:14.828015+0000: client_request(mds.1:2020 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:49 smithi082 ceph-mon[98694]: slow request 393.572388 seconds old, received at 2023-09-09T23:16:14.881674+0000: client_request(mds.1:2052 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:50.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:49 smithi098 ceph-mon[103124]: 176 slow requests, 5 included below; oldest blocked for > 393.857962 secs 2023-09-09T23:22:50.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:49 smithi098 ceph-mon[103124]: slow request 393.798934 seconds old, received at 2023-09-09T23:16:14.655128+0000: client_request(mds.1:1924 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:49 smithi098 ceph-mon[103124]: slow request 393.744005 seconds old, received at 2023-09-09T23:16:14.710057+0000: client_request(mds.1:1956 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:49 smithi098 ceph-mon[103124]: slow request 393.685307 seconds old, received at 2023-09-09T23:16:14.768755+0000: client_request(mds.1:1988 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:49 smithi098 ceph-mon[103124]: slow request 393.626047 seconds old, received at 2023-09-09T23:16:14.828015+0000: client_request(mds.1:2020 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:49 smithi098 ceph-mon[103124]: slow request 393.572388 seconds old, received at 2023-09-09T23:16:14.881674+0000: client_request(mds.1:2052 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:50.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:49 smithi170 ceph-mon[102832]: 176 slow requests, 5 included below; oldest blocked for > 393.857962 secs 2023-09-09T23:22:50.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:49 smithi170 ceph-mon[102832]: slow request 393.798934 seconds old, received at 2023-09-09T23:16:14.655128+0000: client_request(mds.1:1924 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:50.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:49 smithi170 ceph-mon[102832]: slow request 393.744005 seconds old, received at 2023-09-09T23:16:14.710057+0000: client_request(mds.1:1956 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:50.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:49 smithi170 ceph-mon[102832]: slow request 393.685307 seconds old, received at 2023-09-09T23:16:14.768755+0000: client_request(mds.1:1988 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:49 smithi170 ceph-mon[102832]: slow request 393.626047 seconds old, received at 2023-09-09T23:16:14.828015+0000: client_request(mds.1:2020 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:49 smithi170 ceph-mon[102832]: slow request 393.572388 seconds old, received at 2023-09-09T23:16:14.881674+0000: client_request(mds.1:2052 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:50.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:50.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:50.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:50 smithi082 ceph-mon[98694]: pgmap v277: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-09-09T23:22:51.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:51.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:51.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:50 smithi098 ceph-mon[103124]: pgmap v277: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-09-09T23:22:51.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:51.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:51.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:50 smithi170 ceph-mon[102832]: pgmap v277: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-09-09T23:22:51.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:51.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:52.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:52.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:52.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:52.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:52.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:52 smithi082 ceph-mon[98694]: 144 slow requests, 5 included below; oldest blocked for > 385.694829 secs 2023-09-09T23:22:52.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:52 smithi082 ceph-mon[98694]: slow request 385.689324 seconds old, received at 2023-09-09T23:16:26.000857+0000: peer_request:mds.1:1934 currently dispatched 2023-09-09T23:22:52.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:52 smithi082 ceph-mon[98694]: slow request 385.677045 seconds old, received at 2023-09-09T23:16:26.013136+0000: peer_request:mds.1:1966 currently dispatched 2023-09-09T23:22:52.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:52 smithi082 ceph-mon[98694]: slow request 385.664990 seconds old, received at 2023-09-09T23:16:26.025191+0000: peer_request:mds.1:1998 currently dispatched 2023-09-09T23:22:52.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:52 smithi082 ceph-mon[98694]: slow request 385.653529 seconds old, received at 2023-09-09T23:16:26.036652+0000: peer_request:mds.1:2030 currently dispatched 2023-09-09T23:22:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:52 smithi082 ceph-mon[98694]: slow request 377.829390 seconds old, received at 2023-09-09T23:16:33.860791+0000: peer_request:mds.1:2062 currently dispatched 2023-09-09T23:22:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:52 smithi082 ceph-mon[98694]: pgmap v278: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:22:53.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:52 smithi098 ceph-mon[103124]: 144 slow requests, 5 included below; oldest blocked for > 385.694829 secs 2023-09-09T23:22:53.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:52 smithi098 ceph-mon[103124]: slow request 385.689324 seconds old, received at 2023-09-09T23:16:26.000857+0000: peer_request:mds.1:1934 currently dispatched 2023-09-09T23:22:53.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:52 smithi098 ceph-mon[103124]: slow request 385.677045 seconds old, received at 2023-09-09T23:16:26.013136+0000: peer_request:mds.1:1966 currently dispatched 2023-09-09T23:22:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:52 smithi098 ceph-mon[103124]: slow request 385.664990 seconds old, received at 2023-09-09T23:16:26.025191+0000: peer_request:mds.1:1998 currently dispatched 2023-09-09T23:22:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:52 smithi098 ceph-mon[103124]: slow request 385.653529 seconds old, received at 2023-09-09T23:16:26.036652+0000: peer_request:mds.1:2030 currently dispatched 2023-09-09T23:22:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:52 smithi098 ceph-mon[103124]: slow request 377.829390 seconds old, received at 2023-09-09T23:16:33.860791+0000: peer_request:mds.1:2062 currently dispatched 2023-09-09T23:22:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:52 smithi098 ceph-mon[103124]: pgmap v278: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:22:53.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:52 smithi170 ceph-mon[102832]: 144 slow requests, 5 included below; oldest blocked for > 385.694829 secs 2023-09-09T23:22:53.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:52 smithi170 ceph-mon[102832]: slow request 385.689324 seconds old, received at 2023-09-09T23:16:26.000857+0000: peer_request:mds.1:1934 currently dispatched 2023-09-09T23:22:53.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:52 smithi170 ceph-mon[102832]: slow request 385.677045 seconds old, received at 2023-09-09T23:16:26.013136+0000: peer_request:mds.1:1966 currently dispatched 2023-09-09T23:22:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:52 smithi170 ceph-mon[102832]: slow request 385.664990 seconds old, received at 2023-09-09T23:16:26.025191+0000: peer_request:mds.1:1998 currently dispatched 2023-09-09T23:22:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:52 smithi170 ceph-mon[102832]: slow request 385.653529 seconds old, received at 2023-09-09T23:16:26.036652+0000: peer_request:mds.1:2030 currently dispatched 2023-09-09T23:22:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:52 smithi170 ceph-mon[102832]: slow request 377.829390 seconds old, received at 2023-09-09T23:16:33.860791+0000: peer_request:mds.1:2062 currently dispatched 2023-09-09T23:22:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:52 smithi170 ceph-mon[102832]: pgmap v278: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:22:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 205.021245 secs 2023-09-09T23:22:54.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 205.021245 secs 2023-09-09T23:22:54.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 205.021245 secs 2023-09-09T23:22:54.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:54 smithi082 ceph-mon[98694]: 128 slow requests, 5 included below; oldest blocked for > 398.810502 secs 2023-09-09T23:22:54.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:54 smithi082 ceph-mon[98694]: slow request 398.776329 seconds old, received at 2023-09-09T23:16:14.711344+0000: client_request(mds.1:1957 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:54.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:54 smithi082 ceph-mon[98694]: slow request 398.717384 seconds old, received at 2023-09-09T23:16:14.770289+0000: client_request(mds.1:1989 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:54.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:54 smithi082 ceph-mon[98694]: slow request 398.658020 seconds old, received at 2023-09-09T23:16:14.829653+0000: client_request(mds.1:2021 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:54.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:54 smithi082 ceph-mon[98694]: slow request 398.604349 seconds old, received at 2023-09-09T23:16:14.883324+0000: client_request(mds.1:2053 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:54.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:54 smithi082 ceph-mon[98694]: slow request 398.774872 seconds old, received at 2023-09-09T23:16:14.712801+0000: client_request(mds.1:1958 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:54 smithi082 ceph-mon[98694]: pgmap v279: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:22:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:55.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:54 smithi098 ceph-mon[103124]: 128 slow requests, 5 included below; oldest blocked for > 398.810502 secs 2023-09-09T23:22:55.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:54 smithi098 ceph-mon[103124]: slow request 398.776329 seconds old, received at 2023-09-09T23:16:14.711344+0000: client_request(mds.1:1957 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:54 smithi098 ceph-mon[103124]: slow request 398.717384 seconds old, received at 2023-09-09T23:16:14.770289+0000: client_request(mds.1:1989 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:54 smithi098 ceph-mon[103124]: slow request 398.658020 seconds old, received at 2023-09-09T23:16:14.829653+0000: client_request(mds.1:2021 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:54 smithi098 ceph-mon[103124]: slow request 398.604349 seconds old, received at 2023-09-09T23:16:14.883324+0000: client_request(mds.1:2053 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:54 smithi098 ceph-mon[103124]: slow request 398.774872 seconds old, received at 2023-09-09T23:16:14.712801+0000: client_request(mds.1:1958 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:54 smithi098 ceph-mon[103124]: pgmap v279: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:22:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:55.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:54 smithi170 ceph-mon[102832]: 128 slow requests, 5 included below; oldest blocked for > 398.810502 secs 2023-09-09T23:22:55.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:54 smithi170 ceph-mon[102832]: slow request 398.776329 seconds old, received at 2023-09-09T23:16:14.711344+0000: client_request(mds.1:1957 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:55.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:54 smithi170 ceph-mon[102832]: slow request 398.717384 seconds old, received at 2023-09-09T23:16:14.770289+0000: client_request(mds.1:1989 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:54 smithi170 ceph-mon[102832]: slow request 398.658020 seconds old, received at 2023-09-09T23:16:14.829653+0000: client_request(mds.1:2021 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:54 smithi170 ceph-mon[102832]: slow request 398.604349 seconds old, received at 2023-09-09T23:16:14.883324+0000: client_request(mds.1:2053 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:54 smithi170 ceph-mon[102832]: slow request 398.774872 seconds old, received at 2023-09-09T23:16:14.712801+0000: client_request(mds.1:1958 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:54 smithi170 ceph-mon[102832]: pgmap v279: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:22:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:22:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:56.584 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:22:56.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:56.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:56.721 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.722+0000 7f7c4c80d700 1 -- 172.21.15.82:0/2050611623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c480ffbb0 msgr2=0x7f7c48100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:22:56.722 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.722+0000 7f7c4c80d700 1 --2- 172.21.15.82:0/2050611623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c480ffbb0 0x7f7c48100030 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f7c380099c0 tx=0x7f7c38009cd0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.722 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.722+0000 7f7c4c80d700 1 -- 172.21.15.82:0/2050611623 shutdown_connections 2023-09-09T23:22:56.722 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.722+0000 7f7c4c80d700 1 --2- 172.21.15.82:0/2050611623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c48100570 0x7f7c4810c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.722 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.722+0000 7f7c4c80d700 1 --2- 172.21.15.82:0/2050611623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c48105600 0x7f7c481059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.722 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.722+0000 7f7c4c80d700 1 --2- 172.21.15.82:0/2050611623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c480ffbb0 0x7f7c48100030 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.723 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.722+0000 7f7c4c80d700 1 -- 172.21.15.82:0/2050611623 >> 172.21.15.82:0/2050611623 conn(0x7f7c48076a70 msgr2=0x7f7c480fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:22:56.723 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.722+0000 7f7c4c80d700 1 -- 172.21.15.82:0/2050611623 shutdown_connections 2023-09-09T23:22:56.723 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.722+0000 7f7c4c80d700 1 -- 172.21.15.82:0/2050611623 wait complete. 2023-09-09T23:22:56.725 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.723+0000 7f7c4c80d700 1 Processor -- start 2023-09-09T23:22:56.725 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.723+0000 7f7c4c80d700 1 -- start start 2023-09-09T23:22:56.725 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.723+0000 7f7c4c80d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c480ffbb0 0x7f7c481974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:56.726 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.723+0000 7f7c4c80d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c48100570 0x7f7c48197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:56.726 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.723+0000 7f7c4c80d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c48105600 0x7f7c481a2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:56.726 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.723+0000 7f7c4c80d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c481a2a80 con 0x7f7c48105600 2023-09-09T23:22:56.726 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.723+0000 7f7c4c80d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c481a2bc0 con 0x7f7c48100570 2023-09-09T23:22:56.726 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.723+0000 7f7c4c80d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c481a2d00 con 0x7f7c480ffbb0 2023-09-09T23:22:56.727 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.723+0000 7f7c46d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c48105600 0x7f7c481a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:56.727 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.723+0000 7f7c46d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c48105600 0x7f7c481a2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58168/0 (socket says 172.21.15.82:58168) 2023-09-09T23:22:56.727 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.723+0000 7f7c46d9d700 1 -- 172.21.15.82:0/1631475923 learned_addr learned my addr 172.21.15.82:0/1631475923 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:22:56.727 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.723+0000 7f7c4659c700 1 --2- 172.21.15.82:0/1631475923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c480ffbb0 0x7f7c481974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:56.727 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.723+0000 7f7c45d9b700 1 --2- 172.21.15.82:0/1631475923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c48100570 0x7f7c48197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:56.728 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.723+0000 7f7c46d9d700 1 -- 172.21.15.82:0/1631475923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c480ffbb0 msgr2=0x7f7c481974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:22:56.728 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.723+0000 7f7c46d9d700 1 --2- 172.21.15.82:0/1631475923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c480ffbb0 0x7f7c481974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.728 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.723+0000 7f7c46d9d700 1 -- 172.21.15.82:0/1631475923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c48100570 msgr2=0x7f7c48197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:22:56.728 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.724+0000 7f7c46d9d700 1 --2- 172.21.15.82:0/1631475923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c48100570 0x7f7c48197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.728 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.724+0000 7f7c46d9d700 1 -- 172.21.15.82:0/1631475923 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c380095f0 con 0x7f7c48105600 2023-09-09T23:22:56.729 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.724+0000 7f7c46d9d700 1 --2- 172.21.15.82:0/1631475923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c48105600 0x7f7c481a2540 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000ef30 tx=0x7f7c4000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:22:56.729 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.724+0000 7f7c377fe700 1 -- 172.21.15.82:0/1631475923 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c40018070 con 0x7f7c48105600 2023-09-09T23:22:56.729 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.724+0000 7f7c377fe700 1 -- 172.21.15.82:0/1631475923 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c4000f040 con 0x7f7c48105600 2023-09-09T23:22:56.729 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.724+0000 7f7c4c80d700 1 -- 172.21.15.82:0/1631475923 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c481a2f00 con 0x7f7c48105600 2023-09-09T23:22:56.729 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.724+0000 7f7c377fe700 1 -- 172.21.15.82:0/1631475923 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4000cd00 con 0x7f7c48105600 2023-09-09T23:22:56.729 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.724+0000 7f7c4c80d700 1 -- 172.21.15.82:0/1631475923 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c481a3330 con 0x7f7c48105600 2023-09-09T23:22:56.730 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.726+0000 7f7c377fe700 1 -- 172.21.15.82:0/1631475923 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f7c40003a00 con 0x7f7c48105600 2023-09-09T23:22:56.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.727+0000 7f7c377fe700 1 --2- 172.21.15.82:0/1631475923 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f7c300a07b0 0x7f7c300a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:56.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.727+0000 7f7c4659c700 1 --2- 172.21.15.82:0/1631475923 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f7c300a07b0 0x7f7c300a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:56.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.727+0000 7f7c377fe700 1 -- 172.21.15.82:0/1631475923 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f7c400cbab0 con 0x7f7c48105600 2023-09-09T23:22:56.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.727+0000 7f7c4659c700 1 --2- 172.21.15.82:0/1631475923 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f7c300a07b0 0x7f7c300a2c70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7c48072ad0 tx=0x7f7c3c009260 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:22:56.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.727+0000 7f7c589d3b80 1 Processor -- start 2023-09-09T23:22:56.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.728+0000 7f7c589d3b80 1 -- start start 2023-09-09T23:22:56.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.728+0000 7f7c589d3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aeb4e03710 0x55aeb4e03af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:56.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.728+0000 7f7c589d3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aeb4e04030 0x55aeb4e11d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:56.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.728+0000 7f7c589d3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aeb4e12250 0x55aeb4e14640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:56.738 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.728+0000 7f7c589d3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb4d5c730 con 0x55aeb4e03710 2023-09-09T23:22:56.738 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.728+0000 7f7c589d3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb4d5a050 con 0x55aeb4e04030 2023-09-09T23:22:56.738 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.728+0000 7f7c589d3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb4d55940 con 0x55aeb4e12250 2023-09-09T23:22:56.738 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.728+0000 7f7c4659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aeb4e03710 0x55aeb4e03af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:56.738 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.728+0000 7f7c4659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aeb4e03710 0x55aeb4e03af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58172/0 (socket says 172.21.15.82:58172) 2023-09-09T23:22:56.738 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.728+0000 7f7c4659c700 1 -- 172.21.15.82:0/303942703 learned_addr learned my addr 172.21.15.82:0/303942703 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:22:56.739 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.728+0000 7f7c45d9b700 1 --2- 172.21.15.82:0/303942703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aeb4e04030 0x55aeb4e11d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:56.739 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.728+0000 7f7c46d9d700 1 --2- 172.21.15.82:0/303942703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aeb4e12250 0x55aeb4e14640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:56.739 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.728+0000 7f7c4659c700 1 -- 172.21.15.82:0/303942703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aeb4e12250 msgr2=0x55aeb4e14640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:22:56.739 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.728+0000 7f7c4659c700 1 --2- 172.21.15.82:0/303942703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aeb4e12250 0x55aeb4e14640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.728+0000 7f7c4659c700 1 -- 172.21.15.82:0/303942703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aeb4e04030 msgr2=0x55aeb4e11d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:22:56.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.728+0000 7f7c4659c700 1 --2- 172.21.15.82:0/303942703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aeb4e04030 0x55aeb4e11d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.728+0000 7f7c4659c700 1 -- 172.21.15.82:0/303942703 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aeb4b81de0 con 0x55aeb4e03710 2023-09-09T23:22:56.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.728+0000 7f7c4659c700 1 --2- 172.21.15.82:0/303942703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aeb4e03710 0x55aeb4e03af0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c005950 tx=0x7f7c3c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6938c1bcf58d6bcf server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:22:56.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.729+0000 7f7c34ff9700 1 -- 172.21.15.82:0/303942703 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c004220 con 0x55aeb4e03710 2023-09-09T23:22:56.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.729+0000 7f7c34ff9700 1 -- 172.21.15.82:0/303942703 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c3c021070 con 0x55aeb4e03710 2023-09-09T23:22:56.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.729+0000 7f7c34ff9700 1 -- 172.21.15.82:0/303942703 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c003c40 con 0x55aeb4e03710 2023-09-09T23:22:56.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.729+0000 7f7c589d3b80 1 -- 172.21.15.82:0/303942703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aeb4e03710 msgr2=0x55aeb4e03af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:22:56.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.729+0000 7f7c589d3b80 1 --2- 172.21.15.82:0/303942703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aeb4e03710 0x55aeb4e03af0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c005950 tx=0x7f7c3c014000 comp rx=0 tx=0).stop 2023-09-09T23:22:56.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.729+0000 7f7c589d3b80 1 -- 172.21.15.82:0/303942703 shutdown_connections 2023-09-09T23:22:56.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.729+0000 7f7c589d3b80 1 --2- 172.21.15.82:0/303942703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aeb4e12250 0x55aeb4e14640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.729+0000 7f7c589d3b80 1 --2- 172.21.15.82:0/303942703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aeb4e03710 0x55aeb4e03af0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.729+0000 7f7c589d3b80 1 --2- 172.21.15.82:0/303942703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aeb4e04030 0x55aeb4e11d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.729+0000 7f7c589d3b80 1 -- 172.21.15.82:0/303942703 >> 172.21.15.82:0/303942703 conn(0x55aeb4d78090 msgr2=0x55aeb4dfebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:22:56.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.729+0000 7f7c589d3b80 1 -- 172.21.15.82:0/303942703 shutdown_connections 2023-09-09T23:22:56.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.729+0000 7f7c589d3b80 1 -- 172.21.15.82:0/303942703 wait complete. 2023-09-09T23:22:56.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.730+0000 7f7c589d3b80 1 -- start start 2023-09-09T23:22:56.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.730+0000 7f7c589d3b80 1 Processor -- start 2023-09-09T23:22:56.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.730+0000 7f7c589d3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aeb4e03710 0x55aeb4e11c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:56.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.730+0000 7f7c589d3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aeb4e04030 0x55aeb4e0ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:56.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.730+0000 7f7c589d3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aeb4e12250 0x55aeb4e0f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:56.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.730+0000 7f7c589d3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb4cf3b70 con 0x55aeb4e03710 2023-09-09T23:22:56.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.730+0000 7f7c589d3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb4cf3870 con 0x55aeb4e04030 2023-09-09T23:22:56.745 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.730+0000 7f7c589d3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb4bd7920 con 0x55aeb4e12250 2023-09-09T23:22:56.745 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.730+0000 7f7c589d3b80 10 client.0 _ll_register_callbacks cb 0x55aeb4c3da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:22:56.745 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.730+0000 7f7c4659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aeb4e03710 0x55aeb4e11c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:56.745 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.730+0000 7f7c4659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aeb4e03710 0x55aeb4e11c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58178/0 (socket says 172.21.15.82:58178) 2023-09-09T23:22:56.745 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.730+0000 7f7c4659c700 1 -- 172.21.15.82:0/2638605412 learned_addr learned my addr 172.21.15.82:0/2638605412 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:22:56.746 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.730+0000 7f7c46d9d700 1 --2- 172.21.15.82:0/2638605412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aeb4e12250 0x55aeb4e0f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:56.746 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.730+0000 7f7c45d9b700 1 --2- 172.21.15.82:0/2638605412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aeb4e04030 0x55aeb4e0ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:56.746 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.730+0000 7f7c4659c700 1 -- 172.21.15.82:0/2638605412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aeb4e12250 msgr2=0x55aeb4e0f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:22:56.746 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.730+0000 7f7c4659c700 1 --2- 172.21.15.82:0/2638605412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aeb4e12250 0x55aeb4e0f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.746 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.730+0000 7f7c4659c700 1 -- 172.21.15.82:0/2638605412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aeb4e04030 msgr2=0x55aeb4e0ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:22:56.747 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.730+0000 7f7c4659c700 1 --2- 172.21.15.82:0/2638605412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aeb4e04030 0x55aeb4e0ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.747 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.730+0000 7f7c4659c700 1 -- 172.21.15.82:0/2638605412 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c3c017070 con 0x55aeb4e03710 2023-09-09T23:22:56.747 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.730+0000 7f7c45d9b700 1 --2- 172.21.15.82:0/2638605412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aeb4e04030 0x55aeb4e0ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:22:56.747 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.731+0000 7f7c4659c700 1 --2- 172.21.15.82:0/2638605412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aeb4e03710 0x55aeb4e11c80 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009fd0 tx=0x7f7c3c00e8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcdb7d7928aae4a8 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:22:56.747 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.731+0000 7f7c2dffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:22:56.747 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.731+0000 7f7c2dffb700 1 -- 172.21.15.82:0/2638605412 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c026480 con 0x55aeb4e03710 2023-09-09T23:22:56.748 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.731+0000 7f7c2dffb700 1 -- 172.21.15.82:0/2638605412 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f7c3c0265e0 con 0x55aeb4e03710 2023-09-09T23:22:56.748 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.731+0000 7f7c589d3b80 1 -- 172.21.15.82:0/2638605412 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aeb4cad4f0 con 0x55aeb4e03710 2023-09-09T23:22:56.748 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.731+0000 7f7c2dffb700 1 -- 172.21.15.82:0/2638605412 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=123) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c3c0267a0 con 0x55aeb4e03710 2023-09-09T23:22:56.748 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.731+0000 7f7c589d3b80 10 client.16245 fetch_fsmap learned FSMap version 123 2023-09-09T23:22:56.748 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.731+0000 7f7c589d3b80 1 -- 172.21.15.82:0/2638605412 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=123,monmap=4+}) v3 -- 0x55aeb4b6ac60 con 0x55aeb4e03710 2023-09-09T23:22:56.749 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.731+0000 7f7c2dffb700 1 -- 172.21.15.82:0/2638605412 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aeb4cad4f0 con 0x55aeb4e03710 2023-09-09T23:22:56.749 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.731+0000 7f7c2dffb700 1 -- 172.21.15.82:0/2638605412 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 123) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7c3c01d980 con 0x55aeb4e03710 2023-09-09T23:22:56.749 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.731+0000 7f7c589d3b80 10 client.16245 fetch_fsmap finished waiting for FSMap version 123 2023-09-09T23:22:56.749 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.731+0000 7f7c589d3b80 10 client.16245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:22:56.749 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.731+0000 7f7c589d3b80 20 client.16245 populate_metadata read hostname 'smithi082' 2023-09-09T23:22:56.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.731+0000 7f7c589d3b80 1 --2- 172.21.15.82:0/2638605412 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55aeb4e21b80 0x55aeb4e23f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:56.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.731+0000 7f7c589d3b80 4 client.16245 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:22:56.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.731+0000 7f7c589d3b80 1 -- 172.21.15.82:0/2638605412 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aeb4d6a8a0 con 0x55aeb4e21b80 2023-09-09T23:22:56.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.732+0000 7f7c45d9b700 1 --2- 172.21.15.82:0/2638605412 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55aeb4e21b80 0x55aeb4e23f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:56.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.732+0000 7f7c2dffb700 10 client.16245 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:22:56.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.732+0000 7f7c45d9b700 1 --2- 172.21.15.82:0/2638605412 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55aeb4e21b80 0x55aeb4e23f70 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=297a9e373c688ebe server_cookie=7d6eb76169d9af97 in_seq=0 out_seq=0 2023-09-09T23:22:56.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.733+0000 7f7c2dffb700 1 -- 172.21.15.82:0/2638605412 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55aeb4d6a8a0 con 0x55aeb4e21b80 2023-09-09T23:22:56.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.733+0000 7f7c2dffb700 10 client.16245 handle_command_reply: tid=0 2023-09-09T23:22:56.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.733+0000 7f7c589d3b80 1 client.16245 shutdown 2023-09-09T23:22:56.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.733+0000 7f7c589d3b80 1 -- 172.21.15.82:0/2638605412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aeb4e03710 msgr2=0x55aeb4e11c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:22:56.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.733+0000 7f7c589d3b80 1 --2- 172.21.15.82:0/2638605412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aeb4e03710 0x55aeb4e11c80 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009fd0 tx=0x7f7c3c00e8f0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.752 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.733+0000 7f7c589d3b80 1 -- 172.21.15.82:0/2638605412 shutdown_connections 2023-09-09T23:22:56.752 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.733+0000 7f7c589d3b80 1 --2- 172.21.15.82:0/2638605412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aeb4e12250 0x55aeb4e0f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.752 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.733+0000 7f7c589d3b80 1 --2- 172.21.15.82:0/2638605412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aeb4e03710 0x55aeb4e11c80 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.752 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.733+0000 7f7c589d3b80 1 --2- 172.21.15.82:0/2638605412 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55aeb4e21b80 0x55aeb4e23f70 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.752 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.733+0000 7f7c589d3b80 1 --2- 172.21.15.82:0/2638605412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aeb4e04030 0x55aeb4e0ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.733+0000 7f7c2dffb700 0 client.16245 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:22:56.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.733+0000 7f7c589d3b80 1 -- 172.21.15.82:0/2638605412 >> 172.21.15.82:0/2638605412 conn(0x55aeb4d78090 msgr2=0x55aeb4dfe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:22:56.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.733+0000 7f7c589d3b80 1 -- 172.21.15.82:0/2638605412 shutdown_connections 2023-09-09T23:22:56.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.733+0000 7f7c589d3b80 1 -- 172.21.15.82:0/2638605412 wait complete. 2023-09-09T23:22:56.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.733+0000 7f7c589d3b80 20 client.16245 trim_cache size 0 max 16384 2023-09-09T23:22:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:56.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.750+0000 7f7c589d3b80 1 Processor -- start 2023-09-09T23:22:56.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.750+0000 7f7c589d3b80 1 -- start start 2023-09-09T23:22:56.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.750+0000 7f7c589d3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aeb4e03710 0x55aeb4e117b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:56.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.750+0000 7f7c589d3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aeb4e04030 0x55aeb4e0e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:56.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.750+0000 7f7c589d3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aeb4e12250 0x55aeb4e0ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:56.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.750+0000 7f7c589d3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb4dfb410 con 0x55aeb4e03710 2023-09-09T23:22:56.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.750+0000 7f7c589d3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb4e09cb0 con 0x55aeb4e04030 2023-09-09T23:22:56.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.750+0000 7f7c589d3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb4e0f1b0 con 0x55aeb4e12250 2023-09-09T23:22:56.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.750+0000 7f7c4659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aeb4e03710 0x55aeb4e117b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:56.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.750+0000 7f7c4659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aeb4e03710 0x55aeb4e117b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58186/0 (socket says 172.21.15.82:58186) 2023-09-09T23:22:56.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.750+0000 7f7c4659c700 1 -- 172.21.15.82:0/2080069310 learned_addr learned my addr 172.21.15.82:0/2080069310 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:22:56.762 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.750+0000 7f7c45d9b700 1 --2- 172.21.15.82:0/2080069310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aeb4e04030 0x55aeb4e0e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:56.762 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.750+0000 7f7c46d9d700 1 --2- 172.21.15.82:0/2080069310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aeb4e12250 0x55aeb4e0ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:56.762 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.750+0000 7f7c4659c700 1 -- 172.21.15.82:0/2080069310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aeb4e12250 msgr2=0x55aeb4e0ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:22:56.762 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.750+0000 7f7c4659c700 1 --2- 172.21.15.82:0/2080069310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aeb4e12250 0x55aeb4e0ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.762 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.750+0000 7f7c4659c700 1 -- 172.21.15.82:0/2080069310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aeb4e04030 msgr2=0x55aeb4e0e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:22:56.763 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.750+0000 7f7c4659c700 1 --2- 172.21.15.82:0/2080069310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aeb4e04030 0x55aeb4e0e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.763 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.750+0000 7f7c4659c700 1 -- 172.21.15.82:0/2080069310 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aeb4ba8250 con 0x55aeb4e03710 2023-09-09T23:22:56.763 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.751+0000 7f7c46d9d700 1 --2- 172.21.15.82:0/2080069310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aeb4e12250 0x55aeb4e0ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:22:56.763 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.751+0000 7f7c45d9b700 1 --2- 172.21.15.82:0/2080069310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aeb4e04030 0x55aeb4e0e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:22:56.763 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.751+0000 7f7c4659c700 1 --2- 172.21.15.82:0/2080069310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aeb4e03710 0x55aeb4e117b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c011cf0 tx=0x7f7c3c011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abaa9b0b6caa711a server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:22:56.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.751+0000 7f7c357fa700 1 -- 172.21.15.82:0/2080069310 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c00e690 con 0x55aeb4e03710 2023-09-09T23:22:56.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.751+0000 7f7c357fa700 1 -- 172.21.15.82:0/2080069310 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c3c00ed50 con 0x55aeb4e03710 2023-09-09T23:22:56.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.751+0000 7f7c357fa700 1 -- 172.21.15.82:0/2080069310 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c01cb50 con 0x55aeb4e03710 2023-09-09T23:22:56.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.751+0000 7f7c589d3b80 1 -- 172.21.15.82:0/2080069310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aeb4e03710 msgr2=0x55aeb4e117b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:22:56.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.751+0000 7f7c589d3b80 1 --2- 172.21.15.82:0/2080069310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aeb4e03710 0x55aeb4e117b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c011cf0 tx=0x7f7c3c011a90 comp rx=0 tx=0).stop 2023-09-09T23:22:56.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.751+0000 7f7c589d3b80 1 -- 172.21.15.82:0/2080069310 shutdown_connections 2023-09-09T23:22:56.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.751+0000 7f7c589d3b80 1 --2- 172.21.15.82:0/2080069310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aeb4e12250 0x55aeb4e0ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.751+0000 7f7c589d3b80 1 --2- 172.21.15.82:0/2080069310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aeb4e03710 0x55aeb4e117b0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.751+0000 7f7c589d3b80 1 --2- 172.21.15.82:0/2080069310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aeb4e04030 0x55aeb4e0e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.751+0000 7f7c589d3b80 1 -- 172.21.15.82:0/2080069310 >> 172.21.15.82:0/2080069310 conn(0x55aeb4d78090 msgr2=0x55aeb4e0a170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:22:56.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.751+0000 7f7c589d3b80 1 -- 172.21.15.82:0/2080069310 shutdown_connections 2023-09-09T23:22:56.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.751+0000 7f7c589d3b80 1 -- 172.21.15.82:0/2080069310 wait complete. 2023-09-09T23:22:56.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.752+0000 7f7c589d3b80 1 -- start start 2023-09-09T23:22:56.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.752+0000 7f7c589d3b80 1 Processor -- start 2023-09-09T23:22:56.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.752+0000 7f7c589d3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aeb4e03710 0x55aeb4e18e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:56.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.752+0000 7f7c589d3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aeb4e04030 0x55aeb4e17150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:56.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.752+0000 7f7c589d3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aeb4e12250 0x55aeb4e176b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:56.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.752+0000 7f7c589d3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb4e17dc0 con 0x55aeb4e03710 2023-09-09T23:22:56.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.752+0000 7f7c589d3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb4e17f30 con 0x55aeb4e04030 2023-09-09T23:22:56.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.752+0000 7f7c589d3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb4e75980 con 0x55aeb4e12250 2023-09-09T23:22:56.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.752+0000 7f7c589d3b80 10 client.0 _ll_register_callbacks cb 0x7f7c400097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:22:56.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.752+0000 7f7c4659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aeb4e03710 0x55aeb4e18e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:56.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.752+0000 7f7c4659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aeb4e03710 0x55aeb4e18e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58192/0 (socket says 172.21.15.82:58192) 2023-09-09T23:22:56.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.752+0000 7f7c4659c700 1 -- 172.21.15.82:0/1319622763 learned_addr learned my addr 172.21.15.82:0/1319622763 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:22:56.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.752+0000 7f7c45d9b700 1 --2- 172.21.15.82:0/1319622763 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aeb4e04030 0x55aeb4e17150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:56.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.752+0000 7f7c46d9d700 1 --2- 172.21.15.82:0/1319622763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aeb4e12250 0x55aeb4e176b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:56.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.752+0000 7f7c4659c700 1 -- 172.21.15.82:0/1319622763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aeb4e12250 msgr2=0x55aeb4e176b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:22:56.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.752+0000 7f7c4659c700 1 --2- 172.21.15.82:0/1319622763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aeb4e12250 0x55aeb4e176b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.752+0000 7f7c4659c700 1 -- 172.21.15.82:0/1319622763 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aeb4e04030 msgr2=0x55aeb4e17150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:22:56.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.752+0000 7f7c4659c700 1 --2- 172.21.15.82:0/1319622763 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aeb4e04030 0x55aeb4e17150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.752+0000 7f7c4659c700 1 -- 172.21.15.82:0/1319622763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aeb4b6ac60 con 0x55aeb4e03710 2023-09-09T23:22:56.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.753+0000 7f7c4659c700 1 --2- 172.21.15.82:0/1319622763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aeb4e03710 0x55aeb4e18e90 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c011b60 tx=0x7f7c3c005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f47da70cf8d5967 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:22:56.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.753+0000 7f7c2dffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:22:56.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.753+0000 7f7c2dffb700 1 -- 172.21.15.82:0/1319622763 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c01d610 con 0x55aeb4e03710 2023-09-09T23:22:56.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.753+0000 7f7c2dffb700 1 -- 172.21.15.82:0/1319622763 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f7c3c01d770 con 0x55aeb4e03710 2023-09-09T23:22:56.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.753+0000 7f7c589d3b80 1 -- 172.21.15.82:0/1319622763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c3c021070 con 0x55aeb4e03710 2023-09-09T23:22:56.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.753+0000 7f7c2dffb700 1 -- 172.21.15.82:0/1319622763 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=123) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c3c01da50 con 0x55aeb4e03710 2023-09-09T23:22:56.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.753+0000 7f7c589d3b80 10 client.16251 fetch_fsmap learned FSMap version 123 2023-09-09T23:22:56.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.753+0000 7f7c589d3b80 1 -- 172.21.15.82:0/1319622763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=123,monmap=4+}) v3 -- 0x55aeb4b802b0 con 0x55aeb4e03710 2023-09-09T23:22:56.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.753+0000 7f7c2dffb700 1 -- 172.21.15.82:0/1319622763 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c3c021070 con 0x55aeb4e03710 2023-09-09T23:22:56.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.753+0000 7f7c2dffb700 1 -- 172.21.15.82:0/1319622763 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 123) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7c3c042070 con 0x55aeb4e03710 2023-09-09T23:22:56.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.753+0000 7f7c589d3b80 10 client.16251 fetch_fsmap finished waiting for FSMap version 123 2023-09-09T23:22:56.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.753+0000 7f7c589d3b80 10 client.16251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:22:56.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.753+0000 7f7c589d3b80 20 client.16251 populate_metadata read hostname 'smithi082' 2023-09-09T23:22:56.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.753+0000 7f7c589d3b80 1 --2- 172.21.15.82:0/1319622763 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55aeb4e76bc0 0x55aeb4e78fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:22:56.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.753+0000 7f7c589d3b80 4 client.16251 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:22:56.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.753+0000 7f7c589d3b80 1 -- 172.21.15.82:0/1319622763 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55aeb4d4e920 con 0x55aeb4e76bc0 2023-09-09T23:22:56.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.753+0000 7f7c45d9b700 1 --2- 172.21.15.82:0/1319622763 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55aeb4e76bc0 0x55aeb4e78fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:22:56.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.754+0000 7f7c2dffb700 10 client.16251 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:22:56.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.754+0000 7f7c45d9b700 1 --2- 172.21.15.82:0/1319622763 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55aeb4e76bc0 0x55aeb4e78fb0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb798e72c50b8845 server_cookie=2d53aaf9eaae0c0f in_seq=0 out_seq=0 2023-09-09T23:22:56.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.754+0000 7f7c2dffb700 1 -- 172.21.15.82:0/1319622763 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55aeb4d4e920 con 0x55aeb4e76bc0 2023-09-09T23:22:56.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.754+0000 7f7c2dffb700 10 client.16251 handle_command_reply: tid=0 2023-09-09T23:22:56.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.754+0000 7f7c589d3b80 1 client.16251 shutdown 2023-09-09T23:22:56.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.754+0000 7f7c589d3b80 1 -- 172.21.15.82:0/1319622763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aeb4e03710 msgr2=0x55aeb4e18e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:22:56.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.754+0000 7f7c589d3b80 1 --2- 172.21.15.82:0/1319622763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aeb4e03710 0x55aeb4e18e90 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c011b60 tx=0x7f7c3c005720 comp rx=0 tx=0).stop 2023-09-09T23:22:56.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.754+0000 7f7c589d3b80 1 -- 172.21.15.82:0/1319622763 shutdown_connections 2023-09-09T23:22:56.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.754+0000 7f7c589d3b80 1 --2- 172.21.15.82:0/1319622763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aeb4e12250 0x55aeb4e176b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.754+0000 7f7c589d3b80 1 --2- 172.21.15.82:0/1319622763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aeb4e03710 0x55aeb4e18e90 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.754+0000 7f7c589d3b80 1 --2- 172.21.15.82:0/1319622763 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55aeb4e76bc0 0x55aeb4e78fb0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.754+0000 7f7c589d3b80 1 --2- 172.21.15.82:0/1319622763 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aeb4e04030 0x55aeb4e17150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.754+0000 7f7c2dffb700 0 client.16251 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:22:56.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.754+0000 7f7c589d3b80 1 -- 172.21.15.82:0/1319622763 >> 172.21.15.82:0/1319622763 conn(0x55aeb4d78090 msgr2=0x55aeb4e02700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:22:56.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.754+0000 7f7c589d3b80 1 -- 172.21.15.82:0/1319622763 shutdown_connections 2023-09-09T23:22:56.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.754+0000 7f7c589d3b80 1 -- 172.21.15.82:0/1319622763 wait complete. 2023-09-09T23:22:56.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.755+0000 7f7c589d3b80 20 client.16251 trim_cache size 0 max 16384 2023-09-09T23:22:56.776 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:22:56.776 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:22:56.777 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:22:56.777 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:22:56.777 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:22:56.777 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:22:56.777 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:22:56.777 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:22:56.778 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:22:56.778 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:22:56.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.755+0000 7f7c4c80d700 1 -- 172.21.15.82:0/1631475923 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f7c300a07b0 msgr2=0x7f7c300a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:22:56.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.755+0000 7f7c4c80d700 1 --2- 172.21.15.82:0/1631475923 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f7c300a07b0 0x7f7c300a2c70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7c48072ad0 tx=0x7f7c3c009260 comp rx=0 tx=0).stop 2023-09-09T23:22:56.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.755+0000 7f7c4c80d700 1 -- 172.21.15.82:0/1631475923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c48105600 msgr2=0x7f7c481a2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:22:56.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.755+0000 7f7c4c80d700 1 --2- 172.21.15.82:0/1631475923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c48105600 0x7f7c481a2540 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000ef30 tx=0x7f7c4000c3a0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.755+0000 7f7c4c80d700 1 -- 172.21.15.82:0/1631475923 shutdown_connections 2023-09-09T23:22:56.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.755+0000 7f7c4c80d700 1 --2- 172.21.15.82:0/1631475923 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f7c300a07b0 0x7f7c300a2c70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.755+0000 7f7c4c80d700 1 --2- 172.21.15.82:0/1631475923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c48105600 0x7f7c481a2540 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.755+0000 7f7c4c80d700 1 --2- 172.21.15.82:0/1631475923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c480ffbb0 0x7f7c481974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.755+0000 7f7c4c80d700 1 --2- 172.21.15.82:0/1631475923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c48100570 0x7f7c48197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:22:56.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.755+0000 7f7c4c80d700 1 -- 172.21.15.82:0/1631475923 >> 172.21.15.82:0/1631475923 conn(0x7f7c48076a70 msgr2=0x7f7c480fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:22:56.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.755+0000 7f7c4c80d700 1 -- 172.21.15.82:0/1631475923 shutdown_connections 2023-09-09T23:22:56.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:22:56.755+0000 7f7c4c80d700 1 -- 172.21.15.82:0/1631475923 wait complete. 2023-09-09T23:22:56.782 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:22:56.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:22:56.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:22:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:57 smithi082 ceph-mon[98694]: pgmap v280: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 13 op/s 2023-09-09T23:22:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:57 smithi082 ceph-mon[98694]: 84 slow requests, 5 included below; oldest blocked for > 390.672313 secs 2023-09-09T23:22:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:57 smithi082 ceph-mon[98694]: slow request 390.664707 seconds old, received at 2023-09-09T23:16:26.025536+0000: peer_request:mds.1:1999 currently dispatched 2023-09-09T23:22:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:57 smithi082 ceph-mon[98694]: slow request 390.653316 seconds old, received at 2023-09-09T23:16:26.036927+0000: peer_request:mds.1:2031 currently dispatched 2023-09-09T23:22:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:57 smithi082 ceph-mon[98694]: slow request 382.829128 seconds old, received at 2023-09-09T23:16:33.861115+0000: peer_request:mds.1:2063 currently dispatched 2023-09-09T23:22:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:57 smithi082 ceph-mon[98694]: slow request 390.664320 seconds old, received at 2023-09-09T23:16:26.025924+0000: peer_request:mds.1:2000 currently dispatched 2023-09-09T23:22:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:57 smithi082 ceph-mon[98694]: slow request 390.652887 seconds old, received at 2023-09-09T23:16:26.037357+0000: peer_request:mds.1:2032 currently dispatched 2023-09-09T23:22:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:57 smithi098 ceph-mon[103124]: pgmap v280: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 13 op/s 2023-09-09T23:22:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:57 smithi098 ceph-mon[103124]: 84 slow requests, 5 included below; oldest blocked for > 390.672313 secs 2023-09-09T23:22:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:57 smithi098 ceph-mon[103124]: slow request 390.664707 seconds old, received at 2023-09-09T23:16:26.025536+0000: peer_request:mds.1:1999 currently dispatched 2023-09-09T23:22:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:57 smithi098 ceph-mon[103124]: slow request 390.653316 seconds old, received at 2023-09-09T23:16:26.036927+0000: peer_request:mds.1:2031 currently dispatched 2023-09-09T23:22:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:57 smithi098 ceph-mon[103124]: slow request 382.829128 seconds old, received at 2023-09-09T23:16:33.861115+0000: peer_request:mds.1:2063 currently dispatched 2023-09-09T23:22:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:57 smithi098 ceph-mon[103124]: slow request 390.664320 seconds old, received at 2023-09-09T23:16:26.025924+0000: peer_request:mds.1:2000 currently dispatched 2023-09-09T23:22:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:57 smithi098 ceph-mon[103124]: slow request 390.652887 seconds old, received at 2023-09-09T23:16:26.037357+0000: peer_request:mds.1:2032 currently dispatched 2023-09-09T23:22:57.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:57 smithi170 ceph-mon[102832]: pgmap v280: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 13 op/s 2023-09-09T23:22:57.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:57 smithi170 ceph-mon[102832]: 84 slow requests, 5 included below; oldest blocked for > 390.672313 secs 2023-09-09T23:22:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:57 smithi170 ceph-mon[102832]: slow request 390.664707 seconds old, received at 2023-09-09T23:16:26.025536+0000: peer_request:mds.1:1999 currently dispatched 2023-09-09T23:22:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:57 smithi170 ceph-mon[102832]: slow request 390.653316 seconds old, received at 2023-09-09T23:16:26.036927+0000: peer_request:mds.1:2031 currently dispatched 2023-09-09T23:22:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:57 smithi170 ceph-mon[102832]: slow request 382.829128 seconds old, received at 2023-09-09T23:16:33.861115+0000: peer_request:mds.1:2063 currently dispatched 2023-09-09T23:22:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:57 smithi170 ceph-mon[102832]: slow request 390.664320 seconds old, received at 2023-09-09T23:16:26.025924+0000: peer_request:mds.1:2000 currently dispatched 2023-09-09T23:22:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:57 smithi170 ceph-mon[102832]: slow request 390.652887 seconds old, received at 2023-09-09T23:16:26.037357+0000: peer_request:mds.1:2032 currently dispatched 2023-09-09T23:22:58.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 210.021302 secs 2023-09-09T23:22:58.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 210.021302 secs 2023-09-09T23:22:58.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 210.021302 secs 2023-09-09T23:22:59.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:59 smithi082 ceph-mon[98694]: pgmap v281: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 9 op/s 2023-09-09T23:22:59.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:59 smithi082 ceph-mon[98694]: 48 slow requests, 5 included below; oldest blocked for > 403.667886 secs 2023-09-09T23:22:59.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:59 smithi082 ceph-mon[98694]: slow request 403.659834 seconds old, received at 2023-09-09T23:16:14.831296+0000: client_request(mds.1:2022 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:59.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:59 smithi082 ceph-mon[98694]: slow request 403.605888 seconds old, received at 2023-09-09T23:16:14.885243+0000: client_request(mds.1:2054 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:59 smithi082 ceph-mon[98694]: slow request 403.658183 seconds old, received at 2023-09-09T23:16:14.832948+0000: client_request(mds.1:2023 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:59 smithi082 ceph-mon[98694]: slow request 403.604055 seconds old, received at 2023-09-09T23:16:14.887076+0000: client_request(mds.1:2055 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:22:59 smithi082 ceph-mon[98694]: slow request 403.656670 seconds old, received at 2023-09-09T23:16:14.834461+0000: client_request(mds.1:2024 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:59.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:59 smithi098 ceph-mon[103124]: pgmap v281: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 9 op/s 2023-09-09T23:22:59.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:59 smithi098 ceph-mon[103124]: 48 slow requests, 5 included below; oldest blocked for > 403.667886 secs 2023-09-09T23:22:59.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:59 smithi098 ceph-mon[103124]: slow request 403.659834 seconds old, received at 2023-09-09T23:16:14.831296+0000: client_request(mds.1:2022 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:59 smithi098 ceph-mon[103124]: slow request 403.605888 seconds old, received at 2023-09-09T23:16:14.885243+0000: client_request(mds.1:2054 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:59 smithi098 ceph-mon[103124]: slow request 403.658183 seconds old, received at 2023-09-09T23:16:14.832948+0000: client_request(mds.1:2023 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:59 smithi098 ceph-mon[103124]: slow request 403.604055 seconds old, received at 2023-09-09T23:16:14.887076+0000: client_request(mds.1:2055 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:22:59 smithi098 ceph-mon[103124]: slow request 403.656670 seconds old, received at 2023-09-09T23:16:14.834461+0000: client_request(mds.1:2024 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:59.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:59 smithi170 ceph-mon[102832]: pgmap v281: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 9 op/s 2023-09-09T23:22:59.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:59 smithi170 ceph-mon[102832]: 48 slow requests, 5 included below; oldest blocked for > 403.667886 secs 2023-09-09T23:22:59.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:59 smithi170 ceph-mon[102832]: slow request 403.659834 seconds old, received at 2023-09-09T23:16:14.831296+0000: client_request(mds.1:2022 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:59 smithi170 ceph-mon[102832]: slow request 403.605888 seconds old, received at 2023-09-09T23:16:14.885243+0000: client_request(mds.1:2054 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:59 smithi170 ceph-mon[102832]: slow request 403.658183 seconds old, received at 2023-09-09T23:16:14.832948+0000: client_request(mds.1:2023 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:59 smithi170 ceph-mon[102832]: slow request 403.604055 seconds old, received at 2023-09-09T23:16:14.887076+0000: client_request(mds.1:2055 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:22:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:22:59 smithi170 ceph-mon[102832]: slow request 403.656670 seconds old, received at 2023-09-09T23:16:14.834461+0000: client_request(mds.1:2024 rename #0x10000000007/00000001000000010000003A #0x60d/20000000229 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2023-09-09T23:23:00.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:23:00.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:23:00.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:23:00.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:23:00.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:23:00.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:23:01.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:01 smithi082 ceph-mon[98694]: pgmap v282: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 29 KiB/s wr, 15 op/s 2023-09-09T23:23:01.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:01 smithi098 ceph-mon[103124]: pgmap v282: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 29 KiB/s wr, 15 op/s 2023-09-09T23:23:01.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:01 smithi170 ceph-mon[102832]: pgmap v282: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 29 KiB/s wr, 15 op/s 2023-09-09T23:23:02.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:02 smithi098 ceph-mon[103124]: 1 slow requests, 1 included below; oldest blocked for > 213.386822 secs 2023-09-09T23:23:02.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:02 smithi098 ceph-mon[103124]: slow request 213.386822 seconds old, received at 2023-09-09T23:19:28.303507+0000: peer_request:mds.4:47 currently dispatched 2023-09-09T23:23:02.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:02 smithi170 ceph-mon[102832]: 1 slow requests, 1 included below; oldest blocked for > 213.386822 secs 2023-09-09T23:23:02.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:02 smithi170 ceph-mon[102832]: slow request 213.386822 seconds old, received at 2023-09-09T23:19:28.303507+0000: peer_request:mds.4:47 currently dispatched 2023-09-09T23:23:02.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:02 smithi082 ceph-mon[98694]: 1 slow requests, 1 included below; oldest blocked for > 213.386822 secs 2023-09-09T23:23:02.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:02 smithi082 ceph-mon[98694]: slow request 213.386822 seconds old, received at 2023-09-09T23:19:28.303507+0000: peer_request:mds.4:47 currently dispatched 2023-09-09T23:23:03.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:03 smithi098 ceph-mon[103124]: pgmap v283: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 51 KiB/s wr, 24 op/s 2023-09-09T23:23:03.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:03 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:23:03.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:03 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:23:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 215.021359 secs 2023-09-09T23:23:03.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:03 smithi170 ceph-mon[102832]: pgmap v283: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 51 KiB/s wr, 24 op/s 2023-09-09T23:23:03.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:03 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:23:03.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:03 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:23:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 215.021359 secs 2023-09-09T23:23:03.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:03 smithi082 ceph-mon[98694]: pgmap v283: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 51 KiB/s wr, 24 op/s 2023-09-09T23:23:03.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:03 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:23:03.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:03 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:23:03.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 215.021359 secs 2023-09-09T23:23:04.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:23:04.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:23:04.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:04 smithi098 ceph-mon[103124]: 3 slow requests, 3 included below; oldest blocked for > 215.750409 secs 2023-09-09T23:23:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:04 smithi098 ceph-mon[103124]: slow request 215.750408 seconds old, received at 2023-09-09T23:19:27.740800+0000: peer_request:mds.4:47 currently acquired locks 2023-09-09T23:23:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:04 smithi098 ceph-mon[103124]: slow request 215.750174 seconds old, received at 2023-09-09T23:19:27.741034+0000: peer_request(mds.4:48.0 authpin) currently dispatched 2023-09-09T23:23:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:04 smithi098 ceph-mon[103124]: slow request 215.748138 seconds old, received at 2023-09-09T23:19:27.743070+0000: peer_request(client.24913:3869.0 authpin) currently dispatched 2023-09-09T23:23:04.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:23:04.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:23:04.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:04 smithi170 ceph-mon[102832]: 3 slow requests, 3 included below; oldest blocked for > 215.750409 secs 2023-09-09T23:23:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:04 smithi170 ceph-mon[102832]: slow request 215.750408 seconds old, received at 2023-09-09T23:19:27.740800+0000: peer_request:mds.4:47 currently acquired locks 2023-09-09T23:23:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:04 smithi170 ceph-mon[102832]: slow request 215.750174 seconds old, received at 2023-09-09T23:19:27.741034+0000: peer_request(mds.4:48.0 authpin) currently dispatched 2023-09-09T23:23:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:04 smithi170 ceph-mon[102832]: slow request 215.748138 seconds old, received at 2023-09-09T23:19:27.743070+0000: peer_request(client.24913:3869.0 authpin) currently dispatched 2023-09-09T23:23:04.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:23:04.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:23:04.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:04 smithi082 ceph-mon[98694]: 3 slow requests, 3 included below; oldest blocked for > 215.750409 secs 2023-09-09T23:23:04.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:04 smithi082 ceph-mon[98694]: slow request 215.750408 seconds old, received at 2023-09-09T23:19:27.740800+0000: peer_request:mds.4:47 currently acquired locks 2023-09-09T23:23:04.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:04 smithi082 ceph-mon[98694]: slow request 215.750174 seconds old, received at 2023-09-09T23:19:27.741034+0000: peer_request(mds.4:48.0 authpin) currently dispatched 2023-09-09T23:23:04.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:04 smithi082 ceph-mon[98694]: slow request 215.748138 seconds old, received at 2023-09-09T23:19:27.743070+0000: peer_request(client.24913:3869.0 authpin) currently dispatched 2023-09-09T23:23:05.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:05 smithi098 ceph-mon[103124]: pgmap v284: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 51 KiB/s wr, 24 op/s 2023-09-09T23:23:05.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:05 smithi170 ceph-mon[102832]: pgmap v284: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 51 KiB/s wr, 24 op/s 2023-09-09T23:23:05.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:05 smithi082 ceph-mon[98694]: pgmap v284: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 51 KiB/s wr, 24 op/s 2023-09-09T23:23:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:07 smithi098 ceph-mon[103124]: pgmap v285: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 51 KiB/s wr, 29 op/s 2023-09-09T23:23:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:07 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:23:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:07 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:23:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 218.386912 secs 2023-09-09T23:23:07.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:07 smithi170 ceph-mon[102832]: pgmap v285: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 51 KiB/s wr, 29 op/s 2023-09-09T23:23:07.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:07 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:23:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:07 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:23:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 218.386912 secs 2023-09-09T23:23:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:07 smithi082 ceph-mon[98694]: pgmap v285: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 51 KiB/s wr, 29 op/s 2023-09-09T23:23:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:07 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-09T23:23:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:07 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:23:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 218.386912 secs 2023-09-09T23:23:08.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:08 smithi098 ceph-mon[103124]: osdmap e91: 12 total, 12 up, 12 in 2023-09-09T23:23:08.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 220.021422 secs 2023-09-09T23:23:08.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:08 smithi170 ceph-mon[102832]: osdmap e91: 12 total, 12 up, 12 in 2023-09-09T23:23:08.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 220.021422 secs 2023-09-09T23:23:08.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:08 smithi082 ceph-mon[98694]: osdmap e91: 12 total, 12 up, 12 in 2023-09-09T23:23:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 220.021422 secs 2023-09-09T23:23:09.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:09 smithi098 ceph-mon[103124]: pgmap v287: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 61 KiB/s wr, 28 op/s 2023-09-09T23:23:09.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 220.750477 secs 2023-09-09T23:23:09.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:09 smithi170 ceph-mon[102832]: pgmap v287: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 61 KiB/s wr, 28 op/s 2023-09-09T23:23:09.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 220.750477 secs 2023-09-09T23:23:09.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:09 smithi082 ceph-mon[98694]: pgmap v287: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 61 KiB/s wr, 28 op/s 2023-09-09T23:23:09.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 220.750477 secs 2023-09-09T23:23:11.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:11 smithi098 ceph-mon[103124]: pgmap v288: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 30 KiB/s wr, 25 op/s 2023-09-09T23:23:11.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:23:11.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:11 smithi170 ceph-mon[102832]: pgmap v288: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 30 KiB/s wr, 25 op/s 2023-09-09T23:23:11.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:23:11.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:11 smithi082 ceph-mon[98694]: pgmap v288: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 30 KiB/s wr, 25 op/s 2023-09-09T23:23:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:23:12.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 223.387002 secs 2023-09-09T23:23:12.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:23:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:23:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:23:12.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 223.387002 secs 2023-09-09T23:23:12.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:23:12.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:23:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:23:12.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 223.387002 secs 2023-09-09T23:23:12.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:23:12.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:23:12.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:23:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:13 smithi170 ceph-mon[102832]: pgmap v289: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.1 KiB/s wr, 11 op/s 2023-09-09T23:23:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 225.021465 secs 2023-09-09T23:23:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:13 smithi170 ceph-mon[102832]: osdmap e92: 12 total, 12 up, 12 in 2023-09-09T23:23:13.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:13 smithi082 ceph-mon[98694]: pgmap v289: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.1 KiB/s wr, 11 op/s 2023-09-09T23:23:13.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 225.021465 secs 2023-09-09T23:23:13.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:13 smithi082 ceph-mon[98694]: osdmap e92: 12 total, 12 up, 12 in 2023-09-09T23:23:14.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:13 smithi098 ceph-mon[103124]: pgmap v289: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.1 KiB/s wr, 11 op/s 2023-09-09T23:23:14.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 225.021465 secs 2023-09-09T23:23:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:13 smithi098 ceph-mon[103124]: osdmap e92: 12 total, 12 up, 12 in 2023-09-09T23:23:14.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 225.750543 secs 2023-09-09T23:23:14.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 225.750543 secs 2023-09-09T23:23:15.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 225.750543 secs 2023-09-09T23:23:15.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:15 smithi170 ceph-mon[102832]: pgmap v291: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2023-09-09T23:23:15.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:15 smithi082 ceph-mon[98694]: pgmap v291: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2023-09-09T23:23:16.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:15 smithi098 ceph-mon[103124]: pgmap v291: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2023-09-09T23:23:16.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:16 smithi170 ceph-mon[102832]: pgmap v292: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 KiB/s wr, 14 op/s 2023-09-09T23:23:16.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:16 smithi082 ceph-mon[98694]: pgmap v292: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 KiB/s wr, 14 op/s 2023-09-09T23:23:17.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:16 smithi098 ceph-mon[103124]: pgmap v292: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 KiB/s wr, 14 op/s 2023-09-09T23:23:17.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 228.387079 secs 2023-09-09T23:23:17.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 228.387079 secs 2023-09-09T23:23:18.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 228.387079 secs 2023-09-09T23:23:18.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 230.021532 secs 2023-09-09T23:23:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:18 smithi170 ceph-mon[102832]: pgmap v293: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2023-09-09T23:23:18.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 230.021532 secs 2023-09-09T23:23:18.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:18 smithi082 ceph-mon[98694]: pgmap v293: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2023-09-09T23:23:19.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 230.021532 secs 2023-09-09T23:23:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:18 smithi098 ceph-mon[103124]: pgmap v293: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2023-09-09T23:23:19.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 230.750608 secs 2023-09-09T23:23:19.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 230.750608 secs 2023-09-09T23:23:20.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 230.750608 secs 2023-09-09T23:23:20.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:20 smithi170 ceph-mon[102832]: pgmap v294: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 14 op/s 2023-09-09T23:23:20.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:20 smithi082 ceph-mon[98694]: pgmap v294: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 14 op/s 2023-09-09T23:23:21.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:20 smithi098 ceph-mon[103124]: pgmap v294: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 14 op/s 2023-09-09T23:23:22.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:21 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 233.387139 secs 2023-09-09T23:23:22.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:21 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 233.387139 secs 2023-09-09T23:23:22.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:21 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 233.387139 secs 2023-09-09T23:23:23.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:22 smithi170 ceph-mon[102832]: pgmap v295: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 14 op/s 2023-09-09T23:23:23.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:22 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 235.021599 secs 2023-09-09T23:23:23.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:22 smithi082 ceph-mon[98694]: pgmap v295: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 14 op/s 2023-09-09T23:23:23.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:22 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 235.021599 secs 2023-09-09T23:23:23.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:22 smithi098 ceph-mon[103124]: pgmap v295: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 14 op/s 2023-09-09T23:23:23.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:22 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 235.021599 secs 2023-09-09T23:23:24.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 235.750675 secs 2023-09-09T23:23:24.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 235.750675 secs 2023-09-09T23:23:24.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 235.750675 secs 2023-09-09T23:23:25.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:25 smithi082 ceph-mon[98694]: pgmap v296: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 13 op/s 2023-09-09T23:23:25.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:25 smithi098 ceph-mon[103124]: pgmap v296: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 13 op/s 2023-09-09T23:23:25.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:25 smithi170 ceph-mon[102832]: pgmap v296: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 13 op/s 2023-09-09T23:23:26.782 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:23:26.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.922+0000 7f1b1a01d700 1 -- 172.21.15.82:0/1890040922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b140ffe10 msgr2=0x7f1b1410c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:23:26.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.922+0000 7f1b1a01d700 1 --2- 172.21.15.82:0/1890040922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b140ffe10 0x7f1b1410c320 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00b0a0 tx=0x7f1b0c00b3b0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.922+0000 7f1b1a01d700 1 -- 172.21.15.82:0/1890040922 shutdown_connections 2023-09-09T23:23:26.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.922+0000 7f1b1a01d700 1 --2- 172.21.15.82:0/1890040922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b141054f0 0x7f1b141058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.922+0000 7f1b1a01d700 1 --2- 172.21.15.82:0/1890040922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b140ffe10 0x7f1b1410c320 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.922+0000 7f1b1a01d700 1 --2- 172.21.15.82:0/1890040922 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b140ff450 0x7f1b140ff8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.922+0000 7f1b1a01d700 1 -- 172.21.15.82:0/1890040922 >> 172.21.15.82:0/1890040922 conn(0x7f1b140faf20 msgr2=0x7f1b140fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:23:26.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.922+0000 7f1b1a01d700 1 -- 172.21.15.82:0/1890040922 shutdown_connections 2023-09-09T23:23:26.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.922+0000 7f1b1a01d700 1 -- 172.21.15.82:0/1890040922 wait complete. 2023-09-09T23:23:26.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.923+0000 7f1b1a01d700 1 Processor -- start 2023-09-09T23:23:26.928 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.923+0000 7f1b1a01d700 1 -- start start 2023-09-09T23:23:26.928 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.923+0000 7f1b1a01d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b140ff450 0x7f1b140fb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:26.928 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.923+0000 7f1b1a01d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b140ffe10 0x7f1b140fbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:26.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.923+0000 7f1b1a01d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b141054f0 0x7f1b141a47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:26.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.923+0000 7f1b1a01d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b140fc310 con 0x7f1b141054f0 2023-09-09T23:23:26.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.923+0000 7f1b1a01d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b141a4d20 con 0x7f1b140ffe10 2023-09-09T23:23:26.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.923+0000 7f1b1a01d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b141a4e60 con 0x7f1b140ff450 2023-09-09T23:23:26.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.923+0000 7f1b13fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b141054f0 0x7f1b141a47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:26.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.923+0000 7f1b13fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b141054f0 0x7f1b141a47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58212/0 (socket says 172.21.15.82:58212) 2023-09-09T23:23:26.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.923+0000 7f1b13fff700 1 -- 172.21.15.82:0/635603053 learned_addr learned my addr 172.21.15.82:0/635603053 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:23:26.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.923+0000 7f1b12ffd700 1 --2- 172.21.15.82:0/635603053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b140ffe10 0x7f1b140fbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:26.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.923+0000 7f1b137fe700 1 --2- 172.21.15.82:0/635603053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b140ff450 0x7f1b140fb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:26.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.923+0000 7f1b13fff700 1 -- 172.21.15.82:0/635603053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b140ff450 msgr2=0x7f1b140fb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:23:26.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.923+0000 7f1b13fff700 1 --2- 172.21.15.82:0/635603053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b140ff450 0x7f1b140fb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.923+0000 7f1b13fff700 1 -- 172.21.15.82:0/635603053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b140ffe10 msgr2=0x7f1b140fbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:23:26.933 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.923+0000 7f1b13fff700 1 --2- 172.21.15.82:0/635603053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b140ffe10 0x7f1b140fbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.933 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.923+0000 7f1b13fff700 1 -- 172.21.15.82:0/635603053 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0c009d00 con 0x7f1b141054f0 2023-09-09T23:23:26.933 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.923+0000 7f1b12ffd700 1 --2- 172.21.15.82:0/635603053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b140ffe10 0x7f1b140fbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:23:26.934 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.924+0000 7f1b13fff700 1 --2- 172.21.15.82:0/635603053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b141054f0 0x7f1b141a47e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0062a0 tx=0x7f1b0c007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:23:26.934 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.924+0000 7f1b10ff9700 1 -- 172.21.15.82:0/635603053 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c00e040 con 0x7f1b141054f0 2023-09-09T23:23:26.935 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.924+0000 7f1b1a01d700 1 -- 172.21.15.82:0/635603053 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b141a5060 con 0x7f1b141054f0 2023-09-09T23:23:26.935 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.924+0000 7f1b10ff9700 1 -- 172.21.15.82:0/635603053 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b0c0094c0 con 0x7f1b141054f0 2023-09-09T23:23:26.935 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.924+0000 7f1b10ff9700 1 -- 172.21.15.82:0/635603053 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c0052a0 con 0x7f1b141054f0 2023-09-09T23:23:26.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.924+0000 7f1b1a01d700 1 -- 172.21.15.82:0/635603053 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b141a54e0 con 0x7f1b141054f0 2023-09-09T23:23:26.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.926+0000 7f1b10ff9700 1 -- 172.21.15.82:0/635603053 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f1b0c01d470 con 0x7f1b141054f0 2023-09-09T23:23:26.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.927+0000 7f1b10ff9700 1 --2- 172.21.15.82:0/635603053 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f1afc0a07b0 0x7f1afc0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:26.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.927+0000 7f1b10ff9700 1 -- 172.21.15.82:0/635603053 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f1b0c0cd4e0 con 0x7f1b141054f0 2023-09-09T23:23:26.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.927+0000 7f1b137fe700 1 --2- 172.21.15.82:0/635603053 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f1afc0a07b0 0x7f1afc0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:26.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.928+0000 7f1b137fe700 1 --2- 172.21.15.82:0/635603053 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f1afc0a07b0 0x7f1afc0a2c70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1b04009be0 tx=0x7f1b04009260 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:23:26.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.928+0000 7f1b261e3b80 1 Processor -- start 2023-09-09T23:23:26.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.928+0000 7f1b261e3b80 1 -- start start 2023-09-09T23:23:26.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.928+0000 7f1b261e3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557873b0b710 0x557873b0baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:26.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.928+0000 7f1b261e3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557873b0c030 0x557873b15900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:26.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.928+0000 7f1b261e3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557873b15e40 0x557873b18230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:26.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.928+0000 7f1b261e3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557873a64730 con 0x557873b15e40 2023-09-09T23:23:26.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.928+0000 7f1b261e3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x557873a62050 con 0x557873b0c030 2023-09-09T23:23:26.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.928+0000 7f1b261e3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557873a5d940 con 0x557873b0b710 2023-09-09T23:23:26.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.928+0000 7f1b13fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557873b15e40 0x557873b18230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:26.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.928+0000 7f1b13fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557873b15e40 0x557873b18230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58220/0 (socket says 172.21.15.82:58220) 2023-09-09T23:23:26.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.928+0000 7f1b13fff700 1 -- 172.21.15.82:0/955303958 learned_addr learned my addr 172.21.15.82:0/955303958 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:23:26.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.928+0000 7f1b137fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557873b0b710 0x557873b0baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:26.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.928+0000 7f1b12ffd700 1 --2- 172.21.15.82:0/955303958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557873b0c030 0x557873b15900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:26.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.928+0000 7f1b13fff700 1 -- 172.21.15.82:0/955303958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557873b0b710 msgr2=0x557873b0baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:23:26.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.928+0000 7f1b13fff700 1 --2- 172.21.15.82:0/955303958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557873b0b710 0x557873b0baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.928+0000 7f1b13fff700 1 -- 172.21.15.82:0/955303958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557873b0c030 msgr2=0x557873b15900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:23:26.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.929+0000 7f1b13fff700 1 --2- 172.21.15.82:0/955303958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557873b0c030 0x557873b15900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.929+0000 7f1b13fff700 1 -- 172.21.15.82:0/955303958 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557873889de0 con 0x557873b15e40 2023-09-09T23:23:26.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.929+0000 7f1b13fff700 1 --2- 172.21.15.82:0/955303958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557873b15e40 0x557873b18230 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0974d0 tx=0x7f1b0c007aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5a6f9af941f922 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:23:26.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.929+0000 7f1af9ffb700 1 -- 172.21.15.82:0/955303958 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c0ba0b0 con 0x557873b15e40 2023-09-09T23:23:26.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.929+0000 7f1af9ffb700 1 -- 172.21.15.82:0/955303958 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b0c01de90 con 0x557873b15e40 2023-09-09T23:23:26.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.929+0000 7f1af9ffb700 1 -- 172.21.15.82:0/955303958 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c0a2a40 con 0x557873b15e40 2023-09-09T23:23:26.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.929+0000 7f1b261e3b80 1 -- 172.21.15.82:0/955303958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557873b15e40 msgr2=0x557873b18230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:23:26.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.929+0000 7f1b261e3b80 1 --2- 172.21.15.82:0/955303958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557873b15e40 0x557873b18230 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0974d0 tx=0x7f1b0c007aa0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.930+0000 7f1b261e3b80 1 -- 172.21.15.82:0/955303958 shutdown_connections 2023-09-09T23:23:26.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.930+0000 7f1b261e3b80 1 --2- 172.21.15.82:0/955303958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557873b15e40 0x557873b18230 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.930+0000 7f1b261e3b80 1 --2- 172.21.15.82:0/955303958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557873b0b710 0x557873b0baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.930+0000 7f1b261e3b80 1 --2- 172.21.15.82:0/955303958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557873b0c030 0x557873b15900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.930+0000 7f1b261e3b80 1 -- 172.21.15.82:0/955303958 >> 172.21.15.82:0/955303958 conn(0x557873a80090 msgr2=0x557873b06bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:23:26.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.930+0000 7f1b261e3b80 1 -- 172.21.15.82:0/955303958 shutdown_connections 2023-09-09T23:23:26.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.930+0000 7f1b261e3b80 1 -- 172.21.15.82:0/955303958 wait complete. 2023-09-09T23:23:26.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.930+0000 7f1b261e3b80 1 -- start start 2023-09-09T23:23:26.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.930+0000 7f1b261e3b80 1 Processor -- start 2023-09-09T23:23:26.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.931+0000 7f1b261e3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557873b0b710 0x557873b158c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:26.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.931+0000 7f1b261e3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557873b0c030 0x557873b12ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:26.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.931+0000 7f1b261e3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557873b15e40 0x557873b131c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:26.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.931+0000 7f1b261e3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578739fbb70 con 0x557873b15e40 2023-09-09T23:23:26.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.931+0000 7f1b261e3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578739fb870 con 0x557873b0c030 2023-09-09T23:23:26.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.931+0000 7f1b261e3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578738df920 con 0x557873b0b710 2023-09-09T23:23:26.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.931+0000 7f1b261e3b80 10 client.0 _ll_register_callbacks cb 0x557873945a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:23:26.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.931+0000 7f1b13fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557873b15e40 0x557873b131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:26.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.931+0000 7f1b137fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557873b0b710 0x557873b158c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:26.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.931+0000 7f1b12ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557873b0c030 0x557873b12ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:26.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.931+0000 7f1b137fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557873b0b710 0x557873b158c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:56736/0 (socket says 172.21.15.82:56736) 2023-09-09T23:23:26.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.931+0000 7f1b13fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557873b15e40 0x557873b131c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58226/0 (socket says 172.21.15.82:58226) 2023-09-09T23:23:26.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.931+0000 7f1b137fe700 1 -- 172.21.15.82:0/907799904 learned_addr learned my addr 172.21.15.82:0/907799904 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:23:26.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.931+0000 7f1b13fff700 1 -- 172.21.15.82:0/907799904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557873b0b710 msgr2=0x557873b158c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:23:26.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.931+0000 7f1b13fff700 1 --2- 172.21.15.82:0/907799904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557873b0b710 0x557873b158c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.931+0000 7f1b13fff700 1 -- 172.21.15.82:0/907799904 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557873b0c030 msgr2=0x557873b12ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:23:26.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.931+0000 7f1b13fff700 1 --2- 172.21.15.82:0/907799904 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557873b0c030 0x557873b12ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.931+0000 7f1b13fff700 1 -- 172.21.15.82:0/907799904 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b141a54e0 con 0x557873b15e40 2023-09-09T23:23:26.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.931+0000 7f1b12ffd700 1 --2- 172.21.15.82:0/907799904 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557873b0c030 0x557873b12ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:23:26.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.931+0000 7f1b13fff700 1 --2- 172.21.15.82:0/907799904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557873b15e40 0x557873b131c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c096f60 tx=0x7f1b0c0074e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47999f27ceda87f5 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:23:26.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.931+0000 7f1af2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:23:26.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.931+0000 7f1af2ffd700 1 -- 172.21.15.82:0/907799904 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c09b160 con 0x557873b15e40 2023-09-09T23:23:26.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.931+0000 7f1af2ffd700 1 -- 172.21.15.82:0/907799904 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f1b0c0a2a40 con 0x557873b15e40 2023-09-09T23:23:26.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.931+0000 7f1b261e3b80 1 -- 172.21.15.82:0/907799904 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578739b54f0 con 0x557873b15e40 2023-09-09T23:23:26.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.931+0000 7f1af2ffd700 1 -- 172.21.15.82:0/907799904 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b0c09e890 con 0x557873b15e40 2023-09-09T23:23:26.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.931+0000 7f1b261e3b80 10 client.16263 fetch_fsmap learned FSMap version 127 2023-09-09T23:23:26.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.931+0000 7f1b261e3b80 1 -- 172.21.15.82:0/907799904 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x557873872c60 con 0x557873b15e40 2023-09-09T23:23:26.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.932+0000 7f1af2ffd700 1 -- 172.21.15.82:0/907799904 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b0c07d020 con 0x557873b15e40 2023-09-09T23:23:26.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.932+0000 7f1af2ffd700 1 -- 172.21.15.82:0/907799904 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 127) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1b0c0cd040 con 0x557873b15e40 2023-09-09T23:23:26.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.932+0000 7f1b261e3b80 10 client.16263 fetch_fsmap finished waiting for FSMap version 127 2023-09-09T23:23:26.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.932+0000 7f1b261e3b80 10 client.16263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:23:26.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.932+0000 7f1b261e3b80 20 client.16263 populate_metadata read hostname 'smithi082' 2023-09-09T23:23:26.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.932+0000 7f1b261e3b80 1 --2- 172.21.15.82:0/907799904 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x557873b25210 0x557873b27600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:26.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.932+0000 7f1b261e3b80 4 client.16263 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:23:26.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.932+0000 7f1b261e3b80 1 -- 172.21.15.82:0/907799904 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557873a728a0 con 0x557873b25210 2023-09-09T23:23:26.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.932+0000 7f1b137fe700 1 --2- 172.21.15.82:0/907799904 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x557873b25210 0x557873b27600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:26.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.932+0000 7f1af2ffd700 10 client.16263 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:23:26.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.932+0000 7f1b137fe700 1 --2- 172.21.15.82:0/907799904 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x557873b25210 0x557873b27600 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b948403c87c4523b server_cookie=1e795dd26a183742 in_seq=0 out_seq=0 2023-09-09T23:23:26.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.933+0000 7f1af2ffd700 1 -- 172.21.15.82:0/907799904 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557873a728a0 con 0x557873b25210 2023-09-09T23:23:26.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.933+0000 7f1af2ffd700 10 client.16263 handle_command_reply: tid=0 2023-09-09T23:23:26.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.933+0000 7f1b261e3b80 1 client.16263 shutdown 2023-09-09T23:23:26.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.933+0000 7f1b261e3b80 1 -- 172.21.15.82:0/907799904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557873b15e40 msgr2=0x557873b131c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:23:26.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.933+0000 7f1b261e3b80 1 --2- 172.21.15.82:0/907799904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557873b15e40 0x557873b131c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c096f60 tx=0x7f1b0c0074e0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.933+0000 7f1b261e3b80 1 -- 172.21.15.82:0/907799904 shutdown_connections 2023-09-09T23:23:26.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.933+0000 7f1b261e3b80 1 --2- 172.21.15.82:0/907799904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557873b15e40 0x557873b131c0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.933+0000 7f1b261e3b80 1 --2- 172.21.15.82:0/907799904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557873b0b710 0x557873b158c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.933+0000 7f1b261e3b80 1 --2- 172.21.15.82:0/907799904 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x557873b25210 0x557873b27600 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.933+0000 7f1b261e3b80 1 --2- 172.21.15.82:0/907799904 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557873b0c030 0x557873b12ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.933+0000 7f1af2ffd700 0 client.16263 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:23:26.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.933+0000 7f1b261e3b80 1 -- 172.21.15.82:0/907799904 >> 172.21.15.82:0/907799904 conn(0x557873a80090 msgr2=0x557873b06780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:23:26.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.933+0000 7f1b261e3b80 1 -- 172.21.15.82:0/907799904 shutdown_connections 2023-09-09T23:23:26.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.933+0000 7f1b261e3b80 1 -- 172.21.15.82:0/907799904 wait complete. 2023-09-09T23:23:26.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.933+0000 7f1b261e3b80 20 client.16263 trim_cache size 0 max 16384 2023-09-09T23:23:26.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.950+0000 7f1b261e3b80 1 Processor -- start 2023-09-09T23:23:26.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.950+0000 7f1b261e3b80 1 -- start start 2023-09-09T23:23:26.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.950+0000 7f1b261e3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557873b0b710 0x557873b1c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:26.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.950+0000 7f1b261e3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557873b0c030 0x557873b153f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:26.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.950+0000 7f1b261e3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557873b15e40 0x557873b0de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:26.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.950+0000 7f1b261e3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557873b1cdf0 con 0x557873b0c030 2023-09-09T23:23:26.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.950+0000 7f1b261e3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x557873afe400 con 0x557873b0b710 2023-09-09T23:23:26.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.950+0000 7f1b261e3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557873b064d0 con 0x557873b15e40 2023-09-09T23:23:26.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.950+0000 7f1b12ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557873b0c030 0x557873b153f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:26.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.950+0000 7f1b137fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557873b0b710 0x557873b1c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:26.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.950+0000 7f1b137fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557873b0b710 0x557873b1c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:43794/0 (socket says 172.21.15.82:43794) 2023-09-09T23:23:26.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.950+0000 7f1b12ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557873b0c030 0x557873b153f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58232/0 (socket says 172.21.15.82:58232) 2023-09-09T23:23:26.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.950+0000 7f1b137fe700 1 -- 172.21.15.82:0/2469804488 learned_addr learned my addr 172.21.15.82:0/2469804488 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:23:26.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.950+0000 7f1b13fff700 1 --2- 172.21.15.82:0/2469804488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557873b15e40 0x557873b0de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:26.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.951+0000 7f1b137fe700 1 -- 172.21.15.82:0/2469804488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557873b15e40 msgr2=0x557873b0de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:23:26.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.951+0000 7f1b137fe700 1 --2- 172.21.15.82:0/2469804488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557873b15e40 0x557873b0de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.974 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.951+0000 7f1b137fe700 1 -- 172.21.15.82:0/2469804488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557873b0c030 msgr2=0x557873b153f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:23:26.974 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.951+0000 7f1b137fe700 1 --2- 172.21.15.82:0/2469804488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557873b0c030 0x557873b153f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.974 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.951+0000 7f1b137fe700 1 -- 172.21.15.82:0/2469804488 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578738b0250 con 0x557873b0b710 2023-09-09T23:23:26.974 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.951+0000 7f1b12ffd700 1 --2- 172.21.15.82:0/2469804488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557873b0c030 0x557873b153f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:23:26.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.951+0000 7f1b137fe700 1 --2- 172.21.15.82:0/2469804488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557873b0b710 0x557873b1c830 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1b04013010 tx=0x7f1b04014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f3dd9826e3071fc server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:23:26.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.951+0000 7f1afa7fc700 1 -- 172.21.15.82:0/2469804488 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04024040 con 0x557873b0b710 2023-09-09T23:23:26.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.951+0000 7f1afa7fc700 1 -- 172.21.15.82:0/2469804488 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b04029dc0 con 0x557873b0b710 2023-09-09T23:23:26.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.951+0000 7f1afa7fc700 1 -- 172.21.15.82:0/2469804488 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04003e00 con 0x557873b0b710 2023-09-09T23:23:26.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.952+0000 7f1b261e3b80 1 -- 172.21.15.82:0/2469804488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557873b0b710 msgr2=0x557873b1c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:23:26.976 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.952+0000 7f1b261e3b80 1 --2- 172.21.15.82:0/2469804488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557873b0b710 0x557873b1c830 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1b04013010 tx=0x7f1b04014000 comp rx=0 tx=0).stop 2023-09-09T23:23:26.976 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.952+0000 7f1b261e3b80 1 -- 172.21.15.82:0/2469804488 shutdown_connections 2023-09-09T23:23:26.976 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.952+0000 7f1b261e3b80 1 --2- 172.21.15.82:0/2469804488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557873b0b710 0x557873b1c830 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.976 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.952+0000 7f1b261e3b80 1 --2- 172.21.15.82:0/2469804488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557873b15e40 0x557873b0de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.977 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.952+0000 7f1b261e3b80 1 --2- 172.21.15.82:0/2469804488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557873b0c030 0x557873b153f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.977 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.952+0000 7f1b261e3b80 1 -- 172.21.15.82:0/2469804488 >> 172.21.15.82:0/2469804488 conn(0x557873a80090 msgr2=0x557873b05dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:23:26.977 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.952+0000 7f1b261e3b80 1 -- 172.21.15.82:0/2469804488 shutdown_connections 2023-09-09T23:23:26.977 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.952+0000 7f1b261e3b80 1 -- 172.21.15.82:0/2469804488 wait complete. 2023-09-09T23:23:26.977 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.952+0000 7f1b261e3b80 1 -- start start 2023-09-09T23:23:26.977 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.952+0000 7f1b261e3b80 1 Processor -- start 2023-09-09T23:23:26.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.952+0000 7f1b261e3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557873b0b710 0x557873b1b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:26.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.952+0000 7f1b261e3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557873b0c030 0x557873b1b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:26.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.952+0000 7f1b261e3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557873b15e40 0x557873b78fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:26.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.952+0000 7f1b261e3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557873b1bcf0 con 0x557873b0b710 2023-09-09T23:23:26.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.952+0000 7f1b261e3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x557873b1be60 con 0x557873b15e40 2023-09-09T23:23:26.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.952+0000 7f1b261e3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557873b79520 con 0x557873b0c030 2023-09-09T23:23:26.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.953+0000 7f1b261e3b80 10 client.0 _ll_register_callbacks cb 0x7f1b08009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:23:26.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.953+0000 7f1b137fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557873b0b710 0x557873b1b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:26.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.953+0000 7f1b137fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557873b0b710 0x557873b1b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58236/0 (socket says 172.21.15.82:58236) 2023-09-09T23:23:26.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.953+0000 7f1b137fe700 1 -- 172.21.15.82:0/1486627074 learned_addr learned my addr 172.21.15.82:0/1486627074 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:23:26.980 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.953+0000 7f1b12ffd700 1 --2- 172.21.15.82:0/1486627074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557873b0c030 0x557873b1b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:26.980 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.953+0000 7f1b13fff700 1 --2- 172.21.15.82:0/1486627074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557873b15e40 0x557873b78fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:26.980 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.953+0000 7f1b137fe700 1 -- 172.21.15.82:0/1486627074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557873b0c030 msgr2=0x557873b1b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:23:26.980 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.953+0000 7f1b137fe700 1 --2- 172.21.15.82:0/1486627074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557873b0c030 0x557873b1b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.953+0000 7f1b137fe700 1 -- 172.21.15.82:0/1486627074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557873b15e40 msgr2=0x557873b78fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:23:26.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.953+0000 7f1b137fe700 1 --2- 172.21.15.82:0/1486627074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557873b15e40 0x557873b78fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.953+0000 7f1b137fe700 1 -- 172.21.15.82:0/1486627074 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b08009df0 con 0x557873b0b710 2023-09-09T23:23:26.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.953+0000 7f1b137fe700 1 --2- 172.21.15.82:0/1486627074 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557873b0b710 0x557873b1b020 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1b040294a0 tx=0x7f1b040294d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548789dccabb4e30 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:23:26.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.953+0000 7f1af2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:23:26.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.953+0000 7f1af2ffd700 1 -- 172.21.15.82:0/1486627074 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04024070 con 0x557873b0b710 2023-09-09T23:23:26.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.953+0000 7f1b261e3b80 1 -- 172.21.15.82:0/1486627074 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b0c01de90 con 0x557873b0b710 2023-09-09T23:23:26.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.953+0000 7f1af2ffd700 1 -- 172.21.15.82:0/1486627074 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f1b04029b00 con 0x557873b0b710 2023-09-09T23:23:26.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.953+0000 7f1af2ffd700 1 -- 172.21.15.82:0/1486627074 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b04003a80 con 0x557873b0b710 2023-09-09T23:23:26.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.953+0000 7f1b261e3b80 10 client.16269 fetch_fsmap learned FSMap version 127 2023-09-09T23:23:26.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.953+0000 7f1b261e3b80 1 -- 172.21.15.82:0/1486627074 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x7f1b04017070 con 0x557873b0b710 2023-09-09T23:23:26.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.953+0000 7f1af2ffd700 1 -- 172.21.15.82:0/1486627074 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b0c01de90 con 0x557873b0b710 2023-09-09T23:23:26.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.954+0000 7f1af2ffd700 1 -- 172.21.15.82:0/1486627074 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 127) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1b04004750 con 0x557873b0b710 2023-09-09T23:23:26.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.954+0000 7f1b261e3b80 10 client.16269 fetch_fsmap finished waiting for FSMap version 127 2023-09-09T23:23:26.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.954+0000 7f1b261e3b80 10 client.16269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:23:26.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.954+0000 7f1b261e3b80 20 client.16269 populate_metadata read hostname 'smithi082' 2023-09-09T23:23:26.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.954+0000 7f1b261e3b80 1 --2- 172.21.15.82:0/1486627074 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x557873b7acc0 0x557873b7d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:26.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.954+0000 7f1b261e3b80 4 client.16269 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:23:26.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.954+0000 7f1b261e3b80 1 -- 172.21.15.82:0/1486627074 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557873a56920 con 0x557873b7acc0 2023-09-09T23:23:26.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.954+0000 7f1b12ffd700 1 --2- 172.21.15.82:0/1486627074 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x557873b7acc0 0x557873b7d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:26.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.954+0000 7f1af2ffd700 10 client.16269 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:23:26.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.954+0000 7f1b12ffd700 1 --2- 172.21.15.82:0/1486627074 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x557873b7acc0 0x557873b7d0e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf98bb5a0a084962 server_cookie=68c60ce121d88fc1 in_seq=0 out_seq=0 2023-09-09T23:23:26.989 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.954+0000 7f1af2ffd700 1 -- 172.21.15.82:0/1486627074 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x557873a56920 con 0x557873b7acc0 2023-09-09T23:23:26.989 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.954+0000 7f1af2ffd700 10 client.16269 handle_command_reply: tid=0 2023-09-09T23:23:26.989 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.954+0000 7f1b261e3b80 1 client.16269 shutdown 2023-09-09T23:23:26.990 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.955+0000 7f1b261e3b80 1 -- 172.21.15.82:0/1486627074 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557873b0b710 msgr2=0x557873b1b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:23:26.990 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.955+0000 7f1b261e3b80 1 --2- 172.21.15.82:0/1486627074 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557873b0b710 0x557873b1b020 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1b040294a0 tx=0x7f1b040294d0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.991 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.955+0000 7f1b261e3b80 1 -- 172.21.15.82:0/1486627074 shutdown_connections 2023-09-09T23:23:26.991 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.955+0000 7f1b261e3b80 1 --2- 172.21.15.82:0/1486627074 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x557873b7acc0 0x557873b7d0e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.991 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.955+0000 7f1b261e3b80 1 --2- 172.21.15.82:0/1486627074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557873b15e40 0x557873b78fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.955+0000 7f1af2ffd700 0 client.16269 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:23:26.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.955+0000 7f1b261e3b80 1 --2- 172.21.15.82:0/1486627074 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557873b0b710 0x557873b1b020 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.955+0000 7f1b261e3b80 1 --2- 172.21.15.82:0/1486627074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557873b0c030 0x557873b1b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:26.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.955+0000 7f1b261e3b80 1 -- 172.21.15.82:0/1486627074 >> 172.21.15.82:0/1486627074 conn(0x557873a80090 msgr2=0x557873b0de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:23:26.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.955+0000 7f1b261e3b80 1 -- 172.21.15.82:0/1486627074 shutdown_connections 2023-09-09T23:23:26.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.955+0000 7f1b261e3b80 1 -- 172.21.15.82:0/1486627074 wait complete. 2023-09-09T23:23:26.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.955+0000 7f1b261e3b80 20 client.16269 trim_cache size 0 max 16384 2023-09-09T23:23:26.994 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:23:26.995 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:23:26.995 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:23:26.996 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:23:26.996 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:23:26.996 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:23:26.997 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:23:26.997 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:23:26.997 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:23:26.998 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:23:27.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.955+0000 7f1b1a01d700 1 -- 172.21.15.82:0/635603053 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f1afc0a07b0 msgr2=0x7f1afc0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:23:27.001 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.955+0000 7f1b1a01d700 1 --2- 172.21.15.82:0/635603053 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f1afc0a07b0 0x7f1afc0a2c70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1b04009be0 tx=0x7f1b04009260 comp rx=0 tx=0).stop 2023-09-09T23:23:27.001 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.955+0000 7f1b1a01d700 1 -- 172.21.15.82:0/635603053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b141054f0 msgr2=0x7f1b141a47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:23:27.001 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.955+0000 7f1b1a01d700 1 --2- 172.21.15.82:0/635603053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b141054f0 0x7f1b141a47e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0062a0 tx=0x7f1b0c007870 comp rx=0 tx=0).stop 2023-09-09T23:23:27.002 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.955+0000 7f1b1a01d700 1 -- 172.21.15.82:0/635603053 shutdown_connections 2023-09-09T23:23:27.002 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.955+0000 7f1b1a01d700 1 --2- 172.21.15.82:0/635603053 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f1afc0a07b0 0x7f1afc0a2c70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:27.003 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.955+0000 7f1b1a01d700 1 --2- 172.21.15.82:0/635603053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b141054f0 0x7f1b141a47e0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:27.003 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.955+0000 7f1b1a01d700 1 --2- 172.21.15.82:0/635603053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b140ff450 0x7f1b140fb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:27.003 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.955+0000 7f1b1a01d700 1 --2- 172.21.15.82:0/635603053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b140ffe10 0x7f1b140fbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:27.004 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.955+0000 7f1b1a01d700 1 -- 172.21.15.82:0/635603053 >> 172.21.15.82:0/635603053 conn(0x7f1b140faf20 msgr2=0x7f1b14068830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:23:27.004 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.955+0000 7f1b1a01d700 1 -- 172.21.15.82:0/635603053 shutdown_connections 2023-09-09T23:23:27.004 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:26.955+0000 7f1b1a01d700 1 -- 172.21.15.82:0/635603053 wait complete. 2023-09-09T23:23:27.005 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:23:27.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:27 smithi082 ceph-mon[98694]: pgmap v297: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 KiB/s wr, 16 op/s 2023-09-09T23:23:27.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 238.387197 secs 2023-09-09T23:23:27.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:27 smithi098 ceph-mon[103124]: pgmap v297: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 KiB/s wr, 16 op/s 2023-09-09T23:23:27.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 238.387197 secs 2023-09-09T23:23:27.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:27 smithi170 ceph-mon[102832]: pgmap v297: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 KiB/s wr, 16 op/s 2023-09-09T23:23:27.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 238.387197 secs 2023-09-09T23:23:28.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 240.021671 secs 2023-09-09T23:23:28.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 240.021671 secs 2023-09-09T23:23:28.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 240.021671 secs 2023-09-09T23:23:29.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:29 smithi098 ceph-mon[103124]: pgmap v298: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 KiB/s wr, 10 op/s 2023-09-09T23:23:29.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 240.750740 secs 2023-09-09T23:23:29.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:29 smithi170 ceph-mon[102832]: pgmap v298: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 KiB/s wr, 10 op/s 2023-09-09T23:23:29.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 240.750740 secs 2023-09-09T23:23:29.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:29 smithi082 ceph-mon[98694]: pgmap v298: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 KiB/s wr, 10 op/s 2023-09-09T23:23:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 240.750740 secs 2023-09-09T23:23:31.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:31 smithi098 ceph-mon[103124]: pgmap v299: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 13 op/s 2023-09-09T23:23:31.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:31 smithi170 ceph-mon[102832]: pgmap v299: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 13 op/s 2023-09-09T23:23:31.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:31 smithi082 ceph-mon[98694]: pgmap v299: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 13 op/s 2023-09-09T23:23:32.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 243.387260 secs 2023-09-09T23:23:32.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 243.387260 secs 2023-09-09T23:23:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 243.387260 secs 2023-09-09T23:23:33.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:33 smithi098 ceph-mon[103124]: pgmap v300: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2023-09-09T23:23:33.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 245.021739 secs 2023-09-09T23:23:33.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:33 smithi170 ceph-mon[102832]: pgmap v300: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2023-09-09T23:23:33.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 245.021739 secs 2023-09-09T23:23:33.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:33 smithi082 ceph-mon[98694]: pgmap v300: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2023-09-09T23:23:33.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 245.021739 secs 2023-09-09T23:23:34.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 245.750806 secs 2023-09-09T23:23:34.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 245.750806 secs 2023-09-09T23:23:34.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 245.750806 secs 2023-09-09T23:23:35.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:35 smithi098 ceph-mon[103124]: pgmap v301: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2023-09-09T23:23:35.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:35 smithi170 ceph-mon[102832]: pgmap v301: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2023-09-09T23:23:35.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:35 smithi082 ceph-mon[98694]: pgmap v301: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2023-09-09T23:23:37.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:37 smithi098 ceph-mon[103124]: pgmap v302: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2023-09-09T23:23:37.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 248.387375 secs 2023-09-09T23:23:37.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:37 smithi170 ceph-mon[102832]: pgmap v302: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2023-09-09T23:23:37.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 248.387375 secs 2023-09-09T23:23:37.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:37 smithi082 ceph-mon[98694]: pgmap v302: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2023-09-09T23:23:37.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 248.387375 secs 2023-09-09T23:23:38.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 250.021805 secs 2023-09-09T23:23:38.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 250.021805 secs 2023-09-09T23:23:38.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 250.021805 secs 2023-09-09T23:23:39.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:39 smithi098 ceph-mon[103124]: pgmap v303: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2023-09-09T23:23:39.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 250.750871 secs 2023-09-09T23:23:39.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:39 smithi170 ceph-mon[102832]: pgmap v303: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2023-09-09T23:23:39.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 250.750871 secs 2023-09-09T23:23:39.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:39 smithi082 ceph-mon[98694]: pgmap v303: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2023-09-09T23:23:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 250.750871 secs 2023-09-09T23:23:41.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:41 smithi098 ceph-mon[103124]: pgmap v304: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2023-09-09T23:23:41.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:41 smithi170 ceph-mon[102832]: pgmap v304: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2023-09-09T23:23:41.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:41 smithi082 ceph-mon[98694]: pgmap v304: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2023-09-09T23:23:42.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 253.387439 secs 2023-09-09T23:23:42.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 253.387439 secs 2023-09-09T23:23:42.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 253.387439 secs 2023-09-09T23:23:43.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:43 smithi098 ceph-mon[103124]: pgmap v305: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:23:43.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 255.021905 secs 2023-09-09T23:23:43.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:43 smithi170 ceph-mon[102832]: pgmap v305: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:23:43.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 255.021905 secs 2023-09-09T23:23:43.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:43 smithi082 ceph-mon[98694]: pgmap v305: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:23:43.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 255.021905 secs 2023-09-09T23:23:44.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 255.750937 secs 2023-09-09T23:23:44.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 255.750937 secs 2023-09-09T23:23:45.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 255.750937 secs 2023-09-09T23:23:45.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:45 smithi170 ceph-mon[102832]: pgmap v306: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:23:45.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:45 smithi082 ceph-mon[98694]: pgmap v306: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:23:46.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:45 smithi098 ceph-mon[103124]: pgmap v306: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:23:47.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:47 smithi170 ceph-mon[102832]: pgmap v307: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:23:47.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 258.387498 secs 2023-09-09T23:23:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:47 smithi082 ceph-mon[98694]: pgmap v307: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:23:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 258.387498 secs 2023-09-09T23:23:48.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:47 smithi098 ceph-mon[103124]: pgmap v307: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:23:48.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 258.387498 secs 2023-09-09T23:23:48.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 260.021963 secs 2023-09-09T23:23:48.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 260.021963 secs 2023-09-09T23:23:49.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 260.021963 secs 2023-09-09T23:23:49.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:49 smithi170 ceph-mon[102832]: pgmap v308: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2023-09-09T23:23:49.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 260.751004 secs 2023-09-09T23:23:49.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:49 smithi082 ceph-mon[98694]: pgmap v308: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2023-09-09T23:23:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 260.751004 secs 2023-09-09T23:23:50.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:49 smithi098 ceph-mon[103124]: pgmap v308: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2023-09-09T23:23:50.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 260.751004 secs 2023-09-09T23:23:50.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:50 smithi170 ceph-mon[102832]: pgmap v309: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2023-09-09T23:23:50.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:50 smithi082 ceph-mon[98694]: pgmap v309: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2023-09-09T23:23:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:50 smithi098 ceph-mon[103124]: pgmap v309: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2023-09-09T23:23:52.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:51 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 263.387556 secs 2023-09-09T23:23:52.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:51 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 263.387556 secs 2023-09-09T23:23:52.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:51 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 263.387556 secs 2023-09-09T23:23:53.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:52 smithi170 ceph-mon[102832]: pgmap v310: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2023-09-09T23:23:53.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:52 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 265.022023 secs 2023-09-09T23:23:53.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:52 smithi082 ceph-mon[98694]: pgmap v310: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2023-09-09T23:23:53.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:52 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 265.022023 secs 2023-09-09T23:23:53.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:52 smithi098 ceph-mon[103124]: pgmap v310: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2023-09-09T23:23:53.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:52 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 265.022023 secs 2023-09-09T23:23:54.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 265.751069 secs 2023-09-09T23:23:54.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 265.751069 secs 2023-09-09T23:23:54.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 265.751069 secs 2023-09-09T23:23:55.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:55 smithi082 ceph-mon[98694]: pgmap v311: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2023-09-09T23:23:55.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:55 smithi098 ceph-mon[103124]: pgmap v311: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2023-09-09T23:23:55.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:55 smithi170 ceph-mon[102832]: pgmap v311: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2023-09-09T23:23:57.006 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:23:57.146 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.145+0000 7f24eb957700 1 -- 172.21.15.82:0/1038108534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24e4100580 msgr2=0x7f24e4100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:23:57.146 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.145+0000 7f24eb957700 1 --2- 172.21.15.82:0/1038108534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24e4100580 0x7f24e4100a00 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f24d40099c0 tx=0x7f24d4009cd0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.147 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.145+0000 7f24eb957700 1 -- 172.21.15.82:0/1038108534 shutdown_connections 2023-09-09T23:23:57.147 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.145+0000 7f24eb957700 1 --2- 172.21.15.82:0/1038108534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24e4106640 0x7f24e4106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.147 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.145+0000 7f24eb957700 1 --2- 172.21.15.82:0/1038108534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24e4100f40 0x7f24e410d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.147 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.145+0000 7f24eb957700 1 --2- 172.21.15.82:0/1038108534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24e4100580 0x7f24e4100a00 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.147 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.145+0000 7f24eb957700 1 -- 172.21.15.82:0/1038108534 >> 172.21.15.82:0/1038108534 conn(0x7f24e40fc070 msgr2=0x7f24e40fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:23:57.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.145+0000 7f24eb957700 1 -- 172.21.15.82:0/1038108534 shutdown_connections 2023-09-09T23:23:57.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.145+0000 7f24eb957700 1 -- 172.21.15.82:0/1038108534 wait complete. 2023-09-09T23:23:57.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.145+0000 7f24eb957700 1 Processor -- start 2023-09-09T23:23:57.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.145+0000 7f24eb957700 1 -- start start 2023-09-09T23:23:57.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.146+0000 7f24eb957700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24e4100580 0x7f24e4196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:57.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.146+0000 7f24eb957700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24e4100f40 0x7f24e4196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:57.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.146+0000 7f24eb957700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24e4106640 0x7f24e419afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:57.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.146+0000 7f24eb957700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e419b610 con 0x7f24e4100580 2023-09-09T23:23:57.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.146+0000 7f24eb957700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e419b780 con 0x7f24e4106640 2023-09-09T23:23:57.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.146+0000 7f24eb957700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e419b940 con 0x7f24e4100f40 2023-09-09T23:23:57.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.146+0000 7f24e96f3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24e4100580 0x7f24e4196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:57.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.146+0000 7f24e96f3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24e4100580 0x7f24e4196630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58250/0 (socket says 172.21.15.82:58250) 2023-09-09T23:23:57.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.146+0000 7f24e96f3700 1 -- 172.21.15.82:0/1038554055 learned_addr learned my addr 172.21.15.82:0/1038554055 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:23:57.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.146+0000 7f24e9ef4700 1 --2- 172.21.15.82:0/1038554055 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24e4106640 0x7f24e419afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:57.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.146+0000 7f24e8ef2700 1 --2- 172.21.15.82:0/1038554055 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24e4100f40 0x7f24e4196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:57.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.146+0000 7f24e96f3700 1 -- 172.21.15.82:0/1038554055 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24e4100f40 msgr2=0x7f24e4196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:23:57.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.146+0000 7f24e96f3700 1 --2- 172.21.15.82:0/1038554055 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24e4100f40 0x7f24e4196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.146+0000 7f24e96f3700 1 -- 172.21.15.82:0/1038554055 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24e4106640 msgr2=0x7f24e419afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:23:57.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.146+0000 7f24e96f3700 1 --2- 172.21.15.82:0/1038554055 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24e4106640 0x7f24e419afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.146+0000 7f24e96f3700 1 -- 172.21.15.82:0/1038554055 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d40095f0 con 0x7f24e4100580 2023-09-09T23:23:57.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.146+0000 7f24e96f3700 1 --2- 172.21.15.82:0/1038554055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24e4100580 0x7f24e4196630 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f24e000e9a0 tx=0x7f24e000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:23:57.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.146+0000 7f24da7fc700 1 -- 172.21.15.82:0/1038554055 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e0009790 con 0x7f24e4100580 2023-09-09T23:23:57.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.146+0000 7f24da7fc700 1 -- 172.21.15.82:0/1038554055 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24e0010430 con 0x7f24e4100580 2023-09-09T23:23:57.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.146+0000 7f24eb957700 1 -- 172.21.15.82:0/1038554055 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e419bc20 con 0x7f24e4100580 2023-09-09T23:23:57.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.146+0000 7f24da7fc700 1 -- 172.21.15.82:0/1038554055 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e0018680 con 0x7f24e4100580 2023-09-09T23:23:57.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.146+0000 7f24eb957700 1 -- 172.21.15.82:0/1038554055 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e41a3460 con 0x7f24e4100580 2023-09-09T23:23:57.158 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.148+0000 7f24da7fc700 1 -- 172.21.15.82:0/1038554055 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f24e00105a0 con 0x7f24e4100580 2023-09-09T23:23:57.158 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.149+0000 7f24da7fc700 1 --2- 172.21.15.82:0/1038554055 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f24d00a07b0 0x7f24d00a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:57.158 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.149+0000 7f24da7fc700 1 -- 172.21.15.82:0/1038554055 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f24e0014070 con 0x7f24e4100580 2023-09-09T23:23:57.159 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.149+0000 7f24e8ef2700 1 --2- 172.21.15.82:0/1038554055 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f24d00a07b0 0x7f24d00a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:57.159 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.149+0000 7f24e8ef2700 1 --2- 172.21.15.82:0/1038554055 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f24d00a07b0 0x7f24d00a2c70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f24d4009990 tx=0x7f24d4005fb0 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:23:57.159 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.150+0000 7f24f7b1db80 1 Processor -- start 2023-09-09T23:23:57.159 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.150+0000 7f24f7b1db80 1 -- start start 2023-09-09T23:23:57.159 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.150+0000 7f24f7b1db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ac2aa48710 0x55ac2aa48af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:57.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.150+0000 7f24f7b1db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac2aa49030 0x55ac2aa52900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:57.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.150+0000 7f24f7b1db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2aa52e40 0x55ac2aa55230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:57.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.150+0000 7f24f7b1db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2a9a1730 con 0x55ac2aa49030 2023-09-09T23:23:57.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.150+0000 7f24f7b1db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2a99f050 con 0x55ac2aa48710 2023-09-09T23:23:57.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.150+0000 7f24f7b1db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2a99a940 con 0x55ac2aa52e40 2023-09-09T23:23:57.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.150+0000 7f24e8ef2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac2aa49030 0x55ac2aa52900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:57.161 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.150+0000 7f24e8ef2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac2aa49030 0x55ac2aa52900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58260/0 (socket says 172.21.15.82:58260) 2023-09-09T23:23:57.161 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.150+0000 7f24e96f3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ac2aa48710 0x55ac2aa48af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:57.161 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.150+0000 7f24e8ef2700 1 -- 172.21.15.82:0/1190737665 learned_addr learned my addr 172.21.15.82:0/1190737665 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:23:57.161 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.150+0000 7f24e9ef4700 1 --2- 172.21.15.82:0/1190737665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2aa52e40 0x55ac2aa55230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:57.161 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.150+0000 7f24e8ef2700 1 -- 172.21.15.82:0/1190737665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2aa52e40 msgr2=0x55ac2aa55230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:23:57.161 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.150+0000 7f24e8ef2700 1 --2- 172.21.15.82:0/1190737665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2aa52e40 0x55ac2aa55230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.162 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.150+0000 7f24e8ef2700 1 -- 172.21.15.82:0/1190737665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ac2aa48710 msgr2=0x55ac2aa48af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:23:57.162 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.150+0000 7f24e8ef2700 1 --2- 172.21.15.82:0/1190737665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ac2aa48710 0x55ac2aa48af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.162 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.150+0000 7f24e8ef2700 1 -- 172.21.15.82:0/1190737665 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac2a7c6de0 con 0x55ac2aa49030 2023-09-09T23:23:57.162 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.151+0000 7f24e8ef2700 1 --2- 172.21.15.82:0/1190737665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac2aa49030 0x55ac2aa52900 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f24d40099c0 tx=0x7f24d401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dff47bded82afff server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:23:57.162 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.151+0000 7f24cf7fe700 1 -- 172.21.15.82:0/1190737665 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d400b990 con 0x55ac2aa49030 2023-09-09T23:23:57.163 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.151+0000 7f24cf7fe700 1 -- 172.21.15.82:0/1190737665 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24d401e040 con 0x55ac2aa49030 2023-09-09T23:23:57.163 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.151+0000 7f24cf7fe700 1 -- 172.21.15.82:0/1190737665 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d40306e0 con 0x55ac2aa49030 2023-09-09T23:23:57.163 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.151+0000 7f24f7b1db80 1 -- 172.21.15.82:0/1190737665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac2aa49030 msgr2=0x55ac2aa52900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:23:57.163 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.151+0000 7f24f7b1db80 1 --2- 172.21.15.82:0/1190737665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac2aa49030 0x55ac2aa52900 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f24d40099c0 tx=0x7f24d401f000 comp rx=0 tx=0).stop 2023-09-09T23:23:57.163 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.152+0000 7f24f7b1db80 1 -- 172.21.15.82:0/1190737665 shutdown_connections 2023-09-09T23:23:57.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.152+0000 7f24f7b1db80 1 --2- 172.21.15.82:0/1190737665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ac2aa48710 0x55ac2aa48af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.152+0000 7f24f7b1db80 1 --2- 172.21.15.82:0/1190737665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2aa52e40 0x55ac2aa55230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.152+0000 7f24f7b1db80 1 --2- 172.21.15.82:0/1190737665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac2aa49030 0x55ac2aa52900 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.152+0000 7f24f7b1db80 1 -- 172.21.15.82:0/1190737665 >> 172.21.15.82:0/1190737665 conn(0x55ac2a9bd090 msgr2=0x55ac2aa43bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:23:57.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.152+0000 7f24f7b1db80 1 -- 172.21.15.82:0/1190737665 shutdown_connections 2023-09-09T23:23:57.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.152+0000 7f24f7b1db80 1 -- 172.21.15.82:0/1190737665 wait complete. 2023-09-09T23:23:57.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.152+0000 7f24f7b1db80 1 -- start start 2023-09-09T23:23:57.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.152+0000 7f24f7b1db80 1 Processor -- start 2023-09-09T23:23:57.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.152+0000 7f24f7b1db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2aa48710 0x55ac2aa4fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:57.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.152+0000 7f24f7b1db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ac2aa49030 0x55ac2aa500a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:57.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.152+0000 7f24f7b1db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac2aa52e40 0x55ac2aa505e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:57.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.152+0000 7f24f7b1db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2a938b70 con 0x55ac2aa52e40 2023-09-09T23:23:57.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.152+0000 7f24f7b1db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2a938870 con 0x55ac2aa49030 2023-09-09T23:23:57.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.152+0000 7f24f7b1db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2a81c920 con 0x55ac2aa48710 2023-09-09T23:23:57.167 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.153+0000 7f24f7b1db80 10 client.0 _ll_register_callbacks cb 0x55ac2a882a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:23:57.167 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.153+0000 7f24e9ef4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac2aa52e40 0x55ac2aa505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:57.167 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.153+0000 7f24e8ef2700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ac2aa49030 0x55ac2aa500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:57.167 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.153+0000 7f24e96f3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2aa48710 0x55ac2aa4fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:57.167 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.153+0000 7f24e9ef4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac2aa52e40 0x55ac2aa505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58268/0 (socket says 172.21.15.82:58268) 2023-09-09T23:23:57.168 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.153+0000 7f24e8ef2700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ac2aa49030 0x55ac2aa500a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:43830/0 (socket says 172.21.15.82:43830) 2023-09-09T23:23:57.168 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.153+0000 7f24e9ef4700 1 -- 172.21.15.82:0/3126807743 learned_addr learned my addr 172.21.15.82:0/3126807743 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:23:57.168 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.153+0000 7f24e9ef4700 1 -- 172.21.15.82:0/3126807743 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2aa48710 msgr2=0x55ac2aa4fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:23:57.168 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.153+0000 7f24e9ef4700 1 --2- 172.21.15.82:0/3126807743 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2aa48710 0x55ac2aa4fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.168 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.153+0000 7f24e9ef4700 1 -- 172.21.15.82:0/3126807743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ac2aa49030 msgr2=0x55ac2aa500a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:23:57.169 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.153+0000 7f24e9ef4700 1 --2- 172.21.15.82:0/3126807743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ac2aa49030 0x55ac2aa500a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.169 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.153+0000 7f24e9ef4700 1 -- 172.21.15.82:0/3126807743 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e410dd00 con 0x55ac2aa52e40 2023-09-09T23:23:57.169 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.153+0000 7f24e9ef4700 1 --2- 172.21.15.82:0/3126807743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac2aa52e40 0x55ac2aa505e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f24dc011e80 tx=0x7f24dc014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe930ac4d5222d0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:23:57.169 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.153+0000 7f24ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:23:57.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.153+0000 7f24ccff9700 1 -- 172.21.15.82:0/3126807743 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24dc00e040 con 0x55ac2aa52e40 2023-09-09T23:23:57.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.153+0000 7f24ccff9700 1 -- 172.21.15.82:0/3126807743 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f24dc01a070 con 0x55ac2aa52e40 2023-09-09T23:23:57.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.153+0000 7f24f7b1db80 1 -- 172.21.15.82:0/3126807743 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac2a8f24f0 con 0x55ac2aa52e40 2023-09-09T23:23:57.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.153+0000 7f24ccff9700 1 -- 172.21.15.82:0/3126807743 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24dc027420 con 0x55ac2aa52e40 2023-09-09T23:23:57.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.153+0000 7f24f7b1db80 10 client.16281 fetch_fsmap learned FSMap version 127 2023-09-09T23:23:57.171 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.153+0000 7f24f7b1db80 1 -- 172.21.15.82:0/3126807743 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x55ac2a7afc60 con 0x55ac2aa52e40 2023-09-09T23:23:57.171 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.153+0000 7f24ccff9700 1 -- 172.21.15.82:0/3126807743 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ac2a8f24f0 con 0x55ac2aa52e40 2023-09-09T23:23:57.171 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.154+0000 7f24ccff9700 1 -- 172.21.15.82:0/3126807743 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 127) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f24dc016520 con 0x55ac2aa52e40 2023-09-09T23:23:57.171 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.154+0000 7f24f7b1db80 10 client.16281 fetch_fsmap finished waiting for FSMap version 127 2023-09-09T23:23:57.171 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.154+0000 7f24f7b1db80 10 client.16281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:23:57.172 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.154+0000 7f24f7b1db80 20 client.16281 populate_metadata read hostname 'smithi082' 2023-09-09T23:23:57.172 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.154+0000 7f24f7b1db80 1 --2- 172.21.15.82:0/3126807743 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55ac2aa62a70 0x55ac2aa64e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:57.172 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.154+0000 7f24f7b1db80 4 client.16281 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:23:57.172 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.154+0000 7f24f7b1db80 1 -- 172.21.15.82:0/3126807743 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac2a9af8a0 con 0x55ac2aa62a70 2023-09-09T23:23:57.172 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.154+0000 7f24e96f3700 1 --2- 172.21.15.82:0/3126807743 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55ac2aa62a70 0x55ac2aa64e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:57.173 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.154+0000 7f24e96f3700 1 --2- 172.21.15.82:0/3126807743 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55ac2aa62a70 0x55ac2aa64e90 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c077f4d204f7909e server_cookie=7655292148abc9cf in_seq=0 out_seq=0 2023-09-09T23:23:57.173 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.154+0000 7f24ccff9700 10 client.16281 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:23:57.173 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.155+0000 7f24ccff9700 1 -- 172.21.15.82:0/3126807743 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ac2a9af8a0 con 0x55ac2aa62a70 2023-09-09T23:23:57.173 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.155+0000 7f24ccff9700 10 client.16281 handle_command_reply: tid=0 2023-09-09T23:23:57.173 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.155+0000 7f24f7b1db80 1 client.16281 shutdown 2023-09-09T23:23:57.174 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.155+0000 7f24f7b1db80 1 -- 172.21.15.82:0/3126807743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac2aa52e40 msgr2=0x55ac2aa505e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:23:57.174 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.155+0000 7f24f7b1db80 1 --2- 172.21.15.82:0/3126807743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac2aa52e40 0x55ac2aa505e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f24dc011e80 tx=0x7f24dc014600 comp rx=0 tx=0).stop 2023-09-09T23:23:57.174 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.155+0000 7f24f7b1db80 1 -- 172.21.15.82:0/3126807743 shutdown_connections 2023-09-09T23:23:57.174 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.155+0000 7f24f7b1db80 1 --2- 172.21.15.82:0/3126807743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac2aa52e40 0x55ac2aa505e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.174 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.155+0000 7f24f7b1db80 1 --2- 172.21.15.82:0/3126807743 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2aa48710 0x55ac2aa4fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.175 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.155+0000 7f24f7b1db80 1 --2- 172.21.15.82:0/3126807743 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55ac2aa62a70 0x55ac2aa64e90 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.175 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.155+0000 7f24f7b1db80 1 --2- 172.21.15.82:0/3126807743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ac2aa49030 0x55ac2aa500a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.175 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.155+0000 7f24ccff9700 0 client.16281 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:23:57.175 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.155+0000 7f24f7b1db80 1 -- 172.21.15.82:0/3126807743 >> 172.21.15.82:0/3126807743 conn(0x55ac2a9bd090 msgr2=0x55ac2aa437e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:23:57.175 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.155+0000 7f24f7b1db80 1 -- 172.21.15.82:0/3126807743 shutdown_connections 2023-09-09T23:23:57.176 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.155+0000 7f24f7b1db80 1 -- 172.21.15.82:0/3126807743 wait complete. 2023-09-09T23:23:57.176 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.156+0000 7f24f7b1db80 20 client.16281 trim_cache size 0 max 16384 2023-09-09T23:23:57.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.172+0000 7f24f7b1db80 1 Processor -- start 2023-09-09T23:23:57.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.172+0000 7f24f7b1db80 1 -- start start 2023-09-09T23:23:57.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.172+0000 7f24f7b1db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2aa48710 0x55ac2aa59980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:57.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.172+0000 7f24f7b1db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac2aa49030 0x55ac2aa4cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:57.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.172+0000 7f24f7b1db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ac2aa52e40 0x55ac2aa4d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:57.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.172+0000 7f24f7b1db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2aa450f0 con 0x55ac2aa49030 2023-09-09T23:23:57.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.172+0000 7f24f7b1db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2aa62880 con 0x55ac2aa52e40 2023-09-09T23:23:57.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.172+0000 7f24f7b1db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2aa59ef0 con 0x55ac2aa48710 2023-09-09T23:23:57.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.173+0000 7f24e8ef2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac2aa49030 0x55ac2aa4cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:57.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.173+0000 7f24e8ef2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac2aa49030 0x55ac2aa4cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58274/0 (socket says 172.21.15.82:58274) 2023-09-09T23:23:57.184 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.173+0000 7f24e8ef2700 1 -- 172.21.15.82:0/2295699461 learned_addr learned my addr 172.21.15.82:0/2295699461 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:23:57.184 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.173+0000 7f24e9ef4700 1 --2- 172.21.15.82:0/2295699461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ac2aa52e40 0x55ac2aa4d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:57.184 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.173+0000 7f24e96f3700 1 --2- 172.21.15.82:0/2295699461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2aa48710 0x55ac2aa59980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:57.184 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.173+0000 7f24e8ef2700 1 -- 172.21.15.82:0/2295699461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2aa48710 msgr2=0x55ac2aa59980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:23:57.184 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.173+0000 7f24e8ef2700 1 --2- 172.21.15.82:0/2295699461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2aa48710 0x55ac2aa59980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.185 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.173+0000 7f24e8ef2700 1 -- 172.21.15.82:0/2295699461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ac2aa52e40 msgr2=0x55ac2aa4d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:23:57.185 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.173+0000 7f24e8ef2700 1 --2- 172.21.15.82:0/2295699461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ac2aa52e40 0x55ac2aa4d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.185 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.173+0000 7f24e8ef2700 1 -- 172.21.15.82:0/2295699461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac2a7ed250 con 0x55ac2aa49030 2023-09-09T23:23:57.185 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.173+0000 7f24e8ef2700 1 --2- 172.21.15.82:0/2295699461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac2aa49030 0x55ac2aa4cdd0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x55ac2aa3e280 tx=0x7f24d4004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf670abfae4c7f16 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:23:57.185 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.173+0000 7f24cffff700 1 -- 172.21.15.82:0/2295699461 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d4036070 con 0x55ac2aa49030 2023-09-09T23:23:57.186 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.173+0000 7f24cffff700 1 -- 172.21.15.82:0/2295699461 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24d4005040 con 0x55ac2aa49030 2023-09-09T23:23:57.186 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.173+0000 7f24cffff700 1 -- 172.21.15.82:0/2295699461 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d400b9f0 con 0x55ac2aa49030 2023-09-09T23:23:57.186 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.174+0000 7f24f7b1db80 1 -- 172.21.15.82:0/2295699461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac2aa49030 msgr2=0x55ac2aa4cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:23:57.186 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.174+0000 7f24f7b1db80 1 --2- 172.21.15.82:0/2295699461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac2aa49030 0x55ac2aa4cdd0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x55ac2aa3e280 tx=0x7f24d4004540 comp rx=0 tx=0).stop 2023-09-09T23:23:57.186 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.174+0000 7f24f7b1db80 1 -- 172.21.15.82:0/2295699461 shutdown_connections 2023-09-09T23:23:57.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.174+0000 7f24f7b1db80 1 --2- 172.21.15.82:0/2295699461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ac2aa52e40 0x55ac2aa4d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.174+0000 7f24f7b1db80 1 --2- 172.21.15.82:0/2295699461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2aa48710 0x55ac2aa59980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.174+0000 7f24f7b1db80 1 --2- 172.21.15.82:0/2295699461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac2aa49030 0x55ac2aa4cdd0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.174+0000 7f24f7b1db80 1 -- 172.21.15.82:0/2295699461 >> 172.21.15.82:0/2295699461 conn(0x55ac2a9bd090 msgr2=0x55ac2aa4ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:23:57.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.174+0000 7f24f7b1db80 1 -- 172.21.15.82:0/2295699461 shutdown_connections 2023-09-09T23:23:57.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.174+0000 7f24f7b1db80 1 -- 172.21.15.82:0/2295699461 wait complete. 2023-09-09T23:23:57.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.174+0000 7f24f7b1db80 1 -- start start 2023-09-09T23:23:57.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.174+0000 7f24f7b1db80 1 Processor -- start 2023-09-09T23:23:57.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.174+0000 7f24f7b1db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac2aa48710 0x55ac2aa59660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:57.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.174+0000 7f24f7b1db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ac2aa49030 0x55ac2aa578e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:57.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.174+0000 7f24f7b1db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2aa52e40 0x55ac2aa57e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:57.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.174+0000 7f24f7b1db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2aa58360 con 0x55ac2aa48710 2023-09-09T23:23:57.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.174+0000 7f24f7b1db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2aa584a0 con 0x55ac2aa49030 2023-09-09T23:23:57.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.174+0000 7f24f7b1db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2aa58660 con 0x55ac2aa52e40 2023-09-09T23:23:57.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.175+0000 7f24f7b1db80 10 client.0 _ll_register_callbacks cb 0x7f24dc0117b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:23:57.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.175+0000 7f24e96f3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac2aa48710 0x55ac2aa59660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:57.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.175+0000 7f24e96f3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac2aa48710 0x55ac2aa59660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58278/0 (socket says 172.21.15.82:58278) 2023-09-09T23:23:57.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.175+0000 7f24e96f3700 1 -- 172.21.15.82:0/2868642799 learned_addr learned my addr 172.21.15.82:0/2868642799 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:23:57.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.175+0000 7f24e8ef2700 1 --2- 172.21.15.82:0/2868642799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ac2aa49030 0x55ac2aa578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:57.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.175+0000 7f24e9ef4700 1 --2- 172.21.15.82:0/2868642799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2aa52e40 0x55ac2aa57e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:57.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.175+0000 7f24e96f3700 1 -- 172.21.15.82:0/2868642799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2aa52e40 msgr2=0x55ac2aa57e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:23:57.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.175+0000 7f24e96f3700 1 --2- 172.21.15.82:0/2868642799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2aa52e40 0x55ac2aa57e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.175+0000 7f24e96f3700 1 -- 172.21.15.82:0/2868642799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ac2aa49030 msgr2=0x55ac2aa578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:23:57.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.175+0000 7f24e96f3700 1 --2- 172.21.15.82:0/2868642799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ac2aa49030 0x55ac2aa578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.175+0000 7f24e96f3700 1 -- 172.21.15.82:0/2868642799 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac2a7c6de0 con 0x55ac2aa48710 2023-09-09T23:23:57.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.175+0000 7f24e8ef2700 1 --2- 172.21.15.82:0/2868642799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ac2aa49030 0x55ac2aa578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:23:57.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.175+0000 7f24e96f3700 1 --2- 172.21.15.82:0/2868642799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac2aa48710 0x55ac2aa59660 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f24e0099590 tx=0x7f24e0084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2bd08b5c650ab1 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:23:57.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.175+0000 7f24ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:23:57.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.175+0000 7f24ccff9700 1 -- 172.21.15.82:0/2868642799 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e009d430 con 0x55ac2aa48710 2023-09-09T23:23:57.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.175+0000 7f24ccff9700 1 -- 172.21.15.82:0/2868642799 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f24e00a1670 con 0x55ac2aa48710 2023-09-09T23:23:57.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.175+0000 7f24f7b1db80 1 -- 172.21.15.82:0/2868642799 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24d401e040 con 0x55ac2aa48710 2023-09-09T23:23:57.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.175+0000 7f24ccff9700 1 -- 172.21.15.82:0/2868642799 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24e009d120 con 0x55ac2aa48710 2023-09-09T23:23:57.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.175+0000 7f24f7b1db80 10 client.16287 fetch_fsmap learned FSMap version 127 2023-09-09T23:23:57.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.175+0000 7f24f7b1db80 1 -- 172.21.15.82:0/2868642799 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x55ac2a7c52b0 con 0x55ac2aa48710 2023-09-09T23:23:57.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.175+0000 7f24ccff9700 1 -- 172.21.15.82:0/2868642799 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24d401e040 con 0x55ac2aa48710 2023-09-09T23:23:57.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.176+0000 7f24ccff9700 1 -- 172.21.15.82:0/2868642799 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 127) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f24e0084140 con 0x55ac2aa48710 2023-09-09T23:23:57.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.176+0000 7f24f7b1db80 10 client.16287 fetch_fsmap finished waiting for FSMap version 127 2023-09-09T23:23:57.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.176+0000 7f24f7b1db80 10 client.16287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:23:57.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.176+0000 7f24f7b1db80 20 client.16287 populate_metadata read hostname 'smithi082' 2023-09-09T23:23:57.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.176+0000 7f24f7b1db80 1 --2- 172.21.15.82:0/2868642799 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55ac2aab7d80 0x55ac2aaba1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:23:57.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.176+0000 7f24f7b1db80 4 client.16287 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:23:57.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.176+0000 7f24f7b1db80 1 -- 172.21.15.82:0/2868642799 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ac2a993920 con 0x55ac2aab7d80 2023-09-09T23:23:57.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.176+0000 7f24e8ef2700 1 --2- 172.21.15.82:0/2868642799 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55ac2aab7d80 0x55ac2aaba1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:23:57.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.176+0000 7f24ccff9700 10 client.16287 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:23:57.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.176+0000 7f24e8ef2700 1 --2- 172.21.15.82:0/2868642799 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55ac2aab7d80 0x55ac2aaba1a0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1930713d91f38fc4 server_cookie=87f9bbe71959dccb in_seq=0 out_seq=0 2023-09-09T23:23:57.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.176+0000 7f24ccff9700 1 -- 172.21.15.82:0/2868642799 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55ac2a993920 con 0x55ac2aab7d80 2023-09-09T23:23:57.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.176+0000 7f24ccff9700 10 client.16287 handle_command_reply: tid=0 2023-09-09T23:23:57.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.176+0000 7f24f7b1db80 1 client.16287 shutdown 2023-09-09T23:23:57.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.176+0000 7f24f7b1db80 1 -- 172.21.15.82:0/2868642799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac2aa48710 msgr2=0x55ac2aa59660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:23:57.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.176+0000 7f24f7b1db80 1 --2- 172.21.15.82:0/2868642799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac2aa48710 0x55ac2aa59660 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f24e0099590 tx=0x7f24e0084040 comp rx=0 tx=0).stop 2023-09-09T23:23:57.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.176+0000 7f24f7b1db80 1 -- 172.21.15.82:0/2868642799 shutdown_connections 2023-09-09T23:23:57.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.176+0000 7f24f7b1db80 1 --2- 172.21.15.82:0/2868642799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2aa52e40 0x55ac2aa57e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.176+0000 7f24f7b1db80 1 --2- 172.21.15.82:0/2868642799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac2aa48710 0x55ac2aa59660 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.176+0000 7f24f7b1db80 1 --2- 172.21.15.82:0/2868642799 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55ac2aab7d80 0x55ac2aaba1a0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.176+0000 7f24f7b1db80 1 --2- 172.21.15.82:0/2868642799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ac2aa49030 0x55ac2aa578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.176+0000 7f24f7b1db80 1 -- 172.21.15.82:0/2868642799 >> 172.21.15.82:0/2868642799 conn(0x55ac2a9bd090 msgr2=0x55ac2aa4fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:23:57.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.176+0000 7f24ccff9700 0 client.16287 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:23:57.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.177+0000 7f24f7b1db80 1 -- 172.21.15.82:0/2868642799 shutdown_connections 2023-09-09T23:23:57.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.177+0000 7f24f7b1db80 1 -- 172.21.15.82:0/2868642799 wait complete. 2023-09-09T23:23:57.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.177+0000 7f24f7b1db80 20 client.16287 trim_cache size 0 max 16384 2023-09-09T23:23:57.199 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:23:57.199 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:23:57.199 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:23:57.199 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:23:57.199 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:23:57.200 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:23:57.200 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:23:57.200 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:23:57.200 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:23:57.200 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:23:57.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.177+0000 7f24eb957700 1 -- 172.21.15.82:0/1038554055 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f24d00a07b0 msgr2=0x7f24d00a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:23:57.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.177+0000 7f24eb957700 1 --2- 172.21.15.82:0/1038554055 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f24d00a07b0 0x7f24d00a2c70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f24d4009990 tx=0x7f24d4005fb0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.177+0000 7f24eb957700 1 -- 172.21.15.82:0/1038554055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24e4100580 msgr2=0x7f24e4196630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:23:57.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.177+0000 7f24eb957700 1 --2- 172.21.15.82:0/1038554055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24e4100580 0x7f24e4196630 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f24e000e9a0 tx=0x7f24e000ecb0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.177+0000 7f24eb957700 1 -- 172.21.15.82:0/1038554055 shutdown_connections 2023-09-09T23:23:57.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.177+0000 7f24eb957700 1 --2- 172.21.15.82:0/1038554055 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f24d00a07b0 0x7f24d00a2c70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.177+0000 7f24eb957700 1 --2- 172.21.15.82:0/1038554055 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24e4106640 0x7f24e419afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.177+0000 7f24eb957700 1 --2- 172.21.15.82:0/1038554055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24e4100580 0x7f24e4196630 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.177+0000 7f24eb957700 1 --2- 172.21.15.82:0/1038554055 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24e4100f40 0x7f24e4196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:23:57.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.177+0000 7f24eb957700 1 -- 172.21.15.82:0/1038554055 >> 172.21.15.82:0/1038554055 conn(0x7f24e40fc070 msgr2=0x7f24e40fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:23:57.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.177+0000 7f24eb957700 1 -- 172.21.15.82:0/1038554055 shutdown_connections 2023-09-09T23:23:57.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:23:57.177+0000 7f24eb957700 1 -- 172.21.15.82:0/1038554055 wait complete. 2023-09-09T23:23:57.204 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:23:57.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:57 smithi082 ceph-mon[98694]: pgmap v312: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-09T23:23:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 268.387616 secs 2023-09-09T23:23:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:57 smithi098 ceph-mon[103124]: pgmap v312: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-09T23:23:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 268.387616 secs 2023-09-09T23:23:57.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:57 smithi170 ceph-mon[102832]: pgmap v312: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-09T23:23:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 268.387616 secs 2023-09-09T23:23:58.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 270.022094 secs 2023-09-09T23:23:58.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 270.022094 secs 2023-09-09T23:23:58.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 270.022094 secs 2023-09-09T23:23:59.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:59 smithi098 ceph-mon[103124]: pgmap v313: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:23:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:23:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 270.751135 secs 2023-09-09T23:23:59.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:59 smithi170 ceph-mon[102832]: pgmap v313: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:23:59.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:23:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 270.751135 secs 2023-09-09T23:23:59.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:59 smithi082 ceph-mon[98694]: pgmap v313: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:23:59.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:23:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 270.751135 secs 2023-09-09T23:24:01.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:01 smithi098 ceph-mon[103124]: pgmap v314: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 27 KiB/s wr, 14 op/s 2023-09-09T23:24:01.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:01 smithi170 ceph-mon[102832]: pgmap v314: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 27 KiB/s wr, 14 op/s 2023-09-09T23:24:01.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:01 smithi082 ceph-mon[98694]: pgmap v314: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 27 KiB/s wr, 14 op/s 2023-09-09T23:24:02.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 273.387710 secs 2023-09-09T23:24:02.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 273.387710 secs 2023-09-09T23:24:02.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 273.387710 secs 2023-09-09T23:24:03.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:03 smithi098 ceph-mon[103124]: pgmap v315: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 48 KiB/s wr, 24 op/s 2023-09-09T23:24:03.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 275.022150 secs 2023-09-09T23:24:03.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:03 smithi170 ceph-mon[102832]: pgmap v315: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 48 KiB/s wr, 24 op/s 2023-09-09T23:24:03.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 275.022150 secs 2023-09-09T23:24:03.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:03 smithi082 ceph-mon[98694]: pgmap v315: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 48 KiB/s wr, 24 op/s 2023-09-09T23:24:03.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 275.022150 secs 2023-09-09T23:24:04.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 275.751202 secs 2023-09-09T23:24:04.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 275.751202 secs 2023-09-09T23:24:04.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 275.751202 secs 2023-09-09T23:24:05.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:05 smithi098 ceph-mon[103124]: pgmap v316: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 48 KiB/s wr, 23 op/s 2023-09-09T23:24:05.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:05 smithi170 ceph-mon[102832]: pgmap v316: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 48 KiB/s wr, 23 op/s 2023-09-09T23:24:05.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:05 smithi082 ceph-mon[98694]: pgmap v316: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 48 KiB/s wr, 23 op/s 2023-09-09T23:24:07.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:07 smithi098 ceph-mon[103124]: pgmap v317: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 48 KiB/s wr, 27 op/s 2023-09-09T23:24:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 278.387850 secs 2023-09-09T23:24:07.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:07 smithi170 ceph-mon[102832]: pgmap v317: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 48 KiB/s wr, 27 op/s 2023-09-09T23:24:07.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 278.387850 secs 2023-09-09T23:24:07.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:07 smithi082 ceph-mon[98694]: pgmap v317: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 48 KiB/s wr, 27 op/s 2023-09-09T23:24:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 278.387850 secs 2023-09-09T23:24:08.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:08 smithi098 ceph-mon[103124]: osdmap e93: 12 total, 12 up, 12 in 2023-09-09T23:24:08.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 280.022208 secs 2023-09-09T23:24:08.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:08 smithi170 ceph-mon[102832]: osdmap e93: 12 total, 12 up, 12 in 2023-09-09T23:24:08.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 280.022208 secs 2023-09-09T23:24:08.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:08 smithi082 ceph-mon[98694]: osdmap e93: 12 total, 12 up, 12 in 2023-09-09T23:24:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 280.022208 secs 2023-09-09T23:24:09.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:09 smithi098 ceph-mon[103124]: pgmap v319: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 28 op/s 2023-09-09T23:24:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 280.751266 secs 2023-09-09T23:24:09.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:09 smithi170 ceph-mon[102832]: pgmap v319: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 28 op/s 2023-09-09T23:24:09.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 280.751266 secs 2023-09-09T23:24:09.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:09 smithi082 ceph-mon[98694]: pgmap v319: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 28 op/s 2023-09-09T23:24:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 280.751266 secs 2023-09-09T23:24:11.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:11 smithi098 ceph-mon[103124]: pgmap v320: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 25 op/s 2023-09-09T23:24:11.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:11 smithi170 ceph-mon[102832]: pgmap v320: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 25 op/s 2023-09-09T23:24:11.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:11 smithi082 ceph-mon[98694]: pgmap v320: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 25 op/s 2023-09-09T23:24:12.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 283.387910 secs 2023-09-09T23:24:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:24:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 283.387910 secs 2023-09-09T23:24:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:24:12.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 283.387910 secs 2023-09-09T23:24:12.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:24:13.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:13 smithi170 ceph-mon[102832]: pgmap v321: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-09T23:24:13.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 285.022268 secs 2023-09-09T23:24:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:13 smithi170 ceph-mon[102832]: osdmap e94: 12 total, 12 up, 12 in 2023-09-09T23:24:13.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:13 smithi082 ceph-mon[98694]: pgmap v321: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-09T23:24:13.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 285.022268 secs 2023-09-09T23:24:13.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:13 smithi082 ceph-mon[98694]: osdmap e94: 12 total, 12 up, 12 in 2023-09-09T23:24:14.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:13 smithi098 ceph-mon[103124]: pgmap v321: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-09T23:24:14.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 285.022268 secs 2023-09-09T23:24:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:13 smithi098 ceph-mon[103124]: osdmap e94: 12 total, 12 up, 12 in 2023-09-09T23:24:14.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 285.751332 secs 2023-09-09T23:24:14.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 285.751332 secs 2023-09-09T23:24:15.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 285.751332 secs 2023-09-09T23:24:15.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:15 smithi170 ceph-mon[102832]: pgmap v323: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:24:15.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:24:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:24:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:24:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:24:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:15.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:15.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:15 smithi082 ceph-mon[98694]: pgmap v323: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:24:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:24:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:24:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:24:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:24:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:16.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:15 smithi098 ceph-mon[103124]: pgmap v323: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:24:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:24:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:24:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:24:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:24:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:16.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:16.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:16.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:16 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-09T23:24:16.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:16 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-09T23:24:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:16 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-09T23:24:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:16 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-09T23:24:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:24:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:24:16.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:16 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-09T23:24:16.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:16 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-09T23:24:16.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:16 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-09T23:24:16.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:16 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-09T23:24:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:24:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:24:17.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:16 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-09T23:24:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:16 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-09T23:24:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:16 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-09T23:24:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:16 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-09T23:24:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:24:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:24:17.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:17 smithi170 ceph-mon[102832]: pgmap v324: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.9 KiB/s wr, 13 op/s 2023-09-09T23:24:17.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:24:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:24:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:24:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 288.387998 secs 2023-09-09T23:24:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:17 smithi082 ceph-mon[98694]: pgmap v324: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.9 KiB/s wr, 13 op/s 2023-09-09T23:24:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:24:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:24:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:24:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 288.387998 secs 2023-09-09T23:24:18.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:17 smithi098 ceph-mon[103124]: pgmap v324: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.9 KiB/s wr, 13 op/s 2023-09-09T23:24:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:24:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:24:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:24:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:24:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 288.387998 secs 2023-09-09T23:24:18.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:18 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-09T23:24:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:18 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-09T23:24:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 290.022343 secs 2023-09-09T23:24:18.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:18 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-09T23:24:18.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:18 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-09T23:24:18.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 290.022343 secs 2023-09-09T23:24:19.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:18 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-09T23:24:19.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:18 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-09T23:24:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 290.022343 secs 2023-09-09T23:24:19.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:19 smithi170 ceph-mon[102832]: pgmap v325: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.8 KiB/s wr, 11 op/s 2023-09-09T23:24:19.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 290.751395 secs 2023-09-09T23:24:19.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:19 smithi082 ceph-mon[98694]: pgmap v325: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.8 KiB/s wr, 11 op/s 2023-09-09T23:24:19.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 290.751395 secs 2023-09-09T23:24:20.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:19 smithi098 ceph-mon[103124]: pgmap v325: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.8 KiB/s wr, 11 op/s 2023-09-09T23:24:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 290.751395 secs 2023-09-09T23:24:21.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:21 smithi170 ceph-mon[102832]: pgmap v326: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2023-09-09T23:24:21.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:21 smithi082 ceph-mon[98694]: pgmap v326: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2023-09-09T23:24:22.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:21 smithi098 ceph-mon[103124]: pgmap v326: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2023-09-09T23:24:22.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 293.388156 secs 2023-09-09T23:24:22.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 293.388156 secs 2023-09-09T23:24:23.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 293.388156 secs 2023-09-09T23:24:23.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:23 smithi170 ceph-mon[102832]: pgmap v327: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2023-09-09T23:24:23.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 295.022405 secs 2023-09-09T23:24:23.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:23 smithi082 ceph-mon[98694]: pgmap v327: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2023-09-09T23:24:23.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 295.022405 secs 2023-09-09T23:24:24.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:23 smithi098 ceph-mon[103124]: pgmap v327: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2023-09-09T23:24:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 295.022405 secs 2023-09-09T23:24:24.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 295.751457 secs 2023-09-09T23:24:24.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:24 smithi170 ceph-mon[102832]: pgmap v328: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 12 op/s 2023-09-09T23:24:24.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 295.751457 secs 2023-09-09T23:24:24.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:24 smithi082 ceph-mon[98694]: pgmap v328: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 12 op/s 2023-09-09T23:24:25.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 295.751457 secs 2023-09-09T23:24:25.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:24 smithi098 ceph-mon[103124]: pgmap v328: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 12 op/s 2023-09-09T23:24:27.206 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:24:27.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.348+0000 7f66aea6d700 1 -- 172.21.15.82:0/3090686018 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a8105550 msgr2=0x7f66a8105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:24:27.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.348+0000 7f66aea6d700 1 --2- 172.21.15.82:0/3090686018 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a8105550 0x7f66a8105930 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f669c0099c0 tx=0x7f669c009cd0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.348+0000 7f66aea6d700 1 -- 172.21.15.82:0/3090686018 shutdown_connections 2023-09-09T23:24:27.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.348+0000 7f66aea6d700 1 --2- 172.21.15.82:0/3090686018 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66a80ffe50 0x7f66a810c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.348+0000 7f66aea6d700 1 --2- 172.21.15.82:0/3090686018 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a8105550 0x7f66a8105930 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.348+0000 7f66aea6d700 1 --2- 172.21.15.82:0/3090686018 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66a80ff490 0x7f66a80ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.348+0000 7f66aea6d700 1 -- 172.21.15.82:0/3090686018 >> 172.21.15.82:0/3090686018 conn(0x7f66a80faf20 msgr2=0x7f66a80fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:24:27.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.348+0000 7f66aea6d700 1 -- 172.21.15.82:0/3090686018 shutdown_connections 2023-09-09T23:24:27.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.349+0000 7f66aea6d700 1 -- 172.21.15.82:0/3090686018 wait complete. 2023-09-09T23:24:27.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.349+0000 7f66aea6d700 1 Processor -- start 2023-09-09T23:24:27.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.349+0000 7f66aea6d700 1 -- start start 2023-09-09T23:24:27.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.349+0000 7f66aea6d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66a80ff490 0x7f66a819c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:27.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.349+0000 7f66aea6d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a80ffe50 0x7f66a819c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:27.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.349+0000 7f66aea6d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66a8105550 0x7f66a8196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:27.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.349+0000 7f66aea6d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a8196b90 con 0x7f66a80ffe50 2023-09-09T23:24:27.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.349+0000 7f66aea6d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a8196d00 con 0x7f66a8105550 2023-09-09T23:24:27.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.349+0000 7f66aea6d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a8196ec0 con 0x7f66a80ff490 2023-09-09T23:24:27.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.349+0000 7f66a7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a80ffe50 0x7f66a819c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:27.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.349+0000 7f66a7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a80ffe50 0x7f66a819c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58296/0 (socket says 172.21.15.82:58296) 2023-09-09T23:24:27.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.349+0000 7f66a7fff700 1 -- 172.21.15.82:0/1886609575 learned_addr learned my addr 172.21.15.82:0/1886609575 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:24:27.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.349+0000 7f66ad00a700 1 --2- 172.21.15.82:0/1886609575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66a8105550 0x7f66a8196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:27.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.349+0000 7f66ac809700 1 --2- 172.21.15.82:0/1886609575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66a80ff490 0x7f66a819c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:27.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.350+0000 7f66a7fff700 1 -- 172.21.15.82:0/1886609575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66a80ff490 msgr2=0x7f66a819c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:24:27.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.350+0000 7f66a7fff700 1 --2- 172.21.15.82:0/1886609575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66a80ff490 0x7f66a819c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.350+0000 7f66a7fff700 1 -- 172.21.15.82:0/1886609575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66a8105550 msgr2=0x7f66a8196530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:24:27.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.350+0000 7f66a7fff700 1 --2- 172.21.15.82:0/1886609575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66a8105550 0x7f66a8196530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.350+0000 7f66a7fff700 1 -- 172.21.15.82:0/1886609575 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669c0095f0 con 0x7f66a80ffe50 2023-09-09T23:24:27.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.350+0000 7f66a7fff700 1 --2- 172.21.15.82:0/1886609575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a80ffe50 0x7f66a819c9a0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f669400e9a0 tx=0x7f669400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:24:27.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.350+0000 7f66a5ffb700 1 -- 172.21.15.82:0/1886609575 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6694009790 con 0x7f66a80ffe50 2023-09-09T23:24:27.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.350+0000 7f66aea6d700 1 -- 172.21.15.82:0/1886609575 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66a81971a0 con 0x7f66a80ffe50 2023-09-09T23:24:27.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.350+0000 7f66a5ffb700 1 -- 172.21.15.82:0/1886609575 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6694010430 con 0x7f66a80ffe50 2023-09-09T23:24:27.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.350+0000 7f66a5ffb700 1 -- 172.21.15.82:0/1886609575 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6694018680 con 0x7f66a80ffe50 2023-09-09T23:24:27.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.350+0000 7f66aea6d700 1 -- 172.21.15.82:0/1886609575 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a81a3500 con 0x7f66a80ffe50 2023-09-09T23:24:27.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.353+0000 7f66a5ffb700 1 -- 172.21.15.82:0/1886609575 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f66940105a0 con 0x7f66a80ffe50 2023-09-09T23:24:27.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.353+0000 7f66a5ffb700 1 --2- 172.21.15.82:0/1886609575 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f66980a07b0 0x7f66980a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:27.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.353+0000 7f66a5ffb700 1 -- 172.21.15.82:0/1886609575 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f6694014070 con 0x7f66a80ffe50 2023-09-09T23:24:27.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.353+0000 7f66ac809700 1 --2- 172.21.15.82:0/1886609575 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f66980a07b0 0x7f66980a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:27.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.354+0000 7f66ac809700 1 --2- 172.21.15.82:0/1886609575 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f66980a07b0 0x7f66980a2c70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f669c009990 tx=0x7f669c005fb0 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:24:27.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.354+0000 7f66bac33b80 1 Processor -- start 2023-09-09T23:24:27.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.354+0000 7f66bac33b80 1 -- start start 2023-09-09T23:24:27.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.354+0000 7f66bac33b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fb5509710 0x562fb5509af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:27.365 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.354+0000 7f66bac33b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fb550a030 0x562fb5513900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:27.365 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.354+0000 7f66bac33b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fb5513e40 0x562fb5516230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:27.365 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.354+0000 7f66bac33b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb5462730 con 0x562fb5509710 2023-09-09T23:24:27.365 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.354+0000 7f66bac33b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb5460050 con 0x562fb5513e40 2023-09-09T23:24:27.366 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.354+0000 7f66bac33b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb545b940 con 0x562fb550a030 2023-09-09T23:24:27.366 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.354+0000 7f66ac809700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fb5509710 0x562fb5509af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:27.366 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.354+0000 7f66ac809700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fb5509710 0x562fb5509af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58302/0 (socket says 172.21.15.82:58302) 2023-09-09T23:24:27.366 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.354+0000 7f66ac809700 1 -- 172.21.15.82:0/1055094407 learned_addr learned my addr 172.21.15.82:0/1055094407 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:24:27.366 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.354+0000 7f66ad00a700 1 --2- 172.21.15.82:0/1055094407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fb5513e40 0x562fb5516230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:27.367 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.354+0000 7f66a7fff700 1 --2- 172.21.15.82:0/1055094407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fb550a030 0x562fb5513900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:27.367 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.355+0000 7f66ac809700 1 -- 172.21.15.82:0/1055094407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fb550a030 msgr2=0x562fb5513900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:24:27.367 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.355+0000 7f66ac809700 1 --2- 172.21.15.82:0/1055094407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fb550a030 0x562fb5513900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.367 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.355+0000 7f66ac809700 1 -- 172.21.15.82:0/1055094407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fb5513e40 msgr2=0x562fb5516230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:24:27.368 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.355+0000 7f66ac809700 1 --2- 172.21.15.82:0/1055094407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fb5513e40 0x562fb5516230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.368 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.355+0000 7f66ac809700 1 -- 172.21.15.82:0/1055094407 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fb5287de0 con 0x562fb5509710 2023-09-09T23:24:27.368 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.355+0000 7f66ac809700 1 --2- 172.21.15.82:0/1055094407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fb5509710 0x562fb5509af0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f669c0099c0 tx=0x7f669c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b883bbe93f3f9874 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:24:27.368 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.355+0000 7f6692ffd700 1 -- 172.21.15.82:0/1055094407 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c00b990 con 0x562fb5509710 2023-09-09T23:24:27.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.355+0000 7f6692ffd700 1 -- 172.21.15.82:0/1055094407 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f669c01e040 con 0x562fb5509710 2023-09-09T23:24:27.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.355+0000 7f6692ffd700 1 -- 172.21.15.82:0/1055094407 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c00f460 con 0x562fb5509710 2023-09-09T23:24:27.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.356+0000 7f66bac33b80 1 -- 172.21.15.82:0/1055094407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fb5509710 msgr2=0x562fb5509af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:24:27.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.356+0000 7f66bac33b80 1 --2- 172.21.15.82:0/1055094407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fb5509710 0x562fb5509af0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f669c0099c0 tx=0x7f669c01f000 comp rx=0 tx=0).stop 2023-09-09T23:24:27.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.356+0000 7f66bac33b80 1 -- 172.21.15.82:0/1055094407 shutdown_connections 2023-09-09T23:24:27.370 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.356+0000 7f66bac33b80 1 --2- 172.21.15.82:0/1055094407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fb5513e40 0x562fb5516230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.370 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.356+0000 7f66bac33b80 1 --2- 172.21.15.82:0/1055094407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fb5509710 0x562fb5509af0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.370 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.356+0000 7f66bac33b80 1 --2- 172.21.15.82:0/1055094407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fb550a030 0x562fb5513900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.370 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.356+0000 7f66bac33b80 1 -- 172.21.15.82:0/1055094407 >> 172.21.15.82:0/1055094407 conn(0x562fb547e090 msgr2=0x562fb5504bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:24:27.370 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.356+0000 7f66bac33b80 1 -- 172.21.15.82:0/1055094407 shutdown_connections 2023-09-09T23:24:27.371 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.356+0000 7f66bac33b80 1 -- 172.21.15.82:0/1055094407 wait complete. 2023-09-09T23:24:27.371 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.356+0000 7f66bac33b80 1 -- start start 2023-09-09T23:24:27.371 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.356+0000 7f66bac33b80 1 Processor -- start 2023-09-09T23:24:27.371 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.356+0000 7f66bac33b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fb5509710 0x562fb5513840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:27.371 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.356+0000 7f66bac33b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fb550a030 0x562fb5510b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:27.371 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.356+0000 7f66bac33b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fb5513e40 0x562fb55111b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:27.372 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.356+0000 7f66bac33b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb53f9b70 con 0x562fb5509710 2023-09-09T23:24:27.372 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.357+0000 7f66bac33b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb53f9870 con 0x562fb550a030 2023-09-09T23:24:27.372 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.357+0000 7f66bac33b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb52dd920 con 0x562fb5513e40 2023-09-09T23:24:27.372 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.357+0000 7f66bac33b80 10 client.0 _ll_register_callbacks cb 0x562fb5343a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:24:27.373 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.357+0000 7f66ac809700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fb5509710 0x562fb5513840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:27.373 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.357+0000 7f66ac809700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fb5509710 0x562fb5513840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58308/0 (socket says 172.21.15.82:58308) 2023-09-09T23:24:27.373 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.357+0000 7f66ac809700 1 -- 172.21.15.82:0/1747580519 learned_addr learned my addr 172.21.15.82:0/1747580519 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:24:27.373 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.357+0000 7f66a7fff700 1 --2- 172.21.15.82:0/1747580519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fb550a030 0x562fb5510b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:27.373 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.357+0000 7f66ad00a700 1 --2- 172.21.15.82:0/1747580519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fb5513e40 0x562fb55111b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:27.374 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.357+0000 7f66ac809700 1 -- 172.21.15.82:0/1747580519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fb5513e40 msgr2=0x562fb55111b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:24:27.374 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.357+0000 7f66ac809700 1 --2- 172.21.15.82:0/1747580519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fb5513e40 0x562fb55111b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.374 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.357+0000 7f66ac809700 1 -- 172.21.15.82:0/1747580519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fb550a030 msgr2=0x562fb5510b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:24:27.374 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.357+0000 7f66ac809700 1 --2- 172.21.15.82:0/1747580519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fb550a030 0x562fb5510b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.375 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.357+0000 7f66ac809700 1 -- 172.21.15.82:0/1747580519 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a810cc10 con 0x562fb5509710 2023-09-09T23:24:27.375 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.357+0000 7f66ac809700 1 --2- 172.21.15.82:0/1747580519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fb5509710 0x562fb5513840 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f669c0036c0 tx=0x7f669c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eeac33291c0edb4 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:24:27.375 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.357+0000 7f6687fff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:24:27.375 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.357+0000 7f6687fff700 1 -- 172.21.15.82:0/1747580519 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c00bba0 con 0x562fb5509710 2023-09-09T23:24:27.375 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.357+0000 7f6687fff700 1 -- 172.21.15.82:0/1747580519 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f669c004100 con 0x562fb5509710 2023-09-09T23:24:27.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.357+0000 7f66bac33b80 1 -- 172.21.15.82:0/1747580519 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562fb53b34f0 con 0x562fb5509710 2023-09-09T23:24:27.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.357+0000 7f6687fff700 1 -- 172.21.15.82:0/1747580519 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7f669c00bd00 con 0x562fb5509710 2023-09-09T23:24:27.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.357+0000 7f66bac33b80 10 client.16299 fetch_fsmap learned FSMap version 127 2023-09-09T23:24:27.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.357+0000 7f66bac33b80 1 -- 172.21.15.82:0/1747580519 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x562fb5270c60 con 0x562fb5509710 2023-09-09T23:24:27.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.358+0000 7f6687fff700 1 -- 172.21.15.82:0/1747580519 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562fb53b34f0 con 0x562fb5509710 2023-09-09T23:24:27.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.358+0000 7f6687fff700 1 -- 172.21.15.82:0/1747580519 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 127) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f669c030790 con 0x562fb5509710 2023-09-09T23:24:27.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.358+0000 7f66bac33b80 10 client.16299 fetch_fsmap finished waiting for FSMap version 127 2023-09-09T23:24:27.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.358+0000 7f66bac33b80 10 client.16299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:24:27.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.358+0000 7f66bac33b80 20 client.16299 populate_metadata read hostname 'smithi082' 2023-09-09T23:24:27.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.358+0000 7f66bac33b80 1 --2- 172.21.15.82:0/1747580519 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x562fb5523520 0x562fb5525910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:27.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.358+0000 7f66bac33b80 4 client.16299 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:24:27.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.358+0000 7f66bac33b80 1 -- 172.21.15.82:0/1747580519 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562fb54708a0 con 0x562fb5523520 2023-09-09T23:24:27.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.358+0000 7f66a7fff700 1 --2- 172.21.15.82:0/1747580519 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x562fb5523520 0x562fb5525910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:27.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.358+0000 7f6687fff700 10 client.16299 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:24:27.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.358+0000 7f66a7fff700 1 --2- 172.21.15.82:0/1747580519 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x562fb5523520 0x562fb5525910 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d73d349f89a241e0 server_cookie=f851c3589e9c91ee in_seq=0 out_seq=0 2023-09-09T23:24:27.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.359+0000 7f6687fff700 1 -- 172.21.15.82:0/1747580519 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562fb54708a0 con 0x562fb5523520 2023-09-09T23:24:27.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.359+0000 7f6687fff700 10 client.16299 handle_command_reply: tid=0 2023-09-09T23:24:27.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.359+0000 7f66bac33b80 1 client.16299 shutdown 2023-09-09T23:24:27.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.359+0000 7f66bac33b80 1 -- 172.21.15.82:0/1747580519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fb5509710 msgr2=0x562fb5513840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:24:27.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.359+0000 7f66bac33b80 1 --2- 172.21.15.82:0/1747580519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fb5509710 0x562fb5513840 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f669c0036c0 tx=0x7f669c003970 comp rx=0 tx=0).stop 2023-09-09T23:24:27.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.359+0000 7f66bac33b80 1 -- 172.21.15.82:0/1747580519 shutdown_connections 2023-09-09T23:24:27.380 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.359+0000 7f66bac33b80 1 --2- 172.21.15.82:0/1747580519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fb5513e40 0x562fb55111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.380 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.359+0000 7f66bac33b80 1 --2- 172.21.15.82:0/1747580519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fb5509710 0x562fb5513840 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.380 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.359+0000 7f66bac33b80 1 --2- 172.21.15.82:0/1747580519 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x562fb5523520 0x562fb5525910 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.380 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.359+0000 7f66bac33b80 1 --2- 172.21.15.82:0/1747580519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fb550a030 0x562fb5510b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.380 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.359+0000 7f6687fff700 0 client.16299 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:24:27.381 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.359+0000 7f66bac33b80 1 -- 172.21.15.82:0/1747580519 >> 172.21.15.82:0/1747580519 conn(0x562fb547e090 msgr2=0x562fb55047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:24:27.381 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.359+0000 7f66bac33b80 1 -- 172.21.15.82:0/1747580519 shutdown_connections 2023-09-09T23:24:27.381 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.359+0000 7f66bac33b80 1 -- 172.21.15.82:0/1747580519 wait complete. 2023-09-09T23:24:27.381 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.359+0000 7f66bac33b80 20 client.16299 trim_cache size 0 max 16384 2023-09-09T23:24:27.386 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.376+0000 7f66bac33b80 1 Processor -- start 2023-09-09T23:24:27.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.376+0000 7f66bac33b80 1 -- start start 2023-09-09T23:24:27.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.376+0000 7f66bac33b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fb5509710 0x562fb551a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:27.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.376+0000 7f66bac33b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fb550a030 0x562fb551f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:27.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.376+0000 7f66bac33b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fb5513e40 0x562fb55046f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:27.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.376+0000 7f66bac33b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb5520310 con 0x562fb5509710 2023-09-09T23:24:27.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.376+0000 7f66bac33b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb551aed0 con 0x562fb550a030 2023-09-09T23:24:27.388 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.376+0000 7f66bac33b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb550a410 con 0x562fb5513e40 2023-09-09T23:24:27.388 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.376+0000 7f66ac809700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fb5509710 0x562fb551a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:27.388 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.376+0000 7f66ac809700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fb5509710 0x562fb551a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58316/0 (socket says 172.21.15.82:58316) 2023-09-09T23:24:27.388 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.376+0000 7f66ac809700 1 -- 172.21.15.82:0/3312029405 learned_addr learned my addr 172.21.15.82:0/3312029405 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:24:27.388 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.376+0000 7f66ad00a700 1 --2- 172.21.15.82:0/3312029405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fb5513e40 0x562fb55046f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.376+0000 7f66a7fff700 1 --2- 172.21.15.82:0/3312029405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fb550a030 0x562fb551f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.376+0000 7f66ac809700 1 -- 172.21.15.82:0/3312029405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fb5513e40 msgr2=0x562fb55046f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:24:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.376+0000 7f66ac809700 1 --2- 172.21.15.82:0/3312029405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fb5513e40 0x562fb55046f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.376+0000 7f66ac809700 1 -- 172.21.15.82:0/3312029405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fb550a030 msgr2=0x562fb551f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:24:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.376+0000 7f66ac809700 1 --2- 172.21.15.82:0/3312029405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fb550a030 0x562fb551f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.390 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.376+0000 7f66ac809700 1 -- 172.21.15.82:0/3312029405 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fb52ae250 con 0x562fb5509710 2023-09-09T23:24:27.390 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.376+0000 7f66ac809700 1 --2- 172.21.15.82:0/3312029405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fb5509710 0x562fb551a990 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f669c030b40 tx=0x7f669c003e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5aa1352ab102897 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:24:27.390 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.377+0000 7f66937fe700 1 -- 172.21.15.82:0/3312029405 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c004300 con 0x562fb5509710 2023-09-09T23:24:27.390 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.377+0000 7f66937fe700 1 -- 172.21.15.82:0/3312029405 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f669c00b9a0 con 0x562fb5509710 2023-09-09T23:24:27.390 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.377+0000 7f66937fe700 1 -- 172.21.15.82:0/3312029405 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c00faf0 con 0x562fb5509710 2023-09-09T23:24:27.391 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.377+0000 7f66bac33b80 1 -- 172.21.15.82:0/3312029405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fb5509710 msgr2=0x562fb551a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:24:27.391 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.377+0000 7f66bac33b80 1 --2- 172.21.15.82:0/3312029405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fb5509710 0x562fb551a990 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f669c030b40 tx=0x7f669c003e40 comp rx=0 tx=0).stop 2023-09-09T23:24:27.391 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.377+0000 7f66bac33b80 1 -- 172.21.15.82:0/3312029405 shutdown_connections 2023-09-09T23:24:27.391 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.377+0000 7f66bac33b80 1 --2- 172.21.15.82:0/3312029405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fb5513e40 0x562fb55046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.391 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.377+0000 7f66bac33b80 1 --2- 172.21.15.82:0/3312029405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fb5509710 0x562fb551a990 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.392 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.377+0000 7f66bac33b80 1 --2- 172.21.15.82:0/3312029405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fb550a030 0x562fb551f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.392 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.377+0000 7f66bac33b80 1 -- 172.21.15.82:0/3312029405 >> 172.21.15.82:0/3312029405 conn(0x562fb547e090 msgr2=0x562fb551fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:24:27.392 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.377+0000 7f66bac33b80 1 -- 172.21.15.82:0/3312029405 shutdown_connections 2023-09-09T23:24:27.392 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.377+0000 7f66bac33b80 1 -- 172.21.15.82:0/3312029405 wait complete. 2023-09-09T23:24:27.392 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.377+0000 7f66bac33b80 1 -- start start 2023-09-09T23:24:27.393 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.378+0000 7f66bac33b80 1 Processor -- start 2023-09-09T23:24:27.393 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.378+0000 7f66bac33b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fb5509710 0x562fb551a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:27.393 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.378+0000 7f66bac33b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fb550a030 0x562fb551a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:27.393 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.378+0000 7f66bac33b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fb5513e40 0x562fb5518750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:27.393 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.378+0000 7f66bac33b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb5518d20 con 0x562fb550a030 2023-09-09T23:24:27.394 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.378+0000 7f66bac33b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb5518e90 con 0x562fb5513e40 2023-09-09T23:24:27.394 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.378+0000 7f66bac33b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fb5519050 con 0x562fb5509710 2023-09-09T23:24:27.394 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.378+0000 7f66bac33b80 10 client.0 _ll_register_callbacks cb 0x7f66a0009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:24:27.394 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.378+0000 7f66a7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fb550a030 0x562fb551a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:27.394 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.378+0000 7f66ac809700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fb5509710 0x562fb551a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:27.395 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.378+0000 7f66ac809700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fb5509710 0x562fb551a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:56836/0 (socket says 172.21.15.82:56836) 2023-09-09T23:24:27.395 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.378+0000 7f66ac809700 1 -- 172.21.15.82:0/896860531 learned_addr learned my addr 172.21.15.82:0/896860531 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:24:27.395 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.378+0000 7f66ad00a700 1 --2- 172.21.15.82:0/896860531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fb5513e40 0x562fb5518750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:27.395 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.378+0000 7f66ac809700 1 -- 172.21.15.82:0/896860531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fb5513e40 msgr2=0x562fb5518750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:24:27.395 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.378+0000 7f66ac809700 1 --2- 172.21.15.82:0/896860531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fb5513e40 0x562fb5518750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.395 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.378+0000 7f66ac809700 1 -- 172.21.15.82:0/896860531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fb550a030 msgr2=0x562fb551a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:24:27.396 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.378+0000 7f66ac809700 1 --2- 172.21.15.82:0/896860531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fb550a030 0x562fb551a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.396 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.378+0000 7f66ac809700 1 -- 172.21.15.82:0/896860531 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562fb5270c60 con 0x562fb5509710 2023-09-09T23:24:27.396 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.378+0000 7f66a7fff700 1 --2- 172.21.15.82:0/896860531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fb550a030 0x562fb551a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:24:27.396 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.379+0000 7f66ac809700 1 --2- 172.21.15.82:0/896860531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fb5509710 0x562fb551a290 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f669c0058e0 tx=0x7f669c00bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72ac69675fc82c4a server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:24:27.396 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.379+0000 7f6687fff700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-09T23:24:27.397 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.379+0000 7f6687fff700 1 -- 172.21.15.82:0/896860531 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c030890 con 0x562fb5509710 2023-09-09T23:24:27.397 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.379+0000 7f66bac33b80 1 -- 172.21.15.82:0/896860531 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f669c01e040 con 0x562fb5509710 2023-09-09T23:24:27.397 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.379+0000 7f6687fff700 1 -- 172.21.15.82:0/896860531 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f669c00f670 con 0x562fb5509710 2023-09-09T23:24:27.397 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.379+0000 7f6687fff700 1 -- 172.21.15.82:0/896860531 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7f669c00f830 con 0x562fb5509710 2023-09-09T23:24:27.397 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.379+0000 7f66bac33b80 10 client.25672 fetch_fsmap learned FSMap version 127 2023-09-09T23:24:27.398 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.379+0000 7f66bac33b80 1 -- 172.21.15.82:0/896860531 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x562fb52862b0 con 0x562fb5509710 2023-09-09T23:24:27.398 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.379+0000 7f6687fff700 1 -- 172.21.15.82:0/896860531 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f669c01e040 con 0x562fb5509710 2023-09-09T23:24:27.398 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.379+0000 7f6687fff700 1 -- 172.21.15.82:0/896860531 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 127) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f669c049070 con 0x562fb5509710 2023-09-09T23:24:27.398 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.379+0000 7f66bac33b80 10 client.25672 fetch_fsmap finished waiting for FSMap version 127 2023-09-09T23:24:27.398 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.379+0000 7f66bac33b80 10 client.25672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:24:27.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.379+0000 7f66bac33b80 20 client.25672 populate_metadata read hostname 'smithi082' 2023-09-09T23:24:27.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.379+0000 7f66bac33b80 1 --2- 172.21.15.82:0/896860531 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x562fb557c3a0 0x562fb557e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:27.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.379+0000 7f66bac33b80 4 client.25672 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:24:27.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.379+0000 7f66bac33b80 1 -- 172.21.15.82:0/896860531 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562fb5454920 con 0x562fb557c3a0 2023-09-09T23:24:27.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.379+0000 7f66a7fff700 1 --2- 172.21.15.82:0/896860531 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x562fb557c3a0 0x562fb557e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:27.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.380+0000 7f6687fff700 10 client.25672 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:24:27.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.380+0000 7f66a7fff700 1 --2- 172.21.15.82:0/896860531 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x562fb557c3a0 0x562fb557e790 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f0a87935c6e19d1 server_cookie=90656969a243d4a7 in_seq=0 out_seq=0 2023-09-09T23:24:27.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.380+0000 7f6687fff700 1 -- 172.21.15.82:0/896860531 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x562fb5454920 con 0x562fb557c3a0 2023-09-09T23:24:27.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.380+0000 7f6687fff700 10 client.25672 handle_command_reply: tid=0 2023-09-09T23:24:27.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.380+0000 7f66bac33b80 1 client.25672 shutdown 2023-09-09T23:24:27.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.380+0000 7f66bac33b80 1 -- 172.21.15.82:0/896860531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fb5509710 msgr2=0x562fb551a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:24:27.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.380+0000 7f66bac33b80 1 --2- 172.21.15.82:0/896860531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fb5509710 0x562fb551a290 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f669c0058e0 tx=0x7f669c00bda0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.380+0000 7f66bac33b80 1 -- 172.21.15.82:0/896860531 shutdown_connections 2023-09-09T23:24:27.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.380+0000 7f66bac33b80 1 --2- 172.21.15.82:0/896860531 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x562fb557c3a0 0x562fb557e790 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.380+0000 7f66bac33b80 1 --2- 172.21.15.82:0/896860531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562fb5513e40 0x562fb5518750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.380+0000 7f66bac33b80 1 --2- 172.21.15.82:0/896860531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fb5509710 0x562fb551a290 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.380+0000 7f6687fff700 0 client.25672 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:24:27.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.380+0000 7f66bac33b80 1 --2- 172.21.15.82:0/896860531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fb550a030 0x562fb551a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.380+0000 7f66bac33b80 1 -- 172.21.15.82:0/896860531 >> 172.21.15.82:0/896860531 conn(0x562fb547e090 msgr2=0x562fb551f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:24:27.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.380+0000 7f66bac33b80 1 -- 172.21.15.82:0/896860531 shutdown_connections 2023-09-09T23:24:27.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.380+0000 7f66bac33b80 1 -- 172.21.15.82:0/896860531 wait complete. 2023-09-09T23:24:27.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.380+0000 7f66bac33b80 20 client.25672 trim_cache size 0 max 16384 2023-09-09T23:24:27.403 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:24:27.403 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:24:27.403 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:24:27.404 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:24:27.404 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:24:27.404 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:24:27.404 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:24:27.404 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:24:27.405 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:24:27.405 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:24:27.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.381+0000 7f66aea6d700 1 -- 172.21.15.82:0/1886609575 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f66980a07b0 msgr2=0x7f66980a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:24:27.407 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.381+0000 7f66aea6d700 1 --2- 172.21.15.82:0/1886609575 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f66980a07b0 0x7f66980a2c70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f669c009990 tx=0x7f669c005fb0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.407 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.381+0000 7f66aea6d700 1 -- 172.21.15.82:0/1886609575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a80ffe50 msgr2=0x7f66a819c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:24:27.407 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.381+0000 7f66aea6d700 1 --2- 172.21.15.82:0/1886609575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a80ffe50 0x7f66a819c9a0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f669400e9a0 tx=0x7f669400ecb0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.407 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.381+0000 7f66aea6d700 1 -- 172.21.15.82:0/1886609575 shutdown_connections 2023-09-09T23:24:27.407 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.381+0000 7f66aea6d700 1 --2- 172.21.15.82:0/1886609575 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f66980a07b0 0x7f66980a2c70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.408 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.381+0000 7f66aea6d700 1 --2- 172.21.15.82:0/1886609575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66a8105550 0x7f66a8196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.408 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.381+0000 7f66aea6d700 1 --2- 172.21.15.82:0/1886609575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66a80ff490 0x7f66a819c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.408 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.381+0000 7f66aea6d700 1 --2- 172.21.15.82:0/1886609575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a80ffe50 0x7f66a819c9a0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:27.408 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.381+0000 7f66aea6d700 1 -- 172.21.15.82:0/1886609575 >> 172.21.15.82:0/1886609575 conn(0x7f66a80faf20 msgr2=0x7f66a80fc990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:24:27.408 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.381+0000 7f66aea6d700 1 -- 172.21.15.82:0/1886609575 shutdown_connections 2023-09-09T23:24:27.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:27.381+0000 7f66aea6d700 1 -- 172.21.15.82:0/1886609575 wait complete. 2023-09-09T23:24:27.409 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:24:27.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:27 smithi170 ceph-mon[102832]: pgmap v329: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 KiB/s wr, 16 op/s 2023-09-09T23:24:27.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 298.388216 secs 2023-09-09T23:24:27.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:27 smithi082 ceph-mon[98694]: pgmap v329: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 KiB/s wr, 16 op/s 2023-09-09T23:24:27.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 298.388216 secs 2023-09-09T23:24:27.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:27 smithi098 ceph-mon[103124]: pgmap v329: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 KiB/s wr, 16 op/s 2023-09-09T23:24:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 298.388216 secs 2023-09-09T23:24:28.676 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 300.022463 secs 2023-09-09T23:24:28.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 300.022463 secs 2023-09-09T23:24:28.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 300.022463 secs 2023-09-09T23:24:29.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:29 smithi098 ceph-mon[103124]: pgmap v330: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 KiB/s wr, 11 op/s 2023-09-09T23:24:29.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 300.751523 secs 2023-09-09T23:24:29.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:29 smithi170 ceph-mon[102832]: pgmap v330: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 KiB/s wr, 11 op/s 2023-09-09T23:24:29.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 300.751523 secs 2023-09-09T23:24:29.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:29 smithi082 ceph-mon[98694]: pgmap v330: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 KiB/s wr, 11 op/s 2023-09-09T23:24:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 300.751523 secs 2023-09-09T23:24:31.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:31 smithi098 ceph-mon[103124]: pgmap v331: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2023-09-09T23:24:31.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:31 smithi170 ceph-mon[102832]: pgmap v331: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2023-09-09T23:24:31.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:31 smithi082 ceph-mon[98694]: pgmap v331: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2023-09-09T23:24:32.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 303.388279 secs 2023-09-09T23:24:32.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 303.388279 secs 2023-09-09T23:24:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 303.388279 secs 2023-09-09T23:24:33.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:33 smithi098 ceph-mon[103124]: pgmap v332: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2023-09-09T23:24:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 305.022495 secs 2023-09-09T23:24:33.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:33 smithi170 ceph-mon[102832]: pgmap v332: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2023-09-09T23:24:33.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 305.022495 secs 2023-09-09T23:24:33.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:33 smithi082 ceph-mon[98694]: pgmap v332: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2023-09-09T23:24:33.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 305.022495 secs 2023-09-09T23:24:34.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 305.751586 secs 2023-09-09T23:24:34.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 305.751586 secs 2023-09-09T23:24:34.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 305.751586 secs 2023-09-09T23:24:35.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:35 smithi098 ceph-mon[103124]: pgmap v333: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2023-09-09T23:24:35.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:35 smithi170 ceph-mon[102832]: pgmap v333: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2023-09-09T23:24:35.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:35 smithi082 ceph-mon[98694]: pgmap v333: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2023-09-09T23:24:37.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:37 smithi098 ceph-mon[103124]: pgmap v334: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2023-09-09T23:24:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 308.401271 secs 2023-09-09T23:24:37.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:37 smithi170 ceph-mon[102832]: pgmap v334: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2023-09-09T23:24:37.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 308.401271 secs 2023-09-09T23:24:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:37 smithi082 ceph-mon[98694]: pgmap v334: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2023-09-09T23:24:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 308.401271 secs 2023-09-09T23:24:38.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 310.022556 secs 2023-09-09T23:24:38.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 310.022556 secs 2023-09-09T23:24:38.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 310.022556 secs 2023-09-09T23:24:39.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:39 smithi098 ceph-mon[103124]: pgmap v335: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2023-09-09T23:24:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 310.751651 secs 2023-09-09T23:24:39.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:39 smithi170 ceph-mon[102832]: pgmap v335: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2023-09-09T23:24:39.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 310.751651 secs 2023-09-09T23:24:39.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:39 smithi082 ceph-mon[98694]: pgmap v335: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2023-09-09T23:24:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 310.751651 secs 2023-09-09T23:24:41.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:41 smithi098 ceph-mon[103124]: pgmap v336: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2023-09-09T23:24:41.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:41 smithi170 ceph-mon[102832]: pgmap v336: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2023-09-09T23:24:41.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:41 smithi082 ceph-mon[98694]: pgmap v336: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2023-09-09T23:24:42.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 313.401330 secs 2023-09-09T23:24:42.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 313.401330 secs 2023-09-09T23:24:42.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 313.401330 secs 2023-09-09T23:24:43.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:43 smithi098 ceph-mon[103124]: pgmap v337: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:24:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 315.022616 secs 2023-09-09T23:24:43.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:43 smithi170 ceph-mon[102832]: pgmap v337: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:24:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 315.022616 secs 2023-09-09T23:24:43.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:43 smithi082 ceph-mon[98694]: pgmap v337: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:24:43.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 315.022616 secs 2023-09-09T23:24:44.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 315.751718 secs 2023-09-09T23:24:44.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 315.751718 secs 2023-09-09T23:24:45.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 315.751718 secs 2023-09-09T23:24:45.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:45 smithi170 ceph-mon[102832]: pgmap v338: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:24:45.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:45 smithi082 ceph-mon[98694]: pgmap v338: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:24:46.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:45 smithi098 ceph-mon[103124]: pgmap v338: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:24:47.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:47 smithi170 ceph-mon[102832]: pgmap v339: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:24:47.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 318.401391 secs 2023-09-09T23:24:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:47 smithi082 ceph-mon[98694]: pgmap v339: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:24:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 318.401391 secs 2023-09-09T23:24:48.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:47 smithi098 ceph-mon[103124]: pgmap v339: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:24:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 318.401391 secs 2023-09-09T23:24:48.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 320.022676 secs 2023-09-09T23:24:48.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 320.022676 secs 2023-09-09T23:24:49.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 320.022676 secs 2023-09-09T23:24:49.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:49 smithi170 ceph-mon[102832]: pgmap v340: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:24:49.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 320.751779 secs 2023-09-09T23:24:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:49 smithi082 ceph-mon[98694]: pgmap v340: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:24:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 320.751779 secs 2023-09-09T23:24:50.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:49 smithi098 ceph-mon[103124]: pgmap v340: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:24:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 320.751779 secs 2023-09-09T23:24:51.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:51 smithi170 ceph-mon[102832]: pgmap v341: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-09-09T23:24:51.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:51 smithi082 ceph-mon[98694]: pgmap v341: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-09-09T23:24:52.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:51 smithi098 ceph-mon[103124]: pgmap v341: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-09-09T23:24:52.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 323.401453 secs 2023-09-09T23:24:52.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 323.401453 secs 2023-09-09T23:24:53.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 323.401453 secs 2023-09-09T23:24:53.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:53 smithi170 ceph-mon[102832]: pgmap v342: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:24:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 325.022728 secs 2023-09-09T23:24:53.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:53 smithi082 ceph-mon[98694]: pgmap v342: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:24:53.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 325.022728 secs 2023-09-09T23:24:54.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:53 smithi098 ceph-mon[103124]: pgmap v342: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:24:54.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 325.022728 secs 2023-09-09T23:24:54.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 325.751851 secs 2023-09-09T23:24:54.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 325.751851 secs 2023-09-09T23:24:55.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 325.751851 secs 2023-09-09T23:24:55.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:55 smithi170 ceph-mon[102832]: pgmap v343: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:24:55.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:55 smithi082 ceph-mon[98694]: pgmap v343: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:24:56.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:55 smithi098 ceph-mon[103124]: pgmap v343: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:24:57.411 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:24:57.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.562+0000 7f0ba8813700 1 -- 172.21.15.82:0/3469148265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ba4105560 msgr2=0x7f0ba4105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:24:57.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.562+0000 7f0ba8813700 1 --2- 172.21.15.82:0/3469148265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ba4105560 0x7f0ba4105940 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f0b940099c0 tx=0x7f0b94009cd0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.562+0000 7f0ba8813700 1 -- 172.21.15.82:0/3469148265 shutdown_connections 2023-09-09T23:24:57.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.562+0000 7f0ba8813700 1 --2- 172.21.15.82:0/3469148265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ba40ffe60 0x7f0ba410c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.562+0000 7f0ba8813700 1 --2- 172.21.15.82:0/3469148265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ba4105560 0x7f0ba4105940 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.562+0000 7f0ba8813700 1 --2- 172.21.15.82:0/3469148265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ba40ff4a0 0x7f0ba40ff920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.562+0000 7f0ba8813700 1 -- 172.21.15.82:0/3469148265 >> 172.21.15.82:0/3469148265 conn(0x7f0ba40faf50 msgr2=0x7f0ba40fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:24:57.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.562+0000 7f0ba8813700 1 -- 172.21.15.82:0/3469148265 shutdown_connections 2023-09-09T23:24:57.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.562+0000 7f0ba8813700 1 -- 172.21.15.82:0/3469148265 wait complete. 2023-09-09T23:24:57.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.562+0000 7f0ba8813700 1 Processor -- start 2023-09-09T23:24:57.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.563+0000 7f0ba8813700 1 -- start start 2023-09-09T23:24:57.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.563+0000 7f0ba8813700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ba40ff4a0 0x7f0ba4197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:57.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.563+0000 7f0ba8813700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ba40ffe60 0x7f0ba4197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:57.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.563+0000 7f0ba8813700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ba4105560 0x7f0ba41a24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:57.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.563+0000 7f0ba8813700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba41982d0 con 0x7f0ba4105560 2023-09-09T23:24:57.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.563+0000 7f0ba8813700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba41a2a10 con 0x7f0ba40ff4a0 2023-09-09T23:24:57.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.563+0000 7f0ba8813700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba41a2bd0 con 0x7f0ba40ffe60 2023-09-09T23:24:57.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.563+0000 7f0ba259c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ba40ff4a0 0x7f0ba4197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:57.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.563+0000 7f0ba2d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ba4105560 0x7f0ba41a24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:57.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.563+0000 7f0ba259c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ba40ff4a0 0x7f0ba4197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:43900/0 (socket says 172.21.15.82:43900) 2023-09-09T23:24:57.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.563+0000 7f0ba259c700 1 -- 172.21.15.82:0/2686178946 learned_addr learned my addr 172.21.15.82:0/2686178946 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:24:57.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.563+0000 7f0ba1d9b700 1 --2- 172.21.15.82:0/2686178946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ba40ffe60 0x7f0ba4197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:57.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.563+0000 7f0ba259c700 1 -- 172.21.15.82:0/2686178946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ba40ffe60 msgr2=0x7f0ba4197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:24:57.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.563+0000 7f0ba259c700 1 --2- 172.21.15.82:0/2686178946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ba40ffe60 0x7f0ba4197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.563+0000 7f0ba259c700 1 -- 172.21.15.82:0/2686178946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ba4105560 msgr2=0x7f0ba41a24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:24:57.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.563+0000 7f0ba259c700 1 --2- 172.21.15.82:0/2686178946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ba4105560 0x7f0ba41a24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.563+0000 7f0ba259c700 1 -- 172.21.15.82:0/2686178946 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b940095f0 con 0x7f0ba40ff4a0 2023-09-09T23:24:57.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.563+0000 7f0ba2d9d700 1 --2- 172.21.15.82:0/2686178946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ba4105560 0x7f0ba41a24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:24:57.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.563+0000 7f0ba259c700 1 --2- 172.21.15.82:0/2686178946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ba40ff4a0 0x7f0ba4197450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400b580 tx=0x7f0b9400bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:24:57.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.564+0000 7f0b937fe700 1 -- 172.21.15.82:0/2686178946 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9401d070 con 0x7f0ba40ff4a0 2023-09-09T23:24:57.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.564+0000 7f0b937fe700 1 -- 172.21.15.82:0/2686178946 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b94022470 con 0x7f0ba40ff4a0 2023-09-09T23:24:57.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.564+0000 7f0ba8813700 1 -- 172.21.15.82:0/2686178946 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba41a2e50 con 0x7f0ba40ff4a0 2023-09-09T23:24:57.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.564+0000 7f0b937fe700 1 -- 172.21.15.82:0/2686178946 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b94017650 con 0x7f0ba40ff4a0 2023-09-09T23:24:57.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.564+0000 7f0ba8813700 1 -- 172.21.15.82:0/2686178946 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba41a3270 con 0x7f0ba40ff4a0 2023-09-09T23:24:57.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.566+0000 7f0b937fe700 1 -- 172.21.15.82:0/2686178946 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f0b940225e0 con 0x7f0ba40ff4a0 2023-09-09T23:24:57.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.566+0000 7f0b937fe700 1 --2- 172.21.15.82:0/2686178946 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f0b8c0a0830 0x7f0b8c0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:57.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.566+0000 7f0b937fe700 1 -- 172.21.15.82:0/2686178946 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f0b940ccf20 con 0x7f0ba40ff4a0 2023-09-09T23:24:57.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.566+0000 7f0ba1d9b700 1 --2- 172.21.15.82:0/2686178946 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f0b8c0a0830 0x7f0b8c0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:57.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.567+0000 7f0ba1d9b700 1 --2- 172.21.15.82:0/2686178946 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f0b8c0a0830 0x7f0b8c0a2cf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4069000 tx=0x7f0b98009260 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:24:57.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.567+0000 7f0bb49d9b80 1 Processor -- start 2023-09-09T23:24:57.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.567+0000 7f0bb49d9b80 1 -- start start 2023-09-09T23:24:57.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.568+0000 7f0bb49d9b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624f8dc9710 0x5624f8dc9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:57.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.568+0000 7f0bb49d9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624f8dca030 0x5624f8dd3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:57.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.568+0000 7f0bb49d9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f8dd3e40 0x5624f8dd6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:57.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.568+0000 7f0bb49d9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f8d22730 con 0x5624f8dca030 2023-09-09T23:24:57.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.568+0000 7f0bb49d9b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f8d20050 con 0x5624f8dc9710 2023-09-09T23:24:57.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.568+0000 7f0bb49d9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f8d1b940 con 0x5624f8dd3e40 2023-09-09T23:24:57.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.568+0000 7f0ba1d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624f8dca030 0x5624f8dd3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:57.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.568+0000 7f0ba259c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624f8dc9710 0x5624f8dc9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:57.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.568+0000 7f0ba1d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624f8dca030 0x5624f8dd3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58346/0 (socket says 172.21.15.82:58346) 2023-09-09T23:24:57.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.568+0000 7f0ba1d9b700 1 -- 172.21.15.82:0/2366240355 learned_addr learned my addr 172.21.15.82:0/2366240355 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:24:57.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.568+0000 7f0ba2d9d700 1 --2- 172.21.15.82:0/2366240355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f8dd3e40 0x5624f8dd6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:57.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.568+0000 7f0ba1d9b700 1 -- 172.21.15.82:0/2366240355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f8dd3e40 msgr2=0x5624f8dd6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:24:57.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.568+0000 7f0ba1d9b700 1 --2- 172.21.15.82:0/2366240355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f8dd3e40 0x5624f8dd6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.568+0000 7f0ba1d9b700 1 -- 172.21.15.82:0/2366240355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624f8dc9710 msgr2=0x5624f8dc9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:24:57.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.568+0000 7f0ba1d9b700 1 --2- 172.21.15.82:0/2366240355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624f8dc9710 0x5624f8dc9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.568+0000 7f0ba1d9b700 1 -- 172.21.15.82:0/2366240355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624f8b47de0 con 0x5624f8dca030 2023-09-09T23:24:57.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.568+0000 7f0ba259c700 1 --2- 172.21.15.82:0/2366240355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624f8dc9710 0x5624f8dc9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:24:57.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.568+0000 7f0ba1d9b700 1 --2- 172.21.15.82:0/2366240355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624f8dca030 0x5624f8dd3900 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f0b98011d20 tx=0x7f0b98014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55e7e6cc8062055 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:24:57.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.568+0000 7f0b90ff9700 1 -- 172.21.15.82:0/2366240355 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9800eb20 con 0x5624f8dca030 2023-09-09T23:24:57.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.569+0000 7f0b90ff9700 1 -- 172.21.15.82:0/2366240355 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b98021070 con 0x5624f8dca030 2023-09-09T23:24:57.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.569+0000 7f0b90ff9700 1 -- 172.21.15.82:0/2366240355 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b98003c70 con 0x5624f8dca030 2023-09-09T23:24:57.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.569+0000 7f0bb49d9b80 1 -- 172.21.15.82:0/2366240355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624f8dca030 msgr2=0x5624f8dd3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:24:57.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.569+0000 7f0bb49d9b80 1 --2- 172.21.15.82:0/2366240355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624f8dca030 0x5624f8dd3900 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f0b98011d20 tx=0x7f0b98014000 comp rx=0 tx=0).stop 2023-09-09T23:24:57.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.569+0000 7f0bb49d9b80 1 -- 172.21.15.82:0/2366240355 shutdown_connections 2023-09-09T23:24:57.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.569+0000 7f0bb49d9b80 1 --2- 172.21.15.82:0/2366240355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624f8dc9710 0x5624f8dc9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.569+0000 7f0bb49d9b80 1 --2- 172.21.15.82:0/2366240355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f8dd3e40 0x5624f8dd6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.569+0000 7f0bb49d9b80 1 --2- 172.21.15.82:0/2366240355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624f8dca030 0x5624f8dd3900 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.569+0000 7f0bb49d9b80 1 -- 172.21.15.82:0/2366240355 >> 172.21.15.82:0/2366240355 conn(0x5624f8d3e090 msgr2=0x5624f8dc4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:24:57.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.569+0000 7f0bb49d9b80 1 -- 172.21.15.82:0/2366240355 shutdown_connections 2023-09-09T23:24:57.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.569+0000 7f0bb49d9b80 1 -- 172.21.15.82:0/2366240355 wait complete. 2023-09-09T23:24:57.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.570+0000 7f0bb49d9b80 1 -- start start 2023-09-09T23:24:57.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.570+0000 7f0bb49d9b80 1 Processor -- start 2023-09-09T23:24:57.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.570+0000 7f0bb49d9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624f8dc9710 0x5624f8dd0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:57.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.570+0000 7f0bb49d9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f8dca030 0x5624f8dd1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:57.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.570+0000 7f0bb49d9b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624f8dd3e40 0x5624f8dd1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:57.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.570+0000 7f0bb49d9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f8cb9b70 con 0x5624f8dc9710 2023-09-09T23:24:57.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.570+0000 7f0bb49d9b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f8cb9870 con 0x5624f8dd3e40 2023-09-09T23:24:57.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.570+0000 7f0bb49d9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f8b9d920 con 0x5624f8dca030 2023-09-09T23:24:57.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.570+0000 7f0bb49d9b80 10 client.0 _ll_register_callbacks cb 0x5624f8c03a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:24:57.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.570+0000 7f0ba259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624f8dc9710 0x5624f8dd0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:57.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.570+0000 7f0ba2d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624f8dd3e40 0x5624f8dd1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:57.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.570+0000 7f0ba259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624f8dc9710 0x5624f8dd0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58350/0 (socket says 172.21.15.82:58350) 2023-09-09T23:24:57.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.570+0000 7f0ba259c700 1 -- 172.21.15.82:0/2785356888 learned_addr learned my addr 172.21.15.82:0/2785356888 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:24:57.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.570+0000 7f0ba1d9b700 1 --2- 172.21.15.82:0/2785356888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f8dca030 0x5624f8dd1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:57.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.570+0000 7f0ba259c700 1 -- 172.21.15.82:0/2785356888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f8dca030 msgr2=0x5624f8dd1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:24:57.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.570+0000 7f0ba259c700 1 --2- 172.21.15.82:0/2785356888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f8dca030 0x5624f8dd1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.570+0000 7f0ba259c700 1 -- 172.21.15.82:0/2785356888 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624f8dd3e40 msgr2=0x5624f8dd1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:24:57.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.570+0000 7f0ba259c700 1 --2- 172.21.15.82:0/2785356888 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624f8dd3e40 0x5624f8dd1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.570+0000 7f0ba259c700 1 -- 172.21.15.82:0/2785356888 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba41a3270 con 0x5624f8dc9710 2023-09-09T23:24:57.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.571+0000 7f0ba259c700 1 --2- 172.21.15.82:0/2785356888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624f8dc9710 0x5624f8dd0c10 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009990 tx=0x7f0b9400b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ffa8ca5b482944 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:24:57.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.571+0000 7f0b89ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:24:57.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.571+0000 7f0b89ffb700 1 -- 172.21.15.82:0/2785356888 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b940a0040 con 0x5624f8dc9710 2023-09-09T23:24:57.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.571+0000 7f0b89ffb700 1 -- 172.21.15.82:0/2785356888 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f0b9409a5c0 con 0x5624f8dc9710 2023-09-09T23:24:57.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.571+0000 7f0bb49d9b80 1 -- 172.21.15.82:0/2785356888 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5624f8c734f0 con 0x5624f8dc9710 2023-09-09T23:24:57.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.571+0000 7f0b89ffb700 1 -- 172.21.15.82:0/2785356888 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b9409e3f0 con 0x5624f8dc9710 2023-09-09T23:24:57.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.571+0000 7f0bb49d9b80 10 client.16317 fetch_fsmap learned FSMap version 127 2023-09-09T23:24:57.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.571+0000 7f0bb49d9b80 1 -- 172.21.15.82:0/2785356888 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x7f0b98017070 con 0x5624f8dc9710 2023-09-09T23:24:57.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.571+0000 7f0b89ffb700 1 -- 172.21.15.82:0/2785356888 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b940b8020 con 0x5624f8dc9710 2023-09-09T23:24:57.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.571+0000 7f0b89ffb700 1 -- 172.21.15.82:0/2785356888 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 127) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0b94099b40 con 0x5624f8dc9710 2023-09-09T23:24:57.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.571+0000 7f0bb49d9b80 10 client.16317 fetch_fsmap finished waiting for FSMap version 127 2023-09-09T23:24:57.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.571+0000 7f0bb49d9b80 10 client.16317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:24:57.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.571+0000 7f0bb49d9b80 20 client.16317 populate_metadata read hostname 'smithi082' 2023-09-09T23:24:57.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.571+0000 7f0bb49d9b80 1 --2- 172.21.15.82:0/2785356888 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5624f8de3620 0x5624f8de5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:57.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.571+0000 7f0bb49d9b80 4 client.16317 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:24:57.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.571+0000 7f0bb49d9b80 1 -- 172.21.15.82:0/2785356888 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624f8d308a0 con 0x5624f8de3620 2023-09-09T23:24:57.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.571+0000 7f0ba1d9b700 1 --2- 172.21.15.82:0/2785356888 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5624f8de3620 0x5624f8de5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:57.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.572+0000 7f0b89ffb700 10 client.16317 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:24:57.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.572+0000 7f0ba1d9b700 1 --2- 172.21.15.82:0/2785356888 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5624f8de3620 0x5624f8de5a10 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1be919eed9a72a5e server_cookie=f2d1dc6a1477bfd9 in_seq=0 out_seq=0 2023-09-09T23:24:57.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.572+0000 7f0b89ffb700 1 -- 172.21.15.82:0/2785356888 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5624f8d308a0 con 0x5624f8de3620 2023-09-09T23:24:57.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.572+0000 7f0b89ffb700 10 client.16317 handle_command_reply: tid=0 2023-09-09T23:24:57.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.572+0000 7f0bb49d9b80 1 client.16317 shutdown 2023-09-09T23:24:57.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.572+0000 7f0bb49d9b80 1 -- 172.21.15.82:0/2785356888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624f8dc9710 msgr2=0x5624f8dd0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:24:57.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.572+0000 7f0bb49d9b80 1 --2- 172.21.15.82:0/2785356888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624f8dc9710 0x5624f8dd0c10 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009990 tx=0x7f0b9400b5b0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.572+0000 7f0bb49d9b80 1 -- 172.21.15.82:0/2785356888 shutdown_connections 2023-09-09T23:24:57.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.572+0000 7f0bb49d9b80 1 --2- 172.21.15.82:0/2785356888 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5624f8de3620 0x5624f8de5a10 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.572+0000 7f0bb49d9b80 1 --2- 172.21.15.82:0/2785356888 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624f8dd3e40 0x5624f8dd1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.572+0000 7f0b89ffb700 0 client.16317 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:24:57.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.572+0000 7f0bb49d9b80 1 --2- 172.21.15.82:0/2785356888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624f8dc9710 0x5624f8dd0c10 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.572+0000 7f0bb49d9b80 1 --2- 172.21.15.82:0/2785356888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f8dca030 0x5624f8dd1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.572+0000 7f0bb49d9b80 1 -- 172.21.15.82:0/2785356888 >> 172.21.15.82:0/2785356888 conn(0x5624f8d3e090 msgr2=0x5624f8dc47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:24:57.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.572+0000 7f0bb49d9b80 1 -- 172.21.15.82:0/2785356888 shutdown_connections 2023-09-09T23:24:57.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.573+0000 7f0bb49d9b80 1 -- 172.21.15.82:0/2785356888 wait complete. 2023-09-09T23:24:57.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.573+0000 7f0bb49d9b80 20 client.16317 trim_cache size 0 max 16384 2023-09-09T23:24:57.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.589+0000 7f0bb49d9b80 1 Processor -- start 2023-09-09T23:24:57.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.589+0000 7f0bb49d9b80 1 -- start start 2023-09-09T23:24:57.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.589+0000 7f0bb49d9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f8dc9710 0x5624f8ddaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:57.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.589+0000 7f0bb49d9b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624f8dca030 0x5624f8dcdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:57.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.589+0000 7f0bb49d9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624f8dd3e40 0x5624f8dce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:57.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.589+0000 7f0bb49d9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f8dca9f0 con 0x5624f8dd3e40 2023-09-09T23:24:57.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.589+0000 7f0bb49d9b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f8dd2180 con 0x5624f8dca030 2023-09-09T23:24:57.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.589+0000 7f0bb49d9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f8dcba80 con 0x5624f8dc9710 2023-09-09T23:24:57.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.589+0000 7f0ba2d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624f8dd3e40 0x5624f8dce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:57.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.589+0000 7f0ba2d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624f8dd3e40 0x5624f8dce3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58362/0 (socket says 172.21.15.82:58362) 2023-09-09T23:24:57.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.589+0000 7f0ba1d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624f8dca030 0x5624f8dcdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:57.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.589+0000 7f0ba259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f8dc9710 0x5624f8ddaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:57.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.589+0000 7f0ba2d9d700 1 -- 172.21.15.82:0/3164825110 learned_addr learned my addr 172.21.15.82:0/3164825110 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:24:57.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.590+0000 7f0ba2d9d700 1 -- 172.21.15.82:0/3164825110 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f8dc9710 msgr2=0x5624f8ddaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:24:57.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.590+0000 7f0ba2d9d700 1 --2- 172.21.15.82:0/3164825110 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f8dc9710 0x5624f8ddaa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.590+0000 7f0ba2d9d700 1 -- 172.21.15.82:0/3164825110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624f8dca030 msgr2=0x5624f8dcdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:24:57.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.590+0000 7f0ba2d9d700 1 --2- 172.21.15.82:0/3164825110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624f8dca030 0x5624f8dcdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.590+0000 7f0ba2d9d700 1 -- 172.21.15.82:0/3164825110 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624f8b30c60 con 0x5624f8dd3e40 2023-09-09T23:24:57.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.590+0000 7f0ba1d9b700 1 --2- 172.21.15.82:0/3164825110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624f8dca030 0x5624f8dcdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:24:57.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.590+0000 7f0ba2d9d700 1 --2- 172.21.15.82:0/3164825110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624f8dd3e40 0x5624f8dce3e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c00eba0 tx=0x7f0b9c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886e7924f7ef4b2f server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:24:57.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.590+0000 7f0b917fa700 1 -- 172.21.15.82:0/3164825110 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c017880 con 0x5624f8dd3e40 2023-09-09T23:24:57.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.590+0000 7f0b917fa700 1 -- 172.21.15.82:0/3164825110 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b9c00f040 con 0x5624f8dd3e40 2023-09-09T23:24:57.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.590+0000 7f0b917fa700 1 -- 172.21.15.82:0/3164825110 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c007e50 con 0x5624f8dd3e40 2023-09-09T23:24:57.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.590+0000 7f0bb49d9b80 1 -- 172.21.15.82:0/3164825110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624f8dd3e40 msgr2=0x5624f8dce3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:24:57.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.590+0000 7f0bb49d9b80 1 --2- 172.21.15.82:0/3164825110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624f8dd3e40 0x5624f8dce3e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c00eba0 tx=0x7f0b9c009920 comp rx=0 tx=0).stop 2023-09-09T23:24:57.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.591+0000 7f0bb49d9b80 1 -- 172.21.15.82:0/3164825110 shutdown_connections 2023-09-09T23:24:57.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.591+0000 7f0bb49d9b80 1 --2- 172.21.15.82:0/3164825110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624f8dd3e40 0x5624f8dce3e0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.591+0000 7f0bb49d9b80 1 --2- 172.21.15.82:0/3164825110 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f8dc9710 0x5624f8ddaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.591+0000 7f0bb49d9b80 1 --2- 172.21.15.82:0/3164825110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624f8dca030 0x5624f8dcdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.591+0000 7f0bb49d9b80 1 -- 172.21.15.82:0/3164825110 >> 172.21.15.82:0/3164825110 conn(0x5624f8d3e090 msgr2=0x5624f8dd1180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:24:57.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.591+0000 7f0bb49d9b80 1 -- 172.21.15.82:0/3164825110 shutdown_connections 2023-09-09T23:24:57.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.591+0000 7f0bb49d9b80 1 -- 172.21.15.82:0/3164825110 wait complete. 2023-09-09T23:24:57.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.591+0000 7f0bb49d9b80 1 -- start start 2023-09-09T23:24:57.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.591+0000 7f0bb49d9b80 1 Processor -- start 2023-09-09T23:24:57.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.591+0000 7f0bb49d9b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624f8dc9710 0x5624f8e31200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:57.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.591+0000 7f0bb49d9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624f8dca030 0x5624f8e31740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:57.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.591+0000 7f0bb49d9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f8dd3e40 0x5624f8e2b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:57.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.591+0000 7f0bb49d9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f8e2bb80 con 0x5624f8dca030 2023-09-09T23:24:57.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.591+0000 7f0bb49d9b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f8e2bcf0 con 0x5624f8dc9710 2023-09-09T23:24:57.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.591+0000 7f0bb49d9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f8e2beb0 con 0x5624f8dd3e40 2023-09-09T23:24:57.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.591+0000 7f0bb49d9b80 10 client.0 _ll_register_callbacks cb 0x5624f8c03a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:24:57.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.591+0000 7f0ba1d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624f8dca030 0x5624f8e31740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:57.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.591+0000 7f0ba1d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624f8dca030 0x5624f8e31740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58366/0 (socket says 172.21.15.82:58366) 2023-09-09T23:24:57.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.591+0000 7f0ba1d9b700 1 -- 172.21.15.82:0/3052155877 learned_addr learned my addr 172.21.15.82:0/3052155877 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:24:57.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.591+0000 7f0ba2d9d700 1 --2- 172.21.15.82:0/3052155877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f8dd3e40 0x5624f8e2b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:57.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.591+0000 7f0ba259c700 1 --2- 172.21.15.82:0/3052155877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624f8dc9710 0x5624f8e31200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:57.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.592+0000 7f0ba1d9b700 1 -- 172.21.15.82:0/3052155877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f8dd3e40 msgr2=0x5624f8e2b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:24:57.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.592+0000 7f0ba1d9b700 1 --2- 172.21.15.82:0/3052155877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f8dd3e40 0x5624f8e2b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.592+0000 7f0ba1d9b700 1 -- 172.21.15.82:0/3052155877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624f8dc9710 msgr2=0x5624f8e31200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:24:57.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.592+0000 7f0ba1d9b700 1 --2- 172.21.15.82:0/3052155877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624f8dc9710 0x5624f8e31200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.592+0000 7f0ba1d9b700 1 -- 172.21.15.82:0/3052155877 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624f8b47de0 con 0x5624f8dca030 2023-09-09T23:24:57.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.592+0000 7f0ba1d9b700 1 --2- 172.21.15.82:0/3052155877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624f8dca030 0x5624f8e31740 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0b98011d20 tx=0x7f0b98005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8010e66caace6fc5 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:24:57.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.592+0000 7f0b89ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:24:57.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.592+0000 7f0b89ffb700 1 -- 172.21.15.82:0/3052155877 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b98030040 con 0x5624f8dca030 2023-09-09T23:24:57.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.592+0000 7f0b89ffb700 1 -- 172.21.15.82:0/3052155877 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f0b98034330 con 0x5624f8dca030 2023-09-09T23:24:57.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.592+0000 7f0bb49d9b80 1 -- 172.21.15.82:0/3052155877 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b98021070 con 0x5624f8dca030 2023-09-09T23:24:57.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.592+0000 7f0b89ffb700 1 -- 172.21.15.82:0/3052155877 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b98034730 con 0x5624f8dca030 2023-09-09T23:24:57.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.592+0000 7f0bb49d9b80 10 client.16323 fetch_fsmap learned FSMap version 127 2023-09-09T23:24:57.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.592+0000 7f0bb49d9b80 1 -- 172.21.15.82:0/3052155877 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x7f0b9c00e820 con 0x5624f8dca030 2023-09-09T23:24:57.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.592+0000 7f0b89ffb700 1 -- 172.21.15.82:0/3052155877 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b98042020 con 0x5624f8dca030 2023-09-09T23:24:57.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.592+0000 7f0b89ffb700 1 -- 172.21.15.82:0/3052155877 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 127) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0b9802ab60 con 0x5624f8dca030 2023-09-09T23:24:57.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.592+0000 7f0bb49d9b80 10 client.16323 fetch_fsmap finished waiting for FSMap version 127 2023-09-09T23:24:57.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.592+0000 7f0bb49d9b80 10 client.16323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:24:57.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.592+0000 7f0bb49d9b80 20 client.16323 populate_metadata read hostname 'smithi082' 2023-09-09T23:24:57.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.592+0000 7f0bb49d9b80 1 --2- 172.21.15.82:0/3052155877 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5624f8e39090 0x5624f8e3b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:24:57.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.592+0000 7f0bb49d9b80 4 client.16323 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:24:57.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.592+0000 7f0bb49d9b80 1 -- 172.21.15.82:0/3052155877 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5624f8d14920 con 0x5624f8e39090 2023-09-09T23:24:57.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.593+0000 7f0ba259c700 1 --2- 172.21.15.82:0/3052155877 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5624f8e39090 0x5624f8e3b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:24:57.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.593+0000 7f0b89ffb700 10 client.16323 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:24:57.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.593+0000 7f0ba259c700 1 --2- 172.21.15.82:0/3052155877 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5624f8e39090 0x5624f8e3b4b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3349f83f2d6ab71e server_cookie=2f2d19c6373a78ec in_seq=0 out_seq=0 2023-09-09T23:24:57.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.593+0000 7f0b89ffb700 1 -- 172.21.15.82:0/3052155877 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5624f8d14920 con 0x5624f8e39090 2023-09-09T23:24:57.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.593+0000 7f0b89ffb700 10 client.16323 handle_command_reply: tid=0 2023-09-09T23:24:57.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.593+0000 7f0bb49d9b80 1 client.16323 shutdown 2023-09-09T23:24:57.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.593+0000 7f0bb49d9b80 1 -- 172.21.15.82:0/3052155877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624f8dca030 msgr2=0x5624f8e31740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:24:57.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.593+0000 7f0bb49d9b80 1 --2- 172.21.15.82:0/3052155877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624f8dca030 0x5624f8e31740 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0b98011d20 tx=0x7f0b98005c40 comp rx=0 tx=0).stop 2023-09-09T23:24:57.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.593+0000 7f0bb49d9b80 1 -- 172.21.15.82:0/3052155877 shutdown_connections 2023-09-09T23:24:57.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.593+0000 7f0bb49d9b80 1 --2- 172.21.15.82:0/3052155877 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5624f8e39090 0x5624f8e3b4b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.593+0000 7f0bb49d9b80 1 --2- 172.21.15.82:0/3052155877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5624f8dc9710 0x5624f8e31200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.593+0000 7f0b89ffb700 0 client.16323 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:24:57.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.593+0000 7f0bb49d9b80 1 --2- 172.21.15.82:0/3052155877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624f8dd3e40 0x5624f8e2b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.593+0000 7f0bb49d9b80 1 --2- 172.21.15.82:0/3052155877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624f8dca030 0x5624f8e31740 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.593+0000 7f0bb49d9b80 1 -- 172.21.15.82:0/3052155877 >> 172.21.15.82:0/3052155877 conn(0x5624f8d3e090 msgr2=0x5624f8dc8700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:24:57.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.593+0000 7f0bb49d9b80 1 -- 172.21.15.82:0/3052155877 shutdown_connections 2023-09-09T23:24:57.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.593+0000 7f0bb49d9b80 1 -- 172.21.15.82:0/3052155877 wait complete. 2023-09-09T23:24:57.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.593+0000 7f0bb49d9b80 20 client.16323 trim_cache size 0 max 16384 2023-09-09T23:24:57.616 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:24:57.616 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:24:57.616 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:24:57.616 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:24:57.616 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:24:57.617 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:24:57.617 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:24:57.617 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:24:57.617 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:24:57.617 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:24:57.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.594+0000 7f0ba8813700 1 -- 172.21.15.82:0/2686178946 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f0b8c0a0830 msgr2=0x7f0b8c0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:24:57.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.594+0000 7f0ba8813700 1 --2- 172.21.15.82:0/2686178946 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f0b8c0a0830 0x7f0b8c0a2cf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4069000 tx=0x7f0b98009260 comp rx=0 tx=0).stop 2023-09-09T23:24:57.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.594+0000 7f0ba8813700 1 -- 172.21.15.82:0/2686178946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ba40ff4a0 msgr2=0x7f0ba4197450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:24:57.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.594+0000 7f0ba8813700 1 --2- 172.21.15.82:0/2686178946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ba40ff4a0 0x7f0ba4197450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400b580 tx=0x7f0b9400bfd0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.594+0000 7f0ba8813700 1 -- 172.21.15.82:0/2686178946 shutdown_connections 2023-09-09T23:24:57.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.594+0000 7f0ba8813700 1 --2- 172.21.15.82:0/2686178946 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f0b8c0a0830 0x7f0b8c0a2cf0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.594+0000 7f0ba8813700 1 --2- 172.21.15.82:0/2686178946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ba40ff4a0 0x7f0ba4197450 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.594+0000 7f0ba8813700 1 --2- 172.21.15.82:0/2686178946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ba4105560 0x7f0ba41a24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.594+0000 7f0ba8813700 1 --2- 172.21.15.82:0/2686178946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ba40ffe60 0x7f0ba4197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:24:57.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.594+0000 7f0ba8813700 1 -- 172.21.15.82:0/2686178946 >> 172.21.15.82:0/2686178946 conn(0x7f0ba40faf50 msgr2=0x7f0ba40fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:24:57.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.594+0000 7f0ba8813700 1 -- 172.21.15.82:0/2686178946 shutdown_connections 2023-09-09T23:24:57.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:24:57.594+0000 7f0ba8813700 1 -- 172.21.15.82:0/2686178946 wait complete. 2023-09-09T23:24:57.621 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:24:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:57 smithi082 ceph-mon[98694]: pgmap v344: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 12 op/s 2023-09-09T23:24:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 328.401554 secs 2023-09-09T23:24:57.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:57 smithi170 ceph-mon[102832]: pgmap v344: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 12 op/s 2023-09-09T23:24:57.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 328.401554 secs 2023-09-09T23:24:58.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:57 smithi098 ceph-mon[103124]: pgmap v344: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 12 op/s 2023-09-09T23:24:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 328.401554 secs 2023-09-09T23:24:58.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 330.022789 secs 2023-09-09T23:24:58.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 330.022789 secs 2023-09-09T23:24:59.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 330.022789 secs 2023-09-09T23:24:59.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:59 smithi170 ceph-mon[102832]: pgmap v345: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 8 op/s 2023-09-09T23:24:59.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:24:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 330.751911 secs 2023-09-09T23:24:59.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:59 smithi082 ceph-mon[98694]: pgmap v345: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 8 op/s 2023-09-09T23:24:59.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:24:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 330.751911 secs 2023-09-09T23:25:00.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:59 smithi098 ceph-mon[103124]: pgmap v345: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 8 op/s 2023-09-09T23:25:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:24:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 330.751911 secs 2023-09-09T23:25:00.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:00 smithi170 ceph-mon[102832]: pgmap v346: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 16 KiB/s wr, 13 op/s 2023-09-09T23:25:00.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:00 smithi082 ceph-mon[98694]: pgmap v346: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 16 KiB/s wr, 13 op/s 2023-09-09T23:25:01.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:00 smithi098 ceph-mon[103124]: pgmap v346: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 16 KiB/s wr, 13 op/s 2023-09-09T23:25:02.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:01 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 333.401634 secs 2023-09-09T23:25:02.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:01 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 333.401634 secs 2023-09-09T23:25:02.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:01 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 333.401634 secs 2023-09-09T23:25:03.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:02 smithi170 ceph-mon[102832]: pgmap v347: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 38 KiB/s wr, 22 op/s 2023-09-09T23:25:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:02 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 335.022848 secs 2023-09-09T23:25:03.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:02 smithi082 ceph-mon[98694]: pgmap v347: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 38 KiB/s wr, 22 op/s 2023-09-09T23:25:03.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:02 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 335.022848 secs 2023-09-09T23:25:03.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:02 smithi098 ceph-mon[103124]: pgmap v347: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 38 KiB/s wr, 22 op/s 2023-09-09T23:25:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:02 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 335.022848 secs 2023-09-09T23:25:04.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 335.751973 secs 2023-09-09T23:25:04.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 335.751973 secs 2023-09-09T23:25:04.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 335.751973 secs 2023-09-09T23:25:05.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:05 smithi098 ceph-mon[103124]: pgmap v348: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 38 KiB/s wr, 21 op/s 2023-09-09T23:25:05.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:05 smithi170 ceph-mon[102832]: pgmap v348: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 38 KiB/s wr, 21 op/s 2023-09-09T23:25:05.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:05 smithi082 ceph-mon[98694]: pgmap v348: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 38 KiB/s wr, 21 op/s 2023-09-09T23:25:07.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:07 smithi098 ceph-mon[103124]: pgmap v349: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 50 KiB/s wr, 27 op/s 2023-09-09T23:25:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 338.401702 secs 2023-09-09T23:25:07.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:07 smithi170 ceph-mon[102832]: pgmap v349: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 50 KiB/s wr, 27 op/s 2023-09-09T23:25:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 338.401702 secs 2023-09-09T23:25:07.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:07 smithi082 ceph-mon[98694]: pgmap v349: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 50 KiB/s wr, 27 op/s 2023-09-09T23:25:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 338.401702 secs 2023-09-09T23:25:08.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:08 smithi098 ceph-mon[103124]: osdmap e95: 12 total, 12 up, 12 in 2023-09-09T23:25:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 340.022906 secs 2023-09-09T23:25:08.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:08 smithi170 ceph-mon[102832]: osdmap e95: 12 total, 12 up, 12 in 2023-09-09T23:25:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 340.022906 secs 2023-09-09T23:25:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:08 smithi082 ceph-mon[98694]: osdmap e95: 12 total, 12 up, 12 in 2023-09-09T23:25:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 340.022906 secs 2023-09-09T23:25:09.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:09 smithi098 ceph-mon[103124]: pgmap v351: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 KiB/s wr, 28 op/s 2023-09-09T23:25:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 340.752038 secs 2023-09-09T23:25:09.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:09 smithi170 ceph-mon[102832]: pgmap v351: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 KiB/s wr, 28 op/s 2023-09-09T23:25:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 340.752038 secs 2023-09-09T23:25:09.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:09 smithi082 ceph-mon[98694]: pgmap v351: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 KiB/s wr, 28 op/s 2023-09-09T23:25:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 340.752038 secs 2023-09-09T23:25:11.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:11 smithi098 ceph-mon[103124]: pgmap v352: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 40 KiB/s wr, 26 op/s 2023-09-09T23:25:11.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:11 smithi170 ceph-mon[102832]: pgmap v352: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 40 KiB/s wr, 26 op/s 2023-09-09T23:25:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:11 smithi082 ceph-mon[98694]: pgmap v352: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 40 KiB/s wr, 26 op/s 2023-09-09T23:25:12.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 343.401831 secs 2023-09-09T23:25:12.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 343.401831 secs 2023-09-09T23:25:12.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 343.401831 secs 2023-09-09T23:25:13.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:13 smithi098 ceph-mon[103124]: pgmap v353: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-09-09T23:25:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 345.022967 secs 2023-09-09T23:25:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:13 smithi098 ceph-mon[103124]: osdmap e96: 12 total, 12 up, 12 in 2023-09-09T23:25:13.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:13 smithi170 ceph-mon[102832]: pgmap v353: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-09-09T23:25:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 345.022967 secs 2023-09-09T23:25:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:13 smithi170 ceph-mon[102832]: osdmap e96: 12 total, 12 up, 12 in 2023-09-09T23:25:13.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:13 smithi082 ceph-mon[98694]: pgmap v353: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-09-09T23:25:13.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 345.022967 secs 2023-09-09T23:25:13.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:13 smithi082 ceph-mon[98694]: osdmap e96: 12 total, 12 up, 12 in 2023-09-09T23:25:14.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 345.752097 secs 2023-09-09T23:25:14.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 345.752097 secs 2023-09-09T23:25:15.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 345.752097 secs 2023-09-09T23:25:15.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:15 smithi170 ceph-mon[102832]: pgmap v355: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:25:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:15 smithi082 ceph-mon[98694]: pgmap v355: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:25:16.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:15 smithi098 ceph-mon[103124]: pgmap v355: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:25:17.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:17 smithi170 ceph-mon[102832]: pgmap v356: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2023-09-09T23:25:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 348.401891 secs 2023-09-09T23:25:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:25:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:17 smithi082 ceph-mon[98694]: pgmap v356: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2023-09-09T23:25:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 348.401891 secs 2023-09-09T23:25:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:25:18.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:17 smithi098 ceph-mon[103124]: pgmap v356: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2023-09-09T23:25:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 348.401891 secs 2023-09-09T23:25:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:25:18.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:25:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:25:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:25:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 350.023024 secs 2023-09-09T23:25:18.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:25:18.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:25:18.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:25:18.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 350.023024 secs 2023-09-09T23:25:19.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:25:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:25:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:25:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 350.023024 secs 2023-09-09T23:25:19.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:19 smithi170 ceph-mon[102832]: pgmap v357: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.5 KiB/s wr, 11 op/s 2023-09-09T23:25:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 350.752159 secs 2023-09-09T23:25:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:19 smithi082 ceph-mon[98694]: pgmap v357: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.5 KiB/s wr, 11 op/s 2023-09-09T23:25:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 350.752159 secs 2023-09-09T23:25:20.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:19 smithi098 ceph-mon[103124]: pgmap v357: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.5 KiB/s wr, 11 op/s 2023-09-09T23:25:20.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 350.752159 secs 2023-09-09T23:25:21.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:21 smithi170 ceph-mon[102832]: pgmap v358: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-09-09T23:25:21.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:21 smithi082 ceph-mon[98694]: pgmap v358: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-09-09T23:25:22.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:21 smithi098 ceph-mon[103124]: pgmap v358: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-09-09T23:25:22.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 353.401949 secs 2023-09-09T23:25:22.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 353.401949 secs 2023-09-09T23:25:23.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 353.401949 secs 2023-09-09T23:25:23.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:23 smithi170 ceph-mon[102832]: pgmap v359: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-09-09T23:25:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 355.023109 secs 2023-09-09T23:25:23.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:23 smithi082 ceph-mon[98694]: pgmap v359: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-09-09T23:25:23.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 355.023109 secs 2023-09-09T23:25:24.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:23 smithi098 ceph-mon[103124]: pgmap v359: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-09-09T23:25:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 355.023109 secs 2023-09-09T23:25:24.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 355.752224 secs 2023-09-09T23:25:24.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 355.752224 secs 2023-09-09T23:25:25.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 355.752224 secs 2023-09-09T23:25:25.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:25 smithi170 ceph-mon[102832]: pgmap v360: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 13 op/s 2023-09-09T23:25:25.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:25 smithi082 ceph-mon[98694]: pgmap v360: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 13 op/s 2023-09-09T23:25:26.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:25 smithi098 ceph-mon[103124]: pgmap v360: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 13 op/s 2023-09-09T23:25:27.623 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:25:27.762 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.761+0000 7f17bf032700 1 -- 172.21.15.82:0/1706245856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17b8105570 msgr2=0x7f17b8105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:25:27.762 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.761+0000 7f17bf032700 1 --2- 172.21.15.82:0/1706245856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17b8105570 0x7f17b8105950 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f17b000b0d0 tx=0x7f17b000b3e0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.762 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.761+0000 7f17bf032700 1 -- 172.21.15.82:0/1706245856 shutdown_connections 2023-09-09T23:25:27.763 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.761+0000 7f17bf032700 1 --2- 172.21.15.82:0/1706245856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17b80ff3c0 0x7f17b810d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.763 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.761+0000 7f17bf032700 1 --2- 172.21.15.82:0/1706245856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17b8105570 0x7f17b8105950 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.763 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.761+0000 7f17bf032700 1 --2- 172.21.15.82:0/1706245856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17b80fea00 0x7f17b80fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.763 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.761+0000 7f17bf032700 1 -- 172.21.15.82:0/1706245856 >> 172.21.15.82:0/1706245856 conn(0x7f17b8076a70 msgr2=0x7f17b80fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:25:27.763 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.761+0000 7f17bf032700 1 -- 172.21.15.82:0/1706245856 shutdown_connections 2023-09-09T23:25:27.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.761+0000 7f17bf032700 1 -- 172.21.15.82:0/1706245856 wait complete. 2023-09-09T23:25:27.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.762+0000 7f17bf032700 1 Processor -- start 2023-09-09T23:25:27.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.762+0000 7f17bf032700 1 -- start start 2023-09-09T23:25:27.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.762+0000 7f17bf032700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17b80fea00 0x7f17b819c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:27.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.762+0000 7f17bf032700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17b80ff3c0 0x7f17b819caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:27.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.762+0000 7f17bf032700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17b8105570 0x7f17b8196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:27.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.762+0000 7f17bf032700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b8196c90 con 0x7f17b80fea00 2023-09-09T23:25:27.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.762+0000 7f17bf032700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b8196e00 con 0x7f17b8105570 2023-09-09T23:25:27.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.762+0000 7f17bf032700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b8196fc0 con 0x7f17b80ff3c0 2023-09-09T23:25:27.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.762+0000 7f17bcdce700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17b80fea00 0x7f17b819c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:27.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.762+0000 7f17bcdce700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17b80fea00 0x7f17b819c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58376/0 (socket says 172.21.15.82:58376) 2023-09-09T23:25:27.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.762+0000 7f17bcdce700 1 -- 172.21.15.82:0/2027119516 learned_addr learned my addr 172.21.15.82:0/2027119516 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:25:27.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.762+0000 7f17b7fff700 1 --2- 172.21.15.82:0/2027119516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17b80ff3c0 0x7f17b819caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:27.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.762+0000 7f17bd5cf700 1 --2- 172.21.15.82:0/2027119516 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17b8105570 0x7f17b8196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:27.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.763+0000 7f17bcdce700 1 -- 172.21.15.82:0/2027119516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17b80ff3c0 msgr2=0x7f17b819caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:25:27.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.763+0000 7f17bcdce700 1 --2- 172.21.15.82:0/2027119516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17b80ff3c0 0x7f17b819caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.763+0000 7f17bcdce700 1 -- 172.21.15.82:0/2027119516 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17b8105570 msgr2=0x7f17b8196620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:25:27.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.763+0000 7f17bcdce700 1 --2- 172.21.15.82:0/2027119516 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17b8105570 0x7f17b8196620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.763+0000 7f17bcdce700 1 -- 172.21.15.82:0/2027119516 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a8009520 con 0x7f17b80fea00 2023-09-09T23:25:27.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.763+0000 7f17bcdce700 1 --2- 172.21.15.82:0/2027119516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17b80fea00 0x7f17b819c5b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f17b0000f80 tx=0x7f17b0007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:25:27.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.763+0000 7f17b5ffb700 1 -- 172.21.15.82:0/2027119516 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b000e040 con 0x7f17b80fea00 2023-09-09T23:25:27.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.763+0000 7f17b5ffb700 1 -- 172.21.15.82:0/2027119516 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17b0005030 con 0x7f17b80fea00 2023-09-09T23:25:27.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.763+0000 7f17bf032700 1 -- 172.21.15.82:0/2027119516 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b0009d00 con 0x7f17b80fea00 2023-09-09T23:25:27.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.763+0000 7f17b5ffb700 1 -- 172.21.15.82:0/2027119516 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b001d6d0 con 0x7f17b80fea00 2023-09-09T23:25:27.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.763+0000 7f17bf032700 1 -- 172.21.15.82:0/2027119516 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b81a33c0 con 0x7f17b80fea00 2023-09-09T23:25:27.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.765+0000 7f17b5ffb700 1 -- 172.21.15.82:0/2027119516 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f17b00054a0 con 0x7f17b80fea00 2023-09-09T23:25:27.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.766+0000 7f17b5ffb700 1 --2- 172.21.15.82:0/2027119516 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f17a40a07b0 0x7f17a40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:27.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.766+0000 7f17b5ffb700 1 -- 172.21.15.82:0/2027119516 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f17b00cce90 con 0x7f17b80fea00 2023-09-09T23:25:27.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.766+0000 7f17b7fff700 1 --2- 172.21.15.82:0/2027119516 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f17a40a07b0 0x7f17a40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:27.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.766+0000 7f17b7fff700 1 --2- 172.21.15.82:0/2027119516 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f17a40a07b0 0x7f17a40a2c70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f17b81072f0 tx=0x7f17a0009260 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:25:27.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.767+0000 7f17cb1f8b80 1 Processor -- start 2023-09-09T23:25:27.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.767+0000 7f17cb1f8b80 1 -- start start 2023-09-09T23:25:27.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.767+0000 7f17cb1f8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c918206710 0x55c918206af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:27.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.767+0000 7f17cb1f8b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c918207030 0x55c918210900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:27.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.767+0000 7f17cb1f8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c918210e40 0x55c918213230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:27.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.767+0000 7f17cb1f8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91815f730 con 0x55c918206710 2023-09-09T23:25:27.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.767+0000 7f17cb1f8b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91815d050 con 0x55c918207030 2023-09-09T23:25:27.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.767+0000 7f17cb1f8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c918158940 con 0x55c918210e40 2023-09-09T23:25:27.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.767+0000 7f17bcdce700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c918206710 0x55c918206af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:27.777 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.767+0000 7f17b7fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c918207030 0x55c918210900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:27.777 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.767+0000 7f17bcdce700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c918206710 0x55c918206af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58384/0 (socket says 172.21.15.82:58384) 2023-09-09T23:25:27.777 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.767+0000 7f17bd5cf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c918210e40 0x55c918213230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:27.777 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.767+0000 7f17bcdce700 1 -- 172.21.15.82:0/742007146 learned_addr learned my addr 172.21.15.82:0/742007146 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:25:27.777 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.768+0000 7f17b7fff700 1 -- 172.21.15.82:0/742007146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c918210e40 msgr2=0x55c918213230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:25:27.778 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.768+0000 7f17b7fff700 1 --2- 172.21.15.82:0/742007146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c918210e40 0x55c918213230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.778 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.768+0000 7f17b7fff700 1 -- 172.21.15.82:0/742007146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c918206710 msgr2=0x55c918206af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:25:27.778 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.768+0000 7f17b7fff700 1 --2- 172.21.15.82:0/742007146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c918206710 0x55c918206af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.778 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.768+0000 7f17b7fff700 1 -- 172.21.15.82:0/742007146 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c917f84de0 con 0x55c918207030 2023-09-09T23:25:27.778 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.768+0000 7f17bd5cf700 1 --2- 172.21.15.82:0/742007146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c918210e40 0x55c918213230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:25:27.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.768+0000 7f17bcdce700 1 --2- 172.21.15.82:0/742007146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c918206710 0x55c918206af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:25:27.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.768+0000 7f17b7fff700 1 --2- 172.21.15.82:0/742007146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c918207030 0x55c918210900 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f17a0011c90 tx=0x7f17a0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0b1ab549e0b1a12 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:25:27.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.768+0000 7f17aeffd700 1 -- 172.21.15.82:0/742007146 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a0004230 con 0x55c918207030 2023-09-09T23:25:27.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.768+0000 7f17aeffd700 1 -- 172.21.15.82:0/742007146 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17a0021070 con 0x55c918207030 2023-09-09T23:25:27.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.768+0000 7f17aeffd700 1 -- 172.21.15.82:0/742007146 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a0003bf0 con 0x55c918207030 2023-09-09T23:25:27.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.769+0000 7f17cb1f8b80 1 -- 172.21.15.82:0/742007146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c918207030 msgr2=0x55c918210900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:25:27.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.769+0000 7f17cb1f8b80 1 --2- 172.21.15.82:0/742007146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c918207030 0x55c918210900 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f17a0011c90 tx=0x7f17a0014000 comp rx=0 tx=0).stop 2023-09-09T23:25:27.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.769+0000 7f17cb1f8b80 1 -- 172.21.15.82:0/742007146 shutdown_connections 2023-09-09T23:25:27.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.769+0000 7f17cb1f8b80 1 --2- 172.21.15.82:0/742007146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c918210e40 0x55c918213230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.769+0000 7f17cb1f8b80 1 --2- 172.21.15.82:0/742007146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c918206710 0x55c918206af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.769+0000 7f17cb1f8b80 1 --2- 172.21.15.82:0/742007146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c918207030 0x55c918210900 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.769+0000 7f17cb1f8b80 1 -- 172.21.15.82:0/742007146 >> 172.21.15.82:0/742007146 conn(0x55c91817b090 msgr2=0x55c918201bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:25:27.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.769+0000 7f17cb1f8b80 1 -- 172.21.15.82:0/742007146 shutdown_connections 2023-09-09T23:25:27.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.769+0000 7f17cb1f8b80 1 -- 172.21.15.82:0/742007146 wait complete. 2023-09-09T23:25:27.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.769+0000 7f17cb1f8b80 1 -- start start 2023-09-09T23:25:27.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.769+0000 7f17cb1f8b80 1 Processor -- start 2023-09-09T23:25:27.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.770+0000 7f17cb1f8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c918206710 0x55c91820db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:27.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.770+0000 7f17cb1f8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c918207030 0x55c91820e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:27.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.770+0000 7f17cb1f8b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c918210e40 0x55c91820e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:27.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.770+0000 7f17cb1f8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9180f6b70 con 0x55c918206710 2023-09-09T23:25:27.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.770+0000 7f17cb1f8b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9180f6870 con 0x55c918210e40 2023-09-09T23:25:27.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.770+0000 7f17cb1f8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c917fda920 con 0x55c918207030 2023-09-09T23:25:27.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.770+0000 7f17cb1f8b80 10 client.0 _ll_register_callbacks cb 0x55c918040a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:25:27.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.770+0000 7f17bcdce700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c918206710 0x55c91820db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:27.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.770+0000 7f17bcdce700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c918206710 0x55c91820db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58390/0 (socket says 172.21.15.82:58390) 2023-09-09T23:25:27.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.770+0000 7f17bcdce700 1 -- 172.21.15.82:0/194809261 learned_addr learned my addr 172.21.15.82:0/194809261 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:25:27.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.770+0000 7f17b7fff700 1 --2- 172.21.15.82:0/194809261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c918207030 0x55c91820e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:27.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.770+0000 7f17bd5cf700 1 --2- 172.21.15.82:0/194809261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c918210e40 0x55c91820e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:27.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.770+0000 7f17bcdce700 1 -- 172.21.15.82:0/194809261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c918207030 msgr2=0x55c91820e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:25:27.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.770+0000 7f17bcdce700 1 --2- 172.21.15.82:0/194809261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c918207030 0x55c91820e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.770+0000 7f17bcdce700 1 -- 172.21.15.82:0/194809261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c918210e40 msgr2=0x55c91820e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:25:27.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.770+0000 7f17bcdce700 1 --2- 172.21.15.82:0/194809261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c918210e40 0x55c91820e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.770+0000 7f17bcdce700 1 -- 172.21.15.82:0/194809261 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b81a33c0 con 0x55c918206710 2023-09-09T23:25:27.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.770+0000 7f17bcdce700 1 --2- 172.21.15.82:0/194809261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c918206710 0x55c91820db50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f17b0009fd0 tx=0x7f17b00966f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f178809176a36862 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:25:27.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.770+0000 7f1797fff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:25:27.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.770+0000 7f1797fff700 1 -- 172.21.15.82:0/194809261 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b009a810 con 0x55c918206710 2023-09-09T23:25:27.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.770+0000 7f1797fff700 1 -- 172.21.15.82:0/194809261 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f17b00a02c0 con 0x55c918206710 2023-09-09T23:25:27.786 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.770+0000 7f17cb1f8b80 1 -- 172.21.15.82:0/194809261 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c9180b04f0 con 0x55c918206710 2023-09-09T23:25:27.786 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.771+0000 7f1797fff700 1 -- 172.21.15.82:0/194809261 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17b0099ef0 con 0x55c918206710 2023-09-09T23:25:27.786 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.771+0000 7f17cb1f8b80 10 client.16335 fetch_fsmap learned FSMap version 127 2023-09-09T23:25:27.786 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.771+0000 7f17cb1f8b80 1 -- 172.21.15.82:0/194809261 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x7f17a8009a60 con 0x55c918206710 2023-09-09T23:25:27.786 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.771+0000 7f1797fff700 1 -- 172.21.15.82:0/194809261 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c9180b04f0 con 0x55c918206710 2023-09-09T23:25:27.786 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.771+0000 7f1797fff700 1 -- 172.21.15.82:0/194809261 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 127) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f17b00998a0 con 0x55c918206710 2023-09-09T23:25:27.787 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.771+0000 7f17cb1f8b80 10 client.16335 fetch_fsmap finished waiting for FSMap version 127 2023-09-09T23:25:27.787 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.771+0000 7f17cb1f8b80 10 client.16335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:25:27.787 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.771+0000 7f17cb1f8b80 20 client.16335 populate_metadata read hostname 'smithi082' 2023-09-09T23:25:27.787 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.771+0000 7f17cb1f8b80 1 --2- 172.21.15.82:0/194809261 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c9182205f0 0x55c918222a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:27.787 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.771+0000 7f17cb1f8b80 4 client.16335 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:25:27.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.771+0000 7f17cb1f8b80 1 -- 172.21.15.82:0/194809261 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c91816d8a0 con 0x55c9182205f0 2023-09-09T23:25:27.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.771+0000 7f17b7fff700 1 --2- 172.21.15.82:0/194809261 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c9182205f0 0x55c918222a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:27.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.771+0000 7f1797fff700 10 client.16335 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:25:27.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.771+0000 7f17b7fff700 1 --2- 172.21.15.82:0/194809261 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c9182205f0 0x55c918222a10 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f024414ff7fc4b6 server_cookie=bb73132e92172468 in_seq=0 out_seq=0 2023-09-09T23:25:27.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.772+0000 7f1797fff700 1 -- 172.21.15.82:0/194809261 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c91816d8a0 con 0x55c9182205f0 2023-09-09T23:25:27.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.772+0000 7f1797fff700 10 client.16335 handle_command_reply: tid=0 2023-09-09T23:25:27.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.772+0000 7f17cb1f8b80 1 client.16335 shutdown 2023-09-09T23:25:27.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.772+0000 7f17cb1f8b80 1 -- 172.21.15.82:0/194809261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c918206710 msgr2=0x55c91820db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:25:27.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.772+0000 7f17cb1f8b80 1 --2- 172.21.15.82:0/194809261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c918206710 0x55c91820db50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f17b0009fd0 tx=0x7f17b00966f0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.772+0000 7f17cb1f8b80 1 -- 172.21.15.82:0/194809261 shutdown_connections 2023-09-09T23:25:27.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.772+0000 7f17cb1f8b80 1 --2- 172.21.15.82:0/194809261 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c9182205f0 0x55c918222a10 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.772+0000 7f17cb1f8b80 1 --2- 172.21.15.82:0/194809261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c918210e40 0x55c91820e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.772+0000 7f1797fff700 0 client.16335 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:25:27.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.772+0000 7f17cb1f8b80 1 --2- 172.21.15.82:0/194809261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c918206710 0x55c91820db50 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.772+0000 7f17cb1f8b80 1 --2- 172.21.15.82:0/194809261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c918207030 0x55c91820e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.772+0000 7f17cb1f8b80 1 -- 172.21.15.82:0/194809261 >> 172.21.15.82:0/194809261 conn(0x55c91817b090 msgr2=0x55c9182017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:25:27.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.772+0000 7f17cb1f8b80 1 -- 172.21.15.82:0/194809261 shutdown_connections 2023-09-09T23:25:27.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.772+0000 7f17cb1f8b80 1 -- 172.21.15.82:0/194809261 wait complete. 2023-09-09T23:25:27.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.773+0000 7f17cb1f8b80 20 client.16335 trim_cache size 0 max 16384 2023-09-09T23:25:27.796 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.789+0000 7f17cb1f8b80 1 Processor -- start 2023-09-09T23:25:27.796 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.789+0000 7f17cb1f8b80 1 -- start start 2023-09-09T23:25:27.796 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.789+0000 7f17cb1f8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c918206710 0x55c91820e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.789+0000 7f17cb1f8b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c918207030 0x55c918205e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.789+0000 7f17cb1f8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c918210e40 0x55c91820ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.789+0000 7f17cb1f8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9182026e0 con 0x55c918210e40 2023-09-09T23:25:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.789+0000 7f17cb1f8b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9181fbd70 con 0x55c918207030 2023-09-09T23:25:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.789+0000 7f17cb1f8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9182086e0 con 0x55c918206710 2023-09-09T23:25:27.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.789+0000 7f17bd5cf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c918210e40 0x55c91820ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:27.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.789+0000 7f17bcdce700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c918206710 0x55c91820e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:27.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.789+0000 7f17b7fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c918207030 0x55c918205e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:27.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.789+0000 7f17bcdce700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c918206710 0x55c91820e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:56912/0 (socket says 172.21.15.82:56912) 2023-09-09T23:25:27.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.789+0000 7f17bd5cf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c918210e40 0x55c91820ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58402/0 (socket says 172.21.15.82:58402) 2023-09-09T23:25:27.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.789+0000 7f17bcdce700 1 -- 172.21.15.82:0/2169755425 learned_addr learned my addr 172.21.15.82:0/2169755425 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:25:27.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.790+0000 7f17bd5cf700 1 -- 172.21.15.82:0/2169755425 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c918206710 msgr2=0x55c91820e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:25:27.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.790+0000 7f17bd5cf700 1 --2- 172.21.15.82:0/2169755425 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c918206710 0x55c91820e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.790+0000 7f17bd5cf700 1 -- 172.21.15.82:0/2169755425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c918207030 msgr2=0x55c918205e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:25:27.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.790+0000 7f17bd5cf700 1 --2- 172.21.15.82:0/2169755425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c918207030 0x55c918205e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.790+0000 7f17bd5cf700 1 -- 172.21.15.82:0/2169755425 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a0017070 con 0x55c918210e40 2023-09-09T23:25:27.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.790+0000 7f17bd5cf700 1 --2- 172.21.15.82:0/2169755425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c918210e40 0x55c91820ae00 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f17a8012770 tx=0x7f17a8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a887864010e2488f server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:25:27.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.790+0000 7f17af7fe700 1 -- 172.21.15.82:0/2169755425 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a8017050 con 0x55c918210e40 2023-09-09T23:25:27.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.790+0000 7f17af7fe700 1 -- 172.21.15.82:0/2169755425 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17a8018040 con 0x55c918210e40 2023-09-09T23:25:27.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.790+0000 7f17af7fe700 1 -- 172.21.15.82:0/2169755425 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a8003de0 con 0x55c918210e40 2023-09-09T23:25:27.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.791+0000 7f17cb1f8b80 1 -- 172.21.15.82:0/2169755425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c918210e40 msgr2=0x55c91820ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:25:27.801 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.791+0000 7f17cb1f8b80 1 --2- 172.21.15.82:0/2169755425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c918210e40 0x55c91820ae00 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f17a8012770 tx=0x7f17a8009260 comp rx=0 tx=0).stop 2023-09-09T23:25:27.801 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.791+0000 7f17cb1f8b80 1 -- 172.21.15.82:0/2169755425 shutdown_connections 2023-09-09T23:25:27.801 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.791+0000 7f17cb1f8b80 1 --2- 172.21.15.82:0/2169755425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c918210e40 0x55c91820ae00 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.801 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.791+0000 7f17cb1f8b80 1 --2- 172.21.15.82:0/2169755425 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c918206710 0x55c91820e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.801 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.791+0000 7f17cb1f8b80 1 --2- 172.21.15.82:0/2169755425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c918207030 0x55c918205e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.791+0000 7f17cb1f8b80 1 -- 172.21.15.82:0/2169755425 >> 172.21.15.82:0/2169755425 conn(0x55c91817b090 msgr2=0x55c918205700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:25:27.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.791+0000 7f17cb1f8b80 1 -- 172.21.15.82:0/2169755425 shutdown_connections 2023-09-09T23:25:27.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.791+0000 7f17cb1f8b80 1 -- 172.21.15.82:0/2169755425 wait complete. 2023-09-09T23:25:27.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.791+0000 7f17cb1f8b80 1 -- start start 2023-09-09T23:25:27.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.791+0000 7f17cb1f8b80 1 Processor -- start 2023-09-09T23:25:27.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.791+0000 7f17cb1f8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c918206710 0x55c918216070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:27.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.791+0000 7f17cb1f8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c918207030 0x55c9182165b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:27.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.791+0000 7f17cb1f8b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c918210e40 0x55c918216af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:27.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.791+0000 7f17cb1f8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c918274430 con 0x55c918206710 2023-09-09T23:25:27.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.791+0000 7f17cb1f8b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c918274570 con 0x55c918210e40 2023-09-09T23:25:27.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.791+0000 7f17cb1f8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9182746b0 con 0x55c918207030 2023-09-09T23:25:27.804 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.791+0000 7f17cb1f8b80 10 client.0 _ll_register_callbacks cb 0x55c918040a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:25:27.804 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.791+0000 7f17bcdce700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c918206710 0x55c918216070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:27.804 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.792+0000 7f17bcdce700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c918206710 0x55c918216070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58404/0 (socket says 172.21.15.82:58404) 2023-09-09T23:25:27.804 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.792+0000 7f17bcdce700 1 -- 172.21.15.82:0/3655680742 learned_addr learned my addr 172.21.15.82:0/3655680742 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:25:27.804 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.792+0000 7f17b7fff700 1 --2- 172.21.15.82:0/3655680742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c918207030 0x55c9182165b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:27.805 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.792+0000 7f17bd5cf700 1 --2- 172.21.15.82:0/3655680742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c918210e40 0x55c918216af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:27.805 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.792+0000 7f17bcdce700 1 -- 172.21.15.82:0/3655680742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c918207030 msgr2=0x55c9182165b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:25:27.805 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.792+0000 7f17bcdce700 1 --2- 172.21.15.82:0/3655680742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c918207030 0x55c9182165b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.805 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.792+0000 7f17bcdce700 1 -- 172.21.15.82:0/3655680742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c918210e40 msgr2=0x55c918216af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:25:27.805 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.792+0000 7f17bcdce700 1 --2- 172.21.15.82:0/3655680742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c918210e40 0x55c918216af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.806 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.792+0000 7f17bcdce700 1 -- 172.21.15.82:0/3655680742 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a8012450 con 0x55c918206710 2023-09-09T23:25:27.806 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.792+0000 7f17b7fff700 1 --2- 172.21.15.82:0/3655680742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c918207030 0x55c9182165b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:25:27.806 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.792+0000 7f17bd5cf700 1 --2- 172.21.15.82:0/3655680742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c918210e40 0x55c918216af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:25:27.806 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.792+0000 7f17bcdce700 1 --2- 172.21.15.82:0/3655680742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c918206710 0x55c918216070 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f17b0007aa0 tx=0x7f17b009a840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c734f989b126731 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:25:27.806 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.792+0000 7f1797fff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:25:27.806 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.792+0000 7f1797fff700 1 -- 172.21.15.82:0/3655680742 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b00850b0 con 0x55c918206710 2023-09-09T23:25:27.807 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.792+0000 7f1797fff700 1 -- 172.21.15.82:0/3655680742 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f17b00a02c0 con 0x55c918206710 2023-09-09T23:25:27.807 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.792+0000 7f17cb1f8b80 1 -- 172.21.15.82:0/3655680742 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17a0021070 con 0x55c918206710 2023-09-09T23:25:27.807 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.792+0000 7f1797fff700 1 -- 172.21.15.82:0/3655680742 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17b0099ef0 con 0x55c918206710 2023-09-09T23:25:27.807 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.792+0000 7f17cb1f8b80 10 client.16341 fetch_fsmap learned FSMap version 127 2023-09-09T23:25:27.807 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.792+0000 7f17cb1f8b80 1 -- 172.21.15.82:0/3655680742 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x55c917f6dc60 con 0x55c918206710 2023-09-09T23:25:27.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.792+0000 7f1797fff700 1 -- 172.21.15.82:0/3655680742 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17b0064080 con 0x55c918206710 2023-09-09T23:25:27.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.792+0000 7f1797fff700 1 -- 172.21.15.82:0/3655680742 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 127) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f17b0083bb0 con 0x55c918206710 2023-09-09T23:25:27.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.793+0000 7f17cb1f8b80 10 client.16341 fetch_fsmap finished waiting for FSMap version 127 2023-09-09T23:25:27.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.793+0000 7f17cb1f8b80 10 client.16341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:25:27.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.793+0000 7f17cb1f8b80 20 client.16341 populate_metadata read hostname 'smithi082' 2023-09-09T23:25:27.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.793+0000 7f17cb1f8b80 1 --2- 172.21.15.82:0/3655680742 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c9182758e0 0x55c918277cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:27.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.793+0000 7f17cb1f8b80 4 client.16341 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:25:27.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.793+0000 7f17cb1f8b80 1 -- 172.21.15.82:0/3655680742 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c918151920 con 0x55c9182758e0 2023-09-09T23:25:27.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.793+0000 7f17b7fff700 1 --2- 172.21.15.82:0/3655680742 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c9182758e0 0x55c918277cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:27.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.793+0000 7f17b7fff700 1 --2- 172.21.15.82:0/3655680742 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c9182758e0 0x55c918277cd0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6998ac3b2645ae9d server_cookie=6c2dfdd84e329fa5 in_seq=0 out_seq=0 2023-09-09T23:25:27.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.793+0000 7f1797fff700 10 client.16341 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:25:27.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.793+0000 7f1797fff700 1 -- 172.21.15.82:0/3655680742 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55c918151920 con 0x55c9182758e0 2023-09-09T23:25:27.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.793+0000 7f1797fff700 10 client.16341 handle_command_reply: tid=0 2023-09-09T23:25:27.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.793+0000 7f17cb1f8b80 1 client.16341 shutdown 2023-09-09T23:25:27.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.793+0000 7f17cb1f8b80 1 -- 172.21.15.82:0/3655680742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c918206710 msgr2=0x55c918216070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:25:27.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.793+0000 7f17cb1f8b80 1 --2- 172.21.15.82:0/3655680742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c918206710 0x55c918216070 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f17b0007aa0 tx=0x7f17b009a840 comp rx=0 tx=0).stop 2023-09-09T23:25:27.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.793+0000 7f17cb1f8b80 1 -- 172.21.15.82:0/3655680742 shutdown_connections 2023-09-09T23:25:27.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.793+0000 7f17cb1f8b80 1 --2- 172.21.15.82:0/3655680742 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c9182758e0 0x55c918277cd0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.793+0000 7f17cb1f8b80 1 --2- 172.21.15.82:0/3655680742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c918210e40 0x55c918216af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.793+0000 7f1797fff700 0 client.16341 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:25:27.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.793+0000 7f17cb1f8b80 1 --2- 172.21.15.82:0/3655680742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c918206710 0x55c918216070 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.793+0000 7f17cb1f8b80 1 --2- 172.21.15.82:0/3655680742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c918207030 0x55c9182165b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.793+0000 7f17cb1f8b80 1 -- 172.21.15.82:0/3655680742 >> 172.21.15.82:0/3655680742 conn(0x55c91817b090 msgr2=0x55c918205f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:25:27.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.794+0000 7f17cb1f8b80 1 -- 172.21.15.82:0/3655680742 shutdown_connections 2023-09-09T23:25:27.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.794+0000 7f17cb1f8b80 1 -- 172.21.15.82:0/3655680742 wait complete. 2023-09-09T23:25:27.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.794+0000 7f17cb1f8b80 20 client.16341 trim_cache size 0 max 16384 2023-09-09T23:25:27.813 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:25:27.813 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:25:27.813 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:25:27.813 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:25:27.813 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:25:27.814 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:25:27.814 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:25:27.814 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:25:27.814 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:25:27.814 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:25:27.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.794+0000 7f17bf032700 1 -- 172.21.15.82:0/2027119516 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f17a40a07b0 msgr2=0x7f17a40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:25:27.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.794+0000 7f17bf032700 1 --2- 172.21.15.82:0/2027119516 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f17a40a07b0 0x7f17a40a2c70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f17b81072f0 tx=0x7f17a0009260 comp rx=0 tx=0).stop 2023-09-09T23:25:27.816 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.794+0000 7f17bf032700 1 -- 172.21.15.82:0/2027119516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17b80fea00 msgr2=0x7f17b819c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:25:27.816 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.794+0000 7f17bf032700 1 --2- 172.21.15.82:0/2027119516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17b80fea00 0x7f17b819c5b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f17b0000f80 tx=0x7f17b0007870 comp rx=0 tx=0).stop 2023-09-09T23:25:27.816 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.794+0000 7f17bf032700 1 -- 172.21.15.82:0/2027119516 shutdown_connections 2023-09-09T23:25:27.816 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.794+0000 7f17bf032700 1 --2- 172.21.15.82:0/2027119516 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f17a40a07b0 0x7f17a40a2c70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.816 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.794+0000 7f17bf032700 1 --2- 172.21.15.82:0/2027119516 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17b8105570 0x7f17b8196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.794+0000 7f17bf032700 1 --2- 172.21.15.82:0/2027119516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17b80fea00 0x7f17b819c5b0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.794+0000 7f17bf032700 1 --2- 172.21.15.82:0/2027119516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17b80ff3c0 0x7f17b819caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:27.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.794+0000 7f17bf032700 1 -- 172.21.15.82:0/2027119516 >> 172.21.15.82:0/2027119516 conn(0x7f17b8076a70 msgr2=0x7f17b8103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:25:27.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.794+0000 7f17bf032700 1 -- 172.21.15.82:0/2027119516 shutdown_connections 2023-09-09T23:25:27.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:27.794+0000 7f17bf032700 1 -- 172.21.15.82:0/2027119516 wait complete. 2023-09-09T23:25:27.818 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:27 smithi082 ceph-mon[98694]: pgmap v361: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 16 op/s 2023-09-09T23:25:27.818 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 358.402037 secs 2023-09-09T23:25:27.819 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:25:27.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:27 smithi170 ceph-mon[102832]: pgmap v361: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 16 op/s 2023-09-09T23:25:27.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 358.402037 secs 2023-09-09T23:25:28.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:27 smithi098 ceph-mon[103124]: pgmap v361: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 16 op/s 2023-09-09T23:25:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 358.402037 secs 2023-09-09T23:25:28.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:28 smithi170 ceph-mon[102832]: 3 slow requests, 3 included below; oldest blocked for > 360.023175 secs 2023-09-09T23:25:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:28 smithi170 ceph-mon[102832]: slow request 360.023174 seconds old, received at 2023-09-09T23:19:27.740042+0000: client_request(mds.4:47 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently acquired locks 2023-09-09T23:25:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:28 smithi170 ceph-mon[102832]: slow request 360.022794 seconds old, received at 2023-09-09T23:19:27.740422+0000: client_request(mds.4:48 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently requesting remote authpins 2023-09-09T23:25:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:28 smithi170 ceph-mon[102832]: slow request 360.020573 seconds old, received at 2023-09-09T23:19:27.742643+0000: client_request(client.24913:3869 unlink #0x10000000007/00000001000000010000000B 2023-09-09T23:19:27.741863+0000 caller_uid=26, caller_gid=26{26,}) currently requesting remote authpins 2023-09-09T23:25:28.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:28 smithi082 ceph-mon[98694]: 3 slow requests, 3 included below; oldest blocked for > 360.023175 secs 2023-09-09T23:25:28.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:28 smithi082 ceph-mon[98694]: slow request 360.023174 seconds old, received at 2023-09-09T23:19:27.740042+0000: client_request(mds.4:47 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently acquired locks 2023-09-09T23:25:28.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:28 smithi082 ceph-mon[98694]: slow request 360.022794 seconds old, received at 2023-09-09T23:19:27.740422+0000: client_request(mds.4:48 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently requesting remote authpins 2023-09-09T23:25:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:28 smithi082 ceph-mon[98694]: slow request 360.020573 seconds old, received at 2023-09-09T23:19:27.742643+0000: client_request(client.24913:3869 unlink #0x10000000007/00000001000000010000000B 2023-09-09T23:19:27.741863+0000 caller_uid=26, caller_gid=26{26,}) currently requesting remote authpins 2023-09-09T23:25:29.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:28 smithi098 ceph-mon[103124]: 3 slow requests, 3 included below; oldest blocked for > 360.023175 secs 2023-09-09T23:25:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:28 smithi098 ceph-mon[103124]: slow request 360.023174 seconds old, received at 2023-09-09T23:19:27.740042+0000: client_request(mds.4:47 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently acquired locks 2023-09-09T23:25:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:28 smithi098 ceph-mon[103124]: slow request 360.022794 seconds old, received at 2023-09-09T23:19:27.740422+0000: client_request(mds.4:48 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently requesting remote authpins 2023-09-09T23:25:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:28 smithi098 ceph-mon[103124]: slow request 360.020573 seconds old, received at 2023-09-09T23:19:27.742643+0000: client_request(client.24913:3869 unlink #0x10000000007/00000001000000010000000B 2023-09-09T23:19:27.741863+0000 caller_uid=26, caller_gid=26{26,}) currently requesting remote authpins 2023-09-09T23:25:29.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:29 smithi170 ceph-mon[102832]: pgmap v362: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2023-09-09T23:25:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:29 smithi170 ceph-mon[102832]: 3 slow requests, 3 included below; oldest blocked for > 360.752292 secs 2023-09-09T23:25:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:29 smithi170 ceph-mon[102832]: slow request 360.752291 seconds old, received at 2023-09-09T23:19:27.740800+0000: peer_request:mds.4:47 currently acquired locks 2023-09-09T23:25:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:29 smithi170 ceph-mon[102832]: slow request 360.752057 seconds old, received at 2023-09-09T23:19:27.741034+0000: peer_request(mds.4:48.0 authpin) currently dispatched 2023-09-09T23:25:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:29 smithi170 ceph-mon[102832]: slow request 360.750021 seconds old, received at 2023-09-09T23:19:27.743070+0000: peer_request(client.24913:3869.0 authpin) currently dispatched 2023-09-09T23:25:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:29 smithi082 ceph-mon[98694]: pgmap v362: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2023-09-09T23:25:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:29 smithi082 ceph-mon[98694]: 3 slow requests, 3 included below; oldest blocked for > 360.752292 secs 2023-09-09T23:25:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:29 smithi082 ceph-mon[98694]: slow request 360.752291 seconds old, received at 2023-09-09T23:19:27.740800+0000: peer_request:mds.4:47 currently acquired locks 2023-09-09T23:25:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:29 smithi082 ceph-mon[98694]: slow request 360.752057 seconds old, received at 2023-09-09T23:19:27.741034+0000: peer_request(mds.4:48.0 authpin) currently dispatched 2023-09-09T23:25:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:29 smithi082 ceph-mon[98694]: slow request 360.750021 seconds old, received at 2023-09-09T23:19:27.743070+0000: peer_request(client.24913:3869.0 authpin) currently dispatched 2023-09-09T23:25:30.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:29 smithi098 ceph-mon[103124]: pgmap v362: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2023-09-09T23:25:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:29 smithi098 ceph-mon[103124]: 3 slow requests, 3 included below; oldest blocked for > 360.752292 secs 2023-09-09T23:25:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:29 smithi098 ceph-mon[103124]: slow request 360.752291 seconds old, received at 2023-09-09T23:19:27.740800+0000: peer_request:mds.4:47 currently acquired locks 2023-09-09T23:25:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:29 smithi098 ceph-mon[103124]: slow request 360.752057 seconds old, received at 2023-09-09T23:19:27.741034+0000: peer_request(mds.4:48.0 authpin) currently dispatched 2023-09-09T23:25:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:29 smithi098 ceph-mon[103124]: slow request 360.750021 seconds old, received at 2023-09-09T23:19:27.743070+0000: peer_request(client.24913:3869.0 authpin) currently dispatched 2023-09-09T23:25:31.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:31 smithi170 ceph-mon[102832]: pgmap v363: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 14 op/s 2023-09-09T23:25:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:31 smithi082 ceph-mon[98694]: pgmap v363: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 14 op/s 2023-09-09T23:25:32.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:31 smithi098 ceph-mon[103124]: pgmap v363: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 14 op/s 2023-09-09T23:25:32.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:32 smithi170 ceph-mon[102832]: 1 slow requests, 1 included below; oldest blocked for > 363.402094 secs 2023-09-09T23:25:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:32 smithi170 ceph-mon[102832]: slow request 363.402094 seconds old, received at 2023-09-09T23:19:28.303507+0000: peer_request:mds.4:47 currently dispatched 2023-09-09T23:25:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:32 smithi082 ceph-mon[98694]: 1 slow requests, 1 included below; oldest blocked for > 363.402094 secs 2023-09-09T23:25:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:32 smithi082 ceph-mon[98694]: slow request 363.402094 seconds old, received at 2023-09-09T23:19:28.303507+0000: peer_request:mds.4:47 currently dispatched 2023-09-09T23:25:33.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:32 smithi098 ceph-mon[103124]: 1 slow requests, 1 included below; oldest blocked for > 363.402094 secs 2023-09-09T23:25:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:32 smithi098 ceph-mon[103124]: slow request 363.402094 seconds old, received at 2023-09-09T23:19:28.303507+0000: peer_request:mds.4:47 currently dispatched 2023-09-09T23:25:33.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:33 smithi170 ceph-mon[102832]: pgmap v364: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2023-09-09T23:25:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 365.023245 secs 2023-09-09T23:25:33.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:33 smithi082 ceph-mon[98694]: pgmap v364: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2023-09-09T23:25:33.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 365.023245 secs 2023-09-09T23:25:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:33 smithi098 ceph-mon[103124]: pgmap v364: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2023-09-09T23:25:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 365.023245 secs 2023-09-09T23:25:34.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 365.752351 secs 2023-09-09T23:25:34.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 365.752351 secs 2023-09-09T23:25:35.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 365.752351 secs 2023-09-09T23:25:35.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:35 smithi170 ceph-mon[102832]: pgmap v365: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2023-09-09T23:25:35.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:35 smithi082 ceph-mon[98694]: pgmap v365: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2023-09-09T23:25:36.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:35 smithi098 ceph-mon[103124]: pgmap v365: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2023-09-09T23:25:36.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:36 smithi170 ceph-mon[102832]: pgmap v366: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2023-09-09T23:25:36.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:36 smithi082 ceph-mon[98694]: pgmap v366: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2023-09-09T23:25:37.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:36 smithi098 ceph-mon[103124]: pgmap v366: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2023-09-09T23:25:37.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 368.402153 secs 2023-09-09T23:25:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 368.402153 secs 2023-09-09T23:25:38.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 368.402153 secs 2023-09-09T23:25:38.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 370.023305 secs 2023-09-09T23:25:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:38 smithi170 ceph-mon[102832]: pgmap v367: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2023-09-09T23:25:38.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 370.023305 secs 2023-09-09T23:25:38.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:38 smithi082 ceph-mon[98694]: pgmap v367: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2023-09-09T23:25:39.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 370.023305 secs 2023-09-09T23:25:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:38 smithi098 ceph-mon[103124]: pgmap v367: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2023-09-09T23:25:39.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 370.752415 secs 2023-09-09T23:25:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 370.752415 secs 2023-09-09T23:25:40.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 370.752415 secs 2023-09-09T23:25:40.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:40 smithi170 ceph-mon[102832]: pgmap v368: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2023-09-09T23:25:40.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:40 smithi082 ceph-mon[98694]: pgmap v368: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2023-09-09T23:25:41.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:40 smithi098 ceph-mon[103124]: pgmap v368: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2023-09-09T23:25:42.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:41 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 373.402216 secs 2023-09-09T23:25:42.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:41 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 373.402216 secs 2023-09-09T23:25:42.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:41 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 373.402216 secs 2023-09-09T23:25:43.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:42 smithi170 ceph-mon[102832]: pgmap v369: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:25:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:42 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 375.023361 secs 2023-09-09T23:25:43.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:42 smithi082 ceph-mon[98694]: pgmap v369: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:25:43.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:42 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 375.023361 secs 2023-09-09T23:25:43.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:42 smithi098 ceph-mon[103124]: pgmap v369: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:25:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:42 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 375.023361 secs 2023-09-09T23:25:44.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 375.752486 secs 2023-09-09T23:25:44.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 375.752486 secs 2023-09-09T23:25:44.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 375.752486 secs 2023-09-09T23:25:45.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:45 smithi098 ceph-mon[103124]: pgmap v370: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:25:45.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:45 smithi170 ceph-mon[102832]: pgmap v370: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:25:45.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:45 smithi082 ceph-mon[98694]: pgmap v370: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:25:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:47 smithi098 ceph-mon[103124]: pgmap v371: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:25:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 378.402278 secs 2023-09-09T23:25:47.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:47 smithi170 ceph-mon[102832]: pgmap v371: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:25:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 378.402278 secs 2023-09-09T23:25:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:47 smithi082 ceph-mon[98694]: pgmap v371: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:25:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 378.402278 secs 2023-09-09T23:25:48.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 380.023444 secs 2023-09-09T23:25:48.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 380.023444 secs 2023-09-09T23:25:48.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 380.023444 secs 2023-09-09T23:25:49.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:49 smithi098 ceph-mon[103124]: pgmap v372: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:25:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 380.752548 secs 2023-09-09T23:25:49.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:49 smithi170 ceph-mon[102832]: pgmap v372: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:25:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 380.752548 secs 2023-09-09T23:25:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:49 smithi082 ceph-mon[98694]: pgmap v372: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:25:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 380.752548 secs 2023-09-09T23:25:51.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:51 smithi098 ceph-mon[103124]: pgmap v373: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-09-09T23:25:51.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:51 smithi170 ceph-mon[102832]: pgmap v373: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-09-09T23:25:51.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:51 smithi082 ceph-mon[98694]: pgmap v373: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-09-09T23:25:52.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 383.402337 secs 2023-09-09T23:25:52.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 383.402337 secs 2023-09-09T23:25:52.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 383.402337 secs 2023-09-09T23:25:53.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:53 smithi170 ceph-mon[102832]: pgmap v374: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:25:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 385.023523 secs 2023-09-09T23:25:53.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:53 smithi082 ceph-mon[98694]: pgmap v374: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:25:53.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 385.023523 secs 2023-09-09T23:25:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:53 smithi098 ceph-mon[103124]: pgmap v374: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:25:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 385.023523 secs 2023-09-09T23:25:54.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 385.752611 secs 2023-09-09T23:25:54.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 385.752611 secs 2023-09-09T23:25:55.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 385.752611 secs 2023-09-09T23:25:55.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:55 smithi170 ceph-mon[102832]: pgmap v375: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:25:55.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:55 smithi082 ceph-mon[98694]: pgmap v375: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:25:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:55 smithi098 ceph-mon[103124]: pgmap v375: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:25:57.820 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:25:57.851 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:57 smithi082 ceph-mon[98694]: pgmap v376: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-09T23:25:57.851 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 388.402397 secs 2023-09-09T23:25:57.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:57 smithi170 ceph-mon[102832]: pgmap v376: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-09T23:25:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 388.402397 secs 2023-09-09T23:25:57.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.959+0000 7f1e11ec6700 1 -- 172.21.15.82:0/1936330848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e0c1004d0 msgr2=0x7f1e0c10c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:25:57.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.959+0000 7f1e11ec6700 1 --2- 172.21.15.82:0/1936330848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e0c1004d0 0x7f1e0c10c330 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1e0400b0d0 tx=0x7f1e0400b3e0 comp rx=0 tx=0).stop 2023-09-09T23:25:57.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.959+0000 7f1e11ec6700 1 -- 172.21.15.82:0/1936330848 shutdown_connections 2023-09-09T23:25:57.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.959+0000 7f1e11ec6700 1 --2- 172.21.15.82:0/1936330848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e0c1004d0 0x7f1e0c10c330 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:57.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.959+0000 7f1e11ec6700 1 --2- 172.21.15.82:0/1936330848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e0c105570 0x7f1e0c105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:57.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.959+0000 7f1e11ec6700 1 --2- 172.21.15.82:0/1936330848 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e0c0ffb10 0x7f1e0c0fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:57.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.959+0000 7f1e11ec6700 1 -- 172.21.15.82:0/1936330848 >> 172.21.15.82:0/1936330848 conn(0x7f1e0c076a70 msgr2=0x7f1e0c0fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:25:57.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.959+0000 7f1e11ec6700 1 -- 172.21.15.82:0/1936330848 shutdown_connections 2023-09-09T23:25:57.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.959+0000 7f1e11ec6700 1 -- 172.21.15.82:0/1936330848 wait complete. 2023-09-09T23:25:57.970 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.960+0000 7f1e11ec6700 1 Processor -- start 2023-09-09T23:25:57.970 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.960+0000 7f1e11ec6700 1 -- start start 2023-09-09T23:25:57.970 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.960+0000 7f1e11ec6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e0c0ffb10 0x7f1e0c195660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:57.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.960+0000 7f1e11ec6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e0c1004d0 0x7f1e0c195ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:57.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.960+0000 7f1e11ec6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e0c105570 0x7f1e0c19a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:57.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.960+0000 7f1e11ec6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c19a640 con 0x7f1e0c1004d0 2023-09-09T23:25:57.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.960+0000 7f1e11ec6700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c19a7b0 con 0x7f1e0c105570 2023-09-09T23:25:57.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.960+0000 7f1e11ec6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c19a970 con 0x7f1e0c0ffb10 2023-09-09T23:25:57.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.960+0000 7f1e03fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e0c1004d0 0x7f1e0c195ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:57.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.960+0000 7f1e0b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e0c0ffb10 0x7f1e0c195660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:57.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.960+0000 7f1e03fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e0c1004d0 0x7f1e0c195ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58420/0 (socket says 172.21.15.82:58420) 2023-09-09T23:25:57.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.960+0000 7f1e0b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e0c0ffb10 0x7f1e0c195660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:56932/0 (socket says 172.21.15.82:56932) 2023-09-09T23:25:57.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.960+0000 7f1e03fff700 1 -- 172.21.15.82:0/26641571 learned_addr learned my addr 172.21.15.82:0/26641571 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:25:57.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.960+0000 7f1e0bfff700 1 --2- 172.21.15.82:0/26641571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e0c105570 0x7f1e0c19a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:57.974 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.961+0000 7f1e03fff700 1 -- 172.21.15.82:0/26641571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e0c0ffb10 msgr2=0x7f1e0c195660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:25:57.974 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.961+0000 7f1e03fff700 1 --2- 172.21.15.82:0/26641571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e0c0ffb10 0x7f1e0c195660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:57.974 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.961+0000 7f1e03fff700 1 -- 172.21.15.82:0/26641571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e0c105570 msgr2=0x7f1e0c19a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:25:57.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.961+0000 7f1e03fff700 1 --2- 172.21.15.82:0/26641571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e0c105570 0x7f1e0c19a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:57.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.961+0000 7f1e03fff700 1 -- 172.21.15.82:0/26641571 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e04009d00 con 0x7f1e0c1004d0 2023-09-09T23:25:57.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.961+0000 7f1e0bfff700 1 --2- 172.21.15.82:0/26641571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e0c105570 0x7f1e0c19a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:25:57.976 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.961+0000 7f1e03fff700 1 --2- 172.21.15.82:0/26641571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e0c1004d0 0x7f1e0c195ba0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1df400e9a0 tx=0x7f1df400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:25:57.976 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.961+0000 7f1e097fa700 1 -- 172.21.15.82:0/26641571 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4009790 con 0x7f1e0c1004d0 2023-09-09T23:25:57.977 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.961+0000 7f1e11ec6700 1 -- 172.21.15.82:0/26641571 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0c19ac50 con 0x7f1e0c1004d0 2023-09-09T23:25:57.977 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.961+0000 7f1e097fa700 1 -- 172.21.15.82:0/26641571 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1df4010430 con 0x7f1e0c1004d0 2023-09-09T23:25:57.977 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.961+0000 7f1e097fa700 1 -- 172.21.15.82:0/26641571 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4018680 con 0x7f1e0c1004d0 2023-09-09T23:25:57.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.961+0000 7f1e11ec6700 1 -- 172.21.15.82:0/26641571 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0c1a2490 con 0x7f1e0c1004d0 2023-09-09T23:25:57.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.963+0000 7f1e097fa700 1 -- 172.21.15.82:0/26641571 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f1df40105a0 con 0x7f1e0c1004d0 2023-09-09T23:25:57.978 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.963+0000 7f1e097fa700 1 --2- 172.21.15.82:0/26641571 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f1df80a07b0 0x7f1df80a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:57.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.964+0000 7f1e0b7fe700 1 --2- 172.21.15.82:0/26641571 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f1df80a07b0 0x7f1df80a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:57.979 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.964+0000 7f1e097fa700 1 -- 172.21.15.82:0/26641571 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f1df4014070 con 0x7f1e0c1004d0 2023-09-09T23:25:57.980 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.964+0000 7f1e0b7fe700 1 --2- 172.21.15.82:0/26641571 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f1df80a07b0 0x7f1df80a2c70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc009c60 tx=0x7f1dfc009260 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:25:57.980 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.964+0000 7f1e1e08cb80 1 Processor -- start 2023-09-09T23:25:57.980 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.964+0000 7f1e1e08cb80 1 -- start start 2023-09-09T23:25:57.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.965+0000 7f1e1e08cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afe5e1710 0x563afe5e1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:57.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.965+0000 7f1e1e08cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afe5e2030 0x563afe5eb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:57.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.965+0000 7f1e1e08cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563afe5ebe40 0x563afe5ee230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:57.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.965+0000 7f1e1e08cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afe53a730 con 0x563afe5e2030 2023-09-09T23:25:57.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.965+0000 7f1e1e08cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afe538050 con 0x563afe5ebe40 2023-09-09T23:25:57.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.965+0000 7f1e1e08cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afe533940 con 0x563afe5e1710 2023-09-09T23:25:57.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.965+0000 7f1e03fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afe5e2030 0x563afe5eb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:57.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.965+0000 7f1e0b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afe5e1710 0x563afe5e1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:57.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.965+0000 7f1e0bfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563afe5ebe40 0x563afe5ee230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:57.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.965+0000 7f1e03fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afe5e2030 0x563afe5eb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58430/0 (socket says 172.21.15.82:58430) 2023-09-09T23:25:57.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.965+0000 7f1e0b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afe5e1710 0x563afe5e1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:56940/0 (socket says 172.21.15.82:56940) 2023-09-09T23:25:57.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.965+0000 7f1e03fff700 1 -- 172.21.15.82:0/654047744 learned_addr learned my addr 172.21.15.82:0/654047744 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:25:57.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.965+0000 7f1e03fff700 1 -- 172.21.15.82:0/654047744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afe5e1710 msgr2=0x563afe5e1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:25:57.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.965+0000 7f1e03fff700 1 --2- 172.21.15.82:0/654047744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afe5e1710 0x563afe5e1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:57.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.965+0000 7f1e03fff700 1 -- 172.21.15.82:0/654047744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563afe5ebe40 msgr2=0x563afe5ee230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:25:57.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.965+0000 7f1e03fff700 1 --2- 172.21.15.82:0/654047744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563afe5ebe40 0x563afe5ee230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:57.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.965+0000 7f1e03fff700 1 -- 172.21.15.82:0/654047744 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563afe35fde0 con 0x563afe5e2030 2023-09-09T23:25:57.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.965+0000 7f1e0b7fe700 1 --2- 172.21.15.82:0/654047744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afe5e1710 0x563afe5e1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:25:57.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.965+0000 7f1e0bfff700 1 --2- 172.21.15.82:0/654047744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563afe5ebe40 0x563afe5ee230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:25:57.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.965+0000 7f1e03fff700 1 --2- 172.21.15.82:0/654047744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afe5e2030 0x563afe5eb900 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1df40a1140 tx=0x7f1df4098f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674a38d0d688cff6 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:25:57.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.965+0000 7f1e01ffb700 1 -- 172.21.15.82:0/654047744 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df409d430 con 0x563afe5e2030 2023-09-09T23:25:57.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.965+0000 7f1e01ffb700 1 -- 172.21.15.82:0/654047744 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1df40a1670 con 0x563afe5e2030 2023-09-09T23:25:57.989 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.965+0000 7f1e01ffb700 1 -- 172.21.15.82:0/654047744 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4084270 con 0x563afe5e2030 2023-09-09T23:25:57.989 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.966+0000 7f1e1e08cb80 1 -- 172.21.15.82:0/654047744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afe5e2030 msgr2=0x563afe5eb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:25:57.989 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.966+0000 7f1e1e08cb80 1 --2- 172.21.15.82:0/654047744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afe5e2030 0x563afe5eb900 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1df40a1140 tx=0x7f1df4098f60 comp rx=0 tx=0).stop 2023-09-09T23:25:57.990 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.966+0000 7f1e1e08cb80 1 -- 172.21.15.82:0/654047744 shutdown_connections 2023-09-09T23:25:57.990 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.966+0000 7f1e1e08cb80 1 --2- 172.21.15.82:0/654047744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563afe5ebe40 0x563afe5ee230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:57.991 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.966+0000 7f1e1e08cb80 1 --2- 172.21.15.82:0/654047744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afe5e1710 0x563afe5e1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:57.991 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.966+0000 7f1e1e08cb80 1 --2- 172.21.15.82:0/654047744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afe5e2030 0x563afe5eb900 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:57.991 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.966+0000 7f1e1e08cb80 1 -- 172.21.15.82:0/654047744 >> 172.21.15.82:0/654047744 conn(0x563afe556090 msgr2=0x563afe5dcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:25:57.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.966+0000 7f1e1e08cb80 1 -- 172.21.15.82:0/654047744 shutdown_connections 2023-09-09T23:25:57.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.966+0000 7f1e1e08cb80 1 -- 172.21.15.82:0/654047744 wait complete. 2023-09-09T23:25:57.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.966+0000 7f1e1e08cb80 1 -- start start 2023-09-09T23:25:57.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.967+0000 7f1e1e08cb80 1 Processor -- start 2023-09-09T23:25:57.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.967+0000 7f1e1e08cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afe5e1710 0x563afe5e8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:57.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.967+0000 7f1e1e08cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563afe5e2030 0x563afe5e90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:57.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.967+0000 7f1e1e08cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afe5ebe40 0x563afe5e95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:57.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.967+0000 7f1e1e08cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afe4d1b70 con 0x563afe5e1710 2023-09-09T23:25:57.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.967+0000 7f1e1e08cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afe4d1870 con 0x563afe5e2030 2023-09-09T23:25:57.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.967+0000 7f1e1e08cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afe3b5920 con 0x563afe5ebe40 2023-09-09T23:25:57.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.967+0000 7f1e1e08cb80 10 client.0 _ll_register_callbacks cb 0x563afe41ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:25:57.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.967+0000 7f1e0b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afe5e1710 0x563afe5e8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:57.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.967+0000 7f1e0b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afe5e1710 0x563afe5e8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58432/0 (socket says 172.21.15.82:58432) 2023-09-09T23:25:57.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.967+0000 7f1e0b7fe700 1 -- 172.21.15.82:0/1723189588 learned_addr learned my addr 172.21.15.82:0/1723189588 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:25:57.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.967+0000 7f1e03fff700 1 --2- 172.21.15.82:0/1723189588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563afe5e2030 0x563afe5e90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:57.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.967+0000 7f1e0bfff700 1 --2- 172.21.15.82:0/1723189588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afe5ebe40 0x563afe5e95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:57.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.967+0000 7f1e0b7fe700 1 -- 172.21.15.82:0/1723189588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afe5ebe40 msgr2=0x563afe5e95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:25:57.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.967+0000 7f1e0b7fe700 1 --2- 172.21.15.82:0/1723189588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afe5ebe40 0x563afe5e95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:57.996 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.967+0000 7f1e0b7fe700 1 -- 172.21.15.82:0/1723189588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563afe5e2030 msgr2=0x563afe5e90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:25:57.996 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.967+0000 7f1e0b7fe700 1 --2- 172.21.15.82:0/1723189588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563afe5e2030 0x563afe5e90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:57.996 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.967+0000 7f1e0b7fe700 1 -- 172.21.15.82:0/1723189588 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dfc017070 con 0x563afe5e1710 2023-09-09T23:25:57.996 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.967+0000 7f1e03fff700 1 --2- 172.21.15.82:0/1723189588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563afe5e2030 0x563afe5e90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:25:57.996 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.968+0000 7f1e0b7fe700 1 --2- 172.21.15.82:0/1723189588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afe5e1710 0x563afe5e8b60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc009b20 tx=0x7f1dfc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd0651815fa8def server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:25:57.997 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.968+0000 7f1deaffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:25:57.997 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.968+0000 7f1deaffd700 1 -- 172.21.15.82:0/1723189588 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc022070 con 0x563afe5e1710 2023-09-09T23:25:57.997 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.968+0000 7f1deaffd700 1 -- 172.21.15.82:0/1723189588 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f1dfc01d620 con 0x563afe5e1710 2023-09-09T23:25:57.997 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.968+0000 7f1e1e08cb80 1 -- 172.21.15.82:0/1723189588 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563afe48b4f0 con 0x563afe5e1710 2023-09-09T23:25:57.997 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.968+0000 7f1deaffd700 1 -- 172.21.15.82:0/1723189588 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1dfc01d7e0 con 0x563afe5e1710 2023-09-09T23:25:57.997 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.968+0000 7f1e1e08cb80 10 client.16353 fetch_fsmap learned FSMap version 127 2023-09-09T23:25:57.998 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.968+0000 7f1e1e08cb80 1 -- 172.21.15.82:0/1723189588 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x7f1e0c10cc30 con 0x563afe5e1710 2023-09-09T23:25:57.998 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.968+0000 7f1deaffd700 1 -- 172.21.15.82:0/1723189588 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563afe48b4f0 con 0x563afe5e1710 2023-09-09T23:25:57.998 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.968+0000 7f1deaffd700 1 -- 172.21.15.82:0/1723189588 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 127) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1dfc027b30 con 0x563afe5e1710 2023-09-09T23:25:57.998 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.968+0000 7f1e1e08cb80 10 client.16353 fetch_fsmap finished waiting for FSMap version 127 2023-09-09T23:25:57.998 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.968+0000 7f1e1e08cb80 10 client.16353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:25:57.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.968+0000 7f1e1e08cb80 20 client.16353 populate_metadata read hostname 'smithi082' 2023-09-09T23:25:57.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.968+0000 7f1e1e08cb80 1 --2- 172.21.15.82:0/1723189588 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x563afe5fba40 0x563afe5fde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:57.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.968+0000 7f1e1e08cb80 4 client.16353 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:25:57.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.968+0000 7f1e1e08cb80 1 -- 172.21.15.82:0/1723189588 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563afe5488a0 con 0x563afe5fba40 2023-09-09T23:25:57.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.968+0000 7f1e03fff700 1 --2- 172.21.15.82:0/1723189588 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x563afe5fba40 0x563afe5fde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:58.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.969+0000 7f1deaffd700 10 client.16353 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:25:58.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.969+0000 7f1e03fff700 1 --2- 172.21.15.82:0/1723189588 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x563afe5fba40 0x563afe5fde60 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5efd747b2808891 server_cookie=d27a366a612fa2ac in_seq=0 out_seq=0 2023-09-09T23:25:58.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.970+0000 7f1deaffd700 1 -- 172.21.15.82:0/1723189588 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563afe5488a0 con 0x563afe5fba40 2023-09-09T23:25:58.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.970+0000 7f1deaffd700 10 client.16353 handle_command_reply: tid=0 2023-09-09T23:25:58.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.970+0000 7f1e1e08cb80 1 client.16353 shutdown 2023-09-09T23:25:58.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.970+0000 7f1e1e08cb80 1 -- 172.21.15.82:0/1723189588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afe5e1710 msgr2=0x563afe5e8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:25:58.001 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.970+0000 7f1e1e08cb80 1 --2- 172.21.15.82:0/1723189588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afe5e1710 0x563afe5e8b60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc009b20 tx=0x7f1dfc0047c0 comp rx=0 tx=0).stop 2023-09-09T23:25:58.001 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.970+0000 7f1e1e08cb80 1 -- 172.21.15.82:0/1723189588 shutdown_connections 2023-09-09T23:25:58.001 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.970+0000 7f1e1e08cb80 1 --2- 172.21.15.82:0/1723189588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afe5ebe40 0x563afe5e95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:58.001 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.970+0000 7f1e1e08cb80 1 --2- 172.21.15.82:0/1723189588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afe5e1710 0x563afe5e8b60 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:58.001 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.970+0000 7f1e1e08cb80 1 --2- 172.21.15.82:0/1723189588 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x563afe5fba40 0x563afe5fde60 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:58.002 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.970+0000 7f1e1e08cb80 1 --2- 172.21.15.82:0/1723189588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563afe5e2030 0x563afe5e90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:58.002 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.970+0000 7f1deaffd700 0 client.16353 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:25:58.002 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.970+0000 7f1e1e08cb80 1 -- 172.21.15.82:0/1723189588 >> 172.21.15.82:0/1723189588 conn(0x563afe556090 msgr2=0x563afe5dc780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:25:58.002 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.970+0000 7f1e1e08cb80 1 -- 172.21.15.82:0/1723189588 shutdown_connections 2023-09-09T23:25:58.003 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.970+0000 7f1e1e08cb80 1 -- 172.21.15.82:0/1723189588 wait complete. 2023-09-09T23:25:58.003 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.970+0000 7f1e1e08cb80 20 client.16353 trim_cache size 0 max 16384 2023-09-09T23:25:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:57 smithi098 ceph-mon[103124]: pgmap v376: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-09T23:25:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 388.402397 secs 2023-09-09T23:25:58.008 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.987+0000 7f1e1e08cb80 1 Processor -- start 2023-09-09T23:25:58.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.987+0000 7f1e1e08cb80 1 -- start start 2023-09-09T23:25:58.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.987+0000 7f1e1e08cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563afe5e1710 0x563afe5e9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:58.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.987+0000 7f1e1e08cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afe5e2030 0x563afe5dc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:58.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.987+0000 7f1e1e08cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afe5ebe40 0x563afe5dcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:58.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.987+0000 7f1e1e08cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afe5dd170 con 0x563afe5e2030 2023-09-09T23:25:58.010 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.987+0000 7f1e1e08cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afe5e2410 con 0x563afe5e1710 2023-09-09T23:25:58.010 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.987+0000 7f1e1e08cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afe5ee230 con 0x563afe5ebe40 2023-09-09T23:25:58.010 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.987+0000 7f1e03fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afe5e2030 0x563afe5dc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:58.010 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.987+0000 7f1e03fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afe5e2030 0x563afe5dc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58442/0 (socket says 172.21.15.82:58442) 2023-09-09T23:25:58.010 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.987+0000 7f1e03fff700 1 -- 172.21.15.82:0/3290391671 learned_addr learned my addr 172.21.15.82:0/3290391671 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:25:58.011 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.987+0000 7f1e0b7fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563afe5e1710 0x563afe5e9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:58.011 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.987+0000 7f1e0bfff700 1 --2- 172.21.15.82:0/3290391671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afe5ebe40 0x563afe5dcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:58.011 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.987+0000 7f1e03fff700 1 -- 172.21.15.82:0/3290391671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afe5ebe40 msgr2=0x563afe5dcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:25:58.012 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.987+0000 7f1e03fff700 1 --2- 172.21.15.82:0/3290391671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afe5ebe40 0x563afe5dcc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:58.012 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.987+0000 7f1e03fff700 1 -- 172.21.15.82:0/3290391671 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563afe5e1710 msgr2=0x563afe5e9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:25:58.012 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.987+0000 7f1e03fff700 1 --2- 172.21.15.82:0/3290391671 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563afe5e1710 0x563afe5e9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:58.013 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.987+0000 7f1e03fff700 1 -- 172.21.15.82:0/3290391671 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0c1a2490 con 0x563afe5e2030 2023-09-09T23:25:58.013 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.987+0000 7f1e03fff700 1 --2- 172.21.15.82:0/3290391671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afe5e2030 0x563afe5dc6f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f1df4099590 tx=0x7f1df4095310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d0d7efcedf96ec server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:25:58.013 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.988+0000 7f1e027fc700 1 -- 172.21.15.82:0/3290391671 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df40b5070 con 0x563afe5e2030 2023-09-09T23:25:58.013 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.988+0000 7f1e027fc700 1 -- 172.21.15.82:0/3290391671 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1df409d4b0 con 0x563afe5e2030 2023-09-09T23:25:58.013 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.988+0000 7f1e027fc700 1 -- 172.21.15.82:0/3290391671 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df409d120 con 0x563afe5e2030 2023-09-09T23:25:58.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.988+0000 7f1e1e08cb80 1 -- 172.21.15.82:0/3290391671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afe5e2030 msgr2=0x563afe5dc6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:25:58.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.988+0000 7f1e1e08cb80 1 --2- 172.21.15.82:0/3290391671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afe5e2030 0x563afe5dc6f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f1df4099590 tx=0x7f1df4095310 comp rx=0 tx=0).stop 2023-09-09T23:25:58.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.988+0000 7f1e1e08cb80 1 -- 172.21.15.82:0/3290391671 shutdown_connections 2023-09-09T23:25:58.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.988+0000 7f1e1e08cb80 1 --2- 172.21.15.82:0/3290391671 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563afe5e1710 0x563afe5e9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:58.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.988+0000 7f1e1e08cb80 1 --2- 172.21.15.82:0/3290391671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afe5ebe40 0x563afe5dcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:58.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.988+0000 7f1e1e08cb80 1 --2- 172.21.15.82:0/3290391671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afe5e2030 0x563afe5dc6f0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:58.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.988+0000 7f1e1e08cb80 1 -- 172.21.15.82:0/3290391671 >> 172.21.15.82:0/3290391671 conn(0x563afe556090 msgr2=0x563afe5e3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:25:58.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.988+0000 7f1e1e08cb80 1 -- 172.21.15.82:0/3290391671 shutdown_connections 2023-09-09T23:25:58.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.988+0000 7f1e1e08cb80 1 -- 172.21.15.82:0/3290391671 wait complete. 2023-09-09T23:25:58.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.989+0000 7f1e1e08cb80 1 -- start start 2023-09-09T23:25:58.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.989+0000 7f1e1e08cb80 1 Processor -- start 2023-09-09T23:25:58.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.989+0000 7f1e1e08cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afe5e1710 0x563afe5f27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:58.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.989+0000 7f1e1e08cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afe5e2030 0x563afe5f0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:58.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.989+0000 7f1e1e08cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563afe5ebe40 0x563afe5f1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:58.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.989+0000 7f1e1e08cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afe5f1540 con 0x563afe5e1710 2023-09-09T23:25:58.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.989+0000 7f1e1e08cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afe5f16b0 con 0x563afe5ebe40 2023-09-09T23:25:58.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.989+0000 7f1e1e08cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afe5f1870 con 0x563afe5e2030 2023-09-09T23:25:58.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.989+0000 7f1e1e08cb80 10 client.0 _ll_register_callbacks cb 0x563afe41ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:25:58.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.989+0000 7f1e0b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afe5e1710 0x563afe5f27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:58.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.989+0000 7f1e0b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afe5e1710 0x563afe5f27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58446/0 (socket says 172.21.15.82:58446) 2023-09-09T23:25:58.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.989+0000 7f1e0b7fe700 1 -- 172.21.15.82:0/3584077613 learned_addr learned my addr 172.21.15.82:0/3584077613 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:25:58.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.989+0000 7f1e03fff700 1 --2- 172.21.15.82:0/3584077613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afe5e2030 0x563afe5f0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:58.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.989+0000 7f1e0bfff700 1 --2- 172.21.15.82:0/3584077613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563afe5ebe40 0x563afe5f1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:58.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.990+0000 7f1e03fff700 1 -- 172.21.15.82:0/3584077613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563afe5ebe40 msgr2=0x563afe5f1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:25:58.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.990+0000 7f1e03fff700 1 --2- 172.21.15.82:0/3584077613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563afe5ebe40 0x563afe5f1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:58.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.990+0000 7f1e03fff700 1 -- 172.21.15.82:0/3584077613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afe5e1710 msgr2=0x563afe5f27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:25:58.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.990+0000 7f1e03fff700 1 --2- 172.21.15.82:0/3584077613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afe5e1710 0x563afe5f27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:58.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.990+0000 7f1e03fff700 1 -- 172.21.15.82:0/3584077613 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563afe35fde0 con 0x563afe5e2030 2023-09-09T23:25:58.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.990+0000 7f1e0b7fe700 1 --2- 172.21.15.82:0/3584077613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afe5e1710 0x563afe5f27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:25:58.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.990+0000 7f1e03fff700 1 --2- 172.21.15.82:0/3584077613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afe5e2030 0x563afe5f0ac0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1df4099590 tx=0x7f1df40c9970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2804546bc8a1fd0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:25:58.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.990+0000 7f1deaffd700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-09-09T23:25:58.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.990+0000 7f1deaffd700 1 -- 172.21.15.82:0/3584077613 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df40b5030 con 0x563afe5e2030 2023-09-09T23:25:58.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.990+0000 7f1e1e08cb80 1 -- 172.21.15.82:0/3584077613 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1df40a1670 con 0x563afe5e2030 2023-09-09T23:25:58.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.990+0000 7f1deaffd700 1 -- 172.21.15.82:0/3584077613 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f1df40b6070 con 0x563afe5e2030 2023-09-09T23:25:58.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.990+0000 7f1deaffd700 1 -- 172.21.15.82:0/3584077613 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1df4098f60 con 0x563afe5e2030 2023-09-09T23:25:58.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.990+0000 7f1e1e08cb80 10 client.25726 fetch_fsmap learned FSMap version 127 2023-09-09T23:25:58.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.990+0000 7f1e1e08cb80 1 -- 172.21.15.82:0/3584077613 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x563afe348c60 con 0x563afe5e2030 2023-09-09T23:25:58.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.990+0000 7f1deaffd700 1 -- 172.21.15.82:0/3584077613 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1df40a1670 con 0x563afe5e2030 2023-09-09T23:25:58.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.990+0000 7f1deaffd700 1 -- 172.21.15.82:0/3584077613 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 127) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1df40840b0 con 0x563afe5e2030 2023-09-09T23:25:58.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.991+0000 7f1e1e08cb80 10 client.25726 fetch_fsmap finished waiting for FSMap version 127 2023-09-09T23:25:58.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.991+0000 7f1e1e08cb80 10 client.25726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:25:58.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.991+0000 7f1e1e08cb80 20 client.25726 populate_metadata read hostname 'smithi082' 2023-09-09T23:25:58.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.991+0000 7f1e1e08cb80 1 --2- 172.21.15.82:0/3584077613 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x563afe650830 0x563afe652c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:25:58.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.991+0000 7f1e1e08cb80 4 client.25726 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:25:58.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.991+0000 7f1e1e08cb80 1 -- 172.21.15.82:0/3584077613 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563afe52c920 con 0x563afe650830 2023-09-09T23:25:58.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.991+0000 7f1e0b7fe700 1 --2- 172.21.15.82:0/3584077613 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x563afe650830 0x563afe652c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:25:58.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.991+0000 7f1deaffd700 10 client.25726 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:25:58.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.991+0000 7f1e0b7fe700 1 --2- 172.21.15.82:0/3584077613 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x563afe650830 0x563afe652c20 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7c5e4a1e3370ed server_cookie=dee93e444a4c8271 in_seq=0 out_seq=0 2023-09-09T23:25:58.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.991+0000 7f1deaffd700 1 -- 172.21.15.82:0/3584077613 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x563afe52c920 con 0x563afe650830 2023-09-09T23:25:58.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.991+0000 7f1deaffd700 10 client.25726 handle_command_reply: tid=0 2023-09-09T23:25:58.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.991+0000 7f1e1e08cb80 1 client.25726 shutdown 2023-09-09T23:25:58.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.991+0000 7f1e1e08cb80 1 -- 172.21.15.82:0/3584077613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afe5e2030 msgr2=0x563afe5f0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:25:58.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.991+0000 7f1e1e08cb80 1 --2- 172.21.15.82:0/3584077613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afe5e2030 0x563afe5f0ac0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1df4099590 tx=0x7f1df40c9970 comp rx=0 tx=0).stop 2023-09-09T23:25:58.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.992+0000 7f1e1e08cb80 1 -- 172.21.15.82:0/3584077613 shutdown_connections 2023-09-09T23:25:58.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.992+0000 7f1e1e08cb80 1 --2- 172.21.15.82:0/3584077613 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x563afe650830 0x563afe652c20 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:58.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.992+0000 7f1e1e08cb80 1 --2- 172.21.15.82:0/3584077613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563afe5ebe40 0x563afe5f1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:58.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.992+0000 7f1e1e08cb80 1 --2- 172.21.15.82:0/3584077613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afe5e1710 0x563afe5f27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:58.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.992+0000 7f1e1e08cb80 1 --2- 172.21.15.82:0/3584077613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afe5e2030 0x563afe5f0ac0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:58.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.992+0000 7f1deaffd700 0 client.25726 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:25:58.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.992+0000 7f1e1e08cb80 1 -- 172.21.15.82:0/3584077613 >> 172.21.15.82:0/3584077613 conn(0x563afe556090 msgr2=0x563afe5dc670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:25:58.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.992+0000 7f1e1e08cb80 1 -- 172.21.15.82:0/3584077613 shutdown_connections 2023-09-09T23:25:58.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.992+0000 7f1e1e08cb80 1 -- 172.21.15.82:0/3584077613 wait complete. 2023-09-09T23:25:58.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.992+0000 7f1e1e08cb80 20 client.25726 trim_cache size 0 max 16384 2023-09-09T23:25:58.027 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:25:58.027 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:25:58.027 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:25:58.027 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:25:58.028 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:25:58.028 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:25:58.028 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:25:58.028 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:25:58.029 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:25:58.029 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:25:58.030 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.992+0000 7f1e11ec6700 1 -- 172.21.15.82:0/26641571 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f1df80a07b0 msgr2=0x7f1df80a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:25:58.030 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.992+0000 7f1e11ec6700 1 --2- 172.21.15.82:0/26641571 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f1df80a07b0 0x7f1df80a2c70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc009c60 tx=0x7f1dfc009260 comp rx=0 tx=0).stop 2023-09-09T23:25:58.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.992+0000 7f1e11ec6700 1 -- 172.21.15.82:0/26641571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e0c1004d0 msgr2=0x7f1e0c195ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:25:58.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.992+0000 7f1e11ec6700 1 --2- 172.21.15.82:0/26641571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e0c1004d0 0x7f1e0c195ba0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1df400e9a0 tx=0x7f1df400ecb0 comp rx=0 tx=0).stop 2023-09-09T23:25:58.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.992+0000 7f1e11ec6700 1 -- 172.21.15.82:0/26641571 shutdown_connections 2023-09-09T23:25:58.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.992+0000 7f1e11ec6700 1 --2- 172.21.15.82:0/26641571 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f1df80a07b0 0x7f1df80a2c70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:58.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.992+0000 7f1e11ec6700 1 --2- 172.21.15.82:0/26641571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e0c105570 0x7f1e0c19a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:58.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.992+0000 7f1e11ec6700 1 --2- 172.21.15.82:0/26641571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e0c0ffb10 0x7f1e0c195660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:58.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.992+0000 7f1e11ec6700 1 --2- 172.21.15.82:0/26641571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e0c1004d0 0x7f1e0c195ba0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:25:58.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.992+0000 7f1e11ec6700 1 -- 172.21.15.82:0/26641571 >> 172.21.15.82:0/26641571 conn(0x7f1e0c076a70 msgr2=0x7f1e0c103060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:25:58.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.992+0000 7f1e11ec6700 1 -- 172.21.15.82:0/26641571 shutdown_connections 2023-09-09T23:25:58.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:25:57.992+0000 7f1e11ec6700 1 -- 172.21.15.82:0/26641571 wait complete. 2023-09-09T23:25:58.033 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:25:58.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 390.023611 secs 2023-09-09T23:25:58.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 390.023611 secs 2023-09-09T23:25:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 390.023611 secs 2023-09-09T23:25:59.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:59 smithi170 ceph-mon[102832]: pgmap v377: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:25:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:25:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 390.752673 secs 2023-09-09T23:25:59.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:59 smithi082 ceph-mon[98694]: pgmap v377: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:25:59.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:25:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 390.752673 secs 2023-09-09T23:26:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:59 smithi098 ceph-mon[103124]: pgmap v377: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:26:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:25:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 390.752673 secs 2023-09-09T23:26:01.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:01 smithi170 ceph-mon[102832]: pgmap v378: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-09T23:26:01.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:01 smithi082 ceph-mon[98694]: pgmap v378: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-09T23:26:02.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:01 smithi098 ceph-mon[103124]: pgmap v378: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-09T23:26:02.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 393.402462 secs 2023-09-09T23:26:02.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 393.402462 secs 2023-09-09T23:26:03.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 393.402462 secs 2023-09-09T23:26:03.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:03 smithi170 ceph-mon[102832]: pgmap v379: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 21 op/s 2023-09-09T23:26:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 395.023673 secs 2023-09-09T23:26:03.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:03 smithi082 ceph-mon[98694]: pgmap v379: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 21 op/s 2023-09-09T23:26:03.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 395.023673 secs 2023-09-09T23:26:04.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:03 smithi098 ceph-mon[103124]: pgmap v379: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 21 op/s 2023-09-09T23:26:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 395.023673 secs 2023-09-09T23:26:04.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 395.752736 secs 2023-09-09T23:26:04.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 395.752736 secs 2023-09-09T23:26:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 395.752736 secs 2023-09-09T23:26:05.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:05 smithi170 ceph-mon[102832]: pgmap v380: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 20 op/s 2023-09-09T23:26:05.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:05 smithi082 ceph-mon[98694]: pgmap v380: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 20 op/s 2023-09-09T23:26:06.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:05 smithi098 ceph-mon[103124]: pgmap v380: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 20 op/s 2023-09-09T23:26:07.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:07 smithi170 ceph-mon[102832]: pgmap v381: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 27 op/s 2023-09-09T23:26:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 398.402549 secs 2023-09-09T23:26:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:07 smithi082 ceph-mon[98694]: pgmap v381: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 27 op/s 2023-09-09T23:26:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 398.402549 secs 2023-09-09T23:26:08.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:07 smithi098 ceph-mon[103124]: pgmap v381: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 27 op/s 2023-09-09T23:26:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 398.402549 secs 2023-09-09T23:26:08.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:08 smithi170 ceph-mon[102832]: osdmap e97: 12 total, 12 up, 12 in 2023-09-09T23:26:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 400.023758 secs 2023-09-09T23:26:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:08 smithi082 ceph-mon[98694]: osdmap e97: 12 total, 12 up, 12 in 2023-09-09T23:26:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 400.023758 secs 2023-09-09T23:26:09.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:08 smithi098 ceph-mon[103124]: osdmap e97: 12 total, 12 up, 12 in 2023-09-09T23:26:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 400.023758 secs 2023-09-09T23:26:09.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:09 smithi170 ceph-mon[102832]: pgmap v383: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:26:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 400.752808 secs 2023-09-09T23:26:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:09 smithi082 ceph-mon[98694]: pgmap v383: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:26:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 400.752808 secs 2023-09-09T23:26:10.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:09 smithi098 ceph-mon[103124]: pgmap v383: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:26:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 400.752808 secs 2023-09-09T23:26:11.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:11 smithi170 ceph-mon[102832]: pgmap v384: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:26:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:11 smithi082 ceph-mon[98694]: pgmap v384: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:26:12.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:11 smithi098 ceph-mon[103124]: pgmap v384: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:26:12.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 403.402658 secs 2023-09-09T23:26:12.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 403.402658 secs 2023-09-09T23:26:13.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 403.402658 secs 2023-09-09T23:26:13.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:13 smithi170 ceph-mon[102832]: pgmap v385: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 28 KiB/s wr, 13 op/s 2023-09-09T23:26:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 405.023817 secs 2023-09-09T23:26:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:13 smithi170 ceph-mon[102832]: osdmap e98: 12 total, 12 up, 12 in 2023-09-09T23:26:13.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:13 smithi082 ceph-mon[98694]: pgmap v385: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 28 KiB/s wr, 13 op/s 2023-09-09T23:26:13.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 405.023817 secs 2023-09-09T23:26:13.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:13 smithi082 ceph-mon[98694]: osdmap e98: 12 total, 12 up, 12 in 2023-09-09T23:26:14.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:13 smithi098 ceph-mon[103124]: pgmap v385: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 28 KiB/s wr, 13 op/s 2023-09-09T23:26:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 405.023817 secs 2023-09-09T23:26:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:13 smithi098 ceph-mon[103124]: osdmap e98: 12 total, 12 up, 12 in 2023-09-09T23:26:14.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 405.752869 secs 2023-09-09T23:26:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:14 smithi170 ceph-mon[102832]: pgmap v387: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:26:14.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 405.752869 secs 2023-09-09T23:26:14.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:14 smithi082 ceph-mon[98694]: pgmap v387: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:26:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 405.752869 secs 2023-09-09T23:26:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:14 smithi098 ceph-mon[103124]: pgmap v387: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:26:17.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:17 smithi082 ceph-mon[98694]: pgmap v388: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2023-09-09T23:26:17.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 408.402806 secs 2023-09-09T23:26:17.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:17 smithi098 ceph-mon[103124]: pgmap v388: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2023-09-09T23:26:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 408.402806 secs 2023-09-09T23:26:17.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:17 smithi170 ceph-mon[102832]: pgmap v388: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2023-09-09T23:26:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 408.402806 secs 2023-09-09T23:26:18.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:26:18.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 410.023881 secs 2023-09-09T23:26:18.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:26:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:26:18.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:26:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 410.023881 secs 2023-09-09T23:26:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:26:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:26:18.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:26:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 410.023881 secs 2023-09-09T23:26:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:26:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:26:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:19 smithi098 ceph-mon[103124]: pgmap v389: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.8 KiB/s wr, 11 op/s 2023-09-09T23:26:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 410.752936 secs 2023-09-09T23:26:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:26:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:26:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:26:19.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:19 smithi170 ceph-mon[102832]: pgmap v389: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.8 KiB/s wr, 11 op/s 2023-09-09T23:26:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 410.752936 secs 2023-09-09T23:26:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:26:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:26:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:26:19.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:19 smithi082 ceph-mon[98694]: pgmap v389: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.8 KiB/s wr, 11 op/s 2023-09-09T23:26:19.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 410.752936 secs 2023-09-09T23:26:19.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:26:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:26:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:26:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:21 smithi098 ceph-mon[103124]: pgmap v390: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 13 op/s 2023-09-09T23:26:21.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:21 smithi170 ceph-mon[102832]: pgmap v390: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 13 op/s 2023-09-09T23:26:21.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:21 smithi082 ceph-mon[98694]: pgmap v390: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 13 op/s 2023-09-09T23:26:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 413.402889 secs 2023-09-09T23:26:22.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 413.402889 secs 2023-09-09T23:26:22.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 413.402889 secs 2023-09-09T23:26:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:23 smithi098 ceph-mon[103124]: pgmap v391: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 KiB/s wr, 13 op/s 2023-09-09T23:26:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 415.023939 secs 2023-09-09T23:26:23.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:23 smithi170 ceph-mon[102832]: pgmap v391: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 KiB/s wr, 13 op/s 2023-09-09T23:26:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 415.023939 secs 2023-09-09T23:26:23.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:23 smithi082 ceph-mon[98694]: pgmap v391: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 19 KiB/s wr, 13 op/s 2023-09-09T23:26:23.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 415.023939 secs 2023-09-09T23:26:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 415.753004 secs 2023-09-09T23:26:24.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 415.753004 secs 2023-09-09T23:26:24.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 415.753004 secs 2023-09-09T23:26:25.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:25 smithi170 ceph-mon[102832]: pgmap v392: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2023-09-09T23:26:25.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:25 smithi082 ceph-mon[98694]: pgmap v392: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2023-09-09T23:26:26.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:25 smithi098 ceph-mon[103124]: pgmap v392: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2023-09-09T23:26:27.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:27 smithi170 ceph-mon[102832]: pgmap v393: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 16 op/s 2023-09-09T23:26:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 418.402949 secs 2023-09-09T23:26:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:27 smithi082 ceph-mon[98694]: pgmap v393: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 16 op/s 2023-09-09T23:26:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 418.402949 secs 2023-09-09T23:26:28.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:27 smithi098 ceph-mon[103124]: pgmap v393: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 16 op/s 2023-09-09T23:26:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 418.402949 secs 2023-09-09T23:26:28.034 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:26:28.172 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.172+0000 7ff18f78b700 1 -- 172.21.15.82:0/992535980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1880ff9f0 msgr2=0x7ff188103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:26:28.172 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.172+0000 7ff18f78b700 1 --2- 172.21.15.82:0/992535980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1880ff9f0 0x7ff188103850 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ff178009990 tx=0x7ff178009ca0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.172 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.172+0000 7ff18f78b700 1 -- 172.21.15.82:0/992535980 shutdown_connections 2023-09-09T23:26:28.172 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.172+0000 7ff18f78b700 1 --2- 172.21.15.82:0/992535980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1880ff040 0x7ff1880ff420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.173 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.172+0000 7ff18f78b700 1 --2- 172.21.15.82:0/992535980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff188103f80 0x7ff18810bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.173 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.172+0000 7ff18f78b700 1 --2- 172.21.15.82:0/992535980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1880ff9f0 0x7ff188103850 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.173 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.172+0000 7ff18f78b700 1 -- 172.21.15.82:0/992535980 >> 172.21.15.82:0/992535980 conn(0x7ff188075740 msgr2=0x7ff188075b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:26:28.173 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.172+0000 7ff18f78b700 1 -- 172.21.15.82:0/992535980 shutdown_connections 2023-09-09T23:26:28.174 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.172+0000 7ff18f78b700 1 -- 172.21.15.82:0/992535980 wait complete. 2023-09-09T23:26:28.175 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.173+0000 7ff18f78b700 1 Processor -- start 2023-09-09T23:26:28.176 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.173+0000 7ff18f78b700 1 -- start start 2023-09-09T23:26:28.176 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.173+0000 7ff18f78b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1880ff040 0x7ff188197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:28.176 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.173+0000 7ff18f78b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff188103f80 0x7ff188197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:28.176 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.173+0000 7ff18f78b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff188198080 0x7ff1881a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:28.177 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.173+0000 7ff18f78b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1881a2d60 con 0x7ff188103f80 2023-09-09T23:26:28.177 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.173+0000 7ff18f78b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1881a2ed0 con 0x7ff188198080 2023-09-09T23:26:28.177 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.173+0000 7ff18f78b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1881a3090 con 0x7ff1880ff040 2023-09-09T23:26:28.177 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.173+0000 7ff18cd26700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff188103f80 0x7ff188197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:28.177 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.173+0000 7ff18cd26700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff188103f80 0x7ff188197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58466/0 (socket says 172.21.15.82:58466) 2023-09-09T23:26:28.177 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.173+0000 7ff18cd26700 1 -- 172.21.15.82:0/366803001 learned_addr learned my addr 172.21.15.82:0/366803001 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:26:28.178 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.173+0000 7ff18d527700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1880ff040 0x7ff188197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:28.178 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.173+0000 7ff18dd28700 1 --2- 172.21.15.82:0/366803001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff188198080 0x7ff1881a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:28.178 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.173+0000 7ff18cd26700 1 -- 172.21.15.82:0/366803001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1880ff040 msgr2=0x7ff188197560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:26:28.178 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.173+0000 7ff18cd26700 1 --2- 172.21.15.82:0/366803001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1880ff040 0x7ff188197560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.179 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.173+0000 7ff18cd26700 1 -- 172.21.15.82:0/366803001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff188198080 msgr2=0x7ff1881a25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:26:28.179 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.173+0000 7ff18cd26700 1 --2- 172.21.15.82:0/366803001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff188198080 0x7ff1881a25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.179 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.173+0000 7ff18cd26700 1 -- 172.21.15.82:0/366803001 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1780095f0 con 0x7ff188103f80 2023-09-09T23:26:28.179 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.173+0000 7ff18cd26700 1 --2- 172.21.15.82:0/366803001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff188103f80 0x7ff188197aa0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff178000c00 tx=0x7ff1780047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:26:28.180 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.174+0000 7ff17e7fc700 1 -- 172.21.15.82:0/366803001 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17801d070 con 0x7ff188103f80 2023-09-09T23:26:28.180 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.174+0000 7ff17e7fc700 1 -- 172.21.15.82:0/366803001 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff178021410 con 0x7ff188103f80 2023-09-09T23:26:28.180 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.174+0000 7ff18f78b700 1 -- 172.21.15.82:0/366803001 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1881a3310 con 0x7ff188103f80 2023-09-09T23:26:28.180 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.174+0000 7ff17e7fc700 1 -- 172.21.15.82:0/366803001 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff178017650 con 0x7ff188103f80 2023-09-09T23:26:28.180 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.174+0000 7ff18f78b700 1 -- 172.21.15.82:0/366803001 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1881a37c0 con 0x7ff188103f80 2023-09-09T23:26:28.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.177+0000 7ff19b951b80 1 Processor -- start 2023-09-09T23:26:28.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.177+0000 7ff19b951b80 1 -- start start 2023-09-09T23:26:28.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.178+0000 7ff19b951b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55adb7d20710 0x55adb7d20af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:28.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.178+0000 7ff19b951b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adb7d21030 0x55adb7d2a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:28.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.178+0000 7ff19b951b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55adb7d2ae40 0x55adb7d2d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:28.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.178+0000 7ff19b951b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb7c79730 con 0x55adb7d21030 2023-09-09T23:26:28.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.178+0000 7ff19b951b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb7c77050 con 0x55adb7d2ae40 2023-09-09T23:26:28.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.178+0000 7ff19b951b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb7c72940 con 0x55adb7d20710 2023-09-09T23:26:28.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.178+0000 7ff18dd28700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55adb7d20710 0x55adb7d20af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:28.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.178+0000 7ff18dd28700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55adb7d20710 0x55adb7d20af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:56984/0 (socket says 172.21.15.82:56984) 2023-09-09T23:26:28.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.178+0000 7ff18cd26700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55adb7d2ae40 0x55adb7d2d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:28.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.178+0000 7ff18dd28700 1 -- 172.21.15.82:0/3527916033 learned_addr learned my addr 172.21.15.82:0/3527916033 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:26:28.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.178+0000 7ff18dd28700 1 -- 172.21.15.82:0/3527916033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55adb7d2ae40 msgr2=0x55adb7d2d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:26:28.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.178+0000 7ff18dd28700 1 --2- 172.21.15.82:0/3527916033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55adb7d2ae40 0x55adb7d2d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.178+0000 7ff18dd28700 1 -- 172.21.15.82:0/3527916033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adb7d21030 msgr2=0x55adb7d2a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:26:28.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.178+0000 7ff18dd28700 1 --2- 172.21.15.82:0/3527916033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adb7d21030 0x55adb7d2a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.178+0000 7ff18dd28700 1 -- 172.21.15.82:0/3527916033 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adb7a9ede0 con 0x55adb7d20710 2023-09-09T23:26:28.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.178+0000 7ff18dd28700 1 --2- 172.21.15.82:0/3527916033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55adb7d20710 0x55adb7d20af0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff180011680 tx=0x7ff180009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44456018bc3b9d35 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:26:28.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.179+0000 7ff1737fe700 1 -- 172.21.15.82:0/3527916033 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18000e040 con 0x55adb7d20710 2023-09-09T23:26:28.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.179+0000 7ff1737fe700 1 -- 172.21.15.82:0/3527916033 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff18000f040 con 0x55adb7d20710 2023-09-09T23:26:28.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.179+0000 7ff1737fe700 1 -- 172.21.15.82:0/3527916033 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1800048b0 con 0x55adb7d20710 2023-09-09T23:26:28.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.179+0000 7ff17e7fc700 1 -- 172.21.15.82:0/366803001 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) v1 ==== 135390+0+0 (secure 0 0 0) 0x7ff178021580 con 0x7ff188103f80 2023-09-09T23:26:28.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.179+0000 7ff19b951b80 1 -- 172.21.15.82:0/3527916033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55adb7d20710 msgr2=0x55adb7d20af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:26:28.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.179+0000 7ff19b951b80 1 --2- 172.21.15.82:0/3527916033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55adb7d20710 0x55adb7d20af0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff180011680 tx=0x7ff180009920 comp rx=0 tx=0).stop 2023-09-09T23:26:28.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.179+0000 7ff19b951b80 1 -- 172.21.15.82:0/3527916033 shutdown_connections 2023-09-09T23:26:28.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.179+0000 7ff19b951b80 1 --2- 172.21.15.82:0/3527916033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55adb7d2ae40 0x55adb7d2d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.179+0000 7ff19b951b80 1 --2- 172.21.15.82:0/3527916033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55adb7d20710 0x55adb7d20af0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.179+0000 7ff19b951b80 1 --2- 172.21.15.82:0/3527916033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adb7d21030 0x55adb7d2a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.179+0000 7ff19b951b80 1 -- 172.21.15.82:0/3527916033 >> 172.21.15.82:0/3527916033 conn(0x55adb7c95090 msgr2=0x55adb7d1bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:26:28.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.179+0000 7ff19b951b80 1 -- 172.21.15.82:0/3527916033 shutdown_connections 2023-09-09T23:26:28.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.179+0000 7ff19b951b80 1 -- 172.21.15.82:0/3527916033 wait complete. 2023-09-09T23:26:28.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.180+0000 7ff17e7fc700 1 --2- 172.21.15.82:0/366803001 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7ff1740a06c0 0x7ff1740a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:28.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.180+0000 7ff19b951b80 1 -- start start 2023-09-09T23:26:28.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.180+0000 7ff18dd28700 1 --2- 172.21.15.82:0/366803001 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7ff1740a06c0 0x7ff1740a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:28.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.180+0000 7ff17e7fc700 1 -- 172.21.15.82:0/366803001 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8455+0+0 (secure 0 0 0) 0x7ff1780d4100 con 0x7ff188103f80 2023-09-09T23:26:28.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.180+0000 7ff19b951b80 1 Processor -- start 2023-09-09T23:26:28.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.180+0000 7ff18dd28700 1 --2- 172.21.15.82:0/366803001 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7ff1740a06c0 0x7ff1740a2b80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff180000f80 tx=0x7ff1800040c0 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:26:28.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.180+0000 7ff19b951b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55adb7d20710 0x55adb7d2a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:28.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.180+0000 7ff19b951b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55adb7d21030 0x55adb7d279e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:28.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.180+0000 7ff19b951b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adb7d2ae40 0x55adb7d27f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:28.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.180+0000 7ff19b951b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb7c10b70 con 0x55adb7d2ae40 2023-09-09T23:26:28.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.180+0000 7ff19b951b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb7c10870 con 0x55adb7d21030 2023-09-09T23:26:28.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.180+0000 7ff19b951b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb7af4920 con 0x55adb7d20710 2023-09-09T23:26:28.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.181+0000 7ff19b951b80 10 client.0 _ll_register_callbacks cb 0x55adb7b5aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:26:28.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.181+0000 7ff18dd28700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adb7d2ae40 0x55adb7d27f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:28.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.181+0000 7ff18cd26700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55adb7d21030 0x55adb7d279e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:28.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.181+0000 7ff18d527700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55adb7d20710 0x55adb7d2a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:28.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.181+0000 7ff18cd26700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55adb7d21030 0x55adb7d279e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:44044/0 (socket says 172.21.15.82:44044) 2023-09-09T23:26:28.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.181+0000 7ff18d527700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55adb7d20710 0x55adb7d2a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:56992/0 (socket says 172.21.15.82:56992) 2023-09-09T23:26:28.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.181+0000 7ff18dd28700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adb7d2ae40 0x55adb7d27f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58482/0 (socket says 172.21.15.82:58482) 2023-09-09T23:26:28.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.181+0000 7ff18cd26700 1 -- 172.21.15.82:0/3227133684 learned_addr learned my addr 172.21.15.82:0/3227133684 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:26:28.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.181+0000 7ff18cd26700 1 -- 172.21.15.82:0/3227133684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55adb7d20710 msgr2=0x55adb7d2a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:26:28.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.181+0000 7ff18cd26700 1 --2- 172.21.15.82:0/3227133684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55adb7d20710 0x55adb7d2a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.181+0000 7ff18cd26700 1 -- 172.21.15.82:0/3227133684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adb7d2ae40 msgr2=0x55adb7d27f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:26:28.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.181+0000 7ff18cd26700 1 --2- 172.21.15.82:0/3227133684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adb7d2ae40 0x55adb7d27f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.181+0000 7ff18cd26700 1 -- 172.21.15.82:0/3227133684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff180011330 con 0x55adb7d21030 2023-09-09T23:26:28.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.181+0000 7ff18dd28700 1 --2- 172.21.15.82:0/3227133684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adb7d2ae40 0x55adb7d27f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:26:28.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.181+0000 7ff18d527700 1 --2- 172.21.15.82:0/3227133684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55adb7d20710 0x55adb7d2a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:26:28.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.181+0000 7ff18cd26700 1 --2- 172.21.15.82:0/3227133684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55adb7d21030 0x55adb7d279e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff1780d1940 tx=0x7ff178021a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34de307ffb0c3e87 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:26:28.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.181+0000 7ff170ff9700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-09T23:26:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.181+0000 7ff170ff9700 1 -- 172.21.15.82:0/3227133684 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1780a13c0 con 0x55adb7d21030 2023-09-09T23:26:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.181+0000 7ff170ff9700 1 -- 172.21.15.82:0/3227133684 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8455+0+0 (secure 0 0 0) 0x7ff1780a1cf0 con 0x55adb7d21030 2023-09-09T23:26:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.181+0000 7ff19b951b80 1 -- 172.21.15.82:0/3227133684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55adb7bca4f0 con 0x55adb7d21030 2023-09-09T23:26:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.181+0000 7ff170ff9700 1 -- 172.21.15.82:0/3227133684 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1780a1130 con 0x55adb7d21030 2023-09-09T23:26:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.181+0000 7ff19b951b80 10 client.25832 fetch_fsmap learned FSMap version 127 2023-09-09T23:26:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.181+0000 7ff19b951b80 1 -- 172.21.15.82:0/3227133684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x55adb7a87c60 con 0x55adb7d21030 2023-09-09T23:26:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.182+0000 7ff170ff9700 1 -- 172.21.15.82:0/3227133684 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55adb7bca4f0 con 0x55adb7d21030 2023-09-09T23:26:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.182+0000 7ff170ff9700 1 -- 172.21.15.82:0/3227133684 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 127) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff1780a0ae0 con 0x55adb7d21030 2023-09-09T23:26:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.182+0000 7ff19b951b80 10 client.25832 fetch_fsmap finished waiting for FSMap version 127 2023-09-09T23:26:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.182+0000 7ff19b951b80 10 client.25832 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:26:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.182+0000 7ff19b951b80 20 client.25832 populate_metadata read hostname 'smithi082' 2023-09-09T23:26:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.182+0000 7ff19b951b80 1 --2- 172.21.15.82:0/3227133684 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55adb7d3a4c0 0x55adb7d3c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.182+0000 7ff19b951b80 4 client.25832 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:26:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.182+0000 7ff19b951b80 1 -- 172.21.15.82:0/3227133684 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55adb7c878a0 con 0x55adb7d3a4c0 2023-09-09T23:26:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.182+0000 7ff18d527700 1 --2- 172.21.15.82:0/3227133684 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55adb7d3a4c0 0x55adb7d3c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.182+0000 7ff170ff9700 10 client.25832 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:26:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.182+0000 7ff18d527700 1 --2- 172.21.15.82:0/3227133684 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55adb7d3a4c0 0x55adb7d3c8e0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=209d6439f7c909df server_cookie=56cc29c61065c4dd in_seq=0 out_seq=0 2023-09-09T23:26:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.183+0000 7ff170ff9700 1 -- 172.21.15.82:0/3227133684 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55adb7c878a0 con 0x55adb7d3a4c0 2023-09-09T23:26:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.183+0000 7ff170ff9700 10 client.25832 handle_command_reply: tid=0 2023-09-09T23:26:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.183+0000 7ff19b951b80 1 client.25832 shutdown 2023-09-09T23:26:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.183+0000 7ff19b951b80 1 -- 172.21.15.82:0/3227133684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55adb7d21030 msgr2=0x55adb7d279e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:26:28.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.183+0000 7ff19b951b80 1 --2- 172.21.15.82:0/3227133684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55adb7d21030 0x55adb7d279e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff1780d1940 tx=0x7ff178021a90 comp rx=0 tx=0).stop 2023-09-09T23:26:28.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.183+0000 7ff19b951b80 1 -- 172.21.15.82:0/3227133684 shutdown_connections 2023-09-09T23:26:28.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.183+0000 7ff19b951b80 1 --2- 172.21.15.82:0/3227133684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adb7d2ae40 0x55adb7d27f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.183+0000 7ff19b951b80 1 --2- 172.21.15.82:0/3227133684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55adb7d20710 0x55adb7d2a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.183+0000 7ff19b951b80 1 --2- 172.21.15.82:0/3227133684 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55adb7d3a4c0 0x55adb7d3c8e0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.183+0000 7ff19b951b80 1 --2- 172.21.15.82:0/3227133684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55adb7d21030 0x55adb7d279e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.183+0000 7ff170ff9700 0 client.25832 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:26:28.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.183+0000 7ff19b951b80 1 -- 172.21.15.82:0/3227133684 >> 172.21.15.82:0/3227133684 conn(0x55adb7c95090 msgr2=0x55adb7d1b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:26:28.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.184+0000 7ff19b951b80 1 -- 172.21.15.82:0/3227133684 shutdown_connections 2023-09-09T23:26:28.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.184+0000 7ff19b951b80 1 -- 172.21.15.82:0/3227133684 wait complete. 2023-09-09T23:26:28.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.184+0000 7ff19b951b80 20 client.25832 trim_cache size 0 max 16384 2023-09-09T23:26:28.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.200+0000 7ff19b951b80 1 Processor -- start 2023-09-09T23:26:28.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.200+0000 7ff19b951b80 1 -- start start 2023-09-09T23:26:28.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.200+0000 7ff19b951b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adb7d20710 0x55adb7d31a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:28.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.200+0000 7ff19b951b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55adb7d21030 0x55adb7d36180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:28.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.200+0000 7ff19b951b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55adb7d2ae40 0x55adb7d22d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:28.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.200+0000 7ff19b951b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb7d2d750 con 0x55adb7d20710 2023-09-09T23:26:28.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.200+0000 7ff19b951b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb7d37280 con 0x55adb7d2ae40 2023-09-09T23:26:28.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.200+0000 7ff19b951b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb7d28fa0 con 0x55adb7d21030 2023-09-09T23:26:28.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.200+0000 7ff18d527700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adb7d20710 0x55adb7d31a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:28.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.200+0000 7ff18dd28700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55adb7d2ae40 0x55adb7d22d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:28.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.200+0000 7ff18d527700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adb7d20710 0x55adb7d31a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58486/0 (socket says 172.21.15.82:58486) 2023-09-09T23:26:28.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.200+0000 7ff18cd26700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55adb7d21030 0x55adb7d36180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:28.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.200+0000 7ff18d527700 1 -- 172.21.15.82:0/3192857299 learned_addr learned my addr 172.21.15.82:0/3192857299 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:26:28.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.201+0000 7ff18d527700 1 -- 172.21.15.82:0/3192857299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55adb7d21030 msgr2=0x55adb7d36180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:26:28.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.201+0000 7ff18d527700 1 --2- 172.21.15.82:0/3192857299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55adb7d21030 0x55adb7d36180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.201+0000 7ff18d527700 1 -- 172.21.15.82:0/3192857299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55adb7d2ae40 msgr2=0x55adb7d22d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:26:28.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.201+0000 7ff18d527700 1 --2- 172.21.15.82:0/3192857299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55adb7d2ae40 0x55adb7d22d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.201+0000 7ff18d527700 1 -- 172.21.15.82:0/3192857299 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adb7ac5250 con 0x55adb7d20710 2023-09-09T23:26:28.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.201+0000 7ff18d527700 1 --2- 172.21.15.82:0/3192857299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adb7d20710 0x55adb7d31a30 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x55adb7d3d1c0 tx=0x7ff184004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d34bbc2cd7703c server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:26:28.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.201+0000 7ff173fff700 1 -- 172.21.15.82:0/3192857299 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff184016040 con 0x55adb7d20710 2023-09-09T23:26:28.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.201+0000 7ff173fff700 1 -- 172.21.15.82:0/3192857299 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff184018040 con 0x55adb7d20710 2023-09-09T23:26:28.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.201+0000 7ff173fff700 1 -- 172.21.15.82:0/3192857299 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff184011540 con 0x55adb7d20710 2023-09-09T23:26:28.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.202+0000 7ff19b951b80 1 -- 172.21.15.82:0/3192857299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adb7d20710 msgr2=0x55adb7d31a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:26:28.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.202+0000 7ff19b951b80 1 --2- 172.21.15.82:0/3192857299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adb7d20710 0x55adb7d31a30 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x55adb7d3d1c0 tx=0x7ff184004320 comp rx=0 tx=0).stop 2023-09-09T23:26:28.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.202+0000 7ff19b951b80 1 -- 172.21.15.82:0/3192857299 shutdown_connections 2023-09-09T23:26:28.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.202+0000 7ff19b951b80 1 --2- 172.21.15.82:0/3192857299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55adb7d2ae40 0x55adb7d22d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.202+0000 7ff19b951b80 1 --2- 172.21.15.82:0/3192857299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adb7d20710 0x55adb7d31a30 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.202+0000 7ff19b951b80 1 --2- 172.21.15.82:0/3192857299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55adb7d21030 0x55adb7d36180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.202+0000 7ff19b951b80 1 -- 172.21.15.82:0/3192857299 >> 172.21.15.82:0/3192857299 conn(0x55adb7c95090 msgr2=0x55adb7d36b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:26:28.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.202+0000 7ff19b951b80 1 -- 172.21.15.82:0/3192857299 shutdown_connections 2023-09-09T23:26:28.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.202+0000 7ff19b951b80 1 -- 172.21.15.82:0/3192857299 wait complete. 2023-09-09T23:26:28.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.202+0000 7ff19b951b80 1 -- start start 2023-09-09T23:26:28.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.202+0000 7ff19b951b80 1 Processor -- start 2023-09-09T23:26:28.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.202+0000 7ff19b951b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adb7d20710 0x55adb7d82670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:28.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.202+0000 7ff19b951b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55adb7d21030 0x55adb7d82bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:28.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.202+0000 7ff19b951b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55adb7d2ae40 0x55adb7d875b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:28.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.202+0000 7ff19b951b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb7d87cc0 con 0x55adb7d20710 2023-09-09T23:26:28.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.202+0000 7ff19b951b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb7d87e30 con 0x55adb7d2ae40 2023-09-09T23:26:28.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.202+0000 7ff19b951b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb7d87ff0 con 0x55adb7d21030 2023-09-09T23:26:28.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.202+0000 7ff19b951b80 10 client.0 _ll_register_callbacks cb 0x7ff178029eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:26:28.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.203+0000 7ff18d527700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adb7d20710 0x55adb7d82670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:28.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.203+0000 7ff18cd26700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55adb7d21030 0x55adb7d82bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:28.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.203+0000 7ff18d527700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adb7d20710 0x55adb7d82670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58492/0 (socket says 172.21.15.82:58492) 2023-09-09T23:26:28.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.203+0000 7ff18cd26700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55adb7d21030 0x55adb7d82bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.82:57008/0 (socket says 172.21.15.82:57008) 2023-09-09T23:26:28.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.203+0000 7ff18d527700 1 -- 172.21.15.82:0/3339732281 learned_addr learned my addr 172.21.15.82:0/3339732281 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:26:28.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.203+0000 7ff18dd28700 1 --2- 172.21.15.82:0/3339732281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55adb7d2ae40 0x55adb7d875b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:28.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.203+0000 7ff18d527700 1 -- 172.21.15.82:0/3339732281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55adb7d21030 msgr2=0x55adb7d82bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:26:28.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.203+0000 7ff18d527700 1 --2- 172.21.15.82:0/3339732281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55adb7d21030 0x55adb7d82bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.203+0000 7ff18d527700 1 -- 172.21.15.82:0/3339732281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55adb7d2ae40 msgr2=0x55adb7d875b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:26:28.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.203+0000 7ff18d527700 1 --2- 172.21.15.82:0/3339732281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55adb7d2ae40 0x55adb7d875b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.203+0000 7ff18d527700 1 -- 172.21.15.82:0/3339732281 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff184009dc0 con 0x55adb7d20710 2023-09-09T23:26:28.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.203+0000 7ff18d527700 1 --2- 172.21.15.82:0/3339732281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adb7d20710 0x55adb7d82670 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff18401d6e0 tx=0x7ff184003cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea06ccec20d7c268 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:26:28.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.203+0000 7ff170ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:26:28.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.203+0000 7ff170ff9700 1 -- 172.21.15.82:0/3339732281 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff184016070 con 0x55adb7d20710 2023-09-09T23:26:28.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.203+0000 7ff170ff9700 1 -- 172.21.15.82:0/3339732281 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8455+0+0 (secure 0 0 0) 0x7ff184003680 con 0x55adb7d20710 2023-09-09T23:26:28.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.203+0000 7ff19b951b80 1 -- 172.21.15.82:0/3339732281 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff18000f040 con 0x55adb7d20710 2023-09-09T23:26:28.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.203+0000 7ff170ff9700 1 -- 172.21.15.82:0/3339732281 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff184003840 con 0x55adb7d20710 2023-09-09T23:26:28.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.203+0000 7ff19b951b80 10 client.16374 fetch_fsmap learned FSMap version 127 2023-09-09T23:26:28.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.203+0000 7ff19b951b80 1 -- 172.21.15.82:0/3339732281 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x55adb7a9d2b0 con 0x55adb7d20710 2023-09-09T23:26:28.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.203+0000 7ff170ff9700 1 -- 172.21.15.82:0/3339732281 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff18000f040 con 0x55adb7d20710 2023-09-09T23:26:28.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.204+0000 7ff170ff9700 1 -- 172.21.15.82:0/3339732281 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 127) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff184005040 con 0x55adb7d20710 2023-09-09T23:26:28.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.204+0000 7ff19b951b80 10 client.16374 fetch_fsmap finished waiting for FSMap version 127 2023-09-09T23:26:28.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.204+0000 7ff19b951b80 10 client.16374 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:26:28.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.204+0000 7ff19b951b80 20 client.16374 populate_metadata read hostname 'smithi082' 2023-09-09T23:26:28.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.204+0000 7ff19b951b80 1 --2- 172.21.15.82:0/3339732281 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55adb7d8f760 0x55adb7d91b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:28.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.204+0000 7ff19b951b80 4 client.16374 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:26:28.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.204+0000 7ff19b951b80 1 -- 172.21.15.82:0/3339732281 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55adb7c6b920 con 0x55adb7d8f760 2023-09-09T23:26:28.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.204+0000 7ff18cd26700 1 --2- 172.21.15.82:0/3339732281 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55adb7d8f760 0x55adb7d91b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:28.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.204+0000 7ff170ff9700 10 client.16374 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:26:28.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.204+0000 7ff18cd26700 1 --2- 172.21.15.82:0/3339732281 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55adb7d8f760 0x55adb7d91b80 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6268b1440522766 server_cookie=deaf76f0a8992993 in_seq=0 out_seq=0 2023-09-09T23:26:28.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.204+0000 7ff170ff9700 1 -- 172.21.15.82:0/3339732281 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55adb7c6b920 con 0x55adb7d8f760 2023-09-09T23:26:28.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.204+0000 7ff170ff9700 10 client.16374 handle_command_reply: tid=0 2023-09-09T23:26:28.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.204+0000 7ff19b951b80 1 client.16374 shutdown 2023-09-09T23:26:28.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.204+0000 7ff19b951b80 1 -- 172.21.15.82:0/3339732281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adb7d20710 msgr2=0x55adb7d82670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:26:28.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.204+0000 7ff19b951b80 1 --2- 172.21.15.82:0/3339732281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adb7d20710 0x55adb7d82670 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff18401d6e0 tx=0x7ff184003cc0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.204+0000 7ff19b951b80 1 -- 172.21.15.82:0/3339732281 shutdown_connections 2023-09-09T23:26:28.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.204+0000 7ff19b951b80 1 --2- 172.21.15.82:0/3339732281 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55adb7d8f760 0x55adb7d91b80 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.204+0000 7ff19b951b80 1 --2- 172.21.15.82:0/3339732281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55adb7d2ae40 0x55adb7d875b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.204+0000 7ff19b951b80 1 --2- 172.21.15.82:0/3339732281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adb7d20710 0x55adb7d82670 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.204+0000 7ff19b951b80 1 --2- 172.21.15.82:0/3339732281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55adb7d21030 0x55adb7d82bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.204+0000 7ff170ff9700 0 client.16374 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:26:28.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.204+0000 7ff19b951b80 1 -- 172.21.15.82:0/3339732281 >> 172.21.15.82:0/3339732281 conn(0x55adb7c95090 msgr2=0x55adb7d36180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:26:28.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.204+0000 7ff19b951b80 1 -- 172.21.15.82:0/3339732281 shutdown_connections 2023-09-09T23:26:28.227 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.204+0000 7ff19b951b80 1 -- 172.21.15.82:0/3339732281 wait complete. 2023-09-09T23:26:28.227 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.205+0000 7ff19b951b80 20 client.16374 trim_cache size 0 max 16384 2023-09-09T23:26:28.227 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:26:28.227 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:26:28.228 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:26:28.228 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:26:28.228 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:26:28.228 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:26:28.228 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:26:28.229 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:26:28.229 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:26:28.229 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:26:28.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.205+0000 7ff18f78b700 1 -- 172.21.15.82:0/366803001 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7ff1740a06c0 msgr2=0x7ff1740a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:26:28.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.205+0000 7ff18f78b700 1 --2- 172.21.15.82:0/366803001 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7ff1740a06c0 0x7ff1740a2b80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff180000f80 tx=0x7ff1800040c0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.205+0000 7ff18f78b700 1 -- 172.21.15.82:0/366803001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff188103f80 msgr2=0x7ff188197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:26:28.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.205+0000 7ff18f78b700 1 --2- 172.21.15.82:0/366803001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff188103f80 0x7ff188197aa0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff178000c00 tx=0x7ff1780047b0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.205+0000 7ff18f78b700 1 -- 172.21.15.82:0/366803001 shutdown_connections 2023-09-09T23:26:28.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.205+0000 7ff18f78b700 1 --2- 172.21.15.82:0/366803001 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7ff1740a06c0 0x7ff1740a2b80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.205+0000 7ff18f78b700 1 --2- 172.21.15.82:0/366803001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff188198080 0x7ff1881a25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.205+0000 7ff18f78b700 1 --2- 172.21.15.82:0/366803001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1880ff040 0x7ff188197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.205+0000 7ff18f78b700 1 --2- 172.21.15.82:0/366803001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff188103f80 0x7ff188197aa0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:28.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.205+0000 7ff18f78b700 1 -- 172.21.15.82:0/366803001 >> 172.21.15.82:0/366803001 conn(0x7ff188075740 msgr2=0x7ff1880fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:26:28.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.205+0000 7ff18f78b700 1 -- 172.21.15.82:0/366803001 shutdown_connections 2023-09-09T23:26:28.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:28.205+0000 7ff18f78b700 1 -- 172.21.15.82:0/366803001 wait complete. 2023-09-09T23:26:28.233 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:26:28.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 420.023999 secs 2023-09-09T23:26:28.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 420.023999 secs 2023-09-09T23:26:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 420.023999 secs 2023-09-09T23:26:29.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:29 smithi170 ceph-mon[102832]: pgmap v394: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-09-09T23:26:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 420.753067 secs 2023-09-09T23:26:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:29 smithi082 ceph-mon[98694]: pgmap v394: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-09-09T23:26:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 420.753067 secs 2023-09-09T23:26:30.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:29 smithi098 ceph-mon[103124]: pgmap v394: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-09-09T23:26:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 420.753067 secs 2023-09-09T23:26:31.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:31 smithi170 ceph-mon[102832]: pgmap v395: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 14 op/s 2023-09-09T23:26:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:31 smithi082 ceph-mon[98694]: pgmap v395: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 14 op/s 2023-09-09T23:26:32.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:31 smithi098 ceph-mon[103124]: pgmap v395: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 14 op/s 2023-09-09T23:26:32.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 423.403008 secs 2023-09-09T23:26:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 423.403008 secs 2023-09-09T23:26:33.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 423.403008 secs 2023-09-09T23:26:33.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:33 smithi170 ceph-mon[102832]: pgmap v396: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.8 KiB/s wr, 10 op/s 2023-09-09T23:26:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 425.024062 secs 2023-09-09T23:26:33.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:33 smithi082 ceph-mon[98694]: pgmap v396: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.8 KiB/s wr, 10 op/s 2023-09-09T23:26:33.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 425.024062 secs 2023-09-09T23:26:34.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:33 smithi098 ceph-mon[103124]: pgmap v396: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.8 KiB/s wr, 10 op/s 2023-09-09T23:26:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 425.024062 secs 2023-09-09T23:26:34.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 425.753131 secs 2023-09-09T23:26:34.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 425.753131 secs 2023-09-09T23:26:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 425.753131 secs 2023-09-09T23:26:35.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:35 smithi170 ceph-mon[102832]: pgmap v397: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2023-09-09T23:26:35.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:35 smithi082 ceph-mon[98694]: pgmap v397: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2023-09-09T23:26:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:35 smithi098 ceph-mon[103124]: pgmap v397: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2023-09-09T23:26:37.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:37 smithi170 ceph-mon[102832]: pgmap v398: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2023-09-09T23:26:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 428.403081 secs 2023-09-09T23:26:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:37 smithi082 ceph-mon[98694]: pgmap v398: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2023-09-09T23:26:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 428.403081 secs 2023-09-09T23:26:38.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:37 smithi098 ceph-mon[103124]: pgmap v398: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2023-09-09T23:26:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 428.403081 secs 2023-09-09T23:26:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 430.024121 secs 2023-09-09T23:26:38.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 430.024121 secs 2023-09-09T23:26:39.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 430.024121 secs 2023-09-09T23:26:39.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:39 smithi170 ceph-mon[102832]: pgmap v399: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2023-09-09T23:26:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 430.753200 secs 2023-09-09T23:26:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:39 smithi082 ceph-mon[98694]: pgmap v399: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2023-09-09T23:26:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 430.753200 secs 2023-09-09T23:26:40.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:39 smithi098 ceph-mon[103124]: pgmap v399: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2023-09-09T23:26:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 430.753200 secs 2023-09-09T23:26:41.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:41 smithi170 ceph-mon[102832]: pgmap v400: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2023-09-09T23:26:41.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:41 smithi082 ceph-mon[98694]: pgmap v400: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2023-09-09T23:26:42.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:41 smithi098 ceph-mon[103124]: pgmap v400: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2023-09-09T23:26:42.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 433.403189 secs 2023-09-09T23:26:42.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 433.403189 secs 2023-09-09T23:26:43.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 433.403189 secs 2023-09-09T23:26:43.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:43 smithi170 ceph-mon[102832]: pgmap v401: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:26:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 435.024204 secs 2023-09-09T23:26:43.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:43 smithi082 ceph-mon[98694]: pgmap v401: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:26:43.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 435.024204 secs 2023-09-09T23:26:44.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:43 smithi098 ceph-mon[103124]: pgmap v401: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:26:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 435.024204 secs 2023-09-09T23:26:44.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 435.753266 secs 2023-09-09T23:26:44.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 435.753266 secs 2023-09-09T23:26:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 435.753266 secs 2023-09-09T23:26:45.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:45 smithi170 ceph-mon[102832]: pgmap v402: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:26:45.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:45 smithi082 ceph-mon[98694]: pgmap v402: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:26:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:45 smithi098 ceph-mon[103124]: pgmap v402: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:26:47.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:47 smithi170 ceph-mon[102832]: pgmap v403: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:26:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 438.403250 secs 2023-09-09T23:26:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:47 smithi082 ceph-mon[98694]: pgmap v403: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:26:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 438.403250 secs 2023-09-09T23:26:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:47 smithi098 ceph-mon[103124]: pgmap v403: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:26:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 438.403250 secs 2023-09-09T23:26:48.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 440.024301 secs 2023-09-09T23:26:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:48 smithi170 ceph-mon[102832]: pgmap v404: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:26:48.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 440.024301 secs 2023-09-09T23:26:48.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:48 smithi082 ceph-mon[98694]: pgmap v404: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:26:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 440.024301 secs 2023-09-09T23:26:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:48 smithi098 ceph-mon[103124]: pgmap v404: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:26:49.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 440.753335 secs 2023-09-09T23:26:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 440.753335 secs 2023-09-09T23:26:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 440.753335 secs 2023-09-09T23:26:50.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:50 smithi170 ceph-mon[102832]: pgmap v405: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-09T23:26:50.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:50 smithi082 ceph-mon[98694]: pgmap v405: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-09T23:26:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:50 smithi098 ceph-mon[103124]: pgmap v405: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-09T23:26:52.176 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:51 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 443.403367 secs 2023-09-09T23:26:52.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:51 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 443.403367 secs 2023-09-09T23:26:52.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:51 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 443.403367 secs 2023-09-09T23:26:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:52 smithi170 ceph-mon[102832]: pgmap v406: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-09T23:26:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:52 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 445.024361 secs 2023-09-09T23:26:53.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:52 smithi082 ceph-mon[98694]: pgmap v406: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-09T23:26:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:52 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 445.024361 secs 2023-09-09T23:26:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:52 smithi098 ceph-mon[103124]: pgmap v406: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-09T23:26:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:52 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 445.024361 secs 2023-09-09T23:26:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 445.753403 secs 2023-09-09T23:26:54.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 445.753403 secs 2023-09-09T23:26:54.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 445.753403 secs 2023-09-09T23:26:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:55 smithi098 ceph-mon[103124]: pgmap v407: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:26:55.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:55 smithi170 ceph-mon[102832]: pgmap v407: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:26:55.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:55 smithi082 ceph-mon[98694]: pgmap v407: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:26:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:57 smithi098 ceph-mon[103124]: pgmap v408: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-09T23:26:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 448.403476 secs 2023-09-09T23:26:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:57 smithi170 ceph-mon[102832]: pgmap v408: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-09T23:26:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 448.403476 secs 2023-09-09T23:26:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:57 smithi082 ceph-mon[98694]: pgmap v408: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-09T23:26:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 448.403476 secs 2023-09-09T23:26:58.234 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:26:58.374 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.373+0000 7f762c2ac700 1 -- 172.21.15.82:0/2967476518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76240ffef0 msgr2=0x7f7624100370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:26:58.375 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.373+0000 7f762c2ac700 1 --2- 172.21.15.82:0/2967476518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76240ffef0 0x7f7624100370 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f76140099c0 tx=0x7f7614009cd0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.375 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.373+0000 7f762c2ac700 1 -- 172.21.15.82:0/2967476518 shutdown_connections 2023-09-09T23:26:58.375 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.373+0000 7f762c2ac700 1 --2- 172.21.15.82:0/2967476518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7624105560 0x7f7624105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.373+0000 7f762c2ac700 1 --2- 172.21.15.82:0/2967476518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76241008b0 0x7f762410c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.373+0000 7f762c2ac700 1 --2- 172.21.15.82:0/2967476518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76240ffef0 0x7f7624100370 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.373+0000 7f762c2ac700 1 -- 172.21.15.82:0/2967476518 >> 172.21.15.82:0/2967476518 conn(0x7f76240747a0 msgr2=0x7f7624074bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:26:58.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.373+0000 7f762c2ac700 1 -- 172.21.15.82:0/2967476518 shutdown_connections 2023-09-09T23:26:58.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.373+0000 7f762c2ac700 1 -- 172.21.15.82:0/2967476518 wait complete. 2023-09-09T23:26:58.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.374+0000 7f762c2ac700 1 Processor -- start 2023-09-09T23:26:58.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.374+0000 7f762c2ac700 1 -- start start 2023-09-09T23:26:58.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.374+0000 7f762c2ac700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76240ffef0 0x7f76241955c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:58.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.374+0000 7f762c2ac700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76241008b0 0x7f7624195b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:58.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.374+0000 7f762c2ac700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7624105560 0x7f7624199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:58.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.374+0000 7f762c2ac700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762419a5a0 con 0x7f7624105560 2023-09-09T23:26:58.380 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.374+0000 7f762c2ac700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762419a710 con 0x7f76240ffef0 2023-09-09T23:26:58.380 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.374+0000 7f762c2ac700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762419a8d0 con 0x7f76241008b0 2023-09-09T23:26:58.380 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.374+0000 7f762a048700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76240ffef0 0x7f76241955c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:58.381 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.374+0000 7f762a849700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7624105560 0x7f7624199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:58.381 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.374+0000 7f762a048700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76240ffef0 0x7f76241955c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:44070/0 (socket says 172.21.15.82:44070) 2023-09-09T23:26:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.374+0000 7f762a048700 1 -- 172.21.15.82:0/3006262710 learned_addr learned my addr 172.21.15.82:0/3006262710 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:26:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.374+0000 7f7629847700 1 --2- 172.21.15.82:0/3006262710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76241008b0 0x7f7624195b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.375+0000 7f762a849700 1 -- 172.21.15.82:0/3006262710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76241008b0 msgr2=0x7f7624195b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:26:58.383 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.375+0000 7f762a849700 1 --2- 172.21.15.82:0/3006262710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76241008b0 0x7f7624195b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.383 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.375+0000 7f762a849700 1 -- 172.21.15.82:0/3006262710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76240ffef0 msgr2=0x7f76241955c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:26:58.383 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.375+0000 7f762a849700 1 --2- 172.21.15.82:0/3006262710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76240ffef0 0x7f76241955c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.384 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.375+0000 7f762a849700 1 -- 172.21.15.82:0/3006262710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76140095f0 con 0x7f7624105560 2023-09-09T23:26:58.384 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.375+0000 7f762a849700 1 --2- 172.21.15.82:0/3006262710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7624105560 0x7f7624199f60 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f761c00efc0 tx=0x7f761c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:26:58.384 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.375+0000 7f761b7fe700 1 -- 172.21.15.82:0/3006262710 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761c018070 con 0x7f7624105560 2023-09-09T23:26:58.384 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.375+0000 7f762c2ac700 1 -- 172.21.15.82:0/3006262710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f762419ab50 con 0x7f7624105560 2023-09-09T23:26:58.385 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.375+0000 7f761b7fe700 1 -- 172.21.15.82:0/3006262710 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f761c00f040 con 0x7f7624105560 2023-09-09T23:26:58.385 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.375+0000 7f761b7fe700 1 -- 172.21.15.82:0/3006262710 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761c00cd40 con 0x7f7624105560 2023-09-09T23:26:58.385 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.375+0000 7f762c2ac700 1 -- 172.21.15.82:0/3006262710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76241a23f0 con 0x7f7624105560 2023-09-09T23:26:58.391 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.377+0000 7f761b7fe700 1 -- 172.21.15.82:0/3006262710 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f761c003a00 con 0x7f7624105560 2023-09-09T23:26:58.391 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.378+0000 7f761b7fe700 1 --2- 172.21.15.82:0/3006262710 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f76100a07b0 0x7f76100a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:58.391 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.378+0000 7f762a048700 1 --2- 172.21.15.82:0/3006262710 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f76100a07b0 0x7f76100a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:58.392 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.378+0000 7f761b7fe700 1 -- 172.21.15.82:0/3006262710 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f761c0cbb90 con 0x7f7624105560 2023-09-09T23:26:58.392 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.378+0000 7f762a048700 1 --2- 172.21.15.82:0/3006262710 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f76100a07b0 0x7f76100a2c70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7620009ca0 tx=0x7f7620009260 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:26:58.392 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.379+0000 7f7638472b80 1 Processor -- start 2023-09-09T23:26:58.393 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.379+0000 7f7638472b80 1 -- start start 2023-09-09T23:26:58.393 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.379+0000 7f7638472b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563916b4a710 0x563916b4aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:58.393 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.379+0000 7f7638472b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563916b4b030 0x563916b54900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:58.393 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.379+0000 7f7638472b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563916b54e40 0x563916b57230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:58.393 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.379+0000 7f7638472b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563916aa3730 con 0x563916b54e40 2023-09-09T23:26:58.394 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.379+0000 7f7638472b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563916aa1050 con 0x563916b4a710 2023-09-09T23:26:58.394 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.379+0000 7f7638472b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563916a9c940 con 0x563916b4b030 2023-09-09T23:26:58.394 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.379+0000 7f762a048700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563916b4a710 0x563916b4aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:58.394 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.379+0000 7f7629847700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563916b4b030 0x563916b54900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:58.395 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.379+0000 7f762a048700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563916b4a710 0x563916b4aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:44078/0 (socket says 172.21.15.82:44078) 2023-09-09T23:26:58.395 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.379+0000 7f762a849700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563916b54e40 0x563916b57230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:58.395 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.379+0000 7f762a048700 1 -- 172.21.15.82:0/4177811989 learned_addr learned my addr 172.21.15.82:0/4177811989 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:26:58.395 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.379+0000 7f762a849700 1 -- 172.21.15.82:0/4177811989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563916b4b030 msgr2=0x563916b54900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:26:58.396 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.379+0000 7f762a849700 1 --2- 172.21.15.82:0/4177811989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563916b4b030 0x563916b54900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.396 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.379+0000 7f762a849700 1 -- 172.21.15.82:0/4177811989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563916b4a710 msgr2=0x563916b4aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:26:58.396 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.379+0000 7f762a849700 1 --2- 172.21.15.82:0/4177811989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563916b4a710 0x563916b4aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.396 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.379+0000 7f762a849700 1 -- 172.21.15.82:0/4177811989 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639168c8de0 con 0x563916b54e40 2023-09-09T23:26:58.397 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.380+0000 7f762a849700 1 --2- 172.21.15.82:0/4177811989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563916b54e40 0x563916b57230 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f761c0c92c0 tx=0x7f761c098f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87001daa1538e729 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:26:58.397 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.380+0000 7f7618d89700 1 -- 172.21.15.82:0/4177811989 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761c004ad0 con 0x563916b54e40 2023-09-09T23:26:58.397 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.380+0000 7f7618d89700 1 -- 172.21.15.82:0/4177811989 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f761c0997b0 con 0x563916b54e40 2023-09-09T23:26:58.397 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.380+0000 7f7618d89700 1 -- 172.21.15.82:0/4177811989 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761c01b560 con 0x563916b54e40 2023-09-09T23:26:58.398 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.380+0000 7f7638472b80 1 -- 172.21.15.82:0/4177811989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563916b54e40 msgr2=0x563916b57230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:26:58.398 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.380+0000 7f7638472b80 1 --2- 172.21.15.82:0/4177811989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563916b54e40 0x563916b57230 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f761c0c92c0 tx=0x7f761c098f80 comp rx=0 tx=0).stop 2023-09-09T23:26:58.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.380+0000 7f7638472b80 1 -- 172.21.15.82:0/4177811989 shutdown_connections 2023-09-09T23:26:58.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.380+0000 7f7638472b80 1 --2- 172.21.15.82:0/4177811989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563916b4a710 0x563916b4aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.380+0000 7f7638472b80 1 --2- 172.21.15.82:0/4177811989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563916b54e40 0x563916b57230 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.380+0000 7f7638472b80 1 --2- 172.21.15.82:0/4177811989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563916b4b030 0x563916b54900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.380+0000 7f7638472b80 1 -- 172.21.15.82:0/4177811989 >> 172.21.15.82:0/4177811989 conn(0x563916abf090 msgr2=0x563916b45bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:26:58.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.381+0000 7f7638472b80 1 -- 172.21.15.82:0/4177811989 shutdown_connections 2023-09-09T23:26:58.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.381+0000 7f7638472b80 1 -- 172.21.15.82:0/4177811989 wait complete. 2023-09-09T23:26:58.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.381+0000 7f7638472b80 1 -- start start 2023-09-09T23:26:58.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.381+0000 7f7638472b80 1 Processor -- start 2023-09-09T23:26:58.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.381+0000 7f7638472b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563916b4a710 0x563916b548f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:58.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.381+0000 7f7638472b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563916b4b030 0x563916b51b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:58.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.381+0000 7f7638472b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563916b54e40 0x563916b52240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:58.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.381+0000 7f7638472b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563916a3ab70 con 0x563916b54e40 2023-09-09T23:26:58.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.381+0000 7f7638472b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563916a3a870 con 0x563916b4a710 2023-09-09T23:26:58.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.381+0000 7f7638472b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56391691e920 con 0x563916b4b030 2023-09-09T23:26:58.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.381+0000 7f7638472b80 10 client.0 _ll_register_callbacks cb 0x563916984a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:26:58.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.381+0000 7f762a849700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563916b54e40 0x563916b52240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:58.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.381+0000 7f762a849700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563916b54e40 0x563916b52240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58524/0 (socket says 172.21.15.82:58524) 2023-09-09T23:26:58.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.381+0000 7f762a849700 1 -- 172.21.15.82:0/3140833114 learned_addr learned my addr 172.21.15.82:0/3140833114 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:26:58.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.381+0000 7f762a048700 1 --2- 172.21.15.82:0/3140833114 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563916b4a710 0x563916b548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:58.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.381+0000 7f7629847700 1 --2- 172.21.15.82:0/3140833114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563916b4b030 0x563916b51b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:58.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.382+0000 7f762a849700 1 -- 172.21.15.82:0/3140833114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563916b4b030 msgr2=0x563916b51b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:26:58.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.382+0000 7f762a849700 1 --2- 172.21.15.82:0/3140833114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563916b4b030 0x563916b51b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.382+0000 7f762a849700 1 -- 172.21.15.82:0/3140833114 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563916b4a710 msgr2=0x563916b548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:26:58.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.382+0000 7f762a849700 1 --2- 172.21.15.82:0/3140833114 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563916b4a710 0x563916b548f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.382+0000 7f762a849700 1 -- 172.21.15.82:0/3140833114 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76241a23f0 con 0x563916b54e40 2023-09-09T23:26:58.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.382+0000 7f762a849700 1 --2- 172.21.15.82:0/3140833114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563916b54e40 0x563916b52240 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f761c099040 tx=0x7f761c0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dafd2fc3dca48bc6 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:26:58.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.382+0000 7f760dffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:26:58.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.382+0000 7f760dffb700 1 -- 172.21.15.82:0/3140833114 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761c0990b0 con 0x563916b54e40 2023-09-09T23:26:58.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.382+0000 7f7638472b80 1 -- 172.21.15.82:0/3140833114 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639169f44f0 con 0x563916b54e40 2023-09-09T23:26:58.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.382+0000 7f760dffb700 1 -- 172.21.15.82:0/3140833114 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f761c09cee0 con 0x563916b54e40 2023-09-09T23:26:58.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.382+0000 7f760dffb700 1 -- 172.21.15.82:0/3140833114 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7f761c0a1090 con 0x563916b54e40 2023-09-09T23:26:58.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.382+0000 7f7638472b80 10 client.16386 fetch_fsmap learned FSMap version 127 2023-09-09T23:26:58.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.382+0000 7f7638472b80 1 -- 172.21.15.82:0/3140833114 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x5639168b1c60 con 0x563916b54e40 2023-09-09T23:26:58.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.382+0000 7f760dffb700 1 -- 172.21.15.82:0/3140833114 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5639169f44f0 con 0x563916b54e40 2023-09-09T23:26:58.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.382+0000 7f760dffb700 1 -- 172.21.15.82:0/3140833114 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 127) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f761c0a0c00 con 0x563916b54e40 2023-09-09T23:26:58.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.382+0000 7f7638472b80 10 client.16386 fetch_fsmap finished waiting for FSMap version 127 2023-09-09T23:26:58.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.382+0000 7f7638472b80 10 client.16386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:26:58.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.382+0000 7f7638472b80 20 client.16386 populate_metadata read hostname 'smithi082' 2023-09-09T23:26:58.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.382+0000 7f7638472b80 1 --2- 172.21.15.82:0/3140833114 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x563916b644e0 0x563916b668d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:58.407 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.382+0000 7f7638472b80 4 client.16386 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:26:58.407 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.382+0000 7f7638472b80 1 -- 172.21.15.82:0/3140833114 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563916ab18a0 con 0x563916b644e0 2023-09-09T23:26:58.407 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.383+0000 7f762a048700 1 --2- 172.21.15.82:0/3140833114 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x563916b644e0 0x563916b668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:58.407 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.383+0000 7f762a048700 1 --2- 172.21.15.82:0/3140833114 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x563916b644e0 0x563916b668d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85f59f2969749373 server_cookie=58c6434e1dc201da in_seq=0 out_seq=0 2023-09-09T23:26:58.407 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.383+0000 7f760dffb700 10 client.16386 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:26:58.407 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.384+0000 7f760dffb700 1 -- 172.21.15.82:0/3140833114 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563916ab18a0 con 0x563916b644e0 2023-09-09T23:26:58.408 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.384+0000 7f760dffb700 10 client.16386 handle_command_reply: tid=0 2023-09-09T23:26:58.408 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.384+0000 7f7638472b80 1 client.16386 shutdown 2023-09-09T23:26:58.408 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.384+0000 7f7638472b80 1 -- 172.21.15.82:0/3140833114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563916b54e40 msgr2=0x563916b52240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:26:58.408 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.384+0000 7f7638472b80 1 --2- 172.21.15.82:0/3140833114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563916b54e40 0x563916b52240 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f761c099040 tx=0x7f761c0092e0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.408 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.384+0000 7f7638472b80 1 -- 172.21.15.82:0/3140833114 shutdown_connections 2023-09-09T23:26:58.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.384+0000 7f7638472b80 1 --2- 172.21.15.82:0/3140833114 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x563916b644e0 0x563916b668d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.384+0000 7f7638472b80 1 --2- 172.21.15.82:0/3140833114 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563916b4a710 0x563916b548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.384+0000 7f760dffb700 0 client.16386 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:26:58.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.384+0000 7f7638472b80 1 --2- 172.21.15.82:0/3140833114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563916b54e40 0x563916b52240 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.410 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.384+0000 7f7638472b80 1 --2- 172.21.15.82:0/3140833114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563916b4b030 0x563916b51b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.410 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.384+0000 7f7638472b80 1 -- 172.21.15.82:0/3140833114 >> 172.21.15.82:0/3140833114 conn(0x563916abf090 msgr2=0x563916b457e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:26:58.410 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.384+0000 7f7638472b80 1 -- 172.21.15.82:0/3140833114 shutdown_connections 2023-09-09T23:26:58.410 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.384+0000 7f7638472b80 1 -- 172.21.15.82:0/3140833114 wait complete. 2023-09-09T23:26:58.411 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.384+0000 7f7638472b80 20 client.16386 trim_cache size 0 max 16384 2023-09-09T23:26:58.415 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.400+0000 7f7638472b80 1 Processor -- start 2023-09-09T23:26:58.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.400+0000 7f7638472b80 1 -- start start 2023-09-09T23:26:58.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.401+0000 7f7638472b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563916b4a710 0x563916b5b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:58.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.401+0000 7f7638472b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563916b4b030 0x563916b456f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:58.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.401+0000 7f7638472b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563916b54e40 0x563916b4ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:58.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.401+0000 7f7638472b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563916b61380 con 0x563916b4b030 2023-09-09T23:26:58.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.401+0000 7f7638472b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563916b66f00 con 0x563916b4a710 2023-09-09T23:26:58.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.401+0000 7f7638472b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563916b5bf30 con 0x563916b54e40 2023-09-09T23:26:58.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.401+0000 7f7629847700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563916b4b030 0x563916b456f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:58.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.401+0000 7f7629847700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563916b4b030 0x563916b456f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58530/0 (socket says 172.21.15.82:58530) 2023-09-09T23:26:58.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.401+0000 7f7629847700 1 -- 172.21.15.82:0/1188823086 learned_addr learned my addr 172.21.15.82:0/1188823086 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:26:58.418 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.401+0000 7f762a048700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563916b4a710 0x563916b5b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:58.418 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.401+0000 7f762a849700 1 --2- 172.21.15.82:0/1188823086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563916b54e40 0x563916b4ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:58.418 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.401+0000 7f7629847700 1 -- 172.21.15.82:0/1188823086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563916b54e40 msgr2=0x563916b4ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:26:58.418 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.401+0000 7f7629847700 1 --2- 172.21.15.82:0/1188823086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563916b54e40 0x563916b4ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.418 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.401+0000 7f7629847700 1 -- 172.21.15.82:0/1188823086 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563916b4a710 msgr2=0x563916b5b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:26:58.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.401+0000 7f7629847700 1 --2- 172.21.15.82:0/1188823086 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563916b4a710 0x563916b5b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.401+0000 7f7629847700 1 -- 172.21.15.82:0/1188823086 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639168ef250 con 0x563916b4b030 2023-09-09T23:26:58.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.401+0000 7f7629847700 1 --2- 172.21.15.82:0/1188823086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563916b4b030 0x563916b456f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f76140099c0 tx=0x7f7614005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2502f84cb7eb94e0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:26:58.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.401+0000 7f761958a700 1 -- 172.21.15.82:0/1188823086 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7614005460 con 0x563916b4b030 2023-09-09T23:26:58.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.401+0000 7f761958a700 1 -- 172.21.15.82:0/1188823086 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f761401e040 con 0x563916b4b030 2023-09-09T23:26:58.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.401+0000 7f761958a700 1 -- 172.21.15.82:0/1188823086 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761400bcf0 con 0x563916b4b030 2023-09-09T23:26:58.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.402+0000 7f7638472b80 1 -- 172.21.15.82:0/1188823086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563916b4b030 msgr2=0x563916b456f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:26:58.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.402+0000 7f7638472b80 1 --2- 172.21.15.82:0/1188823086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563916b4b030 0x563916b456f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f76140099c0 tx=0x7f7614005040 comp rx=0 tx=0).stop 2023-09-09T23:26:58.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.402+0000 7f7638472b80 1 -- 172.21.15.82:0/1188823086 shutdown_connections 2023-09-09T23:26:58.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.402+0000 7f7638472b80 1 --2- 172.21.15.82:0/1188823086 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563916b4a710 0x563916b5b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.402+0000 7f7638472b80 1 --2- 172.21.15.82:0/1188823086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563916b54e40 0x563916b4ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.402+0000 7f7638472b80 1 --2- 172.21.15.82:0/1188823086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563916b4b030 0x563916b456f0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.402+0000 7f7638472b80 1 -- 172.21.15.82:0/1188823086 >> 172.21.15.82:0/1188823086 conn(0x563916abf090 msgr2=0x563916b60140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:26:58.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.402+0000 7f7638472b80 1 -- 172.21.15.82:0/1188823086 shutdown_connections 2023-09-09T23:26:58.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.402+0000 7f7638472b80 1 -- 172.21.15.82:0/1188823086 wait complete. 2023-09-09T23:26:58.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.402+0000 7f7638472b80 1 -- start start 2023-09-09T23:26:58.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.402+0000 7f7638472b80 1 Processor -- start 2023-09-09T23:26:58.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.402+0000 7f7638472b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563916b4a710 0x563916b5a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:58.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.402+0000 7f7638472b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563916b4b030 0x563916b5ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:58.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.402+0000 7f7638472b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563916b54e40 0x563916bb4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:58.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.402+0000 7f7638472b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563916bb4970 con 0x563916b54e40 2023-09-09T23:26:58.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.402+0000 7f7638472b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x563916bb4ae0 con 0x563916b4a710 2023-09-09T23:26:58.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.402+0000 7f7638472b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563916bb4ca0 con 0x563916b4b030 2023-09-09T23:26:58.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.403+0000 7f7638472b80 10 client.0 _ll_register_callbacks cb 0x7f7620009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:26:58.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.403+0000 7f762a849700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563916b54e40 0x563916bb4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:58.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.403+0000 7f762a048700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563916b4a710 0x563916b5a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:58.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.403+0000 7f762a849700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563916b54e40 0x563916bb4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58538/0 (socket says 172.21.15.82:58538) 2023-09-09T23:26:58.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.403+0000 7f762a849700 1 -- 172.21.15.82:0/1279564959 learned_addr learned my addr 172.21.15.82:0/1279564959 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:26:58.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.403+0000 7f7629847700 1 --2- 172.21.15.82:0/1279564959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563916b4b030 0x563916b5ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:58.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.403+0000 7f762a849700 1 -- 172.21.15.82:0/1279564959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563916b4b030 msgr2=0x563916b5ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:26:58.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.403+0000 7f762a849700 1 --2- 172.21.15.82:0/1279564959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563916b4b030 0x563916b5ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.403+0000 7f762a849700 1 -- 172.21.15.82:0/1279564959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563916b4a710 msgr2=0x563916b5a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:26:58.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.403+0000 7f762a849700 1 --2- 172.21.15.82:0/1279564959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563916b4a710 0x563916b5a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.403+0000 7f762a849700 1 -- 172.21.15.82:0/1279564959 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f762410cc20 con 0x563916b54e40 2023-09-09T23:26:58.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.403+0000 7f7629847700 1 --2- 172.21.15.82:0/1279564959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563916b4b030 0x563916b5ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-09T23:26:58.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.403+0000 7f762a849700 1 --2- 172.21.15.82:0/1279564959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563916b54e40 0x563916bb4430 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f761c0a0f80 tx=0x7f761c097660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=296923b4c5aba5c server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:26:58.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.403+0000 7f760dffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:26:58.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.403+0000 7f760dffb700 1 -- 172.21.15.82:0/1279564959 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761c098bf0 con 0x563916b54e40 2023-09-09T23:26:58.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.403+0000 7f760dffb700 1 -- 172.21.15.82:0/1279564959 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f761c09cee0 con 0x563916b54e40 2023-09-09T23:26:58.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.403+0000 7f7638472b80 1 -- 172.21.15.82:0/1279564959 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f761c0997b0 con 0x563916b54e40 2023-09-09T23:26:58.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.403+0000 7f760dffb700 1 -- 172.21.15.82:0/1279564959 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7f761c0a1090 con 0x563916b54e40 2023-09-09T23:26:58.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.403+0000 7f7638472b80 10 client.16392 fetch_fsmap learned FSMap version 127 2023-09-09T23:26:58.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.403+0000 7f7638472b80 1 -- 172.21.15.82:0/1279564959 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x5639168c72b0 con 0x563916b54e40 2023-09-09T23:26:58.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.403+0000 7f760dffb700 1 -- 172.21.15.82:0/1279564959 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f761c0a7080 con 0x563916b54e40 2023-09-09T23:26:58.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.404+0000 7f760dffb700 1 -- 172.21.15.82:0/1279564959 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 127) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f761c0981f0 con 0x563916b54e40 2023-09-09T23:26:58.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.404+0000 7f7638472b80 10 client.16392 fetch_fsmap finished waiting for FSMap version 127 2023-09-09T23:26:58.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.404+0000 7f7638472b80 10 client.16392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:26:58.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.404+0000 7f7638472b80 20 client.16392 populate_metadata read hostname 'smithi082' 2023-09-09T23:26:58.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.404+0000 7f7638472b80 1 --2- 172.21.15.82:0/1279564959 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x563916bbbf50 0x563916bbe340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:26:58.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.404+0000 7f7638472b80 4 client.16392 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:26:58.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.404+0000 7f7638472b80 1 -- 172.21.15.82:0/1279564959 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563916a95920 con 0x563916bbbf50 2023-09-09T23:26:58.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.404+0000 7f762a048700 1 --2- 172.21.15.82:0/1279564959 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x563916bbbf50 0x563916bbe340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:26:58.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.404+0000 7f760dffb700 10 client.16392 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:26:58.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.404+0000 7f762a048700 1 --2- 172.21.15.82:0/1279564959 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x563916bbbf50 0x563916bbe340 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1683f518089079af server_cookie=46e8bb0c16fe84cd in_seq=0 out_seq=0 2023-09-09T23:26:58.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.404+0000 7f760dffb700 1 -- 172.21.15.82:0/1279564959 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x563916a95920 con 0x563916bbbf50 2023-09-09T23:26:58.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.404+0000 7f760dffb700 10 client.16392 handle_command_reply: tid=0 2023-09-09T23:26:58.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.404+0000 7f7638472b80 1 client.16392 shutdown 2023-09-09T23:26:58.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.404+0000 7f7638472b80 1 -- 172.21.15.82:0/1279564959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563916b54e40 msgr2=0x563916bb4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:26:58.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.404+0000 7f7638472b80 1 --2- 172.21.15.82:0/1279564959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563916b54e40 0x563916bb4430 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f761c0a0f80 tx=0x7f761c097660 comp rx=0 tx=0).stop 2023-09-09T23:26:58.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.404+0000 7f7638472b80 1 -- 172.21.15.82:0/1279564959 shutdown_connections 2023-09-09T23:26:58.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.404+0000 7f7638472b80 1 --2- 172.21.15.82:0/1279564959 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x563916bbbf50 0x563916bbe340 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.404+0000 7f7638472b80 1 --2- 172.21.15.82:0/1279564959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563916b4a710 0x563916b5a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.404+0000 7f7638472b80 1 --2- 172.21.15.82:0/1279564959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563916b54e40 0x563916bb4430 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.404+0000 7f760dffb700 0 client.16392 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:26:58.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.404+0000 7f7638472b80 1 --2- 172.21.15.82:0/1279564959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563916b4b030 0x563916b5ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.404+0000 7f7638472b80 1 -- 172.21.15.82:0/1279564959 >> 172.21.15.82:0/1279564959 conn(0x563916abf090 msgr2=0x563916b45670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:26:58.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.405+0000 7f7638472b80 1 -- 172.21.15.82:0/1279564959 shutdown_connections 2023-09-09T23:26:58.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.405+0000 7f7638472b80 1 -- 172.21.15.82:0/1279564959 wait complete. 2023-09-09T23:26:58.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.405+0000 7f7638472b80 20 client.16392 trim_cache size 0 max 16384 2023-09-09T23:26:58.438 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:26:58.438 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:26:58.438 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:26:58.438 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:26:58.438 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:26:58.439 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:26:58.439 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:26:58.439 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:26:58.439 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:26:58.439 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:26:58.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.405+0000 7f762c2ac700 1 -- 172.21.15.82:0/3006262710 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f76100a07b0 msgr2=0x7f76100a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:26:58.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.405+0000 7f762c2ac700 1 --2- 172.21.15.82:0/3006262710 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f76100a07b0 0x7f76100a2c70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7620009ca0 tx=0x7f7620009260 comp rx=0 tx=0).stop 2023-09-09T23:26:58.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.405+0000 7f762c2ac700 1 -- 172.21.15.82:0/3006262710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7624105560 msgr2=0x7f7624199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:26:58.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.405+0000 7f762c2ac700 1 --2- 172.21.15.82:0/3006262710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7624105560 0x7f7624199f60 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f761c00efc0 tx=0x7f761c00c3a0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.405+0000 7f762c2ac700 1 -- 172.21.15.82:0/3006262710 shutdown_connections 2023-09-09T23:26:58.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.405+0000 7f762c2ac700 1 --2- 172.21.15.82:0/3006262710 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f76100a07b0 0x7f76100a2c70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.405+0000 7f762c2ac700 1 --2- 172.21.15.82:0/3006262710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76240ffef0 0x7f76241955c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.405+0000 7f762c2ac700 1 --2- 172.21.15.82:0/3006262710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7624105560 0x7f7624199f60 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.405+0000 7f762c2ac700 1 --2- 172.21.15.82:0/3006262710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76241008b0 0x7f7624195b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:26:58.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.405+0000 7f762c2ac700 1 -- 172.21.15.82:0/3006262710 >> 172.21.15.82:0/3006262710 conn(0x7f76240747a0 msgr2=0x7f76240fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:26:58.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.405+0000 7f762c2ac700 1 -- 172.21.15.82:0/3006262710 shutdown_connections 2023-09-09T23:26:58.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:26:58.405+0000 7f762c2ac700 1 -- 172.21.15.82:0/3006262710 wait complete. 2023-09-09T23:26:58.443 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:26:58.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 450.024438 secs 2023-09-09T23:26:58.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 450.024438 secs 2023-09-09T23:26:59.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 450.024438 secs 2023-09-09T23:26:59.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:59 smithi170 ceph-mon[102832]: pgmap v409: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:26:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:26:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 450.753475 secs 2023-09-09T23:26:59.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:59 smithi082 ceph-mon[98694]: pgmap v409: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:26:59.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:26:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 450.753475 secs 2023-09-09T23:27:00.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:59 smithi098 ceph-mon[103124]: pgmap v409: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:27:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:26:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 450.753475 secs 2023-09-09T23:27:01.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:01 smithi170 ceph-mon[102832]: pgmap v410: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-09T23:27:01.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:01 smithi082 ceph-mon[98694]: pgmap v410: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-09T23:27:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:01 smithi098 ceph-mon[103124]: pgmap v410: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-09T23:27:02.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 453.403536 secs 2023-09-09T23:27:02.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 453.403536 secs 2023-09-09T23:27:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 453.403536 secs 2023-09-09T23:27:03.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:03 smithi170 ceph-mon[102832]: pgmap v411: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 21 op/s 2023-09-09T23:27:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 455.024497 secs 2023-09-09T23:27:03.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:03 smithi082 ceph-mon[98694]: pgmap v411: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 21 op/s 2023-09-09T23:27:03.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 455.024497 secs 2023-09-09T23:27:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:03 smithi098 ceph-mon[103124]: pgmap v411: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 21 op/s 2023-09-09T23:27:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 455.024497 secs 2023-09-09T23:27:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 455.753488 secs 2023-09-09T23:27:04.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 455.753488 secs 2023-09-09T23:27:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 455.753488 secs 2023-09-09T23:27:05.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:05 smithi170 ceph-mon[102832]: pgmap v412: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 20 op/s 2023-09-09T23:27:05.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:05 smithi082 ceph-mon[98694]: pgmap v412: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 20 op/s 2023-09-09T23:27:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:05 smithi098 ceph-mon[103124]: pgmap v412: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 20 op/s 2023-09-09T23:27:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:07 smithi170 ceph-mon[102832]: pgmap v413: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 27 op/s 2023-09-09T23:27:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 458.403621 secs 2023-09-09T23:27:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:07 smithi082 ceph-mon[98694]: pgmap v413: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 27 op/s 2023-09-09T23:27:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 458.403621 secs 2023-09-09T23:27:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:07 smithi098 ceph-mon[103124]: pgmap v413: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 27 op/s 2023-09-09T23:27:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 458.403621 secs 2023-09-09T23:27:08.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:08 smithi170 ceph-mon[102832]: osdmap e99: 12 total, 12 up, 12 in 2023-09-09T23:27:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 460.024556 secs 2023-09-09T23:27:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:08 smithi082 ceph-mon[98694]: osdmap e99: 12 total, 12 up, 12 in 2023-09-09T23:27:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 460.024556 secs 2023-09-09T23:27:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:08 smithi098 ceph-mon[103124]: osdmap e99: 12 total, 12 up, 12 in 2023-09-09T23:27:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 460.024556 secs 2023-09-09T23:27:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:09 smithi170 ceph-mon[102832]: pgmap v415: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:27:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 460.753542 secs 2023-09-09T23:27:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:09 smithi082 ceph-mon[98694]: pgmap v415: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:27:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 460.753542 secs 2023-09-09T23:27:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:09 smithi098 ceph-mon[103124]: pgmap v415: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:27:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 460.753542 secs 2023-09-09T23:27:11.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:11 smithi170 ceph-mon[102832]: pgmap v416: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:27:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:11 smithi082 ceph-mon[98694]: pgmap v416: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:27:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:11 smithi098 ceph-mon[103124]: pgmap v416: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:27:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 463.403721 secs 2023-09-09T23:27:12.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 463.403721 secs 2023-09-09T23:27:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 463.403721 secs 2023-09-09T23:27:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:13 smithi170 ceph-mon[102832]: pgmap v417: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 28 KiB/s wr, 13 op/s 2023-09-09T23:27:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 465.024617 secs 2023-09-09T23:27:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:13 smithi170 ceph-mon[102832]: osdmap e100: 12 total, 12 up, 12 in 2023-09-09T23:27:13.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:13 smithi082 ceph-mon[98694]: pgmap v417: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 28 KiB/s wr, 13 op/s 2023-09-09T23:27:13.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 465.024617 secs 2023-09-09T23:27:13.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:13 smithi082 ceph-mon[98694]: osdmap e100: 12 total, 12 up, 12 in 2023-09-09T23:27:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:13 smithi098 ceph-mon[103124]: pgmap v417: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 28 KiB/s wr, 13 op/s 2023-09-09T23:27:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 465.024617 secs 2023-09-09T23:27:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:13 smithi098 ceph-mon[103124]: osdmap e100: 12 total, 12 up, 12 in 2023-09-09T23:27:14.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 465.753610 secs 2023-09-09T23:27:14.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 465.753610 secs 2023-09-09T23:27:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 465.753610 secs 2023-09-09T23:27:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:15 smithi170 ceph-mon[102832]: pgmap v419: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:27:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:15 smithi082 ceph-mon[98694]: pgmap v419: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:27:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:15 smithi098 ceph-mon[103124]: pgmap v419: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:27:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:17 smithi170 ceph-mon[102832]: pgmap v420: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.6 KiB/s wr, 12 op/s 2023-09-09T23:27:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 468.424548 secs 2023-09-09T23:27:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:17 smithi082 ceph-mon[98694]: pgmap v420: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.6 KiB/s wr, 12 op/s 2023-09-09T23:27:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 468.424548 secs 2023-09-09T23:27:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:17 smithi098 ceph-mon[103124]: pgmap v420: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.6 KiB/s wr, 12 op/s 2023-09-09T23:27:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 468.424548 secs 2023-09-09T23:27:18.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 470.024674 secs 2023-09-09T23:27:18.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 470.024674 secs 2023-09-09T23:27:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 470.024674 secs 2023-09-09T23:27:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:19 smithi170 ceph-mon[102832]: pgmap v421: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 KiB/s wr, 11 op/s 2023-09-09T23:27:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 470.753698 secs 2023-09-09T23:27:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:27:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:27:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:27:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:27:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:27:19.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:19 smithi082 ceph-mon[98694]: pgmap v421: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 KiB/s wr, 11 op/s 2023-09-09T23:27:19.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 470.753698 secs 2023-09-09T23:27:19.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:27:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:27:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:27:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:27:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:27:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:19 smithi098 ceph-mon[103124]: pgmap v421: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 KiB/s wr, 11 op/s 2023-09-09T23:27:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 470.753698 secs 2023-09-09T23:27:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:27:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:27:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:27:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:27:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:27:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:27:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:27:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:27:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:27:20.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:27:20.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:27:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:27:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:27:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:27:21.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:21 smithi170 ceph-mon[102832]: pgmap v422: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 13 op/s 2023-09-09T23:27:21.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:21 smithi082 ceph-mon[98694]: pgmap v422: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 13 op/s 2023-09-09T23:27:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:21 smithi098 ceph-mon[103124]: pgmap v422: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 13 op/s 2023-09-09T23:27:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 473.424596 secs 2023-09-09T23:27:22.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 473.424596 secs 2023-09-09T23:27:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 473.424596 secs 2023-09-09T23:27:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:23 smithi170 ceph-mon[102832]: pgmap v423: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 KiB/s wr, 14 op/s 2023-09-09T23:27:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 475.024737 secs 2023-09-09T23:27:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:23 smithi082 ceph-mon[98694]: pgmap v423: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 KiB/s wr, 14 op/s 2023-09-09T23:27:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 475.024737 secs 2023-09-09T23:27:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:23 smithi098 ceph-mon[103124]: pgmap v423: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 KiB/s wr, 14 op/s 2023-09-09T23:27:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 475.024737 secs 2023-09-09T23:27:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 475.753762 secs 2023-09-09T23:27:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:24 smithi170 ceph-mon[102832]: pgmap v424: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 12 op/s 2023-09-09T23:27:24.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 475.753762 secs 2023-09-09T23:27:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:24 smithi082 ceph-mon[98694]: pgmap v424: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 12 op/s 2023-09-09T23:27:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 475.753762 secs 2023-09-09T23:27:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:24 smithi098 ceph-mon[103124]: pgmap v424: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 12 op/s 2023-09-09T23:27:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:27 smithi098 ceph-mon[103124]: pgmap v425: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 KiB/s wr, 16 op/s 2023-09-09T23:27:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 478.424659 secs 2023-09-09T23:27:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:27 smithi170 ceph-mon[102832]: pgmap v425: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 KiB/s wr, 16 op/s 2023-09-09T23:27:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 478.424659 secs 2023-09-09T23:27:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:27 smithi082 ceph-mon[98694]: pgmap v425: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 KiB/s wr, 16 op/s 2023-09-09T23:27:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 478.424659 secs 2023-09-09T23:27:28.444 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:27:28.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.583+0000 7efe2a4a3700 1 -- 172.21.15.82:0/3752021901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe24105590 msgr2=0x7efe24105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:27:28.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.583+0000 7efe2a4a3700 1 --2- 172.21.15.82:0/3752021901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe24105590 0x7efe24105970 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7efe180099c0 tx=0x7efe18009cd0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.583+0000 7efe2a4a3700 1 -- 172.21.15.82:0/3752021901 shutdown_connections 2023-09-09T23:27:28.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.583+0000 7efe2a4a3700 1 --2- 172.21.15.82:0/3752021901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe24100530 0x7efe2410c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.583+0000 7efe2a4a3700 1 --2- 172.21.15.82:0/3752021901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe24105590 0x7efe24105970 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.583+0000 7efe2a4a3700 1 --2- 172.21.15.82:0/3752021901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe240ffb70 0x7efe240ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.583+0000 7efe2a4a3700 1 -- 172.21.15.82:0/3752021901 >> 172.21.15.82:0/3752021901 conn(0x7efe24076a70 msgr2=0x7efe240fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:27:28.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.583+0000 7efe2a4a3700 1 -- 172.21.15.82:0/3752021901 shutdown_connections 2023-09-09T23:27:28.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.583+0000 7efe2a4a3700 1 -- 172.21.15.82:0/3752021901 wait complete. 2023-09-09T23:27:28.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.583+0000 7efe2a4a3700 1 Processor -- start 2023-09-09T23:27:28.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.583+0000 7efe2a4a3700 1 -- start start 2023-09-09T23:27:28.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.584+0000 7efe2a4a3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe240ffb70 0x7efe24195280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:28.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.584+0000 7efe2a4a3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe24100530 0x7efe241957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:28.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.584+0000 7efe2a4a3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe24105590 0x7efe24195d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:28.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.584+0000 7efe2a4a3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe241a4ac0 con 0x7efe240ffb70 2023-09-09T23:27:28.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.584+0000 7efe2a4a3700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe241a4c30 con 0x7efe24100530 2023-09-09T23:27:28.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.584+0000 7efe2a4a3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe241a4df0 con 0x7efe24105590 2023-09-09T23:27:28.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.584+0000 7efe23fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe240ffb70 0x7efe24195280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:28.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.584+0000 7efe237fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe24100530 0x7efe241957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:28.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.584+0000 7efe28a40700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe24105590 0x7efe24195d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:28.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.584+0000 7efe23fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe240ffb70 0x7efe24195280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58562/0 (socket says 172.21.15.82:58562) 2023-09-09T23:27:28.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.584+0000 7efe23fff700 1 -- 172.21.15.82:0/2359416252 learned_addr learned my addr 172.21.15.82:0/2359416252 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:27:28.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.584+0000 7efe23fff700 1 -- 172.21.15.82:0/2359416252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe24105590 msgr2=0x7efe24195d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:27:28.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.584+0000 7efe23fff700 1 --2- 172.21.15.82:0/2359416252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe24105590 0x7efe24195d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.584+0000 7efe23fff700 1 -- 172.21.15.82:0/2359416252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe24100530 msgr2=0x7efe241957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:27:28.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.584+0000 7efe23fff700 1 --2- 172.21.15.82:0/2359416252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe24100530 0x7efe241957c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.584+0000 7efe23fff700 1 -- 172.21.15.82:0/2359416252 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe180095f0 con 0x7efe240ffb70 2023-09-09T23:27:28.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.584+0000 7efe23fff700 1 --2- 172.21.15.82:0/2359416252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe240ffb70 0x7efe24195280 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7efe1800b580 tx=0x7efe1800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:27:28.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.585+0000 7efe217fa700 1 -- 172.21.15.82:0/2359416252 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1801d070 con 0x7efe240ffb70 2023-09-09T23:27:28.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.585+0000 7efe2a4a3700 1 -- 172.21.15.82:0/2359416252 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe241a5070 con 0x7efe240ffb70 2023-09-09T23:27:28.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.585+0000 7efe217fa700 1 -- 172.21.15.82:0/2359416252 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe18022470 con 0x7efe240ffb70 2023-09-09T23:27:28.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.585+0000 7efe217fa700 1 -- 172.21.15.82:0/2359416252 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe18017650 con 0x7efe240ffb70 2023-09-09T23:27:28.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.585+0000 7efe2a4a3700 1 -- 172.21.15.82:0/2359416252 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe241a5520 con 0x7efe240ffb70 2023-09-09T23:27:28.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.587+0000 7efe217fa700 1 -- 172.21.15.82:0/2359416252 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) v1 ==== 135390+0+0 (secure 0 0 0) 0x7efe180225e0 con 0x7efe240ffb70 2023-09-09T23:27:28.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.588+0000 7efe217fa700 1 --2- 172.21.15.82:0/2359416252 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7efe0c0a07b0 0x7efe0c0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:28.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.588+0000 7efe237fe700 1 --2- 172.21.15.82:0/2359416252 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7efe0c0a07b0 0x7efe0c0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:28.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.588+0000 7efe217fa700 1 -- 172.21.15.82:0/2359416252 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8455+0+0 (secure 0 0 0) 0x7efe180cd0e0 con 0x7efe240ffb70 2023-09-09T23:27:28.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.588+0000 7efe237fe700 1 --2- 172.21.15.82:0/2359416252 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7efe0c0a07b0 0x7efe0c0a2c70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efe240ff3c0 tx=0x7efe14009260 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:27:28.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.588+0000 7efe36669b80 1 Processor -- start 2023-09-09T23:27:28.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.589+0000 7efe36669b80 1 -- start start 2023-09-09T23:27:28.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.589+0000 7efe36669b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb5e656710 0x55bb5e656af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:28.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.589+0000 7efe36669b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb5e657030 0x55bb5e660900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:28.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.589+0000 7efe36669b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb5e660e40 0x55bb5e663230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:28.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.589+0000 7efe36669b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb5e5af730 con 0x55bb5e656710 2023-09-09T23:27:28.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.589+0000 7efe36669b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb5e5ad050 con 0x55bb5e657030 2023-09-09T23:27:28.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.589+0000 7efe36669b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb5e5a8940 con 0x55bb5e660e40 2023-09-09T23:27:28.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.589+0000 7efe23fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb5e656710 0x55bb5e656af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:28.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.589+0000 7efe237fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb5e657030 0x55bb5e660900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:28.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.589+0000 7efe23fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb5e656710 0x55bb5e656af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58570/0 (socket says 172.21.15.82:58570) 2023-09-09T23:27:28.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.589+0000 7efe237fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb5e657030 0x55bb5e660900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:44136/0 (socket says 172.21.15.82:44136) 2023-09-09T23:27:28.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.589+0000 7efe23fff700 1 -- 172.21.15.82:0/965724060 learned_addr learned my addr 172.21.15.82:0/965724060 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:27:28.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.589+0000 7efe28a40700 1 --2- 172.21.15.82:0/965724060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb5e660e40 0x55bb5e663230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:28.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.589+0000 7efe23fff700 1 -- 172.21.15.82:0/965724060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb5e660e40 msgr2=0x55bb5e663230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:27:28.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.589+0000 7efe23fff700 1 --2- 172.21.15.82:0/965724060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb5e660e40 0x55bb5e663230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.589+0000 7efe23fff700 1 -- 172.21.15.82:0/965724060 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb5e657030 msgr2=0x55bb5e660900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:27:28.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.589+0000 7efe23fff700 1 --2- 172.21.15.82:0/965724060 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb5e657030 0x55bb5e660900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.589+0000 7efe23fff700 1 -- 172.21.15.82:0/965724060 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb5e3d4de0 con 0x55bb5e656710 2023-09-09T23:27:28.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.589+0000 7efe237fe700 1 --2- 172.21.15.82:0/965724060 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb5e657030 0x55bb5e660900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:27:28.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.589+0000 7efe23fff700 1 --2- 172.21.15.82:0/965724060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb5e656710 0x55bb5e656af0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7efe1809a710 tx=0x7efe1809a310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef94dfa1ef7c52b7 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:27:28.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.589+0000 7efe0a7fc700 1 -- 172.21.15.82:0/965724060 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1809a780 con 0x55bb5e656710 2023-09-09T23:27:28.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.590+0000 7efe0a7fc700 1 -- 172.21.15.82:0/965724060 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe1809e5b0 con 0x55bb5e656710 2023-09-09T23:27:28.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.590+0000 7efe0a7fc700 1 -- 172.21.15.82:0/965724060 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe18099c70 con 0x55bb5e656710 2023-09-09T23:27:28.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.590+0000 7efe36669b80 1 -- 172.21.15.82:0/965724060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb5e656710 msgr2=0x55bb5e656af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:27:28.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.590+0000 7efe36669b80 1 --2- 172.21.15.82:0/965724060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb5e656710 0x55bb5e656af0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7efe1809a710 tx=0x7efe1809a310 comp rx=0 tx=0).stop 2023-09-09T23:27:28.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.590+0000 7efe36669b80 1 -- 172.21.15.82:0/965724060 shutdown_connections 2023-09-09T23:27:28.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.590+0000 7efe36669b80 1 --2- 172.21.15.82:0/965724060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb5e660e40 0x55bb5e663230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.590+0000 7efe36669b80 1 --2- 172.21.15.82:0/965724060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb5e656710 0x55bb5e656af0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.590+0000 7efe36669b80 1 --2- 172.21.15.82:0/965724060 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb5e657030 0x55bb5e660900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.590+0000 7efe36669b80 1 -- 172.21.15.82:0/965724060 >> 172.21.15.82:0/965724060 conn(0x55bb5e5cb090 msgr2=0x55bb5e651bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:27:28.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.590+0000 7efe36669b80 1 -- 172.21.15.82:0/965724060 shutdown_connections 2023-09-09T23:27:28.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.590+0000 7efe36669b80 1 -- 172.21.15.82:0/965724060 wait complete. 2023-09-09T23:27:28.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.591+0000 7efe36669b80 1 -- start start 2023-09-09T23:27:28.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.591+0000 7efe36669b80 1 Processor -- start 2023-09-09T23:27:28.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.591+0000 7efe36669b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb5e656710 0x55bb5e661220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:28.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.591+0000 7efe36669b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb5e657030 0x55bb5e661760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:28.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.591+0000 7efe36669b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb5e660e40 0x55bb5e66c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:28.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.591+0000 7efe36669b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb5e546b70 con 0x55bb5e660e40 2023-09-09T23:27:28.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.591+0000 7efe36669b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb5e546870 con 0x55bb5e657030 2023-09-09T23:27:28.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.591+0000 7efe36669b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb5e42a920 con 0x55bb5e656710 2023-09-09T23:27:28.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.591+0000 7efe36669b80 10 client.0 _ll_register_callbacks cb 0x55bb5e490a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:27:28.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.591+0000 7efe28a40700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb5e660e40 0x55bb5e66c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:28.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.591+0000 7efe237fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb5e657030 0x55bb5e661760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:28.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.591+0000 7efe23fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb5e656710 0x55bb5e661220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:28.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.591+0000 7efe237fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb5e657030 0x55bb5e661760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:44142/0 (socket says 172.21.15.82:44142) 2023-09-09T23:27:28.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.591+0000 7efe28a40700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb5e660e40 0x55bb5e66c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58580/0 (socket says 172.21.15.82:58580) 2023-09-09T23:27:28.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.591+0000 7efe237fe700 1 -- 172.21.15.82:0/3618351930 learned_addr learned my addr 172.21.15.82:0/3618351930 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:27:28.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.591+0000 7efe28a40700 1 -- 172.21.15.82:0/3618351930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb5e656710 msgr2=0x55bb5e661220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:27:28.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.591+0000 7efe28a40700 1 --2- 172.21.15.82:0/3618351930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb5e656710 0x55bb5e661220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.591+0000 7efe28a40700 1 -- 172.21.15.82:0/3618351930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb5e657030 msgr2=0x55bb5e661760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:27:28.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.591+0000 7efe28a40700 1 --2- 172.21.15.82:0/3618351930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb5e657030 0x55bb5e661760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.591+0000 7efe28a40700 1 -- 172.21.15.82:0/3618351930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe14017070 con 0x55bb5e660e40 2023-09-09T23:27:28.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.591+0000 7efe23fff700 1 --2- 172.21.15.82:0/3618351930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb5e656710 0x55bb5e661220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:27:28.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.592+0000 7efe237fe700 1 --2- 172.21.15.82:0/3618351930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb5e657030 0x55bb5e661760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:27:28.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.592+0000 7efe28a40700 1 --2- 172.21.15.82:0/3618351930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb5e660e40 0x55bb5e66c2a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7efe1c011e90 tx=0x7efe1c014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8c4f03b8fea283 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:27:28.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.592+0000 7efe037fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:27:28.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.592+0000 7efe037fe700 1 -- 172.21.15.82:0/3618351930 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1c00e040 con 0x55bb5e660e40 2023-09-09T23:27:28.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.592+0000 7efe037fe700 1 -- 172.21.15.82:0/3618351930 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8455+0+0 (secure 0 0 0) 0x7efe1c004bb0 con 0x55bb5e660e40 2023-09-09T23:27:28.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.592+0000 7efe36669b80 1 -- 172.21.15.82:0/3618351930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb5e5004f0 con 0x55bb5e660e40 2023-09-09T23:27:28.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.592+0000 7efe037fe700 1 -- 172.21.15.82:0/3618351930 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe1c004d70 con 0x55bb5e660e40 2023-09-09T23:27:28.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.592+0000 7efe36669b80 10 client.16404 fetch_fsmap learned FSMap version 127 2023-09-09T23:27:28.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.592+0000 7efe36669b80 1 -- 172.21.15.82:0/3618351930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x7efe241a5520 con 0x55bb5e660e40 2023-09-09T23:27:28.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.592+0000 7efe037fe700 1 -- 172.21.15.82:0/3618351930 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bb5e5004f0 con 0x55bb5e660e40 2023-09-09T23:27:28.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.592+0000 7efe037fe700 1 -- 172.21.15.82:0/3618351930 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 127) v1 ==== 7882+0+0 (secure 0 0 0) 0x7efe1c01c030 con 0x55bb5e660e40 2023-09-09T23:27:28.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.592+0000 7efe36669b80 10 client.16404 fetch_fsmap finished waiting for FSMap version 127 2023-09-09T23:27:28.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.592+0000 7efe36669b80 10 client.16404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:27:28.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.592+0000 7efe36669b80 20 client.16404 populate_metadata read hostname 'smithi082' 2023-09-09T23:27:28.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.592+0000 7efe36669b80 1 --2- 172.21.15.82:0/3618351930 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55bb5e670650 0x55bb5e672a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:28.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.592+0000 7efe36669b80 4 client.16404 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:27:28.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.592+0000 7efe36669b80 1 -- 172.21.15.82:0/3618351930 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb5e5bd8a0 con 0x55bb5e670650 2023-09-09T23:27:28.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.592+0000 7efe23fff700 1 --2- 172.21.15.82:0/3618351930 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55bb5e670650 0x55bb5e672a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:28.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.593+0000 7efe037fe700 10 client.16404 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:27:28.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.593+0000 7efe23fff700 1 --2- 172.21.15.82:0/3618351930 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55bb5e670650 0x55bb5e672a70 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b725c33dc3d5f2fa server_cookie=258a43a41a952336 in_seq=0 out_seq=0 2023-09-09T23:27:28.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.593+0000 7efe037fe700 1 -- 172.21.15.82:0/3618351930 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bb5e5bd8a0 con 0x55bb5e670650 2023-09-09T23:27:28.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.593+0000 7efe037fe700 10 client.16404 handle_command_reply: tid=0 2023-09-09T23:27:28.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.593+0000 7efe36669b80 1 client.16404 shutdown 2023-09-09T23:27:28.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.594+0000 7efe36669b80 1 -- 172.21.15.82:0/3618351930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb5e660e40 msgr2=0x55bb5e66c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:27:28.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.594+0000 7efe36669b80 1 --2- 172.21.15.82:0/3618351930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb5e660e40 0x55bb5e66c2a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7efe1c011e90 tx=0x7efe1c014600 comp rx=0 tx=0).stop 2023-09-09T23:27:28.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.594+0000 7efe36669b80 1 -- 172.21.15.82:0/3618351930 shutdown_connections 2023-09-09T23:27:28.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.594+0000 7efe36669b80 1 --2- 172.21.15.82:0/3618351930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb5e660e40 0x55bb5e66c2a0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.594+0000 7efe36669b80 1 --2- 172.21.15.82:0/3618351930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb5e656710 0x55bb5e661220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.594+0000 7efe36669b80 1 --2- 172.21.15.82:0/3618351930 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55bb5e670650 0x55bb5e672a70 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.594+0000 7efe36669b80 1 --2- 172.21.15.82:0/3618351930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb5e657030 0x55bb5e661760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.594+0000 7efe037fe700 0 client.16404 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:27:28.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.594+0000 7efe36669b80 1 -- 172.21.15.82:0/3618351930 >> 172.21.15.82:0/3618351930 conn(0x55bb5e5cb090 msgr2=0x55bb5e6517e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:27:28.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.594+0000 7efe36669b80 1 -- 172.21.15.82:0/3618351930 shutdown_connections 2023-09-09T23:27:28.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.594+0000 7efe36669b80 1 -- 172.21.15.82:0/3618351930 wait complete. 2023-09-09T23:27:28.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.594+0000 7efe36669b80 20 client.16404 trim_cache size 0 max 16384 2023-09-09T23:27:28.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.610+0000 7efe36669b80 1 Processor -- start 2023-09-09T23:27:28.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.610+0000 7efe36669b80 1 -- start start 2023-09-09T23:27:28.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.611+0000 7efe36669b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb5e656710 0x55bb5e65f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:28.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.611+0000 7efe36669b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb5e657030 0x55bb5e66b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:28.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.611+0000 7efe36669b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb5e660e40 0x55bb5e66bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:28.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.611+0000 7efe36669b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb5e65bc10 con 0x55bb5e657030 2023-09-09T23:27:28.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.611+0000 7efe36669b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb5e661e30 con 0x55bb5e656710 2023-09-09T23:27:28.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.611+0000 7efe36669b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb5e657a20 con 0x55bb5e660e40 2023-09-09T23:27:28.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.611+0000 7efe237fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb5e657030 0x55bb5e66b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:28.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.611+0000 7efe23fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb5e656710 0x55bb5e65f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:28.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.611+0000 7efe237fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb5e657030 0x55bb5e66b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58586/0 (socket says 172.21.15.82:58586) 2023-09-09T23:27:28.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.611+0000 7efe237fe700 1 -- 172.21.15.82:0/1178089092 learned_addr learned my addr 172.21.15.82:0/1178089092 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:27:28.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.611+0000 7efe28a40700 1 --2- 172.21.15.82:0/1178089092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb5e660e40 0x55bb5e66bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:28.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.611+0000 7efe23fff700 1 -- 172.21.15.82:0/1178089092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb5e660e40 msgr2=0x55bb5e66bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:27:28.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.611+0000 7efe23fff700 1 --2- 172.21.15.82:0/1178089092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb5e660e40 0x55bb5e66bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.611+0000 7efe23fff700 1 -- 172.21.15.82:0/1178089092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb5e657030 msgr2=0x55bb5e66b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:27:28.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.611+0000 7efe23fff700 1 --2- 172.21.15.82:0/1178089092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb5e657030 0x55bb5e66b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.611+0000 7efe23fff700 1 -- 172.21.15.82:0/1178089092 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb5e3bdc60 con 0x55bb5e656710 2023-09-09T23:27:28.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.611+0000 7efe237fe700 1 --2- 172.21.15.82:0/1178089092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb5e657030 0x55bb5e66b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:27:28.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.611+0000 7efe23fff700 1 --2- 172.21.15.82:0/1178089092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb5e656710 0x55bb5e65f750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efe1809a250 tx=0x7efe18022c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=316cd114f872e511 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:27:28.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.611+0000 7efe0affd700 1 -- 172.21.15.82:0/1178089092 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1809a780 con 0x55bb5e656710 2023-09-09T23:27:28.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.611+0000 7efe0affd700 1 -- 172.21.15.82:0/1178089092 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe180a2760 con 0x55bb5e656710 2023-09-09T23:27:28.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.611+0000 7efe0affd700 1 -- 172.21.15.82:0/1178089092 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe18099ed0 con 0x55bb5e656710 2023-09-09T23:27:28.627 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.612+0000 7efe36669b80 1 -- 172.21.15.82:0/1178089092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb5e656710 msgr2=0x55bb5e65f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:27:28.627 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.612+0000 7efe36669b80 1 --2- 172.21.15.82:0/1178089092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb5e656710 0x55bb5e65f750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efe1809a250 tx=0x7efe18022c60 comp rx=0 tx=0).stop 2023-09-09T23:27:28.627 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.612+0000 7efe36669b80 1 -- 172.21.15.82:0/1178089092 shutdown_connections 2023-09-09T23:27:28.627 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.612+0000 7efe36669b80 1 --2- 172.21.15.82:0/1178089092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb5e656710 0x55bb5e65f750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.627 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.612+0000 7efe36669b80 1 --2- 172.21.15.82:0/1178089092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb5e660e40 0x55bb5e66bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.612+0000 7efe36669b80 1 --2- 172.21.15.82:0/1178089092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb5e657030 0x55bb5e66b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.612+0000 7efe36669b80 1 -- 172.21.15.82:0/1178089092 >> 172.21.15.82:0/1178089092 conn(0x55bb5e5cb090 msgr2=0x55bb5e65a840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:27:28.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.612+0000 7efe36669b80 1 -- 172.21.15.82:0/1178089092 shutdown_connections 2023-09-09T23:27:28.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.612+0000 7efe36669b80 1 -- 172.21.15.82:0/1178089092 wait complete. 2023-09-09T23:27:28.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.612+0000 7efe36669b80 1 -- start start 2023-09-09T23:27:28.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.613+0000 7efe36669b80 1 Processor -- start 2023-09-09T23:27:28.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.613+0000 7efe36669b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb5e656710 0x55bb5e666050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:28.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.613+0000 7efe36669b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb5e657030 0x55bb5e6622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:28.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.613+0000 7efe36669b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb5e660e40 0x55bb5e662830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:28.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.613+0000 7efe36669b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb5e662d70 con 0x55bb5e660e40 2023-09-09T23:27:28.630 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.613+0000 7efe36669b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb5e662ee0 con 0x55bb5e656710 2023-09-09T23:27:28.630 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.613+0000 7efe36669b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb5e6630a0 con 0x55bb5e657030 2023-09-09T23:27:28.630 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.613+0000 7efe36669b80 10 client.0 _ll_register_callbacks cb 0x7efe1c0117c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:27:28.630 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.613+0000 7efe23fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb5e656710 0x55bb5e666050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:28.630 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.613+0000 7efe28a40700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb5e660e40 0x55bb5e662830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:28.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.613+0000 7efe23fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb5e656710 0x55bb5e666050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:44154/0 (socket says 172.21.15.82:44154) 2023-09-09T23:27:28.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.613+0000 7efe23fff700 1 -- 172.21.15.82:0/2484826553 learned_addr learned my addr 172.21.15.82:0/2484826553 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:27:28.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.613+0000 7efe237fe700 1 --2- 172.21.15.82:0/2484826553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb5e657030 0x55bb5e6622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:28.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.613+0000 7efe28a40700 1 -- 172.21.15.82:0/2484826553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb5e657030 msgr2=0x55bb5e6622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:27:28.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.613+0000 7efe28a40700 1 --2- 172.21.15.82:0/2484826553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb5e657030 0x55bb5e6622f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.613+0000 7efe28a40700 1 -- 172.21.15.82:0/2484826553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb5e656710 msgr2=0x55bb5e666050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:27:28.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.613+0000 7efe28a40700 1 --2- 172.21.15.82:0/2484826553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb5e656710 0x55bb5e666050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.613+0000 7efe28a40700 1 -- 172.21.15.82:0/2484826553 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe1401d410 con 0x55bb5e660e40 2023-09-09T23:27:28.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.613+0000 7efe23fff700 1 --2- 172.21.15.82:0/2484826553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb5e656710 0x55bb5e666050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:27:28.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.613+0000 7efe28a40700 1 --2- 172.21.15.82:0/2484826553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb5e660e40 0x55bb5e662830 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7efe1c011e60 tx=0x7efe1c009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbf1a61b9be35e1 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:27:28.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.613+0000 7efe037fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:27:28.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.614+0000 7efe037fe700 1 -- 172.21.15.82:0/2484826553 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1c00e070 con 0x55bb5e660e40 2023-09-09T23:27:28.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.614+0000 7efe037fe700 1 -- 172.21.15.82:0/2484826553 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8455+0+0 (secure 0 0 0) 0x7efe1c02c070 con 0x55bb5e660e40 2023-09-09T23:27:28.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.614+0000 7efe36669b80 1 -- 172.21.15.82:0/2484826553 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe1809e5b0 con 0x55bb5e660e40 2023-09-09T23:27:28.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.614+0000 7efe037fe700 1 -- 172.21.15.82:0/2484826553 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe1c014d50 con 0x55bb5e660e40 2023-09-09T23:27:28.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.614+0000 7efe36669b80 10 client.16410 fetch_fsmap learned FSMap version 127 2023-09-09T23:27:28.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.614+0000 7efe36669b80 1 -- 172.21.15.82:0/2484826553 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x55bb5e3d4de0 con 0x55bb5e660e40 2023-09-09T23:27:28.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.614+0000 7efe037fe700 1 -- 172.21.15.82:0/2484826553 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe1809e5b0 con 0x55bb5e660e40 2023-09-09T23:27:28.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.614+0000 7efe037fe700 1 -- 172.21.15.82:0/2484826553 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 127) v1 ==== 7882+0+0 (secure 0 0 0) 0x7efe1c014320 con 0x55bb5e660e40 2023-09-09T23:27:28.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.614+0000 7efe36669b80 10 client.16410 fetch_fsmap finished waiting for FSMap version 127 2023-09-09T23:27:28.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.614+0000 7efe36669b80 10 client.16410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:27:28.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.614+0000 7efe36669b80 20 client.16410 populate_metadata read hostname 'smithi082' 2023-09-09T23:27:28.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.614+0000 7efe36669b80 1 --2- 172.21.15.82:0/2484826553 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55bb5e6c5860 0x55bb5e6c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:28.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.614+0000 7efe36669b80 4 client.16410 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:27:28.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.614+0000 7efe36669b80 1 -- 172.21.15.82:0/2484826553 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bb5e5a1920 con 0x55bb5e6c5860 2023-09-09T23:27:28.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.614+0000 7efe23fff700 1 --2- 172.21.15.82:0/2484826553 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55bb5e6c5860 0x55bb5e6c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:28.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.615+0000 7efe037fe700 10 client.16410 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:27:28.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.615+0000 7efe23fff700 1 --2- 172.21.15.82:0/2484826553 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55bb5e6c5860 0x55bb5e6c7c50 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a066887c557d9990 server_cookie=ef6c3f6999c19c1 in_seq=0 out_seq=0 2023-09-09T23:27:28.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.615+0000 7efe037fe700 1 -- 172.21.15.82:0/2484826553 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55bb5e5a1920 con 0x55bb5e6c5860 2023-09-09T23:27:28.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.615+0000 7efe037fe700 10 client.16410 handle_command_reply: tid=0 2023-09-09T23:27:28.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.615+0000 7efe36669b80 1 client.16410 shutdown 2023-09-09T23:27:28.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.615+0000 7efe36669b80 1 -- 172.21.15.82:0/2484826553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb5e660e40 msgr2=0x55bb5e662830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:27:28.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.615+0000 7efe36669b80 1 --2- 172.21.15.82:0/2484826553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb5e660e40 0x55bb5e662830 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7efe1c011e60 tx=0x7efe1c009510 comp rx=0 tx=0).stop 2023-09-09T23:27:28.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.615+0000 7efe36669b80 1 -- 172.21.15.82:0/2484826553 shutdown_connections 2023-09-09T23:27:28.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.615+0000 7efe36669b80 1 --2- 172.21.15.82:0/2484826553 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55bb5e6c5860 0x55bb5e6c7c50 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.615+0000 7efe36669b80 1 --2- 172.21.15.82:0/2484826553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb5e656710 0x55bb5e666050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.615+0000 7efe36669b80 1 --2- 172.21.15.82:0/2484826553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb5e660e40 0x55bb5e662830 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.615+0000 7efe36669b80 1 --2- 172.21.15.82:0/2484826553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bb5e657030 0x55bb5e6622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.615+0000 7efe037fe700 0 client.16410 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:27:28.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.615+0000 7efe36669b80 1 -- 172.21.15.82:0/2484826553 >> 172.21.15.82:0/2484826553 conn(0x55bb5e5cb090 msgr2=0x55bb5e655f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:27:28.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.615+0000 7efe36669b80 1 -- 172.21.15.82:0/2484826553 shutdown_connections 2023-09-09T23:27:28.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.615+0000 7efe36669b80 1 -- 172.21.15.82:0/2484826553 wait complete. 2023-09-09T23:27:28.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.615+0000 7efe36669b80 20 client.16410 trim_cache size 0 max 16384 2023-09-09T23:27:28.639 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:27:28.640 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:27:28.640 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:27:28.640 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:27:28.640 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:27:28.640 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:27:28.641 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:27:28.641 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:27:28.641 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:27:28.641 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:27:28.642 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.616+0000 7efe2a4a3700 1 -- 172.21.15.82:0/2359416252 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7efe0c0a07b0 msgr2=0x7efe0c0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:27:28.643 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.616+0000 7efe2a4a3700 1 --2- 172.21.15.82:0/2359416252 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7efe0c0a07b0 0x7efe0c0a2c70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efe240ff3c0 tx=0x7efe14009260 comp rx=0 tx=0).stop 2023-09-09T23:27:28.643 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.616+0000 7efe2a4a3700 1 -- 172.21.15.82:0/2359416252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe240ffb70 msgr2=0x7efe24195280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:27:28.643 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.616+0000 7efe2a4a3700 1 --2- 172.21.15.82:0/2359416252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe240ffb70 0x7efe24195280 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7efe1800b580 tx=0x7efe1800bfd0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.643 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.616+0000 7efe2a4a3700 1 -- 172.21.15.82:0/2359416252 shutdown_connections 2023-09-09T23:27:28.644 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.616+0000 7efe2a4a3700 1 --2- 172.21.15.82:0/2359416252 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7efe0c0a07b0 0x7efe0c0a2c70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.644 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.616+0000 7efe2a4a3700 1 --2- 172.21.15.82:0/2359416252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe24105590 0x7efe24195d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.644 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.616+0000 7efe2a4a3700 1 --2- 172.21.15.82:0/2359416252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe240ffb70 0x7efe24195280 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.644 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.616+0000 7efe2a4a3700 1 --2- 172.21.15.82:0/2359416252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe24100530 0x7efe241957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:28.644 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.616+0000 7efe2a4a3700 1 -- 172.21.15.82:0/2359416252 >> 172.21.15.82:0/2359416252 conn(0x7efe24076a70 msgr2=0x7efe240fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:27:28.644 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.616+0000 7efe2a4a3700 1 -- 172.21.15.82:0/2359416252 shutdown_connections 2023-09-09T23:27:28.645 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:28.616+0000 7efe2a4a3700 1 -- 172.21.15.82:0/2359416252 wait complete. 2023-09-09T23:27:28.645 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:27:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 480.024795 secs 2023-09-09T23:27:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 480.024795 secs 2023-09-09T23:27:28.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 480.024795 secs 2023-09-09T23:27:29.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:29 smithi170 ceph-mon[102832]: pgmap v426: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 11 op/s 2023-09-09T23:27:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 480.753831 secs 2023-09-09T23:27:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:29 smithi082 ceph-mon[98694]: pgmap v426: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 11 op/s 2023-09-09T23:27:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 480.753831 secs 2023-09-09T23:27:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:29 smithi098 ceph-mon[103124]: pgmap v426: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 11 op/s 2023-09-09T23:27:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 480.753831 secs 2023-09-09T23:27:31.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:31 smithi170 ceph-mon[102832]: pgmap v427: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 14 op/s 2023-09-09T23:27:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:31 smithi082 ceph-mon[98694]: pgmap v427: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 14 op/s 2023-09-09T23:27:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:31 smithi098 ceph-mon[103124]: pgmap v427: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 14 op/s 2023-09-09T23:27:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 483.424750 secs 2023-09-09T23:27:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 483.424750 secs 2023-09-09T23:27:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 483.424750 secs 2023-09-09T23:27:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:33 smithi170 ceph-mon[102832]: pgmap v428: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.1 KiB/s wr, 10 op/s 2023-09-09T23:27:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 485.024854 secs 2023-09-09T23:27:33.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:33 smithi082 ceph-mon[98694]: pgmap v428: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.1 KiB/s wr, 10 op/s 2023-09-09T23:27:33.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 485.024854 secs 2023-09-09T23:27:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:33 smithi098 ceph-mon[103124]: pgmap v428: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.1 KiB/s wr, 10 op/s 2023-09-09T23:27:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 485.024854 secs 2023-09-09T23:27:34.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 485.753897 secs 2023-09-09T23:27:34.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 485.753897 secs 2023-09-09T23:27:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 485.753897 secs 2023-09-09T23:27:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:35 smithi170 ceph-mon[102832]: pgmap v429: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2023-09-09T23:27:35.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:35 smithi082 ceph-mon[98694]: pgmap v429: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2023-09-09T23:27:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:35 smithi098 ceph-mon[103124]: pgmap v429: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2023-09-09T23:27:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:37 smithi170 ceph-mon[102832]: pgmap v430: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2023-09-09T23:27:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 488.424812 secs 2023-09-09T23:27:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:37 smithi082 ceph-mon[98694]: pgmap v430: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2023-09-09T23:27:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 488.424812 secs 2023-09-09T23:27:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:37 smithi098 ceph-mon[103124]: pgmap v430: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2023-09-09T23:27:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 488.424812 secs 2023-09-09T23:27:38.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 490.024913 secs 2023-09-09T23:27:38.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 490.024913 secs 2023-09-09T23:27:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 490.024913 secs 2023-09-09T23:27:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:39 smithi170 ceph-mon[102832]: pgmap v431: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2023-09-09T23:27:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 490.753962 secs 2023-09-09T23:27:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:39 smithi082 ceph-mon[98694]: pgmap v431: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2023-09-09T23:27:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 490.753962 secs 2023-09-09T23:27:40.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:39 smithi098 ceph-mon[103124]: pgmap v431: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2023-09-09T23:27:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 490.753962 secs 2023-09-09T23:27:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:41 smithi170 ceph-mon[102832]: pgmap v432: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2023-09-09T23:27:41.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:41 smithi082 ceph-mon[98694]: pgmap v432: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2023-09-09T23:27:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:41 smithi098 ceph-mon[103124]: pgmap v432: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2023-09-09T23:27:42.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 493.424876 secs 2023-09-09T23:27:42.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 493.424876 secs 2023-09-09T23:27:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 493.424876 secs 2023-09-09T23:27:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:43 smithi170 ceph-mon[102832]: pgmap v433: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:27:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 495.024995 secs 2023-09-09T23:27:43.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:43 smithi082 ceph-mon[98694]: pgmap v433: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:27:43.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 495.024995 secs 2023-09-09T23:27:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:43 smithi098 ceph-mon[103124]: pgmap v433: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:27:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 495.024995 secs 2023-09-09T23:27:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 495.754030 secs 2023-09-09T23:27:44.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 495.754030 secs 2023-09-09T23:27:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 495.754030 secs 2023-09-09T23:27:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:45 smithi170 ceph-mon[102832]: pgmap v434: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:27:45.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:45 smithi082 ceph-mon[98694]: pgmap v434: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:27:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:45 smithi098 ceph-mon[103124]: pgmap v434: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:27:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:47 smithi170 ceph-mon[102832]: pgmap v435: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:27:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 498.424906 secs 2023-09-09T23:27:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:47 smithi082 ceph-mon[98694]: pgmap v435: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:27:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 498.424906 secs 2023-09-09T23:27:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:47 smithi098 ceph-mon[103124]: pgmap v435: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:27:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 498.424906 secs 2023-09-09T23:27:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 500.025078 secs 2023-09-09T23:27:48.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 500.025078 secs 2023-09-09T23:27:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 500.025078 secs 2023-09-09T23:27:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:49 smithi170 ceph-mon[102832]: pgmap v436: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:27:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 500.754094 secs 2023-09-09T23:27:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:49 smithi082 ceph-mon[98694]: pgmap v436: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:27:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 500.754094 secs 2023-09-09T23:27:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:49 smithi098 ceph-mon[103124]: pgmap v436: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:27:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 500.754094 secs 2023-09-09T23:27:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:51 smithi170 ceph-mon[102832]: pgmap v437: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-09-09T23:27:51.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:51 smithi082 ceph-mon[98694]: pgmap v437: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-09-09T23:27:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:51 smithi098 ceph-mon[103124]: pgmap v437: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-09-09T23:27:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 503.424964 secs 2023-09-09T23:27:52.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 503.424964 secs 2023-09-09T23:27:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 503.424964 secs 2023-09-09T23:27:53.926 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:53 smithi170 ceph-mon[102832]: pgmap v438: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:27:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 505.025166 secs 2023-09-09T23:27:53.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:53 smithi082 ceph-mon[98694]: pgmap v438: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:27:53.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 505.025166 secs 2023-09-09T23:27:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:53 smithi098 ceph-mon[103124]: pgmap v438: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:27:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 505.025166 secs 2023-09-09T23:27:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 505.754158 secs 2023-09-09T23:27:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 505.754158 secs 2023-09-09T23:27:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 505.754158 secs 2023-09-09T23:27:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:55 smithi170 ceph-mon[102832]: pgmap v439: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:27:55.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:55 smithi082 ceph-mon[98694]: pgmap v439: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:27:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:55 smithi098 ceph-mon[103124]: pgmap v439: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:27:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:56 smithi170 ceph-mon[102832]: pgmap v440: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2023-09-09T23:27:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:56 smithi082 ceph-mon[98694]: pgmap v440: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2023-09-09T23:27:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:56 smithi098 ceph-mon[103124]: pgmap v440: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2023-09-09T23:27:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 508.425026 secs 2023-09-09T23:27:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 508.425026 secs 2023-09-09T23:27:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 508.425026 secs 2023-09-09T23:27:58.646 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:27:58.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.785+0000 7f78b2719700 1 -- 172.21.15.82:0/2360569770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78ac0ff5d0 msgr2=0x7f78ac0ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:27:58.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.785+0000 7f78b2719700 1 --2- 172.21.15.82:0/2360569770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78ac0ff5d0 0x7f78ac0ffa50 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f78a0009990 tx=0x7f78a0009ca0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.785+0000 7f78b2719700 1 -- 172.21.15.82:0/2360569770 shutdown_connections 2023-09-09T23:27:58.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.785+0000 7f78b2719700 1 --2- 172.21.15.82:0/2360569770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78ac1054f0 0x7f78ac1058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.785+0000 7f78b2719700 1 --2- 172.21.15.82:0/2360569770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78ac0fff90 0x7f78ac10c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.786 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.785+0000 7f78b2719700 1 --2- 172.21.15.82:0/2360569770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78ac0ff5d0 0x7f78ac0ffa50 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.786 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.785+0000 7f78b2719700 1 -- 172.21.15.82:0/2360569770 >> 172.21.15.82:0/2360569770 conn(0x7f78ac075020 msgr2=0x7f78ac075430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:27:58.786 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.785+0000 7f78b2719700 1 -- 172.21.15.82:0/2360569770 shutdown_connections 2023-09-09T23:27:58.786 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.785+0000 7f78b2719700 1 -- 172.21.15.82:0/2360569770 wait complete. 2023-09-09T23:27:58.786 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.785+0000 7f78b2719700 1 Processor -- start 2023-09-09T23:27:58.787 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.785+0000 7f78b2719700 1 -- start start 2023-09-09T23:27:58.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.786+0000 7f78b2719700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78ac0ff5d0 0x7f78ac195580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:58.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.786+0000 7f78b2719700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78ac0fff90 0x7f78ac195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:58.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.786+0000 7f78b2719700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78ac1054f0 0x7f78ac199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:58.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.786+0000 7f78b2719700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac19a560 con 0x7f78ac1054f0 2023-09-09T23:27:58.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.786+0000 7f78b2719700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac19a6d0 con 0x7f78ac0ff5d0 2023-09-09T23:27:58.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.786+0000 7f78b2719700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac19a890 con 0x7f78ac0fff90 2023-09-09T23:27:58.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.786+0000 7f78b0cb6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78ac1054f0 0x7f78ac199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:58.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.786+0000 7f78b0cb6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78ac1054f0 0x7f78ac199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58608/0 (socket says 172.21.15.82:58608) 2023-09-09T23:27:58.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.786+0000 7f78b0cb6700 1 -- 172.21.15.82:0/147429603 learned_addr learned my addr 172.21.15.82:0/147429603 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:27:58.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.786+0000 7f78abfff700 1 --2- 172.21.15.82:0/147429603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78ac0ff5d0 0x7f78ac195580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:58.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.786+0000 7f78ab7fe700 1 --2- 172.21.15.82:0/147429603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78ac0fff90 0x7f78ac195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:58.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.786+0000 7f78b0cb6700 1 -- 172.21.15.82:0/147429603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78ac0fff90 msgr2=0x7f78ac195ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:27:58.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.786+0000 7f78b0cb6700 1 --2- 172.21.15.82:0/147429603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78ac0fff90 0x7f78ac195ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.786+0000 7f78b0cb6700 1 -- 172.21.15.82:0/147429603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78ac0ff5d0 msgr2=0x7f78ac195580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:27:58.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.786+0000 7f78b0cb6700 1 --2- 172.21.15.82:0/147429603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78ac0ff5d0 0x7f78ac195580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.786+0000 7f78b0cb6700 1 -- 172.21.15.82:0/147429603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a00095f0 con 0x7f78ac1054f0 2023-09-09T23:27:58.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.786+0000 7f78b0cb6700 1 --2- 172.21.15.82:0/147429603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78ac1054f0 0x7f78ac199f20 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f78a400ef30 tx=0x7f78a400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:27:58.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.787+0000 7f78a97fa700 1 -- 172.21.15.82:0/147429603 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a4018070 con 0x7f78ac1054f0 2023-09-09T23:27:58.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.787+0000 7f78a97fa700 1 -- 172.21.15.82:0/147429603 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78a400f040 con 0x7f78ac1054f0 2023-09-09T23:27:58.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.787+0000 7f78b2719700 1 -- 172.21.15.82:0/147429603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78ac19ab10 con 0x7f78ac1054f0 2023-09-09T23:27:58.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.787+0000 7f78a97fa700 1 -- 172.21.15.82:0/147429603 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a400cdb0 con 0x7f78ac1054f0 2023-09-09T23:27:58.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.787+0000 7f78b2719700 1 -- 172.21.15.82:0/147429603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78ac1a23b0 con 0x7f78ac1054f0 2023-09-09T23:27:58.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.789+0000 7f78a97fa700 1 -- 172.21.15.82:0/147429603 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f78a4003a00 con 0x7f78ac1054f0 2023-09-09T23:27:58.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.790+0000 7f78a97fa700 1 --2- 172.21.15.82:0/147429603 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f78940a07b0 0x7f78940a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:58.794 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.790+0000 7f78a97fa700 1 -- 172.21.15.82:0/147429603 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f78a40cbbd0 con 0x7f78ac1054f0 2023-09-09T23:27:58.794 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.790+0000 7f78abfff700 1 --2- 172.21.15.82:0/147429603 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f78940a07b0 0x7f78940a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:58.801 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.791+0000 7f78abfff700 1 --2- 172.21.15.82:0/147429603 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f78940a07b0 0x7f78940a2c70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f789c009be0 tx=0x7f789c009260 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:27:58.801 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.791+0000 7f78be8dfb80 1 Processor -- start 2023-09-09T23:27:58.801 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.791+0000 7f78be8dfb80 1 -- start start 2023-09-09T23:27:58.801 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.791+0000 7f78be8dfb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8517b1710 0x55c8517b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:58.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.791+0000 7f78be8dfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8517b2030 0x55c8517bb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:58.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.791+0000 7f78be8dfb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8517bbe40 0x55c8517be230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:58.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.791+0000 7f78be8dfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c85170a730 con 0x55c8517b2030 2023-09-09T23:27:58.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.791+0000 7f78be8dfb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c851708050 con 0x55c8517bbe40 2023-09-09T23:27:58.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.791+0000 7f78be8dfb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c851703940 con 0x55c8517b1710 2023-09-09T23:27:58.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.791+0000 7f78ab7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8517b2030 0x55c8517bb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:58.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.791+0000 7f78ab7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8517b2030 0x55c8517bb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58614/0 (socket says 172.21.15.82:58614) 2023-09-09T23:27:58.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.791+0000 7f78ab7fe700 1 -- 172.21.15.82:0/582584819 learned_addr learned my addr 172.21.15.82:0/582584819 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:27:58.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.791+0000 7f78b0cb6700 1 --2- 172.21.15.82:0/582584819 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8517bbe40 0x55c8517be230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:58.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.791+0000 7f78abfff700 1 --2- 172.21.15.82:0/582584819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8517b1710 0x55c8517b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:58.804 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.791+0000 7f78ab7fe700 1 -- 172.21.15.82:0/582584819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8517b1710 msgr2=0x55c8517b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:27:58.804 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.791+0000 7f78ab7fe700 1 --2- 172.21.15.82:0/582584819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8517b1710 0x55c8517b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.804 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.791+0000 7f78ab7fe700 1 -- 172.21.15.82:0/582584819 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8517bbe40 msgr2=0x55c8517be230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:27:58.804 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.791+0000 7f78ab7fe700 1 --2- 172.21.15.82:0/582584819 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8517bbe40 0x55c8517be230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.804 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.791+0000 7f78ab7fe700 1 -- 172.21.15.82:0/582584819 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c85152fde0 con 0x55c8517b2030 2023-09-09T23:27:58.804 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.791+0000 7f78b0cb6700 1 --2- 172.21.15.82:0/582584819 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8517bbe40 0x55c8517be230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:27:58.805 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.791+0000 7f78ab7fe700 1 --2- 172.21.15.82:0/582584819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8517b2030 0x55c8517bb900 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f78a0006010 tx=0x7f78a0005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7443e7a85b61bcc server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:27:58.805 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.792+0000 7f78927fc700 1 -- 172.21.15.82:0/582584819 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a000beb0 con 0x55c8517b2030 2023-09-09T23:27:58.805 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.792+0000 7f78927fc700 1 -- 172.21.15.82:0/582584819 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78a001e040 con 0x55c8517b2030 2023-09-09T23:27:58.805 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.792+0000 7f78927fc700 1 -- 172.21.15.82:0/582584819 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a0004990 con 0x55c8517b2030 2023-09-09T23:27:58.805 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.792+0000 7f78be8dfb80 1 -- 172.21.15.82:0/582584819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8517b2030 msgr2=0x55c8517bb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:27:58.806 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.792+0000 7f78be8dfb80 1 --2- 172.21.15.82:0/582584819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8517b2030 0x55c8517bb900 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f78a0006010 tx=0x7f78a0005f00 comp rx=0 tx=0).stop 2023-09-09T23:27:58.806 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.792+0000 7f78be8dfb80 1 -- 172.21.15.82:0/582584819 shutdown_connections 2023-09-09T23:27:58.806 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.792+0000 7f78be8dfb80 1 --2- 172.21.15.82:0/582584819 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8517bbe40 0x55c8517be230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.806 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.792+0000 7f78be8dfb80 1 --2- 172.21.15.82:0/582584819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8517b1710 0x55c8517b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.806 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.792+0000 7f78be8dfb80 1 --2- 172.21.15.82:0/582584819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8517b2030 0x55c8517bb900 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.807 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.792+0000 7f78be8dfb80 1 -- 172.21.15.82:0/582584819 >> 172.21.15.82:0/582584819 conn(0x55c851726090 msgr2=0x55c8517acbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:27:58.807 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.792+0000 7f78be8dfb80 1 -- 172.21.15.82:0/582584819 shutdown_connections 2023-09-09T23:27:58.807 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.792+0000 7f78be8dfb80 1 -- 172.21.15.82:0/582584819 wait complete. 2023-09-09T23:27:58.807 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.793+0000 7f78be8dfb80 1 -- start start 2023-09-09T23:27:58.807 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.793+0000 7f78be8dfb80 1 Processor -- start 2023-09-09T23:27:58.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.793+0000 7f78be8dfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8517b1710 0x55c8517bb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:58.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.793+0000 7f78be8dfb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8517b2030 0x55c8517b8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:58.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.793+0000 7f78be8dfb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8517bbe40 0x55c8517b9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:58.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.793+0000 7f78be8dfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8516a1b70 con 0x55c8517b1710 2023-09-09T23:27:58.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.793+0000 7f78be8dfb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8516a1870 con 0x55c8517bbe40 2023-09-09T23:27:58.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.793+0000 7f78be8dfb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c851585920 con 0x55c8517b2030 2023-09-09T23:27:58.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.793+0000 7f78be8dfb80 10 client.0 _ll_register_callbacks cb 0x55c8515eba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:27:58.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.793+0000 7f78abfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8517b1710 0x55c8517bb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:58.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.793+0000 7f78abfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8517b1710 0x55c8517bb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58618/0 (socket says 172.21.15.82:58618) 2023-09-09T23:27:58.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.793+0000 7f78abfff700 1 -- 172.21.15.82:0/2385478093 learned_addr learned my addr 172.21.15.82:0/2385478093 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:27:58.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.793+0000 7f78ab7fe700 1 --2- 172.21.15.82:0/2385478093 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8517b2030 0x55c8517b8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:58.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.793+0000 7f78b0cb6700 1 --2- 172.21.15.82:0/2385478093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8517bbe40 0x55c8517b9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:58.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.794+0000 7f78abfff700 1 -- 172.21.15.82:0/2385478093 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8517b2030 msgr2=0x55c8517b8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:27:58.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.794+0000 7f78abfff700 1 --2- 172.21.15.82:0/2385478093 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8517b2030 0x55c8517b8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.794+0000 7f78abfff700 1 -- 172.21.15.82:0/2385478093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8517bbe40 msgr2=0x55c8517b9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:27:58.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.794+0000 7f78abfff700 1 --2- 172.21.15.82:0/2385478093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8517bbe40 0x55c8517b9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.794+0000 7f78abfff700 1 -- 172.21.15.82:0/2385478093 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78ac1a23b0 con 0x55c8517b1710 2023-09-09T23:27:58.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.794+0000 7f78abfff700 1 --2- 172.21.15.82:0/2385478093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8517b1710 0x55c8517bb830 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f789c01fbf0 tx=0x7f789c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b133af9dabd11efb server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:27:58.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.794+0000 7f788b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:27:58.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.794+0000 7f788b7fe700 1 -- 172.21.15.82:0/2385478093 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c011e30 con 0x55c8517b1710 2023-09-09T23:27:58.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.794+0000 7f788b7fe700 1 -- 172.21.15.82:0/2385478093 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f789c004020 con 0x55c8517b1710 2023-09-09T23:27:58.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.794+0000 7f78be8dfb80 1 -- 172.21.15.82:0/2385478093 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c85165b4f0 con 0x55c8517b1710 2023-09-09T23:27:58.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.794+0000 7f788b7fe700 1 -- 172.21.15.82:0/2385478093 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7f789c004220 con 0x55c8517b1710 2023-09-09T23:27:58.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.794+0000 7f78be8dfb80 10 client.16422 fetch_fsmap learned FSMap version 127 2023-09-09T23:27:58.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.794+0000 7f78be8dfb80 1 -- 172.21.15.82:0/2385478093 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x7f78ac10cba0 con 0x55c8517b1710 2023-09-09T23:27:58.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.794+0000 7f788b7fe700 1 -- 172.21.15.82:0/2385478093 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c85165b4f0 con 0x55c8517b1710 2023-09-09T23:27:58.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.794+0000 7f788b7fe700 1 -- 172.21.15.82:0/2385478093 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 127) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f789c0039e0 con 0x55c8517b1710 2023-09-09T23:27:58.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.794+0000 7f78be8dfb80 10 client.16422 fetch_fsmap finished waiting for FSMap version 127 2023-09-09T23:27:58.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.794+0000 7f78be8dfb80 10 client.16422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:27:58.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.794+0000 7f78be8dfb80 20 client.16422 populate_metadata read hostname 'smithi082' 2023-09-09T23:27:58.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.794+0000 7f78be8dfb80 1 --2- 172.21.15.82:0/2385478093 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c8517cb990 0x55c8517cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:58.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.794+0000 7f78be8dfb80 4 client.16422 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:27:58.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.795+0000 7f78be8dfb80 1 -- 172.21.15.82:0/2385478093 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8517188a0 con 0x55c8517cb990 2023-09-09T23:27:58.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.795+0000 7f78ab7fe700 1 --2- 172.21.15.82:0/2385478093 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c8517cb990 0x55c8517cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:58.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.795+0000 7f78ab7fe700 1 --2- 172.21.15.82:0/2385478093 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c8517cb990 0x55c8517cddb0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88ef92932d43ef7f server_cookie=3bb59c57b180f9fc in_seq=0 out_seq=0 2023-09-09T23:27:58.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.795+0000 7f788b7fe700 10 client.16422 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:27:58.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.796+0000 7f788b7fe700 1 -- 172.21.15.82:0/2385478093 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c8517188a0 con 0x55c8517cb990 2023-09-09T23:27:58.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.796+0000 7f788b7fe700 10 client.16422 handle_command_reply: tid=0 2023-09-09T23:27:58.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.796+0000 7f78be8dfb80 1 client.16422 shutdown 2023-09-09T23:27:58.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.796+0000 7f78be8dfb80 1 -- 172.21.15.82:0/2385478093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8517b1710 msgr2=0x55c8517bb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:27:58.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.796+0000 7f78be8dfb80 1 --2- 172.21.15.82:0/2385478093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8517b1710 0x55c8517bb830 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f789c01fbf0 tx=0x7f789c0047c0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.816 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.796+0000 7f78be8dfb80 1 -- 172.21.15.82:0/2385478093 shutdown_connections 2023-09-09T23:27:58.816 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.796+0000 7f78be8dfb80 1 --2- 172.21.15.82:0/2385478093 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c8517cb990 0x55c8517cddb0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.816 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.796+0000 7f78be8dfb80 1 --2- 172.21.15.82:0/2385478093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8517bbe40 0x55c8517b9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.816 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.796+0000 7f788b7fe700 0 client.16422 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:27:58.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.796+0000 7f78be8dfb80 1 --2- 172.21.15.82:0/2385478093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8517b1710 0x55c8517bb830 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.796+0000 7f78be8dfb80 1 --2- 172.21.15.82:0/2385478093 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8517b2030 0x55c8517b8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.796+0000 7f78be8dfb80 1 -- 172.21.15.82:0/2385478093 >> 172.21.15.82:0/2385478093 conn(0x55c851726090 msgr2=0x55c8517ac7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:27:58.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.796+0000 7f78be8dfb80 1 -- 172.21.15.82:0/2385478093 shutdown_connections 2023-09-09T23:27:58.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.796+0000 7f78be8dfb80 1 -- 172.21.15.82:0/2385478093 wait complete. 2023-09-09T23:27:58.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.796+0000 7f78be8dfb80 20 client.16422 trim_cache size 0 max 16384 2023-09-09T23:27:58.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.812+0000 7f78be8dfb80 1 Processor -- start 2023-09-09T23:27:58.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.812+0000 7f78be8dfb80 1 -- start start 2023-09-09T23:27:58.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.812+0000 7f78be8dfb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8517b1710 0x55c8517c29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:58.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.812+0000 7f78be8dfb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8517b2030 0x55c8517bb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:58.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.812+0000 7f78be8dfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8517bbe40 0x55c8517ac6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:58.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.812+0000 7f78be8dfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8517cb770 con 0x55c8517bbe40 2023-09-09T23:27:58.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.812+0000 7f78be8dfb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8517c2f10 con 0x55c8517b1710 2023-09-09T23:27:58.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.812+0000 7f78be8dfb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8517b2410 con 0x55c8517b2030 2023-09-09T23:27:58.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.813+0000 7f78b0cb6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8517bbe40 0x55c8517ac6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:58.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.813+0000 7f78abfff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8517b1710 0x55c8517c29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:58.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.813+0000 7f78b0cb6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8517bbe40 0x55c8517ac6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58630/0 (socket says 172.21.15.82:58630) 2023-09-09T23:27:58.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.813+0000 7f78b0cb6700 1 -- 172.21.15.82:0/3685894999 learned_addr learned my addr 172.21.15.82:0/3685894999 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:27:58.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.813+0000 7f78ab7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8517b2030 0x55c8517bb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:58.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.813+0000 7f78b0cb6700 1 -- 172.21.15.82:0/3685894999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8517b2030 msgr2=0x55c8517bb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:27:58.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.813+0000 7f78b0cb6700 1 --2- 172.21.15.82:0/3685894999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8517b2030 0x55c8517bb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.813+0000 7f78b0cb6700 1 -- 172.21.15.82:0/3685894999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8517b1710 msgr2=0x55c8517c29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:27:58.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.813+0000 7f78b0cb6700 1 --2- 172.21.15.82:0/3685894999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8517b1710 0x55c8517c29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.813+0000 7f78b0cb6700 1 -- 172.21.15.82:0/3685894999 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c851518c60 con 0x55c8517bbe40 2023-09-09T23:27:58.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.813+0000 7f78b0cb6700 1 --2- 172.21.15.82:0/3685894999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8517bbe40 0x55c8517ac6f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f78a40c93c0 tx=0x7f78a409f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0899862f0c47e7c server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:27:58.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.813+0000 7f7892ffd700 1 -- 172.21.15.82:0/3685894999 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a401b320 con 0x55c8517bbe40 2023-09-09T23:27:58.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.813+0000 7f7892ffd700 1 -- 172.21.15.82:0/3685894999 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78a4099850 con 0x55c8517bbe40 2023-09-09T23:27:58.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.813+0000 7f7892ffd700 1 -- 172.21.15.82:0/3685894999 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a40993b0 con 0x55c8517bbe40 2023-09-09T23:27:58.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.814+0000 7f78be8dfb80 1 -- 172.21.15.82:0/3685894999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8517bbe40 msgr2=0x55c8517ac6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:27:58.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.814+0000 7f78be8dfb80 1 --2- 172.21.15.82:0/3685894999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8517bbe40 0x55c8517ac6f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f78a40c93c0 tx=0x7f78a409f040 comp rx=0 tx=0).stop 2023-09-09T23:27:58.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.814+0000 7f78be8dfb80 1 -- 172.21.15.82:0/3685894999 shutdown_connections 2023-09-09T23:27:58.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.814+0000 7f78be8dfb80 1 --2- 172.21.15.82:0/3685894999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8517b1710 0x55c8517c29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.814+0000 7f78be8dfb80 1 --2- 172.21.15.82:0/3685894999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8517bbe40 0x55c8517ac6f0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.814+0000 7f78be8dfb80 1 --2- 172.21.15.82:0/3685894999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8517b2030 0x55c8517bb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.814+0000 7f78be8dfb80 1 -- 172.21.15.82:0/3685894999 >> 172.21.15.82:0/3685894999 conn(0x55c851726090 msgr2=0x55c8517b3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:27:58.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.814+0000 7f78be8dfb80 1 -- 172.21.15.82:0/3685894999 shutdown_connections 2023-09-09T23:27:58.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.814+0000 7f78be8dfb80 1 -- 172.21.15.82:0/3685894999 wait complete. 2023-09-09T23:27:58.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.814+0000 7f78be8dfb80 1 -- start start 2023-09-09T23:27:58.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.814+0000 7f78be8dfb80 1 Processor -- start 2023-09-09T23:27:58.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.814+0000 7f78be8dfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8517b1710 0x55c85181b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:58.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.814+0000 7f78be8dfb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8517b2030 0x55c85181b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:58.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.814+0000 7f78be8dfb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8517bbe40 0x55c851815760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:58.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.814+0000 7f78be8dfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c851815d40 con 0x55c8517b1710 2023-09-09T23:27:58.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.814+0000 7f78be8dfb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c851815eb0 con 0x55c8517b2030 2023-09-09T23:27:58.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.814+0000 7f78be8dfb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c851816070 con 0x55c8517bbe40 2023-09-09T23:27:58.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.814+0000 7f78be8dfb80 10 client.0 _ll_register_callbacks cb 0x7f789c01f7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:27:58.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.815+0000 7f78abfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8517b1710 0x55c85181b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:58.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.815+0000 7f78abfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8517b1710 0x55c85181b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58632/0 (socket says 172.21.15.82:58632) 2023-09-09T23:27:58.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.815+0000 7f78abfff700 1 -- 172.21.15.82:0/2066688536 learned_addr learned my addr 172.21.15.82:0/2066688536 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:27:58.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.815+0000 7f78ab7fe700 1 --2- 172.21.15.82:0/2066688536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8517b2030 0x55c85181b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:58.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.815+0000 7f78b0cb6700 1 --2- 172.21.15.82:0/2066688536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8517bbe40 0x55c851815760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:58.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.815+0000 7f78abfff700 1 -- 172.21.15.82:0/2066688536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8517bbe40 msgr2=0x55c851815760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:27:58.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.815+0000 7f78abfff700 1 --2- 172.21.15.82:0/2066688536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8517bbe40 0x55c851815760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.815+0000 7f78abfff700 1 -- 172.21.15.82:0/2066688536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8517b2030 msgr2=0x55c85181b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:27:58.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.815+0000 7f78abfff700 1 --2- 172.21.15.82:0/2066688536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8517b2030 0x55c85181b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.815+0000 7f78abfff700 1 -- 172.21.15.82:0/2066688536 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78ac19ab10 con 0x55c8517b1710 2023-09-09T23:27:58.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.815+0000 7f78b0cb6700 1 --2- 172.21.15.82:0/2066688536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8517bbe40 0x55c851815760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-09T23:27:58.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.815+0000 7f78abfff700 1 --2- 172.21.15.82:0/2066688536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8517b1710 0x55c85181b1b0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f789c0117d0 tx=0x7f789c0038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4e3d3ed7f6a88d server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:27:58.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.815+0000 7f788b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:27:58.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.815+0000 7f788b7fe700 1 -- 172.21.15.82:0/2066688536 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c004200 con 0x55c8517b1710 2023-09-09T23:27:58.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.815+0000 7f78be8dfb80 1 -- 172.21.15.82:0/2066688536 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f78a001e040 con 0x55c8517b1710 2023-09-09T23:27:58.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.815+0000 7f788b7fe700 1 -- 172.21.15.82:0/2066688536 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f789c004360 con 0x55c8517b1710 2023-09-09T23:27:58.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.815+0000 7f788b7fe700 1 -- 172.21.15.82:0/2066688536 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7f789c004520 con 0x55c8517b1710 2023-09-09T23:27:58.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.815+0000 7f78be8dfb80 10 client.16428 fetch_fsmap learned FSMap version 127 2023-09-09T23:27:58.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.815+0000 7f78be8dfb80 1 -- 172.21.15.82:0/2066688536 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x55c851556250 con 0x55c8517b1710 2023-09-09T23:27:58.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.815+0000 7f788b7fe700 1 -- 172.21.15.82:0/2066688536 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f789c046020 con 0x55c8517b1710 2023-09-09T23:27:58.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.816+0000 7f788b7fe700 1 -- 172.21.15.82:0/2066688536 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 127) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f789c01d930 con 0x55c8517b1710 2023-09-09T23:27:58.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.816+0000 7f78be8dfb80 10 client.16428 fetch_fsmap finished waiting for FSMap version 127 2023-09-09T23:27:58.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.816+0000 7f78be8dfb80 10 client.16428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:27:58.836 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.816+0000 7f78be8dfb80 20 client.16428 populate_metadata read hostname 'smithi082' 2023-09-09T23:27:58.836 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.816+0000 7f78be8dfb80 1 --2- 172.21.15.82:0/2066688536 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c851822f90 0x55c851825380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:27:58.836 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.816+0000 7f78be8dfb80 4 client.16428 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:27:58.836 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.816+0000 7f78be8dfb80 1 -- 172.21.15.82:0/2066688536 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c8516fc920 con 0x55c851822f90 2023-09-09T23:27:58.836 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.816+0000 7f78ab7fe700 1 --2- 172.21.15.82:0/2066688536 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c851822f90 0x55c851825380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:27:58.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.816+0000 7f788b7fe700 10 client.16428 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:27:58.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.816+0000 7f78ab7fe700 1 --2- 172.21.15.82:0/2066688536 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c851822f90 0x55c851825380 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=293b47e540eda4c9 server_cookie=b2ba36f573f647fc in_seq=0 out_seq=0 2023-09-09T23:27:58.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.816+0000 7f788b7fe700 1 -- 172.21.15.82:0/2066688536 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55c8516fc920 con 0x55c851822f90 2023-09-09T23:27:58.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.816+0000 7f788b7fe700 10 client.16428 handle_command_reply: tid=0 2023-09-09T23:27:58.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.816+0000 7f78be8dfb80 1 client.16428 shutdown 2023-09-09T23:27:58.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.816+0000 7f78be8dfb80 1 -- 172.21.15.82:0/2066688536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8517b1710 msgr2=0x55c85181b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:27:58.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.816+0000 7f78be8dfb80 1 --2- 172.21.15.82:0/2066688536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8517b1710 0x55c85181b1b0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f789c0117d0 tx=0x7f789c0038b0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.816+0000 7f78be8dfb80 1 -- 172.21.15.82:0/2066688536 shutdown_connections 2023-09-09T23:27:58.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.816+0000 7f78be8dfb80 1 --2- 172.21.15.82:0/2066688536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c8517bbe40 0x55c851815760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.816+0000 7f78be8dfb80 1 --2- 172.21.15.82:0/2066688536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8517b1710 0x55c85181b1b0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.816+0000 7f78be8dfb80 1 --2- 172.21.15.82:0/2066688536 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c851822f90 0x55c851825380 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.816+0000 7f78be8dfb80 1 --2- 172.21.15.82:0/2066688536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8517b2030 0x55c85181b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.816+0000 7f788b7fe700 0 client.16428 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:27:58.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.816+0000 7f78be8dfb80 1 -- 172.21.15.82:0/2066688536 >> 172.21.15.82:0/2066688536 conn(0x55c851726090 msgr2=0x55c8517ac670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:27:58.839 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.817+0000 7f78be8dfb80 1 -- 172.21.15.82:0/2066688536 shutdown_connections 2023-09-09T23:27:58.840 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.817+0000 7f78be8dfb80 1 -- 172.21.15.82:0/2066688536 wait complete. 2023-09-09T23:27:58.840 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.817+0000 7f78be8dfb80 20 client.16428 trim_cache size 0 max 16384 2023-09-09T23:27:58.841 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:27:58.841 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:27:58.841 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:27:58.841 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:27:58.841 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:27:58.842 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:27:58.842 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:27:58.842 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:27:58.843 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:27:58.843 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:27:58.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.817+0000 7f78b2719700 1 -- 172.21.15.82:0/147429603 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f78940a07b0 msgr2=0x7f78940a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:27:58.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.817+0000 7f78b2719700 1 --2- 172.21.15.82:0/147429603 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f78940a07b0 0x7f78940a2c70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f789c009be0 tx=0x7f789c009260 comp rx=0 tx=0).stop 2023-09-09T23:27:58.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.817+0000 7f78b2719700 1 -- 172.21.15.82:0/147429603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78ac1054f0 msgr2=0x7f78ac199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:27:58.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.817+0000 7f78b2719700 1 --2- 172.21.15.82:0/147429603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78ac1054f0 0x7f78ac199f20 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f78a400ef30 tx=0x7f78a400c3a0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.817+0000 7f78b2719700 1 -- 172.21.15.82:0/147429603 shutdown_connections 2023-09-09T23:27:58.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.817+0000 7f78b2719700 1 --2- 172.21.15.82:0/147429603 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f78940a07b0 0x7f78940a2c70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.817+0000 7f78b2719700 1 --2- 172.21.15.82:0/147429603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78ac0ff5d0 0x7f78ac195580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.817+0000 7f78b2719700 1 --2- 172.21.15.82:0/147429603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78ac1054f0 0x7f78ac199f20 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.817+0000 7f78b2719700 1 --2- 172.21.15.82:0/147429603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78ac0fff90 0x7f78ac195ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:27:58.846 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.817+0000 7f78b2719700 1 -- 172.21.15.82:0/147429603 >> 172.21.15.82:0/147429603 conn(0x7f78ac075020 msgr2=0x7f78ac0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:27:58.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.817+0000 7f78b2719700 1 -- 172.21.15.82:0/147429603 shutdown_connections 2023-09-09T23:27:58.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:27:58.817+0000 7f78b2719700 1 -- 172.21.15.82:0/147429603 wait complete. 2023-09-09T23:27:58.847 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:27:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 510.025224 secs 2023-09-09T23:27:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:58 smithi170 ceph-mon[102832]: pgmap v441: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2023-09-09T23:27:58.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 510.025224 secs 2023-09-09T23:27:58.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:58 smithi082 ceph-mon[98694]: pgmap v441: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2023-09-09T23:27:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 510.025224 secs 2023-09-09T23:27:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:58 smithi098 ceph-mon[103124]: pgmap v441: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2023-09-09T23:27:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:27:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 510.754224 secs 2023-09-09T23:27:59.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:27:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 510.754224 secs 2023-09-09T23:28:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:27:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 510.754224 secs 2023-09-09T23:28:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:00 smithi170 ceph-mon[102832]: pgmap v442: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2023-09-09T23:28:00.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:00 smithi082 ceph-mon[98694]: pgmap v442: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2023-09-09T23:28:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:00 smithi098 ceph-mon[103124]: pgmap v442: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2023-09-09T23:28:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:01 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 513.425118 secs 2023-09-09T23:28:02.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:01 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 513.425118 secs 2023-09-09T23:28:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:01 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 513.425118 secs 2023-09-09T23:28:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:02 smithi170 ceph-mon[102832]: pgmap v443: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-09-09T23:28:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:02 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 515.025284 secs 2023-09-09T23:28:03.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:02 smithi082 ceph-mon[98694]: pgmap v443: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-09-09T23:28:03.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:02 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 515.025284 secs 2023-09-09T23:28:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:02 smithi098 ceph-mon[103124]: pgmap v443: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-09-09T23:28:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:02 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 515.025284 secs 2023-09-09T23:28:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 515.754288 secs 2023-09-09T23:28:04.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 515.754288 secs 2023-09-09T23:28:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 515.754288 secs 2023-09-09T23:28:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:05 smithi170 ceph-mon[102832]: pgmap v444: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-09-09T23:28:05.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:05 smithi082 ceph-mon[98694]: pgmap v444: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-09-09T23:28:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:05 smithi098 ceph-mon[103124]: pgmap v444: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 21 op/s 2023-09-09T23:28:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:07 smithi170 ceph-mon[102832]: pgmap v445: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 49 KiB/s wr, 28 op/s 2023-09-09T23:28:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 518.425204 secs 2023-09-09T23:28:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:07 smithi082 ceph-mon[98694]: pgmap v445: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 49 KiB/s wr, 28 op/s 2023-09-09T23:28:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 518.425204 secs 2023-09-09T23:28:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:07 smithi098 ceph-mon[103124]: pgmap v445: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 49 KiB/s wr, 28 op/s 2023-09-09T23:28:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 518.425204 secs 2023-09-09T23:28:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:08 smithi170 ceph-mon[102832]: osdmap e101: 12 total, 12 up, 12 in 2023-09-09T23:28:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 520.025343 secs 2023-09-09T23:28:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:08 smithi082 ceph-mon[98694]: osdmap e101: 12 total, 12 up, 12 in 2023-09-09T23:28:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 520.025343 secs 2023-09-09T23:28:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:08 smithi098 ceph-mon[103124]: osdmap e101: 12 total, 12 up, 12 in 2023-09-09T23:28:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 520.025343 secs 2023-09-09T23:28:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:09 smithi170 ceph-mon[102832]: pgmap v447: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:28:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 520.754355 secs 2023-09-09T23:28:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:09 smithi082 ceph-mon[98694]: pgmap v447: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:28:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 520.754355 secs 2023-09-09T23:28:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:09 smithi098 ceph-mon[103124]: pgmap v447: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:28:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 520.754355 secs 2023-09-09T23:28:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:11 smithi170 ceph-mon[102832]: pgmap v448: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:28:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:11 smithi082 ceph-mon[98694]: pgmap v448: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:28:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:11 smithi098 ceph-mon[103124]: pgmap v448: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:28:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 523.425264 secs 2023-09-09T23:28:12.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 523.425264 secs 2023-09-09T23:28:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 523.425264 secs 2023-09-09T23:28:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:13 smithi170 ceph-mon[102832]: pgmap v449: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 28 KiB/s wr, 13 op/s 2023-09-09T23:28:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 525.025402 secs 2023-09-09T23:28:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:13 smithi170 ceph-mon[102832]: osdmap e102: 12 total, 12 up, 12 in 2023-09-09T23:28:13.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:13 smithi082 ceph-mon[98694]: pgmap v449: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 28 KiB/s wr, 13 op/s 2023-09-09T23:28:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 525.025402 secs 2023-09-09T23:28:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:13 smithi082 ceph-mon[98694]: osdmap e102: 12 total, 12 up, 12 in 2023-09-09T23:28:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:13 smithi098 ceph-mon[103124]: pgmap v449: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 28 KiB/s wr, 13 op/s 2023-09-09T23:28:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 525.025402 secs 2023-09-09T23:28:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:13 smithi098 ceph-mon[103124]: osdmap e102: 12 total, 12 up, 12 in 2023-09-09T23:28:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 525.754423 secs 2023-09-09T23:28:14.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 525.754423 secs 2023-09-09T23:28:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 525.754423 secs 2023-09-09T23:28:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:15 smithi170 ceph-mon[102832]: pgmap v451: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-09T23:28:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:15 smithi082 ceph-mon[98694]: pgmap v451: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-09T23:28:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:15 smithi098 ceph-mon[103124]: pgmap v451: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-09T23:28:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:17 smithi170 ceph-mon[102832]: pgmap v452: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 KiB/s wr, 13 op/s 2023-09-09T23:28:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 528.425425 secs 2023-09-09T23:28:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:17 smithi082 ceph-mon[98694]: pgmap v452: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 KiB/s wr, 13 op/s 2023-09-09T23:28:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 528.425425 secs 2023-09-09T23:28:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:17 smithi098 ceph-mon[103124]: pgmap v452: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 KiB/s wr, 13 op/s 2023-09-09T23:28:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 528.425425 secs 2023-09-09T23:28:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 530.025477 secs 2023-09-09T23:28:18.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 530.025477 secs 2023-09-09T23:28:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 530.025477 secs 2023-09-09T23:28:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:19 smithi170 ceph-mon[102832]: pgmap v453: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 KiB/s wr, 11 op/s 2023-09-09T23:28:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 530.754490 secs 2023-09-09T23:28:19.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:19 smithi082 ceph-mon[98694]: pgmap v453: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 KiB/s wr, 11 op/s 2023-09-09T23:28:19.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 530.754490 secs 2023-09-09T23:28:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:19 smithi098 ceph-mon[103124]: pgmap v453: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 KiB/s wr, 11 op/s 2023-09-09T23:28:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 530.754490 secs 2023-09-09T23:28:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:28:20.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:28:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:28:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:21 smithi170 ceph-mon[102832]: pgmap v454: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-09T23:28:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:28:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:28:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:28:21.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:21 smithi082 ceph-mon[98694]: pgmap v454: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-09T23:28:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:28:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:28:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:28:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:21 smithi098 ceph-mon[103124]: pgmap v454: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-09T23:28:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:28:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:28:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:28:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 533.425484 secs 2023-09-09T23:28:22.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 533.425484 secs 2023-09-09T23:28:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 533.425484 secs 2023-09-09T23:28:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:23 smithi170 ceph-mon[102832]: pgmap v455: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 14 op/s 2023-09-09T23:28:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 535.025535 secs 2023-09-09T23:28:23.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:23 smithi082 ceph-mon[98694]: pgmap v455: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 14 op/s 2023-09-09T23:28:23.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 535.025535 secs 2023-09-09T23:28:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:23 smithi098 ceph-mon[103124]: pgmap v455: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 14 op/s 2023-09-09T23:28:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 535.025535 secs 2023-09-09T23:28:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 535.754556 secs 2023-09-09T23:28:24.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 535.754556 secs 2023-09-09T23:28:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 535.754556 secs 2023-09-09T23:28:25.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:25 smithi170 ceph-mon[102832]: pgmap v456: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 13 op/s 2023-09-09T23:28:25.969 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:25 smithi082 ceph-mon[98694]: pgmap v456: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 13 op/s 2023-09-09T23:28:26.009 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:25 smithi098 ceph-mon[103124]: pgmap v456: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 13 op/s 2023-09-09T23:28:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:26 smithi170 ceph-mon[102832]: pgmap v457: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 22 KiB/s wr, 16 op/s 2023-09-09T23:28:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:26 smithi082 ceph-mon[98694]: pgmap v457: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 22 KiB/s wr, 16 op/s 2023-09-09T23:28:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:26 smithi098 ceph-mon[103124]: pgmap v457: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 22 KiB/s wr, 16 op/s 2023-09-09T23:28:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 538.425570 secs 2023-09-09T23:28:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 538.425570 secs 2023-09-09T23:28:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 538.425570 secs 2023-09-09T23:28:28.849 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:28:28.878 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 540.025595 secs 2023-09-09T23:28:28.878 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:28 smithi082 ceph-mon[98694]: pgmap v458: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 11 op/s 2023-09-09T23:28:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 540.025595 secs 2023-09-09T23:28:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:28 smithi170 ceph-mon[102832]: pgmap v458: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 11 op/s 2023-09-09T23:28:28.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.985+0000 7fc16cdf6700 1 -- 172.21.15.82:0/2751231153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1680ff3c0 msgr2=0x7fc16810d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:28:28.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.985+0000 7fc16cdf6700 1 --2- 172.21.15.82:0/2751231153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1680ff3c0 0x7fc16810d470 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fc16000b0d0 tx=0x7fc16000b3e0 comp rx=0 tx=0).stop 2023-09-09T23:28:28.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.985+0000 7fc16cdf6700 1 -- 172.21.15.82:0/2751231153 shutdown_connections 2023-09-09T23:28:28.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.985+0000 7fc16cdf6700 1 --2- 172.21.15.82:0/2751231153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc168105570 0x7fc168105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:28.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.985+0000 7fc16cdf6700 1 --2- 172.21.15.82:0/2751231153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1680ff3c0 0x7fc16810d470 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:28.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.985+0000 7fc16cdf6700 1 --2- 172.21.15.82:0/2751231153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1680fea00 0x7fc1680fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:28.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.985+0000 7fc16cdf6700 1 -- 172.21.15.82:0/2751231153 >> 172.21.15.82:0/2751231153 conn(0x7fc168076a70 msgr2=0x7fc1680fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:28:28.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.985+0000 7fc16cdf6700 1 -- 172.21.15.82:0/2751231153 shutdown_connections 2023-09-09T23:28:28.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.985+0000 7fc16cdf6700 1 -- 172.21.15.82:0/2751231153 wait complete. 2023-09-09T23:28:28.990 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.986+0000 7fc16cdf6700 1 Processor -- start 2023-09-09T23:28:28.990 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.986+0000 7fc16cdf6700 1 -- start start 2023-09-09T23:28:28.990 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.986+0000 7fc16cdf6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1680fea00 0x7fc16819c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:28.990 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.986+0000 7fc16cdf6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1680ff3c0 0x7fc16819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:28.991 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.986+0000 7fc16cdf6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc168105570 0x7fc168196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:28.991 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.986+0000 7fc16cdf6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc168196bf0 con 0x7fc168105570 2023-09-09T23:28:28.991 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.986+0000 7fc16cdf6700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc168196d60 con 0x7fc1680fea00 2023-09-09T23:28:28.991 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.986+0000 7fc16cdf6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc168196f20 con 0x7fc1680ff3c0 2023-09-09T23:28:28.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.986+0000 7fc166d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc168105570 0x7fc168196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:28.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.986+0000 7fc16659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1680fea00 0x7fc16819c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:28.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.987+0000 7fc166d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc168105570 0x7fc168196580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58652/0 (socket says 172.21.15.82:58652) 2023-09-09T23:28:28.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.987+0000 7fc16659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1680fea00 0x7fc16819c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:44212/0 (socket says 172.21.15.82:44212) 2023-09-09T23:28:28.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.987+0000 7fc166d9d700 1 -- 172.21.15.82:0/1660340885 learned_addr learned my addr 172.21.15.82:0/1660340885 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:28:28.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.987+0000 7fc165d9b700 1 --2- 172.21.15.82:0/1660340885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1680ff3c0 0x7fc16819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:28.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.987+0000 7fc166d9d700 1 -- 172.21.15.82:0/1660340885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1680ff3c0 msgr2=0x7fc16819ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:28:28.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.987+0000 7fc166d9d700 1 --2- 172.21.15.82:0/1660340885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1680ff3c0 0x7fc16819ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:28.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.987+0000 7fc166d9d700 1 -- 172.21.15.82:0/1660340885 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1680fea00 msgr2=0x7fc16819c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:28:28.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.987+0000 7fc166d9d700 1 --2- 172.21.15.82:0/1660340885 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1680fea00 0x7fc16819c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:28.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.987+0000 7fc166d9d700 1 -- 172.21.15.82:0/1660340885 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc160009d00 con 0x7fc168105570 2023-09-09T23:28:28.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.987+0000 7fc166d9d700 1 --2- 172.21.15.82:0/1660340885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc168105570 0x7fc168196580 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fc160007870 tx=0x7fc1600078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:28:28.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.987+0000 7fc1577fe700 1 -- 172.21.15.82:0/1660340885 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16000e040 con 0x7fc168105570 2023-09-09T23:28:28.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.987+0000 7fc16cdf6700 1 -- 172.21.15.82:0/1660340885 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1681971a0 con 0x7fc168105570 2023-09-09T23:28:28.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.987+0000 7fc1577fe700 1 -- 172.21.15.82:0/1660340885 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc160005070 con 0x7fc168105570 2023-09-09T23:28:28.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.987+0000 7fc1577fe700 1 -- 172.21.15.82:0/1660340885 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16001b6c0 con 0x7fc168105570 2023-09-09T23:28:28.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.987+0000 7fc16cdf6700 1 -- 172.21.15.82:0/1660340885 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1681a3500 con 0x7fc168105570 2023-09-09T23:28:28.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.989+0000 7fc1577fe700 1 -- 172.21.15.82:0/1660340885 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) v1 ==== 135390+0+0 (secure 0 0 0) 0x7fc1600051e0 con 0x7fc168105570 2023-09-09T23:28:28.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.990+0000 7fc1577fe700 1 --2- 172.21.15.82:0/1660340885 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fc1500a07b0 0x7fc1500a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:28.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.990+0000 7fc1577fe700 1 -- 172.21.15.82:0/1660340885 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fc1600cd060 con 0x7fc168105570 2023-09-09T23:28:28.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.990+0000 7fc16659c700 1 --2- 172.21.15.82:0/1660340885 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fc1500a07b0 0x7fc1500a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:29.001 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.990+0000 7fc16659c700 1 --2- 172.21.15.82:0/1660340885 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fc1500a07b0 0x7fc1500a2c70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc15c009c60 tx=0x7fc15c009260 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:28:29.001 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.991+0000 7fc178fbcb80 1 Processor -- start 2023-09-09T23:28:29.002 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.991+0000 7fc178fbcb80 1 -- start start 2023-09-09T23:28:29.002 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.991+0000 7fc178fbcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9e6a61710 0x55c9e6a61af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:29.002 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.991+0000 7fc178fbcb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9e6a62030 0x55c9e6a6b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:29.002 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.991+0000 7fc178fbcb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9e6a6be40 0x55c9e6a6e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:29.002 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.991+0000 7fc178fbcb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e69ba730 con 0x55c9e6a62030 2023-09-09T23:28:29.003 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.991+0000 7fc178fbcb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e69b8050 con 0x55c9e6a6be40 2023-09-09T23:28:29.003 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.991+0000 7fc178fbcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e69b3940 con 0x55c9e6a61710 2023-09-09T23:28:29.003 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.991+0000 7fc165d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9e6a62030 0x55c9e6a6b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:29.003 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.991+0000 7fc165d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9e6a62030 0x55c9e6a6b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58658/0 (socket says 172.21.15.82:58658) 2023-09-09T23:28:29.003 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.991+0000 7fc165d9b700 1 -- 172.21.15.82:0/2308931996 learned_addr learned my addr 172.21.15.82:0/2308931996 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:28:29.004 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.991+0000 7fc16659c700 1 --2- 172.21.15.82:0/2308931996 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9e6a61710 0x55c9e6a61af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:29.004 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.991+0000 7fc166d9d700 1 --2- 172.21.15.82:0/2308931996 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9e6a6be40 0x55c9e6a6e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:29.004 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.991+0000 7fc165d9b700 1 -- 172.21.15.82:0/2308931996 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9e6a61710 msgr2=0x55c9e6a61af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:28:29.004 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.991+0000 7fc165d9b700 1 --2- 172.21.15.82:0/2308931996 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9e6a61710 0x55c9e6a61af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.004 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.991+0000 7fc165d9b700 1 -- 172.21.15.82:0/2308931996 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9e6a6be40 msgr2=0x55c9e6a6e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:28:29.004 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.991+0000 7fc165d9b700 1 --2- 172.21.15.82:0/2308931996 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9e6a6be40 0x55c9e6a6e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.005 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.991+0000 7fc165d9b700 1 -- 172.21.15.82:0/2308931996 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9e67dfde0 con 0x55c9e6a62030 2023-09-09T23:28:29.005 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.992+0000 7fc165d9b700 1 --2- 172.21.15.82:0/2308931996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9e6a62030 0x55c9e6a6b900 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc158009d20 tx=0x7fc158011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab6ca550a602b7cc server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:28:29.005 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.992+0000 7fc154ff9700 1 -- 172.21.15.82:0/2308931996 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc158019630 con 0x55c9e6a62030 2023-09-09T23:28:29.005 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.992+0000 7fc154ff9700 1 -- 172.21.15.82:0/2308931996 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc158019c50 con 0x55c9e6a62030 2023-09-09T23:28:29.005 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.992+0000 7fc154ff9700 1 -- 172.21.15.82:0/2308931996 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15800ebe0 con 0x55c9e6a62030 2023-09-09T23:28:29.006 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.992+0000 7fc178fbcb80 1 -- 172.21.15.82:0/2308931996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9e6a62030 msgr2=0x55c9e6a6b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:28:29.006 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.992+0000 7fc178fbcb80 1 --2- 172.21.15.82:0/2308931996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9e6a62030 0x55c9e6a6b900 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc158009d20 tx=0x7fc158011720 comp rx=0 tx=0).stop 2023-09-09T23:28:29.006 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.992+0000 7fc178fbcb80 1 -- 172.21.15.82:0/2308931996 shutdown_connections 2023-09-09T23:28:29.006 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.992+0000 7fc178fbcb80 1 --2- 172.21.15.82:0/2308931996 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9e6a6be40 0x55c9e6a6e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.006 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.992+0000 7fc178fbcb80 1 --2- 172.21.15.82:0/2308931996 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9e6a61710 0x55c9e6a61af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.007 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.992+0000 7fc178fbcb80 1 --2- 172.21.15.82:0/2308931996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9e6a62030 0x55c9e6a6b900 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.007 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.992+0000 7fc178fbcb80 1 -- 172.21.15.82:0/2308931996 >> 172.21.15.82:0/2308931996 conn(0x55c9e69d6090 msgr2=0x55c9e6a5cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:28:29.007 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.993+0000 7fc178fbcb80 1 -- 172.21.15.82:0/2308931996 shutdown_connections 2023-09-09T23:28:29.007 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.993+0000 7fc178fbcb80 1 -- 172.21.15.82:0/2308931996 wait complete. 2023-09-09T23:28:29.007 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.993+0000 7fc178fbcb80 1 -- start start 2023-09-09T23:28:29.008 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.993+0000 7fc178fbcb80 1 Processor -- start 2023-09-09T23:28:29.008 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.993+0000 7fc178fbcb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9e6a61710 0x55c9e6a6b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:29.008 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.993+0000 7fc178fbcb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9e6a62030 0x55c9e6a68a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:29.008 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.993+0000 7fc178fbcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9e6a6be40 0x55c9e6a69050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:29.008 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.993+0000 7fc178fbcb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e6951b70 con 0x55c9e6a61710 2023-09-09T23:28:29.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.993+0000 7fc178fbcb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e6951870 con 0x55c9e6a62030 2023-09-09T23:28:29.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.993+0000 7fc178fbcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e6835920 con 0x55c9e6a6be40 2023-09-09T23:28:29.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.994+0000 7fc16659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9e6a61710 0x55c9e6a6b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:29.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.994+0000 7fc16659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9e6a61710 0x55c9e6a6b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58662/0 (socket says 172.21.15.82:58662) 2023-09-09T23:28:29.010 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.994+0000 7fc16659c700 1 -- 172.21.15.82:0/494714171 learned_addr learned my addr 172.21.15.82:0/494714171 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:28:29.010 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.994+0000 7fc178fbcb80 10 client.0 _ll_register_callbacks cb 0x55c9e689ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:28:29.010 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.994+0000 7fc165d9b700 1 --2- 172.21.15.82:0/494714171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9e6a62030 0x55c9e6a68a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:29.010 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.994+0000 7fc166d9d700 1 --2- 172.21.15.82:0/494714171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9e6a6be40 0x55c9e6a69050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:29.010 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.994+0000 7fc16659c700 1 -- 172.21.15.82:0/494714171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9e6a6be40 msgr2=0x55c9e6a69050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:28:29.011 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.994+0000 7fc16659c700 1 --2- 172.21.15.82:0/494714171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9e6a6be40 0x55c9e6a69050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.011 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.994+0000 7fc16659c700 1 -- 172.21.15.82:0/494714171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9e6a62030 msgr2=0x55c9e6a68a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:28:29.011 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.994+0000 7fc16659c700 1 --2- 172.21.15.82:0/494714171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9e6a62030 0x55c9e6a68a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.011 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.994+0000 7fc16659c700 1 -- 172.21.15.82:0/494714171 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc158009980 con 0x55c9e6a61710 2023-09-09T23:28:29.011 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.994+0000 7fc16659c700 1 --2- 172.21.15.82:0/494714171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9e6a61710 0x55c9e6a6b830 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fc15c005910 tx=0x7fc15c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc10f3dd89e8784 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:28:29.012 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.994+0000 7fc14dffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:28:29.012 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.994+0000 7fc14dffb700 1 -- 172.21.15.82:0/494714171 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c022070 con 0x55c9e6a61710 2023-09-09T23:28:29.012 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.994+0000 7fc14dffb700 1 -- 172.21.15.82:0/494714171 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fc15c01d720 con 0x55c9e6a61710 2023-09-09T23:28:29.012 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.994+0000 7fc178fbcb80 1 -- 172.21.15.82:0/494714171 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c9e690b4f0 con 0x55c9e6a61710 2023-09-09T23:28:29.012 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.994+0000 7fc14dffb700 1 -- 172.21.15.82:0/494714171 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc15c01d8e0 con 0x55c9e6a61710 2023-09-09T23:28:29.013 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.994+0000 7fc178fbcb80 10 client.16440 fetch_fsmap learned FSMap version 127 2023-09-09T23:28:29.013 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.994+0000 7fc178fbcb80 1 -- 172.21.15.82:0/494714171 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x55c9e67c8c60 con 0x55c9e6a61710 2023-09-09T23:28:29.013 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.994+0000 7fc14dffb700 1 -- 172.21.15.82:0/494714171 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c9e690b4f0 con 0x55c9e6a61710 2023-09-09T23:28:29.013 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.995+0000 7fc14dffb700 1 -- 172.21.15.82:0/494714171 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 127) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc15c01cad0 con 0x55c9e6a61710 2023-09-09T23:28:29.013 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.995+0000 7fc178fbcb80 10 client.16440 fetch_fsmap finished waiting for FSMap version 127 2023-09-09T23:28:29.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.995+0000 7fc178fbcb80 10 client.16440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:28:29.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.995+0000 7fc178fbcb80 20 client.16440 populate_metadata read hostname 'smithi082' 2023-09-09T23:28:29.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.995+0000 7fc178fbcb80 1 --2- 172.21.15.82:0/494714171 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c9e6a7b9c0 0x55c9e6a7dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:29.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.995+0000 7fc178fbcb80 4 client.16440 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:28:29.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.995+0000 7fc178fbcb80 1 -- 172.21.15.82:0/494714171 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9e69c88a0 con 0x55c9e6a7b9c0 2023-09-09T23:28:29.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.995+0000 7fc165d9b700 1 --2- 172.21.15.82:0/494714171 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c9e6a7b9c0 0x55c9e6a7dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:29.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.995+0000 7fc14dffb700 10 client.16440 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:28:29.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.995+0000 7fc165d9b700 1 --2- 172.21.15.82:0/494714171 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c9e6a7b9c0 0x55c9e6a7dde0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f52f113ff5728a22 server_cookie=402edc4b0847571d in_seq=0 out_seq=0 2023-09-09T23:28:29.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.996+0000 7fc14dffb700 1 -- 172.21.15.82:0/494714171 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c9e69c88a0 con 0x55c9e6a7b9c0 2023-09-09T23:28:29.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.996+0000 7fc14dffb700 10 client.16440 handle_command_reply: tid=0 2023-09-09T23:28:29.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.996+0000 7fc178fbcb80 1 client.16440 shutdown 2023-09-09T23:28:29.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.996+0000 7fc178fbcb80 1 -- 172.21.15.82:0/494714171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9e6a61710 msgr2=0x55c9e6a6b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:28:29.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.996+0000 7fc178fbcb80 1 --2- 172.21.15.82:0/494714171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9e6a61710 0x55c9e6a6b830 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fc15c005910 tx=0x7fc15c0047c0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.996+0000 7fc178fbcb80 1 -- 172.21.15.82:0/494714171 shutdown_connections 2023-09-09T23:28:29.016 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.996+0000 7fc178fbcb80 1 --2- 172.21.15.82:0/494714171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9e6a6be40 0x55c9e6a69050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.996+0000 7fc178fbcb80 1 --2- 172.21.15.82:0/494714171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9e6a61710 0x55c9e6a6b830 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.996+0000 7fc178fbcb80 1 --2- 172.21.15.82:0/494714171 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c9e6a7b9c0 0x55c9e6a7dde0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.996+0000 7fc178fbcb80 1 --2- 172.21.15.82:0/494714171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9e6a62030 0x55c9e6a68a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.996+0000 7fc14dffb700 0 client.16440 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:28:29.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.996+0000 7fc178fbcb80 1 -- 172.21.15.82:0/494714171 >> 172.21.15.82:0/494714171 conn(0x55c9e69d6090 msgr2=0x55c9e6a5c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:28:29.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.996+0000 7fc178fbcb80 1 -- 172.21.15.82:0/494714171 shutdown_connections 2023-09-09T23:28:29.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.996+0000 7fc178fbcb80 1 -- 172.21.15.82:0/494714171 wait complete. 2023-09-09T23:28:29.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:28.996+0000 7fc178fbcb80 20 client.16440 trim_cache size 0 max 16384 2023-09-09T23:28:29.019 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 540.025595 secs 2023-09-09T23:28:29.019 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:28 smithi098 ceph-mon[103124]: pgmap v458: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 11 op/s 2023-09-09T23:28:29.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.013+0000 7fc178fbcb80 1 Processor -- start 2023-09-09T23:28:29.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.013+0000 7fc178fbcb80 1 -- start start 2023-09-09T23:28:29.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.013+0000 7fc178fbcb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9e6a61710 0x55c9e6a689f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:29.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.013+0000 7fc178fbcb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9e6a62030 0x55c9e6a6b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:29.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.013+0000 7fc178fbcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9e6a6be40 0x55c9e6a65cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:29.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.013+0000 7fc178fbcb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e6a59440 con 0x55c9e6a62030 2023-09-09T23:28:29.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.013+0000 7fc178fbcb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e6a62a10 con 0x55c9e6a61710 2023-09-09T23:28:29.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.013+0000 7fc178fbcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e6a62440 con 0x55c9e6a6be40 2023-09-09T23:28:29.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.013+0000 7fc165d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9e6a62030 0x55c9e6a6b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:29.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.013+0000 7fc165d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9e6a62030 0x55c9e6a6b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58672/0 (socket says 172.21.15.82:58672) 2023-09-09T23:28:29.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.013+0000 7fc165d9b700 1 -- 172.21.15.82:0/3902618696 learned_addr learned my addr 172.21.15.82:0/3902618696 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:28:29.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.013+0000 7fc16659c700 1 --2- 172.21.15.82:0/3902618696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9e6a61710 0x55c9e6a689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:29.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.013+0000 7fc166d9d700 1 --2- 172.21.15.82:0/3902618696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9e6a6be40 0x55c9e6a65cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:29.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.013+0000 7fc165d9b700 1 -- 172.21.15.82:0/3902618696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9e6a6be40 msgr2=0x55c9e6a65cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:28:29.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.013+0000 7fc165d9b700 1 --2- 172.21.15.82:0/3902618696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9e6a6be40 0x55c9e6a65cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.013+0000 7fc165d9b700 1 -- 172.21.15.82:0/3902618696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9e6a61710 msgr2=0x55c9e6a689f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:28:29.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.013+0000 7fc165d9b700 1 --2- 172.21.15.82:0/3902618696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9e6a61710 0x55c9e6a689f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.013+0000 7fc165d9b700 1 -- 172.21.15.82:0/3902618696 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9e6806250 con 0x55c9e6a62030 2023-09-09T23:28:29.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.013+0000 7fc16659c700 1 --2- 172.21.15.82:0/3902618696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9e6a61710 0x55c9e6a689f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:28:29.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.013+0000 7fc165d9b700 1 --2- 172.21.15.82:0/3902618696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9e6a62030 0x55c9e6a6b360 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x55c9e6a57280 tx=0x7fc1580098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8cb1a5519328a6f server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:28:29.028 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.014+0000 7fc1557fa700 1 -- 172.21.15.82:0/3902618696 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15802a1b0 con 0x55c9e6a62030 2023-09-09T23:28:29.029 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.014+0000 7fc1557fa700 1 -- 172.21.15.82:0/3902618696 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc15802a040 con 0x55c9e6a62030 2023-09-09T23:28:29.029 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.014+0000 7fc1557fa700 1 -- 172.21.15.82:0/3902618696 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1580052a0 con 0x55c9e6a62030 2023-09-09T23:28:29.029 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.014+0000 7fc178fbcb80 1 -- 172.21.15.82:0/3902618696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9e6a62030 msgr2=0x55c9e6a6b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:28:29.029 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.014+0000 7fc178fbcb80 1 --2- 172.21.15.82:0/3902618696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9e6a62030 0x55c9e6a6b360 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x55c9e6a57280 tx=0x7fc1580098e0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.029 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.014+0000 7fc178fbcb80 1 -- 172.21.15.82:0/3902618696 shutdown_connections 2023-09-09T23:28:29.030 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.014+0000 7fc178fbcb80 1 --2- 172.21.15.82:0/3902618696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9e6a61710 0x55c9e6a689f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.030 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.014+0000 7fc178fbcb80 1 --2- 172.21.15.82:0/3902618696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9e6a6be40 0x55c9e6a65cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.030 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.014+0000 7fc178fbcb80 1 --2- 172.21.15.82:0/3902618696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9e6a62030 0x55c9e6a6b360 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.030 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.014+0000 7fc178fbcb80 1 -- 172.21.15.82:0/3902618696 >> 172.21.15.82:0/3902618696 conn(0x55c9e69d6090 msgr2=0x55c9e6a682e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:28:29.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.014+0000 7fc178fbcb80 1 -- 172.21.15.82:0/3902618696 shutdown_connections 2023-09-09T23:28:29.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.014+0000 7fc178fbcb80 1 -- 172.21.15.82:0/3902618696 wait complete. 2023-09-09T23:28:29.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.014+0000 7fc178fbcb80 1 -- start start 2023-09-09T23:28:29.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.015+0000 7fc178fbcb80 1 Processor -- start 2023-09-09T23:28:29.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.015+0000 7fc178fbcb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9e6a61710 0x55c9e6a710c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:29.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.015+0000 7fc178fbcb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9e6a62030 0x55c9e6a71600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:29.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.015+0000 7fc178fbcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9e6a6be40 0x55c9e6a71b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:29.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.015+0000 7fc178fbcb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e6acf800 con 0x55c9e6a62030 2023-09-09T23:28:29.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.015+0000 7fc178fbcb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e6acf970 con 0x55c9e6a61710 2023-09-09T23:28:29.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.015+0000 7fc178fbcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9e6acfb30 con 0x55c9e6a6be40 2023-09-09T23:28:29.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.015+0000 7fc178fbcb80 10 client.0 _ll_register_callbacks cb 0x7fc15c0044e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:28:29.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.015+0000 7fc165d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9e6a62030 0x55c9e6a71600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:29.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.015+0000 7fc165d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9e6a62030 0x55c9e6a71600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58678/0 (socket says 172.21.15.82:58678) 2023-09-09T23:28:29.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.015+0000 7fc16659c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9e6a61710 0x55c9e6a710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:29.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.015+0000 7fc165d9b700 1 -- 172.21.15.82:0/2422740725 learned_addr learned my addr 172.21.15.82:0/2422740725 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:28:29.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.015+0000 7fc166d9d700 1 --2- 172.21.15.82:0/2422740725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9e6a6be40 0x55c9e6a71b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:29.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.015+0000 7fc165d9b700 1 -- 172.21.15.82:0/2422740725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9e6a6be40 msgr2=0x55c9e6a71b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:28:29.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.015+0000 7fc165d9b700 1 --2- 172.21.15.82:0/2422740725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9e6a6be40 0x55c9e6a71b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.015+0000 7fc165d9b700 1 -- 172.21.15.82:0/2422740725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9e6a61710 msgr2=0x55c9e6a710c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:28:29.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.015+0000 7fc165d9b700 1 --2- 172.21.15.82:0/2422740725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9e6a61710 0x55c9e6a710c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.015+0000 7fc165d9b700 1 -- 172.21.15.82:0/2422740725 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9e67c8c60 con 0x55c9e6a62030 2023-09-09T23:28:29.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.015+0000 7fc165d9b700 1 --2- 172.21.15.82:0/2422740725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9e6a62030 0x55c9e6a71600 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc158000c00 tx=0x7fc1580092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b1441e991d1d8c server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:28:29.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.015+0000 7fc14dffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:28:29.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.016+0000 7fc14dffb700 1 -- 172.21.15.82:0/2422740725 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc158019730 con 0x55c9e6a62030 2023-09-09T23:28:29.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.016+0000 7fc14dffb700 1 -- 172.21.15.82:0/2422740725 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8455+0+0 (secure 0 0 0) 0x7fc158019890 con 0x55c9e6a62030 2023-09-09T23:28:29.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.016+0000 7fc178fbcb80 1 -- 172.21.15.82:0/2422740725 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc158019c50 con 0x55c9e6a62030 2023-09-09T23:28:29.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.016+0000 7fc14dffb700 1 -- 172.21.15.82:0/2422740725 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc158019a50 con 0x55c9e6a62030 2023-09-09T23:28:29.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.016+0000 7fc178fbcb80 10 client.16446 fetch_fsmap learned FSMap version 127 2023-09-09T23:28:29.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.016+0000 7fc178fbcb80 1 -- 172.21.15.82:0/2422740725 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x55c9e67dfde0 con 0x55c9e6a62030 2023-09-09T23:28:29.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.016+0000 7fc14dffb700 1 -- 172.21.15.82:0/2422740725 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc158019c50 con 0x55c9e6a62030 2023-09-09T23:28:29.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.016+0000 7fc14dffb700 1 -- 172.21.15.82:0/2422740725 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 127) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc158030020 con 0x55c9e6a62030 2023-09-09T23:28:29.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.016+0000 7fc178fbcb80 10 client.16446 fetch_fsmap finished waiting for FSMap version 127 2023-09-09T23:28:29.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.016+0000 7fc178fbcb80 10 client.16446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:28:29.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.016+0000 7fc178fbcb80 20 client.16446 populate_metadata read hostname 'smithi082' 2023-09-09T23:28:29.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.016+0000 7fc178fbcb80 1 --2- 172.21.15.82:0/2422740725 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c9e6ad0e30 0x55c9e6ad3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:29.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.016+0000 7fc178fbcb80 4 client.16446 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:28:29.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.016+0000 7fc178fbcb80 1 -- 172.21.15.82:0/2422740725 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c9e69ac920 con 0x55c9e6ad0e30 2023-09-09T23:28:29.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.016+0000 7fc16659c700 1 --2- 172.21.15.82:0/2422740725 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c9e6ad0e30 0x55c9e6ad3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:29.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.017+0000 7fc14dffb700 10 client.16446 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:28:29.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.017+0000 7fc16659c700 1 --2- 172.21.15.82:0/2422740725 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c9e6ad0e30 0x55c9e6ad3250 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d0db6feea9dcf5a server_cookie=2f46b8285aaea900 in_seq=0 out_seq=0 2023-09-09T23:28:29.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.017+0000 7fc14dffb700 1 -- 172.21.15.82:0/2422740725 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55c9e69ac920 con 0x55c9e6ad0e30 2023-09-09T23:28:29.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.017+0000 7fc14dffb700 10 client.16446 handle_command_reply: tid=0 2023-09-09T23:28:29.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.017+0000 7fc178fbcb80 1 client.16446 shutdown 2023-09-09T23:28:29.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.017+0000 7fc178fbcb80 1 -- 172.21.15.82:0/2422740725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9e6a62030 msgr2=0x55c9e6a71600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:28:29.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.017+0000 7fc178fbcb80 1 --2- 172.21.15.82:0/2422740725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9e6a62030 0x55c9e6a71600 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc158000c00 tx=0x7fc1580092f0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.017+0000 7fc178fbcb80 1 -- 172.21.15.82:0/2422740725 shutdown_connections 2023-09-09T23:28:29.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.017+0000 7fc178fbcb80 1 --2- 172.21.15.82:0/2422740725 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x55c9e6ad0e30 0x55c9e6ad3250 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.017+0000 7fc178fbcb80 1 --2- 172.21.15.82:0/2422740725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9e6a61710 0x55c9e6a710c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.017+0000 7fc14dffb700 0 client.16446 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:28:29.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.017+0000 7fc178fbcb80 1 --2- 172.21.15.82:0/2422740725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9e6a6be40 0x55c9e6a71b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.017+0000 7fc178fbcb80 1 --2- 172.21.15.82:0/2422740725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9e6a62030 0x55c9e6a71600 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.041 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.017+0000 7fc178fbcb80 1 -- 172.21.15.82:0/2422740725 >> 172.21.15.82:0/2422740725 conn(0x55c9e69d6090 msgr2=0x55c9e6a671f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:28:29.041 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.017+0000 7fc178fbcb80 1 -- 172.21.15.82:0/2422740725 shutdown_connections 2023-09-09T23:28:29.041 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.017+0000 7fc178fbcb80 1 -- 172.21.15.82:0/2422740725 wait complete. 2023-09-09T23:28:29.041 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.017+0000 7fc178fbcb80 20 client.16446 trim_cache size 0 max 16384 2023-09-09T23:28:29.042 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:28:29.042 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:28:29.042 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:28:29.042 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:28:29.042 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:28:29.043 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:28:29.043 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:28:29.043 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:28:29.043 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:28:29.043 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:28:29.045 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.018+0000 7fc16cdf6700 1 -- 172.21.15.82:0/1660340885 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fc1500a07b0 msgr2=0x7fc1500a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:28:29.045 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.018+0000 7fc16cdf6700 1 --2- 172.21.15.82:0/1660340885 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fc1500a07b0 0x7fc1500a2c70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc15c009c60 tx=0x7fc15c009260 comp rx=0 tx=0).stop 2023-09-09T23:28:29.045 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.018+0000 7fc16cdf6700 1 -- 172.21.15.82:0/1660340885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc168105570 msgr2=0x7fc168196580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:28:29.045 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.018+0000 7fc16cdf6700 1 --2- 172.21.15.82:0/1660340885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc168105570 0x7fc168196580 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fc160007870 tx=0x7fc1600078a0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.045 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.018+0000 7fc16cdf6700 1 -- 172.21.15.82:0/1660340885 shutdown_connections 2023-09-09T23:28:29.046 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.018+0000 7fc16cdf6700 1 --2- 172.21.15.82:0/1660340885 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7fc1500a07b0 0x7fc1500a2c70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.046 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.018+0000 7fc16cdf6700 1 --2- 172.21.15.82:0/1660340885 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1680fea00 0x7fc16819c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.046 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.018+0000 7fc16cdf6700 1 --2- 172.21.15.82:0/1660340885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc168105570 0x7fc168196580 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.046 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.018+0000 7fc16cdf6700 1 --2- 172.21.15.82:0/1660340885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1680ff3c0 0x7fc16819ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:29.046 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.018+0000 7fc16cdf6700 1 -- 172.21.15.82:0/1660340885 >> 172.21.15.82:0/1660340885 conn(0x7fc168076a70 msgr2=0x7fc168103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:28:29.047 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.018+0000 7fc16cdf6700 1 -- 172.21.15.82:0/1660340885 shutdown_connections 2023-09-09T23:28:29.047 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:29.018+0000 7fc16cdf6700 1 -- 172.21.15.82:0/1660340885 wait complete. 2023-09-09T23:28:29.047 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:28:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 540.754654 secs 2023-09-09T23:28:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 540.754654 secs 2023-09-09T23:28:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 540.754654 secs 2023-09-09T23:28:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:30 smithi170 ceph-mon[102832]: pgmap v459: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 14 op/s 2023-09-09T23:28:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:30 smithi082 ceph-mon[98694]: pgmap v459: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 14 op/s 2023-09-09T23:28:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:30 smithi098 ceph-mon[103124]: pgmap v459: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 14 op/s 2023-09-09T23:28:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:31 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 543.425687 secs 2023-09-09T23:28:32.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:31 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 543.425687 secs 2023-09-09T23:28:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:31 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 543.425687 secs 2023-09-09T23:28:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:32 smithi170 ceph-mon[102832]: pgmap v460: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2023-09-09T23:28:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:32 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 545.025668 secs 2023-09-09T23:28:33.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:32 smithi082 ceph-mon[98694]: pgmap v460: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2023-09-09T23:28:33.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:32 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 545.025668 secs 2023-09-09T23:28:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:32 smithi098 ceph-mon[103124]: pgmap v460: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2023-09-09T23:28:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:32 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 545.025668 secs 2023-09-09T23:28:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 545.754741 secs 2023-09-09T23:28:34.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 545.754741 secs 2023-09-09T23:28:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 545.754741 secs 2023-09-09T23:28:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:35 smithi170 ceph-mon[102832]: pgmap v461: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2023-09-09T23:28:35.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:35 smithi082 ceph-mon[98694]: pgmap v461: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2023-09-09T23:28:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:35 smithi098 ceph-mon[103124]: pgmap v461: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2023-09-09T23:28:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:37 smithi170 ceph-mon[102832]: pgmap v462: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.0 KiB/s wr, 12 op/s 2023-09-09T23:28:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 548.425747 secs 2023-09-09T23:28:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:37 smithi082 ceph-mon[98694]: pgmap v462: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.0 KiB/s wr, 12 op/s 2023-09-09T23:28:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 548.425747 secs 2023-09-09T23:28:38.160 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:37 smithi098 ceph-mon[103124]: pgmap v462: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.0 KiB/s wr, 12 op/s 2023-09-09T23:28:38.160 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 548.425747 secs 2023-09-09T23:28:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 550.025727 secs 2023-09-09T23:28:38.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 550.025727 secs 2023-09-09T23:28:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 550.025727 secs 2023-09-09T23:28:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:39 smithi170 ceph-mon[102832]: pgmap v463: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2023-09-09T23:28:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 550.754828 secs 2023-09-09T23:28:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:39 smithi082 ceph-mon[98694]: pgmap v463: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2023-09-09T23:28:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 550.754828 secs 2023-09-09T23:28:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:39 smithi098 ceph-mon[103124]: pgmap v463: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2023-09-09T23:28:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 550.754828 secs 2023-09-09T23:28:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:41 smithi170 ceph-mon[102832]: pgmap v464: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2023-09-09T23:28:41.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:41 smithi082 ceph-mon[98694]: pgmap v464: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2023-09-09T23:28:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:41 smithi098 ceph-mon[103124]: pgmap v464: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2023-09-09T23:28:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 553.425794 secs 2023-09-09T23:28:42.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 553.425794 secs 2023-09-09T23:28:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 553.425794 secs 2023-09-09T23:28:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:43 smithi170 ceph-mon[102832]: pgmap v465: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:28:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 555.025789 secs 2023-09-09T23:28:43.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:43 smithi082 ceph-mon[98694]: pgmap v465: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:28:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 555.025789 secs 2023-09-09T23:28:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:43 smithi098 ceph-mon[103124]: pgmap v465: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:28:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 555.025789 secs 2023-09-09T23:28:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 555.754894 secs 2023-09-09T23:28:44.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 555.754894 secs 2023-09-09T23:28:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 555.754894 secs 2023-09-09T23:28:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:45 smithi170 ceph-mon[102832]: pgmap v466: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:28:45.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:45 smithi082 ceph-mon[98694]: pgmap v466: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:28:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:45 smithi098 ceph-mon[103124]: pgmap v466: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:28:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:47 smithi170 ceph-mon[102832]: pgmap v467: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:28:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 558.425853 secs 2023-09-09T23:28:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:47 smithi082 ceph-mon[98694]: pgmap v467: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:28:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 558.425853 secs 2023-09-09T23:28:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:47 smithi098 ceph-mon[103124]: pgmap v467: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:28:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 558.425853 secs 2023-09-09T23:28:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 560.025851 secs 2023-09-09T23:28:48.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 560.025851 secs 2023-09-09T23:28:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 560.025851 secs 2023-09-09T23:28:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:49 smithi170 ceph-mon[102832]: pgmap v468: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:28:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 560.754961 secs 2023-09-09T23:28:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:49 smithi082 ceph-mon[98694]: pgmap v468: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:28:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 560.754961 secs 2023-09-09T23:28:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:49 smithi098 ceph-mon[103124]: pgmap v468: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:28:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 560.754961 secs 2023-09-09T23:28:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:51 smithi170 ceph-mon[102832]: pgmap v469: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-09-09T23:28:51.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:51 smithi082 ceph-mon[98694]: pgmap v469: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-09-09T23:28:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:51 smithi098 ceph-mon[103124]: pgmap v469: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-09-09T23:28:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 563.425943 secs 2023-09-09T23:28:52.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 563.425943 secs 2023-09-09T23:28:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 563.425943 secs 2023-09-09T23:28:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:53 smithi170 ceph-mon[102832]: pgmap v470: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:28:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 565.025910 secs 2023-09-09T23:28:53.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:53 smithi082 ceph-mon[98694]: pgmap v470: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:28:53.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 565.025910 secs 2023-09-09T23:28:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:53 smithi098 ceph-mon[103124]: pgmap v470: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:28:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 565.025910 secs 2023-09-09T23:28:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 565.755025 secs 2023-09-09T23:28:54.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 565.755025 secs 2023-09-09T23:28:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 565.755025 secs 2023-09-09T23:28:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:55 smithi170 ceph-mon[102832]: pgmap v471: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:28:55.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:55 smithi082 ceph-mon[98694]: pgmap v471: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:28:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:55 smithi098 ceph-mon[103124]: pgmap v471: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:28:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:57 smithi170 ceph-mon[102832]: pgmap v472: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-09T23:28:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 568.426003 secs 2023-09-09T23:28:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:57 smithi082 ceph-mon[98694]: pgmap v472: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-09T23:28:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 568.426003 secs 2023-09-09T23:28:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:57 smithi098 ceph-mon[103124]: pgmap v472: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-09T23:28:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 568.426003 secs 2023-09-09T23:28:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 570.025992 secs 2023-09-09T23:28:58.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 570.025992 secs 2023-09-09T23:28:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 570.025992 secs 2023-09-09T23:28:59.048 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-09T23:28:59.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.187+0000 7f973387f700 1 -- 172.21.15.82:0/3148971205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f972c0ff4a0 msgr2=0x7f972c0ff920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:28:59.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.187+0000 7f973387f700 1 --2- 172.21.15.82:0/3148971205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f972c0ff4a0 0x7f972c0ff920 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f97280099c0 tx=0x7f9728009cd0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.187+0000 7f973387f700 1 -- 172.21.15.82:0/3148971205 shutdown_connections 2023-09-09T23:28:59.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.187+0000 7f973387f700 1 --2- 172.21.15.82:0/3148971205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f972c105560 0x7f972c105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.187+0000 7f973387f700 1 --2- 172.21.15.82:0/3148971205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f972c0ffe60 0x7f972c10c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.187+0000 7f973387f700 1 --2- 172.21.15.82:0/3148971205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f972c0ff4a0 0x7f972c0ff920 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.187+0000 7f973387f700 1 -- 172.21.15.82:0/3148971205 >> 172.21.15.82:0/3148971205 conn(0x7f972c0faf50 msgr2=0x7f972c0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:28:59.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.187+0000 7f973387f700 1 -- 172.21.15.82:0/3148971205 shutdown_connections 2023-09-09T23:28:59.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.187+0000 7f973387f700 1 -- 172.21.15.82:0/3148971205 wait complete. 2023-09-09T23:28:59.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.188+0000 7f973387f700 1 Processor -- start 2023-09-09T23:28:59.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.188+0000 7f973387f700 1 -- start start 2023-09-09T23:28:59.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.188+0000 7f973387f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f972c0ff4a0 0x7f972c19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:59.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.188+0000 7f973387f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f972c0ffe60 0x7f972c19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:59.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.188+0000 7f973387f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f972c105560 0x7f972c198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:59.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.188+0000 7f973387f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c198cf0 con 0x7f972c0ffe60 2023-09-09T23:28:59.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.188+0000 7f973387f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c198e60 con 0x7f972c105560 2023-09-09T23:28:59.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.188+0000 7f973387f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c199020 con 0x7f972c0ff4a0 2023-09-09T23:28:59.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.188+0000 7f9730e1a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f972c0ffe60 0x7f972c19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:59.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.188+0000 7f9730e1a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f972c0ffe60 0x7f972c19eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58712/0 (socket says 172.21.15.82:58712) 2023-09-09T23:28:59.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.188+0000 7f9730e1a700 1 -- 172.21.15.82:0/1390860251 learned_addr learned my addr 172.21.15.82:0/1390860251 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:28:59.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.188+0000 7f973161b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f972c0ff4a0 0x7f972c19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:59.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.188+0000 7f9731e1c700 1 --2- 172.21.15.82:0/1390860251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f972c105560 0x7f972c198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:59.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.188+0000 7f9730e1a700 1 -- 172.21.15.82:0/1390860251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f972c0ff4a0 msgr2=0x7f972c19e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:28:59.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.188+0000 7f9730e1a700 1 --2- 172.21.15.82:0/1390860251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f972c0ff4a0 0x7f972c19e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.188+0000 7f9730e1a700 1 -- 172.21.15.82:0/1390860251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f972c105560 msgr2=0x7f972c198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:28:59.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.188+0000 7f9730e1a700 1 --2- 172.21.15.82:0/1390860251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f972c105560 0x7f972c198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.188+0000 7f9730e1a700 1 -- 172.21.15.82:0/1390860251 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97280095f0 con 0x7f972c0ffe60 2023-09-09T23:28:59.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.188+0000 7f9730e1a700 1 --2- 172.21.15.82:0/1390860251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f972c0ffe60 0x7f972c19eb50 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f972800b580 tx=0x7f972800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:28:59.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.188+0000 7f97227fc700 1 -- 172.21.15.82:0/1390860251 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972801d070 con 0x7f972c0ffe60 2023-09-09T23:28:59.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.189+0000 7f973387f700 1 -- 172.21.15.82:0/1390860251 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f972c1992a0 con 0x7f972c0ffe60 2023-09-09T23:28:59.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.189+0000 7f97227fc700 1 -- 172.21.15.82:0/1390860251 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9728022470 con 0x7f972c0ffe60 2023-09-09T23:28:59.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.189+0000 7f97227fc700 1 -- 172.21.15.82:0/1390860251 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9728017650 con 0x7f972c0ffe60 2023-09-09T23:28:59.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.189+0000 7f973387f700 1 -- 172.21.15.82:0/1390860251 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f972c1a5600 con 0x7f972c0ffe60 2023-09-09T23:28:59.200 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.191+0000 7f97227fc700 1 -- 172.21.15.82:0/1390860251 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) v1 ==== 135390+0+0 (secure 0 0 0) 0x7f97280225e0 con 0x7f972c0ffe60 2023-09-09T23:28:59.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.192+0000 7f97227fc700 1 --2- 172.21.15.82:0/1390860251 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f97180a07b0 0x7f97180a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:59.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.192+0000 7f973161b700 1 --2- 172.21.15.82:0/1390860251 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f97180a07b0 0x7f97180a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:59.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.192+0000 7f97227fc700 1 -- 172.21.15.82:0/1390860251 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f97280cd0e0 con 0x7f972c0ffe60 2023-09-09T23:28:59.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.192+0000 7f973161b700 1 --2- 172.21.15.82:0/1390860251 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f97180a07b0 0x7f97180a2c70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f971c009be0 tx=0x7f971c009260 comp rx=0 tx=0).ready entity=mgr.15597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:28:59.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.193+0000 7f973fa45b80 1 Processor -- start 2023-09-09T23:28:59.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.193+0000 7f973fa45b80 1 -- start start 2023-09-09T23:28:59.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.193+0000 7f973fa45b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557763666710 0x557763666af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:59.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.193+0000 7f973fa45b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557763667030 0x557763670900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:59.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.193+0000 7f973fa45b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557763670e40 0x557763673230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:59.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.193+0000 7f973fa45b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577635bf730 con 0x557763666710 2023-09-09T23:28:59.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.193+0000 7f973fa45b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577635bd050 con 0x557763667030 2023-09-09T23:28:59.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.193+0000 7f973fa45b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577635b8940 con 0x557763670e40 2023-09-09T23:28:59.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.193+0000 7f973161b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557763666710 0x557763666af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:59.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.193+0000 7f973161b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557763666710 0x557763666af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58718/0 (socket says 172.21.15.82:58718) 2023-09-09T23:28:59.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.193+0000 7f973161b700 1 -- 172.21.15.82:0/431738280 learned_addr learned my addr 172.21.15.82:0/431738280 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:28:59.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.193+0000 7f9730e1a700 1 --2- 172.21.15.82:0/431738280 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557763667030 0x557763670900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:59.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.193+0000 7f9731e1c700 1 --2- 172.21.15.82:0/431738280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557763670e40 0x557763673230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:59.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.193+0000 7f973161b700 1 -- 172.21.15.82:0/431738280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557763670e40 msgr2=0x557763673230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:28:59.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.193+0000 7f973161b700 1 --2- 172.21.15.82:0/431738280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557763670e40 0x557763673230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.193+0000 7f973161b700 1 -- 172.21.15.82:0/431738280 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557763667030 msgr2=0x557763670900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:28:59.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.193+0000 7f973161b700 1 --2- 172.21.15.82:0/431738280 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557763667030 0x557763670900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.193+0000 7f973161b700 1 -- 172.21.15.82:0/431738280 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577633e4de0 con 0x557763666710 2023-09-09T23:28:59.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.193+0000 7f973161b700 1 --2- 172.21.15.82:0/431738280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557763666710 0x557763666af0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f971c011d70 tx=0x7f971c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813453939fe6f336 server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:28:59.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.193+0000 7f97177fe700 1 -- 172.21.15.82:0/431738280 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c011da0 con 0x557763666710 2023-09-09T23:28:59.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.194+0000 7f97177fe700 1 -- 172.21.15.82:0/431738280 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f971c004020 con 0x557763666710 2023-09-09T23:28:59.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.194+0000 7f97177fe700 1 -- 172.21.15.82:0/431738280 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c003cb0 con 0x557763666710 2023-09-09T23:28:59.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.194+0000 7f973fa45b80 1 -- 172.21.15.82:0/431738280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557763666710 msgr2=0x557763666af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:28:59.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.194+0000 7f973fa45b80 1 --2- 172.21.15.82:0/431738280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557763666710 0x557763666af0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f971c011d70 tx=0x7f971c014000 comp rx=0 tx=0).stop 2023-09-09T23:28:59.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.194+0000 7f973fa45b80 1 -- 172.21.15.82:0/431738280 shutdown_connections 2023-09-09T23:28:59.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.194+0000 7f973fa45b80 1 --2- 172.21.15.82:0/431738280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557763670e40 0x557763673230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.194+0000 7f973fa45b80 1 --2- 172.21.15.82:0/431738280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557763666710 0x557763666af0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.194+0000 7f973fa45b80 1 --2- 172.21.15.82:0/431738280 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557763667030 0x557763670900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.194+0000 7f973fa45b80 1 -- 172.21.15.82:0/431738280 >> 172.21.15.82:0/431738280 conn(0x5577635db090 msgr2=0x557763661bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:28:59.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.194+0000 7f973fa45b80 1 -- 172.21.15.82:0/431738280 shutdown_connections 2023-09-09T23:28:59.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.194+0000 7f973fa45b80 1 -- 172.21.15.82:0/431738280 wait complete. 2023-09-09T23:28:59.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.195+0000 7f973fa45b80 1 -- start start 2023-09-09T23:28:59.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.195+0000 7f973fa45b80 1 Processor -- start 2023-09-09T23:28:59.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.195+0000 7f973fa45b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557763666710 0x557763670750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:59.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.195+0000 7f973fa45b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557763667030 0x55776366da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:59.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.195+0000 7f973fa45b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557763670e40 0x55776366dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:59.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.195+0000 7f973fa45b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557763556b70 con 0x557763667030 2023-09-09T23:28:59.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.195+0000 7f973fa45b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x557763556870 con 0x557763666710 2023-09-09T23:28:59.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.195+0000 7f973fa45b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55776343a920 con 0x557763670e40 2023-09-09T23:28:59.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.195+0000 7f973fa45b80 10 client.0 _ll_register_callbacks cb 0x5577634a0a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:28:59.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.195+0000 7f973161b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557763666710 0x557763670750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:59.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.195+0000 7f9730e1a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557763667030 0x55776366da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:59.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.196+0000 7f973161b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557763666710 0x557763670750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:44288/0 (socket says 172.21.15.82:44288) 2023-09-09T23:28:59.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.196+0000 7f9730e1a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557763667030 0x55776366da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58726/0 (socket says 172.21.15.82:58726) 2023-09-09T23:28:59.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.196+0000 7f973161b700 1 -- 172.21.15.82:0/1125449297 learned_addr learned my addr 172.21.15.82:0/1125449297 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:28:59.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.196+0000 7f9731e1c700 1 --2- 172.21.15.82:0/1125449297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557763670e40 0x55776366dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:59.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.196+0000 7f9730e1a700 1 -- 172.21.15.82:0/1125449297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557763670e40 msgr2=0x55776366dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:28:59.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.196+0000 7f9730e1a700 1 --2- 172.21.15.82:0/1125449297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557763670e40 0x55776366dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.196+0000 7f9730e1a700 1 -- 172.21.15.82:0/1125449297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557763666710 msgr2=0x557763670750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:28:59.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.196+0000 7f9730e1a700 1 --2- 172.21.15.82:0/1125449297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557763666710 0x557763670750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.196+0000 7f9730e1a700 1 -- 172.21.15.82:0/1125449297 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f971c017070 con 0x557763667030 2023-09-09T23:28:59.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.196+0000 7f973161b700 1 --2- 172.21.15.82:0/1125449297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557763666710 0x557763670750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:28:59.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.196+0000 7f9731e1c700 1 --2- 172.21.15.82:0/1125449297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557763670e40 0x55776366dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:28:59.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.196+0000 7f9730e1a700 1 --2- 172.21.15.82:0/1125449297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557763667030 0x55776366da40 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f97280ca870 tx=0x7f9728005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8010843978a2ae server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:28:59.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.196+0000 7f9714ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-09-09T23:28:59.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.196+0000 7f9714ff9700 1 -- 172.21.15.82:0/1125449297 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972809abe0 con 0x557763667030 2023-09-09T23:28:59.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.196+0000 7f9714ff9700 1 -- 172.21.15.82:0/1125449297 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f972809e380 con 0x557763667030 2023-09-09T23:28:59.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.196+0000 7f973fa45b80 1 -- 172.21.15.82:0/1125449297 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577635104f0 con 0x557763667030 2023-09-09T23:28:59.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.196+0000 7f9714ff9700 1 -- 172.21.15.82:0/1125449297 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7f972809a780 con 0x557763667030 2023-09-09T23:28:59.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.196+0000 7f973fa45b80 10 client.16458 fetch_fsmap learned FSMap version 127 2023-09-09T23:28:59.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.196+0000 7f973fa45b80 1 -- 172.21.15.82:0/1125449297 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x5577633cdc60 con 0x557763667030 2023-09-09T23:28:59.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.196+0000 7f9714ff9700 1 -- 172.21.15.82:0/1125449297 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5577635104f0 con 0x557763667030 2023-09-09T23:28:59.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.197+0000 7f9714ff9700 1 -- 172.21.15.82:0/1125449297 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 127) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f97280a22d0 con 0x557763667030 2023-09-09T23:28:59.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.197+0000 7f973fa45b80 10 client.16458 fetch_fsmap finished waiting for FSMap version 127 2023-09-09T23:28:59.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.197+0000 7f973fa45b80 10 client.16458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:28:59.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.197+0000 7f973fa45b80 20 client.16458 populate_metadata read hostname 'smithi082' 2023-09-09T23:28:59.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.197+0000 7f973fa45b80 1 --2- 172.21.15.82:0/1125449297 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x557763680450 0x557763682840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:59.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.197+0000 7f973fa45b80 4 client.16458 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-09T23:28:59.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.197+0000 7f973fa45b80 1 -- 172.21.15.82:0/1125449297 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577635cd8a0 con 0x557763680450 2023-09-09T23:28:59.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.197+0000 7f973161b700 1 --2- 172.21.15.82:0/1125449297 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x557763680450 0x557763682840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:59.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.197+0000 7f9714ff9700 10 client.16458 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:28:59.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.197+0000 7f973161b700 1 --2- 172.21.15.82:0/1125449297 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x557763680450 0x557763682840 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95e44187a185acf6 server_cookie=70115e3bc83fac8b in_seq=0 out_seq=0 2023-09-09T23:28:59.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.198+0000 7f9714ff9700 1 -- 172.21.15.82:0/1125449297 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5577635cd8a0 con 0x557763680450 2023-09-09T23:28:59.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.198+0000 7f9714ff9700 10 client.16458 handle_command_reply: tid=0 2023-09-09T23:28:59.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.198+0000 7f973fa45b80 1 client.16458 shutdown 2023-09-09T23:28:59.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.198+0000 7f973fa45b80 1 -- 172.21.15.82:0/1125449297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557763667030 msgr2=0x55776366da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:28:59.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.198+0000 7f973fa45b80 1 --2- 172.21.15.82:0/1125449297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557763667030 0x55776366da40 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f97280ca870 tx=0x7f9728005e80 comp rx=0 tx=0).stop 2023-09-09T23:28:59.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.198+0000 7f973fa45b80 1 -- 172.21.15.82:0/1125449297 shutdown_connections 2023-09-09T23:28:59.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.198+0000 7f973fa45b80 1 --2- 172.21.15.82:0/1125449297 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x557763680450 0x557763682840 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.198+0000 7f973fa45b80 1 --2- 172.21.15.82:0/1125449297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557763666710 0x557763670750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.198+0000 7f973fa45b80 1 --2- 172.21.15.82:0/1125449297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557763670e40 0x55776366dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.198+0000 7f973fa45b80 1 --2- 172.21.15.82:0/1125449297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557763667030 0x55776366da40 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.198+0000 7f973fa45b80 1 -- 172.21.15.82:0/1125449297 >> 172.21.15.82:0/1125449297 conn(0x5577635db090 msgr2=0x5577636617e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:28:59.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.198+0000 7f9714ff9700 0 client.16458 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:28:59.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.198+0000 7f973fa45b80 1 -- 172.21.15.82:0/1125449297 shutdown_connections 2023-09-09T23:28:59.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.198+0000 7f973fa45b80 1 -- 172.21.15.82:0/1125449297 wait complete. 2023-09-09T23:28:59.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.198+0000 7f973fa45b80 20 client.16458 trim_cache size 0 max 16384 2023-09-09T23:28:59.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.215+0000 7f973fa45b80 1 Processor -- start 2023-09-09T23:28:59.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.215+0000 7f973fa45b80 1 -- start start 2023-09-09T23:28:59.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.215+0000 7f973fa45b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557763666710 0x5577636779f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:59.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.215+0000 7f973fa45b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557763667030 0x557763668d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:59.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.215+0000 7f973fa45b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557763670e40 0x557763669260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:59.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.215+0000 7f973fa45b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577636676d0 con 0x557763667030 2023-09-09T23:28:59.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.215+0000 7f973fa45b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x557763668850 con 0x557763670e40 2023-09-09T23:28:59.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.215+0000 7f973fa45b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557763677f30 con 0x557763666710 2023-09-09T23:28:59.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.215+0000 7f9730e1a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557763667030 0x557763668d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:59.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.215+0000 7f9730e1a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557763667030 0x557763668d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58734/0 (socket says 172.21.15.82:58734) 2023-09-09T23:28:59.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.215+0000 7f9730e1a700 1 -- 172.21.15.82:0/2448618215 learned_addr learned my addr 172.21.15.82:0/2448618215 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:28:59.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.215+0000 7f9731e1c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557763670e40 0x557763669260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:59.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.216+0000 7f973161b700 1 --2- 172.21.15.82:0/2448618215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557763666710 0x5577636779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:59.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.216+0000 7f9730e1a700 1 -- 172.21.15.82:0/2448618215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557763666710 msgr2=0x5577636779f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:28:59.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.216+0000 7f9730e1a700 1 --2- 172.21.15.82:0/2448618215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557763666710 0x5577636779f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.227 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.216+0000 7f9730e1a700 1 -- 172.21.15.82:0/2448618215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557763670e40 msgr2=0x557763669260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:28:59.227 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.216+0000 7f9730e1a700 1 --2- 172.21.15.82:0/2448618215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557763670e40 0x557763669260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.227 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.216+0000 7f9730e1a700 1 -- 172.21.15.82:0/2448618215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55776340b250 con 0x557763667030 2023-09-09T23:28:59.227 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.216+0000 7f9730e1a700 1 --2- 172.21.15.82:0/2448618215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557763667030 0x557763668d20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9728000c00 tx=0x7f9728009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66586ea4d6305da server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:28:59.227 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.216+0000 7f9717fff700 1 -- 172.21.15.82:0/2448618215 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972809a0e0 con 0x557763667030 2023-09-09T23:28:59.228 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.216+0000 7f9717fff700 1 -- 172.21.15.82:0/2448618215 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f972809e380 con 0x557763667030 2023-09-09T23:28:59.228 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.216+0000 7f9717fff700 1 -- 172.21.15.82:0/2448618215 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972809a9e0 con 0x557763667030 2023-09-09T23:28:59.228 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.216+0000 7f973fa45b80 1 -- 172.21.15.82:0/2448618215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557763667030 msgr2=0x557763668d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:28:59.228 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.216+0000 7f973fa45b80 1 --2- 172.21.15.82:0/2448618215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557763667030 0x557763668d20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9728000c00 tx=0x7f9728009290 comp rx=0 tx=0).stop 2023-09-09T23:28:59.228 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.217+0000 7f973fa45b80 1 -- 172.21.15.82:0/2448618215 shutdown_connections 2023-09-09T23:28:59.228 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.217+0000 7f973fa45b80 1 --2- 172.21.15.82:0/2448618215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557763670e40 0x557763669260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.217+0000 7f973fa45b80 1 --2- 172.21.15.82:0/2448618215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557763666710 0x5577636779f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.217+0000 7f973fa45b80 1 --2- 172.21.15.82:0/2448618215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557763667030 0x557763668d20 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.217+0000 7f973fa45b80 1 -- 172.21.15.82:0/2448618215 >> 172.21.15.82:0/2448618215 conn(0x5577635db090 msgr2=0x55776367c140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:28:59.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.217+0000 7f973fa45b80 1 -- 172.21.15.82:0/2448618215 shutdown_connections 2023-09-09T23:28:59.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.217+0000 7f973fa45b80 1 -- 172.21.15.82:0/2448618215 wait complete. 2023-09-09T23:28:59.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.217+0000 7f973fa45b80 1 -- start start 2023-09-09T23:28:59.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.217+0000 7f973fa45b80 1 Processor -- start 2023-09-09T23:28:59.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.217+0000 7f973fa45b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557763666710 0x5577636771b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:59.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.218+0000 7f973fa45b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557763667030 0x5577636776f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:59.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.218+0000 7f973fa45b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557763670e40 0x557763675670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:59.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.218+0000 7f973fa45b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557763675c40 con 0x557763667030 2023-09-09T23:28:59.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.218+0000 7f973fa45b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x557763675db0 con 0x557763666710 2023-09-09T23:28:59.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.218+0000 7f973fa45b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557763675f70 con 0x557763670e40 2023-09-09T23:28:59.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.218+0000 7f973fa45b80 10 client.0 _ll_register_callbacks cb 0x7f972809a640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-09T23:28:59.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.218+0000 7f973161b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557763666710 0x5577636771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:59.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.218+0000 7f9730e1a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557763667030 0x5577636776f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:59.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.218+0000 7f973161b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557763666710 0x5577636771b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.82:44302/0 (socket says 172.21.15.82:44302) 2023-09-09T23:28:59.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.218+0000 7f973161b700 1 -- 172.21.15.82:0/2170861535 learned_addr learned my addr 172.21.15.82:0/2170861535 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-09-09T23:28:59.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.218+0000 7f9731e1c700 1 --2- 172.21.15.82:0/2170861535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557763670e40 0x557763675670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:59.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.218+0000 7f973161b700 1 -- 172.21.15.82:0/2170861535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557763670e40 msgr2=0x557763675670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:28:59.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.218+0000 7f973161b700 1 --2- 172.21.15.82:0/2170861535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557763670e40 0x557763675670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.218+0000 7f973161b700 1 -- 172.21.15.82:0/2170861535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557763667030 msgr2=0x5577636776f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-09T23:28:59.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.218+0000 7f973161b700 1 --2- 172.21.15.82:0/2170861535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557763667030 0x5577636776f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.218+0000 7f973161b700 1 -- 172.21.15.82:0/2170861535 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577633cdc60 con 0x557763666710 2023-09-09T23:28:59.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.218+0000 7f9730e1a700 1 --2- 172.21.15.82:0/2170861535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557763667030 0x5577636776f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-09T23:28:59.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.218+0000 7f973161b700 1 --2- 172.21.15.82:0/2170861535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557763666710 0x5577636771b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f971c012d80 tx=0x7f971c0036f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=583787d1ea8f19ca server_cookie=0 in_seq=0 out_seq=0 2023-09-09T23:28:59.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.218+0000 7f9714ff9700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2023-09-09T23:28:59.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.218+0000 7f9714ff9700 1 -- 172.21.15.82:0/2170861535 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c02c040 con 0x557763666710 2023-09-09T23:28:59.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.218+0000 7f973fa45b80 1 -- 172.21.15.82:0/2170861535 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f971c004020 con 0x557763666710 2023-09-09T23:28:59.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.218+0000 7f9714ff9700 1 -- 172.21.15.82:0/2170861535 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8455+0+0 (secure 0 0 0) 0x7f971c005040 con 0x557763666710 2023-09-09T23:28:59.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.218+0000 7f9714ff9700 1 -- 172.21.15.82:0/2170861535 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=127) v2 ==== 24+0+0 (secure 0 0 0) 0x7f971c00ee30 con 0x557763666710 2023-09-09T23:28:59.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.219+0000 7f973fa45b80 10 client.25928 fetch_fsmap learned FSMap version 127 2023-09-09T23:28:59.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.219+0000 7f973fa45b80 1 -- 172.21.15.82:0/2170861535 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=127,monmap=4+}) v3 -- 0x5577633e32b0 con 0x557763666710 2023-09-09T23:28:59.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.219+0000 7f9714ff9700 1 -- 172.21.15.82:0/2170861535 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f971c004020 con 0x557763666710 2023-09-09T23:28:59.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.219+0000 7f9714ff9700 1 -- 172.21.15.82:0/2170861535 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 127) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f971c038070 con 0x557763666710 2023-09-09T23:28:59.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.219+0000 7f973fa45b80 10 client.25928 fetch_fsmap finished waiting for FSMap version 127 2023-09-09T23:28:59.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.219+0000 7f973fa45b80 10 client.25928 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-09T23:28:59.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.219+0000 7f973fa45b80 20 client.25928 populate_metadata read hostname 'smithi082' 2023-09-09T23:28:59.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.219+0000 7f973fa45b80 1 --2- 172.21.15.82:0/2170861535 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5577636d5330 0x5577636d7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-09T23:28:59.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.219+0000 7f973fa45b80 4 client.25928 mds_command: new command op to 24566 tid=0[{"prefix": "scrub status"}] 2023-09-09T23:28:59.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.219+0000 7f973fa45b80 1 -- 172.21.15.82:0/2170861535 --> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5577635b1920 con 0x5577636d5330 2023-09-09T23:28:59.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.219+0000 7f9730e1a700 1 --2- 172.21.15.82:0/2170861535 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5577636d5330 0x5577636d7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-09T23:28:59.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.219+0000 7f9714ff9700 10 client.25928 ms_handle_connect on v2:172.21.15.98:6838/2785086371 2023-09-09T23:28:59.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.219+0000 7f9730e1a700 1 --2- 172.21.15.82:0/2170861535 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5577636d5330 0x5577636d7750 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=659643f0c2356577 server_cookie=cd6637cdae1dad14 in_seq=0 out_seq=0 2023-09-09T23:28:59.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.220+0000 7f9714ff9700 1 -- 172.21.15.82:0/2170861535 <== mds.0 v2:172.21.15.98:6838/2785086371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5577635b1920 con 0x5577636d5330 2023-09-09T23:28:59.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.220+0000 7f9714ff9700 10 client.25928 handle_command_reply: tid=0 2023-09-09T23:28:59.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.220+0000 7f973fa45b80 1 client.25928 shutdown 2023-09-09T23:28:59.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.220+0000 7f973fa45b80 1 -- 172.21.15.82:0/2170861535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557763666710 msgr2=0x5577636771b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:28:59.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.220+0000 7f973fa45b80 1 --2- 172.21.15.82:0/2170861535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557763666710 0x5577636771b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f971c012d80 tx=0x7f971c0036f0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.220+0000 7f973fa45b80 1 -- 172.21.15.82:0/2170861535 shutdown_connections 2023-09-09T23:28:59.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.220+0000 7f973fa45b80 1 --2- 172.21.15.82:0/2170861535 >> [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] conn(0x5577636d5330 0x5577636d7750 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.220+0000 7f973fa45b80 1 --2- 172.21.15.82:0/2170861535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557763666710 0x5577636771b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.220+0000 7f9714ff9700 0 client.25928 ms_handle_reset on v2:172.21.15.98:6838/2785086371 2023-09-09T23:28:59.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.220+0000 7f973fa45b80 1 --2- 172.21.15.82:0/2170861535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557763670e40 0x557763675670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.220+0000 7f973fa45b80 1 --2- 172.21.15.82:0/2170861535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557763667030 0x5577636776f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.220+0000 7f973fa45b80 1 -- 172.21.15.82:0/2170861535 >> 172.21.15.82:0/2170861535 conn(0x5577635db090 msgr2=0x557763668d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:28:59.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.220+0000 7f973fa45b80 1 -- 172.21.15.82:0/2170861535 shutdown_connections 2023-09-09T23:28:59.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.220+0000 7f973fa45b80 1 -- 172.21.15.82:0/2170861535 wait complete. 2023-09-09T23:28:59.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.220+0000 7f973fa45b80 20 client.25928 trim_cache size 0 max 16384 2023-09-09T23:28:59.240 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-09-09T23:28:59.240 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-09T23:28:59.240 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-09-09T23:28:59.241 INFO:teuthology.orchestra.run.smithi082.stdout: "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6": { 2023-09-09T23:28:59.241 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-09-09T23:28:59.241 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6", 2023-09-09T23:28:59.241 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-09-09T23:28:59.241 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:28:59.242 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-09-09T23:28:59.242 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-09-09T23:28:59.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.220+0000 7f973387f700 1 -- 172.21.15.82:0/1390860251 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f97180a07b0 msgr2=0x7f97180a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:28:59.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.220+0000 7f973387f700 1 --2- 172.21.15.82:0/1390860251 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f97180a07b0 0x7f97180a2c70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f971c009be0 tx=0x7f971c009260 comp rx=0 tx=0).stop 2023-09-09T23:28:59.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.220+0000 7f973387f700 1 -- 172.21.15.82:0/1390860251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f972c0ffe60 msgr2=0x7f972c19eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-09T23:28:59.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.220+0000 7f973387f700 1 --2- 172.21.15.82:0/1390860251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f972c0ffe60 0x7f972c19eb50 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f972800b580 tx=0x7f972800bfd0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.221+0000 7f973387f700 1 -- 172.21.15.82:0/1390860251 shutdown_connections 2023-09-09T23:28:59.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.221+0000 7f973387f700 1 --2- 172.21.15.82:0/1390860251 >> [v2:172.21.15.82:6800/4121269095,v1:172.21.15.82:6801/4121269095] conn(0x7f97180a07b0 0x7f97180a2c70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.221+0000 7f973387f700 1 --2- 172.21.15.82:0/1390860251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f972c105560 0x7f972c198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.221+0000 7f973387f700 1 --2- 172.21.15.82:0/1390860251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f972c0ff4a0 0x7f972c19e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.221+0000 7f973387f700 1 --2- 172.21.15.82:0/1390860251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f972c0ffe60 0x7f972c19eb50 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-09T23:28:59.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.221+0000 7f973387f700 1 -- 172.21.15.82:0/1390860251 >> 172.21.15.82:0/1390860251 conn(0x7f972c0faf50 msgr2=0x7f972c0687b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-09T23:28:59.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.221+0000 7f973387f700 1 -- 172.21.15.82:0/1390860251 shutdown_connections 2023-09-09T23:28:59.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-09-09T23:28:59.221+0000 7f973387f700 1 -- 172.21.15.82:0/1390860251 wait complete. 2023-09-09T23:28:59.246 INFO:tasks.cephfs.filesystem:scrub status for tag:a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6 - {'path': '/', 'tag': 'a32889a1-b159-4ba5-9d68-e0fcf2d8e4e6', 'options': 'recursive,force'} 2023-09-09T23:28:59.246 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_9670cca5bc8e17916865dc84939edfc10b1f2c00/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_9670cca5bc8e17916865dc84939edfc10b1f2c00/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_9670cca5bc8e17916865dc84939edfc10b1f2c00/qa/tasks/fwd_scrub.py", line 76, in _scrub done = self.fs.wait_until_scrub_complete(tag=tag, sleep=30, timeout=self.scrub_timeout) File "/home/teuthworker/src/git.ceph.com_ceph-c_9670cca5bc8e17916865dc84939edfc10b1f2c00/qa/tasks/cephfs/filesystem.py", line 1730, in wait_until_scrub_complete while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (31) after waiting for 900 seconds 2023-09-09T23:28:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:59 smithi170 ceph-mon[102832]: pgmap v473: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:28:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:28:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 570.755090 secs 2023-09-09T23:28:59.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:59 smithi082 ceph-mon[98694]: pgmap v473: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:28:59.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:28:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 570.755090 secs 2023-09-09T23:29:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:59 smithi098 ceph-mon[103124]: pgmap v473: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:29:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:28:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 570.755090 secs 2023-09-09T23:29:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:00 smithi170 ceph-mon[102832]: pgmap v474: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-09T23:29:00.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:00 smithi082 ceph-mon[98694]: pgmap v474: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-09T23:29:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:00 smithi098 ceph-mon[103124]: pgmap v474: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-09T23:29:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:01 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 573.426109 secs 2023-09-09T23:29:02.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:01 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 573.426109 secs 2023-09-09T23:29:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:01 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 573.426109 secs 2023-09-09T23:29:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:02 smithi170 ceph-mon[102832]: pgmap v475: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 21 op/s 2023-09-09T23:29:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:02 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 575.026097 secs 2023-09-09T23:29:03.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:02 smithi082 ceph-mon[98694]: pgmap v475: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 21 op/s 2023-09-09T23:29:03.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:02 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 575.026097 secs 2023-09-09T23:29:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:02 smithi098 ceph-mon[103124]: pgmap v475: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 21 op/s 2023-09-09T23:29:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:02 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 575.026097 secs 2023-09-09T23:29:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 575.755157 secs 2023-09-09T23:29:04.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 575.755157 secs 2023-09-09T23:29:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 575.755157 secs 2023-09-09T23:29:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:05 smithi170 ceph-mon[102832]: pgmap v476: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 20 op/s 2023-09-09T23:29:05.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:05 smithi082 ceph-mon[98694]: pgmap v476: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 20 op/s 2023-09-09T23:29:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:05 smithi098 ceph-mon[103124]: pgmap v476: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 20 op/s 2023-09-09T23:29:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:07 smithi170 ceph-mon[102832]: pgmap v477: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 27 op/s 2023-09-09T23:29:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 578.426229 secs 2023-09-09T23:29:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:07 smithi082 ceph-mon[98694]: pgmap v477: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 27 op/s 2023-09-09T23:29:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 578.426229 secs 2023-09-09T23:29:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:07 smithi098 ceph-mon[103124]: pgmap v477: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 27 op/s 2023-09-09T23:29:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 578.426229 secs 2023-09-09T23:29:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:08 smithi170 ceph-mon[102832]: osdmap e103: 12 total, 12 up, 12 in 2023-09-09T23:29:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 580.026165 secs 2023-09-09T23:29:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:08 smithi082 ceph-mon[98694]: osdmap e103: 12 total, 12 up, 12 in 2023-09-09T23:29:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 580.026165 secs 2023-09-09T23:29:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:08 smithi098 ceph-mon[103124]: osdmap e103: 12 total, 12 up, 12 in 2023-09-09T23:29:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 580.026165 secs 2023-09-09T23:29:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:09 smithi170 ceph-mon[102832]: pgmap v479: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:29:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 580.755224 secs 2023-09-09T23:29:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:09 smithi082 ceph-mon[98694]: pgmap v479: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:29:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 580.755224 secs 2023-09-09T23:29:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:09 smithi098 ceph-mon[103124]: pgmap v479: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:29:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 580.755224 secs 2023-09-09T23:29:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:11 smithi170 ceph-mon[102832]: pgmap v480: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:29:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:11 smithi082 ceph-mon[98694]: pgmap v480: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:29:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:11 smithi098 ceph-mon[103124]: pgmap v480: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:29:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 583.426288 secs 2023-09-09T23:29:12.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 583.426288 secs 2023-09-09T23:29:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 583.426288 secs 2023-09-09T23:29:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:13 smithi170 ceph-mon[102832]: pgmap v481: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 28 KiB/s wr, 13 op/s 2023-09-09T23:29:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 585.026224 secs 2023-09-09T23:29:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:13 smithi170 ceph-mon[102832]: osdmap e104: 12 total, 12 up, 12 in 2023-09-09T23:29:13.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:13 smithi082 ceph-mon[98694]: pgmap v481: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 28 KiB/s wr, 13 op/s 2023-09-09T23:29:13.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 585.026224 secs 2023-09-09T23:29:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:13 smithi082 ceph-mon[98694]: osdmap e104: 12 total, 12 up, 12 in 2023-09-09T23:29:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:13 smithi098 ceph-mon[103124]: pgmap v481: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 28 KiB/s wr, 13 op/s 2023-09-09T23:29:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 585.026224 secs 2023-09-09T23:29:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:13 smithi098 ceph-mon[103124]: osdmap e104: 12 total, 12 up, 12 in 2023-09-09T23:29:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 585.755289 secs 2023-09-09T23:29:14.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 585.755289 secs 2023-09-09T23:29:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 585.755289 secs 2023-09-09T23:29:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:15 smithi170 ceph-mon[102832]: pgmap v483: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:29:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:15 smithi082 ceph-mon[98694]: pgmap v483: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:29:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:15 smithi098 ceph-mon[103124]: pgmap v483: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:29:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:17 smithi170 ceph-mon[102832]: pgmap v484: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.3 KiB/s wr, 12 op/s 2023-09-09T23:29:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 588.426350 secs 2023-09-09T23:29:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:17 smithi082 ceph-mon[98694]: pgmap v484: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.3 KiB/s wr, 12 op/s 2023-09-09T23:29:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 588.426350 secs 2023-09-09T23:29:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:17 smithi098 ceph-mon[103124]: pgmap v484: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.3 KiB/s wr, 12 op/s 2023-09-09T23:29:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 588.426350 secs 2023-09-09T23:29:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 590.026283 secs 2023-09-09T23:29:18.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 590.026283 secs 2023-09-09T23:29:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 590.026283 secs 2023-09-09T23:29:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:19 smithi170 ceph-mon[102832]: pgmap v485: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2023-09-09T23:29:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 590.755355 secs 2023-09-09T23:29:19.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:19 smithi082 ceph-mon[98694]: pgmap v485: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2023-09-09T23:29:19.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 590.755355 secs 2023-09-09T23:29:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:19 smithi098 ceph-mon[103124]: pgmap v485: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2023-09-09T23:29:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 590.755355 secs 2023-09-09T23:29:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:21 smithi170 ceph-mon[102832]: pgmap v486: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-09-09T23:29:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:29:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:29:21.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:29:21.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:29:21.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:21 smithi082 ceph-mon[98694]: pgmap v486: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-09-09T23:29:21.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:29:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:29:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:29:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:29:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:21 smithi098 ceph-mon[103124]: pgmap v486: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-09-09T23:29:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:29:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:29:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:29:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:29:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 593.426409 secs 2023-09-09T23:29:22.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 593.426409 secs 2023-09-09T23:29:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 593.426409 secs 2023-09-09T23:29:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:23 smithi170 ceph-mon[102832]: pgmap v487: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-09-09T23:29:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 595.026342 secs 2023-09-09T23:29:23.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:23 smithi082 ceph-mon[98694]: pgmap v487: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-09-09T23:29:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 595.026342 secs 2023-09-09T23:29:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:23 smithi098 ceph-mon[103124]: pgmap v487: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-09-09T23:29:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 595.026342 secs 2023-09-09T23:29:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 595.755421 secs 2023-09-09T23:29:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:24 smithi170 ceph-mon[102832]: pgmap v488: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-09-09T23:29:24.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 595.755421 secs 2023-09-09T23:29:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:24 smithi082 ceph-mon[98694]: pgmap v488: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-09-09T23:29:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 595.755421 secs 2023-09-09T23:29:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:24 smithi098 ceph-mon[103124]: pgmap v488: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-09-09T23:29:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:27 smithi170 ceph-mon[102832]: pgmap v489: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 KiB/s wr, 16 op/s 2023-09-09T23:29:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 598.426531 secs 2023-09-09T23:29:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:27 smithi082 ceph-mon[98694]: pgmap v489: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 KiB/s wr, 16 op/s 2023-09-09T23:29:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 598.426531 secs 2023-09-09T23:29:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:27 smithi098 ceph-mon[103124]: pgmap v489: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 KiB/s wr, 16 op/s 2023-09-09T23:29:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 598.426531 secs 2023-09-09T23:29:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 600.026432 secs 2023-09-09T23:29:28.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 600.026432 secs 2023-09-09T23:29:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 600.026432 secs 2023-09-09T23:29:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:29 smithi170 ceph-mon[102832]: pgmap v490: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-09T23:29:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 600.755496 secs 2023-09-09T23:29:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:29 smithi082 ceph-mon[98694]: pgmap v490: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-09T23:29:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 600.755496 secs 2023-09-09T23:29:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:29 smithi098 ceph-mon[103124]: pgmap v490: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-09T23:29:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 600.755496 secs 2023-09-09T23:29:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:31 smithi170 ceph-mon[102832]: pgmap v491: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-09-09T23:29:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:31 smithi082 ceph-mon[98694]: pgmap v491: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-09-09T23:29:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:31 smithi098 ceph-mon[103124]: pgmap v491: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-09-09T23:29:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 603.426619 secs 2023-09-09T23:29:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 603.426619 secs 2023-09-09T23:29:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 603.426619 secs 2023-09-09T23:29:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:33 smithi170 ceph-mon[102832]: pgmap v492: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2023-09-09T23:29:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 605.026502 secs 2023-09-09T23:29:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:33 smithi082 ceph-mon[98694]: pgmap v492: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2023-09-09T23:29:33.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 605.026502 secs 2023-09-09T23:29:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:33 smithi098 ceph-mon[103124]: pgmap v492: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2023-09-09T23:29:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 605.026502 secs 2023-09-09T23:29:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 605.755564 secs 2023-09-09T23:29:34.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 605.755564 secs 2023-09-09T23:29:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 605.755564 secs 2023-09-09T23:29:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:35 smithi170 ceph-mon[102832]: pgmap v493: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2023-09-09T23:29:35.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:35 smithi082 ceph-mon[98694]: pgmap v493: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2023-09-09T23:29:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:35 smithi098 ceph-mon[103124]: pgmap v493: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2023-09-09T23:29:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:37 smithi170 ceph-mon[102832]: pgmap v494: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2023-09-09T23:29:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 608.426679 secs 2023-09-09T23:29:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:37 smithi082 ceph-mon[98694]: pgmap v494: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2023-09-09T23:29:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 608.426679 secs 2023-09-09T23:29:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:37 smithi098 ceph-mon[103124]: pgmap v494: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2023-09-09T23:29:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 608.426679 secs 2023-09-09T23:29:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 610.026560 secs 2023-09-09T23:29:38.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 610.026560 secs 2023-09-09T23:29:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 610.026560 secs 2023-09-09T23:29:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:39 smithi170 ceph-mon[102832]: pgmap v495: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.9 KiB/s wr, 8 op/s 2023-09-09T23:29:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 610.755630 secs 2023-09-09T23:29:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:39 smithi082 ceph-mon[98694]: pgmap v495: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.9 KiB/s wr, 8 op/s 2023-09-09T23:29:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 610.755630 secs 2023-09-09T23:29:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:39 smithi098 ceph-mon[103124]: pgmap v495: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.9 KiB/s wr, 8 op/s 2023-09-09T23:29:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 610.755630 secs 2023-09-09T23:29:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:41 smithi170 ceph-mon[102832]: pgmap v496: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2023-09-09T23:29:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:41 smithi082 ceph-mon[98694]: pgmap v496: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2023-09-09T23:29:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:41 smithi098 ceph-mon[103124]: pgmap v496: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2023-09-09T23:29:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 613.426740 secs 2023-09-09T23:29:42.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 613.426740 secs 2023-09-09T23:29:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 613.426740 secs 2023-09-09T23:29:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:43 smithi170 ceph-mon[102832]: pgmap v497: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:29:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 615.026618 secs 2023-09-09T23:29:43.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:43 smithi082 ceph-mon[98694]: pgmap v497: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:29:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 615.026618 secs 2023-09-09T23:29:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:43 smithi098 ceph-mon[103124]: pgmap v497: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:29:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 615.026618 secs 2023-09-09T23:29:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 615.755695 secs 2023-09-09T23:29:44.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 615.755695 secs 2023-09-09T23:29:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 615.755695 secs 2023-09-09T23:29:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:45 smithi170 ceph-mon[102832]: pgmap v498: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:29:45.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:45 smithi082 ceph-mon[98694]: pgmap v498: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:29:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:45 smithi098 ceph-mon[103124]: pgmap v498: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:29:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:47 smithi170 ceph-mon[102832]: pgmap v499: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:29:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 618.426791 secs 2023-09-09T23:29:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:47 smithi082 ceph-mon[98694]: pgmap v499: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:29:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 618.426791 secs 2023-09-09T23:29:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:47 smithi098 ceph-mon[103124]: pgmap v499: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:29:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 618.426791 secs 2023-09-09T23:29:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 620.026687 secs 2023-09-09T23:29:48.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 620.026687 secs 2023-09-09T23:29:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 620.026687 secs 2023-09-09T23:29:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:49 smithi170 ceph-mon[102832]: pgmap v500: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:29:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 620.755762 secs 2023-09-09T23:29:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:49 smithi082 ceph-mon[98694]: pgmap v500: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:29:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 620.755762 secs 2023-09-09T23:29:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:49 smithi098 ceph-mon[103124]: pgmap v500: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:29:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 620.755762 secs 2023-09-09T23:29:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:50 smithi170 ceph-mon[102832]: pgmap v501: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-09-09T23:29:50.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:50 smithi082 ceph-mon[98694]: pgmap v501: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-09-09T23:29:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:50 smithi098 ceph-mon[103124]: pgmap v501: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-09-09T23:29:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:51 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 623.426858 secs 2023-09-09T23:29:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:51 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 623.426858 secs 2023-09-09T23:29:52.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:51 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 623.426858 secs 2023-09-09T23:29:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:52 smithi098 ceph-mon[103124]: pgmap v502: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:29:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:52 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 625.026749 secs 2023-09-09T23:29:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:52 smithi170 ceph-mon[102832]: pgmap v502: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:29:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:52 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 625.026749 secs 2023-09-09T23:29:53.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:52 smithi082 ceph-mon[98694]: pgmap v502: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:29:53.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:52 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 625.026749 secs 2023-09-09T23:29:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 625.755833 secs 2023-09-09T23:29:54.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 625.755833 secs 2023-09-09T23:29:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 625.755833 secs 2023-09-09T23:29:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:55 smithi170 ceph-mon[102832]: pgmap v503: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:29:55.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:55 smithi082 ceph-mon[98694]: pgmap v503: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:29:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:55 smithi098 ceph-mon[103124]: pgmap v503: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:29:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:57 smithi170 ceph-mon[102832]: pgmap v504: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-09T23:29:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 628.426918 secs 2023-09-09T23:29:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:57 smithi082 ceph-mon[98694]: pgmap v504: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-09T23:29:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 628.426918 secs 2023-09-09T23:29:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:57 smithi098 ceph-mon[103124]: pgmap v504: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-09T23:29:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 628.426918 secs 2023-09-09T23:29:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 630.026811 secs 2023-09-09T23:29:58.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 630.026811 secs 2023-09-09T23:29:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 630.026811 secs 2023-09-09T23:29:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:59 smithi170 ceph-mon[102832]: pgmap v505: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:29:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:29:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 630.755900 secs 2023-09-09T23:29:59.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:59 smithi082 ceph-mon[98694]: pgmap v505: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:29:59.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:29:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 630.755900 secs 2023-09-09T23:30:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:59 smithi098 ceph-mon[103124]: pgmap v505: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:30:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:29:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 630.755900 secs 2023-09-09T23:30:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:00 smithi170 ceph-mon[102832]: Health detail: HEALTH_WARN 3 MDSs report slow requests 2023-09-09T23:30:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:00 smithi170 ceph-mon[102832]: [WRN] MDS_SLOW_REQUEST: 3 MDSs report slow requests 2023-09-09T23:30:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:00 smithi170 ceph-mon[102832]: mds.l(mds.4): 3 slow requests are blocked > 180 secs 2023-09-09T23:30:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:00 smithi170 ceph-mon[102832]: mds.h(mds.1): 3 slow requests are blocked > 180 secs 2023-09-09T23:30:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:00 smithi170 ceph-mon[102832]: mds.b(mds.0): 1 slow requests are blocked > 180 secs 2023-09-09T23:30:00.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:00 smithi082 ceph-mon[98694]: Health detail: HEALTH_WARN 3 MDSs report slow requests 2023-09-09T23:30:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:00 smithi082 ceph-mon[98694]: [WRN] MDS_SLOW_REQUEST: 3 MDSs report slow requests 2023-09-09T23:30:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:00 smithi082 ceph-mon[98694]: mds.l(mds.4): 3 slow requests are blocked > 180 secs 2023-09-09T23:30:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:00 smithi082 ceph-mon[98694]: mds.h(mds.1): 3 slow requests are blocked > 180 secs 2023-09-09T23:30:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:00 smithi082 ceph-mon[98694]: mds.b(mds.0): 1 slow requests are blocked > 180 secs 2023-09-09T23:30:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:00 smithi098 ceph-mon[103124]: Health detail: HEALTH_WARN 3 MDSs report slow requests 2023-09-09T23:30:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:00 smithi098 ceph-mon[103124]: [WRN] MDS_SLOW_REQUEST: 3 MDSs report slow requests 2023-09-09T23:30:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:00 smithi098 ceph-mon[103124]: mds.l(mds.4): 3 slow requests are blocked > 180 secs 2023-09-09T23:30:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:00 smithi098 ceph-mon[103124]: mds.h(mds.1): 3 slow requests are blocked > 180 secs 2023-09-09T23:30:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:00 smithi098 ceph-mon[103124]: mds.b(mds.0): 1 slow requests are blocked > 180 secs 2023-09-09T23:30:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:01 smithi170 ceph-mon[102832]: pgmap v506: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-09T23:30:01.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:01 smithi082 ceph-mon[98694]: pgmap v506: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-09T23:30:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:01 smithi098 ceph-mon[103124]: pgmap v506: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-09T23:30:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 633.427000 secs 2023-09-09T23:30:02.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 633.427000 secs 2023-09-09T23:30:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 633.427000 secs 2023-09-09T23:30:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:03 smithi170 ceph-mon[102832]: pgmap v507: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 21 op/s 2023-09-09T23:30:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 635.026865 secs 2023-09-09T23:30:03.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:03 smithi082 ceph-mon[98694]: pgmap v507: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 21 op/s 2023-09-09T23:30:03.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 635.026865 secs 2023-09-09T23:30:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:03 smithi098 ceph-mon[103124]: pgmap v507: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 21 op/s 2023-09-09T23:30:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 635.026865 secs 2023-09-09T23:30:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 635.755969 secs 2023-09-09T23:30:04.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 635.755969 secs 2023-09-09T23:30:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 635.755969 secs 2023-09-09T23:30:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:05 smithi170 ceph-mon[102832]: pgmap v508: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 20 op/s 2023-09-09T23:30:05.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:05 smithi082 ceph-mon[98694]: pgmap v508: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 20 op/s 2023-09-09T23:30:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:05 smithi098 ceph-mon[103124]: pgmap v508: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 20 op/s 2023-09-09T23:30:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:07 smithi170 ceph-mon[102832]: pgmap v509: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 27 op/s 2023-09-09T23:30:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 638.427060 secs 2023-09-09T23:30:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:07 smithi082 ceph-mon[98694]: pgmap v509: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 27 op/s 2023-09-09T23:30:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 638.427060 secs 2023-09-09T23:30:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:07 smithi098 ceph-mon[103124]: pgmap v509: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 27 op/s 2023-09-09T23:30:08.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 638.427060 secs 2023-09-09T23:30:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:08 smithi170 ceph-mon[102832]: osdmap e105: 12 total, 12 up, 12 in 2023-09-09T23:30:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 640.026937 secs 2023-09-09T23:30:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:08 smithi170 ceph-mon[102832]: pgmap v511: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:30:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:08 smithi082 ceph-mon[98694]: osdmap e105: 12 total, 12 up, 12 in 2023-09-09T23:30:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 640.026937 secs 2023-09-09T23:30:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:08 smithi082 ceph-mon[98694]: pgmap v511: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:30:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:08 smithi098 ceph-mon[103124]: osdmap e105: 12 total, 12 up, 12 in 2023-09-09T23:30:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 640.026937 secs 2023-09-09T23:30:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:08 smithi098 ceph-mon[103124]: pgmap v511: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:30:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 640.756034 secs 2023-09-09T23:30:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 640.756034 secs 2023-09-09T23:30:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 640.756034 secs 2023-09-09T23:30:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:10 smithi170 ceph-mon[102832]: pgmap v512: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:30:10.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:10 smithi082 ceph-mon[98694]: pgmap v512: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:30:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:10 smithi098 ceph-mon[103124]: pgmap v512: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:30:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:11 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 643.427180 secs 2023-09-09T23:30:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:11 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 643.427180 secs 2023-09-09T23:30:12.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:11 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 643.427180 secs 2023-09-09T23:30:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:12 smithi098 ceph-mon[103124]: pgmap v513: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 28 KiB/s wr, 13 op/s 2023-09-09T23:30:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:12 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 645.026994 secs 2023-09-09T23:30:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:12 smithi170 ceph-mon[102832]: pgmap v513: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 28 KiB/s wr, 13 op/s 2023-09-09T23:30:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:12 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 645.026994 secs 2023-09-09T23:30:13.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:12 smithi082 ceph-mon[98694]: pgmap v513: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 28 KiB/s wr, 13 op/s 2023-09-09T23:30:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:12 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 645.026994 secs 2023-09-09T23:30:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 645.756101 secs 2023-09-09T23:30:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:14 smithi170 ceph-mon[102832]: osdmap e106: 12 total, 12 up, 12 in 2023-09-09T23:30:14.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 645.756101 secs 2023-09-09T23:30:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:14 smithi082 ceph-mon[98694]: osdmap e106: 12 total, 12 up, 12 in 2023-09-09T23:30:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 645.756101 secs 2023-09-09T23:30:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:14 smithi098 ceph-mon[103124]: osdmap e106: 12 total, 12 up, 12 in 2023-09-09T23:30:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:15 smithi170 ceph-mon[102832]: pgmap v515: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:30:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:15 smithi082 ceph-mon[98694]: pgmap v515: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:30:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:15 smithi098 ceph-mon[103124]: pgmap v515: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:30:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:17 smithi170 ceph-mon[102832]: pgmap v516: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.6 KiB/s wr, 12 op/s 2023-09-09T23:30:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 648.427273 secs 2023-09-09T23:30:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:17 smithi082 ceph-mon[98694]: pgmap v516: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.6 KiB/s wr, 12 op/s 2023-09-09T23:30:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 648.427273 secs 2023-09-09T23:30:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:17 smithi098 ceph-mon[103124]: pgmap v516: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.6 KiB/s wr, 12 op/s 2023-09-09T23:30:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 648.427273 secs 2023-09-09T23:30:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 650.027082 secs 2023-09-09T23:30:18.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 650.027082 secs 2023-09-09T23:30:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 650.027082 secs 2023-09-09T23:30:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:19 smithi170 ceph-mon[102832]: pgmap v517: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.9 KiB/s wr, 11 op/s 2023-09-09T23:30:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 650.756165 secs 2023-09-09T23:30:19.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:19 smithi082 ceph-mon[98694]: pgmap v517: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.9 KiB/s wr, 11 op/s 2023-09-09T23:30:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 650.756165 secs 2023-09-09T23:30:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:19 smithi098 ceph-mon[103124]: pgmap v517: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.9 KiB/s wr, 11 op/s 2023-09-09T23:30:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 650.756165 secs 2023-09-09T23:30:21.862 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:21 smithi082 ceph-mon[98694]: pgmap v518: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-09-09T23:30:21.893 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:21 smithi098 ceph-mon[103124]: pgmap v518: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-09-09T23:30:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:21 smithi170 ceph-mon[102832]: pgmap v518: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-09-09T23:30:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:30:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 653.427350 secs 2023-09-09T23:30:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:30:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:30:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:30:22.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:30:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 653.427350 secs 2023-09-09T23:30:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:30:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:30:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:30:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:30:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 653.427350 secs 2023-09-09T23:30:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:30:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:30:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:30:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:23 smithi170 ceph-mon[102832]: pgmap v519: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 13 op/s 2023-09-09T23:30:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 655.027166 secs 2023-09-09T23:30:23.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:23 smithi082 ceph-mon[98694]: pgmap v519: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 13 op/s 2023-09-09T23:30:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 655.027166 secs 2023-09-09T23:30:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:23 smithi098 ceph-mon[103124]: pgmap v519: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 13 op/s 2023-09-09T23:30:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 655.027166 secs 2023-09-09T23:30:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 655.756232 secs 2023-09-09T23:30:24.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 655.756232 secs 2023-09-09T23:30:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 655.756232 secs 2023-09-09T23:30:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:25 smithi170 ceph-mon[102832]: pgmap v520: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 KiB/s wr, 12 op/s 2023-09-09T23:30:25.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:25 smithi082 ceph-mon[98694]: pgmap v520: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 KiB/s wr, 12 op/s 2023-09-09T23:30:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:25 smithi098 ceph-mon[103124]: pgmap v520: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 KiB/s wr, 12 op/s 2023-09-09T23:30:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:27 smithi170 ceph-mon[102832]: pgmap v521: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 19 KiB/s wr, 16 op/s 2023-09-09T23:30:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 658.427410 secs 2023-09-09T23:30:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:27 smithi082 ceph-mon[98694]: pgmap v521: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 19 KiB/s wr, 16 op/s 2023-09-09T23:30:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 658.427410 secs 2023-09-09T23:30:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:27 smithi098 ceph-mon[103124]: pgmap v521: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 19 KiB/s wr, 16 op/s 2023-09-09T23:30:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 658.427410 secs 2023-09-09T23:30:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 660.027224 secs 2023-09-09T23:30:28.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 660.027224 secs 2023-09-09T23:30:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 660.027224 secs 2023-09-09T23:30:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:29 smithi170 ceph-mon[102832]: pgmap v522: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-09T23:30:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 660.756299 secs 2023-09-09T23:30:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:29 smithi082 ceph-mon[98694]: pgmap v522: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-09T23:30:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 660.756299 secs 2023-09-09T23:30:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:29 smithi098 ceph-mon[103124]: pgmap v522: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-09T23:30:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 660.756299 secs 2023-09-09T23:30:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:31 smithi170 ceph-mon[102832]: pgmap v523: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-09-09T23:30:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:31 smithi082 ceph-mon[98694]: pgmap v523: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-09-09T23:30:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:31 smithi098 ceph-mon[103124]: pgmap v523: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-09-09T23:30:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 663.427470 secs 2023-09-09T23:30:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 663.427470 secs 2023-09-09T23:30:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 663.427470 secs 2023-09-09T23:30:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:33 smithi170 ceph-mon[102832]: pgmap v524: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2023-09-09T23:30:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 665.027283 secs 2023-09-09T23:30:33.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:33 smithi082 ceph-mon[98694]: pgmap v524: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2023-09-09T23:30:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 665.027283 secs 2023-09-09T23:30:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:33 smithi098 ceph-mon[103124]: pgmap v524: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2023-09-09T23:30:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 665.027283 secs 2023-09-09T23:30:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 665.756364 secs 2023-09-09T23:30:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:34 smithi170 ceph-mon[102832]: pgmap v525: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2023-09-09T23:30:34.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 665.756364 secs 2023-09-09T23:30:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:34 smithi082 ceph-mon[98694]: pgmap v525: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2023-09-09T23:30:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 665.756364 secs 2023-09-09T23:30:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:34 smithi098 ceph-mon[103124]: pgmap v525: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2023-09-09T23:30:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:37 smithi170 ceph-mon[102832]: pgmap v526: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2023-09-09T23:30:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 668.427581 secs 2023-09-09T23:30:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:37 smithi082 ceph-mon[98694]: pgmap v526: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2023-09-09T23:30:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 668.427581 secs 2023-09-09T23:30:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:37 smithi098 ceph-mon[103124]: pgmap v526: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2023-09-09T23:30:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 668.427581 secs 2023-09-09T23:30:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 670.027342 secs 2023-09-09T23:30:38.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 670.027342 secs 2023-09-09T23:30:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 670.027342 secs 2023-09-09T23:30:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:39 smithi170 ceph-mon[102832]: pgmap v527: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2023-09-09T23:30:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 670.756432 secs 2023-09-09T23:30:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:39 smithi082 ceph-mon[98694]: pgmap v527: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2023-09-09T23:30:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 670.756432 secs 2023-09-09T23:30:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:39 smithi098 ceph-mon[103124]: pgmap v527: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2023-09-09T23:30:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 670.756432 secs 2023-09-09T23:30:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:41 smithi170 ceph-mon[102832]: pgmap v528: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.1 KiB/s wr, 11 op/s 2023-09-09T23:30:41.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:41 smithi082 ceph-mon[98694]: pgmap v528: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.1 KiB/s wr, 11 op/s 2023-09-09T23:30:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:41 smithi098 ceph-mon[103124]: pgmap v528: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.1 KiB/s wr, 11 op/s 2023-09-09T23:30:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 673.427644 secs 2023-09-09T23:30:42.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 673.427644 secs 2023-09-09T23:30:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 673.427644 secs 2023-09-09T23:30:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:43 smithi170 ceph-mon[102832]: pgmap v529: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:30:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 675.027396 secs 2023-09-09T23:30:43.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:43 smithi082 ceph-mon[98694]: pgmap v529: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:30:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 675.027396 secs 2023-09-09T23:30:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:43 smithi098 ceph-mon[103124]: pgmap v529: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:30:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 675.027396 secs 2023-09-09T23:30:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 675.756491 secs 2023-09-09T23:30:44.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 675.756491 secs 2023-09-09T23:30:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 675.756491 secs 2023-09-09T23:30:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:45 smithi170 ceph-mon[102832]: pgmap v530: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:30:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:45 smithi082 ceph-mon[98694]: pgmap v530: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:30:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:45 smithi098 ceph-mon[103124]: pgmap v530: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:30:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:47 smithi170 ceph-mon[102832]: pgmap v531: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:30:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 678.427746 secs 2023-09-09T23:30:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:47 smithi082 ceph-mon[98694]: pgmap v531: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:30:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 678.427746 secs 2023-09-09T23:30:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:47 smithi098 ceph-mon[103124]: pgmap v531: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:30:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 678.427746 secs 2023-09-09T23:30:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 680.027479 secs 2023-09-09T23:30:48.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 680.027479 secs 2023-09-09T23:30:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 680.027479 secs 2023-09-09T23:30:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:49 smithi170 ceph-mon[102832]: pgmap v532: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:30:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 680.756559 secs 2023-09-09T23:30:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:49 smithi082 ceph-mon[98694]: pgmap v532: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:30:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 680.756559 secs 2023-09-09T23:30:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:49 smithi098 ceph-mon[103124]: pgmap v532: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:30:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 680.756559 secs 2023-09-09T23:30:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:51 smithi170 ceph-mon[102832]: pgmap v533: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-09-09T23:30:51.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:51 smithi082 ceph-mon[98694]: pgmap v533: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-09-09T23:30:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:51 smithi098 ceph-mon[103124]: pgmap v533: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-09-09T23:30:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 683.427790 secs 2023-09-09T23:30:52.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 683.427790 secs 2023-09-09T23:30:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 683.427790 secs 2023-09-09T23:30:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:53 smithi170 ceph-mon[102832]: pgmap v534: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:30:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 685.027525 secs 2023-09-09T23:30:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:53 smithi082 ceph-mon[98694]: pgmap v534: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:30:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 685.027525 secs 2023-09-09T23:30:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:53 smithi098 ceph-mon[103124]: pgmap v534: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:30:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 685.027525 secs 2023-09-09T23:30:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 685.756627 secs 2023-09-09T23:30:54.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 685.756627 secs 2023-09-09T23:30:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 685.756627 secs 2023-09-09T23:30:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:55 smithi170 ceph-mon[102832]: pgmap v535: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:30:55.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:55 smithi082 ceph-mon[98694]: pgmap v535: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:30:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:55 smithi098 ceph-mon[103124]: pgmap v535: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:30:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:56 smithi170 ceph-mon[102832]: pgmap v536: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-09T23:30:56.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:56 smithi082 ceph-mon[98694]: pgmap v536: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-09T23:30:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:56 smithi098 ceph-mon[103124]: pgmap v536: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-09T23:30:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 688.427848 secs 2023-09-09T23:30:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 688.427848 secs 2023-09-09T23:30:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 688.427848 secs 2023-09-09T23:30:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 690.027624 secs 2023-09-09T23:30:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:58 smithi170 ceph-mon[102832]: pgmap v537: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:30:58.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 690.027624 secs 2023-09-09T23:30:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:58 smithi082 ceph-mon[98694]: pgmap v537: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:30:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 690.027624 secs 2023-09-09T23:30:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:58 smithi098 ceph-mon[103124]: pgmap v537: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:30:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:30:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 690.756692 secs 2023-09-09T23:30:59.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:30:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 690.756692 secs 2023-09-09T23:31:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:30:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 690.756692 secs 2023-09-09T23:31:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:00 smithi170 ceph-mon[102832]: pgmap v538: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-09T23:31:00.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:00 smithi082 ceph-mon[98694]: pgmap v538: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-09T23:31:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:00 smithi098 ceph-mon[103124]: pgmap v538: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-09T23:31:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:01 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 693.427908 secs 2023-09-09T23:31:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:01 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 693.427908 secs 2023-09-09T23:31:02.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:01 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 693.427908 secs 2023-09-09T23:31:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:02 smithi098 ceph-mon[103124]: pgmap v539: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 11 KiB/s wr, 13 op/s 2023-09-09T23:31:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:02 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 695.027682 secs 2023-09-09T23:31:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:02 smithi170 ceph-mon[102832]: pgmap v539: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 11 KiB/s wr, 13 op/s 2023-09-09T23:31:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:02 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 695.027682 secs 2023-09-09T23:31:03.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:02 smithi082 ceph-mon[98694]: pgmap v539: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 11 KiB/s wr, 13 op/s 2023-09-09T23:31:03.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:02 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 695.027682 secs 2023-09-09T23:31:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 695.756756 secs 2023-09-09T23:31:04.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 695.756756 secs 2023-09-09T23:31:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 695.756756 secs 2023-09-09T23:31:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:05 smithi170 ceph-mon[102832]: pgmap v540: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 11 KiB/s wr, 13 op/s 2023-09-09T23:31:05.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:05 smithi082 ceph-mon[98694]: pgmap v540: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 11 KiB/s wr, 13 op/s 2023-09-09T23:31:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:05 smithi098 ceph-mon[103124]: pgmap v540: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 11 KiB/s wr, 13 op/s 2023-09-09T23:31:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:07 smithi170 ceph-mon[102832]: pgmap v541: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 27 op/s 2023-09-09T23:31:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 698.427969 secs 2023-09-09T23:31:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:07 smithi082 ceph-mon[98694]: pgmap v541: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 27 op/s 2023-09-09T23:31:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 698.427969 secs 2023-09-09T23:31:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:07 smithi098 ceph-mon[103124]: pgmap v541: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 27 op/s 2023-09-09T23:31:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 698.427969 secs 2023-09-09T23:31:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:08 smithi170 ceph-mon[102832]: osdmap e107: 12 total, 12 up, 12 in 2023-09-09T23:31:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 700.027740 secs 2023-09-09T23:31:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:08 smithi082 ceph-mon[98694]: osdmap e107: 12 total, 12 up, 12 in 2023-09-09T23:31:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 700.027740 secs 2023-09-09T23:31:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:08 smithi098 ceph-mon[103124]: osdmap e107: 12 total, 12 up, 12 in 2023-09-09T23:31:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 700.027740 secs 2023-09-09T23:31:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:09 smithi170 ceph-mon[102832]: pgmap v543: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:31:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 700.756822 secs 2023-09-09T23:31:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:09 smithi082 ceph-mon[98694]: pgmap v543: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:31:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 700.756822 secs 2023-09-09T23:31:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:09 smithi098 ceph-mon[103124]: pgmap v543: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:31:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 700.756822 secs 2023-09-09T23:31:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:11 smithi170 ceph-mon[102832]: pgmap v544: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:31:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:11 smithi082 ceph-mon[98694]: pgmap v544: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:31:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:11 smithi098 ceph-mon[103124]: pgmap v544: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:31:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 703.428083 secs 2023-09-09T23:31:12.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 703.428083 secs 2023-09-09T23:31:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 703.428083 secs 2023-09-09T23:31:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:13 smithi170 ceph-mon[102832]: pgmap v545: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 41 KiB/s wr, 22 op/s 2023-09-09T23:31:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 705.027799 secs 2023-09-09T23:31:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:13 smithi170 ceph-mon[102832]: osdmap e108: 12 total, 12 up, 12 in 2023-09-09T23:31:13.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:13 smithi082 ceph-mon[98694]: pgmap v545: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 41 KiB/s wr, 22 op/s 2023-09-09T23:31:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 705.027799 secs 2023-09-09T23:31:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:13 smithi082 ceph-mon[98694]: osdmap e108: 12 total, 12 up, 12 in 2023-09-09T23:31:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:13 smithi098 ceph-mon[103124]: pgmap v545: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 41 KiB/s wr, 22 op/s 2023-09-09T23:31:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 705.027799 secs 2023-09-09T23:31:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:13 smithi098 ceph-mon[103124]: osdmap e108: 12 total, 12 up, 12 in 2023-09-09T23:31:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 705.756884 secs 2023-09-09T23:31:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 705.756884 secs 2023-09-09T23:31:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 705.756884 secs 2023-09-09T23:31:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:15 smithi170 ceph-mon[102832]: pgmap v547: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:31:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:15 smithi082 ceph-mon[98694]: pgmap v547: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:31:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:15 smithi098 ceph-mon[103124]: pgmap v547: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:31:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:17 smithi170 ceph-mon[102832]: pgmap v548: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 KiB/s wr, 13 op/s 2023-09-09T23:31:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 708.428142 secs 2023-09-09T23:31:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:17 smithi082 ceph-mon[98694]: pgmap v548: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 KiB/s wr, 13 op/s 2023-09-09T23:31:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 708.428142 secs 2023-09-09T23:31:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:17 smithi098 ceph-mon[103124]: pgmap v548: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 KiB/s wr, 13 op/s 2023-09-09T23:31:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 708.428142 secs 2023-09-09T23:31:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 710.027856 secs 2023-09-09T23:31:18.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 710.027856 secs 2023-09-09T23:31:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 710.027856 secs 2023-09-09T23:31:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:19 smithi170 ceph-mon[102832]: pgmap v549: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 KiB/s wr, 11 op/s 2023-09-09T23:31:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 710.756948 secs 2023-09-09T23:31:19.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:19 smithi082 ceph-mon[98694]: pgmap v549: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 KiB/s wr, 11 op/s 2023-09-09T23:31:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 710.756948 secs 2023-09-09T23:31:20.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:19 smithi098 ceph-mon[103124]: pgmap v549: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 KiB/s wr, 11 op/s 2023-09-09T23:31:20.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 710.756948 secs 2023-09-09T23:31:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:20 smithi170 ceph-mon[102832]: pgmap v550: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 KiB/s wr, 13 op/s 2023-09-09T23:31:20.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:20 smithi082 ceph-mon[98694]: pgmap v550: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 KiB/s wr, 13 op/s 2023-09-09T23:31:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:20 smithi098 ceph-mon[103124]: pgmap v550: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 KiB/s wr, 13 op/s 2023-09-09T23:31:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:21 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 713.428247 secs 2023-09-09T23:31:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:21 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 713.428247 secs 2023-09-09T23:31:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:21 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 713.428247 secs 2023-09-09T23:31:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:31:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:23 smithi098 ceph-mon[103124]: pgmap v551: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 KiB/s wr, 13 op/s 2023-09-09T23:31:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 715.027914 secs 2023-09-09T23:31:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:31:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:31:23.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:31:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:31:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:23 smithi170 ceph-mon[102832]: pgmap v551: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 KiB/s wr, 13 op/s 2023-09-09T23:31:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 715.027914 secs 2023-09-09T23:31:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:31:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:31:23.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:31:23.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:31:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:23 smithi082 ceph-mon[98694]: pgmap v551: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 KiB/s wr, 13 op/s 2023-09-09T23:31:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 715.027914 secs 2023-09-09T23:31:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:31:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:31:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:31:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 715.757012 secs 2023-09-09T23:31:24.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 715.757012 secs 2023-09-09T23:31:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 715.757012 secs 2023-09-09T23:31:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:25 smithi170 ceph-mon[102832]: pgmap v552: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 KiB/s wr, 12 op/s 2023-09-09T23:31:25.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:25 smithi082 ceph-mon[98694]: pgmap v552: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 KiB/s wr, 12 op/s 2023-09-09T23:31:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:25 smithi098 ceph-mon[103124]: pgmap v552: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 KiB/s wr, 12 op/s 2023-09-09T23:31:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:27 smithi170 ceph-mon[102832]: pgmap v553: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 16 op/s 2023-09-09T23:31:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 718.428306 secs 2023-09-09T23:31:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:27 smithi082 ceph-mon[98694]: pgmap v553: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 16 op/s 2023-09-09T23:31:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 718.428306 secs 2023-09-09T23:31:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:27 smithi098 ceph-mon[103124]: pgmap v553: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 16 op/s 2023-09-09T23:31:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 718.428306 secs 2023-09-09T23:31:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:28 smithi170 ceph-mon[102832]: 3 slow requests, 3 included below; oldest blocked for > 720.027971 secs 2023-09-09T23:31:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:28 smithi170 ceph-mon[102832]: slow request 720.027971 seconds old, received at 2023-09-09T23:19:27.740042+0000: client_request(mds.4:47 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently acquired locks 2023-09-09T23:31:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:28 smithi170 ceph-mon[102832]: slow request 720.027591 seconds old, received at 2023-09-09T23:19:27.740422+0000: client_request(mds.4:48 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently requesting remote authpins 2023-09-09T23:31:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:28 smithi170 ceph-mon[102832]: slow request 720.025370 seconds old, received at 2023-09-09T23:19:27.742643+0000: client_request(client.24913:3869 unlink #0x10000000007/00000001000000010000000B 2023-09-09T23:19:27.741863+0000 caller_uid=26, caller_gid=26{26,}) currently requesting remote authpins 2023-09-09T23:31:28.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:28 smithi082 ceph-mon[98694]: 3 slow requests, 3 included below; oldest blocked for > 720.027971 secs 2023-09-09T23:31:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:28 smithi082 ceph-mon[98694]: slow request 720.027971 seconds old, received at 2023-09-09T23:19:27.740042+0000: client_request(mds.4:47 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently acquired locks 2023-09-09T23:31:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:28 smithi082 ceph-mon[98694]: slow request 720.027591 seconds old, received at 2023-09-09T23:19:27.740422+0000: client_request(mds.4:48 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently requesting remote authpins 2023-09-09T23:31:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:28 smithi082 ceph-mon[98694]: slow request 720.025370 seconds old, received at 2023-09-09T23:19:27.742643+0000: client_request(client.24913:3869 unlink #0x10000000007/00000001000000010000000B 2023-09-09T23:19:27.741863+0000 caller_uid=26, caller_gid=26{26,}) currently requesting remote authpins 2023-09-09T23:31:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:28 smithi098 ceph-mon[103124]: 3 slow requests, 3 included below; oldest blocked for > 720.027971 secs 2023-09-09T23:31:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:28 smithi098 ceph-mon[103124]: slow request 720.027971 seconds old, received at 2023-09-09T23:19:27.740042+0000: client_request(mds.4:47 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently acquired locks 2023-09-09T23:31:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:28 smithi098 ceph-mon[103124]: slow request 720.027591 seconds old, received at 2023-09-09T23:19:27.740422+0000: client_request(mds.4:48 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently requesting remote authpins 2023-09-09T23:31:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:28 smithi098 ceph-mon[103124]: slow request 720.025370 seconds old, received at 2023-09-09T23:19:27.742643+0000: client_request(client.24913:3869 unlink #0x10000000007/00000001000000010000000B 2023-09-09T23:19:27.741863+0000 caller_uid=26, caller_gid=26{26,}) currently requesting remote authpins 2023-09-09T23:31:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:29 smithi170 ceph-mon[102832]: pgmap v554: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-09T23:31:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:29 smithi170 ceph-mon[102832]: 3 slow requests, 3 included below; oldest blocked for > 720.757079 secs 2023-09-09T23:31:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:29 smithi170 ceph-mon[102832]: slow request 720.757078 seconds old, received at 2023-09-09T23:19:27.740800+0000: peer_request:mds.4:47 currently acquired locks 2023-09-09T23:31:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:29 smithi170 ceph-mon[102832]: slow request 720.756844 seconds old, received at 2023-09-09T23:19:27.741034+0000: peer_request(mds.4:48.0 authpin) currently dispatched 2023-09-09T23:31:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:29 smithi170 ceph-mon[102832]: slow request 720.754808 seconds old, received at 2023-09-09T23:19:27.743070+0000: peer_request(client.24913:3869.0 authpin) currently dispatched 2023-09-09T23:31:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:29 smithi082 ceph-mon[98694]: pgmap v554: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-09T23:31:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:29 smithi082 ceph-mon[98694]: 3 slow requests, 3 included below; oldest blocked for > 720.757079 secs 2023-09-09T23:31:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:29 smithi082 ceph-mon[98694]: slow request 720.757078 seconds old, received at 2023-09-09T23:19:27.740800+0000: peer_request:mds.4:47 currently acquired locks 2023-09-09T23:31:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:29 smithi082 ceph-mon[98694]: slow request 720.756844 seconds old, received at 2023-09-09T23:19:27.741034+0000: peer_request(mds.4:48.0 authpin) currently dispatched 2023-09-09T23:31:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:29 smithi082 ceph-mon[98694]: slow request 720.754808 seconds old, received at 2023-09-09T23:19:27.743070+0000: peer_request(client.24913:3869.0 authpin) currently dispatched 2023-09-09T23:31:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:29 smithi098 ceph-mon[103124]: pgmap v554: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-09T23:31:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:29 smithi098 ceph-mon[103124]: 3 slow requests, 3 included below; oldest blocked for > 720.757079 secs 2023-09-09T23:31:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:29 smithi098 ceph-mon[103124]: slow request 720.757078 seconds old, received at 2023-09-09T23:19:27.740800+0000: peer_request:mds.4:47 currently acquired locks 2023-09-09T23:31:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:29 smithi098 ceph-mon[103124]: slow request 720.756844 seconds old, received at 2023-09-09T23:19:27.741034+0000: peer_request(mds.4:48.0 authpin) currently dispatched 2023-09-09T23:31:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:29 smithi098 ceph-mon[103124]: slow request 720.754808 seconds old, received at 2023-09-09T23:19:27.743070+0000: peer_request(client.24913:3869.0 authpin) currently dispatched 2023-09-09T23:31:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:31 smithi170 ceph-mon[102832]: pgmap v555: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 14 op/s 2023-09-09T23:31:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:31 smithi082 ceph-mon[98694]: pgmap v555: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 14 op/s 2023-09-09T23:31:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:31 smithi098 ceph-mon[103124]: pgmap v555: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 14 op/s 2023-09-09T23:31:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:32 smithi170 ceph-mon[102832]: 1 slow requests, 1 included below; oldest blocked for > 723.428367 secs 2023-09-09T23:31:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:32 smithi170 ceph-mon[102832]: slow request 723.428367 seconds old, received at 2023-09-09T23:19:28.303507+0000: peer_request:mds.4:47 currently dispatched 2023-09-09T23:31:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:32 smithi082 ceph-mon[98694]: 1 slow requests, 1 included below; oldest blocked for > 723.428367 secs 2023-09-09T23:31:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:32 smithi082 ceph-mon[98694]: slow request 723.428367 seconds old, received at 2023-09-09T23:19:28.303507+0000: peer_request:mds.4:47 currently dispatched 2023-09-09T23:31:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:32 smithi098 ceph-mon[103124]: 1 slow requests, 1 included below; oldest blocked for > 723.428367 secs 2023-09-09T23:31:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:32 smithi098 ceph-mon[103124]: slow request 723.428367 seconds old, received at 2023-09-09T23:19:28.303507+0000: peer_request:mds.4:47 currently dispatched 2023-09-09T23:31:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:33 smithi170 ceph-mon[102832]: pgmap v556: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2023-09-09T23:31:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 725.028050 secs 2023-09-09T23:31:33.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:33 smithi082 ceph-mon[98694]: pgmap v556: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2023-09-09T23:31:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 725.028050 secs 2023-09-09T23:31:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:33 smithi098 ceph-mon[103124]: pgmap v556: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2023-09-09T23:31:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 725.028050 secs 2023-09-09T23:31:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 725.757146 secs 2023-09-09T23:31:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 725.757146 secs 2023-09-09T23:31:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 725.757146 secs 2023-09-09T23:31:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:35 smithi170 ceph-mon[102832]: pgmap v557: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2023-09-09T23:31:35.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:35 smithi082 ceph-mon[98694]: pgmap v557: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2023-09-09T23:31:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:35 smithi098 ceph-mon[103124]: pgmap v557: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2023-09-09T23:31:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:37 smithi170 ceph-mon[102832]: pgmap v558: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 KiB/s wr, 13 op/s 2023-09-09T23:31:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 728.428426 secs 2023-09-09T23:31:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:37 smithi082 ceph-mon[98694]: pgmap v558: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 KiB/s wr, 13 op/s 2023-09-09T23:31:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 728.428426 secs 2023-09-09T23:31:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:37 smithi098 ceph-mon[103124]: pgmap v558: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 KiB/s wr, 13 op/s 2023-09-09T23:31:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 728.428426 secs 2023-09-09T23:31:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 730.028108 secs 2023-09-09T23:31:38.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 730.028108 secs 2023-09-09T23:31:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 730.028108 secs 2023-09-09T23:31:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:39 smithi170 ceph-mon[102832]: pgmap v559: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2023-09-09T23:31:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 730.757211 secs 2023-09-09T23:31:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:39 smithi082 ceph-mon[98694]: pgmap v559: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2023-09-09T23:31:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 730.757211 secs 2023-09-09T23:31:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:39 smithi098 ceph-mon[103124]: pgmap v559: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2023-09-09T23:31:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 730.757211 secs 2023-09-09T23:31:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:40 smithi170 ceph-mon[102832]: pgmap v560: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2023-09-09T23:31:40.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:40 smithi082 ceph-mon[98694]: pgmap v560: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2023-09-09T23:31:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:40 smithi098 ceph-mon[103124]: pgmap v560: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2023-09-09T23:31:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:41 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 733.428564 secs 2023-09-09T23:31:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:41 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 733.428564 secs 2023-09-09T23:31:42.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:41 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 733.428564 secs 2023-09-09T23:31:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:43 smithi098 ceph-mon[103124]: pgmap v561: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:31:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 735.028177 secs 2023-09-09T23:31:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:43 smithi170 ceph-mon[102832]: pgmap v561: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:31:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 735.028177 secs 2023-09-09T23:31:43.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:43 smithi082 ceph-mon[98694]: pgmap v561: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:31:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 735.028177 secs 2023-09-09T23:31:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 735.757278 secs 2023-09-09T23:31:44.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 735.757278 secs 2023-09-09T23:31:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 735.757278 secs 2023-09-09T23:31:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:45 smithi170 ceph-mon[102832]: pgmap v562: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:31:45.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:45 smithi082 ceph-mon[98694]: pgmap v562: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:31:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:45 smithi098 ceph-mon[103124]: pgmap v562: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:31:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:47 smithi170 ceph-mon[102832]: pgmap v563: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:31:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 738.428626 secs 2023-09-09T23:31:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:47 smithi082 ceph-mon[98694]: pgmap v563: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:31:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 738.428626 secs 2023-09-09T23:31:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:47 smithi098 ceph-mon[103124]: pgmap v563: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:31:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 738.428626 secs 2023-09-09T23:31:48.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 740.028240 secs 2023-09-09T23:31:48.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 740.028240 secs 2023-09-09T23:31:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 740.028240 secs 2023-09-09T23:31:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:49 smithi170 ceph-mon[102832]: pgmap v564: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:31:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 740.757341 secs 2023-09-09T23:31:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:49 smithi082 ceph-mon[98694]: pgmap v564: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:31:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 740.757341 secs 2023-09-09T23:31:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:49 smithi098 ceph-mon[103124]: pgmap v564: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:31:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 740.757341 secs 2023-09-09T23:31:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:51 smithi170 ceph-mon[102832]: pgmap v565: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-09-09T23:31:51.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:51 smithi082 ceph-mon[98694]: pgmap v565: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-09-09T23:31:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:51 smithi098 ceph-mon[103124]: pgmap v565: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2023-09-09T23:31:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 743.428692 secs 2023-09-09T23:31:52.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 743.428692 secs 2023-09-09T23:31:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 743.428692 secs 2023-09-09T23:31:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:53 smithi170 ceph-mon[102832]: pgmap v566: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:31:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 745.028298 secs 2023-09-09T23:31:53.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:53 smithi082 ceph-mon[98694]: pgmap v566: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:31:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 745.028298 secs 2023-09-09T23:31:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:53 smithi098 ceph-mon[103124]: pgmap v566: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-09T23:31:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 745.028298 secs 2023-09-09T23:31:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 745.757410 secs 2023-09-09T23:31:54.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 745.757410 secs 2023-09-09T23:31:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 745.757410 secs 2023-09-09T23:31:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:55 smithi170 ceph-mon[102832]: pgmap v567: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:31:55.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:55 smithi082 ceph-mon[98694]: pgmap v567: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:31:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:55 smithi098 ceph-mon[103124]: pgmap v567: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:31:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:57 smithi170 ceph-mon[102832]: pgmap v568: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-09T23:31:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 748.428752 secs 2023-09-09T23:31:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:57 smithi082 ceph-mon[98694]: pgmap v568: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-09T23:31:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 748.428752 secs 2023-09-09T23:31:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:57 smithi098 ceph-mon[103124]: pgmap v568: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-09T23:31:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 748.428752 secs 2023-09-09T23:31:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 750.028392 secs 2023-09-09T23:31:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:58 smithi170 ceph-mon[102832]: pgmap v569: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:31:58.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 750.028392 secs 2023-09-09T23:31:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:58 smithi082 ceph-mon[98694]: pgmap v569: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:31:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 750.028392 secs 2023-09-09T23:31:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:58 smithi098 ceph-mon[103124]: pgmap v569: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:31:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:31:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 750.757482 secs 2023-09-09T23:31:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:31:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 750.757482 secs 2023-09-09T23:32:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:31:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 750.757482 secs 2023-09-09T23:32:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:00 smithi170 ceph-mon[102832]: pgmap v570: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-09T23:32:00.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:00 smithi082 ceph-mon[98694]: pgmap v570: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-09T23:32:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:00 smithi098 ceph-mon[103124]: pgmap v570: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2023-09-09T23:32:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:01 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 753.428773 secs 2023-09-09T23:32:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:01 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 753.428773 secs 2023-09-09T23:32:02.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:01 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 753.428773 secs 2023-09-09T23:32:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:03 smithi098 ceph-mon[103124]: pgmap v571: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-09T23:32:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 755.028450 secs 2023-09-09T23:32:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:03 smithi170 ceph-mon[102832]: pgmap v571: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-09T23:32:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 755.028450 secs 2023-09-09T23:32:03.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:03 smithi082 ceph-mon[98694]: pgmap v571: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-09T23:32:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 755.028450 secs 2023-09-09T23:32:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 755.757546 secs 2023-09-09T23:32:04.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 755.757546 secs 2023-09-09T23:32:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 755.757546 secs 2023-09-09T23:32:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:05 smithi170 ceph-mon[102832]: pgmap v572: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:32:05.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:05 smithi082 ceph-mon[98694]: pgmap v572: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:32:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:05 smithi098 ceph-mon[103124]: pgmap v572: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:32:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:07 smithi170 ceph-mon[102832]: pgmap v573: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 27 op/s 2023-09-09T23:32:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 758.428873 secs 2023-09-09T23:32:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:07 smithi082 ceph-mon[98694]: pgmap v573: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 27 op/s 2023-09-09T23:32:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 758.428873 secs 2023-09-09T23:32:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:07 smithi098 ceph-mon[103124]: pgmap v573: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 27 op/s 2023-09-09T23:32:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 758.428873 secs 2023-09-09T23:32:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:08 smithi170 ceph-mon[102832]: osdmap e109: 12 total, 12 up, 12 in 2023-09-09T23:32:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 760.028521 secs 2023-09-09T23:32:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:08 smithi082 ceph-mon[98694]: osdmap e109: 12 total, 12 up, 12 in 2023-09-09T23:32:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 760.028521 secs 2023-09-09T23:32:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:08 smithi098 ceph-mon[103124]: osdmap e109: 12 total, 12 up, 12 in 2023-09-09T23:32:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 760.028521 secs 2023-09-09T23:32:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 760.757612 secs 2023-09-09T23:32:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:09 smithi170 ceph-mon[102832]: pgmap v575: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:32:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 760.757612 secs 2023-09-09T23:32:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:09 smithi082 ceph-mon[98694]: pgmap v575: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:32:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 760.757612 secs 2023-09-09T23:32:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:09 smithi098 ceph-mon[103124]: pgmap v575: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:32:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:10 smithi170 ceph-mon[102832]: pgmap v576: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:32:10.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:10 smithi082 ceph-mon[98694]: pgmap v576: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:32:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:10 smithi098 ceph-mon[103124]: pgmap v576: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:32:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:11 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 763.429023 secs 2023-09-09T23:32:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:11 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 763.429023 secs 2023-09-09T23:32:12.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:11 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 763.429023 secs 2023-09-09T23:32:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:13 smithi170 ceph-mon[102832]: pgmap v577: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:32:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 765.028584 secs 2023-09-09T23:32:13.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:13 smithi082 ceph-mon[98694]: pgmap v577: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:32:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 765.028584 secs 2023-09-09T23:32:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:13 smithi098 ceph-mon[103124]: pgmap v577: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:32:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 765.028584 secs 2023-09-09T23:32:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 765.757681 secs 2023-09-09T23:32:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:14 smithi170 ceph-mon[102832]: osdmap e110: 12 total, 12 up, 12 in 2023-09-09T23:32:14.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 765.757681 secs 2023-09-09T23:32:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:14 smithi082 ceph-mon[98694]: osdmap e110: 12 total, 12 up, 12 in 2023-09-09T23:32:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 765.757681 secs 2023-09-09T23:32:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:14 smithi098 ceph-mon[103124]: osdmap e110: 12 total, 12 up, 12 in 2023-09-09T23:32:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:15 smithi170 ceph-mon[102832]: pgmap v579: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:32:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:15 smithi082 ceph-mon[98694]: pgmap v579: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:32:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:15 smithi098 ceph-mon[103124]: pgmap v579: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:32:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:17 smithi170 ceph-mon[102832]: pgmap v580: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 KiB/s wr, 13 op/s 2023-09-09T23:32:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 768.429084 secs 2023-09-09T23:32:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:17 smithi082 ceph-mon[98694]: pgmap v580: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 KiB/s wr, 13 op/s 2023-09-09T23:32:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 768.429084 secs 2023-09-09T23:32:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:17 smithi098 ceph-mon[103124]: pgmap v580: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 KiB/s wr, 13 op/s 2023-09-09T23:32:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 768.429084 secs 2023-09-09T23:32:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 770.028641 secs 2023-09-09T23:32:18.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 770.028641 secs 2023-09-09T23:32:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 770.028641 secs 2023-09-09T23:32:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 770.757747 secs 2023-09-09T23:32:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:19 smithi170 ceph-mon[102832]: pgmap v581: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.6 KiB/s wr, 12 op/s 2023-09-09T23:32:19.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 770.757747 secs 2023-09-09T23:32:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:19 smithi082 ceph-mon[98694]: pgmap v581: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.6 KiB/s wr, 12 op/s 2023-09-09T23:32:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 770.757747 secs 2023-09-09T23:32:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:19 smithi098 ceph-mon[103124]: pgmap v581: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.6 KiB/s wr, 12 op/s 2023-09-09T23:32:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:21 smithi170 ceph-mon[102832]: pgmap v582: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 KiB/s wr, 13 op/s 2023-09-09T23:32:21.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:21 smithi082 ceph-mon[98694]: pgmap v582: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 KiB/s wr, 13 op/s 2023-09-09T23:32:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:21 smithi098 ceph-mon[103124]: pgmap v582: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 21 KiB/s wr, 13 op/s 2023-09-09T23:32:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 773.429145 secs 2023-09-09T23:32:22.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 773.429145 secs 2023-09-09T23:32:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 773.429145 secs 2023-09-09T23:32:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:23 smithi170 ceph-mon[102832]: pgmap v583: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 KiB/s wr, 14 op/s 2023-09-09T23:32:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 775.028698 secs 2023-09-09T23:32:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:32:23.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:23 smithi082 ceph-mon[98694]: pgmap v583: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 KiB/s wr, 14 op/s 2023-09-09T23:32:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 775.028698 secs 2023-09-09T23:32:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:32:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:23 smithi098 ceph-mon[103124]: pgmap v583: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 KiB/s wr, 14 op/s 2023-09-09T23:32:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 775.028698 secs 2023-09-09T23:32:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:32:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 775.757813 secs 2023-09-09T23:32:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:32:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:32:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:32:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:24 smithi170 ceph-mon[102832]: pgmap v584: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-09-09T23:32:24.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 775.757813 secs 2023-09-09T23:32:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:32:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:32:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:32:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:24 smithi082 ceph-mon[98694]: pgmap v584: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-09-09T23:32:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 775.757813 secs 2023-09-09T23:32:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:32:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:32:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:32:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:24 smithi098 ceph-mon[103124]: pgmap v584: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-09-09T23:32:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:27 smithi170 ceph-mon[102832]: pgmap v585: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 20 KiB/s wr, 16 op/s 2023-09-09T23:32:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 778.429266 secs 2023-09-09T23:32:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:27 smithi082 ceph-mon[98694]: pgmap v585: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 20 KiB/s wr, 16 op/s 2023-09-09T23:32:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 778.429266 secs 2023-09-09T23:32:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:27 smithi098 ceph-mon[103124]: pgmap v585: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 20 KiB/s wr, 16 op/s 2023-09-09T23:32:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 778.429266 secs 2023-09-09T23:32:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 780.028756 secs 2023-09-09T23:32:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 780.028756 secs 2023-09-09T23:32:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 780.028756 secs 2023-09-09T23:32:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 780.757877 secs 2023-09-09T23:32:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:29 smithi170 ceph-mon[102832]: pgmap v586: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 10 op/s 2023-09-09T23:32:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 780.757877 secs 2023-09-09T23:32:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:29 smithi082 ceph-mon[98694]: pgmap v586: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 10 op/s 2023-09-09T23:32:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 780.757877 secs 2023-09-09T23:32:30.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:29 smithi098 ceph-mon[103124]: pgmap v586: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 10 op/s 2023-09-09T23:32:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:31 smithi170 ceph-mon[102832]: pgmap v587: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 14 op/s 2023-09-09T23:32:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:31 smithi082 ceph-mon[98694]: pgmap v587: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 14 op/s 2023-09-09T23:32:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:31 smithi098 ceph-mon[103124]: pgmap v587: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 14 op/s 2023-09-09T23:32:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 783.429324 secs 2023-09-09T23:32:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 783.429324 secs 2023-09-09T23:32:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 783.429324 secs 2023-09-09T23:32:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:33 smithi170 ceph-mon[102832]: pgmap v588: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.2 KiB/s wr, 10 op/s 2023-09-09T23:32:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 785.028814 secs 2023-09-09T23:32:33.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:33 smithi082 ceph-mon[98694]: pgmap v588: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.2 KiB/s wr, 10 op/s 2023-09-09T23:32:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 785.028814 secs 2023-09-09T23:32:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:33 smithi098 ceph-mon[103124]: pgmap v588: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.2 KiB/s wr, 10 op/s 2023-09-09T23:32:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 785.028814 secs 2023-09-09T23:32:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 785.757941 secs 2023-09-09T23:32:34.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 785.757941 secs 2023-09-09T23:32:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 785.757941 secs 2023-09-09T23:32:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:35 smithi170 ceph-mon[102832]: pgmap v589: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2023-09-09T23:32:35.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:35 smithi082 ceph-mon[98694]: pgmap v589: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2023-09-09T23:32:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:35 smithi098 ceph-mon[103124]: pgmap v589: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2023-09-09T23:32:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:37 smithi170 ceph-mon[102832]: pgmap v590: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2023-09-09T23:32:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 788.429384 secs 2023-09-09T23:32:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:37 smithi082 ceph-mon[98694]: pgmap v590: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2023-09-09T23:32:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 788.429384 secs 2023-09-09T23:32:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:37 smithi098 ceph-mon[103124]: pgmap v590: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2023-09-09T23:32:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 788.429384 secs 2023-09-09T23:32:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 790.028872 secs 2023-09-09T23:32:38.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 790.028872 secs 2023-09-09T23:32:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 790.028872 secs 2023-09-09T23:32:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 790.758005 secs 2023-09-09T23:32:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:39 smithi170 ceph-mon[102832]: pgmap v591: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2023-09-09T23:32:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 790.758005 secs 2023-09-09T23:32:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:39 smithi082 ceph-mon[98694]: pgmap v591: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2023-09-09T23:32:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 790.758005 secs 2023-09-09T23:32:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:39 smithi098 ceph-mon[103124]: pgmap v591: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2023-09-09T23:32:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:40 smithi170 ceph-mon[102832]: pgmap v592: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2023-09-09T23:32:40.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:40 smithi082 ceph-mon[98694]: pgmap v592: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2023-09-09T23:32:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:40 smithi098 ceph-mon[103124]: pgmap v592: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2023-09-09T23:32:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 793.429444 secs 2023-09-09T23:32:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 793.429444 secs 2023-09-09T23:32:42.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 793.429444 secs 2023-09-09T23:32:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:43 smithi170 ceph-mon[102832]: pgmap v593: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2023-09-09T23:32:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 795.028934 secs 2023-09-09T23:32:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:43 smithi082 ceph-mon[98694]: pgmap v593: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2023-09-09T23:32:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 795.028934 secs 2023-09-09T23:32:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:43 smithi098 ceph-mon[103124]: pgmap v593: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2023-09-09T23:32:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 795.028934 secs 2023-09-09T23:32:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 795.758069 secs 2023-09-09T23:32:44.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 795.758069 secs 2023-09-09T23:32:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 795.758069 secs 2023-09-09T23:32:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:45 smithi170 ceph-mon[102832]: pgmap v594: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2023-09-09T23:32:45.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:45 smithi082 ceph-mon[98694]: pgmap v594: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2023-09-09T23:32:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:45 smithi098 ceph-mon[103124]: pgmap v594: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2023-09-09T23:32:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:47 smithi170 ceph-mon[102832]: pgmap v595: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 13 op/s 2023-09-09T23:32:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 798.429504 secs 2023-09-09T23:32:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:47 smithi082 ceph-mon[98694]: pgmap v595: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 13 op/s 2023-09-09T23:32:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 798.429504 secs 2023-09-09T23:32:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:47 smithi098 ceph-mon[103124]: pgmap v595: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 13 op/s 2023-09-09T23:32:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 798.429504 secs 2023-09-09T23:32:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 800.028991 secs 2023-09-09T23:32:48.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 800.028991 secs 2023-09-09T23:32:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 800.028991 secs 2023-09-09T23:32:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 800.758133 secs 2023-09-09T23:32:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:49 smithi170 ceph-mon[102832]: pgmap v596: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-09T23:32:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 800.758133 secs 2023-09-09T23:32:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:49 smithi082 ceph-mon[98694]: pgmap v596: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-09T23:32:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 800.758133 secs 2023-09-09T23:32:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:49 smithi098 ceph-mon[103124]: pgmap v596: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-09T23:32:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:51 smithi170 ceph-mon[102832]: pgmap v597: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2023-09-09T23:32:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:51 smithi082 ceph-mon[98694]: pgmap v597: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2023-09-09T23:32:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:51 smithi098 ceph-mon[103124]: pgmap v597: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2023-09-09T23:32:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 803.429562 secs 2023-09-09T23:32:52.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 803.429562 secs 2023-09-09T23:32:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 803.429562 secs 2023-09-09T23:32:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:53 smithi170 ceph-mon[102832]: pgmap v598: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-09-09T23:32:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 805.029092 secs 2023-09-09T23:32:53.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:53 smithi082 ceph-mon[98694]: pgmap v598: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-09-09T23:32:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 805.029092 secs 2023-09-09T23:32:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:53 smithi098 ceph-mon[103124]: pgmap v598: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-09-09T23:32:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 805.029092 secs 2023-09-09T23:32:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 805.758203 secs 2023-09-09T23:32:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:54 smithi170 ceph-mon[102832]: pgmap v599: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-09T23:32:54.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 805.758203 secs 2023-09-09T23:32:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:54 smithi082 ceph-mon[98694]: pgmap v599: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-09T23:32:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 805.758203 secs 2023-09-09T23:32:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:54 smithi098 ceph-mon[103124]: pgmap v599: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-09T23:32:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:57 smithi170 ceph-mon[102832]: pgmap v600: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 13 op/s 2023-09-09T23:32:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 808.429622 secs 2023-09-09T23:32:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:57 smithi082 ceph-mon[98694]: pgmap v600: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 13 op/s 2023-09-09T23:32:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 808.429622 secs 2023-09-09T23:32:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:57 smithi098 ceph-mon[103124]: pgmap v600: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 13 op/s 2023-09-09T23:32:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 808.429622 secs 2023-09-09T23:32:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 810.029153 secs 2023-09-09T23:32:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 810.029153 secs 2023-09-09T23:32:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 810.029153 secs 2023-09-09T23:32:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 810.758264 secs 2023-09-09T23:32:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:32:59 smithi170 ceph-mon[102832]: pgmap v601: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-09T23:32:59.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 810.758264 secs 2023-09-09T23:32:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:32:59 smithi082 ceph-mon[98694]: pgmap v601: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-09T23:33:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 810.758264 secs 2023-09-09T23:33:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:32:59 smithi098 ceph-mon[103124]: pgmap v601: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-09T23:33:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:01 smithi170 ceph-mon[102832]: pgmap v602: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-09T23:33:01.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:01 smithi082 ceph-mon[98694]: pgmap v602: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-09T23:33:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:01 smithi098 ceph-mon[103124]: pgmap v602: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-09T23:33:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 813.429738 secs 2023-09-09T23:33:02.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 813.429738 secs 2023-09-09T23:33:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 813.429738 secs 2023-09-09T23:33:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:03 smithi170 ceph-mon[102832]: pgmap v603: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-09T23:33:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 815.029230 secs 2023-09-09T23:33:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:03 smithi082 ceph-mon[98694]: pgmap v603: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-09T23:33:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 815.029230 secs 2023-09-09T23:33:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:03 smithi098 ceph-mon[103124]: pgmap v603: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-09T23:33:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 815.029230 secs 2023-09-09T23:33:04.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 815.758337 secs 2023-09-09T23:33:04.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 815.758337 secs 2023-09-09T23:33:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 815.758337 secs 2023-09-09T23:33:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:05 smithi170 ceph-mon[102832]: pgmap v604: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-09T23:33:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:05 smithi082 ceph-mon[98694]: pgmap v604: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-09T23:33:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:05 smithi098 ceph-mon[103124]: pgmap v604: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-09T23:33:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:06 smithi170 ceph-mon[102832]: pgmap v605: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 28 op/s 2023-09-09T23:33:06.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:06 smithi082 ceph-mon[98694]: pgmap v605: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 28 op/s 2023-09-09T23:33:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:06 smithi098 ceph-mon[103124]: pgmap v605: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 28 op/s 2023-09-09T23:33:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 818.429780 secs 2023-09-09T23:33:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 818.429780 secs 2023-09-09T23:33:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 818.429780 secs 2023-09-09T23:33:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:08 smithi170 ceph-mon[102832]: osdmap e111: 12 total, 12 up, 12 in 2023-09-09T23:33:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 820.029294 secs 2023-09-09T23:33:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:08 smithi170 ceph-mon[102832]: pgmap v607: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:33:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:08 smithi082 ceph-mon[98694]: osdmap e111: 12 total, 12 up, 12 in 2023-09-09T23:33:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 820.029294 secs 2023-09-09T23:33:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:08 smithi082 ceph-mon[98694]: pgmap v607: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:33:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:08 smithi098 ceph-mon[103124]: osdmap e111: 12 total, 12 up, 12 in 2023-09-09T23:33:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 820.029294 secs 2023-09-09T23:33:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:08 smithi098 ceph-mon[103124]: pgmap v607: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:33:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 820.758398 secs 2023-09-09T23:33:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 820.758398 secs 2023-09-09T23:33:10.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 820.758398 secs 2023-09-09T23:33:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:10 smithi170 ceph-mon[102832]: pgmap v608: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:33:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:10 smithi082 ceph-mon[98694]: pgmap v608: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:33:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:10 smithi098 ceph-mon[103124]: pgmap v608: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:33:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 823.429864 secs 2023-09-09T23:33:12.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 823.429864 secs 2023-09-09T23:33:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 823.429864 secs 2023-09-09T23:33:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:13 smithi170 ceph-mon[102832]: pgmap v609: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:33:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 825.029352 secs 2023-09-09T23:33:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:13 smithi082 ceph-mon[98694]: pgmap v609: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:33:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 825.029352 secs 2023-09-09T23:33:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:13 smithi098 ceph-mon[103124]: pgmap v609: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:33:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 825.029352 secs 2023-09-09T23:33:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 825.758463 secs 2023-09-09T23:33:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:14 smithi170 ceph-mon[102832]: osdmap e112: 12 total, 12 up, 12 in 2023-09-09T23:33:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 825.758463 secs 2023-09-09T23:33:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:14 smithi082 ceph-mon[98694]: osdmap e112: 12 total, 12 up, 12 in 2023-09-09T23:33:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 825.758463 secs 2023-09-09T23:33:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:14 smithi098 ceph-mon[103124]: osdmap e112: 12 total, 12 up, 12 in 2023-09-09T23:33:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:15 smithi170 ceph-mon[102832]: pgmap v611: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:33:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:15 smithi082 ceph-mon[98694]: pgmap v611: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:33:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:15 smithi098 ceph-mon[103124]: pgmap v611: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:33:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:17 smithi170 ceph-mon[102832]: pgmap v612: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 KiB/s wr, 12 op/s 2023-09-09T23:33:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 828.429924 secs 2023-09-09T23:33:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:17 smithi082 ceph-mon[98694]: pgmap v612: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 KiB/s wr, 12 op/s 2023-09-09T23:33:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 828.429924 secs 2023-09-09T23:33:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:17 smithi098 ceph-mon[103124]: pgmap v612: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 KiB/s wr, 12 op/s 2023-09-09T23:33:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 828.429924 secs 2023-09-09T23:33:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 830.029444 secs 2023-09-09T23:33:18.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 830.029444 secs 2023-09-09T23:33:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 830.029444 secs 2023-09-09T23:33:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 830.758489 secs 2023-09-09T23:33:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:19 smithi170 ceph-mon[102832]: pgmap v613: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.7 KiB/s wr, 11 op/s 2023-09-09T23:33:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 830.758489 secs 2023-09-09T23:33:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:19 smithi082 ceph-mon[98694]: pgmap v613: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.7 KiB/s wr, 11 op/s 2023-09-09T23:33:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 830.758489 secs 2023-09-09T23:33:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:19 smithi098 ceph-mon[103124]: pgmap v613: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.7 KiB/s wr, 11 op/s 2023-09-09T23:33:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:20 smithi170 ceph-mon[102832]: pgmap v614: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2023-09-09T23:33:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:20 smithi082 ceph-mon[98694]: pgmap v614: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2023-09-09T23:33:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:20 smithi098 ceph-mon[103124]: pgmap v614: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2023-09-09T23:33:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 833.429987 secs 2023-09-09T23:33:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 833.429987 secs 2023-09-09T23:33:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 833.429987 secs 2023-09-09T23:33:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:23 smithi170 ceph-mon[102832]: pgmap v615: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 12 op/s 2023-09-09T23:33:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 835.029503 secs 2023-09-09T23:33:23.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:23 smithi082 ceph-mon[98694]: pgmap v615: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 12 op/s 2023-09-09T23:33:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 835.029503 secs 2023-09-09T23:33:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:23 smithi098 ceph-mon[103124]: pgmap v615: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 12 op/s 2023-09-09T23:33:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 835.029503 secs 2023-09-09T23:33:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 835.758555 secs 2023-09-09T23:33:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:33:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:33:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:33:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:33:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 835.758555 secs 2023-09-09T23:33:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:33:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:33:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:33:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:33:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 835.758555 secs 2023-09-09T23:33:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:33:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:33:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:33:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:33:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:25 smithi170 ceph-mon[102832]: pgmap v616: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 11 op/s 2023-09-09T23:33:25.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:25 smithi082 ceph-mon[98694]: pgmap v616: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 11 op/s 2023-09-09T23:33:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:25 smithi098 ceph-mon[103124]: pgmap v616: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 11 op/s 2023-09-09T23:33:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:27 smithi170 ceph-mon[102832]: pgmap v617: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-09-09T23:33:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 838.430115 secs 2023-09-09T23:33:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:27 smithi082 ceph-mon[98694]: pgmap v617: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-09-09T23:33:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 838.430115 secs 2023-09-09T23:33:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:27 smithi098 ceph-mon[103124]: pgmap v617: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-09-09T23:33:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 838.430115 secs 2023-09-09T23:33:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 840.029586 secs 2023-09-09T23:33:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 840.029586 secs 2023-09-09T23:33:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 840.029586 secs 2023-09-09T23:33:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 840.758621 secs 2023-09-09T23:33:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:29 smithi170 ceph-mon[102832]: pgmap v618: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 9 op/s 2023-09-09T23:33:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 840.758621 secs 2023-09-09T23:33:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:29 smithi082 ceph-mon[98694]: pgmap v618: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 9 op/s 2023-09-09T23:33:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 840.758621 secs 2023-09-09T23:33:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:29 smithi098 ceph-mon[103124]: pgmap v618: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 9 op/s 2023-09-09T23:33:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:31 smithi170 ceph-mon[102832]: pgmap v619: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 12 op/s 2023-09-09T23:33:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:31 smithi082 ceph-mon[98694]: pgmap v619: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 12 op/s 2023-09-09T23:33:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:31 smithi098 ceph-mon[103124]: pgmap v619: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 12 op/s 2023-09-09T23:33:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 843.430199 secs 2023-09-09T23:33:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 843.430199 secs 2023-09-09T23:33:33.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 843.430199 secs 2023-09-09T23:33:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:33 smithi170 ceph-mon[102832]: pgmap v620: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-09T23:33:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 845.029665 secs 2023-09-09T23:33:33.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:33 smithi082 ceph-mon[98694]: pgmap v620: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-09T23:33:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 845.029665 secs 2023-09-09T23:33:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:33 smithi098 ceph-mon[103124]: pgmap v620: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-09T23:33:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 845.029665 secs 2023-09-09T23:33:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 845.758687 secs 2023-09-09T23:33:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:34 smithi170 ceph-mon[102832]: pgmap v621: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.9 KiB/s wr, 8 op/s 2023-09-09T23:33:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 845.758687 secs 2023-09-09T23:33:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:34 smithi082 ceph-mon[98694]: pgmap v621: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.9 KiB/s wr, 8 op/s 2023-09-09T23:33:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 845.758687 secs 2023-09-09T23:33:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:34 smithi098 ceph-mon[103124]: pgmap v621: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.9 KiB/s wr, 8 op/s 2023-09-09T23:33:37.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:37 smithi170 ceph-mon[102832]: pgmap v622: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.9 KiB/s wr, 12 op/s 2023-09-09T23:33:37.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 848.430257 secs 2023-09-09T23:33:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:37 smithi082 ceph-mon[98694]: pgmap v622: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.9 KiB/s wr, 12 op/s 2023-09-09T23:33:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 848.430257 secs 2023-09-09T23:33:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:37 smithi098 ceph-mon[103124]: pgmap v622: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.9 KiB/s wr, 12 op/s 2023-09-09T23:33:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 848.430257 secs 2023-09-09T23:33:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 850.029722 secs 2023-09-09T23:33:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 850.029722 secs 2023-09-09T23:33:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 850.029722 secs 2023-09-09T23:33:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 850.758751 secs 2023-09-09T23:33:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:39 smithi170 ceph-mon[102832]: pgmap v623: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2023-09-09T23:33:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 850.758751 secs 2023-09-09T23:33:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:39 smithi082 ceph-mon[98694]: pgmap v623: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2023-09-09T23:33:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 850.758751 secs 2023-09-09T23:33:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:39 smithi098 ceph-mon[103124]: pgmap v623: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2023-09-09T23:33:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:41 smithi170 ceph-mon[102832]: pgmap v624: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2023-09-09T23:33:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:41 smithi082 ceph-mon[98694]: pgmap v624: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2023-09-09T23:33:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:41 smithi098 ceph-mon[103124]: pgmap v624: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2023-09-09T23:33:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 853.430358 secs 2023-09-09T23:33:42.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 853.430358 secs 2023-09-09T23:33:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 853.430358 secs 2023-09-09T23:33:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:43 smithi170 ceph-mon[102832]: pgmap v625: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2023-09-09T23:33:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 855.029779 secs 2023-09-09T23:33:43.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:43 smithi082 ceph-mon[98694]: pgmap v625: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2023-09-09T23:33:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 855.029779 secs 2023-09-09T23:33:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:43 smithi098 ceph-mon[103124]: pgmap v625: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2023-09-09T23:33:44.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 855.029779 secs 2023-09-09T23:33:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 855.758816 secs 2023-09-09T23:33:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 855.758816 secs 2023-09-09T23:33:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 855.758816 secs 2023-09-09T23:33:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:45 smithi170 ceph-mon[102832]: pgmap v626: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:33:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:45 smithi082 ceph-mon[98694]: pgmap v626: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:33:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:45 smithi098 ceph-mon[103124]: pgmap v626: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:33:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:46 smithi170 ceph-mon[102832]: pgmap v627: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2023-09-09T23:33:46.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:46 smithi082 ceph-mon[98694]: pgmap v627: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2023-09-09T23:33:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:46 smithi098 ceph-mon[103124]: pgmap v627: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2023-09-09T23:33:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 858.430426 secs 2023-09-09T23:33:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 858.430426 secs 2023-09-09T23:33:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 858.430426 secs 2023-09-09T23:33:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 860.029836 secs 2023-09-09T23:33:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:48 smithi170 ceph-mon[102832]: pgmap v628: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2023-09-09T23:33:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 860.029836 secs 2023-09-09T23:33:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:48 smithi082 ceph-mon[98694]: pgmap v628: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2023-09-09T23:33:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 860.029836 secs 2023-09-09T23:33:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:48 smithi098 ceph-mon[103124]: pgmap v628: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2023-09-09T23:33:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 860.758880 secs 2023-09-09T23:33:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 860.758880 secs 2023-09-09T23:33:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 860.758880 secs 2023-09-09T23:33:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:50 smithi170 ceph-mon[102832]: pgmap v629: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2023-09-09T23:33:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:50 smithi082 ceph-mon[98694]: pgmap v629: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2023-09-09T23:33:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:50 smithi098 ceph-mon[103124]: pgmap v629: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2023-09-09T23:33:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 863.430553 secs 2023-09-09T23:33:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 863.430553 secs 2023-09-09T23:33:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 863.430553 secs 2023-09-09T23:33:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:53 smithi170 ceph-mon[102832]: pgmap v630: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-09-09T23:33:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 865.029894 secs 2023-09-09T23:33:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:53 smithi082 ceph-mon[98694]: pgmap v630: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-09-09T23:33:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 865.029894 secs 2023-09-09T23:33:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:53 smithi098 ceph-mon[103124]: pgmap v630: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-09-09T23:33:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 865.029894 secs 2023-09-09T23:33:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 865.758945 secs 2023-09-09T23:33:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 865.758945 secs 2023-09-09T23:33:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 865.758945 secs 2023-09-09T23:33:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:55 smithi170 ceph-mon[102832]: pgmap v631: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2023-09-09T23:33:55.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:55 smithi082 ceph-mon[98694]: pgmap v631: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2023-09-09T23:33:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:55 smithi098 ceph-mon[103124]: pgmap v631: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2023-09-09T23:33:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:57 smithi170 ceph-mon[102832]: pgmap v632: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2023-09-09T23:33:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 868.430612 secs 2023-09-09T23:33:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:57 smithi082 ceph-mon[98694]: pgmap v632: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2023-09-09T23:33:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 868.430612 secs 2023-09-09T23:33:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:57 smithi098 ceph-mon[103124]: pgmap v632: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 14 op/s 2023-09-09T23:33:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 868.430612 secs 2023-09-09T23:33:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 870.029978 secs 2023-09-09T23:33:58.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 870.029978 secs 2023-09-09T23:33:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 870.029978 secs 2023-09-09T23:33:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 870.759009 secs 2023-09-09T23:33:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:33:59 smithi170 ceph-mon[102832]: pgmap v633: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.7 KiB/s wr, 9 op/s 2023-09-09T23:33:59.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 870.759009 secs 2023-09-09T23:33:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:33:59 smithi082 ceph-mon[98694]: pgmap v633: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.7 KiB/s wr, 9 op/s 2023-09-09T23:34:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 870.759009 secs 2023-09-09T23:34:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:33:59 smithi098 ceph-mon[103124]: pgmap v633: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.7 KiB/s wr, 9 op/s 2023-09-09T23:34:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:00 smithi170 ceph-mon[102832]: pgmap v634: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 KiB/s wr, 12 op/s 2023-09-09T23:34:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:00 smithi082 ceph-mon[98694]: pgmap v634: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 KiB/s wr, 12 op/s 2023-09-09T23:34:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:00 smithi098 ceph-mon[103124]: pgmap v634: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 KiB/s wr, 12 op/s 2023-09-09T23:34:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 873.430674 secs 2023-09-09T23:34:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 873.430674 secs 2023-09-09T23:34:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 873.430674 secs 2023-09-09T23:34:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:03 smithi170 ceph-mon[102832]: pgmap v635: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2023-09-09T23:34:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 875.030038 secs 2023-09-09T23:34:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:03 smithi082 ceph-mon[98694]: pgmap v635: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2023-09-09T23:34:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 875.030038 secs 2023-09-09T23:34:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:03 smithi098 ceph-mon[103124]: pgmap v635: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2023-09-09T23:34:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 875.030038 secs 2023-09-09T23:34:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 875.759075 secs 2023-09-09T23:34:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 875.759075 secs 2023-09-09T23:34:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 875.759075 secs 2023-09-09T23:34:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:05 smithi170 ceph-mon[102832]: pgmap v636: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-09T23:34:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:05 smithi082 ceph-mon[98694]: pgmap v636: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-09T23:34:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:05 smithi098 ceph-mon[103124]: pgmap v636: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-09T23:34:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:07 smithi170 ceph-mon[102832]: pgmap v637: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2023-09-09T23:34:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 878.430733 secs 2023-09-09T23:34:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:07 smithi082 ceph-mon[98694]: pgmap v637: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2023-09-09T23:34:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 878.430733 secs 2023-09-09T23:34:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:07 smithi098 ceph-mon[103124]: pgmap v637: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2023-09-09T23:34:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 878.430733 secs 2023-09-09T23:34:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:08 smithi170 ceph-mon[102832]: osdmap e113: 12 total, 12 up, 12 in 2023-09-09T23:34:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 880.030100 secs 2023-09-09T23:34:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:08 smithi170 ceph-mon[102832]: pgmap v639: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:34:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:08 smithi082 ceph-mon[98694]: osdmap e113: 12 total, 12 up, 12 in 2023-09-09T23:34:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 880.030100 secs 2023-09-09T23:34:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:08 smithi082 ceph-mon[98694]: pgmap v639: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:34:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:08 smithi098 ceph-mon[103124]: osdmap e113: 12 total, 12 up, 12 in 2023-09-09T23:34:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 880.030100 secs 2023-09-09T23:34:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:08 smithi098 ceph-mon[103124]: pgmap v639: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:34:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 880.759141 secs 2023-09-09T23:34:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 880.759141 secs 2023-09-09T23:34:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 880.759141 secs 2023-09-09T23:34:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:10 smithi170 ceph-mon[102832]: pgmap v640: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:34:10.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:10 smithi082 ceph-mon[98694]: pgmap v640: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:34:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:10 smithi098 ceph-mon[103124]: pgmap v640: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:34:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 883.430776 secs 2023-09-09T23:34:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 883.430776 secs 2023-09-09T23:34:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 883.430776 secs 2023-09-09T23:34:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:13 smithi170 ceph-mon[102832]: pgmap v641: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:34:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 885.030159 secs 2023-09-09T23:34:13.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:13 smithi082 ceph-mon[98694]: pgmap v641: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:34:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 885.030159 secs 2023-09-09T23:34:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:13 smithi098 ceph-mon[103124]: pgmap v641: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:34:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 885.030159 secs 2023-09-09T23:34:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 885.759206 secs 2023-09-09T23:34:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:14 smithi170 ceph-mon[102832]: osdmap e114: 12 total, 12 up, 12 in 2023-09-09T23:34:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 885.759206 secs 2023-09-09T23:34:14.986 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:14 smithi082 ceph-mon[98694]: osdmap e114: 12 total, 12 up, 12 in 2023-09-09T23:34:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 885.759206 secs 2023-09-09T23:34:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:14 smithi098 ceph-mon[103124]: osdmap e114: 12 total, 12 up, 12 in 2023-09-09T23:34:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:15 smithi170 ceph-mon[102832]: pgmap v643: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-09T23:34:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:15 smithi082 ceph-mon[98694]: pgmap v643: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-09T23:34:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:15 smithi098 ceph-mon[103124]: pgmap v643: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-09T23:34:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:17 smithi170 ceph-mon[102832]: pgmap v644: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 KiB/s wr, 12 op/s 2023-09-09T23:34:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 888.430836 secs 2023-09-09T23:34:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:17 smithi082 ceph-mon[98694]: pgmap v644: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 KiB/s wr, 12 op/s 2023-09-09T23:34:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 888.430836 secs 2023-09-09T23:34:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:17 smithi098 ceph-mon[103124]: pgmap v644: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 KiB/s wr, 12 op/s 2023-09-09T23:34:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 888.430836 secs 2023-09-09T23:34:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 890.030232 secs 2023-09-09T23:34:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:18 smithi170 ceph-mon[102832]: pgmap v645: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2023-09-09T23:34:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 890.030232 secs 2023-09-09T23:34:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:18 smithi082 ceph-mon[98694]: pgmap v645: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2023-09-09T23:34:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 890.030232 secs 2023-09-09T23:34:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:18 smithi098 ceph-mon[103124]: pgmap v645: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2023-09-09T23:34:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 890.759275 secs 2023-09-09T23:34:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 890.759275 secs 2023-09-09T23:34:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 890.759275 secs 2023-09-09T23:34:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:20 smithi170 ceph-mon[102832]: pgmap v646: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2023-09-09T23:34:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:20 smithi082 ceph-mon[98694]: pgmap v646: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2023-09-09T23:34:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:20 smithi098 ceph-mon[103124]: pgmap v646: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2023-09-09T23:34:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 893.430896 secs 2023-09-09T23:34:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 893.430896 secs 2023-09-09T23:34:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 893.430896 secs 2023-09-09T23:34:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:23 smithi170 ceph-mon[102832]: pgmap v647: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-09-09T23:34:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 895.030320 secs 2023-09-09T23:34:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:23 smithi082 ceph-mon[98694]: pgmap v647: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-09-09T23:34:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 895.030320 secs 2023-09-09T23:34:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:23 smithi098 ceph-mon[103124]: pgmap v647: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-09-09T23:34:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 895.030320 secs 2023-09-09T23:34:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 895.759339 secs 2023-09-09T23:34:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:34:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 895.759339 secs 2023-09-09T23:34:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:34:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 895.759339 secs 2023-09-09T23:34:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:34:25.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:25 smithi170 ceph-mon[102832]: pgmap v648: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 11 op/s 2023-09-09T23:34:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:25 smithi082 ceph-mon[98694]: pgmap v648: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 11 op/s 2023-09-09T23:34:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:25 smithi098 ceph-mon[103124]: pgmap v648: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 11 op/s 2023-09-09T23:34:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:27 smithi170 ceph-mon[102832]: pgmap v649: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 19 KiB/s wr, 14 op/s 2023-09-09T23:34:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 898.430958 secs 2023-09-09T23:34:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:34:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:34:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:34:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:34:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:27 smithi082 ceph-mon[98694]: pgmap v649: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 19 KiB/s wr, 14 op/s 2023-09-09T23:34:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 898.430958 secs 2023-09-09T23:34:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:34:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:34:27.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:34:27.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:34:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:27 smithi098 ceph-mon[103124]: pgmap v649: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 19 KiB/s wr, 14 op/s 2023-09-09T23:34:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 898.430958 secs 2023-09-09T23:34:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:34:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:34:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:34:28.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:34:28.630 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:28.631 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:28.631 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:28.631 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:28.632 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:28.632 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:28.632 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:28.633 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:28.633 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 900.030380 secs 2023-09-09T23:34:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:34:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:34:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:28.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 900.030380 secs 2023-09-09T23:34:28.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:34:28.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:34:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:29.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:29.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:29.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:29.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:29.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:29.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 900.030380 secs 2023-09-09T23:34:29.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:34:29.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:34:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:29 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-09T23:34:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:29 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-09T23:34:29.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:29 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-09T23:34:29.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:29 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-09T23:34:29.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 900.759403 secs 2023-09-09T23:34:29.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:29 smithi170 ceph-mon[102832]: pgmap v650: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 9 op/s 2023-09-09T23:34:29.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:29.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:29.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:29.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:29.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:34:29.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:34:29.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:34:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:29 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-09T23:34:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:29 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-09T23:34:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:29 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-09T23:34:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:29 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-09T23:34:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 900.759403 secs 2023-09-09T23:34:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:29 smithi082 ceph-mon[98694]: pgmap v650: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 9 op/s 2023-09-09T23:34:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:29.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:29.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:34:29.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:34:29.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:34:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:29 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-09T23:34:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:29 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-09T23:34:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:29 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-09T23:34:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:29 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-09T23:34:30.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 900.759403 secs 2023-09-09T23:34:30.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:29 smithi098 ceph-mon[103124]: pgmap v650: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 9 op/s 2023-09-09T23:34:30.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:30.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:30.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:30.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:34:30.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:34:30.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:34:30.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:34:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:30 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-09T23:34:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:30 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-09T23:34:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:30 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-09T23:34:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:30 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-09T23:34:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:30 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-09T23:34:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:30 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-09T23:34:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:31 smithi170 ceph-mon[102832]: pgmap v651: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 12 op/s 2023-09-09T23:34:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:31 smithi082 ceph-mon[98694]: pgmap v651: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 12 op/s 2023-09-09T23:34:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:31 smithi098 ceph-mon[103124]: pgmap v651: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 12 op/s 2023-09-09T23:34:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 903.431020 secs 2023-09-09T23:34:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:32 smithi170 ceph-mon[102832]: pgmap v652: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-09T23:34:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 903.431020 secs 2023-09-09T23:34:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:32 smithi082 ceph-mon[98694]: pgmap v652: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-09T23:34:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 903.431020 secs 2023-09-09T23:34:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:32 smithi098 ceph-mon[103124]: pgmap v652: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-09T23:34:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 905.030466 secs 2023-09-09T23:34:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 905.030466 secs 2023-09-09T23:34:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 905.030466 secs 2023-09-09T23:34:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 905.759478 secs 2023-09-09T23:34:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:34 smithi170 ceph-mon[102832]: pgmap v653: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2023-09-09T23:34:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 905.759478 secs 2023-09-09T23:34:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:34 smithi082 ceph-mon[98694]: pgmap v653: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2023-09-09T23:34:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 905.759478 secs 2023-09-09T23:34:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:34 smithi098 ceph-mon[103124]: pgmap v653: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2023-09-09T23:34:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:37 smithi170 ceph-mon[102832]: pgmap v654: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2023-09-09T23:34:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 908.431104 secs 2023-09-09T23:34:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:37 smithi082 ceph-mon[98694]: pgmap v654: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2023-09-09T23:34:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 908.431104 secs 2023-09-09T23:34:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:37 smithi098 ceph-mon[103124]: pgmap v654: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2023-09-09T23:34:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 908.431104 secs 2023-09-09T23:34:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 910.030518 secs 2023-09-09T23:34:38.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 910.030518 secs 2023-09-09T23:34:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 910.030518 secs 2023-09-09T23:34:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 910.759486 secs 2023-09-09T23:34:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:39 smithi170 ceph-mon[102832]: pgmap v655: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2023-09-09T23:34:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 910.759486 secs 2023-09-09T23:34:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:39 smithi082 ceph-mon[98694]: pgmap v655: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2023-09-09T23:34:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 910.759486 secs 2023-09-09T23:34:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:39 smithi098 ceph-mon[103124]: pgmap v655: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2023-09-09T23:34:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:41 smithi170 ceph-mon[102832]: pgmap v656: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.0 KiB/s wr, 11 op/s 2023-09-09T23:34:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:41 smithi082 ceph-mon[98694]: pgmap v656: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.0 KiB/s wr, 11 op/s 2023-09-09T23:34:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:41 smithi098 ceph-mon[103124]: pgmap v656: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.0 KiB/s wr, 11 op/s 2023-09-09T23:34:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 913.431178 secs 2023-09-09T23:34:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 913.431178 secs 2023-09-09T23:34:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 913.431178 secs 2023-09-09T23:34:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:43 smithi170 ceph-mon[102832]: pgmap v657: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.9 KiB/s wr, 9 op/s 2023-09-09T23:34:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 915.030577 secs 2023-09-09T23:34:43.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:43 smithi082 ceph-mon[98694]: pgmap v657: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.9 KiB/s wr, 9 op/s 2023-09-09T23:34:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 915.030577 secs 2023-09-09T23:34:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:43 smithi098 ceph-mon[103124]: pgmap v657: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.9 KiB/s wr, 9 op/s 2023-09-09T23:34:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 915.030577 secs 2023-09-09T23:34:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 915.759554 secs 2023-09-09T23:34:44.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 915.759554 secs 2023-09-09T23:34:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 915.759554 secs 2023-09-09T23:34:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:45 smithi170 ceph-mon[102832]: pgmap v658: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:34:45.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:45 smithi082 ceph-mon[98694]: pgmap v658: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:34:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:45 smithi098 ceph-mon[103124]: pgmap v658: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:34:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:46 smithi170 ceph-mon[102832]: pgmap v659: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2023-09-09T23:34:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:46 smithi082 ceph-mon[98694]: pgmap v659: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2023-09-09T23:34:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:46 smithi098 ceph-mon[103124]: pgmap v659: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2023-09-09T23:34:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 918.431241 secs 2023-09-09T23:34:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 918.431241 secs 2023-09-09T23:34:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 918.431241 secs 2023-09-09T23:34:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 920.030637 secs 2023-09-09T23:34:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:48 smithi170 ceph-mon[102832]: pgmap v660: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2023-09-09T23:34:48.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 920.030637 secs 2023-09-09T23:34:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:48 smithi082 ceph-mon[98694]: pgmap v660: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2023-09-09T23:34:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 920.030637 secs 2023-09-09T23:34:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:48 smithi098 ceph-mon[103124]: pgmap v660: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2023-09-09T23:34:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 920.759619 secs 2023-09-09T23:34:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 920.759619 secs 2023-09-09T23:34:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 920.759619 secs 2023-09-09T23:34:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:50 smithi170 ceph-mon[102832]: pgmap v661: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.3 KiB/s wr, 12 op/s 2023-09-09T23:34:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:50 smithi082 ceph-mon[98694]: pgmap v661: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.3 KiB/s wr, 12 op/s 2023-09-09T23:34:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:50 smithi098 ceph-mon[103124]: pgmap v661: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.3 KiB/s wr, 12 op/s 2023-09-09T23:34:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 923.431312 secs 2023-09-09T23:34:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 923.431312 secs 2023-09-09T23:34:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 923.431312 secs 2023-09-09T23:34:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:53 smithi170 ceph-mon[102832]: pgmap v662: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-09-09T23:34:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 925.030696 secs 2023-09-09T23:34:53.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:53 smithi082 ceph-mon[98694]: pgmap v662: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-09-09T23:34:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 925.030696 secs 2023-09-09T23:34:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:53 smithi098 ceph-mon[103124]: pgmap v662: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-09-09T23:34:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 925.030696 secs 2023-09-09T23:34:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 925.759685 secs 2023-09-09T23:34:54.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 925.759685 secs 2023-09-09T23:34:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 925.759685 secs 2023-09-09T23:34:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:55 smithi170 ceph-mon[102832]: pgmap v663: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2023-09-09T23:34:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:55 smithi082 ceph-mon[98694]: pgmap v663: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2023-09-09T23:34:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:55 smithi098 ceph-mon[103124]: pgmap v663: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2023-09-09T23:34:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:57 smithi170 ceph-mon[102832]: pgmap v664: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 14 op/s 2023-09-09T23:34:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 928.431372 secs 2023-09-09T23:34:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:57 smithi082 ceph-mon[98694]: pgmap v664: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 14 op/s 2023-09-09T23:34:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 928.431372 secs 2023-09-09T23:34:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:57 smithi098 ceph-mon[103124]: pgmap v664: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 14 op/s 2023-09-09T23:34:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 928.431372 secs 2023-09-09T23:34:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 930.030752 secs 2023-09-09T23:34:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 930.030752 secs 2023-09-09T23:34:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 930.030752 secs 2023-09-09T23:34:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 930.759750 secs 2023-09-09T23:34:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:34:59 smithi170 ceph-mon[102832]: pgmap v665: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 KiB/s wr, 8 op/s 2023-09-09T23:34:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 930.759750 secs 2023-09-09T23:34:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:34:59 smithi082 ceph-mon[98694]: pgmap v665: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 KiB/s wr, 8 op/s 2023-09-09T23:35:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 930.759750 secs 2023-09-09T23:35:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:34:59 smithi098 ceph-mon[103124]: pgmap v665: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 KiB/s wr, 8 op/s 2023-09-09T23:35:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:00 smithi170 ceph-mon[102832]: pgmap v666: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.1 KiB/s wr, 12 op/s 2023-09-09T23:35:00.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:00 smithi082 ceph-mon[98694]: pgmap v666: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.1 KiB/s wr, 12 op/s 2023-09-09T23:35:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:00 smithi098 ceph-mon[103124]: pgmap v666: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.1 KiB/s wr, 12 op/s 2023-09-09T23:35:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 933.431488 secs 2023-09-09T23:35:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 933.431488 secs 2023-09-09T23:35:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 933.431488 secs 2023-09-09T23:35:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:03 smithi170 ceph-mon[102832]: pgmap v667: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 KiB/s wr, 10 op/s 2023-09-09T23:35:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 935.030810 secs 2023-09-09T23:35:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:03 smithi082 ceph-mon[98694]: pgmap v667: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 KiB/s wr, 10 op/s 2023-09-09T23:35:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 935.030810 secs 2023-09-09T23:35:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:03 smithi098 ceph-mon[103124]: pgmap v667: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 KiB/s wr, 10 op/s 2023-09-09T23:35:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 935.030810 secs 2023-09-09T23:35:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 935.759817 secs 2023-09-09T23:35:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 935.759817 secs 2023-09-09T23:35:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 935.759817 secs 2023-09-09T23:35:05.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:05 smithi170 ceph-mon[102832]: pgmap v668: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-09T23:35:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:05 smithi082 ceph-mon[98694]: pgmap v668: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-09T23:35:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:05 smithi098 ceph-mon[103124]: pgmap v668: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-09T23:35:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:07 smithi170 ceph-mon[102832]: pgmap v669: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2023-09-09T23:35:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 938.431551 secs 2023-09-09T23:35:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:07 smithi082 ceph-mon[98694]: pgmap v669: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2023-09-09T23:35:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 938.431551 secs 2023-09-09T23:35:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:07 smithi098 ceph-mon[103124]: pgmap v669: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2023-09-09T23:35:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 938.431551 secs 2023-09-09T23:35:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:08 smithi170 ceph-mon[102832]: osdmap e115: 12 total, 12 up, 12 in 2023-09-09T23:35:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 940.030868 secs 2023-09-09T23:35:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:08 smithi170 ceph-mon[102832]: pgmap v671: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:35:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:08 smithi082 ceph-mon[98694]: osdmap e115: 12 total, 12 up, 12 in 2023-09-09T23:35:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 940.030868 secs 2023-09-09T23:35:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:08 smithi082 ceph-mon[98694]: pgmap v671: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:35:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:08 smithi098 ceph-mon[103124]: osdmap e115: 12 total, 12 up, 12 in 2023-09-09T23:35:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 940.030868 secs 2023-09-09T23:35:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:08 smithi098 ceph-mon[103124]: pgmap v671: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:35:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 940.759884 secs 2023-09-09T23:35:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 940.759884 secs 2023-09-09T23:35:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 940.759884 secs 2023-09-09T23:35:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:10 smithi170 ceph-mon[102832]: pgmap v672: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:35:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:10 smithi082 ceph-mon[98694]: pgmap v672: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:35:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:10 smithi098 ceph-mon[103124]: pgmap v672: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:35:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 943.431621 secs 2023-09-09T23:35:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 943.431621 secs 2023-09-09T23:35:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 943.431621 secs 2023-09-09T23:35:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:13 smithi170 ceph-mon[102832]: pgmap v673: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:35:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 945.030955 secs 2023-09-09T23:35:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:13 smithi082 ceph-mon[98694]: pgmap v673: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:35:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 945.030955 secs 2023-09-09T23:35:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:13 smithi098 ceph-mon[103124]: pgmap v673: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:35:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 945.030955 secs 2023-09-09T23:35:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 945.759949 secs 2023-09-09T23:35:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:14 smithi170 ceph-mon[102832]: osdmap e116: 12 total, 12 up, 12 in 2023-09-09T23:35:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 945.759949 secs 2023-09-09T23:35:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:14 smithi082 ceph-mon[98694]: osdmap e116: 12 total, 12 up, 12 in 2023-09-09T23:35:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 945.759949 secs 2023-09-09T23:35:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:14 smithi098 ceph-mon[103124]: osdmap e116: 12 total, 12 up, 12 in 2023-09-09T23:35:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:15 smithi170 ceph-mon[102832]: pgmap v675: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-09T23:35:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:15 smithi082 ceph-mon[98694]: pgmap v675: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-09T23:35:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:15 smithi098 ceph-mon[103124]: pgmap v675: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-09T23:35:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:16 smithi082 ceph-mon[98694]: pgmap v676: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.2 KiB/s wr, 12 op/s 2023-09-09T23:35:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:16 smithi098 ceph-mon[103124]: pgmap v676: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.2 KiB/s wr, 12 op/s 2023-09-09T23:35:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:16 smithi170 ceph-mon[102832]: pgmap v676: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.2 KiB/s wr, 12 op/s 2023-09-09T23:35:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 948.431684 secs 2023-09-09T23:35:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 948.431684 secs 2023-09-09T23:35:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 948.431684 secs 2023-09-09T23:35:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 950.031013 secs 2023-09-09T23:35:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:18 smithi082 ceph-mon[98694]: pgmap v677: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.3 KiB/s wr, 11 op/s 2023-09-09T23:35:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 950.031013 secs 2023-09-09T23:35:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:18 smithi098 ceph-mon[103124]: pgmap v677: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.3 KiB/s wr, 11 op/s 2023-09-09T23:35:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 950.031013 secs 2023-09-09T23:35:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:18 smithi170 ceph-mon[102832]: pgmap v677: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.3 KiB/s wr, 11 op/s 2023-09-09T23:35:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 950.760020 secs 2023-09-09T23:35:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 950.760020 secs 2023-09-09T23:35:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 950.760020 secs 2023-09-09T23:35:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:20 smithi082 ceph-mon[98694]: pgmap v678: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 12 op/s 2023-09-09T23:35:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:20 smithi098 ceph-mon[103124]: pgmap v678: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 12 op/s 2023-09-09T23:35:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:20 smithi170 ceph-mon[102832]: pgmap v678: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 12 op/s 2023-09-09T23:35:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 953.431743 secs 2023-09-09T23:35:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 953.431743 secs 2023-09-09T23:35:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 953.431743 secs 2023-09-09T23:35:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:23 smithi170 ceph-mon[102832]: pgmap v679: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-09T23:35:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 955.031065 secs 2023-09-09T23:35:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:23 smithi082 ceph-mon[98694]: pgmap v679: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-09T23:35:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 955.031065 secs 2023-09-09T23:35:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:23 smithi098 ceph-mon[103124]: pgmap v679: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-09T23:35:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 955.031065 secs 2023-09-09T23:35:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 955.760083 secs 2023-09-09T23:35:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 955.760083 secs 2023-09-09T23:35:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 955.760083 secs 2023-09-09T23:35:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:25 smithi170 ceph-mon[102832]: pgmap v680: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-09T23:35:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:25 smithi082 ceph-mon[98694]: pgmap v680: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-09T23:35:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:25 smithi098 ceph-mon[103124]: pgmap v680: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-09T23:35:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:26 smithi170 ceph-mon[102832]: pgmap v681: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 19 KiB/s wr, 14 op/s 2023-09-09T23:35:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:26 smithi082 ceph-mon[98694]: pgmap v681: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 19 KiB/s wr, 14 op/s 2023-09-09T23:35:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:26 smithi098 ceph-mon[103124]: pgmap v681: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 19 KiB/s wr, 14 op/s 2023-09-09T23:35:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 958.431777 secs 2023-09-09T23:35:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 958.431777 secs 2023-09-09T23:35:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 958.431777 secs 2023-09-09T23:35:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 960.031126 secs 2023-09-09T23:35:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:28 smithi170 ceph-mon[102832]: pgmap v682: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 9 op/s 2023-09-09T23:35:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 960.031126 secs 2023-09-09T23:35:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:28 smithi082 ceph-mon[98694]: pgmap v682: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 9 op/s 2023-09-09T23:35:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 960.031126 secs 2023-09-09T23:35:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:28 smithi098 ceph-mon[103124]: pgmap v682: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 9 op/s 2023-09-09T23:35:29.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 960.760148 secs 2023-09-09T23:35:29.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:35:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 960.760148 secs 2023-09-09T23:35:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:35:30.395 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 960.760148 secs 2023-09-09T23:35:30.396 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:35:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:30 smithi170 ceph-mon[102832]: pgmap v683: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 12 op/s 2023-09-09T23:35:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:30 smithi082 ceph-mon[98694]: pgmap v683: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 12 op/s 2023-09-09T23:35:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:30 smithi098 ceph-mon[103124]: pgmap v683: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 12 op/s 2023-09-09T23:35:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:35:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:35:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:35:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:35:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:35:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:35:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:35:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:35:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:35:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 963.431857 secs 2023-09-09T23:35:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:32 smithi170 ceph-mon[102832]: pgmap v684: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 KiB/s wr, 10 op/s 2023-09-09T23:35:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 963.431857 secs 2023-09-09T23:35:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:32 smithi082 ceph-mon[98694]: pgmap v684: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 KiB/s wr, 10 op/s 2023-09-09T23:35:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 963.431857 secs 2023-09-09T23:35:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:32 smithi098 ceph-mon[103124]: pgmap v684: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 KiB/s wr, 10 op/s 2023-09-09T23:35:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 965.031206 secs 2023-09-09T23:35:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 965.031206 secs 2023-09-09T23:35:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 965.031206 secs 2023-09-09T23:35:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 965.760212 secs 2023-09-09T23:35:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:34 smithi170 ceph-mon[102832]: pgmap v685: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.3 KiB/s wr, 8 op/s 2023-09-09T23:35:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 965.760212 secs 2023-09-09T23:35:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:34 smithi082 ceph-mon[98694]: pgmap v685: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.3 KiB/s wr, 8 op/s 2023-09-09T23:35:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 965.760212 secs 2023-09-09T23:35:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:34 smithi098 ceph-mon[103124]: pgmap v685: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.3 KiB/s wr, 8 op/s 2023-09-09T23:35:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:37 smithi170 ceph-mon[102832]: pgmap v686: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2023-09-09T23:35:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 968.431951 secs 2023-09-09T23:35:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:37 smithi082 ceph-mon[98694]: pgmap v686: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2023-09-09T23:35:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 968.431951 secs 2023-09-09T23:35:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:37 smithi098 ceph-mon[103124]: pgmap v686: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2023-09-09T23:35:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 968.431951 secs 2023-09-09T23:35:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 970.031263 secs 2023-09-09T23:35:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 970.031263 secs 2023-09-09T23:35:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 970.031263 secs 2023-09-09T23:35:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 970.760276 secs 2023-09-09T23:35:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:39 smithi170 ceph-mon[102832]: pgmap v687: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2023-09-09T23:35:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 970.760276 secs 2023-09-09T23:35:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:39 smithi082 ceph-mon[98694]: pgmap v687: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2023-09-09T23:35:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 970.760276 secs 2023-09-09T23:35:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:39 smithi098 ceph-mon[103124]: pgmap v687: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2023-09-09T23:35:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:41 smithi170 ceph-mon[102832]: pgmap v688: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2023-09-09T23:35:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:41 smithi082 ceph-mon[98694]: pgmap v688: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2023-09-09T23:35:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:41 smithi098 ceph-mon[103124]: pgmap v688: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2023-09-09T23:35:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 973.432056 secs 2023-09-09T23:35:42.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:42 smithi170 ceph-mon[102832]: pgmap v689: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2023-09-09T23:35:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 973.432056 secs 2023-09-09T23:35:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:42 smithi082 ceph-mon[98694]: pgmap v689: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2023-09-09T23:35:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 973.432056 secs 2023-09-09T23:35:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:42 smithi098 ceph-mon[103124]: pgmap v689: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2023-09-09T23:35:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 975.031322 secs 2023-09-09T23:35:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 975.031322 secs 2023-09-09T23:35:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 975.031322 secs 2023-09-09T23:35:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 975.760341 secs 2023-09-09T23:35:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:44 smithi170 ceph-mon[102832]: pgmap v690: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:35:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 975.760341 secs 2023-09-09T23:35:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:44 smithi082 ceph-mon[98694]: pgmap v690: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:35:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 975.760341 secs 2023-09-09T23:35:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:44 smithi098 ceph-mon[103124]: pgmap v690: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:35:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:47 smithi170 ceph-mon[102832]: pgmap v691: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2023-09-09T23:35:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 978.432119 secs 2023-09-09T23:35:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:47 smithi082 ceph-mon[98694]: pgmap v691: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2023-09-09T23:35:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 978.432119 secs 2023-09-09T23:35:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:47 smithi098 ceph-mon[103124]: pgmap v691: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2023-09-09T23:35:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 978.432119 secs 2023-09-09T23:35:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 980.031380 secs 2023-09-09T23:35:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 980.031380 secs 2023-09-09T23:35:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 980.031380 secs 2023-09-09T23:35:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 980.760410 secs 2023-09-09T23:35:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:49 smithi170 ceph-mon[102832]: pgmap v692: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2023-09-09T23:35:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 980.760410 secs 2023-09-09T23:35:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:49 smithi082 ceph-mon[98694]: pgmap v692: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2023-09-09T23:35:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 980.760410 secs 2023-09-09T23:35:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:49 smithi098 ceph-mon[103124]: pgmap v692: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2023-09-09T23:35:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:50 smithi082 ceph-mon[98694]: pgmap v693: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.4 KiB/s wr, 12 op/s 2023-09-09T23:35:51.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:50 smithi098 ceph-mon[103124]: pgmap v693: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.4 KiB/s wr, 12 op/s 2023-09-09T23:35:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:50 smithi170 ceph-mon[102832]: pgmap v693: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.4 KiB/s wr, 12 op/s 2023-09-09T23:35:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 983.432179 secs 2023-09-09T23:35:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 983.432179 secs 2023-09-09T23:35:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 983.432179 secs 2023-09-09T23:35:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:53 smithi170 ceph-mon[102832]: pgmap v694: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2023-09-09T23:35:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 985.031467 secs 2023-09-09T23:35:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:53 smithi082 ceph-mon[98694]: pgmap v694: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2023-09-09T23:35:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 985.031467 secs 2023-09-09T23:35:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:53 smithi098 ceph-mon[103124]: pgmap v694: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2023-09-09T23:35:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 985.031467 secs 2023-09-09T23:35:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 985.760478 secs 2023-09-09T23:35:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 985.760478 secs 2023-09-09T23:35:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 985.760478 secs 2023-09-09T23:35:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:55 smithi170 ceph-mon[102832]: pgmap v695: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2023-09-09T23:35:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:55 smithi082 ceph-mon[98694]: pgmap v695: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2023-09-09T23:35:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:55 smithi098 ceph-mon[103124]: pgmap v695: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2023-09-09T23:35:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:56 smithi170 ceph-mon[102832]: pgmap v696: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 13 KiB/s wr, 14 op/s 2023-09-09T23:35:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:56 smithi082 ceph-mon[98694]: pgmap v696: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 13 KiB/s wr, 14 op/s 2023-09-09T23:35:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:56 smithi098 ceph-mon[103124]: pgmap v696: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 13 KiB/s wr, 14 op/s 2023-09-09T23:35:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 988.432250 secs 2023-09-09T23:35:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 988.432250 secs 2023-09-09T23:35:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 988.432250 secs 2023-09-09T23:35:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 990.031496 secs 2023-09-09T23:35:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:58 smithi170 ceph-mon[102832]: pgmap v697: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 12 KiB/s wr, 9 op/s 2023-09-09T23:35:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 990.031496 secs 2023-09-09T23:35:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:58 smithi082 ceph-mon[98694]: pgmap v697: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 12 KiB/s wr, 9 op/s 2023-09-09T23:35:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 990.031496 secs 2023-09-09T23:35:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:58 smithi098 ceph-mon[103124]: pgmap v697: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 12 KiB/s wr, 9 op/s 2023-09-09T23:35:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:35:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 990.760552 secs 2023-09-09T23:35:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:35:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 990.760552 secs 2023-09-09T23:36:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:35:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 990.760552 secs 2023-09-09T23:36:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:00 smithi170 ceph-mon[102832]: pgmap v698: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 12 KiB/s wr, 13 op/s 2023-09-09T23:36:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:00 smithi082 ceph-mon[98694]: pgmap v698: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 12 KiB/s wr, 13 op/s 2023-09-09T23:36:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:00 smithi098 ceph-mon[103124]: pgmap v698: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 12 KiB/s wr, 13 op/s 2023-09-09T23:36:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 993.432340 secs 2023-09-09T23:36:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 993.432340 secs 2023-09-09T23:36:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 993.432340 secs 2023-09-09T23:36:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:03 smithi170 ceph-mon[102832]: pgmap v699: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 KiB/s wr, 10 op/s 2023-09-09T23:36:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 995.031555 secs 2023-09-09T23:36:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:03 smithi082 ceph-mon[98694]: pgmap v699: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 KiB/s wr, 10 op/s 2023-09-09T23:36:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 995.031555 secs 2023-09-09T23:36:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:03 smithi098 ceph-mon[103124]: pgmap v699: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 KiB/s wr, 10 op/s 2023-09-09T23:36:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 995.031555 secs 2023-09-09T23:36:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 995.760614 secs 2023-09-09T23:36:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 995.760614 secs 2023-09-09T23:36:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 995.760614 secs 2023-09-09T23:36:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:05 smithi170 ceph-mon[102832]: pgmap v700: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.4 KiB/s wr, 9 op/s 2023-09-09T23:36:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:05 smithi082 ceph-mon[98694]: pgmap v700: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.4 KiB/s wr, 9 op/s 2023-09-09T23:36:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:05 smithi098 ceph-mon[103124]: pgmap v700: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.4 KiB/s wr, 9 op/s 2023-09-09T23:36:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:06 smithi170 ceph-mon[102832]: pgmap v701: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 53 KiB/s wr, 28 op/s 2023-09-09T23:36:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:06 smithi082 ceph-mon[98694]: pgmap v701: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 53 KiB/s wr, 28 op/s 2023-09-09T23:36:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:06 smithi098 ceph-mon[103124]: pgmap v701: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 53 KiB/s wr, 28 op/s 2023-09-09T23:36:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 998.432399 secs 2023-09-09T23:36:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 998.432399 secs 2023-09-09T23:36:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 998.432399 secs 2023-09-09T23:36:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:08 smithi170 ceph-mon[102832]: osdmap e117: 12 total, 12 up, 12 in 2023-09-09T23:36:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1000.031627 secs 2023-09-09T23:36:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:08 smithi170 ceph-mon[102832]: pgmap v703: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:36:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:08 smithi082 ceph-mon[98694]: osdmap e117: 12 total, 12 up, 12 in 2023-09-09T23:36:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1000.031627 secs 2023-09-09T23:36:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:08 smithi082 ceph-mon[98694]: pgmap v703: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:36:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:08 smithi098 ceph-mon[103124]: osdmap e117: 12 total, 12 up, 12 in 2023-09-09T23:36:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1000.031627 secs 2023-09-09T23:36:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:08 smithi098 ceph-mon[103124]: pgmap v703: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:36:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1000.760682 secs 2023-09-09T23:36:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1000.760682 secs 2023-09-09T23:36:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1000.760682 secs 2023-09-09T23:36:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:10 smithi082 ceph-mon[98694]: pgmap v704: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:36:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:10 smithi098 ceph-mon[103124]: pgmap v704: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:36:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:10 smithi170 ceph-mon[102832]: pgmap v704: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:36:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1003.432479 secs 2023-09-09T23:36:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1003.432479 secs 2023-09-09T23:36:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1003.432479 secs 2023-09-09T23:36:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:13 smithi170 ceph-mon[102832]: pgmap v705: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:36:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1005.031687 secs 2023-09-09T23:36:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:13 smithi082 ceph-mon[98694]: pgmap v705: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:36:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1005.031687 secs 2023-09-09T23:36:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:13 smithi098 ceph-mon[103124]: pgmap v705: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:36:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1005.031687 secs 2023-09-09T23:36:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1005.760745 secs 2023-09-09T23:36:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:14 smithi170 ceph-mon[102832]: osdmap e118: 12 total, 12 up, 12 in 2023-09-09T23:36:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:14 smithi170 ceph-mon[102832]: pgmap v707: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:36:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1005.760745 secs 2023-09-09T23:36:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:14 smithi082 ceph-mon[98694]: osdmap e118: 12 total, 12 up, 12 in 2023-09-09T23:36:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:14 smithi082 ceph-mon[98694]: pgmap v707: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:36:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1005.760745 secs 2023-09-09T23:36:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:14 smithi098 ceph-mon[103124]: osdmap e118: 12 total, 12 up, 12 in 2023-09-09T23:36:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:14 smithi098 ceph-mon[103124]: pgmap v707: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:36:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:17 smithi170 ceph-mon[102832]: pgmap v708: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.6 KiB/s wr, 12 op/s 2023-09-09T23:36:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1008.432541 secs 2023-09-09T23:36:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:17 smithi082 ceph-mon[98694]: pgmap v708: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.6 KiB/s wr, 12 op/s 2023-09-09T23:36:17.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1008.432541 secs 2023-09-09T23:36:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:17 smithi098 ceph-mon[103124]: pgmap v708: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.6 KiB/s wr, 12 op/s 2023-09-09T23:36:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1008.432541 secs 2023-09-09T23:36:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1010.031746 secs 2023-09-09T23:36:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1010.031746 secs 2023-09-09T23:36:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1010.031746 secs 2023-09-09T23:36:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1010.760814 secs 2023-09-09T23:36:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:19 smithi170 ceph-mon[102832]: pgmap v709: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 KiB/s wr, 11 op/s 2023-09-09T23:36:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1010.760814 secs 2023-09-09T23:36:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:19 smithi082 ceph-mon[98694]: pgmap v709: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 KiB/s wr, 11 op/s 2023-09-09T23:36:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1010.760814 secs 2023-09-09T23:36:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:19 smithi098 ceph-mon[103124]: pgmap v709: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 KiB/s wr, 11 op/s 2023-09-09T23:36:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:21 smithi170 ceph-mon[102832]: pgmap v710: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 12 op/s 2023-09-09T23:36:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:21 smithi082 ceph-mon[98694]: pgmap v710: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 12 op/s 2023-09-09T23:36:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:21 smithi098 ceph-mon[103124]: pgmap v710: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 12 op/s 2023-09-09T23:36:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1013.432678 secs 2023-09-09T23:36:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1013.432678 secs 2023-09-09T23:36:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1013.432678 secs 2023-09-09T23:36:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:23 smithi170 ceph-mon[102832]: pgmap v711: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-09T23:36:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1015.031804 secs 2023-09-09T23:36:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:23 smithi082 ceph-mon[98694]: pgmap v711: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-09T23:36:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1015.031804 secs 2023-09-09T23:36:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:23 smithi098 ceph-mon[103124]: pgmap v711: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-09T23:36:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1015.031804 secs 2023-09-09T23:36:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1015.760880 secs 2023-09-09T23:36:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:24 smithi170 ceph-mon[102832]: pgmap v712: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-09T23:36:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1015.760880 secs 2023-09-09T23:36:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:24 smithi082 ceph-mon[98694]: pgmap v712: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-09T23:36:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1015.760880 secs 2023-09-09T23:36:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:24 smithi098 ceph-mon[103124]: pgmap v712: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-09T23:36:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:27 smithi170 ceph-mon[102832]: pgmap v713: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 20 KiB/s wr, 14 op/s 2023-09-09T23:36:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1018.432739 secs 2023-09-09T23:36:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:27 smithi082 ceph-mon[98694]: pgmap v713: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 20 KiB/s wr, 14 op/s 2023-09-09T23:36:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1018.432739 secs 2023-09-09T23:36:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:27 smithi098 ceph-mon[103124]: pgmap v713: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 20 KiB/s wr, 14 op/s 2023-09-09T23:36:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1018.432739 secs 2023-09-09T23:36:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1020.031863 secs 2023-09-09T23:36:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1020.031863 secs 2023-09-09T23:36:29.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1020.031863 secs 2023-09-09T23:36:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1020.760945 secs 2023-09-09T23:36:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:29 smithi170 ceph-mon[102832]: pgmap v714: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 9 op/s 2023-09-09T23:36:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1020.760945 secs 2023-09-09T23:36:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:29 smithi082 ceph-mon[98694]: pgmap v714: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 9 op/s 2023-09-09T23:36:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1020.760945 secs 2023-09-09T23:36:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:29 smithi098 ceph-mon[103124]: pgmap v714: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 9 op/s 2023-09-09T23:36:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:31 smithi170 ceph-mon[102832]: pgmap v715: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-09-09T23:36:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:36:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:36:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:36:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:31 smithi082 ceph-mon[98694]: pgmap v715: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-09-09T23:36:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:36:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:36:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:36:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:31 smithi098 ceph-mon[103124]: pgmap v715: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-09-09T23:36:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:36:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:36:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:36:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1023.432775 secs 2023-09-09T23:36:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:36:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:36:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:36:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1023.432775 secs 2023-09-09T23:36:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:36:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:36:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:36:33.042 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1023.432775 secs 2023-09-09T23:36:33.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:36:33.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:36:33.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:36:33.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:33 smithi170 ceph-mon[102832]: pgmap v716: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 KiB/s wr, 10 op/s 2023-09-09T23:36:33.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1025.031921 secs 2023-09-09T23:36:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:33 smithi082 ceph-mon[98694]: pgmap v716: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 KiB/s wr, 10 op/s 2023-09-09T23:36:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1025.031921 secs 2023-09-09T23:36:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:33 smithi098 ceph-mon[103124]: pgmap v716: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 KiB/s wr, 10 op/s 2023-09-09T23:36:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1025.031921 secs 2023-09-09T23:36:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1025.761008 secs 2023-09-09T23:36:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:34 smithi170 ceph-mon[102832]: pgmap v717: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2023-09-09T23:36:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1025.761008 secs 2023-09-09T23:36:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:34 smithi082 ceph-mon[98694]: pgmap v717: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2023-09-09T23:36:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1025.761008 secs 2023-09-09T23:36:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:34 smithi098 ceph-mon[103124]: pgmap v717: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2023-09-09T23:36:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:37 smithi170 ceph-mon[102832]: pgmap v718: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.6 KiB/s wr, 12 op/s 2023-09-09T23:36:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1028.432835 secs 2023-09-09T23:36:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:37 smithi082 ceph-mon[98694]: pgmap v718: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.6 KiB/s wr, 12 op/s 2023-09-09T23:36:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1028.432835 secs 2023-09-09T23:36:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:37 smithi098 ceph-mon[103124]: pgmap v718: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.6 KiB/s wr, 12 op/s 2023-09-09T23:36:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1028.432835 secs 2023-09-09T23:36:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1030.031979 secs 2023-09-09T23:36:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1030.031979 secs 2023-09-09T23:36:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1030.031979 secs 2023-09-09T23:36:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1030.761072 secs 2023-09-09T23:36:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:39 smithi170 ceph-mon[102832]: pgmap v719: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.4 KiB/s wr, 8 op/s 2023-09-09T23:36:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1030.761072 secs 2023-09-09T23:36:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:39 smithi082 ceph-mon[98694]: pgmap v719: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.4 KiB/s wr, 8 op/s 2023-09-09T23:36:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1030.761072 secs 2023-09-09T23:36:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:39 smithi098 ceph-mon[103124]: pgmap v719: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.4 KiB/s wr, 8 op/s 2023-09-09T23:36:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:40 smithi082 ceph-mon[98694]: pgmap v720: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 KiB/s wr, 11 op/s 2023-09-09T23:36:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:40 smithi098 ceph-mon[103124]: pgmap v720: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 KiB/s wr, 11 op/s 2023-09-09T23:36:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:40 smithi170 ceph-mon[102832]: pgmap v720: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 KiB/s wr, 11 op/s 2023-09-09T23:36:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1033.432894 secs 2023-09-09T23:36:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1033.432894 secs 2023-09-09T23:36:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1033.432894 secs 2023-09-09T23:36:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:43 smithi170 ceph-mon[102832]: pgmap v721: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2023-09-09T23:36:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1035.032063 secs 2023-09-09T23:36:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:43 smithi082 ceph-mon[98694]: pgmap v721: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2023-09-09T23:36:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1035.032063 secs 2023-09-09T23:36:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:43 smithi098 ceph-mon[103124]: pgmap v721: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2023-09-09T23:36:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1035.032063 secs 2023-09-09T23:36:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1035.761139 secs 2023-09-09T23:36:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:44 smithi170 ceph-mon[102832]: pgmap v722: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2023-09-09T23:36:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1035.761139 secs 2023-09-09T23:36:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:44 smithi082 ceph-mon[98694]: pgmap v722: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2023-09-09T23:36:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1035.761139 secs 2023-09-09T23:36:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:44 smithi098 ceph-mon[103124]: pgmap v722: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2023-09-09T23:36:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:47 smithi170 ceph-mon[102832]: pgmap v723: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.0 KiB/s wr, 12 op/s 2023-09-09T23:36:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1038.432955 secs 2023-09-09T23:36:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:47 smithi082 ceph-mon[98694]: pgmap v723: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.0 KiB/s wr, 12 op/s 2023-09-09T23:36:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1038.432955 secs 2023-09-09T23:36:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:47 smithi098 ceph-mon[103124]: pgmap v723: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.0 KiB/s wr, 12 op/s 2023-09-09T23:36:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1038.432955 secs 2023-09-09T23:36:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1040.032121 secs 2023-09-09T23:36:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1040.032121 secs 2023-09-09T23:36:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1040.032121 secs 2023-09-09T23:36:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1040.761207 secs 2023-09-09T23:36:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:49 smithi170 ceph-mon[102832]: pgmap v724: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2023-09-09T23:36:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1040.761207 secs 2023-09-09T23:36:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:49 smithi082 ceph-mon[98694]: pgmap v724: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2023-09-09T23:36:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1040.761207 secs 2023-09-09T23:36:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:49 smithi098 ceph-mon[103124]: pgmap v724: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2023-09-09T23:36:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:51 smithi170 ceph-mon[102832]: pgmap v725: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2023-09-09T23:36:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:51 smithi082 ceph-mon[98694]: pgmap v725: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2023-09-09T23:36:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:51 smithi098 ceph-mon[103124]: pgmap v725: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2023-09-09T23:36:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1043.433014 secs 2023-09-09T23:36:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1043.433014 secs 2023-09-09T23:36:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1043.433014 secs 2023-09-09T23:36:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:53 smithi170 ceph-mon[102832]: pgmap v726: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2023-09-09T23:36:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1045.032181 secs 2023-09-09T23:36:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:53 smithi082 ceph-mon[98694]: pgmap v726: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2023-09-09T23:36:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1045.032181 secs 2023-09-09T23:36:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:53 smithi098 ceph-mon[103124]: pgmap v726: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2023-09-09T23:36:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1045.032181 secs 2023-09-09T23:36:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1045.761273 secs 2023-09-09T23:36:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:54 smithi170 ceph-mon[102832]: pgmap v727: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2023-09-09T23:36:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1045.761273 secs 2023-09-09T23:36:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:54 smithi082 ceph-mon[98694]: pgmap v727: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2023-09-09T23:36:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1045.761273 secs 2023-09-09T23:36:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:54 smithi098 ceph-mon[103124]: pgmap v727: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2023-09-09T23:36:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:57 smithi170 ceph-mon[102832]: pgmap v728: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 13 KiB/s wr, 14 op/s 2023-09-09T23:36:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1048.433089 secs 2023-09-09T23:36:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:57 smithi082 ceph-mon[98694]: pgmap v728: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 13 KiB/s wr, 14 op/s 2023-09-09T23:36:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1048.433089 secs 2023-09-09T23:36:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:57 smithi098 ceph-mon[103124]: pgmap v728: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 13 KiB/s wr, 14 op/s 2023-09-09T23:36:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1048.433089 secs 2023-09-09T23:36:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1050.032240 secs 2023-09-09T23:36:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1050.032240 secs 2023-09-09T23:36:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1050.032240 secs 2023-09-09T23:36:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1050.761340 secs 2023-09-09T23:36:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:36:59 smithi170 ceph-mon[102832]: pgmap v729: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2023-09-09T23:36:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1050.761340 secs 2023-09-09T23:36:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:36:59 smithi082 ceph-mon[98694]: pgmap v729: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2023-09-09T23:37:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1050.761340 secs 2023-09-09T23:37:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:36:59 smithi098 ceph-mon[103124]: pgmap v729: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2023-09-09T23:37:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:00 smithi082 ceph-mon[98694]: pgmap v730: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.3 KiB/s wr, 12 op/s 2023-09-09T23:37:01.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:00 smithi098 ceph-mon[103124]: pgmap v730: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.3 KiB/s wr, 12 op/s 2023-09-09T23:37:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:00 smithi170 ceph-mon[102832]: pgmap v730: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.3 KiB/s wr, 12 op/s 2023-09-09T23:37:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1053.433147 secs 2023-09-09T23:37:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1053.433147 secs 2023-09-09T23:37:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1053.433147 secs 2023-09-09T23:37:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:03 smithi170 ceph-mon[102832]: pgmap v731: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.7 KiB/s wr, 10 op/s 2023-09-09T23:37:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1055.032421 secs 2023-09-09T23:37:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:03 smithi082 ceph-mon[98694]: pgmap v731: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.7 KiB/s wr, 10 op/s 2023-09-09T23:37:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1055.032421 secs 2023-09-09T23:37:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:03 smithi098 ceph-mon[103124]: pgmap v731: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.7 KiB/s wr, 10 op/s 2023-09-09T23:37:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1055.032421 secs 2023-09-09T23:37:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1055.761407 secs 2023-09-09T23:37:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:04 smithi170 ceph-mon[102832]: pgmap v732: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 KiB/s wr, 8 op/s 2023-09-09T23:37:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1055.761407 secs 2023-09-09T23:37:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:04 smithi082 ceph-mon[98694]: pgmap v732: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 KiB/s wr, 8 op/s 2023-09-09T23:37:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1055.761407 secs 2023-09-09T23:37:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:04 smithi098 ceph-mon[103124]: pgmap v732: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 KiB/s wr, 8 op/s 2023-09-09T23:37:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:07 smithi170 ceph-mon[102832]: pgmap v733: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 53 KiB/s wr, 27 op/s 2023-09-09T23:37:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1058.433206 secs 2023-09-09T23:37:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:07 smithi082 ceph-mon[98694]: pgmap v733: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 53 KiB/s wr, 27 op/s 2023-09-09T23:37:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1058.433206 secs 2023-09-09T23:37:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:07 smithi098 ceph-mon[103124]: pgmap v733: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 53 KiB/s wr, 27 op/s 2023-09-09T23:37:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1058.433206 secs 2023-09-09T23:37:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:08 smithi170 ceph-mon[102832]: osdmap e119: 12 total, 12 up, 12 in 2023-09-09T23:37:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1060.032479 secs 2023-09-09T23:37:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:08 smithi082 ceph-mon[98694]: osdmap e119: 12 total, 12 up, 12 in 2023-09-09T23:37:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1060.032479 secs 2023-09-09T23:37:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:08 smithi098 ceph-mon[103124]: osdmap e119: 12 total, 12 up, 12 in 2023-09-09T23:37:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1060.032479 secs 2023-09-09T23:37:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1060.761476 secs 2023-09-09T23:37:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:09 smithi170 ceph-mon[102832]: pgmap v735: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:37:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1060.761476 secs 2023-09-09T23:37:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:09 smithi082 ceph-mon[98694]: pgmap v735: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:37:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1060.761476 secs 2023-09-09T23:37:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:09 smithi098 ceph-mon[103124]: pgmap v735: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:37:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:10 smithi170 ceph-mon[102832]: pgmap v736: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:37:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:10 smithi082 ceph-mon[98694]: pgmap v736: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:37:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:10 smithi098 ceph-mon[103124]: pgmap v736: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:37:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1063.433265 secs 2023-09-09T23:37:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1063.433265 secs 2023-09-09T23:37:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1063.433265 secs 2023-09-09T23:37:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:13 smithi170 ceph-mon[102832]: pgmap v737: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:37:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1065.032493 secs 2023-09-09T23:37:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:13 smithi082 ceph-mon[98694]: pgmap v737: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:37:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1065.032493 secs 2023-09-09T23:37:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:13 smithi098 ceph-mon[103124]: pgmap v737: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:37:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1065.032493 secs 2023-09-09T23:37:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1065.761541 secs 2023-09-09T23:37:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:14 smithi170 ceph-mon[102832]: osdmap e120: 12 total, 12 up, 12 in 2023-09-09T23:37:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:14 smithi170 ceph-mon[102832]: pgmap v739: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:37:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1065.761541 secs 2023-09-09T23:37:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:14 smithi082 ceph-mon[98694]: osdmap e120: 12 total, 12 up, 12 in 2023-09-09T23:37:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:14 smithi082 ceph-mon[98694]: pgmap v739: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:37:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1065.761541 secs 2023-09-09T23:37:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:14 smithi098 ceph-mon[103124]: osdmap e120: 12 total, 12 up, 12 in 2023-09-09T23:37:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:14 smithi098 ceph-mon[103124]: pgmap v739: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:37:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:17 smithi170 ceph-mon[102832]: pgmap v740: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.8 KiB/s wr, 12 op/s 2023-09-09T23:37:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1068.433328 secs 2023-09-09T23:37:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:17 smithi082 ceph-mon[98694]: pgmap v740: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.8 KiB/s wr, 12 op/s 2023-09-09T23:37:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1068.433328 secs 2023-09-09T23:37:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:17 smithi098 ceph-mon[103124]: pgmap v740: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.8 KiB/s wr, 12 op/s 2023-09-09T23:37:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1068.433328 secs 2023-09-09T23:37:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1070.032522 secs 2023-09-09T23:37:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1070.032522 secs 2023-09-09T23:37:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1070.032522 secs 2023-09-09T23:37:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1070.761607 secs 2023-09-09T23:37:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:19 smithi170 ceph-mon[102832]: pgmap v741: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.9 KiB/s wr, 11 op/s 2023-09-09T23:37:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1070.761607 secs 2023-09-09T23:37:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:19 smithi082 ceph-mon[98694]: pgmap v741: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.9 KiB/s wr, 11 op/s 2023-09-09T23:37:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1070.761607 secs 2023-09-09T23:37:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:19 smithi098 ceph-mon[103124]: pgmap v741: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.9 KiB/s wr, 11 op/s 2023-09-09T23:37:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:20 smithi098 ceph-mon[103124]: pgmap v742: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 12 op/s 2023-09-09T23:37:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:20 smithi170 ceph-mon[102832]: pgmap v742: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 12 op/s 2023-09-09T23:37:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:20 smithi082 ceph-mon[98694]: pgmap v742: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 12 op/s 2023-09-09T23:37:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1073.433390 secs 2023-09-09T23:37:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1073.433390 secs 2023-09-09T23:37:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1073.433390 secs 2023-09-09T23:37:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:23 smithi170 ceph-mon[102832]: pgmap v743: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 12 op/s 2023-09-09T23:37:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1075.032580 secs 2023-09-09T23:37:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:23 smithi082 ceph-mon[98694]: pgmap v743: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 12 op/s 2023-09-09T23:37:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1075.032580 secs 2023-09-09T23:37:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:23 smithi098 ceph-mon[103124]: pgmap v743: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 12 op/s 2023-09-09T23:37:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1075.032580 secs 2023-09-09T23:37:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1075.761673 secs 2023-09-09T23:37:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:24 smithi170 ceph-mon[102832]: pgmap v744: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-09T23:37:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1075.761673 secs 2023-09-09T23:37:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:24 smithi082 ceph-mon[98694]: pgmap v744: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-09T23:37:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1075.761673 secs 2023-09-09T23:37:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:24 smithi098 ceph-mon[103124]: pgmap v744: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-09T23:37:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:27 smithi170 ceph-mon[102832]: pgmap v745: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 21 KiB/s wr, 14 op/s 2023-09-09T23:37:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1078.433575 secs 2023-09-09T23:37:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:27 smithi082 ceph-mon[98694]: pgmap v745: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 21 KiB/s wr, 14 op/s 2023-09-09T23:37:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1078.433575 secs 2023-09-09T23:37:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:27 smithi098 ceph-mon[103124]: pgmap v745: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 21 KiB/s wr, 14 op/s 2023-09-09T23:37:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1078.433575 secs 2023-09-09T23:37:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1080.032637 secs 2023-09-09T23:37:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1080.032637 secs 2023-09-09T23:37:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1080.032637 secs 2023-09-09T23:37:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1080.761738 secs 2023-09-09T23:37:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:29 smithi170 ceph-mon[102832]: pgmap v746: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 9 op/s 2023-09-09T23:37:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1080.761738 secs 2023-09-09T23:37:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:29 smithi082 ceph-mon[98694]: pgmap v746: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 9 op/s 2023-09-09T23:37:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1080.761738 secs 2023-09-09T23:37:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:29 smithi098 ceph-mon[103124]: pgmap v746: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 9 op/s 2023-09-09T23:37:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:30 smithi098 ceph-mon[103124]: pgmap v747: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 12 op/s 2023-09-09T23:37:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:30 smithi170 ceph-mon[102832]: pgmap v747: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 12 op/s 2023-09-09T23:37:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:30 smithi082 ceph-mon[98694]: pgmap v747: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 12 op/s 2023-09-09T23:37:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:37:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:37:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:37:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1083.433634 secs 2023-09-09T23:37:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:37:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:37:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:33 smithi170 ceph-mon[102832]: pgmap v748: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-09T23:37:33.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:37:33.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:37:33.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1085.032699 secs 2023-09-09T23:37:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1083.433634 secs 2023-09-09T23:37:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:37:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:37:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:33 smithi082 ceph-mon[98694]: pgmap v748: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-09T23:37:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:37:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:37:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1085.032699 secs 2023-09-09T23:37:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1083.433634 secs 2023-09-09T23:37:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:37:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:37:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:33 smithi098 ceph-mon[103124]: pgmap v748: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-09T23:37:33.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:37:33.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:37:33.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1085.032699 secs 2023-09-09T23:37:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:37:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:37:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:37:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:37:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:37:34.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:37:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:37:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:37:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:37:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1085.761807 secs 2023-09-09T23:37:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:35 smithi170 ceph-mon[102832]: pgmap v749: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2023-09-09T23:37:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1085.761807 secs 2023-09-09T23:37:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:35 smithi082 ceph-mon[98694]: pgmap v749: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2023-09-09T23:37:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1085.761807 secs 2023-09-09T23:37:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:35 smithi098 ceph-mon[103124]: pgmap v749: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2023-09-09T23:37:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:37 smithi170 ceph-mon[102832]: pgmap v750: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2023-09-09T23:37:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1088.433696 secs 2023-09-09T23:37:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:37 smithi082 ceph-mon[98694]: pgmap v750: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2023-09-09T23:37:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1088.433696 secs 2023-09-09T23:37:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:37 smithi098 ceph-mon[103124]: pgmap v750: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2023-09-09T23:37:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1088.433696 secs 2023-09-09T23:37:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1090.032756 secs 2023-09-09T23:37:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1090.032756 secs 2023-09-09T23:37:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1090.032756 secs 2023-09-09T23:37:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1090.761871 secs 2023-09-09T23:37:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:39 smithi170 ceph-mon[102832]: pgmap v751: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2023-09-09T23:37:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1090.761871 secs 2023-09-09T23:37:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:39 smithi082 ceph-mon[98694]: pgmap v751: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2023-09-09T23:37:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1090.761871 secs 2023-09-09T23:37:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:39 smithi098 ceph-mon[103124]: pgmap v751: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2023-09-09T23:37:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:40 smithi170 ceph-mon[102832]: pgmap v752: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2023-09-09T23:37:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:40 smithi082 ceph-mon[98694]: pgmap v752: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2023-09-09T23:37:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:40 smithi098 ceph-mon[103124]: pgmap v752: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2023-09-09T23:37:42.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1093.433816 secs 2023-09-09T23:37:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1093.433816 secs 2023-09-09T23:37:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1093.433816 secs 2023-09-09T23:37:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:43 smithi170 ceph-mon[102832]: pgmap v753: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.5 KiB/s wr, 9 op/s 2023-09-09T23:37:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1095.032815 secs 2023-09-09T23:37:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:43 smithi082 ceph-mon[98694]: pgmap v753: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.5 KiB/s wr, 9 op/s 2023-09-09T23:37:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1095.032815 secs 2023-09-09T23:37:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:43 smithi098 ceph-mon[103124]: pgmap v753: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.5 KiB/s wr, 9 op/s 2023-09-09T23:37:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1095.032815 secs 2023-09-09T23:37:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1095.761935 secs 2023-09-09T23:37:45.054 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:44 smithi082 ceph-mon[98694]: pgmap v754: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:37:45.055 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1095.761935 secs 2023-09-09T23:37:45.056 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:44 smithi098 ceph-mon[103124]: pgmap v754: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:37:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1095.761935 secs 2023-09-09T23:37:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:44 smithi170 ceph-mon[102832]: pgmap v754: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:37:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:47 smithi170 ceph-mon[102832]: pgmap v755: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2023-09-09T23:37:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1098.433878 secs 2023-09-09T23:37:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:47 smithi082 ceph-mon[98694]: pgmap v755: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2023-09-09T23:37:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1098.433878 secs 2023-09-09T23:37:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:47 smithi098 ceph-mon[103124]: pgmap v755: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2023-09-09T23:37:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1098.433878 secs 2023-09-09T23:37:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1100.032873 secs 2023-09-09T23:37:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1100.032873 secs 2023-09-09T23:37:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1100.032873 secs 2023-09-09T23:37:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1100.762002 secs 2023-09-09T23:37:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:49 smithi170 ceph-mon[102832]: pgmap v756: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2023-09-09T23:37:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1100.762002 secs 2023-09-09T23:37:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:49 smithi082 ceph-mon[98694]: pgmap v756: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2023-09-09T23:37:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1100.762002 secs 2023-09-09T23:37:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:49 smithi098 ceph-mon[103124]: pgmap v756: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2023-09-09T23:37:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:50 smithi170 ceph-mon[102832]: pgmap v757: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2023-09-09T23:37:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:50 smithi082 ceph-mon[98694]: pgmap v757: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2023-09-09T23:37:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:50 smithi098 ceph-mon[103124]: pgmap v757: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2023-09-09T23:37:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1103.433989 secs 2023-09-09T23:37:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1103.433989 secs 2023-09-09T23:37:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1103.433989 secs 2023-09-09T23:37:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:53 smithi170 ceph-mon[102832]: pgmap v758: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-09T23:37:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1105.032931 secs 2023-09-09T23:37:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:53 smithi082 ceph-mon[98694]: pgmap v758: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-09T23:37:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1105.032931 secs 2023-09-09T23:37:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:53 smithi098 ceph-mon[103124]: pgmap v758: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-09T23:37:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1105.032931 secs 2023-09-09T23:37:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1105.762068 secs 2023-09-09T23:37:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:54 smithi082 ceph-mon[98694]: pgmap v759: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-09T23:37:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1105.762068 secs 2023-09-09T23:37:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:54 smithi098 ceph-mon[103124]: pgmap v759: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-09T23:37:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1105.762068 secs 2023-09-09T23:37:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:54 smithi170 ceph-mon[102832]: pgmap v759: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-09T23:37:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:57 smithi170 ceph-mon[102832]: pgmap v760: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 14 op/s 2023-09-09T23:37:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1108.434048 secs 2023-09-09T23:37:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:57 smithi082 ceph-mon[98694]: pgmap v760: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 14 op/s 2023-09-09T23:37:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1108.434048 secs 2023-09-09T23:37:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:57 smithi098 ceph-mon[103124]: pgmap v760: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 14 op/s 2023-09-09T23:37:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1108.434048 secs 2023-09-09T23:37:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1110.033004 secs 2023-09-09T23:37:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1110.033004 secs 2023-09-09T23:37:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1110.033004 secs 2023-09-09T23:37:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1110.762134 secs 2023-09-09T23:37:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:37:59 smithi170 ceph-mon[102832]: pgmap v761: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.6 KiB/s wr, 9 op/s 2023-09-09T23:37:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1110.762134 secs 2023-09-09T23:37:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:37:59 smithi082 ceph-mon[98694]: pgmap v761: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.6 KiB/s wr, 9 op/s 2023-09-09T23:38:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1110.762134 secs 2023-09-09T23:38:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:37:59 smithi098 ceph-mon[103124]: pgmap v761: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.6 KiB/s wr, 9 op/s 2023-09-09T23:38:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:00 smithi170 ceph-mon[102832]: pgmap v762: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-09-09T23:38:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:00 smithi082 ceph-mon[98694]: pgmap v762: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-09-09T23:38:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:00 smithi098 ceph-mon[103124]: pgmap v762: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-09-09T23:38:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1113.434109 secs 2023-09-09T23:38:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1113.434109 secs 2023-09-09T23:38:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1113.434109 secs 2023-09-09T23:38:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:03 smithi170 ceph-mon[102832]: pgmap v763: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-09-09T23:38:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1115.033063 secs 2023-09-09T23:38:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:03 smithi082 ceph-mon[98694]: pgmap v763: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-09-09T23:38:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1115.033063 secs 2023-09-09T23:38:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:03 smithi098 ceph-mon[103124]: pgmap v763: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 53 KiB/s wr, 24 op/s 2023-09-09T23:38:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1115.033063 secs 2023-09-09T23:38:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1115.762199 secs 2023-09-09T23:38:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:04 smithi082 ceph-mon[98694]: pgmap v764: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-09-09T23:38:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1115.762199 secs 2023-09-09T23:38:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:04 smithi098 ceph-mon[103124]: pgmap v764: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-09-09T23:38:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1115.762199 secs 2023-09-09T23:38:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:04 smithi170 ceph-mon[102832]: pgmap v764: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-09-09T23:38:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:07 smithi170 ceph-mon[102832]: pgmap v765: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 45 KiB/s wr, 27 op/s 2023-09-09T23:38:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1118.434169 secs 2023-09-09T23:38:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:07 smithi082 ceph-mon[98694]: pgmap v765: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 45 KiB/s wr, 27 op/s 2023-09-09T23:38:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1118.434169 secs 2023-09-09T23:38:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:07 smithi098 ceph-mon[103124]: pgmap v765: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 45 KiB/s wr, 27 op/s 2023-09-09T23:38:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1118.434169 secs 2023-09-09T23:38:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:08 smithi170 ceph-mon[102832]: osdmap e121: 12 total, 12 up, 12 in 2023-09-09T23:38:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1120.033125 secs 2023-09-09T23:38:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:08 smithi170 ceph-mon[102832]: pgmap v767: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:38:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:08 smithi082 ceph-mon[98694]: osdmap e121: 12 total, 12 up, 12 in 2023-09-09T23:38:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1120.033125 secs 2023-09-09T23:38:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:08 smithi082 ceph-mon[98694]: pgmap v767: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:38:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:08 smithi098 ceph-mon[103124]: osdmap e121: 12 total, 12 up, 12 in 2023-09-09T23:38:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1120.033125 secs 2023-09-09T23:38:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:08 smithi098 ceph-mon[103124]: pgmap v767: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:38:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1120.762262 secs 2023-09-09T23:38:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1120.762262 secs 2023-09-09T23:38:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1120.762262 secs 2023-09-09T23:38:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:10 smithi082 ceph-mon[98694]: pgmap v768: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 25 op/s 2023-09-09T23:38:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:10 smithi098 ceph-mon[103124]: pgmap v768: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 25 op/s 2023-09-09T23:38:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:10 smithi170 ceph-mon[102832]: pgmap v768: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 25 op/s 2023-09-09T23:38:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1123.434234 secs 2023-09-09T23:38:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1123.434234 secs 2023-09-09T23:38:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1123.434234 secs 2023-09-09T23:38:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:13 smithi170 ceph-mon[102832]: pgmap v769: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-09T23:38:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1125.033209 secs 2023-09-09T23:38:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:13 smithi082 ceph-mon[98694]: pgmap v769: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-09T23:38:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1125.033209 secs 2023-09-09T23:38:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:13 smithi098 ceph-mon[103124]: pgmap v769: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-09T23:38:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1125.033209 secs 2023-09-09T23:38:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1125.762326 secs 2023-09-09T23:38:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:14 smithi082 ceph-mon[98694]: osdmap e122: 12 total, 12 up, 12 in 2023-09-09T23:38:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:14 smithi082 ceph-mon[98694]: pgmap v771: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-09T23:38:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1125.762326 secs 2023-09-09T23:38:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:14 smithi098 ceph-mon[103124]: osdmap e122: 12 total, 12 up, 12 in 2023-09-09T23:38:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:14 smithi098 ceph-mon[103124]: pgmap v771: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-09T23:38:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1125.762326 secs 2023-09-09T23:38:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:14 smithi170 ceph-mon[102832]: osdmap e122: 12 total, 12 up, 12 in 2023-09-09T23:38:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:14 smithi170 ceph-mon[102832]: pgmap v771: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-09T23:38:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:17 smithi170 ceph-mon[102832]: pgmap v772: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2023-09-09T23:38:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1128.434294 secs 2023-09-09T23:38:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:17 smithi082 ceph-mon[98694]: pgmap v772: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2023-09-09T23:38:17.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1128.434294 secs 2023-09-09T23:38:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:17 smithi098 ceph-mon[103124]: pgmap v772: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2023-09-09T23:38:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1128.434294 secs 2023-09-09T23:38:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1130.033270 secs 2023-09-09T23:38:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:18 smithi170 ceph-mon[102832]: pgmap v773: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.2 KiB/s wr, 11 op/s 2023-09-09T23:38:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1130.033270 secs 2023-09-09T23:38:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:18 smithi082 ceph-mon[98694]: pgmap v773: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.2 KiB/s wr, 11 op/s 2023-09-09T23:38:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1130.033270 secs 2023-09-09T23:38:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:18 smithi098 ceph-mon[103124]: pgmap v773: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.2 KiB/s wr, 11 op/s 2023-09-09T23:38:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1130.762392 secs 2023-09-09T23:38:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1130.762392 secs 2023-09-09T23:38:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1130.762392 secs 2023-09-09T23:38:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:20 smithi082 ceph-mon[98694]: pgmap v774: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 12 op/s 2023-09-09T23:38:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:20 smithi098 ceph-mon[103124]: pgmap v774: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 12 op/s 2023-09-09T23:38:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:20 smithi170 ceph-mon[102832]: pgmap v774: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 12 op/s 2023-09-09T23:38:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1133.434374 secs 2023-09-09T23:38:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1133.434374 secs 2023-09-09T23:38:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1133.434374 secs 2023-09-09T23:38:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:23 smithi170 ceph-mon[102832]: pgmap v775: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-09T23:38:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1135.033354 secs 2023-09-09T23:38:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:23 smithi082 ceph-mon[98694]: pgmap v775: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-09T23:38:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1135.033354 secs 2023-09-09T23:38:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:23 smithi098 ceph-mon[103124]: pgmap v775: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-09T23:38:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1135.033354 secs 2023-09-09T23:38:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1135.762460 secs 2023-09-09T23:38:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:24 smithi082 ceph-mon[98694]: pgmap v776: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-09T23:38:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1135.762460 secs 2023-09-09T23:38:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:24 smithi098 ceph-mon[103124]: pgmap v776: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-09T23:38:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1135.762460 secs 2023-09-09T23:38:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:24 smithi170 ceph-mon[102832]: pgmap v776: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-09T23:38:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:26 smithi098 ceph-mon[103124]: pgmap v777: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 KiB/s wr, 15 op/s 2023-09-09T23:38:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:26 smithi170 ceph-mon[102832]: pgmap v777: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 KiB/s wr, 15 op/s 2023-09-09T23:38:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:26 smithi082 ceph-mon[98694]: pgmap v777: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 KiB/s wr, 15 op/s 2023-09-09T23:38:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1138.434435 secs 2023-09-09T23:38:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1138.434435 secs 2023-09-09T23:38:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1138.434435 secs 2023-09-09T23:38:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1140.033414 secs 2023-09-09T23:38:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:28 smithi098 ceph-mon[103124]: pgmap v778: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-09T23:38:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1140.033414 secs 2023-09-09T23:38:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:28 smithi170 ceph-mon[102832]: pgmap v778: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-09T23:38:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1140.033414 secs 2023-09-09T23:38:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:28 smithi082 ceph-mon[98694]: pgmap v778: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-09T23:38:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1140.762489 secs 2023-09-09T23:38:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1140.762489 secs 2023-09-09T23:38:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1140.762489 secs 2023-09-09T23:38:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:30 smithi098 ceph-mon[103124]: pgmap v779: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 13 op/s 2023-09-09T23:38:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:30 smithi170 ceph-mon[102832]: pgmap v779: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 13 op/s 2023-09-09T23:38:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:30 smithi082 ceph-mon[98694]: pgmap v779: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 13 op/s 2023-09-09T23:38:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1143.434513 secs 2023-09-09T23:38:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1143.434513 secs 2023-09-09T23:38:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1143.434513 secs 2023-09-09T23:38:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:33 smithi170 ceph-mon[102832]: pgmap v780: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-09-09T23:38:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1145.033472 secs 2023-09-09T23:38:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:33 smithi082 ceph-mon[98694]: pgmap v780: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-09-09T23:38:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1145.033472 secs 2023-09-09T23:38:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:33 smithi098 ceph-mon[103124]: pgmap v780: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-09-09T23:38:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1145.033472 secs 2023-09-09T23:38:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:38:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1145.762557 secs 2023-09-09T23:38:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:38:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1145.762557 secs 2023-09-09T23:38:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:38:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1145.762557 secs 2023-09-09T23:38:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:38:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:38:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:38:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:35 smithi170 ceph-mon[102832]: pgmap v781: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 KiB/s wr, 9 op/s 2023-09-09T23:38:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:38:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:38:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:38:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:35 smithi082 ceph-mon[98694]: pgmap v781: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 KiB/s wr, 9 op/s 2023-09-09T23:38:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:38:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:38:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:38:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:35 smithi098 ceph-mon[103124]: pgmap v781: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 KiB/s wr, 9 op/s 2023-09-09T23:38:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:36 smithi098 ceph-mon[103124]: pgmap v782: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 13 KiB/s wr, 13 op/s 2023-09-09T23:38:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:36 smithi170 ceph-mon[102832]: pgmap v782: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 13 KiB/s wr, 13 op/s 2023-09-09T23:38:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:36 smithi082 ceph-mon[98694]: pgmap v782: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 13 KiB/s wr, 13 op/s 2023-09-09T23:38:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1148.434572 secs 2023-09-09T23:38:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1148.434572 secs 2023-09-09T23:38:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1148.434572 secs 2023-09-09T23:38:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1150.033496 secs 2023-09-09T23:38:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:38 smithi098 ceph-mon[103124]: pgmap v783: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 8 op/s 2023-09-09T23:38:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1150.033496 secs 2023-09-09T23:38:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:38 smithi170 ceph-mon[102832]: pgmap v783: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 8 op/s 2023-09-09T23:38:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1150.033496 secs 2023-09-09T23:38:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:38 smithi082 ceph-mon[98694]: pgmap v783: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 8 op/s 2023-09-09T23:38:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1150.762623 secs 2023-09-09T23:38:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1150.762623 secs 2023-09-09T23:38:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1150.762623 secs 2023-09-09T23:38:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:40 smithi098 ceph-mon[103124]: pgmap v784: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 11 op/s 2023-09-09T23:38:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:40 smithi170 ceph-mon[102832]: pgmap v784: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 11 op/s 2023-09-09T23:38:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:40 smithi082 ceph-mon[98694]: pgmap v784: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 11 op/s 2023-09-09T23:38:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1153.434632 secs 2023-09-09T23:38:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1153.434632 secs 2023-09-09T23:38:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1153.434632 secs 2023-09-09T23:38:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:43 smithi170 ceph-mon[102832]: pgmap v785: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 9 op/s 2023-09-09T23:38:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1155.033568 secs 2023-09-09T23:38:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:43 smithi082 ceph-mon[98694]: pgmap v785: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 9 op/s 2023-09-09T23:38:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1155.033568 secs 2023-09-09T23:38:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:43 smithi098 ceph-mon[103124]: pgmap v785: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 9 op/s 2023-09-09T23:38:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1155.033568 secs 2023-09-09T23:38:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1155.762688 secs 2023-09-09T23:38:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:44 smithi082 ceph-mon[98694]: pgmap v786: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:38:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1155.762688 secs 2023-09-09T23:38:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:44 smithi098 ceph-mon[103124]: pgmap v786: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:38:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1155.762688 secs 2023-09-09T23:38:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:44 smithi170 ceph-mon[102832]: pgmap v786: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:38:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:46 smithi098 ceph-mon[103124]: pgmap v787: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 12 op/s 2023-09-09T23:38:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:46 smithi170 ceph-mon[102832]: pgmap v787: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 12 op/s 2023-09-09T23:38:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:46 smithi082 ceph-mon[98694]: pgmap v787: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 12 op/s 2023-09-09T23:38:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1158.434750 secs 2023-09-09T23:38:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1158.434750 secs 2023-09-09T23:38:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1158.434750 secs 2023-09-09T23:38:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1160.033627 secs 2023-09-09T23:38:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:48 smithi098 ceph-mon[103124]: pgmap v788: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 8 op/s 2023-09-09T23:38:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1160.033627 secs 2023-09-09T23:38:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:48 smithi170 ceph-mon[102832]: pgmap v788: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 8 op/s 2023-09-09T23:38:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1160.033627 secs 2023-09-09T23:38:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:48 smithi082 ceph-mon[98694]: pgmap v788: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 8 op/s 2023-09-09T23:38:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1160.762755 secs 2023-09-09T23:38:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1160.762755 secs 2023-09-09T23:38:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1160.762755 secs 2023-09-09T23:38:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:50 smithi098 ceph-mon[103124]: pgmap v789: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2023-09-09T23:38:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:50 smithi170 ceph-mon[102832]: pgmap v789: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2023-09-09T23:38:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:50 smithi082 ceph-mon[98694]: pgmap v789: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2023-09-09T23:38:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1163.434847 secs 2023-09-09T23:38:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1163.434847 secs 2023-09-09T23:38:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1163.434847 secs 2023-09-09T23:38:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:53 smithi170 ceph-mon[102832]: pgmap v790: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2023-09-09T23:38:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1165.033710 secs 2023-09-09T23:38:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:53 smithi082 ceph-mon[98694]: pgmap v790: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2023-09-09T23:38:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1165.033710 secs 2023-09-09T23:38:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:53 smithi098 ceph-mon[103124]: pgmap v790: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2023-09-09T23:38:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1165.033710 secs 2023-09-09T23:38:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1165.762844 secs 2023-09-09T23:38:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:54 smithi082 ceph-mon[98694]: pgmap v791: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2023-09-09T23:38:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1165.762844 secs 2023-09-09T23:38:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:54 smithi098 ceph-mon[103124]: pgmap v791: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2023-09-09T23:38:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1165.762844 secs 2023-09-09T23:38:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:54 smithi170 ceph-mon[102832]: pgmap v791: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2023-09-09T23:38:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:56 smithi098 ceph-mon[103124]: pgmap v792: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 14 KiB/s wr, 14 op/s 2023-09-09T23:38:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:56 smithi170 ceph-mon[102832]: pgmap v792: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 14 KiB/s wr, 14 op/s 2023-09-09T23:38:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:56 smithi082 ceph-mon[98694]: pgmap v792: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 14 KiB/s wr, 14 op/s 2023-09-09T23:38:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1168.434946 secs 2023-09-09T23:38:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1168.434946 secs 2023-09-09T23:38:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1168.434946 secs 2023-09-09T23:38:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1170.033771 secs 2023-09-09T23:38:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:58 smithi098 ceph-mon[103124]: pgmap v793: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 13 KiB/s wr, 10 op/s 2023-09-09T23:38:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1170.033771 secs 2023-09-09T23:38:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:58 smithi170 ceph-mon[102832]: pgmap v793: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 13 KiB/s wr, 10 op/s 2023-09-09T23:38:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1170.033771 secs 2023-09-09T23:38:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:58 smithi082 ceph-mon[98694]: pgmap v793: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 13 KiB/s wr, 10 op/s 2023-09-09T23:39:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:38:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1170.762911 secs 2023-09-09T23:39:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:38:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1170.762911 secs 2023-09-09T23:39:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:38:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1170.762911 secs 2023-09-09T23:39:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:00 smithi098 ceph-mon[103124]: pgmap v794: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 35 KiB/s wr, 15 op/s 2023-09-09T23:39:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:00 smithi170 ceph-mon[102832]: pgmap v794: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 35 KiB/s wr, 15 op/s 2023-09-09T23:39:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:00 smithi082 ceph-mon[98694]: pgmap v794: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 35 KiB/s wr, 15 op/s 2023-09-09T23:39:02.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1173.435004 secs 2023-09-09T23:39:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1173.435004 secs 2023-09-09T23:39:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1173.435004 secs 2023-09-09T23:39:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:03 smithi170 ceph-mon[102832]: pgmap v795: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 54 KiB/s wr, 25 op/s 2023-09-09T23:39:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1175.033835 secs 2023-09-09T23:39:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:03 smithi082 ceph-mon[98694]: pgmap v795: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 54 KiB/s wr, 25 op/s 2023-09-09T23:39:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1175.033835 secs 2023-09-09T23:39:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:03 smithi098 ceph-mon[103124]: pgmap v795: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 54 KiB/s wr, 25 op/s 2023-09-09T23:39:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1175.033835 secs 2023-09-09T23:39:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1175.762975 secs 2023-09-09T23:39:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:04 smithi082 ceph-mon[98694]: pgmap v796: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-09-09T23:39:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1175.762975 secs 2023-09-09T23:39:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:04 smithi098 ceph-mon[103124]: pgmap v796: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-09-09T23:39:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1175.762975 secs 2023-09-09T23:39:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:04 smithi170 ceph-mon[102832]: pgmap v796: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-09-09T23:39:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:06 smithi098 ceph-mon[103124]: pgmap v797: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:39:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:06 smithi170 ceph-mon[102832]: pgmap v797: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:39:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:06 smithi082 ceph-mon[98694]: pgmap v797: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:39:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1178.435064 secs 2023-09-09T23:39:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1178.435064 secs 2023-09-09T23:39:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1178.435064 secs 2023-09-09T23:39:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:08 smithi098 ceph-mon[103124]: osdmap e123: 12 total, 12 up, 12 in 2023-09-09T23:39:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1180.033897 secs 2023-09-09T23:39:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:08 smithi098 ceph-mon[103124]: pgmap v799: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:39:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:08 smithi170 ceph-mon[102832]: osdmap e123: 12 total, 12 up, 12 in 2023-09-09T23:39:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1180.033897 secs 2023-09-09T23:39:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:08 smithi170 ceph-mon[102832]: pgmap v799: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:39:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:08 smithi082 ceph-mon[98694]: osdmap e123: 12 total, 12 up, 12 in 2023-09-09T23:39:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1180.033897 secs 2023-09-09T23:39:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:08 smithi082 ceph-mon[98694]: pgmap v799: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:39:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1180.763041 secs 2023-09-09T23:39:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1180.763041 secs 2023-09-09T23:39:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1180.763041 secs 2023-09-09T23:39:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:10 smithi098 ceph-mon[103124]: pgmap v800: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 KiB/s wr, 25 op/s 2023-09-09T23:39:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:10 smithi170 ceph-mon[102832]: pgmap v800: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 KiB/s wr, 25 op/s 2023-09-09T23:39:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:10 smithi082 ceph-mon[98694]: pgmap v800: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 KiB/s wr, 25 op/s 2023-09-09T23:39:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1183.435122 secs 2023-09-09T23:39:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1183.435122 secs 2023-09-09T23:39:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1183.435122 secs 2023-09-09T23:39:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:13 smithi170 ceph-mon[102832]: pgmap v801: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-09T23:39:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1185.033980 secs 2023-09-09T23:39:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:13 smithi082 ceph-mon[98694]: pgmap v801: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-09T23:39:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1185.033980 secs 2023-09-09T23:39:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:13 smithi098 ceph-mon[103124]: pgmap v801: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-09T23:39:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1185.033980 secs 2023-09-09T23:39:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1185.763107 secs 2023-09-09T23:39:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:14 smithi082 ceph-mon[98694]: osdmap e124: 12 total, 12 up, 12 in 2023-09-09T23:39:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:14 smithi082 ceph-mon[98694]: pgmap v803: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:39:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1185.763107 secs 2023-09-09T23:39:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:14 smithi098 ceph-mon[103124]: osdmap e124: 12 total, 12 up, 12 in 2023-09-09T23:39:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:14 smithi098 ceph-mon[103124]: pgmap v803: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:39:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1185.763107 secs 2023-09-09T23:39:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:14 smithi170 ceph-mon[102832]: osdmap e124: 12 total, 12 up, 12 in 2023-09-09T23:39:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:14 smithi170 ceph-mon[102832]: pgmap v803: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:39:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:16 smithi098 ceph-mon[103124]: pgmap v804: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 19 KiB/s wr, 13 op/s 2023-09-09T23:39:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:16 smithi170 ceph-mon[102832]: pgmap v804: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 19 KiB/s wr, 13 op/s 2023-09-09T23:39:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:16 smithi082 ceph-mon[98694]: pgmap v804: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 19 KiB/s wr, 13 op/s 2023-09-09T23:39:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1188.435227 secs 2023-09-09T23:39:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1188.435227 secs 2023-09-09T23:39:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1188.435227 secs 2023-09-09T23:39:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1190.034062 secs 2023-09-09T23:39:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:18 smithi098 ceph-mon[103124]: pgmap v805: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 KiB/s wr, 11 op/s 2023-09-09T23:39:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1190.034062 secs 2023-09-09T23:39:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:18 smithi170 ceph-mon[102832]: pgmap v805: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 KiB/s wr, 11 op/s 2023-09-09T23:39:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1190.034062 secs 2023-09-09T23:39:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:18 smithi082 ceph-mon[98694]: pgmap v805: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 KiB/s wr, 11 op/s 2023-09-09T23:39:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1190.763174 secs 2023-09-09T23:39:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1190.763174 secs 2023-09-09T23:39:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1190.763174 secs 2023-09-09T23:39:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:20 smithi098 ceph-mon[103124]: pgmap v806: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-09-09T23:39:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:20 smithi170 ceph-mon[102832]: pgmap v806: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-09-09T23:39:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:20 smithi082 ceph-mon[98694]: pgmap v806: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-09-09T23:39:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1193.435338 secs 2023-09-09T23:39:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1193.435338 secs 2023-09-09T23:39:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1193.435338 secs 2023-09-09T23:39:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:23 smithi170 ceph-mon[102832]: pgmap v807: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 35 KiB/s wr, 13 op/s 2023-09-09T23:39:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1195.034144 secs 2023-09-09T23:39:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:23 smithi082 ceph-mon[98694]: pgmap v807: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 35 KiB/s wr, 13 op/s 2023-09-09T23:39:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1195.034144 secs 2023-09-09T23:39:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:23 smithi098 ceph-mon[103124]: pgmap v807: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 35 KiB/s wr, 13 op/s 2023-09-09T23:39:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1195.034144 secs 2023-09-09T23:39:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1195.763243 secs 2023-09-09T23:39:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:24 smithi098 ceph-mon[103124]: pgmap v808: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-09T23:39:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1195.763243 secs 2023-09-09T23:39:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:24 smithi170 ceph-mon[102832]: pgmap v808: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-09T23:39:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1195.763243 secs 2023-09-09T23:39:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:24 smithi082 ceph-mon[98694]: pgmap v808: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-09T23:39:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:26 smithi098 ceph-mon[103124]: pgmap v809: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 30 KiB/s wr, 15 op/s 2023-09-09T23:39:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:26 smithi170 ceph-mon[102832]: pgmap v809: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 30 KiB/s wr, 15 op/s 2023-09-09T23:39:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:26 smithi082 ceph-mon[98694]: pgmap v809: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 30 KiB/s wr, 15 op/s 2023-09-09T23:39:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1198.435397 secs 2023-09-09T23:39:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1198.435397 secs 2023-09-09T23:39:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1198.435397 secs 2023-09-09T23:39:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1200.034207 secs 2023-09-09T23:39:29.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:28 smithi098 ceph-mon[103124]: pgmap v810: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 9 op/s 2023-09-09T23:39:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1200.034207 secs 2023-09-09T23:39:29.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:28 smithi170 ceph-mon[102832]: pgmap v810: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 9 op/s 2023-09-09T23:39:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1200.034207 secs 2023-09-09T23:39:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:28 smithi082 ceph-mon[98694]: pgmap v810: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 9 op/s 2023-09-09T23:39:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1200.763307 secs 2023-09-09T23:39:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1200.763307 secs 2023-09-09T23:39:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1200.763307 secs 2023-09-09T23:39:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:30 smithi098 ceph-mon[103124]: pgmap v811: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 13 op/s 2023-09-09T23:39:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:30 smithi170 ceph-mon[102832]: pgmap v811: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 13 op/s 2023-09-09T23:39:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:30 smithi082 ceph-mon[98694]: pgmap v811: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 13 op/s 2023-09-09T23:39:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1203.435459 secs 2023-09-09T23:39:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1203.435459 secs 2023-09-09T23:39:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1203.435459 secs 2023-09-09T23:39:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:33 smithi170 ceph-mon[102832]: pgmap v812: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 KiB/s wr, 10 op/s 2023-09-09T23:39:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1205.034266 secs 2023-09-09T23:39:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:33 smithi082 ceph-mon[98694]: pgmap v812: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 KiB/s wr, 10 op/s 2023-09-09T23:39:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1205.034266 secs 2023-09-09T23:39:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:33 smithi098 ceph-mon[103124]: pgmap v812: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 KiB/s wr, 10 op/s 2023-09-09T23:39:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1205.034266 secs 2023-09-09T23:39:35.062 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1205.763374 secs 2023-09-09T23:39:35.062 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:39:35.062 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:34 smithi098 ceph-mon[103124]: pgmap v813: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2023-09-09T23:39:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1205.763374 secs 2023-09-09T23:39:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:39:35.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:34 smithi170 ceph-mon[102832]: pgmap v813: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2023-09-09T23:39:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1205.763374 secs 2023-09-09T23:39:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:39:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:34 smithi082 ceph-mon[98694]: pgmap v813: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2023-09-09T23:39:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:39:36.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:39:36.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:39:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:39:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:39:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:39:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:39:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:39:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:39:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:36 smithi098 ceph-mon[103124]: pgmap v814: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2023-09-09T23:39:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:36 smithi170 ceph-mon[102832]: pgmap v814: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2023-09-09T23:39:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:36 smithi082 ceph-mon[98694]: pgmap v814: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2023-09-09T23:39:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1208.435520 secs 2023-09-09T23:39:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1208.435520 secs 2023-09-09T23:39:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1208.435520 secs 2023-09-09T23:39:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1210.034326 secs 2023-09-09T23:39:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:38 smithi098 ceph-mon[103124]: pgmap v815: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 KiB/s wr, 8 op/s 2023-09-09T23:39:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1210.034326 secs 2023-09-09T23:39:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:38 smithi170 ceph-mon[102832]: pgmap v815: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 KiB/s wr, 8 op/s 2023-09-09T23:39:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1210.034326 secs 2023-09-09T23:39:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:38 smithi082 ceph-mon[98694]: pgmap v815: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 KiB/s wr, 8 op/s 2023-09-09T23:39:40.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1210.763440 secs 2023-09-09T23:39:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1210.763440 secs 2023-09-09T23:39:40.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1210.763440 secs 2023-09-09T23:39:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:40 smithi098 ceph-mon[103124]: pgmap v816: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 KiB/s wr, 11 op/s 2023-09-09T23:39:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:40 smithi170 ceph-mon[102832]: pgmap v816: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 KiB/s wr, 11 op/s 2023-09-09T23:39:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:40 smithi082 ceph-mon[98694]: pgmap v816: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 KiB/s wr, 11 op/s 2023-09-09T23:39:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1213.435581 secs 2023-09-09T23:39:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1213.435581 secs 2023-09-09T23:39:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1213.435581 secs 2023-09-09T23:39:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:43 smithi170 ceph-mon[102832]: pgmap v817: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 9 op/s 2023-09-09T23:39:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1215.034385 secs 2023-09-09T23:39:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:43 smithi082 ceph-mon[98694]: pgmap v817: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 9 op/s 2023-09-09T23:39:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1215.034385 secs 2023-09-09T23:39:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:43 smithi098 ceph-mon[103124]: pgmap v817: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 9 op/s 2023-09-09T23:39:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1215.034385 secs 2023-09-09T23:39:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1215.763488 secs 2023-09-09T23:39:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:44 smithi098 ceph-mon[103124]: pgmap v818: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:39:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1215.763488 secs 2023-09-09T23:39:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:44 smithi170 ceph-mon[102832]: pgmap v818: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:39:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1215.763488 secs 2023-09-09T23:39:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:44 smithi082 ceph-mon[98694]: pgmap v818: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:39:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:46 smithi098 ceph-mon[103124]: pgmap v819: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-09T23:39:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:46 smithi170 ceph-mon[102832]: pgmap v819: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-09T23:39:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:46 smithi082 ceph-mon[98694]: pgmap v819: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-09T23:39:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1218.435642 secs 2023-09-09T23:39:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1218.435642 secs 2023-09-09T23:39:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1218.435642 secs 2023-09-09T23:39:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1220.034447 secs 2023-09-09T23:39:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:48 smithi098 ceph-mon[103124]: pgmap v820: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:39:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1220.034447 secs 2023-09-09T23:39:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:48 smithi170 ceph-mon[102832]: pgmap v820: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:39:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1220.034447 secs 2023-09-09T23:39:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:48 smithi082 ceph-mon[98694]: pgmap v820: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-09T23:39:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1220.763557 secs 2023-09-09T23:39:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1220.763557 secs 2023-09-09T23:39:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1220.763557 secs 2023-09-09T23:39:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:50 smithi098 ceph-mon[103124]: pgmap v821: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.8 KiB/s wr, 12 op/s 2023-09-09T23:39:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:50 smithi170 ceph-mon[102832]: pgmap v821: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.8 KiB/s wr, 12 op/s 2023-09-09T23:39:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:50 smithi082 ceph-mon[98694]: pgmap v821: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.8 KiB/s wr, 12 op/s 2023-09-09T23:39:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1223.435703 secs 2023-09-09T23:39:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1223.435703 secs 2023-09-09T23:39:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1223.435703 secs 2023-09-09T23:39:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:53 smithi170 ceph-mon[102832]: pgmap v822: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.8 KiB/s wr, 11 op/s 2023-09-09T23:39:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1225.034493 secs 2023-09-09T23:39:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:53 smithi082 ceph-mon[98694]: pgmap v822: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.8 KiB/s wr, 11 op/s 2023-09-09T23:39:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1225.034493 secs 2023-09-09T23:39:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:53 smithi098 ceph-mon[103124]: pgmap v822: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.8 KiB/s wr, 11 op/s 2023-09-09T23:39:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1225.034493 secs 2023-09-09T23:39:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1225.763626 secs 2023-09-09T23:39:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:54 smithi098 ceph-mon[103124]: pgmap v823: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2023-09-09T23:39:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1225.763626 secs 2023-09-09T23:39:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:54 smithi170 ceph-mon[102832]: pgmap v823: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2023-09-09T23:39:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1225.763626 secs 2023-09-09T23:39:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:54 smithi082 ceph-mon[98694]: pgmap v823: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2023-09-09T23:39:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:56 smithi098 ceph-mon[103124]: pgmap v824: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 15 KiB/s wr, 14 op/s 2023-09-09T23:39:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:56 smithi170 ceph-mon[102832]: pgmap v824: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 15 KiB/s wr, 14 op/s 2023-09-09T23:39:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:56 smithi082 ceph-mon[98694]: pgmap v824: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 15 KiB/s wr, 14 op/s 2023-09-09T23:39:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1228.435771 secs 2023-09-09T23:39:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1228.435771 secs 2023-09-09T23:39:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1228.435771 secs 2023-09-09T23:39:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1230.034553 secs 2023-09-09T23:39:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:58 smithi098 ceph-mon[103124]: pgmap v825: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-09T23:39:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1230.034553 secs 2023-09-09T23:39:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:58 smithi170 ceph-mon[102832]: pgmap v825: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-09T23:39:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1230.034553 secs 2023-09-09T23:39:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:58 smithi082 ceph-mon[98694]: pgmap v825: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-09T23:40:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:39:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1230.763689 secs 2023-09-09T23:40:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:39:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1230.763689 secs 2023-09-09T23:40:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:39:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1230.763689 secs 2023-09-09T23:40:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-09T23:40:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:00 smithi098 ceph-mon[103124]: pgmap v826: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 37 KiB/s wr, 16 op/s 2023-09-09T23:40:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-09T23:40:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:00 smithi170 ceph-mon[102832]: pgmap v826: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 37 KiB/s wr, 16 op/s 2023-09-09T23:40:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-09T23:40:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:00 smithi082 ceph-mon[98694]: pgmap v826: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 37 KiB/s wr, 16 op/s 2023-09-09T23:40:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1233.435851 secs 2023-09-09T23:40:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1233.435851 secs 2023-09-09T23:40:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1233.435851 secs 2023-09-09T23:40:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:03 smithi170 ceph-mon[102832]: pgmap v827: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 54 KiB/s wr, 25 op/s 2023-09-09T23:40:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1235.034636 secs 2023-09-09T23:40:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:03 smithi082 ceph-mon[98694]: pgmap v827: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 54 KiB/s wr, 25 op/s 2023-09-09T23:40:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1235.034636 secs 2023-09-09T23:40:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:03 smithi098 ceph-mon[103124]: pgmap v827: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 54 KiB/s wr, 25 op/s 2023-09-09T23:40:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1235.034636 secs 2023-09-09T23:40:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1235.763757 secs 2023-09-09T23:40:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:04 smithi098 ceph-mon[103124]: pgmap v828: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 54 KiB/s wr, 24 op/s 2023-09-09T23:40:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1235.763757 secs 2023-09-09T23:40:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:04 smithi170 ceph-mon[102832]: pgmap v828: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 54 KiB/s wr, 24 op/s 2023-09-09T23:40:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1235.763757 secs 2023-09-09T23:40:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:04 smithi082 ceph-mon[98694]: pgmap v828: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 54 KiB/s wr, 24 op/s 2023-09-09T23:40:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:06 smithi098 ceph-mon[103124]: pgmap v829: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 55 KiB/s wr, 29 op/s 2023-09-09T23:40:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:06 smithi170 ceph-mon[102832]: pgmap v829: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 55 KiB/s wr, 29 op/s 2023-09-09T23:40:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:06 smithi082 ceph-mon[98694]: pgmap v829: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 55 KiB/s wr, 29 op/s 2023-09-09T23:40:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1238.435913 secs 2023-09-09T23:40:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1238.435913 secs 2023-09-09T23:40:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1238.435913 secs 2023-09-09T23:40:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:08 smithi098 ceph-mon[103124]: osdmap e125: 12 total, 12 up, 12 in 2023-09-09T23:40:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1240.034695 secs 2023-09-09T23:40:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:08 smithi098 ceph-mon[103124]: pgmap v831: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-09-09T23:40:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:08 smithi170 ceph-mon[102832]: osdmap e125: 12 total, 12 up, 12 in 2023-09-09T23:40:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1240.034695 secs 2023-09-09T23:40:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:08 smithi170 ceph-mon[102832]: pgmap v831: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-09-09T23:40:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:08 smithi082 ceph-mon[98694]: osdmap e125: 12 total, 12 up, 12 in 2023-09-09T23:40:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1240.034695 secs 2023-09-09T23:40:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:08 smithi082 ceph-mon[98694]: pgmap v831: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-09-09T23:40:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1240.763819 secs 2023-09-09T23:40:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1240.763819 secs 2023-09-09T23:40:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1240.763819 secs 2023-09-09T23:40:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:10 smithi170 ceph-mon[102832]: pgmap v832: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 KiB/s wr, 26 op/s 2023-09-09T23:40:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:10 smithi082 ceph-mon[98694]: pgmap v832: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 KiB/s wr, 26 op/s 2023-09-09T23:40:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:10 smithi098 ceph-mon[103124]: pgmap v832: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 KiB/s wr, 26 op/s 2023-09-09T23:40:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1243.435975 secs 2023-09-09T23:40:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1243.435975 secs 2023-09-09T23:40:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1243.435975 secs 2023-09-09T23:40:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:13 smithi170 ceph-mon[102832]: pgmap v833: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-09T23:40:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1245.034757 secs 2023-09-09T23:40:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:13 smithi082 ceph-mon[98694]: pgmap v833: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-09T23:40:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1245.034757 secs 2023-09-09T23:40:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:13 smithi098 ceph-mon[103124]: pgmap v833: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-09T23:40:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1245.034757 secs 2023-09-09T23:40:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1245.763888 secs 2023-09-09T23:40:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:14 smithi098 ceph-mon[103124]: osdmap e126: 12 total, 12 up, 12 in 2023-09-09T23:40:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:14 smithi098 ceph-mon[103124]: pgmap v835: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-09T23:40:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1245.763888 secs 2023-09-09T23:40:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:14 smithi170 ceph-mon[102832]: osdmap e126: 12 total, 12 up, 12 in 2023-09-09T23:40:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:14 smithi170 ceph-mon[102832]: pgmap v835: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-09T23:40:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1245.763888 secs 2023-09-09T23:40:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:14 smithi082 ceph-mon[98694]: osdmap e126: 12 total, 12 up, 12 in 2023-09-09T23:40:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:14 smithi082 ceph-mon[98694]: pgmap v835: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-09T23:40:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:16 smithi098 ceph-mon[103124]: pgmap v836: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2023-09-09T23:40:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:16 smithi170 ceph-mon[102832]: pgmap v836: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2023-09-09T23:40:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:16 smithi082 ceph-mon[98694]: pgmap v836: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2023-09-09T23:40:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1248.436036 secs 2023-09-09T23:40:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1248.436036 secs 2023-09-09T23:40:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1248.436036 secs 2023-09-09T23:40:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1250.034817 secs 2023-09-09T23:40:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:18 smithi098 ceph-mon[103124]: pgmap v837: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.1 KiB/s wr, 13 op/s 2023-09-09T23:40:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1250.034817 secs 2023-09-09T23:40:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:18 smithi170 ceph-mon[102832]: pgmap v837: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.1 KiB/s wr, 13 op/s 2023-09-09T23:40:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1250.034817 secs 2023-09-09T23:40:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:18 smithi082 ceph-mon[98694]: pgmap v837: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.1 KiB/s wr, 13 op/s 2023-09-09T23:40:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1250.763955 secs 2023-09-09T23:40:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1250.763955 secs 2023-09-09T23:40:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1250.763955 secs 2023-09-09T23:40:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:20 smithi098 ceph-mon[103124]: pgmap v838: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 KiB/s wr, 13 op/s 2023-09-09T23:40:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:20 smithi170 ceph-mon[102832]: pgmap v838: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 KiB/s wr, 13 op/s 2023-09-09T23:40:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:20 smithi082 ceph-mon[98694]: pgmap v838: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 KiB/s wr, 13 op/s 2023-09-09T23:40:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1253.436097 secs 2023-09-09T23:40:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1253.436097 secs 2023-09-09T23:40:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1253.436097 secs 2023-09-09T23:40:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:23 smithi170 ceph-mon[102832]: pgmap v839: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 13 op/s 2023-09-09T23:40:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1255.034877 secs 2023-09-09T23:40:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:23 smithi082 ceph-mon[98694]: pgmap v839: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 13 op/s 2023-09-09T23:40:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1255.034877 secs 2023-09-09T23:40:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:23 smithi098 ceph-mon[103124]: pgmap v839: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 13 op/s 2023-09-09T23:40:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1255.034877 secs 2023-09-09T23:40:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1255.764023 secs 2023-09-09T23:40:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:24 smithi098 ceph-mon[103124]: pgmap v840: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 19 KiB/s wr, 11 op/s 2023-09-09T23:40:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1255.764023 secs 2023-09-09T23:40:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:24 smithi170 ceph-mon[102832]: pgmap v840: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 19 KiB/s wr, 11 op/s 2023-09-09T23:40:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1255.764023 secs 2023-09-09T23:40:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:24 smithi082 ceph-mon[98694]: pgmap v840: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 19 KiB/s wr, 11 op/s 2023-09-09T23:40:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:26 smithi098 ceph-mon[103124]: pgmap v841: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 KiB/s wr, 15 op/s 2023-09-09T23:40:27.047 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:26 smithi082 ceph-mon[98694]: pgmap v841: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 KiB/s wr, 15 op/s 2023-09-09T23:40:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:26 smithi170 ceph-mon[102832]: pgmap v841: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 KiB/s wr, 15 op/s 2023-09-09T23:40:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1258.436178 secs 2023-09-09T23:40:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1258.436178 secs 2023-09-09T23:40:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1258.436178 secs 2023-09-09T23:40:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1260.034936 secs 2023-09-09T23:40:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:28 smithi098 ceph-mon[103124]: pgmap v842: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 9 op/s 2023-09-09T23:40:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1260.034936 secs 2023-09-09T23:40:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:28 smithi170 ceph-mon[102832]: pgmap v842: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 9 op/s 2023-09-09T23:40:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1260.034936 secs 2023-09-09T23:40:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:28 smithi082 ceph-mon[98694]: pgmap v842: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 9 op/s 2023-09-09T23:40:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1260.764088 secs 2023-09-09T23:40:30.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1260.764088 secs 2023-09-09T23:40:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1260.764088 secs 2023-09-09T23:40:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:30 smithi082 ceph-mon[98694]: pgmap v843: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 12 op/s 2023-09-09T23:40:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:30 smithi098 ceph-mon[103124]: pgmap v843: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 12 op/s 2023-09-09T23:40:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:30 smithi170 ceph-mon[102832]: pgmap v843: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 12 op/s 2023-09-09T23:40:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1263.436248 secs 2023-09-09T23:40:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1263.436248 secs 2023-09-09T23:40:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1263.436248 secs 2023-09-09T23:40:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:33 smithi170 ceph-mon[102832]: pgmap v844: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 10 op/s 2023-09-09T23:40:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1265.034996 secs 2023-09-09T23:40:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:33 smithi082 ceph-mon[98694]: pgmap v844: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 10 op/s 2023-09-09T23:40:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1265.034996 secs 2023-09-09T23:40:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:33 smithi098 ceph-mon[103124]: pgmap v844: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 10 op/s 2023-09-09T23:40:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1265.034996 secs 2023-09-09T23:40:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1265.764152 secs 2023-09-09T23:40:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:34 smithi098 ceph-mon[103124]: pgmap v845: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.3 KiB/s wr, 8 op/s 2023-09-09T23:40:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1265.764152 secs 2023-09-09T23:40:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:34 smithi170 ceph-mon[102832]: pgmap v845: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.3 KiB/s wr, 8 op/s 2023-09-09T23:40:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1265.764152 secs 2023-09-09T23:40:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:34 smithi082 ceph-mon[98694]: pgmap v845: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.3 KiB/s wr, 8 op/s 2023-09-09T23:40:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:40:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:40:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:40:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:40:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:40:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:40:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:36 smithi082 ceph-mon[98694]: pgmap v846: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.3 KiB/s wr, 13 op/s 2023-09-09T23:40:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:40:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:40:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:40:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:36 smithi098 ceph-mon[103124]: pgmap v846: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.3 KiB/s wr, 13 op/s 2023-09-09T23:40:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:40:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:40:37.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:40:37.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:36 smithi170 ceph-mon[102832]: pgmap v846: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.3 KiB/s wr, 13 op/s 2023-09-09T23:40:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1268.436309 secs 2023-09-09T23:40:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:37 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1270.035051 secs 2023-09-09T23:40:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1268.436309 secs 2023-09-09T23:40:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:37 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1270.035051 secs 2023-09-09T23:40:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1268.436309 secs 2023-09-09T23:40:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:37 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1270.035051 secs 2023-09-09T23:40:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:38 smithi082 ceph-mon[98694]: pgmap v847: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2023-09-09T23:40:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:38 smithi098 ceph-mon[103124]: pgmap v847: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2023-09-09T23:40:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:38 smithi170 ceph-mon[102832]: pgmap v847: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2023-09-09T23:40:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1270.764218 secs 2023-09-09T23:40:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1270.764218 secs 2023-09-09T23:40:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1270.764218 secs 2023-09-09T23:40:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:40 smithi082 ceph-mon[98694]: pgmap v848: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2023-09-09T23:40:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:40 smithi098 ceph-mon[103124]: pgmap v848: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2023-09-09T23:40:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:40 smithi170 ceph-mon[102832]: pgmap v848: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2023-09-09T23:40:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1273.436367 secs 2023-09-09T23:40:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1273.436367 secs 2023-09-09T23:40:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1273.436367 secs 2023-09-09T23:40:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:43 smithi170 ceph-mon[102832]: pgmap v849: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 9 op/s 2023-09-09T23:40:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1275.035109 secs 2023-09-09T23:40:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:43 smithi082 ceph-mon[98694]: pgmap v849: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 9 op/s 2023-09-09T23:40:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1275.035109 secs 2023-09-09T23:40:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:43 smithi098 ceph-mon[103124]: pgmap v849: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 9 op/s 2023-09-09T23:40:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1275.035109 secs 2023-09-09T23:40:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1275.764285 secs 2023-09-09T23:40:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:44 smithi098 ceph-mon[103124]: pgmap v850: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:40:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1275.764285 secs 2023-09-09T23:40:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:44 smithi170 ceph-mon[102832]: pgmap v850: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:40:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1275.764285 secs 2023-09-09T23:40:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:44 smithi082 ceph-mon[98694]: pgmap v850: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:40:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:46 smithi098 ceph-mon[103124]: pgmap v851: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 12 op/s 2023-09-09T23:40:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:46 smithi170 ceph-mon[102832]: pgmap v851: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 12 op/s 2023-09-09T23:40:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:46 smithi082 ceph-mon[98694]: pgmap v851: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 12 op/s 2023-09-09T23:40:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1278.436427 secs 2023-09-09T23:40:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1278.436427 secs 2023-09-09T23:40:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1278.436427 secs 2023-09-09T23:40:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1280.035167 secs 2023-09-09T23:40:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:48 smithi098 ceph-mon[103124]: pgmap v852: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-09T23:40:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1280.035167 secs 2023-09-09T23:40:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:48 smithi170 ceph-mon[102832]: pgmap v852: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-09T23:40:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1280.035167 secs 2023-09-09T23:40:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:48 smithi082 ceph-mon[98694]: pgmap v852: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-09T23:40:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1280.764351 secs 2023-09-09T23:40:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1280.764351 secs 2023-09-09T23:40:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1280.764351 secs 2023-09-09T23:40:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:50 smithi170 ceph-mon[102832]: pgmap v853: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.5 KiB/s wr, 12 op/s 2023-09-09T23:40:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:50 smithi082 ceph-mon[98694]: pgmap v853: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.5 KiB/s wr, 12 op/s 2023-09-09T23:40:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:50 smithi098 ceph-mon[103124]: pgmap v853: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.5 KiB/s wr, 12 op/s 2023-09-09T23:40:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1283.436487 secs 2023-09-09T23:40:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1283.436487 secs 2023-09-09T23:40:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1283.436487 secs 2023-09-09T23:40:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:53 smithi170 ceph-mon[102832]: pgmap v854: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2023-09-09T23:40:53.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1285.035227 secs 2023-09-09T23:40:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:53 smithi082 ceph-mon[98694]: pgmap v854: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2023-09-09T23:40:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1285.035227 secs 2023-09-09T23:40:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:53 smithi098 ceph-mon[103124]: pgmap v854: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2023-09-09T23:40:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1285.035227 secs 2023-09-09T23:40:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1285.764419 secs 2023-09-09T23:40:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:54 smithi098 ceph-mon[103124]: pgmap v855: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2023-09-09T23:40:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1285.764419 secs 2023-09-09T23:40:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:54 smithi170 ceph-mon[102832]: pgmap v855: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2023-09-09T23:40:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1285.764419 secs 2023-09-09T23:40:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:54 smithi082 ceph-mon[98694]: pgmap v855: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2023-09-09T23:40:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:56 smithi098 ceph-mon[103124]: pgmap v856: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 13 KiB/s wr, 14 op/s 2023-09-09T23:40:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:56 smithi170 ceph-mon[102832]: pgmap v856: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 13 KiB/s wr, 14 op/s 2023-09-09T23:40:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:56 smithi082 ceph-mon[98694]: pgmap v856: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 13 KiB/s wr, 14 op/s 2023-09-09T23:40:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1288.436590 secs 2023-09-09T23:40:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1288.436590 secs 2023-09-09T23:40:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1288.436590 secs 2023-09-09T23:40:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1290.035287 secs 2023-09-09T23:40:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:58 smithi098 ceph-mon[103124]: pgmap v857: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 KiB/s wr, 10 op/s 2023-09-09T23:40:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1290.035287 secs 2023-09-09T23:40:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:58 smithi170 ceph-mon[102832]: pgmap v857: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 KiB/s wr, 10 op/s 2023-09-09T23:40:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1290.035287 secs 2023-09-09T23:40:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:58 smithi082 ceph-mon[98694]: pgmap v857: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 KiB/s wr, 10 op/s 2023-09-09T23:41:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:40:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1290.764495 secs 2023-09-09T23:41:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:40:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1290.764495 secs 2023-09-09T23:41:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:40:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1290.764495 secs 2023-09-09T23:41:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:00 smithi170 ceph-mon[102832]: pgmap v858: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 36 KiB/s wr, 15 op/s 2023-09-09T23:41:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:00 smithi082 ceph-mon[98694]: pgmap v858: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 36 KiB/s wr, 15 op/s 2023-09-09T23:41:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:00 smithi098 ceph-mon[103124]: pgmap v858: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 36 KiB/s wr, 15 op/s 2023-09-09T23:41:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1293.436660 secs 2023-09-09T23:41:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1293.436660 secs 2023-09-09T23:41:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1293.436660 secs 2023-09-09T23:41:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:03 smithi170 ceph-mon[102832]: pgmap v859: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 55 KiB/s wr, 24 op/s 2023-09-09T23:41:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1295.035345 secs 2023-09-09T23:41:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:03 smithi082 ceph-mon[98694]: pgmap v859: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 55 KiB/s wr, 24 op/s 2023-09-09T23:41:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1295.035345 secs 2023-09-09T23:41:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:03 smithi098 ceph-mon[103124]: pgmap v859: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 55 KiB/s wr, 24 op/s 2023-09-09T23:41:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1295.035345 secs 2023-09-09T23:41:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1295.764561 secs 2023-09-09T23:41:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:04 smithi098 ceph-mon[103124]: pgmap v860: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 55 KiB/s wr, 23 op/s 2023-09-09T23:41:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1295.764561 secs 2023-09-09T23:41:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:04 smithi170 ceph-mon[102832]: pgmap v860: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 55 KiB/s wr, 23 op/s 2023-09-09T23:41:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1295.764561 secs 2023-09-09T23:41:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:04 smithi082 ceph-mon[98694]: pgmap v860: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 55 KiB/s wr, 23 op/s 2023-09-09T23:41:07.014 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:06 smithi098 ceph-mon[103124]: pgmap v861: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 55 KiB/s wr, 28 op/s 2023-09-09T23:41:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:06 smithi170 ceph-mon[102832]: pgmap v861: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 55 KiB/s wr, 28 op/s 2023-09-09T23:41:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:06 smithi082 ceph-mon[98694]: pgmap v861: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 55 KiB/s wr, 28 op/s 2023-09-09T23:41:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1298.436718 secs 2023-09-09T23:41:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:07 smithi098 ceph-mon[103124]: osdmap e127: 12 total, 12 up, 12 in 2023-09-09T23:41:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1298.436718 secs 2023-09-09T23:41:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:07 smithi170 ceph-mon[102832]: osdmap e127: 12 total, 12 up, 12 in 2023-09-09T23:41:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1298.436718 secs 2023-09-09T23:41:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:07 smithi082 ceph-mon[98694]: osdmap e127: 12 total, 12 up, 12 in 2023-09-09T23:41:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1300.035403 secs 2023-09-09T23:41:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:08 smithi098 ceph-mon[103124]: pgmap v863: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:41:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1300.035403 secs 2023-09-09T23:41:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:08 smithi170 ceph-mon[102832]: pgmap v863: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:41:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1300.035403 secs 2023-09-09T23:41:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:08 smithi082 ceph-mon[98694]: pgmap v863: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:41:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1300.764626 secs 2023-09-09T23:41:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1300.764626 secs 2023-09-09T23:41:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1300.764626 secs 2023-09-09T23:41:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:10 smithi170 ceph-mon[102832]: pgmap v864: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 KiB/s wr, 25 op/s 2023-09-09T23:41:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:10 smithi082 ceph-mon[98694]: pgmap v864: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 KiB/s wr, 25 op/s 2023-09-09T23:41:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:10 smithi098 ceph-mon[103124]: pgmap v864: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 KiB/s wr, 25 op/s 2023-09-09T23:41:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1303.436804 secs 2023-09-09T23:41:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1303.436804 secs 2023-09-09T23:41:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1303.436804 secs 2023-09-09T23:41:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:13 smithi170 ceph-mon[102832]: pgmap v865: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-09T23:41:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1305.035463 secs 2023-09-09T23:41:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:13 smithi082 ceph-mon[98694]: pgmap v865: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-09T23:41:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1305.035463 secs 2023-09-09T23:41:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:13 smithi098 ceph-mon[103124]: pgmap v865: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-09T23:41:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1305.035463 secs 2023-09-09T23:41:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1305.764696 secs 2023-09-09T23:41:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:14 smithi098 ceph-mon[103124]: osdmap e128: 12 total, 12 up, 12 in 2023-09-09T23:41:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:14 smithi098 ceph-mon[103124]: pgmap v867: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:41:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1305.764696 secs 2023-09-09T23:41:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:14 smithi170 ceph-mon[102832]: osdmap e128: 12 total, 12 up, 12 in 2023-09-09T23:41:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:14 smithi170 ceph-mon[102832]: pgmap v867: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:41:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1305.764696 secs 2023-09-09T23:41:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:14 smithi082 ceph-mon[98694]: osdmap e128: 12 total, 12 up, 12 in 2023-09-09T23:41:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:14 smithi082 ceph-mon[98694]: pgmap v867: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:41:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:16 smithi098 ceph-mon[103124]: pgmap v868: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2023-09-09T23:41:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:16 smithi170 ceph-mon[102832]: pgmap v868: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2023-09-09T23:41:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:16 smithi082 ceph-mon[98694]: pgmap v868: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2023-09-09T23:41:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1308.436925 secs 2023-09-09T23:41:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1308.436925 secs 2023-09-09T23:41:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1308.436925 secs 2023-09-09T23:41:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1310.035497 secs 2023-09-09T23:41:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:18 smithi170 ceph-mon[102832]: pgmap v869: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2023-09-09T23:41:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1310.035497 secs 2023-09-09T23:41:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:18 smithi082 ceph-mon[98694]: pgmap v869: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2023-09-09T23:41:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1310.035497 secs 2023-09-09T23:41:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:18 smithi098 ceph-mon[103124]: pgmap v869: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2023-09-09T23:41:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1310.764760 secs 2023-09-09T23:41:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1310.764760 secs 2023-09-09T23:41:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1310.764760 secs 2023-09-09T23:41:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:20 smithi170 ceph-mon[102832]: pgmap v870: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.2 KiB/s wr, 12 op/s 2023-09-09T23:41:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:20 smithi082 ceph-mon[98694]: pgmap v870: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.2 KiB/s wr, 12 op/s 2023-09-09T23:41:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:20 smithi098 ceph-mon[103124]: pgmap v870: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.2 KiB/s wr, 12 op/s 2023-09-09T23:41:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1313.436988 secs 2023-09-09T23:41:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1313.436988 secs 2023-09-09T23:41:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1313.436988 secs 2023-09-09T23:41:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:23 smithi170 ceph-mon[102832]: pgmap v871: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 12 op/s 2023-09-09T23:41:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1315.035554 secs 2023-09-09T23:41:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:23 smithi082 ceph-mon[98694]: pgmap v871: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 12 op/s 2023-09-09T23:41:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1315.035554 secs 2023-09-09T23:41:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:23 smithi098 ceph-mon[103124]: pgmap v871: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 12 op/s 2023-09-09T23:41:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1315.035554 secs 2023-09-09T23:41:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1315.764824 secs 2023-09-09T23:41:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:24 smithi098 ceph-mon[103124]: pgmap v872: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 19 KiB/s wr, 11 op/s 2023-09-09T23:41:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1315.764824 secs 2023-09-09T23:41:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:24 smithi170 ceph-mon[102832]: pgmap v872: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 19 KiB/s wr, 11 op/s 2023-09-09T23:41:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1315.764824 secs 2023-09-09T23:41:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:24 smithi082 ceph-mon[98694]: pgmap v872: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 19 KiB/s wr, 11 op/s 2023-09-09T23:41:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:26 smithi098 ceph-mon[103124]: pgmap v873: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 KiB/s wr, 15 op/s 2023-09-09T23:41:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:26 smithi170 ceph-mon[102832]: pgmap v873: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 KiB/s wr, 15 op/s 2023-09-09T23:41:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:26 smithi082 ceph-mon[98694]: pgmap v873: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 18 KiB/s wr, 15 op/s 2023-09-09T23:41:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1318.437051 secs 2023-09-09T23:41:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1318.437051 secs 2023-09-09T23:41:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1318.437051 secs 2023-09-09T23:41:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1320.035614 secs 2023-09-09T23:41:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:28 smithi170 ceph-mon[102832]: pgmap v874: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 9 op/s 2023-09-09T23:41:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1320.035614 secs 2023-09-09T23:41:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:28 smithi082 ceph-mon[98694]: pgmap v874: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 9 op/s 2023-09-09T23:41:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1320.035614 secs 2023-09-09T23:41:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:28 smithi098 ceph-mon[103124]: pgmap v874: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 9 op/s 2023-09-09T23:41:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1320.764889 secs 2023-09-09T23:41:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1320.764889 secs 2023-09-09T23:41:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1320.764889 secs 2023-09-09T23:41:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:30 smithi170 ceph-mon[102832]: pgmap v875: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-09-09T23:41:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:30 smithi082 ceph-mon[98694]: pgmap v875: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-09-09T23:41:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:30 smithi098 ceph-mon[103124]: pgmap v875: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-09-09T23:41:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1323.437109 secs 2023-09-09T23:41:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1323.437109 secs 2023-09-09T23:41:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1323.437109 secs 2023-09-09T23:41:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:33 smithi170 ceph-mon[102832]: pgmap v876: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 10 op/s 2023-09-09T23:41:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1325.035671 secs 2023-09-09T23:41:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:33 smithi082 ceph-mon[98694]: pgmap v876: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 10 op/s 2023-09-09T23:41:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1325.035671 secs 2023-09-09T23:41:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:33 smithi098 ceph-mon[103124]: pgmap v876: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 10 op/s 2023-09-09T23:41:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1325.035671 secs 2023-09-09T23:41:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1325.764956 secs 2023-09-09T23:41:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:34 smithi098 ceph-mon[103124]: pgmap v877: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 KiB/s wr, 8 op/s 2023-09-09T23:41:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1325.764956 secs 2023-09-09T23:41:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:34 smithi170 ceph-mon[102832]: pgmap v877: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 KiB/s wr, 8 op/s 2023-09-09T23:41:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1325.764956 secs 2023-09-09T23:41:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:34 smithi082 ceph-mon[98694]: pgmap v877: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 KiB/s wr, 8 op/s 2023-09-09T23:41:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:41:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:41:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:41:37.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:37 smithi170 ceph-mon[102832]: pgmap v878: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.6 KiB/s wr, 12 op/s 2023-09-09T23:41:37.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1328.437178 secs 2023-09-09T23:41:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:37 smithi082 ceph-mon[98694]: pgmap v878: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.6 KiB/s wr, 12 op/s 2023-09-09T23:41:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1328.437178 secs 2023-09-09T23:41:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:37 smithi098 ceph-mon[103124]: pgmap v878: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.6 KiB/s wr, 12 op/s 2023-09-09T23:41:38.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1328.437178 secs 2023-09-09T23:41:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1330.035735 secs 2023-09-09T23:41:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1330.035735 secs 2023-09-09T23:41:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1330.035735 secs 2023-09-09T23:41:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1330.765020 secs 2023-09-09T23:41:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:39 smithi170 ceph-mon[102832]: pgmap v879: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 KiB/s wr, 8 op/s 2023-09-09T23:41:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1330.765020 secs 2023-09-09T23:41:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:39 smithi082 ceph-mon[98694]: pgmap v879: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 KiB/s wr, 8 op/s 2023-09-09T23:41:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1330.765020 secs 2023-09-09T23:41:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:39 smithi098 ceph-mon[103124]: pgmap v879: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 KiB/s wr, 8 op/s 2023-09-09T23:41:41.353 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:41:41.354 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:41:41.354 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:41:41.354 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:41:41.354 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:41 smithi098 ceph-mon[103124]: pgmap v880: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 KiB/s wr, 11 op/s 2023-09-09T23:41:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:41:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:41:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:41:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:41:41.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:41 smithi170 ceph-mon[102832]: pgmap v880: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 KiB/s wr, 11 op/s 2023-09-09T23:41:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:41:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:41:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:41:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:41:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:41 smithi082 ceph-mon[98694]: pgmap v880: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 KiB/s wr, 11 op/s 2023-09-09T23:41:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1333.437291 secs 2023-09-09T23:41:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1333.437291 secs 2023-09-09T23:41:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1333.437291 secs 2023-09-09T23:41:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:43 smithi170 ceph-mon[102832]: pgmap v881: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 KiB/s wr, 9 op/s 2023-09-09T23:41:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1335.035793 secs 2023-09-09T23:41:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:43 smithi082 ceph-mon[98694]: pgmap v881: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 KiB/s wr, 9 op/s 2023-09-09T23:41:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1335.035793 secs 2023-09-09T23:41:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:43 smithi098 ceph-mon[103124]: pgmap v881: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 KiB/s wr, 9 op/s 2023-09-09T23:41:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1335.035793 secs 2023-09-09T23:41:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1335.765085 secs 2023-09-09T23:41:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:41:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:41:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:41:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:41:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:41:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:44 smithi098 ceph-mon[103124]: pgmap v882: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:41:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1335.765085 secs 2023-09-09T23:41:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:41:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:41:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:41:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:41:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:41:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:44 smithi170 ceph-mon[102832]: pgmap v882: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:41:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1335.765085 secs 2023-09-09T23:41:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:41:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:41:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:41:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:41:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:41:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:44 smithi082 ceph-mon[98694]: pgmap v882: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:41:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:46 smithi170 ceph-mon[102832]: pgmap v883: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2023-09-09T23:41:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:46 smithi082 ceph-mon[98694]: pgmap v883: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2023-09-09T23:41:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:46 smithi098 ceph-mon[103124]: pgmap v883: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.9 KiB/s wr, 13 op/s 2023-09-09T23:41:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1338.437381 secs 2023-09-09T23:41:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1338.437381 secs 2023-09-09T23:41:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1338.437381 secs 2023-09-09T23:41:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1340.035885 secs 2023-09-09T23:41:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:48 smithi170 ceph-mon[102832]: pgmap v884: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2023-09-09T23:41:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1340.035885 secs 2023-09-09T23:41:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:48 smithi082 ceph-mon[98694]: pgmap v884: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2023-09-09T23:41:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1340.035885 secs 2023-09-09T23:41:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:48 smithi098 ceph-mon[103124]: pgmap v884: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2023-09-09T23:41:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1340.765150 secs 2023-09-09T23:41:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1340.765150 secs 2023-09-09T23:41:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1340.765150 secs 2023-09-09T23:41:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:50 smithi170 ceph-mon[102832]: pgmap v885: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.6 KiB/s wr, 12 op/s 2023-09-09T23:41:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:50 smithi082 ceph-mon[98694]: pgmap v885: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.6 KiB/s wr, 12 op/s 2023-09-09T23:41:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:50 smithi098 ceph-mon[103124]: pgmap v885: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.6 KiB/s wr, 12 op/s 2023-09-09T23:41:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1343.437445 secs 2023-09-09T23:41:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1343.437445 secs 2023-09-09T23:41:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1343.437445 secs 2023-09-09T23:41:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:53 smithi170 ceph-mon[102832]: pgmap v886: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.6 KiB/s wr, 11 op/s 2023-09-09T23:41:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1345.035942 secs 2023-09-09T23:41:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:53 smithi082 ceph-mon[98694]: pgmap v886: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.6 KiB/s wr, 11 op/s 2023-09-09T23:41:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1345.035942 secs 2023-09-09T23:41:53.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:53 smithi098 ceph-mon[103124]: pgmap v886: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.6 KiB/s wr, 11 op/s 2023-09-09T23:41:53.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1345.035942 secs 2023-09-09T23:41:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1345.765216 secs 2023-09-09T23:41:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:54 smithi098 ceph-mon[103124]: pgmap v887: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2023-09-09T23:41:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1345.765216 secs 2023-09-09T23:41:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:54 smithi170 ceph-mon[102832]: pgmap v887: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2023-09-09T23:41:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1345.765216 secs 2023-09-09T23:41:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:54 smithi082 ceph-mon[98694]: pgmap v887: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2023-09-09T23:41:57.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:56 smithi170 ceph-mon[102832]: pgmap v888: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 14 op/s 2023-09-09T23:41:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:56 smithi082 ceph-mon[98694]: pgmap v888: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 14 op/s 2023-09-09T23:41:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:56 smithi098 ceph-mon[103124]: pgmap v888: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 14 op/s 2023-09-09T23:41:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1348.437505 secs 2023-09-09T23:41:58.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1348.437505 secs 2023-09-09T23:41:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1348.437505 secs 2023-09-09T23:41:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1350.036047 secs 2023-09-09T23:41:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:58 smithi170 ceph-mon[102832]: pgmap v889: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 10 op/s 2023-09-09T23:41:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1350.036047 secs 2023-09-09T23:41:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:58 smithi082 ceph-mon[98694]: pgmap v889: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 10 op/s 2023-09-09T23:41:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1350.036047 secs 2023-09-09T23:41:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:58 smithi098 ceph-mon[103124]: pgmap v889: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 10 op/s 2023-09-09T23:42:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:41:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1350.765282 secs 2023-09-09T23:42:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:41:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1350.765282 secs 2023-09-09T23:42:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:41:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1350.765282 secs 2023-09-09T23:42:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:00 smithi170 ceph-mon[102832]: pgmap v890: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-09-09T23:42:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:00 smithi082 ceph-mon[98694]: pgmap v890: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-09-09T23:42:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:00 smithi098 ceph-mon[103124]: pgmap v890: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-09-09T23:42:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1353.437565 secs 2023-09-09T23:42:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1353.437565 secs 2023-09-09T23:42:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1353.437565 secs 2023-09-09T23:42:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:03 smithi170 ceph-mon[102832]: pgmap v891: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 44 KiB/s wr, 23 op/s 2023-09-09T23:42:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1355.036106 secs 2023-09-09T23:42:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:03 smithi082 ceph-mon[98694]: pgmap v891: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 44 KiB/s wr, 23 op/s 2023-09-09T23:42:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1355.036106 secs 2023-09-09T23:42:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:03 smithi098 ceph-mon[103124]: pgmap v891: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 44 KiB/s wr, 23 op/s 2023-09-09T23:42:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1355.036106 secs 2023-09-09T23:42:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1355.765349 secs 2023-09-09T23:42:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:04 smithi170 ceph-mon[102832]: pgmap v892: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-09-09T23:42:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1355.765349 secs 2023-09-09T23:42:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:04 smithi082 ceph-mon[98694]: pgmap v892: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-09-09T23:42:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1355.765349 secs 2023-09-09T23:42:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:04 smithi098 ceph-mon[103124]: pgmap v892: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-09-09T23:42:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:06 smithi170 ceph-mon[102832]: pgmap v893: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 55 KiB/s wr, 28 op/s 2023-09-09T23:42:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:06 smithi082 ceph-mon[98694]: pgmap v893: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 55 KiB/s wr, 28 op/s 2023-09-09T23:42:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:06 smithi098 ceph-mon[103124]: pgmap v893: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 55 KiB/s wr, 28 op/s 2023-09-09T23:42:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1358.437626 secs 2023-09-09T23:42:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:07 smithi170 ceph-mon[102832]: osdmap e129: 12 total, 12 up, 12 in 2023-09-09T23:42:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1358.437626 secs 2023-09-09T23:42:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:07 smithi082 ceph-mon[98694]: osdmap e129: 12 total, 12 up, 12 in 2023-09-09T23:42:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1358.437626 secs 2023-09-09T23:42:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:07 smithi098 ceph-mon[103124]: osdmap e129: 12 total, 12 up, 12 in 2023-09-09T23:42:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1360.036168 secs 2023-09-09T23:42:09.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:08 smithi170 ceph-mon[102832]: pgmap v895: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:42:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1360.036168 secs 2023-09-09T23:42:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:08 smithi082 ceph-mon[98694]: pgmap v895: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:42:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1360.036168 secs 2023-09-09T23:42:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:08 smithi098 ceph-mon[103124]: pgmap v895: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:42:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1360.765413 secs 2023-09-09T23:42:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1360.765413 secs 2023-09-09T23:42:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1360.765413 secs 2023-09-09T23:42:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:10 smithi170 ceph-mon[102832]: pgmap v896: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 40 KiB/s wr, 26 op/s 2023-09-09T23:42:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:10 smithi082 ceph-mon[98694]: pgmap v896: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 40 KiB/s wr, 26 op/s 2023-09-09T23:42:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:10 smithi098 ceph-mon[103124]: pgmap v896: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 40 KiB/s wr, 26 op/s 2023-09-09T23:42:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1363.437684 secs 2023-09-09T23:42:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1363.437684 secs 2023-09-09T23:42:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1363.437684 secs 2023-09-09T23:42:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:13 smithi170 ceph-mon[102832]: pgmap v897: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-09-09T23:42:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1365.036227 secs 2023-09-09T23:42:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:13 smithi082 ceph-mon[98694]: pgmap v897: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-09-09T23:42:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1365.036227 secs 2023-09-09T23:42:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:13 smithi098 ceph-mon[103124]: pgmap v897: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-09-09T23:42:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1365.036227 secs 2023-09-09T23:42:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1365.765483 secs 2023-09-09T23:42:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:14 smithi098 ceph-mon[103124]: osdmap e130: 12 total, 12 up, 12 in 2023-09-09T23:42:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:14 smithi098 ceph-mon[103124]: pgmap v899: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:42:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1365.765483 secs 2023-09-09T23:42:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:14 smithi170 ceph-mon[102832]: osdmap e130: 12 total, 12 up, 12 in 2023-09-09T23:42:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:14 smithi170 ceph-mon[102832]: pgmap v899: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:42:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1365.765483 secs 2023-09-09T23:42:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:14 smithi082 ceph-mon[98694]: osdmap e130: 12 total, 12 up, 12 in 2023-09-09T23:42:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:14 smithi082 ceph-mon[98694]: pgmap v899: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:42:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:16 smithi170 ceph-mon[102832]: pgmap v900: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.5 KiB/s wr, 12 op/s 2023-09-09T23:42:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:16 smithi082 ceph-mon[98694]: pgmap v900: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.5 KiB/s wr, 12 op/s 2023-09-09T23:42:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:16 smithi098 ceph-mon[103124]: pgmap v900: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.5 KiB/s wr, 12 op/s 2023-09-09T23:42:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1368.437785 secs 2023-09-09T23:42:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1368.437785 secs 2023-09-09T23:42:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1368.437785 secs 2023-09-09T23:42:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1370.036286 secs 2023-09-09T23:42:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:18 smithi170 ceph-mon[102832]: pgmap v901: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.9 KiB/s wr, 11 op/s 2023-09-09T23:42:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1370.036286 secs 2023-09-09T23:42:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:18 smithi082 ceph-mon[98694]: pgmap v901: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.9 KiB/s wr, 11 op/s 2023-09-09T23:42:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1370.036286 secs 2023-09-09T23:42:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:18 smithi098 ceph-mon[103124]: pgmap v901: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.9 KiB/s wr, 11 op/s 2023-09-09T23:42:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1370.765550 secs 2023-09-09T23:42:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1370.765550 secs 2023-09-09T23:42:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1370.765550 secs 2023-09-09T23:42:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:20 smithi170 ceph-mon[102832]: pgmap v902: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.1 KiB/s wr, 12 op/s 2023-09-09T23:42:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:20 smithi082 ceph-mon[98694]: pgmap v902: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.1 KiB/s wr, 12 op/s 2023-09-09T23:42:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:20 smithi098 ceph-mon[103124]: pgmap v902: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.1 KiB/s wr, 12 op/s 2023-09-09T23:42:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1373.437852 secs 2023-09-09T23:42:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1373.437852 secs 2023-09-09T23:42:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1373.437852 secs 2023-09-09T23:42:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:23 smithi170 ceph-mon[102832]: pgmap v903: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-09T23:42:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1375.036393 secs 2023-09-09T23:42:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:23 smithi082 ceph-mon[98694]: pgmap v903: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-09T23:42:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1375.036393 secs 2023-09-09T23:42:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:23 smithi098 ceph-mon[103124]: pgmap v903: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-09T23:42:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1375.036393 secs 2023-09-09T23:42:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1375.765616 secs 2023-09-09T23:42:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:24 smithi170 ceph-mon[102832]: pgmap v904: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-09T23:42:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1375.765616 secs 2023-09-09T23:42:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:24 smithi082 ceph-mon[98694]: pgmap v904: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-09T23:42:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1375.765616 secs 2023-09-09T23:42:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:24 smithi098 ceph-mon[103124]: pgmap v904: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-09T23:42:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:26 smithi170 ceph-mon[102832]: pgmap v905: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 20 KiB/s wr, 15 op/s 2023-09-09T23:42:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:26 smithi082 ceph-mon[98694]: pgmap v905: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 20 KiB/s wr, 15 op/s 2023-09-09T23:42:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:26 smithi098 ceph-mon[103124]: pgmap v905: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 20 KiB/s wr, 15 op/s 2023-09-09T23:42:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1378.437960 secs 2023-09-09T23:42:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1378.437960 secs 2023-09-09T23:42:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1378.437960 secs 2023-09-09T23:42:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1380.036454 secs 2023-09-09T23:42:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:28 smithi170 ceph-mon[102832]: pgmap v906: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 9 op/s 2023-09-09T23:42:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1380.036454 secs 2023-09-09T23:42:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:28 smithi082 ceph-mon[98694]: pgmap v906: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 9 op/s 2023-09-09T23:42:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1380.036454 secs 2023-09-09T23:42:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:28 smithi098 ceph-mon[103124]: pgmap v906: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 9 op/s 2023-09-09T23:42:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1380.765681 secs 2023-09-09T23:42:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1380.765681 secs 2023-09-09T23:42:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1380.765681 secs 2023-09-09T23:42:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:30 smithi170 ceph-mon[102832]: pgmap v907: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 12 op/s 2023-09-09T23:42:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:30 smithi082 ceph-mon[98694]: pgmap v907: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 12 op/s 2023-09-09T23:42:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:30 smithi098 ceph-mon[103124]: pgmap v907: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 12 op/s 2023-09-09T23:42:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1383.438020 secs 2023-09-09T23:42:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1383.438020 secs 2023-09-09T23:42:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1383.438020 secs 2023-09-09T23:42:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:33 smithi170 ceph-mon[102832]: pgmap v908: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 10 op/s 2023-09-09T23:42:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1385.036539 secs 2023-09-09T23:42:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:33 smithi082 ceph-mon[98694]: pgmap v908: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 10 op/s 2023-09-09T23:42:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1385.036539 secs 2023-09-09T23:42:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:33 smithi098 ceph-mon[103124]: pgmap v908: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 10 op/s 2023-09-09T23:42:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1385.036539 secs 2023-09-09T23:42:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1385.765749 secs 2023-09-09T23:42:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:34 smithi170 ceph-mon[102832]: pgmap v909: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 8 op/s 2023-09-09T23:42:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1385.765749 secs 2023-09-09T23:42:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:34 smithi082 ceph-mon[98694]: pgmap v909: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 8 op/s 2023-09-09T23:42:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1385.765749 secs 2023-09-09T23:42:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:34 smithi098 ceph-mon[103124]: pgmap v909: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 8 op/s 2023-09-09T23:42:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:36 smithi170 ceph-mon[102832]: pgmap v910: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.7 KiB/s wr, 12 op/s 2023-09-09T23:42:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:36 smithi082 ceph-mon[98694]: pgmap v910: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.7 KiB/s wr, 12 op/s 2023-09-09T23:42:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:36 smithi098 ceph-mon[103124]: pgmap v910: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.7 KiB/s wr, 12 op/s 2023-09-09T23:42:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1388.438079 secs 2023-09-09T23:42:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1388.438079 secs 2023-09-09T23:42:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1388.438079 secs 2023-09-09T23:42:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1390.036602 secs 2023-09-09T23:42:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:38 smithi170 ceph-mon[102832]: pgmap v911: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 KiB/s wr, 8 op/s 2023-09-09T23:42:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1390.036602 secs 2023-09-09T23:42:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:38 smithi082 ceph-mon[98694]: pgmap v911: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 KiB/s wr, 8 op/s 2023-09-09T23:42:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1390.036602 secs 2023-09-09T23:42:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:38 smithi098 ceph-mon[103124]: pgmap v911: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 KiB/s wr, 8 op/s 2023-09-09T23:42:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1390.765811 secs 2023-09-09T23:42:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1390.765811 secs 2023-09-09T23:42:40.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1390.765811 secs 2023-09-09T23:42:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:40 smithi170 ceph-mon[102832]: pgmap v912: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 KiB/s wr, 11 op/s 2023-09-09T23:42:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:40 smithi082 ceph-mon[98694]: pgmap v912: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 KiB/s wr, 11 op/s 2023-09-09T23:42:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:40 smithi098 ceph-mon[103124]: pgmap v912: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 KiB/s wr, 11 op/s 2023-09-09T23:42:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1393.438139 secs 2023-09-09T23:42:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1393.438139 secs 2023-09-09T23:42:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1393.438139 secs 2023-09-09T23:42:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:43 smithi170 ceph-mon[102832]: pgmap v913: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 KiB/s wr, 9 op/s 2023-09-09T23:42:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1395.036661 secs 2023-09-09T23:42:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:43 smithi082 ceph-mon[98694]: pgmap v913: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 KiB/s wr, 9 op/s 2023-09-09T23:42:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1395.036661 secs 2023-09-09T23:42:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:43 smithi098 ceph-mon[103124]: pgmap v913: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 KiB/s wr, 9 op/s 2023-09-09T23:42:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1395.036661 secs 2023-09-09T23:42:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1395.765879 secs 2023-09-09T23:42:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:42:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:44 smithi170 ceph-mon[102832]: pgmap v914: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:42:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1395.765879 secs 2023-09-09T23:42:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:42:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:44 smithi082 ceph-mon[98694]: pgmap v914: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:42:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1395.765879 secs 2023-09-09T23:42:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:42:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:44 smithi098 ceph-mon[103124]: pgmap v914: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:42:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:42:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:42:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:42:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:42:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:42:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:42:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:42:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:42:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:42:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:46 smithi170 ceph-mon[102832]: pgmap v915: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 12 op/s 2023-09-09T23:42:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:46 smithi082 ceph-mon[98694]: pgmap v915: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 12 op/s 2023-09-09T23:42:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:46 smithi098 ceph-mon[103124]: pgmap v915: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 12 op/s 2023-09-09T23:42:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1398.438214 secs 2023-09-09T23:42:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1398.438214 secs 2023-09-09T23:42:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1398.438214 secs 2023-09-09T23:42:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1400.036720 secs 2023-09-09T23:42:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:48 smithi170 ceph-mon[102832]: pgmap v916: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 8 op/s 2023-09-09T23:42:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1400.036720 secs 2023-09-09T23:42:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:48 smithi082 ceph-mon[98694]: pgmap v916: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 8 op/s 2023-09-09T23:42:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1400.036720 secs 2023-09-09T23:42:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:48 smithi098 ceph-mon[103124]: pgmap v916: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 8 op/s 2023-09-09T23:42:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1400.765943 secs 2023-09-09T23:42:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1400.765943 secs 2023-09-09T23:42:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1400.765943 secs 2023-09-09T23:42:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:50 smithi170 ceph-mon[102832]: pgmap v917: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2023-09-09T23:42:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:50 smithi082 ceph-mon[98694]: pgmap v917: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2023-09-09T23:42:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:50 smithi098 ceph-mon[103124]: pgmap v917: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2023-09-09T23:42:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1403.438274 secs 2023-09-09T23:42:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1403.438274 secs 2023-09-09T23:42:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1403.438274 secs 2023-09-09T23:42:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:53 smithi170 ceph-mon[102832]: pgmap v918: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.7 KiB/s wr, 11 op/s 2023-09-09T23:42:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1405.036779 secs 2023-09-09T23:42:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:53 smithi082 ceph-mon[98694]: pgmap v918: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.7 KiB/s wr, 11 op/s 2023-09-09T23:42:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1405.036779 secs 2023-09-09T23:42:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:53 smithi098 ceph-mon[103124]: pgmap v918: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.7 KiB/s wr, 11 op/s 2023-09-09T23:42:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1405.036779 secs 2023-09-09T23:42:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1405.766011 secs 2023-09-09T23:42:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:54 smithi170 ceph-mon[102832]: pgmap v919: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2023-09-09T23:42:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1405.766011 secs 2023-09-09T23:42:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:54 smithi082 ceph-mon[98694]: pgmap v919: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2023-09-09T23:42:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1405.766011 secs 2023-09-09T23:42:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:54 smithi098 ceph-mon[103124]: pgmap v919: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2023-09-09T23:42:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:56 smithi170 ceph-mon[102832]: pgmap v920: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 14 op/s 2023-09-09T23:42:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:56 smithi082 ceph-mon[98694]: pgmap v920: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 14 op/s 2023-09-09T23:42:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:56 smithi098 ceph-mon[103124]: pgmap v920: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 14 op/s 2023-09-09T23:42:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1408.438375 secs 2023-09-09T23:42:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1408.438375 secs 2023-09-09T23:42:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1408.438375 secs 2023-09-09T23:42:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1410.036840 secs 2023-09-09T23:42:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:58 smithi170 ceph-mon[102832]: pgmap v921: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-09T23:42:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1410.036840 secs 2023-09-09T23:42:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:58 smithi082 ceph-mon[98694]: pgmap v921: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-09T23:42:59.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1410.036840 secs 2023-09-09T23:42:59.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:58 smithi098 ceph-mon[103124]: pgmap v921: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-09T23:43:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:42:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1410.766076 secs 2023-09-09T23:43:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:42:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1410.766076 secs 2023-09-09T23:43:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:42:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1410.766076 secs 2023-09-09T23:43:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:00 smithi170 ceph-mon[102832]: pgmap v922: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 13 op/s 2023-09-09T23:43:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:00 smithi082 ceph-mon[98694]: pgmap v922: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 13 op/s 2023-09-09T23:43:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:00 smithi098 ceph-mon[103124]: pgmap v922: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 13 op/s 2023-09-09T23:43:02.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1413.438454 secs 2023-09-09T23:43:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1413.438454 secs 2023-09-09T23:43:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1413.438454 secs 2023-09-09T23:43:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:03 smithi170 ceph-mon[102832]: pgmap v923: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 KiB/s wr, 22 op/s 2023-09-09T23:43:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1415.036924 secs 2023-09-09T23:43:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:03 smithi082 ceph-mon[98694]: pgmap v923: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 KiB/s wr, 22 op/s 2023-09-09T23:43:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1415.036924 secs 2023-09-09T23:43:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:03 smithi098 ceph-mon[103124]: pgmap v923: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 KiB/s wr, 22 op/s 2023-09-09T23:43:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1415.036924 secs 2023-09-09T23:43:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1415.766142 secs 2023-09-09T23:43:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:04 smithi170 ceph-mon[102832]: pgmap v924: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 33 KiB/s wr, 21 op/s 2023-09-09T23:43:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1415.766142 secs 2023-09-09T23:43:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:04 smithi082 ceph-mon[98694]: pgmap v924: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 33 KiB/s wr, 21 op/s 2023-09-09T23:43:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1415.766142 secs 2023-09-09T23:43:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:04 smithi098 ceph-mon[103124]: pgmap v924: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 33 KiB/s wr, 21 op/s 2023-09-09T23:43:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:06 smithi170 ceph-mon[102832]: pgmap v925: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 55 KiB/s wr, 28 op/s 2023-09-09T23:43:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:06 smithi082 ceph-mon[98694]: pgmap v925: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 55 KiB/s wr, 28 op/s 2023-09-09T23:43:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:06 smithi098 ceph-mon[103124]: pgmap v925: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 55 KiB/s wr, 28 op/s 2023-09-09T23:43:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1418.438513 secs 2023-09-09T23:43:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:07 smithi170 ceph-mon[102832]: osdmap e131: 12 total, 12 up, 12 in 2023-09-09T23:43:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1418.438513 secs 2023-09-09T23:43:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:07 smithi082 ceph-mon[98694]: osdmap e131: 12 total, 12 up, 12 in 2023-09-09T23:43:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1418.438513 secs 2023-09-09T23:43:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:07 smithi098 ceph-mon[103124]: osdmap e131: 12 total, 12 up, 12 in 2023-09-09T23:43:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1420.037009 secs 2023-09-09T23:43:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:08 smithi170 ceph-mon[102832]: pgmap v927: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:43:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1420.037009 secs 2023-09-09T23:43:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:08 smithi082 ceph-mon[98694]: pgmap v927: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:43:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1420.037009 secs 2023-09-09T23:43:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:08 smithi098 ceph-mon[103124]: pgmap v927: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:43:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1420.766208 secs 2023-09-09T23:43:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1420.766208 secs 2023-09-09T23:43:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1420.766208 secs 2023-09-09T23:43:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:10 smithi170 ceph-mon[102832]: pgmap v928: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:43:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:10 smithi082 ceph-mon[98694]: pgmap v928: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:43:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:10 smithi098 ceph-mon[103124]: pgmap v928: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:43:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1423.438597 secs 2023-09-09T23:43:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1423.438597 secs 2023-09-09T23:43:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1423.438597 secs 2023-09-09T23:43:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:13 smithi170 ceph-mon[102832]: pgmap v929: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 27 KiB/s wr, 13 op/s 2023-09-09T23:43:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1425.037066 secs 2023-09-09T23:43:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:13 smithi082 ceph-mon[98694]: pgmap v929: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 27 KiB/s wr, 13 op/s 2023-09-09T23:43:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1425.037066 secs 2023-09-09T23:43:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:13 smithi098 ceph-mon[103124]: pgmap v929: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 27 KiB/s wr, 13 op/s 2023-09-09T23:43:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1425.037066 secs 2023-09-09T23:43:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1425.766275 secs 2023-09-09T23:43:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:14 smithi170 ceph-mon[102832]: osdmap e132: 12 total, 12 up, 12 in 2023-09-09T23:43:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:14 smithi170 ceph-mon[102832]: pgmap v931: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:43:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1425.766275 secs 2023-09-09T23:43:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:14 smithi082 ceph-mon[98694]: osdmap e132: 12 total, 12 up, 12 in 2023-09-09T23:43:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:14 smithi082 ceph-mon[98694]: pgmap v931: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:43:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1425.766275 secs 2023-09-09T23:43:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:14 smithi098 ceph-mon[103124]: osdmap e132: 12 total, 12 up, 12 in 2023-09-09T23:43:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:14 smithi098 ceph-mon[103124]: pgmap v931: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:43:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:16 smithi170 ceph-mon[102832]: pgmap v932: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2023-09-09T23:43:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:16 smithi082 ceph-mon[98694]: pgmap v932: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2023-09-09T23:43:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:16 smithi098 ceph-mon[103124]: pgmap v932: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2023-09-09T23:43:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1428.438656 secs 2023-09-09T23:43:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1428.438656 secs 2023-09-09T23:43:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1428.438656 secs 2023-09-09T23:43:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1430.037125 secs 2023-09-09T23:43:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:18 smithi170 ceph-mon[102832]: pgmap v933: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2023-09-09T23:43:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1430.037125 secs 2023-09-09T23:43:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:18 smithi082 ceph-mon[98694]: pgmap v933: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2023-09-09T23:43:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1430.037125 secs 2023-09-09T23:43:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:18 smithi098 ceph-mon[103124]: pgmap v933: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2023-09-09T23:43:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1430.766343 secs 2023-09-09T23:43:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1430.766343 secs 2023-09-09T23:43:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1430.766343 secs 2023-09-09T23:43:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:20 smithi170 ceph-mon[102832]: pgmap v934: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 KiB/s wr, 12 op/s 2023-09-09T23:43:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:20 smithi082 ceph-mon[98694]: pgmap v934: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 KiB/s wr, 12 op/s 2023-09-09T23:43:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:20 smithi098 ceph-mon[103124]: pgmap v934: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 KiB/s wr, 12 op/s 2023-09-09T23:43:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1433.438807 secs 2023-09-09T23:43:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1433.438807 secs 2023-09-09T23:43:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1433.438807 secs 2023-09-09T23:43:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:23 smithi170 ceph-mon[102832]: pgmap v935: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-09T23:43:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1435.037187 secs 2023-09-09T23:43:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:23 smithi082 ceph-mon[98694]: pgmap v935: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-09T23:43:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1435.037187 secs 2023-09-09T23:43:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:23 smithi098 ceph-mon[103124]: pgmap v935: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-09T23:43:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1435.037187 secs 2023-09-09T23:43:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1435.766408 secs 2023-09-09T23:43:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:24 smithi170 ceph-mon[102832]: pgmap v936: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-09T23:43:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1435.766408 secs 2023-09-09T23:43:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:24 smithi082 ceph-mon[98694]: pgmap v936: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-09T23:43:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1435.766408 secs 2023-09-09T23:43:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:24 smithi098 ceph-mon[103124]: pgmap v936: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-09T23:43:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:26 smithi170 ceph-mon[102832]: pgmap v937: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 20 KiB/s wr, 15 op/s 2023-09-09T23:43:27.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:26 smithi082 ceph-mon[98694]: pgmap v937: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 20 KiB/s wr, 15 op/s 2023-09-09T23:43:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:26 smithi098 ceph-mon[103124]: pgmap v937: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 20 KiB/s wr, 15 op/s 2023-09-09T23:43:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1438.438879 secs 2023-09-09T23:43:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1438.438879 secs 2023-09-09T23:43:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1438.438879 secs 2023-09-09T23:43:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:28 smithi170 ceph-mon[102832]: 3 slow requests, 3 included below; oldest blocked for > 1440.037274 secs 2023-09-09T23:43:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:28 smithi170 ceph-mon[102832]: slow request 1440.037274 seconds old, received at 2023-09-09T23:19:27.740042+0000: client_request(mds.4:47 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently acquired locks 2023-09-09T23:43:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:28 smithi170 ceph-mon[102832]: slow request 1440.036894 seconds old, received at 2023-09-09T23:19:27.740422+0000: client_request(mds.4:48 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently requesting remote authpins 2023-09-09T23:43:29.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:28 smithi170 ceph-mon[102832]: slow request 1440.034673 seconds old, received at 2023-09-09T23:19:27.742643+0000: client_request(client.24913:3869 unlink #0x10000000007/00000001000000010000000B 2023-09-09T23:19:27.741863+0000 caller_uid=26, caller_gid=26{26,}) currently requesting remote authpins 2023-09-09T23:43:29.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:28 smithi170 ceph-mon[102832]: pgmap v938: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-09T23:43:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:28 smithi082 ceph-mon[98694]: 3 slow requests, 3 included below; oldest blocked for > 1440.037274 secs 2023-09-09T23:43:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:28 smithi082 ceph-mon[98694]: slow request 1440.037274 seconds old, received at 2023-09-09T23:19:27.740042+0000: client_request(mds.4:47 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently acquired locks 2023-09-09T23:43:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:28 smithi082 ceph-mon[98694]: slow request 1440.036894 seconds old, received at 2023-09-09T23:19:27.740422+0000: client_request(mds.4:48 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently requesting remote authpins 2023-09-09T23:43:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:28 smithi082 ceph-mon[98694]: slow request 1440.034673 seconds old, received at 2023-09-09T23:19:27.742643+0000: client_request(client.24913:3869 unlink #0x10000000007/00000001000000010000000B 2023-09-09T23:19:27.741863+0000 caller_uid=26, caller_gid=26{26,}) currently requesting remote authpins 2023-09-09T23:43:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:28 smithi082 ceph-mon[98694]: pgmap v938: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-09T23:43:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:28 smithi098 ceph-mon[103124]: 3 slow requests, 3 included below; oldest blocked for > 1440.037274 secs 2023-09-09T23:43:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:28 smithi098 ceph-mon[103124]: slow request 1440.037274 seconds old, received at 2023-09-09T23:19:27.740042+0000: client_request(mds.4:47 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently acquired locks 2023-09-09T23:43:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:28 smithi098 ceph-mon[103124]: slow request 1440.036894 seconds old, received at 2023-09-09T23:19:27.740422+0000: client_request(mds.4:48 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently requesting remote authpins 2023-09-09T23:43:29.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:28 smithi098 ceph-mon[103124]: slow request 1440.034673 seconds old, received at 2023-09-09T23:19:27.742643+0000: client_request(client.24913:3869 unlink #0x10000000007/00000001000000010000000B 2023-09-09T23:19:27.741863+0000 caller_uid=26, caller_gid=26{26,}) currently requesting remote authpins 2023-09-09T23:43:29.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:28 smithi098 ceph-mon[103124]: pgmap v938: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-09T23:43:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:29 smithi170 ceph-mon[102832]: 3 slow requests, 3 included below; oldest blocked for > 1440.766477 secs 2023-09-09T23:43:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:29 smithi170 ceph-mon[102832]: slow request 1440.766477 seconds old, received at 2023-09-09T23:19:27.740800+0000: peer_request:mds.4:47 currently acquired locks 2023-09-09T23:43:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:29 smithi170 ceph-mon[102832]: slow request 1440.766243 seconds old, received at 2023-09-09T23:19:27.741034+0000: peer_request(mds.4:48.0 authpin) currently dispatched 2023-09-09T23:43:30.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:29 smithi170 ceph-mon[102832]: slow request 1440.764207 seconds old, received at 2023-09-09T23:19:27.743070+0000: peer_request(client.24913:3869.0 authpin) currently dispatched 2023-09-09T23:43:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:29 smithi082 ceph-mon[98694]: 3 slow requests, 3 included below; oldest blocked for > 1440.766477 secs 2023-09-09T23:43:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:29 smithi082 ceph-mon[98694]: slow request 1440.766477 seconds old, received at 2023-09-09T23:19:27.740800+0000: peer_request:mds.4:47 currently acquired locks 2023-09-09T23:43:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:29 smithi082 ceph-mon[98694]: slow request 1440.766243 seconds old, received at 2023-09-09T23:19:27.741034+0000: peer_request(mds.4:48.0 authpin) currently dispatched 2023-09-09T23:43:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:29 smithi082 ceph-mon[98694]: slow request 1440.764207 seconds old, received at 2023-09-09T23:19:27.743070+0000: peer_request(client.24913:3869.0 authpin) currently dispatched 2023-09-09T23:43:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:29 smithi098 ceph-mon[103124]: 3 slow requests, 3 included below; oldest blocked for > 1440.766477 secs 2023-09-09T23:43:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:29 smithi098 ceph-mon[103124]: slow request 1440.766477 seconds old, received at 2023-09-09T23:19:27.740800+0000: peer_request:mds.4:47 currently acquired locks 2023-09-09T23:43:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:29 smithi098 ceph-mon[103124]: slow request 1440.766243 seconds old, received at 2023-09-09T23:19:27.741034+0000: peer_request(mds.4:48.0 authpin) currently dispatched 2023-09-09T23:43:30.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:29 smithi098 ceph-mon[103124]: slow request 1440.764207 seconds old, received at 2023-09-09T23:19:27.743070+0000: peer_request(client.24913:3869.0 authpin) currently dispatched 2023-09-09T23:43:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:30 smithi170 ceph-mon[102832]: pgmap v939: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 KiB/s wr, 13 op/s 2023-09-09T23:43:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:30 smithi082 ceph-mon[98694]: pgmap v939: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 KiB/s wr, 13 op/s 2023-09-09T23:43:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:30 smithi098 ceph-mon[103124]: pgmap v939: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 KiB/s wr, 13 op/s 2023-09-09T23:43:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:32 smithi170 ceph-mon[102832]: 1 slow requests, 1 included below; oldest blocked for > 1443.438939 secs 2023-09-09T23:43:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:32 smithi170 ceph-mon[102832]: slow request 1443.438939 seconds old, received at 2023-09-09T23:19:28.303507+0000: peer_request:mds.4:47 currently dispatched 2023-09-09T23:43:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:32 smithi082 ceph-mon[98694]: 1 slow requests, 1 included below; oldest blocked for > 1443.438939 secs 2023-09-09T23:43:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:32 smithi082 ceph-mon[98694]: slow request 1443.438939 seconds old, received at 2023-09-09T23:19:28.303507+0000: peer_request:mds.4:47 currently dispatched 2023-09-09T23:43:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:32 smithi098 ceph-mon[103124]: 1 slow requests, 1 included below; oldest blocked for > 1443.438939 secs 2023-09-09T23:43:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:32 smithi098 ceph-mon[103124]: slow request 1443.438939 seconds old, received at 2023-09-09T23:19:28.303507+0000: peer_request:mds.4:47 currently dispatched 2023-09-09T23:43:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:33 smithi170 ceph-mon[102832]: pgmap v940: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-09T23:43:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1445.037334 secs 2023-09-09T23:43:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:33 smithi082 ceph-mon[98694]: pgmap v940: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-09T23:43:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1445.037334 secs 2023-09-09T23:43:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:33 smithi098 ceph-mon[103124]: pgmap v940: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-09T23:43:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1445.037334 secs 2023-09-09T23:43:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1445.766546 secs 2023-09-09T23:43:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:34 smithi170 ceph-mon[102832]: pgmap v941: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-09T23:43:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1445.766546 secs 2023-09-09T23:43:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:34 smithi082 ceph-mon[98694]: pgmap v941: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-09T23:43:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1445.766546 secs 2023-09-09T23:43:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:34 smithi098 ceph-mon[103124]: pgmap v941: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-09T23:43:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:36 smithi170 ceph-mon[102832]: pgmap v942: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 KiB/s wr, 13 op/s 2023-09-09T23:43:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:36 smithi082 ceph-mon[98694]: pgmap v942: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 KiB/s wr, 13 op/s 2023-09-09T23:43:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:36 smithi098 ceph-mon[103124]: pgmap v942: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 KiB/s wr, 13 op/s 2023-09-09T23:43:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1448.439027 secs 2023-09-09T23:43:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1448.439027 secs 2023-09-09T23:43:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1448.439027 secs 2023-09-09T23:43:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1450.037391 secs 2023-09-09T23:43:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:38 smithi170 ceph-mon[102832]: pgmap v943: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 8 op/s 2023-09-09T23:43:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1450.037391 secs 2023-09-09T23:43:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:38 smithi082 ceph-mon[98694]: pgmap v943: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 8 op/s 2023-09-09T23:43:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1450.037391 secs 2023-09-09T23:43:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:38 smithi098 ceph-mon[103124]: pgmap v943: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 8 op/s 2023-09-09T23:43:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1450.766610 secs 2023-09-09T23:43:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1450.766610 secs 2023-09-09T23:43:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1450.766610 secs 2023-09-09T23:43:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:40 smithi170 ceph-mon[102832]: pgmap v944: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-09-09T23:43:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:40 smithi082 ceph-mon[98694]: pgmap v944: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-09-09T23:43:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:40 smithi098 ceph-mon[103124]: pgmap v944: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-09-09T23:43:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1453.439144 secs 2023-09-09T23:43:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1453.439144 secs 2023-09-09T23:43:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1453.439144 secs 2023-09-09T23:43:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:43 smithi170 ceph-mon[102832]: pgmap v945: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-09T23:43:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1455.037449 secs 2023-09-09T23:43:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:43 smithi082 ceph-mon[98694]: pgmap v945: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-09T23:43:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1455.037449 secs 2023-09-09T23:43:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:43 smithi098 ceph-mon[103124]: pgmap v945: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-09T23:43:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1455.037449 secs 2023-09-09T23:43:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1455.766676 secs 2023-09-09T23:43:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:44 smithi170 ceph-mon[102832]: pgmap v946: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2023-09-09T23:43:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1455.766676 secs 2023-09-09T23:43:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:44 smithi082 ceph-mon[98694]: pgmap v946: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2023-09-09T23:43:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1455.766676 secs 2023-09-09T23:43:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:44 smithi098 ceph-mon[103124]: pgmap v946: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2023-09-09T23:43:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:43:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:43:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:43:47.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:43:47.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:43:47.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:43:47.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:46 smithi170 ceph-mon[102832]: pgmap v947: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 170 B/s wr, 12 op/s 2023-09-09T23:43:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:43:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:43:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:43:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:46 smithi082 ceph-mon[98694]: pgmap v947: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 170 B/s wr, 12 op/s 2023-09-09T23:43:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:43:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:43:47.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:43:47.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:46 smithi098 ceph-mon[103124]: pgmap v947: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 170 B/s wr, 12 op/s 2023-09-09T23:43:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1458.439257 secs 2023-09-09T23:43:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1458.439257 secs 2023-09-09T23:43:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1458.439257 secs 2023-09-09T23:43:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1460.037494 secs 2023-09-09T23:43:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:48 smithi170 ceph-mon[102832]: pgmap v948: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-09T23:43:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1460.037494 secs 2023-09-09T23:43:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:48 smithi082 ceph-mon[98694]: pgmap v948: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-09T23:43:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1460.037494 secs 2023-09-09T23:43:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:48 smithi098 ceph-mon[103124]: pgmap v948: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-09T23:43:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1460.766740 secs 2023-09-09T23:43:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1460.766740 secs 2023-09-09T23:43:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1460.766740 secs 2023-09-09T23:43:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:50 smithi170 ceph-mon[102832]: pgmap v949: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.8 KiB/s wr, 12 op/s 2023-09-09T23:43:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:50 smithi082 ceph-mon[98694]: pgmap v949: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.8 KiB/s wr, 12 op/s 2023-09-09T23:43:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:50 smithi098 ceph-mon[103124]: pgmap v949: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.8 KiB/s wr, 12 op/s 2023-09-09T23:43:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1463.439315 secs 2023-09-09T23:43:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1463.439315 secs 2023-09-09T23:43:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1463.439315 secs 2023-09-09T23:43:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:53 smithi170 ceph-mon[102832]: pgmap v950: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2023-09-09T23:43:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1465.037601 secs 2023-09-09T23:43:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:53 smithi082 ceph-mon[98694]: pgmap v950: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2023-09-09T23:43:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1465.037601 secs 2023-09-09T23:43:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:53 smithi098 ceph-mon[103124]: pgmap v950: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2023-09-09T23:43:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1465.037601 secs 2023-09-09T23:43:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1465.766805 secs 2023-09-09T23:43:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:54 smithi170 ceph-mon[102832]: pgmap v951: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2023-09-09T23:43:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1465.766805 secs 2023-09-09T23:43:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:54 smithi082 ceph-mon[98694]: pgmap v951: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2023-09-09T23:43:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1465.766805 secs 2023-09-09T23:43:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:54 smithi098 ceph-mon[103124]: pgmap v951: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2023-09-09T23:43:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:56 smithi170 ceph-mon[102832]: pgmap v952: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 KiB/s wr, 14 op/s 2023-09-09T23:43:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:56 smithi082 ceph-mon[98694]: pgmap v952: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 KiB/s wr, 14 op/s 2023-09-09T23:43:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:56 smithi098 ceph-mon[103124]: pgmap v952: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 KiB/s wr, 14 op/s 2023-09-09T23:43:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1468.439376 secs 2023-09-09T23:43:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1468.439376 secs 2023-09-09T23:43:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1468.439376 secs 2023-09-09T23:43:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1470.037660 secs 2023-09-09T23:43:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:58 smithi170 ceph-mon[102832]: pgmap v953: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-09T23:43:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1470.037660 secs 2023-09-09T23:43:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:58 smithi082 ceph-mon[98694]: pgmap v953: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-09T23:43:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1470.037660 secs 2023-09-09T23:43:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:58 smithi098 ceph-mon[103124]: pgmap v953: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-09T23:44:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:43:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1470.766870 secs 2023-09-09T23:44:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:43:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1470.766870 secs 2023-09-09T23:44:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:43:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1470.766870 secs 2023-09-09T23:44:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:00 smithi170 ceph-mon[102832]: pgmap v954: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 14 op/s 2023-09-09T23:44:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:00 smithi082 ceph-mon[98694]: pgmap v954: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 14 op/s 2023-09-09T23:44:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:00 smithi098 ceph-mon[103124]: pgmap v954: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 14 op/s 2023-09-09T23:44:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1473.439437 secs 2023-09-09T23:44:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1473.439437 secs 2023-09-09T23:44:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1473.439437 secs 2023-09-09T23:44:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:03 smithi170 ceph-mon[102832]: pgmap v955: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 10 op/s 2023-09-09T23:44:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1475.037720 secs 2023-09-09T23:44:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:03 smithi082 ceph-mon[98694]: pgmap v955: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 10 op/s 2023-09-09T23:44:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1475.037720 secs 2023-09-09T23:44:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:03 smithi098 ceph-mon[103124]: pgmap v955: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 10 op/s 2023-09-09T23:44:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1475.037720 secs 2023-09-09T23:44:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1475.766936 secs 2023-09-09T23:44:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:04 smithi170 ceph-mon[102832]: pgmap v956: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 8 op/s 2023-09-09T23:44:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1475.766936 secs 2023-09-09T23:44:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:04 smithi082 ceph-mon[98694]: pgmap v956: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 8 op/s 2023-09-09T23:44:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1475.766936 secs 2023-09-09T23:44:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:04 smithi098 ceph-mon[103124]: pgmap v956: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 8 op/s 2023-09-09T23:44:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:06 smithi170 ceph-mon[102832]: pgmap v957: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 56 KiB/s wr, 27 op/s 2023-09-09T23:44:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:06 smithi082 ceph-mon[98694]: pgmap v957: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 56 KiB/s wr, 27 op/s 2023-09-09T23:44:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:06 smithi098 ceph-mon[103124]: pgmap v957: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 56 KiB/s wr, 27 op/s 2023-09-09T23:44:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1478.439500 secs 2023-09-09T23:44:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:07 smithi170 ceph-mon[102832]: osdmap e133: 12 total, 12 up, 12 in 2023-09-09T23:44:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1478.439500 secs 2023-09-09T23:44:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:07 smithi082 ceph-mon[98694]: osdmap e133: 12 total, 12 up, 12 in 2023-09-09T23:44:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1478.439500 secs 2023-09-09T23:44:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:07 smithi098 ceph-mon[103124]: osdmap e133: 12 total, 12 up, 12 in 2023-09-09T23:44:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1480.037780 secs 2023-09-09T23:44:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:08 smithi170 ceph-mon[102832]: pgmap v959: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:44:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1480.037780 secs 2023-09-09T23:44:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:08 smithi082 ceph-mon[98694]: pgmap v959: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:44:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1480.037780 secs 2023-09-09T23:44:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:08 smithi098 ceph-mon[103124]: pgmap v959: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:44:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1480.767002 secs 2023-09-09T23:44:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1480.767002 secs 2023-09-09T23:44:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1480.767002 secs 2023-09-09T23:44:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:10 smithi170 ceph-mon[102832]: pgmap v960: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:44:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:10 smithi082 ceph-mon[98694]: pgmap v960: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:44:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:10 smithi098 ceph-mon[103124]: pgmap v960: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:44:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1483.439559 secs 2023-09-09T23:44:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1483.439559 secs 2023-09-09T23:44:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1483.439559 secs 2023-09-09T23:44:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:13 smithi170 ceph-mon[102832]: pgmap v961: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:44:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1485.037868 secs 2023-09-09T23:44:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:13 smithi082 ceph-mon[98694]: pgmap v961: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:44:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1485.037868 secs 2023-09-09T23:44:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:13 smithi098 ceph-mon[103124]: pgmap v961: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:44:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1485.037868 secs 2023-09-09T23:44:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1485.767067 secs 2023-09-09T23:44:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:14 smithi170 ceph-mon[102832]: osdmap e134: 12 total, 12 up, 12 in 2023-09-09T23:44:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:14 smithi170 ceph-mon[102832]: pgmap v963: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:44:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1485.767067 secs 2023-09-09T23:44:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:14 smithi082 ceph-mon[98694]: osdmap e134: 12 total, 12 up, 12 in 2023-09-09T23:44:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:14 smithi082 ceph-mon[98694]: pgmap v963: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:44:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1485.767067 secs 2023-09-09T23:44:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:14 smithi098 ceph-mon[103124]: osdmap e134: 12 total, 12 up, 12 in 2023-09-09T23:44:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:14 smithi098 ceph-mon[103124]: pgmap v963: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:44:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:16 smithi170 ceph-mon[102832]: pgmap v964: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-09T23:44:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:16 smithi082 ceph-mon[98694]: pgmap v964: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-09T23:44:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:16 smithi098 ceph-mon[103124]: pgmap v964: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-09T23:44:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1488.439618 secs 2023-09-09T23:44:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1488.439618 secs 2023-09-09T23:44:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1488.439618 secs 2023-09-09T23:44:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1490.037929 secs 2023-09-09T23:44:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:18 smithi170 ceph-mon[102832]: pgmap v965: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-09T23:44:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1490.037929 secs 2023-09-09T23:44:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:18 smithi082 ceph-mon[98694]: pgmap v965: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-09T23:44:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1490.037929 secs 2023-09-09T23:44:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:18 smithi098 ceph-mon[103124]: pgmap v965: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-09T23:44:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1490.767132 secs 2023-09-09T23:44:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1490.767132 secs 2023-09-09T23:44:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1490.767132 secs 2023-09-09T23:44:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:20 smithi170 ceph-mon[102832]: pgmap v966: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 35 KiB/s wr, 13 op/s 2023-09-09T23:44:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:20 smithi082 ceph-mon[98694]: pgmap v966: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 35 KiB/s wr, 13 op/s 2023-09-09T23:44:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:20 smithi098 ceph-mon[103124]: pgmap v966: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 35 KiB/s wr, 13 op/s 2023-09-09T23:44:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1493.439678 secs 2023-09-09T23:44:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1493.439678 secs 2023-09-09T23:44:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1493.439678 secs 2023-09-09T23:44:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:23 smithi170 ceph-mon[102832]: pgmap v967: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 35 KiB/s wr, 13 op/s 2023-09-09T23:44:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1495.037990 secs 2023-09-09T23:44:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:23 smithi082 ceph-mon[98694]: pgmap v967: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 35 KiB/s wr, 13 op/s 2023-09-09T23:44:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1495.037990 secs 2023-09-09T23:44:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:23 smithi098 ceph-mon[103124]: pgmap v967: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 35 KiB/s wr, 13 op/s 2023-09-09T23:44:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1495.037990 secs 2023-09-09T23:44:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1495.767198 secs 2023-09-09T23:44:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:24 smithi170 ceph-mon[102832]: pgmap v968: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-09T23:44:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1495.767198 secs 2023-09-09T23:44:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:24 smithi082 ceph-mon[98694]: pgmap v968: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-09T23:44:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1495.767198 secs 2023-09-09T23:44:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:24 smithi098 ceph-mon[103124]: pgmap v968: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-09T23:44:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:26 smithi170 ceph-mon[102832]: pgmap v969: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 30 KiB/s wr, 15 op/s 2023-09-09T23:44:27.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:26 smithi082 ceph-mon[98694]: pgmap v969: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 30 KiB/s wr, 15 op/s 2023-09-09T23:44:28.337 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1498.439768 secs 2023-09-09T23:44:28.337 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1498.439768 secs 2023-09-09T23:44:28.338 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:26 smithi098 ceph-mon[103124]: pgmap v969: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 30 KiB/s wr, 15 op/s 2023-09-09T23:44:28.338 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1498.439768 secs 2023-09-09T23:44:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1500.038048 secs 2023-09-09T23:44:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:28 smithi170 ceph-mon[102832]: pgmap v970: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 17 KiB/s wr, 10 op/s 2023-09-09T23:44:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1500.038048 secs 2023-09-09T23:44:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:28 smithi082 ceph-mon[98694]: pgmap v970: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 17 KiB/s wr, 10 op/s 2023-09-09T23:44:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1500.038048 secs 2023-09-09T23:44:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:28 smithi098 ceph-mon[103124]: pgmap v970: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 17 KiB/s wr, 10 op/s 2023-09-09T23:44:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1500.767263 secs 2023-09-09T23:44:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1500.767263 secs 2023-09-09T23:44:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1500.767263 secs 2023-09-09T23:44:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:30 smithi170 ceph-mon[102832]: pgmap v971: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 24 KiB/s wr, 14 op/s 2023-09-09T23:44:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:30 smithi082 ceph-mon[98694]: pgmap v971: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 24 KiB/s wr, 14 op/s 2023-09-09T23:44:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:30 smithi098 ceph-mon[103124]: pgmap v971: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 24 KiB/s wr, 14 op/s 2023-09-09T23:44:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1503.439826 secs 2023-09-09T23:44:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1503.439826 secs 2023-09-09T23:44:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1503.439826 secs 2023-09-09T23:44:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:33 smithi170 ceph-mon[102832]: pgmap v972: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2023-09-09T23:44:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1505.038108 secs 2023-09-09T23:44:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:33 smithi082 ceph-mon[98694]: pgmap v972: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2023-09-09T23:44:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1505.038108 secs 2023-09-09T23:44:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:33 smithi098 ceph-mon[103124]: pgmap v972: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2023-09-09T23:44:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1505.038108 secs 2023-09-09T23:44:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1505.767328 secs 2023-09-09T23:44:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:34 smithi170 ceph-mon[102832]: pgmap v973: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2023-09-09T23:44:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1505.767328 secs 2023-09-09T23:44:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:34 smithi082 ceph-mon[98694]: pgmap v973: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2023-09-09T23:44:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1505.767328 secs 2023-09-09T23:44:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:34 smithi098 ceph-mon[103124]: pgmap v973: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2023-09-09T23:44:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:36 smithi170 ceph-mon[102832]: pgmap v974: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.2 KiB/s wr, 13 op/s 2023-09-09T23:44:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:36 smithi082 ceph-mon[98694]: pgmap v974: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.2 KiB/s wr, 13 op/s 2023-09-09T23:44:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:36 smithi098 ceph-mon[103124]: pgmap v974: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.2 KiB/s wr, 13 op/s 2023-09-09T23:44:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1508.439886 secs 2023-09-09T23:44:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1508.439886 secs 2023-09-09T23:44:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1508.439886 secs 2023-09-09T23:44:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1510.038166 secs 2023-09-09T23:44:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:38 smithi170 ceph-mon[102832]: pgmap v975: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 8 op/s 2023-09-09T23:44:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1510.038166 secs 2023-09-09T23:44:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:38 smithi082 ceph-mon[98694]: pgmap v975: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 8 op/s 2023-09-09T23:44:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1510.038166 secs 2023-09-09T23:44:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:38 smithi098 ceph-mon[103124]: pgmap v975: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 8 op/s 2023-09-09T23:44:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1510.767394 secs 2023-09-09T23:44:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1510.767394 secs 2023-09-09T23:44:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1510.767394 secs 2023-09-09T23:44:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:40 smithi170 ceph-mon[102832]: pgmap v976: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.0 KiB/s wr, 12 op/s 2023-09-09T23:44:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:40 smithi082 ceph-mon[98694]: pgmap v976: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.0 KiB/s wr, 12 op/s 2023-09-09T23:44:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:40 smithi098 ceph-mon[103124]: pgmap v976: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.0 KiB/s wr, 12 op/s 2023-09-09T23:44:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1513.439974 secs 2023-09-09T23:44:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1513.439974 secs 2023-09-09T23:44:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1513.439974 secs 2023-09-09T23:44:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:43 smithi170 ceph-mon[102832]: pgmap v977: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:44:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1515.038225 secs 2023-09-09T23:44:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:43 smithi082 ceph-mon[98694]: pgmap v977: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:44:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1515.038225 secs 2023-09-09T23:44:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:43 smithi098 ceph-mon[103124]: pgmap v977: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:44:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1515.038225 secs 2023-09-09T23:44:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1515.767461 secs 2023-09-09T23:44:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:44 smithi170 ceph-mon[102832]: pgmap v978: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:44:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1515.767461 secs 2023-09-09T23:44:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:44 smithi082 ceph-mon[98694]: pgmap v978: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:44:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1515.767461 secs 2023-09-09T23:44:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:44 smithi098 ceph-mon[103124]: pgmap v978: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:44:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:44:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:44:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:44:47.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:47 smithi170 ceph-mon[102832]: pgmap v979: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:44:47.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:47 smithi082 ceph-mon[98694]: pgmap v979: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:44:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:47 smithi098 ceph-mon[103124]: pgmap v979: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:44:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1518.440037 secs 2023-09-09T23:44:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1520.038287 secs 2023-09-09T23:44:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1518.440037 secs 2023-09-09T23:44:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1520.038287 secs 2023-09-09T23:44:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1518.440037 secs 2023-09-09T23:44:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1520.038287 secs 2023-09-09T23:44:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1520.767493 secs 2023-09-09T23:44:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:49.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:49.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:49.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:49.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:49 smithi170 ceph-mon[102832]: pgmap v980: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:44:49.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:49.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:49.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:49.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:49.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:49.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:49.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:49.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:49.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:49.932 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:49.932 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:49.932 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1520.767493 secs 2023-09-09T23:44:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:49.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:49.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:49 smithi082 ceph-mon[98694]: pgmap v980: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:44:49.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:49.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:49.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:49.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:49.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:49.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:49.962 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:49.962 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:49.963 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:49.963 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:49.963 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:49.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1520.767493 secs 2023-09-09T23:44:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:50.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:50.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:50.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:50.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:49 smithi098 ceph-mon[103124]: pgmap v980: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:44:50.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:50.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:50.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:50.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:50.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:50.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:50.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:50.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:50.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:50.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:50.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:50.008 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:50 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-09T23:44:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:50 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-09T23:44:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:50 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-09T23:44:50.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:50 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-09T23:44:50.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:50.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:50 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-09T23:44:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:50 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-09T23:44:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:50 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-09T23:44:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:50 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-09T23:44:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:50.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:50 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-09T23:44:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:50 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-09T23:44:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:50 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-09T23:44:51.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:50 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-09T23:44:51.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:51.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:52 smithi170 ceph-mon[102832]: pgmap v981: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.8 KiB/s wr, 13 op/s 2023-09-09T23:44:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1523.440149 secs 2023-09-09T23:44:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:52 smithi082 ceph-mon[98694]: pgmap v981: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.8 KiB/s wr, 13 op/s 2023-09-09T23:44:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1523.440149 secs 2023-09-09T23:44:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:52 smithi098 ceph-mon[103124]: pgmap v981: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.8 KiB/s wr, 13 op/s 2023-09-09T23:44:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1523.440149 secs 2023-09-09T23:44:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1525.038346 secs 2023-09-09T23:44:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1525.038346 secs 2023-09-09T23:44:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1525.038346 secs 2023-09-09T23:44:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:54 smithi170 ceph-mon[102832]: pgmap v982: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2023-09-09T23:44:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1525.767558 secs 2023-09-09T23:44:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:54.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:54.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:54.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:54 smithi082 ceph-mon[98694]: pgmap v982: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2023-09-09T23:44:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1525.767558 secs 2023-09-09T23:44:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:54 smithi098 ceph-mon[103124]: pgmap v982: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2023-09-09T23:44:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1525.767558 secs 2023-09-09T23:44:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:55.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:55.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:55.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:55 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-09T23:44:55.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:55 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-09T23:44:55.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:44:55.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:44:55.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:55 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-09T23:44:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:55 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-09T23:44:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:44:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:44:55.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:44:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:55 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-09T23:44:56.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:55 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-09T23:44:56.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:44:56.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:44:56.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:44:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:56 smithi170 ceph-mon[102832]: pgmap v983: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2023-09-09T23:44:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:56 smithi082 ceph-mon[98694]: pgmap v983: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2023-09-09T23:44:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:56 smithi098 ceph-mon[103124]: pgmap v983: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2023-09-09T23:44:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1528.440208 secs 2023-09-09T23:44:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1528.440208 secs 2023-09-09T23:44:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1528.440208 secs 2023-09-09T23:44:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:58 smithi170 ceph-mon[102832]: pgmap v984: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 15 KiB/s wr, 14 op/s 2023-09-09T23:44:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1530.038426 secs 2023-09-09T23:44:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:58 smithi082 ceph-mon[98694]: pgmap v984: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 15 KiB/s wr, 14 op/s 2023-09-09T23:44:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1530.038426 secs 2023-09-09T23:44:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:58 smithi098 ceph-mon[103124]: pgmap v984: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 15 KiB/s wr, 14 op/s 2023-09-09T23:44:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1530.038426 secs 2023-09-09T23:45:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1530.767623 secs 2023-09-09T23:45:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:44:59 smithi170 ceph-mon[102832]: pgmap v985: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 15 KiB/s wr, 10 op/s 2023-09-09T23:45:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1530.767623 secs 2023-09-09T23:45:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:44:59 smithi082 ceph-mon[98694]: pgmap v985: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 15 KiB/s wr, 10 op/s 2023-09-09T23:45:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1530.767623 secs 2023-09-09T23:45:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:44:59 smithi098 ceph-mon[103124]: pgmap v985: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 15 KiB/s wr, 10 op/s 2023-09-09T23:45:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:01 smithi170 ceph-mon[102832]: pgmap v986: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 14 op/s 2023-09-09T23:45:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:01 smithi082 ceph-mon[98694]: pgmap v986: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 14 op/s 2023-09-09T23:45:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:01 smithi098 ceph-mon[103124]: pgmap v986: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 14 op/s 2023-09-09T23:45:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1533.440265 secs 2023-09-09T23:45:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1533.440265 secs 2023-09-09T23:45:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1533.440265 secs 2023-09-09T23:45:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:03 smithi170 ceph-mon[102832]: pgmap v987: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 9 op/s 2023-09-09T23:45:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1535.038532 secs 2023-09-09T23:45:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:03 smithi082 ceph-mon[98694]: pgmap v987: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 9 op/s 2023-09-09T23:45:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1535.038532 secs 2023-09-09T23:45:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:03 smithi098 ceph-mon[103124]: pgmap v987: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 9 op/s 2023-09-09T23:45:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1535.038532 secs 2023-09-09T23:45:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1535.767690 secs 2023-09-09T23:45:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1535.767690 secs 2023-09-09T23:45:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1535.767690 secs 2023-09-09T23:45:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:05 smithi170 ceph-mon[102832]: pgmap v988: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 9 op/s 2023-09-09T23:45:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:05 smithi082 ceph-mon[98694]: pgmap v988: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 9 op/s 2023-09-09T23:45:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:05 smithi098 ceph-mon[103124]: pgmap v988: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 9 op/s 2023-09-09T23:45:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:07 smithi170 ceph-mon[102832]: pgmap v989: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 56 KiB/s wr, 28 op/s 2023-09-09T23:45:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1538.440340 secs 2023-09-09T23:45:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:07 smithi170 ceph-mon[102832]: osdmap e135: 12 total, 12 up, 12 in 2023-09-09T23:45:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:07 smithi082 ceph-mon[98694]: pgmap v989: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 56 KiB/s wr, 28 op/s 2023-09-09T23:45:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1538.440340 secs 2023-09-09T23:45:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:07 smithi082 ceph-mon[98694]: osdmap e135: 12 total, 12 up, 12 in 2023-09-09T23:45:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:07 smithi098 ceph-mon[103124]: pgmap v989: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 56 KiB/s wr, 28 op/s 2023-09-09T23:45:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1538.440340 secs 2023-09-09T23:45:08.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:07 smithi098 ceph-mon[103124]: osdmap e135: 12 total, 12 up, 12 in 2023-09-09T23:45:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1540.038588 secs 2023-09-09T23:45:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1540.038588 secs 2023-09-09T23:45:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1540.038588 secs 2023-09-09T23:45:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1540.767758 secs 2023-09-09T23:45:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:09 smithi170 ceph-mon[102832]: pgmap v991: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:45:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1540.767758 secs 2023-09-09T23:45:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:09 smithi082 ceph-mon[98694]: pgmap v991: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:45:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1540.767758 secs 2023-09-09T23:45:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:09 smithi098 ceph-mon[103124]: pgmap v991: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:45:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:11 smithi170 ceph-mon[102832]: pgmap v992: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-09-09T23:45:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:11 smithi082 ceph-mon[98694]: pgmap v992: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-09-09T23:45:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:11 smithi098 ceph-mon[103124]: pgmap v992: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-09-09T23:45:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1543.440399 secs 2023-09-09T23:45:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1543.440399 secs 2023-09-09T23:45:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1543.440399 secs 2023-09-09T23:45:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:13 smithi170 ceph-mon[102832]: pgmap v993: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-09-09T23:45:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1545.038650 secs 2023-09-09T23:45:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:13 smithi170 ceph-mon[102832]: osdmap e136: 12 total, 12 up, 12 in 2023-09-09T23:45:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:13 smithi082 ceph-mon[98694]: pgmap v993: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-09-09T23:45:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1545.038650 secs 2023-09-09T23:45:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:13 smithi082 ceph-mon[98694]: osdmap e136: 12 total, 12 up, 12 in 2023-09-09T23:45:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:13 smithi098 ceph-mon[103124]: pgmap v993: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-09-09T23:45:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1545.038650 secs 2023-09-09T23:45:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:13 smithi098 ceph-mon[103124]: osdmap e136: 12 total, 12 up, 12 in 2023-09-09T23:45:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1545.767824 secs 2023-09-09T23:45:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1545.767824 secs 2023-09-09T23:45:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1545.767824 secs 2023-09-09T23:45:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:15 smithi170 ceph-mon[102832]: pgmap v995: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-09T23:45:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:15 smithi082 ceph-mon[98694]: pgmap v995: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-09T23:45:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:15 smithi098 ceph-mon[103124]: pgmap v995: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-09T23:45:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:17 smithi170 ceph-mon[102832]: pgmap v996: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.8 KiB/s wr, 12 op/s 2023-09-09T23:45:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1548.440466 secs 2023-09-09T23:45:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:17 smithi082 ceph-mon[98694]: pgmap v996: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.8 KiB/s wr, 12 op/s 2023-09-09T23:45:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1548.440466 secs 2023-09-09T23:45:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:17 smithi098 ceph-mon[103124]: pgmap v996: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.8 KiB/s wr, 12 op/s 2023-09-09T23:45:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1548.440466 secs 2023-09-09T23:45:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1550.038708 secs 2023-09-09T23:45:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1550.038708 secs 2023-09-09T23:45:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1550.038708 secs 2023-09-09T23:45:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1550.767891 secs 2023-09-09T23:45:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:19 smithi170 ceph-mon[102832]: pgmap v997: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.8 KiB/s wr, 11 op/s 2023-09-09T23:45:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1550.767891 secs 2023-09-09T23:45:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:19 smithi082 ceph-mon[98694]: pgmap v997: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.8 KiB/s wr, 11 op/s 2023-09-09T23:45:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1550.767891 secs 2023-09-09T23:45:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:19 smithi098 ceph-mon[103124]: pgmap v997: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.8 KiB/s wr, 11 op/s 2023-09-09T23:45:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:21 smithi170 ceph-mon[102832]: pgmap v998: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-09T23:45:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:21 smithi082 ceph-mon[98694]: pgmap v998: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-09T23:45:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:21 smithi098 ceph-mon[103124]: pgmap v998: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-09T23:45:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1553.440564 secs 2023-09-09T23:45:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1553.440564 secs 2023-09-09T23:45:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1553.440564 secs 2023-09-09T23:45:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:23 smithi170 ceph-mon[102832]: pgmap v999: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-09T23:45:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1555.038766 secs 2023-09-09T23:45:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:23 smithi082 ceph-mon[98694]: pgmap v999: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-09T23:45:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1555.038766 secs 2023-09-09T23:45:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:23 smithi098 ceph-mon[103124]: pgmap v999: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-09T23:45:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1555.038766 secs 2023-09-09T23:45:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1555.767958 secs 2023-09-09T23:45:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1555.767958 secs 2023-09-09T23:45:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1555.767958 secs 2023-09-09T23:45:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:25 smithi170 ceph-mon[102832]: pgmap v1000: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-09T23:45:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:25 smithi082 ceph-mon[98694]: pgmap v1000: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-09T23:45:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:25 smithi098 ceph-mon[103124]: pgmap v1000: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-09T23:45:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:27 smithi170 ceph-mon[102832]: pgmap v1001: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 KiB/s wr, 15 op/s 2023-09-09T23:45:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1558.440658 secs 2023-09-09T23:45:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:27 smithi082 ceph-mon[98694]: pgmap v1001: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 KiB/s wr, 15 op/s 2023-09-09T23:45:28.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1558.440658 secs 2023-09-09T23:45:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:27 smithi098 ceph-mon[103124]: pgmap v1001: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 KiB/s wr, 15 op/s 2023-09-09T23:45:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1558.440658 secs 2023-09-09T23:45:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1560.038845 secs 2023-09-09T23:45:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1560.038845 secs 2023-09-09T23:45:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1560.038845 secs 2023-09-09T23:45:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1560.768024 secs 2023-09-09T23:45:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:29 smithi170 ceph-mon[102832]: pgmap v1002: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 KiB/s wr, 9 op/s 2023-09-09T23:45:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1560.768024 secs 2023-09-09T23:45:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:29 smithi082 ceph-mon[98694]: pgmap v1002: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 KiB/s wr, 9 op/s 2023-09-09T23:45:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1560.768024 secs 2023-09-09T23:45:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:29 smithi098 ceph-mon[103124]: pgmap v1002: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 KiB/s wr, 9 op/s 2023-09-09T23:45:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:31 smithi170 ceph-mon[102832]: pgmap v1003: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 24 KiB/s wr, 14 op/s 2023-09-09T23:45:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:31 smithi082 ceph-mon[98694]: pgmap v1003: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 24 KiB/s wr, 14 op/s 2023-09-09T23:45:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:31 smithi098 ceph-mon[103124]: pgmap v1003: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 24 KiB/s wr, 14 op/s 2023-09-09T23:45:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1563.440760 secs 2023-09-09T23:45:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1563.440760 secs 2023-09-09T23:45:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1563.440760 secs 2023-09-09T23:45:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:33 smithi170 ceph-mon[102832]: pgmap v1004: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.3 KiB/s wr, 8 op/s 2023-09-09T23:45:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1565.038904 secs 2023-09-09T23:45:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:33 smithi082 ceph-mon[98694]: pgmap v1004: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.3 KiB/s wr, 8 op/s 2023-09-09T23:45:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1565.038904 secs 2023-09-09T23:45:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:33 smithi098 ceph-mon[103124]: pgmap v1004: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.3 KiB/s wr, 8 op/s 2023-09-09T23:45:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1565.038904 secs 2023-09-09T23:45:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1565.768089 secs 2023-09-09T23:45:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1565.768089 secs 2023-09-09T23:45:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1565.768089 secs 2023-09-09T23:45:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:35 smithi170 ceph-mon[102832]: pgmap v1005: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.3 KiB/s wr, 8 op/s 2023-09-09T23:45:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:35 smithi082 ceph-mon[98694]: pgmap v1005: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.3 KiB/s wr, 8 op/s 2023-09-09T23:45:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:35 smithi098 ceph-mon[103124]: pgmap v1005: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.3 KiB/s wr, 8 op/s 2023-09-09T23:45:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1568.440777 secs 2023-09-09T23:45:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:37 smithi170 ceph-mon[102832]: pgmap v1006: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.3 KiB/s wr, 13 op/s 2023-09-09T23:45:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1568.440777 secs 2023-09-09T23:45:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:37 smithi082 ceph-mon[98694]: pgmap v1006: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.3 KiB/s wr, 13 op/s 2023-09-09T23:45:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1568.440777 secs 2023-09-09T23:45:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:37 smithi098 ceph-mon[103124]: pgmap v1006: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.3 KiB/s wr, 13 op/s 2023-09-09T23:45:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1570.038962 secs 2023-09-09T23:45:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1570.038962 secs 2023-09-09T23:45:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1570.038962 secs 2023-09-09T23:45:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1570.768156 secs 2023-09-09T23:45:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:39 smithi170 ceph-mon[102832]: pgmap v1007: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2023-09-09T23:45:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1570.768156 secs 2023-09-09T23:45:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:39 smithi082 ceph-mon[98694]: pgmap v1007: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2023-09-09T23:45:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1570.768156 secs 2023-09-09T23:45:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:39 smithi098 ceph-mon[103124]: pgmap v1007: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2023-09-09T23:45:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:41 smithi170 ceph-mon[102832]: pgmap v1008: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.2 KiB/s wr, 12 op/s 2023-09-09T23:45:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:41 smithi082 ceph-mon[98694]: pgmap v1008: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.2 KiB/s wr, 12 op/s 2023-09-09T23:45:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:41 smithi098 ceph-mon[103124]: pgmap v1008: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.2 KiB/s wr, 12 op/s 2023-09-09T23:45:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1573.440845 secs 2023-09-09T23:45:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1573.440845 secs 2023-09-09T23:45:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1573.440845 secs 2023-09-09T23:45:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:43 smithi170 ceph-mon[102832]: pgmap v1009: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:45:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1575.039020 secs 2023-09-09T23:45:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:43 smithi082 ceph-mon[98694]: pgmap v1009: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:45:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1575.039020 secs 2023-09-09T23:45:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:43 smithi098 ceph-mon[103124]: pgmap v1009: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:45:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1575.039020 secs 2023-09-09T23:45:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1575.768223 secs 2023-09-09T23:45:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1575.768223 secs 2023-09-09T23:45:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1575.768223 secs 2023-09-09T23:45:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:45 smithi170 ceph-mon[102832]: pgmap v1010: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:45:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:45 smithi082 ceph-mon[98694]: pgmap v1010: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:45:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:45 smithi098 ceph-mon[103124]: pgmap v1010: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:45:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1578.440923 secs 2023-09-09T23:45:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:47 smithi170 ceph-mon[102832]: pgmap v1011: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:45:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1578.440923 secs 2023-09-09T23:45:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:47 smithi082 ceph-mon[98694]: pgmap v1011: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:45:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1578.440923 secs 2023-09-09T23:45:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:47 smithi098 ceph-mon[103124]: pgmap v1011: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:45:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1580.039073 secs 2023-09-09T23:45:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1580.039073 secs 2023-09-09T23:45:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1580.039073 secs 2023-09-09T23:45:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1580.768310 secs 2023-09-09T23:45:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:49 smithi082 ceph-mon[98694]: pgmap v1012: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:45:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1580.768310 secs 2023-09-09T23:45:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:49 smithi098 ceph-mon[103124]: pgmap v1012: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:45:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1580.768310 secs 2023-09-09T23:45:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:49 smithi170 ceph-mon[102832]: pgmap v1012: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:45:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:51 smithi082 ceph-mon[98694]: pgmap v1013: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2023-09-09T23:45:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:51 smithi098 ceph-mon[103124]: pgmap v1013: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2023-09-09T23:45:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:51 smithi170 ceph-mon[102832]: pgmap v1013: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2023-09-09T23:45:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1583.441031 secs 2023-09-09T23:45:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1583.441031 secs 2023-09-09T23:45:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1583.441031 secs 2023-09-09T23:45:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:53 smithi082 ceph-mon[98694]: pgmap v1014: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2023-09-09T23:45:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1585.039133 secs 2023-09-09T23:45:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:53 smithi098 ceph-mon[103124]: pgmap v1014: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2023-09-09T23:45:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1585.039133 secs 2023-09-09T23:45:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:53 smithi170 ceph-mon[102832]: pgmap v1014: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2023-09-09T23:45:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1585.039133 secs 2023-09-09T23:45:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1585.768377 secs 2023-09-09T23:45:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:45:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1585.768377 secs 2023-09-09T23:45:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:45:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1585.768377 secs 2023-09-09T23:45:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:45:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:55 smithi082 ceph-mon[98694]: pgmap v1015: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2023-09-09T23:45:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:45:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:45:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:45:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:55 smithi098 ceph-mon[103124]: pgmap v1015: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2023-09-09T23:45:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:45:56.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:45:56.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:45:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:55 smithi170 ceph-mon[102832]: pgmap v1015: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2023-09-09T23:45:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:45:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:45:56.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:45:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1588.441093 secs 2023-09-09T23:45:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:57 smithi082 ceph-mon[98694]: pgmap v1016: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 16 KiB/s wr, 14 op/s 2023-09-09T23:45:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:57 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1590.039192 secs 2023-09-09T23:45:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1588.441093 secs 2023-09-09T23:45:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:57 smithi098 ceph-mon[103124]: pgmap v1016: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 16 KiB/s wr, 14 op/s 2023-09-09T23:45:58.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:57 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1590.039192 secs 2023-09-09T23:45:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1588.441093 secs 2023-09-09T23:45:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:57 smithi170 ceph-mon[102832]: pgmap v1016: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 16 KiB/s wr, 14 op/s 2023-09-09T23:45:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:57 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1590.039192 secs 2023-09-09T23:46:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1590.768467 secs 2023-09-09T23:46:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:45:59 smithi082 ceph-mon[98694]: pgmap v1017: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-09T23:46:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1590.768467 secs 2023-09-09T23:46:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:45:59 smithi098 ceph-mon[103124]: pgmap v1017: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-09T23:46:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1590.768467 secs 2023-09-09T23:46:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:45:59 smithi170 ceph-mon[102832]: pgmap v1017: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-09T23:46:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:01 smithi082 ceph-mon[98694]: pgmap v1018: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 14 op/s 2023-09-09T23:46:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:01 smithi098 ceph-mon[103124]: pgmap v1018: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 14 op/s 2023-09-09T23:46:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:01 smithi170 ceph-mon[102832]: pgmap v1018: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 14 op/s 2023-09-09T23:46:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1593.441156 secs 2023-09-09T23:46:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:02 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1595.039250 secs 2023-09-09T23:46:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1593.441156 secs 2023-09-09T23:46:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:02 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1595.039250 secs 2023-09-09T23:46:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1593.441156 secs 2023-09-09T23:46:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:02 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1595.039250 secs 2023-09-09T23:46:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:03 smithi098 ceph-mon[103124]: pgmap v1019: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 9 op/s 2023-09-09T23:46:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:03 smithi170 ceph-mon[102832]: pgmap v1019: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 9 op/s 2023-09-09T23:46:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:03 smithi082 ceph-mon[98694]: pgmap v1019: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 9 op/s 2023-09-09T23:46:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1595.768535 secs 2023-09-09T23:46:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1595.768535 secs 2023-09-09T23:46:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1595.768535 secs 2023-09-09T23:46:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:05 smithi098 ceph-mon[103124]: pgmap v1020: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 9 op/s 2023-09-09T23:46:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:05 smithi170 ceph-mon[102832]: pgmap v1020: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 9 op/s 2023-09-09T23:46:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:05 smithi082 ceph-mon[98694]: pgmap v1020: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 9 op/s 2023-09-09T23:46:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1598.441216 secs 2023-09-09T23:46:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:07 smithi098 ceph-mon[103124]: pgmap v1021: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 56 KiB/s wr, 28 op/s 2023-09-09T23:46:08.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:07 smithi098 ceph-mon[103124]: osdmap e137: 12 total, 12 up, 12 in 2023-09-09T23:46:08.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:07 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1600.039309 secs 2023-09-09T23:46:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1598.441216 secs 2023-09-09T23:46:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:07 smithi170 ceph-mon[102832]: pgmap v1021: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 56 KiB/s wr, 28 op/s 2023-09-09T23:46:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:07 smithi170 ceph-mon[102832]: osdmap e137: 12 total, 12 up, 12 in 2023-09-09T23:46:08.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:07 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1600.039309 secs 2023-09-09T23:46:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1598.441216 secs 2023-09-09T23:46:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:07 smithi082 ceph-mon[98694]: pgmap v1021: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 56 KiB/s wr, 28 op/s 2023-09-09T23:46:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:07 smithi082 ceph-mon[98694]: osdmap e137: 12 total, 12 up, 12 in 2023-09-09T23:46:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:07 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1600.039309 secs 2023-09-09T23:46:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1600.768600 secs 2023-09-09T23:46:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:09 smithi098 ceph-mon[103124]: pgmap v1023: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:46:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1600.768600 secs 2023-09-09T23:46:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:09 smithi170 ceph-mon[102832]: pgmap v1023: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:46:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1600.768600 secs 2023-09-09T23:46:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:09 smithi082 ceph-mon[98694]: pgmap v1023: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:46:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:11 smithi098 ceph-mon[103124]: pgmap v1024: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:46:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:11 smithi170 ceph-mon[102832]: pgmap v1024: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:46:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:11 smithi082 ceph-mon[98694]: pgmap v1024: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:46:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1603.441318 secs 2023-09-09T23:46:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:12 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1605.039368 secs 2023-09-09T23:46:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1603.441318 secs 2023-09-09T23:46:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:12 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1605.039368 secs 2023-09-09T23:46:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1603.441318 secs 2023-09-09T23:46:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:12 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1605.039368 secs 2023-09-09T23:46:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:13 smithi098 ceph-mon[103124]: pgmap v1025: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:46:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:13 smithi098 ceph-mon[103124]: osdmap e138: 12 total, 12 up, 12 in 2023-09-09T23:46:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:13 smithi170 ceph-mon[102832]: pgmap v1025: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:46:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:13 smithi170 ceph-mon[102832]: osdmap e138: 12 total, 12 up, 12 in 2023-09-09T23:46:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:13 smithi082 ceph-mon[98694]: pgmap v1025: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:46:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:13 smithi082 ceph-mon[98694]: osdmap e138: 12 total, 12 up, 12 in 2023-09-09T23:46:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1605.768668 secs 2023-09-09T23:46:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1605.768668 secs 2023-09-09T23:46:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1605.768668 secs 2023-09-09T23:46:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:15 smithi098 ceph-mon[103124]: pgmap v1027: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:46:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:15 smithi170 ceph-mon[102832]: pgmap v1027: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:46:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:15 smithi082 ceph-mon[98694]: pgmap v1027: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:46:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1608.441443 secs 2023-09-09T23:46:18.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:17 smithi098 ceph-mon[103124]: pgmap v1028: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.1 KiB/s wr, 11 op/s 2023-09-09T23:46:18.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:17 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1610.039427 secs 2023-09-09T23:46:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1608.441443 secs 2023-09-09T23:46:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:17 smithi170 ceph-mon[102832]: pgmap v1028: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.1 KiB/s wr, 11 op/s 2023-09-09T23:46:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:17 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1610.039427 secs 2023-09-09T23:46:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1608.441443 secs 2023-09-09T23:46:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:17 smithi082 ceph-mon[98694]: pgmap v1028: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.1 KiB/s wr, 11 op/s 2023-09-09T23:46:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:17 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1610.039427 secs 2023-09-09T23:46:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1610.768732 secs 2023-09-09T23:46:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:19 smithi098 ceph-mon[103124]: pgmap v1029: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.0 KiB/s wr, 11 op/s 2023-09-09T23:46:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1610.768732 secs 2023-09-09T23:46:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:19 smithi170 ceph-mon[102832]: pgmap v1029: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.0 KiB/s wr, 11 op/s 2023-09-09T23:46:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1610.768732 secs 2023-09-09T23:46:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:19 smithi082 ceph-mon[98694]: pgmap v1029: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.0 KiB/s wr, 11 op/s 2023-09-09T23:46:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:22 smithi098 ceph-mon[103124]: pgmap v1030: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 12 op/s 2023-09-09T23:46:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:22 smithi170 ceph-mon[102832]: pgmap v1030: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 12 op/s 2023-09-09T23:46:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:22 smithi082 ceph-mon[98694]: pgmap v1030: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 12 op/s 2023-09-09T23:46:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1613.441501 secs 2023-09-09T23:46:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1615.039488 secs 2023-09-09T23:46:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1613.441501 secs 2023-09-09T23:46:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1615.039488 secs 2023-09-09T23:46:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1613.441501 secs 2023-09-09T23:46:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1615.039488 secs 2023-09-09T23:46:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:24 smithi170 ceph-mon[102832]: pgmap v1031: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 12 op/s 2023-09-09T23:46:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:24 smithi082 ceph-mon[98694]: pgmap v1031: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 12 op/s 2023-09-09T23:46:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:24 smithi098 ceph-mon[103124]: pgmap v1031: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 12 op/s 2023-09-09T23:46:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1615.768797 secs 2023-09-09T23:46:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1615.768797 secs 2023-09-09T23:46:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1615.768797 secs 2023-09-09T23:46:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:26 smithi170 ceph-mon[102832]: pgmap v1032: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-09T23:46:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:26 smithi082 ceph-mon[98694]: pgmap v1032: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-09T23:46:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:26 smithi098 ceph-mon[103124]: pgmap v1032: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-09T23:46:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1618.441585 secs 2023-09-09T23:46:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:28 smithi170 ceph-mon[102832]: pgmap v1033: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 KiB/s wr, 14 op/s 2023-09-09T23:46:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1620.039550 secs 2023-09-09T23:46:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1618.441585 secs 2023-09-09T23:46:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:28 smithi082 ceph-mon[98694]: pgmap v1033: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 KiB/s wr, 14 op/s 2023-09-09T23:46:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1620.039550 secs 2023-09-09T23:46:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1618.441585 secs 2023-09-09T23:46:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:28 smithi098 ceph-mon[103124]: pgmap v1033: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 KiB/s wr, 14 op/s 2023-09-09T23:46:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1620.039550 secs 2023-09-09T23:46:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1620.768861 secs 2023-09-09T23:46:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:30 smithi170 ceph-mon[102832]: pgmap v1034: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-09-09T23:46:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1620.768861 secs 2023-09-09T23:46:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:30 smithi082 ceph-mon[98694]: pgmap v1034: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-09-09T23:46:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1620.768861 secs 2023-09-09T23:46:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:30 smithi098 ceph-mon[103124]: pgmap v1034: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-09-09T23:46:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:32 smithi170 ceph-mon[102832]: pgmap v1035: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 25 KiB/s wr, 14 op/s 2023-09-09T23:46:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:32 smithi082 ceph-mon[98694]: pgmap v1035: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 25 KiB/s wr, 14 op/s 2023-09-09T23:46:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:32 smithi098 ceph-mon[103124]: pgmap v1035: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 25 KiB/s wr, 14 op/s 2023-09-09T23:46:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1623.441645 secs 2023-09-09T23:46:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1625.039608 secs 2023-09-09T23:46:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1623.441645 secs 2023-09-09T23:46:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1625.039608 secs 2023-09-09T23:46:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1623.441645 secs 2023-09-09T23:46:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1625.039608 secs 2023-09-09T23:46:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:34 smithi170 ceph-mon[102832]: pgmap v1036: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.5 KiB/s wr, 8 op/s 2023-09-09T23:46:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:34 smithi082 ceph-mon[98694]: pgmap v1036: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.5 KiB/s wr, 8 op/s 2023-09-09T23:46:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:34 smithi098 ceph-mon[103124]: pgmap v1036: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.5 KiB/s wr, 8 op/s 2023-09-09T23:46:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1625.768925 secs 2023-09-09T23:46:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1625.768925 secs 2023-09-09T23:46:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1625.768925 secs 2023-09-09T23:46:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:36 smithi170 ceph-mon[102832]: pgmap v1037: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.5 KiB/s wr, 8 op/s 2023-09-09T23:46:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:36 smithi082 ceph-mon[98694]: pgmap v1037: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.5 KiB/s wr, 8 op/s 2023-09-09T23:46:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:36 smithi098 ceph-mon[103124]: pgmap v1037: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.5 KiB/s wr, 8 op/s 2023-09-09T23:46:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1628.441705 secs 2023-09-09T23:46:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:38 smithi170 ceph-mon[102832]: pgmap v1038: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.5 KiB/s wr, 13 op/s 2023-09-09T23:46:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1630.039669 secs 2023-09-09T23:46:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1628.441705 secs 2023-09-09T23:46:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:38 smithi082 ceph-mon[98694]: pgmap v1038: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.5 KiB/s wr, 13 op/s 2023-09-09T23:46:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1630.039669 secs 2023-09-09T23:46:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1628.441705 secs 2023-09-09T23:46:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:38 smithi098 ceph-mon[103124]: pgmap v1038: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.5 KiB/s wr, 13 op/s 2023-09-09T23:46:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1630.039669 secs 2023-09-09T23:46:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1630.768993 secs 2023-09-09T23:46:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:40 smithi170 ceph-mon[102832]: pgmap v1039: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 KiB/s wr, 8 op/s 2023-09-09T23:46:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1630.768993 secs 2023-09-09T23:46:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:40 smithi082 ceph-mon[98694]: pgmap v1039: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 KiB/s wr, 8 op/s 2023-09-09T23:46:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1630.768993 secs 2023-09-09T23:46:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:40 smithi098 ceph-mon[103124]: pgmap v1039: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 KiB/s wr, 8 op/s 2023-09-09T23:46:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:42 smithi170 ceph-mon[102832]: pgmap v1040: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.3 KiB/s wr, 12 op/s 2023-09-09T23:46:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:42 smithi082 ceph-mon[98694]: pgmap v1040: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.3 KiB/s wr, 12 op/s 2023-09-09T23:46:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:42 smithi098 ceph-mon[103124]: pgmap v1040: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.3 KiB/s wr, 12 op/s 2023-09-09T23:46:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1633.441773 secs 2023-09-09T23:46:43.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1635.039730 secs 2023-09-09T23:46:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1633.441773 secs 2023-09-09T23:46:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1635.039730 secs 2023-09-09T23:46:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1633.441773 secs 2023-09-09T23:46:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1635.039730 secs 2023-09-09T23:46:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:44 smithi170 ceph-mon[102832]: pgmap v1041: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:46:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:44 smithi082 ceph-mon[98694]: pgmap v1041: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:46:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:44 smithi098 ceph-mon[103124]: pgmap v1041: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:46:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1635.769056 secs 2023-09-09T23:46:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1635.769056 secs 2023-09-09T23:46:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1635.769056 secs 2023-09-09T23:46:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:46 smithi170 ceph-mon[102832]: pgmap v1042: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:46:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:46 smithi082 ceph-mon[98694]: pgmap v1042: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:46:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:46 smithi098 ceph-mon[103124]: pgmap v1042: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:46:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1638.441833 secs 2023-09-09T23:46:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:48 smithi170 ceph-mon[102832]: pgmap v1043: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:46:48.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1640.039788 secs 2023-09-09T23:46:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1638.441833 secs 2023-09-09T23:46:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:48 smithi082 ceph-mon[98694]: pgmap v1043: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:46:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1640.039788 secs 2023-09-09T23:46:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1638.441833 secs 2023-09-09T23:46:48.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:48 smithi098 ceph-mon[103124]: pgmap v1043: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-09-09T23:46:48.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1640.039788 secs 2023-09-09T23:46:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1640.769125 secs 2023-09-09T23:46:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:50 smithi170 ceph-mon[102832]: pgmap v1044: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:46:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1640.769125 secs 2023-09-09T23:46:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:50 smithi082 ceph-mon[98694]: pgmap v1044: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:46:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1640.769125 secs 2023-09-09T23:46:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:50 smithi098 ceph-mon[103124]: pgmap v1044: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-09T23:46:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:52 smithi170 ceph-mon[102832]: pgmap v1045: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2023-09-09T23:46:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:52 smithi082 ceph-mon[98694]: pgmap v1045: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2023-09-09T23:46:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:52 smithi098 ceph-mon[103124]: pgmap v1045: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2023-09-09T23:46:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1643.441925 secs 2023-09-09T23:46:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1645.039866 secs 2023-09-09T23:46:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1643.441925 secs 2023-09-09T23:46:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1645.039866 secs 2023-09-09T23:46:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1643.441925 secs 2023-09-09T23:46:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1645.039866 secs 2023-09-09T23:46:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:54 smithi170 ceph-mon[102832]: pgmap v1046: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2023-09-09T23:46:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:54 smithi082 ceph-mon[98694]: pgmap v1046: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2023-09-09T23:46:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:54 smithi098 ceph-mon[103124]: pgmap v1046: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2023-09-09T23:46:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1645.769207 secs 2023-09-09T23:46:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1645.769207 secs 2023-09-09T23:46:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1645.769207 secs 2023-09-09T23:46:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:56 smithi170 ceph-mon[102832]: pgmap v1047: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2023-09-09T23:46:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:46:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:46:56.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:46:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:56 smithi082 ceph-mon[98694]: pgmap v1047: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2023-09-09T23:46:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:46:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:46:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:46:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:56 smithi098 ceph-mon[103124]: pgmap v1047: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2023-09-09T23:46:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:46:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:46:56.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:46:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:46:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:46:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:46:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:46:57.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:46:57.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:46:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:46:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:46:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:46:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1648.441985 secs 2023-09-09T23:46:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:58 smithi170 ceph-mon[102832]: pgmap v1048: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 16 KiB/s wr, 14 op/s 2023-09-09T23:46:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:46:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1650.039924 secs 2023-09-09T23:46:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1648.441985 secs 2023-09-09T23:46:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:58 smithi082 ceph-mon[98694]: pgmap v1048: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 16 KiB/s wr, 14 op/s 2023-09-09T23:46:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:46:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1650.039924 secs 2023-09-09T23:46:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1648.441985 secs 2023-09-09T23:46:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:58 smithi098 ceph-mon[103124]: pgmap v1048: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 16 KiB/s wr, 14 op/s 2023-09-09T23:46:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:46:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1650.039924 secs 2023-09-09T23:47:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1650.769277 secs 2023-09-09T23:47:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:00 smithi170 ceph-mon[102832]: pgmap v1049: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-09T23:47:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1650.769277 secs 2023-09-09T23:47:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:00 smithi082 ceph-mon[98694]: pgmap v1049: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-09T23:47:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1650.769277 secs 2023-09-09T23:47:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:00 smithi098 ceph-mon[103124]: pgmap v1049: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-09T23:47:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:02 smithi170 ceph-mon[102832]: pgmap v1050: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 14 op/s 2023-09-09T23:47:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1653.442071 secs 2023-09-09T23:47:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:02 smithi082 ceph-mon[98694]: pgmap v1050: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 14 op/s 2023-09-09T23:47:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1653.442071 secs 2023-09-09T23:47:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:02 smithi098 ceph-mon[103124]: pgmap v1050: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 14 op/s 2023-09-09T23:47:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1653.442071 secs 2023-09-09T23:47:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1655.039995 secs 2023-09-09T23:47:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1655.039995 secs 2023-09-09T23:47:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1655.039995 secs 2023-09-09T23:47:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:04 smithi170 ceph-mon[102832]: pgmap v1051: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 9 op/s 2023-09-09T23:47:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:04 smithi082 ceph-mon[98694]: pgmap v1051: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 9 op/s 2023-09-09T23:47:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:04 smithi098 ceph-mon[103124]: pgmap v1051: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 9 op/s 2023-09-09T23:47:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1655.769340 secs 2023-09-09T23:47:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1655.769340 secs 2023-09-09T23:47:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1655.769340 secs 2023-09-09T23:47:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:06 smithi170 ceph-mon[102832]: pgmap v1052: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 9 op/s 2023-09-09T23:47:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:06 smithi082 ceph-mon[98694]: pgmap v1052: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 9 op/s 2023-09-09T23:47:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:06 smithi098 ceph-mon[103124]: pgmap v1052: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 9 op/s 2023-09-09T23:47:07.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1658.442140 secs 2023-09-09T23:47:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1658.442140 secs 2023-09-09T23:47:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1658.442140 secs 2023-09-09T23:47:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:08 smithi170 ceph-mon[102832]: pgmap v1053: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 57 KiB/s wr, 28 op/s 2023-09-09T23:47:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:08 smithi170 ceph-mon[102832]: osdmap e139: 12 total, 12 up, 12 in 2023-09-09T23:47:08.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1660.040050 secs 2023-09-09T23:47:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:08 smithi082 ceph-mon[98694]: pgmap v1053: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 57 KiB/s wr, 28 op/s 2023-09-09T23:47:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:08 smithi082 ceph-mon[98694]: osdmap e139: 12 total, 12 up, 12 in 2023-09-09T23:47:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1660.040050 secs 2023-09-09T23:47:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:08 smithi098 ceph-mon[103124]: pgmap v1053: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 57 KiB/s wr, 28 op/s 2023-09-09T23:47:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:08 smithi098 ceph-mon[103124]: osdmap e139: 12 total, 12 up, 12 in 2023-09-09T23:47:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1660.040050 secs 2023-09-09T23:47:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1660.769407 secs 2023-09-09T23:47:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:10 smithi170 ceph-mon[102832]: pgmap v1055: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:47:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1660.769407 secs 2023-09-09T23:47:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:10 smithi082 ceph-mon[98694]: pgmap v1055: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:47:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1660.769407 secs 2023-09-09T23:47:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:10 smithi098 ceph-mon[103124]: pgmap v1055: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:47:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:12 smithi170 ceph-mon[102832]: pgmap v1056: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:47:12.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1663.442200 secs 2023-09-09T23:47:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:12 smithi082 ceph-mon[98694]: pgmap v1056: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:47:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1663.442200 secs 2023-09-09T23:47:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:12 smithi098 ceph-mon[103124]: pgmap v1056: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:47:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1663.442200 secs 2023-09-09T23:47:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1665.040107 secs 2023-09-09T23:47:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1665.040107 secs 2023-09-09T23:47:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1665.040107 secs 2023-09-09T23:47:14.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:14 smithi170 ceph-mon[102832]: pgmap v1057: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:47:14.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1665.769475 secs 2023-09-09T23:47:14.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:14 smithi170 ceph-mon[102832]: osdmap e140: 12 total, 12 up, 12 in 2023-09-09T23:47:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:14 smithi082 ceph-mon[98694]: pgmap v1057: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:47:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1665.769475 secs 2023-09-09T23:47:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:14 smithi082 ceph-mon[98694]: osdmap e140: 12 total, 12 up, 12 in 2023-09-09T23:47:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:14 smithi098 ceph-mon[103124]: pgmap v1057: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:47:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1665.769475 secs 2023-09-09T23:47:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:14 smithi098 ceph-mon[103124]: osdmap e140: 12 total, 12 up, 12 in 2023-09-09T23:47:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:16 smithi170 ceph-mon[102832]: pgmap v1059: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:47:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:16 smithi082 ceph-mon[98694]: pgmap v1059: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:47:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:16 smithi098 ceph-mon[103124]: pgmap v1059: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:47:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1668.442259 secs 2023-09-09T23:47:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1668.442259 secs 2023-09-09T23:47:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1668.442259 secs 2023-09-09T23:47:18.434 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:18 smithi170 ceph-mon[102832]: pgmap v1060: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2023-09-09T23:47:20.486 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1670.040194 secs 2023-09-09T23:47:20.488 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1670.769538 secs 2023-09-09T23:47:20.489 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:19 smithi170 ceph-mon[102832]: pgmap v1061: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.3 KiB/s wr, 11 op/s 2023-09-09T23:47:20.489 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:18 smithi098 ceph-mon[103124]: pgmap v1060: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2023-09-09T23:47:20.490 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1670.040194 secs 2023-09-09T23:47:20.490 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1670.769538 secs 2023-09-09T23:47:20.490 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:19 smithi098 ceph-mon[103124]: pgmap v1061: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.3 KiB/s wr, 11 op/s 2023-09-09T23:47:20.491 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:18 smithi082 ceph-mon[98694]: pgmap v1060: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2023-09-09T23:47:20.491 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1670.040194 secs 2023-09-09T23:47:20.491 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1670.769538 secs 2023-09-09T23:47:20.491 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:19 smithi082 ceph-mon[98694]: pgmap v1061: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.3 KiB/s wr, 11 op/s 2023-09-09T23:47:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:21 smithi170 ceph-mon[102832]: pgmap v1062: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 40 KiB/s wr, 13 op/s 2023-09-09T23:47:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:21 smithi082 ceph-mon[98694]: pgmap v1062: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 40 KiB/s wr, 13 op/s 2023-09-09T23:47:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:21 smithi098 ceph-mon[103124]: pgmap v1062: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 40 KiB/s wr, 13 op/s 2023-09-09T23:47:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1673.442321 secs 2023-09-09T23:47:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1673.442321 secs 2023-09-09T23:47:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1673.442321 secs 2023-09-09T23:47:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:23 smithi170 ceph-mon[102832]: pgmap v1063: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 40 KiB/s wr, 13 op/s 2023-09-09T23:47:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1675.040252 secs 2023-09-09T23:47:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:23 smithi082 ceph-mon[98694]: pgmap v1063: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 40 KiB/s wr, 13 op/s 2023-09-09T23:47:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1675.040252 secs 2023-09-09T23:47:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:23 smithi098 ceph-mon[103124]: pgmap v1063: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 40 KiB/s wr, 13 op/s 2023-09-09T23:47:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1675.040252 secs 2023-09-09T23:47:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1675.769603 secs 2023-09-09T23:47:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1675.769603 secs 2023-09-09T23:47:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1675.769603 secs 2023-09-09T23:47:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:25 smithi170 ceph-mon[102832]: pgmap v1064: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 36 KiB/s wr, 12 op/s 2023-09-09T23:47:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:25 smithi082 ceph-mon[98694]: pgmap v1064: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 36 KiB/s wr, 12 op/s 2023-09-09T23:47:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:25 smithi098 ceph-mon[103124]: pgmap v1064: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 36 KiB/s wr, 12 op/s 2023-09-09T23:47:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:47:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:47:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:47:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1678.442388 secs 2023-09-09T23:47:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:27 smithi170 ceph-mon[102832]: pgmap v1065: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 45 KiB/s wr, 16 op/s 2023-09-09T23:47:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1678.442388 secs 2023-09-09T23:47:28.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:27 smithi082 ceph-mon[98694]: pgmap v1065: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 45 KiB/s wr, 16 op/s 2023-09-09T23:47:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1678.442388 secs 2023-09-09T23:47:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:27 smithi098 ceph-mon[103124]: pgmap v1065: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 45 KiB/s wr, 16 op/s 2023-09-09T23:47:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1680.040310 secs 2023-09-09T23:47:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1680.040310 secs 2023-09-09T23:47:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1680.040310 secs 2023-09-09T23:47:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1680.769692 secs 2023-09-09T23:47:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:29 smithi170 ceph-mon[102832]: pgmap v1066: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-09T23:47:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1680.769692 secs 2023-09-09T23:47:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:29 smithi082 ceph-mon[98694]: pgmap v1066: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-09T23:47:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1680.769692 secs 2023-09-09T23:47:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:29 smithi098 ceph-mon[103124]: pgmap v1066: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-09T23:47:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:31 smithi170 ceph-mon[102832]: pgmap v1067: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 57 KiB/s wr, 17 op/s 2023-09-09T23:47:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:31 smithi082 ceph-mon[98694]: pgmap v1067: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 57 KiB/s wr, 17 op/s 2023-09-09T23:47:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:31 smithi098 ceph-mon[103124]: pgmap v1067: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 57 KiB/s wr, 17 op/s 2023-09-09T23:47:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1683.442531 secs 2023-09-09T23:47:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1683.442531 secs 2023-09-09T23:47:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1683.442531 secs 2023-09-09T23:47:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1685.040369 secs 2023-09-09T23:47:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:33 smithi170 ceph-mon[102832]: pgmap v1068: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-09T23:47:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1685.040369 secs 2023-09-09T23:47:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:33 smithi082 ceph-mon[98694]: pgmap v1068: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-09T23:47:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1685.040369 secs 2023-09-09T23:47:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:33 smithi098 ceph-mon[103124]: pgmap v1068: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-09T23:47:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1685.769758 secs 2023-09-09T23:47:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1685.769758 secs 2023-09-09T23:47:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1685.769758 secs 2023-09-09T23:47:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:35 smithi170 ceph-mon[102832]: pgmap v1069: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-09T23:47:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:35 smithi082 ceph-mon[98694]: pgmap v1069: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-09T23:47:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:35 smithi098 ceph-mon[103124]: pgmap v1069: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-09T23:47:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1688.442596 secs 2023-09-09T23:47:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:37 smithi170 ceph-mon[102832]: pgmap v1070: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 29 KiB/s wr, 15 op/s 2023-09-09T23:47:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1688.442596 secs 2023-09-09T23:47:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:37 smithi082 ceph-mon[98694]: pgmap v1070: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 29 KiB/s wr, 15 op/s 2023-09-09T23:47:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1688.442596 secs 2023-09-09T23:47:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:37 smithi098 ceph-mon[103124]: pgmap v1070: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 29 KiB/s wr, 15 op/s 2023-09-09T23:47:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1690.040430 secs 2023-09-09T23:47:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1690.040430 secs 2023-09-09T23:47:39.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1690.040430 secs 2023-09-09T23:47:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1690.769825 secs 2023-09-09T23:47:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:39 smithi170 ceph-mon[102832]: pgmap v1071: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 17 KiB/s wr, 10 op/s 2023-09-09T23:47:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1690.769825 secs 2023-09-09T23:47:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:39 smithi082 ceph-mon[98694]: pgmap v1071: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 17 KiB/s wr, 10 op/s 2023-09-09T23:47:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1690.769825 secs 2023-09-09T23:47:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:39 smithi098 ceph-mon[103124]: pgmap v1071: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 17 KiB/s wr, 10 op/s 2023-09-09T23:47:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:41 smithi170 ceph-mon[102832]: pgmap v1072: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 22 KiB/s wr, 14 op/s 2023-09-09T23:47:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:41 smithi082 ceph-mon[98694]: pgmap v1072: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 22 KiB/s wr, 14 op/s 2023-09-09T23:47:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:41 smithi098 ceph-mon[103124]: pgmap v1072: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 22 KiB/s wr, 14 op/s 2023-09-09T23:47:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1693.442674 secs 2023-09-09T23:47:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1693.442674 secs 2023-09-09T23:47:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1693.442674 secs 2023-09-09T23:47:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1695.040491 secs 2023-09-09T23:47:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:43 smithi170 ceph-mon[102832]: pgmap v1073: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2023-09-09T23:47:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1695.040491 secs 2023-09-09T23:47:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:43 smithi082 ceph-mon[98694]: pgmap v1073: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2023-09-09T23:47:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1695.040491 secs 2023-09-09T23:47:44.343 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:43 smithi098 ceph-mon[103124]: pgmap v1073: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2023-09-09T23:47:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1695.769889 secs 2023-09-09T23:47:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1695.769889 secs 2023-09-09T23:47:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1695.769889 secs 2023-09-09T23:47:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:45 smithi170 ceph-mon[102832]: pgmap v1074: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2023-09-09T23:47:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:45 smithi082 ceph-mon[98694]: pgmap v1074: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2023-09-09T23:47:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:45 smithi098 ceph-mon[103124]: pgmap v1074: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2023-09-09T23:47:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1698.442759 secs 2023-09-09T23:47:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:47 smithi170 ceph-mon[102832]: pgmap v1075: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2023-09-09T23:47:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1698.442759 secs 2023-09-09T23:47:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:47 smithi082 ceph-mon[98694]: pgmap v1075: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2023-09-09T23:47:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1698.442759 secs 2023-09-09T23:47:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:47 smithi098 ceph-mon[103124]: pgmap v1075: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2023-09-09T23:47:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1700.040550 secs 2023-09-09T23:47:49.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1700.040550 secs 2023-09-09T23:47:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1700.040550 secs 2023-09-09T23:47:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1700.769957 secs 2023-09-09T23:47:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:49 smithi082 ceph-mon[98694]: pgmap v1076: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2023-09-09T23:47:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1700.769957 secs 2023-09-09T23:47:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:49 smithi098 ceph-mon[103124]: pgmap v1076: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2023-09-09T23:47:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1700.769957 secs 2023-09-09T23:47:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:49 smithi170 ceph-mon[102832]: pgmap v1076: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2023-09-09T23:47:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:51 smithi082 ceph-mon[98694]: pgmap v1077: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 KiB/s wr, 13 op/s 2023-09-09T23:47:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:51 smithi098 ceph-mon[103124]: pgmap v1077: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 KiB/s wr, 13 op/s 2023-09-09T23:47:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:51 smithi170 ceph-mon[102832]: pgmap v1077: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 KiB/s wr, 13 op/s 2023-09-09T23:47:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1703.442822 secs 2023-09-09T23:47:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1703.442822 secs 2023-09-09T23:47:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1703.442822 secs 2023-09-09T23:47:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1705.040609 secs 2023-09-09T23:47:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:53 smithi082 ceph-mon[98694]: pgmap v1078: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2023-09-09T23:47:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1705.040609 secs 2023-09-09T23:47:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:53 smithi098 ceph-mon[103124]: pgmap v1078: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2023-09-09T23:47:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1705.040609 secs 2023-09-09T23:47:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:53 smithi170 ceph-mon[102832]: pgmap v1078: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2023-09-09T23:47:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1705.770027 secs 2023-09-09T23:47:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1705.770027 secs 2023-09-09T23:47:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1705.770027 secs 2023-09-09T23:47:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:55 smithi082 ceph-mon[98694]: pgmap v1079: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2023-09-09T23:47:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:55 smithi098 ceph-mon[103124]: pgmap v1079: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2023-09-09T23:47:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:55 smithi170 ceph-mon[102832]: pgmap v1079: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2023-09-09T23:47:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:47:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:47:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:47:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1708.442882 secs 2023-09-09T23:47:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:58 smithi170 ceph-mon[102832]: pgmap v1080: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 17 KiB/s wr, 14 op/s 2023-09-09T23:47:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:47:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:47:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:47:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:47:58.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:47:58.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:47:58.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:47:58.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:47:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1710.040669 secs 2023-09-09T23:47:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1708.442882 secs 2023-09-09T23:47:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:58 smithi082 ceph-mon[98694]: pgmap v1080: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 17 KiB/s wr, 14 op/s 2023-09-09T23:47:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:47:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:47:58.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:47:58.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:47:58.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:47:58.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:47:58.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:47:58.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:47:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1710.040669 secs 2023-09-09T23:47:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1708.442882 secs 2023-09-09T23:47:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:58 smithi098 ceph-mon[103124]: pgmap v1080: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 17 KiB/s wr, 14 op/s 2023-09-09T23:47:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:47:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:47:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:47:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:47:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:47:58.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:47:58.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:47:58.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:47:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1710.040669 secs 2023-09-09T23:48:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1710.770098 secs 2023-09-09T23:48:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:00 smithi170 ceph-mon[102832]: pgmap v1081: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 17 KiB/s wr, 10 op/s 2023-09-09T23:48:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1710.770098 secs 2023-09-09T23:48:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:00 smithi082 ceph-mon[98694]: pgmap v1081: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 17 KiB/s wr, 10 op/s 2023-09-09T23:48:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1710.770098 secs 2023-09-09T23:48:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:00 smithi098 ceph-mon[103124]: pgmap v1081: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 17 KiB/s wr, 10 op/s 2023-09-09T23:48:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:02 smithi170 ceph-mon[102832]: pgmap v1082: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 14 op/s 2023-09-09T23:48:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:02 smithi082 ceph-mon[98694]: pgmap v1082: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 14 op/s 2023-09-09T23:48:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:02 smithi098 ceph-mon[103124]: pgmap v1082: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 14 op/s 2023-09-09T23:48:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1713.442967 secs 2023-09-09T23:48:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1715.040730 secs 2023-09-09T23:48:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1713.442967 secs 2023-09-09T23:48:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1715.040730 secs 2023-09-09T23:48:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1713.442967 secs 2023-09-09T23:48:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1715.040730 secs 2023-09-09T23:48:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:04 smithi170 ceph-mon[102832]: pgmap v1083: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 9 op/s 2023-09-09T23:48:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:04 smithi082 ceph-mon[98694]: pgmap v1083: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 9 op/s 2023-09-09T23:48:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:04 smithi098 ceph-mon[103124]: pgmap v1083: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 9 op/s 2023-09-09T23:48:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1715.770161 secs 2023-09-09T23:48:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1715.770161 secs 2023-09-09T23:48:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1715.770161 secs 2023-09-09T23:48:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:06 smithi170 ceph-mon[102832]: pgmap v1084: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 9 op/s 2023-09-09T23:48:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:06 smithi082 ceph-mon[98694]: pgmap v1084: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 9 op/s 2023-09-09T23:48:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:06 smithi098 ceph-mon[103124]: pgmap v1084: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 9 op/s 2023-09-09T23:48:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1718.443083 secs 2023-09-09T23:48:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:08 smithi170 ceph-mon[102832]: pgmap v1085: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 57 KiB/s wr, 28 op/s 2023-09-09T23:48:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:08 smithi170 ceph-mon[102832]: osdmap e141: 12 total, 12 up, 12 in 2023-09-09T23:48:08.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1720.040818 secs 2023-09-09T23:48:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1718.443083 secs 2023-09-09T23:48:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:08 smithi082 ceph-mon[98694]: pgmap v1085: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 57 KiB/s wr, 28 op/s 2023-09-09T23:48:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:08 smithi082 ceph-mon[98694]: osdmap e141: 12 total, 12 up, 12 in 2023-09-09T23:48:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1720.040818 secs 2023-09-09T23:48:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1718.443083 secs 2023-09-09T23:48:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:08 smithi098 ceph-mon[103124]: pgmap v1085: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 57 KiB/s wr, 28 op/s 2023-09-09T23:48:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:08 smithi098 ceph-mon[103124]: osdmap e141: 12 total, 12 up, 12 in 2023-09-09T23:48:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1720.040818 secs 2023-09-09T23:48:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1720.770219 secs 2023-09-09T23:48:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:10 smithi170 ceph-mon[102832]: pgmap v1087: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:48:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1720.770219 secs 2023-09-09T23:48:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:10 smithi082 ceph-mon[98694]: pgmap v1087: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:48:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1720.770219 secs 2023-09-09T23:48:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:10 smithi098 ceph-mon[103124]: pgmap v1087: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-09T23:48:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:12 smithi170 ceph-mon[102832]: pgmap v1088: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:48:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:12 smithi082 ceph-mon[98694]: pgmap v1088: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:48:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:12 smithi098 ceph-mon[103124]: pgmap v1088: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:48:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1723.443143 secs 2023-09-09T23:48:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1725.040873 secs 2023-09-09T23:48:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1723.443143 secs 2023-09-09T23:48:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1725.040873 secs 2023-09-09T23:48:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1723.443143 secs 2023-09-09T23:48:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1725.040873 secs 2023-09-09T23:48:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:14 smithi170 ceph-mon[102832]: pgmap v1089: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:48:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:14 smithi170 ceph-mon[102832]: osdmap e142: 12 total, 12 up, 12 in 2023-09-09T23:48:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:14 smithi082 ceph-mon[98694]: pgmap v1089: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:48:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:14 smithi082 ceph-mon[98694]: osdmap e142: 12 total, 12 up, 12 in 2023-09-09T23:48:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:14 smithi098 ceph-mon[103124]: pgmap v1089: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-09T23:48:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:14 smithi098 ceph-mon[103124]: osdmap e142: 12 total, 12 up, 12 in 2023-09-09T23:48:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1725.770283 secs 2023-09-09T23:48:15.479 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1725.770283 secs 2023-09-09T23:48:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1725.770283 secs 2023-09-09T23:48:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:16 smithi170 ceph-mon[102832]: pgmap v1091: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:48:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:16 smithi082 ceph-mon[98694]: pgmap v1091: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:48:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:16 smithi098 ceph-mon[103124]: pgmap v1091: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-09T23:48:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1728.443203 secs 2023-09-09T23:48:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:18 smithi170 ceph-mon[102832]: pgmap v1092: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 420 B/s wr, 11 op/s 2023-09-09T23:48:18.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1730.040931 secs 2023-09-09T23:48:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1728.443203 secs 2023-09-09T23:48:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:18 smithi082 ceph-mon[98694]: pgmap v1092: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 420 B/s wr, 11 op/s 2023-09-09T23:48:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1730.040931 secs 2023-09-09T23:48:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1728.443203 secs 2023-09-09T23:48:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:18 smithi098 ceph-mon[103124]: pgmap v1092: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 420 B/s wr, 11 op/s 2023-09-09T23:48:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1730.040931 secs 2023-09-09T23:48:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1730.770348 secs 2023-09-09T23:48:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:20 smithi170 ceph-mon[102832]: pgmap v1093: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 409 B/s wr, 10 op/s 2023-09-09T23:48:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1730.770348 secs 2023-09-09T23:48:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:20 smithi082 ceph-mon[98694]: pgmap v1093: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 409 B/s wr, 10 op/s 2023-09-09T23:48:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1730.770348 secs 2023-09-09T23:48:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:20 smithi098 ceph-mon[103124]: pgmap v1093: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 409 B/s wr, 10 op/s 2023-09-09T23:48:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:22 smithi170 ceph-mon[102832]: pgmap v1094: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-09T23:48:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:22 smithi082 ceph-mon[98694]: pgmap v1094: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-09T23:48:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:22 smithi098 ceph-mon[103124]: pgmap v1094: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-09T23:48:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:48:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1735.041014 secs 2023-09-09T23:48:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:48:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1735.041014 secs 2023-09-09T23:48:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:48:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1735.041014 secs 2023-09-09T23:48:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:24 smithi170 ceph-mon[102832]: pgmap v1095: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-09T23:48:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:48:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:48:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:48:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:24 smithi082 ceph-mon[98694]: pgmap v1095: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-09T23:48:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:48:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:48:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:48:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:24 smithi098 ceph-mon[103124]: pgmap v1095: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-09T23:48:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:48:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:48:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:48:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1735.770416 secs 2023-09-09T23:48:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1735.770416 secs 2023-09-09T23:48:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1735.770416 secs 2023-09-09T23:48:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:26 smithi170 ceph-mon[102832]: pgmap v1096: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-09T23:48:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:26 smithi082 ceph-mon[98694]: pgmap v1096: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-09T23:48:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:26 smithi098 ceph-mon[103124]: pgmap v1096: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-09T23:48:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:28 smithi170 ceph-mon[102832]: pgmap v1097: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 47 KiB/s wr, 14 op/s 2023-09-09T23:48:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1740.041073 secs 2023-09-09T23:48:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:28 smithi082 ceph-mon[98694]: pgmap v1097: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 47 KiB/s wr, 14 op/s 2023-09-09T23:48:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1740.041073 secs 2023-09-09T23:48:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:28 smithi098 ceph-mon[103124]: pgmap v1097: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 47 KiB/s wr, 14 op/s 2023-09-09T23:48:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1740.041073 secs 2023-09-09T23:48:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1740.770487 secs 2023-09-09T23:48:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:30 smithi170 ceph-mon[102832]: pgmap v1098: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 10 op/s 2023-09-09T23:48:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1740.770487 secs 2023-09-09T23:48:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:30 smithi082 ceph-mon[98694]: pgmap v1098: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 10 op/s 2023-09-09T23:48:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1740.770487 secs 2023-09-09T23:48:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:30 smithi098 ceph-mon[103124]: pgmap v1098: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 10 op/s 2023-09-09T23:48:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:32 smithi170 ceph-mon[102832]: pgmap v1099: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 84 KiB/s wr, 16 op/s 2023-09-09T23:48:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:32 smithi082 ceph-mon[98694]: pgmap v1099: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 84 KiB/s wr, 16 op/s 2023-09-09T23:48:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:32 smithi098 ceph-mon[103124]: pgmap v1099: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 84 KiB/s wr, 16 op/s 2023-09-09T23:48:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1745.041143 secs 2023-09-09T23:48:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1745.041143 secs 2023-09-09T23:48:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1745.041143 secs 2023-09-09T23:48:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:34 smithi170 ceph-mon[102832]: pgmap v1100: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-09T23:48:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:34 smithi082 ceph-mon[98694]: pgmap v1100: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-09T23:48:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:34 smithi098 ceph-mon[103124]: pgmap v1100: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-09T23:48:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1745.770553 secs 2023-09-09T23:48:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1745.770553 secs 2023-09-09T23:48:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1745.770553 secs 2023-09-09T23:48:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:36 smithi170 ceph-mon[102832]: pgmap v1101: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-09T23:48:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:36 smithi082 ceph-mon[98694]: pgmap v1101: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-09T23:48:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:36 smithi098 ceph-mon[103124]: pgmap v1101: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-09T23:48:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:38 smithi170 ceph-mon[102832]: pgmap v1102: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 89 KiB/s wr, 17 op/s 2023-09-09T23:48:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1750.041203 secs 2023-09-09T23:48:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:38 smithi082 ceph-mon[98694]: pgmap v1102: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 89 KiB/s wr, 17 op/s 2023-09-09T23:48:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1750.041203 secs 2023-09-09T23:48:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:38 smithi098 ceph-mon[103124]: pgmap v1102: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 89 KiB/s wr, 17 op/s 2023-09-09T23:48:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1750.041203 secs 2023-09-09T23:48:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1750.770619 secs 2023-09-09T23:48:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:40 smithi170 ceph-mon[102832]: pgmap v1103: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 55 KiB/s wr, 12 op/s 2023-09-09T23:48:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1750.770619 secs 2023-09-09T23:48:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:40 smithi082 ceph-mon[98694]: pgmap v1103: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 55 KiB/s wr, 12 op/s 2023-09-09T23:48:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1750.770619 secs 2023-09-09T23:48:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:40 smithi098 ceph-mon[103124]: pgmap v1103: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 55 KiB/s wr, 12 op/s 2023-09-09T23:48:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:42 smithi170 ceph-mon[102832]: pgmap v1104: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 90 KiB/s wr, 18 op/s 2023-09-09T23:48:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:42 smithi082 ceph-mon[98694]: pgmap v1104: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 90 KiB/s wr, 18 op/s 2023-09-09T23:48:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:42 smithi098 ceph-mon[103124]: pgmap v1104: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 90 KiB/s wr, 18 op/s 2023-09-09T23:48:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1755.041259 secs 2023-09-09T23:48:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1755.041259 secs 2023-09-09T23:48:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1755.041259 secs 2023-09-09T23:48:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:44 smithi170 ceph-mon[102832]: pgmap v1105: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 53 KiB/s wr, 12 op/s 2023-09-09T23:48:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:44 smithi082 ceph-mon[98694]: pgmap v1105: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 53 KiB/s wr, 12 op/s 2023-09-09T23:48:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:44 smithi098 ceph-mon[103124]: pgmap v1105: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 53 KiB/s wr, 12 op/s 2023-09-09T23:48:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1755.770684 secs 2023-09-09T23:48:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1755.770684 secs 2023-09-09T23:48:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1755.770684 secs 2023-09-09T23:48:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:46 smithi170 ceph-mon[102832]: pgmap v1106: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 53 KiB/s wr, 12 op/s 2023-09-09T23:48:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:46 smithi082 ceph-mon[98694]: pgmap v1106: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 53 KiB/s wr, 12 op/s 2023-09-09T23:48:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:46 smithi098 ceph-mon[103124]: pgmap v1106: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 53 KiB/s wr, 12 op/s 2023-09-09T23:48:47.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1758.443622 secs 2023-09-09T23:48:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1758.443622 secs 2023-09-09T23:48:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1758.443622 secs 2023-09-09T23:48:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:48 smithi170 ceph-mon[102832]: pgmap v1107: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 80 KiB/s wr, 17 op/s 2023-09-09T23:48:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1760.041334 secs 2023-09-09T23:48:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:48 smithi082 ceph-mon[98694]: pgmap v1107: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 80 KiB/s wr, 17 op/s 2023-09-09T23:48:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1760.041334 secs 2023-09-09T23:48:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:48 smithi098 ceph-mon[103124]: pgmap v1107: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 80 KiB/s wr, 17 op/s 2023-09-09T23:48:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1760.041334 secs 2023-09-09T23:48:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1760.770751 secs 2023-09-09T23:48:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:49 smithi170 ceph-mon[102832]: pgmap v1108: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-09T23:48:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1760.770751 secs 2023-09-09T23:48:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:49 smithi082 ceph-mon[98694]: pgmap v1108: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-09T23:48:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1760.770751 secs 2023-09-09T23:48:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:49 smithi098 ceph-mon[103124]: pgmap v1108: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-09T23:48:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:50 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:48:51.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:48:51.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:48:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:50 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:48:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:48:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:48:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:50 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:48:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:48:51.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:48:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:51 smithi170 ceph-mon[102832]: pgmap v1109: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 88 KiB/s wr, 17 op/s 2023-09-09T23:48:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:51 smithi082 ceph-mon[98694]: pgmap v1109: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 88 KiB/s wr, 17 op/s 2023-09-09T23:48:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:51 smithi098 ceph-mon[103124]: pgmap v1109: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 88 KiB/s wr, 17 op/s 2023-09-09T23:48:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1763.443681 secs 2023-09-09T23:48:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1763.443681 secs 2023-09-09T23:48:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1763.443681 secs 2023-09-09T23:48:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1765.041419 secs 2023-09-09T23:48:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:53 smithi170 ceph-mon[102832]: pgmap v1110: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-09T23:48:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1765.041419 secs 2023-09-09T23:48:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:53 smithi082 ceph-mon[98694]: pgmap v1110: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-09T23:48:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1765.041419 secs 2023-09-09T23:48:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:53 smithi098 ceph-mon[103124]: pgmap v1110: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-09T23:48:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1765.770816 secs 2023-09-09T23:48:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1765.770816 secs 2023-09-09T23:48:55.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1765.770816 secs 2023-09-09T23:48:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:55 smithi082 ceph-mon[98694]: pgmap v1111: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-09T23:48:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:55 smithi098 ceph-mon[103124]: pgmap v1111: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-09T23:48:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:55 smithi170 ceph-mon[102832]: pgmap v1111: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-09T23:48:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1768.443797 secs 2023-09-09T23:48:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:57 smithi082 ceph-mon[98694]: pgmap v1112: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 63 KiB/s wr, 15 op/s 2023-09-09T23:48:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:48:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1768.443797 secs 2023-09-09T23:48:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:57 smithi098 ceph-mon[103124]: pgmap v1112: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 63 KiB/s wr, 15 op/s 2023-09-09T23:48:58.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:48:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1768.443797 secs 2023-09-09T23:48:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:57 smithi170 ceph-mon[102832]: pgmap v1112: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 63 KiB/s wr, 15 op/s 2023-09-09T23:48:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:48:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1770.041477 secs 2023-09-09T23:48:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:48:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:48:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:48:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1770.041477 secs 2023-09-09T23:48:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:48:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:48:59.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:48:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1770.041477 secs 2023-09-09T23:48:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:48:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:48:59.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:49:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1770.770883 secs 2023-09-09T23:49:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:48:59 smithi082 ceph-mon[98694]: pgmap v1113: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-09-09T23:49:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1770.770883 secs 2023-09-09T23:49:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:48:59 smithi098 ceph-mon[103124]: pgmap v1113: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-09-09T23:49:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1770.770883 secs 2023-09-09T23:49:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:48:59 smithi170 ceph-mon[102832]: pgmap v1113: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-09-09T23:49:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:01 smithi082 ceph-mon[98694]: pgmap v1114: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 46 KiB/s wr, 15 op/s 2023-09-09T23:49:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:01 smithi098 ceph-mon[103124]: pgmap v1114: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 46 KiB/s wr, 15 op/s 2023-09-09T23:49:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:01 smithi170 ceph-mon[102832]: pgmap v1114: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 46 KiB/s wr, 15 op/s 2023-09-09T23:49:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1773.443856 secs 2023-09-09T23:49:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1773.443856 secs 2023-09-09T23:49:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1773.443856 secs 2023-09-09T23:49:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1775.041561 secs 2023-09-09T23:49:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:03 smithi082 ceph-mon[98694]: pgmap v1115: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 20 KiB/s wr, 9 op/s 2023-09-09T23:49:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1775.041561 secs 2023-09-09T23:49:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:03 smithi098 ceph-mon[103124]: pgmap v1115: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 20 KiB/s wr, 9 op/s 2023-09-09T23:49:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1775.041561 secs 2023-09-09T23:49:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:03 smithi170 ceph-mon[102832]: pgmap v1115: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 20 KiB/s wr, 9 op/s 2023-09-09T23:49:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1775.770947 secs 2023-09-09T23:49:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1775.770947 secs 2023-09-09T23:49:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1775.770947 secs 2023-09-09T23:49:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:05 smithi082 ceph-mon[98694]: pgmap v1116: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 20 KiB/s wr, 9 op/s 2023-09-09T23:49:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:05 smithi098 ceph-mon[103124]: pgmap v1116: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 20 KiB/s wr, 9 op/s 2023-09-09T23:49:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:05 smithi170 ceph-mon[102832]: pgmap v1116: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 20 KiB/s wr, 9 op/s 2023-09-09T23:49:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1778.443972 secs 2023-09-09T23:49:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:07 smithi098 ceph-mon[103124]: pgmap v1117: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-09T23:49:08.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:07 smithi098 ceph-mon[103124]: osdmap e143: 12 total, 12 up, 12 in 2023-09-09T23:49:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1778.443972 secs 2023-09-09T23:49:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:07 smithi170 ceph-mon[102832]: pgmap v1117: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-09T23:49:08.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:07 smithi170 ceph-mon[102832]: osdmap e143: 12 total, 12 up, 12 in 2023-09-09T23:49:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1778.443972 secs 2023-09-09T23:49:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:07 smithi082 ceph-mon[98694]: pgmap v1117: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-09T23:49:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:07 smithi082 ceph-mon[98694]: osdmap e143: 12 total, 12 up, 12 in 2023-09-09T23:49:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1780.041622 secs 2023-09-09T23:49:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1780.041622 secs 2023-09-09T23:49:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1780.041622 secs 2023-09-09T23:49:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1780.771016 secs 2023-09-09T23:49:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:09 smithi098 ceph-mon[103124]: pgmap v1119: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-09T23:49:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1780.771016 secs 2023-09-09T23:49:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:09 smithi170 ceph-mon[102832]: pgmap v1119: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-09T23:49:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1780.771016 secs 2023-09-09T23:49:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:09 smithi082 ceph-mon[98694]: pgmap v1119: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-09T23:49:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:11 smithi098 ceph-mon[103124]: pgmap v1120: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-09T23:49:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:11 smithi170 ceph-mon[102832]: pgmap v1120: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-09T23:49:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:11 smithi082 ceph-mon[98694]: pgmap v1120: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-09T23:49:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1783.444070 secs 2023-09-09T23:49:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1783.444070 secs 2023-09-09T23:49:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1783.444070 secs 2023-09-09T23:49:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1785.041682 secs 2023-09-09T23:49:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:13 smithi098 ceph-mon[103124]: pgmap v1121: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-09T23:49:14.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:13 smithi098 ceph-mon[103124]: osdmap e144: 12 total, 12 up, 12 in 2023-09-09T23:49:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1785.041682 secs 2023-09-09T23:49:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:13 smithi170 ceph-mon[102832]: pgmap v1121: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-09T23:49:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:13 smithi170 ceph-mon[102832]: osdmap e144: 12 total, 12 up, 12 in 2023-09-09T23:49:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1785.041682 secs 2023-09-09T23:49:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:13 smithi082 ceph-mon[98694]: pgmap v1121: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-09T23:49:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:13 smithi082 ceph-mon[98694]: osdmap e144: 12 total, 12 up, 12 in 2023-09-09T23:49:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1785.771083 secs 2023-09-09T23:49:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1785.771083 secs 2023-09-09T23:49:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1785.771083 secs 2023-09-09T23:49:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:15 smithi098 ceph-mon[103124]: pgmap v1123: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 18 KiB/s wr, 7 op/s 2023-09-09T23:49:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:15 smithi170 ceph-mon[102832]: pgmap v1123: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 18 KiB/s wr, 7 op/s 2023-09-09T23:49:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:15 smithi082 ceph-mon[98694]: pgmap v1123: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 18 KiB/s wr, 7 op/s 2023-09-09T23:49:18.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:17 smithi098 ceph-mon[103124]: pgmap v1124: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 12 op/s 2023-09-09T23:49:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:17 smithi170 ceph-mon[102832]: pgmap v1124: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 12 op/s 2023-09-09T23:49:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:17 smithi082 ceph-mon[98694]: pgmap v1124: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 12 op/s 2023-09-09T23:49:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1790.041772 secs 2023-09-09T23:49:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:18 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:49:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1790.041772 secs 2023-09-09T23:49:19.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:18 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:49:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1790.041772 secs 2023-09-09T23:49:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:18 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:49:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1790.771147 secs 2023-09-09T23:49:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:19 smithi098 ceph-mon[103124]: pgmap v1125: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-09-09T23:49:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:19 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:49:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:19 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:49:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:19 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:49:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1790.771147 secs 2023-09-09T23:49:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:19 smithi170 ceph-mon[102832]: pgmap v1125: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-09-09T23:49:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:19 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:49:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:19 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:49:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:19 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:49:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1790.771147 secs 2023-09-09T23:49:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:19 smithi082 ceph-mon[98694]: pgmap v1125: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-09-09T23:49:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:19 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:49:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:19 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:49:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:19 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:49:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:22 smithi098 ceph-mon[103124]: pgmap v1126: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-09T23:49:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:22 smithi170 ceph-mon[102832]: pgmap v1126: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-09T23:49:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:22 smithi082 ceph-mon[98694]: pgmap v1126: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-09T23:49:23.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1793.444193 secs 2023-09-09T23:49:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1793.444193 secs 2023-09-09T23:49:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1793.444193 secs 2023-09-09T23:49:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1795.041830 secs 2023-09-09T23:49:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:24 smithi170 ceph-mon[102832]: pgmap v1127: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-09T23:49:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:49:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:49:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1795.041830 secs 2023-09-09T23:49:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:24 smithi082 ceph-mon[98694]: pgmap v1127: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-09T23:49:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:49:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:49:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1795.041830 secs 2023-09-09T23:49:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:24 smithi098 ceph-mon[103124]: pgmap v1127: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-09T23:49:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:49:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:49:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1795.771214 secs 2023-09-09T23:49:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1795.771214 secs 2023-09-09T23:49:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1795.771214 secs 2023-09-09T23:49:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:26 smithi170 ceph-mon[102832]: pgmap v1128: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-09T23:49:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:26 smithi082 ceph-mon[98694]: pgmap v1128: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-09T23:49:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:26 smithi098 ceph-mon[103124]: pgmap v1128: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-09T23:49:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1798.444253 secs 2023-09-09T23:49:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:28 smithi170 ceph-mon[102832]: pgmap v1129: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 78 KiB/s wr, 17 op/s 2023-09-09T23:49:28.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1800.041889 secs 2023-09-09T23:49:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1798.444253 secs 2023-09-09T23:49:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:28 smithi082 ceph-mon[98694]: pgmap v1129: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 78 KiB/s wr, 17 op/s 2023-09-09T23:49:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1800.041889 secs 2023-09-09T23:49:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1798.444253 secs 2023-09-09T23:49:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:28 smithi098 ceph-mon[103124]: pgmap v1129: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 78 KiB/s wr, 17 op/s 2023-09-09T23:49:28.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1800.041889 secs 2023-09-09T23:49:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:29 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:49:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:29 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:49:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:29 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:49:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1800.771278 secs 2023-09-09T23:49:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:30 smithi170 ceph-mon[102832]: pgmap v1130: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 77 KiB/s wr, 12 op/s 2023-09-09T23:49:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1800.771278 secs 2023-09-09T23:49:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:30 smithi082 ceph-mon[98694]: pgmap v1130: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 77 KiB/s wr, 12 op/s 2023-09-09T23:49:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1800.771278 secs 2023-09-09T23:49:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:30 smithi098 ceph-mon[103124]: pgmap v1130: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 77 KiB/s wr, 12 op/s 2023-09-09T23:49:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:32 smithi170 ceph-mon[102832]: pgmap v1131: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 112 KiB/s wr, 18 op/s 2023-09-09T23:49:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:32 smithi082 ceph-mon[98694]: pgmap v1131: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 112 KiB/s wr, 18 op/s 2023-09-09T23:49:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:32 smithi098 ceph-mon[103124]: pgmap v1131: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 112 KiB/s wr, 18 op/s 2023-09-09T23:49:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1805.041947 secs 2023-09-09T23:49:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1805.041947 secs 2023-09-09T23:49:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1805.041947 secs 2023-09-09T23:49:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:34 smithi170 ceph-mon[102832]: pgmap v1132: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 52 KiB/s wr, 12 op/s 2023-09-09T23:49:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:34 smithi082 ceph-mon[98694]: pgmap v1132: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 52 KiB/s wr, 12 op/s 2023-09-09T23:49:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:34 smithi098 ceph-mon[103124]: pgmap v1132: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 52 KiB/s wr, 12 op/s 2023-09-09T23:49:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1805.771343 secs 2023-09-09T23:49:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:49:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1805.771343 secs 2023-09-09T23:49:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:49:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1805.771343 secs 2023-09-09T23:49:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:49:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:36 smithi170 ceph-mon[102832]: pgmap v1133: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 52 KiB/s wr, 12 op/s 2023-09-09T23:49:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:49:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:49:36.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:49:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:36 smithi082 ceph-mon[98694]: pgmap v1133: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 52 KiB/s wr, 12 op/s 2023-09-09T23:49:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:49:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:49:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:49:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:36 smithi098 ceph-mon[103124]: pgmap v1133: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 52 KiB/s wr, 12 op/s 2023-09-09T23:49:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:49:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:49:36.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:49:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:38 smithi170 ceph-mon[102832]: pgmap v1134: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 95 KiB/s wr, 19 op/s 2023-09-09T23:49:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1810.042004 secs 2023-09-09T23:49:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:38 smithi082 ceph-mon[98694]: pgmap v1134: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 95 KiB/s wr, 19 op/s 2023-09-09T23:49:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1810.042004 secs 2023-09-09T23:49:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:38 smithi098 ceph-mon[103124]: pgmap v1134: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 95 KiB/s wr, 19 op/s 2023-09-09T23:49:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1810.042004 secs 2023-09-09T23:49:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1810.771409 secs 2023-09-09T23:49:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:40 smithi170 ceph-mon[102832]: pgmap v1135: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 77 KiB/s wr, 12 op/s 2023-09-09T23:49:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1810.771409 secs 2023-09-09T23:49:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:40 smithi082 ceph-mon[98694]: pgmap v1135: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 77 KiB/s wr, 12 op/s 2023-09-09T23:49:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1810.771409 secs 2023-09-09T23:49:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:40 smithi098 ceph-mon[103124]: pgmap v1135: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 77 KiB/s wr, 12 op/s 2023-09-09T23:49:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:42 smithi170 ceph-mon[102832]: pgmap v1136: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 92 KiB/s wr, 17 op/s 2023-09-09T23:49:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:42 smithi082 ceph-mon[98694]: pgmap v1136: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 92 KiB/s wr, 17 op/s 2023-09-09T23:49:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:42 smithi098 ceph-mon[103124]: pgmap v1136: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 92 KiB/s wr, 17 op/s 2023-09-09T23:49:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1813.444540 secs 2023-09-09T23:49:43.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1815.042092 secs 2023-09-09T23:49:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1813.444540 secs 2023-09-09T23:49:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1815.042092 secs 2023-09-09T23:49:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1813.444540 secs 2023-09-09T23:49:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1815.042092 secs 2023-09-09T23:49:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:44 smithi170 ceph-mon[102832]: pgmap v1137: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-09T23:49:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:49:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:49:44.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:49:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:44 smithi082 ceph-mon[98694]: pgmap v1137: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-09T23:49:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:49:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:49:44.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:49:44.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:44 smithi098 ceph-mon[103124]: pgmap v1137: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-09T23:49:44.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:49:44.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:49:44.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:49:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1815.771477 secs 2023-09-09T23:49:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1815.771477 secs 2023-09-09T23:49:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1815.771477 secs 2023-09-09T23:49:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:46 smithi170 ceph-mon[102832]: pgmap v1138: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-09T23:49:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:46 smithi082 ceph-mon[98694]: pgmap v1138: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-09T23:49:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:46 smithi098 ceph-mon[103124]: pgmap v1138: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-09T23:49:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1818.444602 secs 2023-09-09T23:49:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:48 smithi170 ceph-mon[102832]: pgmap v1139: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 79 KiB/s wr, 16 op/s 2023-09-09T23:49:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1820.042182 secs 2023-09-09T23:49:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1818.444602 secs 2023-09-09T23:49:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:48 smithi082 ceph-mon[98694]: pgmap v1139: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 79 KiB/s wr, 16 op/s 2023-09-09T23:49:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1820.042182 secs 2023-09-09T23:49:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1818.444602 secs 2023-09-09T23:49:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:48 smithi098 ceph-mon[103124]: pgmap v1139: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 79 KiB/s wr, 16 op/s 2023-09-09T23:49:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1820.042182 secs 2023-09-09T23:49:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1820.771541 secs 2023-09-09T23:49:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:50 smithi170 ceph-mon[102832]: pgmap v1140: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-09-09T23:49:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1820.771541 secs 2023-09-09T23:49:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:50 smithi082 ceph-mon[98694]: pgmap v1140: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-09-09T23:49:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1820.771541 secs 2023-09-09T23:49:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:50 smithi098 ceph-mon[103124]: pgmap v1140: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-09-09T23:49:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:52 smithi170 ceph-mon[102832]: pgmap v1141: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 49 KiB/s wr, 14 op/s 2023-09-09T23:49:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:52 smithi082 ceph-mon[98694]: pgmap v1141: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 49 KiB/s wr, 14 op/s 2023-09-09T23:49:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:52 smithi098 ceph-mon[103124]: pgmap v1141: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 49 KiB/s wr, 14 op/s 2023-09-09T23:49:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1823.444661 secs 2023-09-09T23:49:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1825.042240 secs 2023-09-09T23:49:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1823.444661 secs 2023-09-09T23:49:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1825.042240 secs 2023-09-09T23:49:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1823.444661 secs 2023-09-09T23:49:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1825.042240 secs 2023-09-09T23:49:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:54 smithi170 ceph-mon[102832]: pgmap v1142: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 9 op/s 2023-09-09T23:49:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:54 smithi082 ceph-mon[98694]: pgmap v1142: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 9 op/s 2023-09-09T23:49:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:54 smithi098 ceph-mon[103124]: pgmap v1142: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 9 op/s 2023-09-09T23:49:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1825.771610 secs 2023-09-09T23:49:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1825.771610 secs 2023-09-09T23:49:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1825.771610 secs 2023-09-09T23:49:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:56 smithi170 ceph-mon[102832]: pgmap v1143: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 9 op/s 2023-09-09T23:49:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:56 smithi082 ceph-mon[98694]: pgmap v1143: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 9 op/s 2023-09-09T23:49:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:56 smithi098 ceph-mon[103124]: pgmap v1143: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 9 op/s 2023-09-09T23:49:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:49:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:49:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:49:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1828.444812 secs 2023-09-09T23:49:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:58 smithi170 ceph-mon[102832]: pgmap v1144: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 14 op/s 2023-09-09T23:49:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1830.042299 secs 2023-09-09T23:49:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1828.444812 secs 2023-09-09T23:49:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:58 smithi082 ceph-mon[98694]: pgmap v1144: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 14 op/s 2023-09-09T23:49:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1830.042299 secs 2023-09-09T23:49:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1828.444812 secs 2023-09-09T23:49:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:58 smithi098 ceph-mon[103124]: pgmap v1144: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 14 op/s 2023-09-09T23:49:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1830.042299 secs 2023-09-09T23:49:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:49:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:49:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:49:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:49:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:49:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:50:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1830.771678 secs 2023-09-09T23:50:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:00 smithi170 ceph-mon[102832]: pgmap v1145: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 22 KiB/s wr, 9 op/s 2023-09-09T23:50:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:50:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:50:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:50:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-09T23:50:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1830.771678 secs 2023-09-09T23:50:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:00 smithi082 ceph-mon[98694]: pgmap v1145: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 22 KiB/s wr, 9 op/s 2023-09-09T23:50:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:50:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:50:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:50:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-09T23:50:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1830.771678 secs 2023-09-09T23:50:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:00 smithi098 ceph-mon[103124]: pgmap v1145: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 22 KiB/s wr, 9 op/s 2023-09-09T23:50:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:50:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:50:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:50:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-09T23:50:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:02 smithi170 ceph-mon[102832]: pgmap v1146: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 23 KiB/s wr, 14 op/s 2023-09-09T23:50:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:02 smithi082 ceph-mon[98694]: pgmap v1146: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 23 KiB/s wr, 14 op/s 2023-09-09T23:50:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:02 smithi098 ceph-mon[103124]: pgmap v1146: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 23 KiB/s wr, 14 op/s 2023-09-09T23:50:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1833.444887 secs 2023-09-09T23:50:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1835.042373 secs 2023-09-09T23:50:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1833.444887 secs 2023-09-09T23:50:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1835.042373 secs 2023-09-09T23:50:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1833.444887 secs 2023-09-09T23:50:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1835.042373 secs 2023-09-09T23:50:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:04 smithi170 ceph-mon[102832]: pgmap v1147: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 10 op/s 2023-09-09T23:50:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:04 smithi082 ceph-mon[98694]: pgmap v1147: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 10 op/s 2023-09-09T23:50:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:04 smithi098 ceph-mon[103124]: pgmap v1147: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 10 op/s 2023-09-09T23:50:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1835.771746 secs 2023-09-09T23:50:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1835.771746 secs 2023-09-09T23:50:05.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1835.771746 secs 2023-09-09T23:50:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:06 smithi170 ceph-mon[102832]: pgmap v1148: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 10 op/s 2023-09-09T23:50:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:06 smithi082 ceph-mon[98694]: pgmap v1148: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 10 op/s 2023-09-09T23:50:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:06 smithi098 ceph-mon[103124]: pgmap v1148: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 10 op/s 2023-09-09T23:50:07.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1838.444995 secs 2023-09-09T23:50:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1838.444995 secs 2023-09-09T23:50:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1838.444995 secs 2023-09-09T23:50:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:08 smithi170 ceph-mon[102832]: pgmap v1149: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 68 KiB/s wr, 29 op/s 2023-09-09T23:50:08.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:08 smithi170 ceph-mon[102832]: osdmap e145: 12 total, 12 up, 12 in 2023-09-09T23:50:08.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1840.042455 secs 2023-09-09T23:50:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:08 smithi082 ceph-mon[98694]: pgmap v1149: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 68 KiB/s wr, 29 op/s 2023-09-09T23:50:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:08 smithi082 ceph-mon[98694]: osdmap e145: 12 total, 12 up, 12 in 2023-09-09T23:50:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1840.042455 secs 2023-09-09T23:50:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:08 smithi098 ceph-mon[103124]: pgmap v1149: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 68 KiB/s wr, 29 op/s 2023-09-09T23:50:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:08 smithi098 ceph-mon[103124]: osdmap e145: 12 total, 12 up, 12 in 2023-09-09T23:50:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1840.042455 secs 2023-09-09T23:50:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1840.771810 secs 2023-09-09T23:50:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:10 smithi170 ceph-mon[102832]: pgmap v1151: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-09T23:50:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1840.771810 secs 2023-09-09T23:50:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:10 smithi082 ceph-mon[98694]: pgmap v1151: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-09T23:50:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1840.771810 secs 2023-09-09T23:50:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:10 smithi098 ceph-mon[103124]: pgmap v1151: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-09T23:50:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:12 smithi170 ceph-mon[102832]: pgmap v1152: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-09T23:50:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1843.445057 secs 2023-09-09T23:50:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:12 smithi082 ceph-mon[98694]: pgmap v1152: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-09T23:50:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1843.445057 secs 2023-09-09T23:50:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:12 smithi098 ceph-mon[103124]: pgmap v1152: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-09T23:50:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1843.445057 secs 2023-09-09T23:50:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1845.042493 secs 2023-09-09T23:50:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1845.042493 secs 2023-09-09T23:50:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1845.042493 secs 2023-09-09T23:50:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:14 smithi170 ceph-mon[102832]: pgmap v1153: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-09T23:50:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:14 smithi170 ceph-mon[102832]: osdmap e146: 12 total, 12 up, 12 in 2023-09-09T23:50:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:14 smithi082 ceph-mon[98694]: pgmap v1153: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-09T23:50:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:14 smithi082 ceph-mon[98694]: osdmap e146: 12 total, 12 up, 12 in 2023-09-09T23:50:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:14 smithi098 ceph-mon[103124]: pgmap v1153: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-09T23:50:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:14 smithi098 ceph-mon[103124]: osdmap e146: 12 total, 12 up, 12 in 2023-09-09T23:50:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1845.771874 secs 2023-09-09T23:50:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1845.771874 secs 2023-09-09T23:50:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1845.771874 secs 2023-09-09T23:50:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:16 smithi170 ceph-mon[102832]: pgmap v1155: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-09-09T23:50:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:16 smithi082 ceph-mon[98694]: pgmap v1155: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-09-09T23:50:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:16 smithi098 ceph-mon[103124]: pgmap v1155: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-09-09T23:50:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:18 smithi170 ceph-mon[102832]: pgmap v1156: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-09T23:50:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1850.042551 secs 2023-09-09T23:50:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:18 smithi082 ceph-mon[98694]: pgmap v1156: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-09T23:50:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1850.042551 secs 2023-09-09T23:50:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:18 smithi098 ceph-mon[103124]: pgmap v1156: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-09T23:50:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1850.042551 secs 2023-09-09T23:50:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1850.771941 secs 2023-09-09T23:50:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:50:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1850.771941 secs 2023-09-09T23:50:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:50:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1850.771941 secs 2023-09-09T23:50:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:50:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:20 smithi170 ceph-mon[102832]: pgmap v1157: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-09T23:50:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:50:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:50:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:50:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:20 smithi082 ceph-mon[98694]: pgmap v1157: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-09T23:50:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:50:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:50:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:50:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:20 smithi098 ceph-mon[103124]: pgmap v1157: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-09T23:50:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:50:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:50:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:50:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:22 smithi170 ceph-mon[102832]: pgmap v1158: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-09T23:50:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:22 smithi082 ceph-mon[98694]: pgmap v1158: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-09T23:50:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:22 smithi098 ceph-mon[103124]: pgmap v1158: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-09T23:50:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1855.042611 secs 2023-09-09T23:50:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1855.042611 secs 2023-09-09T23:50:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1855.042611 secs 2023-09-09T23:50:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:24 smithi170 ceph-mon[102832]: pgmap v1159: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-09T23:50:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1855.772003 secs 2023-09-09T23:50:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:24 smithi082 ceph-mon[98694]: pgmap v1159: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-09T23:50:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1855.772003 secs 2023-09-09T23:50:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:24 smithi098 ceph-mon[103124]: pgmap v1159: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-09T23:50:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1855.772003 secs 2023-09-09T23:50:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:26 smithi170 ceph-mon[102832]: pgmap v1160: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-09T23:50:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:26 smithi082 ceph-mon[98694]: pgmap v1160: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-09T23:50:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:26 smithi098 ceph-mon[103124]: pgmap v1160: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-09T23:50:27.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1858.445324 secs 2023-09-09T23:50:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1858.445324 secs 2023-09-09T23:50:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1858.445324 secs 2023-09-09T23:50:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:28 smithi082 ceph-mon[98694]: pgmap v1161: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 76 KiB/s wr, 16 op/s 2023-09-09T23:50:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1860.042669 secs 2023-09-09T23:50:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:28 smithi098 ceph-mon[103124]: pgmap v1161: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 76 KiB/s wr, 16 op/s 2023-09-09T23:50:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1860.042669 secs 2023-09-09T23:50:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:28 smithi170 ceph-mon[102832]: pgmap v1161: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 76 KiB/s wr, 16 op/s 2023-09-09T23:50:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1860.042669 secs 2023-09-09T23:50:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1860.772068 secs 2023-09-09T23:50:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:29 smithi170 ceph-mon[102832]: pgmap v1162: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-09T23:50:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1860.772068 secs 2023-09-09T23:50:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:29 smithi082 ceph-mon[98694]: pgmap v1162: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-09T23:50:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1860.772068 secs 2023-09-09T23:50:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:29 smithi098 ceph-mon[103124]: pgmap v1162: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-09T23:50:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:30 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:50:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:50:31.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:50:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:30 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:50:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:50:31.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:50:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:30 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:50:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:50:31.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:50:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:31 smithi082 ceph-mon[98694]: pgmap v1163: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 17 op/s 2023-09-09T23:50:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:31 smithi098 ceph-mon[103124]: pgmap v1163: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 17 op/s 2023-09-09T23:50:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:31 smithi170 ceph-mon[102832]: pgmap v1163: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 17 op/s 2023-09-09T23:50:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1865.042727 secs 2023-09-09T23:50:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:33 smithi082 ceph-mon[98694]: pgmap v1164: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 79 KiB/s wr, 12 op/s 2023-09-09T23:50:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1865.042727 secs 2023-09-09T23:50:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:33 smithi098 ceph-mon[103124]: pgmap v1164: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 79 KiB/s wr, 12 op/s 2023-09-09T23:50:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1865.042727 secs 2023-09-09T23:50:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:33 smithi170 ceph-mon[102832]: pgmap v1164: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 79 KiB/s wr, 12 op/s 2023-09-09T23:50:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1865.772132 secs 2023-09-09T23:50:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:34 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:50:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1865.772132 secs 2023-09-09T23:50:35.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:34 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:50:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1865.772132 secs 2023-09-09T23:50:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:34 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:50:36.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:35 smithi098 ceph-mon[103124]: pgmap v1165: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 79 KiB/s wr, 12 op/s 2023-09-09T23:50:36.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:35 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:50:36.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:35 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:50:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:35 smithi170 ceph-mon[102832]: pgmap v1165: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 79 KiB/s wr, 12 op/s 2023-09-09T23:50:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:35 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:50:36.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:35 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:50:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:35 smithi082 ceph-mon[98694]: pgmap v1165: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 79 KiB/s wr, 12 op/s 2023-09-09T23:50:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:35 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:50:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:35 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:50:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1868.445451 secs 2023-09-09T23:50:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:37 smithi098 ceph-mon[103124]: pgmap v1166: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 143 KiB/s wr, 18 op/s 2023-09-09T23:50:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1868.445451 secs 2023-09-09T23:50:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:37 smithi170 ceph-mon[102832]: pgmap v1166: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 143 KiB/s wr, 18 op/s 2023-09-09T23:50:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1868.445451 secs 2023-09-09T23:50:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:37 smithi082 ceph-mon[98694]: pgmap v1166: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 143 KiB/s wr, 18 op/s 2023-09-09T23:50:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1870.042787 secs 2023-09-09T23:50:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:38 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:50:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1870.042787 secs 2023-09-09T23:50:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:38 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:50:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1870.042787 secs 2023-09-09T23:50:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:38 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:50:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1870.772197 secs 2023-09-09T23:50:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:39 smithi098 ceph-mon[103124]: pgmap v1167: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-09T23:50:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:39 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:50:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:39 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:50:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1870.772197 secs 2023-09-09T23:50:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:39 smithi170 ceph-mon[102832]: pgmap v1167: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-09T23:50:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:39 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:50:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:39 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:50:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1870.772197 secs 2023-09-09T23:50:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:39 smithi082 ceph-mon[98694]: pgmap v1167: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-09T23:50:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:39 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:50:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:39 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:50:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:41 smithi098 ceph-mon[103124]: pgmap v1168: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 103 KiB/s wr, 17 op/s 2023-09-09T23:50:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:41 smithi170 ceph-mon[102832]: pgmap v1168: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 103 KiB/s wr, 17 op/s 2023-09-09T23:50:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:41 smithi082 ceph-mon[98694]: pgmap v1168: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 103 KiB/s wr, 17 op/s 2023-09-09T23:50:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1873.445545 secs 2023-09-09T23:50:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1873.445545 secs 2023-09-09T23:50:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1873.445545 secs 2023-09-09T23:50:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1875.042846 secs 2023-09-09T23:50:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:43 smithi098 ceph-mon[103124]: pgmap v1169: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-09T23:50:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:50:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1875.042846 secs 2023-09-09T23:50:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:43 smithi170 ceph-mon[102832]: pgmap v1169: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-09T23:50:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:50:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1875.042846 secs 2023-09-09T23:50:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:43 smithi082 ceph-mon[98694]: pgmap v1169: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-09T23:50:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:50:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1875.772263 secs 2023-09-09T23:50:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1875.772263 secs 2023-09-09T23:50:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1875.772263 secs 2023-09-09T23:50:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:45 smithi098 ceph-mon[103124]: pgmap v1170: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-09T23:50:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:45 smithi170 ceph-mon[102832]: pgmap v1170: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-09T23:50:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:45 smithi082 ceph-mon[98694]: pgmap v1170: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-09T23:50:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1878.445604 secs 2023-09-09T23:50:48.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:47 smithi098 ceph-mon[103124]: pgmap v1171: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 88 KiB/s wr, 17 op/s 2023-09-09T23:50:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1878.445604 secs 2023-09-09T23:50:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:47 smithi170 ceph-mon[102832]: pgmap v1171: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 88 KiB/s wr, 17 op/s 2023-09-09T23:50:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1878.445604 secs 2023-09-09T23:50:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:47 smithi082 ceph-mon[98694]: pgmap v1171: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 88 KiB/s wr, 17 op/s 2023-09-09T23:50:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1880.042931 secs 2023-09-09T23:50:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1880.042931 secs 2023-09-09T23:50:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1880.042931 secs 2023-09-09T23:50:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1880.772332 secs 2023-09-09T23:50:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:49 smithi098 ceph-mon[103124]: pgmap v1172: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 24 KiB/s wr, 10 op/s 2023-09-09T23:50:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1880.772332 secs 2023-09-09T23:50:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:49 smithi170 ceph-mon[102832]: pgmap v1172: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 24 KiB/s wr, 10 op/s 2023-09-09T23:50:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1880.772332 secs 2023-09-09T23:50:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:49 smithi082 ceph-mon[98694]: pgmap v1172: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 24 KiB/s wr, 10 op/s 2023-09-09T23:50:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:51 smithi098 ceph-mon[103124]: pgmap v1173: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 26 KiB/s wr, 16 op/s 2023-09-09T23:50:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:51 smithi170 ceph-mon[102832]: pgmap v1173: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 26 KiB/s wr, 16 op/s 2023-09-09T23:50:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:51 smithi082 ceph-mon[98694]: pgmap v1173: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 26 KiB/s wr, 16 op/s 2023-09-09T23:50:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1883.445672 secs 2023-09-09T23:50:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1883.445672 secs 2023-09-09T23:50:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1883.445672 secs 2023-09-09T23:50:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1885.043002 secs 2023-09-09T23:50:54.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:54 smithi098 ceph-mon[103124]: pgmap v1174: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-09T23:50:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1885.043002 secs 2023-09-09T23:50:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:54 smithi170 ceph-mon[102832]: pgmap v1174: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-09T23:50:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1885.043002 secs 2023-09-09T23:50:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:54 smithi082 ceph-mon[98694]: pgmap v1174: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-09T23:50:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1885.772398 secs 2023-09-09T23:50:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1885.772398 secs 2023-09-09T23:50:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1885.772398 secs 2023-09-09T23:50:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:56 smithi170 ceph-mon[102832]: pgmap v1175: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-09T23:50:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:56 smithi082 ceph-mon[98694]: pgmap v1175: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-09T23:50:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:56 smithi098 ceph-mon[103124]: pgmap v1175: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-09T23:50:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1888.445734 secs 2023-09-09T23:50:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:58 smithi170 ceph-mon[102832]: pgmap v1176: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 24 KiB/s wr, 15 op/s 2023-09-09T23:50:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:50:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1890.043060 secs 2023-09-09T23:50:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1888.445734 secs 2023-09-09T23:50:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:58 smithi082 ceph-mon[98694]: pgmap v1176: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 24 KiB/s wr, 15 op/s 2023-09-09T23:50:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:50:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1890.043060 secs 2023-09-09T23:50:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1888.445734 secs 2023-09-09T23:50:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:58 smithi098 ceph-mon[103124]: pgmap v1176: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 24 KiB/s wr, 15 op/s 2023-09-09T23:50:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:50:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1890.043060 secs 2023-09-09T23:51:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1890.772465 secs 2023-09-09T23:51:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:00 smithi170 ceph-mon[102832]: pgmap v1177: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2023-09-09T23:51:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:51:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1890.772465 secs 2023-09-09T23:51:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:00 smithi082 ceph-mon[98694]: pgmap v1177: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2023-09-09T23:51:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:51:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1890.772465 secs 2023-09-09T23:51:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:00 smithi098 ceph-mon[103124]: pgmap v1177: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2023-09-09T23:51:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:51:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:51:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:51:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:51:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:51:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:51:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:51:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:51:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:51:01.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:51:02.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:02 smithi170 ceph-mon[102832]: pgmap v1178: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 15 op/s 2023-09-09T23:51:02.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1893.445808 secs 2023-09-09T23:51:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:02 smithi082 ceph-mon[98694]: pgmap v1178: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 15 op/s 2023-09-09T23:51:02.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1893.445808 secs 2023-09-09T23:51:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:02 smithi098 ceph-mon[103124]: pgmap v1178: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 15 op/s 2023-09-09T23:51:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1893.445808 secs 2023-09-09T23:51:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1895.043135 secs 2023-09-09T23:51:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1895.043135 secs 2023-09-09T23:51:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1895.043135 secs 2023-09-09T23:51:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:04 smithi170 ceph-mon[102832]: pgmap v1179: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 KiB/s wr, 10 op/s 2023-09-09T23:51:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1895.772490 secs 2023-09-09T23:51:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:04 smithi082 ceph-mon[98694]: pgmap v1179: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 KiB/s wr, 10 op/s 2023-09-09T23:51:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1895.772490 secs 2023-09-09T23:51:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:04 smithi098 ceph-mon[103124]: pgmap v1179: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 KiB/s wr, 10 op/s 2023-09-09T23:51:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1895.772490 secs 2023-09-09T23:51:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:06 smithi170 ceph-mon[102832]: pgmap v1180: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 KiB/s wr, 10 op/s 2023-09-09T23:51:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:06 smithi082 ceph-mon[98694]: pgmap v1180: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 KiB/s wr, 10 op/s 2023-09-09T23:51:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:06 smithi098 ceph-mon[103124]: pgmap v1180: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 KiB/s wr, 10 op/s 2023-09-09T23:51:07.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1898.445869 secs 2023-09-09T23:51:07.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1898.445869 secs 2023-09-09T23:51:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1898.445869 secs 2023-09-09T23:51:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:08 smithi170 ceph-mon[102832]: pgmap v1181: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-09T23:51:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:08 smithi170 ceph-mon[102832]: osdmap e147: 12 total, 12 up, 12 in 2023-09-09T23:51:08.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1900.043243 secs 2023-09-09T23:51:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:08 smithi082 ceph-mon[98694]: pgmap v1181: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-09T23:51:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:08 smithi082 ceph-mon[98694]: osdmap e147: 12 total, 12 up, 12 in 2023-09-09T23:51:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1900.043243 secs 2023-09-09T23:51:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:08 smithi098 ceph-mon[103124]: pgmap v1181: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-09T23:51:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:08 smithi098 ceph-mon[103124]: osdmap e147: 12 total, 12 up, 12 in 2023-09-09T23:51:08.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1900.043243 secs 2023-09-09T23:51:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1900.772558 secs 2023-09-09T23:51:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:09 smithi170 ceph-mon[102832]: pgmap v1183: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-09T23:51:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1900.772558 secs 2023-09-09T23:51:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:09 smithi082 ceph-mon[98694]: pgmap v1183: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-09T23:51:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1900.772558 secs 2023-09-09T23:51:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:09 smithi098 ceph-mon[103124]: pgmap v1183: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-09T23:51:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:11 smithi170 ceph-mon[102832]: pgmap v1184: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-09T23:51:12.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:11 smithi082 ceph-mon[98694]: pgmap v1184: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-09T23:51:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:11 smithi098 ceph-mon[103124]: pgmap v1184: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-09T23:51:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1903.445944 secs 2023-09-09T23:51:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1903.445944 secs 2023-09-09T23:51:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1903.445944 secs 2023-09-09T23:51:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1905.043302 secs 2023-09-09T23:51:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:13 smithi170 ceph-mon[102832]: pgmap v1185: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-09T23:51:14.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:13 smithi170 ceph-mon[102832]: osdmap e148: 12 total, 12 up, 12 in 2023-09-09T23:51:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1905.043302 secs 2023-09-09T23:51:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:13 smithi082 ceph-mon[98694]: pgmap v1185: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-09T23:51:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:13 smithi082 ceph-mon[98694]: osdmap e148: 12 total, 12 up, 12 in 2023-09-09T23:51:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1905.043302 secs 2023-09-09T23:51:14.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:13 smithi098 ceph-mon[103124]: pgmap v1185: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-09T23:51:14.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:13 smithi098 ceph-mon[103124]: osdmap e148: 12 total, 12 up, 12 in 2023-09-09T23:51:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1905.772623 secs 2023-09-09T23:51:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1905.772623 secs 2023-09-09T23:51:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1905.772623 secs 2023-09-09T23:51:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:15 smithi082 ceph-mon[98694]: pgmap v1187: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 383 B/s wr, 6 op/s 2023-09-09T23:51:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:15 smithi098 ceph-mon[103124]: pgmap v1187: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 383 B/s wr, 6 op/s 2023-09-09T23:51:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:15 smithi170 ceph-mon[102832]: pgmap v1187: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 383 B/s wr, 6 op/s 2023-09-09T23:51:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:17 smithi082 ceph-mon[98694]: pgmap v1188: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 966 B/s wr, 12 op/s 2023-09-09T23:51:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:17 smithi098 ceph-mon[103124]: pgmap v1188: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 966 B/s wr, 12 op/s 2023-09-09T23:51:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:17 smithi170 ceph-mon[102832]: pgmap v1188: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 966 B/s wr, 12 op/s 2023-09-09T23:51:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1910.043361 secs 2023-09-09T23:51:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:18 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:51:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1910.043361 secs 2023-09-09T23:51:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:18 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:51:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1910.043361 secs 2023-09-09T23:51:19.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:18 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:51:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1910.772690 secs 2023-09-09T23:51:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:19 smithi082 ceph-mon[98694]: pgmap v1189: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 921 B/s wr, 11 op/s 2023-09-09T23:51:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:19 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:51:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:19 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:51:20.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:19 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:51:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1910.772690 secs 2023-09-09T23:51:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:19 smithi098 ceph-mon[103124]: pgmap v1189: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 921 B/s wr, 11 op/s 2023-09-09T23:51:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:19 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:51:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:19 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:51:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:19 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:51:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1910.772690 secs 2023-09-09T23:51:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:19 smithi170 ceph-mon[102832]: pgmap v1189: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 921 B/s wr, 11 op/s 2023-09-09T23:51:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:19 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:51:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:19 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:51:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:19 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:51:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:21 smithi082 ceph-mon[98694]: pgmap v1190: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 51 KiB/s wr, 12 op/s 2023-09-09T23:51:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:21 smithi098 ceph-mon[103124]: pgmap v1190: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 51 KiB/s wr, 12 op/s 2023-09-09T23:51:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:21 smithi170 ceph-mon[102832]: pgmap v1190: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 51 KiB/s wr, 12 op/s 2023-09-09T23:51:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1915.043444 secs 2023-09-09T23:51:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:23 smithi082 ceph-mon[98694]: pgmap v1191: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 51 KiB/s wr, 12 op/s 2023-09-09T23:51:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1915.043444 secs 2023-09-09T23:51:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:23 smithi098 ceph-mon[103124]: pgmap v1191: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 51 KiB/s wr, 12 op/s 2023-09-09T23:51:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1915.043444 secs 2023-09-09T23:51:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:23 smithi170 ceph-mon[102832]: pgmap v1191: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 51 KiB/s wr, 12 op/s 2023-09-09T23:51:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1915.772754 secs 2023-09-09T23:51:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1915.772754 secs 2023-09-09T23:51:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1915.772754 secs 2023-09-09T23:51:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:25 smithi082 ceph-mon[98694]: pgmap v1192: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-09T23:51:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:25 smithi098 ceph-mon[103124]: pgmap v1192: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-09T23:51:26.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:25 smithi170 ceph-mon[102832]: pgmap v1192: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-09T23:51:28.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:27 smithi082 ceph-mon[98694]: pgmap v1193: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 61 KiB/s wr, 16 op/s 2023-09-09T23:51:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:27 smithi098 ceph-mon[103124]: pgmap v1193: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 61 KiB/s wr, 16 op/s 2023-09-09T23:51:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:27 smithi170 ceph-mon[102832]: pgmap v1193: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 61 KiB/s wr, 16 op/s 2023-09-09T23:51:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1920.043461 secs 2023-09-09T23:51:29.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1920.043461 secs 2023-09-09T23:51:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1920.043461 secs 2023-09-09T23:51:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1920.772819 secs 2023-09-09T23:51:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:29 smithi098 ceph-mon[103124]: pgmap v1194: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-09T23:51:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1920.772819 secs 2023-09-09T23:51:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:29 smithi170 ceph-mon[102832]: pgmap v1194: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-09T23:51:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1920.772819 secs 2023-09-09T23:51:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:29 smithi082 ceph-mon[98694]: pgmap v1194: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-09T23:51:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:31 smithi098 ceph-mon[103124]: pgmap v1195: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 97 KiB/s wr, 16 op/s 2023-09-09T23:51:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:31 smithi170 ceph-mon[102832]: pgmap v1195: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 97 KiB/s wr, 16 op/s 2023-09-09T23:51:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:31 smithi082 ceph-mon[98694]: pgmap v1195: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 97 KiB/s wr, 16 op/s 2023-09-09T23:51:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1925.043549 secs 2023-09-09T23:51:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:33 smithi098 ceph-mon[103124]: pgmap v1196: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-09T23:51:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1925.043549 secs 2023-09-09T23:51:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:33 smithi170 ceph-mon[102832]: pgmap v1196: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-09T23:51:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1925.043549 secs 2023-09-09T23:51:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:33 smithi082 ceph-mon[98694]: pgmap v1196: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-09T23:51:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1925.772886 secs 2023-09-09T23:51:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1925.772886 secs 2023-09-09T23:51:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1925.772886 secs 2023-09-09T23:51:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:35 smithi098 ceph-mon[103124]: pgmap v1197: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-09T23:51:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:35 smithi170 ceph-mon[102832]: pgmap v1197: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-09T23:51:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:35 smithi082 ceph-mon[98694]: pgmap v1197: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-09T23:51:38.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:37 smithi098 ceph-mon[103124]: pgmap v1198: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 69 KiB/s wr, 17 op/s 2023-09-09T23:51:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:37 smithi170 ceph-mon[102832]: pgmap v1198: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 69 KiB/s wr, 17 op/s 2023-09-09T23:51:38.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:37 smithi082 ceph-mon[98694]: pgmap v1198: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 69 KiB/s wr, 17 op/s 2023-09-09T23:51:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1930.043608 secs 2023-09-09T23:51:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1930.043608 secs 2023-09-09T23:51:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1930.043608 secs 2023-09-09T23:51:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1930.772950 secs 2023-09-09T23:51:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:39 smithi098 ceph-mon[103124]: pgmap v1199: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-09T23:51:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1930.772950 secs 2023-09-09T23:51:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:39 smithi170 ceph-mon[102832]: pgmap v1199: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-09T23:51:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1930.772950 secs 2023-09-09T23:51:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:39 smithi082 ceph-mon[98694]: pgmap v1199: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-09T23:51:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:41 smithi098 ceph-mon[103124]: pgmap v1200: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 97 KiB/s wr, 16 op/s 2023-09-09T23:51:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:41 smithi170 ceph-mon[102832]: pgmap v1200: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 97 KiB/s wr, 16 op/s 2023-09-09T23:51:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:41 smithi082 ceph-mon[98694]: pgmap v1200: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 97 KiB/s wr, 16 op/s 2023-09-09T23:51:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1933.446528 secs 2023-09-09T23:51:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1933.446528 secs 2023-09-09T23:51:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1933.446528 secs 2023-09-09T23:51:44.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1935.043666 secs 2023-09-09T23:51:44.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:43 smithi098 ceph-mon[103124]: pgmap v1201: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-09T23:51:44.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:51:44.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:51:44.258 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:51:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1935.043666 secs 2023-09-09T23:51:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:43 smithi170 ceph-mon[102832]: pgmap v1201: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-09T23:51:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:51:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:51:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:51:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1935.043666 secs 2023-09-09T23:51:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:43 smithi082 ceph-mon[98694]: pgmap v1201: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-09T23:51:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:51:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:51:44.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:51:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1935.773019 secs 2023-09-09T23:51:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1935.773019 secs 2023-09-09T23:51:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1935.773019 secs 2023-09-09T23:51:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:45 smithi098 ceph-mon[103124]: pgmap v1202: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-09T23:51:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:45 smithi170 ceph-mon[102832]: pgmap v1202: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-09T23:51:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:45 smithi082 ceph-mon[98694]: pgmap v1202: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-09T23:51:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1938.446626 secs 2023-09-09T23:51:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:47 smithi098 ceph-mon[103124]: pgmap v1203: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 84 KiB/s wr, 16 op/s 2023-09-09T23:51:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1938.446626 secs 2023-09-09T23:51:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:47 smithi170 ceph-mon[102832]: pgmap v1203: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 84 KiB/s wr, 16 op/s 2023-09-09T23:51:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1938.446626 secs 2023-09-09T23:51:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:47 smithi082 ceph-mon[98694]: pgmap v1203: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 84 KiB/s wr, 16 op/s 2023-09-09T23:51:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1940.043748 secs 2023-09-09T23:51:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1940.043748 secs 2023-09-09T23:51:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1940.043748 secs 2023-09-09T23:51:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1940.773086 secs 2023-09-09T23:51:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:50 smithi170 ceph-mon[102832]: pgmap v1204: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 69 KiB/s wr, 10 op/s 2023-09-09T23:51:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1940.773086 secs 2023-09-09T23:51:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:50 smithi082 ceph-mon[98694]: pgmap v1204: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 69 KiB/s wr, 10 op/s 2023-09-09T23:51:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1940.773086 secs 2023-09-09T23:51:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:50 smithi098 ceph-mon[103124]: pgmap v1204: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 69 KiB/s wr, 10 op/s 2023-09-09T23:51:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:52 smithi170 ceph-mon[102832]: pgmap v1205: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 82 KiB/s wr, 15 op/s 2023-09-09T23:51:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:52 smithi082 ceph-mon[98694]: pgmap v1205: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 82 KiB/s wr, 15 op/s 2023-09-09T23:51:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:52 smithi098 ceph-mon[103124]: pgmap v1205: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 82 KiB/s wr, 15 op/s 2023-09-09T23:51:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1943.446730 secs 2023-09-09T23:51:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1943.446730 secs 2023-09-09T23:51:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1943.446730 secs 2023-09-09T23:51:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1945.043804 secs 2023-09-09T23:51:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:54 smithi170 ceph-mon[102832]: pgmap v1206: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-09T23:51:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1945.043804 secs 2023-09-09T23:51:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:54 smithi082 ceph-mon[98694]: pgmap v1206: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-09T23:51:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1945.043804 secs 2023-09-09T23:51:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:54 smithi098 ceph-mon[103124]: pgmap v1206: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-09T23:51:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1945.773151 secs 2023-09-09T23:51:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1945.773151 secs 2023-09-09T23:51:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1945.773151 secs 2023-09-09T23:51:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:56 smithi170 ceph-mon[102832]: pgmap v1207: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-09T23:51:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:56 smithi082 ceph-mon[98694]: pgmap v1207: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-09T23:51:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:56 smithi098 ceph-mon[103124]: pgmap v1207: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-09T23:51:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1948.446799 secs 2023-09-09T23:51:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:58 smithi170 ceph-mon[102832]: pgmap v1208: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 46 KiB/s wr, 15 op/s 2023-09-09T23:51:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:51:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1950.043888 secs 2023-09-09T23:51:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1948.446799 secs 2023-09-09T23:51:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:58 smithi082 ceph-mon[98694]: pgmap v1208: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 46 KiB/s wr, 15 op/s 2023-09-09T23:51:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:51:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1950.043888 secs 2023-09-09T23:51:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1948.446799 secs 2023-09-09T23:51:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:58 smithi098 ceph-mon[103124]: pgmap v1208: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 46 KiB/s wr, 15 op/s 2023-09-09T23:51:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:51:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1950.043888 secs 2023-09-09T23:52:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1950.773230 secs 2023-09-09T23:52:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:00 smithi170 ceph-mon[102832]: pgmap v1209: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-09-09T23:52:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1950.773230 secs 2023-09-09T23:52:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:00 smithi082 ceph-mon[98694]: pgmap v1209: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-09-09T23:52:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1950.773230 secs 2023-09-09T23:52:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:00 smithi098 ceph-mon[103124]: pgmap v1209: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-09-09T23:52:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:52:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:52:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:52:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:02 smithi170 ceph-mon[102832]: pgmap v1210: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 24 KiB/s wr, 14 op/s 2023-09-09T23:52:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:52:02.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:52:02.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:52:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:02 smithi082 ceph-mon[98694]: pgmap v1210: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 24 KiB/s wr, 14 op/s 2023-09-09T23:52:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:52:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:52:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:52:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:02 smithi098 ceph-mon[103124]: pgmap v1210: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 24 KiB/s wr, 14 op/s 2023-09-09T23:52:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:52:02.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:52:02.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:52:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1953.446862 secs 2023-09-09T23:52:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1955.043946 secs 2023-09-09T23:52:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1953.446862 secs 2023-09-09T23:52:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1955.043946 secs 2023-09-09T23:52:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1953.446862 secs 2023-09-09T23:52:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1955.043946 secs 2023-09-09T23:52:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:04 smithi170 ceph-mon[102832]: pgmap v1211: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 10 op/s 2023-09-09T23:52:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:04 smithi082 ceph-mon[98694]: pgmap v1211: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 10 op/s 2023-09-09T23:52:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:04 smithi098 ceph-mon[103124]: pgmap v1211: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 10 op/s 2023-09-09T23:52:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1955.773297 secs 2023-09-09T23:52:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1955.773297 secs 2023-09-09T23:52:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1955.773297 secs 2023-09-09T23:52:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:06 smithi170 ceph-mon[102832]: pgmap v1212: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 10 op/s 2023-09-09T23:52:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:06 smithi082 ceph-mon[98694]: pgmap v1212: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 10 op/s 2023-09-09T23:52:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:06 smithi098 ceph-mon[103124]: pgmap v1212: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 10 op/s 2023-09-09T23:52:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1958.446921 secs 2023-09-09T23:52:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:08 smithi170 ceph-mon[102832]: pgmap v1213: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-09T23:52:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:08 smithi170 ceph-mon[102832]: osdmap e149: 12 total, 12 up, 12 in 2023-09-09T23:52:08.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1960.044006 secs 2023-09-09T23:52:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1958.446921 secs 2023-09-09T23:52:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:08 smithi082 ceph-mon[98694]: pgmap v1213: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-09T23:52:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:08 smithi082 ceph-mon[98694]: osdmap e149: 12 total, 12 up, 12 in 2023-09-09T23:52:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1960.044006 secs 2023-09-09T23:52:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1958.446921 secs 2023-09-09T23:52:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:08 smithi098 ceph-mon[103124]: pgmap v1213: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-09T23:52:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:08 smithi098 ceph-mon[103124]: osdmap e149: 12 total, 12 up, 12 in 2023-09-09T23:52:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1960.044006 secs 2023-09-09T23:52:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1960.773360 secs 2023-09-09T23:52:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:10 smithi170 ceph-mon[102832]: pgmap v1215: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-09T23:52:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1960.773360 secs 2023-09-09T23:52:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:10 smithi082 ceph-mon[98694]: pgmap v1215: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-09T23:52:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1960.773360 secs 2023-09-09T23:52:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:10 smithi098 ceph-mon[103124]: pgmap v1215: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-09T23:52:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:12 smithi170 ceph-mon[102832]: pgmap v1216: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-09T23:52:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:12 smithi082 ceph-mon[98694]: pgmap v1216: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-09T23:52:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:12 smithi098 ceph-mon[103124]: pgmap v1216: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-09T23:52:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1963.447035 secs 2023-09-09T23:52:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1965.044064 secs 2023-09-09T23:52:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1963.447035 secs 2023-09-09T23:52:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1965.044064 secs 2023-09-09T23:52:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1963.447035 secs 2023-09-09T23:52:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1965.044064 secs 2023-09-09T23:52:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:14 smithi170 ceph-mon[102832]: pgmap v1217: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-09T23:52:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:14 smithi170 ceph-mon[102832]: osdmap e150: 12 total, 12 up, 12 in 2023-09-09T23:52:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:14 smithi082 ceph-mon[98694]: pgmap v1217: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-09T23:52:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:14 smithi082 ceph-mon[98694]: osdmap e150: 12 total, 12 up, 12 in 2023-09-09T23:52:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:14 smithi098 ceph-mon[103124]: pgmap v1217: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-09T23:52:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:14 smithi098 ceph-mon[103124]: osdmap e150: 12 total, 12 up, 12 in 2023-09-09T23:52:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1965.773451 secs 2023-09-09T23:52:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1965.773451 secs 2023-09-09T23:52:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1965.773451 secs 2023-09-09T23:52:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:16 smithi170 ceph-mon[102832]: pgmap v1219: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-09-09T23:52:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:16 smithi082 ceph-mon[98694]: pgmap v1219: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-09-09T23:52:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:16 smithi098 ceph-mon[103124]: pgmap v1219: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-09-09T23:52:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:18 smithi170 ceph-mon[102832]: pgmap v1220: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-09-09T23:52:18.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1970.044120 secs 2023-09-09T23:52:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:18 smithi082 ceph-mon[98694]: pgmap v1220: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-09-09T23:52:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1970.044120 secs 2023-09-09T23:52:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:18 smithi098 ceph-mon[103124]: pgmap v1220: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-09-09T23:52:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1970.044120 secs 2023-09-09T23:52:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:52:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:52:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:52:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1970.773488 secs 2023-09-09T23:52:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:20 smithi170 ceph-mon[102832]: pgmap v1221: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-09T23:52:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:52:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:52:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:52:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1970.773488 secs 2023-09-09T23:52:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:20 smithi082 ceph-mon[98694]: pgmap v1221: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-09T23:52:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:52:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:52:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:52:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1970.773488 secs 2023-09-09T23:52:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:20 smithi098 ceph-mon[103124]: pgmap v1221: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-09T23:52:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:52:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:52:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:52:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:22 smithi170 ceph-mon[102832]: pgmap v1222: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 63 KiB/s wr, 14 op/s 2023-09-09T23:52:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:22 smithi082 ceph-mon[98694]: pgmap v1222: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 63 KiB/s wr, 14 op/s 2023-09-09T23:52:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:22 smithi098 ceph-mon[103124]: pgmap v1222: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 63 KiB/s wr, 14 op/s 2023-09-09T23:52:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1975.044175 secs 2023-09-09T23:52:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1975.044175 secs 2023-09-09T23:52:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1975.044175 secs 2023-09-09T23:52:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:24 smithi170 ceph-mon[102832]: pgmap v1223: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 63 KiB/s wr, 14 op/s 2023-09-09T23:52:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:24 smithi082 ceph-mon[98694]: pgmap v1223: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 63 KiB/s wr, 14 op/s 2023-09-09T23:52:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:24 smithi098 ceph-mon[103124]: pgmap v1223: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 63 KiB/s wr, 14 op/s 2023-09-09T23:52:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1975.773554 secs 2023-09-09T23:52:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1975.773554 secs 2023-09-09T23:52:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1975.773554 secs 2023-09-09T23:52:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:26 smithi170 ceph-mon[102832]: pgmap v1224: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 58 KiB/s wr, 13 op/s 2023-09-09T23:52:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:26 smithi082 ceph-mon[98694]: pgmap v1224: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 58 KiB/s wr, 13 op/s 2023-09-09T23:52:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:26 smithi098 ceph-mon[103124]: pgmap v1224: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 58 KiB/s wr, 13 op/s 2023-09-09T23:52:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:28 smithi170 ceph-mon[102832]: pgmap v1225: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 92 KiB/s wr, 35 op/s 2023-09-09T23:52:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1980.044234 secs 2023-09-09T23:52:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:28 smithi082 ceph-mon[98694]: pgmap v1225: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 92 KiB/s wr, 35 op/s 2023-09-09T23:52:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1980.044234 secs 2023-09-09T23:52:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:28 smithi098 ceph-mon[103124]: pgmap v1225: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 92 KiB/s wr, 35 op/s 2023-09-09T23:52:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1980.044234 secs 2023-09-09T23:52:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1980.773621 secs 2023-09-09T23:52:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:30 smithi170 ceph-mon[102832]: pgmap v1226: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 75 KiB/s wr, 30 op/s 2023-09-09T23:52:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1980.773621 secs 2023-09-09T23:52:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:30 smithi082 ceph-mon[98694]: pgmap v1226: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 75 KiB/s wr, 30 op/s 2023-09-09T23:52:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1980.773621 secs 2023-09-09T23:52:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:30 smithi098 ceph-mon[103124]: pgmap v1226: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 75 KiB/s wr, 30 op/s 2023-09-09T23:52:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:32 smithi170 ceph-mon[102832]: pgmap v1227: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 127 KiB/s wr, 36 op/s 2023-09-09T23:52:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:32 smithi082 ceph-mon[98694]: pgmap v1227: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 127 KiB/s wr, 36 op/s 2023-09-09T23:52:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:32 smithi098 ceph-mon[103124]: pgmap v1227: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 127 KiB/s wr, 36 op/s 2023-09-09T23:52:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1985.044303 secs 2023-09-09T23:52:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1985.044303 secs 2023-09-09T23:52:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1985.044303 secs 2023-09-09T23:52:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:34 smithi170 ceph-mon[102832]: pgmap v1228: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-09T23:52:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:34 smithi082 ceph-mon[98694]: pgmap v1228: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-09T23:52:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:34 smithi098 ceph-mon[103124]: pgmap v1228: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-09T23:52:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1985.773686 secs 2023-09-09T23:52:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1985.773686 secs 2023-09-09T23:52:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1985.773686 secs 2023-09-09T23:52:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:36 smithi170 ceph-mon[102832]: pgmap v1229: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-09T23:52:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:36 smithi082 ceph-mon[98694]: pgmap v1229: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-09T23:52:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:36 smithi098 ceph-mon[103124]: pgmap v1229: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-09T23:52:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1988.447478 secs 2023-09-09T23:52:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:38 smithi170 ceph-mon[102832]: pgmap v1230: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 108 KiB/s wr, 36 op/s 2023-09-09T23:52:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1990.044386 secs 2023-09-09T23:52:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1988.447478 secs 2023-09-09T23:52:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:38 smithi082 ceph-mon[98694]: pgmap v1230: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 108 KiB/s wr, 36 op/s 2023-09-09T23:52:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1990.044386 secs 2023-09-09T23:52:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1988.447478 secs 2023-09-09T23:52:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:38 smithi098 ceph-mon[103124]: pgmap v1230: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 108 KiB/s wr, 36 op/s 2023-09-09T23:52:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1990.044386 secs 2023-09-09T23:52:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1990.773750 secs 2023-09-09T23:52:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:39 smithi082 ceph-mon[98694]: pgmap v1231: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-09T23:52:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:39 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:52:40.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:39 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:52:40.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:39 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:52:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1990.773750 secs 2023-09-09T23:52:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:39 smithi098 ceph-mon[103124]: pgmap v1231: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-09T23:52:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:39 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:52:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:39 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:52:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:39 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:52:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1990.773750 secs 2023-09-09T23:52:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:39 smithi170 ceph-mon[102832]: pgmap v1231: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-09T23:52:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:39 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:52:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:39 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:52:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:39 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:52:42.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:41 smithi082 ceph-mon[98694]: pgmap v1232: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 88 KiB/s wr, 17 op/s 2023-09-09T23:52:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:41 smithi098 ceph-mon[103124]: pgmap v1232: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 88 KiB/s wr, 17 op/s 2023-09-09T23:52:42.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:41 smithi170 ceph-mon[102832]: pgmap v1232: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 88 KiB/s wr, 17 op/s 2023-09-09T23:52:43.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1993.447571 secs 2023-09-09T23:52:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1993.447571 secs 2023-09-09T23:52:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1993.447571 secs 2023-09-09T23:52:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1995.044445 secs 2023-09-09T23:52:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:43 smithi082 ceph-mon[98694]: pgmap v1233: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-09T23:52:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1995.044445 secs 2023-09-09T23:52:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:43 smithi098 ceph-mon[103124]: pgmap v1233: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-09T23:52:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1995.044445 secs 2023-09-09T23:52:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:43 smithi170 ceph-mon[102832]: pgmap v1233: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-09T23:52:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 1995.773816 secs 2023-09-09T23:52:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 1995.773816 secs 2023-09-09T23:52:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 1995.773816 secs 2023-09-09T23:52:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:45 smithi082 ceph-mon[98694]: pgmap v1234: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-09T23:52:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:45 smithi098 ceph-mon[103124]: pgmap v1234: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-09T23:52:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:45 smithi170 ceph-mon[102832]: pgmap v1234: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-09T23:52:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 1998.447663 secs 2023-09-09T23:52:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:47 smithi082 ceph-mon[98694]: pgmap v1235: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 46 KiB/s wr, 16 op/s 2023-09-09T23:52:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 1998.447663 secs 2023-09-09T23:52:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:47 smithi098 ceph-mon[103124]: pgmap v1235: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 46 KiB/s wr, 16 op/s 2023-09-09T23:52:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 1998.447663 secs 2023-09-09T23:52:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:47 smithi170 ceph-mon[102832]: pgmap v1235: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 46 KiB/s wr, 16 op/s 2023-09-09T23:52:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2000.044504 secs 2023-09-09T23:52:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2000.044504 secs 2023-09-09T23:52:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2000.044504 secs 2023-09-09T23:52:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2000.773881 secs 2023-09-09T23:52:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:49 smithi082 ceph-mon[98694]: pgmap v1236: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 9 op/s 2023-09-09T23:52:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2000.773881 secs 2023-09-09T23:52:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:49 smithi098 ceph-mon[103124]: pgmap v1236: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 9 op/s 2023-09-09T23:52:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2000.773881 secs 2023-09-09T23:52:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:49 smithi170 ceph-mon[102832]: pgmap v1236: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 9 op/s 2023-09-09T23:52:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:51 smithi098 ceph-mon[103124]: pgmap v1237: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 40 KiB/s wr, 14 op/s 2023-09-09T23:52:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:51 smithi170 ceph-mon[102832]: pgmap v1237: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 40 KiB/s wr, 14 op/s 2023-09-09T23:52:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:51 smithi082 ceph-mon[98694]: pgmap v1237: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 40 KiB/s wr, 14 op/s 2023-09-09T23:52:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2003.447729 secs 2023-09-09T23:52:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2003.447729 secs 2023-09-09T23:52:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2003.447729 secs 2023-09-09T23:52:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2005.044601 secs 2023-09-09T23:52:54.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:53 smithi098 ceph-mon[103124]: pgmap v1238: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 22 KiB/s wr, 9 op/s 2023-09-09T23:52:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2005.044601 secs 2023-09-09T23:52:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:53 smithi170 ceph-mon[102832]: pgmap v1238: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 22 KiB/s wr, 9 op/s 2023-09-09T23:52:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2005.044601 secs 2023-09-09T23:52:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:53 smithi082 ceph-mon[98694]: pgmap v1238: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 22 KiB/s wr, 9 op/s 2023-09-09T23:52:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2005.773950 secs 2023-09-09T23:52:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2005.773950 secs 2023-09-09T23:52:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2005.773950 secs 2023-09-09T23:52:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:55 smithi098 ceph-mon[103124]: pgmap v1239: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 22 KiB/s wr, 9 op/s 2023-09-09T23:52:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:55 smithi170 ceph-mon[102832]: pgmap v1239: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 22 KiB/s wr, 9 op/s 2023-09-09T23:52:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:55 smithi082 ceph-mon[98694]: pgmap v1239: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 22 KiB/s wr, 9 op/s 2023-09-09T23:52:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2008.447828 secs 2023-09-09T23:52:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:57 smithi098 ceph-mon[103124]: pgmap v1240: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 29 KiB/s wr, 14 op/s 2023-09-09T23:52:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2008.447828 secs 2023-09-09T23:52:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:57 smithi170 ceph-mon[102832]: pgmap v1240: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 29 KiB/s wr, 14 op/s 2023-09-09T23:52:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2008.447828 secs 2023-09-09T23:52:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:57 smithi082 ceph-mon[98694]: pgmap v1240: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 29 KiB/s wr, 14 op/s 2023-09-09T23:52:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2010.044657 secs 2023-09-09T23:52:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2010.044657 secs 2023-09-09T23:52:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2010.044657 secs 2023-09-09T23:53:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2010.774015 secs 2023-09-09T23:53:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:52:59 smithi098 ceph-mon[103124]: pgmap v1241: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 9 op/s 2023-09-09T23:53:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2010.774015 secs 2023-09-09T23:53:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:52:59 smithi170 ceph-mon[102832]: pgmap v1241: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 9 op/s 2023-09-09T23:53:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2010.774015 secs 2023-09-09T23:53:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:52:59 smithi082 ceph-mon[98694]: pgmap v1241: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 9 op/s 2023-09-09T23:53:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:01 smithi098 ceph-mon[103124]: pgmap v1242: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-09T23:53:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:53:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:01 smithi170 ceph-mon[102832]: pgmap v1242: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-09T23:53:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:53:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:01 smithi082 ceph-mon[98694]: pgmap v1242: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-09T23:53:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:53:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2013.447934 secs 2023-09-09T23:53:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:02 smithi098 ceph-mon[103124]: osdmap e151: 12 total, 12 up, 12 in 2023-09-09T23:53:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2013.447934 secs 2023-09-09T23:53:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:02 smithi170 ceph-mon[102832]: osdmap e151: 12 total, 12 up, 12 in 2023-09-09T23:53:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2013.447934 secs 2023-09-09T23:53:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:02 smithi082 ceph-mon[98694]: osdmap e151: 12 total, 12 up, 12 in 2023-09-09T23:53:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2015.044715 secs 2023-09-09T23:53:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:03 smithi098 ceph-mon[103124]: pgmap v1244: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-09T23:53:04.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:53:04.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:53:04.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:53:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2015.044715 secs 2023-09-09T23:53:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:03 smithi170 ceph-mon[102832]: pgmap v1244: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-09T23:53:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:53:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:53:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:53:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2015.044715 secs 2023-09-09T23:53:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:03 smithi082 ceph-mon[98694]: pgmap v1244: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-09T23:53:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:53:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:53:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:53:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2015.774084 secs 2023-09-09T23:53:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2015.774084 secs 2023-09-09T23:53:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2015.774084 secs 2023-09-09T23:53:06.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:05 smithi098 ceph-mon[103124]: pgmap v1245: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-09T23:53:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:05 smithi170 ceph-mon[102832]: pgmap v1245: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-09T23:53:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:05 smithi082 ceph-mon[98694]: pgmap v1245: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-09T23:53:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2018.447997 secs 2023-09-09T23:53:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:08 smithi098 ceph-mon[103124]: pgmap v1246: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-09T23:53:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2018.447997 secs 2023-09-09T23:53:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:08 smithi170 ceph-mon[102832]: pgmap v1246: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-09T23:53:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2018.447997 secs 2023-09-09T23:53:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:08 smithi082 ceph-mon[98694]: pgmap v1246: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-09T23:53:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2020.044774 secs 2023-09-09T23:53:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:09 smithi170 ceph-mon[102832]: osdmap e152: 12 total, 12 up, 12 in 2023-09-09T23:53:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2020.044774 secs 2023-09-09T23:53:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:09 smithi082 ceph-mon[98694]: osdmap e152: 12 total, 12 up, 12 in 2023-09-09T23:53:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2020.044774 secs 2023-09-09T23:53:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:09 smithi098 ceph-mon[103124]: osdmap e152: 12 total, 12 up, 12 in 2023-09-09T23:53:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2020.774148 secs 2023-09-09T23:53:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:10 smithi170 ceph-mon[102832]: pgmap v1247: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-09T23:53:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2020.774148 secs 2023-09-09T23:53:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:10 smithi082 ceph-mon[98694]: pgmap v1247: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-09T23:53:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2020.774148 secs 2023-09-09T23:53:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:10 smithi098 ceph-mon[103124]: pgmap v1247: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-09T23:53:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:12 smithi170 ceph-mon[102832]: pgmap v1249: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 12 op/s 2023-09-09T23:53:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:12 smithi082 ceph-mon[98694]: pgmap v1249: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 12 op/s 2023-09-09T23:53:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:12 smithi098 ceph-mon[103124]: pgmap v1249: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 12 op/s 2023-09-09T23:53:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2023.448056 secs 2023-09-09T23:53:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2023.448056 secs 2023-09-09T23:53:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2023.448056 secs 2023-09-09T23:53:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2025.044832 secs 2023-09-09T23:53:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:14 smithi170 ceph-mon[102832]: pgmap v1250: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-09T23:53:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2025.044832 secs 2023-09-09T23:53:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:14 smithi082 ceph-mon[98694]: pgmap v1250: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-09T23:53:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2025.044832 secs 2023-09-09T23:53:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:14 smithi098 ceph-mon[103124]: pgmap v1250: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-09T23:53:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2025.774213 secs 2023-09-09T23:53:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2025.774213 secs 2023-09-09T23:53:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2025.774213 secs 2023-09-09T23:53:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:16 smithi170 ceph-mon[102832]: pgmap v1251: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-09T23:53:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:16 smithi082 ceph-mon[98694]: pgmap v1251: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-09T23:53:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:16 smithi098 ceph-mon[103124]: pgmap v1251: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-09T23:53:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:18 smithi170 ceph-mon[102832]: pgmap v1252: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-09T23:53:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2030.044891 secs 2023-09-09T23:53:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:18 smithi082 ceph-mon[98694]: pgmap v1252: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-09T23:53:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2030.044891 secs 2023-09-09T23:53:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:18 smithi098 ceph-mon[103124]: pgmap v1252: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-09T23:53:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2030.044891 secs 2023-09-09T23:53:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:53:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:53:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:53:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2030.774279 secs 2023-09-09T23:53:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:20 smithi170 ceph-mon[102832]: pgmap v1253: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-09T23:53:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:53:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:53:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:53:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2030.774279 secs 2023-09-09T23:53:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:20 smithi082 ceph-mon[98694]: pgmap v1253: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-09T23:53:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:53:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:53:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:53:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2030.774279 secs 2023-09-09T23:53:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:20 smithi098 ceph-mon[103124]: pgmap v1253: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-09T23:53:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:53:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:53:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:53:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:22 smithi170 ceph-mon[102832]: pgmap v1254: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 28 KiB/s wr, 13 op/s 2023-09-09T23:53:22.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:22 smithi082 ceph-mon[98694]: pgmap v1254: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 28 KiB/s wr, 13 op/s 2023-09-09T23:53:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:22 smithi098 ceph-mon[103124]: pgmap v1254: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 28 KiB/s wr, 13 op/s 2023-09-09T23:53:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2035.044948 secs 2023-09-09T23:53:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2035.044948 secs 2023-09-09T23:53:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2035.044948 secs 2023-09-09T23:53:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:24 smithi170 ceph-mon[102832]: pgmap v1255: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-09T23:53:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:24 smithi082 ceph-mon[98694]: pgmap v1255: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-09T23:53:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:24 smithi098 ceph-mon[103124]: pgmap v1255: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-09T23:53:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2035.774343 secs 2023-09-09T23:53:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2035.774343 secs 2023-09-09T23:53:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2035.774343 secs 2023-09-09T23:53:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:26 smithi170 ceph-mon[102832]: pgmap v1256: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-09T23:53:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:26 smithi082 ceph-mon[98694]: pgmap v1256: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-09T23:53:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:26 smithi098 ceph-mon[103124]: pgmap v1256: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-09T23:53:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:28 smithi170 ceph-mon[102832]: pgmap v1257: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 14 op/s 2023-09-09T23:53:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2040.045025 secs 2023-09-09T23:53:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:28 smithi082 ceph-mon[98694]: pgmap v1257: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 14 op/s 2023-09-09T23:53:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2040.045025 secs 2023-09-09T23:53:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:28 smithi098 ceph-mon[103124]: pgmap v1257: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 14 op/s 2023-09-09T23:53:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2040.045025 secs 2023-09-09T23:53:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2040.774409 secs 2023-09-09T23:53:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:30 smithi170 ceph-mon[102832]: pgmap v1258: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-09T23:53:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2040.774409 secs 2023-09-09T23:53:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:30 smithi082 ceph-mon[98694]: pgmap v1258: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-09T23:53:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2040.774409 secs 2023-09-09T23:53:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:30 smithi098 ceph-mon[103124]: pgmap v1258: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-09T23:53:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:32 smithi170 ceph-mon[102832]: pgmap v1259: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 82 KiB/s wr, 16 op/s 2023-09-09T23:53:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:32 smithi082 ceph-mon[98694]: pgmap v1259: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 82 KiB/s wr, 16 op/s 2023-09-09T23:53:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:32 smithi098 ceph-mon[103124]: pgmap v1259: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 82 KiB/s wr, 16 op/s 2023-09-09T23:53:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2045.045111 secs 2023-09-09T23:53:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2045.045111 secs 2023-09-09T23:53:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2045.045111 secs 2023-09-09T23:53:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:34 smithi170 ceph-mon[102832]: pgmap v1260: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-09T23:53:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:34 smithi082 ceph-mon[98694]: pgmap v1260: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-09T23:53:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:34 smithi098 ceph-mon[103124]: pgmap v1260: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-09T23:53:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2045.774477 secs 2023-09-09T23:53:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2045.774477 secs 2023-09-09T23:53:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2045.774477 secs 2023-09-09T23:53:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:36 smithi170 ceph-mon[102832]: pgmap v1261: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-09T23:53:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:36 smithi082 ceph-mon[98694]: pgmap v1261: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-09T23:53:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:36 smithi098 ceph-mon[103124]: pgmap v1261: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-09T23:53:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:38 smithi170 ceph-mon[102832]: pgmap v1262: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 87 KiB/s wr, 18 op/s 2023-09-09T23:53:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2050.045165 secs 2023-09-09T23:53:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:38 smithi082 ceph-mon[98694]: pgmap v1262: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 87 KiB/s wr, 18 op/s 2023-09-09T23:53:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2050.045165 secs 2023-09-09T23:53:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:38 smithi098 ceph-mon[103124]: pgmap v1262: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 87 KiB/s wr, 18 op/s 2023-09-09T23:53:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2050.045165 secs 2023-09-09T23:53:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2050.774502 secs 2023-09-09T23:53:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:40 smithi170 ceph-mon[102832]: pgmap v1263: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 12 op/s 2023-09-09T23:53:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2050.774502 secs 2023-09-09T23:53:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:40 smithi082 ceph-mon[98694]: pgmap v1263: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 12 op/s 2023-09-09T23:53:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2050.774502 secs 2023-09-09T23:53:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:40 smithi098 ceph-mon[103124]: pgmap v1263: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 12 op/s 2023-09-09T23:53:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:42 smithi170 ceph-mon[102832]: pgmap v1264: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 111 KiB/s wr, 18 op/s 2023-09-09T23:53:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:42 smithi082 ceph-mon[98694]: pgmap v1264: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 111 KiB/s wr, 18 op/s 2023-09-09T23:53:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:42 smithi098 ceph-mon[103124]: pgmap v1264: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 111 KiB/s wr, 18 op/s 2023-09-09T23:53:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2055.045271 secs 2023-09-09T23:53:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2055.045271 secs 2023-09-09T23:53:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2055.045271 secs 2023-09-09T23:53:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:44 smithi170 ceph-mon[102832]: pgmap v1265: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-09T23:53:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:44 smithi082 ceph-mon[98694]: pgmap v1265: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-09T23:53:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:44 smithi098 ceph-mon[103124]: pgmap v1265: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-09T23:53:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2055.774568 secs 2023-09-09T23:53:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2055.774568 secs 2023-09-09T23:53:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2055.774568 secs 2023-09-09T23:53:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:46 smithi170 ceph-mon[102832]: pgmap v1266: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-09T23:53:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:46 smithi082 ceph-mon[98694]: pgmap v1266: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-09T23:53:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:46 smithi098 ceph-mon[103124]: pgmap v1266: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-09T23:53:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2058.448679 secs 2023-09-09T23:53:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:48 smithi170 ceph-mon[102832]: pgmap v1267: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 102 KiB/s wr, 16 op/s 2023-09-09T23:53:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2060.045365 secs 2023-09-09T23:53:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2058.448679 secs 2023-09-09T23:53:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:48 smithi082 ceph-mon[98694]: pgmap v1267: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 102 KiB/s wr, 16 op/s 2023-09-09T23:53:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2060.045365 secs 2023-09-09T23:53:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2058.448679 secs 2023-09-09T23:53:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:48 smithi098 ceph-mon[103124]: pgmap v1267: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 102 KiB/s wr, 16 op/s 2023-09-09T23:53:48.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2060.045365 secs 2023-09-09T23:53:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2060.774634 secs 2023-09-09T23:53:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:50 smithi170 ceph-mon[102832]: pgmap v1268: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-09T23:53:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2060.774634 secs 2023-09-09T23:53:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:50 smithi082 ceph-mon[98694]: pgmap v1268: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-09T23:53:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2060.774634 secs 2023-09-09T23:53:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:50 smithi098 ceph-mon[103124]: pgmap v1268: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-09T23:53:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:53:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:53:51.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:53:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:53:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:53:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:53:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:53:51.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:53:51.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:53:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:52 smithi170 ceph-mon[102832]: pgmap v1269: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 110 KiB/s wr, 16 op/s 2023-09-09T23:53:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2063.448805 secs 2023-09-09T23:53:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:52 smithi082 ceph-mon[98694]: pgmap v1269: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 110 KiB/s wr, 16 op/s 2023-09-09T23:53:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2063.448805 secs 2023-09-09T23:53:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:52 smithi098 ceph-mon[103124]: pgmap v1269: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 110 KiB/s wr, 16 op/s 2023-09-09T23:53:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2063.448805 secs 2023-09-09T23:53:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2065.045436 secs 2023-09-09T23:53:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2065.045436 secs 2023-09-09T23:53:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2065.045436 secs 2023-09-09T23:53:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:54 smithi170 ceph-mon[102832]: pgmap v1270: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 58 KiB/s wr, 10 op/s 2023-09-09T23:53:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:54 smithi082 ceph-mon[98694]: pgmap v1270: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 58 KiB/s wr, 10 op/s 2023-09-09T23:53:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:54 smithi098 ceph-mon[103124]: pgmap v1270: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 58 KiB/s wr, 10 op/s 2023-09-09T23:53:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2065.774699 secs 2023-09-09T23:53:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2065.774699 secs 2023-09-09T23:53:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2065.774699 secs 2023-09-09T23:53:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:56 smithi170 ceph-mon[102832]: pgmap v1271: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 58 KiB/s wr, 10 op/s 2023-09-09T23:53:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:56 smithi082 ceph-mon[98694]: pgmap v1271: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 58 KiB/s wr, 10 op/s 2023-09-09T23:53:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:56 smithi098 ceph-mon[103124]: pgmap v1271: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 58 KiB/s wr, 10 op/s 2023-09-09T23:53:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2068.448891 secs 2023-09-09T23:53:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2068.448891 secs 2023-09-09T23:53:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2068.448891 secs 2023-09-09T23:53:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:58 smithi170 ceph-mon[102832]: pgmap v1272: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 69 KiB/s wr, 16 op/s 2023-09-09T23:53:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2070.045495 secs 2023-09-09T23:53:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:58 smithi082 ceph-mon[98694]: pgmap v1272: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 69 KiB/s wr, 16 op/s 2023-09-09T23:53:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2070.045495 secs 2023-09-09T23:53:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:58 smithi098 ceph-mon[103124]: pgmap v1272: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 69 KiB/s wr, 16 op/s 2023-09-09T23:53:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2070.045495 secs 2023-09-09T23:54:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2070.774763 secs 2023-09-09T23:54:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:53:59 smithi082 ceph-mon[98694]: pgmap v1273: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 39 KiB/s wr, 10 op/s 2023-09-09T23:54:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2070.774763 secs 2023-09-09T23:54:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:53:59 smithi098 ceph-mon[103124]: pgmap v1273: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 39 KiB/s wr, 10 op/s 2023-09-09T23:54:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2070.774763 secs 2023-09-09T23:54:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:53:59 smithi170 ceph-mon[102832]: pgmap v1273: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 39 KiB/s wr, 10 op/s 2023-09-09T23:54:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:01 smithi082 ceph-mon[98694]: pgmap v1274: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 92 KiB/s wr, 29 op/s 2023-09-09T23:54:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:01 smithi098 ceph-mon[103124]: pgmap v1274: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 92 KiB/s wr, 29 op/s 2023-09-09T23:54:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:01 smithi170 ceph-mon[102832]: pgmap v1274: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 92 KiB/s wr, 29 op/s 2023-09-09T23:54:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2073.448949 secs 2023-09-09T23:54:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2073.448949 secs 2023-09-09T23:54:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2073.448949 secs 2023-09-09T23:54:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2075.045579 secs 2023-09-09T23:54:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:03 smithi098 ceph-mon[103124]: pgmap v1275: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 64 KiB/s wr, 23 op/s 2023-09-09T23:54:04.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:54:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2075.045579 secs 2023-09-09T23:54:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:03 smithi170 ceph-mon[102832]: pgmap v1275: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 64 KiB/s wr, 23 op/s 2023-09-09T23:54:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:54:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2075.045579 secs 2023-09-09T23:54:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:03 smithi082 ceph-mon[98694]: pgmap v1275: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 64 KiB/s wr, 23 op/s 2023-09-09T23:54:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:54:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2075.774827 secs 2023-09-09T23:54:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:54:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:54:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:54:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2075.774827 secs 2023-09-09T23:54:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:54:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:54:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:54:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2075.774827 secs 2023-09-09T23:54:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:54:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:54:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:54:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:06 smithi082 ceph-mon[98694]: pgmap v1276: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 64 KiB/s wr, 23 op/s 2023-09-09T23:54:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:06 smithi098 ceph-mon[103124]: pgmap v1276: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 64 KiB/s wr, 23 op/s 2023-09-09T23:54:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:06 smithi170 ceph-mon[102832]: pgmap v1276: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 64 KiB/s wr, 23 op/s 2023-09-09T23:54:07.731 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2078.449033 secs 2023-09-09T23:54:07.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2078.449033 secs 2023-09-09T23:54:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2078.449033 secs 2023-09-09T23:54:08.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:08 smithi098 ceph-mon[103124]: pgmap v1277: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 65 KiB/s wr, 28 op/s 2023-09-09T23:54:08.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:08 smithi098 ceph-mon[103124]: osdmap e153: 12 total, 12 up, 12 in 2023-09-09T23:54:08.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2080.045637 secs 2023-09-09T23:54:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:08 smithi170 ceph-mon[102832]: pgmap v1277: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 65 KiB/s wr, 28 op/s 2023-09-09T23:54:08.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:08 smithi170 ceph-mon[102832]: osdmap e153: 12 total, 12 up, 12 in 2023-09-09T23:54:08.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2080.045637 secs 2023-09-09T23:54:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:08 smithi082 ceph-mon[98694]: pgmap v1277: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 65 KiB/s wr, 28 op/s 2023-09-09T23:54:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:08 smithi082 ceph-mon[98694]: osdmap e153: 12 total, 12 up, 12 in 2023-09-09T23:54:08.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2080.045637 secs 2023-09-09T23:54:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2080.774895 secs 2023-09-09T23:54:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:09 smithi082 ceph-mon[98694]: pgmap v1279: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 65 KiB/s wr, 28 op/s 2023-09-09T23:54:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2080.774895 secs 2023-09-09T23:54:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:09 smithi098 ceph-mon[103124]: pgmap v1279: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 65 KiB/s wr, 28 op/s 2023-09-09T23:54:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2080.774895 secs 2023-09-09T23:54:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:09 smithi170 ceph-mon[102832]: pgmap v1279: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 65 KiB/s wr, 28 op/s 2023-09-09T23:54:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:11 smithi098 ceph-mon[103124]: pgmap v1280: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 17 KiB/s wr, 11 op/s 2023-09-09T23:54:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:11 smithi170 ceph-mon[102832]: pgmap v1280: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 17 KiB/s wr, 11 op/s 2023-09-09T23:54:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:11 smithi082 ceph-mon[98694]: pgmap v1280: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 17 KiB/s wr, 11 op/s 2023-09-09T23:54:13.251 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2083.449144 secs 2023-09-09T23:54:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2083.449144 secs 2023-09-09T23:54:13.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2083.449144 secs 2023-09-09T23:54:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2085.045698 secs 2023-09-09T23:54:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:13 smithi098 ceph-mon[103124]: pgmap v1281: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 17 KiB/s wr, 11 op/s 2023-09-09T23:54:14.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:13 smithi098 ceph-mon[103124]: osdmap e154: 12 total, 12 up, 12 in 2023-09-09T23:54:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2085.045698 secs 2023-09-09T23:54:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:13 smithi170 ceph-mon[102832]: pgmap v1281: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 17 KiB/s wr, 11 op/s 2023-09-09T23:54:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:13 smithi170 ceph-mon[102832]: osdmap e154: 12 total, 12 up, 12 in 2023-09-09T23:54:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2085.045698 secs 2023-09-09T23:54:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:13 smithi082 ceph-mon[98694]: pgmap v1281: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 17 KiB/s wr, 11 op/s 2023-09-09T23:54:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:13 smithi082 ceph-mon[98694]: osdmap e154: 12 total, 12 up, 12 in 2023-09-09T23:54:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2085.774985 secs 2023-09-09T23:54:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2085.774985 secs 2023-09-09T23:54:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2085.774985 secs 2023-09-09T23:54:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:15 smithi098 ceph-mon[103124]: pgmap v1283: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 20 KiB/s wr, 7 op/s 2023-09-09T23:54:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:15 smithi170 ceph-mon[102832]: pgmap v1283: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 20 KiB/s wr, 7 op/s 2023-09-09T23:54:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:15 smithi082 ceph-mon[98694]: pgmap v1283: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 20 KiB/s wr, 7 op/s 2023-09-09T23:54:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:17 smithi082 ceph-mon[98694]: pgmap v1284: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 29 KiB/s wr, 12 op/s 2023-09-09T23:54:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:17 smithi098 ceph-mon[103124]: pgmap v1284: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 29 KiB/s wr, 12 op/s 2023-09-09T23:54:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:17 smithi170 ceph-mon[102832]: pgmap v1284: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 29 KiB/s wr, 12 op/s 2023-09-09T23:54:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2090.045746 secs 2023-09-09T23:54:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:18 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:54:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2090.045746 secs 2023-09-09T23:54:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:18 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:54:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2090.045746 secs 2023-09-09T23:54:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:18 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:54:20.229 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2090.775049 secs 2023-09-09T23:54:20.230 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:19 smithi082 ceph-mon[98694]: pgmap v1285: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-09T23:54:20.230 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:19 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:54:20.230 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:19 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:54:20.230 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:19 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:54:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2090.775049 secs 2023-09-09T23:54:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:19 smithi098 ceph-mon[103124]: pgmap v1285: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-09T23:54:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:19 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:54:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:19 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:54:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:19 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:54:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2090.775049 secs 2023-09-09T23:54:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:19 smithi170 ceph-mon[102832]: pgmap v1285: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-09T23:54:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:19 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:54:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:19 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:54:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:19 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:54:22.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:21 smithi098 ceph-mon[103124]: pgmap v1286: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-09T23:54:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:21 smithi170 ceph-mon[102832]: pgmap v1286: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-09T23:54:22.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:21 smithi082 ceph-mon[98694]: pgmap v1286: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-09T23:54:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2095.045809 secs 2023-09-09T23:54:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:23 smithi098 ceph-mon[103124]: pgmap v1287: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-09T23:54:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2095.045809 secs 2023-09-09T23:54:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:23 smithi170 ceph-mon[102832]: pgmap v1287: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-09T23:54:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2095.045809 secs 2023-09-09T23:54:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:23 smithi082 ceph-mon[98694]: pgmap v1287: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-09T23:54:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2095.775118 secs 2023-09-09T23:54:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2095.775118 secs 2023-09-09T23:54:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2095.775118 secs 2023-09-09T23:54:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:25 smithi098 ceph-mon[103124]: pgmap v1288: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-09T23:54:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:25 smithi170 ceph-mon[102832]: pgmap v1288: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-09T23:54:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:25 smithi082 ceph-mon[98694]: pgmap v1288: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-09T23:54:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:27 smithi098 ceph-mon[103124]: pgmap v1289: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 86 KiB/s wr, 17 op/s 2023-09-09T23:54:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:27 smithi170 ceph-mon[102832]: pgmap v1289: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 86 KiB/s wr, 17 op/s 2023-09-09T23:54:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:27 smithi082 ceph-mon[98694]: pgmap v1289: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 86 KiB/s wr, 17 op/s 2023-09-09T23:54:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2100.045897 secs 2023-09-09T23:54:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2100.045897 secs 2023-09-09T23:54:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2100.045897 secs 2023-09-09T23:54:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2100.775183 secs 2023-09-09T23:54:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:29 smithi098 ceph-mon[103124]: pgmap v1290: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 77 KiB/s wr, 12 op/s 2023-09-09T23:54:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2100.775183 secs 2023-09-09T23:54:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:29 smithi170 ceph-mon[102832]: pgmap v1290: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 77 KiB/s wr, 12 op/s 2023-09-09T23:54:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2100.775183 secs 2023-09-09T23:54:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:29 smithi082 ceph-mon[98694]: pgmap v1290: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 77 KiB/s wr, 12 op/s 2023-09-09T23:54:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:32 smithi098 ceph-mon[103124]: pgmap v1291: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 118 KiB/s wr, 17 op/s 2023-09-09T23:54:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:32 smithi170 ceph-mon[102832]: pgmap v1291: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 118 KiB/s wr, 17 op/s 2023-09-09T23:54:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:32 smithi082 ceph-mon[98694]: pgmap v1291: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 118 KiB/s wr, 17 op/s 2023-09-09T23:54:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2105.045960 secs 2023-09-09T23:54:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:34 smithi170 ceph-mon[102832]: pgmap v1292: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-09T23:54:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2105.045960 secs 2023-09-09T23:54:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:34 smithi082 ceph-mon[98694]: pgmap v1292: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-09T23:54:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2105.045960 secs 2023-09-09T23:54:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:34 smithi098 ceph-mon[103124]: pgmap v1292: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-09T23:54:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2105.775249 secs 2023-09-09T23:54:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2105.775249 secs 2023-09-09T23:54:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2105.775249 secs 2023-09-09T23:54:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:36 smithi170 ceph-mon[102832]: pgmap v1293: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-09T23:54:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:36 smithi082 ceph-mon[98694]: pgmap v1293: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-09T23:54:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:36 smithi098 ceph-mon[103124]: pgmap v1293: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-09T23:54:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:38 smithi170 ceph-mon[102832]: pgmap v1294: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 106 KiB/s wr, 17 op/s 2023-09-09T23:54:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:38 smithi082 ceph-mon[98694]: pgmap v1294: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 106 KiB/s wr, 17 op/s 2023-09-09T23:54:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:38 smithi098 ceph-mon[103124]: pgmap v1294: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 106 KiB/s wr, 17 op/s 2023-09-09T23:54:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2110.046042 secs 2023-09-09T23:54:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2110.046042 secs 2023-09-09T23:54:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2110.046042 secs 2023-09-09T23:54:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2110.775318 secs 2023-09-09T23:54:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:40 smithi170 ceph-mon[102832]: pgmap v1295: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-09T23:54:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2110.775318 secs 2023-09-09T23:54:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:40 smithi082 ceph-mon[98694]: pgmap v1295: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-09T23:54:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2110.775318 secs 2023-09-09T23:54:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:40 smithi098 ceph-mon[103124]: pgmap v1295: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-09T23:54:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:42 smithi170 ceph-mon[102832]: pgmap v1296: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 106 KiB/s wr, 16 op/s 2023-09-09T23:54:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:42 smithi082 ceph-mon[98694]: pgmap v1296: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 106 KiB/s wr, 16 op/s 2023-09-09T23:54:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:42 smithi098 ceph-mon[103124]: pgmap v1296: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 106 KiB/s wr, 16 op/s 2023-09-09T23:54:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2113.449568 secs 2023-09-09T23:54:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2115.046099 secs 2023-09-09T23:54:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2113.449568 secs 2023-09-09T23:54:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2115.046099 secs 2023-09-09T23:54:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2113.449568 secs 2023-09-09T23:54:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2115.046099 secs 2023-09-09T23:54:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:44 smithi170 ceph-mon[102832]: pgmap v1297: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-09T23:54:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:54:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:54:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:54:44.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:44 smithi082 ceph-mon[98694]: pgmap v1297: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-09T23:54:44.462 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:54:44.462 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:54:44.462 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:54:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:44 smithi098 ceph-mon[103124]: pgmap v1297: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-09T23:54:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:54:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:54:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:54:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2115.775380 secs 2023-09-09T23:54:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2115.775380 secs 2023-09-09T23:54:45.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2115.775380 secs 2023-09-09T23:54:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:46 smithi170 ceph-mon[102832]: pgmap v1298: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-09T23:54:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:46 smithi082 ceph-mon[98694]: pgmap v1298: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-09T23:54:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:46 smithi098 ceph-mon[103124]: pgmap v1298: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-09T23:54:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2118.449633 secs 2023-09-09T23:54:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:48 smithi170 ceph-mon[102832]: pgmap v1299: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 100 KiB/s wr, 16 op/s 2023-09-09T23:54:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2120.046159 secs 2023-09-09T23:54:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2118.449633 secs 2023-09-09T23:54:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:48 smithi082 ceph-mon[98694]: pgmap v1299: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 100 KiB/s wr, 16 op/s 2023-09-09T23:54:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2120.046159 secs 2023-09-09T23:54:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2118.449633 secs 2023-09-09T23:54:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:48 smithi098 ceph-mon[103124]: pgmap v1299: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 100 KiB/s wr, 16 op/s 2023-09-09T23:54:48.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2120.046159 secs 2023-09-09T23:54:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2120.775449 secs 2023-09-09T23:54:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:50 smithi170 ceph-mon[102832]: pgmap v1300: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 52 KiB/s wr, 10 op/s 2023-09-09T23:54:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2120.775449 secs 2023-09-09T23:54:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:50 smithi082 ceph-mon[98694]: pgmap v1300: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 52 KiB/s wr, 10 op/s 2023-09-09T23:54:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2120.775449 secs 2023-09-09T23:54:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:50 smithi098 ceph-mon[103124]: pgmap v1300: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 52 KiB/s wr, 10 op/s 2023-09-09T23:54:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:52 smithi170 ceph-mon[102832]: pgmap v1301: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 66 KiB/s wr, 15 op/s 2023-09-09T23:54:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:52 smithi082 ceph-mon[98694]: pgmap v1301: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 66 KiB/s wr, 15 op/s 2023-09-09T23:54:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:52 smithi098 ceph-mon[103124]: pgmap v1301: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 66 KiB/s wr, 15 op/s 2023-09-09T23:54:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2123.449691 secs 2023-09-09T23:54:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2125.046240 secs 2023-09-09T23:54:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2123.449691 secs 2023-09-09T23:54:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2125.046240 secs 2023-09-09T23:54:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2123.449691 secs 2023-09-09T23:54:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2125.046240 secs 2023-09-09T23:54:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:54 smithi170 ceph-mon[102832]: pgmap v1302: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 9 op/s 2023-09-09T23:54:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:54 smithi082 ceph-mon[98694]: pgmap v1302: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 9 op/s 2023-09-09T23:54:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:54 smithi098 ceph-mon[103124]: pgmap v1302: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 9 op/s 2023-09-09T23:54:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2125.775489 secs 2023-09-09T23:54:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2125.775489 secs 2023-09-09T23:54:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2125.775489 secs 2023-09-09T23:54:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:56 smithi170 ceph-mon[102832]: pgmap v1303: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 9 op/s 2023-09-09T23:54:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:56 smithi082 ceph-mon[98694]: pgmap v1303: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 9 op/s 2023-09-09T23:54:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:56 smithi098 ceph-mon[103124]: pgmap v1303: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 9 op/s 2023-09-09T23:54:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2128.449752 secs 2023-09-09T23:54:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:58 smithi170 ceph-mon[102832]: pgmap v1304: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 14 op/s 2023-09-09T23:54:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:54:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2130.046328 secs 2023-09-09T23:54:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2128.449752 secs 2023-09-09T23:54:58.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:58 smithi082 ceph-mon[98694]: pgmap v1304: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 14 op/s 2023-09-09T23:54:58.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:54:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2130.046328 secs 2023-09-09T23:54:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2128.449752 secs 2023-09-09T23:54:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:58 smithi098 ceph-mon[103124]: pgmap v1304: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 14 op/s 2023-09-09T23:54:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:54:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2130.046328 secs 2023-09-09T23:55:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2130.775557 secs 2023-09-09T23:55:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:00 smithi170 ceph-mon[102832]: pgmap v1305: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 25 KiB/s wr, 10 op/s 2023-09-09T23:55:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2130.775557 secs 2023-09-09T23:55:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:00 smithi082 ceph-mon[98694]: pgmap v1305: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 25 KiB/s wr, 10 op/s 2023-09-09T23:55:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2130.775557 secs 2023-09-09T23:55:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:00 smithi098 ceph-mon[103124]: pgmap v1305: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 25 KiB/s wr, 10 op/s 2023-09-09T23:55:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:02 smithi170 ceph-mon[102832]: pgmap v1306: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 60 KiB/s wr, 28 op/s 2023-09-09T23:55:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:02 smithi082 ceph-mon[98694]: pgmap v1306: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 60 KiB/s wr, 28 op/s 2023-09-09T23:55:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:02 smithi098 ceph-mon[103124]: pgmap v1306: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 60 KiB/s wr, 28 op/s 2023-09-09T23:55:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2133.449789 secs 2023-09-09T23:55:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2135.046387 secs 2023-09-09T23:55:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2133.449789 secs 2023-09-09T23:55:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2135.046387 secs 2023-09-09T23:55:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2133.449789 secs 2023-09-09T23:55:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2135.046387 secs 2023-09-09T23:55:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:04 smithi170 ceph-mon[102832]: pgmap v1307: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 46 KiB/s wr, 23 op/s 2023-09-09T23:55:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:04 smithi082 ceph-mon[98694]: pgmap v1307: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 46 KiB/s wr, 23 op/s 2023-09-09T23:55:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:04 smithi098 ceph-mon[103124]: pgmap v1307: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 46 KiB/s wr, 23 op/s 2023-09-09T23:55:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2135.775622 secs 2023-09-09T23:55:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:55:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2135.775622 secs 2023-09-09T23:55:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:55:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2135.775622 secs 2023-09-09T23:55:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:55:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:06 smithi170 ceph-mon[102832]: pgmap v1308: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 46 KiB/s wr, 23 op/s 2023-09-09T23:55:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:06 smithi082 ceph-mon[98694]: pgmap v1308: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 46 KiB/s wr, 23 op/s 2023-09-09T23:55:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:06 smithi098 ceph-mon[103124]: pgmap v1308: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 46 KiB/s wr, 23 op/s 2023-09-09T23:55:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2138.449889 secs 2023-09-09T23:55:08.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:08 smithi170 ceph-mon[102832]: pgmap v1309: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-09-09T23:55:08.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:08 smithi170 ceph-mon[102832]: osdmap e155: 12 total, 12 up, 12 in 2023-09-09T23:55:08.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:55:08.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:55:08.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:55:08.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:55:08.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:08.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:08.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:08.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:08.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:08.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:08.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:08.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:08.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:08 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-09T23:55:08.433 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:08 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-09T23:55:08.433 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:08 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-09T23:55:08.433 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:08 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-09T23:55:08.434 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2140.046446 secs 2023-09-09T23:55:08.434 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:55:08.434 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:55:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2138.449889 secs 2023-09-09T23:55:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:08 smithi082 ceph-mon[98694]: pgmap v1309: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-09-09T23:55:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:08 smithi082 ceph-mon[98694]: osdmap e155: 12 total, 12 up, 12 in 2023-09-09T23:55:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:55:08.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:55:08.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:55:08.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:55:08.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:08.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:08.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:08.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:08.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:08.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:08.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:08.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:08.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:08 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-09T23:55:08.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:08 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-09T23:55:08.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:08 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-09T23:55:08.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:08 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-09T23:55:08.462 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2140.046446 secs 2023-09-09T23:55:08.462 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:55:08.462 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:55:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2138.449889 secs 2023-09-09T23:55:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:08 smithi098 ceph-mon[103124]: pgmap v1309: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-09-09T23:55:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:08 smithi098 ceph-mon[103124]: osdmap e155: 12 total, 12 up, 12 in 2023-09-09T23:55:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:55:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:55:08.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:55:08.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:55:08.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:08.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:08.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:08.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:08.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:08.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:08.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:08.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:08.509 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:08 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-09T23:55:08.509 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:08 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-09T23:55:08.509 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:08 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-09T23:55:08.509 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:08 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-09T23:55:08.509 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2140.046446 secs 2023-09-09T23:55:08.510 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:55:08.510 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:55:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:09 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-09T23:55:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:09 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-09T23:55:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:55:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:55:09.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:55:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:09.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:09 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-09T23:55:09.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:09 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-09T23:55:09.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:55:09.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:55:09.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:55:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-09T23:55:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:09 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-09T23:55:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:09 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-09T23:55:09.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:55:09.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:55:09.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:55:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2140.775692 secs 2023-09-09T23:55:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:10 smithi170 ceph-mon[102832]: pgmap v1311: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-09-09T23:55:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2140.775692 secs 2023-09-09T23:55:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:10 smithi082 ceph-mon[98694]: pgmap v1311: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-09-09T23:55:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2140.775692 secs 2023-09-09T23:55:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:10 smithi098 ceph-mon[103124]: pgmap v1311: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-09-09T23:55:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:12 smithi170 ceph-mon[102832]: pgmap v1312: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-09T23:55:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:12 smithi082 ceph-mon[98694]: pgmap v1312: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-09T23:55:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:12 smithi098 ceph-mon[103124]: pgmap v1312: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-09T23:55:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2143.449954 secs 2023-09-09T23:55:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2145.046496 secs 2023-09-09T23:55:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2143.449954 secs 2023-09-09T23:55:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2145.046496 secs 2023-09-09T23:55:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2143.449954 secs 2023-09-09T23:55:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2145.046496 secs 2023-09-09T23:55:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:14 smithi170 ceph-mon[102832]: pgmap v1313: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-09T23:55:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:14 smithi170 ceph-mon[102832]: osdmap e156: 12 total, 12 up, 12 in 2023-09-09T23:55:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:14 smithi082 ceph-mon[98694]: pgmap v1313: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-09T23:55:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:14 smithi082 ceph-mon[98694]: osdmap e156: 12 total, 12 up, 12 in 2023-09-09T23:55:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:14 smithi098 ceph-mon[103124]: pgmap v1313: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-09T23:55:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:14 smithi098 ceph-mon[103124]: osdmap e156: 12 total, 12 up, 12 in 2023-09-09T23:55:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2145.775757 secs 2023-09-09T23:55:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2145.775757 secs 2023-09-09T23:55:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2145.775757 secs 2023-09-09T23:55:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:16 smithi170 ceph-mon[102832]: pgmap v1315: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2023-09-09T23:55:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:16 smithi082 ceph-mon[98694]: pgmap v1315: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2023-09-09T23:55:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:16 smithi098 ceph-mon[103124]: pgmap v1315: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2023-09-09T23:55:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:18 smithi170 ceph-mon[102832]: pgmap v1316: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-09T23:55:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2150.046557 secs 2023-09-09T23:55:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:18 smithi082 ceph-mon[98694]: pgmap v1316: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-09T23:55:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2150.046557 secs 2023-09-09T23:55:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:18 smithi098 ceph-mon[103124]: pgmap v1316: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-09T23:55:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2150.046557 secs 2023-09-09T23:55:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:55:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:55:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:55:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2150.775824 secs 2023-09-09T23:55:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:20 smithi170 ceph-mon[102832]: pgmap v1317: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 11 op/s 2023-09-09T23:55:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:55:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:55:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:55:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2150.775824 secs 2023-09-09T23:55:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:20 smithi082 ceph-mon[98694]: pgmap v1317: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 11 op/s 2023-09-09T23:55:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:55:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:55:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:55:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2150.775824 secs 2023-09-09T23:55:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:20 smithi098 ceph-mon[103124]: pgmap v1317: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 11 op/s 2023-09-09T23:55:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:55:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:55:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:55:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:22 smithi170 ceph-mon[102832]: pgmap v1318: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-09T23:55:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:22 smithi082 ceph-mon[98694]: pgmap v1318: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-09T23:55:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:22 smithi098 ceph-mon[103124]: pgmap v1318: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-09T23:55:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2155.046614 secs 2023-09-09T23:55:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2155.046614 secs 2023-09-09T23:55:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2155.046614 secs 2023-09-09T23:55:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:24 smithi170 ceph-mon[102832]: pgmap v1319: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-09T23:55:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:24 smithi082 ceph-mon[98694]: pgmap v1319: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-09T23:55:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:24 smithi098 ceph-mon[103124]: pgmap v1319: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-09T23:55:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2155.775888 secs 2023-09-09T23:55:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2155.775888 secs 2023-09-09T23:55:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2155.775888 secs 2023-09-09T23:55:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:26 smithi170 ceph-mon[102832]: pgmap v1320: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 10 op/s 2023-09-09T23:55:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:26 smithi082 ceph-mon[98694]: pgmap v1320: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 10 op/s 2023-09-09T23:55:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:26 smithi098 ceph-mon[103124]: pgmap v1320: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 10 op/s 2023-09-09T23:55:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:28 smithi170 ceph-mon[102832]: pgmap v1321: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 49 KiB/s wr, 15 op/s 2023-09-09T23:55:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2160.046671 secs 2023-09-09T23:55:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:28 smithi082 ceph-mon[98694]: pgmap v1321: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 49 KiB/s wr, 15 op/s 2023-09-09T23:55:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2160.046671 secs 2023-09-09T23:55:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:28 smithi098 ceph-mon[103124]: pgmap v1321: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 49 KiB/s wr, 15 op/s 2023-09-09T23:55:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2160.046671 secs 2023-09-09T23:55:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2160.775952 secs 2023-09-09T23:55:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:30 smithi170 ceph-mon[102832]: pgmap v1322: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 34 KiB/s wr, 10 op/s 2023-09-09T23:55:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2160.775952 secs 2023-09-09T23:55:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:30 smithi082 ceph-mon[98694]: pgmap v1322: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 34 KiB/s wr, 10 op/s 2023-09-09T23:55:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2160.775952 secs 2023-09-09T23:55:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:30 smithi098 ceph-mon[103124]: pgmap v1322: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 34 KiB/s wr, 10 op/s 2023-09-09T23:55:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:32 smithi170 ceph-mon[102832]: pgmap v1323: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 80 KiB/s wr, 16 op/s 2023-09-09T23:55:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:32 smithi082 ceph-mon[98694]: pgmap v1323: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 80 KiB/s wr, 16 op/s 2023-09-09T23:55:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:32 smithi098 ceph-mon[103124]: pgmap v1323: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 80 KiB/s wr, 16 op/s 2023-09-09T23:55:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2165.046731 secs 2023-09-09T23:55:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2165.046731 secs 2023-09-09T23:55:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2165.046731 secs 2023-09-09T23:55:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:34 smithi170 ceph-mon[102832]: pgmap v1324: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-09T23:55:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:34 smithi082 ceph-mon[98694]: pgmap v1324: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-09T23:55:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:34 smithi098 ceph-mon[103124]: pgmap v1324: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-09T23:55:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2165.776016 secs 2023-09-09T23:55:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2165.776016 secs 2023-09-09T23:55:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2165.776016 secs 2023-09-09T23:55:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:36 smithi170 ceph-mon[102832]: pgmap v1325: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-09T23:55:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:36 smithi082 ceph-mon[98694]: pgmap v1325: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-09T23:55:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:36 smithi098 ceph-mon[103124]: pgmap v1325: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-09T23:55:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:38 smithi170 ceph-mon[102832]: pgmap v1326: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 98 KiB/s wr, 17 op/s 2023-09-09T23:55:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2170.046790 secs 2023-09-09T23:55:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:38 smithi082 ceph-mon[98694]: pgmap v1326: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 98 KiB/s wr, 17 op/s 2023-09-09T23:55:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2170.046790 secs 2023-09-09T23:55:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:38 smithi098 ceph-mon[103124]: pgmap v1326: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 98 KiB/s wr, 17 op/s 2023-09-09T23:55:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2170.046790 secs 2023-09-09T23:55:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2170.776097 secs 2023-09-09T23:55:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:40 smithi170 ceph-mon[102832]: pgmap v1327: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-09T23:55:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2170.776097 secs 2023-09-09T23:55:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:40 smithi082 ceph-mon[98694]: pgmap v1327: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-09T23:55:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2170.776097 secs 2023-09-09T23:55:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:40 smithi098 ceph-mon[103124]: pgmap v1327: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-09T23:55:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:42 smithi170 ceph-mon[102832]: pgmap v1328: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 110 KiB/s wr, 19 op/s 2023-09-09T23:55:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:42 smithi082 ceph-mon[98694]: pgmap v1328: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 110 KiB/s wr, 19 op/s 2023-09-09T23:55:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:42 smithi098 ceph-mon[103124]: pgmap v1328: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 110 KiB/s wr, 19 op/s 2023-09-09T23:55:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2175.046853 secs 2023-09-09T23:55:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2175.046853 secs 2023-09-09T23:55:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2175.046853 secs 2023-09-09T23:55:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:44 smithi170 ceph-mon[102832]: pgmap v1329: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-09T23:55:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:44 smithi082 ceph-mon[98694]: pgmap v1329: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-09T23:55:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:44 smithi098 ceph-mon[103124]: pgmap v1329: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-09T23:55:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2175.776164 secs 2023-09-09T23:55:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2175.776164 secs 2023-09-09T23:55:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2175.776164 secs 2023-09-09T23:55:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:46 smithi170 ceph-mon[102832]: pgmap v1330: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-09T23:55:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:46 smithi082 ceph-mon[98694]: pgmap v1330: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-09T23:55:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:46 smithi098 ceph-mon[103124]: pgmap v1330: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-09T23:55:47.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2178.450547 secs 2023-09-09T23:55:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2178.450547 secs 2023-09-09T23:55:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2178.450547 secs 2023-09-09T23:55:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:48 smithi170 ceph-mon[102832]: pgmap v1331: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 99 KiB/s wr, 18 op/s 2023-09-09T23:55:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2180.046911 secs 2023-09-09T23:55:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:48 smithi082 ceph-mon[98694]: pgmap v1331: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 99 KiB/s wr, 18 op/s 2023-09-09T23:55:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2180.046911 secs 2023-09-09T23:55:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:48 smithi098 ceph-mon[103124]: pgmap v1331: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 99 KiB/s wr, 18 op/s 2023-09-09T23:55:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2180.046911 secs 2023-09-09T23:55:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2180.776232 secs 2023-09-09T23:55:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:49 smithi098 ceph-mon[103124]: pgmap v1332: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 76 KiB/s wr, 12 op/s 2023-09-09T23:55:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2180.776232 secs 2023-09-09T23:55:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:49 smithi170 ceph-mon[102832]: pgmap v1332: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 76 KiB/s wr, 12 op/s 2023-09-09T23:55:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2180.776232 secs 2023-09-09T23:55:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:49 smithi082 ceph-mon[98694]: pgmap v1332: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 76 KiB/s wr, 12 op/s 2023-09-09T23:55:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:55:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:55:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:55:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:55:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:55:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:55:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:55:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:55:51.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:55:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:52 smithi170 ceph-mon[102832]: pgmap v1333: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 97 KiB/s wr, 18 op/s 2023-09-09T23:55:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:52 smithi082 ceph-mon[98694]: pgmap v1333: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 97 KiB/s wr, 18 op/s 2023-09-09T23:55:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:52 smithi098 ceph-mon[103124]: pgmap v1333: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 97 KiB/s wr, 18 op/s 2023-09-09T23:55:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2183.450607 secs 2023-09-09T23:55:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2183.450607 secs 2023-09-09T23:55:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2183.450607 secs 2023-09-09T23:55:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2185.046968 secs 2023-09-09T23:55:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:54 smithi170 ceph-mon[102832]: pgmap v1334: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-09T23:55:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2185.046968 secs 2023-09-09T23:55:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:54 smithi082 ceph-mon[98694]: pgmap v1334: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-09T23:55:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2185.046968 secs 2023-09-09T23:55:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:54 smithi098 ceph-mon[103124]: pgmap v1334: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-09T23:55:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2185.776299 secs 2023-09-09T23:55:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2185.776299 secs 2023-09-09T23:55:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2185.776299 secs 2023-09-09T23:55:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:56 smithi170 ceph-mon[102832]: pgmap v1335: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-09T23:55:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:56 smithi082 ceph-mon[98694]: pgmap v1335: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-09T23:55:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:56 smithi098 ceph-mon[103124]: pgmap v1335: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-09T23:55:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2188.450686 secs 2023-09-09T23:55:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:58 smithi170 ceph-mon[102832]: pgmap v1336: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 67 KiB/s wr, 16 op/s 2023-09-09T23:55:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2188.450686 secs 2023-09-09T23:55:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:58 smithi082 ceph-mon[98694]: pgmap v1336: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 67 KiB/s wr, 16 op/s 2023-09-09T23:55:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2188.450686 secs 2023-09-09T23:55:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:58 smithi098 ceph-mon[103124]: pgmap v1336: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 67 KiB/s wr, 16 op/s 2023-09-09T23:55:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:55:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2190.047028 secs 2023-09-09T23:55:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:55:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2190.047028 secs 2023-09-09T23:55:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:55:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2190.047028 secs 2023-09-09T23:56:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2190.776362 secs 2023-09-09T23:56:00.468 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:00 smithi170 ceph-mon[102832]: pgmap v1337: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 32 KiB/s wr, 10 op/s 2023-09-09T23:56:00.469 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2190.776362 secs 2023-09-09T23:56:00.469 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:00 smithi082 ceph-mon[98694]: pgmap v1337: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 32 KiB/s wr, 10 op/s 2023-09-09T23:56:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2190.776362 secs 2023-09-09T23:56:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:00 smithi098 ceph-mon[103124]: pgmap v1337: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 32 KiB/s wr, 10 op/s 2023-09-09T23:56:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:02 smithi170 ceph-mon[102832]: pgmap v1338: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 46 KiB/s wr, 19 op/s 2023-09-09T23:56:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:02 smithi082 ceph-mon[98694]: pgmap v1338: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 46 KiB/s wr, 19 op/s 2023-09-09T23:56:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:02 smithi098 ceph-mon[103124]: pgmap v1338: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 46 KiB/s wr, 19 op/s 2023-09-09T23:56:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2193.450746 secs 2023-09-09T23:56:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2193.450746 secs 2023-09-09T23:56:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2193.450746 secs 2023-09-09T23:56:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2195.047085 secs 2023-09-09T23:56:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:04 smithi170 ceph-mon[102832]: pgmap v1339: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 25 KiB/s wr, 14 op/s 2023-09-09T23:56:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2195.047085 secs 2023-09-09T23:56:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:04 smithi082 ceph-mon[98694]: pgmap v1339: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 25 KiB/s wr, 14 op/s 2023-09-09T23:56:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2195.047085 secs 2023-09-09T23:56:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:04 smithi098 ceph-mon[103124]: pgmap v1339: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 25 KiB/s wr, 14 op/s 2023-09-09T23:56:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2195.776430 secs 2023-09-09T23:56:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2195.776430 secs 2023-09-09T23:56:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2195.776430 secs 2023-09-09T23:56:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:06 smithi170 ceph-mon[102832]: pgmap v1340: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 25 KiB/s wr, 14 op/s 2023-09-09T23:56:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:06 smithi082 ceph-mon[98694]: pgmap v1340: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 25 KiB/s wr, 14 op/s 2023-09-09T23:56:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:06 smithi098 ceph-mon[103124]: pgmap v1340: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 25 KiB/s wr, 14 op/s 2023-09-09T23:56:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2198.450775 secs 2023-09-09T23:56:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:08 smithi170 ceph-mon[102832]: pgmap v1341: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-09-09T23:56:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:08 smithi170 ceph-mon[102832]: osdmap e157: 12 total, 12 up, 12 in 2023-09-09T23:56:08.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2200.063871 secs 2023-09-09T23:56:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2198.450775 secs 2023-09-09T23:56:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:08 smithi082 ceph-mon[98694]: pgmap v1341: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-09-09T23:56:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:08 smithi082 ceph-mon[98694]: osdmap e157: 12 total, 12 up, 12 in 2023-09-09T23:56:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2200.063871 secs 2023-09-09T23:56:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2198.450775 secs 2023-09-09T23:56:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:08 smithi098 ceph-mon[103124]: pgmap v1341: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-09-09T23:56:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:08 smithi098 ceph-mon[103124]: osdmap e157: 12 total, 12 up, 12 in 2023-09-09T23:56:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2200.063871 secs 2023-09-09T23:56:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:56:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:56:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:56:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2200.776496 secs 2023-09-09T23:56:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:10 smithi170 ceph-mon[102832]: pgmap v1343: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-09-09T23:56:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:56:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:56:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:56:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2200.776496 secs 2023-09-09T23:56:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:10 smithi082 ceph-mon[98694]: pgmap v1343: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-09-09T23:56:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:56:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:56:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:56:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2200.776496 secs 2023-09-09T23:56:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:10 smithi098 ceph-mon[103124]: pgmap v1343: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-09-09T23:56:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:56:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:56:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:56:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:12 smithi170 ceph-mon[102832]: pgmap v1344: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 72 KiB/s wr, 24 op/s 2023-09-09T23:56:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:12 smithi082 ceph-mon[98694]: pgmap v1344: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 72 KiB/s wr, 24 op/s 2023-09-09T23:56:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:12 smithi098 ceph-mon[103124]: pgmap v1344: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 72 KiB/s wr, 24 op/s 2023-09-09T23:56:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2203.450842 secs 2023-09-09T23:56:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2205.063930 secs 2023-09-09T23:56:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2203.450842 secs 2023-09-09T23:56:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2205.063930 secs 2023-09-09T23:56:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2203.450842 secs 2023-09-09T23:56:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2205.063930 secs 2023-09-09T23:56:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:14 smithi170 ceph-mon[102832]: pgmap v1345: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 72 KiB/s wr, 24 op/s 2023-09-09T23:56:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:14 smithi170 ceph-mon[102832]: osdmap e158: 12 total, 12 up, 12 in 2023-09-09T23:56:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:14 smithi082 ceph-mon[98694]: pgmap v1345: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 72 KiB/s wr, 24 op/s 2023-09-09T23:56:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:14 smithi082 ceph-mon[98694]: osdmap e158: 12 total, 12 up, 12 in 2023-09-09T23:56:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:14 smithi098 ceph-mon[103124]: pgmap v1345: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 72 KiB/s wr, 24 op/s 2023-09-09T23:56:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:14 smithi098 ceph-mon[103124]: osdmap e158: 12 total, 12 up, 12 in 2023-09-09T23:56:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2205.776564 secs 2023-09-09T23:56:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2205.776564 secs 2023-09-09T23:56:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2205.776564 secs 2023-09-09T23:56:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:16 smithi170 ceph-mon[102832]: pgmap v1347: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 21 KiB/s wr, 7 op/s 2023-09-09T23:56:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:16 smithi082 ceph-mon[98694]: pgmap v1347: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 21 KiB/s wr, 7 op/s 2023-09-09T23:56:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:16 smithi098 ceph-mon[103124]: pgmap v1347: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 21 KiB/s wr, 7 op/s 2023-09-09T23:56:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:18 smithi170 ceph-mon[102832]: pgmap v1348: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-09T23:56:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2210.064049 secs 2023-09-09T23:56:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:18 smithi082 ceph-mon[98694]: pgmap v1348: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-09T23:56:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2210.064049 secs 2023-09-09T23:56:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:18 smithi098 ceph-mon[103124]: pgmap v1348: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-09T23:56:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2210.064049 secs 2023-09-09T23:56:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:56:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:56:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:56:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2210.776627 secs 2023-09-09T23:56:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:20 smithi170 ceph-mon[102832]: pgmap v1349: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-09T23:56:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:56:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:56:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:56:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2210.776627 secs 2023-09-09T23:56:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:20 smithi082 ceph-mon[98694]: pgmap v1349: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-09T23:56:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:56:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:56:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:56:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2210.776627 secs 2023-09-09T23:56:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:20 smithi098 ceph-mon[103124]: pgmap v1349: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-09T23:56:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:56:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:56:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:56:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:22 smithi170 ceph-mon[102832]: pgmap v1350: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 85 KiB/s wr, 12 op/s 2023-09-09T23:56:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:22 smithi082 ceph-mon[98694]: pgmap v1350: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 85 KiB/s wr, 12 op/s 2023-09-09T23:56:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:22 smithi098 ceph-mon[103124]: pgmap v1350: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 85 KiB/s wr, 12 op/s 2023-09-09T23:56:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2215.064106 secs 2023-09-09T23:56:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2215.064106 secs 2023-09-09T23:56:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2215.064106 secs 2023-09-09T23:56:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:24 smithi170 ceph-mon[102832]: pgmap v1351: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 85 KiB/s wr, 12 op/s 2023-09-09T23:56:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:24 smithi082 ceph-mon[98694]: pgmap v1351: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 85 KiB/s wr, 12 op/s 2023-09-09T23:56:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:24 smithi098 ceph-mon[103124]: pgmap v1351: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 85 KiB/s wr, 12 op/s 2023-09-09T23:56:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2215.776692 secs 2023-09-09T23:56:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2215.776692 secs 2023-09-09T23:56:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2215.776692 secs 2023-09-09T23:56:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:26 smithi170 ceph-mon[102832]: pgmap v1352: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-09T23:56:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:26 smithi082 ceph-mon[98694]: pgmap v1352: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-09T23:56:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:26 smithi098 ceph-mon[103124]: pgmap v1352: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-09T23:56:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:28 smithi170 ceph-mon[102832]: pgmap v1353: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 90 KiB/s wr, 16 op/s 2023-09-09T23:56:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2220.064162 secs 2023-09-09T23:56:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:28 smithi082 ceph-mon[98694]: pgmap v1353: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 90 KiB/s wr, 16 op/s 2023-09-09T23:56:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2220.064162 secs 2023-09-09T23:56:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:28 smithi098 ceph-mon[103124]: pgmap v1353: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 90 KiB/s wr, 16 op/s 2023-09-09T23:56:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2220.064162 secs 2023-09-09T23:56:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2220.776756 secs 2023-09-09T23:56:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:30 smithi170 ceph-mon[102832]: pgmap v1354: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-09T23:56:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2220.776756 secs 2023-09-09T23:56:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:30 smithi082 ceph-mon[98694]: pgmap v1354: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-09T23:56:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2220.776756 secs 2023-09-09T23:56:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:30 smithi098 ceph-mon[103124]: pgmap v1354: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-09T23:56:32.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:32 smithi170 ceph-mon[102832]: pgmap v1355: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 131 KiB/s wr, 16 op/s 2023-09-09T23:56:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:32 smithi082 ceph-mon[98694]: pgmap v1355: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 131 KiB/s wr, 16 op/s 2023-09-09T23:56:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:32 smithi098 ceph-mon[103124]: pgmap v1355: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 131 KiB/s wr, 16 op/s 2023-09-09T23:56:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2225.064224 secs 2023-09-09T23:56:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2225.064224 secs 2023-09-09T23:56:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2225.064224 secs 2023-09-09T23:56:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:34 smithi170 ceph-mon[102832]: pgmap v1356: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-09T23:56:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:34 smithi082 ceph-mon[98694]: pgmap v1356: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-09T23:56:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:34 smithi098 ceph-mon[103124]: pgmap v1356: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-09T23:56:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2225.776822 secs 2023-09-09T23:56:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2225.776822 secs 2023-09-09T23:56:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2225.776822 secs 2023-09-09T23:56:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:36 smithi170 ceph-mon[102832]: pgmap v1357: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-09T23:56:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:36 smithi082 ceph-mon[98694]: pgmap v1357: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-09T23:56:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:36 smithi098 ceph-mon[103124]: pgmap v1357: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-09T23:56:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:38 smithi170 ceph-mon[102832]: pgmap v1358: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 111 KiB/s wr, 17 op/s 2023-09-09T23:56:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2230.064280 secs 2023-09-09T23:56:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:38 smithi082 ceph-mon[98694]: pgmap v1358: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 111 KiB/s wr, 17 op/s 2023-09-09T23:56:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2230.064280 secs 2023-09-09T23:56:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:38 smithi098 ceph-mon[103124]: pgmap v1358: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 111 KiB/s wr, 17 op/s 2023-09-09T23:56:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2230.064280 secs 2023-09-09T23:56:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2230.776886 secs 2023-09-09T23:56:40.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:40 smithi170 ceph-mon[102832]: pgmap v1359: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-09T23:56:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2230.776886 secs 2023-09-09T23:56:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:40 smithi082 ceph-mon[98694]: pgmap v1359: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-09T23:56:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2230.776886 secs 2023-09-09T23:56:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:40 smithi098 ceph-mon[103124]: pgmap v1359: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-09T23:56:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:42 smithi170 ceph-mon[102832]: pgmap v1360: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 17 op/s 2023-09-09T23:56:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2233.451255 secs 2023-09-09T23:56:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:42 smithi082 ceph-mon[98694]: pgmap v1360: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 17 op/s 2023-09-09T23:56:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2233.451255 secs 2023-09-09T23:56:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:42 smithi098 ceph-mon[103124]: pgmap v1360: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 17 op/s 2023-09-09T23:56:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2233.451255 secs 2023-09-09T23:56:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2235.064330 secs 2023-09-09T23:56:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2235.064330 secs 2023-09-09T23:56:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2235.064330 secs 2023-09-09T23:56:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:44 smithi170 ceph-mon[102832]: pgmap v1361: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-09T23:56:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:56:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:56:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:56:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:44 smithi082 ceph-mon[98694]: pgmap v1361: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-09T23:56:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:56:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:56:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:56:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:44 smithi098 ceph-mon[103124]: pgmap v1361: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-09T23:56:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:56:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:56:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:56:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2235.776952 secs 2023-09-09T23:56:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2235.776952 secs 2023-09-09T23:56:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2235.776952 secs 2023-09-09T23:56:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:46 smithi170 ceph-mon[102832]: pgmap v1362: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-09T23:56:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:46 smithi082 ceph-mon[98694]: pgmap v1362: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-09T23:56:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:46 smithi098 ceph-mon[103124]: pgmap v1362: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-09T23:56:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2238.451379 secs 2023-09-09T23:56:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2238.451379 secs 2023-09-09T23:56:47.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2238.451379 secs 2023-09-09T23:56:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:48 smithi082 ceph-mon[98694]: pgmap v1363: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 94 KiB/s wr, 16 op/s 2023-09-09T23:56:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2240.064413 secs 2023-09-09T23:56:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:48 smithi098 ceph-mon[103124]: pgmap v1363: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 94 KiB/s wr, 16 op/s 2023-09-09T23:56:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2240.064413 secs 2023-09-09T23:56:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:48 smithi170 ceph-mon[102832]: pgmap v1363: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 94 KiB/s wr, 16 op/s 2023-09-09T23:56:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2240.064413 secs 2023-09-09T23:56:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2240.777017 secs 2023-09-09T23:56:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:50 smithi098 ceph-mon[103124]: pgmap v1364: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-09T23:56:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2240.777017 secs 2023-09-09T23:56:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:50 smithi170 ceph-mon[102832]: pgmap v1364: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-09T23:56:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2240.777017 secs 2023-09-09T23:56:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:50 smithi082 ceph-mon[98694]: pgmap v1364: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-09T23:56:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:52 smithi170 ceph-mon[102832]: pgmap v1365: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 58 KiB/s wr, 15 op/s 2023-09-09T23:56:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:52 smithi082 ceph-mon[98694]: pgmap v1365: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 58 KiB/s wr, 15 op/s 2023-09-09T23:56:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:52 smithi098 ceph-mon[103124]: pgmap v1365: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 58 KiB/s wr, 15 op/s 2023-09-09T23:56:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2243.451439 secs 2023-09-09T23:56:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2243.451439 secs 2023-09-09T23:56:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2243.451439 secs 2023-09-09T23:56:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2245.064470 secs 2023-09-09T23:56:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:54 smithi170 ceph-mon[102832]: pgmap v1366: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 9 op/s 2023-09-09T23:56:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2245.064470 secs 2023-09-09T23:56:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:54 smithi082 ceph-mon[98694]: pgmap v1366: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 9 op/s 2023-09-09T23:56:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2245.064470 secs 2023-09-09T23:56:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:54 smithi098 ceph-mon[103124]: pgmap v1366: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 9 op/s 2023-09-09T23:56:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2245.777082 secs 2023-09-09T23:56:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2245.777082 secs 2023-09-09T23:56:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2245.777082 secs 2023-09-09T23:56:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:56 smithi170 ceph-mon[102832]: pgmap v1367: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 9 op/s 2023-09-09T23:56:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:56 smithi082 ceph-mon[98694]: pgmap v1367: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 9 op/s 2023-09-09T23:56:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:56 smithi098 ceph-mon[103124]: pgmap v1367: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 9 op/s 2023-09-09T23:56:57.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2248.451500 secs 2023-09-09T23:56:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2248.451500 secs 2023-09-09T23:56:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2248.451500 secs 2023-09-09T23:56:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:58 smithi170 ceph-mon[102832]: pgmap v1368: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 15 op/s 2023-09-09T23:56:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:56:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2250.064551 secs 2023-09-09T23:56:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:58 smithi082 ceph-mon[98694]: pgmap v1368: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 15 op/s 2023-09-09T23:56:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:56:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2250.064551 secs 2023-09-09T23:56:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:58 smithi098 ceph-mon[103124]: pgmap v1368: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 15 op/s 2023-09-09T23:56:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:56:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2250.064551 secs 2023-09-09T23:57:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2250.777147 secs 2023-09-09T23:57:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:00 smithi170 ceph-mon[102832]: pgmap v1369: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 26 KiB/s wr, 10 op/s 2023-09-09T23:57:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2250.777147 secs 2023-09-09T23:57:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:00 smithi082 ceph-mon[98694]: pgmap v1369: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 26 KiB/s wr, 10 op/s 2023-09-09T23:57:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2250.777147 secs 2023-09-09T23:57:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:00 smithi098 ceph-mon[103124]: pgmap v1369: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 26 KiB/s wr, 10 op/s 2023-09-09T23:57:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:02 smithi170 ceph-mon[102832]: pgmap v1370: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 28 KiB/s wr, 15 op/s 2023-09-09T23:57:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:02 smithi082 ceph-mon[98694]: pgmap v1370: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 28 KiB/s wr, 15 op/s 2023-09-09T23:57:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:02 smithi098 ceph-mon[103124]: pgmap v1370: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 28 KiB/s wr, 15 op/s 2023-09-09T23:57:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2253.451559 secs 2023-09-09T23:57:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2253.451559 secs 2023-09-09T23:57:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2253.451559 secs 2023-09-09T23:57:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2255.064637 secs 2023-09-09T23:57:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:04 smithi170 ceph-mon[102832]: pgmap v1371: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 10 op/s 2023-09-09T23:57:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2255.064637 secs 2023-09-09T23:57:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:04 smithi082 ceph-mon[98694]: pgmap v1371: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 10 op/s 2023-09-09T23:57:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2255.064637 secs 2023-09-09T23:57:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:04 smithi098 ceph-mon[103124]: pgmap v1371: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 10 op/s 2023-09-09T23:57:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2255.777213 secs 2023-09-09T23:57:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2255.777213 secs 2023-09-09T23:57:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2255.777213 secs 2023-09-09T23:57:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:06 smithi170 ceph-mon[102832]: pgmap v1372: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 10 op/s 2023-09-09T23:57:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:06 smithi082 ceph-mon[98694]: pgmap v1372: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 10 op/s 2023-09-09T23:57:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:06 smithi098 ceph-mon[103124]: pgmap v1372: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 10 op/s 2023-09-09T23:57:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2258.451683 secs 2023-09-09T23:57:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:08 smithi170 ceph-mon[102832]: pgmap v1373: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 72 KiB/s wr, 31 op/s 2023-09-09T23:57:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:08 smithi170 ceph-mon[102832]: osdmap e159: 12 total, 12 up, 12 in 2023-09-09T23:57:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2258.451683 secs 2023-09-09T23:57:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:08 smithi082 ceph-mon[98694]: pgmap v1373: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 72 KiB/s wr, 31 op/s 2023-09-09T23:57:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:08 smithi082 ceph-mon[98694]: osdmap e159: 12 total, 12 up, 12 in 2023-09-09T23:57:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2258.451683 secs 2023-09-09T23:57:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:08 smithi098 ceph-mon[103124]: pgmap v1373: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 72 KiB/s wr, 31 op/s 2023-09-09T23:57:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:08 smithi098 ceph-mon[103124]: osdmap e159: 12 total, 12 up, 12 in 2023-09-09T23:57:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2260.064696 secs 2023-09-09T23:57:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2260.064696 secs 2023-09-09T23:57:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2260.064696 secs 2023-09-09T23:57:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2260.777279 secs 2023-09-09T23:57:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:10 smithi170 ceph-mon[102832]: pgmap v1375: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 72 KiB/s wr, 30 op/s 2023-09-09T23:57:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:57:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:57:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:57:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2260.777279 secs 2023-09-09T23:57:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:10 smithi082 ceph-mon[98694]: pgmap v1375: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 72 KiB/s wr, 30 op/s 2023-09-09T23:57:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:57:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:57:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:57:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2260.777279 secs 2023-09-09T23:57:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:10 smithi098 ceph-mon[103124]: pgmap v1375: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 72 KiB/s wr, 30 op/s 2023-09-09T23:57:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:57:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:57:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:57:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:57:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:57:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:57:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:57:11.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:57:11.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:57:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:57:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:57:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:57:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:12 smithi170 ceph-mon[102832]: pgmap v1376: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-09-09T23:57:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2263.451789 secs 2023-09-09T23:57:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:12 smithi082 ceph-mon[98694]: pgmap v1376: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-09-09T23:57:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2263.451789 secs 2023-09-09T23:57:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:12 smithi098 ceph-mon[103124]: pgmap v1376: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-09-09T23:57:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2263.451789 secs 2023-09-09T23:57:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2265.064754 secs 2023-09-09T23:57:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2265.064754 secs 2023-09-09T23:57:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2265.064754 secs 2023-09-09T23:57:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:14 smithi170 ceph-mon[102832]: pgmap v1377: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-09-09T23:57:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2265.777345 secs 2023-09-09T23:57:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:14 smithi170 ceph-mon[102832]: osdmap e160: 12 total, 12 up, 12 in 2023-09-09T23:57:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:14 smithi082 ceph-mon[98694]: pgmap v1377: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-09-09T23:57:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2265.777345 secs 2023-09-09T23:57:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:14 smithi082 ceph-mon[98694]: osdmap e160: 12 total, 12 up, 12 in 2023-09-09T23:57:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:14 smithi098 ceph-mon[103124]: pgmap v1377: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-09-09T23:57:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2265.777345 secs 2023-09-09T23:57:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:14 smithi098 ceph-mon[103124]: osdmap e160: 12 total, 12 up, 12 in 2023-09-09T23:57:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:16 smithi170 ceph-mon[102832]: pgmap v1379: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2023-09-09T23:57:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:16 smithi082 ceph-mon[98694]: pgmap v1379: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2023-09-09T23:57:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:16 smithi098 ceph-mon[103124]: pgmap v1379: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2023-09-09T23:57:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:18 smithi170 ceph-mon[102832]: pgmap v1380: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-09T23:57:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2270.064811 secs 2023-09-09T23:57:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:18 smithi082 ceph-mon[98694]: pgmap v1380: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-09T23:57:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2270.064811 secs 2023-09-09T23:57:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:18 smithi098 ceph-mon[103124]: pgmap v1380: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-09T23:57:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2270.064811 secs 2023-09-09T23:57:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2270.777410 secs 2023-09-09T23:57:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:57:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2270.777410 secs 2023-09-09T23:57:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:57:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2270.777410 secs 2023-09-09T23:57:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:57:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:20 smithi170 ceph-mon[102832]: pgmap v1381: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-09T23:57:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:57:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:57:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:57:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:20 smithi082 ceph-mon[98694]: pgmap v1381: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-09T23:57:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:57:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:57:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:57:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:20 smithi098 ceph-mon[103124]: pgmap v1381: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-09T23:57:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:57:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:57:20.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:57:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:22 smithi170 ceph-mon[102832]: pgmap v1382: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-09T23:57:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:22 smithi082 ceph-mon[98694]: pgmap v1382: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-09T23:57:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:22 smithi098 ceph-mon[103124]: pgmap v1382: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-09T23:57:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2275.064899 secs 2023-09-09T23:57:23.713 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2275.064899 secs 2023-09-09T23:57:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2275.064899 secs 2023-09-09T23:57:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:24 smithi170 ceph-mon[102832]: pgmap v1383: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-09T23:57:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2275.777482 secs 2023-09-09T23:57:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:24 smithi082 ceph-mon[98694]: pgmap v1383: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-09T23:57:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2275.777482 secs 2023-09-09T23:57:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:24 smithi098 ceph-mon[103124]: pgmap v1383: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-09T23:57:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2275.777482 secs 2023-09-09T23:57:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:26 smithi170 ceph-mon[102832]: pgmap v1384: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-09-09T23:57:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:26 smithi082 ceph-mon[98694]: pgmap v1384: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-09-09T23:57:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:26 smithi098 ceph-mon[103124]: pgmap v1384: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-09-09T23:57:27.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2278.452031 secs 2023-09-09T23:57:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2278.452031 secs 2023-09-09T23:57:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2278.452031 secs 2023-09-09T23:57:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:28 smithi170 ceph-mon[102832]: pgmap v1385: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 52 KiB/s wr, 14 op/s 2023-09-09T23:57:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2280.064983 secs 2023-09-09T23:57:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:28 smithi082 ceph-mon[98694]: pgmap v1385: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 52 KiB/s wr, 14 op/s 2023-09-09T23:57:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2280.064983 secs 2023-09-09T23:57:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:28 smithi098 ceph-mon[103124]: pgmap v1385: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 52 KiB/s wr, 14 op/s 2023-09-09T23:57:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2280.064983 secs 2023-09-09T23:57:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2280.777549 secs 2023-09-09T23:57:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:30 smithi170 ceph-mon[102832]: pgmap v1386: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-09T23:57:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2280.777549 secs 2023-09-09T23:57:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:30 smithi082 ceph-mon[98694]: pgmap v1386: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-09T23:57:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2280.777549 secs 2023-09-09T23:57:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:30 smithi098 ceph-mon[103124]: pgmap v1386: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-09T23:57:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:31 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:57:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:57:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:57:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:31 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:57:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:57:31.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:57:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:31 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:57:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:57:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:57:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:32 smithi170 ceph-mon[102832]: pgmap v1387: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 83 KiB/s wr, 16 op/s 2023-09-09T23:57:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:32 smithi082 ceph-mon[98694]: pgmap v1387: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 83 KiB/s wr, 16 op/s 2023-09-09T23:57:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:32 smithi098 ceph-mon[103124]: pgmap v1387: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 83 KiB/s wr, 16 op/s 2023-09-09T23:57:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2285.065039 secs 2023-09-09T23:57:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:34 smithi170 ceph-mon[102832]: pgmap v1388: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-09T23:57:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2285.065039 secs 2023-09-09T23:57:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:34 smithi082 ceph-mon[98694]: pgmap v1388: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-09T23:57:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2285.065039 secs 2023-09-09T23:57:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:34 smithi098 ceph-mon[103124]: pgmap v1388: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-09T23:57:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2285.777615 secs 2023-09-09T23:57:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:57:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2285.777615 secs 2023-09-09T23:57:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:57:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2285.777615 secs 2023-09-09T23:57:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:57:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:36 smithi170 ceph-mon[102832]: pgmap v1389: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-09T23:57:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:57:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:57:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:36 smithi082 ceph-mon[98694]: pgmap v1389: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-09T23:57:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:57:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:57:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:36 smithi098 ceph-mon[103124]: pgmap v1389: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-09T23:57:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:57:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:57:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:38 smithi170 ceph-mon[102832]: pgmap v1390: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 100 KiB/s wr, 17 op/s 2023-09-09T23:57:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2290.065099 secs 2023-09-09T23:57:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:38 smithi082 ceph-mon[98694]: pgmap v1390: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 100 KiB/s wr, 17 op/s 2023-09-09T23:57:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2290.065099 secs 2023-09-09T23:57:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:38 smithi098 ceph-mon[103124]: pgmap v1390: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 100 KiB/s wr, 17 op/s 2023-09-09T23:57:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2290.065099 secs 2023-09-09T23:57:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:39 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:57:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:39 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:57:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:39 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:57:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2290.777682 secs 2023-09-09T23:57:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:40 smithi170 ceph-mon[102832]: pgmap v1391: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 70 KiB/s wr, 12 op/s 2023-09-09T23:57:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2290.777682 secs 2023-09-09T23:57:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:40 smithi082 ceph-mon[98694]: pgmap v1391: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 70 KiB/s wr, 12 op/s 2023-09-09T23:57:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2290.777682 secs 2023-09-09T23:57:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:40 smithi098 ceph-mon[103124]: pgmap v1391: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 70 KiB/s wr, 12 op/s 2023-09-09T23:57:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:42 smithi170 ceph-mon[102832]: pgmap v1392: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 121 KiB/s wr, 17 op/s 2023-09-09T23:57:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:42 smithi082 ceph-mon[98694]: pgmap v1392: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 121 KiB/s wr, 17 op/s 2023-09-09T23:57:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:42 smithi098 ceph-mon[103124]: pgmap v1392: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 121 KiB/s wr, 17 op/s 2023-09-09T23:57:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2295.065155 secs 2023-09-09T23:57:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2295.065155 secs 2023-09-09T23:57:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2295.065155 secs 2023-09-09T23:57:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:44 smithi170 ceph-mon[102832]: pgmap v1393: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-09T23:57:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:44 smithi082 ceph-mon[98694]: pgmap v1393: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-09T23:57:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:44 smithi098 ceph-mon[103124]: pgmap v1393: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-09T23:57:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2295.777749 secs 2023-09-09T23:57:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2295.777749 secs 2023-09-09T23:57:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2295.777749 secs 2023-09-09T23:57:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:46 smithi170 ceph-mon[102832]: pgmap v1394: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-09T23:57:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:46 smithi082 ceph-mon[98694]: pgmap v1394: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-09T23:57:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:46 smithi098 ceph-mon[103124]: pgmap v1394: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-09T23:57:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2298.452382 secs 2023-09-09T23:57:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:48 smithi170 ceph-mon[102832]: pgmap v1395: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 94 KiB/s wr, 17 op/s 2023-09-09T23:57:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2300.065213 secs 2023-09-09T23:57:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2298.452382 secs 2023-09-09T23:57:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:48 smithi082 ceph-mon[98694]: pgmap v1395: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 94 KiB/s wr, 17 op/s 2023-09-09T23:57:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2300.065213 secs 2023-09-09T23:57:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2298.452382 secs 2023-09-09T23:57:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:48 smithi098 ceph-mon[103124]: pgmap v1395: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 94 KiB/s wr, 17 op/s 2023-09-09T23:57:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2300.065213 secs 2023-09-09T23:57:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2300.777812 secs 2023-09-09T23:57:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:50 smithi170 ceph-mon[102832]: pgmap v1396: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-09T23:57:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2300.777812 secs 2023-09-09T23:57:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:50 smithi082 ceph-mon[98694]: pgmap v1396: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-09T23:57:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2300.777812 secs 2023-09-09T23:57:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:50 smithi098 ceph-mon[103124]: pgmap v1396: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-09T23:57:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:57:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:57:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:57:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:57:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:57:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:57:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:57:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:57:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:57:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:52 smithi170 ceph-mon[102832]: pgmap v1397: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 93 KiB/s wr, 16 op/s 2023-09-09T23:57:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2303.452489 secs 2023-09-09T23:57:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:52 smithi082 ceph-mon[98694]: pgmap v1397: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 93 KiB/s wr, 16 op/s 2023-09-09T23:57:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2303.452489 secs 2023-09-09T23:57:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:52 smithi098 ceph-mon[103124]: pgmap v1397: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 93 KiB/s wr, 16 op/s 2023-09-09T23:57:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2303.452489 secs 2023-09-09T23:57:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2305.065274 secs 2023-09-09T23:57:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2305.065274 secs 2023-09-09T23:57:53.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2305.065274 secs 2023-09-09T23:57:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:54 smithi170 ceph-mon[102832]: pgmap v1398: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-09T23:57:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:54 smithi082 ceph-mon[98694]: pgmap v1398: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-09T23:57:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:54 smithi098 ceph-mon[103124]: pgmap v1398: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-09T23:57:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2305.777877 secs 2023-09-09T23:57:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2305.777877 secs 2023-09-09T23:57:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2305.777877 secs 2023-09-09T23:57:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:56 smithi170 ceph-mon[102832]: pgmap v1399: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-09T23:57:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:56 smithi082 ceph-mon[98694]: pgmap v1399: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-09T23:57:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:56 smithi098 ceph-mon[103124]: pgmap v1399: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-09T23:57:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2308.452547 secs 2023-09-09T23:57:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2308.452547 secs 2023-09-09T23:57:57.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2308.452547 secs 2023-09-09T23:57:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:58 smithi082 ceph-mon[98694]: pgmap v1400: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 54 KiB/s wr, 15 op/s 2023-09-09T23:57:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:57:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2310.065335 secs 2023-09-09T23:57:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:58 smithi098 ceph-mon[103124]: pgmap v1400: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 54 KiB/s wr, 15 op/s 2023-09-09T23:57:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:57:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2310.065335 secs 2023-09-09T23:57:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:58 smithi170 ceph-mon[102832]: pgmap v1400: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 54 KiB/s wr, 15 op/s 2023-09-09T23:57:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:57:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2310.065335 secs 2023-09-09T23:58:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2310.777944 secs 2023-09-09T23:58:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:00 smithi082 ceph-mon[98694]: pgmap v1401: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-09-09T23:58:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2310.777944 secs 2023-09-09T23:58:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:00 smithi098 ceph-mon[103124]: pgmap v1401: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-09-09T23:58:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2310.777944 secs 2023-09-09T23:58:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:00 smithi170 ceph-mon[102832]: pgmap v1401: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-09-09T23:58:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:02 smithi082 ceph-mon[98694]: pgmap v1402: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 14 op/s 2023-09-09T23:58:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2313.452625 secs 2023-09-09T23:58:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:02 smithi098 ceph-mon[103124]: pgmap v1402: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 14 op/s 2023-09-09T23:58:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2313.452625 secs 2023-09-09T23:58:02.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:02 smithi170 ceph-mon[102832]: pgmap v1402: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 14 op/s 2023-09-09T23:58:02.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2313.452625 secs 2023-09-09T23:58:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2315.065399 secs 2023-09-09T23:58:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2315.065399 secs 2023-09-09T23:58:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2315.065399 secs 2023-09-09T23:58:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:04 smithi082 ceph-mon[98694]: pgmap v1403: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-09T23:58:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2315.778015 secs 2023-09-09T23:58:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:04 smithi098 ceph-mon[103124]: pgmap v1403: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-09T23:58:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2315.778015 secs 2023-09-09T23:58:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:04 smithi170 ceph-mon[102832]: pgmap v1403: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-09T23:58:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2315.778015 secs 2023-09-09T23:58:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:06 smithi082 ceph-mon[98694]: pgmap v1404: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-09T23:58:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:06 smithi098 ceph-mon[103124]: pgmap v1404: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-09T23:58:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:06 smithi170 ceph-mon[102832]: pgmap v1404: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-09T23:58:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2318.452684 secs 2023-09-09T23:58:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2318.452684 secs 2023-09-09T23:58:07.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2318.452684 secs 2023-09-09T23:58:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:08 smithi082 ceph-mon[98694]: pgmap v1405: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-09-09T23:58:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:08 smithi082 ceph-mon[98694]: osdmap e161: 12 total, 12 up, 12 in 2023-09-09T23:58:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2320.065475 secs 2023-09-09T23:58:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:08 smithi098 ceph-mon[103124]: pgmap v1405: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-09-09T23:58:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:08 smithi098 ceph-mon[103124]: osdmap e161: 12 total, 12 up, 12 in 2023-09-09T23:58:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2320.065475 secs 2023-09-09T23:58:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:08 smithi170 ceph-mon[102832]: pgmap v1405: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-09-09T23:58:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:08 smithi170 ceph-mon[102832]: osdmap e161: 12 total, 12 up, 12 in 2023-09-09T23:58:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2320.065475 secs 2023-09-09T23:58:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2320.778080 secs 2023-09-09T23:58:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:10 smithi170 ceph-mon[102832]: pgmap v1407: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-09-09T23:58:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2320.778080 secs 2023-09-09T23:58:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:10 smithi082 ceph-mon[98694]: pgmap v1407: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-09-09T23:58:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2320.778080 secs 2023-09-09T23:58:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:10 smithi098 ceph-mon[103124]: pgmap v1407: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-09-09T23:58:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:58:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:58:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:58:11.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:58:11.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:58:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:58:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:58:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:58:11.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:58:11.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:58:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:58:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:58:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:58:11.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:58:11.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:58:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:12 smithi170 ceph-mon[102832]: pgmap v1408: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-09T23:58:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:58:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:58:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:58:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:12 smithi082 ceph-mon[98694]: pgmap v1408: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-09T23:58:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:58:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:58:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:58:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:12 smithi098 ceph-mon[103124]: pgmap v1408: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-09T23:58:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:58:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:58:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:58:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2323.452743 secs 2023-09-09T23:58:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2323.452743 secs 2023-09-09T23:58:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2323.452743 secs 2023-09-09T23:58:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2325.065556 secs 2023-09-09T23:58:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:14 smithi170 ceph-mon[102832]: pgmap v1409: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-09T23:58:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:14 smithi170 ceph-mon[102832]: osdmap e162: 12 total, 12 up, 12 in 2023-09-09T23:58:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2325.065556 secs 2023-09-09T23:58:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:14 smithi082 ceph-mon[98694]: pgmap v1409: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-09T23:58:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:14 smithi082 ceph-mon[98694]: osdmap e162: 12 total, 12 up, 12 in 2023-09-09T23:58:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2325.065556 secs 2023-09-09T23:58:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:14 smithi098 ceph-mon[103124]: pgmap v1409: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-09T23:58:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:14 smithi098 ceph-mon[103124]: osdmap e162: 12 total, 12 up, 12 in 2023-09-09T23:58:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2325.778145 secs 2023-09-09T23:58:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2325.778145 secs 2023-09-09T23:58:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2325.778145 secs 2023-09-09T23:58:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:16 smithi170 ceph-mon[102832]: pgmap v1411: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-09-09T23:58:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:16 smithi082 ceph-mon[98694]: pgmap v1411: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-09-09T23:58:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:16 smithi098 ceph-mon[103124]: pgmap v1411: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-09-09T23:58:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:18 smithi170 ceph-mon[102832]: pgmap v1412: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 12 op/s 2023-09-09T23:58:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:18 smithi082 ceph-mon[98694]: pgmap v1412: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 12 op/s 2023-09-09T23:58:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:18 smithi098 ceph-mon[103124]: pgmap v1412: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 12 op/s 2023-09-09T23:58:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2330.065636 secs 2023-09-09T23:58:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:58:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2330.065636 secs 2023-09-09T23:58:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:58:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2330.065636 secs 2023-09-09T23:58:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:58:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2330.778210 secs 2023-09-09T23:58:20.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:20 smithi170 ceph-mon[102832]: pgmap v1413: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 11 op/s 2023-09-09T23:58:20.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:58:20.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:58:20.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:58:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2330.778210 secs 2023-09-09T23:58:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:20 smithi082 ceph-mon[98694]: pgmap v1413: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 11 op/s 2023-09-09T23:58:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:58:20.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:58:20.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:58:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2330.778210 secs 2023-09-09T23:58:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:20 smithi098 ceph-mon[103124]: pgmap v1413: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 11 op/s 2023-09-09T23:58:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:58:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:58:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:58:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:22 smithi170 ceph-mon[102832]: pgmap v1414: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 89 KiB/s wr, 12 op/s 2023-09-09T23:58:22.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:22 smithi082 ceph-mon[98694]: pgmap v1414: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 89 KiB/s wr, 12 op/s 2023-09-09T23:58:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:22 smithi098 ceph-mon[103124]: pgmap v1414: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 89 KiB/s wr, 12 op/s 2023-09-09T23:58:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2335.065714 secs 2023-09-09T23:58:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:24 smithi170 ceph-mon[102832]: pgmap v1415: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 89 KiB/s wr, 12 op/s 2023-09-09T23:58:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2335.065714 secs 2023-09-09T23:58:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:24 smithi082 ceph-mon[98694]: pgmap v1415: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 89 KiB/s wr, 12 op/s 2023-09-09T23:58:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2335.065714 secs 2023-09-09T23:58:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:24 smithi098 ceph-mon[103124]: pgmap v1415: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 89 KiB/s wr, 12 op/s 2023-09-09T23:58:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2335.778278 secs 2023-09-09T23:58:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2335.778278 secs 2023-09-09T23:58:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2335.778278 secs 2023-09-09T23:58:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:26 smithi170 ceph-mon[102832]: pgmap v1416: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-09T23:58:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:26 smithi082 ceph-mon[98694]: pgmap v1416: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-09T23:58:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:26 smithi098 ceph-mon[103124]: pgmap v1416: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-09T23:58:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:28 smithi170 ceph-mon[102832]: pgmap v1417: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 96 KiB/s wr, 16 op/s 2023-09-09T23:58:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:28 smithi082 ceph-mon[98694]: pgmap v1417: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 96 KiB/s wr, 16 op/s 2023-09-09T23:58:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:28 smithi098 ceph-mon[103124]: pgmap v1417: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 96 KiB/s wr, 16 op/s 2023-09-09T23:58:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2340.065804 secs 2023-09-09T23:58:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2340.065804 secs 2023-09-09T23:58:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2340.065804 secs 2023-09-09T23:58:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2340.778343 secs 2023-09-09T23:58:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:30 smithi170 ceph-mon[102832]: pgmap v1418: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-09T23:58:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2340.778343 secs 2023-09-09T23:58:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:30 smithi082 ceph-mon[98694]: pgmap v1418: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-09T23:58:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2340.778343 secs 2023-09-09T23:58:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:30 smithi098 ceph-mon[103124]: pgmap v1418: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-09T23:58:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:32 smithi170 ceph-mon[102832]: pgmap v1419: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 137 KiB/s wr, 17 op/s 2023-09-09T23:58:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:32 smithi082 ceph-mon[98694]: pgmap v1419: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 137 KiB/s wr, 17 op/s 2023-09-09T23:58:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:32 smithi098 ceph-mon[103124]: pgmap v1419: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 137 KiB/s wr, 17 op/s 2023-09-09T23:58:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2343.453023 secs 2023-09-09T23:58:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2343.453023 secs 2023-09-09T23:58:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2343.453023 secs 2023-09-09T23:58:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2345.065862 secs 2023-09-09T23:58:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:34 smithi170 ceph-mon[102832]: pgmap v1420: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-09T23:58:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2345.065862 secs 2023-09-09T23:58:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:34 smithi082 ceph-mon[98694]: pgmap v1420: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-09T23:58:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2345.065862 secs 2023-09-09T23:58:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:34 smithi098 ceph-mon[103124]: pgmap v1420: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-09T23:58:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2345.778411 secs 2023-09-09T23:58:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2345.778411 secs 2023-09-09T23:58:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2345.778411 secs 2023-09-09T23:58:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:36 smithi170 ceph-mon[102832]: pgmap v1421: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-09T23:58:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:36 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:58:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:58:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:58:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:36 smithi082 ceph-mon[98694]: pgmap v1421: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-09T23:58:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:36 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:58:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:58:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:58:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:36 smithi098 ceph-mon[103124]: pgmap v1421: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-09T23:58:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:36 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:58:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:58:36.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:58:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2348.453087 secs 2023-09-09T23:58:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:38 smithi170 ceph-mon[102832]: pgmap v1422: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 116 KiB/s wr, 19 op/s 2023-09-09T23:58:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2350.065921 secs 2023-09-09T23:58:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2348.453087 secs 2023-09-09T23:58:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:38 smithi082 ceph-mon[98694]: pgmap v1422: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 116 KiB/s wr, 19 op/s 2023-09-09T23:58:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2350.065921 secs 2023-09-09T23:58:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2348.453087 secs 2023-09-09T23:58:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:38 smithi098 ceph-mon[103124]: pgmap v1422: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 116 KiB/s wr, 19 op/s 2023-09-09T23:58:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2350.065921 secs 2023-09-09T23:58:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2350.778481 secs 2023-09-09T23:58:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:40 smithi170 ceph-mon[102832]: pgmap v1423: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-09T23:58:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2350.778481 secs 2023-09-09T23:58:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:40 smithi082 ceph-mon[98694]: pgmap v1423: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-09T23:58:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2350.778481 secs 2023-09-09T23:58:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:40 smithi098 ceph-mon[103124]: pgmap v1423: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-09T23:58:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:42 smithi170 ceph-mon[102832]: pgmap v1424: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 114 KiB/s wr, 18 op/s 2023-09-09T23:58:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:42 smithi082 ceph-mon[98694]: pgmap v1424: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 114 KiB/s wr, 18 op/s 2023-09-09T23:58:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:42 smithi098 ceph-mon[103124]: pgmap v1424: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 114 KiB/s wr, 18 op/s 2023-09-09T23:58:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2353.453148 secs 2023-09-09T23:58:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2355.065979 secs 2023-09-09T23:58:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2353.453148 secs 2023-09-09T23:58:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2355.065979 secs 2023-09-09T23:58:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2353.453148 secs 2023-09-09T23:58:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2355.065979 secs 2023-09-09T23:58:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:44 smithi170 ceph-mon[102832]: pgmap v1425: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-09T23:58:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:44 smithi082 ceph-mon[98694]: pgmap v1425: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-09T23:58:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:44 smithi098 ceph-mon[103124]: pgmap v1425: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-09T23:58:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2355.778547 secs 2023-09-09T23:58:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2355.778547 secs 2023-09-09T23:58:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2355.778547 secs 2023-09-09T23:58:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:46 smithi170 ceph-mon[102832]: pgmap v1426: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-09T23:58:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:46 smithi082 ceph-mon[98694]: pgmap v1426: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-09T23:58:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:46 smithi098 ceph-mon[103124]: pgmap v1426: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-09T23:58:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2358.453239 secs 2023-09-09T23:58:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:48 smithi170 ceph-mon[102832]: pgmap v1427: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 109 KiB/s wr, 17 op/s 2023-09-09T23:58:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2360.066051 secs 2023-09-09T23:58:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2358.453239 secs 2023-09-09T23:58:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:48 smithi082 ceph-mon[98694]: pgmap v1427: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 109 KiB/s wr, 17 op/s 2023-09-09T23:58:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2360.066051 secs 2023-09-09T23:58:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2358.453239 secs 2023-09-09T23:58:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:48 smithi098 ceph-mon[103124]: pgmap v1427: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 109 KiB/s wr, 17 op/s 2023-09-09T23:58:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2360.066051 secs 2023-09-09T23:58:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2360.778614 secs 2023-09-09T23:58:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:50 smithi170 ceph-mon[102832]: pgmap v1428: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 57 KiB/s wr, 10 op/s 2023-09-09T23:58:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2360.778614 secs 2023-09-09T23:58:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:50 smithi082 ceph-mon[98694]: pgmap v1428: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 57 KiB/s wr, 10 op/s 2023-09-09T23:58:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2360.778614 secs 2023-09-09T23:58:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:50 smithi098 ceph-mon[103124]: pgmap v1428: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 57 KiB/s wr, 10 op/s 2023-09-09T23:58:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:52 smithi170 ceph-mon[102832]: pgmap v1429: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 72 KiB/s wr, 15 op/s 2023-09-09T23:58:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:52 smithi082 ceph-mon[98694]: pgmap v1429: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 72 KiB/s wr, 15 op/s 2023-09-09T23:58:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:52 smithi098 ceph-mon[103124]: pgmap v1429: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 72 KiB/s wr, 15 op/s 2023-09-09T23:58:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2363.453321 secs 2023-09-09T23:58:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2365.066154 secs 2023-09-09T23:58:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2363.453321 secs 2023-09-09T23:58:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2365.066154 secs 2023-09-09T23:58:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2363.453321 secs 2023-09-09T23:58:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2365.066154 secs 2023-09-09T23:58:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:54 smithi170 ceph-mon[102832]: pgmap v1430: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 52 KiB/s wr, 10 op/s 2023-09-09T23:58:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:54 smithi082 ceph-mon[98694]: pgmap v1430: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 52 KiB/s wr, 10 op/s 2023-09-09T23:58:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:54 smithi098 ceph-mon[103124]: pgmap v1430: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 52 KiB/s wr, 10 op/s 2023-09-09T23:58:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2365.778682 secs 2023-09-09T23:58:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2365.778682 secs 2023-09-09T23:58:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2365.778682 secs 2023-09-09T23:58:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:56 smithi170 ceph-mon[102832]: pgmap v1431: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 52 KiB/s wr, 10 op/s 2023-09-09T23:58:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:56 smithi082 ceph-mon[98694]: pgmap v1431: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 52 KiB/s wr, 10 op/s 2023-09-09T23:58:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:56 smithi098 ceph-mon[103124]: pgmap v1431: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 52 KiB/s wr, 10 op/s 2023-09-09T23:58:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2368.453380 secs 2023-09-09T23:58:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:58 smithi170 ceph-mon[102832]: pgmap v1432: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 15 op/s 2023-09-09T23:58:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:58:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2370.066237 secs 2023-09-09T23:58:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2368.453380 secs 2023-09-09T23:58:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:58 smithi082 ceph-mon[98694]: pgmap v1432: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 15 op/s 2023-09-09T23:58:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:58:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2370.066237 secs 2023-09-09T23:58:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2368.453380 secs 2023-09-09T23:58:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:58 smithi098 ceph-mon[103124]: pgmap v1432: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 15 op/s 2023-09-09T23:58:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:58:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2370.066237 secs 2023-09-09T23:59:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2370.778748 secs 2023-09-09T23:59:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:00 smithi170 ceph-mon[102832]: pgmap v1433: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-09T23:59:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2370.778748 secs 2023-09-09T23:59:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:00 smithi082 ceph-mon[98694]: pgmap v1433: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-09T23:59:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2370.778748 secs 2023-09-09T23:59:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:00 smithi098 ceph-mon[103124]: pgmap v1433: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-09T23:59:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:02 smithi170 ceph-mon[102832]: pgmap v1434: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 28 KiB/s wr, 14 op/s 2023-09-09T23:59:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:02 smithi082 ceph-mon[98694]: pgmap v1434: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 28 KiB/s wr, 14 op/s 2023-09-09T23:59:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:02 smithi098 ceph-mon[103124]: pgmap v1434: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 28 KiB/s wr, 14 op/s 2023-09-09T23:59:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2373.453440 secs 2023-09-09T23:59:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2375.066294 secs 2023-09-09T23:59:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2373.453440 secs 2023-09-09T23:59:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2375.066294 secs 2023-09-09T23:59:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2373.453440 secs 2023-09-09T23:59:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2375.066294 secs 2023-09-09T23:59:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:04 smithi170 ceph-mon[102832]: pgmap v1435: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-09T23:59:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:04 smithi082 ceph-mon[98694]: pgmap v1435: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-09T23:59:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:04 smithi098 ceph-mon[103124]: pgmap v1435: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-09T23:59:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2375.778817 secs 2023-09-09T23:59:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2375.778817 secs 2023-09-09T23:59:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2375.778817 secs 2023-09-09T23:59:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:06 smithi170 ceph-mon[102832]: pgmap v1436: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-09T23:59:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:06 smithi082 ceph-mon[98694]: pgmap v1436: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-09T23:59:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:06 smithi098 ceph-mon[103124]: pgmap v1436: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-09T23:59:07.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2378.453521 secs 2023-09-09T23:59:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2378.453521 secs 2023-09-09T23:59:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2378.453521 secs 2023-09-09T23:59:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:08 smithi170 ceph-mon[102832]: pgmap v1437: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-09T23:59:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:08 smithi170 ceph-mon[102832]: osdmap e163: 12 total, 12 up, 12 in 2023-09-09T23:59:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2380.066350 secs 2023-09-09T23:59:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:08 smithi082 ceph-mon[98694]: pgmap v1437: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-09T23:59:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:08 smithi082 ceph-mon[98694]: osdmap e163: 12 total, 12 up, 12 in 2023-09-09T23:59:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2380.066350 secs 2023-09-09T23:59:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:08 smithi098 ceph-mon[103124]: pgmap v1437: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-09T23:59:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:08 smithi098 ceph-mon[103124]: osdmap e163: 12 total, 12 up, 12 in 2023-09-09T23:59:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2380.066350 secs 2023-09-09T23:59:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2380.778883 secs 2023-09-09T23:59:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:10 smithi170 ceph-mon[102832]: pgmap v1439: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-09T23:59:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2380.778883 secs 2023-09-09T23:59:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:10 smithi082 ceph-mon[98694]: pgmap v1439: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-09T23:59:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2380.778883 secs 2023-09-09T23:59:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:10 smithi098 ceph-mon[103124]: pgmap v1439: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-09T23:59:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:12 smithi082 ceph-mon[98694]: pgmap v1440: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-09T23:59:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:59:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2383.453717 secs 2023-09-09T23:59:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:12 smithi098 ceph-mon[103124]: pgmap v1440: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-09T23:59:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:59:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2383.453717 secs 2023-09-09T23:59:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:12 smithi170 ceph-mon[102832]: pgmap v1440: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-09T23:59:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-09T23:59:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2383.453717 secs 2023-09-09T23:59:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:59:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:59:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:59:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2385.066406 secs 2023-09-09T23:59:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:59:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:59:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:59:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2385.066406 secs 2023-09-09T23:59:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-09T23:59:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-09T23:59:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-09T23:59:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2385.066406 secs 2023-09-09T23:59:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:14 smithi098 ceph-mon[103124]: pgmap v1441: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-09T23:59:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2385.778948 secs 2023-09-09T23:59:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:14 smithi098 ceph-mon[103124]: osdmap e164: 12 total, 12 up, 12 in 2023-09-09T23:59:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:14 smithi170 ceph-mon[102832]: pgmap v1441: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-09T23:59:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2385.778948 secs 2023-09-09T23:59:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:14 smithi170 ceph-mon[102832]: osdmap e164: 12 total, 12 up, 12 in 2023-09-09T23:59:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:14 smithi082 ceph-mon[98694]: pgmap v1441: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-09T23:59:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2385.778948 secs 2023-09-09T23:59:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:14 smithi082 ceph-mon[98694]: osdmap e164: 12 total, 12 up, 12 in 2023-09-09T23:59:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:16 smithi098 ceph-mon[103124]: pgmap v1443: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2023-09-09T23:59:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:16 smithi170 ceph-mon[102832]: pgmap v1443: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2023-09-09T23:59:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:16 smithi082 ceph-mon[98694]: pgmap v1443: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2023-09-09T23:59:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:18 smithi098 ceph-mon[103124]: pgmap v1444: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-09T23:59:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2390.066464 secs 2023-09-09T23:59:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:18 smithi170 ceph-mon[102832]: pgmap v1444: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-09T23:59:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2390.066464 secs 2023-09-09T23:59:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:18 smithi082 ceph-mon[98694]: pgmap v1444: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-09T23:59:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2390.066464 secs 2023-09-09T23:59:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2390.779037 secs 2023-09-09T23:59:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:59:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2390.779037 secs 2023-09-09T23:59:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:59:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2390.779037 secs 2023-09-09T23:59:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-09T23:59:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:20 smithi098 ceph-mon[103124]: pgmap v1445: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-09T23:59:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:59:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:59:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:59:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:20 smithi170 ceph-mon[102832]: pgmap v1445: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-09T23:59:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:59:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:59:20.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:59:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:20 smithi082 ceph-mon[98694]: pgmap v1445: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-09T23:59:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:59:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:59:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:59:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:22 smithi098 ceph-mon[103124]: pgmap v1446: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-09T23:59:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:22 smithi170 ceph-mon[102832]: pgmap v1446: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-09T23:59:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:22 smithi082 ceph-mon[98694]: pgmap v1446: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-09T23:59:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2395.066493 secs 2023-09-09T23:59:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2395.066493 secs 2023-09-09T23:59:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2395.066493 secs 2023-09-09T23:59:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:24 smithi098 ceph-mon[103124]: pgmap v1447: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-09T23:59:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2395.779104 secs 2023-09-09T23:59:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:24 smithi170 ceph-mon[102832]: pgmap v1447: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-09T23:59:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2395.779104 secs 2023-09-09T23:59:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:24 smithi082 ceph-mon[98694]: pgmap v1447: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-09T23:59:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2395.779104 secs 2023-09-09T23:59:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:26 smithi098 ceph-mon[103124]: pgmap v1448: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 10 op/s 2023-09-09T23:59:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:26 smithi170 ceph-mon[102832]: pgmap v1448: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 10 op/s 2023-09-09T23:59:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:26 smithi082 ceph-mon[98694]: pgmap v1448: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 10 op/s 2023-09-09T23:59:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:28 smithi098 ceph-mon[103124]: pgmap v1449: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 54 KiB/s wr, 15 op/s 2023-09-09T23:59:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2400.066550 secs 2023-09-09T23:59:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:28 smithi170 ceph-mon[102832]: pgmap v1449: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 54 KiB/s wr, 15 op/s 2023-09-09T23:59:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2400.066550 secs 2023-09-09T23:59:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:28 smithi082 ceph-mon[98694]: pgmap v1449: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 54 KiB/s wr, 15 op/s 2023-09-09T23:59:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2400.066550 secs 2023-09-09T23:59:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2400.779167 secs 2023-09-09T23:59:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:30 smithi170 ceph-mon[102832]: pgmap v1450: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-09-09T23:59:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2400.779167 secs 2023-09-09T23:59:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:30 smithi082 ceph-mon[98694]: pgmap v1450: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-09-09T23:59:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2400.779167 secs 2023-09-09T23:59:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:30 smithi098 ceph-mon[103124]: pgmap v1450: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-09-09T23:59:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:32 smithi170 ceph-mon[102832]: pgmap v1451: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 87 KiB/s wr, 17 op/s 2023-09-09T23:59:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:32 smithi082 ceph-mon[98694]: pgmap v1451: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 87 KiB/s wr, 17 op/s 2023-09-09T23:59:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:32 smithi098 ceph-mon[103124]: pgmap v1451: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 87 KiB/s wr, 17 op/s 2023-09-09T23:59:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2405.066610 secs 2023-09-09T23:59:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:34 smithi170 ceph-mon[102832]: pgmap v1452: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-09T23:59:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2405.066610 secs 2023-09-09T23:59:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:34 smithi082 ceph-mon[98694]: pgmap v1452: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-09T23:59:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2405.066610 secs 2023-09-09T23:59:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:34 smithi098 ceph-mon[103124]: pgmap v1452: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-09T23:59:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2405.779234 secs 2023-09-09T23:59:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2405.779234 secs 2023-09-09T23:59:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2405.779234 secs 2023-09-09T23:59:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:36 smithi170 ceph-mon[102832]: pgmap v1453: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-09T23:59:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:36 smithi082 ceph-mon[98694]: pgmap v1453: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-09T23:59:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:36 smithi098 ceph-mon[103124]: pgmap v1453: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-09T23:59:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:38 smithi170 ceph-mon[102832]: pgmap v1454: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 105 KiB/s wr, 18 op/s 2023-09-09T23:59:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:38 smithi082 ceph-mon[98694]: pgmap v1454: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 105 KiB/s wr, 18 op/s 2023-09-09T23:59:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:38 smithi098 ceph-mon[103124]: pgmap v1454: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 105 KiB/s wr, 18 op/s 2023-09-09T23:59:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2410.066670 secs 2023-09-09T23:59:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2410.066670 secs 2023-09-09T23:59:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2410.066670 secs 2023-09-09T23:59:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2410.779300 secs 2023-09-09T23:59:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:40 smithi170 ceph-mon[102832]: pgmap v1455: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 12 op/s 2023-09-09T23:59:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2410.779300 secs 2023-09-09T23:59:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:40 smithi082 ceph-mon[98694]: pgmap v1455: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 12 op/s 2023-09-09T23:59:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2410.779300 secs 2023-09-09T23:59:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:40 smithi098 ceph-mon[103124]: pgmap v1455: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 12 op/s 2023-09-09T23:59:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:42 smithi170 ceph-mon[102832]: pgmap v1456: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 144 KiB/s wr, 18 op/s 2023-09-09T23:59:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:42 smithi082 ceph-mon[98694]: pgmap v1456: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 144 KiB/s wr, 18 op/s 2023-09-09T23:59:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:42 smithi098 ceph-mon[103124]: pgmap v1456: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 144 KiB/s wr, 18 op/s 2023-09-09T23:59:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2415.066727 secs 2023-09-09T23:59:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:44 smithi170 ceph-mon[102832]: pgmap v1457: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-09T23:59:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2415.066727 secs 2023-09-09T23:59:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:44 smithi082 ceph-mon[98694]: pgmap v1457: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-09T23:59:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2415.066727 secs 2023-09-09T23:59:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:44 smithi098 ceph-mon[103124]: pgmap v1457: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-09T23:59:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2415.779388 secs 2023-09-09T23:59:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2415.779388 secs 2023-09-09T23:59:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2415.779388 secs 2023-09-09T23:59:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:46 smithi170 ceph-mon[102832]: pgmap v1458: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-09T23:59:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:46 smithi082 ceph-mon[98694]: pgmap v1458: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-09T23:59:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:46 smithi098 ceph-mon[103124]: pgmap v1458: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-09T23:59:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2418.454261 secs 2023-09-09T23:59:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:48 smithi170 ceph-mon[102832]: pgmap v1459: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 116 KiB/s wr, 17 op/s 2023-09-09T23:59:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2418.454261 secs 2023-09-09T23:59:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:48 smithi082 ceph-mon[98694]: pgmap v1459: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 116 KiB/s wr, 17 op/s 2023-09-09T23:59:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2418.454261 secs 2023-09-09T23:59:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:48 smithi098 ceph-mon[103124]: pgmap v1459: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 116 KiB/s wr, 17 op/s 2023-09-09T23:59:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2420.066815 secs 2023-09-09T23:59:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2420.066815 secs 2023-09-09T23:59:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2420.066815 secs 2023-09-09T23:59:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2420.779455 secs 2023-09-09T23:59:50.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:50 smithi170 ceph-mon[102832]: pgmap v1460: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-09T23:59:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2420.779455 secs 2023-09-09T23:59:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:50 smithi082 ceph-mon[98694]: pgmap v1460: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-09T23:59:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2420.779455 secs 2023-09-09T23:59:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:50 smithi098 ceph-mon[103124]: pgmap v1460: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-09T23:59:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:59:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:59:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:59:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:59:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:59:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:59:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-09T23:59:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-09T23:59:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-09T23:59:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:52 smithi170 ceph-mon[102832]: pgmap v1461: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 115 KiB/s wr, 17 op/s 2023-09-09T23:59:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:52 smithi082 ceph-mon[98694]: pgmap v1461: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 115 KiB/s wr, 17 op/s 2023-09-09T23:59:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:52 smithi098 ceph-mon[103124]: pgmap v1461: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 115 KiB/s wr, 17 op/s 2023-09-09T23:59:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2423.454320 secs 2023-09-09T23:59:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2425.066873 secs 2023-09-09T23:59:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2423.454320 secs 2023-09-09T23:59:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2425.066873 secs 2023-09-09T23:59:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2423.454320 secs 2023-09-09T23:59:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2425.066873 secs 2023-09-09T23:59:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:54 smithi170 ceph-mon[102832]: pgmap v1462: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-09T23:59:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:54 smithi082 ceph-mon[98694]: pgmap v1462: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-09T23:59:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:54 smithi098 ceph-mon[103124]: pgmap v1462: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-09T23:59:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2425.779498 secs 2023-09-09T23:59:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2425.779498 secs 2023-09-09T23:59:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2425.779498 secs 2023-09-09T23:59:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:56 smithi170 ceph-mon[102832]: pgmap v1463: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-09T23:59:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:56 smithi082 ceph-mon[98694]: pgmap v1463: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-09T23:59:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:56 smithi098 ceph-mon[103124]: pgmap v1463: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-09T23:59:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2428.454381 secs 2023-09-09T23:59:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2428.454381 secs 2023-09-09T23:59:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2428.454381 secs 2023-09-09T23:59:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:58 smithi170 ceph-mon[102832]: pgmap v1464: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 56 KiB/s wr, 16 op/s 2023-09-09T23:59:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 09 23:59:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2430.066930 secs 2023-09-09T23:59:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:58 smithi082 ceph-mon[98694]: pgmap v1464: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 56 KiB/s wr, 16 op/s 2023-09-09T23:59:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 09 23:59:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2430.066930 secs 2023-09-09T23:59:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:58 smithi098 ceph-mon[103124]: pgmap v1464: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 56 KiB/s wr, 16 op/s 2023-09-09T23:59:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 09 23:59:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2430.066930 secs 2023-09-10T00:00:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2430.779566 secs 2023-09-10T00:00:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:00 smithi082 ceph-mon[98694]: pgmap v1465: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 10 op/s 2023-09-10T00:00:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T00:00:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2430.779566 secs 2023-09-10T00:00:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:00 smithi098 ceph-mon[103124]: pgmap v1465: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 10 op/s 2023-09-10T00:00:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T00:00:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2430.779566 secs 2023-09-10T00:00:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:00 smithi170 ceph-mon[102832]: pgmap v1465: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 10 op/s 2023-09-10T00:00:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T00:00:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:02 smithi082 ceph-mon[98694]: pgmap v1466: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 15 op/s 2023-09-10T00:00:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:02 smithi098 ceph-mon[103124]: pgmap v1466: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 15 op/s 2023-09-10T00:00:02.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:02 smithi170 ceph-mon[102832]: pgmap v1466: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 15 op/s 2023-09-10T00:00:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:03 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:00:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2435.066989 secs 2023-09-10T00:00:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:03 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:00:03.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2435.066989 secs 2023-09-10T00:00:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:03 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:00:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2435.066989 secs 2023-09-10T00:00:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:04 smithi082 ceph-mon[98694]: pgmap v1467: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-10T00:00:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:04 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:00:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:00:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:00:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:04 smithi098 ceph-mon[103124]: pgmap v1467: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-10T00:00:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:04 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:00:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:00:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:00:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:04 smithi170 ceph-mon[102832]: pgmap v1467: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-10T00:00:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:04 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:00:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:00:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:00:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2435.779630 secs 2023-09-10T00:00:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2435.779630 secs 2023-09-10T00:00:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2435.779630 secs 2023-09-10T00:00:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:06 smithi082 ceph-mon[98694]: pgmap v1468: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-10T00:00:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:06 smithi098 ceph-mon[103124]: pgmap v1468: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-10T00:00:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:06 smithi170 ceph-mon[102832]: pgmap v1468: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-10T00:00:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2438.454573 secs 2023-09-10T00:00:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2438.454573 secs 2023-09-10T00:00:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2438.454573 secs 2023-09-10T00:00:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:08 smithi082 ceph-mon[98694]: pgmap v1469: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T00:00:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:08 smithi082 ceph-mon[98694]: osdmap e165: 12 total, 12 up, 12 in 2023-09-10T00:00:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2440.067049 secs 2023-09-10T00:00:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:08 smithi098 ceph-mon[103124]: pgmap v1469: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T00:00:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:08 smithi098 ceph-mon[103124]: osdmap e165: 12 total, 12 up, 12 in 2023-09-10T00:00:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2440.067049 secs 2023-09-10T00:00:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:08 smithi170 ceph-mon[102832]: pgmap v1469: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T00:00:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:08 smithi170 ceph-mon[102832]: osdmap e165: 12 total, 12 up, 12 in 2023-09-10T00:00:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2440.067049 secs 2023-09-10T00:00:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2440.779696 secs 2023-09-10T00:00:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:10 smithi170 ceph-mon[102832]: pgmap v1471: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:00:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2440.779696 secs 2023-09-10T00:00:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:10 smithi082 ceph-mon[98694]: pgmap v1471: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:00:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2440.779696 secs 2023-09-10T00:00:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:10 smithi098 ceph-mon[103124]: pgmap v1471: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:00:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:11 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:00:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:11 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:00:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:11 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:00:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:11 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:00:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:11 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:00:11.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:11 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:00:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:11 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:00:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:11 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:00:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:11 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:00:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:12 smithi170 ceph-mon[102832]: pgmap v1472: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 109 KiB/s wr, 30 op/s 2023-09-10T00:00:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:12 smithi082 ceph-mon[98694]: pgmap v1472: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 109 KiB/s wr, 30 op/s 2023-09-10T00:00:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:12 smithi098 ceph-mon[103124]: pgmap v1472: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 109 KiB/s wr, 30 op/s 2023-09-10T00:00:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2443.454737 secs 2023-09-10T00:00:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:00:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2445.067117 secs 2023-09-10T00:00:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2443.454737 secs 2023-09-10T00:00:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:00:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2445.067117 secs 2023-09-10T00:00:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2443.454737 secs 2023-09-10T00:00:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:00:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2445.067117 secs 2023-09-10T00:00:14.391 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:14 smithi082 ceph-mon[98694]: pgmap v1473: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 109 KiB/s wr, 30 op/s 2023-09-10T00:00:14.391 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:00:14.391 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:00:14.392 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:00:14.392 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:14 smithi082 ceph-mon[98694]: osdmap e166: 12 total, 12 up, 12 in 2023-09-10T00:00:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:14 smithi170 ceph-mon[102832]: pgmap v1473: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 109 KiB/s wr, 30 op/s 2023-09-10T00:00:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:00:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:00:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:00:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:14 smithi170 ceph-mon[102832]: osdmap e166: 12 total, 12 up, 12 in 2023-09-10T00:00:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:14 smithi098 ceph-mon[103124]: pgmap v1473: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 109 KiB/s wr, 30 op/s 2023-09-10T00:00:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:00:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:00:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:00:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:14 smithi098 ceph-mon[103124]: osdmap e166: 12 total, 12 up, 12 in 2023-09-10T00:00:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2445.779760 secs 2023-09-10T00:00:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2445.779760 secs 2023-09-10T00:00:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2445.779760 secs 2023-09-10T00:00:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:16 smithi170 ceph-mon[102832]: pgmap v1475: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 32 KiB/s wr, 8 op/s 2023-09-10T00:00:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:16 smithi082 ceph-mon[98694]: pgmap v1475: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 32 KiB/s wr, 8 op/s 2023-09-10T00:00:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:16 smithi098 ceph-mon[103124]: pgmap v1475: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 32 KiB/s wr, 8 op/s 2023-09-10T00:00:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:18 smithi170 ceph-mon[102832]: pgmap v1476: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 13 op/s 2023-09-10T00:00:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2450.067196 secs 2023-09-10T00:00:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:18 smithi082 ceph-mon[98694]: pgmap v1476: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 13 op/s 2023-09-10T00:00:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2450.067196 secs 2023-09-10T00:00:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:18 smithi098 ceph-mon[103124]: pgmap v1476: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 13 op/s 2023-09-10T00:00:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2450.067196 secs 2023-09-10T00:00:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:00:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:00:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:00:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2450.779825 secs 2023-09-10T00:00:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:20 smithi170 ceph-mon[102832]: pgmap v1477: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 12 op/s 2023-09-10T00:00:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:00:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:00:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:00:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2450.779825 secs 2023-09-10T00:00:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:20 smithi082 ceph-mon[98694]: pgmap v1477: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 12 op/s 2023-09-10T00:00:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:00:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:00:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:00:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2450.779825 secs 2023-09-10T00:00:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:20 smithi098 ceph-mon[103124]: pgmap v1477: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 12 op/s 2023-09-10T00:00:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:00:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:00:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:00:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:22 smithi170 ceph-mon[102832]: pgmap v1478: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T00:00:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:22 smithi082 ceph-mon[98694]: pgmap v1478: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T00:00:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:22 smithi098 ceph-mon[103124]: pgmap v1478: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T00:00:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2455.067254 secs 2023-09-10T00:00:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2455.067254 secs 2023-09-10T00:00:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2455.067254 secs 2023-09-10T00:00:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:24 smithi170 ceph-mon[102832]: pgmap v1479: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T00:00:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:24 smithi082 ceph-mon[98694]: pgmap v1479: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T00:00:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:24 smithi098 ceph-mon[103124]: pgmap v1479: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T00:00:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2455.779891 secs 2023-09-10T00:00:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2455.779891 secs 2023-09-10T00:00:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2455.779891 secs 2023-09-10T00:00:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:26 smithi170 ceph-mon[102832]: pgmap v1480: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T00:00:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:26 smithi082 ceph-mon[98694]: pgmap v1480: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T00:00:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:26 smithi098 ceph-mon[103124]: pgmap v1480: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T00:00:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:28 smithi170 ceph-mon[102832]: pgmap v1481: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 106 KiB/s wr, 16 op/s 2023-09-10T00:00:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2460.067307 secs 2023-09-10T00:00:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:28 smithi082 ceph-mon[98694]: pgmap v1481: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 106 KiB/s wr, 16 op/s 2023-09-10T00:00:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2460.067307 secs 2023-09-10T00:00:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:28 smithi098 ceph-mon[103124]: pgmap v1481: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 106 KiB/s wr, 16 op/s 2023-09-10T00:00:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2460.067307 secs 2023-09-10T00:00:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2460.779956 secs 2023-09-10T00:00:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:30 smithi170 ceph-mon[102832]: pgmap v1482: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T00:00:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2460.779956 secs 2023-09-10T00:00:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:30 smithi082 ceph-mon[98694]: pgmap v1482: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T00:00:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2460.779956 secs 2023-09-10T00:00:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:30 smithi098 ceph-mon[103124]: pgmap v1482: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T00:00:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:32 smithi170 ceph-mon[102832]: pgmap v1483: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 171 KiB/s wr, 17 op/s 2023-09-10T00:00:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:32 smithi082 ceph-mon[98694]: pgmap v1483: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 171 KiB/s wr, 17 op/s 2023-09-10T00:00:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:32 smithi098 ceph-mon[103124]: pgmap v1483: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 171 KiB/s wr, 17 op/s 2023-09-10T00:00:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2465.067366 secs 2023-09-10T00:00:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2465.067366 secs 2023-09-10T00:00:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2465.067366 secs 2023-09-10T00:00:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:34 smithi170 ceph-mon[102832]: pgmap v1484: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T00:00:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:34 smithi082 ceph-mon[98694]: pgmap v1484: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T00:00:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:34 smithi098 ceph-mon[103124]: pgmap v1484: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T00:00:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2465.780021 secs 2023-09-10T00:00:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2465.780021 secs 2023-09-10T00:00:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2465.780021 secs 2023-09-10T00:00:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:36 smithi170 ceph-mon[102832]: pgmap v1485: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T00:00:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:36 smithi082 ceph-mon[98694]: pgmap v1485: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T00:00:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:36 smithi098 ceph-mon[103124]: pgmap v1485: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T00:00:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:38 smithi170 ceph-mon[102832]: pgmap v1486: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 129 KiB/s wr, 18 op/s 2023-09-10T00:00:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2470.067423 secs 2023-09-10T00:00:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:38 smithi082 ceph-mon[98694]: pgmap v1486: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 129 KiB/s wr, 18 op/s 2023-09-10T00:00:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2470.067423 secs 2023-09-10T00:00:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:38 smithi098 ceph-mon[103124]: pgmap v1486: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 129 KiB/s wr, 18 op/s 2023-09-10T00:00:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2470.067423 secs 2023-09-10T00:00:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2470.780086 secs 2023-09-10T00:00:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:40 smithi170 ceph-mon[102832]: pgmap v1487: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 108 KiB/s wr, 11 op/s 2023-09-10T00:00:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2470.780086 secs 2023-09-10T00:00:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:40 smithi082 ceph-mon[98694]: pgmap v1487: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 108 KiB/s wr, 11 op/s 2023-09-10T00:00:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2470.780086 secs 2023-09-10T00:00:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:40 smithi098 ceph-mon[103124]: pgmap v1487: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 108 KiB/s wr, 11 op/s 2023-09-10T00:00:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:42 smithi170 ceph-mon[102832]: pgmap v1488: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 141 KiB/s wr, 17 op/s 2023-09-10T00:00:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2473.455169 secs 2023-09-10T00:00:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:42 smithi082 ceph-mon[98694]: pgmap v1488: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 141 KiB/s wr, 17 op/s 2023-09-10T00:00:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2473.455169 secs 2023-09-10T00:00:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:42 smithi098 ceph-mon[103124]: pgmap v1488: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 141 KiB/s wr, 17 op/s 2023-09-10T00:00:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2473.455169 secs 2023-09-10T00:00:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2475.067510 secs 2023-09-10T00:00:43.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2475.067510 secs 2023-09-10T00:00:43.661 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2475.067510 secs 2023-09-10T00:00:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:44 smithi170 ceph-mon[102832]: pgmap v1489: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T00:00:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:00:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:00:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:00:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:44 smithi082 ceph-mon[98694]: pgmap v1489: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T00:00:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:00:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:00:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:00:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:44 smithi098 ceph-mon[103124]: pgmap v1489: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T00:00:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:00:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:00:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:00:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2475.780153 secs 2023-09-10T00:00:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2475.780153 secs 2023-09-10T00:00:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2475.780153 secs 2023-09-10T00:00:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:46 smithi082 ceph-mon[98694]: pgmap v1490: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T00:00:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:46 smithi098 ceph-mon[103124]: pgmap v1490: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T00:00:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:46 smithi170 ceph-mon[102832]: pgmap v1490: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T00:00:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2478.455256 secs 2023-09-10T00:00:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2478.455256 secs 2023-09-10T00:00:47.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2478.455256 secs 2023-09-10T00:00:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:48 smithi082 ceph-mon[98694]: pgmap v1491: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 82 KiB/s wr, 17 op/s 2023-09-10T00:00:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2480.067568 secs 2023-09-10T00:00:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:48 smithi098 ceph-mon[103124]: pgmap v1491: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 82 KiB/s wr, 17 op/s 2023-09-10T00:00:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2480.067568 secs 2023-09-10T00:00:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:48 smithi170 ceph-mon[102832]: pgmap v1491: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 82 KiB/s wr, 17 op/s 2023-09-10T00:00:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2480.067568 secs 2023-09-10T00:00:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2480.780216 secs 2023-09-10T00:00:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:50 smithi082 ceph-mon[98694]: pgmap v1492: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T00:00:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2480.780216 secs 2023-09-10T00:00:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:50 smithi098 ceph-mon[103124]: pgmap v1492: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T00:00:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2480.780216 secs 2023-09-10T00:00:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:50 smithi170 ceph-mon[102832]: pgmap v1492: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T00:00:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:52 smithi082 ceph-mon[98694]: pgmap v1493: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 49 KiB/s wr, 16 op/s 2023-09-10T00:00:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2483.455316 secs 2023-09-10T00:00:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:52 smithi098 ceph-mon[103124]: pgmap v1493: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 49 KiB/s wr, 16 op/s 2023-09-10T00:00:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2483.455316 secs 2023-09-10T00:00:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:52 smithi170 ceph-mon[102832]: pgmap v1493: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 49 KiB/s wr, 16 op/s 2023-09-10T00:00:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2483.455316 secs 2023-09-10T00:00:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2485.067656 secs 2023-09-10T00:00:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2485.067656 secs 2023-09-10T00:00:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2485.067656 secs 2023-09-10T00:00:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:54 smithi082 ceph-mon[98694]: pgmap v1494: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-10T00:00:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2485.780281 secs 2023-09-10T00:00:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:54 smithi098 ceph-mon[103124]: pgmap v1494: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-10T00:00:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2485.780281 secs 2023-09-10T00:00:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:54 smithi170 ceph-mon[102832]: pgmap v1494: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-10T00:00:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2485.780281 secs 2023-09-10T00:00:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:56 smithi082 ceph-mon[98694]: pgmap v1495: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-10T00:00:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:56 smithi098 ceph-mon[103124]: pgmap v1495: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-10T00:00:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:56 smithi170 ceph-mon[102832]: pgmap v1495: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 KiB/s wr, 10 op/s 2023-09-10T00:00:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2488.455376 secs 2023-09-10T00:00:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2488.455376 secs 2023-09-10T00:00:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2488.455376 secs 2023-09-10T00:00:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:58 smithi098 ceph-mon[103124]: pgmap v1496: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 34 KiB/s wr, 16 op/s 2023-09-10T00:00:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:00:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2490.067713 secs 2023-09-10T00:00:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:58 smithi170 ceph-mon[102832]: pgmap v1496: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 34 KiB/s wr, 16 op/s 2023-09-10T00:00:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:00:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2490.067713 secs 2023-09-10T00:00:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:58 smithi082 ceph-mon[98694]: pgmap v1496: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 34 KiB/s wr, 16 op/s 2023-09-10T00:00:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:00:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2490.067713 secs 2023-09-10T00:01:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2490.780346 secs 2023-09-10T00:01:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:00 smithi170 ceph-mon[102832]: pgmap v1497: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2023-09-10T00:01:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2490.780346 secs 2023-09-10T00:01:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:00 smithi082 ceph-mon[98694]: pgmap v1497: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2023-09-10T00:01:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2490.780346 secs 2023-09-10T00:01:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:00 smithi098 ceph-mon[103124]: pgmap v1497: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2023-09-10T00:01:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:02 smithi098 ceph-mon[103124]: pgmap v1498: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 36 KiB/s wr, 15 op/s 2023-09-10T00:01:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:02 smithi170 ceph-mon[102832]: pgmap v1498: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 36 KiB/s wr, 15 op/s 2023-09-10T00:01:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:02 smithi082 ceph-mon[98694]: pgmap v1498: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 36 KiB/s wr, 15 op/s 2023-09-10T00:01:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2493.455481 secs 2023-09-10T00:01:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2493.455481 secs 2023-09-10T00:01:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2493.455481 secs 2023-09-10T00:01:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2495.067771 secs 2023-09-10T00:01:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:04 smithi170 ceph-mon[102832]: pgmap v1499: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-09-10T00:01:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2495.067771 secs 2023-09-10T00:01:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:04 smithi082 ceph-mon[98694]: pgmap v1499: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-09-10T00:01:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2495.067771 secs 2023-09-10T00:01:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:04 smithi098 ceph-mon[103124]: pgmap v1499: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-09-10T00:01:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2495.780410 secs 2023-09-10T00:01:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2495.780410 secs 2023-09-10T00:01:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2495.780410 secs 2023-09-10T00:01:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:06 smithi170 ceph-mon[102832]: pgmap v1500: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-09-10T00:01:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:06 smithi082 ceph-mon[98694]: pgmap v1500: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-09-10T00:01:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:06 smithi098 ceph-mon[103124]: pgmap v1500: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-09-10T00:01:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2498.455581 secs 2023-09-10T00:01:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:08 smithi170 ceph-mon[102832]: pgmap v1501: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 87 KiB/s wr, 30 op/s 2023-09-10T00:01:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:08 smithi170 ceph-mon[102832]: osdmap e167: 12 total, 12 up, 12 in 2023-09-10T00:01:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2498.455581 secs 2023-09-10T00:01:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:08 smithi082 ceph-mon[98694]: pgmap v1501: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 87 KiB/s wr, 30 op/s 2023-09-10T00:01:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:08 smithi082 ceph-mon[98694]: osdmap e167: 12 total, 12 up, 12 in 2023-09-10T00:01:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2498.455581 secs 2023-09-10T00:01:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:08 smithi098 ceph-mon[103124]: pgmap v1501: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 87 KiB/s wr, 30 op/s 2023-09-10T00:01:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:08 smithi098 ceph-mon[103124]: osdmap e167: 12 total, 12 up, 12 in 2023-09-10T00:01:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2500.067830 secs 2023-09-10T00:01:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2500.067830 secs 2023-09-10T00:01:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2500.067830 secs 2023-09-10T00:01:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2500.780480 secs 2023-09-10T00:01:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:10 smithi170 ceph-mon[102832]: pgmap v1503: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T00:01:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2500.780480 secs 2023-09-10T00:01:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:10 smithi082 ceph-mon[98694]: pgmap v1503: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T00:01:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2500.780480 secs 2023-09-10T00:01:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:10 smithi098 ceph-mon[103124]: pgmap v1503: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T00:01:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:12 smithi170 ceph-mon[102832]: pgmap v1504: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T00:01:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:12 smithi082 ceph-mon[98694]: pgmap v1504: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T00:01:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:12 smithi098 ceph-mon[103124]: pgmap v1504: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T00:01:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2503.455697 secs 2023-09-10T00:01:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2505.067885 secs 2023-09-10T00:01:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2503.455697 secs 2023-09-10T00:01:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2505.067885 secs 2023-09-10T00:01:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2503.455697 secs 2023-09-10T00:01:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2505.067885 secs 2023-09-10T00:01:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:14 smithi170 ceph-mon[102832]: pgmap v1505: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T00:01:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:14 smithi170 ceph-mon[102832]: osdmap e168: 12 total, 12 up, 12 in 2023-09-10T00:01:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:14 smithi082 ceph-mon[98694]: pgmap v1505: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T00:01:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:14 smithi082 ceph-mon[98694]: osdmap e168: 12 total, 12 up, 12 in 2023-09-10T00:01:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:14 smithi098 ceph-mon[103124]: pgmap v1505: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T00:01:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:14 smithi098 ceph-mon[103124]: osdmap e168: 12 total, 12 up, 12 in 2023-09-10T00:01:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2505.780545 secs 2023-09-10T00:01:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:01:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:01:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:01:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:01:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2505.780545 secs 2023-09-10T00:01:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:01:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:01:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:01:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:01:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2505.780545 secs 2023-09-10T00:01:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:01:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:01:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:01:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:01:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:16 smithi170 ceph-mon[102832]: pgmap v1507: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 27 KiB/s wr, 7 op/s 2023-09-10T00:01:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:16 smithi082 ceph-mon[98694]: pgmap v1507: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 27 KiB/s wr, 7 op/s 2023-09-10T00:01:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:16 smithi098 ceph-mon[103124]: pgmap v1507: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 27 KiB/s wr, 7 op/s 2023-09-10T00:01:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:18 smithi170 ceph-mon[102832]: pgmap v1508: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T00:01:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2510.067938 secs 2023-09-10T00:01:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:18 smithi082 ceph-mon[98694]: pgmap v1508: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T00:01:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2510.067938 secs 2023-09-10T00:01:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:18 smithi098 ceph-mon[103124]: pgmap v1508: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T00:01:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2510.067938 secs 2023-09-10T00:01:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:01:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:01:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:01:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2510.780610 secs 2023-09-10T00:01:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:20 smithi170 ceph-mon[102832]: pgmap v1509: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T00:01:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:01:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:01:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:01:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2510.780610 secs 2023-09-10T00:01:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:20 smithi082 ceph-mon[98694]: pgmap v1509: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T00:01:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:01:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:01:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:01:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2510.780610 secs 2023-09-10T00:01:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:20 smithi098 ceph-mon[103124]: pgmap v1509: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T00:01:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:01:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:01:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:01:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:22 smithi170 ceph-mon[102832]: pgmap v1510: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 33 KiB/s wr, 11 op/s 2023-09-10T00:01:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:22 smithi082 ceph-mon[98694]: pgmap v1510: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 33 KiB/s wr, 11 op/s 2023-09-10T00:01:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:22 smithi098 ceph-mon[103124]: pgmap v1510: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 33 KiB/s wr, 11 op/s 2023-09-10T00:01:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2515.067996 secs 2023-09-10T00:01:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2515.067996 secs 2023-09-10T00:01:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2515.067996 secs 2023-09-10T00:01:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:24 smithi170 ceph-mon[102832]: pgmap v1511: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 33 KiB/s wr, 11 op/s 2023-09-10T00:01:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:24 smithi082 ceph-mon[98694]: pgmap v1511: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 33 KiB/s wr, 11 op/s 2023-09-10T00:01:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:24 smithi098 ceph-mon[103124]: pgmap v1511: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 33 KiB/s wr, 11 op/s 2023-09-10T00:01:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2515.780678 secs 2023-09-10T00:01:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2515.780678 secs 2023-09-10T00:01:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2515.780678 secs 2023-09-10T00:01:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:26 smithi170 ceph-mon[102832]: pgmap v1512: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 31 KiB/s wr, 10 op/s 2023-09-10T00:01:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:26 smithi082 ceph-mon[98694]: pgmap v1512: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 31 KiB/s wr, 10 op/s 2023-09-10T00:01:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:26 smithi098 ceph-mon[103124]: pgmap v1512: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 31 KiB/s wr, 10 op/s 2023-09-10T00:01:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:28 smithi170 ceph-mon[102832]: pgmap v1513: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 89 KiB/s wr, 16 op/s 2023-09-10T00:01:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2520.068054 secs 2023-09-10T00:01:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:28 smithi082 ceph-mon[98694]: pgmap v1513: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 89 KiB/s wr, 16 op/s 2023-09-10T00:01:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2520.068054 secs 2023-09-10T00:01:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:28 smithi098 ceph-mon[103124]: pgmap v1513: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 89 KiB/s wr, 16 op/s 2023-09-10T00:01:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2520.068054 secs 2023-09-10T00:01:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2520.780742 secs 2023-09-10T00:01:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:30 smithi170 ceph-mon[102832]: pgmap v1514: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T00:01:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2520.780742 secs 2023-09-10T00:01:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:30 smithi082 ceph-mon[98694]: pgmap v1514: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T00:01:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2520.780742 secs 2023-09-10T00:01:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:30 smithi098 ceph-mon[103124]: pgmap v1514: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T00:01:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:32 smithi170 ceph-mon[102832]: pgmap v1515: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 148 KiB/s wr, 32 op/s 2023-09-10T00:01:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:32 smithi082 ceph-mon[98694]: pgmap v1515: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 148 KiB/s wr, 32 op/s 2023-09-10T00:01:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:32 smithi098 ceph-mon[103124]: pgmap v1515: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 148 KiB/s wr, 32 op/s 2023-09-10T00:01:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2525.068111 secs 2023-09-10T00:01:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2525.068111 secs 2023-09-10T00:01:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2525.068111 secs 2023-09-10T00:01:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:34 smithi170 ceph-mon[102832]: pgmap v1516: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 120 KiB/s wr, 27 op/s 2023-09-10T00:01:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:34 smithi082 ceph-mon[98694]: pgmap v1516: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 120 KiB/s wr, 27 op/s 2023-09-10T00:01:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:34 smithi098 ceph-mon[103124]: pgmap v1516: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 120 KiB/s wr, 27 op/s 2023-09-10T00:01:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2525.780808 secs 2023-09-10T00:01:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2525.780808 secs 2023-09-10T00:01:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2525.780808 secs 2023-09-10T00:01:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:36 smithi170 ceph-mon[102832]: pgmap v1517: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 120 KiB/s wr, 27 op/s 2023-09-10T00:01:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:36 smithi082 ceph-mon[98694]: pgmap v1517: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 120 KiB/s wr, 27 op/s 2023-09-10T00:01:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:36 smithi098 ceph-mon[103124]: pgmap v1517: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 120 KiB/s wr, 27 op/s 2023-09-10T00:01:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:38 smithi170 ceph-mon[102832]: pgmap v1518: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 138 KiB/s wr, 33 op/s 2023-09-10T00:01:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2530.068171 secs 2023-09-10T00:01:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:38 smithi082 ceph-mon[98694]: pgmap v1518: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 138 KiB/s wr, 33 op/s 2023-09-10T00:01:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2530.068171 secs 2023-09-10T00:01:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:38 smithi098 ceph-mon[103124]: pgmap v1518: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 138 KiB/s wr, 33 op/s 2023-09-10T00:01:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2530.068171 secs 2023-09-10T00:01:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2530.780872 secs 2023-09-10T00:01:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:40 smithi170 ceph-mon[102832]: pgmap v1519: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 77 KiB/s wr, 25 op/s 2023-09-10T00:01:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2530.780872 secs 2023-09-10T00:01:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:40 smithi082 ceph-mon[98694]: pgmap v1519: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 77 KiB/s wr, 25 op/s 2023-09-10T00:01:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2530.780872 secs 2023-09-10T00:01:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:40 smithi098 ceph-mon[103124]: pgmap v1519: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 77 KiB/s wr, 25 op/s 2023-09-10T00:01:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:42 smithi170 ceph-mon[102832]: pgmap v1520: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 167 KiB/s wr, 32 op/s 2023-09-10T00:01:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2533.456138 secs 2023-09-10T00:01:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:42 smithi082 ceph-mon[98694]: pgmap v1520: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 167 KiB/s wr, 32 op/s 2023-09-10T00:01:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2533.456138 secs 2023-09-10T00:01:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:42 smithi098 ceph-mon[103124]: pgmap v1520: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 167 KiB/s wr, 32 op/s 2023-09-10T00:01:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2533.456138 secs 2023-09-10T00:01:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2535.068227 secs 2023-09-10T00:01:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2535.068227 secs 2023-09-10T00:01:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2535.068227 secs 2023-09-10T00:01:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:44 smithi082 ceph-mon[98694]: pgmap v1521: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T00:01:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:01:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:01:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:01:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:44 smithi098 ceph-mon[103124]: pgmap v1521: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T00:01:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:01:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:01:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:01:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:44 smithi170 ceph-mon[102832]: pgmap v1521: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T00:01:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:01:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:01:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:01:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2535.780941 secs 2023-09-10T00:01:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2535.780941 secs 2023-09-10T00:01:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2535.780941 secs 2023-09-10T00:01:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:46 smithi082 ceph-mon[98694]: pgmap v1522: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T00:01:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:46 smithi098 ceph-mon[103124]: pgmap v1522: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T00:01:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:46 smithi170 ceph-mon[102832]: pgmap v1522: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T00:01:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2538.456214 secs 2023-09-10T00:01:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2538.456214 secs 2023-09-10T00:01:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2538.456214 secs 2023-09-10T00:01:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:48 smithi082 ceph-mon[98694]: pgmap v1523: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 135 KiB/s wr, 17 op/s 2023-09-10T00:01:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2540.068286 secs 2023-09-10T00:01:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:48 smithi098 ceph-mon[103124]: pgmap v1523: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 135 KiB/s wr, 17 op/s 2023-09-10T00:01:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2540.068286 secs 2023-09-10T00:01:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:48 smithi170 ceph-mon[102832]: pgmap v1523: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 135 KiB/s wr, 17 op/s 2023-09-10T00:01:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2540.068286 secs 2023-09-10T00:01:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2540.781008 secs 2023-09-10T00:01:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:50 smithi082 ceph-mon[98694]: pgmap v1524: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T00:01:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2540.781008 secs 2023-09-10T00:01:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:50 smithi098 ceph-mon[103124]: pgmap v1524: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T00:01:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2540.781008 secs 2023-09-10T00:01:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:50 smithi170 ceph-mon[102832]: pgmap v1524: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T00:01:52.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:52 smithi098 ceph-mon[103124]: pgmap v1525: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 134 KiB/s wr, 17 op/s 2023-09-10T00:01:52.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2543.456275 secs 2023-09-10T00:01:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:52 smithi170 ceph-mon[102832]: pgmap v1525: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 134 KiB/s wr, 17 op/s 2023-09-10T00:01:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2543.456275 secs 2023-09-10T00:01:52.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:52 smithi082 ceph-mon[98694]: pgmap v1525: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 134 KiB/s wr, 17 op/s 2023-09-10T00:01:52.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2543.456275 secs 2023-09-10T00:01:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2545.068343 secs 2023-09-10T00:01:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2545.068343 secs 2023-09-10T00:01:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2545.068343 secs 2023-09-10T00:01:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:54 smithi098 ceph-mon[103124]: pgmap v1526: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T00:01:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2545.781075 secs 2023-09-10T00:01:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:54 smithi170 ceph-mon[102832]: pgmap v1526: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T00:01:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2545.781075 secs 2023-09-10T00:01:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:54 smithi082 ceph-mon[98694]: pgmap v1526: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T00:01:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2545.781075 secs 2023-09-10T00:01:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:56 smithi098 ceph-mon[103124]: pgmap v1527: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T00:01:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:56 smithi170 ceph-mon[102832]: pgmap v1527: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T00:01:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:56 smithi082 ceph-mon[98694]: pgmap v1527: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T00:01:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2548.456356 secs 2023-09-10T00:01:57.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2548.456356 secs 2023-09-10T00:01:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2548.456356 secs 2023-09-10T00:01:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:58 smithi098 ceph-mon[103124]: pgmap v1528: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 54 KiB/s wr, 16 op/s 2023-09-10T00:01:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:01:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2550.068407 secs 2023-09-10T00:01:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:58 smithi170 ceph-mon[102832]: pgmap v1528: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 54 KiB/s wr, 16 op/s 2023-09-10T00:01:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:01:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2550.068407 secs 2023-09-10T00:01:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:58 smithi082 ceph-mon[98694]: pgmap v1528: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 54 KiB/s wr, 16 op/s 2023-09-10T00:01:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:01:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2550.068407 secs 2023-09-10T00:02:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2550.781142 secs 2023-09-10T00:02:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:00 smithi170 ceph-mon[102832]: pgmap v1529: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 26 KiB/s wr, 10 op/s 2023-09-10T00:02:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2550.781142 secs 2023-09-10T00:02:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:00 smithi082 ceph-mon[98694]: pgmap v1529: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 26 KiB/s wr, 10 op/s 2023-09-10T00:02:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2550.781142 secs 2023-09-10T00:02:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:00 smithi098 ceph-mon[103124]: pgmap v1529: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 26 KiB/s wr, 10 op/s 2023-09-10T00:02:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:02 smithi170 ceph-mon[102832]: pgmap v1530: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 27 KiB/s wr, 15 op/s 2023-09-10T00:02:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:02 smithi082 ceph-mon[98694]: pgmap v1530: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 27 KiB/s wr, 15 op/s 2023-09-10T00:02:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:02 smithi098 ceph-mon[103124]: pgmap v1530: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 27 KiB/s wr, 15 op/s 2023-09-10T00:02:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2553.456428 secs 2023-09-10T00:02:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2553.456428 secs 2023-09-10T00:02:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2553.456428 secs 2023-09-10T00:02:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2555.068464 secs 2023-09-10T00:02:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:04 smithi170 ceph-mon[102832]: pgmap v1531: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 KiB/s wr, 10 op/s 2023-09-10T00:02:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2555.068464 secs 2023-09-10T00:02:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:04 smithi082 ceph-mon[98694]: pgmap v1531: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 KiB/s wr, 10 op/s 2023-09-10T00:02:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2555.068464 secs 2023-09-10T00:02:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:04 smithi098 ceph-mon[103124]: pgmap v1531: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 KiB/s wr, 10 op/s 2023-09-10T00:02:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2555.781210 secs 2023-09-10T00:02:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2555.781210 secs 2023-09-10T00:02:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2555.781210 secs 2023-09-10T00:02:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:06 smithi170 ceph-mon[102832]: pgmap v1532: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 KiB/s wr, 10 op/s 2023-09-10T00:02:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:06 smithi082 ceph-mon[98694]: pgmap v1532: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 KiB/s wr, 10 op/s 2023-09-10T00:02:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:06 smithi098 ceph-mon[103124]: pgmap v1532: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 KiB/s wr, 10 op/s 2023-09-10T00:02:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2558.456552 secs 2023-09-10T00:02:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:08 smithi170 ceph-mon[102832]: pgmap v1533: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 72 KiB/s wr, 30 op/s 2023-09-10T00:02:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:08 smithi170 ceph-mon[102832]: osdmap e169: 12 total, 12 up, 12 in 2023-09-10T00:02:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2560.068521 secs 2023-09-10T00:02:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2558.456552 secs 2023-09-10T00:02:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:08 smithi082 ceph-mon[98694]: pgmap v1533: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 72 KiB/s wr, 30 op/s 2023-09-10T00:02:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:08 smithi082 ceph-mon[98694]: osdmap e169: 12 total, 12 up, 12 in 2023-09-10T00:02:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2560.068521 secs 2023-09-10T00:02:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2558.456552 secs 2023-09-10T00:02:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:08 smithi098 ceph-mon[103124]: pgmap v1533: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 72 KiB/s wr, 30 op/s 2023-09-10T00:02:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:08 smithi098 ceph-mon[103124]: osdmap e169: 12 total, 12 up, 12 in 2023-09-10T00:02:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2560.068521 secs 2023-09-10T00:02:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2560.781281 secs 2023-09-10T00:02:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:10 smithi170 ceph-mon[102832]: pgmap v1535: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-10T00:02:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2560.781281 secs 2023-09-10T00:02:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:10 smithi082 ceph-mon[98694]: pgmap v1535: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-10T00:02:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2560.781281 secs 2023-09-10T00:02:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:10 smithi098 ceph-mon[103124]: pgmap v1535: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-10T00:02:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:12 smithi170 ceph-mon[102832]: pgmap v1536: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-10T00:02:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:12 smithi082 ceph-mon[98694]: pgmap v1536: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-10T00:02:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:12 smithi098 ceph-mon[103124]: pgmap v1536: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-10T00:02:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2563.456614 secs 2023-09-10T00:02:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2565.068576 secs 2023-09-10T00:02:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2563.456614 secs 2023-09-10T00:02:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2565.068576 secs 2023-09-10T00:02:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2563.456614 secs 2023-09-10T00:02:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2565.068576 secs 2023-09-10T00:02:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:14 smithi170 ceph-mon[102832]: pgmap v1537: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-10T00:02:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:14 smithi170 ceph-mon[102832]: osdmap e170: 12 total, 12 up, 12 in 2023-09-10T00:02:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:14 smithi082 ceph-mon[98694]: pgmap v1537: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-10T00:02:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:14 smithi082 ceph-mon[98694]: osdmap e170: 12 total, 12 up, 12 in 2023-09-10T00:02:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:14 smithi098 ceph-mon[103124]: pgmap v1537: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-10T00:02:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:14 smithi098 ceph-mon[103124]: osdmap e170: 12 total, 12 up, 12 in 2023-09-10T00:02:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2565.781347 secs 2023-09-10T00:02:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:02:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2565.781347 secs 2023-09-10T00:02:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:02:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2565.781347 secs 2023-09-10T00:02:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:02:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:16 smithi170 ceph-mon[102832]: pgmap v1539: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-09-10T00:02:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:02:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:02:16.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:02:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:16 smithi082 ceph-mon[98694]: pgmap v1539: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-09-10T00:02:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:02:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:02:16.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:02:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:16 smithi098 ceph-mon[103124]: pgmap v1539: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-09-10T00:02:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:02:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:02:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:02:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:18 smithi170 ceph-mon[102832]: pgmap v1540: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T00:02:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2570.068635 secs 2023-09-10T00:02:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:18 smithi082 ceph-mon[98694]: pgmap v1540: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T00:02:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2570.068635 secs 2023-09-10T00:02:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:18 smithi098 ceph-mon[103124]: pgmap v1540: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T00:02:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2570.068635 secs 2023-09-10T00:02:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:02:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:02:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:02:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2570.781412 secs 2023-09-10T00:02:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:20 smithi170 ceph-mon[102832]: pgmap v1541: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T00:02:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:02:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:02:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:02:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2570.781412 secs 2023-09-10T00:02:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:20 smithi082 ceph-mon[98694]: pgmap v1541: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T00:02:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:02:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:02:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:02:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2570.781412 secs 2023-09-10T00:02:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:20 smithi098 ceph-mon[103124]: pgmap v1541: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T00:02:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:02:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:02:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:02:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:22 smithi170 ceph-mon[102832]: pgmap v1542: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T00:02:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:22 smithi082 ceph-mon[98694]: pgmap v1542: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T00:02:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:22 smithi098 ceph-mon[103124]: pgmap v1542: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T00:02:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2575.068694 secs 2023-09-10T00:02:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2575.068694 secs 2023-09-10T00:02:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2575.068694 secs 2023-09-10T00:02:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:24 smithi170 ceph-mon[102832]: pgmap v1543: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T00:02:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:24 smithi082 ceph-mon[98694]: pgmap v1543: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T00:02:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:24 smithi098 ceph-mon[103124]: pgmap v1543: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T00:02:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2575.781482 secs 2023-09-10T00:02:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2575.781482 secs 2023-09-10T00:02:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2575.781482 secs 2023-09-10T00:02:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:26 smithi170 ceph-mon[102832]: pgmap v1544: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 10 op/s 2023-09-10T00:02:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:26 smithi082 ceph-mon[98694]: pgmap v1544: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 10 op/s 2023-09-10T00:02:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:26 smithi098 ceph-mon[103124]: pgmap v1544: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 10 op/s 2023-09-10T00:02:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:28 smithi170 ceph-mon[102832]: pgmap v1545: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 15 op/s 2023-09-10T00:02:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2580.068751 secs 2023-09-10T00:02:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:28 smithi082 ceph-mon[98694]: pgmap v1545: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 15 op/s 2023-09-10T00:02:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2580.068751 secs 2023-09-10T00:02:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:28 smithi098 ceph-mon[103124]: pgmap v1545: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 15 op/s 2023-09-10T00:02:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2580.068751 secs 2023-09-10T00:02:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2580.781549 secs 2023-09-10T00:02:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:30 smithi170 ceph-mon[102832]: pgmap v1546: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T00:02:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2580.781549 secs 2023-09-10T00:02:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:30 smithi082 ceph-mon[98694]: pgmap v1546: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T00:02:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2580.781549 secs 2023-09-10T00:02:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:30 smithi098 ceph-mon[103124]: pgmap v1546: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T00:02:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:32 smithi170 ceph-mon[102832]: pgmap v1547: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 91 KiB/s wr, 16 op/s 2023-09-10T00:02:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:32 smithi082 ceph-mon[98694]: pgmap v1547: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 91 KiB/s wr, 16 op/s 2023-09-10T00:02:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:32 smithi098 ceph-mon[103124]: pgmap v1547: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 91 KiB/s wr, 16 op/s 2023-09-10T00:02:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2585.068808 secs 2023-09-10T00:02:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2585.068808 secs 2023-09-10T00:02:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2585.068808 secs 2023-09-10T00:02:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:34 smithi082 ceph-mon[98694]: pgmap v1548: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T00:02:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:34 smithi098 ceph-mon[103124]: pgmap v1548: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T00:02:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:34 smithi170 ceph-mon[102832]: pgmap v1548: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T00:02:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2585.781616 secs 2023-09-10T00:02:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2585.781616 secs 2023-09-10T00:02:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2585.781616 secs 2023-09-10T00:02:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:36 smithi082 ceph-mon[98694]: pgmap v1549: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T00:02:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:36 smithi098 ceph-mon[103124]: pgmap v1549: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T00:02:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:36 smithi170 ceph-mon[102832]: pgmap v1549: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T00:02:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:38 smithi082 ceph-mon[98694]: pgmap v1550: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 108 KiB/s wr, 17 op/s 2023-09-10T00:02:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2590.068868 secs 2023-09-10T00:02:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:38 smithi098 ceph-mon[103124]: pgmap v1550: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 108 KiB/s wr, 17 op/s 2023-09-10T00:02:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2590.068868 secs 2023-09-10T00:02:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:38 smithi170 ceph-mon[102832]: pgmap v1550: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 108 KiB/s wr, 17 op/s 2023-09-10T00:02:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2590.068868 secs 2023-09-10T00:02:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2590.781682 secs 2023-09-10T00:02:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:40 smithi082 ceph-mon[98694]: pgmap v1551: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T00:02:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2590.781682 secs 2023-09-10T00:02:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:40 smithi098 ceph-mon[103124]: pgmap v1551: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T00:02:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2590.781682 secs 2023-09-10T00:02:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:40 smithi170 ceph-mon[102832]: pgmap v1551: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T00:02:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:42 smithi082 ceph-mon[98694]: pgmap v1552: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 124 KiB/s wr, 17 op/s 2023-09-10T00:02:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:42 smithi098 ceph-mon[103124]: pgmap v1552: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 124 KiB/s wr, 17 op/s 2023-09-10T00:02:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:42 smithi170 ceph-mon[102832]: pgmap v1552: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 124 KiB/s wr, 17 op/s 2023-09-10T00:02:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2595.068925 secs 2023-09-10T00:02:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2595.068925 secs 2023-09-10T00:02:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2595.068925 secs 2023-09-10T00:02:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:44 smithi082 ceph-mon[98694]: pgmap v1553: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T00:02:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:44 smithi098 ceph-mon[103124]: pgmap v1553: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T00:02:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:44 smithi170 ceph-mon[102832]: pgmap v1553: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T00:02:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2595.781746 secs 2023-09-10T00:02:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2595.781746 secs 2023-09-10T00:02:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2595.781746 secs 2023-09-10T00:02:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:46 smithi098 ceph-mon[103124]: pgmap v1554: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T00:02:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:46 smithi170 ceph-mon[102832]: pgmap v1554: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T00:02:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:46 smithi082 ceph-mon[98694]: pgmap v1554: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T00:02:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2598.457183 secs 2023-09-10T00:02:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:48 smithi098 ceph-mon[103124]: pgmap v1555: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 94 KiB/s wr, 17 op/s 2023-09-10T00:02:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2600.068979 secs 2023-09-10T00:02:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2598.457183 secs 2023-09-10T00:02:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:48 smithi170 ceph-mon[102832]: pgmap v1555: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 94 KiB/s wr, 17 op/s 2023-09-10T00:02:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2600.068979 secs 2023-09-10T00:02:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2598.457183 secs 2023-09-10T00:02:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:48 smithi082 ceph-mon[98694]: pgmap v1555: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 94 KiB/s wr, 17 op/s 2023-09-10T00:02:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2600.068979 secs 2023-09-10T00:02:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2600.781816 secs 2023-09-10T00:02:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:50 smithi170 ceph-mon[102832]: pgmap v1556: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T00:02:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2600.781816 secs 2023-09-10T00:02:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:50 smithi082 ceph-mon[98694]: pgmap v1556: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T00:02:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2600.781816 secs 2023-09-10T00:02:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:50 smithi098 ceph-mon[103124]: pgmap v1556: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T00:02:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:02:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:02:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:02:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:02:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:02:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:02:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:02:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:02:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:02:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:52 smithi170 ceph-mon[102832]: pgmap v1557: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 93 KiB/s wr, 16 op/s 2023-09-10T00:02:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:52 smithi082 ceph-mon[98694]: pgmap v1557: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 93 KiB/s wr, 16 op/s 2023-09-10T00:02:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:52 smithi098 ceph-mon[103124]: pgmap v1557: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 93 KiB/s wr, 16 op/s 2023-09-10T00:02:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2603.457244 secs 2023-09-10T00:02:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2605.069046 secs 2023-09-10T00:02:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2603.457244 secs 2023-09-10T00:02:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2605.069046 secs 2023-09-10T00:02:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2603.457244 secs 2023-09-10T00:02:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2605.069046 secs 2023-09-10T00:02:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:54 smithi170 ceph-mon[102832]: pgmap v1558: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 45 KiB/s wr, 10 op/s 2023-09-10T00:02:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:54 smithi082 ceph-mon[98694]: pgmap v1558: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 45 KiB/s wr, 10 op/s 2023-09-10T00:02:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:54 smithi098 ceph-mon[103124]: pgmap v1558: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 45 KiB/s wr, 10 op/s 2023-09-10T00:02:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2605.781880 secs 2023-09-10T00:02:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2605.781880 secs 2023-09-10T00:02:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2605.781880 secs 2023-09-10T00:02:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:56 smithi170 ceph-mon[102832]: pgmap v1559: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 45 KiB/s wr, 10 op/s 2023-09-10T00:02:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:56 smithi082 ceph-mon[98694]: pgmap v1559: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 45 KiB/s wr, 10 op/s 2023-09-10T00:02:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:56 smithi098 ceph-mon[103124]: pgmap v1559: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 45 KiB/s wr, 10 op/s 2023-09-10T00:02:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2608.457365 secs 2023-09-10T00:02:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:58 smithi170 ceph-mon[102832]: pgmap v1560: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 58 KiB/s wr, 15 op/s 2023-09-10T00:02:58.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:02:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2610.069106 secs 2023-09-10T00:02:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2608.457365 secs 2023-09-10T00:02:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:58 smithi082 ceph-mon[98694]: pgmap v1560: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 58 KiB/s wr, 15 op/s 2023-09-10T00:02:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:02:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2610.069106 secs 2023-09-10T00:02:58.509 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2608.457365 secs 2023-09-10T00:02:58.510 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:58 smithi098 ceph-mon[103124]: pgmap v1560: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 58 KiB/s wr, 15 op/s 2023-09-10T00:02:58.510 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:02:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2610.069106 secs 2023-09-10T00:03:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2610.781948 secs 2023-09-10T00:03:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:00 smithi082 ceph-mon[98694]: pgmap v1561: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T00:03:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2610.781948 secs 2023-09-10T00:03:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:00 smithi098 ceph-mon[103124]: pgmap v1561: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T00:03:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2610.781948 secs 2023-09-10T00:03:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:00 smithi170 ceph-mon[102832]: pgmap v1561: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T00:03:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:02 smithi082 ceph-mon[98694]: pgmap v1562: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 14 op/s 2023-09-10T00:03:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:02 smithi098 ceph-mon[103124]: pgmap v1562: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 14 op/s 2023-09-10T00:03:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:02 smithi170 ceph-mon[102832]: pgmap v1562: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 14 op/s 2023-09-10T00:03:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2613.457458 secs 2023-09-10T00:03:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2615.069164 secs 2023-09-10T00:03:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2613.457458 secs 2023-09-10T00:03:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2615.069164 secs 2023-09-10T00:03:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2613.457458 secs 2023-09-10T00:03:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2615.069164 secs 2023-09-10T00:03:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:04 smithi082 ceph-mon[98694]: pgmap v1563: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-10T00:03:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:04 smithi098 ceph-mon[103124]: pgmap v1563: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-10T00:03:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:04 smithi170 ceph-mon[102832]: pgmap v1563: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-10T00:03:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2615.782013 secs 2023-09-10T00:03:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2615.782013 secs 2023-09-10T00:03:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2615.782013 secs 2023-09-10T00:03:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:06 smithi082 ceph-mon[98694]: pgmap v1564: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-10T00:03:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:06 smithi098 ceph-mon[103124]: pgmap v1564: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-10T00:03:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:06 smithi170 ceph-mon[102832]: pgmap v1564: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-10T00:03:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2618.457518 secs 2023-09-10T00:03:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:08 smithi082 ceph-mon[98694]: pgmap v1565: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 74 KiB/s wr, 29 op/s 2023-09-10T00:03:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:08 smithi082 ceph-mon[98694]: osdmap e171: 12 total, 12 up, 12 in 2023-09-10T00:03:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2620.069223 secs 2023-09-10T00:03:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2618.457518 secs 2023-09-10T00:03:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:08 smithi098 ceph-mon[103124]: pgmap v1565: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 74 KiB/s wr, 29 op/s 2023-09-10T00:03:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:08 smithi098 ceph-mon[103124]: osdmap e171: 12 total, 12 up, 12 in 2023-09-10T00:03:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2620.069223 secs 2023-09-10T00:03:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2618.457518 secs 2023-09-10T00:03:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:08 smithi170 ceph-mon[102832]: pgmap v1565: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 74 KiB/s wr, 29 op/s 2023-09-10T00:03:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:08 smithi170 ceph-mon[102832]: osdmap e171: 12 total, 12 up, 12 in 2023-09-10T00:03:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2620.069223 secs 2023-09-10T00:03:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2620.782081 secs 2023-09-10T00:03:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:10 smithi082 ceph-mon[98694]: pgmap v1567: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-10T00:03:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2620.782081 secs 2023-09-10T00:03:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:10 smithi098 ceph-mon[103124]: pgmap v1567: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-10T00:03:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2620.782081 secs 2023-09-10T00:03:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:10 smithi170 ceph-mon[102832]: pgmap v1567: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-10T00:03:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:12 smithi082 ceph-mon[98694]: pgmap v1568: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T00:03:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:12 smithi098 ceph-mon[103124]: pgmap v1568: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T00:03:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:12 smithi170 ceph-mon[102832]: pgmap v1568: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T00:03:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2623.457577 secs 2023-09-10T00:03:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2625.069306 secs 2023-09-10T00:03:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2623.457577 secs 2023-09-10T00:03:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2625.069306 secs 2023-09-10T00:03:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2623.457577 secs 2023-09-10T00:03:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2625.069306 secs 2023-09-10T00:03:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:14 smithi098 ceph-mon[103124]: pgmap v1569: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T00:03:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:14 smithi098 ceph-mon[103124]: osdmap e172: 12 total, 12 up, 12 in 2023-09-10T00:03:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:14 smithi170 ceph-mon[102832]: pgmap v1569: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T00:03:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:14 smithi170 ceph-mon[102832]: osdmap e172: 12 total, 12 up, 12 in 2023-09-10T00:03:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:14 smithi082 ceph-mon[98694]: pgmap v1569: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T00:03:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:14 smithi082 ceph-mon[98694]: osdmap e172: 12 total, 12 up, 12 in 2023-09-10T00:03:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2625.782145 secs 2023-09-10T00:03:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2625.782145 secs 2023-09-10T00:03:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2625.782145 secs 2023-09-10T00:03:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:16 smithi098 ceph-mon[103124]: pgmap v1571: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 24 KiB/s wr, 7 op/s 2023-09-10T00:03:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:03:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:16 smithi170 ceph-mon[102832]: pgmap v1571: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 24 KiB/s wr, 7 op/s 2023-09-10T00:03:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:03:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:16 smithi082 ceph-mon[98694]: pgmap v1571: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 24 KiB/s wr, 7 op/s 2023-09-10T00:03:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:03:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:03:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:03:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:03:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:03:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:03:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:03:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:03:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:03:17.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:03:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:18 smithi170 ceph-mon[102832]: pgmap v1572: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-09-10T00:03:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2630.069382 secs 2023-09-10T00:03:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:18 smithi082 ceph-mon[98694]: pgmap v1572: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-09-10T00:03:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2630.069382 secs 2023-09-10T00:03:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:18 smithi098 ceph-mon[103124]: pgmap v1572: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-09-10T00:03:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2630.069382 secs 2023-09-10T00:03:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2630.782211 secs 2023-09-10T00:03:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:03:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2630.782211 secs 2023-09-10T00:03:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:03:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2630.782211 secs 2023-09-10T00:03:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:03:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:20 smithi170 ceph-mon[102832]: pgmap v1573: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 11 op/s 2023-09-10T00:03:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:03:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:03:20.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:03:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:20 smithi082 ceph-mon[98694]: pgmap v1573: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 11 op/s 2023-09-10T00:03:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:03:20.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:03:20.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:03:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:20 smithi098 ceph-mon[103124]: pgmap v1573: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 11 op/s 2023-09-10T00:03:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:03:21.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:03:21.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:03:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:22 smithi170 ceph-mon[102832]: pgmap v1574: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T00:03:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:22 smithi082 ceph-mon[98694]: pgmap v1574: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T00:03:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:22 smithi098 ceph-mon[103124]: pgmap v1574: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T00:03:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2635.069441 secs 2023-09-10T00:03:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2635.069441 secs 2023-09-10T00:03:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2635.069441 secs 2023-09-10T00:03:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:24 smithi170 ceph-mon[102832]: pgmap v1575: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T00:03:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2635.782274 secs 2023-09-10T00:03:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:24 smithi082 ceph-mon[98694]: pgmap v1575: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T00:03:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2635.782274 secs 2023-09-10T00:03:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:24 smithi098 ceph-mon[103124]: pgmap v1575: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T00:03:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2635.782274 secs 2023-09-10T00:03:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:26 smithi170 ceph-mon[102832]: pgmap v1576: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T00:03:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:26 smithi082 ceph-mon[98694]: pgmap v1576: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T00:03:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:26 smithi098 ceph-mon[103124]: pgmap v1576: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T00:03:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:28 smithi170 ceph-mon[102832]: pgmap v1577: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 102 KiB/s wr, 16 op/s 2023-09-10T00:03:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2640.069498 secs 2023-09-10T00:03:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:28 smithi082 ceph-mon[98694]: pgmap v1577: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 102 KiB/s wr, 16 op/s 2023-09-10T00:03:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2640.069498 secs 2023-09-10T00:03:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:28 smithi098 ceph-mon[103124]: pgmap v1577: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 102 KiB/s wr, 16 op/s 2023-09-10T00:03:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2640.069498 secs 2023-09-10T00:03:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2640.782340 secs 2023-09-10T00:03:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:30 smithi170 ceph-mon[102832]: pgmap v1578: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 101 KiB/s wr, 11 op/s 2023-09-10T00:03:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2640.782340 secs 2023-09-10T00:03:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:30 smithi082 ceph-mon[98694]: pgmap v1578: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 101 KiB/s wr, 11 op/s 2023-09-10T00:03:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2640.782340 secs 2023-09-10T00:03:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:30 smithi098 ceph-mon[103124]: pgmap v1578: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 101 KiB/s wr, 11 op/s 2023-09-10T00:03:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:32 smithi170 ceph-mon[102832]: pgmap v1579: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 149 KiB/s wr, 17 op/s 2023-09-10T00:03:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:32 smithi082 ceph-mon[98694]: pgmap v1579: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 149 KiB/s wr, 17 op/s 2023-09-10T00:03:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:32 smithi098 ceph-mon[103124]: pgmap v1579: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 149 KiB/s wr, 17 op/s 2023-09-10T00:03:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2645.069553 secs 2023-09-10T00:03:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2645.069553 secs 2023-09-10T00:03:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2645.069553 secs 2023-09-10T00:03:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:34 smithi170 ceph-mon[102832]: pgmap v1580: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T00:03:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:34 smithi082 ceph-mon[98694]: pgmap v1580: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T00:03:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:34 smithi098 ceph-mon[103124]: pgmap v1580: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T00:03:35.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2645.782405 secs 2023-09-10T00:03:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2645.782405 secs 2023-09-10T00:03:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2645.782405 secs 2023-09-10T00:03:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:36 smithi170 ceph-mon[102832]: pgmap v1581: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T00:03:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:36 smithi082 ceph-mon[98694]: pgmap v1581: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T00:03:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:36 smithi098 ceph-mon[103124]: pgmap v1581: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T00:03:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:38 smithi170 ceph-mon[102832]: pgmap v1582: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 126 KiB/s wr, 18 op/s 2023-09-10T00:03:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2650.069615 secs 2023-09-10T00:03:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:38 smithi082 ceph-mon[98694]: pgmap v1582: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 126 KiB/s wr, 18 op/s 2023-09-10T00:03:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2650.069615 secs 2023-09-10T00:03:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:38 smithi098 ceph-mon[103124]: pgmap v1582: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 126 KiB/s wr, 18 op/s 2023-09-10T00:03:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2650.069615 secs 2023-09-10T00:03:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2650.782477 secs 2023-09-10T00:03:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:40 smithi170 ceph-mon[102832]: pgmap v1583: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T00:03:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2650.782477 secs 2023-09-10T00:03:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:40 smithi082 ceph-mon[98694]: pgmap v1583: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T00:03:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2650.782477 secs 2023-09-10T00:03:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:40 smithi098 ceph-mon[103124]: pgmap v1583: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T00:03:42.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:42 smithi170 ceph-mon[102832]: pgmap v1584: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 125 KiB/s wr, 17 op/s 2023-09-10T00:03:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:42 smithi082 ceph-mon[98694]: pgmap v1584: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 125 KiB/s wr, 17 op/s 2023-09-10T00:03:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:42 smithi098 ceph-mon[103124]: pgmap v1584: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 125 KiB/s wr, 17 op/s 2023-09-10T00:03:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2653.457962 secs 2023-09-10T00:03:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2655.069687 secs 2023-09-10T00:03:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2653.457962 secs 2023-09-10T00:03:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2655.069687 secs 2023-09-10T00:03:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2653.457962 secs 2023-09-10T00:03:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2655.069687 secs 2023-09-10T00:03:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:44 smithi170 ceph-mon[102832]: pgmap v1585: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T00:03:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:03:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:03:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:03:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:44 smithi082 ceph-mon[98694]: pgmap v1585: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T00:03:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:03:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:03:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:03:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:44 smithi098 ceph-mon[103124]: pgmap v1585: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T00:03:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:03:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:03:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:03:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2655.782542 secs 2023-09-10T00:03:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2655.782542 secs 2023-09-10T00:03:45.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2655.782542 secs 2023-09-10T00:03:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:46 smithi170 ceph-mon[102832]: pgmap v1586: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T00:03:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:46 smithi082 ceph-mon[98694]: pgmap v1586: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T00:03:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:46 smithi098 ceph-mon[103124]: pgmap v1586: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T00:03:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2658.458059 secs 2023-09-10T00:03:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:48 smithi170 ceph-mon[102832]: pgmap v1587: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 113 KiB/s wr, 16 op/s 2023-09-10T00:03:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2660.069748 secs 2023-09-10T00:03:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2658.458059 secs 2023-09-10T00:03:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:48 smithi082 ceph-mon[98694]: pgmap v1587: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 113 KiB/s wr, 16 op/s 2023-09-10T00:03:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2660.069748 secs 2023-09-10T00:03:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2658.458059 secs 2023-09-10T00:03:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:48 smithi098 ceph-mon[103124]: pgmap v1587: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 113 KiB/s wr, 16 op/s 2023-09-10T00:03:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2660.069748 secs 2023-09-10T00:03:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2660.782608 secs 2023-09-10T00:03:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:50 smithi170 ceph-mon[102832]: pgmap v1588: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T00:03:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2660.782608 secs 2023-09-10T00:03:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:50 smithi082 ceph-mon[98694]: pgmap v1588: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T00:03:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2660.782608 secs 2023-09-10T00:03:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:50 smithi098 ceph-mon[103124]: pgmap v1588: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T00:03:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:52 smithi170 ceph-mon[102832]: pgmap v1589: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 71 KiB/s wr, 15 op/s 2023-09-10T00:03:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:52 smithi082 ceph-mon[98694]: pgmap v1589: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 71 KiB/s wr, 15 op/s 2023-09-10T00:03:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:52 smithi098 ceph-mon[103124]: pgmap v1589: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 71 KiB/s wr, 15 op/s 2023-09-10T00:03:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2663.458121 secs 2023-09-10T00:03:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2665.069806 secs 2023-09-10T00:03:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2663.458121 secs 2023-09-10T00:03:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2665.069806 secs 2023-09-10T00:03:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2663.458121 secs 2023-09-10T00:03:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2665.069806 secs 2023-09-10T00:03:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:54 smithi170 ceph-mon[102832]: pgmap v1590: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 9 op/s 2023-09-10T00:03:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:54 smithi082 ceph-mon[98694]: pgmap v1590: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 9 op/s 2023-09-10T00:03:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:54 smithi098 ceph-mon[103124]: pgmap v1590: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 9 op/s 2023-09-10T00:03:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2665.782675 secs 2023-09-10T00:03:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2665.782675 secs 2023-09-10T00:03:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2665.782675 secs 2023-09-10T00:03:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:56 smithi170 ceph-mon[102832]: pgmap v1591: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 9 op/s 2023-09-10T00:03:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:56 smithi082 ceph-mon[98694]: pgmap v1591: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 9 op/s 2023-09-10T00:03:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:56 smithi098 ceph-mon[103124]: pgmap v1591: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 9 op/s 2023-09-10T00:03:56.848 INFO:journalctl@ceph.osd.0.smithi082.stdout:Sep 10 00:03:56 smithi082 sudo[187725]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-10T00:03:56.849 INFO:journalctl@ceph.osd.0.smithi082.stdout:Sep 10 00:03:56 smithi082 sudo[187725]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:03:56.849 INFO:journalctl@ceph.osd.0.smithi082.stdout:Sep 10 00:03:56 smithi082 sudo[187725]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:03:56.849 INFO:journalctl@ceph.osd.0.smithi082.stdout:Sep 10 00:03:56 smithi082 sudo[187725]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:03:57.207 INFO:journalctl@ceph.osd.0.smithi082.stdout:Sep 10 00:03:56 smithi082 sudo[187728]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-10T00:03:57.207 INFO:journalctl@ceph.osd.0.smithi082.stdout:Sep 10 00:03:56 smithi082 sudo[187728]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:03:57.207 INFO:journalctl@ceph.osd.0.smithi082.stdout:Sep 10 00:03:56 smithi082 sudo[187728]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:03:57.208 INFO:journalctl@ceph.osd.0.smithi082.stdout:Sep 10 00:03:56 smithi082 sudo[187728]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:03:57.689 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 10 00:03:57 smithi098 sudo[155336]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-10T00:03:57.690 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 10 00:03:57 smithi098 sudo[155336]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:03:57.690 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 10 00:03:57 smithi098 sudo[155336]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:03:57.690 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 10 00:03:57 smithi098 sudo[155336]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:03:57.691 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 10 00:03:57 smithi098 sudo[155339]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-10T00:03:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2668.458255 secs 2023-09-10T00:03:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:03:58.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:03:58.005 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 10 00:03:57 smithi098 sudo[155339]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:03:58.005 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 10 00:03:57 smithi098 sudo[155339]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:03:58.006 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 10 00:03:57 smithi098 sudo[155339]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:03:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2668.458255 secs 2023-09-10T00:03:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:03:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:03:58.179 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 10 00:03:58 smithi170 sudo[155495]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-10T00:03:58.179 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 10 00:03:58 smithi170 sudo[155495]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:03:58.179 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 10 00:03:58 smithi170 sudo[155495]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:03:58.179 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 10 00:03:58 smithi170 sudo[155495]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:03:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2668.458255 secs 2023-09-10T00:03:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:03:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:03:58.677 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 10 00:03:58 smithi170 sudo[155498]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-10T00:03:58.678 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 10 00:03:58 smithi170 sudo[155498]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:03:58.678 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 10 00:03:58 smithi170 sudo[155498]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:03:58.678 INFO:journalctl@ceph.osd.2.smithi170.stdout:Sep 10 00:03:58 smithi170 sudo[155498]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:03:58.839 INFO:journalctl@ceph.osd.3.smithi082.stdout:Sep 10 00:03:58 smithi082 sudo[187731]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-10T00:03:58.839 INFO:journalctl@ceph.osd.3.smithi082.stdout:Sep 10 00:03:58 smithi082 sudo[187731]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:03:58.839 INFO:journalctl@ceph.osd.3.smithi082.stdout:Sep 10 00:03:58 smithi082 sudo[187731]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:03:58.840 INFO:journalctl@ceph.osd.3.smithi082.stdout:Sep 10 00:03:58 smithi082 sudo[187731]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:03:59.149 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:58 smithi098 ceph-mon[103124]: pgmap v1592: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 64 KiB/s wr, 15 op/s 2023-09-10T00:03:59.149 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2670.069866 secs 2023-09-10T00:03:59.149 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:03:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:03:59.150 INFO:journalctl@ceph.osd.4.smithi098.stdout:Sep 10 00:03:59 smithi098 sudo[155342]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-10T00:03:59.150 INFO:journalctl@ceph.osd.4.smithi098.stdout:Sep 10 00:03:59 smithi098 sudo[155342]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:03:59.150 INFO:journalctl@ceph.osd.4.smithi098.stdout:Sep 10 00:03:59 smithi098 sudo[155342]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:03:59.150 INFO:journalctl@ceph.osd.4.smithi098.stdout:Sep 10 00:03:59 smithi098 sudo[155342]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:03:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:58 smithi170 ceph-mon[102832]: pgmap v1592: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 64 KiB/s wr, 15 op/s 2023-09-10T00:03:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2670.069866 secs 2023-09-10T00:03:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:03:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:03:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:58 smithi082 ceph-mon[98694]: pgmap v1592: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 64 KiB/s wr, 15 op/s 2023-09-10T00:03:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2670.069866 secs 2023-09-10T00:03:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:03:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:03:59.208 INFO:journalctl@ceph.osd.3.smithi082.stdout:Sep 10 00:03:58 smithi082 sudo[187734]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-10T00:03:59.208 INFO:journalctl@ceph.osd.3.smithi082.stdout:Sep 10 00:03:58 smithi082 sudo[187734]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:03:59.208 INFO:journalctl@ceph.osd.3.smithi082.stdout:Sep 10 00:03:58 smithi082 sudo[187734]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:03:59.209 INFO:journalctl@ceph.osd.3.smithi082.stdout:Sep 10 00:03:58 smithi082 sudo[187734]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:03:59.754 INFO:journalctl@ceph.osd.4.smithi098.stdout:Sep 10 00:03:59 smithi098 sudo[155345]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-10T00:03:59.754 INFO:journalctl@ceph.osd.4.smithi098.stdout:Sep 10 00:03:59 smithi098 sudo[155345]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:03:59.754 INFO:journalctl@ceph.osd.4.smithi098.stdout:Sep 10 00:03:59 smithi098 sudo[155345]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:03:59.755 INFO:journalctl@ceph.osd.4.smithi098.stdout:Sep 10 00:03:59 smithi098 sudo[155345]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:03:59.997 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 10 00:03:59 smithi170 sudo[155501]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-10T00:03:59.998 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 10 00:03:59 smithi170 sudo[155501]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:03:59.998 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 10 00:03:59 smithi170 sudo[155501]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:03:59.998 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 10 00:03:59 smithi170 sudo[155501]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:04:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2670.782748 secs 2023-09-10T00:04:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:00 smithi170 ceph-mon[102832]: pgmap v1593: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 29 KiB/s wr, 10 op/s 2023-09-10T00:04:00.428 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 10 00:03:59 smithi170 sudo[155504]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-10T00:04:00.428 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 10 00:03:59 smithi170 sudo[155504]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:04:00.429 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 10 00:03:59 smithi170 sudo[155504]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:04:00.429 INFO:journalctl@ceph.osd.5.smithi170.stdout:Sep 10 00:03:59 smithi170 sudo[155504]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:04:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2670.782748 secs 2023-09-10T00:04:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:00 smithi082 ceph-mon[98694]: pgmap v1593: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 29 KiB/s wr, 10 op/s 2023-09-10T00:04:00.458 INFO:journalctl@ceph.osd.6.smithi082.stdout:Sep 10 00:04:00 smithi082 sudo[187737]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-10T00:04:00.458 INFO:journalctl@ceph.osd.6.smithi082.stdout:Sep 10 00:04:00 smithi082 sudo[187737]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:04:00.458 INFO:journalctl@ceph.osd.6.smithi082.stdout:Sep 10 00:04:00 smithi082 sudo[187737]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:04:00.458 INFO:journalctl@ceph.osd.6.smithi082.stdout:Sep 10 00:04:00 smithi082 sudo[187737]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:04:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2670.782748 secs 2023-09-10T00:04:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:00 smithi098 ceph-mon[103124]: pgmap v1593: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 29 KiB/s wr, 10 op/s 2023-09-10T00:04:00.957 INFO:journalctl@ceph.osd.6.smithi082.stdout:Sep 10 00:04:00 smithi082 sudo[187740]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-10T00:04:00.957 INFO:journalctl@ceph.osd.6.smithi082.stdout:Sep 10 00:04:00 smithi082 sudo[187740]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:04:00.957 INFO:journalctl@ceph.osd.6.smithi082.stdout:Sep 10 00:04:00 smithi082 sudo[187740]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:04:00.958 INFO:journalctl@ceph.osd.6.smithi082.stdout:Sep 10 00:04:00 smithi082 sudo[187740]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:04:01.127 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 10 00:04:00 smithi098 sudo[155348]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-10T00:04:01.127 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 10 00:04:00 smithi098 sudo[155348]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:04:01.127 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 10 00:04:00 smithi098 sudo[155348]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:04:01.127 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 10 00:04:00 smithi098 sudo[155348]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:04:01.504 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 10 00:04:01 smithi098 sudo[155351]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-10T00:04:01.504 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 10 00:04:01 smithi098 sudo[155351]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:04:01.504 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 10 00:04:01 smithi098 sudo[155351]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:04:01.504 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 10 00:04:01 smithi098 sudo[155351]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:04:01.677 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 10 00:04:01 smithi170 sudo[155508]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-10T00:04:01.678 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 10 00:04:01 smithi170 sudo[155508]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:04:01.678 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 10 00:04:01 smithi170 sudo[155508]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:04:01.678 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 10 00:04:01 smithi170 sudo[155508]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:04:02.127 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 10 00:04:01 smithi170 sudo[155511]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-10T00:04:02.127 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 10 00:04:01 smithi170 sudo[155511]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:04:02.127 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 10 00:04:01 smithi170 sudo[155511]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:04:02.127 INFO:journalctl@ceph.osd.8.smithi170.stdout:Sep 10 00:04:01 smithi170 sudo[155511]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:04:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:02 smithi082 ceph-mon[98694]: pgmap v1594: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 128 KiB/s wr, 23 op/s 2023-09-10T00:04:02.208 INFO:journalctl@ceph.osd.9.smithi082.stdout:Sep 10 00:04:01 smithi082 sudo[187744]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-10T00:04:02.208 INFO:journalctl@ceph.osd.9.smithi082.stdout:Sep 10 00:04:01 smithi082 sudo[187744]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:04:02.208 INFO:journalctl@ceph.osd.9.smithi082.stdout:Sep 10 00:04:01 smithi082 sudo[187744]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:04:02.208 INFO:journalctl@ceph.osd.9.smithi082.stdout:Sep 10 00:04:01 smithi082 sudo[187744]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:04:02.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:02 smithi170 ceph-mon[102832]: pgmap v1594: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 128 KiB/s wr, 23 op/s 2023-09-10T00:04:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:02 smithi098 ceph-mon[103124]: pgmap v1594: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 128 KiB/s wr, 23 op/s 2023-09-10T00:04:02.707 INFO:journalctl@ceph.osd.9.smithi082.stdout:Sep 10 00:04:02 smithi082 sudo[187747]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-10T00:04:02.707 INFO:journalctl@ceph.osd.9.smithi082.stdout:Sep 10 00:04:02 smithi082 sudo[187747]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:04:02.708 INFO:journalctl@ceph.osd.9.smithi082.stdout:Sep 10 00:04:02 smithi082 sudo[187747]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:04:02.708 INFO:journalctl@ceph.osd.9.smithi082.stdout:Sep 10 00:04:02 smithi082 sudo[187747]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:04:02.828 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 10 00:04:02 smithi098 sudo[155356]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-10T00:04:02.828 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 10 00:04:02 smithi098 sudo[155356]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:04:02.828 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 10 00:04:02 smithi098 sudo[155356]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:04:02.828 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 10 00:04:02 smithi098 sudo[155356]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:04:02.829 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 10 00:04:02 smithi098 sudo[155359]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-10T00:04:03.132 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 10 00:04:02 smithi098 sudo[155359]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:04:03.132 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 10 00:04:02 smithi098 sudo[155359]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:04:03.133 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 10 00:04:02 smithi098 sudo[155359]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:04:03.387 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 10 00:04:03 smithi170 sudo[155514]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-10T00:04:03.387 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 10 00:04:03 smithi170 sudo[155514]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:04:03.387 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 10 00:04:03 smithi170 sudo[155514]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:04:03.388 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 10 00:04:03 smithi170 sudo[155514]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:04:03.388 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2673.458336 secs 2023-09-10T00:04:03.388 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2675.069924 secs 2023-09-10T00:04:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2673.458336 secs 2023-09-10T00:04:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2675.069924 secs 2023-09-10T00:04:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2673.458336 secs 2023-09-10T00:04:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2675.069924 secs 2023-09-10T00:04:03.677 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 10 00:04:03 smithi170 sudo[155517]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-10T00:04:03.678 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 10 00:04:03 smithi170 sudo[155517]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:04:03.678 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 10 00:04:03 smithi170 sudo[155517]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:04:03.678 INFO:journalctl@ceph.osd.11.smithi170.stdout:Sep 10 00:04:03 smithi170 sudo[155517]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:04:03.896 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:03 smithi082 sudo[187750]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-10T00:04:03.896 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:03 smithi082 sudo[187750]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:04:03.897 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:03 smithi082 sudo[187750]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:04:03.897 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:03 smithi082 sudo[187750]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:04:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:03 smithi082 sudo[187753]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-10T00:04:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:03 smithi082 sudo[187753]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:04:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:03 smithi082 sudo[187753]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:04:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:03 smithi082 sudo[187753]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:04:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:04 smithi170 sudo[155520]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-10T00:04:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:04 smithi170 sudo[155520]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:04:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:04 smithi170 sudo[155520]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:04:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:04 smithi170 sudo[155520]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:04:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:04 smithi082 ceph-mon[98694]: pgmap v1595: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 112 KiB/s wr, 18 op/s 2023-09-10T00:04:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:04 smithi082 ceph-mon[98694]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-10T00:04:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:04 smithi082 ceph-mon[98694]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-10T00:04:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:04:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:04:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:04:04.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:04:04.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:04:04.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:04:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:04 smithi098 ceph-mon[103124]: pgmap v1595: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 112 KiB/s wr, 18 op/s 2023-09-10T00:04:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:04 smithi098 ceph-mon[103124]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-10T00:04:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:04 smithi098 ceph-mon[103124]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-10T00:04:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:04:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:04:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:04:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:04:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:04:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:04:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:04 smithi170 sudo[155523]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-10T00:04:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:04 smithi170 sudo[155523]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:04:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:04 smithi170 sudo[155523]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:04:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:04 smithi170 sudo[155523]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:04:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:04 smithi170 ceph-mon[102832]: pgmap v1595: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 112 KiB/s wr, 18 op/s 2023-09-10T00:04:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:04 smithi170 ceph-mon[102832]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-10T00:04:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:04 smithi170 ceph-mon[102832]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-10T00:04:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:04:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:04:04.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:04:04.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:04:04.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:04:04.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:04:05.148 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:04 smithi098 sudo[155362]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-10T00:04:05.148 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:04 smithi098 sudo[155362]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:04:05.149 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:04 smithi098 sudo[155362]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:04:05.149 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:04 smithi098 sudo[155362]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:04:05.465 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:05 smithi098 sudo[155365]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-10T00:04:05.465 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:05 smithi098 sudo[155365]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-10T00:04:05.465 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:05 smithi098 sudo[155365]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-10T00:04:05.466 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:05 smithi098 sudo[155365]: pam_unix(sudo:session): session closed for user root 2023-09-10T00:04:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2675.782808 secs 2023-09-10T00:04:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:05 smithi098 ceph-mon[103124]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-10T00:04:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:05 smithi098 ceph-mon[103124]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-10T00:04:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:04:05.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:04:05.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:04:05.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:05 smithi098 ceph-mon[103124]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-10T00:04:05.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:05 smithi098 ceph-mon[103124]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-10T00:04:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2675.782808 secs 2023-09-10T00:04:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:05 smithi170 ceph-mon[102832]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-10T00:04:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:05 smithi170 ceph-mon[102832]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-10T00:04:05.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:04:05.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:04:05.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:04:05.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:05 smithi170 ceph-mon[102832]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-10T00:04:05.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:05 smithi170 ceph-mon[102832]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-10T00:04:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2675.782808 secs 2023-09-10T00:04:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:05 smithi082 ceph-mon[98694]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-10T00:04:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:05 smithi082 ceph-mon[98694]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-10T00:04:05.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-10T00:04:05.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-10T00:04:05.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-10T00:04:05.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:05 smithi082 ceph-mon[98694]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-10T00:04:05.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:05 smithi082 ceph-mon[98694]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-10T00:04:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:06 smithi098 ceph-mon[103124]: pgmap v1596: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 112 KiB/s wr, 18 op/s 2023-09-10T00:04:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:06 smithi170 ceph-mon[102832]: pgmap v1596: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 112 KiB/s wr, 18 op/s 2023-09-10T00:04:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:06 smithi082 ceph-mon[98694]: pgmap v1596: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 112 KiB/s wr, 18 op/s 2023-09-10T00:04:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2678.458402 secs 2023-09-10T00:04:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2678.458402 secs 2023-09-10T00:04:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2678.458402 secs 2023-09-10T00:04:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:08 smithi098 ceph-mon[103124]: pgmap v1597: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 204 KiB/s wr, 40 op/s 2023-09-10T00:04:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:08 smithi098 ceph-mon[103124]: osdmap e173: 12 total, 12 up, 12 in 2023-09-10T00:04:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2680.069989 secs 2023-09-10T00:04:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:08 smithi170 ceph-mon[102832]: pgmap v1597: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 204 KiB/s wr, 40 op/s 2023-09-10T00:04:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:08 smithi170 ceph-mon[102832]: osdmap e173: 12 total, 12 up, 12 in 2023-09-10T00:04:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2680.069989 secs 2023-09-10T00:04:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:08 smithi082 ceph-mon[98694]: pgmap v1597: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 204 KiB/s wr, 40 op/s 2023-09-10T00:04:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:08 smithi082 ceph-mon[98694]: osdmap e173: 12 total, 12 up, 12 in 2023-09-10T00:04:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2680.069989 secs 2023-09-10T00:04:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2680.782872 secs 2023-09-10T00:04:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:10 smithi170 ceph-mon[102832]: pgmap v1599: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 228 KiB/s wr, 42 op/s 2023-09-10T00:04:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2680.782872 secs 2023-09-10T00:04:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:10 smithi082 ceph-mon[98694]: pgmap v1599: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 228 KiB/s wr, 42 op/s 2023-09-10T00:04:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2680.782872 secs 2023-09-10T00:04:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:10 smithi098 ceph-mon[103124]: pgmap v1599: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 228 KiB/s wr, 42 op/s 2023-09-10T00:04:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:12 smithi170 ceph-mon[102832]: pgmap v1600: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 110 KiB/s wr, 31 op/s 2023-09-10T00:04:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:12 smithi082 ceph-mon[98694]: pgmap v1600: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 110 KiB/s wr, 31 op/s 2023-09-10T00:04:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:12 smithi098 ceph-mon[103124]: pgmap v1600: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 110 KiB/s wr, 31 op/s 2023-09-10T00:04:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2683.458531 secs 2023-09-10T00:04:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2685.070047 secs 2023-09-10T00:04:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2683.458531 secs 2023-09-10T00:04:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2685.070047 secs 2023-09-10T00:04:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2683.458531 secs 2023-09-10T00:04:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2685.070047 secs 2023-09-10T00:04:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:14 smithi170 ceph-mon[102832]: pgmap v1601: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 110 KiB/s wr, 31 op/s 2023-09-10T00:04:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:14 smithi170 ceph-mon[102832]: osdmap e174: 12 total, 12 up, 12 in 2023-09-10T00:04:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:14 smithi082 ceph-mon[98694]: pgmap v1601: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 110 KiB/s wr, 31 op/s 2023-09-10T00:04:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:14 smithi082 ceph-mon[98694]: osdmap e174: 12 total, 12 up, 12 in 2023-09-10T00:04:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:14 smithi098 ceph-mon[103124]: pgmap v1601: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 110 KiB/s wr, 31 op/s 2023-09-10T00:04:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:14 smithi098 ceph-mon[103124]: osdmap e174: 12 total, 12 up, 12 in 2023-09-10T00:04:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2685.782940 secs 2023-09-10T00:04:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2685.782940 secs 2023-09-10T00:04:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2685.782940 secs 2023-09-10T00:04:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:16 smithi170 ceph-mon[102832]: pgmap v1603: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-09-10T00:04:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:16 smithi082 ceph-mon[98694]: pgmap v1603: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-09-10T00:04:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:16 smithi098 ceph-mon[103124]: pgmap v1603: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-09-10T00:04:17.437 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:04:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:04:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:04:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:18 smithi170 ceph-mon[102832]: pgmap v1604: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T00:04:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:04:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:04:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:04:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2690.070096 secs 2023-09-10T00:04:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:18 smithi082 ceph-mon[98694]: pgmap v1604: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T00:04:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:04:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:04:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:04:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2690.070096 secs 2023-09-10T00:04:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:18 smithi098 ceph-mon[103124]: pgmap v1604: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T00:04:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:04:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:04:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:04:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2690.070096 secs 2023-09-10T00:04:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:04:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:04:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:04:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2690.783006 secs 2023-09-10T00:04:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:20 smithi170 ceph-mon[102832]: pgmap v1605: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-10T00:04:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:04:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:04:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:04:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2690.783006 secs 2023-09-10T00:04:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:20 smithi082 ceph-mon[98694]: pgmap v1605: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-10T00:04:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:04:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:04:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:04:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2690.783006 secs 2023-09-10T00:04:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:20 smithi098 ceph-mon[103124]: pgmap v1605: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-10T00:04:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:04:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:04:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:04:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:22 smithi170 ceph-mon[102832]: pgmap v1606: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T00:04:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:22 smithi082 ceph-mon[98694]: pgmap v1606: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T00:04:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:22 smithi098 ceph-mon[103124]: pgmap v1606: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T00:04:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2695.070162 secs 2023-09-10T00:04:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2695.070162 secs 2023-09-10T00:04:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2695.070162 secs 2023-09-10T00:04:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:24 smithi170 ceph-mon[102832]: pgmap v1607: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T00:04:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:24 smithi082 ceph-mon[98694]: pgmap v1607: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T00:04:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:24 smithi098 ceph-mon[103124]: pgmap v1607: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T00:04:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2695.783071 secs 2023-09-10T00:04:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2695.783071 secs 2023-09-10T00:04:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2695.783071 secs 2023-09-10T00:04:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:26 smithi170 ceph-mon[102832]: pgmap v1608: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-10T00:04:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:26 smithi082 ceph-mon[98694]: pgmap v1608: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-10T00:04:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:26 smithi098 ceph-mon[103124]: pgmap v1608: 97 pgs: 97 active+clean; 8.8 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-10T00:04:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:28 smithi170 ceph-mon[102832]: pgmap v1609: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 59 KiB/s wr, 15 op/s 2023-09-10T00:04:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2700.070251 secs 2023-09-10T00:04:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:28 smithi082 ceph-mon[98694]: pgmap v1609: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 59 KiB/s wr, 15 op/s 2023-09-10T00:04:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2700.070251 secs 2023-09-10T00:04:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:28 smithi098 ceph-mon[103124]: pgmap v1609: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 59 KiB/s wr, 15 op/s 2023-09-10T00:04:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2700.070251 secs 2023-09-10T00:04:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2700.783138 secs 2023-09-10T00:04:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:30 smithi082 ceph-mon[98694]: pgmap v1610: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T00:04:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2700.783138 secs 2023-09-10T00:04:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:30 smithi098 ceph-mon[103124]: pgmap v1610: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T00:04:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2700.783138 secs 2023-09-10T00:04:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:30 smithi170 ceph-mon[102832]: pgmap v1610: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T00:04:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:32 smithi082 ceph-mon[98694]: pgmap v1611: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 94 KiB/s wr, 16 op/s 2023-09-10T00:04:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:32 smithi098 ceph-mon[103124]: pgmap v1611: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 94 KiB/s wr, 16 op/s 2023-09-10T00:04:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:32 smithi170 ceph-mon[102832]: pgmap v1611: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 94 KiB/s wr, 16 op/s 2023-09-10T00:04:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2705.070309 secs 2023-09-10T00:04:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2705.070309 secs 2023-09-10T00:04:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2705.070309 secs 2023-09-10T00:04:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:34 smithi082 ceph-mon[98694]: pgmap v1612: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T00:04:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:34 smithi098 ceph-mon[103124]: pgmap v1612: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T00:04:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:34 smithi170 ceph-mon[102832]: pgmap v1612: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T00:04:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2705.783204 secs 2023-09-10T00:04:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2705.783204 secs 2023-09-10T00:04:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2705.783204 secs 2023-09-10T00:04:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:36 smithi082 ceph-mon[98694]: pgmap v1613: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T00:04:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:36 smithi098 ceph-mon[103124]: pgmap v1613: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T00:04:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:36 smithi170 ceph-mon[102832]: pgmap v1613: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T00:04:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:38 smithi082 ceph-mon[98694]: pgmap v1614: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 113 KiB/s wr, 18 op/s 2023-09-10T00:04:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2710.070365 secs 2023-09-10T00:04:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:38 smithi098 ceph-mon[103124]: pgmap v1614: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 113 KiB/s wr, 18 op/s 2023-09-10T00:04:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2710.070365 secs 2023-09-10T00:04:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:38 smithi170 ceph-mon[102832]: pgmap v1614: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 113 KiB/s wr, 18 op/s 2023-09-10T00:04:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2710.070365 secs 2023-09-10T00:04:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2710.783269 secs 2023-09-10T00:04:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:40 smithi082 ceph-mon[98694]: pgmap v1615: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 79 KiB/s wr, 12 op/s 2023-09-10T00:04:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2710.783269 secs 2023-09-10T00:04:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:40 smithi098 ceph-mon[103124]: pgmap v1615: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 79 KiB/s wr, 12 op/s 2023-09-10T00:04:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2710.783269 secs 2023-09-10T00:04:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:40 smithi170 ceph-mon[102832]: pgmap v1615: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 79 KiB/s wr, 12 op/s 2023-09-10T00:04:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:42 smithi082 ceph-mon[98694]: pgmap v1616: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 129 KiB/s wr, 19 op/s 2023-09-10T00:04:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:42 smithi098 ceph-mon[103124]: pgmap v1616: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 129 KiB/s wr, 19 op/s 2023-09-10T00:04:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:42 smithi170 ceph-mon[102832]: pgmap v1616: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 129 KiB/s wr, 19 op/s 2023-09-10T00:04:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2715.070472 secs 2023-09-10T00:04:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2715.070472 secs 2023-09-10T00:04:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2715.070472 secs 2023-09-10T00:04:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:44 smithi082 ceph-mon[98694]: pgmap v1617: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-09-10T00:04:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:44 smithi098 ceph-mon[103124]: pgmap v1617: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-09-10T00:04:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:44 smithi170 ceph-mon[102832]: pgmap v1617: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-09-10T00:04:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2715.783336 secs 2023-09-10T00:04:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2715.783336 secs 2023-09-10T00:04:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2715.783336 secs 2023-09-10T00:04:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:46 smithi098 ceph-mon[103124]: pgmap v1618: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-09-10T00:04:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:46 smithi170 ceph-mon[102832]: pgmap v1618: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-09-10T00:04:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:46 smithi082 ceph-mon[98694]: pgmap v1618: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-09-10T00:04:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2718.459123 secs 2023-09-10T00:04:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:48 smithi098 ceph-mon[103124]: pgmap v1619: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 112 KiB/s wr, 19 op/s 2023-09-10T00:04:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2720.070497 secs 2023-09-10T00:04:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2718.459123 secs 2023-09-10T00:04:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:48 smithi170 ceph-mon[102832]: pgmap v1619: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 112 KiB/s wr, 19 op/s 2023-09-10T00:04:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2720.070497 secs 2023-09-10T00:04:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2718.459123 secs 2023-09-10T00:04:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:48 smithi082 ceph-mon[98694]: pgmap v1619: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 112 KiB/s wr, 19 op/s 2023-09-10T00:04:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2720.070497 secs 2023-09-10T00:04:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2720.783401 secs 2023-09-10T00:04:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:50 smithi098 ceph-mon[103124]: pgmap v1620: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T00:04:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2720.783401 secs 2023-09-10T00:04:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:50 smithi170 ceph-mon[102832]: pgmap v1620: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T00:04:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2720.783401 secs 2023-09-10T00:04:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:50 smithi082 ceph-mon[98694]: pgmap v1620: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T00:04:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:04:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:04:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:04:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:04:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:04:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:04:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:04:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:04:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:04:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:52 smithi170 ceph-mon[102832]: pgmap v1621: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 17 op/s 2023-09-10T00:04:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2723.459183 secs 2023-09-10T00:04:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:52 smithi082 ceph-mon[98694]: pgmap v1621: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 17 op/s 2023-09-10T00:04:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2723.459183 secs 2023-09-10T00:04:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:52 smithi098 ceph-mon[103124]: pgmap v1621: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 17 op/s 2023-09-10T00:04:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2723.459183 secs 2023-09-10T00:04:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2725.070558 secs 2023-09-10T00:04:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2725.070558 secs 2023-09-10T00:04:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2725.070558 secs 2023-09-10T00:04:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:54 smithi170 ceph-mon[102832]: pgmap v1622: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T00:04:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2725.783467 secs 2023-09-10T00:04:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:54 smithi082 ceph-mon[98694]: pgmap v1622: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T00:04:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2725.783467 secs 2023-09-10T00:04:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:54 smithi098 ceph-mon[103124]: pgmap v1622: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T00:04:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2725.783467 secs 2023-09-10T00:04:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:56 smithi170 ceph-mon[102832]: pgmap v1623: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T00:04:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:56 smithi082 ceph-mon[98694]: pgmap v1623: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T00:04:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:56 smithi098 ceph-mon[103124]: pgmap v1623: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T00:04:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2728.459256 secs 2023-09-10T00:04:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2728.459256 secs 2023-09-10T00:04:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2728.459256 secs 2023-09-10T00:04:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:58 smithi170 ceph-mon[102832]: pgmap v1624: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 74 KiB/s wr, 16 op/s 2023-09-10T00:04:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:04:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2730.070617 secs 2023-09-10T00:04:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:58 smithi082 ceph-mon[98694]: pgmap v1624: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 74 KiB/s wr, 16 op/s 2023-09-10T00:04:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:04:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2730.070617 secs 2023-09-10T00:04:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:58 smithi098 ceph-mon[103124]: pgmap v1624: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 74 KiB/s wr, 16 op/s 2023-09-10T00:04:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:04:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2730.070617 secs 2023-09-10T00:05:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2730.783489 secs 2023-09-10T00:05:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:00 smithi170 ceph-mon[102832]: pgmap v1625: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 9 op/s 2023-09-10T00:05:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2730.783489 secs 2023-09-10T00:05:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:00 smithi082 ceph-mon[98694]: pgmap v1625: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 9 op/s 2023-09-10T00:05:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2730.783489 secs 2023-09-10T00:05:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:00 smithi098 ceph-mon[103124]: pgmap v1625: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 9 op/s 2023-09-10T00:05:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:02 smithi170 ceph-mon[102832]: pgmap v1626: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 14 op/s 2023-09-10T00:05:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:02 smithi082 ceph-mon[98694]: pgmap v1626: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 14 op/s 2023-09-10T00:05:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:02 smithi098 ceph-mon[103124]: pgmap v1626: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 14 op/s 2023-09-10T00:05:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2733.459314 secs 2023-09-10T00:05:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2735.070677 secs 2023-09-10T00:05:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2733.459314 secs 2023-09-10T00:05:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2735.070677 secs 2023-09-10T00:05:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2733.459314 secs 2023-09-10T00:05:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2735.070677 secs 2023-09-10T00:05:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:04 smithi170 ceph-mon[102832]: pgmap v1627: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 9 op/s 2023-09-10T00:05:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:04 smithi082 ceph-mon[98694]: pgmap v1627: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 9 op/s 2023-09-10T00:05:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:04 smithi098 ceph-mon[103124]: pgmap v1627: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 9 op/s 2023-09-10T00:05:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2735.783554 secs 2023-09-10T00:05:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2735.783554 secs 2023-09-10T00:05:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2735.783554 secs 2023-09-10T00:05:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:06 smithi170 ceph-mon[102832]: pgmap v1628: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 9 op/s 2023-09-10T00:05:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:06 smithi082 ceph-mon[98694]: pgmap v1628: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 9 op/s 2023-09-10T00:05:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:06 smithi098 ceph-mon[103124]: pgmap v1628: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 9 op/s 2023-09-10T00:05:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2738.459373 secs 2023-09-10T00:05:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:08 smithi170 ceph-mon[102832]: pgmap v1629: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 51 KiB/s wr, 26 op/s 2023-09-10T00:05:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2740.070735 secs 2023-09-10T00:05:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2738.459373 secs 2023-09-10T00:05:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:08 smithi082 ceph-mon[98694]: pgmap v1629: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 51 KiB/s wr, 26 op/s 2023-09-10T00:05:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2740.070735 secs 2023-09-10T00:05:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2738.459373 secs 2023-09-10T00:05:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:08 smithi098 ceph-mon[103124]: pgmap v1629: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 51 KiB/s wr, 26 op/s 2023-09-10T00:05:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2740.070735 secs 2023-09-10T00:05:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2740.783619 secs 2023-09-10T00:05:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:10 smithi170 ceph-mon[102832]: pgmap v1630: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 37 KiB/s wr, 21 op/s 2023-09-10T00:05:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2740.783619 secs 2023-09-10T00:05:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:10 smithi082 ceph-mon[98694]: pgmap v1630: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 37 KiB/s wr, 21 op/s 2023-09-10T00:05:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2740.783619 secs 2023-09-10T00:05:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:10 smithi098 ceph-mon[103124]: pgmap v1630: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 37 KiB/s wr, 21 op/s 2023-09-10T00:05:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:12 smithi170 ceph-mon[102832]: pgmap v1631: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 53 KiB/s wr, 26 op/s 2023-09-10T00:05:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:12 smithi082 ceph-mon[98694]: pgmap v1631: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 53 KiB/s wr, 26 op/s 2023-09-10T00:05:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:12 smithi098 ceph-mon[103124]: pgmap v1631: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 53 KiB/s wr, 26 op/s 2023-09-10T00:05:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2743.459471 secs 2023-09-10T00:05:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2745.070799 secs 2023-09-10T00:05:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2743.459471 secs 2023-09-10T00:05:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2745.070799 secs 2023-09-10T00:05:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2743.459471 secs 2023-09-10T00:05:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2745.070799 secs 2023-09-10T00:05:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:14 smithi170 ceph-mon[102832]: pgmap v1632: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 53 KiB/s wr, 22 op/s 2023-09-10T00:05:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:14 smithi082 ceph-mon[98694]: pgmap v1632: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 53 KiB/s wr, 22 op/s 2023-09-10T00:05:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:14 smithi098 ceph-mon[103124]: pgmap v1632: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 53 KiB/s wr, 22 op/s 2023-09-10T00:05:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2745.783686 secs 2023-09-10T00:05:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2745.783686 secs 2023-09-10T00:05:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2745.783686 secs 2023-09-10T00:05:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:16 smithi082 ceph-mon[98694]: pgmap v1633: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 53 KiB/s wr, 22 op/s 2023-09-10T00:05:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:16 smithi098 ceph-mon[103124]: pgmap v1633: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 53 KiB/s wr, 22 op/s 2023-09-10T00:05:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:16 smithi170 ceph-mon[102832]: pgmap v1633: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 53 KiB/s wr, 22 op/s 2023-09-10T00:05:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2748.459540 secs 2023-09-10T00:05:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:18 smithi082 ceph-mon[98694]: pgmap v1634: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T00:05:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:05:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2750.070878 secs 2023-09-10T00:05:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2748.459540 secs 2023-09-10T00:05:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:18 smithi098 ceph-mon[103124]: pgmap v1634: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T00:05:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:05:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2750.070878 secs 2023-09-10T00:05:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2748.459540 secs 2023-09-10T00:05:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:18 smithi170 ceph-mon[102832]: pgmap v1634: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T00:05:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:05:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2750.070878 secs 2023-09-10T00:05:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2750.783754 secs 2023-09-10T00:05:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:20 smithi082 ceph-mon[98694]: pgmap v1635: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 9 op/s 2023-09-10T00:05:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2750.783754 secs 2023-09-10T00:05:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:20 smithi098 ceph-mon[103124]: pgmap v1635: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 9 op/s 2023-09-10T00:05:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2750.783754 secs 2023-09-10T00:05:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:20 smithi170 ceph-mon[102832]: pgmap v1635: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 9 op/s 2023-09-10T00:05:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:05:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:05:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:05:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:05:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:21.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:21.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:21.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:21.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:21 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T00:05:21.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:21 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T00:05:21.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:21.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:21.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:21.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:05:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:05:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:05:21.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:05:21.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:21.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:21.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:21.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:21.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:21 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T00:05:21.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:21 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T00:05:21.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:21.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:21.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:21.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:05:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:05:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:05:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:05:21.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:21.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:21.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:21.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:21.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:21 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T00:05:21.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:21 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T00:05:21.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:21.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:21.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:21.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:22 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T00:05:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:22 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T00:05:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:22 smithi170 ceph-mon[102832]: pgmap v1636: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 101 KiB/s wr, 15 op/s 2023-09-10T00:05:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:05:22.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:05:22.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:22.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:22.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:22.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:22.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:05:22.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:05:22.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:05:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:22 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T00:05:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:22 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T00:05:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:22 smithi082 ceph-mon[98694]: pgmap v1636: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 101 KiB/s wr, 15 op/s 2023-09-10T00:05:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:05:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:05:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:22.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:22.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:05:22.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:05:22.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:05:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:22 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T00:05:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:22 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T00:05:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:22 smithi098 ceph-mon[103124]: pgmap v1636: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 101 KiB/s wr, 15 op/s 2023-09-10T00:05:22.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:05:22.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:05:22.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:22.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:22.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:22.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:05:22.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:05:22.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:05:22.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:05:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:23 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T00:05:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:23 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T00:05:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:05:23.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2755.070946 secs 2023-09-10T00:05:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:23 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T00:05:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:23 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T00:05:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:05:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2755.070946 secs 2023-09-10T00:05:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:23 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T00:05:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:23 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T00:05:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:05:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2755.070946 secs 2023-09-10T00:05:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:24 smithi170 ceph-mon[102832]: pgmap v1637: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T00:05:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:05:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:05:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:05:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2755.783817 secs 2023-09-10T00:05:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:24 smithi082 ceph-mon[98694]: pgmap v1637: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T00:05:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:05:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:05:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:05:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2755.783817 secs 2023-09-10T00:05:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:24 smithi098 ceph-mon[103124]: pgmap v1637: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T00:05:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:05:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:05:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:05:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2755.783817 secs 2023-09-10T00:05:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:26 smithi170 ceph-mon[102832]: pgmap v1638: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T00:05:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:26 smithi082 ceph-mon[98694]: pgmap v1638: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T00:05:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:26 smithi098 ceph-mon[103124]: pgmap v1638: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T00:05:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:28 smithi170 ceph-mon[102832]: pgmap v1639: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 111 KiB/s wr, 17 op/s 2023-09-10T00:05:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2760.071023 secs 2023-09-10T00:05:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:28 smithi082 ceph-mon[98694]: pgmap v1639: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 111 KiB/s wr, 17 op/s 2023-09-10T00:05:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2760.071023 secs 2023-09-10T00:05:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:28 smithi098 ceph-mon[103124]: pgmap v1639: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 111 KiB/s wr, 17 op/s 2023-09-10T00:05:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2760.071023 secs 2023-09-10T00:05:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2760.783882 secs 2023-09-10T00:05:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:30 smithi170 ceph-mon[102832]: pgmap v1640: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T00:05:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2760.783882 secs 2023-09-10T00:05:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:30 smithi082 ceph-mon[98694]: pgmap v1640: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T00:05:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2760.783882 secs 2023-09-10T00:05:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:30 smithi098 ceph-mon[103124]: pgmap v1640: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T00:05:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:32 smithi170 ceph-mon[102832]: pgmap v1641: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 160 KiB/s wr, 18 op/s 2023-09-10T00:05:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:05:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:32 smithi082 ceph-mon[98694]: pgmap v1641: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 160 KiB/s wr, 18 op/s 2023-09-10T00:05:32.535 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:05:32.536 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:32 smithi098 ceph-mon[103124]: pgmap v1641: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 160 KiB/s wr, 18 op/s 2023-09-10T00:05:32.536 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:05:33.435 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2763.459790 secs 2023-09-10T00:05:33.435 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2765.071080 secs 2023-09-10T00:05:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2763.459790 secs 2023-09-10T00:05:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2765.071080 secs 2023-09-10T00:05:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2763.459790 secs 2023-09-10T00:05:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2765.071080 secs 2023-09-10T00:05:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:34 smithi170 ceph-mon[102832]: pgmap v1642: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 77 KiB/s wr, 12 op/s 2023-09-10T00:05:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:34 smithi082 ceph-mon[98694]: pgmap v1642: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 77 KiB/s wr, 12 op/s 2023-09-10T00:05:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:34 smithi098 ceph-mon[103124]: pgmap v1642: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 77 KiB/s wr, 12 op/s 2023-09-10T00:05:35.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2765.783973 secs 2023-09-10T00:05:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2765.783973 secs 2023-09-10T00:05:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2765.783973 secs 2023-09-10T00:05:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:36 smithi170 ceph-mon[102832]: pgmap v1643: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 77 KiB/s wr, 12 op/s 2023-09-10T00:05:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:36 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:05:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:05:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:05:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:36 smithi082 ceph-mon[98694]: pgmap v1643: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 77 KiB/s wr, 12 op/s 2023-09-10T00:05:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:36 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:05:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:05:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:05:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:36 smithi098 ceph-mon[103124]: pgmap v1643: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 77 KiB/s wr, 12 op/s 2023-09-10T00:05:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:36 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:05:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:05:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:05:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2768.459847 secs 2023-09-10T00:05:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:38 smithi170 ceph-mon[102832]: pgmap v1644: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 137 KiB/s wr, 20 op/s 2023-09-10T00:05:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2770.071167 secs 2023-09-10T00:05:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2768.459847 secs 2023-09-10T00:05:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:38 smithi082 ceph-mon[98694]: pgmap v1644: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 137 KiB/s wr, 20 op/s 2023-09-10T00:05:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2770.071167 secs 2023-09-10T00:05:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2768.459847 secs 2023-09-10T00:05:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:38 smithi098 ceph-mon[103124]: pgmap v1644: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 137 KiB/s wr, 20 op/s 2023-09-10T00:05:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2770.071167 secs 2023-09-10T00:05:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2770.784042 secs 2023-09-10T00:05:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:40 smithi082 ceph-mon[98694]: pgmap v1645: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 110 KiB/s wr, 13 op/s 2023-09-10T00:05:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2770.784042 secs 2023-09-10T00:05:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:40 smithi098 ceph-mon[103124]: pgmap v1645: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 110 KiB/s wr, 13 op/s 2023-09-10T00:05:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2770.784042 secs 2023-09-10T00:05:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:40 smithi170 ceph-mon[102832]: pgmap v1645: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 110 KiB/s wr, 13 op/s 2023-09-10T00:05:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:42 smithi082 ceph-mon[98694]: pgmap v1646: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 147 KiB/s wr, 18 op/s 2023-09-10T00:05:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:42 smithi098 ceph-mon[103124]: pgmap v1646: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 147 KiB/s wr, 18 op/s 2023-09-10T00:05:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:42 smithi170 ceph-mon[102832]: pgmap v1646: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 147 KiB/s wr, 18 op/s 2023-09-10T00:05:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2773.459907 secs 2023-09-10T00:05:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2775.071221 secs 2023-09-10T00:05:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2773.459907 secs 2023-09-10T00:05:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2775.071221 secs 2023-09-10T00:05:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2773.459907 secs 2023-09-10T00:05:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2775.071221 secs 2023-09-10T00:05:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:44 smithi082 ceph-mon[98694]: pgmap v1647: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T00:05:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:44 smithi098 ceph-mon[103124]: pgmap v1647: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T00:05:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:44 smithi170 ceph-mon[102832]: pgmap v1647: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T00:05:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2775.784107 secs 2023-09-10T00:05:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2775.784107 secs 2023-09-10T00:05:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2775.784107 secs 2023-09-10T00:05:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:46 smithi082 ceph-mon[98694]: pgmap v1648: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T00:05:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:46 smithi098 ceph-mon[103124]: pgmap v1648: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T00:05:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:46 smithi170 ceph-mon[102832]: pgmap v1648: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T00:05:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2778.459966 secs 2023-09-10T00:05:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:48 smithi082 ceph-mon[98694]: pgmap v1649: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 119 KiB/s wr, 17 op/s 2023-09-10T00:05:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2780.071279 secs 2023-09-10T00:05:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2778.459966 secs 2023-09-10T00:05:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:48 smithi098 ceph-mon[103124]: pgmap v1649: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 119 KiB/s wr, 17 op/s 2023-09-10T00:05:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2780.071279 secs 2023-09-10T00:05:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2778.459966 secs 2023-09-10T00:05:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:48 smithi170 ceph-mon[102832]: pgmap v1649: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 119 KiB/s wr, 17 op/s 2023-09-10T00:05:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2780.071279 secs 2023-09-10T00:05:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2780.784175 secs 2023-09-10T00:05:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:50 smithi082 ceph-mon[98694]: pgmap v1650: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T00:05:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2780.784175 secs 2023-09-10T00:05:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:50 smithi098 ceph-mon[103124]: pgmap v1650: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T00:05:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2780.784175 secs 2023-09-10T00:05:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:50 smithi170 ceph-mon[102832]: pgmap v1650: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T00:05:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:52 smithi082 ceph-mon[98694]: pgmap v1651: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 72 KiB/s wr, 14 op/s 2023-09-10T00:05:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:52 smithi098 ceph-mon[103124]: pgmap v1651: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 72 KiB/s wr, 14 op/s 2023-09-10T00:05:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:52 smithi170 ceph-mon[102832]: pgmap v1651: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 72 KiB/s wr, 14 op/s 2023-09-10T00:05:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2783.460026 secs 2023-09-10T00:05:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2785.071337 secs 2023-09-10T00:05:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2783.460026 secs 2023-09-10T00:05:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2785.071337 secs 2023-09-10T00:05:53.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2783.460026 secs 2023-09-10T00:05:53.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2785.071337 secs 2023-09-10T00:05:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:54 smithi098 ceph-mon[103124]: pgmap v1652: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 35 KiB/s wr, 9 op/s 2023-09-10T00:05:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:54 smithi170 ceph-mon[102832]: pgmap v1652: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 35 KiB/s wr, 9 op/s 2023-09-10T00:05:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:54 smithi082 ceph-mon[98694]: pgmap v1652: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 35 KiB/s wr, 9 op/s 2023-09-10T00:05:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2785.784241 secs 2023-09-10T00:05:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2785.784241 secs 2023-09-10T00:05:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2785.784241 secs 2023-09-10T00:05:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:56 smithi098 ceph-mon[103124]: pgmap v1653: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 35 KiB/s wr, 9 op/s 2023-09-10T00:05:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:56 smithi170 ceph-mon[102832]: pgmap v1653: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 35 KiB/s wr, 9 op/s 2023-09-10T00:05:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:56 smithi082 ceph-mon[98694]: pgmap v1653: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 35 KiB/s wr, 9 op/s 2023-09-10T00:05:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2788.460129 secs 2023-09-10T00:05:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:58 smithi098 ceph-mon[103124]: pgmap v1654: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 38 KiB/s wr, 14 op/s 2023-09-10T00:05:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:05:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2790.071422 secs 2023-09-10T00:05:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2788.460129 secs 2023-09-10T00:05:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:58 smithi170 ceph-mon[102832]: pgmap v1654: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 38 KiB/s wr, 14 op/s 2023-09-10T00:05:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:05:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2790.071422 secs 2023-09-10T00:05:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2788.460129 secs 2023-09-10T00:05:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:58 smithi082 ceph-mon[98694]: pgmap v1654: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 38 KiB/s wr, 14 op/s 2023-09-10T00:05:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:05:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2790.071422 secs 2023-09-10T00:06:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2790.784309 secs 2023-09-10T00:06:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:00 smithi098 ceph-mon[103124]: pgmap v1655: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 KiB/s wr, 9 op/s 2023-09-10T00:06:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2790.784309 secs 2023-09-10T00:06:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:00 smithi170 ceph-mon[102832]: pgmap v1655: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 KiB/s wr, 9 op/s 2023-09-10T00:06:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2790.784309 secs 2023-09-10T00:06:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:00 smithi082 ceph-mon[98694]: pgmap v1655: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 KiB/s wr, 9 op/s 2023-09-10T00:06:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:02 smithi098 ceph-mon[103124]: pgmap v1656: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-09-10T00:06:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:02 smithi170 ceph-mon[102832]: pgmap v1656: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-09-10T00:06:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:02 smithi082 ceph-mon[98694]: pgmap v1656: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-09-10T00:06:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2793.460189 secs 2023-09-10T00:06:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2795.071501 secs 2023-09-10T00:06:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2793.460189 secs 2023-09-10T00:06:03.742 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2795.071501 secs 2023-09-10T00:06:03.743 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2793.460189 secs 2023-09-10T00:06:03.743 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2795.071501 secs 2023-09-10T00:06:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:04 smithi098 ceph-mon[103124]: pgmap v1657: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2023-09-10T00:06:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2795.784373 secs 2023-09-10T00:06:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:04 smithi170 ceph-mon[102832]: pgmap v1657: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2023-09-10T00:06:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2795.784373 secs 2023-09-10T00:06:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:04 smithi082 ceph-mon[98694]: pgmap v1657: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2023-09-10T00:06:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2795.784373 secs 2023-09-10T00:06:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:06 smithi098 ceph-mon[103124]: pgmap v1658: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2023-09-10T00:06:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:06 smithi170 ceph-mon[102832]: pgmap v1658: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2023-09-10T00:06:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:06 smithi082 ceph-mon[98694]: pgmap v1658: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2023-09-10T00:06:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2798.460250 secs 2023-09-10T00:06:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:08 smithi170 ceph-mon[102832]: pgmap v1659: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T00:06:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:08 smithi170 ceph-mon[102832]: osdmap e175: 12 total, 12 up, 12 in 2023-09-10T00:06:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2800.071558 secs 2023-09-10T00:06:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2798.460250 secs 2023-09-10T00:06:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:08 smithi082 ceph-mon[98694]: pgmap v1659: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T00:06:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:08 smithi082 ceph-mon[98694]: osdmap e175: 12 total, 12 up, 12 in 2023-09-10T00:06:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2800.071558 secs 2023-09-10T00:06:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2798.460250 secs 2023-09-10T00:06:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:08 smithi098 ceph-mon[103124]: pgmap v1659: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T00:06:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:08 smithi098 ceph-mon[103124]: osdmap e175: 12 total, 12 up, 12 in 2023-09-10T00:06:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2800.071558 secs 2023-09-10T00:06:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2800.784447 secs 2023-09-10T00:06:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:10 smithi170 ceph-mon[102832]: pgmap v1661: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 75 KiB/s wr, 29 op/s 2023-09-10T00:06:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2800.784447 secs 2023-09-10T00:06:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:10 smithi082 ceph-mon[98694]: pgmap v1661: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 75 KiB/s wr, 29 op/s 2023-09-10T00:06:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2800.784447 secs 2023-09-10T00:06:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:10 smithi098 ceph-mon[103124]: pgmap v1661: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 75 KiB/s wr, 29 op/s 2023-09-10T00:06:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:12 smithi170 ceph-mon[102832]: pgmap v1662: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 74 KiB/s wr, 28 op/s 2023-09-10T00:06:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:12 smithi082 ceph-mon[98694]: pgmap v1662: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 74 KiB/s wr, 28 op/s 2023-09-10T00:06:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:12 smithi098 ceph-mon[103124]: pgmap v1662: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 74 KiB/s wr, 28 op/s 2023-09-10T00:06:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2803.460311 secs 2023-09-10T00:06:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2805.071629 secs 2023-09-10T00:06:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2803.460311 secs 2023-09-10T00:06:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2805.071629 secs 2023-09-10T00:06:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2803.460311 secs 2023-09-10T00:06:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2805.071629 secs 2023-09-10T00:06:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:14 smithi082 ceph-mon[98694]: pgmap v1663: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 74 KiB/s wr, 28 op/s 2023-09-10T00:06:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:14 smithi082 ceph-mon[98694]: osdmap e176: 12 total, 12 up, 12 in 2023-09-10T00:06:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:14 smithi098 ceph-mon[103124]: pgmap v1663: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 74 KiB/s wr, 28 op/s 2023-09-10T00:06:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:14 smithi098 ceph-mon[103124]: osdmap e176: 12 total, 12 up, 12 in 2023-09-10T00:06:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:14 smithi170 ceph-mon[102832]: pgmap v1663: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 74 KiB/s wr, 28 op/s 2023-09-10T00:06:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:14 smithi170 ceph-mon[102832]: osdmap e176: 12 total, 12 up, 12 in 2023-09-10T00:06:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2805.784489 secs 2023-09-10T00:06:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2805.784489 secs 2023-09-10T00:06:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2805.784489 secs 2023-09-10T00:06:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:16 smithi082 ceph-mon[98694]: pgmap v1665: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 383 B/s wr, 6 op/s 2023-09-10T00:06:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:16 smithi098 ceph-mon[103124]: pgmap v1665: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 383 B/s wr, 6 op/s 2023-09-10T00:06:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:16 smithi170 ceph-mon[102832]: pgmap v1665: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 383 B/s wr, 6 op/s 2023-09-10T00:06:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2808.460372 secs 2023-09-10T00:06:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:18 smithi082 ceph-mon[98694]: pgmap v1666: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 519 B/s wr, 10 op/s 2023-09-10T00:06:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2810.071691 secs 2023-09-10T00:06:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2808.460372 secs 2023-09-10T00:06:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:18 smithi098 ceph-mon[103124]: pgmap v1666: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 519 B/s wr, 10 op/s 2023-09-10T00:06:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2810.071691 secs 2023-09-10T00:06:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2808.460372 secs 2023-09-10T00:06:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:18 smithi170 ceph-mon[102832]: pgmap v1666: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 519 B/s wr, 10 op/s 2023-09-10T00:06:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2810.071691 secs 2023-09-10T00:06:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2810.784552 secs 2023-09-10T00:06:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:20 smithi082 ceph-mon[98694]: pgmap v1667: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 511 B/s wr, 10 op/s 2023-09-10T00:06:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2810.784552 secs 2023-09-10T00:06:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:20 smithi098 ceph-mon[103124]: pgmap v1667: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 511 B/s wr, 10 op/s 2023-09-10T00:06:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2810.784552 secs 2023-09-10T00:06:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:20 smithi170 ceph-mon[102832]: pgmap v1667: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 511 B/s wr, 10 op/s 2023-09-10T00:06:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:22 smithi082 ceph-mon[98694]: pgmap v1668: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T00:06:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:06:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:22 smithi098 ceph-mon[103124]: pgmap v1668: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T00:06:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:06:22.714 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:22 smithi170 ceph-mon[102832]: pgmap v1668: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T00:06:22.714 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:06:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:06:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2815.071755 secs 2023-09-10T00:06:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:06:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:06:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:06:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:06:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2815.071755 secs 2023-09-10T00:06:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:06:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:06:23.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:06:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:06:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2815.071755 secs 2023-09-10T00:06:23.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:06:23.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:06:23.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:06:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:24 smithi082 ceph-mon[98694]: pgmap v1669: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T00:06:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:06:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:06:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:06:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:24 smithi098 ceph-mon[103124]: pgmap v1669: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T00:06:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:06:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:06:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:06:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:24 smithi170 ceph-mon[102832]: pgmap v1669: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T00:06:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:06:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:06:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:06:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2815.784619 secs 2023-09-10T00:06:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2815.784619 secs 2023-09-10T00:06:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2815.784619 secs 2023-09-10T00:06:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:26 smithi082 ceph-mon[98694]: pgmap v1670: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 10 op/s 2023-09-10T00:06:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:26 smithi098 ceph-mon[103124]: pgmap v1670: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 10 op/s 2023-09-10T00:06:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:26 smithi170 ceph-mon[102832]: pgmap v1670: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 10 op/s 2023-09-10T00:06:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:28 smithi082 ceph-mon[98694]: pgmap v1671: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 54 KiB/s wr, 14 op/s 2023-09-10T00:06:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2820.071814 secs 2023-09-10T00:06:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:28 smithi098 ceph-mon[103124]: pgmap v1671: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 54 KiB/s wr, 14 op/s 2023-09-10T00:06:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2820.071814 secs 2023-09-10T00:06:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:28 smithi170 ceph-mon[102832]: pgmap v1671: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 54 KiB/s wr, 14 op/s 2023-09-10T00:06:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2820.071814 secs 2023-09-10T00:06:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2820.784687 secs 2023-09-10T00:06:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:30 smithi098 ceph-mon[103124]: pgmap v1672: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 10 op/s 2023-09-10T00:06:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2820.784687 secs 2023-09-10T00:06:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:30 smithi170 ceph-mon[102832]: pgmap v1672: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 10 op/s 2023-09-10T00:06:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2820.784687 secs 2023-09-10T00:06:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:30 smithi082 ceph-mon[98694]: pgmap v1672: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 10 op/s 2023-09-10T00:06:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:32 smithi098 ceph-mon[103124]: pgmap v1673: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 87 KiB/s wr, 16 op/s 2023-09-10T00:06:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:32 smithi170 ceph-mon[102832]: pgmap v1673: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 87 KiB/s wr, 16 op/s 2023-09-10T00:06:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:32 smithi082 ceph-mon[98694]: pgmap v1673: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 87 KiB/s wr, 16 op/s 2023-09-10T00:06:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2825.071873 secs 2023-09-10T00:06:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2825.071873 secs 2023-09-10T00:06:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2825.071873 secs 2023-09-10T00:06:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:34 smithi098 ceph-mon[103124]: pgmap v1674: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T00:06:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:34 smithi170 ceph-mon[102832]: pgmap v1674: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T00:06:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:34 smithi082 ceph-mon[98694]: pgmap v1674: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T00:06:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2825.784757 secs 2023-09-10T00:06:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2825.784757 secs 2023-09-10T00:06:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2825.784757 secs 2023-09-10T00:06:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:36 smithi098 ceph-mon[103124]: pgmap v1675: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T00:06:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:36 smithi170 ceph-mon[102832]: pgmap v1675: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T00:06:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:36 smithi082 ceph-mon[98694]: pgmap v1675: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T00:06:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:38 smithi098 ceph-mon[103124]: pgmap v1676: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 90 KiB/s wr, 16 op/s 2023-09-10T00:06:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2830.071930 secs 2023-09-10T00:06:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:38 smithi170 ceph-mon[102832]: pgmap v1676: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 90 KiB/s wr, 16 op/s 2023-09-10T00:06:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2830.071930 secs 2023-09-10T00:06:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:38 smithi082 ceph-mon[98694]: pgmap v1676: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 90 KiB/s wr, 16 op/s 2023-09-10T00:06:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2830.071930 secs 2023-09-10T00:06:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2830.784822 secs 2023-09-10T00:06:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:40 smithi098 ceph-mon[103124]: pgmap v1677: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 54 KiB/s wr, 11 op/s 2023-09-10T00:06:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2830.784822 secs 2023-09-10T00:06:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:40 smithi170 ceph-mon[102832]: pgmap v1677: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 54 KiB/s wr, 11 op/s 2023-09-10T00:06:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2830.784822 secs 2023-09-10T00:06:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:40 smithi082 ceph-mon[98694]: pgmap v1677: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 54 KiB/s wr, 11 op/s 2023-09-10T00:06:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:42 smithi098 ceph-mon[103124]: pgmap v1678: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 67 KiB/s wr, 17 op/s 2023-09-10T00:06:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:42 smithi170 ceph-mon[102832]: pgmap v1678: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 67 KiB/s wr, 17 op/s 2023-09-10T00:06:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:42 smithi082 ceph-mon[98694]: pgmap v1678: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 67 KiB/s wr, 17 op/s 2023-09-10T00:06:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2835.071988 secs 2023-09-10T00:06:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2835.071988 secs 2023-09-10T00:06:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2835.071988 secs 2023-09-10T00:06:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:44 smithi098 ceph-mon[103124]: pgmap v1679: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T00:06:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2835.784889 secs 2023-09-10T00:06:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:44 smithi170 ceph-mon[102832]: pgmap v1679: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T00:06:44.742 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2835.784889 secs 2023-09-10T00:06:44.742 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:44 smithi082 ceph-mon[98694]: pgmap v1679: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T00:06:44.743 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2835.784889 secs 2023-09-10T00:06:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:46 smithi098 ceph-mon[103124]: pgmap v1680: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T00:06:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:46 smithi170 ceph-mon[102832]: pgmap v1680: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T00:06:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:46 smithi082 ceph-mon[98694]: pgmap v1680: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T00:06:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:48 smithi098 ceph-mon[103124]: pgmap v1681: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 99 KiB/s wr, 16 op/s 2023-09-10T00:06:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2840.072073 secs 2023-09-10T00:06:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:48 smithi170 ceph-mon[102832]: pgmap v1681: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 99 KiB/s wr, 16 op/s 2023-09-10T00:06:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2840.072073 secs 2023-09-10T00:06:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:48 smithi082 ceph-mon[98694]: pgmap v1681: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 99 KiB/s wr, 16 op/s 2023-09-10T00:06:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2840.072073 secs 2023-09-10T00:06:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2840.784953 secs 2023-09-10T00:06:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:50 smithi170 ceph-mon[102832]: pgmap v1682: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T00:06:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2840.784953 secs 2023-09-10T00:06:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:50 smithi082 ceph-mon[98694]: pgmap v1682: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T00:06:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2840.784953 secs 2023-09-10T00:06:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:50 smithi098 ceph-mon[103124]: pgmap v1682: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T00:06:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:52 smithi170 ceph-mon[102832]: pgmap v1683: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 118 KiB/s wr, 17 op/s 2023-09-10T00:06:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:52 smithi082 ceph-mon[98694]: pgmap v1683: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 118 KiB/s wr, 17 op/s 2023-09-10T00:06:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:52 smithi098 ceph-mon[103124]: pgmap v1683: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 118 KiB/s wr, 17 op/s 2023-09-10T00:06:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2843.460889 secs 2023-09-10T00:06:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2845.072130 secs 2023-09-10T00:06:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2843.460889 secs 2023-09-10T00:06:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2845.072130 secs 2023-09-10T00:06:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2843.460889 secs 2023-09-10T00:06:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2845.072130 secs 2023-09-10T00:06:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:54 smithi082 ceph-mon[98694]: pgmap v1684: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T00:06:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:54 smithi098 ceph-mon[103124]: pgmap v1684: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T00:06:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:54 smithi170 ceph-mon[102832]: pgmap v1684: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T00:06:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2845.785018 secs 2023-09-10T00:06:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2845.785018 secs 2023-09-10T00:06:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2845.785018 secs 2023-09-10T00:06:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:56 smithi082 ceph-mon[98694]: pgmap v1685: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T00:06:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:06:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:06:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:06:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:56 smithi098 ceph-mon[103124]: pgmap v1685: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T00:06:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:06:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:06:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:06:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:56 smithi170 ceph-mon[102832]: pgmap v1685: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T00:06:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:06:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:06:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:06:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2848.460947 secs 2023-09-10T00:06:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:58 smithi082 ceph-mon[98694]: pgmap v1686: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 140 KiB/s wr, 17 op/s 2023-09-10T00:06:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:06:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2850.072188 secs 2023-09-10T00:06:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2848.460947 secs 2023-09-10T00:06:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:58 smithi098 ceph-mon[103124]: pgmap v1686: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 140 KiB/s wr, 17 op/s 2023-09-10T00:06:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:06:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2850.072188 secs 2023-09-10T00:06:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2848.460947 secs 2023-09-10T00:06:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:58 smithi170 ceph-mon[102832]: pgmap v1686: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 140 KiB/s wr, 17 op/s 2023-09-10T00:06:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:06:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2850.072188 secs 2023-09-10T00:07:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2850.785088 secs 2023-09-10T00:07:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:00 smithi082 ceph-mon[98694]: pgmap v1687: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T00:07:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2850.785088 secs 2023-09-10T00:07:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:00 smithi098 ceph-mon[103124]: pgmap v1687: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T00:07:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2850.785088 secs 2023-09-10T00:07:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:00 smithi170 ceph-mon[102832]: pgmap v1687: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T00:07:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:02 smithi082 ceph-mon[98694]: pgmap v1688: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 90 KiB/s wr, 16 op/s 2023-09-10T00:07:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:02 smithi098 ceph-mon[103124]: pgmap v1688: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 90 KiB/s wr, 16 op/s 2023-09-10T00:07:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:02 smithi170 ceph-mon[102832]: pgmap v1688: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 90 KiB/s wr, 16 op/s 2023-09-10T00:07:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2853.461030 secs 2023-09-10T00:07:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2855.072246 secs 2023-09-10T00:07:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2853.461030 secs 2023-09-10T00:07:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2855.072246 secs 2023-09-10T00:07:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2853.461030 secs 2023-09-10T00:07:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2855.072246 secs 2023-09-10T00:07:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:04 smithi082 ceph-mon[98694]: pgmap v1689: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T00:07:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:04 smithi098 ceph-mon[103124]: pgmap v1689: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T00:07:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:04 smithi170 ceph-mon[102832]: pgmap v1689: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T00:07:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2855.785153 secs 2023-09-10T00:07:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2855.785153 secs 2023-09-10T00:07:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2855.785153 secs 2023-09-10T00:07:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:06 smithi098 ceph-mon[103124]: pgmap v1690: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T00:07:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:06 smithi170 ceph-mon[102832]: pgmap v1690: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T00:07:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:06 smithi082 ceph-mon[98694]: pgmap v1690: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T00:07:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2858.461121 secs 2023-09-10T00:07:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:08 smithi098 ceph-mon[103124]: pgmap v1691: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 107 KiB/s wr, 30 op/s 2023-09-10T00:07:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:08 smithi098 ceph-mon[103124]: osdmap e177: 12 total, 12 up, 12 in 2023-09-10T00:07:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2860.072303 secs 2023-09-10T00:07:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2858.461121 secs 2023-09-10T00:07:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:08 smithi170 ceph-mon[102832]: pgmap v1691: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 107 KiB/s wr, 30 op/s 2023-09-10T00:07:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:08 smithi170 ceph-mon[102832]: osdmap e177: 12 total, 12 up, 12 in 2023-09-10T00:07:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2860.072303 secs 2023-09-10T00:07:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2858.461121 secs 2023-09-10T00:07:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:08 smithi082 ceph-mon[98694]: pgmap v1691: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 107 KiB/s wr, 30 op/s 2023-09-10T00:07:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:08 smithi082 ceph-mon[98694]: osdmap e177: 12 total, 12 up, 12 in 2023-09-10T00:07:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2860.072303 secs 2023-09-10T00:07:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2860.785217 secs 2023-09-10T00:07:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:10 smithi098 ceph-mon[103124]: pgmap v1693: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T00:07:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2860.785217 secs 2023-09-10T00:07:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:10 smithi170 ceph-mon[102832]: pgmap v1693: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T00:07:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2860.785217 secs 2023-09-10T00:07:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:10 smithi082 ceph-mon[98694]: pgmap v1693: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T00:07:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:12 smithi098 ceph-mon[103124]: pgmap v1694: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 89 KiB/s wr, 30 op/s 2023-09-10T00:07:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:12 smithi170 ceph-mon[102832]: pgmap v1694: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 89 KiB/s wr, 30 op/s 2023-09-10T00:07:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:12 smithi082 ceph-mon[98694]: pgmap v1694: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 89 KiB/s wr, 30 op/s 2023-09-10T00:07:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2863.461182 secs 2023-09-10T00:07:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2865.072361 secs 2023-09-10T00:07:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2863.461182 secs 2023-09-10T00:07:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2865.072361 secs 2023-09-10T00:07:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2863.461182 secs 2023-09-10T00:07:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2865.072361 secs 2023-09-10T00:07:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:14 smithi098 ceph-mon[103124]: pgmap v1695: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 89 KiB/s wr, 30 op/s 2023-09-10T00:07:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:14 smithi098 ceph-mon[103124]: osdmap e178: 12 total, 12 up, 12 in 2023-09-10T00:07:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:14 smithi170 ceph-mon[102832]: pgmap v1695: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 89 KiB/s wr, 30 op/s 2023-09-10T00:07:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:14 smithi170 ceph-mon[102832]: osdmap e178: 12 total, 12 up, 12 in 2023-09-10T00:07:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:14 smithi082 ceph-mon[98694]: pgmap v1695: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 89 KiB/s wr, 30 op/s 2023-09-10T00:07:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:14 smithi082 ceph-mon[98694]: osdmap e178: 12 total, 12 up, 12 in 2023-09-10T00:07:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2865.785284 secs 2023-09-10T00:07:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2865.785284 secs 2023-09-10T00:07:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2865.785284 secs 2023-09-10T00:07:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:16 smithi098 ceph-mon[103124]: pgmap v1697: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 26 KiB/s wr, 8 op/s 2023-09-10T00:07:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:16 smithi170 ceph-mon[102832]: pgmap v1697: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 26 KiB/s wr, 8 op/s 2023-09-10T00:07:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:16 smithi082 ceph-mon[98694]: pgmap v1697: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 26 KiB/s wr, 8 op/s 2023-09-10T00:07:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2868.461258 secs 2023-09-10T00:07:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:18 smithi098 ceph-mon[103124]: pgmap v1698: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T00:07:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2870.072419 secs 2023-09-10T00:07:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2868.461258 secs 2023-09-10T00:07:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:18 smithi170 ceph-mon[102832]: pgmap v1698: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T00:07:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2870.072419 secs 2023-09-10T00:07:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2868.461258 secs 2023-09-10T00:07:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:18 smithi082 ceph-mon[98694]: pgmap v1698: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T00:07:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2870.072419 secs 2023-09-10T00:07:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2870.785349 secs 2023-09-10T00:07:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:20 smithi098 ceph-mon[103124]: pgmap v1699: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T00:07:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2870.785349 secs 2023-09-10T00:07:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:20 smithi170 ceph-mon[102832]: pgmap v1699: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T00:07:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2870.785349 secs 2023-09-10T00:07:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:20 smithi082 ceph-mon[98694]: pgmap v1699: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T00:07:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:22 smithi098 ceph-mon[103124]: pgmap v1700: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T00:07:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:22 smithi170 ceph-mon[102832]: pgmap v1700: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T00:07:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:22 smithi082 ceph-mon[98694]: pgmap v1700: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T00:07:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:07:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2875.072517 secs 2023-09-10T00:07:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:07:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:07:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2875.072517 secs 2023-09-10T00:07:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:07:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:07:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2875.072517 secs 2023-09-10T00:07:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:07:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:24 smithi098 ceph-mon[103124]: pgmap v1701: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T00:07:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:07:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:07:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:07:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:07:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:07:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2875.785413 secs 2023-09-10T00:07:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:07:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:07:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:07:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:24 smithi170 ceph-mon[102832]: pgmap v1701: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T00:07:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:07:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:07:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:07:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:07:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:07:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2875.785413 secs 2023-09-10T00:07:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:07:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:07:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:07:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:24 smithi082 ceph-mon[98694]: pgmap v1701: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T00:07:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:07:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:07:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:07:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:07:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:07:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2875.785413 secs 2023-09-10T00:07:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:07:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:07:24.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:07:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:26 smithi098 ceph-mon[103124]: pgmap v1702: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T00:07:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:26 smithi170 ceph-mon[102832]: pgmap v1702: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T00:07:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:26 smithi082 ceph-mon[98694]: pgmap v1702: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T00:07:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:28 smithi098 ceph-mon[103124]: pgmap v1703: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 114 KiB/s wr, 16 op/s 2023-09-10T00:07:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:28 smithi098 ceph-mon[103124]: 3 slow requests, 3 included below; oldest blocked for > 2880.072602 secs 2023-09-10T00:07:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:28 smithi098 ceph-mon[103124]: slow request 2880.072601 seconds old, received at 2023-09-09T23:19:27.740042+0000: client_request(mds.4:47 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently acquired locks 2023-09-10T00:07:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:28 smithi098 ceph-mon[103124]: slow request 2880.072222 seconds old, received at 2023-09-09T23:19:27.740422+0000: client_request(mds.4:48 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently requesting remote authpins 2023-09-10T00:07:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:28 smithi098 ceph-mon[103124]: slow request 2880.070000 seconds old, received at 2023-09-09T23:19:27.742643+0000: client_request(client.24913:3869 unlink #0x10000000007/00000001000000010000000B 2023-09-09T23:19:27.741863+0000 caller_uid=26, caller_gid=26{26,}) currently requesting remote authpins 2023-09-10T00:07:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:28 smithi170 ceph-mon[102832]: pgmap v1703: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 114 KiB/s wr, 16 op/s 2023-09-10T00:07:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:28 smithi170 ceph-mon[102832]: 3 slow requests, 3 included below; oldest blocked for > 2880.072602 secs 2023-09-10T00:07:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:28 smithi170 ceph-mon[102832]: slow request 2880.072601 seconds old, received at 2023-09-09T23:19:27.740042+0000: client_request(mds.4:47 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently acquired locks 2023-09-10T00:07:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:28 smithi170 ceph-mon[102832]: slow request 2880.072222 seconds old, received at 2023-09-09T23:19:27.740422+0000: client_request(mds.4:48 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently requesting remote authpins 2023-09-10T00:07:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:28 smithi170 ceph-mon[102832]: slow request 2880.070000 seconds old, received at 2023-09-09T23:19:27.742643+0000: client_request(client.24913:3869 unlink #0x10000000007/00000001000000010000000B 2023-09-09T23:19:27.741863+0000 caller_uid=26, caller_gid=26{26,}) currently requesting remote authpins 2023-09-10T00:07:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:28 smithi082 ceph-mon[98694]: pgmap v1703: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 114 KiB/s wr, 16 op/s 2023-09-10T00:07:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:28 smithi082 ceph-mon[98694]: 3 slow requests, 3 included below; oldest blocked for > 2880.072602 secs 2023-09-10T00:07:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:28 smithi082 ceph-mon[98694]: slow request 2880.072601 seconds old, received at 2023-09-09T23:19:27.740042+0000: client_request(mds.4:47 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently acquired locks 2023-09-10T00:07:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:28 smithi082 ceph-mon[98694]: slow request 2880.072222 seconds old, received at 2023-09-09T23:19:27.740422+0000: client_request(mds.4:48 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently requesting remote authpins 2023-09-10T00:07:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:28 smithi082 ceph-mon[98694]: slow request 2880.070000 seconds old, received at 2023-09-09T23:19:27.742643+0000: client_request(client.24913:3869 unlink #0x10000000007/00000001000000010000000B 2023-09-09T23:19:27.741863+0000 caller_uid=26, caller_gid=26{26,}) currently requesting remote authpins 2023-09-10T00:07:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:30 smithi082 ceph-mon[98694]: 3 slow requests, 3 included below; oldest blocked for > 2880.785482 secs 2023-09-10T00:07:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:30 smithi082 ceph-mon[98694]: slow request 2880.785481 seconds old, received at 2023-09-09T23:19:27.740800+0000: peer_request:mds.4:47 currently acquired locks 2023-09-10T00:07:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:30 smithi082 ceph-mon[98694]: slow request 2880.785247 seconds old, received at 2023-09-09T23:19:27.741034+0000: peer_request(mds.4:48.0 authpin) currently dispatched 2023-09-10T00:07:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:30 smithi082 ceph-mon[98694]: slow request 2880.783211 seconds old, received at 2023-09-09T23:19:27.743070+0000: peer_request(client.24913:3869.0 authpin) currently dispatched 2023-09-10T00:07:30.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:30 smithi082 ceph-mon[98694]: pgmap v1704: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T00:07:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:30 smithi098 ceph-mon[103124]: 3 slow requests, 3 included below; oldest blocked for > 2880.785482 secs 2023-09-10T00:07:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:30 smithi098 ceph-mon[103124]: slow request 2880.785481 seconds old, received at 2023-09-09T23:19:27.740800+0000: peer_request:mds.4:47 currently acquired locks 2023-09-10T00:07:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:30 smithi098 ceph-mon[103124]: slow request 2880.785247 seconds old, received at 2023-09-09T23:19:27.741034+0000: peer_request(mds.4:48.0 authpin) currently dispatched 2023-09-10T00:07:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:30 smithi098 ceph-mon[103124]: slow request 2880.783211 seconds old, received at 2023-09-09T23:19:27.743070+0000: peer_request(client.24913:3869.0 authpin) currently dispatched 2023-09-10T00:07:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:30 smithi098 ceph-mon[103124]: pgmap v1704: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T00:07:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:30 smithi170 ceph-mon[102832]: 3 slow requests, 3 included below; oldest blocked for > 2880.785482 secs 2023-09-10T00:07:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:30 smithi170 ceph-mon[102832]: slow request 2880.785481 seconds old, received at 2023-09-09T23:19:27.740800+0000: peer_request:mds.4:47 currently acquired locks 2023-09-10T00:07:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:30 smithi170 ceph-mon[102832]: slow request 2880.785247 seconds old, received at 2023-09-09T23:19:27.741034+0000: peer_request(mds.4:48.0 authpin) currently dispatched 2023-09-10T00:07:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:30 smithi170 ceph-mon[102832]: slow request 2880.783211 seconds old, received at 2023-09-09T23:19:27.743070+0000: peer_request(client.24913:3869.0 authpin) currently dispatched 2023-09-10T00:07:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:30 smithi170 ceph-mon[102832]: pgmap v1704: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T00:07:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:32 smithi082 ceph-mon[98694]: pgmap v1705: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 158 KiB/s wr, 17 op/s 2023-09-10T00:07:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:32 smithi098 ceph-mon[103124]: pgmap v1705: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 158 KiB/s wr, 17 op/s 2023-09-10T00:07:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:32 smithi170 ceph-mon[102832]: pgmap v1705: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 158 KiB/s wr, 17 op/s 2023-09-10T00:07:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2885.072660 secs 2023-09-10T00:07:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2885.072660 secs 2023-09-10T00:07:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2885.072660 secs 2023-09-10T00:07:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:34 smithi082 ceph-mon[98694]: pgmap v1706: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:07:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:34 smithi098 ceph-mon[103124]: pgmap v1706: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:07:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:34 smithi170 ceph-mon[102832]: pgmap v1706: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:07:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2885.785549 secs 2023-09-10T00:07:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2885.785549 secs 2023-09-10T00:07:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2885.785549 secs 2023-09-10T00:07:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:36 smithi098 ceph-mon[103124]: pgmap v1707: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:07:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:36 smithi170 ceph-mon[102832]: pgmap v1707: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:07:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:36 smithi082 ceph-mon[98694]: pgmap v1707: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:07:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:38 smithi098 ceph-mon[103124]: pgmap v1708: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 150 KiB/s wr, 18 op/s 2023-09-10T00:07:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2890.072742 secs 2023-09-10T00:07:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:38 smithi170 ceph-mon[102832]: pgmap v1708: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 150 KiB/s wr, 18 op/s 2023-09-10T00:07:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2890.072742 secs 2023-09-10T00:07:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:38 smithi082 ceph-mon[98694]: pgmap v1708: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 150 KiB/s wr, 18 op/s 2023-09-10T00:07:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2890.072742 secs 2023-09-10T00:07:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2890.785614 secs 2023-09-10T00:07:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:40 smithi098 ceph-mon[103124]: pgmap v1709: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T00:07:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2890.785614 secs 2023-09-10T00:07:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:40 smithi170 ceph-mon[102832]: pgmap v1709: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T00:07:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2890.785614 secs 2023-09-10T00:07:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:40 smithi082 ceph-mon[98694]: pgmap v1709: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T00:07:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:42 smithi098 ceph-mon[103124]: pgmap v1710: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 135 KiB/s wr, 17 op/s 2023-09-10T00:07:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:42 smithi170 ceph-mon[102832]: pgmap v1710: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 135 KiB/s wr, 17 op/s 2023-09-10T00:07:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:42 smithi082 ceph-mon[98694]: pgmap v1710: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 135 KiB/s wr, 17 op/s 2023-09-10T00:07:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:43 smithi098 ceph-mon[103124]: 1 slow requests, 1 included below; oldest blocked for > 2893.461776 secs 2023-09-10T00:07:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:43 smithi098 ceph-mon[103124]: slow request 2893.461776 seconds old, received at 2023-09-09T23:19:28.303507+0000: peer_request:mds.4:47 currently dispatched 2023-09-10T00:07:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2895.072800 secs 2023-09-10T00:07:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:43 smithi170 ceph-mon[102832]: 1 slow requests, 1 included below; oldest blocked for > 2893.461776 secs 2023-09-10T00:07:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:43 smithi170 ceph-mon[102832]: slow request 2893.461776 seconds old, received at 2023-09-09T23:19:28.303507+0000: peer_request:mds.4:47 currently dispatched 2023-09-10T00:07:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2895.072800 secs 2023-09-10T00:07:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:43 smithi082 ceph-mon[98694]: 1 slow requests, 1 included below; oldest blocked for > 2893.461776 secs 2023-09-10T00:07:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:43 smithi082 ceph-mon[98694]: slow request 2893.461776 seconds old, received at 2023-09-09T23:19:28.303507+0000: peer_request:mds.4:47 currently dispatched 2023-09-10T00:07:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2895.072800 secs 2023-09-10T00:07:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:44 smithi098 ceph-mon[103124]: pgmap v1711: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T00:07:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:07:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:07:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:07:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:44 smithi170 ceph-mon[102832]: pgmap v1711: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T00:07:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:07:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:07:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:07:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:44 smithi082 ceph-mon[98694]: pgmap v1711: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T00:07:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:07:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:07:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:07:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2895.785686 secs 2023-09-10T00:07:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2895.785686 secs 2023-09-10T00:07:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2895.785686 secs 2023-09-10T00:07:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:46 smithi098 ceph-mon[103124]: pgmap v1712: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T00:07:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:46 smithi170 ceph-mon[102832]: pgmap v1712: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T00:07:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:46 smithi082 ceph-mon[98694]: pgmap v1712: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T00:07:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2898.461867 secs 2023-09-10T00:07:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:48 smithi098 ceph-mon[103124]: pgmap v1713: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 115 KiB/s wr, 16 op/s 2023-09-10T00:07:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2900.072855 secs 2023-09-10T00:07:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2898.461867 secs 2023-09-10T00:07:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:48 smithi170 ceph-mon[102832]: pgmap v1713: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 115 KiB/s wr, 16 op/s 2023-09-10T00:07:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2900.072855 secs 2023-09-10T00:07:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2898.461867 secs 2023-09-10T00:07:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:48 smithi082 ceph-mon[98694]: pgmap v1713: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 115 KiB/s wr, 16 op/s 2023-09-10T00:07:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2900.072855 secs 2023-09-10T00:07:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2900.785750 secs 2023-09-10T00:07:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:50 smithi098 ceph-mon[103124]: pgmap v1714: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-10T00:07:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2900.785750 secs 2023-09-10T00:07:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:50 smithi170 ceph-mon[102832]: pgmap v1714: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-10T00:07:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2900.785750 secs 2023-09-10T00:07:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:50 smithi082 ceph-mon[98694]: pgmap v1714: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-10T00:07:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:07:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:07:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:07:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:52 smithi170 ceph-mon[102832]: pgmap v1715: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 70 KiB/s wr, 15 op/s 2023-09-10T00:07:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:52 smithi082 ceph-mon[98694]: pgmap v1715: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 70 KiB/s wr, 15 op/s 2023-09-10T00:07:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:52 smithi098 ceph-mon[103124]: pgmap v1715: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 70 KiB/s wr, 15 op/s 2023-09-10T00:07:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2903.461927 secs 2023-09-10T00:07:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2905.072941 secs 2023-09-10T00:07:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2903.461927 secs 2023-09-10T00:07:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2905.072941 secs 2023-09-10T00:07:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2903.461927 secs 2023-09-10T00:07:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2905.072941 secs 2023-09-10T00:07:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:54 smithi170 ceph-mon[102832]: pgmap v1716: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 9 op/s 2023-09-10T00:07:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2905.785816 secs 2023-09-10T00:07:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:54 smithi082 ceph-mon[98694]: pgmap v1716: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 9 op/s 2023-09-10T00:07:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2905.785816 secs 2023-09-10T00:07:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:54 smithi098 ceph-mon[103124]: pgmap v1716: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 9 op/s 2023-09-10T00:07:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2905.785816 secs 2023-09-10T00:07:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:56 smithi170 ceph-mon[102832]: pgmap v1717: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 9 op/s 2023-09-10T00:07:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:56 smithi082 ceph-mon[98694]: pgmap v1717: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 9 op/s 2023-09-10T00:07:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:56 smithi098 ceph-mon[103124]: pgmap v1717: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 9 op/s 2023-09-10T00:07:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2908.462036 secs 2023-09-10T00:07:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:58 smithi170 ceph-mon[102832]: pgmap v1718: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 77 KiB/s wr, 15 op/s 2023-09-10T00:07:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:07:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2910.072998 secs 2023-09-10T00:07:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2908.462036 secs 2023-09-10T00:07:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:58 smithi082 ceph-mon[98694]: pgmap v1718: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 77 KiB/s wr, 15 op/s 2023-09-10T00:07:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:07:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2910.072998 secs 2023-09-10T00:07:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2908.462036 secs 2023-09-10T00:07:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:58 smithi098 ceph-mon[103124]: pgmap v1718: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 77 KiB/s wr, 15 op/s 2023-09-10T00:07:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:07:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2910.072998 secs 2023-09-10T00:08:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2910.785881 secs 2023-09-10T00:08:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:00 smithi082 ceph-mon[98694]: pgmap v1719: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T00:08:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2910.785881 secs 2023-09-10T00:08:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:00 smithi098 ceph-mon[103124]: pgmap v1719: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T00:08:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2910.785881 secs 2023-09-10T00:08:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:00 smithi170 ceph-mon[102832]: pgmap v1719: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T00:08:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:02 smithi082 ceph-mon[98694]: pgmap v1720: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 108 KiB/s wr, 29 op/s 2023-09-10T00:08:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:02 smithi098 ceph-mon[103124]: pgmap v1720: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 108 KiB/s wr, 29 op/s 2023-09-10T00:08:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:02 smithi170 ceph-mon[102832]: pgmap v1720: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 108 KiB/s wr, 29 op/s 2023-09-10T00:08:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2913.462097 secs 2023-09-10T00:08:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:03 smithi098 ceph-mon[103124]: osdmap e179: 12 total, 12 up, 12 in 2023-09-10T00:08:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2915.073058 secs 2023-09-10T00:08:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2913.462097 secs 2023-09-10T00:08:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:03 smithi170 ceph-mon[102832]: osdmap e179: 12 total, 12 up, 12 in 2023-09-10T00:08:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2915.073058 secs 2023-09-10T00:08:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2913.462097 secs 2023-09-10T00:08:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:03 smithi082 ceph-mon[98694]: osdmap e179: 12 total, 12 up, 12 in 2023-09-10T00:08:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2915.073058 secs 2023-09-10T00:08:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:04 smithi098 ceph-mon[103124]: pgmap v1722: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 109 KiB/s wr, 30 op/s 2023-09-10T00:08:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:04 smithi170 ceph-mon[102832]: pgmap v1722: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 109 KiB/s wr, 30 op/s 2023-09-10T00:08:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:04 smithi082 ceph-mon[98694]: pgmap v1722: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 109 KiB/s wr, 30 op/s 2023-09-10T00:08:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2915.785945 secs 2023-09-10T00:08:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2915.785945 secs 2023-09-10T00:08:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2915.785945 secs 2023-09-10T00:08:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:06 smithi098 ceph-mon[103124]: pgmap v1723: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 109 KiB/s wr, 30 op/s 2023-09-10T00:08:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:06 smithi170 ceph-mon[102832]: pgmap v1723: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 109 KiB/s wr, 30 op/s 2023-09-10T00:08:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:06 smithi082 ceph-mon[98694]: pgmap v1723: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 109 KiB/s wr, 30 op/s 2023-09-10T00:08:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2918.462176 secs 2023-09-10T00:08:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:08 smithi098 ceph-mon[103124]: pgmap v1724: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 75 KiB/s wr, 30 op/s 2023-09-10T00:08:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2920.073116 secs 2023-09-10T00:08:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2918.462176 secs 2023-09-10T00:08:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:08 smithi170 ceph-mon[102832]: pgmap v1724: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 75 KiB/s wr, 30 op/s 2023-09-10T00:08:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2920.073116 secs 2023-09-10T00:08:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2918.462176 secs 2023-09-10T00:08:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:08 smithi082 ceph-mon[98694]: pgmap v1724: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 75 KiB/s wr, 30 op/s 2023-09-10T00:08:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2920.073116 secs 2023-09-10T00:08:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2920.786011 secs 2023-09-10T00:08:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:10 smithi082 ceph-mon[98694]: pgmap v1725: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 75 KiB/s wr, 30 op/s 2023-09-10T00:08:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:10 smithi082 ceph-mon[98694]: osdmap e180: 12 total, 12 up, 12 in 2023-09-10T00:08:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2920.786011 secs 2023-09-10T00:08:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:10 smithi098 ceph-mon[103124]: pgmap v1725: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 75 KiB/s wr, 30 op/s 2023-09-10T00:08:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:10 smithi098 ceph-mon[103124]: osdmap e180: 12 total, 12 up, 12 in 2023-09-10T00:08:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2920.786011 secs 2023-09-10T00:08:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:10 smithi170 ceph-mon[102832]: pgmap v1725: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 75 KiB/s wr, 30 op/s 2023-09-10T00:08:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:10 smithi170 ceph-mon[102832]: osdmap e180: 12 total, 12 up, 12 in 2023-09-10T00:08:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:12 smithi082 ceph-mon[98694]: pgmap v1727: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 14 op/s 2023-09-10T00:08:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:12 smithi098 ceph-mon[103124]: pgmap v1727: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 14 op/s 2023-09-10T00:08:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:12 smithi170 ceph-mon[102832]: pgmap v1727: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 14 op/s 2023-09-10T00:08:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2923.462305 secs 2023-09-10T00:08:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2925.073204 secs 2023-09-10T00:08:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2923.462305 secs 2023-09-10T00:08:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2925.073204 secs 2023-09-10T00:08:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2923.462305 secs 2023-09-10T00:08:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2925.073204 secs 2023-09-10T00:08:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:14 smithi098 ceph-mon[103124]: pgmap v1728: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T00:08:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:14 smithi170 ceph-mon[102832]: pgmap v1728: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T00:08:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:14 smithi082 ceph-mon[98694]: pgmap v1728: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T00:08:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2925.786078 secs 2023-09-10T00:08:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2925.786078 secs 2023-09-10T00:08:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2925.786078 secs 2023-09-10T00:08:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:16 smithi098 ceph-mon[103124]: pgmap v1729: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T00:08:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:16 smithi170 ceph-mon[102832]: pgmap v1729: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T00:08:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:16 smithi082 ceph-mon[98694]: pgmap v1729: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T00:08:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:18 smithi098 ceph-mon[103124]: pgmap v1730: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-10T00:08:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2930.073267 secs 2023-09-10T00:08:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:18 smithi170 ceph-mon[102832]: pgmap v1730: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-10T00:08:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2930.073267 secs 2023-09-10T00:08:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:18 smithi082 ceph-mon[98694]: pgmap v1730: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-10T00:08:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2930.073267 secs 2023-09-10T00:08:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:08:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:08:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:08:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2930.786142 secs 2023-09-10T00:08:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:20 smithi098 ceph-mon[103124]: pgmap v1731: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-10T00:08:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:08:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:08:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:08:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2930.786142 secs 2023-09-10T00:08:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:20 smithi170 ceph-mon[102832]: pgmap v1731: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-10T00:08:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:08:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:08:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:08:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2930.786142 secs 2023-09-10T00:08:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:20 smithi082 ceph-mon[98694]: pgmap v1731: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-10T00:08:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:08:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:08:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:08:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:22 smithi098 ceph-mon[103124]: pgmap v1732: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 53 KiB/s wr, 15 op/s 2023-09-10T00:08:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:22 smithi170 ceph-mon[102832]: pgmap v1732: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 53 KiB/s wr, 15 op/s 2023-09-10T00:08:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:22 smithi082 ceph-mon[98694]: pgmap v1732: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 53 KiB/s wr, 15 op/s 2023-09-10T00:08:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2933.462480 secs 2023-09-10T00:08:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2935.073337 secs 2023-09-10T00:08:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2933.462480 secs 2023-09-10T00:08:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2935.073337 secs 2023-09-10T00:08:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2933.462480 secs 2023-09-10T00:08:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2935.073337 secs 2023-09-10T00:08:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:24 smithi098 ceph-mon[103124]: pgmap v1733: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 32 KiB/s wr, 10 op/s 2023-09-10T00:08:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:08:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:08:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:08:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:24 smithi170 ceph-mon[102832]: pgmap v1733: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 32 KiB/s wr, 10 op/s 2023-09-10T00:08:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:08:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:08:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:08:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:24 smithi082 ceph-mon[98694]: pgmap v1733: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 32 KiB/s wr, 10 op/s 2023-09-10T00:08:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:08:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:08:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:08:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2935.786208 secs 2023-09-10T00:08:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:08:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:08:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:08:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:08:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2935.786208 secs 2023-09-10T00:08:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:08:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:08:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:08:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:08:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2935.786208 secs 2023-09-10T00:08:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:08:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:08:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:08:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:08:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:26 smithi098 ceph-mon[103124]: pgmap v1734: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 32 KiB/s wr, 10 op/s 2023-09-10T00:08:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:08:26.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:08:26.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:08:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:26 smithi170 ceph-mon[102832]: pgmap v1734: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 32 KiB/s wr, 10 op/s 2023-09-10T00:08:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:08:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:08:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:08:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:26 smithi082 ceph-mon[98694]: pgmap v1734: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 32 KiB/s wr, 10 op/s 2023-09-10T00:08:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:08:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:08:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:08:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2938.462540 secs 2023-09-10T00:08:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:28 smithi098 ceph-mon[103124]: pgmap v1735: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 114 KiB/s wr, 16 op/s 2023-09-10T00:08:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2940.073394 secs 2023-09-10T00:08:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2938.462540 secs 2023-09-10T00:08:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:28 smithi170 ceph-mon[102832]: pgmap v1735: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 114 KiB/s wr, 16 op/s 2023-09-10T00:08:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2940.073394 secs 2023-09-10T00:08:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2938.462540 secs 2023-09-10T00:08:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:28 smithi082 ceph-mon[98694]: pgmap v1735: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 114 KiB/s wr, 16 op/s 2023-09-10T00:08:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2940.073394 secs 2023-09-10T00:08:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2940.786272 secs 2023-09-10T00:08:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:29 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:08:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2940.786272 secs 2023-09-10T00:08:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:29 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:08:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2940.786272 secs 2023-09-10T00:08:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:29 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:08:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:30 smithi098 ceph-mon[103124]: pgmap v1736: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T00:08:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:30 smithi170 ceph-mon[102832]: pgmap v1736: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T00:08:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:30 smithi082 ceph-mon[98694]: pgmap v1736: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T00:08:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:32 smithi170 ceph-mon[102832]: pgmap v1737: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 166 KiB/s wr, 17 op/s 2023-09-10T00:08:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:32 smithi082 ceph-mon[98694]: pgmap v1737: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 166 KiB/s wr, 17 op/s 2023-09-10T00:08:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:32 smithi098 ceph-mon[103124]: pgmap v1737: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 166 KiB/s wr, 17 op/s 2023-09-10T00:08:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2945.073454 secs 2023-09-10T00:08:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2945.073454 secs 2023-09-10T00:08:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2945.073454 secs 2023-09-10T00:08:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:34 smithi170 ceph-mon[102832]: pgmap v1738: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 135 KiB/s wr, 12 op/s 2023-09-10T00:08:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2945.786339 secs 2023-09-10T00:08:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:34 smithi082 ceph-mon[98694]: pgmap v1738: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 135 KiB/s wr, 12 op/s 2023-09-10T00:08:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2945.786339 secs 2023-09-10T00:08:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:34 smithi098 ceph-mon[103124]: pgmap v1738: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 135 KiB/s wr, 12 op/s 2023-09-10T00:08:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2945.786339 secs 2023-09-10T00:08:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:08:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:08:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:08:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:36 smithi170 ceph-mon[102832]: pgmap v1739: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 135 KiB/s wr, 12 op/s 2023-09-10T00:08:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:08:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:08:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:08:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:36 smithi082 ceph-mon[98694]: pgmap v1739: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 135 KiB/s wr, 12 op/s 2023-09-10T00:08:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:08:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:08:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:08:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:36 smithi098 ceph-mon[103124]: pgmap v1739: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 135 KiB/s wr, 12 op/s 2023-09-10T00:08:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:08:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:08:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:08:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:38 smithi170 ceph-mon[102832]: pgmap v1740: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 155 KiB/s wr, 18 op/s 2023-09-10T00:08:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2950.073492 secs 2023-09-10T00:08:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:38 smithi082 ceph-mon[98694]: pgmap v1740: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 155 KiB/s wr, 18 op/s 2023-09-10T00:08:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2950.073492 secs 2023-09-10T00:08:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:38 smithi098 ceph-mon[103124]: pgmap v1740: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 155 KiB/s wr, 18 op/s 2023-09-10T00:08:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2950.073492 secs 2023-09-10T00:08:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2950.786402 secs 2023-09-10T00:08:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:40 smithi082 ceph-mon[98694]: pgmap v1741: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T00:08:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2950.786402 secs 2023-09-10T00:08:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:40 smithi098 ceph-mon[103124]: pgmap v1741: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T00:08:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2950.786402 secs 2023-09-10T00:08:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:40 smithi170 ceph-mon[102832]: pgmap v1741: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T00:08:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:42 smithi098 ceph-mon[103124]: pgmap v1742: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 140 KiB/s wr, 17 op/s 2023-09-10T00:08:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:42 smithi170 ceph-mon[102832]: pgmap v1742: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 140 KiB/s wr, 17 op/s 2023-09-10T00:08:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:42 smithi082 ceph-mon[98694]: pgmap v1742: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 140 KiB/s wr, 17 op/s 2023-09-10T00:08:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2953.462771 secs 2023-09-10T00:08:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2955.073576 secs 2023-09-10T00:08:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2953.462771 secs 2023-09-10T00:08:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2955.073576 secs 2023-09-10T00:08:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2953.462771 secs 2023-09-10T00:08:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2955.073576 secs 2023-09-10T00:08:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:44 smithi098 ceph-mon[103124]: pgmap v1743: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:08:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:08:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:08:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:08:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:44 smithi170 ceph-mon[102832]: pgmap v1743: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:08:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:08:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:08:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:08:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:44 smithi082 ceph-mon[98694]: pgmap v1743: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:08:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:08:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:08:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:08:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2955.786467 secs 2023-09-10T00:08:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2955.786467 secs 2023-09-10T00:08:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2955.786467 secs 2023-09-10T00:08:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:46 smithi098 ceph-mon[103124]: pgmap v1744: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:08:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:46 smithi170 ceph-mon[102832]: pgmap v1744: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:08:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:46 smithi082 ceph-mon[98694]: pgmap v1744: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:08:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2958.462830 secs 2023-09-10T00:08:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:48 smithi098 ceph-mon[103124]: pgmap v1745: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 119 KiB/s wr, 16 op/s 2023-09-10T00:08:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2960.073657 secs 2023-09-10T00:08:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2958.462830 secs 2023-09-10T00:08:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:48 smithi170 ceph-mon[102832]: pgmap v1745: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 119 KiB/s wr, 16 op/s 2023-09-10T00:08:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2960.073657 secs 2023-09-10T00:08:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2958.462830 secs 2023-09-10T00:08:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:48 smithi082 ceph-mon[98694]: pgmap v1745: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 119 KiB/s wr, 16 op/s 2023-09-10T00:08:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2960.073657 secs 2023-09-10T00:08:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2960.786489 secs 2023-09-10T00:08:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:50 smithi098 ceph-mon[103124]: pgmap v1746: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T00:08:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2960.786489 secs 2023-09-10T00:08:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:50 smithi170 ceph-mon[102832]: pgmap v1746: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T00:08:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2960.786489 secs 2023-09-10T00:08:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:50 smithi082 ceph-mon[98694]: pgmap v1746: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T00:08:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:52 smithi098 ceph-mon[103124]: pgmap v1747: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 118 KiB/s wr, 15 op/s 2023-09-10T00:08:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:52 smithi170 ceph-mon[102832]: pgmap v1747: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 118 KiB/s wr, 15 op/s 2023-09-10T00:08:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:52 smithi082 ceph-mon[98694]: pgmap v1747: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 118 KiB/s wr, 15 op/s 2023-09-10T00:08:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2963.462890 secs 2023-09-10T00:08:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2965.073718 secs 2023-09-10T00:08:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2963.462890 secs 2023-09-10T00:08:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2965.073718 secs 2023-09-10T00:08:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2963.462890 secs 2023-09-10T00:08:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2965.073718 secs 2023-09-10T00:08:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:54 smithi098 ceph-mon[103124]: pgmap v1748: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:08:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:54 smithi170 ceph-mon[102832]: pgmap v1748: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:08:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:54 smithi082 ceph-mon[98694]: pgmap v1748: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:08:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2965.786555 secs 2023-09-10T00:08:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2965.786555 secs 2023-09-10T00:08:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2965.786555 secs 2023-09-10T00:08:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:56 smithi170 ceph-mon[102832]: pgmap v1749: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:08:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:56 smithi082 ceph-mon[98694]: pgmap v1749: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:08:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:56 smithi098 ceph-mon[103124]: pgmap v1749: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:08:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2968.462950 secs 2023-09-10T00:08:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:58 smithi170 ceph-mon[102832]: pgmap v1750: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 63 KiB/s wr, 15 op/s 2023-09-10T00:08:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:08:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2970.073778 secs 2023-09-10T00:08:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2968.462950 secs 2023-09-10T00:08:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:58 smithi082 ceph-mon[98694]: pgmap v1750: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 63 KiB/s wr, 15 op/s 2023-09-10T00:08:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:08:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2970.073778 secs 2023-09-10T00:08:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2968.462950 secs 2023-09-10T00:08:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:58 smithi098 ceph-mon[103124]: pgmap v1750: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 63 KiB/s wr, 15 op/s 2023-09-10T00:08:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:08:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2970.073778 secs 2023-09-10T00:09:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2970.786629 secs 2023-09-10T00:09:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:00 smithi098 ceph-mon[103124]: pgmap v1751: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 32 KiB/s wr, 9 op/s 2023-09-10T00:09:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2970.786629 secs 2023-09-10T00:09:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:00 smithi170 ceph-mon[102832]: pgmap v1751: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 32 KiB/s wr, 9 op/s 2023-09-10T00:09:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2970.786629 secs 2023-09-10T00:09:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:00 smithi082 ceph-mon[98694]: pgmap v1751: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 32 KiB/s wr, 9 op/s 2023-09-10T00:09:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:02 smithi098 ceph-mon[103124]: pgmap v1752: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 80 KiB/s wr, 28 op/s 2023-09-10T00:09:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:02 smithi170 ceph-mon[102832]: pgmap v1752: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 80 KiB/s wr, 28 op/s 2023-09-10T00:09:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:02 smithi082 ceph-mon[98694]: pgmap v1752: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 80 KiB/s wr, 28 op/s 2023-09-10T00:09:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2973.463057 secs 2023-09-10T00:09:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2975.073849 secs 2023-09-10T00:09:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2973.463057 secs 2023-09-10T00:09:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2975.073849 secs 2023-09-10T00:09:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2973.463057 secs 2023-09-10T00:09:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2975.073849 secs 2023-09-10T00:09:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:04 smithi098 ceph-mon[103124]: pgmap v1753: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 60 KiB/s wr, 23 op/s 2023-09-10T00:09:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:04 smithi170 ceph-mon[102832]: pgmap v1753: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 60 KiB/s wr, 23 op/s 2023-09-10T00:09:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:04 smithi082 ceph-mon[98694]: pgmap v1753: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 60 KiB/s wr, 23 op/s 2023-09-10T00:09:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2975.786691 secs 2023-09-10T00:09:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2975.786691 secs 2023-09-10T00:09:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2975.786691 secs 2023-09-10T00:09:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:06 smithi098 ceph-mon[103124]: pgmap v1754: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 60 KiB/s wr, 23 op/s 2023-09-10T00:09:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:06 smithi170 ceph-mon[102832]: pgmap v1754: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 60 KiB/s wr, 23 op/s 2023-09-10T00:09:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:06 smithi082 ceph-mon[98694]: pgmap v1754: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 60 KiB/s wr, 23 op/s 2023-09-10T00:09:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2978.463117 secs 2023-09-10T00:09:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:08 smithi098 ceph-mon[103124]: pgmap v1755: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T00:09:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:08 smithi098 ceph-mon[103124]: osdmap e181: 12 total, 12 up, 12 in 2023-09-10T00:09:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2980.073908 secs 2023-09-10T00:09:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2978.463117 secs 2023-09-10T00:09:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:08 smithi170 ceph-mon[102832]: pgmap v1755: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T00:09:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:08 smithi170 ceph-mon[102832]: osdmap e181: 12 total, 12 up, 12 in 2023-09-10T00:09:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2980.073908 secs 2023-09-10T00:09:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2978.463117 secs 2023-09-10T00:09:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:08 smithi082 ceph-mon[98694]: pgmap v1755: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T00:09:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:08 smithi082 ceph-mon[98694]: osdmap e181: 12 total, 12 up, 12 in 2023-09-10T00:09:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2980.073908 secs 2023-09-10T00:09:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2980.786756 secs 2023-09-10T00:09:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:10 smithi098 ceph-mon[103124]: pgmap v1757: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-10T00:09:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2980.786756 secs 2023-09-10T00:09:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:10 smithi170 ceph-mon[102832]: pgmap v1757: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-10T00:09:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2980.786756 secs 2023-09-10T00:09:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:10 smithi082 ceph-mon[98694]: pgmap v1757: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-10T00:09:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:12 smithi098 ceph-mon[103124]: pgmap v1758: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-09-10T00:09:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:12 smithi170 ceph-mon[102832]: pgmap v1758: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-09-10T00:09:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:12 smithi082 ceph-mon[98694]: pgmap v1758: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-09-10T00:09:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 2983.463205 secs 2023-09-10T00:09:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2985.073964 secs 2023-09-10T00:09:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 2983.463205 secs 2023-09-10T00:09:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2985.073964 secs 2023-09-10T00:09:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 2983.463205 secs 2023-09-10T00:09:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2985.073964 secs 2023-09-10T00:09:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:14 smithi098 ceph-mon[103124]: pgmap v1759: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-09-10T00:09:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:14 smithi098 ceph-mon[103124]: osdmap e182: 12 total, 12 up, 12 in 2023-09-10T00:09:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:14 smithi170 ceph-mon[102832]: pgmap v1759: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-09-10T00:09:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:14 smithi170 ceph-mon[102832]: osdmap e182: 12 total, 12 up, 12 in 2023-09-10T00:09:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:14 smithi082 ceph-mon[98694]: pgmap v1759: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-09-10T00:09:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:14 smithi082 ceph-mon[98694]: osdmap e182: 12 total, 12 up, 12 in 2023-09-10T00:09:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2985.786818 secs 2023-09-10T00:09:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2985.786818 secs 2023-09-10T00:09:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2985.786818 secs 2023-09-10T00:09:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:16 smithi170 ceph-mon[102832]: pgmap v1761: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1023 B/s wr, 7 op/s 2023-09-10T00:09:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:16 smithi082 ceph-mon[98694]: pgmap v1761: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1023 B/s wr, 7 op/s 2023-09-10T00:09:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:16 smithi098 ceph-mon[103124]: pgmap v1761: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1023 B/s wr, 7 op/s 2023-09-10T00:09:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:18 smithi170 ceph-mon[102832]: pgmap v1762: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-09-10T00:09:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2990.074049 secs 2023-09-10T00:09:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:18 smithi082 ceph-mon[98694]: pgmap v1762: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-09-10T00:09:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2990.074049 secs 2023-09-10T00:09:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:18 smithi098 ceph-mon[103124]: pgmap v1762: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-09-10T00:09:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2990.074049 secs 2023-09-10T00:09:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2990.786881 secs 2023-09-10T00:09:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:09:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2990.786881 secs 2023-09-10T00:09:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:09:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2990.786881 secs 2023-09-10T00:09:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:09:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:20 smithi170 ceph-mon[102832]: pgmap v1763: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-09-10T00:09:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:09:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:09:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:09:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:20 smithi082 ceph-mon[98694]: pgmap v1763: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-09-10T00:09:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:09:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:09:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:09:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:20 smithi098 ceph-mon[103124]: pgmap v1763: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-09-10T00:09:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:09:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:09:20.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:09:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:22 smithi170 ceph-mon[102832]: pgmap v1764: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 77 KiB/s wr, 13 op/s 2023-09-10T00:09:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:22 smithi082 ceph-mon[98694]: pgmap v1764: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 77 KiB/s wr, 13 op/s 2023-09-10T00:09:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:22 smithi098 ceph-mon[103124]: pgmap v1764: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 77 KiB/s wr, 13 op/s 2023-09-10T00:09:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2995.074106 secs 2023-09-10T00:09:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2995.074106 secs 2023-09-10T00:09:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2995.074106 secs 2023-09-10T00:09:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:24 smithi170 ceph-mon[102832]: pgmap v1765: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 77 KiB/s wr, 13 op/s 2023-09-10T00:09:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 2995.786946 secs 2023-09-10T00:09:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:24 smithi082 ceph-mon[98694]: pgmap v1765: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 77 KiB/s wr, 13 op/s 2023-09-10T00:09:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 2995.786946 secs 2023-09-10T00:09:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:24 smithi098 ceph-mon[103124]: pgmap v1765: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 77 KiB/s wr, 13 op/s 2023-09-10T00:09:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 2995.786946 secs 2023-09-10T00:09:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:26 smithi170 ceph-mon[102832]: pgmap v1766: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 70 KiB/s wr, 12 op/s 2023-09-10T00:09:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:09:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:26 smithi082 ceph-mon[98694]: pgmap v1766: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 70 KiB/s wr, 12 op/s 2023-09-10T00:09:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:09:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:26 smithi098 ceph-mon[103124]: pgmap v1766: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 70 KiB/s wr, 12 op/s 2023-09-10T00:09:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:09:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:09:27.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:09:27.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:09:27.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:09:27.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:09:27.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:09:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:09:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:09:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:09:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:28 smithi170 ceph-mon[102832]: pgmap v1767: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 88 KiB/s wr, 17 op/s 2023-09-10T00:09:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3000.074164 secs 2023-09-10T00:09:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:28 smithi082 ceph-mon[98694]: pgmap v1767: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 88 KiB/s wr, 17 op/s 2023-09-10T00:09:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3000.074164 secs 2023-09-10T00:09:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:28 smithi098 ceph-mon[103124]: pgmap v1767: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 88 KiB/s wr, 17 op/s 2023-09-10T00:09:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3000.074164 secs 2023-09-10T00:09:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3000.787012 secs 2023-09-10T00:09:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:30 smithi098 ceph-mon[103124]: pgmap v1768: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-10T00:09:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3000.787012 secs 2023-09-10T00:09:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:30 smithi170 ceph-mon[102832]: pgmap v1768: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-10T00:09:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3000.787012 secs 2023-09-10T00:09:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:30 smithi082 ceph-mon[98694]: pgmap v1768: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-10T00:09:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:32 smithi098 ceph-mon[103124]: pgmap v1769: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 155 KiB/s wr, 17 op/s 2023-09-10T00:09:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:32 smithi170 ceph-mon[102832]: pgmap v1769: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 155 KiB/s wr, 17 op/s 2023-09-10T00:09:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:32 smithi082 ceph-mon[98694]: pgmap v1769: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 155 KiB/s wr, 17 op/s 2023-09-10T00:09:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3003.463518 secs 2023-09-10T00:09:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3005.074221 secs 2023-09-10T00:09:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3003.463518 secs 2023-09-10T00:09:33.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3005.074221 secs 2023-09-10T00:09:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3003.463518 secs 2023-09-10T00:09:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3005.074221 secs 2023-09-10T00:09:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:34 smithi098 ceph-mon[103124]: pgmap v1770: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T00:09:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:34 smithi170 ceph-mon[102832]: pgmap v1770: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T00:09:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:34 smithi082 ceph-mon[98694]: pgmap v1770: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T00:09:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3005.787078 secs 2023-09-10T00:09:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3005.787078 secs 2023-09-10T00:09:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3005.787078 secs 2023-09-10T00:09:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:36 smithi098 ceph-mon[103124]: pgmap v1771: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T00:09:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:36 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:09:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:09:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:09:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:36 smithi170 ceph-mon[102832]: pgmap v1771: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T00:09:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:36 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:09:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:09:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:09:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:36 smithi082 ceph-mon[98694]: pgmap v1771: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T00:09:36.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:36 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:09:36.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:09:36.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:09:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3008.463609 secs 2023-09-10T00:09:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:38 smithi098 ceph-mon[103124]: pgmap v1772: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 127 KiB/s wr, 17 op/s 2023-09-10T00:09:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3010.074279 secs 2023-09-10T00:09:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3008.463609 secs 2023-09-10T00:09:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:38 smithi170 ceph-mon[102832]: pgmap v1772: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 127 KiB/s wr, 17 op/s 2023-09-10T00:09:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3010.074279 secs 2023-09-10T00:09:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3008.463609 secs 2023-09-10T00:09:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:38 smithi082 ceph-mon[98694]: pgmap v1772: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 127 KiB/s wr, 17 op/s 2023-09-10T00:09:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3010.074279 secs 2023-09-10T00:09:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3010.787143 secs 2023-09-10T00:09:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:40 smithi098 ceph-mon[103124]: pgmap v1773: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T00:09:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3010.787143 secs 2023-09-10T00:09:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:40 smithi170 ceph-mon[102832]: pgmap v1773: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T00:09:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3010.787143 secs 2023-09-10T00:09:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:40 smithi082 ceph-mon[98694]: pgmap v1773: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T00:09:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:42 smithi098 ceph-mon[103124]: pgmap v1774: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 161 KiB/s wr, 16 op/s 2023-09-10T00:09:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:42 smithi170 ceph-mon[102832]: pgmap v1774: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 161 KiB/s wr, 16 op/s 2023-09-10T00:09:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:42 smithi082 ceph-mon[98694]: pgmap v1774: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 161 KiB/s wr, 16 op/s 2023-09-10T00:09:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3013.463667 secs 2023-09-10T00:09:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3015.074333 secs 2023-09-10T00:09:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3013.463667 secs 2023-09-10T00:09:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3015.074333 secs 2023-09-10T00:09:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3013.463667 secs 2023-09-10T00:09:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3015.074333 secs 2023-09-10T00:09:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:44 smithi098 ceph-mon[103124]: pgmap v1775: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T00:09:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:44 smithi170 ceph-mon[102832]: pgmap v1775: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T00:09:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:44 smithi082 ceph-mon[98694]: pgmap v1775: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T00:09:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3015.787207 secs 2023-09-10T00:09:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3015.787207 secs 2023-09-10T00:09:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3015.787207 secs 2023-09-10T00:09:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:46 smithi170 ceph-mon[102832]: pgmap v1776: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T00:09:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:46 smithi082 ceph-mon[98694]: pgmap v1776: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T00:09:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:46 smithi098 ceph-mon[103124]: pgmap v1776: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T00:09:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3018.463767 secs 2023-09-10T00:09:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:48 smithi170 ceph-mon[102832]: pgmap v1777: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 118 KiB/s wr, 16 op/s 2023-09-10T00:09:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3020.074414 secs 2023-09-10T00:09:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3018.463767 secs 2023-09-10T00:09:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:48 smithi082 ceph-mon[98694]: pgmap v1777: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 118 KiB/s wr, 16 op/s 2023-09-10T00:09:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3020.074414 secs 2023-09-10T00:09:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3018.463767 secs 2023-09-10T00:09:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:48 smithi098 ceph-mon[103124]: pgmap v1777: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 118 KiB/s wr, 16 op/s 2023-09-10T00:09:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3020.074414 secs 2023-09-10T00:09:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3020.787276 secs 2023-09-10T00:09:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:50 smithi098 ceph-mon[103124]: pgmap v1778: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 97 KiB/s wr, 10 op/s 2023-09-10T00:09:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3020.787276 secs 2023-09-10T00:09:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:50 smithi170 ceph-mon[102832]: pgmap v1778: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 97 KiB/s wr, 10 op/s 2023-09-10T00:09:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3020.787276 secs 2023-09-10T00:09:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:50 smithi082 ceph-mon[98694]: pgmap v1778: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 97 KiB/s wr, 10 op/s 2023-09-10T00:09:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:52 smithi098 ceph-mon[103124]: pgmap v1779: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 115 KiB/s wr, 16 op/s 2023-09-10T00:09:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:52 smithi170 ceph-mon[102832]: pgmap v1779: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 115 KiB/s wr, 16 op/s 2023-09-10T00:09:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:52 smithi082 ceph-mon[98694]: pgmap v1779: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 115 KiB/s wr, 16 op/s 2023-09-10T00:09:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3023.463828 secs 2023-09-10T00:09:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3025.074490 secs 2023-09-10T00:09:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3023.463828 secs 2023-09-10T00:09:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3025.074490 secs 2023-09-10T00:09:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3023.463828 secs 2023-09-10T00:09:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3025.074490 secs 2023-09-10T00:09:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:54 smithi098 ceph-mon[103124]: pgmap v1780: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 10 op/s 2023-09-10T00:09:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:54 smithi170 ceph-mon[102832]: pgmap v1780: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 10 op/s 2023-09-10T00:09:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:54 smithi082 ceph-mon[98694]: pgmap v1780: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 10 op/s 2023-09-10T00:09:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3025.787340 secs 2023-09-10T00:09:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3025.787340 secs 2023-09-10T00:09:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3025.787340 secs 2023-09-10T00:09:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:56 smithi098 ceph-mon[103124]: pgmap v1781: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 10 op/s 2023-09-10T00:09:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:56 smithi170 ceph-mon[102832]: pgmap v1781: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 10 op/s 2023-09-10T00:09:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:56 smithi082 ceph-mon[98694]: pgmap v1781: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 10 op/s 2023-09-10T00:09:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3028.463888 secs 2023-09-10T00:09:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:58 smithi098 ceph-mon[103124]: pgmap v1782: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 73 KiB/s wr, 15 op/s 2023-09-10T00:09:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:09:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3030.074538 secs 2023-09-10T00:09:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3028.463888 secs 2023-09-10T00:09:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:58 smithi170 ceph-mon[102832]: pgmap v1782: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 73 KiB/s wr, 15 op/s 2023-09-10T00:09:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:09:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3030.074538 secs 2023-09-10T00:09:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3028.463888 secs 2023-09-10T00:09:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:58 smithi082 ceph-mon[98694]: pgmap v1782: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 73 KiB/s wr, 15 op/s 2023-09-10T00:09:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:09:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3030.074538 secs 2023-09-10T00:10:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3030.787407 secs 2023-09-10T00:10:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:00 smithi098 ceph-mon[103124]: pgmap v1783: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-09-10T00:10:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T00:10:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3030.787407 secs 2023-09-10T00:10:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:00 smithi170 ceph-mon[102832]: pgmap v1783: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-09-10T00:10:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T00:10:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3030.787407 secs 2023-09-10T00:10:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:00 smithi082 ceph-mon[98694]: pgmap v1783: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-09-10T00:10:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T00:10:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:02 smithi170 ceph-mon[102832]: pgmap v1784: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T00:10:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:02 smithi082 ceph-mon[98694]: pgmap v1784: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T00:10:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:02 smithi098 ceph-mon[103124]: pgmap v1784: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T00:10:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3033.463949 secs 2023-09-10T00:10:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3035.074598 secs 2023-09-10T00:10:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3033.463949 secs 2023-09-10T00:10:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3035.074598 secs 2023-09-10T00:10:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3033.463949 secs 2023-09-10T00:10:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3035.074598 secs 2023-09-10T00:10:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:04 smithi170 ceph-mon[102832]: pgmap v1785: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 60 KiB/s wr, 24 op/s 2023-09-10T00:10:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3035.787479 secs 2023-09-10T00:10:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:04 smithi082 ceph-mon[98694]: pgmap v1785: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 60 KiB/s wr, 24 op/s 2023-09-10T00:10:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3035.787479 secs 2023-09-10T00:10:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:04 smithi098 ceph-mon[103124]: pgmap v1785: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 60 KiB/s wr, 24 op/s 2023-09-10T00:10:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3035.787479 secs 2023-09-10T00:10:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:06 smithi170 ceph-mon[102832]: pgmap v1786: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 60 KiB/s wr, 24 op/s 2023-09-10T00:10:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:06 smithi082 ceph-mon[98694]: pgmap v1786: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 60 KiB/s wr, 24 op/s 2023-09-10T00:10:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:06 smithi098 ceph-mon[103124]: pgmap v1786: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 60 KiB/s wr, 24 op/s 2023-09-10T00:10:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3038.464010 secs 2023-09-10T00:10:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:08 smithi170 ceph-mon[102832]: pgmap v1787: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-10T00:10:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:08 smithi170 ceph-mon[102832]: osdmap e183: 12 total, 12 up, 12 in 2023-09-10T00:10:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3040.074657 secs 2023-09-10T00:10:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3038.464010 secs 2023-09-10T00:10:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:08 smithi082 ceph-mon[98694]: pgmap v1787: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-10T00:10:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:08 smithi082 ceph-mon[98694]: osdmap e183: 12 total, 12 up, 12 in 2023-09-10T00:10:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3040.074657 secs 2023-09-10T00:10:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3038.464010 secs 2023-09-10T00:10:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:08 smithi098 ceph-mon[103124]: pgmap v1787: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-10T00:10:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:08 smithi098 ceph-mon[103124]: osdmap e183: 12 total, 12 up, 12 in 2023-09-10T00:10:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3040.074657 secs 2023-09-10T00:10:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3040.787542 secs 2023-09-10T00:10:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:10 smithi098 ceph-mon[103124]: pgmap v1789: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T00:10:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3040.787542 secs 2023-09-10T00:10:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:10 smithi170 ceph-mon[102832]: pgmap v1789: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T00:10:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3040.787542 secs 2023-09-10T00:10:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:10 smithi082 ceph-mon[98694]: pgmap v1789: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T00:10:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:12 smithi098 ceph-mon[103124]: pgmap v1790: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T00:10:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:12 smithi170 ceph-mon[102832]: pgmap v1790: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T00:10:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:12 smithi082 ceph-mon[98694]: pgmap v1790: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T00:10:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3043.464070 secs 2023-09-10T00:10:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3045.074719 secs 2023-09-10T00:10:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3043.464070 secs 2023-09-10T00:10:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3045.074719 secs 2023-09-10T00:10:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3043.464070 secs 2023-09-10T00:10:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3045.074719 secs 2023-09-10T00:10:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:14 smithi098 ceph-mon[103124]: pgmap v1791: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T00:10:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:14 smithi098 ceph-mon[103124]: osdmap e184: 12 total, 12 up, 12 in 2023-09-10T00:10:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:14 smithi170 ceph-mon[102832]: pgmap v1791: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T00:10:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:14 smithi170 ceph-mon[102832]: osdmap e184: 12 total, 12 up, 12 in 2023-09-10T00:10:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:14 smithi082 ceph-mon[98694]: pgmap v1791: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T00:10:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:14 smithi082 ceph-mon[98694]: osdmap e184: 12 total, 12 up, 12 in 2023-09-10T00:10:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3045.787606 secs 2023-09-10T00:10:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3045.787606 secs 2023-09-10T00:10:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3045.787606 secs 2023-09-10T00:10:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:16 smithi098 ceph-mon[103124]: pgmap v1793: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1023 B/s wr, 7 op/s 2023-09-10T00:10:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:16 smithi170 ceph-mon[102832]: pgmap v1793: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1023 B/s wr, 7 op/s 2023-09-10T00:10:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:16 smithi082 ceph-mon[98694]: pgmap v1793: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1023 B/s wr, 7 op/s 2023-09-10T00:10:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:18 smithi098 ceph-mon[103124]: pgmap v1794: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-10T00:10:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3050.074776 secs 2023-09-10T00:10:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:18 smithi170 ceph-mon[102832]: pgmap v1794: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-10T00:10:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3050.074776 secs 2023-09-10T00:10:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:18 smithi082 ceph-mon[98694]: pgmap v1794: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-10T00:10:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3050.074776 secs 2023-09-10T00:10:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:10:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:10:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:10:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3050.787670 secs 2023-09-10T00:10:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:20 smithi170 ceph-mon[102832]: pgmap v1795: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 20 KiB/s wr, 11 op/s 2023-09-10T00:10:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:10:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:10:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:10:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3050.787670 secs 2023-09-10T00:10:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:20 smithi082 ceph-mon[98694]: pgmap v1795: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 20 KiB/s wr, 11 op/s 2023-09-10T00:10:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:10:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:10:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:10:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3050.787670 secs 2023-09-10T00:10:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:20 smithi098 ceph-mon[103124]: pgmap v1795: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 20 KiB/s wr, 11 op/s 2023-09-10T00:10:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:10:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:10:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:10:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:22 smithi170 ceph-mon[102832]: pgmap v1796: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 75 KiB/s wr, 13 op/s 2023-09-10T00:10:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:22 smithi082 ceph-mon[98694]: pgmap v1796: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 75 KiB/s wr, 13 op/s 2023-09-10T00:10:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:22 smithi098 ceph-mon[103124]: pgmap v1796: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 75 KiB/s wr, 13 op/s 2023-09-10T00:10:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3055.074833 secs 2023-09-10T00:10:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3055.074833 secs 2023-09-10T00:10:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3055.074833 secs 2023-09-10T00:10:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:24 smithi170 ceph-mon[102832]: pgmap v1797: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 75 KiB/s wr, 13 op/s 2023-09-10T00:10:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3055.787735 secs 2023-09-10T00:10:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:24 smithi082 ceph-mon[98694]: pgmap v1797: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 75 KiB/s wr, 13 op/s 2023-09-10T00:10:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3055.787735 secs 2023-09-10T00:10:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:24 smithi098 ceph-mon[103124]: pgmap v1797: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 75 KiB/s wr, 13 op/s 2023-09-10T00:10:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3055.787735 secs 2023-09-10T00:10:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:26 smithi170 ceph-mon[102832]: pgmap v1798: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T00:10:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:26 smithi082 ceph-mon[98694]: pgmap v1798: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T00:10:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:26 smithi098 ceph-mon[103124]: pgmap v1798: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T00:10:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:10:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:10:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:10:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:28 smithi170 ceph-mon[102832]: pgmap v1799: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 87 KiB/s wr, 17 op/s 2023-09-10T00:10:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:10:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:10:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:10:28.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3060.074911 secs 2023-09-10T00:10:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:28 smithi082 ceph-mon[98694]: pgmap v1799: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 87 KiB/s wr, 17 op/s 2023-09-10T00:10:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:10:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:10:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:10:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3060.074911 secs 2023-09-10T00:10:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:28 smithi098 ceph-mon[103124]: pgmap v1799: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 87 KiB/s wr, 17 op/s 2023-09-10T00:10:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:10:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:10:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:10:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3060.074911 secs 2023-09-10T00:10:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3060.787801 secs 2023-09-10T00:10:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:30 smithi098 ceph-mon[103124]: pgmap v1800: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T00:10:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3060.787801 secs 2023-09-10T00:10:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:30 smithi170 ceph-mon[102832]: pgmap v1800: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T00:10:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3060.787801 secs 2023-09-10T00:10:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:30 smithi082 ceph-mon[98694]: pgmap v1800: 97 pgs: 97 active+clean; 8.8 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T00:10:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:32 smithi098 ceph-mon[103124]: pgmap v1801: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 155 KiB/s wr, 18 op/s 2023-09-10T00:10:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:32 smithi170 ceph-mon[102832]: pgmap v1801: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 155 KiB/s wr, 18 op/s 2023-09-10T00:10:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:32 smithi082 ceph-mon[98694]: pgmap v1801: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 155 KiB/s wr, 18 op/s 2023-09-10T00:10:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3065.074993 secs 2023-09-10T00:10:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3065.074993 secs 2023-09-10T00:10:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3065.074993 secs 2023-09-10T00:10:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:34 smithi098 ceph-mon[103124]: pgmap v1802: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T00:10:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:34 smithi170 ceph-mon[102832]: pgmap v1802: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T00:10:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:34 smithi082 ceph-mon[98694]: pgmap v1802: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T00:10:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3065.787869 secs 2023-09-10T00:10:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3065.787869 secs 2023-09-10T00:10:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3065.787869 secs 2023-09-10T00:10:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:36 smithi098 ceph-mon[103124]: pgmap v1803: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T00:10:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:36 smithi170 ceph-mon[102832]: pgmap v1803: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T00:10:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:36 smithi082 ceph-mon[98694]: pgmap v1803: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T00:10:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:38 smithi098 ceph-mon[103124]: pgmap v1804: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 129 KiB/s wr, 17 op/s 2023-09-10T00:10:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3070.075078 secs 2023-09-10T00:10:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:38 smithi170 ceph-mon[102832]: pgmap v1804: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 129 KiB/s wr, 17 op/s 2023-09-10T00:10:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3070.075078 secs 2023-09-10T00:10:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:38 smithi082 ceph-mon[98694]: pgmap v1804: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 129 KiB/s wr, 17 op/s 2023-09-10T00:10:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3070.075078 secs 2023-09-10T00:10:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3070.787936 secs 2023-09-10T00:10:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:40 smithi170 ceph-mon[102832]: pgmap v1805: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T00:10:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3070.787936 secs 2023-09-10T00:10:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:40 smithi082 ceph-mon[98694]: pgmap v1805: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T00:10:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3070.787936 secs 2023-09-10T00:10:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:40 smithi098 ceph-mon[103124]: pgmap v1805: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T00:10:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:42 smithi170 ceph-mon[102832]: pgmap v1806: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 16 op/s 2023-09-10T00:10:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:42 smithi082 ceph-mon[98694]: pgmap v1806: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 16 op/s 2023-09-10T00:10:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:42 smithi098 ceph-mon[103124]: pgmap v1806: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 16 op/s 2023-09-10T00:10:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3075.075167 secs 2023-09-10T00:10:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3075.075167 secs 2023-09-10T00:10:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3075.075167 secs 2023-09-10T00:10:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:44 smithi170 ceph-mon[102832]: pgmap v1807: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T00:10:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3075.787999 secs 2023-09-10T00:10:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:44 smithi082 ceph-mon[98694]: pgmap v1807: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T00:10:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3075.787999 secs 2023-09-10T00:10:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:44 smithi098 ceph-mon[103124]: pgmap v1807: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T00:10:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3075.787999 secs 2023-09-10T00:10:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:46 smithi170 ceph-mon[102832]: pgmap v1808: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T00:10:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:46 smithi082 ceph-mon[98694]: pgmap v1808: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T00:10:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:46 smithi098 ceph-mon[103124]: pgmap v1808: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T00:10:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3078.464604 secs 2023-09-10T00:10:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:48 smithi170 ceph-mon[102832]: pgmap v1809: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 90 KiB/s wr, 16 op/s 2023-09-10T00:10:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3080.075234 secs 2023-09-10T00:10:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3078.464604 secs 2023-09-10T00:10:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:48 smithi082 ceph-mon[98694]: pgmap v1809: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 90 KiB/s wr, 16 op/s 2023-09-10T00:10:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3080.075234 secs 2023-09-10T00:10:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3078.464604 secs 2023-09-10T00:10:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:48 smithi098 ceph-mon[103124]: pgmap v1809: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 90 KiB/s wr, 16 op/s 2023-09-10T00:10:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3080.075234 secs 2023-09-10T00:10:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3080.788063 secs 2023-09-10T00:10:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:50 smithi098 ceph-mon[103124]: pgmap v1810: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T00:10:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3080.788063 secs 2023-09-10T00:10:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:50 smithi170 ceph-mon[102832]: pgmap v1810: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T00:10:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3080.788063 secs 2023-09-10T00:10:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:50 smithi082 ceph-mon[98694]: pgmap v1810: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T00:10:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:10:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:10:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:10:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:10:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:10:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:10:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:10:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:10:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:10:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:52 smithi170 ceph-mon[102832]: pgmap v1811: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 15 op/s 2023-09-10T00:10:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3083.464707 secs 2023-09-10T00:10:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:52 smithi082 ceph-mon[98694]: pgmap v1811: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 15 op/s 2023-09-10T00:10:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3083.464707 secs 2023-09-10T00:10:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:52 smithi098 ceph-mon[103124]: pgmap v1811: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 15 op/s 2023-09-10T00:10:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3083.464707 secs 2023-09-10T00:10:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3085.075322 secs 2023-09-10T00:10:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3085.075322 secs 2023-09-10T00:10:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3085.075322 secs 2023-09-10T00:10:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:54 smithi170 ceph-mon[102832]: pgmap v1812: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T00:10:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3085.788131 secs 2023-09-10T00:10:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:54 smithi082 ceph-mon[98694]: pgmap v1812: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T00:10:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3085.788131 secs 2023-09-10T00:10:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:54 smithi098 ceph-mon[103124]: pgmap v1812: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T00:10:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3085.788131 secs 2023-09-10T00:10:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:56 smithi170 ceph-mon[102832]: pgmap v1813: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T00:10:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:56 smithi082 ceph-mon[98694]: pgmap v1813: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T00:10:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:56 smithi098 ceph-mon[103124]: pgmap v1813: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T00:10:57.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3088.464772 secs 2023-09-10T00:10:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3088.464772 secs 2023-09-10T00:10:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3088.464772 secs 2023-09-10T00:10:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:58 smithi170 ceph-mon[102832]: pgmap v1814: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 89 KiB/s wr, 15 op/s 2023-09-10T00:10:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:10:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3090.075380 secs 2023-09-10T00:10:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:58 smithi082 ceph-mon[98694]: pgmap v1814: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 89 KiB/s wr, 15 op/s 2023-09-10T00:10:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:10:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3090.075380 secs 2023-09-10T00:10:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:58 smithi098 ceph-mon[103124]: pgmap v1814: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 89 KiB/s wr, 15 op/s 2023-09-10T00:10:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:10:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3090.075380 secs 2023-09-10T00:11:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3090.788195 secs 2023-09-10T00:11:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:00 smithi098 ceph-mon[103124]: pgmap v1815: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-09-10T00:11:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3090.788195 secs 2023-09-10T00:11:00.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:00 smithi170 ceph-mon[102832]: pgmap v1815: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-09-10T00:11:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3090.788195 secs 2023-09-10T00:11:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:00 smithi082 ceph-mon[98694]: pgmap v1815: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-09-10T00:11:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:02 smithi098 ceph-mon[103124]: pgmap v1816: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 28 op/s 2023-09-10T00:11:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:02 smithi170 ceph-mon[102832]: pgmap v1816: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 28 op/s 2023-09-10T00:11:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:02 smithi082 ceph-mon[98694]: pgmap v1816: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 28 op/s 2023-09-10T00:11:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3093.464832 secs 2023-09-10T00:11:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3095.075469 secs 2023-09-10T00:11:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3093.464832 secs 2023-09-10T00:11:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3095.075469 secs 2023-09-10T00:11:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3093.464832 secs 2023-09-10T00:11:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3095.075469 secs 2023-09-10T00:11:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:04 smithi170 ceph-mon[102832]: pgmap v1817: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 49 KiB/s wr, 23 op/s 2023-09-10T00:11:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:04 smithi082 ceph-mon[98694]: pgmap v1817: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 49 KiB/s wr, 23 op/s 2023-09-10T00:11:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:04 smithi098 ceph-mon[103124]: pgmap v1817: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 49 KiB/s wr, 23 op/s 2023-09-10T00:11:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3095.788264 secs 2023-09-10T00:11:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3095.788264 secs 2023-09-10T00:11:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3095.788264 secs 2023-09-10T00:11:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:06 smithi170 ceph-mon[102832]: pgmap v1818: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 49 KiB/s wr, 23 op/s 2023-09-10T00:11:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:06 smithi082 ceph-mon[98694]: pgmap v1818: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 49 KiB/s wr, 23 op/s 2023-09-10T00:11:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:06 smithi098 ceph-mon[103124]: pgmap v1818: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 49 KiB/s wr, 23 op/s 2023-09-10T00:11:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3098.464914 secs 2023-09-10T00:11:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:08 smithi170 ceph-mon[102832]: pgmap v1819: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 80 KiB/s wr, 29 op/s 2023-09-10T00:11:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:08 smithi170 ceph-mon[102832]: osdmap e185: 12 total, 12 up, 12 in 2023-09-10T00:11:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3100.075495 secs 2023-09-10T00:11:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3098.464914 secs 2023-09-10T00:11:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:08 smithi082 ceph-mon[98694]: pgmap v1819: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 80 KiB/s wr, 29 op/s 2023-09-10T00:11:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:08 smithi082 ceph-mon[98694]: osdmap e185: 12 total, 12 up, 12 in 2023-09-10T00:11:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3100.075495 secs 2023-09-10T00:11:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3098.464914 secs 2023-09-10T00:11:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:08 smithi098 ceph-mon[103124]: pgmap v1819: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 80 KiB/s wr, 29 op/s 2023-09-10T00:11:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:08 smithi098 ceph-mon[103124]: osdmap e185: 12 total, 12 up, 12 in 2023-09-10T00:11:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3100.075495 secs 2023-09-10T00:11:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3100.788327 secs 2023-09-10T00:11:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:10 smithi098 ceph-mon[103124]: pgmap v1821: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 77 KiB/s wr, 29 op/s 2023-09-10T00:11:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3100.788327 secs 2023-09-10T00:11:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:10 smithi170 ceph-mon[102832]: pgmap v1821: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 77 KiB/s wr, 29 op/s 2023-09-10T00:11:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3100.788327 secs 2023-09-10T00:11:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:10 smithi082 ceph-mon[98694]: pgmap v1821: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 77 KiB/s wr, 29 op/s 2023-09-10T00:11:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:12 smithi098 ceph-mon[103124]: pgmap v1822: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 56 KiB/s wr, 13 op/s 2023-09-10T00:11:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:12 smithi170 ceph-mon[102832]: pgmap v1822: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 56 KiB/s wr, 13 op/s 2023-09-10T00:11:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:12 smithi082 ceph-mon[98694]: pgmap v1822: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 56 KiB/s wr, 13 op/s 2023-09-10T00:11:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3103.465025 secs 2023-09-10T00:11:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3105.075561 secs 2023-09-10T00:11:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3103.465025 secs 2023-09-10T00:11:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3105.075561 secs 2023-09-10T00:11:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3103.465025 secs 2023-09-10T00:11:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3105.075561 secs 2023-09-10T00:11:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:14 smithi170 ceph-mon[102832]: pgmap v1823: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 56 KiB/s wr, 13 op/s 2023-09-10T00:11:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:14 smithi170 ceph-mon[102832]: osdmap e186: 12 total, 12 up, 12 in 2023-09-10T00:11:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:14 smithi082 ceph-mon[98694]: pgmap v1823: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 56 KiB/s wr, 13 op/s 2023-09-10T00:11:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:14 smithi082 ceph-mon[98694]: osdmap e186: 12 total, 12 up, 12 in 2023-09-10T00:11:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:14 smithi098 ceph-mon[103124]: pgmap v1823: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 56 KiB/s wr, 13 op/s 2023-09-10T00:11:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:14 smithi098 ceph-mon[103124]: osdmap e186: 12 total, 12 up, 12 in 2023-09-10T00:11:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3105.788392 secs 2023-09-10T00:11:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3105.788392 secs 2023-09-10T00:11:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3105.788392 secs 2023-09-10T00:11:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:16 smithi170 ceph-mon[102832]: pgmap v1825: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 7 op/s 2023-09-10T00:11:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:16 smithi082 ceph-mon[98694]: pgmap v1825: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 7 op/s 2023-09-10T00:11:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:16 smithi098 ceph-mon[103124]: pgmap v1825: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 7 op/s 2023-09-10T00:11:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:18 smithi170 ceph-mon[102832]: pgmap v1826: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T00:11:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3110.075628 secs 2023-09-10T00:11:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:18 smithi082 ceph-mon[98694]: pgmap v1826: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T00:11:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3110.075628 secs 2023-09-10T00:11:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:18 smithi098 ceph-mon[103124]: pgmap v1826: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T00:11:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3110.075628 secs 2023-09-10T00:11:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3110.788455 secs 2023-09-10T00:11:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:11:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3110.788455 secs 2023-09-10T00:11:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:11:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3110.788455 secs 2023-09-10T00:11:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:11:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:20 smithi170 ceph-mon[102832]: pgmap v1827: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T00:11:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:11:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:11:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:11:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:20 smithi082 ceph-mon[98694]: pgmap v1827: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T00:11:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:11:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:11:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:11:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:20 smithi098 ceph-mon[103124]: pgmap v1827: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T00:11:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:11:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:11:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:11:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:22 smithi170 ceph-mon[102832]: pgmap v1828: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T00:11:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:22 smithi082 ceph-mon[98694]: pgmap v1828: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T00:11:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:22 smithi098 ceph-mon[103124]: pgmap v1828: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T00:11:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3115.075682 secs 2023-09-10T00:11:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3115.075682 secs 2023-09-10T00:11:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3115.075682 secs 2023-09-10T00:11:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:24 smithi170 ceph-mon[102832]: pgmap v1829: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T00:11:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3115.788486 secs 2023-09-10T00:11:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:24 smithi082 ceph-mon[98694]: pgmap v1829: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T00:11:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3115.788486 secs 2023-09-10T00:11:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:24 smithi098 ceph-mon[103124]: pgmap v1829: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T00:11:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3115.788486 secs 2023-09-10T00:11:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:26 smithi170 ceph-mon[102832]: pgmap v1830: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 26 KiB/s wr, 10 op/s 2023-09-10T00:11:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:26 smithi082 ceph-mon[98694]: pgmap v1830: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 26 KiB/s wr, 10 op/s 2023-09-10T00:11:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:26 smithi098 ceph-mon[103124]: pgmap v1830: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 26 KiB/s wr, 10 op/s 2023-09-10T00:11:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:28 smithi170 ceph-mon[102832]: pgmap v1831: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 105 KiB/s wr, 16 op/s 2023-09-10T00:11:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:11:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3120.075745 secs 2023-09-10T00:11:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:28 smithi082 ceph-mon[98694]: pgmap v1831: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 105 KiB/s wr, 16 op/s 2023-09-10T00:11:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:11:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3120.075745 secs 2023-09-10T00:11:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:28 smithi098 ceph-mon[103124]: pgmap v1831: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 105 KiB/s wr, 16 op/s 2023-09-10T00:11:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:11:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3120.075745 secs 2023-09-10T00:11:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3120.788550 secs 2023-09-10T00:11:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:11:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:11:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:11:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3120.788550 secs 2023-09-10T00:11:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:11:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:11:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:11:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3120.788550 secs 2023-09-10T00:11:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:11:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:11:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:11:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:30 smithi098 ceph-mon[103124]: pgmap v1832: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T00:11:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:30 smithi170 ceph-mon[102832]: pgmap v1832: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T00:11:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:30 smithi082 ceph-mon[98694]: pgmap v1832: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T00:11:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:32 smithi098 ceph-mon[103124]: pgmap v1833: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 155 KiB/s wr, 18 op/s 2023-09-10T00:11:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:32 smithi170 ceph-mon[102832]: pgmap v1833: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 155 KiB/s wr, 18 op/s 2023-09-10T00:11:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:32 smithi082 ceph-mon[98694]: pgmap v1833: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 155 KiB/s wr, 18 op/s 2023-09-10T00:11:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3125.075817 secs 2023-09-10T00:11:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3125.075817 secs 2023-09-10T00:11:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3125.075817 secs 2023-09-10T00:11:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:34 smithi098 ceph-mon[103124]: pgmap v1834: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 149 KiB/s wr, 13 op/s 2023-09-10T00:11:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3125.788614 secs 2023-09-10T00:11:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:34 smithi170 ceph-mon[102832]: pgmap v1834: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 149 KiB/s wr, 13 op/s 2023-09-10T00:11:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3125.788614 secs 2023-09-10T00:11:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:34 smithi082 ceph-mon[98694]: pgmap v1834: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 149 KiB/s wr, 13 op/s 2023-09-10T00:11:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3125.788614 secs 2023-09-10T00:11:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:36 smithi098 ceph-mon[103124]: pgmap v1835: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 149 KiB/s wr, 13 op/s 2023-09-10T00:11:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:36 smithi170 ceph-mon[102832]: pgmap v1835: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 149 KiB/s wr, 13 op/s 2023-09-10T00:11:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:36 smithi082 ceph-mon[98694]: pgmap v1835: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 149 KiB/s wr, 13 op/s 2023-09-10T00:11:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:38 smithi098 ceph-mon[103124]: pgmap v1836: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 170 KiB/s wr, 18 op/s 2023-09-10T00:11:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3130.075873 secs 2023-09-10T00:11:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:38 smithi170 ceph-mon[102832]: pgmap v1836: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 170 KiB/s wr, 18 op/s 2023-09-10T00:11:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3130.075873 secs 2023-09-10T00:11:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:38 smithi082 ceph-mon[98694]: pgmap v1836: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 170 KiB/s wr, 18 op/s 2023-09-10T00:11:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3130.075873 secs 2023-09-10T00:11:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3130.788684 secs 2023-09-10T00:11:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:40 smithi170 ceph-mon[102832]: pgmap v1837: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 89 KiB/s wr, 12 op/s 2023-09-10T00:11:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3130.788684 secs 2023-09-10T00:11:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:40 smithi082 ceph-mon[98694]: pgmap v1837: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 89 KiB/s wr, 12 op/s 2023-09-10T00:11:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3130.788684 secs 2023-09-10T00:11:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:40 smithi098 ceph-mon[103124]: pgmap v1837: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 89 KiB/s wr, 12 op/s 2023-09-10T00:11:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:42 smithi170 ceph-mon[102832]: pgmap v1838: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 103 KiB/s wr, 17 op/s 2023-09-10T00:11:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:42 smithi082 ceph-mon[98694]: pgmap v1838: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 103 KiB/s wr, 17 op/s 2023-09-10T00:11:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:42 smithi098 ceph-mon[103124]: pgmap v1838: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 103 KiB/s wr, 17 op/s 2023-09-10T00:11:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3135.075931 secs 2023-09-10T00:11:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3135.075931 secs 2023-09-10T00:11:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3135.075931 secs 2023-09-10T00:11:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:44 smithi170 ceph-mon[102832]: pgmap v1839: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T00:11:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:44 smithi082 ceph-mon[98694]: pgmap v1839: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T00:11:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:44 smithi098 ceph-mon[103124]: pgmap v1839: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T00:11:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3135.788745 secs 2023-09-10T00:11:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3135.788745 secs 2023-09-10T00:11:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3135.788745 secs 2023-09-10T00:11:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:46 smithi170 ceph-mon[102832]: pgmap v1840: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T00:11:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:46 smithi082 ceph-mon[98694]: pgmap v1840: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T00:11:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:46 smithi098 ceph-mon[103124]: pgmap v1840: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T00:11:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3138.465652 secs 2023-09-10T00:11:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:48 smithi170 ceph-mon[102832]: pgmap v1841: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 92 KiB/s wr, 16 op/s 2023-09-10T00:11:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3140.075989 secs 2023-09-10T00:11:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3138.465652 secs 2023-09-10T00:11:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:48 smithi082 ceph-mon[98694]: pgmap v1841: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 92 KiB/s wr, 16 op/s 2023-09-10T00:11:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3140.075989 secs 2023-09-10T00:11:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3138.465652 secs 2023-09-10T00:11:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:48 smithi098 ceph-mon[103124]: pgmap v1841: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 92 KiB/s wr, 16 op/s 2023-09-10T00:11:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3140.075989 secs 2023-09-10T00:11:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3140.788806 secs 2023-09-10T00:11:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:50 smithi170 ceph-mon[102832]: pgmap v1842: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 71 KiB/s wr, 10 op/s 2023-09-10T00:11:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3140.788806 secs 2023-09-10T00:11:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:50 smithi082 ceph-mon[98694]: pgmap v1842: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 71 KiB/s wr, 10 op/s 2023-09-10T00:11:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3140.788806 secs 2023-09-10T00:11:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:50 smithi098 ceph-mon[103124]: pgmap v1842: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 71 KiB/s wr, 10 op/s 2023-09-10T00:11:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:11:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:11:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:11:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:11:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:11:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:11:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:11:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:11:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:11:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:52 smithi170 ceph-mon[102832]: pgmap v1843: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 89 KiB/s wr, 16 op/s 2023-09-10T00:11:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3143.465715 secs 2023-09-10T00:11:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:52 smithi082 ceph-mon[98694]: pgmap v1843: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 89 KiB/s wr, 16 op/s 2023-09-10T00:11:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3143.465715 secs 2023-09-10T00:11:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:52 smithi098 ceph-mon[103124]: pgmap v1843: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 89 KiB/s wr, 16 op/s 2023-09-10T00:11:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3143.465715 secs 2023-09-10T00:11:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3145.076047 secs 2023-09-10T00:11:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3145.076047 secs 2023-09-10T00:11:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3145.076047 secs 2023-09-10T00:11:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:54 smithi170 ceph-mon[102832]: pgmap v1844: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T00:11:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3145.788870 secs 2023-09-10T00:11:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:54 smithi082 ceph-mon[98694]: pgmap v1844: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T00:11:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3145.788870 secs 2023-09-10T00:11:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:54 smithi098 ceph-mon[103124]: pgmap v1844: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T00:11:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3145.788870 secs 2023-09-10T00:11:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:56 smithi170 ceph-mon[102832]: pgmap v1845: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T00:11:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:56 smithi082 ceph-mon[98694]: pgmap v1845: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T00:11:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:56 smithi098 ceph-mon[103124]: pgmap v1845: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T00:11:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3148.465772 secs 2023-09-10T00:11:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3148.465772 secs 2023-09-10T00:11:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3148.465772 secs 2023-09-10T00:11:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:58 smithi170 ceph-mon[102832]: pgmap v1846: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 87 KiB/s wr, 15 op/s 2023-09-10T00:11:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:11:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3150.076104 secs 2023-09-10T00:11:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:58 smithi082 ceph-mon[98694]: pgmap v1846: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 87 KiB/s wr, 15 op/s 2023-09-10T00:11:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:11:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3150.076104 secs 2023-09-10T00:11:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:58 smithi098 ceph-mon[103124]: pgmap v1846: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 87 KiB/s wr, 15 op/s 2023-09-10T00:11:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:11:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3150.076104 secs 2023-09-10T00:12:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3150.788958 secs 2023-09-10T00:12:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:00 smithi170 ceph-mon[102832]: pgmap v1847: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 31 KiB/s wr, 9 op/s 2023-09-10T00:12:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3150.788958 secs 2023-09-10T00:12:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:00 smithi082 ceph-mon[98694]: pgmap v1847: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 31 KiB/s wr, 9 op/s 2023-09-10T00:12:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3150.788958 secs 2023-09-10T00:12:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:00 smithi098 ceph-mon[103124]: pgmap v1847: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 31 KiB/s wr, 9 op/s 2023-09-10T00:12:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:02 smithi170 ceph-mon[102832]: pgmap v1848: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-09-10T00:12:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:02 smithi082 ceph-mon[98694]: pgmap v1848: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-09-10T00:12:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:02 smithi098 ceph-mon[103124]: pgmap v1848: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 45 KiB/s wr, 21 op/s 2023-09-10T00:12:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3153.465834 secs 2023-09-10T00:12:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3155.076185 secs 2023-09-10T00:12:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3153.465834 secs 2023-09-10T00:12:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3155.076185 secs 2023-09-10T00:12:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3153.465834 secs 2023-09-10T00:12:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3155.076185 secs 2023-09-10T00:12:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:04 smithi170 ceph-mon[102832]: pgmap v1849: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 27 KiB/s wr, 16 op/s 2023-09-10T00:12:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3155.789021 secs 2023-09-10T00:12:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:04 smithi082 ceph-mon[98694]: pgmap v1849: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 27 KiB/s wr, 16 op/s 2023-09-10T00:12:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3155.789021 secs 2023-09-10T00:12:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:04 smithi098 ceph-mon[103124]: pgmap v1849: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 27 KiB/s wr, 16 op/s 2023-09-10T00:12:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3155.789021 secs 2023-09-10T00:12:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:06 smithi170 ceph-mon[102832]: pgmap v1850: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 27 KiB/s wr, 16 op/s 2023-09-10T00:12:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:06 smithi082 ceph-mon[98694]: pgmap v1850: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 27 KiB/s wr, 16 op/s 2023-09-10T00:12:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:06 smithi098 ceph-mon[103124]: pgmap v1850: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 27 KiB/s wr, 16 op/s 2023-09-10T00:12:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3158.465922 secs 2023-09-10T00:12:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:08 smithi170 ceph-mon[102832]: pgmap v1851: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 80 KiB/s wr, 29 op/s 2023-09-10T00:12:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:08 smithi170 ceph-mon[102832]: osdmap e187: 12 total, 12 up, 12 in 2023-09-10T00:12:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3160.076245 secs 2023-09-10T00:12:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3158.465922 secs 2023-09-10T00:12:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:08 smithi082 ceph-mon[98694]: pgmap v1851: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 80 KiB/s wr, 29 op/s 2023-09-10T00:12:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:08 smithi082 ceph-mon[98694]: osdmap e187: 12 total, 12 up, 12 in 2023-09-10T00:12:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3160.076245 secs 2023-09-10T00:12:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3158.465922 secs 2023-09-10T00:12:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:08 smithi098 ceph-mon[103124]: pgmap v1851: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 80 KiB/s wr, 29 op/s 2023-09-10T00:12:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:08 smithi098 ceph-mon[103124]: osdmap e187: 12 total, 12 up, 12 in 2023-09-10T00:12:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3160.076245 secs 2023-09-10T00:12:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3160.789083 secs 2023-09-10T00:12:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:10 smithi170 ceph-mon[102832]: pgmap v1853: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T00:12:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3160.789083 secs 2023-09-10T00:12:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:10 smithi082 ceph-mon[98694]: pgmap v1853: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T00:12:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3160.789083 secs 2023-09-10T00:12:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:10 smithi098 ceph-mon[103124]: pgmap v1853: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T00:12:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:12 smithi170 ceph-mon[102832]: pgmap v1854: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 64 KiB/s wr, 21 op/s 2023-09-10T00:12:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:12 smithi082 ceph-mon[98694]: pgmap v1854: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 64 KiB/s wr, 21 op/s 2023-09-10T00:12:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:12 smithi098 ceph-mon[103124]: pgmap v1854: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 64 KiB/s wr, 21 op/s 2023-09-10T00:12:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3163.466010 secs 2023-09-10T00:12:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3165.076329 secs 2023-09-10T00:12:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3163.466010 secs 2023-09-10T00:12:13.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3165.076329 secs 2023-09-10T00:12:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3163.466010 secs 2023-09-10T00:12:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3165.076329 secs 2023-09-10T00:12:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:14 smithi170 ceph-mon[102832]: pgmap v1855: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 64 KiB/s wr, 21 op/s 2023-09-10T00:12:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:14 smithi170 ceph-mon[102832]: osdmap e188: 12 total, 12 up, 12 in 2023-09-10T00:12:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:14 smithi082 ceph-mon[98694]: pgmap v1855: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 64 KiB/s wr, 21 op/s 2023-09-10T00:12:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:14 smithi082 ceph-mon[98694]: osdmap e188: 12 total, 12 up, 12 in 2023-09-10T00:12:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:14 smithi098 ceph-mon[103124]: pgmap v1855: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 64 KiB/s wr, 21 op/s 2023-09-10T00:12:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:14 smithi098 ceph-mon[103124]: osdmap e188: 12 total, 12 up, 12 in 2023-09-10T00:12:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3165.789148 secs 2023-09-10T00:12:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3165.789148 secs 2023-09-10T00:12:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3165.789148 secs 2023-09-10T00:12:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:16 smithi170 ceph-mon[102832]: pgmap v1857: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2023-09-10T00:12:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:16 smithi082 ceph-mon[98694]: pgmap v1857: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2023-09-10T00:12:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:16 smithi098 ceph-mon[103124]: pgmap v1857: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2023-09-10T00:12:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:18 smithi170 ceph-mon[102832]: pgmap v1858: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T00:12:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3170.076386 secs 2023-09-10T00:12:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:18 smithi082 ceph-mon[98694]: pgmap v1858: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T00:12:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3170.076386 secs 2023-09-10T00:12:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:18 smithi098 ceph-mon[103124]: pgmap v1858: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T00:12:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3170.076386 secs 2023-09-10T00:12:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3170.789211 secs 2023-09-10T00:12:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:12:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3170.789211 secs 2023-09-10T00:12:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:12:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3170.789211 secs 2023-09-10T00:12:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:12:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:20 smithi170 ceph-mon[102832]: pgmap v1859: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T00:12:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:12:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:12:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:12:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:20 smithi082 ceph-mon[98694]: pgmap v1859: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T00:12:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:12:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:12:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:12:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:20 smithi098 ceph-mon[103124]: pgmap v1859: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T00:12:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:12:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:12:20.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:12:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:22 smithi170 ceph-mon[102832]: pgmap v1860: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T00:12:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:22 smithi082 ceph-mon[98694]: pgmap v1860: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T00:12:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:22 smithi098 ceph-mon[103124]: pgmap v1860: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T00:12:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3175.076443 secs 2023-09-10T00:12:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3175.076443 secs 2023-09-10T00:12:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3175.076443 secs 2023-09-10T00:12:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:24 smithi170 ceph-mon[102832]: pgmap v1861: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T00:12:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3175.789276 secs 2023-09-10T00:12:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:24 smithi082 ceph-mon[98694]: pgmap v1861: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T00:12:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3175.789276 secs 2023-09-10T00:12:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:24 smithi098 ceph-mon[103124]: pgmap v1861: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T00:12:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3175.789276 secs 2023-09-10T00:12:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:26 smithi170 ceph-mon[102832]: pgmap v1862: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 10 op/s 2023-09-10T00:12:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:26 smithi082 ceph-mon[98694]: pgmap v1862: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 10 op/s 2023-09-10T00:12:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:26 smithi098 ceph-mon[103124]: pgmap v1862: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 10 op/s 2023-09-10T00:12:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:28 smithi170 ceph-mon[102832]: pgmap v1863: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 15 op/s 2023-09-10T00:12:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3180.076530 secs 2023-09-10T00:12:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:28 smithi082 ceph-mon[98694]: pgmap v1863: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 15 op/s 2023-09-10T00:12:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3180.076530 secs 2023-09-10T00:12:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:28 smithi098 ceph-mon[103124]: pgmap v1863: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 15 op/s 2023-09-10T00:12:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3180.076530 secs 2023-09-10T00:12:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3180.789339 secs 2023-09-10T00:12:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:12:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3180.789339 secs 2023-09-10T00:12:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:12:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3180.789339 secs 2023-09-10T00:12:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:12:30.601 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:30 smithi170 ceph-mon[102832]: pgmap v1864: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T00:12:30.645 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:30 smithi098 ceph-mon[103124]: pgmap v1864: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T00:12:30.655 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:30 smithi082 ceph-mon[98694]: pgmap v1864: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T00:12:32.656 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:32 smithi098 ceph-mon[103124]: pgmap v1865: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 109 KiB/s wr, 16 op/s 2023-09-10T00:12:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:32 smithi170 ceph-mon[102832]: pgmap v1865: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 109 KiB/s wr, 16 op/s 2023-09-10T00:12:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:32 smithi082 ceph-mon[98694]: pgmap v1865: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 109 KiB/s wr, 16 op/s 2023-09-10T00:12:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:12:33.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:12:33.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:12:33.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:12:33.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3185.076589 secs 2023-09-10T00:12:33.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:12:33.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:12:33.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:12:33.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:12:33.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:12:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:12:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:12:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:12:33.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:12:33.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3185.076589 secs 2023-09-10T00:12:33.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:12:33.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:12:33.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:12:33.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:12:33.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:12:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:12:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:12:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:12:33.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:12:33.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3185.076589 secs 2023-09-10T00:12:33.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:12:33.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:12:33.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:12:33.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:12:33.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:12:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:34 smithi170 ceph-mon[102832]: pgmap v1866: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T00:12:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3185.789402 secs 2023-09-10T00:12:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:34 smithi082 ceph-mon[98694]: pgmap v1866: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T00:12:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3185.789402 secs 2023-09-10T00:12:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:34 smithi098 ceph-mon[103124]: pgmap v1866: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T00:12:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3185.789402 secs 2023-09-10T00:12:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:36 smithi170 ceph-mon[102832]: pgmap v1867: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T00:12:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:36 smithi082 ceph-mon[98694]: pgmap v1867: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T00:12:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:36 smithi098 ceph-mon[103124]: pgmap v1867: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T00:12:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:38 smithi170 ceph-mon[102832]: pgmap v1868: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 130 KiB/s wr, 18 op/s 2023-09-10T00:12:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3190.076646 secs 2023-09-10T00:12:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:38 smithi082 ceph-mon[98694]: pgmap v1868: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 130 KiB/s wr, 18 op/s 2023-09-10T00:12:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3190.076646 secs 2023-09-10T00:12:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:38 smithi098 ceph-mon[103124]: pgmap v1868: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 130 KiB/s wr, 18 op/s 2023-09-10T00:12:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3190.076646 secs 2023-09-10T00:12:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3190.789464 secs 2023-09-10T00:12:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3190.789464 secs 2023-09-10T00:12:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3190.789464 secs 2023-09-10T00:12:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:40 smithi170 ceph-mon[102832]: pgmap v1869: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 90 KiB/s wr, 12 op/s 2023-09-10T00:12:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:40 smithi082 ceph-mon[98694]: pgmap v1869: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 90 KiB/s wr, 12 op/s 2023-09-10T00:12:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:40 smithi098 ceph-mon[103124]: pgmap v1869: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 90 KiB/s wr, 12 op/s 2023-09-10T00:12:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:42 smithi170 ceph-mon[102832]: pgmap v1870: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 149 KiB/s wr, 18 op/s 2023-09-10T00:12:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:42 smithi082 ceph-mon[98694]: pgmap v1870: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 149 KiB/s wr, 18 op/s 2023-09-10T00:12:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:42 smithi098 ceph-mon[103124]: pgmap v1870: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 149 KiB/s wr, 18 op/s 2023-09-10T00:12:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3195.076728 secs 2023-09-10T00:12:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3195.076728 secs 2023-09-10T00:12:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3195.076728 secs 2023-09-10T00:12:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:44 smithi170 ceph-mon[102832]: pgmap v1871: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 12 op/s 2023-09-10T00:12:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3195.789490 secs 2023-09-10T00:12:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:44 smithi082 ceph-mon[98694]: pgmap v1871: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 12 op/s 2023-09-10T00:12:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3195.789490 secs 2023-09-10T00:12:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:44 smithi098 ceph-mon[103124]: pgmap v1871: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 12 op/s 2023-09-10T00:12:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3195.789490 secs 2023-09-10T00:12:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:46 smithi170 ceph-mon[102832]: pgmap v1872: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 12 op/s 2023-09-10T00:12:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:46 smithi082 ceph-mon[98694]: pgmap v1872: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 12 op/s 2023-09-10T00:12:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:46 smithi098 ceph-mon[103124]: pgmap v1872: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 12 op/s 2023-09-10T00:12:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3198.466632 secs 2023-09-10T00:12:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3198.466632 secs 2023-09-10T00:12:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3198.466632 secs 2023-09-10T00:12:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:48 smithi170 ceph-mon[102832]: pgmap v1873: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 133 KiB/s wr, 18 op/s 2023-09-10T00:12:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3200.076812 secs 2023-09-10T00:12:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:48 smithi082 ceph-mon[98694]: pgmap v1873: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 133 KiB/s wr, 18 op/s 2023-09-10T00:12:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3200.076812 secs 2023-09-10T00:12:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:48 smithi098 ceph-mon[103124]: pgmap v1873: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 133 KiB/s wr, 18 op/s 2023-09-10T00:12:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3200.076812 secs 2023-09-10T00:12:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3200.789578 secs 2023-09-10T00:12:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3200.789578 secs 2023-09-10T00:12:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3200.789578 secs 2023-09-10T00:12:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:50 smithi170 ceph-mon[102832]: pgmap v1874: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T00:12:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:50 smithi082 ceph-mon[98694]: pgmap v1874: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T00:12:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:50 smithi098 ceph-mon[103124]: pgmap v1874: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T00:12:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:12:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:12:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:12:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:12:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:12:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:12:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:12:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:12:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:12:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:52 smithi082 ceph-mon[98694]: pgmap v1875: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 133 KiB/s wr, 17 op/s 2023-09-10T00:12:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3203.466702 secs 2023-09-10T00:12:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:52 smithi098 ceph-mon[103124]: pgmap v1875: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 133 KiB/s wr, 17 op/s 2023-09-10T00:12:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3203.466702 secs 2023-09-10T00:12:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:52 smithi170 ceph-mon[102832]: pgmap v1875: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 133 KiB/s wr, 17 op/s 2023-09-10T00:12:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3203.466702 secs 2023-09-10T00:12:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3205.076891 secs 2023-09-10T00:12:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3205.076891 secs 2023-09-10T00:12:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3205.076891 secs 2023-09-10T00:12:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:54 smithi082 ceph-mon[98694]: pgmap v1876: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T00:12:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3205.789668 secs 2023-09-10T00:12:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:54 smithi098 ceph-mon[103124]: pgmap v1876: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T00:12:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3205.789668 secs 2023-09-10T00:12:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:54 smithi170 ceph-mon[102832]: pgmap v1876: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T00:12:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3205.789668 secs 2023-09-10T00:12:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:56 smithi098 ceph-mon[103124]: pgmap v1877: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T00:12:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:56 smithi170 ceph-mon[102832]: pgmap v1877: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T00:12:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:56 smithi082 ceph-mon[98694]: pgmap v1877: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T00:12:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3208.466765 secs 2023-09-10T00:12:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3208.466765 secs 2023-09-10T00:12:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3208.466765 secs 2023-09-10T00:12:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:58 smithi098 ceph-mon[103124]: pgmap v1878: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 90 KiB/s wr, 16 op/s 2023-09-10T00:12:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3210.076948 secs 2023-09-10T00:12:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:58 smithi170 ceph-mon[102832]: pgmap v1878: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 90 KiB/s wr, 16 op/s 2023-09-10T00:12:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3210.076948 secs 2023-09-10T00:12:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:58 smithi082 ceph-mon[98694]: pgmap v1878: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 90 KiB/s wr, 16 op/s 2023-09-10T00:12:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3210.076948 secs 2023-09-10T00:12:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:12:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3210.789732 secs 2023-09-10T00:12:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:12:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3210.789732 secs 2023-09-10T00:12:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:12:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3210.789732 secs 2023-09-10T00:13:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:00 smithi098 ceph-mon[103124]: pgmap v1879: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T00:13:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:00 smithi170 ceph-mon[102832]: pgmap v1879: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T00:13:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:00 smithi082 ceph-mon[98694]: pgmap v1879: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T00:13:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:02 smithi098 ceph-mon[103124]: pgmap v1880: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 47 KiB/s wr, 15 op/s 2023-09-10T00:13:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3213.466823 secs 2023-09-10T00:13:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:02 smithi170 ceph-mon[102832]: pgmap v1880: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 47 KiB/s wr, 15 op/s 2023-09-10T00:13:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3213.466823 secs 2023-09-10T00:13:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:02 smithi082 ceph-mon[98694]: pgmap v1880: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 47 KiB/s wr, 15 op/s 2023-09-10T00:13:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3213.466823 secs 2023-09-10T00:13:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3215.077010 secs 2023-09-10T00:13:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3215.077010 secs 2023-09-10T00:13:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3215.077010 secs 2023-09-10T00:13:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:04 smithi098 ceph-mon[103124]: pgmap v1881: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 19 KiB/s wr, 9 op/s 2023-09-10T00:13:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3215.789798 secs 2023-09-10T00:13:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:04 smithi170 ceph-mon[102832]: pgmap v1881: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 19 KiB/s wr, 9 op/s 2023-09-10T00:13:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3215.789798 secs 2023-09-10T00:13:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:04 smithi082 ceph-mon[98694]: pgmap v1881: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 19 KiB/s wr, 9 op/s 2023-09-10T00:13:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3215.789798 secs 2023-09-10T00:13:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:06 smithi098 ceph-mon[103124]: pgmap v1882: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 19 KiB/s wr, 9 op/s 2023-09-10T00:13:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:06 smithi170 ceph-mon[102832]: pgmap v1882: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 19 KiB/s wr, 9 op/s 2023-09-10T00:13:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:06 smithi082 ceph-mon[98694]: pgmap v1882: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 19 KiB/s wr, 9 op/s 2023-09-10T00:13:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3218.466926 secs 2023-09-10T00:13:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3218.466926 secs 2023-09-10T00:13:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3218.466926 secs 2023-09-10T00:13:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:08 smithi170 ceph-mon[102832]: pgmap v1883: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 80 KiB/s wr, 29 op/s 2023-09-10T00:13:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:08 smithi170 ceph-mon[102832]: osdmap e189: 12 total, 12 up, 12 in 2023-09-10T00:13:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3220.077066 secs 2023-09-10T00:13:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:08 smithi082 ceph-mon[98694]: pgmap v1883: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 80 KiB/s wr, 29 op/s 2023-09-10T00:13:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:08 smithi082 ceph-mon[98694]: osdmap e189: 12 total, 12 up, 12 in 2023-09-10T00:13:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3220.077066 secs 2023-09-10T00:13:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:08 smithi098 ceph-mon[103124]: pgmap v1883: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 80 KiB/s wr, 29 op/s 2023-09-10T00:13:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:08 smithi098 ceph-mon[103124]: osdmap e189: 12 total, 12 up, 12 in 2023-09-10T00:13:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3220.077066 secs 2023-09-10T00:13:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3220.789863 secs 2023-09-10T00:13:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3220.789863 secs 2023-09-10T00:13:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3220.789863 secs 2023-09-10T00:13:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:10 smithi170 ceph-mon[102832]: pgmap v1885: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 29 op/s 2023-09-10T00:13:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:10 smithi082 ceph-mon[98694]: pgmap v1885: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 29 op/s 2023-09-10T00:13:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:10 smithi098 ceph-mon[103124]: pgmap v1885: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 29 op/s 2023-09-10T00:13:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:12 smithi170 ceph-mon[102832]: pgmap v1886: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T00:13:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3223.467041 secs 2023-09-10T00:13:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:12 smithi082 ceph-mon[98694]: pgmap v1886: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T00:13:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3223.467041 secs 2023-09-10T00:13:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:12 smithi098 ceph-mon[103124]: pgmap v1886: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T00:13:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3223.467041 secs 2023-09-10T00:13:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3225.077122 secs 2023-09-10T00:13:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3225.077122 secs 2023-09-10T00:13:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3225.077122 secs 2023-09-10T00:13:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:14 smithi170 ceph-mon[102832]: pgmap v1887: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T00:13:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3225.789933 secs 2023-09-10T00:13:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:14 smithi170 ceph-mon[102832]: osdmap e190: 12 total, 12 up, 12 in 2023-09-10T00:13:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:14 smithi082 ceph-mon[98694]: pgmap v1887: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T00:13:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3225.789933 secs 2023-09-10T00:13:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:14 smithi082 ceph-mon[98694]: osdmap e190: 12 total, 12 up, 12 in 2023-09-10T00:13:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:14 smithi098 ceph-mon[103124]: pgmap v1887: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T00:13:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3225.789933 secs 2023-09-10T00:13:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:14 smithi098 ceph-mon[103124]: osdmap e190: 12 total, 12 up, 12 in 2023-09-10T00:13:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:16 smithi170 ceph-mon[102832]: pgmap v1889: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 28 KiB/s wr, 7 op/s 2023-09-10T00:13:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:16 smithi082 ceph-mon[98694]: pgmap v1889: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 28 KiB/s wr, 7 op/s 2023-09-10T00:13:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:16 smithi098 ceph-mon[103124]: pgmap v1889: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 28 KiB/s wr, 7 op/s 2023-09-10T00:13:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3228.467119 secs 2023-09-10T00:13:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3228.467119 secs 2023-09-10T00:13:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3228.467119 secs 2023-09-10T00:13:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:18 smithi170 ceph-mon[102832]: pgmap v1890: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T00:13:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3230.077180 secs 2023-09-10T00:13:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:18 smithi082 ceph-mon[98694]: pgmap v1890: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T00:13:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3230.077180 secs 2023-09-10T00:13:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:18 smithi098 ceph-mon[103124]: pgmap v1890: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T00:13:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3230.077180 secs 2023-09-10T00:13:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3230.789993 secs 2023-09-10T00:13:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3230.789993 secs 2023-09-10T00:13:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3230.789993 secs 2023-09-10T00:13:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:20 smithi170 ceph-mon[102832]: pgmap v1891: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-10T00:13:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:20 smithi082 ceph-mon[98694]: pgmap v1891: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-10T00:13:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:20 smithi098 ceph-mon[103124]: pgmap v1891: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-10T00:13:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:22 smithi170 ceph-mon[102832]: pgmap v1892: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T00:13:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:22 smithi082 ceph-mon[98694]: pgmap v1892: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T00:13:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:22 smithi098 ceph-mon[103124]: pgmap v1892: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T00:13:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:13:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3235.077237 secs 2023-09-10T00:13:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:13:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3235.077237 secs 2023-09-10T00:13:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:13:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3235.077237 secs 2023-09-10T00:13:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:24 smithi170 ceph-mon[102832]: pgmap v1893: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T00:13:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3235.790056 secs 2023-09-10T00:13:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:13:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:13:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:13:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:24 smithi082 ceph-mon[98694]: pgmap v1893: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T00:13:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3235.790056 secs 2023-09-10T00:13:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:13:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:13:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:13:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:24 smithi098 ceph-mon[103124]: pgmap v1893: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T00:13:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3235.790056 secs 2023-09-10T00:13:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:13:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:13:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:13:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:26 smithi170 ceph-mon[102832]: pgmap v1894: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T00:13:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:26 smithi082 ceph-mon[98694]: pgmap v1894: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T00:13:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:26 smithi098 ceph-mon[103124]: pgmap v1894: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T00:13:28.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:28 smithi170 ceph-mon[102832]: pgmap v1895: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 121 KiB/s wr, 16 op/s 2023-09-10T00:13:28.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3240.077294 secs 2023-09-10T00:13:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:28 smithi082 ceph-mon[98694]: pgmap v1895: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 121 KiB/s wr, 16 op/s 2023-09-10T00:13:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3240.077294 secs 2023-09-10T00:13:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:28 smithi098 ceph-mon[103124]: pgmap v1895: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 121 KiB/s wr, 16 op/s 2023-09-10T00:13:29.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3240.077294 secs 2023-09-10T00:13:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3240.790118 secs 2023-09-10T00:13:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3240.790118 secs 2023-09-10T00:13:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3240.790118 secs 2023-09-10T00:13:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:30 smithi170 ceph-mon[102832]: pgmap v1896: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T00:13:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:30 smithi082 ceph-mon[98694]: pgmap v1896: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T00:13:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:30 smithi098 ceph-mon[103124]: pgmap v1896: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T00:13:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:32 smithi170 ceph-mon[102832]: pgmap v1897: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 17 op/s 2023-09-10T00:13:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:32 smithi082 ceph-mon[98694]: pgmap v1897: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 17 op/s 2023-09-10T00:13:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:32 smithi098 ceph-mon[103124]: pgmap v1897: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 17 op/s 2023-09-10T00:13:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3245.077356 secs 2023-09-10T00:13:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:13:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3245.077356 secs 2023-09-10T00:13:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:13:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3245.077356 secs 2023-09-10T00:13:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:13:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:34 smithi170 ceph-mon[102832]: pgmap v1898: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-10T00:13:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3245.790187 secs 2023-09-10T00:13:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:13:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:13:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:13:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:34 smithi082 ceph-mon[98694]: pgmap v1898: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-10T00:13:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3245.790187 secs 2023-09-10T00:13:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:13:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:13:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:13:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:34 smithi098 ceph-mon[103124]: pgmap v1898: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-10T00:13:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3245.790187 secs 2023-09-10T00:13:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:13:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:13:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:13:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:36 smithi170 ceph-mon[102832]: pgmap v1899: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-10T00:13:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:36 smithi082 ceph-mon[98694]: pgmap v1899: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-10T00:13:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:36 smithi098 ceph-mon[103124]: pgmap v1899: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-10T00:13:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:38 smithi170 ceph-mon[102832]: pgmap v1900: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 150 KiB/s wr, 18 op/s 2023-09-10T00:13:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3250.077445 secs 2023-09-10T00:13:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:38 smithi082 ceph-mon[98694]: pgmap v1900: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 150 KiB/s wr, 18 op/s 2023-09-10T00:13:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3250.077445 secs 2023-09-10T00:13:39.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:38 smithi098 ceph-mon[103124]: pgmap v1900: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 150 KiB/s wr, 18 op/s 2023-09-10T00:13:39.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3250.077445 secs 2023-09-10T00:13:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3250.790251 secs 2023-09-10T00:13:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3250.790251 secs 2023-09-10T00:13:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3250.790251 secs 2023-09-10T00:13:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:40 smithi170 ceph-mon[102832]: pgmap v1901: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T00:13:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:40 smithi082 ceph-mon[98694]: pgmap v1901: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T00:13:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:40 smithi098 ceph-mon[103124]: pgmap v1901: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T00:13:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:42 smithi170 ceph-mon[102832]: pgmap v1902: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 150 KiB/s wr, 18 op/s 2023-09-10T00:13:42.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3253.467480 secs 2023-09-10T00:13:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:42 smithi082 ceph-mon[98694]: pgmap v1902: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 150 KiB/s wr, 18 op/s 2023-09-10T00:13:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3253.467480 secs 2023-09-10T00:13:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:42 smithi098 ceph-mon[103124]: pgmap v1902: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 150 KiB/s wr, 18 op/s 2023-09-10T00:13:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3253.467480 secs 2023-09-10T00:13:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3255.077533 secs 2023-09-10T00:13:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3255.077533 secs 2023-09-10T00:13:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3255.077533 secs 2023-09-10T00:13:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:44 smithi170 ceph-mon[102832]: pgmap v1903: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T00:13:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3255.790312 secs 2023-09-10T00:13:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:13:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:13:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:13:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:44 smithi082 ceph-mon[98694]: pgmap v1903: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T00:13:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3255.790312 secs 2023-09-10T00:13:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:13:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:13:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:13:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:44 smithi098 ceph-mon[103124]: pgmap v1903: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T00:13:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3255.790312 secs 2023-09-10T00:13:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:13:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:13:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:13:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:46 smithi170 ceph-mon[102832]: pgmap v1904: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T00:13:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:46 smithi082 ceph-mon[98694]: pgmap v1904: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T00:13:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:46 smithi098 ceph-mon[103124]: pgmap v1904: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T00:13:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3258.467540 secs 2023-09-10T00:13:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3258.467540 secs 2023-09-10T00:13:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3258.467540 secs 2023-09-10T00:13:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:48 smithi170 ceph-mon[102832]: pgmap v1905: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 142 KiB/s wr, 17 op/s 2023-09-10T00:13:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3260.077593 secs 2023-09-10T00:13:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:48 smithi082 ceph-mon[98694]: pgmap v1905: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 142 KiB/s wr, 17 op/s 2023-09-10T00:13:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3260.077593 secs 2023-09-10T00:13:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:48 smithi098 ceph-mon[103124]: pgmap v1905: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 142 KiB/s wr, 17 op/s 2023-09-10T00:13:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3260.077593 secs 2023-09-10T00:13:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3260.790377 secs 2023-09-10T00:13:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3260.790377 secs 2023-09-10T00:13:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3260.790377 secs 2023-09-10T00:13:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:50 smithi170 ceph-mon[102832]: pgmap v1906: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T00:13:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:50 smithi082 ceph-mon[98694]: pgmap v1906: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T00:13:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:50 smithi098 ceph-mon[103124]: pgmap v1906: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T00:13:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:52 smithi170 ceph-mon[102832]: pgmap v1907: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 93 KiB/s wr, 15 op/s 2023-09-10T00:13:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3263.467619 secs 2023-09-10T00:13:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:52 smithi082 ceph-mon[98694]: pgmap v1907: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 93 KiB/s wr, 15 op/s 2023-09-10T00:13:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3263.467619 secs 2023-09-10T00:13:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:52 smithi098 ceph-mon[103124]: pgmap v1907: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 93 KiB/s wr, 15 op/s 2023-09-10T00:13:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3263.467619 secs 2023-09-10T00:13:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3265.077676 secs 2023-09-10T00:13:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3265.077676 secs 2023-09-10T00:13:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3265.077676 secs 2023-09-10T00:13:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:54 smithi170 ceph-mon[102832]: pgmap v1908: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 68 KiB/s wr, 9 op/s 2023-09-10T00:13:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3265.790438 secs 2023-09-10T00:13:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:54 smithi082 ceph-mon[98694]: pgmap v1908: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 68 KiB/s wr, 9 op/s 2023-09-10T00:13:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3265.790438 secs 2023-09-10T00:13:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:54 smithi098 ceph-mon[103124]: pgmap v1908: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 68 KiB/s wr, 9 op/s 2023-09-10T00:13:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3265.790438 secs 2023-09-10T00:13:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:56 smithi170 ceph-mon[102832]: pgmap v1909: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T00:13:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:56 smithi082 ceph-mon[98694]: pgmap v1909: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T00:13:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:56 smithi098 ceph-mon[103124]: pgmap v1909: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T00:13:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3268.467681 secs 2023-09-10T00:13:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3268.467681 secs 2023-09-10T00:13:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3268.467681 secs 2023-09-10T00:13:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:58 smithi170 ceph-mon[102832]: pgmap v1910: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 15 op/s 2023-09-10T00:13:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3270.077735 secs 2023-09-10T00:13:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:58 smithi082 ceph-mon[98694]: pgmap v1910: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 15 op/s 2023-09-10T00:13:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3270.077735 secs 2023-09-10T00:13:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:58 smithi098 ceph-mon[103124]: pgmap v1910: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 15 op/s 2023-09-10T00:13:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3270.077735 secs 2023-09-10T00:13:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:13:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3270.790485 secs 2023-09-10T00:13:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:13:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3270.790485 secs 2023-09-10T00:14:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:13:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3270.790485 secs 2023-09-10T00:14:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:00 smithi170 ceph-mon[102832]: pgmap v1911: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T00:14:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:00 smithi082 ceph-mon[98694]: pgmap v1911: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T00:14:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:00 smithi098 ceph-mon[103124]: pgmap v1911: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T00:14:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:02 smithi170 ceph-mon[102832]: pgmap v1912: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 36 KiB/s wr, 15 op/s 2023-09-10T00:14:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3273.467746 secs 2023-09-10T00:14:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:02 smithi082 ceph-mon[98694]: pgmap v1912: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 36 KiB/s wr, 15 op/s 2023-09-10T00:14:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3273.467746 secs 2023-09-10T00:14:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:02 smithi098 ceph-mon[103124]: pgmap v1912: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 36 KiB/s wr, 15 op/s 2023-09-10T00:14:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3273.467746 secs 2023-09-10T00:14:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3275.077794 secs 2023-09-10T00:14:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3275.077794 secs 2023-09-10T00:14:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3275.077794 secs 2023-09-10T00:14:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:04 smithi170 ceph-mon[102832]: pgmap v1913: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 KiB/s wr, 10 op/s 2023-09-10T00:14:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3275.790550 secs 2023-09-10T00:14:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:04 smithi082 ceph-mon[98694]: pgmap v1913: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 KiB/s wr, 10 op/s 2023-09-10T00:14:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3275.790550 secs 2023-09-10T00:14:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:04 smithi098 ceph-mon[103124]: pgmap v1913: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 KiB/s wr, 10 op/s 2023-09-10T00:14:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3275.790550 secs 2023-09-10T00:14:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:06 smithi170 ceph-mon[102832]: pgmap v1914: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 61 KiB/s wr, 27 op/s 2023-09-10T00:14:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:06 smithi082 ceph-mon[98694]: pgmap v1914: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 61 KiB/s wr, 27 op/s 2023-09-10T00:14:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:06 smithi098 ceph-mon[103124]: pgmap v1914: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 61 KiB/s wr, 27 op/s 2023-09-10T00:14:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3278.467830 secs 2023-09-10T00:14:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3278.467830 secs 2023-09-10T00:14:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3278.467830 secs 2023-09-10T00:14:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:08 smithi170 ceph-mon[102832]: pgmap v1915: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 78 KiB/s wr, 26 op/s 2023-09-10T00:14:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:08 smithi170 ceph-mon[102832]: osdmap e191: 12 total, 12 up, 12 in 2023-09-10T00:14:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3280.077854 secs 2023-09-10T00:14:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:08 smithi082 ceph-mon[98694]: pgmap v1915: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 78 KiB/s wr, 26 op/s 2023-09-10T00:14:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:08 smithi082 ceph-mon[98694]: osdmap e191: 12 total, 12 up, 12 in 2023-09-10T00:14:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3280.077854 secs 2023-09-10T00:14:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:08 smithi098 ceph-mon[103124]: pgmap v1915: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 78 KiB/s wr, 26 op/s 2023-09-10T00:14:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:08 smithi098 ceph-mon[103124]: osdmap e191: 12 total, 12 up, 12 in 2023-09-10T00:14:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3280.077854 secs 2023-09-10T00:14:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3280.790613 secs 2023-09-10T00:14:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3280.790613 secs 2023-09-10T00:14:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3280.790613 secs 2023-09-10T00:14:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:10 smithi082 ceph-mon[98694]: pgmap v1917: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T00:14:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:10 smithi098 ceph-mon[103124]: pgmap v1917: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T00:14:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:10 smithi170 ceph-mon[102832]: pgmap v1917: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T00:14:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:12 smithi082 ceph-mon[98694]: pgmap v1918: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T00:14:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3283.467890 secs 2023-09-10T00:14:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:12 smithi098 ceph-mon[103124]: pgmap v1918: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T00:14:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3283.467890 secs 2023-09-10T00:14:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:12 smithi170 ceph-mon[102832]: pgmap v1918: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T00:14:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3283.467890 secs 2023-09-10T00:14:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3285.077939 secs 2023-09-10T00:14:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3285.077939 secs 2023-09-10T00:14:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3285.077939 secs 2023-09-10T00:14:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:14 smithi082 ceph-mon[98694]: pgmap v1919: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T00:14:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3285.790676 secs 2023-09-10T00:14:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:14 smithi082 ceph-mon[98694]: osdmap e192: 12 total, 12 up, 12 in 2023-09-10T00:14:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:14 smithi098 ceph-mon[103124]: pgmap v1919: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T00:14:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3285.790676 secs 2023-09-10T00:14:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:14 smithi098 ceph-mon[103124]: osdmap e192: 12 total, 12 up, 12 in 2023-09-10T00:14:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:14 smithi170 ceph-mon[102832]: pgmap v1919: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T00:14:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3285.790676 secs 2023-09-10T00:14:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:14 smithi170 ceph-mon[102832]: osdmap e192: 12 total, 12 up, 12 in 2023-09-10T00:14:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:16 smithi082 ceph-mon[98694]: pgmap v1921: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-09-10T00:14:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:16 smithi098 ceph-mon[103124]: pgmap v1921: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-09-10T00:14:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:16 smithi170 ceph-mon[102832]: pgmap v1921: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-09-10T00:14:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3288.467950 secs 2023-09-10T00:14:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3288.467950 secs 2023-09-10T00:14:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3288.467950 secs 2023-09-10T00:14:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:18 smithi082 ceph-mon[98694]: pgmap v1922: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T00:14:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3290.077999 secs 2023-09-10T00:14:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:18 smithi098 ceph-mon[103124]: pgmap v1922: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T00:14:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3290.077999 secs 2023-09-10T00:14:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:18 smithi170 ceph-mon[102832]: pgmap v1922: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T00:14:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3290.077999 secs 2023-09-10T00:14:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3290.790739 secs 2023-09-10T00:14:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3290.790739 secs 2023-09-10T00:14:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3290.790739 secs 2023-09-10T00:14:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:20 smithi098 ceph-mon[103124]: pgmap v1923: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T00:14:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:20 smithi170 ceph-mon[102832]: pgmap v1923: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T00:14:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:20 smithi082 ceph-mon[98694]: pgmap v1923: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T00:14:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:22 smithi098 ceph-mon[103124]: pgmap v1924: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T00:14:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3293.468012 secs 2023-09-10T00:14:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:22 smithi170 ceph-mon[102832]: pgmap v1924: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T00:14:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3293.468012 secs 2023-09-10T00:14:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:22 smithi082 ceph-mon[98694]: pgmap v1924: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T00:14:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3293.468012 secs 2023-09-10T00:14:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3295.078058 secs 2023-09-10T00:14:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3295.078058 secs 2023-09-10T00:14:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3295.078058 secs 2023-09-10T00:14:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:24 smithi098 ceph-mon[103124]: pgmap v1925: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T00:14:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3295.790802 secs 2023-09-10T00:14:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:24 smithi170 ceph-mon[102832]: pgmap v1925: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T00:14:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3295.790802 secs 2023-09-10T00:14:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:24 smithi082 ceph-mon[98694]: pgmap v1925: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T00:14:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3295.790802 secs 2023-09-10T00:14:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:26 smithi098 ceph-mon[103124]: pgmap v1926: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 14 op/s 2023-09-10T00:14:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:26 smithi170 ceph-mon[102832]: pgmap v1926: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 14 op/s 2023-09-10T00:14:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:26 smithi082 ceph-mon[98694]: pgmap v1926: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 14 op/s 2023-09-10T00:14:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:28 smithi098 ceph-mon[103124]: pgmap v1927: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 70 KiB/s wr, 12 op/s 2023-09-10T00:14:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3300.078115 secs 2023-09-10T00:14:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:28 smithi170 ceph-mon[102832]: pgmap v1927: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 70 KiB/s wr, 12 op/s 2023-09-10T00:14:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3300.078115 secs 2023-09-10T00:14:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:28 smithi082 ceph-mon[98694]: pgmap v1927: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 70 KiB/s wr, 12 op/s 2023-09-10T00:14:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3300.078115 secs 2023-09-10T00:14:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3300.790863 secs 2023-09-10T00:14:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3300.790863 secs 2023-09-10T00:14:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3300.790863 secs 2023-09-10T00:14:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:30 smithi098 ceph-mon[103124]: pgmap v1928: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 47 KiB/s wr, 10 op/s 2023-09-10T00:14:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:30 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:14:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:30 smithi170 ceph-mon[102832]: pgmap v1928: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 47 KiB/s wr, 10 op/s 2023-09-10T00:14:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:30 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:14:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:30 smithi082 ceph-mon[98694]: pgmap v1928: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 47 KiB/s wr, 10 op/s 2023-09-10T00:14:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:30 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:14:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:31 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:14:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:14:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:14:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:31 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:14:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:14:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:14:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:31 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:14:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:14:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:14:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:32 smithi098 ceph-mon[103124]: pgmap v1929: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 112 KiB/s wr, 17 op/s 2023-09-10T00:14:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:32 smithi170 ceph-mon[102832]: pgmap v1929: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 112 KiB/s wr, 17 op/s 2023-09-10T00:14:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:32 smithi082 ceph-mon[98694]: pgmap v1929: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 112 KiB/s wr, 17 op/s 2023-09-10T00:14:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3305.078174 secs 2023-09-10T00:14:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3305.078174 secs 2023-09-10T00:14:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3305.078174 secs 2023-09-10T00:14:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:34 smithi098 ceph-mon[103124]: pgmap v1930: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T00:14:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3305.790925 secs 2023-09-10T00:14:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:14:35.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:14:35.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:14:35.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:14:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:34 smithi170 ceph-mon[102832]: pgmap v1930: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T00:14:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3305.790925 secs 2023-09-10T00:14:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:14:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:14:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:14:35.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:14:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:34 smithi082 ceph-mon[98694]: pgmap v1930: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T00:14:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3305.790925 secs 2023-09-10T00:14:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:14:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:14:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:14:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:14:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:36 smithi098 ceph-mon[103124]: pgmap v1931: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 110 KiB/s wr, 15 op/s 2023-09-10T00:14:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:36 smithi170 ceph-mon[102832]: pgmap v1931: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 110 KiB/s wr, 15 op/s 2023-09-10T00:14:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:36 smithi082 ceph-mon[98694]: pgmap v1931: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 110 KiB/s wr, 15 op/s 2023-09-10T00:14:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:38 smithi098 ceph-mon[103124]: pgmap v1932: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 131 KiB/s wr, 15 op/s 2023-09-10T00:14:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3310.078234 secs 2023-09-10T00:14:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:38 smithi170 ceph-mon[102832]: pgmap v1932: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 131 KiB/s wr, 15 op/s 2023-09-10T00:14:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3310.078234 secs 2023-09-10T00:14:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:38 smithi082 ceph-mon[98694]: pgmap v1932: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 131 KiB/s wr, 15 op/s 2023-09-10T00:14:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3310.078234 secs 2023-09-10T00:14:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3310.790985 secs 2023-09-10T00:14:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3310.790985 secs 2023-09-10T00:14:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3310.790985 secs 2023-09-10T00:14:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:40 smithi170 ceph-mon[102832]: pgmap v1933: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 94 KiB/s wr, 12 op/s 2023-09-10T00:14:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:40 smithi082 ceph-mon[98694]: pgmap v1933: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 94 KiB/s wr, 12 op/s 2023-09-10T00:14:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:40 smithi098 ceph-mon[103124]: pgmap v1933: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 94 KiB/s wr, 12 op/s 2023-09-10T00:14:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:42 smithi170 ceph-mon[102832]: pgmap v1934: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 183 KiB/s wr, 18 op/s 2023-09-10T00:14:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:42 smithi082 ceph-mon[98694]: pgmap v1934: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 183 KiB/s wr, 18 op/s 2023-09-10T00:14:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:42 smithi098 ceph-mon[103124]: pgmap v1934: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 183 KiB/s wr, 18 op/s 2023-09-10T00:14:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3315.078295 secs 2023-09-10T00:14:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3315.078295 secs 2023-09-10T00:14:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3315.078295 secs 2023-09-10T00:14:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:44 smithi170 ceph-mon[102832]: pgmap v1935: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T00:14:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3315.791049 secs 2023-09-10T00:14:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:44 smithi082 ceph-mon[98694]: pgmap v1935: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T00:14:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3315.791049 secs 2023-09-10T00:14:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:44 smithi098 ceph-mon[103124]: pgmap v1935: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T00:14:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3315.791049 secs 2023-09-10T00:14:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:46 smithi170 ceph-mon[102832]: pgmap v1936: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 125 KiB/s wr, 15 op/s 2023-09-10T00:14:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:46 smithi082 ceph-mon[98694]: pgmap v1936: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 125 KiB/s wr, 15 op/s 2023-09-10T00:14:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:46 smithi098 ceph-mon[103124]: pgmap v1936: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 125 KiB/s wr, 15 op/s 2023-09-10T00:14:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3318.468397 secs 2023-09-10T00:14:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3318.468397 secs 2023-09-10T00:14:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3318.468397 secs 2023-09-10T00:14:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:48 smithi170 ceph-mon[102832]: pgmap v1937: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 145 KiB/s wr, 14 op/s 2023-09-10T00:14:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3320.078353 secs 2023-09-10T00:14:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:48 smithi082 ceph-mon[98694]: pgmap v1937: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 145 KiB/s wr, 14 op/s 2023-09-10T00:14:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3320.078353 secs 2023-09-10T00:14:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:48 smithi098 ceph-mon[103124]: pgmap v1937: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 145 KiB/s wr, 14 op/s 2023-09-10T00:14:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3320.078353 secs 2023-09-10T00:14:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3320.791109 secs 2023-09-10T00:14:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3320.791109 secs 2023-09-10T00:14:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3320.791109 secs 2023-09-10T00:14:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:50 smithi170 ceph-mon[102832]: pgmap v1938: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T00:14:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:50 smithi082 ceph-mon[98694]: pgmap v1938: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T00:14:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:50 smithi098 ceph-mon[103124]: pgmap v1938: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T00:14:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:14:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:14:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:14:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:14:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:14:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:14:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:14:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:14:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:14:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:52 smithi170 ceph-mon[102832]: pgmap v1939: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 148 KiB/s wr, 16 op/s 2023-09-10T00:14:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3323.468456 secs 2023-09-10T00:14:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:52 smithi082 ceph-mon[98694]: pgmap v1939: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 148 KiB/s wr, 16 op/s 2023-09-10T00:14:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3323.468456 secs 2023-09-10T00:14:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:52 smithi098 ceph-mon[103124]: pgmap v1939: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 148 KiB/s wr, 16 op/s 2023-09-10T00:14:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3323.468456 secs 2023-09-10T00:14:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3325.078413 secs 2023-09-10T00:14:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3325.078413 secs 2023-09-10T00:14:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3325.078413 secs 2023-09-10T00:14:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:54 smithi170 ceph-mon[102832]: pgmap v1940: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T00:14:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3325.791169 secs 2023-09-10T00:14:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:54 smithi082 ceph-mon[98694]: pgmap v1940: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T00:14:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3325.791169 secs 2023-09-10T00:14:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:54 smithi098 ceph-mon[103124]: pgmap v1940: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T00:14:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3325.791169 secs 2023-09-10T00:14:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:56 smithi170 ceph-mon[102832]: pgmap v1941: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 62 KiB/s wr, 13 op/s 2023-09-10T00:14:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:56 smithi082 ceph-mon[98694]: pgmap v1941: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 62 KiB/s wr, 13 op/s 2023-09-10T00:14:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:56 smithi098 ceph-mon[103124]: pgmap v1941: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 62 KiB/s wr, 13 op/s 2023-09-10T00:14:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3328.468518 secs 2023-09-10T00:14:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3328.468518 secs 2023-09-10T00:14:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3328.468518 secs 2023-09-10T00:14:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:58 smithi170 ceph-mon[102832]: pgmap v1942: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T00:14:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3330.078474 secs 2023-09-10T00:14:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:58 smithi082 ceph-mon[98694]: pgmap v1942: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T00:14:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3330.078474 secs 2023-09-10T00:14:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:58 smithi098 ceph-mon[103124]: pgmap v1942: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T00:14:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3330.078474 secs 2023-09-10T00:15:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:14:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3330.791237 secs 2023-09-10T00:15:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:14:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3330.791237 secs 2023-09-10T00:15:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:14:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3330.791237 secs 2023-09-10T00:15:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:00 smithi170 ceph-mon[102832]: pgmap v1943: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 10 op/s 2023-09-10T00:15:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:00 smithi082 ceph-mon[98694]: pgmap v1943: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 10 op/s 2023-09-10T00:15:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:00 smithi098 ceph-mon[103124]: pgmap v1943: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 10 op/s 2023-09-10T00:15:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:02 smithi170 ceph-mon[102832]: pgmap v1944: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 15 op/s 2023-09-10T00:15:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3333.468660 secs 2023-09-10T00:15:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:02 smithi082 ceph-mon[98694]: pgmap v1944: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 15 op/s 2023-09-10T00:15:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3333.468660 secs 2023-09-10T00:15:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:02 smithi098 ceph-mon[103124]: pgmap v1944: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 15 op/s 2023-09-10T00:15:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3333.468660 secs 2023-09-10T00:15:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3335.078490 secs 2023-09-10T00:15:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3335.078490 secs 2023-09-10T00:15:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3335.078490 secs 2023-09-10T00:15:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:04 smithi170 ceph-mon[102832]: pgmap v1945: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 KiB/s wr, 10 op/s 2023-09-10T00:15:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3335.791299 secs 2023-09-10T00:15:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:04 smithi082 ceph-mon[98694]: pgmap v1945: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 KiB/s wr, 10 op/s 2023-09-10T00:15:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3335.791299 secs 2023-09-10T00:15:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:04 smithi098 ceph-mon[103124]: pgmap v1945: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 KiB/s wr, 10 op/s 2023-09-10T00:15:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3335.791299 secs 2023-09-10T00:15:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:06 smithi170 ceph-mon[102832]: pgmap v1946: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 64 KiB/s wr, 27 op/s 2023-09-10T00:15:07.223 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:06 smithi082 ceph-mon[98694]: pgmap v1946: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 64 KiB/s wr, 27 op/s 2023-09-10T00:15:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:06 smithi098 ceph-mon[103124]: pgmap v1946: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 64 KiB/s wr, 27 op/s 2023-09-10T00:15:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3338.468779 secs 2023-09-10T00:15:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:07 smithi170 ceph-mon[102832]: osdmap e193: 12 total, 12 up, 12 in 2023-09-10T00:15:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3338.468779 secs 2023-09-10T00:15:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:07 smithi082 ceph-mon[98694]: osdmap e193: 12 total, 12 up, 12 in 2023-09-10T00:15:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3338.468779 secs 2023-09-10T00:15:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:07 smithi098 ceph-mon[103124]: osdmap e193: 12 total, 12 up, 12 in 2023-09-10T00:15:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:08 smithi170 ceph-mon[102832]: pgmap v1948: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T00:15:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3340.078548 secs 2023-09-10T00:15:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:08 smithi082 ceph-mon[98694]: pgmap v1948: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T00:15:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3340.078548 secs 2023-09-10T00:15:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:08 smithi098 ceph-mon[103124]: pgmap v1948: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T00:15:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3340.078548 secs 2023-09-10T00:15:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3340.791360 secs 2023-09-10T00:15:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3340.791360 secs 2023-09-10T00:15:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3340.791360 secs 2023-09-10T00:15:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:10 smithi170 ceph-mon[102832]: pgmap v1949: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T00:15:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:10 smithi082 ceph-mon[98694]: pgmap v1949: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T00:15:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:10 smithi098 ceph-mon[103124]: pgmap v1949: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T00:15:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:12 smithi170 ceph-mon[102832]: pgmap v1950: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T00:15:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3343.468910 secs 2023-09-10T00:15:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:12 smithi082 ceph-mon[98694]: pgmap v1950: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T00:15:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3343.468910 secs 2023-09-10T00:15:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:12 smithi098 ceph-mon[103124]: pgmap v1950: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T00:15:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3343.468910 secs 2023-09-10T00:15:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3345.078607 secs 2023-09-10T00:15:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3345.078607 secs 2023-09-10T00:15:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3345.078607 secs 2023-09-10T00:15:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:14 smithi170 ceph-mon[102832]: pgmap v1951: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T00:15:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3345.791422 secs 2023-09-10T00:15:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:14 smithi170 ceph-mon[102832]: osdmap e194: 12 total, 12 up, 12 in 2023-09-10T00:15:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:14 smithi082 ceph-mon[98694]: pgmap v1951: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T00:15:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3345.791422 secs 2023-09-10T00:15:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:14 smithi082 ceph-mon[98694]: osdmap e194: 12 total, 12 up, 12 in 2023-09-10T00:15:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:14 smithi098 ceph-mon[103124]: pgmap v1951: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T00:15:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3345.791422 secs 2023-09-10T00:15:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:14 smithi098 ceph-mon[103124]: osdmap e194: 12 total, 12 up, 12 in 2023-09-10T00:15:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:16 smithi170 ceph-mon[102832]: pgmap v1953: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-09-10T00:15:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:16 smithi082 ceph-mon[98694]: pgmap v1953: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-09-10T00:15:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:16 smithi098 ceph-mon[103124]: pgmap v1953: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-09-10T00:15:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:18 smithi170 ceph-mon[102832]: pgmap v1954: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T00:15:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3350.078665 secs 2023-09-10T00:15:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:18 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:15:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:18 smithi082 ceph-mon[98694]: pgmap v1954: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T00:15:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3350.078665 secs 2023-09-10T00:15:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:18 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:15:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:18 smithi098 ceph-mon[103124]: pgmap v1954: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T00:15:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3350.078665 secs 2023-09-10T00:15:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:18 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:15:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3350.791482 secs 2023-09-10T00:15:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:19 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:15:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:19 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:15:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:19 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:15:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3350.791482 secs 2023-09-10T00:15:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:19 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:15:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:19 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:15:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:19 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:15:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3350.791482 secs 2023-09-10T00:15:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:19 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:15:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:19 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:15:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:19 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:15:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:20 smithi170 ceph-mon[102832]: pgmap v1955: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T00:15:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:20 smithi082 ceph-mon[98694]: pgmap v1955: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T00:15:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:20 smithi098 ceph-mon[103124]: pgmap v1955: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T00:15:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:22 smithi170 ceph-mon[102832]: pgmap v1956: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-10T00:15:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:22 smithi082 ceph-mon[98694]: pgmap v1956: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-10T00:15:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:22 smithi098 ceph-mon[103124]: pgmap v1956: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-10T00:15:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3355.078750 secs 2023-09-10T00:15:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3355.078750 secs 2023-09-10T00:15:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3355.078750 secs 2023-09-10T00:15:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:24 smithi170 ceph-mon[102832]: pgmap v1957: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-10T00:15:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3355.791543 secs 2023-09-10T00:15:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:24 smithi082 ceph-mon[98694]: pgmap v1957: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-10T00:15:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3355.791543 secs 2023-09-10T00:15:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:24 smithi098 ceph-mon[103124]: pgmap v1957: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-10T00:15:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3355.791543 secs 2023-09-10T00:15:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:26 smithi170 ceph-mon[102832]: pgmap v1958: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 79 KiB/s wr, 15 op/s 2023-09-10T00:15:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:26 smithi082 ceph-mon[98694]: pgmap v1958: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 79 KiB/s wr, 15 op/s 2023-09-10T00:15:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:26 smithi098 ceph-mon[103124]: pgmap v1958: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 79 KiB/s wr, 15 op/s 2023-09-10T00:15:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3358.469129 secs 2023-09-10T00:15:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3358.469129 secs 2023-09-10T00:15:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3358.469129 secs 2023-09-10T00:15:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:28 smithi170 ceph-mon[102832]: pgmap v1959: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 94 KiB/s wr, 13 op/s 2023-09-10T00:15:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3360.078824 secs 2023-09-10T00:15:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:28 smithi082 ceph-mon[98694]: pgmap v1959: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 94 KiB/s wr, 13 op/s 2023-09-10T00:15:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3360.078824 secs 2023-09-10T00:15:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:28 smithi098 ceph-mon[103124]: pgmap v1959: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 94 KiB/s wr, 13 op/s 2023-09-10T00:15:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3360.078824 secs 2023-09-10T00:15:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3360.791606 secs 2023-09-10T00:15:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3360.791606 secs 2023-09-10T00:15:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3360.791606 secs 2023-09-10T00:15:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:30 smithi170 ceph-mon[102832]: pgmap v1960: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T00:15:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:30 smithi082 ceph-mon[98694]: pgmap v1960: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T00:15:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:30 smithi098 ceph-mon[103124]: pgmap v1960: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T00:15:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:31 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:15:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:15:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:15:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:31 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:15:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:15:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:15:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:31 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:15:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:15:32.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:15:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:32 smithi170 ceph-mon[102832]: pgmap v1961: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 143 KiB/s wr, 17 op/s 2023-09-10T00:15:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3363.469262 secs 2023-09-10T00:15:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:32 smithi082 ceph-mon[98694]: pgmap v1961: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 143 KiB/s wr, 17 op/s 2023-09-10T00:15:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3363.469262 secs 2023-09-10T00:15:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:32 smithi098 ceph-mon[103124]: pgmap v1961: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 143 KiB/s wr, 17 op/s 2023-09-10T00:15:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3363.469262 secs 2023-09-10T00:15:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3365.078881 secs 2023-09-10T00:15:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3365.078881 secs 2023-09-10T00:15:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3365.078881 secs 2023-09-10T00:15:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:34 smithi170 ceph-mon[102832]: pgmap v1962: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T00:15:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3365.791669 secs 2023-09-10T00:15:35.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:15:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:34 smithi082 ceph-mon[98694]: pgmap v1962: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T00:15:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3365.791669 secs 2023-09-10T00:15:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:15:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:34 smithi098 ceph-mon[103124]: pgmap v1962: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T00:15:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3365.791669 secs 2023-09-10T00:15:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:15:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:36 smithi170 ceph-mon[102832]: pgmap v1963: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 140 KiB/s wr, 17 op/s 2023-09-10T00:15:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:36 smithi082 ceph-mon[98694]: pgmap v1963: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 140 KiB/s wr, 17 op/s 2023-09-10T00:15:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:36 smithi098 ceph-mon[103124]: pgmap v1963: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 140 KiB/s wr, 17 op/s 2023-09-10T00:15:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3368.469326 secs 2023-09-10T00:15:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3368.469326 secs 2023-09-10T00:15:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:38.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:38.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:38.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3368.469326 secs 2023-09-10T00:15:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:38 smithi082 ceph-mon[98694]: pgmap v1964: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 136 KiB/s wr, 13 op/s 2023-09-10T00:15:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3370.078947 secs 2023-09-10T00:15:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:39.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:39.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:39.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:39.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.212 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:15:39.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:15:39.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:38 smithi098 ceph-mon[103124]: pgmap v1964: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 136 KiB/s wr, 13 op/s 2023-09-10T00:15:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3370.078947 secs 2023-09-10T00:15:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:39.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:39.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:39.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:39.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:39.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.258 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:15:39.258 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:15:39.258 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:38 smithi170 ceph-mon[102832]: pgmap v1964: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 136 KiB/s wr, 13 op/s 2023-09-10T00:15:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3370.078947 secs 2023-09-10T00:15:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:39.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:39.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:39.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:39.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:39.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:39.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:15:39.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:15:39.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:15:39.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:15:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:39 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T00:15:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:39 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T00:15:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:39 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T00:15:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:39 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T00:15:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3370.791736 secs 2023-09-10T00:15:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:39 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T00:15:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:39 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T00:15:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:39 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T00:15:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:39 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T00:15:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:39 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T00:15:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:39 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T00:15:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3370.791736 secs 2023-09-10T00:15:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:39 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T00:15:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:39 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T00:15:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:39 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T00:15:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:39 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T00:15:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:39 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T00:15:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:39 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T00:15:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3370.791736 secs 2023-09-10T00:15:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:39 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T00:15:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:39 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T00:15:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:40 smithi082 ceph-mon[98694]: pgmap v1965: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T00:15:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:40 smithi098 ceph-mon[103124]: pgmap v1965: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T00:15:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:40 smithi170 ceph-mon[102832]: pgmap v1965: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T00:15:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:42 smithi082 ceph-mon[98694]: pgmap v1966: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 177 KiB/s wr, 16 op/s 2023-09-10T00:15:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:42 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:15:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:42 smithi098 ceph-mon[103124]: pgmap v1966: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 177 KiB/s wr, 16 op/s 2023-09-10T00:15:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:42 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:15:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:42 smithi170 ceph-mon[102832]: pgmap v1966: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 177 KiB/s wr, 16 op/s 2023-09-10T00:15:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:42 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:15:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3375.079028 secs 2023-09-10T00:15:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:43 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:15:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:15:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:15:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3375.079028 secs 2023-09-10T00:15:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:43 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:15:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:15:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:15:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3375.079028 secs 2023-09-10T00:15:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:43 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:15:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:15:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:15:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:44 smithi098 ceph-mon[103124]: pgmap v1967: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T00:15:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3375.791796 secs 2023-09-10T00:15:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:44 smithi170 ceph-mon[102832]: pgmap v1967: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T00:15:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3375.791796 secs 2023-09-10T00:15:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:44 smithi082 ceph-mon[98694]: pgmap v1967: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T00:15:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3375.791796 secs 2023-09-10T00:15:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:46 smithi098 ceph-mon[103124]: pgmap v1968: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 154 KiB/s wr, 17 op/s 2023-09-10T00:15:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:46 smithi170 ceph-mon[102832]: pgmap v1968: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 154 KiB/s wr, 17 op/s 2023-09-10T00:15:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:46 smithi082 ceph-mon[98694]: pgmap v1968: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 154 KiB/s wr, 17 op/s 2023-09-10T00:15:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3378.469444 secs 2023-09-10T00:15:48.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3378.469444 secs 2023-09-10T00:15:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3378.469444 secs 2023-09-10T00:15:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:48 smithi098 ceph-mon[103124]: pgmap v1969: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T00:15:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3380.079088 secs 2023-09-10T00:15:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:48 smithi170 ceph-mon[102832]: pgmap v1969: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T00:15:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3380.079088 secs 2023-09-10T00:15:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:48 smithi082 ceph-mon[98694]: pgmap v1969: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T00:15:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3380.079088 secs 2023-09-10T00:15:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3380.791857 secs 2023-09-10T00:15:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3380.791857 secs 2023-09-10T00:15:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3380.791857 secs 2023-09-10T00:15:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:50 smithi098 ceph-mon[103124]: pgmap v1970: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T00:15:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:50 smithi170 ceph-mon[102832]: pgmap v1970: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T00:15:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:50 smithi082 ceph-mon[98694]: pgmap v1970: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T00:15:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:15:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:15:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:15:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:15:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:15:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:15:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:15:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:15:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:15:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:52 smithi098 ceph-mon[103124]: pgmap v1971: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 127 KiB/s wr, 16 op/s 2023-09-10T00:15:53.300 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3383.469510 secs 2023-09-10T00:15:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:52 smithi170 ceph-mon[102832]: pgmap v1971: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 127 KiB/s wr, 16 op/s 2023-09-10T00:15:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3383.469510 secs 2023-09-10T00:15:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:52 smithi082 ceph-mon[98694]: pgmap v1971: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 127 KiB/s wr, 16 op/s 2023-09-10T00:15:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3383.469510 secs 2023-09-10T00:15:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3385.079149 secs 2023-09-10T00:15:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3385.079149 secs 2023-09-10T00:15:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3385.079149 secs 2023-09-10T00:15:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:54 smithi098 ceph-mon[103124]: pgmap v1972: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T00:15:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3385.791919 secs 2023-09-10T00:15:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:54 smithi170 ceph-mon[102832]: pgmap v1972: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T00:15:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3385.791919 secs 2023-09-10T00:15:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:54 smithi082 ceph-mon[98694]: pgmap v1972: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T00:15:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3385.791919 secs 2023-09-10T00:15:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:56 smithi098 ceph-mon[103124]: pgmap v1973: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 81 KiB/s wr, 15 op/s 2023-09-10T00:15:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:56 smithi170 ceph-mon[102832]: pgmap v1973: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 81 KiB/s wr, 15 op/s 2023-09-10T00:15:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:56 smithi082 ceph-mon[98694]: pgmap v1973: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 81 KiB/s wr, 15 op/s 2023-09-10T00:15:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3388.469570 secs 2023-09-10T00:15:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3388.469570 secs 2023-09-10T00:15:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3388.469570 secs 2023-09-10T00:15:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:59 smithi170 ceph-mon[102832]: pgmap v1974: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 36 KiB/s wr, 10 op/s 2023-09-10T00:15:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:15:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3390.079207 secs 2023-09-10T00:15:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:59 smithi082 ceph-mon[98694]: pgmap v1974: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 36 KiB/s wr, 10 op/s 2023-09-10T00:15:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:15:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3390.079207 secs 2023-09-10T00:15:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:59 smithi098 ceph-mon[103124]: pgmap v1974: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 36 KiB/s wr, 10 op/s 2023-09-10T00:15:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:15:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3390.079207 secs 2023-09-10T00:16:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3390.791979 secs 2023-09-10T00:16:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3390.791979 secs 2023-09-10T00:16:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3390.791979 secs 2023-09-10T00:16:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:01 smithi170 ceph-mon[102832]: pgmap v1975: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 36 KiB/s wr, 10 op/s 2023-09-10T00:16:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:01 smithi082 ceph-mon[98694]: pgmap v1975: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 36 KiB/s wr, 10 op/s 2023-09-10T00:16:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:01 smithi098 ceph-mon[103124]: pgmap v1975: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 36 KiB/s wr, 10 op/s 2023-09-10T00:16:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:03 smithi170 ceph-mon[102832]: pgmap v1976: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 37 KiB/s wr, 15 op/s 2023-09-10T00:16:03.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3393.469674 secs 2023-09-10T00:16:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:03 smithi082 ceph-mon[98694]: pgmap v1976: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 37 KiB/s wr, 15 op/s 2023-09-10T00:16:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3393.469674 secs 2023-09-10T00:16:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:03 smithi098 ceph-mon[103124]: pgmap v1976: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 37 KiB/s wr, 15 op/s 2023-09-10T00:16:03.557 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3393.469674 secs 2023-09-10T00:16:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3395.079290 secs 2023-09-10T00:16:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3395.079290 secs 2023-09-10T00:16:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3395.079290 secs 2023-09-10T00:16:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:05 smithi170 ceph-mon[102832]: pgmap v1977: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 10 op/s 2023-09-10T00:16:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3395.792044 secs 2023-09-10T00:16:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:05 smithi082 ceph-mon[98694]: pgmap v1977: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 10 op/s 2023-09-10T00:16:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3395.792044 secs 2023-09-10T00:16:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:05 smithi098 ceph-mon[103124]: pgmap v1977: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 10 op/s 2023-09-10T00:16:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3395.792044 secs 2023-09-10T00:16:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:07 smithi170 ceph-mon[102832]: pgmap v1978: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T00:16:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:07 smithi082 ceph-mon[98694]: pgmap v1978: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T00:16:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:07 smithi098 ceph-mon[103124]: pgmap v1978: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T00:16:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3398.469758 secs 2023-09-10T00:16:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:08 smithi170 ceph-mon[102832]: osdmap e195: 12 total, 12 up, 12 in 2023-09-10T00:16:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3398.469758 secs 2023-09-10T00:16:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:08 smithi082 ceph-mon[98694]: osdmap e195: 12 total, 12 up, 12 in 2023-09-10T00:16:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3398.469758 secs 2023-09-10T00:16:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:08 smithi098 ceph-mon[103124]: osdmap e195: 12 total, 12 up, 12 in 2023-09-10T00:16:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:09 smithi170 ceph-mon[102832]: pgmap v1980: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-10T00:16:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3400.079348 secs 2023-09-10T00:16:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:09 smithi082 ceph-mon[98694]: pgmap v1980: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-10T00:16:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3400.079348 secs 2023-09-10T00:16:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:09 smithi098 ceph-mon[103124]: pgmap v1980: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-10T00:16:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3400.079348 secs 2023-09-10T00:16:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3400.792108 secs 2023-09-10T00:16:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3400.792108 secs 2023-09-10T00:16:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3400.792108 secs 2023-09-10T00:16:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:11 smithi170 ceph-mon[102832]: pgmap v1981: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-10T00:16:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:11 smithi082 ceph-mon[98694]: pgmap v1981: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-10T00:16:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:11 smithi098 ceph-mon[103124]: pgmap v1981: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-10T00:16:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:13 smithi170 ceph-mon[102832]: pgmap v1982: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T00:16:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3403.469774 secs 2023-09-10T00:16:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:13 smithi082 ceph-mon[98694]: pgmap v1982: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T00:16:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3403.469774 secs 2023-09-10T00:16:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:13 smithi098 ceph-mon[103124]: pgmap v1982: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T00:16:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3403.469774 secs 2023-09-10T00:16:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3405.079405 secs 2023-09-10T00:16:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3405.079405 secs 2023-09-10T00:16:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3405.079405 secs 2023-09-10T00:16:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:15 smithi170 ceph-mon[102832]: pgmap v1983: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T00:16:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3405.792170 secs 2023-09-10T00:16:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:15 smithi170 ceph-mon[102832]: osdmap e196: 12 total, 12 up, 12 in 2023-09-10T00:16:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:15 smithi082 ceph-mon[98694]: pgmap v1983: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T00:16:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3405.792170 secs 2023-09-10T00:16:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:15 smithi082 ceph-mon[98694]: osdmap e196: 12 total, 12 up, 12 in 2023-09-10T00:16:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:15 smithi098 ceph-mon[103124]: pgmap v1983: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T00:16:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3405.792170 secs 2023-09-10T00:16:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:15 smithi098 ceph-mon[103124]: osdmap e196: 12 total, 12 up, 12 in 2023-09-10T00:16:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:17 smithi170 ceph-mon[102832]: pgmap v1985: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 54 KiB/s wr, 13 op/s 2023-09-10T00:16:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:17 smithi082 ceph-mon[98694]: pgmap v1985: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 54 KiB/s wr, 13 op/s 2023-09-10T00:16:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:17 smithi098 ceph-mon[103124]: pgmap v1985: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 54 KiB/s wr, 13 op/s 2023-09-10T00:16:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:19 smithi170 ceph-mon[102832]: pgmap v1986: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T00:16:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3410.079480 secs 2023-09-10T00:16:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:16:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:19 smithi082 ceph-mon[98694]: pgmap v1986: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T00:16:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3410.079480 secs 2023-09-10T00:16:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:16:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:19 smithi098 ceph-mon[103124]: pgmap v1986: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T00:16:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3410.079480 secs 2023-09-10T00:16:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:16:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3410.792249 secs 2023-09-10T00:16:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:16:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:16:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:16:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:20 smithi170 ceph-mon[102832]: pgmap v1987: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T00:16:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3410.792249 secs 2023-09-10T00:16:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:16:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:16:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:16:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:20 smithi082 ceph-mon[98694]: pgmap v1987: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T00:16:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3410.792249 secs 2023-09-10T00:16:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:16:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:16:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:16:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:20 smithi098 ceph-mon[103124]: pgmap v1987: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T00:16:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:22 smithi170 ceph-mon[102832]: pgmap v1988: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:16:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:22 smithi082 ceph-mon[98694]: pgmap v1988: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:16:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:22 smithi098 ceph-mon[103124]: pgmap v1988: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:16:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3415.079493 secs 2023-09-10T00:16:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3415.079493 secs 2023-09-10T00:16:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3415.079493 secs 2023-09-10T00:16:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:24 smithi170 ceph-mon[102832]: pgmap v1989: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:16:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3415.792312 secs 2023-09-10T00:16:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:24 smithi082 ceph-mon[98694]: pgmap v1989: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:16:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3415.792312 secs 2023-09-10T00:16:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:24 smithi098 ceph-mon[103124]: pgmap v1989: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:16:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3415.792312 secs 2023-09-10T00:16:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:26 smithi170 ceph-mon[102832]: pgmap v1990: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 124 KiB/s wr, 17 op/s 2023-09-10T00:16:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:26 smithi082 ceph-mon[98694]: pgmap v1990: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 124 KiB/s wr, 17 op/s 2023-09-10T00:16:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:26 smithi098 ceph-mon[103124]: pgmap v1990: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 124 KiB/s wr, 17 op/s 2023-09-10T00:16:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:28 smithi170 ceph-mon[102832]: pgmap v1991: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T00:16:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3420.079552 secs 2023-09-10T00:16:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:28 smithi082 ceph-mon[98694]: pgmap v1991: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T00:16:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3420.079552 secs 2023-09-10T00:16:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:28 smithi098 ceph-mon[103124]: pgmap v1991: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T00:16:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3420.079552 secs 2023-09-10T00:16:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3420.792392 secs 2023-09-10T00:16:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3420.792392 secs 2023-09-10T00:16:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3420.792392 secs 2023-09-10T00:16:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:30 smithi170 ceph-mon[102832]: pgmap v1992: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T00:16:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:30 smithi082 ceph-mon[98694]: pgmap v1992: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T00:16:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:30 smithi098 ceph-mon[103124]: pgmap v1992: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T00:16:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:32 smithi170 ceph-mon[102832]: pgmap v1993: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 168 KiB/s wr, 17 op/s 2023-09-10T00:16:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:32 smithi082 ceph-mon[98694]: pgmap v1993: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 168 KiB/s wr, 17 op/s 2023-09-10T00:16:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:32 smithi098 ceph-mon[103124]: pgmap v1993: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 168 KiB/s wr, 17 op/s 2023-09-10T00:16:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3425.079613 secs 2023-09-10T00:16:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3425.079613 secs 2023-09-10T00:16:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3425.079613 secs 2023-09-10T00:16:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:34 smithi170 ceph-mon[102832]: pgmap v1994: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 161 KiB/s wr, 12 op/s 2023-09-10T00:16:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3425.792457 secs 2023-09-10T00:16:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:34 smithi082 ceph-mon[98694]: pgmap v1994: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 161 KiB/s wr, 12 op/s 2023-09-10T00:16:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3425.792457 secs 2023-09-10T00:16:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:34 smithi098 ceph-mon[103124]: pgmap v1994: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 161 KiB/s wr, 12 op/s 2023-09-10T00:16:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3425.792457 secs 2023-09-10T00:16:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:36 smithi170 ceph-mon[102832]: pgmap v1995: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 184 KiB/s wr, 18 op/s 2023-09-10T00:16:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:36 smithi082 ceph-mon[98694]: pgmap v1995: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 184 KiB/s wr, 18 op/s 2023-09-10T00:16:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:36 smithi098 ceph-mon[103124]: pgmap v1995: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 184 KiB/s wr, 18 op/s 2023-09-10T00:16:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3428.470132 secs 2023-09-10T00:16:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3428.470132 secs 2023-09-10T00:16:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3428.470132 secs 2023-09-10T00:16:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:38 smithi170 ceph-mon[102832]: pgmap v1996: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T00:16:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3430.079670 secs 2023-09-10T00:16:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:38 smithi082 ceph-mon[98694]: pgmap v1996: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T00:16:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3430.079670 secs 2023-09-10T00:16:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:38 smithi098 ceph-mon[103124]: pgmap v1996: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T00:16:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3430.079670 secs 2023-09-10T00:16:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3430.792485 secs 2023-09-10T00:16:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:16:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3430.792485 secs 2023-09-10T00:16:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:16:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3430.792485 secs 2023-09-10T00:16:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:16:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:40 smithi170 ceph-mon[102832]: pgmap v1997: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T00:16:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:16:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:16:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:16:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:16:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:16:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:16:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:40 smithi082 ceph-mon[98694]: pgmap v1997: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T00:16:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:16:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:16:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:16:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:16:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:16:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:16:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:40 smithi098 ceph-mon[103124]: pgmap v1997: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T00:16:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:16:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:16:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:16:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:16:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:16:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:16:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:42 smithi170 ceph-mon[102832]: pgmap v1998: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 171 KiB/s wr, 18 op/s 2023-09-10T00:16:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3433.470193 secs 2023-09-10T00:16:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:42 smithi082 ceph-mon[98694]: pgmap v1998: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 171 KiB/s wr, 18 op/s 2023-09-10T00:16:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3433.470193 secs 2023-09-10T00:16:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:42 smithi098 ceph-mon[103124]: pgmap v1998: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 171 KiB/s wr, 18 op/s 2023-09-10T00:16:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3433.470193 secs 2023-09-10T00:16:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3435.079727 secs 2023-09-10T00:16:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3435.079727 secs 2023-09-10T00:16:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3435.079727 secs 2023-09-10T00:16:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:44 smithi170 ceph-mon[102832]: pgmap v1999: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T00:16:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3435.792549 secs 2023-09-10T00:16:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:44 smithi082 ceph-mon[98694]: pgmap v1999: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T00:16:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3435.792549 secs 2023-09-10T00:16:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:44 smithi098 ceph-mon[103124]: pgmap v1999: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T00:16:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3435.792549 secs 2023-09-10T00:16:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:46 smithi170 ceph-mon[102832]: pgmap v2000: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 133 KiB/s wr, 17 op/s 2023-09-10T00:16:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:46 smithi082 ceph-mon[98694]: pgmap v2000: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 133 KiB/s wr, 17 op/s 2023-09-10T00:16:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:46 smithi098 ceph-mon[103124]: pgmap v2000: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 133 KiB/s wr, 17 op/s 2023-09-10T00:16:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3438.470269 secs 2023-09-10T00:16:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3438.470269 secs 2023-09-10T00:16:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3438.470269 secs 2023-09-10T00:16:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:48 smithi170 ceph-mon[102832]: pgmap v2001: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T00:16:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3440.079814 secs 2023-09-10T00:16:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:48 smithi082 ceph-mon[98694]: pgmap v2001: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T00:16:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3440.079814 secs 2023-09-10T00:16:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:48 smithi098 ceph-mon[103124]: pgmap v2001: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T00:16:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3440.079814 secs 2023-09-10T00:16:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3440.792613 secs 2023-09-10T00:16:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3440.792613 secs 2023-09-10T00:16:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3440.792613 secs 2023-09-10T00:16:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:50 smithi170 ceph-mon[102832]: pgmap v2002: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T00:16:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:50 smithi082 ceph-mon[98694]: pgmap v2002: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T00:16:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:50 smithi098 ceph-mon[103124]: pgmap v2002: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T00:16:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:52 smithi170 ceph-mon[102832]: pgmap v2003: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 130 KiB/s wr, 16 op/s 2023-09-10T00:16:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3443.470331 secs 2023-09-10T00:16:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:52 smithi082 ceph-mon[98694]: pgmap v2003: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 130 KiB/s wr, 16 op/s 2023-09-10T00:16:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3443.470331 secs 2023-09-10T00:16:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:52 smithi098 ceph-mon[103124]: pgmap v2003: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 130 KiB/s wr, 16 op/s 2023-09-10T00:16:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3443.470331 secs 2023-09-10T00:16:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3445.079888 secs 2023-09-10T00:16:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3445.079888 secs 2023-09-10T00:16:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3445.079888 secs 2023-09-10T00:16:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:54 smithi082 ceph-mon[98694]: pgmap v2004: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T00:16:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3445.792678 secs 2023-09-10T00:16:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:54 smithi098 ceph-mon[103124]: pgmap v2004: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T00:16:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3445.792678 secs 2023-09-10T00:16:54.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:54 smithi170 ceph-mon[102832]: pgmap v2004: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T00:16:54.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3445.792678 secs 2023-09-10T00:16:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:56 smithi082 ceph-mon[98694]: pgmap v2005: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 69 KiB/s wr, 14 op/s 2023-09-10T00:16:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:56 smithi098 ceph-mon[103124]: pgmap v2005: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 69 KiB/s wr, 14 op/s 2023-09-10T00:16:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:56 smithi170 ceph-mon[102832]: pgmap v2005: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 69 KiB/s wr, 14 op/s 2023-09-10T00:16:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3448.470395 secs 2023-09-10T00:16:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3448.470395 secs 2023-09-10T00:16:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3448.470395 secs 2023-09-10T00:16:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:58 smithi082 ceph-mon[98694]: pgmap v2006: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 33 KiB/s wr, 9 op/s 2023-09-10T00:16:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3450.079957 secs 2023-09-10T00:16:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:58 smithi098 ceph-mon[103124]: pgmap v2006: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 33 KiB/s wr, 9 op/s 2023-09-10T00:16:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3450.079957 secs 2023-09-10T00:16:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:58 smithi170 ceph-mon[102832]: pgmap v2006: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 33 KiB/s wr, 9 op/s 2023-09-10T00:16:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3450.079957 secs 2023-09-10T00:16:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:16:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3450.792744 secs 2023-09-10T00:16:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:16:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3450.792744 secs 2023-09-10T00:16:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:16:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3450.792744 secs 2023-09-10T00:17:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:00 smithi082 ceph-mon[98694]: pgmap v2007: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 33 KiB/s wr, 9 op/s 2023-09-10T00:17:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:00 smithi098 ceph-mon[103124]: pgmap v2007: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 33 KiB/s wr, 9 op/s 2023-09-10T00:17:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:00 smithi170 ceph-mon[102832]: pgmap v2007: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 33 KiB/s wr, 9 op/s 2023-09-10T00:17:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:02 smithi082 ceph-mon[98694]: pgmap v2008: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 37 KiB/s wr, 14 op/s 2023-09-10T00:17:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3453.470462 secs 2023-09-10T00:17:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:02 smithi098 ceph-mon[103124]: pgmap v2008: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 37 KiB/s wr, 14 op/s 2023-09-10T00:17:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3453.470462 secs 2023-09-10T00:17:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:02 smithi170 ceph-mon[102832]: pgmap v2008: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 37 KiB/s wr, 14 op/s 2023-09-10T00:17:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3453.470462 secs 2023-09-10T00:17:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3455.080008 secs 2023-09-10T00:17:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3455.080008 secs 2023-09-10T00:17:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3455.080008 secs 2023-09-10T00:17:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:04 smithi082 ceph-mon[98694]: pgmap v2009: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-09-10T00:17:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3455.792810 secs 2023-09-10T00:17:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:04 smithi098 ceph-mon[103124]: pgmap v2009: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-09-10T00:17:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3455.792810 secs 2023-09-10T00:17:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:04 smithi170 ceph-mon[102832]: pgmap v2009: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-09-10T00:17:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3455.792810 secs 2023-09-10T00:17:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:06 smithi082 ceph-mon[98694]: pgmap v2010: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T00:17:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:06 smithi098 ceph-mon[103124]: pgmap v2010: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T00:17:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:06 smithi170 ceph-mon[102832]: pgmap v2010: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T00:17:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3458.470525 secs 2023-09-10T00:17:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3458.470525 secs 2023-09-10T00:17:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3458.470525 secs 2023-09-10T00:17:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:08 smithi098 ceph-mon[103124]: pgmap v2011: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 69 KiB/s wr, 24 op/s 2023-09-10T00:17:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:08 smithi098 ceph-mon[103124]: osdmap e197: 12 total, 12 up, 12 in 2023-09-10T00:17:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3460.080067 secs 2023-09-10T00:17:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:08 smithi170 ceph-mon[102832]: pgmap v2011: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 69 KiB/s wr, 24 op/s 2023-09-10T00:17:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:08 smithi170 ceph-mon[102832]: osdmap e197: 12 total, 12 up, 12 in 2023-09-10T00:17:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3460.080067 secs 2023-09-10T00:17:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:08 smithi082 ceph-mon[98694]: pgmap v2011: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 69 KiB/s wr, 24 op/s 2023-09-10T00:17:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:08 smithi082 ceph-mon[98694]: osdmap e197: 12 total, 12 up, 12 in 2023-09-10T00:17:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3460.080067 secs 2023-09-10T00:17:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3460.792872 secs 2023-09-10T00:17:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3460.792872 secs 2023-09-10T00:17:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3460.792872 secs 2023-09-10T00:17:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:10 smithi098 ceph-mon[103124]: pgmap v2013: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T00:17:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:10 smithi170 ceph-mon[102832]: pgmap v2013: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T00:17:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:10 smithi082 ceph-mon[98694]: pgmap v2013: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T00:17:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:12 smithi098 ceph-mon[103124]: pgmap v2014: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 79 KiB/s wr, 30 op/s 2023-09-10T00:17:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3463.470603 secs 2023-09-10T00:17:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:12 smithi170 ceph-mon[102832]: pgmap v2014: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 79 KiB/s wr, 30 op/s 2023-09-10T00:17:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3463.470603 secs 2023-09-10T00:17:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:12 smithi082 ceph-mon[98694]: pgmap v2014: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 79 KiB/s wr, 30 op/s 2023-09-10T00:17:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3463.470603 secs 2023-09-10T00:17:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3465.080129 secs 2023-09-10T00:17:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3465.080129 secs 2023-09-10T00:17:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3465.080129 secs 2023-09-10T00:17:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:14 smithi098 ceph-mon[103124]: pgmap v2015: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 79 KiB/s wr, 30 op/s 2023-09-10T00:17:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3465.792936 secs 2023-09-10T00:17:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:14 smithi098 ceph-mon[103124]: osdmap e198: 12 total, 12 up, 12 in 2023-09-10T00:17:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:14 smithi170 ceph-mon[102832]: pgmap v2015: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 79 KiB/s wr, 30 op/s 2023-09-10T00:17:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3465.792936 secs 2023-09-10T00:17:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:14 smithi170 ceph-mon[102832]: osdmap e198: 12 total, 12 up, 12 in 2023-09-10T00:17:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:14 smithi082 ceph-mon[98694]: pgmap v2015: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 79 KiB/s wr, 30 op/s 2023-09-10T00:17:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3465.792936 secs 2023-09-10T00:17:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:14 smithi082 ceph-mon[98694]: osdmap e198: 12 total, 12 up, 12 in 2023-09-10T00:17:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:16 smithi098 ceph-mon[103124]: pgmap v2017: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 14 op/s 2023-09-10T00:17:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:16 smithi170 ceph-mon[102832]: pgmap v2017: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 14 op/s 2023-09-10T00:17:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:16 smithi082 ceph-mon[98694]: pgmap v2017: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 14 op/s 2023-09-10T00:17:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:18 smithi098 ceph-mon[103124]: pgmap v2018: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:17:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3470.080187 secs 2023-09-10T00:17:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:18 smithi170 ceph-mon[102832]: pgmap v2018: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:17:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3470.080187 secs 2023-09-10T00:17:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:18 smithi082 ceph-mon[98694]: pgmap v2018: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:17:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3470.080187 secs 2023-09-10T00:17:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3470.793003 secs 2023-09-10T00:17:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:17:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3470.793003 secs 2023-09-10T00:17:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:17:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3470.793003 secs 2023-09-10T00:17:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:17:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:20 smithi098 ceph-mon[103124]: pgmap v2019: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T00:17:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:17:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:17:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:17:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:20 smithi170 ceph-mon[102832]: pgmap v2019: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T00:17:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:17:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:17:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:17:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:20 smithi082 ceph-mon[98694]: pgmap v2019: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T00:17:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:17:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:17:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:17:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:22 smithi098 ceph-mon[103124]: pgmap v2020: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T00:17:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:22 smithi170 ceph-mon[102832]: pgmap v2020: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T00:17:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:22 smithi082 ceph-mon[98694]: pgmap v2020: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T00:17:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3475.080245 secs 2023-09-10T00:17:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3475.080245 secs 2023-09-10T00:17:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3475.080245 secs 2023-09-10T00:17:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:24 smithi098 ceph-mon[103124]: pgmap v2021: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T00:17:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3475.793070 secs 2023-09-10T00:17:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:24 smithi170 ceph-mon[102832]: pgmap v2021: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T00:17:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3475.793070 secs 2023-09-10T00:17:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:24 smithi082 ceph-mon[98694]: pgmap v2021: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T00:17:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3475.793070 secs 2023-09-10T00:17:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:26 smithi098 ceph-mon[103124]: pgmap v2022: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 17 op/s 2023-09-10T00:17:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:26 smithi170 ceph-mon[102832]: pgmap v2022: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 17 op/s 2023-09-10T00:17:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:26 smithi082 ceph-mon[98694]: pgmap v2022: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 17 op/s 2023-09-10T00:17:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:28 smithi098 ceph-mon[103124]: pgmap v2023: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T00:17:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3480.080302 secs 2023-09-10T00:17:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:28 smithi170 ceph-mon[102832]: pgmap v2023: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T00:17:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3480.080302 secs 2023-09-10T00:17:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:28 smithi082 ceph-mon[98694]: pgmap v2023: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T00:17:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3480.080302 secs 2023-09-10T00:17:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3480.793135 secs 2023-09-10T00:17:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3480.793135 secs 2023-09-10T00:17:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3480.793135 secs 2023-09-10T00:17:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:30 smithi170 ceph-mon[102832]: pgmap v2024: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T00:17:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:30 smithi082 ceph-mon[98694]: pgmap v2024: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T00:17:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:30 smithi098 ceph-mon[103124]: pgmap v2024: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T00:17:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:32 smithi170 ceph-mon[102832]: pgmap v2025: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 118 KiB/s wr, 16 op/s 2023-09-10T00:17:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:32 smithi082 ceph-mon[98694]: pgmap v2025: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 118 KiB/s wr, 16 op/s 2023-09-10T00:17:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:32 smithi098 ceph-mon[103124]: pgmap v2025: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 118 KiB/s wr, 16 op/s 2023-09-10T00:17:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3485.080360 secs 2023-09-10T00:17:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3485.080360 secs 2023-09-10T00:17:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3485.080360 secs 2023-09-10T00:17:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:34 smithi170 ceph-mon[102832]: pgmap v2026: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T00:17:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3485.793203 secs 2023-09-10T00:17:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:34 smithi082 ceph-mon[98694]: pgmap v2026: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T00:17:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3485.793203 secs 2023-09-10T00:17:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:34 smithi098 ceph-mon[103124]: pgmap v2026: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T00:17:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3485.793203 secs 2023-09-10T00:17:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:36 smithi170 ceph-mon[102832]: pgmap v2027: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 141 KiB/s wr, 17 op/s 2023-09-10T00:17:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:36 smithi082 ceph-mon[98694]: pgmap v2027: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 141 KiB/s wr, 17 op/s 2023-09-10T00:17:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:36 smithi098 ceph-mon[103124]: pgmap v2027: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 141 KiB/s wr, 17 op/s 2023-09-10T00:17:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:38 smithi170 ceph-mon[102832]: pgmap v2028: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T00:17:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3490.080445 secs 2023-09-10T00:17:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:38 smithi082 ceph-mon[98694]: pgmap v2028: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T00:17:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3490.080445 secs 2023-09-10T00:17:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:38 smithi098 ceph-mon[103124]: pgmap v2028: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T00:17:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3490.080445 secs 2023-09-10T00:17:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3490.793270 secs 2023-09-10T00:17:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3490.793270 secs 2023-09-10T00:17:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3490.793270 secs 2023-09-10T00:17:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:40 smithi170 ceph-mon[102832]: pgmap v2029: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T00:17:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:17:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:40 smithi082 ceph-mon[98694]: pgmap v2029: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T00:17:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:17:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:40 smithi098 ceph-mon[103124]: pgmap v2029: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T00:17:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:17:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:17:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:17:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:17:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:17:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:17:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:17:42.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:17:42.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:17:42.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:17:42.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:17:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:17:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:17:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:17:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:17:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:17:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:42 smithi170 ceph-mon[102832]: pgmap v2030: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 161 KiB/s wr, 17 op/s 2023-09-10T00:17:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:42 smithi082 ceph-mon[98694]: pgmap v2030: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 161 KiB/s wr, 17 op/s 2023-09-10T00:17:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:42 smithi098 ceph-mon[103124]: pgmap v2030: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 161 KiB/s wr, 17 op/s 2023-09-10T00:17:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3495.080494 secs 2023-09-10T00:17:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3495.080494 secs 2023-09-10T00:17:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3495.080494 secs 2023-09-10T00:17:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:44 smithi170 ceph-mon[102832]: pgmap v2031: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T00:17:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3495.793336 secs 2023-09-10T00:17:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:44 smithi082 ceph-mon[98694]: pgmap v2031: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T00:17:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3495.793336 secs 2023-09-10T00:17:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:44 smithi098 ceph-mon[103124]: pgmap v2031: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T00:17:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3495.793336 secs 2023-09-10T00:17:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:46 smithi170 ceph-mon[102832]: pgmap v2032: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 122 KiB/s wr, 17 op/s 2023-09-10T00:17:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:46 smithi082 ceph-mon[98694]: pgmap v2032: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 122 KiB/s wr, 17 op/s 2023-09-10T00:17:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:46 smithi098 ceph-mon[103124]: pgmap v2032: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 122 KiB/s wr, 17 op/s 2023-09-10T00:17:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3498.471115 secs 2023-09-10T00:17:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3498.471115 secs 2023-09-10T00:17:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3498.471115 secs 2023-09-10T00:17:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:48 smithi170 ceph-mon[102832]: pgmap v2033: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T00:17:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3500.080554 secs 2023-09-10T00:17:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:48 smithi082 ceph-mon[98694]: pgmap v2033: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T00:17:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3500.080554 secs 2023-09-10T00:17:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:48 smithi098 ceph-mon[103124]: pgmap v2033: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T00:17:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3500.080554 secs 2023-09-10T00:17:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3500.793402 secs 2023-09-10T00:17:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3500.793402 secs 2023-09-10T00:17:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3500.793402 secs 2023-09-10T00:17:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:50 smithi170 ceph-mon[102832]: pgmap v2034: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T00:17:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:50 smithi082 ceph-mon[98694]: pgmap v2034: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T00:17:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:50 smithi098 ceph-mon[103124]: pgmap v2034: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T00:17:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:17:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:17:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:17:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:17:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:17:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:17:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:17:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:17:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:17:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:52 smithi170 ceph-mon[102832]: pgmap v2035: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 122 KiB/s wr, 16 op/s 2023-09-10T00:17:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3503.471180 secs 2023-09-10T00:17:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:52 smithi082 ceph-mon[98694]: pgmap v2035: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 122 KiB/s wr, 16 op/s 2023-09-10T00:17:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3503.471180 secs 2023-09-10T00:17:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:52 smithi098 ceph-mon[103124]: pgmap v2035: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 122 KiB/s wr, 16 op/s 2023-09-10T00:17:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3503.471180 secs 2023-09-10T00:17:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3505.080613 secs 2023-09-10T00:17:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3505.080613 secs 2023-09-10T00:17:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3505.080613 secs 2023-09-10T00:17:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:54 smithi170 ceph-mon[102832]: pgmap v2036: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T00:17:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3505.793466 secs 2023-09-10T00:17:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:54 smithi082 ceph-mon[98694]: pgmap v2036: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T00:17:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3505.793466 secs 2023-09-10T00:17:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:54 smithi098 ceph-mon[103124]: pgmap v2036: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T00:17:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3505.793466 secs 2023-09-10T00:17:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:56 smithi170 ceph-mon[102832]: pgmap v2037: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 75 KiB/s wr, 16 op/s 2023-09-10T00:17:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:56 smithi082 ceph-mon[98694]: pgmap v2037: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 75 KiB/s wr, 16 op/s 2023-09-10T00:17:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:56 smithi098 ceph-mon[103124]: pgmap v2037: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 75 KiB/s wr, 16 op/s 2023-09-10T00:17:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3508.471239 secs 2023-09-10T00:17:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3508.471239 secs 2023-09-10T00:17:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3508.471239 secs 2023-09-10T00:17:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:58 smithi170 ceph-mon[102832]: pgmap v2038: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T00:17:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3510.080687 secs 2023-09-10T00:17:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:58 smithi082 ceph-mon[98694]: pgmap v2038: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T00:17:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3510.080687 secs 2023-09-10T00:17:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:58 smithi098 ceph-mon[103124]: pgmap v2038: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T00:17:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3510.080687 secs 2023-09-10T00:18:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:17:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3510.793499 secs 2023-09-10T00:18:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:17:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3510.793499 secs 2023-09-10T00:18:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:17:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3510.793499 secs 2023-09-10T00:18:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:00 smithi170 ceph-mon[102832]: pgmap v2039: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T00:18:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:00 smithi082 ceph-mon[98694]: pgmap v2039: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T00:18:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:00 smithi098 ceph-mon[103124]: pgmap v2039: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T00:18:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:02 smithi170 ceph-mon[102832]: pgmap v2040: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 15 op/s 2023-09-10T00:18:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3513.471297 secs 2023-09-10T00:18:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:02 smithi082 ceph-mon[98694]: pgmap v2040: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 15 op/s 2023-09-10T00:18:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3513.471297 secs 2023-09-10T00:18:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:02 smithi098 ceph-mon[103124]: pgmap v2040: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 15 op/s 2023-09-10T00:18:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3513.471297 secs 2023-09-10T00:18:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3515.080743 secs 2023-09-10T00:18:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3515.080743 secs 2023-09-10T00:18:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3515.080743 secs 2023-09-10T00:18:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:04 smithi170 ceph-mon[102832]: pgmap v2041: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 20 KiB/s wr, 10 op/s 2023-09-10T00:18:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3515.793568 secs 2023-09-10T00:18:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:04 smithi082 ceph-mon[98694]: pgmap v2041: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 20 KiB/s wr, 10 op/s 2023-09-10T00:18:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3515.793568 secs 2023-09-10T00:18:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:04 smithi098 ceph-mon[103124]: pgmap v2041: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 20 KiB/s wr, 10 op/s 2023-09-10T00:18:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3515.793568 secs 2023-09-10T00:18:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:06 smithi170 ceph-mon[102832]: pgmap v2042: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 82 KiB/s wr, 30 op/s 2023-09-10T00:18:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:06 smithi082 ceph-mon[98694]: pgmap v2042: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 82 KiB/s wr, 30 op/s 2023-09-10T00:18:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:06 smithi098 ceph-mon[103124]: pgmap v2042: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 82 KiB/s wr, 30 op/s 2023-09-10T00:18:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3518.471403 secs 2023-09-10T00:18:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:07 smithi170 ceph-mon[102832]: osdmap e199: 12 total, 12 up, 12 in 2023-09-10T00:18:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3518.471403 secs 2023-09-10T00:18:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:07 smithi082 ceph-mon[98694]: osdmap e199: 12 total, 12 up, 12 in 2023-09-10T00:18:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3518.471403 secs 2023-09-10T00:18:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:07 smithi098 ceph-mon[103124]: osdmap e199: 12 total, 12 up, 12 in 2023-09-10T00:18:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:08 smithi170 ceph-mon[102832]: pgmap v2044: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 79 KiB/s wr, 30 op/s 2023-09-10T00:18:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3520.080801 secs 2023-09-10T00:18:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:08 smithi082 ceph-mon[98694]: pgmap v2044: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 79 KiB/s wr, 30 op/s 2023-09-10T00:18:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3520.080801 secs 2023-09-10T00:18:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:08 smithi098 ceph-mon[103124]: pgmap v2044: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 79 KiB/s wr, 30 op/s 2023-09-10T00:18:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3520.080801 secs 2023-09-10T00:18:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3520.793633 secs 2023-09-10T00:18:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3520.793633 secs 2023-09-10T00:18:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3520.793633 secs 2023-09-10T00:18:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:10 smithi170 ceph-mon[102832]: pgmap v2045: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 79 KiB/s wr, 30 op/s 2023-09-10T00:18:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:10 smithi082 ceph-mon[98694]: pgmap v2045: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 79 KiB/s wr, 30 op/s 2023-09-10T00:18:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:10 smithi098 ceph-mon[103124]: pgmap v2045: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 79 KiB/s wr, 30 op/s 2023-09-10T00:18:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:12 smithi170 ceph-mon[102832]: pgmap v2046: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 99 KiB/s wr, 30 op/s 2023-09-10T00:18:13.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3523.471464 secs 2023-09-10T00:18:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:12 smithi082 ceph-mon[98694]: pgmap v2046: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 99 KiB/s wr, 30 op/s 2023-09-10T00:18:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3523.471464 secs 2023-09-10T00:18:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:12 smithi098 ceph-mon[103124]: pgmap v2046: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 99 KiB/s wr, 30 op/s 2023-09-10T00:18:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3523.471464 secs 2023-09-10T00:18:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3525.080899 secs 2023-09-10T00:18:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3525.080899 secs 2023-09-10T00:18:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3525.080899 secs 2023-09-10T00:18:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:14 smithi170 ceph-mon[102832]: pgmap v2047: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 99 KiB/s wr, 30 op/s 2023-09-10T00:18:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3525.793725 secs 2023-09-10T00:18:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:14 smithi170 ceph-mon[102832]: osdmap e200: 12 total, 12 up, 12 in 2023-09-10T00:18:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:14 smithi082 ceph-mon[98694]: pgmap v2047: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 99 KiB/s wr, 30 op/s 2023-09-10T00:18:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3525.793725 secs 2023-09-10T00:18:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:14 smithi082 ceph-mon[98694]: osdmap e200: 12 total, 12 up, 12 in 2023-09-10T00:18:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:14 smithi098 ceph-mon[103124]: pgmap v2047: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 99 KiB/s wr, 30 op/s 2023-09-10T00:18:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3525.793725 secs 2023-09-10T00:18:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:14 smithi098 ceph-mon[103124]: osdmap e200: 12 total, 12 up, 12 in 2023-09-10T00:18:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:16 smithi170 ceph-mon[102832]: pgmap v2049: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 14 op/s 2023-09-10T00:18:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:16 smithi082 ceph-mon[98694]: pgmap v2049: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 14 op/s 2023-09-10T00:18:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:16 smithi098 ceph-mon[103124]: pgmap v2049: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 14 op/s 2023-09-10T00:18:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:18 smithi170 ceph-mon[102832]: pgmap v2050: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T00:18:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3530.080958 secs 2023-09-10T00:18:19.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:18 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:18:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:18 smithi082 ceph-mon[98694]: pgmap v2050: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T00:18:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3530.080958 secs 2023-09-10T00:18:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:18 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:18:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:18 smithi098 ceph-mon[103124]: pgmap v2050: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T00:18:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3530.080958 secs 2023-09-10T00:18:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:18 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:18:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3530.793791 secs 2023-09-10T00:18:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:19 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:18:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:19 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:18:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:19 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:18:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3530.793791 secs 2023-09-10T00:18:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:19 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:18:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:19 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:18:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:19 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:18:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3530.793791 secs 2023-09-10T00:18:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:19 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:18:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:19 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:18:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:19 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:18:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:20 smithi170 ceph-mon[102832]: pgmap v2051: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T00:18:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:20 smithi082 ceph-mon[98694]: pgmap v2051: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T00:18:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:20 smithi098 ceph-mon[103124]: pgmap v2051: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T00:18:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:22 smithi170 ceph-mon[102832]: pgmap v2052: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T00:18:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:22 smithi082 ceph-mon[98694]: pgmap v2052: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T00:18:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:22 smithi098 ceph-mon[103124]: pgmap v2052: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T00:18:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3535.081012 secs 2023-09-10T00:18:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3535.081012 secs 2023-09-10T00:18:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3535.081012 secs 2023-09-10T00:18:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:24 smithi170 ceph-mon[102832]: pgmap v2053: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T00:18:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3535.793854 secs 2023-09-10T00:18:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:24 smithi082 ceph-mon[98694]: pgmap v2053: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T00:18:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3535.793854 secs 2023-09-10T00:18:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:24 smithi098 ceph-mon[103124]: pgmap v2053: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T00:18:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3535.793854 secs 2023-09-10T00:18:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:26 smithi170 ceph-mon[102832]: pgmap v2054: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 145 KiB/s wr, 18 op/s 2023-09-10T00:18:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:26 smithi082 ceph-mon[98694]: pgmap v2054: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 145 KiB/s wr, 18 op/s 2023-09-10T00:18:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:26 smithi098 ceph-mon[103124]: pgmap v2054: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 145 KiB/s wr, 18 op/s 2023-09-10T00:18:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:28 smithi170 ceph-mon[102832]: pgmap v2055: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 132 KiB/s wr, 12 op/s 2023-09-10T00:18:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3540.081072 secs 2023-09-10T00:18:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:28 smithi082 ceph-mon[98694]: pgmap v2055: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 132 KiB/s wr, 12 op/s 2023-09-10T00:18:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3540.081072 secs 2023-09-10T00:18:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:28 smithi098 ceph-mon[103124]: pgmap v2055: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 132 KiB/s wr, 12 op/s 2023-09-10T00:18:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3540.081072 secs 2023-09-10T00:18:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3540.793922 secs 2023-09-10T00:18:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3540.793922 secs 2023-09-10T00:18:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3540.793922 secs 2023-09-10T00:18:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:30 smithi170 ceph-mon[102832]: pgmap v2056: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 132 KiB/s wr, 12 op/s 2023-09-10T00:18:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:30 smithi082 ceph-mon[98694]: pgmap v2056: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 132 KiB/s wr, 12 op/s 2023-09-10T00:18:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:30 smithi098 ceph-mon[103124]: pgmap v2056: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 132 KiB/s wr, 12 op/s 2023-09-10T00:18:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:32 smithi170 ceph-mon[102832]: pgmap v2057: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 191 KiB/s wr, 17 op/s 2023-09-10T00:18:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:32 smithi082 ceph-mon[98694]: pgmap v2057: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 191 KiB/s wr, 17 op/s 2023-09-10T00:18:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:32 smithi098 ceph-mon[103124]: pgmap v2057: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 191 KiB/s wr, 17 op/s 2023-09-10T00:18:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3545.081154 secs 2023-09-10T00:18:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3545.081154 secs 2023-09-10T00:18:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3545.081154 secs 2023-09-10T00:18:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:34 smithi170 ceph-mon[102832]: pgmap v2058: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T00:18:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3545.793987 secs 2023-09-10T00:18:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:34 smithi082 ceph-mon[98694]: pgmap v2058: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T00:18:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3545.793987 secs 2023-09-10T00:18:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:34 smithi098 ceph-mon[103124]: pgmap v2058: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T00:18:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3545.793987 secs 2023-09-10T00:18:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:36 smithi082 ceph-mon[98694]: pgmap v2059: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 161 KiB/s wr, 18 op/s 2023-09-10T00:18:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:36 smithi098 ceph-mon[103124]: pgmap v2059: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 161 KiB/s wr, 18 op/s 2023-09-10T00:18:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:36 smithi170 ceph-mon[102832]: pgmap v2059: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 161 KiB/s wr, 18 op/s 2023-09-10T00:18:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:38 smithi082 ceph-mon[98694]: pgmap v2060: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T00:18:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3550.081211 secs 2023-09-10T00:18:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:38 smithi098 ceph-mon[103124]: pgmap v2060: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T00:18:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3550.081211 secs 2023-09-10T00:18:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:38 smithi170 ceph-mon[102832]: pgmap v2060: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T00:18:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3550.081211 secs 2023-09-10T00:18:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3550.794054 secs 2023-09-10T00:18:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3550.794054 secs 2023-09-10T00:18:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3550.794054 secs 2023-09-10T00:18:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:40 smithi082 ceph-mon[98694]: pgmap v2061: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T00:18:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:40 smithi098 ceph-mon[103124]: pgmap v2061: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T00:18:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:40 smithi170 ceph-mon[102832]: pgmap v2061: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T00:18:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:18:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:18:42.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:18:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:18:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:18:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:18:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:18:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:18:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:18:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:42 smithi082 ceph-mon[98694]: pgmap v2062: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 158 KiB/s wr, 17 op/s 2023-09-10T00:18:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3553.471838 secs 2023-09-10T00:18:43.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:18:43.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:18:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:42 smithi098 ceph-mon[103124]: pgmap v2062: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 158 KiB/s wr, 17 op/s 2023-09-10T00:18:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3553.471838 secs 2023-09-10T00:18:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:18:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:18:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:42 smithi170 ceph-mon[102832]: pgmap v2062: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 158 KiB/s wr, 17 op/s 2023-09-10T00:18:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3553.471838 secs 2023-09-10T00:18:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:18:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:18:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3555.081267 secs 2023-09-10T00:18:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:18:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:18:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:18:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:18:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:18:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:18:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3555.081267 secs 2023-09-10T00:18:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:18:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:18:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:18:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:18:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:18:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:18:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3555.081267 secs 2023-09-10T00:18:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:18:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:18:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:18:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:18:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:18:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:18:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:44 smithi098 ceph-mon[103124]: pgmap v2063: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T00:18:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3555.794122 secs 2023-09-10T00:18:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:44 smithi170 ceph-mon[102832]: pgmap v2063: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T00:18:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3555.794122 secs 2023-09-10T00:18:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:44 smithi082 ceph-mon[98694]: pgmap v2063: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T00:18:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3555.794122 secs 2023-09-10T00:18:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:46 smithi098 ceph-mon[103124]: pgmap v2064: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T00:18:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:46 smithi170 ceph-mon[102832]: pgmap v2064: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T00:18:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:46 smithi082 ceph-mon[98694]: pgmap v2064: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T00:18:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3558.471895 secs 2023-09-10T00:18:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3558.471895 secs 2023-09-10T00:18:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3558.471895 secs 2023-09-10T00:18:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:48 smithi098 ceph-mon[103124]: pgmap v2065: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T00:18:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3560.081363 secs 2023-09-10T00:18:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:48 smithi170 ceph-mon[102832]: pgmap v2065: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T00:18:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3560.081363 secs 2023-09-10T00:18:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:48 smithi082 ceph-mon[98694]: pgmap v2065: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T00:18:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3560.081363 secs 2023-09-10T00:18:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3560.794205 secs 2023-09-10T00:18:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3560.794205 secs 2023-09-10T00:18:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3560.794205 secs 2023-09-10T00:18:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:50 smithi098 ceph-mon[103124]: pgmap v2066: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T00:18:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:50 smithi170 ceph-mon[102832]: pgmap v2066: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T00:18:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:50 smithi082 ceph-mon[98694]: pgmap v2066: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T00:18:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:52 smithi098 ceph-mon[103124]: pgmap v2067: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 94 KiB/s wr, 15 op/s 2023-09-10T00:18:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3563.471959 secs 2023-09-10T00:18:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:52 smithi170 ceph-mon[102832]: pgmap v2067: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 94 KiB/s wr, 15 op/s 2023-09-10T00:18:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3563.471959 secs 2023-09-10T00:18:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:52 smithi082 ceph-mon[98694]: pgmap v2067: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 94 KiB/s wr, 15 op/s 2023-09-10T00:18:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3563.471959 secs 2023-09-10T00:18:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3565.081435 secs 2023-09-10T00:18:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3565.081435 secs 2023-09-10T00:18:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3565.081435 secs 2023-09-10T00:18:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:54 smithi098 ceph-mon[103124]: pgmap v2068: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 68 KiB/s wr, 9 op/s 2023-09-10T00:18:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3565.794272 secs 2023-09-10T00:18:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:54 smithi170 ceph-mon[102832]: pgmap v2068: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 68 KiB/s wr, 9 op/s 2023-09-10T00:18:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3565.794272 secs 2023-09-10T00:18:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:54 smithi082 ceph-mon[98694]: pgmap v2068: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 68 KiB/s wr, 9 op/s 2023-09-10T00:18:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3565.794272 secs 2023-09-10T00:18:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:56 smithi098 ceph-mon[103124]: pgmap v2069: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 85 KiB/s wr, 15 op/s 2023-09-10T00:18:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:56 smithi170 ceph-mon[102832]: pgmap v2069: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 85 KiB/s wr, 15 op/s 2023-09-10T00:18:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:56 smithi082 ceph-mon[98694]: pgmap v2069: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 85 KiB/s wr, 15 op/s 2023-09-10T00:18:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3568.472032 secs 2023-09-10T00:18:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3568.472032 secs 2023-09-10T00:18:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3568.472032 secs 2023-09-10T00:18:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:58 smithi098 ceph-mon[103124]: pgmap v2070: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-09-10T00:18:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:18:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3570.081492 secs 2023-09-10T00:18:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:58 smithi170 ceph-mon[102832]: pgmap v2070: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-09-10T00:18:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:18:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3570.081492 secs 2023-09-10T00:18:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:58 smithi082 ceph-mon[98694]: pgmap v2070: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-09-10T00:18:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:18:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3570.081492 secs 2023-09-10T00:19:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3570.794340 secs 2023-09-10T00:19:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3570.794340 secs 2023-09-10T00:19:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3570.794340 secs 2023-09-10T00:19:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:01 smithi098 ceph-mon[103124]: pgmap v2071: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-09-10T00:19:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:01 smithi170 ceph-mon[102832]: pgmap v2071: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-09-10T00:19:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:01 smithi082 ceph-mon[98694]: pgmap v2071: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-09-10T00:19:03.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:03 smithi170 ceph-mon[102832]: pgmap v2072: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 57 KiB/s wr, 15 op/s 2023-09-10T00:19:03.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3573.472092 secs 2023-09-10T00:19:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:03 smithi082 ceph-mon[98694]: pgmap v2072: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 57 KiB/s wr, 15 op/s 2023-09-10T00:19:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3573.472092 secs 2023-09-10T00:19:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:03 smithi098 ceph-mon[103124]: pgmap v2072: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 57 KiB/s wr, 15 op/s 2023-09-10T00:19:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3573.472092 secs 2023-09-10T00:19:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3575.081561 secs 2023-09-10T00:19:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3575.081561 secs 2023-09-10T00:19:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3575.081561 secs 2023-09-10T00:19:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:05 smithi170 ceph-mon[102832]: pgmap v2073: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T00:19:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3575.794404 secs 2023-09-10T00:19:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:05 smithi082 ceph-mon[98694]: pgmap v2073: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T00:19:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3575.794404 secs 2023-09-10T00:19:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:05 smithi098 ceph-mon[103124]: pgmap v2073: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T00:19:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3575.794404 secs 2023-09-10T00:19:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:07 smithi170 ceph-mon[102832]: pgmap v2074: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 89 KiB/s wr, 30 op/s 2023-09-10T00:19:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:07 smithi082 ceph-mon[98694]: pgmap v2074: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 89 KiB/s wr, 30 op/s 2023-09-10T00:19:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:07 smithi098 ceph-mon[103124]: pgmap v2074: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 89 KiB/s wr, 30 op/s 2023-09-10T00:19:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3578.472165 secs 2023-09-10T00:19:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:08 smithi170 ceph-mon[102832]: osdmap e201: 12 total, 12 up, 12 in 2023-09-10T00:19:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3578.472165 secs 2023-09-10T00:19:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:08 smithi082 ceph-mon[98694]: osdmap e201: 12 total, 12 up, 12 in 2023-09-10T00:19:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3578.472165 secs 2023-09-10T00:19:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:08 smithi098 ceph-mon[103124]: osdmap e201: 12 total, 12 up, 12 in 2023-09-10T00:19:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:09 smithi170 ceph-mon[102832]: pgmap v2076: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T00:19:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3580.081646 secs 2023-09-10T00:19:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:09 smithi082 ceph-mon[98694]: pgmap v2076: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T00:19:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3580.081646 secs 2023-09-10T00:19:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:09 smithi098 ceph-mon[103124]: pgmap v2076: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T00:19:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3580.081646 secs 2023-09-10T00:19:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3580.794479 secs 2023-09-10T00:19:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3580.794479 secs 2023-09-10T00:19:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3580.794479 secs 2023-09-10T00:19:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:11 smithi170 ceph-mon[102832]: pgmap v2077: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T00:19:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:11 smithi082 ceph-mon[98694]: pgmap v2077: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T00:19:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:11 smithi098 ceph-mon[103124]: pgmap v2077: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T00:19:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:13 smithi170 ceph-mon[102832]: pgmap v2078: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 92 KiB/s wr, 29 op/s 2023-09-10T00:19:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3583.472225 secs 2023-09-10T00:19:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:13 smithi082 ceph-mon[98694]: pgmap v2078: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 92 KiB/s wr, 29 op/s 2023-09-10T00:19:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3583.472225 secs 2023-09-10T00:19:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:13 smithi098 ceph-mon[103124]: pgmap v2078: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 92 KiB/s wr, 29 op/s 2023-09-10T00:19:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3583.472225 secs 2023-09-10T00:19:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3585.081704 secs 2023-09-10T00:19:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3585.081704 secs 2023-09-10T00:19:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3585.081704 secs 2023-09-10T00:19:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:15 smithi170 ceph-mon[102832]: pgmap v2079: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 92 KiB/s wr, 29 op/s 2023-09-10T00:19:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3585.794546 secs 2023-09-10T00:19:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:15 smithi170 ceph-mon[102832]: osdmap e202: 12 total, 12 up, 12 in 2023-09-10T00:19:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:15 smithi082 ceph-mon[98694]: pgmap v2079: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 92 KiB/s wr, 29 op/s 2023-09-10T00:19:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3585.794546 secs 2023-09-10T00:19:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:15 smithi082 ceph-mon[98694]: osdmap e202: 12 total, 12 up, 12 in 2023-09-10T00:19:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:15 smithi098 ceph-mon[103124]: pgmap v2079: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 92 KiB/s wr, 29 op/s 2023-09-10T00:19:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3585.794546 secs 2023-09-10T00:19:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:15 smithi098 ceph-mon[103124]: osdmap e202: 12 total, 12 up, 12 in 2023-09-10T00:19:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:17 smithi170 ceph-mon[102832]: pgmap v2081: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 13 op/s 2023-09-10T00:19:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:17 smithi082 ceph-mon[98694]: pgmap v2081: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 13 op/s 2023-09-10T00:19:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:17 smithi098 ceph-mon[103124]: pgmap v2081: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 13 op/s 2023-09-10T00:19:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:19 smithi170 ceph-mon[102832]: pgmap v2082: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:19:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3590.081762 secs 2023-09-10T00:19:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:19:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:19 smithi082 ceph-mon[98694]: pgmap v2082: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:19:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3590.081762 secs 2023-09-10T00:19:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:19:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:19 smithi098 ceph-mon[103124]: pgmap v2082: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:19:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3590.081762 secs 2023-09-10T00:19:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:19:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3590.794611 secs 2023-09-10T00:19:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:19:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:19:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:19:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3590.794611 secs 2023-09-10T00:19:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:19:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:19:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:19:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3590.794611 secs 2023-09-10T00:19:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:19:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:19:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:19:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:21 smithi170 ceph-mon[102832]: pgmap v2083: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:19:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:21 smithi082 ceph-mon[98694]: pgmap v2083: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:19:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:21 smithi098 ceph-mon[103124]: pgmap v2083: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:19:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:23 smithi170 ceph-mon[102832]: pgmap v2084: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T00:19:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:23 smithi082 ceph-mon[98694]: pgmap v2084: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T00:19:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:23 smithi098 ceph-mon[103124]: pgmap v2084: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T00:19:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3595.081820 secs 2023-09-10T00:19:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3595.081820 secs 2023-09-10T00:19:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3595.081820 secs 2023-09-10T00:19:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:25 smithi170 ceph-mon[102832]: pgmap v2085: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T00:19:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3595.794679 secs 2023-09-10T00:19:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:25 smithi082 ceph-mon[98694]: pgmap v2085: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T00:19:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3595.794679 secs 2023-09-10T00:19:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:25 smithi098 ceph-mon[103124]: pgmap v2085: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T00:19:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3595.794679 secs 2023-09-10T00:19:27.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:27 smithi170 ceph-mon[102832]: pgmap v2086: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 132 KiB/s wr, 17 op/s 2023-09-10T00:19:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:27 smithi082 ceph-mon[98694]: pgmap v2086: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 132 KiB/s wr, 17 op/s 2023-09-10T00:19:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:27 smithi098 ceph-mon[103124]: pgmap v2086: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 132 KiB/s wr, 17 op/s 2023-09-10T00:19:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:29 smithi170 ceph-mon[102832]: pgmap v2087: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T00:19:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3600.081878 secs 2023-09-10T00:19:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:29 smithi082 ceph-mon[98694]: pgmap v2087: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T00:19:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3600.081878 secs 2023-09-10T00:19:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:29 smithi098 ceph-mon[103124]: pgmap v2087: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T00:19:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3600.081878 secs 2023-09-10T00:19:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3600.794743 secs 2023-09-10T00:19:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3600.794743 secs 2023-09-10T00:19:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3600.794743 secs 2023-09-10T00:19:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:31 smithi170 ceph-mon[102832]: pgmap v2088: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T00:19:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:31 smithi082 ceph-mon[98694]: pgmap v2088: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T00:19:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:31 smithi098 ceph-mon[103124]: pgmap v2088: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T00:19:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:33 smithi170 ceph-mon[102832]: pgmap v2089: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 182 KiB/s wr, 33 op/s 2023-09-10T00:19:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:33 smithi082 ceph-mon[98694]: pgmap v2089: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 182 KiB/s wr, 33 op/s 2023-09-10T00:19:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:33 smithi098 ceph-mon[103124]: pgmap v2089: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 182 KiB/s wr, 33 op/s 2023-09-10T00:19:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3605.081958 secs 2023-09-10T00:19:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3605.081958 secs 2023-09-10T00:19:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3605.081958 secs 2023-09-10T00:19:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:35 smithi170 ceph-mon[102832]: pgmap v2090: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 153 KiB/s wr, 28 op/s 2023-09-10T00:19:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3605.794810 secs 2023-09-10T00:19:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:35 smithi082 ceph-mon[98694]: pgmap v2090: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 153 KiB/s wr, 28 op/s 2023-09-10T00:19:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3605.794810 secs 2023-09-10T00:19:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:35 smithi098 ceph-mon[103124]: pgmap v2090: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 153 KiB/s wr, 28 op/s 2023-09-10T00:19:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3605.794810 secs 2023-09-10T00:19:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:36 smithi170 ceph-mon[102832]: pgmap v2091: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 177 KiB/s wr, 33 op/s 2023-09-10T00:19:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:36 smithi082 ceph-mon[98694]: pgmap v2091: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 177 KiB/s wr, 33 op/s 2023-09-10T00:19:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:36 smithi098 ceph-mon[103124]: pgmap v2091: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 177 KiB/s wr, 33 op/s 2023-09-10T00:19:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:38 smithi170 ceph-mon[102832]: pgmap v2092: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 86 KiB/s wr, 27 op/s 2023-09-10T00:19:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3610.082015 secs 2023-09-10T00:19:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:38 smithi082 ceph-mon[98694]: pgmap v2092: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 86 KiB/s wr, 27 op/s 2023-09-10T00:19:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3610.082015 secs 2023-09-10T00:19:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:38 smithi098 ceph-mon[103124]: pgmap v2092: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 86 KiB/s wr, 27 op/s 2023-09-10T00:19:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3610.082015 secs 2023-09-10T00:19:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3610.794881 secs 2023-09-10T00:19:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3610.794881 secs 2023-09-10T00:19:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3610.794881 secs 2023-09-10T00:19:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:40 smithi170 ceph-mon[102832]: pgmap v2093: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 86 KiB/s wr, 27 op/s 2023-09-10T00:19:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:40 smithi082 ceph-mon[98694]: pgmap v2093: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 86 KiB/s wr, 27 op/s 2023-09-10T00:19:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:40 smithi098 ceph-mon[103124]: pgmap v2093: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 86 KiB/s wr, 27 op/s 2023-09-10T00:19:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:42 smithi170 ceph-mon[102832]: pgmap v2094: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 107 KiB/s wr, 33 op/s 2023-09-10T00:19:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:42 smithi082 ceph-mon[98694]: pgmap v2094: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 107 KiB/s wr, 33 op/s 2023-09-10T00:19:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:42 smithi098 ceph-mon[103124]: pgmap v2094: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 107 KiB/s wr, 33 op/s 2023-09-10T00:19:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3615.082102 secs 2023-09-10T00:19:43.738 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3615.082102 secs 2023-09-10T00:19:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3615.082102 secs 2023-09-10T00:19:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:44 smithi170 ceph-mon[102832]: pgmap v2095: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T00:19:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:19:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3615.794946 secs 2023-09-10T00:19:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:19:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:19:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:19:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:44 smithi082 ceph-mon[98694]: pgmap v2095: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T00:19:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:19:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3615.794946 secs 2023-09-10T00:19:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:19:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:19:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:19:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:44 smithi098 ceph-mon[103124]: pgmap v2095: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T00:19:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:19:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3615.794946 secs 2023-09-10T00:19:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:19:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:19:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:19:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:46 smithi170 ceph-mon[102832]: pgmap v2096: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 106 KiB/s wr, 17 op/s 2023-09-10T00:19:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:46 smithi082 ceph-mon[98694]: pgmap v2096: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 106 KiB/s wr, 17 op/s 2023-09-10T00:19:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:46 smithi098 ceph-mon[103124]: pgmap v2096: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 106 KiB/s wr, 17 op/s 2023-09-10T00:19:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3618.472773 secs 2023-09-10T00:19:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3618.472773 secs 2023-09-10T00:19:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3618.472773 secs 2023-09-10T00:19:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:48 smithi082 ceph-mon[98694]: pgmap v2097: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T00:19:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3620.082162 secs 2023-09-10T00:19:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:48 smithi098 ceph-mon[103124]: pgmap v2097: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T00:19:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3620.082162 secs 2023-09-10T00:19:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:48 smithi170 ceph-mon[102832]: pgmap v2097: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T00:19:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3620.082162 secs 2023-09-10T00:19:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3620.795011 secs 2023-09-10T00:19:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3620.795011 secs 2023-09-10T00:19:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3620.795011 secs 2023-09-10T00:19:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:50 smithi082 ceph-mon[98694]: pgmap v2098: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T00:19:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:50 smithi098 ceph-mon[103124]: pgmap v2098: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T00:19:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:50 smithi170 ceph-mon[102832]: pgmap v2098: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T00:19:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:19:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:19:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:19:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:19:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:19:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:19:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:19:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:19:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:19:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:52 smithi098 ceph-mon[103124]: pgmap v2099: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 105 KiB/s wr, 17 op/s 2023-09-10T00:19:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3623.472883 secs 2023-09-10T00:19:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:52 smithi170 ceph-mon[102832]: pgmap v2099: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 105 KiB/s wr, 17 op/s 2023-09-10T00:19:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3623.472883 secs 2023-09-10T00:19:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:52 smithi082 ceph-mon[98694]: pgmap v2099: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 105 KiB/s wr, 17 op/s 2023-09-10T00:19:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3623.472883 secs 2023-09-10T00:19:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3625.082221 secs 2023-09-10T00:19:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3625.082221 secs 2023-09-10T00:19:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3625.082221 secs 2023-09-10T00:19:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:54 smithi170 ceph-mon[102832]: pgmap v2100: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T00:19:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3625.795078 secs 2023-09-10T00:19:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:54 smithi082 ceph-mon[98694]: pgmap v2100: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T00:19:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3625.795078 secs 2023-09-10T00:19:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:54 smithi098 ceph-mon[103124]: pgmap v2100: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T00:19:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3625.795078 secs 2023-09-10T00:19:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:56 smithi170 ceph-mon[102832]: pgmap v2101: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 16 op/s 2023-09-10T00:19:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:56 smithi082 ceph-mon[98694]: pgmap v2101: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 16 op/s 2023-09-10T00:19:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:56 smithi098 ceph-mon[103124]: pgmap v2101: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 16 op/s 2023-09-10T00:19:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3628.472979 secs 2023-09-10T00:19:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3628.472979 secs 2023-09-10T00:19:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3628.472979 secs 2023-09-10T00:19:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:58 smithi170 ceph-mon[102832]: pgmap v2102: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-09-10T00:19:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3630.082299 secs 2023-09-10T00:19:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:58 smithi082 ceph-mon[98694]: pgmap v2102: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-09-10T00:19:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3630.082299 secs 2023-09-10T00:19:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:58 smithi098 ceph-mon[103124]: pgmap v2102: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-09-10T00:19:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3630.082299 secs 2023-09-10T00:19:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:19:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3630.795139 secs 2023-09-10T00:19:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:19:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3630.795139 secs 2023-09-10T00:20:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:19:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3630.795139 secs 2023-09-10T00:20:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:00 smithi170 ceph-mon[102832]: pgmap v2103: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-09-10T00:20:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T00:20:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:00 smithi082 ceph-mon[98694]: pgmap v2103: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-09-10T00:20:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T00:20:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:00 smithi098 ceph-mon[103124]: pgmap v2103: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-09-10T00:20:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T00:20:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:02 smithi170 ceph-mon[102832]: pgmap v2104: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 38 KiB/s wr, 14 op/s 2023-09-10T00:20:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3633.473037 secs 2023-09-10T00:20:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:02 smithi082 ceph-mon[98694]: pgmap v2104: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 38 KiB/s wr, 14 op/s 2023-09-10T00:20:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3633.473037 secs 2023-09-10T00:20:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:02 smithi098 ceph-mon[103124]: pgmap v2104: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 38 KiB/s wr, 14 op/s 2023-09-10T00:20:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3633.473037 secs 2023-09-10T00:20:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3635.082380 secs 2023-09-10T00:20:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3635.082380 secs 2023-09-10T00:20:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3635.082380 secs 2023-09-10T00:20:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:04 smithi170 ceph-mon[102832]: pgmap v2105: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-10T00:20:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3635.795204 secs 2023-09-10T00:20:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:04 smithi082 ceph-mon[98694]: pgmap v2105: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-10T00:20:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3635.795204 secs 2023-09-10T00:20:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:04 smithi098 ceph-mon[103124]: pgmap v2105: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-10T00:20:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3635.795204 secs 2023-09-10T00:20:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:06 smithi170 ceph-mon[102832]: pgmap v2106: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T00:20:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:06 smithi082 ceph-mon[98694]: pgmap v2106: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T00:20:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:06 smithi098 ceph-mon[103124]: pgmap v2106: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T00:20:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3638.473107 secs 2023-09-10T00:20:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3638.473107 secs 2023-09-10T00:20:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3638.473107 secs 2023-09-10T00:20:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:08 smithi170 ceph-mon[102832]: pgmap v2107: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 66 KiB/s wr, 24 op/s 2023-09-10T00:20:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:08 smithi170 ceph-mon[102832]: osdmap e203: 12 total, 12 up, 12 in 2023-09-10T00:20:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3640.082438 secs 2023-09-10T00:20:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:08 smithi082 ceph-mon[98694]: pgmap v2107: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 66 KiB/s wr, 24 op/s 2023-09-10T00:20:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:08 smithi082 ceph-mon[98694]: osdmap e203: 12 total, 12 up, 12 in 2023-09-10T00:20:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3640.082438 secs 2023-09-10T00:20:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:08 smithi098 ceph-mon[103124]: pgmap v2107: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 66 KiB/s wr, 24 op/s 2023-09-10T00:20:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:08 smithi098 ceph-mon[103124]: osdmap e203: 12 total, 12 up, 12 in 2023-09-10T00:20:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3640.082438 secs 2023-09-10T00:20:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3640.795272 secs 2023-09-10T00:20:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3640.795272 secs 2023-09-10T00:20:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3640.795272 secs 2023-09-10T00:20:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:10 smithi170 ceph-mon[102832]: pgmap v2109: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 80 KiB/s wr, 28 op/s 2023-09-10T00:20:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:10 smithi082 ceph-mon[98694]: pgmap v2109: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 80 KiB/s wr, 28 op/s 2023-09-10T00:20:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:10 smithi098 ceph-mon[103124]: pgmap v2109: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 80 KiB/s wr, 28 op/s 2023-09-10T00:20:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:12 smithi170 ceph-mon[102832]: pgmap v2110: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 80 KiB/s wr, 29 op/s 2023-09-10T00:20:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3643.473194 secs 2023-09-10T00:20:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:12 smithi082 ceph-mon[98694]: pgmap v2110: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 80 KiB/s wr, 29 op/s 2023-09-10T00:20:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3643.473194 secs 2023-09-10T00:20:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:12 smithi098 ceph-mon[103124]: pgmap v2110: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 80 KiB/s wr, 29 op/s 2023-09-10T00:20:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3643.473194 secs 2023-09-10T00:20:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3645.082522 secs 2023-09-10T00:20:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3645.082522 secs 2023-09-10T00:20:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3645.082522 secs 2023-09-10T00:20:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:14 smithi170 ceph-mon[102832]: pgmap v2111: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 80 KiB/s wr, 29 op/s 2023-09-10T00:20:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3645.795339 secs 2023-09-10T00:20:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:14 smithi170 ceph-mon[102832]: osdmap e204: 12 total, 12 up, 12 in 2023-09-10T00:20:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:14 smithi082 ceph-mon[98694]: pgmap v2111: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 80 KiB/s wr, 29 op/s 2023-09-10T00:20:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3645.795339 secs 2023-09-10T00:20:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:14 smithi082 ceph-mon[98694]: osdmap e204: 12 total, 12 up, 12 in 2023-09-10T00:20:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:14 smithi098 ceph-mon[103124]: pgmap v2111: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 80 KiB/s wr, 29 op/s 2023-09-10T00:20:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3645.795339 secs 2023-09-10T00:20:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:14 smithi098 ceph-mon[103124]: osdmap e204: 12 total, 12 up, 12 in 2023-09-10T00:20:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:16 smithi170 ceph-mon[102832]: pgmap v2113: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 14 op/s 2023-09-10T00:20:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:16 smithi082 ceph-mon[98694]: pgmap v2113: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 14 op/s 2023-09-10T00:20:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:16 smithi098 ceph-mon[103124]: pgmap v2113: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 14 op/s 2023-09-10T00:20:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:18 smithi170 ceph-mon[102832]: pgmap v2114: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T00:20:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3650.082584 secs 2023-09-10T00:20:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:18 smithi082 ceph-mon[98694]: pgmap v2114: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T00:20:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3650.082584 secs 2023-09-10T00:20:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:18 smithi098 ceph-mon[103124]: pgmap v2114: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T00:20:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3650.082584 secs 2023-09-10T00:20:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3650.795403 secs 2023-09-10T00:20:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:20:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3650.795403 secs 2023-09-10T00:20:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:20:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3650.795403 secs 2023-09-10T00:20:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:20:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:20 smithi170 ceph-mon[102832]: pgmap v2115: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:20:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:20:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:20:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:20:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:20 smithi082 ceph-mon[98694]: pgmap v2115: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:20:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:20:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:20:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:20:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:20 smithi098 ceph-mon[103124]: pgmap v2115: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:20:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:20:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:20:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:20:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:22 smithi170 ceph-mon[102832]: pgmap v2116: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T00:20:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:22 smithi082 ceph-mon[98694]: pgmap v2116: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T00:20:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:22 smithi098 ceph-mon[103124]: pgmap v2116: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T00:20:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3655.082659 secs 2023-09-10T00:20:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3655.082659 secs 2023-09-10T00:20:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3655.082659 secs 2023-09-10T00:20:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:24 smithi170 ceph-mon[102832]: pgmap v2117: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T00:20:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3655.795495 secs 2023-09-10T00:20:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:24 smithi082 ceph-mon[98694]: pgmap v2117: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T00:20:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3655.795495 secs 2023-09-10T00:20:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:24 smithi098 ceph-mon[103124]: pgmap v2117: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T00:20:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3655.795495 secs 2023-09-10T00:20:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:26 smithi170 ceph-mon[102832]: pgmap v2118: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 83 KiB/s wr, 16 op/s 2023-09-10T00:20:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:26 smithi082 ceph-mon[98694]: pgmap v2118: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 83 KiB/s wr, 16 op/s 2023-09-10T00:20:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:26 smithi098 ceph-mon[103124]: pgmap v2118: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 83 KiB/s wr, 16 op/s 2023-09-10T00:20:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:28 smithi170 ceph-mon[102832]: pgmap v2119: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:20:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3660.082717 secs 2023-09-10T00:20:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:28 smithi082 ceph-mon[98694]: pgmap v2119: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:20:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3660.082717 secs 2023-09-10T00:20:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:28 smithi098 ceph-mon[103124]: pgmap v2119: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:20:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3660.082717 secs 2023-09-10T00:20:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3660.795561 secs 2023-09-10T00:20:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3660.795561 secs 2023-09-10T00:20:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3660.795561 secs 2023-09-10T00:20:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:30 smithi170 ceph-mon[102832]: pgmap v2120: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:20:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:30 smithi082 ceph-mon[98694]: pgmap v2120: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:20:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:30 smithi098 ceph-mon[103124]: pgmap v2120: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:20:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:32 smithi170 ceph-mon[102832]: pgmap v2121: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 121 KiB/s wr, 16 op/s 2023-09-10T00:20:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:32 smithi082 ceph-mon[98694]: pgmap v2121: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 121 KiB/s wr, 16 op/s 2023-09-10T00:20:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:32 smithi098 ceph-mon[103124]: pgmap v2121: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 121 KiB/s wr, 16 op/s 2023-09-10T00:20:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3665.082775 secs 2023-09-10T00:20:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3665.082775 secs 2023-09-10T00:20:34.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3665.082775 secs 2023-09-10T00:20:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:34 smithi170 ceph-mon[102832]: pgmap v2122: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T00:20:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3665.795625 secs 2023-09-10T00:20:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:34 smithi082 ceph-mon[98694]: pgmap v2122: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T00:20:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3665.795625 secs 2023-09-10T00:20:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:34 smithi098 ceph-mon[103124]: pgmap v2122: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T00:20:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3665.795625 secs 2023-09-10T00:20:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:36 smithi170 ceph-mon[102832]: pgmap v2123: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 145 KiB/s wr, 18 op/s 2023-09-10T00:20:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:36 smithi082 ceph-mon[98694]: pgmap v2123: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 145 KiB/s wr, 18 op/s 2023-09-10T00:20:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:36 smithi098 ceph-mon[103124]: pgmap v2123: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 145 KiB/s wr, 18 op/s 2023-09-10T00:20:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:38 smithi170 ceph-mon[102832]: pgmap v2124: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T00:20:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3670.082832 secs 2023-09-10T00:20:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:38 smithi082 ceph-mon[98694]: pgmap v2124: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T00:20:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3670.082832 secs 2023-09-10T00:20:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:38 smithi098 ceph-mon[103124]: pgmap v2124: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T00:20:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3670.082832 secs 2023-09-10T00:20:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3670.795692 secs 2023-09-10T00:20:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3670.795692 secs 2023-09-10T00:20:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3670.795692 secs 2023-09-10T00:20:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:40 smithi170 ceph-mon[102832]: pgmap v2125: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T00:20:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:40 smithi082 ceph-mon[98694]: pgmap v2125: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T00:20:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:40 smithi098 ceph-mon[103124]: pgmap v2125: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T00:20:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:42 smithi170 ceph-mon[102832]: pgmap v2126: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 167 KiB/s wr, 18 op/s 2023-09-10T00:20:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:42 smithi082 ceph-mon[98694]: pgmap v2126: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 167 KiB/s wr, 18 op/s 2023-09-10T00:20:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:42 smithi098 ceph-mon[103124]: pgmap v2126: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 167 KiB/s wr, 18 op/s 2023-09-10T00:20:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3675.082911 secs 2023-09-10T00:20:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3675.082911 secs 2023-09-10T00:20:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3675.082911 secs 2023-09-10T00:20:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:44 smithi170 ceph-mon[102832]: pgmap v2127: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T00:20:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3675.795755 secs 2023-09-10T00:20:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:20:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:44 smithi082 ceph-mon[98694]: pgmap v2127: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T00:20:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3675.795755 secs 2023-09-10T00:20:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:20:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:44 smithi098 ceph-mon[103124]: pgmap v2127: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T00:20:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3675.795755 secs 2023-09-10T00:20:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:20:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:20:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:20:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:20:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:20:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:20:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:20:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:20:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:20:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:20:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:46 smithi082 ceph-mon[98694]: pgmap v2128: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 145 KiB/s wr, 17 op/s 2023-09-10T00:20:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:46 smithi098 ceph-mon[103124]: pgmap v2128: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 145 KiB/s wr, 17 op/s 2023-09-10T00:20:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:46 smithi170 ceph-mon[102832]: pgmap v2128: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 145 KiB/s wr, 17 op/s 2023-09-10T00:20:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3678.473833 secs 2023-09-10T00:20:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3678.473833 secs 2023-09-10T00:20:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3678.473833 secs 2023-09-10T00:20:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:48 smithi098 ceph-mon[103124]: pgmap v2129: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T00:20:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3680.082998 secs 2023-09-10T00:20:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:48 smithi170 ceph-mon[102832]: pgmap v2129: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T00:20:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3680.082998 secs 2023-09-10T00:20:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:48 smithi082 ceph-mon[98694]: pgmap v2129: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T00:20:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3680.082998 secs 2023-09-10T00:20:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3680.795822 secs 2023-09-10T00:20:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3680.795822 secs 2023-09-10T00:20:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3680.795822 secs 2023-09-10T00:20:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:50 smithi098 ceph-mon[103124]: pgmap v2130: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T00:20:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:50 smithi170 ceph-mon[102832]: pgmap v2130: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T00:20:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:50 smithi082 ceph-mon[98694]: pgmap v2130: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T00:20:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:20:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:20:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:20:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:20:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:20:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:20:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:20:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:20:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:20:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:52 smithi098 ceph-mon[103124]: pgmap v2131: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 144 KiB/s wr, 16 op/s 2023-09-10T00:20:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3683.473893 secs 2023-09-10T00:20:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:52 smithi170 ceph-mon[102832]: pgmap v2131: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 144 KiB/s wr, 16 op/s 2023-09-10T00:20:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3683.473893 secs 2023-09-10T00:20:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:52 smithi082 ceph-mon[98694]: pgmap v2131: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 144 KiB/s wr, 16 op/s 2023-09-10T00:20:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3683.473893 secs 2023-09-10T00:20:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3685.083057 secs 2023-09-10T00:20:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3685.083057 secs 2023-09-10T00:20:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3685.083057 secs 2023-09-10T00:20:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:54 smithi098 ceph-mon[103124]: pgmap v2132: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 78 KiB/s wr, 10 op/s 2023-09-10T00:20:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3685.795885 secs 2023-09-10T00:20:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:54 smithi170 ceph-mon[102832]: pgmap v2132: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 78 KiB/s wr, 10 op/s 2023-09-10T00:20:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3685.795885 secs 2023-09-10T00:20:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:54 smithi082 ceph-mon[98694]: pgmap v2132: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 78 KiB/s wr, 10 op/s 2023-09-10T00:20:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3685.795885 secs 2023-09-10T00:20:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:56 smithi098 ceph-mon[103124]: pgmap v2133: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 96 KiB/s wr, 15 op/s 2023-09-10T00:20:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:56 smithi170 ceph-mon[102832]: pgmap v2133: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 96 KiB/s wr, 15 op/s 2023-09-10T00:20:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:56 smithi082 ceph-mon[98694]: pgmap v2133: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 96 KiB/s wr, 15 op/s 2023-09-10T00:20:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3688.473951 secs 2023-09-10T00:20:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3688.473951 secs 2023-09-10T00:20:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3688.473951 secs 2023-09-10T00:20:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:59 smithi098 ceph-mon[103124]: pgmap v2134: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T00:20:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:20:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3690.083145 secs 2023-09-10T00:20:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:59 smithi170 ceph-mon[102832]: pgmap v2134: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T00:20:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:20:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3690.083145 secs 2023-09-10T00:20:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:59 smithi082 ceph-mon[98694]: pgmap v2134: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T00:20:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:20:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3690.083145 secs 2023-09-10T00:21:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3690.795970 secs 2023-09-10T00:21:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3690.795970 secs 2023-09-10T00:21:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3690.795970 secs 2023-09-10T00:21:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:01 smithi170 ceph-mon[102832]: pgmap v2135: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T00:21:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:01 smithi082 ceph-mon[98694]: pgmap v2135: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T00:21:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:01 smithi098 ceph-mon[103124]: pgmap v2135: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T00:21:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:03 smithi170 ceph-mon[102832]: pgmap v2136: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 51 KiB/s wr, 14 op/s 2023-09-10T00:21:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3693.474014 secs 2023-09-10T00:21:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:03 smithi082 ceph-mon[98694]: pgmap v2136: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 51 KiB/s wr, 14 op/s 2023-09-10T00:21:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3693.474014 secs 2023-09-10T00:21:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:03 smithi098 ceph-mon[103124]: pgmap v2136: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 51 KiB/s wr, 14 op/s 2023-09-10T00:21:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3693.474014 secs 2023-09-10T00:21:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3695.083202 secs 2023-09-10T00:21:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3695.083202 secs 2023-09-10T00:21:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3695.083202 secs 2023-09-10T00:21:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:05 smithi170 ceph-mon[102832]: pgmap v2137: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 21 KiB/s wr, 9 op/s 2023-09-10T00:21:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3695.796037 secs 2023-09-10T00:21:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:05 smithi082 ceph-mon[98694]: pgmap v2137: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 21 KiB/s wr, 9 op/s 2023-09-10T00:21:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3695.796037 secs 2023-09-10T00:21:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:05 smithi098 ceph-mon[103124]: pgmap v2137: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 21 KiB/s wr, 9 op/s 2023-09-10T00:21:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3695.796037 secs 2023-09-10T00:21:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:07 smithi170 ceph-mon[102832]: pgmap v2138: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T00:21:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:07 smithi082 ceph-mon[98694]: pgmap v2138: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T00:21:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:07 smithi098 ceph-mon[103124]: pgmap v2138: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T00:21:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3698.474076 secs 2023-09-10T00:21:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:08 smithi170 ceph-mon[102832]: osdmap e205: 12 total, 12 up, 12 in 2023-09-10T00:21:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3698.474076 secs 2023-09-10T00:21:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:08 smithi082 ceph-mon[98694]: osdmap e205: 12 total, 12 up, 12 in 2023-09-10T00:21:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3698.474076 secs 2023-09-10T00:21:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:08 smithi098 ceph-mon[103124]: osdmap e205: 12 total, 12 up, 12 in 2023-09-10T00:21:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:09 smithi170 ceph-mon[102832]: pgmap v2140: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T00:21:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3700.083285 secs 2023-09-10T00:21:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:09 smithi082 ceph-mon[98694]: pgmap v2140: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T00:21:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3700.083285 secs 2023-09-10T00:21:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:09 smithi098 ceph-mon[103124]: pgmap v2140: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T00:21:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3700.083285 secs 2023-09-10T00:21:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3700.796103 secs 2023-09-10T00:21:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3700.796103 secs 2023-09-10T00:21:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3700.796103 secs 2023-09-10T00:21:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:11 smithi170 ceph-mon[102832]: pgmap v2141: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T00:21:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:11 smithi082 ceph-mon[98694]: pgmap v2141: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T00:21:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:11 smithi098 ceph-mon[103124]: pgmap v2141: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T00:21:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:13 smithi170 ceph-mon[102832]: pgmap v2142: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T00:21:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3703.474136 secs 2023-09-10T00:21:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:13 smithi082 ceph-mon[98694]: pgmap v2142: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T00:21:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3703.474136 secs 2023-09-10T00:21:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:13 smithi098 ceph-mon[103124]: pgmap v2142: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T00:21:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3703.474136 secs 2023-09-10T00:21:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3705.083343 secs 2023-09-10T00:21:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3705.083343 secs 2023-09-10T00:21:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3705.083343 secs 2023-09-10T00:21:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:15 smithi170 ceph-mon[102832]: pgmap v2143: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T00:21:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3705.796173 secs 2023-09-10T00:21:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:15 smithi170 ceph-mon[102832]: osdmap e206: 12 total, 12 up, 12 in 2023-09-10T00:21:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:15 smithi082 ceph-mon[98694]: pgmap v2143: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T00:21:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3705.796173 secs 2023-09-10T00:21:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:15 smithi082 ceph-mon[98694]: osdmap e206: 12 total, 12 up, 12 in 2023-09-10T00:21:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:15 smithi098 ceph-mon[103124]: pgmap v2143: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T00:21:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3705.796173 secs 2023-09-10T00:21:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:15 smithi098 ceph-mon[103124]: osdmap e206: 12 total, 12 up, 12 in 2023-09-10T00:21:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:17 smithi170 ceph-mon[102832]: pgmap v2145: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 29 KiB/s wr, 13 op/s 2023-09-10T00:21:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:17 smithi082 ceph-mon[98694]: pgmap v2145: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 29 KiB/s wr, 13 op/s 2023-09-10T00:21:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:17 smithi098 ceph-mon[103124]: pgmap v2145: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 29 KiB/s wr, 13 op/s 2023-09-10T00:21:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:19 smithi170 ceph-mon[102832]: pgmap v2146: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-10T00:21:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3710.083431 secs 2023-09-10T00:21:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:21:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:19 smithi082 ceph-mon[98694]: pgmap v2146: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-10T00:21:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3710.083431 secs 2023-09-10T00:21:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:21:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:19 smithi098 ceph-mon[103124]: pgmap v2146: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-10T00:21:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3710.083431 secs 2023-09-10T00:21:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:21:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3710.796238 secs 2023-09-10T00:21:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:21:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:21:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:21:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3710.796238 secs 2023-09-10T00:21:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:21:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:21:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:21:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3710.796238 secs 2023-09-10T00:21:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:21:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:21:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:21:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:21 smithi170 ceph-mon[102832]: pgmap v2147: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-10T00:21:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:21 smithi082 ceph-mon[98694]: pgmap v2147: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-10T00:21:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:21 smithi098 ceph-mon[103124]: pgmap v2147: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-10T00:21:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:23 smithi170 ceph-mon[102832]: pgmap v2148: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 89 KiB/s wr, 12 op/s 2023-09-10T00:21:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:23 smithi082 ceph-mon[98694]: pgmap v2148: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 89 KiB/s wr, 12 op/s 2023-09-10T00:21:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:23 smithi098 ceph-mon[103124]: pgmap v2148: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 89 KiB/s wr, 12 op/s 2023-09-10T00:21:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3715.083516 secs 2023-09-10T00:21:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3715.083516 secs 2023-09-10T00:21:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3715.083516 secs 2023-09-10T00:21:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:25 smithi170 ceph-mon[102832]: pgmap v2149: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 89 KiB/s wr, 12 op/s 2023-09-10T00:21:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3715.796303 secs 2023-09-10T00:21:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:25 smithi082 ceph-mon[98694]: pgmap v2149: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 89 KiB/s wr, 12 op/s 2023-09-10T00:21:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3715.796303 secs 2023-09-10T00:21:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:25 smithi098 ceph-mon[103124]: pgmap v2149: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 89 KiB/s wr, 12 op/s 2023-09-10T00:21:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3715.796303 secs 2023-09-10T00:21:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:27 smithi170 ceph-mon[102832]: pgmap v2150: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 111 KiB/s wr, 18 op/s 2023-09-10T00:21:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:27 smithi082 ceph-mon[98694]: pgmap v2150: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 111 KiB/s wr, 18 op/s 2023-09-10T00:21:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:27 smithi098 ceph-mon[103124]: pgmap v2150: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 111 KiB/s wr, 18 op/s 2023-09-10T00:21:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:29 smithi170 ceph-mon[102832]: pgmap v2151: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T00:21:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3720.083602 secs 2023-09-10T00:21:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:29 smithi082 ceph-mon[98694]: pgmap v2151: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T00:21:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3720.083602 secs 2023-09-10T00:21:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:29 smithi098 ceph-mon[103124]: pgmap v2151: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T00:21:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3720.083602 secs 2023-09-10T00:21:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3720.796367 secs 2023-09-10T00:21:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3720.796367 secs 2023-09-10T00:21:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3720.796367 secs 2023-09-10T00:21:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:31 smithi170 ceph-mon[102832]: pgmap v2152: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T00:21:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:31 smithi082 ceph-mon[98694]: pgmap v2152: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T00:21:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:31 smithi098 ceph-mon[103124]: pgmap v2152: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T00:21:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:33 smithi170 ceph-mon[102832]: pgmap v2153: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 182 KiB/s wr, 18 op/s 2023-09-10T00:21:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:33 smithi082 ceph-mon[98694]: pgmap v2153: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 182 KiB/s wr, 18 op/s 2023-09-10T00:21:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:33 smithi098 ceph-mon[103124]: pgmap v2153: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 182 KiB/s wr, 18 op/s 2023-09-10T00:21:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3725.083660 secs 2023-09-10T00:21:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3725.083660 secs 2023-09-10T00:21:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3725.083660 secs 2023-09-10T00:21:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:35 smithi170 ceph-mon[102832]: pgmap v2154: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 12 op/s 2023-09-10T00:21:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3725.796431 secs 2023-09-10T00:21:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:35 smithi082 ceph-mon[98694]: pgmap v2154: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 12 op/s 2023-09-10T00:21:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3725.796431 secs 2023-09-10T00:21:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:35 smithi098 ceph-mon[103124]: pgmap v2154: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 12 op/s 2023-09-10T00:21:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3725.796431 secs 2023-09-10T00:21:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:37 smithi170 ceph-mon[102832]: pgmap v2155: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 152 KiB/s wr, 18 op/s 2023-09-10T00:21:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:37 smithi082 ceph-mon[98694]: pgmap v2155: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 152 KiB/s wr, 18 op/s 2023-09-10T00:21:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:37 smithi098 ceph-mon[103124]: pgmap v2155: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 152 KiB/s wr, 18 op/s 2023-09-10T00:21:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:39 smithi170 ceph-mon[102832]: pgmap v2156: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T00:21:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3730.083718 secs 2023-09-10T00:21:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:39 smithi082 ceph-mon[98694]: pgmap v2156: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T00:21:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3730.083718 secs 2023-09-10T00:21:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:39 smithi098 ceph-mon[103124]: pgmap v2156: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T00:21:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3730.083718 secs 2023-09-10T00:21:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3730.796512 secs 2023-09-10T00:21:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3730.796512 secs 2023-09-10T00:21:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3730.796512 secs 2023-09-10T00:21:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:41 smithi170 ceph-mon[102832]: pgmap v2157: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T00:21:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:41 smithi082 ceph-mon[98694]: pgmap v2157: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T00:21:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:41 smithi098 ceph-mon[103124]: pgmap v2157: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T00:21:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:42 smithi170 ceph-mon[102832]: pgmap v2158: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 17 op/s 2023-09-10T00:21:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:42 smithi082 ceph-mon[98694]: pgmap v2158: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 17 op/s 2023-09-10T00:21:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:42 smithi098 ceph-mon[103124]: pgmap v2158: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 17 op/s 2023-09-10T00:21:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3735.083775 secs 2023-09-10T00:21:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3735.083775 secs 2023-09-10T00:21:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3735.083775 secs 2023-09-10T00:21:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:44 smithi170 ceph-mon[102832]: pgmap v2159: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T00:21:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3735.796577 secs 2023-09-10T00:21:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:44 smithi082 ceph-mon[98694]: pgmap v2159: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T00:21:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3735.796577 secs 2023-09-10T00:21:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:44 smithi098 ceph-mon[103124]: pgmap v2159: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T00:21:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3735.796577 secs 2023-09-10T00:21:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:21:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:21:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:21:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:46 smithi082 ceph-mon[98694]: pgmap v2160: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 106 KiB/s wr, 17 op/s 2023-09-10T00:21:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:21:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:21:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:21:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:46 smithi098 ceph-mon[103124]: pgmap v2160: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 106 KiB/s wr, 17 op/s 2023-09-10T00:21:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:21:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:21:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:21:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:46 smithi170 ceph-mon[102832]: pgmap v2160: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 106 KiB/s wr, 17 op/s 2023-09-10T00:21:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:21:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:21:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:21:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3738.474699 secs 2023-09-10T00:21:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3738.474699 secs 2023-09-10T00:21:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3738.474699 secs 2023-09-10T00:21:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:48 smithi082 ceph-mon[98694]: pgmap v2161: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 82 KiB/s wr, 10 op/s 2023-09-10T00:21:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3740.083833 secs 2023-09-10T00:21:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:48 smithi098 ceph-mon[103124]: pgmap v2161: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 82 KiB/s wr, 10 op/s 2023-09-10T00:21:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3740.083833 secs 2023-09-10T00:21:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:48 smithi170 ceph-mon[102832]: pgmap v2161: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 82 KiB/s wr, 10 op/s 2023-09-10T00:21:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3740.083833 secs 2023-09-10T00:21:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3740.796640 secs 2023-09-10T00:21:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3740.796640 secs 2023-09-10T00:21:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3740.796640 secs 2023-09-10T00:21:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:50 smithi082 ceph-mon[98694]: pgmap v2162: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 82 KiB/s wr, 10 op/s 2023-09-10T00:21:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:50 smithi098 ceph-mon[103124]: pgmap v2162: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 82 KiB/s wr, 10 op/s 2023-09-10T00:21:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:50 smithi170 ceph-mon[102832]: pgmap v2162: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 82 KiB/s wr, 10 op/s 2023-09-10T00:21:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:21:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:21:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:21:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:21:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:21:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:21:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:21:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:21:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:21:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:52 smithi170 ceph-mon[102832]: pgmap v2163: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 102 KiB/s wr, 16 op/s 2023-09-10T00:21:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3743.474765 secs 2023-09-10T00:21:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:52 smithi082 ceph-mon[98694]: pgmap v2163: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 102 KiB/s wr, 16 op/s 2023-09-10T00:21:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3743.474765 secs 2023-09-10T00:21:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:52 smithi098 ceph-mon[103124]: pgmap v2163: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 102 KiB/s wr, 16 op/s 2023-09-10T00:21:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3743.474765 secs 2023-09-10T00:21:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3745.083891 secs 2023-09-10T00:21:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3745.083891 secs 2023-09-10T00:21:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3745.083891 secs 2023-09-10T00:21:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:54 smithi170 ceph-mon[102832]: pgmap v2164: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 69 KiB/s wr, 10 op/s 2023-09-10T00:21:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3745.796708 secs 2023-09-10T00:21:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:54 smithi082 ceph-mon[98694]: pgmap v2164: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 69 KiB/s wr, 10 op/s 2023-09-10T00:21:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3745.796708 secs 2023-09-10T00:21:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:54 smithi098 ceph-mon[103124]: pgmap v2164: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 69 KiB/s wr, 10 op/s 2023-09-10T00:21:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3745.796708 secs 2023-09-10T00:21:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:56 smithi170 ceph-mon[102832]: pgmap v2165: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 87 KiB/s wr, 15 op/s 2023-09-10T00:21:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:56 smithi082 ceph-mon[98694]: pgmap v2165: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 87 KiB/s wr, 15 op/s 2023-09-10T00:21:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:56 smithi098 ceph-mon[103124]: pgmap v2165: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 87 KiB/s wr, 15 op/s 2023-09-10T00:21:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3748.474820 secs 2023-09-10T00:21:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3748.474820 secs 2023-09-10T00:21:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3748.474820 secs 2023-09-10T00:21:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:58 smithi170 ceph-mon[102832]: pgmap v2166: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T00:21:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3750.083950 secs 2023-09-10T00:21:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:58 smithi082 ceph-mon[98694]: pgmap v2166: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T00:21:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3750.083950 secs 2023-09-10T00:21:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:58 smithi098 ceph-mon[103124]: pgmap v2166: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T00:21:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3750.083950 secs 2023-09-10T00:21:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:21:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3750.796773 secs 2023-09-10T00:21:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:21:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3750.796773 secs 2023-09-10T00:22:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:21:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3750.796773 secs 2023-09-10T00:22:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:00 smithi170 ceph-mon[102832]: pgmap v2167: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T00:22:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:00 smithi082 ceph-mon[98694]: pgmap v2167: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T00:22:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:00 smithi098 ceph-mon[103124]: pgmap v2167: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T00:22:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:02 smithi170 ceph-mon[102832]: pgmap v2168: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T00:22:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3753.474914 secs 2023-09-10T00:22:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:02 smithi082 ceph-mon[98694]: pgmap v2168: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T00:22:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3753.474914 secs 2023-09-10T00:22:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:02 smithi098 ceph-mon[103124]: pgmap v2168: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T00:22:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3753.474914 secs 2023-09-10T00:22:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3755.084034 secs 2023-09-10T00:22:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3755.084034 secs 2023-09-10T00:22:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3755.084034 secs 2023-09-10T00:22:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:04 smithi170 ceph-mon[102832]: pgmap v2169: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 63 KiB/s wr, 24 op/s 2023-09-10T00:22:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3755.796840 secs 2023-09-10T00:22:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:04 smithi082 ceph-mon[98694]: pgmap v2169: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 63 KiB/s wr, 24 op/s 2023-09-10T00:22:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3755.796840 secs 2023-09-10T00:22:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:04 smithi098 ceph-mon[103124]: pgmap v2169: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 63 KiB/s wr, 24 op/s 2023-09-10T00:22:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3755.796840 secs 2023-09-10T00:22:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:06 smithi170 ceph-mon[102832]: pgmap v2170: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T00:22:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:06 smithi082 ceph-mon[98694]: pgmap v2170: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T00:22:07.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:06 smithi098 ceph-mon[103124]: pgmap v2170: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T00:22:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3758.474972 secs 2023-09-10T00:22:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3758.474972 secs 2023-09-10T00:22:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3758.474972 secs 2023-09-10T00:22:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:08 smithi170 ceph-mon[102832]: pgmap v2171: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 67 KiB/s wr, 24 op/s 2023-09-10T00:22:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:08 smithi170 ceph-mon[102832]: osdmap e207: 12 total, 12 up, 12 in 2023-09-10T00:22:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3760.084091 secs 2023-09-10T00:22:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:08 smithi082 ceph-mon[98694]: pgmap v2171: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 67 KiB/s wr, 24 op/s 2023-09-10T00:22:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:08 smithi082 ceph-mon[98694]: osdmap e207: 12 total, 12 up, 12 in 2023-09-10T00:22:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3760.084091 secs 2023-09-10T00:22:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:08 smithi098 ceph-mon[103124]: pgmap v2171: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 67 KiB/s wr, 24 op/s 2023-09-10T00:22:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:08 smithi098 ceph-mon[103124]: osdmap e207: 12 total, 12 up, 12 in 2023-09-10T00:22:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3760.084091 secs 2023-09-10T00:22:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3760.796908 secs 2023-09-10T00:22:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3760.796908 secs 2023-09-10T00:22:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3760.796908 secs 2023-09-10T00:22:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:10 smithi170 ceph-mon[102832]: pgmap v2173: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T00:22:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:10 smithi082 ceph-mon[98694]: pgmap v2173: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T00:22:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:10 smithi098 ceph-mon[103124]: pgmap v2173: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T00:22:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:12 smithi170 ceph-mon[102832]: pgmap v2174: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T00:22:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3763.475054 secs 2023-09-10T00:22:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:12 smithi082 ceph-mon[98694]: pgmap v2174: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T00:22:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3763.475054 secs 2023-09-10T00:22:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:12 smithi098 ceph-mon[103124]: pgmap v2174: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T00:22:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3763.475054 secs 2023-09-10T00:22:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3765.084149 secs 2023-09-10T00:22:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3765.084149 secs 2023-09-10T00:22:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3765.084149 secs 2023-09-10T00:22:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:14 smithi170 ceph-mon[102832]: pgmap v2175: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T00:22:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3765.796975 secs 2023-09-10T00:22:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:14 smithi170 ceph-mon[102832]: osdmap e208: 12 total, 12 up, 12 in 2023-09-10T00:22:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:14 smithi082 ceph-mon[98694]: pgmap v2175: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T00:22:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3765.796975 secs 2023-09-10T00:22:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:14 smithi082 ceph-mon[98694]: osdmap e208: 12 total, 12 up, 12 in 2023-09-10T00:22:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:14 smithi098 ceph-mon[103124]: pgmap v2175: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T00:22:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3765.796975 secs 2023-09-10T00:22:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:14 smithi098 ceph-mon[103124]: osdmap e208: 12 total, 12 up, 12 in 2023-09-10T00:22:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:16 smithi170 ceph-mon[102832]: pgmap v2177: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 14 op/s 2023-09-10T00:22:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:16 smithi082 ceph-mon[98694]: pgmap v2177: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 14 op/s 2023-09-10T00:22:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:16 smithi098 ceph-mon[103124]: pgmap v2177: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 14 op/s 2023-09-10T00:22:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3768.475181 secs 2023-09-10T00:22:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3768.475181 secs 2023-09-10T00:22:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3768.475181 secs 2023-09-10T00:22:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:18 smithi170 ceph-mon[102832]: pgmap v2178: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 KiB/s wr, 11 op/s 2023-09-10T00:22:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3770.084207 secs 2023-09-10T00:22:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:18 smithi082 ceph-mon[98694]: pgmap v2178: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 KiB/s wr, 11 op/s 2023-09-10T00:22:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3770.084207 secs 2023-09-10T00:22:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:18 smithi098 ceph-mon[103124]: pgmap v2178: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 KiB/s wr, 11 op/s 2023-09-10T00:22:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3770.084207 secs 2023-09-10T00:22:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3770.797043 secs 2023-09-10T00:22:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3770.797043 secs 2023-09-10T00:22:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3770.797043 secs 2023-09-10T00:22:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:20 smithi170 ceph-mon[102832]: pgmap v2179: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T00:22:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:20 smithi082 ceph-mon[98694]: pgmap v2179: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T00:22:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:20 smithi098 ceph-mon[103124]: pgmap v2179: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T00:22:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:22 smithi170 ceph-mon[102832]: pgmap v2180: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T00:22:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3773.475240 secs 2023-09-10T00:22:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:22 smithi082 ceph-mon[98694]: pgmap v2180: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T00:22:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3773.475240 secs 2023-09-10T00:22:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:22 smithi098 ceph-mon[103124]: pgmap v2180: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T00:22:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3773.475240 secs 2023-09-10T00:22:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3775.084264 secs 2023-09-10T00:22:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3775.084264 secs 2023-09-10T00:22:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3775.084264 secs 2023-09-10T00:22:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:24 smithi170 ceph-mon[102832]: pgmap v2181: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T00:22:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3775.797112 secs 2023-09-10T00:22:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:24 smithi082 ceph-mon[98694]: pgmap v2181: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T00:22:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3775.797112 secs 2023-09-10T00:22:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:24 smithi098 ceph-mon[103124]: pgmap v2181: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T00:22:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3775.797112 secs 2023-09-10T00:22:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:26 smithi170 ceph-mon[102832]: pgmap v2182: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 86 KiB/s wr, 16 op/s 2023-09-10T00:22:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:26 smithi082 ceph-mon[98694]: pgmap v2182: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 86 KiB/s wr, 16 op/s 2023-09-10T00:22:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:26 smithi098 ceph-mon[103124]: pgmap v2182: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 86 KiB/s wr, 16 op/s 2023-09-10T00:22:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:28 smithi170 ceph-mon[102832]: pgmap v2183: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-10T00:22:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3780.084320 secs 2023-09-10T00:22:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:28 smithi082 ceph-mon[98694]: pgmap v2183: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-10T00:22:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3780.084320 secs 2023-09-10T00:22:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:28 smithi098 ceph-mon[103124]: pgmap v2183: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-10T00:22:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3780.084320 secs 2023-09-10T00:22:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3780.797183 secs 2023-09-10T00:22:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3780.797183 secs 2023-09-10T00:22:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3780.797183 secs 2023-09-10T00:22:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:30 smithi170 ceph-mon[102832]: pgmap v2184: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-10T00:22:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:30 smithi082 ceph-mon[98694]: pgmap v2184: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-10T00:22:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:30 smithi098 ceph-mon[103124]: pgmap v2184: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-10T00:22:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:22:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:22:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:22:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:32 smithi170 ceph-mon[102832]: pgmap v2185: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 125 KiB/s wr, 17 op/s 2023-09-10T00:22:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:22:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:22:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:22:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:32 smithi082 ceph-mon[98694]: pgmap v2185: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 125 KiB/s wr, 17 op/s 2023-09-10T00:22:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:22:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:22:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:22:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:32 smithi098 ceph-mon[103124]: pgmap v2185: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 125 KiB/s wr, 17 op/s 2023-09-10T00:22:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:22:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:22:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:22:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3785.084393 secs 2023-09-10T00:22:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3785.084393 secs 2023-09-10T00:22:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3785.084393 secs 2023-09-10T00:22:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:34 smithi170 ceph-mon[102832]: pgmap v2186: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T00:22:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3785.797247 secs 2023-09-10T00:22:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:34 smithi082 ceph-mon[98694]: pgmap v2186: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T00:22:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3785.797247 secs 2023-09-10T00:22:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:34 smithi098 ceph-mon[103124]: pgmap v2186: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T00:22:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3785.797247 secs 2023-09-10T00:22:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:36 smithi170 ceph-mon[102832]: pgmap v2187: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 150 KiB/s wr, 19 op/s 2023-09-10T00:22:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:36 smithi082 ceph-mon[98694]: pgmap v2187: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 150 KiB/s wr, 19 op/s 2023-09-10T00:22:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:36 smithi098 ceph-mon[103124]: pgmap v2187: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 150 KiB/s wr, 19 op/s 2023-09-10T00:22:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:38 smithi170 ceph-mon[102832]: pgmap v2188: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 13 op/s 2023-09-10T00:22:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3790.084451 secs 2023-09-10T00:22:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:38 smithi082 ceph-mon[98694]: pgmap v2188: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 13 op/s 2023-09-10T00:22:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3790.084451 secs 2023-09-10T00:22:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:38 smithi098 ceph-mon[103124]: pgmap v2188: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 13 op/s 2023-09-10T00:22:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3790.084451 secs 2023-09-10T00:22:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3790.797311 secs 2023-09-10T00:22:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3790.797311 secs 2023-09-10T00:22:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3790.797311 secs 2023-09-10T00:22:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:40 smithi170 ceph-mon[102832]: pgmap v2189: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 13 op/s 2023-09-10T00:22:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:40 smithi082 ceph-mon[98694]: pgmap v2189: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 13 op/s 2023-09-10T00:22:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:40 smithi098 ceph-mon[103124]: pgmap v2189: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 13 op/s 2023-09-10T00:22:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:42 smithi170 ceph-mon[102832]: pgmap v2190: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 229 KiB/s wr, 19 op/s 2023-09-10T00:22:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:42 smithi082 ceph-mon[98694]: pgmap v2190: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 229 KiB/s wr, 19 op/s 2023-09-10T00:22:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:42 smithi098 ceph-mon[103124]: pgmap v2190: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 229 KiB/s wr, 19 op/s 2023-09-10T00:22:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3795.084522 secs 2023-09-10T00:22:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3795.084522 secs 2023-09-10T00:22:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3795.084522 secs 2023-09-10T00:22:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:44 smithi170 ceph-mon[102832]: pgmap v2191: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T00:22:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3795.797378 secs 2023-09-10T00:22:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:44 smithi082 ceph-mon[98694]: pgmap v2191: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T00:22:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3795.797378 secs 2023-09-10T00:22:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:44 smithi098 ceph-mon[103124]: pgmap v2191: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T00:22:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3795.797378 secs 2023-09-10T00:22:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:46 smithi170 ceph-mon[102832]: pgmap v2192: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 187 KiB/s wr, 18 op/s 2023-09-10T00:22:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:22:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:22:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:22:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:22:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:46 smithi082 ceph-mon[98694]: pgmap v2192: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 187 KiB/s wr, 18 op/s 2023-09-10T00:22:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:22:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:22:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:22:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:22:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:46 smithi098 ceph-mon[103124]: pgmap v2192: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 187 KiB/s wr, 18 op/s 2023-09-10T00:22:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:22:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:22:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:22:47.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:22:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3798.475751 secs 2023-09-10T00:22:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3798.475751 secs 2023-09-10T00:22:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3798.475751 secs 2023-09-10T00:22:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:48 smithi170 ceph-mon[102832]: pgmap v2193: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T00:22:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3800.084584 secs 2023-09-10T00:22:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:48 smithi082 ceph-mon[98694]: pgmap v2193: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T00:22:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3800.084584 secs 2023-09-10T00:22:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:48 smithi098 ceph-mon[103124]: pgmap v2193: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T00:22:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3800.084584 secs 2023-09-10T00:22:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3800.797443 secs 2023-09-10T00:22:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3800.797443 secs 2023-09-10T00:22:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3800.797443 secs 2023-09-10T00:22:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:50 smithi170 ceph-mon[102832]: pgmap v2194: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T00:22:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:50 smithi082 ceph-mon[98694]: pgmap v2194: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T00:22:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:50 smithi098 ceph-mon[103124]: pgmap v2194: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T00:22:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:22:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:22:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:22:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:22:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:22:51.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:22:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:22:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:22:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:22:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:52 smithi170 ceph-mon[102832]: pgmap v2195: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 186 KiB/s wr, 16 op/s 2023-09-10T00:22:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3803.475777 secs 2023-09-10T00:22:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:52 smithi082 ceph-mon[98694]: pgmap v2195: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 186 KiB/s wr, 16 op/s 2023-09-10T00:22:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3803.475777 secs 2023-09-10T00:22:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:52 smithi098 ceph-mon[103124]: pgmap v2195: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 186 KiB/s wr, 16 op/s 2023-09-10T00:22:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3803.475777 secs 2023-09-10T00:22:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3805.084643 secs 2023-09-10T00:22:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3805.084643 secs 2023-09-10T00:22:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3805.084643 secs 2023-09-10T00:22:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:54 smithi170 ceph-mon[102832]: pgmap v2196: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 61 KiB/s wr, 10 op/s 2023-09-10T00:22:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3805.797485 secs 2023-09-10T00:22:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:54 smithi082 ceph-mon[98694]: pgmap v2196: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 61 KiB/s wr, 10 op/s 2023-09-10T00:22:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3805.797485 secs 2023-09-10T00:22:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:54 smithi098 ceph-mon[103124]: pgmap v2196: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 61 KiB/s wr, 10 op/s 2023-09-10T00:22:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3805.797485 secs 2023-09-10T00:22:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:56 smithi170 ceph-mon[102832]: pgmap v2197: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 83 KiB/s wr, 16 op/s 2023-09-10T00:22:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:56 smithi082 ceph-mon[98694]: pgmap v2197: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 83 KiB/s wr, 16 op/s 2023-09-10T00:22:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:56 smithi098 ceph-mon[103124]: pgmap v2197: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 83 KiB/s wr, 16 op/s 2023-09-10T00:22:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3808.475891 secs 2023-09-10T00:22:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3808.475891 secs 2023-09-10T00:22:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3808.475891 secs 2023-09-10T00:22:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:58 smithi082 ceph-mon[98694]: pgmap v2198: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 10 op/s 2023-09-10T00:22:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3810.084699 secs 2023-09-10T00:22:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:58 smithi098 ceph-mon[103124]: pgmap v2198: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 10 op/s 2023-09-10T00:22:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3810.084699 secs 2023-09-10T00:22:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:58 smithi170 ceph-mon[102832]: pgmap v2198: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 10 op/s 2023-09-10T00:22:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3810.084699 secs 2023-09-10T00:22:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:22:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3810.797551 secs 2023-09-10T00:23:00.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:22:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3810.797551 secs 2023-09-10T00:23:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:22:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3810.797551 secs 2023-09-10T00:23:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:00 smithi082 ceph-mon[98694]: pgmap v2199: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 10 op/s 2023-09-10T00:23:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:00 smithi098 ceph-mon[103124]: pgmap v2199: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 10 op/s 2023-09-10T00:23:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:00 smithi170 ceph-mon[102832]: pgmap v2199: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 10 op/s 2023-09-10T00:23:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:02 smithi082 ceph-mon[98694]: pgmap v2200: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T00:23:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3813.475949 secs 2023-09-10T00:23:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:02 smithi098 ceph-mon[103124]: pgmap v2200: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T00:23:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3813.475949 secs 2023-09-10T00:23:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:02 smithi170 ceph-mon[102832]: pgmap v2200: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T00:23:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3813.475949 secs 2023-09-10T00:23:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3815.084759 secs 2023-09-10T00:23:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3815.084759 secs 2023-09-10T00:23:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3815.084759 secs 2023-09-10T00:23:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:04 smithi082 ceph-mon[98694]: pgmap v2201: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 67 KiB/s wr, 24 op/s 2023-09-10T00:23:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3815.797618 secs 2023-09-10T00:23:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:04 smithi098 ceph-mon[103124]: pgmap v2201: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 67 KiB/s wr, 24 op/s 2023-09-10T00:23:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3815.797618 secs 2023-09-10T00:23:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:04 smithi170 ceph-mon[102832]: pgmap v2201: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 67 KiB/s wr, 24 op/s 2023-09-10T00:23:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3815.797618 secs 2023-09-10T00:23:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:06 smithi082 ceph-mon[98694]: pgmap v2202: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 106 KiB/s wr, 29 op/s 2023-09-10T00:23:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:06 smithi098 ceph-mon[103124]: pgmap v2202: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 106 KiB/s wr, 29 op/s 2023-09-10T00:23:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:06 smithi170 ceph-mon[102832]: pgmap v2202: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 106 KiB/s wr, 29 op/s 2023-09-10T00:23:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3818.476010 secs 2023-09-10T00:23:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3818.476010 secs 2023-09-10T00:23:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3818.476010 secs 2023-09-10T00:23:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:08 smithi098 ceph-mon[103124]: pgmap v2203: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 85 KiB/s wr, 24 op/s 2023-09-10T00:23:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:08 smithi098 ceph-mon[103124]: osdmap e209: 12 total, 12 up, 12 in 2023-09-10T00:23:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3820.084819 secs 2023-09-10T00:23:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:08 smithi170 ceph-mon[102832]: pgmap v2203: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 85 KiB/s wr, 24 op/s 2023-09-10T00:23:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:08 smithi170 ceph-mon[102832]: osdmap e209: 12 total, 12 up, 12 in 2023-09-10T00:23:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3820.084819 secs 2023-09-10T00:23:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:08 smithi082 ceph-mon[98694]: pgmap v2203: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 85 KiB/s wr, 24 op/s 2023-09-10T00:23:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:08 smithi082 ceph-mon[98694]: osdmap e209: 12 total, 12 up, 12 in 2023-09-10T00:23:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3820.084819 secs 2023-09-10T00:23:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3820.797706 secs 2023-09-10T00:23:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3820.797706 secs 2023-09-10T00:23:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3820.797706 secs 2023-09-10T00:23:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:10 smithi098 ceph-mon[103124]: pgmap v2205: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T00:23:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:10 smithi170 ceph-mon[102832]: pgmap v2205: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T00:23:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:10 smithi082 ceph-mon[98694]: pgmap v2205: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T00:23:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:12 smithi098 ceph-mon[103124]: pgmap v2206: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T00:23:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3823.476086 secs 2023-09-10T00:23:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:12 smithi170 ceph-mon[102832]: pgmap v2206: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T00:23:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3823.476086 secs 2023-09-10T00:23:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:12 smithi082 ceph-mon[98694]: pgmap v2206: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T00:23:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3823.476086 secs 2023-09-10T00:23:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3825.084903 secs 2023-09-10T00:23:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3825.084903 secs 2023-09-10T00:23:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3825.084903 secs 2023-09-10T00:23:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:14 smithi098 ceph-mon[103124]: pgmap v2207: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T00:23:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3825.797773 secs 2023-09-10T00:23:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:14 smithi098 ceph-mon[103124]: osdmap e210: 12 total, 12 up, 12 in 2023-09-10T00:23:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:14 smithi170 ceph-mon[102832]: pgmap v2207: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T00:23:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3825.797773 secs 2023-09-10T00:23:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:14 smithi170 ceph-mon[102832]: osdmap e210: 12 total, 12 up, 12 in 2023-09-10T00:23:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:14 smithi082 ceph-mon[98694]: pgmap v2207: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T00:23:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3825.797773 secs 2023-09-10T00:23:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:14 smithi082 ceph-mon[98694]: osdmap e210: 12 total, 12 up, 12 in 2023-09-10T00:23:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:16 smithi098 ceph-mon[103124]: pgmap v2209: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 31 KiB/s wr, 14 op/s 2023-09-10T00:23:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:16 smithi170 ceph-mon[102832]: pgmap v2209: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 31 KiB/s wr, 14 op/s 2023-09-10T00:23:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:16 smithi082 ceph-mon[98694]: pgmap v2209: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 31 KiB/s wr, 14 op/s 2023-09-10T00:23:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:18 smithi098 ceph-mon[103124]: pgmap v2210: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-10T00:23:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3830.084968 secs 2023-09-10T00:23:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:18 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:23:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:18 smithi170 ceph-mon[102832]: pgmap v2210: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-10T00:23:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3830.084968 secs 2023-09-10T00:23:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:18 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:23:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:18 smithi082 ceph-mon[98694]: pgmap v2210: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-10T00:23:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3830.084968 secs 2023-09-10T00:23:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:18 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:23:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3830.797839 secs 2023-09-10T00:23:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:19 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:23:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:19 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:23:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:19 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:23:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3830.797839 secs 2023-09-10T00:23:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:19 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:23:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:19 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:23:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:19 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:23:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3830.797839 secs 2023-09-10T00:23:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:19 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:23:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:19 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:23:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:19 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:23:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:20 smithi098 ceph-mon[103124]: pgmap v2211: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T00:23:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:20 smithi170 ceph-mon[102832]: pgmap v2211: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T00:23:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:20 smithi082 ceph-mon[98694]: pgmap v2211: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T00:23:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:22 smithi098 ceph-mon[103124]: pgmap v2212: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 91 KiB/s wr, 13 op/s 2023-09-10T00:23:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3833.476203 secs 2023-09-10T00:23:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:22 smithi170 ceph-mon[102832]: pgmap v2212: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 91 KiB/s wr, 13 op/s 2023-09-10T00:23:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3833.476203 secs 2023-09-10T00:23:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:22 smithi082 ceph-mon[98694]: pgmap v2212: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 91 KiB/s wr, 13 op/s 2023-09-10T00:23:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3833.476203 secs 2023-09-10T00:23:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:23:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:23:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3835.085026 secs 2023-09-10T00:23:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:23:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:23:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3835.085026 secs 2023-09-10T00:23:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:23:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:23:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3835.085026 secs 2023-09-10T00:23:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:24 smithi170 ceph-mon[102832]: pgmap v2213: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 91 KiB/s wr, 13 op/s 2023-09-10T00:23:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3835.797904 secs 2023-09-10T00:23:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:24 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:23:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:24 smithi082 ceph-mon[98694]: pgmap v2213: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 91 KiB/s wr, 13 op/s 2023-09-10T00:23:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3835.797904 secs 2023-09-10T00:23:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:24 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:23:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:24 smithi098 ceph-mon[103124]: pgmap v2213: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 91 KiB/s wr, 13 op/s 2023-09-10T00:23:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3835.797904 secs 2023-09-10T00:23:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:24 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:23:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:26 smithi170 ceph-mon[102832]: pgmap v2214: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 114 KiB/s wr, 19 op/s 2023-09-10T00:23:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:26 smithi082 ceph-mon[98694]: pgmap v2214: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 114 KiB/s wr, 19 op/s 2023-09-10T00:23:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:26 smithi098 ceph-mon[103124]: pgmap v2214: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 114 KiB/s wr, 19 op/s 2023-09-10T00:23:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:28 smithi170 ceph-mon[102832]: pgmap v2215: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T00:23:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3840.085133 secs 2023-09-10T00:23:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:28 smithi082 ceph-mon[98694]: pgmap v2215: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T00:23:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3840.085133 secs 2023-09-10T00:23:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:28 smithi098 ceph-mon[103124]: pgmap v2215: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T00:23:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3840.085133 secs 2023-09-10T00:23:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3840.797992 secs 2023-09-10T00:23:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3840.797992 secs 2023-09-10T00:23:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3840.797992 secs 2023-09-10T00:23:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:30 smithi170 ceph-mon[102832]: pgmap v2216: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T00:23:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:30 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:23:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:30 smithi082 ceph-mon[98694]: pgmap v2216: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T00:23:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:30 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:23:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:30 smithi098 ceph-mon[103124]: pgmap v2216: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T00:23:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:30 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:23:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:31 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:23:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:23:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:23:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:31 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:23:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:23:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:23:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:31 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:23:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:23:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:23:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:32 smithi170 ceph-mon[102832]: pgmap v2217: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 187 KiB/s wr, 18 op/s 2023-09-10T00:23:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:32 smithi082 ceph-mon[98694]: pgmap v2217: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 187 KiB/s wr, 18 op/s 2023-09-10T00:23:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:32 smithi098 ceph-mon[103124]: pgmap v2217: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 187 KiB/s wr, 18 op/s 2023-09-10T00:23:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3845.085191 secs 2023-09-10T00:23:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3845.085191 secs 2023-09-10T00:23:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3845.085191 secs 2023-09-10T00:23:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:34 smithi170 ceph-mon[102832]: pgmap v2218: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T00:23:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3845.798057 secs 2023-09-10T00:23:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:34 smithi082 ceph-mon[98694]: pgmap v2218: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T00:23:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3845.798057 secs 2023-09-10T00:23:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:34 smithi098 ceph-mon[103124]: pgmap v2218: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T00:23:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3845.798057 secs 2023-09-10T00:23:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:36 smithi170 ceph-mon[102832]: pgmap v2219: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 156 KiB/s wr, 18 op/s 2023-09-10T00:23:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:36 smithi082 ceph-mon[98694]: pgmap v2219: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 156 KiB/s wr, 18 op/s 2023-09-10T00:23:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:36 smithi098 ceph-mon[103124]: pgmap v2219: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 156 KiB/s wr, 18 op/s 2023-09-10T00:23:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:38 smithi170 ceph-mon[102832]: pgmap v2220: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T00:23:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3850.085250 secs 2023-09-10T00:23:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:38 smithi082 ceph-mon[98694]: pgmap v2220: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T00:23:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3850.085250 secs 2023-09-10T00:23:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:38 smithi098 ceph-mon[103124]: pgmap v2220: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T00:23:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3850.085250 secs 2023-09-10T00:23:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3850.798124 secs 2023-09-10T00:23:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3850.798124 secs 2023-09-10T00:23:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3850.798124 secs 2023-09-10T00:23:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:40 smithi170 ceph-mon[102832]: pgmap v2221: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T00:23:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:40 smithi082 ceph-mon[98694]: pgmap v2221: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T00:23:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:40 smithi098 ceph-mon[103124]: pgmap v2221: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T00:23:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:42 smithi170 ceph-mon[102832]: pgmap v2222: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 16 op/s 2023-09-10T00:23:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:42 smithi082 ceph-mon[98694]: pgmap v2222: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 16 op/s 2023-09-10T00:23:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:42 smithi098 ceph-mon[103124]: pgmap v2222: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 16 op/s 2023-09-10T00:23:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3855.085309 secs 2023-09-10T00:23:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3855.085309 secs 2023-09-10T00:23:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3855.085309 secs 2023-09-10T00:23:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:44 smithi170 ceph-mon[102832]: pgmap v2223: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T00:23:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3855.798189 secs 2023-09-10T00:23:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:44 smithi082 ceph-mon[98694]: pgmap v2223: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T00:23:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3855.798189 secs 2023-09-10T00:23:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:44 smithi098 ceph-mon[103124]: pgmap v2223: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T00:23:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3855.798189 secs 2023-09-10T00:23:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:46 smithi170 ceph-mon[102832]: pgmap v2224: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 112 KiB/s wr, 16 op/s 2023-09-10T00:23:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:23:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:46 smithi082 ceph-mon[98694]: pgmap v2224: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 112 KiB/s wr, 16 op/s 2023-09-10T00:23:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:23:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:46 smithi098 ceph-mon[103124]: pgmap v2224: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 112 KiB/s wr, 16 op/s 2023-09-10T00:23:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:23:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3858.476648 secs 2023-09-10T00:23:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:23:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:23:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:23:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3858.476648 secs 2023-09-10T00:23:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:23:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:23:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:23:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3858.476648 secs 2023-09-10T00:23:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:23:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:23:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:23:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:48 smithi170 ceph-mon[102832]: pgmap v2225: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T00:23:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3860.085372 secs 2023-09-10T00:23:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:48 smithi082 ceph-mon[98694]: pgmap v2225: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T00:23:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3860.085372 secs 2023-09-10T00:23:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:48 smithi098 ceph-mon[103124]: pgmap v2225: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T00:23:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3860.085372 secs 2023-09-10T00:23:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3860.798256 secs 2023-09-10T00:23:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3860.798256 secs 2023-09-10T00:23:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3860.798256 secs 2023-09-10T00:23:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:50 smithi170 ceph-mon[102832]: pgmap v2226: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T00:23:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:50 smithi082 ceph-mon[98694]: pgmap v2226: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T00:23:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:50 smithi098 ceph-mon[103124]: pgmap v2226: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T00:23:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:23:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:23:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:23:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:23:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:23:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:23:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:23:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:23:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:23:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:52 smithi170 ceph-mon[102832]: pgmap v2227: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 109 KiB/s wr, 16 op/s 2023-09-10T00:23:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3863.517903 secs 2023-09-10T00:23:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:52 smithi082 ceph-mon[98694]: pgmap v2227: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 109 KiB/s wr, 16 op/s 2023-09-10T00:23:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3863.517903 secs 2023-09-10T00:23:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:52 smithi098 ceph-mon[103124]: pgmap v2227: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 109 KiB/s wr, 16 op/s 2023-09-10T00:23:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3863.517903 secs 2023-09-10T00:23:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3865.085475 secs 2023-09-10T00:23:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3865.085475 secs 2023-09-10T00:23:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3865.085475 secs 2023-09-10T00:23:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:54 smithi170 ceph-mon[102832]: pgmap v2228: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T00:23:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3865.798324 secs 2023-09-10T00:23:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:54 smithi082 ceph-mon[98694]: pgmap v2228: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T00:23:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3865.798324 secs 2023-09-10T00:23:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:54 smithi098 ceph-mon[103124]: pgmap v2228: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T00:23:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3865.798324 secs 2023-09-10T00:23:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:56 smithi170 ceph-mon[102832]: pgmap v2229: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 111 KiB/s wr, 15 op/s 2023-09-10T00:23:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:56 smithi082 ceph-mon[98694]: pgmap v2229: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 111 KiB/s wr, 15 op/s 2023-09-10T00:23:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:56 smithi098 ceph-mon[103124]: pgmap v2229: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 111 KiB/s wr, 15 op/s 2023-09-10T00:23:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3868.517960 secs 2023-09-10T00:23:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3868.517960 secs 2023-09-10T00:23:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3868.517960 secs 2023-09-10T00:23:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:58 smithi170 ceph-mon[102832]: pgmap v2230: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T00:23:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3870.085508 secs 2023-09-10T00:23:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:58 smithi082 ceph-mon[98694]: pgmap v2230: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T00:23:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3870.085508 secs 2023-09-10T00:23:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:58 smithi098 ceph-mon[103124]: pgmap v2230: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T00:23:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3870.085508 secs 2023-09-10T00:24:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:23:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3870.798442 secs 2023-09-10T00:24:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:23:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3870.798442 secs 2023-09-10T00:24:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:23:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3870.798442 secs 2023-09-10T00:24:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:00 smithi170 ceph-mon[102832]: pgmap v2231: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T00:24:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:00 smithi082 ceph-mon[98694]: pgmap v2231: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T00:24:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:00 smithi098 ceph-mon[103124]: pgmap v2231: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T00:24:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:02 smithi170 ceph-mon[102832]: pgmap v2232: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 84 KiB/s wr, 24 op/s 2023-09-10T00:24:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3873.518019 secs 2023-09-10T00:24:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:02 smithi082 ceph-mon[98694]: pgmap v2232: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 84 KiB/s wr, 24 op/s 2023-09-10T00:24:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3873.518019 secs 2023-09-10T00:24:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:02 smithi098 ceph-mon[103124]: pgmap v2232: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 84 KiB/s wr, 24 op/s 2023-09-10T00:24:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3873.518019 secs 2023-09-10T00:24:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3875.085558 secs 2023-09-10T00:24:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3875.085558 secs 2023-09-10T00:24:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3875.085558 secs 2023-09-10T00:24:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:04 smithi170 ceph-mon[102832]: pgmap v2233: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 63 KiB/s wr, 19 op/s 2023-09-10T00:24:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3875.798493 secs 2023-09-10T00:24:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:04 smithi082 ceph-mon[98694]: pgmap v2233: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 63 KiB/s wr, 19 op/s 2023-09-10T00:24:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3875.798493 secs 2023-09-10T00:24:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:04 smithi098 ceph-mon[103124]: pgmap v2233: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 63 KiB/s wr, 19 op/s 2023-09-10T00:24:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3875.798493 secs 2023-09-10T00:24:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:06 smithi170 ceph-mon[102832]: pgmap v2234: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T00:24:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:06 smithi082 ceph-mon[98694]: pgmap v2234: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T00:24:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:06 smithi098 ceph-mon[103124]: pgmap v2234: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T00:24:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3878.518077 secs 2023-09-10T00:24:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:07 smithi170 ceph-mon[102832]: osdmap e211: 12 total, 12 up, 12 in 2023-09-10T00:24:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3878.518077 secs 2023-09-10T00:24:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:07 smithi082 ceph-mon[98694]: osdmap e211: 12 total, 12 up, 12 in 2023-09-10T00:24:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3878.518077 secs 2023-09-10T00:24:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:07 smithi098 ceph-mon[103124]: osdmap e211: 12 total, 12 up, 12 in 2023-09-10T00:24:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:08 smithi170 ceph-mon[102832]: pgmap v2236: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T00:24:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3880.085628 secs 2023-09-10T00:24:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:08 smithi082 ceph-mon[98694]: pgmap v2236: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T00:24:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3880.085628 secs 2023-09-10T00:24:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:08 smithi098 ceph-mon[103124]: pgmap v2236: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T00:24:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3880.085628 secs 2023-09-10T00:24:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3880.798561 secs 2023-09-10T00:24:10.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3880.798561 secs 2023-09-10T00:24:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3880.798561 secs 2023-09-10T00:24:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:10 smithi170 ceph-mon[102832]: pgmap v2237: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T00:24:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:10 smithi082 ceph-mon[98694]: pgmap v2237: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T00:24:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:10 smithi098 ceph-mon[103124]: pgmap v2237: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T00:24:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:12 smithi170 ceph-mon[102832]: pgmap v2238: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 61 KiB/s wr, 19 op/s 2023-09-10T00:24:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3883.518137 secs 2023-09-10T00:24:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:12 smithi082 ceph-mon[98694]: pgmap v2238: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 61 KiB/s wr, 19 op/s 2023-09-10T00:24:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3883.518137 secs 2023-09-10T00:24:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:12 smithi098 ceph-mon[103124]: pgmap v2238: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 61 KiB/s wr, 19 op/s 2023-09-10T00:24:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3883.518137 secs 2023-09-10T00:24:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3885.085708 secs 2023-09-10T00:24:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3885.085708 secs 2023-09-10T00:24:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3885.085708 secs 2023-09-10T00:24:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:14 smithi170 ceph-mon[102832]: pgmap v2239: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 61 KiB/s wr, 19 op/s 2023-09-10T00:24:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3885.798628 secs 2023-09-10T00:24:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:14 smithi170 ceph-mon[102832]: osdmap e212: 12 total, 12 up, 12 in 2023-09-10T00:24:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:14 smithi082 ceph-mon[98694]: pgmap v2239: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 61 KiB/s wr, 19 op/s 2023-09-10T00:24:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3885.798628 secs 2023-09-10T00:24:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:14 smithi082 ceph-mon[98694]: osdmap e212: 12 total, 12 up, 12 in 2023-09-10T00:24:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:14 smithi098 ceph-mon[103124]: pgmap v2239: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 61 KiB/s wr, 19 op/s 2023-09-10T00:24:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3885.798628 secs 2023-09-10T00:24:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:14 smithi098 ceph-mon[103124]: osdmap e212: 12 total, 12 up, 12 in 2023-09-10T00:24:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:16 smithi170 ceph-mon[102832]: pgmap v2241: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 38 KiB/s wr, 14 op/s 2023-09-10T00:24:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:16 smithi082 ceph-mon[98694]: pgmap v2241: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 38 KiB/s wr, 14 op/s 2023-09-10T00:24:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:16 smithi098 ceph-mon[103124]: pgmap v2241: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 38 KiB/s wr, 14 op/s 2023-09-10T00:24:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:18 smithi170 ceph-mon[102832]: pgmap v2242: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2023-09-10T00:24:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3890.085811 secs 2023-09-10T00:24:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:18 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:24:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:18 smithi082 ceph-mon[98694]: pgmap v2242: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2023-09-10T00:24:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3890.085811 secs 2023-09-10T00:24:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:18 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:24:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:18 smithi098 ceph-mon[103124]: pgmap v2242: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2023-09-10T00:24:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3890.085811 secs 2023-09-10T00:24:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:18 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:24:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3890.798693 secs 2023-09-10T00:24:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:19 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:24:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:19 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:24:20.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:19 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:24:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3890.798693 secs 2023-09-10T00:24:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:19 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:24:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:19 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:24:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:19 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:24:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3890.798693 secs 2023-09-10T00:24:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:19 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:24:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:19 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:24:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:19 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:24:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:20 smithi170 ceph-mon[102832]: pgmap v2243: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2023-09-10T00:24:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:20 smithi082 ceph-mon[98694]: pgmap v2243: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2023-09-10T00:24:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:20 smithi098 ceph-mon[103124]: pgmap v2243: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2023-09-10T00:24:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:22 smithi170 ceph-mon[102832]: pgmap v2244: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T00:24:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:22 smithi082 ceph-mon[98694]: pgmap v2244: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T00:24:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:22 smithi098 ceph-mon[103124]: pgmap v2244: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T00:24:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3895.085871 secs 2023-09-10T00:24:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3895.085871 secs 2023-09-10T00:24:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3895.085871 secs 2023-09-10T00:24:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:24 smithi082 ceph-mon[98694]: pgmap v2245: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T00:24:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3895.798760 secs 2023-09-10T00:24:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:24 smithi098 ceph-mon[103124]: pgmap v2245: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T00:24:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3895.798760 secs 2023-09-10T00:24:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:24 smithi170 ceph-mon[102832]: pgmap v2245: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T00:24:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3895.798760 secs 2023-09-10T00:24:27.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:26 smithi082 ceph-mon[98694]: pgmap v2246: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 17 op/s 2023-09-10T00:24:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:26 smithi098 ceph-mon[103124]: pgmap v2246: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 17 op/s 2023-09-10T00:24:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:26 smithi170 ceph-mon[102832]: pgmap v2246: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 17 op/s 2023-09-10T00:24:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3898.518413 secs 2023-09-10T00:24:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3898.518413 secs 2023-09-10T00:24:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3898.518413 secs 2023-09-10T00:24:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:28 smithi082 ceph-mon[98694]: pgmap v2247: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T00:24:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3900.085933 secs 2023-09-10T00:24:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:28 smithi098 ceph-mon[103124]: pgmap v2247: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T00:24:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3900.085933 secs 2023-09-10T00:24:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:28 smithi170 ceph-mon[102832]: pgmap v2247: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T00:24:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3900.085933 secs 2023-09-10T00:24:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3900.798824 secs 2023-09-10T00:24:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3900.798824 secs 2023-09-10T00:24:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3900.798824 secs 2023-09-10T00:24:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:30 smithi082 ceph-mon[98694]: pgmap v2248: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T00:24:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:30 smithi098 ceph-mon[103124]: pgmap v2248: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T00:24:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:30 smithi170 ceph-mon[102832]: pgmap v2248: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T00:24:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:31 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:24:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:24:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:24:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:31 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:24:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:24:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:24:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:31 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:24:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:24:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:24:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:32 smithi098 ceph-mon[103124]: pgmap v2249: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 199 KiB/s wr, 17 op/s 2023-09-10T00:24:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3903.518534 secs 2023-09-10T00:24:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:32 smithi170 ceph-mon[102832]: pgmap v2249: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 199 KiB/s wr, 17 op/s 2023-09-10T00:24:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3903.518534 secs 2023-09-10T00:24:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:32 smithi082 ceph-mon[98694]: pgmap v2249: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 199 KiB/s wr, 17 op/s 2023-09-10T00:24:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3903.518534 secs 2023-09-10T00:24:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3905.085994 secs 2023-09-10T00:24:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3905.085994 secs 2023-09-10T00:24:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3905.085994 secs 2023-09-10T00:24:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:34 smithi098 ceph-mon[103124]: pgmap v2250: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 164 KiB/s wr, 12 op/s 2023-09-10T00:24:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3905.798888 secs 2023-09-10T00:24:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:34 smithi170 ceph-mon[102832]: pgmap v2250: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 164 KiB/s wr, 12 op/s 2023-09-10T00:24:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3905.798888 secs 2023-09-10T00:24:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:34 smithi082 ceph-mon[98694]: pgmap v2250: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 164 KiB/s wr, 12 op/s 2023-09-10T00:24:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3905.798888 secs 2023-09-10T00:24:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:36 smithi098 ceph-mon[103124]: pgmap v2251: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 190 KiB/s wr, 18 op/s 2023-09-10T00:24:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:36 smithi170 ceph-mon[102832]: pgmap v2251: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 190 KiB/s wr, 18 op/s 2023-09-10T00:24:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:36 smithi082 ceph-mon[98694]: pgmap v2251: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 190 KiB/s wr, 18 op/s 2023-09-10T00:24:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3908.518595 secs 2023-09-10T00:24:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3908.518595 secs 2023-09-10T00:24:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3908.518595 secs 2023-09-10T00:24:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:38 smithi098 ceph-mon[103124]: pgmap v2252: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T00:24:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3910.086098 secs 2023-09-10T00:24:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:38 smithi170 ceph-mon[102832]: pgmap v2252: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T00:24:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3910.086098 secs 2023-09-10T00:24:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:38 smithi082 ceph-mon[98694]: pgmap v2252: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T00:24:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3910.086098 secs 2023-09-10T00:24:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3910.798957 secs 2023-09-10T00:24:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3910.798957 secs 2023-09-10T00:24:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3910.798957 secs 2023-09-10T00:24:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:40 smithi098 ceph-mon[103124]: pgmap v2253: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T00:24:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:40 smithi170 ceph-mon[102832]: pgmap v2253: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T00:24:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:40 smithi082 ceph-mon[98694]: pgmap v2253: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T00:24:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:42 smithi098 ceph-mon[103124]: pgmap v2254: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 177 KiB/s wr, 17 op/s 2023-09-10T00:24:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3913.518676 secs 2023-09-10T00:24:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:42 smithi170 ceph-mon[102832]: pgmap v2254: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 177 KiB/s wr, 17 op/s 2023-09-10T00:24:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3913.518676 secs 2023-09-10T00:24:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:42 smithi082 ceph-mon[98694]: pgmap v2254: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 177 KiB/s wr, 17 op/s 2023-09-10T00:24:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3913.518676 secs 2023-09-10T00:24:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3915.086179 secs 2023-09-10T00:24:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3915.086179 secs 2023-09-10T00:24:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3915.086179 secs 2023-09-10T00:24:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:44 smithi098 ceph-mon[103124]: pgmap v2255: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T00:24:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3915.799022 secs 2023-09-10T00:24:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:44 smithi170 ceph-mon[102832]: pgmap v2255: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T00:24:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3915.799022 secs 2023-09-10T00:24:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:44 smithi082 ceph-mon[98694]: pgmap v2255: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T00:24:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3915.799022 secs 2023-09-10T00:24:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:47 smithi098 ceph-mon[103124]: pgmap v2256: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 151 KiB/s wr, 16 op/s 2023-09-10T00:24:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:47 smithi170 ceph-mon[102832]: pgmap v2256: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 151 KiB/s wr, 16 op/s 2023-09-10T00:24:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:47 smithi082 ceph-mon[98694]: pgmap v2256: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 151 KiB/s wr, 16 op/s 2023-09-10T00:24:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3918.518797 secs 2023-09-10T00:24:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:24:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:24:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:24:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:24:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3918.518797 secs 2023-09-10T00:24:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:24:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:24:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:24:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:24:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3918.518797 secs 2023-09-10T00:24:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:24:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:24:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:24:48.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:24:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:49 smithi170 ceph-mon[102832]: pgmap v2257: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T00:24:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3920.086239 secs 2023-09-10T00:24:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:49 smithi082 ceph-mon[98694]: pgmap v2257: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T00:24:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3920.086239 secs 2023-09-10T00:24:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:49 smithi098 ceph-mon[103124]: pgmap v2257: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T00:24:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3920.086239 secs 2023-09-10T00:24:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3920.799089 secs 2023-09-10T00:24:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3920.799089 secs 2023-09-10T00:24:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3920.799089 secs 2023-09-10T00:24:51.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:51 smithi170 ceph-mon[102832]: pgmap v2258: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T00:24:51.553 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:51 smithi098 ceph-mon[103124]: pgmap v2258: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T00:24:51.554 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:51 smithi082 ceph-mon[98694]: pgmap v2258: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T00:24:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:53 smithi170 ceph-mon[102832]: pgmap v2259: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 150 KiB/s wr, 16 op/s 2023-09-10T00:24:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3923.518860 secs 2023-09-10T00:24:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:53 smithi082 ceph-mon[98694]: pgmap v2259: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 150 KiB/s wr, 16 op/s 2023-09-10T00:24:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3923.518860 secs 2023-09-10T00:24:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:53 smithi098 ceph-mon[103124]: pgmap v2259: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 150 KiB/s wr, 16 op/s 2023-09-10T00:24:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3923.518860 secs 2023-09-10T00:24:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3925.086297 secs 2023-09-10T00:24:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3925.086297 secs 2023-09-10T00:24:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3925.086297 secs 2023-09-10T00:24:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:55 smithi170 ceph-mon[102832]: pgmap v2260: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T00:24:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3925.799156 secs 2023-09-10T00:24:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:55 smithi082 ceph-mon[98694]: pgmap v2260: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T00:24:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3925.799156 secs 2023-09-10T00:24:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:55 smithi098 ceph-mon[103124]: pgmap v2260: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T00:24:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3925.799156 secs 2023-09-10T00:24:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:57 smithi170 ceph-mon[102832]: pgmap v2261: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 83 KiB/s wr, 15 op/s 2023-09-10T00:24:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:57 smithi082 ceph-mon[98694]: pgmap v2261: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 83 KiB/s wr, 15 op/s 2023-09-10T00:24:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:57 smithi098 ceph-mon[103124]: pgmap v2261: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 83 KiB/s wr, 15 op/s 2023-09-10T00:24:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3928.518920 secs 2023-09-10T00:24:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3928.518920 secs 2023-09-10T00:24:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3928.518920 secs 2023-09-10T00:24:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:59 smithi170 ceph-mon[102832]: pgmap v2262: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T00:24:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:24:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3930.086353 secs 2023-09-10T00:24:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:59 smithi082 ceph-mon[98694]: pgmap v2262: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T00:24:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:24:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3930.086353 secs 2023-09-10T00:24:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:59 smithi098 ceph-mon[103124]: pgmap v2262: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T00:24:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:24:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3930.086353 secs 2023-09-10T00:25:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3930.799220 secs 2023-09-10T00:25:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3930.799220 secs 2023-09-10T00:25:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3930.799220 secs 2023-09-10T00:25:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:01 smithi170 ceph-mon[102832]: pgmap v2263: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T00:25:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:01 smithi082 ceph-mon[98694]: pgmap v2263: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T00:25:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:01 smithi098 ceph-mon[103124]: pgmap v2263: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T00:25:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:03 smithi170 ceph-mon[102832]: pgmap v2264: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 44 KiB/s wr, 15 op/s 2023-09-10T00:25:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3933.518983 secs 2023-09-10T00:25:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:03 smithi082 ceph-mon[98694]: pgmap v2264: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 44 KiB/s wr, 15 op/s 2023-09-10T00:25:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3933.518983 secs 2023-09-10T00:25:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:03 smithi098 ceph-mon[103124]: pgmap v2264: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 44 KiB/s wr, 15 op/s 2023-09-10T00:25:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3933.518983 secs 2023-09-10T00:25:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3935.086438 secs 2023-09-10T00:25:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3935.086438 secs 2023-09-10T00:25:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3935.086438 secs 2023-09-10T00:25:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:05 smithi170 ceph-mon[102832]: pgmap v2265: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 19 KiB/s wr, 9 op/s 2023-09-10T00:25:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3935.799284 secs 2023-09-10T00:25:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:05 smithi082 ceph-mon[98694]: pgmap v2265: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 19 KiB/s wr, 9 op/s 2023-09-10T00:25:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3935.799284 secs 2023-09-10T00:25:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:05 smithi098 ceph-mon[103124]: pgmap v2265: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 19 KiB/s wr, 9 op/s 2023-09-10T00:25:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3935.799284 secs 2023-09-10T00:25:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:07 smithi170 ceph-mon[102832]: pgmap v2266: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T00:25:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:07 smithi082 ceph-mon[98694]: pgmap v2266: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T00:25:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:07 smithi098 ceph-mon[103124]: pgmap v2266: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T00:25:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3938.519044 secs 2023-09-10T00:25:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:08 smithi170 ceph-mon[102832]: osdmap e213: 12 total, 12 up, 12 in 2023-09-10T00:25:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3938.519044 secs 2023-09-10T00:25:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:08 smithi082 ceph-mon[98694]: osdmap e213: 12 total, 12 up, 12 in 2023-09-10T00:25:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3938.519044 secs 2023-09-10T00:25:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:08 smithi098 ceph-mon[103124]: osdmap e213: 12 total, 12 up, 12 in 2023-09-10T00:25:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:09 smithi170 ceph-mon[102832]: pgmap v2268: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T00:25:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3940.086492 secs 2023-09-10T00:25:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:09 smithi082 ceph-mon[98694]: pgmap v2268: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T00:25:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3940.086492 secs 2023-09-10T00:25:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:09 smithi098 ceph-mon[103124]: pgmap v2268: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T00:25:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3940.086492 secs 2023-09-10T00:25:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3940.799353 secs 2023-09-10T00:25:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3940.799353 secs 2023-09-10T00:25:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3940.799353 secs 2023-09-10T00:25:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:11 smithi170 ceph-mon[102832]: pgmap v2269: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T00:25:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:11 smithi082 ceph-mon[98694]: pgmap v2269: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T00:25:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:11 smithi098 ceph-mon[103124]: pgmap v2269: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T00:25:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:13 smithi170 ceph-mon[102832]: pgmap v2270: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T00:25:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3943.519105 secs 2023-09-10T00:25:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:13 smithi082 ceph-mon[98694]: pgmap v2270: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T00:25:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3943.519105 secs 2023-09-10T00:25:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:13 smithi098 ceph-mon[103124]: pgmap v2270: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T00:25:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3943.519105 secs 2023-09-10T00:25:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3945.086557 secs 2023-09-10T00:25:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3945.086557 secs 2023-09-10T00:25:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3945.086557 secs 2023-09-10T00:25:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:15 smithi170 ceph-mon[102832]: pgmap v2271: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T00:25:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3945.799419 secs 2023-09-10T00:25:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:15 smithi170 ceph-mon[102832]: osdmap e214: 12 total, 12 up, 12 in 2023-09-10T00:25:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:15 smithi082 ceph-mon[98694]: pgmap v2271: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T00:25:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3945.799419 secs 2023-09-10T00:25:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:15 smithi082 ceph-mon[98694]: osdmap e214: 12 total, 12 up, 12 in 2023-09-10T00:25:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:15 smithi098 ceph-mon[103124]: pgmap v2271: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T00:25:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3945.799419 secs 2023-09-10T00:25:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:15 smithi098 ceph-mon[103124]: osdmap e214: 12 total, 12 up, 12 in 2023-09-10T00:25:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:17 smithi170 ceph-mon[102832]: pgmap v2273: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2023-09-10T00:25:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:17 smithi082 ceph-mon[98694]: pgmap v2273: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2023-09-10T00:25:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:17 smithi098 ceph-mon[103124]: pgmap v2273: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 KiB/s wr, 13 op/s 2023-09-10T00:25:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:19 smithi170 ceph-mon[102832]: pgmap v2274: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T00:25:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3950.086614 secs 2023-09-10T00:25:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:25:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:19 smithi082 ceph-mon[98694]: pgmap v2274: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T00:25:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3950.086614 secs 2023-09-10T00:25:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:25:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:19 smithi098 ceph-mon[103124]: pgmap v2274: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T00:25:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3950.086614 secs 2023-09-10T00:25:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:25:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3950.799482 secs 2023-09-10T00:25:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:25:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:25:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:25:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3950.799482 secs 2023-09-10T00:25:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:25:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:25:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:25:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3950.799482 secs 2023-09-10T00:25:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:25:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:25:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:25:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:21 smithi170 ceph-mon[102832]: pgmap v2275: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T00:25:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:21 smithi082 ceph-mon[98694]: pgmap v2275: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T00:25:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:21 smithi098 ceph-mon[103124]: pgmap v2275: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T00:25:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:23 smithi170 ceph-mon[102832]: pgmap v2276: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T00:25:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:23 smithi082 ceph-mon[98694]: pgmap v2276: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T00:25:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:23 smithi098 ceph-mon[103124]: pgmap v2276: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T00:25:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3955.086701 secs 2023-09-10T00:25:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3955.086701 secs 2023-09-10T00:25:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3955.086701 secs 2023-09-10T00:25:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:25 smithi170 ceph-mon[102832]: pgmap v2277: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T00:25:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3955.799548 secs 2023-09-10T00:25:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:25 smithi082 ceph-mon[98694]: pgmap v2277: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T00:25:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3955.799548 secs 2023-09-10T00:25:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:25 smithi098 ceph-mon[103124]: pgmap v2277: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T00:25:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3955.799548 secs 2023-09-10T00:25:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:27 smithi170 ceph-mon[102832]: pgmap v2278: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 90 KiB/s wr, 16 op/s 2023-09-10T00:25:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:27 smithi082 ceph-mon[98694]: pgmap v2278: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 90 KiB/s wr, 16 op/s 2023-09-10T00:25:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:27 smithi098 ceph-mon[103124]: pgmap v2278: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 90 KiB/s wr, 16 op/s 2023-09-10T00:25:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:29 smithi170 ceph-mon[102832]: pgmap v2279: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T00:25:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3960.086758 secs 2023-09-10T00:25:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:29 smithi082 ceph-mon[98694]: pgmap v2279: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T00:25:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3960.086758 secs 2023-09-10T00:25:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:29 smithi098 ceph-mon[103124]: pgmap v2279: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T00:25:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3960.086758 secs 2023-09-10T00:25:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3960.799612 secs 2023-09-10T00:25:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3960.799612 secs 2023-09-10T00:25:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3960.799612 secs 2023-09-10T00:25:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:31 smithi170 ceph-mon[102832]: pgmap v2280: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T00:25:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:31 smithi082 ceph-mon[98694]: pgmap v2280: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T00:25:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:31 smithi098 ceph-mon[103124]: pgmap v2280: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T00:25:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:33 smithi170 ceph-mon[102832]: pgmap v2281: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 130 KiB/s wr, 16 op/s 2023-09-10T00:25:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:33 smithi082 ceph-mon[98694]: pgmap v2281: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 130 KiB/s wr, 16 op/s 2023-09-10T00:25:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:33 smithi098 ceph-mon[103124]: pgmap v2281: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 130 KiB/s wr, 16 op/s 2023-09-10T00:25:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3965.086817 secs 2023-09-10T00:25:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3965.086817 secs 2023-09-10T00:25:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3965.086817 secs 2023-09-10T00:25:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:35 smithi170 ceph-mon[102832]: pgmap v2282: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T00:25:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3965.799676 secs 2023-09-10T00:25:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:35 smithi082 ceph-mon[98694]: pgmap v2282: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T00:25:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3965.799676 secs 2023-09-10T00:25:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:35 smithi098 ceph-mon[103124]: pgmap v2282: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T00:25:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3965.799676 secs 2023-09-10T00:25:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:37 smithi170 ceph-mon[102832]: pgmap v2283: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 156 KiB/s wr, 17 op/s 2023-09-10T00:25:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:37 smithi082 ceph-mon[98694]: pgmap v2283: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 156 KiB/s wr, 17 op/s 2023-09-10T00:25:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:37 smithi098 ceph-mon[103124]: pgmap v2283: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 156 KiB/s wr, 17 op/s 2023-09-10T00:25:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:39 smithi170 ceph-mon[102832]: pgmap v2284: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T00:25:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3970.086876 secs 2023-09-10T00:25:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:39 smithi082 ceph-mon[98694]: pgmap v2284: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T00:25:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3970.086876 secs 2023-09-10T00:25:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:39 smithi098 ceph-mon[103124]: pgmap v2284: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T00:25:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3970.086876 secs 2023-09-10T00:25:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3970.799742 secs 2023-09-10T00:25:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3970.799742 secs 2023-09-10T00:25:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3970.799742 secs 2023-09-10T00:25:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:41 smithi170 ceph-mon[102832]: pgmap v2285: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T00:25:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:41 smithi082 ceph-mon[98694]: pgmap v2285: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T00:25:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:41 smithi098 ceph-mon[103124]: pgmap v2285: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T00:25:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:43 smithi170 ceph-mon[102832]: pgmap v2286: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 178 KiB/s wr, 17 op/s 2023-09-10T00:25:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:43 smithi082 ceph-mon[98694]: pgmap v2286: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 178 KiB/s wr, 17 op/s 2023-09-10T00:25:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:43 smithi098 ceph-mon[103124]: pgmap v2286: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 178 KiB/s wr, 17 op/s 2023-09-10T00:25:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3975.086962 secs 2023-09-10T00:25:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3975.086962 secs 2023-09-10T00:25:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3975.086962 secs 2023-09-10T00:25:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:45 smithi170 ceph-mon[102832]: pgmap v2287: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T00:25:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3975.799806 secs 2023-09-10T00:25:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:45 smithi082 ceph-mon[98694]: pgmap v2287: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T00:25:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3975.799806 secs 2023-09-10T00:25:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:45 smithi098 ceph-mon[103124]: pgmap v2287: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T00:25:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3975.799806 secs 2023-09-10T00:25:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:46 smithi082 ceph-mon[98694]: pgmap v2288: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 138 KiB/s wr, 17 op/s 2023-09-10T00:25:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:46 smithi098 ceph-mon[103124]: pgmap v2288: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 138 KiB/s wr, 17 op/s 2023-09-10T00:25:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:46 smithi170 ceph-mon[102832]: pgmap v2288: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 138 KiB/s wr, 17 op/s 2023-09-10T00:25:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3978.519598 secs 2023-09-10T00:25:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3978.519598 secs 2023-09-10T00:25:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3978.519598 secs 2023-09-10T00:25:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:48 smithi170 ceph-mon[102832]: pgmap v2289: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T00:25:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3980.087045 secs 2023-09-10T00:25:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:25:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:48 smithi082 ceph-mon[98694]: pgmap v2289: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T00:25:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3980.087045 secs 2023-09-10T00:25:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:25:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:48 smithi098 ceph-mon[103124]: pgmap v2289: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T00:25:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3980.087045 secs 2023-09-10T00:25:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:25:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3980.799872 secs 2023-09-10T00:25:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3980.799872 secs 2023-09-10T00:25:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3980.799872 secs 2023-09-10T00:25:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:50 smithi170 ceph-mon[102832]: pgmap v2290: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T00:25:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:50 smithi082 ceph-mon[98694]: pgmap v2290: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T00:25:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:50 smithi098 ceph-mon[103124]: pgmap v2290: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T00:25:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:25:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:25:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:25:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:25:52.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:52.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:52.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:52.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:52.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:52.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:52.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:52.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:52.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:25:52.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:25:52.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:25:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:25:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:25:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:25:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:25:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:52.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:52.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:52.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:52.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:52.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:52.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:25:52.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:25:52.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:25:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:25:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:25:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:25:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:25:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:52.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:52.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:52.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:52.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:52.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:25:52.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:25:52.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:25:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:52 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T00:25:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:52 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T00:25:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:52 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T00:25:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:52 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T00:25:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:52 smithi170 ceph-mon[102832]: pgmap v2291: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 137 KiB/s wr, 16 op/s 2023-09-10T00:25:53.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3983.519658 secs 2023-09-10T00:25:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:52 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T00:25:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:52 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T00:25:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:52 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T00:25:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:52 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T00:25:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:52 smithi082 ceph-mon[98694]: pgmap v2291: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 137 KiB/s wr, 16 op/s 2023-09-10T00:25:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3983.519658 secs 2023-09-10T00:25:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:52 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T00:25:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:52 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T00:25:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:52 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T00:25:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:52 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T00:25:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:52 smithi098 ceph-mon[103124]: pgmap v2291: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 137 KiB/s wr, 16 op/s 2023-09-10T00:25:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3983.519658 secs 2023-09-10T00:25:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3985.087101 secs 2023-09-10T00:25:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:25:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:25:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:54 smithi082 ceph-mon[98694]: pgmap v2292: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T00:25:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:54.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:54.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:54.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:54 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T00:25:54.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:54 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T00:25:54.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:25:54.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:25:54.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:25:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3985.087101 secs 2023-09-10T00:25:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:25:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:25:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:54 smithi098 ceph-mon[103124]: pgmap v2292: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T00:25:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:54.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:54.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:54.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:54 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T00:25:54.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:54 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T00:25:54.509 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:25:54.509 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:25:54.509 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:25:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3985.087101 secs 2023-09-10T00:25:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:25:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:25:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:54 smithi170 ceph-mon[102832]: pgmap v2292: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T00:25:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:54.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:25:54.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:54 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T00:25:54.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:54 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T00:25:54.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:25:54.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:25:54.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:25:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3985.799940 secs 2023-09-10T00:25:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3985.799940 secs 2023-09-10T00:25:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3985.799940 secs 2023-09-10T00:25:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:56 smithi082 ceph-mon[98694]: pgmap v2293: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 86 KiB/s wr, 16 op/s 2023-09-10T00:25:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:56 smithi098 ceph-mon[103124]: pgmap v2293: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 86 KiB/s wr, 16 op/s 2023-09-10T00:25:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:56 smithi170 ceph-mon[102832]: pgmap v2293: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 86 KiB/s wr, 16 op/s 2023-09-10T00:25:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3988.519716 secs 2023-09-10T00:25:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3988.519716 secs 2023-09-10T00:25:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3988.519716 secs 2023-09-10T00:25:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:58 smithi170 ceph-mon[102832]: pgmap v2294: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:25:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3990.087161 secs 2023-09-10T00:25:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:58 smithi082 ceph-mon[98694]: pgmap v2294: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:25:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3990.087161 secs 2023-09-10T00:25:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:58 smithi098 ceph-mon[103124]: pgmap v2294: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:25:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3990.087161 secs 2023-09-10T00:25:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:25:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3990.800003 secs 2023-09-10T00:25:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:25:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3990.800003 secs 2023-09-10T00:26:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:25:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3990.800003 secs 2023-09-10T00:26:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:00 smithi170 ceph-mon[102832]: pgmap v2295: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:26:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:00 smithi082 ceph-mon[98694]: pgmap v2295: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:26:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:00 smithi098 ceph-mon[103124]: pgmap v2295: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:26:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:02 smithi170 ceph-mon[102832]: pgmap v2296: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 15 op/s 2023-09-10T00:26:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3993.519826 secs 2023-09-10T00:26:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:02 smithi082 ceph-mon[98694]: pgmap v2296: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 15 op/s 2023-09-10T00:26:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3993.519826 secs 2023-09-10T00:26:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:02 smithi098 ceph-mon[103124]: pgmap v2296: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 15 op/s 2023-09-10T00:26:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3993.519826 secs 2023-09-10T00:26:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3995.087241 secs 2023-09-10T00:26:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3995.087241 secs 2023-09-10T00:26:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3995.087241 secs 2023-09-10T00:26:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:04 smithi170 ceph-mon[102832]: pgmap v2297: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 23 KiB/s wr, 9 op/s 2023-09-10T00:26:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 3995.800067 secs 2023-09-10T00:26:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:04 smithi082 ceph-mon[98694]: pgmap v2297: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 23 KiB/s wr, 9 op/s 2023-09-10T00:26:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 3995.800067 secs 2023-09-10T00:26:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:04 smithi098 ceph-mon[103124]: pgmap v2297: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 23 KiB/s wr, 9 op/s 2023-09-10T00:26:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 3995.800067 secs 2023-09-10T00:26:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:06 smithi170 ceph-mon[102832]: pgmap v2298: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T00:26:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:06 smithi082 ceph-mon[98694]: pgmap v2298: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T00:26:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:06 smithi098 ceph-mon[103124]: pgmap v2298: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T00:26:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 3998.519887 secs 2023-09-10T00:26:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 3998.519887 secs 2023-09-10T00:26:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 3998.519887 secs 2023-09-10T00:26:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:08 smithi170 ceph-mon[102832]: pgmap v2299: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 90 KiB/s wr, 24 op/s 2023-09-10T00:26:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:08 smithi170 ceph-mon[102832]: osdmap e215: 12 total, 12 up, 12 in 2023-09-10T00:26:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4000.087297 secs 2023-09-10T00:26:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:08 smithi082 ceph-mon[98694]: pgmap v2299: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 90 KiB/s wr, 24 op/s 2023-09-10T00:26:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:08 smithi082 ceph-mon[98694]: osdmap e215: 12 total, 12 up, 12 in 2023-09-10T00:26:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4000.087297 secs 2023-09-10T00:26:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:08 smithi098 ceph-mon[103124]: pgmap v2299: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 90 KiB/s wr, 24 op/s 2023-09-10T00:26:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:08 smithi098 ceph-mon[103124]: osdmap e215: 12 total, 12 up, 12 in 2023-09-10T00:26:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4000.087297 secs 2023-09-10T00:26:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4000.800133 secs 2023-09-10T00:26:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4000.800133 secs 2023-09-10T00:26:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4000.800133 secs 2023-09-10T00:26:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:10 smithi170 ceph-mon[102832]: pgmap v2301: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 108 KiB/s wr, 29 op/s 2023-09-10T00:26:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:10 smithi082 ceph-mon[98694]: pgmap v2301: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 108 KiB/s wr, 29 op/s 2023-09-10T00:26:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:10 smithi098 ceph-mon[103124]: pgmap v2301: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 108 KiB/s wr, 29 op/s 2023-09-10T00:26:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:12 smithi170 ceph-mon[102832]: pgmap v2302: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 104 KiB/s wr, 29 op/s 2023-09-10T00:26:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4003.519946 secs 2023-09-10T00:26:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:12 smithi082 ceph-mon[98694]: pgmap v2302: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 104 KiB/s wr, 29 op/s 2023-09-10T00:26:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4003.519946 secs 2023-09-10T00:26:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:12 smithi098 ceph-mon[103124]: pgmap v2302: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 104 KiB/s wr, 29 op/s 2023-09-10T00:26:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4003.519946 secs 2023-09-10T00:26:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4005.087366 secs 2023-09-10T00:26:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4005.087366 secs 2023-09-10T00:26:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4005.087366 secs 2023-09-10T00:26:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:14 smithi170 ceph-mon[102832]: pgmap v2303: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 104 KiB/s wr, 29 op/s 2023-09-10T00:26:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4005.800197 secs 2023-09-10T00:26:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:14 smithi170 ceph-mon[102832]: osdmap e216: 12 total, 12 up, 12 in 2023-09-10T00:26:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:14 smithi082 ceph-mon[98694]: pgmap v2303: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 104 KiB/s wr, 29 op/s 2023-09-10T00:26:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4005.800197 secs 2023-09-10T00:26:14.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:14 smithi082 ceph-mon[98694]: osdmap e216: 12 total, 12 up, 12 in 2023-09-10T00:26:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:14 smithi098 ceph-mon[103124]: pgmap v2303: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 104 KiB/s wr, 29 op/s 2023-09-10T00:26:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4005.800197 secs 2023-09-10T00:26:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:14 smithi098 ceph-mon[103124]: osdmap e216: 12 total, 12 up, 12 in 2023-09-10T00:26:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:16 smithi170 ceph-mon[102832]: pgmap v2305: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-09-10T00:26:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:16 smithi082 ceph-mon[98694]: pgmap v2305: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-09-10T00:26:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:16 smithi098 ceph-mon[103124]: pgmap v2305: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-09-10T00:26:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:18 smithi170 ceph-mon[102832]: pgmap v2306: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-09-10T00:26:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4010.087423 secs 2023-09-10T00:26:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:18 smithi082 ceph-mon[98694]: pgmap v2306: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-09-10T00:26:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4010.087423 secs 2023-09-10T00:26:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:18 smithi098 ceph-mon[103124]: pgmap v2306: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-09-10T00:26:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4010.087423 secs 2023-09-10T00:26:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4010.800259 secs 2023-09-10T00:26:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:26:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4010.800259 secs 2023-09-10T00:26:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:26:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4010.800259 secs 2023-09-10T00:26:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:26:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:20 smithi170 ceph-mon[102832]: pgmap v2307: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-09-10T00:26:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:26:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:26:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:26:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:20 smithi082 ceph-mon[98694]: pgmap v2307: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-09-10T00:26:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:26:20.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:26:20.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:26:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:20 smithi098 ceph-mon[103124]: pgmap v2307: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-09-10T00:26:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:26:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:26:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:26:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:22 smithi170 ceph-mon[102832]: pgmap v2308: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T00:26:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:22 smithi082 ceph-mon[98694]: pgmap v2308: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T00:26:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:22 smithi098 ceph-mon[103124]: pgmap v2308: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T00:26:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4015.087480 secs 2023-09-10T00:26:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4015.087480 secs 2023-09-10T00:26:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4015.087480 secs 2023-09-10T00:26:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:24 smithi170 ceph-mon[102832]: pgmap v2309: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T00:26:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4015.800325 secs 2023-09-10T00:26:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:24 smithi082 ceph-mon[98694]: pgmap v2309: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T00:26:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4015.800325 secs 2023-09-10T00:26:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:24 smithi098 ceph-mon[103124]: pgmap v2309: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T00:26:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4015.800325 secs 2023-09-10T00:26:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:26 smithi170 ceph-mon[102832]: pgmap v2310: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 119 KiB/s wr, 18 op/s 2023-09-10T00:26:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:26 smithi082 ceph-mon[98694]: pgmap v2310: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 119 KiB/s wr, 18 op/s 2023-09-10T00:26:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:26 smithi098 ceph-mon[103124]: pgmap v2310: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 119 KiB/s wr, 18 op/s 2023-09-10T00:26:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:28 smithi170 ceph-mon[102832]: pgmap v2311: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:26:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4020.087496 secs 2023-09-10T00:26:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:28 smithi082 ceph-mon[98694]: pgmap v2311: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:26:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4020.087496 secs 2023-09-10T00:26:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:28 smithi098 ceph-mon[103124]: pgmap v2311: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:26:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4020.087496 secs 2023-09-10T00:26:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4020.800390 secs 2023-09-10T00:26:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4020.800390 secs 2023-09-10T00:26:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4020.800390 secs 2023-09-10T00:26:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:30 smithi170 ceph-mon[102832]: pgmap v2312: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:26:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:30 smithi082 ceph-mon[98694]: pgmap v2312: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:26:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:30 smithi098 ceph-mon[103124]: pgmap v2312: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:26:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:32 smithi170 ceph-mon[102832]: pgmap v2313: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 195 KiB/s wr, 17 op/s 2023-09-10T00:26:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:32 smithi082 ceph-mon[98694]: pgmap v2313: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 195 KiB/s wr, 17 op/s 2023-09-10T00:26:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:32 smithi098 ceph-mon[103124]: pgmap v2313: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 195 KiB/s wr, 17 op/s 2023-09-10T00:26:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4025.087554 secs 2023-09-10T00:26:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4025.087554 secs 2023-09-10T00:26:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4025.087554 secs 2023-09-10T00:26:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:34 smithi170 ceph-mon[102832]: pgmap v2314: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 137 KiB/s wr, 11 op/s 2023-09-10T00:26:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4025.800457 secs 2023-09-10T00:26:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:34 smithi082 ceph-mon[98694]: pgmap v2314: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 137 KiB/s wr, 11 op/s 2023-09-10T00:26:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4025.800457 secs 2023-09-10T00:26:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:34 smithi098 ceph-mon[103124]: pgmap v2314: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 137 KiB/s wr, 11 op/s 2023-09-10T00:26:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4025.800457 secs 2023-09-10T00:26:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:36 smithi170 ceph-mon[102832]: pgmap v2315: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 163 KiB/s wr, 17 op/s 2023-09-10T00:26:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:36 smithi082 ceph-mon[98694]: pgmap v2315: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 163 KiB/s wr, 17 op/s 2023-09-10T00:26:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:36 smithi098 ceph-mon[103124]: pgmap v2315: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 163 KiB/s wr, 17 op/s 2023-09-10T00:26:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:38 smithi170 ceph-mon[102832]: pgmap v2316: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T00:26:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4030.087611 secs 2023-09-10T00:26:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:38 smithi082 ceph-mon[98694]: pgmap v2316: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T00:26:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4030.087611 secs 2023-09-10T00:26:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:38 smithi098 ceph-mon[103124]: pgmap v2316: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T00:26:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4030.087611 secs 2023-09-10T00:26:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4030.800491 secs 2023-09-10T00:26:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4030.800491 secs 2023-09-10T00:26:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4030.800491 secs 2023-09-10T00:26:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:40 smithi170 ceph-mon[102832]: pgmap v2317: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T00:26:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:40 smithi082 ceph-mon[98694]: pgmap v2317: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T00:26:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:40 smithi098 ceph-mon[103124]: pgmap v2317: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T00:26:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:42 smithi170 ceph-mon[102832]: pgmap v2318: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 169 KiB/s wr, 17 op/s 2023-09-10T00:26:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:42 smithi082 ceph-mon[98694]: pgmap v2318: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 169 KiB/s wr, 17 op/s 2023-09-10T00:26:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:42 smithi098 ceph-mon[103124]: pgmap v2318: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 169 KiB/s wr, 17 op/s 2023-09-10T00:26:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4035.087670 secs 2023-09-10T00:26:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4035.087670 secs 2023-09-10T00:26:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4035.087670 secs 2023-09-10T00:26:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:44 smithi170 ceph-mon[102832]: pgmap v2319: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T00:26:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4035.800554 secs 2023-09-10T00:26:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:44 smithi082 ceph-mon[98694]: pgmap v2319: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T00:26:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4035.800554 secs 2023-09-10T00:26:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:44 smithi098 ceph-mon[103124]: pgmap v2319: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T00:26:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4035.800554 secs 2023-09-10T00:26:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:46 smithi170 ceph-mon[102832]: pgmap v2320: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 114 KiB/s wr, 17 op/s 2023-09-10T00:26:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:46 smithi082 ceph-mon[98694]: pgmap v2320: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 114 KiB/s wr, 17 op/s 2023-09-10T00:26:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:46 smithi098 ceph-mon[103124]: pgmap v2320: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 114 KiB/s wr, 17 op/s 2023-09-10T00:26:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4038.520457 secs 2023-09-10T00:26:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4038.520457 secs 2023-09-10T00:26:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4038.520457 secs 2023-09-10T00:26:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:48 smithi170 ceph-mon[102832]: pgmap v2321: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:26:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4040.087727 secs 2023-09-10T00:26:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:48 smithi082 ceph-mon[98694]: pgmap v2321: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:26:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4040.087727 secs 2023-09-10T00:26:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:48 smithi098 ceph-mon[103124]: pgmap v2321: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:26:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4040.087727 secs 2023-09-10T00:26:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4040.800622 secs 2023-09-10T00:26:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4040.800622 secs 2023-09-10T00:26:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4040.800622 secs 2023-09-10T00:26:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:50 smithi170 ceph-mon[102832]: pgmap v2322: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:26:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:50 smithi082 ceph-mon[98694]: pgmap v2322: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:26:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:50 smithi098 ceph-mon[103124]: pgmap v2322: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:26:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:26:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:26:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:26:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:26:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:26:51.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:26:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:26:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:26:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:26:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:52 smithi170 ceph-mon[102832]: pgmap v2323: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 16 op/s 2023-09-10T00:26:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4043.520515 secs 2023-09-10T00:26:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:52 smithi082 ceph-mon[98694]: pgmap v2323: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 16 op/s 2023-09-10T00:26:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4043.520515 secs 2023-09-10T00:26:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:52 smithi098 ceph-mon[103124]: pgmap v2323: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 16 op/s 2023-09-10T00:26:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4043.520515 secs 2023-09-10T00:26:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4045.087785 secs 2023-09-10T00:26:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4045.087785 secs 2023-09-10T00:26:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4045.087785 secs 2023-09-10T00:26:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:54 smithi170 ceph-mon[102832]: pgmap v2324: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T00:26:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4045.800687 secs 2023-09-10T00:26:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:26:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:26:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:26:54.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:26:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:54 smithi082 ceph-mon[98694]: pgmap v2324: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T00:26:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4045.800687 secs 2023-09-10T00:26:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:26:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:26:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:26:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:26:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:54 smithi098 ceph-mon[103124]: pgmap v2324: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T00:26:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4045.800687 secs 2023-09-10T00:26:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:26:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:26:55.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:26:55.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:26:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:56 smithi170 ceph-mon[102832]: pgmap v2325: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 15 op/s 2023-09-10T00:26:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:56 smithi082 ceph-mon[98694]: pgmap v2325: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 15 op/s 2023-09-10T00:26:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:56 smithi098 ceph-mon[103124]: pgmap v2325: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 15 op/s 2023-09-10T00:26:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4048.520576 secs 2023-09-10T00:26:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4048.520576 secs 2023-09-10T00:26:58.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4048.520576 secs 2023-09-10T00:26:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:58 smithi170 ceph-mon[102832]: pgmap v2326: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 41 KiB/s wr, 10 op/s 2023-09-10T00:26:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4050.087843 secs 2023-09-10T00:26:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:58 smithi082 ceph-mon[98694]: pgmap v2326: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 41 KiB/s wr, 10 op/s 2023-09-10T00:26:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4050.087843 secs 2023-09-10T00:26:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:58 smithi098 ceph-mon[103124]: pgmap v2326: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 41 KiB/s wr, 10 op/s 2023-09-10T00:26:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4050.087843 secs 2023-09-10T00:26:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:26:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4050.800753 secs 2023-09-10T00:26:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:26:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4050.800753 secs 2023-09-10T00:27:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:26:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4050.800753 secs 2023-09-10T00:27:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:00 smithi170 ceph-mon[102832]: pgmap v2327: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 41 KiB/s wr, 10 op/s 2023-09-10T00:27:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:00 smithi082 ceph-mon[98694]: pgmap v2327: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 41 KiB/s wr, 10 op/s 2023-09-10T00:27:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:00 smithi098 ceph-mon[103124]: pgmap v2327: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 41 KiB/s wr, 10 op/s 2023-09-10T00:27:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:02 smithi170 ceph-mon[102832]: pgmap v2328: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 42 KiB/s wr, 15 op/s 2023-09-10T00:27:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4053.520672 secs 2023-09-10T00:27:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:02 smithi082 ceph-mon[98694]: pgmap v2328: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 42 KiB/s wr, 15 op/s 2023-09-10T00:27:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4053.520672 secs 2023-09-10T00:27:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:02 smithi098 ceph-mon[103124]: pgmap v2328: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 42 KiB/s wr, 15 op/s 2023-09-10T00:27:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4053.520672 secs 2023-09-10T00:27:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4055.087900 secs 2023-09-10T00:27:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4055.087900 secs 2023-09-10T00:27:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4055.087900 secs 2023-09-10T00:27:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:04 smithi170 ceph-mon[102832]: pgmap v2329: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 10 op/s 2023-09-10T00:27:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4055.800821 secs 2023-09-10T00:27:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:04 smithi082 ceph-mon[98694]: pgmap v2329: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 10 op/s 2023-09-10T00:27:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4055.800821 secs 2023-09-10T00:27:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:04 smithi098 ceph-mon[103124]: pgmap v2329: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 10 op/s 2023-09-10T00:27:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4055.800821 secs 2023-09-10T00:27:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:06 smithi082 ceph-mon[98694]: pgmap v2330: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T00:27:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:06 smithi098 ceph-mon[103124]: pgmap v2330: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T00:27:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:06 smithi170 ceph-mon[102832]: pgmap v2330: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T00:27:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4058.520729 secs 2023-09-10T00:27:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4058.520729 secs 2023-09-10T00:27:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4058.520729 secs 2023-09-10T00:27:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:08 smithi082 ceph-mon[98694]: pgmap v2331: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 69 KiB/s wr, 24 op/s 2023-09-10T00:27:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:08 smithi082 ceph-mon[98694]: osdmap e217: 12 total, 12 up, 12 in 2023-09-10T00:27:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4060.087961 secs 2023-09-10T00:27:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:08 smithi098 ceph-mon[103124]: pgmap v2331: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 69 KiB/s wr, 24 op/s 2023-09-10T00:27:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:08 smithi098 ceph-mon[103124]: osdmap e217: 12 total, 12 up, 12 in 2023-09-10T00:27:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4060.087961 secs 2023-09-10T00:27:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:08 smithi170 ceph-mon[102832]: pgmap v2331: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 69 KiB/s wr, 24 op/s 2023-09-10T00:27:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:08 smithi170 ceph-mon[102832]: osdmap e217: 12 total, 12 up, 12 in 2023-09-10T00:27:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4060.087961 secs 2023-09-10T00:27:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4060.800889 secs 2023-09-10T00:27:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4060.800889 secs 2023-09-10T00:27:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4060.800889 secs 2023-09-10T00:27:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:10 smithi082 ceph-mon[98694]: pgmap v2333: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T00:27:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:10 smithi098 ceph-mon[103124]: pgmap v2333: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T00:27:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:10 smithi170 ceph-mon[102832]: pgmap v2333: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T00:27:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:12 smithi082 ceph-mon[98694]: pgmap v2334: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T00:27:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4063.520836 secs 2023-09-10T00:27:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:12 smithi098 ceph-mon[103124]: pgmap v2334: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T00:27:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4063.520836 secs 2023-09-10T00:27:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:12 smithi170 ceph-mon[102832]: pgmap v2334: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T00:27:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4063.520836 secs 2023-09-10T00:27:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4065.088018 secs 2023-09-10T00:27:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4065.088018 secs 2023-09-10T00:27:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4065.088018 secs 2023-09-10T00:27:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:14 smithi082 ceph-mon[98694]: pgmap v2335: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T00:27:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4065.800957 secs 2023-09-10T00:27:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:14 smithi082 ceph-mon[98694]: osdmap e218: 12 total, 12 up, 12 in 2023-09-10T00:27:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:14 smithi098 ceph-mon[103124]: pgmap v2335: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T00:27:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4065.800957 secs 2023-09-10T00:27:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:14 smithi098 ceph-mon[103124]: osdmap e218: 12 total, 12 up, 12 in 2023-09-10T00:27:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:14 smithi170 ceph-mon[102832]: pgmap v2335: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T00:27:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4065.800957 secs 2023-09-10T00:27:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:14 smithi170 ceph-mon[102832]: osdmap e218: 12 total, 12 up, 12 in 2023-09-10T00:27:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:16 smithi082 ceph-mon[98694]: pgmap v2337: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 14 op/s 2023-09-10T00:27:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:16 smithi098 ceph-mon[103124]: pgmap v2337: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 14 op/s 2023-09-10T00:27:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:16 smithi170 ceph-mon[102832]: pgmap v2337: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 43 KiB/s wr, 14 op/s 2023-09-10T00:27:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:18 smithi098 ceph-mon[103124]: pgmap v2338: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T00:27:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4070.088091 secs 2023-09-10T00:27:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:18 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:27:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:18 smithi170 ceph-mon[102832]: pgmap v2338: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T00:27:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4070.088091 secs 2023-09-10T00:27:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:18 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:27:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:18 smithi082 ceph-mon[98694]: pgmap v2338: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T00:27:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4070.088091 secs 2023-09-10T00:27:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:18 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:27:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4070.801022 secs 2023-09-10T00:27:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:19 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:27:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:19 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:27:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:19 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:27:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4070.801022 secs 2023-09-10T00:27:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:19 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:27:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:19 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:27:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:19 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:27:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4070.801022 secs 2023-09-10T00:27:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:19 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:27:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:19 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:27:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:19 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:27:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:20 smithi098 ceph-mon[103124]: pgmap v2339: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T00:27:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:20 smithi170 ceph-mon[102832]: pgmap v2339: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T00:27:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:20 smithi082 ceph-mon[98694]: pgmap v2339: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T00:27:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:22 smithi098 ceph-mon[103124]: pgmap v2340: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T00:27:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:22 smithi170 ceph-mon[102832]: pgmap v2340: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T00:27:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:22 smithi082 ceph-mon[98694]: pgmap v2340: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T00:27:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4075.088148 secs 2023-09-10T00:27:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4075.088148 secs 2023-09-10T00:27:24.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4075.088148 secs 2023-09-10T00:27:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:24 smithi098 ceph-mon[103124]: pgmap v2341: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T00:27:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4075.801090 secs 2023-09-10T00:27:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:24 smithi170 ceph-mon[102832]: pgmap v2341: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T00:27:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4075.801090 secs 2023-09-10T00:27:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:24 smithi082 ceph-mon[98694]: pgmap v2341: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T00:27:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4075.801090 secs 2023-09-10T00:27:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:26 smithi098 ceph-mon[103124]: pgmap v2342: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 92 KiB/s wr, 16 op/s 2023-09-10T00:27:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:26 smithi170 ceph-mon[102832]: pgmap v2342: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 92 KiB/s wr, 16 op/s 2023-09-10T00:27:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:26 smithi082 ceph-mon[98694]: pgmap v2342: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 92 KiB/s wr, 16 op/s 2023-09-10T00:27:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:28 smithi098 ceph-mon[103124]: pgmap v2343: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T00:27:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4080.088206 secs 2023-09-10T00:27:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:28 smithi170 ceph-mon[102832]: pgmap v2343: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T00:27:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4080.088206 secs 2023-09-10T00:27:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:28 smithi082 ceph-mon[98694]: pgmap v2343: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T00:27:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4080.088206 secs 2023-09-10T00:27:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4080.801159 secs 2023-09-10T00:27:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4080.801159 secs 2023-09-10T00:27:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4080.801159 secs 2023-09-10T00:27:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:30 smithi098 ceph-mon[103124]: pgmap v2344: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T00:27:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:30 smithi170 ceph-mon[102832]: pgmap v2344: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T00:27:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:30 smithi082 ceph-mon[98694]: pgmap v2344: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T00:27:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:32 smithi098 ceph-mon[103124]: pgmap v2345: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 134 KiB/s wr, 16 op/s 2023-09-10T00:27:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:32 smithi170 ceph-mon[102832]: pgmap v2345: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 134 KiB/s wr, 16 op/s 2023-09-10T00:27:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:32 smithi082 ceph-mon[98694]: pgmap v2345: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 134 KiB/s wr, 16 op/s 2023-09-10T00:27:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4085.088265 secs 2023-09-10T00:27:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4085.088265 secs 2023-09-10T00:27:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4085.088265 secs 2023-09-10T00:27:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:34 smithi098 ceph-mon[103124]: pgmap v2346: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T00:27:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4085.801224 secs 2023-09-10T00:27:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:34 smithi170 ceph-mon[102832]: pgmap v2346: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T00:27:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4085.801224 secs 2023-09-10T00:27:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:34 smithi082 ceph-mon[98694]: pgmap v2346: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T00:27:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4085.801224 secs 2023-09-10T00:27:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:36 smithi098 ceph-mon[103124]: pgmap v2347: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 160 KiB/s wr, 17 op/s 2023-09-10T00:27:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:36 smithi170 ceph-mon[102832]: pgmap v2347: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 160 KiB/s wr, 17 op/s 2023-09-10T00:27:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:36 smithi082 ceph-mon[98694]: pgmap v2347: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 160 KiB/s wr, 17 op/s 2023-09-10T00:27:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:38 smithi098 ceph-mon[103124]: pgmap v2348: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T00:27:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4090.088322 secs 2023-09-10T00:27:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:38 smithi170 ceph-mon[102832]: pgmap v2348: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T00:27:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4090.088322 secs 2023-09-10T00:27:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:38 smithi082 ceph-mon[98694]: pgmap v2348: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T00:27:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4090.088322 secs 2023-09-10T00:27:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4090.801292 secs 2023-09-10T00:27:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4090.801292 secs 2023-09-10T00:27:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4090.801292 secs 2023-09-10T00:27:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:40 smithi170 ceph-mon[102832]: pgmap v2349: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T00:27:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:40 smithi082 ceph-mon[98694]: pgmap v2349: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T00:27:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:40 smithi098 ceph-mon[103124]: pgmap v2349: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T00:27:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:42 smithi170 ceph-mon[102832]: pgmap v2350: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 184 KiB/s wr, 18 op/s 2023-09-10T00:27:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:42 smithi082 ceph-mon[98694]: pgmap v2350: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 184 KiB/s wr, 18 op/s 2023-09-10T00:27:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:42 smithi098 ceph-mon[103124]: pgmap v2350: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 184 KiB/s wr, 18 op/s 2023-09-10T00:27:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4095.088378 secs 2023-09-10T00:27:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4095.088378 secs 2023-09-10T00:27:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4095.088378 secs 2023-09-10T00:27:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:44 smithi170 ceph-mon[102832]: pgmap v2351: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T00:27:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4095.801356 secs 2023-09-10T00:27:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:44 smithi082 ceph-mon[98694]: pgmap v2351: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T00:27:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4095.801356 secs 2023-09-10T00:27:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:44 smithi098 ceph-mon[103124]: pgmap v2351: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T00:27:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4095.801356 secs 2023-09-10T00:27:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:46 smithi170 ceph-mon[102832]: pgmap v2352: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 160 KiB/s wr, 18 op/s 2023-09-10T00:27:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:46 smithi082 ceph-mon[98694]: pgmap v2352: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 160 KiB/s wr, 18 op/s 2023-09-10T00:27:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:46 smithi098 ceph-mon[103124]: pgmap v2352: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 160 KiB/s wr, 18 op/s 2023-09-10T00:27:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4098.521303 secs 2023-09-10T00:27:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4098.521303 secs 2023-09-10T00:27:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4098.521303 secs 2023-09-10T00:27:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:48 smithi170 ceph-mon[102832]: pgmap v2353: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T00:27:49.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4100.088469 secs 2023-09-10T00:27:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:48 smithi082 ceph-mon[98694]: pgmap v2353: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T00:27:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4100.088469 secs 2023-09-10T00:27:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:48 smithi098 ceph-mon[103124]: pgmap v2353: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T00:27:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4100.088469 secs 2023-09-10T00:27:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4100.801422 secs 2023-09-10T00:27:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4100.801422 secs 2023-09-10T00:27:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4100.801422 secs 2023-09-10T00:27:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:50 smithi170 ceph-mon[102832]: pgmap v2354: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T00:27:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:50 smithi082 ceph-mon[98694]: pgmap v2354: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T00:27:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:50 smithi098 ceph-mon[103124]: pgmap v2354: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T00:27:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:27:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:27:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:27:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:27:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:27:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:27:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:27:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:27:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:27:53.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:52 smithi170 ceph-mon[102832]: pgmap v2355: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 159 KiB/s wr, 18 op/s 2023-09-10T00:27:53.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4103.521404 secs 2023-09-10T00:27:53.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:52 smithi082 ceph-mon[98694]: pgmap v2355: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 159 KiB/s wr, 18 op/s 2023-09-10T00:27:53.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4103.521404 secs 2023-09-10T00:27:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:52 smithi098 ceph-mon[103124]: pgmap v2355: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 159 KiB/s wr, 18 op/s 2023-09-10T00:27:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4103.521404 secs 2023-09-10T00:27:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4105.088493 secs 2023-09-10T00:27:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4105.088493 secs 2023-09-10T00:27:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4105.088493 secs 2023-09-10T00:27:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:54 smithi170 ceph-mon[102832]: pgmap v2356: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T00:27:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4105.801494 secs 2023-09-10T00:27:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:27:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:54 smithi082 ceph-mon[98694]: pgmap v2356: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T00:27:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4105.801494 secs 2023-09-10T00:27:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:27:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:54 smithi098 ceph-mon[103124]: pgmap v2356: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T00:27:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4105.801494 secs 2023-09-10T00:27:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:27:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:27:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:27:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:27:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:27:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:27:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:27:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:27:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:27:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:27:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:27:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:27:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:27:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:27:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:27:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:27:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:57 smithi170 ceph-mon[102832]: pgmap v2357: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 110 KiB/s wr, 17 op/s 2023-09-10T00:27:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:57 smithi082 ceph-mon[98694]: pgmap v2357: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 110 KiB/s wr, 17 op/s 2023-09-10T00:27:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:57 smithi098 ceph-mon[103124]: pgmap v2357: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 110 KiB/s wr, 17 op/s 2023-09-10T00:27:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4108.521466 secs 2023-09-10T00:27:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4108.521466 secs 2023-09-10T00:27:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4108.521466 secs 2023-09-10T00:27:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:59 smithi170 ceph-mon[102832]: pgmap v2358: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T00:27:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:27:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4110.088550 secs 2023-09-10T00:27:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:59 smithi082 ceph-mon[98694]: pgmap v2358: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T00:27:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:27:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4110.088550 secs 2023-09-10T00:27:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:59 smithi098 ceph-mon[103124]: pgmap v2358: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T00:27:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:27:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4110.088550 secs 2023-09-10T00:28:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4110.801561 secs 2023-09-10T00:28:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4110.801561 secs 2023-09-10T00:28:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4110.801561 secs 2023-09-10T00:28:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:01 smithi170 ceph-mon[102832]: pgmap v2359: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T00:28:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:01 smithi082 ceph-mon[98694]: pgmap v2359: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T00:28:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:01 smithi098 ceph-mon[103124]: pgmap v2359: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T00:28:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:03 smithi170 ceph-mon[102832]: pgmap v2360: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 57 KiB/s wr, 15 op/s 2023-09-10T00:28:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4113.521523 secs 2023-09-10T00:28:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:03 smithi082 ceph-mon[98694]: pgmap v2360: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 57 KiB/s wr, 15 op/s 2023-09-10T00:28:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4113.521523 secs 2023-09-10T00:28:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:03 smithi098 ceph-mon[103124]: pgmap v2360: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 57 KiB/s wr, 15 op/s 2023-09-10T00:28:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4113.521523 secs 2023-09-10T00:28:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4115.088609 secs 2023-09-10T00:28:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4115.088609 secs 2023-09-10T00:28:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4115.088609 secs 2023-09-10T00:28:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:05 smithi170 ceph-mon[102832]: pgmap v2361: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 24 KiB/s wr, 9 op/s 2023-09-10T00:28:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4115.801626 secs 2023-09-10T00:28:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:05 smithi082 ceph-mon[98694]: pgmap v2361: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 24 KiB/s wr, 9 op/s 2023-09-10T00:28:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4115.801626 secs 2023-09-10T00:28:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:05 smithi098 ceph-mon[103124]: pgmap v2361: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 24 KiB/s wr, 9 op/s 2023-09-10T00:28:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4115.801626 secs 2023-09-10T00:28:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:07 smithi170 ceph-mon[102832]: pgmap v2362: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 111 KiB/s wr, 29 op/s 2023-09-10T00:28:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:07 smithi082 ceph-mon[98694]: pgmap v2362: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 111 KiB/s wr, 29 op/s 2023-09-10T00:28:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:07 smithi098 ceph-mon[103124]: pgmap v2362: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 111 KiB/s wr, 29 op/s 2023-09-10T00:28:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4118.521585 secs 2023-09-10T00:28:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:08 smithi170 ceph-mon[102832]: osdmap e219: 12 total, 12 up, 12 in 2023-09-10T00:28:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4118.521585 secs 2023-09-10T00:28:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:08 smithi082 ceph-mon[98694]: osdmap e219: 12 total, 12 up, 12 in 2023-09-10T00:28:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4118.521585 secs 2023-09-10T00:28:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:08 smithi098 ceph-mon[103124]: osdmap e219: 12 total, 12 up, 12 in 2023-09-10T00:28:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:09 smithi170 ceph-mon[102832]: pgmap v2364: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T00:28:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4120.088667 secs 2023-09-10T00:28:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:09 smithi082 ceph-mon[98694]: pgmap v2364: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T00:28:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4120.088667 secs 2023-09-10T00:28:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:09 smithi098 ceph-mon[103124]: pgmap v2364: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T00:28:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4120.088667 secs 2023-09-10T00:28:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4120.801693 secs 2023-09-10T00:28:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4120.801693 secs 2023-09-10T00:28:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4120.801693 secs 2023-09-10T00:28:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:11 smithi170 ceph-mon[102832]: pgmap v2365: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T00:28:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:11 smithi082 ceph-mon[98694]: pgmap v2365: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T00:28:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:11 smithi098 ceph-mon[103124]: pgmap v2365: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T00:28:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:13 smithi170 ceph-mon[102832]: pgmap v2366: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T00:28:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4123.521645 secs 2023-09-10T00:28:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:13 smithi082 ceph-mon[98694]: pgmap v2366: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T00:28:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4123.521645 secs 2023-09-10T00:28:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:13 smithi098 ceph-mon[103124]: pgmap v2366: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T00:28:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4123.521645 secs 2023-09-10T00:28:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4125.088726 secs 2023-09-10T00:28:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4125.088726 secs 2023-09-10T00:28:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4125.088726 secs 2023-09-10T00:28:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:15 smithi170 ceph-mon[102832]: pgmap v2367: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T00:28:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4125.801759 secs 2023-09-10T00:28:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:15 smithi170 ceph-mon[102832]: osdmap e220: 12 total, 12 up, 12 in 2023-09-10T00:28:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:15 smithi082 ceph-mon[98694]: pgmap v2367: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T00:28:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4125.801759 secs 2023-09-10T00:28:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:15 smithi082 ceph-mon[98694]: osdmap e220: 12 total, 12 up, 12 in 2023-09-10T00:28:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:15 smithi098 ceph-mon[103124]: pgmap v2367: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T00:28:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4125.801759 secs 2023-09-10T00:28:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:15 smithi098 ceph-mon[103124]: osdmap e220: 12 total, 12 up, 12 in 2023-09-10T00:28:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:17 smithi170 ceph-mon[102832]: pgmap v2369: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 32 KiB/s wr, 13 op/s 2023-09-10T00:28:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:17 smithi082 ceph-mon[98694]: pgmap v2369: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 32 KiB/s wr, 13 op/s 2023-09-10T00:28:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:17 smithi098 ceph-mon[103124]: pgmap v2369: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 32 KiB/s wr, 13 op/s 2023-09-10T00:28:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:19 smithi170 ceph-mon[102832]: pgmap v2370: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-09-10T00:28:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4130.088787 secs 2023-09-10T00:28:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:28:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:19 smithi082 ceph-mon[98694]: pgmap v2370: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-09-10T00:28:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4130.088787 secs 2023-09-10T00:28:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:28:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:19 smithi098 ceph-mon[103124]: pgmap v2370: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-09-10T00:28:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4130.088787 secs 2023-09-10T00:28:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:28:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4130.801832 secs 2023-09-10T00:28:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:28:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:28:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:28:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4130.801832 secs 2023-09-10T00:28:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:28:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:28:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:28:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4130.801832 secs 2023-09-10T00:28:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:28:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:28:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:28:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:21 smithi170 ceph-mon[102832]: pgmap v2371: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-09-10T00:28:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:21 smithi082 ceph-mon[98694]: pgmap v2371: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-09-10T00:28:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:21 smithi098 ceph-mon[103124]: pgmap v2371: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-09-10T00:28:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:23 smithi170 ceph-mon[102832]: pgmap v2372: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T00:28:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:23 smithi082 ceph-mon[98694]: pgmap v2372: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T00:28:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:23 smithi098 ceph-mon[103124]: pgmap v2372: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T00:28:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4135.088846 secs 2023-09-10T00:28:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4135.088846 secs 2023-09-10T00:28:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4135.088846 secs 2023-09-10T00:28:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:25 smithi170 ceph-mon[102832]: pgmap v2373: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T00:28:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4135.801896 secs 2023-09-10T00:28:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:25 smithi082 ceph-mon[98694]: pgmap v2373: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T00:28:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4135.801896 secs 2023-09-10T00:28:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:25 smithi098 ceph-mon[103124]: pgmap v2373: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T00:28:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4135.801896 secs 2023-09-10T00:28:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:26 smithi098 ceph-mon[103124]: pgmap v2374: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 122 KiB/s wr, 17 op/s 2023-09-10T00:28:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:26 smithi170 ceph-mon[102832]: pgmap v2374: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 122 KiB/s wr, 17 op/s 2023-09-10T00:28:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:26 smithi082 ceph-mon[98694]: pgmap v2374: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 122 KiB/s wr, 17 op/s 2023-09-10T00:28:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:28 smithi170 ceph-mon[102832]: pgmap v2375: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T00:28:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4140.088905 secs 2023-09-10T00:28:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:28 smithi082 ceph-mon[98694]: pgmap v2375: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T00:28:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4140.088905 secs 2023-09-10T00:28:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:28 smithi098 ceph-mon[103124]: pgmap v2375: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T00:28:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4140.088905 secs 2023-09-10T00:28:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4140.801962 secs 2023-09-10T00:28:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4140.801962 secs 2023-09-10T00:28:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4140.801962 secs 2023-09-10T00:28:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:30 smithi170 ceph-mon[102832]: pgmap v2376: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T00:28:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:30 smithi082 ceph-mon[98694]: pgmap v2376: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T00:28:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:30 smithi098 ceph-mon[103124]: pgmap v2376: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T00:28:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:32 smithi170 ceph-mon[102832]: pgmap v2377: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 200 KiB/s wr, 17 op/s 2023-09-10T00:28:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:32 smithi082 ceph-mon[98694]: pgmap v2377: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 200 KiB/s wr, 17 op/s 2023-09-10T00:28:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:32 smithi098 ceph-mon[103124]: pgmap v2377: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 200 KiB/s wr, 17 op/s 2023-09-10T00:28:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4145.088963 secs 2023-09-10T00:28:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4145.088963 secs 2023-09-10T00:28:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4145.088963 secs 2023-09-10T00:28:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:34 smithi170 ceph-mon[102832]: pgmap v2378: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 140 KiB/s wr, 11 op/s 2023-09-10T00:28:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4145.802027 secs 2023-09-10T00:28:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:34 smithi082 ceph-mon[98694]: pgmap v2378: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 140 KiB/s wr, 11 op/s 2023-09-10T00:28:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4145.802027 secs 2023-09-10T00:28:35.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:34 smithi098 ceph-mon[103124]: pgmap v2378: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 140 KiB/s wr, 11 op/s 2023-09-10T00:28:35.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4145.802027 secs 2023-09-10T00:28:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:36 smithi170 ceph-mon[102832]: pgmap v2379: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 167 KiB/s wr, 18 op/s 2023-09-10T00:28:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:36 smithi082 ceph-mon[98694]: pgmap v2379: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 167 KiB/s wr, 18 op/s 2023-09-10T00:28:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:36 smithi098 ceph-mon[103124]: pgmap v2379: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 167 KiB/s wr, 18 op/s 2023-09-10T00:28:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:38 smithi170 ceph-mon[102832]: pgmap v2380: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T00:28:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4150.089047 secs 2023-09-10T00:28:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:38 smithi082 ceph-mon[98694]: pgmap v2380: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T00:28:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4150.089047 secs 2023-09-10T00:28:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:38 smithi098 ceph-mon[103124]: pgmap v2380: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T00:28:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4150.089047 secs 2023-09-10T00:28:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4150.802092 secs 2023-09-10T00:28:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4150.802092 secs 2023-09-10T00:28:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4150.802092 secs 2023-09-10T00:28:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:40 smithi170 ceph-mon[102832]: pgmap v2381: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T00:28:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:40 smithi082 ceph-mon[98694]: pgmap v2381: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T00:28:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:40 smithi098 ceph-mon[103124]: pgmap v2381: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T00:28:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:42 smithi170 ceph-mon[102832]: pgmap v2382: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 17 op/s 2023-09-10T00:28:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:42 smithi082 ceph-mon[98694]: pgmap v2382: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 17 op/s 2023-09-10T00:28:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:42 smithi098 ceph-mon[103124]: pgmap v2382: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 17 op/s 2023-09-10T00:28:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4155.089106 secs 2023-09-10T00:28:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4155.089106 secs 2023-09-10T00:28:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4155.089106 secs 2023-09-10T00:28:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:44 smithi170 ceph-mon[102832]: pgmap v2383: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T00:28:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4155.802160 secs 2023-09-10T00:28:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:44 smithi082 ceph-mon[98694]: pgmap v2383: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T00:28:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4155.802160 secs 2023-09-10T00:28:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:44 smithi098 ceph-mon[103124]: pgmap v2383: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T00:28:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4155.802160 secs 2023-09-10T00:28:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:46 smithi170 ceph-mon[102832]: pgmap v2384: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 126 KiB/s wr, 17 op/s 2023-09-10T00:28:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:46 smithi082 ceph-mon[98694]: pgmap v2384: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 126 KiB/s wr, 17 op/s 2023-09-10T00:28:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:46 smithi098 ceph-mon[103124]: pgmap v2384: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 126 KiB/s wr, 17 op/s 2023-09-10T00:28:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4158.522203 secs 2023-09-10T00:28:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4158.522203 secs 2023-09-10T00:28:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4158.522203 secs 2023-09-10T00:28:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:48 smithi170 ceph-mon[102832]: pgmap v2385: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T00:28:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4160.089164 secs 2023-09-10T00:28:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:48 smithi082 ceph-mon[98694]: pgmap v2385: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T00:28:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4160.089164 secs 2023-09-10T00:28:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:48 smithi098 ceph-mon[103124]: pgmap v2385: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T00:28:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4160.089164 secs 2023-09-10T00:28:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4160.802225 secs 2023-09-10T00:28:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4160.802225 secs 2023-09-10T00:28:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4160.802225 secs 2023-09-10T00:28:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:50 smithi170 ceph-mon[102832]: pgmap v2386: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T00:28:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:50 smithi082 ceph-mon[98694]: pgmap v2386: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T00:28:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:50 smithi098 ceph-mon[103124]: pgmap v2386: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T00:28:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:28:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:28:51.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:28:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:28:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:28:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:28:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:28:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:28:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:28:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:52 smithi082 ceph-mon[98694]: pgmap v2387: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 122 KiB/s wr, 16 op/s 2023-09-10T00:28:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4163.522262 secs 2023-09-10T00:28:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:52 smithi098 ceph-mon[103124]: pgmap v2387: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 122 KiB/s wr, 16 op/s 2023-09-10T00:28:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4163.522262 secs 2023-09-10T00:28:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:52 smithi170 ceph-mon[102832]: pgmap v2387: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 122 KiB/s wr, 16 op/s 2023-09-10T00:28:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4163.522262 secs 2023-09-10T00:28:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4165.089221 secs 2023-09-10T00:28:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4165.089221 secs 2023-09-10T00:28:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4165.089221 secs 2023-09-10T00:28:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:54 smithi082 ceph-mon[98694]: pgmap v2388: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T00:28:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4165.802291 secs 2023-09-10T00:28:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:54 smithi098 ceph-mon[103124]: pgmap v2388: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T00:28:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4165.802291 secs 2023-09-10T00:28:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:54 smithi170 ceph-mon[102832]: pgmap v2388: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T00:28:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4165.802291 secs 2023-09-10T00:28:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:56 smithi082 ceph-mon[98694]: pgmap v2389: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 15 op/s 2023-09-10T00:28:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:28:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:28:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:28:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:28:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:28:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:56 smithi098 ceph-mon[103124]: pgmap v2389: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 15 op/s 2023-09-10T00:28:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:28:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:28:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:28:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:28:57.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:28:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:56 smithi170 ceph-mon[102832]: pgmap v2389: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 15 op/s 2023-09-10T00:28:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:28:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:28:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:28:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:28:57.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:28:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:28:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:28:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:28:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4168.522322 secs 2023-09-10T00:28:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:28:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:28:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:28:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4168.522322 secs 2023-09-10T00:28:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:28:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:28:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:28:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4168.522322 secs 2023-09-10T00:28:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:58 smithi170 ceph-mon[102832]: pgmap v2390: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T00:28:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4170.089280 secs 2023-09-10T00:28:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:58 smithi082 ceph-mon[98694]: pgmap v2390: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T00:28:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4170.089280 secs 2023-09-10T00:28:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:58 smithi098 ceph-mon[103124]: pgmap v2390: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T00:28:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4170.089280 secs 2023-09-10T00:29:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:28:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4170.802357 secs 2023-09-10T00:29:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:28:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4170.802357 secs 2023-09-10T00:29:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:28:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4170.802357 secs 2023-09-10T00:29:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:00 smithi170 ceph-mon[102832]: pgmap v2391: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T00:29:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:00 smithi082 ceph-mon[98694]: pgmap v2391: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T00:29:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:00 smithi098 ceph-mon[103124]: pgmap v2391: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T00:29:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:02 smithi170 ceph-mon[102832]: pgmap v2392: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 44 KiB/s wr, 15 op/s 2023-09-10T00:29:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4173.522387 secs 2023-09-10T00:29:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:02 smithi082 ceph-mon[98694]: pgmap v2392: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 44 KiB/s wr, 15 op/s 2023-09-10T00:29:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4173.522387 secs 2023-09-10T00:29:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:02 smithi098 ceph-mon[103124]: pgmap v2392: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 44 KiB/s wr, 15 op/s 2023-09-10T00:29:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4173.522387 secs 2023-09-10T00:29:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4175.089338 secs 2023-09-10T00:29:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4175.089338 secs 2023-09-10T00:29:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4175.089338 secs 2023-09-10T00:29:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:04 smithi170 ceph-mon[102832]: pgmap v2393: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 21 KiB/s wr, 10 op/s 2023-09-10T00:29:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4175.802433 secs 2023-09-10T00:29:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:04 smithi082 ceph-mon[98694]: pgmap v2393: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 21 KiB/s wr, 10 op/s 2023-09-10T00:29:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4175.802433 secs 2023-09-10T00:29:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:04 smithi098 ceph-mon[103124]: pgmap v2393: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 21 KiB/s wr, 10 op/s 2023-09-10T00:29:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4175.802433 secs 2023-09-10T00:29:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:06 smithi170 ceph-mon[102832]: pgmap v2394: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T00:29:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:06 smithi082 ceph-mon[98694]: pgmap v2394: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T00:29:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:06 smithi098 ceph-mon[103124]: pgmap v2394: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T00:29:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4178.522446 secs 2023-09-10T00:29:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4178.522446 secs 2023-09-10T00:29:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4178.522446 secs 2023-09-10T00:29:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:08 smithi170 ceph-mon[102832]: pgmap v2395: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 82 KiB/s wr, 24 op/s 2023-09-10T00:29:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:08 smithi170 ceph-mon[102832]: osdmap e221: 12 total, 12 up, 12 in 2023-09-10T00:29:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4180.089398 secs 2023-09-10T00:29:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:08 smithi082 ceph-mon[98694]: pgmap v2395: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 82 KiB/s wr, 24 op/s 2023-09-10T00:29:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:08 smithi082 ceph-mon[98694]: osdmap e221: 12 total, 12 up, 12 in 2023-09-10T00:29:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4180.089398 secs 2023-09-10T00:29:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:08 smithi098 ceph-mon[103124]: pgmap v2395: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 82 KiB/s wr, 24 op/s 2023-09-10T00:29:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:08 smithi098 ceph-mon[103124]: osdmap e221: 12 total, 12 up, 12 in 2023-09-10T00:29:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4180.089398 secs 2023-09-10T00:29:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4180.802488 secs 2023-09-10T00:29:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4180.802488 secs 2023-09-10T00:29:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4180.802488 secs 2023-09-10T00:29:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:10 smithi170 ceph-mon[102832]: pgmap v2397: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T00:29:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:10 smithi082 ceph-mon[98694]: pgmap v2397: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T00:29:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:10 smithi098 ceph-mon[103124]: pgmap v2397: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T00:29:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:12 smithi170 ceph-mon[102832]: pgmap v2398: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 94 KiB/s wr, 29 op/s 2023-09-10T00:29:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4183.522509 secs 2023-09-10T00:29:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:12 smithi082 ceph-mon[98694]: pgmap v2398: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 94 KiB/s wr, 29 op/s 2023-09-10T00:29:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4183.522509 secs 2023-09-10T00:29:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:12 smithi098 ceph-mon[103124]: pgmap v2398: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 94 KiB/s wr, 29 op/s 2023-09-10T00:29:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4183.522509 secs 2023-09-10T00:29:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4185.089470 secs 2023-09-10T00:29:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4185.089470 secs 2023-09-10T00:29:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4185.089470 secs 2023-09-10T00:29:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:14 smithi170 ceph-mon[102832]: pgmap v2399: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 94 KiB/s wr, 29 op/s 2023-09-10T00:29:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4185.802553 secs 2023-09-10T00:29:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:14 smithi170 ceph-mon[102832]: osdmap e222: 12 total, 12 up, 12 in 2023-09-10T00:29:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:14 smithi082 ceph-mon[98694]: pgmap v2399: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 94 KiB/s wr, 29 op/s 2023-09-10T00:29:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4185.802553 secs 2023-09-10T00:29:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:14 smithi082 ceph-mon[98694]: osdmap e222: 12 total, 12 up, 12 in 2023-09-10T00:29:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:14 smithi098 ceph-mon[103124]: pgmap v2399: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 94 KiB/s wr, 29 op/s 2023-09-10T00:29:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4185.802553 secs 2023-09-10T00:29:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:14 smithi098 ceph-mon[103124]: osdmap e222: 12 total, 12 up, 12 in 2023-09-10T00:29:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:16 smithi170 ceph-mon[102832]: pgmap v2401: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 43 KiB/s wr, 14 op/s 2023-09-10T00:29:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:16 smithi082 ceph-mon[98694]: pgmap v2401: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 43 KiB/s wr, 14 op/s 2023-09-10T00:29:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:16 smithi098 ceph-mon[103124]: pgmap v2401: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 43 KiB/s wr, 14 op/s 2023-09-10T00:29:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4188.522615 secs 2023-09-10T00:29:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4188.522615 secs 2023-09-10T00:29:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4188.522615 secs 2023-09-10T00:29:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:18 smithi170 ceph-mon[102832]: pgmap v2402: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T00:29:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4190.089544 secs 2023-09-10T00:29:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:18 smithi082 ceph-mon[98694]: pgmap v2402: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T00:29:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4190.089544 secs 2023-09-10T00:29:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:18 smithi098 ceph-mon[103124]: pgmap v2402: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T00:29:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4190.089544 secs 2023-09-10T00:29:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4190.802617 secs 2023-09-10T00:29:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4190.802617 secs 2023-09-10T00:29:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4190.802617 secs 2023-09-10T00:29:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:20 smithi170 ceph-mon[102832]: pgmap v2403: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T00:29:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:20 smithi082 ceph-mon[98694]: pgmap v2403: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T00:29:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:20 smithi098 ceph-mon[103124]: pgmap v2403: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T00:29:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:22 smithi170 ceph-mon[102832]: pgmap v2404: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T00:29:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:29:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:22 smithi082 ceph-mon[98694]: pgmap v2404: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T00:29:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:29:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:22 smithi098 ceph-mon[103124]: pgmap v2404: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T00:29:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:29:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4195.089601 secs 2023-09-10T00:29:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:29:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:29:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:29:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4195.089601 secs 2023-09-10T00:29:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:29:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:29:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:29:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4195.089601 secs 2023-09-10T00:29:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:29:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:29:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:29:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:24 smithi170 ceph-mon[102832]: pgmap v2405: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T00:29:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4195.802683 secs 2023-09-10T00:29:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:24 smithi082 ceph-mon[98694]: pgmap v2405: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T00:29:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4195.802683 secs 2023-09-10T00:29:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:24 smithi098 ceph-mon[103124]: pgmap v2405: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T00:29:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4195.802683 secs 2023-09-10T00:29:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:26 smithi170 ceph-mon[102832]: pgmap v2406: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 101 KiB/s wr, 16 op/s 2023-09-10T00:29:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:26 smithi082 ceph-mon[98694]: pgmap v2406: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 101 KiB/s wr, 16 op/s 2023-09-10T00:29:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:26 smithi098 ceph-mon[103124]: pgmap v2406: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 101 KiB/s wr, 16 op/s 2023-09-10T00:29:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:28 smithi170 ceph-mon[102832]: pgmap v2407: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T00:29:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4200.089660 secs 2023-09-10T00:29:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:28 smithi082 ceph-mon[98694]: pgmap v2407: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T00:29:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4200.089660 secs 2023-09-10T00:29:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:28 smithi098 ceph-mon[103124]: pgmap v2407: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T00:29:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4200.089660 secs 2023-09-10T00:29:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4200.802746 secs 2023-09-10T00:29:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4200.802746 secs 2023-09-10T00:29:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4200.802746 secs 2023-09-10T00:29:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:30 smithi170 ceph-mon[102832]: pgmap v2408: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T00:29:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:30 smithi082 ceph-mon[98694]: pgmap v2408: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T00:29:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:30 smithi098 ceph-mon[103124]: pgmap v2408: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T00:29:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:32 smithi170 ceph-mon[102832]: pgmap v2409: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 155 KiB/s wr, 17 op/s 2023-09-10T00:29:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:32 smithi082 ceph-mon[98694]: pgmap v2409: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 155 KiB/s wr, 17 op/s 2023-09-10T00:29:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:32 smithi098 ceph-mon[103124]: pgmap v2409: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 155 KiB/s wr, 17 op/s 2023-09-10T00:29:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4205.089718 secs 2023-09-10T00:29:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4205.089718 secs 2023-09-10T00:29:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4205.089718 secs 2023-09-10T00:29:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:34 smithi170 ceph-mon[102832]: pgmap v2410: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T00:29:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4205.802812 secs 2023-09-10T00:29:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:34 smithi082 ceph-mon[98694]: pgmap v2410: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T00:29:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4205.802812 secs 2023-09-10T00:29:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:34 smithi098 ceph-mon[103124]: pgmap v2410: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T00:29:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4205.802812 secs 2023-09-10T00:29:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:36 smithi170 ceph-mon[102832]: pgmap v2411: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 203 KiB/s wr, 18 op/s 2023-09-10T00:29:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:36 smithi082 ceph-mon[98694]: pgmap v2411: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 203 KiB/s wr, 18 op/s 2023-09-10T00:29:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:36 smithi098 ceph-mon[103124]: pgmap v2411: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 203 KiB/s wr, 18 op/s 2023-09-10T00:29:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:38 smithi170 ceph-mon[102832]: pgmap v2412: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 172 KiB/s wr, 12 op/s 2023-09-10T00:29:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4210.089777 secs 2023-09-10T00:29:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:38 smithi082 ceph-mon[98694]: pgmap v2412: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 172 KiB/s wr, 12 op/s 2023-09-10T00:29:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4210.089777 secs 2023-09-10T00:29:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:38 smithi098 ceph-mon[103124]: pgmap v2412: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 172 KiB/s wr, 12 op/s 2023-09-10T00:29:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4210.089777 secs 2023-09-10T00:29:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4210.802875 secs 2023-09-10T00:29:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4210.802875 secs 2023-09-10T00:29:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4210.802875 secs 2023-09-10T00:29:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:40 smithi170 ceph-mon[102832]: pgmap v2413: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 172 KiB/s wr, 12 op/s 2023-09-10T00:29:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:40 smithi082 ceph-mon[98694]: pgmap v2413: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 172 KiB/s wr, 12 op/s 2023-09-10T00:29:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:40 smithi098 ceph-mon[103124]: pgmap v2413: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 172 KiB/s wr, 12 op/s 2023-09-10T00:29:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:42 smithi170 ceph-mon[102832]: pgmap v2414: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 197 KiB/s wr, 18 op/s 2023-09-10T00:29:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:42 smithi082 ceph-mon[98694]: pgmap v2414: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 197 KiB/s wr, 18 op/s 2023-09-10T00:29:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:42 smithi098 ceph-mon[103124]: pgmap v2414: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 197 KiB/s wr, 18 op/s 2023-09-10T00:29:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4215.089835 secs 2023-09-10T00:29:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4215.089835 secs 2023-09-10T00:29:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4215.089835 secs 2023-09-10T00:29:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:44 smithi170 ceph-mon[102832]: pgmap v2415: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T00:29:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4215.802939 secs 2023-09-10T00:29:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:44 smithi082 ceph-mon[98694]: pgmap v2415: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T00:29:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4215.802939 secs 2023-09-10T00:29:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:44 smithi098 ceph-mon[103124]: pgmap v2415: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T00:29:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4215.802939 secs 2023-09-10T00:29:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:46 smithi170 ceph-mon[102832]: pgmap v2416: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 145 KiB/s wr, 16 op/s 2023-09-10T00:29:47.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:46 smithi082 ceph-mon[98694]: pgmap v2416: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 145 KiB/s wr, 16 op/s 2023-09-10T00:29:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:46 smithi098 ceph-mon[103124]: pgmap v2416: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 145 KiB/s wr, 16 op/s 2023-09-10T00:29:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4218.522977 secs 2023-09-10T00:29:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4218.522977 secs 2023-09-10T00:29:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4218.522977 secs 2023-09-10T00:29:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:48 smithi170 ceph-mon[102832]: pgmap v2417: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T00:29:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4220.089894 secs 2023-09-10T00:29:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:48 smithi082 ceph-mon[98694]: pgmap v2417: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T00:29:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4220.089894 secs 2023-09-10T00:29:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:48 smithi098 ceph-mon[103124]: pgmap v2417: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T00:29:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4220.089894 secs 2023-09-10T00:29:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4220.803003 secs 2023-09-10T00:29:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4220.803003 secs 2023-09-10T00:29:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4220.803003 secs 2023-09-10T00:29:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:50 smithi170 ceph-mon[102832]: pgmap v2418: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T00:29:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:50 smithi082 ceph-mon[98694]: pgmap v2418: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T00:29:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:50 smithi098 ceph-mon[103124]: pgmap v2418: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T00:29:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:29:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:29:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:29:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:29:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:29:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:29:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:29:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:29:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:29:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:52 smithi170 ceph-mon[102832]: pgmap v2419: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 120 KiB/s wr, 16 op/s 2023-09-10T00:29:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4223.523035 secs 2023-09-10T00:29:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:52 smithi082 ceph-mon[98694]: pgmap v2419: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 120 KiB/s wr, 16 op/s 2023-09-10T00:29:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4223.523035 secs 2023-09-10T00:29:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:52 smithi098 ceph-mon[103124]: pgmap v2419: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 120 KiB/s wr, 16 op/s 2023-09-10T00:29:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4223.523035 secs 2023-09-10T00:29:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4225.089949 secs 2023-09-10T00:29:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4225.089949 secs 2023-09-10T00:29:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4225.089949 secs 2023-09-10T00:29:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:54 smithi082 ceph-mon[98694]: pgmap v2420: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T00:29:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4225.803068 secs 2023-09-10T00:29:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:54 smithi098 ceph-mon[103124]: pgmap v2420: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T00:29:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4225.803068 secs 2023-09-10T00:29:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:54 smithi170 ceph-mon[102832]: pgmap v2420: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T00:29:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4225.803068 secs 2023-09-10T00:29:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:56 smithi082 ceph-mon[98694]: pgmap v2421: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 116 KiB/s wr, 16 op/s 2023-09-10T00:29:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:29:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:56 smithi098 ceph-mon[103124]: pgmap v2421: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 116 KiB/s wr, 16 op/s 2023-09-10T00:29:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:29:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:56 smithi170 ceph-mon[102832]: pgmap v2421: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 116 KiB/s wr, 16 op/s 2023-09-10T00:29:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:29:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4228.523115 secs 2023-09-10T00:29:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:29:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:29:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:29:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4228.523115 secs 2023-09-10T00:29:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:29:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:29:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:29:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4228.523115 secs 2023-09-10T00:29:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:29:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:29:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:29:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:58 smithi082 ceph-mon[98694]: pgmap v2422: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 76 KiB/s wr, 10 op/s 2023-09-10T00:29:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4230.090021 secs 2023-09-10T00:29:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:58 smithi098 ceph-mon[103124]: pgmap v2422: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 76 KiB/s wr, 10 op/s 2023-09-10T00:29:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4230.090021 secs 2023-09-10T00:29:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:58 smithi170 ceph-mon[102832]: pgmap v2422: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 76 KiB/s wr, 10 op/s 2023-09-10T00:29:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4230.090021 secs 2023-09-10T00:30:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:29:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4230.803136 secs 2023-09-10T00:30:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:29:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4230.803136 secs 2023-09-10T00:30:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:29:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4230.803136 secs 2023-09-10T00:30:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:00 smithi082 ceph-mon[98694]: pgmap v2423: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 76 KiB/s wr, 10 op/s 2023-09-10T00:30:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T00:30:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:00 smithi098 ceph-mon[103124]: pgmap v2423: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 76 KiB/s wr, 10 op/s 2023-09-10T00:30:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T00:30:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:00 smithi170 ceph-mon[102832]: pgmap v2423: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 76 KiB/s wr, 10 op/s 2023-09-10T00:30:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T00:30:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:02 smithi082 ceph-mon[98694]: pgmap v2424: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 15 op/s 2023-09-10T00:30:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4233.523177 secs 2023-09-10T00:30:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:02 smithi098 ceph-mon[103124]: pgmap v2424: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 15 op/s 2023-09-10T00:30:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4233.523177 secs 2023-09-10T00:30:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:02 smithi170 ceph-mon[102832]: pgmap v2424: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 15 op/s 2023-09-10T00:30:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4233.523177 secs 2023-09-10T00:30:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4235.090078 secs 2023-09-10T00:30:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4235.090078 secs 2023-09-10T00:30:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4235.090078 secs 2023-09-10T00:30:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:04 smithi082 ceph-mon[98694]: pgmap v2425: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 10 op/s 2023-09-10T00:30:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4235.803200 secs 2023-09-10T00:30:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:04 smithi098 ceph-mon[103124]: pgmap v2425: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 10 op/s 2023-09-10T00:30:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4235.803200 secs 2023-09-10T00:30:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:04 smithi170 ceph-mon[102832]: pgmap v2425: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 10 op/s 2023-09-10T00:30:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4235.803200 secs 2023-09-10T00:30:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:06 smithi098 ceph-mon[103124]: pgmap v2426: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T00:30:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:06 smithi170 ceph-mon[102832]: pgmap v2426: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T00:30:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:06 smithi082 ceph-mon[98694]: pgmap v2426: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T00:30:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4238.523257 secs 2023-09-10T00:30:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:07 smithi098 ceph-mon[103124]: osdmap e223: 12 total, 12 up, 12 in 2023-09-10T00:30:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4238.523257 secs 2023-09-10T00:30:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:07 smithi170 ceph-mon[102832]: osdmap e223: 12 total, 12 up, 12 in 2023-09-10T00:30:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4238.523257 secs 2023-09-10T00:30:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:07 smithi082 ceph-mon[98694]: osdmap e223: 12 total, 12 up, 12 in 2023-09-10T00:30:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:08 smithi098 ceph-mon[103124]: pgmap v2428: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 56 KiB/s wr, 29 op/s 2023-09-10T00:30:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4240.090165 secs 2023-09-10T00:30:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:08 smithi170 ceph-mon[102832]: pgmap v2428: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 56 KiB/s wr, 29 op/s 2023-09-10T00:30:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4240.090165 secs 2023-09-10T00:30:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:08 smithi082 ceph-mon[98694]: pgmap v2428: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 56 KiB/s wr, 29 op/s 2023-09-10T00:30:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4240.090165 secs 2023-09-10T00:30:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4240.803266 secs 2023-09-10T00:30:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4240.803266 secs 2023-09-10T00:30:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4240.803266 secs 2023-09-10T00:30:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:10 smithi098 ceph-mon[103124]: pgmap v2429: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 56 KiB/s wr, 29 op/s 2023-09-10T00:30:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:10 smithi170 ceph-mon[102832]: pgmap v2429: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 56 KiB/s wr, 29 op/s 2023-09-10T00:30:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:10 smithi082 ceph-mon[98694]: pgmap v2429: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 56 KiB/s wr, 29 op/s 2023-09-10T00:30:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:12 smithi098 ceph-mon[103124]: pgmap v2430: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T00:30:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4243.523318 secs 2023-09-10T00:30:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:12 smithi170 ceph-mon[102832]: pgmap v2430: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T00:30:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4243.523318 secs 2023-09-10T00:30:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:12 smithi082 ceph-mon[98694]: pgmap v2430: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T00:30:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4243.523318 secs 2023-09-10T00:30:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4245.090250 secs 2023-09-10T00:30:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4245.090250 secs 2023-09-10T00:30:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4245.090250 secs 2023-09-10T00:30:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:14 smithi098 ceph-mon[103124]: pgmap v2431: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T00:30:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4245.803333 secs 2023-09-10T00:30:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:14 smithi098 ceph-mon[103124]: osdmap e224: 12 total, 12 up, 12 in 2023-09-10T00:30:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:14 smithi170 ceph-mon[102832]: pgmap v2431: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T00:30:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4245.803333 secs 2023-09-10T00:30:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:14 smithi170 ceph-mon[102832]: osdmap e224: 12 total, 12 up, 12 in 2023-09-10T00:30:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:14 smithi082 ceph-mon[98694]: pgmap v2431: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T00:30:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4245.803333 secs 2023-09-10T00:30:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:14 smithi082 ceph-mon[98694]: osdmap e224: 12 total, 12 up, 12 in 2023-09-10T00:30:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:16 smithi098 ceph-mon[103124]: pgmap v2433: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 13 op/s 2023-09-10T00:30:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:16 smithi170 ceph-mon[102832]: pgmap v2433: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 13 op/s 2023-09-10T00:30:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:16 smithi082 ceph-mon[98694]: pgmap v2433: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 13 op/s 2023-09-10T00:30:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4248.523377 secs 2023-09-10T00:30:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4248.523377 secs 2023-09-10T00:30:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4248.523377 secs 2023-09-10T00:30:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:18 smithi098 ceph-mon[103124]: pgmap v2434: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T00:30:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4250.090311 secs 2023-09-10T00:30:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:18 smithi170 ceph-mon[102832]: pgmap v2434: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T00:30:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4250.090311 secs 2023-09-10T00:30:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:18 smithi082 ceph-mon[98694]: pgmap v2434: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T00:30:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4250.090311 secs 2023-09-10T00:30:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4250.803424 secs 2023-09-10T00:30:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4250.803424 secs 2023-09-10T00:30:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4250.803424 secs 2023-09-10T00:30:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:21 smithi098 ceph-mon[103124]: pgmap v2435: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T00:30:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:21 smithi170 ceph-mon[102832]: pgmap v2435: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T00:30:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:21 smithi082 ceph-mon[98694]: pgmap v2435: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T00:30:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:23 smithi170 ceph-mon[102832]: pgmap v2436: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T00:30:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4253.523493 secs 2023-09-10T00:30:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:23 smithi082 ceph-mon[98694]: pgmap v2436: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T00:30:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4253.523493 secs 2023-09-10T00:30:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:23 smithi098 ceph-mon[103124]: pgmap v2436: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T00:30:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4253.523493 secs 2023-09-10T00:30:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4255.090368 secs 2023-09-10T00:30:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4255.090368 secs 2023-09-10T00:30:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4255.090368 secs 2023-09-10T00:30:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4255.803489 secs 2023-09-10T00:30:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:25 smithi170 ceph-mon[102832]: pgmap v2437: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T00:30:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4255.803489 secs 2023-09-10T00:30:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:25 smithi082 ceph-mon[98694]: pgmap v2437: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T00:30:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4255.803489 secs 2023-09-10T00:30:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:25 smithi098 ceph-mon[103124]: pgmap v2437: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T00:30:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:27 smithi170 ceph-mon[102832]: pgmap v2438: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 95 KiB/s wr, 17 op/s 2023-09-10T00:30:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:27 smithi082 ceph-mon[98694]: pgmap v2438: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 95 KiB/s wr, 17 op/s 2023-09-10T00:30:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:27 smithi098 ceph-mon[103124]: pgmap v2438: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 95 KiB/s wr, 17 op/s 2023-09-10T00:30:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:29 smithi170 ceph-mon[102832]: pgmap v2439: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T00:30:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4260.090424 secs 2023-09-10T00:30:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:29 smithi082 ceph-mon[98694]: pgmap v2439: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T00:30:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4260.090424 secs 2023-09-10T00:30:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:29 smithi098 ceph-mon[103124]: pgmap v2439: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T00:30:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4260.090424 secs 2023-09-10T00:30:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4260.803555 secs 2023-09-10T00:30:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4260.803555 secs 2023-09-10T00:30:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4260.803555 secs 2023-09-10T00:30:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:31 smithi170 ceph-mon[102832]: pgmap v2440: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T00:30:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:30:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:31 smithi082 ceph-mon[98694]: pgmap v2440: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T00:30:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:30:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:31 smithi098 ceph-mon[103124]: pgmap v2440: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T00:30:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:30:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:30:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:30:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:30:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:30:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:30:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:30:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:30:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:30:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:30:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:33 smithi170 ceph-mon[102832]: pgmap v2441: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 132 KiB/s wr, 18 op/s 2023-09-10T00:30:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:33 smithi082 ceph-mon[98694]: pgmap v2441: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 132 KiB/s wr, 18 op/s 2023-09-10T00:30:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:33 smithi098 ceph-mon[103124]: pgmap v2441: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 132 KiB/s wr, 18 op/s 2023-09-10T00:30:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4265.090487 secs 2023-09-10T00:30:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4265.090487 secs 2023-09-10T00:30:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4265.090487 secs 2023-09-10T00:30:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4265.803624 secs 2023-09-10T00:30:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:35 smithi170 ceph-mon[102832]: pgmap v2442: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T00:30:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4265.803624 secs 2023-09-10T00:30:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:35 smithi082 ceph-mon[98694]: pgmap v2442: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T00:30:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4265.803624 secs 2023-09-10T00:30:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:35 smithi098 ceph-mon[103124]: pgmap v2442: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T00:30:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:37 smithi170 ceph-mon[102832]: pgmap v2443: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 167 KiB/s wr, 19 op/s 2023-09-10T00:30:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:37 smithi082 ceph-mon[98694]: pgmap v2443: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 167 KiB/s wr, 19 op/s 2023-09-10T00:30:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:37 smithi098 ceph-mon[103124]: pgmap v2443: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 167 KiB/s wr, 19 op/s 2023-09-10T00:30:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:39 smithi170 ceph-mon[102832]: pgmap v2444: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 116 KiB/s wr, 13 op/s 2023-09-10T00:30:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4270.090544 secs 2023-09-10T00:30:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:39 smithi082 ceph-mon[98694]: pgmap v2444: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 116 KiB/s wr, 13 op/s 2023-09-10T00:30:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4270.090544 secs 2023-09-10T00:30:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:39 smithi098 ceph-mon[103124]: pgmap v2444: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 116 KiB/s wr, 13 op/s 2023-09-10T00:30:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4270.090544 secs 2023-09-10T00:30:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4270.803687 secs 2023-09-10T00:30:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4270.803687 secs 2023-09-10T00:30:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4270.803687 secs 2023-09-10T00:30:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:41 smithi170 ceph-mon[102832]: pgmap v2445: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 116 KiB/s wr, 13 op/s 2023-09-10T00:30:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:41 smithi082 ceph-mon[98694]: pgmap v2445: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 116 KiB/s wr, 13 op/s 2023-09-10T00:30:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:41 smithi098 ceph-mon[103124]: pgmap v2445: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 116 KiB/s wr, 13 op/s 2023-09-10T00:30:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:43 smithi170 ceph-mon[102832]: pgmap v2446: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 173 KiB/s wr, 18 op/s 2023-09-10T00:30:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:43 smithi082 ceph-mon[98694]: pgmap v2446: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 173 KiB/s wr, 18 op/s 2023-09-10T00:30:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:43 smithi098 ceph-mon[103124]: pgmap v2446: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 173 KiB/s wr, 18 op/s 2023-09-10T00:30:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4275.090602 secs 2023-09-10T00:30:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4275.090602 secs 2023-09-10T00:30:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4275.090602 secs 2023-09-10T00:30:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4275.803752 secs 2023-09-10T00:30:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:45 smithi170 ceph-mon[102832]: pgmap v2447: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T00:30:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4275.803752 secs 2023-09-10T00:30:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:45 smithi082 ceph-mon[98694]: pgmap v2447: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T00:30:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4275.803752 secs 2023-09-10T00:30:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:45 smithi098 ceph-mon[103124]: pgmap v2447: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T00:30:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:47 smithi170 ceph-mon[102832]: pgmap v2448: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 219 KiB/s wr, 17 op/s 2023-09-10T00:30:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:47 smithi082 ceph-mon[98694]: pgmap v2448: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 219 KiB/s wr, 17 op/s 2023-09-10T00:30:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:47 smithi098 ceph-mon[103124]: pgmap v2448: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 219 KiB/s wr, 17 op/s 2023-09-10T00:30:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4278.523925 secs 2023-09-10T00:30:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4278.523925 secs 2023-09-10T00:30:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4278.523925 secs 2023-09-10T00:30:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:49 smithi170 ceph-mon[102832]: pgmap v2449: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T00:30:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4280.090701 secs 2023-09-10T00:30:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:49 smithi082 ceph-mon[98694]: pgmap v2449: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T00:30:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4280.090701 secs 2023-09-10T00:30:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:49 smithi098 ceph-mon[103124]: pgmap v2449: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T00:30:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4280.090701 secs 2023-09-10T00:30:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4280.803818 secs 2023-09-10T00:30:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4280.803818 secs 2023-09-10T00:30:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4280.803818 secs 2023-09-10T00:30:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:51 smithi170 ceph-mon[102832]: pgmap v2450: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T00:30:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:51 smithi082 ceph-mon[98694]: pgmap v2450: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T00:30:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:51 smithi098 ceph-mon[103124]: pgmap v2450: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T00:30:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:30:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:30:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:30:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:30:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:30:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:30:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:30:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:30:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:30:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:53 smithi170 ceph-mon[102832]: pgmap v2451: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 173 KiB/s wr, 16 op/s 2023-09-10T00:30:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4283.523989 secs 2023-09-10T00:30:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:53 smithi082 ceph-mon[98694]: pgmap v2451: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 173 KiB/s wr, 16 op/s 2023-09-10T00:30:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4283.523989 secs 2023-09-10T00:30:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:53 smithi098 ceph-mon[103124]: pgmap v2451: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 173 KiB/s wr, 16 op/s 2023-09-10T00:30:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4283.523989 secs 2023-09-10T00:30:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4285.090754 secs 2023-09-10T00:30:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4285.090754 secs 2023-09-10T00:30:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4285.090754 secs 2023-09-10T00:30:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4285.803882 secs 2023-09-10T00:30:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:55 smithi170 ceph-mon[102832]: pgmap v2452: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T00:30:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4285.803882 secs 2023-09-10T00:30:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:55 smithi082 ceph-mon[98694]: pgmap v2452: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T00:30:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4285.803882 secs 2023-09-10T00:30:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:55 smithi098 ceph-mon[103124]: pgmap v2452: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T00:30:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:57 smithi170 ceph-mon[102832]: pgmap v2453: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 167 KiB/s wr, 16 op/s 2023-09-10T00:30:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:57 smithi082 ceph-mon[98694]: pgmap v2453: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 167 KiB/s wr, 16 op/s 2023-09-10T00:30:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:57 smithi098 ceph-mon[103124]: pgmap v2453: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 167 KiB/s wr, 16 op/s 2023-09-10T00:30:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4288.524083 secs 2023-09-10T00:30:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:30:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4288.524083 secs 2023-09-10T00:30:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:30:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4288.524083 secs 2023-09-10T00:30:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:30:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:59 smithi170 ceph-mon[102832]: pgmap v2454: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T00:30:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4290.090838 secs 2023-09-10T00:30:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:30:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:30:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:30:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:30:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:59 smithi082 ceph-mon[98694]: pgmap v2454: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T00:30:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4290.090838 secs 2023-09-10T00:30:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:30:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:30:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:30:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:30:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:59 smithi098 ceph-mon[103124]: pgmap v2454: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T00:30:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4290.090838 secs 2023-09-10T00:30:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:30:59.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:30:59.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:30:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:31:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4290.803948 secs 2023-09-10T00:31:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4290.803948 secs 2023-09-10T00:31:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4290.803948 secs 2023-09-10T00:31:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:01 smithi170 ceph-mon[102832]: pgmap v2455: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T00:31:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:01 smithi082 ceph-mon[98694]: pgmap v2455: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T00:31:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:01 smithi098 ceph-mon[103124]: pgmap v2455: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T00:31:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:03 smithi170 ceph-mon[102832]: pgmap v2456: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 76 KiB/s wr, 14 op/s 2023-09-10T00:31:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4293.524144 secs 2023-09-10T00:31:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:03 smithi082 ceph-mon[98694]: pgmap v2456: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 76 KiB/s wr, 14 op/s 2023-09-10T00:31:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4293.524144 secs 2023-09-10T00:31:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:03 smithi098 ceph-mon[103124]: pgmap v2456: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 76 KiB/s wr, 14 op/s 2023-09-10T00:31:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4293.524144 secs 2023-09-10T00:31:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4295.090921 secs 2023-09-10T00:31:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4295.090921 secs 2023-09-10T00:31:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4295.090921 secs 2023-09-10T00:31:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4295.804037 secs 2023-09-10T00:31:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:05 smithi170 ceph-mon[102832]: pgmap v2457: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 51 KiB/s wr, 9 op/s 2023-09-10T00:31:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4295.804037 secs 2023-09-10T00:31:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:05 smithi082 ceph-mon[98694]: pgmap v2457: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 51 KiB/s wr, 9 op/s 2023-09-10T00:31:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4295.804037 secs 2023-09-10T00:31:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:05 smithi098 ceph-mon[103124]: pgmap v2457: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 51 KiB/s wr, 9 op/s 2023-09-10T00:31:07.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:07 smithi170 ceph-mon[102832]: pgmap v2458: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 116 KiB/s wr, 29 op/s 2023-09-10T00:31:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:07 smithi082 ceph-mon[98694]: pgmap v2458: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 116 KiB/s wr, 29 op/s 2023-09-10T00:31:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:07 smithi098 ceph-mon[103124]: pgmap v2458: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 116 KiB/s wr, 29 op/s 2023-09-10T00:31:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4298.524221 secs 2023-09-10T00:31:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:08 smithi170 ceph-mon[102832]: osdmap e225: 12 total, 12 up, 12 in 2023-09-10T00:31:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4298.524221 secs 2023-09-10T00:31:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:08 smithi082 ceph-mon[98694]: osdmap e225: 12 total, 12 up, 12 in 2023-09-10T00:31:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4298.524221 secs 2023-09-10T00:31:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:08 smithi098 ceph-mon[103124]: osdmap e225: 12 total, 12 up, 12 in 2023-09-10T00:31:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:09 smithi170 ceph-mon[102832]: pgmap v2460: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 79 KiB/s wr, 28 op/s 2023-09-10T00:31:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4300.090981 secs 2023-09-10T00:31:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:09 smithi082 ceph-mon[98694]: pgmap v2460: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 79 KiB/s wr, 28 op/s 2023-09-10T00:31:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4300.090981 secs 2023-09-10T00:31:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:09 smithi098 ceph-mon[103124]: pgmap v2460: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 79 KiB/s wr, 28 op/s 2023-09-10T00:31:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4300.090981 secs 2023-09-10T00:31:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4300.804103 secs 2023-09-10T00:31:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4300.804103 secs 2023-09-10T00:31:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4300.804103 secs 2023-09-10T00:31:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:11 smithi170 ceph-mon[102832]: pgmap v2461: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 79 KiB/s wr, 28 op/s 2023-09-10T00:31:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:11 smithi082 ceph-mon[98694]: pgmap v2461: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 79 KiB/s wr, 28 op/s 2023-09-10T00:31:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:11 smithi098 ceph-mon[103124]: pgmap v2461: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 79 KiB/s wr, 28 op/s 2023-09-10T00:31:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:13 smithi170 ceph-mon[102832]: pgmap v2462: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 107 KiB/s wr, 30 op/s 2023-09-10T00:31:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4303.524281 secs 2023-09-10T00:31:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:13 smithi082 ceph-mon[98694]: pgmap v2462: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 107 KiB/s wr, 30 op/s 2023-09-10T00:31:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4303.524281 secs 2023-09-10T00:31:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:13 smithi098 ceph-mon[103124]: pgmap v2462: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 107 KiB/s wr, 30 op/s 2023-09-10T00:31:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4303.524281 secs 2023-09-10T00:31:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4305.091067 secs 2023-09-10T00:31:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4305.091067 secs 2023-09-10T00:31:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4305.091067 secs 2023-09-10T00:31:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4305.804167 secs 2023-09-10T00:31:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:15 smithi170 ceph-mon[102832]: pgmap v2463: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 107 KiB/s wr, 30 op/s 2023-09-10T00:31:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:15 smithi170 ceph-mon[102832]: osdmap e226: 12 total, 12 up, 12 in 2023-09-10T00:31:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4305.804167 secs 2023-09-10T00:31:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:15 smithi082 ceph-mon[98694]: pgmap v2463: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 107 KiB/s wr, 30 op/s 2023-09-10T00:31:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:15 smithi082 ceph-mon[98694]: osdmap e226: 12 total, 12 up, 12 in 2023-09-10T00:31:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4305.804167 secs 2023-09-10T00:31:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:15 smithi098 ceph-mon[103124]: pgmap v2463: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 107 KiB/s wr, 30 op/s 2023-09-10T00:31:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:15 smithi098 ceph-mon[103124]: osdmap e226: 12 total, 12 up, 12 in 2023-09-10T00:31:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:17 smithi170 ceph-mon[102832]: pgmap v2465: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 66 KiB/s wr, 15 op/s 2023-09-10T00:31:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:17 smithi082 ceph-mon[98694]: pgmap v2465: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 66 KiB/s wr, 15 op/s 2023-09-10T00:31:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:17 smithi098 ceph-mon[103124]: pgmap v2465: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 66 KiB/s wr, 15 op/s 2023-09-10T00:31:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:19 smithi170 ceph-mon[102832]: pgmap v2466: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 56 KiB/s wr, 13 op/s 2023-09-10T00:31:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4310.091127 secs 2023-09-10T00:31:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:31:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:19 smithi082 ceph-mon[98694]: pgmap v2466: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 56 KiB/s wr, 13 op/s 2023-09-10T00:31:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4310.091127 secs 2023-09-10T00:31:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:31:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:19 smithi098 ceph-mon[103124]: pgmap v2466: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 56 KiB/s wr, 13 op/s 2023-09-10T00:31:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4310.091127 secs 2023-09-10T00:31:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:31:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4310.804233 secs 2023-09-10T00:31:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:31:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:31:20.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:31:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4310.804233 secs 2023-09-10T00:31:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:31:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:31:20.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:31:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4310.804233 secs 2023-09-10T00:31:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:31:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:31:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:31:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:21 smithi082 ceph-mon[98694]: pgmap v2467: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 56 KiB/s wr, 13 op/s 2023-09-10T00:31:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:21 smithi098 ceph-mon[103124]: pgmap v2467: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 56 KiB/s wr, 13 op/s 2023-09-10T00:31:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:21 smithi170 ceph-mon[102832]: pgmap v2467: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 56 KiB/s wr, 13 op/s 2023-09-10T00:31:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:23 smithi082 ceph-mon[98694]: pgmap v2468: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T00:31:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4313.524441 secs 2023-09-10T00:31:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:23 smithi098 ceph-mon[103124]: pgmap v2468: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T00:31:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4313.524441 secs 2023-09-10T00:31:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:23 smithi170 ceph-mon[102832]: pgmap v2468: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T00:31:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4313.524441 secs 2023-09-10T00:31:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4315.091185 secs 2023-09-10T00:31:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:31:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:31:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4315.091185 secs 2023-09-10T00:31:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:31:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:31:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4315.091185 secs 2023-09-10T00:31:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:31:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:31:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4315.804300 secs 2023-09-10T00:31:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:25 smithi082 ceph-mon[98694]: pgmap v2469: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T00:31:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:25 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:31:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4315.804300 secs 2023-09-10T00:31:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:25 smithi098 ceph-mon[103124]: pgmap v2469: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T00:31:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:25 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:31:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4315.804300 secs 2023-09-10T00:31:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:25 smithi170 ceph-mon[102832]: pgmap v2469: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T00:31:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:25 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:31:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:27 smithi082 ceph-mon[98694]: pgmap v2470: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 131 KiB/s wr, 19 op/s 2023-09-10T00:31:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:27 smithi098 ceph-mon[103124]: pgmap v2470: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 131 KiB/s wr, 19 op/s 2023-09-10T00:31:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:27 smithi170 ceph-mon[102832]: pgmap v2470: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 131 KiB/s wr, 19 op/s 2023-09-10T00:31:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:29 smithi082 ceph-mon[98694]: pgmap v2471: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T00:31:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4320.091270 secs 2023-09-10T00:31:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:29 smithi098 ceph-mon[103124]: pgmap v2471: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T00:31:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4320.091270 secs 2023-09-10T00:31:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:29 smithi170 ceph-mon[102832]: pgmap v2471: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T00:31:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4320.091270 secs 2023-09-10T00:31:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4320.804366 secs 2023-09-10T00:31:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4320.804366 secs 2023-09-10T00:31:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4320.804366 secs 2023-09-10T00:31:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:31 smithi098 ceph-mon[103124]: pgmap v2472: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T00:31:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:31:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:31 smithi170 ceph-mon[102832]: pgmap v2472: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T00:31:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:31:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:31 smithi082 ceph-mon[98694]: pgmap v2472: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T00:31:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:31:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:31:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:31:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:31:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:31:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:31:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:31:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:31:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:31:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:31:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:33 smithi098 ceph-mon[103124]: pgmap v2473: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 153 KiB/s wr, 17 op/s 2023-09-10T00:31:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:33 smithi170 ceph-mon[102832]: pgmap v2473: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 153 KiB/s wr, 17 op/s 2023-09-10T00:31:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:33 smithi082 ceph-mon[98694]: pgmap v2473: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 153 KiB/s wr, 17 op/s 2023-09-10T00:31:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4325.091335 secs 2023-09-10T00:31:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:34 smithi098 ceph-mon[103124]: pgmap v2474: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T00:31:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4325.091335 secs 2023-09-10T00:31:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:34 smithi170 ceph-mon[102832]: pgmap v2474: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T00:31:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4325.091335 secs 2023-09-10T00:31:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:34 smithi082 ceph-mon[98694]: pgmap v2474: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T00:31:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4325.804430 secs 2023-09-10T00:31:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4325.804430 secs 2023-09-10T00:31:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4325.804430 secs 2023-09-10T00:31:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:36 smithi098 ceph-mon[103124]: pgmap v2475: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 201 KiB/s wr, 17 op/s 2023-09-10T00:31:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:36 smithi170 ceph-mon[102832]: pgmap v2475: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 201 KiB/s wr, 17 op/s 2023-09-10T00:31:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:36 smithi082 ceph-mon[98694]: pgmap v2475: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 201 KiB/s wr, 17 op/s 2023-09-10T00:31:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:38 smithi170 ceph-mon[102832]: pgmap v2476: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T00:31:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4330.091416 secs 2023-09-10T00:31:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:38 smithi082 ceph-mon[98694]: pgmap v2476: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T00:31:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4330.091416 secs 2023-09-10T00:31:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:38 smithi098 ceph-mon[103124]: pgmap v2476: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T00:31:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4330.091416 secs 2023-09-10T00:31:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4330.804487 secs 2023-09-10T00:31:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4330.804487 secs 2023-09-10T00:31:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4330.804487 secs 2023-09-10T00:31:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:40 smithi170 ceph-mon[102832]: pgmap v2477: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T00:31:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:40 smithi082 ceph-mon[98694]: pgmap v2477: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T00:31:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:40 smithi098 ceph-mon[103124]: pgmap v2477: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T00:31:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:42 smithi170 ceph-mon[102832]: pgmap v2478: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 16 op/s 2023-09-10T00:31:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:42 smithi082 ceph-mon[98694]: pgmap v2478: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 16 op/s 2023-09-10T00:31:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:42 smithi098 ceph-mon[103124]: pgmap v2478: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 16 op/s 2023-09-10T00:31:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4335.091471 secs 2023-09-10T00:31:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4335.091471 secs 2023-09-10T00:31:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4335.091471 secs 2023-09-10T00:31:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4335.804553 secs 2023-09-10T00:31:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:44 smithi170 ceph-mon[102832]: pgmap v2479: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T00:31:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4335.804553 secs 2023-09-10T00:31:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:44 smithi082 ceph-mon[98694]: pgmap v2479: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T00:31:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4335.804553 secs 2023-09-10T00:31:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:44 smithi098 ceph-mon[103124]: pgmap v2479: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T00:31:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:46 smithi170 ceph-mon[102832]: pgmap v2480: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 179 KiB/s wr, 17 op/s 2023-09-10T00:31:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:46 smithi082 ceph-mon[98694]: pgmap v2480: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 179 KiB/s wr, 17 op/s 2023-09-10T00:31:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:46 smithi098 ceph-mon[103124]: pgmap v2480: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 179 KiB/s wr, 17 op/s 2023-09-10T00:31:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4338.555656 secs 2023-09-10T00:31:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4338.555656 secs 2023-09-10T00:31:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4338.555656 secs 2023-09-10T00:31:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:48 smithi170 ceph-mon[102832]: pgmap v2481: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T00:31:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4340.091526 secs 2023-09-10T00:31:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:48 smithi082 ceph-mon[98694]: pgmap v2481: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T00:31:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4340.091526 secs 2023-09-10T00:31:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:48 smithi098 ceph-mon[103124]: pgmap v2481: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T00:31:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4340.091526 secs 2023-09-10T00:31:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4340.804622 secs 2023-09-10T00:31:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4340.804622 secs 2023-09-10T00:31:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4340.804622 secs 2023-09-10T00:31:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:50 smithi170 ceph-mon[102832]: pgmap v2482: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T00:31:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:50 smithi082 ceph-mon[98694]: pgmap v2482: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T00:31:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:50 smithi098 ceph-mon[103124]: pgmap v2482: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T00:31:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:31:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:31:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:31:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:31:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:31:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:31:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:31:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:31:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:31:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:52 smithi170 ceph-mon[102832]: pgmap v2483: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 117 KiB/s wr, 17 op/s 2023-09-10T00:31:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4343.555768 secs 2023-09-10T00:31:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:52 smithi082 ceph-mon[98694]: pgmap v2483: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 117 KiB/s wr, 17 op/s 2023-09-10T00:31:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4343.555768 secs 2023-09-10T00:31:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:52 smithi098 ceph-mon[103124]: pgmap v2483: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 117 KiB/s wr, 17 op/s 2023-09-10T00:31:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4343.555768 secs 2023-09-10T00:31:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4345.091585 secs 2023-09-10T00:31:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4345.091585 secs 2023-09-10T00:31:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4345.091585 secs 2023-09-10T00:31:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4345.804689 secs 2023-09-10T00:31:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:54 smithi170 ceph-mon[102832]: pgmap v2484: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 79 KiB/s wr, 12 op/s 2023-09-10T00:31:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4345.804689 secs 2023-09-10T00:31:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:54 smithi082 ceph-mon[98694]: pgmap v2484: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 79 KiB/s wr, 12 op/s 2023-09-10T00:31:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4345.804689 secs 2023-09-10T00:31:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:54 smithi098 ceph-mon[103124]: pgmap v2484: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 79 KiB/s wr, 12 op/s 2023-09-10T00:31:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:56 smithi170 ceph-mon[102832]: pgmap v2485: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 100 KiB/s wr, 17 op/s 2023-09-10T00:31:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:56 smithi082 ceph-mon[98694]: pgmap v2485: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 100 KiB/s wr, 17 op/s 2023-09-10T00:31:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:56 smithi098 ceph-mon[103124]: pgmap v2485: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 100 KiB/s wr, 17 op/s 2023-09-10T00:31:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4348.555828 secs 2023-09-10T00:31:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4348.555828 secs 2023-09-10T00:31:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4348.555828 secs 2023-09-10T00:31:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:58 smithi170 ceph-mon[102832]: pgmap v2486: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T00:31:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4350.091643 secs 2023-09-10T00:31:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:31:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:58 smithi082 ceph-mon[98694]: pgmap v2486: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T00:31:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4350.091643 secs 2023-09-10T00:31:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:31:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:58 smithi098 ceph-mon[103124]: pgmap v2486: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T00:31:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4350.091643 secs 2023-09-10T00:31:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:31:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4350.804759 secs 2023-09-10T00:31:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:31:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:31:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:31:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:31:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4350.804759 secs 2023-09-10T00:31:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:31:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:31:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:31:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:32:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4350.804759 secs 2023-09-10T00:32:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:32:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:32:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:31:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:32:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:01 smithi170 ceph-mon[102832]: pgmap v2487: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T00:32:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:01 smithi082 ceph-mon[98694]: pgmap v2487: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T00:32:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:01 smithi098 ceph-mon[103124]: pgmap v2487: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T00:32:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:03 smithi170 ceph-mon[102832]: pgmap v2488: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 45 KiB/s wr, 16 op/s 2023-09-10T00:32:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4353.555931 secs 2023-09-10T00:32:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:03 smithi082 ceph-mon[98694]: pgmap v2488: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 45 KiB/s wr, 16 op/s 2023-09-10T00:32:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4353.555931 secs 2023-09-10T00:32:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:03 smithi098 ceph-mon[103124]: pgmap v2488: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 45 KiB/s wr, 16 op/s 2023-09-10T00:32:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4353.555931 secs 2023-09-10T00:32:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4355.091717 secs 2023-09-10T00:32:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4355.091717 secs 2023-09-10T00:32:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4355.091717 secs 2023-09-10T00:32:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4355.804823 secs 2023-09-10T00:32:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:05 smithi170 ceph-mon[102832]: pgmap v2489: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 10 op/s 2023-09-10T00:32:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4355.804823 secs 2023-09-10T00:32:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:05 smithi082 ceph-mon[98694]: pgmap v2489: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 10 op/s 2023-09-10T00:32:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4355.804823 secs 2023-09-10T00:32:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:05 smithi098 ceph-mon[103124]: pgmap v2489: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 10 op/s 2023-09-10T00:32:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:07 smithi170 ceph-mon[102832]: pgmap v2490: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T00:32:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:07 smithi082 ceph-mon[98694]: pgmap v2490: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T00:32:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:07 smithi098 ceph-mon[103124]: pgmap v2490: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T00:32:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4358.555992 secs 2023-09-10T00:32:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:08 smithi170 ceph-mon[102832]: osdmap e227: 12 total, 12 up, 12 in 2023-09-10T00:32:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4358.555992 secs 2023-09-10T00:32:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:08 smithi082 ceph-mon[98694]: osdmap e227: 12 total, 12 up, 12 in 2023-09-10T00:32:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4358.555992 secs 2023-09-10T00:32:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:08 smithi098 ceph-mon[103124]: osdmap e227: 12 total, 12 up, 12 in 2023-09-10T00:32:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:09 smithi170 ceph-mon[102832]: pgmap v2492: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T00:32:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4360.091772 secs 2023-09-10T00:32:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:09 smithi082 ceph-mon[98694]: pgmap v2492: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T00:32:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4360.091772 secs 2023-09-10T00:32:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:09 smithi098 ceph-mon[103124]: pgmap v2492: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T00:32:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4360.091772 secs 2023-09-10T00:32:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4360.804888 secs 2023-09-10T00:32:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4360.804888 secs 2023-09-10T00:32:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4360.804888 secs 2023-09-10T00:32:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:11 smithi170 ceph-mon[102832]: pgmap v2493: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T00:32:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:11 smithi082 ceph-mon[98694]: pgmap v2493: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T00:32:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:11 smithi098 ceph-mon[103124]: pgmap v2493: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T00:32:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:13 smithi170 ceph-mon[102832]: pgmap v2494: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 119 KiB/s wr, 29 op/s 2023-09-10T00:32:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4363.556054 secs 2023-09-10T00:32:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:13 smithi082 ceph-mon[98694]: pgmap v2494: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 119 KiB/s wr, 29 op/s 2023-09-10T00:32:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4363.556054 secs 2023-09-10T00:32:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:13 smithi098 ceph-mon[103124]: pgmap v2494: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 119 KiB/s wr, 29 op/s 2023-09-10T00:32:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4363.556054 secs 2023-09-10T00:32:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4365.091867 secs 2023-09-10T00:32:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4365.091867 secs 2023-09-10T00:32:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4365.091867 secs 2023-09-10T00:32:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4365.804956 secs 2023-09-10T00:32:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:15 smithi170 ceph-mon[102832]: pgmap v2495: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 119 KiB/s wr, 29 op/s 2023-09-10T00:32:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:15 smithi170 ceph-mon[102832]: osdmap e228: 12 total, 12 up, 12 in 2023-09-10T00:32:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4365.804956 secs 2023-09-10T00:32:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:15 smithi082 ceph-mon[98694]: pgmap v2495: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 119 KiB/s wr, 29 op/s 2023-09-10T00:32:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:15 smithi082 ceph-mon[98694]: osdmap e228: 12 total, 12 up, 12 in 2023-09-10T00:32:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4365.804956 secs 2023-09-10T00:32:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:15 smithi098 ceph-mon[103124]: pgmap v2495: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 119 KiB/s wr, 29 op/s 2023-09-10T00:32:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:15 smithi098 ceph-mon[103124]: osdmap e228: 12 total, 12 up, 12 in 2023-09-10T00:32:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:17 smithi170 ceph-mon[102832]: pgmap v2497: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 13 op/s 2023-09-10T00:32:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:17 smithi082 ceph-mon[98694]: pgmap v2497: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 13 op/s 2023-09-10T00:32:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:17 smithi098 ceph-mon[103124]: pgmap v2497: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 13 op/s 2023-09-10T00:32:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:19 smithi170 ceph-mon[102832]: pgmap v2498: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T00:32:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4370.091925 secs 2023-09-10T00:32:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:32:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:19 smithi082 ceph-mon[98694]: pgmap v2498: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T00:32:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4370.091925 secs 2023-09-10T00:32:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:32:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:19 smithi098 ceph-mon[103124]: pgmap v2498: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T00:32:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4370.091925 secs 2023-09-10T00:32:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:32:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4370.805020 secs 2023-09-10T00:32:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:32:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:32:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:32:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4370.805020 secs 2023-09-10T00:32:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:32:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:32:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:32:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4370.805020 secs 2023-09-10T00:32:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:32:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:32:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:32:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:21 smithi170 ceph-mon[102832]: pgmap v2499: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T00:32:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:21 smithi082 ceph-mon[98694]: pgmap v2499: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T00:32:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:21 smithi098 ceph-mon[103124]: pgmap v2499: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T00:32:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:23 smithi170 ceph-mon[102832]: pgmap v2500: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T00:32:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:23 smithi082 ceph-mon[98694]: pgmap v2500: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T00:32:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:23 smithi098 ceph-mon[103124]: pgmap v2500: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T00:32:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4375.092025 secs 2023-09-10T00:32:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4375.092025 secs 2023-09-10T00:32:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4375.092025 secs 2023-09-10T00:32:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4375.805088 secs 2023-09-10T00:32:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:25 smithi170 ceph-mon[102832]: pgmap v2501: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T00:32:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4375.805088 secs 2023-09-10T00:32:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:25 smithi082 ceph-mon[98694]: pgmap v2501: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T00:32:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4375.805088 secs 2023-09-10T00:32:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:25 smithi098 ceph-mon[103124]: pgmap v2501: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T00:32:27.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:27 smithi170 ceph-mon[102832]: pgmap v2502: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 134 KiB/s wr, 18 op/s 2023-09-10T00:32:27.532 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:27 smithi098 ceph-mon[103124]: pgmap v2502: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 134 KiB/s wr, 18 op/s 2023-09-10T00:32:27.532 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:27 smithi082 ceph-mon[98694]: pgmap v2502: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 134 KiB/s wr, 18 op/s 2023-09-10T00:32:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4378.556316 secs 2023-09-10T00:32:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4378.556316 secs 2023-09-10T00:32:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4378.556316 secs 2023-09-10T00:32:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:29 smithi170 ceph-mon[102832]: pgmap v2503: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T00:32:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4380.092098 secs 2023-09-10T00:32:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:29 smithi082 ceph-mon[98694]: pgmap v2503: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T00:32:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4380.092098 secs 2023-09-10T00:32:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:29 smithi098 ceph-mon[103124]: pgmap v2503: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T00:32:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4380.092098 secs 2023-09-10T00:32:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4380.805152 secs 2023-09-10T00:32:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4380.805152 secs 2023-09-10T00:32:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4380.805152 secs 2023-09-10T00:32:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:31 smithi170 ceph-mon[102832]: pgmap v2504: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T00:32:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:31 smithi082 ceph-mon[98694]: pgmap v2504: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T00:32:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:31 smithi098 ceph-mon[103124]: pgmap v2504: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T00:32:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:32 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:32:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:32:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:32:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:32 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:32:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:32:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:32:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:32 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:32:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:32:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:32:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:33 smithi170 ceph-mon[102832]: pgmap v2505: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 184 KiB/s wr, 31 op/s 2023-09-10T00:32:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:33 smithi082 ceph-mon[98694]: pgmap v2505: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 184 KiB/s wr, 31 op/s 2023-09-10T00:32:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:33 smithi098 ceph-mon[103124]: pgmap v2505: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 184 KiB/s wr, 31 op/s 2023-09-10T00:32:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4385.092156 secs 2023-09-10T00:32:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4385.092156 secs 2023-09-10T00:32:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4385.092156 secs 2023-09-10T00:32:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4385.805224 secs 2023-09-10T00:32:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:35 smithi170 ceph-mon[102832]: pgmap v2506: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 150 KiB/s wr, 26 op/s 2023-09-10T00:32:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:32:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4385.805224 secs 2023-09-10T00:32:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:35 smithi082 ceph-mon[98694]: pgmap v2506: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 150 KiB/s wr, 26 op/s 2023-09-10T00:32:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:32:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4385.805224 secs 2023-09-10T00:32:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:35 smithi098 ceph-mon[103124]: pgmap v2506: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 150 KiB/s wr, 26 op/s 2023-09-10T00:32:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:32:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:36 smithi170 ceph-mon[102832]: pgmap v2507: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 209 KiB/s wr, 32 op/s 2023-09-10T00:32:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:32:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:32:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:36 smithi082 ceph-mon[98694]: pgmap v2507: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 209 KiB/s wr, 32 op/s 2023-09-10T00:32:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:32:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:32:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:36 smithi098 ceph-mon[103124]: pgmap v2507: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 209 KiB/s wr, 32 op/s 2023-09-10T00:32:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:32:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:32:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:38 smithi170 ceph-mon[102832]: pgmap v2508: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 121 KiB/s wr, 24 op/s 2023-09-10T00:32:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4390.092213 secs 2023-09-10T00:32:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:38 smithi082 ceph-mon[98694]: pgmap v2508: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 121 KiB/s wr, 24 op/s 2023-09-10T00:32:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4390.092213 secs 2023-09-10T00:32:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:38 smithi098 ceph-mon[103124]: pgmap v2508: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 121 KiB/s wr, 24 op/s 2023-09-10T00:32:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4390.092213 secs 2023-09-10T00:32:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4390.805288 secs 2023-09-10T00:32:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:39 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:32:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4390.805288 secs 2023-09-10T00:32:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:39 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:32:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4390.805288 secs 2023-09-10T00:32:40.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:39 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:32:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:40 smithi170 ceph-mon[102832]: pgmap v2509: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 121 KiB/s wr, 24 op/s 2023-09-10T00:32:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:40 smithi082 ceph-mon[98694]: pgmap v2509: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 121 KiB/s wr, 24 op/s 2023-09-10T00:32:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:40 smithi098 ceph-mon[103124]: pgmap v2509: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 121 KiB/s wr, 24 op/s 2023-09-10T00:32:42.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:42 smithi170 ceph-mon[102832]: pgmap v2510: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 216 KiB/s wr, 31 op/s 2023-09-10T00:32:42.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4393.556533 secs 2023-09-10T00:32:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:42 smithi082 ceph-mon[98694]: pgmap v2510: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 216 KiB/s wr, 31 op/s 2023-09-10T00:32:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4393.556533 secs 2023-09-10T00:32:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:42 smithi098 ceph-mon[103124]: pgmap v2510: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 216 KiB/s wr, 31 op/s 2023-09-10T00:32:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4393.556533 secs 2023-09-10T00:32:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4395.092301 secs 2023-09-10T00:32:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4395.092301 secs 2023-09-10T00:32:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4395.092301 secs 2023-09-10T00:32:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4395.805352 secs 2023-09-10T00:32:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:44 smithi170 ceph-mon[102832]: pgmap v2511: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 154 KiB/s wr, 12 op/s 2023-09-10T00:32:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:32:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:32:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:32:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4395.805352 secs 2023-09-10T00:32:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:44 smithi082 ceph-mon[98694]: pgmap v2511: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 154 KiB/s wr, 12 op/s 2023-09-10T00:32:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:32:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:32:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:32:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4395.805352 secs 2023-09-10T00:32:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:44 smithi098 ceph-mon[103124]: pgmap v2511: 97 pgs: 97 active+clean; 8.9 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 154 KiB/s wr, 12 op/s 2023-09-10T00:32:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:32:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:32:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:32:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:46 smithi170 ceph-mon[102832]: pgmap v2512: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 198 KiB/s wr, 17 op/s 2023-09-10T00:32:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:46 smithi082 ceph-mon[98694]: pgmap v2512: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 198 KiB/s wr, 17 op/s 2023-09-10T00:32:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:46 smithi098 ceph-mon[103124]: pgmap v2512: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 198 KiB/s wr, 17 op/s 2023-09-10T00:32:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4398.556592 secs 2023-09-10T00:32:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4398.556592 secs 2023-09-10T00:32:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4398.556592 secs 2023-09-10T00:32:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:48 smithi170 ceph-mon[102832]: pgmap v2513: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T00:32:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4400.092385 secs 2023-09-10T00:32:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:48 smithi082 ceph-mon[98694]: pgmap v2513: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T00:32:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4400.092385 secs 2023-09-10T00:32:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:48 smithi098 ceph-mon[103124]: pgmap v2513: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T00:32:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4400.092385 secs 2023-09-10T00:32:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4400.805418 secs 2023-09-10T00:32:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4400.805418 secs 2023-09-10T00:32:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4400.805418 secs 2023-09-10T00:32:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:50 smithi170 ceph-mon[102832]: pgmap v2514: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T00:32:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:50 smithi082 ceph-mon[98694]: pgmap v2514: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T00:32:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:50 smithi098 ceph-mon[103124]: pgmap v2514: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T00:32:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:52 smithi170 ceph-mon[102832]: pgmap v2515: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 166 KiB/s wr, 16 op/s 2023-09-10T00:32:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4403.556685 secs 2023-09-10T00:32:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:52 smithi082 ceph-mon[98694]: pgmap v2515: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 166 KiB/s wr, 16 op/s 2023-09-10T00:32:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4403.556685 secs 2023-09-10T00:32:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:52 smithi098 ceph-mon[103124]: pgmap v2515: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 166 KiB/s wr, 16 op/s 2023-09-10T00:32:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4403.556685 secs 2023-09-10T00:32:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4405.092442 secs 2023-09-10T00:32:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4405.092442 secs 2023-09-10T00:32:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4405.092442 secs 2023-09-10T00:32:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4405.805485 secs 2023-09-10T00:32:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:54 smithi170 ceph-mon[102832]: pgmap v2516: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 71 KiB/s wr, 10 op/s 2023-09-10T00:32:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4405.805485 secs 2023-09-10T00:32:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:54 smithi082 ceph-mon[98694]: pgmap v2516: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 71 KiB/s wr, 10 op/s 2023-09-10T00:32:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4405.805485 secs 2023-09-10T00:32:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:54 smithi098 ceph-mon[103124]: pgmap v2516: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 71 KiB/s wr, 10 op/s 2023-09-10T00:32:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:56 smithi170 ceph-mon[102832]: pgmap v2517: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 88 KiB/s wr, 15 op/s 2023-09-10T00:32:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:56 smithi082 ceph-mon[98694]: pgmap v2517: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 88 KiB/s wr, 15 op/s 2023-09-10T00:32:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:56 smithi098 ceph-mon[103124]: pgmap v2517: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 88 KiB/s wr, 15 op/s 2023-09-10T00:32:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4408.556743 secs 2023-09-10T00:32:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4408.556743 secs 2023-09-10T00:32:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4408.556743 secs 2023-09-10T00:32:58.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:58 smithi170 ceph-mon[102832]: pgmap v2518: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T00:32:58.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4410.092501 secs 2023-09-10T00:32:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:58 smithi082 ceph-mon[98694]: pgmap v2518: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T00:32:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4410.092501 secs 2023-09-10T00:32:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:58 smithi098 ceph-mon[103124]: pgmap v2518: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T00:32:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4410.092501 secs 2023-09-10T00:32:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4410.805551 secs 2023-09-10T00:32:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:32:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:32:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4410.805551 secs 2023-09-10T00:32:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:32:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:33:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4410.805551 secs 2023-09-10T00:33:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:32:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:33:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:00 smithi170 ceph-mon[102832]: pgmap v2519: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T00:33:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:33:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:33:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:00 smithi082 ceph-mon[98694]: pgmap v2519: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T00:33:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:33:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:33:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:00 smithi098 ceph-mon[103124]: pgmap v2519: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T00:33:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:33:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:33:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:33:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:02 smithi170 ceph-mon[102832]: pgmap v2520: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 45 KiB/s wr, 15 op/s 2023-09-10T00:33:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4413.556776 secs 2023-09-10T00:33:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:02 smithi082 ceph-mon[98694]: pgmap v2520: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 45 KiB/s wr, 15 op/s 2023-09-10T00:33:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4413.556776 secs 2023-09-10T00:33:03.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:02 smithi098 ceph-mon[103124]: pgmap v2520: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 45 KiB/s wr, 15 op/s 2023-09-10T00:33:03.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4413.556776 secs 2023-09-10T00:33:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4415.092560 secs 2023-09-10T00:33:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4415.092560 secs 2023-09-10T00:33:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4415.092560 secs 2023-09-10T00:33:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4415.805617 secs 2023-09-10T00:33:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:04 smithi082 ceph-mon[98694]: pgmap v2521: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 KiB/s wr, 9 op/s 2023-09-10T00:33:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4415.805617 secs 2023-09-10T00:33:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:04 smithi098 ceph-mon[103124]: pgmap v2521: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 KiB/s wr, 9 op/s 2023-09-10T00:33:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4415.805617 secs 2023-09-10T00:33:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:04 smithi170 ceph-mon[102832]: pgmap v2521: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 KiB/s wr, 9 op/s 2023-09-10T00:33:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:06 smithi082 ceph-mon[98694]: pgmap v2522: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:33:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:06 smithi098 ceph-mon[103124]: pgmap v2522: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:33:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:06 smithi170 ceph-mon[102832]: pgmap v2522: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:33:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4418.556841 secs 2023-09-10T00:33:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4418.556841 secs 2023-09-10T00:33:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4418.556841 secs 2023-09-10T00:33:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:08 smithi082 ceph-mon[98694]: pgmap v2523: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 70 KiB/s wr, 24 op/s 2023-09-10T00:33:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:08 smithi082 ceph-mon[98694]: osdmap e229: 12 total, 12 up, 12 in 2023-09-10T00:33:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4420.092621 secs 2023-09-10T00:33:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:08 smithi098 ceph-mon[103124]: pgmap v2523: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 70 KiB/s wr, 24 op/s 2023-09-10T00:33:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:08 smithi098 ceph-mon[103124]: osdmap e229: 12 total, 12 up, 12 in 2023-09-10T00:33:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4420.092621 secs 2023-09-10T00:33:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:08 smithi170 ceph-mon[102832]: pgmap v2523: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 70 KiB/s wr, 24 op/s 2023-09-10T00:33:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:08 smithi170 ceph-mon[102832]: osdmap e229: 12 total, 12 up, 12 in 2023-09-10T00:33:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4420.092621 secs 2023-09-10T00:33:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4420.805685 secs 2023-09-10T00:33:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4420.805685 secs 2023-09-10T00:33:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4420.805685 secs 2023-09-10T00:33:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:10 smithi082 ceph-mon[98694]: pgmap v2525: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T00:33:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:10 smithi098 ceph-mon[103124]: pgmap v2525: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T00:33:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:10 smithi170 ceph-mon[102832]: pgmap v2525: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T00:33:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:12 smithi082 ceph-mon[98694]: pgmap v2526: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T00:33:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4423.556898 secs 2023-09-10T00:33:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:12 smithi098 ceph-mon[103124]: pgmap v2526: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T00:33:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4423.556898 secs 2023-09-10T00:33:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:12 smithi170 ceph-mon[102832]: pgmap v2526: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T00:33:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4423.556898 secs 2023-09-10T00:33:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4425.092678 secs 2023-09-10T00:33:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4425.092678 secs 2023-09-10T00:33:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4425.092678 secs 2023-09-10T00:33:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4425.805748 secs 2023-09-10T00:33:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:14 smithi098 ceph-mon[103124]: pgmap v2527: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T00:33:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:14 smithi098 ceph-mon[103124]: osdmap e230: 12 total, 12 up, 12 in 2023-09-10T00:33:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4425.805748 secs 2023-09-10T00:33:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:14 smithi170 ceph-mon[102832]: pgmap v2527: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T00:33:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:14 smithi170 ceph-mon[102832]: osdmap e230: 12 total, 12 up, 12 in 2023-09-10T00:33:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4425.805748 secs 2023-09-10T00:33:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:14 smithi082 ceph-mon[98694]: pgmap v2527: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T00:33:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:14 smithi082 ceph-mon[98694]: osdmap e230: 12 total, 12 up, 12 in 2023-09-10T00:33:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:16 smithi098 ceph-mon[103124]: pgmap v2529: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 47 KiB/s wr, 14 op/s 2023-09-10T00:33:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:16 smithi170 ceph-mon[102832]: pgmap v2529: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 47 KiB/s wr, 14 op/s 2023-09-10T00:33:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:16 smithi082 ceph-mon[98694]: pgmap v2529: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 47 KiB/s wr, 14 op/s 2023-09-10T00:33:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4428.556957 secs 2023-09-10T00:33:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4428.556957 secs 2023-09-10T00:33:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4428.556957 secs 2023-09-10T00:33:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:18 smithi098 ceph-mon[103124]: pgmap v2530: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T00:33:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4430.092735 secs 2023-09-10T00:33:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:18 smithi170 ceph-mon[102832]: pgmap v2530: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T00:33:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4430.092735 secs 2023-09-10T00:33:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:18 smithi082 ceph-mon[98694]: pgmap v2530: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T00:33:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4430.092735 secs 2023-09-10T00:33:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4430.805814 secs 2023-09-10T00:33:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4430.805814 secs 2023-09-10T00:33:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4430.805814 secs 2023-09-10T00:33:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:20 smithi098 ceph-mon[103124]: pgmap v2531: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T00:33:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:20 smithi170 ceph-mon[102832]: pgmap v2531: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T00:33:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:20 smithi082 ceph-mon[98694]: pgmap v2531: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T00:33:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:22 smithi098 ceph-mon[103124]: pgmap v2532: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T00:33:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4433.557019 secs 2023-09-10T00:33:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:22 smithi170 ceph-mon[102832]: pgmap v2532: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T00:33:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4433.557019 secs 2023-09-10T00:33:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:22 smithi082 ceph-mon[98694]: pgmap v2532: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T00:33:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4433.557019 secs 2023-09-10T00:33:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4435.092820 secs 2023-09-10T00:33:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4435.092820 secs 2023-09-10T00:33:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4435.092820 secs 2023-09-10T00:33:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4435.805879 secs 2023-09-10T00:33:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:24 smithi098 ceph-mon[103124]: pgmap v2533: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T00:33:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4435.805879 secs 2023-09-10T00:33:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:24 smithi170 ceph-mon[102832]: pgmap v2533: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T00:33:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4435.805879 secs 2023-09-10T00:33:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:24 smithi082 ceph-mon[98694]: pgmap v2533: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T00:33:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:26 smithi098 ceph-mon[103124]: pgmap v2534: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 98 KiB/s wr, 16 op/s 2023-09-10T00:33:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:26 smithi170 ceph-mon[102832]: pgmap v2534: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 98 KiB/s wr, 16 op/s 2023-09-10T00:33:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:26 smithi082 ceph-mon[98694]: pgmap v2534: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 98 KiB/s wr, 16 op/s 2023-09-10T00:33:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:28 smithi098 ceph-mon[103124]: pgmap v2535: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T00:33:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4440.092879 secs 2023-09-10T00:33:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:28 smithi170 ceph-mon[102832]: pgmap v2535: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T00:33:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4440.092879 secs 2023-09-10T00:33:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:28 smithi082 ceph-mon[98694]: pgmap v2535: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T00:33:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4440.092879 secs 2023-09-10T00:33:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4440.805947 secs 2023-09-10T00:33:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4440.805947 secs 2023-09-10T00:33:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4440.805947 secs 2023-09-10T00:33:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:30 smithi170 ceph-mon[102832]: pgmap v2536: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T00:33:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:30 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:33:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:30 smithi082 ceph-mon[98694]: pgmap v2536: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T00:33:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:30 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:33:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:30 smithi098 ceph-mon[103124]: pgmap v2536: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T00:33:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:30 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:33:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:31 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:33:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:33:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:33:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:31 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:33:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:33:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:33:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:31 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:33:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:33:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:33:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:32 smithi170 ceph-mon[102832]: pgmap v2537: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 143 KiB/s wr, 16 op/s 2023-09-10T00:33:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:32 smithi082 ceph-mon[98694]: pgmap v2537: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 143 KiB/s wr, 16 op/s 2023-09-10T00:33:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:32 smithi098 ceph-mon[103124]: pgmap v2537: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 143 KiB/s wr, 16 op/s 2023-09-10T00:33:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4445.092938 secs 2023-09-10T00:33:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4445.092938 secs 2023-09-10T00:33:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4445.092938 secs 2023-09-10T00:33:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4445.806009 secs 2023-09-10T00:33:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:34 smithi170 ceph-mon[102832]: pgmap v2538: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T00:33:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4445.806009 secs 2023-09-10T00:33:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:34 smithi082 ceph-mon[98694]: pgmap v2538: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T00:33:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4445.806009 secs 2023-09-10T00:33:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:34 smithi098 ceph-mon[103124]: pgmap v2538: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T00:33:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:36 smithi170 ceph-mon[102832]: pgmap v2539: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 171 KiB/s wr, 17 op/s 2023-09-10T00:33:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:36 smithi082 ceph-mon[98694]: pgmap v2539: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 171 KiB/s wr, 17 op/s 2023-09-10T00:33:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:36 smithi098 ceph-mon[103124]: pgmap v2539: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 171 KiB/s wr, 17 op/s 2023-09-10T00:33:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:38 smithi170 ceph-mon[102832]: pgmap v2540: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T00:33:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4450.092997 secs 2023-09-10T00:33:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:38 smithi082 ceph-mon[98694]: pgmap v2540: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T00:33:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4450.092997 secs 2023-09-10T00:33:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:38 smithi098 ceph-mon[103124]: pgmap v2540: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T00:33:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4450.092997 secs 2023-09-10T00:33:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4450.806075 secs 2023-09-10T00:33:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4450.806075 secs 2023-09-10T00:33:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4450.806075 secs 2023-09-10T00:33:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:40 smithi170 ceph-mon[102832]: pgmap v2541: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T00:33:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:40 smithi082 ceph-mon[98694]: pgmap v2541: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T00:33:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:40 smithi098 ceph-mon[103124]: pgmap v2541: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T00:33:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:42 smithi170 ceph-mon[102832]: pgmap v2542: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 196 KiB/s wr, 17 op/s 2023-09-10T00:33:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:42 smithi082 ceph-mon[98694]: pgmap v2542: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 196 KiB/s wr, 17 op/s 2023-09-10T00:33:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:42 smithi098 ceph-mon[103124]: pgmap v2542: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 196 KiB/s wr, 17 op/s 2023-09-10T00:33:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4455.093061 secs 2023-09-10T00:33:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4455.093061 secs 2023-09-10T00:33:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4455.093061 secs 2023-09-10T00:33:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4455.806145 secs 2023-09-10T00:33:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:44 smithi170 ceph-mon[102832]: pgmap v2543: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T00:33:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4455.806145 secs 2023-09-10T00:33:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:44 smithi082 ceph-mon[98694]: pgmap v2543: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T00:33:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4455.806145 secs 2023-09-10T00:33:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:44 smithi098 ceph-mon[103124]: pgmap v2543: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T00:33:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:46 smithi170 ceph-mon[102832]: pgmap v2544: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 148 KiB/s wr, 17 op/s 2023-09-10T00:33:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:46 smithi082 ceph-mon[98694]: pgmap v2544: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 148 KiB/s wr, 17 op/s 2023-09-10T00:33:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:46 smithi098 ceph-mon[103124]: pgmap v2544: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 148 KiB/s wr, 17 op/s 2023-09-10T00:33:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4458.557418 secs 2023-09-10T00:33:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4458.557418 secs 2023-09-10T00:33:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4458.557418 secs 2023-09-10T00:33:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:48 smithi170 ceph-mon[102832]: pgmap v2545: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T00:33:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4460.093118 secs 2023-09-10T00:33:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:48 smithi082 ceph-mon[98694]: pgmap v2545: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T00:33:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4460.093118 secs 2023-09-10T00:33:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:48 smithi098 ceph-mon[103124]: pgmap v2545: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T00:33:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4460.093118 secs 2023-09-10T00:33:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4460.806213 secs 2023-09-10T00:33:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4460.806213 secs 2023-09-10T00:33:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4460.806213 secs 2023-09-10T00:33:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:50 smithi170 ceph-mon[102832]: pgmap v2546: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T00:33:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:50 smithi082 ceph-mon[98694]: pgmap v2546: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T00:33:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:50 smithi098 ceph-mon[103124]: pgmap v2546: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T00:33:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:33:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:33:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:33:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:33:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:33:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:33:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:33:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:33:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:33:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:52 smithi170 ceph-mon[102832]: pgmap v2547: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T00:33:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4463.557531 secs 2023-09-10T00:33:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:52 smithi082 ceph-mon[98694]: pgmap v2547: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T00:33:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4463.557531 secs 2023-09-10T00:33:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:52 smithi098 ceph-mon[103124]: pgmap v2547: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T00:33:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4463.557531 secs 2023-09-10T00:33:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4465.093176 secs 2023-09-10T00:33:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4465.093176 secs 2023-09-10T00:33:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4465.093176 secs 2023-09-10T00:33:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4465.806282 secs 2023-09-10T00:33:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:54 smithi170 ceph-mon[102832]: pgmap v2548: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T00:33:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4465.806282 secs 2023-09-10T00:33:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:54 smithi082 ceph-mon[98694]: pgmap v2548: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T00:33:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4465.806282 secs 2023-09-10T00:33:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:54 smithi098 ceph-mon[103124]: pgmap v2548: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T00:33:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:56 smithi170 ceph-mon[102832]: pgmap v2549: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 94 KiB/s wr, 16 op/s 2023-09-10T00:33:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:56 smithi082 ceph-mon[98694]: pgmap v2549: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 94 KiB/s wr, 16 op/s 2023-09-10T00:33:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:56 smithi098 ceph-mon[103124]: pgmap v2549: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 94 KiB/s wr, 16 op/s 2023-09-10T00:33:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4468.557590 secs 2023-09-10T00:33:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4468.557590 secs 2023-09-10T00:33:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4468.557590 secs 2023-09-10T00:33:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:58 smithi170 ceph-mon[102832]: pgmap v2550: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T00:33:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4470.093238 secs 2023-09-10T00:33:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:58 smithi082 ceph-mon[98694]: pgmap v2550: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T00:33:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4470.093238 secs 2023-09-10T00:33:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:58 smithi098 ceph-mon[103124]: pgmap v2550: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T00:33:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4470.093238 secs 2023-09-10T00:34:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:33:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4470.806349 secs 2023-09-10T00:34:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:33:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4470.806349 secs 2023-09-10T00:34:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:33:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4470.806349 secs 2023-09-10T00:34:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:00 smithi170 ceph-mon[102832]: pgmap v2551: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T00:34:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:34:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:34:01.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:34:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:00 smithi082 ceph-mon[98694]: pgmap v2551: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T00:34:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:34:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:34:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:34:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:00 smithi098 ceph-mon[103124]: pgmap v2551: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T00:34:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:34:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:34:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:34:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:34:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:02 smithi170 ceph-mon[102832]: pgmap v2552: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 15 op/s 2023-09-10T00:34:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4473.557704 secs 2023-09-10T00:34:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:02 smithi082 ceph-mon[98694]: pgmap v2552: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 15 op/s 2023-09-10T00:34:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4473.557704 secs 2023-09-10T00:34:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:02 smithi098 ceph-mon[103124]: pgmap v2552: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 15 op/s 2023-09-10T00:34:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4473.557704 secs 2023-09-10T00:34:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4475.093295 secs 2023-09-10T00:34:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4475.093295 secs 2023-09-10T00:34:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4475.093295 secs 2023-09-10T00:34:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4475.806414 secs 2023-09-10T00:34:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:04 smithi170 ceph-mon[102832]: pgmap v2553: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 25 KiB/s wr, 10 op/s 2023-09-10T00:34:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4475.806414 secs 2023-09-10T00:34:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:04 smithi082 ceph-mon[98694]: pgmap v2553: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 25 KiB/s wr, 10 op/s 2023-09-10T00:34:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4475.806414 secs 2023-09-10T00:34:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:04 smithi098 ceph-mon[103124]: pgmap v2553: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 25 KiB/s wr, 10 op/s 2023-09-10T00:34:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:06 smithi170 ceph-mon[102832]: pgmap v2554: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 115 KiB/s wr, 29 op/s 2023-09-10T00:34:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:06 smithi082 ceph-mon[98694]: pgmap v2554: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 115 KiB/s wr, 29 op/s 2023-09-10T00:34:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:06 smithi098 ceph-mon[103124]: pgmap v2554: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 115 KiB/s wr, 29 op/s 2023-09-10T00:34:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4478.557769 secs 2023-09-10T00:34:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4478.557769 secs 2023-09-10T00:34:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4478.557769 secs 2023-09-10T00:34:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:08 smithi170 ceph-mon[102832]: pgmap v2555: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 91 KiB/s wr, 24 op/s 2023-09-10T00:34:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4480.093355 secs 2023-09-10T00:34:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:08 smithi170 ceph-mon[102832]: osdmap e231: 12 total, 12 up, 12 in 2023-09-10T00:34:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:08 smithi082 ceph-mon[98694]: pgmap v2555: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 91 KiB/s wr, 24 op/s 2023-09-10T00:34:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4480.093355 secs 2023-09-10T00:34:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:08 smithi082 ceph-mon[98694]: osdmap e231: 12 total, 12 up, 12 in 2023-09-10T00:34:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:08 smithi098 ceph-mon[103124]: pgmap v2555: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 91 KiB/s wr, 24 op/s 2023-09-10T00:34:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4480.093355 secs 2023-09-10T00:34:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:08 smithi098 ceph-mon[103124]: osdmap e231: 12 total, 12 up, 12 in 2023-09-10T00:34:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4480.806494 secs 2023-09-10T00:34:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4480.806494 secs 2023-09-10T00:34:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4480.806494 secs 2023-09-10T00:34:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:10 smithi170 ceph-mon[102832]: pgmap v2557: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T00:34:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:10 smithi082 ceph-mon[98694]: pgmap v2557: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T00:34:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:10 smithi098 ceph-mon[103124]: pgmap v2557: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T00:34:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:12 smithi170 ceph-mon[102832]: pgmap v2558: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T00:34:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4483.557884 secs 2023-09-10T00:34:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:12 smithi082 ceph-mon[98694]: pgmap v2558: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T00:34:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4483.557884 secs 2023-09-10T00:34:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:12 smithi098 ceph-mon[103124]: pgmap v2558: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T00:34:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4483.557884 secs 2023-09-10T00:34:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4485.093415 secs 2023-09-10T00:34:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4485.093415 secs 2023-09-10T00:34:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4485.093415 secs 2023-09-10T00:34:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4485.806558 secs 2023-09-10T00:34:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:14 smithi170 ceph-mon[102832]: pgmap v2559: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T00:34:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:14 smithi170 ceph-mon[102832]: osdmap e232: 12 total, 12 up, 12 in 2023-09-10T00:34:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4485.806558 secs 2023-09-10T00:34:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:14 smithi082 ceph-mon[98694]: pgmap v2559: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T00:34:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:14 smithi082 ceph-mon[98694]: osdmap e232: 12 total, 12 up, 12 in 2023-09-10T00:34:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4485.806558 secs 2023-09-10T00:34:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:14 smithi098 ceph-mon[103124]: pgmap v2559: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T00:34:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:14 smithi098 ceph-mon[103124]: osdmap e232: 12 total, 12 up, 12 in 2023-09-10T00:34:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:16 smithi170 ceph-mon[102832]: pgmap v2561: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 KiB/s wr, 14 op/s 2023-09-10T00:34:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:16 smithi082 ceph-mon[98694]: pgmap v2561: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 KiB/s wr, 14 op/s 2023-09-10T00:34:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:16 smithi098 ceph-mon[103124]: pgmap v2561: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 KiB/s wr, 14 op/s 2023-09-10T00:34:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:18 smithi170 ceph-mon[102832]: pgmap v2562: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T00:34:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4490.093474 secs 2023-09-10T00:34:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:18 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:34:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:18 smithi082 ceph-mon[98694]: pgmap v2562: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T00:34:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4490.093474 secs 2023-09-10T00:34:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:18 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:34:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:18 smithi098 ceph-mon[103124]: pgmap v2562: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T00:34:19.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4490.093474 secs 2023-09-10T00:34:19.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:18 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:34:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4490.806648 secs 2023-09-10T00:34:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:19 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:34:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:19 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:34:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:19 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:34:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4490.806648 secs 2023-09-10T00:34:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:19 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:34:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:19 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:34:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:19 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:34:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4490.806648 secs 2023-09-10T00:34:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:19 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:34:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:19 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:34:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:19 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:34:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:20 smithi170 ceph-mon[102832]: pgmap v2563: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T00:34:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:20 smithi082 ceph-mon[98694]: pgmap v2563: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T00:34:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:20 smithi098 ceph-mon[103124]: pgmap v2563: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T00:34:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:22 smithi170 ceph-mon[102832]: pgmap v2564: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T00:34:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4493.558105 secs 2023-09-10T00:34:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:22 smithi082 ceph-mon[98694]: pgmap v2564: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T00:34:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4493.558105 secs 2023-09-10T00:34:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:22 smithi098 ceph-mon[103124]: pgmap v2564: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T00:34:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4493.558105 secs 2023-09-10T00:34:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4495.093535 secs 2023-09-10T00:34:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:34:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:34:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4495.093535 secs 2023-09-10T00:34:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:34:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:34:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4495.093535 secs 2023-09-10T00:34:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:34:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:34:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4495.806712 secs 2023-09-10T00:34:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:24 smithi170 ceph-mon[102832]: pgmap v2565: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T00:34:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:24 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:34:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4495.806712 secs 2023-09-10T00:34:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:24 smithi082 ceph-mon[98694]: pgmap v2565: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T00:34:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:24 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:34:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4495.806712 secs 2023-09-10T00:34:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:24 smithi098 ceph-mon[103124]: pgmap v2565: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T00:34:25.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:24 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:34:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:26 smithi170 ceph-mon[102832]: pgmap v2566: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 130 KiB/s wr, 18 op/s 2023-09-10T00:34:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:26 smithi082 ceph-mon[98694]: pgmap v2566: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 130 KiB/s wr, 18 op/s 2023-09-10T00:34:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:26 smithi098 ceph-mon[103124]: pgmap v2566: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 130 KiB/s wr, 18 op/s 2023-09-10T00:34:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4498.558185 secs 2023-09-10T00:34:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4498.558185 secs 2023-09-10T00:34:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4498.558185 secs 2023-09-10T00:34:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:28 smithi170 ceph-mon[102832]: pgmap v2567: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T00:34:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4500.093620 secs 2023-09-10T00:34:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:28 smithi082 ceph-mon[98694]: pgmap v2567: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T00:34:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4500.093620 secs 2023-09-10T00:34:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:28 smithi098 ceph-mon[103124]: pgmap v2567: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T00:34:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4500.093620 secs 2023-09-10T00:34:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4500.806783 secs 2023-09-10T00:34:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4500.806783 secs 2023-09-10T00:34:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4500.806783 secs 2023-09-10T00:34:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:30 smithi170 ceph-mon[102832]: pgmap v2568: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T00:34:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:30 smithi082 ceph-mon[98694]: pgmap v2568: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T00:34:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:30 smithi098 ceph-mon[103124]: pgmap v2568: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T00:34:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:32 smithi170 ceph-mon[102832]: pgmap v2569: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 214 KiB/s wr, 18 op/s 2023-09-10T00:34:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:32 smithi082 ceph-mon[98694]: pgmap v2569: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 214 KiB/s wr, 18 op/s 2023-09-10T00:34:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:32 smithi098 ceph-mon[103124]: pgmap v2569: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 214 KiB/s wr, 18 op/s 2023-09-10T00:34:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4505.093688 secs 2023-09-10T00:34:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4505.093688 secs 2023-09-10T00:34:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4505.093688 secs 2023-09-10T00:34:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4505.806851 secs 2023-09-10T00:34:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:34 smithi082 ceph-mon[98694]: pgmap v2570: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T00:34:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:34 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:34:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4505.806851 secs 2023-09-10T00:34:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:34 smithi098 ceph-mon[103124]: pgmap v2570: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T00:34:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:34 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:34:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4505.806851 secs 2023-09-10T00:34:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:34 smithi170 ceph-mon[102832]: pgmap v2570: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T00:34:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:34 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:34:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:35 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:34:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:35 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:34:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:35 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:34:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:35 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:34:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:35 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:34:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:35 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:34:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:35 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:34:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:35 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:34:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:35 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:34:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:36 smithi082 ceph-mon[98694]: pgmap v2571: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 179 KiB/s wr, 18 op/s 2023-09-10T00:34:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:36 smithi098 ceph-mon[103124]: pgmap v2571: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 179 KiB/s wr, 18 op/s 2023-09-10T00:34:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:36 smithi170 ceph-mon[102832]: pgmap v2571: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 179 KiB/s wr, 18 op/s 2023-09-10T00:34:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:38 smithi082 ceph-mon[98694]: pgmap v2572: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T00:34:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4510.093744 secs 2023-09-10T00:34:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:38 smithi098 ceph-mon[103124]: pgmap v2572: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T00:34:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4510.093744 secs 2023-09-10T00:34:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:38 smithi170 ceph-mon[102832]: pgmap v2572: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T00:34:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4510.093744 secs 2023-09-10T00:34:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4510.806937 secs 2023-09-10T00:34:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4510.806937 secs 2023-09-10T00:34:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4510.806937 secs 2023-09-10T00:34:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:40 smithi082 ceph-mon[98694]: pgmap v2573: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T00:34:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:40 smithi098 ceph-mon[103124]: pgmap v2573: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T00:34:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:40 smithi170 ceph-mon[102832]: pgmap v2573: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T00:34:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:42 smithi082 ceph-mon[98694]: pgmap v2574: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 166 KiB/s wr, 17 op/s 2023-09-10T00:34:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:42 smithi098 ceph-mon[103124]: pgmap v2574: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 166 KiB/s wr, 17 op/s 2023-09-10T00:34:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:42 smithi170 ceph-mon[102832]: pgmap v2574: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 166 KiB/s wr, 17 op/s 2023-09-10T00:34:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4515.093806 secs 2023-09-10T00:34:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4515.093806 secs 2023-09-10T00:34:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4515.093806 secs 2023-09-10T00:34:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4515.807002 secs 2023-09-10T00:34:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:44 smithi098 ceph-mon[103124]: pgmap v2575: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T00:34:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4515.807002 secs 2023-09-10T00:34:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:44 smithi170 ceph-mon[102832]: pgmap v2575: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T00:34:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4515.807002 secs 2023-09-10T00:34:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:44 smithi082 ceph-mon[98694]: pgmap v2575: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T00:34:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:46 smithi098 ceph-mon[103124]: pgmap v2576: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 145 KiB/s wr, 16 op/s 2023-09-10T00:34:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:46 smithi170 ceph-mon[102832]: pgmap v2576: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 145 KiB/s wr, 16 op/s 2023-09-10T00:34:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:46 smithi082 ceph-mon[98694]: pgmap v2576: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 145 KiB/s wr, 16 op/s 2023-09-10T00:34:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4518.558558 secs 2023-09-10T00:34:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4518.558558 secs 2023-09-10T00:34:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4518.558558 secs 2023-09-10T00:34:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:48 smithi098 ceph-mon[103124]: pgmap v2577: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T00:34:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4520.093866 secs 2023-09-10T00:34:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:48 smithi170 ceph-mon[102832]: pgmap v2577: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T00:34:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4520.093866 secs 2023-09-10T00:34:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:48 smithi082 ceph-mon[98694]: pgmap v2577: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T00:34:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4520.093866 secs 2023-09-10T00:34:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4520.807067 secs 2023-09-10T00:34:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4520.807067 secs 2023-09-10T00:34:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4520.807067 secs 2023-09-10T00:34:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:50 smithi098 ceph-mon[103124]: pgmap v2578: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T00:34:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:50 smithi170 ceph-mon[102832]: pgmap v2578: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T00:34:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:50 smithi082 ceph-mon[98694]: pgmap v2578: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T00:34:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:34:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:34:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:34:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:34:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:34:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:34:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:34:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:34:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:34:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:52 smithi098 ceph-mon[103124]: pgmap v2579: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 141 KiB/s wr, 16 op/s 2023-09-10T00:34:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4523.558618 secs 2023-09-10T00:34:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:52 smithi170 ceph-mon[102832]: pgmap v2579: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 141 KiB/s wr, 16 op/s 2023-09-10T00:34:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4523.558618 secs 2023-09-10T00:34:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:52 smithi082 ceph-mon[98694]: pgmap v2579: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 141 KiB/s wr, 16 op/s 2023-09-10T00:34:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4523.558618 secs 2023-09-10T00:34:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4525.093944 secs 2023-09-10T00:34:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4525.093944 secs 2023-09-10T00:34:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4525.093944 secs 2023-09-10T00:34:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4525.807135 secs 2023-09-10T00:34:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:55 smithi098 ceph-mon[103124]: pgmap v2580: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T00:34:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4525.807135 secs 2023-09-10T00:34:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:55 smithi170 ceph-mon[102832]: pgmap v2580: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T00:34:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4525.807135 secs 2023-09-10T00:34:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:55 smithi082 ceph-mon[98694]: pgmap v2580: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T00:34:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:57 smithi098 ceph-mon[103124]: pgmap v2581: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 144 KiB/s wr, 16 op/s 2023-09-10T00:34:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:57 smithi170 ceph-mon[102832]: pgmap v2581: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 144 KiB/s wr, 16 op/s 2023-09-10T00:34:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:57 smithi082 ceph-mon[98694]: pgmap v2581: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 144 KiB/s wr, 16 op/s 2023-09-10T00:34:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4528.558694 secs 2023-09-10T00:34:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4528.558694 secs 2023-09-10T00:34:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4528.558694 secs 2023-09-10T00:34:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:59 smithi170 ceph-mon[102832]: pgmap v2582: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T00:34:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:34:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4530.094029 secs 2023-09-10T00:34:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:59 smithi082 ceph-mon[98694]: pgmap v2582: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T00:34:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:34:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4530.094029 secs 2023-09-10T00:34:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:59 smithi098 ceph-mon[103124]: pgmap v2582: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T00:34:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:34:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4530.094029 secs 2023-09-10T00:35:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4530.807204 secs 2023-09-10T00:35:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4530.807204 secs 2023-09-10T00:35:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4530.807204 secs 2023-09-10T00:35:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:01 smithi170 ceph-mon[102832]: pgmap v2583: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T00:35:01.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:35:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:01 smithi082 ceph-mon[98694]: pgmap v2583: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T00:35:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:35:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:01 smithi098 ceph-mon[103124]: pgmap v2583: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T00:35:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:35:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:35:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:35:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:35:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:35:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:35:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:35:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:35:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:03 smithi082 ceph-mon[98694]: pgmap v2584: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 116 KiB/s wr, 30 op/s 2023-09-10T00:35:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4533.558753 secs 2023-09-10T00:35:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:03 smithi082 ceph-mon[98694]: osdmap e233: 12 total, 12 up, 12 in 2023-09-10T00:35:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:03 smithi098 ceph-mon[103124]: pgmap v2584: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 116 KiB/s wr, 30 op/s 2023-09-10T00:35:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4533.558753 secs 2023-09-10T00:35:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:03 smithi098 ceph-mon[103124]: osdmap e233: 12 total, 12 up, 12 in 2023-09-10T00:35:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:03 smithi170 ceph-mon[102832]: pgmap v2584: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 116 KiB/s wr, 30 op/s 2023-09-10T00:35:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4533.558753 secs 2023-09-10T00:35:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:03 smithi170 ceph-mon[102832]: osdmap e233: 12 total, 12 up, 12 in 2023-09-10T00:35:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4535.094110 secs 2023-09-10T00:35:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4535.094110 secs 2023-09-10T00:35:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4535.094110 secs 2023-09-10T00:35:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4535.807275 secs 2023-09-10T00:35:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:05 smithi082 ceph-mon[98694]: pgmap v2586: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T00:35:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4535.807275 secs 2023-09-10T00:35:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:05 smithi098 ceph-mon[103124]: pgmap v2586: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T00:35:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4535.807275 secs 2023-09-10T00:35:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:05 smithi170 ceph-mon[102832]: pgmap v2586: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T00:35:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:07 smithi098 ceph-mon[103124]: pgmap v2587: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T00:35:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:07 smithi170 ceph-mon[102832]: pgmap v2587: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T00:35:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:07 smithi082 ceph-mon[98694]: pgmap v2587: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T00:35:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4538.558775 secs 2023-09-10T00:35:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4538.558775 secs 2023-09-10T00:35:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4538.558775 secs 2023-09-10T00:35:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:09 smithi098 ceph-mon[103124]: pgmap v2588: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T00:35:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4540.094188 secs 2023-09-10T00:35:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:09 smithi170 ceph-mon[102832]: pgmap v2588: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T00:35:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4540.094188 secs 2023-09-10T00:35:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:09 smithi082 ceph-mon[98694]: pgmap v2588: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T00:35:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4540.094188 secs 2023-09-10T00:35:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4540.807338 secs 2023-09-10T00:35:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:10 smithi098 ceph-mon[103124]: osdmap e234: 12 total, 12 up, 12 in 2023-09-10T00:35:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4540.807338 secs 2023-09-10T00:35:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:10 smithi170 ceph-mon[102832]: osdmap e234: 12 total, 12 up, 12 in 2023-09-10T00:35:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4540.807338 secs 2023-09-10T00:35:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:10 smithi082 ceph-mon[98694]: osdmap e234: 12 total, 12 up, 12 in 2023-09-10T00:35:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:11 smithi098 ceph-mon[103124]: pgmap v2589: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T00:35:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:11 smithi170 ceph-mon[102832]: pgmap v2589: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T00:35:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:11 smithi082 ceph-mon[98694]: pgmap v2589: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T00:35:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:13 smithi098 ceph-mon[103124]: pgmap v2591: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 13 op/s 2023-09-10T00:35:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4543.558835 secs 2023-09-10T00:35:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:13 smithi170 ceph-mon[102832]: pgmap v2591: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 13 op/s 2023-09-10T00:35:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4543.558835 secs 2023-09-10T00:35:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:13 smithi082 ceph-mon[98694]: pgmap v2591: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 13 op/s 2023-09-10T00:35:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4543.558835 secs 2023-09-10T00:35:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4545.094263 secs 2023-09-10T00:35:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4545.094263 secs 2023-09-10T00:35:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4545.094263 secs 2023-09-10T00:35:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4545.807407 secs 2023-09-10T00:35:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:15 smithi098 ceph-mon[103124]: pgmap v2592: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 55 KiB/s wr, 12 op/s 2023-09-10T00:35:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4545.807407 secs 2023-09-10T00:35:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:15 smithi170 ceph-mon[102832]: pgmap v2592: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 55 KiB/s wr, 12 op/s 2023-09-10T00:35:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4545.807407 secs 2023-09-10T00:35:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:15 smithi082 ceph-mon[98694]: pgmap v2592: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 55 KiB/s wr, 12 op/s 2023-09-10T00:35:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:17 smithi098 ceph-mon[103124]: pgmap v2593: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 12 op/s 2023-09-10T00:35:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:17 smithi170 ceph-mon[102832]: pgmap v2593: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 12 op/s 2023-09-10T00:35:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:17 smithi082 ceph-mon[98694]: pgmap v2593: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 12 op/s 2023-09-10T00:35:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:19 smithi098 ceph-mon[103124]: pgmap v2594: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 12 op/s 2023-09-10T00:35:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4550.094319 secs 2023-09-10T00:35:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:35:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:19 smithi170 ceph-mon[102832]: pgmap v2594: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 12 op/s 2023-09-10T00:35:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4550.094319 secs 2023-09-10T00:35:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:35:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:19 smithi082 ceph-mon[98694]: pgmap v2594: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 12 op/s 2023-09-10T00:35:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4550.094319 secs 2023-09-10T00:35:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:35:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4550.807473 secs 2023-09-10T00:35:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:35:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:35:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:35:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4550.807473 secs 2023-09-10T00:35:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:35:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:35:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:35:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4550.807473 secs 2023-09-10T00:35:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:35:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:35:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:35:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:21 smithi098 ceph-mon[103124]: pgmap v2595: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 12 op/s 2023-09-10T00:35:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:21 smithi170 ceph-mon[102832]: pgmap v2595: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 12 op/s 2023-09-10T00:35:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:21 smithi082 ceph-mon[98694]: pgmap v2595: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 12 op/s 2023-09-10T00:35:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:23 smithi098 ceph-mon[103124]: pgmap v2596: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 74 KiB/s wr, 16 op/s 2023-09-10T00:35:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:23 smithi170 ceph-mon[102832]: pgmap v2596: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 74 KiB/s wr, 16 op/s 2023-09-10T00:35:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:23 smithi082 ceph-mon[98694]: pgmap v2596: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 74 KiB/s wr, 16 op/s 2023-09-10T00:35:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4555.094407 secs 2023-09-10T00:35:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4555.094407 secs 2023-09-10T00:35:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4555.094407 secs 2023-09-10T00:35:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4555.807541 secs 2023-09-10T00:35:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:25 smithi170 ceph-mon[102832]: pgmap v2597: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T00:35:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4555.807541 secs 2023-09-10T00:35:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:25 smithi082 ceph-mon[98694]: pgmap v2597: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T00:35:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4555.807541 secs 2023-09-10T00:35:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:25 smithi098 ceph-mon[103124]: pgmap v2597: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T00:35:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:26 smithi170 ceph-mon[102832]: pgmap v2598: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 155 KiB/s wr, 17 op/s 2023-09-10T00:35:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:26 smithi082 ceph-mon[98694]: pgmap v2598: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 155 KiB/s wr, 17 op/s 2023-09-10T00:35:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:26 smithi098 ceph-mon[103124]: pgmap v2598: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 155 KiB/s wr, 17 op/s 2023-09-10T00:35:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:28 smithi082 ceph-mon[98694]: pgmap v2599: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T00:35:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4560.094465 secs 2023-09-10T00:35:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:28 smithi098 ceph-mon[103124]: pgmap v2599: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T00:35:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4560.094465 secs 2023-09-10T00:35:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:28 smithi170 ceph-mon[102832]: pgmap v2599: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T00:35:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4560.094465 secs 2023-09-10T00:35:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4560.807606 secs 2023-09-10T00:35:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4560.807606 secs 2023-09-10T00:35:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4560.807606 secs 2023-09-10T00:35:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:30 smithi082 ceph-mon[98694]: pgmap v2600: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T00:35:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:30 smithi098 ceph-mon[103124]: pgmap v2600: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T00:35:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:30 smithi170 ceph-mon[102832]: pgmap v2600: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T00:35:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:32 smithi082 ceph-mon[98694]: pgmap v2601: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 223 KiB/s wr, 18 op/s 2023-09-10T00:35:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4563.559190 secs 2023-09-10T00:35:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:32 smithi098 ceph-mon[103124]: pgmap v2601: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 223 KiB/s wr, 18 op/s 2023-09-10T00:35:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4563.559190 secs 2023-09-10T00:35:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:32 smithi170 ceph-mon[102832]: pgmap v2601: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 223 KiB/s wr, 18 op/s 2023-09-10T00:35:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4563.559190 secs 2023-09-10T00:35:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4565.094547 secs 2023-09-10T00:35:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4565.094547 secs 2023-09-10T00:35:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4565.094547 secs 2023-09-10T00:35:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4565.807673 secs 2023-09-10T00:35:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:34 smithi082 ceph-mon[98694]: pgmap v2602: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 188 KiB/s wr, 12 op/s 2023-09-10T00:35:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4565.807673 secs 2023-09-10T00:35:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:34 smithi098 ceph-mon[103124]: pgmap v2602: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 188 KiB/s wr, 12 op/s 2023-09-10T00:35:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4565.807673 secs 2023-09-10T00:35:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:34 smithi170 ceph-mon[102832]: pgmap v2602: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 188 KiB/s wr, 12 op/s 2023-09-10T00:35:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:35 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:35:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:35 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:35:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:35 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:35:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:35 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:35:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:35 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:35:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:35 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:35:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:35 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:35:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:35 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:35:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:35 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:35:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:36 smithi098 ceph-mon[103124]: pgmap v2603: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 217 KiB/s wr, 18 op/s 2023-09-10T00:35:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:36 smithi170 ceph-mon[102832]: pgmap v2603: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 217 KiB/s wr, 18 op/s 2023-09-10T00:35:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:36 smithi082 ceph-mon[98694]: pgmap v2603: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 217 KiB/s wr, 18 op/s 2023-09-10T00:35:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4568.559289 secs 2023-09-10T00:35:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4568.559289 secs 2023-09-10T00:35:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4568.559289 secs 2023-09-10T00:35:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:38 smithi098 ceph-mon[103124]: pgmap v2604: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T00:35:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4570.094608 secs 2023-09-10T00:35:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:38 smithi170 ceph-mon[102832]: pgmap v2604: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T00:35:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4570.094608 secs 2023-09-10T00:35:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:38 smithi082 ceph-mon[98694]: pgmap v2604: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T00:35:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4570.094608 secs 2023-09-10T00:35:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4570.807739 secs 2023-09-10T00:35:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4570.807739 secs 2023-09-10T00:35:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4570.807739 secs 2023-09-10T00:35:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:40 smithi098 ceph-mon[103124]: pgmap v2605: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T00:35:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:40 smithi170 ceph-mon[102832]: pgmap v2605: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T00:35:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:40 smithi082 ceph-mon[98694]: pgmap v2605: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T00:35:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:42 smithi098 ceph-mon[103124]: pgmap v2606: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 131 KiB/s wr, 17 op/s 2023-09-10T00:35:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4573.559349 secs 2023-09-10T00:35:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:42 smithi170 ceph-mon[102832]: pgmap v2606: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 131 KiB/s wr, 17 op/s 2023-09-10T00:35:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4573.559349 secs 2023-09-10T00:35:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:42 smithi082 ceph-mon[98694]: pgmap v2606: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 131 KiB/s wr, 17 op/s 2023-09-10T00:35:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4573.559349 secs 2023-09-10T00:35:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4575.094689 secs 2023-09-10T00:35:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4575.094689 secs 2023-09-10T00:35:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4575.094689 secs 2023-09-10T00:35:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4575.807803 secs 2023-09-10T00:35:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:44 smithi098 ceph-mon[103124]: pgmap v2607: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T00:35:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4575.807803 secs 2023-09-10T00:35:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:44 smithi170 ceph-mon[102832]: pgmap v2607: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T00:35:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4575.807803 secs 2023-09-10T00:35:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:44 smithi082 ceph-mon[98694]: pgmap v2607: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T00:35:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:46 smithi098 ceph-mon[103124]: pgmap v2608: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 129 KiB/s wr, 16 op/s 2023-09-10T00:35:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:46 smithi170 ceph-mon[102832]: pgmap v2608: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 129 KiB/s wr, 16 op/s 2023-09-10T00:35:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:46 smithi082 ceph-mon[98694]: pgmap v2608: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 129 KiB/s wr, 16 op/s 2023-09-10T00:35:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4578.559408 secs 2023-09-10T00:35:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4578.559408 secs 2023-09-10T00:35:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4578.559408 secs 2023-09-10T00:35:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:48 smithi098 ceph-mon[103124]: pgmap v2609: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T00:35:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4580.094748 secs 2023-09-10T00:35:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:48 smithi170 ceph-mon[102832]: pgmap v2609: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T00:35:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4580.094748 secs 2023-09-10T00:35:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:48 smithi082 ceph-mon[98694]: pgmap v2609: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T00:35:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4580.094748 secs 2023-09-10T00:35:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4580.807870 secs 2023-09-10T00:35:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4580.807870 secs 2023-09-10T00:35:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4580.807870 secs 2023-09-10T00:35:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:50 smithi098 ceph-mon[103124]: pgmap v2610: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T00:35:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:50 smithi170 ceph-mon[102832]: pgmap v2610: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T00:35:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:50 smithi082 ceph-mon[98694]: pgmap v2610: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T00:35:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:52 smithi170 ceph-mon[102832]: pgmap v2611: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 128 KiB/s wr, 16 op/s 2023-09-10T00:35:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4583.559532 secs 2023-09-10T00:35:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:52 smithi082 ceph-mon[98694]: pgmap v2611: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 128 KiB/s wr, 16 op/s 2023-09-10T00:35:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4583.559532 secs 2023-09-10T00:35:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:52 smithi098 ceph-mon[103124]: pgmap v2611: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 128 KiB/s wr, 16 op/s 2023-09-10T00:35:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4583.559532 secs 2023-09-10T00:35:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4585.094834 secs 2023-09-10T00:35:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4585.094834 secs 2023-09-10T00:35:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4585.094834 secs 2023-09-10T00:35:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4585.807961 secs 2023-09-10T00:35:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:54 smithi170 ceph-mon[102832]: pgmap v2612: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T00:35:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4585.807961 secs 2023-09-10T00:35:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:54 smithi082 ceph-mon[98694]: pgmap v2612: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T00:35:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4585.807961 secs 2023-09-10T00:35:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:54 smithi098 ceph-mon[103124]: pgmap v2612: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T00:35:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:56 smithi170 ceph-mon[102832]: pgmap v2613: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 123 KiB/s wr, 16 op/s 2023-09-10T00:35:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:56 smithi082 ceph-mon[98694]: pgmap v2613: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 123 KiB/s wr, 16 op/s 2023-09-10T00:35:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:56 smithi098 ceph-mon[103124]: pgmap v2613: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 123 KiB/s wr, 16 op/s 2023-09-10T00:35:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4588.559658 secs 2023-09-10T00:35:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4588.559658 secs 2023-09-10T00:35:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4588.559658 secs 2023-09-10T00:35:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:58 smithi170 ceph-mon[102832]: pgmap v2614: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T00:35:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4590.094893 secs 2023-09-10T00:35:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:58 smithi082 ceph-mon[98694]: pgmap v2614: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T00:35:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4590.094893 secs 2023-09-10T00:35:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:58 smithi098 ceph-mon[103124]: pgmap v2614: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T00:35:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4590.094893 secs 2023-09-10T00:36:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:35:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4590.808049 secs 2023-09-10T00:36:00.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:35:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4590.808049 secs 2023-09-10T00:36:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:35:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4590.808049 secs 2023-09-10T00:36:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:00 smithi170 ceph-mon[102832]: pgmap v2615: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T00:36:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:00 smithi082 ceph-mon[98694]: pgmap v2615: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T00:36:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:00 smithi098 ceph-mon[103124]: pgmap v2615: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T00:36:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:36:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:36:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:36:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:02 smithi170 ceph-mon[102832]: pgmap v2616: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 95 KiB/s wr, 31 op/s 2023-09-10T00:36:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4593.559746 secs 2023-09-10T00:36:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:02 smithi082 ceph-mon[98694]: pgmap v2616: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 95 KiB/s wr, 31 op/s 2023-09-10T00:36:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4593.559746 secs 2023-09-10T00:36:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:02 smithi098 ceph-mon[103124]: pgmap v2616: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 95 KiB/s wr, 31 op/s 2023-09-10T00:36:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4593.559746 secs 2023-09-10T00:36:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4595.094958 secs 2023-09-10T00:36:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4595.094958 secs 2023-09-10T00:36:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4595.094958 secs 2023-09-10T00:36:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4595.808115 secs 2023-09-10T00:36:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:05 smithi170 ceph-mon[102832]: pgmap v2617: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 67 KiB/s wr, 25 op/s 2023-09-10T00:36:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:36:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:36:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:36:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:36:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:36:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:36:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:05.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:05.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:05.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:05.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:05.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:05.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:05.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4595.808115 secs 2023-09-10T00:36:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:05 smithi082 ceph-mon[98694]: pgmap v2617: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 67 KiB/s wr, 25 op/s 2023-09-10T00:36:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:36:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:36:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:36:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:36:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:36:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:36:05.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:05.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:05.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:05.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:05.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:05.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:05.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:05.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4595.808115 secs 2023-09-10T00:36:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:05 smithi098 ceph-mon[103124]: pgmap v2617: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 67 KiB/s wr, 25 op/s 2023-09-10T00:36:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:36:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:36:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:36:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:36:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:36:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:36:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:05.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:05.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:05.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:05.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:05.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:05.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:05.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:06 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T00:36:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:06 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T00:36:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:06 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T00:36:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:06 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T00:36:06.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:06.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:06.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:06.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:06.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:06 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T00:36:06.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:06 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T00:36:06.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:06.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:36:06.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:36:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:06 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T00:36:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:06 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T00:36:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:06 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T00:36:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:06 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T00:36:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:06 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T00:36:06.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:06 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T00:36:06.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:06.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:36:06.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:36:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:06 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T00:36:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:06 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T00:36:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:06 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T00:36:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:06 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T00:36:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:06.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:06.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:06.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:36:06.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:06 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T00:36:06.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:06 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T00:36:06.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:36:06.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:36:06.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:36:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:07 smithi170 ceph-mon[102832]: pgmap v2618: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 92 KiB/s wr, 30 op/s 2023-09-10T00:36:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:07 smithi082 ceph-mon[98694]: pgmap v2618: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 92 KiB/s wr, 30 op/s 2023-09-10T00:36:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:07 smithi098 ceph-mon[103124]: pgmap v2618: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 92 KiB/s wr, 30 op/s 2023-09-10T00:36:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4598.559775 secs 2023-09-10T00:36:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:08 smithi170 ceph-mon[102832]: osdmap e235: 12 total, 12 up, 12 in 2023-09-10T00:36:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4598.559775 secs 2023-09-10T00:36:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:08 smithi082 ceph-mon[98694]: osdmap e235: 12 total, 12 up, 12 in 2023-09-10T00:36:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4598.559775 secs 2023-09-10T00:36:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:08 smithi098 ceph-mon[103124]: osdmap e235: 12 total, 12 up, 12 in 2023-09-10T00:36:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:09 smithi170 ceph-mon[102832]: pgmap v2620: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-09-10T00:36:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4600.095017 secs 2023-09-10T00:36:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:09 smithi082 ceph-mon[98694]: pgmap v2620: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-09-10T00:36:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4600.095017 secs 2023-09-10T00:36:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:09 smithi098 ceph-mon[103124]: pgmap v2620: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-09-10T00:36:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4600.095017 secs 2023-09-10T00:36:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4600.808180 secs 2023-09-10T00:36:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4600.808180 secs 2023-09-10T00:36:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4600.808180 secs 2023-09-10T00:36:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:11 smithi170 ceph-mon[102832]: pgmap v2621: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-09-10T00:36:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:11 smithi082 ceph-mon[98694]: pgmap v2621: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-09-10T00:36:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:11 smithi098 ceph-mon[103124]: pgmap v2621: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-09-10T00:36:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:13 smithi170 ceph-mon[102832]: pgmap v2622: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:36:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4603.559833 secs 2023-09-10T00:36:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:13 smithi082 ceph-mon[98694]: pgmap v2622: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:36:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4603.559833 secs 2023-09-10T00:36:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:13 smithi098 ceph-mon[103124]: pgmap v2622: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:36:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4603.559833 secs 2023-09-10T00:36:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4605.095076 secs 2023-09-10T00:36:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4605.095076 secs 2023-09-10T00:36:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4605.095076 secs 2023-09-10T00:36:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4605.808243 secs 2023-09-10T00:36:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:15 smithi170 ceph-mon[102832]: pgmap v2623: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:36:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:15 smithi170 ceph-mon[102832]: osdmap e236: 12 total, 12 up, 12 in 2023-09-10T00:36:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4605.808243 secs 2023-09-10T00:36:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:15 smithi082 ceph-mon[98694]: pgmap v2623: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:36:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:15 smithi082 ceph-mon[98694]: osdmap e236: 12 total, 12 up, 12 in 2023-09-10T00:36:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4605.808243 secs 2023-09-10T00:36:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:15 smithi098 ceph-mon[103124]: pgmap v2623: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:36:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:15 smithi098 ceph-mon[103124]: osdmap e236: 12 total, 12 up, 12 in 2023-09-10T00:36:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:17 smithi170 ceph-mon[102832]: pgmap v2625: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 45 KiB/s wr, 13 op/s 2023-09-10T00:36:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:17 smithi082 ceph-mon[98694]: pgmap v2625: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 45 KiB/s wr, 13 op/s 2023-09-10T00:36:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:17 smithi098 ceph-mon[103124]: pgmap v2625: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 45 KiB/s wr, 13 op/s 2023-09-10T00:36:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:19 smithi170 ceph-mon[102832]: pgmap v2626: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T00:36:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4610.095129 secs 2023-09-10T00:36:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:36:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:19 smithi082 ceph-mon[98694]: pgmap v2626: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T00:36:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4610.095129 secs 2023-09-10T00:36:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:36:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:19 smithi098 ceph-mon[103124]: pgmap v2626: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T00:36:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4610.095129 secs 2023-09-10T00:36:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:36:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4610.808309 secs 2023-09-10T00:36:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:36:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:36:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:36:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4610.808309 secs 2023-09-10T00:36:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:36:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:36:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:36:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4610.808309 secs 2023-09-10T00:36:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:36:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:36:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:36:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:21 smithi170 ceph-mon[102832]: pgmap v2627: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T00:36:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:21 smithi082 ceph-mon[98694]: pgmap v2627: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T00:36:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:21 smithi098 ceph-mon[103124]: pgmap v2627: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T00:36:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:23 smithi170 ceph-mon[102832]: pgmap v2628: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T00:36:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:23 smithi082 ceph-mon[98694]: pgmap v2628: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T00:36:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:23 smithi098 ceph-mon[103124]: pgmap v2628: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T00:36:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4615.095187 secs 2023-09-10T00:36:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4615.095187 secs 2023-09-10T00:36:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4615.095187 secs 2023-09-10T00:36:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4615.808375 secs 2023-09-10T00:36:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:25 smithi170 ceph-mon[102832]: pgmap v2629: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T00:36:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4615.808375 secs 2023-09-10T00:36:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:25 smithi082 ceph-mon[98694]: pgmap v2629: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T00:36:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4615.808375 secs 2023-09-10T00:36:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:25 smithi098 ceph-mon[103124]: pgmap v2629: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T00:36:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:27 smithi082 ceph-mon[98694]: pgmap v2630: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 102 KiB/s wr, 16 op/s 2023-09-10T00:36:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:27 smithi098 ceph-mon[103124]: pgmap v2630: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 102 KiB/s wr, 16 op/s 2023-09-10T00:36:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:27 smithi170 ceph-mon[102832]: pgmap v2630: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 102 KiB/s wr, 16 op/s 2023-09-10T00:36:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:29 smithi082 ceph-mon[98694]: pgmap v2631: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 62 KiB/s wr, 10 op/s 2023-09-10T00:36:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4620.095239 secs 2023-09-10T00:36:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:29 smithi098 ceph-mon[103124]: pgmap v2631: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 62 KiB/s wr, 10 op/s 2023-09-10T00:36:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4620.095239 secs 2023-09-10T00:36:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:29 smithi170 ceph-mon[102832]: pgmap v2631: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 62 KiB/s wr, 10 op/s 2023-09-10T00:36:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4620.095239 secs 2023-09-10T00:36:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4620.808445 secs 2023-09-10T00:36:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4620.808445 secs 2023-09-10T00:36:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4620.808445 secs 2023-09-10T00:36:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:31 smithi082 ceph-mon[98694]: pgmap v2632: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 62 KiB/s wr, 10 op/s 2023-09-10T00:36:31.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:31 smithi098 ceph-mon[103124]: pgmap v2632: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 62 KiB/s wr, 10 op/s 2023-09-10T00:36:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:31 smithi170 ceph-mon[102832]: pgmap v2632: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 62 KiB/s wr, 10 op/s 2023-09-10T00:36:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:33 smithi082 ceph-mon[98694]: pgmap v2633: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 120 KiB/s wr, 16 op/s 2023-09-10T00:36:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:33 smithi098 ceph-mon[103124]: pgmap v2633: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 120 KiB/s wr, 16 op/s 2023-09-10T00:36:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:33 smithi170 ceph-mon[102832]: pgmap v2633: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 120 KiB/s wr, 16 op/s 2023-09-10T00:36:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4625.095327 secs 2023-09-10T00:36:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4625.095327 secs 2023-09-10T00:36:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4625.095327 secs 2023-09-10T00:36:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4625.808492 secs 2023-09-10T00:36:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:35 smithi082 ceph-mon[98694]: pgmap v2634: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T00:36:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4625.808492 secs 2023-09-10T00:36:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:35 smithi098 ceph-mon[103124]: pgmap v2634: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T00:36:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4625.808492 secs 2023-09-10T00:36:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:35 smithi170 ceph-mon[102832]: pgmap v2634: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T00:36:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:37 smithi082 ceph-mon[98694]: pgmap v2635: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 125 KiB/s wr, 17 op/s 2023-09-10T00:36:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:37 smithi098 ceph-mon[103124]: pgmap v2635: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 125 KiB/s wr, 17 op/s 2023-09-10T00:36:37.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:37 smithi170 ceph-mon[102832]: pgmap v2635: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 125 KiB/s wr, 17 op/s 2023-09-10T00:36:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:39 smithi082 ceph-mon[98694]: pgmap v2636: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T00:36:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4630.095384 secs 2023-09-10T00:36:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:39 smithi098 ceph-mon[103124]: pgmap v2636: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T00:36:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4630.095384 secs 2023-09-10T00:36:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:39 smithi170 ceph-mon[102832]: pgmap v2636: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T00:36:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4630.095384 secs 2023-09-10T00:36:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4630.808556 secs 2023-09-10T00:36:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4630.808556 secs 2023-09-10T00:36:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4630.808556 secs 2023-09-10T00:36:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:41 smithi098 ceph-mon[103124]: pgmap v2637: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T00:36:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:41 smithi170 ceph-mon[102832]: pgmap v2637: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T00:36:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:41 smithi082 ceph-mon[98694]: pgmap v2637: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T00:36:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:43 smithi098 ceph-mon[103124]: pgmap v2638: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 181 KiB/s wr, 17 op/s 2023-09-10T00:36:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:43 smithi170 ceph-mon[102832]: pgmap v2638: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 181 KiB/s wr, 17 op/s 2023-09-10T00:36:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:43 smithi082 ceph-mon[98694]: pgmap v2638: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 181 KiB/s wr, 17 op/s 2023-09-10T00:36:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4635.095444 secs 2023-09-10T00:36:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4635.095444 secs 2023-09-10T00:36:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4635.095444 secs 2023-09-10T00:36:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4635.808622 secs 2023-09-10T00:36:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:45 smithi098 ceph-mon[103124]: pgmap v2639: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T00:36:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4635.808622 secs 2023-09-10T00:36:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:45 smithi170 ceph-mon[102832]: pgmap v2639: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T00:36:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4635.808622 secs 2023-09-10T00:36:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:45 smithi082 ceph-mon[98694]: pgmap v2639: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T00:36:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:47 smithi098 ceph-mon[103124]: pgmap v2640: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 240 KiB/s wr, 18 op/s 2023-09-10T00:36:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:47 smithi170 ceph-mon[102832]: pgmap v2640: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 240 KiB/s wr, 18 op/s 2023-09-10T00:36:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:47 smithi082 ceph-mon[98694]: pgmap v2640: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 240 KiB/s wr, 18 op/s 2023-09-10T00:36:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4638.560344 secs 2023-09-10T00:36:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4638.560344 secs 2023-09-10T00:36:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4638.560344 secs 2023-09-10T00:36:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:49 smithi098 ceph-mon[103124]: pgmap v2641: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 206 KiB/s wr, 12 op/s 2023-09-10T00:36:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4640.095530 secs 2023-09-10T00:36:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:49 smithi170 ceph-mon[102832]: pgmap v2641: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 206 KiB/s wr, 12 op/s 2023-09-10T00:36:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4640.095530 secs 2023-09-10T00:36:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:49 smithi082 ceph-mon[98694]: pgmap v2641: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 206 KiB/s wr, 12 op/s 2023-09-10T00:36:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4640.095530 secs 2023-09-10T00:36:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4640.808689 secs 2023-09-10T00:36:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4640.808689 secs 2023-09-10T00:36:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4640.808689 secs 2023-09-10T00:36:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:51 smithi098 ceph-mon[103124]: pgmap v2642: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 206 KiB/s wr, 12 op/s 2023-09-10T00:36:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:51 smithi170 ceph-mon[102832]: pgmap v2642: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 206 KiB/s wr, 12 op/s 2023-09-10T00:36:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:51 smithi082 ceph-mon[98694]: pgmap v2642: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 206 KiB/s wr, 12 op/s 2023-09-10T00:36:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:36:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:36:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:36:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:36:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:36:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:36:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:36:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:36:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:36:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:53 smithi098 ceph-mon[103124]: pgmap v2643: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 242 KiB/s wr, 18 op/s 2023-09-10T00:36:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4643.560420 secs 2023-09-10T00:36:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:53 smithi170 ceph-mon[102832]: pgmap v2643: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 242 KiB/s wr, 18 op/s 2023-09-10T00:36:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4643.560420 secs 2023-09-10T00:36:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:53 smithi082 ceph-mon[98694]: pgmap v2643: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 242 KiB/s wr, 18 op/s 2023-09-10T00:36:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4643.560420 secs 2023-09-10T00:36:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4645.095592 secs 2023-09-10T00:36:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4645.095592 secs 2023-09-10T00:36:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4645.095592 secs 2023-09-10T00:36:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4645.808754 secs 2023-09-10T00:36:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:55 smithi098 ceph-mon[103124]: pgmap v2644: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 153 KiB/s wr, 12 op/s 2023-09-10T00:36:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4645.808754 secs 2023-09-10T00:36:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:55 smithi170 ceph-mon[102832]: pgmap v2644: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 153 KiB/s wr, 12 op/s 2023-09-10T00:36:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4645.808754 secs 2023-09-10T00:36:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:55 smithi082 ceph-mon[98694]: pgmap v2644: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 153 KiB/s wr, 12 op/s 2023-09-10T00:36:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:57 smithi170 ceph-mon[102832]: pgmap v2645: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 211 KiB/s wr, 18 op/s 2023-09-10T00:36:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:57 smithi082 ceph-mon[98694]: pgmap v2645: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 211 KiB/s wr, 18 op/s 2023-09-10T00:36:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:57 smithi098 ceph-mon[103124]: pgmap v2645: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 211 KiB/s wr, 18 op/s 2023-09-10T00:36:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4648.560479 secs 2023-09-10T00:36:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4648.560479 secs 2023-09-10T00:36:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4648.560479 secs 2023-09-10T00:36:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:59 smithi170 ceph-mon[102832]: pgmap v2646: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 94 KiB/s wr, 12 op/s 2023-09-10T00:36:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:36:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4650.095656 secs 2023-09-10T00:36:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:59 smithi082 ceph-mon[98694]: pgmap v2646: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 94 KiB/s wr, 12 op/s 2023-09-10T00:36:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:36:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4650.095656 secs 2023-09-10T00:36:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:59 smithi098 ceph-mon[103124]: pgmap v2646: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 94 KiB/s wr, 12 op/s 2023-09-10T00:36:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:36:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4650.095656 secs 2023-09-10T00:37:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4650.808821 secs 2023-09-10T00:37:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4650.808821 secs 2023-09-10T00:37:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4650.808821 secs 2023-09-10T00:37:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:01 smithi170 ceph-mon[102832]: pgmap v2647: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 94 KiB/s wr, 12 op/s 2023-09-10T00:37:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:01 smithi082 ceph-mon[98694]: pgmap v2647: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 94 KiB/s wr, 12 op/s 2023-09-10T00:37:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:01 smithi098 ceph-mon[103124]: pgmap v2647: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 94 KiB/s wr, 12 op/s 2023-09-10T00:37:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:02 smithi170 ceph-mon[102832]: pgmap v2648: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 127 KiB/s wr, 30 op/s 2023-09-10T00:37:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:02 smithi082 ceph-mon[98694]: pgmap v2648: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 127 KiB/s wr, 30 op/s 2023-09-10T00:37:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:02 smithi098 ceph-mon[103124]: pgmap v2648: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 127 KiB/s wr, 30 op/s 2023-09-10T00:37:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4653.560540 secs 2023-09-10T00:37:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4653.560540 secs 2023-09-10T00:37:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4653.560540 secs 2023-09-10T00:37:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4655.095713 secs 2023-09-10T00:37:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:04 smithi170 ceph-mon[102832]: pgmap v2649: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 92 KiB/s wr, 24 op/s 2023-09-10T00:37:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4655.095713 secs 2023-09-10T00:37:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:04 smithi082 ceph-mon[98694]: pgmap v2649: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 92 KiB/s wr, 24 op/s 2023-09-10T00:37:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4655.095713 secs 2023-09-10T00:37:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:04 smithi098 ceph-mon[103124]: pgmap v2649: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 92 KiB/s wr, 24 op/s 2023-09-10T00:37:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4655.808885 secs 2023-09-10T00:37:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:37:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4655.808885 secs 2023-09-10T00:37:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:37:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4655.808885 secs 2023-09-10T00:37:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:37:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:06 smithi170 ceph-mon[102832]: pgmap v2650: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 122 KiB/s wr, 30 op/s 2023-09-10T00:37:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:06 smithi082 ceph-mon[98694]: pgmap v2650: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 122 KiB/s wr, 30 op/s 2023-09-10T00:37:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:06 smithi098 ceph-mon[103124]: pgmap v2650: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 122 KiB/s wr, 30 op/s 2023-09-10T00:37:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:07.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:37:07.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:37:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:37:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:37:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:37:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:37:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:37:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4658.560705 secs 2023-09-10T00:37:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:08 smithi170 ceph-mon[102832]: pgmap v2651: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 24 op/s 2023-09-10T00:37:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:08 smithi170 ceph-mon[102832]: osdmap e237: 12 total, 12 up, 12 in 2023-09-10T00:37:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4658.560705 secs 2023-09-10T00:37:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:08 smithi082 ceph-mon[98694]: pgmap v2651: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 24 op/s 2023-09-10T00:37:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:08 smithi082 ceph-mon[98694]: osdmap e237: 12 total, 12 up, 12 in 2023-09-10T00:37:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4658.560705 secs 2023-09-10T00:37:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:08 smithi098 ceph-mon[103124]: pgmap v2651: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 24 op/s 2023-09-10T00:37:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:08 smithi098 ceph-mon[103124]: osdmap e237: 12 total, 12 up, 12 in 2023-09-10T00:37:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4660.095785 secs 2023-09-10T00:37:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4660.095785 secs 2023-09-10T00:37:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4660.095785 secs 2023-09-10T00:37:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4660.808951 secs 2023-09-10T00:37:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:10 smithi170 ceph-mon[102832]: pgmap v2653: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 76 KiB/s wr, 28 op/s 2023-09-10T00:37:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4660.808951 secs 2023-09-10T00:37:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:10 smithi082 ceph-mon[98694]: pgmap v2653: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 76 KiB/s wr, 28 op/s 2023-09-10T00:37:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4660.808951 secs 2023-09-10T00:37:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:10 smithi098 ceph-mon[103124]: pgmap v2653: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 76 KiB/s wr, 28 op/s 2023-09-10T00:37:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:12 smithi098 ceph-mon[103124]: pgmap v2654: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 13 op/s 2023-09-10T00:37:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4663.560766 secs 2023-09-10T00:37:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:12 smithi170 ceph-mon[102832]: pgmap v2654: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 13 op/s 2023-09-10T00:37:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4663.560766 secs 2023-09-10T00:37:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:12 smithi082 ceph-mon[98694]: pgmap v2654: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 13 op/s 2023-09-10T00:37:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4663.560766 secs 2023-09-10T00:37:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4665.095842 secs 2023-09-10T00:37:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4665.095842 secs 2023-09-10T00:37:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4665.095842 secs 2023-09-10T00:37:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4665.809015 secs 2023-09-10T00:37:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:14 smithi098 ceph-mon[103124]: pgmap v2655: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 13 op/s 2023-09-10T00:37:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:14 smithi098 ceph-mon[103124]: osdmap e238: 12 total, 12 up, 12 in 2023-09-10T00:37:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4665.809015 secs 2023-09-10T00:37:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:14 smithi170 ceph-mon[102832]: pgmap v2655: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 13 op/s 2023-09-10T00:37:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:14 smithi170 ceph-mon[102832]: osdmap e238: 12 total, 12 up, 12 in 2023-09-10T00:37:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4665.809015 secs 2023-09-10T00:37:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:14 smithi082 ceph-mon[98694]: pgmap v2655: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 13 op/s 2023-09-10T00:37:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:14 smithi082 ceph-mon[98694]: osdmap e238: 12 total, 12 up, 12 in 2023-09-10T00:37:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:16 smithi098 ceph-mon[103124]: pgmap v2657: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 60 KiB/s wr, 14 op/s 2023-09-10T00:37:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:16 smithi170 ceph-mon[102832]: pgmap v2657: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 60 KiB/s wr, 14 op/s 2023-09-10T00:37:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:16 smithi082 ceph-mon[98694]: pgmap v2657: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 60 KiB/s wr, 14 op/s 2023-09-10T00:37:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:18 smithi098 ceph-mon[103124]: pgmap v2658: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T00:37:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4670.095902 secs 2023-09-10T00:37:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:18 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:37:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:18 smithi170 ceph-mon[102832]: pgmap v2658: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T00:37:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4670.095902 secs 2023-09-10T00:37:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:18 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:37:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:18 smithi082 ceph-mon[98694]: pgmap v2658: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T00:37:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4670.095902 secs 2023-09-10T00:37:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:18 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:37:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4670.809079 secs 2023-09-10T00:37:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:19 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:37:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:19 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:37:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:19 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:37:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4670.809079 secs 2023-09-10T00:37:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:19 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:37:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:19 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:37:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:19 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:37:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4670.809079 secs 2023-09-10T00:37:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:19 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:37:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:19 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:37:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:19 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:37:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:20 smithi170 ceph-mon[102832]: pgmap v2659: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T00:37:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:20 smithi082 ceph-mon[98694]: pgmap v2659: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T00:37:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:20 smithi098 ceph-mon[103124]: pgmap v2659: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T00:37:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:22 smithi170 ceph-mon[102832]: pgmap v2660: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 129 KiB/s wr, 12 op/s 2023-09-10T00:37:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:22 smithi082 ceph-mon[98694]: pgmap v2660: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 129 KiB/s wr, 12 op/s 2023-09-10T00:37:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:22 smithi098 ceph-mon[103124]: pgmap v2660: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 129 KiB/s wr, 12 op/s 2023-09-10T00:37:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4675.095979 secs 2023-09-10T00:37:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4675.095979 secs 2023-09-10T00:37:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4675.095979 secs 2023-09-10T00:37:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4675.809146 secs 2023-09-10T00:37:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:24 smithi170 ceph-mon[102832]: pgmap v2661: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 129 KiB/s wr, 12 op/s 2023-09-10T00:37:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4675.809146 secs 2023-09-10T00:37:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:24 smithi082 ceph-mon[98694]: pgmap v2661: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 129 KiB/s wr, 12 op/s 2023-09-10T00:37:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4675.809146 secs 2023-09-10T00:37:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:24 smithi098 ceph-mon[103124]: pgmap v2661: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 129 KiB/s wr, 12 op/s 2023-09-10T00:37:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:26 smithi170 ceph-mon[102832]: pgmap v2662: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 210 KiB/s wr, 49 op/s 2023-09-10T00:37:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:26 smithi082 ceph-mon[98694]: pgmap v2662: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 210 KiB/s wr, 49 op/s 2023-09-10T00:37:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:26 smithi098 ceph-mon[103124]: pgmap v2662: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 210 KiB/s wr, 49 op/s 2023-09-10T00:37:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:28 smithi170 ceph-mon[102832]: pgmap v2663: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 178 KiB/s wr, 40 op/s 2023-09-10T00:37:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4680.096058 secs 2023-09-10T00:37:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:28 smithi082 ceph-mon[98694]: pgmap v2663: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 178 KiB/s wr, 40 op/s 2023-09-10T00:37:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4680.096058 secs 2023-09-10T00:37:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:28 smithi098 ceph-mon[103124]: pgmap v2663: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 178 KiB/s wr, 40 op/s 2023-09-10T00:37:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4680.096058 secs 2023-09-10T00:37:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4680.809212 secs 2023-09-10T00:37:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4680.809212 secs 2023-09-10T00:37:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4680.809212 secs 2023-09-10T00:37:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:30 smithi170 ceph-mon[102832]: pgmap v2664: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 178 KiB/s wr, 40 op/s 2023-09-10T00:37:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:30 smithi082 ceph-mon[98694]: pgmap v2664: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 178 KiB/s wr, 40 op/s 2023-09-10T00:37:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:30 smithi098 ceph-mon[103124]: pgmap v2664: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 178 KiB/s wr, 40 op/s 2023-09-10T00:37:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:32 smithi170 ceph-mon[102832]: pgmap v2665: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 240 KiB/s wr, 47 op/s 2023-09-10T00:37:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:32 smithi082 ceph-mon[98694]: pgmap v2665: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 240 KiB/s wr, 47 op/s 2023-09-10T00:37:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:32 smithi098 ceph-mon[103124]: pgmap v2665: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 240 KiB/s wr, 47 op/s 2023-09-10T00:37:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4685.096119 secs 2023-09-10T00:37:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4685.096119 secs 2023-09-10T00:37:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4685.096119 secs 2023-09-10T00:37:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4685.809278 secs 2023-09-10T00:37:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:34 smithi170 ceph-mon[102832]: pgmap v2666: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 147 KiB/s wr, 41 op/s 2023-09-10T00:37:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4685.809278 secs 2023-09-10T00:37:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:34 smithi082 ceph-mon[98694]: pgmap v2666: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 147 KiB/s wr, 41 op/s 2023-09-10T00:37:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4685.809278 secs 2023-09-10T00:37:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:34 smithi098 ceph-mon[103124]: pgmap v2666: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 147 KiB/s wr, 41 op/s 2023-09-10T00:37:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:36 smithi170 ceph-mon[102832]: pgmap v2667: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 277 KiB/s wr, 48 op/s 2023-09-10T00:37:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:36 smithi082 ceph-mon[98694]: pgmap v2667: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 277 KiB/s wr, 48 op/s 2023-09-10T00:37:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:36 smithi098 ceph-mon[103124]: pgmap v2667: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 277 KiB/s wr, 48 op/s 2023-09-10T00:37:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4688.561076 secs 2023-09-10T00:37:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4688.561076 secs 2023-09-10T00:37:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4688.561076 secs 2023-09-10T00:37:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:38 smithi170 ceph-mon[102832]: pgmap v2668: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 193 KiB/s wr, 13 op/s 2023-09-10T00:37:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4690.096179 secs 2023-09-10T00:37:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:38 smithi082 ceph-mon[98694]: pgmap v2668: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 193 KiB/s wr, 13 op/s 2023-09-10T00:37:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4690.096179 secs 2023-09-10T00:37:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:38 smithi098 ceph-mon[103124]: pgmap v2668: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 193 KiB/s wr, 13 op/s 2023-09-10T00:37:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4690.096179 secs 2023-09-10T00:37:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4690.809345 secs 2023-09-10T00:37:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:39 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:37:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:39 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:37:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:39 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:37:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4690.809345 secs 2023-09-10T00:37:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:39 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:37:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:39 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:37:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:39 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:37:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4690.809345 secs 2023-09-10T00:37:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:39 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:37:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:39 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:37:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:39 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:37:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:40 smithi170 ceph-mon[102832]: pgmap v2669: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 193 KiB/s wr, 13 op/s 2023-09-10T00:37:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:40 smithi082 ceph-mon[98694]: pgmap v2669: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 193 KiB/s wr, 13 op/s 2023-09-10T00:37:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:40 smithi098 ceph-mon[103124]: pgmap v2669: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 193 KiB/s wr, 13 op/s 2023-09-10T00:37:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:42 smithi170 ceph-mon[102832]: pgmap v2670: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 218 KiB/s wr, 18 op/s 2023-09-10T00:37:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4693.561137 secs 2023-09-10T00:37:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:42 smithi082 ceph-mon[98694]: pgmap v2670: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 218 KiB/s wr, 18 op/s 2023-09-10T00:37:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4693.561137 secs 2023-09-10T00:37:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:42 smithi098 ceph-mon[103124]: pgmap v2670: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 218 KiB/s wr, 18 op/s 2023-09-10T00:37:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4693.561137 secs 2023-09-10T00:37:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4695.096239 secs 2023-09-10T00:37:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4695.096239 secs 2023-09-10T00:37:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4695.096239 secs 2023-09-10T00:37:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4695.809410 secs 2023-09-10T00:37:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:44 smithi170 ceph-mon[102832]: pgmap v2671: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T00:37:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4695.809410 secs 2023-09-10T00:37:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:44 smithi082 ceph-mon[98694]: pgmap v2671: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T00:37:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4695.809410 secs 2023-09-10T00:37:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:44 smithi098 ceph-mon[103124]: pgmap v2671: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T00:37:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:46 smithi170 ceph-mon[102832]: pgmap v2672: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 181 KiB/s wr, 17 op/s 2023-09-10T00:37:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:46 smithi082 ceph-mon[98694]: pgmap v2672: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 181 KiB/s wr, 17 op/s 2023-09-10T00:37:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:46 smithi098 ceph-mon[103124]: pgmap v2672: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 181 KiB/s wr, 17 op/s 2023-09-10T00:37:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4698.561254 secs 2023-09-10T00:37:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4698.561254 secs 2023-09-10T00:37:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4698.561254 secs 2023-09-10T00:37:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:48 smithi170 ceph-mon[102832]: pgmap v2673: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:37:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4700.096301 secs 2023-09-10T00:37:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:48 smithi082 ceph-mon[98694]: pgmap v2673: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:37:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4700.096301 secs 2023-09-10T00:37:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:48 smithi098 ceph-mon[103124]: pgmap v2673: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:37:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4700.096301 secs 2023-09-10T00:37:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4700.809495 secs 2023-09-10T00:37:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4700.809495 secs 2023-09-10T00:37:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4700.809495 secs 2023-09-10T00:37:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:50 smithi170 ceph-mon[102832]: pgmap v2674: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:37:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:50 smithi082 ceph-mon[98694]: pgmap v2674: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:37:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:50 smithi098 ceph-mon[103124]: pgmap v2674: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:37:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:52 smithi170 ceph-mon[102832]: pgmap v2675: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 78 KiB/s wr, 15 op/s 2023-09-10T00:37:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4703.561331 secs 2023-09-10T00:37:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:52 smithi082 ceph-mon[98694]: pgmap v2675: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 78 KiB/s wr, 15 op/s 2023-09-10T00:37:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4703.561331 secs 2023-09-10T00:37:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:52 smithi098 ceph-mon[103124]: pgmap v2675: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 78 KiB/s wr, 15 op/s 2023-09-10T00:37:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4703.561331 secs 2023-09-10T00:37:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4705.096362 secs 2023-09-10T00:37:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4705.096362 secs 2023-09-10T00:37:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4705.096362 secs 2023-09-10T00:37:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4705.809562 secs 2023-09-10T00:37:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:54 smithi170 ceph-mon[102832]: pgmap v2676: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 9 op/s 2023-09-10T00:37:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4705.809562 secs 2023-09-10T00:37:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:54 smithi082 ceph-mon[98694]: pgmap v2676: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 9 op/s 2023-09-10T00:37:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4705.809562 secs 2023-09-10T00:37:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:54 smithi098 ceph-mon[103124]: pgmap v2676: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 9 op/s 2023-09-10T00:37:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:56 smithi170 ceph-mon[102832]: pgmap v2677: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 68 KiB/s wr, 15 op/s 2023-09-10T00:37:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:56 smithi082 ceph-mon[98694]: pgmap v2677: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 68 KiB/s wr, 15 op/s 2023-09-10T00:37:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:56 smithi098 ceph-mon[103124]: pgmap v2677: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 68 KiB/s wr, 15 op/s 2023-09-10T00:37:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4708.561391 secs 2023-09-10T00:37:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4708.561391 secs 2023-09-10T00:37:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4708.561391 secs 2023-09-10T00:37:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:58 smithi170 ceph-mon[102832]: pgmap v2678: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-10T00:37:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4710.096449 secs 2023-09-10T00:37:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:58 smithi082 ceph-mon[98694]: pgmap v2678: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-10T00:37:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4710.096449 secs 2023-09-10T00:37:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:58 smithi098 ceph-mon[103124]: pgmap v2678: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-10T00:37:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4710.096449 secs 2023-09-10T00:38:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:37:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4710.809629 secs 2023-09-10T00:38:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:37:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4710.809629 secs 2023-09-10T00:38:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:37:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4710.809629 secs 2023-09-10T00:38:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:00 smithi170 ceph-mon[102832]: pgmap v2679: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-10T00:38:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:00 smithi082 ceph-mon[98694]: pgmap v2679: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-10T00:38:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:00 smithi098 ceph-mon[103124]: pgmap v2679: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-10T00:38:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:02 smithi170 ceph-mon[102832]: pgmap v2680: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 90 KiB/s wr, 27 op/s 2023-09-10T00:38:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4713.561452 secs 2023-09-10T00:38:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:02 smithi082 ceph-mon[98694]: pgmap v2680: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 90 KiB/s wr, 27 op/s 2023-09-10T00:38:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4713.561452 secs 2023-09-10T00:38:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:02 smithi098 ceph-mon[103124]: pgmap v2680: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 90 KiB/s wr, 27 op/s 2023-09-10T00:38:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4713.561452 secs 2023-09-10T00:38:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4715.096510 secs 2023-09-10T00:38:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4715.096510 secs 2023-09-10T00:38:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4715.096510 secs 2023-09-10T00:38:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4715.809694 secs 2023-09-10T00:38:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:04 smithi170 ceph-mon[102832]: pgmap v2681: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 63 KiB/s wr, 23 op/s 2023-09-10T00:38:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4715.809694 secs 2023-09-10T00:38:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:04 smithi082 ceph-mon[98694]: pgmap v2681: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 63 KiB/s wr, 23 op/s 2023-09-10T00:38:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4715.809694 secs 2023-09-10T00:38:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:04 smithi098 ceph-mon[103124]: pgmap v2681: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 63 KiB/s wr, 23 op/s 2023-09-10T00:38:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:06 smithi170 ceph-mon[102832]: pgmap v2682: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-09-10T00:38:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:06 smithi082 ceph-mon[98694]: pgmap v2682: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-09-10T00:38:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:06 smithi098 ceph-mon[103124]: pgmap v2682: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-09-10T00:38:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4718.561519 secs 2023-09-10T00:38:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:38:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4718.561519 secs 2023-09-10T00:38:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:38:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4718.561519 secs 2023-09-10T00:38:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:38:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:08 smithi170 ceph-mon[102832]: pgmap v2683: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 71 KiB/s wr, 24 op/s 2023-09-10T00:38:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4720.096571 secs 2023-09-10T00:38:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:08 smithi170 ceph-mon[102832]: osdmap e239: 12 total, 12 up, 12 in 2023-09-10T00:38:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:38:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:38:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:09.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:38:09.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:38:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:08 smithi082 ceph-mon[98694]: pgmap v2683: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 71 KiB/s wr, 24 op/s 2023-09-10T00:38:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4720.096571 secs 2023-09-10T00:38:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:08 smithi082 ceph-mon[98694]: osdmap e239: 12 total, 12 up, 12 in 2023-09-10T00:38:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:38:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:38:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:09.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:38:09.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:38:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:08 smithi098 ceph-mon[103124]: pgmap v2683: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 71 KiB/s wr, 24 op/s 2023-09-10T00:38:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4720.096571 secs 2023-09-10T00:38:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:08 smithi098 ceph-mon[103124]: osdmap e239: 12 total, 12 up, 12 in 2023-09-10T00:38:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:38:09.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:38:09.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:38:09.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:38:09.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:38:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4720.809759 secs 2023-09-10T00:38:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4720.809759 secs 2023-09-10T00:38:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4720.809759 secs 2023-09-10T00:38:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:10 smithi170 ceph-mon[102832]: pgmap v2685: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T00:38:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:10 smithi082 ceph-mon[98694]: pgmap v2685: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T00:38:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:10 smithi098 ceph-mon[103124]: pgmap v2685: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T00:38:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:12 smithi170 ceph-mon[102832]: pgmap v2686: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 58 KiB/s wr, 14 op/s 2023-09-10T00:38:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4723.561598 secs 2023-09-10T00:38:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:12 smithi082 ceph-mon[98694]: pgmap v2686: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 58 KiB/s wr, 14 op/s 2023-09-10T00:38:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4723.561598 secs 2023-09-10T00:38:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:12 smithi098 ceph-mon[103124]: pgmap v2686: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 58 KiB/s wr, 14 op/s 2023-09-10T00:38:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4723.561598 secs 2023-09-10T00:38:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4725.096630 secs 2023-09-10T00:38:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4725.096630 secs 2023-09-10T00:38:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4725.096630 secs 2023-09-10T00:38:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4725.809823 secs 2023-09-10T00:38:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:14 smithi170 ceph-mon[102832]: pgmap v2687: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 58 KiB/s wr, 14 op/s 2023-09-10T00:38:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:14 smithi170 ceph-mon[102832]: osdmap e240: 12 total, 12 up, 12 in 2023-09-10T00:38:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4725.809823 secs 2023-09-10T00:38:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:14 smithi082 ceph-mon[98694]: pgmap v2687: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 58 KiB/s wr, 14 op/s 2023-09-10T00:38:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:14 smithi082 ceph-mon[98694]: osdmap e240: 12 total, 12 up, 12 in 2023-09-10T00:38:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4725.809823 secs 2023-09-10T00:38:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:14 smithi098 ceph-mon[103124]: pgmap v2687: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 58 KiB/s wr, 14 op/s 2023-09-10T00:38:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:14 smithi098 ceph-mon[103124]: osdmap e240: 12 total, 12 up, 12 in 2023-09-10T00:38:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:16 smithi170 ceph-mon[102832]: pgmap v2689: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 39 KiB/s wr, 14 op/s 2023-09-10T00:38:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:16 smithi082 ceph-mon[98694]: pgmap v2689: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 39 KiB/s wr, 14 op/s 2023-09-10T00:38:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:16 smithi098 ceph-mon[103124]: pgmap v2689: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 39 KiB/s wr, 14 op/s 2023-09-10T00:38:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:18 smithi170 ceph-mon[102832]: pgmap v2690: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T00:38:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4730.096690 secs 2023-09-10T00:38:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:18 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:38:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:18 smithi082 ceph-mon[98694]: pgmap v2690: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T00:38:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4730.096690 secs 2023-09-10T00:38:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:18 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:38:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:18 smithi098 ceph-mon[103124]: pgmap v2690: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T00:38:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4730.096690 secs 2023-09-10T00:38:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:18 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:38:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4730.809886 secs 2023-09-10T00:38:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:19 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:38:20.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:19 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:38:20.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:19 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:38:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4730.809886 secs 2023-09-10T00:38:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:19 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:38:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:19 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:38:20.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:19 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:38:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4730.809886 secs 2023-09-10T00:38:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:19 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:38:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:19 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:38:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:19 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:38:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:20 smithi170 ceph-mon[102832]: pgmap v2691: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:38:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:20 smithi082 ceph-mon[98694]: pgmap v2691: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:38:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:20 smithi098 ceph-mon[103124]: pgmap v2691: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:38:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:22 smithi170 ceph-mon[102832]: pgmap v2692: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T00:38:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:22 smithi082 ceph-mon[98694]: pgmap v2692: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T00:38:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:22 smithi098 ceph-mon[103124]: pgmap v2692: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T00:38:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4735.096748 secs 2023-09-10T00:38:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4735.096748 secs 2023-09-10T00:38:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4735.096748 secs 2023-09-10T00:38:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4735.809959 secs 2023-09-10T00:38:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:24 smithi170 ceph-mon[102832]: pgmap v2693: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T00:38:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4735.809959 secs 2023-09-10T00:38:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:24 smithi082 ceph-mon[98694]: pgmap v2693: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T00:38:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4735.809959 secs 2023-09-10T00:38:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:24 smithi098 ceph-mon[103124]: pgmap v2693: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T00:38:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:26 smithi170 ceph-mon[102832]: pgmap v2694: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 69 KiB/s wr, 16 op/s 2023-09-10T00:38:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:26 smithi082 ceph-mon[98694]: pgmap v2694: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 69 KiB/s wr, 16 op/s 2023-09-10T00:38:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:26 smithi098 ceph-mon[103124]: pgmap v2694: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 69 KiB/s wr, 16 op/s 2023-09-10T00:38:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:28 smithi170 ceph-mon[102832]: pgmap v2695: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 10 op/s 2023-09-10T00:38:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4740.096807 secs 2023-09-10T00:38:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:28 smithi082 ceph-mon[98694]: pgmap v2695: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 10 op/s 2023-09-10T00:38:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4740.096807 secs 2023-09-10T00:38:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:28 smithi098 ceph-mon[103124]: pgmap v2695: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 10 op/s 2023-09-10T00:38:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4740.096807 secs 2023-09-10T00:38:30.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4740.810022 secs 2023-09-10T00:38:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4740.810022 secs 2023-09-10T00:38:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4740.810022 secs 2023-09-10T00:38:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:30 smithi082 ceph-mon[98694]: pgmap v2696: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 10 op/s 2023-09-10T00:38:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:30 smithi098 ceph-mon[103124]: pgmap v2696: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 10 op/s 2023-09-10T00:38:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:30 smithi170 ceph-mon[102832]: pgmap v2696: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 10 op/s 2023-09-10T00:38:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:32 smithi082 ceph-mon[98694]: pgmap v2697: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 142 KiB/s wr, 16 op/s 2023-09-10T00:38:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:32 smithi098 ceph-mon[103124]: pgmap v2697: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 142 KiB/s wr, 16 op/s 2023-09-10T00:38:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:32 smithi170 ceph-mon[102832]: pgmap v2697: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 142 KiB/s wr, 16 op/s 2023-09-10T00:38:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4745.096868 secs 2023-09-10T00:38:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4745.096868 secs 2023-09-10T00:38:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4745.096868 secs 2023-09-10T00:38:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4745.810088 secs 2023-09-10T00:38:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:34 smithi082 ceph-mon[98694]: pgmap v2698: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T00:38:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4745.810088 secs 2023-09-10T00:38:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:34 smithi098 ceph-mon[103124]: pgmap v2698: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T00:38:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4745.810088 secs 2023-09-10T00:38:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:34 smithi170 ceph-mon[102832]: pgmap v2698: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T00:38:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:36 smithi082 ceph-mon[98694]: pgmap v2699: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 151 KiB/s wr, 18 op/s 2023-09-10T00:38:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:36 smithi098 ceph-mon[103124]: pgmap v2699: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 151 KiB/s wr, 18 op/s 2023-09-10T00:38:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:36 smithi170 ceph-mon[102832]: pgmap v2699: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 151 KiB/s wr, 18 op/s 2023-09-10T00:38:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:38 smithi082 ceph-mon[98694]: pgmap v2700: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T00:38:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4750.096928 secs 2023-09-10T00:38:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:38 smithi098 ceph-mon[103124]: pgmap v2700: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T00:38:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4750.096928 secs 2023-09-10T00:38:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:38 smithi170 ceph-mon[102832]: pgmap v2700: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T00:38:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4750.096928 secs 2023-09-10T00:38:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4750.810151 secs 2023-09-10T00:38:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4750.810151 secs 2023-09-10T00:38:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4750.810151 secs 2023-09-10T00:38:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:40 smithi098 ceph-mon[103124]: pgmap v2701: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T00:38:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:40 smithi170 ceph-mon[102832]: pgmap v2701: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T00:38:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:40 smithi082 ceph-mon[98694]: pgmap v2701: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T00:38:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:42 smithi098 ceph-mon[103124]: pgmap v2702: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 242 KiB/s wr, 18 op/s 2023-09-10T00:38:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:42 smithi170 ceph-mon[102832]: pgmap v2702: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 242 KiB/s wr, 18 op/s 2023-09-10T00:38:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:42 smithi082 ceph-mon[98694]: pgmap v2702: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 242 KiB/s wr, 18 op/s 2023-09-10T00:38:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4755.097006 secs 2023-09-10T00:38:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4755.097006 secs 2023-09-10T00:38:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4755.097006 secs 2023-09-10T00:38:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4755.810219 secs 2023-09-10T00:38:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:44 smithi098 ceph-mon[103124]: pgmap v2703: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-09-10T00:38:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4755.810219 secs 2023-09-10T00:38:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:44 smithi170 ceph-mon[102832]: pgmap v2703: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-09-10T00:38:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4755.810219 secs 2023-09-10T00:38:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:44 smithi082 ceph-mon[98694]: pgmap v2703: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-09-10T00:38:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:46 smithi098 ceph-mon[103124]: pgmap v2704: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 225 KiB/s wr, 18 op/s 2023-09-10T00:38:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:46 smithi170 ceph-mon[102832]: pgmap v2704: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 225 KiB/s wr, 18 op/s 2023-09-10T00:38:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:46 smithi082 ceph-mon[98694]: pgmap v2704: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 225 KiB/s wr, 18 op/s 2023-09-10T00:38:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4758.562069 secs 2023-09-10T00:38:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4758.562069 secs 2023-09-10T00:38:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4758.562069 secs 2023-09-10T00:38:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:48 smithi098 ceph-mon[103124]: pgmap v2705: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 187 KiB/s wr, 12 op/s 2023-09-10T00:38:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4760.097063 secs 2023-09-10T00:38:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:48 smithi170 ceph-mon[102832]: pgmap v2705: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 187 KiB/s wr, 12 op/s 2023-09-10T00:38:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4760.097063 secs 2023-09-10T00:38:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:48 smithi082 ceph-mon[98694]: pgmap v2705: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 187 KiB/s wr, 12 op/s 2023-09-10T00:38:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4760.097063 secs 2023-09-10T00:38:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4760.810283 secs 2023-09-10T00:38:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4760.810283 secs 2023-09-10T00:38:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4760.810283 secs 2023-09-10T00:38:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:50 smithi098 ceph-mon[103124]: pgmap v2706: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 187 KiB/s wr, 12 op/s 2023-09-10T00:38:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:50 smithi170 ceph-mon[102832]: pgmap v2706: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 187 KiB/s wr, 12 op/s 2023-09-10T00:38:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:50 smithi082 ceph-mon[98694]: pgmap v2706: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 187 KiB/s wr, 12 op/s 2023-09-10T00:38:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:38:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:38:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:38:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:38:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:38:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:38:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:38:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:38:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:38:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:52 smithi098 ceph-mon[103124]: pgmap v2707: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 242 KiB/s wr, 17 op/s 2023-09-10T00:38:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4763.562156 secs 2023-09-10T00:38:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:52 smithi170 ceph-mon[102832]: pgmap v2707: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 242 KiB/s wr, 17 op/s 2023-09-10T00:38:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4763.562156 secs 2023-09-10T00:38:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:52 smithi082 ceph-mon[98694]: pgmap v2707: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 242 KiB/s wr, 17 op/s 2023-09-10T00:38:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4763.562156 secs 2023-09-10T00:38:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4765.097122 secs 2023-09-10T00:38:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4765.097122 secs 2023-09-10T00:38:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4765.097122 secs 2023-09-10T00:38:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4765.810347 secs 2023-09-10T00:38:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:55 smithi098 ceph-mon[103124]: pgmap v2708: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T00:38:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4765.810347 secs 2023-09-10T00:38:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:55 smithi170 ceph-mon[102832]: pgmap v2708: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T00:38:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4765.810347 secs 2023-09-10T00:38:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:55 smithi082 ceph-mon[98694]: pgmap v2708: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T00:38:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:57 smithi170 ceph-mon[102832]: pgmap v2709: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 139 KiB/s wr, 17 op/s 2023-09-10T00:38:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:57 smithi082 ceph-mon[98694]: pgmap v2709: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 139 KiB/s wr, 17 op/s 2023-09-10T00:38:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:57 smithi098 ceph-mon[103124]: pgmap v2709: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 139 KiB/s wr, 17 op/s 2023-09-10T00:38:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4768.562216 secs 2023-09-10T00:38:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4768.562216 secs 2023-09-10T00:38:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4768.562216 secs 2023-09-10T00:38:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:59 smithi170 ceph-mon[102832]: pgmap v2710: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 77 KiB/s wr, 10 op/s 2023-09-10T00:38:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:38:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4770.097180 secs 2023-09-10T00:38:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:59 smithi082 ceph-mon[98694]: pgmap v2710: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 77 KiB/s wr, 10 op/s 2023-09-10T00:38:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:38:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4770.097180 secs 2023-09-10T00:38:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:59 smithi098 ceph-mon[103124]: pgmap v2710: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 77 KiB/s wr, 10 op/s 2023-09-10T00:38:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:38:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4770.097180 secs 2023-09-10T00:39:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4770.810413 secs 2023-09-10T00:39:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4770.810413 secs 2023-09-10T00:39:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4770.810413 secs 2023-09-10T00:39:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:01 smithi170 ceph-mon[102832]: pgmap v2711: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 77 KiB/s wr, 10 op/s 2023-09-10T00:39:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:01 smithi082 ceph-mon[98694]: pgmap v2711: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 77 KiB/s wr, 10 op/s 2023-09-10T00:39:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:01 smithi098 ceph-mon[103124]: pgmap v2711: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 77 KiB/s wr, 10 op/s 2023-09-10T00:39:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:03 smithi170 ceph-mon[102832]: pgmap v2712: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 106 KiB/s wr, 22 op/s 2023-09-10T00:39:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4773.562295 secs 2023-09-10T00:39:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:03 smithi082 ceph-mon[98694]: pgmap v2712: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 106 KiB/s wr, 22 op/s 2023-09-10T00:39:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4773.562295 secs 2023-09-10T00:39:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:03 smithi098 ceph-mon[103124]: pgmap v2712: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 106 KiB/s wr, 22 op/s 2023-09-10T00:39:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4773.562295 secs 2023-09-10T00:39:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4775.097239 secs 2023-09-10T00:39:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4775.097239 secs 2023-09-10T00:39:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4775.097239 secs 2023-09-10T00:39:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4775.810485 secs 2023-09-10T00:39:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:05 smithi170 ceph-mon[102832]: pgmap v2713: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 51 KiB/s wr, 16 op/s 2023-09-10T00:39:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4775.810485 secs 2023-09-10T00:39:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:05 smithi082 ceph-mon[98694]: pgmap v2713: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 51 KiB/s wr, 16 op/s 2023-09-10T00:39:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4775.810485 secs 2023-09-10T00:39:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:05 smithi098 ceph-mon[103124]: pgmap v2713: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 51 KiB/s wr, 16 op/s 2023-09-10T00:39:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:07 smithi170 ceph-mon[102832]: pgmap v2714: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 111 KiB/s wr, 29 op/s 2023-09-10T00:39:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:07 smithi082 ceph-mon[98694]: pgmap v2714: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 111 KiB/s wr, 29 op/s 2023-09-10T00:39:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:07 smithi098 ceph-mon[103124]: pgmap v2714: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 111 KiB/s wr, 29 op/s 2023-09-10T00:39:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4778.562355 secs 2023-09-10T00:39:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:08 smithi170 ceph-mon[102832]: osdmap e241: 12 total, 12 up, 12 in 2023-09-10T00:39:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4778.562355 secs 2023-09-10T00:39:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:08 smithi082 ceph-mon[98694]: osdmap e241: 12 total, 12 up, 12 in 2023-09-10T00:39:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4778.562355 secs 2023-09-10T00:39:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:08 smithi098 ceph-mon[103124]: osdmap e241: 12 total, 12 up, 12 in 2023-09-10T00:39:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:09 smithi170 ceph-mon[102832]: pgmap v2716: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 107 KiB/s wr, 28 op/s 2023-09-10T00:39:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4780.097297 secs 2023-09-10T00:39:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:39:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:09 smithi082 ceph-mon[98694]: pgmap v2716: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 107 KiB/s wr, 28 op/s 2023-09-10T00:39:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4780.097297 secs 2023-09-10T00:39:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:39:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:09 smithi098 ceph-mon[103124]: pgmap v2716: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 107 KiB/s wr, 28 op/s 2023-09-10T00:39:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4780.097297 secs 2023-09-10T00:39:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:39:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4780.810551 secs 2023-09-10T00:39:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:39:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:39:10.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:39:10.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:39:10.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:10 smithi170 ceph-mon[102832]: pgmap v2717: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 107 KiB/s wr, 28 op/s 2023-09-10T00:39:10.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:39:10.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:39:10.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:39:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4780.810551 secs 2023-09-10T00:39:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:39:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:39:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:39:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:39:10.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:10 smithi082 ceph-mon[98694]: pgmap v2717: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 107 KiB/s wr, 28 op/s 2023-09-10T00:39:10.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:39:10.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:39:10.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:39:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4780.810551 secs 2023-09-10T00:39:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:39:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:39:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:39:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:39:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:10 smithi098 ceph-mon[103124]: pgmap v2717: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 107 KiB/s wr, 28 op/s 2023-09-10T00:39:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:39:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:39:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:39:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:12 smithi170 ceph-mon[102832]: pgmap v2718: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 73 KiB/s wr, 20 op/s 2023-09-10T00:39:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4783.562416 secs 2023-09-10T00:39:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:12 smithi082 ceph-mon[98694]: pgmap v2718: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 73 KiB/s wr, 20 op/s 2023-09-10T00:39:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4783.562416 secs 2023-09-10T00:39:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:12 smithi098 ceph-mon[103124]: pgmap v2718: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 73 KiB/s wr, 20 op/s 2023-09-10T00:39:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4783.562416 secs 2023-09-10T00:39:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4785.097370 secs 2023-09-10T00:39:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4785.097370 secs 2023-09-10T00:39:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4785.097370 secs 2023-09-10T00:39:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4785.810615 secs 2023-09-10T00:39:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:14 smithi170 ceph-mon[102832]: pgmap v2719: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 73 KiB/s wr, 20 op/s 2023-09-10T00:39:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:14 smithi170 ceph-mon[102832]: osdmap e242: 12 total, 12 up, 12 in 2023-09-10T00:39:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4785.810615 secs 2023-09-10T00:39:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:14 smithi082 ceph-mon[98694]: pgmap v2719: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 73 KiB/s wr, 20 op/s 2023-09-10T00:39:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:14 smithi082 ceph-mon[98694]: osdmap e242: 12 total, 12 up, 12 in 2023-09-10T00:39:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4785.810615 secs 2023-09-10T00:39:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:14 smithi098 ceph-mon[103124]: pgmap v2719: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 73 KiB/s wr, 20 op/s 2023-09-10T00:39:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:14 smithi098 ceph-mon[103124]: osdmap e242: 12 total, 12 up, 12 in 2023-09-10T00:39:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:16 smithi170 ceph-mon[102832]: pgmap v2721: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 35 KiB/s wr, 13 op/s 2023-09-10T00:39:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:16 smithi082 ceph-mon[98694]: pgmap v2721: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 35 KiB/s wr, 13 op/s 2023-09-10T00:39:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:16 smithi098 ceph-mon[103124]: pgmap v2721: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 35 KiB/s wr, 13 op/s 2023-09-10T00:39:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:18 smithi170 ceph-mon[102832]: pgmap v2722: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T00:39:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4790.097454 secs 2023-09-10T00:39:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:18 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:39:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:18 smithi082 ceph-mon[98694]: pgmap v2722: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T00:39:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4790.097454 secs 2023-09-10T00:39:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:18 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:39:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:18 smithi098 ceph-mon[103124]: pgmap v2722: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T00:39:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4790.097454 secs 2023-09-10T00:39:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:18 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:39:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4790.810683 secs 2023-09-10T00:39:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:19 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:39:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:19 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:39:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:19 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:39:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4790.810683 secs 2023-09-10T00:39:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:19 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:39:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:19 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:39:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:19 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:39:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4790.810683 secs 2023-09-10T00:39:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:19 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:39:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:19 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:39:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:19 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:39:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:20 smithi170 ceph-mon[102832]: pgmap v2723: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T00:39:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:20 smithi082 ceph-mon[98694]: pgmap v2723: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T00:39:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:20 smithi098 ceph-mon[103124]: pgmap v2723: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T00:39:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:22 smithi170 ceph-mon[102832]: pgmap v2724: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T00:39:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:22 smithi082 ceph-mon[98694]: pgmap v2724: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T00:39:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:22 smithi098 ceph-mon[103124]: pgmap v2724: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T00:39:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4795.097492 secs 2023-09-10T00:39:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4795.097492 secs 2023-09-10T00:39:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4795.097492 secs 2023-09-10T00:39:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4795.810746 secs 2023-09-10T00:39:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:24 smithi170 ceph-mon[102832]: pgmap v2725: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T00:39:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4795.810746 secs 2023-09-10T00:39:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:24 smithi082 ceph-mon[98694]: pgmap v2725: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T00:39:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4795.810746 secs 2023-09-10T00:39:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:24 smithi098 ceph-mon[103124]: pgmap v2725: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T00:39:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:26 smithi170 ceph-mon[102832]: pgmap v2726: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 139 KiB/s wr, 18 op/s 2023-09-10T00:39:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:26 smithi082 ceph-mon[98694]: pgmap v2726: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 139 KiB/s wr, 18 op/s 2023-09-10T00:39:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:26 smithi098 ceph-mon[103124]: pgmap v2726: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 139 KiB/s wr, 18 op/s 2023-09-10T00:39:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:28 smithi170 ceph-mon[102832]: pgmap v2727: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T00:39:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4800.097554 secs 2023-09-10T00:39:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:28 smithi082 ceph-mon[98694]: pgmap v2727: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T00:39:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4800.097554 secs 2023-09-10T00:39:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:28 smithi098 ceph-mon[103124]: pgmap v2727: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T00:39:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4800.097554 secs 2023-09-10T00:39:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4800.810811 secs 2023-09-10T00:39:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4800.810811 secs 2023-09-10T00:39:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4800.810811 secs 2023-09-10T00:39:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:30 smithi170 ceph-mon[102832]: pgmap v2728: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T00:39:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:30 smithi082 ceph-mon[98694]: pgmap v2728: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T00:39:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:30 smithi098 ceph-mon[103124]: pgmap v2728: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T00:39:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:32 smithi170 ceph-mon[102832]: pgmap v2729: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 227 KiB/s wr, 17 op/s 2023-09-10T00:39:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:32 smithi082 ceph-mon[98694]: pgmap v2729: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 227 KiB/s wr, 17 op/s 2023-09-10T00:39:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:32 smithi098 ceph-mon[103124]: pgmap v2729: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 227 KiB/s wr, 17 op/s 2023-09-10T00:39:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4805.097644 secs 2023-09-10T00:39:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4805.097644 secs 2023-09-10T00:39:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4805.097644 secs 2023-09-10T00:39:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4805.810875 secs 2023-09-10T00:39:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:34 smithi170 ceph-mon[102832]: pgmap v2730: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 160 KiB/s wr, 11 op/s 2023-09-10T00:39:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4805.810875 secs 2023-09-10T00:39:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:34 smithi082 ceph-mon[98694]: pgmap v2730: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 160 KiB/s wr, 11 op/s 2023-09-10T00:39:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4805.810875 secs 2023-09-10T00:39:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:34 smithi098 ceph-mon[103124]: pgmap v2730: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 160 KiB/s wr, 11 op/s 2023-09-10T00:39:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:36 smithi170 ceph-mon[102832]: pgmap v2731: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 190 KiB/s wr, 18 op/s 2023-09-10T00:39:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:36 smithi082 ceph-mon[98694]: pgmap v2731: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 190 KiB/s wr, 18 op/s 2023-09-10T00:39:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:36 smithi098 ceph-mon[103124]: pgmap v2731: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 190 KiB/s wr, 18 op/s 2023-09-10T00:39:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:38 smithi170 ceph-mon[102832]: pgmap v2732: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T00:39:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4810.097703 secs 2023-09-10T00:39:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:38 smithi082 ceph-mon[98694]: pgmap v2732: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T00:39:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4810.097703 secs 2023-09-10T00:39:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:38 smithi098 ceph-mon[103124]: pgmap v2732: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T00:39:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4810.097703 secs 2023-09-10T00:39:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4810.810943 secs 2023-09-10T00:39:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4810.810943 secs 2023-09-10T00:39:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4810.810943 secs 2023-09-10T00:39:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:40 smithi170 ceph-mon[102832]: pgmap v2733: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T00:39:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:40 smithi082 ceph-mon[98694]: pgmap v2733: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T00:39:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:40 smithi098 ceph-mon[103124]: pgmap v2733: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T00:39:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:42 smithi170 ceph-mon[102832]: pgmap v2734: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 175 KiB/s wr, 17 op/s 2023-09-10T00:39:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:42 smithi082 ceph-mon[98694]: pgmap v2734: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 175 KiB/s wr, 17 op/s 2023-09-10T00:39:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:42 smithi098 ceph-mon[103124]: pgmap v2734: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 175 KiB/s wr, 17 op/s 2023-09-10T00:39:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4815.097763 secs 2023-09-10T00:39:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4815.097763 secs 2023-09-10T00:39:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4815.097763 secs 2023-09-10T00:39:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4815.811009 secs 2023-09-10T00:39:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:44 smithi170 ceph-mon[102832]: pgmap v2735: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T00:39:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4815.811009 secs 2023-09-10T00:39:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:44 smithi082 ceph-mon[98694]: pgmap v2735: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T00:39:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4815.811009 secs 2023-09-10T00:39:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:44 smithi098 ceph-mon[103124]: pgmap v2735: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T00:39:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:46 smithi170 ceph-mon[102832]: pgmap v2736: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 136 KiB/s wr, 17 op/s 2023-09-10T00:39:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:46 smithi082 ceph-mon[98694]: pgmap v2736: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 136 KiB/s wr, 17 op/s 2023-09-10T00:39:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:46 smithi098 ceph-mon[103124]: pgmap v2736: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 136 KiB/s wr, 17 op/s 2023-09-10T00:39:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4818.562904 secs 2023-09-10T00:39:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4818.562904 secs 2023-09-10T00:39:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4818.562904 secs 2023-09-10T00:39:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:48 smithi170 ceph-mon[102832]: pgmap v2737: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T00:39:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4820.097842 secs 2023-09-10T00:39:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:48 smithi082 ceph-mon[98694]: pgmap v2737: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T00:39:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4820.097842 secs 2023-09-10T00:39:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:48 smithi098 ceph-mon[103124]: pgmap v2737: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T00:39:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4820.097842 secs 2023-09-10T00:39:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4820.811075 secs 2023-09-10T00:39:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4820.811075 secs 2023-09-10T00:39:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4820.811075 secs 2023-09-10T00:39:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:50 smithi170 ceph-mon[102832]: pgmap v2738: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T00:39:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:50 smithi082 ceph-mon[98694]: pgmap v2738: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T00:39:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:50 smithi098 ceph-mon[103124]: pgmap v2738: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T00:39:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:39:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:39:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:39:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:39:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:39:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:39:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:39:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:39:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:39:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:52 smithi170 ceph-mon[102832]: pgmap v2739: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 132 KiB/s wr, 16 op/s 2023-09-10T00:39:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4823.562968 secs 2023-09-10T00:39:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:52 smithi082 ceph-mon[98694]: pgmap v2739: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 132 KiB/s wr, 16 op/s 2023-09-10T00:39:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4823.562968 secs 2023-09-10T00:39:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:52 smithi098 ceph-mon[103124]: pgmap v2739: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 132 KiB/s wr, 16 op/s 2023-09-10T00:39:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4823.562968 secs 2023-09-10T00:39:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4825.097900 secs 2023-09-10T00:39:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4825.097900 secs 2023-09-10T00:39:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4825.097900 secs 2023-09-10T00:39:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4825.811139 secs 2023-09-10T00:39:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:54 smithi170 ceph-mon[102832]: pgmap v2740: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 112 KiB/s wr, 10 op/s 2023-09-10T00:39:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4825.811139 secs 2023-09-10T00:39:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:54 smithi082 ceph-mon[98694]: pgmap v2740: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 112 KiB/s wr, 10 op/s 2023-09-10T00:39:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4825.811139 secs 2023-09-10T00:39:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:54 smithi098 ceph-mon[103124]: pgmap v2740: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 112 KiB/s wr, 10 op/s 2023-09-10T00:39:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:56 smithi170 ceph-mon[102832]: pgmap v2741: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 134 KiB/s wr, 16 op/s 2023-09-10T00:39:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:56 smithi082 ceph-mon[98694]: pgmap v2741: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 134 KiB/s wr, 16 op/s 2023-09-10T00:39:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:56 smithi098 ceph-mon[103124]: pgmap v2741: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 134 KiB/s wr, 16 op/s 2023-09-10T00:39:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4828.563081 secs 2023-09-10T00:39:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4828.563081 secs 2023-09-10T00:39:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4828.563081 secs 2023-09-10T00:39:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:58 smithi170 ceph-mon[102832]: pgmap v2742: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 49 KiB/s wr, 10 op/s 2023-09-10T00:39:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4830.097958 secs 2023-09-10T00:39:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:58 smithi082 ceph-mon[98694]: pgmap v2742: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 49 KiB/s wr, 10 op/s 2023-09-10T00:39:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4830.097958 secs 2023-09-10T00:39:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:58 smithi098 ceph-mon[103124]: pgmap v2742: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 49 KiB/s wr, 10 op/s 2023-09-10T00:39:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4830.097958 secs 2023-09-10T00:40:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:39:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4830.811205 secs 2023-09-10T00:40:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:39:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4830.811205 secs 2023-09-10T00:40:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:39:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4830.811205 secs 2023-09-10T00:40:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:00 smithi170 ceph-mon[102832]: pgmap v2743: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 49 KiB/s wr, 10 op/s 2023-09-10T00:40:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T00:40:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:00 smithi082 ceph-mon[98694]: pgmap v2743: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 49 KiB/s wr, 10 op/s 2023-09-10T00:40:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T00:40:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:00 smithi098 ceph-mon[103124]: pgmap v2743: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 49 KiB/s wr, 10 op/s 2023-09-10T00:40:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T00:40:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:02 smithi170 ceph-mon[102832]: pgmap v2744: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 75 KiB/s wr, 15 op/s 2023-09-10T00:40:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4833.563142 secs 2023-09-10T00:40:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:02 smithi082 ceph-mon[98694]: pgmap v2744: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 75 KiB/s wr, 15 op/s 2023-09-10T00:40:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4833.563142 secs 2023-09-10T00:40:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:02 smithi098 ceph-mon[103124]: pgmap v2744: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 75 KiB/s wr, 15 op/s 2023-09-10T00:40:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4833.563142 secs 2023-09-10T00:40:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4835.098044 secs 2023-09-10T00:40:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4835.098044 secs 2023-09-10T00:40:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4835.098044 secs 2023-09-10T00:40:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4835.811271 secs 2023-09-10T00:40:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:04 smithi170 ceph-mon[102832]: pgmap v2745: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 49 KiB/s wr, 10 op/s 2023-09-10T00:40:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4835.811271 secs 2023-09-10T00:40:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:04 smithi082 ceph-mon[98694]: pgmap v2745: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 49 KiB/s wr, 10 op/s 2023-09-10T00:40:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4835.811271 secs 2023-09-10T00:40:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:04 smithi098 ceph-mon[103124]: pgmap v2745: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 49 KiB/s wr, 10 op/s 2023-09-10T00:40:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:06 smithi170 ceph-mon[102832]: pgmap v2746: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 94 KiB/s wr, 29 op/s 2023-09-10T00:40:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:06 smithi082 ceph-mon[98694]: pgmap v2746: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 94 KiB/s wr, 29 op/s 2023-09-10T00:40:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:06 smithi098 ceph-mon[103124]: pgmap v2746: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 94 KiB/s wr, 29 op/s 2023-09-10T00:40:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4838.563202 secs 2023-09-10T00:40:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:07 smithi170 ceph-mon[102832]: osdmap e243: 12 total, 12 up, 12 in 2023-09-10T00:40:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4838.563202 secs 2023-09-10T00:40:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:07 smithi082 ceph-mon[98694]: osdmap e243: 12 total, 12 up, 12 in 2023-09-10T00:40:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4838.563202 secs 2023-09-10T00:40:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:07 smithi098 ceph-mon[103124]: osdmap e243: 12 total, 12 up, 12 in 2023-09-10T00:40:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:08 smithi170 ceph-mon[102832]: pgmap v2748: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T00:40:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4840.098142 secs 2023-09-10T00:40:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:08 smithi082 ceph-mon[98694]: pgmap v2748: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T00:40:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4840.098142 secs 2023-09-10T00:40:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:08 smithi098 ceph-mon[103124]: pgmap v2748: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T00:40:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4840.098142 secs 2023-09-10T00:40:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4840.811337 secs 2023-09-10T00:40:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4840.811337 secs 2023-09-10T00:40:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4840.811337 secs 2023-09-10T00:40:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:10 smithi170 ceph-mon[102832]: pgmap v2749: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T00:40:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:40:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:40:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:40:11.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:40:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:10 smithi082 ceph-mon[98694]: pgmap v2749: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T00:40:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:40:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:40:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:40:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:40:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:10 smithi098 ceph-mon[103124]: pgmap v2749: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T00:40:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:40:11.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:40:11.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:40:11.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:40:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:12 smithi170 ceph-mon[102832]: pgmap v2750: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 94 KiB/s wr, 29 op/s 2023-09-10T00:40:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4843.563303 secs 2023-09-10T00:40:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:12 smithi082 ceph-mon[98694]: pgmap v2750: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 94 KiB/s wr, 29 op/s 2023-09-10T00:40:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4843.563303 secs 2023-09-10T00:40:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:12 smithi098 ceph-mon[103124]: pgmap v2750: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 94 KiB/s wr, 29 op/s 2023-09-10T00:40:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4843.563303 secs 2023-09-10T00:40:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4845.098201 secs 2023-09-10T00:40:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4845.098201 secs 2023-09-10T00:40:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4845.098201 secs 2023-09-10T00:40:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4845.811402 secs 2023-09-10T00:40:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:14 smithi170 ceph-mon[102832]: pgmap v2751: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 94 KiB/s wr, 29 op/s 2023-09-10T00:40:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:14 smithi170 ceph-mon[102832]: osdmap e244: 12 total, 12 up, 12 in 2023-09-10T00:40:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4845.811402 secs 2023-09-10T00:40:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:14 smithi082 ceph-mon[98694]: pgmap v2751: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 94 KiB/s wr, 29 op/s 2023-09-10T00:40:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:14 smithi082 ceph-mon[98694]: osdmap e244: 12 total, 12 up, 12 in 2023-09-10T00:40:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4845.811402 secs 2023-09-10T00:40:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:14 smithi098 ceph-mon[103124]: pgmap v2751: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 94 KiB/s wr, 29 op/s 2023-09-10T00:40:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:14 smithi098 ceph-mon[103124]: osdmap e244: 12 total, 12 up, 12 in 2023-09-10T00:40:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:16 smithi170 ceph-mon[102832]: pgmap v2753: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T00:40:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:16 smithi082 ceph-mon[98694]: pgmap v2753: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T00:40:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:16 smithi098 ceph-mon[103124]: pgmap v2753: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T00:40:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:18 smithi082 ceph-mon[98694]: pgmap v2754: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T00:40:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4850.098262 secs 2023-09-10T00:40:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:18 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:40:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:18 smithi098 ceph-mon[103124]: pgmap v2754: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T00:40:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4850.098262 secs 2023-09-10T00:40:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:18 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:40:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:18 smithi170 ceph-mon[102832]: pgmap v2754: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T00:40:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4850.098262 secs 2023-09-10T00:40:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:18 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:40:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4850.811467 secs 2023-09-10T00:40:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:19 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:40:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:19 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:40:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:19 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:40:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4850.811467 secs 2023-09-10T00:40:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:19 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:40:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:19 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:40:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:19 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:40:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4850.811467 secs 2023-09-10T00:40:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:19 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:40:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:19 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:40:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:19 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:40:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:20 smithi082 ceph-mon[98694]: pgmap v2755: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T00:40:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:20 smithi098 ceph-mon[103124]: pgmap v2755: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T00:40:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:20 smithi170 ceph-mon[102832]: pgmap v2755: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T00:40:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:22 smithi082 ceph-mon[98694]: pgmap v2756: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T00:40:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:22 smithi098 ceph-mon[103124]: pgmap v2756: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T00:40:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:22 smithi170 ceph-mon[102832]: pgmap v2756: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T00:40:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4855.098319 secs 2023-09-10T00:40:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4855.098319 secs 2023-09-10T00:40:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4855.098319 secs 2023-09-10T00:40:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4855.811489 secs 2023-09-10T00:40:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:24 smithi082 ceph-mon[98694]: pgmap v2757: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T00:40:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4855.811489 secs 2023-09-10T00:40:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:24 smithi098 ceph-mon[103124]: pgmap v2757: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T00:40:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4855.811489 secs 2023-09-10T00:40:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:24 smithi170 ceph-mon[102832]: pgmap v2757: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T00:40:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:26 smithi082 ceph-mon[98694]: pgmap v2758: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 171 KiB/s wr, 19 op/s 2023-09-10T00:40:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:26 smithi098 ceph-mon[103124]: pgmap v2758: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 171 KiB/s wr, 19 op/s 2023-09-10T00:40:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:26 smithi170 ceph-mon[102832]: pgmap v2758: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 171 KiB/s wr, 19 op/s 2023-09-10T00:40:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:28 smithi098 ceph-mon[103124]: pgmap v2759: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 156 KiB/s wr, 13 op/s 2023-09-10T00:40:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4860.098376 secs 2023-09-10T00:40:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:28 smithi170 ceph-mon[102832]: pgmap v2759: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 156 KiB/s wr, 13 op/s 2023-09-10T00:40:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4860.098376 secs 2023-09-10T00:40:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:28 smithi082 ceph-mon[98694]: pgmap v2759: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 156 KiB/s wr, 13 op/s 2023-09-10T00:40:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4860.098376 secs 2023-09-10T00:40:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4860.811552 secs 2023-09-10T00:40:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4860.811552 secs 2023-09-10T00:40:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4860.811552 secs 2023-09-10T00:40:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:30 smithi098 ceph-mon[103124]: pgmap v2760: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 156 KiB/s wr, 13 op/s 2023-09-10T00:40:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:30 smithi170 ceph-mon[102832]: pgmap v2760: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 156 KiB/s wr, 13 op/s 2023-09-10T00:40:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:30 smithi082 ceph-mon[98694]: pgmap v2760: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 156 KiB/s wr, 13 op/s 2023-09-10T00:40:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:32 smithi098 ceph-mon[103124]: pgmap v2761: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 237 KiB/s wr, 32 op/s 2023-09-10T00:40:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:32 smithi170 ceph-mon[102832]: pgmap v2761: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 237 KiB/s wr, 32 op/s 2023-09-10T00:40:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:32 smithi082 ceph-mon[98694]: pgmap v2761: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 237 KiB/s wr, 32 op/s 2023-09-10T00:40:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4865.098437 secs 2023-09-10T00:40:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4865.098437 secs 2023-09-10T00:40:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4865.098437 secs 2023-09-10T00:40:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4865.811617 secs 2023-09-10T00:40:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:34 smithi098 ceph-mon[103124]: pgmap v2762: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 199 KiB/s wr, 27 op/s 2023-09-10T00:40:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4865.811617 secs 2023-09-10T00:40:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:34 smithi170 ceph-mon[102832]: pgmap v2762: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 199 KiB/s wr, 27 op/s 2023-09-10T00:40:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4865.811617 secs 2023-09-10T00:40:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:34 smithi082 ceph-mon[98694]: pgmap v2762: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 199 KiB/s wr, 27 op/s 2023-09-10T00:40:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:36 smithi098 ceph-mon[103124]: pgmap v2763: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 230 KiB/s wr, 33 op/s 2023-09-10T00:40:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:36 smithi170 ceph-mon[102832]: pgmap v2763: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 230 KiB/s wr, 33 op/s 2023-09-10T00:40:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:36 smithi082 ceph-mon[98694]: pgmap v2763: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 230 KiB/s wr, 33 op/s 2023-09-10T00:40:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:38 smithi098 ceph-mon[103124]: pgmap v2764: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 111 KiB/s wr, 24 op/s 2023-09-10T00:40:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4870.098519 secs 2023-09-10T00:40:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:38 smithi170 ceph-mon[102832]: pgmap v2764: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 111 KiB/s wr, 24 op/s 2023-09-10T00:40:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4870.098519 secs 2023-09-10T00:40:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:38 smithi082 ceph-mon[98694]: pgmap v2764: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 111 KiB/s wr, 24 op/s 2023-09-10T00:40:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4870.098519 secs 2023-09-10T00:40:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4870.811682 secs 2023-09-10T00:40:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4870.811682 secs 2023-09-10T00:40:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4870.811682 secs 2023-09-10T00:40:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:40 smithi098 ceph-mon[103124]: pgmap v2765: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 111 KiB/s wr, 24 op/s 2023-09-10T00:40:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:40 smithi170 ceph-mon[102832]: pgmap v2765: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 111 KiB/s wr, 24 op/s 2023-09-10T00:40:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:40 smithi082 ceph-mon[98694]: pgmap v2765: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 111 KiB/s wr, 24 op/s 2023-09-10T00:40:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:42 smithi098 ceph-mon[103124]: pgmap v2766: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 215 KiB/s wr, 30 op/s 2023-09-10T00:40:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4873.563778 secs 2023-09-10T00:40:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:42 smithi170 ceph-mon[102832]: pgmap v2766: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 215 KiB/s wr, 30 op/s 2023-09-10T00:40:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4873.563778 secs 2023-09-10T00:40:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:42 smithi082 ceph-mon[98694]: pgmap v2766: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 215 KiB/s wr, 30 op/s 2023-09-10T00:40:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4873.563778 secs 2023-09-10T00:40:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4875.098577 secs 2023-09-10T00:40:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:40:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:40:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:40:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4875.098577 secs 2023-09-10T00:40:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:40:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:40:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:40:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4875.098577 secs 2023-09-10T00:40:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:40:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:40:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:40:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4875.811747 secs 2023-09-10T00:40:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:45 smithi170 ceph-mon[102832]: pgmap v2767: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T00:40:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4875.811747 secs 2023-09-10T00:40:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:45 smithi082 ceph-mon[98694]: pgmap v2767: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T00:40:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4875.811747 secs 2023-09-10T00:40:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:45 smithi098 ceph-mon[103124]: pgmap v2767: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T00:40:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:47 smithi170 ceph-mon[102832]: pgmap v2768: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 183 KiB/s wr, 16 op/s 2023-09-10T00:40:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:47 smithi082 ceph-mon[98694]: pgmap v2768: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 183 KiB/s wr, 16 op/s 2023-09-10T00:40:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:47 smithi098 ceph-mon[103124]: pgmap v2768: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 183 KiB/s wr, 16 op/s 2023-09-10T00:40:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4878.563832 secs 2023-09-10T00:40:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4878.563832 secs 2023-09-10T00:40:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4878.563832 secs 2023-09-10T00:40:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:49 smithi170 ceph-mon[102832]: pgmap v2769: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T00:40:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4880.098664 secs 2023-09-10T00:40:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:49 smithi082 ceph-mon[98694]: pgmap v2769: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T00:40:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4880.098664 secs 2023-09-10T00:40:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:49 smithi098 ceph-mon[103124]: pgmap v2769: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T00:40:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4880.098664 secs 2023-09-10T00:40:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4880.811812 secs 2023-09-10T00:40:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4880.811812 secs 2023-09-10T00:40:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4880.811812 secs 2023-09-10T00:40:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:51 smithi170 ceph-mon[102832]: pgmap v2770: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T00:40:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:51 smithi082 ceph-mon[98694]: pgmap v2770: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T00:40:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:51 smithi098 ceph-mon[103124]: pgmap v2770: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T00:40:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:53 smithi170 ceph-mon[102832]: pgmap v2771: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 183 KiB/s wr, 16 op/s 2023-09-10T00:40:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4883.563893 secs 2023-09-10T00:40:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:53 smithi082 ceph-mon[98694]: pgmap v2771: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 183 KiB/s wr, 16 op/s 2023-09-10T00:40:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4883.563893 secs 2023-09-10T00:40:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:53 smithi098 ceph-mon[103124]: pgmap v2771: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 183 KiB/s wr, 16 op/s 2023-09-10T00:40:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4883.563893 secs 2023-09-10T00:40:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4885.098746 secs 2023-09-10T00:40:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4885.098746 secs 2023-09-10T00:40:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4885.098746 secs 2023-09-10T00:40:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4885.811875 secs 2023-09-10T00:40:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:55 smithi170 ceph-mon[102832]: pgmap v2772: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 78 KiB/s wr, 10 op/s 2023-09-10T00:40:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4885.811875 secs 2023-09-10T00:40:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:55 smithi082 ceph-mon[98694]: pgmap v2772: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 78 KiB/s wr, 10 op/s 2023-09-10T00:40:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4885.811875 secs 2023-09-10T00:40:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:55 smithi098 ceph-mon[103124]: pgmap v2772: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 78 KiB/s wr, 10 op/s 2023-09-10T00:40:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:57 smithi170 ceph-mon[102832]: pgmap v2773: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 97 KiB/s wr, 16 op/s 2023-09-10T00:40:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:57 smithi082 ceph-mon[98694]: pgmap v2773: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 97 KiB/s wr, 16 op/s 2023-09-10T00:40:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:57 smithi098 ceph-mon[103124]: pgmap v2773: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 97 KiB/s wr, 16 op/s 2023-09-10T00:40:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4888.564011 secs 2023-09-10T00:40:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4888.564011 secs 2023-09-10T00:40:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4888.564011 secs 2023-09-10T00:40:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:59 smithi170 ceph-mon[102832]: pgmap v2774: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T00:40:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:40:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4890.098808 secs 2023-09-10T00:40:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:59 smithi082 ceph-mon[98694]: pgmap v2774: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T00:40:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:40:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4890.098808 secs 2023-09-10T00:40:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:59 smithi098 ceph-mon[103124]: pgmap v2774: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T00:40:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:40:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4890.098808 secs 2023-09-10T00:41:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4890.811942 secs 2023-09-10T00:41:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4890.811942 secs 2023-09-10T00:41:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4890.811942 secs 2023-09-10T00:41:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:01 smithi170 ceph-mon[102832]: pgmap v2775: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T00:41:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:01 smithi082 ceph-mon[98694]: pgmap v2775: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T00:41:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:01 smithi098 ceph-mon[103124]: pgmap v2775: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T00:41:03.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:03 smithi170 ceph-mon[102832]: pgmap v2776: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 49 KiB/s wr, 15 op/s 2023-09-10T00:41:03.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4893.564113 secs 2023-09-10T00:41:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:03 smithi082 ceph-mon[98694]: pgmap v2776: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 49 KiB/s wr, 15 op/s 2023-09-10T00:41:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4893.564113 secs 2023-09-10T00:41:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:03 smithi098 ceph-mon[103124]: pgmap v2776: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 49 KiB/s wr, 15 op/s 2023-09-10T00:41:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4893.564113 secs 2023-09-10T00:41:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4895.098890 secs 2023-09-10T00:41:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4895.098890 secs 2023-09-10T00:41:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4895.098890 secs 2023-09-10T00:41:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4895.812010 secs 2023-09-10T00:41:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:05 smithi170 ceph-mon[102832]: pgmap v2777: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 19 KiB/s wr, 9 op/s 2023-09-10T00:41:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4895.812010 secs 2023-09-10T00:41:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:05 smithi082 ceph-mon[98694]: pgmap v2777: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 19 KiB/s wr, 9 op/s 2023-09-10T00:41:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4895.812010 secs 2023-09-10T00:41:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:05 smithi098 ceph-mon[103124]: pgmap v2777: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 19 KiB/s wr, 9 op/s 2023-09-10T00:41:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:07 smithi170 ceph-mon[102832]: pgmap v2778: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T00:41:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:07 smithi082 ceph-mon[98694]: pgmap v2778: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T00:41:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:07 smithi098 ceph-mon[103124]: pgmap v2778: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T00:41:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4898.564173 secs 2023-09-10T00:41:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:08 smithi170 ceph-mon[102832]: osdmap e245: 12 total, 12 up, 12 in 2023-09-10T00:41:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4898.564173 secs 2023-09-10T00:41:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:08 smithi082 ceph-mon[98694]: osdmap e245: 12 total, 12 up, 12 in 2023-09-10T00:41:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4898.564173 secs 2023-09-10T00:41:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:08 smithi098 ceph-mon[103124]: osdmap e245: 12 total, 12 up, 12 in 2023-09-10T00:41:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:09 smithi170 ceph-mon[102832]: pgmap v2780: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:41:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4900.098989 secs 2023-09-10T00:41:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:09 smithi082 ceph-mon[98694]: pgmap v2780: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:41:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4900.098989 secs 2023-09-10T00:41:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:09 smithi098 ceph-mon[103124]: pgmap v2780: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:41:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4900.098989 secs 2023-09-10T00:41:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4900.812078 secs 2023-09-10T00:41:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4900.812078 secs 2023-09-10T00:41:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4900.812078 secs 2023-09-10T00:41:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:11 smithi170 ceph-mon[102832]: pgmap v2781: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:41:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:41:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:11 smithi082 ceph-mon[98694]: pgmap v2781: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:41:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:41:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:11 smithi098 ceph-mon[103124]: pgmap v2781: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:41:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:41:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:41:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:41:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:41:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:41:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:41:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:41:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:41:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:41:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:41:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:13 smithi170 ceph-mon[102832]: pgmap v2782: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:41:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4903.564301 secs 2023-09-10T00:41:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:13 smithi082 ceph-mon[98694]: pgmap v2782: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:41:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4903.564301 secs 2023-09-10T00:41:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:13 smithi098 ceph-mon[103124]: pgmap v2782: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:41:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4903.564301 secs 2023-09-10T00:41:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4905.099082 secs 2023-09-10T00:41:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4905.099082 secs 2023-09-10T00:41:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4905.099082 secs 2023-09-10T00:41:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4905.812145 secs 2023-09-10T00:41:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:15 smithi170 ceph-mon[102832]: pgmap v2783: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:41:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:15 smithi170 ceph-mon[102832]: osdmap e246: 12 total, 12 up, 12 in 2023-09-10T00:41:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4905.812145 secs 2023-09-10T00:41:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:15 smithi082 ceph-mon[98694]: pgmap v2783: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:41:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:15 smithi082 ceph-mon[98694]: osdmap e246: 12 total, 12 up, 12 in 2023-09-10T00:41:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4905.812145 secs 2023-09-10T00:41:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:15 smithi098 ceph-mon[103124]: pgmap v2783: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:41:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:15 smithi098 ceph-mon[103124]: osdmap e246: 12 total, 12 up, 12 in 2023-09-10T00:41:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:17 smithi170 ceph-mon[102832]: pgmap v2785: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 47 KiB/s wr, 13 op/s 2023-09-10T00:41:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:17 smithi082 ceph-mon[98694]: pgmap v2785: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 47 KiB/s wr, 13 op/s 2023-09-10T00:41:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:17 smithi098 ceph-mon[103124]: pgmap v2785: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 47 KiB/s wr, 13 op/s 2023-09-10T00:41:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:19 smithi170 ceph-mon[102832]: pgmap v2786: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T00:41:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4910.099160 secs 2023-09-10T00:41:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:41:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:19 smithi082 ceph-mon[98694]: pgmap v2786: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T00:41:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4910.099160 secs 2023-09-10T00:41:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:41:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:19 smithi098 ceph-mon[103124]: pgmap v2786: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T00:41:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4910.099160 secs 2023-09-10T00:41:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:41:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4910.812207 secs 2023-09-10T00:41:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:41:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:41:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:41:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4910.812207 secs 2023-09-10T00:41:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:41:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:41:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:41:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4910.812207 secs 2023-09-10T00:41:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:41:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:41:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:41:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:21 smithi170 ceph-mon[102832]: pgmap v2787: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T00:41:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:21 smithi082 ceph-mon[98694]: pgmap v2787: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T00:41:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:21 smithi098 ceph-mon[103124]: pgmap v2787: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T00:41:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:23 smithi170 ceph-mon[102832]: pgmap v2788: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T00:41:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:23 smithi082 ceph-mon[98694]: pgmap v2788: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T00:41:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:23 smithi098 ceph-mon[103124]: pgmap v2788: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T00:41:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4915.099216 secs 2023-09-10T00:41:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4915.099216 secs 2023-09-10T00:41:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4915.099216 secs 2023-09-10T00:41:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4915.812272 secs 2023-09-10T00:41:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:25 smithi170 ceph-mon[102832]: pgmap v2789: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T00:41:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4915.812272 secs 2023-09-10T00:41:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:25 smithi082 ceph-mon[98694]: pgmap v2789: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T00:41:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4915.812272 secs 2023-09-10T00:41:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:25 smithi098 ceph-mon[103124]: pgmap v2789: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T00:41:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:27 smithi170 ceph-mon[102832]: pgmap v2790: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 108 KiB/s wr, 16 op/s 2023-09-10T00:41:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:27 smithi082 ceph-mon[98694]: pgmap v2790: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 108 KiB/s wr, 16 op/s 2023-09-10T00:41:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:27 smithi098 ceph-mon[103124]: pgmap v2790: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 108 KiB/s wr, 16 op/s 2023-09-10T00:41:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:29 smithi170 ceph-mon[102832]: pgmap v2791: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 66 KiB/s wr, 10 op/s 2023-09-10T00:41:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4920.099285 secs 2023-09-10T00:41:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:29 smithi082 ceph-mon[98694]: pgmap v2791: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 66 KiB/s wr, 10 op/s 2023-09-10T00:41:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4920.099285 secs 2023-09-10T00:41:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:29 smithi098 ceph-mon[103124]: pgmap v2791: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 66 KiB/s wr, 10 op/s 2023-09-10T00:41:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4920.099285 secs 2023-09-10T00:41:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4920.812340 secs 2023-09-10T00:41:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4920.812340 secs 2023-09-10T00:41:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4920.812340 secs 2023-09-10T00:41:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:31 smithi170 ceph-mon[102832]: pgmap v2792: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 66 KiB/s wr, 10 op/s 2023-09-10T00:41:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:31 smithi082 ceph-mon[98694]: pgmap v2792: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 66 KiB/s wr, 10 op/s 2023-09-10T00:41:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:31 smithi098 ceph-mon[103124]: pgmap v2792: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 66 KiB/s wr, 10 op/s 2023-09-10T00:41:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:33 smithi170 ceph-mon[102832]: pgmap v2793: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 158 KiB/s wr, 17 op/s 2023-09-10T00:41:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:33 smithi082 ceph-mon[98694]: pgmap v2793: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 158 KiB/s wr, 17 op/s 2023-09-10T00:41:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:33 smithi098 ceph-mon[103124]: pgmap v2793: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 158 KiB/s wr, 17 op/s 2023-09-10T00:41:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4925.099355 secs 2023-09-10T00:41:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4925.099355 secs 2023-09-10T00:41:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4925.099355 secs 2023-09-10T00:41:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4925.812407 secs 2023-09-10T00:41:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:35 smithi170 ceph-mon[102832]: pgmap v2794: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T00:41:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4925.812407 secs 2023-09-10T00:41:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:35 smithi082 ceph-mon[98694]: pgmap v2794: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T00:41:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4925.812407 secs 2023-09-10T00:41:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:35 smithi098 ceph-mon[103124]: pgmap v2794: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T00:41:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:37 smithi170 ceph-mon[102832]: pgmap v2795: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 167 KiB/s wr, 18 op/s 2023-09-10T00:41:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:37 smithi082 ceph-mon[98694]: pgmap v2795: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 167 KiB/s wr, 18 op/s 2023-09-10T00:41:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:37 smithi098 ceph-mon[103124]: pgmap v2795: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 167 KiB/s wr, 18 op/s 2023-09-10T00:41:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:39 smithi170 ceph-mon[102832]: pgmap v2796: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T00:41:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4930.099440 secs 2023-09-10T00:41:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:39 smithi082 ceph-mon[98694]: pgmap v2796: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T00:41:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4930.099440 secs 2023-09-10T00:41:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:39 smithi098 ceph-mon[103124]: pgmap v2796: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T00:41:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4930.099440 secs 2023-09-10T00:41:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4930.812477 secs 2023-09-10T00:41:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4930.812477 secs 2023-09-10T00:41:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4930.812477 secs 2023-09-10T00:41:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:41 smithi170 ceph-mon[102832]: pgmap v2797: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T00:41:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:41 smithi082 ceph-mon[98694]: pgmap v2797: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T00:41:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:41 smithi098 ceph-mon[103124]: pgmap v2797: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T00:41:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:43 smithi170 ceph-mon[102832]: pgmap v2798: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 216 KiB/s wr, 17 op/s 2023-09-10T00:41:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:43 smithi082 ceph-mon[98694]: pgmap v2798: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 216 KiB/s wr, 17 op/s 2023-09-10T00:41:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:43 smithi098 ceph-mon[103124]: pgmap v2798: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 216 KiB/s wr, 17 op/s 2023-09-10T00:41:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4935.099578 secs 2023-09-10T00:41:44.509 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4935.099578 secs 2023-09-10T00:41:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4935.099578 secs 2023-09-10T00:41:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4935.812542 secs 2023-09-10T00:41:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:45 smithi082 ceph-mon[98694]: pgmap v2799: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T00:41:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4935.812542 secs 2023-09-10T00:41:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:45 smithi098 ceph-mon[103124]: pgmap v2799: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T00:41:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4935.812542 secs 2023-09-10T00:41:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:45 smithi170 ceph-mon[102832]: pgmap v2799: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T00:41:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:47 smithi082 ceph-mon[98694]: pgmap v2800: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 166 KiB/s wr, 17 op/s 2023-09-10T00:41:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:47 smithi098 ceph-mon[103124]: pgmap v2800: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 166 KiB/s wr, 17 op/s 2023-09-10T00:41:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:47 smithi170 ceph-mon[102832]: pgmap v2800: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 166 KiB/s wr, 17 op/s 2023-09-10T00:41:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4938.564773 secs 2023-09-10T00:41:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4938.564773 secs 2023-09-10T00:41:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4938.564773 secs 2023-09-10T00:41:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:49 smithi082 ceph-mon[98694]: pgmap v2801: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T00:41:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4940.099637 secs 2023-09-10T00:41:49.521 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:49 smithi098 ceph-mon[103124]: pgmap v2801: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T00:41:49.522 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4940.099637 secs 2023-09-10T00:41:49.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:49 smithi170 ceph-mon[102832]: pgmap v2801: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T00:41:49.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4940.099637 secs 2023-09-10T00:41:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4940.812609 secs 2023-09-10T00:41:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4940.812609 secs 2023-09-10T00:41:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4940.812609 secs 2023-09-10T00:41:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:51 smithi082 ceph-mon[98694]: pgmap v2802: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T00:41:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:51 smithi098 ceph-mon[103124]: pgmap v2802: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T00:41:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:51 smithi170 ceph-mon[102832]: pgmap v2802: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T00:41:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:41:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:41:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:41:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:41:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:41:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:41:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:41:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:41:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:41:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:53 smithi098 ceph-mon[103124]: pgmap v2803: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 166 KiB/s wr, 17 op/s 2023-09-10T00:41:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4943.564833 secs 2023-09-10T00:41:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:53 smithi170 ceph-mon[102832]: pgmap v2803: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 166 KiB/s wr, 17 op/s 2023-09-10T00:41:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4943.564833 secs 2023-09-10T00:41:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:53 smithi082 ceph-mon[98694]: pgmap v2803: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 166 KiB/s wr, 17 op/s 2023-09-10T00:41:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4943.564833 secs 2023-09-10T00:41:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4945.099697 secs 2023-09-10T00:41:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4945.099697 secs 2023-09-10T00:41:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4945.099697 secs 2023-09-10T00:41:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4945.812683 secs 2023-09-10T00:41:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:55 smithi098 ceph-mon[103124]: pgmap v2804: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 81 KiB/s wr, 12 op/s 2023-09-10T00:41:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4945.812683 secs 2023-09-10T00:41:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:55 smithi170 ceph-mon[102832]: pgmap v2804: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 81 KiB/s wr, 12 op/s 2023-09-10T00:41:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4945.812683 secs 2023-09-10T00:41:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:55 smithi082 ceph-mon[98694]: pgmap v2804: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 81 KiB/s wr, 12 op/s 2023-09-10T00:41:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:57 smithi098 ceph-mon[103124]: pgmap v2805: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 104 KiB/s wr, 17 op/s 2023-09-10T00:41:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:57 smithi170 ceph-mon[102832]: pgmap v2805: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 104 KiB/s wr, 17 op/s 2023-09-10T00:41:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:57 smithi082 ceph-mon[98694]: pgmap v2805: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 104 KiB/s wr, 17 op/s 2023-09-10T00:41:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4948.564933 secs 2023-09-10T00:41:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4948.564933 secs 2023-09-10T00:41:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4948.564933 secs 2023-09-10T00:41:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:59 smithi098 ceph-mon[103124]: pgmap v2806: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T00:41:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:41:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4950.099752 secs 2023-09-10T00:41:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:59 smithi170 ceph-mon[102832]: pgmap v2806: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T00:41:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:41:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4950.099752 secs 2023-09-10T00:41:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:59 smithi082 ceph-mon[98694]: pgmap v2806: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T00:41:59.749 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:41:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4950.099752 secs 2023-09-10T00:42:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4950.812745 secs 2023-09-10T00:42:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4950.812745 secs 2023-09-10T00:42:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4950.812745 secs 2023-09-10T00:42:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:01 smithi098 ceph-mon[103124]: pgmap v2807: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T00:42:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:01 smithi170 ceph-mon[102832]: pgmap v2807: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T00:42:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:01 smithi082 ceph-mon[98694]: pgmap v2807: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T00:42:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:03 smithi098 ceph-mon[103124]: pgmap v2808: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 62 KiB/s wr, 15 op/s 2023-09-10T00:42:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4953.565006 secs 2023-09-10T00:42:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:03 smithi170 ceph-mon[102832]: pgmap v2808: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 62 KiB/s wr, 15 op/s 2023-09-10T00:42:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4953.565006 secs 2023-09-10T00:42:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:03 smithi082 ceph-mon[98694]: pgmap v2808: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 62 KiB/s wr, 15 op/s 2023-09-10T00:42:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4953.565006 secs 2023-09-10T00:42:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4955.099818 secs 2023-09-10T00:42:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4955.099818 secs 2023-09-10T00:42:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4955.099818 secs 2023-09-10T00:42:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4955.812810 secs 2023-09-10T00:42:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:05 smithi170 ceph-mon[102832]: pgmap v2809: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 24 KiB/s wr, 9 op/s 2023-09-10T00:42:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4955.812810 secs 2023-09-10T00:42:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:05 smithi082 ceph-mon[98694]: pgmap v2809: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 24 KiB/s wr, 9 op/s 2023-09-10T00:42:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4955.812810 secs 2023-09-10T00:42:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:05 smithi098 ceph-mon[103124]: pgmap v2809: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 24 KiB/s wr, 9 op/s 2023-09-10T00:42:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:07 smithi170 ceph-mon[102832]: pgmap v2810: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 118 KiB/s wr, 29 op/s 2023-09-10T00:42:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:07 smithi082 ceph-mon[98694]: pgmap v2810: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 118 KiB/s wr, 29 op/s 2023-09-10T00:42:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:07 smithi098 ceph-mon[103124]: pgmap v2810: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 118 KiB/s wr, 29 op/s 2023-09-10T00:42:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4958.565066 secs 2023-09-10T00:42:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:08 smithi170 ceph-mon[102832]: osdmap e247: 12 total, 12 up, 12 in 2023-09-10T00:42:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4958.565066 secs 2023-09-10T00:42:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:08 smithi082 ceph-mon[98694]: osdmap e247: 12 total, 12 up, 12 in 2023-09-10T00:42:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4958.565066 secs 2023-09-10T00:42:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:08 smithi098 ceph-mon[103124]: osdmap e247: 12 total, 12 up, 12 in 2023-09-10T00:42:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:09 smithi170 ceph-mon[102832]: pgmap v2812: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T00:42:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4960.099901 secs 2023-09-10T00:42:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:09 smithi082 ceph-mon[98694]: pgmap v2812: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T00:42:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4960.099901 secs 2023-09-10T00:42:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:09 smithi098 ceph-mon[103124]: pgmap v2812: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T00:42:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4960.099901 secs 2023-09-10T00:42:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4960.812876 secs 2023-09-10T00:42:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4960.812876 secs 2023-09-10T00:42:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4960.812876 secs 2023-09-10T00:42:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:11 smithi170 ceph-mon[102832]: pgmap v2813: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T00:42:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:11 smithi082 ceph-mon[98694]: pgmap v2813: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T00:42:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:11 smithi098 ceph-mon[103124]: pgmap v2813: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T00:42:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:42:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:42:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:42:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:13 smithi170 ceph-mon[102832]: pgmap v2814: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T00:42:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4963.565145 secs 2023-09-10T00:42:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:42:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:42:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:42:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:13 smithi082 ceph-mon[98694]: pgmap v2814: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T00:42:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4963.565145 secs 2023-09-10T00:42:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:42:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:42:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:42:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:13 smithi098 ceph-mon[103124]: pgmap v2814: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T00:42:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4963.565145 secs 2023-09-10T00:42:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:42:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:42:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:42:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4965.099958 secs 2023-09-10T00:42:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4965.099958 secs 2023-09-10T00:42:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4965.099958 secs 2023-09-10T00:42:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4965.812939 secs 2023-09-10T00:42:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:15 smithi170 ceph-mon[102832]: pgmap v2815: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T00:42:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:15 smithi170 ceph-mon[102832]: osdmap e248: 12 total, 12 up, 12 in 2023-09-10T00:42:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4965.812939 secs 2023-09-10T00:42:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:15 smithi082 ceph-mon[98694]: pgmap v2815: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T00:42:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:15 smithi082 ceph-mon[98694]: osdmap e248: 12 total, 12 up, 12 in 2023-09-10T00:42:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4965.812939 secs 2023-09-10T00:42:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:15 smithi098 ceph-mon[103124]: pgmap v2815: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T00:42:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:15 smithi098 ceph-mon[103124]: osdmap e248: 12 total, 12 up, 12 in 2023-09-10T00:42:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:17 smithi170 ceph-mon[102832]: pgmap v2817: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 37 KiB/s wr, 13 op/s 2023-09-10T00:42:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:17 smithi082 ceph-mon[98694]: pgmap v2817: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 37 KiB/s wr, 13 op/s 2023-09-10T00:42:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:17 smithi098 ceph-mon[103124]: pgmap v2817: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 37 KiB/s wr, 13 op/s 2023-09-10T00:42:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:19 smithi170 ceph-mon[102832]: pgmap v2818: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:42:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4970.100017 secs 2023-09-10T00:42:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:42:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:19 smithi082 ceph-mon[98694]: pgmap v2818: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:42:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4970.100017 secs 2023-09-10T00:42:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:42:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:19 smithi098 ceph-mon[103124]: pgmap v2818: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:42:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4970.100017 secs 2023-09-10T00:42:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:42:20.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4970.813007 secs 2023-09-10T00:42:20.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:42:20.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:42:20.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:42:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4970.813007 secs 2023-09-10T00:42:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:42:20.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:42:20.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:42:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4970.813007 secs 2023-09-10T00:42:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:42:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:42:20.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:42:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:21 smithi170 ceph-mon[102832]: pgmap v2819: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:42:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:21 smithi082 ceph-mon[98694]: pgmap v2819: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:42:21.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:21 smithi098 ceph-mon[103124]: pgmap v2819: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T00:42:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:22 smithi170 ceph-mon[102832]: pgmap v2820: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T00:42:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:22 smithi082 ceph-mon[98694]: pgmap v2820: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T00:42:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:22 smithi098 ceph-mon[103124]: pgmap v2820: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T00:42:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4975.100078 secs 2023-09-10T00:42:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4975.100078 secs 2023-09-10T00:42:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4975.100078 secs 2023-09-10T00:42:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:24 smithi082 ceph-mon[98694]: pgmap v2821: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T00:42:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:24 smithi098 ceph-mon[103124]: pgmap v2821: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T00:42:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:24 smithi170 ceph-mon[102832]: pgmap v2821: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T00:42:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4975.813075 secs 2023-09-10T00:42:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4975.813075 secs 2023-09-10T00:42:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4975.813075 secs 2023-09-10T00:42:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:26 smithi082 ceph-mon[98694]: pgmap v2822: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 144 KiB/s wr, 18 op/s 2023-09-10T00:42:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:26 smithi098 ceph-mon[103124]: pgmap v2822: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 144 KiB/s wr, 18 op/s 2023-09-10T00:42:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:26 smithi170 ceph-mon[102832]: pgmap v2822: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 144 KiB/s wr, 18 op/s 2023-09-10T00:42:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:28 smithi170 ceph-mon[102832]: pgmap v2823: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T00:42:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4980.100161 secs 2023-09-10T00:42:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:28 smithi082 ceph-mon[98694]: pgmap v2823: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T00:42:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4980.100161 secs 2023-09-10T00:42:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:28 smithi098 ceph-mon[103124]: pgmap v2823: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T00:42:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4980.100161 secs 2023-09-10T00:42:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4980.813139 secs 2023-09-10T00:42:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4980.813139 secs 2023-09-10T00:42:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4980.813139 secs 2023-09-10T00:42:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:30 smithi170 ceph-mon[102832]: pgmap v2824: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T00:42:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:30 smithi082 ceph-mon[98694]: pgmap v2824: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T00:42:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:30 smithi098 ceph-mon[103124]: pgmap v2824: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T00:42:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:32 smithi170 ceph-mon[102832]: pgmap v2825: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 235 KiB/s wr, 17 op/s 2023-09-10T00:42:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:32 smithi082 ceph-mon[98694]: pgmap v2825: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 235 KiB/s wr, 17 op/s 2023-09-10T00:42:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:32 smithi098 ceph-mon[103124]: pgmap v2825: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 235 KiB/s wr, 17 op/s 2023-09-10T00:42:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4985.100222 secs 2023-09-10T00:42:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4985.100222 secs 2023-09-10T00:42:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4985.100222 secs 2023-09-10T00:42:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4985.813202 secs 2023-09-10T00:42:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:34 smithi170 ceph-mon[102832]: pgmap v2826: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T00:42:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4985.813202 secs 2023-09-10T00:42:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:34 smithi082 ceph-mon[98694]: pgmap v2826: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T00:42:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4985.813202 secs 2023-09-10T00:42:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:34 smithi098 ceph-mon[103124]: pgmap v2826: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T00:42:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:36 smithi170 ceph-mon[102832]: pgmap v2827: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 196 KiB/s wr, 17 op/s 2023-09-10T00:42:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:36 smithi082 ceph-mon[98694]: pgmap v2827: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 196 KiB/s wr, 17 op/s 2023-09-10T00:42:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:36 smithi098 ceph-mon[103124]: pgmap v2827: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 196 KiB/s wr, 17 op/s 2023-09-10T00:42:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:38 smithi170 ceph-mon[102832]: pgmap v2828: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 11 op/s 2023-09-10T00:42:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4990.100283 secs 2023-09-10T00:42:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:38 smithi082 ceph-mon[98694]: pgmap v2828: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 11 op/s 2023-09-10T00:42:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4990.100283 secs 2023-09-10T00:42:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:38 smithi098 ceph-mon[103124]: pgmap v2828: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 11 op/s 2023-09-10T00:42:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4990.100283 secs 2023-09-10T00:42:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4990.813270 secs 2023-09-10T00:42:40.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4990.813270 secs 2023-09-10T00:42:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4990.813270 secs 2023-09-10T00:42:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:40 smithi170 ceph-mon[102832]: pgmap v2829: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 11 op/s 2023-09-10T00:42:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:40 smithi082 ceph-mon[98694]: pgmap v2829: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 11 op/s 2023-09-10T00:42:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:40 smithi098 ceph-mon[103124]: pgmap v2829: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 11 op/s 2023-09-10T00:42:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:42 smithi170 ceph-mon[102832]: pgmap v2830: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 204 KiB/s wr, 17 op/s 2023-09-10T00:42:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:42 smithi082 ceph-mon[98694]: pgmap v2830: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 204 KiB/s wr, 17 op/s 2023-09-10T00:42:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:42 smithi098 ceph-mon[103124]: pgmap v2830: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 204 KiB/s wr, 17 op/s 2023-09-10T00:42:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4995.100344 secs 2023-09-10T00:42:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4995.100344 secs 2023-09-10T00:42:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4995.100344 secs 2023-09-10T00:42:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 4995.813337 secs 2023-09-10T00:42:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:44 smithi170 ceph-mon[102832]: pgmap v2831: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T00:42:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 4995.813337 secs 2023-09-10T00:42:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:44 smithi082 ceph-mon[98694]: pgmap v2831: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T00:42:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 4995.813337 secs 2023-09-10T00:42:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:44 smithi098 ceph-mon[103124]: pgmap v2831: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T00:42:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:46 smithi170 ceph-mon[102832]: pgmap v2832: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 137 KiB/s wr, 17 op/s 2023-09-10T00:42:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:46 smithi082 ceph-mon[98694]: pgmap v2832: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 137 KiB/s wr, 17 op/s 2023-09-10T00:42:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:46 smithi098 ceph-mon[103124]: pgmap v2832: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 137 KiB/s wr, 17 op/s 2023-09-10T00:42:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 4998.565756 secs 2023-09-10T00:42:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 4998.565756 secs 2023-09-10T00:42:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 4998.565756 secs 2023-09-10T00:42:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:48 smithi170 ceph-mon[102832]: pgmap v2833: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T00:42:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5000.100429 secs 2023-09-10T00:42:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:48 smithi082 ceph-mon[98694]: pgmap v2833: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T00:42:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5000.100429 secs 2023-09-10T00:42:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:48 smithi098 ceph-mon[103124]: pgmap v2833: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T00:42:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5000.100429 secs 2023-09-10T00:42:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5000.813400 secs 2023-09-10T00:42:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5000.813400 secs 2023-09-10T00:42:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5000.813400 secs 2023-09-10T00:42:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:50 smithi170 ceph-mon[102832]: pgmap v2834: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T00:42:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:50 smithi082 ceph-mon[98694]: pgmap v2834: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T00:42:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:50 smithi098 ceph-mon[103124]: pgmap v2834: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T00:42:52.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:42:52.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:42:52.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:42:52.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:42:52.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:42:52.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:42:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:42:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:42:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:42:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:52 smithi170 ceph-mon[102832]: pgmap v2835: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 133 KiB/s wr, 16 op/s 2023-09-10T00:42:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5003.565773 secs 2023-09-10T00:42:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:52 smithi082 ceph-mon[98694]: pgmap v2835: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 133 KiB/s wr, 16 op/s 2023-09-10T00:42:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5003.565773 secs 2023-09-10T00:42:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:52 smithi098 ceph-mon[103124]: pgmap v2835: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 133 KiB/s wr, 16 op/s 2023-09-10T00:42:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5003.565773 secs 2023-09-10T00:42:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5005.100513 secs 2023-09-10T00:42:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5005.100513 secs 2023-09-10T00:42:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5005.100513 secs 2023-09-10T00:42:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5005.813464 secs 2023-09-10T00:42:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:54 smithi170 ceph-mon[102832]: pgmap v2836: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T00:42:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5005.813464 secs 2023-09-10T00:42:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:54 smithi082 ceph-mon[98694]: pgmap v2836: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T00:42:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5005.813464 secs 2023-09-10T00:42:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:54 smithi098 ceph-mon[103124]: pgmap v2836: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T00:42:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:56 smithi170 ceph-mon[102832]: pgmap v2837: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 113 KiB/s wr, 15 op/s 2023-09-10T00:42:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:56 smithi082 ceph-mon[98694]: pgmap v2837: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 113 KiB/s wr, 15 op/s 2023-09-10T00:42:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:56 smithi098 ceph-mon[103124]: pgmap v2837: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 113 KiB/s wr, 15 op/s 2023-09-10T00:42:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5008.565887 secs 2023-09-10T00:42:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5008.565887 secs 2023-09-10T00:42:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5008.565887 secs 2023-09-10T00:42:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:58 smithi170 ceph-mon[102832]: pgmap v2838: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:42:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5010.100612 secs 2023-09-10T00:42:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:58 smithi082 ceph-mon[98694]: pgmap v2838: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:42:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5010.100612 secs 2023-09-10T00:42:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:58 smithi098 ceph-mon[103124]: pgmap v2838: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:42:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5010.100612 secs 2023-09-10T00:43:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:42:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5010.813528 secs 2023-09-10T00:43:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:42:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5010.813528 secs 2023-09-10T00:43:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:42:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5010.813528 secs 2023-09-10T00:43:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:00 smithi170 ceph-mon[102832]: pgmap v2839: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:43:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:00 smithi082 ceph-mon[98694]: pgmap v2839: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:43:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:00 smithi098 ceph-mon[103124]: pgmap v2839: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:43:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:02 smithi170 ceph-mon[102832]: pgmap v2840: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 15 op/s 2023-09-10T00:43:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5013.565946 secs 2023-09-10T00:43:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:02 smithi082 ceph-mon[98694]: pgmap v2840: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 15 op/s 2023-09-10T00:43:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5013.565946 secs 2023-09-10T00:43:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:02 smithi098 ceph-mon[103124]: pgmap v2840: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 15 op/s 2023-09-10T00:43:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5013.565946 secs 2023-09-10T00:43:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5015.100669 secs 2023-09-10T00:43:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5015.100669 secs 2023-09-10T00:43:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5015.100669 secs 2023-09-10T00:43:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5015.813590 secs 2023-09-10T00:43:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:04 smithi170 ceph-mon[102832]: pgmap v2841: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:43:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5015.813590 secs 2023-09-10T00:43:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:04 smithi082 ceph-mon[98694]: pgmap v2841: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:43:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5015.813590 secs 2023-09-10T00:43:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:04 smithi098 ceph-mon[103124]: pgmap v2841: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:43:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:06 smithi170 ceph-mon[102832]: pgmap v2842: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T00:43:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:06 smithi082 ceph-mon[98694]: pgmap v2842: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T00:43:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:06 smithi098 ceph-mon[103124]: pgmap v2842: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T00:43:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5018.566038 secs 2023-09-10T00:43:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:07 smithi170 ceph-mon[102832]: osdmap e249: 12 total, 12 up, 12 in 2023-09-10T00:43:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5018.566038 secs 2023-09-10T00:43:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:07 smithi082 ceph-mon[98694]: osdmap e249: 12 total, 12 up, 12 in 2023-09-10T00:43:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5018.566038 secs 2023-09-10T00:43:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:07 smithi098 ceph-mon[103124]: osdmap e249: 12 total, 12 up, 12 in 2023-09-10T00:43:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:08 smithi170 ceph-mon[102832]: pgmap v2844: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:43:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5020.100729 secs 2023-09-10T00:43:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:08 smithi082 ceph-mon[98694]: pgmap v2844: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:43:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5020.100729 secs 2023-09-10T00:43:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:08 smithi098 ceph-mon[103124]: pgmap v2844: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:43:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5020.100729 secs 2023-09-10T00:43:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5020.813653 secs 2023-09-10T00:43:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5020.813653 secs 2023-09-10T00:43:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5020.813653 secs 2023-09-10T00:43:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:10 smithi082 ceph-mon[98694]: pgmap v2845: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:43:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:10 smithi098 ceph-mon[103124]: pgmap v2845: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:43:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:10 smithi170 ceph-mon[102832]: pgmap v2845: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:43:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:12 smithi082 ceph-mon[98694]: pgmap v2846: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T00:43:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5023.566097 secs 2023-09-10T00:43:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:43:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:12 smithi098 ceph-mon[103124]: pgmap v2846: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T00:43:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5023.566097 secs 2023-09-10T00:43:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:43:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:12 smithi170 ceph-mon[102832]: pgmap v2846: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T00:43:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5023.566097 secs 2023-09-10T00:43:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:43:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5025.100789 secs 2023-09-10T00:43:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5025.100789 secs 2023-09-10T00:43:14.381 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5025.100789 secs 2023-09-10T00:43:15.140 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5025.813719 secs 2023-09-10T00:43:15.141 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:14 smithi098 ceph-mon[103124]: pgmap v2847: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T00:43:15.141 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:14 smithi098 ceph-mon[103124]: osdmap e250: 12 total, 12 up, 12 in 2023-09-10T00:43:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5025.813719 secs 2023-09-10T00:43:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:14 smithi082 ceph-mon[98694]: pgmap v2847: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T00:43:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:14 smithi082 ceph-mon[98694]: osdmap e250: 12 total, 12 up, 12 in 2023-09-10T00:43:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5025.813719 secs 2023-09-10T00:43:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:14 smithi170 ceph-mon[102832]: pgmap v2847: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T00:43:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:14 smithi170 ceph-mon[102832]: osdmap e250: 12 total, 12 up, 12 in 2023-09-10T00:43:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:17 smithi098 ceph-mon[103124]: pgmap v2849: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 13 op/s 2023-09-10T00:43:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:43:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:43:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:43:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:43:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:17 smithi170 ceph-mon[102832]: pgmap v2849: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 13 op/s 2023-09-10T00:43:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:43:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:43:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:43:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:43:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:17 smithi082 ceph-mon[98694]: pgmap v2849: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 13 op/s 2023-09-10T00:43:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:43:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:43:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:43:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:43:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:43:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:43:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:43:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:43:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:43:18.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:18 smithi170 ceph-mon[102832]: pgmap v2850: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T00:43:18.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5030.100847 secs 2023-09-10T00:43:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:43:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:43:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:43:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:43:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:43:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:18 smithi082 ceph-mon[98694]: pgmap v2850: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T00:43:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5030.100847 secs 2023-09-10T00:43:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:43:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:43:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:43:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:43:18.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:43:18.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:18 smithi098 ceph-mon[103124]: pgmap v2850: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T00:43:18.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5030.100847 secs 2023-09-10T00:43:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:43:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:43:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:43:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5030.813783 secs 2023-09-10T00:43:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:43:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:43:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:43:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:20 smithi082 ceph-mon[98694]: pgmap v2851: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T00:43:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5030.813783 secs 2023-09-10T00:43:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:43:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:43:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:43:20.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:20 smithi098 ceph-mon[103124]: pgmap v2851: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T00:43:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5030.813783 secs 2023-09-10T00:43:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:43:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:43:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:43:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:20 smithi170 ceph-mon[102832]: pgmap v2851: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T00:43:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:22 smithi170 ceph-mon[102832]: pgmap v2852: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T00:43:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:22 smithi082 ceph-mon[98694]: pgmap v2852: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T00:43:23.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:22 smithi098 ceph-mon[103124]: pgmap v2852: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T00:43:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5035.100903 secs 2023-09-10T00:43:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5035.100903 secs 2023-09-10T00:43:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5035.100903 secs 2023-09-10T00:43:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5035.813851 secs 2023-09-10T00:43:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:24 smithi170 ceph-mon[102832]: pgmap v2853: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T00:43:25.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5035.813851 secs 2023-09-10T00:43:25.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:24 smithi082 ceph-mon[98694]: pgmap v2853: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T00:43:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5035.813851 secs 2023-09-10T00:43:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:24 smithi098 ceph-mon[103124]: pgmap v2853: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T00:43:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:26 smithi170 ceph-mon[102832]: pgmap v2854: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 152 KiB/s wr, 17 op/s 2023-09-10T00:43:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:26 smithi082 ceph-mon[98694]: pgmap v2854: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 152 KiB/s wr, 17 op/s 2023-09-10T00:43:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:26 smithi098 ceph-mon[103124]: pgmap v2854: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 152 KiB/s wr, 17 op/s 2023-09-10T00:43:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:28 smithi170 ceph-mon[102832]: pgmap v2855: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T00:43:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5040.100961 secs 2023-09-10T00:43:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:28 smithi082 ceph-mon[98694]: pgmap v2855: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T00:43:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5040.100961 secs 2023-09-10T00:43:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:28 smithi098 ceph-mon[103124]: pgmap v2855: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T00:43:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5040.100961 secs 2023-09-10T00:43:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5040.813914 secs 2023-09-10T00:43:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5040.813914 secs 2023-09-10T00:43:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5040.813914 secs 2023-09-10T00:43:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:30 smithi170 ceph-mon[102832]: pgmap v2856: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T00:43:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:30 smithi082 ceph-mon[98694]: pgmap v2856: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T00:43:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:30 smithi098 ceph-mon[103124]: pgmap v2856: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T00:43:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:32 smithi170 ceph-mon[102832]: pgmap v2857: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 245 KiB/s wr, 18 op/s 2023-09-10T00:43:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:32 smithi082 ceph-mon[98694]: pgmap v2857: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 245 KiB/s wr, 18 op/s 2023-09-10T00:43:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:32 smithi098 ceph-mon[103124]: pgmap v2857: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 245 KiB/s wr, 18 op/s 2023-09-10T00:43:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5045.101019 secs 2023-09-10T00:43:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5045.101019 secs 2023-09-10T00:43:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5045.101019 secs 2023-09-10T00:43:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5045.813977 secs 2023-09-10T00:43:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:34 smithi170 ceph-mon[102832]: pgmap v2858: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 206 KiB/s wr, 12 op/s 2023-09-10T00:43:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5045.813977 secs 2023-09-10T00:43:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:34 smithi082 ceph-mon[98694]: pgmap v2858: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 206 KiB/s wr, 12 op/s 2023-09-10T00:43:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5045.813977 secs 2023-09-10T00:43:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:34 smithi098 ceph-mon[103124]: pgmap v2858: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 206 KiB/s wr, 12 op/s 2023-09-10T00:43:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:36 smithi170 ceph-mon[102832]: pgmap v2859: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 237 KiB/s wr, 18 op/s 2023-09-10T00:43:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:36 smithi082 ceph-mon[98694]: pgmap v2859: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 237 KiB/s wr, 18 op/s 2023-09-10T00:43:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:36 smithi098 ceph-mon[103124]: pgmap v2859: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 237 KiB/s wr, 18 op/s 2023-09-10T00:43:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:38 smithi170 ceph-mon[102832]: pgmap v2860: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T00:43:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5050.101080 secs 2023-09-10T00:43:39.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:38 smithi082 ceph-mon[98694]: pgmap v2860: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T00:43:39.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5050.101080 secs 2023-09-10T00:43:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:38 smithi098 ceph-mon[103124]: pgmap v2860: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T00:43:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5050.101080 secs 2023-09-10T00:43:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5050.814044 secs 2023-09-10T00:43:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5050.814044 secs 2023-09-10T00:43:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5050.814044 secs 2023-09-10T00:43:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:40 smithi170 ceph-mon[102832]: pgmap v2861: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T00:43:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:40 smithi082 ceph-mon[98694]: pgmap v2861: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T00:43:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:40 smithi098 ceph-mon[103124]: pgmap v2861: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T00:43:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:42 smithi170 ceph-mon[102832]: pgmap v2862: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 166 KiB/s wr, 17 op/s 2023-09-10T00:43:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:42 smithi082 ceph-mon[98694]: pgmap v2862: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 166 KiB/s wr, 17 op/s 2023-09-10T00:43:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:42 smithi098 ceph-mon[103124]: pgmap v2862: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 166 KiB/s wr, 17 op/s 2023-09-10T00:43:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5055.101139 secs 2023-09-10T00:43:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5055.101139 secs 2023-09-10T00:43:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5055.101139 secs 2023-09-10T00:43:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5055.814112 secs 2023-09-10T00:43:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:44 smithi170 ceph-mon[102832]: pgmap v2863: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 60 KiB/s wr, 11 op/s 2023-09-10T00:43:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5055.814112 secs 2023-09-10T00:43:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:44 smithi082 ceph-mon[98694]: pgmap v2863: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 60 KiB/s wr, 11 op/s 2023-09-10T00:43:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5055.814112 secs 2023-09-10T00:43:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:44 smithi098 ceph-mon[103124]: pgmap v2863: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 60 KiB/s wr, 11 op/s 2023-09-10T00:43:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:46 smithi170 ceph-mon[102832]: pgmap v2864: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 141 KiB/s wr, 16 op/s 2023-09-10T00:43:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:46 smithi082 ceph-mon[98694]: pgmap v2864: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 141 KiB/s wr, 16 op/s 2023-09-10T00:43:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:46 smithi098 ceph-mon[103124]: pgmap v2864: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 141 KiB/s wr, 16 op/s 2023-09-10T00:43:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5058.566560 secs 2023-09-10T00:43:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5058.566560 secs 2023-09-10T00:43:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5058.566560 secs 2023-09-10T00:43:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:48 smithi170 ceph-mon[102832]: pgmap v2865: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 110 KiB/s wr, 10 op/s 2023-09-10T00:43:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5060.101222 secs 2023-09-10T00:43:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:48 smithi082 ceph-mon[98694]: pgmap v2865: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 110 KiB/s wr, 10 op/s 2023-09-10T00:43:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5060.101222 secs 2023-09-10T00:43:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:48 smithi098 ceph-mon[103124]: pgmap v2865: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 110 KiB/s wr, 10 op/s 2023-09-10T00:43:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5060.101222 secs 2023-09-10T00:43:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5060.814180 secs 2023-09-10T00:43:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5060.814180 secs 2023-09-10T00:43:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5060.814180 secs 2023-09-10T00:43:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:50 smithi170 ceph-mon[102832]: pgmap v2866: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 110 KiB/s wr, 10 op/s 2023-09-10T00:43:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:50 smithi082 ceph-mon[98694]: pgmap v2866: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 110 KiB/s wr, 10 op/s 2023-09-10T00:43:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:50 smithi098 ceph-mon[103124]: pgmap v2866: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 110 KiB/s wr, 10 op/s 2023-09-10T00:43:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:43:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:43:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:43:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:43:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:43:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:43:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:43:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:43:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:43:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:52 smithi170 ceph-mon[102832]: pgmap v2867: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 141 KiB/s wr, 16 op/s 2023-09-10T00:43:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5063.566620 secs 2023-09-10T00:43:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:52 smithi082 ceph-mon[98694]: pgmap v2867: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 141 KiB/s wr, 16 op/s 2023-09-10T00:43:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5063.566620 secs 2023-09-10T00:43:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:52 smithi098 ceph-mon[103124]: pgmap v2867: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 141 KiB/s wr, 16 op/s 2023-09-10T00:43:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5063.566620 secs 2023-09-10T00:43:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5065.101304 secs 2023-09-10T00:43:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5065.101304 secs 2023-09-10T00:43:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5065.101304 secs 2023-09-10T00:43:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5065.814252 secs 2023-09-10T00:43:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:54 smithi170 ceph-mon[102832]: pgmap v2868: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T00:43:55.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5065.814252 secs 2023-09-10T00:43:55.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:54 smithi082 ceph-mon[98694]: pgmap v2868: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T00:43:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5065.814252 secs 2023-09-10T00:43:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:54 smithi098 ceph-mon[103124]: pgmap v2868: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T00:43:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:56 smithi082 ceph-mon[98694]: pgmap v2869: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 131 KiB/s wr, 16 op/s 2023-09-10T00:43:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:56 smithi098 ceph-mon[103124]: pgmap v2869: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 131 KiB/s wr, 16 op/s 2023-09-10T00:43:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:56 smithi170 ceph-mon[102832]: pgmap v2869: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 131 KiB/s wr, 16 op/s 2023-09-10T00:43:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5068.566681 secs 2023-09-10T00:43:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5068.566681 secs 2023-09-10T00:43:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5068.566681 secs 2023-09-10T00:43:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:58 smithi082 ceph-mon[98694]: pgmap v2870: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T00:43:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5070.101361 secs 2023-09-10T00:43:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:58 smithi098 ceph-mon[103124]: pgmap v2870: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T00:43:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5070.101361 secs 2023-09-10T00:43:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:58 smithi170 ceph-mon[102832]: pgmap v2870: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T00:43:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5070.101361 secs 2023-09-10T00:44:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:43:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5070.814318 secs 2023-09-10T00:44:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:43:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5070.814318 secs 2023-09-10T00:44:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:43:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5070.814318 secs 2023-09-10T00:44:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:00 smithi082 ceph-mon[98694]: pgmap v2871: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T00:44:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:00 smithi098 ceph-mon[103124]: pgmap v2871: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T00:44:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:00 smithi170 ceph-mon[102832]: pgmap v2871: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T00:44:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:02 smithi082 ceph-mon[98694]: pgmap v2872: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 55 KiB/s wr, 16 op/s 2023-09-10T00:44:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5073.566777 secs 2023-09-10T00:44:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:02 smithi098 ceph-mon[103124]: pgmap v2872: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 55 KiB/s wr, 16 op/s 2023-09-10T00:44:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5073.566777 secs 2023-09-10T00:44:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:02 smithi170 ceph-mon[102832]: pgmap v2872: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 55 KiB/s wr, 16 op/s 2023-09-10T00:44:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5073.566777 secs 2023-09-10T00:44:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5075.101420 secs 2023-09-10T00:44:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5075.101420 secs 2023-09-10T00:44:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5075.101420 secs 2023-09-10T00:44:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5075.814383 secs 2023-09-10T00:44:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:04 smithi082 ceph-mon[98694]: pgmap v2873: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 24 KiB/s wr, 10 op/s 2023-09-10T00:44:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5075.814383 secs 2023-09-10T00:44:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:04 smithi098 ceph-mon[103124]: pgmap v2873: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 24 KiB/s wr, 10 op/s 2023-09-10T00:44:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5075.814383 secs 2023-09-10T00:44:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:04 smithi170 ceph-mon[102832]: pgmap v2873: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 24 KiB/s wr, 10 op/s 2023-09-10T00:44:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:06 smithi082 ceph-mon[98694]: pgmap v2874: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 96 KiB/s wr, 30 op/s 2023-09-10T00:44:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:06 smithi098 ceph-mon[103124]: pgmap v2874: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 96 KiB/s wr, 30 op/s 2023-09-10T00:44:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:06 smithi170 ceph-mon[102832]: pgmap v2874: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 96 KiB/s wr, 30 op/s 2023-09-10T00:44:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5078.566885 secs 2023-09-10T00:44:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:07 smithi098 ceph-mon[103124]: osdmap e251: 12 total, 12 up, 12 in 2023-09-10T00:44:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5078.566885 secs 2023-09-10T00:44:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:07 smithi170 ceph-mon[102832]: osdmap e251: 12 total, 12 up, 12 in 2023-09-10T00:44:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5078.566885 secs 2023-09-10T00:44:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:07 smithi082 ceph-mon[98694]: osdmap e251: 12 total, 12 up, 12 in 2023-09-10T00:44:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:08 smithi098 ceph-mon[103124]: pgmap v2876: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 93 KiB/s wr, 30 op/s 2023-09-10T00:44:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5080.101479 secs 2023-09-10T00:44:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:08 smithi170 ceph-mon[102832]: pgmap v2876: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 93 KiB/s wr, 30 op/s 2023-09-10T00:44:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5080.101479 secs 2023-09-10T00:44:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:08 smithi082 ceph-mon[98694]: pgmap v2876: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 93 KiB/s wr, 30 op/s 2023-09-10T00:44:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5080.101479 secs 2023-09-10T00:44:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5080.814449 secs 2023-09-10T00:44:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5080.814449 secs 2023-09-10T00:44:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5080.814449 secs 2023-09-10T00:44:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:10 smithi098 ceph-mon[103124]: pgmap v2877: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 93 KiB/s wr, 30 op/s 2023-09-10T00:44:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:10 smithi170 ceph-mon[102832]: pgmap v2877: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 93 KiB/s wr, 30 op/s 2023-09-10T00:44:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:10 smithi082 ceph-mon[98694]: pgmap v2877: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 93 KiB/s wr, 30 op/s 2023-09-10T00:44:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:12 smithi098 ceph-mon[103124]: pgmap v2878: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T00:44:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5083.566950 secs 2023-09-10T00:44:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:12 smithi170 ceph-mon[102832]: pgmap v2878: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T00:44:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5083.566950 secs 2023-09-10T00:44:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:12 smithi082 ceph-mon[98694]: pgmap v2878: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T00:44:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5083.566950 secs 2023-09-10T00:44:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5085.101564 secs 2023-09-10T00:44:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5085.101564 secs 2023-09-10T00:44:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5085.101564 secs 2023-09-10T00:44:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5085.814512 secs 2023-09-10T00:44:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:14 smithi098 ceph-mon[103124]: pgmap v2879: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T00:44:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:14 smithi098 ceph-mon[103124]: osdmap e252: 12 total, 12 up, 12 in 2023-09-10T00:44:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5085.814512 secs 2023-09-10T00:44:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:14 smithi170 ceph-mon[102832]: pgmap v2879: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T00:44:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:14 smithi170 ceph-mon[102832]: osdmap e252: 12 total, 12 up, 12 in 2023-09-10T00:44:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5085.814512 secs 2023-09-10T00:44:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:14 smithi082 ceph-mon[98694]: pgmap v2879: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T00:44:15.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:14 smithi082 ceph-mon[98694]: osdmap e252: 12 total, 12 up, 12 in 2023-09-10T00:44:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:16 smithi098 ceph-mon[103124]: pgmap v2881: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 48 KiB/s wr, 13 op/s 2023-09-10T00:44:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:16 smithi170 ceph-mon[102832]: pgmap v2881: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 48 KiB/s wr, 13 op/s 2023-09-10T00:44:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:16 smithi082 ceph-mon[98694]: pgmap v2881: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 48 KiB/s wr, 13 op/s 2023-09-10T00:44:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:44:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:44:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:44:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:19 smithi098 ceph-mon[103124]: pgmap v2882: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T00:44:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5090.101624 secs 2023-09-10T00:44:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:44:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:44:19.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:44:19.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:44:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:19 smithi170 ceph-mon[102832]: pgmap v2882: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T00:44:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5090.101624 secs 2023-09-10T00:44:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:44:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:44:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:44:19.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:44:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:19 smithi082 ceph-mon[98694]: pgmap v2882: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T00:44:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5090.101624 secs 2023-09-10T00:44:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:44:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:44:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:44:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:44:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5090.814579 secs 2023-09-10T00:44:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:44:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:44:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:44:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5090.814579 secs 2023-09-10T00:44:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:44:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:44:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:44:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5090.814579 secs 2023-09-10T00:44:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:44:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:44:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:44:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:21 smithi098 ceph-mon[103124]: pgmap v2883: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T00:44:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:21 smithi170 ceph-mon[102832]: pgmap v2883: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T00:44:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:21 smithi082 ceph-mon[98694]: pgmap v2883: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T00:44:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:23 smithi098 ceph-mon[103124]: pgmap v2884: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T00:44:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:23 smithi170 ceph-mon[102832]: pgmap v2884: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T00:44:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:23 smithi082 ceph-mon[98694]: pgmap v2884: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T00:44:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5095.101688 secs 2023-09-10T00:44:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5095.101688 secs 2023-09-10T00:44:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5095.101688 secs 2023-09-10T00:44:25.240 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5095.814641 secs 2023-09-10T00:44:25.240 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:25 smithi082 ceph-mon[98694]: pgmap v2885: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T00:44:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5095.814641 secs 2023-09-10T00:44:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:25 smithi098 ceph-mon[103124]: pgmap v2885: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T00:44:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5095.814641 secs 2023-09-10T00:44:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:25 smithi170 ceph-mon[102832]: pgmap v2885: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T00:44:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:27 smithi098 ceph-mon[103124]: pgmap v2886: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 112 KiB/s wr, 16 op/s 2023-09-10T00:44:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:27 smithi170 ceph-mon[102832]: pgmap v2886: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 112 KiB/s wr, 16 op/s 2023-09-10T00:44:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:27 smithi082 ceph-mon[98694]: pgmap v2886: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 112 KiB/s wr, 16 op/s 2023-09-10T00:44:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:29 smithi098 ceph-mon[103124]: pgmap v2887: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 68 KiB/s wr, 10 op/s 2023-09-10T00:44:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5100.101776 secs 2023-09-10T00:44:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:29 smithi170 ceph-mon[102832]: pgmap v2887: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 68 KiB/s wr, 10 op/s 2023-09-10T00:44:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5100.101776 secs 2023-09-10T00:44:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:29 smithi082 ceph-mon[98694]: pgmap v2887: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 68 KiB/s wr, 10 op/s 2023-09-10T00:44:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5100.101776 secs 2023-09-10T00:44:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5100.814704 secs 2023-09-10T00:44:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5100.814704 secs 2023-09-10T00:44:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5100.814704 secs 2023-09-10T00:44:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:31 smithi170 ceph-mon[102832]: pgmap v2888: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 68 KiB/s wr, 10 op/s 2023-09-10T00:44:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:31 smithi082 ceph-mon[98694]: pgmap v2888: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 68 KiB/s wr, 10 op/s 2023-09-10T00:44:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:31 smithi098 ceph-mon[103124]: pgmap v2888: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 68 KiB/s wr, 10 op/s 2023-09-10T00:44:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:33 smithi170 ceph-mon[102832]: pgmap v2889: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 163 KiB/s wr, 16 op/s 2023-09-10T00:44:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:33 smithi082 ceph-mon[98694]: pgmap v2889: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 163 KiB/s wr, 16 op/s 2023-09-10T00:44:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:33 smithi098 ceph-mon[103124]: pgmap v2889: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 163 KiB/s wr, 16 op/s 2023-09-10T00:44:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5105.101833 secs 2023-09-10T00:44:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5105.101833 secs 2023-09-10T00:44:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5105.101833 secs 2023-09-10T00:44:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5105.814771 secs 2023-09-10T00:44:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:35 smithi170 ceph-mon[102832]: pgmap v2890: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T00:44:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5105.814771 secs 2023-09-10T00:44:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:35 smithi082 ceph-mon[98694]: pgmap v2890: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T00:44:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5105.814771 secs 2023-09-10T00:44:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:35 smithi098 ceph-mon[103124]: pgmap v2890: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T00:44:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:37 smithi170 ceph-mon[102832]: pgmap v2891: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 173 KiB/s wr, 18 op/s 2023-09-10T00:44:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:37 smithi082 ceph-mon[98694]: pgmap v2891: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 173 KiB/s wr, 18 op/s 2023-09-10T00:44:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:37 smithi098 ceph-mon[103124]: pgmap v2891: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 173 KiB/s wr, 18 op/s 2023-09-10T00:44:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:39 smithi170 ceph-mon[102832]: pgmap v2892: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 136 KiB/s wr, 12 op/s 2023-09-10T00:44:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5110.101897 secs 2023-09-10T00:44:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:39 smithi082 ceph-mon[98694]: pgmap v2892: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 136 KiB/s wr, 12 op/s 2023-09-10T00:44:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5110.101897 secs 2023-09-10T00:44:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:39 smithi098 ceph-mon[103124]: pgmap v2892: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 136 KiB/s wr, 12 op/s 2023-09-10T00:44:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5110.101897 secs 2023-09-10T00:44:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5110.814837 secs 2023-09-10T00:44:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5110.814837 secs 2023-09-10T00:44:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5110.814837 secs 2023-09-10T00:44:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:41 smithi170 ceph-mon[102832]: pgmap v2893: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 136 KiB/s wr, 12 op/s 2023-09-10T00:44:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:41 smithi082 ceph-mon[98694]: pgmap v2893: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 136 KiB/s wr, 12 op/s 2023-09-10T00:44:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:41 smithi098 ceph-mon[103124]: pgmap v2893: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 136 KiB/s wr, 12 op/s 2023-09-10T00:44:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:43 smithi170 ceph-mon[102832]: pgmap v2894: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 224 KiB/s wr, 19 op/s 2023-09-10T00:44:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:43 smithi082 ceph-mon[98694]: pgmap v2894: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 224 KiB/s wr, 19 op/s 2023-09-10T00:44:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:43 smithi098 ceph-mon[103124]: pgmap v2894: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 224 KiB/s wr, 19 op/s 2023-09-10T00:44:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5115.101959 secs 2023-09-10T00:44:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5115.101959 secs 2023-09-10T00:44:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5115.101959 secs 2023-09-10T00:44:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5115.814904 secs 2023-09-10T00:44:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:45 smithi170 ceph-mon[102832]: pgmap v2895: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 129 KiB/s wr, 12 op/s 2023-09-10T00:44:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5115.814904 secs 2023-09-10T00:44:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:45 smithi082 ceph-mon[98694]: pgmap v2895: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 129 KiB/s wr, 12 op/s 2023-09-10T00:44:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5115.814904 secs 2023-09-10T00:44:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:45 smithi098 ceph-mon[103124]: pgmap v2895: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 129 KiB/s wr, 12 op/s 2023-09-10T00:44:47.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:47 smithi170 ceph-mon[102832]: pgmap v2896: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 197 KiB/s wr, 19 op/s 2023-09-10T00:44:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:47 smithi082 ceph-mon[98694]: pgmap v2896: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 197 KiB/s wr, 19 op/s 2023-09-10T00:44:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:47 smithi098 ceph-mon[103124]: pgmap v2896: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 197 KiB/s wr, 19 op/s 2023-09-10T00:44:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5118.567484 secs 2023-09-10T00:44:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5118.567484 secs 2023-09-10T00:44:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5118.567484 secs 2023-09-10T00:44:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:49 smithi170 ceph-mon[102832]: pgmap v2897: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T00:44:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5120.102017 secs 2023-09-10T00:44:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:49 smithi082 ceph-mon[98694]: pgmap v2897: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T00:44:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5120.102017 secs 2023-09-10T00:44:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:49 smithi098 ceph-mon[103124]: pgmap v2897: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T00:44:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5120.102017 secs 2023-09-10T00:44:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5120.814970 secs 2023-09-10T00:44:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5120.814970 secs 2023-09-10T00:44:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5120.814970 secs 2023-09-10T00:44:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:51 smithi170 ceph-mon[102832]: pgmap v2898: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T00:44:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:51 smithi082 ceph-mon[98694]: pgmap v2898: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T00:44:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:51 smithi098 ceph-mon[103124]: pgmap v2898: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T00:44:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:44:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:44:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:44:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:44:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:44:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:44:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:44:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:44:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:44:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:53 smithi170 ceph-mon[102832]: pgmap v2899: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 197 KiB/s wr, 18 op/s 2023-09-10T00:44:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5123.567547 secs 2023-09-10T00:44:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:53 smithi082 ceph-mon[98694]: pgmap v2899: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 197 KiB/s wr, 18 op/s 2023-09-10T00:44:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5123.567547 secs 2023-09-10T00:44:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:53 smithi098 ceph-mon[103124]: pgmap v2899: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 197 KiB/s wr, 18 op/s 2023-09-10T00:44:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5123.567547 secs 2023-09-10T00:44:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5125.102075 secs 2023-09-10T00:44:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5125.102075 secs 2023-09-10T00:44:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5125.102075 secs 2023-09-10T00:44:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5125.815035 secs 2023-09-10T00:44:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:55 smithi170 ceph-mon[102832]: pgmap v2900: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T00:44:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5125.815035 secs 2023-09-10T00:44:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:55 smithi082 ceph-mon[98694]: pgmap v2900: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T00:44:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5125.815035 secs 2023-09-10T00:44:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:55 smithi098 ceph-mon[103124]: pgmap v2900: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T00:44:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:57 smithi170 ceph-mon[102832]: pgmap v2901: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 133 KiB/s wr, 18 op/s 2023-09-10T00:44:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:57 smithi082 ceph-mon[98694]: pgmap v2901: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 133 KiB/s wr, 18 op/s 2023-09-10T00:44:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:57 smithi098 ceph-mon[103124]: pgmap v2901: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 133 KiB/s wr, 18 op/s 2023-09-10T00:44:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5128.567608 secs 2023-09-10T00:44:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5128.567608 secs 2023-09-10T00:44:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5128.567608 secs 2023-09-10T00:44:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:59 smithi170 ceph-mon[102832]: pgmap v2902: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T00:44:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:44:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5130.102132 secs 2023-09-10T00:44:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:59 smithi082 ceph-mon[98694]: pgmap v2902: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T00:44:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:44:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5130.102132 secs 2023-09-10T00:44:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:59 smithi098 ceph-mon[103124]: pgmap v2902: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T00:44:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:44:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5130.102132 secs 2023-09-10T00:45:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5130.815098 secs 2023-09-10T00:45:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5130.815098 secs 2023-09-10T00:45:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5130.815098 secs 2023-09-10T00:45:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:01 smithi170 ceph-mon[102832]: pgmap v2903: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T00:45:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:01 smithi082 ceph-mon[98694]: pgmap v2903: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T00:45:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:01 smithi098 ceph-mon[103124]: pgmap v2903: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T00:45:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:03 smithi170 ceph-mon[102832]: pgmap v2904: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 65 KiB/s wr, 16 op/s 2023-09-10T00:45:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5133.567723 secs 2023-09-10T00:45:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:03 smithi082 ceph-mon[98694]: pgmap v2904: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 65 KiB/s wr, 16 op/s 2023-09-10T00:45:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5133.567723 secs 2023-09-10T00:45:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:03 smithi098 ceph-mon[103124]: pgmap v2904: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 65 KiB/s wr, 16 op/s 2023-09-10T00:45:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5133.567723 secs 2023-09-10T00:45:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5135.102221 secs 2023-09-10T00:45:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5135.102221 secs 2023-09-10T00:45:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5135.102221 secs 2023-09-10T00:45:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5135.815305 secs 2023-09-10T00:45:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:05 smithi170 ceph-mon[102832]: pgmap v2905: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 24 KiB/s wr, 10 op/s 2023-09-10T00:45:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5135.815305 secs 2023-09-10T00:45:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:05 smithi082 ceph-mon[98694]: pgmap v2905: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 24 KiB/s wr, 10 op/s 2023-09-10T00:45:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5135.815305 secs 2023-09-10T00:45:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:05 smithi098 ceph-mon[103124]: pgmap v2905: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 24 KiB/s wr, 10 op/s 2023-09-10T00:45:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:07 smithi170 ceph-mon[102832]: pgmap v2906: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 120 KiB/s wr, 29 op/s 2023-09-10T00:45:07.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:07 smithi082 ceph-mon[98694]: pgmap v2906: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 120 KiB/s wr, 29 op/s 2023-09-10T00:45:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:07 smithi098 ceph-mon[103124]: pgmap v2906: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 120 KiB/s wr, 29 op/s 2023-09-10T00:45:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5138.567778 secs 2023-09-10T00:45:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:08 smithi170 ceph-mon[102832]: osdmap e253: 12 total, 12 up, 12 in 2023-09-10T00:45:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5138.567778 secs 2023-09-10T00:45:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:08 smithi082 ceph-mon[98694]: osdmap e253: 12 total, 12 up, 12 in 2023-09-10T00:45:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5138.567778 secs 2023-09-10T00:45:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:08 smithi098 ceph-mon[103124]: osdmap e253: 12 total, 12 up, 12 in 2023-09-10T00:45:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:09 smithi170 ceph-mon[102832]: pgmap v2908: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 116 KiB/s wr, 29 op/s 2023-09-10T00:45:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5140.102280 secs 2023-09-10T00:45:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:09 smithi082 ceph-mon[98694]: pgmap v2908: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 116 KiB/s wr, 29 op/s 2023-09-10T00:45:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5140.102280 secs 2023-09-10T00:45:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:09 smithi098 ceph-mon[103124]: pgmap v2908: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 116 KiB/s wr, 29 op/s 2023-09-10T00:45:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5140.102280 secs 2023-09-10T00:45:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5140.815370 secs 2023-09-10T00:45:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:10 smithi170 ceph-mon[102832]: pgmap v2909: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 116 KiB/s wr, 29 op/s 2023-09-10T00:45:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5140.815370 secs 2023-09-10T00:45:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:10 smithi082 ceph-mon[98694]: pgmap v2909: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 116 KiB/s wr, 29 op/s 2023-09-10T00:45:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5140.815370 secs 2023-09-10T00:45:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:10 smithi098 ceph-mon[103124]: pgmap v2909: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 116 KiB/s wr, 29 op/s 2023-09-10T00:45:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:12 smithi170 ceph-mon[102832]: pgmap v2910: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 116 KiB/s wr, 29 op/s 2023-09-10T00:45:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5143.567838 secs 2023-09-10T00:45:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:12 smithi082 ceph-mon[98694]: pgmap v2910: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 116 KiB/s wr, 29 op/s 2023-09-10T00:45:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5143.567838 secs 2023-09-10T00:45:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:12 smithi098 ceph-mon[103124]: pgmap v2910: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 116 KiB/s wr, 29 op/s 2023-09-10T00:45:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5143.567838 secs 2023-09-10T00:45:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5145.102365 secs 2023-09-10T00:45:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5145.102365 secs 2023-09-10T00:45:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5145.102365 secs 2023-09-10T00:45:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5145.815433 secs 2023-09-10T00:45:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:14 smithi170 ceph-mon[102832]: pgmap v2911: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 116 KiB/s wr, 29 op/s 2023-09-10T00:45:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:14 smithi170 ceph-mon[102832]: osdmap e254: 12 total, 12 up, 12 in 2023-09-10T00:45:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5145.815433 secs 2023-09-10T00:45:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:14 smithi082 ceph-mon[98694]: pgmap v2911: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 116 KiB/s wr, 29 op/s 2023-09-10T00:45:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:14 smithi082 ceph-mon[98694]: osdmap e254: 12 total, 12 up, 12 in 2023-09-10T00:45:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5145.815433 secs 2023-09-10T00:45:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:14 smithi098 ceph-mon[103124]: pgmap v2911: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 116 KiB/s wr, 29 op/s 2023-09-10T00:45:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:14 smithi098 ceph-mon[103124]: osdmap e254: 12 total, 12 up, 12 in 2023-09-10T00:45:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:16 smithi170 ceph-mon[102832]: pgmap v2913: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 38 KiB/s wr, 13 op/s 2023-09-10T00:45:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:16 smithi082 ceph-mon[98694]: pgmap v2913: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 38 KiB/s wr, 13 op/s 2023-09-10T00:45:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:16 smithi098 ceph-mon[103124]: pgmap v2913: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 38 KiB/s wr, 13 op/s 2023-09-10T00:45:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:18 smithi170 ceph-mon[102832]: pgmap v2914: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T00:45:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5150.102423 secs 2023-09-10T00:45:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:45:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:18 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:45:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:18 smithi082 ceph-mon[98694]: pgmap v2914: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T00:45:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5150.102423 secs 2023-09-10T00:45:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:45:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:18 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:45:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:18 smithi098 ceph-mon[103124]: pgmap v2914: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T00:45:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5150.102423 secs 2023-09-10T00:45:19.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:45:19.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:18 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:45:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5150.815491 secs 2023-09-10T00:45:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:19 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:45:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:19 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:45:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:19 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:45:20.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:45:20.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:45:20.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:45:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5150.815491 secs 2023-09-10T00:45:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:19 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:45:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:19 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:45:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:19 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:45:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:45:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:45:20.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:45:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5150.815491 secs 2023-09-10T00:45:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:19 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:45:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:19 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:45:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:19 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:45:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:45:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:45:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:45:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:20 smithi082 ceph-mon[98694]: pgmap v2915: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T00:45:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:20 smithi098 ceph-mon[103124]: pgmap v2915: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T00:45:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:20 smithi170 ceph-mon[102832]: pgmap v2915: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T00:45:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:22 smithi082 ceph-mon[98694]: pgmap v2916: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T00:45:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:22 smithi098 ceph-mon[103124]: pgmap v2916: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T00:45:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:22 smithi170 ceph-mon[102832]: pgmap v2916: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T00:45:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5155.102505 secs 2023-09-10T00:45:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5155.102505 secs 2023-09-10T00:45:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5155.102505 secs 2023-09-10T00:45:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5155.815555 secs 2023-09-10T00:45:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:24 smithi082 ceph-mon[98694]: pgmap v2917: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T00:45:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5155.815555 secs 2023-09-10T00:45:25.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:24 smithi098 ceph-mon[103124]: pgmap v2917: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T00:45:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5155.815555 secs 2023-09-10T00:45:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:24 smithi170 ceph-mon[102832]: pgmap v2917: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T00:45:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:26 smithi082 ceph-mon[98694]: pgmap v2918: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 147 KiB/s wr, 17 op/s 2023-09-10T00:45:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:26 smithi098 ceph-mon[103124]: pgmap v2918: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 147 KiB/s wr, 17 op/s 2023-09-10T00:45:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:26 smithi170 ceph-mon[102832]: pgmap v2918: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 147 KiB/s wr, 17 op/s 2023-09-10T00:45:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:28 smithi082 ceph-mon[98694]: pgmap v2919: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 108 KiB/s wr, 11 op/s 2023-09-10T00:45:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5160.102565 secs 2023-09-10T00:45:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:28 smithi098 ceph-mon[103124]: pgmap v2919: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 108 KiB/s wr, 11 op/s 2023-09-10T00:45:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5160.102565 secs 2023-09-10T00:45:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:28 smithi170 ceph-mon[102832]: pgmap v2919: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 108 KiB/s wr, 11 op/s 2023-09-10T00:45:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5160.102565 secs 2023-09-10T00:45:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5160.815619 secs 2023-09-10T00:45:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5160.815619 secs 2023-09-10T00:45:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5160.815619 secs 2023-09-10T00:45:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:30 smithi082 ceph-mon[98694]: pgmap v2920: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 108 KiB/s wr, 11 op/s 2023-09-10T00:45:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:30 smithi098 ceph-mon[103124]: pgmap v2920: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 108 KiB/s wr, 11 op/s 2023-09-10T00:45:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:30 smithi170 ceph-mon[102832]: pgmap v2920: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 108 KiB/s wr, 11 op/s 2023-09-10T00:45:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:32 smithi082 ceph-mon[98694]: pgmap v2921: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 242 KiB/s wr, 17 op/s 2023-09-10T00:45:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:32 smithi098 ceph-mon[103124]: pgmap v2921: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 242 KiB/s wr, 17 op/s 2023-09-10T00:45:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:32 smithi170 ceph-mon[102832]: pgmap v2921: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 242 KiB/s wr, 17 op/s 2023-09-10T00:45:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5165.102622 secs 2023-09-10T00:45:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5165.102622 secs 2023-09-10T00:45:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5165.102622 secs 2023-09-10T00:45:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5165.815683 secs 2023-09-10T00:45:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:34 smithi098 ceph-mon[103124]: pgmap v2922: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 170 KiB/s wr, 12 op/s 2023-09-10T00:45:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5165.815683 secs 2023-09-10T00:45:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:34 smithi170 ceph-mon[102832]: pgmap v2922: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 170 KiB/s wr, 12 op/s 2023-09-10T00:45:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5165.815683 secs 2023-09-10T00:45:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:34 smithi082 ceph-mon[98694]: pgmap v2922: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 170 KiB/s wr, 12 op/s 2023-09-10T00:45:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:36 smithi098 ceph-mon[103124]: pgmap v2923: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 202 KiB/s wr, 18 op/s 2023-09-10T00:45:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:36 smithi170 ceph-mon[102832]: pgmap v2923: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 202 KiB/s wr, 18 op/s 2023-09-10T00:45:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:36 smithi082 ceph-mon[98694]: pgmap v2923: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 202 KiB/s wr, 18 op/s 2023-09-10T00:45:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:38 smithi098 ceph-mon[103124]: pgmap v2924: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 166 KiB/s wr, 12 op/s 2023-09-10T00:45:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5170.102680 secs 2023-09-10T00:45:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:38 smithi170 ceph-mon[102832]: pgmap v2924: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 166 KiB/s wr, 12 op/s 2023-09-10T00:45:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5170.102680 secs 2023-09-10T00:45:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:38 smithi082 ceph-mon[98694]: pgmap v2924: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 166 KiB/s wr, 12 op/s 2023-09-10T00:45:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5170.102680 secs 2023-09-10T00:45:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5170.815709 secs 2023-09-10T00:45:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5170.815709 secs 2023-09-10T00:45:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5170.815709 secs 2023-09-10T00:45:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:40 smithi098 ceph-mon[103124]: pgmap v2925: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 166 KiB/s wr, 12 op/s 2023-09-10T00:45:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:40 smithi170 ceph-mon[102832]: pgmap v2925: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 166 KiB/s wr, 12 op/s 2023-09-10T00:45:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:40 smithi082 ceph-mon[98694]: pgmap v2925: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 166 KiB/s wr, 12 op/s 2023-09-10T00:45:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:42 smithi098 ceph-mon[103124]: pgmap v2926: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 187 KiB/s wr, 17 op/s 2023-09-10T00:45:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:42 smithi170 ceph-mon[102832]: pgmap v2926: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 187 KiB/s wr, 17 op/s 2023-09-10T00:45:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:42 smithi082 ceph-mon[98694]: pgmap v2926: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 187 KiB/s wr, 17 op/s 2023-09-10T00:45:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5175.102739 secs 2023-09-10T00:45:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5175.102739 secs 2023-09-10T00:45:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5175.102739 secs 2023-09-10T00:45:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5175.815777 secs 2023-09-10T00:45:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:44 smithi098 ceph-mon[103124]: pgmap v2927: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T00:45:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5175.815777 secs 2023-09-10T00:45:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:44 smithi170 ceph-mon[102832]: pgmap v2927: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T00:45:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5175.815777 secs 2023-09-10T00:45:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:44 smithi082 ceph-mon[98694]: pgmap v2927: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T00:45:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:46 smithi098 ceph-mon[103124]: pgmap v2928: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 163 KiB/s wr, 17 op/s 2023-09-10T00:45:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:46 smithi170 ceph-mon[102832]: pgmap v2928: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 163 KiB/s wr, 17 op/s 2023-09-10T00:45:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:46 smithi082 ceph-mon[98694]: pgmap v2928: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 163 KiB/s wr, 17 op/s 2023-09-10T00:45:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5178.568443 secs 2023-09-10T00:45:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5178.568443 secs 2023-09-10T00:45:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5178.568443 secs 2023-09-10T00:45:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:48 smithi098 ceph-mon[103124]: pgmap v2929: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 131 KiB/s wr, 11 op/s 2023-09-10T00:45:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5180.102811 secs 2023-09-10T00:45:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:48 smithi170 ceph-mon[102832]: pgmap v2929: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 131 KiB/s wr, 11 op/s 2023-09-10T00:45:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5180.102811 secs 2023-09-10T00:45:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:48 smithi082 ceph-mon[98694]: pgmap v2929: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 131 KiB/s wr, 11 op/s 2023-09-10T00:45:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5180.102811 secs 2023-09-10T00:45:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5180.815840 secs 2023-09-10T00:45:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5180.815840 secs 2023-09-10T00:45:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5180.815840 secs 2023-09-10T00:45:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:51 smithi098 ceph-mon[103124]: pgmap v2930: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 131 KiB/s wr, 11 op/s 2023-09-10T00:45:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:51 smithi170 ceph-mon[102832]: pgmap v2930: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 131 KiB/s wr, 11 op/s 2023-09-10T00:45:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:51 smithi082 ceph-mon[98694]: pgmap v2930: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 131 KiB/s wr, 11 op/s 2023-09-10T00:45:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:45:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:45:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:45:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:45:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:45:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:45:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:45:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:45:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:45:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:53 smithi170 ceph-mon[102832]: pgmap v2931: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 158 KiB/s wr, 16 op/s 2023-09-10T00:45:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5183.568522 secs 2023-09-10T00:45:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:53 smithi082 ceph-mon[98694]: pgmap v2931: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 158 KiB/s wr, 16 op/s 2023-09-10T00:45:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5183.568522 secs 2023-09-10T00:45:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:53 smithi098 ceph-mon[103124]: pgmap v2931: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 158 KiB/s wr, 16 op/s 2023-09-10T00:45:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5183.568522 secs 2023-09-10T00:45:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5185.102875 secs 2023-09-10T00:45:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5185.102875 secs 2023-09-10T00:45:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5185.102875 secs 2023-09-10T00:45:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5185.815903 secs 2023-09-10T00:45:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:55 smithi170 ceph-mon[102832]: pgmap v2932: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T00:45:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5185.815903 secs 2023-09-10T00:45:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:55 smithi082 ceph-mon[98694]: pgmap v2932: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T00:45:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5185.815903 secs 2023-09-10T00:45:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:55 smithi098 ceph-mon[103124]: pgmap v2932: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T00:45:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:57 smithi170 ceph-mon[102832]: pgmap v2933: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 161 KiB/s wr, 16 op/s 2023-09-10T00:45:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:57 smithi082 ceph-mon[98694]: pgmap v2933: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 161 KiB/s wr, 16 op/s 2023-09-10T00:45:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:57 smithi098 ceph-mon[103124]: pgmap v2933: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 161 KiB/s wr, 16 op/s 2023-09-10T00:45:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5188.568631 secs 2023-09-10T00:45:59.033 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5188.568631 secs 2023-09-10T00:45:59.034 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5188.568631 secs 2023-09-10T00:45:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:59 smithi170 ceph-mon[102832]: pgmap v2934: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:45:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:45:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5190.102933 secs 2023-09-10T00:45:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:59 smithi082 ceph-mon[98694]: pgmap v2934: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:45:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:45:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5190.102933 secs 2023-09-10T00:45:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:59 smithi098 ceph-mon[103124]: pgmap v2934: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:45:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:45:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5190.102933 secs 2023-09-10T00:46:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5190.815967 secs 2023-09-10T00:46:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5190.815967 secs 2023-09-10T00:46:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5190.815967 secs 2023-09-10T00:46:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:01 smithi170 ceph-mon[102832]: pgmap v2935: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:46:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:01 smithi082 ceph-mon[98694]: pgmap v2935: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:46:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:01 smithi098 ceph-mon[103124]: pgmap v2935: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T00:46:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:03 smithi170 ceph-mon[102832]: pgmap v2936: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 79 KiB/s wr, 15 op/s 2023-09-10T00:46:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5193.568746 secs 2023-09-10T00:46:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:03 smithi082 ceph-mon[98694]: pgmap v2936: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 79 KiB/s wr, 15 op/s 2023-09-10T00:46:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5193.568746 secs 2023-09-10T00:46:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:03 smithi098 ceph-mon[103124]: pgmap v2936: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 79 KiB/s wr, 15 op/s 2023-09-10T00:46:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5193.568746 secs 2023-09-10T00:46:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5195.102992 secs 2023-09-10T00:46:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5195.102992 secs 2023-09-10T00:46:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5195.102992 secs 2023-09-10T00:46:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5195.816031 secs 2023-09-10T00:46:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:05 smithi170 ceph-mon[102832]: pgmap v2937: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 52 KiB/s wr, 10 op/s 2023-09-10T00:46:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5195.816031 secs 2023-09-10T00:46:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:05 smithi082 ceph-mon[98694]: pgmap v2937: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 52 KiB/s wr, 10 op/s 2023-09-10T00:46:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5195.816031 secs 2023-09-10T00:46:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:05 smithi098 ceph-mon[103124]: pgmap v2937: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 52 KiB/s wr, 10 op/s 2023-09-10T00:46:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:07 smithi170 ceph-mon[102832]: pgmap v2938: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T00:46:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:07 smithi082 ceph-mon[98694]: pgmap v2938: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T00:46:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:07 smithi098 ceph-mon[103124]: pgmap v2938: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T00:46:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5198.568820 secs 2023-09-10T00:46:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:08 smithi170 ceph-mon[102832]: osdmap e255: 12 total, 12 up, 12 in 2023-09-10T00:46:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5198.568820 secs 2023-09-10T00:46:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:08 smithi082 ceph-mon[98694]: osdmap e255: 12 total, 12 up, 12 in 2023-09-10T00:46:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5198.568820 secs 2023-09-10T00:46:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:08 smithi098 ceph-mon[103124]: osdmap e255: 12 total, 12 up, 12 in 2023-09-10T00:46:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:09 smithi170 ceph-mon[102832]: pgmap v2940: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T00:46:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5200.103049 secs 2023-09-10T00:46:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:09 smithi082 ceph-mon[98694]: pgmap v2940: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T00:46:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5200.103049 secs 2023-09-10T00:46:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:09 smithi098 ceph-mon[103124]: pgmap v2940: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T00:46:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5200.103049 secs 2023-09-10T00:46:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5200.816095 secs 2023-09-10T00:46:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5200.816095 secs 2023-09-10T00:46:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5200.816095 secs 2023-09-10T00:46:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:11 smithi170 ceph-mon[102832]: pgmap v2941: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T00:46:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:11 smithi082 ceph-mon[98694]: pgmap v2941: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T00:46:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:11 smithi098 ceph-mon[103124]: pgmap v2941: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T00:46:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:13 smithi170 ceph-mon[102832]: pgmap v2942: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T00:46:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5203.568893 secs 2023-09-10T00:46:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:13 smithi082 ceph-mon[98694]: pgmap v2942: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T00:46:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5203.568893 secs 2023-09-10T00:46:13.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:13 smithi098 ceph-mon[103124]: pgmap v2942: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T00:46:13.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5203.568893 secs 2023-09-10T00:46:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5205.103107 secs 2023-09-10T00:46:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5205.103107 secs 2023-09-10T00:46:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5205.103107 secs 2023-09-10T00:46:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5205.816157 secs 2023-09-10T00:46:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:15 smithi170 ceph-mon[102832]: pgmap v2943: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T00:46:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:15 smithi170 ceph-mon[102832]: osdmap e256: 12 total, 12 up, 12 in 2023-09-10T00:46:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5205.816157 secs 2023-09-10T00:46:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:15 smithi082 ceph-mon[98694]: pgmap v2943: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T00:46:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:15 smithi082 ceph-mon[98694]: osdmap e256: 12 total, 12 up, 12 in 2023-09-10T00:46:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5205.816157 secs 2023-09-10T00:46:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:15 smithi098 ceph-mon[103124]: pgmap v2943: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T00:46:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:15 smithi098 ceph-mon[103124]: osdmap e256: 12 total, 12 up, 12 in 2023-09-10T00:46:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:17 smithi170 ceph-mon[102832]: pgmap v2945: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 48 KiB/s wr, 14 op/s 2023-09-10T00:46:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:17 smithi082 ceph-mon[98694]: pgmap v2945: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 48 KiB/s wr, 14 op/s 2023-09-10T00:46:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:17 smithi098 ceph-mon[103124]: pgmap v2945: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 48 KiB/s wr, 14 op/s 2023-09-10T00:46:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5208.568954 secs 2023-09-10T00:46:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5208.568954 secs 2023-09-10T00:46:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5208.568954 secs 2023-09-10T00:46:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:19 smithi170 ceph-mon[102832]: pgmap v2946: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T00:46:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5210.103165 secs 2023-09-10T00:46:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:46:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:19 smithi082 ceph-mon[98694]: pgmap v2946: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T00:46:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5210.103165 secs 2023-09-10T00:46:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:46:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:19 smithi098 ceph-mon[103124]: pgmap v2946: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T00:46:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5210.103165 secs 2023-09-10T00:46:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:46:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5210.816222 secs 2023-09-10T00:46:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5210.816222 secs 2023-09-10T00:46:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5210.816222 secs 2023-09-10T00:46:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:21 smithi170 ceph-mon[102832]: pgmap v2947: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T00:46:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:21 smithi082 ceph-mon[98694]: pgmap v2947: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T00:46:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:21 smithi098 ceph-mon[103124]: pgmap v2947: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T00:46:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:22 smithi170 ceph-mon[102832]: pgmap v2948: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 48 KiB/s wr, 12 op/s 2023-09-10T00:46:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:23.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:23.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:23.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:23.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:46:23.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:23.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:23.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:23.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:23.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:23.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:22 smithi082 ceph-mon[98694]: pgmap v2948: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 48 KiB/s wr, 12 op/s 2023-09-10T00:46:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:23.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:23.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:46:23.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:23.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:23.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:23.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:23.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:23.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:22 smithi098 ceph-mon[103124]: pgmap v2948: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 48 KiB/s wr, 12 op/s 2023-09-10T00:46:23.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:23.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:23.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:23.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:23.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:46:23.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:23.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:23.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:23.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:23.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:23.258 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:24 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T00:46:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:24 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T00:46:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5215.103258 secs 2023-09-10T00:46:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:24 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T00:46:24.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:24 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T00:46:24.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:46:24.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:46:24.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:46:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:24 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T00:46:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:24 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T00:46:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5215.103258 secs 2023-09-10T00:46:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:24.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:24.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:24.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:24.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:24 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T00:46:24.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:24 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T00:46:24.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:46:24.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:46:24.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:46:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:24 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T00:46:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:24 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T00:46:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5215.103258 secs 2023-09-10T00:46:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:24.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:24.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:24 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T00:46:24.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:24 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T00:46:24.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:46:24.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:46:24.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:46:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5215.816287 secs 2023-09-10T00:46:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:25 smithi170 ceph-mon[102832]: pgmap v2949: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 48 KiB/s wr, 12 op/s 2023-09-10T00:46:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5215.816287 secs 2023-09-10T00:46:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:25 smithi082 ceph-mon[98694]: pgmap v2949: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 48 KiB/s wr, 12 op/s 2023-09-10T00:46:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5215.816287 secs 2023-09-10T00:46:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:25 smithi098 ceph-mon[103124]: pgmap v2949: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 48 KiB/s wr, 12 op/s 2023-09-10T00:46:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:25.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:26.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:26.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:26.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:26.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:26 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T00:46:26.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:26 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T00:46:26.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:46:26.711 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:46:26.711 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:26.711 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:26 smithi082 ceph-mon[98694]: pgmap v2950: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 182 KiB/s wr, 18 op/s 2023-09-10T00:46:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:26.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:26.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:26.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:26.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:26 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T00:46:26.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:26 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T00:46:26.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:46:26.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:46:26.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:26.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:26 smithi098 ceph-mon[103124]: pgmap v2950: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 182 KiB/s wr, 18 op/s 2023-09-10T00:46:26.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:26.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:26.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:26.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:26.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:26.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:46:26.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:26 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T00:46:26.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:26 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T00:46:26.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:46:26.932 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:46:26.932 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:46:26.932 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:26 smithi170 ceph-mon[102832]: pgmap v2950: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 182 KiB/s wr, 18 op/s 2023-09-10T00:46:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:28 smithi170 ceph-mon[102832]: pgmap v2951: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T00:46:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5220.103318 secs 2023-09-10T00:46:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:28 smithi082 ceph-mon[98694]: pgmap v2951: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T00:46:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5220.103318 secs 2023-09-10T00:46:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:28 smithi098 ceph-mon[103124]: pgmap v2951: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T00:46:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5220.103318 secs 2023-09-10T00:46:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5220.816351 secs 2023-09-10T00:46:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5220.816351 secs 2023-09-10T00:46:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5220.816351 secs 2023-09-10T00:46:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:30 smithi170 ceph-mon[102832]: pgmap v2952: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T00:46:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:30 smithi082 ceph-mon[98694]: pgmap v2952: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T00:46:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:30 smithi098 ceph-mon[103124]: pgmap v2952: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T00:46:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:32 smithi170 ceph-mon[102832]: pgmap v2953: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 252 KiB/s wr, 17 op/s 2023-09-10T00:46:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:32 smithi082 ceph-mon[98694]: pgmap v2953: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 252 KiB/s wr, 17 op/s 2023-09-10T00:46:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:32 smithi098 ceph-mon[103124]: pgmap v2953: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 252 KiB/s wr, 17 op/s 2023-09-10T00:46:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5225.103400 secs 2023-09-10T00:46:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5225.103400 secs 2023-09-10T00:46:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5225.103400 secs 2023-09-10T00:46:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5225.816414 secs 2023-09-10T00:46:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:34 smithi170 ceph-mon[102832]: pgmap v2954: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 212 KiB/s wr, 12 op/s 2023-09-10T00:46:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5225.816414 secs 2023-09-10T00:46:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:34 smithi082 ceph-mon[98694]: pgmap v2954: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 212 KiB/s wr, 12 op/s 2023-09-10T00:46:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5225.816414 secs 2023-09-10T00:46:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:34 smithi098 ceph-mon[103124]: pgmap v2954: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 212 KiB/s wr, 12 op/s 2023-09-10T00:46:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:36 smithi170 ceph-mon[102832]: pgmap v2955: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 245 KiB/s wr, 18 op/s 2023-09-10T00:46:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:36 smithi082 ceph-mon[98694]: pgmap v2955: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 245 KiB/s wr, 18 op/s 2023-09-10T00:46:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:36 smithi098 ceph-mon[103124]: pgmap v2955: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 245 KiB/s wr, 18 op/s 2023-09-10T00:46:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:38 smithi170 ceph-mon[102832]: pgmap v2956: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T00:46:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5230.103461 secs 2023-09-10T00:46:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:38 smithi082 ceph-mon[98694]: pgmap v2956: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T00:46:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5230.103461 secs 2023-09-10T00:46:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:38 smithi098 ceph-mon[103124]: pgmap v2956: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T00:46:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5230.103461 secs 2023-09-10T00:46:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5230.816480 secs 2023-09-10T00:46:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5230.816480 secs 2023-09-10T00:46:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5230.816480 secs 2023-09-10T00:46:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:40 smithi170 ceph-mon[102832]: pgmap v2957: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T00:46:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:40 smithi082 ceph-mon[98694]: pgmap v2957: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T00:46:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:40 smithi098 ceph-mon[103124]: pgmap v2957: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T00:46:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:42 smithi170 ceph-mon[102832]: pgmap v2958: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 229 KiB/s wr, 17 op/s 2023-09-10T00:46:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5233.569436 secs 2023-09-10T00:46:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:42 smithi082 ceph-mon[98694]: pgmap v2958: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 229 KiB/s wr, 17 op/s 2023-09-10T00:46:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5233.569436 secs 2023-09-10T00:46:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:42 smithi098 ceph-mon[103124]: pgmap v2958: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 229 KiB/s wr, 17 op/s 2023-09-10T00:46:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5233.569436 secs 2023-09-10T00:46:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5235.103519 secs 2023-09-10T00:46:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:46:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:46:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:46:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5235.103519 secs 2023-09-10T00:46:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:46:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:46:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:46:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5235.103519 secs 2023-09-10T00:46:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:46:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:46:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:46:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5235.816546 secs 2023-09-10T00:46:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:44 smithi170 ceph-mon[102832]: pgmap v2959: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 143 KiB/s wr, 11 op/s 2023-09-10T00:46:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5235.816546 secs 2023-09-10T00:46:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:44 smithi082 ceph-mon[98694]: pgmap v2959: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 143 KiB/s wr, 11 op/s 2023-09-10T00:46:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5235.816546 secs 2023-09-10T00:46:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:44 smithi098 ceph-mon[103124]: pgmap v2959: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 143 KiB/s wr, 11 op/s 2023-09-10T00:46:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:46 smithi082 ceph-mon[98694]: pgmap v2960: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 195 KiB/s wr, 17 op/s 2023-09-10T00:46:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:46 smithi098 ceph-mon[103124]: pgmap v2960: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 195 KiB/s wr, 17 op/s 2023-09-10T00:46:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:46 smithi170 ceph-mon[102832]: pgmap v2960: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 195 KiB/s wr, 17 op/s 2023-09-10T00:46:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5238.569498 secs 2023-09-10T00:46:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5238.569498 secs 2023-09-10T00:46:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5238.569498 secs 2023-09-10T00:46:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:48 smithi082 ceph-mon[98694]: pgmap v2961: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T00:46:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5240.103603 secs 2023-09-10T00:46:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:48 smithi098 ceph-mon[103124]: pgmap v2961: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T00:46:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5240.103603 secs 2023-09-10T00:46:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:48 smithi170 ceph-mon[102832]: pgmap v2961: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T00:46:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5240.103603 secs 2023-09-10T00:46:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5240.816612 secs 2023-09-10T00:46:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5240.816612 secs 2023-09-10T00:46:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5240.816612 secs 2023-09-10T00:46:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:50 smithi082 ceph-mon[98694]: pgmap v2962: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T00:46:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:50 smithi098 ceph-mon[103124]: pgmap v2962: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T00:46:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:50 smithi170 ceph-mon[102832]: pgmap v2962: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T00:46:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:52 smithi082 ceph-mon[98694]: pgmap v2963: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 194 KiB/s wr, 16 op/s 2023-09-10T00:46:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5243.569610 secs 2023-09-10T00:46:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:52 smithi098 ceph-mon[103124]: pgmap v2963: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 194 KiB/s wr, 16 op/s 2023-09-10T00:46:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5243.569610 secs 2023-09-10T00:46:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:52 smithi170 ceph-mon[102832]: pgmap v2963: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 194 KiB/s wr, 16 op/s 2023-09-10T00:46:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5243.569610 secs 2023-09-10T00:46:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5245.103661 secs 2023-09-10T00:46:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5245.103661 secs 2023-09-10T00:46:54.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5245.103661 secs 2023-09-10T00:46:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5245.816677 secs 2023-09-10T00:46:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:54 smithi082 ceph-mon[98694]: pgmap v2964: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-09-10T00:46:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5245.816677 secs 2023-09-10T00:46:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:54 smithi098 ceph-mon[103124]: pgmap v2964: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-09-10T00:46:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5245.816677 secs 2023-09-10T00:46:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:54 smithi170 ceph-mon[102832]: pgmap v2964: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-09-10T00:46:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:56 smithi082 ceph-mon[98694]: pgmap v2965: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 107 KiB/s wr, 15 op/s 2023-09-10T00:46:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:56 smithi098 ceph-mon[103124]: pgmap v2965: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 107 KiB/s wr, 15 op/s 2023-09-10T00:46:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:56 smithi170 ceph-mon[102832]: pgmap v2965: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 107 KiB/s wr, 15 op/s 2023-09-10T00:46:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5248.569667 secs 2023-09-10T00:46:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5248.569667 secs 2023-09-10T00:46:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5248.569667 secs 2023-09-10T00:46:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:58 smithi098 ceph-mon[103124]: pgmap v2966: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T00:46:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5250.103721 secs 2023-09-10T00:46:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:58 smithi170 ceph-mon[102832]: pgmap v2966: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T00:46:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5250.103721 secs 2023-09-10T00:46:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:58 smithi082 ceph-mon[98694]: pgmap v2966: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T00:46:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5250.103721 secs 2023-09-10T00:47:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:46:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5250.816745 secs 2023-09-10T00:47:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:46:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5250.816745 secs 2023-09-10T00:47:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:46:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5250.816745 secs 2023-09-10T00:47:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:00 smithi098 ceph-mon[103124]: pgmap v2967: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T00:47:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:00 smithi170 ceph-mon[102832]: pgmap v2967: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T00:47:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:00 smithi082 ceph-mon[98694]: pgmap v2967: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T00:47:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:02 smithi098 ceph-mon[103124]: pgmap v2968: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 56 KiB/s wr, 15 op/s 2023-09-10T00:47:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5253.569730 secs 2023-09-10T00:47:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:02 smithi170 ceph-mon[102832]: pgmap v2968: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 56 KiB/s wr, 15 op/s 2023-09-10T00:47:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5253.569730 secs 2023-09-10T00:47:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:02 smithi082 ceph-mon[98694]: pgmap v2968: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 56 KiB/s wr, 15 op/s 2023-09-10T00:47:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5253.569730 secs 2023-09-10T00:47:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5255.103781 secs 2023-09-10T00:47:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5255.103781 secs 2023-09-10T00:47:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5255.103781 secs 2023-09-10T00:47:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5255.816809 secs 2023-09-10T00:47:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:04 smithi098 ceph-mon[103124]: pgmap v2969: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 KiB/s wr, 10 op/s 2023-09-10T00:47:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5255.816809 secs 2023-09-10T00:47:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:04 smithi170 ceph-mon[102832]: pgmap v2969: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 KiB/s wr, 10 op/s 2023-09-10T00:47:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5255.816809 secs 2023-09-10T00:47:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:04 smithi082 ceph-mon[98694]: pgmap v2969: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 KiB/s wr, 10 op/s 2023-09-10T00:47:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:06 smithi098 ceph-mon[103124]: pgmap v2970: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T00:47:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:06 smithi170 ceph-mon[102832]: pgmap v2970: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T00:47:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:06 smithi082 ceph-mon[98694]: pgmap v2970: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T00:47:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5258.569775 secs 2023-09-10T00:47:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:07 smithi098 ceph-mon[103124]: osdmap e257: 12 total, 12 up, 12 in 2023-09-10T00:47:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5258.569775 secs 2023-09-10T00:47:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:07 smithi170 ceph-mon[102832]: osdmap e257: 12 total, 12 up, 12 in 2023-09-10T00:47:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5258.569775 secs 2023-09-10T00:47:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:07 smithi082 ceph-mon[98694]: osdmap e257: 12 total, 12 up, 12 in 2023-09-10T00:47:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5260.103839 secs 2023-09-10T00:47:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:08 smithi098 ceph-mon[103124]: pgmap v2972: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 89 KiB/s wr, 29 op/s 2023-09-10T00:47:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5260.103839 secs 2023-09-10T00:47:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:08 smithi170 ceph-mon[102832]: pgmap v2972: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 89 KiB/s wr, 29 op/s 2023-09-10T00:47:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5260.103839 secs 2023-09-10T00:47:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:08 smithi082 ceph-mon[98694]: pgmap v2972: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 89 KiB/s wr, 29 op/s 2023-09-10T00:47:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5260.816877 secs 2023-09-10T00:47:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5260.816877 secs 2023-09-10T00:47:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5260.816877 secs 2023-09-10T00:47:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:11 smithi098 ceph-mon[103124]: pgmap v2973: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 89 KiB/s wr, 29 op/s 2023-09-10T00:47:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:11 smithi170 ceph-mon[102832]: pgmap v2973: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 89 KiB/s wr, 29 op/s 2023-09-10T00:47:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:11 smithi082 ceph-mon[98694]: pgmap v2973: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 89 KiB/s wr, 29 op/s 2023-09-10T00:47:13.260 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:13 smithi098 ceph-mon[103124]: pgmap v2974: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 89 KiB/s wr, 29 op/s 2023-09-10T00:47:13.260 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5263.569875 secs 2023-09-10T00:47:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:13 smithi170 ceph-mon[102832]: pgmap v2974: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 89 KiB/s wr, 29 op/s 2023-09-10T00:47:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5263.569875 secs 2023-09-10T00:47:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:13 smithi082 ceph-mon[98694]: pgmap v2974: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 89 KiB/s wr, 29 op/s 2023-09-10T00:47:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5263.569875 secs 2023-09-10T00:47:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5265.103898 secs 2023-09-10T00:47:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5265.103898 secs 2023-09-10T00:47:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5265.103898 secs 2023-09-10T00:47:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5265.816941 secs 2023-09-10T00:47:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:15 smithi170 ceph-mon[102832]: pgmap v2975: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 89 KiB/s wr, 29 op/s 2023-09-10T00:47:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:15 smithi170 ceph-mon[102832]: osdmap e258: 12 total, 12 up, 12 in 2023-09-10T00:47:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5265.816941 secs 2023-09-10T00:47:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:15 smithi082 ceph-mon[98694]: pgmap v2975: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 89 KiB/s wr, 29 op/s 2023-09-10T00:47:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:15 smithi082 ceph-mon[98694]: osdmap e258: 12 total, 12 up, 12 in 2023-09-10T00:47:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5265.816941 secs 2023-09-10T00:47:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:15 smithi098 ceph-mon[103124]: pgmap v2975: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 89 KiB/s wr, 29 op/s 2023-09-10T00:47:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:15 smithi098 ceph-mon[103124]: osdmap e258: 12 total, 12 up, 12 in 2023-09-10T00:47:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:17 smithi170 ceph-mon[102832]: pgmap v2977: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 49 KiB/s wr, 12 op/s 2023-09-10T00:47:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:17 smithi082 ceph-mon[98694]: pgmap v2977: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 49 KiB/s wr, 12 op/s 2023-09-10T00:47:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:17 smithi098 ceph-mon[103124]: pgmap v2977: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 49 KiB/s wr, 12 op/s 2023-09-10T00:47:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5270.103983 secs 2023-09-10T00:47:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:19 smithi170 ceph-mon[102832]: pgmap v2978: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T00:47:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:47:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5270.103983 secs 2023-09-10T00:47:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:19 smithi082 ceph-mon[98694]: pgmap v2978: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T00:47:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:47:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5270.103983 secs 2023-09-10T00:47:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:19 smithi098 ceph-mon[103124]: pgmap v2978: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T00:47:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:47:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5270.817003 secs 2023-09-10T00:47:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:47:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:47:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:47:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5270.817003 secs 2023-09-10T00:47:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:47:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:47:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:47:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5270.817003 secs 2023-09-10T00:47:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:47:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:47:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:47:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:21 smithi170 ceph-mon[102832]: pgmap v2979: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T00:47:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:21 smithi082 ceph-mon[98694]: pgmap v2979: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T00:47:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:21 smithi098 ceph-mon[103124]: pgmap v2979: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T00:47:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:23 smithi170 ceph-mon[102832]: pgmap v2980: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T00:47:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5273.570042 secs 2023-09-10T00:47:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:23 smithi082 ceph-mon[98694]: pgmap v2980: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T00:47:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5273.570042 secs 2023-09-10T00:47:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:23 smithi098 ceph-mon[103124]: pgmap v2980: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T00:47:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5273.570042 secs 2023-09-10T00:47:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5275.104040 secs 2023-09-10T00:47:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:47:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:47:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5275.104040 secs 2023-09-10T00:47:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:47:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:47:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5275.104040 secs 2023-09-10T00:47:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:47:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:47:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5275.817068 secs 2023-09-10T00:47:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:25 smithi170 ceph-mon[102832]: pgmap v2981: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T00:47:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:25 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:47:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5275.817068 secs 2023-09-10T00:47:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:25 smithi082 ceph-mon[98694]: pgmap v2981: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T00:47:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:25 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:47:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5275.817068 secs 2023-09-10T00:47:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:25 smithi098 ceph-mon[103124]: pgmap v2981: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T00:47:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:25 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:47:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:47:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:47:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:47:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:27 smithi170 ceph-mon[102832]: pgmap v2982: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 115 KiB/s wr, 16 op/s 2023-09-10T00:47:27.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:47:27.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:47:27.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:47:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:27 smithi082 ceph-mon[98694]: pgmap v2982: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 115 KiB/s wr, 16 op/s 2023-09-10T00:47:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:47:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:47:27.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:47:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:27 smithi098 ceph-mon[103124]: pgmap v2982: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 115 KiB/s wr, 16 op/s 2023-09-10T00:47:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:47:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:47:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:47:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5278.570105 secs 2023-09-10T00:47:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5278.570105 secs 2023-09-10T00:47:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5278.570105 secs 2023-09-10T00:47:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5280.104099 secs 2023-09-10T00:47:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:29 smithi170 ceph-mon[102832]: pgmap v2983: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T00:47:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5280.104099 secs 2023-09-10T00:47:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:29 smithi082 ceph-mon[98694]: pgmap v2983: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T00:47:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5280.104099 secs 2023-09-10T00:47:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:29 smithi098 ceph-mon[103124]: pgmap v2983: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T00:47:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5280.817131 secs 2023-09-10T00:47:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5280.817131 secs 2023-09-10T00:47:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5280.817131 secs 2023-09-10T00:47:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:31 smithi170 ceph-mon[102832]: pgmap v2984: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T00:47:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:31 smithi082 ceph-mon[98694]: pgmap v2984: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T00:47:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:31 smithi098 ceph-mon[103124]: pgmap v2984: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T00:47:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:33 smithi170 ceph-mon[102832]: pgmap v2985: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 144 KiB/s wr, 16 op/s 2023-09-10T00:47:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:33 smithi082 ceph-mon[98694]: pgmap v2985: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 144 KiB/s wr, 16 op/s 2023-09-10T00:47:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:33 smithi098 ceph-mon[103124]: pgmap v2985: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 144 KiB/s wr, 16 op/s 2023-09-10T00:47:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5285.104157 secs 2023-09-10T00:47:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5285.104157 secs 2023-09-10T00:47:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5285.104157 secs 2023-09-10T00:47:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5285.817198 secs 2023-09-10T00:47:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:35 smithi170 ceph-mon[102832]: pgmap v2986: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T00:47:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:47:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5285.817198 secs 2023-09-10T00:47:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:35 smithi082 ceph-mon[98694]: pgmap v2986: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T00:47:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:47:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5285.817198 secs 2023-09-10T00:47:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:35 smithi098 ceph-mon[103124]: pgmap v2986: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T00:47:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:47:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:47:36.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:47:36.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:47:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:47:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:47:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:47:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:47:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:47:36.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:47:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:37 smithi170 ceph-mon[102832]: pgmap v2987: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 176 KiB/s wr, 18 op/s 2023-09-10T00:47:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:37 smithi082 ceph-mon[98694]: pgmap v2987: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 176 KiB/s wr, 18 op/s 2023-09-10T00:47:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:37 smithi098 ceph-mon[103124]: pgmap v2987: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 176 KiB/s wr, 18 op/s 2023-09-10T00:47:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5290.104243 secs 2023-09-10T00:47:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:39 smithi170 ceph-mon[102832]: pgmap v2988: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 12 op/s 2023-09-10T00:47:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5290.104243 secs 2023-09-10T00:47:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:39 smithi082 ceph-mon[98694]: pgmap v2988: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 12 op/s 2023-09-10T00:47:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5290.104243 secs 2023-09-10T00:47:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:39 smithi098 ceph-mon[103124]: pgmap v2988: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 12 op/s 2023-09-10T00:47:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5290.817262 secs 2023-09-10T00:47:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5290.817262 secs 2023-09-10T00:47:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5290.817262 secs 2023-09-10T00:47:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:41 smithi170 ceph-mon[102832]: pgmap v2989: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 12 op/s 2023-09-10T00:47:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:41 smithi082 ceph-mon[98694]: pgmap v2989: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 12 op/s 2023-09-10T00:47:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:41 smithi098 ceph-mon[103124]: pgmap v2989: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 12 op/s 2023-09-10T00:47:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:43 smithi170 ceph-mon[102832]: pgmap v2990: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 212 KiB/s wr, 18 op/s 2023-09-10T00:47:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:43 smithi082 ceph-mon[98694]: pgmap v2990: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 212 KiB/s wr, 18 op/s 2023-09-10T00:47:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:43 smithi098 ceph-mon[103124]: pgmap v2990: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 212 KiB/s wr, 18 op/s 2023-09-10T00:47:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5295.104300 secs 2023-09-10T00:47:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5295.104300 secs 2023-09-10T00:47:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5295.104300 secs 2023-09-10T00:47:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5295.817327 secs 2023-09-10T00:47:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:45 smithi170 ceph-mon[102832]: pgmap v2991: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T00:47:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5295.817327 secs 2023-09-10T00:47:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:45 smithi082 ceph-mon[98694]: pgmap v2991: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T00:47:45.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5295.817327 secs 2023-09-10T00:47:45.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:45 smithi098 ceph-mon[103124]: pgmap v2991: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T00:47:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:47 smithi170 ceph-mon[102832]: pgmap v2992: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 242 KiB/s wr, 18 op/s 2023-09-10T00:47:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:47 smithi082 ceph-mon[98694]: pgmap v2992: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 242 KiB/s wr, 18 op/s 2023-09-10T00:47:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:47 smithi098 ceph-mon[103124]: pgmap v2992: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 242 KiB/s wr, 18 op/s 2023-09-10T00:47:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5298.570445 secs 2023-09-10T00:47:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5298.570445 secs 2023-09-10T00:47:48.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5298.570445 secs 2023-09-10T00:47:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5300.104357 secs 2023-09-10T00:47:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:49 smithi170 ceph-mon[102832]: pgmap v2993: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T00:47:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5300.104357 secs 2023-09-10T00:47:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:49 smithi082 ceph-mon[98694]: pgmap v2993: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T00:47:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5300.104357 secs 2023-09-10T00:47:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:49 smithi098 ceph-mon[103124]: pgmap v2993: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T00:47:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5300.817391 secs 2023-09-10T00:47:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5300.817391 secs 2023-09-10T00:47:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5300.817391 secs 2023-09-10T00:47:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:51 smithi170 ceph-mon[102832]: pgmap v2994: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T00:47:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:51 smithi082 ceph-mon[98694]: pgmap v2994: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T00:47:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:51 smithi098 ceph-mon[103124]: pgmap v2994: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T00:47:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:47:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:47:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:47:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:47:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:47:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:47:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:47:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:47:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:47:53.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:53 smithi170 ceph-mon[102832]: pgmap v2995: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 206 KiB/s wr, 16 op/s 2023-09-10T00:47:53.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5303.570505 secs 2023-09-10T00:47:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:53 smithi082 ceph-mon[98694]: pgmap v2995: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 206 KiB/s wr, 16 op/s 2023-09-10T00:47:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5303.570505 secs 2023-09-10T00:47:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:53 smithi098 ceph-mon[103124]: pgmap v2995: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 206 KiB/s wr, 16 op/s 2023-09-10T00:47:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5303.570505 secs 2023-09-10T00:47:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5305.104441 secs 2023-09-10T00:47:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5305.104441 secs 2023-09-10T00:47:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5305.104441 secs 2023-09-10T00:47:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5305.817457 secs 2023-09-10T00:47:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:55 smithi170 ceph-mon[102832]: pgmap v2996: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 134 KiB/s wr, 10 op/s 2023-09-10T00:47:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5305.817457 secs 2023-09-10T00:47:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:55 smithi082 ceph-mon[98694]: pgmap v2996: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 134 KiB/s wr, 10 op/s 2023-09-10T00:47:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5305.817457 secs 2023-09-10T00:47:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:55 smithi098 ceph-mon[103124]: pgmap v2996: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 134 KiB/s wr, 10 op/s 2023-09-10T00:47:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:57 smithi170 ceph-mon[102832]: pgmap v2997: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 199 KiB/s wr, 16 op/s 2023-09-10T00:47:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:57 smithi082 ceph-mon[98694]: pgmap v2997: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 199 KiB/s wr, 16 op/s 2023-09-10T00:47:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:57 smithi098 ceph-mon[103124]: pgmap v2997: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 199 KiB/s wr, 16 op/s 2023-09-10T00:47:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5308.570565 secs 2023-09-10T00:47:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5308.570565 secs 2023-09-10T00:47:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5308.570565 secs 2023-09-10T00:47:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5310.104531 secs 2023-09-10T00:47:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:47:59 smithi170 ceph-mon[102832]: pgmap v2998: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 94 KiB/s wr, 10 op/s 2023-09-10T00:47:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5310.104531 secs 2023-09-10T00:47:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:47:59 smithi082 ceph-mon[98694]: pgmap v2998: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 94 KiB/s wr, 10 op/s 2023-09-10T00:47:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5310.104531 secs 2023-09-10T00:47:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:47:59 smithi098 ceph-mon[103124]: pgmap v2998: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 94 KiB/s wr, 10 op/s 2023-09-10T00:48:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5310.817484 secs 2023-09-10T00:48:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5310.817484 secs 2023-09-10T00:48:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5310.817484 secs 2023-09-10T00:48:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:01 smithi170 ceph-mon[102832]: pgmap v2999: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 94 KiB/s wr, 10 op/s 2023-09-10T00:48:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:01 smithi082 ceph-mon[98694]: pgmap v2999: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 94 KiB/s wr, 10 op/s 2023-09-10T00:48:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:01 smithi098 ceph-mon[103124]: pgmap v2999: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 94 KiB/s wr, 10 op/s 2023-09-10T00:48:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:03 smithi170 ceph-mon[102832]: pgmap v3000: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 139 KiB/s wr, 29 op/s 2023-09-10T00:48:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5313.570642 secs 2023-09-10T00:48:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:03 smithi082 ceph-mon[98694]: pgmap v3000: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 139 KiB/s wr, 29 op/s 2023-09-10T00:48:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5313.570642 secs 2023-09-10T00:48:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:03 smithi098 ceph-mon[103124]: pgmap v3000: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 139 KiB/s wr, 29 op/s 2023-09-10T00:48:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5313.570642 secs 2023-09-10T00:48:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5315.104591 secs 2023-09-10T00:48:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5315.104591 secs 2023-09-10T00:48:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5315.104591 secs 2023-09-10T00:48:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5315.817552 secs 2023-09-10T00:48:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:05 smithi082 ceph-mon[98694]: pgmap v3001: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 110 KiB/s wr, 24 op/s 2023-09-10T00:48:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5315.817552 secs 2023-09-10T00:48:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:05 smithi098 ceph-mon[103124]: pgmap v3001: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 110 KiB/s wr, 24 op/s 2023-09-10T00:48:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5315.817552 secs 2023-09-10T00:48:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:05 smithi170 ceph-mon[102832]: pgmap v3001: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 110 KiB/s wr, 24 op/s 2023-09-10T00:48:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:07 smithi082 ceph-mon[98694]: pgmap v3002: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 158 KiB/s wr, 30 op/s 2023-09-10T00:48:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:07 smithi098 ceph-mon[103124]: pgmap v3002: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 158 KiB/s wr, 30 op/s 2023-09-10T00:48:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:07 smithi170 ceph-mon[102832]: pgmap v3002: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 158 KiB/s wr, 30 op/s 2023-09-10T00:48:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5318.570771 secs 2023-09-10T00:48:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:08 smithi082 ceph-mon[98694]: osdmap e259: 12 total, 12 up, 12 in 2023-09-10T00:48:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5318.570771 secs 2023-09-10T00:48:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:08 smithi098 ceph-mon[103124]: osdmap e259: 12 total, 12 up, 12 in 2023-09-10T00:48:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5318.570771 secs 2023-09-10T00:48:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:08 smithi170 ceph-mon[102832]: osdmap e259: 12 total, 12 up, 12 in 2023-09-10T00:48:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5320.104657 secs 2023-09-10T00:48:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:09 smithi082 ceph-mon[98694]: pgmap v3004: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 112 KiB/s wr, 29 op/s 2023-09-10T00:48:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5320.104657 secs 2023-09-10T00:48:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:09 smithi098 ceph-mon[103124]: pgmap v3004: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 112 KiB/s wr, 29 op/s 2023-09-10T00:48:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5320.104657 secs 2023-09-10T00:48:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:09 smithi170 ceph-mon[102832]: pgmap v3004: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 112 KiB/s wr, 29 op/s 2023-09-10T00:48:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5320.817618 secs 2023-09-10T00:48:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5320.817618 secs 2023-09-10T00:48:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5320.817618 secs 2023-09-10T00:48:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:11 smithi082 ceph-mon[98694]: pgmap v3005: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 112 KiB/s wr, 29 op/s 2023-09-10T00:48:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:11 smithi098 ceph-mon[103124]: pgmap v3005: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 112 KiB/s wr, 29 op/s 2023-09-10T00:48:11.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:11 smithi170 ceph-mon[102832]: pgmap v3005: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 112 KiB/s wr, 29 op/s 2023-09-10T00:48:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:13 smithi098 ceph-mon[103124]: pgmap v3006: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T00:48:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5323.570907 secs 2023-09-10T00:48:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:13 smithi170 ceph-mon[102832]: pgmap v3006: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T00:48:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5323.570907 secs 2023-09-10T00:48:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:13 smithi082 ceph-mon[98694]: pgmap v3006: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T00:48:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5323.570907 secs 2023-09-10T00:48:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5325.104713 secs 2023-09-10T00:48:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5325.104713 secs 2023-09-10T00:48:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5325.104713 secs 2023-09-10T00:48:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5325.817689 secs 2023-09-10T00:48:15.689 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:15 smithi098 ceph-mon[103124]: pgmap v3007: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T00:48:15.690 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:15 smithi098 ceph-mon[103124]: osdmap e260: 12 total, 12 up, 12 in 2023-09-10T00:48:15.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5325.817689 secs 2023-09-10T00:48:15.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:15 smithi170 ceph-mon[102832]: pgmap v3007: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T00:48:15.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:15 smithi170 ceph-mon[102832]: osdmap e260: 12 total, 12 up, 12 in 2023-09-10T00:48:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5325.817689 secs 2023-09-10T00:48:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:15 smithi082 ceph-mon[98694]: pgmap v3007: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T00:48:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:15 smithi082 ceph-mon[98694]: osdmap e260: 12 total, 12 up, 12 in 2023-09-10T00:48:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:17 smithi098 ceph-mon[103124]: pgmap v3009: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 14 op/s 2023-09-10T00:48:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:17 smithi170 ceph-mon[102832]: pgmap v3009: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 14 op/s 2023-09-10T00:48:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:17 smithi082 ceph-mon[98694]: pgmap v3009: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 14 op/s 2023-09-10T00:48:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5330.104770 secs 2023-09-10T00:48:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:19 smithi098 ceph-mon[103124]: pgmap v3010: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T00:48:19.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:48:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5330.104770 secs 2023-09-10T00:48:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:19 smithi170 ceph-mon[102832]: pgmap v3010: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T00:48:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:48:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5330.104770 secs 2023-09-10T00:48:19.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:19 smithi082 ceph-mon[98694]: pgmap v3010: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T00:48:19.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:48:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5330.817755 secs 2023-09-10T00:48:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:48:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:48:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:48:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5330.817755 secs 2023-09-10T00:48:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:48:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:48:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:48:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5330.817755 secs 2023-09-10T00:48:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:48:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:48:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:48:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:21 smithi098 ceph-mon[103124]: pgmap v3011: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T00:48:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:21 smithi170 ceph-mon[102832]: pgmap v3011: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T00:48:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:21 smithi082 ceph-mon[98694]: pgmap v3011: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T00:48:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:23 smithi098 ceph-mon[103124]: pgmap v3012: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T00:48:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:23 smithi170 ceph-mon[102832]: pgmap v3012: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T00:48:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:23 smithi082 ceph-mon[98694]: pgmap v3012: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T00:48:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5335.104828 secs 2023-09-10T00:48:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5335.104828 secs 2023-09-10T00:48:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5335.104828 secs 2023-09-10T00:48:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5335.817822 secs 2023-09-10T00:48:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:25 smithi098 ceph-mon[103124]: pgmap v3013: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T00:48:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5335.817822 secs 2023-09-10T00:48:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:25 smithi170 ceph-mon[102832]: pgmap v3013: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T00:48:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5335.817822 secs 2023-09-10T00:48:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:25 smithi082 ceph-mon[98694]: pgmap v3013: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T00:48:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:27 smithi098 ceph-mon[103124]: pgmap v3014: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 152 KiB/s wr, 18 op/s 2023-09-10T00:48:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:48:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:48:27.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:48:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:27 smithi170 ceph-mon[102832]: pgmap v3014: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 152 KiB/s wr, 18 op/s 2023-09-10T00:48:27.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:48:27.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:48:27.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:48:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:27 smithi082 ceph-mon[98694]: pgmap v3014: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 152 KiB/s wr, 18 op/s 2023-09-10T00:48:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:48:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:48:27.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:48:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:48:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:48:28.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:48:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:48:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:48:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:48:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:48:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:48:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:48:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5340.104887 secs 2023-09-10T00:48:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:29 smithi170 ceph-mon[102832]: pgmap v3015: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T00:48:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5340.104887 secs 2023-09-10T00:48:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:29 smithi082 ceph-mon[98694]: pgmap v3015: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T00:48:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5340.104887 secs 2023-09-10T00:48:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:29 smithi098 ceph-mon[103124]: pgmap v3015: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T00:48:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5340.817888 secs 2023-09-10T00:48:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5340.817888 secs 2023-09-10T00:48:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5340.817888 secs 2023-09-10T00:48:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:31 smithi170 ceph-mon[102832]: pgmap v3016: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T00:48:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:31 smithi082 ceph-mon[98694]: pgmap v3016: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T00:48:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:31 smithi098 ceph-mon[103124]: pgmap v3016: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T00:48:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:33 smithi170 ceph-mon[102832]: pgmap v3017: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 250 KiB/s wr, 17 op/s 2023-09-10T00:48:33.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5343.571145 secs 2023-09-10T00:48:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:33 smithi082 ceph-mon[98694]: pgmap v3017: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 250 KiB/s wr, 17 op/s 2023-09-10T00:48:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5343.571145 secs 2023-09-10T00:48:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:33 smithi098 ceph-mon[103124]: pgmap v3017: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 250 KiB/s wr, 17 op/s 2023-09-10T00:48:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5343.571145 secs 2023-09-10T00:48:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5345.104970 secs 2023-09-10T00:48:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5345.104970 secs 2023-09-10T00:48:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5345.104970 secs 2023-09-10T00:48:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5345.817950 secs 2023-09-10T00:48:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:35 smithi170 ceph-mon[102832]: pgmap v3018: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 175 KiB/s wr, 11 op/s 2023-09-10T00:48:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5345.817950 secs 2023-09-10T00:48:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:35 smithi082 ceph-mon[98694]: pgmap v3018: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 175 KiB/s wr, 11 op/s 2023-09-10T00:48:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5345.817950 secs 2023-09-10T00:48:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:35 smithi098 ceph-mon[103124]: pgmap v3018: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 175 KiB/s wr, 11 op/s 2023-09-10T00:48:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:36 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:48:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:48:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:48:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:36 smithi170 ceph-mon[102832]: pgmap v3019: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 209 KiB/s wr, 17 op/s 2023-09-10T00:48:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:36 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:48:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:48:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:48:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:36 smithi082 ceph-mon[98694]: pgmap v3019: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 209 KiB/s wr, 17 op/s 2023-09-10T00:48:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:36 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:48:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:48:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:48:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:36 smithi098 ceph-mon[103124]: pgmap v3019: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 209 KiB/s wr, 17 op/s 2023-09-10T00:48:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5348.571225 secs 2023-09-10T00:48:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5348.571225 secs 2023-09-10T00:48:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5348.571225 secs 2023-09-10T00:48:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5350.105056 secs 2023-09-10T00:48:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:38 smithi098 ceph-mon[103124]: pgmap v3020: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 171 KiB/s wr, 11 op/s 2023-09-10T00:48:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5350.105056 secs 2023-09-10T00:48:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:38 smithi170 ceph-mon[102832]: pgmap v3020: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 171 KiB/s wr, 11 op/s 2023-09-10T00:48:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5350.105056 secs 2023-09-10T00:48:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:38 smithi082 ceph-mon[98694]: pgmap v3020: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 171 KiB/s wr, 11 op/s 2023-09-10T00:48:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5350.818017 secs 2023-09-10T00:48:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5350.818017 secs 2023-09-10T00:48:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5350.818017 secs 2023-09-10T00:48:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:40 smithi170 ceph-mon[102832]: pgmap v3021: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 171 KiB/s wr, 11 op/s 2023-09-10T00:48:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:40 smithi082 ceph-mon[98694]: pgmap v3021: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 171 KiB/s wr, 11 op/s 2023-09-10T00:48:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:40 smithi098 ceph-mon[103124]: pgmap v3021: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 171 KiB/s wr, 11 op/s 2023-09-10T00:48:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:42 smithi170 ceph-mon[102832]: pgmap v3022: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 216 KiB/s wr, 16 op/s 2023-09-10T00:48:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5353.571323 secs 2023-09-10T00:48:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:42 smithi082 ceph-mon[98694]: pgmap v3022: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 216 KiB/s wr, 16 op/s 2023-09-10T00:48:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5353.571323 secs 2023-09-10T00:48:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:42 smithi098 ceph-mon[103124]: pgmap v3022: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 216 KiB/s wr, 16 op/s 2023-09-10T00:48:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5353.571323 secs 2023-09-10T00:48:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5355.105113 secs 2023-09-10T00:48:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5355.105113 secs 2023-09-10T00:48:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5355.105113 secs 2023-09-10T00:48:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5355.818080 secs 2023-09-10T00:48:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:44 smithi170 ceph-mon[102832]: pgmap v3023: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T00:48:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5355.818080 secs 2023-09-10T00:48:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:44 smithi082 ceph-mon[98694]: pgmap v3023: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T00:48:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5355.818080 secs 2023-09-10T00:48:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:44 smithi098 ceph-mon[103124]: pgmap v3023: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T00:48:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:46 smithi170 ceph-mon[102832]: pgmap v3024: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 145 KiB/s wr, 17 op/s 2023-09-10T00:48:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:46 smithi082 ceph-mon[98694]: pgmap v3024: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 145 KiB/s wr, 17 op/s 2023-09-10T00:48:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:46 smithi098 ceph-mon[103124]: pgmap v3024: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 145 KiB/s wr, 17 op/s 2023-09-10T00:48:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5358.571452 secs 2023-09-10T00:48:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5358.571452 secs 2023-09-10T00:48:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5358.571452 secs 2023-09-10T00:48:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5360.105171 secs 2023-09-10T00:48:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:48 smithi170 ceph-mon[102832]: pgmap v3025: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T00:48:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5360.105171 secs 2023-09-10T00:48:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:48 smithi082 ceph-mon[98694]: pgmap v3025: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T00:48:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5360.105171 secs 2023-09-10T00:48:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:48 smithi098 ceph-mon[103124]: pgmap v3025: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T00:48:50.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5360.818146 secs 2023-09-10T00:48:50.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5360.818146 secs 2023-09-10T00:48:50.262 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5360.818146 secs 2023-09-10T00:48:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:50 smithi170 ceph-mon[102832]: pgmap v3026: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T00:48:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:50 smithi082 ceph-mon[98694]: pgmap v3026: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T00:48:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:50 smithi098 ceph-mon[103124]: pgmap v3026: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T00:48:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5363.571544 secs 2023-09-10T00:48:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:52 smithi170 ceph-mon[102832]: pgmap v3027: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 140 KiB/s wr, 16 op/s 2023-09-10T00:48:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5363.571544 secs 2023-09-10T00:48:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:52 smithi082 ceph-mon[98694]: pgmap v3027: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 140 KiB/s wr, 16 op/s 2023-09-10T00:48:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5363.571544 secs 2023-09-10T00:48:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:52 smithi098 ceph-mon[103124]: pgmap v3027: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 140 KiB/s wr, 16 op/s 2023-09-10T00:48:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5365.105228 secs 2023-09-10T00:48:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5365.105228 secs 2023-09-10T00:48:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5365.105228 secs 2023-09-10T00:48:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5365.818212 secs 2023-09-10T00:48:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:54 smithi082 ceph-mon[98694]: pgmap v3028: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 10 op/s 2023-09-10T00:48:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5365.818212 secs 2023-09-10T00:48:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:54 smithi098 ceph-mon[103124]: pgmap v3028: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 10 op/s 2023-09-10T00:48:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5365.818212 secs 2023-09-10T00:48:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:54 smithi170 ceph-mon[102832]: pgmap v3028: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 10 op/s 2023-09-10T00:48:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:56 smithi082 ceph-mon[98694]: pgmap v3029: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 15 op/s 2023-09-10T00:48:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:56 smithi098 ceph-mon[103124]: pgmap v3029: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 15 op/s 2023-09-10T00:48:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:56 smithi170 ceph-mon[102832]: pgmap v3029: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 15 op/s 2023-09-10T00:48:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5368.571672 secs 2023-09-10T00:48:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5368.571672 secs 2023-09-10T00:48:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5368.571672 secs 2023-09-10T00:48:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5370.105300 secs 2023-09-10T00:48:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:58 smithi082 ceph-mon[98694]: pgmap v3030: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-10T00:48:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5370.105300 secs 2023-09-10T00:48:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:58 smithi098 ceph-mon[103124]: pgmap v3030: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-10T00:48:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5370.105300 secs 2023-09-10T00:48:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:58 smithi170 ceph-mon[102832]: pgmap v3030: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-10T00:49:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:48:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5370.818277 secs 2023-09-10T00:49:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:48:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5370.818277 secs 2023-09-10T00:49:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:48:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5370.818277 secs 2023-09-10T00:49:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:00 smithi082 ceph-mon[98694]: pgmap v3031: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-10T00:49:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:00 smithi098 ceph-mon[103124]: pgmap v3031: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-10T00:49:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:00 smithi170 ceph-mon[102832]: pgmap v3031: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-10T00:49:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5373.571751 secs 2023-09-10T00:49:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:02 smithi082 ceph-mon[98694]: pgmap v3032: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T00:49:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:02 smithi082 ceph-mon[98694]: osdmap e261: 12 total, 12 up, 12 in 2023-09-10T00:49:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5373.571751 secs 2023-09-10T00:49:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:02 smithi098 ceph-mon[103124]: pgmap v3032: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T00:49:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:02 smithi098 ceph-mon[103124]: osdmap e261: 12 total, 12 up, 12 in 2023-09-10T00:49:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5373.571751 secs 2023-09-10T00:49:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:02 smithi170 ceph-mon[102832]: pgmap v3032: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T00:49:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:02 smithi170 ceph-mon[102832]: osdmap e261: 12 total, 12 up, 12 in 2023-09-10T00:49:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5375.105357 secs 2023-09-10T00:49:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5375.105357 secs 2023-09-10T00:49:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5375.105357 secs 2023-09-10T00:49:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5375.818342 secs 2023-09-10T00:49:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:04 smithi098 ceph-mon[103124]: pgmap v3034: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 118 KiB/s wr, 29 op/s 2023-09-10T00:49:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5375.818342 secs 2023-09-10T00:49:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:04 smithi170 ceph-mon[102832]: pgmap v3034: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 118 KiB/s wr, 29 op/s 2023-09-10T00:49:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5375.818342 secs 2023-09-10T00:49:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:04 smithi082 ceph-mon[98694]: pgmap v3034: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 118 KiB/s wr, 29 op/s 2023-09-10T00:49:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:06 smithi098 ceph-mon[103124]: pgmap v3035: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T00:49:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:06 smithi170 ceph-mon[102832]: pgmap v3035: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T00:49:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:06 smithi082 ceph-mon[98694]: pgmap v3035: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T00:49:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5378.571793 secs 2023-09-10T00:49:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5378.571793 secs 2023-09-10T00:49:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5378.571793 secs 2023-09-10T00:49:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5380.105415 secs 2023-09-10T00:49:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:08 smithi098 ceph-mon[103124]: pgmap v3036: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T00:49:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5380.105415 secs 2023-09-10T00:49:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:08 smithi170 ceph-mon[102832]: pgmap v3036: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T00:49:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5380.105415 secs 2023-09-10T00:49:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:08 smithi082 ceph-mon[98694]: pgmap v3036: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T00:49:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5380.818404 secs 2023-09-10T00:49:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:09 smithi098 ceph-mon[103124]: osdmap e262: 12 total, 12 up, 12 in 2023-09-10T00:49:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5380.818404 secs 2023-09-10T00:49:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:09 smithi170 ceph-mon[102832]: osdmap e262: 12 total, 12 up, 12 in 2023-09-10T00:49:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5380.818404 secs 2023-09-10T00:49:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:09 smithi082 ceph-mon[98694]: osdmap e262: 12 total, 12 up, 12 in 2023-09-10T00:49:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:10 smithi098 ceph-mon[103124]: pgmap v3037: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T00:49:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:10 smithi170 ceph-mon[102832]: pgmap v3037: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T00:49:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:10 smithi082 ceph-mon[98694]: pgmap v3037: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T00:49:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5383.571877 secs 2023-09-10T00:49:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:12 smithi098 ceph-mon[103124]: pgmap v3039: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T00:49:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5383.571877 secs 2023-09-10T00:49:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:12 smithi170 ceph-mon[102832]: pgmap v3039: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T00:49:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5383.571877 secs 2023-09-10T00:49:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:12 smithi082 ceph-mon[98694]: pgmap v3039: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T00:49:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5385.105475 secs 2023-09-10T00:49:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5385.105475 secs 2023-09-10T00:49:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5385.105475 secs 2023-09-10T00:49:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5385.818474 secs 2023-09-10T00:49:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:14 smithi098 ceph-mon[103124]: pgmap v3040: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T00:49:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5385.818474 secs 2023-09-10T00:49:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:14 smithi170 ceph-mon[102832]: pgmap v3040: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T00:49:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5385.818474 secs 2023-09-10T00:49:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:14 smithi082 ceph-mon[98694]: pgmap v3040: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T00:49:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:16 smithi098 ceph-mon[103124]: pgmap v3041: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T00:49:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:16 smithi170 ceph-mon[102832]: pgmap v3041: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T00:49:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:16 smithi082 ceph-mon[98694]: pgmap v3041: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T00:49:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5390.105532 secs 2023-09-10T00:49:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:18 smithi098 ceph-mon[103124]: pgmap v3042: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T00:49:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:18 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:49:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5390.105532 secs 2023-09-10T00:49:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:18 smithi170 ceph-mon[102832]: pgmap v3042: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T00:49:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:18 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:49:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5390.105532 secs 2023-09-10T00:49:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:18 smithi082 ceph-mon[98694]: pgmap v3042: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T00:49:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:18 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:49:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5390.818490 secs 2023-09-10T00:49:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:19 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:49:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:19 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:49:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:19 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:49:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5390.818490 secs 2023-09-10T00:49:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:19 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:49:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:19 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:49:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:19 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:49:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5390.818490 secs 2023-09-10T00:49:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:19 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:49:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:19 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:49:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:19 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:49:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:20 smithi098 ceph-mon[103124]: pgmap v3043: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T00:49:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:20 smithi170 ceph-mon[102832]: pgmap v3043: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T00:49:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:20 smithi082 ceph-mon[98694]: pgmap v3043: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T00:49:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:23 smithi098 ceph-mon[103124]: pgmap v3044: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 78 KiB/s wr, 14 op/s 2023-09-10T00:49:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:23 smithi170 ceph-mon[102832]: pgmap v3044: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 78 KiB/s wr, 14 op/s 2023-09-10T00:49:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:23 smithi082 ceph-mon[98694]: pgmap v3044: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 78 KiB/s wr, 14 op/s 2023-09-10T00:49:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5395.105591 secs 2023-09-10T00:49:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5395.105591 secs 2023-09-10T00:49:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5395.105591 secs 2023-09-10T00:49:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5395.818554 secs 2023-09-10T00:49:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:25 smithi170 ceph-mon[102832]: pgmap v3045: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 42 KiB/s wr, 9 op/s 2023-09-10T00:49:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5395.818554 secs 2023-09-10T00:49:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:25 smithi082 ceph-mon[98694]: pgmap v3045: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 42 KiB/s wr, 9 op/s 2023-09-10T00:49:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5395.818554 secs 2023-09-10T00:49:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:25 smithi098 ceph-mon[103124]: pgmap v3045: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 42 KiB/s wr, 9 op/s 2023-09-10T00:49:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:27 smithi170 ceph-mon[102832]: pgmap v3046: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 166 KiB/s wr, 15 op/s 2023-09-10T00:49:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:27 smithi082 ceph-mon[98694]: pgmap v3046: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 166 KiB/s wr, 15 op/s 2023-09-10T00:49:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:27 smithi098 ceph-mon[103124]: pgmap v3046: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 166 KiB/s wr, 15 op/s 2023-09-10T00:49:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:49:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:49:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:49:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:49:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:49:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:49:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:49:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:49:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:49:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:49:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:49:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:49:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:49:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:49:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:49:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5400.105648 secs 2023-09-10T00:49:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:29 smithi170 ceph-mon[102832]: pgmap v3047: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 166 KiB/s wr, 11 op/s 2023-09-10T00:49:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:49:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:49:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:49:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5400.105648 secs 2023-09-10T00:49:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:29 smithi082 ceph-mon[98694]: pgmap v3047: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 166 KiB/s wr, 11 op/s 2023-09-10T00:49:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:49:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:49:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:49:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5400.105648 secs 2023-09-10T00:49:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:29 smithi098 ceph-mon[103124]: pgmap v3047: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 166 KiB/s wr, 11 op/s 2023-09-10T00:49:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:49:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:49:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:49:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5400.818619 secs 2023-09-10T00:49:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5400.818619 secs 2023-09-10T00:49:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5400.818619 secs 2023-09-10T00:49:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:31 smithi170 ceph-mon[102832]: pgmap v3048: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 166 KiB/s wr, 11 op/s 2023-09-10T00:49:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:31 smithi082 ceph-mon[98694]: pgmap v3048: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 166 KiB/s wr, 11 op/s 2023-09-10T00:49:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:31 smithi098 ceph-mon[103124]: pgmap v3048: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 166 KiB/s wr, 11 op/s 2023-09-10T00:49:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:33 smithi170 ceph-mon[102832]: pgmap v3049: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 259 KiB/s wr, 17 op/s 2023-09-10T00:49:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:33 smithi082 ceph-mon[98694]: pgmap v3049: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 259 KiB/s wr, 17 op/s 2023-09-10T00:49:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:33 smithi098 ceph-mon[103124]: pgmap v3049: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 259 KiB/s wr, 17 op/s 2023-09-10T00:49:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5405.105708 secs 2023-09-10T00:49:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5405.105708 secs 2023-09-10T00:49:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5405.105708 secs 2023-09-10T00:49:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5405.818683 secs 2023-09-10T00:49:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:35 smithi170 ceph-mon[102832]: pgmap v3050: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 218 KiB/s wr, 12 op/s 2023-09-10T00:49:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5405.818683 secs 2023-09-10T00:49:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:35 smithi082 ceph-mon[98694]: pgmap v3050: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 218 KiB/s wr, 12 op/s 2023-09-10T00:49:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5405.818683 secs 2023-09-10T00:49:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:35 smithi098 ceph-mon[103124]: pgmap v3050: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 218 KiB/s wr, 12 op/s 2023-09-10T00:49:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:37 smithi170 ceph-mon[102832]: pgmap v3051: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 252 KiB/s wr, 18 op/s 2023-09-10T00:49:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:37 smithi082 ceph-mon[98694]: pgmap v3051: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 252 KiB/s wr, 18 op/s 2023-09-10T00:49:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:37 smithi098 ceph-mon[103124]: pgmap v3051: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 252 KiB/s wr, 18 op/s 2023-09-10T00:49:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5410.105769 secs 2023-09-10T00:49:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:39 smithi170 ceph-mon[102832]: pgmap v3052: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T00:49:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5410.105769 secs 2023-09-10T00:49:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:39 smithi082 ceph-mon[98694]: pgmap v3052: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T00:49:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5410.105769 secs 2023-09-10T00:49:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:39 smithi098 ceph-mon[103124]: pgmap v3052: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T00:49:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5410.818746 secs 2023-09-10T00:49:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5410.818746 secs 2023-09-10T00:49:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5410.818746 secs 2023-09-10T00:49:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:41 smithi170 ceph-mon[102832]: pgmap v3053: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T00:49:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:41 smithi082 ceph-mon[98694]: pgmap v3053: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T00:49:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:41 smithi098 ceph-mon[103124]: pgmap v3053: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T00:49:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5413.572355 secs 2023-09-10T00:49:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:43 smithi170 ceph-mon[102832]: pgmap v3054: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 241 KiB/s wr, 18 op/s 2023-09-10T00:49:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5413.572355 secs 2023-09-10T00:49:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:43 smithi082 ceph-mon[98694]: pgmap v3054: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 241 KiB/s wr, 18 op/s 2023-09-10T00:49:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5413.572355 secs 2023-09-10T00:49:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:43 smithi098 ceph-mon[103124]: pgmap v3054: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 241 KiB/s wr, 18 op/s 2023-09-10T00:49:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5415.105849 secs 2023-09-10T00:49:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:49:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:49:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:49:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5415.105849 secs 2023-09-10T00:49:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:49:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:49:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:49:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5415.105849 secs 2023-09-10T00:49:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:49:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:49:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:49:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5415.818814 secs 2023-09-10T00:49:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:45 smithi170 ceph-mon[102832]: pgmap v3055: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T00:49:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5415.818814 secs 2023-09-10T00:49:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:45 smithi082 ceph-mon[98694]: pgmap v3055: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T00:49:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5415.818814 secs 2023-09-10T00:49:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:45 smithi098 ceph-mon[103124]: pgmap v3055: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T00:49:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:47 smithi170 ceph-mon[102832]: pgmap v3056: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 201 KiB/s wr, 16 op/s 2023-09-10T00:49:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:47 smithi082 ceph-mon[98694]: pgmap v3056: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 201 KiB/s wr, 16 op/s 2023-09-10T00:49:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:47 smithi098 ceph-mon[103124]: pgmap v3056: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 201 KiB/s wr, 16 op/s 2023-09-10T00:49:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5418.572430 secs 2023-09-10T00:49:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5418.572430 secs 2023-09-10T00:49:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5418.572430 secs 2023-09-10T00:49:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5420.105898 secs 2023-09-10T00:49:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:49 smithi170 ceph-mon[102832]: pgmap v3057: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T00:49:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5420.105898 secs 2023-09-10T00:49:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:49 smithi082 ceph-mon[98694]: pgmap v3057: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T00:49:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5420.105898 secs 2023-09-10T00:49:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:49 smithi098 ceph-mon[103124]: pgmap v3057: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T00:49:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5420.818876 secs 2023-09-10T00:49:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5420.818876 secs 2023-09-10T00:49:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5420.818876 secs 2023-09-10T00:49:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:51 smithi170 ceph-mon[102832]: pgmap v3058: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T00:49:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:51 smithi082 ceph-mon[98694]: pgmap v3058: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T00:49:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:51 smithi098 ceph-mon[103124]: pgmap v3058: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T00:49:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5423.572492 secs 2023-09-10T00:49:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:53 smithi170 ceph-mon[102832]: pgmap v3059: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 200 KiB/s wr, 16 op/s 2023-09-10T00:49:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5423.572492 secs 2023-09-10T00:49:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:53 smithi082 ceph-mon[98694]: pgmap v3059: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 200 KiB/s wr, 16 op/s 2023-09-10T00:49:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5423.572492 secs 2023-09-10T00:49:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:53 smithi098 ceph-mon[103124]: pgmap v3059: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 200 KiB/s wr, 16 op/s 2023-09-10T00:49:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5425.105955 secs 2023-09-10T00:49:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5425.105955 secs 2023-09-10T00:49:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5425.105955 secs 2023-09-10T00:49:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5425.818938 secs 2023-09-10T00:49:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:55 smithi170 ceph-mon[102832]: pgmap v3060: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 86 KiB/s wr, 10 op/s 2023-09-10T00:49:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5425.818938 secs 2023-09-10T00:49:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:55 smithi082 ceph-mon[98694]: pgmap v3060: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 86 KiB/s wr, 10 op/s 2023-09-10T00:49:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5425.818938 secs 2023-09-10T00:49:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:55 smithi098 ceph-mon[103124]: pgmap v3060: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 86 KiB/s wr, 10 op/s 2023-09-10T00:49:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:57 smithi170 ceph-mon[102832]: pgmap v3061: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 110 KiB/s wr, 15 op/s 2023-09-10T00:49:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:57 smithi082 ceph-mon[98694]: pgmap v3061: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 110 KiB/s wr, 15 op/s 2023-09-10T00:49:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:57 smithi098 ceph-mon[103124]: pgmap v3061: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 110 KiB/s wr, 15 op/s 2023-09-10T00:49:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5428.572553 secs 2023-09-10T00:49:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5428.572553 secs 2023-09-10T00:49:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5428.572553 secs 2023-09-10T00:49:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5430.106012 secs 2023-09-10T00:49:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:49:59 smithi170 ceph-mon[102832]: pgmap v3062: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 57 KiB/s wr, 10 op/s 2023-09-10T00:49:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5430.106012 secs 2023-09-10T00:49:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:49:59 smithi082 ceph-mon[98694]: pgmap v3062: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 57 KiB/s wr, 10 op/s 2023-09-10T00:49:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5430.106012 secs 2023-09-10T00:49:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:49:59 smithi098 ceph-mon[103124]: pgmap v3062: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 57 KiB/s wr, 10 op/s 2023-09-10T00:50:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5430.819008 secs 2023-09-10T00:50:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T00:50:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5430.819008 secs 2023-09-10T00:50:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T00:50:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5430.819008 secs 2023-09-10T00:50:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T00:50:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:01 smithi170 ceph-mon[102832]: pgmap v3063: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 57 KiB/s wr, 10 op/s 2023-09-10T00:50:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:01 smithi082 ceph-mon[98694]: pgmap v3063: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 57 KiB/s wr, 10 op/s 2023-09-10T00:50:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:01 smithi098 ceph-mon[103124]: pgmap v3063: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 57 KiB/s wr, 10 op/s 2023-09-10T00:50:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5433.572674 secs 2023-09-10T00:50:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:03 smithi170 ceph-mon[102832]: pgmap v3064: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 92 KiB/s wr, 28 op/s 2023-09-10T00:50:03.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5433.572674 secs 2023-09-10T00:50:03.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:03 smithi082 ceph-mon[98694]: pgmap v3064: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 92 KiB/s wr, 28 op/s 2023-09-10T00:50:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5433.572674 secs 2023-09-10T00:50:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:03 smithi098 ceph-mon[103124]: pgmap v3064: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 92 KiB/s wr, 28 op/s 2023-09-10T00:50:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5435.106072 secs 2023-09-10T00:50:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5435.106072 secs 2023-09-10T00:50:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5435.106072 secs 2023-09-10T00:50:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5435.819075 secs 2023-09-10T00:50:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:05 smithi170 ceph-mon[102832]: pgmap v3065: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 59 KiB/s wr, 23 op/s 2023-09-10T00:50:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5435.819075 secs 2023-09-10T00:50:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:05 smithi082 ceph-mon[98694]: pgmap v3065: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 59 KiB/s wr, 23 op/s 2023-09-10T00:50:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5435.819075 secs 2023-09-10T00:50:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:05 smithi098 ceph-mon[103124]: pgmap v3065: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 59 KiB/s wr, 23 op/s 2023-09-10T00:50:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:07 smithi170 ceph-mon[102832]: pgmap v3066: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 99 KiB/s wr, 29 op/s 2023-09-10T00:50:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:07 smithi082 ceph-mon[98694]: pgmap v3066: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 99 KiB/s wr, 29 op/s 2023-09-10T00:50:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:07 smithi098 ceph-mon[103124]: pgmap v3066: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 99 KiB/s wr, 29 op/s 2023-09-10T00:50:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5438.572735 secs 2023-09-10T00:50:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:08 smithi170 ceph-mon[102832]: osdmap e263: 12 total, 12 up, 12 in 2023-09-10T00:50:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5438.572735 secs 2023-09-10T00:50:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:08 smithi082 ceph-mon[98694]: osdmap e263: 12 total, 12 up, 12 in 2023-09-10T00:50:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5438.572735 secs 2023-09-10T00:50:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:08 smithi098 ceph-mon[103124]: osdmap e263: 12 total, 12 up, 12 in 2023-09-10T00:50:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5440.106131 secs 2023-09-10T00:50:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:09 smithi170 ceph-mon[102832]: pgmap v3068: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T00:50:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5440.106131 secs 2023-09-10T00:50:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:09 smithi082 ceph-mon[98694]: pgmap v3068: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T00:50:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5440.106131 secs 2023-09-10T00:50:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:09 smithi098 ceph-mon[103124]: pgmap v3068: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T00:50:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5440.819137 secs 2023-09-10T00:50:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5440.819137 secs 2023-09-10T00:50:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5440.819137 secs 2023-09-10T00:50:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:11 smithi170 ceph-mon[102832]: pgmap v3069: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T00:50:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:11 smithi082 ceph-mon[98694]: pgmap v3069: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T00:50:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:11 smithi098 ceph-mon[103124]: pgmap v3069: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T00:50:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5443.572778 secs 2023-09-10T00:50:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:13 smithi170 ceph-mon[102832]: pgmap v3070: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 49 KiB/s wr, 14 op/s 2023-09-10T00:50:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5443.572778 secs 2023-09-10T00:50:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:13 smithi082 ceph-mon[98694]: pgmap v3070: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 49 KiB/s wr, 14 op/s 2023-09-10T00:50:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5443.572778 secs 2023-09-10T00:50:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:13 smithi098 ceph-mon[103124]: pgmap v3070: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 49 KiB/s wr, 14 op/s 2023-09-10T00:50:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5445.106189 secs 2023-09-10T00:50:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5445.106189 secs 2023-09-10T00:50:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5445.106189 secs 2023-09-10T00:50:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5445.819202 secs 2023-09-10T00:50:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:15 smithi170 ceph-mon[102832]: pgmap v3071: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 49 KiB/s wr, 14 op/s 2023-09-10T00:50:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:15 smithi170 ceph-mon[102832]: osdmap e264: 12 total, 12 up, 12 in 2023-09-10T00:50:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5445.819202 secs 2023-09-10T00:50:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:15 smithi082 ceph-mon[98694]: pgmap v3071: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 49 KiB/s wr, 14 op/s 2023-09-10T00:50:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:15 smithi082 ceph-mon[98694]: osdmap e264: 12 total, 12 up, 12 in 2023-09-10T00:50:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5445.819202 secs 2023-09-10T00:50:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:15 smithi098 ceph-mon[103124]: pgmap v3071: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 49 KiB/s wr, 14 op/s 2023-09-10T00:50:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:15 smithi098 ceph-mon[103124]: osdmap e264: 12 total, 12 up, 12 in 2023-09-10T00:50:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:17 smithi170 ceph-mon[102832]: pgmap v3073: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 51 KiB/s wr, 14 op/s 2023-09-10T00:50:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:17 smithi082 ceph-mon[98694]: pgmap v3073: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 51 KiB/s wr, 14 op/s 2023-09-10T00:50:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:17 smithi098 ceph-mon[103124]: pgmap v3073: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 51 KiB/s wr, 14 op/s 2023-09-10T00:50:19.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5450.106248 secs 2023-09-10T00:50:19.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:19 smithi170 ceph-mon[102832]: pgmap v3074: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T00:50:19.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:50:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5450.106248 secs 2023-09-10T00:50:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:19 smithi082 ceph-mon[98694]: pgmap v3074: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T00:50:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:50:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5450.106248 secs 2023-09-10T00:50:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:19 smithi098 ceph-mon[103124]: pgmap v3074: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T00:50:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:50:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5450.819266 secs 2023-09-10T00:50:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:50:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:50:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:50:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5450.819266 secs 2023-09-10T00:50:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:50:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:50:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:50:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5450.819266 secs 2023-09-10T00:50:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:50:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:50:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:50:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:21 smithi170 ceph-mon[102832]: pgmap v3075: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T00:50:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:21 smithi082 ceph-mon[98694]: pgmap v3075: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T00:50:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:21 smithi098 ceph-mon[103124]: pgmap v3075: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T00:50:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:23 smithi170 ceph-mon[102832]: pgmap v3076: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T00:50:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:23 smithi082 ceph-mon[98694]: pgmap v3076: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T00:50:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:23 smithi098 ceph-mon[103124]: pgmap v3076: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T00:50:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5455.106335 secs 2023-09-10T00:50:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5455.106335 secs 2023-09-10T00:50:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5455.106335 secs 2023-09-10T00:50:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5455.819329 secs 2023-09-10T00:50:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:25 smithi170 ceph-mon[102832]: pgmap v3077: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T00:50:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5455.819329 secs 2023-09-10T00:50:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:25 smithi082 ceph-mon[98694]: pgmap v3077: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T00:50:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5455.819329 secs 2023-09-10T00:50:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:25 smithi098 ceph-mon[103124]: pgmap v3077: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T00:50:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:27 smithi170 ceph-mon[102832]: pgmap v3078: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 118 KiB/s wr, 17 op/s 2023-09-10T00:50:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:27 smithi082 ceph-mon[98694]: pgmap v3078: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 118 KiB/s wr, 17 op/s 2023-09-10T00:50:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:27 smithi098 ceph-mon[103124]: pgmap v3078: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 118 KiB/s wr, 17 op/s 2023-09-10T00:50:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5460.106394 secs 2023-09-10T00:50:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:29 smithi170 ceph-mon[102832]: pgmap v3079: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T00:50:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:50:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5460.106394 secs 2023-09-10T00:50:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:29 smithi082 ceph-mon[98694]: pgmap v3079: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T00:50:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:50:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5460.106394 secs 2023-09-10T00:50:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:29 smithi098 ceph-mon[103124]: pgmap v3079: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T00:50:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:50:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5460.819394 secs 2023-09-10T00:50:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:50:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:50:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:50:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5460.819394 secs 2023-09-10T00:50:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:50:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:50:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:50:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5460.819394 secs 2023-09-10T00:50:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:50:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:50:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:50:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:31 smithi170 ceph-mon[102832]: pgmap v3080: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T00:50:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:31 smithi082 ceph-mon[98694]: pgmap v3080: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T00:50:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:31 smithi098 ceph-mon[103124]: pgmap v3080: 97 pgs: 97 active+clean; 9.0 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T00:50:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:33 smithi170 ceph-mon[102832]: pgmap v3081: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 173 KiB/s wr, 17 op/s 2023-09-10T00:50:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:33 smithi082 ceph-mon[98694]: pgmap v3081: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 173 KiB/s wr, 17 op/s 2023-09-10T00:50:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:33 smithi098 ceph-mon[103124]: pgmap v3081: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 173 KiB/s wr, 17 op/s 2023-09-10T00:50:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5465.106450 secs 2023-09-10T00:50:34.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5465.106450 secs 2023-09-10T00:50:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5465.106450 secs 2023-09-10T00:50:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5465.819461 secs 2023-09-10T00:50:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:35 smithi082 ceph-mon[98694]: pgmap v3082: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 139 KiB/s wr, 12 op/s 2023-09-10T00:50:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5465.819461 secs 2023-09-10T00:50:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:35 smithi098 ceph-mon[103124]: pgmap v3082: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 139 KiB/s wr, 12 op/s 2023-09-10T00:50:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5465.819461 secs 2023-09-10T00:50:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:35 smithi170 ceph-mon[102832]: pgmap v3082: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 139 KiB/s wr, 12 op/s 2023-09-10T00:50:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:37 smithi082 ceph-mon[98694]: pgmap v3083: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 182 KiB/s wr, 17 op/s 2023-09-10T00:50:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:37 smithi098 ceph-mon[103124]: pgmap v3083: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 182 KiB/s wr, 17 op/s 2023-09-10T00:50:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:37 smithi170 ceph-mon[102832]: pgmap v3083: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 182 KiB/s wr, 17 op/s 2023-09-10T00:50:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5470.106535 secs 2023-09-10T00:50:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:39 smithi082 ceph-mon[98694]: pgmap v3084: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 12 op/s 2023-09-10T00:50:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5470.106535 secs 2023-09-10T00:50:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:39 smithi098 ceph-mon[103124]: pgmap v3084: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 12 op/s 2023-09-10T00:50:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5470.106535 secs 2023-09-10T00:50:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:39 smithi170 ceph-mon[102832]: pgmap v3084: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 12 op/s 2023-09-10T00:50:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5470.819485 secs 2023-09-10T00:50:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5470.819485 secs 2023-09-10T00:50:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5470.819485 secs 2023-09-10T00:50:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:41 smithi082 ceph-mon[98694]: pgmap v3085: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 12 op/s 2023-09-10T00:50:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:41 smithi098 ceph-mon[103124]: pgmap v3085: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 12 op/s 2023-09-10T00:50:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:41 smithi170 ceph-mon[102832]: pgmap v3085: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 12 op/s 2023-09-10T00:50:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:43 smithi082 ceph-mon[98694]: pgmap v3086: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 236 KiB/s wr, 17 op/s 2023-09-10T00:50:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:43 smithi098 ceph-mon[103124]: pgmap v3086: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 236 KiB/s wr, 17 op/s 2023-09-10T00:50:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:43 smithi170 ceph-mon[102832]: pgmap v3086: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 236 KiB/s wr, 17 op/s 2023-09-10T00:50:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5475.106596 secs 2023-09-10T00:50:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5475.106596 secs 2023-09-10T00:50:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5475.106596 secs 2023-09-10T00:50:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5475.819547 secs 2023-09-10T00:50:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:45 smithi082 ceph-mon[98694]: pgmap v3087: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T00:50:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5475.819547 secs 2023-09-10T00:50:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:45 smithi098 ceph-mon[103124]: pgmap v3087: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T00:50:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5475.819547 secs 2023-09-10T00:50:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:45 smithi170 ceph-mon[102832]: pgmap v3087: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T00:50:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:47 smithi082 ceph-mon[98694]: pgmap v3088: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 178 KiB/s wr, 17 op/s 2023-09-10T00:50:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:47 smithi098 ceph-mon[103124]: pgmap v3088: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 178 KiB/s wr, 17 op/s 2023-09-10T00:50:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:47 smithi170 ceph-mon[102832]: pgmap v3088: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 178 KiB/s wr, 17 op/s 2023-09-10T00:50:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5478.573225 secs 2023-09-10T00:50:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5478.573225 secs 2023-09-10T00:50:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5478.573225 secs 2023-09-10T00:50:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5480.106656 secs 2023-09-10T00:50:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:49 smithi082 ceph-mon[98694]: pgmap v3089: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T00:50:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5480.106656 secs 2023-09-10T00:50:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:49 smithi098 ceph-mon[103124]: pgmap v3089: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T00:50:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5480.106656 secs 2023-09-10T00:50:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:49 smithi170 ceph-mon[102832]: pgmap v3089: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T00:50:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5480.819613 secs 2023-09-10T00:50:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5480.819613 secs 2023-09-10T00:50:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5480.819613 secs 2023-09-10T00:50:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:51 smithi098 ceph-mon[103124]: pgmap v3090: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T00:50:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:51 smithi170 ceph-mon[102832]: pgmap v3090: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T00:50:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:51 smithi082 ceph-mon[98694]: pgmap v3090: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T00:50:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:50:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:50:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:50:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:50:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:50:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:50:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:50:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:50:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:50:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5483.573287 secs 2023-09-10T00:50:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:53 smithi098 ceph-mon[103124]: pgmap v3091: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 177 KiB/s wr, 17 op/s 2023-09-10T00:50:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5483.573287 secs 2023-09-10T00:50:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:53 smithi170 ceph-mon[102832]: pgmap v3091: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 177 KiB/s wr, 17 op/s 2023-09-10T00:50:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5483.573287 secs 2023-09-10T00:50:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:53 smithi082 ceph-mon[98694]: pgmap v3091: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 177 KiB/s wr, 17 op/s 2023-09-10T00:50:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5485.106739 secs 2023-09-10T00:50:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5485.106739 secs 2023-09-10T00:50:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5485.106739 secs 2023-09-10T00:50:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5485.819674 secs 2023-09-10T00:50:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:55 smithi098 ceph-mon[103124]: pgmap v3092: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T00:50:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5485.819674 secs 2023-09-10T00:50:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:55 smithi170 ceph-mon[102832]: pgmap v3092: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T00:50:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5485.819674 secs 2023-09-10T00:50:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:55 smithi082 ceph-mon[98694]: pgmap v3092: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T00:50:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:57 smithi098 ceph-mon[103124]: pgmap v3093: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 110 KiB/s wr, 17 op/s 2023-09-10T00:50:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:57 smithi170 ceph-mon[102832]: pgmap v3093: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 110 KiB/s wr, 17 op/s 2023-09-10T00:50:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:57 smithi082 ceph-mon[98694]: pgmap v3093: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 110 KiB/s wr, 17 op/s 2023-09-10T00:50:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5488.573346 secs 2023-09-10T00:50:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5488.573346 secs 2023-09-10T00:50:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5488.573346 secs 2023-09-10T00:50:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5490.106794 secs 2023-09-10T00:50:59.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:50:59 smithi098 ceph-mon[103124]: pgmap v3094: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T00:50:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5490.106794 secs 2023-09-10T00:50:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:50:59 smithi170 ceph-mon[102832]: pgmap v3094: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T00:50:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5490.106794 secs 2023-09-10T00:50:59.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:50:59 smithi082 ceph-mon[98694]: pgmap v3094: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T00:51:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5490.819736 secs 2023-09-10T00:51:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5490.819736 secs 2023-09-10T00:51:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5490.819736 secs 2023-09-10T00:51:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:01 smithi098 ceph-mon[103124]: pgmap v3095: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T00:51:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:01 smithi170 ceph-mon[102832]: pgmap v3095: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T00:51:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:01 smithi082 ceph-mon[98694]: pgmap v3095: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T00:51:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5493.573418 secs 2023-09-10T00:51:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:03 smithi098 ceph-mon[103124]: pgmap v3096: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 79 KiB/s wr, 17 op/s 2023-09-10T00:51:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5493.573418 secs 2023-09-10T00:51:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:03 smithi170 ceph-mon[102832]: pgmap v3096: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 79 KiB/s wr, 17 op/s 2023-09-10T00:51:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5493.573418 secs 2023-09-10T00:51:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:03 smithi082 ceph-mon[98694]: pgmap v3096: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 79 KiB/s wr, 17 op/s 2023-09-10T00:51:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5495.106876 secs 2023-09-10T00:51:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5495.106876 secs 2023-09-10T00:51:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5495.106876 secs 2023-09-10T00:51:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5495.819799 secs 2023-09-10T00:51:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:05 smithi098 ceph-mon[103124]: pgmap v3097: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 37 KiB/s wr, 12 op/s 2023-09-10T00:51:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5495.819799 secs 2023-09-10T00:51:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:05 smithi170 ceph-mon[102832]: pgmap v3097: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 37 KiB/s wr, 12 op/s 2023-09-10T00:51:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5495.819799 secs 2023-09-10T00:51:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:05 smithi082 ceph-mon[98694]: pgmap v3097: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 37 KiB/s wr, 12 op/s 2023-09-10T00:51:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:07 smithi098 ceph-mon[103124]: pgmap v3098: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 125 KiB/s wr, 29 op/s 2023-09-10T00:51:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:07 smithi170 ceph-mon[102832]: pgmap v3098: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 125 KiB/s wr, 29 op/s 2023-09-10T00:51:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:07 smithi082 ceph-mon[98694]: pgmap v3098: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 125 KiB/s wr, 29 op/s 2023-09-10T00:51:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5498.573477 secs 2023-09-10T00:51:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:08 smithi170 ceph-mon[102832]: osdmap e265: 12 total, 12 up, 12 in 2023-09-10T00:51:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5498.573477 secs 2023-09-10T00:51:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:08 smithi082 ceph-mon[98694]: osdmap e265: 12 total, 12 up, 12 in 2023-09-10T00:51:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5498.573477 secs 2023-09-10T00:51:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:08 smithi098 ceph-mon[103124]: osdmap e265: 12 total, 12 up, 12 in 2023-09-10T00:51:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5500.106960 secs 2023-09-10T00:51:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:09 smithi170 ceph-mon[102832]: pgmap v3100: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 120 KiB/s wr, 29 op/s 2023-09-10T00:51:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5500.106960 secs 2023-09-10T00:51:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:09 smithi082 ceph-mon[98694]: pgmap v3100: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 120 KiB/s wr, 29 op/s 2023-09-10T00:51:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5500.106960 secs 2023-09-10T00:51:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:09 smithi098 ceph-mon[103124]: pgmap v3100: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 120 KiB/s wr, 29 op/s 2023-09-10T00:51:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5500.819863 secs 2023-09-10T00:51:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5500.819863 secs 2023-09-10T00:51:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5500.819863 secs 2023-09-10T00:51:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:11 smithi170 ceph-mon[102832]: pgmap v3101: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 120 KiB/s wr, 29 op/s 2023-09-10T00:51:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:11 smithi082 ceph-mon[98694]: pgmap v3101: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 120 KiB/s wr, 29 op/s 2023-09-10T00:51:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:11 smithi098 ceph-mon[103124]: pgmap v3101: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 120 KiB/s wr, 29 op/s 2023-09-10T00:51:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5503.573537 secs 2023-09-10T00:51:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:13 smithi170 ceph-mon[102832]: pgmap v3102: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 106 KiB/s wr, 26 op/s 2023-09-10T00:51:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5503.573537 secs 2023-09-10T00:51:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:13 smithi082 ceph-mon[98694]: pgmap v3102: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 106 KiB/s wr, 26 op/s 2023-09-10T00:51:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5503.573537 secs 2023-09-10T00:51:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:13 smithi098 ceph-mon[103124]: pgmap v3102: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 106 KiB/s wr, 26 op/s 2023-09-10T00:51:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5505.107018 secs 2023-09-10T00:51:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5505.107018 secs 2023-09-10T00:51:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5505.107018 secs 2023-09-10T00:51:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5505.819925 secs 2023-09-10T00:51:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:15 smithi170 ceph-mon[102832]: pgmap v3103: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 106 KiB/s wr, 26 op/s 2023-09-10T00:51:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:15 smithi170 ceph-mon[102832]: osdmap e266: 12 total, 12 up, 12 in 2023-09-10T00:51:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5505.819925 secs 2023-09-10T00:51:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:15 smithi082 ceph-mon[98694]: pgmap v3103: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 106 KiB/s wr, 26 op/s 2023-09-10T00:51:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:15 smithi082 ceph-mon[98694]: osdmap e266: 12 total, 12 up, 12 in 2023-09-10T00:51:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5505.819925 secs 2023-09-10T00:51:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:15 smithi098 ceph-mon[103124]: pgmap v3103: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 106 KiB/s wr, 26 op/s 2023-09-10T00:51:15.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:15 smithi098 ceph-mon[103124]: osdmap e266: 12 total, 12 up, 12 in 2023-09-10T00:51:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:17 smithi170 ceph-mon[102832]: pgmap v3105: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 40 KiB/s wr, 13 op/s 2023-09-10T00:51:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:17 smithi082 ceph-mon[98694]: pgmap v3105: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 40 KiB/s wr, 13 op/s 2023-09-10T00:51:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:17 smithi098 ceph-mon[103124]: pgmap v3105: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 40 KiB/s wr, 13 op/s 2023-09-10T00:51:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5510.107105 secs 2023-09-10T00:51:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:19 smithi170 ceph-mon[102832]: pgmap v3106: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T00:51:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:51:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5510.107105 secs 2023-09-10T00:51:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:19 smithi082 ceph-mon[98694]: pgmap v3106: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T00:51:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:51:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5510.107105 secs 2023-09-10T00:51:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:19 smithi098 ceph-mon[103124]: pgmap v3106: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T00:51:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:51:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5510.819989 secs 2023-09-10T00:51:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:51:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:51:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:51:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5510.819989 secs 2023-09-10T00:51:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:51:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:51:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:51:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5510.819989 secs 2023-09-10T00:51:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:51:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:51:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:51:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:21 smithi170 ceph-mon[102832]: pgmap v3107: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T00:51:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:21 smithi082 ceph-mon[98694]: pgmap v3107: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T00:51:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:21 smithi098 ceph-mon[103124]: pgmap v3107: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T00:51:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:23 smithi170 ceph-mon[102832]: pgmap v3108: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T00:51:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:23 smithi082 ceph-mon[98694]: pgmap v3108: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T00:51:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:23 smithi098 ceph-mon[103124]: pgmap v3108: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T00:51:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5515.107189 secs 2023-09-10T00:51:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5515.107189 secs 2023-09-10T00:51:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5515.107189 secs 2023-09-10T00:51:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5515.820054 secs 2023-09-10T00:51:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:25 smithi170 ceph-mon[102832]: pgmap v3109: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T00:51:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5515.820054 secs 2023-09-10T00:51:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:25 smithi082 ceph-mon[98694]: pgmap v3109: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T00:51:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5515.820054 secs 2023-09-10T00:51:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:25 smithi098 ceph-mon[103124]: pgmap v3109: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T00:51:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:27 smithi170 ceph-mon[102832]: pgmap v3110: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 156 KiB/s wr, 17 op/s 2023-09-10T00:51:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:27 smithi082 ceph-mon[98694]: pgmap v3110: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 156 KiB/s wr, 17 op/s 2023-09-10T00:51:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:27 smithi098 ceph-mon[103124]: pgmap v3110: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 156 KiB/s wr, 17 op/s 2023-09-10T00:51:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5520.107249 secs 2023-09-10T00:51:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:29 smithi170 ceph-mon[102832]: pgmap v3111: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T00:51:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5520.107249 secs 2023-09-10T00:51:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:29 smithi082 ceph-mon[98694]: pgmap v3111: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T00:51:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5520.107249 secs 2023-09-10T00:51:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:29 smithi098 ceph-mon[103124]: pgmap v3111: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T00:51:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5520.820119 secs 2023-09-10T00:51:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:51:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5520.820119 secs 2023-09-10T00:51:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:51:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5520.820119 secs 2023-09-10T00:51:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:51:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:31 smithi170 ceph-mon[102832]: pgmap v3112: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T00:51:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:51:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:51:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:51:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:31 smithi082 ceph-mon[98694]: pgmap v3112: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T00:51:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:51:31.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:51:31.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:51:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:31 smithi098 ceph-mon[103124]: pgmap v3112: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T00:51:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:51:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:51:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:51:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:33 smithi170 ceph-mon[102832]: pgmap v3113: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 257 KiB/s wr, 17 op/s 2023-09-10T00:51:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:33 smithi082 ceph-mon[98694]: pgmap v3113: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 257 KiB/s wr, 17 op/s 2023-09-10T00:51:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:33 smithi098 ceph-mon[103124]: pgmap v3113: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 257 KiB/s wr, 17 op/s 2023-09-10T00:51:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5525.107341 secs 2023-09-10T00:51:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5525.107341 secs 2023-09-10T00:51:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5525.107341 secs 2023-09-10T00:51:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5525.820181 secs 2023-09-10T00:51:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:35 smithi170 ceph-mon[102832]: pgmap v3114: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 181 KiB/s wr, 12 op/s 2023-09-10T00:51:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5525.820181 secs 2023-09-10T00:51:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:35 smithi082 ceph-mon[98694]: pgmap v3114: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 181 KiB/s wr, 12 op/s 2023-09-10T00:51:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5525.820181 secs 2023-09-10T00:51:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:35 smithi098 ceph-mon[103124]: pgmap v3114: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 181 KiB/s wr, 12 op/s 2023-09-10T00:51:37.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:37 smithi170 ceph-mon[102832]: pgmap v3115: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 215 KiB/s wr, 18 op/s 2023-09-10T00:51:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:37 smithi082 ceph-mon[98694]: pgmap v3115: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 215 KiB/s wr, 18 op/s 2023-09-10T00:51:37.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:37 smithi098 ceph-mon[103124]: pgmap v3115: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 215 KiB/s wr, 18 op/s 2023-09-10T00:51:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5530.107397 secs 2023-09-10T00:51:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:39 smithi170 ceph-mon[102832]: pgmap v3116: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 12 op/s 2023-09-10T00:51:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5530.107397 secs 2023-09-10T00:51:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:39 smithi082 ceph-mon[98694]: pgmap v3116: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 12 op/s 2023-09-10T00:51:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5530.107397 secs 2023-09-10T00:51:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:39 smithi098 ceph-mon[103124]: pgmap v3116: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 12 op/s 2023-09-10T00:51:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5530.820244 secs 2023-09-10T00:51:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5530.820244 secs 2023-09-10T00:51:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5530.820244 secs 2023-09-10T00:51:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:41 smithi170 ceph-mon[102832]: pgmap v3117: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 12 op/s 2023-09-10T00:51:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:41 smithi082 ceph-mon[98694]: pgmap v3117: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 12 op/s 2023-09-10T00:51:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:41 smithi098 ceph-mon[103124]: pgmap v3117: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 12 op/s 2023-09-10T00:51:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:43 smithi170 ceph-mon[102832]: pgmap v3118: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 199 KiB/s wr, 18 op/s 2023-09-10T00:51:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:43 smithi082 ceph-mon[98694]: pgmap v3118: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 199 KiB/s wr, 18 op/s 2023-09-10T00:51:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:43 smithi098 ceph-mon[103124]: pgmap v3118: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 199 KiB/s wr, 18 op/s 2023-09-10T00:51:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5535.107455 secs 2023-09-10T00:51:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5535.107455 secs 2023-09-10T00:51:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5535.107455 secs 2023-09-10T00:51:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5535.820308 secs 2023-09-10T00:51:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:45 smithi170 ceph-mon[102832]: pgmap v3119: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T00:51:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5535.820308 secs 2023-09-10T00:51:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:45 smithi082 ceph-mon[98694]: pgmap v3119: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T00:51:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5535.820308 secs 2023-09-10T00:51:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:45 smithi098 ceph-mon[103124]: pgmap v3119: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T00:51:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:47 smithi170 ceph-mon[102832]: pgmap v3120: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 162 KiB/s wr, 17 op/s 2023-09-10T00:51:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:47 smithi082 ceph-mon[98694]: pgmap v3120: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 162 KiB/s wr, 17 op/s 2023-09-10T00:51:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:47 smithi098 ceph-mon[103124]: pgmap v3120: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 162 KiB/s wr, 17 op/s 2023-09-10T00:51:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5538.573984 secs 2023-09-10T00:51:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5538.573984 secs 2023-09-10T00:51:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5538.573984 secs 2023-09-10T00:51:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5540.107495 secs 2023-09-10T00:51:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:49 smithi170 ceph-mon[102832]: pgmap v3121: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T00:51:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5540.107495 secs 2023-09-10T00:51:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:49 smithi082 ceph-mon[98694]: pgmap v3121: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T00:51:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5540.107495 secs 2023-09-10T00:51:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:49 smithi098 ceph-mon[103124]: pgmap v3121: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T00:51:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5540.820373 secs 2023-09-10T00:51:50.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5540.820373 secs 2023-09-10T00:51:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5540.820373 secs 2023-09-10T00:51:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:51 smithi170 ceph-mon[102832]: pgmap v3122: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T00:51:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:51 smithi082 ceph-mon[98694]: pgmap v3122: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T00:51:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:51 smithi098 ceph-mon[103124]: pgmap v3122: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T00:51:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:51:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:51:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:51:52.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:51:52.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:51:52.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:51:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:51:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:51:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:51:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5543.574071 secs 2023-09-10T00:51:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:53 smithi170 ceph-mon[102832]: pgmap v3123: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 157 KiB/s wr, 16 op/s 2023-09-10T00:51:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5543.574071 secs 2023-09-10T00:51:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:53 smithi082 ceph-mon[98694]: pgmap v3123: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 157 KiB/s wr, 16 op/s 2023-09-10T00:51:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5543.574071 secs 2023-09-10T00:51:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:53 smithi098 ceph-mon[103124]: pgmap v3123: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 157 KiB/s wr, 16 op/s 2023-09-10T00:51:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5545.107578 secs 2023-09-10T00:51:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5545.107578 secs 2023-09-10T00:51:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5545.107578 secs 2023-09-10T00:51:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5545.820439 secs 2023-09-10T00:51:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:55 smithi170 ceph-mon[102832]: pgmap v3124: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 134 KiB/s wr, 10 op/s 2023-09-10T00:51:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5545.820439 secs 2023-09-10T00:51:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:55 smithi082 ceph-mon[98694]: pgmap v3124: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 134 KiB/s wr, 10 op/s 2023-09-10T00:51:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5545.820439 secs 2023-09-10T00:51:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:55 smithi098 ceph-mon[103124]: pgmap v3124: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 134 KiB/s wr, 10 op/s 2023-09-10T00:51:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:57 smithi170 ceph-mon[102832]: pgmap v3125: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 160 KiB/s wr, 16 op/s 2023-09-10T00:51:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:57 smithi082 ceph-mon[98694]: pgmap v3125: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 160 KiB/s wr, 16 op/s 2023-09-10T00:51:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:57 smithi098 ceph-mon[103124]: pgmap v3125: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 160 KiB/s wr, 16 op/s 2023-09-10T00:51:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5548.574131 secs 2023-09-10T00:51:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5548.574131 secs 2023-09-10T00:51:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5548.574131 secs 2023-09-10T00:51:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5550.107637 secs 2023-09-10T00:51:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:51:59 smithi170 ceph-mon[102832]: pgmap v3126: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T00:51:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5550.107637 secs 2023-09-10T00:51:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:51:59 smithi082 ceph-mon[98694]: pgmap v3126: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T00:51:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5550.107637 secs 2023-09-10T00:51:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:51:59 smithi098 ceph-mon[103124]: pgmap v3126: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T00:52:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5550.820484 secs 2023-09-10T00:52:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5550.820484 secs 2023-09-10T00:52:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5550.820484 secs 2023-09-10T00:52:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:01 smithi170 ceph-mon[102832]: pgmap v3127: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T00:52:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:01 smithi082 ceph-mon[98694]: pgmap v3127: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T00:52:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:01 smithi098 ceph-mon[103124]: pgmap v3127: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T00:52:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5553.574189 secs 2023-09-10T00:52:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:03 smithi170 ceph-mon[102832]: pgmap v3128: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 15 op/s 2023-09-10T00:52:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5553.574189 secs 2023-09-10T00:52:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:03 smithi082 ceph-mon[98694]: pgmap v3128: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 15 op/s 2023-09-10T00:52:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5553.574189 secs 2023-09-10T00:52:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:03 smithi098 ceph-mon[103124]: pgmap v3128: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 15 op/s 2023-09-10T00:52:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5555.107692 secs 2023-09-10T00:52:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5555.107692 secs 2023-09-10T00:52:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5555.107692 secs 2023-09-10T00:52:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5555.820549 secs 2023-09-10T00:52:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:05 smithi170 ceph-mon[102832]: pgmap v3129: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T00:52:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5555.820549 secs 2023-09-10T00:52:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:05 smithi082 ceph-mon[98694]: pgmap v3129: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T00:52:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5555.820549 secs 2023-09-10T00:52:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:05 smithi098 ceph-mon[103124]: pgmap v3129: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T00:52:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:07 smithi170 ceph-mon[102832]: pgmap v3130: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 113 KiB/s wr, 30 op/s 2023-09-10T00:52:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:07 smithi082 ceph-mon[98694]: pgmap v3130: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 113 KiB/s wr, 30 op/s 2023-09-10T00:52:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:07 smithi098 ceph-mon[103124]: pgmap v3130: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 113 KiB/s wr, 30 op/s 2023-09-10T00:52:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5558.574282 secs 2023-09-10T00:52:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:08 smithi170 ceph-mon[102832]: osdmap e267: 12 total, 12 up, 12 in 2023-09-10T00:52:08.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:08 smithi170 ceph-mon[102832]: pgmap v3132: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T00:52:08.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5558.574282 secs 2023-09-10T00:52:08.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:08 smithi082 ceph-mon[98694]: osdmap e267: 12 total, 12 up, 12 in 2023-09-10T00:52:08.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:08 smithi082 ceph-mon[98694]: pgmap v3132: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T00:52:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5558.574282 secs 2023-09-10T00:52:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:08 smithi098 ceph-mon[103124]: osdmap e267: 12 total, 12 up, 12 in 2023-09-10T00:52:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:08 smithi098 ceph-mon[103124]: pgmap v3132: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T00:52:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5560.107753 secs 2023-09-10T00:52:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5560.107753 secs 2023-09-10T00:52:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5560.107753 secs 2023-09-10T00:52:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5560.820612 secs 2023-09-10T00:52:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:10 smithi082 ceph-mon[98694]: pgmap v3133: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T00:52:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5560.820612 secs 2023-09-10T00:52:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:10 smithi098 ceph-mon[103124]: pgmap v3133: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T00:52:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5560.820612 secs 2023-09-10T00:52:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:10 smithi170 ceph-mon[102832]: pgmap v3133: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T00:52:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5563.574343 secs 2023-09-10T00:52:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:12 smithi098 ceph-mon[103124]: pgmap v3134: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T00:52:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5563.574343 secs 2023-09-10T00:52:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:12 smithi170 ceph-mon[102832]: pgmap v3134: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T00:52:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5563.574343 secs 2023-09-10T00:52:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:12 smithi082 ceph-mon[98694]: pgmap v3134: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T00:52:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5565.107859 secs 2023-09-10T00:52:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5565.107859 secs 2023-09-10T00:52:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5565.107859 secs 2023-09-10T00:52:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5565.820675 secs 2023-09-10T00:52:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:14 smithi098 ceph-mon[103124]: pgmap v3135: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T00:52:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:14 smithi098 ceph-mon[103124]: osdmap e268: 12 total, 12 up, 12 in 2023-09-10T00:52:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5565.820675 secs 2023-09-10T00:52:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:14 smithi170 ceph-mon[102832]: pgmap v3135: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T00:52:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:14 smithi170 ceph-mon[102832]: osdmap e268: 12 total, 12 up, 12 in 2023-09-10T00:52:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5565.820675 secs 2023-09-10T00:52:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:14 smithi082 ceph-mon[98694]: pgmap v3135: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T00:52:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:14 smithi082 ceph-mon[98694]: osdmap e268: 12 total, 12 up, 12 in 2023-09-10T00:52:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:16 smithi098 ceph-mon[103124]: pgmap v3137: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 14 op/s 2023-09-10T00:52:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:16 smithi170 ceph-mon[102832]: pgmap v3137: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 14 op/s 2023-09-10T00:52:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:16 smithi082 ceph-mon[98694]: pgmap v3137: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 14 op/s 2023-09-10T00:52:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5570.107913 secs 2023-09-10T00:52:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:19 smithi170 ceph-mon[102832]: pgmap v3138: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T00:52:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:52:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5570.107913 secs 2023-09-10T00:52:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:19 smithi082 ceph-mon[98694]: pgmap v3138: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T00:52:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:52:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5570.107913 secs 2023-09-10T00:52:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:19 smithi098 ceph-mon[103124]: pgmap v3138: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T00:52:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:52:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5570.820741 secs 2023-09-10T00:52:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:52:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:52:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:52:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5570.820741 secs 2023-09-10T00:52:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:52:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:52:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:52:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5570.820741 secs 2023-09-10T00:52:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:52:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:52:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:52:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:21 smithi170 ceph-mon[102832]: pgmap v3139: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T00:52:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:21 smithi082 ceph-mon[98694]: pgmap v3139: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T00:52:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:21 smithi098 ceph-mon[103124]: pgmap v3139: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T00:52:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:23 smithi170 ceph-mon[102832]: pgmap v3140: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 51 KiB/s wr, 12 op/s 2023-09-10T00:52:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:23 smithi082 ceph-mon[98694]: pgmap v3140: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 51 KiB/s wr, 12 op/s 2023-09-10T00:52:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:23 smithi098 ceph-mon[103124]: pgmap v3140: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 51 KiB/s wr, 12 op/s 2023-09-10T00:52:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5575.107970 secs 2023-09-10T00:52:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5575.107970 secs 2023-09-10T00:52:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5575.107970 secs 2023-09-10T00:52:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5575.820803 secs 2023-09-10T00:52:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:25 smithi170 ceph-mon[102832]: pgmap v3141: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 51 KiB/s wr, 12 op/s 2023-09-10T00:52:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5575.820803 secs 2023-09-10T00:52:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:25 smithi082 ceph-mon[98694]: pgmap v3141: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 51 KiB/s wr, 12 op/s 2023-09-10T00:52:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5575.820803 secs 2023-09-10T00:52:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:25 smithi098 ceph-mon[103124]: pgmap v3141: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 51 KiB/s wr, 12 op/s 2023-09-10T00:52:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:27 smithi170 ceph-mon[102832]: pgmap v3142: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 198 KiB/s wr, 17 op/s 2023-09-10T00:52:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:27 smithi082 ceph-mon[98694]: pgmap v3142: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 198 KiB/s wr, 17 op/s 2023-09-10T00:52:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:27 smithi098 ceph-mon[103124]: pgmap v3142: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 198 KiB/s wr, 17 op/s 2023-09-10T00:52:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5580.108054 secs 2023-09-10T00:52:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:29 smithi170 ceph-mon[102832]: pgmap v3143: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 181 KiB/s wr, 11 op/s 2023-09-10T00:52:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5580.108054 secs 2023-09-10T00:52:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:29 smithi082 ceph-mon[98694]: pgmap v3143: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 181 KiB/s wr, 11 op/s 2023-09-10T00:52:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5580.108054 secs 2023-09-10T00:52:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:29 smithi098 ceph-mon[103124]: pgmap v3143: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 181 KiB/s wr, 11 op/s 2023-09-10T00:52:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5580.820868 secs 2023-09-10T00:52:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5580.820868 secs 2023-09-10T00:52:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5580.820868 secs 2023-09-10T00:52:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:31 smithi170 ceph-mon[102832]: pgmap v3144: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 181 KiB/s wr, 11 op/s 2023-09-10T00:52:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:52:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:31 smithi082 ceph-mon[98694]: pgmap v3144: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 181 KiB/s wr, 11 op/s 2023-09-10T00:52:31.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:52:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:31 smithi098 ceph-mon[103124]: pgmap v3144: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 181 KiB/s wr, 11 op/s 2023-09-10T00:52:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:52:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:52:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:52:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:52:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:52:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:52:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:52:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:52:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:52:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:52:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:33 smithi170 ceph-mon[102832]: pgmap v3145: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 267 KiB/s wr, 17 op/s 2023-09-10T00:52:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:33 smithi082 ceph-mon[98694]: pgmap v3145: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 267 KiB/s wr, 17 op/s 2023-09-10T00:52:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:33 smithi098 ceph-mon[103124]: pgmap v3145: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 267 KiB/s wr, 17 op/s 2023-09-10T00:52:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5585.108113 secs 2023-09-10T00:52:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5585.108113 secs 2023-09-10T00:52:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5585.108113 secs 2023-09-10T00:52:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5585.820931 secs 2023-09-10T00:52:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:35 smithi170 ceph-mon[102832]: pgmap v3146: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 225 KiB/s wr, 12 op/s 2023-09-10T00:52:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5585.820931 secs 2023-09-10T00:52:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:35 smithi082 ceph-mon[98694]: pgmap v3146: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 225 KiB/s wr, 12 op/s 2023-09-10T00:52:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5585.820931 secs 2023-09-10T00:52:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:35 smithi098 ceph-mon[103124]: pgmap v3146: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 225 KiB/s wr, 12 op/s 2023-09-10T00:52:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:37 smithi170 ceph-mon[102832]: pgmap v3147: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 260 KiB/s wr, 18 op/s 2023-09-10T00:52:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:37 smithi082 ceph-mon[98694]: pgmap v3147: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 260 KiB/s wr, 18 op/s 2023-09-10T00:52:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:37 smithi098 ceph-mon[103124]: pgmap v3147: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 260 KiB/s wr, 18 op/s 2023-09-10T00:52:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5590.108171 secs 2023-09-10T00:52:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:39 smithi170 ceph-mon[102832]: pgmap v3148: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T00:52:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5590.108171 secs 2023-09-10T00:52:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:39 smithi082 ceph-mon[98694]: pgmap v3148: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T00:52:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5590.108171 secs 2023-09-10T00:52:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:39 smithi098 ceph-mon[103124]: pgmap v3148: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T00:52:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5590.820996 secs 2023-09-10T00:52:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5590.820996 secs 2023-09-10T00:52:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5590.820996 secs 2023-09-10T00:52:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:41 smithi170 ceph-mon[102832]: pgmap v3149: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T00:52:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:41 smithi082 ceph-mon[98694]: pgmap v3149: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T00:52:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:41 smithi098 ceph-mon[103124]: pgmap v3149: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T00:52:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:43 smithi170 ceph-mon[102832]: pgmap v3150: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 172 KiB/s wr, 17 op/s 2023-09-10T00:52:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:43 smithi082 ceph-mon[98694]: pgmap v3150: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 172 KiB/s wr, 17 op/s 2023-09-10T00:52:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:43 smithi098 ceph-mon[103124]: pgmap v3150: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 172 KiB/s wr, 17 op/s 2023-09-10T00:52:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5595.108234 secs 2023-09-10T00:52:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5595.108234 secs 2023-09-10T00:52:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5595.108234 secs 2023-09-10T00:52:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5595.821058 secs 2023-09-10T00:52:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:45 smithi170 ceph-mon[102832]: pgmap v3151: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T00:52:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5595.821058 secs 2023-09-10T00:52:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:45 smithi082 ceph-mon[98694]: pgmap v3151: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T00:52:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5595.821058 secs 2023-09-10T00:52:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:45 smithi098 ceph-mon[103124]: pgmap v3151: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T00:52:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:47 smithi170 ceph-mon[102832]: pgmap v3152: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 151 KiB/s wr, 17 op/s 2023-09-10T00:52:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:47 smithi082 ceph-mon[98694]: pgmap v3152: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 151 KiB/s wr, 17 op/s 2023-09-10T00:52:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:47 smithi098 ceph-mon[103124]: pgmap v3152: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 151 KiB/s wr, 17 op/s 2023-09-10T00:52:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5598.574964 secs 2023-09-10T00:52:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5598.574964 secs 2023-09-10T00:52:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5598.574964 secs 2023-09-10T00:52:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5600.108320 secs 2023-09-10T00:52:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:49 smithi170 ceph-mon[102832]: pgmap v3153: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T00:52:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5600.108320 secs 2023-09-10T00:52:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:49 smithi082 ceph-mon[98694]: pgmap v3153: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T00:52:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5600.108320 secs 2023-09-10T00:52:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:49 smithi098 ceph-mon[103124]: pgmap v3153: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T00:52:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5600.821126 secs 2023-09-10T00:52:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5600.821126 secs 2023-09-10T00:52:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5600.821126 secs 2023-09-10T00:52:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:51 smithi170 ceph-mon[102832]: pgmap v3154: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T00:52:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:51 smithi082 ceph-mon[98694]: pgmap v3154: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T00:52:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:51 smithi098 ceph-mon[103124]: pgmap v3154: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T00:52:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:52:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:52:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:52:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:52:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:52:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:52:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:52:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:52:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:52:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5603.575029 secs 2023-09-10T00:52:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:53 smithi170 ceph-mon[102832]: pgmap v3155: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 146 KiB/s wr, 16 op/s 2023-09-10T00:52:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5603.575029 secs 2023-09-10T00:52:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:53 smithi082 ceph-mon[98694]: pgmap v3155: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 146 KiB/s wr, 16 op/s 2023-09-10T00:52:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5603.575029 secs 2023-09-10T00:52:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:53 smithi098 ceph-mon[103124]: pgmap v3155: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 146 KiB/s wr, 16 op/s 2023-09-10T00:52:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5605.108379 secs 2023-09-10T00:52:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5605.108379 secs 2023-09-10T00:52:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5605.108379 secs 2023-09-10T00:52:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5605.821191 secs 2023-09-10T00:52:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:55 smithi170 ceph-mon[102832]: pgmap v3156: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T00:52:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5605.821191 secs 2023-09-10T00:52:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:55 smithi082 ceph-mon[98694]: pgmap v3156: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T00:52:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5605.821191 secs 2023-09-10T00:52:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:55 smithi098 ceph-mon[103124]: pgmap v3156: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T00:52:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:57 smithi170 ceph-mon[102832]: pgmap v3157: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 120 KiB/s wr, 16 op/s 2023-09-10T00:52:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:57 smithi082 ceph-mon[98694]: pgmap v3157: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 120 KiB/s wr, 16 op/s 2023-09-10T00:52:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:57 smithi098 ceph-mon[103124]: pgmap v3157: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 120 KiB/s wr, 16 op/s 2023-09-10T00:52:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5608.575088 secs 2023-09-10T00:52:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5608.575088 secs 2023-09-10T00:52:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5608.575088 secs 2023-09-10T00:52:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5610.108445 secs 2023-09-10T00:52:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:52:59 smithi170 ceph-mon[102832]: pgmap v3158: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T00:52:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5610.108445 secs 2023-09-10T00:52:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:52:59 smithi082 ceph-mon[98694]: pgmap v3158: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T00:52:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5610.108445 secs 2023-09-10T00:52:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:52:59 smithi098 ceph-mon[103124]: pgmap v3158: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T00:53:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5610.821253 secs 2023-09-10T00:53:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5610.821253 secs 2023-09-10T00:53:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5610.821253 secs 2023-09-10T00:53:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:01 smithi170 ceph-mon[102832]: pgmap v3159: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T00:53:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:01 smithi082 ceph-mon[98694]: pgmap v3159: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T00:53:01.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:01 smithi098 ceph-mon[103124]: pgmap v3159: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T00:53:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5613.575149 secs 2023-09-10T00:53:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:03 smithi170 ceph-mon[102832]: pgmap v3160: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 56 KiB/s wr, 15 op/s 2023-09-10T00:53:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5613.575149 secs 2023-09-10T00:53:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:03 smithi082 ceph-mon[98694]: pgmap v3160: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 56 KiB/s wr, 15 op/s 2023-09-10T00:53:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5613.575149 secs 2023-09-10T00:53:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:03 smithi098 ceph-mon[103124]: pgmap v3160: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 56 KiB/s wr, 15 op/s 2023-09-10T00:53:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5615.108560 secs 2023-09-10T00:53:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5615.108560 secs 2023-09-10T00:53:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5615.108560 secs 2023-09-10T00:53:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5615.821318 secs 2023-09-10T00:53:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:05 smithi170 ceph-mon[102832]: pgmap v3161: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 KiB/s wr, 10 op/s 2023-09-10T00:53:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5615.821318 secs 2023-09-10T00:53:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:05 smithi082 ceph-mon[98694]: pgmap v3161: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 KiB/s wr, 10 op/s 2023-09-10T00:53:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5615.821318 secs 2023-09-10T00:53:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:05 smithi098 ceph-mon[103124]: pgmap v3161: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 KiB/s wr, 10 op/s 2023-09-10T00:53:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:07 smithi170 ceph-mon[102832]: pgmap v3162: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T00:53:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:07 smithi082 ceph-mon[98694]: pgmap v3162: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T00:53:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:07 smithi098 ceph-mon[103124]: pgmap v3162: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T00:53:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5618.575246 secs 2023-09-10T00:53:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:08 smithi170 ceph-mon[102832]: osdmap e269: 12 total, 12 up, 12 in 2023-09-10T00:53:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5618.575246 secs 2023-09-10T00:53:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:08 smithi082 ceph-mon[98694]: osdmap e269: 12 total, 12 up, 12 in 2023-09-10T00:53:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5618.575246 secs 2023-09-10T00:53:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:08 smithi098 ceph-mon[103124]: osdmap e269: 12 total, 12 up, 12 in 2023-09-10T00:53:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5620.108618 secs 2023-09-10T00:53:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:09 smithi170 ceph-mon[102832]: pgmap v3164: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T00:53:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5620.108618 secs 2023-09-10T00:53:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:09 smithi082 ceph-mon[98694]: pgmap v3164: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T00:53:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5620.108618 secs 2023-09-10T00:53:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:09 smithi098 ceph-mon[103124]: pgmap v3164: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T00:53:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5620.821380 secs 2023-09-10T00:53:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5620.821380 secs 2023-09-10T00:53:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5620.821380 secs 2023-09-10T00:53:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:11 smithi170 ceph-mon[102832]: pgmap v3165: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T00:53:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:11 smithi082 ceph-mon[98694]: pgmap v3165: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T00:53:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:11 smithi098 ceph-mon[103124]: pgmap v3165: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T00:53:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5623.575335 secs 2023-09-10T00:53:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:13 smithi170 ceph-mon[102832]: pgmap v3166: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T00:53:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5623.575335 secs 2023-09-10T00:53:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:13 smithi082 ceph-mon[98694]: pgmap v3166: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T00:53:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5623.575335 secs 2023-09-10T00:53:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:13 smithi098 ceph-mon[103124]: pgmap v3166: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T00:53:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5625.108678 secs 2023-09-10T00:53:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5625.108678 secs 2023-09-10T00:53:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5625.108678 secs 2023-09-10T00:53:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5625.821442 secs 2023-09-10T00:53:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:15 smithi170 ceph-mon[102832]: pgmap v3167: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T00:53:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:15 smithi170 ceph-mon[102832]: osdmap e270: 12 total, 12 up, 12 in 2023-09-10T00:53:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5625.821442 secs 2023-09-10T00:53:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:15 smithi082 ceph-mon[98694]: pgmap v3167: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T00:53:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:15 smithi082 ceph-mon[98694]: osdmap e270: 12 total, 12 up, 12 in 2023-09-10T00:53:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5625.821442 secs 2023-09-10T00:53:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:15 smithi098 ceph-mon[103124]: pgmap v3167: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T00:53:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:15 smithi098 ceph-mon[103124]: osdmap e270: 12 total, 12 up, 12 in 2023-09-10T00:53:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:17 smithi170 ceph-mon[102832]: pgmap v3169: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 52 KiB/s wr, 13 op/s 2023-09-10T00:53:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:17 smithi082 ceph-mon[98694]: pgmap v3169: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 52 KiB/s wr, 13 op/s 2023-09-10T00:53:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:17 smithi098 ceph-mon[103124]: pgmap v3169: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 52 KiB/s wr, 13 op/s 2023-09-10T00:53:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5630.108738 secs 2023-09-10T00:53:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:19 smithi082 ceph-mon[98694]: pgmap v3170: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T00:53:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:53:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5630.108738 secs 2023-09-10T00:53:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:19 smithi098 ceph-mon[103124]: pgmap v3170: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T00:53:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:53:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5630.108738 secs 2023-09-10T00:53:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:19 smithi170 ceph-mon[102832]: pgmap v3170: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T00:53:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:53:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5630.821482 secs 2023-09-10T00:53:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:53:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:53:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:53:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5630.821482 secs 2023-09-10T00:53:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:53:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:53:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:53:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5630.821482 secs 2023-09-10T00:53:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:53:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:53:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:53:21.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:21 smithi082 ceph-mon[98694]: pgmap v3171: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T00:53:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:21 smithi098 ceph-mon[103124]: pgmap v3171: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T00:53:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:21 smithi170 ceph-mon[102832]: pgmap v3171: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T00:53:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:23 smithi082 ceph-mon[98694]: pgmap v3172: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T00:53:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:23 smithi098 ceph-mon[103124]: pgmap v3172: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T00:53:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:23 smithi170 ceph-mon[102832]: pgmap v3172: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T00:53:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5635.108797 secs 2023-09-10T00:53:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5635.108797 secs 2023-09-10T00:53:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5635.108797 secs 2023-09-10T00:53:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5635.821546 secs 2023-09-10T00:53:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:25 smithi082 ceph-mon[98694]: pgmap v3173: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T00:53:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5635.821546 secs 2023-09-10T00:53:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:25 smithi098 ceph-mon[103124]: pgmap v3173: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T00:53:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5635.821546 secs 2023-09-10T00:53:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:25 smithi170 ceph-mon[102832]: pgmap v3173: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T00:53:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:27 smithi082 ceph-mon[98694]: pgmap v3174: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 122 KiB/s wr, 16 op/s 2023-09-10T00:53:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:27 smithi098 ceph-mon[103124]: pgmap v3174: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 122 KiB/s wr, 16 op/s 2023-09-10T00:53:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:27 smithi170 ceph-mon[102832]: pgmap v3174: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 122 KiB/s wr, 16 op/s 2023-09-10T00:53:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5640.108857 secs 2023-09-10T00:53:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:29 smithi098 ceph-mon[103124]: pgmap v3175: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T00:53:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5640.108857 secs 2023-09-10T00:53:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:29 smithi170 ceph-mon[102832]: pgmap v3175: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T00:53:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5640.108857 secs 2023-09-10T00:53:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:29 smithi082 ceph-mon[98694]: pgmap v3175: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T00:53:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5640.821610 secs 2023-09-10T00:53:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5640.821610 secs 2023-09-10T00:53:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5640.821610 secs 2023-09-10T00:53:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:31 smithi098 ceph-mon[103124]: pgmap v3176: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T00:53:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:53:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:31 smithi170 ceph-mon[102832]: pgmap v3176: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T00:53:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:53:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:31 smithi082 ceph-mon[98694]: pgmap v3176: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T00:53:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:53:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:53:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:53:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:53:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:53:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:53:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:53:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:53:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:53:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:53:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:33 smithi098 ceph-mon[103124]: pgmap v3177: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 180 KiB/s wr, 16 op/s 2023-09-10T00:53:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:33 smithi170 ceph-mon[102832]: pgmap v3177: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 180 KiB/s wr, 16 op/s 2023-09-10T00:53:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:33 smithi082 ceph-mon[98694]: pgmap v3177: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 180 KiB/s wr, 16 op/s 2023-09-10T00:53:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5645.108940 secs 2023-09-10T00:53:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5645.108940 secs 2023-09-10T00:53:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5645.108940 secs 2023-09-10T00:53:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5645.821675 secs 2023-09-10T00:53:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:35 smithi098 ceph-mon[103124]: pgmap v3178: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T00:53:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5645.821675 secs 2023-09-10T00:53:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:35 smithi170 ceph-mon[102832]: pgmap v3178: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T00:53:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5645.821675 secs 2023-09-10T00:53:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:35 smithi082 ceph-mon[98694]: pgmap v3178: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T00:53:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:37 smithi098 ceph-mon[103124]: pgmap v3179: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 188 KiB/s wr, 18 op/s 2023-09-10T00:53:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:37 smithi170 ceph-mon[102832]: pgmap v3179: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 188 KiB/s wr, 18 op/s 2023-09-10T00:53:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:37 smithi082 ceph-mon[98694]: pgmap v3179: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 188 KiB/s wr, 18 op/s 2023-09-10T00:53:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5650.108997 secs 2023-09-10T00:53:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:39 smithi098 ceph-mon[103124]: pgmap v3180: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 149 KiB/s wr, 13 op/s 2023-09-10T00:53:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5650.108997 secs 2023-09-10T00:53:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:39 smithi170 ceph-mon[102832]: pgmap v3180: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 149 KiB/s wr, 13 op/s 2023-09-10T00:53:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5650.108997 secs 2023-09-10T00:53:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:39 smithi082 ceph-mon[98694]: pgmap v3180: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 149 KiB/s wr, 13 op/s 2023-09-10T00:53:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5650.821736 secs 2023-09-10T00:53:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5650.821736 secs 2023-09-10T00:53:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5650.821736 secs 2023-09-10T00:53:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:41 smithi098 ceph-mon[103124]: pgmap v3181: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 149 KiB/s wr, 13 op/s 2023-09-10T00:53:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:41 smithi170 ceph-mon[102832]: pgmap v3181: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 149 KiB/s wr, 13 op/s 2023-09-10T00:53:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:41 smithi082 ceph-mon[98694]: pgmap v3181: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 149 KiB/s wr, 13 op/s 2023-09-10T00:53:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:43 smithi098 ceph-mon[103124]: pgmap v3182: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 253 KiB/s wr, 19 op/s 2023-09-10T00:53:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:43 smithi170 ceph-mon[102832]: pgmap v3182: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 253 KiB/s wr, 19 op/s 2023-09-10T00:53:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:43 smithi082 ceph-mon[98694]: pgmap v3182: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 253 KiB/s wr, 19 op/s 2023-09-10T00:53:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5655.109084 secs 2023-09-10T00:53:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5655.109084 secs 2023-09-10T00:53:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5655.109084 secs 2023-09-10T00:53:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5655.821802 secs 2023-09-10T00:53:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:45 smithi098 ceph-mon[103124]: pgmap v3183: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 12 op/s 2023-09-10T00:53:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5655.821802 secs 2023-09-10T00:53:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:45 smithi170 ceph-mon[102832]: pgmap v3183: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 12 op/s 2023-09-10T00:53:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5655.821802 secs 2023-09-10T00:53:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:45 smithi082 ceph-mon[98694]: pgmap v3183: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 12 op/s 2023-09-10T00:53:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:47 smithi098 ceph-mon[103124]: pgmap v3184: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 217 KiB/s wr, 18 op/s 2023-09-10T00:53:47.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:47 smithi170 ceph-mon[102832]: pgmap v3184: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 217 KiB/s wr, 18 op/s 2023-09-10T00:53:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:47 smithi082 ceph-mon[98694]: pgmap v3184: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 217 KiB/s wr, 18 op/s 2023-09-10T00:53:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5658.575758 secs 2023-09-10T00:53:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5658.575758 secs 2023-09-10T00:53:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5658.575758 secs 2023-09-10T00:53:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5660.109152 secs 2023-09-10T00:53:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:49 smithi170 ceph-mon[102832]: pgmap v3185: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T00:53:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5660.109152 secs 2023-09-10T00:53:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:49 smithi082 ceph-mon[98694]: pgmap v3185: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T00:53:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5660.109152 secs 2023-09-10T00:53:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:49 smithi098 ceph-mon[103124]: pgmap v3185: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T00:53:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5660.821863 secs 2023-09-10T00:53:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5660.821863 secs 2023-09-10T00:53:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5660.821863 secs 2023-09-10T00:53:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:51 smithi170 ceph-mon[102832]: pgmap v3186: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T00:53:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:51 smithi082 ceph-mon[98694]: pgmap v3186: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T00:53:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:51 smithi098 ceph-mon[103124]: pgmap v3186: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T00:53:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:53:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:53:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:53:52.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:53:52.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:53:52.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:53:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:53:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:53:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:53:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5663.575791 secs 2023-09-10T00:53:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:53 smithi170 ceph-mon[102832]: pgmap v3187: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 218 KiB/s wr, 17 op/s 2023-09-10T00:53:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5663.575791 secs 2023-09-10T00:53:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:53 smithi082 ceph-mon[98694]: pgmap v3187: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 218 KiB/s wr, 17 op/s 2023-09-10T00:53:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5663.575791 secs 2023-09-10T00:53:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:53 smithi098 ceph-mon[103124]: pgmap v3187: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 218 KiB/s wr, 17 op/s 2023-09-10T00:53:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5665.109207 secs 2023-09-10T00:53:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5665.109207 secs 2023-09-10T00:53:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5665.109207 secs 2023-09-10T00:53:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5665.821928 secs 2023-09-10T00:53:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:55 smithi170 ceph-mon[102832]: pgmap v3188: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T00:53:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5665.821928 secs 2023-09-10T00:53:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:55 smithi082 ceph-mon[98694]: pgmap v3188: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T00:53:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5665.821928 secs 2023-09-10T00:53:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:55 smithi098 ceph-mon[103124]: pgmap v3188: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T00:53:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:57 smithi170 ceph-mon[102832]: pgmap v3189: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 139 KiB/s wr, 16 op/s 2023-09-10T00:53:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:57 smithi082 ceph-mon[98694]: pgmap v3189: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 139 KiB/s wr, 16 op/s 2023-09-10T00:53:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:57 smithi098 ceph-mon[103124]: pgmap v3189: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 139 KiB/s wr, 16 op/s 2023-09-10T00:53:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5668.575851 secs 2023-09-10T00:53:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5668.575851 secs 2023-09-10T00:53:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5668.575851 secs 2023-09-10T00:53:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5670.109264 secs 2023-09-10T00:53:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:53:59 smithi170 ceph-mon[102832]: pgmap v3190: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T00:53:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5670.109264 secs 2023-09-10T00:53:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:53:59 smithi082 ceph-mon[98694]: pgmap v3190: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T00:53:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5670.109264 secs 2023-09-10T00:53:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:53:59 smithi098 ceph-mon[103124]: pgmap v3190: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T00:54:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5670.821990 secs 2023-09-10T00:54:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5670.821990 secs 2023-09-10T00:54:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5670.821990 secs 2023-09-10T00:54:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:01 smithi170 ceph-mon[102832]: pgmap v3191: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T00:54:01.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:01 smithi082 ceph-mon[98694]: pgmap v3191: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T00:54:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:01 smithi098 ceph-mon[103124]: pgmap v3191: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T00:54:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5673.575910 secs 2023-09-10T00:54:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:03 smithi170 ceph-mon[102832]: pgmap v3192: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 71 KiB/s wr, 14 op/s 2023-09-10T00:54:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5673.575910 secs 2023-09-10T00:54:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:03 smithi082 ceph-mon[98694]: pgmap v3192: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 71 KiB/s wr, 14 op/s 2023-09-10T00:54:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5673.575910 secs 2023-09-10T00:54:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:03 smithi098 ceph-mon[103124]: pgmap v3192: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 71 KiB/s wr, 14 op/s 2023-09-10T00:54:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5675.109351 secs 2023-09-10T00:54:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5675.109351 secs 2023-09-10T00:54:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5675.109351 secs 2023-09-10T00:54:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5675.822055 secs 2023-09-10T00:54:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:05 smithi170 ceph-mon[102832]: pgmap v3193: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 27 KiB/s wr, 9 op/s 2023-09-10T00:54:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5675.822055 secs 2023-09-10T00:54:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:05 smithi082 ceph-mon[98694]: pgmap v3193: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 27 KiB/s wr, 9 op/s 2023-09-10T00:54:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5675.822055 secs 2023-09-10T00:54:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:05 smithi098 ceph-mon[103124]: pgmap v3193: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 27 KiB/s wr, 9 op/s 2023-09-10T00:54:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:07 smithi170 ceph-mon[102832]: pgmap v3194: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T00:54:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:07 smithi082 ceph-mon[98694]: pgmap v3194: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T00:54:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:07 smithi098 ceph-mon[103124]: pgmap v3194: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T00:54:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5678.575966 secs 2023-09-10T00:54:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:08 smithi170 ceph-mon[102832]: osdmap e271: 12 total, 12 up, 12 in 2023-09-10T00:54:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5678.575966 secs 2023-09-10T00:54:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:08 smithi082 ceph-mon[98694]: osdmap e271: 12 total, 12 up, 12 in 2023-09-10T00:54:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5678.575966 secs 2023-09-10T00:54:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:08 smithi098 ceph-mon[103124]: osdmap e271: 12 total, 12 up, 12 in 2023-09-10T00:54:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5680.109430 secs 2023-09-10T00:54:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:09 smithi170 ceph-mon[102832]: pgmap v3196: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 121 KiB/s wr, 29 op/s 2023-09-10T00:54:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5680.109430 secs 2023-09-10T00:54:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:09 smithi082 ceph-mon[98694]: pgmap v3196: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 121 KiB/s wr, 29 op/s 2023-09-10T00:54:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5680.109430 secs 2023-09-10T00:54:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:09 smithi098 ceph-mon[103124]: pgmap v3196: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 121 KiB/s wr, 29 op/s 2023-09-10T00:54:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5680.822117 secs 2023-09-10T00:54:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5680.822117 secs 2023-09-10T00:54:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5680.822117 secs 2023-09-10T00:54:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:11 smithi170 ceph-mon[102832]: pgmap v3197: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 121 KiB/s wr, 29 op/s 2023-09-10T00:54:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:11 smithi082 ceph-mon[98694]: pgmap v3197: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 121 KiB/s wr, 29 op/s 2023-09-10T00:54:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:11 smithi098 ceph-mon[103124]: pgmap v3197: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 121 KiB/s wr, 29 op/s 2023-09-10T00:54:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5683.576027 secs 2023-09-10T00:54:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:13 smithi170 ceph-mon[102832]: pgmap v3198: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 29 op/s 2023-09-10T00:54:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5683.576027 secs 2023-09-10T00:54:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:13 smithi082 ceph-mon[98694]: pgmap v3198: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 29 op/s 2023-09-10T00:54:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5683.576027 secs 2023-09-10T00:54:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:13 smithi098 ceph-mon[103124]: pgmap v3198: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 29 op/s 2023-09-10T00:54:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5685.109490 secs 2023-09-10T00:54:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5685.109490 secs 2023-09-10T00:54:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5685.109490 secs 2023-09-10T00:54:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5685.822180 secs 2023-09-10T00:54:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:15 smithi170 ceph-mon[102832]: pgmap v3199: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 29 op/s 2023-09-10T00:54:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:15 smithi170 ceph-mon[102832]: osdmap e272: 12 total, 12 up, 12 in 2023-09-10T00:54:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5685.822180 secs 2023-09-10T00:54:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:15 smithi082 ceph-mon[98694]: pgmap v3199: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 29 op/s 2023-09-10T00:54:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:15 smithi082 ceph-mon[98694]: osdmap e272: 12 total, 12 up, 12 in 2023-09-10T00:54:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5685.822180 secs 2023-09-10T00:54:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:15 smithi098 ceph-mon[103124]: pgmap v3199: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 29 op/s 2023-09-10T00:54:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:15 smithi098 ceph-mon[103124]: osdmap e272: 12 total, 12 up, 12 in 2023-09-10T00:54:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:17 smithi170 ceph-mon[102832]: pgmap v3201: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 41 KiB/s wr, 13 op/s 2023-09-10T00:54:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:17 smithi082 ceph-mon[98694]: pgmap v3201: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 41 KiB/s wr, 13 op/s 2023-09-10T00:54:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:17 smithi098 ceph-mon[103124]: pgmap v3201: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 41 KiB/s wr, 13 op/s 2023-09-10T00:54:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5688.576086 secs 2023-09-10T00:54:18.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5688.576086 secs 2023-09-10T00:54:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5688.576086 secs 2023-09-10T00:54:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5690.109549 secs 2023-09-10T00:54:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:19 smithi170 ceph-mon[102832]: pgmap v3202: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T00:54:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5690.109549 secs 2023-09-10T00:54:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:19 smithi082 ceph-mon[98694]: pgmap v3202: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T00:54:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5690.109549 secs 2023-09-10T00:54:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:19 smithi098 ceph-mon[103124]: pgmap v3202: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T00:54:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5690.822246 secs 2023-09-10T00:54:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5690.822246 secs 2023-09-10T00:54:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5690.822246 secs 2023-09-10T00:54:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:21 smithi170 ceph-mon[102832]: pgmap v3203: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T00:54:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:21 smithi082 ceph-mon[98694]: pgmap v3203: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T00:54:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:21 smithi098 ceph-mon[103124]: pgmap v3203: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T00:54:23.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:23 smithi170 ceph-mon[102832]: pgmap v3204: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 129 KiB/s wr, 13 op/s 2023-09-10T00:54:23.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:54:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:23 smithi082 ceph-mon[98694]: pgmap v3204: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 129 KiB/s wr, 13 op/s 2023-09-10T00:54:23.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:54:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:23 smithi098 ceph-mon[103124]: pgmap v3204: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 129 KiB/s wr, 13 op/s 2023-09-10T00:54:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:54:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5695.109608 secs 2023-09-10T00:54:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:54:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:54:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:54:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5695.109608 secs 2023-09-10T00:54:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:54:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:54:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:54:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5695.109608 secs 2023-09-10T00:54:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:54:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:54:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:54:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5695.822308 secs 2023-09-10T00:54:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:25 smithi170 ceph-mon[102832]: pgmap v3205: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 129 KiB/s wr, 13 op/s 2023-09-10T00:54:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5695.822308 secs 2023-09-10T00:54:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:25 smithi082 ceph-mon[98694]: pgmap v3205: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 129 KiB/s wr, 13 op/s 2023-09-10T00:54:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5695.822308 secs 2023-09-10T00:54:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:25 smithi098 ceph-mon[103124]: pgmap v3205: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 129 KiB/s wr, 13 op/s 2023-09-10T00:54:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:27 smithi170 ceph-mon[102832]: pgmap v3206: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 162 KiB/s wr, 19 op/s 2023-09-10T00:54:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:27 smithi082 ceph-mon[98694]: pgmap v3206: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 162 KiB/s wr, 19 op/s 2023-09-10T00:54:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:27 smithi098 ceph-mon[103124]: pgmap v3206: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 162 KiB/s wr, 19 op/s 2023-09-10T00:54:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5700.109671 secs 2023-09-10T00:54:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:29 smithi170 ceph-mon[102832]: pgmap v3207: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T00:54:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5700.109671 secs 2023-09-10T00:54:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:29 smithi082 ceph-mon[98694]: pgmap v3207: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T00:54:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5700.109671 secs 2023-09-10T00:54:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:29 smithi098 ceph-mon[103124]: pgmap v3207: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T00:54:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5700.822371 secs 2023-09-10T00:54:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5700.822371 secs 2023-09-10T00:54:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5700.822371 secs 2023-09-10T00:54:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:31 smithi170 ceph-mon[102832]: pgmap v3208: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T00:54:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:54:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:31 smithi082 ceph-mon[98694]: pgmap v3208: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T00:54:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:54:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:31 smithi098 ceph-mon[103124]: pgmap v3208: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T00:54:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:54:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:54:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:54:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:54:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:33 smithi170 ceph-mon[102832]: pgmap v3209: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 270 KiB/s wr, 19 op/s 2023-09-10T00:54:33.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:54:33.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:54:33.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:54:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:33 smithi082 ceph-mon[98694]: pgmap v3209: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 270 KiB/s wr, 19 op/s 2023-09-10T00:54:33.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:54:33.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:54:33.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:54:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:33 smithi098 ceph-mon[103124]: pgmap v3209: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 270 KiB/s wr, 19 op/s 2023-09-10T00:54:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:54:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:54:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:54:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5705.109727 secs 2023-09-10T00:54:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5705.109727 secs 2023-09-10T00:54:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5705.109727 secs 2023-09-10T00:54:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5705.822436 secs 2023-09-10T00:54:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:35 smithi170 ceph-mon[102832]: pgmap v3210: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 191 KiB/s wr, 13 op/s 2023-09-10T00:54:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5705.822436 secs 2023-09-10T00:54:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:35 smithi082 ceph-mon[98694]: pgmap v3210: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 191 KiB/s wr, 13 op/s 2023-09-10T00:54:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5705.822436 secs 2023-09-10T00:54:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:35 smithi098 ceph-mon[103124]: pgmap v3210: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 191 KiB/s wr, 13 op/s 2023-09-10T00:54:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:37 smithi170 ceph-mon[102832]: pgmap v3211: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 290 KiB/s wr, 23 op/s 2023-09-10T00:54:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:37 smithi082 ceph-mon[98694]: pgmap v3211: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 290 KiB/s wr, 23 op/s 2023-09-10T00:54:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:37 smithi098 ceph-mon[103124]: pgmap v3211: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 290 KiB/s wr, 23 op/s 2023-09-10T00:54:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5708.576328 secs 2023-09-10T00:54:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5708.576328 secs 2023-09-10T00:54:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5708.576328 secs 2023-09-10T00:54:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5710.109790 secs 2023-09-10T00:54:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:39 smithi170 ceph-mon[102832]: pgmap v3212: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 250 KiB/s wr, 17 op/s 2023-09-10T00:54:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5710.109790 secs 2023-09-10T00:54:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:39 smithi082 ceph-mon[98694]: pgmap v3212: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 250 KiB/s wr, 17 op/s 2023-09-10T00:54:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5710.109790 secs 2023-09-10T00:54:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:39 smithi098 ceph-mon[103124]: pgmap v3212: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 250 KiB/s wr, 17 op/s 2023-09-10T00:54:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5710.822489 secs 2023-09-10T00:54:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:54:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:54:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:54:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5710.822489 secs 2023-09-10T00:54:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:54:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:54:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:54:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5710.822489 secs 2023-09-10T00:54:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:54:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:54:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:54:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:41 smithi170 ceph-mon[102832]: pgmap v3213: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 250 KiB/s wr, 17 op/s 2023-09-10T00:54:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:41 smithi082 ceph-mon[98694]: pgmap v3213: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 250 KiB/s wr, 17 op/s 2023-09-10T00:54:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:41 smithi098 ceph-mon[103124]: pgmap v3213: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 250 KiB/s wr, 17 op/s 2023-09-10T00:54:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5713.576417 secs 2023-09-10T00:54:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:43 smithi170 ceph-mon[102832]: pgmap v3214: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 329 KiB/s wr, 23 op/s 2023-09-10T00:54:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5713.576417 secs 2023-09-10T00:54:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:43 smithi082 ceph-mon[98694]: pgmap v3214: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 329 KiB/s wr, 23 op/s 2023-09-10T00:54:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5713.576417 secs 2023-09-10T00:54:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:43 smithi098 ceph-mon[103124]: pgmap v3214: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 329 KiB/s wr, 23 op/s 2023-09-10T00:54:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5715.109849 secs 2023-09-10T00:54:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5715.109849 secs 2023-09-10T00:54:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5715.109849 secs 2023-09-10T00:54:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5715.822551 secs 2023-09-10T00:54:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:45 smithi170 ceph-mon[102832]: pgmap v3215: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 178 KiB/s wr, 16 op/s 2023-09-10T00:54:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5715.822551 secs 2023-09-10T00:54:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:45 smithi082 ceph-mon[98694]: pgmap v3215: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 178 KiB/s wr, 16 op/s 2023-09-10T00:54:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5715.822551 secs 2023-09-10T00:54:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:45 smithi098 ceph-mon[103124]: pgmap v3215: 97 pgs: 97 active+clean; 9.1 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 178 KiB/s wr, 16 op/s 2023-09-10T00:54:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:47 smithi082 ceph-mon[98694]: pgmap v3216: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 270 KiB/s wr, 25 op/s 2023-09-10T00:54:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:47 smithi098 ceph-mon[103124]: pgmap v3216: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 270 KiB/s wr, 25 op/s 2023-09-10T00:54:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:47 smithi170 ceph-mon[102832]: pgmap v3216: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 270 KiB/s wr, 25 op/s 2023-09-10T00:54:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5718.576478 secs 2023-09-10T00:54:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5718.576478 secs 2023-09-10T00:54:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5718.576478 secs 2023-09-10T00:54:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5720.109936 secs 2023-09-10T00:54:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:49 smithi082 ceph-mon[98694]: pgmap v3217: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 172 KiB/s wr, 15 op/s 2023-09-10T00:54:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5720.109936 secs 2023-09-10T00:54:49.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:49 smithi098 ceph-mon[103124]: pgmap v3217: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 172 KiB/s wr, 15 op/s 2023-09-10T00:54:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5720.109936 secs 2023-09-10T00:54:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:49 smithi170 ceph-mon[102832]: pgmap v3217: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 172 KiB/s wr, 15 op/s 2023-09-10T00:54:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5720.822615 secs 2023-09-10T00:54:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5720.822615 secs 2023-09-10T00:54:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5720.822615 secs 2023-09-10T00:54:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:51 smithi082 ceph-mon[98694]: pgmap v3218: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 172 KiB/s wr, 15 op/s 2023-09-10T00:54:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:51 smithi098 ceph-mon[103124]: pgmap v3218: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 172 KiB/s wr, 15 op/s 2023-09-10T00:54:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:51 smithi170 ceph-mon[102832]: pgmap v3218: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 172 KiB/s wr, 15 op/s 2023-09-10T00:54:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:52 smithi082 ceph-mon[98694]: pgmap v3219: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 217 KiB/s wr, 23 op/s 2023-09-10T00:54:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:52 smithi098 ceph-mon[103124]: pgmap v3219: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 217 KiB/s wr, 23 op/s 2023-09-10T00:54:52.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:52 smithi170 ceph-mon[102832]: pgmap v3219: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 217 KiB/s wr, 23 op/s 2023-09-10T00:54:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5723.576539 secs 2023-09-10T00:54:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5723.576539 secs 2023-09-10T00:54:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5723.576539 secs 2023-09-10T00:54:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5725.109994 secs 2023-09-10T00:54:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:54 smithi082 ceph-mon[98694]: pgmap v3220: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 138 KiB/s wr, 16 op/s 2023-09-10T00:54:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5725.109994 secs 2023-09-10T00:54:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:54 smithi098 ceph-mon[103124]: pgmap v3220: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 138 KiB/s wr, 16 op/s 2023-09-10T00:54:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5725.109994 secs 2023-09-10T00:54:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:54 smithi170 ceph-mon[102832]: pgmap v3220: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 138 KiB/s wr, 16 op/s 2023-09-10T00:54:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5725.822701 secs 2023-09-10T00:54:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5725.822701 secs 2023-09-10T00:54:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5725.822701 secs 2023-09-10T00:54:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:56 smithi098 ceph-mon[103124]: pgmap v3221: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 199 KiB/s wr, 25 op/s 2023-09-10T00:54:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:56 smithi170 ceph-mon[102832]: pgmap v3221: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 199 KiB/s wr, 25 op/s 2023-09-10T00:54:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:56 smithi082 ceph-mon[98694]: pgmap v3221: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 199 KiB/s wr, 25 op/s 2023-09-10T00:54:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5728.576617 secs 2023-09-10T00:54:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5728.576617 secs 2023-09-10T00:54:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5728.576617 secs 2023-09-10T00:54:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5730.110051 secs 2023-09-10T00:54:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:58 smithi170 ceph-mon[102832]: pgmap v3222: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 106 KiB/s wr, 16 op/s 2023-09-10T00:54:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5730.110051 secs 2023-09-10T00:54:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:58 smithi082 ceph-mon[98694]: pgmap v3222: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 106 KiB/s wr, 16 op/s 2023-09-10T00:54:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5730.110051 secs 2023-09-10T00:54:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:58 smithi098 ceph-mon[103124]: pgmap v3222: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 106 KiB/s wr, 16 op/s 2023-09-10T00:55:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:54:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5730.822768 secs 2023-09-10T00:55:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:54:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5730.822768 secs 2023-09-10T00:55:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:54:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5730.822768 secs 2023-09-10T00:55:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:01 smithi098 ceph-mon[103124]: pgmap v3223: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 106 KiB/s wr, 16 op/s 2023-09-10T00:55:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:01 smithi170 ceph-mon[102832]: pgmap v3223: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 106 KiB/s wr, 16 op/s 2023-09-10T00:55:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:01 smithi082 ceph-mon[98694]: pgmap v3223: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 106 KiB/s wr, 16 op/s 2023-09-10T00:55:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5733.576708 secs 2023-09-10T00:55:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:03 smithi170 ceph-mon[102832]: pgmap v3224: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 172 KiB/s wr, 22 op/s 2023-09-10T00:55:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5733.576708 secs 2023-09-10T00:55:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:03 smithi082 ceph-mon[98694]: pgmap v3224: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 172 KiB/s wr, 22 op/s 2023-09-10T00:55:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5733.576708 secs 2023-09-10T00:55:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:03 smithi098 ceph-mon[103124]: pgmap v3224: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 172 KiB/s wr, 22 op/s 2023-09-10T00:55:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5735.110115 secs 2023-09-10T00:55:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5735.110115 secs 2023-09-10T00:55:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5735.110115 secs 2023-09-10T00:55:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5735.822830 secs 2023-09-10T00:55:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:05 smithi170 ceph-mon[102832]: pgmap v3225: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 127 KiB/s wr, 15 op/s 2023-09-10T00:55:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5735.822830 secs 2023-09-10T00:55:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:05 smithi082 ceph-mon[98694]: pgmap v3225: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 127 KiB/s wr, 15 op/s 2023-09-10T00:55:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5735.822830 secs 2023-09-10T00:55:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:05 smithi098 ceph-mon[103124]: pgmap v3225: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 127 KiB/s wr, 15 op/s 2023-09-10T00:55:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:07 smithi170 ceph-mon[102832]: pgmap v3226: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 199 KiB/s wr, 36 op/s 2023-09-10T00:55:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:07 smithi082 ceph-mon[98694]: pgmap v3226: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 199 KiB/s wr, 36 op/s 2023-09-10T00:55:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:07 smithi098 ceph-mon[103124]: pgmap v3226: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 199 KiB/s wr, 36 op/s 2023-09-10T00:55:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5738.576771 secs 2023-09-10T00:55:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:08 smithi170 ceph-mon[102832]: osdmap e273: 12 total, 12 up, 12 in 2023-09-10T00:55:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5738.576771 secs 2023-09-10T00:55:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:08 smithi082 ceph-mon[98694]: osdmap e273: 12 total, 12 up, 12 in 2023-09-10T00:55:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5738.576771 secs 2023-09-10T00:55:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:08 smithi098 ceph-mon[103124]: osdmap e273: 12 total, 12 up, 12 in 2023-09-10T00:55:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5740.110172 secs 2023-09-10T00:55:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:09 smithi170 ceph-mon[102832]: pgmap v3228: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 166 KiB/s wr, 33 op/s 2023-09-10T00:55:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5740.110172 secs 2023-09-10T00:55:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:09 smithi082 ceph-mon[98694]: pgmap v3228: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 166 KiB/s wr, 33 op/s 2023-09-10T00:55:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5740.110172 secs 2023-09-10T00:55:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:09 smithi098 ceph-mon[103124]: pgmap v3228: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 166 KiB/s wr, 33 op/s 2023-09-10T00:55:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5740.822890 secs 2023-09-10T00:55:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5740.822890 secs 2023-09-10T00:55:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5740.822890 secs 2023-09-10T00:55:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:11 smithi170 ceph-mon[102832]: pgmap v3229: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 166 KiB/s wr, 33 op/s 2023-09-10T00:55:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:11 smithi082 ceph-mon[98694]: pgmap v3229: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 166 KiB/s wr, 33 op/s 2023-09-10T00:55:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:11 smithi098 ceph-mon[103124]: pgmap v3229: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 166 KiB/s wr, 33 op/s 2023-09-10T00:55:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5743.576832 secs 2023-09-10T00:55:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:13 smithi170 ceph-mon[102832]: pgmap v3230: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 125 KiB/s wr, 31 op/s 2023-09-10T00:55:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5743.576832 secs 2023-09-10T00:55:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:13 smithi082 ceph-mon[98694]: pgmap v3230: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 125 KiB/s wr, 31 op/s 2023-09-10T00:55:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5743.576832 secs 2023-09-10T00:55:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:13 smithi098 ceph-mon[103124]: pgmap v3230: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 125 KiB/s wr, 31 op/s 2023-09-10T00:55:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5745.110230 secs 2023-09-10T00:55:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5745.110230 secs 2023-09-10T00:55:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5745.110230 secs 2023-09-10T00:55:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5745.822953 secs 2023-09-10T00:55:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:15 smithi170 ceph-mon[102832]: pgmap v3231: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 125 KiB/s wr, 31 op/s 2023-09-10T00:55:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:15 smithi170 ceph-mon[102832]: osdmap e274: 12 total, 12 up, 12 in 2023-09-10T00:55:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5745.822953 secs 2023-09-10T00:55:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:15 smithi082 ceph-mon[98694]: pgmap v3231: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 125 KiB/s wr, 31 op/s 2023-09-10T00:55:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:15 smithi082 ceph-mon[98694]: osdmap e274: 12 total, 12 up, 12 in 2023-09-10T00:55:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5745.822953 secs 2023-09-10T00:55:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:15 smithi098 ceph-mon[103124]: pgmap v3231: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 125 KiB/s wr, 31 op/s 2023-09-10T00:55:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:15 smithi098 ceph-mon[103124]: osdmap e274: 12 total, 12 up, 12 in 2023-09-10T00:55:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:17 smithi170 ceph-mon[102832]: pgmap v3233: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 13 op/s 2023-09-10T00:55:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:17 smithi082 ceph-mon[98694]: pgmap v3233: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 13 op/s 2023-09-10T00:55:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:17 smithi098 ceph-mon[103124]: pgmap v3233: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 13 op/s 2023-09-10T00:55:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5750.110295 secs 2023-09-10T00:55:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:19 smithi170 ceph-mon[102832]: pgmap v3234: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T00:55:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:19 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:55:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5750.110295 secs 2023-09-10T00:55:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:19 smithi082 ceph-mon[98694]: pgmap v3234: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T00:55:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:19 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:55:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5750.110295 secs 2023-09-10T00:55:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:19 smithi098 ceph-mon[103124]: pgmap v3234: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T00:55:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:19 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:55:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5750.823018 secs 2023-09-10T00:55:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:20 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:55:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:55:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5750.823018 secs 2023-09-10T00:55:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:20 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:55:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:55:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5750.823018 secs 2023-09-10T00:55:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:20 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:55:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:55:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:21 smithi170 ceph-mon[102832]: pgmap v3235: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T00:55:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:21 smithi082 ceph-mon[98694]: pgmap v3235: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T00:55:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:21 smithi098 ceph-mon[103124]: pgmap v3235: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T00:55:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5753.576951 secs 2023-09-10T00:55:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:23 smithi170 ceph-mon[102832]: pgmap v3236: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 62 KiB/s wr, 12 op/s 2023-09-10T00:55:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5753.576951 secs 2023-09-10T00:55:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:23 smithi082 ceph-mon[98694]: pgmap v3236: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 62 KiB/s wr, 12 op/s 2023-09-10T00:55:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5753.576951 secs 2023-09-10T00:55:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:23 smithi098 ceph-mon[103124]: pgmap v3236: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 62 KiB/s wr, 12 op/s 2023-09-10T00:55:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5755.110355 secs 2023-09-10T00:55:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:55:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5755.110355 secs 2023-09-10T00:55:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:55:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5755.110355 secs 2023-09-10T00:55:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:55:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5755.823082 secs 2023-09-10T00:55:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:25 smithi170 ceph-mon[102832]: pgmap v3237: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 62 KiB/s wr, 12 op/s 2023-09-10T00:55:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:25 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:55:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5755.823082 secs 2023-09-10T00:55:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:25 smithi082 ceph-mon[98694]: pgmap v3237: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 62 KiB/s wr, 12 op/s 2023-09-10T00:55:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:25 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:55:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5755.823082 secs 2023-09-10T00:55:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:25 smithi098 ceph-mon[103124]: pgmap v3237: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 62 KiB/s wr, 12 op/s 2023-09-10T00:55:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:25 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:55:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:27 smithi170 ceph-mon[102832]: pgmap v3238: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 100 KiB/s wr, 18 op/s 2023-09-10T00:55:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:27 smithi082 ceph-mon[98694]: pgmap v3238: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 100 KiB/s wr, 18 op/s 2023-09-10T00:55:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:27 smithi098 ceph-mon[103124]: pgmap v3238: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 100 KiB/s wr, 18 op/s 2023-09-10T00:55:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5758.577010 secs 2023-09-10T00:55:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5758.577010 secs 2023-09-10T00:55:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5758.577010 secs 2023-09-10T00:55:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:29 smithi170 ceph-mon[102832]: 3 slow requests, 3 included below; oldest blocked for > 5760.110411 secs 2023-09-10T00:55:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:29 smithi170 ceph-mon[102832]: slow request 5760.110411 seconds old, received at 2023-09-09T23:19:27.740042+0000: client_request(mds.4:47 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently acquired locks 2023-09-10T00:55:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:29 smithi170 ceph-mon[102832]: slow request 5760.110031 seconds old, received at 2023-09-09T23:19:27.740422+0000: client_request(mds.4:48 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently requesting remote authpins 2023-09-10T00:55:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:29 smithi170 ceph-mon[102832]: slow request 5760.107810 seconds old, received at 2023-09-09T23:19:27.742643+0000: client_request(client.24913:3869 unlink #0x10000000007/00000001000000010000000B 2023-09-09T23:19:27.741863+0000 caller_uid=26, caller_gid=26{26,}) currently requesting remote authpins 2023-09-10T00:55:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:29 smithi170 ceph-mon[102832]: pgmap v3239: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T00:55:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:29 smithi082 ceph-mon[98694]: 3 slow requests, 3 included below; oldest blocked for > 5760.110411 secs 2023-09-10T00:55:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:29 smithi082 ceph-mon[98694]: slow request 5760.110411 seconds old, received at 2023-09-09T23:19:27.740042+0000: client_request(mds.4:47 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently acquired locks 2023-09-10T00:55:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:29 smithi082 ceph-mon[98694]: slow request 5760.110031 seconds old, received at 2023-09-09T23:19:27.740422+0000: client_request(mds.4:48 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently requesting remote authpins 2023-09-10T00:55:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:29 smithi082 ceph-mon[98694]: slow request 5760.107810 seconds old, received at 2023-09-09T23:19:27.742643+0000: client_request(client.24913:3869 unlink #0x10000000007/00000001000000010000000B 2023-09-09T23:19:27.741863+0000 caller_uid=26, caller_gid=26{26,}) currently requesting remote authpins 2023-09-10T00:55:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:29 smithi082 ceph-mon[98694]: pgmap v3239: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T00:55:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:29 smithi098 ceph-mon[103124]: 3 slow requests, 3 included below; oldest blocked for > 5760.110411 secs 2023-09-10T00:55:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:29 smithi098 ceph-mon[103124]: slow request 5760.110411 seconds old, received at 2023-09-09T23:19:27.740042+0000: client_request(mds.4:47 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently acquired locks 2023-09-10T00:55:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:29 smithi098 ceph-mon[103124]: slow request 5760.110031 seconds old, received at 2023-09-09T23:19:27.740422+0000: client_request(mds.4:48 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently requesting remote authpins 2023-09-10T00:55:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:29 smithi098 ceph-mon[103124]: slow request 5760.107810 seconds old, received at 2023-09-09T23:19:27.742643+0000: client_request(client.24913:3869 unlink #0x10000000007/00000001000000010000000B 2023-09-09T23:19:27.741863+0000 caller_uid=26, caller_gid=26{26,}) currently requesting remote authpins 2023-09-10T00:55:29.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:29 smithi098 ceph-mon[103124]: pgmap v3239: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T00:55:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:30 smithi170 ceph-mon[102832]: 3 slow requests, 3 included below; oldest blocked for > 5760.823143 secs 2023-09-10T00:55:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:30 smithi170 ceph-mon[102832]: slow request 5760.823142 seconds old, received at 2023-09-09T23:19:27.740800+0000: peer_request:mds.4:47 currently acquired locks 2023-09-10T00:55:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:30 smithi170 ceph-mon[102832]: slow request 5760.822908 seconds old, received at 2023-09-09T23:19:27.741034+0000: peer_request(mds.4:48.0 authpin) currently dispatched 2023-09-10T00:55:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:30 smithi170 ceph-mon[102832]: slow request 5760.820873 seconds old, received at 2023-09-09T23:19:27.743070+0000: peer_request(client.24913:3869.0 authpin) currently dispatched 2023-09-10T00:55:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:30 smithi082 ceph-mon[98694]: 3 slow requests, 3 included below; oldest blocked for > 5760.823143 secs 2023-09-10T00:55:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:30 smithi082 ceph-mon[98694]: slow request 5760.823142 seconds old, received at 2023-09-09T23:19:27.740800+0000: peer_request:mds.4:47 currently acquired locks 2023-09-10T00:55:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:30 smithi082 ceph-mon[98694]: slow request 5760.822908 seconds old, received at 2023-09-09T23:19:27.741034+0000: peer_request(mds.4:48.0 authpin) currently dispatched 2023-09-10T00:55:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:30 smithi082 ceph-mon[98694]: slow request 5760.820873 seconds old, received at 2023-09-09T23:19:27.743070+0000: peer_request(client.24913:3869.0 authpin) currently dispatched 2023-09-10T00:55:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:30 smithi098 ceph-mon[103124]: 3 slow requests, 3 included below; oldest blocked for > 5760.823143 secs 2023-09-10T00:55:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:30 smithi098 ceph-mon[103124]: slow request 5760.823142 seconds old, received at 2023-09-09T23:19:27.740800+0000: peer_request:mds.4:47 currently acquired locks 2023-09-10T00:55:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:30 smithi098 ceph-mon[103124]: slow request 5760.822908 seconds old, received at 2023-09-09T23:19:27.741034+0000: peer_request(mds.4:48.0 authpin) currently dispatched 2023-09-10T00:55:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:30 smithi098 ceph-mon[103124]: slow request 5760.820873 seconds old, received at 2023-09-09T23:19:27.743070+0000: peer_request(client.24913:3869.0 authpin) currently dispatched 2023-09-10T00:55:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:31 smithi170 ceph-mon[102832]: pgmap v3240: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T00:55:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:31 smithi082 ceph-mon[98694]: pgmap v3240: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T00:55:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:31 smithi098 ceph-mon[103124]: pgmap v3240: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T00:55:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:33 smithi170 ceph-mon[102832]: pgmap v3241: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 199 KiB/s wr, 19 op/s 2023-09-10T00:55:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:55:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:33 smithi082 ceph-mon[98694]: pgmap v3241: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 199 KiB/s wr, 19 op/s 2023-09-10T00:55:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:55:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:33 smithi098 ceph-mon[103124]: pgmap v3241: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 199 KiB/s wr, 19 op/s 2023-09-10T00:55:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:55:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5765.110472 secs 2023-09-10T00:55:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:55:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:55:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:55:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5765.110472 secs 2023-09-10T00:55:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:55:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:55:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:55:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5765.110472 secs 2023-09-10T00:55:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:55:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:55:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:55:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5765.823202 secs 2023-09-10T00:55:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:35 smithi098 ceph-mon[103124]: pgmap v3242: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 164 KiB/s wr, 14 op/s 2023-09-10T00:55:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:55:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5765.823202 secs 2023-09-10T00:55:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:35 smithi170 ceph-mon[102832]: pgmap v3242: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 164 KiB/s wr, 14 op/s 2023-09-10T00:55:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:55:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5765.823202 secs 2023-09-10T00:55:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:35 smithi082 ceph-mon[98694]: pgmap v3242: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 164 KiB/s wr, 14 op/s 2023-09-10T00:55:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:55:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:55:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:55:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:55:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:55:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:55:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:55:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:37 smithi098 ceph-mon[103124]: pgmap v3243: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 263 KiB/s wr, 25 op/s 2023-09-10T00:55:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:37 smithi170 ceph-mon[102832]: pgmap v3243: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 263 KiB/s wr, 25 op/s 2023-09-10T00:55:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:37 smithi082 ceph-mon[98694]: pgmap v3243: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 263 KiB/s wr, 25 op/s 2023-09-10T00:55:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:38 smithi170 ceph-mon[102832]: 1 slow requests, 1 included below; oldest blocked for > 5768.577139 secs 2023-09-10T00:55:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:38 smithi170 ceph-mon[102832]: slow request 5768.577139 seconds old, received at 2023-09-09T23:19:28.303507+0000: peer_request:mds.4:47 currently dispatched 2023-09-10T00:55:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:38 smithi082 ceph-mon[98694]: 1 slow requests, 1 included below; oldest blocked for > 5768.577139 secs 2023-09-10T00:55:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:38 smithi082 ceph-mon[98694]: slow request 5768.577139 seconds old, received at 2023-09-09T23:19:28.303507+0000: peer_request:mds.4:47 currently dispatched 2023-09-10T00:55:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:38 smithi098 ceph-mon[103124]: 1 slow requests, 1 included below; oldest blocked for > 5768.577139 secs 2023-09-10T00:55:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:38 smithi098 ceph-mon[103124]: slow request 5768.577139 seconds old, received at 2023-09-09T23:19:28.303507+0000: peer_request:mds.4:47 currently dispatched 2023-09-10T00:55:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5770.110527 secs 2023-09-10T00:55:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:39 smithi170 ceph-mon[102832]: pgmap v3244: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 223 KiB/s wr, 18 op/s 2023-09-10T00:55:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5770.110527 secs 2023-09-10T00:55:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:39 smithi082 ceph-mon[98694]: pgmap v3244: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 223 KiB/s wr, 18 op/s 2023-09-10T00:55:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5770.110527 secs 2023-09-10T00:55:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:39 smithi098 ceph-mon[103124]: pgmap v3244: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 223 KiB/s wr, 18 op/s 2023-09-10T00:55:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5770.823266 secs 2023-09-10T00:55:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:55:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5770.823266 secs 2023-09-10T00:55:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:55:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5770.823266 secs 2023-09-10T00:55:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:55:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:41 smithi170 ceph-mon[102832]: pgmap v3245: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 223 KiB/s wr, 18 op/s 2023-09-10T00:55:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:41 smithi082 ceph-mon[98694]: pgmap v3245: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 223 KiB/s wr, 18 op/s 2023-09-10T00:55:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:41 smithi098 ceph-mon[103124]: pgmap v3245: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 223 KiB/s wr, 18 op/s 2023-09-10T00:55:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5773.577200 secs 2023-09-10T00:55:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:43 smithi170 ceph-mon[102832]: pgmap v3246: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 348 KiB/s wr, 28 op/s 2023-09-10T00:55:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5773.577200 secs 2023-09-10T00:55:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:43 smithi082 ceph-mon[98694]: pgmap v3246: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 348 KiB/s wr, 28 op/s 2023-09-10T00:55:43.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5773.577200 secs 2023-09-10T00:55:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:43 smithi098 ceph-mon[103124]: pgmap v3246: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 348 KiB/s wr, 28 op/s 2023-09-10T00:55:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5775.110590 secs 2023-09-10T00:55:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5775.110590 secs 2023-09-10T00:55:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5775.110590 secs 2023-09-10T00:55:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5775.823331 secs 2023-09-10T00:55:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:45 smithi170 ceph-mon[102832]: pgmap v3247: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 223 KiB/s wr, 20 op/s 2023-09-10T00:55:45.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:45.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:45 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:55:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5775.823331 secs 2023-09-10T00:55:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:45 smithi082 ceph-mon[98694]: pgmap v3247: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 223 KiB/s wr, 20 op/s 2023-09-10T00:55:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:45 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:55:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5775.823331 secs 2023-09-10T00:55:45.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:45 smithi098 ceph-mon[103124]: pgmap v3247: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 223 KiB/s wr, 20 op/s 2023-09-10T00:55:45.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:45.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:45 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:55:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:47 smithi170 ceph-mon[102832]: pgmap v3248: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 369 KiB/s wr, 31 op/s 2023-09-10T00:55:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:47 smithi082 ceph-mon[98694]: pgmap v3248: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 369 KiB/s wr, 31 op/s 2023-09-10T00:55:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:47 smithi098 ceph-mon[103124]: pgmap v3248: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 369 KiB/s wr, 31 op/s 2023-09-10T00:55:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5778.577299 secs 2023-09-10T00:55:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:48 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5778.577299 secs 2023-09-10T00:55:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:48 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5778.577299 secs 2023-09-10T00:55:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:48 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5780.110655 secs 2023-09-10T00:55:50.708 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:49 smithi170 ceph-mon[102832]: pgmap v3249: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 270 KiB/s wr, 20 op/s 2023-09-10T00:55:50.709 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5780.110655 secs 2023-09-10T00:55:50.709 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:49 smithi098 ceph-mon[103124]: pgmap v3249: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 270 KiB/s wr, 20 op/s 2023-09-10T00:55:50.710 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5780.823393 secs 2023-09-10T00:55:50.710 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:50.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5780.110655 secs 2023-09-10T00:55:50.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:49 smithi082 ceph-mon[98694]: pgmap v3249: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 270 KiB/s wr, 20 op/s 2023-09-10T00:55:50.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5780.823393 secs 2023-09-10T00:55:50.711 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5780.823393 secs 2023-09-10T00:55:50.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:51 smithi170 ceph-mon[102832]: pgmap v3250: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 270 KiB/s wr, 20 op/s 2023-09-10T00:55:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:51 smithi082 ceph-mon[98694]: pgmap v3250: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 270 KiB/s wr, 20 op/s 2023-09-10T00:55:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:51 smithi098 ceph-mon[103124]: pgmap v3250: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 270 KiB/s wr, 20 op/s 2023-09-10T00:55:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5783.577361 secs 2023-09-10T00:55:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:53 smithi170 ceph-mon[102832]: pgmap v3251: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 356 KiB/s wr, 30 op/s 2023-09-10T00:55:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5783.577361 secs 2023-09-10T00:55:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:53 smithi082 ceph-mon[98694]: pgmap v3251: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 356 KiB/s wr, 30 op/s 2023-09-10T00:55:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5783.577361 secs 2023-09-10T00:55:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:53 smithi098 ceph-mon[103124]: pgmap v3251: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 356 KiB/s wr, 30 op/s 2023-09-10T00:55:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5785.110712 secs 2023-09-10T00:55:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5785.110712 secs 2023-09-10T00:55:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5785.110712 secs 2023-09-10T00:55:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5785.823455 secs 2023-09-10T00:55:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:55 smithi170 ceph-mon[102832]: pgmap v3252: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 231 KiB/s wr, 21 op/s 2023-09-10T00:55:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5785.823455 secs 2023-09-10T00:55:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:55 smithi082 ceph-mon[98694]: pgmap v3252: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 231 KiB/s wr, 21 op/s 2023-09-10T00:55:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5785.823455 secs 2023-09-10T00:55:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:55 smithi098 ceph-mon[103124]: pgmap v3252: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 231 KiB/s wr, 21 op/s 2023-09-10T00:55:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:57 smithi170 ceph-mon[102832]: pgmap v3253: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 493 KiB/s wr, 33 op/s 2023-09-10T00:55:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:57 smithi082 ceph-mon[98694]: pgmap v3253: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 493 KiB/s wr, 33 op/s 2023-09-10T00:55:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:57 smithi098 ceph-mon[103124]: pgmap v3253: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 493 KiB/s wr, 33 op/s 2023-09-10T00:55:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5788.577422 secs 2023-09-10T00:55:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5788.577422 secs 2023-09-10T00:55:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5788.577422 secs 2023-09-10T00:55:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:55:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5790.110772 secs 2023-09-10T00:55:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:55:59 smithi170 ceph-mon[102832]: pgmap v3254: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 348 KiB/s wr, 21 op/s 2023-09-10T00:55:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5790.110772 secs 2023-09-10T00:55:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:55:59 smithi082 ceph-mon[98694]: pgmap v3254: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 348 KiB/s wr, 21 op/s 2023-09-10T00:55:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5790.110772 secs 2023-09-10T00:55:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:55:59 smithi098 ceph-mon[103124]: pgmap v3254: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 348 KiB/s wr, 21 op/s 2023-09-10T00:56:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5790.823486 secs 2023-09-10T00:56:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5790.823486 secs 2023-09-10T00:56:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5790.823486 secs 2023-09-10T00:56:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:01 smithi170 ceph-mon[102832]: pgmap v3255: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 348 KiB/s wr, 21 op/s 2023-09-10T00:56:01.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:01 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:01 smithi082 ceph-mon[98694]: pgmap v3255: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 348 KiB/s wr, 21 op/s 2023-09-10T00:56:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:01 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:01 smithi098 ceph-mon[103124]: pgmap v3255: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 348 KiB/s wr, 21 op/s 2023-09-10T00:56:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:01 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:03 smithi170 ceph-mon[102832]: pgmap v3256: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 567 KiB/s wr, 31 op/s 2023-09-10T00:56:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:03 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:03 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:56:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:03 smithi082 ceph-mon[98694]: pgmap v3256: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 567 KiB/s wr, 31 op/s 2023-09-10T00:56:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:03 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:03 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:56:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:03 smithi098 ceph-mon[103124]: pgmap v3256: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 567 KiB/s wr, 31 op/s 2023-09-10T00:56:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:03 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:03 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:56:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5795.110830 secs 2023-09-10T00:56:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:04 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:56:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:56:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5795.110830 secs 2023-09-10T00:56:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:04 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:56:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:56:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5795.110830 secs 2023-09-10T00:56:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:04 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:56:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:56:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5795.823555 secs 2023-09-10T00:56:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:05 smithi170 ceph-mon[102832]: pgmap v3257: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 481 KiB/s wr, 21 op/s 2023-09-10T00:56:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5795.823555 secs 2023-09-10T00:56:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:05 smithi082 ceph-mon[98694]: pgmap v3257: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 481 KiB/s wr, 21 op/s 2023-09-10T00:56:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5795.823555 secs 2023-09-10T00:56:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:05 smithi098 ceph-mon[103124]: pgmap v3257: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 481 KiB/s wr, 21 op/s 2023-09-10T00:56:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:07 smithi170 ceph-mon[102832]: pgmap v3258: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 669 KiB/s wr, 45 op/s 2023-09-10T00:56:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:07 smithi082 ceph-mon[98694]: pgmap v3258: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 669 KiB/s wr, 45 op/s 2023-09-10T00:56:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:07 smithi098 ceph-mon[103124]: pgmap v3258: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 669 KiB/s wr, 45 op/s 2023-09-10T00:56:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5798.577590 secs 2023-09-10T00:56:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:08 smithi170 ceph-mon[102832]: osdmap e275: 12 total, 12 up, 12 in 2023-09-10T00:56:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5798.577590 secs 2023-09-10T00:56:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:08 smithi082 ceph-mon[98694]: osdmap e275: 12 total, 12 up, 12 in 2023-09-10T00:56:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5798.577590 secs 2023-09-10T00:56:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:08 smithi098 ceph-mon[103124]: osdmap e275: 12 total, 12 up, 12 in 2023-09-10T00:56:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5800.110890 secs 2023-09-10T00:56:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:09 smithi170 ceph-mon[102832]: pgmap v3260: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 488 KiB/s wr, 40 op/s 2023-09-10T00:56:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:09 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5800.110890 secs 2023-09-10T00:56:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:09 smithi082 ceph-mon[98694]: pgmap v3260: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 488 KiB/s wr, 40 op/s 2023-09-10T00:56:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:09 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5800.110890 secs 2023-09-10T00:56:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:09 smithi098 ceph-mon[103124]: pgmap v3260: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 488 KiB/s wr, 40 op/s 2023-09-10T00:56:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:09 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5800.823620 secs 2023-09-10T00:56:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5800.823620 secs 2023-09-10T00:56:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5800.823620 secs 2023-09-10T00:56:11.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:11 smithi170 ceph-mon[102832]: pgmap v3261: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 488 KiB/s wr, 40 op/s 2023-09-10T00:56:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:11 smithi082 ceph-mon[98694]: pgmap v3261: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 488 KiB/s wr, 40 op/s 2023-09-10T00:56:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:11 smithi098 ceph-mon[103124]: pgmap v3261: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 488 KiB/s wr, 40 op/s 2023-09-10T00:56:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:12 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:56:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:56:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:12 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:56:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:56:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:12 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:56:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:56:12.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5803.577695 secs 2023-09-10T00:56:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:13 smithi170 ceph-mon[102832]: pgmap v3262: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 267 KiB/s wr, 35 op/s 2023-09-10T00:56:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5803.577695 secs 2023-09-10T00:56:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:13 smithi082 ceph-mon[98694]: pgmap v3262: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 267 KiB/s wr, 35 op/s 2023-09-10T00:56:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5803.577695 secs 2023-09-10T00:56:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:13 smithi098 ceph-mon[103124]: pgmap v3262: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 267 KiB/s wr, 35 op/s 2023-09-10T00:56:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5805.110975 secs 2023-09-10T00:56:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5805.110975 secs 2023-09-10T00:56:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5805.110975 secs 2023-09-10T00:56:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5805.823683 secs 2023-09-10T00:56:15.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:15 smithi170 ceph-mon[102832]: pgmap v3263: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 267 KiB/s wr, 35 op/s 2023-09-10T00:56:15.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:15 smithi170 ceph-mon[102832]: osdmap e276: 12 total, 12 up, 12 in 2023-09-10T00:56:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5805.823683 secs 2023-09-10T00:56:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:15 smithi082 ceph-mon[98694]: pgmap v3263: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 267 KiB/s wr, 35 op/s 2023-09-10T00:56:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:15 smithi082 ceph-mon[98694]: osdmap e276: 12 total, 12 up, 12 in 2023-09-10T00:56:15.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5805.823683 secs 2023-09-10T00:56:15.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:15 smithi098 ceph-mon[103124]: pgmap v3263: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 267 KiB/s wr, 35 op/s 2023-09-10T00:56:15.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:15 smithi098 ceph-mon[103124]: osdmap e276: 12 total, 12 up, 12 in 2023-09-10T00:56:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:17 smithi170 ceph-mon[102832]: pgmap v3265: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 78 KiB/s wr, 13 op/s 2023-09-10T00:56:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:17 smithi082 ceph-mon[98694]: pgmap v3265: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 78 KiB/s wr, 13 op/s 2023-09-10T00:56:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:17 smithi098 ceph-mon[103124]: pgmap v3265: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 78 KiB/s wr, 13 op/s 2023-09-10T00:56:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5808.577755 secs 2023-09-10T00:56:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5808.577755 secs 2023-09-10T00:56:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5808.577755 secs 2023-09-10T00:56:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5810.111037 secs 2023-09-10T00:56:19.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:19 smithi170 ceph-mon[102832]: pgmap v3266: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T00:56:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5810.111037 secs 2023-09-10T00:56:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:19 smithi082 ceph-mon[98694]: pgmap v3266: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T00:56:19.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5810.111037 secs 2023-09-10T00:56:19.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:19 smithi098 ceph-mon[103124]: pgmap v3266: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T00:56:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5810.823746 secs 2023-09-10T00:56:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5810.823746 secs 2023-09-10T00:56:20.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5810.823746 secs 2023-09-10T00:56:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:21 smithi082 ceph-mon[98694]: pgmap v3267: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T00:56:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:21 smithi098 ceph-mon[103124]: pgmap v3267: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T00:56:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:21 smithi170 ceph-mon[102832]: pgmap v3267: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T00:56:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:23 smithi082 ceph-mon[98694]: pgmap v3268: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-09-10T00:56:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:56:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:23 smithi098 ceph-mon[103124]: pgmap v3268: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-09-10T00:56:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:56:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:23 smithi170 ceph-mon[102832]: pgmap v3268: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-09-10T00:56:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:56:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5815.111121 secs 2023-09-10T00:56:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:56:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:56:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5815.111121 secs 2023-09-10T00:56:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:56:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:56:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5815.111121 secs 2023-09-10T00:56:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:56:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:56:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5815.823808 secs 2023-09-10T00:56:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:25 smithi082 ceph-mon[98694]: pgmap v3269: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-09-10T00:56:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5815.823808 secs 2023-09-10T00:56:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:25 smithi098 ceph-mon[103124]: pgmap v3269: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-09-10T00:56:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5815.823808 secs 2023-09-10T00:56:25.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:25 smithi170 ceph-mon[102832]: pgmap v3269: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-09-10T00:56:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:27 smithi082 ceph-mon[98694]: pgmap v3270: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 76 KiB/s wr, 15 op/s 2023-09-10T00:56:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:27 smithi098 ceph-mon[103124]: pgmap v3270: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 76 KiB/s wr, 15 op/s 2023-09-10T00:56:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:27 smithi170 ceph-mon[102832]: pgmap v3270: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 76 KiB/s wr, 15 op/s 2023-09-10T00:56:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5820.111180 secs 2023-09-10T00:56:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:29 smithi082 ceph-mon[98694]: pgmap v3271: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 9 op/s 2023-09-10T00:56:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5820.111180 secs 2023-09-10T00:56:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:29 smithi098 ceph-mon[103124]: pgmap v3271: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 9 op/s 2023-09-10T00:56:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5820.111180 secs 2023-09-10T00:56:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:29 smithi170 ceph-mon[102832]: pgmap v3271: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 9 op/s 2023-09-10T00:56:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5820.823873 secs 2023-09-10T00:56:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5820.823873 secs 2023-09-10T00:56:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5820.823873 secs 2023-09-10T00:56:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:31 smithi098 ceph-mon[103124]: pgmap v3272: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 9 op/s 2023-09-10T00:56:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:31 smithi170 ceph-mon[102832]: pgmap v3272: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 9 op/s 2023-09-10T00:56:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:31 smithi082 ceph-mon[98694]: pgmap v3272: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 9 op/s 2023-09-10T00:56:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:32 smithi098 ceph-mon[103124]: pgmap v3273: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 105 KiB/s wr, 14 op/s 2023-09-10T00:56:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:32 smithi170 ceph-mon[102832]: pgmap v3273: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 105 KiB/s wr, 14 op/s 2023-09-10T00:56:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:32 smithi082 ceph-mon[98694]: pgmap v3273: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 105 KiB/s wr, 14 op/s 2023-09-10T00:56:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5825.111238 secs 2023-09-10T00:56:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:56:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5825.111238 secs 2023-09-10T00:56:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:56:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5825.111238 secs 2023-09-10T00:56:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:56:34.475 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:34 smithi170 ceph-mon[102832]: pgmap v3274: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T00:56:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:34 smithi098 ceph-mon[103124]: pgmap v3274: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T00:56:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:34 smithi082 ceph-mon[98694]: pgmap v3274: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T00:56:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5825.823944 secs 2023-09-10T00:56:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5825.823944 secs 2023-09-10T00:56:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5825.823944 secs 2023-09-10T00:56:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:37 smithi170 ceph-mon[102832]: pgmap v3275: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 211 KiB/s wr, 16 op/s 2023-09-10T00:56:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:56:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:56:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:56:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:56:37.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:56:37.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:56:37.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:37.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:37.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:37.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:37.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:37.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:37.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:37.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:37 smithi082 ceph-mon[98694]: pgmap v3275: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 211 KiB/s wr, 16 op/s 2023-09-10T00:56:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:56:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:56:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:56:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:56:37.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:56:37.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:56:37.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:37.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:37.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:37.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:37.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:37.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:37.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:37.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:37 smithi098 ceph-mon[103124]: pgmap v3275: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 211 KiB/s wr, 16 op/s 2023-09-10T00:56:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:56:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:56:37.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:56:37.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:56:37.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:56:37.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:56:37.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:37.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:37.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:37.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:37.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:37.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:37.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:37.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:38 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T00:56:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:38 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T00:56:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:38 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T00:56:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:38 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T00:56:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:38 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T00:56:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:38 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T00:56:38.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:56:38.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:56:38.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:56:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:38 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T00:56:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:38 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T00:56:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:38 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T00:56:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:38 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T00:56:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:38.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:38.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:38 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T00:56:38.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:38 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T00:56:38.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:56:38.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:56:38.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:56:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:38 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T00:56:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:38 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T00:56:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:38 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T00:56:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:38 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T00:56:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T00:56:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:38 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T00:56:38.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:38 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T00:56:38.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:56:38.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:56:38.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:56:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5830.111326 secs 2023-09-10T00:56:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:39 smithi170 ceph-mon[102832]: pgmap v3276: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T00:56:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5830.111326 secs 2023-09-10T00:56:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:39 smithi082 ceph-mon[98694]: pgmap v3276: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T00:56:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5830.111326 secs 2023-09-10T00:56:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:39 smithi098 ceph-mon[103124]: pgmap v3276: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T00:56:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5830.824006 secs 2023-09-10T00:56:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5830.824006 secs 2023-09-10T00:56:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5830.824006 secs 2023-09-10T00:56:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:41 smithi170 ceph-mon[102832]: pgmap v3277: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T00:56:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:41 smithi082 ceph-mon[98694]: pgmap v3277: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T00:56:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:41 smithi098 ceph-mon[103124]: pgmap v3277: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T00:56:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:43 smithi170 ceph-mon[102832]: pgmap v3278: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 253 KiB/s wr, 18 op/s 2023-09-10T00:56:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:43 smithi082 ceph-mon[98694]: pgmap v3278: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 253 KiB/s wr, 18 op/s 2023-09-10T00:56:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:43 smithi098 ceph-mon[103124]: pgmap v3278: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 253 KiB/s wr, 18 op/s 2023-09-10T00:56:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5835.111414 secs 2023-09-10T00:56:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5835.111414 secs 2023-09-10T00:56:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5835.111414 secs 2023-09-10T00:56:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5835.824069 secs 2023-09-10T00:56:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:45 smithi170 ceph-mon[102832]: pgmap v3279: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 196 KiB/s wr, 12 op/s 2023-09-10T00:56:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5835.824069 secs 2023-09-10T00:56:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:45 smithi082 ceph-mon[98694]: pgmap v3279: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 196 KiB/s wr, 12 op/s 2023-09-10T00:56:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5835.824069 secs 2023-09-10T00:56:45.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:45 smithi098 ceph-mon[103124]: pgmap v3279: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 196 KiB/s wr, 12 op/s 2023-09-10T00:56:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:47 smithi170 ceph-mon[102832]: pgmap v3280: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 288 KiB/s wr, 19 op/s 2023-09-10T00:56:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:47 smithi082 ceph-mon[98694]: pgmap v3280: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 288 KiB/s wr, 19 op/s 2023-09-10T00:56:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:47 smithi098 ceph-mon[103124]: pgmap v3280: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 288 KiB/s wr, 19 op/s 2023-09-10T00:56:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5838.578149 secs 2023-09-10T00:56:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5838.578149 secs 2023-09-10T00:56:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5838.578149 secs 2023-09-10T00:56:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5840.111473 secs 2023-09-10T00:56:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:49 smithi170 ceph-mon[102832]: pgmap v3281: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 183 KiB/s wr, 12 op/s 2023-09-10T00:56:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5840.111473 secs 2023-09-10T00:56:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:49 smithi082 ceph-mon[98694]: pgmap v3281: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 183 KiB/s wr, 12 op/s 2023-09-10T00:56:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5840.111473 secs 2023-09-10T00:56:49.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:49 smithi098 ceph-mon[103124]: pgmap v3281: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 183 KiB/s wr, 12 op/s 2023-09-10T00:56:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5840.824139 secs 2023-09-10T00:56:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5840.824139 secs 2023-09-10T00:56:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5840.824139 secs 2023-09-10T00:56:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:51 smithi082 ceph-mon[98694]: pgmap v3282: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 183 KiB/s wr, 12 op/s 2023-09-10T00:56:51.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:51 smithi098 ceph-mon[103124]: pgmap v3282: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 183 KiB/s wr, 12 op/s 2023-09-10T00:56:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:51 smithi170 ceph-mon[102832]: pgmap v3282: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 183 KiB/s wr, 12 op/s 2023-09-10T00:56:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:56:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:56:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:56:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:56:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:56:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:56:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:56:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5843.578207 secs 2023-09-10T00:56:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:53 smithi082 ceph-mon[98694]: pgmap v3283: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 236 KiB/s wr, 18 op/s 2023-09-10T00:56:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5843.578207 secs 2023-09-10T00:56:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:53 smithi098 ceph-mon[103124]: pgmap v3283: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 236 KiB/s wr, 18 op/s 2023-09-10T00:56:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5843.578207 secs 2023-09-10T00:56:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:53 smithi170 ceph-mon[102832]: pgmap v3283: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 236 KiB/s wr, 18 op/s 2023-09-10T00:56:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5845.111532 secs 2023-09-10T00:56:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5845.111532 secs 2023-09-10T00:56:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5845.111532 secs 2023-09-10T00:56:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5845.824199 secs 2023-09-10T00:56:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:55 smithi082 ceph-mon[98694]: pgmap v3284: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T00:56:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5845.824199 secs 2023-09-10T00:56:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:55 smithi098 ceph-mon[103124]: pgmap v3284: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T00:56:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5845.824199 secs 2023-09-10T00:56:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:55 smithi170 ceph-mon[102832]: pgmap v3284: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T00:56:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:57 smithi082 ceph-mon[98694]: pgmap v3285: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 173 KiB/s wr, 17 op/s 2023-09-10T00:56:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:57 smithi098 ceph-mon[103124]: pgmap v3285: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 173 KiB/s wr, 17 op/s 2023-09-10T00:56:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:57 smithi170 ceph-mon[102832]: pgmap v3285: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 173 KiB/s wr, 17 op/s 2023-09-10T00:56:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5848.578266 secs 2023-09-10T00:56:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5848.578266 secs 2023-09-10T00:56:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5848.578266 secs 2023-09-10T00:56:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5850.111589 secs 2023-09-10T00:56:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:56:59 smithi098 ceph-mon[103124]: pgmap v3286: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T00:56:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5850.111589 secs 2023-09-10T00:56:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:56:59 smithi170 ceph-mon[102832]: pgmap v3286: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T00:56:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5850.111589 secs 2023-09-10T00:56:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:56:59 smithi082 ceph-mon[98694]: pgmap v3286: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T00:57:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5850.824261 secs 2023-09-10T00:57:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5850.824261 secs 2023-09-10T00:57:00.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5850.824261 secs 2023-09-10T00:57:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:01 smithi098 ceph-mon[103124]: pgmap v3287: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T00:57:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:01 smithi170 ceph-mon[102832]: pgmap v3287: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T00:57:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:01 smithi082 ceph-mon[98694]: pgmap v3287: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T00:57:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5853.578437 secs 2023-09-10T00:57:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:03 smithi098 ceph-mon[103124]: pgmap v3288: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 102 KiB/s wr, 15 op/s 2023-09-10T00:57:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5853.578437 secs 2023-09-10T00:57:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:03 smithi170 ceph-mon[102832]: pgmap v3288: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 102 KiB/s wr, 15 op/s 2023-09-10T00:57:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5853.578437 secs 2023-09-10T00:57:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:03 smithi082 ceph-mon[98694]: pgmap v3288: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 102 KiB/s wr, 15 op/s 2023-09-10T00:57:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5855.111650 secs 2023-09-10T00:57:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5855.111650 secs 2023-09-10T00:57:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5855.111650 secs 2023-09-10T00:57:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5855.824325 secs 2023-09-10T00:57:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:05 smithi098 ceph-mon[103124]: pgmap v3289: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 49 KiB/s wr, 9 op/s 2023-09-10T00:57:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5855.824325 secs 2023-09-10T00:57:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:05 smithi170 ceph-mon[102832]: pgmap v3289: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 49 KiB/s wr, 9 op/s 2023-09-10T00:57:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5855.824325 secs 2023-09-10T00:57:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:05 smithi082 ceph-mon[98694]: pgmap v3289: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 49 KiB/s wr, 9 op/s 2023-09-10T00:57:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:07 smithi098 ceph-mon[103124]: pgmap v3290: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 94 KiB/s wr, 29 op/s 2023-09-10T00:57:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:07 smithi170 ceph-mon[102832]: pgmap v3290: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 94 KiB/s wr, 29 op/s 2023-09-10T00:57:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:07 smithi082 ceph-mon[98694]: pgmap v3290: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 94 KiB/s wr, 29 op/s 2023-09-10T00:57:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5858.578497 secs 2023-09-10T00:57:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:08 smithi098 ceph-mon[103124]: osdmap e277: 12 total, 12 up, 12 in 2023-09-10T00:57:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5858.578497 secs 2023-09-10T00:57:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:08 smithi170 ceph-mon[102832]: osdmap e277: 12 total, 12 up, 12 in 2023-09-10T00:57:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5858.578497 secs 2023-09-10T00:57:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:08 smithi082 ceph-mon[98694]: osdmap e277: 12 total, 12 up, 12 in 2023-09-10T00:57:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5860.111747 secs 2023-09-10T00:57:09.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:09 smithi098 ceph-mon[103124]: pgmap v3292: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T00:57:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5860.111747 secs 2023-09-10T00:57:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:09 smithi170 ceph-mon[102832]: pgmap v3292: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T00:57:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5860.111747 secs 2023-09-10T00:57:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:09 smithi082 ceph-mon[98694]: pgmap v3292: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T00:57:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5860.824390 secs 2023-09-10T00:57:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5860.824390 secs 2023-09-10T00:57:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5860.824390 secs 2023-09-10T00:57:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:11 smithi098 ceph-mon[103124]: pgmap v3293: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T00:57:11.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:11 smithi170 ceph-mon[102832]: pgmap v3293: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T00:57:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:11 smithi082 ceph-mon[98694]: pgmap v3293: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T00:57:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5863.578557 secs 2023-09-10T00:57:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:13 smithi170 ceph-mon[102832]: pgmap v3294: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 56 KiB/s wr, 29 op/s 2023-09-10T00:57:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5863.578557 secs 2023-09-10T00:57:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:13 smithi082 ceph-mon[98694]: pgmap v3294: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 56 KiB/s wr, 29 op/s 2023-09-10T00:57:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5863.578557 secs 2023-09-10T00:57:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:13 smithi098 ceph-mon[103124]: pgmap v3294: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 56 KiB/s wr, 29 op/s 2023-09-10T00:57:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5865.111805 secs 2023-09-10T00:57:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5865.111805 secs 2023-09-10T00:57:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5865.111805 secs 2023-09-10T00:57:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5865.824451 secs 2023-09-10T00:57:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:15 smithi170 ceph-mon[102832]: pgmap v3295: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 56 KiB/s wr, 29 op/s 2023-09-10T00:57:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:15 smithi170 ceph-mon[102832]: osdmap e278: 12 total, 12 up, 12 in 2023-09-10T00:57:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5865.824451 secs 2023-09-10T00:57:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:15 smithi082 ceph-mon[98694]: pgmap v3295: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 56 KiB/s wr, 29 op/s 2023-09-10T00:57:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:15 smithi082 ceph-mon[98694]: osdmap e278: 12 total, 12 up, 12 in 2023-09-10T00:57:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5865.824451 secs 2023-09-10T00:57:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:15 smithi098 ceph-mon[103124]: pgmap v3295: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 56 KiB/s wr, 29 op/s 2023-09-10T00:57:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:15 smithi098 ceph-mon[103124]: osdmap e278: 12 total, 12 up, 12 in 2023-09-10T00:57:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:17 smithi170 ceph-mon[102832]: pgmap v3297: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 13 op/s 2023-09-10T00:57:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:17 smithi082 ceph-mon[98694]: pgmap v3297: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 13 op/s 2023-09-10T00:57:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:17 smithi098 ceph-mon[103124]: pgmap v3297: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 13 op/s 2023-09-10T00:57:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5868.578666 secs 2023-09-10T00:57:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5868.578666 secs 2023-09-10T00:57:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5868.578666 secs 2023-09-10T00:57:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5870.111880 secs 2023-09-10T00:57:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:19 smithi170 ceph-mon[102832]: pgmap v3298: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T00:57:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5870.111880 secs 2023-09-10T00:57:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:19 smithi082 ceph-mon[98694]: pgmap v3298: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T00:57:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5870.111880 secs 2023-09-10T00:57:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:19 smithi098 ceph-mon[103124]: pgmap v3298: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T00:57:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5870.824484 secs 2023-09-10T00:57:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5870.824484 secs 2023-09-10T00:57:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5870.824484 secs 2023-09-10T00:57:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:21 smithi170 ceph-mon[102832]: pgmap v3299: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T00:57:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:21 smithi082 ceph-mon[98694]: pgmap v3299: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T00:57:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:21 smithi098 ceph-mon[103124]: pgmap v3299: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T00:57:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:23 smithi170 ceph-mon[102832]: pgmap v3300: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T00:57:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:57:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:23 smithi082 ceph-mon[98694]: pgmap v3300: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T00:57:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:57:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:23 smithi098 ceph-mon[103124]: pgmap v3300: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T00:57:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:57:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5875.111948 secs 2023-09-10T00:57:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:57:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:57:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:57:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5875.111948 secs 2023-09-10T00:57:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:57:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:57:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:57:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5875.111948 secs 2023-09-10T00:57:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:57:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:57:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:57:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5875.824544 secs 2023-09-10T00:57:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:25 smithi170 ceph-mon[102832]: pgmap v3301: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T00:57:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5875.824544 secs 2023-09-10T00:57:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:25 smithi082 ceph-mon[98694]: pgmap v3301: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T00:57:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5875.824544 secs 2023-09-10T00:57:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:25 smithi098 ceph-mon[103124]: pgmap v3301: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T00:57:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:27 smithi170 ceph-mon[102832]: pgmap v3302: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 131 KiB/s wr, 17 op/s 2023-09-10T00:57:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:27 smithi082 ceph-mon[98694]: pgmap v3302: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 131 KiB/s wr, 17 op/s 2023-09-10T00:57:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:27 smithi098 ceph-mon[103124]: pgmap v3302: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 131 KiB/s wr, 17 op/s 2023-09-10T00:57:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5880.112005 secs 2023-09-10T00:57:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:29 smithi170 ceph-mon[102832]: pgmap v3303: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T00:57:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5880.112005 secs 2023-09-10T00:57:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:29 smithi082 ceph-mon[98694]: pgmap v3303: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T00:57:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5880.112005 secs 2023-09-10T00:57:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:29 smithi098 ceph-mon[103124]: pgmap v3303: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T00:57:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5880.824607 secs 2023-09-10T00:57:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5880.824607 secs 2023-09-10T00:57:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5880.824607 secs 2023-09-10T00:57:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:31 smithi170 ceph-mon[102832]: pgmap v3304: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T00:57:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:31 smithi082 ceph-mon[98694]: pgmap v3304: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T00:57:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:31 smithi098 ceph-mon[103124]: pgmap v3304: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T00:57:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:33 smithi170 ceph-mon[102832]: pgmap v3305: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 213 KiB/s wr, 17 op/s 2023-09-10T00:57:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:33 smithi082 ceph-mon[98694]: pgmap v3305: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 213 KiB/s wr, 17 op/s 2023-09-10T00:57:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:33 smithi098 ceph-mon[103124]: pgmap v3305: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 213 KiB/s wr, 17 op/s 2023-09-10T00:57:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5885.112058 secs 2023-09-10T00:57:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5885.112058 secs 2023-09-10T00:57:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5885.112058 secs 2023-09-10T00:57:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5885.824671 secs 2023-09-10T00:57:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:35 smithi170 ceph-mon[102832]: pgmap v3306: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 169 KiB/s wr, 12 op/s 2023-09-10T00:57:35.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5885.824671 secs 2023-09-10T00:57:35.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:35 smithi082 ceph-mon[98694]: pgmap v3306: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 169 KiB/s wr, 12 op/s 2023-09-10T00:57:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5885.824671 secs 2023-09-10T00:57:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:35 smithi098 ceph-mon[103124]: pgmap v3306: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 169 KiB/s wr, 12 op/s 2023-09-10T00:57:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:37 smithi170 ceph-mon[102832]: pgmap v3307: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 184 KiB/s wr, 18 op/s 2023-09-10T00:57:37.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:57:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:37 smithi082 ceph-mon[98694]: pgmap v3307: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 184 KiB/s wr, 18 op/s 2023-09-10T00:57:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:57:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:37 smithi098 ceph-mon[103124]: pgmap v3307: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 184 KiB/s wr, 18 op/s 2023-09-10T00:57:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:57:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5890.112139 secs 2023-09-10T00:57:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:39 smithi170 ceph-mon[102832]: pgmap v3308: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 143 KiB/s wr, 12 op/s 2023-09-10T00:57:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:57:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:57:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:57:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5890.112139 secs 2023-09-10T00:57:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:39 smithi082 ceph-mon[98694]: pgmap v3308: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 143 KiB/s wr, 12 op/s 2023-09-10T00:57:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:57:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:57:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:57:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5890.112139 secs 2023-09-10T00:57:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:39 smithi098 ceph-mon[103124]: pgmap v3308: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 143 KiB/s wr, 12 op/s 2023-09-10T00:57:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:57:39.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:57:39.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:57:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5890.824733 secs 2023-09-10T00:57:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5890.824733 secs 2023-09-10T00:57:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5890.824733 secs 2023-09-10T00:57:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:41 smithi170 ceph-mon[102832]: pgmap v3309: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 143 KiB/s wr, 12 op/s 2023-09-10T00:57:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:41 smithi082 ceph-mon[98694]: pgmap v3309: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 143 KiB/s wr, 12 op/s 2023-09-10T00:57:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:41 smithi098 ceph-mon[103124]: pgmap v3309: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 143 KiB/s wr, 12 op/s 2023-09-10T00:57:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:43 smithi170 ceph-mon[102832]: pgmap v3310: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 276 KiB/s wr, 17 op/s 2023-09-10T00:57:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:43 smithi082 ceph-mon[98694]: pgmap v3310: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 276 KiB/s wr, 17 op/s 2023-09-10T00:57:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:43 smithi098 ceph-mon[103124]: pgmap v3310: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 276 KiB/s wr, 17 op/s 2023-09-10T00:57:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5895.112223 secs 2023-09-10T00:57:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5895.112223 secs 2023-09-10T00:57:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5895.112223 secs 2023-09-10T00:57:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5895.824797 secs 2023-09-10T00:57:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:45 smithi170 ceph-mon[102832]: pgmap v3311: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T00:57:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5895.824797 secs 2023-09-10T00:57:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:45 smithi082 ceph-mon[98694]: pgmap v3311: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T00:57:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5895.824797 secs 2023-09-10T00:57:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:45 smithi098 ceph-mon[103124]: pgmap v3311: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T00:57:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:47 smithi170 ceph-mon[102832]: pgmap v3312: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 191 KiB/s wr, 17 op/s 2023-09-10T00:57:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:47 smithi082 ceph-mon[98694]: pgmap v3312: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 191 KiB/s wr, 17 op/s 2023-09-10T00:57:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:47 smithi098 ceph-mon[103124]: pgmap v3312: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 191 KiB/s wr, 17 op/s 2023-09-10T00:57:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5898.579124 secs 2023-09-10T00:57:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5898.579124 secs 2023-09-10T00:57:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5898.579124 secs 2023-09-10T00:57:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5900.112282 secs 2023-09-10T00:57:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:49 smithi170 ceph-mon[102832]: pgmap v3313: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T00:57:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5900.112282 secs 2023-09-10T00:57:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:49 smithi082 ceph-mon[98694]: pgmap v3313: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T00:57:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5900.112282 secs 2023-09-10T00:57:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:49 smithi098 ceph-mon[103124]: pgmap v3313: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T00:57:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5900.824857 secs 2023-09-10T00:57:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5900.824857 secs 2023-09-10T00:57:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5900.824857 secs 2023-09-10T00:57:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:51 smithi170 ceph-mon[102832]: pgmap v3314: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T00:57:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:51 smithi082 ceph-mon[98694]: pgmap v3314: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T00:57:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:51 smithi098 ceph-mon[103124]: pgmap v3314: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T00:57:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:57:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:57:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:57:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:57:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:57:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:57:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:57:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:57:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:57:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5903.579213 secs 2023-09-10T00:57:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:53 smithi170 ceph-mon[102832]: pgmap v3315: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 265 KiB/s wr, 17 op/s 2023-09-10T00:57:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5903.579213 secs 2023-09-10T00:57:53.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:53 smithi082 ceph-mon[98694]: pgmap v3315: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 265 KiB/s wr, 17 op/s 2023-09-10T00:57:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5903.579213 secs 2023-09-10T00:57:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:53 smithi098 ceph-mon[103124]: pgmap v3315: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 265 KiB/s wr, 17 op/s 2023-09-10T00:57:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5905.112340 secs 2023-09-10T00:57:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5905.112340 secs 2023-09-10T00:57:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5905.112340 secs 2023-09-10T00:57:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5905.824919 secs 2023-09-10T00:57:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:55 smithi170 ceph-mon[102832]: pgmap v3316: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T00:57:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5905.824919 secs 2023-09-10T00:57:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:55 smithi082 ceph-mon[98694]: pgmap v3316: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T00:57:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5905.824919 secs 2023-09-10T00:57:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:55 smithi098 ceph-mon[103124]: pgmap v3316: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T00:57:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:57 smithi170 ceph-mon[102832]: pgmap v3317: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 132 KiB/s wr, 17 op/s 2023-09-10T00:57:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:57 smithi082 ceph-mon[98694]: pgmap v3317: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 132 KiB/s wr, 17 op/s 2023-09-10T00:57:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:57 smithi098 ceph-mon[103124]: pgmap v3317: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 132 KiB/s wr, 17 op/s 2023-09-10T00:57:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5908.579309 secs 2023-09-10T00:57:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5908.579309 secs 2023-09-10T00:57:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5908.579309 secs 2023-09-10T00:57:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5910.112397 secs 2023-09-10T00:57:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:57:59 smithi170 ceph-mon[102832]: pgmap v3318: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:57:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5910.112397 secs 2023-09-10T00:57:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:57:59 smithi082 ceph-mon[98694]: pgmap v3318: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:57:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5910.112397 secs 2023-09-10T00:57:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:57:59 smithi098 ceph-mon[103124]: pgmap v3318: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:58:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5910.825006 secs 2023-09-10T00:58:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5910.825006 secs 2023-09-10T00:58:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5910.825006 secs 2023-09-10T00:58:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:01 smithi170 ceph-mon[102832]: pgmap v3319: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:58:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:01 smithi082 ceph-mon[98694]: pgmap v3319: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:58:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:01 smithi098 ceph-mon[103124]: pgmap v3319: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T00:58:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5913.579423 secs 2023-09-10T00:58:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:03 smithi170 ceph-mon[102832]: pgmap v3320: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 111 KiB/s wr, 16 op/s 2023-09-10T00:58:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5913.579423 secs 2023-09-10T00:58:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:03 smithi082 ceph-mon[98694]: pgmap v3320: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 111 KiB/s wr, 16 op/s 2023-09-10T00:58:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5913.579423 secs 2023-09-10T00:58:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:03 smithi098 ceph-mon[103124]: pgmap v3320: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 111 KiB/s wr, 16 op/s 2023-09-10T00:58:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5915.112497 secs 2023-09-10T00:58:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5915.112497 secs 2023-09-10T00:58:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5915.112497 secs 2023-09-10T00:58:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5915.825067 secs 2023-09-10T00:58:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:05 smithi170 ceph-mon[102832]: pgmap v3321: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-09-10T00:58:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5915.825067 secs 2023-09-10T00:58:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:05 smithi082 ceph-mon[98694]: pgmap v3321: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-09-10T00:58:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5915.825067 secs 2023-09-10T00:58:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:05 smithi098 ceph-mon[103124]: pgmap v3321: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-09-10T00:58:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:07 smithi082 ceph-mon[98694]: pgmap v3322: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 68 KiB/s wr, 29 op/s 2023-09-10T00:58:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:07 smithi098 ceph-mon[103124]: pgmap v3322: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 68 KiB/s wr, 29 op/s 2023-09-10T00:58:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:07 smithi170 ceph-mon[102832]: pgmap v3322: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 68 KiB/s wr, 29 op/s 2023-09-10T00:58:08.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5918.579485 secs 2023-09-10T00:58:08.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:08 smithi082 ceph-mon[98694]: osdmap e279: 12 total, 12 up, 12 in 2023-09-10T00:58:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5918.579485 secs 2023-09-10T00:58:08.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:08 smithi098 ceph-mon[103124]: osdmap e279: 12 total, 12 up, 12 in 2023-09-10T00:58:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5918.579485 secs 2023-09-10T00:58:08.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:08 smithi170 ceph-mon[102832]: osdmap e279: 12 total, 12 up, 12 in 2023-09-10T00:58:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5920.112583 secs 2023-09-10T00:58:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:09 smithi082 ceph-mon[98694]: pgmap v3324: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T00:58:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:09 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5920.112583 secs 2023-09-10T00:58:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:09 smithi098 ceph-mon[103124]: pgmap v3324: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T00:58:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:09 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5920.112583 secs 2023-09-10T00:58:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:09 smithi170 ceph-mon[102832]: pgmap v3324: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T00:58:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:09 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5920.825129 secs 2023-09-10T00:58:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5920.825129 secs 2023-09-10T00:58:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5920.825129 secs 2023-09-10T00:58:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:11 smithi082 ceph-mon[98694]: pgmap v3325: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T00:58:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:11 smithi098 ceph-mon[103124]: pgmap v3325: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T00:58:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:11 smithi170 ceph-mon[102832]: pgmap v3325: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T00:58:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5923.579576 secs 2023-09-10T00:58:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:13 smithi098 ceph-mon[103124]: pgmap v3326: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 117 KiB/s wr, 29 op/s 2023-09-10T00:58:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5923.579576 secs 2023-09-10T00:58:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:13 smithi170 ceph-mon[102832]: pgmap v3326: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 117 KiB/s wr, 29 op/s 2023-09-10T00:58:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5923.579576 secs 2023-09-10T00:58:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:13 smithi082 ceph-mon[98694]: pgmap v3326: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 117 KiB/s wr, 29 op/s 2023-09-10T00:58:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5925.112641 secs 2023-09-10T00:58:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5925.112641 secs 2023-09-10T00:58:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5925.112641 secs 2023-09-10T00:58:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5925.825192 secs 2023-09-10T00:58:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:15 smithi098 ceph-mon[103124]: pgmap v3327: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 117 KiB/s wr, 29 op/s 2023-09-10T00:58:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:15 smithi098 ceph-mon[103124]: osdmap e280: 12 total, 12 up, 12 in 2023-09-10T00:58:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5925.825192 secs 2023-09-10T00:58:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:15 smithi170 ceph-mon[102832]: pgmap v3327: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 117 KiB/s wr, 29 op/s 2023-09-10T00:58:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:15 smithi170 ceph-mon[102832]: osdmap e280: 12 total, 12 up, 12 in 2023-09-10T00:58:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5925.825192 secs 2023-09-10T00:58:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:15 smithi082 ceph-mon[98694]: pgmap v3327: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 117 KiB/s wr, 29 op/s 2023-09-10T00:58:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:15 smithi082 ceph-mon[98694]: osdmap e280: 12 total, 12 up, 12 in 2023-09-10T00:58:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:17 smithi098 ceph-mon[103124]: pgmap v3329: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 115 KiB/s wr, 13 op/s 2023-09-10T00:58:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:17 smithi170 ceph-mon[102832]: pgmap v3329: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 115 KiB/s wr, 13 op/s 2023-09-10T00:58:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:17 smithi082 ceph-mon[98694]: pgmap v3329: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 115 KiB/s wr, 13 op/s 2023-09-10T00:58:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5928.579635 secs 2023-09-10T00:58:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5928.579635 secs 2023-09-10T00:58:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5928.579635 secs 2023-09-10T00:58:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5930.112703 secs 2023-09-10T00:58:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:19 smithi098 ceph-mon[103124]: pgmap v3330: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T00:58:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:19 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5930.112703 secs 2023-09-10T00:58:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:19 smithi170 ceph-mon[102832]: pgmap v3330: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T00:58:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:19 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5930.112703 secs 2023-09-10T00:58:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:19 smithi082 ceph-mon[98694]: pgmap v3330: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T00:58:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:19 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5930.825255 secs 2023-09-10T00:58:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:20 smithi098 ceph-mon[103124]: pgmap v3331: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T00:58:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5930.825255 secs 2023-09-10T00:58:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:20 smithi170 ceph-mon[102832]: pgmap v3331: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T00:58:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5930.825255 secs 2023-09-10T00:58:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:20 smithi082 ceph-mon[98694]: pgmap v3331: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T00:58:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:21 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:21 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:21 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:22 smithi098 ceph-mon[103124]: pgmap v3332: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T00:58:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:22 smithi170 ceph-mon[102832]: pgmap v3332: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T00:58:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:22 smithi082 ceph-mon[98694]: pgmap v3332: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T00:58:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:58:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5935.112789 secs 2023-09-10T00:58:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:58:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5935.112789 secs 2023-09-10T00:58:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:58:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5935.112789 secs 2023-09-10T00:58:24.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:58:24.762 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:58:24.762 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:24.762 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:24 smithi098 ceph-mon[103124]: pgmap v3333: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T00:58:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:58:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:58:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:24 smithi170 ceph-mon[102832]: pgmap v3333: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T00:58:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:58:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:58:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:24 smithi082 ceph-mon[98694]: pgmap v3333: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T00:58:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5935.825318 secs 2023-09-10T00:58:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5935.825318 secs 2023-09-10T00:58:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5935.825318 secs 2023-09-10T00:58:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:26 smithi170 ceph-mon[102832]: pgmap v3334: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 126 KiB/s wr, 16 op/s 2023-09-10T00:58:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:26 smithi082 ceph-mon[98694]: pgmap v3334: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 126 KiB/s wr, 16 op/s 2023-09-10T00:58:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:26 smithi098 ceph-mon[103124]: pgmap v3334: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 126 KiB/s wr, 16 op/s 2023-09-10T00:58:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:28 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5940.112876 secs 2023-09-10T00:58:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:28 smithi170 ceph-mon[102832]: pgmap v3335: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 86 KiB/s wr, 10 op/s 2023-09-10T00:58:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:28 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5940.112876 secs 2023-09-10T00:58:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:28 smithi082 ceph-mon[98694]: pgmap v3335: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 86 KiB/s wr, 10 op/s 2023-09-10T00:58:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:28 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5940.112876 secs 2023-09-10T00:58:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:28 smithi098 ceph-mon[103124]: pgmap v3335: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 86 KiB/s wr, 10 op/s 2023-09-10T00:58:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5940.825380 secs 2023-09-10T00:58:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5940.825380 secs 2023-09-10T00:58:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5940.825380 secs 2023-09-10T00:58:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:31 smithi170 ceph-mon[102832]: pgmap v3336: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 86 KiB/s wr, 10 op/s 2023-09-10T00:58:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:31 smithi082 ceph-mon[98694]: pgmap v3336: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 86 KiB/s wr, 10 op/s 2023-09-10T00:58:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:31 smithi098 ceph-mon[103124]: pgmap v3336: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 86 KiB/s wr, 10 op/s 2023-09-10T00:58:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:33 smithi170 ceph-mon[102832]: pgmap v3337: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 223 KiB/s wr, 16 op/s 2023-09-10T00:58:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:33 smithi082 ceph-mon[98694]: pgmap v3337: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 223 KiB/s wr, 16 op/s 2023-09-10T00:58:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:33 smithi098 ceph-mon[103124]: pgmap v3337: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 223 KiB/s wr, 16 op/s 2023-09-10T00:58:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5945.112939 secs 2023-09-10T00:58:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5945.112939 secs 2023-09-10T00:58:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5945.112939 secs 2023-09-10T00:58:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5945.825444 secs 2023-09-10T00:58:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:35 smithi170 ceph-mon[102832]: pgmap v3338: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 153 KiB/s wr, 12 op/s 2023-09-10T00:58:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5945.825444 secs 2023-09-10T00:58:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:35 smithi082 ceph-mon[98694]: pgmap v3338: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 153 KiB/s wr, 12 op/s 2023-09-10T00:58:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5945.825444 secs 2023-09-10T00:58:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:35 smithi098 ceph-mon[103124]: pgmap v3338: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 153 KiB/s wr, 12 op/s 2023-09-10T00:58:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:37 smithi170 ceph-mon[102832]: pgmap v3339: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 168 KiB/s wr, 17 op/s 2023-09-10T00:58:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:37 smithi082 ceph-mon[98694]: pgmap v3339: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 168 KiB/s wr, 17 op/s 2023-09-10T00:58:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:37 smithi098 ceph-mon[103124]: pgmap v3339: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 168 KiB/s wr, 17 op/s 2023-09-10T00:58:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5950.112997 secs 2023-09-10T00:58:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:39 smithi170 ceph-mon[102832]: pgmap v3340: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 12 op/s 2023-09-10T00:58:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:58:39.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:58:39.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:58:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5950.112997 secs 2023-09-10T00:58:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:39 smithi082 ceph-mon[98694]: pgmap v3340: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 12 op/s 2023-09-10T00:58:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:58:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:58:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:58:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5950.112997 secs 2023-09-10T00:58:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:39 smithi098 ceph-mon[103124]: pgmap v3340: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 12 op/s 2023-09-10T00:58:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:58:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:58:39.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:58:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5950.825490 secs 2023-09-10T00:58:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:58:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:58:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:58:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5950.825490 secs 2023-09-10T00:58:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:58:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:58:40.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:58:40.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5950.825490 secs 2023-09-10T00:58:40.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:58:40.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:58:40.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:58:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:41 smithi170 ceph-mon[102832]: pgmap v3341: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 12 op/s 2023-09-10T00:58:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:41 smithi082 ceph-mon[98694]: pgmap v3341: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 12 op/s 2023-09-10T00:58:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:41 smithi098 ceph-mon[103124]: pgmap v3341: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 12 op/s 2023-09-10T00:58:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:43 smithi170 ceph-mon[102832]: pgmap v3342: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 287 KiB/s wr, 18 op/s 2023-09-10T00:58:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:43 smithi082 ceph-mon[98694]: pgmap v3342: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 287 KiB/s wr, 18 op/s 2023-09-10T00:58:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:43 smithi098 ceph-mon[103124]: pgmap v3342: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 287 KiB/s wr, 18 op/s 2023-09-10T00:58:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5955.113056 secs 2023-09-10T00:58:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5955.113056 secs 2023-09-10T00:58:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5955.113056 secs 2023-09-10T00:58:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5955.825550 secs 2023-09-10T00:58:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:45 smithi170 ceph-mon[102832]: pgmap v3343: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T00:58:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5955.825550 secs 2023-09-10T00:58:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:45 smithi082 ceph-mon[98694]: pgmap v3343: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T00:58:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5955.825550 secs 2023-09-10T00:58:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:45 smithi098 ceph-mon[103124]: pgmap v3343: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T00:58:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:47 smithi170 ceph-mon[102832]: pgmap v3344: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 164 KiB/s wr, 17 op/s 2023-09-10T00:58:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:47 smithi082 ceph-mon[98694]: pgmap v3344: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 164 KiB/s wr, 17 op/s 2023-09-10T00:58:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:47 smithi098 ceph-mon[103124]: pgmap v3344: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 164 KiB/s wr, 17 op/s 2023-09-10T00:58:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5958.580063 secs 2023-09-10T00:58:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5958.580063 secs 2023-09-10T00:58:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5958.580063 secs 2023-09-10T00:58:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5960.113114 secs 2023-09-10T00:58:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:49 smithi170 ceph-mon[102832]: pgmap v3345: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T00:58:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5960.113114 secs 2023-09-10T00:58:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:49 smithi082 ceph-mon[98694]: pgmap v3345: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T00:58:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5960.113114 secs 2023-09-10T00:58:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:49 smithi098 ceph-mon[103124]: pgmap v3345: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T00:58:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5960.825614 secs 2023-09-10T00:58:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5960.825614 secs 2023-09-10T00:58:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5960.825614 secs 2023-09-10T00:58:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:51 smithi170 ceph-mon[102832]: pgmap v3346: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T00:58:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:51 smithi082 ceph-mon[98694]: pgmap v3346: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T00:58:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:51 smithi098 ceph-mon[103124]: pgmap v3346: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T00:58:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:58:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:58:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:58:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:58:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:58:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:58:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5963.580124 secs 2023-09-10T00:58:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:53 smithi170 ceph-mon[102832]: pgmap v3347: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 233 KiB/s wr, 16 op/s 2023-09-10T00:58:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5963.580124 secs 2023-09-10T00:58:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:53 smithi082 ceph-mon[98694]: pgmap v3347: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 233 KiB/s wr, 16 op/s 2023-09-10T00:58:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5963.580124 secs 2023-09-10T00:58:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:53 smithi098 ceph-mon[103124]: pgmap v3347: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 233 KiB/s wr, 16 op/s 2023-09-10T00:58:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5965.113172 secs 2023-09-10T00:58:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5965.113172 secs 2023-09-10T00:58:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5965.113172 secs 2023-09-10T00:58:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5965.825675 secs 2023-09-10T00:58:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:55 smithi170 ceph-mon[102832]: pgmap v3348: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T00:58:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5965.825675 secs 2023-09-10T00:58:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:55 smithi082 ceph-mon[98694]: pgmap v3348: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T00:58:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5965.825675 secs 2023-09-10T00:58:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:55 smithi098 ceph-mon[103124]: pgmap v3348: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T00:58:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:57 smithi170 ceph-mon[102832]: pgmap v3349: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 99 KiB/s wr, 15 op/s 2023-09-10T00:58:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:57 smithi082 ceph-mon[98694]: pgmap v3349: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 99 KiB/s wr, 15 op/s 2023-09-10T00:58:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:57 smithi098 ceph-mon[103124]: pgmap v3349: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 99 KiB/s wr, 15 op/s 2023-09-10T00:58:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:58:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5968.580219 secs 2023-09-10T00:58:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5968.580219 secs 2023-09-10T00:58:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5968.580219 secs 2023-09-10T00:58:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5970.113236 secs 2023-09-10T00:58:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:58:59 smithi170 ceph-mon[102832]: pgmap v3350: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 9 op/s 2023-09-10T00:58:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5970.113236 secs 2023-09-10T00:58:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:58:59 smithi082 ceph-mon[98694]: pgmap v3350: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 9 op/s 2023-09-10T00:58:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5970.113236 secs 2023-09-10T00:58:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:58:59 smithi098 ceph-mon[103124]: pgmap v3350: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 9 op/s 2023-09-10T00:59:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5970.825737 secs 2023-09-10T00:59:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5970.825737 secs 2023-09-10T00:59:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5970.825737 secs 2023-09-10T00:59:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:01 smithi170 ceph-mon[102832]: pgmap v3351: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 9 op/s 2023-09-10T00:59:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:01 smithi082 ceph-mon[98694]: pgmap v3351: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 9 op/s 2023-09-10T00:59:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:01 smithi098 ceph-mon[103124]: pgmap v3351: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 9 op/s 2023-09-10T00:59:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5973.580280 secs 2023-09-10T00:59:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:03 smithi170 ceph-mon[102832]: pgmap v3352: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 15 op/s 2023-09-10T00:59:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5973.580280 secs 2023-09-10T00:59:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:03 smithi082 ceph-mon[98694]: pgmap v3352: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 15 op/s 2023-09-10T00:59:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5973.580280 secs 2023-09-10T00:59:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:03 smithi098 ceph-mon[103124]: pgmap v3352: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 15 op/s 2023-09-10T00:59:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5975.113332 secs 2023-09-10T00:59:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5975.113332 secs 2023-09-10T00:59:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5975.113332 secs 2023-09-10T00:59:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5975.825799 secs 2023-09-10T00:59:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:05 smithi170 ceph-mon[102832]: pgmap v3353: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 27 KiB/s wr, 9 op/s 2023-09-10T00:59:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5975.825799 secs 2023-09-10T00:59:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:05 smithi082 ceph-mon[98694]: pgmap v3353: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 27 KiB/s wr, 9 op/s 2023-09-10T00:59:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5975.825799 secs 2023-09-10T00:59:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:05 smithi098 ceph-mon[103124]: pgmap v3353: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 27 KiB/s wr, 9 op/s 2023-09-10T00:59:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:07 smithi170 ceph-mon[102832]: pgmap v3354: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-10T00:59:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:07 smithi082 ceph-mon[98694]: pgmap v3354: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-10T00:59:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:07 smithi098 ceph-mon[103124]: pgmap v3354: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-10T00:59:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5978.580383 secs 2023-09-10T00:59:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:08 smithi170 ceph-mon[102832]: osdmap e281: 12 total, 12 up, 12 in 2023-09-10T00:59:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5978.580383 secs 2023-09-10T00:59:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:08 smithi082 ceph-mon[98694]: osdmap e281: 12 total, 12 up, 12 in 2023-09-10T00:59:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5978.580383 secs 2023-09-10T00:59:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:08 smithi098 ceph-mon[103124]: osdmap e281: 12 total, 12 up, 12 in 2023-09-10T00:59:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5980.113417 secs 2023-09-10T00:59:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:09 smithi170 ceph-mon[102832]: pgmap v3356: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:59:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5980.113417 secs 2023-09-10T00:59:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:09 smithi082 ceph-mon[98694]: pgmap v3356: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:59:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5980.113417 secs 2023-09-10T00:59:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:09 smithi098 ceph-mon[103124]: pgmap v3356: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:59:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5980.825861 secs 2023-09-10T00:59:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5980.825861 secs 2023-09-10T00:59:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5980.825861 secs 2023-09-10T00:59:11.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:11 smithi082 ceph-mon[98694]: pgmap v3357: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:59:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:11 smithi098 ceph-mon[103124]: pgmap v3357: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:59:11.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:11 smithi170 ceph-mon[102832]: pgmap v3357: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T00:59:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5983.580469 secs 2023-09-10T00:59:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:13 smithi082 ceph-mon[98694]: pgmap v3358: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 118 KiB/s wr, 29 op/s 2023-09-10T00:59:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5983.580469 secs 2023-09-10T00:59:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:13 smithi098 ceph-mon[103124]: pgmap v3358: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 118 KiB/s wr, 29 op/s 2023-09-10T00:59:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5983.580469 secs 2023-09-10T00:59:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:13 smithi170 ceph-mon[102832]: pgmap v3358: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 118 KiB/s wr, 29 op/s 2023-09-10T00:59:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5985.113476 secs 2023-09-10T00:59:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5985.113476 secs 2023-09-10T00:59:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5985.113476 secs 2023-09-10T00:59:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5985.825924 secs 2023-09-10T00:59:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:15 smithi082 ceph-mon[98694]: pgmap v3359: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 118 KiB/s wr, 29 op/s 2023-09-10T00:59:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:15 smithi082 ceph-mon[98694]: osdmap e282: 12 total, 12 up, 12 in 2023-09-10T00:59:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5985.825924 secs 2023-09-10T00:59:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:15 smithi098 ceph-mon[103124]: pgmap v3359: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 118 KiB/s wr, 29 op/s 2023-09-10T00:59:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:15 smithi098 ceph-mon[103124]: osdmap e282: 12 total, 12 up, 12 in 2023-09-10T00:59:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5985.825924 secs 2023-09-10T00:59:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:15 smithi170 ceph-mon[102832]: pgmap v3359: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 118 KiB/s wr, 29 op/s 2023-09-10T00:59:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:15 smithi170 ceph-mon[102832]: osdmap e282: 12 total, 12 up, 12 in 2023-09-10T00:59:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:17 smithi082 ceph-mon[98694]: pgmap v3361: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T00:59:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:17 smithi098 ceph-mon[103124]: pgmap v3361: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T00:59:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:17 smithi170 ceph-mon[102832]: pgmap v3361: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T00:59:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 5988.580530 secs 2023-09-10T00:59:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 5988.580530 secs 2023-09-10T00:59:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 5988.580530 secs 2023-09-10T00:59:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5990.113495 secs 2023-09-10T00:59:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:19 smithi082 ceph-mon[98694]: pgmap v3362: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T00:59:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5990.113495 secs 2023-09-10T00:59:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:19 smithi098 ceph-mon[103124]: pgmap v3362: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T00:59:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5990.113495 secs 2023-09-10T00:59:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:19 smithi170 ceph-mon[102832]: pgmap v3362: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T00:59:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5990.825987 secs 2023-09-10T00:59:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5990.825987 secs 2023-09-10T00:59:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5990.825987 secs 2023-09-10T00:59:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:21 smithi098 ceph-mon[103124]: pgmap v3363: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T00:59:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:21 smithi170 ceph-mon[102832]: pgmap v3363: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T00:59:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:21 smithi082 ceph-mon[98694]: pgmap v3363: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T00:59:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:23 smithi098 ceph-mon[103124]: pgmap v3364: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T00:59:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:59:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:23 smithi170 ceph-mon[102832]: pgmap v3364: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T00:59:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:59:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:23 smithi082 ceph-mon[98694]: pgmap v3364: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T00:59:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T00:59:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5995.113555 secs 2023-09-10T00:59:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:59:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:59:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:59:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5995.113555 secs 2023-09-10T00:59:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:59:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:59:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:59:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5995.113555 secs 2023-09-10T00:59:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:59:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:59:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:59:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 5995.826050 secs 2023-09-10T00:59:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:25 smithi098 ceph-mon[103124]: pgmap v3365: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T00:59:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 5995.826050 secs 2023-09-10T00:59:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:25 smithi170 ceph-mon[102832]: pgmap v3365: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T00:59:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 5995.826050 secs 2023-09-10T00:59:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:25 smithi082 ceph-mon[98694]: pgmap v3365: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T00:59:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:27 smithi098 ceph-mon[103124]: pgmap v3366: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 127 KiB/s wr, 17 op/s 2023-09-10T00:59:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:27 smithi170 ceph-mon[102832]: pgmap v3366: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 127 KiB/s wr, 17 op/s 2023-09-10T00:59:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:27 smithi082 ceph-mon[98694]: pgmap v3366: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 127 KiB/s wr, 17 op/s 2023-09-10T00:59:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6000.113616 secs 2023-09-10T00:59:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:29 smithi098 ceph-mon[103124]: pgmap v3367: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 10 op/s 2023-09-10T00:59:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6000.113616 secs 2023-09-10T00:59:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:29 smithi170 ceph-mon[102832]: pgmap v3367: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 10 op/s 2023-09-10T00:59:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6000.113616 secs 2023-09-10T00:59:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:29 smithi082 ceph-mon[98694]: pgmap v3367: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 10 op/s 2023-09-10T00:59:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6000.826111 secs 2023-09-10T00:59:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6000.826111 secs 2023-09-10T00:59:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6000.826111 secs 2023-09-10T00:59:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:31 smithi098 ceph-mon[103124]: pgmap v3368: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 10 op/s 2023-09-10T00:59:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:31 smithi170 ceph-mon[102832]: pgmap v3368: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 10 op/s 2023-09-10T00:59:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:31 smithi082 ceph-mon[98694]: pgmap v3368: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 10 op/s 2023-09-10T00:59:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:33 smithi098 ceph-mon[103124]: pgmap v3369: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 184 KiB/s wr, 16 op/s 2023-09-10T00:59:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:33 smithi170 ceph-mon[102832]: pgmap v3369: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 184 KiB/s wr, 16 op/s 2023-09-10T00:59:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:33 smithi082 ceph-mon[98694]: pgmap v3369: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 184 KiB/s wr, 16 op/s 2023-09-10T00:59:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6005.113676 secs 2023-09-10T00:59:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6005.113676 secs 2023-09-10T00:59:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6005.113676 secs 2023-09-10T00:59:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6005.826173 secs 2023-09-10T00:59:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:35 smithi170 ceph-mon[102832]: pgmap v3370: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T00:59:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6005.826173 secs 2023-09-10T00:59:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:35 smithi082 ceph-mon[98694]: pgmap v3370: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T00:59:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6005.826173 secs 2023-09-10T00:59:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:35 smithi098 ceph-mon[103124]: pgmap v3370: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T00:59:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:37 smithi170 ceph-mon[102832]: pgmap v3371: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 240 KiB/s wr, 17 op/s 2023-09-10T00:59:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:37 smithi082 ceph-mon[98694]: pgmap v3371: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 240 KiB/s wr, 17 op/s 2023-09-10T00:59:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:37 smithi098 ceph-mon[103124]: pgmap v3371: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 240 KiB/s wr, 17 op/s 2023-09-10T00:59:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6010.113738 secs 2023-09-10T00:59:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:39 smithi170 ceph-mon[102832]: pgmap v3372: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 12 op/s 2023-09-10T00:59:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6010.113738 secs 2023-09-10T00:59:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:39 smithi082 ceph-mon[98694]: pgmap v3372: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 12 op/s 2023-09-10T00:59:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6010.113738 secs 2023-09-10T00:59:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:39 smithi098 ceph-mon[103124]: pgmap v3372: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 12 op/s 2023-09-10T00:59:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6010.826237 secs 2023-09-10T00:59:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:59:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:59:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:59:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6010.826237 secs 2023-09-10T00:59:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:59:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:59:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:59:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6010.826237 secs 2023-09-10T00:59:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T00:59:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:59:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:59:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:41 smithi170 ceph-mon[102832]: pgmap v3373: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 12 op/s 2023-09-10T00:59:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:59:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:59:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:59:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:59:41.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:59:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:41 smithi082 ceph-mon[98694]: pgmap v3373: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 12 op/s 2023-09-10T00:59:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:59:41.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:59:41.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:59:41.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:59:41.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:59:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:41 smithi098 ceph-mon[103124]: pgmap v3373: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 12 op/s 2023-09-10T00:59:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:59:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:59:41.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T00:59:41.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T00:59:41.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T00:59:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:43 smithi170 ceph-mon[102832]: pgmap v3374: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 238 KiB/s wr, 17 op/s 2023-09-10T00:59:43.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:43 smithi082 ceph-mon[98694]: pgmap v3374: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 238 KiB/s wr, 17 op/s 2023-09-10T00:59:43.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:43 smithi098 ceph-mon[103124]: pgmap v3374: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 238 KiB/s wr, 17 op/s 2023-09-10T00:59:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6015.113795 secs 2023-09-10T00:59:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6015.113795 secs 2023-09-10T00:59:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6015.113795 secs 2023-09-10T00:59:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6015.826299 secs 2023-09-10T00:59:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:45 smithi170 ceph-mon[102832]: pgmap v3375: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 170 KiB/s wr, 11 op/s 2023-09-10T00:59:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6015.826299 secs 2023-09-10T00:59:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:45 smithi082 ceph-mon[98694]: pgmap v3375: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 170 KiB/s wr, 11 op/s 2023-09-10T00:59:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6015.826299 secs 2023-09-10T00:59:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:45 smithi098 ceph-mon[103124]: pgmap v3375: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 170 KiB/s wr, 11 op/s 2023-09-10T00:59:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:47 smithi170 ceph-mon[102832]: pgmap v3376: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 289 KiB/s wr, 18 op/s 2023-09-10T00:59:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:47 smithi082 ceph-mon[98694]: pgmap v3376: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 289 KiB/s wr, 18 op/s 2023-09-10T00:59:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:47 smithi098 ceph-mon[103124]: pgmap v3376: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 289 KiB/s wr, 18 op/s 2023-09-10T00:59:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6018.580896 secs 2023-09-10T00:59:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6018.580896 secs 2023-09-10T00:59:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6018.580896 secs 2023-09-10T00:59:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6020.113871 secs 2023-09-10T00:59:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:49 smithi170 ceph-mon[102832]: pgmap v3377: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 203 KiB/s wr, 12 op/s 2023-09-10T00:59:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6020.113871 secs 2023-09-10T00:59:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:49 smithi082 ceph-mon[98694]: pgmap v3377: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 203 KiB/s wr, 12 op/s 2023-09-10T00:59:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6020.113871 secs 2023-09-10T00:59:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:49 smithi098 ceph-mon[103124]: pgmap v3377: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 203 KiB/s wr, 12 op/s 2023-09-10T00:59:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6020.826377 secs 2023-09-10T00:59:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6020.826377 secs 2023-09-10T00:59:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6020.826377 secs 2023-09-10T00:59:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:51 smithi170 ceph-mon[102832]: pgmap v3378: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 203 KiB/s wr, 12 op/s 2023-09-10T00:59:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:51 smithi082 ceph-mon[98694]: pgmap v3378: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 203 KiB/s wr, 12 op/s 2023-09-10T00:59:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:51 smithi098 ceph-mon[103124]: pgmap v3378: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 203 KiB/s wr, 12 op/s 2023-09-10T00:59:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:59:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:59:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:59:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:59:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:59:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:59:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T00:59:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T00:59:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T00:59:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6023.580956 secs 2023-09-10T00:59:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:53 smithi170 ceph-mon[102832]: pgmap v3379: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 283 KiB/s wr, 18 op/s 2023-09-10T00:59:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6023.580956 secs 2023-09-10T00:59:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:53 smithi082 ceph-mon[98694]: pgmap v3379: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 283 KiB/s wr, 18 op/s 2023-09-10T00:59:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6023.580956 secs 2023-09-10T00:59:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:53 smithi098 ceph-mon[103124]: pgmap v3379: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 283 KiB/s wr, 18 op/s 2023-09-10T00:59:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6025.113929 secs 2023-09-10T00:59:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6025.113929 secs 2023-09-10T00:59:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6025.113929 secs 2023-09-10T00:59:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6025.826442 secs 2023-09-10T00:59:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:55 smithi170 ceph-mon[102832]: pgmap v3380: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 200 KiB/s wr, 12 op/s 2023-09-10T00:59:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6025.826442 secs 2023-09-10T00:59:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:55 smithi082 ceph-mon[98694]: pgmap v3380: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 200 KiB/s wr, 12 op/s 2023-09-10T00:59:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6025.826442 secs 2023-09-10T00:59:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:55 smithi098 ceph-mon[103124]: pgmap v3380: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 200 KiB/s wr, 12 op/s 2023-09-10T00:59:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:57 smithi170 ceph-mon[102832]: pgmap v3381: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 204 KiB/s wr, 18 op/s 2023-09-10T00:59:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:57 smithi082 ceph-mon[98694]: pgmap v3381: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 204 KiB/s wr, 18 op/s 2023-09-10T00:59:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:57 smithi098 ceph-mon[103124]: pgmap v3381: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 204 KiB/s wr, 18 op/s 2023-09-10T00:59:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6028.581043 secs 2023-09-10T00:59:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6028.581043 secs 2023-09-10T00:59:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6028.581043 secs 2023-09-10T00:59:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6030.114010 secs 2023-09-10T00:59:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 00:59:59 smithi170 ceph-mon[102832]: pgmap v3382: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T00:59:59.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6030.114010 secs 2023-09-10T00:59:59.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 00:59:59 smithi082 ceph-mon[98694]: pgmap v3382: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T00:59:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6030.114010 secs 2023-09-10T00:59:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 00:59:59 smithi098 ceph-mon[103124]: pgmap v3382: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T01:00:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6030.826489 secs 2023-09-10T01:00:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T01:00:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6030.826489 secs 2023-09-10T01:00:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T01:00:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6030.826489 secs 2023-09-10T01:00:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T01:00:01.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:01 smithi170 ceph-mon[102832]: pgmap v3383: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T01:00:01.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:01 smithi082 ceph-mon[98694]: pgmap v3383: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T01:00:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:01 smithi098 ceph-mon[103124]: pgmap v3383: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T01:00:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6033.581103 secs 2023-09-10T01:00:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:03 smithi170 ceph-mon[102832]: pgmap v3384: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 108 KiB/s wr, 16 op/s 2023-09-10T01:00:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6033.581103 secs 2023-09-10T01:00:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:03 smithi082 ceph-mon[98694]: pgmap v3384: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 108 KiB/s wr, 16 op/s 2023-09-10T01:00:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6033.581103 secs 2023-09-10T01:00:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:03 smithi098 ceph-mon[103124]: pgmap v3384: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 108 KiB/s wr, 16 op/s 2023-09-10T01:00:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6035.114089 secs 2023-09-10T01:00:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6035.114089 secs 2023-09-10T01:00:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6035.114089 secs 2023-09-10T01:00:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6035.826548 secs 2023-09-10T01:00:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:05 smithi170 ceph-mon[102832]: pgmap v3385: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 28 KiB/s wr, 10 op/s 2023-09-10T01:00:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6035.826548 secs 2023-09-10T01:00:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:05 smithi082 ceph-mon[98694]: pgmap v3385: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 28 KiB/s wr, 10 op/s 2023-09-10T01:00:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6035.826548 secs 2023-09-10T01:00:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:05 smithi098 ceph-mon[103124]: pgmap v3385: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 28 KiB/s wr, 10 op/s 2023-09-10T01:00:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:07 smithi170 ceph-mon[102832]: pgmap v3386: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 108 KiB/s wr, 30 op/s 2023-09-10T01:00:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:07 smithi082 ceph-mon[98694]: pgmap v3386: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 108 KiB/s wr, 30 op/s 2023-09-10T01:00:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:07 smithi098 ceph-mon[103124]: pgmap v3386: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 108 KiB/s wr, 30 op/s 2023-09-10T01:00:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6038.581193 secs 2023-09-10T01:00:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:08 smithi082 ceph-mon[98694]: osdmap e283: 12 total, 12 up, 12 in 2023-09-10T01:00:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6038.581193 secs 2023-09-10T01:00:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:08 smithi098 ceph-mon[103124]: osdmap e283: 12 total, 12 up, 12 in 2023-09-10T01:00:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6038.581193 secs 2023-09-10T01:00:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:08 smithi170 ceph-mon[102832]: osdmap e283: 12 total, 12 up, 12 in 2023-09-10T01:00:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6040.114148 secs 2023-09-10T01:00:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:09 smithi082 ceph-mon[98694]: pgmap v3388: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 124 KiB/s wr, 29 op/s 2023-09-10T01:00:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6040.114148 secs 2023-09-10T01:00:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:09 smithi098 ceph-mon[103124]: pgmap v3388: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 124 KiB/s wr, 29 op/s 2023-09-10T01:00:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6040.114148 secs 2023-09-10T01:00:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:09 smithi170 ceph-mon[102832]: pgmap v3388: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 124 KiB/s wr, 29 op/s 2023-09-10T01:00:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6040.826607 secs 2023-09-10T01:00:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6040.826607 secs 2023-09-10T01:00:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6040.826607 secs 2023-09-10T01:00:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:11 smithi082 ceph-mon[98694]: pgmap v3389: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 124 KiB/s wr, 29 op/s 2023-09-10T01:00:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:11 smithi098 ceph-mon[103124]: pgmap v3389: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 124 KiB/s wr, 29 op/s 2023-09-10T01:00:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:11 smithi170 ceph-mon[102832]: pgmap v3389: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 124 KiB/s wr, 29 op/s 2023-09-10T01:00:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6043.581253 secs 2023-09-10T01:00:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:13 smithi082 ceph-mon[98694]: pgmap v3390: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 119 KiB/s wr, 29 op/s 2023-09-10T01:00:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6043.581253 secs 2023-09-10T01:00:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:13 smithi098 ceph-mon[103124]: pgmap v3390: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 119 KiB/s wr, 29 op/s 2023-09-10T01:00:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6043.581253 secs 2023-09-10T01:00:13.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:13 smithi170 ceph-mon[102832]: pgmap v3390: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 119 KiB/s wr, 29 op/s 2023-09-10T01:00:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6045.114206 secs 2023-09-10T01:00:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6045.114206 secs 2023-09-10T01:00:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6045.114206 secs 2023-09-10T01:00:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6045.826670 secs 2023-09-10T01:00:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:15 smithi082 ceph-mon[98694]: pgmap v3391: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 119 KiB/s wr, 29 op/s 2023-09-10T01:00:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:15 smithi082 ceph-mon[98694]: osdmap e284: 12 total, 12 up, 12 in 2023-09-10T01:00:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6045.826670 secs 2023-09-10T01:00:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:15 smithi098 ceph-mon[103124]: pgmap v3391: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 119 KiB/s wr, 29 op/s 2023-09-10T01:00:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:15 smithi098 ceph-mon[103124]: osdmap e284: 12 total, 12 up, 12 in 2023-09-10T01:00:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6045.826670 secs 2023-09-10T01:00:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:15 smithi170 ceph-mon[102832]: pgmap v3391: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 119 KiB/s wr, 29 op/s 2023-09-10T01:00:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:15 smithi170 ceph-mon[102832]: osdmap e284: 12 total, 12 up, 12 in 2023-09-10T01:00:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:17 smithi082 ceph-mon[98694]: pgmap v3393: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 67 KiB/s wr, 13 op/s 2023-09-10T01:00:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:17 smithi098 ceph-mon[103124]: pgmap v3393: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 67 KiB/s wr, 13 op/s 2023-09-10T01:00:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:17 smithi170 ceph-mon[102832]: pgmap v3393: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 67 KiB/s wr, 13 op/s 2023-09-10T01:00:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6048.581314 secs 2023-09-10T01:00:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6048.581314 secs 2023-09-10T01:00:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6048.581314 secs 2023-09-10T01:00:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6050.114274 secs 2023-09-10T01:00:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:19 smithi098 ceph-mon[103124]: pgmap v3394: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T01:00:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6050.114274 secs 2023-09-10T01:00:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:19 smithi170 ceph-mon[102832]: pgmap v3394: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T01:00:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6050.114274 secs 2023-09-10T01:00:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:19 smithi082 ceph-mon[98694]: pgmap v3394: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T01:00:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6050.826731 secs 2023-09-10T01:00:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6050.826731 secs 2023-09-10T01:00:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6050.826731 secs 2023-09-10T01:00:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:21 smithi098 ceph-mon[103124]: pgmap v3395: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T01:00:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:21 smithi170 ceph-mon[102832]: pgmap v3395: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T01:00:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:21 smithi082 ceph-mon[98694]: pgmap v3395: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T01:00:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:23 smithi098 ceph-mon[103124]: pgmap v3396: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 109 KiB/s wr, 11 op/s 2023-09-10T01:00:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:00:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:23 smithi170 ceph-mon[102832]: pgmap v3396: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 109 KiB/s wr, 11 op/s 2023-09-10T01:00:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:00:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:23 smithi082 ceph-mon[98694]: pgmap v3396: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 109 KiB/s wr, 11 op/s 2023-09-10T01:00:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:00:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6055.114329 secs 2023-09-10T01:00:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:00:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:00:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:00:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6055.114329 secs 2023-09-10T01:00:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:00:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:00:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:00:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6055.114329 secs 2023-09-10T01:00:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:00:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:00:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:00:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6055.826793 secs 2023-09-10T01:00:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:25 smithi098 ceph-mon[103124]: pgmap v3397: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 109 KiB/s wr, 11 op/s 2023-09-10T01:00:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6055.826793 secs 2023-09-10T01:00:25.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:25 smithi170 ceph-mon[102832]: pgmap v3397: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 109 KiB/s wr, 11 op/s 2023-09-10T01:00:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6055.826793 secs 2023-09-10T01:00:25.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:25 smithi082 ceph-mon[98694]: pgmap v3397: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 109 KiB/s wr, 11 op/s 2023-09-10T01:00:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:27 smithi098 ceph-mon[103124]: pgmap v3398: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 128 KiB/s wr, 17 op/s 2023-09-10T01:00:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:27 smithi170 ceph-mon[102832]: pgmap v3398: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 128 KiB/s wr, 17 op/s 2023-09-10T01:00:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:27 smithi082 ceph-mon[98694]: pgmap v3398: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 128 KiB/s wr, 17 op/s 2023-09-10T01:00:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6060.114387 secs 2023-09-10T01:00:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:29 smithi098 ceph-mon[103124]: pgmap v3399: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T01:00:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6060.114387 secs 2023-09-10T01:00:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:29 smithi170 ceph-mon[102832]: pgmap v3399: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T01:00:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6060.114387 secs 2023-09-10T01:00:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:29 smithi082 ceph-mon[98694]: pgmap v3399: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T01:00:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6060.826859 secs 2023-09-10T01:00:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6060.826859 secs 2023-09-10T01:00:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6060.826859 secs 2023-09-10T01:00:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:31 smithi098 ceph-mon[103124]: pgmap v3400: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T01:00:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:31 smithi170 ceph-mon[102832]: pgmap v3400: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T01:00:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:31 smithi082 ceph-mon[98694]: pgmap v3400: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T01:00:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:33 smithi098 ceph-mon[103124]: pgmap v3401: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 161 KiB/s wr, 17 op/s 2023-09-10T01:00:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:33 smithi170 ceph-mon[102832]: pgmap v3401: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 161 KiB/s wr, 17 op/s 2023-09-10T01:00:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:33 smithi082 ceph-mon[98694]: pgmap v3401: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 161 KiB/s wr, 17 op/s 2023-09-10T01:00:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6065.114471 secs 2023-09-10T01:00:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:34 smithi170 ceph-mon[102832]: pgmap v3402: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 100 KiB/s wr, 12 op/s 2023-09-10T01:00:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6065.114471 secs 2023-09-10T01:00:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:34 smithi082 ceph-mon[98694]: pgmap v3402: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 100 KiB/s wr, 12 op/s 2023-09-10T01:00:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6065.114471 secs 2023-09-10T01:00:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:34 smithi098 ceph-mon[103124]: pgmap v3402: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 100 KiB/s wr, 12 op/s 2023-09-10T01:00:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6065.826922 secs 2023-09-10T01:00:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6065.826922 secs 2023-09-10T01:00:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6065.826922 secs 2023-09-10T01:00:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:36 smithi170 ceph-mon[102832]: pgmap v3403: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 182 KiB/s wr, 18 op/s 2023-09-10T01:00:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:36 smithi082 ceph-mon[98694]: pgmap v3403: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 182 KiB/s wr, 18 op/s 2023-09-10T01:00:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:36 smithi098 ceph-mon[103124]: pgmap v3403: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 182 KiB/s wr, 18 op/s 2023-09-10T01:00:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6070.114510 secs 2023-09-10T01:00:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:39 smithi170 ceph-mon[102832]: pgmap v3404: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T01:00:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6070.114510 secs 2023-09-10T01:00:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:39 smithi082 ceph-mon[98694]: pgmap v3404: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T01:00:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6070.114510 secs 2023-09-10T01:00:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:39 smithi098 ceph-mon[103124]: pgmap v3404: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T01:00:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6070.826986 secs 2023-09-10T01:00:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6070.826986 secs 2023-09-10T01:00:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6070.826986 secs 2023-09-10T01:00:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:41 smithi170 ceph-mon[102832]: pgmap v3405: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T01:00:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:00:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:41 smithi082 ceph-mon[98694]: pgmap v3405: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T01:00:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:00:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:41 smithi098 ceph-mon[103124]: pgmap v3405: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T01:00:41.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:00:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:00:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:00:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:00:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:00:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:00:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:00:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:00:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:00:42.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:00:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:43 smithi170 ceph-mon[102832]: pgmap v3406: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 219 KiB/s wr, 17 op/s 2023-09-10T01:00:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:43 smithi082 ceph-mon[98694]: pgmap v3406: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 219 KiB/s wr, 17 op/s 2023-09-10T01:00:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:43 smithi098 ceph-mon[103124]: pgmap v3406: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 219 KiB/s wr, 17 op/s 2023-09-10T01:00:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6075.114568 secs 2023-09-10T01:00:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6075.114568 secs 2023-09-10T01:00:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6075.114568 secs 2023-09-10T01:00:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6075.827047 secs 2023-09-10T01:00:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:45 smithi170 ceph-mon[102832]: pgmap v3407: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T01:00:45.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6075.827047 secs 2023-09-10T01:00:45.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:45 smithi082 ceph-mon[98694]: pgmap v3407: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T01:00:45.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6075.827047 secs 2023-09-10T01:00:45.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:45 smithi098 ceph-mon[103124]: pgmap v3407: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T01:00:47.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:47 smithi170 ceph-mon[102832]: pgmap v3408: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 233 KiB/s wr, 17 op/s 2023-09-10T01:00:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:47 smithi082 ceph-mon[98694]: pgmap v3408: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 233 KiB/s wr, 17 op/s 2023-09-10T01:00:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:47 smithi098 ceph-mon[103124]: pgmap v3408: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 233 KiB/s wr, 17 op/s 2023-09-10T01:00:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6078.581754 secs 2023-09-10T01:00:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6078.581754 secs 2023-09-10T01:00:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6078.581754 secs 2023-09-10T01:00:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6080.114630 secs 2023-09-10T01:00:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:49 smithi170 ceph-mon[102832]: pgmap v3409: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T01:00:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6080.114630 secs 2023-09-10T01:00:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:49 smithi082 ceph-mon[98694]: pgmap v3409: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T01:00:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6080.114630 secs 2023-09-10T01:00:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:49 smithi098 ceph-mon[103124]: pgmap v3409: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T01:00:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6080.827110 secs 2023-09-10T01:00:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6080.827110 secs 2023-09-10T01:00:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6080.827110 secs 2023-09-10T01:00:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:51 smithi170 ceph-mon[102832]: pgmap v3410: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T01:00:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:51 smithi082 ceph-mon[98694]: pgmap v3410: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T01:00:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:51 smithi098 ceph-mon[103124]: pgmap v3410: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T01:00:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:00:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:00:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:00:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:00:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:00:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:00:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:00:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:00:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:00:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6083.581775 secs 2023-09-10T01:00:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:53 smithi082 ceph-mon[98694]: pgmap v3411: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 183 KiB/s wr, 16 op/s 2023-09-10T01:00:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6083.581775 secs 2023-09-10T01:00:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:53 smithi098 ceph-mon[103124]: pgmap v3411: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 183 KiB/s wr, 16 op/s 2023-09-10T01:00:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6083.581775 secs 2023-09-10T01:00:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:53 smithi170 ceph-mon[102832]: pgmap v3411: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 183 KiB/s wr, 16 op/s 2023-09-10T01:00:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6085.114716 secs 2023-09-10T01:00:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6085.114716 secs 2023-09-10T01:00:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6085.114716 secs 2023-09-10T01:00:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6085.827170 secs 2023-09-10T01:00:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:55 smithi082 ceph-mon[98694]: pgmap v3412: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 10 op/s 2023-09-10T01:00:55.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6085.827170 secs 2023-09-10T01:00:55.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:55 smithi098 ceph-mon[103124]: pgmap v3412: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 10 op/s 2023-09-10T01:00:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6085.827170 secs 2023-09-10T01:00:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:55 smithi170 ceph-mon[102832]: pgmap v3412: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 10 op/s 2023-09-10T01:00:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:57 smithi082 ceph-mon[98694]: pgmap v3413: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 174 KiB/s wr, 16 op/s 2023-09-10T01:00:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:57 smithi098 ceph-mon[103124]: pgmap v3413: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 174 KiB/s wr, 16 op/s 2023-09-10T01:00:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:57 smithi170 ceph-mon[102832]: pgmap v3413: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 174 KiB/s wr, 16 op/s 2023-09-10T01:00:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6088.581853 secs 2023-09-10T01:00:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6088.581853 secs 2023-09-10T01:00:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6088.581853 secs 2023-09-10T01:00:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6090.114773 secs 2023-09-10T01:00:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:00:59 smithi082 ceph-mon[98694]: pgmap v3414: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T01:00:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6090.114773 secs 2023-09-10T01:00:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:00:59 smithi098 ceph-mon[103124]: pgmap v3414: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T01:00:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6090.114773 secs 2023-09-10T01:00:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:00:59 smithi170 ceph-mon[102832]: pgmap v3414: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T01:01:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6090.827233 secs 2023-09-10T01:01:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6090.827233 secs 2023-09-10T01:01:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6090.827233 secs 2023-09-10T01:01:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:01 smithi082 ceph-mon[98694]: pgmap v3415: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T01:01:01.485 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:01 smithi170 ceph-mon[102832]: pgmap v3415: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T01:01:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:01 smithi098 ceph-mon[103124]: pgmap v3415: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T01:01:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6093.581912 secs 2023-09-10T01:01:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:03 smithi098 ceph-mon[103124]: pgmap v3416: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 158 KiB/s wr, 31 op/s 2023-09-10T01:01:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6093.581912 secs 2023-09-10T01:01:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:03 smithi170 ceph-mon[102832]: pgmap v3416: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 158 KiB/s wr, 31 op/s 2023-09-10T01:01:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6093.581912 secs 2023-09-10T01:01:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:03 smithi082 ceph-mon[98694]: pgmap v3416: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 158 KiB/s wr, 31 op/s 2023-09-10T01:01:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6095.114832 secs 2023-09-10T01:01:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6095.114832 secs 2023-09-10T01:01:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6095.114832 secs 2023-09-10T01:01:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6095.827294 secs 2023-09-10T01:01:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:05 smithi098 ceph-mon[103124]: pgmap v3417: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 126 KiB/s wr, 25 op/s 2023-09-10T01:01:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6095.827294 secs 2023-09-10T01:01:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:05 smithi170 ceph-mon[102832]: pgmap v3417: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 126 KiB/s wr, 25 op/s 2023-09-10T01:01:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6095.827294 secs 2023-09-10T01:01:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:05 smithi082 ceph-mon[98694]: pgmap v3417: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 126 KiB/s wr, 25 op/s 2023-09-10T01:01:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:07 smithi098 ceph-mon[103124]: pgmap v3418: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 127 KiB/s wr, 31 op/s 2023-09-10T01:01:07.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:07 smithi170 ceph-mon[102832]: pgmap v3418: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 127 KiB/s wr, 31 op/s 2023-09-10T01:01:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:07 smithi082 ceph-mon[98694]: pgmap v3418: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 127 KiB/s wr, 31 op/s 2023-09-10T01:01:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6098.581971 secs 2023-09-10T01:01:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:08 smithi098 ceph-mon[103124]: osdmap e285: 12 total, 12 up, 12 in 2023-09-10T01:01:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6098.581971 secs 2023-09-10T01:01:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:08 smithi170 ceph-mon[102832]: osdmap e285: 12 total, 12 up, 12 in 2023-09-10T01:01:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6098.581971 secs 2023-09-10T01:01:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:08 smithi082 ceph-mon[98694]: osdmap e285: 12 total, 12 up, 12 in 2023-09-10T01:01:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6100.114891 secs 2023-09-10T01:01:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:09 smithi098 ceph-mon[103124]: pgmap v3420: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 30 op/s 2023-09-10T01:01:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6100.114891 secs 2023-09-10T01:01:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:09 smithi170 ceph-mon[102832]: pgmap v3420: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 30 op/s 2023-09-10T01:01:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6100.114891 secs 2023-09-10T01:01:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:09 smithi082 ceph-mon[98694]: pgmap v3420: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 30 op/s 2023-09-10T01:01:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6100.827355 secs 2023-09-10T01:01:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6100.827355 secs 2023-09-10T01:01:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6100.827355 secs 2023-09-10T01:01:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:11 smithi098 ceph-mon[103124]: pgmap v3421: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 30 op/s 2023-09-10T01:01:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:11 smithi170 ceph-mon[102832]: pgmap v3421: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 30 op/s 2023-09-10T01:01:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:11 smithi082 ceph-mon[98694]: pgmap v3421: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 30 op/s 2023-09-10T01:01:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6103.582032 secs 2023-09-10T01:01:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:13 smithi098 ceph-mon[103124]: pgmap v3422: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T01:01:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6103.582032 secs 2023-09-10T01:01:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:13 smithi170 ceph-mon[102832]: pgmap v3422: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T01:01:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6103.582032 secs 2023-09-10T01:01:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:13 smithi082 ceph-mon[98694]: pgmap v3422: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T01:01:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6105.114948 secs 2023-09-10T01:01:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6105.114948 secs 2023-09-10T01:01:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6105.114948 secs 2023-09-10T01:01:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6105.827415 secs 2023-09-10T01:01:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:15 smithi098 ceph-mon[103124]: pgmap v3423: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T01:01:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:15 smithi098 ceph-mon[103124]: osdmap e286: 12 total, 12 up, 12 in 2023-09-10T01:01:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6105.827415 secs 2023-09-10T01:01:15.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:15 smithi170 ceph-mon[102832]: pgmap v3423: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T01:01:15.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:15 smithi170 ceph-mon[102832]: osdmap e286: 12 total, 12 up, 12 in 2023-09-10T01:01:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6105.827415 secs 2023-09-10T01:01:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:15 smithi082 ceph-mon[98694]: pgmap v3423: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T01:01:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:15 smithi082 ceph-mon[98694]: osdmap e286: 12 total, 12 up, 12 in 2023-09-10T01:01:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:17 smithi098 ceph-mon[103124]: pgmap v3425: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 13 op/s 2023-09-10T01:01:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:17 smithi170 ceph-mon[102832]: pgmap v3425: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 13 op/s 2023-09-10T01:01:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:17 smithi082 ceph-mon[98694]: pgmap v3425: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 13 op/s 2023-09-10T01:01:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6108.582092 secs 2023-09-10T01:01:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6108.582092 secs 2023-09-10T01:01:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6108.582092 secs 2023-09-10T01:01:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6110.115029 secs 2023-09-10T01:01:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:19 smithi098 ceph-mon[103124]: pgmap v3426: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T01:01:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6110.115029 secs 2023-09-10T01:01:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:19 smithi170 ceph-mon[102832]: pgmap v3426: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T01:01:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6110.115029 secs 2023-09-10T01:01:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:19 smithi082 ceph-mon[98694]: pgmap v3426: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T01:01:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6110.827480 secs 2023-09-10T01:01:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6110.827480 secs 2023-09-10T01:01:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6110.827480 secs 2023-09-10T01:01:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:21 smithi170 ceph-mon[102832]: pgmap v3427: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T01:01:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:21 smithi082 ceph-mon[98694]: pgmap v3427: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T01:01:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:21 smithi098 ceph-mon[103124]: pgmap v3427: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T01:01:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6113.582150 secs 2023-09-10T01:01:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:23 smithi170 ceph-mon[102832]: pgmap v3428: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T01:01:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6113.582150 secs 2023-09-10T01:01:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:23 smithi082 ceph-mon[98694]: pgmap v3428: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T01:01:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6113.582150 secs 2023-09-10T01:01:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:23 smithi098 ceph-mon[103124]: pgmap v3428: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T01:01:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6115.115084 secs 2023-09-10T01:01:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6115.115084 secs 2023-09-10T01:01:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6115.115084 secs 2023-09-10T01:01:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6115.827541 secs 2023-09-10T01:01:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:25 smithi170 ceph-mon[102832]: pgmap v3429: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T01:01:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6115.827541 secs 2023-09-10T01:01:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:25 smithi082 ceph-mon[98694]: pgmap v3429: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T01:01:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6115.827541 secs 2023-09-10T01:01:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:25 smithi098 ceph-mon[103124]: pgmap v3429: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T01:01:27.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:27 smithi170 ceph-mon[102832]: pgmap v3430: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 129 KiB/s wr, 16 op/s 2023-09-10T01:01:27.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:27 smithi082 ceph-mon[98694]: pgmap v3430: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 129 KiB/s wr, 16 op/s 2023-09-10T01:01:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:27 smithi098 ceph-mon[103124]: pgmap v3430: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 129 KiB/s wr, 16 op/s 2023-09-10T01:01:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6120.115142 secs 2023-09-10T01:01:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:29 smithi170 ceph-mon[102832]: pgmap v3431: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T01:01:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6120.115142 secs 2023-09-10T01:01:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:29 smithi082 ceph-mon[98694]: pgmap v3431: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T01:01:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6120.115142 secs 2023-09-10T01:01:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:29 smithi098 ceph-mon[103124]: pgmap v3431: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T01:01:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6120.827603 secs 2023-09-10T01:01:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6120.827603 secs 2023-09-10T01:01:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6120.827603 secs 2023-09-10T01:01:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:31 smithi170 ceph-mon[102832]: pgmap v3432: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T01:01:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:01:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:31 smithi082 ceph-mon[98694]: pgmap v3432: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T01:01:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:01:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:31 smithi098 ceph-mon[103124]: pgmap v3432: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T01:01:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:01:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:01:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:01:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:01:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:01:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:01:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:01:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:01:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:01:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:01:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:33 smithi170 ceph-mon[102832]: pgmap v3433: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 187 KiB/s wr, 16 op/s 2023-09-10T01:01:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:33 smithi082 ceph-mon[98694]: pgmap v3433: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 187 KiB/s wr, 16 op/s 2023-09-10T01:01:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:33 smithi098 ceph-mon[103124]: pgmap v3433: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 187 KiB/s wr, 16 op/s 2023-09-10T01:01:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6125.115199 secs 2023-09-10T01:01:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6125.115199 secs 2023-09-10T01:01:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6125.115199 secs 2023-09-10T01:01:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6125.827665 secs 2023-09-10T01:01:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:35 smithi170 ceph-mon[102832]: pgmap v3434: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 11 op/s 2023-09-10T01:01:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6125.827665 secs 2023-09-10T01:01:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:35 smithi082 ceph-mon[98694]: pgmap v3434: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 11 op/s 2023-09-10T01:01:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6125.827665 secs 2023-09-10T01:01:35.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:35 smithi098 ceph-mon[103124]: pgmap v3434: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 11 op/s 2023-09-10T01:01:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:37 smithi170 ceph-mon[102832]: pgmap v3435: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 244 KiB/s wr, 18 op/s 2023-09-10T01:01:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:37 smithi082 ceph-mon[98694]: pgmap v3435: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 244 KiB/s wr, 18 op/s 2023-09-10T01:01:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:37 smithi098 ceph-mon[103124]: pgmap v3435: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 244 KiB/s wr, 18 op/s 2023-09-10T01:01:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6130.115257 secs 2023-09-10T01:01:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:39 smithi170 ceph-mon[102832]: pgmap v3436: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 12 op/s 2023-09-10T01:01:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6130.115257 secs 2023-09-10T01:01:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:39 smithi082 ceph-mon[98694]: pgmap v3436: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 12 op/s 2023-09-10T01:01:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6130.115257 secs 2023-09-10T01:01:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:39 smithi098 ceph-mon[103124]: pgmap v3436: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 12 op/s 2023-09-10T01:01:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6130.827728 secs 2023-09-10T01:01:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6130.827728 secs 2023-09-10T01:01:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6130.827728 secs 2023-09-10T01:01:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:41 smithi170 ceph-mon[102832]: pgmap v3437: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 12 op/s 2023-09-10T01:01:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:41 smithi082 ceph-mon[98694]: pgmap v3437: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 12 op/s 2023-09-10T01:01:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:41 smithi098 ceph-mon[103124]: pgmap v3437: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 12 op/s 2023-09-10T01:01:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:01:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:01:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:01:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:43 smithi170 ceph-mon[102832]: pgmap v3438: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 221 KiB/s wr, 18 op/s 2023-09-10T01:01:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:01:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:01:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:01:43.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:43 smithi082 ceph-mon[98694]: pgmap v3438: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 221 KiB/s wr, 18 op/s 2023-09-10T01:01:43.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:01:43.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:01:43.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:01:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:43 smithi098 ceph-mon[103124]: pgmap v3438: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 221 KiB/s wr, 18 op/s 2023-09-10T01:01:43.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:01:43.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:01:43.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:01:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6135.115316 secs 2023-09-10T01:01:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6135.115316 secs 2023-09-10T01:01:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6135.115316 secs 2023-09-10T01:01:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6135.827788 secs 2023-09-10T01:01:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:45 smithi170 ceph-mon[102832]: pgmap v3439: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 12 op/s 2023-09-10T01:01:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6135.827788 secs 2023-09-10T01:01:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:45 smithi082 ceph-mon[98694]: pgmap v3439: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 12 op/s 2023-09-10T01:01:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6135.827788 secs 2023-09-10T01:01:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:45 smithi098 ceph-mon[103124]: pgmap v3439: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 12 op/s 2023-09-10T01:01:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:47 smithi170 ceph-mon[102832]: pgmap v3440: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 250 KiB/s wr, 18 op/s 2023-09-10T01:01:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:47 smithi082 ceph-mon[98694]: pgmap v3440: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 250 KiB/s wr, 18 op/s 2023-09-10T01:01:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:47 smithi098 ceph-mon[103124]: pgmap v3440: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 250 KiB/s wr, 18 op/s 2023-09-10T01:01:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6138.582497 secs 2023-09-10T01:01:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6138.582497 secs 2023-09-10T01:01:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6138.582497 secs 2023-09-10T01:01:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6140.115377 secs 2023-09-10T01:01:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:49 smithi170 ceph-mon[102832]: pgmap v3441: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-09-10T01:01:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6140.115377 secs 2023-09-10T01:01:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:49 smithi082 ceph-mon[98694]: pgmap v3441: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-09-10T01:01:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6140.115377 secs 2023-09-10T01:01:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:49 smithi098 ceph-mon[103124]: pgmap v3441: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-09-10T01:01:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6140.827850 secs 2023-09-10T01:01:50.713 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6140.827850 secs 2023-09-10T01:01:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6140.827850 secs 2023-09-10T01:01:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:51 smithi170 ceph-mon[102832]: pgmap v3442: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-09-10T01:01:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:51 smithi082 ceph-mon[98694]: pgmap v3442: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-09-10T01:01:51.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:51 smithi098 ceph-mon[103124]: pgmap v3442: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-09-10T01:01:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:01:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:01:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:01:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:01:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:01:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:01:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:01:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:01:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:01:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6143.582558 secs 2023-09-10T01:01:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:53 smithi170 ceph-mon[102832]: pgmap v3443: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 190 KiB/s wr, 16 op/s 2023-09-10T01:01:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6143.582558 secs 2023-09-10T01:01:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:53 smithi082 ceph-mon[98694]: pgmap v3443: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 190 KiB/s wr, 16 op/s 2023-09-10T01:01:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6143.582558 secs 2023-09-10T01:01:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:53 smithi098 ceph-mon[103124]: pgmap v3443: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 190 KiB/s wr, 16 op/s 2023-09-10T01:01:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6145.115435 secs 2023-09-10T01:01:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6145.115435 secs 2023-09-10T01:01:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6145.115435 secs 2023-09-10T01:01:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6145.827911 secs 2023-09-10T01:01:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:55 smithi170 ceph-mon[102832]: pgmap v3444: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T01:01:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6145.827911 secs 2023-09-10T01:01:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:55 smithi082 ceph-mon[98694]: pgmap v3444: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T01:01:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6145.827911 secs 2023-09-10T01:01:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:55 smithi098 ceph-mon[103124]: pgmap v3444: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T01:01:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:57 smithi170 ceph-mon[102832]: pgmap v3445: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 16 op/s 2023-09-10T01:01:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:57 smithi082 ceph-mon[98694]: pgmap v3445: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 16 op/s 2023-09-10T01:01:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:57 smithi098 ceph-mon[103124]: pgmap v3445: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 16 op/s 2023-09-10T01:01:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6148.582642 secs 2023-09-10T01:01:58.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6148.582642 secs 2023-09-10T01:01:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6148.582642 secs 2023-09-10T01:01:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6150.115492 secs 2023-09-10T01:01:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:01:59 smithi170 ceph-mon[102832]: pgmap v3446: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T01:01:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6150.115492 secs 2023-09-10T01:01:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:01:59 smithi082 ceph-mon[98694]: pgmap v3446: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T01:01:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6150.115492 secs 2023-09-10T01:01:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:01:59 smithi098 ceph-mon[103124]: pgmap v3446: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T01:02:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6150.827972 secs 2023-09-10T01:02:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6150.827972 secs 2023-09-10T01:02:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6150.827972 secs 2023-09-10T01:02:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:01 smithi170 ceph-mon[102832]: pgmap v3447: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T01:02:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:01 smithi082 ceph-mon[98694]: pgmap v3447: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T01:02:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:01 smithi098 ceph-mon[103124]: pgmap v3447: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T01:02:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6153.582726 secs 2023-09-10T01:02:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:03 smithi170 ceph-mon[102832]: pgmap v3448: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 143 KiB/s wr, 29 op/s 2023-09-10T01:02:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6153.582726 secs 2023-09-10T01:02:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:03 smithi082 ceph-mon[98694]: pgmap v3448: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 143 KiB/s wr, 29 op/s 2023-09-10T01:02:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6153.582726 secs 2023-09-10T01:02:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:03 smithi098 ceph-mon[103124]: pgmap v3448: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 143 KiB/s wr, 29 op/s 2023-09-10T01:02:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6155.115578 secs 2023-09-10T01:02:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6155.115578 secs 2023-09-10T01:02:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6155.115578 secs 2023-09-10T01:02:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6155.828034 secs 2023-09-10T01:02:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:05 smithi170 ceph-mon[102832]: pgmap v3449: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 116 KiB/s wr, 24 op/s 2023-09-10T01:02:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6155.828034 secs 2023-09-10T01:02:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:05 smithi082 ceph-mon[98694]: pgmap v3449: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 116 KiB/s wr, 24 op/s 2023-09-10T01:02:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6155.828034 secs 2023-09-10T01:02:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:05 smithi098 ceph-mon[103124]: pgmap v3449: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 116 KiB/s wr, 24 op/s 2023-09-10T01:02:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:07 smithi170 ceph-mon[102832]: pgmap v3450: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T01:02:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:07 smithi082 ceph-mon[98694]: pgmap v3450: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T01:02:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:07 smithi098 ceph-mon[103124]: pgmap v3450: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T01:02:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6158.582839 secs 2023-09-10T01:02:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:08 smithi170 ceph-mon[102832]: osdmap e287: 12 total, 12 up, 12 in 2023-09-10T01:02:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6158.582839 secs 2023-09-10T01:02:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:08 smithi082 ceph-mon[98694]: osdmap e287: 12 total, 12 up, 12 in 2023-09-10T01:02:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6158.582839 secs 2023-09-10T01:02:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:08 smithi098 ceph-mon[103124]: osdmap e287: 12 total, 12 up, 12 in 2023-09-10T01:02:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6160.115647 secs 2023-09-10T01:02:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:09 smithi170 ceph-mon[102832]: pgmap v3452: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T01:02:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6160.115647 secs 2023-09-10T01:02:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:09 smithi082 ceph-mon[98694]: pgmap v3452: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T01:02:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6160.115647 secs 2023-09-10T01:02:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:09 smithi098 ceph-mon[103124]: pgmap v3452: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T01:02:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6160.828098 secs 2023-09-10T01:02:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6160.828098 secs 2023-09-10T01:02:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6160.828098 secs 2023-09-10T01:02:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:11 smithi170 ceph-mon[102832]: pgmap v3453: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T01:02:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:11 smithi082 ceph-mon[98694]: pgmap v3453: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T01:02:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:11 smithi098 ceph-mon[103124]: pgmap v3453: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T01:02:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6163.582922 secs 2023-09-10T01:02:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:13 smithi170 ceph-mon[102832]: pgmap v3454: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 79 KiB/s wr, 13 op/s 2023-09-10T01:02:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6163.582922 secs 2023-09-10T01:02:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:13 smithi082 ceph-mon[98694]: pgmap v3454: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 79 KiB/s wr, 13 op/s 2023-09-10T01:02:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6163.582922 secs 2023-09-10T01:02:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:13 smithi098 ceph-mon[103124]: pgmap v3454: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 79 KiB/s wr, 13 op/s 2023-09-10T01:02:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6165.115707 secs 2023-09-10T01:02:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6165.115707 secs 2023-09-10T01:02:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6165.115707 secs 2023-09-10T01:02:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6165.828159 secs 2023-09-10T01:02:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:15 smithi170 ceph-mon[102832]: pgmap v3455: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 79 KiB/s wr, 13 op/s 2023-09-10T01:02:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:15 smithi170 ceph-mon[102832]: osdmap e288: 12 total, 12 up, 12 in 2023-09-10T01:02:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6165.828159 secs 2023-09-10T01:02:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:15 smithi082 ceph-mon[98694]: pgmap v3455: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 79 KiB/s wr, 13 op/s 2023-09-10T01:02:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:15 smithi082 ceph-mon[98694]: osdmap e288: 12 total, 12 up, 12 in 2023-09-10T01:02:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6165.828159 secs 2023-09-10T01:02:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:15 smithi098 ceph-mon[103124]: pgmap v3455: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 79 KiB/s wr, 13 op/s 2023-09-10T01:02:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:15 smithi098 ceph-mon[103124]: osdmap e288: 12 total, 12 up, 12 in 2023-09-10T01:02:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:17 smithi170 ceph-mon[102832]: pgmap v3457: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 13 op/s 2023-09-10T01:02:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:17 smithi082 ceph-mon[98694]: pgmap v3457: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 13 op/s 2023-09-10T01:02:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:17 smithi098 ceph-mon[103124]: pgmap v3457: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 13 op/s 2023-09-10T01:02:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6168.582999 secs 2023-09-10T01:02:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6168.582999 secs 2023-09-10T01:02:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6168.582999 secs 2023-09-10T01:02:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6170.115768 secs 2023-09-10T01:02:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:19 smithi170 ceph-mon[102832]: pgmap v3458: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T01:02:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6170.115768 secs 2023-09-10T01:02:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:19 smithi082 ceph-mon[98694]: pgmap v3458: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T01:02:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6170.115768 secs 2023-09-10T01:02:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:19 smithi098 ceph-mon[103124]: pgmap v3458: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T01:02:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6170.828220 secs 2023-09-10T01:02:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6170.828220 secs 2023-09-10T01:02:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6170.828220 secs 2023-09-10T01:02:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:21 smithi170 ceph-mon[102832]: pgmap v3459: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T01:02:21.711 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:21 smithi082 ceph-mon[98694]: pgmap v3459: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T01:02:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:21 smithi098 ceph-mon[103124]: pgmap v3459: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T01:02:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6173.583061 secs 2023-09-10T01:02:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:23 smithi170 ceph-mon[102832]: pgmap v3460: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 12 op/s 2023-09-10T01:02:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6173.583061 secs 2023-09-10T01:02:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:23 smithi082 ceph-mon[98694]: pgmap v3460: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 12 op/s 2023-09-10T01:02:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6173.583061 secs 2023-09-10T01:02:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:23 smithi098 ceph-mon[103124]: pgmap v3460: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 12 op/s 2023-09-10T01:02:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6175.115859 secs 2023-09-10T01:02:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6175.115859 secs 2023-09-10T01:02:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6175.115859 secs 2023-09-10T01:02:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6175.828281 secs 2023-09-10T01:02:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:25 smithi082 ceph-mon[98694]: pgmap v3461: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 12 op/s 2023-09-10T01:02:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6175.828281 secs 2023-09-10T01:02:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:25 smithi098 ceph-mon[103124]: pgmap v3461: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 12 op/s 2023-09-10T01:02:25.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6175.828281 secs 2023-09-10T01:02:25.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:25 smithi170 ceph-mon[102832]: pgmap v3461: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 12 op/s 2023-09-10T01:02:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:27 smithi082 ceph-mon[98694]: pgmap v3462: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 131 KiB/s wr, 17 op/s 2023-09-10T01:02:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:27 smithi098 ceph-mon[103124]: pgmap v3462: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 131 KiB/s wr, 17 op/s 2023-09-10T01:02:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:27 smithi170 ceph-mon[102832]: pgmap v3462: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 131 KiB/s wr, 17 op/s 2023-09-10T01:02:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6178.583121 secs 2023-09-10T01:02:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6178.583121 secs 2023-09-10T01:02:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6178.583121 secs 2023-09-10T01:02:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6180.115918 secs 2023-09-10T01:02:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:29 smithi082 ceph-mon[98694]: pgmap v3463: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T01:02:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6180.115918 secs 2023-09-10T01:02:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:29 smithi098 ceph-mon[103124]: pgmap v3463: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T01:02:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6180.115918 secs 2023-09-10T01:02:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:29 smithi170 ceph-mon[102832]: pgmap v3463: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T01:02:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6180.828342 secs 2023-09-10T01:02:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6180.828342 secs 2023-09-10T01:02:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6180.828342 secs 2023-09-10T01:02:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:31 smithi082 ceph-mon[98694]: pgmap v3464: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T01:02:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:31 smithi098 ceph-mon[103124]: pgmap v3464: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T01:02:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:31 smithi170 ceph-mon[102832]: pgmap v3464: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T01:02:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:33 smithi082 ceph-mon[98694]: pgmap v3465: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 194 KiB/s wr, 18 op/s 2023-09-10T01:02:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:33 smithi098 ceph-mon[103124]: pgmap v3465: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 194 KiB/s wr, 18 op/s 2023-09-10T01:02:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:33 smithi170 ceph-mon[102832]: pgmap v3465: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 194 KiB/s wr, 18 op/s 2023-09-10T01:02:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6185.116012 secs 2023-09-10T01:02:34.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6185.116012 secs 2023-09-10T01:02:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6185.116012 secs 2023-09-10T01:02:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6185.828401 secs 2023-09-10T01:02:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:35 smithi098 ceph-mon[103124]: pgmap v3466: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T01:02:35.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:02:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6185.828401 secs 2023-09-10T01:02:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:35 smithi170 ceph-mon[102832]: pgmap v3466: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T01:02:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:02:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6185.828401 secs 2023-09-10T01:02:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:35 smithi082 ceph-mon[98694]: pgmap v3466: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T01:02:35.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:02:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:02:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:02:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:02:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:02:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:02:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:02:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:02:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:02:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:02:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:37 smithi098 ceph-mon[103124]: pgmap v3467: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 247 KiB/s wr, 18 op/s 2023-09-10T01:02:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:37 smithi170 ceph-mon[102832]: pgmap v3467: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 247 KiB/s wr, 18 op/s 2023-09-10T01:02:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:37 smithi082 ceph-mon[98694]: pgmap v3467: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 247 KiB/s wr, 18 op/s 2023-09-10T01:02:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6190.116070 secs 2023-09-10T01:02:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:39 smithi098 ceph-mon[103124]: pgmap v3468: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T01:02:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6190.116070 secs 2023-09-10T01:02:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:39 smithi170 ceph-mon[102832]: pgmap v3468: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T01:02:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6190.116070 secs 2023-09-10T01:02:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:39 smithi082 ceph-mon[98694]: pgmap v3468: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T01:02:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6190.828465 secs 2023-09-10T01:02:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6190.828465 secs 2023-09-10T01:02:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6190.828465 secs 2023-09-10T01:02:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:41 smithi098 ceph-mon[103124]: pgmap v3469: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T01:02:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:41 smithi170 ceph-mon[102832]: pgmap v3469: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T01:02:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:41 smithi082 ceph-mon[98694]: pgmap v3469: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T01:02:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:43 smithi098 ceph-mon[103124]: pgmap v3470: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 223 KiB/s wr, 17 op/s 2023-09-10T01:02:43.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:02:43.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:02:43.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:02:43.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:02:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:43 smithi170 ceph-mon[102832]: pgmap v3470: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 223 KiB/s wr, 17 op/s 2023-09-10T01:02:43.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:02:43.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:02:43.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:02:43.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:02:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:43 smithi082 ceph-mon[98694]: pgmap v3470: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 223 KiB/s wr, 17 op/s 2023-09-10T01:02:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:02:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:02:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:02:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:02:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6195.116134 secs 2023-09-10T01:02:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6195.116134 secs 2023-09-10T01:02:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6195.116134 secs 2023-09-10T01:02:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6195.828486 secs 2023-09-10T01:02:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:45 smithi098 ceph-mon[103124]: pgmap v3471: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T01:02:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6195.828486 secs 2023-09-10T01:02:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:45 smithi170 ceph-mon[102832]: pgmap v3471: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T01:02:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6195.828486 secs 2023-09-10T01:02:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:45 smithi082 ceph-mon[98694]: pgmap v3471: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T01:02:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:47 smithi098 ceph-mon[103124]: pgmap v3472: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 237 KiB/s wr, 17 op/s 2023-09-10T01:02:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:47 smithi170 ceph-mon[102832]: pgmap v3472: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 237 KiB/s wr, 17 op/s 2023-09-10T01:02:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:47 smithi082 ceph-mon[98694]: pgmap v3472: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 237 KiB/s wr, 17 op/s 2023-09-10T01:02:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6198.583423 secs 2023-09-10T01:02:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6198.583423 secs 2023-09-10T01:02:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6198.583423 secs 2023-09-10T01:02:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6200.116188 secs 2023-09-10T01:02:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:49 smithi098 ceph-mon[103124]: pgmap v3473: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T01:02:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6200.116188 secs 2023-09-10T01:02:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:49 smithi170 ceph-mon[102832]: pgmap v3473: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T01:02:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6200.116188 secs 2023-09-10T01:02:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:49 smithi082 ceph-mon[98694]: pgmap v3473: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T01:02:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6200.828550 secs 2023-09-10T01:02:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6200.828550 secs 2023-09-10T01:02:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6200.828550 secs 2023-09-10T01:02:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:51 smithi098 ceph-mon[103124]: pgmap v3474: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T01:02:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:51 smithi170 ceph-mon[102832]: pgmap v3474: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T01:02:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:51 smithi082 ceph-mon[98694]: pgmap v3474: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T01:02:52.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:02:52.932 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:02:52.932 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:02:52.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:02:52.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:02:52.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:02:53.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:02:53.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:02:53.008 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:02:53.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6203.583483 secs 2023-09-10T01:02:53.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:53 smithi170 ceph-mon[102832]: pgmap v3475: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 186 KiB/s wr, 16 op/s 2023-09-10T01:02:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6203.583483 secs 2023-09-10T01:02:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:53 smithi082 ceph-mon[98694]: pgmap v3475: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 186 KiB/s wr, 16 op/s 2023-09-10T01:02:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6203.583483 secs 2023-09-10T01:02:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:53 smithi098 ceph-mon[103124]: pgmap v3475: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 186 KiB/s wr, 16 op/s 2023-09-10T01:02:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6205.116274 secs 2023-09-10T01:02:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6205.116274 secs 2023-09-10T01:02:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6205.116274 secs 2023-09-10T01:02:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6205.828613 secs 2023-09-10T01:02:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:55 smithi170 ceph-mon[102832]: pgmap v3476: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T01:02:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6205.828613 secs 2023-09-10T01:02:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:55 smithi082 ceph-mon[98694]: pgmap v3476: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T01:02:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6205.828613 secs 2023-09-10T01:02:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:55 smithi098 ceph-mon[103124]: pgmap v3476: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T01:02:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:57 smithi170 ceph-mon[102832]: pgmap v3477: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 176 KiB/s wr, 16 op/s 2023-09-10T01:02:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:57 smithi082 ceph-mon[98694]: pgmap v3477: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 176 KiB/s wr, 16 op/s 2023-09-10T01:02:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:57 smithi098 ceph-mon[103124]: pgmap v3477: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 176 KiB/s wr, 16 op/s 2023-09-10T01:02:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6208.583542 secs 2023-09-10T01:02:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6208.583542 secs 2023-09-10T01:02:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6208.583542 secs 2023-09-10T01:02:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6210.116399 secs 2023-09-10T01:02:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:02:59 smithi170 ceph-mon[102832]: pgmap v3478: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T01:02:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6210.116399 secs 2023-09-10T01:02:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:02:59 smithi082 ceph-mon[98694]: pgmap v3478: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T01:03:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6210.116399 secs 2023-09-10T01:03:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:02:59 smithi098 ceph-mon[103124]: pgmap v3478: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T01:03:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6210.828675 secs 2023-09-10T01:03:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6210.828675 secs 2023-09-10T01:03:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6210.828675 secs 2023-09-10T01:03:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:01 smithi170 ceph-mon[102832]: pgmap v3479: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T01:03:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:01 smithi082 ceph-mon[98694]: pgmap v3479: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T01:03:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:01 smithi098 ceph-mon[103124]: pgmap v3479: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T01:03:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:02 smithi170 ceph-mon[102832]: pgmap v3480: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 137 KiB/s wr, 28 op/s 2023-09-10T01:03:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:02 smithi082 ceph-mon[98694]: pgmap v3480: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 137 KiB/s wr, 28 op/s 2023-09-10T01:03:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:02 smithi098 ceph-mon[103124]: pgmap v3480: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 137 KiB/s wr, 28 op/s 2023-09-10T01:03:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6213.583603 secs 2023-09-10T01:03:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6213.583603 secs 2023-09-10T01:03:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6213.583603 secs 2023-09-10T01:03:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6215.116456 secs 2023-09-10T01:03:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:04 smithi170 ceph-mon[102832]: pgmap v3481: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 105 KiB/s wr, 22 op/s 2023-09-10T01:03:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6215.116456 secs 2023-09-10T01:03:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:04 smithi082 ceph-mon[98694]: pgmap v3481: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 105 KiB/s wr, 22 op/s 2023-09-10T01:03:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6215.116456 secs 2023-09-10T01:03:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:04 smithi098 ceph-mon[103124]: pgmap v3481: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 105 KiB/s wr, 22 op/s 2023-09-10T01:03:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6215.828735 secs 2023-09-10T01:03:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6215.828735 secs 2023-09-10T01:03:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6215.828735 secs 2023-09-10T01:03:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:06 smithi170 ceph-mon[102832]: pgmap v3482: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:03:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:06 smithi082 ceph-mon[98694]: pgmap v3482: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:03:07.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:06 smithi098 ceph-mon[103124]: pgmap v3482: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:03:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6218.583662 secs 2023-09-10T01:03:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:08 smithi170 ceph-mon[102832]: osdmap e289: 12 total, 12 up, 12 in 2023-09-10T01:03:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6220.116495 secs 2023-09-10T01:03:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:08 smithi170 ceph-mon[102832]: pgmap v3484: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T01:03:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6218.583662 secs 2023-09-10T01:03:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:08 smithi082 ceph-mon[98694]: osdmap e289: 12 total, 12 up, 12 in 2023-09-10T01:03:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6220.116495 secs 2023-09-10T01:03:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:08 smithi082 ceph-mon[98694]: pgmap v3484: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T01:03:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6218.583662 secs 2023-09-10T01:03:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:08 smithi098 ceph-mon[103124]: osdmap e289: 12 total, 12 up, 12 in 2023-09-10T01:03:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6220.116495 secs 2023-09-10T01:03:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:08 smithi098 ceph-mon[103124]: pgmap v3484: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T01:03:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6220.828797 secs 2023-09-10T01:03:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6220.828797 secs 2023-09-10T01:03:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6220.828797 secs 2023-09-10T01:03:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:11 smithi170 ceph-mon[102832]: pgmap v3485: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T01:03:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:11 smithi082 ceph-mon[98694]: pgmap v3485: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T01:03:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:11 smithi098 ceph-mon[103124]: pgmap v3485: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T01:03:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6223.583729 secs 2023-09-10T01:03:13.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:13 smithi170 ceph-mon[102832]: pgmap v3486: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 94 KiB/s wr, 14 op/s 2023-09-10T01:03:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6223.583729 secs 2023-09-10T01:03:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:13 smithi082 ceph-mon[98694]: pgmap v3486: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 94 KiB/s wr, 14 op/s 2023-09-10T01:03:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6223.583729 secs 2023-09-10T01:03:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:13 smithi098 ceph-mon[103124]: pgmap v3486: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 94 KiB/s wr, 14 op/s 2023-09-10T01:03:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6225.116553 secs 2023-09-10T01:03:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6225.116553 secs 2023-09-10T01:03:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6225.116553 secs 2023-09-10T01:03:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6225.828861 secs 2023-09-10T01:03:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:15 smithi170 ceph-mon[102832]: pgmap v3487: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 94 KiB/s wr, 14 op/s 2023-09-10T01:03:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:15 smithi170 ceph-mon[102832]: osdmap e290: 12 total, 12 up, 12 in 2023-09-10T01:03:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6225.828861 secs 2023-09-10T01:03:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:15 smithi082 ceph-mon[98694]: pgmap v3487: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 94 KiB/s wr, 14 op/s 2023-09-10T01:03:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:15 smithi082 ceph-mon[98694]: osdmap e290: 12 total, 12 up, 12 in 2023-09-10T01:03:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6225.828861 secs 2023-09-10T01:03:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:15 smithi098 ceph-mon[103124]: pgmap v3487: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 94 KiB/s wr, 14 op/s 2023-09-10T01:03:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:15 smithi098 ceph-mon[103124]: osdmap e290: 12 total, 12 up, 12 in 2023-09-10T01:03:17.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:17 smithi170 ceph-mon[102832]: pgmap v3489: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 14 op/s 2023-09-10T01:03:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:17 smithi082 ceph-mon[98694]: pgmap v3489: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 14 op/s 2023-09-10T01:03:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:17 smithi098 ceph-mon[103124]: pgmap v3489: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 14 op/s 2023-09-10T01:03:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6228.583781 secs 2023-09-10T01:03:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6228.583781 secs 2023-09-10T01:03:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6228.583781 secs 2023-09-10T01:03:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6230.116607 secs 2023-09-10T01:03:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:19 smithi170 ceph-mon[102832]: pgmap v3490: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T01:03:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6230.116607 secs 2023-09-10T01:03:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:19 smithi082 ceph-mon[98694]: pgmap v3490: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T01:03:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6230.116607 secs 2023-09-10T01:03:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:19 smithi098 ceph-mon[103124]: pgmap v3490: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T01:03:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6230.828922 secs 2023-09-10T01:03:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6230.828922 secs 2023-09-10T01:03:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6230.828922 secs 2023-09-10T01:03:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:21 smithi170 ceph-mon[102832]: pgmap v3491: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T01:03:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:21 smithi082 ceph-mon[98694]: pgmap v3491: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T01:03:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:21 smithi098 ceph-mon[103124]: pgmap v3491: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T01:03:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:23 smithi170 ceph-mon[102832]: pgmap v3492: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:03:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:03:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:23 smithi082 ceph-mon[98694]: pgmap v3492: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:03:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:03:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:23 smithi098 ceph-mon[103124]: pgmap v3492: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:03:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:03:24.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6235.116687 secs 2023-09-10T01:03:24.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:03:24.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:03:24.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:03:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6235.116687 secs 2023-09-10T01:03:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:03:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:03:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:03:24.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6235.116687 secs 2023-09-10T01:03:24.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:03:24.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:03:24.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:03:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6235.828983 secs 2023-09-10T01:03:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:25 smithi082 ceph-mon[98694]: pgmap v3493: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:03:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6235.828983 secs 2023-09-10T01:03:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:25 smithi098 ceph-mon[103124]: pgmap v3493: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:03:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6235.828983 secs 2023-09-10T01:03:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:25 smithi170 ceph-mon[102832]: pgmap v3493: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:03:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:27 smithi082 ceph-mon[98694]: pgmap v3494: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 140 KiB/s wr, 17 op/s 2023-09-10T01:03:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:27 smithi098 ceph-mon[103124]: pgmap v3494: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 140 KiB/s wr, 17 op/s 2023-09-10T01:03:27.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:27 smithi170 ceph-mon[102832]: pgmap v3494: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 140 KiB/s wr, 17 op/s 2023-09-10T01:03:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6240.116745 secs 2023-09-10T01:03:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:29 smithi082 ceph-mon[98694]: pgmap v3495: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 128 KiB/s wr, 10 op/s 2023-09-10T01:03:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6240.116745 secs 2023-09-10T01:03:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:29 smithi098 ceph-mon[103124]: pgmap v3495: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 128 KiB/s wr, 10 op/s 2023-09-10T01:03:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6240.116745 secs 2023-09-10T01:03:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:29 smithi170 ceph-mon[102832]: pgmap v3495: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 128 KiB/s wr, 10 op/s 2023-09-10T01:03:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6240.829048 secs 2023-09-10T01:03:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6240.829048 secs 2023-09-10T01:03:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6240.829048 secs 2023-09-10T01:03:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:31 smithi082 ceph-mon[98694]: pgmap v3496: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 128 KiB/s wr, 10 op/s 2023-09-10T01:03:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:31 smithi098 ceph-mon[103124]: pgmap v3496: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 128 KiB/s wr, 10 op/s 2023-09-10T01:03:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:31 smithi170 ceph-mon[102832]: pgmap v3496: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 128 KiB/s wr, 10 op/s 2023-09-10T01:03:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6243.584016 secs 2023-09-10T01:03:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:33 smithi082 ceph-mon[98694]: pgmap v3497: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 191 KiB/s wr, 17 op/s 2023-09-10T01:03:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6243.584016 secs 2023-09-10T01:03:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:33 smithi098 ceph-mon[103124]: pgmap v3497: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 191 KiB/s wr, 17 op/s 2023-09-10T01:03:33.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6243.584016 secs 2023-09-10T01:03:33.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:33 smithi170 ceph-mon[102832]: pgmap v3497: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 191 KiB/s wr, 17 op/s 2023-09-10T01:03:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6245.116825 secs 2023-09-10T01:03:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6245.116825 secs 2023-09-10T01:03:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6245.116825 secs 2023-09-10T01:03:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6245.829108 secs 2023-09-10T01:03:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:35 smithi098 ceph-mon[103124]: pgmap v3498: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 160 KiB/s wr, 13 op/s 2023-09-10T01:03:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6245.829108 secs 2023-09-10T01:03:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:35 smithi170 ceph-mon[102832]: pgmap v3498: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 160 KiB/s wr, 13 op/s 2023-09-10T01:03:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6245.829108 secs 2023-09-10T01:03:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:35 smithi082 ceph-mon[98694]: pgmap v3498: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 160 KiB/s wr, 13 op/s 2023-09-10T01:03:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:36 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:03:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:03:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:03:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:36 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:03:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:03:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:03:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:36 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:03:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:03:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:03:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:37 smithi098 ceph-mon[103124]: pgmap v3499: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 249 KiB/s wr, 19 op/s 2023-09-10T01:03:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:37 smithi170 ceph-mon[102832]: pgmap v3499: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 249 KiB/s wr, 19 op/s 2023-09-10T01:03:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:37 smithi082 ceph-mon[98694]: pgmap v3499: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 249 KiB/s wr, 19 op/s 2023-09-10T01:03:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6248.584075 secs 2023-09-10T01:03:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6248.584075 secs 2023-09-10T01:03:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6248.584075 secs 2023-09-10T01:03:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6250.116902 secs 2023-09-10T01:03:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:39 smithi098 ceph-mon[103124]: pgmap v3500: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 152 KiB/s wr, 13 op/s 2023-09-10T01:03:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6250.116902 secs 2023-09-10T01:03:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:39 smithi170 ceph-mon[102832]: pgmap v3500: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 152 KiB/s wr, 13 op/s 2023-09-10T01:03:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6250.116902 secs 2023-09-10T01:03:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:39 smithi082 ceph-mon[98694]: pgmap v3500: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 152 KiB/s wr, 13 op/s 2023-09-10T01:03:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6250.829170 secs 2023-09-10T01:03:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6250.829170 secs 2023-09-10T01:03:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6250.829170 secs 2023-09-10T01:03:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:41 smithi098 ceph-mon[103124]: pgmap v3501: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 152 KiB/s wr, 13 op/s 2023-09-10T01:03:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:41 smithi170 ceph-mon[102832]: pgmap v3501: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 152 KiB/s wr, 13 op/s 2023-09-10T01:03:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:41 smithi082 ceph-mon[98694]: pgmap v3501: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 152 KiB/s wr, 13 op/s 2023-09-10T01:03:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:43 smithi098 ceph-mon[103124]: pgmap v3502: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 217 KiB/s wr, 18 op/s 2023-09-10T01:03:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:03:43.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:03:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:43 smithi170 ceph-mon[102832]: pgmap v3502: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 217 KiB/s wr, 18 op/s 2023-09-10T01:03:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:03:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:03:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:43 smithi082 ceph-mon[98694]: pgmap v3502: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 217 KiB/s wr, 18 op/s 2023-09-10T01:03:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:03:43.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:03:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6255.116986 secs 2023-09-10T01:03:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:44 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:03:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:03:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:03:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:03:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:03:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:03:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6255.116986 secs 2023-09-10T01:03:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:44 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:03:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:03:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:03:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:03:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:03:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:03:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6255.116986 secs 2023-09-10T01:03:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:44 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:03:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:03:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:03:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:03:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:03:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:03:45.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6255.829229 secs 2023-09-10T01:03:45.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:45 smithi098 ceph-mon[103124]: pgmap v3503: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T01:03:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6255.829229 secs 2023-09-10T01:03:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:45 smithi170 ceph-mon[102832]: pgmap v3503: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T01:03:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6255.829229 secs 2023-09-10T01:03:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:45 smithi082 ceph-mon[98694]: pgmap v3503: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T01:03:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:47 smithi098 ceph-mon[103124]: pgmap v3504: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 298 KiB/s wr, 17 op/s 2023-09-10T01:03:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:47 smithi170 ceph-mon[102832]: pgmap v3504: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 298 KiB/s wr, 17 op/s 2023-09-10T01:03:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:47 smithi082 ceph-mon[98694]: pgmap v3504: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 298 KiB/s wr, 17 op/s 2023-09-10T01:03:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6258.584193 secs 2023-09-10T01:03:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6258.584193 secs 2023-09-10T01:03:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6258.584193 secs 2023-09-10T01:03:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6260.117053 secs 2023-09-10T01:03:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:49 smithi170 ceph-mon[102832]: pgmap v3505: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T01:03:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6260.117053 secs 2023-09-10T01:03:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:49 smithi082 ceph-mon[98694]: pgmap v3505: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T01:03:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6260.117053 secs 2023-09-10T01:03:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:49 smithi098 ceph-mon[103124]: pgmap v3505: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T01:03:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6260.829288 secs 2023-09-10T01:03:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6260.829288 secs 2023-09-10T01:03:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6260.829288 secs 2023-09-10T01:03:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:51 smithi170 ceph-mon[102832]: pgmap v3506: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T01:03:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:51 smithi082 ceph-mon[98694]: pgmap v3506: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T01:03:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:51 smithi098 ceph-mon[103124]: pgmap v3506: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T01:03:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:03:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:03:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:03:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:03:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:03:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:03:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:03:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:03:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:03:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6263.584281 secs 2023-09-10T01:03:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:53 smithi170 ceph-mon[102832]: pgmap v3507: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 242 KiB/s wr, 16 op/s 2023-09-10T01:03:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6263.584281 secs 2023-09-10T01:03:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:53 smithi082 ceph-mon[98694]: pgmap v3507: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 242 KiB/s wr, 16 op/s 2023-09-10T01:03:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6263.584281 secs 2023-09-10T01:03:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:53 smithi098 ceph-mon[103124]: pgmap v3507: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 242 KiB/s wr, 16 op/s 2023-09-10T01:03:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6265.117136 secs 2023-09-10T01:03:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6265.117136 secs 2023-09-10T01:03:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6265.117136 secs 2023-09-10T01:03:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6265.829348 secs 2023-09-10T01:03:56.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:55 smithi170 ceph-mon[102832]: pgmap v3508: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T01:03:56.645 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6265.829348 secs 2023-09-10T01:03:56.645 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:55 smithi098 ceph-mon[103124]: pgmap v3508: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T01:03:56.646 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6265.829348 secs 2023-09-10T01:03:56.646 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:55 smithi082 ceph-mon[98694]: pgmap v3508: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T01:03:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:57 smithi170 ceph-mon[102832]: pgmap v3509: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 232 KiB/s wr, 17 op/s 2023-09-10T01:03:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:57 smithi082 ceph-mon[98694]: pgmap v3509: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 232 KiB/s wr, 17 op/s 2023-09-10T01:03:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:57 smithi098 ceph-mon[103124]: pgmap v3509: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 232 KiB/s wr, 17 op/s 2023-09-10T01:03:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6268.584341 secs 2023-09-10T01:03:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6268.584341 secs 2023-09-10T01:03:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6268.584341 secs 2023-09-10T01:03:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6270.117220 secs 2023-09-10T01:03:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:03:59 smithi170 ceph-mon[102832]: pgmap v3510: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T01:03:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6270.117220 secs 2023-09-10T01:03:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:03:59 smithi082 ceph-mon[98694]: pgmap v3510: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T01:03:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6270.117220 secs 2023-09-10T01:03:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:03:59 smithi098 ceph-mon[103124]: pgmap v3510: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T01:04:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6270.829409 secs 2023-09-10T01:04:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6270.829409 secs 2023-09-10T01:04:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6270.829409 secs 2023-09-10T01:04:01.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:01 smithi170 ceph-mon[102832]: pgmap v3511: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T01:04:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:01 smithi082 ceph-mon[98694]: pgmap v3511: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T01:04:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:01 smithi098 ceph-mon[103124]: pgmap v3511: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T01:04:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6273.584399 secs 2023-09-10T01:04:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:03 smithi170 ceph-mon[102832]: pgmap v3512: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 122 KiB/s wr, 21 op/s 2023-09-10T01:04:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6273.584399 secs 2023-09-10T01:04:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:03 smithi082 ceph-mon[98694]: pgmap v3512: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 122 KiB/s wr, 21 op/s 2023-09-10T01:04:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6273.584399 secs 2023-09-10T01:04:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:03 smithi098 ceph-mon[103124]: pgmap v3512: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 122 KiB/s wr, 21 op/s 2023-09-10T01:04:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6275.117316 secs 2023-09-10T01:04:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6275.117316 secs 2023-09-10T01:04:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6275.117316 secs 2023-09-10T01:04:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6275.829485 secs 2023-09-10T01:04:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:05 smithi170 ceph-mon[102832]: pgmap v3513: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 89 KiB/s wr, 15 op/s 2023-09-10T01:04:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6275.829485 secs 2023-09-10T01:04:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:05 smithi082 ceph-mon[98694]: pgmap v3513: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 89 KiB/s wr, 15 op/s 2023-09-10T01:04:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6275.829485 secs 2023-09-10T01:04:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:05 smithi098 ceph-mon[103124]: pgmap v3513: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 89 KiB/s wr, 15 op/s 2023-09-10T01:04:07.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:07 smithi170 ceph-mon[102832]: pgmap v3514: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 124 KiB/s wr, 30 op/s 2023-09-10T01:04:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:07 smithi082 ceph-mon[98694]: pgmap v3514: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 124 KiB/s wr, 30 op/s 2023-09-10T01:04:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:07 smithi098 ceph-mon[103124]: pgmap v3514: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 124 KiB/s wr, 30 op/s 2023-09-10T01:04:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6278.584494 secs 2023-09-10T01:04:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:08 smithi170 ceph-mon[102832]: osdmap e291: 12 total, 12 up, 12 in 2023-09-10T01:04:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6278.584494 secs 2023-09-10T01:04:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:08 smithi082 ceph-mon[98694]: osdmap e291: 12 total, 12 up, 12 in 2023-09-10T01:04:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6278.584494 secs 2023-09-10T01:04:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:08 smithi098 ceph-mon[103124]: osdmap e291: 12 total, 12 up, 12 in 2023-09-10T01:04:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6280.117374 secs 2023-09-10T01:04:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:09 smithi170 ceph-mon[102832]: pgmap v3516: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T01:04:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6280.117374 secs 2023-09-10T01:04:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:09 smithi082 ceph-mon[98694]: pgmap v3516: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T01:04:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6280.117374 secs 2023-09-10T01:04:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:09 smithi098 ceph-mon[103124]: pgmap v3516: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T01:04:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6280.829548 secs 2023-09-10T01:04:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6280.829548 secs 2023-09-10T01:04:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6280.829548 secs 2023-09-10T01:04:11.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:11 smithi170 ceph-mon[102832]: pgmap v3517: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T01:04:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:11 smithi082 ceph-mon[98694]: pgmap v3517: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T01:04:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:11 smithi098 ceph-mon[103124]: pgmap v3517: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T01:04:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6283.584566 secs 2023-09-10T01:04:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:13 smithi170 ceph-mon[102832]: pgmap v3518: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 107 KiB/s wr, 23 op/s 2023-09-10T01:04:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6283.584566 secs 2023-09-10T01:04:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:13 smithi082 ceph-mon[98694]: pgmap v3518: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 107 KiB/s wr, 23 op/s 2023-09-10T01:04:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6283.584566 secs 2023-09-10T01:04:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:13 smithi098 ceph-mon[103124]: pgmap v3518: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 107 KiB/s wr, 23 op/s 2023-09-10T01:04:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6285.117434 secs 2023-09-10T01:04:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6285.117434 secs 2023-09-10T01:04:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6285.117434 secs 2023-09-10T01:04:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6285.829609 secs 2023-09-10T01:04:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:15 smithi170 ceph-mon[102832]: pgmap v3519: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 107 KiB/s wr, 23 op/s 2023-09-10T01:04:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:15 smithi170 ceph-mon[102832]: osdmap e292: 12 total, 12 up, 12 in 2023-09-10T01:04:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6285.829609 secs 2023-09-10T01:04:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:15 smithi082 ceph-mon[98694]: pgmap v3519: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 107 KiB/s wr, 23 op/s 2023-09-10T01:04:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:15 smithi082 ceph-mon[98694]: osdmap e292: 12 total, 12 up, 12 in 2023-09-10T01:04:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6285.829609 secs 2023-09-10T01:04:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:15 smithi098 ceph-mon[103124]: pgmap v3519: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 107 KiB/s wr, 23 op/s 2023-09-10T01:04:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:15 smithi098 ceph-mon[103124]: osdmap e292: 12 total, 12 up, 12 in 2023-09-10T01:04:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:17 smithi170 ceph-mon[102832]: pgmap v3521: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 13 op/s 2023-09-10T01:04:17.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:17 smithi082 ceph-mon[98694]: pgmap v3521: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 13 op/s 2023-09-10T01:04:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:17 smithi098 ceph-mon[103124]: pgmap v3521: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 13 op/s 2023-09-10T01:04:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6288.584641 secs 2023-09-10T01:04:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6288.584641 secs 2023-09-10T01:04:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6288.584641 secs 2023-09-10T01:04:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6290.117492 secs 2023-09-10T01:04:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:19 smithi170 ceph-mon[102832]: pgmap v3522: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T01:04:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6290.117492 secs 2023-09-10T01:04:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:19 smithi082 ceph-mon[98694]: pgmap v3522: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T01:04:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6290.117492 secs 2023-09-10T01:04:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:19 smithi098 ceph-mon[103124]: pgmap v3522: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T01:04:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6290.829675 secs 2023-09-10T01:04:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6290.829675 secs 2023-09-10T01:04:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6290.829675 secs 2023-09-10T01:04:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:21 smithi170 ceph-mon[102832]: pgmap v3523: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T01:04:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:21 smithi082 ceph-mon[98694]: pgmap v3523: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T01:04:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:21 smithi098 ceph-mon[103124]: pgmap v3523: 97 pgs: 97 active+clean; 9.1 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T01:04:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:23 smithi170 ceph-mon[102832]: pgmap v3524: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:04:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:04:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:23 smithi082 ceph-mon[98694]: pgmap v3524: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:04:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:04:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:23 smithi098 ceph-mon[103124]: pgmap v3524: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:04:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:04:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6295.117593 secs 2023-09-10T01:04:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:04:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:04:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:04:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6295.117593 secs 2023-09-10T01:04:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:04:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:04:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:04:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6295.117593 secs 2023-09-10T01:04:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:04:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:04:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:04:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6295.829737 secs 2023-09-10T01:04:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:25 smithi170 ceph-mon[102832]: pgmap v3525: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:04:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6295.829737 secs 2023-09-10T01:04:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:25 smithi082 ceph-mon[98694]: pgmap v3525: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:04:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6295.829737 secs 2023-09-10T01:04:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:25 smithi098 ceph-mon[103124]: pgmap v3525: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:04:27.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:27 smithi170 ceph-mon[102832]: pgmap v3526: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 133 KiB/s wr, 16 op/s 2023-09-10T01:04:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:27 smithi082 ceph-mon[98694]: pgmap v3526: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 133 KiB/s wr, 16 op/s 2023-09-10T01:04:27.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:27 smithi098 ceph-mon[103124]: pgmap v3526: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 133 KiB/s wr, 16 op/s 2023-09-10T01:04:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6300.117651 secs 2023-09-10T01:04:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:29 smithi170 ceph-mon[102832]: pgmap v3527: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 10 op/s 2023-09-10T01:04:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6300.117651 secs 2023-09-10T01:04:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:29 smithi082 ceph-mon[98694]: pgmap v3527: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 10 op/s 2023-09-10T01:04:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6300.117651 secs 2023-09-10T01:04:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:29 smithi098 ceph-mon[103124]: pgmap v3527: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 10 op/s 2023-09-10T01:04:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6300.829798 secs 2023-09-10T01:04:30.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6300.829798 secs 2023-09-10T01:04:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6300.829798 secs 2023-09-10T01:04:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:31 smithi170 ceph-mon[102832]: pgmap v3528: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 10 op/s 2023-09-10T01:04:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:31 smithi082 ceph-mon[98694]: pgmap v3528: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 10 op/s 2023-09-10T01:04:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:31 smithi098 ceph-mon[103124]: pgmap v3528: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 10 op/s 2023-09-10T01:04:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6303.584869 secs 2023-09-10T01:04:33.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:33 smithi170 ceph-mon[102832]: pgmap v3529: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 207 KiB/s wr, 17 op/s 2023-09-10T01:04:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6303.584869 secs 2023-09-10T01:04:33.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:33 smithi082 ceph-mon[98694]: pgmap v3529: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 207 KiB/s wr, 17 op/s 2023-09-10T01:04:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6303.584869 secs 2023-09-10T01:04:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:33 smithi098 ceph-mon[103124]: pgmap v3529: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 207 KiB/s wr, 17 op/s 2023-09-10T01:04:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6305.117707 secs 2023-09-10T01:04:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6305.117707 secs 2023-09-10T01:04:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6305.117707 secs 2023-09-10T01:04:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6305.829860 secs 2023-09-10T01:04:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:35 smithi170 ceph-mon[102832]: pgmap v3530: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 176 KiB/s wr, 12 op/s 2023-09-10T01:04:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6305.829860 secs 2023-09-10T01:04:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:35 smithi082 ceph-mon[98694]: pgmap v3530: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 176 KiB/s wr, 12 op/s 2023-09-10T01:04:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6305.829860 secs 2023-09-10T01:04:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:35 smithi098 ceph-mon[103124]: pgmap v3530: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 176 KiB/s wr, 12 op/s 2023-09-10T01:04:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:36 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:04:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:04:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:04:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:36 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:04:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:04:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:04:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:36 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:04:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:04:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:04:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:37 smithi170 ceph-mon[102832]: pgmap v3531: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 250 KiB/s wr, 18 op/s 2023-09-10T01:04:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:37 smithi082 ceph-mon[98694]: pgmap v3531: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 250 KiB/s wr, 18 op/s 2023-09-10T01:04:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:37 smithi098 ceph-mon[103124]: pgmap v3531: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 250 KiB/s wr, 18 op/s 2023-09-10T01:04:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6308.584930 secs 2023-09-10T01:04:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6308.584930 secs 2023-09-10T01:04:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6308.584930 secs 2023-09-10T01:04:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6310.117767 secs 2023-09-10T01:04:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:39 smithi170 ceph-mon[102832]: pgmap v3532: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 161 KiB/s wr, 12 op/s 2023-09-10T01:04:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6310.117767 secs 2023-09-10T01:04:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:39 smithi082 ceph-mon[98694]: pgmap v3532: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 161 KiB/s wr, 12 op/s 2023-09-10T01:04:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6310.117767 secs 2023-09-10T01:04:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:39 smithi098 ceph-mon[103124]: pgmap v3532: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 161 KiB/s wr, 12 op/s 2023-09-10T01:04:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6310.829921 secs 2023-09-10T01:04:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6310.829921 secs 2023-09-10T01:04:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6310.829921 secs 2023-09-10T01:04:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:41 smithi082 ceph-mon[98694]: pgmap v3533: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 161 KiB/s wr, 12 op/s 2023-09-10T01:04:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:41 smithi098 ceph-mon[103124]: pgmap v3533: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 161 KiB/s wr, 12 op/s 2023-09-10T01:04:41.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:41 smithi170 ceph-mon[102832]: pgmap v3533: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 161 KiB/s wr, 12 op/s 2023-09-10T01:04:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6313.584993 secs 2023-09-10T01:04:43.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:43 smithi082 ceph-mon[98694]: pgmap v3534: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 246 KiB/s wr, 17 op/s 2023-09-10T01:04:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6313.584993 secs 2023-09-10T01:04:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:43 smithi098 ceph-mon[103124]: pgmap v3534: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 246 KiB/s wr, 17 op/s 2023-09-10T01:04:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6313.584993 secs 2023-09-10T01:04:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:43 smithi170 ceph-mon[102832]: pgmap v3534: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 246 KiB/s wr, 17 op/s 2023-09-10T01:04:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6315.117828 secs 2023-09-10T01:04:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:04:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6315.117828 secs 2023-09-10T01:04:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:04:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6315.117828 secs 2023-09-10T01:04:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:04:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6315.830005 secs 2023-09-10T01:04:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:45 smithi082 ceph-mon[98694]: pgmap v3535: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T01:04:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:04:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:04:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:04:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6315.830005 secs 2023-09-10T01:04:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:45 smithi098 ceph-mon[103124]: pgmap v3535: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T01:04:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:04:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:04:45.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:04:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6315.830005 secs 2023-09-10T01:04:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:45 smithi170 ceph-mon[102832]: pgmap v3535: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T01:04:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:04:45.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:04:45.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:04:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:47 smithi082 ceph-mon[98694]: pgmap v3536: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 230 KiB/s wr, 16 op/s 2023-09-10T01:04:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:47 smithi098 ceph-mon[103124]: pgmap v3536: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 230 KiB/s wr, 16 op/s 2023-09-10T01:04:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:47 smithi170 ceph-mon[102832]: pgmap v3536: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 230 KiB/s wr, 16 op/s 2023-09-10T01:04:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6318.585056 secs 2023-09-10T01:04:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6318.585056 secs 2023-09-10T01:04:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6318.585056 secs 2023-09-10T01:04:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6320.117887 secs 2023-09-10T01:04:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:49 smithi082 ceph-mon[98694]: pgmap v3537: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 156 KiB/s wr, 10 op/s 2023-09-10T01:04:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6320.117887 secs 2023-09-10T01:04:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:49 smithi098 ceph-mon[103124]: pgmap v3537: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 156 KiB/s wr, 10 op/s 2023-09-10T01:04:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6320.117887 secs 2023-09-10T01:04:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:49 smithi170 ceph-mon[102832]: pgmap v3537: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 156 KiB/s wr, 10 op/s 2023-09-10T01:04:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6320.830072 secs 2023-09-10T01:04:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6320.830072 secs 2023-09-10T01:04:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6320.830072 secs 2023-09-10T01:04:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:51 smithi098 ceph-mon[103124]: pgmap v3538: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 156 KiB/s wr, 10 op/s 2023-09-10T01:04:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:51 smithi170 ceph-mon[102832]: pgmap v3538: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 156 KiB/s wr, 10 op/s 2023-09-10T01:04:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:51 smithi082 ceph-mon[98694]: pgmap v3538: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 156 KiB/s wr, 10 op/s 2023-09-10T01:04:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6323.585117 secs 2023-09-10T01:04:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:53 smithi098 ceph-mon[103124]: pgmap v3539: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 194 KiB/s wr, 15 op/s 2023-09-10T01:04:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6323.585117 secs 2023-09-10T01:04:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:53 smithi170 ceph-mon[102832]: pgmap v3539: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 194 KiB/s wr, 15 op/s 2023-09-10T01:04:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6323.585117 secs 2023-09-10T01:04:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:53 smithi082 ceph-mon[98694]: pgmap v3539: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 194 KiB/s wr, 15 op/s 2023-09-10T01:04:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6325.117943 secs 2023-09-10T01:04:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6325.830133 secs 2023-09-10T01:04:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6325.117943 secs 2023-09-10T01:04:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6325.830133 secs 2023-09-10T01:04:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6325.117943 secs 2023-09-10T01:04:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6325.830133 secs 2023-09-10T01:04:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:55 smithi098 ceph-mon[103124]: pgmap v3540: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T01:04:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:55 smithi170 ceph-mon[102832]: pgmap v3540: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T01:04:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:55 smithi082 ceph-mon[98694]: pgmap v3540: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T01:04:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:57 smithi098 ceph-mon[103124]: pgmap v3541: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 164 KiB/s wr, 16 op/s 2023-09-10T01:04:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:57 smithi170 ceph-mon[102832]: pgmap v3541: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 164 KiB/s wr, 16 op/s 2023-09-10T01:04:58.648 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:57 smithi082 ceph-mon[98694]: pgmap v3541: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 164 KiB/s wr, 16 op/s 2023-09-10T01:04:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6328.585262 secs 2023-09-10T01:04:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6328.585262 secs 2023-09-10T01:04:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6328.585262 secs 2023-09-10T01:04:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6330.118005 secs 2023-09-10T01:04:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:59 smithi098 ceph-mon[103124]: pgmap v3542: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 93 KiB/s wr, 10 op/s 2023-09-10T01:04:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:04:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6330.830196 secs 2023-09-10T01:04:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6330.118005 secs 2023-09-10T01:04:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:59 smithi170 ceph-mon[102832]: pgmap v3542: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 93 KiB/s wr, 10 op/s 2023-09-10T01:04:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:04:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6330.830196 secs 2023-09-10T01:04:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6330.118005 secs 2023-09-10T01:04:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:59 smithi082 ceph-mon[98694]: pgmap v3542: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 93 KiB/s wr, 10 op/s 2023-09-10T01:04:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:04:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6330.830196 secs 2023-09-10T01:05:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:01 smithi098 ceph-mon[103124]: pgmap v3543: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 93 KiB/s wr, 10 op/s 2023-09-10T01:05:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:01 smithi170 ceph-mon[102832]: pgmap v3543: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 93 KiB/s wr, 10 op/s 2023-09-10T01:05:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:01 smithi082 ceph-mon[98694]: pgmap v3543: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 93 KiB/s wr, 10 op/s 2023-09-10T01:05:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6333.585328 secs 2023-09-10T01:05:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:03 smithi098 ceph-mon[103124]: pgmap v3544: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 15 op/s 2023-09-10T01:05:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6333.585328 secs 2023-09-10T01:05:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:03 smithi170 ceph-mon[102832]: pgmap v3544: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 15 op/s 2023-09-10T01:05:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6333.585328 secs 2023-09-10T01:05:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:03 smithi082 ceph-mon[98694]: pgmap v3544: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 15 op/s 2023-09-10T01:05:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6335.118061 secs 2023-09-10T01:05:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6335.830260 secs 2023-09-10T01:05:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6335.118061 secs 2023-09-10T01:05:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6335.830260 secs 2023-09-10T01:05:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6335.118061 secs 2023-09-10T01:05:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6335.830260 secs 2023-09-10T01:05:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:05 smithi098 ceph-mon[103124]: pgmap v3545: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 79 KiB/s wr, 10 op/s 2023-09-10T01:05:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:05 smithi170 ceph-mon[102832]: pgmap v3545: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 79 KiB/s wr, 10 op/s 2023-09-10T01:05:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:05 smithi082 ceph-mon[98694]: pgmap v3545: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 79 KiB/s wr, 10 op/s 2023-09-10T01:05:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:07 smithi098 ceph-mon[103124]: pgmap v3546: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 102 KiB/s wr, 27 op/s 2023-09-10T01:05:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:07 smithi170 ceph-mon[102832]: pgmap v3546: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 102 KiB/s wr, 27 op/s 2023-09-10T01:05:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:07 smithi082 ceph-mon[98694]: pgmap v3546: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 102 KiB/s wr, 27 op/s 2023-09-10T01:05:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6338.585429 secs 2023-09-10T01:05:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6338.585429 secs 2023-09-10T01:05:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6338.585429 secs 2023-09-10T01:05:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6340.118120 secs 2023-09-10T01:05:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:09 smithi098 ceph-mon[103124]: pgmap v3547: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 47 KiB/s wr, 22 op/s 2023-09-10T01:05:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6340.830324 secs 2023-09-10T01:05:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6340.118120 secs 2023-09-10T01:05:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:09 smithi170 ceph-mon[102832]: pgmap v3547: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 47 KiB/s wr, 22 op/s 2023-09-10T01:05:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6340.830324 secs 2023-09-10T01:05:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6340.118120 secs 2023-09-10T01:05:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:09 smithi082 ceph-mon[98694]: pgmap v3547: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 47 KiB/s wr, 22 op/s 2023-09-10T01:05:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6340.830324 secs 2023-09-10T01:05:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:11 smithi170 ceph-mon[102832]: pgmap v3548: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 47 KiB/s wr, 22 op/s 2023-09-10T01:05:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:11 smithi082 ceph-mon[98694]: pgmap v3548: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 47 KiB/s wr, 22 op/s 2023-09-10T01:05:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:11 smithi098 ceph-mon[103124]: pgmap v3548: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 47 KiB/s wr, 22 op/s 2023-09-10T01:05:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6343.585533 secs 2023-09-10T01:05:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:13 smithi170 ceph-mon[102832]: pgmap v3549: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 102 KiB/s wr, 28 op/s 2023-09-10T01:05:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6343.585533 secs 2023-09-10T01:05:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:13 smithi082 ceph-mon[98694]: pgmap v3549: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 102 KiB/s wr, 28 op/s 2023-09-10T01:05:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6343.585533 secs 2023-09-10T01:05:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:13 smithi098 ceph-mon[103124]: pgmap v3549: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 102 KiB/s wr, 28 op/s 2023-09-10T01:05:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6345.118178 secs 2023-09-10T01:05:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6345.830390 secs 2023-09-10T01:05:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6345.118178 secs 2023-09-10T01:05:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6345.830390 secs 2023-09-10T01:05:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6345.118178 secs 2023-09-10T01:05:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6345.830390 secs 2023-09-10T01:05:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:15 smithi170 ceph-mon[102832]: pgmap v3550: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 78 KiB/s wr, 22 op/s 2023-09-10T01:05:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:15 smithi082 ceph-mon[98694]: pgmap v3550: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 78 KiB/s wr, 22 op/s 2023-09-10T01:05:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:15 smithi098 ceph-mon[103124]: pgmap v3550: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 78 KiB/s wr, 22 op/s 2023-09-10T01:05:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:17 smithi170 ceph-mon[102832]: pgmap v3551: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 79 KiB/s wr, 28 op/s 2023-09-10T01:05:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:17 smithi082 ceph-mon[98694]: pgmap v3551: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 79 KiB/s wr, 28 op/s 2023-09-10T01:05:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:17 smithi098 ceph-mon[103124]: pgmap v3551: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 79 KiB/s wr, 28 op/s 2023-09-10T01:05:18.785 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6348.585686 secs 2023-09-10T01:05:18.937 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6348.585686 secs 2023-09-10T01:05:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6348.585686 secs 2023-09-10T01:05:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6350.118236 secs 2023-09-10T01:05:19.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:19 smithi098 ceph-mon[103124]: pgmap v3552: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T01:05:19.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6350.830456 secs 2023-09-10T01:05:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6350.118236 secs 2023-09-10T01:05:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:19 smithi170 ceph-mon[102832]: pgmap v3552: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T01:05:19.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6350.830456 secs 2023-09-10T01:05:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6350.118236 secs 2023-09-10T01:05:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:19 smithi082 ceph-mon[98694]: pgmap v3552: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T01:05:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6350.830456 secs 2023-09-10T01:05:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:21 smithi170 ceph-mon[102832]: pgmap v3553: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T01:05:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:21 smithi082 ceph-mon[98694]: pgmap v3553: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T01:05:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:21 smithi098 ceph-mon[103124]: pgmap v3553: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T01:05:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:23 smithi170 ceph-mon[102832]: pgmap v3554: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 88 KiB/s wr, 16 op/s 2023-09-10T01:05:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:05:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:23 smithi082 ceph-mon[98694]: pgmap v3554: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 88 KiB/s wr, 16 op/s 2023-09-10T01:05:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:05:24.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:23 smithi098 ceph-mon[103124]: pgmap v3554: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 88 KiB/s wr, 16 op/s 2023-09-10T01:05:24.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:05:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6355.118295 secs 2023-09-10T01:05:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:05:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:05:24.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:05:24.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6355.830484 secs 2023-09-10T01:05:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6355.118295 secs 2023-09-10T01:05:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:05:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:05:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:05:24.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6355.830484 secs 2023-09-10T01:05:24.966 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6355.118295 secs 2023-09-10T01:05:24.966 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:05:24.966 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:05:24.967 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:05:24.967 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6355.830484 secs 2023-09-10T01:05:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:25 smithi170 ceph-mon[102832]: pgmap v3555: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 10 op/s 2023-09-10T01:05:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:25 smithi082 ceph-mon[98694]: pgmap v3555: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 10 op/s 2023-09-10T01:05:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:25 smithi098 ceph-mon[103124]: pgmap v3555: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 10 op/s 2023-09-10T01:05:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:27 smithi170 ceph-mon[102832]: pgmap v3556: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 16 op/s 2023-09-10T01:05:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:27 smithi082 ceph-mon[98694]: pgmap v3556: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 16 op/s 2023-09-10T01:05:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:27 smithi098 ceph-mon[103124]: pgmap v3556: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 16 op/s 2023-09-10T01:05:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6360.118382 secs 2023-09-10T01:05:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:29 smithi170 ceph-mon[102832]: pgmap v3557: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T01:05:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6360.830551 secs 2023-09-10T01:05:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6360.118382 secs 2023-09-10T01:05:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:29 smithi082 ceph-mon[98694]: pgmap v3557: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T01:05:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6360.830551 secs 2023-09-10T01:05:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6360.118382 secs 2023-09-10T01:05:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:29 smithi098 ceph-mon[103124]: pgmap v3557: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T01:05:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6360.830551 secs 2023-09-10T01:05:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:31 smithi170 ceph-mon[102832]: pgmap v3558: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T01:05:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:31 smithi082 ceph-mon[98694]: pgmap v3558: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T01:05:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:31 smithi098 ceph-mon[103124]: pgmap v3558: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T01:05:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:33 smithi170 ceph-mon[102832]: pgmap v3559: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 209 KiB/s wr, 17 op/s 2023-09-10T01:05:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:33 smithi082 ceph-mon[98694]: pgmap v3559: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 209 KiB/s wr, 17 op/s 2023-09-10T01:05:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:33 smithi098 ceph-mon[103124]: pgmap v3559: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 209 KiB/s wr, 17 op/s 2023-09-10T01:05:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6365.118452 secs 2023-09-10T01:05:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6365.830621 secs 2023-09-10T01:05:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6365.118452 secs 2023-09-10T01:05:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6365.830621 secs 2023-09-10T01:05:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6365.118452 secs 2023-09-10T01:05:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6365.830621 secs 2023-09-10T01:05:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:35 smithi170 ceph-mon[102832]: pgmap v3560: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T01:05:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:35 smithi082 ceph-mon[98694]: pgmap v3560: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T01:05:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:35 smithi098 ceph-mon[103124]: pgmap v3560: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T01:05:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:36 smithi170 ceph-mon[102832]: pgmap v3561: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 244 KiB/s wr, 17 op/s 2023-09-10T01:05:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:36 smithi082 ceph-mon[98694]: pgmap v3561: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 244 KiB/s wr, 17 op/s 2023-09-10T01:05:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:36 smithi098 ceph-mon[103124]: pgmap v3561: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 244 KiB/s wr, 17 op/s 2023-09-10T01:05:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6370.118540 secs 2023-09-10T01:05:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:39 smithi098 ceph-mon[103124]: pgmap v3562: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T01:05:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6370.830684 secs 2023-09-10T01:05:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6370.118540 secs 2023-09-10T01:05:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:39 smithi170 ceph-mon[102832]: pgmap v3562: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T01:05:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6370.830684 secs 2023-09-10T01:05:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6370.118540 secs 2023-09-10T01:05:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:39 smithi082 ceph-mon[98694]: pgmap v3562: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T01:05:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6370.830684 secs 2023-09-10T01:05:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:41 smithi098 ceph-mon[103124]: pgmap v3563: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T01:05:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:41 smithi170 ceph-mon[102832]: pgmap v3563: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T01:05:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:41 smithi082 ceph-mon[98694]: pgmap v3563: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T01:05:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:43 smithi098 ceph-mon[103124]: pgmap v3564: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 239 KiB/s wr, 17 op/s 2023-09-10T01:05:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:43 smithi170 ceph-mon[102832]: pgmap v3564: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 239 KiB/s wr, 17 op/s 2023-09-10T01:05:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:43 smithi082 ceph-mon[98694]: pgmap v3564: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 239 KiB/s wr, 17 op/s 2023-09-10T01:05:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6375.118597 secs 2023-09-10T01:05:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6375.830748 secs 2023-09-10T01:05:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6375.118597 secs 2023-09-10T01:05:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6375.830748 secs 2023-09-10T01:05:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6375.118597 secs 2023-09-10T01:05:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6375.830748 secs 2023-09-10T01:05:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:45 smithi098 ceph-mon[103124]: pgmap v3565: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T01:05:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:05:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:05:45.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:05:45.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:05:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:45 smithi170 ceph-mon[102832]: pgmap v3565: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T01:05:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:05:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:05:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:05:45.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:05:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:45 smithi082 ceph-mon[98694]: pgmap v3565: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T01:05:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:05:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:05:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:05:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:05:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:47 smithi098 ceph-mon[103124]: pgmap v3566: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 229 KiB/s wr, 17 op/s 2023-09-10T01:05:47.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:47 smithi170 ceph-mon[102832]: pgmap v3566: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 229 KiB/s wr, 17 op/s 2023-09-10T01:05:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:47 smithi082 ceph-mon[98694]: pgmap v3566: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 229 KiB/s wr, 17 op/s 2023-09-10T01:05:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6378.586142 secs 2023-09-10T01:05:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6378.586142 secs 2023-09-10T01:05:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6378.586142 secs 2023-09-10T01:05:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6380.118682 secs 2023-09-10T01:05:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:49 smithi098 ceph-mon[103124]: pgmap v3567: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T01:05:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6380.830812 secs 2023-09-10T01:05:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6380.118682 secs 2023-09-10T01:05:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:49 smithi170 ceph-mon[102832]: pgmap v3567: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T01:05:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6380.830812 secs 2023-09-10T01:05:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6380.118682 secs 2023-09-10T01:05:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:49 smithi082 ceph-mon[98694]: pgmap v3567: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T01:05:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6380.830812 secs 2023-09-10T01:05:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:51 smithi170 ceph-mon[102832]: pgmap v3568: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T01:05:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:51 smithi082 ceph-mon[98694]: pgmap v3568: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T01:05:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:51 smithi098 ceph-mon[103124]: pgmap v3568: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T01:05:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:05:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:05:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:05:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:05:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:05:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:05:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:05:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:05:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:05:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6383.586200 secs 2023-09-10T01:05:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:53 smithi170 ceph-mon[102832]: pgmap v3569: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 201 KiB/s wr, 17 op/s 2023-09-10T01:05:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6383.586200 secs 2023-09-10T01:05:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:53 smithi082 ceph-mon[98694]: pgmap v3569: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 201 KiB/s wr, 17 op/s 2023-09-10T01:05:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6383.586200 secs 2023-09-10T01:05:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:53 smithi098 ceph-mon[103124]: pgmap v3569: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 201 KiB/s wr, 17 op/s 2023-09-10T01:05:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6385.118743 secs 2023-09-10T01:05:55.697 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6385.830880 secs 2023-09-10T01:05:55.699 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6385.118743 secs 2023-09-10T01:05:55.699 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6385.830880 secs 2023-09-10T01:05:55.699 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6385.118743 secs 2023-09-10T01:05:55.700 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6385.830880 secs 2023-09-10T01:05:55.700 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:55 smithi170 ceph-mon[102832]: pgmap v3570: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T01:05:55.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:55 smithi082 ceph-mon[98694]: pgmap v3570: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T01:05:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:55 smithi098 ceph-mon[103124]: pgmap v3570: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T01:05:57.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:57 smithi170 ceph-mon[102832]: pgmap v3571: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 170 KiB/s wr, 17 op/s 2023-09-10T01:05:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:57 smithi082 ceph-mon[98694]: pgmap v3571: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 170 KiB/s wr, 17 op/s 2023-09-10T01:05:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:57 smithi098 ceph-mon[103124]: pgmap v3571: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 170 KiB/s wr, 17 op/s 2023-09-10T01:05:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6388.586259 secs 2023-09-10T01:05:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6388.586259 secs 2023-09-10T01:05:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6388.586259 secs 2023-09-10T01:05:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6390.118800 secs 2023-09-10T01:05:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:59 smithi170 ceph-mon[102832]: pgmap v3572: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:05:59.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:05:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6390.830947 secs 2023-09-10T01:05:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6390.118800 secs 2023-09-10T01:05:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:59 smithi082 ceph-mon[98694]: pgmap v3572: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:05:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:05:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6390.830947 secs 2023-09-10T01:05:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6390.118800 secs 2023-09-10T01:05:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:59 smithi098 ceph-mon[103124]: pgmap v3572: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:05:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:05:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6390.830947 secs 2023-09-10T01:06:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:01 smithi170 ceph-mon[102832]: pgmap v3573: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:06:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:01 smithi082 ceph-mon[98694]: pgmap v3573: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:06:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:01 smithi098 ceph-mon[103124]: pgmap v3573: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:06:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6393.586316 secs 2023-09-10T01:06:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:03 smithi170 ceph-mon[102832]: pgmap v3574: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 118 KiB/s wr, 16 op/s 2023-09-10T01:06:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6393.586316 secs 2023-09-10T01:06:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:03 smithi082 ceph-mon[98694]: pgmap v3574: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 118 KiB/s wr, 16 op/s 2023-09-10T01:06:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6393.586316 secs 2023-09-10T01:06:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:03 smithi098 ceph-mon[103124]: pgmap v3574: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 118 KiB/s wr, 16 op/s 2023-09-10T01:06:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6395.118887 secs 2023-09-10T01:06:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6395.831009 secs 2023-09-10T01:06:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6395.118887 secs 2023-09-10T01:06:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6395.831009 secs 2023-09-10T01:06:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6395.118887 secs 2023-09-10T01:06:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6395.831009 secs 2023-09-10T01:06:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:05 smithi170 ceph-mon[102832]: pgmap v3575: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T01:06:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:05 smithi082 ceph-mon[98694]: pgmap v3575: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T01:06:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:05 smithi098 ceph-mon[103124]: pgmap v3575: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T01:06:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:07 smithi170 ceph-mon[102832]: pgmap v3576: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 125 KiB/s wr, 30 op/s 2023-09-10T01:06:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:07 smithi082 ceph-mon[98694]: pgmap v3576: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 125 KiB/s wr, 30 op/s 2023-09-10T01:06:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:07 smithi098 ceph-mon[103124]: pgmap v3576: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 125 KiB/s wr, 30 op/s 2023-09-10T01:06:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6398.586420 secs 2023-09-10T01:06:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:08 smithi170 ceph-mon[102832]: osdmap e293: 12 total, 12 up, 12 in 2023-09-10T01:06:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6398.586420 secs 2023-09-10T01:06:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:08 smithi082 ceph-mon[98694]: osdmap e293: 12 total, 12 up, 12 in 2023-09-10T01:06:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6398.586420 secs 2023-09-10T01:06:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:08 smithi098 ceph-mon[103124]: osdmap e293: 12 total, 12 up, 12 in 2023-09-10T01:06:09.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6400.118973 secs 2023-09-10T01:06:09.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:09 smithi170 ceph-mon[102832]: pgmap v3578: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T01:06:09.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6400.831073 secs 2023-09-10T01:06:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6400.118973 secs 2023-09-10T01:06:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:09 smithi082 ceph-mon[98694]: pgmap v3578: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T01:06:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6400.831073 secs 2023-09-10T01:06:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6400.118973 secs 2023-09-10T01:06:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:09 smithi098 ceph-mon[103124]: pgmap v3578: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T01:06:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6400.831073 secs 2023-09-10T01:06:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:11 smithi170 ceph-mon[102832]: pgmap v3579: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T01:06:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:11 smithi082 ceph-mon[98694]: pgmap v3579: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T01:06:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:11 smithi098 ceph-mon[103124]: pgmap v3579: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T01:06:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6403.586510 secs 2023-09-10T01:06:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:13 smithi170 ceph-mon[102832]: pgmap v3580: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 122 KiB/s wr, 29 op/s 2023-09-10T01:06:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6403.586510 secs 2023-09-10T01:06:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:13 smithi082 ceph-mon[98694]: pgmap v3580: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 122 KiB/s wr, 29 op/s 2023-09-10T01:06:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6403.586510 secs 2023-09-10T01:06:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:13 smithi098 ceph-mon[103124]: pgmap v3580: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 122 KiB/s wr, 29 op/s 2023-09-10T01:06:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6405.119056 secs 2023-09-10T01:06:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6405.831134 secs 2023-09-10T01:06:14.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6405.119056 secs 2023-09-10T01:06:14.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6405.831134 secs 2023-09-10T01:06:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6405.119056 secs 2023-09-10T01:06:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6405.831134 secs 2023-09-10T01:06:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:15 smithi170 ceph-mon[102832]: pgmap v3581: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 122 KiB/s wr, 29 op/s 2023-09-10T01:06:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:15 smithi170 ceph-mon[102832]: osdmap e294: 12 total, 12 up, 12 in 2023-09-10T01:06:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:15 smithi082 ceph-mon[98694]: pgmap v3581: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 122 KiB/s wr, 29 op/s 2023-09-10T01:06:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:15 smithi082 ceph-mon[98694]: osdmap e294: 12 total, 12 up, 12 in 2023-09-10T01:06:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:15 smithi098 ceph-mon[103124]: pgmap v3581: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 122 KiB/s wr, 29 op/s 2023-09-10T01:06:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:15 smithi098 ceph-mon[103124]: osdmap e294: 12 total, 12 up, 12 in 2023-09-10T01:06:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:17 smithi170 ceph-mon[102832]: pgmap v3583: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 76 KiB/s wr, 13 op/s 2023-09-10T01:06:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:17 smithi082 ceph-mon[98694]: pgmap v3583: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 76 KiB/s wr, 13 op/s 2023-09-10T01:06:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:17 smithi098 ceph-mon[103124]: pgmap v3583: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 76 KiB/s wr, 13 op/s 2023-09-10T01:06:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6408.586586 secs 2023-09-10T01:06:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6408.586586 secs 2023-09-10T01:06:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6408.586586 secs 2023-09-10T01:06:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6410.119116 secs 2023-09-10T01:06:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:19 smithi170 ceph-mon[102832]: pgmap v3584: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:06:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6410.831198 secs 2023-09-10T01:06:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6410.119116 secs 2023-09-10T01:06:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:19 smithi082 ceph-mon[98694]: pgmap v3584: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:06:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6410.831198 secs 2023-09-10T01:06:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6410.119116 secs 2023-09-10T01:06:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:19 smithi098 ceph-mon[103124]: pgmap v3584: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:06:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6410.831198 secs 2023-09-10T01:06:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:21 smithi170 ceph-mon[102832]: pgmap v3585: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:06:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:21 smithi082 ceph-mon[98694]: pgmap v3585: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:06:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:21 smithi098 ceph-mon[103124]: pgmap v3585: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:06:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:23 smithi170 ceph-mon[102832]: pgmap v3586: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T01:06:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:06:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:23 smithi082 ceph-mon[98694]: pgmap v3586: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T01:06:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:06:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:23 smithi098 ceph-mon[103124]: pgmap v3586: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T01:06:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:06:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6415.119173 secs 2023-09-10T01:06:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:06:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:06:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:06:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6415.831260 secs 2023-09-10T01:06:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6415.119173 secs 2023-09-10T01:06:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:06:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:06:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:06:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6415.831260 secs 2023-09-10T01:06:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6415.119173 secs 2023-09-10T01:06:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:06:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:06:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:06:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6415.831260 secs 2023-09-10T01:06:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:25 smithi170 ceph-mon[102832]: pgmap v3587: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T01:06:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:25 smithi082 ceph-mon[98694]: pgmap v3587: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T01:06:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:25 smithi098 ceph-mon[103124]: pgmap v3587: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T01:06:27.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:27 smithi170 ceph-mon[102832]: pgmap v3588: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 135 KiB/s wr, 17 op/s 2023-09-10T01:06:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:27 smithi082 ceph-mon[98694]: pgmap v3588: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 135 KiB/s wr, 17 op/s 2023-09-10T01:06:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:27 smithi098 ceph-mon[103124]: pgmap v3588: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 135 KiB/s wr, 17 op/s 2023-09-10T01:06:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6420.119235 secs 2023-09-10T01:06:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:29 smithi170 ceph-mon[102832]: pgmap v3589: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T01:06:29.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6420.831323 secs 2023-09-10T01:06:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6420.119235 secs 2023-09-10T01:06:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:29 smithi082 ceph-mon[98694]: pgmap v3589: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T01:06:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6420.831323 secs 2023-09-10T01:06:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6420.119235 secs 2023-09-10T01:06:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:29 smithi098 ceph-mon[103124]: pgmap v3589: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T01:06:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6420.831323 secs 2023-09-10T01:06:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:31 smithi170 ceph-mon[102832]: pgmap v3590: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T01:06:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:31 smithi082 ceph-mon[98694]: pgmap v3590: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T01:06:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:31 smithi098 ceph-mon[103124]: pgmap v3590: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T01:06:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:33 smithi170 ceph-mon[102832]: pgmap v3591: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 211 KiB/s wr, 18 op/s 2023-09-10T01:06:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:33 smithi082 ceph-mon[98694]: pgmap v3591: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 211 KiB/s wr, 18 op/s 2023-09-10T01:06:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:33 smithi098 ceph-mon[103124]: pgmap v3591: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 211 KiB/s wr, 18 op/s 2023-09-10T01:06:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6425.119293 secs 2023-09-10T01:06:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6425.831389 secs 2023-09-10T01:06:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6425.119293 secs 2023-09-10T01:06:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6425.831389 secs 2023-09-10T01:06:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6425.119293 secs 2023-09-10T01:06:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6425.831389 secs 2023-09-10T01:06:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:35 smithi170 ceph-mon[102832]: pgmap v3592: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 141 KiB/s wr, 13 op/s 2023-09-10T01:06:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:35 smithi082 ceph-mon[98694]: pgmap v3592: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 141 KiB/s wr, 13 op/s 2023-09-10T01:06:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:35 smithi098 ceph-mon[103124]: pgmap v3592: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 141 KiB/s wr, 13 op/s 2023-09-10T01:06:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:37 smithi170 ceph-mon[102832]: pgmap v3593: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 217 KiB/s wr, 18 op/s 2023-09-10T01:06:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:37 smithi082 ceph-mon[98694]: pgmap v3593: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 217 KiB/s wr, 18 op/s 2023-09-10T01:06:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:37 smithi098 ceph-mon[103124]: pgmap v3593: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 217 KiB/s wr, 18 op/s 2023-09-10T01:06:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6430.119349 secs 2023-09-10T01:06:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:39 smithi170 ceph-mon[102832]: pgmap v3594: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 12 op/s 2023-09-10T01:06:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6430.831452 secs 2023-09-10T01:06:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6430.119349 secs 2023-09-10T01:06:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:39 smithi082 ceph-mon[98694]: pgmap v3594: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 12 op/s 2023-09-10T01:06:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6430.831452 secs 2023-09-10T01:06:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6430.119349 secs 2023-09-10T01:06:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:39 smithi098 ceph-mon[103124]: pgmap v3594: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 12 op/s 2023-09-10T01:06:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6430.831452 secs 2023-09-10T01:06:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:41 smithi170 ceph-mon[102832]: pgmap v3595: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 12 op/s 2023-09-10T01:06:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:41 smithi082 ceph-mon[98694]: pgmap v3595: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 12 op/s 2023-09-10T01:06:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:41 smithi098 ceph-mon[103124]: pgmap v3595: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 12 op/s 2023-09-10T01:06:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:43 smithi170 ceph-mon[102832]: pgmap v3596: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 256 KiB/s wr, 18 op/s 2023-09-10T01:06:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:43 smithi082 ceph-mon[98694]: pgmap v3596: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 256 KiB/s wr, 18 op/s 2023-09-10T01:06:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:43 smithi098 ceph-mon[103124]: pgmap v3596: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 256 KiB/s wr, 18 op/s 2023-09-10T01:06:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6435.119397 secs 2023-09-10T01:06:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6435.831484 secs 2023-09-10T01:06:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6435.119397 secs 2023-09-10T01:06:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6435.831484 secs 2023-09-10T01:06:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6435.119397 secs 2023-09-10T01:06:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6435.831484 secs 2023-09-10T01:06:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:45 smithi170 ceph-mon[102832]: pgmap v3597: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 168 KiB/s wr, 11 op/s 2023-09-10T01:06:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:06:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:45 smithi082 ceph-mon[98694]: pgmap v3597: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 168 KiB/s wr, 11 op/s 2023-09-10T01:06:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:06:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:45 smithi098 ceph-mon[103124]: pgmap v3597: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 168 KiB/s wr, 11 op/s 2023-09-10T01:06:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:06:47.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:47 smithi170 ceph-mon[102832]: pgmap v3598: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 235 KiB/s wr, 17 op/s 2023-09-10T01:06:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:47 smithi082 ceph-mon[98694]: pgmap v3598: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 235 KiB/s wr, 17 op/s 2023-09-10T01:06:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:47 smithi098 ceph-mon[103124]: pgmap v3598: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 235 KiB/s wr, 17 op/s 2023-09-10T01:06:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6438.587030 secs 2023-09-10T01:06:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:06:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:06:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:06:48.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:06:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6438.587030 secs 2023-09-10T01:06:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:06:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:06:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:06:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:06:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6438.587030 secs 2023-09-10T01:06:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:06:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:06:48.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:06:48.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:06:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6440.119457 secs 2023-09-10T01:06:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:49 smithi170 ceph-mon[102832]: pgmap v3599: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T01:06:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:06:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:06:49.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6440.831550 secs 2023-09-10T01:06:49.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:49.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:49.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:49.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:49.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:49.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:49.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:49.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:49.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:49.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:49.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:49.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:49.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:06:49.932 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:06:49.932 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:06:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6440.119457 secs 2023-09-10T01:06:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:49 smithi082 ceph-mon[98694]: pgmap v3599: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T01:06:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:06:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:06:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6440.831550 secs 2023-09-10T01:06:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:49.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:49.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:49.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:49.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:49.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:49.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:49.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:49.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:49.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:49.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:49.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:49.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:06:49.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:06:49.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:06:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6440.119457 secs 2023-09-10T01:06:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:49 smithi098 ceph-mon[103124]: pgmap v3599: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T01:06:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:06:50.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:06:50.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6440.831550 secs 2023-09-10T01:06:50.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:50.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:50.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:50.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:50.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:50.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:50.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:50.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:50.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:50.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:50.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:50.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:06:50.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:06:50.008 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:06:50.008 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:06:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:50 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T01:06:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:50 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:06:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:50 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T01:06:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:50 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T01:06:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:50 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T01:06:50.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:50 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T01:06:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:50 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T01:06:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:50 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:06:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:50 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T01:06:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:50 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T01:06:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:50 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T01:06:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:50 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T01:06:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:50 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T01:06:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:50 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:06:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:50 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T01:06:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:50 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T01:06:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:50 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T01:06:51.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:50 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T01:06:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:51 smithi170 ceph-mon[102832]: pgmap v3600: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T01:06:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:51 smithi082 ceph-mon[98694]: pgmap v3600: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T01:06:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:51 smithi098 ceph-mon[103124]: pgmap v3600: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T01:06:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:06:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:06:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:06:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:06:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:06:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:06:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:06:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:06:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:06:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6443.587089 secs 2023-09-10T01:06:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:53 smithi170 ceph-mon[102832]: pgmap v3601: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 249 KiB/s wr, 17 op/s 2023-09-10T01:06:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6443.587089 secs 2023-09-10T01:06:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:53 smithi082 ceph-mon[98694]: pgmap v3601: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 249 KiB/s wr, 17 op/s 2023-09-10T01:06:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6443.587089 secs 2023-09-10T01:06:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:53 smithi098 ceph-mon[103124]: pgmap v3601: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 249 KiB/s wr, 17 op/s 2023-09-10T01:06:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6445.119517 secs 2023-09-10T01:06:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6445.831614 secs 2023-09-10T01:06:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6445.119517 secs 2023-09-10T01:06:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6445.831614 secs 2023-09-10T01:06:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6445.119517 secs 2023-09-10T01:06:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6445.831614 secs 2023-09-10T01:06:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:55 smithi170 ceph-mon[102832]: pgmap v3602: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 157 KiB/s wr, 11 op/s 2023-09-10T01:06:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:55 smithi082 ceph-mon[98694]: pgmap v3602: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 157 KiB/s wr, 11 op/s 2023-09-10T01:06:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:55 smithi098 ceph-mon[103124]: pgmap v3602: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 157 KiB/s wr, 11 op/s 2023-09-10T01:06:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:57 smithi170 ceph-mon[102832]: pgmap v3603: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 157 KiB/s wr, 16 op/s 2023-09-10T01:06:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:57 smithi082 ceph-mon[98694]: pgmap v3603: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 157 KiB/s wr, 16 op/s 2023-09-10T01:06:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:57 smithi098 ceph-mon[103124]: pgmap v3603: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 157 KiB/s wr, 16 op/s 2023-09-10T01:06:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6448.587191 secs 2023-09-10T01:06:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6448.587191 secs 2023-09-10T01:06:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6448.587191 secs 2023-09-10T01:06:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6450.119579 secs 2023-09-10T01:06:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:59 smithi170 ceph-mon[102832]: pgmap v3604: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T01:06:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:06:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6450.831681 secs 2023-09-10T01:06:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6450.119579 secs 2023-09-10T01:06:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:59 smithi082 ceph-mon[98694]: pgmap v3604: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T01:06:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:06:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6450.831681 secs 2023-09-10T01:07:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6450.119579 secs 2023-09-10T01:07:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:59 smithi098 ceph-mon[103124]: pgmap v3604: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T01:07:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:06:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6450.831681 secs 2023-09-10T01:07:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:01 smithi170 ceph-mon[102832]: pgmap v3605: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T01:07:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:01 smithi082 ceph-mon[98694]: pgmap v3605: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T01:07:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:01 smithi098 ceph-mon[103124]: pgmap v3605: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T01:07:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6453.587299 secs 2023-09-10T01:07:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:03 smithi170 ceph-mon[102832]: pgmap v3606: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 114 KiB/s wr, 15 op/s 2023-09-10T01:07:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6453.587299 secs 2023-09-10T01:07:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:03 smithi082 ceph-mon[98694]: pgmap v3606: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 114 KiB/s wr, 15 op/s 2023-09-10T01:07:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6453.587299 secs 2023-09-10T01:07:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:03 smithi098 ceph-mon[103124]: pgmap v3606: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 114 KiB/s wr, 15 op/s 2023-09-10T01:07:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6455.119637 secs 2023-09-10T01:07:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6455.831744 secs 2023-09-10T01:07:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6455.119637 secs 2023-09-10T01:07:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6455.831744 secs 2023-09-10T01:07:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6455.119637 secs 2023-09-10T01:07:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6455.831744 secs 2023-09-10T01:07:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:05 smithi170 ceph-mon[102832]: pgmap v3607: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 25 KiB/s wr, 9 op/s 2023-09-10T01:07:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:05 smithi082 ceph-mon[98694]: pgmap v3607: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 25 KiB/s wr, 9 op/s 2023-09-10T01:07:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:05 smithi098 ceph-mon[103124]: pgmap v3607: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 25 KiB/s wr, 9 op/s 2023-09-10T01:07:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:06 smithi170 ceph-mon[102832]: pgmap v3608: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 107 KiB/s wr, 29 op/s 2023-09-10T01:07:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:06 smithi082 ceph-mon[98694]: pgmap v3608: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 107 KiB/s wr, 29 op/s 2023-09-10T01:07:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:06 smithi098 ceph-mon[103124]: pgmap v3608: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 107 KiB/s wr, 29 op/s 2023-09-10T01:07:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6458.587357 secs 2023-09-10T01:07:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:08 smithi170 ceph-mon[102832]: osdmap e295: 12 total, 12 up, 12 in 2023-09-10T01:07:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6460.119707 secs 2023-09-10T01:07:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:08 smithi170 ceph-mon[102832]: pgmap v3610: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:07:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6458.587357 secs 2023-09-10T01:07:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:08 smithi082 ceph-mon[98694]: osdmap e295: 12 total, 12 up, 12 in 2023-09-10T01:07:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6460.119707 secs 2023-09-10T01:07:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:08 smithi082 ceph-mon[98694]: pgmap v3610: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:07:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6458.587357 secs 2023-09-10T01:07:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:08 smithi098 ceph-mon[103124]: osdmap e295: 12 total, 12 up, 12 in 2023-09-10T01:07:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6460.119707 secs 2023-09-10T01:07:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:08 smithi098 ceph-mon[103124]: pgmap v3610: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:07:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6460.831804 secs 2023-09-10T01:07:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6460.831804 secs 2023-09-10T01:07:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6460.831804 secs 2023-09-10T01:07:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:11 smithi098 ceph-mon[103124]: pgmap v3611: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:07:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:11 smithi170 ceph-mon[102832]: pgmap v3611: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:07:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:11 smithi082 ceph-mon[98694]: pgmap v3611: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:07:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6463.587417 secs 2023-09-10T01:07:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:13 smithi098 ceph-mon[103124]: pgmap v3612: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 122 KiB/s wr, 29 op/s 2023-09-10T01:07:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6463.587417 secs 2023-09-10T01:07:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:13 smithi170 ceph-mon[102832]: pgmap v3612: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 122 KiB/s wr, 29 op/s 2023-09-10T01:07:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6463.587417 secs 2023-09-10T01:07:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:13 smithi082 ceph-mon[98694]: pgmap v3612: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 122 KiB/s wr, 29 op/s 2023-09-10T01:07:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6465.119790 secs 2023-09-10T01:07:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6465.831864 secs 2023-09-10T01:07:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6465.119790 secs 2023-09-10T01:07:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6465.831864 secs 2023-09-10T01:07:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6465.119790 secs 2023-09-10T01:07:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6465.831864 secs 2023-09-10T01:07:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:15 smithi098 ceph-mon[103124]: pgmap v3613: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 122 KiB/s wr, 29 op/s 2023-09-10T01:07:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:15 smithi098 ceph-mon[103124]: osdmap e296: 12 total, 12 up, 12 in 2023-09-10T01:07:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:15 smithi170 ceph-mon[102832]: pgmap v3613: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 122 KiB/s wr, 29 op/s 2023-09-10T01:07:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:15 smithi170 ceph-mon[102832]: osdmap e296: 12 total, 12 up, 12 in 2023-09-10T01:07:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:15 smithi082 ceph-mon[98694]: pgmap v3613: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 122 KiB/s wr, 29 op/s 2023-09-10T01:07:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:15 smithi082 ceph-mon[98694]: osdmap e296: 12 total, 12 up, 12 in 2023-09-10T01:07:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:17 smithi098 ceph-mon[103124]: pgmap v3615: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T01:07:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:17 smithi170 ceph-mon[102832]: pgmap v3615: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T01:07:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:17 smithi082 ceph-mon[98694]: pgmap v3615: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T01:07:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6468.587535 secs 2023-09-10T01:07:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6468.587535 secs 2023-09-10T01:07:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6468.587535 secs 2023-09-10T01:07:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6470.119868 secs 2023-09-10T01:07:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:19 smithi098 ceph-mon[103124]: pgmap v3616: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T01:07:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6470.831928 secs 2023-09-10T01:07:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6470.119868 secs 2023-09-10T01:07:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:19 smithi170 ceph-mon[102832]: pgmap v3616: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T01:07:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6470.831928 secs 2023-09-10T01:07:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6470.119868 secs 2023-09-10T01:07:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:19 smithi082 ceph-mon[98694]: pgmap v3616: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T01:07:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6470.831928 secs 2023-09-10T01:07:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:21 smithi098 ceph-mon[103124]: pgmap v3617: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T01:07:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:21 smithi170 ceph-mon[102832]: pgmap v3617: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T01:07:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:21 smithi082 ceph-mon[98694]: pgmap v3617: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T01:07:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:23 smithi098 ceph-mon[103124]: pgmap v3618: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 130 KiB/s wr, 11 op/s 2023-09-10T01:07:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:07:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:23 smithi170 ceph-mon[102832]: pgmap v3618: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 130 KiB/s wr, 11 op/s 2023-09-10T01:07:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:07:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:23 smithi082 ceph-mon[98694]: pgmap v3618: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 130 KiB/s wr, 11 op/s 2023-09-10T01:07:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:07:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6475.119928 secs 2023-09-10T01:07:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:07:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:07:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:07:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6475.831994 secs 2023-09-10T01:07:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6475.119928 secs 2023-09-10T01:07:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:07:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:07:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:07:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6475.831994 secs 2023-09-10T01:07:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6475.119928 secs 2023-09-10T01:07:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:07:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:07:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:07:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6475.831994 secs 2023-09-10T01:07:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:25 smithi098 ceph-mon[103124]: pgmap v3619: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 130 KiB/s wr, 11 op/s 2023-09-10T01:07:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:25 smithi170 ceph-mon[102832]: pgmap v3619: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 130 KiB/s wr, 11 op/s 2023-09-10T01:07:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:25 smithi082 ceph-mon[98694]: pgmap v3619: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 130 KiB/s wr, 11 op/s 2023-09-10T01:07:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:27 smithi098 ceph-mon[103124]: pgmap v3620: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 136 KiB/s wr, 16 op/s 2023-09-10T01:07:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:27 smithi170 ceph-mon[102832]: pgmap v3620: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 136 KiB/s wr, 16 op/s 2023-09-10T01:07:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:27 smithi082 ceph-mon[98694]: pgmap v3620: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 136 KiB/s wr, 16 op/s 2023-09-10T01:07:29.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6480.119986 secs 2023-09-10T01:07:29.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:29 smithi098 ceph-mon[103124]: pgmap v3621: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T01:07:29.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6480.832051 secs 2023-09-10T01:07:29.683 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6480.119986 secs 2023-09-10T01:07:29.683 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:29 smithi170 ceph-mon[102832]: pgmap v3621: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T01:07:29.683 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6480.832051 secs 2023-09-10T01:07:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6480.119986 secs 2023-09-10T01:07:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:29 smithi082 ceph-mon[98694]: pgmap v3621: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T01:07:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6480.832051 secs 2023-09-10T01:07:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:31 smithi170 ceph-mon[102832]: pgmap v3622: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T01:07:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:31 smithi082 ceph-mon[98694]: pgmap v3622: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T01:07:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:31 smithi098 ceph-mon[103124]: pgmap v3622: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T01:07:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:33 smithi170 ceph-mon[102832]: pgmap v3623: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 241 KiB/s wr, 17 op/s 2023-09-10T01:07:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:33 smithi082 ceph-mon[98694]: pgmap v3623: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 241 KiB/s wr, 17 op/s 2023-09-10T01:07:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:33 smithi098 ceph-mon[103124]: pgmap v3623: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 241 KiB/s wr, 17 op/s 2023-09-10T01:07:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6485.120070 secs 2023-09-10T01:07:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6485.832111 secs 2023-09-10T01:07:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6485.120070 secs 2023-09-10T01:07:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6485.832111 secs 2023-09-10T01:07:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6485.120070 secs 2023-09-10T01:07:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6485.832111 secs 2023-09-10T01:07:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:35 smithi170 ceph-mon[102832]: pgmap v3624: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 165 KiB/s wr, 12 op/s 2023-09-10T01:07:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:35 smithi082 ceph-mon[98694]: pgmap v3624: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 165 KiB/s wr, 12 op/s 2023-09-10T01:07:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:35 smithi098 ceph-mon[103124]: pgmap v3624: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 165 KiB/s wr, 12 op/s 2023-09-10T01:07:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:37 smithi170 ceph-mon[102832]: pgmap v3625: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 181 KiB/s wr, 18 op/s 2023-09-10T01:07:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:37 smithi082 ceph-mon[98694]: pgmap v3625: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 181 KiB/s wr, 18 op/s 2023-09-10T01:07:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:37 smithi098 ceph-mon[103124]: pgmap v3625: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 181 KiB/s wr, 18 op/s 2023-09-10T01:07:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6490.120130 secs 2023-09-10T01:07:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:39 smithi170 ceph-mon[102832]: pgmap v3626: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 165 KiB/s wr, 13 op/s 2023-09-10T01:07:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6490.832174 secs 2023-09-10T01:07:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6490.120130 secs 2023-09-10T01:07:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:39 smithi082 ceph-mon[98694]: pgmap v3626: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 165 KiB/s wr, 13 op/s 2023-09-10T01:07:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6490.832174 secs 2023-09-10T01:07:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6490.120130 secs 2023-09-10T01:07:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:39 smithi098 ceph-mon[103124]: pgmap v3626: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 165 KiB/s wr, 13 op/s 2023-09-10T01:07:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6490.832174 secs 2023-09-10T01:07:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:41 smithi170 ceph-mon[102832]: pgmap v3627: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 165 KiB/s wr, 13 op/s 2023-09-10T01:07:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:41 smithi082 ceph-mon[98694]: pgmap v3627: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 165 KiB/s wr, 13 op/s 2023-09-10T01:07:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:41 smithi098 ceph-mon[103124]: pgmap v3627: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 165 KiB/s wr, 13 op/s 2023-09-10T01:07:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:43 smithi170 ceph-mon[102832]: pgmap v3628: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 309 KiB/s wr, 19 op/s 2023-09-10T01:07:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:43 smithi082 ceph-mon[98694]: pgmap v3628: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 309 KiB/s wr, 19 op/s 2023-09-10T01:07:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:43 smithi098 ceph-mon[103124]: pgmap v3628: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 309 KiB/s wr, 19 op/s 2023-09-10T01:07:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6495.120190 secs 2023-09-10T01:07:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6495.832235 secs 2023-09-10T01:07:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6495.120190 secs 2023-09-10T01:07:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6495.832235 secs 2023-09-10T01:07:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6495.120190 secs 2023-09-10T01:07:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6495.832235 secs 2023-09-10T01:07:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:45 smithi170 ceph-mon[102832]: pgmap v3629: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 161 KiB/s wr, 12 op/s 2023-09-10T01:07:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:45 smithi082 ceph-mon[98694]: pgmap v3629: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 161 KiB/s wr, 12 op/s 2023-09-10T01:07:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:45 smithi098 ceph-mon[103124]: pgmap v3629: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 161 KiB/s wr, 12 op/s 2023-09-10T01:07:47.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:47 smithi170 ceph-mon[102832]: pgmap v3630: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 177 KiB/s wr, 18 op/s 2023-09-10T01:07:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:47 smithi082 ceph-mon[98694]: pgmap v3630: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 177 KiB/s wr, 18 op/s 2023-09-10T01:07:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:47 smithi098 ceph-mon[103124]: pgmap v3630: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 177 KiB/s wr, 18 op/s 2023-09-10T01:07:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6498.588134 secs 2023-09-10T01:07:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6498.588134 secs 2023-09-10T01:07:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6498.588134 secs 2023-09-10T01:07:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6500.120249 secs 2023-09-10T01:07:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:49 smithi170 ceph-mon[102832]: pgmap v3631: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 161 KiB/s wr, 11 op/s 2023-09-10T01:07:49.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6500.832296 secs 2023-09-10T01:07:49.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:07:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6500.120249 secs 2023-09-10T01:07:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:49 smithi082 ceph-mon[98694]: pgmap v3631: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 161 KiB/s wr, 11 op/s 2023-09-10T01:07:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6500.832296 secs 2023-09-10T01:07:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:07:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6500.120249 secs 2023-09-10T01:07:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:49 smithi098 ceph-mon[103124]: pgmap v3631: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 161 KiB/s wr, 11 op/s 2023-09-10T01:07:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6500.832296 secs 2023-09-10T01:07:49.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:07:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:07:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:07:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:07:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:07:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:07:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:07:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:07:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:07:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:07:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:51 smithi170 ceph-mon[102832]: pgmap v3632: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 161 KiB/s wr, 11 op/s 2023-09-10T01:07:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:51 smithi082 ceph-mon[98694]: pgmap v3632: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 161 KiB/s wr, 11 op/s 2023-09-10T01:07:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:51 smithi098 ceph-mon[103124]: pgmap v3632: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 161 KiB/s wr, 11 op/s 2023-09-10T01:07:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:07:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:07:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:07:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:07:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:07:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:07:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:07:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:07:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:07:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6503.588194 secs 2023-09-10T01:07:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:53 smithi170 ceph-mon[102832]: pgmap v3633: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 258 KiB/s wr, 17 op/s 2023-09-10T01:07:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6503.588194 secs 2023-09-10T01:07:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:53 smithi082 ceph-mon[98694]: pgmap v3633: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 258 KiB/s wr, 17 op/s 2023-09-10T01:07:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6503.588194 secs 2023-09-10T01:07:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:53 smithi098 ceph-mon[103124]: pgmap v3633: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 258 KiB/s wr, 17 op/s 2023-09-10T01:07:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6505.120337 secs 2023-09-10T01:07:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6505.832357 secs 2023-09-10T01:07:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6505.120337 secs 2023-09-10T01:07:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6505.832357 secs 2023-09-10T01:07:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6505.120337 secs 2023-09-10T01:07:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6505.832357 secs 2023-09-10T01:07:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:55 smithi170 ceph-mon[102832]: pgmap v3634: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T01:07:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:55 smithi082 ceph-mon[98694]: pgmap v3634: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T01:07:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:55 smithi098 ceph-mon[103124]: pgmap v3634: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T01:07:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:57 smithi170 ceph-mon[102832]: pgmap v3635: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 135 KiB/s wr, 16 op/s 2023-09-10T01:07:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:57 smithi082 ceph-mon[98694]: pgmap v3635: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 135 KiB/s wr, 16 op/s 2023-09-10T01:07:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:57 smithi098 ceph-mon[103124]: pgmap v3635: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 135 KiB/s wr, 16 op/s 2023-09-10T01:07:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6508.588255 secs 2023-09-10T01:07:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6508.588255 secs 2023-09-10T01:07:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6508.588255 secs 2023-09-10T01:07:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6510.120431 secs 2023-09-10T01:08:00.132 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:59 smithi170 ceph-mon[102832]: pgmap v3636: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T01:08:00.132 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:07:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6510.832417 secs 2023-09-10T01:08:00.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6510.120431 secs 2023-09-10T01:08:00.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:59 smithi098 ceph-mon[103124]: pgmap v3636: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T01:08:00.134 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:07:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6510.832417 secs 2023-09-10T01:08:00.135 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6510.120431 secs 2023-09-10T01:08:00.135 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:59 smithi082 ceph-mon[98694]: pgmap v3636: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T01:08:00.135 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:07:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6510.832417 secs 2023-09-10T01:08:01.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:01 smithi170 ceph-mon[102832]: pgmap v3637: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T01:08:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:01 smithi082 ceph-mon[98694]: pgmap v3637: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T01:08:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:01 smithi098 ceph-mon[103124]: pgmap v3637: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T01:08:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6513.588400 secs 2023-09-10T01:08:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:03 smithi170 ceph-mon[102832]: pgmap v3638: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 16 op/s 2023-09-10T01:08:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6513.588400 secs 2023-09-10T01:08:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:03 smithi082 ceph-mon[98694]: pgmap v3638: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 16 op/s 2023-09-10T01:08:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6513.588400 secs 2023-09-10T01:08:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:03 smithi098 ceph-mon[103124]: pgmap v3638: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 16 op/s 2023-09-10T01:08:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6515.120491 secs 2023-09-10T01:08:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6515.832510 secs 2023-09-10T01:08:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6515.120491 secs 2023-09-10T01:08:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6515.832510 secs 2023-09-10T01:08:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6515.120491 secs 2023-09-10T01:08:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6515.832510 secs 2023-09-10T01:08:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:05 smithi170 ceph-mon[102832]: pgmap v3639: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 47 KiB/s wr, 10 op/s 2023-09-10T01:08:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:05 smithi082 ceph-mon[98694]: pgmap v3639: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 47 KiB/s wr, 10 op/s 2023-09-10T01:08:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:05 smithi098 ceph-mon[103124]: pgmap v3639: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 47 KiB/s wr, 10 op/s 2023-09-10T01:08:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:07 smithi170 ceph-mon[102832]: pgmap v3640: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 92 KiB/s wr, 30 op/s 2023-09-10T01:08:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:07 smithi082 ceph-mon[98694]: pgmap v3640: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 92 KiB/s wr, 30 op/s 2023-09-10T01:08:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:07 smithi098 ceph-mon[103124]: pgmap v3640: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 92 KiB/s wr, 30 op/s 2023-09-10T01:08:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6518.588460 secs 2023-09-10T01:08:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:08 smithi170 ceph-mon[102832]: osdmap e297: 12 total, 12 up, 12 in 2023-09-10T01:08:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6518.588460 secs 2023-09-10T01:08:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:08 smithi082 ceph-mon[98694]: osdmap e297: 12 total, 12 up, 12 in 2023-09-10T01:08:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6518.588460 secs 2023-09-10T01:08:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:08 smithi098 ceph-mon[103124]: osdmap e297: 12 total, 12 up, 12 in 2023-09-10T01:08:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6520.120576 secs 2023-09-10T01:08:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:09 smithi170 ceph-mon[102832]: pgmap v3642: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T01:08:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6520.832571 secs 2023-09-10T01:08:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6520.120576 secs 2023-09-10T01:08:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:09 smithi082 ceph-mon[98694]: pgmap v3642: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T01:08:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6520.832571 secs 2023-09-10T01:08:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6520.120576 secs 2023-09-10T01:08:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:09 smithi098 ceph-mon[103124]: pgmap v3642: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T01:08:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6520.832571 secs 2023-09-10T01:08:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:11 smithi170 ceph-mon[102832]: pgmap v3643: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T01:08:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:11 smithi082 ceph-mon[98694]: pgmap v3643: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T01:08:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:11 smithi098 ceph-mon[103124]: pgmap v3643: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T01:08:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6523.588520 secs 2023-09-10T01:08:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:13 smithi170 ceph-mon[102832]: pgmap v3644: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T01:08:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6523.588520 secs 2023-09-10T01:08:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:13 smithi082 ceph-mon[98694]: pgmap v3644: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T01:08:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6523.588520 secs 2023-09-10T01:08:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:13 smithi098 ceph-mon[103124]: pgmap v3644: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T01:08:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6525.120635 secs 2023-09-10T01:08:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6525.832635 secs 2023-09-10T01:08:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6525.120635 secs 2023-09-10T01:08:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6525.832635 secs 2023-09-10T01:08:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6525.120635 secs 2023-09-10T01:08:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6525.832635 secs 2023-09-10T01:08:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:15 smithi170 ceph-mon[102832]: pgmap v3645: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T01:08:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:15 smithi170 ceph-mon[102832]: osdmap e298: 12 total, 12 up, 12 in 2023-09-10T01:08:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:15 smithi082 ceph-mon[98694]: pgmap v3645: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T01:08:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:15 smithi082 ceph-mon[98694]: osdmap e298: 12 total, 12 up, 12 in 2023-09-10T01:08:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:15 smithi098 ceph-mon[103124]: pgmap v3645: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T01:08:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:15 smithi098 ceph-mon[103124]: osdmap e298: 12 total, 12 up, 12 in 2023-09-10T01:08:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:17 smithi170 ceph-mon[102832]: pgmap v3647: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 121 KiB/s wr, 13 op/s 2023-09-10T01:08:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:17 smithi082 ceph-mon[98694]: pgmap v3647: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 121 KiB/s wr, 13 op/s 2023-09-10T01:08:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:17 smithi098 ceph-mon[103124]: pgmap v3647: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 121 KiB/s wr, 13 op/s 2023-09-10T01:08:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6528.588591 secs 2023-09-10T01:08:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6528.588591 secs 2023-09-10T01:08:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6528.588591 secs 2023-09-10T01:08:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6530.120697 secs 2023-09-10T01:08:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:19 smithi170 ceph-mon[102832]: pgmap v3648: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 108 KiB/s wr, 11 op/s 2023-09-10T01:08:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6530.832696 secs 2023-09-10T01:08:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6530.120697 secs 2023-09-10T01:08:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:19 smithi082 ceph-mon[98694]: pgmap v3648: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 108 KiB/s wr, 11 op/s 2023-09-10T01:08:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6530.832696 secs 2023-09-10T01:08:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6530.120697 secs 2023-09-10T01:08:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:19 smithi098 ceph-mon[103124]: pgmap v3648: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 108 KiB/s wr, 11 op/s 2023-09-10T01:08:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6530.832696 secs 2023-09-10T01:08:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:21 smithi170 ceph-mon[102832]: pgmap v3649: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 108 KiB/s wr, 11 op/s 2023-09-10T01:08:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:21 smithi082 ceph-mon[98694]: pgmap v3649: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 108 KiB/s wr, 11 op/s 2023-09-10T01:08:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:21 smithi098 ceph-mon[103124]: pgmap v3649: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 108 KiB/s wr, 11 op/s 2023-09-10T01:08:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:23 smithi170 ceph-mon[102832]: pgmap v3650: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T01:08:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:08:23.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:23 smithi082 ceph-mon[98694]: pgmap v3650: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T01:08:23.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:08:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:23 smithi098 ceph-mon[103124]: pgmap v3650: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T01:08:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:08:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6535.120753 secs 2023-09-10T01:08:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:08:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:08:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:08:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6535.832757 secs 2023-09-10T01:08:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6535.120753 secs 2023-09-10T01:08:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:08:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:08:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:08:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6535.832757 secs 2023-09-10T01:08:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6535.120753 secs 2023-09-10T01:08:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:08:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:08:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:08:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6535.832757 secs 2023-09-10T01:08:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:25 smithi170 ceph-mon[102832]: pgmap v3651: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T01:08:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:25 smithi082 ceph-mon[98694]: pgmap v3651: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T01:08:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:25 smithi098 ceph-mon[103124]: pgmap v3651: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T01:08:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:27 smithi170 ceph-mon[102832]: pgmap v3652: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 137 KiB/s wr, 16 op/s 2023-09-10T01:08:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:27 smithi082 ceph-mon[98694]: pgmap v3652: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 137 KiB/s wr, 16 op/s 2023-09-10T01:08:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:27 smithi098 ceph-mon[103124]: pgmap v3652: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 137 KiB/s wr, 16 op/s 2023-09-10T01:08:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6540.120812 secs 2023-09-10T01:08:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:29 smithi170 ceph-mon[102832]: pgmap v3653: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 93 KiB/s wr, 10 op/s 2023-09-10T01:08:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6540.832817 secs 2023-09-10T01:08:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6540.120812 secs 2023-09-10T01:08:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:29 smithi082 ceph-mon[98694]: pgmap v3653: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 93 KiB/s wr, 10 op/s 2023-09-10T01:08:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6540.832817 secs 2023-09-10T01:08:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6540.120812 secs 2023-09-10T01:08:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:29 smithi098 ceph-mon[103124]: pgmap v3653: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 93 KiB/s wr, 10 op/s 2023-09-10T01:08:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6540.832817 secs 2023-09-10T01:08:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:31 smithi170 ceph-mon[102832]: pgmap v3654: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 93 KiB/s wr, 10 op/s 2023-09-10T01:08:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:31 smithi082 ceph-mon[98694]: pgmap v3654: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 93 KiB/s wr, 10 op/s 2023-09-10T01:08:31.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:31 smithi098 ceph-mon[103124]: pgmap v3654: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 93 KiB/s wr, 10 op/s 2023-09-10T01:08:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:33 smithi170 ceph-mon[102832]: pgmap v3655: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 243 KiB/s wr, 16 op/s 2023-09-10T01:08:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:33 smithi082 ceph-mon[98694]: pgmap v3655: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 243 KiB/s wr, 16 op/s 2023-09-10T01:08:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:33 smithi098 ceph-mon[103124]: pgmap v3655: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 243 KiB/s wr, 16 op/s 2023-09-10T01:08:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6545.120870 secs 2023-09-10T01:08:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6545.832877 secs 2023-09-10T01:08:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6545.120870 secs 2023-09-10T01:08:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6545.832877 secs 2023-09-10T01:08:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6545.120870 secs 2023-09-10T01:08:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6545.832877 secs 2023-09-10T01:08:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:35 smithi170 ceph-mon[102832]: pgmap v3656: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 204 KiB/s wr, 12 op/s 2023-09-10T01:08:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:35 smithi082 ceph-mon[98694]: pgmap v3656: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 204 KiB/s wr, 12 op/s 2023-09-10T01:08:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:35 smithi098 ceph-mon[103124]: pgmap v3656: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 204 KiB/s wr, 12 op/s 2023-09-10T01:08:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:37 smithi170 ceph-mon[102832]: pgmap v3657: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 220 KiB/s wr, 18 op/s 2023-09-10T01:08:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:37 smithi082 ceph-mon[98694]: pgmap v3657: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 220 KiB/s wr, 18 op/s 2023-09-10T01:08:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:37 smithi098 ceph-mon[103124]: pgmap v3657: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 220 KiB/s wr, 18 op/s 2023-09-10T01:08:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6550.120930 secs 2023-09-10T01:08:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:39 smithi170 ceph-mon[102832]: pgmap v3658: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 167 KiB/s wr, 12 op/s 2023-09-10T01:08:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6550.832939 secs 2023-09-10T01:08:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6550.120930 secs 2023-09-10T01:08:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:39 smithi082 ceph-mon[98694]: pgmap v3658: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 167 KiB/s wr, 12 op/s 2023-09-10T01:08:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6550.832939 secs 2023-09-10T01:08:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6550.120930 secs 2023-09-10T01:08:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:39 smithi098 ceph-mon[103124]: pgmap v3658: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 167 KiB/s wr, 12 op/s 2023-09-10T01:08:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6550.832939 secs 2023-09-10T01:08:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:41 smithi170 ceph-mon[102832]: pgmap v3659: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 167 KiB/s wr, 12 op/s 2023-09-10T01:08:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:41 smithi082 ceph-mon[98694]: pgmap v3659: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 167 KiB/s wr, 12 op/s 2023-09-10T01:08:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:41 smithi098 ceph-mon[103124]: pgmap v3659: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 167 KiB/s wr, 12 op/s 2023-09-10T01:08:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:43 smithi170 ceph-mon[102832]: pgmap v3660: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 312 KiB/s wr, 19 op/s 2023-09-10T01:08:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:43 smithi082 ceph-mon[98694]: pgmap v3660: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 312 KiB/s wr, 19 op/s 2023-09-10T01:08:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:43 smithi098 ceph-mon[103124]: pgmap v3660: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 312 KiB/s wr, 19 op/s 2023-09-10T01:08:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6555.120987 secs 2023-09-10T01:08:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6555.833006 secs 2023-09-10T01:08:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6555.120987 secs 2023-09-10T01:08:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6555.833006 secs 2023-09-10T01:08:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6555.120987 secs 2023-09-10T01:08:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6555.833006 secs 2023-09-10T01:08:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:45 smithi082 ceph-mon[98694]: pgmap v3661: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 162 KiB/s wr, 12 op/s 2023-09-10T01:08:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:45 smithi098 ceph-mon[103124]: pgmap v3661: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 162 KiB/s wr, 12 op/s 2023-09-10T01:08:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:45 smithi170 ceph-mon[102832]: pgmap v3661: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 162 KiB/s wr, 12 op/s 2023-09-10T01:08:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:47 smithi082 ceph-mon[98694]: pgmap v3662: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 178 KiB/s wr, 18 op/s 2023-09-10T01:08:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:47 smithi098 ceph-mon[103124]: pgmap v3662: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 178 KiB/s wr, 18 op/s 2023-09-10T01:08:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:47 smithi170 ceph-mon[102832]: pgmap v3662: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 178 KiB/s wr, 18 op/s 2023-09-10T01:08:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6558.589054 secs 2023-09-10T01:08:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6558.589054 secs 2023-09-10T01:08:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6558.589054 secs 2023-09-10T01:08:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6560.121046 secs 2023-09-10T01:08:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:49 smithi082 ceph-mon[98694]: pgmap v3663: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 162 KiB/s wr, 12 op/s 2023-09-10T01:08:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6560.833074 secs 2023-09-10T01:08:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6560.121046 secs 2023-09-10T01:08:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:49 smithi098 ceph-mon[103124]: pgmap v3663: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 162 KiB/s wr, 12 op/s 2023-09-10T01:08:49.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6560.833074 secs 2023-09-10T01:08:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6560.121046 secs 2023-09-10T01:08:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:49 smithi170 ceph-mon[102832]: pgmap v3663: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 162 KiB/s wr, 12 op/s 2023-09-10T01:08:49.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6560.833074 secs 2023-09-10T01:08:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:08:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:08:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:08:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:08:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:08:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:08:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:08:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:08:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:08:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:51 smithi082 ceph-mon[98694]: pgmap v3664: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 162 KiB/s wr, 12 op/s 2023-09-10T01:08:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:08:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:08:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:08:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:51 smithi098 ceph-mon[103124]: pgmap v3664: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 162 KiB/s wr, 12 op/s 2023-09-10T01:08:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:08:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:08:51.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:08:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:51 smithi170 ceph-mon[102832]: pgmap v3664: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 162 KiB/s wr, 12 op/s 2023-09-10T01:08:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:08:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:08:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:08:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:08:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:08:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:08:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:08:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:08:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:08:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:08:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:08:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:08:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6563.589124 secs 2023-09-10T01:08:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:53 smithi098 ceph-mon[103124]: pgmap v3665: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 253 KiB/s wr, 18 op/s 2023-09-10T01:08:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6563.589124 secs 2023-09-10T01:08:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:53 smithi170 ceph-mon[102832]: pgmap v3665: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 253 KiB/s wr, 18 op/s 2023-09-10T01:08:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6563.589124 secs 2023-09-10T01:08:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:53 smithi082 ceph-mon[98694]: pgmap v3665: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 253 KiB/s wr, 18 op/s 2023-09-10T01:08:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6565.121104 secs 2023-09-10T01:08:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6565.833142 secs 2023-09-10T01:08:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6565.121104 secs 2023-09-10T01:08:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6565.833142 secs 2023-09-10T01:08:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6565.121104 secs 2023-09-10T01:08:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6565.833142 secs 2023-09-10T01:08:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:55 smithi098 ceph-mon[103124]: pgmap v3666: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T01:08:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:55 smithi170 ceph-mon[102832]: pgmap v3666: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T01:08:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:55 smithi082 ceph-mon[98694]: pgmap v3666: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T01:08:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:57 smithi098 ceph-mon[103124]: pgmap v3667: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 108 KiB/s wr, 16 op/s 2023-09-10T01:08:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:57 smithi170 ceph-mon[102832]: pgmap v3667: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 108 KiB/s wr, 16 op/s 2023-09-10T01:08:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:57 smithi082 ceph-mon[98694]: pgmap v3667: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 108 KiB/s wr, 16 op/s 2023-09-10T01:08:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6568.589205 secs 2023-09-10T01:08:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6568.589205 secs 2023-09-10T01:08:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6568.589205 secs 2023-09-10T01:08:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6570.121157 secs 2023-09-10T01:08:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:59 smithi098 ceph-mon[103124]: pgmap v3668: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T01:08:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:08:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6570.833208 secs 2023-09-10T01:08:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6570.121157 secs 2023-09-10T01:08:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:59 smithi170 ceph-mon[102832]: pgmap v3668: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T01:08:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:08:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6570.833208 secs 2023-09-10T01:08:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6570.121157 secs 2023-09-10T01:08:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:59 smithi082 ceph-mon[98694]: pgmap v3668: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T01:08:59.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:08:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6570.833208 secs 2023-09-10T01:09:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:01 smithi098 ceph-mon[103124]: pgmap v3669: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T01:09:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:01 smithi170 ceph-mon[102832]: pgmap v3669: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T01:09:01.962 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:01 smithi082 ceph-mon[98694]: pgmap v3669: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T01:09:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6573.589293 secs 2023-09-10T01:09:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:03 smithi098 ceph-mon[103124]: pgmap v3670: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 116 KiB/s wr, 15 op/s 2023-09-10T01:09:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6573.589293 secs 2023-09-10T01:09:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:03 smithi170 ceph-mon[102832]: pgmap v3670: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 116 KiB/s wr, 15 op/s 2023-09-10T01:09:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6573.589293 secs 2023-09-10T01:09:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:03 smithi082 ceph-mon[98694]: pgmap v3670: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 116 KiB/s wr, 15 op/s 2023-09-10T01:09:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6575.121210 secs 2023-09-10T01:09:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6575.833296 secs 2023-09-10T01:09:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6575.121210 secs 2023-09-10T01:09:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6575.833296 secs 2023-09-10T01:09:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6575.121210 secs 2023-09-10T01:09:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6575.833296 secs 2023-09-10T01:09:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:05 smithi098 ceph-mon[103124]: pgmap v3671: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 25 KiB/s wr, 9 op/s 2023-09-10T01:09:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:05 smithi170 ceph-mon[102832]: pgmap v3671: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 25 KiB/s wr, 9 op/s 2023-09-10T01:09:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:05 smithi082 ceph-mon[98694]: pgmap v3671: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 25 KiB/s wr, 9 op/s 2023-09-10T01:09:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:07 smithi098 ceph-mon[103124]: pgmap v3672: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 108 KiB/s wr, 29 op/s 2023-09-10T01:09:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:07 smithi170 ceph-mon[102832]: pgmap v3672: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 108 KiB/s wr, 29 op/s 2023-09-10T01:09:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:07 smithi082 ceph-mon[98694]: pgmap v3672: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 108 KiB/s wr, 29 op/s 2023-09-10T01:09:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6578.589359 secs 2023-09-10T01:09:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:08 smithi170 ceph-mon[102832]: osdmap e299: 12 total, 12 up, 12 in 2023-09-10T01:09:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6578.589359 secs 2023-09-10T01:09:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:08 smithi082 ceph-mon[98694]: osdmap e299: 12 total, 12 up, 12 in 2023-09-10T01:09:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6578.589359 secs 2023-09-10T01:09:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:08 smithi098 ceph-mon[103124]: osdmap e299: 12 total, 12 up, 12 in 2023-09-10T01:09:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6580.121293 secs 2023-09-10T01:09:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:09 smithi170 ceph-mon[102832]: pgmap v3674: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 130 KiB/s wr, 29 op/s 2023-09-10T01:09:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6580.833359 secs 2023-09-10T01:09:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6580.121293 secs 2023-09-10T01:09:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:09 smithi082 ceph-mon[98694]: pgmap v3674: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 130 KiB/s wr, 29 op/s 2023-09-10T01:09:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6580.833359 secs 2023-09-10T01:09:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6580.121293 secs 2023-09-10T01:09:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:09 smithi098 ceph-mon[103124]: pgmap v3674: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 130 KiB/s wr, 29 op/s 2023-09-10T01:09:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6580.833359 secs 2023-09-10T01:09:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:11 smithi170 ceph-mon[102832]: pgmap v3675: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 130 KiB/s wr, 29 op/s 2023-09-10T01:09:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:11 smithi082 ceph-mon[98694]: pgmap v3675: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 130 KiB/s wr, 29 op/s 2023-09-10T01:09:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:11 smithi098 ceph-mon[103124]: pgmap v3675: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 130 KiB/s wr, 29 op/s 2023-09-10T01:09:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6583.589421 secs 2023-09-10T01:09:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:13 smithi170 ceph-mon[102832]: pgmap v3676: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T01:09:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6583.589421 secs 2023-09-10T01:09:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:13 smithi082 ceph-mon[98694]: pgmap v3676: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T01:09:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6583.589421 secs 2023-09-10T01:09:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:13 smithi098 ceph-mon[103124]: pgmap v3676: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T01:09:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6585.121350 secs 2023-09-10T01:09:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6585.833428 secs 2023-09-10T01:09:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6585.121350 secs 2023-09-10T01:09:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6585.833428 secs 2023-09-10T01:09:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6585.121350 secs 2023-09-10T01:09:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6585.833428 secs 2023-09-10T01:09:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:15 smithi170 ceph-mon[102832]: pgmap v3677: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T01:09:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:15 smithi170 ceph-mon[102832]: osdmap e300: 12 total, 12 up, 12 in 2023-09-10T01:09:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:15 smithi082 ceph-mon[98694]: pgmap v3677: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T01:09:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:15 smithi082 ceph-mon[98694]: osdmap e300: 12 total, 12 up, 12 in 2023-09-10T01:09:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:15 smithi098 ceph-mon[103124]: pgmap v3677: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T01:09:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:15 smithi098 ceph-mon[103124]: osdmap e300: 12 total, 12 up, 12 in 2023-09-10T01:09:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:17 smithi170 ceph-mon[102832]: pgmap v3679: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 72 KiB/s wr, 14 op/s 2023-09-10T01:09:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:17 smithi082 ceph-mon[98694]: pgmap v3679: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 72 KiB/s wr, 14 op/s 2023-09-10T01:09:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:17 smithi098 ceph-mon[103124]: pgmap v3679: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 72 KiB/s wr, 14 op/s 2023-09-10T01:09:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6588.589533 secs 2023-09-10T01:09:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6588.589533 secs 2023-09-10T01:09:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6588.589533 secs 2023-09-10T01:09:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6590.121407 secs 2023-09-10T01:09:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:19 smithi170 ceph-mon[102832]: pgmap v3680: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 63 KiB/s wr, 12 op/s 2023-09-10T01:09:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6590.833491 secs 2023-09-10T01:09:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6590.121407 secs 2023-09-10T01:09:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:19 smithi082 ceph-mon[98694]: pgmap v3680: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 63 KiB/s wr, 12 op/s 2023-09-10T01:09:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6590.833491 secs 2023-09-10T01:09:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6590.121407 secs 2023-09-10T01:09:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:19 smithi098 ceph-mon[103124]: pgmap v3680: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 63 KiB/s wr, 12 op/s 2023-09-10T01:09:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6590.833491 secs 2023-09-10T01:09:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:21 smithi170 ceph-mon[102832]: pgmap v3681: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 63 KiB/s wr, 12 op/s 2023-09-10T01:09:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:21 smithi082 ceph-mon[98694]: pgmap v3681: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 63 KiB/s wr, 12 op/s 2023-09-10T01:09:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:21 smithi098 ceph-mon[103124]: pgmap v3681: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 63 KiB/s wr, 12 op/s 2023-09-10T01:09:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:23 smithi170 ceph-mon[102832]: pgmap v3682: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T01:09:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:09:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:23 smithi082 ceph-mon[98694]: pgmap v3682: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T01:09:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:09:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:23 smithi098 ceph-mon[103124]: pgmap v3682: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T01:09:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:09:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6595.121464 secs 2023-09-10T01:09:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:09:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:09:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:09:24.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6595.833553 secs 2023-09-10T01:09:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6595.121464 secs 2023-09-10T01:09:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:09:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:09:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:09:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6595.833553 secs 2023-09-10T01:09:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6595.121464 secs 2023-09-10T01:09:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:09:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:09:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:09:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6595.833553 secs 2023-09-10T01:09:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:25 smithi170 ceph-mon[102832]: pgmap v3683: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T01:09:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:25 smithi082 ceph-mon[98694]: pgmap v3683: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T01:09:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:25 smithi098 ceph-mon[103124]: pgmap v3683: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T01:09:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:27 smithi170 ceph-mon[102832]: pgmap v3684: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 144 KiB/s wr, 17 op/s 2023-09-10T01:09:27.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:27 smithi082 ceph-mon[98694]: pgmap v3684: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 144 KiB/s wr, 17 op/s 2023-09-10T01:09:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:27 smithi098 ceph-mon[103124]: pgmap v3684: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 144 KiB/s wr, 17 op/s 2023-09-10T01:09:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6600.121497 secs 2023-09-10T01:09:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:29 smithi170 ceph-mon[102832]: pgmap v3685: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T01:09:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6600.833620 secs 2023-09-10T01:09:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6600.121497 secs 2023-09-10T01:09:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:29 smithi082 ceph-mon[98694]: pgmap v3685: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T01:09:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6600.833620 secs 2023-09-10T01:09:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6600.121497 secs 2023-09-10T01:09:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:29 smithi098 ceph-mon[103124]: pgmap v3685: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T01:09:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6600.833620 secs 2023-09-10T01:09:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:31 smithi170 ceph-mon[102832]: pgmap v3686: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T01:09:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:31 smithi082 ceph-mon[98694]: pgmap v3686: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T01:09:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:31 smithi098 ceph-mon[103124]: pgmap v3686: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T01:09:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:33 smithi170 ceph-mon[102832]: pgmap v3687: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 236 KiB/s wr, 16 op/s 2023-09-10T01:09:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:33 smithi082 ceph-mon[98694]: pgmap v3687: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 236 KiB/s wr, 16 op/s 2023-09-10T01:09:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:33 smithi098 ceph-mon[103124]: pgmap v3687: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 236 KiB/s wr, 16 op/s 2023-09-10T01:09:34.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6605.121559 secs 2023-09-10T01:09:34.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6605.833684 secs 2023-09-10T01:09:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6605.121559 secs 2023-09-10T01:09:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6605.833684 secs 2023-09-10T01:09:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6605.121559 secs 2023-09-10T01:09:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6605.833684 secs 2023-09-10T01:09:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:35 smithi170 ceph-mon[102832]: pgmap v3688: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T01:09:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:35 smithi082 ceph-mon[98694]: pgmap v3688: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T01:09:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:35 smithi098 ceph-mon[103124]: pgmap v3688: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T01:09:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:37 smithi170 ceph-mon[102832]: pgmap v3689: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 222 KiB/s wr, 17 op/s 2023-09-10T01:09:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:37 smithi082 ceph-mon[98694]: pgmap v3689: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 222 KiB/s wr, 17 op/s 2023-09-10T01:09:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:37 smithi098 ceph-mon[103124]: pgmap v3689: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 222 KiB/s wr, 17 op/s 2023-09-10T01:09:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6610.121617 secs 2023-09-10T01:09:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6610.121617 secs 2023-09-10T01:09:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6610.121617 secs 2023-09-10T01:09:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:39 smithi170 ceph-mon[102832]: pgmap v3690: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 168 KiB/s wr, 12 op/s 2023-09-10T01:09:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6610.833751 secs 2023-09-10T01:09:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:39 smithi082 ceph-mon[98694]: pgmap v3690: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 168 KiB/s wr, 12 op/s 2023-09-10T01:09:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6610.833751 secs 2023-09-10T01:09:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:39 smithi098 ceph-mon[103124]: pgmap v3690: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 168 KiB/s wr, 12 op/s 2023-09-10T01:09:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6610.833751 secs 2023-09-10T01:09:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:41 smithi170 ceph-mon[102832]: pgmap v3691: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 168 KiB/s wr, 12 op/s 2023-09-10T01:09:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:41 smithi082 ceph-mon[98694]: pgmap v3691: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 168 KiB/s wr, 12 op/s 2023-09-10T01:09:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:41 smithi098 ceph-mon[103124]: pgmap v3691: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 168 KiB/s wr, 12 op/s 2023-09-10T01:09:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:42 smithi170 ceph-mon[102832]: pgmap v3692: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 325 KiB/s wr, 18 op/s 2023-09-10T01:09:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:42 smithi082 ceph-mon[98694]: pgmap v3692: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 325 KiB/s wr, 18 op/s 2023-09-10T01:09:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:42 smithi098 ceph-mon[103124]: pgmap v3692: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 325 KiB/s wr, 18 op/s 2023-09-10T01:09:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6615.121701 secs 2023-09-10T01:09:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6615.833814 secs 2023-09-10T01:09:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6615.121701 secs 2023-09-10T01:09:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6615.833814 secs 2023-09-10T01:09:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6615.121701 secs 2023-09-10T01:09:44.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6615.833814 secs 2023-09-10T01:09:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:44 smithi170 ceph-mon[102832]: pgmap v3693: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 188 KiB/s wr, 11 op/s 2023-09-10T01:09:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:44 smithi082 ceph-mon[98694]: pgmap v3693: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 188 KiB/s wr, 11 op/s 2023-09-10T01:09:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:44 smithi098 ceph-mon[103124]: pgmap v3693: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 188 KiB/s wr, 11 op/s 2023-09-10T01:09:47.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:47 smithi170 ceph-mon[102832]: pgmap v3694: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 205 KiB/s wr, 18 op/s 2023-09-10T01:09:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:47 smithi082 ceph-mon[98694]: pgmap v3694: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 205 KiB/s wr, 18 op/s 2023-09-10T01:09:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:47 smithi098 ceph-mon[103124]: pgmap v3694: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 205 KiB/s wr, 18 op/s 2023-09-10T01:09:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6618.590011 secs 2023-09-10T01:09:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6618.590011 secs 2023-09-10T01:09:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6618.590011 secs 2023-09-10T01:09:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6620.121759 secs 2023-09-10T01:09:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:49 smithi170 ceph-mon[102832]: pgmap v3695: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 174 KiB/s wr, 12 op/s 2023-09-10T01:09:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6620.833880 secs 2023-09-10T01:09:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6620.121759 secs 2023-09-10T01:09:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:49 smithi082 ceph-mon[98694]: pgmap v3695: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 174 KiB/s wr, 12 op/s 2023-09-10T01:09:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6620.833880 secs 2023-09-10T01:09:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6620.121759 secs 2023-09-10T01:09:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:49 smithi098 ceph-mon[103124]: pgmap v3695: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 174 KiB/s wr, 12 op/s 2023-09-10T01:09:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6620.833880 secs 2023-09-10T01:09:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:51 smithi170 ceph-mon[102832]: pgmap v3696: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 174 KiB/s wr, 12 op/s 2023-09-10T01:09:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:09:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:51 smithi082 ceph-mon[98694]: pgmap v3696: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 174 KiB/s wr, 12 op/s 2023-09-10T01:09:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:09:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:51 smithi098 ceph-mon[103124]: pgmap v3696: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 174 KiB/s wr, 12 op/s 2023-09-10T01:09:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:09:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:09:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:09:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:09:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:09:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:09:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:09:52.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:09:52.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:09:52.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:09:52.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:09:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:09:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:09:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:09:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:09:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:09:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:09:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:09:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:09:52.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:09:52.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:09:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:09:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:09:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:09:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:09:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:09:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:09:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:09:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:09:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:09:52.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:09:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6623.590091 secs 2023-09-10T01:09:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:53 smithi170 ceph-mon[102832]: pgmap v3697: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 260 KiB/s wr, 17 op/s 2023-09-10T01:09:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6623.590091 secs 2023-09-10T01:09:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:53 smithi082 ceph-mon[98694]: pgmap v3697: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 260 KiB/s wr, 17 op/s 2023-09-10T01:09:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6623.590091 secs 2023-09-10T01:09:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:53 smithi098 ceph-mon[103124]: pgmap v3697: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 260 KiB/s wr, 17 op/s 2023-09-10T01:09:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6625.121834 secs 2023-09-10T01:09:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6625.833947 secs 2023-09-10T01:09:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6625.121834 secs 2023-09-10T01:09:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6625.833947 secs 2023-09-10T01:09:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6625.121834 secs 2023-09-10T01:09:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6625.833947 secs 2023-09-10T01:09:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:55 smithi170 ceph-mon[102832]: pgmap v3698: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T01:09:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:55 smithi082 ceph-mon[98694]: pgmap v3698: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T01:09:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:55 smithi098 ceph-mon[103124]: pgmap v3698: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T01:09:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:57 smithi170 ceph-mon[102832]: pgmap v3699: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 103 KiB/s wr, 17 op/s 2023-09-10T01:09:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:57 smithi082 ceph-mon[98694]: pgmap v3699: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 103 KiB/s wr, 17 op/s 2023-09-10T01:09:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:57 smithi098 ceph-mon[103124]: pgmap v3699: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 103 KiB/s wr, 17 op/s 2023-09-10T01:09:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6628.590186 secs 2023-09-10T01:09:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6628.590186 secs 2023-09-10T01:09:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6628.590186 secs 2023-09-10T01:09:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6630.121890 secs 2023-09-10T01:09:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:59 smithi170 ceph-mon[102832]: pgmap v3700: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T01:09:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:09:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6630.834014 secs 2023-09-10T01:09:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6630.121890 secs 2023-09-10T01:09:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:59 smithi082 ceph-mon[98694]: pgmap v3700: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T01:09:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:09:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6630.834014 secs 2023-09-10T01:09:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6630.121890 secs 2023-09-10T01:09:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:59 smithi098 ceph-mon[103124]: pgmap v3700: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T01:09:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:09:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6630.834014 secs 2023-09-10T01:10:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T01:10:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T01:10:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T01:10:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:01 smithi170 ceph-mon[102832]: pgmap v3701: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T01:10:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:01 smithi082 ceph-mon[98694]: pgmap v3701: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T01:10:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:01 smithi098 ceph-mon[103124]: pgmap v3701: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T01:10:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6633.590249 secs 2023-09-10T01:10:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:03 smithi170 ceph-mon[102832]: pgmap v3702: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 15 op/s 2023-09-10T01:10:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6633.590249 secs 2023-09-10T01:10:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:03 smithi082 ceph-mon[98694]: pgmap v3702: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 15 op/s 2023-09-10T01:10:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6633.590249 secs 2023-09-10T01:10:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:03 smithi098 ceph-mon[103124]: pgmap v3702: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 15 op/s 2023-09-10T01:10:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6635.121972 secs 2023-09-10T01:10:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6635.834080 secs 2023-09-10T01:10:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6635.121972 secs 2023-09-10T01:10:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6635.834080 secs 2023-09-10T01:10:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6635.121972 secs 2023-09-10T01:10:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6635.834080 secs 2023-09-10T01:10:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:05 smithi170 ceph-mon[102832]: pgmap v3703: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 25 KiB/s wr, 10 op/s 2023-09-10T01:10:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:05 smithi082 ceph-mon[98694]: pgmap v3703: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 25 KiB/s wr, 10 op/s 2023-09-10T01:10:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:05 smithi098 ceph-mon[103124]: pgmap v3703: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 25 KiB/s wr, 10 op/s 2023-09-10T01:10:07.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:07 smithi170 ceph-mon[102832]: pgmap v3704: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T01:10:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:07 smithi082 ceph-mon[98694]: pgmap v3704: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T01:10:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:07 smithi098 ceph-mon[103124]: pgmap v3704: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T01:10:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6638.590311 secs 2023-09-10T01:10:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:08 smithi170 ceph-mon[102832]: osdmap e301: 12 total, 12 up, 12 in 2023-09-10T01:10:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6638.590311 secs 2023-09-10T01:10:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:08 smithi082 ceph-mon[98694]: osdmap e301: 12 total, 12 up, 12 in 2023-09-10T01:10:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6638.590311 secs 2023-09-10T01:10:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:08 smithi098 ceph-mon[103124]: osdmap e301: 12 total, 12 up, 12 in 2023-09-10T01:10:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6640.122029 secs 2023-09-10T01:10:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:09 smithi170 ceph-mon[102832]: pgmap v3706: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 130 KiB/s wr, 29 op/s 2023-09-10T01:10:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6640.834146 secs 2023-09-10T01:10:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6640.122029 secs 2023-09-10T01:10:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:09 smithi082 ceph-mon[98694]: pgmap v3706: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 130 KiB/s wr, 29 op/s 2023-09-10T01:10:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6640.834146 secs 2023-09-10T01:10:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6640.122029 secs 2023-09-10T01:10:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:09 smithi098 ceph-mon[103124]: pgmap v3706: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 130 KiB/s wr, 29 op/s 2023-09-10T01:10:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6640.834146 secs 2023-09-10T01:10:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:11 smithi170 ceph-mon[102832]: pgmap v3707: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 130 KiB/s wr, 30 op/s 2023-09-10T01:10:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:11 smithi082 ceph-mon[98694]: pgmap v3707: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 130 KiB/s wr, 30 op/s 2023-09-10T01:10:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:11 smithi098 ceph-mon[103124]: pgmap v3707: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 130 KiB/s wr, 30 op/s 2023-09-10T01:10:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6643.590389 secs 2023-09-10T01:10:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:13 smithi170 ceph-mon[102832]: pgmap v3708: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T01:10:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6643.590389 secs 2023-09-10T01:10:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:13 smithi082 ceph-mon[98694]: pgmap v3708: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T01:10:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6643.590389 secs 2023-09-10T01:10:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:13 smithi098 ceph-mon[103124]: pgmap v3708: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T01:10:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6645.122086 secs 2023-09-10T01:10:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6645.834210 secs 2023-09-10T01:10:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6645.122086 secs 2023-09-10T01:10:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6645.834210 secs 2023-09-10T01:10:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6645.122086 secs 2023-09-10T01:10:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6645.834210 secs 2023-09-10T01:10:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:15 smithi170 ceph-mon[102832]: pgmap v3709: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T01:10:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:15 smithi170 ceph-mon[102832]: osdmap e302: 12 total, 12 up, 12 in 2023-09-10T01:10:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:15 smithi082 ceph-mon[98694]: pgmap v3709: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T01:10:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:15 smithi082 ceph-mon[98694]: osdmap e302: 12 total, 12 up, 12 in 2023-09-10T01:10:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:15 smithi098 ceph-mon[103124]: pgmap v3709: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T01:10:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:15 smithi098 ceph-mon[103124]: osdmap e302: 12 total, 12 up, 12 in 2023-09-10T01:10:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:17 smithi170 ceph-mon[102832]: pgmap v3711: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 71 KiB/s wr, 13 op/s 2023-09-10T01:10:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:17 smithi082 ceph-mon[98694]: pgmap v3711: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 71 KiB/s wr, 13 op/s 2023-09-10T01:10:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:17 smithi098 ceph-mon[103124]: pgmap v3711: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 71 KiB/s wr, 13 op/s 2023-09-10T01:10:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6648.590462 secs 2023-09-10T01:10:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6648.590462 secs 2023-09-10T01:10:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6648.590462 secs 2023-09-10T01:10:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6650.122149 secs 2023-09-10T01:10:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:19 smithi170 ceph-mon[102832]: pgmap v3712: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T01:10:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6650.834275 secs 2023-09-10T01:10:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6650.122149 secs 2023-09-10T01:10:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:19 smithi082 ceph-mon[98694]: pgmap v3712: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T01:10:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6650.834275 secs 2023-09-10T01:10:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6650.122149 secs 2023-09-10T01:10:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:19 smithi098 ceph-mon[103124]: pgmap v3712: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T01:10:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6650.834275 secs 2023-09-10T01:10:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:21 smithi170 ceph-mon[102832]: pgmap v3713: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 12 op/s 2023-09-10T01:10:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:21 smithi082 ceph-mon[98694]: pgmap v3713: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 12 op/s 2023-09-10T01:10:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:21 smithi098 ceph-mon[103124]: pgmap v3713: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 12 op/s 2023-09-10T01:10:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:23 smithi170 ceph-mon[102832]: pgmap v3714: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T01:10:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:10:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:23 smithi082 ceph-mon[98694]: pgmap v3714: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T01:10:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:10:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:23 smithi098 ceph-mon[103124]: pgmap v3714: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T01:10:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:10:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6655.122237 secs 2023-09-10T01:10:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:10:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:10:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:10:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6655.834341 secs 2023-09-10T01:10:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6655.122237 secs 2023-09-10T01:10:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:10:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:10:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:10:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6655.834341 secs 2023-09-10T01:10:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6655.122237 secs 2023-09-10T01:10:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:10:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:10:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:10:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6655.834341 secs 2023-09-10T01:10:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:25 smithi170 ceph-mon[102832]: pgmap v3715: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T01:10:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:25 smithi082 ceph-mon[98694]: pgmap v3715: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T01:10:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:25 smithi098 ceph-mon[103124]: pgmap v3715: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T01:10:27.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:27 smithi170 ceph-mon[102832]: pgmap v3716: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 140 KiB/s wr, 14 op/s 2023-09-10T01:10:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:27 smithi082 ceph-mon[98694]: pgmap v3716: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 140 KiB/s wr, 14 op/s 2023-09-10T01:10:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:27 smithi098 ceph-mon[103124]: pgmap v3716: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 140 KiB/s wr, 14 op/s 2023-09-10T01:10:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6660.122295 secs 2023-09-10T01:10:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:29 smithi170 ceph-mon[102832]: pgmap v3717: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 95 KiB/s wr, 10 op/s 2023-09-10T01:10:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6660.834410 secs 2023-09-10T01:10:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6660.122295 secs 2023-09-10T01:10:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:29 smithi082 ceph-mon[98694]: pgmap v3717: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 95 KiB/s wr, 10 op/s 2023-09-10T01:10:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6660.834410 secs 2023-09-10T01:10:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6660.122295 secs 2023-09-10T01:10:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:29 smithi098 ceph-mon[103124]: pgmap v3717: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 95 KiB/s wr, 10 op/s 2023-09-10T01:10:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6660.834410 secs 2023-09-10T01:10:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:31 smithi170 ceph-mon[102832]: pgmap v3718: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 213 KiB/s wr, 13 op/s 2023-09-10T01:10:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:31 smithi082 ceph-mon[98694]: pgmap v3718: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 213 KiB/s wr, 13 op/s 2023-09-10T01:10:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:31 smithi098 ceph-mon[103124]: pgmap v3718: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 213 KiB/s wr, 13 op/s 2023-09-10T01:10:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:33 smithi170 ceph-mon[102832]: pgmap v3719: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 213 KiB/s wr, 17 op/s 2023-09-10T01:10:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:33 smithi082 ceph-mon[98694]: pgmap v3719: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 213 KiB/s wr, 17 op/s 2023-09-10T01:10:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:33 smithi098 ceph-mon[103124]: pgmap v3719: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 213 KiB/s wr, 17 op/s 2023-09-10T01:10:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6665.122379 secs 2023-09-10T01:10:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6665.834486 secs 2023-09-10T01:10:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6665.122379 secs 2023-09-10T01:10:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6665.834486 secs 2023-09-10T01:10:34.708 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6665.122379 secs 2023-09-10T01:10:34.709 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6665.834486 secs 2023-09-10T01:10:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:35 smithi170 ceph-mon[102832]: pgmap v3720: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 179 KiB/s wr, 12 op/s 2023-09-10T01:10:35.741 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:35 smithi082 ceph-mon[98694]: pgmap v3720: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 179 KiB/s wr, 12 op/s 2023-09-10T01:10:35.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:35 smithi098 ceph-mon[103124]: pgmap v3720: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 179 KiB/s wr, 12 op/s 2023-09-10T01:10:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:37 smithi170 ceph-mon[102832]: pgmap v3721: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 224 KiB/s wr, 18 op/s 2023-09-10T01:10:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:37 smithi082 ceph-mon[98694]: pgmap v3721: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 224 KiB/s wr, 18 op/s 2023-09-10T01:10:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:37 smithi098 ceph-mon[103124]: pgmap v3721: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 224 KiB/s wr, 18 op/s 2023-09-10T01:10:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6670.122438 secs 2023-09-10T01:10:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:39 smithi170 ceph-mon[102832]: pgmap v3722: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 169 KiB/s wr, 12 op/s 2023-09-10T01:10:39.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6670.834552 secs 2023-09-10T01:10:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6670.122438 secs 2023-09-10T01:10:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:39 smithi082 ceph-mon[98694]: pgmap v3722: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 169 KiB/s wr, 12 op/s 2023-09-10T01:10:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6670.834552 secs 2023-09-10T01:10:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6670.122438 secs 2023-09-10T01:10:39.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:39 smithi098 ceph-mon[103124]: pgmap v3722: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 169 KiB/s wr, 12 op/s 2023-09-10T01:10:39.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6670.834552 secs 2023-09-10T01:10:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:41 smithi170 ceph-mon[102832]: pgmap v3723: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 298 KiB/s wr, 16 op/s 2023-09-10T01:10:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:41 smithi082 ceph-mon[98694]: pgmap v3723: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 298 KiB/s wr, 16 op/s 2023-09-10T01:10:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:41 smithi098 ceph-mon[103124]: pgmap v3723: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 298 KiB/s wr, 16 op/s 2023-09-10T01:10:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:43 smithi170 ceph-mon[102832]: pgmap v3724: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 185 KiB/s wr, 15 op/s 2023-09-10T01:10:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:43 smithi082 ceph-mon[98694]: pgmap v3724: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 185 KiB/s wr, 15 op/s 2023-09-10T01:10:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:43 smithi098 ceph-mon[103124]: pgmap v3724: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 185 KiB/s wr, 15 op/s 2023-09-10T01:10:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6675.122492 secs 2023-09-10T01:10:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6675.834618 secs 2023-09-10T01:10:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6675.122492 secs 2023-09-10T01:10:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6675.834618 secs 2023-09-10T01:10:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6675.122492 secs 2023-09-10T01:10:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6675.834618 secs 2023-09-10T01:10:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:45 smithi170 ceph-mon[102832]: pgmap v3725: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 179 KiB/s wr, 11 op/s 2023-09-10T01:10:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:45 smithi082 ceph-mon[98694]: pgmap v3725: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 179 KiB/s wr, 11 op/s 2023-09-10T01:10:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:45 smithi098 ceph-mon[103124]: pgmap v3725: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 179 KiB/s wr, 11 op/s 2023-09-10T01:10:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:47 smithi170 ceph-mon[102832]: pgmap v3726: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 210 KiB/s wr, 17 op/s 2023-09-10T01:10:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:47 smithi082 ceph-mon[98694]: pgmap v3726: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 210 KiB/s wr, 17 op/s 2023-09-10T01:10:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:47 smithi098 ceph-mon[103124]: pgmap v3726: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 210 KiB/s wr, 17 op/s 2023-09-10T01:10:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6678.590902 secs 2023-09-10T01:10:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6678.590902 secs 2023-09-10T01:10:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6678.590902 secs 2023-09-10T01:10:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6680.122575 secs 2023-09-10T01:10:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:49 smithi170 ceph-mon[102832]: pgmap v3727: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 11 op/s 2023-09-10T01:10:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6680.834690 secs 2023-09-10T01:10:49.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6680.122575 secs 2023-09-10T01:10:49.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:49 smithi082 ceph-mon[98694]: pgmap v3727: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 11 op/s 2023-09-10T01:10:49.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6680.834690 secs 2023-09-10T01:10:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6680.122575 secs 2023-09-10T01:10:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:49 smithi098 ceph-mon[103124]: pgmap v3727: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 11 op/s 2023-09-10T01:10:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6680.834690 secs 2023-09-10T01:10:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:51 smithi170 ceph-mon[102832]: pgmap v3728: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 199 KiB/s wr, 14 op/s 2023-09-10T01:10:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:51 smithi082 ceph-mon[98694]: pgmap v3728: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 199 KiB/s wr, 14 op/s 2023-09-10T01:10:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:51 smithi098 ceph-mon[103124]: pgmap v3728: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 199 KiB/s wr, 14 op/s 2023-09-10T01:10:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:10:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:10:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:10:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:10:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:10:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:10:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:10:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:10:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:10:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:10:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:10:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:10:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6683.590966 secs 2023-09-10T01:10:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:53 smithi170 ceph-mon[102832]: pgmap v3729: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 76 KiB/s wr, 13 op/s 2023-09-10T01:10:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:10:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:10:53.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:10:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6683.590966 secs 2023-09-10T01:10:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:53 smithi082 ceph-mon[98694]: pgmap v3729: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 76 KiB/s wr, 13 op/s 2023-09-10T01:10:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:10:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:10:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:10:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6683.590966 secs 2023-09-10T01:10:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:53 smithi098 ceph-mon[103124]: pgmap v3729: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 76 KiB/s wr, 13 op/s 2023-09-10T01:10:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:10:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:10:53.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:10:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6685.122633 secs 2023-09-10T01:10:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6685.834758 secs 2023-09-10T01:10:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6685.122633 secs 2023-09-10T01:10:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6685.834758 secs 2023-09-10T01:10:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6685.122633 secs 2023-09-10T01:10:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6685.834758 secs 2023-09-10T01:10:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:55 smithi170 ceph-mon[102832]: pgmap v3730: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T01:10:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:55 smithi082 ceph-mon[98694]: pgmap v3730: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T01:10:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:55 smithi098 ceph-mon[103124]: pgmap v3730: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T01:10:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:57 smithi170 ceph-mon[102832]: pgmap v3731: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 129 KiB/s wr, 16 op/s 2023-09-10T01:10:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:57 smithi082 ceph-mon[98694]: pgmap v3731: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 129 KiB/s wr, 16 op/s 2023-09-10T01:10:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:57 smithi098 ceph-mon[103124]: pgmap v3731: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 129 KiB/s wr, 16 op/s 2023-09-10T01:10:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6688.591074 secs 2023-09-10T01:10:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6688.591074 secs 2023-09-10T01:10:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6688.591074 secs 2023-09-10T01:10:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6690.122692 secs 2023-09-10T01:10:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:59 smithi082 ceph-mon[98694]: pgmap v3732: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 98 KiB/s wr, 10 op/s 2023-09-10T01:10:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:10:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6690.834825 secs 2023-09-10T01:10:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6690.122692 secs 2023-09-10T01:10:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:59 smithi098 ceph-mon[103124]: pgmap v3732: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 98 KiB/s wr, 10 op/s 2023-09-10T01:10:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:10:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6690.834825 secs 2023-09-10T01:10:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6690.122692 secs 2023-09-10T01:10:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:59 smithi170 ceph-mon[102832]: pgmap v3732: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 98 KiB/s wr, 10 op/s 2023-09-10T01:10:59.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:10:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6690.834825 secs 2023-09-10T01:11:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:01 smithi082 ceph-mon[98694]: pgmap v3733: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 122 KiB/s wr, 13 op/s 2023-09-10T01:11:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:01 smithi098 ceph-mon[103124]: pgmap v3733: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 122 KiB/s wr, 13 op/s 2023-09-10T01:11:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:01 smithi170 ceph-mon[102832]: pgmap v3733: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 122 KiB/s wr, 13 op/s 2023-09-10T01:11:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6693.591149 secs 2023-09-10T01:11:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:03 smithi082 ceph-mon[98694]: pgmap v3734: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T01:11:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6693.591149 secs 2023-09-10T01:11:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:03 smithi098 ceph-mon[103124]: pgmap v3734: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T01:11:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6693.591149 secs 2023-09-10T01:11:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:03 smithi170 ceph-mon[102832]: pgmap v3734: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T01:11:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6695.122751 secs 2023-09-10T01:11:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6695.834892 secs 2023-09-10T01:11:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6695.122751 secs 2023-09-10T01:11:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6695.834892 secs 2023-09-10T01:11:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6695.122751 secs 2023-09-10T01:11:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6695.834892 secs 2023-09-10T01:11:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:05 smithi082 ceph-mon[98694]: pgmap v3735: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-09-10T01:11:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:05 smithi098 ceph-mon[103124]: pgmap v3735: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-09-10T01:11:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:05 smithi170 ceph-mon[102832]: pgmap v3735: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-09-10T01:11:07.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:07 smithi082 ceph-mon[98694]: pgmap v3736: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 129 KiB/s wr, 29 op/s 2023-09-10T01:11:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:07 smithi098 ceph-mon[103124]: pgmap v3736: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 129 KiB/s wr, 29 op/s 2023-09-10T01:11:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:07 smithi170 ceph-mon[102832]: pgmap v3736: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 129 KiB/s wr, 29 op/s 2023-09-10T01:11:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6698.591225 secs 2023-09-10T01:11:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:08 smithi082 ceph-mon[98694]: osdmap e303: 12 total, 12 up, 12 in 2023-09-10T01:11:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6698.591225 secs 2023-09-10T01:11:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:08 smithi098 ceph-mon[103124]: osdmap e303: 12 total, 12 up, 12 in 2023-09-10T01:11:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6698.591225 secs 2023-09-10T01:11:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:08 smithi170 ceph-mon[102832]: osdmap e303: 12 total, 12 up, 12 in 2023-09-10T01:11:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6700.122813 secs 2023-09-10T01:11:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:09 smithi082 ceph-mon[98694]: pgmap v3738: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T01:11:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6700.834957 secs 2023-09-10T01:11:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6700.122813 secs 2023-09-10T01:11:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:09 smithi098 ceph-mon[103124]: pgmap v3738: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T01:11:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6700.834957 secs 2023-09-10T01:11:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6700.122813 secs 2023-09-10T01:11:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:09 smithi170 ceph-mon[102832]: pgmap v3738: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T01:11:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6700.834957 secs 2023-09-10T01:11:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:11 smithi098 ceph-mon[103124]: pgmap v3739: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 124 KiB/s wr, 29 op/s 2023-09-10T01:11:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:11 smithi170 ceph-mon[102832]: pgmap v3739: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 124 KiB/s wr, 29 op/s 2023-09-10T01:11:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:11 smithi082 ceph-mon[98694]: pgmap v3739: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 124 KiB/s wr, 29 op/s 2023-09-10T01:11:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6703.591283 secs 2023-09-10T01:11:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:13 smithi098 ceph-mon[103124]: pgmap v3740: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 124 KiB/s wr, 29 op/s 2023-09-10T01:11:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6703.591283 secs 2023-09-10T01:11:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:13 smithi170 ceph-mon[102832]: pgmap v3740: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 124 KiB/s wr, 29 op/s 2023-09-10T01:11:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6703.591283 secs 2023-09-10T01:11:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:13 smithi082 ceph-mon[98694]: pgmap v3740: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 124 KiB/s wr, 29 op/s 2023-09-10T01:11:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6705.122875 secs 2023-09-10T01:11:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6705.835025 secs 2023-09-10T01:11:14.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6705.122875 secs 2023-09-10T01:11:14.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6705.835025 secs 2023-09-10T01:11:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6705.122875 secs 2023-09-10T01:11:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6705.835025 secs 2023-09-10T01:11:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:15 smithi098 ceph-mon[103124]: pgmap v3741: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 124 KiB/s wr, 29 op/s 2023-09-10T01:11:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:15 smithi098 ceph-mon[103124]: osdmap e304: 12 total, 12 up, 12 in 2023-09-10T01:11:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:15 smithi170 ceph-mon[102832]: pgmap v3741: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 124 KiB/s wr, 29 op/s 2023-09-10T01:11:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:15 smithi170 ceph-mon[102832]: osdmap e304: 12 total, 12 up, 12 in 2023-09-10T01:11:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:15 smithi082 ceph-mon[98694]: pgmap v3741: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 124 KiB/s wr, 29 op/s 2023-09-10T01:11:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:15 smithi082 ceph-mon[98694]: osdmap e304: 12 total, 12 up, 12 in 2023-09-10T01:11:17.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:17 smithi098 ceph-mon[103124]: pgmap v3743: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 125 KiB/s wr, 13 op/s 2023-09-10T01:11:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:17 smithi170 ceph-mon[102832]: pgmap v3743: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 125 KiB/s wr, 13 op/s 2023-09-10T01:11:17.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:17 smithi082 ceph-mon[98694]: pgmap v3743: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 125 KiB/s wr, 13 op/s 2023-09-10T01:11:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6708.591344 secs 2023-09-10T01:11:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6708.591344 secs 2023-09-10T01:11:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6708.591344 secs 2023-09-10T01:11:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6710.122935 secs 2023-09-10T01:11:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:19 smithi098 ceph-mon[103124]: pgmap v3744: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T01:11:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6710.835114 secs 2023-09-10T01:11:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6710.122935 secs 2023-09-10T01:11:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:19 smithi170 ceph-mon[102832]: pgmap v3744: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T01:11:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6710.835114 secs 2023-09-10T01:11:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6710.122935 secs 2023-09-10T01:11:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:19 smithi082 ceph-mon[98694]: pgmap v3744: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T01:11:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6710.835114 secs 2023-09-10T01:11:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:21 smithi098 ceph-mon[103124]: pgmap v3745: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:11:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:21 smithi170 ceph-mon[102832]: pgmap v3745: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:11:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:21 smithi082 ceph-mon[98694]: pgmap v3745: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:11:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:23 smithi098 ceph-mon[103124]: pgmap v3746: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:11:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:11:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:23 smithi170 ceph-mon[102832]: pgmap v3746: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:11:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:11:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:23 smithi082 ceph-mon[98694]: pgmap v3746: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:11:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:11:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6715.123022 secs 2023-09-10T01:11:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:11:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:11:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:11:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6715.835179 secs 2023-09-10T01:11:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6715.123022 secs 2023-09-10T01:11:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:11:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:11:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:11:24.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6715.835179 secs 2023-09-10T01:11:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6715.123022 secs 2023-09-10T01:11:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:11:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:11:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:11:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6715.835179 secs 2023-09-10T01:11:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:25 smithi098 ceph-mon[103124]: pgmap v3747: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:11:25.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:25 smithi170 ceph-mon[102832]: pgmap v3747: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:11:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:25 smithi082 ceph-mon[98694]: pgmap v3747: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:11:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:27 smithi098 ceph-mon[103124]: pgmap v3748: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 135 KiB/s wr, 14 op/s 2023-09-10T01:11:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:27 smithi170 ceph-mon[102832]: pgmap v3748: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 135 KiB/s wr, 14 op/s 2023-09-10T01:11:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:27 smithi082 ceph-mon[98694]: pgmap v3748: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 135 KiB/s wr, 14 op/s 2023-09-10T01:11:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6720.123080 secs 2023-09-10T01:11:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:29 smithi098 ceph-mon[103124]: pgmap v3749: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T01:11:29.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6720.835244 secs 2023-09-10T01:11:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6720.123080 secs 2023-09-10T01:11:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:29 smithi170 ceph-mon[102832]: pgmap v3749: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T01:11:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6720.835244 secs 2023-09-10T01:11:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6720.123080 secs 2023-09-10T01:11:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:29 smithi082 ceph-mon[98694]: pgmap v3749: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T01:11:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6720.835244 secs 2023-09-10T01:11:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:31 smithi170 ceph-mon[102832]: pgmap v3750: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 215 KiB/s wr, 14 op/s 2023-09-10T01:11:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:31 smithi082 ceph-mon[98694]: pgmap v3750: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 215 KiB/s wr, 14 op/s 2023-09-10T01:11:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:31 smithi098 ceph-mon[103124]: pgmap v3750: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 215 KiB/s wr, 14 op/s 2023-09-10T01:11:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:33 smithi170 ceph-mon[102832]: pgmap v3751: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 180 KiB/s wr, 14 op/s 2023-09-10T01:11:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:33 smithi082 ceph-mon[98694]: pgmap v3751: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 180 KiB/s wr, 14 op/s 2023-09-10T01:11:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:33 smithi098 ceph-mon[103124]: pgmap v3751: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 180 KiB/s wr, 14 op/s 2023-09-10T01:11:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6725.123138 secs 2023-09-10T01:11:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6725.835310 secs 2023-09-10T01:11:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6725.123138 secs 2023-09-10T01:11:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6725.835310 secs 2023-09-10T01:11:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6725.123138 secs 2023-09-10T01:11:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6725.835310 secs 2023-09-10T01:11:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:35 smithi170 ceph-mon[102832]: pgmap v3752: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 180 KiB/s wr, 12 op/s 2023-09-10T01:11:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:35 smithi082 ceph-mon[98694]: pgmap v3752: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 180 KiB/s wr, 12 op/s 2023-09-10T01:11:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:35 smithi098 ceph-mon[103124]: pgmap v3752: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 180 KiB/s wr, 12 op/s 2023-09-10T01:11:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:37 smithi170 ceph-mon[102832]: pgmap v3753: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 226 KiB/s wr, 18 op/s 2023-09-10T01:11:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:37 smithi082 ceph-mon[98694]: pgmap v3753: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 226 KiB/s wr, 18 op/s 2023-09-10T01:11:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:37 smithi098 ceph-mon[103124]: pgmap v3753: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 226 KiB/s wr, 18 op/s 2023-09-10T01:11:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6730.123222 secs 2023-09-10T01:11:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:39 smithi170 ceph-mon[102832]: pgmap v3754: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 143 KiB/s wr, 12 op/s 2023-09-10T01:11:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6730.835378 secs 2023-09-10T01:11:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6730.123222 secs 2023-09-10T01:11:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:39 smithi082 ceph-mon[98694]: pgmap v3754: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 143 KiB/s wr, 12 op/s 2023-09-10T01:11:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6730.835378 secs 2023-09-10T01:11:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6730.123222 secs 2023-09-10T01:11:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:39 smithi098 ceph-mon[103124]: pgmap v3754: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 143 KiB/s wr, 12 op/s 2023-09-10T01:11:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6730.835378 secs 2023-09-10T01:11:41.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:41 smithi170 ceph-mon[102832]: pgmap v3755: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 331 KiB/s wr, 16 op/s 2023-09-10T01:11:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:41 smithi082 ceph-mon[98694]: pgmap v3755: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 331 KiB/s wr, 16 op/s 2023-09-10T01:11:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:41 smithi098 ceph-mon[103124]: pgmap v3755: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 331 KiB/s wr, 16 op/s 2023-09-10T01:11:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:43 smithi170 ceph-mon[102832]: pgmap v3756: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 245 KiB/s wr, 14 op/s 2023-09-10T01:11:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:43 smithi082 ceph-mon[98694]: pgmap v3756: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 245 KiB/s wr, 14 op/s 2023-09-10T01:11:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:43 smithi098 ceph-mon[103124]: pgmap v3756: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 245 KiB/s wr, 14 op/s 2023-09-10T01:11:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6735.123283 secs 2023-09-10T01:11:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6735.835444 secs 2023-09-10T01:11:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6735.123283 secs 2023-09-10T01:11:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6735.835444 secs 2023-09-10T01:11:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6735.123283 secs 2023-09-10T01:11:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6735.835444 secs 2023-09-10T01:11:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:45 smithi170 ceph-mon[102832]: pgmap v3757: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 239 KiB/s wr, 11 op/s 2023-09-10T01:11:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:45 smithi082 ceph-mon[98694]: pgmap v3757: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 239 KiB/s wr, 11 op/s 2023-09-10T01:11:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:45 smithi098 ceph-mon[103124]: pgmap v3757: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 239 KiB/s wr, 11 op/s 2023-09-10T01:11:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:47 smithi170 ceph-mon[102832]: pgmap v3758: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 289 KiB/s wr, 17 op/s 2023-09-10T01:11:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:47 smithi082 ceph-mon[98694]: pgmap v3758: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 289 KiB/s wr, 17 op/s 2023-09-10T01:11:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:47 smithi098 ceph-mon[103124]: pgmap v3758: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 289 KiB/s wr, 17 op/s 2023-09-10T01:11:48.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6738.639693 secs 2023-09-10T01:11:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6738.639693 secs 2023-09-10T01:11:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6738.639693 secs 2023-09-10T01:11:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6740.123340 secs 2023-09-10T01:11:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:49 smithi170 ceph-mon[102832]: pgmap v3759: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 244 KiB/s wr, 11 op/s 2023-09-10T01:11:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6740.835485 secs 2023-09-10T01:11:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6740.123340 secs 2023-09-10T01:11:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:49 smithi082 ceph-mon[98694]: pgmap v3759: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 244 KiB/s wr, 11 op/s 2023-09-10T01:11:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6740.835485 secs 2023-09-10T01:11:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6740.123340 secs 2023-09-10T01:11:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:49 smithi098 ceph-mon[103124]: pgmap v3759: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 244 KiB/s wr, 11 op/s 2023-09-10T01:11:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6740.835485 secs 2023-09-10T01:11:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:51 smithi170 ceph-mon[102832]: pgmap v3760: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 273 KiB/s wr, 14 op/s 2023-09-10T01:11:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:51 smithi082 ceph-mon[98694]: pgmap v3760: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 273 KiB/s wr, 14 op/s 2023-09-10T01:11:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:51 smithi098 ceph-mon[103124]: pgmap v3760: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 273 KiB/s wr, 14 op/s 2023-09-10T01:11:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:11:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:11:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:11:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:11:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:11:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:11:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:11:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:11:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:11:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6743.639750 secs 2023-09-10T01:11:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:53 smithi170 ceph-mon[102832]: pgmap v3761: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 90 KiB/s wr, 13 op/s 2023-09-10T01:11:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:11:53.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:11:53.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:11:53.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:11:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6743.639750 secs 2023-09-10T01:11:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:53 smithi082 ceph-mon[98694]: pgmap v3761: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 90 KiB/s wr, 13 op/s 2023-09-10T01:11:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:11:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:11:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:11:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:11:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6743.639750 secs 2023-09-10T01:11:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:53 smithi098 ceph-mon[103124]: pgmap v3761: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 90 KiB/s wr, 13 op/s 2023-09-10T01:11:54.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:11:54.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:11:54.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:11:54.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:11:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6745.123396 secs 2023-09-10T01:11:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6745.835551 secs 2023-09-10T01:11:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6745.123396 secs 2023-09-10T01:11:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6745.835551 secs 2023-09-10T01:11:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6745.123396 secs 2023-09-10T01:11:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6745.835551 secs 2023-09-10T01:11:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:55 smithi170 ceph-mon[102832]: pgmap v3762: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T01:11:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:55 smithi082 ceph-mon[98694]: pgmap v3762: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T01:11:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:55 smithi098 ceph-mon[103124]: pgmap v3762: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T01:11:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:57 smithi170 ceph-mon[102832]: pgmap v3763: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 144 KiB/s wr, 17 op/s 2023-09-10T01:11:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:57 smithi082 ceph-mon[98694]: pgmap v3763: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 144 KiB/s wr, 17 op/s 2023-09-10T01:11:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:57 smithi098 ceph-mon[103124]: pgmap v3763: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 144 KiB/s wr, 17 op/s 2023-09-10T01:11:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6748.639790 secs 2023-09-10T01:11:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6748.639790 secs 2023-09-10T01:11:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6748.639790 secs 2023-09-10T01:11:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6750.123454 secs 2023-09-10T01:11:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:59 smithi170 ceph-mon[102832]: pgmap v3764: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T01:11:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:11:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6750.835619 secs 2023-09-10T01:11:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6750.123454 secs 2023-09-10T01:11:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:59 smithi082 ceph-mon[98694]: pgmap v3764: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T01:11:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:11:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6750.835619 secs 2023-09-10T01:12:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6750.123454 secs 2023-09-10T01:12:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:59 smithi098 ceph-mon[103124]: pgmap v3764: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T01:12:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:11:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6750.835619 secs 2023-09-10T01:12:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:01 smithi170 ceph-mon[102832]: pgmap v3765: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 118 KiB/s wr, 14 op/s 2023-09-10T01:12:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:01 smithi082 ceph-mon[98694]: pgmap v3765: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 118 KiB/s wr, 14 op/s 2023-09-10T01:12:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:01 smithi098 ceph-mon[103124]: pgmap v3765: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 118 KiB/s wr, 14 op/s 2023-09-10T01:12:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6753.639849 secs 2023-09-10T01:12:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:03 smithi170 ceph-mon[102832]: pgmap v3766: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 13 op/s 2023-09-10T01:12:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6753.639849 secs 2023-09-10T01:12:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:03 smithi082 ceph-mon[98694]: pgmap v3766: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 13 op/s 2023-09-10T01:12:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6753.639849 secs 2023-09-10T01:12:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:03 smithi098 ceph-mon[103124]: pgmap v3766: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 13 op/s 2023-09-10T01:12:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6755.123511 secs 2023-09-10T01:12:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6755.835687 secs 2023-09-10T01:12:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6755.123511 secs 2023-09-10T01:12:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6755.835687 secs 2023-09-10T01:12:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6755.123511 secs 2023-09-10T01:12:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6755.835687 secs 2023-09-10T01:12:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:05 smithi170 ceph-mon[102832]: pgmap v3767: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T01:12:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:05 smithi082 ceph-mon[98694]: pgmap v3767: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T01:12:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:05 smithi098 ceph-mon[103124]: pgmap v3767: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T01:12:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:07 smithi170 ceph-mon[102832]: pgmap v3768: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 129 KiB/s wr, 30 op/s 2023-09-10T01:12:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:07 smithi082 ceph-mon[98694]: pgmap v3768: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 129 KiB/s wr, 30 op/s 2023-09-10T01:12:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:07 smithi098 ceph-mon[103124]: pgmap v3768: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 129 KiB/s wr, 30 op/s 2023-09-10T01:12:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6758.639908 secs 2023-09-10T01:12:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:08 smithi170 ceph-mon[102832]: osdmap e305: 12 total, 12 up, 12 in 2023-09-10T01:12:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6760.123568 secs 2023-09-10T01:12:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6758.639908 secs 2023-09-10T01:12:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:08 smithi082 ceph-mon[98694]: osdmap e305: 12 total, 12 up, 12 in 2023-09-10T01:12:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6760.123568 secs 2023-09-10T01:12:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6758.639908 secs 2023-09-10T01:12:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:08 smithi098 ceph-mon[103124]: osdmap e305: 12 total, 12 up, 12 in 2023-09-10T01:12:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6760.123568 secs 2023-09-10T01:12:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:09 smithi170 ceph-mon[102832]: pgmap v3770: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T01:12:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6760.835752 secs 2023-09-10T01:12:09.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:09 smithi082 ceph-mon[98694]: pgmap v3770: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T01:12:09.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6760.835752 secs 2023-09-10T01:12:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:09 smithi098 ceph-mon[103124]: pgmap v3770: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T01:12:10.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6760.835752 secs 2023-09-10T01:12:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:11 smithi170 ceph-mon[102832]: pgmap v3771: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 102 KiB/s wr, 30 op/s 2023-09-10T01:12:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:11 smithi082 ceph-mon[98694]: pgmap v3771: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 102 KiB/s wr, 30 op/s 2023-09-10T01:12:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:11 smithi098 ceph-mon[103124]: pgmap v3771: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 102 KiB/s wr, 30 op/s 2023-09-10T01:12:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6763.639998 secs 2023-09-10T01:12:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:13 smithi170 ceph-mon[102832]: pgmap v3772: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T01:12:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6765.123643 secs 2023-09-10T01:12:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6763.639998 secs 2023-09-10T01:12:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:13 smithi082 ceph-mon[98694]: pgmap v3772: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T01:12:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6765.123643 secs 2023-09-10T01:12:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6763.639998 secs 2023-09-10T01:12:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:13 smithi098 ceph-mon[103124]: pgmap v3772: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T01:12:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6765.123643 secs 2023-09-10T01:12:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6765.835818 secs 2023-09-10T01:12:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6765.835818 secs 2023-09-10T01:12:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6765.835818 secs 2023-09-10T01:12:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:15 smithi170 ceph-mon[102832]: pgmap v3773: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T01:12:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:15 smithi170 ceph-mon[102832]: osdmap e306: 12 total, 12 up, 12 in 2023-09-10T01:12:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:15 smithi082 ceph-mon[98694]: pgmap v3773: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T01:12:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:15 smithi082 ceph-mon[98694]: osdmap e306: 12 total, 12 up, 12 in 2023-09-10T01:12:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:15 smithi098 ceph-mon[103124]: pgmap v3773: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T01:12:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:15 smithi098 ceph-mon[103124]: osdmap e306: 12 total, 12 up, 12 in 2023-09-10T01:12:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:17 smithi170 ceph-mon[102832]: pgmap v3775: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 101 KiB/s wr, 13 op/s 2023-09-10T01:12:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:17 smithi082 ceph-mon[98694]: pgmap v3775: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 101 KiB/s wr, 13 op/s 2023-09-10T01:12:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:17 smithi098 ceph-mon[103124]: pgmap v3775: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 101 KiB/s wr, 13 op/s 2023-09-10T01:12:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6768.640058 secs 2023-09-10T01:12:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6770.123703 secs 2023-09-10T01:12:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6770.835939 secs 2023-09-10T01:12:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6768.640058 secs 2023-09-10T01:12:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6770.123703 secs 2023-09-10T01:12:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6770.835939 secs 2023-09-10T01:12:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6768.640058 secs 2023-09-10T01:12:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6770.123703 secs 2023-09-10T01:12:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6770.835939 secs 2023-09-10T01:12:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:19 smithi170 ceph-mon[102832]: pgmap v3776: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T01:12:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:19 smithi082 ceph-mon[98694]: pgmap v3776: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T01:12:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:19 smithi098 ceph-mon[103124]: pgmap v3776: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T01:12:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:21 smithi170 ceph-mon[102832]: pgmap v3777: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T01:12:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:21 smithi082 ceph-mon[98694]: pgmap v3777: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T01:12:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:21 smithi098 ceph-mon[103124]: pgmap v3777: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T01:12:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:23 smithi170 ceph-mon[102832]: pgmap v3778: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:12:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:12:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:23 smithi082 ceph-mon[98694]: pgmap v3778: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:12:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:12:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:23 smithi098 ceph-mon[103124]: pgmap v3778: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:12:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:12:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6775.123786 secs 2023-09-10T01:12:24.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:12:24.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:12:24.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:12:24.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6775.836005 secs 2023-09-10T01:12:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6775.123786 secs 2023-09-10T01:12:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:12:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:12:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:12:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6775.836005 secs 2023-09-10T01:12:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6775.123786 secs 2023-09-10T01:12:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:12:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:12:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:12:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6775.836005 secs 2023-09-10T01:12:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:25 smithi170 ceph-mon[102832]: pgmap v3779: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:12:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:25 smithi082 ceph-mon[98694]: pgmap v3779: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:12:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:25 smithi098 ceph-mon[103124]: pgmap v3779: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:12:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:27 smithi170 ceph-mon[102832]: pgmap v3780: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T01:12:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:27 smithi082 ceph-mon[98694]: pgmap v3780: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T01:12:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:27 smithi098 ceph-mon[103124]: pgmap v3780: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T01:12:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6780.123846 secs 2023-09-10T01:12:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:29 smithi170 ceph-mon[102832]: pgmap v3781: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T01:12:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6780.836070 secs 2023-09-10T01:12:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6780.123846 secs 2023-09-10T01:12:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:29 smithi082 ceph-mon[98694]: pgmap v3781: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T01:12:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6780.836070 secs 2023-09-10T01:12:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6780.123846 secs 2023-09-10T01:12:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:29 smithi098 ceph-mon[103124]: pgmap v3781: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T01:12:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6780.836070 secs 2023-09-10T01:12:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:31 smithi170 ceph-mon[102832]: pgmap v3782: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 222 KiB/s wr, 16 op/s 2023-09-10T01:12:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:31 smithi082 ceph-mon[98694]: pgmap v3782: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 222 KiB/s wr, 16 op/s 2023-09-10T01:12:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:31 smithi098 ceph-mon[103124]: pgmap v3782: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 222 KiB/s wr, 16 op/s 2023-09-10T01:12:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:33 smithi170 ceph-mon[102832]: pgmap v3783: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 182 KiB/s wr, 13 op/s 2023-09-10T01:12:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:33 smithi082 ceph-mon[98694]: pgmap v3783: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 182 KiB/s wr, 13 op/s 2023-09-10T01:12:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:33 smithi098 ceph-mon[103124]: pgmap v3783: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 182 KiB/s wr, 13 op/s 2023-09-10T01:12:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6785.123903 secs 2023-09-10T01:12:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6785.836136 secs 2023-09-10T01:12:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6785.123903 secs 2023-09-10T01:12:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6785.836136 secs 2023-09-10T01:12:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6785.123903 secs 2023-09-10T01:12:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6785.836136 secs 2023-09-10T01:12:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:35 smithi170 ceph-mon[102832]: pgmap v3784: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 182 KiB/s wr, 12 op/s 2023-09-10T01:12:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:35 smithi082 ceph-mon[98694]: pgmap v3784: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 182 KiB/s wr, 12 op/s 2023-09-10T01:12:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:35 smithi098 ceph-mon[103124]: pgmap v3784: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 182 KiB/s wr, 12 op/s 2023-09-10T01:12:37.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:37 smithi170 ceph-mon[102832]: pgmap v3785: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 228 KiB/s wr, 18 op/s 2023-09-10T01:12:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:37 smithi082 ceph-mon[98694]: pgmap v3785: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 228 KiB/s wr, 18 op/s 2023-09-10T01:12:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:37 smithi098 ceph-mon[103124]: pgmap v3785: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 228 KiB/s wr, 18 op/s 2023-09-10T01:12:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6790.123961 secs 2023-09-10T01:12:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:39 smithi170 ceph-mon[102832]: pgmap v3786: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 172 KiB/s wr, 12 op/s 2023-09-10T01:12:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6790.836209 secs 2023-09-10T01:12:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6790.123961 secs 2023-09-10T01:12:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:39 smithi082 ceph-mon[98694]: pgmap v3786: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 172 KiB/s wr, 12 op/s 2023-09-10T01:12:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6790.836209 secs 2023-09-10T01:12:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6790.123961 secs 2023-09-10T01:12:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:39 smithi098 ceph-mon[103124]: pgmap v3786: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 172 KiB/s wr, 12 op/s 2023-09-10T01:12:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6790.836209 secs 2023-09-10T01:12:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:41 smithi170 ceph-mon[102832]: pgmap v3787: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 308 KiB/s wr, 16 op/s 2023-09-10T01:12:41.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:41 smithi082 ceph-mon[98694]: pgmap v3787: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 308 KiB/s wr, 16 op/s 2023-09-10T01:12:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:41 smithi098 ceph-mon[103124]: pgmap v3787: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 308 KiB/s wr, 16 op/s 2023-09-10T01:12:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:43 smithi170 ceph-mon[102832]: pgmap v3788: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 182 KiB/s wr, 12 op/s 2023-09-10T01:12:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:43 smithi082 ceph-mon[98694]: pgmap v3788: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 182 KiB/s wr, 12 op/s 2023-09-10T01:12:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:43 smithi098 ceph-mon[103124]: pgmap v3788: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 182 KiB/s wr, 12 op/s 2023-09-10T01:12:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6795.124017 secs 2023-09-10T01:12:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6795.836272 secs 2023-09-10T01:12:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6795.124017 secs 2023-09-10T01:12:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6795.836272 secs 2023-09-10T01:12:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6795.124017 secs 2023-09-10T01:12:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6795.836272 secs 2023-09-10T01:12:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:45 smithi170 ceph-mon[102832]: pgmap v3789: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 182 KiB/s wr, 10 op/s 2023-09-10T01:12:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:45 smithi082 ceph-mon[98694]: pgmap v3789: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 182 KiB/s wr, 10 op/s 2023-09-10T01:12:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:45 smithi098 ceph-mon[103124]: pgmap v3789: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 182 KiB/s wr, 10 op/s 2023-09-10T01:12:47.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:47 smithi170 ceph-mon[102832]: pgmap v3790: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 213 KiB/s wr, 16 op/s 2023-09-10T01:12:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:47 smithi082 ceph-mon[98694]: pgmap v3790: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 213 KiB/s wr, 16 op/s 2023-09-10T01:12:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:47 smithi098 ceph-mon[103124]: pgmap v3790: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 213 KiB/s wr, 16 op/s 2023-09-10T01:12:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6798.640550 secs 2023-09-10T01:12:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6798.640550 secs 2023-09-10T01:12:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6798.640550 secs 2023-09-10T01:12:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6800.124075 secs 2023-09-10T01:12:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:49 smithi170 ceph-mon[102832]: pgmap v3791: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 10 op/s 2023-09-10T01:12:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6800.836339 secs 2023-09-10T01:12:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6800.124075 secs 2023-09-10T01:12:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:49 smithi082 ceph-mon[98694]: pgmap v3791: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 10 op/s 2023-09-10T01:12:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6800.836339 secs 2023-09-10T01:12:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6800.124075 secs 2023-09-10T01:12:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:49 smithi098 ceph-mon[103124]: pgmap v3791: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 10 op/s 2023-09-10T01:12:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6800.836339 secs 2023-09-10T01:12:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:51 smithi170 ceph-mon[102832]: pgmap v3792: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 207 KiB/s wr, 14 op/s 2023-09-10T01:12:51.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:51 smithi082 ceph-mon[98694]: pgmap v3792: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 207 KiB/s wr, 14 op/s 2023-09-10T01:12:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:51 smithi098 ceph-mon[103124]: pgmap v3792: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 207 KiB/s wr, 14 op/s 2023-09-10T01:12:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:12:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:12:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:12:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:12:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:12:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:12:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:12:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:12:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:12:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6803.640612 secs 2023-09-10T01:12:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:53 smithi170 ceph-mon[102832]: pgmap v3793: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T01:12:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6803.640612 secs 2023-09-10T01:12:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:53 smithi082 ceph-mon[98694]: pgmap v3793: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T01:12:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6803.640612 secs 2023-09-10T01:12:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:53 smithi098 ceph-mon[103124]: pgmap v3793: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T01:12:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6805.124134 secs 2023-09-10T01:12:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6805.836403 secs 2023-09-10T01:12:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:12:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:12:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:12:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:12:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6805.124134 secs 2023-09-10T01:12:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6805.836403 secs 2023-09-10T01:12:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:12:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:12:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:12:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:12:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6805.124134 secs 2023-09-10T01:12:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6805.836403 secs 2023-09-10T01:12:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:12:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:12:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:12:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:12:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:55 smithi170 ceph-mon[102832]: pgmap v3794: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 72 KiB/s wr, 10 op/s 2023-09-10T01:12:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:55 smithi082 ceph-mon[98694]: pgmap v3794: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 72 KiB/s wr, 10 op/s 2023-09-10T01:12:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:55 smithi098 ceph-mon[103124]: pgmap v3794: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 72 KiB/s wr, 10 op/s 2023-09-10T01:12:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:57 smithi170 ceph-mon[102832]: pgmap v3795: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 131 KiB/s wr, 16 op/s 2023-09-10T01:12:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:57 smithi082 ceph-mon[98694]: pgmap v3795: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 131 KiB/s wr, 16 op/s 2023-09-10T01:12:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:57 smithi098 ceph-mon[103124]: pgmap v3795: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 131 KiB/s wr, 16 op/s 2023-09-10T01:12:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6808.640680 secs 2023-09-10T01:12:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6808.640680 secs 2023-09-10T01:12:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6808.640680 secs 2023-09-10T01:12:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6810.124193 secs 2023-09-10T01:12:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:59 smithi170 ceph-mon[102832]: pgmap v3796: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 100 KiB/s wr, 10 op/s 2023-09-10T01:12:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:12:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6810.836474 secs 2023-09-10T01:12:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6810.124193 secs 2023-09-10T01:12:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:59 smithi082 ceph-mon[98694]: pgmap v3796: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 100 KiB/s wr, 10 op/s 2023-09-10T01:12:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:12:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6810.836474 secs 2023-09-10T01:12:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6810.124193 secs 2023-09-10T01:12:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:59 smithi098 ceph-mon[103124]: pgmap v3796: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 100 KiB/s wr, 10 op/s 2023-09-10T01:12:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:12:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6810.836474 secs 2023-09-10T01:13:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:01 smithi170 ceph-mon[102832]: pgmap v3797: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 148 KiB/s wr, 18 op/s 2023-09-10T01:13:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:01 smithi082 ceph-mon[98694]: pgmap v3797: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 148 KiB/s wr, 18 op/s 2023-09-10T01:13:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:01 smithi098 ceph-mon[103124]: pgmap v3797: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 148 KiB/s wr, 18 op/s 2023-09-10T01:13:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6813.640789 secs 2023-09-10T01:13:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:03 smithi170 ceph-mon[102832]: pgmap v3798: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 130 KiB/s wr, 27 op/s 2023-09-10T01:13:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6813.640789 secs 2023-09-10T01:13:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:03 smithi082 ceph-mon[98694]: pgmap v3798: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 130 KiB/s wr, 27 op/s 2023-09-10T01:13:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6813.640789 secs 2023-09-10T01:13:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:03 smithi098 ceph-mon[103124]: pgmap v3798: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 130 KiB/s wr, 27 op/s 2023-09-10T01:13:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6815.124252 secs 2023-09-10T01:13:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6815.836538 secs 2023-09-10T01:13:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6815.124252 secs 2023-09-10T01:13:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6815.836538 secs 2023-09-10T01:13:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6815.124252 secs 2023-09-10T01:13:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6815.836538 secs 2023-09-10T01:13:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:05 smithi170 ceph-mon[102832]: pgmap v3799: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 129 KiB/s wr, 25 op/s 2023-09-10T01:13:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:05 smithi082 ceph-mon[98694]: pgmap v3799: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 129 KiB/s wr, 25 op/s 2023-09-10T01:13:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:05 smithi098 ceph-mon[103124]: pgmap v3799: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 129 KiB/s wr, 25 op/s 2023-09-10T01:13:07.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:07 smithi170 ceph-mon[102832]: pgmap v3800: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 130 KiB/s wr, 31 op/s 2023-09-10T01:13:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:07 smithi082 ceph-mon[98694]: pgmap v3800: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 130 KiB/s wr, 31 op/s 2023-09-10T01:13:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:07 smithi098 ceph-mon[103124]: pgmap v3800: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 130 KiB/s wr, 31 op/s 2023-09-10T01:13:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6818.640859 secs 2023-09-10T01:13:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:08 smithi082 ceph-mon[98694]: osdmap e307: 12 total, 12 up, 12 in 2023-09-10T01:13:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6818.640859 secs 2023-09-10T01:13:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:08 smithi098 ceph-mon[103124]: osdmap e307: 12 total, 12 up, 12 in 2023-09-10T01:13:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6818.640859 secs 2023-09-10T01:13:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:08 smithi170 ceph-mon[102832]: osdmap e307: 12 total, 12 up, 12 in 2023-09-10T01:13:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6820.124338 secs 2023-09-10T01:13:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:09 smithi082 ceph-mon[98694]: pgmap v3802: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-09-10T01:13:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6820.836604 secs 2023-09-10T01:13:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6820.124338 secs 2023-09-10T01:13:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:09 smithi098 ceph-mon[103124]: pgmap v3802: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-09-10T01:13:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6820.836604 secs 2023-09-10T01:13:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6820.124338 secs 2023-09-10T01:13:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:09 smithi170 ceph-mon[102832]: pgmap v3802: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-09-10T01:13:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6820.836604 secs 2023-09-10T01:13:11.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:11 smithi082 ceph-mon[98694]: pgmap v3803: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 97 KiB/s wr, 26 op/s 2023-09-10T01:13:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:11 smithi098 ceph-mon[103124]: pgmap v3803: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 97 KiB/s wr, 26 op/s 2023-09-10T01:13:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:11 smithi170 ceph-mon[102832]: pgmap v3803: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 97 KiB/s wr, 26 op/s 2023-09-10T01:13:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6823.640921 secs 2023-09-10T01:13:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:13 smithi082 ceph-mon[98694]: pgmap v3804: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T01:13:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6823.640921 secs 2023-09-10T01:13:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:13 smithi098 ceph-mon[103124]: pgmap v3804: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T01:13:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6823.640921 secs 2023-09-10T01:13:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:13 smithi170 ceph-mon[102832]: pgmap v3804: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T01:13:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6825.124410 secs 2023-09-10T01:13:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6825.836668 secs 2023-09-10T01:13:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6825.124410 secs 2023-09-10T01:13:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6825.836668 secs 2023-09-10T01:13:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6825.124410 secs 2023-09-10T01:13:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6825.836668 secs 2023-09-10T01:13:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:15 smithi082 ceph-mon[98694]: pgmap v3805: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T01:13:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:15 smithi082 ceph-mon[98694]: osdmap e308: 12 total, 12 up, 12 in 2023-09-10T01:13:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:15 smithi098 ceph-mon[103124]: pgmap v3805: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T01:13:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:15 smithi098 ceph-mon[103124]: osdmap e308: 12 total, 12 up, 12 in 2023-09-10T01:13:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:15 smithi170 ceph-mon[102832]: pgmap v3805: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T01:13:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:15 smithi170 ceph-mon[102832]: osdmap e308: 12 total, 12 up, 12 in 2023-09-10T01:13:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:17 smithi082 ceph-mon[98694]: pgmap v3807: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 126 KiB/s wr, 13 op/s 2023-09-10T01:13:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:17 smithi098 ceph-mon[103124]: pgmap v3807: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 126 KiB/s wr, 13 op/s 2023-09-10T01:13:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:17 smithi170 ceph-mon[102832]: pgmap v3807: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 126 KiB/s wr, 13 op/s 2023-09-10T01:13:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6828.641026 secs 2023-09-10T01:13:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6828.641026 secs 2023-09-10T01:13:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6828.641026 secs 2023-09-10T01:13:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6830.124467 secs 2023-09-10T01:13:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:19 smithi082 ceph-mon[98694]: pgmap v3808: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T01:13:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6830.836731 secs 2023-09-10T01:13:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6830.124467 secs 2023-09-10T01:13:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:19 smithi098 ceph-mon[103124]: pgmap v3808: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T01:13:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6830.836731 secs 2023-09-10T01:13:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6830.124467 secs 2023-09-10T01:13:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:19 smithi170 ceph-mon[102832]: pgmap v3808: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T01:13:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6830.836731 secs 2023-09-10T01:13:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:21 smithi098 ceph-mon[103124]: pgmap v3809: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T01:13:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:21 smithi170 ceph-mon[102832]: pgmap v3809: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T01:13:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:21 smithi082 ceph-mon[98694]: pgmap v3809: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T01:13:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6833.641091 secs 2023-09-10T01:13:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:23 smithi098 ceph-mon[103124]: pgmap v3810: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T01:13:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6833.641091 secs 2023-09-10T01:13:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:23 smithi170 ceph-mon[102832]: pgmap v3810: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T01:13:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6833.641091 secs 2023-09-10T01:13:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:23 smithi082 ceph-mon[98694]: pgmap v3810: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T01:13:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6835.124496 secs 2023-09-10T01:13:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6835.836798 secs 2023-09-10T01:13:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6835.124496 secs 2023-09-10T01:13:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6835.836798 secs 2023-09-10T01:13:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6835.124496 secs 2023-09-10T01:13:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6835.836798 secs 2023-09-10T01:13:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:25 smithi098 ceph-mon[103124]: pgmap v3811: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T01:13:25.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:25 smithi170 ceph-mon[102832]: pgmap v3811: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T01:13:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:25 smithi082 ceph-mon[98694]: pgmap v3811: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T01:13:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:27 smithi098 ceph-mon[103124]: pgmap v3812: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 106 KiB/s wr, 13 op/s 2023-09-10T01:13:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:27 smithi170 ceph-mon[102832]: pgmap v3812: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 106 KiB/s wr, 13 op/s 2023-09-10T01:13:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:27 smithi082 ceph-mon[98694]: pgmap v3812: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 106 KiB/s wr, 13 op/s 2023-09-10T01:13:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6838.641151 secs 2023-09-10T01:13:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6838.641151 secs 2023-09-10T01:13:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6838.641151 secs 2023-09-10T01:13:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6840.124555 secs 2023-09-10T01:13:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:29 smithi098 ceph-mon[103124]: pgmap v3813: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 97 KiB/s wr, 10 op/s 2023-09-10T01:13:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6840.836866 secs 2023-09-10T01:13:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6840.124555 secs 2023-09-10T01:13:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:29 smithi170 ceph-mon[102832]: pgmap v3813: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 97 KiB/s wr, 10 op/s 2023-09-10T01:13:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6840.836866 secs 2023-09-10T01:13:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6840.124555 secs 2023-09-10T01:13:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:29 smithi082 ceph-mon[98694]: pgmap v3813: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 97 KiB/s wr, 10 op/s 2023-09-10T01:13:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6840.836866 secs 2023-09-10T01:13:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:31 smithi098 ceph-mon[103124]: pgmap v3814: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 223 KiB/s wr, 16 op/s 2023-09-10T01:13:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:31 smithi170 ceph-mon[102832]: pgmap v3814: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 223 KiB/s wr, 16 op/s 2023-09-10T01:13:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:31 smithi082 ceph-mon[98694]: pgmap v3814: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 223 KiB/s wr, 16 op/s 2023-09-10T01:13:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:33 smithi098 ceph-mon[103124]: pgmap v3815: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 183 KiB/s wr, 13 op/s 2023-09-10T01:13:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:33 smithi170 ceph-mon[102832]: pgmap v3815: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 183 KiB/s wr, 13 op/s 2023-09-10T01:13:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:33 smithi082 ceph-mon[98694]: pgmap v3815: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 183 KiB/s wr, 13 op/s 2023-09-10T01:13:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6845.124616 secs 2023-09-10T01:13:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6845.836929 secs 2023-09-10T01:13:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6845.124616 secs 2023-09-10T01:13:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6845.836929 secs 2023-09-10T01:13:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6845.124616 secs 2023-09-10T01:13:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6845.836929 secs 2023-09-10T01:13:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:35 smithi098 ceph-mon[103124]: pgmap v3816: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 183 KiB/s wr, 12 op/s 2023-09-10T01:13:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:13:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:35 smithi170 ceph-mon[102832]: pgmap v3816: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 183 KiB/s wr, 12 op/s 2023-09-10T01:13:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:13:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:35 smithi082 ceph-mon[98694]: pgmap v3816: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 183 KiB/s wr, 12 op/s 2023-09-10T01:13:35.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:13:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:13:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:13:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:13:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:13:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:13:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:13:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:13:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:13:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:13:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:37 smithi098 ceph-mon[103124]: pgmap v3817: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 229 KiB/s wr, 19 op/s 2023-09-10T01:13:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:37 smithi170 ceph-mon[102832]: pgmap v3817: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 229 KiB/s wr, 19 op/s 2023-09-10T01:13:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:37 smithi082 ceph-mon[98694]: pgmap v3817: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 229 KiB/s wr, 19 op/s 2023-09-10T01:13:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6850.124675 secs 2023-09-10T01:13:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:39 smithi170 ceph-mon[102832]: pgmap v3818: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 173 KiB/s wr, 12 op/s 2023-09-10T01:13:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6850.836997 secs 2023-09-10T01:13:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6850.124675 secs 2023-09-10T01:13:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:39 smithi082 ceph-mon[98694]: pgmap v3818: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 173 KiB/s wr, 12 op/s 2023-09-10T01:13:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6850.836997 secs 2023-09-10T01:13:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6850.124675 secs 2023-09-10T01:13:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:39 smithi098 ceph-mon[103124]: pgmap v3818: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 173 KiB/s wr, 12 op/s 2023-09-10T01:13:40.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6850.836997 secs 2023-09-10T01:13:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:41 smithi170 ceph-mon[102832]: pgmap v3819: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 309 KiB/s wr, 17 op/s 2023-09-10T01:13:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:41 smithi082 ceph-mon[98694]: pgmap v3819: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 309 KiB/s wr, 17 op/s 2023-09-10T01:13:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:41 smithi098 ceph-mon[103124]: pgmap v3819: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 309 KiB/s wr, 17 op/s 2023-09-10T01:13:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:43 smithi170 ceph-mon[102832]: pgmap v3820: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 183 KiB/s wr, 13 op/s 2023-09-10T01:13:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:43 smithi082 ceph-mon[98694]: pgmap v3820: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 183 KiB/s wr, 13 op/s 2023-09-10T01:13:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:43 smithi098 ceph-mon[103124]: pgmap v3820: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 183 KiB/s wr, 13 op/s 2023-09-10T01:13:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6855.124733 secs 2023-09-10T01:13:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6855.837062 secs 2023-09-10T01:13:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6855.124733 secs 2023-09-10T01:13:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6855.837062 secs 2023-09-10T01:13:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6855.124733 secs 2023-09-10T01:13:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6855.837062 secs 2023-09-10T01:13:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:45 smithi170 ceph-mon[102832]: pgmap v3821: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 182 KiB/s wr, 11 op/s 2023-09-10T01:13:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:45 smithi082 ceph-mon[98694]: pgmap v3821: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 182 KiB/s wr, 11 op/s 2023-09-10T01:13:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:45 smithi098 ceph-mon[103124]: pgmap v3821: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 182 KiB/s wr, 11 op/s 2023-09-10T01:13:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:47 smithi170 ceph-mon[102832]: pgmap v3822: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 224 KiB/s wr, 17 op/s 2023-09-10T01:13:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:47 smithi082 ceph-mon[98694]: pgmap v3822: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 224 KiB/s wr, 17 op/s 2023-09-10T01:13:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:47 smithi098 ceph-mon[103124]: pgmap v3822: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 224 KiB/s wr, 17 op/s 2023-09-10T01:13:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6858.641486 secs 2023-09-10T01:13:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6858.641486 secs 2023-09-10T01:13:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6858.641486 secs 2023-09-10T01:13:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6860.124791 secs 2023-09-10T01:13:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:49 smithi170 ceph-mon[102832]: pgmap v3823: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T01:13:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6860.837132 secs 2023-09-10T01:13:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6860.124791 secs 2023-09-10T01:13:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:49 smithi082 ceph-mon[98694]: pgmap v3823: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T01:13:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6860.837132 secs 2023-09-10T01:13:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6860.124791 secs 2023-09-10T01:13:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:49 smithi098 ceph-mon[103124]: pgmap v3823: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T01:13:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6860.837132 secs 2023-09-10T01:13:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:51 smithi170 ceph-mon[102832]: pgmap v3824: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 219 KiB/s wr, 14 op/s 2023-09-10T01:13:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:51 smithi082 ceph-mon[98694]: pgmap v3824: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 219 KiB/s wr, 14 op/s 2023-09-10T01:13:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:51 smithi098 ceph-mon[103124]: pgmap v3824: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 219 KiB/s wr, 14 op/s 2023-09-10T01:13:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:13:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:13:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:13:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:13:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:13:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:13:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:13:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:13:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:13:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6863.641548 secs 2023-09-10T01:13:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:53 smithi170 ceph-mon[102832]: pgmap v3825: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T01:13:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6863.641548 secs 2023-09-10T01:13:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:53 smithi082 ceph-mon[98694]: pgmap v3825: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T01:13:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6863.641548 secs 2023-09-10T01:13:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:53 smithi098 ceph-mon[103124]: pgmap v3825: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T01:13:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6865.124848 secs 2023-09-10T01:13:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6865.837203 secs 2023-09-10T01:13:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:13:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6865.124848 secs 2023-09-10T01:13:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6865.837203 secs 2023-09-10T01:13:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:13:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6865.124848 secs 2023-09-10T01:13:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6865.837203 secs 2023-09-10T01:13:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:13:55.808 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:55 smithi082 ceph-mon[98694]: pgmap v3826: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 82 KiB/s wr, 10 op/s 2023-09-10T01:13:55.857 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:55 smithi098 ceph-mon[103124]: pgmap v3826: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 82 KiB/s wr, 10 op/s 2023-09-10T01:13:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:55 smithi170 ceph-mon[102832]: pgmap v3826: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 82 KiB/s wr, 10 op/s 2023-09-10T01:13:57.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:57 smithi082 ceph-mon[98694]: pgmap v3827: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 141 KiB/s wr, 15 op/s 2023-09-10T01:13:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:57 smithi170 ceph-mon[102832]: pgmap v3827: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 141 KiB/s wr, 15 op/s 2023-09-10T01:13:58.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:57 smithi098 ceph-mon[103124]: pgmap v3827: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 141 KiB/s wr, 15 op/s 2023-09-10T01:13:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6868.641609 secs 2023-09-10T01:13:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:13:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:13:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:13:58.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:13:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6868.641609 secs 2023-09-10T01:13:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:13:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:13:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:13:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:13:58.959 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6868.641609 secs 2023-09-10T01:13:58.959 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:13:58.959 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:13:58.959 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:13:58.960 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:13:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6870.124910 secs 2023-09-10T01:13:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:59 smithi170 ceph-mon[102832]: pgmap v3828: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T01:13:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:13:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6870.837268 secs 2023-09-10T01:13:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6870.124910 secs 2023-09-10T01:13:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:59 smithi082 ceph-mon[98694]: pgmap v3828: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T01:13:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:13:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6870.837268 secs 2023-09-10T01:14:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6870.124910 secs 2023-09-10T01:14:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:59 smithi098 ceph-mon[103124]: pgmap v3828: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T01:14:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:13:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6870.837268 secs 2023-09-10T01:14:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:01 smithi082 ceph-mon[98694]: pgmap v3829: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 159 KiB/s wr, 28 op/s 2023-09-10T01:14:01.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:14:01.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:14:01.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:14:01.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:14:01.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:14:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:01 smithi098 ceph-mon[103124]: pgmap v3829: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 159 KiB/s wr, 28 op/s 2023-09-10T01:14:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:14:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:14:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:14:01.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:14:01.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:14:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:01 smithi170 ceph-mon[102832]: pgmap v3829: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 159 KiB/s wr, 28 op/s 2023-09-10T01:14:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:14:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:14:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:14:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:14:01.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:14:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6873.641686 secs 2023-09-10T01:14:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:03 smithi082 ceph-mon[98694]: pgmap v3830: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 119 KiB/s wr, 24 op/s 2023-09-10T01:14:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6873.641686 secs 2023-09-10T01:14:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:03 smithi098 ceph-mon[103124]: pgmap v3830: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 119 KiB/s wr, 24 op/s 2023-09-10T01:14:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6873.641686 secs 2023-09-10T01:14:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:03 smithi170 ceph-mon[102832]: pgmap v3830: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 119 KiB/s wr, 24 op/s 2023-09-10T01:14:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6875.124991 secs 2023-09-10T01:14:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6875.837335 secs 2023-09-10T01:14:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6875.124991 secs 2023-09-10T01:14:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6875.837335 secs 2023-09-10T01:14:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6875.124991 secs 2023-09-10T01:14:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6875.837335 secs 2023-09-10T01:14:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:05 smithi082 ceph-mon[98694]: pgmap v3831: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 119 KiB/s wr, 23 op/s 2023-09-10T01:14:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:05 smithi098 ceph-mon[103124]: pgmap v3831: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 119 KiB/s wr, 23 op/s 2023-09-10T01:14:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:05 smithi170 ceph-mon[102832]: pgmap v3831: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 119 KiB/s wr, 23 op/s 2023-09-10T01:14:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:07 smithi098 ceph-mon[103124]: pgmap v3832: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 130 KiB/s wr, 29 op/s 2023-09-10T01:14:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:07 smithi170 ceph-mon[102832]: pgmap v3832: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 130 KiB/s wr, 29 op/s 2023-09-10T01:14:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:07 smithi082 ceph-mon[98694]: pgmap v3832: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 130 KiB/s wr, 29 op/s 2023-09-10T01:14:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6878.641755 secs 2023-09-10T01:14:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:08 smithi098 ceph-mon[103124]: osdmap e309: 12 total, 12 up, 12 in 2023-09-10T01:14:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6878.641755 secs 2023-09-10T01:14:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:08 smithi170 ceph-mon[102832]: osdmap e309: 12 total, 12 up, 12 in 2023-09-10T01:14:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6878.641755 secs 2023-09-10T01:14:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:08 smithi082 ceph-mon[98694]: osdmap e309: 12 total, 12 up, 12 in 2023-09-10T01:14:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6880.125076 secs 2023-09-10T01:14:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:09 smithi098 ceph-mon[103124]: pgmap v3834: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T01:14:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6880.837423 secs 2023-09-10T01:14:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6880.125076 secs 2023-09-10T01:14:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:09 smithi170 ceph-mon[102832]: pgmap v3834: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T01:14:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6880.837423 secs 2023-09-10T01:14:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6880.125076 secs 2023-09-10T01:14:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:09 smithi082 ceph-mon[98694]: pgmap v3834: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T01:14:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6880.837423 secs 2023-09-10T01:14:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:11 smithi098 ceph-mon[103124]: pgmap v3835: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 84 KiB/s wr, 14 op/s 2023-09-10T01:14:11.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:11 smithi170 ceph-mon[102832]: pgmap v3835: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 84 KiB/s wr, 14 op/s 2023-09-10T01:14:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:11 smithi082 ceph-mon[98694]: pgmap v3835: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 84 KiB/s wr, 14 op/s 2023-09-10T01:14:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6883.641857 secs 2023-09-10T01:14:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:13 smithi098 ceph-mon[103124]: pgmap v3836: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 84 KiB/s wr, 14 op/s 2023-09-10T01:14:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6883.641857 secs 2023-09-10T01:14:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:13 smithi170 ceph-mon[102832]: pgmap v3836: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 84 KiB/s wr, 14 op/s 2023-09-10T01:14:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6883.641857 secs 2023-09-10T01:14:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:13 smithi082 ceph-mon[98694]: pgmap v3836: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 84 KiB/s wr, 14 op/s 2023-09-10T01:14:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6885.125136 secs 2023-09-10T01:14:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6885.837494 secs 2023-09-10T01:14:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6885.125136 secs 2023-09-10T01:14:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6885.837494 secs 2023-09-10T01:14:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6885.125136 secs 2023-09-10T01:14:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6885.837494 secs 2023-09-10T01:14:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:15 smithi098 ceph-mon[103124]: pgmap v3837: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 84 KiB/s wr, 14 op/s 2023-09-10T01:14:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:15 smithi098 ceph-mon[103124]: osdmap e310: 12 total, 12 up, 12 in 2023-09-10T01:14:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:15 smithi170 ceph-mon[102832]: pgmap v3837: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 84 KiB/s wr, 14 op/s 2023-09-10T01:14:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:15 smithi170 ceph-mon[102832]: osdmap e310: 12 total, 12 up, 12 in 2023-09-10T01:14:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:15 smithi082 ceph-mon[98694]: pgmap v3837: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 84 KiB/s wr, 14 op/s 2023-09-10T01:14:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:15 smithi082 ceph-mon[98694]: osdmap e310: 12 total, 12 up, 12 in 2023-09-10T01:14:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:17 smithi098 ceph-mon[103124]: pgmap v3839: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 14 op/s 2023-09-10T01:14:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:17 smithi170 ceph-mon[102832]: pgmap v3839: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 14 op/s 2023-09-10T01:14:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:17 smithi082 ceph-mon[98694]: pgmap v3839: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 14 op/s 2023-09-10T01:14:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6888.641967 secs 2023-09-10T01:14:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6888.641967 secs 2023-09-10T01:14:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6888.641967 secs 2023-09-10T01:14:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6890.125190 secs 2023-09-10T01:14:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:19 smithi098 ceph-mon[103124]: pgmap v3840: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T01:14:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6890.837557 secs 2023-09-10T01:14:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6890.125190 secs 2023-09-10T01:14:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:19 smithi170 ceph-mon[102832]: pgmap v3840: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T01:14:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6890.837557 secs 2023-09-10T01:14:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6890.125190 secs 2023-09-10T01:14:20.279 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:19 smithi082 ceph-mon[98694]: pgmap v3840: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T01:14:20.279 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6890.837557 secs 2023-09-10T01:14:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:21 smithi098 ceph-mon[103124]: pgmap v3841: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T01:14:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:21 smithi170 ceph-mon[102832]: pgmap v3841: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T01:14:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:21 smithi082 ceph-mon[98694]: pgmap v3841: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T01:14:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6893.642031 secs 2023-09-10T01:14:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:23 smithi098 ceph-mon[103124]: pgmap v3842: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T01:14:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6893.642031 secs 2023-09-10T01:14:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:23 smithi170 ceph-mon[102832]: pgmap v3842: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T01:14:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6893.642031 secs 2023-09-10T01:14:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:23 smithi082 ceph-mon[98694]: pgmap v3842: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T01:14:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6895.125283 secs 2023-09-10T01:14:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6895.837621 secs 2023-09-10T01:14:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6895.125283 secs 2023-09-10T01:14:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6895.837621 secs 2023-09-10T01:14:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6895.125283 secs 2023-09-10T01:14:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6895.837621 secs 2023-09-10T01:14:25.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:25 smithi170 ceph-mon[102832]: pgmap v3843: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T01:14:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:25 smithi082 ceph-mon[98694]: pgmap v3843: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T01:14:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:25 smithi098 ceph-mon[103124]: pgmap v3843: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T01:14:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:27 smithi170 ceph-mon[102832]: pgmap v3844: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T01:14:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:27 smithi082 ceph-mon[98694]: pgmap v3844: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T01:14:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:27 smithi098 ceph-mon[103124]: pgmap v3844: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T01:14:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6900.125342 secs 2023-09-10T01:14:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:29 smithi170 ceph-mon[102832]: pgmap v3845: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T01:14:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6900.837685 secs 2023-09-10T01:14:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6900.125342 secs 2023-09-10T01:14:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:29 smithi082 ceph-mon[98694]: pgmap v3845: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T01:14:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6900.837685 secs 2023-09-10T01:14:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6900.125342 secs 2023-09-10T01:14:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:29 smithi098 ceph-mon[103124]: pgmap v3845: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T01:14:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6900.837685 secs 2023-09-10T01:14:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:31 smithi170 ceph-mon[102832]: pgmap v3846: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 219 KiB/s wr, 16 op/s 2023-09-10T01:14:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:14:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:31 smithi082 ceph-mon[98694]: pgmap v3846: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 219 KiB/s wr, 16 op/s 2023-09-10T01:14:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:14:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:31 smithi098 ceph-mon[103124]: pgmap v3846: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 219 KiB/s wr, 16 op/s 2023-09-10T01:14:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:14:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:14:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:14:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:14:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:14:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:14:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:14:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:14:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:14:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:14:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:33 smithi170 ceph-mon[102832]: pgmap v3847: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T01:14:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:33 smithi082 ceph-mon[98694]: pgmap v3847: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T01:14:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:33 smithi098 ceph-mon[103124]: pgmap v3847: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T01:14:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6905.125420 secs 2023-09-10T01:14:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6905.837751 secs 2023-09-10T01:14:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6905.125420 secs 2023-09-10T01:14:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6905.837751 secs 2023-09-10T01:14:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6905.125420 secs 2023-09-10T01:14:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6905.837751 secs 2023-09-10T01:14:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:35 smithi170 ceph-mon[102832]: pgmap v3848: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T01:14:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:35 smithi082 ceph-mon[98694]: pgmap v3848: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T01:14:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:35 smithi098 ceph-mon[103124]: pgmap v3848: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T01:14:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:37 smithi170 ceph-mon[102832]: pgmap v3849: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 290 KiB/s wr, 18 op/s 2023-09-10T01:14:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:37 smithi082 ceph-mon[98694]: pgmap v3849: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 290 KiB/s wr, 18 op/s 2023-09-10T01:14:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:37 smithi098 ceph-mon[103124]: pgmap v3849: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 290 KiB/s wr, 18 op/s 2023-09-10T01:14:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6910.125479 secs 2023-09-10T01:14:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:39 smithi170 ceph-mon[102832]: pgmap v3850: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 187 KiB/s wr, 12 op/s 2023-09-10T01:14:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6910.837814 secs 2023-09-10T01:14:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6910.125479 secs 2023-09-10T01:14:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:39 smithi082 ceph-mon[98694]: pgmap v3850: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 187 KiB/s wr, 12 op/s 2023-09-10T01:14:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6910.837814 secs 2023-09-10T01:14:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6910.125479 secs 2023-09-10T01:14:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:39 smithi098 ceph-mon[103124]: pgmap v3850: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 187 KiB/s wr, 12 op/s 2023-09-10T01:14:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6910.837814 secs 2023-09-10T01:14:41.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:41 smithi170 ceph-mon[102832]: pgmap v3851: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 312 KiB/s wr, 18 op/s 2023-09-10T01:14:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:41 smithi082 ceph-mon[98694]: pgmap v3851: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 312 KiB/s wr, 18 op/s 2023-09-10T01:14:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:41 smithi098 ceph-mon[103124]: pgmap v3851: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 312 KiB/s wr, 18 op/s 2023-09-10T01:14:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:43 smithi170 ceph-mon[102832]: pgmap v3852: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 229 KiB/s wr, 12 op/s 2023-09-10T01:14:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:43 smithi082 ceph-mon[98694]: pgmap v3852: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 229 KiB/s wr, 12 op/s 2023-09-10T01:14:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:43 smithi098 ceph-mon[103124]: pgmap v3852: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 229 KiB/s wr, 12 op/s 2023-09-10T01:14:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6915.125565 secs 2023-09-10T01:14:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6915.837885 secs 2023-09-10T01:14:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6915.125565 secs 2023-09-10T01:14:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6915.837885 secs 2023-09-10T01:14:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6915.125565 secs 2023-09-10T01:14:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6915.837885 secs 2023-09-10T01:14:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:45 smithi170 ceph-mon[102832]: pgmap v3853: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 229 KiB/s wr, 12 op/s 2023-09-10T01:14:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:45 smithi082 ceph-mon[98694]: pgmap v3853: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 229 KiB/s wr, 12 op/s 2023-09-10T01:14:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:45 smithi098 ceph-mon[103124]: pgmap v3853: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 229 KiB/s wr, 12 op/s 2023-09-10T01:14:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:47 smithi170 ceph-mon[102832]: pgmap v3854: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 261 KiB/s wr, 18 op/s 2023-09-10T01:14:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:47 smithi082 ceph-mon[98694]: pgmap v3854: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 261 KiB/s wr, 18 op/s 2023-09-10T01:14:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:47 smithi098 ceph-mon[103124]: pgmap v3854: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 261 KiB/s wr, 18 op/s 2023-09-10T01:14:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6918.642419 secs 2023-09-10T01:14:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6918.642419 secs 2023-09-10T01:14:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6918.642419 secs 2023-09-10T01:14:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6920.125623 secs 2023-09-10T01:14:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:49 smithi170 ceph-mon[102832]: pgmap v3855: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T01:14:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6920.837949 secs 2023-09-10T01:14:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6920.125623 secs 2023-09-10T01:14:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:49 smithi082 ceph-mon[98694]: pgmap v3855: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T01:14:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6920.837949 secs 2023-09-10T01:14:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6920.125623 secs 2023-09-10T01:14:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:49 smithi098 ceph-mon[103124]: pgmap v3855: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T01:14:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6920.837949 secs 2023-09-10T01:14:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:51 smithi170 ceph-mon[102832]: pgmap v3856: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 198 KiB/s wr, 18 op/s 2023-09-10T01:14:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:51 smithi082 ceph-mon[98694]: pgmap v3856: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 198 KiB/s wr, 18 op/s 2023-09-10T01:14:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:51 smithi098 ceph-mon[103124]: pgmap v3856: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 198 KiB/s wr, 18 op/s 2023-09-10T01:14:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:14:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:14:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:14:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:14:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:14:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:14:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:14:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:14:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:14:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6923.642478 secs 2023-09-10T01:14:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:53 smithi170 ceph-mon[102832]: pgmap v3857: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T01:14:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6923.642478 secs 2023-09-10T01:14:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:53 smithi082 ceph-mon[98694]: pgmap v3857: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T01:14:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6923.642478 secs 2023-09-10T01:14:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:53 smithi098 ceph-mon[103124]: pgmap v3857: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T01:14:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6925.125683 secs 2023-09-10T01:14:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6925.838015 secs 2023-09-10T01:14:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6925.125683 secs 2023-09-10T01:14:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6925.838015 secs 2023-09-10T01:14:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6925.125683 secs 2023-09-10T01:14:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6925.838015 secs 2023-09-10T01:14:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:55 smithi170 ceph-mon[102832]: pgmap v3858: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T01:14:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:55 smithi082 ceph-mon[98694]: pgmap v3858: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T01:14:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:55 smithi098 ceph-mon[103124]: pgmap v3858: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T01:14:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:57 smithi170 ceph-mon[102832]: pgmap v3859: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 127 KiB/s wr, 16 op/s 2023-09-10T01:14:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:57 smithi082 ceph-mon[98694]: pgmap v3859: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 127 KiB/s wr, 16 op/s 2023-09-10T01:14:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:57 smithi098 ceph-mon[103124]: pgmap v3859: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 127 KiB/s wr, 16 op/s 2023-09-10T01:14:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6928.642538 secs 2023-09-10T01:14:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6928.642538 secs 2023-09-10T01:14:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6928.642538 secs 2023-09-10T01:14:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6930.125742 secs 2023-09-10T01:14:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:59 smithi170 ceph-mon[102832]: pgmap v3860: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T01:14:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:14:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6930.838083 secs 2023-09-10T01:14:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6930.125742 secs 2023-09-10T01:14:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:59 smithi082 ceph-mon[98694]: pgmap v3860: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T01:14:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:14:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6930.838083 secs 2023-09-10T01:15:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6930.125742 secs 2023-09-10T01:15:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:59 smithi098 ceph-mon[103124]: pgmap v3860: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T01:15:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:14:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6930.838083 secs 2023-09-10T01:15:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:01 smithi170 ceph-mon[102832]: pgmap v3861: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 133 KiB/s wr, 18 op/s 2023-09-10T01:15:01.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:15:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:01 smithi082 ceph-mon[98694]: pgmap v3861: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 133 KiB/s wr, 18 op/s 2023-09-10T01:15:01.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:15:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:01 smithi098 ceph-mon[103124]: pgmap v3861: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 133 KiB/s wr, 18 op/s 2023-09-10T01:15:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:15:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:15:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:15:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:15:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:15:02.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:15:02.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:15:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:15:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:15:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:15:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6933.642602 secs 2023-09-10T01:15:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:03 smithi170 ceph-mon[102832]: pgmap v3862: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T01:15:03.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6935.125797 secs 2023-09-10T01:15:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6933.642602 secs 2023-09-10T01:15:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:03 smithi082 ceph-mon[98694]: pgmap v3862: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T01:15:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6935.125797 secs 2023-09-10T01:15:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6933.642602 secs 2023-09-10T01:15:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:03 smithi098 ceph-mon[103124]: pgmap v3862: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T01:15:04.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6935.125797 secs 2023-09-10T01:15:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6935.838148 secs 2023-09-10T01:15:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6935.838148 secs 2023-09-10T01:15:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6935.838148 secs 2023-09-10T01:15:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:05 smithi170 ceph-mon[102832]: pgmap v3863: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T01:15:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:05 smithi082 ceph-mon[98694]: pgmap v3863: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T01:15:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:05 smithi098 ceph-mon[103124]: pgmap v3863: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T01:15:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:07 smithi170 ceph-mon[102832]: pgmap v3864: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 126 KiB/s wr, 29 op/s 2023-09-10T01:15:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:07 smithi082 ceph-mon[98694]: pgmap v3864: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 126 KiB/s wr, 29 op/s 2023-09-10T01:15:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:07 smithi098 ceph-mon[103124]: pgmap v3864: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 126 KiB/s wr, 29 op/s 2023-09-10T01:15:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6938.642718 secs 2023-09-10T01:15:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:08 smithi170 ceph-mon[102832]: osdmap e311: 12 total, 12 up, 12 in 2023-09-10T01:15:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6940.125854 secs 2023-09-10T01:15:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6940.838213 secs 2023-09-10T01:15:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6938.642718 secs 2023-09-10T01:15:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:08 smithi082 ceph-mon[98694]: osdmap e311: 12 total, 12 up, 12 in 2023-09-10T01:15:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6940.125854 secs 2023-09-10T01:15:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6940.838213 secs 2023-09-10T01:15:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6938.642718 secs 2023-09-10T01:15:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:08 smithi098 ceph-mon[103124]: osdmap e311: 12 total, 12 up, 12 in 2023-09-10T01:15:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6940.125854 secs 2023-09-10T01:15:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6940.838213 secs 2023-09-10T01:15:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:09 smithi170 ceph-mon[102832]: pgmap v3866: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T01:15:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:09 smithi082 ceph-mon[98694]: pgmap v3866: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T01:15:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:09 smithi098 ceph-mon[103124]: pgmap v3866: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T01:15:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:11 smithi170 ceph-mon[102832]: pgmap v3867: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 112 KiB/s wr, 26 op/s 2023-09-10T01:15:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:11 smithi082 ceph-mon[98694]: pgmap v3867: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 112 KiB/s wr, 26 op/s 2023-09-10T01:15:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:11 smithi098 ceph-mon[103124]: pgmap v3867: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 112 KiB/s wr, 26 op/s 2023-09-10T01:15:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6943.642832 secs 2023-09-10T01:15:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:13 smithi170 ceph-mon[102832]: pgmap v3868: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 112 KiB/s wr, 26 op/s 2023-09-10T01:15:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6945.125912 secs 2023-09-10T01:15:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6945.838278 secs 2023-09-10T01:15:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6943.642832 secs 2023-09-10T01:15:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:13 smithi082 ceph-mon[98694]: pgmap v3868: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 112 KiB/s wr, 26 op/s 2023-09-10T01:15:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6945.125912 secs 2023-09-10T01:15:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6945.838278 secs 2023-09-10T01:15:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6943.642832 secs 2023-09-10T01:15:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:13 smithi098 ceph-mon[103124]: pgmap v3868: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 112 KiB/s wr, 26 op/s 2023-09-10T01:15:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6945.125912 secs 2023-09-10T01:15:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6945.838278 secs 2023-09-10T01:15:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:15 smithi170 ceph-mon[102832]: pgmap v3869: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 112 KiB/s wr, 26 op/s 2023-09-10T01:15:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:15 smithi170 ceph-mon[102832]: osdmap e312: 12 total, 12 up, 12 in 2023-09-10T01:15:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:15 smithi082 ceph-mon[98694]: pgmap v3869: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 112 KiB/s wr, 26 op/s 2023-09-10T01:15:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:15 smithi082 ceph-mon[98694]: osdmap e312: 12 total, 12 up, 12 in 2023-09-10T01:15:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:15 smithi098 ceph-mon[103124]: pgmap v3869: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 112 KiB/s wr, 26 op/s 2023-09-10T01:15:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:15 smithi098 ceph-mon[103124]: osdmap e312: 12 total, 12 up, 12 in 2023-09-10T01:15:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:16 smithi170 ceph-mon[102832]: pgmap v3871: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 130 KiB/s wr, 13 op/s 2023-09-10T01:15:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:16 smithi082 ceph-mon[98694]: pgmap v3871: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 130 KiB/s wr, 13 op/s 2023-09-10T01:15:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:16 smithi098 ceph-mon[103124]: pgmap v3871: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 130 KiB/s wr, 13 op/s 2023-09-10T01:15:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6948.642952 secs 2023-09-10T01:15:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6948.642952 secs 2023-09-10T01:15:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6948.642952 secs 2023-09-10T01:15:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6950.125971 secs 2023-09-10T01:15:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:18 smithi170 ceph-mon[102832]: pgmap v3872: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T01:15:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6950.838344 secs 2023-09-10T01:15:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6950.125971 secs 2023-09-10T01:15:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:18 smithi082 ceph-mon[98694]: pgmap v3872: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T01:15:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6950.838344 secs 2023-09-10T01:15:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6950.125971 secs 2023-09-10T01:15:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:18 smithi098 ceph-mon[103124]: pgmap v3872: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T01:15:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6950.838344 secs 2023-09-10T01:15:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:21 smithi170 ceph-mon[102832]: pgmap v3873: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T01:15:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:21 smithi082 ceph-mon[98694]: pgmap v3873: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T01:15:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:21 smithi098 ceph-mon[103124]: pgmap v3873: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T01:15:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6953.643012 secs 2023-09-10T01:15:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:23 smithi170 ceph-mon[102832]: pgmap v3874: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T01:15:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6953.643012 secs 2023-09-10T01:15:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:23 smithi082 ceph-mon[98694]: pgmap v3874: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T01:15:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6953.643012 secs 2023-09-10T01:15:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:23 smithi098 ceph-mon[103124]: pgmap v3874: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T01:15:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6955.126029 secs 2023-09-10T01:15:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6955.838412 secs 2023-09-10T01:15:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6955.126029 secs 2023-09-10T01:15:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6955.838412 secs 2023-09-10T01:15:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6955.126029 secs 2023-09-10T01:15:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6955.838412 secs 2023-09-10T01:15:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:25 smithi170 ceph-mon[102832]: pgmap v3875: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T01:15:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:25 smithi082 ceph-mon[98694]: pgmap v3875: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T01:15:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:25 smithi098 ceph-mon[103124]: pgmap v3875: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T01:15:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:27 smithi170 ceph-mon[102832]: pgmap v3876: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T01:15:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:27 smithi082 ceph-mon[98694]: pgmap v3876: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T01:15:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:27 smithi098 ceph-mon[103124]: pgmap v3876: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T01:15:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6958.643072 secs 2023-09-10T01:15:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6958.643072 secs 2023-09-10T01:15:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6958.643072 secs 2023-09-10T01:15:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6960.126136 secs 2023-09-10T01:15:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:29 smithi170 ceph-mon[102832]: pgmap v3877: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 98 KiB/s wr, 10 op/s 2023-09-10T01:15:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6960.838482 secs 2023-09-10T01:15:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6960.126136 secs 2023-09-10T01:15:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:29 smithi082 ceph-mon[98694]: pgmap v3877: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 98 KiB/s wr, 10 op/s 2023-09-10T01:15:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6960.838482 secs 2023-09-10T01:15:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6960.126136 secs 2023-09-10T01:15:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:29 smithi098 ceph-mon[103124]: pgmap v3877: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 98 KiB/s wr, 10 op/s 2023-09-10T01:15:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6960.838482 secs 2023-09-10T01:15:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:31 smithi170 ceph-mon[102832]: pgmap v3878: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 227 KiB/s wr, 16 op/s 2023-09-10T01:15:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:31 smithi082 ceph-mon[98694]: pgmap v3878: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 227 KiB/s wr, 16 op/s 2023-09-10T01:15:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:31 smithi098 ceph-mon[103124]: pgmap v3878: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 227 KiB/s wr, 16 op/s 2023-09-10T01:15:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:33 smithi082 ceph-mon[98694]: pgmap v3879: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T01:15:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:33 smithi098 ceph-mon[103124]: pgmap v3879: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T01:15:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:33 smithi170 ceph-mon[102832]: pgmap v3879: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T01:15:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6965.126191 secs 2023-09-10T01:15:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6965.838548 secs 2023-09-10T01:15:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6965.126191 secs 2023-09-10T01:15:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6965.838548 secs 2023-09-10T01:15:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6965.126191 secs 2023-09-10T01:15:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6965.838548 secs 2023-09-10T01:15:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:35 smithi082 ceph-mon[98694]: pgmap v3880: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T01:15:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:15:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:35 smithi098 ceph-mon[103124]: pgmap v3880: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T01:15:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:15:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:35 smithi170 ceph-mon[102832]: pgmap v3880: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T01:15:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:15:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:15:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:15:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:15:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:15:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:15:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:15:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:15:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:15:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:15:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:37 smithi082 ceph-mon[98694]: pgmap v3881: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 232 KiB/s wr, 17 op/s 2023-09-10T01:15:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:37 smithi098 ceph-mon[103124]: pgmap v3881: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 232 KiB/s wr, 17 op/s 2023-09-10T01:15:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:37 smithi170 ceph-mon[102832]: pgmap v3881: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 232 KiB/s wr, 17 op/s 2023-09-10T01:15:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6970.126248 secs 2023-09-10T01:15:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:39 smithi082 ceph-mon[98694]: pgmap v3882: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 175 KiB/s wr, 11 op/s 2023-09-10T01:15:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6970.838613 secs 2023-09-10T01:15:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6970.126248 secs 2023-09-10T01:15:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:39 smithi098 ceph-mon[103124]: pgmap v3882: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 175 KiB/s wr, 11 op/s 2023-09-10T01:15:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6970.838613 secs 2023-09-10T01:15:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6970.126248 secs 2023-09-10T01:15:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:39 smithi170 ceph-mon[102832]: pgmap v3882: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 175 KiB/s wr, 11 op/s 2023-09-10T01:15:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6970.838613 secs 2023-09-10T01:15:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:41 smithi082 ceph-mon[98694]: pgmap v3883: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 275 KiB/s wr, 18 op/s 2023-09-10T01:15:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:41 smithi098 ceph-mon[103124]: pgmap v3883: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 275 KiB/s wr, 18 op/s 2023-09-10T01:15:41.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:41 smithi170 ceph-mon[102832]: pgmap v3883: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 275 KiB/s wr, 18 op/s 2023-09-10T01:15:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:43 smithi098 ceph-mon[103124]: pgmap v3884: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T01:15:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:43 smithi170 ceph-mon[102832]: pgmap v3884: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T01:15:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:43 smithi082 ceph-mon[98694]: pgmap v3884: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T01:15:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6975.126306 secs 2023-09-10T01:15:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6975.838675 secs 2023-09-10T01:15:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6975.126306 secs 2023-09-10T01:15:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6975.838675 secs 2023-09-10T01:15:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6975.126306 secs 2023-09-10T01:15:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6975.838675 secs 2023-09-10T01:15:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:45 smithi098 ceph-mon[103124]: pgmap v3885: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T01:15:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:45 smithi170 ceph-mon[102832]: pgmap v3885: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T01:15:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:45 smithi082 ceph-mon[98694]: pgmap v3885: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T01:15:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:47 smithi098 ceph-mon[103124]: pgmap v3886: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 278 KiB/s wr, 18 op/s 2023-09-10T01:15:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:47 smithi170 ceph-mon[102832]: pgmap v3886: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 278 KiB/s wr, 18 op/s 2023-09-10T01:15:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:47 smithi082 ceph-mon[98694]: pgmap v3886: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 278 KiB/s wr, 18 op/s 2023-09-10T01:15:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6978.643368 secs 2023-09-10T01:15:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6978.643368 secs 2023-09-10T01:15:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6978.643368 secs 2023-09-10T01:15:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6980.126363 secs 2023-09-10T01:15:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:49 smithi098 ceph-mon[103124]: pgmap v3887: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 231 KiB/s wr, 12 op/s 2023-09-10T01:15:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6980.838744 secs 2023-09-10T01:15:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6980.126363 secs 2023-09-10T01:15:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:49 smithi170 ceph-mon[102832]: pgmap v3887: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 231 KiB/s wr, 12 op/s 2023-09-10T01:15:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6980.838744 secs 2023-09-10T01:15:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6980.126363 secs 2023-09-10T01:15:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:49 smithi082 ceph-mon[98694]: pgmap v3887: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 231 KiB/s wr, 12 op/s 2023-09-10T01:15:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6980.838744 secs 2023-09-10T01:15:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:51 smithi098 ceph-mon[103124]: pgmap v3888: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 273 KiB/s wr, 18 op/s 2023-09-10T01:15:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:51 smithi170 ceph-mon[102832]: pgmap v3888: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 273 KiB/s wr, 18 op/s 2023-09-10T01:15:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:51 smithi082 ceph-mon[98694]: pgmap v3888: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 273 KiB/s wr, 18 op/s 2023-09-10T01:15:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:15:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:15:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:15:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:15:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:15:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:15:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:15:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:15:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:15:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6983.643430 secs 2023-09-10T01:15:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:53 smithi098 ceph-mon[103124]: pgmap v3889: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 173 KiB/s wr, 11 op/s 2023-09-10T01:15:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6983.643430 secs 2023-09-10T01:15:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:53 smithi170 ceph-mon[102832]: pgmap v3889: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 173 KiB/s wr, 11 op/s 2023-09-10T01:15:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6983.643430 secs 2023-09-10T01:15:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:53 smithi082 ceph-mon[98694]: pgmap v3889: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 173 KiB/s wr, 11 op/s 2023-09-10T01:15:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6985.126432 secs 2023-09-10T01:15:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6985.838807 secs 2023-09-10T01:15:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6985.126432 secs 2023-09-10T01:15:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6985.838807 secs 2023-09-10T01:15:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6985.126432 secs 2023-09-10T01:15:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6985.838807 secs 2023-09-10T01:15:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:55 smithi098 ceph-mon[103124]: pgmap v3890: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 173 KiB/s wr, 11 op/s 2023-09-10T01:15:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:55 smithi170 ceph-mon[102832]: pgmap v3890: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 173 KiB/s wr, 11 op/s 2023-09-10T01:15:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:55 smithi082 ceph-mon[98694]: pgmap v3890: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 173 KiB/s wr, 11 op/s 2023-09-10T01:15:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:57 smithi098 ceph-mon[103124]: pgmap v3891: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 228 KiB/s wr, 17 op/s 2023-09-10T01:15:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:57 smithi170 ceph-mon[102832]: pgmap v3891: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 228 KiB/s wr, 17 op/s 2023-09-10T01:15:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:57 smithi082 ceph-mon[98694]: pgmap v3891: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 228 KiB/s wr, 17 op/s 2023-09-10T01:15:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6988.643506 secs 2023-09-10T01:15:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6988.643506 secs 2023-09-10T01:15:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6988.643506 secs 2023-09-10T01:15:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6990.126491 secs 2023-09-10T01:15:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:59 smithi170 ceph-mon[102832]: pgmap v3892: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T01:15:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:15:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6990.838873 secs 2023-09-10T01:15:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6990.126491 secs 2023-09-10T01:15:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:59 smithi082 ceph-mon[98694]: pgmap v3892: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T01:15:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:15:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6990.838873 secs 2023-09-10T01:16:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6990.126491 secs 2023-09-10T01:16:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:59 smithi098 ceph-mon[103124]: pgmap v3892: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T01:16:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:15:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6990.838873 secs 2023-09-10T01:16:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:01 smithi170 ceph-mon[102832]: pgmap v3893: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 122 KiB/s wr, 16 op/s 2023-09-10T01:16:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:01 smithi082 ceph-mon[98694]: pgmap v3893: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 122 KiB/s wr, 16 op/s 2023-09-10T01:16:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:01 smithi098 ceph-mon[103124]: pgmap v3893: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 122 KiB/s wr, 16 op/s 2023-09-10T01:16:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:16:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:16:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:16:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6993.643566 secs 2023-09-10T01:16:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:03 smithi170 ceph-mon[102832]: pgmap v3894: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T01:16:03.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:16:03.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:16:03.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:16:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6993.643566 secs 2023-09-10T01:16:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:03 smithi082 ceph-mon[98694]: pgmap v3894: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T01:16:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:16:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:16:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:16:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6993.643566 secs 2023-09-10T01:16:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:03 smithi098 ceph-mon[103124]: pgmap v3894: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T01:16:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:16:04.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:16:04.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:16:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6995.126554 secs 2023-09-10T01:16:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 6995.838937 secs 2023-09-10T01:16:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6995.126554 secs 2023-09-10T01:16:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 6995.838937 secs 2023-09-10T01:16:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6995.126554 secs 2023-09-10T01:16:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 6995.838937 secs 2023-09-10T01:16:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:05 smithi170 ceph-mon[102832]: pgmap v3895: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T01:16:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:05 smithi082 ceph-mon[98694]: pgmap v3895: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T01:16:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:05 smithi098 ceph-mon[103124]: pgmap v3895: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T01:16:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:07 smithi170 ceph-mon[102832]: pgmap v3896: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 126 KiB/s wr, 30 op/s 2023-09-10T01:16:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:07 smithi082 ceph-mon[98694]: pgmap v3896: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 126 KiB/s wr, 30 op/s 2023-09-10T01:16:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:07 smithi098 ceph-mon[103124]: pgmap v3896: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 126 KiB/s wr, 30 op/s 2023-09-10T01:16:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 6998.643645 secs 2023-09-10T01:16:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:08 smithi170 ceph-mon[102832]: osdmap e313: 12 total, 12 up, 12 in 2023-09-10T01:16:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 6998.643645 secs 2023-09-10T01:16:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:08 smithi082 ceph-mon[98694]: osdmap e313: 12 total, 12 up, 12 in 2023-09-10T01:16:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 6998.643645 secs 2023-09-10T01:16:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:08 smithi098 ceph-mon[103124]: osdmap e313: 12 total, 12 up, 12 in 2023-09-10T01:16:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7000.126612 secs 2023-09-10T01:16:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:09 smithi170 ceph-mon[102832]: pgmap v3898: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T01:16:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7000.839005 secs 2023-09-10T01:16:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7000.126612 secs 2023-09-10T01:16:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:09 smithi082 ceph-mon[98694]: pgmap v3898: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T01:16:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7000.839005 secs 2023-09-10T01:16:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7000.126612 secs 2023-09-10T01:16:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:09 smithi098 ceph-mon[103124]: pgmap v3898: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T01:16:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7000.839005 secs 2023-09-10T01:16:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:11 smithi170 ceph-mon[102832]: pgmap v3899: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T01:16:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:11 smithi082 ceph-mon[98694]: pgmap v3899: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T01:16:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:11 smithi098 ceph-mon[103124]: pgmap v3899: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T01:16:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7003.643767 secs 2023-09-10T01:16:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:13 smithi170 ceph-mon[102832]: pgmap v3900: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T01:16:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7003.643767 secs 2023-09-10T01:16:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:13 smithi082 ceph-mon[98694]: pgmap v3900: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T01:16:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7003.643767 secs 2023-09-10T01:16:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:13 smithi098 ceph-mon[103124]: pgmap v3900: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T01:16:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7005.126686 secs 2023-09-10T01:16:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7005.839075 secs 2023-09-10T01:16:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7005.126686 secs 2023-09-10T01:16:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7005.839075 secs 2023-09-10T01:16:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7005.126686 secs 2023-09-10T01:16:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7005.839075 secs 2023-09-10T01:16:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:15 smithi170 ceph-mon[102832]: pgmap v3901: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T01:16:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:15 smithi170 ceph-mon[102832]: osdmap e314: 12 total, 12 up, 12 in 2023-09-10T01:16:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:15 smithi082 ceph-mon[98694]: pgmap v3901: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T01:16:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:15 smithi082 ceph-mon[98694]: osdmap e314: 12 total, 12 up, 12 in 2023-09-10T01:16:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:15 smithi098 ceph-mon[103124]: pgmap v3901: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T01:16:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:15 smithi098 ceph-mon[103124]: osdmap e314: 12 total, 12 up, 12 in 2023-09-10T01:16:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:17 smithi170 ceph-mon[102832]: pgmap v3903: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 130 KiB/s wr, 13 op/s 2023-09-10T01:16:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:17 smithi082 ceph-mon[98694]: pgmap v3903: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 130 KiB/s wr, 13 op/s 2023-09-10T01:16:18.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:17 smithi098 ceph-mon[103124]: pgmap v3903: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 130 KiB/s wr, 13 op/s 2023-09-10T01:16:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7008.643798 secs 2023-09-10T01:16:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7008.643798 secs 2023-09-10T01:16:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7008.643798 secs 2023-09-10T01:16:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7010.126747 secs 2023-09-10T01:16:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:19 smithi170 ceph-mon[102832]: pgmap v3904: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T01:16:19.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7010.839140 secs 2023-09-10T01:16:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7010.126747 secs 2023-09-10T01:16:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:19 smithi082 ceph-mon[98694]: pgmap v3904: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T01:16:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7010.839140 secs 2023-09-10T01:16:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7010.126747 secs 2023-09-10T01:16:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:19 smithi098 ceph-mon[103124]: pgmap v3904: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T01:16:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7010.839140 secs 2023-09-10T01:16:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:21 smithi170 ceph-mon[102832]: pgmap v3905: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T01:16:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:21 smithi082 ceph-mon[98694]: pgmap v3905: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T01:16:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:21 smithi098 ceph-mon[103124]: pgmap v3905: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T01:16:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:23 smithi170 ceph-mon[102832]: pgmap v3906: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T01:16:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:16:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:23 smithi082 ceph-mon[98694]: pgmap v3906: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T01:16:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:16:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:23 smithi098 ceph-mon[103124]: pgmap v3906: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T01:16:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:16:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7015.126814 secs 2023-09-10T01:16:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:16:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:16:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:16:24.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7015.839202 secs 2023-09-10T01:16:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7015.126814 secs 2023-09-10T01:16:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:16:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:16:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:16:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7015.839202 secs 2023-09-10T01:16:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7015.126814 secs 2023-09-10T01:16:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:16:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:16:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:16:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7015.839202 secs 2023-09-10T01:16:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:25 smithi170 ceph-mon[102832]: pgmap v3907: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T01:16:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:25 smithi082 ceph-mon[98694]: pgmap v3907: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T01:16:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:25 smithi098 ceph-mon[103124]: pgmap v3907: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T01:16:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:27 smithi170 ceph-mon[102832]: pgmap v3908: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T01:16:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:27 smithi082 ceph-mon[98694]: pgmap v3908: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T01:16:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:27 smithi098 ceph-mon[103124]: pgmap v3908: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T01:16:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7018.643919 secs 2023-09-10T01:16:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7018.643919 secs 2023-09-10T01:16:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7018.643919 secs 2023-09-10T01:16:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7020.126898 secs 2023-09-10T01:16:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:29 smithi170 ceph-mon[102832]: pgmap v3909: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T01:16:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7020.839267 secs 2023-09-10T01:16:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7020.126898 secs 2023-09-10T01:16:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:29 smithi082 ceph-mon[98694]: pgmap v3909: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T01:16:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7020.839267 secs 2023-09-10T01:16:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7020.126898 secs 2023-09-10T01:16:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:29 smithi098 ceph-mon[103124]: pgmap v3909: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T01:16:30.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7020.839267 secs 2023-09-10T01:16:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:31 smithi170 ceph-mon[102832]: pgmap v3910: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 229 KiB/s wr, 17 op/s 2023-09-10T01:16:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:31 smithi082 ceph-mon[98694]: pgmap v3910: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 229 KiB/s wr, 17 op/s 2023-09-10T01:16:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:31 smithi098 ceph-mon[103124]: pgmap v3910: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 229 KiB/s wr, 17 op/s 2023-09-10T01:16:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:32 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:16:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:16:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:16:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:32 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:16:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:16:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:16:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:32 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:16:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:16:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:16:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:33 smithi170 ceph-mon[102832]: pgmap v3911: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 187 KiB/s wr, 12 op/s 2023-09-10T01:16:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:33 smithi082 ceph-mon[98694]: pgmap v3911: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 187 KiB/s wr, 12 op/s 2023-09-10T01:16:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:33 smithi098 ceph-mon[103124]: pgmap v3911: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 187 KiB/s wr, 12 op/s 2023-09-10T01:16:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7025.126986 secs 2023-09-10T01:16:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7025.839334 secs 2023-09-10T01:16:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7025.126986 secs 2023-09-10T01:16:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7025.839334 secs 2023-09-10T01:16:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7025.126986 secs 2023-09-10T01:16:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7025.839334 secs 2023-09-10T01:16:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:35 smithi170 ceph-mon[102832]: pgmap v3912: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 187 KiB/s wr, 12 op/s 2023-09-10T01:16:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:16:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:35 smithi082 ceph-mon[98694]: pgmap v3912: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 187 KiB/s wr, 12 op/s 2023-09-10T01:16:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:16:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:35 smithi098 ceph-mon[103124]: pgmap v3912: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 187 KiB/s wr, 12 op/s 2023-09-10T01:16:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:16:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:16:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:16:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:16:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:16:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:16:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:16:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:37 smithi170 ceph-mon[102832]: pgmap v3913: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 234 KiB/s wr, 18 op/s 2023-09-10T01:16:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:37 smithi082 ceph-mon[98694]: pgmap v3913: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 234 KiB/s wr, 18 op/s 2023-09-10T01:16:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:37 smithi098 ceph-mon[103124]: pgmap v3913: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 234 KiB/s wr, 18 op/s 2023-09-10T01:16:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7030.127044 secs 2023-09-10T01:16:39.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:39 smithi170 ceph-mon[102832]: pgmap v3914: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 177 KiB/s wr, 12 op/s 2023-09-10T01:16:39.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7030.839399 secs 2023-09-10T01:16:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7030.127044 secs 2023-09-10T01:16:39.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:39 smithi082 ceph-mon[98694]: pgmap v3914: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 177 KiB/s wr, 12 op/s 2023-09-10T01:16:39.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7030.839399 secs 2023-09-10T01:16:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7030.127044 secs 2023-09-10T01:16:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:39 smithi098 ceph-mon[103124]: pgmap v3914: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 177 KiB/s wr, 12 op/s 2023-09-10T01:16:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7030.839399 secs 2023-09-10T01:16:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:16:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:16:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:16:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:41 smithi170 ceph-mon[102832]: pgmap v3915: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 255 KiB/s wr, 18 op/s 2023-09-10T01:16:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:41 smithi082 ceph-mon[98694]: pgmap v3915: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 255 KiB/s wr, 18 op/s 2023-09-10T01:16:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:41 smithi098 ceph-mon[103124]: pgmap v3915: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 255 KiB/s wr, 18 op/s 2023-09-10T01:16:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:43 smithi170 ceph-mon[102832]: pgmap v3916: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T01:16:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7035.127132 secs 2023-09-10T01:16:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7035.839464 secs 2023-09-10T01:16:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:43 smithi082 ceph-mon[98694]: pgmap v3916: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T01:16:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7035.127132 secs 2023-09-10T01:16:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7035.839464 secs 2023-09-10T01:16:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:43 smithi098 ceph-mon[103124]: pgmap v3916: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T01:16:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7035.127132 secs 2023-09-10T01:16:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7035.839464 secs 2023-09-10T01:16:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:45 smithi170 ceph-mon[102832]: pgmap v3917: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T01:16:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:45 smithi082 ceph-mon[98694]: pgmap v3917: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T01:16:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:45 smithi098 ceph-mon[103124]: pgmap v3917: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T01:16:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:47 smithi170 ceph-mon[102832]: pgmap v3918: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 219 KiB/s wr, 18 op/s 2023-09-10T01:16:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:47 smithi082 ceph-mon[98694]: pgmap v3918: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 219 KiB/s wr, 18 op/s 2023-09-10T01:16:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:47 smithi098 ceph-mon[103124]: pgmap v3918: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 219 KiB/s wr, 18 op/s 2023-09-10T01:16:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7038.644178 secs 2023-09-10T01:16:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7040.127192 secs 2023-09-10T01:16:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7040.839488 secs 2023-09-10T01:16:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7038.644178 secs 2023-09-10T01:16:48.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7040.127192 secs 2023-09-10T01:16:48.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7040.839488 secs 2023-09-10T01:16:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7038.644178 secs 2023-09-10T01:16:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7040.127192 secs 2023-09-10T01:16:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7040.839488 secs 2023-09-10T01:16:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:49 smithi170 ceph-mon[102832]: pgmap v3919: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 172 KiB/s wr, 12 op/s 2023-09-10T01:16:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:49 smithi082 ceph-mon[98694]: pgmap v3919: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 172 KiB/s wr, 12 op/s 2023-09-10T01:16:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:49 smithi098 ceph-mon[103124]: pgmap v3919: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 172 KiB/s wr, 12 op/s 2023-09-10T01:16:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:51 smithi170 ceph-mon[102832]: pgmap v3920: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 214 KiB/s wr, 18 op/s 2023-09-10T01:16:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:51 smithi082 ceph-mon[98694]: pgmap v3920: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 214 KiB/s wr, 18 op/s 2023-09-10T01:16:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:51 smithi098 ceph-mon[103124]: pgmap v3920: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 214 KiB/s wr, 18 op/s 2023-09-10T01:16:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:16:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:16:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:16:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:16:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:16:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:16:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:16:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:16:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:16:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7043.644235 secs 2023-09-10T01:16:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:53 smithi170 ceph-mon[102832]: pgmap v3921: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 136 KiB/s wr, 12 op/s 2023-09-10T01:16:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7045.127266 secs 2023-09-10T01:16:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7045.839549 secs 2023-09-10T01:16:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7043.644235 secs 2023-09-10T01:16:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:53 smithi082 ceph-mon[98694]: pgmap v3921: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 136 KiB/s wr, 12 op/s 2023-09-10T01:16:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7045.127266 secs 2023-09-10T01:16:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7045.839549 secs 2023-09-10T01:16:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7043.644235 secs 2023-09-10T01:16:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:53 smithi098 ceph-mon[103124]: pgmap v3921: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 136 KiB/s wr, 12 op/s 2023-09-10T01:16:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7045.127266 secs 2023-09-10T01:16:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7045.839549 secs 2023-09-10T01:16:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:55 smithi170 ceph-mon[102832]: pgmap v3922: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 136 KiB/s wr, 12 op/s 2023-09-10T01:16:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:55 smithi082 ceph-mon[98694]: pgmap v3922: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 136 KiB/s wr, 12 op/s 2023-09-10T01:16:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:55 smithi098 ceph-mon[103124]: pgmap v3922: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 136 KiB/s wr, 12 op/s 2023-09-10T01:16:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:57 smithi170 ceph-mon[102832]: pgmap v3923: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 191 KiB/s wr, 17 op/s 2023-09-10T01:16:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:57 smithi082 ceph-mon[98694]: pgmap v3923: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 191 KiB/s wr, 17 op/s 2023-09-10T01:16:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:57 smithi098 ceph-mon[103124]: pgmap v3923: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 191 KiB/s wr, 17 op/s 2023-09-10T01:16:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7048.644294 secs 2023-09-10T01:16:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7050.127325 secs 2023-09-10T01:16:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:58 smithi170 ceph-mon[102832]: pgmap v3924: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 97 KiB/s wr, 10 op/s 2023-09-10T01:16:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:16:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7050.839613 secs 2023-09-10T01:16:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7048.644294 secs 2023-09-10T01:16:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7050.127325 secs 2023-09-10T01:16:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:58 smithi082 ceph-mon[98694]: pgmap v3924: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 97 KiB/s wr, 10 op/s 2023-09-10T01:16:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:16:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7050.839613 secs 2023-09-10T01:16:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7048.644294 secs 2023-09-10T01:16:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7050.127325 secs 2023-09-10T01:16:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:58 smithi098 ceph-mon[103124]: pgmap v3924: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 97 KiB/s wr, 10 op/s 2023-09-10T01:16:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:16:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7050.839613 secs 2023-09-10T01:17:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:01 smithi082 ceph-mon[98694]: pgmap v3925: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 15 op/s 2023-09-10T01:17:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:01 smithi098 ceph-mon[103124]: pgmap v3925: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 15 op/s 2023-09-10T01:17:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:01 smithi170 ceph-mon[102832]: pgmap v3925: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 15 op/s 2023-09-10T01:17:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7053.644353 secs 2023-09-10T01:17:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:03 smithi082 ceph-mon[98694]: pgmap v3926: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 81 KiB/s wr, 9 op/s 2023-09-10T01:17:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:17:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7053.644353 secs 2023-09-10T01:17:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:03 smithi098 ceph-mon[103124]: pgmap v3926: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 81 KiB/s wr, 9 op/s 2023-09-10T01:17:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:17:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7053.644353 secs 2023-09-10T01:17:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:03 smithi170 ceph-mon[102832]: pgmap v3926: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 81 KiB/s wr, 9 op/s 2023-09-10T01:17:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:17:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7055.127381 secs 2023-09-10T01:17:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7055.839677 secs 2023-09-10T01:17:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7055.127381 secs 2023-09-10T01:17:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7055.839677 secs 2023-09-10T01:17:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7055.127381 secs 2023-09-10T01:17:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7055.839677 secs 2023-09-10T01:17:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:05 smithi082 ceph-mon[98694]: pgmap v3927: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 81 KiB/s wr, 9 op/s 2023-09-10T01:17:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:05 smithi098 ceph-mon[103124]: pgmap v3927: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 81 KiB/s wr, 9 op/s 2023-09-10T01:17:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:05 smithi170 ceph-mon[102832]: pgmap v3927: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 81 KiB/s wr, 9 op/s 2023-09-10T01:17:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:06.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:06.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:06.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:06.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:06.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:06.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:06.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:06.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:06.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:06.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:06.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:06.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:06.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:07.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:07.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:07.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:07.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:07.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:07.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:07.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:07.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:07.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:07 smithi170 ceph-mon[102832]: pgmap v3928: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T01:17:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:07 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T01:17:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:07 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:17:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:07.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:07.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:07.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:07.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:07.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:07.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:07 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T01:17:07.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:07 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T01:17:07.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:07.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:07.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:07.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:07.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:07.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:07.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:17:07.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:17:07.932 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:07 smithi082 ceph-mon[98694]: pgmap v3928: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T01:17:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:07 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T01:17:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:07 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:17:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:07.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:07.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:07.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:07.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:07 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T01:17:07.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:07 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T01:17:07.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:07.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:07.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:07.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:07.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:07.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:07.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:17:07.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:17:07.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:07 smithi098 ceph-mon[103124]: pgmap v3928: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T01:17:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:07 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T01:17:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:07 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:17:08.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:08.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:08.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:08.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:08.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:08.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:08.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:07 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T01:17:08.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:07 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T01:17:08.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:08.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:08.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:08.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:08.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:08.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:17:08.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:17:08.008 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:17:08.008 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:17:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7058.644415 secs 2023-09-10T01:17:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:08 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T01:17:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:08 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T01:17:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:08 smithi170 ceph-mon[102832]: osdmap e315: 12 total, 12 up, 12 in 2023-09-10T01:17:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7058.644415 secs 2023-09-10T01:17:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:08 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T01:17:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:08 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T01:17:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:08 smithi082 ceph-mon[98694]: osdmap e315: 12 total, 12 up, 12 in 2023-09-10T01:17:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7058.644415 secs 2023-09-10T01:17:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:08 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T01:17:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:08 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T01:17:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:08 smithi098 ceph-mon[103124]: osdmap e315: 12 total, 12 up, 12 in 2023-09-10T01:17:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7060.127441 secs 2023-09-10T01:17:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:09 smithi170 ceph-mon[102832]: pgmap v3930: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T01:17:09.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7060.839743 secs 2023-09-10T01:17:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7060.127441 secs 2023-09-10T01:17:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:09 smithi082 ceph-mon[98694]: pgmap v3930: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T01:17:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7060.839743 secs 2023-09-10T01:17:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7060.127441 secs 2023-09-10T01:17:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:09 smithi098 ceph-mon[103124]: pgmap v3930: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T01:17:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7060.839743 secs 2023-09-10T01:17:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:11 smithi170 ceph-mon[102832]: pgmap v3931: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T01:17:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:11 smithi082 ceph-mon[98694]: pgmap v3931: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T01:17:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:11 smithi098 ceph-mon[103124]: pgmap v3931: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T01:17:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7063.644499 secs 2023-09-10T01:17:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:13 smithi170 ceph-mon[102832]: pgmap v3932: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T01:17:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7063.644499 secs 2023-09-10T01:17:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:13 smithi082 ceph-mon[98694]: pgmap v3932: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T01:17:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7063.644499 secs 2023-09-10T01:17:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:13 smithi098 ceph-mon[103124]: pgmap v3932: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T01:17:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7065.127528 secs 2023-09-10T01:17:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7065.839808 secs 2023-09-10T01:17:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7065.127528 secs 2023-09-10T01:17:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7065.839808 secs 2023-09-10T01:17:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7065.127528 secs 2023-09-10T01:17:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7065.839808 secs 2023-09-10T01:17:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:15 smithi170 ceph-mon[102832]: pgmap v3933: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T01:17:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:15 smithi170 ceph-mon[102832]: osdmap e316: 12 total, 12 up, 12 in 2023-09-10T01:17:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:15 smithi082 ceph-mon[98694]: pgmap v3933: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T01:17:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:15 smithi082 ceph-mon[98694]: osdmap e316: 12 total, 12 up, 12 in 2023-09-10T01:17:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:15 smithi098 ceph-mon[103124]: pgmap v3933: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T01:17:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:15 smithi098 ceph-mon[103124]: osdmap e316: 12 total, 12 up, 12 in 2023-09-10T01:17:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:17 smithi170 ceph-mon[102832]: pgmap v3935: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 84 KiB/s wr, 13 op/s 2023-09-10T01:17:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:17 smithi082 ceph-mon[98694]: pgmap v3935: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 84 KiB/s wr, 13 op/s 2023-09-10T01:17:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:17 smithi098 ceph-mon[103124]: pgmap v3935: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 84 KiB/s wr, 13 op/s 2023-09-10T01:17:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7068.644558 secs 2023-09-10T01:17:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7068.644558 secs 2023-09-10T01:17:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7068.644558 secs 2023-09-10T01:17:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7070.127587 secs 2023-09-10T01:17:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:19 smithi170 ceph-mon[102832]: pgmap v3936: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T01:17:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7070.839874 secs 2023-09-10T01:17:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7070.127587 secs 2023-09-10T01:17:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:19 smithi082 ceph-mon[98694]: pgmap v3936: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T01:17:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7070.839874 secs 2023-09-10T01:17:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7070.127587 secs 2023-09-10T01:17:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:19 smithi098 ceph-mon[103124]: pgmap v3936: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T01:17:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7070.839874 secs 2023-09-10T01:17:21.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:21 smithi170 ceph-mon[102832]: pgmap v3937: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T01:17:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:21 smithi082 ceph-mon[98694]: pgmap v3937: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T01:17:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:21 smithi098 ceph-mon[103124]: pgmap v3937: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T01:17:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:23 smithi170 ceph-mon[102832]: pgmap v3938: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T01:17:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:17:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:23 smithi082 ceph-mon[98694]: pgmap v3938: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T01:17:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:17:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:23 smithi098 ceph-mon[103124]: pgmap v3938: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T01:17:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:17:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7075.127665 secs 2023-09-10T01:17:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7075.839939 secs 2023-09-10T01:17:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:17:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:17:24.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:17:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7075.127665 secs 2023-09-10T01:17:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7075.839939 secs 2023-09-10T01:17:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:17:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:17:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:17:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7075.127665 secs 2023-09-10T01:17:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7075.839939 secs 2023-09-10T01:17:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:17:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:17:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:17:25.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:25 smithi170 ceph-mon[102832]: pgmap v3939: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T01:17:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:25 smithi082 ceph-mon[98694]: pgmap v3939: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T01:17:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:25 smithi098 ceph-mon[103124]: pgmap v3939: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T01:17:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:27 smithi170 ceph-mon[102832]: pgmap v3940: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 147 KiB/s wr, 12 op/s 2023-09-10T01:17:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:27 smithi082 ceph-mon[98694]: pgmap v3940: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 147 KiB/s wr, 12 op/s 2023-09-10T01:17:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:27 smithi098 ceph-mon[103124]: pgmap v3940: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 147 KiB/s wr, 12 op/s 2023-09-10T01:17:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7080.127725 secs 2023-09-10T01:17:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7080.840002 secs 2023-09-10T01:17:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7080.127725 secs 2023-09-10T01:17:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7080.840002 secs 2023-09-10T01:17:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7080.127725 secs 2023-09-10T01:17:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7080.840002 secs 2023-09-10T01:17:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:29 smithi170 ceph-mon[102832]: pgmap v3941: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 134 KiB/s wr, 11 op/s 2023-09-10T01:17:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:29 smithi082 ceph-mon[98694]: pgmap v3941: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 134 KiB/s wr, 11 op/s 2023-09-10T01:17:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:29 smithi098 ceph-mon[103124]: pgmap v3941: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 134 KiB/s wr, 11 op/s 2023-09-10T01:17:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:31 smithi170 ceph-mon[102832]: pgmap v3942: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 213 KiB/s wr, 17 op/s 2023-09-10T01:17:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:31 smithi082 ceph-mon[98694]: pgmap v3942: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 213 KiB/s wr, 17 op/s 2023-09-10T01:17:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:31 smithi098 ceph-mon[103124]: pgmap v3942: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 213 KiB/s wr, 17 op/s 2023-09-10T01:17:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7083.644830 secs 2023-09-10T01:17:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:33 smithi170 ceph-mon[102832]: pgmap v3943: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T01:17:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7085.127783 secs 2023-09-10T01:17:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7085.840068 secs 2023-09-10T01:17:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7083.644830 secs 2023-09-10T01:17:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:33 smithi082 ceph-mon[98694]: pgmap v3943: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T01:17:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7085.127783 secs 2023-09-10T01:17:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7085.840068 secs 2023-09-10T01:17:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7083.644830 secs 2023-09-10T01:17:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:33 smithi098 ceph-mon[103124]: pgmap v3943: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T01:17:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7085.127783 secs 2023-09-10T01:17:34.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7085.840068 secs 2023-09-10T01:17:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:35 smithi170 ceph-mon[102832]: pgmap v3944: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T01:17:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:35 smithi082 ceph-mon[98694]: pgmap v3944: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T01:17:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:35 smithi098 ceph-mon[103124]: pgmap v3944: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T01:17:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:36 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:17:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:17:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:17:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:36 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:17:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:17:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:17:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:36 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:17:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:17:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:17:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:37 smithi170 ceph-mon[102832]: pgmap v3945: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 237 KiB/s wr, 18 op/s 2023-09-10T01:17:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:37 smithi082 ceph-mon[98694]: pgmap v3945: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 237 KiB/s wr, 18 op/s 2023-09-10T01:17:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:37 smithi098 ceph-mon[103124]: pgmap v3945: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 237 KiB/s wr, 18 op/s 2023-09-10T01:17:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7088.644892 secs 2023-09-10T01:17:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7090.127835 secs 2023-09-10T01:17:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:38 smithi170 ceph-mon[102832]: pgmap v3946: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 178 KiB/s wr, 12 op/s 2023-09-10T01:17:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7090.840133 secs 2023-09-10T01:17:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7088.644892 secs 2023-09-10T01:17:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7090.127835 secs 2023-09-10T01:17:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:38 smithi082 ceph-mon[98694]: pgmap v3946: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 178 KiB/s wr, 12 op/s 2023-09-10T01:17:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7090.840133 secs 2023-09-10T01:17:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7088.644892 secs 2023-09-10T01:17:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7090.127835 secs 2023-09-10T01:17:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:38 smithi098 ceph-mon[103124]: pgmap v3946: 97 pgs: 97 active+clean; 9.2 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 178 KiB/s wr, 12 op/s 2023-09-10T01:17:39.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7090.840133 secs 2023-09-10T01:17:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:41 smithi082 ceph-mon[98694]: pgmap v3947: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 279 KiB/s wr, 18 op/s 2023-09-10T01:17:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:41 smithi098 ceph-mon[103124]: pgmap v3947: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 279 KiB/s wr, 18 op/s 2023-09-10T01:17:41.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:41 smithi170 ceph-mon[102832]: pgmap v3947: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 279 KiB/s wr, 18 op/s 2023-09-10T01:17:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:43 smithi082 ceph-mon[98694]: pgmap v3948: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T01:17:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:17:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:43 smithi098 ceph-mon[103124]: pgmap v3948: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T01:17:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:17:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:43 smithi170 ceph-mon[102832]: pgmap v3948: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T01:17:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:17:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7095.127922 secs 2023-09-10T01:17:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:44 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:17:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:17:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:17:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7095.840196 secs 2023-09-10T01:17:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7095.127922 secs 2023-09-10T01:17:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:44 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:17:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:17:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:17:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7095.840196 secs 2023-09-10T01:17:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7095.127922 secs 2023-09-10T01:17:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:44 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:17:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:17:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:17:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7095.840196 secs 2023-09-10T01:17:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:45 smithi098 ceph-mon[103124]: pgmap v3949: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T01:17:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:45 smithi170 ceph-mon[102832]: pgmap v3949: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T01:17:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:45 smithi082 ceph-mon[98694]: pgmap v3949: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T01:17:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:47 smithi098 ceph-mon[103124]: pgmap v3950: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 18 op/s 2023-09-10T01:17:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:47 smithi170 ceph-mon[102832]: pgmap v3950: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 18 op/s 2023-09-10T01:17:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:47 smithi082 ceph-mon[98694]: pgmap v3950: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 18 op/s 2023-09-10T01:17:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7098.645051 secs 2023-09-10T01:17:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7098.645051 secs 2023-09-10T01:17:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7098.645051 secs 2023-09-10T01:17:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7100.127981 secs 2023-09-10T01:17:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:49 smithi098 ceph-mon[103124]: pgmap v3951: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 184 KiB/s wr, 12 op/s 2023-09-10T01:17:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7100.840261 secs 2023-09-10T01:17:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7100.127981 secs 2023-09-10T01:17:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:49 smithi170 ceph-mon[102832]: pgmap v3951: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 184 KiB/s wr, 12 op/s 2023-09-10T01:17:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7100.840261 secs 2023-09-10T01:17:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7100.127981 secs 2023-09-10T01:17:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:49 smithi082 ceph-mon[98694]: pgmap v3951: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 184 KiB/s wr, 12 op/s 2023-09-10T01:17:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7100.840261 secs 2023-09-10T01:17:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:51 smithi098 ceph-mon[103124]: pgmap v3952: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 221 KiB/s wr, 17 op/s 2023-09-10T01:17:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:51 smithi170 ceph-mon[102832]: pgmap v3952: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 221 KiB/s wr, 17 op/s 2023-09-10T01:17:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:51 smithi082 ceph-mon[98694]: pgmap v3952: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 221 KiB/s wr, 17 op/s 2023-09-10T01:17:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:17:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:17:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:17:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:17:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:17:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:17:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:17:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:17:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:17:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7103.645127 secs 2023-09-10T01:17:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:53 smithi098 ceph-mon[103124]: pgmap v3953: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T01:17:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7103.645127 secs 2023-09-10T01:17:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:53 smithi170 ceph-mon[102832]: pgmap v3953: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T01:17:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7103.645127 secs 2023-09-10T01:17:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:53 smithi082 ceph-mon[98694]: pgmap v3953: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T01:17:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7105.128038 secs 2023-09-10T01:17:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7105.840327 secs 2023-09-10T01:17:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7105.128038 secs 2023-09-10T01:17:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7105.840327 secs 2023-09-10T01:17:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7105.128038 secs 2023-09-10T01:17:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7105.840327 secs 2023-09-10T01:17:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:55 smithi098 ceph-mon[103124]: pgmap v3954: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T01:17:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:55 smithi170 ceph-mon[102832]: pgmap v3954: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T01:17:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:55 smithi082 ceph-mon[98694]: pgmap v3954: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T01:17:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:57 smithi098 ceph-mon[103124]: pgmap v3955: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 180 KiB/s wr, 17 op/s 2023-09-10T01:17:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:57 smithi170 ceph-mon[102832]: pgmap v3955: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 180 KiB/s wr, 17 op/s 2023-09-10T01:17:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:57 smithi082 ceph-mon[98694]: pgmap v3955: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 180 KiB/s wr, 17 op/s 2023-09-10T01:17:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7108.645187 secs 2023-09-10T01:17:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7108.645187 secs 2023-09-10T01:17:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7108.645187 secs 2023-09-10T01:17:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7110.128093 secs 2023-09-10T01:17:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:59 smithi098 ceph-mon[103124]: pgmap v3956: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 97 KiB/s wr, 11 op/s 2023-09-10T01:17:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:17:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7110.840393 secs 2023-09-10T01:17:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7110.128093 secs 2023-09-10T01:17:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:59 smithi170 ceph-mon[102832]: pgmap v3956: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 97 KiB/s wr, 11 op/s 2023-09-10T01:17:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:17:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7110.840393 secs 2023-09-10T01:17:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7110.128093 secs 2023-09-10T01:17:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:59 smithi082 ceph-mon[98694]: pgmap v3956: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 97 KiB/s wr, 11 op/s 2023-09-10T01:17:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:17:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7110.840393 secs 2023-09-10T01:18:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:01 smithi098 ceph-mon[103124]: pgmap v3957: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 16 op/s 2023-09-10T01:18:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:01 smithi170 ceph-mon[102832]: pgmap v3957: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 16 op/s 2023-09-10T01:18:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:01 smithi082 ceph-mon[98694]: pgmap v3957: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 16 op/s 2023-09-10T01:18:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7113.645277 secs 2023-09-10T01:18:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:03 smithi170 ceph-mon[102832]: pgmap v3958: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T01:18:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7113.645277 secs 2023-09-10T01:18:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:03 smithi082 ceph-mon[98694]: pgmap v3958: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T01:18:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7113.645277 secs 2023-09-10T01:18:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:03 smithi098 ceph-mon[103124]: pgmap v3958: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T01:18:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7115.128148 secs 2023-09-10T01:18:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7115.840458 secs 2023-09-10T01:18:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7115.128148 secs 2023-09-10T01:18:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7115.840458 secs 2023-09-10T01:18:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7115.128148 secs 2023-09-10T01:18:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7115.840458 secs 2023-09-10T01:18:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:05 smithi170 ceph-mon[102832]: pgmap v3959: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T01:18:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:05 smithi082 ceph-mon[98694]: pgmap v3959: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T01:18:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:05 smithi098 ceph-mon[103124]: pgmap v3959: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T01:18:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:07 smithi170 ceph-mon[102832]: pgmap v3960: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 133 KiB/s wr, 30 op/s 2023-09-10T01:18:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:18:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:07 smithi082 ceph-mon[98694]: pgmap v3960: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 133 KiB/s wr, 30 op/s 2023-09-10T01:18:07.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:18:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:07 smithi098 ceph-mon[103124]: pgmap v3960: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 133 KiB/s wr, 30 op/s 2023-09-10T01:18:08.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:18:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7118.645396 secs 2023-09-10T01:18:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:08 smithi170 ceph-mon[102832]: osdmap e317: 12 total, 12 up, 12 in 2023-09-10T01:18:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:18:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:18:08.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:18:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7118.645396 secs 2023-09-10T01:18:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:08 smithi082 ceph-mon[98694]: osdmap e317: 12 total, 12 up, 12 in 2023-09-10T01:18:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:18:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:18:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:18:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7118.645396 secs 2023-09-10T01:18:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:08 smithi098 ceph-mon[103124]: osdmap e317: 12 total, 12 up, 12 in 2023-09-10T01:18:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:18:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:18:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:18:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7120.128208 secs 2023-09-10T01:18:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:09 smithi170 ceph-mon[102832]: pgmap v3962: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T01:18:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7120.840489 secs 2023-09-10T01:18:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7120.128208 secs 2023-09-10T01:18:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:09 smithi082 ceph-mon[98694]: pgmap v3962: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T01:18:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7120.840489 secs 2023-09-10T01:18:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7120.128208 secs 2023-09-10T01:18:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:09 smithi098 ceph-mon[103124]: pgmap v3962: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T01:18:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7120.840489 secs 2023-09-10T01:18:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:11 smithi170 ceph-mon[102832]: pgmap v3963: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:18:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:11 smithi082 ceph-mon[98694]: pgmap v3963: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:18:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:11 smithi098 ceph-mon[103124]: pgmap v3963: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:18:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7123.645456 secs 2023-09-10T01:18:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:13 smithi170 ceph-mon[102832]: pgmap v3964: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:18:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7123.645456 secs 2023-09-10T01:18:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:13 smithi082 ceph-mon[98694]: pgmap v3964: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:18:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7123.645456 secs 2023-09-10T01:18:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:13 smithi098 ceph-mon[103124]: pgmap v3964: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:18:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7125.128288 secs 2023-09-10T01:18:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7125.840553 secs 2023-09-10T01:18:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7125.128288 secs 2023-09-10T01:18:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7125.840553 secs 2023-09-10T01:18:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7125.128288 secs 2023-09-10T01:18:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7125.840553 secs 2023-09-10T01:18:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:15 smithi170 ceph-mon[102832]: pgmap v3965: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:18:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:15 smithi170 ceph-mon[102832]: osdmap e318: 12 total, 12 up, 12 in 2023-09-10T01:18:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:15 smithi082 ceph-mon[98694]: pgmap v3965: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:18:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:15 smithi082 ceph-mon[98694]: osdmap e318: 12 total, 12 up, 12 in 2023-09-10T01:18:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:15 smithi098 ceph-mon[103124]: pgmap v3965: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:18:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:15 smithi098 ceph-mon[103124]: osdmap e318: 12 total, 12 up, 12 in 2023-09-10T01:18:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:17 smithi170 ceph-mon[102832]: pgmap v3967: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 131 KiB/s wr, 13 op/s 2023-09-10T01:18:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:17 smithi082 ceph-mon[98694]: pgmap v3967: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 131 KiB/s wr, 13 op/s 2023-09-10T01:18:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:17 smithi098 ceph-mon[103124]: pgmap v3967: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 131 KiB/s wr, 13 op/s 2023-09-10T01:18:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7128.645516 secs 2023-09-10T01:18:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7128.645516 secs 2023-09-10T01:18:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7128.645516 secs 2023-09-10T01:18:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7130.128397 secs 2023-09-10T01:18:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:19 smithi170 ceph-mon[102832]: pgmap v3968: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T01:18:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7130.840618 secs 2023-09-10T01:18:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7130.128397 secs 2023-09-10T01:18:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:19 smithi082 ceph-mon[98694]: pgmap v3968: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T01:18:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7130.840618 secs 2023-09-10T01:18:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7130.128397 secs 2023-09-10T01:18:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:19 smithi098 ceph-mon[103124]: pgmap v3968: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T01:18:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7130.840618 secs 2023-09-10T01:18:21.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:21 smithi170 ceph-mon[102832]: pgmap v3969: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:18:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:21 smithi082 ceph-mon[98694]: pgmap v3969: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:18:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:21 smithi098 ceph-mon[103124]: pgmap v3969: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:18:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:23 smithi170 ceph-mon[102832]: pgmap v3970: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:18:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:18:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:23 smithi082 ceph-mon[98694]: pgmap v3970: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:18:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:18:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:23 smithi098 ceph-mon[103124]: pgmap v3970: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:18:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:18:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7135.128479 secs 2023-09-10T01:18:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:18:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:18:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:18:24.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7135.840683 secs 2023-09-10T01:18:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7135.128479 secs 2023-09-10T01:18:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:18:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:18:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:18:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7135.840683 secs 2023-09-10T01:18:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7135.128479 secs 2023-09-10T01:18:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:18:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:18:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:18:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7135.840683 secs 2023-09-10T01:18:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:25 smithi170 ceph-mon[102832]: pgmap v3971: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:18:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:25 smithi082 ceph-mon[98694]: pgmap v3971: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:18:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:25 smithi098 ceph-mon[103124]: pgmap v3971: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:18:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:27 smithi170 ceph-mon[102832]: pgmap v3972: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T01:18:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:27 smithi082 ceph-mon[98694]: pgmap v3972: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T01:18:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:27 smithi098 ceph-mon[103124]: pgmap v3972: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T01:18:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7140.128567 secs 2023-09-10T01:18:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:29 smithi170 ceph-mon[102832]: pgmap v3973: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 101 KiB/s wr, 10 op/s 2023-09-10T01:18:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7140.840749 secs 2023-09-10T01:18:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7140.128567 secs 2023-09-10T01:18:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:29 smithi082 ceph-mon[98694]: pgmap v3973: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 101 KiB/s wr, 10 op/s 2023-09-10T01:18:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7140.840749 secs 2023-09-10T01:18:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7140.128567 secs 2023-09-10T01:18:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:29 smithi098 ceph-mon[103124]: pgmap v3973: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 101 KiB/s wr, 10 op/s 2023-09-10T01:18:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7140.840749 secs 2023-09-10T01:18:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:31 smithi170 ceph-mon[102832]: pgmap v3974: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 262 KiB/s wr, 17 op/s 2023-09-10T01:18:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:31 smithi082 ceph-mon[98694]: pgmap v3974: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 262 KiB/s wr, 17 op/s 2023-09-10T01:18:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:31 smithi098 ceph-mon[103124]: pgmap v3974: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 262 KiB/s wr, 17 op/s 2023-09-10T01:18:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7143.645696 secs 2023-09-10T01:18:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:33 smithi170 ceph-mon[102832]: pgmap v3975: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 220 KiB/s wr, 13 op/s 2023-09-10T01:18:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7143.645696 secs 2023-09-10T01:18:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:33 smithi082 ceph-mon[98694]: pgmap v3975: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 220 KiB/s wr, 13 op/s 2023-09-10T01:18:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7143.645696 secs 2023-09-10T01:18:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:33 smithi098 ceph-mon[103124]: pgmap v3975: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 220 KiB/s wr, 13 op/s 2023-09-10T01:18:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7145.128626 secs 2023-09-10T01:18:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7145.840816 secs 2023-09-10T01:18:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7145.128626 secs 2023-09-10T01:18:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7145.840816 secs 2023-09-10T01:18:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7145.128626 secs 2023-09-10T01:18:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7145.840816 secs 2023-09-10T01:18:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:35 smithi170 ceph-mon[102832]: pgmap v3976: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 220 KiB/s wr, 13 op/s 2023-09-10T01:18:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:35 smithi082 ceph-mon[98694]: pgmap v3976: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 220 KiB/s wr, 13 op/s 2023-09-10T01:18:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:35 smithi098 ceph-mon[103124]: pgmap v3976: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 220 KiB/s wr, 13 op/s 2023-09-10T01:18:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:36 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:18:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:18:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:18:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:36 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:18:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:18:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:18:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:36 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:18:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:18:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:18:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:37 smithi170 ceph-mon[102832]: pgmap v3977: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 238 KiB/s wr, 18 op/s 2023-09-10T01:18:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:37 smithi082 ceph-mon[98694]: pgmap v3977: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 238 KiB/s wr, 18 op/s 2023-09-10T01:18:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:37 smithi098 ceph-mon[103124]: pgmap v3977: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 238 KiB/s wr, 18 op/s 2023-09-10T01:18:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7148.645780 secs 2023-09-10T01:18:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7148.645780 secs 2023-09-10T01:18:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7148.645780 secs 2023-09-10T01:18:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7150.128685 secs 2023-09-10T01:18:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:39 smithi170 ceph-mon[102832]: pgmap v3978: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 12 op/s 2023-09-10T01:18:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7150.840880 secs 2023-09-10T01:18:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7150.128685 secs 2023-09-10T01:18:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:39 smithi082 ceph-mon[98694]: pgmap v3978: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 12 op/s 2023-09-10T01:18:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7150.840880 secs 2023-09-10T01:18:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7150.128685 secs 2023-09-10T01:18:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:39 smithi098 ceph-mon[103124]: pgmap v3978: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 12 op/s 2023-09-10T01:18:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7150.840880 secs 2023-09-10T01:18:41.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:41 smithi170 ceph-mon[102832]: pgmap v3979: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 336 KiB/s wr, 18 op/s 2023-09-10T01:18:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:41 smithi082 ceph-mon[98694]: pgmap v3979: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 336 KiB/s wr, 18 op/s 2023-09-10T01:18:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:41 smithi098 ceph-mon[103124]: pgmap v3979: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 336 KiB/s wr, 18 op/s 2023-09-10T01:18:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7153.645838 secs 2023-09-10T01:18:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:43 smithi170 ceph-mon[102832]: pgmap v3980: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 175 KiB/s wr, 11 op/s 2023-09-10T01:18:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7153.645838 secs 2023-09-10T01:18:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:43 smithi082 ceph-mon[98694]: pgmap v3980: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 175 KiB/s wr, 11 op/s 2023-09-10T01:18:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7153.645838 secs 2023-09-10T01:18:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:43 smithi098 ceph-mon[103124]: pgmap v3980: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 175 KiB/s wr, 11 op/s 2023-09-10T01:18:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7155.128744 secs 2023-09-10T01:18:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7155.840946 secs 2023-09-10T01:18:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7155.128744 secs 2023-09-10T01:18:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7155.840946 secs 2023-09-10T01:18:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7155.128744 secs 2023-09-10T01:18:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7155.840946 secs 2023-09-10T01:18:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:45 smithi170 ceph-mon[102832]: pgmap v3981: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 175 KiB/s wr, 11 op/s 2023-09-10T01:18:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:45 smithi082 ceph-mon[98694]: pgmap v3981: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 175 KiB/s wr, 11 op/s 2023-09-10T01:18:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:45 smithi098 ceph-mon[103124]: pgmap v3981: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 175 KiB/s wr, 11 op/s 2023-09-10T01:18:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:47 smithi170 ceph-mon[102832]: pgmap v3982: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 192 KiB/s wr, 16 op/s 2023-09-10T01:18:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:47 smithi082 ceph-mon[98694]: pgmap v3982: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 192 KiB/s wr, 16 op/s 2023-09-10T01:18:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:47 smithi098 ceph-mon[103124]: pgmap v3982: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 192 KiB/s wr, 16 op/s 2023-09-10T01:18:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7158.645901 secs 2023-09-10T01:18:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7158.645901 secs 2023-09-10T01:18:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7158.645901 secs 2023-09-10T01:18:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7160.128801 secs 2023-09-10T01:18:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:49 smithi170 ceph-mon[102832]: pgmap v3983: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T01:18:49.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7160.841012 secs 2023-09-10T01:18:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7160.128801 secs 2023-09-10T01:18:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:49 smithi082 ceph-mon[98694]: pgmap v3983: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T01:18:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7160.841012 secs 2023-09-10T01:18:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7160.128801 secs 2023-09-10T01:18:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:49 smithi098 ceph-mon[103124]: pgmap v3983: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T01:18:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7160.841012 secs 2023-09-10T01:18:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:51 smithi170 ceph-mon[102832]: pgmap v3984: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 217 KiB/s wr, 16 op/s 2023-09-10T01:18:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:51 smithi082 ceph-mon[98694]: pgmap v3984: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 217 KiB/s wr, 16 op/s 2023-09-10T01:18:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:51 smithi098 ceph-mon[103124]: pgmap v3984: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 217 KiB/s wr, 16 op/s 2023-09-10T01:18:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7163.645964 secs 2023-09-10T01:18:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:53 smithi170 ceph-mon[102832]: pgmap v3985: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 60 KiB/s wr, 11 op/s 2023-09-10T01:18:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7163.645964 secs 2023-09-10T01:18:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:53 smithi082 ceph-mon[98694]: pgmap v3985: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 60 KiB/s wr, 11 op/s 2023-09-10T01:18:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7163.645964 secs 2023-09-10T01:18:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:53 smithi098 ceph-mon[103124]: pgmap v3985: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 60 KiB/s wr, 11 op/s 2023-09-10T01:18:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7165.128871 secs 2023-09-10T01:18:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7165.841074 secs 2023-09-10T01:18:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7165.128871 secs 2023-09-10T01:18:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7165.841074 secs 2023-09-10T01:18:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7165.128871 secs 2023-09-10T01:18:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7165.841074 secs 2023-09-10T01:18:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:55 smithi170 ceph-mon[102832]: pgmap v3986: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 60 KiB/s wr, 11 op/s 2023-09-10T01:18:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:55 smithi082 ceph-mon[98694]: pgmap v3986: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 60 KiB/s wr, 11 op/s 2023-09-10T01:18:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:55 smithi098 ceph-mon[103124]: pgmap v3986: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 60 KiB/s wr, 11 op/s 2023-09-10T01:18:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:57 smithi170 ceph-mon[102832]: pgmap v3987: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 121 KiB/s wr, 16 op/s 2023-09-10T01:18:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:57 smithi082 ceph-mon[98694]: pgmap v3987: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 121 KiB/s wr, 16 op/s 2023-09-10T01:18:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:57 smithi098 ceph-mon[103124]: pgmap v3987: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 121 KiB/s wr, 16 op/s 2023-09-10T01:18:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7168.646021 secs 2023-09-10T01:18:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7168.646021 secs 2023-09-10T01:18:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7168.646021 secs 2023-09-10T01:18:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7170.128953 secs 2023-09-10T01:18:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:59 smithi170 ceph-mon[102832]: pgmap v3988: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T01:18:59.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:18:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7170.841142 secs 2023-09-10T01:18:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7170.128953 secs 2023-09-10T01:18:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:59 smithi082 ceph-mon[98694]: pgmap v3988: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T01:18:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:18:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7170.841142 secs 2023-09-10T01:19:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7170.128953 secs 2023-09-10T01:19:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:59 smithi098 ceph-mon[103124]: pgmap v3988: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T01:19:00.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:18:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7170.841142 secs 2023-09-10T01:19:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:01 smithi170 ceph-mon[102832]: pgmap v3989: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 130 KiB/s wr, 16 op/s 2023-09-10T01:19:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:01 smithi082 ceph-mon[98694]: pgmap v3989: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 130 KiB/s wr, 16 op/s 2023-09-10T01:19:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:01 smithi098 ceph-mon[103124]: pgmap v3989: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 130 KiB/s wr, 16 op/s 2023-09-10T01:19:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7173.646126 secs 2023-09-10T01:19:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:03 smithi170 ceph-mon[102832]: pgmap v3990: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T01:19:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7175.129010 secs 2023-09-10T01:19:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7173.646126 secs 2023-09-10T01:19:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:03 smithi082 ceph-mon[98694]: pgmap v3990: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T01:19:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7175.129010 secs 2023-09-10T01:19:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7173.646126 secs 2023-09-10T01:19:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:03 smithi098 ceph-mon[103124]: pgmap v3990: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T01:19:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7175.129010 secs 2023-09-10T01:19:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7175.841206 secs 2023-09-10T01:19:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7175.841206 secs 2023-09-10T01:19:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7175.841206 secs 2023-09-10T01:19:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:05 smithi170 ceph-mon[102832]: pgmap v3991: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T01:19:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:05 smithi082 ceph-mon[98694]: pgmap v3991: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T01:19:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:05 smithi098 ceph-mon[103124]: pgmap v3991: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T01:19:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:07 smithi170 ceph-mon[102832]: pgmap v3992: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 133 KiB/s wr, 30 op/s 2023-09-10T01:19:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:07 smithi082 ceph-mon[98694]: pgmap v3992: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 133 KiB/s wr, 30 op/s 2023-09-10T01:19:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:07 smithi098 ceph-mon[103124]: pgmap v3992: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 133 KiB/s wr, 30 op/s 2023-09-10T01:19:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7178.646235 secs 2023-09-10T01:19:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:08 smithi170 ceph-mon[102832]: osdmap e319: 12 total, 12 up, 12 in 2023-09-10T01:19:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7180.129096 secs 2023-09-10T01:19:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:19:08.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7180.841274 secs 2023-09-10T01:19:08.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:19:08.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:19:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7178.646235 secs 2023-09-10T01:19:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:08 smithi082 ceph-mon[98694]: osdmap e319: 12 total, 12 up, 12 in 2023-09-10T01:19:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7180.129096 secs 2023-09-10T01:19:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:19:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7180.841274 secs 2023-09-10T01:19:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:19:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:19:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7178.646235 secs 2023-09-10T01:19:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:08 smithi098 ceph-mon[103124]: osdmap e319: 12 total, 12 up, 12 in 2023-09-10T01:19:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7180.129096 secs 2023-09-10T01:19:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:19:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7180.841274 secs 2023-09-10T01:19:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:19:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:19:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:09 smithi170 ceph-mon[102832]: pgmap v3994: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 86 KiB/s wr, 30 op/s 2023-09-10T01:19:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:19:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:19:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:19:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:09 smithi082 ceph-mon[98694]: pgmap v3994: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 86 KiB/s wr, 30 op/s 2023-09-10T01:19:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:19:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:19:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:19:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:09 smithi098 ceph-mon[103124]: pgmap v3994: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 86 KiB/s wr, 30 op/s 2023-09-10T01:19:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:19:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:19:10.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:19:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:11 smithi170 ceph-mon[102832]: pgmap v3995: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:19:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:11 smithi082 ceph-mon[98694]: pgmap v3995: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:19:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:11 smithi098 ceph-mon[103124]: pgmap v3995: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:19:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7183.646336 secs 2023-09-10T01:19:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:13 smithi170 ceph-mon[102832]: pgmap v3996: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:19:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7185.129182 secs 2023-09-10T01:19:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7185.841338 secs 2023-09-10T01:19:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7183.646336 secs 2023-09-10T01:19:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:13 smithi082 ceph-mon[98694]: pgmap v3996: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:19:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7185.129182 secs 2023-09-10T01:19:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7185.841338 secs 2023-09-10T01:19:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7183.646336 secs 2023-09-10T01:19:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:13 smithi098 ceph-mon[103124]: pgmap v3996: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:19:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7185.129182 secs 2023-09-10T01:19:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7185.841338 secs 2023-09-10T01:19:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:15 smithi170 ceph-mon[102832]: pgmap v3997: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:19:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:15 smithi170 ceph-mon[102832]: osdmap e320: 12 total, 12 up, 12 in 2023-09-10T01:19:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:15 smithi082 ceph-mon[98694]: pgmap v3997: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:19:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:15 smithi082 ceph-mon[98694]: osdmap e320: 12 total, 12 up, 12 in 2023-09-10T01:19:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:15 smithi098 ceph-mon[103124]: pgmap v3997: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:19:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:15 smithi098 ceph-mon[103124]: osdmap e320: 12 total, 12 up, 12 in 2023-09-10T01:19:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:17 smithi170 ceph-mon[102832]: pgmap v3999: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 133 KiB/s wr, 13 op/s 2023-09-10T01:19:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:17 smithi082 ceph-mon[98694]: pgmap v3999: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 133 KiB/s wr, 13 op/s 2023-09-10T01:19:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:17 smithi098 ceph-mon[103124]: pgmap v3999: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 133 KiB/s wr, 13 op/s 2023-09-10T01:19:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7188.646451 secs 2023-09-10T01:19:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7190.129286 secs 2023-09-10T01:19:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7190.841402 secs 2023-09-10T01:19:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7188.646451 secs 2023-09-10T01:19:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7190.129286 secs 2023-09-10T01:19:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7190.841402 secs 2023-09-10T01:19:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7188.646451 secs 2023-09-10T01:19:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7190.129286 secs 2023-09-10T01:19:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7190.841402 secs 2023-09-10T01:19:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:19 smithi082 ceph-mon[98694]: pgmap v4000: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T01:19:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:19 smithi098 ceph-mon[103124]: pgmap v4000: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T01:19:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:19 smithi170 ceph-mon[102832]: pgmap v4000: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T01:19:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:21 smithi170 ceph-mon[102832]: pgmap v4001: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:19:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:21 smithi082 ceph-mon[98694]: pgmap v4001: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:19:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:21 smithi098 ceph-mon[103124]: pgmap v4001: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:19:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:23 smithi082 ceph-mon[98694]: pgmap v4002: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:19:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:19:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:23 smithi098 ceph-mon[103124]: pgmap v4002: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:19:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:19:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:23 smithi170 ceph-mon[102832]: pgmap v4002: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:19:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:19:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7195.129343 secs 2023-09-10T01:19:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:19:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:19:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:19:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7195.841473 secs 2023-09-10T01:19:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7195.129343 secs 2023-09-10T01:19:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:19:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:19:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:19:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7195.841473 secs 2023-09-10T01:19:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7195.129343 secs 2023-09-10T01:19:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:19:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:19:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:19:24.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7195.841473 secs 2023-09-10T01:19:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:25 smithi082 ceph-mon[98694]: pgmap v4003: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:19:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:25 smithi098 ceph-mon[103124]: pgmap v4003: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:19:25.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:25 smithi170 ceph-mon[102832]: pgmap v4003: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:19:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:27 smithi082 ceph-mon[98694]: pgmap v4004: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T01:19:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:27 smithi098 ceph-mon[103124]: pgmap v4004: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T01:19:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:27 smithi170 ceph-mon[102832]: pgmap v4004: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T01:19:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7200.129406 secs 2023-09-10T01:19:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:29 smithi082 ceph-mon[98694]: pgmap v4005: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 136 KiB/s wr, 10 op/s 2023-09-10T01:19:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7200.841486 secs 2023-09-10T01:19:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7200.129406 secs 2023-09-10T01:19:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:29 smithi098 ceph-mon[103124]: pgmap v4005: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 136 KiB/s wr, 10 op/s 2023-09-10T01:19:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7200.841486 secs 2023-09-10T01:19:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7200.129406 secs 2023-09-10T01:19:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:29 smithi170 ceph-mon[102832]: pgmap v4005: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 136 KiB/s wr, 10 op/s 2023-09-10T01:19:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7200.841486 secs 2023-09-10T01:19:31.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:31 smithi082 ceph-mon[98694]: pgmap v4006: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 182 KiB/s wr, 17 op/s 2023-09-10T01:19:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:31 smithi098 ceph-mon[103124]: pgmap v4006: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 182 KiB/s wr, 17 op/s 2023-09-10T01:19:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:31 smithi170 ceph-mon[102832]: pgmap v4006: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 182 KiB/s wr, 17 op/s 2023-09-10T01:19:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:33 smithi082 ceph-mon[98694]: pgmap v4007: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 139 KiB/s wr, 12 op/s 2023-09-10T01:19:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:33 smithi098 ceph-mon[103124]: pgmap v4007: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 139 KiB/s wr, 12 op/s 2023-09-10T01:19:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:33 smithi170 ceph-mon[102832]: pgmap v4007: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 139 KiB/s wr, 12 op/s 2023-09-10T01:19:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7205.129466 secs 2023-09-10T01:19:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7205.841550 secs 2023-09-10T01:19:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7205.129466 secs 2023-09-10T01:19:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7205.841550 secs 2023-09-10T01:19:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7205.129466 secs 2023-09-10T01:19:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7205.841550 secs 2023-09-10T01:19:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:35 smithi098 ceph-mon[103124]: pgmap v4008: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 139 KiB/s wr, 12 op/s 2023-09-10T01:19:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:35 smithi170 ceph-mon[102832]: pgmap v4008: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 139 KiB/s wr, 12 op/s 2023-09-10T01:19:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:35 smithi082 ceph-mon[98694]: pgmap v4008: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 139 KiB/s wr, 12 op/s 2023-09-10T01:19:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:37 smithi098 ceph-mon[103124]: pgmap v4009: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 239 KiB/s wr, 18 op/s 2023-09-10T01:19:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:37 smithi170 ceph-mon[102832]: pgmap v4009: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 239 KiB/s wr, 18 op/s 2023-09-10T01:19:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:37 smithi082 ceph-mon[98694]: pgmap v4009: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 239 KiB/s wr, 18 op/s 2023-09-10T01:19:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7208.646715 secs 2023-09-10T01:19:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7208.646715 secs 2023-09-10T01:19:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7208.646715 secs 2023-09-10T01:19:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7210.129527 secs 2023-09-10T01:19:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:39 smithi098 ceph-mon[103124]: pgmap v4010: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T01:19:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7210.841616 secs 2023-09-10T01:19:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7210.129527 secs 2023-09-10T01:19:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:39 smithi170 ceph-mon[102832]: pgmap v4010: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T01:19:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7210.841616 secs 2023-09-10T01:19:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7210.129527 secs 2023-09-10T01:19:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:39 smithi082 ceph-mon[98694]: pgmap v4010: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T01:19:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7210.841616 secs 2023-09-10T01:19:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:19:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:19:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:19:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:19:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:19:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:19:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:19:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:19:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:19:41.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:41 smithi098 ceph-mon[103124]: pgmap v4011: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 226 KiB/s wr, 18 op/s 2023-09-10T01:19:41.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:41 smithi170 ceph-mon[102832]: pgmap v4011: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 226 KiB/s wr, 18 op/s 2023-09-10T01:19:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:41 smithi082 ceph-mon[98694]: pgmap v4011: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 226 KiB/s wr, 18 op/s 2023-09-10T01:19:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7213.646824 secs 2023-09-10T01:19:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:43 smithi098 ceph-mon[103124]: pgmap v4012: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 180 KiB/s wr, 11 op/s 2023-09-10T01:19:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7213.646824 secs 2023-09-10T01:19:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:43 smithi170 ceph-mon[102832]: pgmap v4012: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 180 KiB/s wr, 11 op/s 2023-09-10T01:19:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7213.646824 secs 2023-09-10T01:19:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:43 smithi082 ceph-mon[98694]: pgmap v4012: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 180 KiB/s wr, 11 op/s 2023-09-10T01:19:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7215.129585 secs 2023-09-10T01:19:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7215.841691 secs 2023-09-10T01:19:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7215.129585 secs 2023-09-10T01:19:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7215.841691 secs 2023-09-10T01:19:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7215.129585 secs 2023-09-10T01:19:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7215.841691 secs 2023-09-10T01:19:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:45 smithi098 ceph-mon[103124]: pgmap v4013: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 180 KiB/s wr, 11 op/s 2023-09-10T01:19:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:45 smithi170 ceph-mon[102832]: pgmap v4013: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 180 KiB/s wr, 11 op/s 2023-09-10T01:19:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:45 smithi082 ceph-mon[98694]: pgmap v4013: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 180 KiB/s wr, 11 op/s 2023-09-10T01:19:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:47 smithi098 ceph-mon[103124]: pgmap v4014: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 339 KiB/s wr, 18 op/s 2023-09-10T01:19:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:47 smithi170 ceph-mon[102832]: pgmap v4014: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 339 KiB/s wr, 18 op/s 2023-09-10T01:19:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:47 smithi082 ceph-mon[98694]: pgmap v4014: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 339 KiB/s wr, 18 op/s 2023-09-10T01:19:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7218.646885 secs 2023-09-10T01:19:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7218.646885 secs 2023-09-10T01:19:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7218.646885 secs 2023-09-10T01:19:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7220.129643 secs 2023-09-10T01:19:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:49 smithi098 ceph-mon[103124]: pgmap v4015: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 239 KiB/s wr, 11 op/s 2023-09-10T01:19:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7220.841760 secs 2023-09-10T01:19:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7220.129643 secs 2023-09-10T01:19:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:49 smithi170 ceph-mon[102832]: pgmap v4015: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 239 KiB/s wr, 11 op/s 2023-09-10T01:19:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7220.841760 secs 2023-09-10T01:19:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7220.129643 secs 2023-09-10T01:19:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:49 smithi082 ceph-mon[98694]: pgmap v4015: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 239 KiB/s wr, 11 op/s 2023-09-10T01:19:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7220.841760 secs 2023-09-10T01:19:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:51 smithi098 ceph-mon[103124]: pgmap v4016: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 276 KiB/s wr, 17 op/s 2023-09-10T01:19:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:51 smithi170 ceph-mon[102832]: pgmap v4016: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 276 KiB/s wr, 17 op/s 2023-09-10T01:19:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:51 smithi082 ceph-mon[98694]: pgmap v4016: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 276 KiB/s wr, 17 op/s 2023-09-10T01:19:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7223.646949 secs 2023-09-10T01:19:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:53 smithi170 ceph-mon[102832]: pgmap v4017: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 196 KiB/s wr, 12 op/s 2023-09-10T01:19:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7223.646949 secs 2023-09-10T01:19:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:53 smithi082 ceph-mon[98694]: pgmap v4017: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 196 KiB/s wr, 12 op/s 2023-09-10T01:19:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7223.646949 secs 2023-09-10T01:19:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:53 smithi098 ceph-mon[103124]: pgmap v4017: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 196 KiB/s wr, 12 op/s 2023-09-10T01:19:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7225.129700 secs 2023-09-10T01:19:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7225.841827 secs 2023-09-10T01:19:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7225.129700 secs 2023-09-10T01:19:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7225.841827 secs 2023-09-10T01:19:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7225.129700 secs 2023-09-10T01:19:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7225.841827 secs 2023-09-10T01:19:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:55 smithi170 ceph-mon[102832]: pgmap v4018: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 196 KiB/s wr, 12 op/s 2023-09-10T01:19:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:55 smithi082 ceph-mon[98694]: pgmap v4018: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 196 KiB/s wr, 12 op/s 2023-09-10T01:19:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:55 smithi098 ceph-mon[103124]: pgmap v4018: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 196 KiB/s wr, 12 op/s 2023-09-10T01:19:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:57 smithi170 ceph-mon[102832]: pgmap v4019: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 252 KiB/s wr, 17 op/s 2023-09-10T01:19:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:57 smithi082 ceph-mon[98694]: pgmap v4019: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 252 KiB/s wr, 17 op/s 2023-09-10T01:19:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:57 smithi098 ceph-mon[103124]: pgmap v4019: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 252 KiB/s wr, 17 op/s 2023-09-10T01:19:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7228.647010 secs 2023-09-10T01:19:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7228.647010 secs 2023-09-10T01:19:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7228.647010 secs 2023-09-10T01:19:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7230.129782 secs 2023-09-10T01:19:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:59 smithi170 ceph-mon[102832]: pgmap v4020: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T01:19:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:19:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7230.841892 secs 2023-09-10T01:19:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7230.129782 secs 2023-09-10T01:19:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:59 smithi082 ceph-mon[98694]: pgmap v4020: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T01:19:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:19:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7230.841892 secs 2023-09-10T01:20:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7230.129782 secs 2023-09-10T01:20:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:59 smithi098 ceph-mon[103124]: pgmap v4020: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T01:20:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:19:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7230.841892 secs 2023-09-10T01:20:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T01:20:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T01:20:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T01:20:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:01 smithi170 ceph-mon[102832]: pgmap v4021: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 16 op/s 2023-09-10T01:20:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:01 smithi082 ceph-mon[98694]: pgmap v4021: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 16 op/s 2023-09-10T01:20:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:01 smithi098 ceph-mon[103124]: pgmap v4021: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 16 op/s 2023-09-10T01:20:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7233.647069 secs 2023-09-10T01:20:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:03 smithi170 ceph-mon[102832]: pgmap v4022: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-09-10T01:20:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7233.647069 secs 2023-09-10T01:20:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:03 smithi082 ceph-mon[98694]: pgmap v4022: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-09-10T01:20:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7233.647069 secs 2023-09-10T01:20:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:03 smithi098 ceph-mon[103124]: pgmap v4022: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-09-10T01:20:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7235.129861 secs 2023-09-10T01:20:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7235.841959 secs 2023-09-10T01:20:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7235.129861 secs 2023-09-10T01:20:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7235.841959 secs 2023-09-10T01:20:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7235.129861 secs 2023-09-10T01:20:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7235.841959 secs 2023-09-10T01:20:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:05 smithi170 ceph-mon[102832]: pgmap v4023: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-09-10T01:20:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:05 smithi082 ceph-mon[98694]: pgmap v4023: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-09-10T01:20:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:05 smithi098 ceph-mon[103124]: pgmap v4023: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-09-10T01:20:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:07 smithi170 ceph-mon[102832]: pgmap v4024: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:20:07.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:07 smithi082 ceph-mon[98694]: pgmap v4024: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:20:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:07 smithi098 ceph-mon[103124]: pgmap v4024: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:20:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7238.647139 secs 2023-09-10T01:20:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:08 smithi170 ceph-mon[102832]: osdmap e321: 12 total, 12 up, 12 in 2023-09-10T01:20:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7238.647139 secs 2023-09-10T01:20:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:08 smithi082 ceph-mon[98694]: osdmap e321: 12 total, 12 up, 12 in 2023-09-10T01:20:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7238.647139 secs 2023-09-10T01:20:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:08 smithi098 ceph-mon[103124]: osdmap e321: 12 total, 12 up, 12 in 2023-09-10T01:20:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7240.129921 secs 2023-09-10T01:20:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:09 smithi082 ceph-mon[98694]: pgmap v4026: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:20:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7240.842027 secs 2023-09-10T01:20:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:20:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7240.129921 secs 2023-09-10T01:20:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:09 smithi170 ceph-mon[102832]: pgmap v4026: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:20:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7240.842027 secs 2023-09-10T01:20:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:20:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7240.129921 secs 2023-09-10T01:20:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:09 smithi098 ceph-mon[103124]: pgmap v4026: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:20:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7240.842027 secs 2023-09-10T01:20:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:20:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:20:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:20:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:20:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:20:10.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:20:10.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:20:10.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:20:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:20:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:20:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:20:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:20:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:20:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:20:10.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:20:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:20:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:20:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:20:11.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:20:11.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:20:11.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:20:11.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:20:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:11 smithi170 ceph-mon[102832]: pgmap v4027: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 129 KiB/s wr, 29 op/s 2023-09-10T01:20:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:11 smithi082 ceph-mon[98694]: pgmap v4027: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 129 KiB/s wr, 29 op/s 2023-09-10T01:20:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:11 smithi098 ceph-mon[103124]: pgmap v4027: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 129 KiB/s wr, 29 op/s 2023-09-10T01:20:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7243.647201 secs 2023-09-10T01:20:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:13 smithi170 ceph-mon[102832]: pgmap v4028: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 129 KiB/s wr, 29 op/s 2023-09-10T01:20:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7243.647201 secs 2023-09-10T01:20:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:13 smithi082 ceph-mon[98694]: pgmap v4028: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 129 KiB/s wr, 29 op/s 2023-09-10T01:20:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7243.647201 secs 2023-09-10T01:20:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:13 smithi098 ceph-mon[103124]: pgmap v4028: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 129 KiB/s wr, 29 op/s 2023-09-10T01:20:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7245.129982 secs 2023-09-10T01:20:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7245.842094 secs 2023-09-10T01:20:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7245.129982 secs 2023-09-10T01:20:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7245.842094 secs 2023-09-10T01:20:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7245.129982 secs 2023-09-10T01:20:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7245.842094 secs 2023-09-10T01:20:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:15 smithi170 ceph-mon[102832]: pgmap v4029: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 129 KiB/s wr, 29 op/s 2023-09-10T01:20:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:15 smithi170 ceph-mon[102832]: osdmap e322: 12 total, 12 up, 12 in 2023-09-10T01:20:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:15 smithi082 ceph-mon[98694]: pgmap v4029: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 129 KiB/s wr, 29 op/s 2023-09-10T01:20:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:15 smithi082 ceph-mon[98694]: osdmap e322: 12 total, 12 up, 12 in 2023-09-10T01:20:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:15 smithi098 ceph-mon[103124]: pgmap v4029: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 129 KiB/s wr, 29 op/s 2023-09-10T01:20:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:15 smithi098 ceph-mon[103124]: osdmap e322: 12 total, 12 up, 12 in 2023-09-10T01:20:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:17 smithi170 ceph-mon[102832]: pgmap v4031: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 133 KiB/s wr, 13 op/s 2023-09-10T01:20:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:17 smithi082 ceph-mon[98694]: pgmap v4031: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 133 KiB/s wr, 13 op/s 2023-09-10T01:20:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:17 smithi098 ceph-mon[103124]: pgmap v4031: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 133 KiB/s wr, 13 op/s 2023-09-10T01:20:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7248.647263 secs 2023-09-10T01:20:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7248.647263 secs 2023-09-10T01:20:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7248.647263 secs 2023-09-10T01:20:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7250.130073 secs 2023-09-10T01:20:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:19 smithi170 ceph-mon[102832]: pgmap v4032: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T01:20:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7250.842157 secs 2023-09-10T01:20:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7250.130073 secs 2023-09-10T01:20:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:19 smithi082 ceph-mon[98694]: pgmap v4032: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T01:20:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7250.842157 secs 2023-09-10T01:20:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7250.130073 secs 2023-09-10T01:20:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:19 smithi098 ceph-mon[103124]: pgmap v4032: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T01:20:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7250.842157 secs 2023-09-10T01:20:21.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:21 smithi170 ceph-mon[102832]: pgmap v4033: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T01:20:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:21 smithi082 ceph-mon[98694]: pgmap v4033: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T01:20:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:21 smithi098 ceph-mon[103124]: pgmap v4033: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T01:20:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:23 smithi170 ceph-mon[102832]: pgmap v4034: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T01:20:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:20:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7255.130159 secs 2023-09-10T01:20:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7255.842223 secs 2023-09-10T01:20:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:23 smithi082 ceph-mon[98694]: pgmap v4034: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T01:20:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:20:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7255.130159 secs 2023-09-10T01:20:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7255.842223 secs 2023-09-10T01:20:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:23 smithi098 ceph-mon[103124]: pgmap v4034: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T01:20:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:20:24.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7255.130159 secs 2023-09-10T01:20:24.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7255.842223 secs 2023-09-10T01:20:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:20:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:20:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:20:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:20:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:20:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:20:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:20:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:20:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:20:25.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:25 smithi170 ceph-mon[102832]: pgmap v4035: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T01:20:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:25 smithi082 ceph-mon[98694]: pgmap v4035: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T01:20:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:25 smithi098 ceph-mon[103124]: pgmap v4035: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T01:20:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:27 smithi170 ceph-mon[102832]: pgmap v4036: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 130 KiB/s wr, 11 op/s 2023-09-10T01:20:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:27 smithi082 ceph-mon[98694]: pgmap v4036: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 130 KiB/s wr, 11 op/s 2023-09-10T01:20:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:27 smithi098 ceph-mon[103124]: pgmap v4036: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 130 KiB/s wr, 11 op/s 2023-09-10T01:20:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7260.130218 secs 2023-09-10T01:20:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7260.842289 secs 2023-09-10T01:20:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7260.130218 secs 2023-09-10T01:20:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7260.842289 secs 2023-09-10T01:20:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7260.130218 secs 2023-09-10T01:20:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7260.842289 secs 2023-09-10T01:20:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:29 smithi082 ceph-mon[98694]: pgmap v4037: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T01:20:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:29 smithi098 ceph-mon[103124]: pgmap v4037: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T01:20:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:29 smithi170 ceph-mon[102832]: pgmap v4037: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T01:20:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:31 smithi082 ceph-mon[98694]: pgmap v4038: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 236 KiB/s wr, 17 op/s 2023-09-10T01:20:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:31 smithi098 ceph-mon[103124]: pgmap v4038: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 236 KiB/s wr, 17 op/s 2023-09-10T01:20:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:31 smithi170 ceph-mon[102832]: pgmap v4038: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 236 KiB/s wr, 17 op/s 2023-09-10T01:20:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:33 smithi082 ceph-mon[98694]: pgmap v4039: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 193 KiB/s wr, 12 op/s 2023-09-10T01:20:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7265.130301 secs 2023-09-10T01:20:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7265.842357 secs 2023-09-10T01:20:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:33 smithi098 ceph-mon[103124]: pgmap v4039: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 193 KiB/s wr, 12 op/s 2023-09-10T01:20:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7265.130301 secs 2023-09-10T01:20:34.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7265.842357 secs 2023-09-10T01:20:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:33 smithi170 ceph-mon[102832]: pgmap v4039: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 193 KiB/s wr, 12 op/s 2023-09-10T01:20:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7265.130301 secs 2023-09-10T01:20:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7265.842357 secs 2023-09-10T01:20:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:35 smithi082 ceph-mon[98694]: pgmap v4040: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 193 KiB/s wr, 12 op/s 2023-09-10T01:20:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:35 smithi098 ceph-mon[103124]: pgmap v4040: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 193 KiB/s wr, 12 op/s 2023-09-10T01:20:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:35 smithi170 ceph-mon[102832]: pgmap v4040: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 193 KiB/s wr, 12 op/s 2023-09-10T01:20:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:37 smithi082 ceph-mon[98694]: pgmap v4041: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 241 KiB/s wr, 18 op/s 2023-09-10T01:20:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:37 smithi098 ceph-mon[103124]: pgmap v4041: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 241 KiB/s wr, 18 op/s 2023-09-10T01:20:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:37 smithi170 ceph-mon[102832]: pgmap v4041: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 241 KiB/s wr, 18 op/s 2023-09-10T01:20:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7270.130386 secs 2023-09-10T01:20:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7270.842421 secs 2023-09-10T01:20:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7270.130386 secs 2023-09-10T01:20:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7270.842421 secs 2023-09-10T01:20:39.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7270.130386 secs 2023-09-10T01:20:39.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7270.842421 secs 2023-09-10T01:20:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:39 smithi082 ceph-mon[98694]: pgmap v4042: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 166 KiB/s wr, 12 op/s 2023-09-10T01:20:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:39 smithi098 ceph-mon[103124]: pgmap v4042: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 166 KiB/s wr, 12 op/s 2023-09-10T01:20:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:39 smithi170 ceph-mon[102832]: pgmap v4042: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 166 KiB/s wr, 12 op/s 2023-09-10T01:20:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:41 smithi082 ceph-mon[98694]: pgmap v4043: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 310 KiB/s wr, 18 op/s 2023-09-10T01:20:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:41 smithi098 ceph-mon[103124]: pgmap v4043: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 310 KiB/s wr, 18 op/s 2023-09-10T01:20:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:41 smithi170 ceph-mon[102832]: pgmap v4043: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 310 KiB/s wr, 18 op/s 2023-09-10T01:20:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7273.647625 secs 2023-09-10T01:20:43.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:43 smithi082 ceph-mon[98694]: pgmap v4044: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 192 KiB/s wr, 11 op/s 2023-09-10T01:20:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7273.647625 secs 2023-09-10T01:20:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:43 smithi098 ceph-mon[103124]: pgmap v4044: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 192 KiB/s wr, 11 op/s 2023-09-10T01:20:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7273.647625 secs 2023-09-10T01:20:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:43 smithi170 ceph-mon[102832]: pgmap v4044: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 192 KiB/s wr, 11 op/s 2023-09-10T01:20:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7275.130445 secs 2023-09-10T01:20:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:20:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:20:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:20:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7275.842487 secs 2023-09-10T01:20:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7275.130445 secs 2023-09-10T01:20:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:20:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:20:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:20:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7275.842487 secs 2023-09-10T01:20:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7275.130445 secs 2023-09-10T01:20:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:20:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:20:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:20:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7275.842487 secs 2023-09-10T01:20:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:45 smithi098 ceph-mon[103124]: pgmap v4045: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 192 KiB/s wr, 11 op/s 2023-09-10T01:20:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:45 smithi170 ceph-mon[102832]: pgmap v4045: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 192 KiB/s wr, 11 op/s 2023-09-10T01:20:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:45 smithi082 ceph-mon[98694]: pgmap v4045: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 192 KiB/s wr, 11 op/s 2023-09-10T01:20:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:47 smithi098 ceph-mon[103124]: pgmap v4046: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 225 KiB/s wr, 16 op/s 2023-09-10T01:20:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:47 smithi170 ceph-mon[102832]: pgmap v4046: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 225 KiB/s wr, 16 op/s 2023-09-10T01:20:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:47 smithi082 ceph-mon[98694]: pgmap v4046: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 225 KiB/s wr, 16 op/s 2023-09-10T01:20:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7278.647684 secs 2023-09-10T01:20:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7278.647684 secs 2023-09-10T01:20:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7278.647684 secs 2023-09-10T01:20:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7280.130538 secs 2023-09-10T01:20:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:49 smithi098 ceph-mon[103124]: pgmap v4047: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 10 op/s 2023-09-10T01:20:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7280.842557 secs 2023-09-10T01:20:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7280.130538 secs 2023-09-10T01:20:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:49 smithi170 ceph-mon[102832]: pgmap v4047: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 10 op/s 2023-09-10T01:20:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7280.842557 secs 2023-09-10T01:20:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7280.130538 secs 2023-09-10T01:20:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:49 smithi082 ceph-mon[98694]: pgmap v4047: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 10 op/s 2023-09-10T01:20:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7280.842557 secs 2023-09-10T01:20:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:51 smithi098 ceph-mon[103124]: pgmap v4048: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 220 KiB/s wr, 16 op/s 2023-09-10T01:20:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:51 smithi170 ceph-mon[102832]: pgmap v4048: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 220 KiB/s wr, 16 op/s 2023-09-10T01:20:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:51 smithi082 ceph-mon[98694]: pgmap v4048: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 220 KiB/s wr, 16 op/s 2023-09-10T01:20:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7283.647789 secs 2023-09-10T01:20:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:53 smithi098 ceph-mon[103124]: pgmap v4049: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T01:20:53.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7283.647789 secs 2023-09-10T01:20:53.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:53 smithi170 ceph-mon[102832]: pgmap v4049: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T01:20:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7283.647789 secs 2023-09-10T01:20:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:53 smithi082 ceph-mon[98694]: pgmap v4049: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T01:20:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7285.130596 secs 2023-09-10T01:20:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7285.842624 secs 2023-09-10T01:20:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7285.130596 secs 2023-09-10T01:20:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7285.842624 secs 2023-09-10T01:20:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7285.130596 secs 2023-09-10T01:20:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7285.842624 secs 2023-09-10T01:20:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:55 smithi098 ceph-mon[103124]: pgmap v4050: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T01:20:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:55 smithi170 ceph-mon[102832]: pgmap v4050: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T01:20:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:55 smithi082 ceph-mon[98694]: pgmap v4050: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T01:20:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:57 smithi098 ceph-mon[103124]: pgmap v4051: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 138 KiB/s wr, 17 op/s 2023-09-10T01:20:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:57 smithi170 ceph-mon[102832]: pgmap v4051: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 138 KiB/s wr, 17 op/s 2023-09-10T01:20:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:57 smithi082 ceph-mon[98694]: pgmap v4051: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 138 KiB/s wr, 17 op/s 2023-09-10T01:20:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7288.647916 secs 2023-09-10T01:20:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7288.647916 secs 2023-09-10T01:20:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7288.647916 secs 2023-09-10T01:20:59.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7290.130673 secs 2023-09-10T01:20:59.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:59 smithi098 ceph-mon[103124]: pgmap v4052: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T01:20:59.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:20:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7290.842693 secs 2023-09-10T01:20:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7290.130673 secs 2023-09-10T01:20:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:59 smithi170 ceph-mon[102832]: pgmap v4052: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T01:20:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:20:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7290.842693 secs 2023-09-10T01:20:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7290.130673 secs 2023-09-10T01:20:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:59 smithi082 ceph-mon[98694]: pgmap v4052: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T01:20:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:20:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7290.842693 secs 2023-09-10T01:21:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:01 smithi098 ceph-mon[103124]: pgmap v4053: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 132 KiB/s wr, 17 op/s 2023-09-10T01:21:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:01 smithi170 ceph-mon[102832]: pgmap v4053: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 132 KiB/s wr, 17 op/s 2023-09-10T01:21:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:01 smithi082 ceph-mon[98694]: pgmap v4053: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 132 KiB/s wr, 17 op/s 2023-09-10T01:21:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7293.647996 secs 2023-09-10T01:21:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:03 smithi098 ceph-mon[103124]: pgmap v4054: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T01:21:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7293.647996 secs 2023-09-10T01:21:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:03 smithi170 ceph-mon[102832]: pgmap v4054: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T01:21:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7293.647996 secs 2023-09-10T01:21:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:03 smithi082 ceph-mon[98694]: pgmap v4054: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T01:21:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7295.130728 secs 2023-09-10T01:21:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7295.842758 secs 2023-09-10T01:21:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7295.130728 secs 2023-09-10T01:21:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7295.842758 secs 2023-09-10T01:21:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7295.130728 secs 2023-09-10T01:21:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7295.842758 secs 2023-09-10T01:21:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:05 smithi170 ceph-mon[102832]: pgmap v4055: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T01:21:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:05 smithi082 ceph-mon[98694]: pgmap v4055: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T01:21:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:05 smithi098 ceph-mon[103124]: pgmap v4055: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T01:21:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:07 smithi170 ceph-mon[102832]: pgmap v4056: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 134 KiB/s wr, 30 op/s 2023-09-10T01:21:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:07 smithi082 ceph-mon[98694]: pgmap v4056: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 134 KiB/s wr, 30 op/s 2023-09-10T01:21:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:07 smithi098 ceph-mon[103124]: pgmap v4056: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 134 KiB/s wr, 30 op/s 2023-09-10T01:21:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7298.648147 secs 2023-09-10T01:21:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:08 smithi170 ceph-mon[102832]: osdmap e323: 12 total, 12 up, 12 in 2023-09-10T01:21:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7298.648147 secs 2023-09-10T01:21:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:08 smithi082 ceph-mon[98694]: osdmap e323: 12 total, 12 up, 12 in 2023-09-10T01:21:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7298.648147 secs 2023-09-10T01:21:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:08 smithi098 ceph-mon[103124]: osdmap e323: 12 total, 12 up, 12 in 2023-09-10T01:21:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7300.130788 secs 2023-09-10T01:21:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:09 smithi170 ceph-mon[102832]: pgmap v4058: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 30 op/s 2023-09-10T01:21:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7300.842823 secs 2023-09-10T01:21:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7300.130788 secs 2023-09-10T01:21:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:09 smithi082 ceph-mon[98694]: pgmap v4058: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 30 op/s 2023-09-10T01:21:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7300.842823 secs 2023-09-10T01:21:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7300.130788 secs 2023-09-10T01:21:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:09 smithi098 ceph-mon[103124]: pgmap v4058: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 30 op/s 2023-09-10T01:21:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7300.842823 secs 2023-09-10T01:21:11.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:11 smithi170 ceph-mon[102832]: pgmap v4059: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 129 KiB/s wr, 29 op/s 2023-09-10T01:21:11.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:21:11.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:21:11.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:21:11.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:21:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:11 smithi082 ceph-mon[98694]: pgmap v4059: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 129 KiB/s wr, 29 op/s 2023-09-10T01:21:11.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:21:11.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:21:11.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:21:11.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:21:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:11 smithi098 ceph-mon[103124]: pgmap v4059: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 129 KiB/s wr, 29 op/s 2023-09-10T01:21:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:21:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:21:12.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:21:12.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:21:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7303.648205 secs 2023-09-10T01:21:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:13 smithi170 ceph-mon[102832]: pgmap v4060: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 129 KiB/s wr, 29 op/s 2023-09-10T01:21:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7303.648205 secs 2023-09-10T01:21:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:13 smithi082 ceph-mon[98694]: pgmap v4060: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 129 KiB/s wr, 29 op/s 2023-09-10T01:21:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7303.648205 secs 2023-09-10T01:21:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:13 smithi098 ceph-mon[103124]: pgmap v4060: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 129 KiB/s wr, 29 op/s 2023-09-10T01:21:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7305.130847 secs 2023-09-10T01:21:15.030 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7305.842910 secs 2023-09-10T01:21:15.031 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7305.130847 secs 2023-09-10T01:21:15.031 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7305.842910 secs 2023-09-10T01:21:15.031 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7305.130847 secs 2023-09-10T01:21:15.032 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7305.842910 secs 2023-09-10T01:21:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:15 smithi170 ceph-mon[102832]: pgmap v4061: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 129 KiB/s wr, 29 op/s 2023-09-10T01:21:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:15 smithi170 ceph-mon[102832]: osdmap e324: 12 total, 12 up, 12 in 2023-09-10T01:21:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:15 smithi082 ceph-mon[98694]: pgmap v4061: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 129 KiB/s wr, 29 op/s 2023-09-10T01:21:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:15 smithi082 ceph-mon[98694]: osdmap e324: 12 total, 12 up, 12 in 2023-09-10T01:21:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:15 smithi098 ceph-mon[103124]: pgmap v4061: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 129 KiB/s wr, 29 op/s 2023-09-10T01:21:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:15 smithi098 ceph-mon[103124]: osdmap e324: 12 total, 12 up, 12 in 2023-09-10T01:21:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:17 smithi170 ceph-mon[102832]: pgmap v4063: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 13 op/s 2023-09-10T01:21:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:17 smithi082 ceph-mon[98694]: pgmap v4063: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 13 op/s 2023-09-10T01:21:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:17 smithi098 ceph-mon[103124]: pgmap v4063: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 13 op/s 2023-09-10T01:21:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7308.648271 secs 2023-09-10T01:21:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7308.648271 secs 2023-09-10T01:21:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7308.648271 secs 2023-09-10T01:21:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7310.130904 secs 2023-09-10T01:21:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:19 smithi170 ceph-mon[102832]: pgmap v4064: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T01:21:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7310.842975 secs 2023-09-10T01:21:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7310.130904 secs 2023-09-10T01:21:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:19 smithi082 ceph-mon[98694]: pgmap v4064: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T01:21:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7310.842975 secs 2023-09-10T01:21:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7310.130904 secs 2023-09-10T01:21:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:19 smithi098 ceph-mon[103124]: pgmap v4064: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T01:21:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7310.842975 secs 2023-09-10T01:21:21.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:21 smithi170 ceph-mon[102832]: pgmap v4065: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T01:21:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:21 smithi082 ceph-mon[98694]: pgmap v4065: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T01:21:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:21 smithi098 ceph-mon[103124]: pgmap v4065: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T01:21:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:23 smithi170 ceph-mon[102832]: pgmap v4066: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T01:21:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:21:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:23 smithi082 ceph-mon[98694]: pgmap v4066: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T01:21:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:21:24.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:23 smithi098 ceph-mon[103124]: pgmap v4066: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T01:21:24.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:21:24.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7315.130988 secs 2023-09-10T01:21:24.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:21:24.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:21:24.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:21:24.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7315.843035 secs 2023-09-10T01:21:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7315.130988 secs 2023-09-10T01:21:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:21:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:21:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:21:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7315.843035 secs 2023-09-10T01:21:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7315.130988 secs 2023-09-10T01:21:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:21:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:21:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:21:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7315.843035 secs 2023-09-10T01:21:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:25 smithi170 ceph-mon[102832]: pgmap v4067: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T01:21:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:25 smithi082 ceph-mon[98694]: pgmap v4067: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T01:21:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:25 smithi098 ceph-mon[103124]: pgmap v4067: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T01:21:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:27 smithi170 ceph-mon[102832]: pgmap v4068: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T01:21:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:27 smithi082 ceph-mon[98694]: pgmap v4068: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T01:21:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:27 smithi098 ceph-mon[103124]: pgmap v4068: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T01:21:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7320.131049 secs 2023-09-10T01:21:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:29 smithi170 ceph-mon[102832]: pgmap v4069: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T01:21:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7320.843099 secs 2023-09-10T01:21:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7320.131049 secs 2023-09-10T01:21:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:29 smithi082 ceph-mon[98694]: pgmap v4069: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T01:21:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7320.843099 secs 2023-09-10T01:21:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7320.131049 secs 2023-09-10T01:21:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:29 smithi098 ceph-mon[103124]: pgmap v4069: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T01:21:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7320.843099 secs 2023-09-10T01:21:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:31 smithi170 ceph-mon[102832]: pgmap v4070: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 238 KiB/s wr, 17 op/s 2023-09-10T01:21:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:31 smithi082 ceph-mon[98694]: pgmap v4070: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 238 KiB/s wr, 17 op/s 2023-09-10T01:21:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:31 smithi098 ceph-mon[103124]: pgmap v4070: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 238 KiB/s wr, 17 op/s 2023-09-10T01:21:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:33 smithi170 ceph-mon[102832]: pgmap v4071: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T01:21:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:33 smithi082 ceph-mon[98694]: pgmap v4071: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T01:21:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:33 smithi098 ceph-mon[103124]: pgmap v4071: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T01:21:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7325.131108 secs 2023-09-10T01:21:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7325.843164 secs 2023-09-10T01:21:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7325.131108 secs 2023-09-10T01:21:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7325.843164 secs 2023-09-10T01:21:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7325.131108 secs 2023-09-10T01:21:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7325.843164 secs 2023-09-10T01:21:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:35 smithi170 ceph-mon[102832]: pgmap v4072: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T01:21:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:35 smithi082 ceph-mon[98694]: pgmap v4072: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T01:21:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:35 smithi098 ceph-mon[103124]: pgmap v4072: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T01:21:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:37 smithi170 ceph-mon[102832]: pgmap v4073: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 243 KiB/s wr, 18 op/s 2023-09-10T01:21:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:37 smithi082 ceph-mon[98694]: pgmap v4073: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 243 KiB/s wr, 18 op/s 2023-09-10T01:21:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:37 smithi098 ceph-mon[103124]: pgmap v4073: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 243 KiB/s wr, 18 op/s 2023-09-10T01:21:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7330.131168 secs 2023-09-10T01:21:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:39 smithi170 ceph-mon[102832]: pgmap v4074: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 184 KiB/s wr, 12 op/s 2023-09-10T01:21:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7330.843229 secs 2023-09-10T01:21:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7330.131168 secs 2023-09-10T01:21:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:39 smithi082 ceph-mon[98694]: pgmap v4074: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 184 KiB/s wr, 12 op/s 2023-09-10T01:21:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7330.843229 secs 2023-09-10T01:21:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7330.131168 secs 2023-09-10T01:21:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:39 smithi098 ceph-mon[103124]: pgmap v4074: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 184 KiB/s wr, 12 op/s 2023-09-10T01:21:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7330.843229 secs 2023-09-10T01:21:41.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:41 smithi170 ceph-mon[102832]: pgmap v4075: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 287 KiB/s wr, 18 op/s 2023-09-10T01:21:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:41 smithi082 ceph-mon[98694]: pgmap v4075: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 287 KiB/s wr, 18 op/s 2023-09-10T01:21:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:41 smithi098 ceph-mon[103124]: pgmap v4075: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 287 KiB/s wr, 18 op/s 2023-09-10T01:21:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:43 smithi170 ceph-mon[102832]: pgmap v4076: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T01:21:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:43 smithi082 ceph-mon[98694]: pgmap v4076: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T01:21:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:43 smithi098 ceph-mon[103124]: pgmap v4076: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T01:21:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7335.131222 secs 2023-09-10T01:21:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7335.843292 secs 2023-09-10T01:21:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7335.131222 secs 2023-09-10T01:21:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7335.843292 secs 2023-09-10T01:21:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7335.131222 secs 2023-09-10T01:21:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7335.843292 secs 2023-09-10T01:21:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:45 smithi170 ceph-mon[102832]: pgmap v4077: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T01:21:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:45 smithi082 ceph-mon[98694]: pgmap v4077: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T01:21:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:45 smithi098 ceph-mon[103124]: pgmap v4077: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T01:21:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:47 smithi170 ceph-mon[102832]: pgmap v4078: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 269 KiB/s wr, 17 op/s 2023-09-10T01:21:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:47 smithi082 ceph-mon[98694]: pgmap v4078: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 269 KiB/s wr, 17 op/s 2023-09-10T01:21:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:47 smithi098 ceph-mon[103124]: pgmap v4078: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 269 KiB/s wr, 17 op/s 2023-09-10T01:21:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7338.648733 secs 2023-09-10T01:21:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7338.648733 secs 2023-09-10T01:21:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7338.648733 secs 2023-09-10T01:21:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7340.131280 secs 2023-09-10T01:21:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:49 smithi170 ceph-mon[102832]: pgmap v4079: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 184 KiB/s wr, 11 op/s 2023-09-10T01:21:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7340.843361 secs 2023-09-10T01:21:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7340.131280 secs 2023-09-10T01:21:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:49 smithi082 ceph-mon[98694]: pgmap v4079: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 184 KiB/s wr, 11 op/s 2023-09-10T01:21:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7340.843361 secs 2023-09-10T01:21:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7340.131280 secs 2023-09-10T01:21:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:49 smithi098 ceph-mon[103124]: pgmap v4079: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 184 KiB/s wr, 11 op/s 2023-09-10T01:21:50.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7340.843361 secs 2023-09-10T01:21:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:51 smithi170 ceph-mon[102832]: pgmap v4080: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 228 KiB/s wr, 16 op/s 2023-09-10T01:21:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:51 smithi082 ceph-mon[98694]: pgmap v4080: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 228 KiB/s wr, 16 op/s 2023-09-10T01:21:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:51 smithi098 ceph-mon[103124]: pgmap v4080: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 228 KiB/s wr, 16 op/s 2023-09-10T01:21:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:21:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:21:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:21:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:21:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:21:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:21:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:21:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:21:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:21:53.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7343.648779 secs 2023-09-10T01:21:53.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:53 smithi170 ceph-mon[102832]: pgmap v4081: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T01:21:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7343.648779 secs 2023-09-10T01:21:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:53 smithi082 ceph-mon[98694]: pgmap v4081: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T01:21:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7343.648779 secs 2023-09-10T01:21:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:53 smithi098 ceph-mon[103124]: pgmap v4081: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T01:21:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7345.131337 secs 2023-09-10T01:21:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7345.843426 secs 2023-09-10T01:21:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7345.131337 secs 2023-09-10T01:21:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7345.843426 secs 2023-09-10T01:21:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7345.131337 secs 2023-09-10T01:21:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7345.843426 secs 2023-09-10T01:21:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:55 smithi170 ceph-mon[102832]: pgmap v4082: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T01:21:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:55 smithi082 ceph-mon[98694]: pgmap v4082: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T01:21:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:55 smithi098 ceph-mon[103124]: pgmap v4082: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T01:21:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:57 smithi170 ceph-mon[102832]: pgmap v4083: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 187 KiB/s wr, 16 op/s 2023-09-10T01:21:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:57 smithi082 ceph-mon[98694]: pgmap v4083: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 187 KiB/s wr, 16 op/s 2023-09-10T01:21:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:57 smithi098 ceph-mon[103124]: pgmap v4083: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 187 KiB/s wr, 16 op/s 2023-09-10T01:21:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7348.648836 secs 2023-09-10T01:21:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7348.648836 secs 2023-09-10T01:21:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7348.648836 secs 2023-09-10T01:21:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7350.131442 secs 2023-09-10T01:21:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:59 smithi170 ceph-mon[102832]: pgmap v4084: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 106 KiB/s wr, 10 op/s 2023-09-10T01:21:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:21:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7350.843488 secs 2023-09-10T01:21:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7350.131442 secs 2023-09-10T01:21:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:59 smithi082 ceph-mon[98694]: pgmap v4084: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 106 KiB/s wr, 10 op/s 2023-09-10T01:21:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:21:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7350.843488 secs 2023-09-10T01:22:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7350.131442 secs 2023-09-10T01:22:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:59 smithi098 ceph-mon[103124]: pgmap v4084: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 106 KiB/s wr, 10 op/s 2023-09-10T01:22:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:21:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7350.843488 secs 2023-09-10T01:22:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:01 smithi170 ceph-mon[102832]: pgmap v4085: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 15 op/s 2023-09-10T01:22:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:01 smithi082 ceph-mon[98694]: pgmap v4085: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 15 op/s 2023-09-10T01:22:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:01 smithi098 ceph-mon[103124]: pgmap v4085: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 15 op/s 2023-09-10T01:22:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7353.648897 secs 2023-09-10T01:22:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:03 smithi170 ceph-mon[102832]: pgmap v4086: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T01:22:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7353.648897 secs 2023-09-10T01:22:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:03 smithi082 ceph-mon[98694]: pgmap v4086: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T01:22:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7353.648897 secs 2023-09-10T01:22:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:03 smithi098 ceph-mon[103124]: pgmap v4086: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T01:22:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7355.131526 secs 2023-09-10T01:22:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7355.843554 secs 2023-09-10T01:22:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7355.131526 secs 2023-09-10T01:22:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7355.843554 secs 2023-09-10T01:22:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7355.131526 secs 2023-09-10T01:22:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7355.843554 secs 2023-09-10T01:22:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:05 smithi170 ceph-mon[102832]: pgmap v4087: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T01:22:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:05 smithi082 ceph-mon[98694]: pgmap v4087: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T01:22:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:05 smithi098 ceph-mon[103124]: pgmap v4087: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T01:22:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:07 smithi170 ceph-mon[102832]: pgmap v4088: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 177 KiB/s wr, 30 op/s 2023-09-10T01:22:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:07 smithi082 ceph-mon[98694]: pgmap v4088: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 177 KiB/s wr, 30 op/s 2023-09-10T01:22:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:07 smithi098 ceph-mon[103124]: pgmap v4088: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 177 KiB/s wr, 30 op/s 2023-09-10T01:22:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7358.648955 secs 2023-09-10T01:22:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:08 smithi170 ceph-mon[102832]: osdmap e325: 12 total, 12 up, 12 in 2023-09-10T01:22:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7360.131585 secs 2023-09-10T01:22:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7358.648955 secs 2023-09-10T01:22:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:08 smithi082 ceph-mon[98694]: osdmap e325: 12 total, 12 up, 12 in 2023-09-10T01:22:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7360.131585 secs 2023-09-10T01:22:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7358.648955 secs 2023-09-10T01:22:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:08 smithi098 ceph-mon[103124]: osdmap e325: 12 total, 12 up, 12 in 2023-09-10T01:22:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7360.131585 secs 2023-09-10T01:22:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:09 smithi170 ceph-mon[102832]: pgmap v4090: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 137 KiB/s wr, 29 op/s 2023-09-10T01:22:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7360.843619 secs 2023-09-10T01:22:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:09 smithi082 ceph-mon[98694]: pgmap v4090: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 137 KiB/s wr, 29 op/s 2023-09-10T01:22:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7360.843619 secs 2023-09-10T01:22:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:09 smithi098 ceph-mon[103124]: pgmap v4090: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 137 KiB/s wr, 29 op/s 2023-09-10T01:22:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7360.843619 secs 2023-09-10T01:22:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:11 smithi170 ceph-mon[102832]: pgmap v4091: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 130 KiB/s wr, 29 op/s 2023-09-10T01:22:11.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:22:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:11 smithi082 ceph-mon[98694]: pgmap v4091: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 130 KiB/s wr, 29 op/s 2023-09-10T01:22:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:22:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:11 smithi098 ceph-mon[103124]: pgmap v4091: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 130 KiB/s wr, 29 op/s 2023-09-10T01:22:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:22:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:22:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:22:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:22:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:22:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:22:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:22:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:22:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:22:13.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:22:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7363.649014 secs 2023-09-10T01:22:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:13 smithi170 ceph-mon[102832]: pgmap v4092: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 130 KiB/s wr, 29 op/s 2023-09-10T01:22:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7365.131645 secs 2023-09-10T01:22:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7365.843683 secs 2023-09-10T01:22:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7363.649014 secs 2023-09-10T01:22:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:13 smithi082 ceph-mon[98694]: pgmap v4092: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 130 KiB/s wr, 29 op/s 2023-09-10T01:22:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7365.131645 secs 2023-09-10T01:22:13.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7365.843683 secs 2023-09-10T01:22:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7363.649014 secs 2023-09-10T01:22:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:13 smithi098 ceph-mon[103124]: pgmap v4092: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 130 KiB/s wr, 29 op/s 2023-09-10T01:22:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7365.131645 secs 2023-09-10T01:22:14.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7365.843683 secs 2023-09-10T01:22:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:15 smithi082 ceph-mon[98694]: pgmap v4093: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 130 KiB/s wr, 29 op/s 2023-09-10T01:22:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:15 smithi082 ceph-mon[98694]: osdmap e326: 12 total, 12 up, 12 in 2023-09-10T01:22:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:15 smithi098 ceph-mon[103124]: pgmap v4093: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 130 KiB/s wr, 29 op/s 2023-09-10T01:22:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:15 smithi098 ceph-mon[103124]: osdmap e326: 12 total, 12 up, 12 in 2023-09-10T01:22:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:15 smithi170 ceph-mon[102832]: pgmap v4093: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 130 KiB/s wr, 29 op/s 2023-09-10T01:22:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:15 smithi170 ceph-mon[102832]: osdmap e326: 12 total, 12 up, 12 in 2023-09-10T01:22:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:17 smithi082 ceph-mon[98694]: pgmap v4095: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 79 KiB/s wr, 13 op/s 2023-09-10T01:22:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:17 smithi098 ceph-mon[103124]: pgmap v4095: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 79 KiB/s wr, 13 op/s 2023-09-10T01:22:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:17 smithi170 ceph-mon[102832]: pgmap v4095: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 79 KiB/s wr, 13 op/s 2023-09-10T01:22:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7368.649099 secs 2023-09-10T01:22:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7370.131704 secs 2023-09-10T01:22:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7370.843748 secs 2023-09-10T01:22:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7368.649099 secs 2023-09-10T01:22:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7370.131704 secs 2023-09-10T01:22:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7370.843748 secs 2023-09-10T01:22:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7368.649099 secs 2023-09-10T01:22:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7370.131704 secs 2023-09-10T01:22:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7370.843748 secs 2023-09-10T01:22:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:19 smithi082 ceph-mon[98694]: pgmap v4096: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T01:22:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:19 smithi098 ceph-mon[103124]: pgmap v4096: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T01:22:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:19 smithi170 ceph-mon[102832]: pgmap v4096: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T01:22:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:21 smithi082 ceph-mon[98694]: pgmap v4097: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T01:22:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:21 smithi098 ceph-mon[103124]: pgmap v4097: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T01:22:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:21 smithi170 ceph-mon[102832]: pgmap v4097: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T01:22:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:22:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:22:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:22:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:23 smithi082 ceph-mon[98694]: pgmap v4098: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T01:22:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:22:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:22:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:22:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7375.131790 secs 2023-09-10T01:22:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7375.843811 secs 2023-09-10T01:22:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:23 smithi098 ceph-mon[103124]: pgmap v4098: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T01:22:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:22:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:22:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:22:24.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7375.131790 secs 2023-09-10T01:22:24.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7375.843811 secs 2023-09-10T01:22:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:23 smithi170 ceph-mon[102832]: pgmap v4098: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T01:22:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:22:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:22:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:22:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7375.131790 secs 2023-09-10T01:22:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7375.843811 secs 2023-09-10T01:22:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:25 smithi082 ceph-mon[98694]: pgmap v4099: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T01:22:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:25 smithi098 ceph-mon[103124]: pgmap v4099: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T01:22:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:25 smithi170 ceph-mon[102832]: pgmap v4099: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T01:22:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:27 smithi082 ceph-mon[98694]: pgmap v4100: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T01:22:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:27 smithi098 ceph-mon[103124]: pgmap v4100: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T01:22:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:27 smithi170 ceph-mon[102832]: pgmap v4100: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T01:22:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7380.131847 secs 2023-09-10T01:22:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7380.843875 secs 2023-09-10T01:22:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7380.131847 secs 2023-09-10T01:22:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7380.843875 secs 2023-09-10T01:22:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7380.131847 secs 2023-09-10T01:22:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7380.843875 secs 2023-09-10T01:22:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:29 smithi098 ceph-mon[103124]: pgmap v4101: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 10 op/s 2023-09-10T01:22:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:29 smithi170 ceph-mon[102832]: pgmap v4101: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 10 op/s 2023-09-10T01:22:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:29 smithi082 ceph-mon[98694]: pgmap v4101: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 10 op/s 2023-09-10T01:22:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:31 smithi098 ceph-mon[103124]: pgmap v4102: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 271 KiB/s wr, 17 op/s 2023-09-10T01:22:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:31 smithi170 ceph-mon[102832]: pgmap v4102: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 271 KiB/s wr, 17 op/s 2023-09-10T01:22:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:31 smithi082 ceph-mon[98694]: pgmap v4102: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 271 KiB/s wr, 17 op/s 2023-09-10T01:22:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:33 smithi098 ceph-mon[103124]: pgmap v4103: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 185 KiB/s wr, 12 op/s 2023-09-10T01:22:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7385.131907 secs 2023-09-10T01:22:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7385.843943 secs 2023-09-10T01:22:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:33 smithi170 ceph-mon[102832]: pgmap v4103: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 185 KiB/s wr, 12 op/s 2023-09-10T01:22:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7385.131907 secs 2023-09-10T01:22:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7385.843943 secs 2023-09-10T01:22:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:33 smithi082 ceph-mon[98694]: pgmap v4103: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 185 KiB/s wr, 12 op/s 2023-09-10T01:22:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7385.131907 secs 2023-09-10T01:22:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7385.843943 secs 2023-09-10T01:22:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:35 smithi098 ceph-mon[103124]: pgmap v4104: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 185 KiB/s wr, 12 op/s 2023-09-10T01:22:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:35 smithi170 ceph-mon[102832]: pgmap v4104: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 185 KiB/s wr, 12 op/s 2023-09-10T01:22:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:35 smithi082 ceph-mon[98694]: pgmap v4104: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 185 KiB/s wr, 12 op/s 2023-09-10T01:22:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:36 smithi170 ceph-mon[102832]: pgmap v4105: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 204 KiB/s wr, 28 op/s 2023-09-10T01:22:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:36 smithi082 ceph-mon[98694]: pgmap v4105: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 204 KiB/s wr, 28 op/s 2023-09-10T01:22:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:36 smithi098 ceph-mon[103124]: pgmap v4105: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 204 KiB/s wr, 28 op/s 2023-09-10T01:22:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7390.131989 secs 2023-09-10T01:22:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:39 smithi170 ceph-mon[102832]: pgmap v4106: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 186 KiB/s wr, 22 op/s 2023-09-10T01:22:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7390.844007 secs 2023-09-10T01:22:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7390.131989 secs 2023-09-10T01:22:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:39 smithi082 ceph-mon[98694]: pgmap v4106: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 186 KiB/s wr, 22 op/s 2023-09-10T01:22:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7390.844007 secs 2023-09-10T01:22:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7390.131989 secs 2023-09-10T01:22:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:39 smithi098 ceph-mon[103124]: pgmap v4106: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 186 KiB/s wr, 22 op/s 2023-09-10T01:22:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7390.844007 secs 2023-09-10T01:22:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:41 smithi170 ceph-mon[102832]: pgmap v4107: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 227 KiB/s wr, 29 op/s 2023-09-10T01:22:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:41 smithi082 ceph-mon[98694]: pgmap v4107: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 227 KiB/s wr, 29 op/s 2023-09-10T01:22:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:41 smithi098 ceph-mon[103124]: pgmap v4107: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 227 KiB/s wr, 29 op/s 2023-09-10T01:22:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:43 smithi170 ceph-mon[102832]: pgmap v4108: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 60 KiB/s wr, 22 op/s 2023-09-10T01:22:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:43 smithi082 ceph-mon[98694]: pgmap v4108: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 60 KiB/s wr, 22 op/s 2023-09-10T01:22:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:43 smithi098 ceph-mon[103124]: pgmap v4108: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 60 KiB/s wr, 22 op/s 2023-09-10T01:22:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7395.132043 secs 2023-09-10T01:22:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7395.844071 secs 2023-09-10T01:22:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7395.132043 secs 2023-09-10T01:22:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7395.844071 secs 2023-09-10T01:22:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7395.132043 secs 2023-09-10T01:22:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7395.844071 secs 2023-09-10T01:22:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:45 smithi170 ceph-mon[102832]: pgmap v4109: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 60 KiB/s wr, 22 op/s 2023-09-10T01:22:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:45 smithi082 ceph-mon[98694]: pgmap v4109: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 60 KiB/s wr, 22 op/s 2023-09-10T01:22:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:45 smithi098 ceph-mon[103124]: pgmap v4109: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 60 KiB/s wr, 22 op/s 2023-09-10T01:22:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:47 smithi170 ceph-mon[102832]: pgmap v4110: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 67 KiB/s wr, 28 op/s 2023-09-10T01:22:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:47 smithi082 ceph-mon[98694]: pgmap v4110: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 67 KiB/s wr, 28 op/s 2023-09-10T01:22:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:47 smithi098 ceph-mon[103124]: pgmap v4110: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 67 KiB/s wr, 28 op/s 2023-09-10T01:22:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7398.649460 secs 2023-09-10T01:22:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7398.649460 secs 2023-09-10T01:22:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7398.649460 secs 2023-09-10T01:22:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7400.132097 secs 2023-09-10T01:22:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:49 smithi170 ceph-mon[102832]: pgmap v4111: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 KiB/s wr, 12 op/s 2023-09-10T01:22:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7400.844137 secs 2023-09-10T01:22:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7400.132097 secs 2023-09-10T01:22:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:49 smithi082 ceph-mon[98694]: pgmap v4111: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 KiB/s wr, 12 op/s 2023-09-10T01:22:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7400.844137 secs 2023-09-10T01:22:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7400.132097 secs 2023-09-10T01:22:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:49 smithi098 ceph-mon[103124]: pgmap v4111: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 KiB/s wr, 12 op/s 2023-09-10T01:22:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7400.844137 secs 2023-09-10T01:22:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:51 smithi170 ceph-mon[102832]: pgmap v4112: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 79 KiB/s wr, 18 op/s 2023-09-10T01:22:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:51 smithi082 ceph-mon[98694]: pgmap v4112: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 79 KiB/s wr, 18 op/s 2023-09-10T01:22:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:51 smithi098 ceph-mon[103124]: pgmap v4112: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 79 KiB/s wr, 18 op/s 2023-09-10T01:22:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:22:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:22:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:22:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:22:52.994 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:22:52.995 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:22:53.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:22:53.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:22:53.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:22:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7403.649521 secs 2023-09-10T01:22:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:53 smithi170 ceph-mon[102832]: pgmap v4113: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:22:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7403.649521 secs 2023-09-10T01:22:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:53 smithi082 ceph-mon[98694]: pgmap v4113: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:22:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7403.649521 secs 2023-09-10T01:22:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:53 smithi098 ceph-mon[103124]: pgmap v4113: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:22:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7405.132165 secs 2023-09-10T01:22:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7405.844201 secs 2023-09-10T01:22:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7405.132165 secs 2023-09-10T01:22:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7405.844201 secs 2023-09-10T01:22:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7405.132165 secs 2023-09-10T01:22:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7405.844201 secs 2023-09-10T01:22:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:55 smithi170 ceph-mon[102832]: pgmap v4114: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:22:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:55 smithi082 ceph-mon[98694]: pgmap v4114: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:22:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:55 smithi098 ceph-mon[103124]: pgmap v4114: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:22:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:57 smithi170 ceph-mon[102832]: pgmap v4115: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 39 KiB/s wr, 16 op/s 2023-09-10T01:22:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:57 smithi082 ceph-mon[98694]: pgmap v4115: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 39 KiB/s wr, 16 op/s 2023-09-10T01:22:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:57 smithi098 ceph-mon[103124]: pgmap v4115: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 39 KiB/s wr, 16 op/s 2023-09-10T01:22:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7408.649599 secs 2023-09-10T01:22:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7408.649599 secs 2023-09-10T01:22:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7408.649599 secs 2023-09-10T01:22:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7410.132249 secs 2023-09-10T01:22:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:59 smithi170 ceph-mon[102832]: pgmap v4116: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 10 op/s 2023-09-10T01:22:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:22:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7410.844268 secs 2023-09-10T01:22:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7410.132249 secs 2023-09-10T01:22:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:59 smithi082 ceph-mon[98694]: pgmap v4116: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 10 op/s 2023-09-10T01:22:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:22:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7410.844268 secs 2023-09-10T01:23:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7410.132249 secs 2023-09-10T01:23:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:59 smithi098 ceph-mon[103124]: pgmap v4116: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 10 op/s 2023-09-10T01:23:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:22:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7410.844268 secs 2023-09-10T01:23:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:01 smithi170 ceph-mon[102832]: pgmap v4117: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 KiB/s wr, 15 op/s 2023-09-10T01:23:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:01 smithi082 ceph-mon[98694]: pgmap v4117: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 KiB/s wr, 15 op/s 2023-09-10T01:23:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:01 smithi098 ceph-mon[103124]: pgmap v4117: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 KiB/s wr, 15 op/s 2023-09-10T01:23:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7413.649661 secs 2023-09-10T01:23:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:03 smithi170 ceph-mon[102832]: pgmap v4118: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.7 KiB/s wr, 9 op/s 2023-09-10T01:23:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7413.649661 secs 2023-09-10T01:23:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:03 smithi082 ceph-mon[98694]: pgmap v4118: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.7 KiB/s wr, 9 op/s 2023-09-10T01:23:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7413.649661 secs 2023-09-10T01:23:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:03 smithi098 ceph-mon[103124]: pgmap v4118: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.7 KiB/s wr, 9 op/s 2023-09-10T01:23:04.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7415.132304 secs 2023-09-10T01:23:04.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7415.844334 secs 2023-09-10T01:23:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7415.132304 secs 2023-09-10T01:23:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7415.844334 secs 2023-09-10T01:23:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7415.132304 secs 2023-09-10T01:23:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7415.844334 secs 2023-09-10T01:23:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:05 smithi170 ceph-mon[102832]: pgmap v4119: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.7 KiB/s wr, 9 op/s 2023-09-10T01:23:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:05 smithi082 ceph-mon[98694]: pgmap v4119: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.7 KiB/s wr, 9 op/s 2023-09-10T01:23:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:05 smithi098 ceph-mon[103124]: pgmap v4119: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.7 KiB/s wr, 9 op/s 2023-09-10T01:23:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:07 smithi170 ceph-mon[102832]: pgmap v4120: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T01:23:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:07 smithi082 ceph-mon[98694]: pgmap v4120: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T01:23:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:07 smithi098 ceph-mon[103124]: pgmap v4120: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T01:23:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7418.649721 secs 2023-09-10T01:23:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:08 smithi170 ceph-mon[102832]: osdmap e327: 12 total, 12 up, 12 in 2023-09-10T01:23:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7418.649721 secs 2023-09-10T01:23:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:08 smithi082 ceph-mon[98694]: osdmap e327: 12 total, 12 up, 12 in 2023-09-10T01:23:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7418.649721 secs 2023-09-10T01:23:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:08 smithi098 ceph-mon[103124]: osdmap e327: 12 total, 12 up, 12 in 2023-09-10T01:23:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7420.132360 secs 2023-09-10T01:23:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:09 smithi170 ceph-mon[102832]: pgmap v4122: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-10T01:23:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7420.844398 secs 2023-09-10T01:23:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7420.132360 secs 2023-09-10T01:23:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:09 smithi082 ceph-mon[98694]: pgmap v4122: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-10T01:23:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7420.844398 secs 2023-09-10T01:23:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7420.132360 secs 2023-09-10T01:23:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:09 smithi098 ceph-mon[103124]: pgmap v4122: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-10T01:23:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7420.844398 secs 2023-09-10T01:23:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:11 smithi170 ceph-mon[102832]: pgmap v4123: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T01:23:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:11 smithi082 ceph-mon[98694]: pgmap v4123: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T01:23:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:11 smithi098 ceph-mon[103124]: pgmap v4123: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T01:23:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:23:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:23:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:23:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7423.649776 secs 2023-09-10T01:23:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:13 smithi170 ceph-mon[102832]: pgmap v4124: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T01:23:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:23:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:23:13.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:23:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7423.649776 secs 2023-09-10T01:23:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:13 smithi082 ceph-mon[98694]: pgmap v4124: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T01:23:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:23:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:23:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:23:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7423.649776 secs 2023-09-10T01:23:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:13 smithi098 ceph-mon[103124]: pgmap v4124: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T01:23:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:23:14.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:23:14.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:23:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7425.132418 secs 2023-09-10T01:23:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7425.844464 secs 2023-09-10T01:23:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7425.132418 secs 2023-09-10T01:23:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7425.844464 secs 2023-09-10T01:23:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7425.132418 secs 2023-09-10T01:23:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7425.844464 secs 2023-09-10T01:23:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:15 smithi170 ceph-mon[102832]: pgmap v4125: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T01:23:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:15 smithi170 ceph-mon[102832]: osdmap e328: 12 total, 12 up, 12 in 2023-09-10T01:23:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:15 smithi082 ceph-mon[98694]: pgmap v4125: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T01:23:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:15 smithi082 ceph-mon[98694]: osdmap e328: 12 total, 12 up, 12 in 2023-09-10T01:23:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:15 smithi098 ceph-mon[103124]: pgmap v4125: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T01:23:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:15 smithi098 ceph-mon[103124]: osdmap e328: 12 total, 12 up, 12 in 2023-09-10T01:23:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:17 smithi170 ceph-mon[102832]: pgmap v4127: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-09-10T01:23:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:17 smithi082 ceph-mon[98694]: pgmap v4127: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-09-10T01:23:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:17 smithi098 ceph-mon[103124]: pgmap v4127: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-09-10T01:23:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7428.649836 secs 2023-09-10T01:23:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7428.649836 secs 2023-09-10T01:23:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7428.649836 secs 2023-09-10T01:23:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7430.132523 secs 2023-09-10T01:23:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:19 smithi170 ceph-mon[102832]: pgmap v4128: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 19 KiB/s wr, 11 op/s 2023-09-10T01:23:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7430.844492 secs 2023-09-10T01:23:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7430.132523 secs 2023-09-10T01:23:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:19 smithi082 ceph-mon[98694]: pgmap v4128: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 19 KiB/s wr, 11 op/s 2023-09-10T01:23:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7430.844492 secs 2023-09-10T01:23:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7430.132523 secs 2023-09-10T01:23:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:19 smithi098 ceph-mon[103124]: pgmap v4128: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 19 KiB/s wr, 11 op/s 2023-09-10T01:23:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7430.844492 secs 2023-09-10T01:23:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:21 smithi170 ceph-mon[102832]: pgmap v4129: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T01:23:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:21 smithi082 ceph-mon[98694]: pgmap v4129: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T01:23:22.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:21 smithi098 ceph-mon[103124]: pgmap v4129: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T01:23:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:23 smithi170 ceph-mon[102832]: pgmap v4130: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T01:23:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:23:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:23 smithi082 ceph-mon[98694]: pgmap v4130: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T01:23:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:23:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:23 smithi098 ceph-mon[103124]: pgmap v4130: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T01:23:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:23:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7435.132620 secs 2023-09-10T01:23:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7435.844557 secs 2023-09-10T01:23:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:23:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:23:24.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:23:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7435.132620 secs 2023-09-10T01:23:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7435.844557 secs 2023-09-10T01:23:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:23:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:23:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:23:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7435.132620 secs 2023-09-10T01:23:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7435.844557 secs 2023-09-10T01:23:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:23:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:23:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:23:25.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:25 smithi170 ceph-mon[102832]: pgmap v4131: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T01:23:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:25 smithi082 ceph-mon[98694]: pgmap v4131: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T01:23:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:25 smithi098 ceph-mon[103124]: pgmap v4131: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T01:23:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:27 smithi170 ceph-mon[102832]: pgmap v4132: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 31 KiB/s wr, 10 op/s 2023-09-10T01:23:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:27 smithi082 ceph-mon[98694]: pgmap v4132: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 31 KiB/s wr, 10 op/s 2023-09-10T01:23:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:27 smithi098 ceph-mon[103124]: pgmap v4132: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 31 KiB/s wr, 10 op/s 2023-09-10T01:23:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7440.132706 secs 2023-09-10T01:23:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7440.132706 secs 2023-09-10T01:23:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7440.132706 secs 2023-09-10T01:23:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:29 smithi170 ceph-mon[102832]: pgmap v4133: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 29 KiB/s wr, 10 op/s 2023-09-10T01:23:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7440.844622 secs 2023-09-10T01:23:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:29 smithi082 ceph-mon[98694]: pgmap v4133: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 29 KiB/s wr, 10 op/s 2023-09-10T01:23:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7440.844622 secs 2023-09-10T01:23:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:29 smithi098 ceph-mon[103124]: pgmap v4133: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 29 KiB/s wr, 10 op/s 2023-09-10T01:23:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7440.844622 secs 2023-09-10T01:23:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:31 smithi082 ceph-mon[98694]: pgmap v4134: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 71 KiB/s wr, 16 op/s 2023-09-10T01:23:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:31 smithi098 ceph-mon[103124]: pgmap v4134: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 71 KiB/s wr, 16 op/s 2023-09-10T01:23:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:31 smithi170 ceph-mon[102832]: pgmap v4134: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 71 KiB/s wr, 16 op/s 2023-09-10T01:23:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:33 smithi082 ceph-mon[98694]: pgmap v4135: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T01:23:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7445.132780 secs 2023-09-10T01:23:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7445.844689 secs 2023-09-10T01:23:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:33 smithi098 ceph-mon[103124]: pgmap v4135: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T01:23:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7445.132780 secs 2023-09-10T01:23:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7445.844689 secs 2023-09-10T01:23:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:33 smithi170 ceph-mon[102832]: pgmap v4135: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T01:23:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7445.132780 secs 2023-09-10T01:23:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7445.844689 secs 2023-09-10T01:23:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:35 smithi082 ceph-mon[98694]: pgmap v4136: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T01:23:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:35 smithi098 ceph-mon[103124]: pgmap v4136: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T01:23:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:35 smithi170 ceph-mon[102832]: pgmap v4136: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T01:23:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:37 smithi082 ceph-mon[98694]: pgmap v4137: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 58 KiB/s wr, 17 op/s 2023-09-10T01:23:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:37 smithi098 ceph-mon[103124]: pgmap v4137: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 58 KiB/s wr, 17 op/s 2023-09-10T01:23:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:37 smithi170 ceph-mon[102832]: pgmap v4137: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 58 KiB/s wr, 17 op/s 2023-09-10T01:23:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7450.132864 secs 2023-09-10T01:23:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7450.844756 secs 2023-09-10T01:23:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7450.132864 secs 2023-09-10T01:23:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7450.844756 secs 2023-09-10T01:23:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7450.132864 secs 2023-09-10T01:23:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7450.844756 secs 2023-09-10T01:23:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:39 smithi082 ceph-mon[98694]: pgmap v4138: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 51 KiB/s wr, 12 op/s 2023-09-10T01:23:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:39 smithi098 ceph-mon[103124]: pgmap v4138: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 51 KiB/s wr, 12 op/s 2023-09-10T01:23:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:39 smithi170 ceph-mon[102832]: pgmap v4138: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 51 KiB/s wr, 12 op/s 2023-09-10T01:23:41.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:41 smithi082 ceph-mon[98694]: pgmap v4139: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 93 KiB/s wr, 17 op/s 2023-09-10T01:23:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:41 smithi098 ceph-mon[103124]: pgmap v4139: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 93 KiB/s wr, 17 op/s 2023-09-10T01:23:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:41 smithi170 ceph-mon[102832]: pgmap v4139: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 93 KiB/s wr, 17 op/s 2023-09-10T01:23:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:43 smithi082 ceph-mon[98694]: pgmap v4140: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T01:23:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7455.132953 secs 2023-09-10T01:23:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7455.844824 secs 2023-09-10T01:23:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:43 smithi098 ceph-mon[103124]: pgmap v4140: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T01:23:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7455.132953 secs 2023-09-10T01:23:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7455.844824 secs 2023-09-10T01:23:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:43 smithi170 ceph-mon[102832]: pgmap v4140: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T01:23:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7455.132953 secs 2023-09-10T01:23:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7455.844824 secs 2023-09-10T01:23:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:45 smithi082 ceph-mon[98694]: pgmap v4141: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T01:23:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:45 smithi098 ceph-mon[103124]: pgmap v4141: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T01:23:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:45 smithi170 ceph-mon[102832]: pgmap v4141: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T01:23:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:47 smithi082 ceph-mon[98694]: pgmap v4142: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 57 KiB/s wr, 17 op/s 2023-09-10T01:23:47.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7458.650318 secs 2023-09-10T01:23:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:47 smithi098 ceph-mon[103124]: pgmap v4142: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 57 KiB/s wr, 17 op/s 2023-09-10T01:23:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7458.650318 secs 2023-09-10T01:23:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:47 smithi170 ceph-mon[102832]: pgmap v4142: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 57 KiB/s wr, 17 op/s 2023-09-10T01:23:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7458.650318 secs 2023-09-10T01:23:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7460.133011 secs 2023-09-10T01:23:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7460.844888 secs 2023-09-10T01:23:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7460.133011 secs 2023-09-10T01:23:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7460.844888 secs 2023-09-10T01:23:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7460.133011 secs 2023-09-10T01:23:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7460.844888 secs 2023-09-10T01:23:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:49 smithi098 ceph-mon[103124]: pgmap v4143: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T01:23:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:49 smithi170 ceph-mon[102832]: pgmap v4143: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T01:23:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:49 smithi082 ceph-mon[98694]: pgmap v4143: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T01:23:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:51 smithi170 ceph-mon[102832]: pgmap v4144: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 89 KiB/s wr, 16 op/s 2023-09-10T01:23:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:51 smithi082 ceph-mon[98694]: pgmap v4144: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 89 KiB/s wr, 16 op/s 2023-09-10T01:23:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:51 smithi098 ceph-mon[103124]: pgmap v4144: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 89 KiB/s wr, 16 op/s 2023-09-10T01:23:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:23:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:23:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:23:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7463.650419 secs 2023-09-10T01:23:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:23:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:23:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:23:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7463.650419 secs 2023-09-10T01:23:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:23:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:23:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:23:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7463.650419 secs 2023-09-10T01:23:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:53 smithi170 ceph-mon[102832]: pgmap v4145: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T01:23:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:53 smithi082 ceph-mon[98694]: pgmap v4145: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T01:23:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:53 smithi098 ceph-mon[103124]: pgmap v4145: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T01:23:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7465.133069 secs 2023-09-10T01:23:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7465.844955 secs 2023-09-10T01:23:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7465.133069 secs 2023-09-10T01:23:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7465.844955 secs 2023-09-10T01:23:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7465.133069 secs 2023-09-10T01:23:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7465.844955 secs 2023-09-10T01:23:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:55 smithi170 ceph-mon[102832]: pgmap v4146: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T01:23:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:55 smithi082 ceph-mon[98694]: pgmap v4146: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T01:23:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:55 smithi098 ceph-mon[103124]: pgmap v4146: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T01:23:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:57 smithi170 ceph-mon[102832]: pgmap v4147: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 53 KiB/s wr, 16 op/s 2023-09-10T01:23:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7468.650479 secs 2023-09-10T01:23:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:57 smithi082 ceph-mon[98694]: pgmap v4147: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 53 KiB/s wr, 16 op/s 2023-09-10T01:23:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7468.650479 secs 2023-09-10T01:23:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:57 smithi098 ceph-mon[103124]: pgmap v4147: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 53 KiB/s wr, 16 op/s 2023-09-10T01:23:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7468.650479 secs 2023-09-10T01:23:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7470.133143 secs 2023-09-10T01:23:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:59 smithi170 ceph-mon[102832]: pgmap v4148: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 45 KiB/s wr, 10 op/s 2023-09-10T01:23:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:23:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7470.845018 secs 2023-09-10T01:23:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7470.133143 secs 2023-09-10T01:23:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:59 smithi082 ceph-mon[98694]: pgmap v4148: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 45 KiB/s wr, 10 op/s 2023-09-10T01:23:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:23:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7470.845018 secs 2023-09-10T01:24:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7470.133143 secs 2023-09-10T01:24:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:59 smithi098 ceph-mon[103124]: pgmap v4148: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 45 KiB/s wr, 10 op/s 2023-09-10T01:24:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:23:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7470.845018 secs 2023-09-10T01:24:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:01 smithi170 ceph-mon[102832]: pgmap v4149: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 15 op/s 2023-09-10T01:24:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:01 smithi082 ceph-mon[98694]: pgmap v4149: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 15 op/s 2023-09-10T01:24:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:01 smithi098 ceph-mon[103124]: pgmap v4149: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 15 op/s 2023-09-10T01:24:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7473.650540 secs 2023-09-10T01:24:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7473.650540 secs 2023-09-10T01:24:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7473.650540 secs 2023-09-10T01:24:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:03 smithi170 ceph-mon[102832]: pgmap v4150: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 KiB/s wr, 10 op/s 2023-09-10T01:24:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:03 smithi082 ceph-mon[98694]: pgmap v4150: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 KiB/s wr, 10 op/s 2023-09-10T01:24:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:03 smithi098 ceph-mon[103124]: pgmap v4150: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 KiB/s wr, 10 op/s 2023-09-10T01:24:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7475.133205 secs 2023-09-10T01:24:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7475.845085 secs 2023-09-10T01:24:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7475.133205 secs 2023-09-10T01:24:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7475.845085 secs 2023-09-10T01:24:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7475.133205 secs 2023-09-10T01:24:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7475.845085 secs 2023-09-10T01:24:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:05 smithi170 ceph-mon[102832]: pgmap v4151: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 KiB/s wr, 10 op/s 2023-09-10T01:24:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:05 smithi082 ceph-mon[98694]: pgmap v4151: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 KiB/s wr, 10 op/s 2023-09-10T01:24:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:05 smithi098 ceph-mon[103124]: pgmap v4151: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 KiB/s wr, 10 op/s 2023-09-10T01:24:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:07 smithi170 ceph-mon[102832]: pgmap v4152: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-10T01:24:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7478.650658 secs 2023-09-10T01:24:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:07 smithi082 ceph-mon[98694]: pgmap v4152: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-10T01:24:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7478.650658 secs 2023-09-10T01:24:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:07 smithi098 ceph-mon[103124]: pgmap v4152: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-10T01:24:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7478.650658 secs 2023-09-10T01:24:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:08 smithi170 ceph-mon[102832]: osdmap e329: 12 total, 12 up, 12 in 2023-09-10T01:24:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:08 smithi082 ceph-mon[98694]: osdmap e329: 12 total, 12 up, 12 in 2023-09-10T01:24:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:08 smithi098 ceph-mon[103124]: osdmap e329: 12 total, 12 up, 12 in 2023-09-10T01:24:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7480.133263 secs 2023-09-10T01:24:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:09 smithi170 ceph-mon[102832]: pgmap v4154: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 80 KiB/s wr, 29 op/s 2023-09-10T01:24:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7480.845153 secs 2023-09-10T01:24:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7480.133263 secs 2023-09-10T01:24:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:09 smithi082 ceph-mon[98694]: pgmap v4154: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 80 KiB/s wr, 29 op/s 2023-09-10T01:24:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7480.845153 secs 2023-09-10T01:24:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7480.133263 secs 2023-09-10T01:24:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:09 smithi098 ceph-mon[103124]: pgmap v4154: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 80 KiB/s wr, 29 op/s 2023-09-10T01:24:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7480.845153 secs 2023-09-10T01:24:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:11 smithi170 ceph-mon[102832]: pgmap v4155: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-10T01:24:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:11 smithi082 ceph-mon[98694]: pgmap v4155: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-10T01:24:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:11 smithi098 ceph-mon[103124]: pgmap v4155: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-10T01:24:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7483.650719 secs 2023-09-10T01:24:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7483.650719 secs 2023-09-10T01:24:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7483.650719 secs 2023-09-10T01:24:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:13 smithi170 ceph-mon[102832]: pgmap v4156: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-10T01:24:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:24:13.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:24:13.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:24:13.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:24:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:13 smithi082 ceph-mon[98694]: pgmap v4156: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-10T01:24:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:24:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:24:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:24:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:24:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:13 smithi098 ceph-mon[103124]: pgmap v4156: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-10T01:24:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:24:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:24:14.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:24:14.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:24:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7485.133326 secs 2023-09-10T01:24:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7485.845219 secs 2023-09-10T01:24:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7485.133326 secs 2023-09-10T01:24:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7485.845219 secs 2023-09-10T01:24:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7485.133326 secs 2023-09-10T01:24:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7485.845219 secs 2023-09-10T01:24:15.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:15 smithi170 ceph-mon[102832]: pgmap v4157: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-10T01:24:15.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:15 smithi170 ceph-mon[102832]: osdmap e330: 12 total, 12 up, 12 in 2023-09-10T01:24:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:15 smithi082 ceph-mon[98694]: pgmap v4157: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-10T01:24:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:15 smithi082 ceph-mon[98694]: osdmap e330: 12 total, 12 up, 12 in 2023-09-10T01:24:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:15 smithi098 ceph-mon[103124]: pgmap v4157: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-10T01:24:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:15 smithi098 ceph-mon[103124]: osdmap e330: 12 total, 12 up, 12 in 2023-09-10T01:24:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:17 smithi170 ceph-mon[102832]: pgmap v4159: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 KiB/s wr, 14 op/s 2023-09-10T01:24:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7488.650773 secs 2023-09-10T01:24:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:17 smithi082 ceph-mon[98694]: pgmap v4159: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 KiB/s wr, 14 op/s 2023-09-10T01:24:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7488.650773 secs 2023-09-10T01:24:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:17 smithi098 ceph-mon[103124]: pgmap v4159: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 KiB/s wr, 14 op/s 2023-09-10T01:24:18.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7488.650773 secs 2023-09-10T01:24:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7490.133383 secs 2023-09-10T01:24:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:19 smithi170 ceph-mon[102832]: pgmap v4160: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-09-10T01:24:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7490.845285 secs 2023-09-10T01:24:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7490.133383 secs 2023-09-10T01:24:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:19 smithi082 ceph-mon[98694]: pgmap v4160: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-09-10T01:24:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7490.845285 secs 2023-09-10T01:24:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7490.133383 secs 2023-09-10T01:24:20.058 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:19 smithi098 ceph-mon[103124]: pgmap v4160: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-09-10T01:24:20.059 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7490.845285 secs 2023-09-10T01:24:21.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:21 smithi170 ceph-mon[102832]: pgmap v4161: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 37 KiB/s wr, 12 op/s 2023-09-10T01:24:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:21 smithi082 ceph-mon[98694]: pgmap v4161: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 37 KiB/s wr, 12 op/s 2023-09-10T01:24:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:21 smithi098 ceph-mon[103124]: pgmap v4161: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 37 KiB/s wr, 12 op/s 2023-09-10T01:24:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:23 smithi170 ceph-mon[102832]: pgmap v4162: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 37 KiB/s wr, 12 op/s 2023-09-10T01:24:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:24:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:23 smithi082 ceph-mon[98694]: pgmap v4162: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 37 KiB/s wr, 12 op/s 2023-09-10T01:24:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:24:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:23 smithi098 ceph-mon[103124]: pgmap v4162: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 37 KiB/s wr, 12 op/s 2023-09-10T01:24:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:24:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7495.133465 secs 2023-09-10T01:24:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:24:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:24:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:24:24.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7495.845349 secs 2023-09-10T01:24:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7495.133465 secs 2023-09-10T01:24:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:24:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:24:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:24:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7495.845349 secs 2023-09-10T01:24:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7495.133465 secs 2023-09-10T01:24:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:24:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:24:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:24:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7495.845349 secs 2023-09-10T01:24:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:25 smithi170 ceph-mon[102832]: pgmap v4163: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 37 KiB/s wr, 12 op/s 2023-09-10T01:24:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:25 smithi082 ceph-mon[98694]: pgmap v4163: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 37 KiB/s wr, 12 op/s 2023-09-10T01:24:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:25 smithi098 ceph-mon[103124]: pgmap v4163: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 37 KiB/s wr, 12 op/s 2023-09-10T01:24:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:27 smithi170 ceph-mon[102832]: pgmap v4164: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T01:24:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:27 smithi082 ceph-mon[98694]: pgmap v4164: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T01:24:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:27 smithi098 ceph-mon[103124]: pgmap v4164: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T01:24:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7500.133528 secs 2023-09-10T01:24:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:29 smithi170 ceph-mon[102832]: pgmap v4165: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 29 KiB/s wr, 10 op/s 2023-09-10T01:24:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7500.845415 secs 2023-09-10T01:24:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7500.133528 secs 2023-09-10T01:24:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:29 smithi082 ceph-mon[98694]: pgmap v4165: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 29 KiB/s wr, 10 op/s 2023-09-10T01:24:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7500.845415 secs 2023-09-10T01:24:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7500.133528 secs 2023-09-10T01:24:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:29 smithi098 ceph-mon[103124]: pgmap v4165: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 29 KiB/s wr, 10 op/s 2023-09-10T01:24:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7500.845415 secs 2023-09-10T01:24:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:31 smithi170 ceph-mon[102832]: pgmap v4166: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 74 KiB/s wr, 16 op/s 2023-09-10T01:24:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:31 smithi082 ceph-mon[98694]: pgmap v4166: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 74 KiB/s wr, 16 op/s 2023-09-10T01:24:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:31 smithi098 ceph-mon[103124]: pgmap v4166: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 74 KiB/s wr, 16 op/s 2023-09-10T01:24:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:33 smithi170 ceph-mon[102832]: pgmap v4167: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T01:24:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:33 smithi082 ceph-mon[98694]: pgmap v4167: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T01:24:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:33 smithi098 ceph-mon[103124]: pgmap v4167: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T01:24:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7505.133610 secs 2023-09-10T01:24:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7505.845482 secs 2023-09-10T01:24:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7505.133610 secs 2023-09-10T01:24:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7505.845482 secs 2023-09-10T01:24:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7505.133610 secs 2023-09-10T01:24:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7505.845482 secs 2023-09-10T01:24:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:35 smithi170 ceph-mon[102832]: pgmap v4168: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T01:24:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:35 smithi082 ceph-mon[98694]: pgmap v4168: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T01:24:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:35 smithi098 ceph-mon[103124]: pgmap v4168: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T01:24:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:37 smithi170 ceph-mon[102832]: pgmap v4169: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 60 KiB/s wr, 17 op/s 2023-09-10T01:24:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:37 smithi082 ceph-mon[98694]: pgmap v4169: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 60 KiB/s wr, 17 op/s 2023-09-10T01:24:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:37 smithi098 ceph-mon[103124]: pgmap v4169: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 60 KiB/s wr, 17 op/s 2023-09-10T01:24:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7510.133691 secs 2023-09-10T01:24:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:39 smithi170 ceph-mon[102832]: pgmap v4170: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 52 KiB/s wr, 12 op/s 2023-09-10T01:24:39.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7510.845573 secs 2023-09-10T01:24:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7510.133691 secs 2023-09-10T01:24:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:39 smithi082 ceph-mon[98694]: pgmap v4170: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 52 KiB/s wr, 12 op/s 2023-09-10T01:24:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7510.845573 secs 2023-09-10T01:24:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7510.133691 secs 2023-09-10T01:24:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:39 smithi098 ceph-mon[103124]: pgmap v4170: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 52 KiB/s wr, 12 op/s 2023-09-10T01:24:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7510.845573 secs 2023-09-10T01:24:41.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:41 smithi170 ceph-mon[102832]: pgmap v4171: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 95 KiB/s wr, 18 op/s 2023-09-10T01:24:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:41 smithi082 ceph-mon[98694]: pgmap v4171: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 95 KiB/s wr, 18 op/s 2023-09-10T01:24:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:41 smithi098 ceph-mon[103124]: pgmap v4171: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 95 KiB/s wr, 18 op/s 2023-09-10T01:24:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:43 smithi170 ceph-mon[102832]: pgmap v4172: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:24:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:43 smithi082 ceph-mon[98694]: pgmap v4172: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:24:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:43 smithi098 ceph-mon[103124]: pgmap v4172: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:24:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7515.133749 secs 2023-09-10T01:24:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7515.845636 secs 2023-09-10T01:24:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7515.133749 secs 2023-09-10T01:24:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7515.845636 secs 2023-09-10T01:24:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7515.133749 secs 2023-09-10T01:24:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7515.845636 secs 2023-09-10T01:24:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:45 smithi170 ceph-mon[102832]: pgmap v4173: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:24:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:45 smithi082 ceph-mon[98694]: pgmap v4173: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:24:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:45 smithi098 ceph-mon[103124]: pgmap v4173: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:24:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:47 smithi170 ceph-mon[102832]: pgmap v4174: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 59 KiB/s wr, 17 op/s 2023-09-10T01:24:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7518.651203 secs 2023-09-10T01:24:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:47 smithi082 ceph-mon[98694]: pgmap v4174: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 59 KiB/s wr, 17 op/s 2023-09-10T01:24:47.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7518.651203 secs 2023-09-10T01:24:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:47 smithi098 ceph-mon[103124]: pgmap v4174: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 59 KiB/s wr, 17 op/s 2023-09-10T01:24:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7518.651203 secs 2023-09-10T01:24:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7520.133803 secs 2023-09-10T01:24:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:49 smithi170 ceph-mon[102832]: pgmap v4175: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:24:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7520.845702 secs 2023-09-10T01:24:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7520.133803 secs 2023-09-10T01:24:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:49 smithi082 ceph-mon[98694]: pgmap v4175: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:24:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7520.845702 secs 2023-09-10T01:24:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7520.133803 secs 2023-09-10T01:24:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:49 smithi098 ceph-mon[103124]: pgmap v4175: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:24:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7520.845702 secs 2023-09-10T01:24:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:51 smithi170 ceph-mon[102832]: pgmap v4176: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 83 KiB/s wr, 16 op/s 2023-09-10T01:24:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:51 smithi082 ceph-mon[98694]: pgmap v4176: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 83 KiB/s wr, 16 op/s 2023-09-10T01:24:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:51 smithi098 ceph-mon[103124]: pgmap v4176: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 83 KiB/s wr, 16 op/s 2023-09-10T01:24:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:24:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:24:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:24:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7523.651266 secs 2023-09-10T01:24:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:24:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:24:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:24:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7523.651266 secs 2023-09-10T01:24:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:24:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:24:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:24:53.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7523.651266 secs 2023-09-10T01:24:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:53 smithi170 ceph-mon[102832]: pgmap v4177: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T01:24:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:53 smithi082 ceph-mon[98694]: pgmap v4177: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T01:24:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:53 smithi098 ceph-mon[103124]: pgmap v4177: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T01:24:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7525.133859 secs 2023-09-10T01:24:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7525.845770 secs 2023-09-10T01:24:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7525.133859 secs 2023-09-10T01:24:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7525.845770 secs 2023-09-10T01:24:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7525.133859 secs 2023-09-10T01:24:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7525.845770 secs 2023-09-10T01:24:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:55 smithi170 ceph-mon[102832]: pgmap v4178: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T01:24:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:55 smithi082 ceph-mon[98694]: pgmap v4178: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T01:24:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:55 smithi098 ceph-mon[103124]: pgmap v4178: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T01:24:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:57 smithi170 ceph-mon[102832]: pgmap v4179: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 46 KiB/s wr, 15 op/s 2023-09-10T01:24:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7528.651375 secs 2023-09-10T01:24:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:57 smithi082 ceph-mon[98694]: pgmap v4179: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 46 KiB/s wr, 15 op/s 2023-09-10T01:24:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7528.651375 secs 2023-09-10T01:24:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:57 smithi098 ceph-mon[103124]: pgmap v4179: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 46 KiB/s wr, 15 op/s 2023-09-10T01:24:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7528.651375 secs 2023-09-10T01:24:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7530.133948 secs 2023-09-10T01:24:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:59 smithi170 ceph-mon[102832]: pgmap v4180: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T01:24:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:24:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7530.845834 secs 2023-09-10T01:24:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7530.133948 secs 2023-09-10T01:24:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:59 smithi082 ceph-mon[98694]: pgmap v4180: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T01:24:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:24:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7530.845834 secs 2023-09-10T01:25:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7530.133948 secs 2023-09-10T01:25:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:59 smithi098 ceph-mon[103124]: pgmap v4180: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T01:25:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:24:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7530.845834 secs 2023-09-10T01:25:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:01 smithi170 ceph-mon[102832]: pgmap v4181: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 92 KiB/s wr, 30 op/s 2023-09-10T01:25:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:01 smithi082 ceph-mon[98694]: pgmap v4181: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 92 KiB/s wr, 30 op/s 2023-09-10T01:25:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:01 smithi098 ceph-mon[103124]: pgmap v4181: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 92 KiB/s wr, 30 op/s 2023-09-10T01:25:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7533.651435 secs 2023-09-10T01:25:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7533.651435 secs 2023-09-10T01:25:03.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7533.651435 secs 2023-09-10T01:25:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:03 smithi170 ceph-mon[102832]: pgmap v4182: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 60 KiB/s wr, 24 op/s 2023-09-10T01:25:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:03 smithi082 ceph-mon[98694]: pgmap v4182: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 60 KiB/s wr, 24 op/s 2023-09-10T01:25:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:03 smithi098 ceph-mon[103124]: pgmap v4182: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 60 KiB/s wr, 24 op/s 2023-09-10T01:25:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7535.134023 secs 2023-09-10T01:25:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7535.845900 secs 2023-09-10T01:25:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7535.134023 secs 2023-09-10T01:25:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7535.845900 secs 2023-09-10T01:25:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7535.134023 secs 2023-09-10T01:25:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7535.845900 secs 2023-09-10T01:25:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:05 smithi170 ceph-mon[102832]: pgmap v4183: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 60 KiB/s wr, 24 op/s 2023-09-10T01:25:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:05 smithi082 ceph-mon[98694]: pgmap v4183: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 60 KiB/s wr, 24 op/s 2023-09-10T01:25:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:05 smithi098 ceph-mon[103124]: pgmap v4183: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 60 KiB/s wr, 24 op/s 2023-09-10T01:25:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:07 smithi082 ceph-mon[98694]: pgmap v4184: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-10T01:25:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7538.651494 secs 2023-09-10T01:25:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:07 smithi098 ceph-mon[103124]: pgmap v4184: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-10T01:25:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7538.651494 secs 2023-09-10T01:25:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:07 smithi170 ceph-mon[102832]: pgmap v4184: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-10T01:25:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7538.651494 secs 2023-09-10T01:25:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:08 smithi082 ceph-mon[98694]: osdmap e331: 12 total, 12 up, 12 in 2023-09-10T01:25:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7540.134108 secs 2023-09-10T01:25:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:08 smithi098 ceph-mon[103124]: osdmap e331: 12 total, 12 up, 12 in 2023-09-10T01:25:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7540.134108 secs 2023-09-10T01:25:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:08 smithi170 ceph-mon[102832]: osdmap e331: 12 total, 12 up, 12 in 2023-09-10T01:25:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7540.134108 secs 2023-09-10T01:25:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:09 smithi082 ceph-mon[98694]: pgmap v4186: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 80 KiB/s wr, 29 op/s 2023-09-10T01:25:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7540.845963 secs 2023-09-10T01:25:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:09 smithi098 ceph-mon[103124]: pgmap v4186: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 80 KiB/s wr, 29 op/s 2023-09-10T01:25:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7540.845963 secs 2023-09-10T01:25:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:09 smithi170 ceph-mon[102832]: pgmap v4186: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 80 KiB/s wr, 29 op/s 2023-09-10T01:25:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7540.845963 secs 2023-09-10T01:25:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:11 smithi082 ceph-mon[98694]: pgmap v4187: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T01:25:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:11 smithi098 ceph-mon[103124]: pgmap v4187: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T01:25:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:11 smithi170 ceph-mon[102832]: pgmap v4187: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T01:25:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7543.651554 secs 2023-09-10T01:25:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7543.651554 secs 2023-09-10T01:25:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7543.651554 secs 2023-09-10T01:25:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:13 smithi082 ceph-mon[98694]: pgmap v4188: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T01:25:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7545.134167 secs 2023-09-10T01:25:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:25:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7545.846027 secs 2023-09-10T01:25:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:13 smithi098 ceph-mon[103124]: pgmap v4188: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T01:25:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7545.134167 secs 2023-09-10T01:25:14.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:25:14.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7545.846027 secs 2023-09-10T01:25:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:13 smithi170 ceph-mon[102832]: pgmap v4188: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T01:25:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7545.134167 secs 2023-09-10T01:25:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:25:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7545.846027 secs 2023-09-10T01:25:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:25:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:25:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:25:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:25:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:25:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:25:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:25:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:25:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:25:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:15 smithi082 ceph-mon[98694]: pgmap v4189: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T01:25:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:15 smithi082 ceph-mon[98694]: osdmap e332: 12 total, 12 up, 12 in 2023-09-10T01:25:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:15 smithi098 ceph-mon[103124]: pgmap v4189: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T01:25:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:15 smithi098 ceph-mon[103124]: osdmap e332: 12 total, 12 up, 12 in 2023-09-10T01:25:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:15 smithi170 ceph-mon[102832]: pgmap v4189: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T01:25:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:15 smithi170 ceph-mon[102832]: osdmap e332: 12 total, 12 up, 12 in 2023-09-10T01:25:17.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:17 smithi082 ceph-mon[98694]: pgmap v4191: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 23 KiB/s wr, 13 op/s 2023-09-10T01:25:17.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7548.651613 secs 2023-09-10T01:25:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:17 smithi098 ceph-mon[103124]: pgmap v4191: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 23 KiB/s wr, 13 op/s 2023-09-10T01:25:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7548.651613 secs 2023-09-10T01:25:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:17 smithi170 ceph-mon[102832]: pgmap v4191: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 23 KiB/s wr, 13 op/s 2023-09-10T01:25:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7548.651613 secs 2023-09-10T01:25:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7550.134226 secs 2023-09-10T01:25:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7550.846091 secs 2023-09-10T01:25:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7550.134226 secs 2023-09-10T01:25:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7550.846091 secs 2023-09-10T01:25:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7550.134226 secs 2023-09-10T01:25:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7550.846091 secs 2023-09-10T01:25:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:19 smithi082 ceph-mon[98694]: pgmap v4192: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-10T01:25:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:19 smithi098 ceph-mon[103124]: pgmap v4192: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-10T01:25:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:19 smithi170 ceph-mon[102832]: pgmap v4192: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-10T01:25:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:21 smithi098 ceph-mon[103124]: pgmap v4193: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:25:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:21 smithi170 ceph-mon[102832]: pgmap v4193: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:25:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:21 smithi082 ceph-mon[98694]: pgmap v4193: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:25:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:25:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:25:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:25:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:23 smithi098 ceph-mon[103124]: pgmap v4194: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:25:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:25:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:25:24.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:25:24.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7555.134310 secs 2023-09-10T01:25:24.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7555.846160 secs 2023-09-10T01:25:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:23 smithi170 ceph-mon[102832]: pgmap v4194: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:25:24.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:25:24.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:25:24.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:25:24.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7555.134310 secs 2023-09-10T01:25:24.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7555.846160 secs 2023-09-10T01:25:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:23 smithi082 ceph-mon[98694]: pgmap v4194: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:25:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:25:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:25:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:25:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7555.134310 secs 2023-09-10T01:25:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7555.846160 secs 2023-09-10T01:25:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:25 smithi098 ceph-mon[103124]: pgmap v4195: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:25:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:25 smithi170 ceph-mon[102832]: pgmap v4195: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:25:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:25 smithi082 ceph-mon[98694]: pgmap v4195: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:25:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:27 smithi098 ceph-mon[103124]: pgmap v4196: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 33 KiB/s wr, 11 op/s 2023-09-10T01:25:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:27 smithi170 ceph-mon[102832]: pgmap v4196: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 33 KiB/s wr, 11 op/s 2023-09-10T01:25:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:27 smithi082 ceph-mon[98694]: pgmap v4196: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 33 KiB/s wr, 11 op/s 2023-09-10T01:25:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7560.134373 secs 2023-09-10T01:25:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7560.846227 secs 2023-09-10T01:25:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7560.134373 secs 2023-09-10T01:25:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7560.846227 secs 2023-09-10T01:25:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7560.134373 secs 2023-09-10T01:25:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7560.846227 secs 2023-09-10T01:25:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:29 smithi098 ceph-mon[103124]: pgmap v4197: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 30 KiB/s wr, 10 op/s 2023-09-10T01:25:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:29 smithi170 ceph-mon[102832]: pgmap v4197: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 30 KiB/s wr, 10 op/s 2023-09-10T01:25:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:29 smithi082 ceph-mon[98694]: pgmap v4197: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 30 KiB/s wr, 10 op/s 2023-09-10T01:25:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:31 smithi098 ceph-mon[103124]: pgmap v4198: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 75 KiB/s wr, 16 op/s 2023-09-10T01:25:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:31 smithi170 ceph-mon[102832]: pgmap v4198: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 75 KiB/s wr, 16 op/s 2023-09-10T01:25:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:31 smithi082 ceph-mon[98694]: pgmap v4198: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 75 KiB/s wr, 16 op/s 2023-09-10T01:25:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:33 smithi098 ceph-mon[103124]: pgmap v4199: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T01:25:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7565.134433 secs 2023-09-10T01:25:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7565.846292 secs 2023-09-10T01:25:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:33 smithi170 ceph-mon[102832]: pgmap v4199: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T01:25:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7565.134433 secs 2023-09-10T01:25:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7565.846292 secs 2023-09-10T01:25:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:33 smithi082 ceph-mon[98694]: pgmap v4199: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T01:25:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7565.134433 secs 2023-09-10T01:25:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7565.846292 secs 2023-09-10T01:25:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:35 smithi098 ceph-mon[103124]: pgmap v4200: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T01:25:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:35 smithi170 ceph-mon[102832]: pgmap v4200: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T01:25:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:35 smithi082 ceph-mon[98694]: pgmap v4200: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T01:25:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:37 smithi098 ceph-mon[103124]: pgmap v4201: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 61 KiB/s wr, 17 op/s 2023-09-10T01:25:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:37 smithi170 ceph-mon[102832]: pgmap v4201: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 61 KiB/s wr, 17 op/s 2023-09-10T01:25:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:37 smithi082 ceph-mon[98694]: pgmap v4201: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 61 KiB/s wr, 17 op/s 2023-09-10T01:25:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7570.134532 secs 2023-09-10T01:25:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7570.846358 secs 2023-09-10T01:25:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7570.134532 secs 2023-09-10T01:25:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7570.846358 secs 2023-09-10T01:25:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7570.134532 secs 2023-09-10T01:25:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7570.846358 secs 2023-09-10T01:25:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:39 smithi098 ceph-mon[103124]: pgmap v4202: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 12 op/s 2023-09-10T01:25:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:39 smithi170 ceph-mon[102832]: pgmap v4202: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 12 op/s 2023-09-10T01:25:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:39 smithi082 ceph-mon[98694]: pgmap v4202: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 12 op/s 2023-09-10T01:25:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:41 smithi098 ceph-mon[103124]: pgmap v4203: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 98 KiB/s wr, 17 op/s 2023-09-10T01:25:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:41 smithi170 ceph-mon[102832]: pgmap v4203: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 98 KiB/s wr, 17 op/s 2023-09-10T01:25:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:41 smithi082 ceph-mon[98694]: pgmap v4203: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 98 KiB/s wr, 17 op/s 2023-09-10T01:25:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:43 smithi098 ceph-mon[103124]: pgmap v4204: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T01:25:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7575.134616 secs 2023-09-10T01:25:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7575.846421 secs 2023-09-10T01:25:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:43 smithi170 ceph-mon[102832]: pgmap v4204: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T01:25:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7575.134616 secs 2023-09-10T01:25:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7575.846421 secs 2023-09-10T01:25:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:43 smithi082 ceph-mon[98694]: pgmap v4204: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T01:25:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7575.134616 secs 2023-09-10T01:25:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7575.846421 secs 2023-09-10T01:25:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:45 smithi098 ceph-mon[103124]: pgmap v4205: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T01:25:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:45 smithi170 ceph-mon[102832]: pgmap v4205: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T01:25:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:45 smithi082 ceph-mon[98694]: pgmap v4205: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T01:25:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:47 smithi098 ceph-mon[103124]: pgmap v4206: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 61 KiB/s wr, 16 op/s 2023-09-10T01:25:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7578.652049 secs 2023-09-10T01:25:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:47 smithi170 ceph-mon[102832]: pgmap v4206: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 61 KiB/s wr, 16 op/s 2023-09-10T01:25:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7578.652049 secs 2023-09-10T01:25:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:47 smithi082 ceph-mon[98694]: pgmap v4206: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 61 KiB/s wr, 16 op/s 2023-09-10T01:25:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7578.652049 secs 2023-09-10T01:25:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7580.134674 secs 2023-09-10T01:25:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7580.846492 secs 2023-09-10T01:25:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7580.134674 secs 2023-09-10T01:25:49.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7580.846492 secs 2023-09-10T01:25:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7580.134674 secs 2023-09-10T01:25:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7580.846492 secs 2023-09-10T01:25:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:49 smithi170 ceph-mon[102832]: pgmap v4207: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-10T01:25:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:49 smithi082 ceph-mon[98694]: pgmap v4207: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-10T01:25:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:49 smithi098 ceph-mon[103124]: pgmap v4207: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-10T01:25:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:51 smithi170 ceph-mon[102832]: pgmap v4208: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 85 KiB/s wr, 16 op/s 2023-09-10T01:25:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:51 smithi082 ceph-mon[98694]: pgmap v4208: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 85 KiB/s wr, 16 op/s 2023-09-10T01:25:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:51 smithi098 ceph-mon[103124]: pgmap v4208: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 85 KiB/s wr, 16 op/s 2023-09-10T01:25:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:25:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:25:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:25:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7583.652152 secs 2023-09-10T01:25:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:25:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:25:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:25:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7583.652152 secs 2023-09-10T01:25:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:25:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:25:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:25:53.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7583.652152 secs 2023-09-10T01:25:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:53 smithi170 ceph-mon[102832]: pgmap v4209: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T01:25:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:53 smithi082 ceph-mon[98694]: pgmap v4209: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T01:25:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:53 smithi098 ceph-mon[103124]: pgmap v4209: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T01:25:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7585.134731 secs 2023-09-10T01:25:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7585.846557 secs 2023-09-10T01:25:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7585.134731 secs 2023-09-10T01:25:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7585.846557 secs 2023-09-10T01:25:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7585.134731 secs 2023-09-10T01:25:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7585.846557 secs 2023-09-10T01:25:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:55 smithi170 ceph-mon[102832]: pgmap v4210: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T01:25:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:55 smithi082 ceph-mon[98694]: pgmap v4210: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T01:25:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:55 smithi098 ceph-mon[103124]: pgmap v4210: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T01:25:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:57 smithi170 ceph-mon[102832]: pgmap v4211: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 47 KiB/s wr, 15 op/s 2023-09-10T01:25:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7588.652213 secs 2023-09-10T01:25:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:57 smithi082 ceph-mon[98694]: pgmap v4211: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 47 KiB/s wr, 15 op/s 2023-09-10T01:25:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7588.652213 secs 2023-09-10T01:25:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:57 smithi098 ceph-mon[103124]: pgmap v4211: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 47 KiB/s wr, 15 op/s 2023-09-10T01:25:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7588.652213 secs 2023-09-10T01:25:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7590.134817 secs 2023-09-10T01:25:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:59 smithi170 ceph-mon[102832]: pgmap v4212: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 10 op/s 2023-09-10T01:25:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:25:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7590.846622 secs 2023-09-10T01:25:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7590.134817 secs 2023-09-10T01:25:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:59 smithi082 ceph-mon[98694]: pgmap v4212: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 10 op/s 2023-09-10T01:25:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:25:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7590.846622 secs 2023-09-10T01:26:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7590.134817 secs 2023-09-10T01:26:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:59 smithi098 ceph-mon[103124]: pgmap v4212: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 10 op/s 2023-09-10T01:26:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:25:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7590.846622 secs 2023-09-10T01:26:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:01 smithi170 ceph-mon[102832]: pgmap v4213: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 92 KiB/s wr, 30 op/s 2023-09-10T01:26:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:01 smithi082 ceph-mon[98694]: pgmap v4213: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 92 KiB/s wr, 30 op/s 2023-09-10T01:26:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:01 smithi098 ceph-mon[103124]: pgmap v4213: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 92 KiB/s wr, 30 op/s 2023-09-10T01:26:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7593.652336 secs 2023-09-10T01:26:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7593.652336 secs 2023-09-10T01:26:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7593.652336 secs 2023-09-10T01:26:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:03 smithi170 ceph-mon[102832]: pgmap v4214: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 60 KiB/s wr, 24 op/s 2023-09-10T01:26:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:03 smithi082 ceph-mon[98694]: pgmap v4214: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 60 KiB/s wr, 24 op/s 2023-09-10T01:26:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:03 smithi098 ceph-mon[103124]: pgmap v4214: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 60 KiB/s wr, 24 op/s 2023-09-10T01:26:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7595.134900 secs 2023-09-10T01:26:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7595.846688 secs 2023-09-10T01:26:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7595.134900 secs 2023-09-10T01:26:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7595.846688 secs 2023-09-10T01:26:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7595.134900 secs 2023-09-10T01:26:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7595.846688 secs 2023-09-10T01:26:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:05 smithi170 ceph-mon[102832]: pgmap v4215: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 60 KiB/s wr, 24 op/s 2023-09-10T01:26:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:05 smithi082 ceph-mon[98694]: pgmap v4215: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 60 KiB/s wr, 24 op/s 2023-09-10T01:26:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:05 smithi098 ceph-mon[103124]: pgmap v4215: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 60 KiB/s wr, 24 op/s 2023-09-10T01:26:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:07 smithi170 ceph-mon[102832]: pgmap v4216: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 74 KiB/s wr, 29 op/s 2023-09-10T01:26:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7598.652395 secs 2023-09-10T01:26:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:07 smithi082 ceph-mon[98694]: pgmap v4216: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 74 KiB/s wr, 29 op/s 2023-09-10T01:26:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7598.652395 secs 2023-09-10T01:26:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:07 smithi098 ceph-mon[103124]: pgmap v4216: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 74 KiB/s wr, 29 op/s 2023-09-10T01:26:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7598.652395 secs 2023-09-10T01:26:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:08 smithi170 ceph-mon[102832]: osdmap e333: 12 total, 12 up, 12 in 2023-09-10T01:26:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:08 smithi082 ceph-mon[98694]: osdmap e333: 12 total, 12 up, 12 in 2023-09-10T01:26:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:08 smithi098 ceph-mon[103124]: osdmap e333: 12 total, 12 up, 12 in 2023-09-10T01:26:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7600.134983 secs 2023-09-10T01:26:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:09 smithi170 ceph-mon[102832]: pgmap v4218: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T01:26:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7600.846753 secs 2023-09-10T01:26:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7600.134983 secs 2023-09-10T01:26:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:09 smithi082 ceph-mon[98694]: pgmap v4218: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T01:26:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7600.846753 secs 2023-09-10T01:26:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7600.134983 secs 2023-09-10T01:26:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:09 smithi098 ceph-mon[103124]: pgmap v4218: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T01:26:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7600.846753 secs 2023-09-10T01:26:11.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:11 smithi170 ceph-mon[102832]: pgmap v4219: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-10T01:26:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:11 smithi082 ceph-mon[98694]: pgmap v4219: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-10T01:26:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:11 smithi098 ceph-mon[103124]: pgmap v4219: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-10T01:26:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7603.652481 secs 2023-09-10T01:26:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7603.652481 secs 2023-09-10T01:26:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7603.652481 secs 2023-09-10T01:26:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:13 smithi170 ceph-mon[102832]: pgmap v4220: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-10T01:26:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:13 smithi082 ceph-mon[98694]: pgmap v4220: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-10T01:26:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:13 smithi098 ceph-mon[103124]: pgmap v4220: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-10T01:26:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7605.135039 secs 2023-09-10T01:26:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7605.846820 secs 2023-09-10T01:26:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:26:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7605.135039 secs 2023-09-10T01:26:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7605.846820 secs 2023-09-10T01:26:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:26:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7605.135039 secs 2023-09-10T01:26:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7605.846820 secs 2023-09-10T01:26:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:26:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:15 smithi170 ceph-mon[102832]: pgmap v4221: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-10T01:26:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:26:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:26:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:26:15.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:15 smithi170 ceph-mon[102832]: osdmap e334: 12 total, 12 up, 12 in 2023-09-10T01:26:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:15 smithi082 ceph-mon[98694]: pgmap v4221: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-10T01:26:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:26:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:26:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:26:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:15 smithi082 ceph-mon[98694]: osdmap e334: 12 total, 12 up, 12 in 2023-09-10T01:26:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:15 smithi098 ceph-mon[103124]: pgmap v4221: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-10T01:26:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:26:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:26:16.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:26:16.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:15 smithi098 ceph-mon[103124]: osdmap e334: 12 total, 12 up, 12 in 2023-09-10T01:26:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:17 smithi170 ceph-mon[102832]: pgmap v4223: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 13 op/s 2023-09-10T01:26:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7608.652543 secs 2023-09-10T01:26:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:17 smithi082 ceph-mon[98694]: pgmap v4223: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 13 op/s 2023-09-10T01:26:17.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7608.652543 secs 2023-09-10T01:26:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:17 smithi098 ceph-mon[103124]: pgmap v4223: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 13 op/s 2023-09-10T01:26:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7608.652543 secs 2023-09-10T01:26:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7610.135112 secs 2023-09-10T01:26:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:19 smithi170 ceph-mon[102832]: pgmap v4224: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 12 op/s 2023-09-10T01:26:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7610.846888 secs 2023-09-10T01:26:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7610.135112 secs 2023-09-10T01:26:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:19 smithi082 ceph-mon[98694]: pgmap v4224: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 12 op/s 2023-09-10T01:26:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7610.846888 secs 2023-09-10T01:26:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7610.135112 secs 2023-09-10T01:26:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:19 smithi098 ceph-mon[103124]: pgmap v4224: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 12 op/s 2023-09-10T01:26:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7610.846888 secs 2023-09-10T01:26:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:21 smithi170 ceph-mon[102832]: pgmap v4225: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T01:26:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:21 smithi082 ceph-mon[98694]: pgmap v4225: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T01:26:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:21 smithi098 ceph-mon[103124]: pgmap v4225: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T01:26:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:23 smithi170 ceph-mon[102832]: pgmap v4226: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T01:26:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:26:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:23 smithi082 ceph-mon[98694]: pgmap v4226: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T01:26:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:26:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:23 smithi098 ceph-mon[103124]: pgmap v4226: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T01:26:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:26:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7615.135170 secs 2023-09-10T01:26:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7615.846951 secs 2023-09-10T01:26:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:26:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:26:24.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:26:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7615.135170 secs 2023-09-10T01:26:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7615.846951 secs 2023-09-10T01:26:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:26:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:26:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:26:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7615.135170 secs 2023-09-10T01:26:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7615.846951 secs 2023-09-10T01:26:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:26:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:26:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:26:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:25 smithi170 ceph-mon[102832]: pgmap v4227: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T01:26:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:25 smithi082 ceph-mon[98694]: pgmap v4227: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T01:26:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:25 smithi098 ceph-mon[103124]: pgmap v4227: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T01:26:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:26:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:26:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:26:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:27 smithi170 ceph-mon[102832]: pgmap v4228: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 47 KiB/s wr, 26 op/s 2023-09-10T01:26:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:27 smithi082 ceph-mon[98694]: pgmap v4228: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 47 KiB/s wr, 26 op/s 2023-09-10T01:26:28.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:27 smithi098 ceph-mon[103124]: pgmap v4228: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 47 KiB/s wr, 26 op/s 2023-09-10T01:26:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7620.135228 secs 2023-09-10T01:26:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:29 smithi170 ceph-mon[102832]: pgmap v4229: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 43 KiB/s wr, 24 op/s 2023-09-10T01:26:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7620.847017 secs 2023-09-10T01:26:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7620.135228 secs 2023-09-10T01:26:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:29 smithi082 ceph-mon[98694]: pgmap v4229: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 43 KiB/s wr, 24 op/s 2023-09-10T01:26:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7620.847017 secs 2023-09-10T01:26:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7620.135228 secs 2023-09-10T01:26:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:29 smithi098 ceph-mon[103124]: pgmap v4229: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 43 KiB/s wr, 24 op/s 2023-09-10T01:26:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7620.847017 secs 2023-09-10T01:26:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:31 smithi170 ceph-mon[102832]: pgmap v4230: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 139 KiB/s wr, 33 op/s 2023-09-10T01:26:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:31 smithi082 ceph-mon[98694]: pgmap v4230: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 139 KiB/s wr, 33 op/s 2023-09-10T01:26:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:31 smithi098 ceph-mon[103124]: pgmap v4230: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 139 KiB/s wr, 33 op/s 2023-09-10T01:26:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:33 smithi170 ceph-mon[102832]: pgmap v4231: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 116 KiB/s wr, 28 op/s 2023-09-10T01:26:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:33 smithi082 ceph-mon[98694]: pgmap v4231: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 116 KiB/s wr, 28 op/s 2023-09-10T01:26:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:33 smithi098 ceph-mon[103124]: pgmap v4231: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 116 KiB/s wr, 28 op/s 2023-09-10T01:26:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7625.135287 secs 2023-09-10T01:26:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7625.847083 secs 2023-09-10T01:26:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7625.135287 secs 2023-09-10T01:26:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7625.847083 secs 2023-09-10T01:26:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7625.135287 secs 2023-09-10T01:26:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7625.847083 secs 2023-09-10T01:26:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:35 smithi170 ceph-mon[102832]: pgmap v4232: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 116 KiB/s wr, 28 op/s 2023-09-10T01:26:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:35 smithi082 ceph-mon[98694]: pgmap v4232: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 116 KiB/s wr, 28 op/s 2023-09-10T01:26:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:35 smithi098 ceph-mon[103124]: pgmap v4232: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 116 KiB/s wr, 28 op/s 2023-09-10T01:26:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:37 smithi170 ceph-mon[102832]: pgmap v4233: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 122 KiB/s wr, 33 op/s 2023-09-10T01:26:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:37 smithi082 ceph-mon[98694]: pgmap v4233: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 122 KiB/s wr, 33 op/s 2023-09-10T01:26:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:37 smithi098 ceph-mon[103124]: pgmap v4233: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 122 KiB/s wr, 33 op/s 2023-09-10T01:26:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7630.135345 secs 2023-09-10T01:26:39.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:39 smithi170 ceph-mon[102832]: pgmap v4234: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 14 op/s 2023-09-10T01:26:39.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7630.847146 secs 2023-09-10T01:26:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7630.135345 secs 2023-09-10T01:26:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:39 smithi082 ceph-mon[98694]: pgmap v4234: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 14 op/s 2023-09-10T01:26:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7630.847146 secs 2023-09-10T01:26:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7630.135345 secs 2023-09-10T01:26:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:39 smithi098 ceph-mon[103124]: pgmap v4234: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 14 op/s 2023-09-10T01:26:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7630.847146 secs 2023-09-10T01:26:41.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:41 smithi170 ceph-mon[102832]: pgmap v4235: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 129 KiB/s wr, 20 op/s 2023-09-10T01:26:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:41 smithi082 ceph-mon[98694]: pgmap v4235: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 129 KiB/s wr, 20 op/s 2023-09-10T01:26:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:41 smithi098 ceph-mon[103124]: pgmap v4235: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 129 KiB/s wr, 20 op/s 2023-09-10T01:26:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:43 smithi170 ceph-mon[102832]: pgmap v4236: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 33 KiB/s wr, 11 op/s 2023-09-10T01:26:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:43 smithi082 ceph-mon[98694]: pgmap v4236: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 33 KiB/s wr, 11 op/s 2023-09-10T01:26:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:43 smithi098 ceph-mon[103124]: pgmap v4236: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 33 KiB/s wr, 11 op/s 2023-09-10T01:26:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7635.135403 secs 2023-09-10T01:26:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7635.847210 secs 2023-09-10T01:26:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7635.135403 secs 2023-09-10T01:26:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7635.847210 secs 2023-09-10T01:26:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7635.135403 secs 2023-09-10T01:26:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7635.847210 secs 2023-09-10T01:26:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:45 smithi170 ceph-mon[102832]: pgmap v4237: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 33 KiB/s wr, 11 op/s 2023-09-10T01:26:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:45 smithi082 ceph-mon[98694]: pgmap v4237: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 33 KiB/s wr, 11 op/s 2023-09-10T01:26:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:45 smithi098 ceph-mon[103124]: pgmap v4237: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 33 KiB/s wr, 11 op/s 2023-09-10T01:26:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:47 smithi170 ceph-mon[102832]: pgmap v4238: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 59 KiB/s wr, 17 op/s 2023-09-10T01:26:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7638.653107 secs 2023-09-10T01:26:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:47 smithi082 ceph-mon[98694]: pgmap v4238: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 59 KiB/s wr, 17 op/s 2023-09-10T01:26:47.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7638.653107 secs 2023-09-10T01:26:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:47 smithi098 ceph-mon[103124]: pgmap v4238: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 59 KiB/s wr, 17 op/s 2023-09-10T01:26:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7638.653107 secs 2023-09-10T01:26:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7640.135459 secs 2023-09-10T01:26:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:49 smithi170 ceph-mon[102832]: pgmap v4239: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T01:26:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7640.847277 secs 2023-09-10T01:26:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7640.135459 secs 2023-09-10T01:26:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:49 smithi082 ceph-mon[98694]: pgmap v4239: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T01:26:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7640.847277 secs 2023-09-10T01:26:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7640.135459 secs 2023-09-10T01:26:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:49 smithi098 ceph-mon[103124]: pgmap v4239: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T01:26:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7640.847277 secs 2023-09-10T01:26:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:51 smithi082 ceph-mon[98694]: pgmap v4240: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 68 KiB/s wr, 16 op/s 2023-09-10T01:26:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:51 smithi098 ceph-mon[103124]: pgmap v4240: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 68 KiB/s wr, 16 op/s 2023-09-10T01:26:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:51 smithi170 ceph-mon[102832]: pgmap v4240: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 68 KiB/s wr, 16 op/s 2023-09-10T01:26:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:26:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:26:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:26:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7643.653187 secs 2023-09-10T01:26:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:26:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:26:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:26:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7643.653187 secs 2023-09-10T01:26:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:26:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:26:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:26:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7643.653187 secs 2023-09-10T01:26:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:53 smithi082 ceph-mon[98694]: pgmap v4241: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 10 op/s 2023-09-10T01:26:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7645.135502 secs 2023-09-10T01:26:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:53 smithi098 ceph-mon[103124]: pgmap v4241: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 10 op/s 2023-09-10T01:26:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7645.135502 secs 2023-09-10T01:26:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:53 smithi170 ceph-mon[102832]: pgmap v4241: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 10 op/s 2023-09-10T01:26:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7645.135502 secs 2023-09-10T01:26:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7645.847344 secs 2023-09-10T01:26:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7645.847344 secs 2023-09-10T01:26:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7645.847344 secs 2023-09-10T01:26:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:55 smithi082 ceph-mon[98694]: pgmap v4242: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 10 op/s 2023-09-10T01:26:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:55 smithi098 ceph-mon[103124]: pgmap v4242: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 10 op/s 2023-09-10T01:26:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:55 smithi170 ceph-mon[102832]: pgmap v4242: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 10 op/s 2023-09-10T01:26:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:57 smithi082 ceph-mon[98694]: pgmap v4243: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 48 KiB/s wr, 15 op/s 2023-09-10T01:26:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7648.653309 secs 2023-09-10T01:26:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:57 smithi098 ceph-mon[103124]: pgmap v4243: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 48 KiB/s wr, 15 op/s 2023-09-10T01:26:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7648.653309 secs 2023-09-10T01:26:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:57 smithi170 ceph-mon[102832]: pgmap v4243: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 48 KiB/s wr, 15 op/s 2023-09-10T01:26:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7648.653309 secs 2023-09-10T01:26:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7650.135559 secs 2023-09-10T01:26:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7650.847409 secs 2023-09-10T01:26:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7650.135559 secs 2023-09-10T01:26:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7650.847409 secs 2023-09-10T01:26:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7650.135559 secs 2023-09-10T01:26:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7650.847409 secs 2023-09-10T01:26:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:26:59 smithi082 ceph-mon[98694]: pgmap v4244: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 22 KiB/s wr, 10 op/s 2023-09-10T01:27:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:26:59 smithi098 ceph-mon[103124]: pgmap v4244: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 22 KiB/s wr, 10 op/s 2023-09-10T01:27:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:26:59 smithi170 ceph-mon[102832]: pgmap v4244: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 22 KiB/s wr, 10 op/s 2023-09-10T01:27:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:01 smithi098 ceph-mon[103124]: pgmap v4245: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 65 KiB/s wr, 25 op/s 2023-09-10T01:27:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:01 smithi170 ceph-mon[102832]: pgmap v4245: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 65 KiB/s wr, 25 op/s 2023-09-10T01:27:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:01 smithi082 ceph-mon[98694]: pgmap v4245: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 65 KiB/s wr, 25 op/s 2023-09-10T01:27:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7653.653397 secs 2023-09-10T01:27:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7653.653397 secs 2023-09-10T01:27:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7653.653397 secs 2023-09-10T01:27:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:03 smithi098 ceph-mon[103124]: pgmap v4246: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 49 KiB/s wr, 20 op/s 2023-09-10T01:27:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7655.135617 secs 2023-09-10T01:27:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7655.847480 secs 2023-09-10T01:27:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:03 smithi170 ceph-mon[102832]: pgmap v4246: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 49 KiB/s wr, 20 op/s 2023-09-10T01:27:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7655.135617 secs 2023-09-10T01:27:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7655.847480 secs 2023-09-10T01:27:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:03 smithi082 ceph-mon[98694]: pgmap v4246: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 49 KiB/s wr, 20 op/s 2023-09-10T01:27:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7655.135617 secs 2023-09-10T01:27:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7655.847480 secs 2023-09-10T01:27:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:05 smithi098 ceph-mon[103124]: pgmap v4247: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 49 KiB/s wr, 20 op/s 2023-09-10T01:27:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:05 smithi170 ceph-mon[102832]: pgmap v4247: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 49 KiB/s wr, 20 op/s 2023-09-10T01:27:06.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:05 smithi082 ceph-mon[98694]: pgmap v4247: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 49 KiB/s wr, 20 op/s 2023-09-10T01:27:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:07 smithi098 ceph-mon[103124]: pgmap v4248: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 74 KiB/s wr, 29 op/s 2023-09-10T01:27:08.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7658.653455 secs 2023-09-10T01:27:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:07 smithi170 ceph-mon[102832]: pgmap v4248: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 74 KiB/s wr, 29 op/s 2023-09-10T01:27:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7658.653455 secs 2023-09-10T01:27:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:07 smithi082 ceph-mon[98694]: pgmap v4248: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 74 KiB/s wr, 29 op/s 2023-09-10T01:27:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7658.653455 secs 2023-09-10T01:27:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:08 smithi098 ceph-mon[103124]: osdmap e335: 12 total, 12 up, 12 in 2023-09-10T01:27:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7660.135676 secs 2023-09-10T01:27:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7660.847544 secs 2023-09-10T01:27:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:08 smithi170 ceph-mon[102832]: osdmap e335: 12 total, 12 up, 12 in 2023-09-10T01:27:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7660.135676 secs 2023-09-10T01:27:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7660.847544 secs 2023-09-10T01:27:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:08 smithi082 ceph-mon[98694]: osdmap e335: 12 total, 12 up, 12 in 2023-09-10T01:27:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7660.135676 secs 2023-09-10T01:27:09.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7660.847544 secs 2023-09-10T01:27:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:09 smithi098 ceph-mon[103124]: pgmap v4250: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T01:27:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:09 smithi170 ceph-mon[102832]: pgmap v4250: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T01:27:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:09 smithi082 ceph-mon[98694]: pgmap v4250: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T01:27:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:11 smithi098 ceph-mon[103124]: pgmap v4251: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 31 KiB/s wr, 16 op/s 2023-09-10T01:27:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:11 smithi170 ceph-mon[102832]: pgmap v4251: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 31 KiB/s wr, 16 op/s 2023-09-10T01:27:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:11 smithi082 ceph-mon[98694]: pgmap v4251: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 31 KiB/s wr, 16 op/s 2023-09-10T01:27:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7663.653517 secs 2023-09-10T01:27:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7663.653517 secs 2023-09-10T01:27:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7663.653517 secs 2023-09-10T01:27:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:13 smithi098 ceph-mon[103124]: pgmap v4252: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 31 KiB/s wr, 16 op/s 2023-09-10T01:27:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7665.135756 secs 2023-09-10T01:27:14.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7665.847613 secs 2023-09-10T01:27:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:13 smithi170 ceph-mon[102832]: pgmap v4252: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 31 KiB/s wr, 16 op/s 2023-09-10T01:27:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7665.135756 secs 2023-09-10T01:27:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7665.847613 secs 2023-09-10T01:27:14.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:13 smithi082 ceph-mon[98694]: pgmap v4252: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 31 KiB/s wr, 16 op/s 2023-09-10T01:27:14.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7665.135756 secs 2023-09-10T01:27:14.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7665.847613 secs 2023-09-10T01:27:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:15 smithi098 ceph-mon[103124]: pgmap v4253: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 31 KiB/s wr, 16 op/s 2023-09-10T01:27:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:27:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:15 smithi098 ceph-mon[103124]: osdmap e336: 12 total, 12 up, 12 in 2023-09-10T01:27:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:15 smithi170 ceph-mon[102832]: pgmap v4253: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 31 KiB/s wr, 16 op/s 2023-09-10T01:27:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:27:16.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:15 smithi170 ceph-mon[102832]: osdmap e336: 12 total, 12 up, 12 in 2023-09-10T01:27:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:15 smithi082 ceph-mon[98694]: pgmap v4253: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 31 KiB/s wr, 16 op/s 2023-09-10T01:27:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:27:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:15 smithi082 ceph-mon[98694]: osdmap e336: 12 total, 12 up, 12 in 2023-09-10T01:27:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:17 smithi098 ceph-mon[103124]: pgmap v4255: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-09-10T01:27:18.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7668.653579 secs 2023-09-10T01:27:18.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:27:18.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:27:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:17 smithi170 ceph-mon[102832]: pgmap v4255: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-09-10T01:27:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7668.653579 secs 2023-09-10T01:27:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:27:18.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:27:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:17 smithi082 ceph-mon[98694]: pgmap v4255: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-09-10T01:27:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7668.653579 secs 2023-09-10T01:27:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:27:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:27:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:27:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:27:19.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7670.135984 secs 2023-09-10T01:27:19.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:19.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:19.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:19.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:19.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:27:19.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:27:19.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:19.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:19.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:19.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:19.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7670.847681 secs 2023-09-10T01:27:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:27:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:27:19.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7670.135984 secs 2023-09-10T01:27:19.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:19.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:19.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:19.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:19.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:27:19.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:27:19.212 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:19.212 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:19.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:19.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:19.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7670.847681 secs 2023-09-10T01:27:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:27:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:27:19.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7670.135984 secs 2023-09-10T01:27:19.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:19.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:19.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:19.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:19.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:27:19.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:27:19.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:19.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:19.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:19.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:19.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7670.847681 secs 2023-09-10T01:27:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:19 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T01:27:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:19 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T01:27:20.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:19 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T01:27:20.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:19 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:27:20.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:19 smithi170 ceph-mon[102832]: pgmap v4256: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 11 op/s 2023-09-10T01:27:20.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:20.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:20.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:20.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:20.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:27:20.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:27:20.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:27:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:19 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T01:27:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:19 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T01:27:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:19 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T01:27:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:19 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:27:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:19 smithi082 ceph-mon[98694]: pgmap v4256: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 11 op/s 2023-09-10T01:27:20.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:20.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:20.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:20.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:20.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:27:20.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:27:20.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:27:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:19 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T01:27:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:19 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T01:27:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:19 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T01:27:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:19 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:27:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:19 smithi098 ceph-mon[103124]: pgmap v4256: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 11 op/s 2023-09-10T01:27:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:20.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:20.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:27:20.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:27:20.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:27:20.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:27:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:20 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T01:27:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:20 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T01:27:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:20 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T01:27:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:20 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T01:27:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:20 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T01:27:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:20 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T01:27:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:21 smithi170 ceph-mon[102832]: pgmap v4257: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:27:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:21 smithi082 ceph-mon[98694]: pgmap v4257: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:27:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:21 smithi098 ceph-mon[103124]: pgmap v4257: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:27:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:22 smithi170 ceph-mon[102832]: pgmap v4258: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:27:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:27:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:22 smithi082 ceph-mon[98694]: pgmap v4258: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:27:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:27:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:22 smithi098 ceph-mon[103124]: pgmap v4258: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:27:23.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:27:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:27:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:27:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:27:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7675.136055 secs 2023-09-10T01:27:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7675.847747 secs 2023-09-10T01:27:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:27:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:27:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:27:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7675.136055 secs 2023-09-10T01:27:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7675.847747 secs 2023-09-10T01:27:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:27:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:27:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:27:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7675.136055 secs 2023-09-10T01:27:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7675.847747 secs 2023-09-10T01:27:25.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:24 smithi170 ceph-mon[102832]: pgmap v4259: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:27:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:24 smithi082 ceph-mon[98694]: pgmap v4259: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:27:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:24 smithi098 ceph-mon[103124]: pgmap v4259: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:27:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:27 smithi170 ceph-mon[102832]: pgmap v4260: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 21 KiB/s wr, 10 op/s 2023-09-10T01:27:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:27 smithi082 ceph-mon[98694]: pgmap v4260: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 21 KiB/s wr, 10 op/s 2023-09-10T01:27:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:27 smithi098 ceph-mon[103124]: pgmap v4260: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 21 KiB/s wr, 10 op/s 2023-09-10T01:27:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7680.136110 secs 2023-09-10T01:27:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:29 smithi170 ceph-mon[102832]: pgmap v4261: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 19 KiB/s wr, 9 op/s 2023-09-10T01:27:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7680.847812 secs 2023-09-10T01:27:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7680.136110 secs 2023-09-10T01:27:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:29 smithi082 ceph-mon[98694]: pgmap v4261: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 19 KiB/s wr, 9 op/s 2023-09-10T01:27:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7680.847812 secs 2023-09-10T01:27:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7680.136110 secs 2023-09-10T01:27:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:29 smithi098 ceph-mon[103124]: pgmap v4261: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 19 KiB/s wr, 9 op/s 2023-09-10T01:27:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7680.847812 secs 2023-09-10T01:27:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:31 smithi170 ceph-mon[102832]: pgmap v4262: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 64 KiB/s wr, 15 op/s 2023-09-10T01:27:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:31 smithi082 ceph-mon[98694]: pgmap v4262: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 64 KiB/s wr, 15 op/s 2023-09-10T01:27:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:31 smithi098 ceph-mon[103124]: pgmap v4262: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 64 KiB/s wr, 15 op/s 2023-09-10T01:27:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:33 smithi170 ceph-mon[102832]: pgmap v4263: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T01:27:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:33 smithi082 ceph-mon[98694]: pgmap v4263: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T01:27:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:33 smithi098 ceph-mon[103124]: pgmap v4263: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T01:27:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7685.136170 secs 2023-09-10T01:27:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7685.847875 secs 2023-09-10T01:27:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7685.136170 secs 2023-09-10T01:27:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7685.847875 secs 2023-09-10T01:27:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7685.136170 secs 2023-09-10T01:27:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7685.847875 secs 2023-09-10T01:27:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:35 smithi170 ceph-mon[102832]: pgmap v4264: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T01:27:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:35 smithi082 ceph-mon[98694]: pgmap v4264: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T01:27:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:35 smithi098 ceph-mon[103124]: pgmap v4264: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T01:27:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:37 smithi170 ceph-mon[102832]: pgmap v4265: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 110 KiB/s wr, 17 op/s 2023-09-10T01:27:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:37 smithi082 ceph-mon[98694]: pgmap v4265: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 110 KiB/s wr, 17 op/s 2023-09-10T01:27:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:37 smithi098 ceph-mon[103124]: pgmap v4265: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 110 KiB/s wr, 17 op/s 2023-09-10T01:27:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7690.136256 secs 2023-09-10T01:27:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:39 smithi170 ceph-mon[102832]: pgmap v4266: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T01:27:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7690.847942 secs 2023-09-10T01:27:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7690.136256 secs 2023-09-10T01:27:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:39 smithi082 ceph-mon[98694]: pgmap v4266: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T01:27:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7690.847942 secs 2023-09-10T01:27:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7690.136256 secs 2023-09-10T01:27:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:39 smithi098 ceph-mon[103124]: pgmap v4266: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T01:27:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7690.847942 secs 2023-09-10T01:27:41.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:41 smithi170 ceph-mon[102832]: pgmap v4267: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 102 KiB/s wr, 18 op/s 2023-09-10T01:27:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:41 smithi082 ceph-mon[98694]: pgmap v4267: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 102 KiB/s wr, 18 op/s 2023-09-10T01:27:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:41 smithi098 ceph-mon[103124]: pgmap v4267: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 102 KiB/s wr, 18 op/s 2023-09-10T01:27:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:43 smithi170 ceph-mon[102832]: pgmap v4268: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T01:27:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:43 smithi082 ceph-mon[98694]: pgmap v4268: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T01:27:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:43 smithi098 ceph-mon[103124]: pgmap v4268: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T01:27:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7695.136339 secs 2023-09-10T01:27:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7695.848007 secs 2023-09-10T01:27:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7695.136339 secs 2023-09-10T01:27:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7695.848007 secs 2023-09-10T01:27:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7695.136339 secs 2023-09-10T01:27:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7695.848007 secs 2023-09-10T01:27:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:45 smithi170 ceph-mon[102832]: pgmap v4269: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T01:27:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:45 smithi082 ceph-mon[98694]: pgmap v4269: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T01:27:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:45 smithi098 ceph-mon[103124]: pgmap v4269: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T01:27:47.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:47 smithi170 ceph-mon[102832]: pgmap v4270: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 17 op/s 2023-09-10T01:27:47.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7698.653974 secs 2023-09-10T01:27:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:47 smithi082 ceph-mon[98694]: pgmap v4270: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 17 op/s 2023-09-10T01:27:47.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7698.653974 secs 2023-09-10T01:27:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:47 smithi098 ceph-mon[103124]: pgmap v4270: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 17 op/s 2023-09-10T01:27:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7698.653974 secs 2023-09-10T01:27:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7700.136395 secs 2023-09-10T01:27:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:49 smithi170 ceph-mon[102832]: pgmap v4271: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T01:27:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7700.848072 secs 2023-09-10T01:27:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7700.136395 secs 2023-09-10T01:27:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:49 smithi082 ceph-mon[98694]: pgmap v4271: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T01:27:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7700.848072 secs 2023-09-10T01:27:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7700.136395 secs 2023-09-10T01:27:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:49 smithi098 ceph-mon[103124]: pgmap v4271: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T01:27:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7700.848072 secs 2023-09-10T01:27:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:51 smithi170 ceph-mon[102832]: pgmap v4272: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 99 KiB/s wr, 17 op/s 2023-09-10T01:27:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:51 smithi082 ceph-mon[98694]: pgmap v4272: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 99 KiB/s wr, 17 op/s 2023-09-10T01:27:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:51 smithi098 ceph-mon[103124]: pgmap v4272: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 99 KiB/s wr, 17 op/s 2023-09-10T01:27:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:27:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:27:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:27:52.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7703.654064 secs 2023-09-10T01:27:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:27:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:27:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:27:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7703.654064 secs 2023-09-10T01:27:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:27:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:27:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:27:53.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7703.654064 secs 2023-09-10T01:27:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:53 smithi170 ceph-mon[102832]: pgmap v4273: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T01:27:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:53 smithi082 ceph-mon[98694]: pgmap v4273: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T01:27:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:53 smithi098 ceph-mon[103124]: pgmap v4273: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T01:27:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7705.136498 secs 2023-09-10T01:27:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7705.848139 secs 2023-09-10T01:27:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7705.136498 secs 2023-09-10T01:27:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7705.848139 secs 2023-09-10T01:27:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7705.136498 secs 2023-09-10T01:27:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7705.848139 secs 2023-09-10T01:27:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:55 smithi170 ceph-mon[102832]: pgmap v4274: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T01:27:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:55 smithi082 ceph-mon[98694]: pgmap v4274: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T01:27:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:55 smithi098 ceph-mon[103124]: pgmap v4274: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T01:27:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:57 smithi170 ceph-mon[102832]: pgmap v4275: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 18 op/s 2023-09-10T01:27:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7708.654125 secs 2023-09-10T01:27:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:57 smithi082 ceph-mon[98694]: pgmap v4275: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 18 op/s 2023-09-10T01:27:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7708.654125 secs 2023-09-10T01:27:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:57 smithi098 ceph-mon[103124]: pgmap v4275: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 18 op/s 2023-09-10T01:27:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7708.654125 secs 2023-09-10T01:27:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7710.136586 secs 2023-09-10T01:27:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:59 smithi170 ceph-mon[102832]: pgmap v4276: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 43 KiB/s wr, 12 op/s 2023-09-10T01:27:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:27:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7710.848204 secs 2023-09-10T01:27:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7710.136586 secs 2023-09-10T01:27:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:59 smithi082 ceph-mon[98694]: pgmap v4276: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 43 KiB/s wr, 12 op/s 2023-09-10T01:27:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:27:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7710.848204 secs 2023-09-10T01:28:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7710.136586 secs 2023-09-10T01:28:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:59 smithi098 ceph-mon[103124]: pgmap v4276: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 43 KiB/s wr, 12 op/s 2023-09-10T01:28:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:27:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7710.848204 secs 2023-09-10T01:28:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:01 smithi170 ceph-mon[102832]: pgmap v4277: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 68 KiB/s wr, 18 op/s 2023-09-10T01:28:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:01 smithi082 ceph-mon[98694]: pgmap v4277: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 68 KiB/s wr, 18 op/s 2023-09-10T01:28:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:01 smithi098 ceph-mon[103124]: pgmap v4277: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 68 KiB/s wr, 18 op/s 2023-09-10T01:28:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7713.654212 secs 2023-09-10T01:28:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7713.654212 secs 2023-09-10T01:28:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7713.654212 secs 2023-09-10T01:28:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:03 smithi170 ceph-mon[102832]: pgmap v4278: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 36 KiB/s wr, 13 op/s 2023-09-10T01:28:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:03 smithi082 ceph-mon[98694]: pgmap v4278: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 36 KiB/s wr, 13 op/s 2023-09-10T01:28:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:03 smithi098 ceph-mon[103124]: pgmap v4278: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 36 KiB/s wr, 13 op/s 2023-09-10T01:28:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7715.136647 secs 2023-09-10T01:28:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7715.848267 secs 2023-09-10T01:28:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7715.136647 secs 2023-09-10T01:28:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7715.848267 secs 2023-09-10T01:28:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7715.136647 secs 2023-09-10T01:28:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7715.848267 secs 2023-09-10T01:28:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:05 smithi170 ceph-mon[102832]: pgmap v4279: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 36 KiB/s wr, 13 op/s 2023-09-10T01:28:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:05 smithi082 ceph-mon[98694]: pgmap v4279: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 36 KiB/s wr, 13 op/s 2023-09-10T01:28:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:05 smithi098 ceph-mon[103124]: pgmap v4279: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 36 KiB/s wr, 13 op/s 2023-09-10T01:28:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:07 smithi170 ceph-mon[102832]: pgmap v4280: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-09-10T01:28:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7718.654272 secs 2023-09-10T01:28:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:07 smithi082 ceph-mon[98694]: pgmap v4280: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-09-10T01:28:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7718.654272 secs 2023-09-10T01:28:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:07 smithi098 ceph-mon[103124]: pgmap v4280: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-09-10T01:28:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7718.654272 secs 2023-09-10T01:28:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:08 smithi170 ceph-mon[102832]: osdmap e337: 12 total, 12 up, 12 in 2023-09-10T01:28:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:08 smithi082 ceph-mon[98694]: osdmap e337: 12 total, 12 up, 12 in 2023-09-10T01:28:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:08 smithi098 ceph-mon[103124]: osdmap e337: 12 total, 12 up, 12 in 2023-09-10T01:28:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7720.136705 secs 2023-09-10T01:28:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:09 smithi170 ceph-mon[102832]: pgmap v4282: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 81 KiB/s wr, 30 op/s 2023-09-10T01:28:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7720.848335 secs 2023-09-10T01:28:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7720.136705 secs 2023-09-10T01:28:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:09 smithi082 ceph-mon[98694]: pgmap v4282: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 81 KiB/s wr, 30 op/s 2023-09-10T01:28:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7720.848335 secs 2023-09-10T01:28:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7720.136705 secs 2023-09-10T01:28:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:09 smithi098 ceph-mon[103124]: pgmap v4282: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 81 KiB/s wr, 30 op/s 2023-09-10T01:28:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7720.848335 secs 2023-09-10T01:28:11.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:11 smithi170 ceph-mon[102832]: pgmap v4283: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 66 KiB/s wr, 28 op/s 2023-09-10T01:28:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:11 smithi082 ceph-mon[98694]: pgmap v4283: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 66 KiB/s wr, 28 op/s 2023-09-10T01:28:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:11 smithi098 ceph-mon[103124]: pgmap v4283: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 66 KiB/s wr, 28 op/s 2023-09-10T01:28:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7723.654346 secs 2023-09-10T01:28:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7723.654346 secs 2023-09-10T01:28:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7723.654346 secs 2023-09-10T01:28:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:13 smithi082 ceph-mon[98694]: pgmap v4284: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 66 KiB/s wr, 28 op/s 2023-09-10T01:28:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:13 smithi098 ceph-mon[103124]: pgmap v4284: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 66 KiB/s wr, 28 op/s 2023-09-10T01:28:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:13 smithi170 ceph-mon[102832]: pgmap v4284: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 66 KiB/s wr, 28 op/s 2023-09-10T01:28:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7725.136766 secs 2023-09-10T01:28:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7725.848398 secs 2023-09-10T01:28:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7725.136766 secs 2023-09-10T01:28:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7725.848398 secs 2023-09-10T01:28:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7725.136766 secs 2023-09-10T01:28:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7725.848398 secs 2023-09-10T01:28:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:15 smithi082 ceph-mon[98694]: pgmap v4285: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 66 KiB/s wr, 28 op/s 2023-09-10T01:28:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:15 smithi082 ceph-mon[98694]: osdmap e338: 12 total, 12 up, 12 in 2023-09-10T01:28:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:15 smithi098 ceph-mon[103124]: pgmap v4285: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 66 KiB/s wr, 28 op/s 2023-09-10T01:28:16.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:15 smithi098 ceph-mon[103124]: osdmap e338: 12 total, 12 up, 12 in 2023-09-10T01:28:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:15 smithi170 ceph-mon[102832]: pgmap v4285: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 66 KiB/s wr, 28 op/s 2023-09-10T01:28:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:15 smithi170 ceph-mon[102832]: osdmap e338: 12 total, 12 up, 12 in 2023-09-10T01:28:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:17 smithi082 ceph-mon[98694]: pgmap v4287: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 13 op/s 2023-09-10T01:28:17.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7728.654407 secs 2023-09-10T01:28:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:17 smithi098 ceph-mon[103124]: pgmap v4287: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 13 op/s 2023-09-10T01:28:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7728.654407 secs 2023-09-10T01:28:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:17 smithi170 ceph-mon[102832]: pgmap v4287: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 13 op/s 2023-09-10T01:28:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7728.654407 secs 2023-09-10T01:28:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7730.136824 secs 2023-09-10T01:28:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:19 smithi082 ceph-mon[98694]: pgmap v4288: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:28:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7730.848462 secs 2023-09-10T01:28:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:28:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7730.136824 secs 2023-09-10T01:28:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:19 smithi098 ceph-mon[103124]: pgmap v4288: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:28:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7730.848462 secs 2023-09-10T01:28:20.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:28:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7730.136824 secs 2023-09-10T01:28:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:19 smithi170 ceph-mon[102832]: pgmap v4288: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:28:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7730.848462 secs 2023-09-10T01:28:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:28:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:28:20.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:28:20.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:28:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:28:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:28:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:28:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:28:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:28:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:28:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:21 smithi082 ceph-mon[98694]: pgmap v4289: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T01:28:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:21 smithi098 ceph-mon[103124]: pgmap v4289: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T01:28:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:21 smithi170 ceph-mon[102832]: pgmap v4289: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T01:28:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:28:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:28:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:28:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:23 smithi082 ceph-mon[98694]: pgmap v4290: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T01:28:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:28:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:28:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:28:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7735.136891 secs 2023-09-10T01:28:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:23 smithi098 ceph-mon[103124]: pgmap v4290: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T01:28:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:28:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:28:24.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:28:24.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7735.136891 secs 2023-09-10T01:28:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:23 smithi170 ceph-mon[102832]: pgmap v4290: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T01:28:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:28:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:28:24.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:28:24.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7735.136891 secs 2023-09-10T01:28:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7735.848484 secs 2023-09-10T01:28:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7735.848484 secs 2023-09-10T01:28:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7735.848484 secs 2023-09-10T01:28:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:25 smithi098 ceph-mon[103124]: pgmap v4291: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T01:28:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:25 smithi170 ceph-mon[102832]: pgmap v4291: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T01:28:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:25 smithi082 ceph-mon[98694]: pgmap v4291: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T01:28:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:27 smithi098 ceph-mon[103124]: pgmap v4292: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:28:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:27 smithi170 ceph-mon[102832]: pgmap v4292: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:28:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:27 smithi082 ceph-mon[98694]: pgmap v4292: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:28:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7740.136932 secs 2023-09-10T01:28:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7740.848552 secs 2023-09-10T01:28:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7740.136932 secs 2023-09-10T01:28:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7740.848552 secs 2023-09-10T01:28:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7740.136932 secs 2023-09-10T01:28:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7740.848552 secs 2023-09-10T01:28:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:29 smithi098 ceph-mon[103124]: pgmap v4293: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 10 op/s 2023-09-10T01:28:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:29 smithi170 ceph-mon[102832]: pgmap v4293: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 10 op/s 2023-09-10T01:28:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:29 smithi082 ceph-mon[98694]: pgmap v4293: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 10 op/s 2023-09-10T01:28:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:31 smithi098 ceph-mon[103124]: pgmap v4294: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 83 KiB/s wr, 16 op/s 2023-09-10T01:28:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:31 smithi170 ceph-mon[102832]: pgmap v4294: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 83 KiB/s wr, 16 op/s 2023-09-10T01:28:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:31 smithi082 ceph-mon[98694]: pgmap v4294: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 83 KiB/s wr, 16 op/s 2023-09-10T01:28:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:33 smithi098 ceph-mon[103124]: pgmap v4295: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T01:28:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7745.136992 secs 2023-09-10T01:28:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7745.848616 secs 2023-09-10T01:28:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:33 smithi170 ceph-mon[102832]: pgmap v4295: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T01:28:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7745.136992 secs 2023-09-10T01:28:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7745.848616 secs 2023-09-10T01:28:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:33 smithi082 ceph-mon[98694]: pgmap v4295: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T01:28:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7745.136992 secs 2023-09-10T01:28:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7745.848616 secs 2023-09-10T01:28:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:35 smithi098 ceph-mon[103124]: pgmap v4296: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T01:28:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:35 smithi170 ceph-mon[102832]: pgmap v4296: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T01:28:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:35 smithi082 ceph-mon[98694]: pgmap v4296: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T01:28:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:37 smithi098 ceph-mon[103124]: pgmap v4297: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 86 KiB/s wr, 17 op/s 2023-09-10T01:28:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:37 smithi170 ceph-mon[102832]: pgmap v4297: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 86 KiB/s wr, 17 op/s 2023-09-10T01:28:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:37 smithi082 ceph-mon[98694]: pgmap v4297: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 86 KiB/s wr, 17 op/s 2023-09-10T01:28:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7750.137050 secs 2023-09-10T01:28:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7750.848683 secs 2023-09-10T01:28:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7750.137050 secs 2023-09-10T01:28:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7750.848683 secs 2023-09-10T01:28:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7750.137050 secs 2023-09-10T01:28:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7750.848683 secs 2023-09-10T01:28:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:39 smithi098 ceph-mon[103124]: pgmap v4298: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 77 KiB/s wr, 12 op/s 2023-09-10T01:28:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:39 smithi170 ceph-mon[102832]: pgmap v4298: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 77 KiB/s wr, 12 op/s 2023-09-10T01:28:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:39 smithi082 ceph-mon[98694]: pgmap v4298: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 77 KiB/s wr, 12 op/s 2023-09-10T01:28:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:41 smithi098 ceph-mon[103124]: pgmap v4299: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 124 KiB/s wr, 18 op/s 2023-09-10T01:28:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:41 smithi170 ceph-mon[102832]: pgmap v4299: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 124 KiB/s wr, 18 op/s 2023-09-10T01:28:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:41 smithi082 ceph-mon[98694]: pgmap v4299: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 124 KiB/s wr, 18 op/s 2023-09-10T01:28:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7753.654775 secs 2023-09-10T01:28:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7753.654775 secs 2023-09-10T01:28:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7753.654775 secs 2023-09-10T01:28:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:43 smithi098 ceph-mon[103124]: pgmap v4300: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-09-10T01:28:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:28:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:28:44.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:28:44.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7755.137108 secs 2023-09-10T01:28:44.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7755.848745 secs 2023-09-10T01:28:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:43 smithi170 ceph-mon[102832]: pgmap v4300: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-09-10T01:28:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:28:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:28:44.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:28:44.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7755.137108 secs 2023-09-10T01:28:44.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7755.848745 secs 2023-09-10T01:28:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:43 smithi082 ceph-mon[98694]: pgmap v4300: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-09-10T01:28:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:28:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:28:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:28:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7755.137108 secs 2023-09-10T01:28:44.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7755.848745 secs 2023-09-10T01:28:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:45 smithi098 ceph-mon[103124]: pgmap v4301: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-09-10T01:28:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:45 smithi170 ceph-mon[102832]: pgmap v4301: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-09-10T01:28:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:45 smithi082 ceph-mon[98694]: pgmap v4301: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-09-10T01:28:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:47 smithi098 ceph-mon[103124]: pgmap v4302: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 76 KiB/s wr, 17 op/s 2023-09-10T01:28:48.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7758.654834 secs 2023-09-10T01:28:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:47 smithi170 ceph-mon[102832]: pgmap v4302: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 76 KiB/s wr, 17 op/s 2023-09-10T01:28:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7758.654834 secs 2023-09-10T01:28:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:47 smithi082 ceph-mon[98694]: pgmap v4302: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 76 KiB/s wr, 17 op/s 2023-09-10T01:28:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7758.654834 secs 2023-09-10T01:28:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7760.137211 secs 2023-09-10T01:28:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7760.848810 secs 2023-09-10T01:28:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7760.137211 secs 2023-09-10T01:28:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7760.848810 secs 2023-09-10T01:28:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7760.137211 secs 2023-09-10T01:28:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7760.848810 secs 2023-09-10T01:28:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:49 smithi170 ceph-mon[102832]: pgmap v4303: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T01:28:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:49 smithi082 ceph-mon[98694]: pgmap v4303: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T01:28:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:49 smithi098 ceph-mon[103124]: pgmap v4303: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T01:28:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:51 smithi170 ceph-mon[102832]: pgmap v4304: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 83 KiB/s wr, 16 op/s 2023-09-10T01:28:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:51 smithi082 ceph-mon[98694]: pgmap v4304: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 83 KiB/s wr, 16 op/s 2023-09-10T01:28:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:51 smithi098 ceph-mon[103124]: pgmap v4304: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 83 KiB/s wr, 16 op/s 2023-09-10T01:28:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7763.654920 secs 2023-09-10T01:28:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7763.654920 secs 2023-09-10T01:28:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7763.654920 secs 2023-09-10T01:28:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:53 smithi170 ceph-mon[102832]: pgmap v4305: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-09-10T01:28:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7765.137294 secs 2023-09-10T01:28:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7765.848878 secs 2023-09-10T01:28:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:53 smithi082 ceph-mon[98694]: pgmap v4305: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-09-10T01:28:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7765.137294 secs 2023-09-10T01:28:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7765.848878 secs 2023-09-10T01:28:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:53 smithi098 ceph-mon[103124]: pgmap v4305: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-09-10T01:28:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7765.137294 secs 2023-09-10T01:28:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7765.848878 secs 2023-09-10T01:28:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:55 smithi170 ceph-mon[102832]: pgmap v4306: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-09-10T01:28:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:55 smithi082 ceph-mon[98694]: pgmap v4306: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-09-10T01:28:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:55 smithi098 ceph-mon[103124]: pgmap v4306: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 37 KiB/s wr, 10 op/s 2023-09-10T01:28:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:28:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:28:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:28:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:57 smithi170 ceph-mon[102832]: pgmap v4307: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 37 KiB/s wr, 15 op/s 2023-09-10T01:28:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7768.655057 secs 2023-09-10T01:28:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:57 smithi082 ceph-mon[98694]: pgmap v4307: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 37 KiB/s wr, 15 op/s 2023-09-10T01:28:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7768.655057 secs 2023-09-10T01:28:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:57 smithi098 ceph-mon[103124]: pgmap v4307: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 37 KiB/s wr, 15 op/s 2023-09-10T01:28:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7768.655057 secs 2023-09-10T01:28:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7770.137361 secs 2023-09-10T01:28:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7770.848944 secs 2023-09-10T01:28:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:28:58 smithi170 ceph-mon[102832]: pgmap v4308: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 10 op/s 2023-09-10T01:28:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7770.137361 secs 2023-09-10T01:28:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7770.848944 secs 2023-09-10T01:28:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:28:58 smithi082 ceph-mon[98694]: pgmap v4308: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 10 op/s 2023-09-10T01:28:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7770.137361 secs 2023-09-10T01:28:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7770.848944 secs 2023-09-10T01:28:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:28:58 smithi098 ceph-mon[103124]: pgmap v4308: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 10 op/s 2023-09-10T01:29:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:01 smithi170 ceph-mon[102832]: pgmap v4309: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 50 KiB/s wr, 15 op/s 2023-09-10T01:29:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:01 smithi082 ceph-mon[98694]: pgmap v4309: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 50 KiB/s wr, 15 op/s 2023-09-10T01:29:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:01 smithi098 ceph-mon[103124]: pgmap v4309: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 50 KiB/s wr, 15 op/s 2023-09-10T01:29:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7773.655117 secs 2023-09-10T01:29:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7773.655117 secs 2023-09-10T01:29:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7773.655117 secs 2023-09-10T01:29:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:03 smithi170 ceph-mon[102832]: pgmap v4310: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-10T01:29:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:03 smithi082 ceph-mon[98694]: pgmap v4310: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-10T01:29:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:03 smithi098 ceph-mon[103124]: pgmap v4310: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-10T01:29:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7775.137435 secs 2023-09-10T01:29:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7775.849008 secs 2023-09-10T01:29:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7775.137435 secs 2023-09-10T01:29:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7775.849008 secs 2023-09-10T01:29:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7775.137435 secs 2023-09-10T01:29:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7775.849008 secs 2023-09-10T01:29:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:05 smithi170 ceph-mon[102832]: pgmap v4311: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-10T01:29:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:05 smithi082 ceph-mon[98694]: pgmap v4311: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-10T01:29:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:05 smithi098 ceph-mon[103124]: pgmap v4311: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-10T01:29:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:07 smithi170 ceph-mon[102832]: pgmap v4312: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 60 KiB/s wr, 29 op/s 2023-09-10T01:29:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7778.655177 secs 2023-09-10T01:29:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:07 smithi082 ceph-mon[98694]: pgmap v4312: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 60 KiB/s wr, 29 op/s 2023-09-10T01:29:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7778.655177 secs 2023-09-10T01:29:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:07 smithi098 ceph-mon[103124]: pgmap v4312: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 60 KiB/s wr, 29 op/s 2023-09-10T01:29:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7778.655177 secs 2023-09-10T01:29:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:08 smithi170 ceph-mon[102832]: osdmap e339: 12 total, 12 up, 12 in 2023-09-10T01:29:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:08 smithi082 ceph-mon[98694]: osdmap e339: 12 total, 12 up, 12 in 2023-09-10T01:29:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:08 smithi098 ceph-mon[103124]: osdmap e339: 12 total, 12 up, 12 in 2023-09-10T01:29:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7780.137515 secs 2023-09-10T01:29:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7780.849074 secs 2023-09-10T01:29:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:09 smithi170 ceph-mon[102832]: pgmap v4314: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-09-10T01:29:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7780.137515 secs 2023-09-10T01:29:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7780.849074 secs 2023-09-10T01:29:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:09 smithi082 ceph-mon[98694]: pgmap v4314: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-09-10T01:29:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7780.137515 secs 2023-09-10T01:29:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7780.849074 secs 2023-09-10T01:29:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:09 smithi098 ceph-mon[103124]: pgmap v4314: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-09-10T01:29:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:11 smithi082 ceph-mon[98694]: pgmap v4315: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-09-10T01:29:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:11 smithi098 ceph-mon[103124]: pgmap v4315: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-09-10T01:29:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:11 smithi170 ceph-mon[102832]: pgmap v4315: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-09-10T01:29:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7783.655293 secs 2023-09-10T01:29:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7783.655293 secs 2023-09-10T01:29:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7783.655293 secs 2023-09-10T01:29:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:13 smithi082 ceph-mon[98694]: pgmap v4316: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-09-10T01:29:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:13 smithi098 ceph-mon[103124]: pgmap v4316: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-09-10T01:29:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:13 smithi170 ceph-mon[102832]: pgmap v4316: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-09-10T01:29:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7785.137583 secs 2023-09-10T01:29:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7785.849150 secs 2023-09-10T01:29:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7785.137583 secs 2023-09-10T01:29:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7785.849150 secs 2023-09-10T01:29:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7785.137583 secs 2023-09-10T01:29:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7785.849150 secs 2023-09-10T01:29:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:15 smithi082 ceph-mon[98694]: pgmap v4317: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-09-10T01:29:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:15 smithi082 ceph-mon[98694]: osdmap e340: 12 total, 12 up, 12 in 2023-09-10T01:29:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:15 smithi098 ceph-mon[103124]: pgmap v4317: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-09-10T01:29:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:15 smithi098 ceph-mon[103124]: osdmap e340: 12 total, 12 up, 12 in 2023-09-10T01:29:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:15 smithi170 ceph-mon[102832]: pgmap v4317: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-09-10T01:29:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:15 smithi170 ceph-mon[102832]: osdmap e340: 12 total, 12 up, 12 in 2023-09-10T01:29:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:17 smithi082 ceph-mon[98694]: pgmap v4319: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 13 op/s 2023-09-10T01:29:17.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7788.655379 secs 2023-09-10T01:29:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:17 smithi098 ceph-mon[103124]: pgmap v4319: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 13 op/s 2023-09-10T01:29:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7788.655379 secs 2023-09-10T01:29:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:17 smithi170 ceph-mon[102832]: pgmap v4319: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 13 op/s 2023-09-10T01:29:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7788.655379 secs 2023-09-10T01:29:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7790.137643 secs 2023-09-10T01:29:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7790.849208 secs 2023-09-10T01:29:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:19 smithi082 ceph-mon[98694]: pgmap v4320: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:29:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7790.137643 secs 2023-09-10T01:29:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7790.849208 secs 2023-09-10T01:29:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:19 smithi098 ceph-mon[103124]: pgmap v4320: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:29:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7790.137643 secs 2023-09-10T01:29:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7790.849208 secs 2023-09-10T01:29:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:19 smithi170 ceph-mon[102832]: pgmap v4320: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:29:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:29:20.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:29:20.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:29:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:29:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:29:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:29:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:29:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:29:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:29:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:21 smithi082 ceph-mon[98694]: pgmap v4321: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 11 op/s 2023-09-10T01:29:21.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:29:21.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:29:21.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:29:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:21 smithi098 ceph-mon[103124]: pgmap v4321: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 11 op/s 2023-09-10T01:29:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:29:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:29:22.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:29:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:21 smithi170 ceph-mon[102832]: pgmap v4321: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 11 op/s 2023-09-10T01:29:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:29:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:29:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:29:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:29:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:29:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:29:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:23 smithi082 ceph-mon[98694]: pgmap v4322: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 11 op/s 2023-09-10T01:29:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:29:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:29:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:29:23.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7795.137701 secs 2023-09-10T01:29:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:23 smithi098 ceph-mon[103124]: pgmap v4322: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 11 op/s 2023-09-10T01:29:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:29:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:29:24.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:29:24.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7795.137701 secs 2023-09-10T01:29:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:23 smithi170 ceph-mon[102832]: pgmap v4322: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 11 op/s 2023-09-10T01:29:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:29:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:29:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:29:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7795.137701 secs 2023-09-10T01:29:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7795.849273 secs 2023-09-10T01:29:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7795.849273 secs 2023-09-10T01:29:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7795.849273 secs 2023-09-10T01:29:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:25 smithi098 ceph-mon[103124]: pgmap v4323: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 11 op/s 2023-09-10T01:29:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:25 smithi170 ceph-mon[102832]: pgmap v4323: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 11 op/s 2023-09-10T01:29:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:25 smithi082 ceph-mon[98694]: pgmap v4323: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 11 op/s 2023-09-10T01:29:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:27 smithi098 ceph-mon[103124]: pgmap v4324: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-10T01:29:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7798.655500 secs 2023-09-10T01:29:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:27 smithi170 ceph-mon[102832]: pgmap v4324: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-10T01:29:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7798.655500 secs 2023-09-10T01:29:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:27 smithi082 ceph-mon[98694]: pgmap v4324: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-10T01:29:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7798.655500 secs 2023-09-10T01:29:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7800.137786 secs 2023-09-10T01:29:29.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7800.849337 secs 2023-09-10T01:29:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7800.137786 secs 2023-09-10T01:29:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7800.849337 secs 2023-09-10T01:29:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7800.137786 secs 2023-09-10T01:29:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7800.849337 secs 2023-09-10T01:29:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:29 smithi098 ceph-mon[103124]: pgmap v4325: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 9 op/s 2023-09-10T01:29:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:29 smithi170 ceph-mon[102832]: pgmap v4325: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 9 op/s 2023-09-10T01:29:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:29 smithi082 ceph-mon[98694]: pgmap v4325: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 9 op/s 2023-09-10T01:29:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:31 smithi098 ceph-mon[103124]: pgmap v4326: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 52 KiB/s wr, 14 op/s 2023-09-10T01:29:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:31 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:29:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:29:32.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:29:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:31 smithi170 ceph-mon[102832]: pgmap v4326: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 52 KiB/s wr, 14 op/s 2023-09-10T01:29:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:31 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:29:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:29:32.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:29:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:31 smithi082 ceph-mon[98694]: pgmap v4326: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 52 KiB/s wr, 14 op/s 2023-09-10T01:29:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:31 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:29:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:29:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:29:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7803.655559 secs 2023-09-10T01:29:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7803.655559 secs 2023-09-10T01:29:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7803.655559 secs 2023-09-10T01:29:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:33 smithi098 ceph-mon[103124]: pgmap v4327: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T01:29:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7805.137848 secs 2023-09-10T01:29:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7805.849406 secs 2023-09-10T01:29:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:33 smithi170 ceph-mon[102832]: pgmap v4327: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T01:29:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7805.137848 secs 2023-09-10T01:29:34.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7805.849406 secs 2023-09-10T01:29:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:33 smithi082 ceph-mon[98694]: pgmap v4327: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T01:29:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7805.137848 secs 2023-09-10T01:29:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7805.849406 secs 2023-09-10T01:29:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:35 smithi098 ceph-mon[103124]: pgmap v4328: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T01:29:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:35 smithi170 ceph-mon[102832]: pgmap v4328: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T01:29:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:35 smithi082 ceph-mon[98694]: pgmap v4328: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T01:29:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:37 smithi098 ceph-mon[103124]: pgmap v4329: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 57 KiB/s wr, 16 op/s 2023-09-10T01:29:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:37 smithi170 ceph-mon[102832]: pgmap v4329: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 57 KiB/s wr, 16 op/s 2023-09-10T01:29:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:37 smithi082 ceph-mon[98694]: pgmap v4329: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 57 KiB/s wr, 16 op/s 2023-09-10T01:29:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7810.137906 secs 2023-09-10T01:29:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7810.849475 secs 2023-09-10T01:29:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:38 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:29:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7810.137906 secs 2023-09-10T01:29:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7810.849475 secs 2023-09-10T01:29:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:38 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:29:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7810.137906 secs 2023-09-10T01:29:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7810.849475 secs 2023-09-10T01:29:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:38 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:29:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:39 smithi098 ceph-mon[103124]: pgmap v4330: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T01:29:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:39 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:29:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:39 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:29:40.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:39 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:29:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:39 smithi170 ceph-mon[102832]: pgmap v4330: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T01:29:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:39 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:29:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:39 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:29:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:39 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:29:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:39 smithi082 ceph-mon[98694]: pgmap v4330: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T01:29:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:39 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:29:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:39 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:29:40.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:39 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:29:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:41 smithi098 ceph-mon[103124]: pgmap v4331: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 85 KiB/s wr, 17 op/s 2023-09-10T01:29:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:41 smithi170 ceph-mon[102832]: pgmap v4331: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 85 KiB/s wr, 17 op/s 2023-09-10T01:29:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:41 smithi082 ceph-mon[98694]: pgmap v4331: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 85 KiB/s wr, 17 op/s 2023-09-10T01:29:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:43 smithi098 ceph-mon[103124]: pgmap v4332: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T01:29:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7815.137965 secs 2023-09-10T01:29:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7815.849542 secs 2023-09-10T01:29:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:43 smithi170 ceph-mon[102832]: pgmap v4332: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T01:29:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7815.137965 secs 2023-09-10T01:29:44.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7815.849542 secs 2023-09-10T01:29:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:43 smithi082 ceph-mon[98694]: pgmap v4332: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T01:29:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7815.137965 secs 2023-09-10T01:29:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7815.849542 secs 2023-09-10T01:29:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:45 smithi098 ceph-mon[103124]: pgmap v4333: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T01:29:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:45 smithi170 ceph-mon[102832]: pgmap v4333: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T01:29:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:45 smithi082 ceph-mon[98694]: pgmap v4333: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T01:29:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:47 smithi098 ceph-mon[103124]: pgmap v4334: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 61 KiB/s wr, 17 op/s 2023-09-10T01:29:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:47 smithi170 ceph-mon[102832]: pgmap v4334: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 61 KiB/s wr, 17 op/s 2023-09-10T01:29:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:47 smithi082 ceph-mon[98694]: pgmap v4334: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 61 KiB/s wr, 17 op/s 2023-09-10T01:29:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7820.138022 secs 2023-09-10T01:29:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7820.849612 secs 2023-09-10T01:29:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7820.138022 secs 2023-09-10T01:29:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7820.849612 secs 2023-09-10T01:29:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7820.138022 secs 2023-09-10T01:29:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7820.849612 secs 2023-09-10T01:29:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:49 smithi170 ceph-mon[102832]: pgmap v4335: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T01:29:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:49 smithi082 ceph-mon[98694]: pgmap v4335: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T01:29:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:49 smithi098 ceph-mon[103124]: pgmap v4335: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T01:29:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:51 smithi170 ceph-mon[102832]: pgmap v4336: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 18 op/s 2023-09-10T01:29:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:51 smithi082 ceph-mon[98694]: pgmap v4336: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 18 op/s 2023-09-10T01:29:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:51 smithi098 ceph-mon[103124]: pgmap v4336: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 18 op/s 2023-09-10T01:29:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7823.655867 secs 2023-09-10T01:29:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7823.655867 secs 2023-09-10T01:29:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7823.655867 secs 2023-09-10T01:29:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:53 smithi170 ceph-mon[102832]: pgmap v4337: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T01:29:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7825.138104 secs 2023-09-10T01:29:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7825.849678 secs 2023-09-10T01:29:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:53 smithi082 ceph-mon[98694]: pgmap v4337: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T01:29:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7825.138104 secs 2023-09-10T01:29:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7825.849678 secs 2023-09-10T01:29:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:53 smithi098 ceph-mon[103124]: pgmap v4337: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T01:29:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7825.138104 secs 2023-09-10T01:29:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7825.849678 secs 2023-09-10T01:29:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:54 smithi170 ceph-mon[102832]: pgmap v4338: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T01:29:55.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:54 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:29:55.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:29:55.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:29:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:54 smithi082 ceph-mon[98694]: pgmap v4338: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T01:29:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:54 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:29:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:29:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:29:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:54 smithi098 ceph-mon[103124]: pgmap v4338: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T01:29:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:54 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:29:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:29:55.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:29:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:57 smithi170 ceph-mon[102832]: pgmap v4339: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 83 KiB/s wr, 18 op/s 2023-09-10T01:29:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7828.655927 secs 2023-09-10T01:29:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:57 smithi082 ceph-mon[98694]: pgmap v4339: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 83 KiB/s wr, 18 op/s 2023-09-10T01:29:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7828.655927 secs 2023-09-10T01:29:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:57 smithi098 ceph-mon[103124]: pgmap v4339: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 83 KiB/s wr, 18 op/s 2023-09-10T01:29:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7828.655927 secs 2023-09-10T01:29:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7830.138161 secs 2023-09-10T01:29:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7830.849745 secs 2023-09-10T01:29:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:29:59 smithi170 ceph-mon[102832]: pgmap v4340: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T01:29:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7830.138161 secs 2023-09-10T01:29:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7830.849745 secs 2023-09-10T01:29:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:29:59 smithi082 ceph-mon[98694]: pgmap v4340: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T01:30:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7830.138161 secs 2023-09-10T01:30:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7830.849745 secs 2023-09-10T01:30:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:29:59 smithi098 ceph-mon[103124]: pgmap v4340: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T01:30:00.792 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T01:30:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T01:30:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T01:30:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:01 smithi170 ceph-mon[102832]: pgmap v4341: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 109 KiB/s wr, 17 op/s 2023-09-10T01:30:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:01 smithi082 ceph-mon[98694]: pgmap v4341: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 109 KiB/s wr, 17 op/s 2023-09-10T01:30:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:01 smithi098 ceph-mon[103124]: pgmap v4341: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 109 KiB/s wr, 17 op/s 2023-09-10T01:30:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7833.656000 secs 2023-09-10T01:30:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7833.656000 secs 2023-09-10T01:30:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7833.656000 secs 2023-09-10T01:30:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:03 smithi170 ceph-mon[102832]: pgmap v4342: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 54 KiB/s wr, 10 op/s 2023-09-10T01:30:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:03 smithi082 ceph-mon[98694]: pgmap v4342: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 54 KiB/s wr, 10 op/s 2023-09-10T01:30:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:03 smithi098 ceph-mon[103124]: pgmap v4342: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 54 KiB/s wr, 10 op/s 2023-09-10T01:30:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7835.138217 secs 2023-09-10T01:30:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7835.849813 secs 2023-09-10T01:30:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7835.138217 secs 2023-09-10T01:30:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7835.849813 secs 2023-09-10T01:30:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7835.138217 secs 2023-09-10T01:30:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7835.849813 secs 2023-09-10T01:30:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:05 smithi082 ceph-mon[98694]: pgmap v4343: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 54 KiB/s wr, 10 op/s 2023-09-10T01:30:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:05 smithi098 ceph-mon[103124]: pgmap v4343: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 54 KiB/s wr, 10 op/s 2023-09-10T01:30:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:05 smithi170 ceph-mon[102832]: pgmap v4343: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 54 KiB/s wr, 10 op/s 2023-09-10T01:30:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:07 smithi082 ceph-mon[98694]: pgmap v4344: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 108 KiB/s wr, 30 op/s 2023-09-10T01:30:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7838.656068 secs 2023-09-10T01:30:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:07 smithi098 ceph-mon[103124]: pgmap v4344: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 108 KiB/s wr, 30 op/s 2023-09-10T01:30:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7838.656068 secs 2023-09-10T01:30:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:07 smithi170 ceph-mon[102832]: pgmap v4344: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 108 KiB/s wr, 30 op/s 2023-09-10T01:30:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7838.656068 secs 2023-09-10T01:30:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:08 smithi082 ceph-mon[98694]: osdmap e341: 12 total, 12 up, 12 in 2023-09-10T01:30:09.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:08 smithi098 ceph-mon[103124]: osdmap e341: 12 total, 12 up, 12 in 2023-09-10T01:30:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:08 smithi170 ceph-mon[102832]: osdmap e341: 12 total, 12 up, 12 in 2023-09-10T01:30:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7840.138283 secs 2023-09-10T01:30:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7840.849879 secs 2023-09-10T01:30:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:09 smithi082 ceph-mon[98694]: pgmap v4346: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T01:30:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7840.138283 secs 2023-09-10T01:30:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7840.849879 secs 2023-09-10T01:30:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:09 smithi098 ceph-mon[103124]: pgmap v4346: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T01:30:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7840.138283 secs 2023-09-10T01:30:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7840.849879 secs 2023-09-10T01:30:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:09 smithi170 ceph-mon[102832]: pgmap v4346: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T01:30:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:11 smithi082 ceph-mon[98694]: pgmap v4347: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T01:30:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:11 smithi098 ceph-mon[103124]: pgmap v4347: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T01:30:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:11 smithi170 ceph-mon[102832]: pgmap v4347: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T01:30:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7843.656128 secs 2023-09-10T01:30:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7843.656128 secs 2023-09-10T01:30:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7843.656128 secs 2023-09-10T01:30:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:13 smithi098 ceph-mon[103124]: pgmap v4348: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T01:30:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7845.138366 secs 2023-09-10T01:30:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:13 smithi170 ceph-mon[102832]: pgmap v4348: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T01:30:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7845.138366 secs 2023-09-10T01:30:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:13 smithi082 ceph-mon[98694]: pgmap v4348: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T01:30:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7845.138366 secs 2023-09-10T01:30:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7845.849947 secs 2023-09-10T01:30:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7845.849947 secs 2023-09-10T01:30:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7845.849947 secs 2023-09-10T01:30:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:15 smithi098 ceph-mon[103124]: pgmap v4349: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T01:30:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:15 smithi098 ceph-mon[103124]: osdmap e342: 12 total, 12 up, 12 in 2023-09-10T01:30:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:15 smithi170 ceph-mon[102832]: pgmap v4349: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T01:30:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:15 smithi170 ceph-mon[102832]: osdmap e342: 12 total, 12 up, 12 in 2023-09-10T01:30:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:15 smithi082 ceph-mon[98694]: pgmap v4349: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T01:30:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:15 smithi082 ceph-mon[98694]: osdmap e342: 12 total, 12 up, 12 in 2023-09-10T01:30:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:17 smithi098 ceph-mon[103124]: pgmap v4351: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 13 op/s 2023-09-10T01:30:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7848.656236 secs 2023-09-10T01:30:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:17 smithi170 ceph-mon[102832]: pgmap v4351: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 13 op/s 2023-09-10T01:30:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7848.656236 secs 2023-09-10T01:30:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:17 smithi082 ceph-mon[98694]: pgmap v4351: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 13 op/s 2023-09-10T01:30:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7848.656236 secs 2023-09-10T01:30:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7850.138449 secs 2023-09-10T01:30:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7850.850014 secs 2023-09-10T01:30:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7850.138449 secs 2023-09-10T01:30:19.230 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7850.850014 secs 2023-09-10T01:30:19.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7850.138449 secs 2023-09-10T01:30:19.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7850.850014 secs 2023-09-10T01:30:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:19 smithi098 ceph-mon[103124]: pgmap v4352: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 11 op/s 2023-09-10T01:30:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:19 smithi170 ceph-mon[102832]: pgmap v4352: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 11 op/s 2023-09-10T01:30:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:19 smithi082 ceph-mon[98694]: pgmap v4352: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 11 op/s 2023-09-10T01:30:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:21 smithi082 ceph-mon[98694]: pgmap v4353: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:30:21.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:30:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:21 smithi098 ceph-mon[103124]: pgmap v4353: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:30:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:30:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:21 smithi170 ceph-mon[102832]: pgmap v4353: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:30:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:30:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:30:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:30:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:30:23.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:30:23.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7853.656299 secs 2023-09-10T01:30:23.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:30:23.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:30:23.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:30:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:30:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:30:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:30:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:30:23.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7853.656299 secs 2023-09-10T01:30:23.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:30:23.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:30:23.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:30:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:30:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:30:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:30:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:30:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7853.656299 secs 2023-09-10T01:30:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:30:23.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:30:23.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:30:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:23 smithi098 ceph-mon[103124]: pgmap v4354: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:30:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7855.138519 secs 2023-09-10T01:30:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7855.850079 secs 2023-09-10T01:30:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:23 smithi170 ceph-mon[102832]: pgmap v4354: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:30:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7855.138519 secs 2023-09-10T01:30:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7855.850079 secs 2023-09-10T01:30:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:23 smithi082 ceph-mon[98694]: pgmap v4354: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:30:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7855.138519 secs 2023-09-10T01:30:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7855.850079 secs 2023-09-10T01:30:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:25 smithi098 ceph-mon[103124]: pgmap v4355: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:30:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:25 smithi170 ceph-mon[102832]: pgmap v4355: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:30:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:25 smithi082 ceph-mon[98694]: pgmap v4355: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:30:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:27 smithi098 ceph-mon[103124]: pgmap v4356: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-09-10T01:30:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7858.656358 secs 2023-09-10T01:30:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:27 smithi170 ceph-mon[102832]: pgmap v4356: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-09-10T01:30:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7858.656358 secs 2023-09-10T01:30:28.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:27 smithi082 ceph-mon[98694]: pgmap v4356: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-09-10T01:30:28.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7858.656358 secs 2023-09-10T01:30:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7860.138578 secs 2023-09-10T01:30:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7860.850145 secs 2023-09-10T01:30:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7860.138578 secs 2023-09-10T01:30:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7860.850145 secs 2023-09-10T01:30:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7860.138578 secs 2023-09-10T01:30:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7860.850145 secs 2023-09-10T01:30:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:29 smithi170 ceph-mon[102832]: pgmap v4357: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 9 op/s 2023-09-10T01:30:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:29 smithi082 ceph-mon[98694]: pgmap v4357: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 9 op/s 2023-09-10T01:30:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:29 smithi098 ceph-mon[103124]: pgmap v4357: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 9 op/s 2023-09-10T01:30:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:31 smithi170 ceph-mon[102832]: pgmap v4358: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 58 KiB/s wr, 15 op/s 2023-09-10T01:30:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:31 smithi082 ceph-mon[98694]: pgmap v4358: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 58 KiB/s wr, 15 op/s 2023-09-10T01:30:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:31 smithi098 ceph-mon[103124]: pgmap v4358: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 58 KiB/s wr, 15 op/s 2023-09-10T01:30:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:33 smithi170 ceph-mon[102832]: pgmap v4359: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T01:30:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7865.138637 secs 2023-09-10T01:30:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7865.850212 secs 2023-09-10T01:30:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:33 smithi082 ceph-mon[98694]: pgmap v4359: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T01:30:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7865.138637 secs 2023-09-10T01:30:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7865.850212 secs 2023-09-10T01:30:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:33 smithi098 ceph-mon[103124]: pgmap v4359: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T01:30:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7865.138637 secs 2023-09-10T01:30:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7865.850212 secs 2023-09-10T01:30:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:34 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:30:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:34 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:30:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:34 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:30:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:35 smithi170 ceph-mon[102832]: pgmap v4360: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T01:30:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:35 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:30:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:35 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:30:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:35 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:30:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:35 smithi082 ceph-mon[98694]: pgmap v4360: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T01:30:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:35 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:30:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:35 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:30:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:35 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:30:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:35 smithi098 ceph-mon[103124]: pgmap v4360: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T01:30:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:35 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:30:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:35 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:30:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:35 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:30:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:37 smithi170 ceph-mon[102832]: pgmap v4361: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 89 KiB/s wr, 17 op/s 2023-09-10T01:30:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:37 smithi082 ceph-mon[98694]: pgmap v4361: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 89 KiB/s wr, 17 op/s 2023-09-10T01:30:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:37 smithi098 ceph-mon[103124]: pgmap v4361: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 89 KiB/s wr, 17 op/s 2023-09-10T01:30:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7870.138725 secs 2023-09-10T01:30:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7870.850279 secs 2023-09-10T01:30:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7870.138725 secs 2023-09-10T01:30:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7870.850279 secs 2023-09-10T01:30:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7870.138725 secs 2023-09-10T01:30:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7870.850279 secs 2023-09-10T01:30:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:39 smithi170 ceph-mon[102832]: pgmap v4362: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 59 KiB/s wr, 12 op/s 2023-09-10T01:30:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:39 smithi082 ceph-mon[98694]: pgmap v4362: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 59 KiB/s wr, 12 op/s 2023-09-10T01:30:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:39 smithi098 ceph-mon[103124]: pgmap v4362: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 59 KiB/s wr, 12 op/s 2023-09-10T01:30:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:41 smithi170 ceph-mon[102832]: pgmap v4363: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 98 KiB/s wr, 18 op/s 2023-09-10T01:30:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:41 smithi082 ceph-mon[98694]: pgmap v4363: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 98 KiB/s wr, 18 op/s 2023-09-10T01:30:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:41 smithi098 ceph-mon[103124]: pgmap v4363: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 98 KiB/s wr, 18 op/s 2023-09-10T01:30:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7873.656567 secs 2023-09-10T01:30:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7873.656567 secs 2023-09-10T01:30:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7873.656567 secs 2023-09-10T01:30:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:43 smithi170 ceph-mon[102832]: pgmap v4364: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T01:30:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:30:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:30:44.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:30:44.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7875.138786 secs 2023-09-10T01:30:44.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7875.850346 secs 2023-09-10T01:30:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:43 smithi082 ceph-mon[98694]: pgmap v4364: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T01:30:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:30:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:30:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:30:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7875.138786 secs 2023-09-10T01:30:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7875.850346 secs 2023-09-10T01:30:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:43 smithi098 ceph-mon[103124]: pgmap v4364: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T01:30:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:30:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:30:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:30:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7875.138786 secs 2023-09-10T01:30:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7875.850346 secs 2023-09-10T01:30:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:45 smithi170 ceph-mon[102832]: pgmap v4365: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T01:30:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:45 smithi082 ceph-mon[98694]: pgmap v4365: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T01:30:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:45 smithi098 ceph-mon[103124]: pgmap v4365: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T01:30:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:47 smithi170 ceph-mon[102832]: pgmap v4366: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 100 KiB/s wr, 17 op/s 2023-09-10T01:30:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7878.656646 secs 2023-09-10T01:30:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:47 smithi082 ceph-mon[98694]: pgmap v4366: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 100 KiB/s wr, 17 op/s 2023-09-10T01:30:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7878.656646 secs 2023-09-10T01:30:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:47 smithi098 ceph-mon[103124]: pgmap v4366: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 100 KiB/s wr, 17 op/s 2023-09-10T01:30:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7878.656646 secs 2023-09-10T01:30:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7880.138846 secs 2023-09-10T01:30:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7880.850411 secs 2023-09-10T01:30:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7880.138846 secs 2023-09-10T01:30:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7880.850411 secs 2023-09-10T01:30:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7880.138846 secs 2023-09-10T01:30:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7880.850411 secs 2023-09-10T01:30:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:49 smithi170 ceph-mon[102832]: pgmap v4367: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T01:30:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:49 smithi082 ceph-mon[98694]: pgmap v4367: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T01:30:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:49 smithi098 ceph-mon[103124]: pgmap v4367: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T01:30:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:51 smithi170 ceph-mon[102832]: pgmap v4368: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 83 KiB/s wr, 16 op/s 2023-09-10T01:30:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:51 smithi082 ceph-mon[98694]: pgmap v4368: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 83 KiB/s wr, 16 op/s 2023-09-10T01:30:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:51 smithi098 ceph-mon[103124]: pgmap v4368: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 83 KiB/s wr, 16 op/s 2023-09-10T01:30:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7883.656736 secs 2023-09-10T01:30:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7883.656736 secs 2023-09-10T01:30:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7883.656736 secs 2023-09-10T01:30:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:53 smithi170 ceph-mon[102832]: pgmap v4369: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T01:30:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:53 smithi082 ceph-mon[98694]: pgmap v4369: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T01:30:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:53 smithi098 ceph-mon[103124]: pgmap v4369: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T01:30:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7885.138904 secs 2023-09-10T01:30:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7885.850478 secs 2023-09-10T01:30:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7885.138904 secs 2023-09-10T01:30:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7885.850478 secs 2023-09-10T01:30:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7885.138904 secs 2023-09-10T01:30:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7885.850478 secs 2023-09-10T01:30:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:55 smithi170 ceph-mon[102832]: pgmap v4370: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T01:30:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:55 smithi082 ceph-mon[98694]: pgmap v4370: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T01:30:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:55 smithi098 ceph-mon[103124]: pgmap v4370: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T01:30:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:57 smithi170 ceph-mon[102832]: pgmap v4371: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 50 KiB/s wr, 15 op/s 2023-09-10T01:30:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7888.656773 secs 2023-09-10T01:30:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:57 smithi082 ceph-mon[98694]: pgmap v4371: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 50 KiB/s wr, 15 op/s 2023-09-10T01:30:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7888.656773 secs 2023-09-10T01:30:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:57 smithi098 ceph-mon[103124]: pgmap v4371: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 50 KiB/s wr, 15 op/s 2023-09-10T01:30:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7888.656773 secs 2023-09-10T01:30:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7890.138964 secs 2023-09-10T01:30:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7890.850549 secs 2023-09-10T01:30:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:30:59 smithi170 ceph-mon[102832]: pgmap v4372: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 10 op/s 2023-09-10T01:30:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7890.138964 secs 2023-09-10T01:30:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7890.850549 secs 2023-09-10T01:30:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:30:59 smithi082 ceph-mon[98694]: pgmap v4372: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 10 op/s 2023-09-10T01:31:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7890.138964 secs 2023-09-10T01:31:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7890.850549 secs 2023-09-10T01:31:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:30:59 smithi098 ceph-mon[103124]: pgmap v4372: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 10 op/s 2023-09-10T01:31:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:01 smithi170 ceph-mon[102832]: pgmap v4373: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 39 KiB/s wr, 15 op/s 2023-09-10T01:31:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:01 smithi082 ceph-mon[98694]: pgmap v4373: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 39 KiB/s wr, 15 op/s 2023-09-10T01:31:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:01 smithi098 ceph-mon[103124]: pgmap v4373: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 39 KiB/s wr, 15 op/s 2023-09-10T01:31:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7893.656895 secs 2023-09-10T01:31:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7893.656895 secs 2023-09-10T01:31:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7893.656895 secs 2023-09-10T01:31:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:03 smithi170 ceph-mon[102832]: pgmap v4374: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 9 op/s 2023-09-10T01:31:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:03 smithi082 ceph-mon[98694]: pgmap v4374: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 9 op/s 2023-09-10T01:31:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:03 smithi098 ceph-mon[103124]: pgmap v4374: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 9 op/s 2023-09-10T01:31:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7895.139022 secs 2023-09-10T01:31:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7895.850612 secs 2023-09-10T01:31:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7895.139022 secs 2023-09-10T01:31:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7895.850612 secs 2023-09-10T01:31:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7895.139022 secs 2023-09-10T01:31:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7895.850612 secs 2023-09-10T01:31:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:05 smithi170 ceph-mon[102832]: pgmap v4375: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 9 op/s 2023-09-10T01:31:05.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:05 smithi082 ceph-mon[98694]: pgmap v4375: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 9 op/s 2023-09-10T01:31:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:05 smithi098 ceph-mon[103124]: pgmap v4375: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 9 op/s 2023-09-10T01:31:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:07 smithi170 ceph-mon[102832]: pgmap v4376: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 77 KiB/s wr, 29 op/s 2023-09-10T01:31:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7898.657010 secs 2023-09-10T01:31:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:07 smithi082 ceph-mon[98694]: pgmap v4376: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 77 KiB/s wr, 29 op/s 2023-09-10T01:31:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7898.657010 secs 2023-09-10T01:31:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:07 smithi098 ceph-mon[103124]: pgmap v4376: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 77 KiB/s wr, 29 op/s 2023-09-10T01:31:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7898.657010 secs 2023-09-10T01:31:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:08 smithi170 ceph-mon[102832]: osdmap e343: 12 total, 12 up, 12 in 2023-09-10T01:31:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:08 smithi082 ceph-mon[98694]: osdmap e343: 12 total, 12 up, 12 in 2023-09-10T01:31:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:08 smithi098 ceph-mon[103124]: osdmap e343: 12 total, 12 up, 12 in 2023-09-10T01:31:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7900.139080 secs 2023-09-10T01:31:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7900.850676 secs 2023-09-10T01:31:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:09 smithi170 ceph-mon[102832]: pgmap v4378: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T01:31:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7900.139080 secs 2023-09-10T01:31:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7900.850676 secs 2023-09-10T01:31:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:09 smithi082 ceph-mon[98694]: pgmap v4378: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T01:31:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7900.139080 secs 2023-09-10T01:31:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7900.850676 secs 2023-09-10T01:31:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:09 smithi098 ceph-mon[103124]: pgmap v4378: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 29 op/s 2023-09-10T01:31:11.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:11 smithi170 ceph-mon[102832]: pgmap v4379: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-09-10T01:31:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:11 smithi082 ceph-mon[98694]: pgmap v4379: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-09-10T01:31:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:11 smithi098 ceph-mon[103124]: pgmap v4379: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-09-10T01:31:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7903.657073 secs 2023-09-10T01:31:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7903.657073 secs 2023-09-10T01:31:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7903.657073 secs 2023-09-10T01:31:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:13 smithi082 ceph-mon[98694]: pgmap v4380: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-09-10T01:31:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:13 smithi098 ceph-mon[103124]: pgmap v4380: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-09-10T01:31:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:13 smithi170 ceph-mon[102832]: pgmap v4380: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-09-10T01:31:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7905.139138 secs 2023-09-10T01:31:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7905.850739 secs 2023-09-10T01:31:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7905.139138 secs 2023-09-10T01:31:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7905.850739 secs 2023-09-10T01:31:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7905.139138 secs 2023-09-10T01:31:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7905.850739 secs 2023-09-10T01:31:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:15 smithi082 ceph-mon[98694]: pgmap v4381: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-09-10T01:31:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:15 smithi082 ceph-mon[98694]: osdmap e344: 12 total, 12 up, 12 in 2023-09-10T01:31:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:15 smithi098 ceph-mon[103124]: pgmap v4381: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-09-10T01:31:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:15 smithi098 ceph-mon[103124]: osdmap e344: 12 total, 12 up, 12 in 2023-09-10T01:31:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:15 smithi170 ceph-mon[102832]: pgmap v4381: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-09-10T01:31:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:15 smithi170 ceph-mon[102832]: osdmap e344: 12 total, 12 up, 12 in 2023-09-10T01:31:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:17 smithi082 ceph-mon[98694]: pgmap v4383: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 13 op/s 2023-09-10T01:31:17.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7908.657189 secs 2023-09-10T01:31:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:17 smithi098 ceph-mon[103124]: pgmap v4383: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 13 op/s 2023-09-10T01:31:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7908.657189 secs 2023-09-10T01:31:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:17 smithi170 ceph-mon[102832]: pgmap v4383: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 13 op/s 2023-09-10T01:31:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7908.657189 secs 2023-09-10T01:31:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7910.139195 secs 2023-09-10T01:31:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7910.850805 secs 2023-09-10T01:31:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:19 smithi082 ceph-mon[98694]: pgmap v4384: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T01:31:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7910.139195 secs 2023-09-10T01:31:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7910.850805 secs 2023-09-10T01:31:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:19 smithi098 ceph-mon[103124]: pgmap v4384: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T01:31:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7910.139195 secs 2023-09-10T01:31:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7910.850805 secs 2023-09-10T01:31:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:19 smithi170 ceph-mon[102832]: pgmap v4384: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T01:31:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:21 smithi082 ceph-mon[98694]: pgmap v4385: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:31:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:21 smithi098 ceph-mon[103124]: pgmap v4385: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:31:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:21 smithi170 ceph-mon[102832]: pgmap v4385: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:31:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:31:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:31:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:31:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:31:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:31:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:31:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:23 smithi082 ceph-mon[98694]: pgmap v4386: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:31:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:31:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:31:23.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:31:23.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:31:23.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:31:23.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:31:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:23 smithi098 ceph-mon[103124]: pgmap v4386: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:31:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:31:24.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:31:24.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:31:24.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:31:24.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:31:24.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:31:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:23 smithi170 ceph-mon[102832]: pgmap v4386: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:31:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:31:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:31:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:31:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:31:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:31:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:31:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7915.139254 secs 2023-09-10T01:31:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7915.850871 secs 2023-09-10T01:31:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7915.139254 secs 2023-09-10T01:31:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7915.850871 secs 2023-09-10T01:31:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7915.139254 secs 2023-09-10T01:31:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7915.850871 secs 2023-09-10T01:31:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:25 smithi098 ceph-mon[103124]: pgmap v4387: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:31:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:25 smithi170 ceph-mon[102832]: pgmap v4387: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:31:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:25 smithi082 ceph-mon[98694]: pgmap v4387: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 11 op/s 2023-09-10T01:31:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:27 smithi098 ceph-mon[103124]: pgmap v4388: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T01:31:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:27 smithi170 ceph-mon[102832]: pgmap v4388: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T01:31:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:27 smithi082 ceph-mon[98694]: pgmap v4388: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T01:31:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7920.139312 secs 2023-09-10T01:31:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7920.139312 secs 2023-09-10T01:31:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7920.139312 secs 2023-09-10T01:31:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7920.850935 secs 2023-09-10T01:31:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:29 smithi098 ceph-mon[103124]: pgmap v4389: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 9 op/s 2023-09-10T01:31:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7920.850935 secs 2023-09-10T01:31:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:29 smithi170 ceph-mon[102832]: pgmap v4389: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 9 op/s 2023-09-10T01:31:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7920.850935 secs 2023-09-10T01:31:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:29 smithi082 ceph-mon[98694]: pgmap v4389: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 9 op/s 2023-09-10T01:31:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:31 smithi098 ceph-mon[103124]: pgmap v4390: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 64 KiB/s wr, 15 op/s 2023-09-10T01:31:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:31 smithi170 ceph-mon[102832]: pgmap v4390: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 64 KiB/s wr, 15 op/s 2023-09-10T01:31:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:31 smithi082 ceph-mon[98694]: pgmap v4390: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 64 KiB/s wr, 15 op/s 2023-09-10T01:31:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7923.657467 secs 2023-09-10T01:31:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7923.657467 secs 2023-09-10T01:31:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7923.657467 secs 2023-09-10T01:31:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:33 smithi098 ceph-mon[103124]: pgmap v4391: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T01:31:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7925.139399 secs 2023-09-10T01:31:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7925.851000 secs 2023-09-10T01:31:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:33 smithi170 ceph-mon[102832]: pgmap v4391: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T01:31:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7925.139399 secs 2023-09-10T01:31:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7925.851000 secs 2023-09-10T01:31:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:33 smithi082 ceph-mon[98694]: pgmap v4391: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T01:31:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7925.139399 secs 2023-09-10T01:31:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7925.851000 secs 2023-09-10T01:31:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:34 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:31:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:31:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:31:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:34 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:31:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:31:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:31:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:34 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:31:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:31:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:31:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:35 smithi170 ceph-mon[102832]: pgmap v4392: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T01:31:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:35 smithi082 ceph-mon[98694]: pgmap v4392: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T01:31:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:35 smithi098 ceph-mon[103124]: pgmap v4392: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T01:31:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:37 smithi170 ceph-mon[102832]: pgmap v4393: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 97 KiB/s wr, 17 op/s 2023-09-10T01:31:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7928.657524 secs 2023-09-10T01:31:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:37 smithi082 ceph-mon[98694]: pgmap v4393: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 97 KiB/s wr, 17 op/s 2023-09-10T01:31:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7928.657524 secs 2023-09-10T01:31:38.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:37 smithi098 ceph-mon[103124]: pgmap v4393: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 97 KiB/s wr, 17 op/s 2023-09-10T01:31:38.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7928.657524 secs 2023-09-10T01:31:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7930.139456 secs 2023-09-10T01:31:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7930.851069 secs 2023-09-10T01:31:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7930.139456 secs 2023-09-10T01:31:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7930.851069 secs 2023-09-10T01:31:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7930.139456 secs 2023-09-10T01:31:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7930.851069 secs 2023-09-10T01:31:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:39 smithi170 ceph-mon[102832]: pgmap v4394: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 62 KiB/s wr, 12 op/s 2023-09-10T01:31:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:39 smithi082 ceph-mon[98694]: pgmap v4394: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 62 KiB/s wr, 12 op/s 2023-09-10T01:31:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:39 smithi098 ceph-mon[103124]: pgmap v4394: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 62 KiB/s wr, 12 op/s 2023-09-10T01:31:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:41 smithi170 ceph-mon[102832]: pgmap v4395: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 82 KiB/s wr, 18 op/s 2023-09-10T01:31:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:41 smithi082 ceph-mon[98694]: pgmap v4395: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 82 KiB/s wr, 18 op/s 2023-09-10T01:31:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:41 smithi098 ceph-mon[103124]: pgmap v4395: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 82 KiB/s wr, 18 op/s 2023-09-10T01:31:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:42 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:31:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:42 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:31:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:42 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:31:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:43 smithi170 ceph-mon[102832]: pgmap v4396: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 54 KiB/s wr, 12 op/s 2023-09-10T01:31:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:43 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:31:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:31:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:31:44.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7935.139514 secs 2023-09-10T01:31:44.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7935.851135 secs 2023-09-10T01:31:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:43 smithi082 ceph-mon[98694]: pgmap v4396: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 54 KiB/s wr, 12 op/s 2023-09-10T01:31:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:43 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:31:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:31:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:31:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7935.139514 secs 2023-09-10T01:31:44.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7935.851135 secs 2023-09-10T01:31:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:43 smithi098 ceph-mon[103124]: pgmap v4396: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 54 KiB/s wr, 12 op/s 2023-09-10T01:31:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:43 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:31:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:31:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:31:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7935.139514 secs 2023-09-10T01:31:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7935.851135 secs 2023-09-10T01:31:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:45 smithi170 ceph-mon[102832]: pgmap v4397: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 54 KiB/s wr, 12 op/s 2023-09-10T01:31:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:45 smithi082 ceph-mon[98694]: pgmap v4397: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 54 KiB/s wr, 12 op/s 2023-09-10T01:31:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:45 smithi098 ceph-mon[103124]: pgmap v4397: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 54 KiB/s wr, 12 op/s 2023-09-10T01:31:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:47 smithi170 ceph-mon[102832]: pgmap v4398: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 94 KiB/s wr, 18 op/s 2023-09-10T01:31:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7938.657676 secs 2023-09-10T01:31:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:47 smithi082 ceph-mon[98694]: pgmap v4398: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 94 KiB/s wr, 18 op/s 2023-09-10T01:31:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7938.657676 secs 2023-09-10T01:31:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:47 smithi098 ceph-mon[103124]: pgmap v4398: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 94 KiB/s wr, 18 op/s 2023-09-10T01:31:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7938.657676 secs 2023-09-10T01:31:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7940.139573 secs 2023-09-10T01:31:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7940.851203 secs 2023-09-10T01:31:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7940.139573 secs 2023-09-10T01:31:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7940.851203 secs 2023-09-10T01:31:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7940.139573 secs 2023-09-10T01:31:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7940.851203 secs 2023-09-10T01:31:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:49 smithi170 ceph-mon[102832]: pgmap v4399: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 60 KiB/s wr, 12 op/s 2023-09-10T01:31:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:49 smithi082 ceph-mon[98694]: pgmap v4399: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 60 KiB/s wr, 12 op/s 2023-09-10T01:31:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:49 smithi098 ceph-mon[103124]: pgmap v4399: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 60 KiB/s wr, 12 op/s 2023-09-10T01:31:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:51 smithi170 ceph-mon[102832]: pgmap v4400: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 81 KiB/s wr, 18 op/s 2023-09-10T01:31:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:31:52.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:31:52.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:31:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:51 smithi082 ceph-mon[98694]: pgmap v4400: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 81 KiB/s wr, 18 op/s 2023-09-10T01:31:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:31:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:31:52.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:31:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:51 smithi098 ceph-mon[103124]: pgmap v4400: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 81 KiB/s wr, 18 op/s 2023-09-10T01:31:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:31:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:31:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:31:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7943.657735 secs 2023-09-10T01:31:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:52 smithi170 ceph-mon[102832]: pgmap v4401: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 12 op/s 2023-09-10T01:31:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7943.657735 secs 2023-09-10T01:31:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:52 smithi082 ceph-mon[98694]: pgmap v4401: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 12 op/s 2023-09-10T01:31:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7943.657735 secs 2023-09-10T01:31:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:52 smithi098 ceph-mon[103124]: pgmap v4401: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 12 op/s 2023-09-10T01:31:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7945.139652 secs 2023-09-10T01:31:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7945.851267 secs 2023-09-10T01:31:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7945.139652 secs 2023-09-10T01:31:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7945.851267 secs 2023-09-10T01:31:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7945.139652 secs 2023-09-10T01:31:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7945.851267 secs 2023-09-10T01:31:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:54 smithi170 ceph-mon[102832]: pgmap v4402: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 12 op/s 2023-09-10T01:31:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:54 smithi082 ceph-mon[98694]: pgmap v4402: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 12 op/s 2023-09-10T01:31:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:54 smithi098 ceph-mon[103124]: pgmap v4402: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 12 op/s 2023-09-10T01:31:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:57 smithi082 ceph-mon[98694]: pgmap v4403: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 88 KiB/s wr, 17 op/s 2023-09-10T01:31:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7948.657809 secs 2023-09-10T01:31:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:57 smithi098 ceph-mon[103124]: pgmap v4403: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 88 KiB/s wr, 17 op/s 2023-09-10T01:31:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7948.657809 secs 2023-09-10T01:31:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:57 smithi170 ceph-mon[102832]: pgmap v4403: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 88 KiB/s wr, 17 op/s 2023-09-10T01:31:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7948.657809 secs 2023-09-10T01:31:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7950.139708 secs 2023-09-10T01:31:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7950.851335 secs 2023-09-10T01:31:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:31:59 smithi082 ceph-mon[98694]: pgmap v4404: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T01:32:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7950.139708 secs 2023-09-10T01:32:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7950.851335 secs 2023-09-10T01:32:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:31:59 smithi098 ceph-mon[103124]: pgmap v4404: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T01:32:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7950.139708 secs 2023-09-10T01:32:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7950.851335 secs 2023-09-10T01:32:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:31:59 smithi170 ceph-mon[102832]: pgmap v4404: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T01:32:01.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:01 smithi082 ceph-mon[98694]: pgmap v4405: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 16 op/s 2023-09-10T01:32:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:01 smithi098 ceph-mon[103124]: pgmap v4405: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 16 op/s 2023-09-10T01:32:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:01 smithi170 ceph-mon[102832]: pgmap v4405: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 16 op/s 2023-09-10T01:32:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7953.657922 secs 2023-09-10T01:32:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7953.657922 secs 2023-09-10T01:32:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7953.657922 secs 2023-09-10T01:32:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:03 smithi098 ceph-mon[103124]: pgmap v4406: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T01:32:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:03 smithi170 ceph-mon[102832]: pgmap v4406: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T01:32:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:03 smithi082 ceph-mon[98694]: pgmap v4406: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T01:32:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7955.139768 secs 2023-09-10T01:32:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7955.851406 secs 2023-09-10T01:32:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7955.139768 secs 2023-09-10T01:32:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7955.851406 secs 2023-09-10T01:32:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7955.139768 secs 2023-09-10T01:32:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7955.851406 secs 2023-09-10T01:32:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:05 smithi098 ceph-mon[103124]: pgmap v4407: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T01:32:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:05 smithi170 ceph-mon[102832]: pgmap v4407: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T01:32:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:05 smithi082 ceph-mon[98694]: pgmap v4407: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T01:32:08.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:07 smithi098 ceph-mon[103124]: pgmap v4408: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 88 KiB/s wr, 30 op/s 2023-09-10T01:32:08.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7958.658038 secs 2023-09-10T01:32:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:07 smithi170 ceph-mon[102832]: pgmap v4408: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 88 KiB/s wr, 30 op/s 2023-09-10T01:32:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7958.658038 secs 2023-09-10T01:32:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:07 smithi082 ceph-mon[98694]: pgmap v4408: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 88 KiB/s wr, 30 op/s 2023-09-10T01:32:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7958.658038 secs 2023-09-10T01:32:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:08 smithi098 ceph-mon[103124]: osdmap e345: 12 total, 12 up, 12 in 2023-09-10T01:32:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7960.139824 secs 2023-09-10T01:32:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:08 smithi170 ceph-mon[102832]: osdmap e345: 12 total, 12 up, 12 in 2023-09-10T01:32:09.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7960.139824 secs 2023-09-10T01:32:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:08 smithi082 ceph-mon[98694]: osdmap e345: 12 total, 12 up, 12 in 2023-09-10T01:32:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7960.139824 secs 2023-09-10T01:32:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7960.851473 secs 2023-09-10T01:32:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:09 smithi098 ceph-mon[103124]: pgmap v4410: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 74 KiB/s wr, 29 op/s 2023-09-10T01:32:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7960.851473 secs 2023-09-10T01:32:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:09 smithi170 ceph-mon[102832]: pgmap v4410: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 74 KiB/s wr, 29 op/s 2023-09-10T01:32:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7960.851473 secs 2023-09-10T01:32:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:09 smithi082 ceph-mon[98694]: pgmap v4410: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 74 KiB/s wr, 29 op/s 2023-09-10T01:32:11.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:11 smithi170 ceph-mon[102832]: pgmap v4411: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T01:32:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:11 smithi082 ceph-mon[98694]: pgmap v4411: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T01:32:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:11 smithi098 ceph-mon[103124]: pgmap v4411: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T01:32:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7963.658098 secs 2023-09-10T01:32:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7963.658098 secs 2023-09-10T01:32:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7963.658098 secs 2023-09-10T01:32:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:13 smithi170 ceph-mon[102832]: pgmap v4412: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T01:32:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:13 smithi082 ceph-mon[98694]: pgmap v4412: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T01:32:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:13 smithi098 ceph-mon[103124]: pgmap v4412: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T01:32:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7965.139888 secs 2023-09-10T01:32:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7965.851544 secs 2023-09-10T01:32:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7965.139888 secs 2023-09-10T01:32:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7965.851544 secs 2023-09-10T01:32:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7965.139888 secs 2023-09-10T01:32:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7965.851544 secs 2023-09-10T01:32:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:15 smithi170 ceph-mon[102832]: pgmap v4413: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T01:32:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:15 smithi170 ceph-mon[102832]: osdmap e346: 12 total, 12 up, 12 in 2023-09-10T01:32:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:15 smithi082 ceph-mon[98694]: pgmap v4413: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T01:32:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:15 smithi082 ceph-mon[98694]: osdmap e346: 12 total, 12 up, 12 in 2023-09-10T01:32:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:15 smithi098 ceph-mon[103124]: pgmap v4413: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T01:32:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:15 smithi098 ceph-mon[103124]: osdmap e346: 12 total, 12 up, 12 in 2023-09-10T01:32:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:17 smithi170 ceph-mon[102832]: pgmap v4415: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 13 op/s 2023-09-10T01:32:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7968.658159 secs 2023-09-10T01:32:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:17 smithi082 ceph-mon[98694]: pgmap v4415: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 13 op/s 2023-09-10T01:32:17.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7968.658159 secs 2023-09-10T01:32:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:17 smithi098 ceph-mon[103124]: pgmap v4415: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 13 op/s 2023-09-10T01:32:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7968.658159 secs 2023-09-10T01:32:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7970.139972 secs 2023-09-10T01:32:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7970.851615 secs 2023-09-10T01:32:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:19 smithi170 ceph-mon[102832]: pgmap v4416: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T01:32:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7970.139972 secs 2023-09-10T01:32:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7970.851615 secs 2023-09-10T01:32:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:19 smithi082 ceph-mon[98694]: pgmap v4416: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T01:32:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7970.139972 secs 2023-09-10T01:32:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7970.851615 secs 2023-09-10T01:32:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:19 smithi098 ceph-mon[103124]: pgmap v4416: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T01:32:21.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:21 smithi170 ceph-mon[102832]: pgmap v4417: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T01:32:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:21 smithi082 ceph-mon[98694]: pgmap v4417: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T01:32:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:21 smithi098 ceph-mon[103124]: pgmap v4417: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T01:32:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:23 smithi082 ceph-mon[98694]: pgmap v4418: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T01:32:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:32:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:32:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:23 smithi098 ceph-mon[103124]: pgmap v4418: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T01:32:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:32:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:32:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:23 smithi170 ceph-mon[102832]: pgmap v4418: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T01:32:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:32:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:32:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7975.140062 secs 2023-09-10T01:32:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7975.851683 secs 2023-09-10T01:32:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:32:24.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:32:24.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:32:24.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:32:24.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:32:24.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:32:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7975.140062 secs 2023-09-10T01:32:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7975.851683 secs 2023-09-10T01:32:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:32:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:32:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:32:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:32:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:32:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:32:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7975.140062 secs 2023-09-10T01:32:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7975.851683 secs 2023-09-10T01:32:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:32:25.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:32:25.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:32:25.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:32:25.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:32:25.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:32:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:25 smithi082 ceph-mon[98694]: pgmap v4419: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T01:32:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:25 smithi098 ceph-mon[103124]: pgmap v4419: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T01:32:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:25 smithi170 ceph-mon[102832]: pgmap v4419: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T01:32:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:27 smithi082 ceph-mon[98694]: pgmap v4420: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:32:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:27 smithi098 ceph-mon[103124]: pgmap v4420: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:32:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:27 smithi170 ceph-mon[102832]: pgmap v4420: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:32:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7980.140146 secs 2023-09-10T01:32:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7980.851747 secs 2023-09-10T01:32:29.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:29 smithi082 ceph-mon[98694]: pgmap v4421: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 10 op/s 2023-09-10T01:32:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7980.140146 secs 2023-09-10T01:32:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7980.851747 secs 2023-09-10T01:32:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:29 smithi098 ceph-mon[103124]: pgmap v4421: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 10 op/s 2023-09-10T01:32:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7980.140146 secs 2023-09-10T01:32:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7980.851747 secs 2023-09-10T01:32:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:29 smithi170 ceph-mon[102832]: pgmap v4421: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 10 op/s 2023-09-10T01:32:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:31 smithi082 ceph-mon[98694]: pgmap v4422: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 76 KiB/s wr, 16 op/s 2023-09-10T01:32:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:31 smithi098 ceph-mon[103124]: pgmap v4422: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 76 KiB/s wr, 16 op/s 2023-09-10T01:32:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:31 smithi170 ceph-mon[102832]: pgmap v4422: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 76 KiB/s wr, 16 op/s 2023-09-10T01:32:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7983.658391 secs 2023-09-10T01:32:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7983.658391 secs 2023-09-10T01:32:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7983.658391 secs 2023-09-10T01:32:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:33 smithi082 ceph-mon[98694]: pgmap v4423: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T01:32:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:33 smithi098 ceph-mon[103124]: pgmap v4423: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T01:32:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:33 smithi170 ceph-mon[102832]: pgmap v4423: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T01:32:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7985.140228 secs 2023-09-10T01:32:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7985.851814 secs 2023-09-10T01:32:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7985.140228 secs 2023-09-10T01:32:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7985.851814 secs 2023-09-10T01:32:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7985.140228 secs 2023-09-10T01:32:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7985.851814 secs 2023-09-10T01:32:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:35 smithi082 ceph-mon[98694]: pgmap v4424: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T01:32:35.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:35 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:32:35.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:35 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:32:35.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:35 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:32:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:35 smithi098 ceph-mon[103124]: pgmap v4424: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T01:32:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:35 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:32:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:35 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:32:36.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:35 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:32:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:35 smithi170 ceph-mon[102832]: pgmap v4424: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T01:32:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:35 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:32:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:35 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:32:36.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:35 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:32:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:37 smithi082 ceph-mon[98694]: pgmap v4425: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 99 KiB/s wr, 17 op/s 2023-09-10T01:32:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7988.658497 secs 2023-09-10T01:32:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:37 smithi098 ceph-mon[103124]: pgmap v4425: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 99 KiB/s wr, 17 op/s 2023-09-10T01:32:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7988.658497 secs 2023-09-10T01:32:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:37 smithi170 ceph-mon[102832]: pgmap v4425: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 99 KiB/s wr, 17 op/s 2023-09-10T01:32:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7988.658497 secs 2023-09-10T01:32:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7990.140286 secs 2023-09-10T01:32:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7990.851882 secs 2023-09-10T01:32:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:39 smithi082 ceph-mon[98694]: pgmap v4426: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 63 KiB/s wr, 12 op/s 2023-09-10T01:32:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7990.140286 secs 2023-09-10T01:32:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7990.851882 secs 2023-09-10T01:32:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:39 smithi098 ceph-mon[103124]: pgmap v4426: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 63 KiB/s wr, 12 op/s 2023-09-10T01:32:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7990.140286 secs 2023-09-10T01:32:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7990.851882 secs 2023-09-10T01:32:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:39 smithi170 ceph-mon[102832]: pgmap v4426: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 63 KiB/s wr, 12 op/s 2023-09-10T01:32:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:41 smithi098 ceph-mon[103124]: pgmap v4427: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 85 KiB/s wr, 17 op/s 2023-09-10T01:32:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:41 smithi170 ceph-mon[102832]: pgmap v4427: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 85 KiB/s wr, 17 op/s 2023-09-10T01:32:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:41 smithi082 ceph-mon[98694]: pgmap v4427: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 85 KiB/s wr, 17 op/s 2023-09-10T01:32:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:42 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:32:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:42 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:32:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:42 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:32:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:43 smithi098 ceph-mon[103124]: pgmap v4428: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T01:32:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:43 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:32:44.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:32:44.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:32:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:43 smithi170 ceph-mon[102832]: pgmap v4428: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T01:32:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:43 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:32:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:32:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:32:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:43 smithi082 ceph-mon[98694]: pgmap v4428: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T01:32:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:43 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:32:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:32:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:32:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7995.140374 secs 2023-09-10T01:32:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 7995.851953 secs 2023-09-10T01:32:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7995.140374 secs 2023-09-10T01:32:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 7995.851953 secs 2023-09-10T01:32:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7995.140374 secs 2023-09-10T01:32:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 7995.851953 secs 2023-09-10T01:32:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:45 smithi098 ceph-mon[103124]: pgmap v4429: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T01:32:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:45 smithi170 ceph-mon[102832]: pgmap v4429: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T01:32:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:45 smithi082 ceph-mon[98694]: pgmap v4429: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T01:32:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:47 smithi098 ceph-mon[103124]: pgmap v4430: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 96 KiB/s wr, 17 op/s 2023-09-10T01:32:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 7998.658618 secs 2023-09-10T01:32:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:47 smithi170 ceph-mon[102832]: pgmap v4430: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 96 KiB/s wr, 17 op/s 2023-09-10T01:32:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 7998.658618 secs 2023-09-10T01:32:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:47 smithi082 ceph-mon[98694]: pgmap v4430: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 96 KiB/s wr, 17 op/s 2023-09-10T01:32:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 7998.658618 secs 2023-09-10T01:32:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8000.140434 secs 2023-09-10T01:32:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8000.140434 secs 2023-09-10T01:32:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8000.140434 secs 2023-09-10T01:32:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8000.852041 secs 2023-09-10T01:32:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:49 smithi098 ceph-mon[103124]: pgmap v4431: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T01:32:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8000.852041 secs 2023-09-10T01:32:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:49 smithi170 ceph-mon[102832]: pgmap v4431: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T01:32:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8000.852041 secs 2023-09-10T01:32:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:49 smithi082 ceph-mon[98694]: pgmap v4431: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T01:32:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:51 smithi170 ceph-mon[102832]: pgmap v4432: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 16 op/s 2023-09-10T01:32:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:51 smithi082 ceph-mon[98694]: pgmap v4432: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 16 op/s 2023-09-10T01:32:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:51 smithi098 ceph-mon[103124]: pgmap v4432: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 16 op/s 2023-09-10T01:32:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:32:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:32:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:32:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8003.658675 secs 2023-09-10T01:32:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:32:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:32:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:32:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8003.658675 secs 2023-09-10T01:32:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:32:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:32:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:32:53.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8003.658675 secs 2023-09-10T01:32:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:53 smithi170 ceph-mon[102832]: pgmap v4433: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:32:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:53 smithi082 ceph-mon[98694]: pgmap v4433: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:32:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:53 smithi098 ceph-mon[103124]: pgmap v4433: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:32:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8005.140492 secs 2023-09-10T01:32:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8005.852129 secs 2023-09-10T01:32:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8005.140492 secs 2023-09-10T01:32:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8005.852129 secs 2023-09-10T01:32:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8005.140492 secs 2023-09-10T01:32:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8005.852129 secs 2023-09-10T01:32:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:55 smithi082 ceph-mon[98694]: pgmap v4434: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:32:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:55 smithi098 ceph-mon[103124]: pgmap v4434: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:32:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:55 smithi170 ceph-mon[102832]: pgmap v4434: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:32:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:57 smithi082 ceph-mon[98694]: pgmap v4435: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 16 op/s 2023-09-10T01:32:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8008.658734 secs 2023-09-10T01:32:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:57 smithi098 ceph-mon[103124]: pgmap v4435: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 16 op/s 2023-09-10T01:32:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8008.658734 secs 2023-09-10T01:32:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:57 smithi170 ceph-mon[102832]: pgmap v4435: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 16 op/s 2023-09-10T01:32:58.198 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8008.658734 secs 2023-09-10T01:32:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8010.140551 secs 2023-09-10T01:32:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8010.852195 secs 2023-09-10T01:32:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:32:59 smithi082 ceph-mon[98694]: pgmap v4436: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T01:33:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8010.140551 secs 2023-09-10T01:33:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8010.852195 secs 2023-09-10T01:33:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:32:59 smithi098 ceph-mon[103124]: pgmap v4436: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T01:33:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8010.140551 secs 2023-09-10T01:33:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8010.852195 secs 2023-09-10T01:33:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:32:59 smithi170 ceph-mon[102832]: pgmap v4436: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T01:33:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:01 smithi170 ceph-mon[102832]: pgmap v4437: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 15 op/s 2023-09-10T01:33:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:01 smithi082 ceph-mon[98694]: pgmap v4437: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 15 op/s 2023-09-10T01:33:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:01 smithi098 ceph-mon[103124]: pgmap v4437: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 15 op/s 2023-09-10T01:33:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8013.658781 secs 2023-09-10T01:33:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8013.658781 secs 2023-09-10T01:33:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8013.658781 secs 2023-09-10T01:33:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:03 smithi170 ceph-mon[102832]: pgmap v4438: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 9 op/s 2023-09-10T01:33:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:03 smithi082 ceph-mon[98694]: pgmap v4438: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 9 op/s 2023-09-10T01:33:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:03 smithi098 ceph-mon[103124]: pgmap v4438: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 9 op/s 2023-09-10T01:33:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8015.140608 secs 2023-09-10T01:33:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8015.852262 secs 2023-09-10T01:33:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8015.140608 secs 2023-09-10T01:33:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8015.852262 secs 2023-09-10T01:33:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8015.140608 secs 2023-09-10T01:33:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8015.852262 secs 2023-09-10T01:33:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:05 smithi082 ceph-mon[98694]: pgmap v4439: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 9 op/s 2023-09-10T01:33:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:05 smithi098 ceph-mon[103124]: pgmap v4439: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 9 op/s 2023-09-10T01:33:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:05 smithi170 ceph-mon[102832]: pgmap v4439: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 9 op/s 2023-09-10T01:33:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:07 smithi082 ceph-mon[98694]: pgmap v4440: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T01:33:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8018.658841 secs 2023-09-10T01:33:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:07 smithi098 ceph-mon[103124]: pgmap v4440: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T01:33:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8018.658841 secs 2023-09-10T01:33:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:07 smithi170 ceph-mon[102832]: pgmap v4440: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T01:33:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8018.658841 secs 2023-09-10T01:33:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:08 smithi082 ceph-mon[98694]: osdmap e347: 12 total, 12 up, 12 in 2023-09-10T01:33:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:08 smithi098 ceph-mon[103124]: osdmap e347: 12 total, 12 up, 12 in 2023-09-10T01:33:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:08 smithi170 ceph-mon[102832]: osdmap e347: 12 total, 12 up, 12 in 2023-09-10T01:33:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8020.140706 secs 2023-09-10T01:33:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8020.852327 secs 2023-09-10T01:33:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:09 smithi082 ceph-mon[98694]: pgmap v4442: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T01:33:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8020.140706 secs 2023-09-10T01:33:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8020.852327 secs 2023-09-10T01:33:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:09 smithi098 ceph-mon[103124]: pgmap v4442: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T01:33:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8020.140706 secs 2023-09-10T01:33:10.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8020.852327 secs 2023-09-10T01:33:10.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:09 smithi170 ceph-mon[102832]: pgmap v4442: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T01:33:11.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:11 smithi170 ceph-mon[102832]: pgmap v4443: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T01:33:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:11 smithi082 ceph-mon[98694]: pgmap v4443: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T01:33:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:11 smithi098 ceph-mon[103124]: pgmap v4443: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T01:33:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8023.658900 secs 2023-09-10T01:33:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8023.658900 secs 2023-09-10T01:33:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8023.658900 secs 2023-09-10T01:33:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:13 smithi170 ceph-mon[102832]: pgmap v4444: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T01:33:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:13 smithi082 ceph-mon[98694]: pgmap v4444: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T01:33:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:13 smithi098 ceph-mon[103124]: pgmap v4444: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T01:33:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8025.140763 secs 2023-09-10T01:33:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8025.852393 secs 2023-09-10T01:33:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8025.140763 secs 2023-09-10T01:33:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8025.852393 secs 2023-09-10T01:33:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8025.140763 secs 2023-09-10T01:33:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8025.852393 secs 2023-09-10T01:33:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:15 smithi082 ceph-mon[98694]: pgmap v4445: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T01:33:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:15 smithi082 ceph-mon[98694]: osdmap e348: 12 total, 12 up, 12 in 2023-09-10T01:33:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:15 smithi098 ceph-mon[103124]: pgmap v4445: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T01:33:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:15 smithi098 ceph-mon[103124]: osdmap e348: 12 total, 12 up, 12 in 2023-09-10T01:33:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:15 smithi170 ceph-mon[102832]: pgmap v4445: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T01:33:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:15 smithi170 ceph-mon[102832]: osdmap e348: 12 total, 12 up, 12 in 2023-09-10T01:33:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:17 smithi082 ceph-mon[98694]: pgmap v4447: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 13 op/s 2023-09-10T01:33:17.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8028.658961 secs 2023-09-10T01:33:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:17 smithi098 ceph-mon[103124]: pgmap v4447: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 13 op/s 2023-09-10T01:33:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8028.658961 secs 2023-09-10T01:33:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:17 smithi170 ceph-mon[102832]: pgmap v4447: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 13 op/s 2023-09-10T01:33:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8028.658961 secs 2023-09-10T01:33:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8030.140849 secs 2023-09-10T01:33:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8030.852458 secs 2023-09-10T01:33:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:19 smithi082 ceph-mon[98694]: pgmap v4448: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T01:33:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8030.140849 secs 2023-09-10T01:33:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8030.852458 secs 2023-09-10T01:33:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:19 smithi098 ceph-mon[103124]: pgmap v4448: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T01:33:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8030.140849 secs 2023-09-10T01:33:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8030.852458 secs 2023-09-10T01:33:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:19 smithi170 ceph-mon[102832]: pgmap v4448: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T01:33:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:21 smithi082 ceph-mon[98694]: pgmap v4449: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T01:33:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:21 smithi098 ceph-mon[103124]: pgmap v4449: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T01:33:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:21 smithi170 ceph-mon[102832]: pgmap v4449: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T01:33:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:23 smithi082 ceph-mon[98694]: pgmap v4450: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T01:33:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:33:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:23 smithi098 ceph-mon[103124]: pgmap v4450: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T01:33:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:33:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:23 smithi170 ceph-mon[102832]: pgmap v4450: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T01:33:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:33:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8035.140907 secs 2023-09-10T01:33:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8035.852492 secs 2023-09-10T01:33:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:33:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:33:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:33:24.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:33:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8035.140907 secs 2023-09-10T01:33:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8035.852492 secs 2023-09-10T01:33:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:33:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:33:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:33:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:33:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8035.140907 secs 2023-09-10T01:33:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8035.852492 secs 2023-09-10T01:33:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:33:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:33:25.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:33:25.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:33:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:25 smithi082 ceph-mon[98694]: pgmap v4451: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T01:33:25.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:33:25.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:33:25.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:33:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:25 smithi098 ceph-mon[103124]: pgmap v4451: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T01:33:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:33:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:33:26.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:33:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:25 smithi170 ceph-mon[102832]: pgmap v4451: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T01:33:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:33:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:33:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:33:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:27 smithi082 ceph-mon[98694]: pgmap v4452: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T01:33:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:27 smithi098 ceph-mon[103124]: pgmap v4452: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T01:33:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:27 smithi170 ceph-mon[102832]: pgmap v4452: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T01:33:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8040.140973 secs 2023-09-10T01:33:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8040.852559 secs 2023-09-10T01:33:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:29 smithi098 ceph-mon[103124]: pgmap v4453: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T01:33:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8040.140973 secs 2023-09-10T01:33:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8040.852559 secs 2023-09-10T01:33:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:29 smithi170 ceph-mon[102832]: pgmap v4453: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T01:33:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8040.140973 secs 2023-09-10T01:33:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8040.852559 secs 2023-09-10T01:33:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:29 smithi082 ceph-mon[98694]: pgmap v4453: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T01:33:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:31 smithi098 ceph-mon[103124]: pgmap v4454: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 78 KiB/s wr, 17 op/s 2023-09-10T01:33:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:31 smithi170 ceph-mon[102832]: pgmap v4454: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 78 KiB/s wr, 17 op/s 2023-09-10T01:33:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:31 smithi082 ceph-mon[98694]: pgmap v4454: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 78 KiB/s wr, 17 op/s 2023-09-10T01:33:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:33 smithi098 ceph-mon[103124]: pgmap v4455: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T01:33:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:33 smithi170 ceph-mon[102832]: pgmap v4455: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T01:33:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:33 smithi082 ceph-mon[98694]: pgmap v4455: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T01:33:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8045.141031 secs 2023-09-10T01:33:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8045.852628 secs 2023-09-10T01:33:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8045.141031 secs 2023-09-10T01:33:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8045.852628 secs 2023-09-10T01:33:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8045.141031 secs 2023-09-10T01:33:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8045.852628 secs 2023-09-10T01:33:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:35 smithi098 ceph-mon[103124]: pgmap v4456: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T01:33:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:35 smithi170 ceph-mon[102832]: pgmap v4456: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T01:33:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:35 smithi082 ceph-mon[98694]: pgmap v4456: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T01:33:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:37 smithi098 ceph-mon[103124]: pgmap v4457: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 102 KiB/s wr, 18 op/s 2023-09-10T01:33:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:37 smithi170 ceph-mon[102832]: pgmap v4457: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 102 KiB/s wr, 18 op/s 2023-09-10T01:33:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:37 smithi082 ceph-mon[98694]: pgmap v4457: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 102 KiB/s wr, 18 op/s 2023-09-10T01:33:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8050.141088 secs 2023-09-10T01:33:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8050.141088 secs 2023-09-10T01:33:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8050.141088 secs 2023-09-10T01:33:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8050.852694 secs 2023-09-10T01:33:40.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:39 smithi098 ceph-mon[103124]: pgmap v4458: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 12 op/s 2023-09-10T01:33:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8050.852694 secs 2023-09-10T01:33:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:39 smithi170 ceph-mon[102832]: pgmap v4458: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 12 op/s 2023-09-10T01:33:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8050.852694 secs 2023-09-10T01:33:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:39 smithi082 ceph-mon[98694]: pgmap v4458: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 12 op/s 2023-09-10T01:33:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:41 smithi098 ceph-mon[103124]: pgmap v4459: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 88 KiB/s wr, 18 op/s 2023-09-10T01:33:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:41 smithi170 ceph-mon[102832]: pgmap v4459: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 88 KiB/s wr, 18 op/s 2023-09-10T01:33:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:41 smithi082 ceph-mon[98694]: pgmap v4459: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 88 KiB/s wr, 18 op/s 2023-09-10T01:33:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8053.659308 secs 2023-09-10T01:33:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8053.659308 secs 2023-09-10T01:33:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8053.659308 secs 2023-09-10T01:33:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:43 smithi098 ceph-mon[103124]: pgmap v4460: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T01:33:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:33:44.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:33:44.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:33:44.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8055.141146 secs 2023-09-10T01:33:44.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8055.852759 secs 2023-09-10T01:33:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:43 smithi170 ceph-mon[102832]: pgmap v4460: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T01:33:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:33:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:33:44.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:33:44.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8055.141146 secs 2023-09-10T01:33:44.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8055.852759 secs 2023-09-10T01:33:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:43 smithi082 ceph-mon[98694]: pgmap v4460: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T01:33:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:33:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:33:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:33:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8055.141146 secs 2023-09-10T01:33:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8055.852759 secs 2023-09-10T01:33:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:45 smithi098 ceph-mon[103124]: pgmap v4461: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T01:33:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:45 smithi170 ceph-mon[102832]: pgmap v4461: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T01:33:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:45 smithi082 ceph-mon[98694]: pgmap v4461: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T01:33:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:47 smithi098 ceph-mon[103124]: pgmap v4462: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 18 op/s 2023-09-10T01:33:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8058.659393 secs 2023-09-10T01:33:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:47 smithi170 ceph-mon[102832]: pgmap v4462: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 18 op/s 2023-09-10T01:33:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8058.659393 secs 2023-09-10T01:33:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:47 smithi082 ceph-mon[98694]: pgmap v4462: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 18 op/s 2023-09-10T01:33:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8058.659393 secs 2023-09-10T01:33:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8060.141199 secs 2023-09-10T01:33:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8060.852824 secs 2023-09-10T01:33:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8060.141199 secs 2023-09-10T01:33:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8060.852824 secs 2023-09-10T01:33:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8060.141199 secs 2023-09-10T01:33:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8060.852824 secs 2023-09-10T01:33:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:49 smithi098 ceph-mon[103124]: pgmap v4463: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T01:33:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:49 smithi170 ceph-mon[102832]: pgmap v4463: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T01:33:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:49 smithi082 ceph-mon[98694]: pgmap v4463: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T01:33:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:51 smithi170 ceph-mon[102832]: pgmap v4464: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 16 op/s 2023-09-10T01:33:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:51 smithi082 ceph-mon[98694]: pgmap v4464: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 16 op/s 2023-09-10T01:33:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:51 smithi098 ceph-mon[103124]: pgmap v4464: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 16 op/s 2023-09-10T01:33:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8063.659452 secs 2023-09-10T01:33:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8063.659452 secs 2023-09-10T01:33:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8063.659452 secs 2023-09-10T01:33:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:53 smithi170 ceph-mon[102832]: pgmap v4465: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T01:33:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8065.141285 secs 2023-09-10T01:33:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8065.852890 secs 2023-09-10T01:33:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:53 smithi082 ceph-mon[98694]: pgmap v4465: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T01:33:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8065.141285 secs 2023-09-10T01:33:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8065.852890 secs 2023-09-10T01:33:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:53 smithi098 ceph-mon[103124]: pgmap v4465: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T01:33:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8065.141285 secs 2023-09-10T01:33:54.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8065.852890 secs 2023-09-10T01:33:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:55 smithi170 ceph-mon[102832]: pgmap v4466: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T01:33:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:55 smithi082 ceph-mon[98694]: pgmap v4466: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T01:33:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:55 smithi098 ceph-mon[103124]: pgmap v4466: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T01:33:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:57 smithi170 ceph-mon[102832]: pgmap v4467: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 79 KiB/s wr, 16 op/s 2023-09-10T01:33:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8068.659545 secs 2023-09-10T01:33:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:57 smithi082 ceph-mon[98694]: pgmap v4467: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 79 KiB/s wr, 16 op/s 2023-09-10T01:33:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8068.659545 secs 2023-09-10T01:33:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:57 smithi098 ceph-mon[103124]: pgmap v4467: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 79 KiB/s wr, 16 op/s 2023-09-10T01:33:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8068.659545 secs 2023-09-10T01:33:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8070.141343 secs 2023-09-10T01:33:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8070.852958 secs 2023-09-10T01:33:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8070.141343 secs 2023-09-10T01:33:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8070.852958 secs 2023-09-10T01:33:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8070.141343 secs 2023-09-10T01:33:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8070.852958 secs 2023-09-10T01:34:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:33:59 smithi170 ceph-mon[102832]: pgmap v4468: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 10 op/s 2023-09-10T01:34:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:33:59 smithi082 ceph-mon[98694]: pgmap v4468: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 10 op/s 2023-09-10T01:34:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:33:59 smithi098 ceph-mon[103124]: pgmap v4468: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 10 op/s 2023-09-10T01:34:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:01 smithi170 ceph-mon[102832]: pgmap v4469: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 KiB/s wr, 15 op/s 2023-09-10T01:34:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:01 smithi082 ceph-mon[98694]: pgmap v4469: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 KiB/s wr, 15 op/s 2023-09-10T01:34:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:01 smithi098 ceph-mon[103124]: pgmap v4469: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 KiB/s wr, 15 op/s 2023-09-10T01:34:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8073.659605 secs 2023-09-10T01:34:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8073.659605 secs 2023-09-10T01:34:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8073.659605 secs 2023-09-10T01:34:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:03 smithi170 ceph-mon[102832]: pgmap v4470: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T01:34:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8075.141406 secs 2023-09-10T01:34:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8075.853024 secs 2023-09-10T01:34:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:03 smithi082 ceph-mon[98694]: pgmap v4470: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T01:34:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8075.141406 secs 2023-09-10T01:34:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8075.853024 secs 2023-09-10T01:34:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:03 smithi098 ceph-mon[103124]: pgmap v4470: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T01:34:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8075.141406 secs 2023-09-10T01:34:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8075.853024 secs 2023-09-10T01:34:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:05 smithi170 ceph-mon[102832]: pgmap v4471: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T01:34:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:05 smithi082 ceph-mon[98694]: pgmap v4471: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T01:34:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:05 smithi098 ceph-mon[103124]: pgmap v4471: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T01:34:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:07 smithi170 ceph-mon[102832]: pgmap v4472: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T01:34:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8078.659700 secs 2023-09-10T01:34:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:07 smithi082 ceph-mon[98694]: pgmap v4472: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T01:34:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8078.659700 secs 2023-09-10T01:34:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:07 smithi098 ceph-mon[103124]: pgmap v4472: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T01:34:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8078.659700 secs 2023-09-10T01:34:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:08 smithi170 ceph-mon[102832]: osdmap e349: 12 total, 12 up, 12 in 2023-09-10T01:34:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8080.141472 secs 2023-09-10T01:34:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8080.853093 secs 2023-09-10T01:34:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:08 smithi082 ceph-mon[98694]: osdmap e349: 12 total, 12 up, 12 in 2023-09-10T01:34:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8080.141472 secs 2023-09-10T01:34:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8080.853093 secs 2023-09-10T01:34:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:08 smithi098 ceph-mon[103124]: osdmap e349: 12 total, 12 up, 12 in 2023-09-10T01:34:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8080.141472 secs 2023-09-10T01:34:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8080.853093 secs 2023-09-10T01:34:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:09 smithi170 ceph-mon[102832]: pgmap v4474: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 75 KiB/s wr, 29 op/s 2023-09-10T01:34:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:09 smithi082 ceph-mon[98694]: pgmap v4474: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 75 KiB/s wr, 29 op/s 2023-09-10T01:34:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:09 smithi098 ceph-mon[103124]: pgmap v4474: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 75 KiB/s wr, 29 op/s 2023-09-10T01:34:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:11 smithi170 ceph-mon[102832]: pgmap v4475: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T01:34:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:11 smithi082 ceph-mon[98694]: pgmap v4475: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T01:34:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:11 smithi098 ceph-mon[103124]: pgmap v4475: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T01:34:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8083.659759 secs 2023-09-10T01:34:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8083.659759 secs 2023-09-10T01:34:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8083.659759 secs 2023-09-10T01:34:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:13 smithi170 ceph-mon[102832]: pgmap v4476: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T01:34:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8085.141530 secs 2023-09-10T01:34:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8085.853153 secs 2023-09-10T01:34:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:13 smithi082 ceph-mon[98694]: pgmap v4476: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T01:34:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8085.141530 secs 2023-09-10T01:34:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8085.853153 secs 2023-09-10T01:34:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:13 smithi098 ceph-mon[103124]: pgmap v4476: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T01:34:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8085.141530 secs 2023-09-10T01:34:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8085.853153 secs 2023-09-10T01:34:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:15 smithi170 ceph-mon[102832]: pgmap v4477: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T01:34:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:15 smithi170 ceph-mon[102832]: osdmap e350: 12 total, 12 up, 12 in 2023-09-10T01:34:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:15 smithi082 ceph-mon[98694]: pgmap v4477: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T01:34:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:15 smithi082 ceph-mon[98694]: osdmap e350: 12 total, 12 up, 12 in 2023-09-10T01:34:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:15 smithi098 ceph-mon[103124]: pgmap v4477: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T01:34:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:15 smithi098 ceph-mon[103124]: osdmap e350: 12 total, 12 up, 12 in 2023-09-10T01:34:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:17 smithi170 ceph-mon[102832]: pgmap v4479: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 13 op/s 2023-09-10T01:34:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8088.659825 secs 2023-09-10T01:34:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:17 smithi082 ceph-mon[98694]: pgmap v4479: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 13 op/s 2023-09-10T01:34:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8088.659825 secs 2023-09-10T01:34:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:17 smithi098 ceph-mon[103124]: pgmap v4479: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 13 op/s 2023-09-10T01:34:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8088.659825 secs 2023-09-10T01:34:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8090.141602 secs 2023-09-10T01:34:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8090.853219 secs 2023-09-10T01:34:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8090.141602 secs 2023-09-10T01:34:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8090.853219 secs 2023-09-10T01:34:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8090.141602 secs 2023-09-10T01:34:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8090.853219 secs 2023-09-10T01:34:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:19 smithi170 ceph-mon[102832]: pgmap v4480: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T01:34:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:19 smithi082 ceph-mon[98694]: pgmap v4480: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T01:34:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:19 smithi098 ceph-mon[103124]: pgmap v4480: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T01:34:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:21 smithi170 ceph-mon[102832]: pgmap v4481: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-09-10T01:34:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:21 smithi082 ceph-mon[98694]: pgmap v4481: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-09-10T01:34:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:21 smithi098 ceph-mon[103124]: pgmap v4481: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-09-10T01:34:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:34:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:34:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:34:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:23 smithi170 ceph-mon[102832]: pgmap v4482: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-09-10T01:34:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:34:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:34:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:34:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8095.141663 secs 2023-09-10T01:34:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8095.853285 secs 2023-09-10T01:34:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:23 smithi082 ceph-mon[98694]: pgmap v4482: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-09-10T01:34:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:34:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:34:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:34:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8095.141663 secs 2023-09-10T01:34:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8095.853285 secs 2023-09-10T01:34:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:23 smithi098 ceph-mon[103124]: pgmap v4482: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-09-10T01:34:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:34:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:34:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:34:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8095.141663 secs 2023-09-10T01:34:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8095.853285 secs 2023-09-10T01:34:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:25 smithi170 ceph-mon[102832]: pgmap v4483: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-09-10T01:34:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:34:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:25 smithi082 ceph-mon[98694]: pgmap v4483: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-09-10T01:34:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:34:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:25 smithi098 ceph-mon[103124]: pgmap v4483: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-09-10T01:34:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:34:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:34:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:34:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:34:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:26 smithi170 ceph-mon[102832]: pgmap v4484: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 11 op/s 2023-09-10T01:34:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:34:27.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:34:27.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:34:27.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:26 smithi082 ceph-mon[98694]: pgmap v4484: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 11 op/s 2023-09-10T01:34:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:34:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:34:27.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:34:27.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:26 smithi098 ceph-mon[103124]: pgmap v4484: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 11 op/s 2023-09-10T01:34:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8100.141720 secs 2023-09-10T01:34:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8100.853348 secs 2023-09-10T01:34:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8100.141720 secs 2023-09-10T01:34:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8100.853348 secs 2023-09-10T01:34:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8100.141720 secs 2023-09-10T01:34:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8100.853348 secs 2023-09-10T01:34:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:29 smithi170 ceph-mon[102832]: pgmap v4485: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 10 op/s 2023-09-10T01:34:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:29 smithi082 ceph-mon[98694]: pgmap v4485: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 10 op/s 2023-09-10T01:34:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:29 smithi098 ceph-mon[103124]: pgmap v4485: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 10 op/s 2023-09-10T01:34:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:31 smithi170 ceph-mon[102832]: pgmap v4486: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 80 KiB/s wr, 16 op/s 2023-09-10T01:34:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:31 smithi082 ceph-mon[98694]: pgmap v4486: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 80 KiB/s wr, 16 op/s 2023-09-10T01:34:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:31 smithi098 ceph-mon[103124]: pgmap v4486: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 80 KiB/s wr, 16 op/s 2023-09-10T01:34:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:33 smithi170 ceph-mon[102832]: pgmap v4487: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:34:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8105.141777 secs 2023-09-10T01:34:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8105.853414 secs 2023-09-10T01:34:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:33 smithi082 ceph-mon[98694]: pgmap v4487: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:34:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8105.141777 secs 2023-09-10T01:34:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8105.853414 secs 2023-09-10T01:34:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:33 smithi098 ceph-mon[103124]: pgmap v4487: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:34:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8105.141777 secs 2023-09-10T01:34:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8105.853414 secs 2023-09-10T01:34:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:35 smithi170 ceph-mon[102832]: pgmap v4488: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:34:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:35 smithi082 ceph-mon[98694]: pgmap v4488: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:34:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:35 smithi098 ceph-mon[103124]: pgmap v4488: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:34:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:37 smithi170 ceph-mon[102832]: pgmap v4489: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 17 op/s 2023-09-10T01:34:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:37 smithi082 ceph-mon[98694]: pgmap v4489: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 17 op/s 2023-09-10T01:34:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:37 smithi098 ceph-mon[103124]: pgmap v4489: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 17 op/s 2023-09-10T01:34:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8110.141852 secs 2023-09-10T01:34:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8110.853482 secs 2023-09-10T01:34:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8110.141852 secs 2023-09-10T01:34:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8110.853482 secs 2023-09-10T01:34:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8110.141852 secs 2023-09-10T01:34:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8110.853482 secs 2023-09-10T01:34:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:39 smithi170 ceph-mon[102832]: pgmap v4490: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T01:34:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:39 smithi082 ceph-mon[98694]: pgmap v4490: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T01:34:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:39 smithi098 ceph-mon[103124]: pgmap v4490: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T01:34:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:41 smithi170 ceph-mon[102832]: pgmap v4491: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 89 KiB/s wr, 17 op/s 2023-09-10T01:34:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:41 smithi082 ceph-mon[98694]: pgmap v4491: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 89 KiB/s wr, 17 op/s 2023-09-10T01:34:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:41 smithi098 ceph-mon[103124]: pgmap v4491: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 89 KiB/s wr, 17 op/s 2023-09-10T01:34:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:43 smithi170 ceph-mon[102832]: pgmap v4492: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T01:34:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8115.141936 secs 2023-09-10T01:34:44.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8115.853554 secs 2023-09-10T01:34:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:43 smithi082 ceph-mon[98694]: pgmap v4492: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T01:34:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8115.141936 secs 2023-09-10T01:34:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8115.853554 secs 2023-09-10T01:34:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:43 smithi098 ceph-mon[103124]: pgmap v4492: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T01:34:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8115.141936 secs 2023-09-10T01:34:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8115.853554 secs 2023-09-10T01:34:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:45 smithi170 ceph-mon[102832]: pgmap v4493: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T01:34:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:45 smithi082 ceph-mon[98694]: pgmap v4493: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T01:34:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:45 smithi098 ceph-mon[103124]: pgmap v4493: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T01:34:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:47 smithi170 ceph-mon[102832]: pgmap v4494: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 100 KiB/s wr, 16 op/s 2023-09-10T01:34:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8118.660252 secs 2023-09-10T01:34:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:47 smithi082 ceph-mon[98694]: pgmap v4494: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 100 KiB/s wr, 16 op/s 2023-09-10T01:34:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8118.660252 secs 2023-09-10T01:34:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:47 smithi098 ceph-mon[103124]: pgmap v4494: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 100 KiB/s wr, 16 op/s 2023-09-10T01:34:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8118.660252 secs 2023-09-10T01:34:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8120.141994 secs 2023-09-10T01:34:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8120.853618 secs 2023-09-10T01:34:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8120.141994 secs 2023-09-10T01:34:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8120.853618 secs 2023-09-10T01:34:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8120.141994 secs 2023-09-10T01:34:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8120.853618 secs 2023-09-10T01:34:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:49 smithi170 ceph-mon[102832]: pgmap v4495: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T01:34:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:49 smithi082 ceph-mon[98694]: pgmap v4495: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T01:34:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:49 smithi098 ceph-mon[103124]: pgmap v4495: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T01:34:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:51 smithi170 ceph-mon[102832]: pgmap v4496: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 89 KiB/s wr, 16 op/s 2023-09-10T01:34:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:34:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:34:52.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:34:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:51 smithi082 ceph-mon[98694]: pgmap v4496: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 89 KiB/s wr, 16 op/s 2023-09-10T01:34:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:34:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:34:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:34:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:51 smithi098 ceph-mon[103124]: pgmap v4496: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 89 KiB/s wr, 16 op/s 2023-09-10T01:34:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:34:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:34:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:34:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8123.660316 secs 2023-09-10T01:34:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8123.660316 secs 2023-09-10T01:34:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8123.660316 secs 2023-09-10T01:34:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:53 smithi170 ceph-mon[102832]: pgmap v4497: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T01:34:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8125.142052 secs 2023-09-10T01:34:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8125.853681 secs 2023-09-10T01:34:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:53 smithi082 ceph-mon[98694]: pgmap v4497: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T01:34:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8125.142052 secs 2023-09-10T01:34:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8125.853681 secs 2023-09-10T01:34:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:53 smithi098 ceph-mon[103124]: pgmap v4497: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T01:34:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8125.142052 secs 2023-09-10T01:34:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8125.853681 secs 2023-09-10T01:34:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:55 smithi170 ceph-mon[102832]: pgmap v4498: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T01:34:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:55 smithi082 ceph-mon[98694]: pgmap v4498: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T01:34:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:55 smithi098 ceph-mon[103124]: pgmap v4498: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T01:34:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:57 smithi170 ceph-mon[102832]: pgmap v4499: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 16 op/s 2023-09-10T01:34:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8128.660373 secs 2023-09-10T01:34:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:57 smithi082 ceph-mon[98694]: pgmap v4499: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 16 op/s 2023-09-10T01:34:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8128.660373 secs 2023-09-10T01:34:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:57 smithi098 ceph-mon[103124]: pgmap v4499: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 16 op/s 2023-09-10T01:34:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8128.660373 secs 2023-09-10T01:34:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8130.142111 secs 2023-09-10T01:34:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8130.853746 secs 2023-09-10T01:34:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8130.142111 secs 2023-09-10T01:34:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8130.853746 secs 2023-09-10T01:34:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8130.142111 secs 2023-09-10T01:34:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8130.853746 secs 2023-09-10T01:35:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:34:59 smithi170 ceph-mon[102832]: pgmap v4500: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T01:35:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:34:59 smithi082 ceph-mon[98694]: pgmap v4500: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T01:35:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:34:59 smithi098 ceph-mon[103124]: pgmap v4500: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T01:35:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:01 smithi082 ceph-mon[98694]: pgmap v4501: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 16 op/s 2023-09-10T01:35:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:01 smithi098 ceph-mon[103124]: pgmap v4501: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 16 op/s 2023-09-10T01:35:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:01 smithi170 ceph-mon[102832]: pgmap v4501: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 16 op/s 2023-09-10T01:35:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8133.660434 secs 2023-09-10T01:35:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8133.660434 secs 2023-09-10T01:35:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8133.660434 secs 2023-09-10T01:35:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:03 smithi098 ceph-mon[103124]: pgmap v4502: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T01:35:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:03 smithi170 ceph-mon[102832]: pgmap v4502: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T01:35:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:03 smithi082 ceph-mon[98694]: pgmap v4502: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T01:35:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8135.142185 secs 2023-09-10T01:35:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8135.853811 secs 2023-09-10T01:35:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8135.142185 secs 2023-09-10T01:35:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8135.853811 secs 2023-09-10T01:35:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8135.142185 secs 2023-09-10T01:35:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8135.853811 secs 2023-09-10T01:35:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:05 smithi098 ceph-mon[103124]: pgmap v4503: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T01:35:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:05 smithi170 ceph-mon[102832]: pgmap v4503: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T01:35:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:05 smithi082 ceph-mon[98694]: pgmap v4503: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T01:35:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:07 smithi098 ceph-mon[103124]: pgmap v4504: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 93 KiB/s wr, 31 op/s 2023-09-10T01:35:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8138.660509 secs 2023-09-10T01:35:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:07 smithi170 ceph-mon[102832]: pgmap v4504: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 93 KiB/s wr, 31 op/s 2023-09-10T01:35:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8138.660509 secs 2023-09-10T01:35:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:07 smithi082 ceph-mon[98694]: pgmap v4504: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 93 KiB/s wr, 31 op/s 2023-09-10T01:35:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8138.660509 secs 2023-09-10T01:35:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:08 smithi098 ceph-mon[103124]: osdmap e351: 12 total, 12 up, 12 in 2023-09-10T01:35:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:08 smithi170 ceph-mon[102832]: osdmap e351: 12 total, 12 up, 12 in 2023-09-10T01:35:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:08 smithi082 ceph-mon[98694]: osdmap e351: 12 total, 12 up, 12 in 2023-09-10T01:35:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8140.142288 secs 2023-09-10T01:35:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8140.853876 secs 2023-09-10T01:35:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:09 smithi098 ceph-mon[103124]: pgmap v4506: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 94 KiB/s wr, 30 op/s 2023-09-10T01:35:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8140.142288 secs 2023-09-10T01:35:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8140.853876 secs 2023-09-10T01:35:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:09 smithi170 ceph-mon[102832]: pgmap v4506: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 94 KiB/s wr, 30 op/s 2023-09-10T01:35:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8140.142288 secs 2023-09-10T01:35:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8140.853876 secs 2023-09-10T01:35:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:09 smithi082 ceph-mon[98694]: pgmap v4506: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 94 KiB/s wr, 30 op/s 2023-09-10T01:35:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:11 smithi082 ceph-mon[98694]: pgmap v4507: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-09-10T01:35:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:11 smithi098 ceph-mon[103124]: pgmap v4507: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-09-10T01:35:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:11 smithi170 ceph-mon[102832]: pgmap v4507: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-09-10T01:35:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8143.660570 secs 2023-09-10T01:35:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8143.660570 secs 2023-09-10T01:35:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8143.660570 secs 2023-09-10T01:35:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:13 smithi098 ceph-mon[103124]: pgmap v4508: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-09-10T01:35:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:13 smithi170 ceph-mon[102832]: pgmap v4508: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-09-10T01:35:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:13 smithi082 ceph-mon[98694]: pgmap v4508: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-09-10T01:35:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8145.142375 secs 2023-09-10T01:35:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8145.853945 secs 2023-09-10T01:35:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8145.142375 secs 2023-09-10T01:35:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8145.853945 secs 2023-09-10T01:35:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8145.142375 secs 2023-09-10T01:35:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8145.853945 secs 2023-09-10T01:35:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:15 smithi098 ceph-mon[103124]: pgmap v4509: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-09-10T01:35:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:15 smithi098 ceph-mon[103124]: osdmap e352: 12 total, 12 up, 12 in 2023-09-10T01:35:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:15 smithi170 ceph-mon[102832]: pgmap v4509: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-09-10T01:35:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:15 smithi170 ceph-mon[102832]: osdmap e352: 12 total, 12 up, 12 in 2023-09-10T01:35:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:15 smithi082 ceph-mon[98694]: pgmap v4509: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-09-10T01:35:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:15 smithi082 ceph-mon[98694]: osdmap e352: 12 total, 12 up, 12 in 2023-09-10T01:35:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:17 smithi098 ceph-mon[103124]: pgmap v4511: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 29 KiB/s wr, 13 op/s 2023-09-10T01:35:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8148.660647 secs 2023-09-10T01:35:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:17 smithi170 ceph-mon[102832]: pgmap v4511: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 29 KiB/s wr, 13 op/s 2023-09-10T01:35:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8148.660647 secs 2023-09-10T01:35:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:17 smithi082 ceph-mon[98694]: pgmap v4511: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 29 KiB/s wr, 13 op/s 2023-09-10T01:35:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8148.660647 secs 2023-09-10T01:35:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8150.142434 secs 2023-09-10T01:35:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8150.142434 secs 2023-09-10T01:35:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8150.142434 secs 2023-09-10T01:35:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8150.854011 secs 2023-09-10T01:35:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:19 smithi098 ceph-mon[103124]: pgmap v4512: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-09-10T01:35:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8150.854011 secs 2023-09-10T01:35:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:19 smithi170 ceph-mon[102832]: pgmap v4512: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-09-10T01:35:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8150.854011 secs 2023-09-10T01:35:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:19 smithi082 ceph-mon[98694]: pgmap v4512: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-09-10T01:35:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:21 smithi098 ceph-mon[103124]: pgmap v4513: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T01:35:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:21 smithi170 ceph-mon[102832]: pgmap v4513: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T01:35:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:21 smithi082 ceph-mon[98694]: pgmap v4513: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T01:35:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:35:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:35:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:35:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:23 smithi098 ceph-mon[103124]: pgmap v4514: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T01:35:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:35:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:35:24.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:35:24.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8155.142494 secs 2023-09-10T01:35:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:23 smithi170 ceph-mon[102832]: pgmap v4514: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T01:35:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:35:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:35:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:35:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8155.142494 secs 2023-09-10T01:35:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:23 smithi082 ceph-mon[98694]: pgmap v4514: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T01:35:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:35:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:35:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:35:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8155.142494 secs 2023-09-10T01:35:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8155.854077 secs 2023-09-10T01:35:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8155.854077 secs 2023-09-10T01:35:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8155.854077 secs 2023-09-10T01:35:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:25 smithi098 ceph-mon[103124]: pgmap v4515: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T01:35:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:25 smithi170 ceph-mon[102832]: pgmap v4515: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T01:35:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:25 smithi082 ceph-mon[98694]: pgmap v4515: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T01:35:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:35:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:35:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:35:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:27 smithi098 ceph-mon[103124]: pgmap v4516: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T01:35:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:35:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:35:28.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:35:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:27 smithi170 ceph-mon[102832]: pgmap v4516: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T01:35:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:35:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:35:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:35:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:27 smithi082 ceph-mon[98694]: pgmap v4516: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T01:35:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:35:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:35:28.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:35:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8160.142552 secs 2023-09-10T01:35:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8160.854142 secs 2023-09-10T01:35:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8160.142552 secs 2023-09-10T01:35:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8160.854142 secs 2023-09-10T01:35:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8160.142552 secs 2023-09-10T01:35:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8160.854142 secs 2023-09-10T01:35:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:29 smithi170 ceph-mon[102832]: pgmap v4517: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 39 KiB/s wr, 10 op/s 2023-09-10T01:35:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:29 smithi082 ceph-mon[98694]: pgmap v4517: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 39 KiB/s wr, 10 op/s 2023-09-10T01:35:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:29 smithi098 ceph-mon[103124]: pgmap v4517: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 39 KiB/s wr, 10 op/s 2023-09-10T01:35:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:31 smithi170 ceph-mon[102832]: pgmap v4518: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 96 KiB/s wr, 17 op/s 2023-09-10T01:35:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:31 smithi082 ceph-mon[98694]: pgmap v4518: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 96 KiB/s wr, 17 op/s 2023-09-10T01:35:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:31 smithi098 ceph-mon[103124]: pgmap v4518: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 96 KiB/s wr, 17 op/s 2023-09-10T01:35:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:33 smithi170 ceph-mon[102832]: pgmap v4519: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-10T01:35:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8165.142614 secs 2023-09-10T01:35:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8165.854206 secs 2023-09-10T01:35:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:33 smithi082 ceph-mon[98694]: pgmap v4519: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-10T01:35:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8165.142614 secs 2023-09-10T01:35:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8165.854206 secs 2023-09-10T01:35:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:33 smithi098 ceph-mon[103124]: pgmap v4519: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-10T01:35:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8165.142614 secs 2023-09-10T01:35:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8165.854206 secs 2023-09-10T01:35:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:35 smithi170 ceph-mon[102832]: pgmap v4520: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-10T01:35:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:35 smithi082 ceph-mon[98694]: pgmap v4520: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-10T01:35:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:35 smithi098 ceph-mon[103124]: pgmap v4520: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-10T01:35:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:37 smithi170 ceph-mon[102832]: pgmap v4521: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 78 KiB/s wr, 17 op/s 2023-09-10T01:35:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:37 smithi082 ceph-mon[98694]: pgmap v4521: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 78 KiB/s wr, 17 op/s 2023-09-10T01:35:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:37 smithi098 ceph-mon[103124]: pgmap v4521: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 78 KiB/s wr, 17 op/s 2023-09-10T01:35:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8170.142669 secs 2023-09-10T01:35:39.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8170.854270 secs 2023-09-10T01:35:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8170.142669 secs 2023-09-10T01:35:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8170.854270 secs 2023-09-10T01:35:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8170.142669 secs 2023-09-10T01:35:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8170.854270 secs 2023-09-10T01:35:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:39 smithi170 ceph-mon[102832]: pgmap v4522: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-10T01:35:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:39 smithi082 ceph-mon[98694]: pgmap v4522: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-10T01:35:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:39 smithi098 ceph-mon[103124]: pgmap v4522: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-10T01:35:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:41 smithi170 ceph-mon[102832]: pgmap v4523: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 117 KiB/s wr, 17 op/s 2023-09-10T01:35:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:41 smithi082 ceph-mon[98694]: pgmap v4523: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 117 KiB/s wr, 17 op/s 2023-09-10T01:35:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:41 smithi098 ceph-mon[103124]: pgmap v4523: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 117 KiB/s wr, 17 op/s 2023-09-10T01:35:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:43 smithi170 ceph-mon[102832]: pgmap v4524: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T01:35:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8175.142728 secs 2023-09-10T01:35:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8175.854337 secs 2023-09-10T01:35:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:43 smithi082 ceph-mon[98694]: pgmap v4524: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T01:35:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8175.142728 secs 2023-09-10T01:35:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8175.854337 secs 2023-09-10T01:35:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:43 smithi098 ceph-mon[103124]: pgmap v4524: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T01:35:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8175.142728 secs 2023-09-10T01:35:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8175.854337 secs 2023-09-10T01:35:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:45 smithi170 ceph-mon[102832]: pgmap v4525: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T01:35:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:45 smithi082 ceph-mon[98694]: pgmap v4525: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T01:35:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:45 smithi098 ceph-mon[103124]: pgmap v4525: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T01:35:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:47 smithi170 ceph-mon[102832]: pgmap v4526: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 76 KiB/s wr, 16 op/s 2023-09-10T01:35:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8178.661067 secs 2023-09-10T01:35:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:47 smithi082 ceph-mon[98694]: pgmap v4526: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 76 KiB/s wr, 16 op/s 2023-09-10T01:35:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8178.661067 secs 2023-09-10T01:35:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:47 smithi098 ceph-mon[103124]: pgmap v4526: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 76 KiB/s wr, 16 op/s 2023-09-10T01:35:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8178.661067 secs 2023-09-10T01:35:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8180.142805 secs 2023-09-10T01:35:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8180.854402 secs 2023-09-10T01:35:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8180.142805 secs 2023-09-10T01:35:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8180.854402 secs 2023-09-10T01:35:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8180.142805 secs 2023-09-10T01:35:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8180.854402 secs 2023-09-10T01:35:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:49 smithi170 ceph-mon[102832]: pgmap v4527: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 10 op/s 2023-09-10T01:35:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:49 smithi082 ceph-mon[98694]: pgmap v4527: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 10 op/s 2023-09-10T01:35:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:49 smithi098 ceph-mon[103124]: pgmap v4527: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 10 op/s 2023-09-10T01:35:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:51 smithi098 ceph-mon[103124]: pgmap v4528: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 99 KiB/s wr, 16 op/s 2023-09-10T01:35:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:35:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:35:52.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:35:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:51 smithi170 ceph-mon[102832]: pgmap v4528: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 99 KiB/s wr, 16 op/s 2023-09-10T01:35:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:35:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:35:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:35:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:51 smithi082 ceph-mon[98694]: pgmap v4528: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 99 KiB/s wr, 16 op/s 2023-09-10T01:35:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:35:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:35:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:35:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8183.661154 secs 2023-09-10T01:35:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8183.661154 secs 2023-09-10T01:35:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8183.661154 secs 2023-09-10T01:35:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:53 smithi098 ceph-mon[103124]: pgmap v4529: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T01:35:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:53 smithi170 ceph-mon[102832]: pgmap v4529: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T01:35:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:53 smithi082 ceph-mon[98694]: pgmap v4529: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T01:35:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8185.142891 secs 2023-09-10T01:35:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8185.854466 secs 2023-09-10T01:35:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8185.142891 secs 2023-09-10T01:35:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8185.854466 secs 2023-09-10T01:35:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8185.142891 secs 2023-09-10T01:35:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8185.854466 secs 2023-09-10T01:35:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:55 smithi098 ceph-mon[103124]: pgmap v4530: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T01:35:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:55 smithi170 ceph-mon[102832]: pgmap v4530: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T01:35:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:55 smithi082 ceph-mon[98694]: pgmap v4530: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T01:35:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:57 smithi098 ceph-mon[103124]: pgmap v4531: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 57 KiB/s wr, 15 op/s 2023-09-10T01:35:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8188.661213 secs 2023-09-10T01:35:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:57 smithi170 ceph-mon[102832]: pgmap v4531: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 57 KiB/s wr, 15 op/s 2023-09-10T01:35:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8188.661213 secs 2023-09-10T01:35:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:57 smithi082 ceph-mon[98694]: pgmap v4531: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 57 KiB/s wr, 15 op/s 2023-09-10T01:35:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8188.661213 secs 2023-09-10T01:35:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8190.142951 secs 2023-09-10T01:35:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8190.142951 secs 2023-09-10T01:35:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8190.142951 secs 2023-09-10T01:36:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8190.854487 secs 2023-09-10T01:36:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:35:59 smithi098 ceph-mon[103124]: pgmap v4532: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 10 op/s 2023-09-10T01:36:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8190.854487 secs 2023-09-10T01:36:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:35:59 smithi170 ceph-mon[102832]: pgmap v4532: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 10 op/s 2023-09-10T01:36:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8190.854487 secs 2023-09-10T01:36:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:35:59 smithi082 ceph-mon[98694]: pgmap v4532: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 10 op/s 2023-09-10T01:36:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:01 smithi098 ceph-mon[103124]: pgmap v4533: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 96 KiB/s wr, 30 op/s 2023-09-10T01:36:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:01 smithi170 ceph-mon[102832]: pgmap v4533: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 96 KiB/s wr, 30 op/s 2023-09-10T01:36:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:01 smithi082 ceph-mon[98694]: pgmap v4533: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 96 KiB/s wr, 30 op/s 2023-09-10T01:36:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8193.661296 secs 2023-09-10T01:36:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8193.661296 secs 2023-09-10T01:36:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8193.661296 secs 2023-09-10T01:36:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:03 smithi098 ceph-mon[103124]: pgmap v4534: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 63 KiB/s wr, 24 op/s 2023-09-10T01:36:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8195.143009 secs 2023-09-10T01:36:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8195.854554 secs 2023-09-10T01:36:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:03 smithi170 ceph-mon[102832]: pgmap v4534: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 63 KiB/s wr, 24 op/s 2023-09-10T01:36:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8195.143009 secs 2023-09-10T01:36:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8195.854554 secs 2023-09-10T01:36:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:03 smithi082 ceph-mon[98694]: pgmap v4534: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 63 KiB/s wr, 24 op/s 2023-09-10T01:36:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8195.143009 secs 2023-09-10T01:36:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8195.854554 secs 2023-09-10T01:36:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:05 smithi098 ceph-mon[103124]: pgmap v4535: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 63 KiB/s wr, 24 op/s 2023-09-10T01:36:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:05 smithi170 ceph-mon[102832]: pgmap v4535: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 63 KiB/s wr, 24 op/s 2023-09-10T01:36:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:05 smithi082 ceph-mon[98694]: pgmap v4535: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 63 KiB/s wr, 24 op/s 2023-09-10T01:36:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:07 smithi170 ceph-mon[102832]: pgmap v4536: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-10T01:36:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8198.661358 secs 2023-09-10T01:36:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:07 smithi082 ceph-mon[98694]: pgmap v4536: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-10T01:36:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8198.661358 secs 2023-09-10T01:36:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:07 smithi098 ceph-mon[103124]: pgmap v4536: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 79 KiB/s wr, 29 op/s 2023-09-10T01:36:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8198.661358 secs 2023-09-10T01:36:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:08 smithi170 ceph-mon[102832]: osdmap e353: 12 total, 12 up, 12 in 2023-09-10T01:36:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8200.143067 secs 2023-09-10T01:36:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8200.854619 secs 2023-09-10T01:36:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:08 smithi082 ceph-mon[98694]: osdmap e353: 12 total, 12 up, 12 in 2023-09-10T01:36:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8200.143067 secs 2023-09-10T01:36:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8200.854619 secs 2023-09-10T01:36:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:08 smithi098 ceph-mon[103124]: osdmap e353: 12 total, 12 up, 12 in 2023-09-10T01:36:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8200.143067 secs 2023-09-10T01:36:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8200.854619 secs 2023-09-10T01:36:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:09 smithi170 ceph-mon[102832]: pgmap v4538: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T01:36:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:09 smithi082 ceph-mon[98694]: pgmap v4538: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T01:36:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:09 smithi098 ceph-mon[103124]: pgmap v4538: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T01:36:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:11 smithi170 ceph-mon[102832]: pgmap v4539: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T01:36:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:11 smithi082 ceph-mon[98694]: pgmap v4539: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T01:36:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:11 smithi098 ceph-mon[103124]: pgmap v4539: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T01:36:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8203.661429 secs 2023-09-10T01:36:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8203.661429 secs 2023-09-10T01:36:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8203.661429 secs 2023-09-10T01:36:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:13 smithi170 ceph-mon[102832]: pgmap v4540: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T01:36:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8205.143155 secs 2023-09-10T01:36:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8205.854687 secs 2023-09-10T01:36:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:13 smithi082 ceph-mon[98694]: pgmap v4540: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T01:36:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8205.143155 secs 2023-09-10T01:36:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8205.854687 secs 2023-09-10T01:36:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:13 smithi098 ceph-mon[103124]: pgmap v4540: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T01:36:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8205.143155 secs 2023-09-10T01:36:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8205.854687 secs 2023-09-10T01:36:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:15 smithi170 ceph-mon[102832]: pgmap v4541: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T01:36:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:15 smithi170 ceph-mon[102832]: osdmap e354: 12 total, 12 up, 12 in 2023-09-10T01:36:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:15 smithi082 ceph-mon[98694]: pgmap v4541: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T01:36:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:15 smithi082 ceph-mon[98694]: osdmap e354: 12 total, 12 up, 12 in 2023-09-10T01:36:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:15 smithi098 ceph-mon[103124]: pgmap v4541: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T01:36:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:15 smithi098 ceph-mon[103124]: osdmap e354: 12 total, 12 up, 12 in 2023-09-10T01:36:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:17 smithi170 ceph-mon[102832]: pgmap v4543: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 30 KiB/s wr, 13 op/s 2023-09-10T01:36:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8208.661523 secs 2023-09-10T01:36:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:17 smithi082 ceph-mon[98694]: pgmap v4543: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 30 KiB/s wr, 13 op/s 2023-09-10T01:36:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8208.661523 secs 2023-09-10T01:36:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:17 smithi098 ceph-mon[103124]: pgmap v4543: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 30 KiB/s wr, 13 op/s 2023-09-10T01:36:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8208.661523 secs 2023-09-10T01:36:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8210.143212 secs 2023-09-10T01:36:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8210.854751 secs 2023-09-10T01:36:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8210.143212 secs 2023-09-10T01:36:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8210.854751 secs 2023-09-10T01:36:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8210.143212 secs 2023-09-10T01:36:19.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8210.854751 secs 2023-09-10T01:36:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:19 smithi170 ceph-mon[102832]: pgmap v4544: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T01:36:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:19 smithi082 ceph-mon[98694]: pgmap v4544: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T01:36:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:19 smithi098 ceph-mon[103124]: pgmap v4544: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T01:36:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:21 smithi170 ceph-mon[102832]: pgmap v4545: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T01:36:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:21 smithi082 ceph-mon[98694]: pgmap v4545: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T01:36:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:21 smithi098 ceph-mon[103124]: pgmap v4545: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T01:36:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:36:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:36:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:36:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:23 smithi170 ceph-mon[102832]: pgmap v4546: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T01:36:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:36:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:36:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:36:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8215.143269 secs 2023-09-10T01:36:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8215.854817 secs 2023-09-10T01:36:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:23 smithi082 ceph-mon[98694]: pgmap v4546: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T01:36:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:36:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:36:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:36:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8215.143269 secs 2023-09-10T01:36:24.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8215.854817 secs 2023-09-10T01:36:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:23 smithi098 ceph-mon[103124]: pgmap v4546: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T01:36:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:36:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:36:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:36:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8215.143269 secs 2023-09-10T01:36:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8215.854817 secs 2023-09-10T01:36:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:25 smithi170 ceph-mon[102832]: pgmap v4547: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T01:36:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:25 smithi082 ceph-mon[98694]: pgmap v4547: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T01:36:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:25 smithi098 ceph-mon[103124]: pgmap v4547: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T01:36:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:27 smithi170 ceph-mon[102832]: pgmap v4548: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T01:36:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:36:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:27 smithi082 ceph-mon[98694]: pgmap v4548: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T01:36:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:36:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:27 smithi098 ceph-mon[103124]: pgmap v4548: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T01:36:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:36:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8220.143328 secs 2023-09-10T01:36:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:36:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:36:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:36:29.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8220.854883 secs 2023-09-10T01:36:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8220.143328 secs 2023-09-10T01:36:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:36:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:36:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:36:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8220.854883 secs 2023-09-10T01:36:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8220.143328 secs 2023-09-10T01:36:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:36:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:36:29.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:36:29.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8220.854883 secs 2023-09-10T01:36:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:29 smithi170 ceph-mon[102832]: pgmap v4549: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 39 KiB/s wr, 10 op/s 2023-09-10T01:36:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:29 smithi082 ceph-mon[98694]: pgmap v4549: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 39 KiB/s wr, 10 op/s 2023-09-10T01:36:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:29 smithi098 ceph-mon[103124]: pgmap v4549: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 39 KiB/s wr, 10 op/s 2023-09-10T01:36:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:31 smithi170 ceph-mon[102832]: pgmap v4550: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 98 KiB/s wr, 16 op/s 2023-09-10T01:36:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:31 smithi082 ceph-mon[98694]: pgmap v4550: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 98 KiB/s wr, 16 op/s 2023-09-10T01:36:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:31 smithi098 ceph-mon[103124]: pgmap v4550: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 98 KiB/s wr, 16 op/s 2023-09-10T01:36:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:33 smithi170 ceph-mon[102832]: pgmap v4551: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T01:36:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8225.143401 secs 2023-09-10T01:36:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8225.855002 secs 2023-09-10T01:36:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:33 smithi082 ceph-mon[98694]: pgmap v4551: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T01:36:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8225.143401 secs 2023-09-10T01:36:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8225.855002 secs 2023-09-10T01:36:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:33 smithi098 ceph-mon[103124]: pgmap v4551: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T01:36:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8225.143401 secs 2023-09-10T01:36:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8225.855002 secs 2023-09-10T01:36:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:35 smithi170 ceph-mon[102832]: pgmap v4552: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T01:36:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:35 smithi082 ceph-mon[98694]: pgmap v4552: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T01:36:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:35 smithi098 ceph-mon[103124]: pgmap v4552: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T01:36:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:37 smithi170 ceph-mon[102832]: pgmap v4553: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 79 KiB/s wr, 18 op/s 2023-09-10T01:36:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:37 smithi082 ceph-mon[98694]: pgmap v4553: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 79 KiB/s wr, 18 op/s 2023-09-10T01:36:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:37 smithi098 ceph-mon[103124]: pgmap v4553: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 79 KiB/s wr, 18 op/s 2023-09-10T01:36:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8230.143461 secs 2023-09-10T01:36:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8230.855069 secs 2023-09-10T01:36:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8230.143461 secs 2023-09-10T01:36:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8230.855069 secs 2023-09-10T01:36:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8230.143461 secs 2023-09-10T01:36:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8230.855069 secs 2023-09-10T01:36:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:39 smithi170 ceph-mon[102832]: pgmap v4554: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T01:36:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:39 smithi082 ceph-mon[98694]: pgmap v4554: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T01:36:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:39 smithi098 ceph-mon[103124]: pgmap v4554: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T01:36:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:41 smithi170 ceph-mon[102832]: pgmap v4555: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 119 KiB/s wr, 18 op/s 2023-09-10T01:36:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:41 smithi082 ceph-mon[98694]: pgmap v4555: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 119 KiB/s wr, 18 op/s 2023-09-10T01:36:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:41 smithi098 ceph-mon[103124]: pgmap v4555: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 119 KiB/s wr, 18 op/s 2023-09-10T01:36:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:43 smithi170 ceph-mon[102832]: pgmap v4556: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 12 op/s 2023-09-10T01:36:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8235.143508 secs 2023-09-10T01:36:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8235.855135 secs 2023-09-10T01:36:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:43 smithi082 ceph-mon[98694]: pgmap v4556: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 12 op/s 2023-09-10T01:36:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8235.143508 secs 2023-09-10T01:36:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8235.855135 secs 2023-09-10T01:36:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:43 smithi098 ceph-mon[103124]: pgmap v4556: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 12 op/s 2023-09-10T01:36:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8235.143508 secs 2023-09-10T01:36:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8235.855135 secs 2023-09-10T01:36:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:45 smithi170 ceph-mon[102832]: pgmap v4557: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 12 op/s 2023-09-10T01:36:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:45 smithi082 ceph-mon[98694]: pgmap v4557: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 12 op/s 2023-09-10T01:36:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:45 smithi098 ceph-mon[103124]: pgmap v4557: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 12 op/s 2023-09-10T01:36:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:47 smithi170 ceph-mon[102832]: pgmap v4558: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 77 KiB/s wr, 18 op/s 2023-09-10T01:36:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8238.661896 secs 2023-09-10T01:36:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:47 smithi082 ceph-mon[98694]: pgmap v4558: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 77 KiB/s wr, 18 op/s 2023-09-10T01:36:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8238.661896 secs 2023-09-10T01:36:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:47 smithi098 ceph-mon[103124]: pgmap v4558: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 77 KiB/s wr, 18 op/s 2023-09-10T01:36:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8238.661896 secs 2023-09-10T01:36:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8240.143565 secs 2023-09-10T01:36:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8240.855200 secs 2023-09-10T01:36:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8240.143565 secs 2023-09-10T01:36:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8240.855200 secs 2023-09-10T01:36:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8240.143565 secs 2023-09-10T01:36:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8240.855200 secs 2023-09-10T01:36:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:49 smithi170 ceph-mon[102832]: pgmap v4559: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T01:36:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:49 smithi082 ceph-mon[98694]: pgmap v4559: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T01:36:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:49 smithi098 ceph-mon[103124]: pgmap v4559: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T01:36:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:51 smithi098 ceph-mon[103124]: pgmap v4560: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 119 KiB/s wr, 17 op/s 2023-09-10T01:36:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:36:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:36:52.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:36:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:51 smithi170 ceph-mon[102832]: pgmap v4560: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 119 KiB/s wr, 17 op/s 2023-09-10T01:36:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:36:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:36:52.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:36:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:51 smithi082 ceph-mon[98694]: pgmap v4560: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 119 KiB/s wr, 17 op/s 2023-09-10T01:36:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:36:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:36:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:36:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8243.661995 secs 2023-09-10T01:36:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8243.661995 secs 2023-09-10T01:36:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8243.661995 secs 2023-09-10T01:36:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:53 smithi098 ceph-mon[103124]: pgmap v4561: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:36:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:53 smithi170 ceph-mon[102832]: pgmap v4561: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:36:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:53 smithi082 ceph-mon[98694]: pgmap v4561: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:36:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8245.143624 secs 2023-09-10T01:36:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8245.855266 secs 2023-09-10T01:36:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8245.143624 secs 2023-09-10T01:36:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8245.855266 secs 2023-09-10T01:36:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8245.143624 secs 2023-09-10T01:36:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8245.855266 secs 2023-09-10T01:36:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:55 smithi098 ceph-mon[103124]: pgmap v4562: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:36:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:55 smithi170 ceph-mon[102832]: pgmap v4562: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:36:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:55 smithi082 ceph-mon[98694]: pgmap v4562: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:36:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:57 smithi098 ceph-mon[103124]: pgmap v4563: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 76 KiB/s wr, 16 op/s 2023-09-10T01:36:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8248.662095 secs 2023-09-10T01:36:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:57 smithi170 ceph-mon[102832]: pgmap v4563: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 76 KiB/s wr, 16 op/s 2023-09-10T01:36:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8248.662095 secs 2023-09-10T01:36:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:57 smithi082 ceph-mon[98694]: pgmap v4563: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 76 KiB/s wr, 16 op/s 2023-09-10T01:36:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8248.662095 secs 2023-09-10T01:36:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8250.143683 secs 2023-09-10T01:36:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8250.143683 secs 2023-09-10T01:36:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8250.143683 secs 2023-09-10T01:37:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8250.855329 secs 2023-09-10T01:37:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:36:59 smithi098 ceph-mon[103124]: pgmap v4564: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T01:37:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8250.855329 secs 2023-09-10T01:37:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:36:59 smithi170 ceph-mon[102832]: pgmap v4564: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T01:37:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8250.855329 secs 2023-09-10T01:37:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:36:59 smithi082 ceph-mon[98694]: pgmap v4564: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T01:37:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:01 smithi170 ceph-mon[102832]: pgmap v4565: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 115 KiB/s wr, 30 op/s 2023-09-10T01:37:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:01 smithi082 ceph-mon[98694]: pgmap v4565: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 115 KiB/s wr, 30 op/s 2023-09-10T01:37:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:01 smithi098 ceph-mon[103124]: pgmap v4565: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 115 KiB/s wr, 30 op/s 2023-09-10T01:37:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8253.662179 secs 2023-09-10T01:37:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8253.662179 secs 2023-09-10T01:37:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8253.662179 secs 2023-09-10T01:37:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:03 smithi170 ceph-mon[102832]: pgmap v4566: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 63 KiB/s wr, 24 op/s 2023-09-10T01:37:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8255.143741 secs 2023-09-10T01:37:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8255.855393 secs 2023-09-10T01:37:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:03 smithi082 ceph-mon[98694]: pgmap v4566: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 63 KiB/s wr, 24 op/s 2023-09-10T01:37:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8255.143741 secs 2023-09-10T01:37:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8255.855393 secs 2023-09-10T01:37:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:03 smithi098 ceph-mon[103124]: pgmap v4566: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 63 KiB/s wr, 24 op/s 2023-09-10T01:37:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8255.143741 secs 2023-09-10T01:37:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8255.855393 secs 2023-09-10T01:37:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:05 smithi170 ceph-mon[102832]: pgmap v4567: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 63 KiB/s wr, 24 op/s 2023-09-10T01:37:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:05 smithi082 ceph-mon[98694]: pgmap v4567: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 63 KiB/s wr, 24 op/s 2023-09-10T01:37:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:05 smithi098 ceph-mon[103124]: pgmap v4567: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 63 KiB/s wr, 24 op/s 2023-09-10T01:37:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:07 smithi170 ceph-mon[102832]: pgmap v4568: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 80 KiB/s wr, 29 op/s 2023-09-10T01:37:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8258.662240 secs 2023-09-10T01:37:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:07 smithi082 ceph-mon[98694]: pgmap v4568: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 80 KiB/s wr, 29 op/s 2023-09-10T01:37:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8258.662240 secs 2023-09-10T01:37:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:07 smithi098 ceph-mon[103124]: pgmap v4568: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 80 KiB/s wr, 29 op/s 2023-09-10T01:37:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8258.662240 secs 2023-09-10T01:37:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:08 smithi170 ceph-mon[102832]: osdmap e355: 12 total, 12 up, 12 in 2023-09-10T01:37:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8260.143799 secs 2023-09-10T01:37:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8260.855460 secs 2023-09-10T01:37:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:08 smithi082 ceph-mon[98694]: osdmap e355: 12 total, 12 up, 12 in 2023-09-10T01:37:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8260.143799 secs 2023-09-10T01:37:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8260.855460 secs 2023-09-10T01:37:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:08 smithi098 ceph-mon[103124]: osdmap e355: 12 total, 12 up, 12 in 2023-09-10T01:37:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8260.143799 secs 2023-09-10T01:37:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8260.855460 secs 2023-09-10T01:37:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:09 smithi170 ceph-mon[102832]: pgmap v4570: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:37:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:09 smithi082 ceph-mon[98694]: pgmap v4570: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:37:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:09 smithi098 ceph-mon[103124]: pgmap v4570: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:37:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:11 smithi170 ceph-mon[102832]: pgmap v4571: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T01:37:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:11 smithi082 ceph-mon[98694]: pgmap v4571: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T01:37:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:11 smithi098 ceph-mon[103124]: pgmap v4571: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T01:37:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8263.662301 secs 2023-09-10T01:37:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8263.662301 secs 2023-09-10T01:37:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8263.662301 secs 2023-09-10T01:37:14.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:13 smithi170 ceph-mon[102832]: pgmap v4572: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T01:37:14.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8265.143858 secs 2023-09-10T01:37:14.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8265.855486 secs 2023-09-10T01:37:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:13 smithi082 ceph-mon[98694]: pgmap v4572: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T01:37:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8265.143858 secs 2023-09-10T01:37:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8265.855486 secs 2023-09-10T01:37:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:13 smithi098 ceph-mon[103124]: pgmap v4572: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T01:37:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8265.143858 secs 2023-09-10T01:37:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8265.855486 secs 2023-09-10T01:37:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:15 smithi170 ceph-mon[102832]: pgmap v4573: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T01:37:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:15 smithi170 ceph-mon[102832]: osdmap e356: 12 total, 12 up, 12 in 2023-09-10T01:37:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:15 smithi082 ceph-mon[98694]: pgmap v4573: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T01:37:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:15 smithi082 ceph-mon[98694]: osdmap e356: 12 total, 12 up, 12 in 2023-09-10T01:37:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:15 smithi098 ceph-mon[103124]: pgmap v4573: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T01:37:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:15 smithi098 ceph-mon[103124]: osdmap e356: 12 total, 12 up, 12 in 2023-09-10T01:37:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:17 smithi170 ceph-mon[102832]: pgmap v4575: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 30 KiB/s wr, 13 op/s 2023-09-10T01:37:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8268.662358 secs 2023-09-10T01:37:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:17 smithi082 ceph-mon[98694]: pgmap v4575: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 30 KiB/s wr, 13 op/s 2023-09-10T01:37:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8268.662358 secs 2023-09-10T01:37:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:17 smithi098 ceph-mon[103124]: pgmap v4575: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 30 KiB/s wr, 13 op/s 2023-09-10T01:37:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8268.662358 secs 2023-09-10T01:37:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8270.143915 secs 2023-09-10T01:37:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8270.855552 secs 2023-09-10T01:37:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8270.143915 secs 2023-09-10T01:37:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8270.855552 secs 2023-09-10T01:37:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8270.143915 secs 2023-09-10T01:37:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8270.855552 secs 2023-09-10T01:37:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:19 smithi170 ceph-mon[102832]: pgmap v4576: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T01:37:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:19 smithi082 ceph-mon[98694]: pgmap v4576: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T01:37:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:19 smithi098 ceph-mon[103124]: pgmap v4576: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T01:37:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:21 smithi170 ceph-mon[102832]: pgmap v4577: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:37:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:21 smithi082 ceph-mon[98694]: pgmap v4577: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:37:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:21 smithi098 ceph-mon[103124]: pgmap v4577: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:37:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:37:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:37:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:37:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:23 smithi170 ceph-mon[102832]: pgmap v4578: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:37:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:37:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:37:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:37:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8275.143975 secs 2023-09-10T01:37:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8275.855617 secs 2023-09-10T01:37:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:23 smithi082 ceph-mon[98694]: pgmap v4578: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:37:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:37:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:37:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:37:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8275.143975 secs 2023-09-10T01:37:24.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8275.855617 secs 2023-09-10T01:37:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:23 smithi098 ceph-mon[103124]: pgmap v4578: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:37:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:37:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:37:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:37:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8275.143975 secs 2023-09-10T01:37:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8275.855617 secs 2023-09-10T01:37:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:25 smithi170 ceph-mon[102832]: pgmap v4579: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:37:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:25 smithi082 ceph-mon[98694]: pgmap v4579: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:37:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:25 smithi098 ceph-mon[103124]: pgmap v4579: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:37:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:27 smithi170 ceph-mon[102832]: pgmap v4580: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T01:37:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:27 smithi082 ceph-mon[98694]: pgmap v4580: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T01:37:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:27 smithi098 ceph-mon[103124]: pgmap v4580: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T01:37:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8280.144033 secs 2023-09-10T01:37:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8280.855682 secs 2023-09-10T01:37:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:37:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8280.144033 secs 2023-09-10T01:37:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8280.855682 secs 2023-09-10T01:37:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:37:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8280.144033 secs 2023-09-10T01:37:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8280.855682 secs 2023-09-10T01:37:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:37:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:29 smithi170 ceph-mon[102832]: pgmap v4581: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T01:37:30.189 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:29 smithi098 ceph-mon[103124]: pgmap v4581: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T01:37:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:29 smithi082 ceph-mon[98694]: pgmap v4581: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 40 KiB/s wr, 10 op/s 2023-09-10T01:37:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:31 smithi170 ceph-mon[102832]: pgmap v4582: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 100 KiB/s wr, 17 op/s 2023-09-10T01:37:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:37:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:37:32.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:37:32.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:37:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:31 smithi082 ceph-mon[98694]: pgmap v4582: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 100 KiB/s wr, 17 op/s 2023-09-10T01:37:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:37:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:37:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:37:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:37:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:31 smithi098 ceph-mon[103124]: pgmap v4582: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 100 KiB/s wr, 17 op/s 2023-09-10T01:37:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:37:32.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:37:32.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:37:32.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:37:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:33.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:33.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:32 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T01:37:33.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:32 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:37:33.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:33.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:33.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:33.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:33.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:37:33.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:37:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:33.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:32 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T01:37:33.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:32 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:37:33.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:33.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:33.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:33.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:33.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:37:33.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:37:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:33.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:33.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:33.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:32 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T01:37:33.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:32 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:37:33.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:33.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:33.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:33.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:33.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:37:33.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:37:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:33 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T01:37:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:33 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T01:37:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:33 smithi170 ceph-mon[102832]: pgmap v4583: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T01:37:34.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:34.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:34.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:34.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:34.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:37:34.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:37:34.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:37:34.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8285.144094 secs 2023-09-10T01:37:34.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8285.855745 secs 2023-09-10T01:37:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:33 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T01:37:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:33 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T01:37:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:33 smithi082 ceph-mon[98694]: pgmap v4583: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T01:37:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:34.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:34.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:34.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:37:34.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:37:34.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:37:34.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8285.144094 secs 2023-09-10T01:37:34.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8285.855745 secs 2023-09-10T01:37:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:33 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T01:37:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:33 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T01:37:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:33 smithi098 ceph-mon[103124]: pgmap v4583: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T01:37:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:34.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:37:34.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:37:34.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:37:34.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:37:34.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8285.144094 secs 2023-09-10T01:37:34.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8285.855745 secs 2023-09-10T01:37:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:34 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T01:37:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:34 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T01:37:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:34 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T01:37:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:34 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T01:37:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:34 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T01:37:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:34 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T01:37:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:35 smithi170 ceph-mon[102832]: pgmap v4584: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T01:37:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:35 smithi082 ceph-mon[98694]: pgmap v4584: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T01:37:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:35 smithi098 ceph-mon[103124]: pgmap v4584: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T01:37:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:37 smithi170 ceph-mon[102832]: pgmap v4585: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 81 KiB/s wr, 17 op/s 2023-09-10T01:37:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:37 smithi082 ceph-mon[98694]: pgmap v4585: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 81 KiB/s wr, 17 op/s 2023-09-10T01:37:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:37 smithi098 ceph-mon[103124]: pgmap v4585: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 81 KiB/s wr, 17 op/s 2023-09-10T01:37:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8290.144196 secs 2023-09-10T01:37:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8290.855834 secs 2023-09-10T01:37:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8290.144196 secs 2023-09-10T01:37:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8290.855834 secs 2023-09-10T01:37:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8290.144196 secs 2023-09-10T01:37:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8290.855834 secs 2023-09-10T01:37:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:39 smithi170 ceph-mon[102832]: pgmap v4586: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T01:37:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:39 smithi082 ceph-mon[98694]: pgmap v4586: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T01:37:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:39 smithi098 ceph-mon[103124]: pgmap v4586: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T01:37:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:41 smithi170 ceph-mon[102832]: pgmap v4587: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 130 KiB/s wr, 17 op/s 2023-09-10T01:37:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:41 smithi082 ceph-mon[98694]: pgmap v4587: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 130 KiB/s wr, 17 op/s 2023-09-10T01:37:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:41 smithi098 ceph-mon[103124]: pgmap v4587: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 130 KiB/s wr, 17 op/s 2023-09-10T01:37:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:43 smithi170 ceph-mon[102832]: pgmap v4588: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T01:37:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8295.144284 secs 2023-09-10T01:37:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8295.855905 secs 2023-09-10T01:37:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:43 smithi082 ceph-mon[98694]: pgmap v4588: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T01:37:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8295.144284 secs 2023-09-10T01:37:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8295.855905 secs 2023-09-10T01:37:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:43 smithi098 ceph-mon[103124]: pgmap v4588: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T01:37:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8295.144284 secs 2023-09-10T01:37:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8295.855905 secs 2023-09-10T01:37:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:45 smithi170 ceph-mon[102832]: pgmap v4589: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T01:37:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:45 smithi082 ceph-mon[98694]: pgmap v4589: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T01:37:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:45 smithi098 ceph-mon[103124]: pgmap v4589: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T01:37:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:47 smithi170 ceph-mon[102832]: pgmap v4590: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 80 KiB/s wr, 16 op/s 2023-09-10T01:37:48.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8298.714947 secs 2023-09-10T01:37:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:47 smithi082 ceph-mon[98694]: pgmap v4590: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 80 KiB/s wr, 16 op/s 2023-09-10T01:37:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8298.714947 secs 2023-09-10T01:37:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:47 smithi098 ceph-mon[103124]: pgmap v4590: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 80 KiB/s wr, 16 op/s 2023-09-10T01:37:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8298.714947 secs 2023-09-10T01:37:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8300.144369 secs 2023-09-10T01:37:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8300.855972 secs 2023-09-10T01:37:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8300.144369 secs 2023-09-10T01:37:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8300.855972 secs 2023-09-10T01:37:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8300.144369 secs 2023-09-10T01:37:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8300.855972 secs 2023-09-10T01:37:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:49 smithi170 ceph-mon[102832]: pgmap v4591: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T01:37:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:49 smithi082 ceph-mon[98694]: pgmap v4591: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T01:37:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:49 smithi098 ceph-mon[103124]: pgmap v4591: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T01:37:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:51 smithi098 ceph-mon[103124]: pgmap v4592: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 105 KiB/s wr, 16 op/s 2023-09-10T01:37:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:37:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:37:52.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:37:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:51 smithi170 ceph-mon[102832]: pgmap v4592: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 105 KiB/s wr, 16 op/s 2023-09-10T01:37:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:37:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:37:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:37:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:51 smithi082 ceph-mon[98694]: pgmap v4592: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 105 KiB/s wr, 16 op/s 2023-09-10T01:37:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:37:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:37:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:37:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8303.715006 secs 2023-09-10T01:37:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8303.715006 secs 2023-09-10T01:37:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8303.715006 secs 2023-09-10T01:37:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:53 smithi170 ceph-mon[102832]: pgmap v4593: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 45 KiB/s wr, 10 op/s 2023-09-10T01:37:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8305.144426 secs 2023-09-10T01:37:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:53 smithi082 ceph-mon[98694]: pgmap v4593: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 45 KiB/s wr, 10 op/s 2023-09-10T01:37:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8305.144426 secs 2023-09-10T01:37:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:53 smithi098 ceph-mon[103124]: pgmap v4593: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 45 KiB/s wr, 10 op/s 2023-09-10T01:37:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8305.144426 secs 2023-09-10T01:37:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8305.856040 secs 2023-09-10T01:37:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8305.856040 secs 2023-09-10T01:37:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8305.856040 secs 2023-09-10T01:37:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:55 smithi170 ceph-mon[102832]: pgmap v4594: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 45 KiB/s wr, 10 op/s 2023-09-10T01:37:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:55 smithi082 ceph-mon[98694]: pgmap v4594: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 45 KiB/s wr, 10 op/s 2023-09-10T01:37:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:55 smithi098 ceph-mon[103124]: pgmap v4594: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 45 KiB/s wr, 10 op/s 2023-09-10T01:37:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:57 smithi170 ceph-mon[102832]: pgmap v4595: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 52 KiB/s wr, 16 op/s 2023-09-10T01:37:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8308.715065 secs 2023-09-10T01:37:58.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:57 smithi082 ceph-mon[98694]: pgmap v4595: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 52 KiB/s wr, 16 op/s 2023-09-10T01:37:58.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8308.715065 secs 2023-09-10T01:37:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:57 smithi098 ceph-mon[103124]: pgmap v4595: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 52 KiB/s wr, 16 op/s 2023-09-10T01:37:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8308.715065 secs 2023-09-10T01:37:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8310.144511 secs 2023-09-10T01:37:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8310.856107 secs 2023-09-10T01:37:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8310.144511 secs 2023-09-10T01:37:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8310.856107 secs 2023-09-10T01:37:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8310.144511 secs 2023-09-10T01:37:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8310.856107 secs 2023-09-10T01:38:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:37:59 smithi170 ceph-mon[102832]: pgmap v4596: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-10T01:38:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:37:59 smithi082 ceph-mon[98694]: pgmap v4596: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-10T01:38:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:37:59 smithi098 ceph-mon[103124]: pgmap v4596: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-10T01:38:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:01 smithi170 ceph-mon[102832]: pgmap v4597: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 82 KiB/s wr, 22 op/s 2023-09-10T01:38:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:01 smithi082 ceph-mon[98694]: pgmap v4597: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 82 KiB/s wr, 22 op/s 2023-09-10T01:38:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:01 smithi098 ceph-mon[103124]: pgmap v4597: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 82 KiB/s wr, 22 op/s 2023-09-10T01:38:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8313.715123 secs 2023-09-10T01:38:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8313.715123 secs 2023-09-10T01:38:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8313.715123 secs 2023-09-10T01:38:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:03 smithi170 ceph-mon[102832]: pgmap v4598: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 47 KiB/s wr, 17 op/s 2023-09-10T01:38:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8315.144601 secs 2023-09-10T01:38:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8315.856197 secs 2023-09-10T01:38:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:03 smithi082 ceph-mon[98694]: pgmap v4598: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 47 KiB/s wr, 17 op/s 2023-09-10T01:38:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8315.144601 secs 2023-09-10T01:38:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8315.856197 secs 2023-09-10T01:38:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:03 smithi098 ceph-mon[103124]: pgmap v4598: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 47 KiB/s wr, 17 op/s 2023-09-10T01:38:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8315.144601 secs 2023-09-10T01:38:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8315.856197 secs 2023-09-10T01:38:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:05 smithi170 ceph-mon[102832]: pgmap v4599: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 47 KiB/s wr, 17 op/s 2023-09-10T01:38:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:05 smithi082 ceph-mon[98694]: pgmap v4599: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 47 KiB/s wr, 17 op/s 2023-09-10T01:38:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:05 smithi098 ceph-mon[103124]: pgmap v4599: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 47 KiB/s wr, 17 op/s 2023-09-10T01:38:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:07 smithi170 ceph-mon[102832]: pgmap v4600: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 70 KiB/s wr, 31 op/s 2023-09-10T01:38:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8318.715184 secs 2023-09-10T01:38:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:07 smithi082 ceph-mon[98694]: pgmap v4600: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 70 KiB/s wr, 31 op/s 2023-09-10T01:38:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8318.715184 secs 2023-09-10T01:38:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:07 smithi098 ceph-mon[103124]: pgmap v4600: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 70 KiB/s wr, 31 op/s 2023-09-10T01:38:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8318.715184 secs 2023-09-10T01:38:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:08 smithi170 ceph-mon[102832]: osdmap e357: 12 total, 12 up, 12 in 2023-09-10T01:38:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8320.144659 secs 2023-09-10T01:38:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8320.856262 secs 2023-09-10T01:38:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:08 smithi082 ceph-mon[98694]: osdmap e357: 12 total, 12 up, 12 in 2023-09-10T01:38:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8320.144659 secs 2023-09-10T01:38:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8320.856262 secs 2023-09-10T01:38:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:08 smithi098 ceph-mon[103124]: osdmap e357: 12 total, 12 up, 12 in 2023-09-10T01:38:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8320.144659 secs 2023-09-10T01:38:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8320.856262 secs 2023-09-10T01:38:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:09 smithi170 ceph-mon[102832]: pgmap v4602: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 31 op/s 2023-09-10T01:38:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:09 smithi082 ceph-mon[98694]: pgmap v4602: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 31 op/s 2023-09-10T01:38:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:09 smithi098 ceph-mon[103124]: pgmap v4602: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 31 op/s 2023-09-10T01:38:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:11 smithi170 ceph-mon[102832]: pgmap v4603: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 59 KiB/s wr, 23 op/s 2023-09-10T01:38:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:11 smithi082 ceph-mon[98694]: pgmap v4603: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 59 KiB/s wr, 23 op/s 2023-09-10T01:38:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:11 smithi098 ceph-mon[103124]: pgmap v4603: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 59 KiB/s wr, 23 op/s 2023-09-10T01:38:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8323.715291 secs 2023-09-10T01:38:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8323.715291 secs 2023-09-10T01:38:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8323.715291 secs 2023-09-10T01:38:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:13 smithi170 ceph-mon[102832]: pgmap v4604: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 59 KiB/s wr, 23 op/s 2023-09-10T01:38:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8325.144721 secs 2023-09-10T01:38:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8325.856326 secs 2023-09-10T01:38:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:13 smithi082 ceph-mon[98694]: pgmap v4604: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 59 KiB/s wr, 23 op/s 2023-09-10T01:38:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8325.144721 secs 2023-09-10T01:38:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8325.856326 secs 2023-09-10T01:38:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:13 smithi098 ceph-mon[103124]: pgmap v4604: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 59 KiB/s wr, 23 op/s 2023-09-10T01:38:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8325.144721 secs 2023-09-10T01:38:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8325.856326 secs 2023-09-10T01:38:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:15 smithi170 ceph-mon[102832]: pgmap v4605: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 59 KiB/s wr, 23 op/s 2023-09-10T01:38:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:15 smithi170 ceph-mon[102832]: osdmap e358: 12 total, 12 up, 12 in 2023-09-10T01:38:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:15 smithi082 ceph-mon[98694]: pgmap v4605: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 59 KiB/s wr, 23 op/s 2023-09-10T01:38:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:15 smithi082 ceph-mon[98694]: osdmap e358: 12 total, 12 up, 12 in 2023-09-10T01:38:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:15 smithi098 ceph-mon[103124]: pgmap v4605: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 59 KiB/s wr, 23 op/s 2023-09-10T01:38:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:15 smithi098 ceph-mon[103124]: osdmap e358: 12 total, 12 up, 12 in 2023-09-10T01:38:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:17 smithi170 ceph-mon[102832]: pgmap v4607: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 14 op/s 2023-09-10T01:38:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8328.715349 secs 2023-09-10T01:38:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:17 smithi082 ceph-mon[98694]: pgmap v4607: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 14 op/s 2023-09-10T01:38:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8328.715349 secs 2023-09-10T01:38:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:17 smithi098 ceph-mon[103124]: pgmap v4607: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 14 op/s 2023-09-10T01:38:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8328.715349 secs 2023-09-10T01:38:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8330.144784 secs 2023-09-10T01:38:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8330.856394 secs 2023-09-10T01:38:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8330.144784 secs 2023-09-10T01:38:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8330.856394 secs 2023-09-10T01:38:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8330.144784 secs 2023-09-10T01:38:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8330.856394 secs 2023-09-10T01:38:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:19 smithi170 ceph-mon[102832]: pgmap v4608: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T01:38:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:19 smithi082 ceph-mon[98694]: pgmap v4608: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T01:38:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:19 smithi098 ceph-mon[103124]: pgmap v4608: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T01:38:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:21 smithi170 ceph-mon[102832]: pgmap v4609: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-10T01:38:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:21 smithi082 ceph-mon[98694]: pgmap v4609: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-10T01:38:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:21 smithi098 ceph-mon[103124]: pgmap v4609: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-10T01:38:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:38:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:38:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:38:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:23 smithi170 ceph-mon[102832]: pgmap v4610: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-10T01:38:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:38:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:38:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:38:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8335.144857 secs 2023-09-10T01:38:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8335.856462 secs 2023-09-10T01:38:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:23 smithi082 ceph-mon[98694]: pgmap v4610: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-10T01:38:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:38:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:38:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:38:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8335.144857 secs 2023-09-10T01:38:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8335.856462 secs 2023-09-10T01:38:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:23 smithi098 ceph-mon[103124]: pgmap v4610: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-10T01:38:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:38:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:38:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:38:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8335.144857 secs 2023-09-10T01:38:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8335.856462 secs 2023-09-10T01:38:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:25 smithi170 ceph-mon[102832]: pgmap v4611: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-10T01:38:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:25 smithi082 ceph-mon[98694]: pgmap v4611: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-10T01:38:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:25 smithi098 ceph-mon[103124]: pgmap v4611: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-10T01:38:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:27 smithi170 ceph-mon[102832]: pgmap v4612: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T01:38:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:27 smithi082 ceph-mon[98694]: pgmap v4612: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T01:38:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:27 smithi098 ceph-mon[103124]: pgmap v4612: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T01:38:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8340.144941 secs 2023-09-10T01:38:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8340.856487 secs 2023-09-10T01:38:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8340.144941 secs 2023-09-10T01:38:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8340.856487 secs 2023-09-10T01:38:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8340.144941 secs 2023-09-10T01:38:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8340.856487 secs 2023-09-10T01:38:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:29 smithi170 ceph-mon[102832]: pgmap v4613: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-10T01:38:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:29 smithi082 ceph-mon[98694]: pgmap v4613: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-10T01:38:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:29 smithi098 ceph-mon[103124]: pgmap v4613: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-10T01:38:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:31 smithi170 ceph-mon[102832]: pgmap v4614: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 86 KiB/s wr, 16 op/s 2023-09-10T01:38:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:31 smithi082 ceph-mon[98694]: pgmap v4614: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 86 KiB/s wr, 16 op/s 2023-09-10T01:38:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:31 smithi098 ceph-mon[103124]: pgmap v4614: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 86 KiB/s wr, 16 op/s 2023-09-10T01:38:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:33 smithi170 ceph-mon[102832]: pgmap v4615: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T01:38:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8345.145029 secs 2023-09-10T01:38:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:38:34.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8345.856553 secs 2023-09-10T01:38:34.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:38:34.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:38:34.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:38:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:33 smithi082 ceph-mon[98694]: pgmap v4615: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T01:38:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8345.145029 secs 2023-09-10T01:38:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:38:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8345.856553 secs 2023-09-10T01:38:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:38:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:38:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:38:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:33 smithi098 ceph-mon[103124]: pgmap v4615: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T01:38:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8345.145029 secs 2023-09-10T01:38:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:38:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8345.856553 secs 2023-09-10T01:38:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:38:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:38:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:38:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:35 smithi170 ceph-mon[102832]: pgmap v4616: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T01:38:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:35 smithi082 ceph-mon[98694]: pgmap v4616: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T01:38:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:35 smithi098 ceph-mon[103124]: pgmap v4616: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T01:38:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:37 smithi170 ceph-mon[102832]: pgmap v4617: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 112 KiB/s wr, 17 op/s 2023-09-10T01:38:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:37 smithi082 ceph-mon[98694]: pgmap v4617: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 112 KiB/s wr, 17 op/s 2023-09-10T01:38:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:37 smithi098 ceph-mon[103124]: pgmap v4617: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 112 KiB/s wr, 17 op/s 2023-09-10T01:38:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8350.145088 secs 2023-09-10T01:38:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8350.856619 secs 2023-09-10T01:38:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8350.145088 secs 2023-09-10T01:38:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8350.856619 secs 2023-09-10T01:38:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8350.145088 secs 2023-09-10T01:38:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8350.856619 secs 2023-09-10T01:38:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:39 smithi170 ceph-mon[102832]: pgmap v4618: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T01:38:40.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:39 smithi082 ceph-mon[98694]: pgmap v4618: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T01:38:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:39 smithi098 ceph-mon[103124]: pgmap v4618: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T01:38:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:41 smithi170 ceph-mon[102832]: pgmap v4619: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 97 KiB/s wr, 17 op/s 2023-09-10T01:38:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:41 smithi082 ceph-mon[98694]: pgmap v4619: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 97 KiB/s wr, 17 op/s 2023-09-10T01:38:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:41 smithi098 ceph-mon[103124]: pgmap v4619: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 97 KiB/s wr, 17 op/s 2023-09-10T01:38:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:43 smithi170 ceph-mon[102832]: pgmap v4620: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T01:38:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8355.145147 secs 2023-09-10T01:38:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8355.856686 secs 2023-09-10T01:38:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:43 smithi082 ceph-mon[98694]: pgmap v4620: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T01:38:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8355.145147 secs 2023-09-10T01:38:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8355.856686 secs 2023-09-10T01:38:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:43 smithi098 ceph-mon[103124]: pgmap v4620: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T01:38:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8355.145147 secs 2023-09-10T01:38:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8355.856686 secs 2023-09-10T01:38:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:45 smithi170 ceph-mon[102832]: pgmap v4621: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T01:38:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:45 smithi082 ceph-mon[98694]: pgmap v4621: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T01:38:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:45 smithi098 ceph-mon[103124]: pgmap v4621: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T01:38:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:47 smithi170 ceph-mon[102832]: pgmap v4622: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 109 KiB/s wr, 17 op/s 2023-09-10T01:38:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8358.715775 secs 2023-09-10T01:38:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:47 smithi082 ceph-mon[98694]: pgmap v4622: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 109 KiB/s wr, 17 op/s 2023-09-10T01:38:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8358.715775 secs 2023-09-10T01:38:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:47 smithi098 ceph-mon[103124]: pgmap v4622: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 109 KiB/s wr, 17 op/s 2023-09-10T01:38:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8358.715775 secs 2023-09-10T01:38:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8360.145206 secs 2023-09-10T01:38:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8360.856751 secs 2023-09-10T01:38:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8360.145206 secs 2023-09-10T01:38:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8360.856751 secs 2023-09-10T01:38:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8360.145206 secs 2023-09-10T01:38:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8360.856751 secs 2023-09-10T01:38:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:49 smithi170 ceph-mon[102832]: pgmap v4623: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T01:38:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:49 smithi082 ceph-mon[98694]: pgmap v4623: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T01:38:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:49 smithi098 ceph-mon[103124]: pgmap v4623: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T01:38:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:51 smithi170 ceph-mon[102832]: pgmap v4624: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 83 KiB/s wr, 16 op/s 2023-09-10T01:38:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:38:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:38:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:38:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:51 smithi082 ceph-mon[98694]: pgmap v4624: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 83 KiB/s wr, 16 op/s 2023-09-10T01:38:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:38:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:38:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:38:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:51 smithi098 ceph-mon[103124]: pgmap v4624: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 83 KiB/s wr, 16 op/s 2023-09-10T01:38:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:38:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:38:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:38:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8363.715837 secs 2023-09-10T01:38:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8363.715837 secs 2023-09-10T01:38:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8363.715837 secs 2023-09-10T01:38:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:53 smithi170 ceph-mon[102832]: pgmap v4625: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T01:38:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8365.145263 secs 2023-09-10T01:38:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8365.856817 secs 2023-09-10T01:38:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:53 smithi082 ceph-mon[98694]: pgmap v4625: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T01:38:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8365.145263 secs 2023-09-10T01:38:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8365.856817 secs 2023-09-10T01:38:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:53 smithi098 ceph-mon[103124]: pgmap v4625: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T01:38:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8365.145263 secs 2023-09-10T01:38:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8365.856817 secs 2023-09-10T01:38:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:55 smithi170 ceph-mon[102832]: pgmap v4626: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T01:38:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:55 smithi082 ceph-mon[98694]: pgmap v4626: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T01:38:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:55 smithi098 ceph-mon[103124]: pgmap v4626: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T01:38:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:57 smithi170 ceph-mon[102832]: pgmap v4627: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 87 KiB/s wr, 16 op/s 2023-09-10T01:38:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8368.715894 secs 2023-09-10T01:38:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:57 smithi082 ceph-mon[98694]: pgmap v4627: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 87 KiB/s wr, 16 op/s 2023-09-10T01:38:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8368.715894 secs 2023-09-10T01:38:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:57 smithi098 ceph-mon[103124]: pgmap v4627: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 87 KiB/s wr, 16 op/s 2023-09-10T01:38:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8368.715894 secs 2023-09-10T01:38:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8370.145348 secs 2023-09-10T01:38:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8370.856882 secs 2023-09-10T01:38:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8370.145348 secs 2023-09-10T01:38:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8370.856882 secs 2023-09-10T01:38:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8370.145348 secs 2023-09-10T01:38:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8370.856882 secs 2023-09-10T01:39:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:38:59 smithi170 ceph-mon[102832]: pgmap v4628: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-10T01:39:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:38:59 smithi082 ceph-mon[98694]: pgmap v4628: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-10T01:39:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:38:59 smithi098 ceph-mon[103124]: pgmap v4628: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-10T01:39:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:01 smithi170 ceph-mon[102832]: pgmap v4629: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 15 op/s 2023-09-10T01:39:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:01 smithi082 ceph-mon[98694]: pgmap v4629: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 15 op/s 2023-09-10T01:39:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:01 smithi098 ceph-mon[103124]: pgmap v4629: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 15 op/s 2023-09-10T01:39:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8373.715957 secs 2023-09-10T01:39:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8373.715957 secs 2023-09-10T01:39:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8373.715957 secs 2023-09-10T01:39:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:03 smithi170 ceph-mon[102832]: pgmap v4630: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T01:39:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8375.145403 secs 2023-09-10T01:39:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8375.856949 secs 2023-09-10T01:39:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:03 smithi082 ceph-mon[98694]: pgmap v4630: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T01:39:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8375.145403 secs 2023-09-10T01:39:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8375.856949 secs 2023-09-10T01:39:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:03 smithi098 ceph-mon[103124]: pgmap v4630: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T01:39:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8375.145403 secs 2023-09-10T01:39:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8375.856949 secs 2023-09-10T01:39:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:05 smithi170 ceph-mon[102832]: pgmap v4631: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T01:39:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:05 smithi082 ceph-mon[98694]: pgmap v4631: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T01:39:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:05 smithi098 ceph-mon[103124]: pgmap v4631: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T01:39:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:07 smithi170 ceph-mon[102832]: pgmap v4632: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:39:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8378.716016 secs 2023-09-10T01:39:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:07 smithi082 ceph-mon[98694]: pgmap v4632: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:39:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8378.716016 secs 2023-09-10T01:39:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:07 smithi098 ceph-mon[103124]: pgmap v4632: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:39:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8378.716016 secs 2023-09-10T01:39:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:08 smithi170 ceph-mon[102832]: osdmap e359: 12 total, 12 up, 12 in 2023-09-10T01:39:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8380.145465 secs 2023-09-10T01:39:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8380.857017 secs 2023-09-10T01:39:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:08 smithi082 ceph-mon[98694]: osdmap e359: 12 total, 12 up, 12 in 2023-09-10T01:39:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8380.145465 secs 2023-09-10T01:39:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8380.857017 secs 2023-09-10T01:39:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:08 smithi098 ceph-mon[103124]: osdmap e359: 12 total, 12 up, 12 in 2023-09-10T01:39:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8380.145465 secs 2023-09-10T01:39:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8380.857017 secs 2023-09-10T01:39:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:09 smithi170 ceph-mon[102832]: pgmap v4634: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T01:39:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:09 smithi082 ceph-mon[98694]: pgmap v4634: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T01:39:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:09 smithi098 ceph-mon[103124]: pgmap v4634: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T01:39:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:11 smithi170 ceph-mon[102832]: pgmap v4635: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:39:12.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:11 smithi082 ceph-mon[98694]: pgmap v4635: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:39:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:11 smithi098 ceph-mon[103124]: pgmap v4635: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:39:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8383.716083 secs 2023-09-10T01:39:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8383.716083 secs 2023-09-10T01:39:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8383.716083 secs 2023-09-10T01:39:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:13 smithi170 ceph-mon[102832]: pgmap v4636: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:39:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8385.145494 secs 2023-09-10T01:39:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8385.857089 secs 2023-09-10T01:39:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:13 smithi082 ceph-mon[98694]: pgmap v4636: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:39:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8385.145494 secs 2023-09-10T01:39:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8385.857089 secs 2023-09-10T01:39:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:13 smithi098 ceph-mon[103124]: pgmap v4636: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:39:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8385.145494 secs 2023-09-10T01:39:14.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8385.857089 secs 2023-09-10T01:39:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:15 smithi170 ceph-mon[102832]: pgmap v4637: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:39:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:15 smithi170 ceph-mon[102832]: osdmap e360: 12 total, 12 up, 12 in 2023-09-10T01:39:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:15 smithi082 ceph-mon[98694]: pgmap v4637: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:39:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:15 smithi082 ceph-mon[98694]: osdmap e360: 12 total, 12 up, 12 in 2023-09-10T01:39:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:15 smithi098 ceph-mon[103124]: pgmap v4637: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:39:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:15 smithi098 ceph-mon[103124]: osdmap e360: 12 total, 12 up, 12 in 2023-09-10T01:39:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:17 smithi170 ceph-mon[102832]: pgmap v4639: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 13 op/s 2023-09-10T01:39:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8388.716155 secs 2023-09-10T01:39:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:17 smithi082 ceph-mon[98694]: pgmap v4639: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 13 op/s 2023-09-10T01:39:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8388.716155 secs 2023-09-10T01:39:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:17 smithi098 ceph-mon[103124]: pgmap v4639: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 13 op/s 2023-09-10T01:39:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8388.716155 secs 2023-09-10T01:39:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8390.145554 secs 2023-09-10T01:39:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8390.857151 secs 2023-09-10T01:39:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8390.145554 secs 2023-09-10T01:39:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8390.857151 secs 2023-09-10T01:39:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8390.145554 secs 2023-09-10T01:39:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8390.857151 secs 2023-09-10T01:39:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:19 smithi170 ceph-mon[102832]: pgmap v4640: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 11 op/s 2023-09-10T01:39:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:19 smithi082 ceph-mon[98694]: pgmap v4640: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 11 op/s 2023-09-10T01:39:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:19 smithi098 ceph-mon[103124]: pgmap v4640: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 11 op/s 2023-09-10T01:39:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:21 smithi170 ceph-mon[102832]: pgmap v4641: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-10T01:39:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:21 smithi082 ceph-mon[98694]: pgmap v4641: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-10T01:39:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:21 smithi098 ceph-mon[103124]: pgmap v4641: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-10T01:39:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:39:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:39:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:39:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:23 smithi170 ceph-mon[102832]: pgmap v4642: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-10T01:39:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:39:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:39:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:39:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8395.145612 secs 2023-09-10T01:39:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8395.857217 secs 2023-09-10T01:39:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:23 smithi082 ceph-mon[98694]: pgmap v4642: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-10T01:39:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:39:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:39:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:39:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8395.145612 secs 2023-09-10T01:39:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8395.857217 secs 2023-09-10T01:39:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:23 smithi098 ceph-mon[103124]: pgmap v4642: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-10T01:39:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:39:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:39:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:39:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8395.145612 secs 2023-09-10T01:39:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8395.857217 secs 2023-09-10T01:39:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:25 smithi170 ceph-mon[102832]: pgmap v4643: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-10T01:39:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:25 smithi082 ceph-mon[98694]: pgmap v4643: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-10T01:39:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:25 smithi098 ceph-mon[103124]: pgmap v4643: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-10T01:39:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:27 smithi170 ceph-mon[102832]: pgmap v4644: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T01:39:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:27 smithi082 ceph-mon[98694]: pgmap v4644: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T01:39:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:27 smithi098 ceph-mon[103124]: pgmap v4644: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T01:39:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8400.145695 secs 2023-09-10T01:39:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8400.857279 secs 2023-09-10T01:39:29.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8400.145695 secs 2023-09-10T01:39:29.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8400.857279 secs 2023-09-10T01:39:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8400.145695 secs 2023-09-10T01:39:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8400.857279 secs 2023-09-10T01:39:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:29 smithi170 ceph-mon[102832]: pgmap v4645: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 10 op/s 2023-09-10T01:39:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:29 smithi082 ceph-mon[98694]: pgmap v4645: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 10 op/s 2023-09-10T01:39:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:29 smithi098 ceph-mon[103124]: pgmap v4645: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 10 op/s 2023-09-10T01:39:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:31 smithi170 ceph-mon[102832]: pgmap v4646: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 99 KiB/s wr, 16 op/s 2023-09-10T01:39:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:31 smithi082 ceph-mon[98694]: pgmap v4646: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 99 KiB/s wr, 16 op/s 2023-09-10T01:39:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:31 smithi098 ceph-mon[103124]: pgmap v4646: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 99 KiB/s wr, 16 op/s 2023-09-10T01:39:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:33 smithi170 ceph-mon[102832]: pgmap v4647: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T01:39:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8405.145754 secs 2023-09-10T01:39:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8405.857344 secs 2023-09-10T01:39:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:39:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:33 smithi082 ceph-mon[98694]: pgmap v4647: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T01:39:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8405.145754 secs 2023-09-10T01:39:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8405.857344 secs 2023-09-10T01:39:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:39:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:33 smithi098 ceph-mon[103124]: pgmap v4647: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T01:39:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8405.145754 secs 2023-09-10T01:39:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8405.857344 secs 2023-09-10T01:39:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:39:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:39:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:39:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:39:35.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:39:35.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:39:35.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:35 smithi098 ceph-mon[103124]: pgmap v4648: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T01:39:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:39:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:39:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:39:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:39:35.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:39:35.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:35 smithi170 ceph-mon[102832]: pgmap v4648: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T01:39:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:39:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:39:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:39:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:39:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:39:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:35 smithi082 ceph-mon[98694]: pgmap v4648: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T01:39:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:37 smithi170 ceph-mon[102832]: pgmap v4649: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 114 KiB/s wr, 18 op/s 2023-09-10T01:39:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:37 smithi082 ceph-mon[98694]: pgmap v4649: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 114 KiB/s wr, 18 op/s 2023-09-10T01:39:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:37 smithi098 ceph-mon[103124]: pgmap v4649: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 114 KiB/s wr, 18 op/s 2023-09-10T01:39:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8410.145810 secs 2023-09-10T01:39:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8410.857411 secs 2023-09-10T01:39:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8410.145810 secs 2023-09-10T01:39:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8410.857411 secs 2023-09-10T01:39:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8410.145810 secs 2023-09-10T01:39:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8410.857411 secs 2023-09-10T01:39:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:39 smithi170 ceph-mon[102832]: pgmap v4650: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 12 op/s 2023-09-10T01:39:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:39 smithi082 ceph-mon[98694]: pgmap v4650: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 12 op/s 2023-09-10T01:39:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:39 smithi098 ceph-mon[103124]: pgmap v4650: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 12 op/s 2023-09-10T01:39:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:41 smithi170 ceph-mon[102832]: pgmap v4651: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 105 KiB/s wr, 18 op/s 2023-09-10T01:39:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:41 smithi082 ceph-mon[98694]: pgmap v4651: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 105 KiB/s wr, 18 op/s 2023-09-10T01:39:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:41 smithi098 ceph-mon[103124]: pgmap v4651: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 105 KiB/s wr, 18 op/s 2023-09-10T01:39:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:43 smithi170 ceph-mon[102832]: pgmap v4652: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 60 KiB/s wr, 12 op/s 2023-09-10T01:39:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8415.145897 secs 2023-09-10T01:39:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8415.857483 secs 2023-09-10T01:39:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:43 smithi082 ceph-mon[98694]: pgmap v4652: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 60 KiB/s wr, 12 op/s 2023-09-10T01:39:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8415.145897 secs 2023-09-10T01:39:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8415.857483 secs 2023-09-10T01:39:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:43 smithi098 ceph-mon[103124]: pgmap v4652: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 60 KiB/s wr, 12 op/s 2023-09-10T01:39:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8415.145897 secs 2023-09-10T01:39:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8415.857483 secs 2023-09-10T01:39:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:45 smithi170 ceph-mon[102832]: pgmap v4653: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 60 KiB/s wr, 12 op/s 2023-09-10T01:39:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:45 smithi082 ceph-mon[98694]: pgmap v4653: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 60 KiB/s wr, 12 op/s 2023-09-10T01:39:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:45 smithi098 ceph-mon[103124]: pgmap v4653: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 60 KiB/s wr, 12 op/s 2023-09-10T01:39:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:47 smithi170 ceph-mon[102832]: pgmap v4654: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 100 KiB/s wr, 18 op/s 2023-09-10T01:39:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8418.716558 secs 2023-09-10T01:39:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:47 smithi082 ceph-mon[98694]: pgmap v4654: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 100 KiB/s wr, 18 op/s 2023-09-10T01:39:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8418.716558 secs 2023-09-10T01:39:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:47 smithi098 ceph-mon[103124]: pgmap v4654: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 100 KiB/s wr, 18 op/s 2023-09-10T01:39:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8418.716558 secs 2023-09-10T01:39:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8420.145956 secs 2023-09-10T01:39:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8420.857550 secs 2023-09-10T01:39:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8420.145956 secs 2023-09-10T01:39:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8420.857550 secs 2023-09-10T01:39:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8420.145956 secs 2023-09-10T01:39:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8420.857550 secs 2023-09-10T01:39:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:49 smithi170 ceph-mon[102832]: pgmap v4655: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T01:39:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:49 smithi082 ceph-mon[98694]: pgmap v4655: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T01:39:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:49 smithi098 ceph-mon[103124]: pgmap v4655: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T01:39:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:51 smithi170 ceph-mon[102832]: pgmap v4656: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 84 KiB/s wr, 17 op/s 2023-09-10T01:39:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:39:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:39:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:39:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:51 smithi082 ceph-mon[98694]: pgmap v4656: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 84 KiB/s wr, 17 op/s 2023-09-10T01:39:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:39:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:39:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:39:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:51 smithi098 ceph-mon[103124]: pgmap v4656: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 84 KiB/s wr, 17 op/s 2023-09-10T01:39:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:39:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:39:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:39:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8423.716618 secs 2023-09-10T01:39:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8423.716618 secs 2023-09-10T01:39:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8423.716618 secs 2023-09-10T01:39:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:53 smithi170 ceph-mon[102832]: pgmap v4657: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T01:39:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8425.146013 secs 2023-09-10T01:39:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8425.857615 secs 2023-09-10T01:39:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:53 smithi082 ceph-mon[98694]: pgmap v4657: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T01:39:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8425.146013 secs 2023-09-10T01:39:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8425.857615 secs 2023-09-10T01:39:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:53 smithi098 ceph-mon[103124]: pgmap v4657: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T01:39:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8425.146013 secs 2023-09-10T01:39:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8425.857615 secs 2023-09-10T01:39:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:55 smithi170 ceph-mon[102832]: pgmap v4658: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T01:39:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:55 smithi082 ceph-mon[98694]: pgmap v4658: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T01:39:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:55 smithi098 ceph-mon[103124]: pgmap v4658: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T01:39:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:57 smithi170 ceph-mon[102832]: pgmap v4659: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 83 KiB/s wr, 16 op/s 2023-09-10T01:39:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8428.716714 secs 2023-09-10T01:39:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:57 smithi082 ceph-mon[98694]: pgmap v4659: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 83 KiB/s wr, 16 op/s 2023-09-10T01:39:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8428.716714 secs 2023-09-10T01:39:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:57 smithi098 ceph-mon[103124]: pgmap v4659: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 83 KiB/s wr, 16 op/s 2023-09-10T01:39:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8428.716714 secs 2023-09-10T01:39:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8430.146071 secs 2023-09-10T01:39:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8430.857679 secs 2023-09-10T01:39:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8430.146071 secs 2023-09-10T01:39:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8430.857679 secs 2023-09-10T01:39:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8430.146071 secs 2023-09-10T01:39:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8430.857679 secs 2023-09-10T01:40:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:39:59 smithi170 ceph-mon[102832]: pgmap v4660: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T01:40:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:39:59 smithi082 ceph-mon[98694]: pgmap v4660: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T01:40:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:39:59 smithi098 ceph-mon[103124]: pgmap v4660: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T01:40:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T01:40:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T01:40:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T01:40:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:01 smithi170 ceph-mon[102832]: pgmap v4661: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 55 KiB/s wr, 15 op/s 2023-09-10T01:40:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:01 smithi082 ceph-mon[98694]: pgmap v4661: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 55 KiB/s wr, 15 op/s 2023-09-10T01:40:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:01 smithi098 ceph-mon[103124]: pgmap v4661: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 55 KiB/s wr, 15 op/s 2023-09-10T01:40:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8433.716807 secs 2023-09-10T01:40:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8433.716807 secs 2023-09-10T01:40:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8433.716807 secs 2023-09-10T01:40:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:03 smithi170 ceph-mon[102832]: pgmap v4662: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 10 op/s 2023-09-10T01:40:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8435.146163 secs 2023-09-10T01:40:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8435.857746 secs 2023-09-10T01:40:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:03 smithi082 ceph-mon[98694]: pgmap v4662: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 10 op/s 2023-09-10T01:40:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8435.146163 secs 2023-09-10T01:40:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8435.857746 secs 2023-09-10T01:40:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:03 smithi098 ceph-mon[103124]: pgmap v4662: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 10 op/s 2023-09-10T01:40:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8435.146163 secs 2023-09-10T01:40:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8435.857746 secs 2023-09-10T01:40:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:05 smithi170 ceph-mon[102832]: pgmap v4663: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 10 op/s 2023-09-10T01:40:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:05 smithi082 ceph-mon[98694]: pgmap v4663: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 10 op/s 2023-09-10T01:40:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:05 smithi098 ceph-mon[103124]: pgmap v4663: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 10 op/s 2023-09-10T01:40:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:07 smithi170 ceph-mon[102832]: pgmap v4664: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:40:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8438.716868 secs 2023-09-10T01:40:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:07 smithi082 ceph-mon[98694]: pgmap v4664: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:40:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8438.716868 secs 2023-09-10T01:40:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:07 smithi098 ceph-mon[103124]: pgmap v4664: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:40:08.284 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8438.716868 secs 2023-09-10T01:40:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:08 smithi170 ceph-mon[102832]: osdmap e361: 12 total, 12 up, 12 in 2023-09-10T01:40:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8440.146221 secs 2023-09-10T01:40:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8440.857811 secs 2023-09-10T01:40:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:08 smithi082 ceph-mon[98694]: osdmap e361: 12 total, 12 up, 12 in 2023-09-10T01:40:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8440.146221 secs 2023-09-10T01:40:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8440.857811 secs 2023-09-10T01:40:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:08 smithi098 ceph-mon[103124]: osdmap e361: 12 total, 12 up, 12 in 2023-09-10T01:40:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8440.146221 secs 2023-09-10T01:40:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8440.857811 secs 2023-09-10T01:40:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:09 smithi170 ceph-mon[102832]: pgmap v4666: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 68 KiB/s wr, 29 op/s 2023-09-10T01:40:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:09 smithi082 ceph-mon[98694]: pgmap v4666: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 68 KiB/s wr, 29 op/s 2023-09-10T01:40:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:09 smithi098 ceph-mon[103124]: pgmap v4666: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 68 KiB/s wr, 29 op/s 2023-09-10T01:40:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:11 smithi170 ceph-mon[102832]: pgmap v4667: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:40:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:11 smithi082 ceph-mon[98694]: pgmap v4667: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:40:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:11 smithi098 ceph-mon[103124]: pgmap v4667: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:40:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8443.716985 secs 2023-09-10T01:40:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8443.716985 secs 2023-09-10T01:40:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8443.716985 secs 2023-09-10T01:40:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:13 smithi170 ceph-mon[102832]: pgmap v4668: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:40:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8445.146279 secs 2023-09-10T01:40:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8445.857875 secs 2023-09-10T01:40:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:13 smithi082 ceph-mon[98694]: pgmap v4668: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:40:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8445.146279 secs 2023-09-10T01:40:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8445.857875 secs 2023-09-10T01:40:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:13 smithi098 ceph-mon[103124]: pgmap v4668: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:40:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8445.146279 secs 2023-09-10T01:40:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8445.857875 secs 2023-09-10T01:40:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:15 smithi170 ceph-mon[102832]: pgmap v4669: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:40:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:15 smithi170 ceph-mon[102832]: osdmap e362: 12 total, 12 up, 12 in 2023-09-10T01:40:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:15 smithi082 ceph-mon[98694]: pgmap v4669: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:40:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:15 smithi082 ceph-mon[98694]: osdmap e362: 12 total, 12 up, 12 in 2023-09-10T01:40:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:15 smithi098 ceph-mon[103124]: pgmap v4669: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:40:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:15 smithi098 ceph-mon[103124]: osdmap e362: 12 total, 12 up, 12 in 2023-09-10T01:40:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:17 smithi170 ceph-mon[102832]: pgmap v4671: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 55 KiB/s wr, 13 op/s 2023-09-10T01:40:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8448.717047 secs 2023-09-10T01:40:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:17 smithi082 ceph-mon[98694]: pgmap v4671: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 55 KiB/s wr, 13 op/s 2023-09-10T01:40:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8448.717047 secs 2023-09-10T01:40:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:17 smithi098 ceph-mon[103124]: pgmap v4671: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 55 KiB/s wr, 13 op/s 2023-09-10T01:40:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8448.717047 secs 2023-09-10T01:40:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8450.146365 secs 2023-09-10T01:40:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8450.857941 secs 2023-09-10T01:40:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8450.146365 secs 2023-09-10T01:40:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8450.857941 secs 2023-09-10T01:40:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8450.146365 secs 2023-09-10T01:40:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8450.857941 secs 2023-09-10T01:40:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:19 smithi170 ceph-mon[102832]: pgmap v4672: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T01:40:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:19 smithi082 ceph-mon[98694]: pgmap v4672: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T01:40:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:19 smithi098 ceph-mon[103124]: pgmap v4672: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T01:40:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:21 smithi170 ceph-mon[102832]: pgmap v4673: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T01:40:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:21 smithi082 ceph-mon[98694]: pgmap v4673: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T01:40:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:21 smithi098 ceph-mon[103124]: pgmap v4673: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T01:40:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:40:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:40:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:40:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:23 smithi170 ceph-mon[102832]: pgmap v4674: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T01:40:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:40:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:40:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:40:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8455.146424 secs 2023-09-10T01:40:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8455.858005 secs 2023-09-10T01:40:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:23 smithi082 ceph-mon[98694]: pgmap v4674: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T01:40:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:40:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:40:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:40:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8455.146424 secs 2023-09-10T01:40:24.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8455.858005 secs 2023-09-10T01:40:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:23 smithi098 ceph-mon[103124]: pgmap v4674: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T01:40:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:40:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:40:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:40:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8455.146424 secs 2023-09-10T01:40:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8455.858005 secs 2023-09-10T01:40:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:25 smithi170 ceph-mon[102832]: pgmap v4675: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T01:40:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:25 smithi082 ceph-mon[98694]: pgmap v4675: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T01:40:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:25 smithi098 ceph-mon[103124]: pgmap v4675: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T01:40:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:27 smithi170 ceph-mon[102832]: pgmap v4676: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T01:40:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:27 smithi082 ceph-mon[98694]: pgmap v4676: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T01:40:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:27 smithi098 ceph-mon[103124]: pgmap v4676: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T01:40:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8460.146487 secs 2023-09-10T01:40:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8460.858071 secs 2023-09-10T01:40:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8460.146487 secs 2023-09-10T01:40:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8460.858071 secs 2023-09-10T01:40:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8460.146487 secs 2023-09-10T01:40:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8460.858071 secs 2023-09-10T01:40:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:29 smithi170 ceph-mon[102832]: pgmap v4677: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-10T01:40:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:29 smithi082 ceph-mon[98694]: pgmap v4677: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-10T01:40:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:29 smithi098 ceph-mon[103124]: pgmap v4677: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-10T01:40:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:31 smithi170 ceph-mon[102832]: pgmap v4678: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 93 KiB/s wr, 16 op/s 2023-09-10T01:40:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:31 smithi082 ceph-mon[98694]: pgmap v4678: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 93 KiB/s wr, 16 op/s 2023-09-10T01:40:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:31 smithi098 ceph-mon[103124]: pgmap v4678: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 93 KiB/s wr, 16 op/s 2023-09-10T01:40:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:33 smithi170 ceph-mon[102832]: pgmap v4679: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T01:40:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8465.146518 secs 2023-09-10T01:40:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8465.858135 secs 2023-09-10T01:40:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:33 smithi082 ceph-mon[98694]: pgmap v4679: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T01:40:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8465.146518 secs 2023-09-10T01:40:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8465.858135 secs 2023-09-10T01:40:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:33 smithi098 ceph-mon[103124]: pgmap v4679: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T01:40:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8465.146518 secs 2023-09-10T01:40:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8465.858135 secs 2023-09-10T01:40:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:40:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:40:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:40:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:35 smithi170 ceph-mon[102832]: pgmap v4680: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T01:40:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:40:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:40:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:40:36.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:40:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:35 smithi082 ceph-mon[98694]: pgmap v4680: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T01:40:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:40:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:40:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:40:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:40:36.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:35 smithi098 ceph-mon[103124]: pgmap v4680: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T01:40:36.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:40:36.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:40:36.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:40:36.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:40:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:40:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:40:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:40:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:40:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:40:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:40:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:40:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:40:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:40:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:37 smithi170 ceph-mon[102832]: pgmap v4681: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 102 KiB/s wr, 17 op/s 2023-09-10T01:40:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:37 smithi082 ceph-mon[98694]: pgmap v4681: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 102 KiB/s wr, 17 op/s 2023-09-10T01:40:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:37 smithi098 ceph-mon[103124]: pgmap v4681: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 102 KiB/s wr, 17 op/s 2023-09-10T01:40:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8470.146576 secs 2023-09-10T01:40:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8470.858200 secs 2023-09-10T01:40:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8470.146576 secs 2023-09-10T01:40:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8470.858200 secs 2023-09-10T01:40:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8470.146576 secs 2023-09-10T01:40:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8470.858200 secs 2023-09-10T01:40:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:39 smithi170 ceph-mon[102832]: pgmap v4682: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 74 KiB/s wr, 12 op/s 2023-09-10T01:40:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:39 smithi082 ceph-mon[98694]: pgmap v4682: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 74 KiB/s wr, 12 op/s 2023-09-10T01:40:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:39 smithi098 ceph-mon[103124]: pgmap v4682: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 74 KiB/s wr, 12 op/s 2023-09-10T01:40:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:41 smithi170 ceph-mon[102832]: pgmap v4683: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 123 KiB/s wr, 17 op/s 2023-09-10T01:40:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:41 smithi082 ceph-mon[98694]: pgmap v4683: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 123 KiB/s wr, 17 op/s 2023-09-10T01:40:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:41 smithi098 ceph-mon[103124]: pgmap v4683: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 123 KiB/s wr, 17 op/s 2023-09-10T01:40:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:42 smithi082 ceph-mon[98694]: pgmap v4684: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T01:40:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:42 smithi098 ceph-mon[103124]: pgmap v4684: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T01:40:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:42 smithi170 ceph-mon[102832]: pgmap v4684: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T01:40:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8475.146634 secs 2023-09-10T01:40:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8475.858266 secs 2023-09-10T01:40:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8475.146634 secs 2023-09-10T01:40:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8475.858266 secs 2023-09-10T01:40:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8475.146634 secs 2023-09-10T01:40:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8475.858266 secs 2023-09-10T01:40:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:44 smithi082 ceph-mon[98694]: pgmap v4685: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T01:40:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:44 smithi098 ceph-mon[103124]: pgmap v4685: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T01:40:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:44 smithi170 ceph-mon[102832]: pgmap v4685: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T01:40:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:47 smithi170 ceph-mon[102832]: pgmap v4686: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 96 KiB/s wr, 17 op/s 2023-09-10T01:40:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8478.717572 secs 2023-09-10T01:40:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:47 smithi082 ceph-mon[98694]: pgmap v4686: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 96 KiB/s wr, 17 op/s 2023-09-10T01:40:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8478.717572 secs 2023-09-10T01:40:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:47 smithi098 ceph-mon[103124]: pgmap v4686: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 96 KiB/s wr, 17 op/s 2023-09-10T01:40:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8478.717572 secs 2023-09-10T01:40:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8480.146695 secs 2023-09-10T01:40:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8480.858331 secs 2023-09-10T01:40:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8480.146695 secs 2023-09-10T01:40:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8480.858331 secs 2023-09-10T01:40:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8480.146695 secs 2023-09-10T01:40:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8480.858331 secs 2023-09-10T01:40:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:49 smithi170 ceph-mon[102832]: pgmap v4687: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T01:40:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:49 smithi082 ceph-mon[98694]: pgmap v4687: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T01:40:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:49 smithi098 ceph-mon[103124]: pgmap v4687: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T01:40:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:51 smithi170 ceph-mon[102832]: pgmap v4688: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 93 KiB/s wr, 16 op/s 2023-09-10T01:40:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:40:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:40:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:40:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:51 smithi082 ceph-mon[98694]: pgmap v4688: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 93 KiB/s wr, 16 op/s 2023-09-10T01:40:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:40:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:40:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:40:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:51 smithi098 ceph-mon[103124]: pgmap v4688: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 93 KiB/s wr, 16 op/s 2023-09-10T01:40:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:40:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:40:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:40:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8483.717660 secs 2023-09-10T01:40:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8483.717660 secs 2023-09-10T01:40:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8483.717660 secs 2023-09-10T01:40:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:53 smithi170 ceph-mon[102832]: pgmap v4689: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T01:40:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8485.146754 secs 2023-09-10T01:40:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8485.858394 secs 2023-09-10T01:40:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:53 smithi082 ceph-mon[98694]: pgmap v4689: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T01:40:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8485.146754 secs 2023-09-10T01:40:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8485.858394 secs 2023-09-10T01:40:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:53 smithi098 ceph-mon[103124]: pgmap v4689: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T01:40:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8485.146754 secs 2023-09-10T01:40:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8485.858394 secs 2023-09-10T01:40:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:55 smithi170 ceph-mon[102832]: pgmap v4690: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T01:40:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:55 smithi082 ceph-mon[98694]: pgmap v4690: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T01:40:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:55 smithi098 ceph-mon[103124]: pgmap v4690: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T01:40:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:57 smithi170 ceph-mon[102832]: pgmap v4691: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 75 KiB/s wr, 16 op/s 2023-09-10T01:40:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8488.717719 secs 2023-09-10T01:40:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:57 smithi082 ceph-mon[98694]: pgmap v4691: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 75 KiB/s wr, 16 op/s 2023-09-10T01:40:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8488.717719 secs 2023-09-10T01:40:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:57 smithi098 ceph-mon[103124]: pgmap v4691: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 75 KiB/s wr, 16 op/s 2023-09-10T01:40:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8488.717719 secs 2023-09-10T01:40:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8490.146838 secs 2023-09-10T01:40:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8490.858465 secs 2023-09-10T01:40:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8490.146838 secs 2023-09-10T01:40:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8490.858465 secs 2023-09-10T01:40:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8490.146838 secs 2023-09-10T01:40:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8490.858465 secs 2023-09-10T01:41:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:40:59 smithi170 ceph-mon[102832]: pgmap v4692: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T01:41:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:40:59 smithi082 ceph-mon[98694]: pgmap v4692: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T01:41:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:40:59 smithi098 ceph-mon[103124]: pgmap v4692: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T01:41:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:01 smithi170 ceph-mon[102832]: pgmap v4693: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 16 op/s 2023-09-10T01:41:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:01 smithi082 ceph-mon[98694]: pgmap v4693: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 16 op/s 2023-09-10T01:41:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:01 smithi098 ceph-mon[103124]: pgmap v4693: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 16 op/s 2023-09-10T01:41:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8493.717795 secs 2023-09-10T01:41:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8493.717795 secs 2023-09-10T01:41:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8493.717795 secs 2023-09-10T01:41:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:03 smithi170 ceph-mon[102832]: pgmap v4694: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T01:41:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8495.146922 secs 2023-09-10T01:41:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8495.858489 secs 2023-09-10T01:41:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:03 smithi082 ceph-mon[98694]: pgmap v4694: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T01:41:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8495.146922 secs 2023-09-10T01:41:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8495.858489 secs 2023-09-10T01:41:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:03 smithi098 ceph-mon[103124]: pgmap v4694: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T01:41:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8495.146922 secs 2023-09-10T01:41:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8495.858489 secs 2023-09-10T01:41:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:05 smithi170 ceph-mon[102832]: pgmap v4695: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T01:41:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:05 smithi082 ceph-mon[98694]: pgmap v4695: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T01:41:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:05 smithi098 ceph-mon[103124]: pgmap v4695: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T01:41:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:07 smithi170 ceph-mon[102832]: pgmap v4696: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 88 KiB/s wr, 31 op/s 2023-09-10T01:41:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8498.717856 secs 2023-09-10T01:41:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:07 smithi082 ceph-mon[98694]: pgmap v4696: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 88 KiB/s wr, 31 op/s 2023-09-10T01:41:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8498.717856 secs 2023-09-10T01:41:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:07 smithi098 ceph-mon[103124]: pgmap v4696: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 88 KiB/s wr, 31 op/s 2023-09-10T01:41:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8498.717856 secs 2023-09-10T01:41:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:08 smithi170 ceph-mon[102832]: osdmap e363: 12 total, 12 up, 12 in 2023-09-10T01:41:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8500.147010 secs 2023-09-10T01:41:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8500.858556 secs 2023-09-10T01:41:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:08 smithi082 ceph-mon[98694]: osdmap e363: 12 total, 12 up, 12 in 2023-09-10T01:41:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8500.147010 secs 2023-09-10T01:41:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8500.858556 secs 2023-09-10T01:41:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:08 smithi098 ceph-mon[103124]: osdmap e363: 12 total, 12 up, 12 in 2023-09-10T01:41:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8500.147010 secs 2023-09-10T01:41:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8500.858556 secs 2023-09-10T01:41:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:09 smithi170 ceph-mon[102832]: pgmap v4698: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 69 KiB/s wr, 30 op/s 2023-09-10T01:41:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:09 smithi082 ceph-mon[98694]: pgmap v4698: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 69 KiB/s wr, 30 op/s 2023-09-10T01:41:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:09 smithi098 ceph-mon[103124]: pgmap v4698: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 69 KiB/s wr, 30 op/s 2023-09-10T01:41:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:11 smithi170 ceph-mon[102832]: pgmap v4699: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T01:41:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:11 smithi082 ceph-mon[98694]: pgmap v4699: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T01:41:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:11 smithi098 ceph-mon[103124]: pgmap v4699: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T01:41:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8503.717957 secs 2023-09-10T01:41:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8503.717957 secs 2023-09-10T01:41:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8503.717957 secs 2023-09-10T01:41:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:13 smithi170 ceph-mon[102832]: pgmap v4700: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T01:41:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8505.147064 secs 2023-09-10T01:41:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8505.858623 secs 2023-09-10T01:41:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:13 smithi082 ceph-mon[98694]: pgmap v4700: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T01:41:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8505.147064 secs 2023-09-10T01:41:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8505.858623 secs 2023-09-10T01:41:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:13 smithi098 ceph-mon[103124]: pgmap v4700: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T01:41:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8505.147064 secs 2023-09-10T01:41:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8505.858623 secs 2023-09-10T01:41:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:15 smithi170 ceph-mon[102832]: pgmap v4701: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T01:41:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:15 smithi170 ceph-mon[102832]: osdmap e364: 12 total, 12 up, 12 in 2023-09-10T01:41:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:15 smithi082 ceph-mon[98694]: pgmap v4701: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T01:41:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:15 smithi082 ceph-mon[98694]: osdmap e364: 12 total, 12 up, 12 in 2023-09-10T01:41:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:15 smithi098 ceph-mon[103124]: pgmap v4701: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T01:41:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:15 smithi098 ceph-mon[103124]: osdmap e364: 12 total, 12 up, 12 in 2023-09-10T01:41:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:17 smithi170 ceph-mon[102832]: pgmap v4703: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 13 op/s 2023-09-10T01:41:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8508.718053 secs 2023-09-10T01:41:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:17 smithi082 ceph-mon[98694]: pgmap v4703: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 13 op/s 2023-09-10T01:41:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8508.718053 secs 2023-09-10T01:41:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:17 smithi098 ceph-mon[103124]: pgmap v4703: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 13 op/s 2023-09-10T01:41:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8508.718053 secs 2023-09-10T01:41:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8510.147122 secs 2023-09-10T01:41:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8510.858689 secs 2023-09-10T01:41:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8510.147122 secs 2023-09-10T01:41:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8510.858689 secs 2023-09-10T01:41:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8510.147122 secs 2023-09-10T01:41:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8510.858689 secs 2023-09-10T01:41:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:19 smithi170 ceph-mon[102832]: pgmap v4704: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T01:41:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:19 smithi082 ceph-mon[98694]: pgmap v4704: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T01:41:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:19 smithi098 ceph-mon[103124]: pgmap v4704: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T01:41:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:21 smithi170 ceph-mon[102832]: pgmap v4705: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T01:41:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:21 smithi082 ceph-mon[98694]: pgmap v4705: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T01:41:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:21 smithi098 ceph-mon[103124]: pgmap v4705: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T01:41:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8513.718113 secs 2023-09-10T01:41:23.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8513.718113 secs 2023-09-10T01:41:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8513.718113 secs 2023-09-10T01:41:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:23 smithi170 ceph-mon[102832]: pgmap v4706: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T01:41:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8515.147206 secs 2023-09-10T01:41:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8515.858754 secs 2023-09-10T01:41:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:23 smithi082 ceph-mon[98694]: pgmap v4706: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T01:41:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8515.147206 secs 2023-09-10T01:41:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8515.858754 secs 2023-09-10T01:41:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:23 smithi098 ceph-mon[103124]: pgmap v4706: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T01:41:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8515.147206 secs 2023-09-10T01:41:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8515.858754 secs 2023-09-10T01:41:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:25 smithi170 ceph-mon[102832]: pgmap v4707: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T01:41:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:25 smithi082 ceph-mon[98694]: pgmap v4707: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T01:41:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:25 smithi098 ceph-mon[103124]: pgmap v4707: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T01:41:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:27 smithi170 ceph-mon[102832]: pgmap v4708: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T01:41:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8518.718207 secs 2023-09-10T01:41:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:27 smithi082 ceph-mon[98694]: pgmap v4708: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T01:41:28.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8518.718207 secs 2023-09-10T01:41:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:27 smithi098 ceph-mon[103124]: pgmap v4708: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T01:41:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8518.718207 secs 2023-09-10T01:41:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8520.147265 secs 2023-09-10T01:41:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8520.858820 secs 2023-09-10T01:41:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8520.147265 secs 2023-09-10T01:41:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8520.858820 secs 2023-09-10T01:41:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8520.147265 secs 2023-09-10T01:41:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8520.858820 secs 2023-09-10T01:41:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:29 smithi170 ceph-mon[102832]: pgmap v4709: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T01:41:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:29 smithi082 ceph-mon[98694]: pgmap v4709: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T01:41:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:29 smithi098 ceph-mon[103124]: pgmap v4709: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T01:41:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:31 smithi170 ceph-mon[102832]: pgmap v4710: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 96 KiB/s wr, 16 op/s 2023-09-10T01:41:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:31 smithi082 ceph-mon[98694]: pgmap v4710: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 96 KiB/s wr, 16 op/s 2023-09-10T01:41:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:31 smithi098 ceph-mon[103124]: pgmap v4710: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 96 KiB/s wr, 16 op/s 2023-09-10T01:41:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:33 smithi170 ceph-mon[102832]: pgmap v4711: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:41:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8525.147342 secs 2023-09-10T01:41:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8525.858888 secs 2023-09-10T01:41:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:33 smithi082 ceph-mon[98694]: pgmap v4711: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:41:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8525.147342 secs 2023-09-10T01:41:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8525.858888 secs 2023-09-10T01:41:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:33 smithi098 ceph-mon[103124]: pgmap v4711: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:41:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8525.147342 secs 2023-09-10T01:41:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8525.858888 secs 2023-09-10T01:41:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:34 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:41:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:34 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:41:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:34 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:41:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:35 smithi170 ceph-mon[102832]: pgmap v4712: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:41:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:35 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:41:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:35 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:41:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:35 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:41:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:35 smithi082 ceph-mon[98694]: pgmap v4712: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:41:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:35 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:41:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:35 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:41:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:35 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:41:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:35 smithi098 ceph-mon[103124]: pgmap v4712: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:41:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:35 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:41:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:35 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:41:36.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:35 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:41:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:41:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:41:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:41:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:41:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:41:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:41:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:41:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:41:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:41:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:41:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:41:37.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:41:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:37 smithi170 ceph-mon[102832]: pgmap v4713: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 105 KiB/s wr, 17 op/s 2023-09-10T01:41:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:37 smithi082 ceph-mon[98694]: pgmap v4713: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 105 KiB/s wr, 17 op/s 2023-09-10T01:41:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:37 smithi098 ceph-mon[103124]: pgmap v4713: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 105 KiB/s wr, 17 op/s 2023-09-10T01:41:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8530.147416 secs 2023-09-10T01:41:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8530.858953 secs 2023-09-10T01:41:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8530.147416 secs 2023-09-10T01:41:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8530.858953 secs 2023-09-10T01:41:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8530.147416 secs 2023-09-10T01:41:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8530.858953 secs 2023-09-10T01:41:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:39 smithi170 ceph-mon[102832]: pgmap v4714: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 12 op/s 2023-09-10T01:41:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:39 smithi082 ceph-mon[98694]: pgmap v4714: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 12 op/s 2023-09-10T01:41:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:39 smithi098 ceph-mon[103124]: pgmap v4714: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 12 op/s 2023-09-10T01:41:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:41 smithi170 ceph-mon[102832]: pgmap v4715: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 108 KiB/s wr, 17 op/s 2023-09-10T01:41:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:41 smithi082 ceph-mon[98694]: pgmap v4715: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 108 KiB/s wr, 17 op/s 2023-09-10T01:41:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:41 smithi098 ceph-mon[103124]: pgmap v4715: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 108 KiB/s wr, 17 op/s 2023-09-10T01:41:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:43 smithi170 ceph-mon[102832]: pgmap v4716: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:41:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8535.147519 secs 2023-09-10T01:41:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8535.859016 secs 2023-09-10T01:41:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:43 smithi082 ceph-mon[98694]: pgmap v4716: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:41:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8535.147519 secs 2023-09-10T01:41:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8535.859016 secs 2023-09-10T01:41:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:43 smithi098 ceph-mon[103124]: pgmap v4716: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:41:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8535.147519 secs 2023-09-10T01:41:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8535.859016 secs 2023-09-10T01:41:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:45 smithi170 ceph-mon[102832]: pgmap v4717: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:41:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:45 smithi082 ceph-mon[98694]: pgmap v4717: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:41:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:45 smithi098 ceph-mon[103124]: pgmap v4717: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:41:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:47 smithi170 ceph-mon[102832]: pgmap v4718: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 110 KiB/s wr, 17 op/s 2023-09-10T01:41:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8538.718523 secs 2023-09-10T01:41:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:47 smithi082 ceph-mon[98694]: pgmap v4718: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 110 KiB/s wr, 17 op/s 2023-09-10T01:41:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8538.718523 secs 2023-09-10T01:41:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:47 smithi098 ceph-mon[103124]: pgmap v4718: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 110 KiB/s wr, 17 op/s 2023-09-10T01:41:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8538.718523 secs 2023-09-10T01:41:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8540.147602 secs 2023-09-10T01:41:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8540.859106 secs 2023-09-10T01:41:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8540.147602 secs 2023-09-10T01:41:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8540.859106 secs 2023-09-10T01:41:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8540.147602 secs 2023-09-10T01:41:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8540.859106 secs 2023-09-10T01:41:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:49 smithi170 ceph-mon[102832]: pgmap v4719: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 10 op/s 2023-09-10T01:41:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:49 smithi082 ceph-mon[98694]: pgmap v4719: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 10 op/s 2023-09-10T01:41:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:49 smithi098 ceph-mon[103124]: pgmap v4719: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 10 op/s 2023-09-10T01:41:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:51 smithi170 ceph-mon[102832]: pgmap v4720: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 105 KiB/s wr, 16 op/s 2023-09-10T01:41:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:41:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:41:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:41:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:51 smithi082 ceph-mon[98694]: pgmap v4720: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 105 KiB/s wr, 16 op/s 2023-09-10T01:41:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:41:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:41:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:41:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:51 smithi098 ceph-mon[103124]: pgmap v4720: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 105 KiB/s wr, 16 op/s 2023-09-10T01:41:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:41:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:41:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:41:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8543.718590 secs 2023-09-10T01:41:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8543.718590 secs 2023-09-10T01:41:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8543.718590 secs 2023-09-10T01:41:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:53 smithi170 ceph-mon[102832]: pgmap v4721: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T01:41:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8545.147663 secs 2023-09-10T01:41:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8545.859173 secs 2023-09-10T01:41:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:53 smithi082 ceph-mon[98694]: pgmap v4721: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T01:41:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8545.147663 secs 2023-09-10T01:41:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8545.859173 secs 2023-09-10T01:41:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:53 smithi098 ceph-mon[103124]: pgmap v4721: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T01:41:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8545.147663 secs 2023-09-10T01:41:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8545.859173 secs 2023-09-10T01:41:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:55 smithi170 ceph-mon[102832]: pgmap v4722: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T01:41:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:55 smithi082 ceph-mon[98694]: pgmap v4722: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T01:41:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:55 smithi098 ceph-mon[103124]: pgmap v4722: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T01:41:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:57 smithi170 ceph-mon[102832]: pgmap v4723: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 105 KiB/s wr, 16 op/s 2023-09-10T01:41:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8548.718672 secs 2023-09-10T01:41:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:57 smithi082 ceph-mon[98694]: pgmap v4723: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 105 KiB/s wr, 16 op/s 2023-09-10T01:41:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8548.718672 secs 2023-09-10T01:41:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:57 smithi098 ceph-mon[103124]: pgmap v4723: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 105 KiB/s wr, 16 op/s 2023-09-10T01:41:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8548.718672 secs 2023-09-10T01:41:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8550.147721 secs 2023-09-10T01:41:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8550.859238 secs 2023-09-10T01:41:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8550.147721 secs 2023-09-10T01:41:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8550.859238 secs 2023-09-10T01:41:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8550.147721 secs 2023-09-10T01:41:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8550.859238 secs 2023-09-10T01:42:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:41:59 smithi170 ceph-mon[102832]: pgmap v4724: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T01:42:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:41:59 smithi082 ceph-mon[98694]: pgmap v4724: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T01:42:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:41:59 smithi098 ceph-mon[103124]: pgmap v4724: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T01:42:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:01 smithi170 ceph-mon[102832]: pgmap v4725: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 15 op/s 2023-09-10T01:42:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:01 smithi082 ceph-mon[98694]: pgmap v4725: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 15 op/s 2023-09-10T01:42:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:01 smithi098 ceph-mon[103124]: pgmap v4725: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 15 op/s 2023-09-10T01:42:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8553.718795 secs 2023-09-10T01:42:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8553.718795 secs 2023-09-10T01:42:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8553.718795 secs 2023-09-10T01:42:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:03 smithi170 ceph-mon[102832]: pgmap v4726: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T01:42:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8555.147782 secs 2023-09-10T01:42:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8555.859320 secs 2023-09-10T01:42:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:03 smithi082 ceph-mon[98694]: pgmap v4726: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T01:42:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8555.147782 secs 2023-09-10T01:42:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8555.859320 secs 2023-09-10T01:42:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:03 smithi098 ceph-mon[103124]: pgmap v4726: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T01:42:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8555.147782 secs 2023-09-10T01:42:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8555.859320 secs 2023-09-10T01:42:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:05 smithi170 ceph-mon[102832]: pgmap v4727: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T01:42:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:05 smithi082 ceph-mon[98694]: pgmap v4727: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T01:42:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:05 smithi098 ceph-mon[103124]: pgmap v4727: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 43 KiB/s wr, 10 op/s 2023-09-10T01:42:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:07 smithi170 ceph-mon[102832]: pgmap v4728: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T01:42:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8558.718871 secs 2023-09-10T01:42:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:07 smithi082 ceph-mon[98694]: pgmap v4728: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T01:42:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8558.718871 secs 2023-09-10T01:42:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:07 smithi098 ceph-mon[103124]: pgmap v4728: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T01:42:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8558.718871 secs 2023-09-10T01:42:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:08 smithi170 ceph-mon[102832]: osdmap e365: 12 total, 12 up, 12 in 2023-09-10T01:42:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8560.147868 secs 2023-09-10T01:42:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8560.859385 secs 2023-09-10T01:42:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:08 smithi082 ceph-mon[98694]: osdmap e365: 12 total, 12 up, 12 in 2023-09-10T01:42:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8560.147868 secs 2023-09-10T01:42:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8560.859385 secs 2023-09-10T01:42:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:08 smithi098 ceph-mon[103124]: osdmap e365: 12 total, 12 up, 12 in 2023-09-10T01:42:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8560.147868 secs 2023-09-10T01:42:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8560.859385 secs 2023-09-10T01:42:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:09 smithi170 ceph-mon[102832]: pgmap v4730: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-10T01:42:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:09 smithi082 ceph-mon[98694]: pgmap v4730: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-10T01:42:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:09 smithi098 ceph-mon[103124]: pgmap v4730: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-10T01:42:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:11 smithi170 ceph-mon[102832]: pgmap v4731: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T01:42:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:11 smithi082 ceph-mon[98694]: pgmap v4731: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T01:42:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:11 smithi098 ceph-mon[103124]: pgmap v4731: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T01:42:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8563.718931 secs 2023-09-10T01:42:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8563.718931 secs 2023-09-10T01:42:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8563.718931 secs 2023-09-10T01:42:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:13 smithi170 ceph-mon[102832]: pgmap v4732: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T01:42:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8565.147927 secs 2023-09-10T01:42:14.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8565.859449 secs 2023-09-10T01:42:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:13 smithi082 ceph-mon[98694]: pgmap v4732: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T01:42:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8565.147927 secs 2023-09-10T01:42:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8565.859449 secs 2023-09-10T01:42:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:13 smithi098 ceph-mon[103124]: pgmap v4732: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T01:42:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8565.147927 secs 2023-09-10T01:42:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8565.859449 secs 2023-09-10T01:42:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:15 smithi170 ceph-mon[102832]: pgmap v4733: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T01:42:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:15 smithi170 ceph-mon[102832]: osdmap e366: 12 total, 12 up, 12 in 2023-09-10T01:42:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:15 smithi082 ceph-mon[98694]: pgmap v4733: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T01:42:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:15 smithi082 ceph-mon[98694]: osdmap e366: 12 total, 12 up, 12 in 2023-09-10T01:42:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:15 smithi098 ceph-mon[103124]: pgmap v4733: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T01:42:16.281 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:15 smithi098 ceph-mon[103124]: osdmap e366: 12 total, 12 up, 12 in 2023-09-10T01:42:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:17 smithi170 ceph-mon[102832]: pgmap v4735: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 43 KiB/s wr, 13 op/s 2023-09-10T01:42:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8568.718990 secs 2023-09-10T01:42:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:17 smithi082 ceph-mon[98694]: pgmap v4735: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 43 KiB/s wr, 13 op/s 2023-09-10T01:42:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8568.718990 secs 2023-09-10T01:42:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:17 smithi098 ceph-mon[103124]: pgmap v4735: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 43 KiB/s wr, 13 op/s 2023-09-10T01:42:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8568.718990 secs 2023-09-10T01:42:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8570.147986 secs 2023-09-10T01:42:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8570.859484 secs 2023-09-10T01:42:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8570.147986 secs 2023-09-10T01:42:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8570.859484 secs 2023-09-10T01:42:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8570.147986 secs 2023-09-10T01:42:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8570.859484 secs 2023-09-10T01:42:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:19 smithi170 ceph-mon[102832]: pgmap v4736: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T01:42:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:19 smithi082 ceph-mon[98694]: pgmap v4736: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T01:42:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:19 smithi098 ceph-mon[103124]: pgmap v4736: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T01:42:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:21 smithi170 ceph-mon[102832]: pgmap v4737: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T01:42:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:21 smithi082 ceph-mon[98694]: pgmap v4737: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T01:42:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:21 smithi098 ceph-mon[103124]: pgmap v4737: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T01:42:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:42:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:42:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:42:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:23 smithi170 ceph-mon[102832]: pgmap v4738: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T01:42:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:42:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8575.148048 secs 2023-09-10T01:42:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:42:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:42:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8575.859551 secs 2023-09-10T01:42:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:23 smithi082 ceph-mon[98694]: pgmap v4738: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T01:42:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:42:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8575.148048 secs 2023-09-10T01:42:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:42:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:42:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8575.859551 secs 2023-09-10T01:42:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:23 smithi098 ceph-mon[103124]: pgmap v4738: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T01:42:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:42:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8575.148048 secs 2023-09-10T01:42:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:42:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:42:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8575.859551 secs 2023-09-10T01:42:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:25 smithi170 ceph-mon[102832]: pgmap v4739: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T01:42:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:25 smithi082 ceph-mon[98694]: pgmap v4739: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T01:42:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:25 smithi098 ceph-mon[103124]: pgmap v4739: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T01:42:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:42:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:42:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:42:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:27 smithi170 ceph-mon[102832]: pgmap v4740: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 57 KiB/s wr, 12 op/s 2023-09-10T01:42:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:27 smithi082 ceph-mon[98694]: pgmap v4740: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 57 KiB/s wr, 12 op/s 2023-09-10T01:42:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:27 smithi098 ceph-mon[103124]: pgmap v4740: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 57 KiB/s wr, 12 op/s 2023-09-10T01:42:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8580.148133 secs 2023-09-10T01:42:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8580.859615 secs 2023-09-10T01:42:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8580.148133 secs 2023-09-10T01:42:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8580.859615 secs 2023-09-10T01:42:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8580.148133 secs 2023-09-10T01:42:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8580.859615 secs 2023-09-10T01:42:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:29 smithi170 ceph-mon[102832]: pgmap v4741: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T01:42:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:29 smithi082 ceph-mon[98694]: pgmap v4741: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T01:42:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:29 smithi098 ceph-mon[103124]: pgmap v4741: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T01:42:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:31 smithi170 ceph-mon[102832]: pgmap v4742: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 159 KiB/s wr, 22 op/s 2023-09-10T01:42:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:31 smithi082 ceph-mon[98694]: pgmap v4742: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 159 KiB/s wr, 22 op/s 2023-09-10T01:42:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:31 smithi098 ceph-mon[103124]: pgmap v4742: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 159 KiB/s wr, 22 op/s 2023-09-10T01:42:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:33 smithi170 ceph-mon[102832]: pgmap v4743: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 143 KiB/s wr, 17 op/s 2023-09-10T01:42:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8585.148211 secs 2023-09-10T01:42:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8585.859680 secs 2023-09-10T01:42:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:33 smithi082 ceph-mon[98694]: pgmap v4743: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 143 KiB/s wr, 17 op/s 2023-09-10T01:42:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8585.148211 secs 2023-09-10T01:42:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8585.859680 secs 2023-09-10T01:42:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:33 smithi098 ceph-mon[103124]: pgmap v4743: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 143 KiB/s wr, 17 op/s 2023-09-10T01:42:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8585.148211 secs 2023-09-10T01:42:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8585.859680 secs 2023-09-10T01:42:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:35 smithi170 ceph-mon[102832]: pgmap v4744: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 143 KiB/s wr, 17 op/s 2023-09-10T01:42:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:35 smithi082 ceph-mon[98694]: pgmap v4744: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 143 KiB/s wr, 17 op/s 2023-09-10T01:42:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:35 smithi098 ceph-mon[103124]: pgmap v4744: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 143 KiB/s wr, 17 op/s 2023-09-10T01:42:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:42:37.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:42:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:42:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:37 smithi170 ceph-mon[102832]: pgmap v4745: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 259 KiB/s wr, 25 op/s 2023-09-10T01:42:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8588.719336 secs 2023-09-10T01:42:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:42:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:42:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:42:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:37 smithi082 ceph-mon[98694]: pgmap v4745: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 259 KiB/s wr, 25 op/s 2023-09-10T01:42:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8588.719336 secs 2023-09-10T01:42:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:42:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:42:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:42:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:37 smithi098 ceph-mon[103124]: pgmap v4745: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 259 KiB/s wr, 25 op/s 2023-09-10T01:42:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8588.719336 secs 2023-09-10T01:42:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:42:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:42:38.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:42:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8590.148272 secs 2023-09-10T01:42:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8590.859746 secs 2023-09-10T01:42:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:38 smithi170 ceph-mon[102832]: pgmap v4746: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 222 KiB/s wr, 18 op/s 2023-09-10T01:42:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8590.148272 secs 2023-09-10T01:42:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8590.859746 secs 2023-09-10T01:42:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:38 smithi082 ceph-mon[98694]: pgmap v4746: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 222 KiB/s wr, 18 op/s 2023-09-10T01:42:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8590.148272 secs 2023-09-10T01:42:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8590.859746 secs 2023-09-10T01:42:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:38 smithi098 ceph-mon[103124]: pgmap v4746: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 222 KiB/s wr, 18 op/s 2023-09-10T01:42:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:39 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:42:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:39 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:42:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:39 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:42:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:39 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:42:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:39 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:42:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:39 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:42:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:39 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:42:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:39 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:42:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:39 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:42:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:41 smithi170 ceph-mon[102832]: pgmap v4747: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 249 KiB/s wr, 26 op/s 2023-09-10T01:42:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:41 smithi082 ceph-mon[98694]: pgmap v4747: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 249 KiB/s wr, 26 op/s 2023-09-10T01:42:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:41 smithi098 ceph-mon[103124]: pgmap v4747: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 249 KiB/s wr, 26 op/s 2023-09-10T01:42:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8593.719395 secs 2023-09-10T01:42:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8593.719395 secs 2023-09-10T01:42:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8593.719395 secs 2023-09-10T01:42:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:43 smithi170 ceph-mon[102832]: pgmap v4748: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 143 KiB/s wr, 15 op/s 2023-09-10T01:42:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8595.148364 secs 2023-09-10T01:42:44.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8595.859811 secs 2023-09-10T01:42:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:43 smithi082 ceph-mon[98694]: pgmap v4748: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 143 KiB/s wr, 15 op/s 2023-09-10T01:42:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8595.148364 secs 2023-09-10T01:42:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8595.859811 secs 2023-09-10T01:42:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:43 smithi098 ceph-mon[103124]: pgmap v4748: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 143 KiB/s wr, 15 op/s 2023-09-10T01:42:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8595.148364 secs 2023-09-10T01:42:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8595.859811 secs 2023-09-10T01:42:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:45 smithi170 ceph-mon[102832]: pgmap v4749: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 143 KiB/s wr, 15 op/s 2023-09-10T01:42:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:45 smithi082 ceph-mon[98694]: pgmap v4749: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 143 KiB/s wr, 15 op/s 2023-09-10T01:42:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:45 smithi098 ceph-mon[103124]: pgmap v4749: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 143 KiB/s wr, 15 op/s 2023-09-10T01:42:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:47 smithi170 ceph-mon[102832]: pgmap v4750: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 198 KiB/s wr, 23 op/s 2023-09-10T01:42:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8598.719455 secs 2023-09-10T01:42:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:47 smithi082 ceph-mon[98694]: pgmap v4750: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 198 KiB/s wr, 23 op/s 2023-09-10T01:42:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8598.719455 secs 2023-09-10T01:42:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:47 smithi098 ceph-mon[103124]: pgmap v4750: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 198 KiB/s wr, 23 op/s 2023-09-10T01:42:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8598.719455 secs 2023-09-10T01:42:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8600.148461 secs 2023-09-10T01:42:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8600.859876 secs 2023-09-10T01:42:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8600.148461 secs 2023-09-10T01:42:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8600.859876 secs 2023-09-10T01:42:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8600.148461 secs 2023-09-10T01:42:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8600.859876 secs 2023-09-10T01:42:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:49 smithi170 ceph-mon[102832]: pgmap v4751: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 82 KiB/s wr, 16 op/s 2023-09-10T01:42:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:49 smithi082 ceph-mon[98694]: pgmap v4751: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 82 KiB/s wr, 16 op/s 2023-09-10T01:42:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:49 smithi098 ceph-mon[103124]: pgmap v4751: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 82 KiB/s wr, 16 op/s 2023-09-10T01:42:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:51 smithi170 ceph-mon[102832]: pgmap v4752: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 118 KiB/s wr, 24 op/s 2023-09-10T01:42:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:51 smithi082 ceph-mon[98694]: pgmap v4752: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 118 KiB/s wr, 24 op/s 2023-09-10T01:42:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:51 smithi098 ceph-mon[103124]: pgmap v4752: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 118 KiB/s wr, 24 op/s 2023-09-10T01:42:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8603.719550 secs 2023-09-10T01:42:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8603.719550 secs 2023-09-10T01:42:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8603.719550 secs 2023-09-10T01:42:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:53 smithi170 ceph-mon[102832]: pgmap v4753: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 91 KiB/s wr, 16 op/s 2023-09-10T01:42:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8605.148493 secs 2023-09-10T01:42:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8605.859941 secs 2023-09-10T01:42:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:53 smithi082 ceph-mon[98694]: pgmap v4753: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 91 KiB/s wr, 16 op/s 2023-09-10T01:42:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8605.148493 secs 2023-09-10T01:42:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8605.859941 secs 2023-09-10T01:42:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:53 smithi098 ceph-mon[103124]: pgmap v4753: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 91 KiB/s wr, 16 op/s 2023-09-10T01:42:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8605.148493 secs 2023-09-10T01:42:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8605.859941 secs 2023-09-10T01:42:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:55 smithi170 ceph-mon[102832]: pgmap v4754: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 91 KiB/s wr, 16 op/s 2023-09-10T01:42:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:55 smithi082 ceph-mon[98694]: pgmap v4754: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 91 KiB/s wr, 16 op/s 2023-09-10T01:42:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:55 smithi098 ceph-mon[103124]: pgmap v4754: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 91 KiB/s wr, 16 op/s 2023-09-10T01:42:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:57 smithi170 ceph-mon[102832]: pgmap v4755: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 126 KiB/s wr, 24 op/s 2023-09-10T01:42:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8608.719660 secs 2023-09-10T01:42:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:57 smithi082 ceph-mon[98694]: pgmap v4755: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 126 KiB/s wr, 24 op/s 2023-09-10T01:42:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8608.719660 secs 2023-09-10T01:42:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:57 smithi098 ceph-mon[103124]: pgmap v4755: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 126 KiB/s wr, 24 op/s 2023-09-10T01:42:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8608.719660 secs 2023-09-10T01:42:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8610.148550 secs 2023-09-10T01:42:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8610.860006 secs 2023-09-10T01:42:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8610.148550 secs 2023-09-10T01:42:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8610.860006 secs 2023-09-10T01:42:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8610.148550 secs 2023-09-10T01:42:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8610.860006 secs 2023-09-10T01:43:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:42:59 smithi170 ceph-mon[102832]: pgmap v4756: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 71 KiB/s wr, 15 op/s 2023-09-10T01:43:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:42:59 smithi082 ceph-mon[98694]: pgmap v4756: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 71 KiB/s wr, 15 op/s 2023-09-10T01:43:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:42:59 smithi098 ceph-mon[103124]: pgmap v4756: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 71 KiB/s wr, 15 op/s 2023-09-10T01:43:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:01 smithi170 ceph-mon[102832]: pgmap v4757: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 82 KiB/s wr, 21 op/s 2023-09-10T01:43:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:01 smithi082 ceph-mon[98694]: pgmap v4757: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 82 KiB/s wr, 21 op/s 2023-09-10T01:43:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:01 smithi098 ceph-mon[103124]: pgmap v4757: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 82 KiB/s wr, 21 op/s 2023-09-10T01:43:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8613.719767 secs 2023-09-10T01:43:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8613.719767 secs 2023-09-10T01:43:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8613.719767 secs 2023-09-10T01:43:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:03 smithi170 ceph-mon[102832]: pgmap v4758: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 46 KiB/s wr, 13 op/s 2023-09-10T01:43:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8615.148614 secs 2023-09-10T01:43:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8615.860073 secs 2023-09-10T01:43:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:03 smithi082 ceph-mon[98694]: pgmap v4758: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 46 KiB/s wr, 13 op/s 2023-09-10T01:43:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8615.148614 secs 2023-09-10T01:43:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8615.860073 secs 2023-09-10T01:43:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:03 smithi098 ceph-mon[103124]: pgmap v4758: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 46 KiB/s wr, 13 op/s 2023-09-10T01:43:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8615.148614 secs 2023-09-10T01:43:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8615.860073 secs 2023-09-10T01:43:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:05 smithi170 ceph-mon[102832]: pgmap v4759: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 46 KiB/s wr, 13 op/s 2023-09-10T01:43:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:05 smithi082 ceph-mon[98694]: pgmap v4759: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 46 KiB/s wr, 13 op/s 2023-09-10T01:43:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:05 smithi098 ceph-mon[103124]: pgmap v4759: 97 pgs: 97 active+clean; 9.3 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 46 KiB/s wr, 13 op/s 2023-09-10T01:43:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:07 smithi170 ceph-mon[102832]: pgmap v4760: 97 pgs: 97 active+clean; 9.3 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 107 KiB/s wr, 32 op/s 2023-09-10T01:43:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8618.719775 secs 2023-09-10T01:43:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:07 smithi082 ceph-mon[98694]: pgmap v4760: 97 pgs: 97 active+clean; 9.3 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 107 KiB/s wr, 32 op/s 2023-09-10T01:43:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8618.719775 secs 2023-09-10T01:43:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:07 smithi098 ceph-mon[103124]: pgmap v4760: 97 pgs: 97 active+clean; 9.3 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 107 KiB/s wr, 32 op/s 2023-09-10T01:43:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8618.719775 secs 2023-09-10T01:43:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8620.148673 secs 2023-09-10T01:43:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:08 smithi170 ceph-mon[102832]: osdmap e367: 12 total, 12 up, 12 in 2023-09-10T01:43:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8620.860139 secs 2023-09-10T01:43:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8620.148673 secs 2023-09-10T01:43:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:08 smithi082 ceph-mon[98694]: osdmap e367: 12 total, 12 up, 12 in 2023-09-10T01:43:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8620.860139 secs 2023-09-10T01:43:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8620.148673 secs 2023-09-10T01:43:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:08 smithi098 ceph-mon[103124]: osdmap e367: 12 total, 12 up, 12 in 2023-09-10T01:43:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8620.860139 secs 2023-09-10T01:43:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:09 smithi170 ceph-mon[102832]: pgmap v4762: 97 pgs: 97 active+clean; 9.3 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 86 KiB/s wr, 30 op/s 2023-09-10T01:43:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:09 smithi082 ceph-mon[98694]: pgmap v4762: 97 pgs: 97 active+clean; 9.3 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 86 KiB/s wr, 30 op/s 2023-09-10T01:43:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:09 smithi098 ceph-mon[103124]: pgmap v4762: 97 pgs: 97 active+clean; 9.3 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 86 KiB/s wr, 30 op/s 2023-09-10T01:43:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:11 smithi170 ceph-mon[102832]: pgmap v4763: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T01:43:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:11 smithi082 ceph-mon[98694]: pgmap v4763: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T01:43:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:11 smithi098 ceph-mon[103124]: pgmap v4763: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T01:43:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8623.719843 secs 2023-09-10T01:43:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8623.719843 secs 2023-09-10T01:43:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8623.719843 secs 2023-09-10T01:43:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:13 smithi170 ceph-mon[102832]: pgmap v4764: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T01:43:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8625.148733 secs 2023-09-10T01:43:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8625.860206 secs 2023-09-10T01:43:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:13 smithi082 ceph-mon[98694]: pgmap v4764: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T01:43:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8625.148733 secs 2023-09-10T01:43:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8625.860206 secs 2023-09-10T01:43:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:13 smithi098 ceph-mon[103124]: pgmap v4764: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T01:43:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8625.148733 secs 2023-09-10T01:43:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8625.860206 secs 2023-09-10T01:43:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:15 smithi170 ceph-mon[102832]: pgmap v4765: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T01:43:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:15 smithi170 ceph-mon[102832]: osdmap e368: 12 total, 12 up, 12 in 2023-09-10T01:43:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:15 smithi082 ceph-mon[98694]: pgmap v4765: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T01:43:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:15 smithi082 ceph-mon[98694]: osdmap e368: 12 total, 12 up, 12 in 2023-09-10T01:43:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:15 smithi098 ceph-mon[103124]: pgmap v4765: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T01:43:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:15 smithi098 ceph-mon[103124]: osdmap e368: 12 total, 12 up, 12 in 2023-09-10T01:43:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:17 smithi170 ceph-mon[102832]: pgmap v4767: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 39 KiB/s wr, 12 op/s 2023-09-10T01:43:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8628.719934 secs 2023-09-10T01:43:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:17 smithi082 ceph-mon[98694]: pgmap v4767: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 39 KiB/s wr, 12 op/s 2023-09-10T01:43:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8628.719934 secs 2023-09-10T01:43:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:17 smithi098 ceph-mon[103124]: pgmap v4767: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 39 KiB/s wr, 12 op/s 2023-09-10T01:43:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8628.719934 secs 2023-09-10T01:43:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8630.148791 secs 2023-09-10T01:43:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8630.860270 secs 2023-09-10T01:43:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8630.148791 secs 2023-09-10T01:43:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8630.860270 secs 2023-09-10T01:43:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8630.148791 secs 2023-09-10T01:43:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8630.860270 secs 2023-09-10T01:43:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:19 smithi170 ceph-mon[102832]: pgmap v4768: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T01:43:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:19 smithi082 ceph-mon[98694]: pgmap v4768: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T01:43:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:19 smithi098 ceph-mon[103124]: pgmap v4768: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T01:43:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:21 smithi170 ceph-mon[102832]: pgmap v4769: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-10T01:43:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:21 smithi082 ceph-mon[98694]: pgmap v4769: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-10T01:43:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:21 smithi098 ceph-mon[103124]: pgmap v4769: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-10T01:43:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:43:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:43:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:43:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:23 smithi170 ceph-mon[102832]: pgmap v4770: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-10T01:43:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:43:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:43:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:43:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8635.148850 secs 2023-09-10T01:43:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8635.860338 secs 2023-09-10T01:43:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:23 smithi082 ceph-mon[98694]: pgmap v4770: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-10T01:43:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:43:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:43:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:43:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8635.148850 secs 2023-09-10T01:43:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8635.860338 secs 2023-09-10T01:43:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:23 smithi098 ceph-mon[103124]: pgmap v4770: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-10T01:43:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:43:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:43:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:43:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8635.148850 secs 2023-09-10T01:43:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8635.860338 secs 2023-09-10T01:43:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:25 smithi170 ceph-mon[102832]: pgmap v4771: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-10T01:43:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:25 smithi082 ceph-mon[98694]: pgmap v4771: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-10T01:43:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:25 smithi098 ceph-mon[103124]: pgmap v4771: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-09-10T01:43:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:27 smithi170 ceph-mon[102832]: pgmap v4772: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 26 KiB/s wr, 10 op/s 2023-09-10T01:43:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:27 smithi082 ceph-mon[98694]: pgmap v4772: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 26 KiB/s wr, 10 op/s 2023-09-10T01:43:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:27 smithi098 ceph-mon[103124]: pgmap v4772: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 26 KiB/s wr, 10 op/s 2023-09-10T01:43:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8640.148936 secs 2023-09-10T01:43:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8640.860408 secs 2023-09-10T01:43:29.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8640.148936 secs 2023-09-10T01:43:29.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8640.860408 secs 2023-09-10T01:43:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8640.148936 secs 2023-09-10T01:43:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8640.860408 secs 2023-09-10T01:43:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:29 smithi170 ceph-mon[102832]: pgmap v4773: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 24 KiB/s wr, 9 op/s 2023-09-10T01:43:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:29 smithi082 ceph-mon[98694]: pgmap v4773: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 24 KiB/s wr, 9 op/s 2023-09-10T01:43:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:29 smithi098 ceph-mon[103124]: pgmap v4773: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 24 KiB/s wr, 9 op/s 2023-09-10T01:43:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:31 smithi170 ceph-mon[102832]: pgmap v4774: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 52 KiB/s wr, 15 op/s 2023-09-10T01:43:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:31 smithi082 ceph-mon[98694]: pgmap v4774: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 52 KiB/s wr, 15 op/s 2023-09-10T01:43:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:31 smithi098 ceph-mon[103124]: pgmap v4774: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 52 KiB/s wr, 15 op/s 2023-09-10T01:43:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:33 smithi170 ceph-mon[102832]: pgmap v4775: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T01:43:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8645.148997 secs 2023-09-10T01:43:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8645.860471 secs 2023-09-10T01:43:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:33 smithi082 ceph-mon[98694]: pgmap v4775: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T01:43:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8645.148997 secs 2023-09-10T01:43:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8645.860471 secs 2023-09-10T01:43:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:33 smithi098 ceph-mon[103124]: pgmap v4775: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T01:43:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8645.148997 secs 2023-09-10T01:43:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8645.860471 secs 2023-09-10T01:43:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:35 smithi170 ceph-mon[102832]: pgmap v4776: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T01:43:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:35 smithi082 ceph-mon[98694]: pgmap v4776: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T01:43:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:35 smithi098 ceph-mon[103124]: pgmap v4776: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T01:43:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:37 smithi170 ceph-mon[102832]: pgmap v4777: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 95 KiB/s wr, 17 op/s 2023-09-10T01:43:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:43:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:37 smithi082 ceph-mon[98694]: pgmap v4777: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 95 KiB/s wr, 17 op/s 2023-09-10T01:43:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:43:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:37 smithi098 ceph-mon[103124]: pgmap v4777: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 95 KiB/s wr, 17 op/s 2023-09-10T01:43:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:43:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8650.149056 secs 2023-09-10T01:43:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:43:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:43:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:43:39.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8650.860489 secs 2023-09-10T01:43:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8650.149056 secs 2023-09-10T01:43:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:43:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:43:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:43:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8650.860489 secs 2023-09-10T01:43:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8650.149056 secs 2023-09-10T01:43:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:43:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:43:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:43:39.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8650.860489 secs 2023-09-10T01:43:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:39 smithi170 ceph-mon[102832]: pgmap v4778: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 71 KiB/s wr, 13 op/s 2023-09-10T01:43:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:39 smithi082 ceph-mon[98694]: pgmap v4778: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 71 KiB/s wr, 13 op/s 2023-09-10T01:43:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:39 smithi098 ceph-mon[103124]: pgmap v4778: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 71 KiB/s wr, 13 op/s 2023-09-10T01:43:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:41 smithi170 ceph-mon[102832]: pgmap v4779: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 95 KiB/s wr, 18 op/s 2023-09-10T01:43:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:41 smithi082 ceph-mon[98694]: pgmap v4779: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 95 KiB/s wr, 18 op/s 2023-09-10T01:43:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:41 smithi098 ceph-mon[103124]: pgmap v4779: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 95 KiB/s wr, 18 op/s 2023-09-10T01:43:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:43 smithi170 ceph-mon[102832]: pgmap v4780: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 67 KiB/s wr, 12 op/s 2023-09-10T01:43:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8655.149113 secs 2023-09-10T01:43:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8655.860555 secs 2023-09-10T01:43:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:43 smithi082 ceph-mon[98694]: pgmap v4780: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 67 KiB/s wr, 12 op/s 2023-09-10T01:43:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8655.149113 secs 2023-09-10T01:43:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8655.860555 secs 2023-09-10T01:43:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:43 smithi098 ceph-mon[103124]: pgmap v4780: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 67 KiB/s wr, 12 op/s 2023-09-10T01:43:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8655.149113 secs 2023-09-10T01:43:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8655.860555 secs 2023-09-10T01:43:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:45 smithi170 ceph-mon[102832]: pgmap v4781: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 67 KiB/s wr, 12 op/s 2023-09-10T01:43:46.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:45 smithi082 ceph-mon[98694]: pgmap v4781: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 67 KiB/s wr, 12 op/s 2023-09-10T01:43:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:45 smithi098 ceph-mon[103124]: pgmap v4781: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 67 KiB/s wr, 12 op/s 2023-09-10T01:43:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:47 smithi170 ceph-mon[102832]: pgmap v4782: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 81 KiB/s wr, 17 op/s 2023-09-10T01:43:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:47 smithi082 ceph-mon[98694]: pgmap v4782: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 81 KiB/s wr, 17 op/s 2023-09-10T01:43:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:47 smithi098 ceph-mon[103124]: pgmap v4782: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 81 KiB/s wr, 17 op/s 2023-09-10T01:43:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8660.149185 secs 2023-09-10T01:43:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8660.860623 secs 2023-09-10T01:43:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8660.149185 secs 2023-09-10T01:43:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8660.860623 secs 2023-09-10T01:43:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8660.149185 secs 2023-09-10T01:43:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8660.860623 secs 2023-09-10T01:43:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:49 smithi170 ceph-mon[102832]: pgmap v4783: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T01:43:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:49 smithi082 ceph-mon[98694]: pgmap v4783: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T01:43:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:49 smithi098 ceph-mon[103124]: pgmap v4783: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T01:43:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:51 smithi170 ceph-mon[102832]: pgmap v4784: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 102 KiB/s wr, 16 op/s 2023-09-10T01:43:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:51 smithi082 ceph-mon[98694]: pgmap v4784: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 102 KiB/s wr, 16 op/s 2023-09-10T01:43:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:51 smithi098 ceph-mon[103124]: pgmap v4784: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 102 KiB/s wr, 16 op/s 2023-09-10T01:43:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:53 smithi170 ceph-mon[102832]: pgmap v4785: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T01:43:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8665.149242 secs 2023-09-10T01:43:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8665.860687 secs 2023-09-10T01:43:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:53 smithi082 ceph-mon[98694]: pgmap v4785: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T01:43:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8665.149242 secs 2023-09-10T01:43:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8665.860687 secs 2023-09-10T01:43:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:53 smithi098 ceph-mon[103124]: pgmap v4785: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T01:43:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8665.149242 secs 2023-09-10T01:43:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8665.860687 secs 2023-09-10T01:43:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:55 smithi170 ceph-mon[102832]: pgmap v4786: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T01:43:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:55 smithi082 ceph-mon[98694]: pgmap v4786: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T01:43:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:55 smithi098 ceph-mon[103124]: pgmap v4786: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T01:43:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:57 smithi170 ceph-mon[102832]: pgmap v4787: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 142 KiB/s wr, 17 op/s 2023-09-10T01:43:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8668.720532 secs 2023-09-10T01:43:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:57 smithi082 ceph-mon[98694]: pgmap v4787: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 142 KiB/s wr, 17 op/s 2023-09-10T01:43:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8668.720532 secs 2023-09-10T01:43:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:57 smithi098 ceph-mon[103124]: pgmap v4787: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 142 KiB/s wr, 17 op/s 2023-09-10T01:43:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8668.720532 secs 2023-09-10T01:43:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8670.149297 secs 2023-09-10T01:43:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8670.860755 secs 2023-09-10T01:43:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8670.149297 secs 2023-09-10T01:43:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8670.860755 secs 2023-09-10T01:43:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8670.149297 secs 2023-09-10T01:43:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8670.860755 secs 2023-09-10T01:44:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:59 smithi082 ceph-mon[98694]: pgmap v4788: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T01:44:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:59 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:44:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:59 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:44:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:43:59 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:44:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:59 smithi098 ceph-mon[103124]: pgmap v4788: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T01:44:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:59 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:44:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:59 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:44:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:43:59 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:44:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:59 smithi170 ceph-mon[102832]: pgmap v4788: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T01:44:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:59 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:44:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:59 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:44:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:43:59 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:44:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:01 smithi170 ceph-mon[102832]: pgmap v4789: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 159 KiB/s wr, 17 op/s 2023-09-10T01:44:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:01 smithi082 ceph-mon[98694]: pgmap v4789: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 159 KiB/s wr, 17 op/s 2023-09-10T01:44:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:01 smithi098 ceph-mon[103124]: pgmap v4789: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 159 KiB/s wr, 17 op/s 2023-09-10T01:44:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8673.720591 secs 2023-09-10T01:44:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8673.720591 secs 2023-09-10T01:44:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8673.720591 secs 2023-09-10T01:44:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:03 smithi170 ceph-mon[102832]: pgmap v4790: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T01:44:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8675.149356 secs 2023-09-10T01:44:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8675.860818 secs 2023-09-10T01:44:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:03 smithi082 ceph-mon[98694]: pgmap v4790: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T01:44:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8675.149356 secs 2023-09-10T01:44:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8675.860818 secs 2023-09-10T01:44:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:03 smithi098 ceph-mon[103124]: pgmap v4790: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T01:44:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8675.149356 secs 2023-09-10T01:44:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8675.860818 secs 2023-09-10T01:44:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:05 smithi170 ceph-mon[102832]: pgmap v4791: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T01:44:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:05 smithi082 ceph-mon[98694]: pgmap v4791: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T01:44:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:05 smithi098 ceph-mon[103124]: pgmap v4791: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T01:44:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:07 smithi170 ceph-mon[102832]: pgmap v4792: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 167 KiB/s wr, 31 op/s 2023-09-10T01:44:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8678.720680 secs 2023-09-10T01:44:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:07 smithi082 ceph-mon[98694]: pgmap v4792: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 167 KiB/s wr, 31 op/s 2023-09-10T01:44:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8678.720680 secs 2023-09-10T01:44:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:07 smithi098 ceph-mon[103124]: pgmap v4792: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 167 KiB/s wr, 31 op/s 2023-09-10T01:44:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8678.720680 secs 2023-09-10T01:44:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:08 smithi170 ceph-mon[102832]: osdmap e369: 12 total, 12 up, 12 in 2023-09-10T01:44:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8680.149418 secs 2023-09-10T01:44:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8680.860886 secs 2023-09-10T01:44:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:08 smithi082 ceph-mon[98694]: osdmap e369: 12 total, 12 up, 12 in 2023-09-10T01:44:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8680.149418 secs 2023-09-10T01:44:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8680.860886 secs 2023-09-10T01:44:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:08 smithi098 ceph-mon[103124]: osdmap e369: 12 total, 12 up, 12 in 2023-09-10T01:44:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8680.149418 secs 2023-09-10T01:44:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8680.860886 secs 2023-09-10T01:44:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:09 smithi170 ceph-mon[102832]: pgmap v4794: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 122 KiB/s wr, 30 op/s 2023-09-10T01:44:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:09 smithi082 ceph-mon[98694]: pgmap v4794: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 122 KiB/s wr, 30 op/s 2023-09-10T01:44:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:09 smithi098 ceph-mon[103124]: pgmap v4794: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 122 KiB/s wr, 30 op/s 2023-09-10T01:44:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:11 smithi170 ceph-mon[102832]: pgmap v4795: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T01:44:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:11 smithi082 ceph-mon[98694]: pgmap v4795: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T01:44:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:11 smithi098 ceph-mon[103124]: pgmap v4795: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T01:44:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8683.720784 secs 2023-09-10T01:44:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8683.720784 secs 2023-09-10T01:44:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8683.720784 secs 2023-09-10T01:44:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:13 smithi170 ceph-mon[102832]: pgmap v4796: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T01:44:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8685.149527 secs 2023-09-10T01:44:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8685.860951 secs 2023-09-10T01:44:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:13 smithi082 ceph-mon[98694]: pgmap v4796: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T01:44:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8685.149527 secs 2023-09-10T01:44:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8685.860951 secs 2023-09-10T01:44:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:13 smithi098 ceph-mon[103124]: pgmap v4796: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T01:44:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8685.149527 secs 2023-09-10T01:44:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8685.860951 secs 2023-09-10T01:44:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:15 smithi170 ceph-mon[102832]: pgmap v4797: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T01:44:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:15 smithi170 ceph-mon[102832]: osdmap e370: 12 total, 12 up, 12 in 2023-09-10T01:44:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:15 smithi082 ceph-mon[98694]: pgmap v4797: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T01:44:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:15 smithi082 ceph-mon[98694]: osdmap e370: 12 total, 12 up, 12 in 2023-09-10T01:44:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:15 smithi098 ceph-mon[103124]: pgmap v4797: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T01:44:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:15 smithi098 ceph-mon[103124]: osdmap e370: 12 total, 12 up, 12 in 2023-09-10T01:44:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:17 smithi170 ceph-mon[102832]: pgmap v4799: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 12 op/s 2023-09-10T01:44:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8688.720862 secs 2023-09-10T01:44:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:17 smithi082 ceph-mon[98694]: pgmap v4799: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 12 op/s 2023-09-10T01:44:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8688.720862 secs 2023-09-10T01:44:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:17 smithi098 ceph-mon[103124]: pgmap v4799: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 12 op/s 2023-09-10T01:44:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8688.720862 secs 2023-09-10T01:44:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8690.149584 secs 2023-09-10T01:44:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8690.861017 secs 2023-09-10T01:44:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8690.149584 secs 2023-09-10T01:44:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8690.861017 secs 2023-09-10T01:44:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8690.149584 secs 2023-09-10T01:44:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8690.861017 secs 2023-09-10T01:44:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:19 smithi170 ceph-mon[102832]: pgmap v4800: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-09-10T01:44:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:19 smithi082 ceph-mon[98694]: pgmap v4800: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-09-10T01:44:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:19 smithi098 ceph-mon[103124]: pgmap v4800: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-09-10T01:44:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:21 smithi170 ceph-mon[102832]: pgmap v4801: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T01:44:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:21 smithi082 ceph-mon[98694]: pgmap v4801: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T01:44:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:21 smithi098 ceph-mon[103124]: pgmap v4801: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T01:44:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:44:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:44:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:44:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:23 smithi170 ceph-mon[102832]: pgmap v4802: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T01:44:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8695.149641 secs 2023-09-10T01:44:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:44:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:44:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:44:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8695.861100 secs 2023-09-10T01:44:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:23 smithi082 ceph-mon[98694]: pgmap v4802: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T01:44:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8695.149641 secs 2023-09-10T01:44:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:44:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:44:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:44:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8695.861100 secs 2023-09-10T01:44:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:23 smithi098 ceph-mon[103124]: pgmap v4802: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T01:44:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8695.149641 secs 2023-09-10T01:44:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:44:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:44:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:44:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8695.861100 secs 2023-09-10T01:44:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:25 smithi170 ceph-mon[102832]: pgmap v4803: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T01:44:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:25 smithi082 ceph-mon[98694]: pgmap v4803: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T01:44:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:25 smithi098 ceph-mon[103124]: pgmap v4803: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T01:44:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:27 smithi170 ceph-mon[102832]: pgmap v4804: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T01:44:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:27 smithi082 ceph-mon[98694]: pgmap v4804: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T01:44:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:27 smithi098 ceph-mon[103124]: pgmap v4804: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T01:44:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8700.149687 secs 2023-09-10T01:44:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8700.861165 secs 2023-09-10T01:44:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8700.149687 secs 2023-09-10T01:44:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8700.861165 secs 2023-09-10T01:44:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8700.149687 secs 2023-09-10T01:44:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8700.861165 secs 2023-09-10T01:44:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:29 smithi082 ceph-mon[98694]: pgmap v4805: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 95 KiB/s wr, 10 op/s 2023-09-10T01:44:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:29 smithi098 ceph-mon[103124]: pgmap v4805: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 95 KiB/s wr, 10 op/s 2023-09-10T01:44:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:29 smithi170 ceph-mon[102832]: pgmap v4805: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 95 KiB/s wr, 10 op/s 2023-09-10T01:44:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:31 smithi170 ceph-mon[102832]: pgmap v4806: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 132 KiB/s wr, 17 op/s 2023-09-10T01:44:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:31 smithi082 ceph-mon[98694]: pgmap v4806: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 132 KiB/s wr, 17 op/s 2023-09-10T01:44:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:31 smithi098 ceph-mon[103124]: pgmap v4806: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 132 KiB/s wr, 17 op/s 2023-09-10T01:44:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:33 smithi170 ceph-mon[102832]: pgmap v4807: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T01:44:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8705.149745 secs 2023-09-10T01:44:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8705.861231 secs 2023-09-10T01:44:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:33 smithi082 ceph-mon[98694]: pgmap v4807: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T01:44:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8705.149745 secs 2023-09-10T01:44:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8705.861231 secs 2023-09-10T01:44:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:33 smithi098 ceph-mon[103124]: pgmap v4807: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T01:44:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8705.149745 secs 2023-09-10T01:44:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8705.861231 secs 2023-09-10T01:44:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:35 smithi170 ceph-mon[102832]: pgmap v4808: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T01:44:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:35 smithi082 ceph-mon[98694]: pgmap v4808: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T01:44:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:35 smithi098 ceph-mon[103124]: pgmap v4808: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T01:44:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:37 smithi170 ceph-mon[102832]: pgmap v4809: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 150 KiB/s wr, 18 op/s 2023-09-10T01:44:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:37 smithi082 ceph-mon[98694]: pgmap v4809: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 150 KiB/s wr, 18 op/s 2023-09-10T01:44:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:37 smithi098 ceph-mon[103124]: pgmap v4809: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 150 KiB/s wr, 18 op/s 2023-09-10T01:44:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8710.149806 secs 2023-09-10T01:44:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:44:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8710.861298 secs 2023-09-10T01:44:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8710.149806 secs 2023-09-10T01:44:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:44:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8710.861298 secs 2023-09-10T01:44:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8710.149806 secs 2023-09-10T01:44:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:44:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8710.861298 secs 2023-09-10T01:44:39.947 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:39 smithi098 ceph-mon[103124]: pgmap v4810: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 82 KiB/s wr, 13 op/s 2023-09-10T01:44:40.037 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:39 smithi170 ceph-mon[102832]: pgmap v4810: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 82 KiB/s wr, 13 op/s 2023-09-10T01:44:40.059 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:39 smithi082 ceph-mon[98694]: pgmap v4810: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 82 KiB/s wr, 13 op/s 2023-09-10T01:44:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:41 smithi170 ceph-mon[102832]: pgmap v4811: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 148 KiB/s wr, 19 op/s 2023-09-10T01:44:42.187 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:41 smithi098 ceph-mon[103124]: pgmap v4811: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 148 KiB/s wr, 19 op/s 2023-09-10T01:44:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:41 smithi082 ceph-mon[98694]: pgmap v4811: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 148 KiB/s wr, 19 op/s 2023-09-10T01:44:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:44:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:44:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:44:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:44:43.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:44:43.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:44:43.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:44:43.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:44:43.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:44:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:44:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:44:43.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:44:43.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:44:43.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:44:43.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:44:43.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:44:43.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:44:43.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:44:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:44:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:44:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:44:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:44:43.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:44:43.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:44:43.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:44:43.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:44:43.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:44:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:43 smithi170 ceph-mon[102832]: pgmap v4812: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T01:44:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8715.149875 secs 2023-09-10T01:44:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8715.861363 secs 2023-09-10T01:44:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:43 smithi082 ceph-mon[98694]: pgmap v4812: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T01:44:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8715.149875 secs 2023-09-10T01:44:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8715.861363 secs 2023-09-10T01:44:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:43 smithi098 ceph-mon[103124]: pgmap v4812: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T01:44:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8715.149875 secs 2023-09-10T01:44:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8715.861363 secs 2023-09-10T01:44:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:45 smithi170 ceph-mon[102832]: pgmap v4813: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T01:44:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:45 smithi082 ceph-mon[98694]: pgmap v4813: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T01:44:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:45 smithi098 ceph-mon[103124]: pgmap v4813: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T01:44:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:47 smithi170 ceph-mon[102832]: pgmap v4814: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 134 KiB/s wr, 17 op/s 2023-09-10T01:44:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8718.721283 secs 2023-09-10T01:44:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:47 smithi082 ceph-mon[98694]: pgmap v4814: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 134 KiB/s wr, 17 op/s 2023-09-10T01:44:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8718.721283 secs 2023-09-10T01:44:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:47 smithi098 ceph-mon[103124]: pgmap v4814: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 134 KiB/s wr, 17 op/s 2023-09-10T01:44:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8718.721283 secs 2023-09-10T01:44:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8720.149948 secs 2023-09-10T01:44:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8720.861427 secs 2023-09-10T01:44:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8720.149948 secs 2023-09-10T01:44:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8720.861427 secs 2023-09-10T01:44:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8720.149948 secs 2023-09-10T01:44:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8720.861427 secs 2023-09-10T01:44:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:49 smithi170 ceph-mon[102832]: pgmap v4815: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:44:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:49 smithi082 ceph-mon[98694]: pgmap v4815: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:44:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:49 smithi098 ceph-mon[103124]: pgmap v4815: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:44:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:51 smithi170 ceph-mon[102832]: pgmap v4816: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 102 KiB/s wr, 16 op/s 2023-09-10T01:44:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:44:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:44:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:44:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:51 smithi082 ceph-mon[98694]: pgmap v4816: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 102 KiB/s wr, 16 op/s 2023-09-10T01:44:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:44:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:44:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:44:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:51 smithi098 ceph-mon[103124]: pgmap v4816: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 102 KiB/s wr, 16 op/s 2023-09-10T01:44:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:44:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:44:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:44:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8723.721350 secs 2023-09-10T01:44:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8723.721350 secs 2023-09-10T01:44:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8723.721350 secs 2023-09-10T01:44:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:53 smithi170 ceph-mon[102832]: pgmap v4817: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 36 KiB/s wr, 10 op/s 2023-09-10T01:44:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8725.150006 secs 2023-09-10T01:44:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8725.861492 secs 2023-09-10T01:44:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:53 smithi082 ceph-mon[98694]: pgmap v4817: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 36 KiB/s wr, 10 op/s 2023-09-10T01:44:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8725.150006 secs 2023-09-10T01:44:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8725.861492 secs 2023-09-10T01:44:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:53 smithi098 ceph-mon[103124]: pgmap v4817: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 36 KiB/s wr, 10 op/s 2023-09-10T01:44:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8725.150006 secs 2023-09-10T01:44:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8725.861492 secs 2023-09-10T01:44:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:55 smithi170 ceph-mon[102832]: pgmap v4818: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 36 KiB/s wr, 10 op/s 2023-09-10T01:44:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:55 smithi082 ceph-mon[98694]: pgmap v4818: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 36 KiB/s wr, 10 op/s 2023-09-10T01:44:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:55 smithi098 ceph-mon[103124]: pgmap v4818: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 36 KiB/s wr, 10 op/s 2023-09-10T01:44:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:57 smithi170 ceph-mon[102832]: pgmap v4819: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 68 KiB/s wr, 15 op/s 2023-09-10T01:44:58.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8728.721473 secs 2023-09-10T01:44:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:57 smithi082 ceph-mon[98694]: pgmap v4819: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 68 KiB/s wr, 15 op/s 2023-09-10T01:44:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8728.721473 secs 2023-09-10T01:44:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:57 smithi098 ceph-mon[103124]: pgmap v4819: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 68 KiB/s wr, 15 op/s 2023-09-10T01:44:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8728.721473 secs 2023-09-10T01:44:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8730.150064 secs 2023-09-10T01:44:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8730.861555 secs 2023-09-10T01:44:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8730.150064 secs 2023-09-10T01:44:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8730.861555 secs 2023-09-10T01:44:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8730.150064 secs 2023-09-10T01:44:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8730.861555 secs 2023-09-10T01:45:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:44:59 smithi170 ceph-mon[102832]: pgmap v4820: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 45 KiB/s wr, 9 op/s 2023-09-10T01:45:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:44:59 smithi082 ceph-mon[98694]: pgmap v4820: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 45 KiB/s wr, 9 op/s 2023-09-10T01:45:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:44:59 smithi098 ceph-mon[103124]: pgmap v4820: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 45 KiB/s wr, 9 op/s 2023-09-10T01:45:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:01 smithi170 ceph-mon[102832]: pgmap v4821: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 64 KiB/s wr, 14 op/s 2023-09-10T01:45:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:01 smithi082 ceph-mon[98694]: pgmap v4821: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 64 KiB/s wr, 14 op/s 2023-09-10T01:45:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:01 smithi098 ceph-mon[103124]: pgmap v4821: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 64 KiB/s wr, 14 op/s 2023-09-10T01:45:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8733.721535 secs 2023-09-10T01:45:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8733.721535 secs 2023-09-10T01:45:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8733.721535 secs 2023-09-10T01:45:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:03 smithi170 ceph-mon[102832]: pgmap v4822: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 50 KiB/s wr, 9 op/s 2023-09-10T01:45:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8735.150121 secs 2023-09-10T01:45:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8735.861623 secs 2023-09-10T01:45:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:03 smithi082 ceph-mon[98694]: pgmap v4822: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 50 KiB/s wr, 9 op/s 2023-09-10T01:45:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8735.150121 secs 2023-09-10T01:45:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8735.861623 secs 2023-09-10T01:45:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:03 smithi098 ceph-mon[103124]: pgmap v4822: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 50 KiB/s wr, 9 op/s 2023-09-10T01:45:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8735.150121 secs 2023-09-10T01:45:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8735.861623 secs 2023-09-10T01:45:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:05 smithi170 ceph-mon[102832]: pgmap v4823: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 50 KiB/s wr, 9 op/s 2023-09-10T01:45:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:05 smithi082 ceph-mon[98694]: pgmap v4823: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 50 KiB/s wr, 9 op/s 2023-09-10T01:45:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:05 smithi098 ceph-mon[103124]: pgmap v4823: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 50 KiB/s wr, 9 op/s 2023-09-10T01:45:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:07 smithi170 ceph-mon[102832]: pgmap v4824: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T01:45:08.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8738.721596 secs 2023-09-10T01:45:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:07 smithi082 ceph-mon[98694]: pgmap v4824: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T01:45:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8738.721596 secs 2023-09-10T01:45:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:07 smithi098 ceph-mon[103124]: pgmap v4824: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T01:45:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8738.721596 secs 2023-09-10T01:45:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8740.150178 secs 2023-09-10T01:45:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:08 smithi170 ceph-mon[102832]: osdmap e371: 12 total, 12 up, 12 in 2023-09-10T01:45:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8740.861688 secs 2023-09-10T01:45:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8740.150178 secs 2023-09-10T01:45:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:08 smithi082 ceph-mon[98694]: osdmap e371: 12 total, 12 up, 12 in 2023-09-10T01:45:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8740.861688 secs 2023-09-10T01:45:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8740.150178 secs 2023-09-10T01:45:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:08 smithi098 ceph-mon[103124]: osdmap e371: 12 total, 12 up, 12 in 2023-09-10T01:45:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8740.861688 secs 2023-09-10T01:45:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:09 smithi170 ceph-mon[102832]: pgmap v4826: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T01:45:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:09 smithi082 ceph-mon[98694]: pgmap v4826: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T01:45:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:09 smithi098 ceph-mon[103124]: pgmap v4826: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T01:45:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:11 smithi170 ceph-mon[102832]: pgmap v4827: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-10T01:45:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:11 smithi082 ceph-mon[98694]: pgmap v4827: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-10T01:45:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:11 smithi098 ceph-mon[103124]: pgmap v4827: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-10T01:45:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8743.721657 secs 2023-09-10T01:45:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8743.721657 secs 2023-09-10T01:45:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8743.721657 secs 2023-09-10T01:45:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:13 smithi170 ceph-mon[102832]: pgmap v4828: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-10T01:45:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8745.150236 secs 2023-09-10T01:45:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8745.861756 secs 2023-09-10T01:45:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:13 smithi082 ceph-mon[98694]: pgmap v4828: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-10T01:45:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8745.150236 secs 2023-09-10T01:45:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8745.861756 secs 2023-09-10T01:45:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:13 smithi098 ceph-mon[103124]: pgmap v4828: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-10T01:45:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8745.150236 secs 2023-09-10T01:45:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8745.861756 secs 2023-09-10T01:45:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:15 smithi170 ceph-mon[102832]: pgmap v4829: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-10T01:45:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:15 smithi170 ceph-mon[102832]: osdmap e372: 12 total, 12 up, 12 in 2023-09-10T01:45:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:15 smithi082 ceph-mon[98694]: pgmap v4829: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-10T01:45:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:15 smithi082 ceph-mon[98694]: osdmap e372: 12 total, 12 up, 12 in 2023-09-10T01:45:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:15 smithi098 ceph-mon[103124]: pgmap v4829: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-10T01:45:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:15 smithi098 ceph-mon[103124]: osdmap e372: 12 total, 12 up, 12 in 2023-09-10T01:45:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:17 smithi170 ceph-mon[102832]: pgmap v4831: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 42 KiB/s wr, 13 op/s 2023-09-10T01:45:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8748.721728 secs 2023-09-10T01:45:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:17 smithi082 ceph-mon[98694]: pgmap v4831: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 42 KiB/s wr, 13 op/s 2023-09-10T01:45:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8748.721728 secs 2023-09-10T01:45:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:17 smithi098 ceph-mon[103124]: pgmap v4831: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 42 KiB/s wr, 13 op/s 2023-09-10T01:45:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8748.721728 secs 2023-09-10T01:45:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8750.150294 secs 2023-09-10T01:45:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8750.861820 secs 2023-09-10T01:45:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8750.150294 secs 2023-09-10T01:45:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8750.861820 secs 2023-09-10T01:45:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8750.150294 secs 2023-09-10T01:45:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8750.861820 secs 2023-09-10T01:45:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:19 smithi170 ceph-mon[102832]: pgmap v4832: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T01:45:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:19 smithi082 ceph-mon[98694]: pgmap v4832: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T01:45:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:19 smithi098 ceph-mon[103124]: pgmap v4832: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T01:45:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:21 smithi170 ceph-mon[102832]: pgmap v4833: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T01:45:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:21 smithi082 ceph-mon[98694]: pgmap v4833: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T01:45:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:21 smithi098 ceph-mon[103124]: pgmap v4833: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T01:45:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:45:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:45:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:45:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:23 smithi170 ceph-mon[102832]: pgmap v4834: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T01:45:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8755.150354 secs 2023-09-10T01:45:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:45:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:45:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:45:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8755.861886 secs 2023-09-10T01:45:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:23 smithi082 ceph-mon[98694]: pgmap v4834: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T01:45:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8755.150354 secs 2023-09-10T01:45:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:45:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:45:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:45:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8755.861886 secs 2023-09-10T01:45:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:23 smithi098 ceph-mon[103124]: pgmap v4834: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T01:45:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8755.150354 secs 2023-09-10T01:45:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:45:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:45:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:45:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8755.861886 secs 2023-09-10T01:45:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:25 smithi170 ceph-mon[102832]: pgmap v4835: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T01:45:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:25 smithi082 ceph-mon[98694]: pgmap v4835: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T01:45:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:25 smithi098 ceph-mon[103124]: pgmap v4835: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T01:45:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:27 smithi170 ceph-mon[102832]: pgmap v4836: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-10T01:45:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:27 smithi082 ceph-mon[98694]: pgmap v4836: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-10T01:45:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:27 smithi098 ceph-mon[103124]: pgmap v4836: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-10T01:45:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8760.150428 secs 2023-09-10T01:45:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8760.861949 secs 2023-09-10T01:45:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8760.150428 secs 2023-09-10T01:45:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8760.861949 secs 2023-09-10T01:45:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8760.150428 secs 2023-09-10T01:45:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8760.861949 secs 2023-09-10T01:45:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:29 smithi170 ceph-mon[102832]: pgmap v4837: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T01:45:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:29 smithi082 ceph-mon[98694]: pgmap v4837: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T01:45:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:29 smithi098 ceph-mon[103124]: pgmap v4837: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T01:45:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:31 smithi170 ceph-mon[102832]: pgmap v4838: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 94 KiB/s wr, 16 op/s 2023-09-10T01:45:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:31 smithi082 ceph-mon[98694]: pgmap v4838: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 94 KiB/s wr, 16 op/s 2023-09-10T01:45:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:31 smithi098 ceph-mon[103124]: pgmap v4838: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 94 KiB/s wr, 16 op/s 2023-09-10T01:45:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:33 smithi170 ceph-mon[102832]: pgmap v4839: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T01:45:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8765.150492 secs 2023-09-10T01:45:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8765.862015 secs 2023-09-10T01:45:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:33 smithi082 ceph-mon[98694]: pgmap v4839: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T01:45:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8765.150492 secs 2023-09-10T01:45:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8765.862015 secs 2023-09-10T01:45:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:33 smithi098 ceph-mon[103124]: pgmap v4839: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T01:45:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8765.150492 secs 2023-09-10T01:45:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8765.862015 secs 2023-09-10T01:45:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:35 smithi170 ceph-mon[102832]: pgmap v4840: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T01:45:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:35 smithi082 ceph-mon[98694]: pgmap v4840: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T01:45:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:35 smithi098 ceph-mon[103124]: pgmap v4840: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T01:45:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:37 smithi170 ceph-mon[102832]: pgmap v4841: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 145 KiB/s wr, 18 op/s 2023-09-10T01:45:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:37 smithi082 ceph-mon[98694]: pgmap v4841: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 145 KiB/s wr, 18 op/s 2023-09-10T01:45:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:37 smithi098 ceph-mon[103124]: pgmap v4841: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 145 KiB/s wr, 18 op/s 2023-09-10T01:45:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8770.150551 secs 2023-09-10T01:45:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8770.862081 secs 2023-09-10T01:45:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8770.150551 secs 2023-09-10T01:45:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8770.862081 secs 2023-09-10T01:45:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8770.150551 secs 2023-09-10T01:45:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8770.862081 secs 2023-09-10T01:45:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:39 smithi082 ceph-mon[98694]: pgmap v4842: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T01:45:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:39 smithi098 ceph-mon[103124]: pgmap v4842: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T01:45:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:39 smithi170 ceph-mon[102832]: pgmap v4842: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T01:45:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:41 smithi082 ceph-mon[98694]: pgmap v4843: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 169 KiB/s wr, 19 op/s 2023-09-10T01:45:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:41 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:45:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:41 smithi098 ceph-mon[103124]: pgmap v4843: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 169 KiB/s wr, 19 op/s 2023-09-10T01:45:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:41 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:45:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:41 smithi170 ceph-mon[102832]: pgmap v4843: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 169 KiB/s wr, 19 op/s 2023-09-10T01:45:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:41 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:45:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8773.722092 secs 2023-09-10T01:45:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:45:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8773.722092 secs 2023-09-10T01:45:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:45:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8773.722092 secs 2023-09-10T01:45:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:45:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:43 smithi082 ceph-mon[98694]: pgmap v4844: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 124 KiB/s wr, 13 op/s 2023-09-10T01:45:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8775.150609 secs 2023-09-10T01:45:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:45:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:45:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:45:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:45:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:45:44.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:45:44.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8775.862147 secs 2023-09-10T01:45:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:43 smithi098 ceph-mon[103124]: pgmap v4844: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 124 KiB/s wr, 13 op/s 2023-09-10T01:45:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8775.150609 secs 2023-09-10T01:45:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:45:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:45:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:45:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:45:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:45:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:45:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8775.862147 secs 2023-09-10T01:45:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:43 smithi170 ceph-mon[102832]: pgmap v4844: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 124 KiB/s wr, 13 op/s 2023-09-10T01:45:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8775.150609 secs 2023-09-10T01:45:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:45:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:45:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:45:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:45:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:45:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:45:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8775.862147 secs 2023-09-10T01:45:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:45 smithi082 ceph-mon[98694]: pgmap v4845: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 124 KiB/s wr, 13 op/s 2023-09-10T01:45:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:45 smithi098 ceph-mon[103124]: pgmap v4845: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 124 KiB/s wr, 13 op/s 2023-09-10T01:45:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:45 smithi170 ceph-mon[102832]: pgmap v4845: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 124 KiB/s wr, 13 op/s 2023-09-10T01:45:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:47 smithi082 ceph-mon[98694]: pgmap v4846: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 132 KiB/s wr, 18 op/s 2023-09-10T01:45:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8778.722151 secs 2023-09-10T01:45:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:47 smithi098 ceph-mon[103124]: pgmap v4846: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 132 KiB/s wr, 18 op/s 2023-09-10T01:45:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8778.722151 secs 2023-09-10T01:45:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:47 smithi170 ceph-mon[102832]: pgmap v4846: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 132 KiB/s wr, 18 op/s 2023-09-10T01:45:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8778.722151 secs 2023-09-10T01:45:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8780.150669 secs 2023-09-10T01:45:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8780.862210 secs 2023-09-10T01:45:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8780.150669 secs 2023-09-10T01:45:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8780.862210 secs 2023-09-10T01:45:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8780.150669 secs 2023-09-10T01:45:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8780.862210 secs 2023-09-10T01:45:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:49 smithi082 ceph-mon[98694]: pgmap v4847: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T01:45:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:49 smithi098 ceph-mon[103124]: pgmap v4847: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T01:45:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:49 smithi170 ceph-mon[102832]: pgmap v4847: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T01:45:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:51 smithi170 ceph-mon[102832]: pgmap v4848: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 104 KiB/s wr, 17 op/s 2023-09-10T01:45:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:51 smithi082 ceph-mon[98694]: pgmap v4848: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 104 KiB/s wr, 17 op/s 2023-09-10T01:45:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:51 smithi098 ceph-mon[103124]: pgmap v4848: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 104 KiB/s wr, 17 op/s 2023-09-10T01:45:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8783.722233 secs 2023-09-10T01:45:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8783.722233 secs 2023-09-10T01:45:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8783.722233 secs 2023-09-10T01:45:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:53 smithi170 ceph-mon[102832]: pgmap v4849: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T01:45:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8785.150729 secs 2023-09-10T01:45:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8785.862277 secs 2023-09-10T01:45:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:53 smithi082 ceph-mon[98694]: pgmap v4849: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T01:45:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8785.150729 secs 2023-09-10T01:45:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8785.862277 secs 2023-09-10T01:45:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:53 smithi098 ceph-mon[103124]: pgmap v4849: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T01:45:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8785.150729 secs 2023-09-10T01:45:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8785.862277 secs 2023-09-10T01:45:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:55 smithi170 ceph-mon[102832]: pgmap v4850: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T01:45:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:55 smithi082 ceph-mon[98694]: pgmap v4850: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T01:45:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:55 smithi098 ceph-mon[103124]: pgmap v4850: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T01:45:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:57 smithi170 ceph-mon[102832]: pgmap v4851: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 53 KiB/s wr, 15 op/s 2023-09-10T01:45:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8788.722340 secs 2023-09-10T01:45:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:57 smithi082 ceph-mon[98694]: pgmap v4851: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 53 KiB/s wr, 15 op/s 2023-09-10T01:45:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8788.722340 secs 2023-09-10T01:45:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:57 smithi098 ceph-mon[103124]: pgmap v4851: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 53 KiB/s wr, 15 op/s 2023-09-10T01:45:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8788.722340 secs 2023-09-10T01:45:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8790.150787 secs 2023-09-10T01:45:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8790.862345 secs 2023-09-10T01:45:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8790.150787 secs 2023-09-10T01:45:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8790.862345 secs 2023-09-10T01:45:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8790.150787 secs 2023-09-10T01:45:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8790.862345 secs 2023-09-10T01:46:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:45:59 smithi170 ceph-mon[102832]: pgmap v4852: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T01:46:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:45:59 smithi082 ceph-mon[98694]: pgmap v4852: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T01:46:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:45:59 smithi098 ceph-mon[103124]: pgmap v4852: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T01:46:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:01 smithi170 ceph-mon[102832]: pgmap v4853: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 15 op/s 2023-09-10T01:46:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:01 smithi082 ceph-mon[98694]: pgmap v4853: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 15 op/s 2023-09-10T01:46:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:01 smithi098 ceph-mon[103124]: pgmap v4853: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 15 op/s 2023-09-10T01:46:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8793.722417 secs 2023-09-10T01:46:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8793.722417 secs 2023-09-10T01:46:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8793.722417 secs 2023-09-10T01:46:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:03 smithi170 ceph-mon[102832]: pgmap v4854: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-09-10T01:46:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8795.150844 secs 2023-09-10T01:46:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8795.862409 secs 2023-09-10T01:46:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:03 smithi082 ceph-mon[98694]: pgmap v4854: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-09-10T01:46:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8795.150844 secs 2023-09-10T01:46:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8795.862409 secs 2023-09-10T01:46:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:03 smithi098 ceph-mon[103124]: pgmap v4854: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-09-10T01:46:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8795.150844 secs 2023-09-10T01:46:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8795.862409 secs 2023-09-10T01:46:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:05 smithi170 ceph-mon[102832]: pgmap v4855: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-09-10T01:46:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:05 smithi082 ceph-mon[98694]: pgmap v4855: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-09-10T01:46:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:05 smithi098 ceph-mon[103124]: pgmap v4855: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-09-10T01:46:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:07 smithi170 ceph-mon[102832]: pgmap v4856: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T01:46:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8798.722480 secs 2023-09-10T01:46:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:07 smithi082 ceph-mon[98694]: pgmap v4856: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T01:46:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8798.722480 secs 2023-09-10T01:46:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:07 smithi098 ceph-mon[103124]: pgmap v4856: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T01:46:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8798.722480 secs 2023-09-10T01:46:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8800.150909 secs 2023-09-10T01:46:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:08 smithi082 ceph-mon[98694]: osdmap e373: 12 total, 12 up, 12 in 2023-09-10T01:46:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8800.862479 secs 2023-09-10T01:46:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8800.150909 secs 2023-09-10T01:46:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:08 smithi098 ceph-mon[103124]: osdmap e373: 12 total, 12 up, 12 in 2023-09-10T01:46:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8800.862479 secs 2023-09-10T01:46:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8800.150909 secs 2023-09-10T01:46:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:08 smithi170 ceph-mon[102832]: osdmap e373: 12 total, 12 up, 12 in 2023-09-10T01:46:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8800.862479 secs 2023-09-10T01:46:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:09 smithi082 ceph-mon[98694]: pgmap v4858: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T01:46:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:09 smithi098 ceph-mon[103124]: pgmap v4858: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T01:46:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:09 smithi170 ceph-mon[102832]: pgmap v4858: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T01:46:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:11 smithi082 ceph-mon[98694]: pgmap v4859: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 29 op/s 2023-09-10T01:46:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:11 smithi098 ceph-mon[103124]: pgmap v4859: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 29 op/s 2023-09-10T01:46:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:11 smithi170 ceph-mon[102832]: pgmap v4859: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 29 op/s 2023-09-10T01:46:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8803.722540 secs 2023-09-10T01:46:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8803.722540 secs 2023-09-10T01:46:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8803.722540 secs 2023-09-10T01:46:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:13 smithi082 ceph-mon[98694]: pgmap v4860: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 29 op/s 2023-09-10T01:46:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8805.150997 secs 2023-09-10T01:46:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8805.862545 secs 2023-09-10T01:46:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:13 smithi098 ceph-mon[103124]: pgmap v4860: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 29 op/s 2023-09-10T01:46:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8805.150997 secs 2023-09-10T01:46:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8805.862545 secs 2023-09-10T01:46:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:13 smithi170 ceph-mon[102832]: pgmap v4860: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 29 op/s 2023-09-10T01:46:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8805.150997 secs 2023-09-10T01:46:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8805.862545 secs 2023-09-10T01:46:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:15 smithi082 ceph-mon[98694]: pgmap v4861: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 29 op/s 2023-09-10T01:46:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:15 smithi082 ceph-mon[98694]: osdmap e374: 12 total, 12 up, 12 in 2023-09-10T01:46:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:15 smithi098 ceph-mon[103124]: pgmap v4861: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 29 op/s 2023-09-10T01:46:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:15 smithi098 ceph-mon[103124]: osdmap e374: 12 total, 12 up, 12 in 2023-09-10T01:46:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:15 smithi170 ceph-mon[102832]: pgmap v4861: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 29 op/s 2023-09-10T01:46:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:15 smithi170 ceph-mon[102832]: osdmap e374: 12 total, 12 up, 12 in 2023-09-10T01:46:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:17 smithi082 ceph-mon[98694]: pgmap v4863: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-10T01:46:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8808.722602 secs 2023-09-10T01:46:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:17 smithi098 ceph-mon[103124]: pgmap v4863: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-10T01:46:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8808.722602 secs 2023-09-10T01:46:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:17 smithi170 ceph-mon[102832]: pgmap v4863: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-10T01:46:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8808.722602 secs 2023-09-10T01:46:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8810.151101 secs 2023-09-10T01:46:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8810.862609 secs 2023-09-10T01:46:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8810.151101 secs 2023-09-10T01:46:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8810.862609 secs 2023-09-10T01:46:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8810.151101 secs 2023-09-10T01:46:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8810.862609 secs 2023-09-10T01:46:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:19 smithi098 ceph-mon[103124]: pgmap v4864: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T01:46:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:19 smithi170 ceph-mon[102832]: pgmap v4864: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T01:46:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:19 smithi082 ceph-mon[98694]: pgmap v4864: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T01:46:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:21 smithi170 ceph-mon[102832]: pgmap v4865: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T01:46:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:21 smithi082 ceph-mon[98694]: pgmap v4865: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T01:46:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:21 smithi098 ceph-mon[103124]: pgmap v4865: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T01:46:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:46:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:46:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:46:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8815.151157 secs 2023-09-10T01:46:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:23 smithi170 ceph-mon[102832]: pgmap v4866: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T01:46:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:46:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:46:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:46:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8815.862675 secs 2023-09-10T01:46:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8815.151157 secs 2023-09-10T01:46:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:23 smithi082 ceph-mon[98694]: pgmap v4866: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T01:46:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:46:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:46:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:46:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8815.862675 secs 2023-09-10T01:46:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8815.151157 secs 2023-09-10T01:46:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:23 smithi098 ceph-mon[103124]: pgmap v4866: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T01:46:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:46:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:46:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:46:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8815.862675 secs 2023-09-10T01:46:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:25 smithi170 ceph-mon[102832]: pgmap v4867: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T01:46:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:25 smithi082 ceph-mon[98694]: pgmap v4867: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T01:46:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:25 smithi098 ceph-mon[103124]: pgmap v4867: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T01:46:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:27 smithi170 ceph-mon[102832]: pgmap v4868: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 18 KiB/s wr, 10 op/s 2023-09-10T01:46:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:27 smithi082 ceph-mon[98694]: pgmap v4868: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 18 KiB/s wr, 10 op/s 2023-09-10T01:46:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:27 smithi098 ceph-mon[103124]: pgmap v4868: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 18 KiB/s wr, 10 op/s 2023-09-10T01:46:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8820.151244 secs 2023-09-10T01:46:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8820.862738 secs 2023-09-10T01:46:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8820.151244 secs 2023-09-10T01:46:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8820.862738 secs 2023-09-10T01:46:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8820.151244 secs 2023-09-10T01:46:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8820.862738 secs 2023-09-10T01:46:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:29 smithi170 ceph-mon[102832]: pgmap v4869: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 17 KiB/s wr, 9 op/s 2023-09-10T01:46:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:29 smithi082 ceph-mon[98694]: pgmap v4869: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 17 KiB/s wr, 9 op/s 2023-09-10T01:46:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:29 smithi098 ceph-mon[103124]: pgmap v4869: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 17 KiB/s wr, 9 op/s 2023-09-10T01:46:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:31 smithi170 ceph-mon[102832]: pgmap v4870: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 54 KiB/s wr, 14 op/s 2023-09-10T01:46:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:31 smithi082 ceph-mon[98694]: pgmap v4870: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 54 KiB/s wr, 14 op/s 2023-09-10T01:46:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:31 smithi098 ceph-mon[103124]: pgmap v4870: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 54 KiB/s wr, 14 op/s 2023-09-10T01:46:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8825.151302 secs 2023-09-10T01:46:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:33 smithi170 ceph-mon[102832]: pgmap v4871: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 54 KiB/s wr, 9 op/s 2023-09-10T01:46:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8825.862803 secs 2023-09-10T01:46:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8825.151302 secs 2023-09-10T01:46:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:33 smithi082 ceph-mon[98694]: pgmap v4871: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 54 KiB/s wr, 9 op/s 2023-09-10T01:46:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8825.862803 secs 2023-09-10T01:46:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8825.151302 secs 2023-09-10T01:46:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:33 smithi098 ceph-mon[103124]: pgmap v4871: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 54 KiB/s wr, 9 op/s 2023-09-10T01:46:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8825.862803 secs 2023-09-10T01:46:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:35 smithi170 ceph-mon[102832]: pgmap v4872: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 54 KiB/s wr, 9 op/s 2023-09-10T01:46:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:35 smithi082 ceph-mon[98694]: pgmap v4872: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 54 KiB/s wr, 9 op/s 2023-09-10T01:46:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:35 smithi098 ceph-mon[103124]: pgmap v4872: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 54 KiB/s wr, 9 op/s 2023-09-10T01:46:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:37 smithi082 ceph-mon[98694]: pgmap v4873: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 108 KiB/s wr, 15 op/s 2023-09-10T01:46:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:37 smithi098 ceph-mon[103124]: pgmap v4873: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 108 KiB/s wr, 15 op/s 2023-09-10T01:46:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:37 smithi170 ceph-mon[102832]: pgmap v4873: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 108 KiB/s wr, 15 op/s 2023-09-10T01:46:39.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8830.151359 secs 2023-09-10T01:46:39.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8830.862873 secs 2023-09-10T01:46:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8830.151359 secs 2023-09-10T01:46:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8830.862873 secs 2023-09-10T01:46:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8830.151359 secs 2023-09-10T01:46:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8830.862873 secs 2023-09-10T01:46:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:39 smithi082 ceph-mon[98694]: pgmap v4874: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T01:46:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:39 smithi098 ceph-mon[103124]: pgmap v4874: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T01:46:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:39 smithi170 ceph-mon[102832]: pgmap v4874: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T01:46:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:41 smithi082 ceph-mon[98694]: pgmap v4875: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 117 KiB/s wr, 17 op/s 2023-09-10T01:46:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:41 smithi098 ceph-mon[103124]: pgmap v4875: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 117 KiB/s wr, 17 op/s 2023-09-10T01:46:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:41 smithi170 ceph-mon[102832]: pgmap v4875: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 117 KiB/s wr, 17 op/s 2023-09-10T01:46:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8835.151414 secs 2023-09-10T01:46:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:43 smithi082 ceph-mon[98694]: pgmap v4876: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T01:46:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:46:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8835.863112 secs 2023-09-10T01:46:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8835.151414 secs 2023-09-10T01:46:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:43 smithi098 ceph-mon[103124]: pgmap v4876: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T01:46:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:46:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8835.863112 secs 2023-09-10T01:46:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8835.151414 secs 2023-09-10T01:46:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:43 smithi170 ceph-mon[102832]: pgmap v4876: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T01:46:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:46:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8835.863112 secs 2023-09-10T01:46:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:46:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:46:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:46:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:46:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:46:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:46:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:46:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:46:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:46:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:45 smithi082 ceph-mon[98694]: pgmap v4877: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T01:46:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:45 smithi098 ceph-mon[103124]: pgmap v4877: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T01:46:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:45 smithi170 ceph-mon[102832]: pgmap v4877: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T01:46:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:47 smithi082 ceph-mon[98694]: pgmap v4878: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 127 KiB/s wr, 17 op/s 2023-09-10T01:46:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:47 smithi098 ceph-mon[103124]: pgmap v4878: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 127 KiB/s wr, 17 op/s 2023-09-10T01:46:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:47 smithi170 ceph-mon[102832]: pgmap v4878: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 127 KiB/s wr, 17 op/s 2023-09-10T01:46:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8840.151467 secs 2023-09-10T01:46:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8840.863176 secs 2023-09-10T01:46:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8840.151467 secs 2023-09-10T01:46:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8840.863176 secs 2023-09-10T01:46:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8840.151467 secs 2023-09-10T01:46:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8840.863176 secs 2023-09-10T01:46:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:49 smithi082 ceph-mon[98694]: pgmap v4879: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T01:46:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:49 smithi098 ceph-mon[103124]: pgmap v4879: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T01:46:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:49 smithi170 ceph-mon[102832]: pgmap v4879: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T01:46:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:51 smithi082 ceph-mon[98694]: pgmap v4880: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 129 KiB/s wr, 17 op/s 2023-09-10T01:46:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:51 smithi098 ceph-mon[103124]: pgmap v4880: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 129 KiB/s wr, 17 op/s 2023-09-10T01:46:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:51 smithi170 ceph-mon[102832]: pgmap v4880: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 129 KiB/s wr, 17 op/s 2023-09-10T01:46:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8843.723036 secs 2023-09-10T01:46:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8843.723036 secs 2023-09-10T01:46:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8843.723036 secs 2023-09-10T01:46:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8845.151525 secs 2023-09-10T01:46:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:53 smithi098 ceph-mon[103124]: pgmap v4881: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T01:46:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8845.863238 secs 2023-09-10T01:46:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8845.151525 secs 2023-09-10T01:46:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:53 smithi170 ceph-mon[102832]: pgmap v4881: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T01:46:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8845.863238 secs 2023-09-10T01:46:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8845.151525 secs 2023-09-10T01:46:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:53 smithi082 ceph-mon[98694]: pgmap v4881: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T01:46:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8845.863238 secs 2023-09-10T01:46:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:54 smithi098 ceph-mon[103124]: pgmap v4882: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T01:46:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:54 smithi170 ceph-mon[102832]: pgmap v4882: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T01:46:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:54 smithi082 ceph-mon[98694]: pgmap v4882: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T01:46:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:46:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:46:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:46:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:46:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:46:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:46:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:46:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:46:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:46:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:56 smithi098 ceph-mon[103124]: pgmap v4883: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 141 KiB/s wr, 17 op/s 2023-09-10T01:46:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:56 smithi170 ceph-mon[102832]: pgmap v4883: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 141 KiB/s wr, 17 op/s 2023-09-10T01:46:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:56 smithi082 ceph-mon[98694]: pgmap v4883: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 141 KiB/s wr, 17 op/s 2023-09-10T01:46:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8848.723145 secs 2023-09-10T01:46:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8848.723145 secs 2023-09-10T01:46:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8848.723145 secs 2023-09-10T01:46:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8850.151583 secs 2023-09-10T01:46:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8850.863304 secs 2023-09-10T01:46:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:46:58 smithi098 ceph-mon[103124]: pgmap v4884: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:46:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8850.151583 secs 2023-09-10T01:46:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8850.863304 secs 2023-09-10T01:46:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:46:58 smithi170 ceph-mon[102832]: pgmap v4884: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:46:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8850.151583 secs 2023-09-10T01:46:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8850.863304 secs 2023-09-10T01:46:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:46:58 smithi082 ceph-mon[98694]: pgmap v4884: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:47:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:01 smithi098 ceph-mon[103124]: pgmap v4885: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 159 KiB/s wr, 31 op/s 2023-09-10T01:47:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:01 smithi170 ceph-mon[102832]: pgmap v4885: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 159 KiB/s wr, 31 op/s 2023-09-10T01:47:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:01 smithi082 ceph-mon[98694]: pgmap v4885: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 159 KiB/s wr, 31 op/s 2023-09-10T01:47:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8853.723205 secs 2023-09-10T01:47:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8853.723205 secs 2023-09-10T01:47:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8853.723205 secs 2023-09-10T01:47:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8855.151636 secs 2023-09-10T01:47:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:03 smithi098 ceph-mon[103124]: pgmap v4886: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 103 KiB/s wr, 25 op/s 2023-09-10T01:47:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8855.863370 secs 2023-09-10T01:47:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8855.151636 secs 2023-09-10T01:47:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:03 smithi170 ceph-mon[102832]: pgmap v4886: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 103 KiB/s wr, 25 op/s 2023-09-10T01:47:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8855.863370 secs 2023-09-10T01:47:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8855.151636 secs 2023-09-10T01:47:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:03 smithi082 ceph-mon[98694]: pgmap v4886: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 103 KiB/s wr, 25 op/s 2023-09-10T01:47:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8855.863370 secs 2023-09-10T01:47:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:04 smithi098 ceph-mon[103124]: pgmap v4887: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 103 KiB/s wr, 25 op/s 2023-09-10T01:47:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:04 smithi170 ceph-mon[102832]: pgmap v4887: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 103 KiB/s wr, 25 op/s 2023-09-10T01:47:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:04 smithi082 ceph-mon[98694]: pgmap v4887: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 103 KiB/s wr, 25 op/s 2023-09-10T01:47:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:07 smithi170 ceph-mon[102832]: pgmap v4888: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 116 KiB/s wr, 31 op/s 2023-09-10T01:47:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8858.723264 secs 2023-09-10T01:47:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:07 smithi082 ceph-mon[98694]: pgmap v4888: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 116 KiB/s wr, 31 op/s 2023-09-10T01:47:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8858.723264 secs 2023-09-10T01:47:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:07 smithi098 ceph-mon[103124]: pgmap v4888: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 116 KiB/s wr, 31 op/s 2023-09-10T01:47:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8858.723264 secs 2023-09-10T01:47:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8860.151689 secs 2023-09-10T01:47:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:08 smithi170 ceph-mon[102832]: osdmap e375: 12 total, 12 up, 12 in 2023-09-10T01:47:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8860.863432 secs 2023-09-10T01:47:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8860.151689 secs 2023-09-10T01:47:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:08 smithi082 ceph-mon[98694]: osdmap e375: 12 total, 12 up, 12 in 2023-09-10T01:47:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8860.863432 secs 2023-09-10T01:47:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8860.151689 secs 2023-09-10T01:47:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:08 smithi098 ceph-mon[103124]: osdmap e375: 12 total, 12 up, 12 in 2023-09-10T01:47:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8860.863432 secs 2023-09-10T01:47:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:09 smithi170 ceph-mon[102832]: pgmap v4890: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 94 KiB/s wr, 30 op/s 2023-09-10T01:47:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:09 smithi082 ceph-mon[98694]: pgmap v4890: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 94 KiB/s wr, 30 op/s 2023-09-10T01:47:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:09 smithi098 ceph-mon[103124]: pgmap v4890: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 94 KiB/s wr, 30 op/s 2023-09-10T01:47:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:11 smithi170 ceph-mon[102832]: pgmap v4891: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 39 KiB/s wr, 13 op/s 2023-09-10T01:47:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:11 smithi082 ceph-mon[98694]: pgmap v4891: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 39 KiB/s wr, 13 op/s 2023-09-10T01:47:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:11 smithi098 ceph-mon[103124]: pgmap v4891: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 39 KiB/s wr, 13 op/s 2023-09-10T01:47:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8863.723340 secs 2023-09-10T01:47:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8863.723340 secs 2023-09-10T01:47:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8863.723340 secs 2023-09-10T01:47:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8865.151750 secs 2023-09-10T01:47:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:13 smithi082 ceph-mon[98694]: pgmap v4892: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 39 KiB/s wr, 13 op/s 2023-09-10T01:47:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8865.863484 secs 2023-09-10T01:47:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8865.151750 secs 2023-09-10T01:47:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:13 smithi098 ceph-mon[103124]: pgmap v4892: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 39 KiB/s wr, 13 op/s 2023-09-10T01:47:14.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8865.863484 secs 2023-09-10T01:47:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8865.151750 secs 2023-09-10T01:47:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:13 smithi170 ceph-mon[102832]: pgmap v4892: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 39 KiB/s wr, 13 op/s 2023-09-10T01:47:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8865.863484 secs 2023-09-10T01:47:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:15 smithi082 ceph-mon[98694]: pgmap v4893: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 39 KiB/s wr, 13 op/s 2023-09-10T01:47:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:15 smithi082 ceph-mon[98694]: osdmap e376: 12 total, 12 up, 12 in 2023-09-10T01:47:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:15 smithi098 ceph-mon[103124]: pgmap v4893: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 39 KiB/s wr, 13 op/s 2023-09-10T01:47:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:15 smithi098 ceph-mon[103124]: osdmap e376: 12 total, 12 up, 12 in 2023-09-10T01:47:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:15 smithi170 ceph-mon[102832]: pgmap v4893: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 39 KiB/s wr, 13 op/s 2023-09-10T01:47:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:15 smithi170 ceph-mon[102832]: osdmap e376: 12 total, 12 up, 12 in 2023-09-10T01:47:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:17 smithi082 ceph-mon[98694]: pgmap v4895: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 13 op/s 2023-09-10T01:47:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8868.723400 secs 2023-09-10T01:47:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:17 smithi098 ceph-mon[103124]: pgmap v4895: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 13 op/s 2023-09-10T01:47:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8868.723400 secs 2023-09-10T01:47:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:17 smithi170 ceph-mon[102832]: pgmap v4895: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 13 op/s 2023-09-10T01:47:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8868.723400 secs 2023-09-10T01:47:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8870.151805 secs 2023-09-10T01:47:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8870.863548 secs 2023-09-10T01:47:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8870.151805 secs 2023-09-10T01:47:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8870.863548 secs 2023-09-10T01:47:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8870.151805 secs 2023-09-10T01:47:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8870.863548 secs 2023-09-10T01:47:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:19 smithi082 ceph-mon[98694]: pgmap v4896: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T01:47:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:19 smithi098 ceph-mon[103124]: pgmap v4896: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T01:47:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:19 smithi170 ceph-mon[102832]: pgmap v4896: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T01:47:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:21 smithi082 ceph-mon[98694]: pgmap v4897: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T01:47:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:21 smithi098 ceph-mon[103124]: pgmap v4897: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T01:47:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:21 smithi170 ceph-mon[102832]: pgmap v4897: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T01:47:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:47:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:47:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:47:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8875.151864 secs 2023-09-10T01:47:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:23 smithi082 ceph-mon[98694]: pgmap v4898: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T01:47:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:47:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:47:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:47:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8875.863615 secs 2023-09-10T01:47:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8875.151864 secs 2023-09-10T01:47:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:23 smithi098 ceph-mon[103124]: pgmap v4898: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T01:47:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:47:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:47:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:47:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8875.863615 secs 2023-09-10T01:47:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8875.151864 secs 2023-09-10T01:47:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:23 smithi170 ceph-mon[102832]: pgmap v4898: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T01:47:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:47:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:47:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:47:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8875.863615 secs 2023-09-10T01:47:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:25 smithi082 ceph-mon[98694]: pgmap v4899: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T01:47:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:25 smithi098 ceph-mon[103124]: pgmap v4899: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T01:47:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:25 smithi170 ceph-mon[102832]: pgmap v4899: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T01:47:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:27 smithi082 ceph-mon[98694]: pgmap v4900: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T01:47:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:27 smithi098 ceph-mon[103124]: pgmap v4900: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T01:47:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:27 smithi170 ceph-mon[102832]: pgmap v4900: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T01:47:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8880.151923 secs 2023-09-10T01:47:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8880.863680 secs 2023-09-10T01:47:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8880.151923 secs 2023-09-10T01:47:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8880.863680 secs 2023-09-10T01:47:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8880.151923 secs 2023-09-10T01:47:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8880.863680 secs 2023-09-10T01:47:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:29 smithi098 ceph-mon[103124]: pgmap v4901: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 9 op/s 2023-09-10T01:47:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:29 smithi170 ceph-mon[102832]: pgmap v4901: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 9 op/s 2023-09-10T01:47:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:29 smithi082 ceph-mon[98694]: pgmap v4901: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 9 op/s 2023-09-10T01:47:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:31 smithi098 ceph-mon[103124]: pgmap v4902: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 113 KiB/s wr, 15 op/s 2023-09-10T01:47:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:31 smithi170 ceph-mon[102832]: pgmap v4902: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 113 KiB/s wr, 15 op/s 2023-09-10T01:47:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:31 smithi082 ceph-mon[98694]: pgmap v4902: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 113 KiB/s wr, 15 op/s 2023-09-10T01:47:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8885.151981 secs 2023-09-10T01:47:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:33 smithi098 ceph-mon[103124]: pgmap v4903: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T01:47:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8885.863746 secs 2023-09-10T01:47:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8885.151981 secs 2023-09-10T01:47:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:33 smithi170 ceph-mon[102832]: pgmap v4903: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T01:47:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8885.863746 secs 2023-09-10T01:47:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8885.151981 secs 2023-09-10T01:47:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:33 smithi082 ceph-mon[98694]: pgmap v4903: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T01:47:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8885.863746 secs 2023-09-10T01:47:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:35 smithi098 ceph-mon[103124]: pgmap v4904: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T01:47:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:35 smithi170 ceph-mon[102832]: pgmap v4904: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T01:47:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:35 smithi082 ceph-mon[98694]: pgmap v4904: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T01:47:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:37 smithi170 ceph-mon[102832]: pgmap v4905: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 122 KiB/s wr, 17 op/s 2023-09-10T01:47:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:37 smithi082 ceph-mon[98694]: pgmap v4905: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 122 KiB/s wr, 17 op/s 2023-09-10T01:47:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:37 smithi098 ceph-mon[103124]: pgmap v4905: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 122 KiB/s wr, 17 op/s 2023-09-10T01:47:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8890.152040 secs 2023-09-10T01:47:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8890.863816 secs 2023-09-10T01:47:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8890.152040 secs 2023-09-10T01:47:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8890.863816 secs 2023-09-10T01:47:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8890.152040 secs 2023-09-10T01:47:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8890.863816 secs 2023-09-10T01:47:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:39 smithi170 ceph-mon[102832]: pgmap v4906: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T01:47:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:39 smithi082 ceph-mon[98694]: pgmap v4906: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T01:47:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:39 smithi098 ceph-mon[103124]: pgmap v4906: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T01:47:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:41 smithi098 ceph-mon[103124]: pgmap v4907: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 177 KiB/s wr, 18 op/s 2023-09-10T01:47:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:41 smithi170 ceph-mon[102832]: pgmap v4907: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 177 KiB/s wr, 18 op/s 2023-09-10T01:47:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:41 smithi082 ceph-mon[98694]: pgmap v4907: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 177 KiB/s wr, 18 op/s 2023-09-10T01:47:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8893.723872 secs 2023-09-10T01:47:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:42 smithi098 ceph-mon[103124]: pgmap v4908: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T01:47:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8893.723872 secs 2023-09-10T01:47:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:42 smithi170 ceph-mon[102832]: pgmap v4908: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T01:47:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8893.723872 secs 2023-09-10T01:47:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:42 smithi082 ceph-mon[98694]: pgmap v4908: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T01:47:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8895.152096 secs 2023-09-10T01:47:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:47:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:47:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:47:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8895.863878 secs 2023-09-10T01:47:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8895.152096 secs 2023-09-10T01:47:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:47:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:47:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:47:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8895.863878 secs 2023-09-10T01:47:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8895.152096 secs 2023-09-10T01:47:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:47:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:47:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:47:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8895.863878 secs 2023-09-10T01:47:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:47:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:44 smithi098 ceph-mon[103124]: pgmap v4909: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T01:47:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:47:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:44 smithi170 ceph-mon[102832]: pgmap v4909: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T01:47:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:47:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:44 smithi082 ceph-mon[98694]: pgmap v4909: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T01:47:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:47 smithi082 ceph-mon[98694]: pgmap v4910: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 109 KiB/s wr, 17 op/s 2023-09-10T01:47:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8898.723932 secs 2023-09-10T01:47:48.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:48.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:48.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:48.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:48.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:48.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:48.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:48.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:47 smithi098 ceph-mon[103124]: pgmap v4910: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 109 KiB/s wr, 17 op/s 2023-09-10T01:47:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8898.723932 secs 2023-09-10T01:47:48.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:48.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:48.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:48.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:48.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:48.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:48.258 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:48.258 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:47 smithi170 ceph-mon[102832]: pgmap v4910: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 109 KiB/s wr, 17 op/s 2023-09-10T01:47:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8898.723932 secs 2023-09-10T01:47:48.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:48.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:48.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:48.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:48.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:48.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:48.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:48.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:48.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:48 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T01:47:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:48 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:47:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8900.152165 secs 2023-09-10T01:47:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:48 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T01:47:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:48 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T01:47:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:49.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:49.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:49.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:49.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:49.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:49.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:47:49.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:47:49.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:49.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8900.863947 secs 2023-09-10T01:47:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:48 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T01:47:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:48 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:47:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8900.152165 secs 2023-09-10T01:47:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:48 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T01:47:49.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:48 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T01:47:49.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:49.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:49.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:49.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:49.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:49.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:49.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:47:49.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:47:49.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:49.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8900.863947 secs 2023-09-10T01:47:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:48 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T01:47:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:48 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:47:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8900.152165 secs 2023-09-10T01:47:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:48 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T01:47:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:48 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T01:47:49.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:49.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:49.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:49.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:49.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:49.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:47:49.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:47:49.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:47:49.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:47:49.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8900.863947 secs 2023-09-10T01:47:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:49 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T01:47:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:49 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T01:47:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:49 smithi082 ceph-mon[98694]: pgmap v4911: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T01:47:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:49 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T01:47:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:49 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T01:47:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:49 smithi098 ceph-mon[103124]: pgmap v4911: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T01:47:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:49 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T01:47:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:49 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T01:47:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:49 smithi170 ceph-mon[102832]: pgmap v4911: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T01:47:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:51 smithi082 ceph-mon[98694]: pgmap v4912: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 81 KiB/s wr, 16 op/s 2023-09-10T01:47:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:51 smithi098 ceph-mon[103124]: pgmap v4912: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 81 KiB/s wr, 16 op/s 2023-09-10T01:47:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:51 smithi170 ceph-mon[102832]: pgmap v4912: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 81 KiB/s wr, 16 op/s 2023-09-10T01:47:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8903.724025 secs 2023-09-10T01:47:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8903.724025 secs 2023-09-10T01:47:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8903.724025 secs 2023-09-10T01:47:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8905.152223 secs 2023-09-10T01:47:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:53 smithi098 ceph-mon[103124]: pgmap v4913: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 16 KiB/s wr, 9 op/s 2023-09-10T01:47:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8905.864015 secs 2023-09-10T01:47:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8905.152223 secs 2023-09-10T01:47:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:53 smithi170 ceph-mon[102832]: pgmap v4913: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 16 KiB/s wr, 9 op/s 2023-09-10T01:47:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8905.864015 secs 2023-09-10T01:47:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8905.152223 secs 2023-09-10T01:47:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:53 smithi082 ceph-mon[98694]: pgmap v4913: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 16 KiB/s wr, 9 op/s 2023-09-10T01:47:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8905.864015 secs 2023-09-10T01:47:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:55 smithi098 ceph-mon[103124]: pgmap v4914: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 16 KiB/s wr, 9 op/s 2023-09-10T01:47:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:55 smithi170 ceph-mon[102832]: pgmap v4914: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 16 KiB/s wr, 9 op/s 2023-09-10T01:47:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:55 smithi082 ceph-mon[98694]: pgmap v4914: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 16 KiB/s wr, 9 op/s 2023-09-10T01:47:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:57 smithi098 ceph-mon[103124]: pgmap v4915: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 44 KiB/s wr, 14 op/s 2023-09-10T01:47:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8908.724087 secs 2023-09-10T01:47:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:57 smithi170 ceph-mon[102832]: pgmap v4915: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 44 KiB/s wr, 14 op/s 2023-09-10T01:47:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8908.724087 secs 2023-09-10T01:47:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:57 smithi082 ceph-mon[98694]: pgmap v4915: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 44 KiB/s wr, 14 op/s 2023-09-10T01:47:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8908.724087 secs 2023-09-10T01:47:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8910.152281 secs 2023-09-10T01:47:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8910.864104 secs 2023-09-10T01:47:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8910.152281 secs 2023-09-10T01:47:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8910.864104 secs 2023-09-10T01:47:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8910.152281 secs 2023-09-10T01:47:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8910.864104 secs 2023-09-10T01:48:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:47:59 smithi098 ceph-mon[103124]: pgmap v4916: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 44 KiB/s wr, 9 op/s 2023-09-10T01:48:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:47:59 smithi170 ceph-mon[102832]: pgmap v4916: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 44 KiB/s wr, 9 op/s 2023-09-10T01:48:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:47:59 smithi082 ceph-mon[98694]: pgmap v4916: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 44 KiB/s wr, 9 op/s 2023-09-10T01:48:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:01 smithi082 ceph-mon[98694]: pgmap v4917: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T01:48:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:01 smithi098 ceph-mon[103124]: pgmap v4917: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T01:48:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:01 smithi170 ceph-mon[102832]: pgmap v4917: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T01:48:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8913.724191 secs 2023-09-10T01:48:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8913.724191 secs 2023-09-10T01:48:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8913.724191 secs 2023-09-10T01:48:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8915.152337 secs 2023-09-10T01:48:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:03 smithi082 ceph-mon[98694]: pgmap v4918: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 81 KiB/s wr, 23 op/s 2023-09-10T01:48:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8915.864169 secs 2023-09-10T01:48:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8915.152337 secs 2023-09-10T01:48:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:03 smithi098 ceph-mon[103124]: pgmap v4918: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 81 KiB/s wr, 23 op/s 2023-09-10T01:48:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8915.864169 secs 2023-09-10T01:48:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8915.152337 secs 2023-09-10T01:48:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:03 smithi170 ceph-mon[102832]: pgmap v4918: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 81 KiB/s wr, 23 op/s 2023-09-10T01:48:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8915.864169 secs 2023-09-10T01:48:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:05 smithi082 ceph-mon[98694]: pgmap v4919: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 81 KiB/s wr, 23 op/s 2023-09-10T01:48:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:05 smithi098 ceph-mon[103124]: pgmap v4919: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 81 KiB/s wr, 23 op/s 2023-09-10T01:48:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:05 smithi170 ceph-mon[102832]: pgmap v4919: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 81 KiB/s wr, 23 op/s 2023-09-10T01:48:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:07 smithi082 ceph-mon[98694]: pgmap v4920: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:48:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8918.724253 secs 2023-09-10T01:48:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:07 smithi098 ceph-mon[103124]: pgmap v4920: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:48:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8918.724253 secs 2023-09-10T01:48:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:07 smithi170 ceph-mon[102832]: pgmap v4920: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:48:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8918.724253 secs 2023-09-10T01:48:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8920.152427 secs 2023-09-10T01:48:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:08 smithi082 ceph-mon[98694]: osdmap e377: 12 total, 12 up, 12 in 2023-09-10T01:48:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8920.864233 secs 2023-09-10T01:48:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8920.152427 secs 2023-09-10T01:48:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:08 smithi098 ceph-mon[103124]: osdmap e377: 12 total, 12 up, 12 in 2023-09-10T01:48:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8920.864233 secs 2023-09-10T01:48:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8920.152427 secs 2023-09-10T01:48:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:08 smithi170 ceph-mon[102832]: osdmap e377: 12 total, 12 up, 12 in 2023-09-10T01:48:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8920.864233 secs 2023-09-10T01:48:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:09 smithi082 ceph-mon[98694]: pgmap v4922: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T01:48:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:09 smithi098 ceph-mon[103124]: pgmap v4922: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T01:48:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:09 smithi170 ceph-mon[102832]: pgmap v4922: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T01:48:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:11 smithi082 ceph-mon[98694]: pgmap v4923: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 38 KiB/s wr, 13 op/s 2023-09-10T01:48:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:11 smithi098 ceph-mon[103124]: pgmap v4923: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 38 KiB/s wr, 13 op/s 2023-09-10T01:48:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:11 smithi170 ceph-mon[102832]: pgmap v4923: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 38 KiB/s wr, 13 op/s 2023-09-10T01:48:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8923.724339 secs 2023-09-10T01:48:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8923.724339 secs 2023-09-10T01:48:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8923.724339 secs 2023-09-10T01:48:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8925.152506 secs 2023-09-10T01:48:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:13 smithi098 ceph-mon[103124]: pgmap v4924: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 38 KiB/s wr, 13 op/s 2023-09-10T01:48:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8925.864304 secs 2023-09-10T01:48:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8925.152506 secs 2023-09-10T01:48:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:13 smithi170 ceph-mon[102832]: pgmap v4924: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 38 KiB/s wr, 13 op/s 2023-09-10T01:48:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8925.864304 secs 2023-09-10T01:48:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8925.152506 secs 2023-09-10T01:48:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:13 smithi082 ceph-mon[98694]: pgmap v4924: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 38 KiB/s wr, 13 op/s 2023-09-10T01:48:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8925.864304 secs 2023-09-10T01:48:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:15 smithi098 ceph-mon[103124]: pgmap v4925: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 38 KiB/s wr, 13 op/s 2023-09-10T01:48:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:15 smithi098 ceph-mon[103124]: osdmap e378: 12 total, 12 up, 12 in 2023-09-10T01:48:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:15 smithi170 ceph-mon[102832]: pgmap v4925: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 38 KiB/s wr, 13 op/s 2023-09-10T01:48:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:15 smithi170 ceph-mon[102832]: osdmap e378: 12 total, 12 up, 12 in 2023-09-10T01:48:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:15 smithi082 ceph-mon[98694]: pgmap v4925: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 38 KiB/s wr, 13 op/s 2023-09-10T01:48:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:15 smithi082 ceph-mon[98694]: osdmap e378: 12 total, 12 up, 12 in 2023-09-10T01:48:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:17 smithi098 ceph-mon[103124]: pgmap v4927: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 13 op/s 2023-09-10T01:48:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8928.724407 secs 2023-09-10T01:48:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:17 smithi170 ceph-mon[102832]: pgmap v4927: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 13 op/s 2023-09-10T01:48:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8928.724407 secs 2023-09-10T01:48:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:17 smithi082 ceph-mon[98694]: pgmap v4927: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 13 op/s 2023-09-10T01:48:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8928.724407 secs 2023-09-10T01:48:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8930.152567 secs 2023-09-10T01:48:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8930.864367 secs 2023-09-10T01:48:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8930.152567 secs 2023-09-10T01:48:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8930.864367 secs 2023-09-10T01:48:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8930.152567 secs 2023-09-10T01:48:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8930.864367 secs 2023-09-10T01:48:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:19 smithi098 ceph-mon[103124]: pgmap v4928: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-10T01:48:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:19 smithi170 ceph-mon[102832]: pgmap v4928: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-10T01:48:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:19 smithi082 ceph-mon[98694]: pgmap v4928: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-10T01:48:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:21 smithi082 ceph-mon[98694]: pgmap v4929: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-10T01:48:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:21 smithi098 ceph-mon[103124]: pgmap v4929: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-10T01:48:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:21 smithi170 ceph-mon[102832]: pgmap v4929: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-10T01:48:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:48:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:48:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:48:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8935.152634 secs 2023-09-10T01:48:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:23 smithi082 ceph-mon[98694]: pgmap v4930: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-10T01:48:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:48:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:48:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:48:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8935.864434 secs 2023-09-10T01:48:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8935.152634 secs 2023-09-10T01:48:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:23 smithi098 ceph-mon[103124]: pgmap v4930: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-10T01:48:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:48:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:48:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:48:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8935.864434 secs 2023-09-10T01:48:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8935.152634 secs 2023-09-10T01:48:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:23 smithi170 ceph-mon[102832]: pgmap v4930: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-10T01:48:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:48:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:48:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:48:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8935.864434 secs 2023-09-10T01:48:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:25 smithi082 ceph-mon[98694]: pgmap v4931: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-10T01:48:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:25 smithi098 ceph-mon[103124]: pgmap v4931: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-10T01:48:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:25 smithi170 ceph-mon[102832]: pgmap v4931: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-09-10T01:48:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:27 smithi082 ceph-mon[98694]: pgmap v4932: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T01:48:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:27 smithi098 ceph-mon[103124]: pgmap v4932: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T01:48:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:27 smithi170 ceph-mon[102832]: pgmap v4932: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T01:48:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8940.152689 secs 2023-09-10T01:48:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8940.864487 secs 2023-09-10T01:48:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8940.152689 secs 2023-09-10T01:48:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8940.864487 secs 2023-09-10T01:48:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8940.152689 secs 2023-09-10T01:48:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8940.864487 secs 2023-09-10T01:48:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:29 smithi082 ceph-mon[98694]: pgmap v4933: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T01:48:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:29 smithi098 ceph-mon[103124]: pgmap v4933: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T01:48:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:29 smithi170 ceph-mon[102832]: pgmap v4933: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T01:48:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:31 smithi082 ceph-mon[98694]: pgmap v4934: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 88 KiB/s wr, 16 op/s 2023-09-10T01:48:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:31 smithi098 ceph-mon[103124]: pgmap v4934: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 88 KiB/s wr, 16 op/s 2023-09-10T01:48:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:31 smithi170 ceph-mon[102832]: pgmap v4934: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 88 KiB/s wr, 16 op/s 2023-09-10T01:48:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8945.152748 secs 2023-09-10T01:48:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:33 smithi082 ceph-mon[98694]: pgmap v4935: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T01:48:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8945.864551 secs 2023-09-10T01:48:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8945.152748 secs 2023-09-10T01:48:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:33 smithi098 ceph-mon[103124]: pgmap v4935: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T01:48:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8945.864551 secs 2023-09-10T01:48:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8945.152748 secs 2023-09-10T01:48:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:33 smithi170 ceph-mon[102832]: pgmap v4935: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T01:48:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8945.864551 secs 2023-09-10T01:48:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:35 smithi098 ceph-mon[103124]: pgmap v4936: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T01:48:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:35 smithi170 ceph-mon[102832]: pgmap v4936: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T01:48:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:35 smithi082 ceph-mon[98694]: pgmap v4936: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T01:48:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:37 smithi098 ceph-mon[103124]: pgmap v4937: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 116 KiB/s wr, 17 op/s 2023-09-10T01:48:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:37 smithi170 ceph-mon[102832]: pgmap v4937: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 116 KiB/s wr, 17 op/s 2023-09-10T01:48:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:37 smithi082 ceph-mon[98694]: pgmap v4937: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 116 KiB/s wr, 17 op/s 2023-09-10T01:48:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8950.152806 secs 2023-09-10T01:48:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8950.864640 secs 2023-09-10T01:48:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8950.152806 secs 2023-09-10T01:48:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8950.864640 secs 2023-09-10T01:48:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8950.152806 secs 2023-09-10T01:48:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8950.864640 secs 2023-09-10T01:48:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:39 smithi098 ceph-mon[103124]: pgmap v4938: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 12 op/s 2023-09-10T01:48:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:39 smithi170 ceph-mon[102832]: pgmap v4938: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 12 op/s 2023-09-10T01:48:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:39 smithi082 ceph-mon[98694]: pgmap v4938: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 12 op/s 2023-09-10T01:48:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:41 smithi098 ceph-mon[103124]: pgmap v4939: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 120 KiB/s wr, 17 op/s 2023-09-10T01:48:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:41 smithi170 ceph-mon[102832]: pgmap v4939: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 120 KiB/s wr, 17 op/s 2023-09-10T01:48:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:41 smithi082 ceph-mon[98694]: pgmap v4939: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 120 KiB/s wr, 17 op/s 2023-09-10T01:48:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8955.152883 secs 2023-09-10T01:48:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:43 smithi098 ceph-mon[103124]: pgmap v4940: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T01:48:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8955.864704 secs 2023-09-10T01:48:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8955.152883 secs 2023-09-10T01:48:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:43 smithi170 ceph-mon[102832]: pgmap v4940: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T01:48:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8955.864704 secs 2023-09-10T01:48:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8955.152883 secs 2023-09-10T01:48:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:43 smithi082 ceph-mon[98694]: pgmap v4940: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T01:48:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8955.864704 secs 2023-09-10T01:48:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:45 smithi098 ceph-mon[103124]: pgmap v4941: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T01:48:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:45 smithi170 ceph-mon[102832]: pgmap v4941: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T01:48:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:45 smithi082 ceph-mon[98694]: pgmap v4941: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T01:48:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:47 smithi098 ceph-mon[103124]: pgmap v4942: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 17 op/s 2023-09-10T01:48:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8958.724894 secs 2023-09-10T01:48:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:47 smithi170 ceph-mon[102832]: pgmap v4942: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 17 op/s 2023-09-10T01:48:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8958.724894 secs 2023-09-10T01:48:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:47 smithi082 ceph-mon[98694]: pgmap v4942: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 17 op/s 2023-09-10T01:48:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8958.724894 secs 2023-09-10T01:48:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8960.152942 secs 2023-09-10T01:48:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8960.864770 secs 2023-09-10T01:48:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:48:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8960.152942 secs 2023-09-10T01:48:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8960.864770 secs 2023-09-10T01:48:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:48:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8960.152942 secs 2023-09-10T01:48:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8960.864770 secs 2023-09-10T01:48:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:48:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:49 smithi098 ceph-mon[103124]: pgmap v4943: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T01:48:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:48:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:48:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:48:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:49 smithi170 ceph-mon[102832]: pgmap v4943: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T01:48:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:48:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:48:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:48:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:49 smithi082 ceph-mon[98694]: pgmap v4943: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T01:48:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:48:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:48:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:48:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:51 smithi082 ceph-mon[98694]: pgmap v4944: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 122 KiB/s wr, 17 op/s 2023-09-10T01:48:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:48:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:48:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:48:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:51 smithi098 ceph-mon[103124]: pgmap v4944: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 122 KiB/s wr, 17 op/s 2023-09-10T01:48:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:48:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:48:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:48:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:51 smithi170 ceph-mon[102832]: pgmap v4944: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 122 KiB/s wr, 17 op/s 2023-09-10T01:48:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:48:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:48:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:48:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8963.725011 secs 2023-09-10T01:48:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8963.725011 secs 2023-09-10T01:48:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8963.725011 secs 2023-09-10T01:48:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8965.153017 secs 2023-09-10T01:48:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:53 smithi082 ceph-mon[98694]: pgmap v4945: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T01:48:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8965.864834 secs 2023-09-10T01:48:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8965.153017 secs 2023-09-10T01:48:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:53 smithi098 ceph-mon[103124]: pgmap v4945: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T01:48:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8965.864834 secs 2023-09-10T01:48:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8965.153017 secs 2023-09-10T01:48:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:53 smithi170 ceph-mon[102832]: pgmap v4945: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T01:48:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8965.864834 secs 2023-09-10T01:48:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:55 smithi082 ceph-mon[98694]: pgmap v4946: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T01:48:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:55 smithi098 ceph-mon[103124]: pgmap v4946: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T01:48:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:55 smithi170 ceph-mon[102832]: pgmap v4946: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T01:48:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:57 smithi082 ceph-mon[98694]: pgmap v4947: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 16 op/s 2023-09-10T01:48:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8968.725152 secs 2023-09-10T01:48:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:57 smithi098 ceph-mon[103124]: pgmap v4947: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 16 op/s 2023-09-10T01:48:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8968.725152 secs 2023-09-10T01:48:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:57 smithi170 ceph-mon[102832]: pgmap v4947: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 16 op/s 2023-09-10T01:48:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8968.725152 secs 2023-09-10T01:48:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8970.153075 secs 2023-09-10T01:48:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8970.864901 secs 2023-09-10T01:48:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8970.153075 secs 2023-09-10T01:48:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8970.864901 secs 2023-09-10T01:48:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8970.153075 secs 2023-09-10T01:48:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8970.864901 secs 2023-09-10T01:49:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:48:59 smithi082 ceph-mon[98694]: pgmap v4948: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 73 KiB/s wr, 10 op/s 2023-09-10T01:49:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:48:59 smithi098 ceph-mon[103124]: pgmap v4948: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 73 KiB/s wr, 10 op/s 2023-09-10T01:49:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:48:59 smithi170 ceph-mon[102832]: pgmap v4948: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 73 KiB/s wr, 10 op/s 2023-09-10T01:49:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:01 smithi098 ceph-mon[103124]: pgmap v4949: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 109 KiB/s wr, 28 op/s 2023-09-10T01:49:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:01 smithi170 ceph-mon[102832]: pgmap v4949: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 109 KiB/s wr, 28 op/s 2023-09-10T01:49:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:01 smithi082 ceph-mon[98694]: pgmap v4949: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 109 KiB/s wr, 28 op/s 2023-09-10T01:49:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8973.725215 secs 2023-09-10T01:49:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8973.725215 secs 2023-09-10T01:49:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8973.725215 secs 2023-09-10T01:49:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8975.153131 secs 2023-09-10T01:49:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:03 smithi098 ceph-mon[103124]: pgmap v4950: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 69 KiB/s wr, 22 op/s 2023-09-10T01:49:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8975.864967 secs 2023-09-10T01:49:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8975.153131 secs 2023-09-10T01:49:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:03 smithi170 ceph-mon[102832]: pgmap v4950: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 69 KiB/s wr, 22 op/s 2023-09-10T01:49:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8975.864967 secs 2023-09-10T01:49:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8975.153131 secs 2023-09-10T01:49:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:03 smithi082 ceph-mon[98694]: pgmap v4950: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 69 KiB/s wr, 22 op/s 2023-09-10T01:49:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8975.864967 secs 2023-09-10T01:49:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:05 smithi098 ceph-mon[103124]: pgmap v4951: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 69 KiB/s wr, 22 op/s 2023-09-10T01:49:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:05 smithi170 ceph-mon[102832]: pgmap v4951: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 69 KiB/s wr, 22 op/s 2023-09-10T01:49:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:05 smithi082 ceph-mon[98694]: pgmap v4951: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 69 KiB/s wr, 22 op/s 2023-09-10T01:49:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:07 smithi098 ceph-mon[103124]: pgmap v4952: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 92 KiB/s wr, 29 op/s 2023-09-10T01:49:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8978.725276 secs 2023-09-10T01:49:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:07 smithi170 ceph-mon[102832]: pgmap v4952: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 92 KiB/s wr, 29 op/s 2023-09-10T01:49:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8978.725276 secs 2023-09-10T01:49:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:07 smithi082 ceph-mon[98694]: pgmap v4952: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 92 KiB/s wr, 29 op/s 2023-09-10T01:49:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8978.725276 secs 2023-09-10T01:49:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8980.153189 secs 2023-09-10T01:49:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:08 smithi098 ceph-mon[103124]: osdmap e379: 12 total, 12 up, 12 in 2023-09-10T01:49:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8980.865033 secs 2023-09-10T01:49:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8980.153189 secs 2023-09-10T01:49:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:08 smithi170 ceph-mon[102832]: osdmap e379: 12 total, 12 up, 12 in 2023-09-10T01:49:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8980.865033 secs 2023-09-10T01:49:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8980.153189 secs 2023-09-10T01:49:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:08 smithi082 ceph-mon[98694]: osdmap e379: 12 total, 12 up, 12 in 2023-09-10T01:49:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8980.865033 secs 2023-09-10T01:49:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:09 smithi098 ceph-mon[103124]: pgmap v4954: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-10T01:49:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:09 smithi170 ceph-mon[102832]: pgmap v4954: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-10T01:49:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:09 smithi082 ceph-mon[98694]: pgmap v4954: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-10T01:49:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:11 smithi082 ceph-mon[98694]: pgmap v4955: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 65 KiB/s wr, 14 op/s 2023-09-10T01:49:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:11 smithi098 ceph-mon[103124]: pgmap v4955: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 65 KiB/s wr, 14 op/s 2023-09-10T01:49:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:11 smithi170 ceph-mon[102832]: pgmap v4955: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 65 KiB/s wr, 14 op/s 2023-09-10T01:49:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8983.725395 secs 2023-09-10T01:49:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8983.725395 secs 2023-09-10T01:49:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8983.725395 secs 2023-09-10T01:49:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8985.153246 secs 2023-09-10T01:49:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:13 smithi082 ceph-mon[98694]: pgmap v4956: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 65 KiB/s wr, 14 op/s 2023-09-10T01:49:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8985.865098 secs 2023-09-10T01:49:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8985.153246 secs 2023-09-10T01:49:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:13 smithi098 ceph-mon[103124]: pgmap v4956: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 65 KiB/s wr, 14 op/s 2023-09-10T01:49:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8985.865098 secs 2023-09-10T01:49:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8985.153246 secs 2023-09-10T01:49:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:13 smithi170 ceph-mon[102832]: pgmap v4956: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 65 KiB/s wr, 14 op/s 2023-09-10T01:49:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8985.865098 secs 2023-09-10T01:49:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:15 smithi082 ceph-mon[98694]: pgmap v4957: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 65 KiB/s wr, 14 op/s 2023-09-10T01:49:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:15 smithi082 ceph-mon[98694]: osdmap e380: 12 total, 12 up, 12 in 2023-09-10T01:49:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:15 smithi098 ceph-mon[103124]: pgmap v4957: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 65 KiB/s wr, 14 op/s 2023-09-10T01:49:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:15 smithi098 ceph-mon[103124]: osdmap e380: 12 total, 12 up, 12 in 2023-09-10T01:49:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:15 smithi170 ceph-mon[102832]: pgmap v4957: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 65 KiB/s wr, 14 op/s 2023-09-10T01:49:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:15 smithi170 ceph-mon[102832]: osdmap e380: 12 total, 12 up, 12 in 2023-09-10T01:49:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:17 smithi098 ceph-mon[103124]: pgmap v4959: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 42 KiB/s wr, 13 op/s 2023-09-10T01:49:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8988.725455 secs 2023-09-10T01:49:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:17 smithi170 ceph-mon[102832]: pgmap v4959: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 42 KiB/s wr, 13 op/s 2023-09-10T01:49:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8988.725455 secs 2023-09-10T01:49:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:17 smithi082 ceph-mon[98694]: pgmap v4959: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 42 KiB/s wr, 13 op/s 2023-09-10T01:49:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8988.725455 secs 2023-09-10T01:49:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8990.153311 secs 2023-09-10T01:49:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8990.865164 secs 2023-09-10T01:49:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8990.153311 secs 2023-09-10T01:49:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8990.865164 secs 2023-09-10T01:49:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8990.153311 secs 2023-09-10T01:49:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8990.865164 secs 2023-09-10T01:49:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:19 smithi098 ceph-mon[103124]: pgmap v4960: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:49:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:19 smithi170 ceph-mon[102832]: pgmap v4960: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:49:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:19 smithi082 ceph-mon[98694]: pgmap v4960: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:49:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:21 smithi098 ceph-mon[103124]: pgmap v4961: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-10T01:49:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:21 smithi170 ceph-mon[102832]: pgmap v4961: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-10T01:49:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:21 smithi082 ceph-mon[98694]: pgmap v4961: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-10T01:49:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 8993.725537 secs 2023-09-10T01:49:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 8993.725537 secs 2023-09-10T01:49:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 8993.725537 secs 2023-09-10T01:49:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8995.153392 secs 2023-09-10T01:49:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:23 smithi098 ceph-mon[103124]: pgmap v4962: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-10T01:49:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 8995.865234 secs 2023-09-10T01:49:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8995.153392 secs 2023-09-10T01:49:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:23 smithi170 ceph-mon[102832]: pgmap v4962: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-10T01:49:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 8995.865234 secs 2023-09-10T01:49:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8995.153392 secs 2023-09-10T01:49:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:23 smithi082 ceph-mon[98694]: pgmap v4962: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-10T01:49:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 8995.865234 secs 2023-09-10T01:49:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:25 smithi098 ceph-mon[103124]: pgmap v4963: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-10T01:49:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:25 smithi170 ceph-mon[102832]: pgmap v4963: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-10T01:49:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:25 smithi082 ceph-mon[98694]: pgmap v4963: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-10T01:49:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:27 smithi098 ceph-mon[103124]: pgmap v4964: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T01:49:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:27 smithi170 ceph-mon[102832]: pgmap v4964: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T01:49:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:27 smithi082 ceph-mon[98694]: pgmap v4964: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T01:49:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9000.153491 secs 2023-09-10T01:49:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9000.865299 secs 2023-09-10T01:49:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9000.153491 secs 2023-09-10T01:49:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9000.865299 secs 2023-09-10T01:49:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9000.153491 secs 2023-09-10T01:49:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9000.865299 secs 2023-09-10T01:49:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:29 smithi098 ceph-mon[103124]: pgmap v4965: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T01:49:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:29 smithi170 ceph-mon[102832]: pgmap v4965: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T01:49:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:29 smithi082 ceph-mon[98694]: pgmap v4965: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T01:49:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:30 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:49:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:30 smithi098 ceph-mon[103124]: pgmap v4966: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 105 KiB/s wr, 16 op/s 2023-09-10T01:49:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:30 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:49:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:30 smithi170 ceph-mon[102832]: pgmap v4966: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 105 KiB/s wr, 16 op/s 2023-09-10T01:49:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:30 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:49:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:30 smithi082 ceph-mon[98694]: pgmap v4966: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 105 KiB/s wr, 16 op/s 2023-09-10T01:49:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:31 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:49:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:49:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:49:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:31 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:49:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:49:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:49:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:31 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:49:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:49:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:49:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:33 smithi098 ceph-mon[103124]: pgmap v4967: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T01:49:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:33 smithi170 ceph-mon[102832]: pgmap v4967: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T01:49:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:33 smithi082 ceph-mon[98694]: pgmap v4967: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T01:49:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9005.153549 secs 2023-09-10T01:49:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9005.865368 secs 2023-09-10T01:49:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9005.153549 secs 2023-09-10T01:49:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9005.865368 secs 2023-09-10T01:49:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9005.153549 secs 2023-09-10T01:49:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9005.865368 secs 2023-09-10T01:49:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:35 smithi170 ceph-mon[102832]: pgmap v4968: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T01:49:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:35 smithi082 ceph-mon[98694]: pgmap v4968: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T01:49:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:35 smithi098 ceph-mon[103124]: pgmap v4968: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T01:49:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:37 smithi082 ceph-mon[98694]: pgmap v4969: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 136 KiB/s wr, 18 op/s 2023-09-10T01:49:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:37 smithi098 ceph-mon[103124]: pgmap v4969: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 136 KiB/s wr, 18 op/s 2023-09-10T01:49:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:37 smithi170 ceph-mon[102832]: pgmap v4969: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 136 KiB/s wr, 18 op/s 2023-09-10T01:49:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9010.153607 secs 2023-09-10T01:49:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9010.865437 secs 2023-09-10T01:49:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9010.153607 secs 2023-09-10T01:49:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9010.865437 secs 2023-09-10T01:49:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9010.153607 secs 2023-09-10T01:49:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9010.865437 secs 2023-09-10T01:49:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:39 smithi098 ceph-mon[103124]: pgmap v4970: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T01:49:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:39 smithi170 ceph-mon[102832]: pgmap v4970: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T01:49:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:39 smithi082 ceph-mon[98694]: pgmap v4970: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T01:49:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:41 smithi098 ceph-mon[103124]: pgmap v4971: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 127 KiB/s wr, 18 op/s 2023-09-10T01:49:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:41 smithi170 ceph-mon[102832]: pgmap v4971: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 127 KiB/s wr, 18 op/s 2023-09-10T01:49:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:41 smithi082 ceph-mon[98694]: pgmap v4971: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 127 KiB/s wr, 18 op/s 2023-09-10T01:49:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9015.153670 secs 2023-09-10T01:49:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:43 smithi098 ceph-mon[103124]: pgmap v4972: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T01:49:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9015.865487 secs 2023-09-10T01:49:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9015.153670 secs 2023-09-10T01:49:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:43 smithi170 ceph-mon[102832]: pgmap v4972: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T01:49:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9015.865487 secs 2023-09-10T01:49:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9015.153670 secs 2023-09-10T01:49:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:43 smithi082 ceph-mon[98694]: pgmap v4972: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T01:49:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9015.865487 secs 2023-09-10T01:49:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:45 smithi098 ceph-mon[103124]: pgmap v4973: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T01:49:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:45 smithi170 ceph-mon[102832]: pgmap v4973: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T01:49:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:45 smithi082 ceph-mon[98694]: pgmap v4973: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T01:49:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:47 smithi098 ceph-mon[103124]: pgmap v4974: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 135 KiB/s wr, 17 op/s 2023-09-10T01:49:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9018.725933 secs 2023-09-10T01:49:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:47 smithi170 ceph-mon[102832]: pgmap v4974: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 135 KiB/s wr, 17 op/s 2023-09-10T01:49:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9018.725933 secs 2023-09-10T01:49:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:47 smithi082 ceph-mon[98694]: pgmap v4974: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 135 KiB/s wr, 17 op/s 2023-09-10T01:49:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9018.725933 secs 2023-09-10T01:49:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9020.153727 secs 2023-09-10T01:49:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9020.865558 secs 2023-09-10T01:49:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9020.153727 secs 2023-09-10T01:49:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9020.865558 secs 2023-09-10T01:49:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9020.153727 secs 2023-09-10T01:49:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9020.865558 secs 2023-09-10T01:49:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:49 smithi098 ceph-mon[103124]: pgmap v4975: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T01:49:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:49:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:49 smithi170 ceph-mon[102832]: pgmap v4975: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T01:49:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:49:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:49 smithi082 ceph-mon[98694]: pgmap v4975: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T01:49:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:49:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:49:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:49:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:49:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:49:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:49:51.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:51 smithi170 ceph-mon[102832]: pgmap v4976: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 109 KiB/s wr, 17 op/s 2023-09-10T01:49:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:49:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:49:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:49:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:49:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:49:51.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:51 smithi082 ceph-mon[98694]: pgmap v4976: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 109 KiB/s wr, 17 op/s 2023-09-10T01:49:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:49:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:49:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:49:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:49:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:49:51.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:51 smithi098 ceph-mon[103124]: pgmap v4976: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 109 KiB/s wr, 17 op/s 2023-09-10T01:49:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:49:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:49:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:49:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:49:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:49:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:49:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:49:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:49:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:49:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9023.726021 secs 2023-09-10T01:49:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:53 smithi170 ceph-mon[102832]: pgmap v4977: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T01:49:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9023.726021 secs 2023-09-10T01:49:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:53 smithi082 ceph-mon[98694]: pgmap v4977: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T01:49:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9023.726021 secs 2023-09-10T01:49:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:53 smithi098 ceph-mon[103124]: pgmap v4977: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T01:49:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9025.153812 secs 2023-09-10T01:49:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9025.865623 secs 2023-09-10T01:49:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9025.153812 secs 2023-09-10T01:49:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9025.865623 secs 2023-09-10T01:49:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9025.153812 secs 2023-09-10T01:49:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9025.865623 secs 2023-09-10T01:49:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:55 smithi170 ceph-mon[102832]: pgmap v4978: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T01:49:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:55 smithi082 ceph-mon[98694]: pgmap v4978: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T01:49:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:55 smithi098 ceph-mon[103124]: pgmap v4978: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T01:49:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:57 smithi170 ceph-mon[102832]: pgmap v4979: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 97 KiB/s wr, 16 op/s 2023-09-10T01:49:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:57 smithi082 ceph-mon[98694]: pgmap v4979: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 97 KiB/s wr, 16 op/s 2023-09-10T01:49:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:57 smithi098 ceph-mon[103124]: pgmap v4979: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 97 KiB/s wr, 16 op/s 2023-09-10T01:49:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9028.726081 secs 2023-09-10T01:49:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9028.726081 secs 2023-09-10T01:49:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9028.726081 secs 2023-09-10T01:49:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9030.153895 secs 2023-09-10T01:49:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9030.865688 secs 2023-09-10T01:49:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:49:59 smithi170 ceph-mon[102832]: pgmap v4980: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 49 KiB/s wr, 10 op/s 2023-09-10T01:49:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9030.153895 secs 2023-09-10T01:49:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9030.865688 secs 2023-09-10T01:49:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:49:59 smithi082 ceph-mon[98694]: pgmap v4980: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 49 KiB/s wr, 10 op/s 2023-09-10T01:49:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9030.153895 secs 2023-09-10T01:49:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9030.865688 secs 2023-09-10T01:49:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:49:59 smithi098 ceph-mon[103124]: pgmap v4980: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 49 KiB/s wr, 10 op/s 2023-09-10T01:50:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T01:50:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T01:50:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T01:50:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:01 smithi098 ceph-mon[103124]: pgmap v4981: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 73 KiB/s wr, 18 op/s 2023-09-10T01:50:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:01 smithi170 ceph-mon[102832]: pgmap v4981: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 73 KiB/s wr, 18 op/s 2023-09-10T01:50:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:01 smithi082 ceph-mon[98694]: pgmap v4981: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 73 KiB/s wr, 18 op/s 2023-09-10T01:50:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9033.726145 secs 2023-09-10T01:50:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9033.726145 secs 2023-09-10T01:50:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9033.726145 secs 2023-09-10T01:50:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9035.153966 secs 2023-09-10T01:50:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:03 smithi098 ceph-mon[103124]: pgmap v4982: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 12 op/s 2023-09-10T01:50:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9035.865754 secs 2023-09-10T01:50:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9035.153966 secs 2023-09-10T01:50:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:03 smithi170 ceph-mon[102832]: pgmap v4982: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 12 op/s 2023-09-10T01:50:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9035.865754 secs 2023-09-10T01:50:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9035.153966 secs 2023-09-10T01:50:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:03 smithi082 ceph-mon[98694]: pgmap v4982: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 12 op/s 2023-09-10T01:50:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9035.865754 secs 2023-09-10T01:50:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:05 smithi098 ceph-mon[103124]: pgmap v4983: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 12 op/s 2023-09-10T01:50:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:05 smithi170 ceph-mon[102832]: pgmap v4983: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 12 op/s 2023-09-10T01:50:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:05 smithi082 ceph-mon[98694]: pgmap v4983: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 12 op/s 2023-09-10T01:50:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:07 smithi098 ceph-mon[103124]: pgmap v4984: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T01:50:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9038.726286 secs 2023-09-10T01:50:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:07 smithi170 ceph-mon[102832]: pgmap v4984: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T01:50:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9038.726286 secs 2023-09-10T01:50:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:07 smithi082 ceph-mon[98694]: pgmap v4984: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T01:50:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9038.726286 secs 2023-09-10T01:50:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9040.154023 secs 2023-09-10T01:50:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:08 smithi098 ceph-mon[103124]: osdmap e381: 12 total, 12 up, 12 in 2023-09-10T01:50:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9040.865821 secs 2023-09-10T01:50:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9040.154023 secs 2023-09-10T01:50:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:08 smithi170 ceph-mon[102832]: osdmap e381: 12 total, 12 up, 12 in 2023-09-10T01:50:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9040.865821 secs 2023-09-10T01:50:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9040.154023 secs 2023-09-10T01:50:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:08 smithi082 ceph-mon[98694]: osdmap e381: 12 total, 12 up, 12 in 2023-09-10T01:50:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9040.865821 secs 2023-09-10T01:50:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:09 smithi098 ceph-mon[103124]: pgmap v4986: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-10T01:50:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:09 smithi170 ceph-mon[102832]: pgmap v4986: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-10T01:50:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:09 smithi082 ceph-mon[98694]: pgmap v4986: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-10T01:50:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:11 smithi098 ceph-mon[103124]: pgmap v4987: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 79 KiB/s wr, 26 op/s 2023-09-10T01:50:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:11 smithi170 ceph-mon[102832]: pgmap v4987: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 79 KiB/s wr, 26 op/s 2023-09-10T01:50:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:11 smithi082 ceph-mon[98694]: pgmap v4987: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 79 KiB/s wr, 26 op/s 2023-09-10T01:50:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9043.726367 secs 2023-09-10T01:50:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9043.726367 secs 2023-09-10T01:50:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9043.726367 secs 2023-09-10T01:50:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9045.154110 secs 2023-09-10T01:50:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:13 smithi098 ceph-mon[103124]: pgmap v4988: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 79 KiB/s wr, 26 op/s 2023-09-10T01:50:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9045.865885 secs 2023-09-10T01:50:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9045.154110 secs 2023-09-10T01:50:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:13 smithi170 ceph-mon[102832]: pgmap v4988: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 79 KiB/s wr, 26 op/s 2023-09-10T01:50:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9045.865885 secs 2023-09-10T01:50:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9045.154110 secs 2023-09-10T01:50:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:13 smithi082 ceph-mon[98694]: pgmap v4988: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 79 KiB/s wr, 26 op/s 2023-09-10T01:50:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9045.865885 secs 2023-09-10T01:50:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:15 smithi098 ceph-mon[103124]: pgmap v4989: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 79 KiB/s wr, 26 op/s 2023-09-10T01:50:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:15 smithi098 ceph-mon[103124]: osdmap e382: 12 total, 12 up, 12 in 2023-09-10T01:50:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:15 smithi170 ceph-mon[102832]: pgmap v4989: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 79 KiB/s wr, 26 op/s 2023-09-10T01:50:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:15 smithi170 ceph-mon[102832]: osdmap e382: 12 total, 12 up, 12 in 2023-09-10T01:50:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:15 smithi082 ceph-mon[98694]: pgmap v4989: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 79 KiB/s wr, 26 op/s 2023-09-10T01:50:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:15 smithi082 ceph-mon[98694]: osdmap e382: 12 total, 12 up, 12 in 2023-09-10T01:50:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:17 smithi098 ceph-mon[103124]: pgmap v4991: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 42 KiB/s wr, 13 op/s 2023-09-10T01:50:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9048.726496 secs 2023-09-10T01:50:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:17 smithi170 ceph-mon[102832]: pgmap v4991: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 42 KiB/s wr, 13 op/s 2023-09-10T01:50:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9048.726496 secs 2023-09-10T01:50:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:17 smithi082 ceph-mon[98694]: pgmap v4991: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 42 KiB/s wr, 13 op/s 2023-09-10T01:50:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9048.726496 secs 2023-09-10T01:50:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9050.154193 secs 2023-09-10T01:50:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9050.865949 secs 2023-09-10T01:50:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9050.154193 secs 2023-09-10T01:50:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9050.865949 secs 2023-09-10T01:50:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9050.154193 secs 2023-09-10T01:50:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9050.865949 secs 2023-09-10T01:50:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:20 smithi098 ceph-mon[103124]: pgmap v4992: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:50:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:20 smithi170 ceph-mon[102832]: pgmap v4992: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:50:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:20 smithi082 ceph-mon[98694]: pgmap v4992: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T01:50:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:21 smithi098 ceph-mon[103124]: pgmap v4993: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T01:50:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:21 smithi170 ceph-mon[102832]: pgmap v4993: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T01:50:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:21 smithi082 ceph-mon[98694]: pgmap v4993: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T01:50:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9053.726556 secs 2023-09-10T01:50:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9053.726556 secs 2023-09-10T01:50:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9053.726556 secs 2023-09-10T01:50:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9055.154249 secs 2023-09-10T01:50:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:23 smithi098 ceph-mon[103124]: pgmap v4994: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T01:50:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9055.866015 secs 2023-09-10T01:50:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9055.154249 secs 2023-09-10T01:50:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:23 smithi170 ceph-mon[102832]: pgmap v4994: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T01:50:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9055.866015 secs 2023-09-10T01:50:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9055.154249 secs 2023-09-10T01:50:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:23 smithi082 ceph-mon[98694]: pgmap v4994: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T01:50:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9055.866015 secs 2023-09-10T01:50:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:25 smithi098 ceph-mon[103124]: pgmap v4995: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T01:50:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:25 smithi170 ceph-mon[102832]: pgmap v4995: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T01:50:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:25 smithi082 ceph-mon[98694]: pgmap v4995: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T01:50:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:27 smithi098 ceph-mon[103124]: pgmap v4996: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T01:50:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9058.726617 secs 2023-09-10T01:50:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:27 smithi170 ceph-mon[102832]: pgmap v4996: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T01:50:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9058.726617 secs 2023-09-10T01:50:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:27 smithi082 ceph-mon[98694]: pgmap v4996: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T01:50:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9058.726617 secs 2023-09-10T01:50:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9060.154332 secs 2023-09-10T01:50:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9060.866081 secs 2023-09-10T01:50:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9060.154332 secs 2023-09-10T01:50:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9060.866081 secs 2023-09-10T01:50:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9060.154332 secs 2023-09-10T01:50:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9060.866081 secs 2023-09-10T01:50:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:29 smithi098 ceph-mon[103124]: pgmap v4997: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T01:50:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:29 smithi170 ceph-mon[102832]: pgmap v4997: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T01:50:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:29 smithi082 ceph-mon[98694]: pgmap v4997: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T01:50:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:31 smithi098 ceph-mon[103124]: pgmap v4998: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 109 KiB/s wr, 16 op/s 2023-09-10T01:50:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:31 smithi170 ceph-mon[102832]: pgmap v4998: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 109 KiB/s wr, 16 op/s 2023-09-10T01:50:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:31 smithi082 ceph-mon[98694]: pgmap v4998: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 109 KiB/s wr, 16 op/s 2023-09-10T01:50:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9065.154416 secs 2023-09-10T01:50:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:33 smithi098 ceph-mon[103124]: pgmap v4999: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T01:50:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9065.866144 secs 2023-09-10T01:50:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9065.154416 secs 2023-09-10T01:50:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:33 smithi170 ceph-mon[102832]: pgmap v4999: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T01:50:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9065.866144 secs 2023-09-10T01:50:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9065.154416 secs 2023-09-10T01:50:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:33 smithi082 ceph-mon[98694]: pgmap v4999: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T01:50:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9065.866144 secs 2023-09-10T01:50:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:34 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:50:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:34 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:50:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:34 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:50:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:35 smithi098 ceph-mon[103124]: pgmap v5000: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T01:50:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:35 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:50:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:35 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:50:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:35 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:50:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:35 smithi170 ceph-mon[102832]: pgmap v5000: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T01:50:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:35 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:50:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:35 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:50:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:35 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:50:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:35 smithi082 ceph-mon[98694]: pgmap v5000: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T01:50:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:35 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:50:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:35 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:50:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:35 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:50:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:37 smithi170 ceph-mon[102832]: pgmap v5001: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 147 KiB/s wr, 18 op/s 2023-09-10T01:50:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:37 smithi082 ceph-mon[98694]: pgmap v5001: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 147 KiB/s wr, 18 op/s 2023-09-10T01:50:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:37 smithi098 ceph-mon[103124]: pgmap v5001: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 147 KiB/s wr, 18 op/s 2023-09-10T01:50:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9070.866208 secs 2023-09-10T01:50:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9070.866208 secs 2023-09-10T01:50:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9070.866208 secs 2023-09-10T01:50:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9070.154474 secs 2023-09-10T01:50:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:39 smithi170 ceph-mon[102832]: pgmap v5002: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T01:50:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9070.154474 secs 2023-09-10T01:50:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:39 smithi082 ceph-mon[98694]: pgmap v5002: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T01:50:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9070.154474 secs 2023-09-10T01:50:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:39 smithi098 ceph-mon[103124]: pgmap v5002: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T01:50:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:41 smithi098 ceph-mon[103124]: pgmap v5003: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 129 KiB/s wr, 18 op/s 2023-09-10T01:50:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:41 smithi170 ceph-mon[102832]: pgmap v5003: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 129 KiB/s wr, 18 op/s 2023-09-10T01:50:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:41 smithi082 ceph-mon[98694]: pgmap v5003: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 129 KiB/s wr, 18 op/s 2023-09-10T01:50:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9075.154506 secs 2023-09-10T01:50:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:43 smithi098 ceph-mon[103124]: pgmap v5004: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 87 KiB/s wr, 12 op/s 2023-09-10T01:50:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9075.866275 secs 2023-09-10T01:50:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9075.154506 secs 2023-09-10T01:50:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:43 smithi170 ceph-mon[102832]: pgmap v5004: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 87 KiB/s wr, 12 op/s 2023-09-10T01:50:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9075.866275 secs 2023-09-10T01:50:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9075.154506 secs 2023-09-10T01:50:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:43 smithi082 ceph-mon[98694]: pgmap v5004: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 87 KiB/s wr, 12 op/s 2023-09-10T01:50:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9075.866275 secs 2023-09-10T01:50:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:45 smithi098 ceph-mon[103124]: pgmap v5005: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 87 KiB/s wr, 12 op/s 2023-09-10T01:50:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:45 smithi170 ceph-mon[102832]: pgmap v5005: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 87 KiB/s wr, 12 op/s 2023-09-10T01:50:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:45 smithi082 ceph-mon[98694]: pgmap v5005: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 87 KiB/s wr, 12 op/s 2023-09-10T01:50:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:47 smithi098 ceph-mon[103124]: pgmap v5006: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 132 KiB/s wr, 18 op/s 2023-09-10T01:50:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9078.727058 secs 2023-09-10T01:50:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:47 smithi170 ceph-mon[102832]: pgmap v5006: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 132 KiB/s wr, 18 op/s 2023-09-10T01:50:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9078.727058 secs 2023-09-10T01:50:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:47 smithi082 ceph-mon[98694]: pgmap v5006: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 132 KiB/s wr, 18 op/s 2023-09-10T01:50:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9078.727058 secs 2023-09-10T01:50:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9080.154566 secs 2023-09-10T01:50:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9080.866340 secs 2023-09-10T01:50:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9080.154566 secs 2023-09-10T01:50:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9080.866340 secs 2023-09-10T01:50:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9080.154566 secs 2023-09-10T01:50:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9080.866340 secs 2023-09-10T01:50:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:49 smithi098 ceph-mon[103124]: pgmap v5007: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 77 KiB/s wr, 12 op/s 2023-09-10T01:50:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:49 smithi170 ceph-mon[102832]: pgmap v5007: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 77 KiB/s wr, 12 op/s 2023-09-10T01:50:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:49 smithi082 ceph-mon[98694]: pgmap v5007: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 77 KiB/s wr, 12 op/s 2023-09-10T01:50:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:50:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:50:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:50:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:51 smithi098 ceph-mon[103124]: pgmap v5008: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 100 KiB/s wr, 17 op/s 2023-09-10T01:50:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:50:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:50:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:50:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:50:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:50:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:50:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:50:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:50:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:50:52.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:50:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:51 smithi170 ceph-mon[102832]: pgmap v5008: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 100 KiB/s wr, 17 op/s 2023-09-10T01:50:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:50:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:50:52.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:50:52.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:50:52.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:50:52.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:50:52.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:50:52.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:50:52.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:50:52.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:50:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:51 smithi082 ceph-mon[98694]: pgmap v5008: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 100 KiB/s wr, 17 op/s 2023-09-10T01:50:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:50:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:50:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:50:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:50:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:50:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:50:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:50:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:50:52.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:50:52.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:50:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9083.727120 secs 2023-09-10T01:50:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9083.727120 secs 2023-09-10T01:50:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9083.727120 secs 2023-09-10T01:50:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9085.154624 secs 2023-09-10T01:50:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:53 smithi098 ceph-mon[103124]: pgmap v5009: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T01:50:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9085.866405 secs 2023-09-10T01:50:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9085.154624 secs 2023-09-10T01:50:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:53 smithi170 ceph-mon[102832]: pgmap v5009: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T01:50:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9085.866405 secs 2023-09-10T01:50:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9085.154624 secs 2023-09-10T01:50:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:53 smithi082 ceph-mon[98694]: pgmap v5009: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T01:50:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9085.866405 secs 2023-09-10T01:50:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:56 smithi170 ceph-mon[102832]: pgmap v5010: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T01:50:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:56 smithi082 ceph-mon[98694]: pgmap v5010: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T01:50:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:56 smithi098 ceph-mon[103124]: pgmap v5010: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T01:50:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:57 smithi170 ceph-mon[102832]: pgmap v5011: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 97 KiB/s wr, 17 op/s 2023-09-10T01:50:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:57 smithi082 ceph-mon[98694]: pgmap v5011: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 97 KiB/s wr, 17 op/s 2023-09-10T01:50:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:57 smithi098 ceph-mon[103124]: pgmap v5011: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 97 KiB/s wr, 17 op/s 2023-09-10T01:50:58.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9088.727184 secs 2023-09-10T01:50:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9088.727184 secs 2023-09-10T01:50:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9088.727184 secs 2023-09-10T01:50:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9090.154715 secs 2023-09-10T01:50:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9090.866487 secs 2023-09-10T01:50:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:50:59 smithi170 ceph-mon[102832]: pgmap v5012: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:50:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9090.154715 secs 2023-09-10T01:50:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9090.866487 secs 2023-09-10T01:50:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:50:59 smithi082 ceph-mon[98694]: pgmap v5012: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:50:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9090.154715 secs 2023-09-10T01:50:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9090.866487 secs 2023-09-10T01:50:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:50:59 smithi098 ceph-mon[103124]: pgmap v5012: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T01:51:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:01 smithi098 ceph-mon[103124]: pgmap v5013: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 16 op/s 2023-09-10T01:51:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:01 smithi170 ceph-mon[102832]: pgmap v5013: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 16 op/s 2023-09-10T01:51:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:01 smithi082 ceph-mon[98694]: pgmap v5013: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 16 op/s 2023-09-10T01:51:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9093.727277 secs 2023-09-10T01:51:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9093.727277 secs 2023-09-10T01:51:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9093.727277 secs 2023-09-10T01:51:04.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9095.154778 secs 2023-09-10T01:51:04.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:03 smithi098 ceph-mon[103124]: pgmap v5014: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 41 KiB/s wr, 10 op/s 2023-09-10T01:51:04.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9095.866554 secs 2023-09-10T01:51:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9095.154778 secs 2023-09-10T01:51:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:03 smithi170 ceph-mon[102832]: pgmap v5014: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 41 KiB/s wr, 10 op/s 2023-09-10T01:51:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9095.866554 secs 2023-09-10T01:51:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9095.154778 secs 2023-09-10T01:51:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:03 smithi082 ceph-mon[98694]: pgmap v5014: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 41 KiB/s wr, 10 op/s 2023-09-10T01:51:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9095.866554 secs 2023-09-10T01:51:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:05 smithi098 ceph-mon[103124]: pgmap v5015: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 41 KiB/s wr, 10 op/s 2023-09-10T01:51:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:05 smithi170 ceph-mon[102832]: pgmap v5015: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 41 KiB/s wr, 10 op/s 2023-09-10T01:51:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:05 smithi082 ceph-mon[98694]: pgmap v5015: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 41 KiB/s wr, 10 op/s 2023-09-10T01:51:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:07 smithi098 ceph-mon[103124]: pgmap v5016: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T01:51:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9098.727339 secs 2023-09-10T01:51:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:07 smithi170 ceph-mon[102832]: pgmap v5016: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T01:51:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9098.727339 secs 2023-09-10T01:51:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:07 smithi082 ceph-mon[98694]: pgmap v5016: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T01:51:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9098.727339 secs 2023-09-10T01:51:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9100.154833 secs 2023-09-10T01:51:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:08 smithi098 ceph-mon[103124]: osdmap e383: 12 total, 12 up, 12 in 2023-09-10T01:51:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9100.866621 secs 2023-09-10T01:51:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9100.154833 secs 2023-09-10T01:51:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:08 smithi170 ceph-mon[102832]: osdmap e383: 12 total, 12 up, 12 in 2023-09-10T01:51:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9100.866621 secs 2023-09-10T01:51:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9100.154833 secs 2023-09-10T01:51:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:08 smithi082 ceph-mon[98694]: osdmap e383: 12 total, 12 up, 12 in 2023-09-10T01:51:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9100.866621 secs 2023-09-10T01:51:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:09 smithi098 ceph-mon[103124]: pgmap v5018: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-09-10T01:51:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:09 smithi170 ceph-mon[102832]: pgmap v5018: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-09-10T01:51:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:09 smithi082 ceph-mon[98694]: pgmap v5018: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-09-10T01:51:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:12 smithi098 ceph-mon[103124]: pgmap v5019: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:51:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:12 smithi170 ceph-mon[102832]: pgmap v5019: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:51:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:12 smithi082 ceph-mon[98694]: pgmap v5019: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:51:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9103.727434 secs 2023-09-10T01:51:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9103.727434 secs 2023-09-10T01:51:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9103.727434 secs 2023-09-10T01:51:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9105.154919 secs 2023-09-10T01:51:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:14 smithi170 ceph-mon[102832]: pgmap v5020: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:51:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9105.866690 secs 2023-09-10T01:51:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9105.154919 secs 2023-09-10T01:51:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:14 smithi082 ceph-mon[98694]: pgmap v5020: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:51:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9105.866690 secs 2023-09-10T01:51:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9105.154919 secs 2023-09-10T01:51:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:14 smithi098 ceph-mon[103124]: pgmap v5020: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:51:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9105.866690 secs 2023-09-10T01:51:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:15 smithi170 ceph-mon[102832]: pgmap v5021: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:51:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:15 smithi082 ceph-mon[98694]: pgmap v5021: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:51:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:15 smithi098 ceph-mon[103124]: pgmap v5021: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:51:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:16 smithi098 ceph-mon[103124]: osdmap e384: 12 total, 12 up, 12 in 2023-09-10T01:51:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:16 smithi170 ceph-mon[102832]: osdmap e384: 12 total, 12 up, 12 in 2023-09-10T01:51:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:16 smithi082 ceph-mon[98694]: osdmap e384: 12 total, 12 up, 12 in 2023-09-10T01:51:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:17 smithi098 ceph-mon[103124]: pgmap v5023: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 13 op/s 2023-09-10T01:51:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9108.727493 secs 2023-09-10T01:51:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:17 smithi170 ceph-mon[102832]: pgmap v5023: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 13 op/s 2023-09-10T01:51:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9108.727493 secs 2023-09-10T01:51:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:17 smithi082 ceph-mon[98694]: pgmap v5023: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 13 op/s 2023-09-10T01:51:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9108.727493 secs 2023-09-10T01:51:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9110.155000 secs 2023-09-10T01:51:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9110.866753 secs 2023-09-10T01:51:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9110.155000 secs 2023-09-10T01:51:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9110.866753 secs 2023-09-10T01:51:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9110.155000 secs 2023-09-10T01:51:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9110.866753 secs 2023-09-10T01:51:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:19 smithi098 ceph-mon[103124]: pgmap v5024: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T01:51:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:19 smithi170 ceph-mon[102832]: pgmap v5024: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T01:51:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:19 smithi082 ceph-mon[98694]: pgmap v5024: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T01:51:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:21 smithi098 ceph-mon[103124]: pgmap v5025: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T01:51:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:21 smithi170 ceph-mon[102832]: pgmap v5025: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T01:51:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:21 smithi082 ceph-mon[98694]: pgmap v5025: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T01:51:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9113.727574 secs 2023-09-10T01:51:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9113.727574 secs 2023-09-10T01:51:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9113.727574 secs 2023-09-10T01:51:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9115.155059 secs 2023-09-10T01:51:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:24 smithi170 ceph-mon[102832]: pgmap v5026: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T01:51:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9115.866817 secs 2023-09-10T01:51:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9115.155059 secs 2023-09-10T01:51:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:24 smithi082 ceph-mon[98694]: pgmap v5026: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T01:51:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9115.866817 secs 2023-09-10T01:51:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9115.155059 secs 2023-09-10T01:51:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:24 smithi098 ceph-mon[103124]: pgmap v5026: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T01:51:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9115.866817 secs 2023-09-10T01:51:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:26 smithi170 ceph-mon[102832]: pgmap v5027: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T01:51:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:26 smithi082 ceph-mon[98694]: pgmap v5027: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T01:51:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:26 smithi098 ceph-mon[103124]: pgmap v5027: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T01:51:27.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:27 smithi170 ceph-mon[102832]: pgmap v5028: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 12 op/s 2023-09-10T01:51:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:27 smithi082 ceph-mon[98694]: pgmap v5028: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 12 op/s 2023-09-10T01:51:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:27 smithi098 ceph-mon[103124]: pgmap v5028: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 12 op/s 2023-09-10T01:51:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9118.727633 secs 2023-09-10T01:51:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9118.727633 secs 2023-09-10T01:51:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9118.727633 secs 2023-09-10T01:51:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9120.155117 secs 2023-09-10T01:51:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9120.866889 secs 2023-09-10T01:51:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:29 smithi170 ceph-mon[102832]: pgmap v5029: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:51:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9120.155117 secs 2023-09-10T01:51:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9120.866889 secs 2023-09-10T01:51:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:29 smithi082 ceph-mon[98694]: pgmap v5029: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:51:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9120.155117 secs 2023-09-10T01:51:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9120.866889 secs 2023-09-10T01:51:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:29 smithi098 ceph-mon[103124]: pgmap v5029: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T01:51:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:31 smithi098 ceph-mon[103124]: pgmap v5030: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 108 KiB/s wr, 17 op/s 2023-09-10T01:51:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:31 smithi170 ceph-mon[102832]: pgmap v5030: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 108 KiB/s wr, 17 op/s 2023-09-10T01:51:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:31 smithi082 ceph-mon[98694]: pgmap v5030: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 108 KiB/s wr, 17 op/s 2023-09-10T01:51:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9125.155200 secs 2023-09-10T01:51:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:33 smithi098 ceph-mon[103124]: pgmap v5031: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 90 KiB/s wr, 12 op/s 2023-09-10T01:51:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9125.866953 secs 2023-09-10T01:51:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9125.155200 secs 2023-09-10T01:51:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:33 smithi170 ceph-mon[102832]: pgmap v5031: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 90 KiB/s wr, 12 op/s 2023-09-10T01:51:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9125.866953 secs 2023-09-10T01:51:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9125.155200 secs 2023-09-10T01:51:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:33 smithi082 ceph-mon[98694]: pgmap v5031: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 90 KiB/s wr, 12 op/s 2023-09-10T01:51:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9125.866953 secs 2023-09-10T01:51:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:34 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:51:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:34 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:51:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:34 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:51:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:35 smithi098 ceph-mon[103124]: pgmap v5032: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 90 KiB/s wr, 12 op/s 2023-09-10T01:51:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:35 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:51:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:35 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:51:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:35 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:51:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:35 smithi170 ceph-mon[102832]: pgmap v5032: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 90 KiB/s wr, 12 op/s 2023-09-10T01:51:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:35 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:51:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:35 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:51:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:35 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:51:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:35 smithi082 ceph-mon[98694]: pgmap v5032: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 90 KiB/s wr, 12 op/s 2023-09-10T01:51:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:35 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:51:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:35 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:51:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:35 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:51:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:37 smithi098 ceph-mon[103124]: pgmap v5033: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 140 KiB/s wr, 18 op/s 2023-09-10T01:51:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:37 smithi170 ceph-mon[102832]: pgmap v5033: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 140 KiB/s wr, 18 op/s 2023-09-10T01:51:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:37 smithi082 ceph-mon[98694]: pgmap v5033: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 140 KiB/s wr, 18 op/s 2023-09-10T01:51:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9130.155260 secs 2023-09-10T01:51:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9130.867019 secs 2023-09-10T01:51:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9130.155260 secs 2023-09-10T01:51:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9130.867019 secs 2023-09-10T01:51:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9130.155260 secs 2023-09-10T01:51:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9130.867019 secs 2023-09-10T01:51:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:40 smithi098 ceph-mon[103124]: pgmap v5034: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 90 KiB/s wr, 12 op/s 2023-09-10T01:51:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:40 smithi170 ceph-mon[102832]: pgmap v5034: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 90 KiB/s wr, 12 op/s 2023-09-10T01:51:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:40 smithi082 ceph-mon[98694]: pgmap v5034: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 90 KiB/s wr, 12 op/s 2023-09-10T01:51:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:42 smithi170 ceph-mon[102832]: pgmap v5035: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 129 KiB/s wr, 17 op/s 2023-09-10T01:51:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:42 smithi082 ceph-mon[98694]: pgmap v5035: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 129 KiB/s wr, 17 op/s 2023-09-10T01:51:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:42 smithi098 ceph-mon[103124]: pgmap v5035: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 129 KiB/s wr, 17 op/s 2023-09-10T01:51:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9135.155342 secs 2023-09-10T01:51:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:44 smithi170 ceph-mon[102832]: pgmap v5036: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:51:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9135.867089 secs 2023-09-10T01:51:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9135.155342 secs 2023-09-10T01:51:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:44 smithi082 ceph-mon[98694]: pgmap v5036: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:51:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9135.867089 secs 2023-09-10T01:51:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9135.155342 secs 2023-09-10T01:51:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:44 smithi098 ceph-mon[103124]: pgmap v5036: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:51:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9135.867089 secs 2023-09-10T01:51:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:45 smithi170 ceph-mon[102832]: pgmap v5037: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:51:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:45 smithi082 ceph-mon[98694]: pgmap v5037: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:51:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:45 smithi098 ceph-mon[103124]: pgmap v5037: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:51:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:47 smithi098 ceph-mon[103124]: pgmap v5038: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 164 KiB/s wr, 17 op/s 2023-09-10T01:51:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9138.727900 secs 2023-09-10T01:51:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:47 smithi170 ceph-mon[102832]: pgmap v5038: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 164 KiB/s wr, 17 op/s 2023-09-10T01:51:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9138.727900 secs 2023-09-10T01:51:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:47 smithi082 ceph-mon[98694]: pgmap v5038: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 164 KiB/s wr, 17 op/s 2023-09-10T01:51:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9138.727900 secs 2023-09-10T01:51:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9140.155400 secs 2023-09-10T01:51:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9140.867154 secs 2023-09-10T01:51:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9140.155400 secs 2023-09-10T01:51:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9140.867154 secs 2023-09-10T01:51:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9140.155400 secs 2023-09-10T01:51:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9140.867154 secs 2023-09-10T01:51:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:49 smithi098 ceph-mon[103124]: pgmap v5039: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T01:51:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:49 smithi170 ceph-mon[102832]: pgmap v5039: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T01:51:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:49 smithi082 ceph-mon[98694]: pgmap v5039: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T01:51:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:51 smithi098 ceph-mon[103124]: pgmap v5040: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 136 KiB/s wr, 17 op/s 2023-09-10T01:51:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:51:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:51:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:51:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:51:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:51 smithi170 ceph-mon[102832]: pgmap v5040: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 136 KiB/s wr, 17 op/s 2023-09-10T01:51:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:51:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:51:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:51:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:51:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:51 smithi082 ceph-mon[98694]: pgmap v5040: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 136 KiB/s wr, 17 op/s 2023-09-10T01:51:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:51:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:51:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:51:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:51:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9143.727963 secs 2023-09-10T01:51:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:51:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:51:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:51:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9143.727963 secs 2023-09-10T01:51:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:51:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:51:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:51:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9143.727963 secs 2023-09-10T01:51:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:51:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:51:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:51:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9145.155495 secs 2023-09-10T01:51:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:53 smithi098 ceph-mon[103124]: pgmap v5041: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T01:51:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9145.867220 secs 2023-09-10T01:51:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9145.155495 secs 2023-09-10T01:51:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:53 smithi170 ceph-mon[102832]: pgmap v5041: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T01:51:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9145.867220 secs 2023-09-10T01:51:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9145.155495 secs 2023-09-10T01:51:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:53 smithi082 ceph-mon[98694]: pgmap v5041: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T01:51:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9145.867220 secs 2023-09-10T01:51:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:55 smithi098 ceph-mon[103124]: pgmap v5042: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T01:51:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:55 smithi170 ceph-mon[102832]: pgmap v5042: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T01:51:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:55 smithi082 ceph-mon[98694]: pgmap v5042: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T01:51:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:57 smithi098 ceph-mon[103124]: pgmap v5043: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 125 KiB/s wr, 17 op/s 2023-09-10T01:51:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9148.728021 secs 2023-09-10T01:51:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:57 smithi170 ceph-mon[102832]: pgmap v5043: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 125 KiB/s wr, 17 op/s 2023-09-10T01:51:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9148.728021 secs 2023-09-10T01:51:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:57 smithi082 ceph-mon[98694]: pgmap v5043: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 125 KiB/s wr, 17 op/s 2023-09-10T01:51:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9148.728021 secs 2023-09-10T01:51:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9150.155553 secs 2023-09-10T01:51:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:51:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9150.867285 secs 2023-09-10T01:51:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9150.155553 secs 2023-09-10T01:51:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:51:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9150.867285 secs 2023-09-10T01:51:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9150.155553 secs 2023-09-10T01:51:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:51:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9150.867285 secs 2023-09-10T01:52:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:00 smithi170 ceph-mon[102832]: pgmap v5044: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T01:52:00.473 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:00 smithi082 ceph-mon[98694]: pgmap v5044: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T01:52:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:00 smithi098 ceph-mon[103124]: pgmap v5044: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T01:52:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:02 smithi170 ceph-mon[102832]: pgmap v5045: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 15 op/s 2023-09-10T01:52:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:02 smithi082 ceph-mon[98694]: pgmap v5045: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 15 op/s 2023-09-10T01:52:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:02 smithi098 ceph-mon[103124]: pgmap v5045: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 15 op/s 2023-09-10T01:52:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9153.728110 secs 2023-09-10T01:52:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9153.728110 secs 2023-09-10T01:52:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9153.728110 secs 2023-09-10T01:52:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9155.155678 secs 2023-09-10T01:52:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:04 smithi170 ceph-mon[102832]: pgmap v5046: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 41 KiB/s wr, 9 op/s 2023-09-10T01:52:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9155.867355 secs 2023-09-10T01:52:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9155.155678 secs 2023-09-10T01:52:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:04 smithi082 ceph-mon[98694]: pgmap v5046: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 41 KiB/s wr, 9 op/s 2023-09-10T01:52:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9155.867355 secs 2023-09-10T01:52:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9155.155678 secs 2023-09-10T01:52:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:04 smithi098 ceph-mon[103124]: pgmap v5046: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 41 KiB/s wr, 9 op/s 2023-09-10T01:52:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9155.867355 secs 2023-09-10T01:52:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:05 smithi170 ceph-mon[102832]: pgmap v5047: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 41 KiB/s wr, 9 op/s 2023-09-10T01:52:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:05 smithi082 ceph-mon[98694]: pgmap v5047: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 41 KiB/s wr, 9 op/s 2023-09-10T01:52:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:05 smithi098 ceph-mon[103124]: pgmap v5047: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 41 KiB/s wr, 9 op/s 2023-09-10T01:52:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:07 smithi098 ceph-mon[103124]: pgmap v5048: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:52:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9158.728195 secs 2023-09-10T01:52:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:07 smithi170 ceph-mon[102832]: pgmap v5048: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:52:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9158.728195 secs 2023-09-10T01:52:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:07 smithi082 ceph-mon[98694]: pgmap v5048: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T01:52:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9158.728195 secs 2023-09-10T01:52:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9160.155735 secs 2023-09-10T01:52:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:09 smithi170 ceph-mon[102832]: osdmap e385: 12 total, 12 up, 12 in 2023-09-10T01:52:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9160.867422 secs 2023-09-10T01:52:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9160.155735 secs 2023-09-10T01:52:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:09 smithi082 ceph-mon[98694]: osdmap e385: 12 total, 12 up, 12 in 2023-09-10T01:52:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9160.867422 secs 2023-09-10T01:52:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9160.155735 secs 2023-09-10T01:52:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:09 smithi098 ceph-mon[103124]: osdmap e385: 12 total, 12 up, 12 in 2023-09-10T01:52:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9160.867422 secs 2023-09-10T01:52:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:10 smithi170 ceph-mon[102832]: pgmap v5050: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 71 KiB/s wr, 28 op/s 2023-09-10T01:52:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:10 smithi082 ceph-mon[98694]: pgmap v5050: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 71 KiB/s wr, 28 op/s 2023-09-10T01:52:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:10 smithi098 ceph-mon[103124]: pgmap v5050: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 71 KiB/s wr, 28 op/s 2023-09-10T01:52:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:12 smithi170 ceph-mon[102832]: pgmap v5051: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:52:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:12 smithi082 ceph-mon[98694]: pgmap v5051: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:52:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:12 smithi098 ceph-mon[103124]: pgmap v5051: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:52:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9163.728257 secs 2023-09-10T01:52:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9163.728257 secs 2023-09-10T01:52:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9163.728257 secs 2023-09-10T01:52:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9165.155793 secs 2023-09-10T01:52:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:14 smithi170 ceph-mon[102832]: pgmap v5052: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:52:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9165.867488 secs 2023-09-10T01:52:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9165.155793 secs 2023-09-10T01:52:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:14 smithi082 ceph-mon[98694]: pgmap v5052: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:52:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9165.867488 secs 2023-09-10T01:52:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9165.155793 secs 2023-09-10T01:52:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:14 smithi098 ceph-mon[103124]: pgmap v5052: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:52:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9165.867488 secs 2023-09-10T01:52:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:15 smithi170 ceph-mon[102832]: pgmap v5053: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:52:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:15 smithi082 ceph-mon[98694]: pgmap v5053: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:52:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:15 smithi098 ceph-mon[103124]: pgmap v5053: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:52:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:16 smithi098 ceph-mon[103124]: osdmap e386: 12 total, 12 up, 12 in 2023-09-10T01:52:17.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:16 smithi170 ceph-mon[102832]: osdmap e386: 12 total, 12 up, 12 in 2023-09-10T01:52:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:16 smithi082 ceph-mon[98694]: osdmap e386: 12 total, 12 up, 12 in 2023-09-10T01:52:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:18 smithi170 ceph-mon[102832]: pgmap v5055: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 13 op/s 2023-09-10T01:52:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9168.728366 secs 2023-09-10T01:52:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:18 smithi082 ceph-mon[98694]: pgmap v5055: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 13 op/s 2023-09-10T01:52:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9168.728366 secs 2023-09-10T01:52:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:18 smithi098 ceph-mon[103124]: pgmap v5055: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 13 op/s 2023-09-10T01:52:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9168.728366 secs 2023-09-10T01:52:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9170.155861 secs 2023-09-10T01:52:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9170.867556 secs 2023-09-10T01:52:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9170.155861 secs 2023-09-10T01:52:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9170.867556 secs 2023-09-10T01:52:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9170.155861 secs 2023-09-10T01:52:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9170.867556 secs 2023-09-10T01:52:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:20 smithi170 ceph-mon[102832]: pgmap v5056: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T01:52:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:20 smithi082 ceph-mon[98694]: pgmap v5056: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T01:52:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:20 smithi098 ceph-mon[103124]: pgmap v5056: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T01:52:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:22 smithi098 ceph-mon[103124]: pgmap v5057: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T01:52:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:22 smithi170 ceph-mon[102832]: pgmap v5057: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T01:52:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:22 smithi082 ceph-mon[98694]: pgmap v5057: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T01:52:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9173.728425 secs 2023-09-10T01:52:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9173.728425 secs 2023-09-10T01:52:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9173.728425 secs 2023-09-10T01:52:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9175.155923 secs 2023-09-10T01:52:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:24 smithi170 ceph-mon[102832]: pgmap v5058: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T01:52:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9175.867621 secs 2023-09-10T01:52:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9175.155923 secs 2023-09-10T01:52:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:24 smithi082 ceph-mon[98694]: pgmap v5058: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T01:52:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9175.867621 secs 2023-09-10T01:52:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9175.155923 secs 2023-09-10T01:52:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:24 smithi098 ceph-mon[103124]: pgmap v5058: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T01:52:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9175.867621 secs 2023-09-10T01:52:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:26 smithi170 ceph-mon[102832]: pgmap v5059: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T01:52:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:26 smithi082 ceph-mon[98694]: pgmap v5059: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T01:52:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:26 smithi098 ceph-mon[103124]: pgmap v5059: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T01:52:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:28 smithi170 ceph-mon[102832]: pgmap v5060: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T01:52:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:28 smithi082 ceph-mon[98694]: pgmap v5060: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T01:52:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:28 smithi098 ceph-mon[103124]: pgmap v5060: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T01:52:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9180.155979 secs 2023-09-10T01:52:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9180.867686 secs 2023-09-10T01:52:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9180.155979 secs 2023-09-10T01:52:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9180.867686 secs 2023-09-10T01:52:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9180.155979 secs 2023-09-10T01:52:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9180.867686 secs 2023-09-10T01:52:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:30 smithi170 ceph-mon[102832]: pgmap v5061: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 69 KiB/s wr, 10 op/s 2023-09-10T01:52:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:30 smithi082 ceph-mon[98694]: pgmap v5061: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 69 KiB/s wr, 10 op/s 2023-09-10T01:52:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:30 smithi098 ceph-mon[103124]: pgmap v5061: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 69 KiB/s wr, 10 op/s 2023-09-10T01:52:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:52:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:31 smithi170 ceph-mon[102832]: pgmap v5062: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 112 KiB/s wr, 16 op/s 2023-09-10T01:52:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:52:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:31 smithi082 ceph-mon[98694]: pgmap v5062: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 112 KiB/s wr, 16 op/s 2023-09-10T01:52:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:52:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:31 smithi098 ceph-mon[103124]: pgmap v5062: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 112 KiB/s wr, 16 op/s 2023-09-10T01:52:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:52:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:52:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:52:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:52:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:52:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:52:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:52:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:52:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:52:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:33 smithi170 ceph-mon[102832]: pgmap v5063: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T01:52:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:33 smithi082 ceph-mon[98694]: pgmap v5063: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T01:52:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:33 smithi098 ceph-mon[103124]: pgmap v5063: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T01:52:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9185.156066 secs 2023-09-10T01:52:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9185.867750 secs 2023-09-10T01:52:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9185.156066 secs 2023-09-10T01:52:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9185.867750 secs 2023-09-10T01:52:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9185.156066 secs 2023-09-10T01:52:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9185.867750 secs 2023-09-10T01:52:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:35 smithi170 ceph-mon[102832]: pgmap v5064: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T01:52:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:35 smithi082 ceph-mon[98694]: pgmap v5064: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T01:52:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:35 smithi098 ceph-mon[103124]: pgmap v5064: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T01:52:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:38 smithi170 ceph-mon[102832]: pgmap v5065: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 141 KiB/s wr, 18 op/s 2023-09-10T01:52:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:38 smithi082 ceph-mon[98694]: pgmap v5065: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 141 KiB/s wr, 18 op/s 2023-09-10T01:52:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:38 smithi098 ceph-mon[103124]: pgmap v5065: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 141 KiB/s wr, 18 op/s 2023-09-10T01:52:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9190.156125 secs 2023-09-10T01:52:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9190.867815 secs 2023-09-10T01:52:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9190.156125 secs 2023-09-10T01:52:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9190.867815 secs 2023-09-10T01:52:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9190.156125 secs 2023-09-10T01:52:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9190.867815 secs 2023-09-10T01:52:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:40 smithi170 ceph-mon[102832]: pgmap v5066: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T01:52:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:40 smithi082 ceph-mon[98694]: pgmap v5066: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T01:52:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:40 smithi098 ceph-mon[103124]: pgmap v5066: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T01:52:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:42 smithi170 ceph-mon[102832]: pgmap v5067: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 132 KiB/s wr, 17 op/s 2023-09-10T01:52:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:42 smithi082 ceph-mon[98694]: pgmap v5067: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 132 KiB/s wr, 17 op/s 2023-09-10T01:52:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:42 smithi098 ceph-mon[103124]: pgmap v5067: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 132 KiB/s wr, 17 op/s 2023-09-10T01:52:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9195.156185 secs 2023-09-10T01:52:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9195.156185 secs 2023-09-10T01:52:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9195.156185 secs 2023-09-10T01:52:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:44 smithi170 ceph-mon[102832]: pgmap v5068: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:52:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9195.867878 secs 2023-09-10T01:52:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:44 smithi082 ceph-mon[98694]: pgmap v5068: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:52:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9195.867878 secs 2023-09-10T01:52:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:44 smithi098 ceph-mon[103124]: pgmap v5068: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:52:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9195.867878 secs 2023-09-10T01:52:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:45 smithi170 ceph-mon[102832]: pgmap v5069: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:52:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:45 smithi082 ceph-mon[98694]: pgmap v5069: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:52:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:45 smithi098 ceph-mon[103124]: pgmap v5069: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T01:52:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:47 smithi170 ceph-mon[102832]: pgmap v5070: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 135 KiB/s wr, 16 op/s 2023-09-10T01:52:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:47 smithi082 ceph-mon[98694]: pgmap v5070: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 135 KiB/s wr, 16 op/s 2023-09-10T01:52:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:47 smithi098 ceph-mon[103124]: pgmap v5070: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 135 KiB/s wr, 16 op/s 2023-09-10T01:52:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9198.728849 secs 2023-09-10T01:52:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9200.156242 secs 2023-09-10T01:52:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9198.728849 secs 2023-09-10T01:52:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9200.156242 secs 2023-09-10T01:52:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9198.728849 secs 2023-09-10T01:52:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9200.156242 secs 2023-09-10T01:52:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9200.867942 secs 2023-09-10T01:52:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:49 smithi170 ceph-mon[102832]: pgmap v5071: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T01:52:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9200.867942 secs 2023-09-10T01:52:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:49 smithi082 ceph-mon[98694]: pgmap v5071: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T01:52:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9200.867942 secs 2023-09-10T01:52:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:49 smithi098 ceph-mon[103124]: pgmap v5071: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T01:52:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:50 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:52:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:52:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:52:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:50 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:52:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:52:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:52:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:50 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:52:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:52:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:52:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:52 smithi170 ceph-mon[102832]: pgmap v5072: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 111 KiB/s wr, 16 op/s 2023-09-10T01:52:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:52 smithi082 ceph-mon[98694]: pgmap v5072: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 111 KiB/s wr, 16 op/s 2023-09-10T01:52:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:52 smithi098 ceph-mon[103124]: pgmap v5072: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 111 KiB/s wr, 16 op/s 2023-09-10T01:52:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9203.728915 secs 2023-09-10T01:52:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:52:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9205.156307 secs 2023-09-10T01:52:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9203.728915 secs 2023-09-10T01:52:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:52:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9205.156307 secs 2023-09-10T01:52:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9203.728915 secs 2023-09-10T01:52:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:52:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9205.156307 secs 2023-09-10T01:52:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:54 smithi170 ceph-mon[102832]: pgmap v5073: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T01:52:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:52:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:52:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:52:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9205.868004 secs 2023-09-10T01:52:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:54 smithi082 ceph-mon[98694]: pgmap v5073: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T01:52:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:52:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:52:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:52:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9205.868004 secs 2023-09-10T01:52:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:54 smithi098 ceph-mon[103124]: pgmap v5073: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T01:52:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:52:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:52:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:52:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9205.868004 secs 2023-09-10T01:52:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:55 smithi170 ceph-mon[102832]: pgmap v5074: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T01:52:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:55 smithi082 ceph-mon[98694]: pgmap v5074: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T01:52:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:55 smithi098 ceph-mon[103124]: pgmap v5074: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T01:52:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:57 smithi170 ceph-mon[102832]: pgmap v5075: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 99 KiB/s wr, 17 op/s 2023-09-10T01:52:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:57 smithi082 ceph-mon[98694]: pgmap v5075: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 99 KiB/s wr, 17 op/s 2023-09-10T01:52:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:57 smithi098 ceph-mon[103124]: pgmap v5075: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 99 KiB/s wr, 17 op/s 2023-09-10T01:52:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9208.728999 secs 2023-09-10T01:52:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9210.156394 secs 2023-09-10T01:52:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9208.728999 secs 2023-09-10T01:52:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9210.156394 secs 2023-09-10T01:52:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9208.728999 secs 2023-09-10T01:52:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9210.156394 secs 2023-09-10T01:52:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9210.868072 secs 2023-09-10T01:52:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:52:59 smithi170 ceph-mon[102832]: pgmap v5076: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T01:52:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9210.868072 secs 2023-09-10T01:52:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:52:59 smithi082 ceph-mon[98694]: pgmap v5076: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T01:52:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9210.868072 secs 2023-09-10T01:52:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:52:59 smithi098 ceph-mon[103124]: pgmap v5076: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T01:53:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:02 smithi170 ceph-mon[102832]: pgmap v5077: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 16 op/s 2023-09-10T01:53:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:02 smithi082 ceph-mon[98694]: pgmap v5077: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 16 op/s 2023-09-10T01:53:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:02 smithi098 ceph-mon[103124]: pgmap v5077: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 16 op/s 2023-09-10T01:53:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9213.729061 secs 2023-09-10T01:53:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9215.156450 secs 2023-09-10T01:53:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9213.729061 secs 2023-09-10T01:53:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9215.156450 secs 2023-09-10T01:53:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9213.729061 secs 2023-09-10T01:53:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9215.156450 secs 2023-09-10T01:53:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:04 smithi170 ceph-mon[102832]: pgmap v5078: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-10T01:53:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9215.868136 secs 2023-09-10T01:53:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:04 smithi082 ceph-mon[98694]: pgmap v5078: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-10T01:53:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9215.868136 secs 2023-09-10T01:53:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:04 smithi098 ceph-mon[103124]: pgmap v5078: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-10T01:53:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9215.868136 secs 2023-09-10T01:53:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:05 smithi170 ceph-mon[102832]: pgmap v5079: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-10T01:53:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:05 smithi082 ceph-mon[98694]: pgmap v5079: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-10T01:53:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:05 smithi098 ceph-mon[103124]: pgmap v5079: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-10T01:53:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:07 smithi170 ceph-mon[102832]: pgmap v5080: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 87 KiB/s wr, 30 op/s 2023-09-10T01:53:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:07 smithi082 ceph-mon[98694]: pgmap v5080: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 87 KiB/s wr, 30 op/s 2023-09-10T01:53:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:07 smithi098 ceph-mon[103124]: pgmap v5080: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 87 KiB/s wr, 30 op/s 2023-09-10T01:53:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9218.729148 secs 2023-09-10T01:53:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:08 smithi170 ceph-mon[102832]: osdmap e387: 12 total, 12 up, 12 in 2023-09-10T01:53:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9220.156510 secs 2023-09-10T01:53:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9218.729148 secs 2023-09-10T01:53:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:08 smithi082 ceph-mon[98694]: osdmap e387: 12 total, 12 up, 12 in 2023-09-10T01:53:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9220.156510 secs 2023-09-10T01:53:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9218.729148 secs 2023-09-10T01:53:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:08 smithi098 ceph-mon[103124]: osdmap e387: 12 total, 12 up, 12 in 2023-09-10T01:53:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9220.156510 secs 2023-09-10T01:53:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9220.868200 secs 2023-09-10T01:53:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:09 smithi170 ceph-mon[102832]: pgmap v5082: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-09-10T01:53:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9220.868200 secs 2023-09-10T01:53:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:09 smithi082 ceph-mon[98694]: pgmap v5082: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-09-10T01:53:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9220.868200 secs 2023-09-10T01:53:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:09 smithi098 ceph-mon[103124]: pgmap v5082: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 71 KiB/s wr, 29 op/s 2023-09-10T01:53:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:12 smithi170 ceph-mon[102832]: pgmap v5083: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:53:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:12 smithi082 ceph-mon[98694]: pgmap v5083: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:53:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:12 smithi098 ceph-mon[103124]: pgmap v5083: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:53:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9223.729208 secs 2023-09-10T01:53:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9225.156599 secs 2023-09-10T01:53:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9223.729208 secs 2023-09-10T01:53:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9225.156599 secs 2023-09-10T01:53:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9223.729208 secs 2023-09-10T01:53:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9225.156599 secs 2023-09-10T01:53:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:14 smithi170 ceph-mon[102832]: pgmap v5084: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:53:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9225.868264 secs 2023-09-10T01:53:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:14 smithi082 ceph-mon[98694]: pgmap v5084: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:53:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9225.868264 secs 2023-09-10T01:53:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:14 smithi098 ceph-mon[103124]: pgmap v5084: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:53:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9225.868264 secs 2023-09-10T01:53:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:15 smithi170 ceph-mon[102832]: pgmap v5085: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:53:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:15 smithi082 ceph-mon[98694]: pgmap v5085: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:53:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:15 smithi098 ceph-mon[103124]: pgmap v5085: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:53:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:17 smithi170 ceph-mon[102832]: osdmap e388: 12 total, 12 up, 12 in 2023-09-10T01:53:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:17 smithi082 ceph-mon[98694]: osdmap e388: 12 total, 12 up, 12 in 2023-09-10T01:53:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:17 smithi098 ceph-mon[103124]: osdmap e388: 12 total, 12 up, 12 in 2023-09-10T01:53:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:18 smithi170 ceph-mon[102832]: pgmap v5087: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T01:53:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9228.729306 secs 2023-09-10T01:53:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9230.156685 secs 2023-09-10T01:53:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:18 smithi082 ceph-mon[98694]: pgmap v5087: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T01:53:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9228.729306 secs 2023-09-10T01:53:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9230.156685 secs 2023-09-10T01:53:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:18 smithi098 ceph-mon[103124]: pgmap v5087: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T01:53:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9228.729306 secs 2023-09-10T01:53:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9230.156685 secs 2023-09-10T01:53:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9230.868325 secs 2023-09-10T01:53:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9230.868325 secs 2023-09-10T01:53:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9230.868325 secs 2023-09-10T01:53:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:20 smithi170 ceph-mon[102832]: pgmap v5088: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T01:53:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:20 smithi082 ceph-mon[98694]: pgmap v5088: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T01:53:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:20 smithi098 ceph-mon[103124]: pgmap v5088: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T01:53:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:22 smithi170 ceph-mon[102832]: pgmap v5089: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T01:53:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:22 smithi082 ceph-mon[98694]: pgmap v5089: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T01:53:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:22 smithi098 ceph-mon[103124]: pgmap v5089: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T01:53:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9233.729396 secs 2023-09-10T01:53:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9235.156741 secs 2023-09-10T01:53:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9233.729396 secs 2023-09-10T01:53:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9235.156741 secs 2023-09-10T01:53:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9233.729396 secs 2023-09-10T01:53:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9235.156741 secs 2023-09-10T01:53:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:24 smithi170 ceph-mon[102832]: pgmap v5090: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T01:53:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9235.868392 secs 2023-09-10T01:53:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:24 smithi082 ceph-mon[98694]: pgmap v5090: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T01:53:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9235.868392 secs 2023-09-10T01:53:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:24 smithi098 ceph-mon[103124]: pgmap v5090: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T01:53:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9235.868392 secs 2023-09-10T01:53:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:25 smithi170 ceph-mon[102832]: pgmap v5091: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T01:53:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:25 smithi082 ceph-mon[98694]: pgmap v5091: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T01:53:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:25 smithi098 ceph-mon[103124]: pgmap v5091: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T01:53:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:27 smithi170 ceph-mon[102832]: pgmap v5092: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T01:53:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:27 smithi082 ceph-mon[98694]: pgmap v5092: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T01:53:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:27 smithi098 ceph-mon[103124]: pgmap v5092: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T01:53:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9238.729506 secs 2023-09-10T01:53:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9240.156800 secs 2023-09-10T01:53:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9238.729506 secs 2023-09-10T01:53:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9240.156800 secs 2023-09-10T01:53:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9238.729506 secs 2023-09-10T01:53:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9240.156800 secs 2023-09-10T01:53:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9240.868457 secs 2023-09-10T01:53:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:29 smithi170 ceph-mon[102832]: pgmap v5093: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 10 op/s 2023-09-10T01:53:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9240.868457 secs 2023-09-10T01:53:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:29 smithi082 ceph-mon[98694]: pgmap v5093: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 10 op/s 2023-09-10T01:53:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9240.868457 secs 2023-09-10T01:53:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:29 smithi098 ceph-mon[103124]: pgmap v5093: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 10 op/s 2023-09-10T01:53:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:32 smithi170 ceph-mon[102832]: pgmap v5094: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 111 KiB/s wr, 16 op/s 2023-09-10T01:53:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:32 smithi082 ceph-mon[98694]: pgmap v5094: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 111 KiB/s wr, 16 op/s 2023-09-10T01:53:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:32 smithi098 ceph-mon[103124]: pgmap v5094: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 111 KiB/s wr, 16 op/s 2023-09-10T01:53:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9243.729566 secs 2023-09-10T01:53:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9245.156863 secs 2023-09-10T01:53:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9243.729566 secs 2023-09-10T01:53:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9245.156863 secs 2023-09-10T01:53:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9243.729566 secs 2023-09-10T01:53:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9245.156863 secs 2023-09-10T01:53:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:34 smithi170 ceph-mon[102832]: pgmap v5095: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T01:53:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9245.868487 secs 2023-09-10T01:53:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:34 smithi082 ceph-mon[98694]: pgmap v5095: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T01:53:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9245.868487 secs 2023-09-10T01:53:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:34 smithi098 ceph-mon[103124]: pgmap v5095: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T01:53:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9245.868487 secs 2023-09-10T01:53:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:35 smithi170 ceph-mon[102832]: pgmap v5096: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T01:53:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:35 smithi082 ceph-mon[98694]: pgmap v5096: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T01:53:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:35 smithi098 ceph-mon[103124]: pgmap v5096: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T01:53:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:37 smithi170 ceph-mon[102832]: pgmap v5097: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 143 KiB/s wr, 17 op/s 2023-09-10T01:53:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:37 smithi082 ceph-mon[98694]: pgmap v5097: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 143 KiB/s wr, 17 op/s 2023-09-10T01:53:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:37 smithi098 ceph-mon[103124]: pgmap v5097: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 143 KiB/s wr, 17 op/s 2023-09-10T01:53:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9250.156921 secs 2023-09-10T01:53:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9250.156921 secs 2023-09-10T01:53:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9250.156921 secs 2023-09-10T01:53:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9250.868556 secs 2023-09-10T01:53:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:53:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:39 smithi170 ceph-mon[102832]: pgmap v5098: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T01:53:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9250.868556 secs 2023-09-10T01:53:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:53:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:39 smithi082 ceph-mon[98694]: pgmap v5098: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T01:53:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9250.868556 secs 2023-09-10T01:53:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:53:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:39 smithi098 ceph-mon[103124]: pgmap v5098: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T01:53:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:53:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:53:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:53:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:53:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:53:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:53:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:53:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:53:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:53:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:42 smithi170 ceph-mon[102832]: pgmap v5099: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 133 KiB/s wr, 17 op/s 2023-09-10T01:53:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:42 smithi082 ceph-mon[98694]: pgmap v5099: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 133 KiB/s wr, 17 op/s 2023-09-10T01:53:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:42 smithi098 ceph-mon[103124]: pgmap v5099: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 133 KiB/s wr, 17 op/s 2023-09-10T01:53:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9255.157004 secs 2023-09-10T01:53:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9255.157004 secs 2023-09-10T01:53:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9255.157004 secs 2023-09-10T01:53:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:44 smithi170 ceph-mon[102832]: pgmap v5100: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T01:53:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9255.868621 secs 2023-09-10T01:53:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:44 smithi082 ceph-mon[98694]: pgmap v5100: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T01:53:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9255.868621 secs 2023-09-10T01:53:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:44 smithi098 ceph-mon[103124]: pgmap v5100: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T01:53:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9255.868621 secs 2023-09-10T01:53:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:45 smithi170 ceph-mon[102832]: pgmap v5101: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T01:53:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:45 smithi082 ceph-mon[98694]: pgmap v5101: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T01:53:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:45 smithi098 ceph-mon[103124]: pgmap v5101: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T01:53:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:48 smithi170 ceph-mon[102832]: pgmap v5102: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 140 KiB/s wr, 17 op/s 2023-09-10T01:53:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9258.729835 secs 2023-09-10T01:53:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9260.157061 secs 2023-09-10T01:53:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:48 smithi082 ceph-mon[98694]: pgmap v5102: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 140 KiB/s wr, 17 op/s 2023-09-10T01:53:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9258.729835 secs 2023-09-10T01:53:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9260.157061 secs 2023-09-10T01:53:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:48 smithi098 ceph-mon[103124]: pgmap v5102: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 140 KiB/s wr, 17 op/s 2023-09-10T01:53:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9258.729835 secs 2023-09-10T01:53:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9260.157061 secs 2023-09-10T01:53:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9260.868687 secs 2023-09-10T01:53:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9260.868687 secs 2023-09-10T01:53:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9260.868687 secs 2023-09-10T01:53:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:50 smithi170 ceph-mon[102832]: pgmap v5103: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T01:53:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:50 smithi082 ceph-mon[98694]: pgmap v5103: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T01:53:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:50 smithi098 ceph-mon[103124]: pgmap v5103: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T01:53:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:52 smithi170 ceph-mon[102832]: pgmap v5104: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 114 KiB/s wr, 16 op/s 2023-09-10T01:53:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:53:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:53:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:53:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:52 smithi082 ceph-mon[98694]: pgmap v5104: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 114 KiB/s wr, 16 op/s 2023-09-10T01:53:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:53:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:53:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:53:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:52 smithi098 ceph-mon[103124]: pgmap v5104: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 114 KiB/s wr, 16 op/s 2023-09-10T01:53:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:53:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:53:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:53:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9263.729961 secs 2023-09-10T01:53:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9265.157143 secs 2023-09-10T01:53:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:53 smithi170 ceph-mon[102832]: pgmap v5105: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T01:53:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9263.729961 secs 2023-09-10T01:53:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9265.157143 secs 2023-09-10T01:53:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:53 smithi082 ceph-mon[98694]: pgmap v5105: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T01:53:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9263.729961 secs 2023-09-10T01:53:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9265.157143 secs 2023-09-10T01:53:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:53 smithi098 ceph-mon[103124]: pgmap v5105: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T01:53:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:53:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9265.868755 secs 2023-09-10T01:53:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:53:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:53:54.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:53:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:53:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9265.868755 secs 2023-09-10T01:53:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:53:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:53:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:53:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:53:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9265.868755 secs 2023-09-10T01:53:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:53:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:53:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:53:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:55 smithi170 ceph-mon[102832]: pgmap v5106: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T01:53:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:55 smithi082 ceph-mon[98694]: pgmap v5106: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T01:53:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:55 smithi098 ceph-mon[103124]: pgmap v5106: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T01:53:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:57 smithi170 ceph-mon[102832]: pgmap v5107: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 109 KiB/s wr, 16 op/s 2023-09-10T01:53:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:57 smithi082 ceph-mon[98694]: pgmap v5107: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 109 KiB/s wr, 16 op/s 2023-09-10T01:53:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:57 smithi098 ceph-mon[103124]: pgmap v5107: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 109 KiB/s wr, 16 op/s 2023-09-10T01:53:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9268.730052 secs 2023-09-10T01:53:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9270.157228 secs 2023-09-10T01:53:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9268.730052 secs 2023-09-10T01:53:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9270.157228 secs 2023-09-10T01:53:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9268.730052 secs 2023-09-10T01:53:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9270.157228 secs 2023-09-10T01:53:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9270.868822 secs 2023-09-10T01:53:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:53:59 smithi170 ceph-mon[102832]: pgmap v5108: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T01:53:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9270.868822 secs 2023-09-10T01:53:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:53:59 smithi082 ceph-mon[98694]: pgmap v5108: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T01:53:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9270.868822 secs 2023-09-10T01:53:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:53:59 smithi098 ceph-mon[103124]: pgmap v5108: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T01:54:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:01 smithi170 ceph-mon[102832]: pgmap v5109: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 74 KiB/s wr, 16 op/s 2023-09-10T01:54:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:01 smithi082 ceph-mon[98694]: pgmap v5109: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 74 KiB/s wr, 16 op/s 2023-09-10T01:54:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:01 smithi098 ceph-mon[103124]: pgmap v5109: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 74 KiB/s wr, 16 op/s 2023-09-10T01:54:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9273.730158 secs 2023-09-10T01:54:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9273.730158 secs 2023-09-10T01:54:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9273.730158 secs 2023-09-10T01:54:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9275.157284 secs 2023-09-10T01:54:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:03 smithi170 ceph-mon[102832]: pgmap v5110: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T01:54:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9275.157284 secs 2023-09-10T01:54:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:03 smithi082 ceph-mon[98694]: pgmap v5110: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T01:54:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9275.157284 secs 2023-09-10T01:54:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:03 smithi098 ceph-mon[103124]: pgmap v5110: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T01:54:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9275.868889 secs 2023-09-10T01:54:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9275.868889 secs 2023-09-10T01:54:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9275.868889 secs 2023-09-10T01:54:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:05 smithi170 ceph-mon[102832]: pgmap v5111: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T01:54:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:05 smithi082 ceph-mon[98694]: pgmap v5111: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T01:54:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:05 smithi098 ceph-mon[103124]: pgmap v5111: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T01:54:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:07 smithi170 ceph-mon[102832]: pgmap v5112: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 95 KiB/s wr, 30 op/s 2023-09-10T01:54:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:07 smithi082 ceph-mon[98694]: pgmap v5112: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 95 KiB/s wr, 30 op/s 2023-09-10T01:54:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:07 smithi098 ceph-mon[103124]: pgmap v5112: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 95 KiB/s wr, 30 op/s 2023-09-10T01:54:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9278.730239 secs 2023-09-10T01:54:08.476 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9280.157374 secs 2023-09-10T01:54:08.476 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:08 smithi170 ceph-mon[102832]: osdmap e389: 12 total, 12 up, 12 in 2023-09-10T01:54:08.477 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9278.730239 secs 2023-09-10T01:54:08.477 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9280.157374 secs 2023-09-10T01:54:08.477 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:08 smithi082 ceph-mon[98694]: osdmap e389: 12 total, 12 up, 12 in 2023-09-10T01:54:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9278.730239 secs 2023-09-10T01:54:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9280.157374 secs 2023-09-10T01:54:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:08 smithi098 ceph-mon[103124]: osdmap e389: 12 total, 12 up, 12 in 2023-09-10T01:54:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9280.868955 secs 2023-09-10T01:54:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:09 smithi170 ceph-mon[102832]: pgmap v5114: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 30 op/s 2023-09-10T01:54:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9280.868955 secs 2023-09-10T01:54:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:09 smithi082 ceph-mon[98694]: pgmap v5114: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 30 op/s 2023-09-10T01:54:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9280.868955 secs 2023-09-10T01:54:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:09 smithi098 ceph-mon[103124]: pgmap v5114: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 30 op/s 2023-09-10T01:54:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:12 smithi170 ceph-mon[102832]: pgmap v5115: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 94 KiB/s wr, 30 op/s 2023-09-10T01:54:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:12 smithi082 ceph-mon[98694]: pgmap v5115: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 94 KiB/s wr, 30 op/s 2023-09-10T01:54:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:12 smithi098 ceph-mon[103124]: pgmap v5115: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 94 KiB/s wr, 30 op/s 2023-09-10T01:54:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9283.730299 secs 2023-09-10T01:54:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9285.157430 secs 2023-09-10T01:54:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9283.730299 secs 2023-09-10T01:54:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9285.157430 secs 2023-09-10T01:54:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9283.730299 secs 2023-09-10T01:54:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9285.157430 secs 2023-09-10T01:54:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:14 smithi170 ceph-mon[102832]: pgmap v5116: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 94 KiB/s wr, 30 op/s 2023-09-10T01:54:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9285.869019 secs 2023-09-10T01:54:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:14 smithi082 ceph-mon[98694]: pgmap v5116: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 94 KiB/s wr, 30 op/s 2023-09-10T01:54:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9285.869019 secs 2023-09-10T01:54:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:14 smithi098 ceph-mon[103124]: pgmap v5116: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 94 KiB/s wr, 30 op/s 2023-09-10T01:54:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9285.869019 secs 2023-09-10T01:54:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:16 smithi170 ceph-mon[102832]: pgmap v5117: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 94 KiB/s wr, 30 op/s 2023-09-10T01:54:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:16 smithi170 ceph-mon[102832]: osdmap e390: 12 total, 12 up, 12 in 2023-09-10T01:54:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:16 smithi082 ceph-mon[98694]: pgmap v5117: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 94 KiB/s wr, 30 op/s 2023-09-10T01:54:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:16 smithi082 ceph-mon[98694]: osdmap e390: 12 total, 12 up, 12 in 2023-09-10T01:54:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:16 smithi098 ceph-mon[103124]: pgmap v5117: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 94 KiB/s wr, 30 op/s 2023-09-10T01:54:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:16 smithi098 ceph-mon[103124]: osdmap e390: 12 total, 12 up, 12 in 2023-09-10T01:54:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:18 smithi170 ceph-mon[102832]: pgmap v5119: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 13 op/s 2023-09-10T01:54:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9288.730359 secs 2023-09-10T01:54:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9290.157514 secs 2023-09-10T01:54:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:18 smithi082 ceph-mon[98694]: pgmap v5119: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 13 op/s 2023-09-10T01:54:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9288.730359 secs 2023-09-10T01:54:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9290.157514 secs 2023-09-10T01:54:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:18 smithi098 ceph-mon[103124]: pgmap v5119: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 13 op/s 2023-09-10T01:54:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9288.730359 secs 2023-09-10T01:54:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9290.157514 secs 2023-09-10T01:54:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9290.869084 secs 2023-09-10T01:54:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9290.869084 secs 2023-09-10T01:54:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9290.869084 secs 2023-09-10T01:54:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:20 smithi170 ceph-mon[102832]: pgmap v5120: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T01:54:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:20 smithi082 ceph-mon[98694]: pgmap v5120: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T01:54:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:20 smithi098 ceph-mon[103124]: pgmap v5120: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T01:54:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:22 smithi170 ceph-mon[102832]: pgmap v5121: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T01:54:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:22 smithi082 ceph-mon[98694]: pgmap v5121: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T01:54:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:22 smithi098 ceph-mon[103124]: pgmap v5121: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T01:54:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:54:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9295.157575 secs 2023-09-10T01:54:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:54:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9295.157575 secs 2023-09-10T01:54:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:54:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9295.157575 secs 2023-09-10T01:54:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:24 smithi170 ceph-mon[102832]: pgmap v5122: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T01:54:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:54:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:54:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:54:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9295.869149 secs 2023-09-10T01:54:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:24 smithi082 ceph-mon[98694]: pgmap v5122: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T01:54:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:54:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:54:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:54:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9295.869149 secs 2023-09-10T01:54:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:24 smithi098 ceph-mon[103124]: pgmap v5122: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T01:54:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:54:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:54:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:54:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9295.869149 secs 2023-09-10T01:54:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:26 smithi170 ceph-mon[102832]: pgmap v5123: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T01:54:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:26 smithi082 ceph-mon[98694]: pgmap v5123: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T01:54:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:26 smithi098 ceph-mon[103124]: pgmap v5123: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T01:54:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:28 smithi170 ceph-mon[102832]: pgmap v5124: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T01:54:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9300.157660 secs 2023-09-10T01:54:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:28 smithi082 ceph-mon[98694]: pgmap v5124: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T01:54:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9300.157660 secs 2023-09-10T01:54:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:28 smithi098 ceph-mon[103124]: pgmap v5124: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T01:54:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9300.157660 secs 2023-09-10T01:54:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9300.869212 secs 2023-09-10T01:54:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9300.869212 secs 2023-09-10T01:54:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9300.869212 secs 2023-09-10T01:54:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:30 smithi170 ceph-mon[102832]: pgmap v5125: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 10 op/s 2023-09-10T01:54:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:30 smithi082 ceph-mon[98694]: pgmap v5125: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 10 op/s 2023-09-10T01:54:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:30 smithi098 ceph-mon[103124]: pgmap v5125: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 10 op/s 2023-09-10T01:54:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:32 smithi170 ceph-mon[102832]: pgmap v5126: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 119 KiB/s wr, 16 op/s 2023-09-10T01:54:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:32 smithi082 ceph-mon[98694]: pgmap v5126: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 119 KiB/s wr, 16 op/s 2023-09-10T01:54:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:32 smithi098 ceph-mon[103124]: pgmap v5126: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 119 KiB/s wr, 16 op/s 2023-09-10T01:54:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9303.730541 secs 2023-09-10T01:54:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9305.157760 secs 2023-09-10T01:54:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9303.730541 secs 2023-09-10T01:54:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9305.157760 secs 2023-09-10T01:54:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9303.730541 secs 2023-09-10T01:54:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9305.157760 secs 2023-09-10T01:54:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:34 smithi170 ceph-mon[102832]: pgmap v5127: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T01:54:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9305.869280 secs 2023-09-10T01:54:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:34 smithi082 ceph-mon[98694]: pgmap v5127: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T01:54:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9305.869280 secs 2023-09-10T01:54:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:34 smithi098 ceph-mon[103124]: pgmap v5127: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T01:54:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9305.869280 secs 2023-09-10T01:54:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:35 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:54:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:35 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:54:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:35 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:54:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:35 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:54:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:35 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:54:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:35 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:54:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:35 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:54:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:35 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:54:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:35 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:54:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:36 smithi082 ceph-mon[98694]: pgmap v5128: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T01:54:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:36 smithi098 ceph-mon[103124]: pgmap v5128: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T01:54:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:36 smithi170 ceph-mon[102832]: pgmap v5128: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T01:54:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9308.730602 secs 2023-09-10T01:54:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:38 smithi082 ceph-mon[98694]: pgmap v5129: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 126 KiB/s wr, 17 op/s 2023-09-10T01:54:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9310.157818 secs 2023-09-10T01:54:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9308.730602 secs 2023-09-10T01:54:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:38 smithi098 ceph-mon[103124]: pgmap v5129: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 126 KiB/s wr, 17 op/s 2023-09-10T01:54:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9310.157818 secs 2023-09-10T01:54:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9308.730602 secs 2023-09-10T01:54:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:38 smithi170 ceph-mon[102832]: pgmap v5129: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 126 KiB/s wr, 17 op/s 2023-09-10T01:54:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9310.157818 secs 2023-09-10T01:54:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9310.869342 secs 2023-09-10T01:54:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9310.869342 secs 2023-09-10T01:54:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9310.869342 secs 2023-09-10T01:54:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:40 smithi082 ceph-mon[98694]: pgmap v5130: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:54:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:40 smithi098 ceph-mon[103124]: pgmap v5130: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:54:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:40 smithi170 ceph-mon[102832]: pgmap v5130: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T01:54:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:42 smithi082 ceph-mon[98694]: pgmap v5131: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 143 KiB/s wr, 17 op/s 2023-09-10T01:54:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:42 smithi098 ceph-mon[103124]: pgmap v5131: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 143 KiB/s wr, 17 op/s 2023-09-10T01:54:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:42 smithi170 ceph-mon[102832]: pgmap v5131: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 143 KiB/s wr, 17 op/s 2023-09-10T01:54:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:54:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9315.157872 secs 2023-09-10T01:54:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:54:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9315.157872 secs 2023-09-10T01:54:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:54:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9315.157872 secs 2023-09-10T01:54:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:44 smithi082 ceph-mon[98694]: pgmap v5132: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T01:54:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:44 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:54:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:54:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:54:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9315.869407 secs 2023-09-10T01:54:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:44 smithi098 ceph-mon[103124]: pgmap v5132: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T01:54:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:44 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:54:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:54:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:54:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9315.869407 secs 2023-09-10T01:54:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:44 smithi170 ceph-mon[102832]: pgmap v5132: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T01:54:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:44 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:54:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:54:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:54:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9315.869407 secs 2023-09-10T01:54:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:46 smithi098 ceph-mon[103124]: pgmap v5133: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T01:54:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:46 smithi170 ceph-mon[102832]: pgmap v5133: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T01:54:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:46 smithi082 ceph-mon[98694]: pgmap v5133: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T01:54:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9318.730828 secs 2023-09-10T01:54:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:48 smithi098 ceph-mon[103124]: pgmap v5134: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 135 KiB/s wr, 17 op/s 2023-09-10T01:54:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9320.157931 secs 2023-09-10T01:54:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9318.730828 secs 2023-09-10T01:54:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:48 smithi170 ceph-mon[102832]: pgmap v5134: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 135 KiB/s wr, 17 op/s 2023-09-10T01:54:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9320.157931 secs 2023-09-10T01:54:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9318.730828 secs 2023-09-10T01:54:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:48 smithi082 ceph-mon[98694]: pgmap v5134: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 135 KiB/s wr, 17 op/s 2023-09-10T01:54:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9320.157931 secs 2023-09-10T01:54:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9320.869475 secs 2023-09-10T01:54:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9320.869475 secs 2023-09-10T01:54:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9320.869475 secs 2023-09-10T01:54:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:50 smithi098 ceph-mon[103124]: pgmap v5135: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T01:54:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:50 smithi170 ceph-mon[102832]: pgmap v5135: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T01:54:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:50 smithi082 ceph-mon[98694]: pgmap v5135: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T01:54:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:52 smithi170 ceph-mon[102832]: pgmap v5136: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 114 KiB/s wr, 16 op/s 2023-09-10T01:54:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:54:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:54:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:54:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:52 smithi082 ceph-mon[98694]: pgmap v5136: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 114 KiB/s wr, 16 op/s 2023-09-10T01:54:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:54:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:54:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:54:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:52 smithi098 ceph-mon[103124]: pgmap v5136: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 114 KiB/s wr, 16 op/s 2023-09-10T01:54:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:54:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:54:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:54:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9323.730890 secs 2023-09-10T01:54:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9325.157989 secs 2023-09-10T01:54:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9323.730890 secs 2023-09-10T01:54:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9325.157989 secs 2023-09-10T01:54:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9323.730890 secs 2023-09-10T01:54:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9325.157989 secs 2023-09-10T01:54:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:54 smithi170 ceph-mon[102832]: pgmap v5137: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T01:54:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9325.869542 secs 2023-09-10T01:54:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:54 smithi082 ceph-mon[98694]: pgmap v5137: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T01:54:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9325.869542 secs 2023-09-10T01:54:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:54 smithi098 ceph-mon[103124]: pgmap v5137: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T01:54:54.555 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9325.869542 secs 2023-09-10T01:54:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:54:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:54:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:54:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:54:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:54:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:54:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:54:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:54:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:54:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:54:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:54:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:54:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:56 smithi170 ceph-mon[102832]: pgmap v5138: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T01:54:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:56 smithi082 ceph-mon[98694]: pgmap v5138: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T01:54:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:56 smithi098 ceph-mon[103124]: pgmap v5138: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T01:54:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9328.730953 secs 2023-09-10T01:54:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9328.730953 secs 2023-09-10T01:54:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9328.730953 secs 2023-09-10T01:54:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:58 smithi170 ceph-mon[102832]: pgmap v5139: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 101 KiB/s wr, 15 op/s 2023-09-10T01:54:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9330.158047 secs 2023-09-10T01:54:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:58 smithi082 ceph-mon[98694]: pgmap v5139: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 101 KiB/s wr, 15 op/s 2023-09-10T01:54:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9330.158047 secs 2023-09-10T01:54:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:58 smithi098 ceph-mon[103124]: pgmap v5139: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 101 KiB/s wr, 15 op/s 2023-09-10T01:54:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9330.158047 secs 2023-09-10T01:55:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:54:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9330.869605 secs 2023-09-10T01:55:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:54:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9330.869605 secs 2023-09-10T01:55:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:54:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9330.869605 secs 2023-09-10T01:55:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:00 smithi170 ceph-mon[102832]: pgmap v5140: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 10 op/s 2023-09-10T01:55:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:00 smithi082 ceph-mon[98694]: pgmap v5140: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 10 op/s 2023-09-10T01:55:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:00 smithi098 ceph-mon[103124]: pgmap v5140: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 10 op/s 2023-09-10T01:55:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:02 smithi170 ceph-mon[102832]: pgmap v5141: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 15 op/s 2023-09-10T01:55:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9333.731061 secs 2023-09-10T01:55:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:02 smithi082 ceph-mon[98694]: pgmap v5141: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 15 op/s 2023-09-10T01:55:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9333.731061 secs 2023-09-10T01:55:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:02 smithi098 ceph-mon[103124]: pgmap v5141: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 15 op/s 2023-09-10T01:55:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9333.731061 secs 2023-09-10T01:55:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9335.158106 secs 2023-09-10T01:55:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9335.158106 secs 2023-09-10T01:55:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9335.158106 secs 2023-09-10T01:55:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:04 smithi170 ceph-mon[102832]: pgmap v5142: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T01:55:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9335.869672 secs 2023-09-10T01:55:05.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:04 smithi082 ceph-mon[98694]: pgmap v5142: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T01:55:05.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9335.869672 secs 2023-09-10T01:55:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:04 smithi098 ceph-mon[103124]: pgmap v5142: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T01:55:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9335.869672 secs 2023-09-10T01:55:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:06 smithi170 ceph-mon[102832]: pgmap v5143: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T01:55:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:06 smithi082 ceph-mon[98694]: pgmap v5143: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T01:55:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:06 smithi098 ceph-mon[103124]: pgmap v5143: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 10 op/s 2023-09-10T01:55:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9338.731125 secs 2023-09-10T01:55:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9338.731125 secs 2023-09-10T01:55:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9338.731125 secs 2023-09-10T01:55:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:08 smithi170 ceph-mon[102832]: pgmap v5144: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T01:55:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:08 smithi170 ceph-mon[102832]: osdmap e391: 12 total, 12 up, 12 in 2023-09-10T01:55:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9340.158165 secs 2023-09-10T01:55:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:08 smithi082 ceph-mon[98694]: pgmap v5144: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T01:55:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:08 smithi082 ceph-mon[98694]: osdmap e391: 12 total, 12 up, 12 in 2023-09-10T01:55:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9340.158165 secs 2023-09-10T01:55:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:08 smithi098 ceph-mon[103124]: pgmap v5144: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T01:55:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:08 smithi098 ceph-mon[103124]: osdmap e391: 12 total, 12 up, 12 in 2023-09-10T01:55:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9340.158165 secs 2023-09-10T01:55:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9340.869734 secs 2023-09-10T01:55:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9340.869734 secs 2023-09-10T01:55:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9340.869734 secs 2023-09-10T01:55:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:10 smithi170 ceph-mon[102832]: pgmap v5146: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-09-10T01:55:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:10 smithi082 ceph-mon[98694]: pgmap v5146: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-09-10T01:55:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:10 smithi098 ceph-mon[103124]: pgmap v5146: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-09-10T01:55:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:12 smithi170 ceph-mon[102832]: pgmap v5147: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T01:55:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9343.731235 secs 2023-09-10T01:55:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:12 smithi082 ceph-mon[98694]: pgmap v5147: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T01:55:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9343.731235 secs 2023-09-10T01:55:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:12 smithi098 ceph-mon[103124]: pgmap v5147: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T01:55:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9343.731235 secs 2023-09-10T01:55:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9345.158223 secs 2023-09-10T01:55:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9345.158223 secs 2023-09-10T01:55:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9345.158223 secs 2023-09-10T01:55:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:14 smithi170 ceph-mon[102832]: pgmap v5148: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T01:55:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9345.869799 secs 2023-09-10T01:55:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:14 smithi082 ceph-mon[98694]: pgmap v5148: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T01:55:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9345.869799 secs 2023-09-10T01:55:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:14 smithi098 ceph-mon[103124]: pgmap v5148: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T01:55:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9345.869799 secs 2023-09-10T01:55:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:16 smithi170 ceph-mon[102832]: pgmap v5149: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T01:55:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:16 smithi170 ceph-mon[102832]: osdmap e392: 12 total, 12 up, 12 in 2023-09-10T01:55:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:16 smithi082 ceph-mon[98694]: pgmap v5149: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T01:55:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:16 smithi082 ceph-mon[98694]: osdmap e392: 12 total, 12 up, 12 in 2023-09-10T01:55:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:16 smithi098 ceph-mon[103124]: pgmap v5149: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T01:55:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:16 smithi098 ceph-mon[103124]: osdmap e392: 12 total, 12 up, 12 in 2023-09-10T01:55:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9348.731353 secs 2023-09-10T01:55:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9348.731353 secs 2023-09-10T01:55:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9348.731353 secs 2023-09-10T01:55:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:18 smithi170 ceph-mon[102832]: pgmap v5151: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 69 KiB/s wr, 14 op/s 2023-09-10T01:55:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9350.158309 secs 2023-09-10T01:55:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:18 smithi082 ceph-mon[98694]: pgmap v5151: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 69 KiB/s wr, 14 op/s 2023-09-10T01:55:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9350.158309 secs 2023-09-10T01:55:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:18 smithi098 ceph-mon[103124]: pgmap v5151: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 69 KiB/s wr, 14 op/s 2023-09-10T01:55:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9350.158309 secs 2023-09-10T01:55:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9350.869884 secs 2023-09-10T01:55:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9350.869884 secs 2023-09-10T01:55:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9350.869884 secs 2023-09-10T01:55:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:20 smithi170 ceph-mon[102832]: pgmap v5152: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T01:55:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:20 smithi082 ceph-mon[98694]: pgmap v5152: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T01:55:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:20 smithi098 ceph-mon[103124]: pgmap v5152: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T01:55:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:22 smithi170 ceph-mon[102832]: pgmap v5153: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 74 KiB/s wr, 12 op/s 2023-09-10T01:55:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:55:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:22 smithi082 ceph-mon[98694]: pgmap v5153: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 74 KiB/s wr, 12 op/s 2023-09-10T01:55:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:55:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:22 smithi098 ceph-mon[103124]: pgmap v5153: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 74 KiB/s wr, 12 op/s 2023-09-10T01:55:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:55:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:55:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:55:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:55:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9355.158394 secs 2023-09-10T01:55:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9355.869951 secs 2023-09-10T01:55:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:55:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:55:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:55:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9355.158394 secs 2023-09-10T01:55:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9355.869951 secs 2023-09-10T01:55:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:55:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:55:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:55:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9355.158394 secs 2023-09-10T01:55:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9355.869951 secs 2023-09-10T01:55:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:24 smithi170 ceph-mon[102832]: pgmap v5154: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 74 KiB/s wr, 12 op/s 2023-09-10T01:55:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:24 smithi082 ceph-mon[98694]: pgmap v5154: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 74 KiB/s wr, 12 op/s 2023-09-10T01:55:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:24 smithi098 ceph-mon[103124]: pgmap v5154: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 74 KiB/s wr, 12 op/s 2023-09-10T01:55:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:26 smithi170 ceph-mon[102832]: pgmap v5155: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 74 KiB/s wr, 12 op/s 2023-09-10T01:55:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:26 smithi082 ceph-mon[98694]: pgmap v5155: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 74 KiB/s wr, 12 op/s 2023-09-10T01:55:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:26 smithi098 ceph-mon[103124]: pgmap v5155: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 74 KiB/s wr, 12 op/s 2023-09-10T01:55:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:28 smithi170 ceph-mon[102832]: pgmap v5156: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T01:55:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9360.158452 secs 2023-09-10T01:55:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9360.870018 secs 2023-09-10T01:55:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:28 smithi082 ceph-mon[98694]: pgmap v5156: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T01:55:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9360.158452 secs 2023-09-10T01:55:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9360.870018 secs 2023-09-10T01:55:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:28 smithi098 ceph-mon[103124]: pgmap v5156: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T01:55:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9360.158452 secs 2023-09-10T01:55:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9360.870018 secs 2023-09-10T01:55:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:30 smithi170 ceph-mon[102832]: pgmap v5157: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 54 KiB/s wr, 10 op/s 2023-09-10T01:55:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:30 smithi082 ceph-mon[98694]: pgmap v5157: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 54 KiB/s wr, 10 op/s 2023-09-10T01:55:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:30 smithi098 ceph-mon[103124]: pgmap v5157: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 54 KiB/s wr, 10 op/s 2023-09-10T01:55:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:32 smithi170 ceph-mon[102832]: pgmap v5158: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 137 KiB/s wr, 16 op/s 2023-09-10T01:55:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9363.731578 secs 2023-09-10T01:55:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:32 smithi082 ceph-mon[98694]: pgmap v5158: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 137 KiB/s wr, 16 op/s 2023-09-10T01:55:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9363.731578 secs 2023-09-10T01:55:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:32 smithi098 ceph-mon[103124]: pgmap v5158: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 137 KiB/s wr, 16 op/s 2023-09-10T01:55:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9363.731578 secs 2023-09-10T01:55:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9365.158500 secs 2023-09-10T01:55:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9365.870081 secs 2023-09-10T01:55:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9365.158500 secs 2023-09-10T01:55:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9365.870081 secs 2023-09-10T01:55:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9365.158500 secs 2023-09-10T01:55:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9365.870081 secs 2023-09-10T01:55:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:34 smithi170 ceph-mon[102832]: pgmap v5159: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T01:55:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:34 smithi082 ceph-mon[98694]: pgmap v5159: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T01:55:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:34 smithi098 ceph-mon[103124]: pgmap v5159: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T01:55:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:35 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:55:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:35 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:55:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:35 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:55:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:35 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:55:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:35 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:55:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:35 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:55:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:35 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:55:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:35 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:55:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:35 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:55:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:36 smithi170 ceph-mon[102832]: pgmap v5160: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T01:55:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:36 smithi082 ceph-mon[98694]: pgmap v5160: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T01:55:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:36 smithi098 ceph-mon[103124]: pgmap v5160: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T01:55:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:38 smithi170 ceph-mon[102832]: pgmap v5161: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 107 KiB/s wr, 17 op/s 2023-09-10T01:55:39.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9370.158557 secs 2023-09-10T01:55:39.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9370.870147 secs 2023-09-10T01:55:39.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:38 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:55:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:38 smithi082 ceph-mon[98694]: pgmap v5161: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 107 KiB/s wr, 17 op/s 2023-09-10T01:55:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9370.158557 secs 2023-09-10T01:55:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9370.870147 secs 2023-09-10T01:55:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:38 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:55:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:38 smithi098 ceph-mon[103124]: pgmap v5161: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 107 KiB/s wr, 17 op/s 2023-09-10T01:55:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9370.158557 secs 2023-09-10T01:55:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9370.870147 secs 2023-09-10T01:55:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:38 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:55:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:39 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:55:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:39 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:55:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:39 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:55:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:39 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:55:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:39 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:55:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:39 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:55:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:40 smithi170 ceph-mon[102832]: pgmap v5162: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T01:55:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:40 smithi082 ceph-mon[98694]: pgmap v5162: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T01:55:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:40 smithi098 ceph-mon[103124]: pgmap v5162: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T01:55:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:41 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:55:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:41 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:55:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:41 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:55:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:42 smithi170 ceph-mon[102832]: pgmap v5163: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 174 KiB/s wr, 18 op/s 2023-09-10T01:55:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:42 smithi082 ceph-mon[98694]: pgmap v5163: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 174 KiB/s wr, 18 op/s 2023-09-10T01:55:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:42 smithi098 ceph-mon[103124]: pgmap v5163: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 174 KiB/s wr, 18 op/s 2023-09-10T01:55:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9375.158616 secs 2023-09-10T01:55:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9375.870216 secs 2023-09-10T01:55:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9375.158616 secs 2023-09-10T01:55:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9375.870216 secs 2023-09-10T01:55:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9375.158616 secs 2023-09-10T01:55:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9375.870216 secs 2023-09-10T01:55:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:44 smithi082 ceph-mon[98694]: pgmap v5164: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T01:55:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:44 smithi098 ceph-mon[103124]: pgmap v5164: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T01:55:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:44 smithi170 ceph-mon[102832]: pgmap v5164: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T01:55:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:46 smithi082 ceph-mon[98694]: pgmap v5165: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T01:55:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:46 smithi098 ceph-mon[103124]: pgmap v5165: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T01:55:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:46 smithi170 ceph-mon[102832]: pgmap v5165: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T01:55:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9378.731759 secs 2023-09-10T01:55:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9378.731759 secs 2023-09-10T01:55:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9378.731759 secs 2023-09-10T01:55:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:48 smithi082 ceph-mon[98694]: pgmap v5166: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 103 KiB/s wr, 17 op/s 2023-09-10T01:55:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9380.158673 secs 2023-09-10T01:55:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9380.870280 secs 2023-09-10T01:55:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:48 smithi098 ceph-mon[103124]: pgmap v5166: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 103 KiB/s wr, 17 op/s 2023-09-10T01:55:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9380.158673 secs 2023-09-10T01:55:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9380.870280 secs 2023-09-10T01:55:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:48 smithi170 ceph-mon[102832]: pgmap v5166: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 103 KiB/s wr, 17 op/s 2023-09-10T01:55:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9380.158673 secs 2023-09-10T01:55:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9380.870280 secs 2023-09-10T01:55:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:50 smithi082 ceph-mon[98694]: pgmap v5167: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T01:55:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:50 smithi098 ceph-mon[103124]: pgmap v5167: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T01:55:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:50 smithi170 ceph-mon[102832]: pgmap v5167: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T01:55:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:55:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:55:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:55:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:55:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:55:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:55:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:55:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:55:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:55:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:52 smithi098 ceph-mon[103124]: pgmap v5168: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 160 KiB/s wr, 17 op/s 2023-09-10T01:55:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9383.731856 secs 2023-09-10T01:55:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:52 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9385.158760 secs 2023-09-10T01:55:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:52 smithi170 ceph-mon[102832]: pgmap v5168: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 160 KiB/s wr, 17 op/s 2023-09-10T01:55:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9383.731856 secs 2023-09-10T01:55:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:52 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9385.158760 secs 2023-09-10T01:55:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:52 smithi082 ceph-mon[98694]: pgmap v5168: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 160 KiB/s wr, 17 op/s 2023-09-10T01:55:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9383.731856 secs 2023-09-10T01:55:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:52 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9385.158760 secs 2023-09-10T01:55:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9385.870343 secs 2023-09-10T01:55:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9385.870343 secs 2023-09-10T01:55:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9385.870343 secs 2023-09-10T01:55:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:54 smithi098 ceph-mon[103124]: pgmap v5169: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T01:55:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:55:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:54 smithi170 ceph-mon[102832]: pgmap v5169: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T01:55:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:55:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:54 smithi082 ceph-mon[98694]: pgmap v5169: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T01:55:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:55:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:55:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:55:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:55:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:55:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:55:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:55:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:55:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:55:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:55:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:56 smithi098 ceph-mon[103124]: pgmap v5170: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T01:55:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:56 smithi170 ceph-mon[102832]: pgmap v5170: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T01:55:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:56 smithi082 ceph-mon[98694]: pgmap v5170: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T01:55:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9388.731961 secs 2023-09-10T01:55:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:57 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9390.158817 secs 2023-09-10T01:55:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9388.731961 secs 2023-09-10T01:55:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:57 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9390.158817 secs 2023-09-10T01:55:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9388.731961 secs 2023-09-10T01:55:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:57 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9390.158817 secs 2023-09-10T01:55:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:58 smithi098 ceph-mon[103124]: pgmap v5171: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 82 KiB/s wr, 16 op/s 2023-09-10T01:55:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:55:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9390.870408 secs 2023-09-10T01:55:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:58 smithi170 ceph-mon[102832]: pgmap v5171: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 82 KiB/s wr, 16 op/s 2023-09-10T01:55:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:55:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9390.870408 secs 2023-09-10T01:55:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:58 smithi082 ceph-mon[98694]: pgmap v5171: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 82 KiB/s wr, 16 op/s 2023-09-10T01:55:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:55:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9390.870408 secs 2023-09-10T01:56:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:00 smithi098 ceph-mon[103124]: pgmap v5172: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T01:56:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:00 smithi170 ceph-mon[102832]: pgmap v5172: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T01:56:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:00 smithi082 ceph-mon[98694]: pgmap v5172: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T01:56:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:02 smithi098 ceph-mon[103124]: pgmap v5173: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 84 KiB/s wr, 15 op/s 2023-09-10T01:56:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9393.732033 secs 2023-09-10T01:56:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:02 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9395.158877 secs 2023-09-10T01:56:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:02 smithi170 ceph-mon[102832]: pgmap v5173: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 84 KiB/s wr, 15 op/s 2023-09-10T01:56:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9393.732033 secs 2023-09-10T01:56:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:02 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9395.158877 secs 2023-09-10T01:56:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:02 smithi082 ceph-mon[98694]: pgmap v5173: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 84 KiB/s wr, 15 op/s 2023-09-10T01:56:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9393.732033 secs 2023-09-10T01:56:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:02 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9395.158877 secs 2023-09-10T01:56:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9395.870481 secs 2023-09-10T01:56:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9395.870481 secs 2023-09-10T01:56:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9395.870481 secs 2023-09-10T01:56:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:04 smithi098 ceph-mon[103124]: pgmap v5174: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-10T01:56:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:04 smithi170 ceph-mon[102832]: pgmap v5174: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-10T01:56:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:04 smithi082 ceph-mon[98694]: pgmap v5174: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-10T01:56:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:06 smithi098 ceph-mon[103124]: pgmap v5175: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-10T01:56:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:06 smithi170 ceph-mon[102832]: pgmap v5175: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-10T01:56:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:06 smithi082 ceph-mon[98694]: pgmap v5175: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 15 KiB/s wr, 9 op/s 2023-09-10T01:56:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9398.732133 secs 2023-09-10T01:56:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9400.158937 secs 2023-09-10T01:56:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9398.732133 secs 2023-09-10T01:56:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:07 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9400.158937 secs 2023-09-10T01:56:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9398.732133 secs 2023-09-10T01:56:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9400.158937 secs 2023-09-10T01:56:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:09 smithi098 ceph-mon[103124]: pgmap v5176: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T01:56:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:09 smithi098 ceph-mon[103124]: osdmap e393: 12 total, 12 up, 12 in 2023-09-10T01:56:09.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9400.870547 secs 2023-09-10T01:56:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:09 smithi170 ceph-mon[102832]: pgmap v5176: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T01:56:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:09 smithi170 ceph-mon[102832]: osdmap e393: 12 total, 12 up, 12 in 2023-09-10T01:56:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9400.870547 secs 2023-09-10T01:56:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:09 smithi082 ceph-mon[98694]: pgmap v5176: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T01:56:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:09 smithi082 ceph-mon[98694]: osdmap e393: 12 total, 12 up, 12 in 2023-09-10T01:56:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9400.870547 secs 2023-09-10T01:56:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:11 smithi098 ceph-mon[103124]: pgmap v5178: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T01:56:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:11 smithi170 ceph-mon[102832]: pgmap v5178: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T01:56:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:11 smithi082 ceph-mon[98694]: pgmap v5178: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T01:56:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:13 smithi170 ceph-mon[102832]: pgmap v5179: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T01:56:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9403.732218 secs 2023-09-10T01:56:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9405.159015 secs 2023-09-10T01:56:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:13 smithi082 ceph-mon[98694]: pgmap v5179: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T01:56:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9403.732218 secs 2023-09-10T01:56:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9405.159015 secs 2023-09-10T01:56:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:13 smithi098 ceph-mon[103124]: pgmap v5179: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T01:56:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9403.732218 secs 2023-09-10T01:56:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9405.159015 secs 2023-09-10T01:56:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9405.870611 secs 2023-09-10T01:56:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9405.870611 secs 2023-09-10T01:56:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9405.870611 secs 2023-09-10T01:56:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:15 smithi170 ceph-mon[102832]: pgmap v5180: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T01:56:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:15 smithi082 ceph-mon[98694]: pgmap v5180: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T01:56:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:15 smithi098 ceph-mon[103124]: pgmap v5180: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T01:56:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:17 smithi170 ceph-mon[102832]: pgmap v5181: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T01:56:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:17 smithi170 ceph-mon[102832]: osdmap e394: 12 total, 12 up, 12 in 2023-09-10T01:56:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:17 smithi082 ceph-mon[98694]: pgmap v5181: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T01:56:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:17 smithi082 ceph-mon[98694]: osdmap e394: 12 total, 12 up, 12 in 2023-09-10T01:56:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:17 smithi098 ceph-mon[103124]: pgmap v5181: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T01:56:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:17 smithi098 ceph-mon[103124]: osdmap e394: 12 total, 12 up, 12 in 2023-09-10T01:56:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9408.732281 secs 2023-09-10T01:56:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9410.159073 secs 2023-09-10T01:56:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9408.732281 secs 2023-09-10T01:56:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9410.159073 secs 2023-09-10T01:56:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9408.732281 secs 2023-09-10T01:56:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9410.159073 secs 2023-09-10T01:56:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:19 smithi170 ceph-mon[102832]: pgmap v5183: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 43 KiB/s wr, 13 op/s 2023-09-10T01:56:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9410.870677 secs 2023-09-10T01:56:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:19 smithi082 ceph-mon[98694]: pgmap v5183: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 43 KiB/s wr, 13 op/s 2023-09-10T01:56:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9410.870677 secs 2023-09-10T01:56:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:19 smithi098 ceph-mon[103124]: pgmap v5183: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 43 KiB/s wr, 13 op/s 2023-09-10T01:56:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9410.870677 secs 2023-09-10T01:56:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:21 smithi170 ceph-mon[102832]: pgmap v5184: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T01:56:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:21 smithi082 ceph-mon[98694]: pgmap v5184: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T01:56:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:21 smithi098 ceph-mon[103124]: pgmap v5184: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T01:56:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:23 smithi170 ceph-mon[102832]: pgmap v5185: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 73 KiB/s wr, 12 op/s 2023-09-10T01:56:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:56:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9415.159130 secs 2023-09-10T01:56:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:23 smithi082 ceph-mon[98694]: pgmap v5185: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 73 KiB/s wr, 12 op/s 2023-09-10T01:56:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:56:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9415.159130 secs 2023-09-10T01:56:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:23 smithi098 ceph-mon[103124]: pgmap v5185: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 73 KiB/s wr, 12 op/s 2023-09-10T01:56:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:56:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9415.159130 secs 2023-09-10T01:56:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:56:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:56:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:56:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9415.870738 secs 2023-09-10T01:56:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:56:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:56:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:56:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9415.870738 secs 2023-09-10T01:56:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:56:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:56:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:56:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9415.870738 secs 2023-09-10T01:56:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:25 smithi170 ceph-mon[102832]: pgmap v5186: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 73 KiB/s wr, 12 op/s 2023-09-10T01:56:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:25 smithi082 ceph-mon[98694]: pgmap v5186: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 73 KiB/s wr, 12 op/s 2023-09-10T01:56:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:25 smithi098 ceph-mon[103124]: pgmap v5186: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 73 KiB/s wr, 12 op/s 2023-09-10T01:56:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:27 smithi170 ceph-mon[102832]: pgmap v5187: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 73 KiB/s wr, 12 op/s 2023-09-10T01:56:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:27 smithi082 ceph-mon[98694]: pgmap v5187: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 73 KiB/s wr, 12 op/s 2023-09-10T01:56:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:27 smithi098 ceph-mon[103124]: pgmap v5187: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 73 KiB/s wr, 12 op/s 2023-09-10T01:56:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9420.159209 secs 2023-09-10T01:56:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9420.159209 secs 2023-09-10T01:56:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9420.159209 secs 2023-09-10T01:56:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:29 smithi170 ceph-mon[102832]: pgmap v5188: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T01:56:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9420.870802 secs 2023-09-10T01:56:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:29 smithi082 ceph-mon[98694]: pgmap v5188: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T01:56:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9420.870802 secs 2023-09-10T01:56:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:29 smithi098 ceph-mon[103124]: pgmap v5188: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T01:56:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9420.870802 secs 2023-09-10T01:56:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:31 smithi170 ceph-mon[102832]: pgmap v5189: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 54 KiB/s wr, 10 op/s 2023-09-10T01:56:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:31 smithi082 ceph-mon[98694]: pgmap v5189: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 54 KiB/s wr, 10 op/s 2023-09-10T01:56:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:31 smithi098 ceph-mon[103124]: pgmap v5189: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 54 KiB/s wr, 10 op/s 2023-09-10T01:56:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:32 smithi170 ceph-mon[102832]: pgmap v5190: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 138 KiB/s wr, 17 op/s 2023-09-10T01:56:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:32 smithi082 ceph-mon[98694]: pgmap v5190: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 138 KiB/s wr, 17 op/s 2023-09-10T01:56:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:32 smithi098 ceph-mon[103124]: pgmap v5190: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 138 KiB/s wr, 17 op/s 2023-09-10T01:56:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9423.732572 secs 2023-09-10T01:56:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9425.159298 secs 2023-09-10T01:56:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9423.732572 secs 2023-09-10T01:56:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9425.159298 secs 2023-09-10T01:56:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9423.732572 secs 2023-09-10T01:56:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9425.159298 secs 2023-09-10T01:56:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:34 smithi170 ceph-mon[102832]: pgmap v5191: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T01:56:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9425.870866 secs 2023-09-10T01:56:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:34 smithi082 ceph-mon[98694]: pgmap v5191: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T01:56:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9425.870866 secs 2023-09-10T01:56:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:34 smithi098 ceph-mon[103124]: pgmap v5191: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T01:56:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9425.870866 secs 2023-09-10T01:56:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:35 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:56:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:35 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:56:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:35 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:56:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:35 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:56:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:35 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:56:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:35 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:56:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:35 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:56:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:35 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:56:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:35 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:56:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:36 smithi170 ceph-mon[102832]: pgmap v5192: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T01:56:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:36 smithi082 ceph-mon[98694]: pgmap v5192: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T01:56:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:36 smithi098 ceph-mon[103124]: pgmap v5192: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T01:56:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9428.732632 secs 2023-09-10T01:56:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:38 smithi170 ceph-mon[102832]: pgmap v5193: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 109 KiB/s wr, 17 op/s 2023-09-10T01:56:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9430.159387 secs 2023-09-10T01:56:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9428.732632 secs 2023-09-10T01:56:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:38 smithi082 ceph-mon[98694]: pgmap v5193: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 109 KiB/s wr, 17 op/s 2023-09-10T01:56:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9430.159387 secs 2023-09-10T01:56:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9428.732632 secs 2023-09-10T01:56:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:38 smithi098 ceph-mon[103124]: pgmap v5193: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 109 KiB/s wr, 17 op/s 2023-09-10T01:56:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9430.159387 secs 2023-09-10T01:56:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9430.870934 secs 2023-09-10T01:56:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9430.870934 secs 2023-09-10T01:56:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9430.870934 secs 2023-09-10T01:56:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:40 smithi170 ceph-mon[102832]: pgmap v5194: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T01:56:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:40 smithi082 ceph-mon[98694]: pgmap v5194: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T01:56:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:40 smithi098 ceph-mon[103124]: pgmap v5194: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T01:56:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:42 smithi170 ceph-mon[102832]: pgmap v5195: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 177 KiB/s wr, 18 op/s 2023-09-10T01:56:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:42 smithi082 ceph-mon[98694]: pgmap v5195: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 177 KiB/s wr, 18 op/s 2023-09-10T01:56:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:42 smithi098 ceph-mon[103124]: pgmap v5195: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 177 KiB/s wr, 18 op/s 2023-09-10T01:56:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9433.732705 secs 2023-09-10T01:56:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9435.159476 secs 2023-09-10T01:56:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9433.732705 secs 2023-09-10T01:56:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9435.159476 secs 2023-09-10T01:56:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9433.732705 secs 2023-09-10T01:56:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9435.159476 secs 2023-09-10T01:56:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:44 smithi170 ceph-mon[102832]: pgmap v5196: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T01:56:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9435.870999 secs 2023-09-10T01:56:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:44 smithi082 ceph-mon[98694]: pgmap v5196: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T01:56:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9435.870999 secs 2023-09-10T01:56:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:44 smithi098 ceph-mon[103124]: pgmap v5196: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T01:56:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9435.870999 secs 2023-09-10T01:56:46.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:46 smithi170 ceph-mon[102832]: pgmap v5197: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T01:56:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:46 smithi082 ceph-mon[98694]: pgmap v5197: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T01:56:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:46 smithi098 ceph-mon[103124]: pgmap v5197: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T01:56:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9438.732787 secs 2023-09-10T01:56:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:48 smithi170 ceph-mon[102832]: pgmap v5198: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 105 KiB/s wr, 17 op/s 2023-09-10T01:56:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9440.159492 secs 2023-09-10T01:56:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9438.732787 secs 2023-09-10T01:56:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:48 smithi082 ceph-mon[98694]: pgmap v5198: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 105 KiB/s wr, 17 op/s 2023-09-10T01:56:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9440.159492 secs 2023-09-10T01:56:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9438.732787 secs 2023-09-10T01:56:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:48 smithi098 ceph-mon[103124]: pgmap v5198: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 105 KiB/s wr, 17 op/s 2023-09-10T01:56:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9440.159492 secs 2023-09-10T01:56:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9440.871066 secs 2023-09-10T01:56:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9440.871066 secs 2023-09-10T01:56:49.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9440.871066 secs 2023-09-10T01:56:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:50 smithi170 ceph-mon[102832]: pgmap v5199: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T01:56:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:50 smithi082 ceph-mon[98694]: pgmap v5199: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T01:56:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:50 smithi098 ceph-mon[103124]: pgmap v5199: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T01:56:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:52 smithi170 ceph-mon[102832]: pgmap v5200: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T01:56:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:52 smithi082 ceph-mon[98694]: pgmap v5200: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T01:56:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:52 smithi098 ceph-mon[103124]: pgmap v5200: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T01:56:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9443.732846 secs 2023-09-10T01:56:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9445.159549 secs 2023-09-10T01:56:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9443.732846 secs 2023-09-10T01:56:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9445.159549 secs 2023-09-10T01:56:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9443.732846 secs 2023-09-10T01:56:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9445.159549 secs 2023-09-10T01:56:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:54 smithi170 ceph-mon[102832]: pgmap v5201: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T01:56:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9445.871131 secs 2023-09-10T01:56:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:54 smithi082 ceph-mon[98694]: pgmap v5201: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T01:56:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9445.871131 secs 2023-09-10T01:56:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:54 smithi098 ceph-mon[103124]: pgmap v5201: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T01:56:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9445.871131 secs 2023-09-10T01:56:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:56 smithi170 ceph-mon[102832]: pgmap v5202: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T01:56:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:56 smithi082 ceph-mon[98694]: pgmap v5202: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T01:56:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:56 smithi098 ceph-mon[103124]: pgmap v5202: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T01:56:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:56:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:56:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:56:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:56:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:56:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:56:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:56:57.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:56:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:56:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:56:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:56:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:56:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9448.732905 secs 2023-09-10T01:56:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:58 smithi170 ceph-mon[102832]: pgmap v5203: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 74 KiB/s wr, 15 op/s 2023-09-10T01:56:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9450.159635 secs 2023-09-10T01:56:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9448.732905 secs 2023-09-10T01:56:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:58 smithi082 ceph-mon[98694]: pgmap v5203: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 74 KiB/s wr, 15 op/s 2023-09-10T01:56:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9450.159635 secs 2023-09-10T01:56:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9448.732905 secs 2023-09-10T01:56:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:58 smithi098 ceph-mon[103124]: pgmap v5203: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 74 KiB/s wr, 15 op/s 2023-09-10T01:56:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9450.159635 secs 2023-09-10T01:56:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:56:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9450.871197 secs 2023-09-10T01:56:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:56:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9450.871197 secs 2023-09-10T01:56:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:56:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9450.871197 secs 2023-09-10T01:57:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:00 smithi170 ceph-mon[102832]: pgmap v5204: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 62 KiB/s wr, 10 op/s 2023-09-10T01:57:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:00 smithi082 ceph-mon[98694]: pgmap v5204: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 62 KiB/s wr, 10 op/s 2023-09-10T01:57:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:00 smithi098 ceph-mon[103124]: pgmap v5204: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 62 KiB/s wr, 10 op/s 2023-09-10T01:57:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:02 smithi170 ceph-mon[102832]: pgmap v5205: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 15 op/s 2023-09-10T01:57:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:02 smithi082 ceph-mon[98694]: pgmap v5205: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 15 op/s 2023-09-10T01:57:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:02 smithi098 ceph-mon[103124]: pgmap v5205: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 15 op/s 2023-09-10T01:57:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9453.732986 secs 2023-09-10T01:57:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9455.159690 secs 2023-09-10T01:57:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9453.732986 secs 2023-09-10T01:57:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9455.159690 secs 2023-09-10T01:57:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9453.732986 secs 2023-09-10T01:57:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9455.159690 secs 2023-09-10T01:57:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:04 smithi170 ceph-mon[102832]: pgmap v5206: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-09-10T01:57:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9455.871285 secs 2023-09-10T01:57:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:04 smithi082 ceph-mon[98694]: pgmap v5206: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-09-10T01:57:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9455.871285 secs 2023-09-10T01:57:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:04 smithi098 ceph-mon[103124]: pgmap v5206: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-09-10T01:57:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9455.871285 secs 2023-09-10T01:57:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:06 smithi170 ceph-mon[102832]: pgmap v5207: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-09-10T01:57:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:06 smithi082 ceph-mon[98694]: pgmap v5207: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-09-10T01:57:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:06 smithi098 ceph-mon[103124]: pgmap v5207: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-09-10T01:57:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9458.733048 secs 2023-09-10T01:57:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:08 smithi170 ceph-mon[102832]: pgmap v5208: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T01:57:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:08 smithi170 ceph-mon[102832]: osdmap e395: 12 total, 12 up, 12 in 2023-09-10T01:57:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9460.159751 secs 2023-09-10T01:57:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9458.733048 secs 2023-09-10T01:57:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:08 smithi082 ceph-mon[98694]: pgmap v5208: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T01:57:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:08 smithi082 ceph-mon[98694]: osdmap e395: 12 total, 12 up, 12 in 2023-09-10T01:57:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9460.159751 secs 2023-09-10T01:57:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9458.733048 secs 2023-09-10T01:57:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:08 smithi098 ceph-mon[103124]: pgmap v5208: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T01:57:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:08 smithi098 ceph-mon[103124]: osdmap e395: 12 total, 12 up, 12 in 2023-09-10T01:57:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9460.159751 secs 2023-09-10T01:57:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9460.871351 secs 2023-09-10T01:57:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9460.871351 secs 2023-09-10T01:57:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9460.871351 secs 2023-09-10T01:57:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:10 smithi170 ceph-mon[102832]: pgmap v5210: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T01:57:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:10 smithi082 ceph-mon[98694]: pgmap v5210: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T01:57:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:10 smithi098 ceph-mon[103124]: pgmap v5210: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T01:57:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:12 smithi170 ceph-mon[102832]: pgmap v5211: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T01:57:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:12 smithi082 ceph-mon[98694]: pgmap v5211: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T01:57:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:12 smithi098 ceph-mon[103124]: pgmap v5211: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T01:57:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9463.733110 secs 2023-09-10T01:57:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9465.159815 secs 2023-09-10T01:57:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9463.733110 secs 2023-09-10T01:57:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9465.159815 secs 2023-09-10T01:57:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9463.733110 secs 2023-09-10T01:57:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9465.159815 secs 2023-09-10T01:57:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:14 smithi170 ceph-mon[102832]: pgmap v5212: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T01:57:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9465.871412 secs 2023-09-10T01:57:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:14 smithi082 ceph-mon[98694]: pgmap v5212: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T01:57:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9465.871412 secs 2023-09-10T01:57:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:14 smithi098 ceph-mon[103124]: pgmap v5212: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T01:57:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9465.871412 secs 2023-09-10T01:57:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:16 smithi170 ceph-mon[102832]: pgmap v5213: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T01:57:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:16 smithi170 ceph-mon[102832]: osdmap e396: 12 total, 12 up, 12 in 2023-09-10T01:57:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:16 smithi082 ceph-mon[98694]: pgmap v5213: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T01:57:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:16 smithi082 ceph-mon[98694]: osdmap e396: 12 total, 12 up, 12 in 2023-09-10T01:57:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:16 smithi098 ceph-mon[103124]: pgmap v5213: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T01:57:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:16 smithi098 ceph-mon[103124]: osdmap e396: 12 total, 12 up, 12 in 2023-09-10T01:57:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9468.733190 secs 2023-09-10T01:57:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:18 smithi170 ceph-mon[102832]: pgmap v5215: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T01:57:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9470.159891 secs 2023-09-10T01:57:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9468.733190 secs 2023-09-10T01:57:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:18 smithi082 ceph-mon[98694]: pgmap v5215: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T01:57:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9470.159891 secs 2023-09-10T01:57:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9468.733190 secs 2023-09-10T01:57:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:18 smithi098 ceph-mon[103124]: pgmap v5215: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T01:57:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9470.159891 secs 2023-09-10T01:57:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9470.871483 secs 2023-09-10T01:57:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9470.871483 secs 2023-09-10T01:57:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9470.871483 secs 2023-09-10T01:57:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:20 smithi170 ceph-mon[102832]: pgmap v5216: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T01:57:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:20 smithi082 ceph-mon[98694]: pgmap v5216: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T01:57:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:20 smithi098 ceph-mon[103124]: pgmap v5216: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T01:57:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:22 smithi170 ceph-mon[102832]: pgmap v5217: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T01:57:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:22 smithi082 ceph-mon[98694]: pgmap v5217: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T01:57:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:22 smithi098 ceph-mon[103124]: pgmap v5217: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T01:57:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:57:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9475.159927 secs 2023-09-10T01:57:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:57:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9475.159927 secs 2023-09-10T01:57:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:57:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9475.159927 secs 2023-09-10T01:57:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:24 smithi170 ceph-mon[102832]: pgmap v5218: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T01:57:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:57:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:57:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:57:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9475.871550 secs 2023-09-10T01:57:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:24 smithi082 ceph-mon[98694]: pgmap v5218: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T01:57:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:57:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:57:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:57:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9475.871550 secs 2023-09-10T01:57:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:24 smithi098 ceph-mon[103124]: pgmap v5218: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T01:57:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:57:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:57:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:57:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9475.871550 secs 2023-09-10T01:57:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:26 smithi170 ceph-mon[102832]: pgmap v5219: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T01:57:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:26 smithi082 ceph-mon[98694]: pgmap v5219: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T01:57:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:26 smithi098 ceph-mon[103124]: pgmap v5219: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T01:57:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:28 smithi170 ceph-mon[102832]: pgmap v5220: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T01:57:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9480.159986 secs 2023-09-10T01:57:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:28 smithi082 ceph-mon[98694]: pgmap v5220: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T01:57:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9480.159986 secs 2023-09-10T01:57:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:28 smithi098 ceph-mon[103124]: pgmap v5220: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T01:57:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9480.159986 secs 2023-09-10T01:57:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9480.871611 secs 2023-09-10T01:57:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9480.871611 secs 2023-09-10T01:57:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9480.871611 secs 2023-09-10T01:57:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:30 smithi170 ceph-mon[102832]: pgmap v5221: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T01:57:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:30 smithi082 ceph-mon[98694]: pgmap v5221: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T01:57:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:30 smithi098 ceph-mon[103124]: pgmap v5221: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T01:57:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:32 smithi170 ceph-mon[102832]: pgmap v5222: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 124 KiB/s wr, 16 op/s 2023-09-10T01:57:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:32 smithi082 ceph-mon[98694]: pgmap v5222: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 124 KiB/s wr, 16 op/s 2023-09-10T01:57:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:32 smithi098 ceph-mon[103124]: pgmap v5222: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 124 KiB/s wr, 16 op/s 2023-09-10T01:57:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9485.160045 secs 2023-09-10T01:57:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9485.160045 secs 2023-09-10T01:57:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9485.160045 secs 2023-09-10T01:57:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:34 smithi170 ceph-mon[102832]: pgmap v5223: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T01:57:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9485.871678 secs 2023-09-10T01:57:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:34 smithi082 ceph-mon[98694]: pgmap v5223: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T01:57:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9485.871678 secs 2023-09-10T01:57:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:34 smithi098 ceph-mon[103124]: pgmap v5223: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T01:57:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9485.871678 secs 2023-09-10T01:57:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:36 smithi170 ceph-mon[102832]: pgmap v5224: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T01:57:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:36 smithi082 ceph-mon[98694]: pgmap v5224: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T01:57:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:36 smithi098 ceph-mon[103124]: pgmap v5224: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T01:57:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:38 smithi170 ceph-mon[102832]: pgmap v5225: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 169 KiB/s wr, 18 op/s 2023-09-10T01:57:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9490.160103 secs 2023-09-10T01:57:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:38 smithi082 ceph-mon[98694]: pgmap v5225: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 169 KiB/s wr, 18 op/s 2023-09-10T01:57:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9490.160103 secs 2023-09-10T01:57:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:38 smithi098 ceph-mon[103124]: pgmap v5225: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 169 KiB/s wr, 18 op/s 2023-09-10T01:57:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9490.160103 secs 2023-09-10T01:57:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9490.871740 secs 2023-09-10T01:57:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9490.871740 secs 2023-09-10T01:57:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9490.871740 secs 2023-09-10T01:57:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:40 smithi170 ceph-mon[102832]: pgmap v5226: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T01:57:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:40 smithi082 ceph-mon[98694]: pgmap v5226: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T01:57:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:40 smithi098 ceph-mon[103124]: pgmap v5226: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T01:57:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:42 smithi170 ceph-mon[102832]: pgmap v5227: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 144 KiB/s wr, 18 op/s 2023-09-10T01:57:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:42 smithi082 ceph-mon[98694]: pgmap v5227: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 144 KiB/s wr, 18 op/s 2023-09-10T01:57:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:42 smithi098 ceph-mon[103124]: pgmap v5227: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 144 KiB/s wr, 18 op/s 2023-09-10T01:57:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9495.160191 secs 2023-09-10T01:57:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9495.160191 secs 2023-09-10T01:57:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9495.160191 secs 2023-09-10T01:57:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:44 smithi170 ceph-mon[102832]: pgmap v5228: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 90 KiB/s wr, 12 op/s 2023-09-10T01:57:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9495.871805 secs 2023-09-10T01:57:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:44 smithi082 ceph-mon[98694]: pgmap v5228: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 90 KiB/s wr, 12 op/s 2023-09-10T01:57:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9495.871805 secs 2023-09-10T01:57:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:44 smithi098 ceph-mon[103124]: pgmap v5228: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 90 KiB/s wr, 12 op/s 2023-09-10T01:57:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9495.871805 secs 2023-09-10T01:57:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:46 smithi170 ceph-mon[102832]: pgmap v5229: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 90 KiB/s wr, 12 op/s 2023-09-10T01:57:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:46 smithi082 ceph-mon[98694]: pgmap v5229: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 90 KiB/s wr, 12 op/s 2023-09-10T01:57:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:46 smithi098 ceph-mon[103124]: pgmap v5229: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 90 KiB/s wr, 12 op/s 2023-09-10T01:57:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9498.733614 secs 2023-09-10T01:57:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:48 smithi170 ceph-mon[102832]: pgmap v5230: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 145 KiB/s wr, 17 op/s 2023-09-10T01:57:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9500.160249 secs 2023-09-10T01:57:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9498.733614 secs 2023-09-10T01:57:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:48 smithi082 ceph-mon[98694]: pgmap v5230: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 145 KiB/s wr, 17 op/s 2023-09-10T01:57:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9500.160249 secs 2023-09-10T01:57:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9498.733614 secs 2023-09-10T01:57:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:48 smithi098 ceph-mon[103124]: pgmap v5230: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 145 KiB/s wr, 17 op/s 2023-09-10T01:57:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9500.160249 secs 2023-09-10T01:57:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9500.871868 secs 2023-09-10T01:57:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9500.871868 secs 2023-09-10T01:57:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9500.871868 secs 2023-09-10T01:57:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:50 smithi170 ceph-mon[102832]: pgmap v5231: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T01:57:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:50 smithi082 ceph-mon[98694]: pgmap v5231: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T01:57:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:50 smithi098 ceph-mon[103124]: pgmap v5231: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T01:57:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:52 smithi170 ceph-mon[102832]: pgmap v5232: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 111 KiB/s wr, 16 op/s 2023-09-10T01:57:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:57:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:57:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:57:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:52 smithi082 ceph-mon[98694]: pgmap v5232: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 111 KiB/s wr, 16 op/s 2023-09-10T01:57:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:57:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:57:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:57:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:52 smithi098 ceph-mon[103124]: pgmap v5232: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 111 KiB/s wr, 16 op/s 2023-09-10T01:57:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:57:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:57:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:57:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9503.733675 secs 2023-09-10T01:57:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9505.160331 secs 2023-09-10T01:57:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9503.733675 secs 2023-09-10T01:57:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9505.160331 secs 2023-09-10T01:57:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9503.733675 secs 2023-09-10T01:57:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9505.160331 secs 2023-09-10T01:57:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:54 smithi170 ceph-mon[102832]: pgmap v5233: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T01:57:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9505.871933 secs 2023-09-10T01:57:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:54 smithi082 ceph-mon[98694]: pgmap v5233: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T01:57:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9505.871933 secs 2023-09-10T01:57:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:54 smithi098 ceph-mon[103124]: pgmap v5233: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T01:57:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9505.871933 secs 2023-09-10T01:57:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:56 smithi170 ceph-mon[102832]: pgmap v5234: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T01:57:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:56 smithi082 ceph-mon[98694]: pgmap v5234: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T01:57:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:56 smithi098 ceph-mon[103124]: pgmap v5234: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T01:57:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:57:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:57:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:57:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9508.733739 secs 2023-09-10T01:57:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:58 smithi170 ceph-mon[102832]: pgmap v5235: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 81 KiB/s wr, 15 op/s 2023-09-10T01:57:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9510.160390 secs 2023-09-10T01:57:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9508.733739 secs 2023-09-10T01:57:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:58 smithi082 ceph-mon[98694]: pgmap v5235: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 81 KiB/s wr, 15 op/s 2023-09-10T01:57:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9510.160390 secs 2023-09-10T01:57:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9508.733739 secs 2023-09-10T01:57:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:58 smithi098 ceph-mon[103124]: pgmap v5235: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 81 KiB/s wr, 15 op/s 2023-09-10T01:57:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9510.160390 secs 2023-09-10T01:57:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:57:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9510.872000 secs 2023-09-10T01:57:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:57:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9510.872000 secs 2023-09-10T01:57:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:57:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9510.872000 secs 2023-09-10T01:58:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:00 smithi170 ceph-mon[102832]: pgmap v5236: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 26 KiB/s wr, 10 op/s 2023-09-10T01:58:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:58:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:58:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:58:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:58:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:58:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:58:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:00.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:00.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:00.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:00.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:00.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:00 smithi082 ceph-mon[98694]: pgmap v5236: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 26 KiB/s wr, 10 op/s 2023-09-10T01:58:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:58:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:58:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:58:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:58:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:58:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:58:00.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:00.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:00.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:00.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:00.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:00.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:00.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:00.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:00 smithi098 ceph-mon[103124]: pgmap v5236: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 26 KiB/s wr, 10 op/s 2023-09-10T01:58:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:58:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:58:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:58:01.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:58:01.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:58:01.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:58:01.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:01.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:01.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:01.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:01.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:01.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:01.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:01.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:01 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T01:58:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:01 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:58:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:01 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T01:58:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:01 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T01:58:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:01.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:01.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:01.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:01.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:01 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T01:58:01.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:01 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T01:58:01.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:58:01.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:58:01.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:58:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:01 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T01:58:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:01 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:58:01.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:01 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T01:58:01.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:01 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T01:58:01.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:01.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:01.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:01.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:01.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:01 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T01:58:01.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:01 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T01:58:01.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:58:01.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:58:01.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:58:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:01 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T01:58:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:01 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T01:58:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:01 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T01:58:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:01 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T01:58:02.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:02.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:02.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:02.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T01:58:02.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:01 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T01:58:02.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:01 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T01:58:02.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:58:02.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:58:02.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:58:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:02 smithi170 ceph-mon[102832]: pgmap v5237: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:58:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9513.733776 secs 2023-09-10T01:58:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:02 smithi082 ceph-mon[98694]: pgmap v5237: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:58:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9513.733776 secs 2023-09-10T01:58:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:02 smithi098 ceph-mon[103124]: pgmap v5237: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T01:58:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9513.733776 secs 2023-09-10T01:58:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9515.160473 secs 2023-09-10T01:58:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9515.160473 secs 2023-09-10T01:58:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9515.160473 secs 2023-09-10T01:58:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:04 smithi170 ceph-mon[102832]: pgmap v5238: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 67 KiB/s wr, 24 op/s 2023-09-10T01:58:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9515.872065 secs 2023-09-10T01:58:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:04 smithi082 ceph-mon[98694]: pgmap v5238: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 67 KiB/s wr, 24 op/s 2023-09-10T01:58:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9515.872065 secs 2023-09-10T01:58:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:04 smithi098 ceph-mon[103124]: pgmap v5238: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 67 KiB/s wr, 24 op/s 2023-09-10T01:58:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9515.872065 secs 2023-09-10T01:58:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:06 smithi170 ceph-mon[102832]: pgmap v5239: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 67 KiB/s wr, 24 op/s 2023-09-10T01:58:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:06 smithi082 ceph-mon[98694]: pgmap v5239: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 67 KiB/s wr, 24 op/s 2023-09-10T01:58:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:06 smithi098 ceph-mon[103124]: pgmap v5239: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 67 KiB/s wr, 24 op/s 2023-09-10T01:58:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9518.733888 secs 2023-09-10T01:58:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9518.733888 secs 2023-09-10T01:58:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9518.733888 secs 2023-09-10T01:58:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:08 smithi170 ceph-mon[102832]: pgmap v5240: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T01:58:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:08 smithi170 ceph-mon[102832]: osdmap e397: 12 total, 12 up, 12 in 2023-09-10T01:58:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9520.160512 secs 2023-09-10T01:58:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:08 smithi082 ceph-mon[98694]: pgmap v5240: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T01:58:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:08 smithi082 ceph-mon[98694]: osdmap e397: 12 total, 12 up, 12 in 2023-09-10T01:58:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9520.160512 secs 2023-09-10T01:58:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:08 smithi098 ceph-mon[103124]: pgmap v5240: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T01:58:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:08 smithi098 ceph-mon[103124]: osdmap e397: 12 total, 12 up, 12 in 2023-09-10T01:58:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9520.160512 secs 2023-09-10T01:58:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9520.872131 secs 2023-09-10T01:58:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9520.872131 secs 2023-09-10T01:58:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9520.872131 secs 2023-09-10T01:58:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:10 smithi170 ceph-mon[102832]: pgmap v5242: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 108 KiB/s wr, 29 op/s 2023-09-10T01:58:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:10 smithi082 ceph-mon[98694]: pgmap v5242: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 108 KiB/s wr, 29 op/s 2023-09-10T01:58:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:10 smithi098 ceph-mon[103124]: pgmap v5242: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 108 KiB/s wr, 29 op/s 2023-09-10T01:58:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:12 smithi170 ceph-mon[102832]: pgmap v5243: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 28 KiB/s wr, 12 op/s 2023-09-10T01:58:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9523.733951 secs 2023-09-10T01:58:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:12 smithi082 ceph-mon[98694]: pgmap v5243: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 28 KiB/s wr, 12 op/s 2023-09-10T01:58:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9523.733951 secs 2023-09-10T01:58:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:12 smithi098 ceph-mon[103124]: pgmap v5243: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 28 KiB/s wr, 12 op/s 2023-09-10T01:58:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9523.733951 secs 2023-09-10T01:58:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9525.160570 secs 2023-09-10T01:58:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9525.160570 secs 2023-09-10T01:58:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9525.160570 secs 2023-09-10T01:58:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:14 smithi170 ceph-mon[102832]: pgmap v5244: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 28 KiB/s wr, 12 op/s 2023-09-10T01:58:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9525.872198 secs 2023-09-10T01:58:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:14 smithi082 ceph-mon[98694]: pgmap v5244: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 28 KiB/s wr, 12 op/s 2023-09-10T01:58:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9525.872198 secs 2023-09-10T01:58:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:14 smithi098 ceph-mon[103124]: pgmap v5244: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 28 KiB/s wr, 12 op/s 2023-09-10T01:58:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9525.872198 secs 2023-09-10T01:58:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:16 smithi170 ceph-mon[102832]: pgmap v5245: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 28 KiB/s wr, 12 op/s 2023-09-10T01:58:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:16 smithi170 ceph-mon[102832]: osdmap e398: 12 total, 12 up, 12 in 2023-09-10T01:58:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:16 smithi082 ceph-mon[98694]: pgmap v5245: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 28 KiB/s wr, 12 op/s 2023-09-10T01:58:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:16 smithi082 ceph-mon[98694]: osdmap e398: 12 total, 12 up, 12 in 2023-09-10T01:58:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:16 smithi098 ceph-mon[103124]: pgmap v5245: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 28 KiB/s wr, 12 op/s 2023-09-10T01:58:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:16 smithi098 ceph-mon[103124]: osdmap e398: 12 total, 12 up, 12 in 2023-09-10T01:58:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9528.734066 secs 2023-09-10T01:58:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9528.734066 secs 2023-09-10T01:58:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9528.734066 secs 2023-09-10T01:58:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:18 smithi170 ceph-mon[102832]: pgmap v5247: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T01:58:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9530.160628 secs 2023-09-10T01:58:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:18 smithi082 ceph-mon[98694]: pgmap v5247: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T01:58:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9530.160628 secs 2023-09-10T01:58:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:18 smithi098 ceph-mon[103124]: pgmap v5247: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T01:58:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9530.160628 secs 2023-09-10T01:58:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9530.872264 secs 2023-09-10T01:58:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9530.872264 secs 2023-09-10T01:58:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9530.872264 secs 2023-09-10T01:58:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:20 smithi170 ceph-mon[102832]: pgmap v5248: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T01:58:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:20 smithi082 ceph-mon[98694]: pgmap v5248: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T01:58:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:20 smithi098 ceph-mon[103124]: pgmap v5248: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T01:58:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:22 smithi170 ceph-mon[102832]: pgmap v5249: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T01:58:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:22 smithi082 ceph-mon[98694]: pgmap v5249: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T01:58:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:22 smithi098 ceph-mon[103124]: pgmap v5249: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T01:58:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:58:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9535.160686 secs 2023-09-10T01:58:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:58:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9535.160686 secs 2023-09-10T01:58:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:58:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9535.160686 secs 2023-09-10T01:58:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:24 smithi170 ceph-mon[102832]: pgmap v5250: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T01:58:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9535.872329 secs 2023-09-10T01:58:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:58:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:58:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:58:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:24 smithi082 ceph-mon[98694]: pgmap v5250: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T01:58:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9535.872329 secs 2023-09-10T01:58:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:58:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:58:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:58:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:24 smithi098 ceph-mon[103124]: pgmap v5250: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T01:58:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9535.872329 secs 2023-09-10T01:58:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:58:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:58:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:58:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:26 smithi170 ceph-mon[102832]: pgmap v5251: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T01:58:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:26 smithi082 ceph-mon[98694]: pgmap v5251: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T01:58:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:26 smithi098 ceph-mon[103124]: pgmap v5251: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T01:58:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:28 smithi170 ceph-mon[102832]: pgmap v5252: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-09-10T01:58:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9540.160746 secs 2023-09-10T01:58:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:28 smithi082 ceph-mon[98694]: pgmap v5252: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-09-10T01:58:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9540.160746 secs 2023-09-10T01:58:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:28 smithi098 ceph-mon[103124]: pgmap v5252: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-09-10T01:58:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9540.160746 secs 2023-09-10T01:58:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9540.872391 secs 2023-09-10T01:58:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9540.872391 secs 2023-09-10T01:58:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9540.872391 secs 2023-09-10T01:58:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:30 smithi170 ceph-mon[102832]: pgmap v5253: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 21 KiB/s wr, 9 op/s 2023-09-10T01:58:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:30 smithi082 ceph-mon[98694]: pgmap v5253: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 21 KiB/s wr, 9 op/s 2023-09-10T01:58:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:30 smithi098 ceph-mon[103124]: pgmap v5253: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 21 KiB/s wr, 9 op/s 2023-09-10T01:58:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:32 smithi170 ceph-mon[102832]: pgmap v5254: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 66 KiB/s wr, 15 op/s 2023-09-10T01:58:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:32 smithi082 ceph-mon[98694]: pgmap v5254: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 66 KiB/s wr, 15 op/s 2023-09-10T01:58:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:32 smithi098 ceph-mon[103124]: pgmap v5254: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 66 KiB/s wr, 15 op/s 2023-09-10T01:58:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9545.160829 secs 2023-09-10T01:58:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9545.160829 secs 2023-09-10T01:58:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9545.160829 secs 2023-09-10T01:58:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:34 smithi170 ceph-mon[102832]: pgmap v5255: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T01:58:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9545.872454 secs 2023-09-10T01:58:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:34 smithi082 ceph-mon[98694]: pgmap v5255: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T01:58:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9545.872454 secs 2023-09-10T01:58:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:34 smithi098 ceph-mon[103124]: pgmap v5255: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T01:58:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9545.872454 secs 2023-09-10T01:58:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:36 smithi170 ceph-mon[102832]: pgmap v5256: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T01:58:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:36 smithi082 ceph-mon[98694]: pgmap v5256: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T01:58:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:36 smithi098 ceph-mon[103124]: pgmap v5256: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T01:58:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:38 smithi170 ceph-mon[102832]: pgmap v5257: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 130 KiB/s wr, 18 op/s 2023-09-10T01:58:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9550.160912 secs 2023-09-10T01:58:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:38 smithi082 ceph-mon[98694]: pgmap v5257: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 130 KiB/s wr, 18 op/s 2023-09-10T01:58:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9550.160912 secs 2023-09-10T01:58:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:38 smithi098 ceph-mon[103124]: pgmap v5257: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 130 KiB/s wr, 18 op/s 2023-09-10T01:58:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9550.160912 secs 2023-09-10T01:58:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9550.872490 secs 2023-09-10T01:58:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9550.872490 secs 2023-09-10T01:58:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9550.872490 secs 2023-09-10T01:58:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:40 smithi170 ceph-mon[102832]: pgmap v5258: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 110 KiB/s wr, 13 op/s 2023-09-10T01:58:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:40 smithi082 ceph-mon[98694]: pgmap v5258: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 110 KiB/s wr, 13 op/s 2023-09-10T01:58:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:40 smithi098 ceph-mon[103124]: pgmap v5258: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 110 KiB/s wr, 13 op/s 2023-09-10T01:58:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:42 smithi170 ceph-mon[102832]: pgmap v5259: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 141 KiB/s wr, 19 op/s 2023-09-10T01:58:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:42 smithi082 ceph-mon[98694]: pgmap v5259: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 141 KiB/s wr, 19 op/s 2023-09-10T01:58:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:42 smithi098 ceph-mon[103124]: pgmap v5259: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 141 KiB/s wr, 19 op/s 2023-09-10T01:58:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9555.160969 secs 2023-09-10T01:58:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9555.160969 secs 2023-09-10T01:58:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9555.160969 secs 2023-09-10T01:58:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:44 smithi082 ceph-mon[98694]: pgmap v5260: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 96 KiB/s wr, 13 op/s 2023-09-10T01:58:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9555.872552 secs 2023-09-10T01:58:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:44 smithi098 ceph-mon[103124]: pgmap v5260: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 96 KiB/s wr, 13 op/s 2023-09-10T01:58:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9555.872552 secs 2023-09-10T01:58:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:44 smithi170 ceph-mon[102832]: pgmap v5260: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 96 KiB/s wr, 13 op/s 2023-09-10T01:58:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9555.872552 secs 2023-09-10T01:58:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:46 smithi082 ceph-mon[98694]: pgmap v5261: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 96 KiB/s wr, 13 op/s 2023-09-10T01:58:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:46 smithi098 ceph-mon[103124]: pgmap v5261: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 96 KiB/s wr, 13 op/s 2023-09-10T01:58:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:46 smithi170 ceph-mon[102832]: pgmap v5261: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 96 KiB/s wr, 13 op/s 2023-09-10T01:58:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:48 smithi082 ceph-mon[98694]: pgmap v5262: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 161 KiB/s wr, 19 op/s 2023-09-10T01:58:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9560.161028 secs 2023-09-10T01:58:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:48 smithi098 ceph-mon[103124]: pgmap v5262: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 161 KiB/s wr, 19 op/s 2023-09-10T01:58:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9560.161028 secs 2023-09-10T01:58:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:48 smithi170 ceph-mon[102832]: pgmap v5262: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 161 KiB/s wr, 19 op/s 2023-09-10T01:58:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9560.161028 secs 2023-09-10T01:58:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9560.872618 secs 2023-09-10T01:58:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9560.872618 secs 2023-09-10T01:58:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9560.872618 secs 2023-09-10T01:58:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:50 smithi082 ceph-mon[98694]: pgmap v5263: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T01:58:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:50 smithi098 ceph-mon[103124]: pgmap v5263: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T01:58:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:50 smithi170 ceph-mon[102832]: pgmap v5263: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T01:58:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:52 smithi082 ceph-mon[98694]: pgmap v5264: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 155 KiB/s wr, 18 op/s 2023-09-10T01:58:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9563.734555 secs 2023-09-10T01:58:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:52 smithi098 ceph-mon[103124]: pgmap v5264: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 155 KiB/s wr, 18 op/s 2023-09-10T01:58:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9563.734555 secs 2023-09-10T01:58:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:52 smithi170 ceph-mon[102832]: pgmap v5264: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 155 KiB/s wr, 18 op/s 2023-09-10T01:58:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9563.734555 secs 2023-09-10T01:58:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9565.161084 secs 2023-09-10T01:58:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9565.161084 secs 2023-09-10T01:58:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9565.161084 secs 2023-09-10T01:58:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:54 smithi082 ceph-mon[98694]: pgmap v5265: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T01:58:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9565.872678 secs 2023-09-10T01:58:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:54 smithi098 ceph-mon[103124]: pgmap v5265: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T01:58:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9565.872678 secs 2023-09-10T01:58:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:54 smithi170 ceph-mon[102832]: pgmap v5265: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T01:58:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9565.872678 secs 2023-09-10T01:58:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:58:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:58:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:58:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:58:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:58:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:58:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:58:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:58:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:58:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:56 smithi170 ceph-mon[102832]: pgmap v5266: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T01:58:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:56 smithi082 ceph-mon[98694]: pgmap v5266: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T01:58:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:56 smithi098 ceph-mon[103124]: pgmap v5266: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T01:58:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9568.734679 secs 2023-09-10T01:58:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9568.734679 secs 2023-09-10T01:58:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9568.734679 secs 2023-09-10T01:58:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:58 smithi170 ceph-mon[102832]: pgmap v5267: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 17 op/s 2023-09-10T01:58:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9570.161143 secs 2023-09-10T01:58:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:58 smithi082 ceph-mon[98694]: pgmap v5267: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 17 op/s 2023-09-10T01:58:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9570.161143 secs 2023-09-10T01:58:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:58 smithi098 ceph-mon[103124]: pgmap v5267: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 17 op/s 2023-09-10T01:58:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9570.161143 secs 2023-09-10T01:59:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:58:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9570.872743 secs 2023-09-10T01:59:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:58:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9570.872743 secs 2023-09-10T01:59:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:58:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9570.872743 secs 2023-09-10T01:59:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:00 smithi170 ceph-mon[102832]: pgmap v5268: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T01:59:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:59:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:00 smithi082 ceph-mon[98694]: pgmap v5268: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T01:59:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:59:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:00 smithi098 ceph-mon[103124]: pgmap v5268: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T01:59:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T01:59:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:02 smithi170 ceph-mon[102832]: pgmap v5269: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 149 KiB/s wr, 20 op/s 2023-09-10T01:59:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:59:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:59:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:59:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:02 smithi082 ceph-mon[98694]: pgmap v5269: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 149 KiB/s wr, 20 op/s 2023-09-10T01:59:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:59:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:59:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:59:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:02 smithi098 ceph-mon[103124]: pgmap v5269: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 149 KiB/s wr, 20 op/s 2023-09-10T01:59:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T01:59:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T01:59:02.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T01:59:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9573.734739 secs 2023-09-10T01:59:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9575.161229 secs 2023-09-10T01:59:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9573.734739 secs 2023-09-10T01:59:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9575.161229 secs 2023-09-10T01:59:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9573.734739 secs 2023-09-10T01:59:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9575.161229 secs 2023-09-10T01:59:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:04 smithi170 ceph-mon[102832]: pgmap v5270: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 89 KiB/s wr, 14 op/s 2023-09-10T01:59:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9575.872803 secs 2023-09-10T01:59:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:04 smithi082 ceph-mon[98694]: pgmap v5270: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 89 KiB/s wr, 14 op/s 2023-09-10T01:59:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9575.872803 secs 2023-09-10T01:59:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:04 smithi098 ceph-mon[103124]: pgmap v5270: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 89 KiB/s wr, 14 op/s 2023-09-10T01:59:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9575.872803 secs 2023-09-10T01:59:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:06 smithi170 ceph-mon[102832]: pgmap v5271: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 89 KiB/s wr, 14 op/s 2023-09-10T01:59:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:06 smithi082 ceph-mon[98694]: pgmap v5271: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 89 KiB/s wr, 14 op/s 2023-09-10T01:59:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:06 smithi098 ceph-mon[103124]: pgmap v5271: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 89 KiB/s wr, 14 op/s 2023-09-10T01:59:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9578.734772 secs 2023-09-10T01:59:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:08 smithi170 ceph-mon[102832]: pgmap v5272: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 122 KiB/s wr, 29 op/s 2023-09-10T01:59:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:08 smithi170 ceph-mon[102832]: osdmap e399: 12 total, 12 up, 12 in 2023-09-10T01:59:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9580.161287 secs 2023-09-10T01:59:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9578.734772 secs 2023-09-10T01:59:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:08 smithi082 ceph-mon[98694]: pgmap v5272: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 122 KiB/s wr, 29 op/s 2023-09-10T01:59:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:08 smithi082 ceph-mon[98694]: osdmap e399: 12 total, 12 up, 12 in 2023-09-10T01:59:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9580.161287 secs 2023-09-10T01:59:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9578.734772 secs 2023-09-10T01:59:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:08 smithi098 ceph-mon[103124]: pgmap v5272: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 122 KiB/s wr, 29 op/s 2023-09-10T01:59:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:08 smithi098 ceph-mon[103124]: osdmap e399: 12 total, 12 up, 12 in 2023-09-10T01:59:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9580.161287 secs 2023-09-10T01:59:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9580.872867 secs 2023-09-10T01:59:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9580.872867 secs 2023-09-10T01:59:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9580.872867 secs 2023-09-10T01:59:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:10 smithi170 ceph-mon[102832]: pgmap v5274: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:59:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:10 smithi082 ceph-mon[98694]: pgmap v5274: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:59:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:10 smithi098 ceph-mon[103124]: pgmap v5274: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T01:59:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:12 smithi170 ceph-mon[102832]: pgmap v5275: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 39 KiB/s wr, 23 op/s 2023-09-10T01:59:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:12 smithi082 ceph-mon[98694]: pgmap v5275: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 39 KiB/s wr, 23 op/s 2023-09-10T01:59:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:12 smithi098 ceph-mon[103124]: pgmap v5275: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 39 KiB/s wr, 23 op/s 2023-09-10T01:59:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9583.734833 secs 2023-09-10T01:59:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9585.161374 secs 2023-09-10T01:59:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9583.734833 secs 2023-09-10T01:59:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9585.161374 secs 2023-09-10T01:59:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9583.734833 secs 2023-09-10T01:59:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9585.161374 secs 2023-09-10T01:59:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:14 smithi170 ceph-mon[102832]: pgmap v5276: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 39 KiB/s wr, 23 op/s 2023-09-10T01:59:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9585.872930 secs 2023-09-10T01:59:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:14 smithi082 ceph-mon[98694]: pgmap v5276: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 39 KiB/s wr, 23 op/s 2023-09-10T01:59:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9585.872930 secs 2023-09-10T01:59:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:14 smithi098 ceph-mon[103124]: pgmap v5276: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 39 KiB/s wr, 23 op/s 2023-09-10T01:59:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9585.872930 secs 2023-09-10T01:59:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:16 smithi170 ceph-mon[102832]: pgmap v5277: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 39 KiB/s wr, 23 op/s 2023-09-10T01:59:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:16 smithi170 ceph-mon[102832]: osdmap e400: 12 total, 12 up, 12 in 2023-09-10T01:59:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:16 smithi082 ceph-mon[98694]: pgmap v5277: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 39 KiB/s wr, 23 op/s 2023-09-10T01:59:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:16 smithi082 ceph-mon[98694]: osdmap e400: 12 total, 12 up, 12 in 2023-09-10T01:59:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:16 smithi098 ceph-mon[103124]: pgmap v5277: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 39 KiB/s wr, 23 op/s 2023-09-10T01:59:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:16 smithi098 ceph-mon[103124]: osdmap e400: 12 total, 12 up, 12 in 2023-09-10T01:59:18.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9588.734937 secs 2023-09-10T01:59:18.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:18 smithi170 ceph-mon[102832]: pgmap v5279: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T01:59:18.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9590.161432 secs 2023-09-10T01:59:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9588.734937 secs 2023-09-10T01:59:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:18 smithi082 ceph-mon[98694]: pgmap v5279: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T01:59:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9590.161432 secs 2023-09-10T01:59:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9588.734937 secs 2023-09-10T01:59:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:18 smithi098 ceph-mon[103124]: pgmap v5279: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T01:59:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9590.161432 secs 2023-09-10T01:59:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9590.872993 secs 2023-09-10T01:59:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9590.872993 secs 2023-09-10T01:59:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9590.872993 secs 2023-09-10T01:59:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:20 smithi170 ceph-mon[102832]: pgmap v5280: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T01:59:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:20 smithi082 ceph-mon[98694]: pgmap v5280: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T01:59:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:20 smithi098 ceph-mon[103124]: pgmap v5280: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T01:59:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:22 smithi170 ceph-mon[102832]: pgmap v5281: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T01:59:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:22 smithi082 ceph-mon[98694]: pgmap v5281: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T01:59:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:22 smithi098 ceph-mon[103124]: pgmap v5281: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T01:59:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:59:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9595.161494 secs 2023-09-10T01:59:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:59:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9595.161494 secs 2023-09-10T01:59:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T01:59:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9595.161494 secs 2023-09-10T01:59:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:24 smithi170 ceph-mon[102832]: pgmap v5282: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T01:59:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:59:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:59:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:59:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9595.873058 secs 2023-09-10T01:59:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:24 smithi082 ceph-mon[98694]: pgmap v5282: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T01:59:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:59:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:59:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:59:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9595.873058 secs 2023-09-10T01:59:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:24 smithi098 ceph-mon[103124]: pgmap v5282: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T01:59:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:59:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:59:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:59:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9595.873058 secs 2023-09-10T01:59:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:26 smithi170 ceph-mon[102832]: pgmap v5283: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T01:59:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:26 smithi082 ceph-mon[98694]: pgmap v5283: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T01:59:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:26 smithi098 ceph-mon[103124]: pgmap v5283: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T01:59:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:28 smithi170 ceph-mon[102832]: pgmap v5284: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T01:59:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9600.161567 secs 2023-09-10T01:59:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:28 smithi082 ceph-mon[98694]: pgmap v5284: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T01:59:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9600.161567 secs 2023-09-10T01:59:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:28 smithi098 ceph-mon[103124]: pgmap v5284: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T01:59:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9600.161567 secs 2023-09-10T01:59:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9600.873118 secs 2023-09-10T01:59:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9600.873118 secs 2023-09-10T01:59:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9600.873118 secs 2023-09-10T01:59:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:30 smithi082 ceph-mon[98694]: pgmap v5285: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T01:59:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:30 smithi098 ceph-mon[103124]: pgmap v5285: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T01:59:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:30 smithi170 ceph-mon[102832]: pgmap v5285: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T01:59:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:32 smithi082 ceph-mon[98694]: pgmap v5286: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 98 KiB/s wr, 16 op/s 2023-09-10T01:59:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:32 smithi098 ceph-mon[103124]: pgmap v5286: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 98 KiB/s wr, 16 op/s 2023-09-10T01:59:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:32 smithi170 ceph-mon[102832]: pgmap v5286: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 98 KiB/s wr, 16 op/s 2023-09-10T01:59:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9605.161621 secs 2023-09-10T01:59:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9605.161621 secs 2023-09-10T01:59:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9605.161621 secs 2023-09-10T01:59:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:34 smithi082 ceph-mon[98694]: pgmap v5287: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T01:59:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9605.873182 secs 2023-09-10T01:59:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:34 smithi098 ceph-mon[103124]: pgmap v5287: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T01:59:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9605.873182 secs 2023-09-10T01:59:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:34 smithi170 ceph-mon[102832]: pgmap v5287: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T01:59:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9605.873182 secs 2023-09-10T01:59:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:36 smithi082 ceph-mon[98694]: pgmap v5288: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T01:59:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:36 smithi098 ceph-mon[103124]: pgmap v5288: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T01:59:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:36 smithi170 ceph-mon[102832]: pgmap v5288: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T01:59:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:38 smithi082 ceph-mon[98694]: pgmap v5289: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 147 KiB/s wr, 18 op/s 2023-09-10T01:59:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9610.161658 secs 2023-09-10T01:59:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:38 smithi098 ceph-mon[103124]: pgmap v5289: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 147 KiB/s wr, 18 op/s 2023-09-10T01:59:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9610.161658 secs 2023-09-10T01:59:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:38 smithi170 ceph-mon[102832]: pgmap v5289: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 147 KiB/s wr, 18 op/s 2023-09-10T01:59:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9610.161658 secs 2023-09-10T01:59:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9610.873245 secs 2023-09-10T01:59:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9610.873245 secs 2023-09-10T01:59:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9610.873245 secs 2023-09-10T01:59:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:40 smithi082 ceph-mon[98694]: pgmap v5290: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T01:59:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:40 smithi098 ceph-mon[103124]: pgmap v5290: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T01:59:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:40 smithi170 ceph-mon[102832]: pgmap v5290: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T01:59:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:42 smithi082 ceph-mon[98694]: pgmap v5291: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 149 KiB/s wr, 19 op/s 2023-09-10T01:59:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:42 smithi098 ceph-mon[103124]: pgmap v5291: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 149 KiB/s wr, 19 op/s 2023-09-10T01:59:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:42 smithi170 ceph-mon[102832]: pgmap v5291: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 149 KiB/s wr, 19 op/s 2023-09-10T01:59:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9613.735340 secs 2023-09-10T01:59:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9615.161713 secs 2023-09-10T01:59:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9613.735340 secs 2023-09-10T01:59:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9615.161713 secs 2023-09-10T01:59:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9613.735340 secs 2023-09-10T01:59:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9615.161713 secs 2023-09-10T01:59:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:44 smithi098 ceph-mon[103124]: pgmap v5292: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T01:59:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:59:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:59:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:59:44.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9615.873304 secs 2023-09-10T01:59:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:44 smithi170 ceph-mon[102832]: pgmap v5292: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T01:59:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:59:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:59:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:59:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9615.873304 secs 2023-09-10T01:59:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:44 smithi082 ceph-mon[98694]: pgmap v5292: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T01:59:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T01:59:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T01:59:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T01:59:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9615.873304 secs 2023-09-10T01:59:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:46 smithi098 ceph-mon[103124]: pgmap v5293: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T01:59:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:46 smithi170 ceph-mon[102832]: pgmap v5293: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T01:59:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:46 smithi082 ceph-mon[98694]: pgmap v5293: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T01:59:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9618.735409 secs 2023-09-10T01:59:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:48 smithi098 ceph-mon[103124]: pgmap v5294: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 164 KiB/s wr, 18 op/s 2023-09-10T01:59:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9620.161771 secs 2023-09-10T01:59:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9618.735409 secs 2023-09-10T01:59:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:48 smithi170 ceph-mon[102832]: pgmap v5294: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 164 KiB/s wr, 18 op/s 2023-09-10T01:59:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9620.161771 secs 2023-09-10T01:59:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9618.735409 secs 2023-09-10T01:59:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:48 smithi082 ceph-mon[98694]: pgmap v5294: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 164 KiB/s wr, 18 op/s 2023-09-10T01:59:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9620.161771 secs 2023-09-10T01:59:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9620.873370 secs 2023-09-10T01:59:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9620.873370 secs 2023-09-10T01:59:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9620.873370 secs 2023-09-10T01:59:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:50 smithi098 ceph-mon[103124]: pgmap v5295: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T01:59:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:50 smithi170 ceph-mon[102832]: pgmap v5295: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T01:59:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:50 smithi082 ceph-mon[98694]: pgmap v5295: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T01:59:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:52 smithi098 ceph-mon[103124]: pgmap v5296: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 134 KiB/s wr, 17 op/s 2023-09-10T01:59:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:52 smithi170 ceph-mon[102832]: pgmap v5296: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 134 KiB/s wr, 17 op/s 2023-09-10T01:59:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:52 smithi082 ceph-mon[98694]: pgmap v5296: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 134 KiB/s wr, 17 op/s 2023-09-10T01:59:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9623.735469 secs 2023-09-10T01:59:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9625.161848 secs 2023-09-10T01:59:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9623.735469 secs 2023-09-10T01:59:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9625.161848 secs 2023-09-10T01:59:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9623.735469 secs 2023-09-10T01:59:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9625.161848 secs 2023-09-10T01:59:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:54 smithi098 ceph-mon[103124]: pgmap v5297: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T01:59:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9625.873432 secs 2023-09-10T01:59:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:54 smithi170 ceph-mon[102832]: pgmap v5297: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T01:59:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9625.873432 secs 2023-09-10T01:59:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:54 smithi082 ceph-mon[98694]: pgmap v5297: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T01:59:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9625.873432 secs 2023-09-10T01:59:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:56 smithi098 ceph-mon[103124]: pgmap v5298: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T01:59:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:56 smithi170 ceph-mon[102832]: pgmap v5298: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T01:59:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:56 smithi082 ceph-mon[98694]: pgmap v5298: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T01:59:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9628.735573 secs 2023-09-10T01:59:58.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:58 smithi098 ceph-mon[103124]: pgmap v5299: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 85 KiB/s wr, 15 op/s 2023-09-10T01:59:58.634 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9630.161904 secs 2023-09-10T01:59:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9628.735573 secs 2023-09-10T01:59:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:58 smithi170 ceph-mon[102832]: pgmap v5299: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 85 KiB/s wr, 15 op/s 2023-09-10T01:59:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9630.161904 secs 2023-09-10T01:59:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9628.735573 secs 2023-09-10T01:59:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:58 smithi082 ceph-mon[98694]: pgmap v5299: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 85 KiB/s wr, 15 op/s 2023-09-10T01:59:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9630.161904 secs 2023-09-10T01:59:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 01:59:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9630.873570 secs 2023-09-10T01:59:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 01:59:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9630.873570 secs 2023-09-10T01:59:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 01:59:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9630.873570 secs 2023-09-10T02:00:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:00 smithi098 ceph-mon[103124]: pgmap v5300: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 9 op/s 2023-09-10T02:00:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T02:00:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:00 smithi170 ceph-mon[102832]: pgmap v5300: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 9 op/s 2023-09-10T02:00:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T02:00:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:00 smithi082 ceph-mon[98694]: pgmap v5300: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 9 op/s 2023-09-10T02:00:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T02:00:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:02 smithi170 ceph-mon[102832]: pgmap v5301: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 65 KiB/s wr, 15 op/s 2023-09-10T02:00:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:00:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:00:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:00:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:02 smithi082 ceph-mon[98694]: pgmap v5301: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 65 KiB/s wr, 15 op/s 2023-09-10T02:00:02.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:00:02.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:00:02.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:00:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:02 smithi098 ceph-mon[103124]: pgmap v5301: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 65 KiB/s wr, 15 op/s 2023-09-10T02:00:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:00:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:00:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:00:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9633.735632 secs 2023-09-10T02:00:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:00:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:00:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:00:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9635.161961 secs 2023-09-10T02:00:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9633.735632 secs 2023-09-10T02:00:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:00:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:00:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:00:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9635.161961 secs 2023-09-10T02:00:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9633.735632 secs 2023-09-10T02:00:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:00:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:00:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:00:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9635.161961 secs 2023-09-10T02:00:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:04 smithi170 ceph-mon[102832]: pgmap v5302: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T02:00:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9635.873645 secs 2023-09-10T02:00:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:04 smithi082 ceph-mon[98694]: pgmap v5302: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T02:00:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9635.873645 secs 2023-09-10T02:00:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:04 smithi098 ceph-mon[103124]: pgmap v5302: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T02:00:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9635.873645 secs 2023-09-10T02:00:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:06 smithi170 ceph-mon[102832]: pgmap v5303: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T02:00:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:06 smithi082 ceph-mon[98694]: pgmap v5303: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T02:00:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:06 smithi098 ceph-mon[103124]: pgmap v5303: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T02:00:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9638.735694 secs 2023-09-10T02:00:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9638.735694 secs 2023-09-10T02:00:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9638.735694 secs 2023-09-10T02:00:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:08 smithi170 ceph-mon[102832]: pgmap v5304: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T02:00:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:08 smithi170 ceph-mon[102832]: osdmap e401: 12 total, 12 up, 12 in 2023-09-10T02:00:08.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9640.162047 secs 2023-09-10T02:00:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:08 smithi082 ceph-mon[98694]: pgmap v5304: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T02:00:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:08 smithi082 ceph-mon[98694]: osdmap e401: 12 total, 12 up, 12 in 2023-09-10T02:00:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9640.162047 secs 2023-09-10T02:00:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:08 smithi098 ceph-mon[103124]: pgmap v5304: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T02:00:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:08 smithi098 ceph-mon[103124]: osdmap e401: 12 total, 12 up, 12 in 2023-09-10T02:00:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9640.162047 secs 2023-09-10T02:00:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9640.873711 secs 2023-09-10T02:00:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9640.873711 secs 2023-09-10T02:00:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9640.873711 secs 2023-09-10T02:00:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:10 smithi170 ceph-mon[102832]: pgmap v5306: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T02:00:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:10 smithi082 ceph-mon[98694]: pgmap v5306: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T02:00:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:10 smithi098 ceph-mon[103124]: pgmap v5306: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T02:00:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:12 smithi170 ceph-mon[102832]: pgmap v5307: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 58 KiB/s wr, 28 op/s 2023-09-10T02:00:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9643.735771 secs 2023-09-10T02:00:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:12 smithi082 ceph-mon[98694]: pgmap v5307: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 58 KiB/s wr, 28 op/s 2023-09-10T02:00:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9643.735771 secs 2023-09-10T02:00:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:12 smithi098 ceph-mon[103124]: pgmap v5307: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 58 KiB/s wr, 28 op/s 2023-09-10T02:00:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9643.735771 secs 2023-09-10T02:00:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9645.162103 secs 2023-09-10T02:00:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9645.162103 secs 2023-09-10T02:00:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9645.162103 secs 2023-09-10T02:00:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:14 smithi170 ceph-mon[102832]: pgmap v5308: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 58 KiB/s wr, 28 op/s 2023-09-10T02:00:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9645.873774 secs 2023-09-10T02:00:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:14 smithi082 ceph-mon[98694]: pgmap v5308: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 58 KiB/s wr, 28 op/s 2023-09-10T02:00:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9645.873774 secs 2023-09-10T02:00:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:14 smithi098 ceph-mon[103124]: pgmap v5308: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 58 KiB/s wr, 28 op/s 2023-09-10T02:00:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9645.873774 secs 2023-09-10T02:00:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:16 smithi170 ceph-mon[102832]: pgmap v5309: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 58 KiB/s wr, 28 op/s 2023-09-10T02:00:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:16 smithi170 ceph-mon[102832]: osdmap e402: 12 total, 12 up, 12 in 2023-09-10T02:00:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:16 smithi082 ceph-mon[98694]: pgmap v5309: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 58 KiB/s wr, 28 op/s 2023-09-10T02:00:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:16 smithi082 ceph-mon[98694]: osdmap e402: 12 total, 12 up, 12 in 2023-09-10T02:00:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:16 smithi098 ceph-mon[103124]: pgmap v5309: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 58 KiB/s wr, 28 op/s 2023-09-10T02:00:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:16 smithi098 ceph-mon[103124]: osdmap e402: 12 total, 12 up, 12 in 2023-09-10T02:00:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9648.735863 secs 2023-09-10T02:00:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9648.735863 secs 2023-09-10T02:00:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9648.735863 secs 2023-09-10T02:00:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:18 smithi170 ceph-mon[102832]: pgmap v5311: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T02:00:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9650.162160 secs 2023-09-10T02:00:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:18 smithi082 ceph-mon[98694]: pgmap v5311: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T02:00:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9650.162160 secs 2023-09-10T02:00:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:18 smithi098 ceph-mon[103124]: pgmap v5311: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T02:00:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9650.162160 secs 2023-09-10T02:00:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9650.873836 secs 2023-09-10T02:00:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9650.873836 secs 2023-09-10T02:00:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9650.873836 secs 2023-09-10T02:00:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:20 smithi170 ceph-mon[102832]: pgmap v5312: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T02:00:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:20 smithi082 ceph-mon[98694]: pgmap v5312: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T02:00:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:20 smithi098 ceph-mon[103124]: pgmap v5312: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T02:00:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:22 smithi170 ceph-mon[102832]: pgmap v5313: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T02:00:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:22 smithi082 ceph-mon[98694]: pgmap v5313: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T02:00:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:22 smithi098 ceph-mon[103124]: pgmap v5313: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T02:00:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:00:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9655.162247 secs 2023-09-10T02:00:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:00:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9655.162247 secs 2023-09-10T02:00:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:00:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9655.162247 secs 2023-09-10T02:00:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:24 smithi170 ceph-mon[102832]: pgmap v5314: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T02:00:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9655.873900 secs 2023-09-10T02:00:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:00:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:00:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:00:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:24 smithi082 ceph-mon[98694]: pgmap v5314: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T02:00:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9655.873900 secs 2023-09-10T02:00:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:00:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:00:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:00:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:24 smithi098 ceph-mon[103124]: pgmap v5314: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T02:00:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9655.873900 secs 2023-09-10T02:00:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:00:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:00:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:00:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:26 smithi170 ceph-mon[102832]: pgmap v5315: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T02:00:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:26 smithi082 ceph-mon[98694]: pgmap v5315: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T02:00:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:26 smithi098 ceph-mon[103124]: pgmap v5315: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T02:00:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:28 smithi170 ceph-mon[102832]: pgmap v5316: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T02:00:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9660.162304 secs 2023-09-10T02:00:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:28 smithi082 ceph-mon[98694]: pgmap v5316: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T02:00:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9660.162304 secs 2023-09-10T02:00:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:28 smithi098 ceph-mon[103124]: pgmap v5316: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T02:00:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9660.162304 secs 2023-09-10T02:00:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9660.873962 secs 2023-09-10T02:00:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9660.873962 secs 2023-09-10T02:00:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9660.873962 secs 2023-09-10T02:00:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:30 smithi170 ceph-mon[102832]: pgmap v5317: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 10 op/s 2023-09-10T02:00:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:30 smithi082 ceph-mon[98694]: pgmap v5317: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 10 op/s 2023-09-10T02:00:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:30 smithi098 ceph-mon[103124]: pgmap v5317: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 10 op/s 2023-09-10T02:00:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:32 smithi170 ceph-mon[102832]: pgmap v5318: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 121 KiB/s wr, 16 op/s 2023-09-10T02:00:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:32 smithi082 ceph-mon[98694]: pgmap v5318: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 121 KiB/s wr, 16 op/s 2023-09-10T02:00:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:32 smithi098 ceph-mon[103124]: pgmap v5318: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 121 KiB/s wr, 16 op/s 2023-09-10T02:00:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9665.162361 secs 2023-09-10T02:00:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9665.162361 secs 2023-09-10T02:00:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9665.162361 secs 2023-09-10T02:00:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:34 smithi170 ceph-mon[102832]: pgmap v5319: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T02:00:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9665.874027 secs 2023-09-10T02:00:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:34 smithi082 ceph-mon[98694]: pgmap v5319: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T02:00:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9665.874027 secs 2023-09-10T02:00:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:34 smithi098 ceph-mon[103124]: pgmap v5319: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T02:00:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9665.874027 secs 2023-09-10T02:00:36.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:36 smithi170 ceph-mon[102832]: pgmap v5320: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T02:00:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:36 smithi082 ceph-mon[98694]: pgmap v5320: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T02:00:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:36 smithi098 ceph-mon[103124]: pgmap v5320: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T02:00:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:38 smithi170 ceph-mon[102832]: pgmap v5321: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 152 KiB/s wr, 18 op/s 2023-09-10T02:00:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9670.162443 secs 2023-09-10T02:00:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:38 smithi082 ceph-mon[98694]: pgmap v5321: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 152 KiB/s wr, 18 op/s 2023-09-10T02:00:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9670.162443 secs 2023-09-10T02:00:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:38 smithi098 ceph-mon[103124]: pgmap v5321: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 152 KiB/s wr, 18 op/s 2023-09-10T02:00:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9670.162443 secs 2023-09-10T02:00:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9670.874091 secs 2023-09-10T02:00:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9670.874091 secs 2023-09-10T02:00:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9670.874091 secs 2023-09-10T02:00:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:40 smithi170 ceph-mon[102832]: pgmap v5322: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T02:00:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:40 smithi082 ceph-mon[98694]: pgmap v5322: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T02:00:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:40 smithi098 ceph-mon[103124]: pgmap v5322: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T02:00:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:42 smithi082 ceph-mon[98694]: pgmap v5323: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 173 KiB/s wr, 18 op/s 2023-09-10T02:00:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:42 smithi098 ceph-mon[103124]: pgmap v5323: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 173 KiB/s wr, 18 op/s 2023-09-10T02:00:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:42 smithi170 ceph-mon[102832]: pgmap v5323: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 173 KiB/s wr, 18 op/s 2023-09-10T02:00:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9675.162524 secs 2023-09-10T02:00:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9675.162524 secs 2023-09-10T02:00:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9675.162524 secs 2023-09-10T02:00:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:44 smithi082 ceph-mon[98694]: pgmap v5324: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T02:00:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9675.874154 secs 2023-09-10T02:00:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:44 smithi098 ceph-mon[103124]: pgmap v5324: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T02:00:45.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9675.874154 secs 2023-09-10T02:00:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:44 smithi170 ceph-mon[102832]: pgmap v5324: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T02:00:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9675.874154 secs 2023-09-10T02:00:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:46 smithi082 ceph-mon[98694]: pgmap v5325: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T02:00:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:46 smithi098 ceph-mon[103124]: pgmap v5325: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T02:00:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:46 smithi170 ceph-mon[102832]: pgmap v5325: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T02:00:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9678.736257 secs 2023-09-10T02:00:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9678.736257 secs 2023-09-10T02:00:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9678.736257 secs 2023-09-10T02:00:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:48 smithi082 ceph-mon[98694]: pgmap v5326: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 17 op/s 2023-09-10T02:00:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9680.162581 secs 2023-09-10T02:00:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:48 smithi098 ceph-mon[103124]: pgmap v5326: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 17 op/s 2023-09-10T02:00:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9680.162581 secs 2023-09-10T02:00:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:48 smithi170 ceph-mon[102832]: pgmap v5326: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 17 op/s 2023-09-10T02:00:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9680.162581 secs 2023-09-10T02:00:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9680.874217 secs 2023-09-10T02:00:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9680.874217 secs 2023-09-10T02:00:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9680.874217 secs 2023-09-10T02:00:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:50 smithi098 ceph-mon[103124]: pgmap v5327: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T02:00:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:50 smithi170 ceph-mon[102832]: pgmap v5327: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T02:00:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:50 smithi082 ceph-mon[98694]: pgmap v5327: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T02:00:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:52 smithi170 ceph-mon[102832]: pgmap v5328: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 168 KiB/s wr, 17 op/s 2023-09-10T02:00:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:00:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:00:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:00:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:52 smithi082 ceph-mon[98694]: pgmap v5328: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 168 KiB/s wr, 17 op/s 2023-09-10T02:00:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:00:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:00:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:00:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:52 smithi098 ceph-mon[103124]: pgmap v5328: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 168 KiB/s wr, 17 op/s 2023-09-10T02:00:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:00:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:00:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:00:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9683.736328 secs 2023-09-10T02:00:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9685.162638 secs 2023-09-10T02:00:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9683.736328 secs 2023-09-10T02:00:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9685.162638 secs 2023-09-10T02:00:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9683.736328 secs 2023-09-10T02:00:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9685.162638 secs 2023-09-10T02:00:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:54 smithi170 ceph-mon[102832]: pgmap v5329: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T02:00:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9685.874279 secs 2023-09-10T02:00:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:54 smithi082 ceph-mon[98694]: pgmap v5329: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T02:00:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9685.874279 secs 2023-09-10T02:00:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:54 smithi098 ceph-mon[103124]: pgmap v5329: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T02:00:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9685.874279 secs 2023-09-10T02:00:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:56 smithi170 ceph-mon[102832]: pgmap v5330: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T02:00:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:56 smithi082 ceph-mon[98694]: pgmap v5330: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T02:00:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:56 smithi098 ceph-mon[103124]: pgmap v5330: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 90 KiB/s wr, 11 op/s 2023-09-10T02:00:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9688.736387 secs 2023-09-10T02:00:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:58 smithi170 ceph-mon[102832]: pgmap v5331: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 98 KiB/s wr, 16 op/s 2023-09-10T02:00:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9690.162766 secs 2023-09-10T02:00:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9688.736387 secs 2023-09-10T02:00:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:58 smithi082 ceph-mon[98694]: pgmap v5331: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 98 KiB/s wr, 16 op/s 2023-09-10T02:00:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9690.162766 secs 2023-09-10T02:00:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9688.736387 secs 2023-09-10T02:00:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:58 smithi098 ceph-mon[103124]: pgmap v5331: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 98 KiB/s wr, 16 op/s 2023-09-10T02:00:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9690.162766 secs 2023-09-10T02:00:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:00:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9690.874342 secs 2023-09-10T02:00:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:00:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9690.874342 secs 2023-09-10T02:00:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:00:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9690.874342 secs 2023-09-10T02:01:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:00 smithi170 ceph-mon[102832]: pgmap v5332: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 58 KiB/s wr, 10 op/s 2023-09-10T02:01:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:00 smithi082 ceph-mon[98694]: pgmap v5332: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 58 KiB/s wr, 10 op/s 2023-09-10T02:01:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:00 smithi098 ceph-mon[103124]: pgmap v5332: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 58 KiB/s wr, 10 op/s 2023-09-10T02:01:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:02 smithi170 ceph-mon[102832]: pgmap v5333: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 73 KiB/s wr, 15 op/s 2023-09-10T02:01:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:02 smithi082 ceph-mon[98694]: pgmap v5333: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 73 KiB/s wr, 15 op/s 2023-09-10T02:01:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:02 smithi098 ceph-mon[103124]: pgmap v5333: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 73 KiB/s wr, 15 op/s 2023-09-10T02:01:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9693.736480 secs 2023-09-10T02:01:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:01:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9695.162833 secs 2023-09-10T02:01:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:01:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:01:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9693.736480 secs 2023-09-10T02:01:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:01:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9695.162833 secs 2023-09-10T02:01:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:01:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:01:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9693.736480 secs 2023-09-10T02:01:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:01:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9695.162833 secs 2023-09-10T02:01:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:01:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:01:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:04 smithi082 ceph-mon[98694]: pgmap v5334: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 23 KiB/s wr, 9 op/s 2023-09-10T02:01:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:01:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:01:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9695.874377 secs 2023-09-10T02:01:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:01:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:01:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:01:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:04 smithi098 ceph-mon[103124]: pgmap v5334: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 23 KiB/s wr, 9 op/s 2023-09-10T02:01:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:01:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:01:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9695.874377 secs 2023-09-10T02:01:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:01:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:01:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:01:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:04 smithi170 ceph-mon[102832]: pgmap v5334: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 23 KiB/s wr, 9 op/s 2023-09-10T02:01:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:01:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:01:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9695.874377 secs 2023-09-10T02:01:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:01:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:01:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:01:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:06 smithi082 ceph-mon[98694]: pgmap v5335: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 23 KiB/s wr, 9 op/s 2023-09-10T02:01:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:06 smithi098 ceph-mon[103124]: pgmap v5335: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 23 KiB/s wr, 9 op/s 2023-09-10T02:01:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:06 smithi170 ceph-mon[102832]: pgmap v5335: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 23 KiB/s wr, 9 op/s 2023-09-10T02:01:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9698.736541 secs 2023-09-10T02:01:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:08 smithi082 ceph-mon[98694]: pgmap v5336: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 92 KiB/s wr, 29 op/s 2023-09-10T02:01:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:08 smithi082 ceph-mon[98694]: osdmap e403: 12 total, 12 up, 12 in 2023-09-10T02:01:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9700.162891 secs 2023-09-10T02:01:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9698.736541 secs 2023-09-10T02:01:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:08 smithi098 ceph-mon[103124]: pgmap v5336: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 92 KiB/s wr, 29 op/s 2023-09-10T02:01:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:08 smithi098 ceph-mon[103124]: osdmap e403: 12 total, 12 up, 12 in 2023-09-10T02:01:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9700.162891 secs 2023-09-10T02:01:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9698.736541 secs 2023-09-10T02:01:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:08 smithi170 ceph-mon[102832]: pgmap v5336: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 92 KiB/s wr, 29 op/s 2023-09-10T02:01:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:08 smithi170 ceph-mon[102832]: osdmap e403: 12 total, 12 up, 12 in 2023-09-10T02:01:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9700.162891 secs 2023-09-10T02:01:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9700.874438 secs 2023-09-10T02:01:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9700.874438 secs 2023-09-10T02:01:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9700.874438 secs 2023-09-10T02:01:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:10 smithi098 ceph-mon[103124]: pgmap v5338: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T02:01:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:10 smithi170 ceph-mon[102832]: pgmap v5338: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T02:01:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:10 smithi082 ceph-mon[98694]: pgmap v5338: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T02:01:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:12 smithi170 ceph-mon[102832]: pgmap v5339: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T02:01:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:12 smithi082 ceph-mon[98694]: pgmap v5339: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T02:01:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:12 smithi098 ceph-mon[103124]: pgmap v5339: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T02:01:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9703.736618 secs 2023-09-10T02:01:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9705.162947 secs 2023-09-10T02:01:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9703.736618 secs 2023-09-10T02:01:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9705.162947 secs 2023-09-10T02:01:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9703.736618 secs 2023-09-10T02:01:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9705.162947 secs 2023-09-10T02:01:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:14 smithi170 ceph-mon[102832]: pgmap v5340: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T02:01:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9705.874486 secs 2023-09-10T02:01:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:14 smithi082 ceph-mon[98694]: pgmap v5340: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T02:01:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9705.874486 secs 2023-09-10T02:01:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:14 smithi098 ceph-mon[103124]: pgmap v5340: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T02:01:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9705.874486 secs 2023-09-10T02:01:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:16 smithi170 ceph-mon[102832]: pgmap v5341: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T02:01:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:16 smithi170 ceph-mon[102832]: osdmap e404: 12 total, 12 up, 12 in 2023-09-10T02:01:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:16 smithi082 ceph-mon[98694]: pgmap v5341: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T02:01:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:16 smithi082 ceph-mon[98694]: osdmap e404: 12 total, 12 up, 12 in 2023-09-10T02:01:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:16 smithi098 ceph-mon[103124]: pgmap v5341: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T02:01:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:16 smithi098 ceph-mon[103124]: osdmap e404: 12 total, 12 up, 12 in 2023-09-10T02:01:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9708.736740 secs 2023-09-10T02:01:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:18 smithi170 ceph-mon[102832]: pgmap v5343: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T02:01:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9710.163019 secs 2023-09-10T02:01:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9708.736740 secs 2023-09-10T02:01:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:18 smithi082 ceph-mon[98694]: pgmap v5343: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T02:01:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9710.163019 secs 2023-09-10T02:01:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9708.736740 secs 2023-09-10T02:01:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:18 smithi098 ceph-mon[103124]: pgmap v5343: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T02:01:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9710.163019 secs 2023-09-10T02:01:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9710.874549 secs 2023-09-10T02:01:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9710.874549 secs 2023-09-10T02:01:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9710.874549 secs 2023-09-10T02:01:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:20 smithi170 ceph-mon[102832]: pgmap v5344: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:01:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:20 smithi082 ceph-mon[98694]: pgmap v5344: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:01:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:20 smithi098 ceph-mon[103124]: pgmap v5344: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:01:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:22 smithi170 ceph-mon[102832]: pgmap v5345: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T02:01:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:22 smithi082 ceph-mon[98694]: pgmap v5345: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T02:01:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:22 smithi098 ceph-mon[103124]: pgmap v5345: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T02:01:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:01:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9715.163101 secs 2023-09-10T02:01:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:01:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9715.163101 secs 2023-09-10T02:01:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:01:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9715.163101 secs 2023-09-10T02:01:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:24 smithi170 ceph-mon[102832]: pgmap v5346: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T02:01:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:01:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:01:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:01:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9715.874615 secs 2023-09-10T02:01:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:24 smithi082 ceph-mon[98694]: pgmap v5346: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T02:01:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:01:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:01:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:01:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9715.874615 secs 2023-09-10T02:01:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:24 smithi098 ceph-mon[103124]: pgmap v5346: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T02:01:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:01:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:01:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:01:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9715.874615 secs 2023-09-10T02:01:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:26 smithi170 ceph-mon[102832]: pgmap v5347: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T02:01:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:26 smithi082 ceph-mon[98694]: pgmap v5347: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T02:01:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:26 smithi098 ceph-mon[103124]: pgmap v5347: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T02:01:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:28 smithi170 ceph-mon[102832]: pgmap v5348: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T02:01:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9720.163186 secs 2023-09-10T02:01:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:28 smithi082 ceph-mon[98694]: pgmap v5348: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T02:01:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9720.163186 secs 2023-09-10T02:01:28.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:28 smithi098 ceph-mon[103124]: pgmap v5348: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T02:01:28.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9720.163186 secs 2023-09-10T02:01:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9720.874678 secs 2023-09-10T02:01:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9720.874678 secs 2023-09-10T02:01:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9720.874678 secs 2023-09-10T02:01:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:30 smithi170 ceph-mon[102832]: pgmap v5349: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 57 KiB/s wr, 10 op/s 2023-09-10T02:01:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:30 smithi082 ceph-mon[98694]: pgmap v5349: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 57 KiB/s wr, 10 op/s 2023-09-10T02:01:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:30 smithi098 ceph-mon[103124]: pgmap v5349: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 57 KiB/s wr, 10 op/s 2023-09-10T02:01:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:32 smithi170 ceph-mon[102832]: pgmap v5350: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T02:01:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:32 smithi082 ceph-mon[98694]: pgmap v5350: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T02:01:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:32 smithi098 ceph-mon[103124]: pgmap v5350: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T02:01:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9725.163267 secs 2023-09-10T02:01:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9725.163267 secs 2023-09-10T02:01:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9725.163267 secs 2023-09-10T02:01:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:34 smithi170 ceph-mon[102832]: pgmap v5351: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T02:01:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9725.874744 secs 2023-09-10T02:01:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:34 smithi082 ceph-mon[98694]: pgmap v5351: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T02:01:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9725.874744 secs 2023-09-10T02:01:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:34 smithi098 ceph-mon[103124]: pgmap v5351: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T02:01:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9725.874744 secs 2023-09-10T02:01:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:36 smithi170 ceph-mon[102832]: pgmap v5352: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T02:01:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:36 smithi082 ceph-mon[98694]: pgmap v5352: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T02:01:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:36 smithi098 ceph-mon[103124]: pgmap v5352: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T02:01:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:38 smithi170 ceph-mon[102832]: pgmap v5353: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 116 KiB/s wr, 18 op/s 2023-09-10T02:01:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9730.163326 secs 2023-09-10T02:01:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:38 smithi082 ceph-mon[98694]: pgmap v5353: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 116 KiB/s wr, 18 op/s 2023-09-10T02:01:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9730.163326 secs 2023-09-10T02:01:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:38 smithi098 ceph-mon[103124]: pgmap v5353: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 116 KiB/s wr, 18 op/s 2023-09-10T02:01:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9730.163326 secs 2023-09-10T02:01:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9730.874811 secs 2023-09-10T02:01:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9730.874811 secs 2023-09-10T02:01:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9730.874811 secs 2023-09-10T02:01:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:40 smithi082 ceph-mon[98694]: pgmap v5354: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T02:01:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:40 smithi098 ceph-mon[103124]: pgmap v5354: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T02:01:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:40 smithi170 ceph-mon[102832]: pgmap v5354: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T02:01:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:42 smithi170 ceph-mon[102832]: pgmap v5355: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 181 KiB/s wr, 18 op/s 2023-09-10T02:01:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:42 smithi082 ceph-mon[98694]: pgmap v5355: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 181 KiB/s wr, 18 op/s 2023-09-10T02:01:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:42 smithi098 ceph-mon[103124]: pgmap v5355: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 181 KiB/s wr, 18 op/s 2023-09-10T02:01:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9735.163384 secs 2023-09-10T02:01:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9735.163384 secs 2023-09-10T02:01:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9735.163384 secs 2023-09-10T02:01:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:44 smithi170 ceph-mon[102832]: pgmap v5356: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T02:01:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9735.874878 secs 2023-09-10T02:01:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:44 smithi082 ceph-mon[98694]: pgmap v5356: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T02:01:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9735.874878 secs 2023-09-10T02:01:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:44 smithi098 ceph-mon[103124]: pgmap v5356: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T02:01:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9735.874878 secs 2023-09-10T02:01:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:46 smithi170 ceph-mon[102832]: pgmap v5357: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T02:01:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:46 smithi082 ceph-mon[98694]: pgmap v5357: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T02:01:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:46 smithi098 ceph-mon[103124]: pgmap v5357: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T02:01:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9738.737179 secs 2023-09-10T02:01:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:48 smithi170 ceph-mon[102832]: pgmap v5358: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 112 KiB/s wr, 18 op/s 2023-09-10T02:01:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9740.163453 secs 2023-09-10T02:01:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9738.737179 secs 2023-09-10T02:01:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:48 smithi082 ceph-mon[98694]: pgmap v5358: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 112 KiB/s wr, 18 op/s 2023-09-10T02:01:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9740.163453 secs 2023-09-10T02:01:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9738.737179 secs 2023-09-10T02:01:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:48 smithi098 ceph-mon[103124]: pgmap v5358: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 112 KiB/s wr, 18 op/s 2023-09-10T02:01:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9740.163453 secs 2023-09-10T02:01:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9740.874942 secs 2023-09-10T02:01:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9740.874942 secs 2023-09-10T02:01:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9740.874942 secs 2023-09-10T02:01:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:50 smithi082 ceph-mon[98694]: pgmap v5359: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 11 op/s 2023-09-10T02:01:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:50 smithi098 ceph-mon[103124]: pgmap v5359: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 11 op/s 2023-09-10T02:01:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:50 smithi170 ceph-mon[102832]: pgmap v5359: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 11 op/s 2023-09-10T02:01:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:52 smithi170 ceph-mon[102832]: pgmap v5360: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 157 KiB/s wr, 17 op/s 2023-09-10T02:01:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:01:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:01:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:01:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:52 smithi082 ceph-mon[98694]: pgmap v5360: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 157 KiB/s wr, 17 op/s 2023-09-10T02:01:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:01:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:01:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:01:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:52 smithi098 ceph-mon[103124]: pgmap v5360: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 157 KiB/s wr, 17 op/s 2023-09-10T02:01:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:01:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:01:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:01:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9743.737240 secs 2023-09-10T02:01:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9745.163494 secs 2023-09-10T02:01:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9743.737240 secs 2023-09-10T02:01:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9745.163494 secs 2023-09-10T02:01:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9743.737240 secs 2023-09-10T02:01:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9745.163494 secs 2023-09-10T02:01:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:54 smithi170 ceph-mon[102832]: pgmap v5361: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T02:01:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9745.875009 secs 2023-09-10T02:01:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:54 smithi082 ceph-mon[98694]: pgmap v5361: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T02:01:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9745.875009 secs 2023-09-10T02:01:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:54 smithi098 ceph-mon[103124]: pgmap v5361: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T02:01:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9745.875009 secs 2023-09-10T02:01:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:56 smithi170 ceph-mon[102832]: pgmap v5362: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T02:01:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:56 smithi082 ceph-mon[98694]: pgmap v5362: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T02:01:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:56 smithi098 ceph-mon[103124]: pgmap v5362: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T02:01:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9748.737302 secs 2023-09-10T02:01:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:58 smithi082 ceph-mon[98694]: pgmap v5363: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 86 KiB/s wr, 16 op/s 2023-09-10T02:01:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9750.163551 secs 2023-09-10T02:01:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9748.737302 secs 2023-09-10T02:01:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:58 smithi098 ceph-mon[103124]: pgmap v5363: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 86 KiB/s wr, 16 op/s 2023-09-10T02:01:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9750.163551 secs 2023-09-10T02:01:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9748.737302 secs 2023-09-10T02:01:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:58 smithi170 ceph-mon[102832]: pgmap v5363: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 86 KiB/s wr, 16 op/s 2023-09-10T02:01:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9750.163551 secs 2023-09-10T02:01:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:01:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9750.875073 secs 2023-09-10T02:01:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:01:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9750.875073 secs 2023-09-10T02:01:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:01:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9750.875073 secs 2023-09-10T02:02:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:00 smithi082 ceph-mon[98694]: pgmap v5364: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 10 op/s 2023-09-10T02:02:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:00 smithi098 ceph-mon[103124]: pgmap v5364: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 10 op/s 2023-09-10T02:02:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:00 smithi170 ceph-mon[102832]: pgmap v5364: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 10 op/s 2023-09-10T02:02:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:02 smithi082 ceph-mon[98694]: pgmap v5365: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 82 KiB/s wr, 15 op/s 2023-09-10T02:02:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:02 smithi098 ceph-mon[103124]: pgmap v5365: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 82 KiB/s wr, 15 op/s 2023-09-10T02:02:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:02 smithi170 ceph-mon[102832]: pgmap v5365: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 82 KiB/s wr, 15 op/s 2023-09-10T02:02:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9753.737396 secs 2023-09-10T02:02:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9755.163607 secs 2023-09-10T02:02:03.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9753.737396 secs 2023-09-10T02:02:03.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9755.163607 secs 2023-09-10T02:02:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9753.737396 secs 2023-09-10T02:02:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9755.163607 secs 2023-09-10T02:02:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:04 smithi082 ceph-mon[98694]: pgmap v5366: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 24 KiB/s wr, 9 op/s 2023-09-10T02:02:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9755.875136 secs 2023-09-10T02:02:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:02:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:04 smithi098 ceph-mon[103124]: pgmap v5366: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 24 KiB/s wr, 9 op/s 2023-09-10T02:02:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9755.875136 secs 2023-09-10T02:02:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:02:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:04 smithi170 ceph-mon[102832]: pgmap v5366: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 24 KiB/s wr, 9 op/s 2023-09-10T02:02:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9755.875136 secs 2023-09-10T02:02:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:02:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:02:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:02:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:02:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:02:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:02:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:02:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:02:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:02:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:02:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:06 smithi098 ceph-mon[103124]: pgmap v5367: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 24 KiB/s wr, 9 op/s 2023-09-10T02:02:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:06 smithi170 ceph-mon[102832]: pgmap v5367: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 24 KiB/s wr, 9 op/s 2023-09-10T02:02:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:06 smithi082 ceph-mon[98694]: pgmap v5367: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 24 KiB/s wr, 9 op/s 2023-09-10T02:02:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9758.737455 secs 2023-09-10T02:02:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9758.737455 secs 2023-09-10T02:02:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9758.737455 secs 2023-09-10T02:02:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:08 smithi098 ceph-mon[103124]: pgmap v5368: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 92 KiB/s wr, 29 op/s 2023-09-10T02:02:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:08 smithi098 ceph-mon[103124]: osdmap e405: 12 total, 12 up, 12 in 2023-09-10T02:02:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9760.163664 secs 2023-09-10T02:02:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:08 smithi170 ceph-mon[102832]: pgmap v5368: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 92 KiB/s wr, 29 op/s 2023-09-10T02:02:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:08 smithi170 ceph-mon[102832]: osdmap e405: 12 total, 12 up, 12 in 2023-09-10T02:02:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9760.163664 secs 2023-09-10T02:02:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:08 smithi082 ceph-mon[98694]: pgmap v5368: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 92 KiB/s wr, 29 op/s 2023-09-10T02:02:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:08 smithi082 ceph-mon[98694]: osdmap e405: 12 total, 12 up, 12 in 2023-09-10T02:02:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9760.163664 secs 2023-09-10T02:02:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9760.875225 secs 2023-09-10T02:02:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9760.875225 secs 2023-09-10T02:02:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9760.875225 secs 2023-09-10T02:02:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:10 smithi098 ceph-mon[103124]: pgmap v5370: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T02:02:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:10 smithi170 ceph-mon[102832]: pgmap v5370: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T02:02:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:10 smithi082 ceph-mon[98694]: pgmap v5370: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T02:02:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:12 smithi098 ceph-mon[103124]: pgmap v5371: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T02:02:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9763.737515 secs 2023-09-10T02:02:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:12 smithi170 ceph-mon[102832]: pgmap v5371: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T02:02:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9763.737515 secs 2023-09-10T02:02:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:12 smithi082 ceph-mon[98694]: pgmap v5371: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T02:02:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9763.737515 secs 2023-09-10T02:02:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9765.163721 secs 2023-09-10T02:02:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9765.163721 secs 2023-09-10T02:02:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9765.163721 secs 2023-09-10T02:02:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:14 smithi098 ceph-mon[103124]: pgmap v5372: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T02:02:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9765.875290 secs 2023-09-10T02:02:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:14 smithi170 ceph-mon[102832]: pgmap v5372: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T02:02:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9765.875290 secs 2023-09-10T02:02:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:14 smithi082 ceph-mon[98694]: pgmap v5372: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T02:02:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9765.875290 secs 2023-09-10T02:02:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:16 smithi098 ceph-mon[103124]: pgmap v5373: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T02:02:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:16 smithi098 ceph-mon[103124]: osdmap e406: 12 total, 12 up, 12 in 2023-09-10T02:02:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:16 smithi170 ceph-mon[102832]: pgmap v5373: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T02:02:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:16 smithi170 ceph-mon[102832]: osdmap e406: 12 total, 12 up, 12 in 2023-09-10T02:02:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:16 smithi082 ceph-mon[98694]: pgmap v5373: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 29 op/s 2023-09-10T02:02:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:16 smithi082 ceph-mon[98694]: osdmap e406: 12 total, 12 up, 12 in 2023-09-10T02:02:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9768.737643 secs 2023-09-10T02:02:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9768.737643 secs 2023-09-10T02:02:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9768.737643 secs 2023-09-10T02:02:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:18 smithi170 ceph-mon[102832]: pgmap v5375: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T02:02:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9770.163836 secs 2023-09-10T02:02:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:18 smithi082 ceph-mon[98694]: pgmap v5375: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T02:02:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9770.163836 secs 2023-09-10T02:02:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:18 smithi098 ceph-mon[103124]: pgmap v5375: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T02:02:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9770.163836 secs 2023-09-10T02:02:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9770.875379 secs 2023-09-10T02:02:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9770.875379 secs 2023-09-10T02:02:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9770.875379 secs 2023-09-10T02:02:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:20 smithi170 ceph-mon[102832]: pgmap v5376: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T02:02:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:20 smithi082 ceph-mon[98694]: pgmap v5376: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T02:02:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:20 smithi098 ceph-mon[103124]: pgmap v5376: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T02:02:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:22 smithi170 ceph-mon[102832]: pgmap v5377: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T02:02:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:22 smithi082 ceph-mon[98694]: pgmap v5377: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T02:02:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:22 smithi098 ceph-mon[103124]: pgmap v5377: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T02:02:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:02:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9775.163922 secs 2023-09-10T02:02:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:02:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9775.163922 secs 2023-09-10T02:02:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:02:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9775.163922 secs 2023-09-10T02:02:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:24 smithi170 ceph-mon[102832]: pgmap v5378: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T02:02:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:02:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:02:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:02:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9775.875443 secs 2023-09-10T02:02:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:24 smithi082 ceph-mon[98694]: pgmap v5378: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T02:02:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:02:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:02:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:02:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9775.875443 secs 2023-09-10T02:02:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:24 smithi098 ceph-mon[103124]: pgmap v5378: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T02:02:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:02:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:02:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:02:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9775.875443 secs 2023-09-10T02:02:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:26 smithi170 ceph-mon[102832]: pgmap v5379: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T02:02:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:26 smithi082 ceph-mon[98694]: pgmap v5379: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T02:02:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:26 smithi098 ceph-mon[103124]: pgmap v5379: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T02:02:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:28 smithi170 ceph-mon[102832]: pgmap v5380: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-09-10T02:02:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9780.163980 secs 2023-09-10T02:02:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:28 smithi082 ceph-mon[98694]: pgmap v5380: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-09-10T02:02:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9780.163980 secs 2023-09-10T02:02:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:28 smithi098 ceph-mon[103124]: pgmap v5380: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-09-10T02:02:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9780.163980 secs 2023-09-10T02:02:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9780.875494 secs 2023-09-10T02:02:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9780.875494 secs 2023-09-10T02:02:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9780.875494 secs 2023-09-10T02:02:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:30 smithi170 ceph-mon[102832]: pgmap v5381: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T02:02:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:30 smithi082 ceph-mon[98694]: pgmap v5381: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T02:02:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:30 smithi098 ceph-mon[103124]: pgmap v5381: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T02:02:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:32 smithi170 ceph-mon[102832]: pgmap v5382: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 114 KiB/s wr, 16 op/s 2023-09-10T02:02:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:32 smithi082 ceph-mon[98694]: pgmap v5382: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 114 KiB/s wr, 16 op/s 2023-09-10T02:02:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:32 smithi098 ceph-mon[103124]: pgmap v5382: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 114 KiB/s wr, 16 op/s 2023-09-10T02:02:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9785.164040 secs 2023-09-10T02:02:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9785.164040 secs 2023-09-10T02:02:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9785.164040 secs 2023-09-10T02:02:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:34 smithi170 ceph-mon[102832]: pgmap v5383: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T02:02:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9785.875559 secs 2023-09-10T02:02:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:34 smithi082 ceph-mon[98694]: pgmap v5383: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T02:02:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9785.875559 secs 2023-09-10T02:02:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:34 smithi098 ceph-mon[103124]: pgmap v5383: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T02:02:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9785.875559 secs 2023-09-10T02:02:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:36 smithi170 ceph-mon[102832]: pgmap v5384: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T02:02:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:36 smithi082 ceph-mon[98694]: pgmap v5384: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T02:02:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:36 smithi098 ceph-mon[103124]: pgmap v5384: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T02:02:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:38 smithi170 ceph-mon[102832]: pgmap v5385: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 145 KiB/s wr, 17 op/s 2023-09-10T02:02:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9790.164102 secs 2023-09-10T02:02:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:38 smithi082 ceph-mon[98694]: pgmap v5385: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 145 KiB/s wr, 17 op/s 2023-09-10T02:02:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9790.164102 secs 2023-09-10T02:02:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:38 smithi098 ceph-mon[103124]: pgmap v5385: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 145 KiB/s wr, 17 op/s 2023-09-10T02:02:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9790.164102 secs 2023-09-10T02:02:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9790.875628 secs 2023-09-10T02:02:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9790.875628 secs 2023-09-10T02:02:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9790.875628 secs 2023-09-10T02:02:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:40 smithi170 ceph-mon[102832]: pgmap v5386: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 79 KiB/s wr, 12 op/s 2023-09-10T02:02:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:40 smithi082 ceph-mon[98694]: pgmap v5386: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 79 KiB/s wr, 12 op/s 2023-09-10T02:02:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:40 smithi098 ceph-mon[103124]: pgmap v5386: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 79 KiB/s wr, 12 op/s 2023-09-10T02:02:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:42 smithi170 ceph-mon[102832]: pgmap v5387: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 162 KiB/s wr, 19 op/s 2023-09-10T02:02:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:42 smithi082 ceph-mon[98694]: pgmap v5387: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 162 KiB/s wr, 19 op/s 2023-09-10T02:02:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:42 smithi098 ceph-mon[103124]: pgmap v5387: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 162 KiB/s wr, 19 op/s 2023-09-10T02:02:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9795.164160 secs 2023-09-10T02:02:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9795.164160 secs 2023-09-10T02:02:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9795.164160 secs 2023-09-10T02:02:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:44 smithi170 ceph-mon[102832]: pgmap v5388: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T02:02:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9795.875694 secs 2023-09-10T02:02:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:44 smithi082 ceph-mon[98694]: pgmap v5388: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T02:02:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9795.875694 secs 2023-09-10T02:02:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:44 smithi098 ceph-mon[103124]: pgmap v5388: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T02:02:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9795.875694 secs 2023-09-10T02:02:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:46 smithi170 ceph-mon[102832]: pgmap v5389: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T02:02:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:46 smithi082 ceph-mon[98694]: pgmap v5389: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T02:02:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:46 smithi098 ceph-mon[103124]: pgmap v5389: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T02:02:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9798.738062 secs 2023-09-10T02:02:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9798.738062 secs 2023-09-10T02:02:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9798.738062 secs 2023-09-10T02:02:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:48 smithi170 ceph-mon[102832]: pgmap v5390: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 17 op/s 2023-09-10T02:02:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9800.164218 secs 2023-09-10T02:02:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:48 smithi082 ceph-mon[98694]: pgmap v5390: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 17 op/s 2023-09-10T02:02:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9800.164218 secs 2023-09-10T02:02:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:48 smithi098 ceph-mon[103124]: pgmap v5390: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 17 op/s 2023-09-10T02:02:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9800.164218 secs 2023-09-10T02:02:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9800.875760 secs 2023-09-10T02:02:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9800.875760 secs 2023-09-10T02:02:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9800.875760 secs 2023-09-10T02:02:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:50 smithi170 ceph-mon[102832]: pgmap v5391: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T02:02:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:50 smithi082 ceph-mon[98694]: pgmap v5391: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T02:02:51.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:50 smithi098 ceph-mon[103124]: pgmap v5391: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T02:02:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:52 smithi082 ceph-mon[98694]: pgmap v5392: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 16 op/s 2023-09-10T02:02:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:02:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:02:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:02:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:52 smithi098 ceph-mon[103124]: pgmap v5392: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 16 op/s 2023-09-10T02:02:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:02:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:02:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:02:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:52 smithi170 ceph-mon[102832]: pgmap v5392: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 16 op/s 2023-09-10T02:02:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:02:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:02:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:02:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9803.738153 secs 2023-09-10T02:02:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9805.164274 secs 2023-09-10T02:02:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9803.738153 secs 2023-09-10T02:02:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9805.164274 secs 2023-09-10T02:02:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9803.738153 secs 2023-09-10T02:02:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9805.164274 secs 2023-09-10T02:02:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:54 smithi082 ceph-mon[98694]: pgmap v5393: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 10 op/s 2023-09-10T02:02:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9805.875827 secs 2023-09-10T02:02:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:54 smithi098 ceph-mon[103124]: pgmap v5393: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 10 op/s 2023-09-10T02:02:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9805.875827 secs 2023-09-10T02:02:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:54 smithi170 ceph-mon[102832]: pgmap v5393: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 10 op/s 2023-09-10T02:02:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9805.875827 secs 2023-09-10T02:02:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:56 smithi082 ceph-mon[98694]: pgmap v5394: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 10 op/s 2023-09-10T02:02:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:56 smithi098 ceph-mon[103124]: pgmap v5394: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 10 op/s 2023-09-10T02:02:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:56 smithi170 ceph-mon[102832]: pgmap v5394: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 10 op/s 2023-09-10T02:02:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9808.738213 secs 2023-09-10T02:02:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:58 smithi082 ceph-mon[98694]: pgmap v5395: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 132 KiB/s wr, 15 op/s 2023-09-10T02:02:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9810.164336 secs 2023-09-10T02:02:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9808.738213 secs 2023-09-10T02:02:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:58 smithi098 ceph-mon[103124]: pgmap v5395: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 132 KiB/s wr, 15 op/s 2023-09-10T02:02:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9810.164336 secs 2023-09-10T02:02:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9808.738213 secs 2023-09-10T02:02:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:58 smithi170 ceph-mon[102832]: pgmap v5395: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 132 KiB/s wr, 15 op/s 2023-09-10T02:02:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9810.164336 secs 2023-09-10T02:02:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:02:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9810.875891 secs 2023-09-10T02:02:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:02:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9810.875891 secs 2023-09-10T02:02:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:02:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9810.875891 secs 2023-09-10T02:03:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:00 smithi082 ceph-mon[98694]: pgmap v5396: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 10 op/s 2023-09-10T02:03:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:00 smithi098 ceph-mon[103124]: pgmap v5396: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 10 op/s 2023-09-10T02:03:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:00 smithi170 ceph-mon[102832]: pgmap v5396: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 10 op/s 2023-09-10T02:03:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:02 smithi082 ceph-mon[98694]: pgmap v5397: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 15 op/s 2023-09-10T02:03:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:02 smithi098 ceph-mon[103124]: pgmap v5397: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 15 op/s 2023-09-10T02:03:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:02 smithi170 ceph-mon[102832]: pgmap v5397: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 15 op/s 2023-09-10T02:03:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9813.738274 secs 2023-09-10T02:03:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9815.164397 secs 2023-09-10T02:03:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9813.738274 secs 2023-09-10T02:03:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9815.164397 secs 2023-09-10T02:03:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9813.738274 secs 2023-09-10T02:03:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9815.164397 secs 2023-09-10T02:03:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:04 smithi098 ceph-mon[103124]: pgmap v5398: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 9 op/s 2023-09-10T02:03:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9815.875957 secs 2023-09-10T02:03:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:04 smithi170 ceph-mon[102832]: pgmap v5398: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 9 op/s 2023-09-10T02:03:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9815.875957 secs 2023-09-10T02:03:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:04 smithi082 ceph-mon[98694]: pgmap v5398: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 9 op/s 2023-09-10T02:03:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9815.875957 secs 2023-09-10T02:03:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:03:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:03:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:03:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:03:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:03:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:03:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:03:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:03:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:03:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:03:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:03:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:03:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:06 smithi098 ceph-mon[103124]: pgmap v5399: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 9 op/s 2023-09-10T02:03:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:06 smithi170 ceph-mon[102832]: pgmap v5399: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 9 op/s 2023-09-10T02:03:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:06 smithi082 ceph-mon[98694]: pgmap v5399: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 9 op/s 2023-09-10T02:03:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9818.738332 secs 2023-09-10T02:03:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:08 smithi098 ceph-mon[103124]: pgmap v5400: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T02:03:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:08 smithi098 ceph-mon[103124]: osdmap e407: 12 total, 12 up, 12 in 2023-09-10T02:03:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9820.164456 secs 2023-09-10T02:03:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9818.738332 secs 2023-09-10T02:03:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:08 smithi170 ceph-mon[102832]: pgmap v5400: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T02:03:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:08 smithi170 ceph-mon[102832]: osdmap e407: 12 total, 12 up, 12 in 2023-09-10T02:03:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9820.164456 secs 2023-09-10T02:03:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9818.738332 secs 2023-09-10T02:03:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:08 smithi082 ceph-mon[98694]: pgmap v5400: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T02:03:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:08 smithi082 ceph-mon[98694]: osdmap e407: 12 total, 12 up, 12 in 2023-09-10T02:03:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9820.164456 secs 2023-09-10T02:03:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9820.876021 secs 2023-09-10T02:03:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9820.876021 secs 2023-09-10T02:03:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9820.876021 secs 2023-09-10T02:03:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:10 smithi098 ceph-mon[103124]: pgmap v5402: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 29 op/s 2023-09-10T02:03:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:10 smithi170 ceph-mon[102832]: pgmap v5402: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 29 op/s 2023-09-10T02:03:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:10 smithi082 ceph-mon[98694]: pgmap v5402: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 29 op/s 2023-09-10T02:03:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:12 smithi098 ceph-mon[103124]: pgmap v5403: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 92 KiB/s wr, 29 op/s 2023-09-10T02:03:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:12 smithi170 ceph-mon[102832]: pgmap v5403: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 92 KiB/s wr, 29 op/s 2023-09-10T02:03:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:12 smithi082 ceph-mon[98694]: pgmap v5403: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 92 KiB/s wr, 29 op/s 2023-09-10T02:03:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9823.738392 secs 2023-09-10T02:03:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9825.164509 secs 2023-09-10T02:03:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9823.738392 secs 2023-09-10T02:03:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9825.164509 secs 2023-09-10T02:03:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9823.738392 secs 2023-09-10T02:03:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9825.164509 secs 2023-09-10T02:03:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:14 smithi098 ceph-mon[103124]: pgmap v5404: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 92 KiB/s wr, 29 op/s 2023-09-10T02:03:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9825.876089 secs 2023-09-10T02:03:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:14 smithi170 ceph-mon[102832]: pgmap v5404: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 92 KiB/s wr, 29 op/s 2023-09-10T02:03:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9825.876089 secs 2023-09-10T02:03:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:14 smithi082 ceph-mon[98694]: pgmap v5404: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 92 KiB/s wr, 29 op/s 2023-09-10T02:03:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9825.876089 secs 2023-09-10T02:03:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:16 smithi098 ceph-mon[103124]: pgmap v5405: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 92 KiB/s wr, 29 op/s 2023-09-10T02:03:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:16 smithi098 ceph-mon[103124]: osdmap e408: 12 total, 12 up, 12 in 2023-09-10T02:03:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:16 smithi170 ceph-mon[102832]: pgmap v5405: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 92 KiB/s wr, 29 op/s 2023-09-10T02:03:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:16 smithi170 ceph-mon[102832]: osdmap e408: 12 total, 12 up, 12 in 2023-09-10T02:03:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:16 smithi082 ceph-mon[98694]: pgmap v5405: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 92 KiB/s wr, 29 op/s 2023-09-10T02:03:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:16 smithi082 ceph-mon[98694]: osdmap e408: 12 total, 12 up, 12 in 2023-09-10T02:03:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9828.738454 secs 2023-09-10T02:03:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:18 smithi098 ceph-mon[103124]: pgmap v5407: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T02:03:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9830.164567 secs 2023-09-10T02:03:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9828.738454 secs 2023-09-10T02:03:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:18 smithi170 ceph-mon[102832]: pgmap v5407: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T02:03:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9830.164567 secs 2023-09-10T02:03:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9828.738454 secs 2023-09-10T02:03:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:18 smithi082 ceph-mon[98694]: pgmap v5407: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T02:03:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9830.164567 secs 2023-09-10T02:03:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9830.876154 secs 2023-09-10T02:03:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9830.876154 secs 2023-09-10T02:03:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9830.876154 secs 2023-09-10T02:03:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:20 smithi098 ceph-mon[103124]: pgmap v5408: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T02:03:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:20 smithi170 ceph-mon[102832]: pgmap v5408: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T02:03:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:20 smithi082 ceph-mon[98694]: pgmap v5408: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T02:03:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:22 smithi098 ceph-mon[103124]: pgmap v5409: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T02:03:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:22 smithi170 ceph-mon[102832]: pgmap v5409: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T02:03:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:22 smithi082 ceph-mon[98694]: pgmap v5409: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T02:03:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:03:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9835.164624 secs 2023-09-10T02:03:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:03:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9835.164624 secs 2023-09-10T02:03:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:03:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9835.164624 secs 2023-09-10T02:03:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:24 smithi170 ceph-mon[102832]: pgmap v5410: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T02:03:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:03:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:03:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:03:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9835.876218 secs 2023-09-10T02:03:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:24 smithi082 ceph-mon[98694]: pgmap v5410: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T02:03:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:03:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:03:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:03:24.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9835.876218 secs 2023-09-10T02:03:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:24 smithi098 ceph-mon[103124]: pgmap v5410: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T02:03:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:03:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:03:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:03:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9835.876218 secs 2023-09-10T02:03:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:26 smithi170 ceph-mon[102832]: pgmap v5411: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T02:03:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:26 smithi082 ceph-mon[98694]: pgmap v5411: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T02:03:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:26 smithi098 ceph-mon[103124]: pgmap v5411: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T02:03:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:28 smithi170 ceph-mon[102832]: pgmap v5412: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T02:03:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9840.164683 secs 2023-09-10T02:03:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:28 smithi082 ceph-mon[98694]: pgmap v5412: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T02:03:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9840.164683 secs 2023-09-10T02:03:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:28 smithi098 ceph-mon[103124]: pgmap v5412: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T02:03:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9840.164683 secs 2023-09-10T02:03:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9840.876283 secs 2023-09-10T02:03:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9840.876283 secs 2023-09-10T02:03:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9840.876283 secs 2023-09-10T02:03:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:30 smithi170 ceph-mon[102832]: pgmap v5413: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 9 op/s 2023-09-10T02:03:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:30 smithi082 ceph-mon[98694]: pgmap v5413: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 9 op/s 2023-09-10T02:03:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:30 smithi098 ceph-mon[103124]: pgmap v5413: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 9 op/s 2023-09-10T02:03:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:32 smithi170 ceph-mon[102832]: pgmap v5414: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 98 KiB/s wr, 14 op/s 2023-09-10T02:03:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:32 smithi082 ceph-mon[98694]: pgmap v5414: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 98 KiB/s wr, 14 op/s 2023-09-10T02:03:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:32 smithi098 ceph-mon[103124]: pgmap v5414: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 98 KiB/s wr, 14 op/s 2023-09-10T02:03:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9845.164741 secs 2023-09-10T02:03:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9845.164741 secs 2023-09-10T02:03:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9845.164741 secs 2023-09-10T02:03:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:34 smithi170 ceph-mon[102832]: pgmap v5415: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T02:03:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9845.876349 secs 2023-09-10T02:03:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:34 smithi082 ceph-mon[98694]: pgmap v5415: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T02:03:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9845.876349 secs 2023-09-10T02:03:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:34 smithi098 ceph-mon[103124]: pgmap v5415: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T02:03:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9845.876349 secs 2023-09-10T02:03:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:36 smithi170 ceph-mon[102832]: pgmap v5416: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T02:03:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:36 smithi082 ceph-mon[98694]: pgmap v5416: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T02:03:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:36 smithi098 ceph-mon[103124]: pgmap v5416: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T02:03:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:38 smithi170 ceph-mon[102832]: pgmap v5417: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 107 KiB/s wr, 16 op/s 2023-09-10T02:03:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9850.164799 secs 2023-09-10T02:03:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:38 smithi082 ceph-mon[98694]: pgmap v5417: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 107 KiB/s wr, 16 op/s 2023-09-10T02:03:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9850.164799 secs 2023-09-10T02:03:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:38 smithi098 ceph-mon[103124]: pgmap v5417: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 107 KiB/s wr, 16 op/s 2023-09-10T02:03:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9850.164799 secs 2023-09-10T02:03:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9850.876412 secs 2023-09-10T02:03:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9850.876412 secs 2023-09-10T02:03:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9850.876412 secs 2023-09-10T02:03:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:40 smithi170 ceph-mon[102832]: pgmap v5418: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T02:03:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:40 smithi082 ceph-mon[98694]: pgmap v5418: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T02:03:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:40 smithi098 ceph-mon[103124]: pgmap v5418: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T02:03:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:42 smithi170 ceph-mon[102832]: pgmap v5419: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 149 KiB/s wr, 18 op/s 2023-09-10T02:03:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:42 smithi082 ceph-mon[98694]: pgmap v5419: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 149 KiB/s wr, 18 op/s 2023-09-10T02:03:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:42 smithi098 ceph-mon[103124]: pgmap v5419: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 149 KiB/s wr, 18 op/s 2023-09-10T02:03:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9855.164883 secs 2023-09-10T02:03:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9855.164883 secs 2023-09-10T02:03:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9855.164883 secs 2023-09-10T02:03:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:44 smithi170 ceph-mon[102832]: pgmap v5420: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T02:03:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9855.876485 secs 2023-09-10T02:03:44.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:44 smithi082 ceph-mon[98694]: pgmap v5420: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T02:03:44.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9855.876485 secs 2023-09-10T02:03:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:44 smithi098 ceph-mon[103124]: pgmap v5420: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T02:03:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9855.876485 secs 2023-09-10T02:03:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:46 smithi170 ceph-mon[102832]: pgmap v5421: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T02:03:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:46 smithi082 ceph-mon[98694]: pgmap v5421: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T02:03:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:46 smithi098 ceph-mon[103124]: pgmap v5421: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T02:03:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:48 smithi170 ceph-mon[102832]: pgmap v5422: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 150 KiB/s wr, 18 op/s 2023-09-10T02:03:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9860.164973 secs 2023-09-10T02:03:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:48 smithi082 ceph-mon[98694]: pgmap v5422: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 150 KiB/s wr, 18 op/s 2023-09-10T02:03:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9860.164973 secs 2023-09-10T02:03:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:48 smithi098 ceph-mon[103124]: pgmap v5422: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 150 KiB/s wr, 18 op/s 2023-09-10T02:03:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9860.164973 secs 2023-09-10T02:03:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9860.876548 secs 2023-09-10T02:03:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9860.876548 secs 2023-09-10T02:03:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9860.876548 secs 2023-09-10T02:03:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:50 smithi170 ceph-mon[102832]: pgmap v5423: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T02:03:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:50 smithi082 ceph-mon[98694]: pgmap v5423: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T02:03:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:50 smithi098 ceph-mon[103124]: pgmap v5423: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T02:03:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:52 smithi170 ceph-mon[102832]: pgmap v5424: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 187 KiB/s wr, 18 op/s 2023-09-10T02:03:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:52 smithi082 ceph-mon[98694]: pgmap v5424: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 187 KiB/s wr, 18 op/s 2023-09-10T02:03:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:52 smithi098 ceph-mon[103124]: pgmap v5424: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 187 KiB/s wr, 18 op/s 2023-09-10T02:03:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9863.738987 secs 2023-09-10T02:03:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9865.165063 secs 2023-09-10T02:03:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9863.738987 secs 2023-09-10T02:03:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9865.165063 secs 2023-09-10T02:03:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9863.738987 secs 2023-09-10T02:03:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9865.165063 secs 2023-09-10T02:03:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:54 smithi170 ceph-mon[102832]: pgmap v5425: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T02:03:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9865.876613 secs 2023-09-10T02:03:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:54 smithi082 ceph-mon[98694]: pgmap v5425: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T02:03:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9865.876613 secs 2023-09-10T02:03:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:54 smithi098 ceph-mon[103124]: pgmap v5425: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T02:03:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9865.876613 secs 2023-09-10T02:03:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:03:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:03:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:03:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:03:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:03:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:03:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:03:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:03:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:03:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:56 smithi170 ceph-mon[102832]: pgmap v5426: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T02:03:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:56 smithi082 ceph-mon[98694]: pgmap v5426: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T02:03:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:56 smithi098 ceph-mon[103124]: pgmap v5426: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T02:03:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9868.739089 secs 2023-09-10T02:03:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9868.739089 secs 2023-09-10T02:03:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9868.739089 secs 2023-09-10T02:03:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:58 smithi170 ceph-mon[102832]: pgmap v5427: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 16 op/s 2023-09-10T02:03:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9870.165122 secs 2023-09-10T02:03:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:58 smithi082 ceph-mon[98694]: pgmap v5427: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 16 op/s 2023-09-10T02:03:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9870.165122 secs 2023-09-10T02:03:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:58 smithi098 ceph-mon[103124]: pgmap v5427: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 16 op/s 2023-09-10T02:03:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9870.165122 secs 2023-09-10T02:03:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:03:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9870.876678 secs 2023-09-10T02:03:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:03:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9870.876678 secs 2023-09-10T02:03:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:03:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9870.876678 secs 2023-09-10T02:04:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:00 smithi170 ceph-mon[102832]: pgmap v5428: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T02:04:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:00 smithi082 ceph-mon[98694]: pgmap v5428: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T02:04:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:00 smithi098 ceph-mon[103124]: pgmap v5428: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T02:04:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:02 smithi170 ceph-mon[102832]: pgmap v5429: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 142 KiB/s wr, 16 op/s 2023-09-10T02:04:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9873.739177 secs 2023-09-10T02:04:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:02 smithi082 ceph-mon[98694]: pgmap v5429: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 142 KiB/s wr, 16 op/s 2023-09-10T02:04:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9873.739177 secs 2023-09-10T02:04:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:02 smithi098 ceph-mon[103124]: pgmap v5429: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 142 KiB/s wr, 16 op/s 2023-09-10T02:04:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9873.739177 secs 2023-09-10T02:04:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9875.165193 secs 2023-09-10T02:04:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9875.165193 secs 2023-09-10T02:04:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9875.165193 secs 2023-09-10T02:04:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:04 smithi170 ceph-mon[102832]: pgmap v5430: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 72 KiB/s wr, 10 op/s 2023-09-10T02:04:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9875.876743 secs 2023-09-10T02:04:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:04 smithi082 ceph-mon[98694]: pgmap v5430: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 72 KiB/s wr, 10 op/s 2023-09-10T02:04:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9875.876743 secs 2023-09-10T02:04:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:04 smithi098 ceph-mon[103124]: pgmap v5430: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 72 KiB/s wr, 10 op/s 2023-09-10T02:04:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9875.876743 secs 2023-09-10T02:04:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:04:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:04:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:04:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:06 smithi170 ceph-mon[102832]: pgmap v5431: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 72 KiB/s wr, 10 op/s 2023-09-10T02:04:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:04:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:04:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:04:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:06 smithi082 ceph-mon[98694]: pgmap v5431: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 72 KiB/s wr, 10 op/s 2023-09-10T02:04:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:04:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:04:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:04:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:06 smithi098 ceph-mon[103124]: pgmap v5431: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 72 KiB/s wr, 10 op/s 2023-09-10T02:04:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:04:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:04:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:04:07.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9878.739238 secs 2023-09-10T02:04:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9878.739238 secs 2023-09-10T02:04:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9878.739238 secs 2023-09-10T02:04:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:08 smithi170 ceph-mon[102832]: pgmap v5432: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 141 KiB/s wr, 30 op/s 2023-09-10T02:04:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:08 smithi170 ceph-mon[102832]: osdmap e409: 12 total, 12 up, 12 in 2023-09-10T02:04:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9880.165255 secs 2023-09-10T02:04:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:08 smithi082 ceph-mon[98694]: pgmap v5432: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 141 KiB/s wr, 30 op/s 2023-09-10T02:04:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:08 smithi082 ceph-mon[98694]: osdmap e409: 12 total, 12 up, 12 in 2023-09-10T02:04:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9880.165255 secs 2023-09-10T02:04:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:08 smithi098 ceph-mon[103124]: pgmap v5432: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 141 KiB/s wr, 30 op/s 2023-09-10T02:04:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:08 smithi098 ceph-mon[103124]: osdmap e409: 12 total, 12 up, 12 in 2023-09-10T02:04:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9880.165255 secs 2023-09-10T02:04:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9880.876806 secs 2023-09-10T02:04:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9880.876806 secs 2023-09-10T02:04:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9880.876806 secs 2023-09-10T02:04:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:10 smithi170 ceph-mon[102832]: pgmap v5434: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 142 KiB/s wr, 29 op/s 2023-09-10T02:04:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:10 smithi082 ceph-mon[98694]: pgmap v5434: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 142 KiB/s wr, 29 op/s 2023-09-10T02:04:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:10 smithi098 ceph-mon[103124]: pgmap v5434: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 142 KiB/s wr, 29 op/s 2023-09-10T02:04:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:12 smithi098 ceph-mon[103124]: pgmap v5435: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T02:04:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:12 smithi170 ceph-mon[102832]: pgmap v5435: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T02:04:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:12 smithi082 ceph-mon[98694]: pgmap v5435: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T02:04:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9883.739305 secs 2023-09-10T02:04:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9885.165316 secs 2023-09-10T02:04:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9883.739305 secs 2023-09-10T02:04:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9885.165316 secs 2023-09-10T02:04:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9883.739305 secs 2023-09-10T02:04:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9885.165316 secs 2023-09-10T02:04:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:14 smithi098 ceph-mon[103124]: pgmap v5436: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T02:04:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9885.876877 secs 2023-09-10T02:04:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:14 smithi170 ceph-mon[102832]: pgmap v5436: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T02:04:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9885.876877 secs 2023-09-10T02:04:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:14 smithi082 ceph-mon[98694]: pgmap v5436: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T02:04:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9885.876877 secs 2023-09-10T02:04:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:16 smithi098 ceph-mon[103124]: pgmap v5437: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T02:04:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:16 smithi098 ceph-mon[103124]: osdmap e410: 12 total, 12 up, 12 in 2023-09-10T02:04:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:16 smithi170 ceph-mon[102832]: pgmap v5437: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T02:04:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:16 smithi170 ceph-mon[102832]: osdmap e410: 12 total, 12 up, 12 in 2023-09-10T02:04:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:16 smithi082 ceph-mon[98694]: pgmap v5437: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T02:04:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:16 smithi082 ceph-mon[98694]: osdmap e410: 12 total, 12 up, 12 in 2023-09-10T02:04:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9888.739417 secs 2023-09-10T02:04:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:18 smithi098 ceph-mon[103124]: pgmap v5439: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-10T02:04:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9890.165399 secs 2023-09-10T02:04:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9888.739417 secs 2023-09-10T02:04:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:18 smithi170 ceph-mon[102832]: pgmap v5439: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-10T02:04:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9890.165399 secs 2023-09-10T02:04:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9888.739417 secs 2023-09-10T02:04:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:18 smithi082 ceph-mon[98694]: pgmap v5439: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-10T02:04:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9890.165399 secs 2023-09-10T02:04:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9890.876939 secs 2023-09-10T02:04:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9890.876939 secs 2023-09-10T02:04:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9890.876939 secs 2023-09-10T02:04:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:20 smithi098 ceph-mon[103124]: pgmap v5440: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T02:04:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:20 smithi170 ceph-mon[102832]: pgmap v5440: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T02:04:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:20 smithi082 ceph-mon[98694]: pgmap v5440: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T02:04:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:22 smithi098 ceph-mon[103124]: pgmap v5441: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T02:04:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:22 smithi170 ceph-mon[102832]: pgmap v5441: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T02:04:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:22 smithi082 ceph-mon[98694]: pgmap v5441: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T02:04:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:04:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9895.165480 secs 2023-09-10T02:04:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:04:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9895.165480 secs 2023-09-10T02:04:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:04:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9895.165480 secs 2023-09-10T02:04:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:24 smithi098 ceph-mon[103124]: pgmap v5442: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T02:04:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:04:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:04:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:04:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9895.877003 secs 2023-09-10T02:04:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:24 smithi170 ceph-mon[102832]: pgmap v5442: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T02:04:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:04:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:04:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:04:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9895.877003 secs 2023-09-10T02:04:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:24 smithi082 ceph-mon[98694]: pgmap v5442: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T02:04:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:04:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:04:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:04:24.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9895.877003 secs 2023-09-10T02:04:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:26 smithi098 ceph-mon[103124]: pgmap v5443: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T02:04:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:26 smithi170 ceph-mon[102832]: pgmap v5443: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T02:04:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:26 smithi082 ceph-mon[98694]: pgmap v5443: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T02:04:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:28 smithi098 ceph-mon[103124]: pgmap v5444: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T02:04:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9900.165495 secs 2023-09-10T02:04:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:28 smithi170 ceph-mon[102832]: pgmap v5444: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T02:04:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9900.165495 secs 2023-09-10T02:04:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:28 smithi082 ceph-mon[98694]: pgmap v5444: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T02:04:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9900.165495 secs 2023-09-10T02:04:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9900.877072 secs 2023-09-10T02:04:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9900.877072 secs 2023-09-10T02:04:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9900.877072 secs 2023-09-10T02:04:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:30 smithi098 ceph-mon[103124]: pgmap v5445: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 60 KiB/s wr, 9 op/s 2023-09-10T02:04:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:30 smithi170 ceph-mon[102832]: pgmap v5445: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 60 KiB/s wr, 9 op/s 2023-09-10T02:04:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:30 smithi082 ceph-mon[98694]: pgmap v5445: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 60 KiB/s wr, 9 op/s 2023-09-10T02:04:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:32 smithi098 ceph-mon[103124]: pgmap v5446: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 112 KiB/s wr, 16 op/s 2023-09-10T02:04:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:32 smithi170 ceph-mon[102832]: pgmap v5446: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 112 KiB/s wr, 16 op/s 2023-09-10T02:04:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:32 smithi082 ceph-mon[98694]: pgmap v5446: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 112 KiB/s wr, 16 op/s 2023-09-10T02:04:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9905.165556 secs 2023-09-10T02:04:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9905.165556 secs 2023-09-10T02:04:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9905.165556 secs 2023-09-10T02:04:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:34 smithi098 ceph-mon[103124]: pgmap v5447: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T02:04:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9905.877139 secs 2023-09-10T02:04:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:34 smithi170 ceph-mon[102832]: pgmap v5447: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T02:04:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9905.877139 secs 2023-09-10T02:04:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:34 smithi082 ceph-mon[98694]: pgmap v5447: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T02:04:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9905.877139 secs 2023-09-10T02:04:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:36 smithi098 ceph-mon[103124]: pgmap v5448: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T02:04:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:36 smithi170 ceph-mon[102832]: pgmap v5448: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T02:04:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:36 smithi082 ceph-mon[98694]: pgmap v5448: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T02:04:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:38 smithi098 ceph-mon[103124]: pgmap v5449: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 155 KiB/s wr, 18 op/s 2023-09-10T02:04:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9910.165614 secs 2023-09-10T02:04:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:38 smithi170 ceph-mon[102832]: pgmap v5449: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 155 KiB/s wr, 18 op/s 2023-09-10T02:04:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9910.165614 secs 2023-09-10T02:04:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:38 smithi082 ceph-mon[98694]: pgmap v5449: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 155 KiB/s wr, 18 op/s 2023-09-10T02:04:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9910.165614 secs 2023-09-10T02:04:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9910.877204 secs 2023-09-10T02:04:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9910.877204 secs 2023-09-10T02:04:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9910.877204 secs 2023-09-10T02:04:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:40 smithi098 ceph-mon[103124]: pgmap v5450: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 139 KiB/s wr, 12 op/s 2023-09-10T02:04:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:40 smithi170 ceph-mon[102832]: pgmap v5450: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 139 KiB/s wr, 12 op/s 2023-09-10T02:04:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:40 smithi082 ceph-mon[98694]: pgmap v5450: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 139 KiB/s wr, 12 op/s 2023-09-10T02:04:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:42 smithi098 ceph-mon[103124]: pgmap v5451: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 205 KiB/s wr, 19 op/s 2023-09-10T02:04:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:42 smithi170 ceph-mon[102832]: pgmap v5451: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 205 KiB/s wr, 19 op/s 2023-09-10T02:04:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:42 smithi082 ceph-mon[98694]: pgmap v5451: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 205 KiB/s wr, 19 op/s 2023-09-10T02:04:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9913.739775 secs 2023-09-10T02:04:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9915.165685 secs 2023-09-10T02:04:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9913.739775 secs 2023-09-10T02:04:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9915.165685 secs 2023-09-10T02:04:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9913.739775 secs 2023-09-10T02:04:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9915.165685 secs 2023-09-10T02:04:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:44 smithi098 ceph-mon[103124]: pgmap v5452: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 153 KiB/s wr, 13 op/s 2023-09-10T02:04:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:04:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:04:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:04:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9915.877269 secs 2023-09-10T02:04:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:44 smithi170 ceph-mon[102832]: pgmap v5452: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 153 KiB/s wr, 13 op/s 2023-09-10T02:04:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:04:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:04:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:04:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9915.877269 secs 2023-09-10T02:04:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:44 smithi082 ceph-mon[98694]: pgmap v5452: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 153 KiB/s wr, 13 op/s 2023-09-10T02:04:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:04:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:04:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:04:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9915.877269 secs 2023-09-10T02:04:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:46 smithi098 ceph-mon[103124]: pgmap v5453: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 153 KiB/s wr, 13 op/s 2023-09-10T02:04:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:46 smithi170 ceph-mon[102832]: pgmap v5453: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 153 KiB/s wr, 13 op/s 2023-09-10T02:04:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:46 smithi082 ceph-mon[98694]: pgmap v5453: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 153 KiB/s wr, 13 op/s 2023-09-10T02:04:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9918.739837 secs 2023-09-10T02:04:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:48 smithi098 ceph-mon[103124]: pgmap v5454: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 169 KiB/s wr, 18 op/s 2023-09-10T02:04:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9920.165770 secs 2023-09-10T02:04:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9918.739837 secs 2023-09-10T02:04:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:48 smithi170 ceph-mon[102832]: pgmap v5454: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 169 KiB/s wr, 18 op/s 2023-09-10T02:04:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9920.165770 secs 2023-09-10T02:04:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9918.739837 secs 2023-09-10T02:04:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:48 smithi082 ceph-mon[98694]: pgmap v5454: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 169 KiB/s wr, 18 op/s 2023-09-10T02:04:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9920.165770 secs 2023-09-10T02:04:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9920.877337 secs 2023-09-10T02:04:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9920.877337 secs 2023-09-10T02:04:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9920.877337 secs 2023-09-10T02:04:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:50 smithi098 ceph-mon[103124]: pgmap v5455: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 83 KiB/s wr, 12 op/s 2023-09-10T02:04:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:50 smithi170 ceph-mon[102832]: pgmap v5455: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 83 KiB/s wr, 12 op/s 2023-09-10T02:04:50.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:50 smithi082 ceph-mon[98694]: pgmap v5455: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 83 KiB/s wr, 12 op/s 2023-09-10T02:04:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:52 smithi098 ceph-mon[103124]: pgmap v5456: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 124 KiB/s wr, 17 op/s 2023-09-10T02:04:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:52 smithi170 ceph-mon[102832]: pgmap v5456: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 124 KiB/s wr, 17 op/s 2023-09-10T02:04:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:52 smithi082 ceph-mon[98694]: pgmap v5456: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 124 KiB/s wr, 17 op/s 2023-09-10T02:04:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9923.739898 secs 2023-09-10T02:04:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9925.165827 secs 2023-09-10T02:04:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9923.739898 secs 2023-09-10T02:04:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9925.165827 secs 2023-09-10T02:04:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9923.739898 secs 2023-09-10T02:04:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9925.165827 secs 2023-09-10T02:04:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:54 smithi098 ceph-mon[103124]: pgmap v5457: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 58 KiB/s wr, 10 op/s 2023-09-10T02:04:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9925.877404 secs 2023-09-10T02:04:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:54 smithi170 ceph-mon[102832]: pgmap v5457: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 58 KiB/s wr, 10 op/s 2023-09-10T02:04:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9925.877404 secs 2023-09-10T02:04:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:54 smithi082 ceph-mon[98694]: pgmap v5457: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 58 KiB/s wr, 10 op/s 2023-09-10T02:04:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9925.877404 secs 2023-09-10T02:04:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:56 smithi098 ceph-mon[103124]: pgmap v5458: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 58 KiB/s wr, 10 op/s 2023-09-10T02:04:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:56 smithi170 ceph-mon[102832]: pgmap v5458: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 58 KiB/s wr, 10 op/s 2023-09-10T02:04:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:56 smithi082 ceph-mon[98694]: pgmap v5458: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 58 KiB/s wr, 10 op/s 2023-09-10T02:04:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9928.739959 secs 2023-09-10T02:04:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:58 smithi098 ceph-mon[103124]: pgmap v5459: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 66 KiB/s wr, 15 op/s 2023-09-10T02:04:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9930.165888 secs 2023-09-10T02:04:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9928.739959 secs 2023-09-10T02:04:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:58 smithi170 ceph-mon[102832]: pgmap v5459: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 66 KiB/s wr, 15 op/s 2023-09-10T02:04:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9930.165888 secs 2023-09-10T02:04:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9928.739959 secs 2023-09-10T02:04:59.014 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:58 smithi082 ceph-mon[98694]: pgmap v5459: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 66 KiB/s wr, 15 op/s 2023-09-10T02:04:59.015 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9930.165888 secs 2023-09-10T02:04:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:04:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9930.877475 secs 2023-09-10T02:04:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:04:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9930.877475 secs 2023-09-10T02:04:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:04:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9930.877475 secs 2023-09-10T02:05:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:00 smithi098 ceph-mon[103124]: pgmap v5460: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T02:05:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:00 smithi170 ceph-mon[102832]: pgmap v5460: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T02:05:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:00 smithi082 ceph-mon[98694]: pgmap v5460: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T02:05:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:02 smithi098 ceph-mon[103124]: pgmap v5461: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 89 KiB/s wr, 15 op/s 2023-09-10T02:05:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:02 smithi170 ceph-mon[102832]: pgmap v5461: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 89 KiB/s wr, 15 op/s 2023-09-10T02:05:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:02 smithi082 ceph-mon[98694]: pgmap v5461: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 89 KiB/s wr, 15 op/s 2023-09-10T02:05:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9933.740021 secs 2023-09-10T02:05:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9935.165971 secs 2023-09-10T02:05:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9933.740021 secs 2023-09-10T02:05:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9935.165971 secs 2023-09-10T02:05:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9933.740021 secs 2023-09-10T02:05:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9935.165971 secs 2023-09-10T02:05:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:04 smithi098 ceph-mon[103124]: pgmap v5462: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 47 KiB/s wr, 10 op/s 2023-09-10T02:05:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9935.877537 secs 2023-09-10T02:05:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:04 smithi170 ceph-mon[102832]: pgmap v5462: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 47 KiB/s wr, 10 op/s 2023-09-10T02:05:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9935.877537 secs 2023-09-10T02:05:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:04 smithi082 ceph-mon[98694]: pgmap v5462: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 47 KiB/s wr, 10 op/s 2023-09-10T02:05:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9935.877537 secs 2023-09-10T02:05:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:06 smithi098 ceph-mon[103124]: pgmap v5463: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 47 KiB/s wr, 10 op/s 2023-09-10T02:05:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:05:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:06 smithi170 ceph-mon[102832]: pgmap v5463: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 47 KiB/s wr, 10 op/s 2023-09-10T02:05:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:05:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:06 smithi082 ceph-mon[98694]: pgmap v5463: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 47 KiB/s wr, 10 op/s 2023-09-10T02:05:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:05:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:05:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:05:07.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:05:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:05:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:05:07.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:05:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:05:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:05:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:05:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9938.740081 secs 2023-09-10T02:05:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:08 smithi098 ceph-mon[103124]: pgmap v5464: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 70 KiB/s wr, 28 op/s 2023-09-10T02:05:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9940.166029 secs 2023-09-10T02:05:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9938.740081 secs 2023-09-10T02:05:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:08 smithi170 ceph-mon[102832]: pgmap v5464: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 70 KiB/s wr, 28 op/s 2023-09-10T02:05:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9940.166029 secs 2023-09-10T02:05:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9938.740081 secs 2023-09-10T02:05:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:08 smithi082 ceph-mon[98694]: pgmap v5464: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 70 KiB/s wr, 28 op/s 2023-09-10T02:05:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9940.166029 secs 2023-09-10T02:05:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9940.877602 secs 2023-09-10T02:05:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9940.877602 secs 2023-09-10T02:05:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9940.877602 secs 2023-09-10T02:05:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:10 smithi098 ceph-mon[103124]: pgmap v5465: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 62 KiB/s wr, 23 op/s 2023-09-10T02:05:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:10 smithi170 ceph-mon[102832]: pgmap v5465: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 62 KiB/s wr, 23 op/s 2023-09-10T02:05:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:10 smithi082 ceph-mon[98694]: pgmap v5465: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 62 KiB/s wr, 23 op/s 2023-09-10T02:05:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:12 smithi098 ceph-mon[103124]: pgmap v5466: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 63 KiB/s wr, 29 op/s 2023-09-10T02:05:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:12 smithi170 ceph-mon[102832]: pgmap v5466: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 63 KiB/s wr, 29 op/s 2023-09-10T02:05:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:12 smithi082 ceph-mon[98694]: pgmap v5466: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 63 KiB/s wr, 29 op/s 2023-09-10T02:05:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9943.740197 secs 2023-09-10T02:05:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9945.166089 secs 2023-09-10T02:05:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9943.740197 secs 2023-09-10T02:05:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9945.166089 secs 2023-09-10T02:05:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9943.740197 secs 2023-09-10T02:05:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9945.166089 secs 2023-09-10T02:05:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:14 smithi098 ceph-mon[103124]: pgmap v5467: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 23 op/s 2023-09-10T02:05:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9945.877668 secs 2023-09-10T02:05:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:14 smithi170 ceph-mon[102832]: pgmap v5467: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 23 op/s 2023-09-10T02:05:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9945.877668 secs 2023-09-10T02:05:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:14 smithi082 ceph-mon[98694]: pgmap v5467: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 23 op/s 2023-09-10T02:05:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9945.877668 secs 2023-09-10T02:05:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:16 smithi098 ceph-mon[103124]: pgmap v5468: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 23 op/s 2023-09-10T02:05:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:16 smithi170 ceph-mon[102832]: pgmap v5468: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 23 op/s 2023-09-10T02:05:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:16 smithi082 ceph-mon[98694]: pgmap v5468: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 23 op/s 2023-09-10T02:05:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9948.740308 secs 2023-09-10T02:05:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:18 smithi098 ceph-mon[103124]: pgmap v5469: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 49 KiB/s wr, 28 op/s 2023-09-10T02:05:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9950.166147 secs 2023-09-10T02:05:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9948.740308 secs 2023-09-10T02:05:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:18 smithi170 ceph-mon[102832]: pgmap v5469: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 49 KiB/s wr, 28 op/s 2023-09-10T02:05:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9950.166147 secs 2023-09-10T02:05:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9948.740308 secs 2023-09-10T02:05:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:18 smithi082 ceph-mon[98694]: pgmap v5469: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 49 KiB/s wr, 28 op/s 2023-09-10T02:05:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9950.166147 secs 2023-09-10T02:05:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9950.877734 secs 2023-09-10T02:05:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9950.877734 secs 2023-09-10T02:05:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9950.877734 secs 2023-09-10T02:05:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:20 smithi170 ceph-mon[102832]: pgmap v5470: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 10 op/s 2023-09-10T02:05:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:20 smithi082 ceph-mon[98694]: pgmap v5470: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 10 op/s 2023-09-10T02:05:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:20 smithi098 ceph-mon[103124]: pgmap v5470: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 10 op/s 2023-09-10T02:05:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:22 smithi170 ceph-mon[102832]: pgmap v5471: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 72 KiB/s wr, 14 op/s 2023-09-10T02:05:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:22 smithi082 ceph-mon[98694]: pgmap v5471: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 72 KiB/s wr, 14 op/s 2023-09-10T02:05:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:22 smithi098 ceph-mon[103124]: pgmap v5471: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 72 KiB/s wr, 14 op/s 2023-09-10T02:05:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9953.740369 secs 2023-09-10T02:05:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9955.166205 secs 2023-09-10T02:05:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9953.740369 secs 2023-09-10T02:05:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9955.166205 secs 2023-09-10T02:05:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9953.740369 secs 2023-09-10T02:05:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9955.166205 secs 2023-09-10T02:05:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:24 smithi170 ceph-mon[102832]: pgmap v5472: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T02:05:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9955.877797 secs 2023-09-10T02:05:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:24 smithi082 ceph-mon[98694]: pgmap v5472: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T02:05:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9955.877797 secs 2023-09-10T02:05:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:24 smithi098 ceph-mon[103124]: pgmap v5472: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T02:05:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9955.877797 secs 2023-09-10T02:05:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:26 smithi170 ceph-mon[102832]: pgmap v5473: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T02:05:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:26 smithi082 ceph-mon[98694]: pgmap v5473: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T02:05:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:26 smithi098 ceph-mon[103124]: pgmap v5473: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T02:05:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:28 smithi170 ceph-mon[102832]: pgmap v5474: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 90 KiB/s wr, 15 op/s 2023-09-10T02:05:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9960.166263 secs 2023-09-10T02:05:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:28 smithi082 ceph-mon[98694]: pgmap v5474: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 90 KiB/s wr, 15 op/s 2023-09-10T02:05:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9960.166263 secs 2023-09-10T02:05:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:28 smithi098 ceph-mon[103124]: pgmap v5474: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 90 KiB/s wr, 15 op/s 2023-09-10T02:05:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9960.166263 secs 2023-09-10T02:05:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9960.877864 secs 2023-09-10T02:05:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9960.877864 secs 2023-09-10T02:05:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9960.877864 secs 2023-09-10T02:05:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:30 smithi170 ceph-mon[102832]: pgmap v5475: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 66 KiB/s wr, 10 op/s 2023-09-10T02:05:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:30 smithi082 ceph-mon[98694]: pgmap v5475: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 66 KiB/s wr, 10 op/s 2023-09-10T02:05:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:30 smithi098 ceph-mon[103124]: pgmap v5475: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 66 KiB/s wr, 10 op/s 2023-09-10T02:05:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:05:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:05:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:05:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:32 smithi170 ceph-mon[102832]: pgmap v5476: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 111 KiB/s wr, 17 op/s 2023-09-10T02:05:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:05:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:05:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:05:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:32 smithi082 ceph-mon[98694]: pgmap v5476: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 111 KiB/s wr, 17 op/s 2023-09-10T02:05:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:05:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:05:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:05:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:32 smithi098 ceph-mon[103124]: pgmap v5476: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 111 KiB/s wr, 17 op/s 2023-09-10T02:05:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:05:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:05:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:05:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9965.166333 secs 2023-09-10T02:05:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9965.166333 secs 2023-09-10T02:05:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9965.166333 secs 2023-09-10T02:05:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:34 smithi170 ceph-mon[102832]: pgmap v5477: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T02:05:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9965.877927 secs 2023-09-10T02:05:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:34 smithi082 ceph-mon[98694]: pgmap v5477: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T02:05:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9965.877927 secs 2023-09-10T02:05:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:34 smithi098 ceph-mon[103124]: pgmap v5477: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T02:05:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9965.877927 secs 2023-09-10T02:05:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:36 smithi170 ceph-mon[102832]: pgmap v5478: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T02:05:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:36 smithi082 ceph-mon[98694]: pgmap v5478: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T02:05:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:36 smithi098 ceph-mon[103124]: pgmap v5478: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T02:05:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:38 smithi170 ceph-mon[102832]: pgmap v5479: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 115 KiB/s wr, 18 op/s 2023-09-10T02:05:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9970.166392 secs 2023-09-10T02:05:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:38 smithi082 ceph-mon[98694]: pgmap v5479: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 115 KiB/s wr, 18 op/s 2023-09-10T02:05:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9970.166392 secs 2023-09-10T02:05:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:38 smithi098 ceph-mon[103124]: pgmap v5479: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 115 KiB/s wr, 18 op/s 2023-09-10T02:05:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9970.166392 secs 2023-09-10T02:05:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9970.877995 secs 2023-09-10T02:05:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9970.877995 secs 2023-09-10T02:05:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9970.877995 secs 2023-09-10T02:05:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:40 smithi170 ceph-mon[102832]: pgmap v5480: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 96 KiB/s wr, 13 op/s 2023-09-10T02:05:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:40 smithi082 ceph-mon[98694]: pgmap v5480: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 96 KiB/s wr, 13 op/s 2023-09-10T02:05:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:40 smithi098 ceph-mon[103124]: pgmap v5480: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 96 KiB/s wr, 13 op/s 2023-09-10T02:05:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:42 smithi170 ceph-mon[102832]: pgmap v5481: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 145 KiB/s wr, 19 op/s 2023-09-10T02:05:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:42 smithi082 ceph-mon[98694]: pgmap v5481: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 145 KiB/s wr, 19 op/s 2023-09-10T02:05:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:42 smithi098 ceph-mon[103124]: pgmap v5481: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 145 KiB/s wr, 19 op/s 2023-09-10T02:05:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9975.166473 secs 2023-09-10T02:05:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9975.166473 secs 2023-09-10T02:05:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9975.166473 secs 2023-09-10T02:05:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:44 smithi170 ceph-mon[102832]: pgmap v5482: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T02:05:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9975.878058 secs 2023-09-10T02:05:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:44 smithi082 ceph-mon[98694]: pgmap v5482: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T02:05:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9975.878058 secs 2023-09-10T02:05:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:44 smithi098 ceph-mon[103124]: pgmap v5482: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T02:05:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9975.878058 secs 2023-09-10T02:05:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:46 smithi170 ceph-mon[102832]: pgmap v5483: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T02:05:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:46 smithi082 ceph-mon[98694]: pgmap v5483: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T02:05:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:46 smithi098 ceph-mon[103124]: pgmap v5483: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T02:05:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9978.740807 secs 2023-09-10T02:05:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:48 smithi170 ceph-mon[102832]: pgmap v5484: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 18 op/s 2023-09-10T02:05:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9980.166492 secs 2023-09-10T02:05:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9978.740807 secs 2023-09-10T02:05:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:48 smithi082 ceph-mon[98694]: pgmap v5484: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 18 op/s 2023-09-10T02:05:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9980.166492 secs 2023-09-10T02:05:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9978.740807 secs 2023-09-10T02:05:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:48 smithi098 ceph-mon[103124]: pgmap v5484: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 18 op/s 2023-09-10T02:05:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9980.166492 secs 2023-09-10T02:05:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9980.878127 secs 2023-09-10T02:05:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9980.878127 secs 2023-09-10T02:05:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9980.878127 secs 2023-09-10T02:05:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:50 smithi170 ceph-mon[102832]: pgmap v5485: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T02:05:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:50 smithi082 ceph-mon[98694]: pgmap v5485: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T02:05:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:50 smithi098 ceph-mon[103124]: pgmap v5485: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T02:05:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:52 smithi098 ceph-mon[103124]: pgmap v5486: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 154 KiB/s wr, 17 op/s 2023-09-10T02:05:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:05:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:05:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:05:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:52 smithi170 ceph-mon[102832]: pgmap v5486: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 154 KiB/s wr, 17 op/s 2023-09-10T02:05:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:05:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:05:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:05:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:52 smithi082 ceph-mon[98694]: pgmap v5486: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 154 KiB/s wr, 17 op/s 2023-09-10T02:05:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:05:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:05:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:05:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9983.740907 secs 2023-09-10T02:05:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9985.166552 secs 2023-09-10T02:05:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9983.740907 secs 2023-09-10T02:05:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9985.166552 secs 2023-09-10T02:05:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9983.740907 secs 2023-09-10T02:05:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9985.166552 secs 2023-09-10T02:05:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:54 smithi098 ceph-mon[103124]: pgmap v5487: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T02:05:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9985.878190 secs 2023-09-10T02:05:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:54 smithi170 ceph-mon[102832]: pgmap v5487: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T02:05:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9985.878190 secs 2023-09-10T02:05:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:54 smithi082 ceph-mon[98694]: pgmap v5487: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T02:05:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9985.878190 secs 2023-09-10T02:05:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:56 smithi098 ceph-mon[103124]: pgmap v5488: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T02:05:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:56 smithi170 ceph-mon[102832]: pgmap v5488: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T02:05:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:56 smithi082 ceph-mon[98694]: pgmap v5488: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T02:05:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9988.740967 secs 2023-09-10T02:05:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:58 smithi098 ceph-mon[103124]: pgmap v5489: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 16 op/s 2023-09-10T02:05:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9990.166635 secs 2023-09-10T02:05:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9988.740967 secs 2023-09-10T02:05:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:58 smithi170 ceph-mon[102832]: pgmap v5489: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 16 op/s 2023-09-10T02:05:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9990.166635 secs 2023-09-10T02:05:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9988.740967 secs 2023-09-10T02:05:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:58 smithi082 ceph-mon[98694]: pgmap v5489: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 16 op/s 2023-09-10T02:05:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9990.166635 secs 2023-09-10T02:05:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:05:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9990.878256 secs 2023-09-10T02:05:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:05:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9990.878256 secs 2023-09-10T02:05:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:05:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9990.878256 secs 2023-09-10T02:06:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:00 smithi170 ceph-mon[102832]: pgmap v5490: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-09-10T02:06:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:00 smithi082 ceph-mon[98694]: pgmap v5490: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-09-10T02:06:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:00 smithi098 ceph-mon[103124]: pgmap v5490: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-09-10T02:06:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:02 smithi098 ceph-mon[103124]: pgmap v5491: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 99 KiB/s wr, 15 op/s 2023-09-10T02:06:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:02 smithi170 ceph-mon[102832]: pgmap v5491: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 99 KiB/s wr, 15 op/s 2023-09-10T02:06:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:02 smithi082 ceph-mon[98694]: pgmap v5491: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 99 KiB/s wr, 15 op/s 2023-09-10T02:06:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9993.741030 secs 2023-09-10T02:06:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9995.166694 secs 2023-09-10T02:06:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9993.741030 secs 2023-09-10T02:06:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9995.166694 secs 2023-09-10T02:06:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9993.741030 secs 2023-09-10T02:06:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9995.166694 secs 2023-09-10T02:06:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:04 smithi098 ceph-mon[103124]: pgmap v5492: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T02:06:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 9995.878322 secs 2023-09-10T02:06:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:04 smithi170 ceph-mon[102832]: pgmap v5492: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T02:06:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 9995.878322 secs 2023-09-10T02:06:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:04 smithi082 ceph-mon[98694]: pgmap v5492: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T02:06:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 9995.878322 secs 2023-09-10T02:06:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:06 smithi098 ceph-mon[103124]: pgmap v5493: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T02:06:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:06 smithi170 ceph-mon[102832]: pgmap v5493: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T02:06:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:06 smithi082 ceph-mon[98694]: pgmap v5493: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 10 op/s 2023-09-10T02:06:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:06:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:06:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:06:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 9998.741092 secs 2023-09-10T02:06:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:08 smithi170 ceph-mon[102832]: pgmap v5494: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 95 KiB/s wr, 28 op/s 2023-09-10T02:06:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:08 smithi170 ceph-mon[102832]: osdmap e411: 12 total, 12 up, 12 in 2023-09-10T02:06:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:06:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:06:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:06:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10000.166767 secs 2023-09-10T02:06:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 9998.741092 secs 2023-09-10T02:06:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:08 smithi082 ceph-mon[98694]: pgmap v5494: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 95 KiB/s wr, 28 op/s 2023-09-10T02:06:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:08 smithi082 ceph-mon[98694]: osdmap e411: 12 total, 12 up, 12 in 2023-09-10T02:06:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:06:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:06:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:06:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10000.166767 secs 2023-09-10T02:06:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 9998.741092 secs 2023-09-10T02:06:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:08 smithi098 ceph-mon[103124]: pgmap v5494: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 95 KiB/s wr, 28 op/s 2023-09-10T02:06:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:08 smithi098 ceph-mon[103124]: osdmap e411: 12 total, 12 up, 12 in 2023-09-10T02:06:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:06:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:06:08.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:06:08.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10000.166767 secs 2023-09-10T02:06:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10000.878404 secs 2023-09-10T02:06:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10000.878404 secs 2023-09-10T02:06:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10000.878404 secs 2023-09-10T02:06:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:10 smithi170 ceph-mon[102832]: pgmap v5496: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 74 KiB/s wr, 29 op/s 2023-09-10T02:06:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:10 smithi082 ceph-mon[98694]: pgmap v5496: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 74 KiB/s wr, 29 op/s 2023-09-10T02:06:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:10 smithi098 ceph-mon[103124]: pgmap v5496: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 74 KiB/s wr, 29 op/s 2023-09-10T02:06:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:12 smithi098 ceph-mon[103124]: pgmap v5497: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T02:06:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:12 smithi170 ceph-mon[102832]: pgmap v5497: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T02:06:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:12 smithi082 ceph-mon[98694]: pgmap v5497: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T02:06:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10003.741153 secs 2023-09-10T02:06:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10005.166852 secs 2023-09-10T02:06:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10003.741153 secs 2023-09-10T02:06:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10005.166852 secs 2023-09-10T02:06:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10003.741153 secs 2023-09-10T02:06:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10005.166852 secs 2023-09-10T02:06:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:14 smithi098 ceph-mon[103124]: pgmap v5498: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T02:06:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10005.878472 secs 2023-09-10T02:06:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:14 smithi170 ceph-mon[102832]: pgmap v5498: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T02:06:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10005.878472 secs 2023-09-10T02:06:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:14 smithi082 ceph-mon[98694]: pgmap v5498: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T02:06:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10005.878472 secs 2023-09-10T02:06:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:16 smithi098 ceph-mon[103124]: pgmap v5499: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T02:06:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:16 smithi098 ceph-mon[103124]: osdmap e412: 12 total, 12 up, 12 in 2023-09-10T02:06:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:16 smithi170 ceph-mon[102832]: pgmap v5499: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T02:06:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:16 smithi170 ceph-mon[102832]: osdmap e412: 12 total, 12 up, 12 in 2023-09-10T02:06:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:16 smithi082 ceph-mon[98694]: pgmap v5499: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T02:06:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:16 smithi082 ceph-mon[98694]: osdmap e412: 12 total, 12 up, 12 in 2023-09-10T02:06:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10008.741211 secs 2023-09-10T02:06:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:18 smithi170 ceph-mon[102832]: pgmap v5501: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T02:06:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10010.166910 secs 2023-09-10T02:06:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10008.741211 secs 2023-09-10T02:06:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:18 smithi082 ceph-mon[98694]: pgmap v5501: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T02:06:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10010.166910 secs 2023-09-10T02:06:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10008.741211 secs 2023-09-10T02:06:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:18 smithi098 ceph-mon[103124]: pgmap v5501: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T02:06:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10010.166910 secs 2023-09-10T02:06:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10010.878488 secs 2023-09-10T02:06:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10010.878488 secs 2023-09-10T02:06:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10010.878488 secs 2023-09-10T02:06:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:20 smithi170 ceph-mon[102832]: pgmap v5502: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T02:06:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:20 smithi082 ceph-mon[98694]: pgmap v5502: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T02:06:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:20 smithi098 ceph-mon[103124]: pgmap v5502: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T02:06:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:22 smithi098 ceph-mon[103124]: pgmap v5503: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9.9 KiB/s wr, 11 op/s 2023-09-10T02:06:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:22 smithi170 ceph-mon[102832]: pgmap v5503: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9.9 KiB/s wr, 11 op/s 2023-09-10T02:06:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:22 smithi082 ceph-mon[98694]: pgmap v5503: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9.9 KiB/s wr, 11 op/s 2023-09-10T02:06:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10013.741270 secs 2023-09-10T02:06:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10015.166967 secs 2023-09-10T02:06:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10013.741270 secs 2023-09-10T02:06:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10015.166967 secs 2023-09-10T02:06:23.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10013.741270 secs 2023-09-10T02:06:23.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10015.166967 secs 2023-09-10T02:06:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:24 smithi098 ceph-mon[103124]: pgmap v5504: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9.9 KiB/s wr, 11 op/s 2023-09-10T02:06:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10015.878552 secs 2023-09-10T02:06:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:24 smithi170 ceph-mon[102832]: pgmap v5504: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9.9 KiB/s wr, 11 op/s 2023-09-10T02:06:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10015.878552 secs 2023-09-10T02:06:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:24 smithi082 ceph-mon[98694]: pgmap v5504: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9.9 KiB/s wr, 11 op/s 2023-09-10T02:06:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10015.878552 secs 2023-09-10T02:06:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:26 smithi170 ceph-mon[102832]: pgmap v5505: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T02:06:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:26 smithi082 ceph-mon[98694]: pgmap v5505: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T02:06:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:26 smithi098 ceph-mon[103124]: pgmap v5505: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T02:06:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10018.741408 secs 2023-09-10T02:06:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:28 smithi170 ceph-mon[102832]: pgmap v5506: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T02:06:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10020.167025 secs 2023-09-10T02:06:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10018.741408 secs 2023-09-10T02:06:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:28 smithi082 ceph-mon[98694]: pgmap v5506: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T02:06:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10020.167025 secs 2023-09-10T02:06:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10018.741408 secs 2023-09-10T02:06:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:28 smithi098 ceph-mon[103124]: pgmap v5506: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T02:06:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10020.167025 secs 2023-09-10T02:06:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10020.878616 secs 2023-09-10T02:06:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10020.878616 secs 2023-09-10T02:06:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10020.878616 secs 2023-09-10T02:06:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:30 smithi170 ceph-mon[102832]: pgmap v5507: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T02:06:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:30 smithi082 ceph-mon[98694]: pgmap v5507: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T02:06:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:30 smithi098 ceph-mon[103124]: pgmap v5507: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T02:06:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:32 smithi098 ceph-mon[103124]: pgmap v5508: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 162 KiB/s wr, 17 op/s 2023-09-10T02:06:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:32 smithi170 ceph-mon[102832]: pgmap v5508: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 162 KiB/s wr, 17 op/s 2023-09-10T02:06:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:32 smithi082 ceph-mon[98694]: pgmap v5508: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 162 KiB/s wr, 17 op/s 2023-09-10T02:06:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10025.167082 secs 2023-09-10T02:06:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10025.167082 secs 2023-09-10T02:06:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10025.167082 secs 2023-09-10T02:06:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:34 smithi170 ceph-mon[102832]: pgmap v5509: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 154 KiB/s wr, 13 op/s 2023-09-10T02:06:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10025.878680 secs 2023-09-10T02:06:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:34 smithi082 ceph-mon[98694]: pgmap v5509: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 154 KiB/s wr, 13 op/s 2023-09-10T02:06:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10025.878680 secs 2023-09-10T02:06:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:34 smithi098 ceph-mon[103124]: pgmap v5509: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 154 KiB/s wr, 13 op/s 2023-09-10T02:06:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10025.878680 secs 2023-09-10T02:06:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:06:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:06:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:06:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:36 smithi170 ceph-mon[102832]: pgmap v5510: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 154 KiB/s wr, 15 op/s 2023-09-10T02:06:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:06:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:06:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:06:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:36 smithi082 ceph-mon[98694]: pgmap v5510: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 154 KiB/s wr, 15 op/s 2023-09-10T02:06:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:06:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:06:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:06:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:36 smithi098 ceph-mon[103124]: pgmap v5510: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 154 KiB/s wr, 15 op/s 2023-09-10T02:06:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:06:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:06:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:06:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:38 smithi170 ceph-mon[102832]: pgmap v5511: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 164 KiB/s wr, 18 op/s 2023-09-10T02:06:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10030.167139 secs 2023-09-10T02:06:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:38 smithi082 ceph-mon[98694]: pgmap v5511: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 164 KiB/s wr, 18 op/s 2023-09-10T02:06:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10030.167139 secs 2023-09-10T02:06:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:38 smithi098 ceph-mon[103124]: pgmap v5511: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 164 KiB/s wr, 18 op/s 2023-09-10T02:06:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10030.167139 secs 2023-09-10T02:06:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10030.878749 secs 2023-09-10T02:06:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10030.878749 secs 2023-09-10T02:06:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10030.878749 secs 2023-09-10T02:06:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:40 smithi170 ceph-mon[102832]: pgmap v5512: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T02:06:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:40 smithi082 ceph-mon[98694]: pgmap v5512: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T02:06:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:40 smithi098 ceph-mon[103124]: pgmap v5512: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T02:06:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:42 smithi170 ceph-mon[102832]: pgmap v5513: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 273 KiB/s wr, 19 op/s 2023-09-10T02:06:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:42 smithi082 ceph-mon[98694]: pgmap v5513: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 273 KiB/s wr, 19 op/s 2023-09-10T02:06:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:42 smithi098 ceph-mon[103124]: pgmap v5513: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 273 KiB/s wr, 19 op/s 2023-09-10T02:06:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10033.741613 secs 2023-09-10T02:06:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10035.167197 secs 2023-09-10T02:06:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10033.741613 secs 2023-09-10T02:06:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10035.167197 secs 2023-09-10T02:06:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10033.741613 secs 2023-09-10T02:06:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10035.167197 secs 2023-09-10T02:06:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:44 smithi170 ceph-mon[102832]: pgmap v5514: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 178 KiB/s wr, 12 op/s 2023-09-10T02:06:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:06:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:06:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:06:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10035.878810 secs 2023-09-10T02:06:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:44 smithi082 ceph-mon[98694]: pgmap v5514: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 178 KiB/s wr, 12 op/s 2023-09-10T02:06:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:06:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:06:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:06:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10035.878810 secs 2023-09-10T02:06:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:44 smithi098 ceph-mon[103124]: pgmap v5514: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 178 KiB/s wr, 12 op/s 2023-09-10T02:06:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:06:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:06:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:06:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10035.878810 secs 2023-09-10T02:06:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:46 smithi170 ceph-mon[102832]: pgmap v5515: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 178 KiB/s wr, 14 op/s 2023-09-10T02:06:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:46 smithi082 ceph-mon[98694]: pgmap v5515: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 178 KiB/s wr, 14 op/s 2023-09-10T02:06:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:46 smithi098 ceph-mon[103124]: pgmap v5515: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 178 KiB/s wr, 14 op/s 2023-09-10T02:06:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10038.741709 secs 2023-09-10T02:06:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:48 smithi170 ceph-mon[102832]: pgmap v5516: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 201 KiB/s wr, 15 op/s 2023-09-10T02:06:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10040.167282 secs 2023-09-10T02:06:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10038.741709 secs 2023-09-10T02:06:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:48 smithi082 ceph-mon[98694]: pgmap v5516: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 201 KiB/s wr, 15 op/s 2023-09-10T02:06:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10040.167282 secs 2023-09-10T02:06:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10038.741709 secs 2023-09-10T02:06:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:48 smithi098 ceph-mon[103124]: pgmap v5516: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 201 KiB/s wr, 15 op/s 2023-09-10T02:06:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10040.167282 secs 2023-09-10T02:06:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10040.878943 secs 2023-09-10T02:06:49.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10040.878943 secs 2023-09-10T02:06:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10040.878943 secs 2023-09-10T02:06:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:50 smithi170 ceph-mon[102832]: pgmap v5517: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T02:06:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:50 smithi082 ceph-mon[98694]: pgmap v5517: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T02:06:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:50 smithi098 ceph-mon[103124]: pgmap v5517: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T02:06:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:52 smithi170 ceph-mon[102832]: pgmap v5518: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 175 KiB/s wr, 16 op/s 2023-09-10T02:06:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:52 smithi082 ceph-mon[98694]: pgmap v5518: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 175 KiB/s wr, 16 op/s 2023-09-10T02:06:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:52 smithi098 ceph-mon[103124]: pgmap v5518: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 175 KiB/s wr, 16 op/s 2023-09-10T02:06:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10043.741771 secs 2023-09-10T02:06:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10045.167336 secs 2023-09-10T02:06:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10043.741771 secs 2023-09-10T02:06:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10045.167336 secs 2023-09-10T02:06:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10043.741771 secs 2023-09-10T02:06:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10045.167336 secs 2023-09-10T02:06:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:54 smithi170 ceph-mon[102832]: pgmap v5519: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 53 KiB/s wr, 9 op/s 2023-09-10T02:06:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10045.879008 secs 2023-09-10T02:06:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:54 smithi082 ceph-mon[98694]: pgmap v5519: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 53 KiB/s wr, 9 op/s 2023-09-10T02:06:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10045.879008 secs 2023-09-10T02:06:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:54 smithi098 ceph-mon[103124]: pgmap v5519: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 53 KiB/s wr, 9 op/s 2023-09-10T02:06:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10045.879008 secs 2023-09-10T02:06:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:56 smithi170 ceph-mon[102832]: pgmap v5520: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T02:06:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:56 smithi082 ceph-mon[98694]: pgmap v5520: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T02:06:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:56 smithi098 ceph-mon[103124]: pgmap v5520: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T02:06:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10048.741830 secs 2023-09-10T02:06:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:58 smithi170 ceph-mon[102832]: pgmap v5521: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 61 KiB/s wr, 13 op/s 2023-09-10T02:06:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10050.167394 secs 2023-09-10T02:06:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10048.741830 secs 2023-09-10T02:06:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:58 smithi082 ceph-mon[98694]: pgmap v5521: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 61 KiB/s wr, 13 op/s 2023-09-10T02:06:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10050.167394 secs 2023-09-10T02:06:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10048.741830 secs 2023-09-10T02:06:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:58 smithi098 ceph-mon[103124]: pgmap v5521: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 61 KiB/s wr, 13 op/s 2023-09-10T02:06:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10050.167394 secs 2023-09-10T02:06:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:06:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10050.879080 secs 2023-09-10T02:06:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:06:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10050.879080 secs 2023-09-10T02:06:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:06:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10050.879080 secs 2023-09-10T02:07:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:00 smithi170 ceph-mon[102832]: pgmap v5522: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T02:07:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:00 smithi082 ceph-mon[98694]: pgmap v5522: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T02:07:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:00 smithi098 ceph-mon[103124]: pgmap v5522: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T02:07:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:02 smithi170 ceph-mon[102832]: pgmap v5523: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 55 KiB/s wr, 15 op/s 2023-09-10T02:07:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:02 smithi082 ceph-mon[98694]: pgmap v5523: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 55 KiB/s wr, 15 op/s 2023-09-10T02:07:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:02 smithi098 ceph-mon[103124]: pgmap v5523: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 55 KiB/s wr, 15 op/s 2023-09-10T02:07:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10053.741890 secs 2023-09-10T02:07:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10055.167451 secs 2023-09-10T02:07:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10053.741890 secs 2023-09-10T02:07:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10055.167451 secs 2023-09-10T02:07:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10053.741890 secs 2023-09-10T02:07:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10055.167451 secs 2023-09-10T02:07:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:04 smithi170 ceph-mon[102832]: pgmap v5524: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 25 KiB/s wr, 10 op/s 2023-09-10T02:07:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10055.879139 secs 2023-09-10T02:07:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:04 smithi082 ceph-mon[98694]: pgmap v5524: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 25 KiB/s wr, 10 op/s 2023-09-10T02:07:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10055.879139 secs 2023-09-10T02:07:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:04 smithi098 ceph-mon[103124]: pgmap v5524: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 25 KiB/s wr, 10 op/s 2023-09-10T02:07:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10055.879139 secs 2023-09-10T02:07:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:06 smithi170 ceph-mon[102832]: pgmap v5525: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T02:07:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:06 smithi082 ceph-mon[98694]: pgmap v5525: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T02:07:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:06 smithi098 ceph-mon[103124]: pgmap v5525: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T02:07:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10058.741978 secs 2023-09-10T02:07:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:08 smithi170 ceph-mon[102832]: pgmap v5526: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 94 KiB/s wr, 27 op/s 2023-09-10T02:07:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:08 smithi170 ceph-mon[102832]: osdmap e413: 12 total, 12 up, 12 in 2023-09-10T02:07:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:07:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10060.167499 secs 2023-09-10T02:07:08.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:07:08.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:07:08.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:07:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10058.741978 secs 2023-09-10T02:07:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:08 smithi082 ceph-mon[98694]: pgmap v5526: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 94 KiB/s wr, 27 op/s 2023-09-10T02:07:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:08 smithi082 ceph-mon[98694]: osdmap e413: 12 total, 12 up, 12 in 2023-09-10T02:07:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:07:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10060.167499 secs 2023-09-10T02:07:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:07:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:07:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:07:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10058.741978 secs 2023-09-10T02:07:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:08 smithi098 ceph-mon[103124]: pgmap v5526: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 94 KiB/s wr, 27 op/s 2023-09-10T02:07:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:08 smithi098 ceph-mon[103124]: osdmap e413: 12 total, 12 up, 12 in 2023-09-10T02:07:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:07:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10060.167499 secs 2023-09-10T02:07:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:07:08.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:07:08.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:07:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10060.879206 secs 2023-09-10T02:07:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10060.879206 secs 2023-09-10T02:07:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10060.879206 secs 2023-09-10T02:07:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:10 smithi170 ceph-mon[102832]: pgmap v5528: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 104 KiB/s wr, 29 op/s 2023-09-10T02:07:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:10 smithi082 ceph-mon[98694]: pgmap v5528: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 104 KiB/s wr, 29 op/s 2023-09-10T02:07:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:10 smithi098 ceph-mon[103124]: pgmap v5528: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 104 KiB/s wr, 29 op/s 2023-09-10T02:07:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:12 smithi170 ceph-mon[102832]: pgmap v5529: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T02:07:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:12 smithi082 ceph-mon[98694]: pgmap v5529: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T02:07:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:12 smithi098 ceph-mon[103124]: pgmap v5529: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T02:07:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10063.742040 secs 2023-09-10T02:07:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10065.167555 secs 2023-09-10T02:07:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10063.742040 secs 2023-09-10T02:07:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10065.167555 secs 2023-09-10T02:07:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10063.742040 secs 2023-09-10T02:07:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10065.167555 secs 2023-09-10T02:07:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:14 smithi170 ceph-mon[102832]: pgmap v5530: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T02:07:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10065.879269 secs 2023-09-10T02:07:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:14 smithi082 ceph-mon[98694]: pgmap v5530: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T02:07:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10065.879269 secs 2023-09-10T02:07:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:14 smithi098 ceph-mon[103124]: pgmap v5530: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T02:07:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10065.879269 secs 2023-09-10T02:07:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:16 smithi170 ceph-mon[102832]: pgmap v5531: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T02:07:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:16 smithi170 ceph-mon[102832]: osdmap e414: 12 total, 12 up, 12 in 2023-09-10T02:07:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:16 smithi082 ceph-mon[98694]: pgmap v5531: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T02:07:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:16 smithi082 ceph-mon[98694]: osdmap e414: 12 total, 12 up, 12 in 2023-09-10T02:07:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:16 smithi098 ceph-mon[103124]: pgmap v5531: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T02:07:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:16 smithi098 ceph-mon[103124]: osdmap e414: 12 total, 12 up, 12 in 2023-09-10T02:07:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10068.742100 secs 2023-09-10T02:07:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:18 smithi170 ceph-mon[102832]: pgmap v5533: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2023-09-10T02:07:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10070.167643 secs 2023-09-10T02:07:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10068.742100 secs 2023-09-10T02:07:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:18 smithi082 ceph-mon[98694]: pgmap v5533: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2023-09-10T02:07:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10070.167643 secs 2023-09-10T02:07:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10068.742100 secs 2023-09-10T02:07:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:18 smithi098 ceph-mon[103124]: pgmap v5533: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2023-09-10T02:07:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10070.167643 secs 2023-09-10T02:07:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10070.879335 secs 2023-09-10T02:07:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10070.879335 secs 2023-09-10T02:07:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10070.879335 secs 2023-09-10T02:07:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:20 smithi170 ceph-mon[102832]: pgmap v5534: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2023-09-10T02:07:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:20 smithi082 ceph-mon[98694]: pgmap v5534: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2023-09-10T02:07:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:20 smithi098 ceph-mon[103124]: pgmap v5534: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2023-09-10T02:07:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:22 smithi170 ceph-mon[102832]: pgmap v5535: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T02:07:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:22 smithi082 ceph-mon[98694]: pgmap v5535: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T02:07:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:22 smithi098 ceph-mon[103124]: pgmap v5535: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T02:07:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:07:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10075.167700 secs 2023-09-10T02:07:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:07:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10075.167700 secs 2023-09-10T02:07:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:07:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10075.167700 secs 2023-09-10T02:07:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:24 smithi170 ceph-mon[102832]: pgmap v5536: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T02:07:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:07:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:07:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:07:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10075.879423 secs 2023-09-10T02:07:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:24 smithi082 ceph-mon[98694]: pgmap v5536: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T02:07:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:07:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:07:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:07:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10075.879423 secs 2023-09-10T02:07:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:24 smithi098 ceph-mon[103124]: pgmap v5536: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T02:07:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:07:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:07:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:07:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10075.879423 secs 2023-09-10T02:07:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:26 smithi170 ceph-mon[102832]: pgmap v5537: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T02:07:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:26 smithi082 ceph-mon[98694]: pgmap v5537: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T02:07:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:26 smithi098 ceph-mon[103124]: pgmap v5537: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T02:07:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10078.742246 secs 2023-09-10T02:07:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:28 smithi170 ceph-mon[102832]: pgmap v5538: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T02:07:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10080.167758 secs 2023-09-10T02:07:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10078.742246 secs 2023-09-10T02:07:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:28 smithi082 ceph-mon[98694]: pgmap v5538: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T02:07:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10080.167758 secs 2023-09-10T02:07:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10078.742246 secs 2023-09-10T02:07:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:28 smithi098 ceph-mon[103124]: pgmap v5538: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T02:07:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10080.167758 secs 2023-09-10T02:07:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10080.879483 secs 2023-09-10T02:07:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10080.879483 secs 2023-09-10T02:07:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10080.879483 secs 2023-09-10T02:07:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:30 smithi170 ceph-mon[102832]: pgmap v5539: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T02:07:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:30 smithi082 ceph-mon[98694]: pgmap v5539: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T02:07:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:30 smithi098 ceph-mon[103124]: pgmap v5539: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 44 KiB/s wr, 10 op/s 2023-09-10T02:07:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:32 smithi170 ceph-mon[102832]: pgmap v5540: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 94 KiB/s wr, 16 op/s 2023-09-10T02:07:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:32 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:07:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:07:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:07:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:32 smithi082 ceph-mon[98694]: pgmap v5540: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 94 KiB/s wr, 16 op/s 2023-09-10T02:07:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:32 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:07:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:07:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:07:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:32 smithi098 ceph-mon[103124]: pgmap v5540: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 94 KiB/s wr, 16 op/s 2023-09-10T02:07:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:32 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:07:32.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:07:32.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:07:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10085.167817 secs 2023-09-10T02:07:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10085.167817 secs 2023-09-10T02:07:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10085.167817 secs 2023-09-10T02:07:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:34 smithi170 ceph-mon[102832]: pgmap v5541: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 10 op/s 2023-09-10T02:07:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10085.879549 secs 2023-09-10T02:07:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:34 smithi082 ceph-mon[98694]: pgmap v5541: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 10 op/s 2023-09-10T02:07:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10085.879549 secs 2023-09-10T02:07:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:34 smithi098 ceph-mon[103124]: pgmap v5541: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 10 op/s 2023-09-10T02:07:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10085.879549 secs 2023-09-10T02:07:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:07:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:07:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:07:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:36 smithi170 ceph-mon[102832]: pgmap v5542: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 13 op/s 2023-09-10T02:07:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:07:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:07:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:36 smithi082 ceph-mon[98694]: pgmap v5542: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 13 op/s 2023-09-10T02:07:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:07:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:07:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:36 smithi098 ceph-mon[103124]: pgmap v5542: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 13 op/s 2023-09-10T02:07:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:07:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:07:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:38 smithi170 ceph-mon[102832]: pgmap v5543: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 15 op/s 2023-09-10T02:07:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10090.167875 secs 2023-09-10T02:07:38.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:38 smithi082 ceph-mon[98694]: pgmap v5543: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 15 op/s 2023-09-10T02:07:38.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10090.167875 secs 2023-09-10T02:07:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:38 smithi098 ceph-mon[103124]: pgmap v5543: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 15 op/s 2023-09-10T02:07:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10090.167875 secs 2023-09-10T02:07:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10090.879616 secs 2023-09-10T02:07:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10090.879616 secs 2023-09-10T02:07:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10090.879616 secs 2023-09-10T02:07:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:40 smithi170 ceph-mon[102832]: pgmap v5544: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T02:07:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:40 smithi082 ceph-mon[98694]: pgmap v5544: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T02:07:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:40 smithi098 ceph-mon[103124]: pgmap v5544: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T02:07:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:41 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:07:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:41 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:07:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:41 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:07:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:42 smithi170 ceph-mon[102832]: pgmap v5545: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 157 KiB/s wr, 18 op/s 2023-09-10T02:07:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:42 smithi082 ceph-mon[98694]: pgmap v5545: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 157 KiB/s wr, 18 op/s 2023-09-10T02:07:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:42 smithi098 ceph-mon[103124]: pgmap v5545: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 157 KiB/s wr, 18 op/s 2023-09-10T02:07:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10095.167945 secs 2023-09-10T02:07:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10095.167945 secs 2023-09-10T02:07:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10095.167945 secs 2023-09-10T02:07:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:44 smithi170 ceph-mon[102832]: pgmap v5546: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 107 KiB/s wr, 12 op/s 2023-09-10T02:07:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10095.879681 secs 2023-09-10T02:07:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:44 smithi082 ceph-mon[98694]: pgmap v5546: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 107 KiB/s wr, 12 op/s 2023-09-10T02:07:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10095.879681 secs 2023-09-10T02:07:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:44 smithi098 ceph-mon[103124]: pgmap v5546: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 107 KiB/s wr, 12 op/s 2023-09-10T02:07:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10095.879681 secs 2023-09-10T02:07:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:46 smithi170 ceph-mon[102832]: pgmap v5547: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 114 KiB/s wr, 14 op/s 2023-09-10T02:07:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:46 smithi082 ceph-mon[98694]: pgmap v5547: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 114 KiB/s wr, 14 op/s 2023-09-10T02:07:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:46 smithi098 ceph-mon[103124]: pgmap v5547: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 114 KiB/s wr, 14 op/s 2023-09-10T02:07:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:48 smithi170 ceph-mon[102832]: pgmap v5548: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 96 KiB/s wr, 14 op/s 2023-09-10T02:07:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10100.168001 secs 2023-09-10T02:07:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:48 smithi082 ceph-mon[98694]: pgmap v5548: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 96 KiB/s wr, 14 op/s 2023-09-10T02:07:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10100.168001 secs 2023-09-10T02:07:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:48 smithi098 ceph-mon[103124]: pgmap v5548: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 96 KiB/s wr, 14 op/s 2023-09-10T02:07:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10100.168001 secs 2023-09-10T02:07:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10100.879745 secs 2023-09-10T02:07:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10100.879745 secs 2023-09-10T02:07:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10100.879745 secs 2023-09-10T02:07:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:50 smithi170 ceph-mon[102832]: pgmap v5549: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T02:07:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:50 smithi082 ceph-mon[98694]: pgmap v5549: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T02:07:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:50 smithi098 ceph-mon[103124]: pgmap v5549: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T02:07:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:52 smithi170 ceph-mon[102832]: pgmap v5550: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 187 KiB/s wr, 17 op/s 2023-09-10T02:07:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:52 smithi082 ceph-mon[98694]: pgmap v5550: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 187 KiB/s wr, 17 op/s 2023-09-10T02:07:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:52 smithi098 ceph-mon[103124]: pgmap v5550: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 187 KiB/s wr, 17 op/s 2023-09-10T02:07:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10103.742673 secs 2023-09-10T02:07:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10105.168057 secs 2023-09-10T02:07:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10103.742673 secs 2023-09-10T02:07:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10105.168057 secs 2023-09-10T02:07:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10103.742673 secs 2023-09-10T02:07:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10105.168057 secs 2023-09-10T02:07:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:54 smithi170 ceph-mon[102832]: pgmap v5551: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T02:07:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10105.879810 secs 2023-09-10T02:07:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:54 smithi082 ceph-mon[98694]: pgmap v5551: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T02:07:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10105.879810 secs 2023-09-10T02:07:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:54 smithi098 ceph-mon[103124]: pgmap v5551: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T02:07:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10105.879810 secs 2023-09-10T02:07:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:07:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:07:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:07:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:07:55.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:07:55.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:07:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:07:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:07:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:07:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:56 smithi170 ceph-mon[102832]: pgmap v5552: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 156 KiB/s wr, 15 op/s 2023-09-10T02:07:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:56 smithi082 ceph-mon[98694]: pgmap v5552: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 156 KiB/s wr, 15 op/s 2023-09-10T02:07:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:56 smithi098 ceph-mon[103124]: pgmap v5552: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 156 KiB/s wr, 15 op/s 2023-09-10T02:07:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10108.742793 secs 2023-09-10T02:07:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:58 smithi170 ceph-mon[102832]: pgmap v5553: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 150 KiB/s wr, 15 op/s 2023-09-10T02:07:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10110.168115 secs 2023-09-10T02:07:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10108.742793 secs 2023-09-10T02:07:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:58 smithi082 ceph-mon[98694]: pgmap v5553: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 150 KiB/s wr, 15 op/s 2023-09-10T02:07:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10110.168115 secs 2023-09-10T02:07:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10108.742793 secs 2023-09-10T02:07:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:58 smithi098 ceph-mon[103124]: pgmap v5553: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 150 KiB/s wr, 15 op/s 2023-09-10T02:07:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10110.168115 secs 2023-09-10T02:07:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:07:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10110.879876 secs 2023-09-10T02:07:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:07:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10110.879876 secs 2023-09-10T02:07:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:07:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10110.879876 secs 2023-09-10T02:08:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:00 smithi170 ceph-mon[102832]: pgmap v5554: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 141 KiB/s wr, 12 op/s 2023-09-10T02:08:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:00 smithi082 ceph-mon[98694]: pgmap v5554: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 141 KiB/s wr, 12 op/s 2023-09-10T02:08:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:00 smithi098 ceph-mon[103124]: pgmap v5554: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 141 KiB/s wr, 12 op/s 2023-09-10T02:08:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:02 smithi170 ceph-mon[102832]: pgmap v5555: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 238 KiB/s wr, 32 op/s 2023-09-10T02:08:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:02 smithi082 ceph-mon[98694]: pgmap v5555: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 238 KiB/s wr, 32 op/s 2023-09-10T02:08:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:02 smithi098 ceph-mon[103124]: pgmap v5555: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 238 KiB/s wr, 32 op/s 2023-09-10T02:08:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10113.742850 secs 2023-09-10T02:08:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10115.168172 secs 2023-09-10T02:08:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10113.742850 secs 2023-09-10T02:08:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10115.168172 secs 2023-09-10T02:08:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10113.742850 secs 2023-09-10T02:08:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10115.168172 secs 2023-09-10T02:08:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:04 smithi170 ceph-mon[102832]: pgmap v5556: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 140 KiB/s wr, 25 op/s 2023-09-10T02:08:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10115.879942 secs 2023-09-10T02:08:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:04 smithi082 ceph-mon[98694]: pgmap v5556: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 140 KiB/s wr, 25 op/s 2023-09-10T02:08:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10115.879942 secs 2023-09-10T02:08:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:04 smithi098 ceph-mon[103124]: pgmap v5556: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 140 KiB/s wr, 25 op/s 2023-09-10T02:08:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10115.879942 secs 2023-09-10T02:08:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:06 smithi170 ceph-mon[102832]: pgmap v5557: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 157 KiB/s wr, 29 op/s 2023-09-10T02:08:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:06 smithi082 ceph-mon[98694]: pgmap v5557: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 157 KiB/s wr, 29 op/s 2023-09-10T02:08:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:06 smithi098 ceph-mon[103124]: pgmap v5557: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 157 KiB/s wr, 29 op/s 2023-09-10T02:08:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10118.742911 secs 2023-09-10T02:08:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:08 smithi170 ceph-mon[102832]: pgmap v5558: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 115 KiB/s wr, 27 op/s 2023-09-10T02:08:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:08 smithi170 ceph-mon[102832]: osdmap e415: 12 total, 12 up, 12 in 2023-09-10T02:08:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10120.168230 secs 2023-09-10T02:08:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:08:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10118.742911 secs 2023-09-10T02:08:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:08 smithi082 ceph-mon[98694]: pgmap v5558: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 115 KiB/s wr, 27 op/s 2023-09-10T02:08:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:08 smithi082 ceph-mon[98694]: osdmap e415: 12 total, 12 up, 12 in 2023-09-10T02:08:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10120.168230 secs 2023-09-10T02:08:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:08:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10118.742911 secs 2023-09-10T02:08:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:08 smithi098 ceph-mon[103124]: pgmap v5558: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 115 KiB/s wr, 27 op/s 2023-09-10T02:08:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:08 smithi098 ceph-mon[103124]: osdmap e415: 12 total, 12 up, 12 in 2023-09-10T02:08:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10120.168230 secs 2023-09-10T02:08:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:08:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10120.880007 secs 2023-09-10T02:08:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10120.880007 secs 2023-09-10T02:08:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10120.880007 secs 2023-09-10T02:08:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:10 smithi170 ceph-mon[102832]: pgmap v5560: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 137 KiB/s wr, 30 op/s 2023-09-10T02:08:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:10 smithi082 ceph-mon[98694]: pgmap v5560: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 137 KiB/s wr, 30 op/s 2023-09-10T02:08:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:10 smithi098 ceph-mon[103124]: pgmap v5560: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 137 KiB/s wr, 30 op/s 2023-09-10T02:08:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:08:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:08:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:08:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:08:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:08:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:08:12.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:08:12.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:08:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:08:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:08:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:08:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:08:12.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:08:12.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:08:12.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:08:12.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:08:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:08:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:08:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:08:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:08:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:08:12.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:08:12.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:08:12.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:08:12.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:08:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:12 smithi170 ceph-mon[102832]: pgmap v5561: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-10T02:08:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:12 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T02:08:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:12 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T02:08:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:12 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T02:08:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:12 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T02:08:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:12 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T02:08:13.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:12 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T02:08:13.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:08:13.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10123.742972 secs 2023-09-10T02:08:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:12 smithi082 ceph-mon[98694]: pgmap v5561: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-10T02:08:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:12 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T02:08:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:12 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T02:08:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:12 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T02:08:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:12 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T02:08:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:12 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T02:08:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:12 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T02:08:13.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:08:13.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10123.742972 secs 2023-09-10T02:08:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:12 smithi098 ceph-mon[103124]: pgmap v5561: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-10T02:08:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:12 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-09-10T02:08:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:12 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-09-10T02:08:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:12 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T02:08:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:12 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T02:08:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:12 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T02:08:13.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:12 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T02:08:13.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:08:13.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10123.742972 secs 2023-09-10T02:08:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10125.168286 secs 2023-09-10T02:08:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10125.168286 secs 2023-09-10T02:08:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10125.168286 secs 2023-09-10T02:08:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:14 smithi170 ceph-mon[102832]: pgmap v5562: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-10T02:08:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10125.880074 secs 2023-09-10T02:08:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:14 smithi082 ceph-mon[98694]: pgmap v5562: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-10T02:08:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10125.880074 secs 2023-09-10T02:08:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:14 smithi098 ceph-mon[103124]: pgmap v5562: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-09-10T02:08:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10125.880074 secs 2023-09-10T02:08:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:16 smithi170 ceph-mon[102832]: pgmap v5563: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T02:08:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:16 smithi170 ceph-mon[102832]: osdmap e416: 12 total, 12 up, 12 in 2023-09-10T02:08:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:16 smithi082 ceph-mon[98694]: pgmap v5563: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T02:08:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:16 smithi082 ceph-mon[98694]: osdmap e416: 12 total, 12 up, 12 in 2023-09-10T02:08:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:16 smithi098 ceph-mon[103124]: pgmap v5563: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T02:08:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:16 smithi098 ceph-mon[103124]: osdmap e416: 12 total, 12 up, 12 in 2023-09-10T02:08:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10128.743032 secs 2023-09-10T02:08:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10128.743032 secs 2023-09-10T02:08:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10128.743032 secs 2023-09-10T02:08:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:18 smithi082 ceph-mon[98694]: pgmap v5565: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T02:08:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10130.168344 secs 2023-09-10T02:08:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10130.880137 secs 2023-09-10T02:08:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:18 smithi098 ceph-mon[103124]: pgmap v5565: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T02:08:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10130.168344 secs 2023-09-10T02:08:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10130.880137 secs 2023-09-10T02:08:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:18 smithi170 ceph-mon[102832]: pgmap v5565: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T02:08:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10130.168344 secs 2023-09-10T02:08:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10130.880137 secs 2023-09-10T02:08:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:20 smithi082 ceph-mon[98694]: pgmap v5566: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T02:08:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:20 smithi098 ceph-mon[103124]: pgmap v5566: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T02:08:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:20 smithi170 ceph-mon[102832]: pgmap v5566: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T02:08:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:22 smithi170 ceph-mon[102832]: pgmap v5567: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T02:08:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:22 smithi082 ceph-mon[98694]: pgmap v5567: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T02:08:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:22 smithi098 ceph-mon[103124]: pgmap v5567: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T02:08:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10133.743093 secs 2023-09-10T02:08:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10135.168399 secs 2023-09-10T02:08:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10133.743093 secs 2023-09-10T02:08:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10135.168399 secs 2023-09-10T02:08:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10133.743093 secs 2023-09-10T02:08:23.767 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10135.168399 secs 2023-09-10T02:08:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:24 smithi170 ceph-mon[102832]: pgmap v5568: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T02:08:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10135.880200 secs 2023-09-10T02:08:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:24 smithi082 ceph-mon[98694]: pgmap v5568: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T02:08:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10135.880200 secs 2023-09-10T02:08:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:24 smithi098 ceph-mon[103124]: pgmap v5568: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T02:08:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10135.880200 secs 2023-09-10T02:08:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:26 smithi170 ceph-mon[102832]: pgmap v5569: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T02:08:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:26 smithi082 ceph-mon[98694]: pgmap v5569: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T02:08:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:26 smithi098 ceph-mon[103124]: pgmap v5569: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T02:08:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:28 smithi170 ceph-mon[102832]: pgmap v5570: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T02:08:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10140.168487 secs 2023-09-10T02:08:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:28 smithi082 ceph-mon[98694]: pgmap v5570: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T02:08:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10140.168487 secs 2023-09-10T02:08:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:28 smithi098 ceph-mon[103124]: pgmap v5570: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-09-10T02:08:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10140.168487 secs 2023-09-10T02:08:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10140.880261 secs 2023-09-10T02:08:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10140.880261 secs 2023-09-10T02:08:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10140.880261 secs 2023-09-10T02:08:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:30 smithi170 ceph-mon[102832]: pgmap v5571: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 KiB/s wr, 9 op/s 2023-09-10T02:08:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:30 smithi082 ceph-mon[98694]: pgmap v5571: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 KiB/s wr, 9 op/s 2023-09-10T02:08:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:30 smithi098 ceph-mon[103124]: pgmap v5571: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 KiB/s wr, 9 op/s 2023-09-10T02:08:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:08:31.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:08:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:08:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:32 smithi170 ceph-mon[102832]: pgmap v5572: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 154 KiB/s wr, 16 op/s 2023-09-10T02:08:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:08:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:08:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:08:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:32 smithi082 ceph-mon[98694]: pgmap v5572: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 154 KiB/s wr, 16 op/s 2023-09-10T02:08:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:08:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:08:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:08:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:32 smithi098 ceph-mon[103124]: pgmap v5572: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 154 KiB/s wr, 16 op/s 2023-09-10T02:08:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:08:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:08:32.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:08:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10145.168529 secs 2023-09-10T02:08:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10145.168529 secs 2023-09-10T02:08:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10145.168529 secs 2023-09-10T02:08:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:34 smithi170 ceph-mon[102832]: pgmap v5573: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T02:08:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10145.880324 secs 2023-09-10T02:08:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:34 smithi082 ceph-mon[98694]: pgmap v5573: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T02:08:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10145.880324 secs 2023-09-10T02:08:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:34 smithi098 ceph-mon[103124]: pgmap v5573: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T02:08:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10145.880324 secs 2023-09-10T02:08:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:36 smithi170 ceph-mon[102832]: pgmap v5574: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 162 KiB/s wr, 14 op/s 2023-09-10T02:08:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:36 smithi082 ceph-mon[98694]: pgmap v5574: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 162 KiB/s wr, 14 op/s 2023-09-10T02:08:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:36 smithi098 ceph-mon[103124]: pgmap v5574: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 162 KiB/s wr, 14 op/s 2023-09-10T02:08:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:38 smithi170 ceph-mon[102832]: pgmap v5575: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T02:08:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10150.168635 secs 2023-09-10T02:08:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:38 smithi082 ceph-mon[98694]: pgmap v5575: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T02:08:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10150.168635 secs 2023-09-10T02:08:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:38 smithi098 ceph-mon[103124]: pgmap v5575: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T02:08:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10150.168635 secs 2023-09-10T02:08:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10150.880389 secs 2023-09-10T02:08:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10150.880389 secs 2023-09-10T02:08:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10150.880389 secs 2023-09-10T02:08:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:40 smithi170 ceph-mon[102832]: pgmap v5576: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T02:08:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:40 smithi082 ceph-mon[98694]: pgmap v5576: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T02:08:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:40 smithi098 ceph-mon[103124]: pgmap v5576: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T02:08:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:42 smithi170 ceph-mon[102832]: pgmap v5577: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 242 KiB/s wr, 17 op/s 2023-09-10T02:08:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:42 smithi082 ceph-mon[98694]: pgmap v5577: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 242 KiB/s wr, 17 op/s 2023-09-10T02:08:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:42 smithi098 ceph-mon[103124]: pgmap v5577: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 242 KiB/s wr, 17 op/s 2023-09-10T02:08:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10155.168708 secs 2023-09-10T02:08:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10155.168708 secs 2023-09-10T02:08:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10155.168708 secs 2023-09-10T02:08:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:44 smithi170 ceph-mon[102832]: pgmap v5578: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T02:08:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10155.880452 secs 2023-09-10T02:08:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:44 smithi082 ceph-mon[98694]: pgmap v5578: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T02:08:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10155.880452 secs 2023-09-10T02:08:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:44 smithi098 ceph-mon[103124]: pgmap v5578: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T02:08:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10155.880452 secs 2023-09-10T02:08:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:46 smithi170 ceph-mon[102832]: pgmap v5579: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 178 KiB/s wr, 14 op/s 2023-09-10T02:08:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:46 smithi082 ceph-mon[98694]: pgmap v5579: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 178 KiB/s wr, 14 op/s 2023-09-10T02:08:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:46 smithi098 ceph-mon[103124]: pgmap v5579: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 178 KiB/s wr, 14 op/s 2023-09-10T02:08:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10158.743463 secs 2023-09-10T02:08:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:48 smithi170 ceph-mon[102832]: pgmap v5580: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T02:08:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10160.168766 secs 2023-09-10T02:08:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10158.743463 secs 2023-09-10T02:08:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:48 smithi082 ceph-mon[98694]: pgmap v5580: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T02:08:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10160.168766 secs 2023-09-10T02:08:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10158.743463 secs 2023-09-10T02:08:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:48 smithi098 ceph-mon[103124]: pgmap v5580: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T02:08:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10160.168766 secs 2023-09-10T02:08:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10160.880481 secs 2023-09-10T02:08:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10160.880481 secs 2023-09-10T02:08:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10160.880481 secs 2023-09-10T02:08:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:50 smithi170 ceph-mon[102832]: pgmap v5581: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 176 KiB/s wr, 12 op/s 2023-09-10T02:08:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:50 smithi082 ceph-mon[98694]: pgmap v5581: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 176 KiB/s wr, 12 op/s 2023-09-10T02:08:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:50 smithi098 ceph-mon[103124]: pgmap v5581: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 176 KiB/s wr, 12 op/s 2023-09-10T02:08:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:52 smithi170 ceph-mon[102832]: pgmap v5582: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 236 KiB/s wr, 18 op/s 2023-09-10T02:08:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:08:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:08:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:08:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:52 smithi082 ceph-mon[98694]: pgmap v5582: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 236 KiB/s wr, 18 op/s 2023-09-10T02:08:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:08:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:08:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:08:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:52 smithi098 ceph-mon[103124]: pgmap v5582: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 236 KiB/s wr, 18 op/s 2023-09-10T02:08:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:08:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:08:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:08:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10163.743524 secs 2023-09-10T02:08:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10165.168823 secs 2023-09-10T02:08:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10163.743524 secs 2023-09-10T02:08:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10165.168823 secs 2023-09-10T02:08:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10163.743524 secs 2023-09-10T02:08:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10165.168823 secs 2023-09-10T02:08:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:54 smithi170 ceph-mon[102832]: pgmap v5583: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 132 KiB/s wr, 12 op/s 2023-09-10T02:08:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10165.880545 secs 2023-09-10T02:08:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:54 smithi082 ceph-mon[98694]: pgmap v5583: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 132 KiB/s wr, 12 op/s 2023-09-10T02:08:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10165.880545 secs 2023-09-10T02:08:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:54 smithi098 ceph-mon[103124]: pgmap v5583: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 132 KiB/s wr, 12 op/s 2023-09-10T02:08:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10165.880545 secs 2023-09-10T02:08:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:56 smithi170 ceph-mon[102832]: pgmap v5584: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 132 KiB/s wr, 15 op/s 2023-09-10T02:08:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:56 smithi082 ceph-mon[98694]: pgmap v5584: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 132 KiB/s wr, 15 op/s 2023-09-10T02:08:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:56 smithi098 ceph-mon[103124]: pgmap v5584: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 132 KiB/s wr, 15 op/s 2023-09-10T02:08:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10168.743603 secs 2023-09-10T02:08:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:58 smithi170 ceph-mon[102832]: pgmap v5585: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 74 KiB/s wr, 13 op/s 2023-09-10T02:08:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10170.168880 secs 2023-09-10T02:08:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10168.743603 secs 2023-09-10T02:08:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:58 smithi082 ceph-mon[98694]: pgmap v5585: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 74 KiB/s wr, 13 op/s 2023-09-10T02:08:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10170.168880 secs 2023-09-10T02:08:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10168.743603 secs 2023-09-10T02:08:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:58 smithi098 ceph-mon[103124]: pgmap v5585: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 74 KiB/s wr, 13 op/s 2023-09-10T02:08:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10170.168880 secs 2023-09-10T02:08:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:08:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10170.880612 secs 2023-09-10T02:08:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:08:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10170.880612 secs 2023-09-10T02:08:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:08:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10170.880612 secs 2023-09-10T02:09:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:00 smithi170 ceph-mon[102832]: pgmap v5586: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 10 op/s 2023-09-10T02:09:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:00 smithi082 ceph-mon[98694]: pgmap v5586: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 10 op/s 2023-09-10T02:09:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:00 smithi098 ceph-mon[103124]: pgmap v5586: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 10 op/s 2023-09-10T02:09:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:02 smithi170 ceph-mon[102832]: pgmap v5587: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 130 KiB/s wr, 30 op/s 2023-09-10T02:09:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:02 smithi082 ceph-mon[98694]: pgmap v5587: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 130 KiB/s wr, 30 op/s 2023-09-10T02:09:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:02 smithi098 ceph-mon[103124]: pgmap v5587: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 130 KiB/s wr, 30 op/s 2023-09-10T02:09:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10173.743704 secs 2023-09-10T02:09:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10175.168936 secs 2023-09-10T02:09:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10173.743704 secs 2023-09-10T02:09:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10175.168936 secs 2023-09-10T02:09:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10173.743704 secs 2023-09-10T02:09:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10175.168936 secs 2023-09-10T02:09:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:04 smithi170 ceph-mon[102832]: pgmap v5588: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 70 KiB/s wr, 24 op/s 2023-09-10T02:09:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10175.880676 secs 2023-09-10T02:09:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:04 smithi082 ceph-mon[98694]: pgmap v5588: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 70 KiB/s wr, 24 op/s 2023-09-10T02:09:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10175.880676 secs 2023-09-10T02:09:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:04 smithi098 ceph-mon[103124]: pgmap v5588: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 70 KiB/s wr, 24 op/s 2023-09-10T02:09:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10175.880676 secs 2023-09-10T02:09:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:06 smithi170 ceph-mon[102832]: pgmap v5589: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 70 KiB/s wr, 27 op/s 2023-09-10T02:09:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:06 smithi082 ceph-mon[98694]: pgmap v5589: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 70 KiB/s wr, 27 op/s 2023-09-10T02:09:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:06 smithi098 ceph-mon[103124]: pgmap v5589: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 70 KiB/s wr, 27 op/s 2023-09-10T02:09:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10178.743803 secs 2023-09-10T02:09:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:08 smithi170 ceph-mon[102832]: pgmap v5590: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 96 KiB/s wr, 27 op/s 2023-09-10T02:09:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:08 smithi170 ceph-mon[102832]: osdmap e417: 12 total, 12 up, 12 in 2023-09-10T02:09:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10180.169009 secs 2023-09-10T02:09:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10178.743803 secs 2023-09-10T02:09:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:08 smithi082 ceph-mon[98694]: pgmap v5590: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 96 KiB/s wr, 27 op/s 2023-09-10T02:09:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:08 smithi082 ceph-mon[98694]: osdmap e417: 12 total, 12 up, 12 in 2023-09-10T02:09:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10180.169009 secs 2023-09-10T02:09:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10178.743803 secs 2023-09-10T02:09:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:08 smithi098 ceph-mon[103124]: pgmap v5590: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 96 KiB/s wr, 27 op/s 2023-09-10T02:09:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:08 smithi098 ceph-mon[103124]: osdmap e417: 12 total, 12 up, 12 in 2023-09-10T02:09:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10180.169009 secs 2023-09-10T02:09:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10180.880739 secs 2023-09-10T02:09:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10180.880739 secs 2023-09-10T02:09:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10180.880739 secs 2023-09-10T02:09:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:10 smithi170 ceph-mon[102832]: pgmap v5592: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T02:09:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:10 smithi082 ceph-mon[98694]: pgmap v5592: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T02:09:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:10 smithi098 ceph-mon[103124]: pgmap v5592: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T02:09:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:12 smithi170 ceph-mon[102832]: pgmap v5593: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 49 KiB/s wr, 12 op/s 2023-09-10T02:09:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:09:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:12 smithi082 ceph-mon[98694]: pgmap v5593: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 49 KiB/s wr, 12 op/s 2023-09-10T02:09:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:09:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:12 smithi098 ceph-mon[103124]: pgmap v5593: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 49 KiB/s wr, 12 op/s 2023-09-10T02:09:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:09:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10183.743865 secs 2023-09-10T02:09:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:09:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:09:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:09:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10185.169094 secs 2023-09-10T02:09:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10183.743865 secs 2023-09-10T02:09:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:09:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:09:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:09:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10185.169094 secs 2023-09-10T02:09:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10183.743865 secs 2023-09-10T02:09:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:09:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:09:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:09:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10185.169094 secs 2023-09-10T02:09:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:14 smithi170 ceph-mon[102832]: pgmap v5594: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 49 KiB/s wr, 12 op/s 2023-09-10T02:09:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10185.880803 secs 2023-09-10T02:09:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:14 smithi082 ceph-mon[98694]: pgmap v5594: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 49 KiB/s wr, 12 op/s 2023-09-10T02:09:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10185.880803 secs 2023-09-10T02:09:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:14 smithi098 ceph-mon[103124]: pgmap v5594: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 49 KiB/s wr, 12 op/s 2023-09-10T02:09:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10185.880803 secs 2023-09-10T02:09:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:16 smithi082 ceph-mon[98694]: pgmap v5595: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T02:09:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:16 smithi082 ceph-mon[98694]: osdmap e418: 12 total, 12 up, 12 in 2023-09-10T02:09:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:16 smithi098 ceph-mon[103124]: pgmap v5595: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T02:09:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:16 smithi098 ceph-mon[103124]: osdmap e418: 12 total, 12 up, 12 in 2023-09-10T02:09:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:16 smithi170 ceph-mon[102832]: pgmap v5595: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T02:09:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:16 smithi170 ceph-mon[102832]: osdmap e418: 12 total, 12 up, 12 in 2023-09-10T02:09:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10188.743981 secs 2023-09-10T02:09:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10188.743981 secs 2023-09-10T02:09:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10188.743981 secs 2023-09-10T02:09:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:18 smithi082 ceph-mon[98694]: pgmap v5597: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 57 KiB/s wr, 12 op/s 2023-09-10T02:09:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10190.169150 secs 2023-09-10T02:09:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:18 smithi098 ceph-mon[103124]: pgmap v5597: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 57 KiB/s wr, 12 op/s 2023-09-10T02:09:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10190.169150 secs 2023-09-10T02:09:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:18 smithi170 ceph-mon[102832]: pgmap v5597: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 57 KiB/s wr, 12 op/s 2023-09-10T02:09:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10190.169150 secs 2023-09-10T02:09:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10190.880866 secs 2023-09-10T02:09:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10190.880866 secs 2023-09-10T02:09:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10190.880866 secs 2023-09-10T02:09:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:20 smithi082 ceph-mon[98694]: pgmap v5598: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 56 KiB/s wr, 12 op/s 2023-09-10T02:09:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:20 smithi098 ceph-mon[103124]: pgmap v5598: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 56 KiB/s wr, 12 op/s 2023-09-10T02:09:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:20 smithi170 ceph-mon[102832]: pgmap v5598: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 56 KiB/s wr, 12 op/s 2023-09-10T02:09:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:22 smithi170 ceph-mon[102832]: pgmap v5599: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T02:09:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:22 smithi082 ceph-mon[98694]: pgmap v5599: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T02:09:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:22 smithi098 ceph-mon[103124]: pgmap v5599: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T02:09:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10193.744039 secs 2023-09-10T02:09:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10195.169208 secs 2023-09-10T02:09:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10193.744039 secs 2023-09-10T02:09:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10195.169208 secs 2023-09-10T02:09:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10193.744039 secs 2023-09-10T02:09:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10195.169208 secs 2023-09-10T02:09:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:24 smithi170 ceph-mon[102832]: pgmap v5600: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T02:09:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10195.880932 secs 2023-09-10T02:09:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:24 smithi082 ceph-mon[98694]: pgmap v5600: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T02:09:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10195.880932 secs 2023-09-10T02:09:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:24 smithi098 ceph-mon[103124]: pgmap v5600: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T02:09:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10195.880932 secs 2023-09-10T02:09:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:26 smithi170 ceph-mon[102832]: pgmap v5601: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T02:09:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:26 smithi082 ceph-mon[98694]: pgmap v5601: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T02:09:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:26 smithi098 ceph-mon[103124]: pgmap v5601: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T02:09:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10198.744157 secs 2023-09-10T02:09:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:28 smithi170 ceph-mon[102832]: pgmap v5602: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 45 KiB/s wr, 10 op/s 2023-09-10T02:09:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10200.169266 secs 2023-09-10T02:09:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10198.744157 secs 2023-09-10T02:09:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:28 smithi082 ceph-mon[98694]: pgmap v5602: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 45 KiB/s wr, 10 op/s 2023-09-10T02:09:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10200.169266 secs 2023-09-10T02:09:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10198.744157 secs 2023-09-10T02:09:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:28 smithi098 ceph-mon[103124]: pgmap v5602: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 45 KiB/s wr, 10 op/s 2023-09-10T02:09:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10200.169266 secs 2023-09-10T02:09:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10200.880998 secs 2023-09-10T02:09:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10200.880998 secs 2023-09-10T02:09:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10200.880998 secs 2023-09-10T02:09:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:30 smithi170 ceph-mon[102832]: pgmap v5603: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 41 KiB/s wr, 9 op/s 2023-09-10T02:09:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:30 smithi082 ceph-mon[98694]: pgmap v5603: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 41 KiB/s wr, 9 op/s 2023-09-10T02:09:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:30 smithi098 ceph-mon[103124]: pgmap v5603: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 41 KiB/s wr, 9 op/s 2023-09-10T02:09:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:32 smithi170 ceph-mon[102832]: pgmap v5604: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 136 KiB/s wr, 15 op/s 2023-09-10T02:09:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:32 smithi082 ceph-mon[98694]: pgmap v5604: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 136 KiB/s wr, 15 op/s 2023-09-10T02:09:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:32 smithi098 ceph-mon[103124]: pgmap v5604: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 136 KiB/s wr, 15 op/s 2023-09-10T02:09:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10205.169326 secs 2023-09-10T02:09:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10205.169326 secs 2023-09-10T02:09:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10205.169326 secs 2023-09-10T02:09:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:34 smithi170 ceph-mon[102832]: pgmap v5605: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 135 KiB/s wr, 10 op/s 2023-09-10T02:09:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10205.881060 secs 2023-09-10T02:09:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:34 smithi082 ceph-mon[98694]: pgmap v5605: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 135 KiB/s wr, 10 op/s 2023-09-10T02:09:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10205.881060 secs 2023-09-10T02:09:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:34 smithi098 ceph-mon[103124]: pgmap v5605: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 135 KiB/s wr, 10 op/s 2023-09-10T02:09:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10205.881060 secs 2023-09-10T02:09:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:09:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:09:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:09:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:36 smithi170 ceph-mon[102832]: pgmap v5606: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 136 KiB/s wr, 28 op/s 2023-09-10T02:09:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:09:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:09:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:09:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:36 smithi082 ceph-mon[98694]: pgmap v5606: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 136 KiB/s wr, 28 op/s 2023-09-10T02:09:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:09:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:09:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:09:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:36 smithi098 ceph-mon[103124]: pgmap v5606: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 136 KiB/s wr, 28 op/s 2023-09-10T02:09:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:09:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:09:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:09:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10208.744293 secs 2023-09-10T02:09:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:38 smithi170 ceph-mon[102832]: pgmap v5607: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 228 KiB/s wr, 29 op/s 2023-09-10T02:09:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10210.169382 secs 2023-09-10T02:09:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10208.744293 secs 2023-09-10T02:09:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:38 smithi082 ceph-mon[98694]: pgmap v5607: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 228 KiB/s wr, 29 op/s 2023-09-10T02:09:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10210.169382 secs 2023-09-10T02:09:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10208.744293 secs 2023-09-10T02:09:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:38 smithi098 ceph-mon[103124]: pgmap v5607: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 228 KiB/s wr, 29 op/s 2023-09-10T02:09:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10210.169382 secs 2023-09-10T02:09:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10210.881123 secs 2023-09-10T02:09:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10210.881123 secs 2023-09-10T02:09:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10210.881123 secs 2023-09-10T02:09:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:40 smithi170 ceph-mon[102832]: pgmap v5608: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 188 KiB/s wr, 26 op/s 2023-09-10T02:09:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:09:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:09:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:40 smithi082 ceph-mon[98694]: pgmap v5608: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 188 KiB/s wr, 26 op/s 2023-09-10T02:09:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:09:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:09:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:40 smithi098 ceph-mon[103124]: pgmap v5608: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 188 KiB/s wr, 26 op/s 2023-09-10T02:09:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:09:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:09:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:41 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:09:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:41 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:09:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:41 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:09:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:42 smithi170 ceph-mon[102832]: pgmap v5609: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 212 KiB/s wr, 33 op/s 2023-09-10T02:09:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:42 smithi082 ceph-mon[98694]: pgmap v5609: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 212 KiB/s wr, 33 op/s 2023-09-10T02:09:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:42 smithi098 ceph-mon[103124]: pgmap v5609: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 212 KiB/s wr, 33 op/s 2023-09-10T02:09:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10213.744405 secs 2023-09-10T02:09:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10215.169463 secs 2023-09-10T02:09:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10213.744405 secs 2023-09-10T02:09:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10215.169463 secs 2023-09-10T02:09:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10213.744405 secs 2023-09-10T02:09:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10215.169463 secs 2023-09-10T02:09:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:44 smithi170 ceph-mon[102832]: pgmap v5610: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 118 KiB/s wr, 27 op/s 2023-09-10T02:09:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10215.881186 secs 2023-09-10T02:09:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:44 smithi082 ceph-mon[98694]: pgmap v5610: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 118 KiB/s wr, 27 op/s 2023-09-10T02:09:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10215.881186 secs 2023-09-10T02:09:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:44 smithi098 ceph-mon[103124]: pgmap v5610: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 118 KiB/s wr, 27 op/s 2023-09-10T02:09:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10215.881186 secs 2023-09-10T02:09:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:46 smithi170 ceph-mon[102832]: pgmap v5611: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 118 KiB/s wr, 29 op/s 2023-09-10T02:09:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:46 smithi082 ceph-mon[98694]: pgmap v5611: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 118 KiB/s wr, 29 op/s 2023-09-10T02:09:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:46 smithi098 ceph-mon[103124]: pgmap v5611: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 118 KiB/s wr, 29 op/s 2023-09-10T02:09:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10218.744500 secs 2023-09-10T02:09:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:48 smithi170 ceph-mon[102832]: pgmap v5612: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 204 KiB/s wr, 15 op/s 2023-09-10T02:09:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10220.169554 secs 2023-09-10T02:09:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10218.744500 secs 2023-09-10T02:09:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:48 smithi082 ceph-mon[98694]: pgmap v5612: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 204 KiB/s wr, 15 op/s 2023-09-10T02:09:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10220.169554 secs 2023-09-10T02:09:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10218.744500 secs 2023-09-10T02:09:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:48 smithi098 ceph-mon[103124]: pgmap v5612: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 204 KiB/s wr, 15 op/s 2023-09-10T02:09:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10220.169554 secs 2023-09-10T02:09:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10220.881246 secs 2023-09-10T02:09:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10220.881246 secs 2023-09-10T02:09:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10220.881246 secs 2023-09-10T02:09:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:50 smithi170 ceph-mon[102832]: pgmap v5613: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T02:09:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:50 smithi082 ceph-mon[98694]: pgmap v5613: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T02:09:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:50 smithi098 ceph-mon[103124]: pgmap v5613: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T02:09:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:52 smithi170 ceph-mon[102832]: pgmap v5614: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 187 KiB/s wr, 18 op/s 2023-09-10T02:09:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:52 smithi082 ceph-mon[98694]: pgmap v5614: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 187 KiB/s wr, 18 op/s 2023-09-10T02:09:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:52 smithi098 ceph-mon[103124]: pgmap v5614: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 187 KiB/s wr, 18 op/s 2023-09-10T02:09:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10223.744621 secs 2023-09-10T02:09:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10225.169610 secs 2023-09-10T02:09:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10223.744621 secs 2023-09-10T02:09:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10225.169610 secs 2023-09-10T02:09:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10223.744621 secs 2023-09-10T02:09:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10225.169610 secs 2023-09-10T02:09:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:54 smithi170 ceph-mon[102832]: pgmap v5615: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T02:09:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10225.881310 secs 2023-09-10T02:09:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:54 smithi082 ceph-mon[98694]: pgmap v5615: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T02:09:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10225.881310 secs 2023-09-10T02:09:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:54 smithi098 ceph-mon[103124]: pgmap v5615: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T02:09:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10225.881310 secs 2023-09-10T02:09:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:56 smithi170 ceph-mon[102832]: pgmap v5616: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 163 KiB/s wr, 15 op/s 2023-09-10T02:09:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:56 smithi082 ceph-mon[98694]: pgmap v5616: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 163 KiB/s wr, 15 op/s 2023-09-10T02:09:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:56 smithi098 ceph-mon[103124]: pgmap v5616: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 163 KiB/s wr, 15 op/s 2023-09-10T02:09:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10228.744679 secs 2023-09-10T02:09:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:58 smithi170 ceph-mon[102832]: pgmap v5617: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 182 KiB/s wr, 15 op/s 2023-09-10T02:09:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10230.169668 secs 2023-09-10T02:09:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10228.744679 secs 2023-09-10T02:09:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:58 smithi082 ceph-mon[98694]: pgmap v5617: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 182 KiB/s wr, 15 op/s 2023-09-10T02:09:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10230.169668 secs 2023-09-10T02:09:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10228.744679 secs 2023-09-10T02:09:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:58 smithi098 ceph-mon[103124]: pgmap v5617: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 182 KiB/s wr, 15 op/s 2023-09-10T02:09:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10230.169668 secs 2023-09-10T02:09:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:09:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10230.881374 secs 2023-09-10T02:09:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:09:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10230.881374 secs 2023-09-10T02:09:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:09:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10230.881374 secs 2023-09-10T02:10:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:00 smithi170 ceph-mon[102832]: pgmap v5618: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T02:10:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T02:10:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:00 smithi082 ceph-mon[98694]: pgmap v5618: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T02:10:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T02:10:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:00 smithi098 ceph-mon[103124]: pgmap v5618: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T02:10:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T02:10:02.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:02 smithi170 ceph-mon[102832]: pgmap v5619: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 135 KiB/s wr, 22 op/s 2023-09-10T02:10:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:02 smithi082 ceph-mon[98694]: pgmap v5619: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 135 KiB/s wr, 22 op/s 2023-09-10T02:10:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:02 smithi098 ceph-mon[103124]: pgmap v5619: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 135 KiB/s wr, 22 op/s 2023-09-10T02:10:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10233.744801 secs 2023-09-10T02:10:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10235.169724 secs 2023-09-10T02:10:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10233.744801 secs 2023-09-10T02:10:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10235.169724 secs 2023-09-10T02:10:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10233.744801 secs 2023-09-10T02:10:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10235.169724 secs 2023-09-10T02:10:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:04 smithi170 ceph-mon[102832]: pgmap v5620: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 59 KiB/s wr, 16 op/s 2023-09-10T02:10:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10235.881438 secs 2023-09-10T02:10:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:04 smithi082 ceph-mon[98694]: pgmap v5620: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 59 KiB/s wr, 16 op/s 2023-09-10T02:10:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10235.881438 secs 2023-09-10T02:10:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:04 smithi098 ceph-mon[103124]: pgmap v5620: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 59 KiB/s wr, 16 op/s 2023-09-10T02:10:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10235.881438 secs 2023-09-10T02:10:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:06 smithi170 ceph-mon[102832]: pgmap v5621: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 70 KiB/s wr, 25 op/s 2023-09-10T02:10:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:06 smithi082 ceph-mon[98694]: pgmap v5621: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 70 KiB/s wr, 25 op/s 2023-09-10T02:10:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:06 smithi098 ceph-mon[103124]: pgmap v5621: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 70 KiB/s wr, 25 op/s 2023-09-10T02:10:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10238.744872 secs 2023-09-10T02:10:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:08 smithi170 ceph-mon[102832]: pgmap v5622: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 81 KiB/s wr, 26 op/s 2023-09-10T02:10:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:08 smithi170 ceph-mon[102832]: osdmap e419: 12 total, 12 up, 12 in 2023-09-10T02:10:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10240.169756 secs 2023-09-10T02:10:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10238.744872 secs 2023-09-10T02:10:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:08 smithi082 ceph-mon[98694]: pgmap v5622: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 81 KiB/s wr, 26 op/s 2023-09-10T02:10:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:08 smithi082 ceph-mon[98694]: osdmap e419: 12 total, 12 up, 12 in 2023-09-10T02:10:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10240.169756 secs 2023-09-10T02:10:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10238.744872 secs 2023-09-10T02:10:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:08 smithi098 ceph-mon[103124]: pgmap v5622: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 81 KiB/s wr, 26 op/s 2023-09-10T02:10:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:08 smithi098 ceph-mon[103124]: osdmap e419: 12 total, 12 up, 12 in 2023-09-10T02:10:08.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10240.169756 secs 2023-09-10T02:10:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10240.881489 secs 2023-09-10T02:10:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10240.881489 secs 2023-09-10T02:10:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10240.881489 secs 2023-09-10T02:10:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:10 smithi170 ceph-mon[102832]: pgmap v5624: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 74 KiB/s wr, 28 op/s 2023-09-10T02:10:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:10 smithi082 ceph-mon[98694]: pgmap v5624: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 74 KiB/s wr, 28 op/s 2023-09-10T02:10:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:10 smithi098 ceph-mon[103124]: pgmap v5624: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 74 KiB/s wr, 28 op/s 2023-09-10T02:10:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:12 smithi170 ceph-mon[102832]: pgmap v5625: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 28 KiB/s wr, 21 op/s 2023-09-10T02:10:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:12 smithi082 ceph-mon[98694]: pgmap v5625: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 28 KiB/s wr, 21 op/s 2023-09-10T02:10:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:12 smithi098 ceph-mon[103124]: pgmap v5625: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 28 KiB/s wr, 21 op/s 2023-09-10T02:10:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10243.744935 secs 2023-09-10T02:10:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:10:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10245.169825 secs 2023-09-10T02:10:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:10:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:10:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10243.744935 secs 2023-09-10T02:10:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:10:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10245.169825 secs 2023-09-10T02:10:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:10:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:10:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10243.744935 secs 2023-09-10T02:10:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:10:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10245.169825 secs 2023-09-10T02:10:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:10:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:10:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:14 smithi170 ceph-mon[102832]: pgmap v5626: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 28 KiB/s wr, 21 op/s 2023-09-10T02:10:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10245.881553 secs 2023-09-10T02:10:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:10:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:10:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:10:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:14 smithi082 ceph-mon[98694]: pgmap v5626: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 28 KiB/s wr, 21 op/s 2023-09-10T02:10:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10245.881553 secs 2023-09-10T02:10:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:10:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:10:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:10:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:14 smithi098 ceph-mon[103124]: pgmap v5626: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 28 KiB/s wr, 21 op/s 2023-09-10T02:10:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10245.881553 secs 2023-09-10T02:10:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:10:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:10:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:10:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:16 smithi170 ceph-mon[102832]: pgmap v5627: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-09-10T02:10:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:16 smithi170 ceph-mon[102832]: osdmap e420: 12 total, 12 up, 12 in 2023-09-10T02:10:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:16 smithi082 ceph-mon[98694]: pgmap v5627: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-09-10T02:10:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:16 smithi082 ceph-mon[98694]: osdmap e420: 12 total, 12 up, 12 in 2023-09-10T02:10:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:16 smithi098 ceph-mon[103124]: pgmap v5627: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-09-10T02:10:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:16 smithi098 ceph-mon[103124]: osdmap e420: 12 total, 12 up, 12 in 2023-09-10T02:10:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10248.745005 secs 2023-09-10T02:10:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:18 smithi170 ceph-mon[102832]: pgmap v5629: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T02:10:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10250.169886 secs 2023-09-10T02:10:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10248.745005 secs 2023-09-10T02:10:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:18 smithi082 ceph-mon[98694]: pgmap v5629: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T02:10:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10250.169886 secs 2023-09-10T02:10:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10248.745005 secs 2023-09-10T02:10:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:18 smithi098 ceph-mon[103124]: pgmap v5629: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T02:10:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10250.169886 secs 2023-09-10T02:10:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10250.881618 secs 2023-09-10T02:10:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10250.881618 secs 2023-09-10T02:10:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10250.881618 secs 2023-09-10T02:10:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:20 smithi170 ceph-mon[102832]: pgmap v5630: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T02:10:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:20 smithi082 ceph-mon[98694]: pgmap v5630: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T02:10:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:20 smithi098 ceph-mon[103124]: pgmap v5630: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T02:10:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:22 smithi170 ceph-mon[102832]: pgmap v5631: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T02:10:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:22 smithi082 ceph-mon[98694]: pgmap v5631: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T02:10:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:22 smithi098 ceph-mon[103124]: pgmap v5631: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T02:10:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:10:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10255.169964 secs 2023-09-10T02:10:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:10:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10255.169964 secs 2023-09-10T02:10:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:10:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10255.169964 secs 2023-09-10T02:10:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:24 smithi170 ceph-mon[102832]: pgmap v5632: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T02:10:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:10:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:10:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:10:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10255.881681 secs 2023-09-10T02:10:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:24 smithi082 ceph-mon[98694]: pgmap v5632: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T02:10:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:10:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:10:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:10:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10255.881681 secs 2023-09-10T02:10:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:24 smithi098 ceph-mon[103124]: pgmap v5632: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T02:10:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:10:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:10:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:10:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10255.881681 secs 2023-09-10T02:10:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:26 smithi170 ceph-mon[102832]: pgmap v5633: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T02:10:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:26 smithi082 ceph-mon[98694]: pgmap v5633: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T02:10:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:26 smithi098 ceph-mon[103124]: pgmap v5633: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T02:10:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:28 smithi170 ceph-mon[102832]: pgmap v5634: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 137 KiB/s wr, 11 op/s 2023-09-10T02:10:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10260.170025 secs 2023-09-10T02:10:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:28 smithi082 ceph-mon[98694]: pgmap v5634: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 137 KiB/s wr, 11 op/s 2023-09-10T02:10:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10260.170025 secs 2023-09-10T02:10:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:28 smithi098 ceph-mon[103124]: pgmap v5634: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 137 KiB/s wr, 11 op/s 2023-09-10T02:10:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10260.170025 secs 2023-09-10T02:10:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10260.881747 secs 2023-09-10T02:10:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10260.881747 secs 2023-09-10T02:10:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10260.881747 secs 2023-09-10T02:10:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:30 smithi170 ceph-mon[102832]: pgmap v5635: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 126 KiB/s wr, 10 op/s 2023-09-10T02:10:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:30 smithi082 ceph-mon[98694]: pgmap v5635: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 126 KiB/s wr, 10 op/s 2023-09-10T02:10:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:30 smithi098 ceph-mon[103124]: pgmap v5635: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 126 KiB/s wr, 10 op/s 2023-09-10T02:10:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:32 smithi170 ceph-mon[102832]: pgmap v5636: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 159 KiB/s wr, 16 op/s 2023-09-10T02:10:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:32 smithi082 ceph-mon[98694]: pgmap v5636: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 159 KiB/s wr, 16 op/s 2023-09-10T02:10:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:32 smithi098 ceph-mon[103124]: pgmap v5636: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 159 KiB/s wr, 16 op/s 2023-09-10T02:10:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10265.170083 secs 2023-09-10T02:10:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10265.170083 secs 2023-09-10T02:10:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10265.170083 secs 2023-09-10T02:10:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:34 smithi170 ceph-mon[102832]: pgmap v5637: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T02:10:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10265.881810 secs 2023-09-10T02:10:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:34 smithi082 ceph-mon[98694]: pgmap v5637: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T02:10:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10265.881810 secs 2023-09-10T02:10:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:34 smithi098 ceph-mon[103124]: pgmap v5637: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T02:10:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10265.881810 secs 2023-09-10T02:10:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:36 smithi170 ceph-mon[102832]: pgmap v5638: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 158 KiB/s wr, 14 op/s 2023-09-10T02:10:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:36 smithi082 ceph-mon[98694]: pgmap v5638: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 158 KiB/s wr, 14 op/s 2023-09-10T02:10:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:36 smithi098 ceph-mon[103124]: pgmap v5638: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 158 KiB/s wr, 14 op/s 2023-09-10T02:10:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:38 smithi170 ceph-mon[102832]: pgmap v5639: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 232 KiB/s wr, 15 op/s 2023-09-10T02:10:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10270.170139 secs 2023-09-10T02:10:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:38 smithi082 ceph-mon[98694]: pgmap v5639: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 232 KiB/s wr, 15 op/s 2023-09-10T02:10:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10270.170139 secs 2023-09-10T02:10:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:38 smithi098 ceph-mon[103124]: pgmap v5639: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 232 KiB/s wr, 15 op/s 2023-09-10T02:10:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10270.170139 secs 2023-09-10T02:10:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10270.881873 secs 2023-09-10T02:10:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10270.881873 secs 2023-09-10T02:10:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10270.881873 secs 2023-09-10T02:10:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:40 smithi170 ceph-mon[102832]: pgmap v5640: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T02:10:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:40 smithi082 ceph-mon[98694]: pgmap v5640: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T02:10:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:40 smithi098 ceph-mon[103124]: pgmap v5640: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T02:10:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:42 smithi170 ceph-mon[102832]: pgmap v5641: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 194 KiB/s wr, 18 op/s 2023-09-10T02:10:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:42 smithi082 ceph-mon[98694]: pgmap v5641: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 194 KiB/s wr, 18 op/s 2023-09-10T02:10:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:42 smithi098 ceph-mon[103124]: pgmap v5641: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 194 KiB/s wr, 18 op/s 2023-09-10T02:10:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10273.745443 secs 2023-09-10T02:10:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10275.170194 secs 2023-09-10T02:10:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10273.745443 secs 2023-09-10T02:10:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10275.170194 secs 2023-09-10T02:10:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10273.745443 secs 2023-09-10T02:10:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10275.170194 secs 2023-09-10T02:10:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:44 smithi170 ceph-mon[102832]: pgmap v5642: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T02:10:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:10:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:10:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:10:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10275.881936 secs 2023-09-10T02:10:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:44 smithi082 ceph-mon[98694]: pgmap v5642: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T02:10:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:10:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:10:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:10:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10275.881936 secs 2023-09-10T02:10:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:44 smithi098 ceph-mon[103124]: pgmap v5642: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T02:10:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:10:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:10:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:10:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10275.881936 secs 2023-09-10T02:10:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:46 smithi170 ceph-mon[102832]: pgmap v5643: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 160 KiB/s wr, 14 op/s 2023-09-10T02:10:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:46 smithi082 ceph-mon[98694]: pgmap v5643: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 160 KiB/s wr, 14 op/s 2023-09-10T02:10:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:46 smithi098 ceph-mon[103124]: pgmap v5643: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 160 KiB/s wr, 14 op/s 2023-09-10T02:10:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10278.745504 secs 2023-09-10T02:10:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:48 smithi170 ceph-mon[102832]: pgmap v5644: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 15 op/s 2023-09-10T02:10:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10280.170255 secs 2023-09-10T02:10:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10278.745504 secs 2023-09-10T02:10:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:48 smithi082 ceph-mon[98694]: pgmap v5644: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 15 op/s 2023-09-10T02:10:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10280.170255 secs 2023-09-10T02:10:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10278.745504 secs 2023-09-10T02:10:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:48 smithi098 ceph-mon[103124]: pgmap v5644: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 15 op/s 2023-09-10T02:10:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10280.170255 secs 2023-09-10T02:10:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10280.882023 secs 2023-09-10T02:10:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10280.882023 secs 2023-09-10T02:10:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10280.882023 secs 2023-09-10T02:10:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:50 smithi170 ceph-mon[102832]: pgmap v5645: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T02:10:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:50 smithi082 ceph-mon[98694]: pgmap v5645: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T02:10:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:50 smithi098 ceph-mon[103124]: pgmap v5645: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T02:10:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:52 smithi170 ceph-mon[102832]: pgmap v5646: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 184 KiB/s wr, 17 op/s 2023-09-10T02:10:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:52 smithi082 ceph-mon[98694]: pgmap v5646: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 184 KiB/s wr, 17 op/s 2023-09-10T02:10:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:52 smithi098 ceph-mon[103124]: pgmap v5646: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 184 KiB/s wr, 17 op/s 2023-09-10T02:10:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10283.745564 secs 2023-09-10T02:10:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10285.170311 secs 2023-09-10T02:10:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10283.745564 secs 2023-09-10T02:10:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10285.170311 secs 2023-09-10T02:10:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10283.745564 secs 2023-09-10T02:10:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10285.170311 secs 2023-09-10T02:10:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:54 smithi170 ceph-mon[102832]: pgmap v5647: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T02:10:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10285.882087 secs 2023-09-10T02:10:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:54 smithi082 ceph-mon[98694]: pgmap v5647: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T02:10:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10285.882087 secs 2023-09-10T02:10:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:54 smithi098 ceph-mon[103124]: pgmap v5647: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T02:10:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10285.882087 secs 2023-09-10T02:10:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:56 smithi170 ceph-mon[102832]: pgmap v5648: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 99 KiB/s wr, 13 op/s 2023-09-10T02:10:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:56 smithi082 ceph-mon[98694]: pgmap v5648: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 99 KiB/s wr, 13 op/s 2023-09-10T02:10:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:56 smithi098 ceph-mon[103124]: pgmap v5648: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 99 KiB/s wr, 13 op/s 2023-09-10T02:10:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10288.745664 secs 2023-09-10T02:10:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:58 smithi170 ceph-mon[102832]: pgmap v5649: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 118 KiB/s wr, 13 op/s 2023-09-10T02:10:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10290.170371 secs 2023-09-10T02:10:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10288.745664 secs 2023-09-10T02:10:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:58 smithi082 ceph-mon[98694]: pgmap v5649: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 118 KiB/s wr, 13 op/s 2023-09-10T02:10:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10290.170371 secs 2023-09-10T02:10:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10288.745664 secs 2023-09-10T02:10:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:58 smithi098 ceph-mon[103124]: pgmap v5649: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 118 KiB/s wr, 13 op/s 2023-09-10T02:10:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10290.170371 secs 2023-09-10T02:10:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:10:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10290.882154 secs 2023-09-10T02:10:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:10:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10290.882154 secs 2023-09-10T02:10:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:10:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10290.882154 secs 2023-09-10T02:11:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:00 smithi170 ceph-mon[102832]: pgmap v5650: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 10 op/s 2023-09-10T02:11:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:00 smithi082 ceph-mon[98694]: pgmap v5650: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 10 op/s 2023-09-10T02:11:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:00 smithi098 ceph-mon[103124]: pgmap v5650: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 10 op/s 2023-09-10T02:11:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:02 smithi170 ceph-mon[102832]: pgmap v5651: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 112 KiB/s wr, 15 op/s 2023-09-10T02:11:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:02 smithi082 ceph-mon[98694]: pgmap v5651: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 112 KiB/s wr, 15 op/s 2023-09-10T02:11:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:02 smithi098 ceph-mon[103124]: pgmap v5651: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 112 KiB/s wr, 15 op/s 2023-09-10T02:11:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10293.745726 secs 2023-09-10T02:11:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10295.170422 secs 2023-09-10T02:11:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10293.745726 secs 2023-09-10T02:11:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10295.170422 secs 2023-09-10T02:11:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10293.745726 secs 2023-09-10T02:11:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10295.170422 secs 2023-09-10T02:11:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:04 smithi170 ceph-mon[102832]: pgmap v5652: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T02:11:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10295.882216 secs 2023-09-10T02:11:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:04 smithi082 ceph-mon[98694]: pgmap v5652: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T02:11:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10295.882216 secs 2023-09-10T02:11:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:04 smithi098 ceph-mon[103124]: pgmap v5652: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T02:11:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10295.882216 secs 2023-09-10T02:11:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:06 smithi170 ceph-mon[102832]: pgmap v5653: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 67 KiB/s wr, 24 op/s 2023-09-10T02:11:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:06 smithi082 ceph-mon[98694]: pgmap v5653: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 67 KiB/s wr, 24 op/s 2023-09-10T02:11:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:06 smithi098 ceph-mon[103124]: pgmap v5653: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 67 KiB/s wr, 24 op/s 2023-09-10T02:11:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10298.745836 secs 2023-09-10T02:11:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:08 smithi170 ceph-mon[102832]: pgmap v5654: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 116 KiB/s wr, 27 op/s 2023-09-10T02:11:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:08 smithi170 ceph-mon[102832]: osdmap e421: 12 total, 12 up, 12 in 2023-09-10T02:11:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10300.170481 secs 2023-09-10T02:11:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10298.745836 secs 2023-09-10T02:11:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:08 smithi082 ceph-mon[98694]: pgmap v5654: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 116 KiB/s wr, 27 op/s 2023-09-10T02:11:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:08 smithi082 ceph-mon[98694]: osdmap e421: 12 total, 12 up, 12 in 2023-09-10T02:11:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10300.170481 secs 2023-09-10T02:11:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10298.745836 secs 2023-09-10T02:11:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:08 smithi098 ceph-mon[103124]: pgmap v5654: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 116 KiB/s wr, 27 op/s 2023-09-10T02:11:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:08 smithi098 ceph-mon[103124]: osdmap e421: 12 total, 12 up, 12 in 2023-09-10T02:11:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10300.170481 secs 2023-09-10T02:11:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10300.882281 secs 2023-09-10T02:11:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10300.882281 secs 2023-09-10T02:11:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10300.882281 secs 2023-09-10T02:11:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:10 smithi170 ceph-mon[102832]: pgmap v5656: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 116 KiB/s wr, 29 op/s 2023-09-10T02:11:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:10 smithi082 ceph-mon[98694]: pgmap v5656: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 116 KiB/s wr, 29 op/s 2023-09-10T02:11:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:10 smithi098 ceph-mon[103124]: pgmap v5656: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 116 KiB/s wr, 29 op/s 2023-09-10T02:11:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:12 smithi170 ceph-mon[102832]: pgmap v5657: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 96 KiB/s wr, 28 op/s 2023-09-10T02:11:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:12 smithi082 ceph-mon[98694]: pgmap v5657: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 96 KiB/s wr, 28 op/s 2023-09-10T02:11:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:12 smithi098 ceph-mon[103124]: pgmap v5657: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 96 KiB/s wr, 28 op/s 2023-09-10T02:11:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10303.745897 secs 2023-09-10T02:11:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10305.170491 secs 2023-09-10T02:11:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10303.745897 secs 2023-09-10T02:11:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10305.170491 secs 2023-09-10T02:11:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10303.745897 secs 2023-09-10T02:11:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10305.170491 secs 2023-09-10T02:11:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:14 smithi170 ceph-mon[102832]: pgmap v5658: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 96 KiB/s wr, 28 op/s 2023-09-10T02:11:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10305.882343 secs 2023-09-10T02:11:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:11:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:11:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:11:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:14 smithi082 ceph-mon[98694]: pgmap v5658: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 96 KiB/s wr, 28 op/s 2023-09-10T02:11:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10305.882343 secs 2023-09-10T02:11:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:11:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:11:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:11:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:14 smithi098 ceph-mon[103124]: pgmap v5658: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 96 KiB/s wr, 28 op/s 2023-09-10T02:11:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10305.882343 secs 2023-09-10T02:11:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:11:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:11:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:11:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:11:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:11:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:11:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:11:15.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:11:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:11:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:11:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:11:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:11:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:11:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:11:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:11:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:11:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:11:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:11:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:17 smithi098 ceph-mon[103124]: pgmap v5659: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 71 KiB/s wr, 14 op/s 2023-09-10T02:11:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:17 smithi098 ceph-mon[103124]: osdmap e422: 12 total, 12 up, 12 in 2023-09-10T02:11:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:17 smithi170 ceph-mon[102832]: pgmap v5659: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 71 KiB/s wr, 14 op/s 2023-09-10T02:11:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:17 smithi170 ceph-mon[102832]: osdmap e422: 12 total, 12 up, 12 in 2023-09-10T02:11:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:17 smithi082 ceph-mon[98694]: pgmap v5659: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 71 KiB/s wr, 14 op/s 2023-09-10T02:11:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:17 smithi082 ceph-mon[98694]: osdmap e422: 12 total, 12 up, 12 in 2023-09-10T02:11:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10308.745963 secs 2023-09-10T02:11:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10308.745963 secs 2023-09-10T02:11:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10308.745963 secs 2023-09-10T02:11:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:19 smithi170 ceph-mon[102832]: pgmap v5661: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T02:11:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10310.170548 secs 2023-09-10T02:11:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10310.882406 secs 2023-09-10T02:11:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:19 smithi082 ceph-mon[98694]: pgmap v5661: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T02:11:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10310.170548 secs 2023-09-10T02:11:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10310.882406 secs 2023-09-10T02:11:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:19 smithi098 ceph-mon[103124]: pgmap v5661: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T02:11:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10310.170548 secs 2023-09-10T02:11:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10310.882406 secs 2023-09-10T02:11:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:21 smithi170 ceph-mon[102832]: pgmap v5662: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T02:11:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:21 smithi082 ceph-mon[98694]: pgmap v5662: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T02:11:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:21 smithi098 ceph-mon[103124]: pgmap v5662: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T02:11:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:23 smithi170 ceph-mon[102832]: pgmap v5663: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2023-09-10T02:11:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:11:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:23 smithi082 ceph-mon[98694]: pgmap v5663: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2023-09-10T02:11:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:11:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:23 smithi098 ceph-mon[103124]: pgmap v5663: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2023-09-10T02:11:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:11:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10315.170606 secs 2023-09-10T02:11:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:11:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:11:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:11:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10315.882473 secs 2023-09-10T02:11:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10315.170606 secs 2023-09-10T02:11:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:11:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:11:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:11:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10315.882473 secs 2023-09-10T02:11:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10315.170606 secs 2023-09-10T02:11:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:11:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:11:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:11:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10315.882473 secs 2023-09-10T02:11:25.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:25 smithi170 ceph-mon[102832]: pgmap v5664: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2023-09-10T02:11:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:25 smithi082 ceph-mon[98694]: pgmap v5664: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2023-09-10T02:11:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:25 smithi098 ceph-mon[103124]: pgmap v5664: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2023-09-10T02:11:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:27 smithi170 ceph-mon[102832]: pgmap v5665: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 13 op/s 2023-09-10T02:11:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:27 smithi082 ceph-mon[98694]: pgmap v5665: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 13 op/s 2023-09-10T02:11:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:27 smithi098 ceph-mon[103124]: pgmap v5665: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 13 op/s 2023-09-10T02:11:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10320.170663 secs 2023-09-10T02:11:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10320.170663 secs 2023-09-10T02:11:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10320.170663 secs 2023-09-10T02:11:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:29 smithi170 ceph-mon[102832]: pgmap v5666: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T02:11:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10320.882485 secs 2023-09-10T02:11:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:29 smithi082 ceph-mon[98694]: pgmap v5666: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T02:11:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10320.882485 secs 2023-09-10T02:11:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:29 smithi098 ceph-mon[103124]: pgmap v5666: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T02:11:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10320.882485 secs 2023-09-10T02:11:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:31 smithi170 ceph-mon[102832]: pgmap v5667: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 10 op/s 2023-09-10T02:11:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:31 smithi082 ceph-mon[98694]: pgmap v5667: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 10 op/s 2023-09-10T02:11:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:31 smithi098 ceph-mon[103124]: pgmap v5667: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 10 op/s 2023-09-10T02:11:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:33 smithi170 ceph-mon[102832]: pgmap v5668: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 132 KiB/s wr, 17 op/s 2023-09-10T02:11:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10325.170721 secs 2023-09-10T02:11:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:33 smithi082 ceph-mon[98694]: pgmap v5668: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 132 KiB/s wr, 17 op/s 2023-09-10T02:11:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10325.170721 secs 2023-09-10T02:11:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:33 smithi098 ceph-mon[103124]: pgmap v5668: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 132 KiB/s wr, 17 op/s 2023-09-10T02:11:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10325.170721 secs 2023-09-10T02:11:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10325.882548 secs 2023-09-10T02:11:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10325.882548 secs 2023-09-10T02:11:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10325.882548 secs 2023-09-10T02:11:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:35 smithi170 ceph-mon[102832]: pgmap v5669: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T02:11:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:35 smithi082 ceph-mon[98694]: pgmap v5669: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T02:11:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:35 smithi098 ceph-mon[103124]: pgmap v5669: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T02:11:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:37 smithi170 ceph-mon[102832]: pgmap v5670: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 177 KiB/s wr, 16 op/s 2023-09-10T02:11:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:37 smithi082 ceph-mon[98694]: pgmap v5670: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 177 KiB/s wr, 16 op/s 2023-09-10T02:11:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:37 smithi098 ceph-mon[103124]: pgmap v5670: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 177 KiB/s wr, 16 op/s 2023-09-10T02:11:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10330.170783 secs 2023-09-10T02:11:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10330.170783 secs 2023-09-10T02:11:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10330.170783 secs 2023-09-10T02:11:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:39 smithi170 ceph-mon[102832]: pgmap v5671: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 123 KiB/s wr, 15 op/s 2023-09-10T02:11:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10330.882608 secs 2023-09-10T02:11:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:39 smithi082 ceph-mon[98694]: pgmap v5671: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 123 KiB/s wr, 15 op/s 2023-09-10T02:11:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10330.882608 secs 2023-09-10T02:11:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:39 smithi098 ceph-mon[103124]: pgmap v5671: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 123 KiB/s wr, 15 op/s 2023-09-10T02:11:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10330.882608 secs 2023-09-10T02:11:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:41 smithi170 ceph-mon[102832]: pgmap v5672: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T02:11:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:41 smithi082 ceph-mon[98694]: pgmap v5672: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T02:11:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:41 smithi098 ceph-mon[103124]: pgmap v5672: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T02:11:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:43 smithi170 ceph-mon[102832]: pgmap v5673: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 204 KiB/s wr, 19 op/s 2023-09-10T02:11:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10333.746368 secs 2023-09-10T02:11:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10335.170864 secs 2023-09-10T02:11:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:43 smithi082 ceph-mon[98694]: pgmap v5673: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 204 KiB/s wr, 19 op/s 2023-09-10T02:11:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10333.746368 secs 2023-09-10T02:11:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10335.170864 secs 2023-09-10T02:11:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:43 smithi098 ceph-mon[103124]: pgmap v5673: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 204 KiB/s wr, 19 op/s 2023-09-10T02:11:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10333.746368 secs 2023-09-10T02:11:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10335.170864 secs 2023-09-10T02:11:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:11:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:11:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:11:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10335.882671 secs 2023-09-10T02:11:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:11:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:11:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:11:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10335.882671 secs 2023-09-10T02:11:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:11:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:11:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:11:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10335.882671 secs 2023-09-10T02:11:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:45 smithi170 ceph-mon[102832]: pgmap v5674: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 143 KiB/s wr, 13 op/s 2023-09-10T02:11:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:45 smithi082 ceph-mon[98694]: pgmap v5674: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 143 KiB/s wr, 13 op/s 2023-09-10T02:11:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:45 smithi098 ceph-mon[103124]: pgmap v5674: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 143 KiB/s wr, 13 op/s 2023-09-10T02:11:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:47 smithi170 ceph-mon[102832]: pgmap v5675: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 191 KiB/s wr, 16 op/s 2023-09-10T02:11:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:47 smithi082 ceph-mon[98694]: pgmap v5675: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 191 KiB/s wr, 16 op/s 2023-09-10T02:11:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:47 smithi098 ceph-mon[103124]: pgmap v5675: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 191 KiB/s wr, 16 op/s 2023-09-10T02:11:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10338.746430 secs 2023-09-10T02:11:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10340.170923 secs 2023-09-10T02:11:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10338.746430 secs 2023-09-10T02:11:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10340.170923 secs 2023-09-10T02:11:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10338.746430 secs 2023-09-10T02:11:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10340.170923 secs 2023-09-10T02:11:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:49 smithi170 ceph-mon[102832]: pgmap v5676: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 145 KiB/s wr, 15 op/s 2023-09-10T02:11:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10340.882732 secs 2023-09-10T02:11:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:49 smithi082 ceph-mon[98694]: pgmap v5676: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 145 KiB/s wr, 15 op/s 2023-09-10T02:11:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10340.882732 secs 2023-09-10T02:11:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:49 smithi098 ceph-mon[103124]: pgmap v5676: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 145 KiB/s wr, 15 op/s 2023-09-10T02:11:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10340.882732 secs 2023-09-10T02:11:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:51 smithi170 ceph-mon[102832]: pgmap v5677: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T02:11:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:51 smithi082 ceph-mon[98694]: pgmap v5677: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T02:11:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:51 smithi098 ceph-mon[103124]: pgmap v5677: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T02:11:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:52 smithi170 ceph-mon[102832]: pgmap v5678: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 169 KiB/s wr, 17 op/s 2023-09-10T02:11:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:52 smithi082 ceph-mon[98694]: pgmap v5678: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 169 KiB/s wr, 17 op/s 2023-09-10T02:11:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:52 smithi098 ceph-mon[103124]: pgmap v5678: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 169 KiB/s wr, 17 op/s 2023-09-10T02:11:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10343.746523 secs 2023-09-10T02:11:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10345.170982 secs 2023-09-10T02:11:53.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10343.746523 secs 2023-09-10T02:11:53.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10345.170982 secs 2023-09-10T02:11:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10343.746523 secs 2023-09-10T02:11:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10345.170982 secs 2023-09-10T02:11:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:54 smithi170 ceph-mon[102832]: pgmap v5679: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T02:11:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10345.882795 secs 2023-09-10T02:11:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:54 smithi082 ceph-mon[98694]: pgmap v5679: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T02:11:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10345.882795 secs 2023-09-10T02:11:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:54 smithi098 ceph-mon[103124]: pgmap v5679: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T02:11:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10345.882795 secs 2023-09-10T02:11:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:56 smithi170 ceph-mon[102832]: pgmap v5680: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 13 op/s 2023-09-10T02:11:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:56 smithi082 ceph-mon[98694]: pgmap v5680: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 13 op/s 2023-09-10T02:11:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:56 smithi098 ceph-mon[103124]: pgmap v5680: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 13 op/s 2023-09-10T02:11:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10348.746608 secs 2023-09-10T02:11:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:58 smithi170 ceph-mon[102832]: pgmap v5681: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T02:11:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10350.171040 secs 2023-09-10T02:11:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10348.746608 secs 2023-09-10T02:11:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:58 smithi082 ceph-mon[98694]: pgmap v5681: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T02:11:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10350.171040 secs 2023-09-10T02:11:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10348.746608 secs 2023-09-10T02:11:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:58 smithi098 ceph-mon[103124]: pgmap v5681: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T02:11:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10350.171040 secs 2023-09-10T02:11:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:11:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10350.882857 secs 2023-09-10T02:11:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:11:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10350.882857 secs 2023-09-10T02:11:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:11:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10350.882857 secs 2023-09-10T02:12:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:00 smithi170 ceph-mon[102832]: pgmap v5682: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 63 KiB/s wr, 10 op/s 2023-09-10T02:12:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:00 smithi082 ceph-mon[98694]: pgmap v5682: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 63 KiB/s wr, 10 op/s 2023-09-10T02:12:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:00 smithi098 ceph-mon[103124]: pgmap v5682: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 63 KiB/s wr, 10 op/s 2023-09-10T02:12:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:02 smithi170 ceph-mon[102832]: pgmap v5683: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 79 KiB/s wr, 15 op/s 2023-09-10T02:12:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:02 smithi082 ceph-mon[98694]: pgmap v5683: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 79 KiB/s wr, 15 op/s 2023-09-10T02:12:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:02 smithi098 ceph-mon[103124]: pgmap v5683: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 79 KiB/s wr, 15 op/s 2023-09-10T02:12:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10353.746714 secs 2023-09-10T02:12:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10355.171098 secs 2023-09-10T02:12:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10353.746714 secs 2023-09-10T02:12:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10355.171098 secs 2023-09-10T02:12:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10353.746714 secs 2023-09-10T02:12:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10355.171098 secs 2023-09-10T02:12:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:04 smithi170 ceph-mon[102832]: pgmap v5684: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 47 KiB/s wr, 10 op/s 2023-09-10T02:12:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10355.882918 secs 2023-09-10T02:12:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:04 smithi082 ceph-mon[98694]: pgmap v5684: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 47 KiB/s wr, 10 op/s 2023-09-10T02:12:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10355.882918 secs 2023-09-10T02:12:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:04 smithi098 ceph-mon[103124]: pgmap v5684: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 47 KiB/s wr, 10 op/s 2023-09-10T02:12:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10355.882918 secs 2023-09-10T02:12:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:06 smithi170 ceph-mon[102832]: pgmap v5685: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 69 KiB/s wr, 26 op/s 2023-09-10T02:12:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:06 smithi082 ceph-mon[98694]: pgmap v5685: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 69 KiB/s wr, 26 op/s 2023-09-10T02:12:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:06 smithi098 ceph-mon[103124]: pgmap v5685: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 69 KiB/s wr, 26 op/s 2023-09-10T02:12:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10358.746820 secs 2023-09-10T02:12:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:08 smithi170 ceph-mon[102832]: pgmap v5686: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 68 KiB/s wr, 27 op/s 2023-09-10T02:12:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:08 smithi170 ceph-mon[102832]: osdmap e423: 12 total, 12 up, 12 in 2023-09-10T02:12:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10360.171159 secs 2023-09-10T02:12:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10358.746820 secs 2023-09-10T02:12:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:08 smithi082 ceph-mon[98694]: pgmap v5686: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 68 KiB/s wr, 27 op/s 2023-09-10T02:12:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:08 smithi082 ceph-mon[98694]: osdmap e423: 12 total, 12 up, 12 in 2023-09-10T02:12:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10360.171159 secs 2023-09-10T02:12:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10358.746820 secs 2023-09-10T02:12:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:08 smithi098 ceph-mon[103124]: pgmap v5686: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 68 KiB/s wr, 27 op/s 2023-09-10T02:12:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:08 smithi098 ceph-mon[103124]: osdmap e423: 12 total, 12 up, 12 in 2023-09-10T02:12:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10360.171159 secs 2023-09-10T02:12:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10360.882979 secs 2023-09-10T02:12:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10360.882979 secs 2023-09-10T02:12:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10360.882979 secs 2023-09-10T02:12:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:10 smithi170 ceph-mon[102832]: pgmap v5688: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 73 KiB/s wr, 30 op/s 2023-09-10T02:12:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:10 smithi082 ceph-mon[98694]: pgmap v5688: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 73 KiB/s wr, 30 op/s 2023-09-10T02:12:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:10 smithi098 ceph-mon[103124]: pgmap v5688: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 73 KiB/s wr, 30 op/s 2023-09-10T02:12:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:12 smithi170 ceph-mon[102832]: pgmap v5689: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T02:12:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:12 smithi082 ceph-mon[98694]: pgmap v5689: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T02:12:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:12 smithi098 ceph-mon[103124]: pgmap v5689: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T02:12:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10363.746878 secs 2023-09-10T02:12:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10365.171235 secs 2023-09-10T02:12:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10363.746878 secs 2023-09-10T02:12:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10365.171235 secs 2023-09-10T02:12:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10363.746878 secs 2023-09-10T02:12:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10365.171235 secs 2023-09-10T02:12:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:14 smithi170 ceph-mon[102832]: pgmap v5690: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T02:12:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10365.883042 secs 2023-09-10T02:12:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:14 smithi082 ceph-mon[98694]: pgmap v5690: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T02:12:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10365.883042 secs 2023-09-10T02:12:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:14 smithi098 ceph-mon[103124]: pgmap v5690: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T02:12:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10365.883042 secs 2023-09-10T02:12:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:12:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:12:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:12:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:16 smithi170 ceph-mon[102832]: pgmap v5691: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 59 KiB/s wr, 14 op/s 2023-09-10T02:12:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:12:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:12:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:12:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:16 smithi170 ceph-mon[102832]: osdmap e424: 12 total, 12 up, 12 in 2023-09-10T02:12:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:16 smithi082 ceph-mon[98694]: pgmap v5691: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 59 KiB/s wr, 14 op/s 2023-09-10T02:12:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:12:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:12:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:12:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:16 smithi082 ceph-mon[98694]: osdmap e424: 12 total, 12 up, 12 in 2023-09-10T02:12:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:16 smithi098 ceph-mon[103124]: pgmap v5691: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 59 KiB/s wr, 14 op/s 2023-09-10T02:12:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:12:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:12:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:12:16.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:16 smithi098 ceph-mon[103124]: osdmap e424: 12 total, 12 up, 12 in 2023-09-10T02:12:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10368.746938 secs 2023-09-10T02:12:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:18 smithi170 ceph-mon[102832]: pgmap v5693: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T02:12:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10370.171293 secs 2023-09-10T02:12:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10368.746938 secs 2023-09-10T02:12:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:18 smithi082 ceph-mon[98694]: pgmap v5693: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T02:12:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10370.171293 secs 2023-09-10T02:12:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10368.746938 secs 2023-09-10T02:12:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:18 smithi098 ceph-mon[103124]: pgmap v5693: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T02:12:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10370.171293 secs 2023-09-10T02:12:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10370.883110 secs 2023-09-10T02:12:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10370.883110 secs 2023-09-10T02:12:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10370.883110 secs 2023-09-10T02:12:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:20 smithi170 ceph-mon[102832]: pgmap v5694: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T02:12:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:20 smithi082 ceph-mon[98694]: pgmap v5694: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T02:12:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:20 smithi098 ceph-mon[103124]: pgmap v5694: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T02:12:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:22 smithi170 ceph-mon[102832]: pgmap v5695: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 716 B/s wr, 11 op/s 2023-09-10T02:12:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:22 smithi082 ceph-mon[98694]: pgmap v5695: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 716 B/s wr, 11 op/s 2023-09-10T02:12:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:22 smithi098 ceph-mon[103124]: pgmap v5695: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 716 B/s wr, 11 op/s 2023-09-10T02:12:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:12:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10375.171351 secs 2023-09-10T02:12:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:12:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10375.171351 secs 2023-09-10T02:12:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:12:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10375.171351 secs 2023-09-10T02:12:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:24 smithi170 ceph-mon[102832]: pgmap v5696: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 716 B/s wr, 11 op/s 2023-09-10T02:12:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:12:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:12:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:12:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10375.883172 secs 2023-09-10T02:12:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:24 smithi082 ceph-mon[98694]: pgmap v5696: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 716 B/s wr, 11 op/s 2023-09-10T02:12:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:12:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:12:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:12:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10375.883172 secs 2023-09-10T02:12:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:24 smithi098 ceph-mon[103124]: pgmap v5696: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 716 B/s wr, 11 op/s 2023-09-10T02:12:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:12:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:12:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:12:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10375.883172 secs 2023-09-10T02:12:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:26 smithi170 ceph-mon[102832]: pgmap v5697: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T02:12:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:26 smithi082 ceph-mon[98694]: pgmap v5697: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T02:12:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:26 smithi098 ceph-mon[103124]: pgmap v5697: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T02:12:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:28 smithi170 ceph-mon[102832]: pgmap v5698: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 10 op/s 2023-09-10T02:12:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10380.171432 secs 2023-09-10T02:12:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:28 smithi082 ceph-mon[98694]: pgmap v5698: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 10 op/s 2023-09-10T02:12:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10380.171432 secs 2023-09-10T02:12:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:28 smithi098 ceph-mon[103124]: pgmap v5698: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 10 op/s 2023-09-10T02:12:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10380.171432 secs 2023-09-10T02:12:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10380.883232 secs 2023-09-10T02:12:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10380.883232 secs 2023-09-10T02:12:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10380.883232 secs 2023-09-10T02:12:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:30 smithi170 ceph-mon[102832]: pgmap v5699: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 9 op/s 2023-09-10T02:12:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:30 smithi082 ceph-mon[98694]: pgmap v5699: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 9 op/s 2023-09-10T02:12:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:30 smithi098 ceph-mon[103124]: pgmap v5699: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 9 op/s 2023-09-10T02:12:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:32 smithi170 ceph-mon[102832]: pgmap v5700: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 118 KiB/s wr, 15 op/s 2023-09-10T02:12:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:32 smithi082 ceph-mon[98694]: pgmap v5700: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 118 KiB/s wr, 15 op/s 2023-09-10T02:12:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:32 smithi098 ceph-mon[103124]: pgmap v5700: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 118 KiB/s wr, 15 op/s 2023-09-10T02:12:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10385.171513 secs 2023-09-10T02:12:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10385.171513 secs 2023-09-10T02:12:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10385.171513 secs 2023-09-10T02:12:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:34 smithi170 ceph-mon[102832]: pgmap v5701: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T02:12:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10385.883296 secs 2023-09-10T02:12:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:34 smithi082 ceph-mon[98694]: pgmap v5701: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T02:12:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10385.883296 secs 2023-09-10T02:12:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:34 smithi098 ceph-mon[103124]: pgmap v5701: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T02:12:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10385.883296 secs 2023-09-10T02:12:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:36 smithi170 ceph-mon[102832]: pgmap v5702: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 167 KiB/s wr, 15 op/s 2023-09-10T02:12:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:36 smithi082 ceph-mon[98694]: pgmap v5702: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 167 KiB/s wr, 15 op/s 2023-09-10T02:12:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:36 smithi098 ceph-mon[103124]: pgmap v5702: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 167 KiB/s wr, 15 op/s 2023-09-10T02:12:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:38 smithi170 ceph-mon[102832]: pgmap v5703: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 118 KiB/s wr, 13 op/s 2023-09-10T02:12:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10390.171572 secs 2023-09-10T02:12:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:38 smithi082 ceph-mon[98694]: pgmap v5703: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 118 KiB/s wr, 13 op/s 2023-09-10T02:12:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10390.171572 secs 2023-09-10T02:12:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:38 smithi098 ceph-mon[103124]: pgmap v5703: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 118 KiB/s wr, 13 op/s 2023-09-10T02:12:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10390.171572 secs 2023-09-10T02:12:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10390.883361 secs 2023-09-10T02:12:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10390.883361 secs 2023-09-10T02:12:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10390.883361 secs 2023-09-10T02:12:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:40 smithi170 ceph-mon[102832]: pgmap v5704: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T02:12:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:40 smithi082 ceph-mon[98694]: pgmap v5704: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T02:12:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:40 smithi098 ceph-mon[103124]: pgmap v5704: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T02:12:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:42 smithi170 ceph-mon[102832]: pgmap v5705: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 163 KiB/s wr, 17 op/s 2023-09-10T02:12:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:42 smithi082 ceph-mon[98694]: pgmap v5705: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 163 KiB/s wr, 17 op/s 2023-09-10T02:12:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:42 smithi098 ceph-mon[103124]: pgmap v5705: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 163 KiB/s wr, 17 op/s 2023-09-10T02:12:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10395.171654 secs 2023-09-10T02:12:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10395.171654 secs 2023-09-10T02:12:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10395.171654 secs 2023-09-10T02:12:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:44 smithi170 ceph-mon[102832]: pgmap v5706: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T02:12:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10395.883428 secs 2023-09-10T02:12:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:44 smithi082 ceph-mon[98694]: pgmap v5706: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T02:12:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10395.883428 secs 2023-09-10T02:12:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:44 smithi098 ceph-mon[103124]: pgmap v5706: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T02:12:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10395.883428 secs 2023-09-10T02:12:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:46 smithi170 ceph-mon[102832]: pgmap v5707: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 162 KiB/s wr, 15 op/s 2023-09-10T02:12:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:46 smithi082 ceph-mon[98694]: pgmap v5707: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 162 KiB/s wr, 15 op/s 2023-09-10T02:12:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:46 smithi098 ceph-mon[103124]: pgmap v5707: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 162 KiB/s wr, 15 op/s 2023-09-10T02:12:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10398.747452 secs 2023-09-10T02:12:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:48 smithi170 ceph-mon[102832]: pgmap v5708: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 14 op/s 2023-09-10T02:12:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10400.171713 secs 2023-09-10T02:12:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10398.747452 secs 2023-09-10T02:12:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:48 smithi082 ceph-mon[98694]: pgmap v5708: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 14 op/s 2023-09-10T02:12:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10400.171713 secs 2023-09-10T02:12:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10398.747452 secs 2023-09-10T02:12:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:48 smithi098 ceph-mon[103124]: pgmap v5708: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 14 op/s 2023-09-10T02:12:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10400.171713 secs 2023-09-10T02:12:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10400.883486 secs 2023-09-10T02:12:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10400.883486 secs 2023-09-10T02:12:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10400.883486 secs 2023-09-10T02:12:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:50 smithi170 ceph-mon[102832]: pgmap v5709: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T02:12:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:50 smithi082 ceph-mon[98694]: pgmap v5709: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T02:12:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:50 smithi098 ceph-mon[103124]: pgmap v5709: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T02:12:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:52 smithi170 ceph-mon[102832]: pgmap v5710: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 167 KiB/s wr, 17 op/s 2023-09-10T02:12:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:12:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:12:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:12:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:52 smithi082 ceph-mon[98694]: pgmap v5710: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 167 KiB/s wr, 17 op/s 2023-09-10T02:12:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:12:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:12:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:12:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:52 smithi098 ceph-mon[103124]: pgmap v5710: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 167 KiB/s wr, 17 op/s 2023-09-10T02:12:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:12:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:12:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:12:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10403.747511 secs 2023-09-10T02:12:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10405.171771 secs 2023-09-10T02:12:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10403.747511 secs 2023-09-10T02:12:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10405.171771 secs 2023-09-10T02:12:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10403.747511 secs 2023-09-10T02:12:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10405.171771 secs 2023-09-10T02:12:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:54 smithi170 ceph-mon[102832]: pgmap v5711: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T02:12:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10405.883549 secs 2023-09-10T02:12:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:54 smithi082 ceph-mon[98694]: pgmap v5711: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T02:12:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10405.883549 secs 2023-09-10T02:12:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:54 smithi098 ceph-mon[103124]: pgmap v5711: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T02:12:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10405.883549 secs 2023-09-10T02:12:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:56 smithi170 ceph-mon[102832]: pgmap v5712: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 149 KiB/s wr, 16 op/s 2023-09-10T02:12:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:56 smithi082 ceph-mon[98694]: pgmap v5712: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 149 KiB/s wr, 16 op/s 2023-09-10T02:12:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:56 smithi098 ceph-mon[103124]: pgmap v5712: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 149 KiB/s wr, 16 op/s 2023-09-10T02:12:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10408.780121 secs 2023-09-10T02:12:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:58 smithi170 ceph-mon[102832]: pgmap v5713: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 98 KiB/s wr, 13 op/s 2023-09-10T02:12:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10410.171831 secs 2023-09-10T02:12:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10408.780121 secs 2023-09-10T02:12:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:58 smithi082 ceph-mon[98694]: pgmap v5713: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 98 KiB/s wr, 13 op/s 2023-09-10T02:12:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10410.171831 secs 2023-09-10T02:12:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10408.780121 secs 2023-09-10T02:12:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:58 smithi098 ceph-mon[103124]: pgmap v5713: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 98 KiB/s wr, 13 op/s 2023-09-10T02:12:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10410.171831 secs 2023-09-10T02:12:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:12:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10410.883610 secs 2023-09-10T02:12:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:12:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10410.883610 secs 2023-09-10T02:12:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:12:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10410.883610 secs 2023-09-10T02:13:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:00 smithi170 ceph-mon[102832]: pgmap v5714: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T02:13:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:00 smithi082 ceph-mon[98694]: pgmap v5714: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T02:13:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:00 smithi098 ceph-mon[103124]: pgmap v5714: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T02:13:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:02 smithi170 ceph-mon[102832]: pgmap v5715: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 16 op/s 2023-09-10T02:13:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:02 smithi082 ceph-mon[98694]: pgmap v5715: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 16 op/s 2023-09-10T02:13:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:02 smithi098 ceph-mon[103124]: pgmap v5715: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 16 op/s 2023-09-10T02:13:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10413.780212 secs 2023-09-10T02:13:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10415.171889 secs 2023-09-10T02:13:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10413.780212 secs 2023-09-10T02:13:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10415.171889 secs 2023-09-10T02:13:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10413.780212 secs 2023-09-10T02:13:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10415.171889 secs 2023-09-10T02:13:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:04 smithi170 ceph-mon[102832]: pgmap v5716: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 54 KiB/s wr, 10 op/s 2023-09-10T02:13:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10415.883674 secs 2023-09-10T02:13:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:04 smithi082 ceph-mon[98694]: pgmap v5716: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 54 KiB/s wr, 10 op/s 2023-09-10T02:13:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10415.883674 secs 2023-09-10T02:13:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:04 smithi098 ceph-mon[103124]: pgmap v5716: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 54 KiB/s wr, 10 op/s 2023-09-10T02:13:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10415.883674 secs 2023-09-10T02:13:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:06 smithi170 ceph-mon[102832]: pgmap v5717: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 99 KiB/s wr, 29 op/s 2023-09-10T02:13:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:06 smithi082 ceph-mon[98694]: pgmap v5717: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 99 KiB/s wr, 29 op/s 2023-09-10T02:13:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:06 smithi098 ceph-mon[103124]: pgmap v5717: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 99 KiB/s wr, 29 op/s 2023-09-10T02:13:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10418.780274 secs 2023-09-10T02:13:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:08 smithi170 ceph-mon[102832]: pgmap v5718: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 63 KiB/s wr, 25 op/s 2023-09-10T02:13:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:08 smithi170 ceph-mon[102832]: osdmap e425: 12 total, 12 up, 12 in 2023-09-10T02:13:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10420.171976 secs 2023-09-10T02:13:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10418.780274 secs 2023-09-10T02:13:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:08 smithi082 ceph-mon[98694]: pgmap v5718: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 63 KiB/s wr, 25 op/s 2023-09-10T02:13:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:08 smithi082 ceph-mon[98694]: osdmap e425: 12 total, 12 up, 12 in 2023-09-10T02:13:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10420.171976 secs 2023-09-10T02:13:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10418.780274 secs 2023-09-10T02:13:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:08 smithi098 ceph-mon[103124]: pgmap v5718: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 63 KiB/s wr, 25 op/s 2023-09-10T02:13:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:08 smithi098 ceph-mon[103124]: osdmap e425: 12 total, 12 up, 12 in 2023-09-10T02:13:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10420.171976 secs 2023-09-10T02:13:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10420.883737 secs 2023-09-10T02:13:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10420.883737 secs 2023-09-10T02:13:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10420.883737 secs 2023-09-10T02:13:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:10 smithi170 ceph-mon[102832]: pgmap v5720: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T02:13:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:10 smithi082 ceph-mon[98694]: pgmap v5720: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T02:13:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:10 smithi098 ceph-mon[103124]: pgmap v5720: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T02:13:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:12 smithi170 ceph-mon[102832]: pgmap v5721: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T02:13:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:12 smithi082 ceph-mon[98694]: pgmap v5721: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T02:13:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:12 smithi098 ceph-mon[103124]: pgmap v5721: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T02:13:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10423.780349 secs 2023-09-10T02:13:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10425.172034 secs 2023-09-10T02:13:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10423.780349 secs 2023-09-10T02:13:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10425.172034 secs 2023-09-10T02:13:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10423.780349 secs 2023-09-10T02:13:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10425.172034 secs 2023-09-10T02:13:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:14 smithi170 ceph-mon[102832]: pgmap v5722: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T02:13:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10425.883799 secs 2023-09-10T02:13:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:14 smithi082 ceph-mon[98694]: pgmap v5722: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T02:13:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10425.883799 secs 2023-09-10T02:13:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:14 smithi098 ceph-mon[103124]: pgmap v5722: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T02:13:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10425.883799 secs 2023-09-10T02:13:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:16 smithi170 ceph-mon[102832]: pgmap v5723: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T02:13:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:16 smithi170 ceph-mon[102832]: osdmap e426: 12 total, 12 up, 12 in 2023-09-10T02:13:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:16 smithi082 ceph-mon[98694]: pgmap v5723: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T02:13:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:16 smithi082 ceph-mon[98694]: osdmap e426: 12 total, 12 up, 12 in 2023-09-10T02:13:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:16 smithi098 ceph-mon[103124]: pgmap v5723: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T02:13:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:16 smithi098 ceph-mon[103124]: osdmap e426: 12 total, 12 up, 12 in 2023-09-10T02:13:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:13:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:13:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:13:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:13:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:13:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:13:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:13:17.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:13:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:13:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:13:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:13:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:13:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10428.780411 secs 2023-09-10T02:13:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:18 smithi170 ceph-mon[102832]: pgmap v5725: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T02:13:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10430.172093 secs 2023-09-10T02:13:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10428.780411 secs 2023-09-10T02:13:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:18 smithi082 ceph-mon[98694]: pgmap v5725: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T02:13:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10430.172093 secs 2023-09-10T02:13:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10428.780411 secs 2023-09-10T02:13:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:18 smithi098 ceph-mon[103124]: pgmap v5725: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T02:13:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10430.172093 secs 2023-09-10T02:13:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10430.883862 secs 2023-09-10T02:13:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10430.883862 secs 2023-09-10T02:13:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10430.883862 secs 2023-09-10T02:13:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:20 smithi170 ceph-mon[102832]: pgmap v5726: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T02:13:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:20 smithi082 ceph-mon[98694]: pgmap v5726: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T02:13:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:20 smithi098 ceph-mon[103124]: pgmap v5726: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T02:13:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:22 smithi170 ceph-mon[102832]: pgmap v5727: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T02:13:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:22 smithi082 ceph-mon[98694]: pgmap v5727: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T02:13:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:22 smithi098 ceph-mon[103124]: pgmap v5727: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T02:13:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:13:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10435.172150 secs 2023-09-10T02:13:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:13:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10435.172150 secs 2023-09-10T02:13:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:13:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10435.172150 secs 2023-09-10T02:13:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:24 smithi170 ceph-mon[102832]: pgmap v5728: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T02:13:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:13:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:13:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:13:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10435.883926 secs 2023-09-10T02:13:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:24 smithi082 ceph-mon[98694]: pgmap v5728: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T02:13:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:13:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:13:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:13:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10435.883926 secs 2023-09-10T02:13:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:24 smithi098 ceph-mon[103124]: pgmap v5728: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-09-10T02:13:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:13:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:13:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:13:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10435.883926 secs 2023-09-10T02:13:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:26 smithi170 ceph-mon[102832]: pgmap v5729: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T02:13:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:26 smithi082 ceph-mon[98694]: pgmap v5729: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T02:13:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:26 smithi098 ceph-mon[103124]: pgmap v5729: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T02:13:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:28 smithi170 ceph-mon[102832]: pgmap v5730: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 63 KiB/s wr, 10 op/s 2023-09-10T02:13:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10440.172203 secs 2023-09-10T02:13:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:28 smithi082 ceph-mon[98694]: pgmap v5730: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 63 KiB/s wr, 10 op/s 2023-09-10T02:13:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10440.172203 secs 2023-09-10T02:13:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:28 smithi098 ceph-mon[103124]: pgmap v5730: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 63 KiB/s wr, 10 op/s 2023-09-10T02:13:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10440.172203 secs 2023-09-10T02:13:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10440.883987 secs 2023-09-10T02:13:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10440.883987 secs 2023-09-10T02:13:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10440.883987 secs 2023-09-10T02:13:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:30 smithi170 ceph-mon[102832]: pgmap v5731: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T02:13:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:30 smithi082 ceph-mon[98694]: pgmap v5731: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T02:13:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:30 smithi098 ceph-mon[103124]: pgmap v5731: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T02:13:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:32 smithi170 ceph-mon[102832]: pgmap v5732: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 144 KiB/s wr, 15 op/s 2023-09-10T02:13:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:32 smithi082 ceph-mon[98694]: pgmap v5732: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 144 KiB/s wr, 15 op/s 2023-09-10T02:13:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:32 smithi098 ceph-mon[103124]: pgmap v5732: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 144 KiB/s wr, 15 op/s 2023-09-10T02:13:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10445.172262 secs 2023-09-10T02:13:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10445.172262 secs 2023-09-10T02:13:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10445.172262 secs 2023-09-10T02:13:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:34 smithi170 ceph-mon[102832]: pgmap v5733: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T02:13:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10445.884048 secs 2023-09-10T02:13:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:34 smithi082 ceph-mon[98694]: pgmap v5733: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T02:13:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10445.884048 secs 2023-09-10T02:13:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:34 smithi098 ceph-mon[103124]: pgmap v5733: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T02:13:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10445.884048 secs 2023-09-10T02:13:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:36 smithi170 ceph-mon[102832]: pgmap v5734: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 166 KiB/s wr, 16 op/s 2023-09-10T02:13:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:36 smithi082 ceph-mon[98694]: pgmap v5734: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 166 KiB/s wr, 16 op/s 2023-09-10T02:13:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:36 smithi098 ceph-mon[103124]: pgmap v5734: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 166 KiB/s wr, 16 op/s 2023-09-10T02:13:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:38 smithi170 ceph-mon[102832]: pgmap v5735: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 132 KiB/s wr, 13 op/s 2023-09-10T02:13:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10450.172322 secs 2023-09-10T02:13:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:38 smithi082 ceph-mon[98694]: pgmap v5735: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 132 KiB/s wr, 13 op/s 2023-09-10T02:13:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10450.172322 secs 2023-09-10T02:13:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:38 smithi098 ceph-mon[103124]: pgmap v5735: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 132 KiB/s wr, 13 op/s 2023-09-10T02:13:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10450.172322 secs 2023-09-10T02:13:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10450.884110 secs 2023-09-10T02:13:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10450.884110 secs 2023-09-10T02:13:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10450.884110 secs 2023-09-10T02:13:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:40 smithi170 ceph-mon[102832]: pgmap v5736: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 132 KiB/s wr, 12 op/s 2023-09-10T02:13:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:40 smithi082 ceph-mon[98694]: pgmap v5736: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 132 KiB/s wr, 12 op/s 2023-09-10T02:13:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:40 smithi098 ceph-mon[103124]: pgmap v5736: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 132 KiB/s wr, 12 op/s 2023-09-10T02:13:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:42 smithi170 ceph-mon[102832]: pgmap v5737: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 227 KiB/s wr, 18 op/s 2023-09-10T02:13:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:42 smithi082 ceph-mon[98694]: pgmap v5737: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 227 KiB/s wr, 18 op/s 2023-09-10T02:13:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:42 smithi098 ceph-mon[103124]: pgmap v5737: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 227 KiB/s wr, 18 op/s 2023-09-10T02:13:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10455.172376 secs 2023-09-10T02:13:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10455.172376 secs 2023-09-10T02:13:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10455.172376 secs 2023-09-10T02:13:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:44 smithi170 ceph-mon[102832]: pgmap v5738: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 141 KiB/s wr, 12 op/s 2023-09-10T02:13:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10455.884175 secs 2023-09-10T02:13:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:44 smithi082 ceph-mon[98694]: pgmap v5738: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 141 KiB/s wr, 12 op/s 2023-09-10T02:13:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10455.884175 secs 2023-09-10T02:13:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:44 smithi098 ceph-mon[103124]: pgmap v5738: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 141 KiB/s wr, 12 op/s 2023-09-10T02:13:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10455.884175 secs 2023-09-10T02:13:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:46 smithi170 ceph-mon[102832]: pgmap v5739: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 166 KiB/s wr, 16 op/s 2023-09-10T02:13:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:46 smithi082 ceph-mon[98694]: pgmap v5739: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 166 KiB/s wr, 16 op/s 2023-09-10T02:13:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:46 smithi098 ceph-mon[103124]: pgmap v5739: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 166 KiB/s wr, 16 op/s 2023-09-10T02:13:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10458.780871 secs 2023-09-10T02:13:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:48 smithi170 ceph-mon[102832]: pgmap v5740: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T02:13:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10460.172436 secs 2023-09-10T02:13:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10458.780871 secs 2023-09-10T02:13:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:48 smithi082 ceph-mon[98694]: pgmap v5740: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T02:13:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10460.172436 secs 2023-09-10T02:13:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10458.780871 secs 2023-09-10T02:13:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:48 smithi098 ceph-mon[103124]: pgmap v5740: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T02:13:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10460.172436 secs 2023-09-10T02:13:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10460.884239 secs 2023-09-10T02:13:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10460.884239 secs 2023-09-10T02:13:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10460.884239 secs 2023-09-10T02:13:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:50 smithi170 ceph-mon[102832]: pgmap v5741: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T02:13:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:50 smithi082 ceph-mon[98694]: pgmap v5741: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T02:13:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:50 smithi098 ceph-mon[103124]: pgmap v5741: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T02:13:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:52 smithi170 ceph-mon[102832]: pgmap v5742: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 193 KiB/s wr, 17 op/s 2023-09-10T02:13:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:13:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:13:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:13:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:52 smithi082 ceph-mon[98694]: pgmap v5742: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 193 KiB/s wr, 17 op/s 2023-09-10T02:13:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:13:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:13:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:13:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:52 smithi098 ceph-mon[103124]: pgmap v5742: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 193 KiB/s wr, 17 op/s 2023-09-10T02:13:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:13:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:13:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:13:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10463.780929 secs 2023-09-10T02:13:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10465.172520 secs 2023-09-10T02:13:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10463.780929 secs 2023-09-10T02:13:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10465.172520 secs 2023-09-10T02:13:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10463.780929 secs 2023-09-10T02:13:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10465.172520 secs 2023-09-10T02:13:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:54 smithi170 ceph-mon[102832]: pgmap v5743: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 98 KiB/s wr, 11 op/s 2023-09-10T02:13:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10465.884305 secs 2023-09-10T02:13:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:54 smithi082 ceph-mon[98694]: pgmap v5743: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 98 KiB/s wr, 11 op/s 2023-09-10T02:13:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10465.884305 secs 2023-09-10T02:13:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:54 smithi098 ceph-mon[103124]: pgmap v5743: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 98 KiB/s wr, 11 op/s 2023-09-10T02:13:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10465.884305 secs 2023-09-10T02:13:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:56 smithi170 ceph-mon[102832]: pgmap v5744: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 98 KiB/s wr, 16 op/s 2023-09-10T02:13:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:56 smithi082 ceph-mon[98694]: pgmap v5744: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 98 KiB/s wr, 16 op/s 2023-09-10T02:13:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:56 smithi098 ceph-mon[103124]: pgmap v5744: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 98 KiB/s wr, 16 op/s 2023-09-10T02:13:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10468.780990 secs 2023-09-10T02:13:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:58 smithi170 ceph-mon[102832]: pgmap v5745: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 12 op/s 2023-09-10T02:13:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10470.172602 secs 2023-09-10T02:13:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10468.780990 secs 2023-09-10T02:13:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:58 smithi082 ceph-mon[98694]: pgmap v5745: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 12 op/s 2023-09-10T02:13:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10470.172602 secs 2023-09-10T02:13:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10468.780990 secs 2023-09-10T02:13:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:58 smithi098 ceph-mon[103124]: pgmap v5745: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 12 op/s 2023-09-10T02:13:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10470.172602 secs 2023-09-10T02:13:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:13:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10470.884370 secs 2023-09-10T02:13:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:13:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10470.884370 secs 2023-09-10T02:13:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:13:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10470.884370 secs 2023-09-10T02:14:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:00 smithi170 ceph-mon[102832]: pgmap v5746: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T02:14:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:00 smithi082 ceph-mon[98694]: pgmap v5746: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T02:14:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:00 smithi098 ceph-mon[103124]: pgmap v5746: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T02:14:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:02 smithi170 ceph-mon[102832]: pgmap v5747: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 91 KiB/s wr, 16 op/s 2023-09-10T02:14:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:02 smithi082 ceph-mon[98694]: pgmap v5747: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 91 KiB/s wr, 16 op/s 2023-09-10T02:14:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:02 smithi098 ceph-mon[103124]: pgmap v5747: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 91 KiB/s wr, 16 op/s 2023-09-10T02:14:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10473.781049 secs 2023-09-10T02:14:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10475.172663 secs 2023-09-10T02:14:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10473.781049 secs 2023-09-10T02:14:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10475.172663 secs 2023-09-10T02:14:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10473.781049 secs 2023-09-10T02:14:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10475.172663 secs 2023-09-10T02:14:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:04 smithi170 ceph-mon[102832]: pgmap v5748: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 KiB/s wr, 9 op/s 2023-09-10T02:14:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10475.884434 secs 2023-09-10T02:14:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:04 smithi082 ceph-mon[98694]: pgmap v5748: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 KiB/s wr, 9 op/s 2023-09-10T02:14:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10475.884434 secs 2023-09-10T02:14:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:04 smithi098 ceph-mon[103124]: pgmap v5748: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 KiB/s wr, 9 op/s 2023-09-10T02:14:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10475.884434 secs 2023-09-10T02:14:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:06 smithi170 ceph-mon[102832]: pgmap v5749: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 63 KiB/s wr, 29 op/s 2023-09-10T02:14:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:06 smithi082 ceph-mon[98694]: pgmap v5749: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 63 KiB/s wr, 29 op/s 2023-09-10T02:14:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:06 smithi098 ceph-mon[103124]: pgmap v5749: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 63 KiB/s wr, 29 op/s 2023-09-10T02:14:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10478.781108 secs 2023-09-10T02:14:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:08 smithi170 ceph-mon[102832]: pgmap v5750: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 63 KiB/s wr, 24 op/s 2023-09-10T02:14:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:08 smithi170 ceph-mon[102832]: osdmap e427: 12 total, 12 up, 12 in 2023-09-10T02:14:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10480.172765 secs 2023-09-10T02:14:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10478.781108 secs 2023-09-10T02:14:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:08 smithi082 ceph-mon[98694]: pgmap v5750: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 63 KiB/s wr, 24 op/s 2023-09-10T02:14:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:08 smithi082 ceph-mon[98694]: osdmap e427: 12 total, 12 up, 12 in 2023-09-10T02:14:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10480.172765 secs 2023-09-10T02:14:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10478.781108 secs 2023-09-10T02:14:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:08 smithi098 ceph-mon[103124]: pgmap v5750: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 63 KiB/s wr, 24 op/s 2023-09-10T02:14:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:08 smithi098 ceph-mon[103124]: osdmap e427: 12 total, 12 up, 12 in 2023-09-10T02:14:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10480.172765 secs 2023-09-10T02:14:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10480.884484 secs 2023-09-10T02:14:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10480.884484 secs 2023-09-10T02:14:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10480.884484 secs 2023-09-10T02:14:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:10 smithi170 ceph-mon[102832]: pgmap v5752: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 28 op/s 2023-09-10T02:14:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:10 smithi082 ceph-mon[98694]: pgmap v5752: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 28 op/s 2023-09-10T02:14:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:10 smithi098 ceph-mon[103124]: pgmap v5752: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 28 op/s 2023-09-10T02:14:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:12 smithi170 ceph-mon[102832]: pgmap v5753: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 29 op/s 2023-09-10T02:14:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:12 smithi082 ceph-mon[98694]: pgmap v5753: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 29 op/s 2023-09-10T02:14:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:12 smithi098 ceph-mon[103124]: pgmap v5753: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 29 op/s 2023-09-10T02:14:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10483.781197 secs 2023-09-10T02:14:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10485.172842 secs 2023-09-10T02:14:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10483.781197 secs 2023-09-10T02:14:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10485.172842 secs 2023-09-10T02:14:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10483.781197 secs 2023-09-10T02:14:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10485.172842 secs 2023-09-10T02:14:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:14 smithi170 ceph-mon[102832]: pgmap v5754: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 29 op/s 2023-09-10T02:14:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10485.884546 secs 2023-09-10T02:14:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:14 smithi082 ceph-mon[98694]: pgmap v5754: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 29 op/s 2023-09-10T02:14:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10485.884546 secs 2023-09-10T02:14:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:14 smithi098 ceph-mon[103124]: pgmap v5754: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 29 op/s 2023-09-10T02:14:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10485.884546 secs 2023-09-10T02:14:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:16 smithi170 ceph-mon[102832]: pgmap v5755: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T02:14:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:16 smithi170 ceph-mon[102832]: osdmap e428: 12 total, 12 up, 12 in 2023-09-10T02:14:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:16 smithi082 ceph-mon[98694]: pgmap v5755: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T02:14:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:16 smithi082 ceph-mon[98694]: osdmap e428: 12 total, 12 up, 12 in 2023-09-10T02:14:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:16 smithi098 ceph-mon[103124]: pgmap v5755: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T02:14:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:16 smithi098 ceph-mon[103124]: osdmap e428: 12 total, 12 up, 12 in 2023-09-10T02:14:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:14:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:14:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:14:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10488.781257 secs 2023-09-10T02:14:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:18 smithi170 ceph-mon[102832]: pgmap v5757: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T02:14:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:14:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:14:18.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:14:18.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10490.172930 secs 2023-09-10T02:14:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10488.781257 secs 2023-09-10T02:14:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:18 smithi082 ceph-mon[98694]: pgmap v5757: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T02:14:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:14:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:14:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:14:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10490.172930 secs 2023-09-10T02:14:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10488.781257 secs 2023-09-10T02:14:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:18 smithi098 ceph-mon[103124]: pgmap v5757: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T02:14:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:14:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:14:18.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:14:18.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10490.172930 secs 2023-09-10T02:14:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10490.884612 secs 2023-09-10T02:14:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10490.884612 secs 2023-09-10T02:14:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10490.884612 secs 2023-09-10T02:14:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:20 smithi170 ceph-mon[102832]: pgmap v5758: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T02:14:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:20 smithi082 ceph-mon[98694]: pgmap v5758: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T02:14:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:20 smithi098 ceph-mon[103124]: pgmap v5758: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 12 op/s 2023-09-10T02:14:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:22 smithi170 ceph-mon[102832]: pgmap v5759: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 11 op/s 2023-09-10T02:14:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:22 smithi082 ceph-mon[98694]: pgmap v5759: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 11 op/s 2023-09-10T02:14:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:22 smithi098 ceph-mon[103124]: pgmap v5759: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 11 op/s 2023-09-10T02:14:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:14:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10495.172988 secs 2023-09-10T02:14:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:14:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10495.172988 secs 2023-09-10T02:14:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:14:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10495.172988 secs 2023-09-10T02:14:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:24 smithi170 ceph-mon[102832]: pgmap v5760: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 11 op/s 2023-09-10T02:14:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:14:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:14:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:14:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10495.884675 secs 2023-09-10T02:14:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:24 smithi082 ceph-mon[98694]: pgmap v5760: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 11 op/s 2023-09-10T02:14:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:14:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:14:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:14:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10495.884675 secs 2023-09-10T02:14:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:24 smithi098 ceph-mon[103124]: pgmap v5760: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 11 op/s 2023-09-10T02:14:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:14:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:14:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:14:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10495.884675 secs 2023-09-10T02:14:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:26 smithi170 ceph-mon[102832]: pgmap v5761: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T02:14:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:26 smithi082 ceph-mon[98694]: pgmap v5761: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T02:14:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:26 smithi098 ceph-mon[103124]: pgmap v5761: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T02:14:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:28 smithi170 ceph-mon[102832]: pgmap v5762: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 10 op/s 2023-09-10T02:14:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10500.173047 secs 2023-09-10T02:14:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:28 smithi082 ceph-mon[98694]: pgmap v5762: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 10 op/s 2023-09-10T02:14:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10500.173047 secs 2023-09-10T02:14:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:28 smithi098 ceph-mon[103124]: pgmap v5762: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 10 op/s 2023-09-10T02:14:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10500.173047 secs 2023-09-10T02:14:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10500.884740 secs 2023-09-10T02:14:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10500.884740 secs 2023-09-10T02:14:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10500.884740 secs 2023-09-10T02:14:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:30 smithi170 ceph-mon[102832]: pgmap v5763: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T02:14:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:30 smithi082 ceph-mon[98694]: pgmap v5763: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T02:14:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:30 smithi098 ceph-mon[103124]: pgmap v5763: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T02:14:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:32 smithi170 ceph-mon[102832]: pgmap v5764: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 124 KiB/s wr, 16 op/s 2023-09-10T02:14:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:32 smithi082 ceph-mon[98694]: pgmap v5764: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 124 KiB/s wr, 16 op/s 2023-09-10T02:14:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:32 smithi098 ceph-mon[103124]: pgmap v5764: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 124 KiB/s wr, 16 op/s 2023-09-10T02:14:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10505.173125 secs 2023-09-10T02:14:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10505.173125 secs 2023-09-10T02:14:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10505.173125 secs 2023-09-10T02:14:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:34 smithi170 ceph-mon[102832]: pgmap v5765: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T02:14:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10505.884803 secs 2023-09-10T02:14:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:34 smithi082 ceph-mon[98694]: pgmap v5765: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T02:14:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10505.884803 secs 2023-09-10T02:14:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:34 smithi098 ceph-mon[103124]: pgmap v5765: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T02:14:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10505.884803 secs 2023-09-10T02:14:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:36 smithi170 ceph-mon[102832]: pgmap v5766: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 179 KiB/s wr, 17 op/s 2023-09-10T02:14:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:36 smithi082 ceph-mon[98694]: pgmap v5766: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 179 KiB/s wr, 17 op/s 2023-09-10T02:14:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:36 smithi098 ceph-mon[103124]: pgmap v5766: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 179 KiB/s wr, 17 op/s 2023-09-10T02:14:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:38 smithi170 ceph-mon[102832]: pgmap v5767: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T02:14:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10510.173217 secs 2023-09-10T02:14:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:38 smithi082 ceph-mon[98694]: pgmap v5767: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T02:14:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10510.173217 secs 2023-09-10T02:14:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:38 smithi098 ceph-mon[103124]: pgmap v5767: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T02:14:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10510.173217 secs 2023-09-10T02:14:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10510.884867 secs 2023-09-10T02:14:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10510.884867 secs 2023-09-10T02:14:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10510.884867 secs 2023-09-10T02:14:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:40 smithi170 ceph-mon[102832]: pgmap v5768: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T02:14:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:40 smithi082 ceph-mon[98694]: pgmap v5768: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T02:14:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:40 smithi098 ceph-mon[103124]: pgmap v5768: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T02:14:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:42 smithi170 ceph-mon[102832]: pgmap v5769: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 159 KiB/s wr, 17 op/s 2023-09-10T02:14:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:42 smithi082 ceph-mon[98694]: pgmap v5769: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 159 KiB/s wr, 17 op/s 2023-09-10T02:14:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:42 smithi098 ceph-mon[103124]: pgmap v5769: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 159 KiB/s wr, 17 op/s 2023-09-10T02:14:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10515.173273 secs 2023-09-10T02:14:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10515.173273 secs 2023-09-10T02:14:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10515.173273 secs 2023-09-10T02:14:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:44 smithi170 ceph-mon[102832]: pgmap v5770: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T02:14:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10515.884931 secs 2023-09-10T02:14:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:44 smithi082 ceph-mon[98694]: pgmap v5770: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T02:14:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10515.884931 secs 2023-09-10T02:14:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:44 smithi098 ceph-mon[103124]: pgmap v5770: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T02:14:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10515.884931 secs 2023-09-10T02:14:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:46 smithi170 ceph-mon[102832]: pgmap v5771: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 171 KiB/s wr, 17 op/s 2023-09-10T02:14:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:46 smithi082 ceph-mon[98694]: pgmap v5771: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 171 KiB/s wr, 17 op/s 2023-09-10T02:14:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:46 smithi098 ceph-mon[103124]: pgmap v5771: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 171 KiB/s wr, 17 op/s 2023-09-10T02:14:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10518.781736 secs 2023-09-10T02:14:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:48 smithi170 ceph-mon[102832]: pgmap v5772: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T02:14:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10520.173335 secs 2023-09-10T02:14:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10518.781736 secs 2023-09-10T02:14:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:48 smithi082 ceph-mon[98694]: pgmap v5772: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T02:14:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10520.173335 secs 2023-09-10T02:14:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10518.781736 secs 2023-09-10T02:14:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:48 smithi098 ceph-mon[103124]: pgmap v5772: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T02:14:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10520.173335 secs 2023-09-10T02:14:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10520.884994 secs 2023-09-10T02:14:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10520.884994 secs 2023-09-10T02:14:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10520.884994 secs 2023-09-10T02:14:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:50 smithi170 ceph-mon[102832]: pgmap v5773: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T02:14:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:50 smithi082 ceph-mon[98694]: pgmap v5773: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T02:14:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:50 smithi098 ceph-mon[103124]: pgmap v5773: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T02:14:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:52 smithi170 ceph-mon[102832]: pgmap v5774: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 163 KiB/s wr, 17 op/s 2023-09-10T02:14:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:14:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:14:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:14:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:52 smithi082 ceph-mon[98694]: pgmap v5774: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 163 KiB/s wr, 17 op/s 2023-09-10T02:14:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:14:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:14:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:14:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:52 smithi098 ceph-mon[103124]: pgmap v5774: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 163 KiB/s wr, 17 op/s 2023-09-10T02:14:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:14:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:14:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:14:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10523.781780 secs 2023-09-10T02:14:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10525.173393 secs 2023-09-10T02:14:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10523.781780 secs 2023-09-10T02:14:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10525.173393 secs 2023-09-10T02:14:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10523.781780 secs 2023-09-10T02:14:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10525.173393 secs 2023-09-10T02:14:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:54 smithi170 ceph-mon[102832]: pgmap v5775: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T02:14:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10525.885059 secs 2023-09-10T02:14:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:54 smithi082 ceph-mon[98694]: pgmap v5775: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T02:14:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10525.885059 secs 2023-09-10T02:14:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:54 smithi098 ceph-mon[103124]: pgmap v5775: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T02:14:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10525.885059 secs 2023-09-10T02:14:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:56 smithi170 ceph-mon[102832]: pgmap v5776: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 160 KiB/s wr, 17 op/s 2023-09-10T02:14:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:56 smithi082 ceph-mon[98694]: pgmap v5776: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 160 KiB/s wr, 17 op/s 2023-09-10T02:14:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:56 smithi098 ceph-mon[103124]: pgmap v5776: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 160 KiB/s wr, 17 op/s 2023-09-10T02:14:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10528.781861 secs 2023-09-10T02:14:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:58 smithi170 ceph-mon[102832]: pgmap v5777: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T02:14:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10530.173450 secs 2023-09-10T02:14:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10528.781861 secs 2023-09-10T02:14:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:58 smithi082 ceph-mon[98694]: pgmap v5777: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T02:14:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10530.173450 secs 2023-09-10T02:14:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10528.781861 secs 2023-09-10T02:14:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:58 smithi098 ceph-mon[103124]: pgmap v5777: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T02:14:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10530.173450 secs 2023-09-10T02:14:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:14:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10530.885123 secs 2023-09-10T02:14:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:14:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10530.885123 secs 2023-09-10T02:14:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:14:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10530.885123 secs 2023-09-10T02:15:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:00 smithi170 ceph-mon[102832]: pgmap v5778: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T02:15:00.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:00 smithi082 ceph-mon[98694]: pgmap v5778: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T02:15:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:00 smithi098 ceph-mon[103124]: pgmap v5778: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T02:15:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:02 smithi170 ceph-mon[102832]: pgmap v5779: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 109 KiB/s wr, 16 op/s 2023-09-10T02:15:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:02 smithi082 ceph-mon[98694]: pgmap v5779: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 109 KiB/s wr, 16 op/s 2023-09-10T02:15:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:02 smithi098 ceph-mon[103124]: pgmap v5779: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 109 KiB/s wr, 16 op/s 2023-09-10T02:15:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10533.781972 secs 2023-09-10T02:15:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10535.173497 secs 2023-09-10T02:15:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10533.781972 secs 2023-09-10T02:15:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10535.173497 secs 2023-09-10T02:15:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10533.781972 secs 2023-09-10T02:15:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10535.173497 secs 2023-09-10T02:15:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:04 smithi170 ceph-mon[102832]: pgmap v5780: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 10 op/s 2023-09-10T02:15:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10535.885188 secs 2023-09-10T02:15:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:04 smithi082 ceph-mon[98694]: pgmap v5780: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 10 op/s 2023-09-10T02:15:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10535.885188 secs 2023-09-10T02:15:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:04 smithi098 ceph-mon[103124]: pgmap v5780: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 10 op/s 2023-09-10T02:15:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10535.885188 secs 2023-09-10T02:15:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:06 smithi170 ceph-mon[102832]: pgmap v5781: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 107 KiB/s wr, 29 op/s 2023-09-10T02:15:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:06 smithi082 ceph-mon[98694]: pgmap v5781: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 107 KiB/s wr, 29 op/s 2023-09-10T02:15:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:06 smithi098 ceph-mon[103124]: pgmap v5781: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 107 KiB/s wr, 29 op/s 2023-09-10T02:15:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10538.782071 secs 2023-09-10T02:15:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:08 smithi170 ceph-mon[102832]: pgmap v5782: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 64 KiB/s wr, 24 op/s 2023-09-10T02:15:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:08 smithi170 ceph-mon[102832]: osdmap e429: 12 total, 12 up, 12 in 2023-09-10T02:15:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10540.173515 secs 2023-09-10T02:15:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10538.782071 secs 2023-09-10T02:15:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:08 smithi082 ceph-mon[98694]: pgmap v5782: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 64 KiB/s wr, 24 op/s 2023-09-10T02:15:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:08 smithi082 ceph-mon[98694]: osdmap e429: 12 total, 12 up, 12 in 2023-09-10T02:15:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10540.173515 secs 2023-09-10T02:15:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10538.782071 secs 2023-09-10T02:15:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:08 smithi098 ceph-mon[103124]: pgmap v5782: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 64 KiB/s wr, 24 op/s 2023-09-10T02:15:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:08 smithi098 ceph-mon[103124]: osdmap e429: 12 total, 12 up, 12 in 2023-09-10T02:15:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10540.173515 secs 2023-09-10T02:15:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10540.885276 secs 2023-09-10T02:15:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10540.885276 secs 2023-09-10T02:15:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10540.885276 secs 2023-09-10T02:15:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:10 smithi170 ceph-mon[102832]: pgmap v5784: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 28 op/s 2023-09-10T02:15:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:10 smithi082 ceph-mon[98694]: pgmap v5784: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 28 op/s 2023-09-10T02:15:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:10 smithi098 ceph-mon[103124]: pgmap v5784: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 28 op/s 2023-09-10T02:15:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:12 smithi170 ceph-mon[102832]: pgmap v5785: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 28 op/s 2023-09-10T02:15:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:12 smithi082 ceph-mon[98694]: pgmap v5785: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 28 op/s 2023-09-10T02:15:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:12 smithi098 ceph-mon[103124]: pgmap v5785: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 28 op/s 2023-09-10T02:15:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10543.782132 secs 2023-09-10T02:15:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10545.173599 secs 2023-09-10T02:15:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10543.782132 secs 2023-09-10T02:15:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10545.173599 secs 2023-09-10T02:15:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10543.782132 secs 2023-09-10T02:15:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10545.173599 secs 2023-09-10T02:15:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:14 smithi170 ceph-mon[102832]: pgmap v5786: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 28 op/s 2023-09-10T02:15:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10545.885340 secs 2023-09-10T02:15:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:14 smithi082 ceph-mon[98694]: pgmap v5786: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 28 op/s 2023-09-10T02:15:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10545.885340 secs 2023-09-10T02:15:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:14 smithi098 ceph-mon[103124]: pgmap v5786: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 28 op/s 2023-09-10T02:15:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10545.885340 secs 2023-09-10T02:15:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:16 smithi170 ceph-mon[102832]: pgmap v5787: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-10T02:15:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:16 smithi170 ceph-mon[102832]: osdmap e430: 12 total, 12 up, 12 in 2023-09-10T02:15:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:16 smithi082 ceph-mon[98694]: pgmap v5787: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-10T02:15:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:16 smithi082 ceph-mon[98694]: osdmap e430: 12 total, 12 up, 12 in 2023-09-10T02:15:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:16 smithi098 ceph-mon[103124]: pgmap v5787: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-10T02:15:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:16 smithi098 ceph-mon[103124]: osdmap e430: 12 total, 12 up, 12 in 2023-09-10T02:15:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10548.782252 secs 2023-09-10T02:15:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:18 smithi170 ceph-mon[102832]: pgmap v5789: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 12 op/s 2023-09-10T02:15:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10550.173685 secs 2023-09-10T02:15:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:15:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10548.782252 secs 2023-09-10T02:15:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:18 smithi082 ceph-mon[98694]: pgmap v5789: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 12 op/s 2023-09-10T02:15:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10550.173685 secs 2023-09-10T02:15:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:15:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10548.782252 secs 2023-09-10T02:15:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:18 smithi098 ceph-mon[103124]: pgmap v5789: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 12 op/s 2023-09-10T02:15:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10550.173685 secs 2023-09-10T02:15:18.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:15:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10550.885407 secs 2023-09-10T02:15:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10550.885407 secs 2023-09-10T02:15:19.734 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10550.885407 secs 2023-09-10T02:15:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:20 smithi170 ceph-mon[102832]: pgmap v5790: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-10T02:15:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:20 smithi082 ceph-mon[98694]: pgmap v5790: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-10T02:15:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:20 smithi098 ceph-mon[103124]: pgmap v5790: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-10T02:15:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:22 smithi170 ceph-mon[102832]: pgmap v5791: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T02:15:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:15:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:15:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:15:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:15:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:22 smithi082 ceph-mon[98694]: pgmap v5791: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T02:15:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:15:22.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:15:22.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:15:22.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:15:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:22 smithi098 ceph-mon[103124]: pgmap v5791: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T02:15:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:15:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:15:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:15:23.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:15:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:15:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10555.173740 secs 2023-09-10T02:15:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:15:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10555.173740 secs 2023-09-10T02:15:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:15:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10555.173740 secs 2023-09-10T02:15:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:24 smithi098 ceph-mon[103124]: pgmap v5792: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T02:15:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:15:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:15:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:15:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10555.885475 secs 2023-09-10T02:15:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:15:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:15:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:15:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:15:24.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:15:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:24 smithi170 ceph-mon[102832]: pgmap v5792: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T02:15:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:15:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:15:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:15:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10555.885475 secs 2023-09-10T02:15:24.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:15:24.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:15:24.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:15:24.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:15:24.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:15:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:24 smithi082 ceph-mon[98694]: pgmap v5792: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T02:15:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:15:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:15:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:15:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10555.885475 secs 2023-09-10T02:15:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:15:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:15:24.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:15:24.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:15:24.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:15:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:26 smithi170 ceph-mon[102832]: pgmap v5793: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 89 KiB/s wr, 12 op/s 2023-09-10T02:15:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:26 smithi082 ceph-mon[98694]: pgmap v5793: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 89 KiB/s wr, 12 op/s 2023-09-10T02:15:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:26 smithi098 ceph-mon[103124]: pgmap v5793: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 89 KiB/s wr, 12 op/s 2023-09-10T02:15:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:28 smithi170 ceph-mon[102832]: pgmap v5794: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T02:15:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10560.173797 secs 2023-09-10T02:15:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:28 smithi082 ceph-mon[98694]: pgmap v5794: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T02:15:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10560.173797 secs 2023-09-10T02:15:29.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:28 smithi098 ceph-mon[103124]: pgmap v5794: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T02:15:29.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10560.173797 secs 2023-09-10T02:15:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10560.885490 secs 2023-09-10T02:15:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10560.885490 secs 2023-09-10T02:15:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10560.885490 secs 2023-09-10T02:15:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:30 smithi170 ceph-mon[102832]: pgmap v5795: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T02:15:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:30 smithi082 ceph-mon[98694]: pgmap v5795: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T02:15:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:30 smithi098 ceph-mon[103124]: pgmap v5795: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T02:15:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:32 smithi170 ceph-mon[102832]: pgmap v5796: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 132 KiB/s wr, 16 op/s 2023-09-10T02:15:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:32 smithi082 ceph-mon[98694]: pgmap v5796: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 132 KiB/s wr, 16 op/s 2023-09-10T02:15:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:32 smithi098 ceph-mon[103124]: pgmap v5796: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 132 KiB/s wr, 16 op/s 2023-09-10T02:15:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10565.173878 secs 2023-09-10T02:15:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10565.173878 secs 2023-09-10T02:15:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10565.173878 secs 2023-09-10T02:15:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:34 smithi170 ceph-mon[102832]: pgmap v5797: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T02:15:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10565.885556 secs 2023-09-10T02:15:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:34 smithi082 ceph-mon[98694]: pgmap v5797: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T02:15:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10565.885556 secs 2023-09-10T02:15:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:34 smithi098 ceph-mon[103124]: pgmap v5797: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T02:15:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10565.885556 secs 2023-09-10T02:15:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:36 smithi170 ceph-mon[102832]: pgmap v5798: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 209 KiB/s wr, 18 op/s 2023-09-10T02:15:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:36 smithi082 ceph-mon[98694]: pgmap v5798: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 209 KiB/s wr, 18 op/s 2023-09-10T02:15:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:36 smithi098 ceph-mon[103124]: pgmap v5798: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 209 KiB/s wr, 18 op/s 2023-09-10T02:15:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:38 smithi170 ceph-mon[102832]: pgmap v5799: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 135 KiB/s wr, 12 op/s 2023-09-10T02:15:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10570.173937 secs 2023-09-10T02:15:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:38 smithi082 ceph-mon[98694]: pgmap v5799: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 135 KiB/s wr, 12 op/s 2023-09-10T02:15:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10570.173937 secs 2023-09-10T02:15:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:38 smithi098 ceph-mon[103124]: pgmap v5799: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 135 KiB/s wr, 12 op/s 2023-09-10T02:15:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10570.173937 secs 2023-09-10T02:15:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10570.885621 secs 2023-09-10T02:15:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10570.885621 secs 2023-09-10T02:15:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10570.885621 secs 2023-09-10T02:15:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:40 smithi170 ceph-mon[102832]: pgmap v5800: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 135 KiB/s wr, 12 op/s 2023-09-10T02:15:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:40 smithi082 ceph-mon[98694]: pgmap v5800: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 135 KiB/s wr, 12 op/s 2023-09-10T02:15:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:40 smithi098 ceph-mon[103124]: pgmap v5800: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 135 KiB/s wr, 12 op/s 2023-09-10T02:15:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:42 smithi170 ceph-mon[102832]: pgmap v5801: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 205 KiB/s wr, 18 op/s 2023-09-10T02:15:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:42 smithi082 ceph-mon[98694]: pgmap v5801: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 205 KiB/s wr, 18 op/s 2023-09-10T02:15:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:42 smithi098 ceph-mon[103124]: pgmap v5801: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 205 KiB/s wr, 18 op/s 2023-09-10T02:15:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10573.782682 secs 2023-09-10T02:15:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10575.173996 secs 2023-09-10T02:15:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10573.782682 secs 2023-09-10T02:15:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10575.173996 secs 2023-09-10T02:15:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10573.782682 secs 2023-09-10T02:15:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10575.173996 secs 2023-09-10T02:15:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:44 smithi170 ceph-mon[102832]: pgmap v5802: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 147 KiB/s wr, 12 op/s 2023-09-10T02:15:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:15:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:15:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:15:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10575.885685 secs 2023-09-10T02:15:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:44 smithi082 ceph-mon[98694]: pgmap v5802: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 147 KiB/s wr, 12 op/s 2023-09-10T02:15:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:15:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:15:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:15:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10575.885685 secs 2023-09-10T02:15:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:44 smithi098 ceph-mon[103124]: pgmap v5802: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 147 KiB/s wr, 12 op/s 2023-09-10T02:15:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:15:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:15:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:15:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10575.885685 secs 2023-09-10T02:15:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:46 smithi170 ceph-mon[102832]: pgmap v5803: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 246 KiB/s wr, 19 op/s 2023-09-10T02:15:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:46 smithi082 ceph-mon[98694]: pgmap v5803: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 246 KiB/s wr, 19 op/s 2023-09-10T02:15:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:46 smithi098 ceph-mon[103124]: pgmap v5803: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 246 KiB/s wr, 19 op/s 2023-09-10T02:15:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10578.782794 secs 2023-09-10T02:15:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:48 smithi170 ceph-mon[102832]: pgmap v5804: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 169 KiB/s wr, 12 op/s 2023-09-10T02:15:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10580.174050 secs 2023-09-10T02:15:48.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10578.782794 secs 2023-09-10T02:15:48.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:48 smithi082 ceph-mon[98694]: pgmap v5804: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 169 KiB/s wr, 12 op/s 2023-09-10T02:15:48.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10580.174050 secs 2023-09-10T02:15:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10578.782794 secs 2023-09-10T02:15:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:48 smithi098 ceph-mon[103124]: pgmap v5804: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 169 KiB/s wr, 12 op/s 2023-09-10T02:15:48.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10580.174050 secs 2023-09-10T02:15:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10580.885754 secs 2023-09-10T02:15:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10580.885754 secs 2023-09-10T02:15:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10580.885754 secs 2023-09-10T02:15:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:50 smithi170 ceph-mon[102832]: pgmap v5805: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 169 KiB/s wr, 12 op/s 2023-09-10T02:15:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:50 smithi082 ceph-mon[98694]: pgmap v5805: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 169 KiB/s wr, 12 op/s 2023-09-10T02:15:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:50 smithi098 ceph-mon[103124]: pgmap v5805: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 169 KiB/s wr, 12 op/s 2023-09-10T02:15:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:52 smithi170 ceph-mon[102832]: pgmap v5806: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 222 KiB/s wr, 18 op/s 2023-09-10T02:15:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:52 smithi082 ceph-mon[98694]: pgmap v5806: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 222 KiB/s wr, 18 op/s 2023-09-10T02:15:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:52 smithi098 ceph-mon[103124]: pgmap v5806: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 222 KiB/s wr, 18 op/s 2023-09-10T02:15:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10583.782854 secs 2023-09-10T02:15:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10585.174109 secs 2023-09-10T02:15:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10583.782854 secs 2023-09-10T02:15:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10585.174109 secs 2023-09-10T02:15:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10583.782854 secs 2023-09-10T02:15:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10585.174109 secs 2023-09-10T02:15:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:54 smithi170 ceph-mon[102832]: pgmap v5807: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 12 op/s 2023-09-10T02:15:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10585.885827 secs 2023-09-10T02:15:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:54 smithi082 ceph-mon[98694]: pgmap v5807: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 12 op/s 2023-09-10T02:15:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10585.885827 secs 2023-09-10T02:15:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:54 smithi098 ceph-mon[103124]: pgmap v5807: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 12 op/s 2023-09-10T02:15:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10585.885827 secs 2023-09-10T02:15:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:56 smithi170 ceph-mon[102832]: pgmap v5808: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 160 KiB/s wr, 17 op/s 2023-09-10T02:15:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:56 smithi082 ceph-mon[98694]: pgmap v5808: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 160 KiB/s wr, 17 op/s 2023-09-10T02:15:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:56 smithi098 ceph-mon[103124]: pgmap v5808: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 160 KiB/s wr, 17 op/s 2023-09-10T02:15:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10588.782915 secs 2023-09-10T02:15:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:58 smithi170 ceph-mon[102832]: pgmap v5809: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T02:15:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10590.174190 secs 2023-09-10T02:15:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10588.782915 secs 2023-09-10T02:15:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:58 smithi082 ceph-mon[98694]: pgmap v5809: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T02:15:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10590.174190 secs 2023-09-10T02:15:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10588.782915 secs 2023-09-10T02:15:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:58 smithi098 ceph-mon[103124]: pgmap v5809: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T02:15:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10590.174190 secs 2023-09-10T02:15:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:15:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10590.885893 secs 2023-09-10T02:15:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:15:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10590.885893 secs 2023-09-10T02:15:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:15:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10590.885893 secs 2023-09-10T02:16:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:00 smithi082 ceph-mon[98694]: pgmap v5810: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T02:16:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:00 smithi098 ceph-mon[103124]: pgmap v5810: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T02:16:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:00 smithi170 ceph-mon[102832]: pgmap v5810: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T02:16:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:02 smithi170 ceph-mon[102832]: pgmap v5811: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 16 op/s 2023-09-10T02:16:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:02 smithi082 ceph-mon[98694]: pgmap v5811: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 16 op/s 2023-09-10T02:16:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:02 smithi098 ceph-mon[103124]: pgmap v5811: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 16 op/s 2023-09-10T02:16:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10593.782978 secs 2023-09-10T02:16:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10595.174262 secs 2023-09-10T02:16:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10593.782978 secs 2023-09-10T02:16:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10595.174262 secs 2023-09-10T02:16:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10593.782978 secs 2023-09-10T02:16:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10595.174262 secs 2023-09-10T02:16:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:04 smithi170 ceph-mon[102832]: pgmap v5812: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 10 op/s 2023-09-10T02:16:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10595.885957 secs 2023-09-10T02:16:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:04 smithi082 ceph-mon[98694]: pgmap v5812: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 10 op/s 2023-09-10T02:16:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10595.885957 secs 2023-09-10T02:16:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:04 smithi098 ceph-mon[103124]: pgmap v5812: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 10 op/s 2023-09-10T02:16:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10595.885957 secs 2023-09-10T02:16:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:06 smithi170 ceph-mon[102832]: pgmap v5813: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-10T02:16:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:06 smithi082 ceph-mon[98694]: pgmap v5813: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-10T02:16:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:06 smithi098 ceph-mon[103124]: pgmap v5813: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-10T02:16:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10598.783040 secs 2023-09-10T02:16:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:08 smithi082 ceph-mon[98694]: pgmap v5814: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 61 KiB/s wr, 24 op/s 2023-09-10T02:16:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:08 smithi082 ceph-mon[98694]: osdmap e431: 12 total, 12 up, 12 in 2023-09-10T02:16:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10600.174319 secs 2023-09-10T02:16:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10598.783040 secs 2023-09-10T02:16:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:08 smithi098 ceph-mon[103124]: pgmap v5814: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 61 KiB/s wr, 24 op/s 2023-09-10T02:16:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:08 smithi098 ceph-mon[103124]: osdmap e431: 12 total, 12 up, 12 in 2023-09-10T02:16:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10600.174319 secs 2023-09-10T02:16:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10598.783040 secs 2023-09-10T02:16:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:08 smithi170 ceph-mon[102832]: pgmap v5814: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 61 KiB/s wr, 24 op/s 2023-09-10T02:16:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:08 smithi170 ceph-mon[102832]: osdmap e431: 12 total, 12 up, 12 in 2023-09-10T02:16:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10600.174319 secs 2023-09-10T02:16:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10600.886021 secs 2023-09-10T02:16:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10600.886021 secs 2023-09-10T02:16:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10600.886021 secs 2023-09-10T02:16:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:10 smithi082 ceph-mon[98694]: pgmap v5816: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-10T02:16:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:10 smithi098 ceph-mon[103124]: pgmap v5816: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-10T02:16:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:10 smithi170 ceph-mon[102832]: pgmap v5816: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 73 KiB/s wr, 29 op/s 2023-09-10T02:16:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:12 smithi170 ceph-mon[102832]: pgmap v5817: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 28 op/s 2023-09-10T02:16:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:12 smithi082 ceph-mon[98694]: pgmap v5817: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 28 op/s 2023-09-10T02:16:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:12 smithi098 ceph-mon[103124]: pgmap v5817: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 28 op/s 2023-09-10T02:16:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10603.783122 secs 2023-09-10T02:16:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10605.174379 secs 2023-09-10T02:16:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10603.783122 secs 2023-09-10T02:16:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10605.174379 secs 2023-09-10T02:16:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10603.783122 secs 2023-09-10T02:16:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10605.174379 secs 2023-09-10T02:16:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:14 smithi170 ceph-mon[102832]: pgmap v5818: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 28 op/s 2023-09-10T02:16:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10605.886086 secs 2023-09-10T02:16:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:14 smithi082 ceph-mon[98694]: pgmap v5818: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 28 op/s 2023-09-10T02:16:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10605.886086 secs 2023-09-10T02:16:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:14 smithi098 ceph-mon[103124]: pgmap v5818: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 28 op/s 2023-09-10T02:16:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10605.886086 secs 2023-09-10T02:16:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:16 smithi082 ceph-mon[98694]: pgmap v5819: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-09-10T02:16:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:16 smithi082 ceph-mon[98694]: osdmap e432: 12 total, 12 up, 12 in 2023-09-10T02:16:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:16 smithi098 ceph-mon[103124]: pgmap v5819: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-09-10T02:16:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:16 smithi098 ceph-mon[103124]: osdmap e432: 12 total, 12 up, 12 in 2023-09-10T02:16:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:16 smithi170 ceph-mon[102832]: pgmap v5819: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-09-10T02:16:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:16 smithi170 ceph-mon[102832]: osdmap e432: 12 total, 12 up, 12 in 2023-09-10T02:16:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10608.783181 secs 2023-09-10T02:16:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:18 smithi082 ceph-mon[98694]: pgmap v5821: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T02:16:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10610.174433 secs 2023-09-10T02:16:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10608.783181 secs 2023-09-10T02:16:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:18 smithi098 ceph-mon[103124]: pgmap v5821: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T02:16:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10610.174433 secs 2023-09-10T02:16:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10608.783181 secs 2023-09-10T02:16:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:18 smithi170 ceph-mon[102832]: pgmap v5821: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T02:16:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10610.174433 secs 2023-09-10T02:16:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10610.886150 secs 2023-09-10T02:16:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10610.886150 secs 2023-09-10T02:16:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10610.886150 secs 2023-09-10T02:16:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:20 smithi082 ceph-mon[98694]: pgmap v5822: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-09-10T02:16:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:20 smithi098 ceph-mon[103124]: pgmap v5822: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-09-10T02:16:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:20 smithi170 ceph-mon[102832]: pgmap v5822: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-09-10T02:16:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:22 smithi170 ceph-mon[102832]: pgmap v5823: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T02:16:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:22 smithi082 ceph-mon[98694]: pgmap v5823: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T02:16:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:22 smithi098 ceph-mon[103124]: pgmap v5823: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T02:16:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:16:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10615.174506 secs 2023-09-10T02:16:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:16:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10615.174506 secs 2023-09-10T02:16:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:16:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10615.174506 secs 2023-09-10T02:16:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:24 smithi170 ceph-mon[102832]: pgmap v5824: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T02:16:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:16:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:16:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:16:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10615.886214 secs 2023-09-10T02:16:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:24 smithi082 ceph-mon[98694]: pgmap v5824: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T02:16:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:16:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:16:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:16:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10615.886214 secs 2023-09-10T02:16:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:24 smithi098 ceph-mon[103124]: pgmap v5824: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T02:16:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:16:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:16:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:16:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10615.886214 secs 2023-09-10T02:16:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:16:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:16:25.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:16:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:26 smithi082 ceph-mon[98694]: pgmap v5825: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T02:16:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:16:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:16:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:16:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:26 smithi098 ceph-mon[103124]: pgmap v5825: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T02:16:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:16:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:16:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:16:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:26 smithi170 ceph-mon[102832]: pgmap v5825: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T02:16:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:16:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:16:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:16:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:28 smithi082 ceph-mon[98694]: pgmap v5826: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T02:16:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10620.174589 secs 2023-09-10T02:16:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:28 smithi098 ceph-mon[103124]: pgmap v5826: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T02:16:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10620.174589 secs 2023-09-10T02:16:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:28 smithi170 ceph-mon[102832]: pgmap v5826: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T02:16:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10620.174589 secs 2023-09-10T02:16:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10620.886279 secs 2023-09-10T02:16:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10620.886279 secs 2023-09-10T02:16:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10620.886279 secs 2023-09-10T02:16:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:30 smithi082 ceph-mon[98694]: pgmap v5827: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 77 KiB/s wr, 9 op/s 2023-09-10T02:16:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:30 smithi098 ceph-mon[103124]: pgmap v5827: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 77 KiB/s wr, 9 op/s 2023-09-10T02:16:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:30 smithi170 ceph-mon[102832]: pgmap v5827: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 77 KiB/s wr, 9 op/s 2023-09-10T02:16:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:32 smithi170 ceph-mon[102832]: pgmap v5828: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 154 KiB/s wr, 16 op/s 2023-09-10T02:16:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:32 smithi082 ceph-mon[98694]: pgmap v5828: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 154 KiB/s wr, 16 op/s 2023-09-10T02:16:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:32 smithi098 ceph-mon[103124]: pgmap v5828: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 154 KiB/s wr, 16 op/s 2023-09-10T02:16:33.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10625.174647 secs 2023-09-10T02:16:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10625.174647 secs 2023-09-10T02:16:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10625.174647 secs 2023-09-10T02:16:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:34 smithi082 ceph-mon[98694]: pgmap v5829: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T02:16:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10625.886343 secs 2023-09-10T02:16:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:34 smithi098 ceph-mon[103124]: pgmap v5829: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T02:16:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10625.886343 secs 2023-09-10T02:16:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:34 smithi170 ceph-mon[102832]: pgmap v5829: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T02:16:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10625.886343 secs 2023-09-10T02:16:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:36 smithi082 ceph-mon[98694]: pgmap v5830: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 132 KiB/s wr, 17 op/s 2023-09-10T02:16:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:36 smithi098 ceph-mon[103124]: pgmap v5830: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 132 KiB/s wr, 17 op/s 2023-09-10T02:16:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:36 smithi170 ceph-mon[102832]: pgmap v5830: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 132 KiB/s wr, 17 op/s 2023-09-10T02:16:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:38 smithi082 ceph-mon[98694]: pgmap v5831: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 83 KiB/s wr, 12 op/s 2023-09-10T02:16:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10630.174740 secs 2023-09-10T02:16:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:38 smithi098 ceph-mon[103124]: pgmap v5831: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 83 KiB/s wr, 12 op/s 2023-09-10T02:16:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10630.174740 secs 2023-09-10T02:16:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:38 smithi170 ceph-mon[102832]: pgmap v5831: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 83 KiB/s wr, 12 op/s 2023-09-10T02:16:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10630.174740 secs 2023-09-10T02:16:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10630.886411 secs 2023-09-10T02:16:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10630.886411 secs 2023-09-10T02:16:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10630.886411 secs 2023-09-10T02:16:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:40 smithi082 ceph-mon[98694]: pgmap v5832: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 83 KiB/s wr, 12 op/s 2023-09-10T02:16:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:40 smithi098 ceph-mon[103124]: pgmap v5832: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 83 KiB/s wr, 12 op/s 2023-09-10T02:16:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:40 smithi170 ceph-mon[102832]: pgmap v5832: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 83 KiB/s wr, 12 op/s 2023-09-10T02:16:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:42 smithi082 ceph-mon[98694]: pgmap v5833: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 180 KiB/s wr, 18 op/s 2023-09-10T02:16:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:42 smithi098 ceph-mon[103124]: pgmap v5833: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 180 KiB/s wr, 18 op/s 2023-09-10T02:16:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:42 smithi170 ceph-mon[102832]: pgmap v5833: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 180 KiB/s wr, 18 op/s 2023-09-10T02:16:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10635.174796 secs 2023-09-10T02:16:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10635.174796 secs 2023-09-10T02:16:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10635.174796 secs 2023-09-10T02:16:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:44 smithi098 ceph-mon[103124]: pgmap v5834: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 10 op/s 2023-09-10T02:16:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10635.886484 secs 2023-09-10T02:16:44.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:44 smithi170 ceph-mon[102832]: pgmap v5834: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 10 op/s 2023-09-10T02:16:44.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10635.886484 secs 2023-09-10T02:16:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:44 smithi082 ceph-mon[98694]: pgmap v5834: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 10 op/s 2023-09-10T02:16:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10635.886484 secs 2023-09-10T02:16:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:46 smithi098 ceph-mon[103124]: pgmap v5835: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 16 op/s 2023-09-10T02:16:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:46 smithi170 ceph-mon[102832]: pgmap v5835: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 16 op/s 2023-09-10T02:16:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:46 smithi082 ceph-mon[98694]: pgmap v5835: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 16 op/s 2023-09-10T02:16:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10638.783692 secs 2023-09-10T02:16:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:48 smithi098 ceph-mon[103124]: pgmap v5836: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T02:16:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10640.174857 secs 2023-09-10T02:16:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10638.783692 secs 2023-09-10T02:16:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:48 smithi170 ceph-mon[102832]: pgmap v5836: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T02:16:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10640.174857 secs 2023-09-10T02:16:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10638.783692 secs 2023-09-10T02:16:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:48 smithi082 ceph-mon[98694]: pgmap v5836: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T02:16:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10640.174857 secs 2023-09-10T02:16:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10640.886550 secs 2023-09-10T02:16:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10640.886550 secs 2023-09-10T02:16:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10640.886550 secs 2023-09-10T02:16:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:50 smithi098 ceph-mon[103124]: pgmap v5837: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T02:16:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:50 smithi170 ceph-mon[102832]: pgmap v5837: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T02:16:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:50 smithi082 ceph-mon[98694]: pgmap v5837: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T02:16:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:52 smithi082 ceph-mon[98694]: pgmap v5838: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 213 KiB/s wr, 16 op/s 2023-09-10T02:16:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:16:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:16:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:16:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:52 smithi098 ceph-mon[103124]: pgmap v5838: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 213 KiB/s wr, 16 op/s 2023-09-10T02:16:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:16:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:16:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:16:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:52 smithi170 ceph-mon[102832]: pgmap v5838: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 213 KiB/s wr, 16 op/s 2023-09-10T02:16:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:16:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:16:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:16:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10643.783760 secs 2023-09-10T02:16:53.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10645.174913 secs 2023-09-10T02:16:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10643.783760 secs 2023-09-10T02:16:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10645.174913 secs 2023-09-10T02:16:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10643.783760 secs 2023-09-10T02:16:53.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10645.174913 secs 2023-09-10T02:16:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:54 smithi082 ceph-mon[98694]: pgmap v5839: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T02:16:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10645.886615 secs 2023-09-10T02:16:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:54 smithi098 ceph-mon[103124]: pgmap v5839: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T02:16:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10645.886615 secs 2023-09-10T02:16:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:54 smithi170 ceph-mon[102832]: pgmap v5839: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T02:16:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10645.886615 secs 2023-09-10T02:16:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:56 smithi082 ceph-mon[98694]: pgmap v5840: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 16 op/s 2023-09-10T02:16:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:56 smithi098 ceph-mon[103124]: pgmap v5840: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 16 op/s 2023-09-10T02:16:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:56 smithi170 ceph-mon[102832]: pgmap v5840: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 16 op/s 2023-09-10T02:16:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10648.783779 secs 2023-09-10T02:16:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:58 smithi082 ceph-mon[98694]: pgmap v5841: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 10 op/s 2023-09-10T02:16:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10650.174985 secs 2023-09-10T02:16:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10648.783779 secs 2023-09-10T02:16:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:58 smithi098 ceph-mon[103124]: pgmap v5841: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 10 op/s 2023-09-10T02:16:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10650.174985 secs 2023-09-10T02:16:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10648.783779 secs 2023-09-10T02:16:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:58 smithi170 ceph-mon[102832]: pgmap v5841: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 10 op/s 2023-09-10T02:16:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10650.174985 secs 2023-09-10T02:16:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:16:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10650.886680 secs 2023-09-10T02:16:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:16:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10650.886680 secs 2023-09-10T02:16:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:16:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10650.886680 secs 2023-09-10T02:17:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:00 smithi082 ceph-mon[98694]: pgmap v5842: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 10 op/s 2023-09-10T02:17:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:00 smithi098 ceph-mon[103124]: pgmap v5842: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 10 op/s 2023-09-10T02:17:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:00 smithi170 ceph-mon[102832]: pgmap v5842: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 10 op/s 2023-09-10T02:17:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:02 smithi082 ceph-mon[98694]: pgmap v5843: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 179 KiB/s wr, 15 op/s 2023-09-10T02:17:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:02 smithi098 ceph-mon[103124]: pgmap v5843: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 179 KiB/s wr, 15 op/s 2023-09-10T02:17:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:02 smithi170 ceph-mon[102832]: pgmap v5843: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 179 KiB/s wr, 15 op/s 2023-09-10T02:17:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10653.783841 secs 2023-09-10T02:17:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10655.175042 secs 2023-09-10T02:17:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10653.783841 secs 2023-09-10T02:17:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10655.175042 secs 2023-09-10T02:17:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10653.783841 secs 2023-09-10T02:17:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10655.175042 secs 2023-09-10T02:17:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:04 smithi082 ceph-mon[98694]: pgmap v5844: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 9 op/s 2023-09-10T02:17:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10655.886748 secs 2023-09-10T02:17:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:04 smithi098 ceph-mon[103124]: pgmap v5844: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 9 op/s 2023-09-10T02:17:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10655.886748 secs 2023-09-10T02:17:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:04 smithi170 ceph-mon[102832]: pgmap v5844: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 9 op/s 2023-09-10T02:17:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10655.886748 secs 2023-09-10T02:17:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:06 smithi082 ceph-mon[98694]: pgmap v5845: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 137 KiB/s wr, 29 op/s 2023-09-10T02:17:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:06 smithi098 ceph-mon[103124]: pgmap v5845: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 137 KiB/s wr, 29 op/s 2023-09-10T02:17:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:06 smithi170 ceph-mon[102832]: pgmap v5845: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 137 KiB/s wr, 29 op/s 2023-09-10T02:17:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10658.783933 secs 2023-09-10T02:17:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:08 smithi098 ceph-mon[103124]: pgmap v5846: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 72 KiB/s wr, 24 op/s 2023-09-10T02:17:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:08 smithi098 ceph-mon[103124]: osdmap e433: 12 total, 12 up, 12 in 2023-09-10T02:17:08.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10660.175101 secs 2023-09-10T02:17:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10658.783933 secs 2023-09-10T02:17:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:08 smithi170 ceph-mon[102832]: pgmap v5846: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 72 KiB/s wr, 24 op/s 2023-09-10T02:17:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:08 smithi170 ceph-mon[102832]: osdmap e433: 12 total, 12 up, 12 in 2023-09-10T02:17:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10660.175101 secs 2023-09-10T02:17:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10658.783933 secs 2023-09-10T02:17:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:08 smithi082 ceph-mon[98694]: pgmap v5846: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 72 KiB/s wr, 24 op/s 2023-09-10T02:17:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:08 smithi082 ceph-mon[98694]: osdmap e433: 12 total, 12 up, 12 in 2023-09-10T02:17:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10660.175101 secs 2023-09-10T02:17:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10660.886811 secs 2023-09-10T02:17:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10660.886811 secs 2023-09-10T02:17:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10660.886811 secs 2023-09-10T02:17:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:10 smithi098 ceph-mon[103124]: pgmap v5848: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T02:17:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:10 smithi170 ceph-mon[102832]: pgmap v5848: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T02:17:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:10 smithi082 ceph-mon[98694]: pgmap v5848: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 86 KiB/s wr, 29 op/s 2023-09-10T02:17:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:12 smithi098 ceph-mon[103124]: pgmap v5849: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T02:17:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:12 smithi170 ceph-mon[102832]: pgmap v5849: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T02:17:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:12 smithi082 ceph-mon[98694]: pgmap v5849: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T02:17:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10663.783993 secs 2023-09-10T02:17:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10665.175184 secs 2023-09-10T02:17:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10663.783993 secs 2023-09-10T02:17:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10665.175184 secs 2023-09-10T02:17:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10663.783993 secs 2023-09-10T02:17:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10665.175184 secs 2023-09-10T02:17:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:14 smithi098 ceph-mon[103124]: pgmap v5850: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T02:17:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10665.886878 secs 2023-09-10T02:17:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:14 smithi170 ceph-mon[102832]: pgmap v5850: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T02:17:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10665.886878 secs 2023-09-10T02:17:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:14 smithi082 ceph-mon[98694]: pgmap v5850: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T02:17:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10665.886878 secs 2023-09-10T02:17:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:16 smithi098 ceph-mon[103124]: pgmap v5851: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:17:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:16 smithi098 ceph-mon[103124]: osdmap e434: 12 total, 12 up, 12 in 2023-09-10T02:17:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:16 smithi170 ceph-mon[102832]: pgmap v5851: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:17:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:16 smithi170 ceph-mon[102832]: osdmap e434: 12 total, 12 up, 12 in 2023-09-10T02:17:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:16 smithi082 ceph-mon[98694]: pgmap v5851: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:17:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:16 smithi082 ceph-mon[98694]: osdmap e434: 12 total, 12 up, 12 in 2023-09-10T02:17:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10668.784052 secs 2023-09-10T02:17:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10668.784052 secs 2023-09-10T02:17:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10668.784052 secs 2023-09-10T02:17:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:18 smithi098 ceph-mon[103124]: pgmap v5853: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:17:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10670.175269 secs 2023-09-10T02:17:18.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:18 smithi170 ceph-mon[102832]: pgmap v5853: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:17:18.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10670.175269 secs 2023-09-10T02:17:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:18 smithi082 ceph-mon[98694]: pgmap v5853: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:17:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10670.175269 secs 2023-09-10T02:17:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10670.886946 secs 2023-09-10T02:17:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10670.886946 secs 2023-09-10T02:17:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10670.886946 secs 2023-09-10T02:17:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:20 smithi170 ceph-mon[102832]: pgmap v5854: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:17:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:20 smithi082 ceph-mon[98694]: pgmap v5854: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:17:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:20 smithi098 ceph-mon[103124]: pgmap v5854: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:17:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:22 smithi170 ceph-mon[102832]: pgmap v5855: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 34 KiB/s wr, 12 op/s 2023-09-10T02:17:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:22 smithi082 ceph-mon[98694]: pgmap v5855: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 34 KiB/s wr, 12 op/s 2023-09-10T02:17:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:22 smithi098 ceph-mon[103124]: pgmap v5855: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 34 KiB/s wr, 12 op/s 2023-09-10T02:17:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:17:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10675.175356 secs 2023-09-10T02:17:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:17:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10675.175356 secs 2023-09-10T02:17:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:17:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10675.175356 secs 2023-09-10T02:17:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:24 smithi170 ceph-mon[102832]: pgmap v5856: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 34 KiB/s wr, 12 op/s 2023-09-10T02:17:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:17:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:17:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:17:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10675.887008 secs 2023-09-10T02:17:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:24 smithi082 ceph-mon[98694]: pgmap v5856: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 34 KiB/s wr, 12 op/s 2023-09-10T02:17:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:17:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:17:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:17:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10675.887008 secs 2023-09-10T02:17:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:24 smithi098 ceph-mon[103124]: pgmap v5856: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 34 KiB/s wr, 12 op/s 2023-09-10T02:17:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:17:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:17:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:17:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10675.887008 secs 2023-09-10T02:17:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:26 smithi170 ceph-mon[102832]: pgmap v5857: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 92 KiB/s wr, 13 op/s 2023-09-10T02:17:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:17:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:26 smithi082 ceph-mon[98694]: pgmap v5857: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 92 KiB/s wr, 13 op/s 2023-09-10T02:17:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:17:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:26 smithi098 ceph-mon[103124]: pgmap v5857: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 92 KiB/s wr, 13 op/s 2023-09-10T02:17:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:17:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:17:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:17:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:17:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:17:27.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:17:27.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:17:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:17:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:17:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:17:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:28 smithi170 ceph-mon[102832]: pgmap v5858: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T02:17:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10680.175417 secs 2023-09-10T02:17:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:28 smithi082 ceph-mon[98694]: pgmap v5858: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T02:17:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10680.175417 secs 2023-09-10T02:17:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:28 smithi098 ceph-mon[103124]: pgmap v5858: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T02:17:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10680.175417 secs 2023-09-10T02:17:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10680.887097 secs 2023-09-10T02:17:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10680.887097 secs 2023-09-10T02:17:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10680.887097 secs 2023-09-10T02:17:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:30 smithi170 ceph-mon[102832]: pgmap v5859: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T02:17:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:30 smithi082 ceph-mon[98694]: pgmap v5859: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T02:17:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:30 smithi098 ceph-mon[103124]: pgmap v5859: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T02:17:32.781 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:32 smithi082 ceph-mon[98694]: pgmap v5860: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 147 KiB/s wr, 17 op/s 2023-09-10T02:17:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:32 smithi170 ceph-mon[102832]: pgmap v5860: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 147 KiB/s wr, 17 op/s 2023-09-10T02:17:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:32 smithi098 ceph-mon[103124]: pgmap v5860: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 147 KiB/s wr, 17 op/s 2023-09-10T02:17:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10685.175501 secs 2023-09-10T02:17:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10685.175501 secs 2023-09-10T02:17:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10685.175501 secs 2023-09-10T02:17:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:34 smithi170 ceph-mon[102832]: pgmap v5861: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T02:17:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10685.887163 secs 2023-09-10T02:17:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:34 smithi082 ceph-mon[98694]: pgmap v5861: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T02:17:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10685.887163 secs 2023-09-10T02:17:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:34 smithi098 ceph-mon[103124]: pgmap v5861: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T02:17:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10685.887163 secs 2023-09-10T02:17:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:36 smithi170 ceph-mon[102832]: pgmap v5862: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 195 KiB/s wr, 19 op/s 2023-09-10T02:17:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:36 smithi082 ceph-mon[98694]: pgmap v5862: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 195 KiB/s wr, 19 op/s 2023-09-10T02:17:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:36 smithi098 ceph-mon[103124]: pgmap v5862: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 195 KiB/s wr, 19 op/s 2023-09-10T02:17:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:38 smithi170 ceph-mon[102832]: pgmap v5863: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T02:17:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10690.175585 secs 2023-09-10T02:17:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:38 smithi082 ceph-mon[98694]: pgmap v5863: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T02:17:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10690.175585 secs 2023-09-10T02:17:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:38 smithi098 ceph-mon[103124]: pgmap v5863: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T02:17:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10690.175585 secs 2023-09-10T02:17:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10690.887231 secs 2023-09-10T02:17:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10690.887231 secs 2023-09-10T02:17:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10690.887231 secs 2023-09-10T02:17:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:40 smithi170 ceph-mon[102832]: pgmap v5864: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T02:17:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:40 smithi082 ceph-mon[98694]: pgmap v5864: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T02:17:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:40 smithi098 ceph-mon[103124]: pgmap v5864: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T02:17:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:42 smithi170 ceph-mon[102832]: pgmap v5865: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 224 KiB/s wr, 18 op/s 2023-09-10T02:17:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10693.784471 secs 2023-09-10T02:17:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:42 smithi082 ceph-mon[98694]: pgmap v5865: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 224 KiB/s wr, 18 op/s 2023-09-10T02:17:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10693.784471 secs 2023-09-10T02:17:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:42 smithi098 ceph-mon[103124]: pgmap v5865: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 224 KiB/s wr, 18 op/s 2023-09-10T02:17:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10693.784471 secs 2023-09-10T02:17:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:17:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:17:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:17:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10695.175643 secs 2023-09-10T02:17:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:17:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:17:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:17:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10695.175643 secs 2023-09-10T02:17:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:17:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:17:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:17:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10695.175643 secs 2023-09-10T02:17:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:44 smithi170 ceph-mon[102832]: pgmap v5866: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 153 KiB/s wr, 12 op/s 2023-09-10T02:17:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10695.887294 secs 2023-09-10T02:17:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:44 smithi082 ceph-mon[98694]: pgmap v5866: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 153 KiB/s wr, 12 op/s 2023-09-10T02:17:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10695.887294 secs 2023-09-10T02:17:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:44 smithi098 ceph-mon[103124]: pgmap v5866: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 153 KiB/s wr, 12 op/s 2023-09-10T02:17:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10695.887294 secs 2023-09-10T02:17:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:46 smithi170 ceph-mon[102832]: pgmap v5867: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 201 KiB/s wr, 17 op/s 2023-09-10T02:17:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:46 smithi082 ceph-mon[98694]: pgmap v5867: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 201 KiB/s wr, 17 op/s 2023-09-10T02:17:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:46 smithi098 ceph-mon[103124]: pgmap v5867: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 201 KiB/s wr, 17 op/s 2023-09-10T02:17:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10698.784581 secs 2023-09-10T02:17:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10698.784581 secs 2023-09-10T02:17:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10698.784581 secs 2023-09-10T02:17:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:48 smithi170 ceph-mon[102832]: pgmap v5868: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T02:17:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10700.175748 secs 2023-09-10T02:17:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:48 smithi082 ceph-mon[98694]: pgmap v5868: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T02:17:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10700.175748 secs 2023-09-10T02:17:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:48 smithi098 ceph-mon[103124]: pgmap v5868: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T02:17:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10700.175748 secs 2023-09-10T02:17:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10700.887359 secs 2023-09-10T02:17:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10700.887359 secs 2023-09-10T02:17:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10700.887359 secs 2023-09-10T02:17:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:50 smithi170 ceph-mon[102832]: pgmap v5869: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T02:17:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:50 smithi082 ceph-mon[98694]: pgmap v5869: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T02:17:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:50 smithi098 ceph-mon[103124]: pgmap v5869: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T02:17:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:52 smithi170 ceph-mon[102832]: pgmap v5870: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 155 KiB/s wr, 16 op/s 2023-09-10T02:17:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10703.784641 secs 2023-09-10T02:17:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:52 smithi082 ceph-mon[98694]: pgmap v5870: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 155 KiB/s wr, 16 op/s 2023-09-10T02:17:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10703.784641 secs 2023-09-10T02:17:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:52 smithi098 ceph-mon[103124]: pgmap v5870: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 155 KiB/s wr, 16 op/s 2023-09-10T02:17:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10703.784641 secs 2023-09-10T02:17:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10705.175806 secs 2023-09-10T02:17:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10705.175806 secs 2023-09-10T02:17:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10705.175806 secs 2023-09-10T02:17:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:54 smithi170 ceph-mon[102832]: pgmap v5871: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 77 KiB/s wr, 10 op/s 2023-09-10T02:17:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10705.887427 secs 2023-09-10T02:17:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:54 smithi082 ceph-mon[98694]: pgmap v5871: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 77 KiB/s wr, 10 op/s 2023-09-10T02:17:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10705.887427 secs 2023-09-10T02:17:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:54 smithi098 ceph-mon[103124]: pgmap v5871: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 77 KiB/s wr, 10 op/s 2023-09-10T02:17:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10705.887427 secs 2023-09-10T02:17:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:56 smithi170 ceph-mon[102832]: pgmap v5872: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 111 KiB/s wr, 16 op/s 2023-09-10T02:17:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:56 smithi082 ceph-mon[98694]: pgmap v5872: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 111 KiB/s wr, 16 op/s 2023-09-10T02:17:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:56 smithi098 ceph-mon[103124]: pgmap v5872: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 111 KiB/s wr, 16 op/s 2023-09-10T02:17:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10708.784701 secs 2023-09-10T02:17:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10708.784701 secs 2023-09-10T02:17:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10708.784701 secs 2023-09-10T02:17:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:58 smithi170 ceph-mon[102832]: pgmap v5873: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T02:17:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10710.175860 secs 2023-09-10T02:17:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:58 smithi082 ceph-mon[98694]: pgmap v5873: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T02:17:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10710.175860 secs 2023-09-10T02:17:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:58 smithi098 ceph-mon[103124]: pgmap v5873: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T02:17:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10710.175860 secs 2023-09-10T02:18:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:17:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10710.887488 secs 2023-09-10T02:18:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:17:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10710.887488 secs 2023-09-10T02:18:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:17:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10710.887488 secs 2023-09-10T02:18:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:00 smithi170 ceph-mon[102832]: pgmap v5874: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T02:18:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:00 smithi082 ceph-mon[98694]: pgmap v5874: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T02:18:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:00 smithi098 ceph-mon[103124]: pgmap v5874: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T02:18:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:02 smithi170 ceph-mon[102832]: pgmap v5875: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 121 KiB/s wr, 31 op/s 2023-09-10T02:18:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10713.784810 secs 2023-09-10T02:18:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:02 smithi082 ceph-mon[98694]: pgmap v5875: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 121 KiB/s wr, 31 op/s 2023-09-10T02:18:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10713.784810 secs 2023-09-10T02:18:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:02 smithi098 ceph-mon[103124]: pgmap v5875: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 121 KiB/s wr, 31 op/s 2023-09-10T02:18:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10713.784810 secs 2023-09-10T02:18:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10715.175916 secs 2023-09-10T02:18:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10715.175916 secs 2023-09-10T02:18:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10715.175916 secs 2023-09-10T02:18:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:04 smithi170 ceph-mon[102832]: pgmap v5876: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 91 KiB/s wr, 26 op/s 2023-09-10T02:18:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10715.887549 secs 2023-09-10T02:18:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:04 smithi082 ceph-mon[98694]: pgmap v5876: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 91 KiB/s wr, 26 op/s 2023-09-10T02:18:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10715.887549 secs 2023-09-10T02:18:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:04 smithi098 ceph-mon[103124]: pgmap v5876: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 91 KiB/s wr, 26 op/s 2023-09-10T02:18:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10715.887549 secs 2023-09-10T02:18:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:06 smithi170 ceph-mon[102832]: pgmap v5877: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 100 KiB/s wr, 32 op/s 2023-09-10T02:18:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:06 smithi082 ceph-mon[98694]: pgmap v5877: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 100 KiB/s wr, 32 op/s 2023-09-10T02:18:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:06 smithi098 ceph-mon[103124]: pgmap v5877: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 100 KiB/s wr, 32 op/s 2023-09-10T02:18:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10718.784870 secs 2023-09-10T02:18:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10718.784870 secs 2023-09-10T02:18:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10718.784870 secs 2023-09-10T02:18:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:08 smithi170 ceph-mon[102832]: pgmap v5878: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 67 KiB/s wr, 26 op/s 2023-09-10T02:18:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:08 smithi170 ceph-mon[102832]: osdmap e435: 12 total, 12 up, 12 in 2023-09-10T02:18:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10720.175975 secs 2023-09-10T02:18:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:08 smithi082 ceph-mon[98694]: pgmap v5878: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 67 KiB/s wr, 26 op/s 2023-09-10T02:18:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:08 smithi082 ceph-mon[98694]: osdmap e435: 12 total, 12 up, 12 in 2023-09-10T02:18:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10720.175975 secs 2023-09-10T02:18:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:08 smithi098 ceph-mon[103124]: pgmap v5878: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 67 KiB/s wr, 26 op/s 2023-09-10T02:18:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:08 smithi098 ceph-mon[103124]: osdmap e435: 12 total, 12 up, 12 in 2023-09-10T02:18:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10720.175975 secs 2023-09-10T02:18:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10720.887616 secs 2023-09-10T02:18:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10720.887616 secs 2023-09-10T02:18:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10720.887616 secs 2023-09-10T02:18:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:10 smithi170 ceph-mon[102832]: pgmap v5880: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 80 KiB/s wr, 31 op/s 2023-09-10T02:18:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:10 smithi082 ceph-mon[98694]: pgmap v5880: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 80 KiB/s wr, 31 op/s 2023-09-10T02:18:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:10 smithi098 ceph-mon[103124]: pgmap v5880: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 80 KiB/s wr, 31 op/s 2023-09-10T02:18:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:12 smithi170 ceph-mon[102832]: pgmap v5881: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 13 op/s 2023-09-10T02:18:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10723.784929 secs 2023-09-10T02:18:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:12 smithi082 ceph-mon[98694]: pgmap v5881: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 13 op/s 2023-09-10T02:18:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10723.784929 secs 2023-09-10T02:18:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:12 smithi098 ceph-mon[103124]: pgmap v5881: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 13 op/s 2023-09-10T02:18:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10723.784929 secs 2023-09-10T02:18:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10725.176036 secs 2023-09-10T02:18:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10725.176036 secs 2023-09-10T02:18:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10725.176036 secs 2023-09-10T02:18:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:14 smithi170 ceph-mon[102832]: pgmap v5882: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 13 op/s 2023-09-10T02:18:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10725.887681 secs 2023-09-10T02:18:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:14 smithi082 ceph-mon[98694]: pgmap v5882: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 13 op/s 2023-09-10T02:18:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10725.887681 secs 2023-09-10T02:18:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:14 smithi098 ceph-mon[103124]: pgmap v5882: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 13 op/s 2023-09-10T02:18:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10725.887681 secs 2023-09-10T02:18:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:16 smithi170 ceph-mon[102832]: pgmap v5883: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T02:18:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:16 smithi170 ceph-mon[102832]: osdmap e436: 12 total, 12 up, 12 in 2023-09-10T02:18:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:16 smithi082 ceph-mon[98694]: pgmap v5883: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T02:18:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:16 smithi082 ceph-mon[98694]: osdmap e436: 12 total, 12 up, 12 in 2023-09-10T02:18:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:16 smithi098 ceph-mon[103124]: pgmap v5883: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T02:18:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:16 smithi098 ceph-mon[103124]: osdmap e436: 12 total, 12 up, 12 in 2023-09-10T02:18:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10728.785008 secs 2023-09-10T02:18:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10728.785008 secs 2023-09-10T02:18:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10728.785008 secs 2023-09-10T02:18:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:18 smithi170 ceph-mon[102832]: pgmap v5885: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 12 op/s 2023-09-10T02:18:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10730.176095 secs 2023-09-10T02:18:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:18 smithi082 ceph-mon[98694]: pgmap v5885: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 12 op/s 2023-09-10T02:18:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10730.176095 secs 2023-09-10T02:18:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:18 smithi098 ceph-mon[103124]: pgmap v5885: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 12 op/s 2023-09-10T02:18:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10730.176095 secs 2023-09-10T02:18:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10730.887747 secs 2023-09-10T02:18:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10730.887747 secs 2023-09-10T02:18:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10730.887747 secs 2023-09-10T02:18:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:20 smithi170 ceph-mon[102832]: pgmap v5886: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T02:18:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:20 smithi082 ceph-mon[98694]: pgmap v5886: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T02:18:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:20 smithi098 ceph-mon[103124]: pgmap v5886: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-09-10T02:18:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:22 smithi082 ceph-mon[98694]: pgmap v5887: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T02:18:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:18:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:22 smithi098 ceph-mon[103124]: pgmap v5887: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T02:18:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:18:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:22 smithi170 ceph-mon[102832]: pgmap v5887: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T02:18:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:18:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10735.176177 secs 2023-09-10T02:18:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:18:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:18:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:18:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10735.176177 secs 2023-09-10T02:18:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:18:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:18:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:18:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10735.176177 secs 2023-09-10T02:18:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:18:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:18:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:18:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:24 smithi082 ceph-mon[98694]: pgmap v5888: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T02:18:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10735.887813 secs 2023-09-10T02:18:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:24 smithi098 ceph-mon[103124]: pgmap v5888: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T02:18:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10735.887813 secs 2023-09-10T02:18:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:24 smithi170 ceph-mon[102832]: pgmap v5888: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T02:18:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10735.887813 secs 2023-09-10T02:18:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:26 smithi082 ceph-mon[98694]: pgmap v5889: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T02:18:27.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:18:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:26 smithi098 ceph-mon[103124]: pgmap v5889: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T02:18:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:18:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:26 smithi170 ceph-mon[102832]: pgmap v5889: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T02:18:27.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:18:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:28 smithi082 ceph-mon[98694]: pgmap v5890: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T02:18:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10740.176236 secs 2023-09-10T02:18:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:28 smithi098 ceph-mon[103124]: pgmap v5890: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T02:18:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10740.176236 secs 2023-09-10T02:18:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:28 smithi170 ceph-mon[102832]: pgmap v5890: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T02:18:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10740.176236 secs 2023-09-10T02:18:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10740.887885 secs 2023-09-10T02:18:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10740.887885 secs 2023-09-10T02:18:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:30.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:30.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:30.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10740.887885 secs 2023-09-10T02:18:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:30.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:30.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:30 smithi082 ceph-mon[98694]: pgmap v5891: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T02:18:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:31.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:31.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:31.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:31.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:18:31.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:18:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:30 smithi098 ceph-mon[103124]: pgmap v5891: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T02:18:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:31.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:31.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:31.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:31.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:31.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.258 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:18:31.258 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:18:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:30 smithi170 ceph-mon[102832]: pgmap v5891: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T02:18:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:31.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:31.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:31.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:31.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:18:31.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:18:31.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:18:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:31 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 195.7M 2023-09-10T02:18:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:31 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 205208383: error parsing value: Value '205208383' is below minimum 939524096 2023-09-10T02:18:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:31 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T02:18:32.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:31 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T02:18:32.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:31 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T02:18:32.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:31 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T02:18:32.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:31 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 195.7M 2023-09-10T02:18:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:31 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 205208383: error parsing value: Value '205208383' is below minimum 939524096 2023-09-10T02:18:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:31 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T02:18:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:31 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T02:18:32.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:31 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T02:18:32.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:31 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T02:18:32.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:31 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 195.7M 2023-09-10T02:18:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:31 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 205208383: error parsing value: Value '205208383' is below minimum 939524096 2023-09-10T02:18:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:31 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T02:18:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:31 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T02:18:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:31 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T02:18:32.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:31 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T02:18:32.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:18:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:32 smithi082 ceph-mon[98694]: pgmap v5892: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 135 KiB/s wr, 16 op/s 2023-09-10T02:18:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:32 smithi098 ceph-mon[103124]: pgmap v5892: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 135 KiB/s wr, 16 op/s 2023-09-10T02:18:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:32 smithi170 ceph-mon[102832]: pgmap v5892: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 135 KiB/s wr, 16 op/s 2023-09-10T02:18:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10745.176292 secs 2023-09-10T02:18:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10745.176292 secs 2023-09-10T02:18:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10745.176292 secs 2023-09-10T02:18:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:34 smithi098 ceph-mon[103124]: pgmap v5893: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 108 KiB/s wr, 11 op/s 2023-09-10T02:18:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10745.887946 secs 2023-09-10T02:18:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:34 smithi170 ceph-mon[102832]: pgmap v5893: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 108 KiB/s wr, 11 op/s 2023-09-10T02:18:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10745.887946 secs 2023-09-10T02:18:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:34 smithi082 ceph-mon[98694]: pgmap v5893: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 108 KiB/s wr, 11 op/s 2023-09-10T02:18:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10745.887946 secs 2023-09-10T02:18:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:36 smithi098 ceph-mon[103124]: pgmap v5894: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 160 KiB/s wr, 17 op/s 2023-09-10T02:18:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:36 smithi170 ceph-mon[102832]: pgmap v5894: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 160 KiB/s wr, 17 op/s 2023-09-10T02:18:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:36 smithi082 ceph-mon[98694]: pgmap v5894: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 160 KiB/s wr, 17 op/s 2023-09-10T02:18:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:38 smithi098 ceph-mon[103124]: pgmap v5895: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T02:18:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10750.176374 secs 2023-09-10T02:18:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:38 smithi170 ceph-mon[102832]: pgmap v5895: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T02:18:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10750.176374 secs 2023-09-10T02:18:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:38 smithi082 ceph-mon[98694]: pgmap v5895: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T02:18:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10750.176374 secs 2023-09-10T02:18:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10750.888015 secs 2023-09-10T02:18:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10750.888015 secs 2023-09-10T02:18:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10750.888015 secs 2023-09-10T02:18:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:40 smithi098 ceph-mon[103124]: pgmap v5896: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T02:18:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:40 smithi170 ceph-mon[102832]: pgmap v5896: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T02:18:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:40 smithi082 ceph-mon[98694]: pgmap v5896: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T02:18:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:42 smithi098 ceph-mon[103124]: pgmap v5897: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 178 KiB/s wr, 17 op/s 2023-09-10T02:18:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:42 smithi170 ceph-mon[102832]: pgmap v5897: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 178 KiB/s wr, 17 op/s 2023-09-10T02:18:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:42 smithi082 ceph-mon[98694]: pgmap v5897: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 178 KiB/s wr, 17 op/s 2023-09-10T02:18:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10755.176430 secs 2023-09-10T02:18:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10755.176430 secs 2023-09-10T02:18:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10755.176430 secs 2023-09-10T02:18:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:44 smithi098 ceph-mon[103124]: pgmap v5898: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T02:18:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10755.888081 secs 2023-09-10T02:18:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:44 smithi170 ceph-mon[102832]: pgmap v5898: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T02:18:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10755.888081 secs 2023-09-10T02:18:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:44 smithi082 ceph-mon[98694]: pgmap v5898: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T02:18:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10755.888081 secs 2023-09-10T02:18:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:46 smithi098 ceph-mon[103124]: pgmap v5899: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 173 KiB/s wr, 17 op/s 2023-09-10T02:18:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:46 smithi170 ceph-mon[102832]: pgmap v5899: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 173 KiB/s wr, 17 op/s 2023-09-10T02:18:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:46 smithi082 ceph-mon[98694]: pgmap v5899: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 173 KiB/s wr, 17 op/s 2023-09-10T02:18:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10758.785439 secs 2023-09-10T02:18:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10758.785439 secs 2023-09-10T02:18:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10758.785439 secs 2023-09-10T02:18:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:48 smithi098 ceph-mon[103124]: pgmap v5900: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T02:18:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10760.176496 secs 2023-09-10T02:18:49.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10760.888148 secs 2023-09-10T02:18:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:48 smithi170 ceph-mon[102832]: pgmap v5900: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T02:18:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10760.176496 secs 2023-09-10T02:18:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10760.888148 secs 2023-09-10T02:18:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:48 smithi082 ceph-mon[98694]: pgmap v5900: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T02:18:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10760.176496 secs 2023-09-10T02:18:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10760.888148 secs 2023-09-10T02:18:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:50 smithi098 ceph-mon[103124]: pgmap v5901: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T02:18:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:50 smithi170 ceph-mon[102832]: pgmap v5901: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T02:18:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:50 smithi082 ceph-mon[98694]: pgmap v5901: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T02:18:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:18:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:18:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:18:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:18:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:18:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:18:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:18:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:18:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:18:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:53 smithi170 ceph-mon[102832]: pgmap v5902: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 170 KiB/s wr, 17 op/s 2023-09-10T02:18:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10763.785499 secs 2023-09-10T02:18:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:53 smithi082 ceph-mon[98694]: pgmap v5902: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 170 KiB/s wr, 17 op/s 2023-09-10T02:18:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10763.785499 secs 2023-09-10T02:18:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:53 smithi098 ceph-mon[103124]: pgmap v5902: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 170 KiB/s wr, 17 op/s 2023-09-10T02:18:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10763.785499 secs 2023-09-10T02:18:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10765.176552 secs 2023-09-10T02:18:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10765.888216 secs 2023-09-10T02:18:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10765.176552 secs 2023-09-10T02:18:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10765.888216 secs 2023-09-10T02:18:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10765.176552 secs 2023-09-10T02:18:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10765.888216 secs 2023-09-10T02:18:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:55 smithi170 ceph-mon[102832]: pgmap v5903: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T02:18:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:55 smithi082 ceph-mon[98694]: pgmap v5903: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T02:18:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:55 smithi098 ceph-mon[103124]: pgmap v5903: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T02:18:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:57 smithi170 ceph-mon[102832]: pgmap v5904: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 155 KiB/s wr, 17 op/s 2023-09-10T02:18:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:57 smithi082 ceph-mon[98694]: pgmap v5904: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 155 KiB/s wr, 17 op/s 2023-09-10T02:18:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:57 smithi098 ceph-mon[103124]: pgmap v5904: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 155 KiB/s wr, 17 op/s 2023-09-10T02:18:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10768.785559 secs 2023-09-10T02:18:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10768.785559 secs 2023-09-10T02:18:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10768.785559 secs 2023-09-10T02:18:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:59 smithi170 ceph-mon[102832]: pgmap v5905: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T02:18:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10770.176610 secs 2023-09-10T02:18:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:18:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10770.888281 secs 2023-09-10T02:18:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:59 smithi082 ceph-mon[98694]: pgmap v5905: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T02:18:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10770.176610 secs 2023-09-10T02:18:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:18:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10770.888281 secs 2023-09-10T02:18:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:59 smithi098 ceph-mon[103124]: pgmap v5905: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T02:18:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10770.176610 secs 2023-09-10T02:18:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:18:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10770.888281 secs 2023-09-10T02:19:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:01 smithi170 ceph-mon[102832]: pgmap v5906: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T02:19:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:01 smithi082 ceph-mon[98694]: pgmap v5906: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T02:19:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:01 smithi098 ceph-mon[103124]: pgmap v5906: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T02:19:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:03 smithi170 ceph-mon[102832]: pgmap v5907: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 137 KiB/s wr, 28 op/s 2023-09-10T02:19:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10773.785626 secs 2023-09-10T02:19:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:03 smithi082 ceph-mon[98694]: pgmap v5907: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 137 KiB/s wr, 28 op/s 2023-09-10T02:19:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10773.785626 secs 2023-09-10T02:19:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:03 smithi098 ceph-mon[103124]: pgmap v5907: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 137 KiB/s wr, 28 op/s 2023-09-10T02:19:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10773.785626 secs 2023-09-10T02:19:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10775.176659 secs 2023-09-10T02:19:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10775.888346 secs 2023-09-10T02:19:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10775.176659 secs 2023-09-10T02:19:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10775.888346 secs 2023-09-10T02:19:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10775.176659 secs 2023-09-10T02:19:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10775.888346 secs 2023-09-10T02:19:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:05 smithi170 ceph-mon[102832]: pgmap v5908: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 87 KiB/s wr, 22 op/s 2023-09-10T02:19:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:05 smithi082 ceph-mon[98694]: pgmap v5908: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 87 KiB/s wr, 22 op/s 2023-09-10T02:19:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:05 smithi098 ceph-mon[103124]: pgmap v5908: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 87 KiB/s wr, 22 op/s 2023-09-10T02:19:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:07 smithi170 ceph-mon[102832]: pgmap v5909: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 110 KiB/s wr, 29 op/s 2023-09-10T02:19:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:07 smithi082 ceph-mon[98694]: pgmap v5909: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 110 KiB/s wr, 29 op/s 2023-09-10T02:19:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:07 smithi098 ceph-mon[103124]: pgmap v5909: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 110 KiB/s wr, 29 op/s 2023-09-10T02:19:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10778.785718 secs 2023-09-10T02:19:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10778.785718 secs 2023-09-10T02:19:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10778.785718 secs 2023-09-10T02:19:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:09 smithi170 ceph-mon[102832]: pgmap v5910: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 64 KiB/s wr, 23 op/s 2023-09-10T02:19:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10780.176745 secs 2023-09-10T02:19:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:09 smithi170 ceph-mon[102832]: osdmap e437: 12 total, 12 up, 12 in 2023-09-10T02:19:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10780.888411 secs 2023-09-10T02:19:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:09 smithi082 ceph-mon[98694]: pgmap v5910: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 64 KiB/s wr, 23 op/s 2023-09-10T02:19:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10780.176745 secs 2023-09-10T02:19:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:09 smithi082 ceph-mon[98694]: osdmap e437: 12 total, 12 up, 12 in 2023-09-10T02:19:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10780.888411 secs 2023-09-10T02:19:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:09 smithi098 ceph-mon[103124]: pgmap v5910: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 64 KiB/s wr, 23 op/s 2023-09-10T02:19:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10780.176745 secs 2023-09-10T02:19:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:09 smithi098 ceph-mon[103124]: osdmap e437: 12 total, 12 up, 12 in 2023-09-10T02:19:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10780.888411 secs 2023-09-10T02:19:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:11 smithi170 ceph-mon[102832]: pgmap v5912: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 28 op/s 2023-09-10T02:19:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:11 smithi082 ceph-mon[98694]: pgmap v5912: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 28 op/s 2023-09-10T02:19:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:11 smithi098 ceph-mon[103124]: pgmap v5912: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 28 op/s 2023-09-10T02:19:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:13 smithi170 ceph-mon[102832]: pgmap v5913: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 50 KiB/s wr, 13 op/s 2023-09-10T02:19:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10783.785773 secs 2023-09-10T02:19:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:13 smithi082 ceph-mon[98694]: pgmap v5913: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 50 KiB/s wr, 13 op/s 2023-09-10T02:19:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10783.785773 secs 2023-09-10T02:19:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:13 smithi098 ceph-mon[103124]: pgmap v5913: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 50 KiB/s wr, 13 op/s 2023-09-10T02:19:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10783.785773 secs 2023-09-10T02:19:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10785.176802 secs 2023-09-10T02:19:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10785.888484 secs 2023-09-10T02:19:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10785.176802 secs 2023-09-10T02:19:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10785.888484 secs 2023-09-10T02:19:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10785.176802 secs 2023-09-10T02:19:14.546 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10785.888484 secs 2023-09-10T02:19:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:15 smithi170 ceph-mon[102832]: pgmap v5914: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 50 KiB/s wr, 13 op/s 2023-09-10T02:19:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:15 smithi082 ceph-mon[98694]: pgmap v5914: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 50 KiB/s wr, 13 op/s 2023-09-10T02:19:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:15 smithi098 ceph-mon[103124]: pgmap v5914: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 50 KiB/s wr, 13 op/s 2023-09-10T02:19:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:17 smithi170 ceph-mon[102832]: pgmap v5915: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T02:19:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:17 smithi170 ceph-mon[102832]: osdmap e438: 12 total, 12 up, 12 in 2023-09-10T02:19:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:17 smithi082 ceph-mon[98694]: pgmap v5915: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T02:19:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:17 smithi082 ceph-mon[98694]: osdmap e438: 12 total, 12 up, 12 in 2023-09-10T02:19:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:17 smithi098 ceph-mon[103124]: pgmap v5915: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T02:19:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:17 smithi098 ceph-mon[103124]: osdmap e438: 12 total, 12 up, 12 in 2023-09-10T02:19:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10788.785833 secs 2023-09-10T02:19:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10788.785833 secs 2023-09-10T02:19:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10788.785833 secs 2023-09-10T02:19:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:19 smithi170 ceph-mon[102832]: pgmap v5917: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 12 op/s 2023-09-10T02:19:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10790.176861 secs 2023-09-10T02:19:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10790.888551 secs 2023-09-10T02:19:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:19 smithi082 ceph-mon[98694]: pgmap v5917: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 12 op/s 2023-09-10T02:19:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10790.176861 secs 2023-09-10T02:19:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10790.888551 secs 2023-09-10T02:19:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:19 smithi098 ceph-mon[103124]: pgmap v5917: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 12 op/s 2023-09-10T02:19:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10790.176861 secs 2023-09-10T02:19:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10790.888551 secs 2023-09-10T02:19:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:21 smithi170 ceph-mon[102832]: pgmap v5918: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T02:19:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:21 smithi082 ceph-mon[98694]: pgmap v5918: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T02:19:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:21 smithi098 ceph-mon[103124]: pgmap v5918: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T02:19:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:23 smithi170 ceph-mon[102832]: pgmap v5919: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:19:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:19:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10795.176921 secs 2023-09-10T02:19:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:23 smithi082 ceph-mon[98694]: pgmap v5919: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:19:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:19:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10795.176921 secs 2023-09-10T02:19:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:23 smithi098 ceph-mon[103124]: pgmap v5919: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:19:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:19:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10795.176921 secs 2023-09-10T02:19:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:19:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:19:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:19:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10795.888617 secs 2023-09-10T02:19:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:19:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:19:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:19:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10795.888617 secs 2023-09-10T02:19:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:19:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:19:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:19:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10795.888617 secs 2023-09-10T02:19:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:25 smithi170 ceph-mon[102832]: pgmap v5920: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:19:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:25 smithi082 ceph-mon[98694]: pgmap v5920: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:19:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:25 smithi098 ceph-mon[103124]: pgmap v5920: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:19:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:27 smithi170 ceph-mon[102832]: pgmap v5921: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T02:19:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:27 smithi082 ceph-mon[98694]: pgmap v5921: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T02:19:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:27 smithi098 ceph-mon[103124]: pgmap v5921: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T02:19:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10800.176979 secs 2023-09-10T02:19:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10800.176979 secs 2023-09-10T02:19:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10800.176979 secs 2023-09-10T02:19:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:29 smithi170 ceph-mon[102832]: pgmap v5922: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T02:19:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10800.888692 secs 2023-09-10T02:19:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:29 smithi082 ceph-mon[98694]: pgmap v5922: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T02:19:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10800.888692 secs 2023-09-10T02:19:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:29 smithi098 ceph-mon[103124]: pgmap v5922: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T02:19:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10800.888692 secs 2023-09-10T02:19:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:31 smithi170 ceph-mon[102832]: pgmap v5923: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 9 op/s 2023-09-10T02:19:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:19:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:31 smithi082 ceph-mon[98694]: pgmap v5923: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 9 op/s 2023-09-10T02:19:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:19:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:31 smithi098 ceph-mon[103124]: pgmap v5923: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 9 op/s 2023-09-10T02:19:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:19:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:32 smithi098 ceph-mon[103124]: pgmap v5924: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 138 KiB/s wr, 15 op/s 2023-09-10T02:19:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:19:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:19:32.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:19:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:32 smithi170 ceph-mon[102832]: pgmap v5924: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 138 KiB/s wr, 15 op/s 2023-09-10T02:19:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:19:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:19:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:19:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:32 smithi082 ceph-mon[98694]: pgmap v5924: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 138 KiB/s wr, 15 op/s 2023-09-10T02:19:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:19:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:19:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:19:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10805.177043 secs 2023-09-10T02:19:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10805.177043 secs 2023-09-10T02:19:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10805.177043 secs 2023-09-10T02:19:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:34 smithi098 ceph-mon[103124]: pgmap v5925: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T02:19:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10805.888733 secs 2023-09-10T02:19:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:34 smithi170 ceph-mon[102832]: pgmap v5925: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T02:19:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10805.888733 secs 2023-09-10T02:19:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:34 smithi082 ceph-mon[98694]: pgmap v5925: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T02:19:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10805.888733 secs 2023-09-10T02:19:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:36 smithi098 ceph-mon[103124]: pgmap v5926: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 202 KiB/s wr, 17 op/s 2023-09-10T02:19:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:36 smithi170 ceph-mon[102832]: pgmap v5926: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 202 KiB/s wr, 17 op/s 2023-09-10T02:19:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:36 smithi082 ceph-mon[98694]: pgmap v5926: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 202 KiB/s wr, 17 op/s 2023-09-10T02:19:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:38 smithi098 ceph-mon[103124]: pgmap v5927: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T02:19:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10810.177129 secs 2023-09-10T02:19:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:38 smithi170 ceph-mon[102832]: pgmap v5927: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T02:19:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10810.177129 secs 2023-09-10T02:19:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:38 smithi082 ceph-mon[98694]: pgmap v5927: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T02:19:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10810.177129 secs 2023-09-10T02:19:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10810.888798 secs 2023-09-10T02:19:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10810.888798 secs 2023-09-10T02:19:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10810.888798 secs 2023-09-10T02:19:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:40 smithi098 ceph-mon[103124]: pgmap v5928: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T02:19:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:40 smithi170 ceph-mon[102832]: pgmap v5928: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T02:19:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:40 smithi082 ceph-mon[98694]: pgmap v5928: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T02:19:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:42 smithi098 ceph-mon[103124]: pgmap v5929: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 214 KiB/s wr, 18 op/s 2023-09-10T02:19:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:42 smithi170 ceph-mon[102832]: pgmap v5929: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 214 KiB/s wr, 18 op/s 2023-09-10T02:19:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:42 smithi082 ceph-mon[98694]: pgmap v5929: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 214 KiB/s wr, 18 op/s 2023-09-10T02:19:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10813.786178 secs 2023-09-10T02:19:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10815.177186 secs 2023-09-10T02:19:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10813.786178 secs 2023-09-10T02:19:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10815.177186 secs 2023-09-10T02:19:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10813.786178 secs 2023-09-10T02:19:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10815.177186 secs 2023-09-10T02:19:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:44 smithi098 ceph-mon[103124]: pgmap v5930: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 154 KiB/s wr, 12 op/s 2023-09-10T02:19:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:19:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:19:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:19:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10815.888863 secs 2023-09-10T02:19:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:44 smithi170 ceph-mon[102832]: pgmap v5930: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 154 KiB/s wr, 12 op/s 2023-09-10T02:19:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:19:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:19:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:19:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10815.888863 secs 2023-09-10T02:19:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:44 smithi082 ceph-mon[98694]: pgmap v5930: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 154 KiB/s wr, 12 op/s 2023-09-10T02:19:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:19:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:19:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:19:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10815.888863 secs 2023-09-10T02:19:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:46 smithi098 ceph-mon[103124]: pgmap v5931: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 236 KiB/s wr, 18 op/s 2023-09-10T02:19:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:46 smithi170 ceph-mon[102832]: pgmap v5931: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 236 KiB/s wr, 18 op/s 2023-09-10T02:19:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:46 smithi082 ceph-mon[98694]: pgmap v5931: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 236 KiB/s wr, 18 op/s 2023-09-10T02:19:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10818.786268 secs 2023-09-10T02:19:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:48 smithi170 ceph-mon[102832]: pgmap v5932: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 171 KiB/s wr, 12 op/s 2023-09-10T02:19:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10820.177246 secs 2023-09-10T02:19:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10818.786268 secs 2023-09-10T02:19:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:48 smithi082 ceph-mon[98694]: pgmap v5932: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 171 KiB/s wr, 12 op/s 2023-09-10T02:19:48.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10820.177246 secs 2023-09-10T02:19:49.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10818.786268 secs 2023-09-10T02:19:49.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:48 smithi098 ceph-mon[103124]: pgmap v5932: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 171 KiB/s wr, 12 op/s 2023-09-10T02:19:49.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10820.177246 secs 2023-09-10T02:19:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10820.888931 secs 2023-09-10T02:19:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10820.888931 secs 2023-09-10T02:19:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10820.888931 secs 2023-09-10T02:19:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:50 smithi170 ceph-mon[102832]: pgmap v5933: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 171 KiB/s wr, 12 op/s 2023-09-10T02:19:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:50 smithi082 ceph-mon[98694]: pgmap v5933: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 171 KiB/s wr, 12 op/s 2023-09-10T02:19:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:50 smithi098 ceph-mon[103124]: pgmap v5933: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 171 KiB/s wr, 12 op/s 2023-09-10T02:19:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:52 smithi098 ceph-mon[103124]: pgmap v5934: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 199 KiB/s wr, 18 op/s 2023-09-10T02:19:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:52 smithi170 ceph-mon[102832]: pgmap v5934: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 199 KiB/s wr, 18 op/s 2023-09-10T02:19:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:52 smithi082 ceph-mon[98694]: pgmap v5934: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 199 KiB/s wr, 18 op/s 2023-09-10T02:19:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10823.786330 secs 2023-09-10T02:19:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10825.177304 secs 2023-09-10T02:19:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10823.786330 secs 2023-09-10T02:19:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10825.177304 secs 2023-09-10T02:19:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10823.786330 secs 2023-09-10T02:19:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10825.177304 secs 2023-09-10T02:19:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:54 smithi098 ceph-mon[103124]: pgmap v5935: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 11 op/s 2023-09-10T02:19:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10825.888999 secs 2023-09-10T02:19:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:54 smithi170 ceph-mon[102832]: pgmap v5935: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 11 op/s 2023-09-10T02:19:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10825.888999 secs 2023-09-10T02:19:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:54 smithi082 ceph-mon[98694]: pgmap v5935: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 11 op/s 2023-09-10T02:19:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10825.888999 secs 2023-09-10T02:19:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:56 smithi098 ceph-mon[103124]: pgmap v5936: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 143 KiB/s wr, 16 op/s 2023-09-10T02:19:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:56 smithi170 ceph-mon[102832]: pgmap v5936: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 143 KiB/s wr, 16 op/s 2023-09-10T02:19:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:56 smithi082 ceph-mon[98694]: pgmap v5936: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 143 KiB/s wr, 16 op/s 2023-09-10T02:19:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10828.786409 secs 2023-09-10T02:19:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:58 smithi098 ceph-mon[103124]: pgmap v5937: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 10 op/s 2023-09-10T02:19:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10830.177361 secs 2023-09-10T02:19:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10828.786409 secs 2023-09-10T02:19:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:58 smithi170 ceph-mon[102832]: pgmap v5937: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 10 op/s 2023-09-10T02:19:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10830.177361 secs 2023-09-10T02:19:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10828.786409 secs 2023-09-10T02:19:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:58 smithi082 ceph-mon[98694]: pgmap v5937: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 10 op/s 2023-09-10T02:19:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10830.177361 secs 2023-09-10T02:19:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:19:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10830.889065 secs 2023-09-10T02:19:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:19:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10830.889065 secs 2023-09-10T02:20:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:19:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10830.889065 secs 2023-09-10T02:20:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:00 smithi170 ceph-mon[102832]: pgmap v5938: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 10 op/s 2023-09-10T02:20:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T02:20:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:00 smithi082 ceph-mon[98694]: pgmap v5938: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 10 op/s 2023-09-10T02:20:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T02:20:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:00 smithi098 ceph-mon[103124]: pgmap v5938: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 10 op/s 2023-09-10T02:20:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T02:20:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:02 smithi098 ceph-mon[103124]: pgmap v5939: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 85 KiB/s wr, 16 op/s 2023-09-10T02:20:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:02 smithi170 ceph-mon[102832]: pgmap v5939: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 85 KiB/s wr, 16 op/s 2023-09-10T02:20:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:02 smithi082 ceph-mon[98694]: pgmap v5939: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 85 KiB/s wr, 16 op/s 2023-09-10T02:20:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10833.786513 secs 2023-09-10T02:20:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10835.177417 secs 2023-09-10T02:20:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10833.786513 secs 2023-09-10T02:20:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10835.177417 secs 2023-09-10T02:20:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10833.786513 secs 2023-09-10T02:20:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10835.177417 secs 2023-09-10T02:20:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:04 smithi098 ceph-mon[103124]: pgmap v5940: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T02:20:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10835.889132 secs 2023-09-10T02:20:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:04 smithi170 ceph-mon[102832]: pgmap v5940: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T02:20:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10835.889132 secs 2023-09-10T02:20:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:04 smithi082 ceph-mon[98694]: pgmap v5940: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T02:20:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10835.889132 secs 2023-09-10T02:20:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:06 smithi098 ceph-mon[103124]: pgmap v5941: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 103 KiB/s wr, 29 op/s 2023-09-10T02:20:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:06 smithi170 ceph-mon[102832]: pgmap v5941: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 103 KiB/s wr, 29 op/s 2023-09-10T02:20:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:06 smithi082 ceph-mon[98694]: pgmap v5941: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 103 KiB/s wr, 29 op/s 2023-09-10T02:20:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10838.786572 secs 2023-09-10T02:20:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:08 smithi170 ceph-mon[102832]: pgmap v5942: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 69 KiB/s wr, 24 op/s 2023-09-10T02:20:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:08 smithi170 ceph-mon[102832]: osdmap e439: 12 total, 12 up, 12 in 2023-09-10T02:20:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10840.177475 secs 2023-09-10T02:20:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10838.786572 secs 2023-09-10T02:20:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:08 smithi082 ceph-mon[98694]: pgmap v5942: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 69 KiB/s wr, 24 op/s 2023-09-10T02:20:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:08 smithi082 ceph-mon[98694]: osdmap e439: 12 total, 12 up, 12 in 2023-09-10T02:20:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10840.177475 secs 2023-09-10T02:20:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10838.786572 secs 2023-09-10T02:20:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:08 smithi098 ceph-mon[103124]: pgmap v5942: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 69 KiB/s wr, 24 op/s 2023-09-10T02:20:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:08 smithi098 ceph-mon[103124]: osdmap e439: 12 total, 12 up, 12 in 2023-09-10T02:20:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10840.177475 secs 2023-09-10T02:20:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10840.889200 secs 2023-09-10T02:20:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10840.889200 secs 2023-09-10T02:20:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10840.889200 secs 2023-09-10T02:20:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:10 smithi170 ceph-mon[102832]: pgmap v5944: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T02:20:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:10 smithi082 ceph-mon[98694]: pgmap v5944: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T02:20:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:10 smithi098 ceph-mon[103124]: pgmap v5944: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 29 op/s 2023-09-10T02:20:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:12 smithi098 ceph-mon[103124]: pgmap v5945: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 56 KiB/s wr, 27 op/s 2023-09-10T02:20:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:12 smithi170 ceph-mon[102832]: pgmap v5945: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 56 KiB/s wr, 27 op/s 2023-09-10T02:20:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:12 smithi082 ceph-mon[98694]: pgmap v5945: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 56 KiB/s wr, 27 op/s 2023-09-10T02:20:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10843.786695 secs 2023-09-10T02:20:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10845.177494 secs 2023-09-10T02:20:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10843.786695 secs 2023-09-10T02:20:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10845.177494 secs 2023-09-10T02:20:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10843.786695 secs 2023-09-10T02:20:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10845.177494 secs 2023-09-10T02:20:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:14 smithi098 ceph-mon[103124]: pgmap v5946: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 56 KiB/s wr, 27 op/s 2023-09-10T02:20:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10845.889267 secs 2023-09-10T02:20:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:14 smithi170 ceph-mon[102832]: pgmap v5946: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 56 KiB/s wr, 27 op/s 2023-09-10T02:20:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10845.889267 secs 2023-09-10T02:20:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:14 smithi082 ceph-mon[98694]: pgmap v5946: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 56 KiB/s wr, 27 op/s 2023-09-10T02:20:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10845.889267 secs 2023-09-10T02:20:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:16 smithi098 ceph-mon[103124]: pgmap v5947: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:20:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:16 smithi098 ceph-mon[103124]: osdmap e440: 12 total, 12 up, 12 in 2023-09-10T02:20:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:16 smithi170 ceph-mon[102832]: pgmap v5947: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:20:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:16 smithi170 ceph-mon[102832]: osdmap e440: 12 total, 12 up, 12 in 2023-09-10T02:20:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:16 smithi082 ceph-mon[98694]: pgmap v5947: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:20:16.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:16 smithi082 ceph-mon[98694]: osdmap e440: 12 total, 12 up, 12 in 2023-09-10T02:20:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10848.786776 secs 2023-09-10T02:20:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:18 smithi098 ceph-mon[103124]: pgmap v5949: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:20:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10850.177555 secs 2023-09-10T02:20:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10848.786776 secs 2023-09-10T02:20:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:18 smithi170 ceph-mon[102832]: pgmap v5949: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:20:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10850.177555 secs 2023-09-10T02:20:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10848.786776 secs 2023-09-10T02:20:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:18 smithi082 ceph-mon[98694]: pgmap v5949: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:20:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10850.177555 secs 2023-09-10T02:20:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10850.889334 secs 2023-09-10T02:20:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10850.889334 secs 2023-09-10T02:20:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10850.889334 secs 2023-09-10T02:20:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:20 smithi170 ceph-mon[102832]: pgmap v5950: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:20:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:20 smithi082 ceph-mon[98694]: pgmap v5950: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:20:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:20 smithi098 ceph-mon[103124]: pgmap v5950: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:20:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:22 smithi098 ceph-mon[103124]: pgmap v5951: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:20:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:22 smithi170 ceph-mon[102832]: pgmap v5951: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:20:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:22 smithi082 ceph-mon[98694]: pgmap v5951: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:20:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:20:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10855.177612 secs 2023-09-10T02:20:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:20:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10855.177612 secs 2023-09-10T02:20:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:20:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10855.177612 secs 2023-09-10T02:20:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:24 smithi098 ceph-mon[103124]: pgmap v5952: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:20:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:20:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:20:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:20:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10855.889397 secs 2023-09-10T02:20:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:24 smithi170 ceph-mon[102832]: pgmap v5952: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:20:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:20:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:20:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:20:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10855.889397 secs 2023-09-10T02:20:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:24 smithi082 ceph-mon[98694]: pgmap v5952: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-09-10T02:20:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:20:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:20:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:20:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10855.889397 secs 2023-09-10T02:20:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:26 smithi098 ceph-mon[103124]: pgmap v5953: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 12 op/s 2023-09-10T02:20:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:26 smithi170 ceph-mon[102832]: pgmap v5953: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 12 op/s 2023-09-10T02:20:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:26 smithi082 ceph-mon[98694]: pgmap v5953: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 12 op/s 2023-09-10T02:20:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:28 smithi098 ceph-mon[103124]: pgmap v5954: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 10 op/s 2023-09-10T02:20:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10860.177675 secs 2023-09-10T02:20:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:28 smithi170 ceph-mon[102832]: pgmap v5954: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 10 op/s 2023-09-10T02:20:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10860.177675 secs 2023-09-10T02:20:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:28 smithi082 ceph-mon[98694]: pgmap v5954: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 10 op/s 2023-09-10T02:20:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10860.177675 secs 2023-09-10T02:20:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10860.889463 secs 2023-09-10T02:20:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10860.889463 secs 2023-09-10T02:20:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10860.889463 secs 2023-09-10T02:20:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:30 smithi170 ceph-mon[102832]: pgmap v5955: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 10 op/s 2023-09-10T02:20:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:30 smithi082 ceph-mon[98694]: pgmap v5955: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 10 op/s 2023-09-10T02:20:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:30 smithi098 ceph-mon[103124]: pgmap v5955: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 10 op/s 2023-09-10T02:20:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:32 smithi098 ceph-mon[103124]: pgmap v5956: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 126 KiB/s wr, 16 op/s 2023-09-10T02:20:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:20:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:20:32.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:20:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:32 smithi170 ceph-mon[102832]: pgmap v5956: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 126 KiB/s wr, 16 op/s 2023-09-10T02:20:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:20:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:20:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:20:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:32 smithi082 ceph-mon[98694]: pgmap v5956: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 126 KiB/s wr, 16 op/s 2023-09-10T02:20:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:20:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:20:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:20:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:20:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:20:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:20:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10865.177734 secs 2023-09-10T02:20:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:20:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:20:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:20:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10865.177734 secs 2023-09-10T02:20:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:20:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:20:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:20:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10865.177734 secs 2023-09-10T02:20:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:34 smithi098 ceph-mon[103124]: pgmap v5957: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T02:20:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10865.889490 secs 2023-09-10T02:20:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:34 smithi170 ceph-mon[102832]: pgmap v5957: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T02:20:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10865.889490 secs 2023-09-10T02:20:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:34 smithi082 ceph-mon[98694]: pgmap v5957: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T02:20:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10865.889490 secs 2023-09-10T02:20:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:36 smithi098 ceph-mon[103124]: pgmap v5958: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 179 KiB/s wr, 18 op/s 2023-09-10T02:20:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:36 smithi170 ceph-mon[102832]: pgmap v5958: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 179 KiB/s wr, 18 op/s 2023-09-10T02:20:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:36 smithi082 ceph-mon[98694]: pgmap v5958: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 179 KiB/s wr, 18 op/s 2023-09-10T02:20:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:38 smithi098 ceph-mon[103124]: pgmap v5959: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T02:20:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10870.177811 secs 2023-09-10T02:20:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:38 smithi170 ceph-mon[102832]: pgmap v5959: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T02:20:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10870.177811 secs 2023-09-10T02:20:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:38 smithi082 ceph-mon[98694]: pgmap v5959: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T02:20:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10870.177811 secs 2023-09-10T02:20:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10870.889557 secs 2023-09-10T02:20:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10870.889557 secs 2023-09-10T02:20:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10870.889557 secs 2023-09-10T02:20:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:40 smithi098 ceph-mon[103124]: pgmap v5960: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T02:20:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:40 smithi170 ceph-mon[102832]: pgmap v5960: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T02:20:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:40 smithi082 ceph-mon[98694]: pgmap v5960: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T02:20:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:42 smithi098 ceph-mon[103124]: pgmap v5961: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 220 KiB/s wr, 19 op/s 2023-09-10T02:20:42.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:42 smithi170 ceph-mon[102832]: pgmap v5961: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 220 KiB/s wr, 19 op/s 2023-09-10T02:20:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:42 smithi082 ceph-mon[98694]: pgmap v5961: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 220 KiB/s wr, 19 op/s 2023-09-10T02:20:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10875.177871 secs 2023-09-10T02:20:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10875.177871 secs 2023-09-10T02:20:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10875.177871 secs 2023-09-10T02:20:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:44 smithi170 ceph-mon[102832]: pgmap v5962: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T02:20:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10875.889624 secs 2023-09-10T02:20:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:44 smithi082 ceph-mon[98694]: pgmap v5962: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T02:20:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10875.889624 secs 2023-09-10T02:20:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:44 smithi098 ceph-mon[103124]: pgmap v5962: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T02:20:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10875.889624 secs 2023-09-10T02:20:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:46 smithi170 ceph-mon[102832]: pgmap v5963: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 204 KiB/s wr, 18 op/s 2023-09-10T02:20:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:46 smithi082 ceph-mon[98694]: pgmap v5963: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 204 KiB/s wr, 18 op/s 2023-09-10T02:20:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:46 smithi098 ceph-mon[103124]: pgmap v5963: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 204 KiB/s wr, 18 op/s 2023-09-10T02:20:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10878.787273 secs 2023-09-10T02:20:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:48 smithi170 ceph-mon[102832]: pgmap v5964: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 152 KiB/s wr, 12 op/s 2023-09-10T02:20:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10880.177930 secs 2023-09-10T02:20:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10878.787273 secs 2023-09-10T02:20:48.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:48 smithi082 ceph-mon[98694]: pgmap v5964: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 152 KiB/s wr, 12 op/s 2023-09-10T02:20:48.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10880.177930 secs 2023-09-10T02:20:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10878.787273 secs 2023-09-10T02:20:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:48 smithi098 ceph-mon[103124]: pgmap v5964: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 152 KiB/s wr, 12 op/s 2023-09-10T02:20:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10880.177930 secs 2023-09-10T02:20:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10880.889691 secs 2023-09-10T02:20:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10880.889691 secs 2023-09-10T02:20:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10880.889691 secs 2023-09-10T02:20:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:50 smithi170 ceph-mon[102832]: pgmap v5965: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 152 KiB/s wr, 12 op/s 2023-09-10T02:20:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:50 smithi082 ceph-mon[98694]: pgmap v5965: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 152 KiB/s wr, 12 op/s 2023-09-10T02:20:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:50 smithi098 ceph-mon[103124]: pgmap v5965: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 152 KiB/s wr, 12 op/s 2023-09-10T02:20:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:52 smithi098 ceph-mon[103124]: pgmap v5966: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 182 KiB/s wr, 17 op/s 2023-09-10T02:20:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:20:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:20:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:20:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:52 smithi170 ceph-mon[102832]: pgmap v5966: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 182 KiB/s wr, 17 op/s 2023-09-10T02:20:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:20:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:20:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:20:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:52 smithi082 ceph-mon[98694]: pgmap v5966: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 182 KiB/s wr, 17 op/s 2023-09-10T02:20:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:20:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:20:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:20:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10883.787338 secs 2023-09-10T02:20:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10885.177989 secs 2023-09-10T02:20:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10883.787338 secs 2023-09-10T02:20:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10885.177989 secs 2023-09-10T02:20:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10883.787338 secs 2023-09-10T02:20:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10885.177989 secs 2023-09-10T02:20:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:54 smithi170 ceph-mon[102832]: pgmap v5967: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T02:20:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10885.889754 secs 2023-09-10T02:20:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:54 smithi082 ceph-mon[98694]: pgmap v5967: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T02:20:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10885.889754 secs 2023-09-10T02:20:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:54 smithi098 ceph-mon[103124]: pgmap v5967: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T02:20:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10885.889754 secs 2023-09-10T02:20:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:56 smithi170 ceph-mon[102832]: pgmap v5968: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 135 KiB/s wr, 16 op/s 2023-09-10T02:20:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:56 smithi082 ceph-mon[98694]: pgmap v5968: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 135 KiB/s wr, 16 op/s 2023-09-10T02:20:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:56 smithi098 ceph-mon[103124]: pgmap v5968: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 135 KiB/s wr, 16 op/s 2023-09-10T02:20:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10888.787425 secs 2023-09-10T02:20:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:58 smithi170 ceph-mon[102832]: pgmap v5969: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 77 KiB/s wr, 10 op/s 2023-09-10T02:20:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10890.178047 secs 2023-09-10T02:20:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10888.787425 secs 2023-09-10T02:20:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:58 smithi082 ceph-mon[98694]: pgmap v5969: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 77 KiB/s wr, 10 op/s 2023-09-10T02:20:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10890.178047 secs 2023-09-10T02:20:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10888.787425 secs 2023-09-10T02:20:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:58 smithi098 ceph-mon[103124]: pgmap v5969: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 77 KiB/s wr, 10 op/s 2023-09-10T02:20:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10890.178047 secs 2023-09-10T02:20:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:20:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10890.889820 secs 2023-09-10T02:20:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:20:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10890.889820 secs 2023-09-10T02:21:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:20:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10890.889820 secs 2023-09-10T02:21:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:00 smithi170 ceph-mon[102832]: pgmap v5970: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 77 KiB/s wr, 10 op/s 2023-09-10T02:21:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:00 smithi082 ceph-mon[98694]: pgmap v5970: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 77 KiB/s wr, 10 op/s 2023-09-10T02:21:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:00 smithi098 ceph-mon[103124]: pgmap v5970: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 77 KiB/s wr, 10 op/s 2023-09-10T02:21:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:02 smithi170 ceph-mon[102832]: pgmap v5971: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 96 KiB/s wr, 15 op/s 2023-09-10T02:21:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:02 smithi082 ceph-mon[98694]: pgmap v5971: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 96 KiB/s wr, 15 op/s 2023-09-10T02:21:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:02 smithi098 ceph-mon[103124]: pgmap v5971: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 96 KiB/s wr, 15 op/s 2023-09-10T02:21:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10893.787579 secs 2023-09-10T02:21:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10895.178107 secs 2023-09-10T02:21:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10893.787579 secs 2023-09-10T02:21:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10895.178107 secs 2023-09-10T02:21:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10893.787579 secs 2023-09-10T02:21:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10895.178107 secs 2023-09-10T02:21:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:04 smithi170 ceph-mon[102832]: pgmap v5972: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 9 op/s 2023-09-10T02:21:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10895.889888 secs 2023-09-10T02:21:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:04 smithi082 ceph-mon[98694]: pgmap v5972: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 9 op/s 2023-09-10T02:21:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10895.889888 secs 2023-09-10T02:21:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:04 smithi098 ceph-mon[103124]: pgmap v5972: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 9 op/s 2023-09-10T02:21:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10895.889888 secs 2023-09-10T02:21:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:06 smithi170 ceph-mon[102832]: pgmap v5973: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 111 KiB/s wr, 29 op/s 2023-09-10T02:21:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:06 smithi082 ceph-mon[98694]: pgmap v5973: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 111 KiB/s wr, 29 op/s 2023-09-10T02:21:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:06 smithi098 ceph-mon[103124]: pgmap v5973: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 111 KiB/s wr, 29 op/s 2023-09-10T02:21:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10898.787660 secs 2023-09-10T02:21:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10898.787660 secs 2023-09-10T02:21:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10898.787660 secs 2023-09-10T02:21:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:08 smithi170 ceph-mon[102832]: pgmap v5974: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 65 KiB/s wr, 24 op/s 2023-09-10T02:21:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:08 smithi170 ceph-mon[102832]: osdmap e441: 12 total, 12 up, 12 in 2023-09-10T02:21:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10900.178166 secs 2023-09-10T02:21:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:08 smithi082 ceph-mon[98694]: pgmap v5974: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 65 KiB/s wr, 24 op/s 2023-09-10T02:21:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:08 smithi082 ceph-mon[98694]: osdmap e441: 12 total, 12 up, 12 in 2023-09-10T02:21:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10900.178166 secs 2023-09-10T02:21:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:08 smithi098 ceph-mon[103124]: pgmap v5974: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 65 KiB/s wr, 24 op/s 2023-09-10T02:21:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:08 smithi098 ceph-mon[103124]: osdmap e441: 12 total, 12 up, 12 in 2023-09-10T02:21:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10900.178166 secs 2023-09-10T02:21:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10900.889954 secs 2023-09-10T02:21:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10900.889954 secs 2023-09-10T02:21:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10900.889954 secs 2023-09-10T02:21:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:10 smithi170 ceph-mon[102832]: pgmap v5976: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T02:21:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:10 smithi082 ceph-mon[98694]: pgmap v5976: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T02:21:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:10 smithi098 ceph-mon[103124]: pgmap v5976: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T02:21:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:12 smithi170 ceph-mon[102832]: pgmap v5977: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T02:21:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:12 smithi082 ceph-mon[98694]: pgmap v5977: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T02:21:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:12 smithi098 ceph-mon[103124]: pgmap v5977: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T02:21:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10903.787805 secs 2023-09-10T02:21:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10905.178260 secs 2023-09-10T02:21:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10903.787805 secs 2023-09-10T02:21:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10905.178260 secs 2023-09-10T02:21:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10903.787805 secs 2023-09-10T02:21:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10905.178260 secs 2023-09-10T02:21:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:14 smithi170 ceph-mon[102832]: pgmap v5978: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T02:21:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10905.890023 secs 2023-09-10T02:21:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:14 smithi082 ceph-mon[98694]: pgmap v5978: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T02:21:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10905.890023 secs 2023-09-10T02:21:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:14 smithi098 ceph-mon[103124]: pgmap v5978: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T02:21:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10905.890023 secs 2023-09-10T02:21:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:16 smithi170 ceph-mon[102832]: pgmap v5979: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T02:21:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:16 smithi170 ceph-mon[102832]: osdmap e442: 12 total, 12 up, 12 in 2023-09-10T02:21:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:16 smithi082 ceph-mon[98694]: pgmap v5979: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T02:21:16.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:16 smithi082 ceph-mon[98694]: osdmap e442: 12 total, 12 up, 12 in 2023-09-10T02:21:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:16 smithi098 ceph-mon[103124]: pgmap v5979: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T02:21:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:16 smithi098 ceph-mon[103124]: osdmap e442: 12 total, 12 up, 12 in 2023-09-10T02:21:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10908.787864 secs 2023-09-10T02:21:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10908.787864 secs 2023-09-10T02:21:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10908.787864 secs 2023-09-10T02:21:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:18 smithi170 ceph-mon[102832]: pgmap v5981: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T02:21:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10910.178342 secs 2023-09-10T02:21:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:18 smithi082 ceph-mon[98694]: pgmap v5981: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T02:21:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10910.178342 secs 2023-09-10T02:21:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:18 smithi098 ceph-mon[103124]: pgmap v5981: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T02:21:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10910.178342 secs 2023-09-10T02:21:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10910.890086 secs 2023-09-10T02:21:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10910.890086 secs 2023-09-10T02:21:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10910.890086 secs 2023-09-10T02:21:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:20 smithi170 ceph-mon[102832]: pgmap v5982: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T02:21:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:20 smithi082 ceph-mon[98694]: pgmap v5982: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T02:21:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:20 smithi098 ceph-mon[103124]: pgmap v5982: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T02:21:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:22 smithi170 ceph-mon[102832]: pgmap v5983: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T02:21:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:22 smithi082 ceph-mon[98694]: pgmap v5983: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T02:21:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:22 smithi098 ceph-mon[103124]: pgmap v5983: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T02:21:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:21:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10915.178423 secs 2023-09-10T02:21:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:21:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10915.178423 secs 2023-09-10T02:21:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:21:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10915.178423 secs 2023-09-10T02:21:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:24 smithi170 ceph-mon[102832]: pgmap v5984: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T02:21:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:21:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:21:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:21:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10915.890154 secs 2023-09-10T02:21:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:24 smithi082 ceph-mon[98694]: pgmap v5984: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T02:21:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:21:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:21:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:21:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10915.890154 secs 2023-09-10T02:21:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:24 smithi098 ceph-mon[103124]: pgmap v5984: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T02:21:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:21:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:21:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:21:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10915.890154 secs 2023-09-10T02:21:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:26 smithi170 ceph-mon[102832]: pgmap v5985: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T02:21:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:26 smithi082 ceph-mon[98694]: pgmap v5985: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T02:21:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:26 smithi098 ceph-mon[103124]: pgmap v5985: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T02:21:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:28 smithi170 ceph-mon[102832]: pgmap v5986: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T02:21:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10920.178535 secs 2023-09-10T02:21:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:28 smithi082 ceph-mon[98694]: pgmap v5986: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T02:21:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10920.178535 secs 2023-09-10T02:21:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:28 smithi098 ceph-mon[103124]: pgmap v5986: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T02:21:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10920.178535 secs 2023-09-10T02:21:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10920.890224 secs 2023-09-10T02:21:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10920.890224 secs 2023-09-10T02:21:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10920.890224 secs 2023-09-10T02:21:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:30 smithi170 ceph-mon[102832]: pgmap v5987: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 79 KiB/s wr, 10 op/s 2023-09-10T02:21:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:30 smithi082 ceph-mon[98694]: pgmap v5987: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 79 KiB/s wr, 10 op/s 2023-09-10T02:21:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:30 smithi098 ceph-mon[103124]: pgmap v5987: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 79 KiB/s wr, 10 op/s 2023-09-10T02:21:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:32 smithi170 ceph-mon[102832]: pgmap v5988: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 153 KiB/s wr, 17 op/s 2023-09-10T02:21:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:32 smithi082 ceph-mon[98694]: pgmap v5988: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 153 KiB/s wr, 17 op/s 2023-09-10T02:21:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:32 smithi098 ceph-mon[103124]: pgmap v5988: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 153 KiB/s wr, 17 op/s 2023-09-10T02:21:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:21:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10925.178595 secs 2023-09-10T02:21:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:21:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:21:33.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:21:33.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:21:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:21:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10925.178595 secs 2023-09-10T02:21:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:21:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:21:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:21:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:21:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:21:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10925.178595 secs 2023-09-10T02:21:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:21:34.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:21:34.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:21:34.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:21:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:34 smithi170 ceph-mon[102832]: pgmap v5989: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 12 op/s 2023-09-10T02:21:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10925.890291 secs 2023-09-10T02:21:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:21:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:21:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:21:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:34 smithi082 ceph-mon[98694]: pgmap v5989: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 12 op/s 2023-09-10T02:21:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10925.890291 secs 2023-09-10T02:21:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:21:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:21:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:21:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:34 smithi098 ceph-mon[103124]: pgmap v5989: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 12 op/s 2023-09-10T02:21:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10925.890291 secs 2023-09-10T02:21:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:21:35.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:21:35.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:21:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:36 smithi170 ceph-mon[102832]: pgmap v5990: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 194 KiB/s wr, 18 op/s 2023-09-10T02:21:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:36 smithi082 ceph-mon[98694]: pgmap v5990: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 194 KiB/s wr, 18 op/s 2023-09-10T02:21:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:36 smithi098 ceph-mon[103124]: pgmap v5990: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 194 KiB/s wr, 18 op/s 2023-09-10T02:21:38.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:38 smithi170 ceph-mon[102832]: pgmap v5991: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 179 KiB/s wr, 12 op/s 2023-09-10T02:21:38.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10930.178653 secs 2023-09-10T02:21:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:38 smithi082 ceph-mon[98694]: pgmap v5991: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 179 KiB/s wr, 12 op/s 2023-09-10T02:21:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10930.178653 secs 2023-09-10T02:21:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:38 smithi098 ceph-mon[103124]: pgmap v5991: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 179 KiB/s wr, 12 op/s 2023-09-10T02:21:39.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10930.178653 secs 2023-09-10T02:21:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10930.890355 secs 2023-09-10T02:21:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10930.890355 secs 2023-09-10T02:21:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10930.890355 secs 2023-09-10T02:21:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:40 smithi170 ceph-mon[102832]: pgmap v5992: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 179 KiB/s wr, 12 op/s 2023-09-10T02:21:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:40 smithi082 ceph-mon[98694]: pgmap v5992: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 179 KiB/s wr, 12 op/s 2023-09-10T02:21:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:40 smithi098 ceph-mon[103124]: pgmap v5992: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 179 KiB/s wr, 12 op/s 2023-09-10T02:21:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:42 smithi170 ceph-mon[102832]: pgmap v5993: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 269 KiB/s wr, 19 op/s 2023-09-10T02:21:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:42 smithi082 ceph-mon[98694]: pgmap v5993: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 269 KiB/s wr, 19 op/s 2023-09-10T02:21:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:42 smithi098 ceph-mon[103124]: pgmap v5993: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 269 KiB/s wr, 19 op/s 2023-09-10T02:21:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10933.788288 secs 2023-09-10T02:21:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10935.178728 secs 2023-09-10T02:21:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10933.788288 secs 2023-09-10T02:21:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10935.178728 secs 2023-09-10T02:21:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10933.788288 secs 2023-09-10T02:21:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10935.178728 secs 2023-09-10T02:21:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:44 smithi170 ceph-mon[102832]: pgmap v5994: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 195 KiB/s wr, 12 op/s 2023-09-10T02:21:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:21:44.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:21:44.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:21:44.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10935.890421 secs 2023-09-10T02:21:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:44 smithi082 ceph-mon[98694]: pgmap v5994: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 195 KiB/s wr, 12 op/s 2023-09-10T02:21:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:21:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:21:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:21:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10935.890421 secs 2023-09-10T02:21:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:44 smithi098 ceph-mon[103124]: pgmap v5994: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 195 KiB/s wr, 12 op/s 2023-09-10T02:21:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:21:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:21:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:21:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10935.890421 secs 2023-09-10T02:21:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:46 smithi170 ceph-mon[102832]: pgmap v5995: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 239 KiB/s wr, 17 op/s 2023-09-10T02:21:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:46 smithi082 ceph-mon[98694]: pgmap v5995: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 239 KiB/s wr, 17 op/s 2023-09-10T02:21:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:46 smithi098 ceph-mon[103124]: pgmap v5995: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 239 KiB/s wr, 17 op/s 2023-09-10T02:21:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10938.788348 secs 2023-09-10T02:21:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10938.788348 secs 2023-09-10T02:21:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10938.788348 secs 2023-09-10T02:21:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:48 smithi170 ceph-mon[102832]: pgmap v5996: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 134 KiB/s wr, 11 op/s 2023-09-10T02:21:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10940.178786 secs 2023-09-10T02:21:48.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:48 smithi082 ceph-mon[98694]: pgmap v5996: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 134 KiB/s wr, 11 op/s 2023-09-10T02:21:48.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10940.178786 secs 2023-09-10T02:21:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:48 smithi098 ceph-mon[103124]: pgmap v5996: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 134 KiB/s wr, 11 op/s 2023-09-10T02:21:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10940.178786 secs 2023-09-10T02:21:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10940.890491 secs 2023-09-10T02:21:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10940.890491 secs 2023-09-10T02:21:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10940.890491 secs 2023-09-10T02:21:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:50 smithi170 ceph-mon[102832]: pgmap v5997: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 134 KiB/s wr, 11 op/s 2023-09-10T02:21:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:50 smithi082 ceph-mon[98694]: pgmap v5997: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 134 KiB/s wr, 11 op/s 2023-09-10T02:21:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:50 smithi098 ceph-mon[103124]: pgmap v5997: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 134 KiB/s wr, 11 op/s 2023-09-10T02:21:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:52 smithi170 ceph-mon[102832]: pgmap v5998: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 165 KiB/s wr, 16 op/s 2023-09-10T02:21:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:52 smithi082 ceph-mon[98694]: pgmap v5998: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 165 KiB/s wr, 16 op/s 2023-09-10T02:21:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:52 smithi098 ceph-mon[103124]: pgmap v5998: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 165 KiB/s wr, 16 op/s 2023-09-10T02:21:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10943.788408 secs 2023-09-10T02:21:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10945.178845 secs 2023-09-10T02:21:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10943.788408 secs 2023-09-10T02:21:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10945.178845 secs 2023-09-10T02:21:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10943.788408 secs 2023-09-10T02:21:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10945.178845 secs 2023-09-10T02:21:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:54 smithi170 ceph-mon[102832]: pgmap v5999: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T02:21:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10945.890557 secs 2023-09-10T02:21:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:54 smithi082 ceph-mon[98694]: pgmap v5999: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T02:21:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10945.890557 secs 2023-09-10T02:21:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:54 smithi098 ceph-mon[103124]: pgmap v5999: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T02:21:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10945.890557 secs 2023-09-10T02:21:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:56 smithi170 ceph-mon[102832]: pgmap v6000: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 76 KiB/s wr, 14 op/s 2023-09-10T02:21:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:56 smithi082 ceph-mon[98694]: pgmap v6000: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 76 KiB/s wr, 14 op/s 2023-09-10T02:21:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:56 smithi098 ceph-mon[103124]: pgmap v6000: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 76 KiB/s wr, 14 op/s 2023-09-10T02:21:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10948.788477 secs 2023-09-10T02:21:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10948.788477 secs 2023-09-10T02:21:58.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10948.788477 secs 2023-09-10T02:21:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:58 smithi170 ceph-mon[102832]: pgmap v6001: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 31 KiB/s wr, 9 op/s 2023-09-10T02:21:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10950.178929 secs 2023-09-10T02:21:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:58 smithi082 ceph-mon[98694]: pgmap v6001: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 31 KiB/s wr, 9 op/s 2023-09-10T02:21:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10950.178929 secs 2023-09-10T02:21:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:58 smithi098 ceph-mon[103124]: pgmap v6001: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 31 KiB/s wr, 9 op/s 2023-09-10T02:21:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10950.178929 secs 2023-09-10T02:21:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:21:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10950.890625 secs 2023-09-10T02:21:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:21:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10950.890625 secs 2023-09-10T02:22:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:21:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10950.890625 secs 2023-09-10T02:22:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:00 smithi170 ceph-mon[102832]: pgmap v6002: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 31 KiB/s wr, 9 op/s 2023-09-10T02:22:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:00 smithi082 ceph-mon[98694]: pgmap v6002: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 31 KiB/s wr, 9 op/s 2023-09-10T02:22:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:00 smithi098 ceph-mon[103124]: pgmap v6002: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 31 KiB/s wr, 9 op/s 2023-09-10T02:22:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:02 smithi170 ceph-mon[102832]: pgmap v6003: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 55 KiB/s wr, 14 op/s 2023-09-10T02:22:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:02 smithi082 ceph-mon[98694]: pgmap v6003: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 55 KiB/s wr, 14 op/s 2023-09-10T02:22:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:02 smithi098 ceph-mon[103124]: pgmap v6003: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 55 KiB/s wr, 14 op/s 2023-09-10T02:22:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10953.788535 secs 2023-09-10T02:22:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10955.178986 secs 2023-09-10T02:22:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10953.788535 secs 2023-09-10T02:22:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10955.178986 secs 2023-09-10T02:22:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10953.788535 secs 2023-09-10T02:22:04.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10955.178986 secs 2023-09-10T02:22:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:04 smithi170 ceph-mon[102832]: pgmap v6004: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 9 op/s 2023-09-10T02:22:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10955.890690 secs 2023-09-10T02:22:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:04 smithi082 ceph-mon[98694]: pgmap v6004: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 9 op/s 2023-09-10T02:22:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10955.890690 secs 2023-09-10T02:22:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:04 smithi098 ceph-mon[103124]: pgmap v6004: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 9 op/s 2023-09-10T02:22:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10955.890690 secs 2023-09-10T02:22:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:06 smithi170 ceph-mon[102832]: pgmap v6005: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T02:22:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:06 smithi082 ceph-mon[98694]: pgmap v6005: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T02:22:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:06 smithi098 ceph-mon[103124]: pgmap v6005: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T02:22:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10958.788622 secs 2023-09-10T02:22:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10958.788622 secs 2023-09-10T02:22:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10958.788622 secs 2023-09-10T02:22:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:08 smithi170 ceph-mon[102832]: pgmap v6006: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 78 KiB/s wr, 24 op/s 2023-09-10T02:22:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:08 smithi170 ceph-mon[102832]: osdmap e443: 12 total, 12 up, 12 in 2023-09-10T02:22:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10960.179047 secs 2023-09-10T02:22:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:08 smithi082 ceph-mon[98694]: pgmap v6006: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 78 KiB/s wr, 24 op/s 2023-09-10T02:22:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:08 smithi082 ceph-mon[98694]: osdmap e443: 12 total, 12 up, 12 in 2023-09-10T02:22:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10960.179047 secs 2023-09-10T02:22:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:08 smithi098 ceph-mon[103124]: pgmap v6006: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 78 KiB/s wr, 24 op/s 2023-09-10T02:22:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:08 smithi098 ceph-mon[103124]: osdmap e443: 12 total, 12 up, 12 in 2023-09-10T02:22:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10960.179047 secs 2023-09-10T02:22:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10960.890754 secs 2023-09-10T02:22:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10960.890754 secs 2023-09-10T02:22:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10960.890754 secs 2023-09-10T02:22:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:10 smithi170 ceph-mon[102832]: pgmap v6008: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T02:22:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:10 smithi082 ceph-mon[98694]: pgmap v6008: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T02:22:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:10 smithi098 ceph-mon[103124]: pgmap v6008: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 93 KiB/s wr, 29 op/s 2023-09-10T02:22:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:12 smithi170 ceph-mon[102832]: pgmap v6009: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 29 op/s 2023-09-10T02:22:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10963.788682 secs 2023-09-10T02:22:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:12 smithi082 ceph-mon[98694]: pgmap v6009: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 29 op/s 2023-09-10T02:22:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10963.788682 secs 2023-09-10T02:22:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:12 smithi098 ceph-mon[103124]: pgmap v6009: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 29 op/s 2023-09-10T02:22:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10963.788682 secs 2023-09-10T02:22:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10965.179104 secs 2023-09-10T02:22:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10965.179104 secs 2023-09-10T02:22:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10965.179104 secs 2023-09-10T02:22:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:14 smithi170 ceph-mon[102832]: pgmap v6010: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 29 op/s 2023-09-10T02:22:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10965.890821 secs 2023-09-10T02:22:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:14 smithi082 ceph-mon[98694]: pgmap v6010: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 29 op/s 2023-09-10T02:22:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10965.890821 secs 2023-09-10T02:22:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:14 smithi098 ceph-mon[103124]: pgmap v6010: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 29 op/s 2023-09-10T02:22:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10965.890821 secs 2023-09-10T02:22:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:16 smithi170 ceph-mon[102832]: pgmap v6011: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 12 op/s 2023-09-10T02:22:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:16 smithi170 ceph-mon[102832]: osdmap e444: 12 total, 12 up, 12 in 2023-09-10T02:22:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:16 smithi082 ceph-mon[98694]: pgmap v6011: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 12 op/s 2023-09-10T02:22:16.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:16 smithi082 ceph-mon[98694]: osdmap e444: 12 total, 12 up, 12 in 2023-09-10T02:22:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:16 smithi098 ceph-mon[103124]: pgmap v6011: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 12 op/s 2023-09-10T02:22:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:16 smithi098 ceph-mon[103124]: osdmap e444: 12 total, 12 up, 12 in 2023-09-10T02:22:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10968.788773 secs 2023-09-10T02:22:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10968.788773 secs 2023-09-10T02:22:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10968.788773 secs 2023-09-10T02:22:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:18 smithi170 ceph-mon[102832]: pgmap v6013: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 12 op/s 2023-09-10T02:22:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10970.179163 secs 2023-09-10T02:22:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:18 smithi082 ceph-mon[98694]: pgmap v6013: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 12 op/s 2023-09-10T02:22:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10970.179163 secs 2023-09-10T02:22:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:18 smithi098 ceph-mon[103124]: pgmap v6013: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 12 op/s 2023-09-10T02:22:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10970.179163 secs 2023-09-10T02:22:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10970.890886 secs 2023-09-10T02:22:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10970.890886 secs 2023-09-10T02:22:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10970.890886 secs 2023-09-10T02:22:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:20 smithi170 ceph-mon[102832]: pgmap v6014: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 12 op/s 2023-09-10T02:22:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:20 smithi082 ceph-mon[98694]: pgmap v6014: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 12 op/s 2023-09-10T02:22:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:20 smithi098 ceph-mon[103124]: pgmap v6014: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 12 op/s 2023-09-10T02:22:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:22 smithi170 ceph-mon[102832]: pgmap v6015: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T02:22:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:22 smithi082 ceph-mon[98694]: pgmap v6015: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T02:22:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:22 smithi098 ceph-mon[103124]: pgmap v6015: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T02:22:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:22:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10975.179217 secs 2023-09-10T02:22:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:22:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10975.179217 secs 2023-09-10T02:22:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:22:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10975.179217 secs 2023-09-10T02:22:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:24 smithi170 ceph-mon[102832]: pgmap v6016: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T02:22:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:22:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:22:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:22:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10975.890952 secs 2023-09-10T02:22:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:24 smithi082 ceph-mon[98694]: pgmap v6016: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T02:22:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:22:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:22:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:22:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10975.890952 secs 2023-09-10T02:22:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:24 smithi098 ceph-mon[103124]: pgmap v6016: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T02:22:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:22:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:22:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:22:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10975.890952 secs 2023-09-10T02:22:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:26 smithi170 ceph-mon[102832]: pgmap v6017: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 13 op/s 2023-09-10T02:22:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:26 smithi082 ceph-mon[98694]: pgmap v6017: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 13 op/s 2023-09-10T02:22:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:26 smithi098 ceph-mon[103124]: pgmap v6017: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 13 op/s 2023-09-10T02:22:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:28 smithi170 ceph-mon[102832]: pgmap v6018: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 12 op/s 2023-09-10T02:22:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10980.179275 secs 2023-09-10T02:22:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:28 smithi082 ceph-mon[98694]: pgmap v6018: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 12 op/s 2023-09-10T02:22:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10980.179275 secs 2023-09-10T02:22:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:28 smithi098 ceph-mon[103124]: pgmap v6018: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 12 op/s 2023-09-10T02:22:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10980.179275 secs 2023-09-10T02:22:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10980.891022 secs 2023-09-10T02:22:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10980.891022 secs 2023-09-10T02:22:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10980.891022 secs 2023-09-10T02:22:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:30 smithi170 ceph-mon[102832]: pgmap v6019: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 54 KiB/s wr, 11 op/s 2023-09-10T02:22:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:30 smithi082 ceph-mon[98694]: pgmap v6019: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 54 KiB/s wr, 11 op/s 2023-09-10T02:22:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:30 smithi098 ceph-mon[103124]: pgmap v6019: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 54 KiB/s wr, 11 op/s 2023-09-10T02:22:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:32 smithi170 ceph-mon[102832]: pgmap v6020: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 158 KiB/s wr, 17 op/s 2023-09-10T02:22:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:32 smithi082 ceph-mon[98694]: pgmap v6020: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 158 KiB/s wr, 17 op/s 2023-09-10T02:22:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:32 smithi098 ceph-mon[103124]: pgmap v6020: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 158 KiB/s wr, 17 op/s 2023-09-10T02:22:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10985.179333 secs 2023-09-10T02:22:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10985.179333 secs 2023-09-10T02:22:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10985.179333 secs 2023-09-10T02:22:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:34 smithi170 ceph-mon[102832]: pgmap v6021: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 129 KiB/s wr, 12 op/s 2023-09-10T02:22:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10985.891085 secs 2023-09-10T02:22:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:22:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:34 smithi082 ceph-mon[98694]: pgmap v6021: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 129 KiB/s wr, 12 op/s 2023-09-10T02:22:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10985.891085 secs 2023-09-10T02:22:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:22:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:34 smithi098 ceph-mon[103124]: pgmap v6021: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 129 KiB/s wr, 12 op/s 2023-09-10T02:22:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10985.891085 secs 2023-09-10T02:22:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:22:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:22:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:22:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:22:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:22:35.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:22:35.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:22:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:22:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:22:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:22:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:36 smithi082 ceph-mon[98694]: pgmap v6022: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 155 KiB/s wr, 18 op/s 2023-09-10T02:22:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:36 smithi098 ceph-mon[103124]: pgmap v6022: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 155 KiB/s wr, 18 op/s 2023-09-10T02:22:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:36 smithi170 ceph-mon[102832]: pgmap v6022: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 155 KiB/s wr, 18 op/s 2023-09-10T02:22:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:38 smithi082 ceph-mon[98694]: pgmap v6023: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 129 KiB/s wr, 12 op/s 2023-09-10T02:22:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10990.179396 secs 2023-09-10T02:22:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:38 smithi098 ceph-mon[103124]: pgmap v6023: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 129 KiB/s wr, 12 op/s 2023-09-10T02:22:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10990.179396 secs 2023-09-10T02:22:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:38 smithi170 ceph-mon[102832]: pgmap v6023: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 129 KiB/s wr, 12 op/s 2023-09-10T02:22:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10990.179396 secs 2023-09-10T02:22:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10990.891156 secs 2023-09-10T02:22:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10990.891156 secs 2023-09-10T02:22:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10990.891156 secs 2023-09-10T02:22:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:40 smithi082 ceph-mon[98694]: pgmap v6024: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 129 KiB/s wr, 12 op/s 2023-09-10T02:22:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:40 smithi098 ceph-mon[103124]: pgmap v6024: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 129 KiB/s wr, 12 op/s 2023-09-10T02:22:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:40 smithi170 ceph-mon[102832]: pgmap v6024: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 129 KiB/s wr, 12 op/s 2023-09-10T02:22:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:42 smithi082 ceph-mon[98694]: pgmap v6025: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 224 KiB/s wr, 18 op/s 2023-09-10T02:22:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:42 smithi098 ceph-mon[103124]: pgmap v6025: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 224 KiB/s wr, 18 op/s 2023-09-10T02:22:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:42 smithi170 ceph-mon[102832]: pgmap v6025: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 224 KiB/s wr, 18 op/s 2023-09-10T02:22:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10995.179475 secs 2023-09-10T02:22:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10995.179475 secs 2023-09-10T02:22:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10995.179475 secs 2023-09-10T02:22:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:44 smithi098 ceph-mon[103124]: pgmap v6026: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T02:22:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 10995.891222 secs 2023-09-10T02:22:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:44 smithi170 ceph-mon[102832]: pgmap v6026: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T02:22:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 10995.891222 secs 2023-09-10T02:22:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:44 smithi082 ceph-mon[98694]: pgmap v6026: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T02:22:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 10995.891222 secs 2023-09-10T02:22:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:46 smithi098 ceph-mon[103124]: pgmap v6027: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 180 KiB/s wr, 17 op/s 2023-09-10T02:22:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:46 smithi170 ceph-mon[102832]: pgmap v6027: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 180 KiB/s wr, 17 op/s 2023-09-10T02:22:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:46 smithi082 ceph-mon[98694]: pgmap v6027: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 180 KiB/s wr, 17 op/s 2023-09-10T02:22:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 10998.789229 secs 2023-09-10T02:22:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 10998.789229 secs 2023-09-10T02:22:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 10998.789229 secs 2023-09-10T02:22:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:48 smithi098 ceph-mon[103124]: pgmap v6028: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T02:22:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11000.179556 secs 2023-09-10T02:22:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:48 smithi170 ceph-mon[102832]: pgmap v6028: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T02:22:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11000.179556 secs 2023-09-10T02:22:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:48 smithi082 ceph-mon[98694]: pgmap v6028: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T02:22:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11000.179556 secs 2023-09-10T02:22:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11000.891288 secs 2023-09-10T02:22:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11000.891288 secs 2023-09-10T02:22:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11000.891288 secs 2023-09-10T02:22:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:50 smithi098 ceph-mon[103124]: pgmap v6029: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T02:22:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:50 smithi170 ceph-mon[102832]: pgmap v6029: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T02:22:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:50 smithi082 ceph-mon[98694]: pgmap v6029: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T02:22:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:52 smithi170 ceph-mon[102832]: pgmap v6030: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 17 op/s 2023-09-10T02:22:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:22:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:22:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:22:52.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11003.789288 secs 2023-09-10T02:22:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:52 smithi082 ceph-mon[98694]: pgmap v6030: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 17 op/s 2023-09-10T02:22:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:22:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:22:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:22:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11003.789288 secs 2023-09-10T02:22:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:52 smithi098 ceph-mon[103124]: pgmap v6030: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 17 op/s 2023-09-10T02:22:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:22:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:22:53.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:22:53.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11003.789288 secs 2023-09-10T02:22:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11005.179612 secs 2023-09-10T02:22:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11005.179612 secs 2023-09-10T02:22:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11005.179612 secs 2023-09-10T02:22:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:54 smithi170 ceph-mon[102832]: pgmap v6031: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T02:22:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11005.891372 secs 2023-09-10T02:22:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:54 smithi082 ceph-mon[98694]: pgmap v6031: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T02:22:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11005.891372 secs 2023-09-10T02:22:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:54 smithi098 ceph-mon[103124]: pgmap v6031: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T02:22:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11005.891372 secs 2023-09-10T02:22:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:56 smithi170 ceph-mon[102832]: pgmap v6032: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 125 KiB/s wr, 16 op/s 2023-09-10T02:22:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:56 smithi082 ceph-mon[98694]: pgmap v6032: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 125 KiB/s wr, 16 op/s 2023-09-10T02:22:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:56 smithi098 ceph-mon[103124]: pgmap v6032: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 125 KiB/s wr, 16 op/s 2023-09-10T02:22:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11008.789372 secs 2023-09-10T02:22:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11008.789372 secs 2023-09-10T02:22:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11008.789372 secs 2023-09-10T02:22:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:58 smithi170 ceph-mon[102832]: pgmap v6033: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T02:22:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11010.179662 secs 2023-09-10T02:22:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:58 smithi082 ceph-mon[98694]: pgmap v6033: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T02:22:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11010.179662 secs 2023-09-10T02:22:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:58 smithi098 ceph-mon[103124]: pgmap v6033: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T02:22:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11010.179662 secs 2023-09-10T02:22:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:22:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11010.891438 secs 2023-09-10T02:22:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:22:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11010.891438 secs 2023-09-10T02:23:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:22:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11010.891438 secs 2023-09-10T02:23:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:00 smithi170 ceph-mon[102832]: pgmap v6034: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T02:23:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:00 smithi082 ceph-mon[98694]: pgmap v6034: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T02:23:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:00 smithi098 ceph-mon[103124]: pgmap v6034: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T02:23:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:02 smithi170 ceph-mon[102832]: pgmap v6035: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 92 KiB/s wr, 16 op/s 2023-09-10T02:23:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11013.789431 secs 2023-09-10T02:23:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:02 smithi082 ceph-mon[98694]: pgmap v6035: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 92 KiB/s wr, 16 op/s 2023-09-10T02:23:02.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11013.789431 secs 2023-09-10T02:23:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:02 smithi098 ceph-mon[103124]: pgmap v6035: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 92 KiB/s wr, 16 op/s 2023-09-10T02:23:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11013.789431 secs 2023-09-10T02:23:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11015.179719 secs 2023-09-10T02:23:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11015.179719 secs 2023-09-10T02:23:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11015.179719 secs 2023-09-10T02:23:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:04 smithi170 ceph-mon[102832]: pgmap v6036: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T02:23:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11015.891487 secs 2023-09-10T02:23:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:04 smithi082 ceph-mon[98694]: pgmap v6036: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T02:23:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11015.891487 secs 2023-09-10T02:23:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:04 smithi098 ceph-mon[103124]: pgmap v6036: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T02:23:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11015.891487 secs 2023-09-10T02:23:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:06 smithi170 ceph-mon[102832]: pgmap v6037: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 112 KiB/s wr, 29 op/s 2023-09-10T02:23:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:06 smithi082 ceph-mon[98694]: pgmap v6037: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 112 KiB/s wr, 29 op/s 2023-09-10T02:23:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:06 smithi098 ceph-mon[103124]: pgmap v6037: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 112 KiB/s wr, 29 op/s 2023-09-10T02:23:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11018.789491 secs 2023-09-10T02:23:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11018.789491 secs 2023-09-10T02:23:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11018.789491 secs 2023-09-10T02:23:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:08 smithi170 ceph-mon[102832]: pgmap v6038: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 73 KiB/s wr, 24 op/s 2023-09-10T02:23:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:08 smithi170 ceph-mon[102832]: osdmap e445: 12 total, 12 up, 12 in 2023-09-10T02:23:08.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11020.179804 secs 2023-09-10T02:23:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:08 smithi082 ceph-mon[98694]: pgmap v6038: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 73 KiB/s wr, 24 op/s 2023-09-10T02:23:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:08 smithi082 ceph-mon[98694]: osdmap e445: 12 total, 12 up, 12 in 2023-09-10T02:23:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11020.179804 secs 2023-09-10T02:23:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:08 smithi098 ceph-mon[103124]: pgmap v6038: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 73 KiB/s wr, 24 op/s 2023-09-10T02:23:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:08 smithi098 ceph-mon[103124]: osdmap e445: 12 total, 12 up, 12 in 2023-09-10T02:23:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11020.179804 secs 2023-09-10T02:23:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11020.891550 secs 2023-09-10T02:23:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11020.891550 secs 2023-09-10T02:23:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11020.891550 secs 2023-09-10T02:23:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:10 smithi170 ceph-mon[102832]: pgmap v6040: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T02:23:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:10 smithi082 ceph-mon[98694]: pgmap v6040: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T02:23:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:10 smithi098 ceph-mon[103124]: pgmap v6040: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T02:23:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:12 smithi170 ceph-mon[102832]: pgmap v6041: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 82 KiB/s wr, 28 op/s 2023-09-10T02:23:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11023.789603 secs 2023-09-10T02:23:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:12 smithi082 ceph-mon[98694]: pgmap v6041: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 82 KiB/s wr, 28 op/s 2023-09-10T02:23:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11023.789603 secs 2023-09-10T02:23:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:12 smithi098 ceph-mon[103124]: pgmap v6041: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 82 KiB/s wr, 28 op/s 2023-09-10T02:23:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11023.789603 secs 2023-09-10T02:23:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11025.179886 secs 2023-09-10T02:23:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11025.179886 secs 2023-09-10T02:23:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11025.179886 secs 2023-09-10T02:23:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:14 smithi170 ceph-mon[102832]: pgmap v6042: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 82 KiB/s wr, 28 op/s 2023-09-10T02:23:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11025.891616 secs 2023-09-10T02:23:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:14 smithi082 ceph-mon[98694]: pgmap v6042: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 82 KiB/s wr, 28 op/s 2023-09-10T02:23:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11025.891616 secs 2023-09-10T02:23:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:14 smithi098 ceph-mon[103124]: pgmap v6042: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 82 KiB/s wr, 28 op/s 2023-09-10T02:23:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11025.891616 secs 2023-09-10T02:23:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:16 smithi170 ceph-mon[102832]: pgmap v6043: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 12 op/s 2023-09-10T02:23:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:16 smithi170 ceph-mon[102832]: osdmap e446: 12 total, 12 up, 12 in 2023-09-10T02:23:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:16 smithi082 ceph-mon[98694]: pgmap v6043: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 12 op/s 2023-09-10T02:23:16.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:16 smithi082 ceph-mon[98694]: osdmap e446: 12 total, 12 up, 12 in 2023-09-10T02:23:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:16 smithi098 ceph-mon[103124]: pgmap v6043: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 12 op/s 2023-09-10T02:23:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:16 smithi098 ceph-mon[103124]: osdmap e446: 12 total, 12 up, 12 in 2023-09-10T02:23:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11028.789664 secs 2023-09-10T02:23:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11028.789664 secs 2023-09-10T02:23:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11028.789664 secs 2023-09-10T02:23:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:18 smithi170 ceph-mon[102832]: pgmap v6045: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T02:23:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11030.179945 secs 2023-09-10T02:23:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:18 smithi082 ceph-mon[98694]: pgmap v6045: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T02:23:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11030.179945 secs 2023-09-10T02:23:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:18 smithi098 ceph-mon[103124]: pgmap v6045: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T02:23:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11030.179945 secs 2023-09-10T02:23:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11030.891680 secs 2023-09-10T02:23:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11030.891680 secs 2023-09-10T02:23:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11030.891680 secs 2023-09-10T02:23:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:20 smithi170 ceph-mon[102832]: pgmap v6046: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 12 op/s 2023-09-10T02:23:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:20 smithi082 ceph-mon[98694]: pgmap v6046: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 12 op/s 2023-09-10T02:23:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:20 smithi098 ceph-mon[103124]: pgmap v6046: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 12 op/s 2023-09-10T02:23:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:22 smithi170 ceph-mon[102832]: pgmap v6047: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T02:23:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:22 smithi082 ceph-mon[98694]: pgmap v6047: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T02:23:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:22 smithi098 ceph-mon[103124]: pgmap v6047: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T02:23:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:23:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11035.180003 secs 2023-09-10T02:23:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:23:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11035.180003 secs 2023-09-10T02:23:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:23:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11035.180003 secs 2023-09-10T02:23:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:24 smithi170 ceph-mon[102832]: pgmap v6048: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T02:23:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11035.891747 secs 2023-09-10T02:23:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:23:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:23:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:23:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:24 smithi082 ceph-mon[98694]: pgmap v6048: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T02:23:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11035.891747 secs 2023-09-10T02:23:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:23:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:23:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:23:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:24 smithi098 ceph-mon[103124]: pgmap v6048: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T02:23:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11035.891747 secs 2023-09-10T02:23:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:23:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:23:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:23:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:26 smithi170 ceph-mon[102832]: pgmap v6049: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T02:23:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:26 smithi082 ceph-mon[98694]: pgmap v6049: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T02:23:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:26 smithi098 ceph-mon[103124]: pgmap v6049: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T02:23:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:28 smithi170 ceph-mon[102832]: pgmap v6050: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T02:23:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11040.180062 secs 2023-09-10T02:23:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:28 smithi082 ceph-mon[98694]: pgmap v6050: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T02:23:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11040.180062 secs 2023-09-10T02:23:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:28 smithi098 ceph-mon[103124]: pgmap v6050: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T02:23:29.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11040.180062 secs 2023-09-10T02:23:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11040.891830 secs 2023-09-10T02:23:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11040.891830 secs 2023-09-10T02:23:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11040.891830 secs 2023-09-10T02:23:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:30 smithi170 ceph-mon[102832]: pgmap v6051: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T02:23:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:30 smithi082 ceph-mon[98694]: pgmap v6051: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T02:23:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:30 smithi098 ceph-mon[103124]: pgmap v6051: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T02:23:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:32 smithi170 ceph-mon[102832]: pgmap v6052: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 139 KiB/s wr, 16 op/s 2023-09-10T02:23:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:32 smithi082 ceph-mon[98694]: pgmap v6052: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 139 KiB/s wr, 16 op/s 2023-09-10T02:23:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:32 smithi098 ceph-mon[103124]: pgmap v6052: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 139 KiB/s wr, 16 op/s 2023-09-10T02:23:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11045.180122 secs 2023-09-10T02:23:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11045.180122 secs 2023-09-10T02:23:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11045.180122 secs 2023-09-10T02:23:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:34 smithi170 ceph-mon[102832]: pgmap v6053: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T02:23:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11045.891892 secs 2023-09-10T02:23:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:34 smithi082 ceph-mon[98694]: pgmap v6053: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T02:23:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11045.891892 secs 2023-09-10T02:23:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:34 smithi098 ceph-mon[103124]: pgmap v6053: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T02:23:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11045.891892 secs 2023-09-10T02:23:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:23:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:23:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:23:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:23:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:23:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:23:35.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:23:35.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:23:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:23:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:23:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:23:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:23:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:36 smithi170 ceph-mon[102832]: pgmap v6054: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 193 KiB/s wr, 17 op/s 2023-09-10T02:23:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:36 smithi082 ceph-mon[98694]: pgmap v6054: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 193 KiB/s wr, 17 op/s 2023-09-10T02:23:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:36 smithi098 ceph-mon[103124]: pgmap v6054: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 193 KiB/s wr, 17 op/s 2023-09-10T02:23:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:38 smithi170 ceph-mon[102832]: pgmap v6055: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T02:23:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11050.180205 secs 2023-09-10T02:23:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:38 smithi082 ceph-mon[98694]: pgmap v6055: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T02:23:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11050.180205 secs 2023-09-10T02:23:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:38 smithi098 ceph-mon[103124]: pgmap v6055: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T02:23:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11050.180205 secs 2023-09-10T02:23:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11050.891960 secs 2023-09-10T02:23:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11050.891960 secs 2023-09-10T02:23:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11050.891960 secs 2023-09-10T02:23:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:40 smithi170 ceph-mon[102832]: pgmap v6056: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T02:23:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:40 smithi082 ceph-mon[98694]: pgmap v6056: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T02:23:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:40 smithi098 ceph-mon[103124]: pgmap v6056: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T02:23:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:42 smithi170 ceph-mon[102832]: pgmap v6057: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 224 KiB/s wr, 18 op/s 2023-09-10T02:23:42.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11053.790113 secs 2023-09-10T02:23:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:42 smithi082 ceph-mon[98694]: pgmap v6057: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 224 KiB/s wr, 18 op/s 2023-09-10T02:23:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11053.790113 secs 2023-09-10T02:23:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:42 smithi098 ceph-mon[103124]: pgmap v6057: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 224 KiB/s wr, 18 op/s 2023-09-10T02:23:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11053.790113 secs 2023-09-10T02:23:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11055.180262 secs 2023-09-10T02:23:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11055.180262 secs 2023-09-10T02:23:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11055.180262 secs 2023-09-10T02:23:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:44 smithi082 ceph-mon[98694]: pgmap v6058: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T02:23:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11055.892022 secs 2023-09-10T02:23:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:23:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:23:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:23:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:44 smithi098 ceph-mon[103124]: pgmap v6058: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T02:23:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11055.892022 secs 2023-09-10T02:23:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:23:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:23:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:23:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:44 smithi170 ceph-mon[102832]: pgmap v6058: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T02:23:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11055.892022 secs 2023-09-10T02:23:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:23:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:23:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:23:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:46 smithi082 ceph-mon[98694]: pgmap v6059: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 269 KiB/s wr, 18 op/s 2023-09-10T02:23:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:46 smithi098 ceph-mon[103124]: pgmap v6059: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 269 KiB/s wr, 18 op/s 2023-09-10T02:23:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:46 smithi170 ceph-mon[102832]: pgmap v6059: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 269 KiB/s wr, 18 op/s 2023-09-10T02:23:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11058.790173 secs 2023-09-10T02:23:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11058.790173 secs 2023-09-10T02:23:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11058.790173 secs 2023-09-10T02:23:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:48 smithi082 ceph-mon[98694]: pgmap v6060: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T02:23:48.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11060.180320 secs 2023-09-10T02:23:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:48 smithi098 ceph-mon[103124]: pgmap v6060: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T02:23:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11060.180320 secs 2023-09-10T02:23:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:48 smithi170 ceph-mon[102832]: pgmap v6060: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T02:23:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11060.180320 secs 2023-09-10T02:23:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11060.892090 secs 2023-09-10T02:23:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11060.892090 secs 2023-09-10T02:23:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11060.892090 secs 2023-09-10T02:23:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:50 smithi082 ceph-mon[98694]: pgmap v6061: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T02:23:51.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:50 smithi098 ceph-mon[103124]: pgmap v6061: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T02:23:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:50 smithi170 ceph-mon[102832]: pgmap v6061: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T02:23:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:52 smithi082 ceph-mon[98694]: pgmap v6062: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 257 KiB/s wr, 18 op/s 2023-09-10T02:23:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11063.790264 secs 2023-09-10T02:23:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:52 smithi098 ceph-mon[103124]: pgmap v6062: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 257 KiB/s wr, 18 op/s 2023-09-10T02:23:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11063.790264 secs 2023-09-10T02:23:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:52 smithi170 ceph-mon[102832]: pgmap v6062: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 257 KiB/s wr, 18 op/s 2023-09-10T02:23:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11063.790264 secs 2023-09-10T02:23:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11065.180404 secs 2023-09-10T02:23:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11065.180404 secs 2023-09-10T02:23:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11065.180404 secs 2023-09-10T02:23:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:54 smithi098 ceph-mon[103124]: pgmap v6063: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 164 KiB/s wr, 12 op/s 2023-09-10T02:23:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11065.892155 secs 2023-09-10T02:23:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:54 smithi170 ceph-mon[102832]: pgmap v6063: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 164 KiB/s wr, 12 op/s 2023-09-10T02:23:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11065.892155 secs 2023-09-10T02:23:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:54 smithi082 ceph-mon[98694]: pgmap v6063: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 164 KiB/s wr, 12 op/s 2023-09-10T02:23:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11065.892155 secs 2023-09-10T02:23:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:56 smithi098 ceph-mon[103124]: pgmap v6064: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 172 KiB/s wr, 16 op/s 2023-09-10T02:23:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:56 smithi170 ceph-mon[102832]: pgmap v6064: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 172 KiB/s wr, 16 op/s 2023-09-10T02:23:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:56 smithi082 ceph-mon[98694]: pgmap v6064: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 172 KiB/s wr, 16 op/s 2023-09-10T02:23:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11068.790327 secs 2023-09-10T02:23:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11068.790327 secs 2023-09-10T02:23:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11068.790327 secs 2023-09-10T02:23:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:58 smithi098 ceph-mon[103124]: pgmap v6065: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T02:23:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11070.180493 secs 2023-09-10T02:23:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:58 smithi170 ceph-mon[102832]: pgmap v6065: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T02:23:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11070.180493 secs 2023-09-10T02:23:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:58 smithi082 ceph-mon[98694]: pgmap v6065: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T02:23:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11070.180493 secs 2023-09-10T02:24:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:23:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11070.892219 secs 2023-09-10T02:24:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:23:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11070.892219 secs 2023-09-10T02:24:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:23:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11070.892219 secs 2023-09-10T02:24:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:00 smithi098 ceph-mon[103124]: pgmap v6066: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T02:24:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:00 smithi170 ceph-mon[102832]: pgmap v6066: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T02:24:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:00 smithi082 ceph-mon[98694]: pgmap v6066: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T02:24:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:02 smithi098 ceph-mon[103124]: pgmap v6067: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 84 KiB/s wr, 15 op/s 2023-09-10T02:24:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11073.790442 secs 2023-09-10T02:24:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:02 smithi170 ceph-mon[102832]: pgmap v6067: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 84 KiB/s wr, 15 op/s 2023-09-10T02:24:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11073.790442 secs 2023-09-10T02:24:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:02 smithi082 ceph-mon[98694]: pgmap v6067: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 84 KiB/s wr, 15 op/s 2023-09-10T02:24:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11073.790442 secs 2023-09-10T02:24:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11075.180575 secs 2023-09-10T02:24:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11075.180575 secs 2023-09-10T02:24:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11075.180575 secs 2023-09-10T02:24:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:04 smithi098 ceph-mon[103124]: pgmap v6068: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 9 op/s 2023-09-10T02:24:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11075.892286 secs 2023-09-10T02:24:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:04 smithi170 ceph-mon[102832]: pgmap v6068: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 9 op/s 2023-09-10T02:24:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11075.892286 secs 2023-09-10T02:24:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:04 smithi082 ceph-mon[98694]: pgmap v6068: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 9 op/s 2023-09-10T02:24:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11075.892286 secs 2023-09-10T02:24:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:06 smithi098 ceph-mon[103124]: pgmap v6069: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 73 KiB/s wr, 28 op/s 2023-09-10T02:24:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:06 smithi170 ceph-mon[102832]: pgmap v6069: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 73 KiB/s wr, 28 op/s 2023-09-10T02:24:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:06 smithi082 ceph-mon[98694]: pgmap v6069: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 73 KiB/s wr, 28 op/s 2023-09-10T02:24:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11078.790500 secs 2023-09-10T02:24:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11078.790500 secs 2023-09-10T02:24:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11078.790500 secs 2023-09-10T02:24:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:08 smithi098 ceph-mon[103124]: pgmap v6070: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 65 KiB/s wr, 23 op/s 2023-09-10T02:24:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:08 smithi098 ceph-mon[103124]: osdmap e447: 12 total, 12 up, 12 in 2023-09-10T02:24:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11080.180634 secs 2023-09-10T02:24:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:08 smithi170 ceph-mon[102832]: pgmap v6070: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 65 KiB/s wr, 23 op/s 2023-09-10T02:24:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:08 smithi170 ceph-mon[102832]: osdmap e447: 12 total, 12 up, 12 in 2023-09-10T02:24:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11080.180634 secs 2023-09-10T02:24:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:08 smithi082 ceph-mon[98694]: pgmap v6070: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 65 KiB/s wr, 23 op/s 2023-09-10T02:24:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:08 smithi082 ceph-mon[98694]: osdmap e447: 12 total, 12 up, 12 in 2023-09-10T02:24:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11080.180634 secs 2023-09-10T02:24:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11080.892354 secs 2023-09-10T02:24:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11080.892354 secs 2023-09-10T02:24:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11080.892354 secs 2023-09-10T02:24:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:10 smithi170 ceph-mon[102832]: pgmap v6072: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 78 KiB/s wr, 28 op/s 2023-09-10T02:24:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:10 smithi082 ceph-mon[98694]: pgmap v6072: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 78 KiB/s wr, 28 op/s 2023-09-10T02:24:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:10 smithi098 ceph-mon[103124]: pgmap v6072: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 78 KiB/s wr, 28 op/s 2023-09-10T02:24:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:12 smithi170 ceph-mon[102832]: pgmap v6073: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 28 op/s 2023-09-10T02:24:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11083.790636 secs 2023-09-10T02:24:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:12 smithi082 ceph-mon[98694]: pgmap v6073: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 28 op/s 2023-09-10T02:24:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11083.790636 secs 2023-09-10T02:24:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:12 smithi098 ceph-mon[103124]: pgmap v6073: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 28 op/s 2023-09-10T02:24:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11083.790636 secs 2023-09-10T02:24:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11085.180688 secs 2023-09-10T02:24:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11085.180688 secs 2023-09-10T02:24:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11085.180688 secs 2023-09-10T02:24:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:14 smithi170 ceph-mon[102832]: pgmap v6074: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 28 op/s 2023-09-10T02:24:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11085.892419 secs 2023-09-10T02:24:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:14 smithi082 ceph-mon[98694]: pgmap v6074: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 28 op/s 2023-09-10T02:24:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11085.892419 secs 2023-09-10T02:24:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:14 smithi098 ceph-mon[103124]: pgmap v6074: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 28 op/s 2023-09-10T02:24:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11085.892419 secs 2023-09-10T02:24:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:16 smithi170 ceph-mon[102832]: pgmap v6075: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T02:24:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:16 smithi170 ceph-mon[102832]: osdmap e448: 12 total, 12 up, 12 in 2023-09-10T02:24:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:16 smithi082 ceph-mon[98694]: pgmap v6075: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T02:24:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:16 smithi082 ceph-mon[98694]: osdmap e448: 12 total, 12 up, 12 in 2023-09-10T02:24:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:16 smithi098 ceph-mon[103124]: pgmap v6075: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T02:24:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:16 smithi098 ceph-mon[103124]: osdmap e448: 12 total, 12 up, 12 in 2023-09-10T02:24:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11088.790694 secs 2023-09-10T02:24:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11088.790694 secs 2023-09-10T02:24:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11088.790694 secs 2023-09-10T02:24:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:18 smithi170 ceph-mon[102832]: pgmap v6077: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 12 op/s 2023-09-10T02:24:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11090.180746 secs 2023-09-10T02:24:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:18 smithi082 ceph-mon[98694]: pgmap v6077: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 12 op/s 2023-09-10T02:24:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11090.180746 secs 2023-09-10T02:24:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:18 smithi098 ceph-mon[103124]: pgmap v6077: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 12 op/s 2023-09-10T02:24:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11090.180746 secs 2023-09-10T02:24:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11090.892487 secs 2023-09-10T02:24:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11090.892487 secs 2023-09-10T02:24:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11090.892487 secs 2023-09-10T02:24:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:20 smithi170 ceph-mon[102832]: pgmap v6078: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T02:24:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:20 smithi082 ceph-mon[98694]: pgmap v6078: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T02:24:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:20 smithi098 ceph-mon[103124]: pgmap v6078: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T02:24:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:22 smithi170 ceph-mon[102832]: pgmap v6079: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T02:24:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:24:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:22 smithi082 ceph-mon[98694]: pgmap v6079: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T02:24:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:24:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:22 smithi098 ceph-mon[103124]: pgmap v6079: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T02:24:23.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:24:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:24:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:24:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:24:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11095.180828 secs 2023-09-10T02:24:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:24:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:24:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:24:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11095.180828 secs 2023-09-10T02:24:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:24:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:24:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:24:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11095.180828 secs 2023-09-10T02:24:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:24 smithi170 ceph-mon[102832]: pgmap v6080: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T02:24:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11095.892552 secs 2023-09-10T02:24:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:24 smithi082 ceph-mon[98694]: pgmap v6080: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T02:24:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11095.892552 secs 2023-09-10T02:24:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:24 smithi098 ceph-mon[103124]: pgmap v6080: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T02:24:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11095.892552 secs 2023-09-10T02:24:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:26 smithi170 ceph-mon[102832]: pgmap v6081: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T02:24:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:26 smithi082 ceph-mon[98694]: pgmap v6081: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T02:24:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:26 smithi098 ceph-mon[103124]: pgmap v6081: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T02:24:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:28 smithi170 ceph-mon[102832]: pgmap v6082: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 10 op/s 2023-09-10T02:24:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11100.180886 secs 2023-09-10T02:24:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:28 smithi082 ceph-mon[98694]: pgmap v6082: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 10 op/s 2023-09-10T02:24:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11100.180886 secs 2023-09-10T02:24:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:28 smithi098 ceph-mon[103124]: pgmap v6082: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 10 op/s 2023-09-10T02:24:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11100.180886 secs 2023-09-10T02:24:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11100.892617 secs 2023-09-10T02:24:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11100.892617 secs 2023-09-10T02:24:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11100.892617 secs 2023-09-10T02:24:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:30 smithi170 ceph-mon[102832]: pgmap v6083: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 9 op/s 2023-09-10T02:24:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:30 smithi082 ceph-mon[98694]: pgmap v6083: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 9 op/s 2023-09-10T02:24:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:30 smithi098 ceph-mon[103124]: pgmap v6083: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 9 op/s 2023-09-10T02:24:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:32 smithi170 ceph-mon[102832]: pgmap v6084: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 138 KiB/s wr, 17 op/s 2023-09-10T02:24:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:32 smithi082 ceph-mon[98694]: pgmap v6084: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 138 KiB/s wr, 17 op/s 2023-09-10T02:24:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:32 smithi098 ceph-mon[103124]: pgmap v6084: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 138 KiB/s wr, 17 op/s 2023-09-10T02:24:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11105.180976 secs 2023-09-10T02:24:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11105.180976 secs 2023-09-10T02:24:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11105.180976 secs 2023-09-10T02:24:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:34 smithi170 ceph-mon[102832]: pgmap v6085: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T02:24:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11105.892684 secs 2023-09-10T02:24:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:34 smithi082 ceph-mon[98694]: pgmap v6085: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T02:24:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11105.892684 secs 2023-09-10T02:24:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:34 smithi098 ceph-mon[103124]: pgmap v6085: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T02:24:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11105.892684 secs 2023-09-10T02:24:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:24:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:24:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:24:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:36 smithi170 ceph-mon[102832]: pgmap v6086: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 144 KiB/s wr, 18 op/s 2023-09-10T02:24:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:24:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:24:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:24:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:36 smithi082 ceph-mon[98694]: pgmap v6086: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 144 KiB/s wr, 18 op/s 2023-09-10T02:24:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:24:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:24:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:24:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:36 smithi098 ceph-mon[103124]: pgmap v6086: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 144 KiB/s wr, 18 op/s 2023-09-10T02:24:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:24:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:24:37.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:24:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:38 smithi170 ceph-mon[102832]: pgmap v6087: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 13 op/s 2023-09-10T02:24:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11110.181036 secs 2023-09-10T02:24:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:38 smithi082 ceph-mon[98694]: pgmap v6087: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 13 op/s 2023-09-10T02:24:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11110.181036 secs 2023-09-10T02:24:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:38 smithi098 ceph-mon[103124]: pgmap v6087: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 13 op/s 2023-09-10T02:24:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11110.181036 secs 2023-09-10T02:24:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11110.892750 secs 2023-09-10T02:24:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11110.892750 secs 2023-09-10T02:24:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11110.892750 secs 2023-09-10T02:24:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:40 smithi170 ceph-mon[102832]: pgmap v6088: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 13 op/s 2023-09-10T02:24:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:40 smithi082 ceph-mon[98694]: pgmap v6088: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 13 op/s 2023-09-10T02:24:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:40 smithi098 ceph-mon[103124]: pgmap v6088: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 13 op/s 2023-09-10T02:24:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:42 smithi170 ceph-mon[102832]: pgmap v6089: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 222 KiB/s wr, 20 op/s 2023-09-10T02:24:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:42 smithi082 ceph-mon[98694]: pgmap v6089: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 222 KiB/s wr, 20 op/s 2023-09-10T02:24:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:42 smithi098 ceph-mon[103124]: pgmap v6089: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 222 KiB/s wr, 20 op/s 2023-09-10T02:24:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11115.181107 secs 2023-09-10T02:24:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11115.181107 secs 2023-09-10T02:24:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11115.181107 secs 2023-09-10T02:24:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:44 smithi170 ceph-mon[102832]: pgmap v6090: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T02:24:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11115.892820 secs 2023-09-10T02:24:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:44 smithi082 ceph-mon[98694]: pgmap v6090: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T02:24:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11115.892820 secs 2023-09-10T02:24:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:44 smithi098 ceph-mon[103124]: pgmap v6090: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T02:24:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11115.892820 secs 2023-09-10T02:24:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:46 smithi170 ceph-mon[102832]: pgmap v6091: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 135 KiB/s wr, 17 op/s 2023-09-10T02:24:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:46 smithi082 ceph-mon[98694]: pgmap v6091: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 135 KiB/s wr, 17 op/s 2023-09-10T02:24:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:46 smithi098 ceph-mon[103124]: pgmap v6091: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 135 KiB/s wr, 17 op/s 2023-09-10T02:24:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11118.791110 secs 2023-09-10T02:24:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11118.791110 secs 2023-09-10T02:24:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11118.791110 secs 2023-09-10T02:24:48.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:48 smithi170 ceph-mon[102832]: pgmap v6092: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 12 op/s 2023-09-10T02:24:48.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11120.181167 secs 2023-09-10T02:24:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:48 smithi082 ceph-mon[98694]: pgmap v6092: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 12 op/s 2023-09-10T02:24:48.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11120.181167 secs 2023-09-10T02:24:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:48 smithi098 ceph-mon[103124]: pgmap v6092: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 12 op/s 2023-09-10T02:24:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11120.181167 secs 2023-09-10T02:24:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11120.892885 secs 2023-09-10T02:24:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11120.892885 secs 2023-09-10T02:24:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11120.892885 secs 2023-09-10T02:24:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:50 smithi170 ceph-mon[102832]: pgmap v6093: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 12 op/s 2023-09-10T02:24:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:50 smithi082 ceph-mon[98694]: pgmap v6093: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 12 op/s 2023-09-10T02:24:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:50 smithi098 ceph-mon[103124]: pgmap v6093: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 12 op/s 2023-09-10T02:24:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:52 smithi170 ceph-mon[102832]: pgmap v6094: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 231 KiB/s wr, 17 op/s 2023-09-10T02:24:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:24:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:24:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:24:52.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11123.791175 secs 2023-09-10T02:24:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:52 smithi082 ceph-mon[98694]: pgmap v6094: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 231 KiB/s wr, 17 op/s 2023-09-10T02:24:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:24:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:24:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:24:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11123.791175 secs 2023-09-10T02:24:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:52 smithi098 ceph-mon[103124]: pgmap v6094: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 231 KiB/s wr, 17 op/s 2023-09-10T02:24:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:24:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:24:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:24:53.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11123.791175 secs 2023-09-10T02:24:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11125.181222 secs 2023-09-10T02:24:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11125.181222 secs 2023-09-10T02:24:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11125.181222 secs 2023-09-10T02:24:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:54 smithi170 ceph-mon[102832]: pgmap v6095: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 125 KiB/s wr, 10 op/s 2023-09-10T02:24:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11125.892952 secs 2023-09-10T02:24:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:54 smithi082 ceph-mon[98694]: pgmap v6095: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 125 KiB/s wr, 10 op/s 2023-09-10T02:24:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11125.892952 secs 2023-09-10T02:24:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:54 smithi098 ceph-mon[103124]: pgmap v6095: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 125 KiB/s wr, 10 op/s 2023-09-10T02:24:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11125.892952 secs 2023-09-10T02:24:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:56 smithi170 ceph-mon[102832]: pgmap v6096: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 195 KiB/s wr, 16 op/s 2023-09-10T02:24:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:56 smithi082 ceph-mon[98694]: pgmap v6096: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 195 KiB/s wr, 16 op/s 2023-09-10T02:24:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:56 smithi098 ceph-mon[103124]: pgmap v6096: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 195 KiB/s wr, 16 op/s 2023-09-10T02:24:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11128.791235 secs 2023-09-10T02:24:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11128.791235 secs 2023-09-10T02:24:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11128.791235 secs 2023-09-10T02:24:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:58 smithi170 ceph-mon[102832]: pgmap v6097: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T02:24:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11130.181282 secs 2023-09-10T02:24:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:58 smithi082 ceph-mon[98694]: pgmap v6097: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T02:24:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11130.181282 secs 2023-09-10T02:24:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:58 smithi098 ceph-mon[103124]: pgmap v6097: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T02:24:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11130.181282 secs 2023-09-10T02:24:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:24:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11130.893017 secs 2023-09-10T02:24:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:24:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11130.893017 secs 2023-09-10T02:25:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:24:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11130.893017 secs 2023-09-10T02:25:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:00 smithi170 ceph-mon[102832]: pgmap v6098: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T02:25:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:00 smithi082 ceph-mon[98694]: pgmap v6098: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T02:25:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:00 smithi098 ceph-mon[103124]: pgmap v6098: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T02:25:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:02 smithi170 ceph-mon[102832]: pgmap v6099: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 193 KiB/s wr, 15 op/s 2023-09-10T02:25:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11133.791294 secs 2023-09-10T02:25:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:02 smithi082 ceph-mon[98694]: pgmap v6099: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 193 KiB/s wr, 15 op/s 2023-09-10T02:25:02.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11133.791294 secs 2023-09-10T02:25:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:02 smithi098 ceph-mon[103124]: pgmap v6099: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 193 KiB/s wr, 15 op/s 2023-09-10T02:25:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11133.791294 secs 2023-09-10T02:25:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11135.181366 secs 2023-09-10T02:25:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11135.181366 secs 2023-09-10T02:25:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11135.181366 secs 2023-09-10T02:25:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:04 smithi170 ceph-mon[102832]: pgmap v6100: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 9 op/s 2023-09-10T02:25:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11135.893085 secs 2023-09-10T02:25:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:04 smithi082 ceph-mon[98694]: pgmap v6100: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 9 op/s 2023-09-10T02:25:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11135.893085 secs 2023-09-10T02:25:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:04 smithi098 ceph-mon[103124]: pgmap v6100: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 9 op/s 2023-09-10T02:25:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11135.893085 secs 2023-09-10T02:25:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:06 smithi170 ceph-mon[102832]: pgmap v6101: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 136 KiB/s wr, 28 op/s 2023-09-10T02:25:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:06 smithi082 ceph-mon[98694]: pgmap v6101: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 136 KiB/s wr, 28 op/s 2023-09-10T02:25:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:06 smithi098 ceph-mon[103124]: pgmap v6101: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 136 KiB/s wr, 28 op/s 2023-09-10T02:25:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11138.791355 secs 2023-09-10T02:25:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11138.791355 secs 2023-09-10T02:25:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11138.791355 secs 2023-09-10T02:25:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:08 smithi170 ceph-mon[102832]: pgmap v6102: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 65 KiB/s wr, 23 op/s 2023-09-10T02:25:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:08 smithi170 ceph-mon[102832]: osdmap e449: 12 total, 12 up, 12 in 2023-09-10T02:25:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11140.181449 secs 2023-09-10T02:25:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:08 smithi082 ceph-mon[98694]: pgmap v6102: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 65 KiB/s wr, 23 op/s 2023-09-10T02:25:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:08 smithi082 ceph-mon[98694]: osdmap e449: 12 total, 12 up, 12 in 2023-09-10T02:25:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11140.181449 secs 2023-09-10T02:25:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:08 smithi098 ceph-mon[103124]: pgmap v6102: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 65 KiB/s wr, 23 op/s 2023-09-10T02:25:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:08 smithi098 ceph-mon[103124]: osdmap e449: 12 total, 12 up, 12 in 2023-09-10T02:25:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11140.181449 secs 2023-09-10T02:25:09.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11140.893151 secs 2023-09-10T02:25:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11140.893151 secs 2023-09-10T02:25:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11140.893151 secs 2023-09-10T02:25:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:10 smithi170 ceph-mon[102832]: pgmap v6104: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 28 op/s 2023-09-10T02:25:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:10 smithi082 ceph-mon[98694]: pgmap v6104: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 28 op/s 2023-09-10T02:25:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:10 smithi098 ceph-mon[103124]: pgmap v6104: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 28 op/s 2023-09-10T02:25:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:12 smithi170 ceph-mon[102832]: pgmap v6105: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 29 op/s 2023-09-10T02:25:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11143.791455 secs 2023-09-10T02:25:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:12 smithi082 ceph-mon[98694]: pgmap v6105: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 29 op/s 2023-09-10T02:25:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11143.791455 secs 2023-09-10T02:25:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:12 smithi098 ceph-mon[103124]: pgmap v6105: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 29 op/s 2023-09-10T02:25:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11143.791455 secs 2023-09-10T02:25:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11145.181491 secs 2023-09-10T02:25:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11145.181491 secs 2023-09-10T02:25:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11145.181491 secs 2023-09-10T02:25:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:14 smithi170 ceph-mon[102832]: pgmap v6106: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 29 op/s 2023-09-10T02:25:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11145.893216 secs 2023-09-10T02:25:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:14 smithi082 ceph-mon[98694]: pgmap v6106: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 29 op/s 2023-09-10T02:25:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11145.893216 secs 2023-09-10T02:25:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:14 smithi098 ceph-mon[103124]: pgmap v6106: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 29 op/s 2023-09-10T02:25:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11145.893216 secs 2023-09-10T02:25:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:16 smithi170 ceph-mon[102832]: pgmap v6107: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 12 op/s 2023-09-10T02:25:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:16 smithi170 ceph-mon[102832]: osdmap e450: 12 total, 12 up, 12 in 2023-09-10T02:25:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:16 smithi082 ceph-mon[98694]: pgmap v6107: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 12 op/s 2023-09-10T02:25:16.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:16 smithi082 ceph-mon[98694]: osdmap e450: 12 total, 12 up, 12 in 2023-09-10T02:25:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:16 smithi098 ceph-mon[103124]: pgmap v6107: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 12 op/s 2023-09-10T02:25:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:16 smithi098 ceph-mon[103124]: osdmap e450: 12 total, 12 up, 12 in 2023-09-10T02:25:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11148.791574 secs 2023-09-10T02:25:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11148.791574 secs 2023-09-10T02:25:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11148.791574 secs 2023-09-10T02:25:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:18 smithi170 ceph-mon[102832]: pgmap v6109: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 KiB/s wr, 12 op/s 2023-09-10T02:25:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11150.181551 secs 2023-09-10T02:25:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:18 smithi082 ceph-mon[98694]: pgmap v6109: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 KiB/s wr, 12 op/s 2023-09-10T02:25:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11150.181551 secs 2023-09-10T02:25:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:18 smithi098 ceph-mon[103124]: pgmap v6109: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 KiB/s wr, 12 op/s 2023-09-10T02:25:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11150.181551 secs 2023-09-10T02:25:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11150.893283 secs 2023-09-10T02:25:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11150.893283 secs 2023-09-10T02:25:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11150.893283 secs 2023-09-10T02:25:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:20 smithi170 ceph-mon[102832]: pgmap v6110: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 12 op/s 2023-09-10T02:25:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:20 smithi082 ceph-mon[98694]: pgmap v6110: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 12 op/s 2023-09-10T02:25:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:20 smithi098 ceph-mon[103124]: pgmap v6110: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 12 op/s 2023-09-10T02:25:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:22 smithi170 ceph-mon[102832]: pgmap v6111: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T02:25:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:22 smithi082 ceph-mon[98694]: pgmap v6111: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T02:25:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:22 smithi098 ceph-mon[103124]: pgmap v6111: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T02:25:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:25:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11155.181610 secs 2023-09-10T02:25:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:25:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11155.181610 secs 2023-09-10T02:25:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:25:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11155.181610 secs 2023-09-10T02:25:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:24 smithi170 ceph-mon[102832]: pgmap v6112: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T02:25:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:25:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:25:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:25:24.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11155.893348 secs 2023-09-10T02:25:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:24 smithi082 ceph-mon[98694]: pgmap v6112: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T02:25:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:25:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:25:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:25:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11155.893348 secs 2023-09-10T02:25:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:24 smithi098 ceph-mon[103124]: pgmap v6112: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T02:25:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:25:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:25:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:25:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11155.893348 secs 2023-09-10T02:25:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:26 smithi170 ceph-mon[102832]: pgmap v6113: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 64 KiB/s wr, 13 op/s 2023-09-10T02:25:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:26 smithi082 ceph-mon[98694]: pgmap v6113: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 64 KiB/s wr, 13 op/s 2023-09-10T02:25:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:26 smithi098 ceph-mon[103124]: pgmap v6113: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 64 KiB/s wr, 13 op/s 2023-09-10T02:25:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:28 smithi170 ceph-mon[102832]: pgmap v6114: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T02:25:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11160.181668 secs 2023-09-10T02:25:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:28 smithi082 ceph-mon[98694]: pgmap v6114: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T02:25:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11160.181668 secs 2023-09-10T02:25:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:28 smithi098 ceph-mon[103124]: pgmap v6114: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T02:25:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11160.181668 secs 2023-09-10T02:25:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11160.893419 secs 2023-09-10T02:25:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11160.893419 secs 2023-09-10T02:25:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11160.893419 secs 2023-09-10T02:25:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:30 smithi170 ceph-mon[102832]: pgmap v6115: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T02:25:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:30 smithi082 ceph-mon[98694]: pgmap v6115: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T02:25:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:30 smithi098 ceph-mon[103124]: pgmap v6115: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T02:25:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:32 smithi170 ceph-mon[102832]: pgmap v6116: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 154 KiB/s wr, 17 op/s 2023-09-10T02:25:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:32 smithi082 ceph-mon[98694]: pgmap v6116: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 154 KiB/s wr, 17 op/s 2023-09-10T02:25:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:32 smithi098 ceph-mon[103124]: pgmap v6116: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 154 KiB/s wr, 17 op/s 2023-09-10T02:25:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11165.181761 secs 2023-09-10T02:25:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11165.181761 secs 2023-09-10T02:25:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11165.181761 secs 2023-09-10T02:25:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:34 smithi170 ceph-mon[102832]: pgmap v6117: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 148 KiB/s wr, 12 op/s 2023-09-10T02:25:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11165.893496 secs 2023-09-10T02:25:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:34 smithi082 ceph-mon[98694]: pgmap v6117: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 148 KiB/s wr, 12 op/s 2023-09-10T02:25:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11165.893496 secs 2023-09-10T02:25:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:34 smithi098 ceph-mon[103124]: pgmap v6117: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 148 KiB/s wr, 12 op/s 2023-09-10T02:25:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11165.893496 secs 2023-09-10T02:25:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:36 smithi170 ceph-mon[102832]: pgmap v6118: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 203 KiB/s wr, 17 op/s 2023-09-10T02:25:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:25:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:36 smithi082 ceph-mon[98694]: pgmap v6118: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 203 KiB/s wr, 17 op/s 2023-09-10T02:25:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:25:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:36 smithi098 ceph-mon[103124]: pgmap v6118: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 203 KiB/s wr, 17 op/s 2023-09-10T02:25:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:25:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:25:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:25:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:25:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:25:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:25:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:25:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:25:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:25:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:25:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:38 smithi170 ceph-mon[102832]: pgmap v6119: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 156 KiB/s wr, 11 op/s 2023-09-10T02:25:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11170.181853 secs 2023-09-10T02:25:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:38 smithi082 ceph-mon[98694]: pgmap v6119: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 156 KiB/s wr, 11 op/s 2023-09-10T02:25:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11170.181853 secs 2023-09-10T02:25:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:38 smithi098 ceph-mon[103124]: pgmap v6119: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 156 KiB/s wr, 11 op/s 2023-09-10T02:25:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11170.181853 secs 2023-09-10T02:25:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11170.893562 secs 2023-09-10T02:25:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11170.893562 secs 2023-09-10T02:25:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11170.893562 secs 2023-09-10T02:25:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:40 smithi170 ceph-mon[102832]: pgmap v6120: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 156 KiB/s wr, 11 op/s 2023-09-10T02:25:41.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:40 smithi082 ceph-mon[98694]: pgmap v6120: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 156 KiB/s wr, 11 op/s 2023-09-10T02:25:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:40 smithi098 ceph-mon[103124]: pgmap v6120: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 156 KiB/s wr, 11 op/s 2023-09-10T02:25:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:42 smithi170 ceph-mon[102832]: pgmap v6121: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 304 KiB/s wr, 18 op/s 2023-09-10T02:25:42.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11173.791887 secs 2023-09-10T02:25:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:42 smithi082 ceph-mon[98694]: pgmap v6121: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 304 KiB/s wr, 18 op/s 2023-09-10T02:25:42.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11173.791887 secs 2023-09-10T02:25:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:42 smithi098 ceph-mon[103124]: pgmap v6121: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 304 KiB/s wr, 18 op/s 2023-09-10T02:25:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11173.791887 secs 2023-09-10T02:25:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11175.181912 secs 2023-09-10T02:25:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11175.181912 secs 2023-09-10T02:25:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11175.181912 secs 2023-09-10T02:25:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:44 smithi170 ceph-mon[102832]: pgmap v6122: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 204 KiB/s wr, 11 op/s 2023-09-10T02:25:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:25:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:25:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:25:44.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11175.893627 secs 2023-09-10T02:25:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:44 smithi082 ceph-mon[98694]: pgmap v6122: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 204 KiB/s wr, 11 op/s 2023-09-10T02:25:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:25:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:25:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:25:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11175.893627 secs 2023-09-10T02:25:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:44 smithi098 ceph-mon[103124]: pgmap v6122: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 204 KiB/s wr, 11 op/s 2023-09-10T02:25:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:25:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:25:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:25:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11175.893627 secs 2023-09-10T02:25:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:46 smithi170 ceph-mon[102832]: pgmap v6123: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 266 KiB/s wr, 17 op/s 2023-09-10T02:25:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:46 smithi082 ceph-mon[98694]: pgmap v6123: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 266 KiB/s wr, 17 op/s 2023-09-10T02:25:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:46 smithi098 ceph-mon[103124]: pgmap v6123: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 266 KiB/s wr, 17 op/s 2023-09-10T02:25:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11178.791951 secs 2023-09-10T02:25:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11178.791951 secs 2023-09-10T02:25:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11178.791951 secs 2023-09-10T02:25:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:48 smithi170 ceph-mon[102832]: pgmap v6124: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 211 KiB/s wr, 11 op/s 2023-09-10T02:25:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11180.181996 secs 2023-09-10T02:25:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:48 smithi082 ceph-mon[98694]: pgmap v6124: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 211 KiB/s wr, 11 op/s 2023-09-10T02:25:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11180.181996 secs 2023-09-10T02:25:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:48 smithi098 ceph-mon[103124]: pgmap v6124: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 211 KiB/s wr, 11 op/s 2023-09-10T02:25:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11180.181996 secs 2023-09-10T02:25:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11180.893692 secs 2023-09-10T02:25:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11180.893692 secs 2023-09-10T02:25:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11180.893692 secs 2023-09-10T02:25:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:50 smithi170 ceph-mon[102832]: pgmap v6125: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 211 KiB/s wr, 11 op/s 2023-09-10T02:25:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:50 smithi082 ceph-mon[98694]: pgmap v6125: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 211 KiB/s wr, 11 op/s 2023-09-10T02:25:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:50 smithi098 ceph-mon[103124]: pgmap v6125: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 211 KiB/s wr, 11 op/s 2023-09-10T02:25:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:52 smithi170 ceph-mon[102832]: pgmap v6126: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 240 KiB/s wr, 17 op/s 2023-09-10T02:25:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11183.792014 secs 2023-09-10T02:25:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:52 smithi082 ceph-mon[98694]: pgmap v6126: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 240 KiB/s wr, 17 op/s 2023-09-10T02:25:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11183.792014 secs 2023-09-10T02:25:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:52 smithi098 ceph-mon[103124]: pgmap v6126: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 240 KiB/s wr, 17 op/s 2023-09-10T02:25:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11183.792014 secs 2023-09-10T02:25:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11185.182053 secs 2023-09-10T02:25:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11185.182053 secs 2023-09-10T02:25:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11185.182053 secs 2023-09-10T02:25:54.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:54 smithi170 ceph-mon[102832]: pgmap v6127: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T02:25:54.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11185.893755 secs 2023-09-10T02:25:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:54 smithi082 ceph-mon[98694]: pgmap v6127: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T02:25:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11185.893755 secs 2023-09-10T02:25:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:54 smithi098 ceph-mon[103124]: pgmap v6127: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T02:25:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11185.893755 secs 2023-09-10T02:25:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:56 smithi170 ceph-mon[102832]: pgmap v6128: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 15 op/s 2023-09-10T02:25:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:56 smithi082 ceph-mon[98694]: pgmap v6128: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 15 op/s 2023-09-10T02:25:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:56 smithi098 ceph-mon[103124]: pgmap v6128: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 15 op/s 2023-09-10T02:25:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11188.792074 secs 2023-09-10T02:25:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11188.792074 secs 2023-09-10T02:25:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11188.792074 secs 2023-09-10T02:25:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:58 smithi170 ceph-mon[102832]: pgmap v6129: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T02:25:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11190.182112 secs 2023-09-10T02:25:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:58 smithi082 ceph-mon[98694]: pgmap v6129: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T02:25:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11190.182112 secs 2023-09-10T02:25:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:58 smithi098 ceph-mon[103124]: pgmap v6129: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T02:25:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11190.182112 secs 2023-09-10T02:25:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:25:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11190.893821 secs 2023-09-10T02:25:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:25:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11190.893821 secs 2023-09-10T02:26:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:25:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11190.893821 secs 2023-09-10T02:26:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:00 smithi170 ceph-mon[102832]: pgmap v6130: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T02:26:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:00 smithi082 ceph-mon[98694]: pgmap v6130: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T02:26:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:00 smithi098 ceph-mon[103124]: pgmap v6130: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T02:26:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:02 smithi170 ceph-mon[102832]: pgmap v6131: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 74 KiB/s wr, 15 op/s 2023-09-10T02:26:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11193.792146 secs 2023-09-10T02:26:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:02 smithi082 ceph-mon[98694]: pgmap v6131: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 74 KiB/s wr, 15 op/s 2023-09-10T02:26:02.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11193.792146 secs 2023-09-10T02:26:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:02 smithi098 ceph-mon[103124]: pgmap v6131: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 74 KiB/s wr, 15 op/s 2023-09-10T02:26:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11193.792146 secs 2023-09-10T02:26:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11195.182187 secs 2023-09-10T02:26:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11195.182187 secs 2023-09-10T02:26:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11195.182187 secs 2023-09-10T02:26:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:04 smithi170 ceph-mon[102832]: pgmap v6132: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T02:26:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11195.893890 secs 2023-09-10T02:26:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:04 smithi082 ceph-mon[98694]: pgmap v6132: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T02:26:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11195.893890 secs 2023-09-10T02:26:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:04 smithi098 ceph-mon[103124]: pgmap v6132: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 10 op/s 2023-09-10T02:26:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11195.893890 secs 2023-09-10T02:26:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:06 smithi170 ceph-mon[102832]: pgmap v6133: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 129 KiB/s wr, 30 op/s 2023-09-10T02:26:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:06 smithi082 ceph-mon[98694]: pgmap v6133: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 129 KiB/s wr, 30 op/s 2023-09-10T02:26:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:06 smithi098 ceph-mon[103124]: pgmap v6133: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 129 KiB/s wr, 30 op/s 2023-09-10T02:26:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11198.792209 secs 2023-09-10T02:26:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11198.792209 secs 2023-09-10T02:26:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11198.792209 secs 2023-09-10T02:26:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:08 smithi170 ceph-mon[102832]: pgmap v6134: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 109 KiB/s wr, 25 op/s 2023-09-10T02:26:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:08 smithi170 ceph-mon[102832]: osdmap e451: 12 total, 12 up, 12 in 2023-09-10T02:26:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11200.182292 secs 2023-09-10T02:26:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:08 smithi082 ceph-mon[98694]: pgmap v6134: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 109 KiB/s wr, 25 op/s 2023-09-10T02:26:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:08 smithi082 ceph-mon[98694]: osdmap e451: 12 total, 12 up, 12 in 2023-09-10T02:26:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11200.182292 secs 2023-09-10T02:26:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:08 smithi098 ceph-mon[103124]: pgmap v6134: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 109 KiB/s wr, 25 op/s 2023-09-10T02:26:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:08 smithi098 ceph-mon[103124]: osdmap e451: 12 total, 12 up, 12 in 2023-09-10T02:26:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11200.182292 secs 2023-09-10T02:26:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11200.893957 secs 2023-09-10T02:26:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11200.893957 secs 2023-09-10T02:26:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11200.893957 secs 2023-09-10T02:26:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:10 smithi170 ceph-mon[102832]: pgmap v6136: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 131 KiB/s wr, 30 op/s 2023-09-10T02:26:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:10 smithi082 ceph-mon[98694]: pgmap v6136: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 131 KiB/s wr, 30 op/s 2023-09-10T02:26:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:10 smithi098 ceph-mon[103124]: pgmap v6136: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 131 KiB/s wr, 30 op/s 2023-09-10T02:26:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:12 smithi170 ceph-mon[102832]: pgmap v6137: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T02:26:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11203.792270 secs 2023-09-10T02:26:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:12 smithi082 ceph-mon[98694]: pgmap v6137: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T02:26:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11203.792270 secs 2023-09-10T02:26:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:12 smithi098 ceph-mon[103124]: pgmap v6137: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T02:26:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11203.792270 secs 2023-09-10T02:26:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11205.182346 secs 2023-09-10T02:26:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11205.182346 secs 2023-09-10T02:26:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11205.182346 secs 2023-09-10T02:26:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:14 smithi170 ceph-mon[102832]: pgmap v6138: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T02:26:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11205.894023 secs 2023-09-10T02:26:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:14 smithi082 ceph-mon[98694]: pgmap v6138: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T02:26:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11205.894023 secs 2023-09-10T02:26:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:14 smithi098 ceph-mon[103124]: pgmap v6138: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T02:26:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11205.894023 secs 2023-09-10T02:26:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:16 smithi170 ceph-mon[102832]: pgmap v6139: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T02:26:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:16 smithi170 ceph-mon[102832]: osdmap e452: 12 total, 12 up, 12 in 2023-09-10T02:26:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:16 smithi082 ceph-mon[98694]: pgmap v6139: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T02:26:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:16 smithi082 ceph-mon[98694]: osdmap e452: 12 total, 12 up, 12 in 2023-09-10T02:26:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:16 smithi098 ceph-mon[103124]: pgmap v6139: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T02:26:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:16 smithi098 ceph-mon[103124]: osdmap e452: 12 total, 12 up, 12 in 2023-09-10T02:26:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11208.792331 secs 2023-09-10T02:26:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11208.792331 secs 2023-09-10T02:26:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11208.792331 secs 2023-09-10T02:26:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:18 smithi170 ceph-mon[102832]: pgmap v6141: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T02:26:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11210.182431 secs 2023-09-10T02:26:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:18 smithi082 ceph-mon[98694]: pgmap v6141: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T02:26:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11210.182431 secs 2023-09-10T02:26:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:18 smithi098 ceph-mon[103124]: pgmap v6141: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T02:26:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11210.182431 secs 2023-09-10T02:26:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11210.894089 secs 2023-09-10T02:26:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11210.894089 secs 2023-09-10T02:26:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11210.894089 secs 2023-09-10T02:26:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:20 smithi170 ceph-mon[102832]: pgmap v6142: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T02:26:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:20 smithi082 ceph-mon[98694]: pgmap v6142: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T02:26:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:20 smithi098 ceph-mon[103124]: pgmap v6142: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T02:26:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:22 smithi170 ceph-mon[102832]: pgmap v6143: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T02:26:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:22 smithi082 ceph-mon[98694]: pgmap v6143: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T02:26:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:22 smithi098 ceph-mon[103124]: pgmap v6143: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T02:26:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:26:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11215.182512 secs 2023-09-10T02:26:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:26:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11215.182512 secs 2023-09-10T02:26:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:26:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11215.182512 secs 2023-09-10T02:26:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:24 smithi170 ceph-mon[102832]: pgmap v6144: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T02:26:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:26:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:26:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:26:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11215.894151 secs 2023-09-10T02:26:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:24 smithi082 ceph-mon[98694]: pgmap v6144: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T02:26:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:26:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:26:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:26:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11215.894151 secs 2023-09-10T02:26:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:24 smithi098 ceph-mon[103124]: pgmap v6144: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 11 op/s 2023-09-10T02:26:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:26:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:26:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:26:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11215.894151 secs 2023-09-10T02:26:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:26 smithi170 ceph-mon[102832]: pgmap v6145: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T02:26:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:26 smithi082 ceph-mon[98694]: pgmap v6145: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T02:26:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:26 smithi098 ceph-mon[103124]: pgmap v6145: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 36 KiB/s wr, 11 op/s 2023-09-10T02:26:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:28 smithi170 ceph-mon[102832]: pgmap v6146: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-09-10T02:26:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11220.182570 secs 2023-09-10T02:26:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:28 smithi082 ceph-mon[98694]: pgmap v6146: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-09-10T02:26:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11220.182570 secs 2023-09-10T02:26:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:28 smithi098 ceph-mon[103124]: pgmap v6146: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-09-10T02:26:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11220.182570 secs 2023-09-10T02:26:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11220.894216 secs 2023-09-10T02:26:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11220.894216 secs 2023-09-10T02:26:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11220.894216 secs 2023-09-10T02:26:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:30 smithi170 ceph-mon[102832]: pgmap v6147: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 30 KiB/s wr, 9 op/s 2023-09-10T02:26:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:30 smithi082 ceph-mon[98694]: pgmap v6147: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 30 KiB/s wr, 9 op/s 2023-09-10T02:26:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:30 smithi098 ceph-mon[103124]: pgmap v6147: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 30 KiB/s wr, 9 op/s 2023-09-10T02:26:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:32 smithi170 ceph-mon[102832]: pgmap v6148: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 91 KiB/s wr, 14 op/s 2023-09-10T02:26:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:32 smithi082 ceph-mon[98694]: pgmap v6148: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 91 KiB/s wr, 14 op/s 2023-09-10T02:26:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:32 smithi098 ceph-mon[103124]: pgmap v6148: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 91 KiB/s wr, 14 op/s 2023-09-10T02:26:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11225.182653 secs 2023-09-10T02:26:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11225.182653 secs 2023-09-10T02:26:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11225.182653 secs 2023-09-10T02:26:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:34 smithi170 ceph-mon[102832]: pgmap v6149: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T02:26:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11225.894283 secs 2023-09-10T02:26:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:34 smithi082 ceph-mon[98694]: pgmap v6149: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T02:26:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11225.894283 secs 2023-09-10T02:26:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:34 smithi098 ceph-mon[103124]: pgmap v6149: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T02:26:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11225.894283 secs 2023-09-10T02:26:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:36 smithi170 ceph-mon[102832]: pgmap v6150: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 170 KiB/s wr, 16 op/s 2023-09-10T02:26:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:36 smithi082 ceph-mon[98694]: pgmap v6150: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 170 KiB/s wr, 16 op/s 2023-09-10T02:26:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:36 smithi098 ceph-mon[103124]: pgmap v6150: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 170 KiB/s wr, 16 op/s 2023-09-10T02:26:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:26:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:26:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:26:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:26:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:26:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:26:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:26:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:26:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:26:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:26:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:26:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:26:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:38 smithi170 ceph-mon[102832]: pgmap v6151: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 140 KiB/s wr, 11 op/s 2023-09-10T02:26:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11230.182714 secs 2023-09-10T02:26:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:38 smithi082 ceph-mon[98694]: pgmap v6151: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 140 KiB/s wr, 11 op/s 2023-09-10T02:26:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11230.182714 secs 2023-09-10T02:26:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:38 smithi098 ceph-mon[103124]: pgmap v6151: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 140 KiB/s wr, 11 op/s 2023-09-10T02:26:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11230.182714 secs 2023-09-10T02:26:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11230.894349 secs 2023-09-10T02:26:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11230.894349 secs 2023-09-10T02:26:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11230.894349 secs 2023-09-10T02:26:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:40 smithi170 ceph-mon[102832]: pgmap v6152: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 140 KiB/s wr, 11 op/s 2023-09-10T02:26:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:40 smithi082 ceph-mon[98694]: pgmap v6152: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 140 KiB/s wr, 11 op/s 2023-09-10T02:26:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:40 smithi098 ceph-mon[103124]: pgmap v6152: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 140 KiB/s wr, 11 op/s 2023-09-10T02:26:42.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:42 smithi170 ceph-mon[102832]: pgmap v6153: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 192 KiB/s wr, 17 op/s 2023-09-10T02:26:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:42 smithi082 ceph-mon[98694]: pgmap v6153: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 192 KiB/s wr, 17 op/s 2023-09-10T02:26:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:42 smithi098 ceph-mon[103124]: pgmap v6153: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 192 KiB/s wr, 17 op/s 2023-09-10T02:26:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11235.182797 secs 2023-09-10T02:26:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11235.182797 secs 2023-09-10T02:26:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11235.182797 secs 2023-09-10T02:26:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:44 smithi170 ceph-mon[102832]: pgmap v6154: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T02:26:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11235.894415 secs 2023-09-10T02:26:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:44 smithi082 ceph-mon[98694]: pgmap v6154: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T02:26:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11235.894415 secs 2023-09-10T02:26:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:44 smithi098 ceph-mon[103124]: pgmap v6154: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T02:26:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11235.894415 secs 2023-09-10T02:26:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:46 smithi170 ceph-mon[102832]: pgmap v6155: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 201 KiB/s wr, 18 op/s 2023-09-10T02:26:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:46 smithi082 ceph-mon[98694]: pgmap v6155: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 201 KiB/s wr, 18 op/s 2023-09-10T02:26:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:46 smithi098 ceph-mon[103124]: pgmap v6155: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 201 KiB/s wr, 18 op/s 2023-09-10T02:26:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:48 smithi170 ceph-mon[102832]: pgmap v6156: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T02:26:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11240.182855 secs 2023-09-10T02:26:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:48 smithi082 ceph-mon[98694]: pgmap v6156: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T02:26:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11240.182855 secs 2023-09-10T02:26:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:48 smithi098 ceph-mon[103124]: pgmap v6156: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T02:26:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11240.182855 secs 2023-09-10T02:26:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11240.894482 secs 2023-09-10T02:26:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11240.894482 secs 2023-09-10T02:26:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11240.894482 secs 2023-09-10T02:26:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:50 smithi170 ceph-mon[102832]: pgmap v6157: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T02:26:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:50 smithi082 ceph-mon[98694]: pgmap v6157: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T02:26:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:50 smithi098 ceph-mon[103124]: pgmap v6157: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T02:26:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:52 smithi170 ceph-mon[102832]: pgmap v6158: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 214 KiB/s wr, 18 op/s 2023-09-10T02:26:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11243.792924 secs 2023-09-10T02:26:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:52 smithi082 ceph-mon[98694]: pgmap v6158: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 214 KiB/s wr, 18 op/s 2023-09-10T02:26:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11243.792924 secs 2023-09-10T02:26:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:52 smithi098 ceph-mon[103124]: pgmap v6158: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 214 KiB/s wr, 18 op/s 2023-09-10T02:26:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11243.792924 secs 2023-09-10T02:26:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11245.182914 secs 2023-09-10T02:26:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11245.182914 secs 2023-09-10T02:26:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11245.182914 secs 2023-09-10T02:26:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:54 smithi170 ceph-mon[102832]: pgmap v6159: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 162 KiB/s wr, 12 op/s 2023-09-10T02:26:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11245.894548 secs 2023-09-10T02:26:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:54 smithi082 ceph-mon[98694]: pgmap v6159: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 162 KiB/s wr, 12 op/s 2023-09-10T02:26:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11245.894548 secs 2023-09-10T02:26:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:54 smithi098 ceph-mon[103124]: pgmap v6159: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 162 KiB/s wr, 12 op/s 2023-09-10T02:26:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11245.894548 secs 2023-09-10T02:26:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:26:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:26:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:26:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:26:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:26:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:26:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:26:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:26:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:26:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:56 smithi170 ceph-mon[102832]: pgmap v6160: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 222 KiB/s wr, 18 op/s 2023-09-10T02:26:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:56 smithi082 ceph-mon[98694]: pgmap v6160: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 222 KiB/s wr, 18 op/s 2023-09-10T02:26:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:56 smithi098 ceph-mon[103124]: pgmap v6160: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 222 KiB/s wr, 18 op/s 2023-09-10T02:26:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11248.792985 secs 2023-09-10T02:26:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11248.792985 secs 2023-09-10T02:26:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11248.792985 secs 2023-09-10T02:26:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:58 smithi082 ceph-mon[98694]: pgmap v6161: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T02:26:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11250.182973 secs 2023-09-10T02:26:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:58 smithi098 ceph-mon[103124]: pgmap v6161: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T02:26:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11250.182973 secs 2023-09-10T02:26:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:58 smithi170 ceph-mon[102832]: pgmap v6161: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T02:26:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11250.182973 secs 2023-09-10T02:26:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:26:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11250.894613 secs 2023-09-10T02:27:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:26:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11250.894613 secs 2023-09-10T02:27:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:26:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11250.894613 secs 2023-09-10T02:27:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:00 smithi082 ceph-mon[98694]: pgmap v6162: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T02:27:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:00 smithi098 ceph-mon[103124]: pgmap v6162: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T02:27:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:00 smithi170 ceph-mon[102832]: pgmap v6162: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T02:27:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:02 smithi170 ceph-mon[102832]: pgmap v6163: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 184 KiB/s wr, 16 op/s 2023-09-10T02:27:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11253.793084 secs 2023-09-10T02:27:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:02 smithi082 ceph-mon[98694]: pgmap v6163: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 184 KiB/s wr, 16 op/s 2023-09-10T02:27:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11253.793084 secs 2023-09-10T02:27:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:02 smithi098 ceph-mon[103124]: pgmap v6163: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 184 KiB/s wr, 16 op/s 2023-09-10T02:27:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11253.793084 secs 2023-09-10T02:27:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11255.183048 secs 2023-09-10T02:27:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11255.183048 secs 2023-09-10T02:27:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11255.183048 secs 2023-09-10T02:27:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:04 smithi170 ceph-mon[102832]: pgmap v6164: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T02:27:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11255.894679 secs 2023-09-10T02:27:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:04 smithi082 ceph-mon[98694]: pgmap v6164: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T02:27:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11255.894679 secs 2023-09-10T02:27:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:04 smithi098 ceph-mon[103124]: pgmap v6164: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T02:27:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11255.894679 secs 2023-09-10T02:27:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:06 smithi170 ceph-mon[102832]: pgmap v6165: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 157 KiB/s wr, 29 op/s 2023-09-10T02:27:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:06 smithi082 ceph-mon[98694]: pgmap v6165: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 157 KiB/s wr, 29 op/s 2023-09-10T02:27:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:06 smithi098 ceph-mon[103124]: pgmap v6165: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 157 KiB/s wr, 29 op/s 2023-09-10T02:27:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11258.793145 secs 2023-09-10T02:27:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11258.793145 secs 2023-09-10T02:27:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11258.793145 secs 2023-09-10T02:27:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:08 smithi170 ceph-mon[102832]: pgmap v6166: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 97 KiB/s wr, 24 op/s 2023-09-10T02:27:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:08 smithi170 ceph-mon[102832]: osdmap e453: 12 total, 12 up, 12 in 2023-09-10T02:27:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11260.183106 secs 2023-09-10T02:27:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:08 smithi082 ceph-mon[98694]: pgmap v6166: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 97 KiB/s wr, 24 op/s 2023-09-10T02:27:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:08 smithi082 ceph-mon[98694]: osdmap e453: 12 total, 12 up, 12 in 2023-09-10T02:27:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11260.183106 secs 2023-09-10T02:27:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:08 smithi098 ceph-mon[103124]: pgmap v6166: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 97 KiB/s wr, 24 op/s 2023-09-10T02:27:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:08 smithi098 ceph-mon[103124]: osdmap e453: 12 total, 12 up, 12 in 2023-09-10T02:27:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11260.183106 secs 2023-09-10T02:27:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11260.894746 secs 2023-09-10T02:27:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11260.894746 secs 2023-09-10T02:27:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11260.894746 secs 2023-09-10T02:27:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:10 smithi170 ceph-mon[102832]: pgmap v6168: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 116 KiB/s wr, 29 op/s 2023-09-10T02:27:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:10 smithi082 ceph-mon[98694]: pgmap v6168: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 116 KiB/s wr, 29 op/s 2023-09-10T02:27:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:10 smithi098 ceph-mon[103124]: pgmap v6168: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 116 KiB/s wr, 29 op/s 2023-09-10T02:27:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:12 smithi170 ceph-mon[102832]: pgmap v6169: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 79 KiB/s wr, 28 op/s 2023-09-10T02:27:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11263.793226 secs 2023-09-10T02:27:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:12 smithi082 ceph-mon[98694]: pgmap v6169: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 79 KiB/s wr, 28 op/s 2023-09-10T02:27:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11263.793226 secs 2023-09-10T02:27:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:12 smithi098 ceph-mon[103124]: pgmap v6169: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 79 KiB/s wr, 28 op/s 2023-09-10T02:27:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11263.793226 secs 2023-09-10T02:27:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11265.183163 secs 2023-09-10T02:27:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11265.183163 secs 2023-09-10T02:27:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11265.183163 secs 2023-09-10T02:27:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:14 smithi170 ceph-mon[102832]: pgmap v6170: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 79 KiB/s wr, 28 op/s 2023-09-10T02:27:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11265.894811 secs 2023-09-10T02:27:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:14 smithi082 ceph-mon[98694]: pgmap v6170: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 79 KiB/s wr, 28 op/s 2023-09-10T02:27:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11265.894811 secs 2023-09-10T02:27:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:14 smithi098 ceph-mon[103124]: pgmap v6170: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 79 KiB/s wr, 28 op/s 2023-09-10T02:27:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11265.894811 secs 2023-09-10T02:27:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:16 smithi170 ceph-mon[102832]: pgmap v6171: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:27:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:16 smithi170 ceph-mon[102832]: osdmap e454: 12 total, 12 up, 12 in 2023-09-10T02:27:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:16 smithi082 ceph-mon[98694]: pgmap v6171: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:27:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:16 smithi082 ceph-mon[98694]: osdmap e454: 12 total, 12 up, 12 in 2023-09-10T02:27:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:16 smithi098 ceph-mon[103124]: pgmap v6171: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:27:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:16 smithi098 ceph-mon[103124]: osdmap e454: 12 total, 12 up, 12 in 2023-09-10T02:27:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11268.793285 secs 2023-09-10T02:27:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11268.793285 secs 2023-09-10T02:27:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11268.793285 secs 2023-09-10T02:27:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:18 smithi170 ceph-mon[102832]: pgmap v6173: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:27:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11270.183221 secs 2023-09-10T02:27:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:18 smithi082 ceph-mon[98694]: pgmap v6173: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:27:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11270.183221 secs 2023-09-10T02:27:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:18 smithi098 ceph-mon[103124]: pgmap v6173: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:27:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11270.183221 secs 2023-09-10T02:27:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11270.894879 secs 2023-09-10T02:27:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11270.894879 secs 2023-09-10T02:27:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11270.894879 secs 2023-09-10T02:27:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:20 smithi170 ceph-mon[102832]: pgmap v6174: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:27:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:20 smithi082 ceph-mon[98694]: pgmap v6174: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:27:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:20 smithi098 ceph-mon[103124]: pgmap v6174: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:27:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:22 smithi170 ceph-mon[102832]: pgmap v6175: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:27:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11273.793393 secs 2023-09-10T02:27:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:22 smithi082 ceph-mon[98694]: pgmap v6175: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:27:22.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11273.793393 secs 2023-09-10T02:27:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:22 smithi098 ceph-mon[103124]: pgmap v6175: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:27:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11273.793393 secs 2023-09-10T02:27:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11275.183277 secs 2023-09-10T02:27:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11275.183277 secs 2023-09-10T02:27:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11275.183277 secs 2023-09-10T02:27:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:24 smithi170 ceph-mon[102832]: pgmap v6176: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:27:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11275.894943 secs 2023-09-10T02:27:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:24 smithi082 ceph-mon[98694]: pgmap v6176: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:27:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11275.894943 secs 2023-09-10T02:27:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:24 smithi098 ceph-mon[103124]: pgmap v6176: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:27:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11275.894943 secs 2023-09-10T02:27:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:26 smithi170 ceph-mon[102832]: pgmap v6177: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T02:27:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:26 smithi082 ceph-mon[98694]: pgmap v6177: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T02:27:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:26 smithi098 ceph-mon[103124]: pgmap v6177: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T02:27:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:28 smithi170 ceph-mon[102832]: pgmap v6178: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 10 op/s 2023-09-10T02:27:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11280.183335 secs 2023-09-10T02:27:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:28 smithi082 ceph-mon[98694]: pgmap v6178: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 10 op/s 2023-09-10T02:27:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11280.183335 secs 2023-09-10T02:27:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:28 smithi098 ceph-mon[103124]: pgmap v6178: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 10 op/s 2023-09-10T02:27:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11280.183335 secs 2023-09-10T02:27:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11280.895011 secs 2023-09-10T02:27:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11280.895011 secs 2023-09-10T02:27:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11280.895011 secs 2023-09-10T02:27:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:30 smithi170 ceph-mon[102832]: pgmap v6179: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T02:27:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:30 smithi082 ceph-mon[98694]: pgmap v6179: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T02:27:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:30 smithi098 ceph-mon[103124]: pgmap v6179: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T02:27:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:27:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:27:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:27:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:32 smithi170 ceph-mon[102832]: pgmap v6180: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 190 KiB/s wr, 16 op/s 2023-09-10T02:27:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:27:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:27:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:27:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:32 smithi082 ceph-mon[98694]: pgmap v6180: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 190 KiB/s wr, 16 op/s 2023-09-10T02:27:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:27:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:27:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:27:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:32 smithi098 ceph-mon[103124]: pgmap v6180: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 190 KiB/s wr, 16 op/s 2023-09-10T02:27:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:27:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:27:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:27:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11285.183417 secs 2023-09-10T02:27:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11285.183417 secs 2023-09-10T02:27:34.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11285.183417 secs 2023-09-10T02:27:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:34 smithi170 ceph-mon[102832]: pgmap v6181: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T02:27:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11285.895075 secs 2023-09-10T02:27:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:34 smithi082 ceph-mon[98694]: pgmap v6181: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T02:27:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11285.895075 secs 2023-09-10T02:27:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:34 smithi098 ceph-mon[103124]: pgmap v6181: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T02:27:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11285.895075 secs 2023-09-10T02:27:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:36 smithi170 ceph-mon[102832]: pgmap v6182: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 199 KiB/s wr, 17 op/s 2023-09-10T02:27:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:36 smithi082 ceph-mon[98694]: pgmap v6182: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 199 KiB/s wr, 17 op/s 2023-09-10T02:27:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:36 smithi098 ceph-mon[103124]: pgmap v6182: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 199 KiB/s wr, 17 op/s 2023-09-10T02:27:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:38 smithi170 ceph-mon[102832]: pgmap v6183: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 140 KiB/s wr, 12 op/s 2023-09-10T02:27:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:27:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11290.183476 secs 2023-09-10T02:27:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:27:38.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:27:38.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:27:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:38 smithi082 ceph-mon[98694]: pgmap v6183: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 140 KiB/s wr, 12 op/s 2023-09-10T02:27:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:27:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11290.183476 secs 2023-09-10T02:27:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:27:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:27:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:27:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:38 smithi098 ceph-mon[103124]: pgmap v6183: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 140 KiB/s wr, 12 op/s 2023-09-10T02:27:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:27:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11290.183476 secs 2023-09-10T02:27:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:27:39.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:27:39.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:27:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11290.895143 secs 2023-09-10T02:27:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11290.895143 secs 2023-09-10T02:27:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11290.895143 secs 2023-09-10T02:27:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:40 smithi082 ceph-mon[98694]: pgmap v6184: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 140 KiB/s wr, 12 op/s 2023-09-10T02:27:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:40 smithi098 ceph-mon[103124]: pgmap v6184: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 140 KiB/s wr, 12 op/s 2023-09-10T02:27:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:40 smithi170 ceph-mon[102832]: pgmap v6184: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 140 KiB/s wr, 12 op/s 2023-09-10T02:27:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:42 smithi082 ceph-mon[98694]: pgmap v6185: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 260 KiB/s wr, 18 op/s 2023-09-10T02:27:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:42 smithi098 ceph-mon[103124]: pgmap v6185: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 260 KiB/s wr, 18 op/s 2023-09-10T02:27:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:42 smithi170 ceph-mon[102832]: pgmap v6185: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 260 KiB/s wr, 18 op/s 2023-09-10T02:27:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11295.183551 secs 2023-09-10T02:27:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11295.183551 secs 2023-09-10T02:27:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11295.183551 secs 2023-09-10T02:27:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:44 smithi082 ceph-mon[98694]: pgmap v6186: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 130 KiB/s wr, 12 op/s 2023-09-10T02:27:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11295.895210 secs 2023-09-10T02:27:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:44 smithi098 ceph-mon[103124]: pgmap v6186: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 130 KiB/s wr, 12 op/s 2023-09-10T02:27:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11295.895210 secs 2023-09-10T02:27:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:44 smithi170 ceph-mon[102832]: pgmap v6186: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 130 KiB/s wr, 12 op/s 2023-09-10T02:27:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11295.895210 secs 2023-09-10T02:27:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:46 smithi082 ceph-mon[98694]: pgmap v6187: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 232 KiB/s wr, 18 op/s 2023-09-10T02:27:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:46 smithi098 ceph-mon[103124]: pgmap v6187: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 232 KiB/s wr, 18 op/s 2023-09-10T02:27:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:46 smithi170 ceph-mon[102832]: pgmap v6187: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 232 KiB/s wr, 18 op/s 2023-09-10T02:27:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11298.793743 secs 2023-09-10T02:27:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11298.793743 secs 2023-09-10T02:27:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11298.793743 secs 2023-09-10T02:27:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:48 smithi082 ceph-mon[98694]: pgmap v6188: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 221 KiB/s wr, 12 op/s 2023-09-10T02:27:48.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11300.183625 secs 2023-09-10T02:27:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:48 smithi098 ceph-mon[103124]: pgmap v6188: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 221 KiB/s wr, 12 op/s 2023-09-10T02:27:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11300.183625 secs 2023-09-10T02:27:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:48 smithi170 ceph-mon[102832]: pgmap v6188: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 221 KiB/s wr, 12 op/s 2023-09-10T02:27:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11300.183625 secs 2023-09-10T02:27:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11300.895274 secs 2023-09-10T02:27:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11300.895274 secs 2023-09-10T02:27:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11300.895274 secs 2023-09-10T02:27:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:50 smithi098 ceph-mon[103124]: pgmap v6189: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 221 KiB/s wr, 12 op/s 2023-09-10T02:27:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:50 smithi170 ceph-mon[102832]: pgmap v6189: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 221 KiB/s wr, 12 op/s 2023-09-10T02:27:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:50 smithi082 ceph-mon[98694]: pgmap v6189: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 221 KiB/s wr, 12 op/s 2023-09-10T02:27:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:52 smithi170 ceph-mon[102832]: pgmap v6190: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 248 KiB/s wr, 17 op/s 2023-09-10T02:27:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:27:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:27:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:27:52.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11303.793773 secs 2023-09-10T02:27:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:52 smithi082 ceph-mon[98694]: pgmap v6190: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 248 KiB/s wr, 17 op/s 2023-09-10T02:27:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:27:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:27:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:27:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11303.793773 secs 2023-09-10T02:27:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:52 smithi098 ceph-mon[103124]: pgmap v6190: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 248 KiB/s wr, 17 op/s 2023-09-10T02:27:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:27:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:27:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:27:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11303.793773 secs 2023-09-10T02:27:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11305.183705 secs 2023-09-10T02:27:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11305.183705 secs 2023-09-10T02:27:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11305.183705 secs 2023-09-10T02:27:54.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:54 smithi170 ceph-mon[102832]: pgmap v6191: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T02:27:54.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11305.895340 secs 2023-09-10T02:27:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:54 smithi082 ceph-mon[98694]: pgmap v6191: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T02:27:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11305.895340 secs 2023-09-10T02:27:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:54 smithi098 ceph-mon[103124]: pgmap v6191: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T02:27:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11305.895340 secs 2023-09-10T02:27:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:56 smithi170 ceph-mon[102832]: pgmap v6192: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 192 KiB/s wr, 16 op/s 2023-09-10T02:27:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:56 smithi082 ceph-mon[98694]: pgmap v6192: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 192 KiB/s wr, 16 op/s 2023-09-10T02:27:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:56 smithi098 ceph-mon[103124]: pgmap v6192: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 192 KiB/s wr, 16 op/s 2023-09-10T02:27:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11308.793900 secs 2023-09-10T02:27:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11308.793900 secs 2023-09-10T02:27:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11308.793900 secs 2023-09-10T02:27:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:58 smithi170 ceph-mon[102832]: pgmap v6193: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T02:27:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11310.183765 secs 2023-09-10T02:27:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:58 smithi082 ceph-mon[98694]: pgmap v6193: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T02:27:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11310.183765 secs 2023-09-10T02:27:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:58 smithi098 ceph-mon[103124]: pgmap v6193: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T02:27:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11310.183765 secs 2023-09-10T02:27:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:27:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11310.895408 secs 2023-09-10T02:27:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:27:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11310.895408 secs 2023-09-10T02:28:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:27:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11310.895408 secs 2023-09-10T02:28:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:00 smithi170 ceph-mon[102832]: pgmap v6194: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T02:28:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:00 smithi082 ceph-mon[98694]: pgmap v6194: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T02:28:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:00 smithi098 ceph-mon[103124]: pgmap v6194: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T02:28:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:02 smithi170 ceph-mon[102832]: pgmap v6195: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 164 KiB/s wr, 30 op/s 2023-09-10T02:28:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11313.793962 secs 2023-09-10T02:28:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:02 smithi082 ceph-mon[98694]: pgmap v6195: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 164 KiB/s wr, 30 op/s 2023-09-10T02:28:02.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11313.793962 secs 2023-09-10T02:28:03.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:02 smithi098 ceph-mon[103124]: pgmap v6195: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 164 KiB/s wr, 30 op/s 2023-09-10T02:28:03.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11313.793962 secs 2023-09-10T02:28:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11315.183851 secs 2023-09-10T02:28:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11315.183851 secs 2023-09-10T02:28:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11315.183851 secs 2023-09-10T02:28:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:04 smithi170 ceph-mon[102832]: pgmap v6196: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 138 KiB/s wr, 24 op/s 2023-09-10T02:28:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11315.895487 secs 2023-09-10T02:28:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:04 smithi082 ceph-mon[98694]: pgmap v6196: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 138 KiB/s wr, 24 op/s 2023-09-10T02:28:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11315.895487 secs 2023-09-10T02:28:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:04 smithi098 ceph-mon[103124]: pgmap v6196: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 138 KiB/s wr, 24 op/s 2023-09-10T02:28:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11315.895487 secs 2023-09-10T02:28:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:06 smithi170 ceph-mon[102832]: pgmap v6197: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 147 KiB/s wr, 29 op/s 2023-09-10T02:28:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:06 smithi082 ceph-mon[98694]: pgmap v6197: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 147 KiB/s wr, 29 op/s 2023-09-10T02:28:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:06 smithi098 ceph-mon[103124]: pgmap v6197: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 147 KiB/s wr, 29 op/s 2023-09-10T02:28:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11318.794024 secs 2023-09-10T02:28:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11318.794024 secs 2023-09-10T02:28:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11318.794024 secs 2023-09-10T02:28:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:08 smithi170 ceph-mon[102832]: pgmap v6198: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 83 KiB/s wr, 24 op/s 2023-09-10T02:28:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:08 smithi170 ceph-mon[102832]: osdmap e455: 12 total, 12 up, 12 in 2023-09-10T02:28:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11320.183911 secs 2023-09-10T02:28:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:08 smithi082 ceph-mon[98694]: pgmap v6198: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 83 KiB/s wr, 24 op/s 2023-09-10T02:28:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:08 smithi082 ceph-mon[98694]: osdmap e455: 12 total, 12 up, 12 in 2023-09-10T02:28:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11320.183911 secs 2023-09-10T02:28:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:08 smithi098 ceph-mon[103124]: pgmap v6198: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 83 KiB/s wr, 24 op/s 2023-09-10T02:28:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:08 smithi098 ceph-mon[103124]: osdmap e455: 12 total, 12 up, 12 in 2023-09-10T02:28:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11320.183911 secs 2023-09-10T02:28:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11320.895555 secs 2023-09-10T02:28:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11320.895555 secs 2023-09-10T02:28:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11320.895555 secs 2023-09-10T02:28:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:10 smithi170 ceph-mon[102832]: pgmap v6200: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 99 KiB/s wr, 29 op/s 2023-09-10T02:28:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:10 smithi082 ceph-mon[98694]: pgmap v6200: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 99 KiB/s wr, 29 op/s 2023-09-10T02:28:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:10 smithi098 ceph-mon[103124]: pgmap v6200: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 99 KiB/s wr, 29 op/s 2023-09-10T02:28:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:12 smithi170 ceph-mon[102832]: pgmap v6201: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T02:28:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11323.794110 secs 2023-09-10T02:28:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:12 smithi082 ceph-mon[98694]: pgmap v6201: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T02:28:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11323.794110 secs 2023-09-10T02:28:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:12 smithi098 ceph-mon[103124]: pgmap v6201: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T02:28:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11323.794110 secs 2023-09-10T02:28:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11325.183994 secs 2023-09-10T02:28:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11325.183994 secs 2023-09-10T02:28:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11325.183994 secs 2023-09-10T02:28:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:14 smithi082 ceph-mon[98694]: pgmap v6202: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T02:28:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11325.895620 secs 2023-09-10T02:28:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:14 smithi098 ceph-mon[103124]: pgmap v6202: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T02:28:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11325.895620 secs 2023-09-10T02:28:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:14 smithi170 ceph-mon[102832]: pgmap v6202: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T02:28:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11325.895620 secs 2023-09-10T02:28:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:16 smithi082 ceph-mon[98694]: pgmap v6203: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 12 op/s 2023-09-10T02:28:16.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:16 smithi082 ceph-mon[98694]: osdmap e456: 12 total, 12 up, 12 in 2023-09-10T02:28:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:16 smithi098 ceph-mon[103124]: pgmap v6203: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 12 op/s 2023-09-10T02:28:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:16 smithi098 ceph-mon[103124]: osdmap e456: 12 total, 12 up, 12 in 2023-09-10T02:28:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:16 smithi170 ceph-mon[102832]: pgmap v6203: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 12 op/s 2023-09-10T02:28:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:16 smithi170 ceph-mon[102832]: osdmap e456: 12 total, 12 up, 12 in 2023-09-10T02:28:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11328.794233 secs 2023-09-10T02:28:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11328.794233 secs 2023-09-10T02:28:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11328.794233 secs 2023-09-10T02:28:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:18 smithi082 ceph-mon[98694]: pgmap v6205: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 12 op/s 2023-09-10T02:28:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11330.184052 secs 2023-09-10T02:28:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:18 smithi098 ceph-mon[103124]: pgmap v6205: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 12 op/s 2023-09-10T02:28:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11330.184052 secs 2023-09-10T02:28:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:18 smithi170 ceph-mon[102832]: pgmap v6205: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 12 op/s 2023-09-10T02:28:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11330.184052 secs 2023-09-10T02:28:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11330.895684 secs 2023-09-10T02:28:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11330.895684 secs 2023-09-10T02:28:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11330.895684 secs 2023-09-10T02:28:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:20 smithi082 ceph-mon[98694]: pgmap v6206: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 12 op/s 2023-09-10T02:28:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:20 smithi098 ceph-mon[103124]: pgmap v6206: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 12 op/s 2023-09-10T02:28:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:20 smithi170 ceph-mon[102832]: pgmap v6206: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 12 op/s 2023-09-10T02:28:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:22 smithi082 ceph-mon[98694]: pgmap v6207: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T02:28:22.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11333.794309 secs 2023-09-10T02:28:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:22 smithi098 ceph-mon[103124]: pgmap v6207: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T02:28:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11333.794309 secs 2023-09-10T02:28:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:22 smithi170 ceph-mon[102832]: pgmap v6207: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T02:28:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11333.794309 secs 2023-09-10T02:28:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11335.184110 secs 2023-09-10T02:28:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11335.184110 secs 2023-09-10T02:28:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11335.184110 secs 2023-09-10T02:28:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:24 smithi098 ceph-mon[103124]: pgmap v6208: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T02:28:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11335.895748 secs 2023-09-10T02:28:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:24 smithi170 ceph-mon[102832]: pgmap v6208: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T02:28:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11335.895748 secs 2023-09-10T02:28:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:24 smithi082 ceph-mon[98694]: pgmap v6208: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T02:28:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11335.895748 secs 2023-09-10T02:28:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:26 smithi098 ceph-mon[103124]: pgmap v6209: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T02:28:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:26 smithi170 ceph-mon[102832]: pgmap v6209: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T02:28:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:26 smithi082 ceph-mon[98694]: pgmap v6209: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T02:28:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:28 smithi098 ceph-mon[103124]: pgmap v6210: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 10 op/s 2023-09-10T02:28:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11340.184178 secs 2023-09-10T02:28:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:28 smithi170 ceph-mon[102832]: pgmap v6210: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 10 op/s 2023-09-10T02:28:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11340.184178 secs 2023-09-10T02:28:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:28 smithi082 ceph-mon[98694]: pgmap v6210: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 10 op/s 2023-09-10T02:28:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11340.184178 secs 2023-09-10T02:28:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11340.895812 secs 2023-09-10T02:28:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11340.895812 secs 2023-09-10T02:28:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11340.895812 secs 2023-09-10T02:28:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:30 smithi098 ceph-mon[103124]: pgmap v6211: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T02:28:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:30 smithi170 ceph-mon[102832]: pgmap v6211: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T02:28:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:30 smithi082 ceph-mon[98694]: pgmap v6211: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T02:28:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:28:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:28:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:28:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:32 smithi098 ceph-mon[103124]: pgmap v6212: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 176 KiB/s wr, 16 op/s 2023-09-10T02:28:33.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:28:33.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:28:33.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:28:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:32 smithi170 ceph-mon[102832]: pgmap v6212: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 176 KiB/s wr, 16 op/s 2023-09-10T02:28:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:28:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:28:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:28:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:32 smithi082 ceph-mon[98694]: pgmap v6212: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 176 KiB/s wr, 16 op/s 2023-09-10T02:28:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:28:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:28:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:28:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11345.184262 secs 2023-09-10T02:28:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11345.184262 secs 2023-09-10T02:28:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11345.184262 secs 2023-09-10T02:28:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:34 smithi098 ceph-mon[103124]: pgmap v6213: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T02:28:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11345.895876 secs 2023-09-10T02:28:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:34 smithi170 ceph-mon[102832]: pgmap v6213: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T02:28:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11345.895876 secs 2023-09-10T02:28:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:34 smithi082 ceph-mon[98694]: pgmap v6213: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T02:28:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11345.895876 secs 2023-09-10T02:28:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:36 smithi098 ceph-mon[103124]: pgmap v6214: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 199 KiB/s wr, 17 op/s 2023-09-10T02:28:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:36 smithi170 ceph-mon[102832]: pgmap v6214: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 199 KiB/s wr, 17 op/s 2023-09-10T02:28:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:36 smithi082 ceph-mon[98694]: pgmap v6214: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 199 KiB/s wr, 17 op/s 2023-09-10T02:28:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:38 smithi098 ceph-mon[103124]: pgmap v6215: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 164 KiB/s wr, 12 op/s 2023-09-10T02:28:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11350.184322 secs 2023-09-10T02:28:39.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:28:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:38 smithi170 ceph-mon[102832]: pgmap v6215: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 164 KiB/s wr, 12 op/s 2023-09-10T02:28:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11350.184322 secs 2023-09-10T02:28:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:28:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:38 smithi082 ceph-mon[98694]: pgmap v6215: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 164 KiB/s wr, 12 op/s 2023-09-10T02:28:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11350.184322 secs 2023-09-10T02:28:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:28:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11350.895948 secs 2023-09-10T02:28:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11350.895948 secs 2023-09-10T02:28:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11350.895948 secs 2023-09-10T02:28:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:40 smithi098 ceph-mon[103124]: pgmap v6216: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 164 KiB/s wr, 12 op/s 2023-09-10T02:28:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:40 smithi170 ceph-mon[102832]: pgmap v6216: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 164 KiB/s wr, 12 op/s 2023-09-10T02:28:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:40 smithi082 ceph-mon[98694]: pgmap v6216: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 164 KiB/s wr, 12 op/s 2023-09-10T02:28:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:28:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:28:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:28:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:28:42.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:28:42.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:28:42.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:42.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:42.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:42.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:42.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:42.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:42.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:42.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:28:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:28:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:28:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:28:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:28:42.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:28:42.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:42.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:42.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:42.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:42.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:42.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:42.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:42.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:28:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:28:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:28:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:28:42.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:28:42.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:28:42.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:42.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:42.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:42.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:42.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:42.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:42.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:42.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:43 smithi170 ceph-mon[102832]: pgmap v6217: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 236 KiB/s wr, 18 op/s 2023-09-10T02:28:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:43 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T02:28:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:43 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 183.1M 2023-09-10T02:28:43.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:43 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T02:28:43.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:43 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 192055046: error parsing value: Value '192055046' is below minimum 939524096 2023-09-10T02:28:43.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11353.794699 secs 2023-09-10T02:28:43.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:43.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:43.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:43.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:43.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:43 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T02:28:43.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:43 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T02:28:43.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:28:43.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:28:43.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:28:43.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11355.184382 secs 2023-09-10T02:28:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:43 smithi082 ceph-mon[98694]: pgmap v6217: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 236 KiB/s wr, 18 op/s 2023-09-10T02:28:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:43 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T02:28:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:43 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 183.1M 2023-09-10T02:28:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:43 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T02:28:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:43 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 192055046: error parsing value: Value '192055046' is below minimum 939524096 2023-09-10T02:28:43.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11353.794699 secs 2023-09-10T02:28:43.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:43.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:43.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:43.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:43.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:43 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T02:28:43.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:43 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T02:28:43.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:28:43.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:28:43.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:28:43.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11355.184382 secs 2023-09-10T02:28:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:43 smithi098 ceph-mon[103124]: pgmap v6217: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 236 KiB/s wr, 18 op/s 2023-09-10T02:28:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:43 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T02:28:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:43 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 183.1M 2023-09-10T02:28:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:43 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T02:28:43.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:43 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 192055046: error parsing value: Value '192055046' is below minimum 939524096 2023-09-10T02:28:43.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11353.794699 secs 2023-09-10T02:28:43.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:43.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:43.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:43.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:28:43.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:43 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T02:28:43.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:43 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T02:28:43.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:28:43.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:28:43.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:28:43.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11355.184382 secs 2023-09-10T02:28:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:28:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:28:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:28:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11355.896012 secs 2023-09-10T02:28:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:28:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:28:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:28:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11355.896012 secs 2023-09-10T02:28:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:28:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:28:44.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:28:44.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11355.896012 secs 2023-09-10T02:28:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:45 smithi170 ceph-mon[102832]: pgmap v6218: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T02:28:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:45 smithi082 ceph-mon[98694]: pgmap v6218: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T02:28:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:45 smithi098 ceph-mon[103124]: pgmap v6218: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T02:28:47.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:47 smithi170 ceph-mon[102832]: pgmap v6219: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 192 KiB/s wr, 18 op/s 2023-09-10T02:28:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:47 smithi082 ceph-mon[98694]: pgmap v6219: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 192 KiB/s wr, 18 op/s 2023-09-10T02:28:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:47 smithi098 ceph-mon[103124]: pgmap v6219: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 192 KiB/s wr, 18 op/s 2023-09-10T02:28:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11358.794759 secs 2023-09-10T02:28:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11360.184443 secs 2023-09-10T02:28:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11358.794759 secs 2023-09-10T02:28:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11360.184443 secs 2023-09-10T02:28:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11358.794759 secs 2023-09-10T02:28:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11360.184443 secs 2023-09-10T02:28:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:49 smithi082 ceph-mon[98694]: pgmap v6220: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T02:28:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11360.896078 secs 2023-09-10T02:28:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:49 smithi098 ceph-mon[103124]: pgmap v6220: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T02:28:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11360.896078 secs 2023-09-10T02:28:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:49 smithi170 ceph-mon[102832]: pgmap v6220: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T02:28:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11360.896078 secs 2023-09-10T02:28:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:51 smithi082 ceph-mon[98694]: pgmap v6221: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T02:28:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:51 smithi098 ceph-mon[103124]: pgmap v6221: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T02:28:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:51 smithi170 ceph-mon[102832]: pgmap v6221: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T02:28:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:53 smithi082 ceph-mon[98694]: pgmap v6222: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 172 KiB/s wr, 17 op/s 2023-09-10T02:28:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11363.794775 secs 2023-09-10T02:28:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11365.184500 secs 2023-09-10T02:28:53.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:53 smithi098 ceph-mon[103124]: pgmap v6222: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 172 KiB/s wr, 17 op/s 2023-09-10T02:28:53.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11363.794775 secs 2023-09-10T02:28:53.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11365.184500 secs 2023-09-10T02:28:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:53 smithi170 ceph-mon[102832]: pgmap v6222: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 172 KiB/s wr, 17 op/s 2023-09-10T02:28:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11363.794775 secs 2023-09-10T02:28:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11365.184500 secs 2023-09-10T02:28:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11365.896140 secs 2023-09-10T02:28:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11365.896140 secs 2023-09-10T02:28:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11365.896140 secs 2023-09-10T02:28:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:55 smithi082 ceph-mon[98694]: pgmap v6223: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T02:28:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:55 smithi098 ceph-mon[103124]: pgmap v6223: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T02:28:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:55 smithi170 ceph-mon[102832]: pgmap v6223: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T02:28:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:57 smithi082 ceph-mon[98694]: pgmap v6224: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 137 KiB/s wr, 16 op/s 2023-09-10T02:28:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:57 smithi098 ceph-mon[103124]: pgmap v6224: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 137 KiB/s wr, 16 op/s 2023-09-10T02:28:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:57 smithi170 ceph-mon[102832]: pgmap v6224: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 137 KiB/s wr, 16 op/s 2023-09-10T02:28:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11368.794835 secs 2023-09-10T02:28:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11370.184560 secs 2023-09-10T02:28:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11368.794835 secs 2023-09-10T02:28:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11370.184560 secs 2023-09-10T02:28:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11368.794835 secs 2023-09-10T02:28:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11370.184560 secs 2023-09-10T02:28:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:59 smithi082 ceph-mon[98694]: pgmap v6225: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T02:28:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:28:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11370.896210 secs 2023-09-10T02:28:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:59 smithi098 ceph-mon[103124]: pgmap v6225: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T02:28:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:28:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11370.896210 secs 2023-09-10T02:28:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:59 smithi170 ceph-mon[102832]: pgmap v6225: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T02:28:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:28:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11370.896210 secs 2023-09-10T02:29:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:01 smithi082 ceph-mon[98694]: pgmap v6226: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T02:29:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:01 smithi098 ceph-mon[103124]: pgmap v6226: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T02:29:01.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:01 smithi170 ceph-mon[102832]: pgmap v6226: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T02:29:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:03 smithi098 ceph-mon[103124]: pgmap v6227: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 108 KiB/s wr, 19 op/s 2023-09-10T02:29:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11373.794909 secs 2023-09-10T02:29:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11375.184616 secs 2023-09-10T02:29:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:03 smithi170 ceph-mon[102832]: pgmap v6227: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 108 KiB/s wr, 19 op/s 2023-09-10T02:29:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11373.794909 secs 2023-09-10T02:29:03.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11375.184616 secs 2023-09-10T02:29:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:03 smithi082 ceph-mon[98694]: pgmap v6227: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 108 KiB/s wr, 19 op/s 2023-09-10T02:29:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11373.794909 secs 2023-09-10T02:29:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11375.184616 secs 2023-09-10T02:29:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11375.896273 secs 2023-09-10T02:29:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11375.896273 secs 2023-09-10T02:29:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11375.896273 secs 2023-09-10T02:29:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:05 smithi098 ceph-mon[103124]: pgmap v6228: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 74 KiB/s wr, 14 op/s 2023-09-10T02:29:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:05 smithi170 ceph-mon[102832]: pgmap v6228: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 74 KiB/s wr, 14 op/s 2023-09-10T02:29:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:05 smithi082 ceph-mon[98694]: pgmap v6228: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 74 KiB/s wr, 14 op/s 2023-09-10T02:29:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:07 smithi098 ceph-mon[103124]: pgmap v6229: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 97 KiB/s wr, 28 op/s 2023-09-10T02:29:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:07 smithi170 ceph-mon[102832]: pgmap v6229: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 97 KiB/s wr, 28 op/s 2023-09-10T02:29:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:07 smithi082 ceph-mon[98694]: pgmap v6229: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 97 KiB/s wr, 28 op/s 2023-09-10T02:29:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11378.794970 secs 2023-09-10T02:29:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:08 smithi098 ceph-mon[103124]: osdmap e457: 12 total, 12 up, 12 in 2023-09-10T02:29:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11380.184674 secs 2023-09-10T02:29:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11378.794970 secs 2023-09-10T02:29:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:08 smithi170 ceph-mon[102832]: osdmap e457: 12 total, 12 up, 12 in 2023-09-10T02:29:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11380.184674 secs 2023-09-10T02:29:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11378.794970 secs 2023-09-10T02:29:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:08 smithi082 ceph-mon[98694]: osdmap e457: 12 total, 12 up, 12 in 2023-09-10T02:29:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11380.184674 secs 2023-09-10T02:29:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:09 smithi098 ceph-mon[103124]: pgmap v6230: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 60 KiB/s wr, 23 op/s 2023-09-10T02:29:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11380.896339 secs 2023-09-10T02:29:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:09 smithi170 ceph-mon[102832]: pgmap v6230: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 60 KiB/s wr, 23 op/s 2023-09-10T02:29:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11380.896339 secs 2023-09-10T02:29:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:09 smithi082 ceph-mon[98694]: pgmap v6230: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 60 KiB/s wr, 23 op/s 2023-09-10T02:29:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11380.896339 secs 2023-09-10T02:29:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:11 smithi098 ceph-mon[103124]: pgmap v6232: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 28 op/s 2023-09-10T02:29:11.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:11 smithi170 ceph-mon[102832]: pgmap v6232: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 28 op/s 2023-09-10T02:29:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:11 smithi082 ceph-mon[98694]: pgmap v6232: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 72 KiB/s wr, 28 op/s 2023-09-10T02:29:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:12 smithi170 ceph-mon[102832]: pgmap v6233: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 KiB/s wr, 22 op/s 2023-09-10T02:29:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11383.795032 secs 2023-09-10T02:29:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:12 smithi082 ceph-mon[98694]: pgmap v6233: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 KiB/s wr, 22 op/s 2023-09-10T02:29:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11383.795032 secs 2023-09-10T02:29:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:12 smithi098 ceph-mon[103124]: pgmap v6233: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 KiB/s wr, 22 op/s 2023-09-10T02:29:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11383.795032 secs 2023-09-10T02:29:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11385.184761 secs 2023-09-10T02:29:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11385.184761 secs 2023-09-10T02:29:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11385.184761 secs 2023-09-10T02:29:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11385.896421 secs 2023-09-10T02:29:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:14 smithi170 ceph-mon[102832]: pgmap v6234: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 KiB/s wr, 22 op/s 2023-09-10T02:29:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11385.896421 secs 2023-09-10T02:29:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:14 smithi082 ceph-mon[98694]: pgmap v6234: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 KiB/s wr, 22 op/s 2023-09-10T02:29:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11385.896421 secs 2023-09-10T02:29:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:14 smithi098 ceph-mon[103124]: pgmap v6234: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 KiB/s wr, 22 op/s 2023-09-10T02:29:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:16 smithi170 ceph-mon[102832]: pgmap v6235: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:29:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:16 smithi170 ceph-mon[102832]: osdmap e458: 12 total, 12 up, 12 in 2023-09-10T02:29:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:16 smithi082 ceph-mon[98694]: pgmap v6235: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:29:16.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:16 smithi082 ceph-mon[98694]: osdmap e458: 12 total, 12 up, 12 in 2023-09-10T02:29:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:16 smithi098 ceph-mon[103124]: pgmap v6235: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:29:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:16 smithi098 ceph-mon[103124]: osdmap e458: 12 total, 12 up, 12 in 2023-09-10T02:29:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11388.795093 secs 2023-09-10T02:29:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11388.795093 secs 2023-09-10T02:29:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11388.795093 secs 2023-09-10T02:29:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:18 smithi170 ceph-mon[102832]: pgmap v6237: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:29:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11390.184818 secs 2023-09-10T02:29:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:18 smithi082 ceph-mon[98694]: pgmap v6237: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:29:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11390.184818 secs 2023-09-10T02:29:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:18 smithi098 ceph-mon[103124]: pgmap v6237: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:29:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11390.184818 secs 2023-09-10T02:29:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11390.896486 secs 2023-09-10T02:29:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11390.896486 secs 2023-09-10T02:29:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11390.896486 secs 2023-09-10T02:29:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:20 smithi170 ceph-mon[102832]: pgmap v6238: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:29:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:20 smithi082 ceph-mon[98694]: pgmap v6238: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:29:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:20 smithi098 ceph-mon[103124]: pgmap v6238: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:29:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:22 smithi170 ceph-mon[102832]: pgmap v6239: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:29:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:22 smithi082 ceph-mon[98694]: pgmap v6239: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:29:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:22 smithi098 ceph-mon[103124]: pgmap v6239: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:29:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:29:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11395.184880 secs 2023-09-10T02:29:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:29:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11395.184880 secs 2023-09-10T02:29:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:29:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11395.184880 secs 2023-09-10T02:29:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11395.896549 secs 2023-09-10T02:29:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:24 smithi170 ceph-mon[102832]: pgmap v6240: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:29:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:29:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:29:24.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:29:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11395.896549 secs 2023-09-10T02:29:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:24 smithi082 ceph-mon[98694]: pgmap v6240: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:29:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:29:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:29:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:29:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11395.896549 secs 2023-09-10T02:29:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:24 smithi098 ceph-mon[103124]: pgmap v6240: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:29:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:29:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:29:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:29:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:26 smithi170 ceph-mon[102832]: pgmap v6241: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:29:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:26 smithi082 ceph-mon[98694]: pgmap v6241: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:29:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:26 smithi098 ceph-mon[103124]: pgmap v6241: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:29:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11398.795212 secs 2023-09-10T02:29:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11398.795212 secs 2023-09-10T02:29:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11398.795212 secs 2023-09-10T02:29:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:28 smithi170 ceph-mon[102832]: pgmap v6242: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 10 op/s 2023-09-10T02:29:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11400.184938 secs 2023-09-10T02:29:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:28 smithi082 ceph-mon[98694]: pgmap v6242: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 10 op/s 2023-09-10T02:29:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11400.184938 secs 2023-09-10T02:29:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:28 smithi098 ceph-mon[103124]: pgmap v6242: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 10 op/s 2023-09-10T02:29:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11400.184938 secs 2023-09-10T02:29:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11400.896617 secs 2023-09-10T02:29:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11400.896617 secs 2023-09-10T02:29:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11400.896617 secs 2023-09-10T02:29:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:30 smithi170 ceph-mon[102832]: pgmap v6243: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 9 op/s 2023-09-10T02:29:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:30 smithi082 ceph-mon[98694]: pgmap v6243: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 9 op/s 2023-09-10T02:29:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:30 smithi098 ceph-mon[103124]: pgmap v6243: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 9 op/s 2023-09-10T02:29:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:32 smithi170 ceph-mon[102832]: pgmap v6244: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 142 KiB/s wr, 15 op/s 2023-09-10T02:29:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:32 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:29:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:29:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:29:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:32 smithi082 ceph-mon[98694]: pgmap v6244: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 142 KiB/s wr, 15 op/s 2023-09-10T02:29:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:32 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:29:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:29:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:29:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:32 smithi098 ceph-mon[103124]: pgmap v6244: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 142 KiB/s wr, 15 op/s 2023-09-10T02:29:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:32 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:29:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:29:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:29:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11405.184995 secs 2023-09-10T02:29:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11405.184995 secs 2023-09-10T02:29:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11405.184995 secs 2023-09-10T02:29:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11405.896682 secs 2023-09-10T02:29:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:34 smithi170 ceph-mon[102832]: pgmap v6245: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T02:29:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11405.896682 secs 2023-09-10T02:29:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:34 smithi082 ceph-mon[98694]: pgmap v6245: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T02:29:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11405.896682 secs 2023-09-10T02:29:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:34 smithi098 ceph-mon[103124]: pgmap v6245: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T02:29:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:29:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:29:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:29:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:36 smithi170 ceph-mon[102832]: pgmap v6246: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 151 KiB/s wr, 17 op/s 2023-09-10T02:29:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:29:36.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:29:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:36 smithi082 ceph-mon[98694]: pgmap v6246: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 151 KiB/s wr, 17 op/s 2023-09-10T02:29:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:29:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:29:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:36 smithi098 ceph-mon[103124]: pgmap v6246: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 151 KiB/s wr, 17 op/s 2023-09-10T02:29:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:29:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:29:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:38 smithi170 ceph-mon[102832]: pgmap v6247: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T02:29:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11410.185080 secs 2023-09-10T02:29:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:38 smithi082 ceph-mon[98694]: pgmap v6247: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T02:29:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11410.185080 secs 2023-09-10T02:29:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:38 smithi098 ceph-mon[103124]: pgmap v6247: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T02:29:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11410.185080 secs 2023-09-10T02:29:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11410.896748 secs 2023-09-10T02:29:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11410.896748 secs 2023-09-10T02:29:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11410.896748 secs 2023-09-10T02:29:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:40 smithi170 ceph-mon[102832]: pgmap v6248: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T02:29:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:40 smithi082 ceph-mon[98694]: pgmap v6248: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T02:29:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:40 smithi098 ceph-mon[103124]: pgmap v6248: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T02:29:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:41 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:29:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:41 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:29:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:41 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:29:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:42 smithi082 ceph-mon[98694]: pgmap v6249: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 239 KiB/s wr, 18 op/s 2023-09-10T02:29:42.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:29:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:42 smithi098 ceph-mon[103124]: pgmap v6249: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 239 KiB/s wr, 18 op/s 2023-09-10T02:29:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:29:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:42 smithi170 ceph-mon[102832]: pgmap v6249: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 239 KiB/s wr, 18 op/s 2023-09-10T02:29:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:29:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:29:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:29:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:29:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11415.185140 secs 2023-09-10T02:29:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:29:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:29:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:29:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11415.185140 secs 2023-09-10T02:29:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:29:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:29:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:29:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11415.185140 secs 2023-09-10T02:29:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11415.896812 secs 2023-09-10T02:29:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:44 smithi170 ceph-mon[102832]: pgmap v6250: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 129 KiB/s wr, 12 op/s 2023-09-10T02:29:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11415.896812 secs 2023-09-10T02:29:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:44 smithi082 ceph-mon[98694]: pgmap v6250: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 129 KiB/s wr, 12 op/s 2023-09-10T02:29:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11415.896812 secs 2023-09-10T02:29:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:44 smithi098 ceph-mon[103124]: pgmap v6250: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 129 KiB/s wr, 12 op/s 2023-09-10T02:29:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:46 smithi170 ceph-mon[102832]: pgmap v6251: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 151 KiB/s wr, 17 op/s 2023-09-10T02:29:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:46 smithi082 ceph-mon[98694]: pgmap v6251: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 151 KiB/s wr, 17 op/s 2023-09-10T02:29:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:46 smithi098 ceph-mon[103124]: pgmap v6251: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 151 KiB/s wr, 17 op/s 2023-09-10T02:29:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11418.795465 secs 2023-09-10T02:29:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11418.795465 secs 2023-09-10T02:29:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11418.795465 secs 2023-09-10T02:29:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:48 smithi082 ceph-mon[98694]: pgmap v6252: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T02:29:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11420.185237 secs 2023-09-10T02:29:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:48 smithi098 ceph-mon[103124]: pgmap v6252: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T02:29:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11420.185237 secs 2023-09-10T02:29:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:48 smithi170 ceph-mon[102832]: pgmap v6252: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T02:29:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11420.185237 secs 2023-09-10T02:29:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11420.896880 secs 2023-09-10T02:29:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11420.896880 secs 2023-09-10T02:29:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11420.896880 secs 2023-09-10T02:29:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:50 smithi082 ceph-mon[98694]: pgmap v6253: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T02:29:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:50 smithi098 ceph-mon[103124]: pgmap v6253: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T02:29:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:50 smithi170 ceph-mon[102832]: pgmap v6253: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T02:29:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:29:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:29:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:29:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:29:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:29:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:29:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:29:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:29:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:29:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:52 smithi082 ceph-mon[98694]: pgmap v6254: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 250 KiB/s wr, 17 op/s 2023-09-10T02:29:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11423.795528 secs 2023-09-10T02:29:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:52 smithi098 ceph-mon[103124]: pgmap v6254: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 250 KiB/s wr, 17 op/s 2023-09-10T02:29:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11423.795528 secs 2023-09-10T02:29:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:52 smithi170 ceph-mon[102832]: pgmap v6254: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 250 KiB/s wr, 17 op/s 2023-09-10T02:29:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11423.795528 secs 2023-09-10T02:29:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11425.185320 secs 2023-09-10T02:29:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11425.185320 secs 2023-09-10T02:29:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11425.185320 secs 2023-09-10T02:29:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11425.896945 secs 2023-09-10T02:29:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:54 smithi082 ceph-mon[98694]: pgmap v6255: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 10 op/s 2023-09-10T02:29:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11425.896945 secs 2023-09-10T02:29:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:54 smithi098 ceph-mon[103124]: pgmap v6255: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 10 op/s 2023-09-10T02:29:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11425.896945 secs 2023-09-10T02:29:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:54 smithi170 ceph-mon[102832]: pgmap v6255: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 10 op/s 2023-09-10T02:29:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:56 smithi082 ceph-mon[98694]: pgmap v6256: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 197 KiB/s wr, 15 op/s 2023-09-10T02:29:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:56 smithi098 ceph-mon[103124]: pgmap v6256: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 197 KiB/s wr, 15 op/s 2023-09-10T02:29:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:56 smithi170 ceph-mon[102832]: pgmap v6256: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 197 KiB/s wr, 15 op/s 2023-09-10T02:29:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11428.795586 secs 2023-09-10T02:29:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11428.795586 secs 2023-09-10T02:29:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11428.795586 secs 2023-09-10T02:29:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:58 smithi098 ceph-mon[103124]: pgmap v6257: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T02:29:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11430.185378 secs 2023-09-10T02:29:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:58 smithi170 ceph-mon[102832]: pgmap v6257: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T02:29:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11430.185378 secs 2023-09-10T02:29:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:58 smithi082 ceph-mon[98694]: pgmap v6257: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T02:29:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11430.185378 secs 2023-09-10T02:30:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:29:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11430.897012 secs 2023-09-10T02:30:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:29:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11430.897012 secs 2023-09-10T02:30:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:29:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11430.897012 secs 2023-09-10T02:30:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:00 smithi098 ceph-mon[103124]: pgmap v6258: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T02:30:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T02:30:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:00 smithi170 ceph-mon[102832]: pgmap v6258: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T02:30:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T02:30:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:00 smithi082 ceph-mon[98694]: pgmap v6258: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T02:30:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T02:30:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:02 smithi170 ceph-mon[102832]: pgmap v6259: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 203 KiB/s wr, 15 op/s 2023-09-10T02:30:03.025 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11433.795648 secs 2023-09-10T02:30:03.026 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:02 smithi098 ceph-mon[103124]: pgmap v6259: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 203 KiB/s wr, 15 op/s 2023-09-10T02:30:03.026 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11433.795648 secs 2023-09-10T02:30:03.027 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:02 smithi082 ceph-mon[98694]: pgmap v6259: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 203 KiB/s wr, 15 op/s 2023-09-10T02:30:03.027 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11433.795648 secs 2023-09-10T02:30:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11435.185435 secs 2023-09-10T02:30:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11435.185435 secs 2023-09-10T02:30:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11435.185435 secs 2023-09-10T02:30:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11435.897079 secs 2023-09-10T02:30:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:04 smithi170 ceph-mon[102832]: pgmap v6260: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 95 KiB/s wr, 10 op/s 2023-09-10T02:30:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11435.897079 secs 2023-09-10T02:30:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:04 smithi082 ceph-mon[98694]: pgmap v6260: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 95 KiB/s wr, 10 op/s 2023-09-10T02:30:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11435.897079 secs 2023-09-10T02:30:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:04 smithi098 ceph-mon[103124]: pgmap v6260: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 95 KiB/s wr, 10 op/s 2023-09-10T02:30:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:06 smithi170 ceph-mon[102832]: pgmap v6261: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 149 KiB/s wr, 29 op/s 2023-09-10T02:30:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:06 smithi082 ceph-mon[98694]: pgmap v6261: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 149 KiB/s wr, 29 op/s 2023-09-10T02:30:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:06 smithi098 ceph-mon[103124]: pgmap v6261: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 149 KiB/s wr, 29 op/s 2023-09-10T02:30:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11438.795709 secs 2023-09-10T02:30:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11438.795709 secs 2023-09-10T02:30:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11438.795709 secs 2023-09-10T02:30:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:08 smithi082 ceph-mon[98694]: pgmap v6262: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 83 KiB/s wr, 24 op/s 2023-09-10T02:30:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:08 smithi082 ceph-mon[98694]: osdmap e459: 12 total, 12 up, 12 in 2023-09-10T02:30:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11440.185498 secs 2023-09-10T02:30:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:08 smithi098 ceph-mon[103124]: pgmap v6262: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 83 KiB/s wr, 24 op/s 2023-09-10T02:30:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:08 smithi098 ceph-mon[103124]: osdmap e459: 12 total, 12 up, 12 in 2023-09-10T02:30:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11440.185498 secs 2023-09-10T02:30:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:08 smithi170 ceph-mon[102832]: pgmap v6262: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 83 KiB/s wr, 24 op/s 2023-09-10T02:30:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:08 smithi170 ceph-mon[102832]: osdmap e459: 12 total, 12 up, 12 in 2023-09-10T02:30:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11440.185498 secs 2023-09-10T02:30:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11440.897162 secs 2023-09-10T02:30:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11440.897162 secs 2023-09-10T02:30:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11440.897162 secs 2023-09-10T02:30:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:10 smithi082 ceph-mon[98694]: pgmap v6264: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T02:30:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:10 smithi098 ceph-mon[103124]: pgmap v6264: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T02:30:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:10 smithi170 ceph-mon[102832]: pgmap v6264: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T02:30:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:12 smithi170 ceph-mon[102832]: pgmap v6265: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 66 KiB/s wr, 28 op/s 2023-09-10T02:30:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11443.795773 secs 2023-09-10T02:30:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:12 smithi082 ceph-mon[98694]: pgmap v6265: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 66 KiB/s wr, 28 op/s 2023-09-10T02:30:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11443.795773 secs 2023-09-10T02:30:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:12 smithi098 ceph-mon[103124]: pgmap v6265: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 66 KiB/s wr, 28 op/s 2023-09-10T02:30:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11443.795773 secs 2023-09-10T02:30:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11445.185581 secs 2023-09-10T02:30:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11445.185581 secs 2023-09-10T02:30:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11445.185581 secs 2023-09-10T02:30:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11445.897230 secs 2023-09-10T02:30:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:14 smithi170 ceph-mon[102832]: pgmap v6266: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 66 KiB/s wr, 28 op/s 2023-09-10T02:30:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11445.897230 secs 2023-09-10T02:30:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:14 smithi082 ceph-mon[98694]: pgmap v6266: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 66 KiB/s wr, 28 op/s 2023-09-10T02:30:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11445.897230 secs 2023-09-10T02:30:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:14 smithi098 ceph-mon[103124]: pgmap v6266: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 66 KiB/s wr, 28 op/s 2023-09-10T02:30:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:16 smithi170 ceph-mon[102832]: pgmap v6267: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 12 op/s 2023-09-10T02:30:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:16 smithi170 ceph-mon[102832]: osdmap e460: 12 total, 12 up, 12 in 2023-09-10T02:30:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:16 smithi082 ceph-mon[98694]: pgmap v6267: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 12 op/s 2023-09-10T02:30:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:16 smithi082 ceph-mon[98694]: osdmap e460: 12 total, 12 up, 12 in 2023-09-10T02:30:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:16 smithi098 ceph-mon[103124]: pgmap v6267: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 12 op/s 2023-09-10T02:30:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:16 smithi098 ceph-mon[103124]: osdmap e460: 12 total, 12 up, 12 in 2023-09-10T02:30:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11448.795835 secs 2023-09-10T02:30:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11448.795835 secs 2023-09-10T02:30:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11448.795835 secs 2023-09-10T02:30:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:18 smithi082 ceph-mon[98694]: pgmap v6269: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 12 op/s 2023-09-10T02:30:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11450.185639 secs 2023-09-10T02:30:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:18 smithi098 ceph-mon[103124]: pgmap v6269: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 12 op/s 2023-09-10T02:30:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11450.185639 secs 2023-09-10T02:30:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:18 smithi170 ceph-mon[102832]: pgmap v6269: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 12 op/s 2023-09-10T02:30:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11450.185639 secs 2023-09-10T02:30:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11450.897292 secs 2023-09-10T02:30:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11450.897292 secs 2023-09-10T02:30:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11450.897292 secs 2023-09-10T02:30:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:20 smithi082 ceph-mon[98694]: pgmap v6270: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 12 op/s 2023-09-10T02:30:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:20 smithi098 ceph-mon[103124]: pgmap v6270: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 12 op/s 2023-09-10T02:30:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:20 smithi170 ceph-mon[102832]: pgmap v6270: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 12 op/s 2023-09-10T02:30:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:22 smithi170 ceph-mon[102832]: pgmap v6271: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 12 op/s 2023-09-10T02:30:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:22 smithi082 ceph-mon[98694]: pgmap v6271: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 12 op/s 2023-09-10T02:30:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:22 smithi098 ceph-mon[103124]: pgmap v6271: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 12 op/s 2023-09-10T02:30:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:30:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11455.185696 secs 2023-09-10T02:30:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:30:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11455.185696 secs 2023-09-10T02:30:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:30:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11455.185696 secs 2023-09-10T02:30:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11455.897357 secs 2023-09-10T02:30:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:24 smithi170 ceph-mon[102832]: pgmap v6272: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 12 op/s 2023-09-10T02:30:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:30:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:30:24.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:30:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11455.897357 secs 2023-09-10T02:30:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:24 smithi082 ceph-mon[98694]: pgmap v6272: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 12 op/s 2023-09-10T02:30:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:30:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:30:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:30:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11455.897357 secs 2023-09-10T02:30:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:24 smithi098 ceph-mon[103124]: pgmap v6272: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 12 op/s 2023-09-10T02:30:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:30:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:30:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:30:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:26 smithi170 ceph-mon[102832]: pgmap v6273: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 13 op/s 2023-09-10T02:30:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:26 smithi082 ceph-mon[98694]: pgmap v6273: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 13 op/s 2023-09-10T02:30:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:26 smithi098 ceph-mon[103124]: pgmap v6273: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 13 op/s 2023-09-10T02:30:27.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11458.795992 secs 2023-09-10T02:30:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11458.795992 secs 2023-09-10T02:30:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11458.795992 secs 2023-09-10T02:30:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:28 smithi082 ceph-mon[98694]: pgmap v6274: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T02:30:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11460.185755 secs 2023-09-10T02:30:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:28 smithi098 ceph-mon[103124]: pgmap v6274: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T02:30:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11460.185755 secs 2023-09-10T02:30:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:28 smithi170 ceph-mon[102832]: pgmap v6274: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T02:30:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11460.185755 secs 2023-09-10T02:30:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11460.897423 secs 2023-09-10T02:30:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11460.897423 secs 2023-09-10T02:30:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11460.897423 secs 2023-09-10T02:30:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:30 smithi082 ceph-mon[98694]: pgmap v6275: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T02:30:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:30 smithi098 ceph-mon[103124]: pgmap v6275: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T02:30:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:30 smithi170 ceph-mon[102832]: pgmap v6275: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T02:30:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:32 smithi170 ceph-mon[102832]: pgmap v6276: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 162 KiB/s wr, 18 op/s 2023-09-10T02:30:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:32 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:30:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:30:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:30:32.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11463.796070 secs 2023-09-10T02:30:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:32 smithi082 ceph-mon[98694]: pgmap v6276: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 162 KiB/s wr, 18 op/s 2023-09-10T02:30:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:32 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:30:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:30:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:30:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11463.796070 secs 2023-09-10T02:30:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:32 smithi098 ceph-mon[103124]: pgmap v6276: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 162 KiB/s wr, 18 op/s 2023-09-10T02:30:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:32 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:30:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:30:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:30:33.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11463.796070 secs 2023-09-10T02:30:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11465.185815 secs 2023-09-10T02:30:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11465.185815 secs 2023-09-10T02:30:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11465.185815 secs 2023-09-10T02:30:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11465.897485 secs 2023-09-10T02:30:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:34 smithi082 ceph-mon[98694]: pgmap v6277: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 162 KiB/s wr, 13 op/s 2023-09-10T02:30:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11465.897485 secs 2023-09-10T02:30:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:34 smithi098 ceph-mon[103124]: pgmap v6277: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 162 KiB/s wr, 13 op/s 2023-09-10T02:30:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11465.897485 secs 2023-09-10T02:30:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:34 smithi170 ceph-mon[102832]: pgmap v6277: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 162 KiB/s wr, 13 op/s 2023-09-10T02:30:36.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:36 smithi082 ceph-mon[98694]: pgmap v6278: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 219 KiB/s wr, 19 op/s 2023-09-10T02:30:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:36 smithi098 ceph-mon[103124]: pgmap v6278: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 219 KiB/s wr, 19 op/s 2023-09-10T02:30:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:36 smithi170 ceph-mon[102832]: pgmap v6278: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 219 KiB/s wr, 19 op/s 2023-09-10T02:30:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:38 smithi082 ceph-mon[98694]: pgmap v6279: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 164 KiB/s wr, 13 op/s 2023-09-10T02:30:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11470.185904 secs 2023-09-10T02:30:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:38 smithi098 ceph-mon[103124]: pgmap v6279: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 164 KiB/s wr, 13 op/s 2023-09-10T02:30:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11470.185904 secs 2023-09-10T02:30:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:38 smithi170 ceph-mon[102832]: pgmap v6279: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 164 KiB/s wr, 13 op/s 2023-09-10T02:30:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11470.185904 secs 2023-09-10T02:30:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11470.897549 secs 2023-09-10T02:30:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:30:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11470.897549 secs 2023-09-10T02:30:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:30:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11470.897549 secs 2023-09-10T02:30:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:30:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:40 smithi082 ceph-mon[98694]: pgmap v6280: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 164 KiB/s wr, 13 op/s 2023-09-10T02:30:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:30:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:30:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:30:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:40 smithi098 ceph-mon[103124]: pgmap v6280: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 164 KiB/s wr, 13 op/s 2023-09-10T02:30:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:30:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:30:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:30:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:40 smithi170 ceph-mon[102832]: pgmap v6280: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 164 KiB/s wr, 13 op/s 2023-09-10T02:30:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:30:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:30:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:30:42.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:42 smithi170 ceph-mon[102832]: pgmap v6281: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 324 KiB/s wr, 19 op/s 2023-09-10T02:30:42.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11473.796233 secs 2023-09-10T02:30:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:42 smithi082 ceph-mon[98694]: pgmap v6281: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 324 KiB/s wr, 19 op/s 2023-09-10T02:30:42.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11473.796233 secs 2023-09-10T02:30:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:42 smithi098 ceph-mon[103124]: pgmap v6281: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 324 KiB/s wr, 19 op/s 2023-09-10T02:30:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11473.796233 secs 2023-09-10T02:30:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11475.185960 secs 2023-09-10T02:30:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:30:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:30:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:30:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11475.185960 secs 2023-09-10T02:30:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:30:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:30:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:30:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11475.185960 secs 2023-09-10T02:30:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:30:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:30:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:30:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11475.897612 secs 2023-09-10T02:30:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:44 smithi082 ceph-mon[98694]: pgmap v6282: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 218 KiB/s wr, 12 op/s 2023-09-10T02:30:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:30:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:30:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:30:44.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:30:44.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:30:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11475.897612 secs 2023-09-10T02:30:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:44 smithi098 ceph-mon[103124]: pgmap v6282: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 218 KiB/s wr, 12 op/s 2023-09-10T02:30:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:30:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:30:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:30:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:30:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:30:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11475.897612 secs 2023-09-10T02:30:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:44 smithi170 ceph-mon[102832]: pgmap v6282: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 218 KiB/s wr, 12 op/s 2023-09-10T02:30:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:30:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:30:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:30:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:30:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:30:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:46 smithi082 ceph-mon[98694]: pgmap v6283: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 250 KiB/s wr, 18 op/s 2023-09-10T02:30:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:46 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:30:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:46 smithi098 ceph-mon[103124]: pgmap v6283: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 250 KiB/s wr, 18 op/s 2023-09-10T02:30:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:46 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:30:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:46 smithi170 ceph-mon[102832]: pgmap v6283: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 250 KiB/s wr, 18 op/s 2023-09-10T02:30:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:46 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:30:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11478.796294 secs 2023-09-10T02:30:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11478.796294 secs 2023-09-10T02:30:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11478.796294 secs 2023-09-10T02:30:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:48 smithi082 ceph-mon[98694]: pgmap v6284: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 11 op/s 2023-09-10T02:30:48.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11480.186047 secs 2023-09-10T02:30:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:48 smithi098 ceph-mon[103124]: pgmap v6284: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 11 op/s 2023-09-10T02:30:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11480.186047 secs 2023-09-10T02:30:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:48 smithi170 ceph-mon[102832]: pgmap v6284: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 11 op/s 2023-09-10T02:30:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11480.186047 secs 2023-09-10T02:30:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11480.897677 secs 2023-09-10T02:30:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11480.897677 secs 2023-09-10T02:30:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11480.897677 secs 2023-09-10T02:30:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:50 smithi082 ceph-mon[98694]: pgmap v6285: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 11 op/s 2023-09-10T02:30:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:50 smithi098 ceph-mon[103124]: pgmap v6285: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 11 op/s 2023-09-10T02:30:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:50 smithi170 ceph-mon[102832]: pgmap v6285: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 11 op/s 2023-09-10T02:30:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:52 smithi082 ceph-mon[98694]: pgmap v6286: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 259 KiB/s wr, 17 op/s 2023-09-10T02:30:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11483.796354 secs 2023-09-10T02:30:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:52 smithi098 ceph-mon[103124]: pgmap v6286: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 259 KiB/s wr, 17 op/s 2023-09-10T02:30:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11483.796354 secs 2023-09-10T02:30:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:52 smithi170 ceph-mon[102832]: pgmap v6286: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 259 KiB/s wr, 17 op/s 2023-09-10T02:30:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11483.796354 secs 2023-09-10T02:30:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11485.186105 secs 2023-09-10T02:30:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11485.186105 secs 2023-09-10T02:30:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11485.186105 secs 2023-09-10T02:30:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11485.897743 secs 2023-09-10T02:30:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:54 smithi082 ceph-mon[98694]: pgmap v6287: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T02:30:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11485.897743 secs 2023-09-10T02:30:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:54 smithi098 ceph-mon[103124]: pgmap v6287: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T02:30:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11485.897743 secs 2023-09-10T02:30:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:54 smithi170 ceph-mon[102832]: pgmap v6287: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T02:30:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:56 smithi082 ceph-mon[98694]: pgmap v6288: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 99 KiB/s wr, 16 op/s 2023-09-10T02:30:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:56 smithi098 ceph-mon[103124]: pgmap v6288: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 99 KiB/s wr, 16 op/s 2023-09-10T02:30:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:56 smithi170 ceph-mon[102832]: pgmap v6288: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 99 KiB/s wr, 16 op/s 2023-09-10T02:30:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11488.796413 secs 2023-09-10T02:30:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11488.796413 secs 2023-09-10T02:30:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11488.796413 secs 2023-09-10T02:30:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:58 smithi082 ceph-mon[98694]: pgmap v6289: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 68 KiB/s wr, 10 op/s 2023-09-10T02:30:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11490.186185 secs 2023-09-10T02:30:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:58 smithi098 ceph-mon[103124]: pgmap v6289: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 68 KiB/s wr, 10 op/s 2023-09-10T02:30:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11490.186185 secs 2023-09-10T02:30:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:58 smithi170 ceph-mon[102832]: pgmap v6289: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 68 KiB/s wr, 10 op/s 2023-09-10T02:30:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11490.186185 secs 2023-09-10T02:30:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:30:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11490.897807 secs 2023-09-10T02:31:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:30:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11490.897807 secs 2023-09-10T02:31:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:30:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11490.897807 secs 2023-09-10T02:31:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:00 smithi082 ceph-mon[98694]: pgmap v6290: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 68 KiB/s wr, 10 op/s 2023-09-10T02:31:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:00 smithi098 ceph-mon[103124]: pgmap v6290: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 68 KiB/s wr, 10 op/s 2023-09-10T02:31:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:00 smithi170 ceph-mon[102832]: pgmap v6290: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 68 KiB/s wr, 10 op/s 2023-09-10T02:31:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:02 smithi082 ceph-mon[98694]: pgmap v6291: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 15 op/s 2023-09-10T02:31:02.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11493.796475 secs 2023-09-10T02:31:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:02 smithi098 ceph-mon[103124]: pgmap v6291: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 15 op/s 2023-09-10T02:31:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11493.796475 secs 2023-09-10T02:31:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:02 smithi170 ceph-mon[102832]: pgmap v6291: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 15 op/s 2023-09-10T02:31:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11493.796475 secs 2023-09-10T02:31:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11495.186243 secs 2023-09-10T02:31:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11495.186243 secs 2023-09-10T02:31:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11495.186243 secs 2023-09-10T02:31:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11495.897871 secs 2023-09-10T02:31:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:04 smithi082 ceph-mon[98694]: pgmap v6292: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 10 op/s 2023-09-10T02:31:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11495.897871 secs 2023-09-10T02:31:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:04 smithi098 ceph-mon[103124]: pgmap v6292: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 10 op/s 2023-09-10T02:31:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11495.897871 secs 2023-09-10T02:31:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:04 smithi170 ceph-mon[102832]: pgmap v6292: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 10 op/s 2023-09-10T02:31:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:06 smithi082 ceph-mon[98694]: pgmap v6293: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T02:31:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:06 smithi098 ceph-mon[103124]: pgmap v6293: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T02:31:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:06 smithi170 ceph-mon[102832]: pgmap v6293: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 84 KiB/s wr, 29 op/s 2023-09-10T02:31:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11498.796533 secs 2023-09-10T02:31:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11498.796533 secs 2023-09-10T02:31:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11498.796533 secs 2023-09-10T02:31:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:08 smithi082 ceph-mon[98694]: pgmap v6294: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 84 KiB/s wr, 25 op/s 2023-09-10T02:31:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:08 smithi082 ceph-mon[98694]: osdmap e461: 12 total, 12 up, 12 in 2023-09-10T02:31:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11500.186303 secs 2023-09-10T02:31:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:08 smithi098 ceph-mon[103124]: pgmap v6294: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 84 KiB/s wr, 25 op/s 2023-09-10T02:31:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:08 smithi098 ceph-mon[103124]: osdmap e461: 12 total, 12 up, 12 in 2023-09-10T02:31:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11500.186303 secs 2023-09-10T02:31:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:08 smithi170 ceph-mon[102832]: pgmap v6294: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 84 KiB/s wr, 25 op/s 2023-09-10T02:31:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:08 smithi170 ceph-mon[102832]: osdmap e461: 12 total, 12 up, 12 in 2023-09-10T02:31:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11500.186303 secs 2023-09-10T02:31:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11500.897960 secs 2023-09-10T02:31:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11500.897960 secs 2023-09-10T02:31:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11500.897960 secs 2023-09-10T02:31:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:10 smithi082 ceph-mon[98694]: pgmap v6296: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 101 KiB/s wr, 30 op/s 2023-09-10T02:31:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:10 smithi098 ceph-mon[103124]: pgmap v6296: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 101 KiB/s wr, 30 op/s 2023-09-10T02:31:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:10 smithi170 ceph-mon[102832]: pgmap v6296: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 101 KiB/s wr, 30 op/s 2023-09-10T02:31:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:12 smithi082 ceph-mon[98694]: pgmap v6297: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 103 KiB/s wr, 29 op/s 2023-09-10T02:31:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11503.796631 secs 2023-09-10T02:31:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:12 smithi098 ceph-mon[103124]: pgmap v6297: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 103 KiB/s wr, 29 op/s 2023-09-10T02:31:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11503.796631 secs 2023-09-10T02:31:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:12 smithi170 ceph-mon[102832]: pgmap v6297: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 103 KiB/s wr, 29 op/s 2023-09-10T02:31:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11503.796631 secs 2023-09-10T02:31:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11505.186386 secs 2023-09-10T02:31:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11505.186386 secs 2023-09-10T02:31:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11505.186386 secs 2023-09-10T02:31:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11505.898025 secs 2023-09-10T02:31:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:14 smithi082 ceph-mon[98694]: pgmap v6298: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 103 KiB/s wr, 29 op/s 2023-09-10T02:31:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11505.898025 secs 2023-09-10T02:31:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:14 smithi098 ceph-mon[103124]: pgmap v6298: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 103 KiB/s wr, 29 op/s 2023-09-10T02:31:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11505.898025 secs 2023-09-10T02:31:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:14 smithi170 ceph-mon[102832]: pgmap v6298: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 103 KiB/s wr, 29 op/s 2023-09-10T02:31:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:16 smithi082 ceph-mon[98694]: pgmap v6299: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T02:31:16.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:16 smithi082 ceph-mon[98694]: osdmap e462: 12 total, 12 up, 12 in 2023-09-10T02:31:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:16 smithi098 ceph-mon[103124]: pgmap v6299: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T02:31:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:16 smithi098 ceph-mon[103124]: osdmap e462: 12 total, 12 up, 12 in 2023-09-10T02:31:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:16 smithi170 ceph-mon[102832]: pgmap v6299: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T02:31:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:16 smithi170 ceph-mon[102832]: osdmap e462: 12 total, 12 up, 12 in 2023-09-10T02:31:17.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11508.796691 secs 2023-09-10T02:31:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11508.796691 secs 2023-09-10T02:31:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11508.796691 secs 2023-09-10T02:31:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:18 smithi082 ceph-mon[98694]: pgmap v6301: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T02:31:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11510.186470 secs 2023-09-10T02:31:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:18 smithi098 ceph-mon[103124]: pgmap v6301: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T02:31:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11510.186470 secs 2023-09-10T02:31:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:18 smithi170 ceph-mon[102832]: pgmap v6301: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T02:31:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11510.186470 secs 2023-09-10T02:31:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11510.898088 secs 2023-09-10T02:31:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11510.898088 secs 2023-09-10T02:31:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11510.898088 secs 2023-09-10T02:31:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:20 smithi082 ceph-mon[98694]: pgmap v6302: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T02:31:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:20 smithi098 ceph-mon[103124]: pgmap v6302: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T02:31:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:20 smithi170 ceph-mon[102832]: pgmap v6302: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 11 op/s 2023-09-10T02:31:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:22 smithi082 ceph-mon[98694]: pgmap v6303: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:31:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:22 smithi098 ceph-mon[103124]: pgmap v6303: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:31:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:22 smithi170 ceph-mon[102832]: pgmap v6303: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:31:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:31:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11515.186526 secs 2023-09-10T02:31:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:31:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11515.186526 secs 2023-09-10T02:31:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:31:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11515.186526 secs 2023-09-10T02:31:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11515.898155 secs 2023-09-10T02:31:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:24 smithi082 ceph-mon[98694]: pgmap v6304: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:31:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:31:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:31:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:31:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11515.898155 secs 2023-09-10T02:31:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:24 smithi098 ceph-mon[103124]: pgmap v6304: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:31:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:31:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:31:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:31:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11515.898155 secs 2023-09-10T02:31:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:24 smithi170 ceph-mon[102832]: pgmap v6304: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:31:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:31:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:31:25.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:31:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:26 smithi082 ceph-mon[98694]: pgmap v6305: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T02:31:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:26 smithi098 ceph-mon[103124]: pgmap v6305: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T02:31:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:26 smithi170 ceph-mon[102832]: pgmap v6305: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T02:31:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:28 smithi082 ceph-mon[98694]: pgmap v6306: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 10 op/s 2023-09-10T02:31:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:28 smithi082 ceph-mon[98694]: 3 slow requests, 3 included below; oldest blocked for > 11520.186602 secs 2023-09-10T02:31:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:28 smithi082 ceph-mon[98694]: slow request 11520.186602 seconds old, received at 2023-09-09T23:19:27.740042+0000: client_request(mds.4:47 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently acquired locks 2023-09-10T02:31:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:28 smithi082 ceph-mon[98694]: slow request 11520.186222 seconds old, received at 2023-09-09T23:19:27.740422+0000: client_request(mds.4:48 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently requesting remote authpins 2023-09-10T02:31:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:28 smithi082 ceph-mon[98694]: slow request 11520.184001 seconds old, received at 2023-09-09T23:19:27.742643+0000: client_request(client.24913:3869 unlink #0x10000000007/00000001000000010000000B 2023-09-09T23:19:27.741863+0000 caller_uid=26, caller_gid=26{26,}) currently requesting remote authpins 2023-09-10T02:31:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:28 smithi098 ceph-mon[103124]: pgmap v6306: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 10 op/s 2023-09-10T02:31:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:28 smithi098 ceph-mon[103124]: 3 slow requests, 3 included below; oldest blocked for > 11520.186602 secs 2023-09-10T02:31:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:28 smithi098 ceph-mon[103124]: slow request 11520.186602 seconds old, received at 2023-09-09T23:19:27.740042+0000: client_request(mds.4:47 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently acquired locks 2023-09-10T02:31:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:28 smithi098 ceph-mon[103124]: slow request 11520.186222 seconds old, received at 2023-09-09T23:19:27.740422+0000: client_request(mds.4:48 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently requesting remote authpins 2023-09-10T02:31:29.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:28 smithi098 ceph-mon[103124]: slow request 11520.184001 seconds old, received at 2023-09-09T23:19:27.742643+0000: client_request(client.24913:3869 unlink #0x10000000007/00000001000000010000000B 2023-09-09T23:19:27.741863+0000 caller_uid=26, caller_gid=26{26,}) currently requesting remote authpins 2023-09-10T02:31:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:28 smithi170 ceph-mon[102832]: pgmap v6306: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 10 op/s 2023-09-10T02:31:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:28 smithi170 ceph-mon[102832]: 3 slow requests, 3 included below; oldest blocked for > 11520.186602 secs 2023-09-10T02:31:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:28 smithi170 ceph-mon[102832]: slow request 11520.186602 seconds old, received at 2023-09-09T23:19:27.740042+0000: client_request(mds.4:47 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently acquired locks 2023-09-10T02:31:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:28 smithi170 ceph-mon[102832]: slow request 11520.186222 seconds old, received at 2023-09-09T23:19:27.740422+0000: client_request(mds.4:48 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently requesting remote authpins 2023-09-10T02:31:29.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:28 smithi170 ceph-mon[102832]: slow request 11520.184001 seconds old, received at 2023-09-09T23:19:27.742643+0000: client_request(client.24913:3869 unlink #0x10000000007/00000001000000010000000B 2023-09-09T23:19:27.741863+0000 caller_uid=26, caller_gid=26{26,}) currently requesting remote authpins 2023-09-10T02:31:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:29 smithi098 ceph-mon[103124]: 3 slow requests, 3 included below; oldest blocked for > 11520.898244 secs 2023-09-10T02:31:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:29 smithi098 ceph-mon[103124]: slow request 11520.898244 seconds old, received at 2023-09-09T23:19:27.740800+0000: peer_request:mds.4:47 currently acquired locks 2023-09-10T02:31:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:29 smithi098 ceph-mon[103124]: slow request 11520.898010 seconds old, received at 2023-09-09T23:19:27.741034+0000: peer_request(mds.4:48.0 authpin) currently dispatched 2023-09-10T02:31:30.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:29 smithi098 ceph-mon[103124]: slow request 11520.895974 seconds old, received at 2023-09-09T23:19:27.743070+0000: peer_request(client.24913:3869.0 authpin) currently dispatched 2023-09-10T02:31:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:29 smithi170 ceph-mon[102832]: 3 slow requests, 3 included below; oldest blocked for > 11520.898244 secs 2023-09-10T02:31:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:29 smithi170 ceph-mon[102832]: slow request 11520.898244 seconds old, received at 2023-09-09T23:19:27.740800+0000: peer_request:mds.4:47 currently acquired locks 2023-09-10T02:31:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:29 smithi170 ceph-mon[102832]: slow request 11520.898010 seconds old, received at 2023-09-09T23:19:27.741034+0000: peer_request(mds.4:48.0 authpin) currently dispatched 2023-09-10T02:31:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:29 smithi170 ceph-mon[102832]: slow request 11520.895974 seconds old, received at 2023-09-09T23:19:27.743070+0000: peer_request(client.24913:3869.0 authpin) currently dispatched 2023-09-10T02:31:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:29 smithi082 ceph-mon[98694]: 3 slow requests, 3 included below; oldest blocked for > 11520.898244 secs 2023-09-10T02:31:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:29 smithi082 ceph-mon[98694]: slow request 11520.898244 seconds old, received at 2023-09-09T23:19:27.740800+0000: peer_request:mds.4:47 currently acquired locks 2023-09-10T02:31:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:29 smithi082 ceph-mon[98694]: slow request 11520.898010 seconds old, received at 2023-09-09T23:19:27.741034+0000: peer_request(mds.4:48.0 authpin) currently dispatched 2023-09-10T02:31:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:29 smithi082 ceph-mon[98694]: slow request 11520.895974 seconds old, received at 2023-09-09T23:19:27.743070+0000: peer_request(client.24913:3869.0 authpin) currently dispatched 2023-09-10T02:31:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:30 smithi098 ceph-mon[103124]: pgmap v6307: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 9 op/s 2023-09-10T02:31:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:30 smithi170 ceph-mon[102832]: pgmap v6307: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 9 op/s 2023-09-10T02:31:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:30 smithi082 ceph-mon[98694]: pgmap v6307: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 9 op/s 2023-09-10T02:31:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:32 smithi082 ceph-mon[98694]: pgmap v6308: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 128 KiB/s wr, 14 op/s 2023-09-10T02:31:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:32 smithi098 ceph-mon[103124]: pgmap v6308: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 128 KiB/s wr, 14 op/s 2023-09-10T02:31:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:32 smithi170 ceph-mon[102832]: pgmap v6308: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 128 KiB/s wr, 14 op/s 2023-09-10T02:31:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11525.186659 secs 2023-09-10T02:31:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11525.186659 secs 2023-09-10T02:31:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11525.186659 secs 2023-09-10T02:31:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11525.898310 secs 2023-09-10T02:31:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:34 smithi082 ceph-mon[98694]: pgmap v6309: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T02:31:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11525.898310 secs 2023-09-10T02:31:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:34 smithi098 ceph-mon[103124]: pgmap v6309: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T02:31:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11525.898310 secs 2023-09-10T02:31:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:34 smithi170 ceph-mon[102832]: pgmap v6309: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T02:31:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:36 smithi082 ceph-mon[98694]: pgmap v6310: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T02:31:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:36 smithi098 ceph-mon[103124]: pgmap v6310: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T02:31:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:36 smithi170 ceph-mon[102832]: pgmap v6310: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T02:31:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:37 smithi082 ceph-mon[98694]: 1 slow requests, 1 included below; oldest blocked for > 11528.797048 secs 2023-09-10T02:31:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:37 smithi082 ceph-mon[98694]: slow request 11528.797047 seconds old, received at 2023-09-09T23:19:28.303507+0000: peer_request:mds.4:47 currently dispatched 2023-09-10T02:31:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:37 smithi098 ceph-mon[103124]: 1 slow requests, 1 included below; oldest blocked for > 11528.797048 secs 2023-09-10T02:31:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:37 smithi098 ceph-mon[103124]: slow request 11528.797047 seconds old, received at 2023-09-09T23:19:28.303507+0000: peer_request:mds.4:47 currently dispatched 2023-09-10T02:31:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:37 smithi170 ceph-mon[102832]: 1 slow requests, 1 included below; oldest blocked for > 11528.797048 secs 2023-09-10T02:31:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:37 smithi170 ceph-mon[102832]: slow request 11528.797047 seconds old, received at 2023-09-09T23:19:28.303507+0000: peer_request:mds.4:47 currently dispatched 2023-09-10T02:31:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:38 smithi082 ceph-mon[98694]: pgmap v6311: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T02:31:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11530.186720 secs 2023-09-10T02:31:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:38 smithi098 ceph-mon[103124]: pgmap v6311: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T02:31:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11530.186720 secs 2023-09-10T02:31:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:38 smithi170 ceph-mon[102832]: pgmap v6311: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T02:31:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11530.186720 secs 2023-09-10T02:31:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11530.898378 secs 2023-09-10T02:31:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11530.898378 secs 2023-09-10T02:31:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11530.898378 secs 2023-09-10T02:31:41.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:40 smithi098 ceph-mon[103124]: pgmap v6312: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T02:31:41.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:31:41.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:31:41.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:31:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:40 smithi170 ceph-mon[102832]: pgmap v6312: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T02:31:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:31:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:31:41.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:31:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:40 smithi082 ceph-mon[98694]: pgmap v6312: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T02:31:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:31:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:31:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:31:42.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:42 smithi082 ceph-mon[98694]: pgmap v6313: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 203 KiB/s wr, 18 op/s 2023-09-10T02:31:42.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11533.797135 secs 2023-09-10T02:31:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:42 smithi098 ceph-mon[103124]: pgmap v6313: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 203 KiB/s wr, 18 op/s 2023-09-10T02:31:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11533.797135 secs 2023-09-10T02:31:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:42 smithi170 ceph-mon[102832]: pgmap v6313: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 203 KiB/s wr, 18 op/s 2023-09-10T02:31:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11533.797135 secs 2023-09-10T02:31:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11535.186780 secs 2023-09-10T02:31:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11535.186780 secs 2023-09-10T02:31:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11535.186780 secs 2023-09-10T02:31:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11535.898442 secs 2023-09-10T02:31:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:44 smithi082 ceph-mon[98694]: pgmap v6314: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T02:31:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:31:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:31:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:31:44.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:31:44.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:31:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11535.898442 secs 2023-09-10T02:31:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:44 smithi098 ceph-mon[103124]: pgmap v6314: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T02:31:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:31:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:31:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:31:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:31:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:31:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11535.898442 secs 2023-09-10T02:31:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:44 smithi170 ceph-mon[102832]: pgmap v6314: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T02:31:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:31:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:31:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:31:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:31:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:31:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:31:45.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:31:45.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:31:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:31:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:31:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:31:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:31:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:31:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:31:47.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:47 smithi170 ceph-mon[102832]: pgmap v6315: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 173 KiB/s wr, 18 op/s 2023-09-10T02:31:47.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:47 smithi082 ceph-mon[98694]: pgmap v6315: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 173 KiB/s wr, 18 op/s 2023-09-10T02:31:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:47 smithi098 ceph-mon[103124]: pgmap v6315: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 173 KiB/s wr, 18 op/s 2023-09-10T02:31:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11538.797195 secs 2023-09-10T02:31:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11540.186840 secs 2023-09-10T02:31:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11538.797195 secs 2023-09-10T02:31:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11540.186840 secs 2023-09-10T02:31:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11538.797195 secs 2023-09-10T02:31:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11540.186840 secs 2023-09-10T02:31:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:49 smithi170 ceph-mon[102832]: pgmap v6316: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T02:31:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11540.898508 secs 2023-09-10T02:31:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:49 smithi082 ceph-mon[98694]: pgmap v6316: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T02:31:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11540.898508 secs 2023-09-10T02:31:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:49 smithi098 ceph-mon[103124]: pgmap v6316: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T02:31:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11540.898508 secs 2023-09-10T02:31:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:51 smithi170 ceph-mon[102832]: pgmap v6317: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T02:31:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:51 smithi082 ceph-mon[98694]: pgmap v6317: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T02:31:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:51 smithi098 ceph-mon[103124]: pgmap v6317: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T02:31:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:52 smithi082 ceph-mon[98694]: pgmap v6318: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 252 KiB/s wr, 18 op/s 2023-09-10T02:31:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11543.797273 secs 2023-09-10T02:31:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:52 smithi098 ceph-mon[103124]: pgmap v6318: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 252 KiB/s wr, 18 op/s 2023-09-10T02:31:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11543.797273 secs 2023-09-10T02:31:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:52 smithi170 ceph-mon[102832]: pgmap v6318: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 252 KiB/s wr, 18 op/s 2023-09-10T02:31:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11543.797273 secs 2023-09-10T02:31:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11545.186898 secs 2023-09-10T02:31:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11545.186898 secs 2023-09-10T02:31:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11545.186898 secs 2023-09-10T02:31:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11545.898574 secs 2023-09-10T02:31:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:54 smithi082 ceph-mon[98694]: pgmap v6319: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T02:31:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11545.898574 secs 2023-09-10T02:31:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:54 smithi098 ceph-mon[103124]: pgmap v6319: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T02:31:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11545.898574 secs 2023-09-10T02:31:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:54 smithi170 ceph-mon[102832]: pgmap v6319: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T02:31:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:56 smithi082 ceph-mon[98694]: pgmap v6320: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 219 KiB/s wr, 17 op/s 2023-09-10T02:31:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:56 smithi098 ceph-mon[103124]: pgmap v6320: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 219 KiB/s wr, 17 op/s 2023-09-10T02:31:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:56 smithi170 ceph-mon[102832]: pgmap v6320: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 219 KiB/s wr, 17 op/s 2023-09-10T02:31:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11548.797331 secs 2023-09-10T02:31:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11548.797331 secs 2023-09-10T02:31:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11548.797331 secs 2023-09-10T02:31:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:58 smithi098 ceph-mon[103124]: pgmap v6321: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 184 KiB/s wr, 11 op/s 2023-09-10T02:31:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11550.186957 secs 2023-09-10T02:31:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:58 smithi170 ceph-mon[102832]: pgmap v6321: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 184 KiB/s wr, 11 op/s 2023-09-10T02:31:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11550.186957 secs 2023-09-10T02:31:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:58 smithi082 ceph-mon[98694]: pgmap v6321: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 184 KiB/s wr, 11 op/s 2023-09-10T02:31:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11550.186957 secs 2023-09-10T02:32:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:31:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11550.898640 secs 2023-09-10T02:32:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:31:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11550.898640 secs 2023-09-10T02:32:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:31:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11550.898640 secs 2023-09-10T02:32:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:00 smithi098 ceph-mon[103124]: pgmap v6322: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 184 KiB/s wr, 11 op/s 2023-09-10T02:32:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:00 smithi170 ceph-mon[102832]: pgmap v6322: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 184 KiB/s wr, 11 op/s 2023-09-10T02:32:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:00 smithi082 ceph-mon[98694]: pgmap v6322: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 184 KiB/s wr, 11 op/s 2023-09-10T02:32:02.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:02 smithi082 ceph-mon[98694]: pgmap v6323: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 260 KiB/s wr, 16 op/s 2023-09-10T02:32:02.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11553.797396 secs 2023-09-10T02:32:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:02 smithi098 ceph-mon[103124]: pgmap v6323: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 260 KiB/s wr, 16 op/s 2023-09-10T02:32:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11553.797396 secs 2023-09-10T02:32:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:02 smithi170 ceph-mon[102832]: pgmap v6323: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 260 KiB/s wr, 16 op/s 2023-09-10T02:32:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11553.797396 secs 2023-09-10T02:32:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11555.187014 secs 2023-09-10T02:32:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11555.187014 secs 2023-09-10T02:32:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11555.187014 secs 2023-09-10T02:32:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11555.898705 secs 2023-09-10T02:32:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:04 smithi098 ceph-mon[103124]: pgmap v6324: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 130 KiB/s wr, 10 op/s 2023-09-10T02:32:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11555.898705 secs 2023-09-10T02:32:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:04 smithi170 ceph-mon[102832]: pgmap v6324: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 130 KiB/s wr, 10 op/s 2023-09-10T02:32:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11555.898705 secs 2023-09-10T02:32:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:04 smithi082 ceph-mon[98694]: pgmap v6324: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 130 KiB/s wr, 10 op/s 2023-09-10T02:32:07.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:06 smithi098 ceph-mon[103124]: pgmap v6325: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 196 KiB/s wr, 29 op/s 2023-09-10T02:32:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:06 smithi170 ceph-mon[102832]: pgmap v6325: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 196 KiB/s wr, 29 op/s 2023-09-10T02:32:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:06 smithi082 ceph-mon[98694]: pgmap v6325: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 196 KiB/s wr, 29 op/s 2023-09-10T02:32:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11558.797459 secs 2023-09-10T02:32:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11558.797459 secs 2023-09-10T02:32:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11558.797459 secs 2023-09-10T02:32:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:08 smithi098 ceph-mon[103124]: pgmap v6326: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 142 KiB/s wr, 24 op/s 2023-09-10T02:32:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:08 smithi098 ceph-mon[103124]: osdmap e463: 12 total, 12 up, 12 in 2023-09-10T02:32:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11560.187073 secs 2023-09-10T02:32:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:08 smithi170 ceph-mon[102832]: pgmap v6326: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 142 KiB/s wr, 24 op/s 2023-09-10T02:32:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:08 smithi170 ceph-mon[102832]: osdmap e463: 12 total, 12 up, 12 in 2023-09-10T02:32:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11560.187073 secs 2023-09-10T02:32:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:08 smithi082 ceph-mon[98694]: pgmap v6326: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 142 KiB/s wr, 24 op/s 2023-09-10T02:32:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:08 smithi082 ceph-mon[98694]: osdmap e463: 12 total, 12 up, 12 in 2023-09-10T02:32:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11560.187073 secs 2023-09-10T02:32:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11560.898769 secs 2023-09-10T02:32:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11560.898769 secs 2023-09-10T02:32:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11560.898769 secs 2023-09-10T02:32:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:10 smithi098 ceph-mon[103124]: pgmap v6328: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 170 KiB/s wr, 29 op/s 2023-09-10T02:32:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:10 smithi170 ceph-mon[102832]: pgmap v6328: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 170 KiB/s wr, 29 op/s 2023-09-10T02:32:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:10 smithi082 ceph-mon[98694]: pgmap v6328: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 170 KiB/s wr, 29 op/s 2023-09-10T02:32:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:12 smithi082 ceph-mon[98694]: pgmap v6329: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T02:32:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11563.797531 secs 2023-09-10T02:32:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:12 smithi098 ceph-mon[103124]: pgmap v6329: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T02:32:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11563.797531 secs 2023-09-10T02:32:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:12 smithi170 ceph-mon[102832]: pgmap v6329: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T02:32:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11563.797531 secs 2023-09-10T02:32:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11565.187130 secs 2023-09-10T02:32:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11565.187130 secs 2023-09-10T02:32:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11565.187130 secs 2023-09-10T02:32:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11565.898834 secs 2023-09-10T02:32:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:14 smithi082 ceph-mon[98694]: pgmap v6330: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T02:32:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11565.898834 secs 2023-09-10T02:32:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:14 smithi098 ceph-mon[103124]: pgmap v6330: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T02:32:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11565.898834 secs 2023-09-10T02:32:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:14 smithi170 ceph-mon[102832]: pgmap v6330: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T02:32:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:16 smithi098 ceph-mon[103124]: pgmap v6331: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-09-10T02:32:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:16 smithi098 ceph-mon[103124]: osdmap e464: 12 total, 12 up, 12 in 2023-09-10T02:32:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:16 smithi170 ceph-mon[102832]: pgmap v6331: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-09-10T02:32:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:16 smithi170 ceph-mon[102832]: osdmap e464: 12 total, 12 up, 12 in 2023-09-10T02:32:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:16 smithi082 ceph-mon[98694]: pgmap v6331: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-09-10T02:32:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:16 smithi082 ceph-mon[98694]: osdmap e464: 12 total, 12 up, 12 in 2023-09-10T02:32:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11568.797593 secs 2023-09-10T02:32:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11568.797593 secs 2023-09-10T02:32:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11568.797593 secs 2023-09-10T02:32:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:18 smithi098 ceph-mon[103124]: pgmap v6333: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-09-10T02:32:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11570.187190 secs 2023-09-10T02:32:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:18 smithi170 ceph-mon[102832]: pgmap v6333: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-09-10T02:32:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11570.187190 secs 2023-09-10T02:32:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:18 smithi082 ceph-mon[98694]: pgmap v6333: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-09-10T02:32:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11570.187190 secs 2023-09-10T02:32:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11570.898900 secs 2023-09-10T02:32:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11570.898900 secs 2023-09-10T02:32:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11570.898900 secs 2023-09-10T02:32:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:20 smithi098 ceph-mon[103124]: pgmap v6334: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-09-10T02:32:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:20 smithi170 ceph-mon[102832]: pgmap v6334: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-09-10T02:32:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:20 smithi082 ceph-mon[98694]: pgmap v6334: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-09-10T02:32:22.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:22 smithi082 ceph-mon[98694]: pgmap v6335: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T02:32:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:22 smithi098 ceph-mon[103124]: pgmap v6335: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T02:32:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:22 smithi170 ceph-mon[102832]: pgmap v6335: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T02:32:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:32:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11575.187265 secs 2023-09-10T02:32:24.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:32:24.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11575.187265 secs 2023-09-10T02:32:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:32:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11575.187265 secs 2023-09-10T02:32:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11575.898967 secs 2023-09-10T02:32:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:24 smithi082 ceph-mon[98694]: pgmap v6336: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T02:32:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:32:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:32:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:32:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11575.898967 secs 2023-09-10T02:32:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:24 smithi098 ceph-mon[103124]: pgmap v6336: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T02:32:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:32:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:32:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:32:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11575.898967 secs 2023-09-10T02:32:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:24 smithi170 ceph-mon[102832]: pgmap v6336: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T02:32:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:32:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:32:25.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:32:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:26 smithi098 ceph-mon[103124]: pgmap v6337: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T02:32:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:26 smithi170 ceph-mon[102832]: pgmap v6337: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T02:32:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:26 smithi082 ceph-mon[98694]: pgmap v6337: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T02:32:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:28 smithi098 ceph-mon[103124]: pgmap v6338: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T02:32:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11580.187323 secs 2023-09-10T02:32:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:28 smithi170 ceph-mon[102832]: pgmap v6338: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T02:32:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11580.187323 secs 2023-09-10T02:32:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:28 smithi082 ceph-mon[98694]: pgmap v6338: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T02:32:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11580.187323 secs 2023-09-10T02:32:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11580.899033 secs 2023-09-10T02:32:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11580.899033 secs 2023-09-10T02:32:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11580.899033 secs 2023-09-10T02:32:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:30 smithi098 ceph-mon[103124]: pgmap v6339: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T02:32:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:30 smithi170 ceph-mon[102832]: pgmap v6339: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T02:32:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:30 smithi082 ceph-mon[98694]: pgmap v6339: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T02:32:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:32 smithi082 ceph-mon[98694]: pgmap v6340: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 232 KiB/s wr, 17 op/s 2023-09-10T02:32:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:32 smithi098 ceph-mon[103124]: pgmap v6340: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 232 KiB/s wr, 17 op/s 2023-09-10T02:32:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:32 smithi170 ceph-mon[102832]: pgmap v6340: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 232 KiB/s wr, 17 op/s 2023-09-10T02:32:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11585.187379 secs 2023-09-10T02:32:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11585.187379 secs 2023-09-10T02:32:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11585.187379 secs 2023-09-10T02:32:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11585.899101 secs 2023-09-10T02:32:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:34 smithi098 ceph-mon[103124]: pgmap v6341: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 199 KiB/s wr, 11 op/s 2023-09-10T02:32:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11585.899101 secs 2023-09-10T02:32:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:34 smithi170 ceph-mon[102832]: pgmap v6341: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 199 KiB/s wr, 11 op/s 2023-09-10T02:32:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11585.899101 secs 2023-09-10T02:32:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:34 smithi082 ceph-mon[98694]: pgmap v6341: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 199 KiB/s wr, 11 op/s 2023-09-10T02:32:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:36 smithi098 ceph-mon[103124]: pgmap v6342: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 210 KiB/s wr, 16 op/s 2023-09-10T02:32:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:36 smithi170 ceph-mon[102832]: pgmap v6342: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 210 KiB/s wr, 16 op/s 2023-09-10T02:32:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:36 smithi082 ceph-mon[98694]: pgmap v6342: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 210 KiB/s wr, 16 op/s 2023-09-10T02:32:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:38 smithi098 ceph-mon[103124]: pgmap v6343: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T02:32:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11590.187437 secs 2023-09-10T02:32:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:38 smithi170 ceph-mon[102832]: pgmap v6343: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T02:32:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11590.187437 secs 2023-09-10T02:32:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:38 smithi082 ceph-mon[98694]: pgmap v6343: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T02:32:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11590.187437 secs 2023-09-10T02:32:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11590.899168 secs 2023-09-10T02:32:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11590.899168 secs 2023-09-10T02:32:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11590.899168 secs 2023-09-10T02:32:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:40 smithi098 ceph-mon[103124]: pgmap v6344: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T02:32:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:40 smithi170 ceph-mon[102832]: pgmap v6344: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T02:32:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:40 smithi082 ceph-mon[98694]: pgmap v6344: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T02:32:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:42 smithi098 ceph-mon[103124]: pgmap v6345: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 294 KiB/s wr, 17 op/s 2023-09-10T02:32:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:42 smithi170 ceph-mon[102832]: pgmap v6345: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 294 KiB/s wr, 17 op/s 2023-09-10T02:32:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:42 smithi082 ceph-mon[98694]: pgmap v6345: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 294 KiB/s wr, 17 op/s 2023-09-10T02:32:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11595.187546 secs 2023-09-10T02:32:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11595.187546 secs 2023-09-10T02:32:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11595.187546 secs 2023-09-10T02:32:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11595.899232 secs 2023-09-10T02:32:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:44 smithi098 ceph-mon[103124]: pgmap v6346: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T02:32:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11595.899232 secs 2023-09-10T02:32:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:44 smithi170 ceph-mon[102832]: pgmap v6346: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T02:32:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11595.899232 secs 2023-09-10T02:32:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:44 smithi082 ceph-mon[98694]: pgmap v6346: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T02:32:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:32:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:32:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:32:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:46 smithi098 ceph-mon[103124]: pgmap v6347: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 244 KiB/s wr, 17 op/s 2023-09-10T02:32:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:32:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:32:47.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:32:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:46 smithi170 ceph-mon[102832]: pgmap v6347: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 244 KiB/s wr, 17 op/s 2023-09-10T02:32:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:32:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:32:47.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:32:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:46 smithi082 ceph-mon[98694]: pgmap v6347: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 244 KiB/s wr, 17 op/s 2023-09-10T02:32:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:32:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:32:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:32:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11598.798080 secs 2023-09-10T02:32:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11598.798080 secs 2023-09-10T02:32:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11598.798080 secs 2023-09-10T02:32:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:48 smithi098 ceph-mon[103124]: pgmap v6348: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 233 KiB/s wr, 12 op/s 2023-09-10T02:32:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11600.187605 secs 2023-09-10T02:32:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:48 smithi170 ceph-mon[102832]: pgmap v6348: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 233 KiB/s wr, 12 op/s 2023-09-10T02:32:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11600.187605 secs 2023-09-10T02:32:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:48 smithi082 ceph-mon[98694]: pgmap v6348: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 233 KiB/s wr, 12 op/s 2023-09-10T02:32:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11600.187605 secs 2023-09-10T02:32:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11600.899298 secs 2023-09-10T02:32:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11600.899298 secs 2023-09-10T02:32:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11600.899298 secs 2023-09-10T02:32:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:50 smithi098 ceph-mon[103124]: pgmap v6349: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 233 KiB/s wr, 12 op/s 2023-09-10T02:32:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:50 smithi170 ceph-mon[102832]: pgmap v6349: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 233 KiB/s wr, 12 op/s 2023-09-10T02:32:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:50 smithi082 ceph-mon[98694]: pgmap v6349: 97 pgs: 97 active+clean; 9.6 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 233 KiB/s wr, 12 op/s 2023-09-10T02:32:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:52 smithi098 ceph-mon[103124]: pgmap v6350: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 275 KiB/s wr, 18 op/s 2023-09-10T02:32:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:32:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:32:53.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:32:53.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11603.798139 secs 2023-09-10T02:32:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:52 smithi170 ceph-mon[102832]: pgmap v6350: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 275 KiB/s wr, 18 op/s 2023-09-10T02:32:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:32:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:32:53.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:32:53.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11603.798139 secs 2023-09-10T02:32:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:52 smithi082 ceph-mon[98694]: pgmap v6350: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 275 KiB/s wr, 18 op/s 2023-09-10T02:32:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:32:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:32:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:32:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11603.798139 secs 2023-09-10T02:32:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11605.187660 secs 2023-09-10T02:32:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11605.187660 secs 2023-09-10T02:32:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11605.187660 secs 2023-09-10T02:32:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11605.899360 secs 2023-09-10T02:32:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:54 smithi098 ceph-mon[103124]: pgmap v6351: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 160 KiB/s wr, 11 op/s 2023-09-10T02:32:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11605.899360 secs 2023-09-10T02:32:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:54 smithi170 ceph-mon[102832]: pgmap v6351: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 160 KiB/s wr, 11 op/s 2023-09-10T02:32:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11605.899360 secs 2023-09-10T02:32:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:54 smithi082 ceph-mon[98694]: pgmap v6351: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 160 KiB/s wr, 11 op/s 2023-09-10T02:32:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:56 smithi098 ceph-mon[103124]: pgmap v6352: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 227 KiB/s wr, 16 op/s 2023-09-10T02:32:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:56 smithi170 ceph-mon[102832]: pgmap v6352: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 227 KiB/s wr, 16 op/s 2023-09-10T02:32:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:56 smithi082 ceph-mon[98694]: pgmap v6352: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 227 KiB/s wr, 16 op/s 2023-09-10T02:32:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11608.798260 secs 2023-09-10T02:32:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11608.798260 secs 2023-09-10T02:32:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11608.798260 secs 2023-09-10T02:32:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:58 smithi098 ceph-mon[103124]: pgmap v6353: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T02:32:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11610.187718 secs 2023-09-10T02:32:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:58 smithi170 ceph-mon[102832]: pgmap v6353: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T02:32:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11610.187718 secs 2023-09-10T02:32:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:58 smithi082 ceph-mon[98694]: pgmap v6353: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T02:32:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11610.187718 secs 2023-09-10T02:33:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:32:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11610.899424 secs 2023-09-10T02:33:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:32:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11610.899424 secs 2023-09-10T02:33:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:32:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11610.899424 secs 2023-09-10T02:33:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:00 smithi098 ceph-mon[103124]: pgmap v6354: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T02:33:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:00 smithi170 ceph-mon[102832]: pgmap v6354: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T02:33:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:00 smithi082 ceph-mon[98694]: pgmap v6354: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T02:33:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:02 smithi098 ceph-mon[103124]: pgmap v6355: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 140 KiB/s wr, 15 op/s 2023-09-10T02:33:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11613.798322 secs 2023-09-10T02:33:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:02 smithi170 ceph-mon[102832]: pgmap v6355: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 140 KiB/s wr, 15 op/s 2023-09-10T02:33:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11613.798322 secs 2023-09-10T02:33:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:02 smithi082 ceph-mon[98694]: pgmap v6355: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 140 KiB/s wr, 15 op/s 2023-09-10T02:33:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11613.798322 secs 2023-09-10T02:33:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11615.187773 secs 2023-09-10T02:33:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11615.187773 secs 2023-09-10T02:33:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11615.187773 secs 2023-09-10T02:33:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11615.899490 secs 2023-09-10T02:33:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:04 smithi098 ceph-mon[103124]: pgmap v6356: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 9 op/s 2023-09-10T02:33:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11615.899490 secs 2023-09-10T02:33:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:04 smithi170 ceph-mon[102832]: pgmap v6356: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 9 op/s 2023-09-10T02:33:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11615.899490 secs 2023-09-10T02:33:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:04 smithi082 ceph-mon[98694]: pgmap v6356: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 9 op/s 2023-09-10T02:33:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:06 smithi098 ceph-mon[103124]: pgmap v6357: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 152 KiB/s wr, 29 op/s 2023-09-10T02:33:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:06 smithi170 ceph-mon[102832]: pgmap v6357: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 152 KiB/s wr, 29 op/s 2023-09-10T02:33:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:06 smithi082 ceph-mon[98694]: pgmap v6357: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 152 KiB/s wr, 29 op/s 2023-09-10T02:33:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11618.798383 secs 2023-09-10T02:33:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11618.798383 secs 2023-09-10T02:33:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11618.798383 secs 2023-09-10T02:33:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:08 smithi098 ceph-mon[103124]: pgmap v6358: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 85 KiB/s wr, 24 op/s 2023-09-10T02:33:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:08 smithi098 ceph-mon[103124]: osdmap e465: 12 total, 12 up, 12 in 2023-09-10T02:33:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11620.187881 secs 2023-09-10T02:33:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:08 smithi170 ceph-mon[102832]: pgmap v6358: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 85 KiB/s wr, 24 op/s 2023-09-10T02:33:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:08 smithi170 ceph-mon[102832]: osdmap e465: 12 total, 12 up, 12 in 2023-09-10T02:33:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11620.187881 secs 2023-09-10T02:33:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:08 smithi082 ceph-mon[98694]: pgmap v6358: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 85 KiB/s wr, 24 op/s 2023-09-10T02:33:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:08 smithi082 ceph-mon[98694]: osdmap e465: 12 total, 12 up, 12 in 2023-09-10T02:33:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11620.187881 secs 2023-09-10T02:33:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11620.899558 secs 2023-09-10T02:33:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11620.899558 secs 2023-09-10T02:33:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11620.899558 secs 2023-09-10T02:33:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:10 smithi098 ceph-mon[103124]: pgmap v6360: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T02:33:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:10 smithi170 ceph-mon[102832]: pgmap v6360: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T02:33:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:10 smithi082 ceph-mon[98694]: pgmap v6360: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T02:33:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:12 smithi098 ceph-mon[103124]: pgmap v6361: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T02:33:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11623.798445 secs 2023-09-10T02:33:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:12 smithi170 ceph-mon[102832]: pgmap v6361: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T02:33:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11623.798445 secs 2023-09-10T02:33:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:12 smithi082 ceph-mon[98694]: pgmap v6361: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T02:33:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11623.798445 secs 2023-09-10T02:33:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11625.187967 secs 2023-09-10T02:33:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11625.187967 secs 2023-09-10T02:33:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11625.187967 secs 2023-09-10T02:33:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11625.899624 secs 2023-09-10T02:33:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:14 smithi098 ceph-mon[103124]: pgmap v6362: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T02:33:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11625.899624 secs 2023-09-10T02:33:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:14 smithi170 ceph-mon[102832]: pgmap v6362: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T02:33:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11625.899624 secs 2023-09-10T02:33:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:14 smithi082 ceph-mon[98694]: pgmap v6362: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T02:33:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:16 smithi098 ceph-mon[103124]: pgmap v6363: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T02:33:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:16 smithi098 ceph-mon[103124]: osdmap e466: 12 total, 12 up, 12 in 2023-09-10T02:33:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:16 smithi170 ceph-mon[102832]: pgmap v6363: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T02:33:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:16 smithi170 ceph-mon[102832]: osdmap e466: 12 total, 12 up, 12 in 2023-09-10T02:33:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:16 smithi082 ceph-mon[98694]: pgmap v6363: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T02:33:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:16 smithi082 ceph-mon[98694]: osdmap e466: 12 total, 12 up, 12 in 2023-09-10T02:33:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11628.798503 secs 2023-09-10T02:33:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11628.798503 secs 2023-09-10T02:33:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11628.798503 secs 2023-09-10T02:33:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:18 smithi098 ceph-mon[103124]: pgmap v6365: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T02:33:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11630.188047 secs 2023-09-10T02:33:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:18 smithi170 ceph-mon[102832]: pgmap v6365: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T02:33:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11630.188047 secs 2023-09-10T02:33:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:18 smithi082 ceph-mon[98694]: pgmap v6365: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T02:33:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11630.188047 secs 2023-09-10T02:33:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11630.899687 secs 2023-09-10T02:33:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11630.899687 secs 2023-09-10T02:33:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11630.899687 secs 2023-09-10T02:33:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:20 smithi098 ceph-mon[103124]: pgmap v6366: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T02:33:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:20 smithi170 ceph-mon[102832]: pgmap v6366: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T02:33:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:20 smithi082 ceph-mon[98694]: pgmap v6366: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T02:33:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:22 smithi098 ceph-mon[103124]: pgmap v6367: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T02:33:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:22 smithi170 ceph-mon[102832]: pgmap v6367: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T02:33:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:22 smithi082 ceph-mon[98694]: pgmap v6367: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T02:33:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:33:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11635.188108 secs 2023-09-10T02:33:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:33:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11635.188108 secs 2023-09-10T02:33:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:33:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11635.188108 secs 2023-09-10T02:33:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11635.899750 secs 2023-09-10T02:33:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:24 smithi098 ceph-mon[103124]: pgmap v6368: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T02:33:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:33:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:33:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:33:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11635.899750 secs 2023-09-10T02:33:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:24 smithi170 ceph-mon[102832]: pgmap v6368: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T02:33:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:33:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:33:25.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:33:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11635.899750 secs 2023-09-10T02:33:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:24 smithi082 ceph-mon[98694]: pgmap v6368: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 11 op/s 2023-09-10T02:33:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:33:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:33:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:33:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:26 smithi098 ceph-mon[103124]: pgmap v6369: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-10T02:33:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:26 smithi170 ceph-mon[102832]: pgmap v6369: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-10T02:33:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:26 smithi082 ceph-mon[98694]: pgmap v6369: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-10T02:33:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:28 smithi098 ceph-mon[103124]: pgmap v6370: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T02:33:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11640.188193 secs 2023-09-10T02:33:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:28 smithi170 ceph-mon[102832]: pgmap v6370: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T02:33:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11640.188193 secs 2023-09-10T02:33:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:28 smithi082 ceph-mon[98694]: pgmap v6370: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T02:33:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11640.188193 secs 2023-09-10T02:33:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11640.899816 secs 2023-09-10T02:33:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11640.899816 secs 2023-09-10T02:33:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11640.899816 secs 2023-09-10T02:33:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:30 smithi098 ceph-mon[103124]: pgmap v6371: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 10 op/s 2023-09-10T02:33:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:30 smithi170 ceph-mon[102832]: pgmap v6371: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 10 op/s 2023-09-10T02:33:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:30 smithi082 ceph-mon[98694]: pgmap v6371: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 10 op/s 2023-09-10T02:33:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:32 smithi098 ceph-mon[103124]: pgmap v6372: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 145 KiB/s wr, 16 op/s 2023-09-10T02:33:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:32 smithi170 ceph-mon[102832]: pgmap v6372: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 145 KiB/s wr, 16 op/s 2023-09-10T02:33:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:32 smithi082 ceph-mon[98694]: pgmap v6372: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 145 KiB/s wr, 16 op/s 2023-09-10T02:33:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11645.188251 secs 2023-09-10T02:33:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11645.188251 secs 2023-09-10T02:33:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11645.188251 secs 2023-09-10T02:33:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11645.899880 secs 2023-09-10T02:33:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:34 smithi098 ceph-mon[103124]: pgmap v6373: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 145 KiB/s wr, 12 op/s 2023-09-10T02:33:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11645.899880 secs 2023-09-10T02:33:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:34 smithi170 ceph-mon[102832]: pgmap v6373: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 145 KiB/s wr, 12 op/s 2023-09-10T02:33:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11645.899880 secs 2023-09-10T02:33:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:34 smithi082 ceph-mon[98694]: pgmap v6373: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 145 KiB/s wr, 12 op/s 2023-09-10T02:33:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:36 smithi098 ceph-mon[103124]: pgmap v6374: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 213 KiB/s wr, 18 op/s 2023-09-10T02:33:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:36 smithi170 ceph-mon[102832]: pgmap v6374: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 213 KiB/s wr, 18 op/s 2023-09-10T02:33:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:36 smithi082 ceph-mon[98694]: pgmap v6374: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 213 KiB/s wr, 18 op/s 2023-09-10T02:33:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:38 smithi098 ceph-mon[103124]: pgmap v6375: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 13 op/s 2023-09-10T02:33:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11650.188310 secs 2023-09-10T02:33:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:38 smithi170 ceph-mon[102832]: pgmap v6375: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 13 op/s 2023-09-10T02:33:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11650.188310 secs 2023-09-10T02:33:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:38 smithi082 ceph-mon[98694]: pgmap v6375: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 13 op/s 2023-09-10T02:33:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11650.188310 secs 2023-09-10T02:33:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11650.899946 secs 2023-09-10T02:33:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11650.899946 secs 2023-09-10T02:33:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11650.899946 secs 2023-09-10T02:33:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:40 smithi098 ceph-mon[103124]: pgmap v6376: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 13 op/s 2023-09-10T02:33:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:40 smithi170 ceph-mon[102832]: pgmap v6376: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 13 op/s 2023-09-10T02:33:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:40 smithi082 ceph-mon[98694]: pgmap v6376: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 13 op/s 2023-09-10T02:33:43.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:42 smithi098 ceph-mon[103124]: pgmap v6377: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 247 KiB/s wr, 19 op/s 2023-09-10T02:33:43.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11653.798925 secs 2023-09-10T02:33:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:42 smithi170 ceph-mon[102832]: pgmap v6377: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 247 KiB/s wr, 19 op/s 2023-09-10T02:33:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11653.798925 secs 2023-09-10T02:33:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:42 smithi082 ceph-mon[98694]: pgmap v6377: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 247 KiB/s wr, 19 op/s 2023-09-10T02:33:43.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11653.798925 secs 2023-09-10T02:33:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11655.188401 secs 2023-09-10T02:33:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11655.188401 secs 2023-09-10T02:33:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11655.188401 secs 2023-09-10T02:33:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11655.900015 secs 2023-09-10T02:33:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:44 smithi098 ceph-mon[103124]: pgmap v6378: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 170 KiB/s wr, 12 op/s 2023-09-10T02:33:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:33:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:33:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:33:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11655.900015 secs 2023-09-10T02:33:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:44 smithi170 ceph-mon[102832]: pgmap v6378: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 170 KiB/s wr, 12 op/s 2023-09-10T02:33:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:33:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:33:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:33:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11655.900015 secs 2023-09-10T02:33:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:44 smithi082 ceph-mon[98694]: pgmap v6378: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 170 KiB/s wr, 12 op/s 2023-09-10T02:33:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:33:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:33:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:33:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:46 smithi098 ceph-mon[103124]: pgmap v6379: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 240 KiB/s wr, 18 op/s 2023-09-10T02:33:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:33:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:33:47.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:33:47.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:33:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:46 smithi170 ceph-mon[102832]: pgmap v6379: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 240 KiB/s wr, 18 op/s 2023-09-10T02:33:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:33:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:33:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:33:47.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:33:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:46 smithi082 ceph-mon[98694]: pgmap v6379: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 240 KiB/s wr, 18 op/s 2023-09-10T02:33:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:33:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:33:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:33:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:33:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11658.798985 secs 2023-09-10T02:33:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11658.798985 secs 2023-09-10T02:33:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11658.798985 secs 2023-09-10T02:33:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:48 smithi098 ceph-mon[103124]: pgmap v6380: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 173 KiB/s wr, 12 op/s 2023-09-10T02:33:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11660.188481 secs 2023-09-10T02:33:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:48 smithi170 ceph-mon[102832]: pgmap v6380: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 173 KiB/s wr, 12 op/s 2023-09-10T02:33:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11660.188481 secs 2023-09-10T02:33:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:48 smithi082 ceph-mon[98694]: pgmap v6380: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 173 KiB/s wr, 12 op/s 2023-09-10T02:33:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11660.188481 secs 2023-09-10T02:33:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11660.900082 secs 2023-09-10T02:33:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11660.900082 secs 2023-09-10T02:33:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11660.900082 secs 2023-09-10T02:33:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:50 smithi098 ceph-mon[103124]: pgmap v6381: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 173 KiB/s wr, 12 op/s 2023-09-10T02:33:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:50 smithi170 ceph-mon[102832]: pgmap v6381: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 173 KiB/s wr, 12 op/s 2023-09-10T02:33:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:50 smithi082 ceph-mon[98694]: pgmap v6381: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 173 KiB/s wr, 12 op/s 2023-09-10T02:33:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:52 smithi098 ceph-mon[103124]: pgmap v6382: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 208 KiB/s wr, 17 op/s 2023-09-10T02:33:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11663.799045 secs 2023-09-10T02:33:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:52 smithi170 ceph-mon[102832]: pgmap v6382: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 208 KiB/s wr, 17 op/s 2023-09-10T02:33:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11663.799045 secs 2023-09-10T02:33:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:52 smithi082 ceph-mon[98694]: pgmap v6382: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 208 KiB/s wr, 17 op/s 2023-09-10T02:33:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11663.799045 secs 2023-09-10T02:33:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11665.188492 secs 2023-09-10T02:33:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11665.188492 secs 2023-09-10T02:33:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11665.188492 secs 2023-09-10T02:33:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11665.900149 secs 2023-09-10T02:33:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:54 smithi098 ceph-mon[103124]: pgmap v6383: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 10 op/s 2023-09-10T02:33:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11665.900149 secs 2023-09-10T02:33:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:54 smithi170 ceph-mon[102832]: pgmap v6383: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 10 op/s 2023-09-10T02:33:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11665.900149 secs 2023-09-10T02:33:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:54 smithi082 ceph-mon[98694]: pgmap v6383: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 10 op/s 2023-09-10T02:33:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:56 smithi098 ceph-mon[103124]: pgmap v6384: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 134 KiB/s wr, 15 op/s 2023-09-10T02:33:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:56 smithi170 ceph-mon[102832]: pgmap v6384: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 134 KiB/s wr, 15 op/s 2023-09-10T02:33:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:56 smithi082 ceph-mon[98694]: pgmap v6384: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 134 KiB/s wr, 15 op/s 2023-09-10T02:33:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11668.799140 secs 2023-09-10T02:33:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11668.799140 secs 2023-09-10T02:33:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11668.799140 secs 2023-09-10T02:33:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:58 smithi098 ceph-mon[103124]: pgmap v6385: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 9 op/s 2023-09-10T02:33:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11670.188548 secs 2023-09-10T02:33:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:58 smithi170 ceph-mon[102832]: pgmap v6385: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 9 op/s 2023-09-10T02:33:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11670.188548 secs 2023-09-10T02:33:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:58 smithi082 ceph-mon[98694]: pgmap v6385: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 9 op/s 2023-09-10T02:33:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11670.188548 secs 2023-09-10T02:34:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:33:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11670.900214 secs 2023-09-10T02:34:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:33:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11670.900214 secs 2023-09-10T02:34:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:33:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11670.900214 secs 2023-09-10T02:34:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:00 smithi098 ceph-mon[103124]: pgmap v6386: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 9 op/s 2023-09-10T02:34:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:00 smithi170 ceph-mon[102832]: pgmap v6386: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 9 op/s 2023-09-10T02:34:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:00 smithi082 ceph-mon[98694]: pgmap v6386: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 9 op/s 2023-09-10T02:34:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:02 smithi098 ceph-mon[103124]: pgmap v6387: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 88 KiB/s wr, 14 op/s 2023-09-10T02:34:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11673.799199 secs 2023-09-10T02:34:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:02 smithi170 ceph-mon[102832]: pgmap v6387: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 88 KiB/s wr, 14 op/s 2023-09-10T02:34:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11673.799199 secs 2023-09-10T02:34:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:02 smithi082 ceph-mon[98694]: pgmap v6387: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 88 KiB/s wr, 14 op/s 2023-09-10T02:34:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11673.799199 secs 2023-09-10T02:34:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11675.188635 secs 2023-09-10T02:34:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11675.188635 secs 2023-09-10T02:34:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11675.188635 secs 2023-09-10T02:34:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11675.900283 secs 2023-09-10T02:34:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:04 smithi098 ceph-mon[103124]: pgmap v6388: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 9 op/s 2023-09-10T02:34:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11675.900283 secs 2023-09-10T02:34:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:04 smithi170 ceph-mon[102832]: pgmap v6388: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 9 op/s 2023-09-10T02:34:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11675.900283 secs 2023-09-10T02:34:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:04 smithi082 ceph-mon[98694]: pgmap v6388: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 9 op/s 2023-09-10T02:34:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:06 smithi098 ceph-mon[103124]: pgmap v6389: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 108 KiB/s wr, 29 op/s 2023-09-10T02:34:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:06 smithi170 ceph-mon[102832]: pgmap v6389: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 108 KiB/s wr, 29 op/s 2023-09-10T02:34:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:06 smithi082 ceph-mon[98694]: pgmap v6389: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 108 KiB/s wr, 29 op/s 2023-09-10T02:34:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11678.799271 secs 2023-09-10T02:34:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11678.799271 secs 2023-09-10T02:34:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11678.799271 secs 2023-09-10T02:34:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:08 smithi098 ceph-mon[103124]: pgmap v6390: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 78 KiB/s wr, 24 op/s 2023-09-10T02:34:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:08 smithi098 ceph-mon[103124]: osdmap e467: 12 total, 12 up, 12 in 2023-09-10T02:34:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11680.188696 secs 2023-09-10T02:34:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:08 smithi170 ceph-mon[102832]: pgmap v6390: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 78 KiB/s wr, 24 op/s 2023-09-10T02:34:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:08 smithi170 ceph-mon[102832]: osdmap e467: 12 total, 12 up, 12 in 2023-09-10T02:34:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11680.188696 secs 2023-09-10T02:34:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:08 smithi082 ceph-mon[98694]: pgmap v6390: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 78 KiB/s wr, 24 op/s 2023-09-10T02:34:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:08 smithi082 ceph-mon[98694]: osdmap e467: 12 total, 12 up, 12 in 2023-09-10T02:34:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11680.188696 secs 2023-09-10T02:34:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11680.900347 secs 2023-09-10T02:34:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11680.900347 secs 2023-09-10T02:34:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11680.900347 secs 2023-09-10T02:34:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:10 smithi170 ceph-mon[102832]: pgmap v6392: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 94 KiB/s wr, 29 op/s 2023-09-10T02:34:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:10 smithi082 ceph-mon[98694]: pgmap v6392: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 94 KiB/s wr, 29 op/s 2023-09-10T02:34:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:10 smithi098 ceph-mon[103124]: pgmap v6392: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 94 KiB/s wr, 29 op/s 2023-09-10T02:34:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:12 smithi098 ceph-mon[103124]: pgmap v6393: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T02:34:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11683.799372 secs 2023-09-10T02:34:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:12 smithi170 ceph-mon[102832]: pgmap v6393: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T02:34:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11683.799372 secs 2023-09-10T02:34:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:12 smithi082 ceph-mon[98694]: pgmap v6393: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T02:34:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11683.799372 secs 2023-09-10T02:34:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11685.188753 secs 2023-09-10T02:34:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11685.188753 secs 2023-09-10T02:34:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11685.188753 secs 2023-09-10T02:34:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11685.900413 secs 2023-09-10T02:34:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:14 smithi098 ceph-mon[103124]: pgmap v6394: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T02:34:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11685.900413 secs 2023-09-10T02:34:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:14 smithi170 ceph-mon[102832]: pgmap v6394: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T02:34:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11685.900413 secs 2023-09-10T02:34:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:14 smithi082 ceph-mon[98694]: pgmap v6394: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-09-10T02:34:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:16 smithi098 ceph-mon[103124]: pgmap v6395: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T02:34:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:16 smithi098 ceph-mon[103124]: osdmap e468: 12 total, 12 up, 12 in 2023-09-10T02:34:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:16 smithi170 ceph-mon[102832]: pgmap v6395: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T02:34:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:16 smithi170 ceph-mon[102832]: osdmap e468: 12 total, 12 up, 12 in 2023-09-10T02:34:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:16 smithi082 ceph-mon[98694]: pgmap v6395: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T02:34:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:16 smithi082 ceph-mon[98694]: osdmap e468: 12 total, 12 up, 12 in 2023-09-10T02:34:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11688.799430 secs 2023-09-10T02:34:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11688.799430 secs 2023-09-10T02:34:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11688.799430 secs 2023-09-10T02:34:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:18 smithi098 ceph-mon[103124]: pgmap v6397: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T02:34:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11690.188810 secs 2023-09-10T02:34:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:18 smithi170 ceph-mon[102832]: pgmap v6397: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T02:34:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11690.188810 secs 2023-09-10T02:34:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:18 smithi082 ceph-mon[98694]: pgmap v6397: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T02:34:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11690.188810 secs 2023-09-10T02:34:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11690.900482 secs 2023-09-10T02:34:20.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11690.900482 secs 2023-09-10T02:34:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11690.900482 secs 2023-09-10T02:34:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:20 smithi170 ceph-mon[102832]: pgmap v6398: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T02:34:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:20 smithi082 ceph-mon[98694]: pgmap v6398: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T02:34:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:20 smithi098 ceph-mon[103124]: pgmap v6398: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T02:34:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:22 smithi098 ceph-mon[103124]: pgmap v6399: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T02:34:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:22 smithi170 ceph-mon[102832]: pgmap v6399: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T02:34:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:22 smithi082 ceph-mon[98694]: pgmap v6399: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T02:34:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:34:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11695.188867 secs 2023-09-10T02:34:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:34:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11695.188867 secs 2023-09-10T02:34:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:34:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11695.188867 secs 2023-09-10T02:34:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11695.900546 secs 2023-09-10T02:34:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:24 smithi098 ceph-mon[103124]: pgmap v6400: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T02:34:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:34:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:34:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:34:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11695.900546 secs 2023-09-10T02:34:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:24 smithi170 ceph-mon[102832]: pgmap v6400: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T02:34:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:34:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:34:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:34:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11695.900546 secs 2023-09-10T02:34:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:24 smithi082 ceph-mon[98694]: pgmap v6400: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 12 op/s 2023-09-10T02:34:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:34:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:34:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:34:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:26 smithi098 ceph-mon[103124]: pgmap v6401: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 13 op/s 2023-09-10T02:34:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:26 smithi170 ceph-mon[102832]: pgmap v6401: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 13 op/s 2023-09-10T02:34:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:26 smithi082 ceph-mon[98694]: pgmap v6401: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 13 op/s 2023-09-10T02:34:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:28 smithi098 ceph-mon[103124]: pgmap v6402: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T02:34:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11700.188926 secs 2023-09-10T02:34:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:28 smithi170 ceph-mon[102832]: pgmap v6402: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T02:34:29.215 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11700.188926 secs 2023-09-10T02:34:29.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:28 smithi082 ceph-mon[98694]: pgmap v6402: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T02:34:29.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11700.188926 secs 2023-09-10T02:34:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11700.900611 secs 2023-09-10T02:34:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11700.900611 secs 2023-09-10T02:34:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11700.900611 secs 2023-09-10T02:34:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:30 smithi098 ceph-mon[103124]: pgmap v6403: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 10 op/s 2023-09-10T02:34:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:30 smithi170 ceph-mon[102832]: pgmap v6403: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 10 op/s 2023-09-10T02:34:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:30 smithi082 ceph-mon[98694]: pgmap v6403: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 10 op/s 2023-09-10T02:34:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:32 smithi098 ceph-mon[103124]: pgmap v6404: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 131 KiB/s wr, 16 op/s 2023-09-10T02:34:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:32 smithi170 ceph-mon[102832]: pgmap v6404: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 131 KiB/s wr, 16 op/s 2023-09-10T02:34:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:32 smithi082 ceph-mon[98694]: pgmap v6404: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 131 KiB/s wr, 16 op/s 2023-09-10T02:34:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11705.188984 secs 2023-09-10T02:34:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11705.188984 secs 2023-09-10T02:34:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11705.188984 secs 2023-09-10T02:34:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11705.900677 secs 2023-09-10T02:34:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:34 smithi098 ceph-mon[103124]: pgmap v6405: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 131 KiB/s wr, 11 op/s 2023-09-10T02:34:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11705.900677 secs 2023-09-10T02:34:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:34 smithi170 ceph-mon[102832]: pgmap v6405: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 131 KiB/s wr, 11 op/s 2023-09-10T02:34:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11705.900677 secs 2023-09-10T02:34:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:34 smithi082 ceph-mon[98694]: pgmap v6405: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 131 KiB/s wr, 11 op/s 2023-09-10T02:34:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:36 smithi098 ceph-mon[103124]: pgmap v6406: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 206 KiB/s wr, 17 op/s 2023-09-10T02:34:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:36 smithi170 ceph-mon[102832]: pgmap v6406: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 206 KiB/s wr, 17 op/s 2023-09-10T02:34:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:36 smithi082 ceph-mon[98694]: pgmap v6406: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 206 KiB/s wr, 17 op/s 2023-09-10T02:34:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:38 smithi170 ceph-mon[102832]: pgmap v6407: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T02:34:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11710.189071 secs 2023-09-10T02:34:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:38 smithi082 ceph-mon[98694]: pgmap v6407: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T02:34:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11710.189071 secs 2023-09-10T02:34:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:38 smithi098 ceph-mon[103124]: pgmap v6407: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T02:34:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11710.189071 secs 2023-09-10T02:34:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11710.900741 secs 2023-09-10T02:34:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11710.900741 secs 2023-09-10T02:34:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11710.900741 secs 2023-09-10T02:34:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:40 smithi170 ceph-mon[102832]: pgmap v6408: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T02:34:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:40 smithi082 ceph-mon[98694]: pgmap v6408: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T02:34:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:40 smithi098 ceph-mon[103124]: pgmap v6408: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T02:34:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:42 smithi098 ceph-mon[103124]: pgmap v6409: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 203 KiB/s wr, 17 op/s 2023-09-10T02:34:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:42 smithi170 ceph-mon[102832]: pgmap v6409: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 203 KiB/s wr, 17 op/s 2023-09-10T02:34:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:42 smithi082 ceph-mon[98694]: pgmap v6409: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 203 KiB/s wr, 17 op/s 2023-09-10T02:34:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11715.189124 secs 2023-09-10T02:34:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11715.189124 secs 2023-09-10T02:34:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11715.189124 secs 2023-09-10T02:34:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11715.900809 secs 2023-09-10T02:34:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:44 smithi098 ceph-mon[103124]: pgmap v6410: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T02:34:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11715.900809 secs 2023-09-10T02:34:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:44 smithi170 ceph-mon[102832]: pgmap v6410: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T02:34:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11715.900809 secs 2023-09-10T02:34:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:44 smithi082 ceph-mon[98694]: pgmap v6410: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T02:34:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:46 smithi170 ceph-mon[102832]: pgmap v6411: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 220 KiB/s wr, 17 op/s 2023-09-10T02:34:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:34:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:46 smithi082 ceph-mon[98694]: pgmap v6411: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 220 KiB/s wr, 17 op/s 2023-09-10T02:34:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:34:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:46 smithi098 ceph-mon[103124]: pgmap v6411: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 220 KiB/s wr, 17 op/s 2023-09-10T02:34:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:34:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11718.799934 secs 2023-09-10T02:34:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:34:48.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:34:48.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:34:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11718.799934 secs 2023-09-10T02:34:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:34:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:34:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:34:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11718.799934 secs 2023-09-10T02:34:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:34:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:34:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:34:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:48 smithi170 ceph-mon[102832]: pgmap v6412: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T02:34:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11720.189181 secs 2023-09-10T02:34:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:48 smithi082 ceph-mon[98694]: pgmap v6412: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T02:34:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11720.189181 secs 2023-09-10T02:34:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:48 smithi098 ceph-mon[103124]: pgmap v6412: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T02:34:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11720.189181 secs 2023-09-10T02:34:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11720.900872 secs 2023-09-10T02:34:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11720.900872 secs 2023-09-10T02:34:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11720.900872 secs 2023-09-10T02:34:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:50 smithi170 ceph-mon[102832]: pgmap v6413: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T02:34:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:50 smithi082 ceph-mon[98694]: pgmap v6413: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T02:34:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:50 smithi098 ceph-mon[103124]: pgmap v6413: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T02:34:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:34:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:34:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:34:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:34:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:34:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:34:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:34:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:34:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:34:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:52 smithi170 ceph-mon[102832]: pgmap v6414: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 210 KiB/s wr, 17 op/s 2023-09-10T02:34:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11723.799994 secs 2023-09-10T02:34:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:52 smithi082 ceph-mon[98694]: pgmap v6414: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 210 KiB/s wr, 17 op/s 2023-09-10T02:34:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11723.799994 secs 2023-09-10T02:34:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:52 smithi098 ceph-mon[103124]: pgmap v6414: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 210 KiB/s wr, 17 op/s 2023-09-10T02:34:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11723.799994 secs 2023-09-10T02:34:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11725.189238 secs 2023-09-10T02:34:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11725.189238 secs 2023-09-10T02:34:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11725.189238 secs 2023-09-10T02:34:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11725.900937 secs 2023-09-10T02:34:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:54 smithi170 ceph-mon[102832]: pgmap v6415: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T02:34:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11725.900937 secs 2023-09-10T02:34:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:54 smithi082 ceph-mon[98694]: pgmap v6415: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T02:34:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11725.900937 secs 2023-09-10T02:34:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:54 smithi098 ceph-mon[103124]: pgmap v6415: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T02:34:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:56 smithi170 ceph-mon[102832]: pgmap v6416: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 196 KiB/s wr, 16 op/s 2023-09-10T02:34:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:56 smithi082 ceph-mon[98694]: pgmap v6416: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 196 KiB/s wr, 16 op/s 2023-09-10T02:34:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:56 smithi098 ceph-mon[103124]: pgmap v6416: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 196 KiB/s wr, 16 op/s 2023-09-10T02:34:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11728.800102 secs 2023-09-10T02:34:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11728.800102 secs 2023-09-10T02:34:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11728.800102 secs 2023-09-10T02:34:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:58 smithi170 ceph-mon[102832]: pgmap v6417: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T02:34:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11730.189296 secs 2023-09-10T02:34:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:58 smithi082 ceph-mon[98694]: pgmap v6417: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T02:34:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11730.189296 secs 2023-09-10T02:34:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:58 smithi098 ceph-mon[103124]: pgmap v6417: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T02:34:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11730.189296 secs 2023-09-10T02:35:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:34:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11730.901000 secs 2023-09-10T02:35:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:34:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11730.901000 secs 2023-09-10T02:35:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:34:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11730.901000 secs 2023-09-10T02:35:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:00 smithi170 ceph-mon[102832]: pgmap v6418: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T02:35:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:00 smithi082 ceph-mon[98694]: pgmap v6418: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T02:35:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:00 smithi098 ceph-mon[103124]: pgmap v6418: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T02:35:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:02 smithi098 ceph-mon[103124]: pgmap v6419: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 140 KiB/s wr, 16 op/s 2023-09-10T02:35:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11733.800162 secs 2023-09-10T02:35:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:02 smithi170 ceph-mon[102832]: pgmap v6419: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 140 KiB/s wr, 16 op/s 2023-09-10T02:35:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11733.800162 secs 2023-09-10T02:35:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:02 smithi082 ceph-mon[98694]: pgmap v6419: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 140 KiB/s wr, 16 op/s 2023-09-10T02:35:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11733.800162 secs 2023-09-10T02:35:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11735.189353 secs 2023-09-10T02:35:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11735.189353 secs 2023-09-10T02:35:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11735.189353 secs 2023-09-10T02:35:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11735.901064 secs 2023-09-10T02:35:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:04 smithi170 ceph-mon[102832]: pgmap v6420: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T02:35:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11735.901064 secs 2023-09-10T02:35:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:04 smithi082 ceph-mon[98694]: pgmap v6420: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T02:35:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11735.901064 secs 2023-09-10T02:35:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:04 smithi098 ceph-mon[103124]: pgmap v6420: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T02:35:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:06 smithi170 ceph-mon[102832]: pgmap v6421: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 129 KiB/s wr, 30 op/s 2023-09-10T02:35:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:06 smithi082 ceph-mon[98694]: pgmap v6421: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 129 KiB/s wr, 30 op/s 2023-09-10T02:35:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:06 smithi098 ceph-mon[103124]: pgmap v6421: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 129 KiB/s wr, 30 op/s 2023-09-10T02:35:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11738.800220 secs 2023-09-10T02:35:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11738.800220 secs 2023-09-10T02:35:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11738.800220 secs 2023-09-10T02:35:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:08 smithi170 ceph-mon[102832]: pgmap v6422: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 86 KiB/s wr, 25 op/s 2023-09-10T02:35:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:08 smithi170 ceph-mon[102832]: osdmap e469: 12 total, 12 up, 12 in 2023-09-10T02:35:09.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11740.189439 secs 2023-09-10T02:35:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:08 smithi082 ceph-mon[98694]: pgmap v6422: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 86 KiB/s wr, 25 op/s 2023-09-10T02:35:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:08 smithi082 ceph-mon[98694]: osdmap e469: 12 total, 12 up, 12 in 2023-09-10T02:35:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11740.189439 secs 2023-09-10T02:35:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:08 smithi098 ceph-mon[103124]: pgmap v6422: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 86 KiB/s wr, 25 op/s 2023-09-10T02:35:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:08 smithi098 ceph-mon[103124]: osdmap e469: 12 total, 12 up, 12 in 2023-09-10T02:35:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11740.189439 secs 2023-09-10T02:35:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11740.901128 secs 2023-09-10T02:35:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11740.901128 secs 2023-09-10T02:35:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11740.901128 secs 2023-09-10T02:35:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:10 smithi170 ceph-mon[102832]: pgmap v6424: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 103 KiB/s wr, 30 op/s 2023-09-10T02:35:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:10 smithi082 ceph-mon[98694]: pgmap v6424: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 103 KiB/s wr, 30 op/s 2023-09-10T02:35:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:10 smithi098 ceph-mon[103124]: pgmap v6424: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 103 KiB/s wr, 30 op/s 2023-09-10T02:35:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:12 smithi098 ceph-mon[103124]: pgmap v6425: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T02:35:13.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11743.800280 secs 2023-09-10T02:35:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:12 smithi170 ceph-mon[102832]: pgmap v6425: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T02:35:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11743.800280 secs 2023-09-10T02:35:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:12 smithi082 ceph-mon[98694]: pgmap v6425: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T02:35:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11743.800280 secs 2023-09-10T02:35:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11745.189524 secs 2023-09-10T02:35:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11745.189524 secs 2023-09-10T02:35:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11745.189524 secs 2023-09-10T02:35:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11745.901193 secs 2023-09-10T02:35:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:14 smithi170 ceph-mon[102832]: pgmap v6426: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T02:35:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11745.901193 secs 2023-09-10T02:35:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:14 smithi082 ceph-mon[98694]: pgmap v6426: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T02:35:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11745.901193 secs 2023-09-10T02:35:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:14 smithi098 ceph-mon[103124]: pgmap v6426: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 88 KiB/s wr, 29 op/s 2023-09-10T02:35:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:16 smithi170 ceph-mon[102832]: pgmap v6427: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T02:35:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:16 smithi170 ceph-mon[102832]: osdmap e470: 12 total, 12 up, 12 in 2023-09-10T02:35:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:16 smithi082 ceph-mon[98694]: pgmap v6427: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T02:35:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:16 smithi082 ceph-mon[98694]: osdmap e470: 12 total, 12 up, 12 in 2023-09-10T02:35:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:16 smithi098 ceph-mon[103124]: pgmap v6427: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T02:35:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:16 smithi098 ceph-mon[103124]: osdmap e470: 12 total, 12 up, 12 in 2023-09-10T02:35:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11748.800339 secs 2023-09-10T02:35:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11748.800339 secs 2023-09-10T02:35:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11748.800339 secs 2023-09-10T02:35:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:18 smithi170 ceph-mon[102832]: pgmap v6429: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 62 KiB/s wr, 12 op/s 2023-09-10T02:35:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11750.189583 secs 2023-09-10T02:35:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:18 smithi082 ceph-mon[98694]: pgmap v6429: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 62 KiB/s wr, 12 op/s 2023-09-10T02:35:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11750.189583 secs 2023-09-10T02:35:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:18 smithi098 ceph-mon[103124]: pgmap v6429: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 62 KiB/s wr, 12 op/s 2023-09-10T02:35:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11750.189583 secs 2023-09-10T02:35:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11750.901259 secs 2023-09-10T02:35:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11750.901259 secs 2023-09-10T02:35:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11750.901259 secs 2023-09-10T02:35:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:20 smithi170 ceph-mon[102832]: pgmap v6430: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T02:35:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:20 smithi082 ceph-mon[98694]: pgmap v6430: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T02:35:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:20 smithi098 ceph-mon[103124]: pgmap v6430: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T02:35:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:22 smithi170 ceph-mon[102832]: pgmap v6431: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T02:35:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:35:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:22 smithi082 ceph-mon[98694]: pgmap v6431: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T02:35:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:35:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:22 smithi098 ceph-mon[103124]: pgmap v6431: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T02:35:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:35:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:35:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:35:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:35:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11755.189642 secs 2023-09-10T02:35:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:35:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:35:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:35:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11755.189642 secs 2023-09-10T02:35:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:35:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:35:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:35:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11755.189642 secs 2023-09-10T02:35:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11755.901324 secs 2023-09-10T02:35:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:24 smithi170 ceph-mon[102832]: pgmap v6432: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T02:35:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11755.901324 secs 2023-09-10T02:35:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:24 smithi082 ceph-mon[98694]: pgmap v6432: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T02:35:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11755.901324 secs 2023-09-10T02:35:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:24 smithi098 ceph-mon[103124]: pgmap v6432: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T02:35:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:26 smithi170 ceph-mon[102832]: pgmap v6433: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T02:35:27.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:26 smithi082 ceph-mon[98694]: pgmap v6433: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T02:35:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:26 smithi098 ceph-mon[103124]: pgmap v6433: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T02:35:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:28 smithi170 ceph-mon[102832]: pgmap v6434: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 10 op/s 2023-09-10T02:35:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11760.189724 secs 2023-09-10T02:35:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:28 smithi082 ceph-mon[98694]: pgmap v6434: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 10 op/s 2023-09-10T02:35:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11760.189724 secs 2023-09-10T02:35:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:28 smithi098 ceph-mon[103124]: pgmap v6434: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 10 op/s 2023-09-10T02:35:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11760.189724 secs 2023-09-10T02:35:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11760.901390 secs 2023-09-10T02:35:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11760.901390 secs 2023-09-10T02:35:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11760.901390 secs 2023-09-10T02:35:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:30 smithi170 ceph-mon[102832]: pgmap v6435: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 9 op/s 2023-09-10T02:35:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:30 smithi082 ceph-mon[98694]: pgmap v6435: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 9 op/s 2023-09-10T02:35:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:30 smithi098 ceph-mon[103124]: pgmap v6435: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 9 op/s 2023-09-10T02:35:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:32 smithi170 ceph-mon[102832]: pgmap v6436: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 155 KiB/s wr, 15 op/s 2023-09-10T02:35:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:32 smithi082 ceph-mon[98694]: pgmap v6436: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 155 KiB/s wr, 15 op/s 2023-09-10T02:35:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:32 smithi098 ceph-mon[103124]: pgmap v6436: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 155 KiB/s wr, 15 op/s 2023-09-10T02:35:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11765.189783 secs 2023-09-10T02:35:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11765.189783 secs 2023-09-10T02:35:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11765.189783 secs 2023-09-10T02:35:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11765.901455 secs 2023-09-10T02:35:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:34 smithi170 ceph-mon[102832]: pgmap v6437: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T02:35:35.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11765.901455 secs 2023-09-10T02:35:35.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:34 smithi082 ceph-mon[98694]: pgmap v6437: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T02:35:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11765.901455 secs 2023-09-10T02:35:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:34 smithi098 ceph-mon[103124]: pgmap v6437: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T02:35:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:36 smithi170 ceph-mon[102832]: pgmap v6438: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 213 KiB/s wr, 17 op/s 2023-09-10T02:35:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:36 smithi082 ceph-mon[98694]: pgmap v6438: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 213 KiB/s wr, 17 op/s 2023-09-10T02:35:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:36 smithi098 ceph-mon[103124]: pgmap v6438: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 213 KiB/s wr, 17 op/s 2023-09-10T02:35:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:38 smithi170 ceph-mon[102832]: pgmap v6439: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 12 op/s 2023-09-10T02:35:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11770.189842 secs 2023-09-10T02:35:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:38 smithi082 ceph-mon[98694]: pgmap v6439: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 12 op/s 2023-09-10T02:35:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11770.189842 secs 2023-09-10T02:35:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:38 smithi098 ceph-mon[103124]: pgmap v6439: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 12 op/s 2023-09-10T02:35:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11770.189842 secs 2023-09-10T02:35:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11770.901503 secs 2023-09-10T02:35:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11770.901503 secs 2023-09-10T02:35:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11770.901503 secs 2023-09-10T02:35:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:40 smithi170 ceph-mon[102832]: pgmap v6440: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 12 op/s 2023-09-10T02:35:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:40 smithi082 ceph-mon[98694]: pgmap v6440: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 12 op/s 2023-09-10T02:35:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:40 smithi098 ceph-mon[103124]: pgmap v6440: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 12 op/s 2023-09-10T02:35:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:42 smithi170 ceph-mon[102832]: pgmap v6441: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 280 KiB/s wr, 19 op/s 2023-09-10T02:35:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11773.800747 secs 2023-09-10T02:35:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:42 smithi082 ceph-mon[98694]: pgmap v6441: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 280 KiB/s wr, 19 op/s 2023-09-10T02:35:43.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11773.800747 secs 2023-09-10T02:35:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:42 smithi098 ceph-mon[103124]: pgmap v6441: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 280 KiB/s wr, 19 op/s 2023-09-10T02:35:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11773.800747 secs 2023-09-10T02:35:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:35:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:35:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:35:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11775.189900 secs 2023-09-10T02:35:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:35:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:35:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:35:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11775.189900 secs 2023-09-10T02:35:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:35:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:35:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:35:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11775.189900 secs 2023-09-10T02:35:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11775.901567 secs 2023-09-10T02:35:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:44 smithi170 ceph-mon[102832]: pgmap v6442: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 194 KiB/s wr, 13 op/s 2023-09-10T02:35:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11775.901567 secs 2023-09-10T02:35:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:44 smithi082 ceph-mon[98694]: pgmap v6442: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 194 KiB/s wr, 13 op/s 2023-09-10T02:35:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11775.901567 secs 2023-09-10T02:35:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:44 smithi098 ceph-mon[103124]: pgmap v6442: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 194 KiB/s wr, 13 op/s 2023-09-10T02:35:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:46 smithi170 ceph-mon[102832]: pgmap v6443: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 264 KiB/s wr, 20 op/s 2023-09-10T02:35:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:46 smithi082 ceph-mon[98694]: pgmap v6443: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 264 KiB/s wr, 20 op/s 2023-09-10T02:35:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:46 smithi098 ceph-mon[103124]: pgmap v6443: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 264 KiB/s wr, 20 op/s 2023-09-10T02:35:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11778.800777 secs 2023-09-10T02:35:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:35:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11778.800777 secs 2023-09-10T02:35:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:35:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11778.800777 secs 2023-09-10T02:35:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:35:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:48 smithi170 ceph-mon[102832]: pgmap v6444: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 174 KiB/s wr, 14 op/s 2023-09-10T02:35:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11780.189966 secs 2023-09-10T02:35:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:35:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:35:49.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:35:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:48 smithi082 ceph-mon[98694]: pgmap v6444: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 174 KiB/s wr, 14 op/s 2023-09-10T02:35:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11780.189966 secs 2023-09-10T02:35:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:35:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:35:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:35:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:48 smithi098 ceph-mon[103124]: pgmap v6444: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 174 KiB/s wr, 14 op/s 2023-09-10T02:35:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11780.189966 secs 2023-09-10T02:35:49.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:35:49.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:35:49.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:35:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11780.901636 secs 2023-09-10T02:35:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11780.901636 secs 2023-09-10T02:35:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11780.901636 secs 2023-09-10T02:35:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:50 smithi170 ceph-mon[102832]: pgmap v6445: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 174 KiB/s wr, 14 op/s 2023-09-10T02:35:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:50 smithi082 ceph-mon[98694]: pgmap v6445: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 174 KiB/s wr, 14 op/s 2023-09-10T02:35:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:50 smithi098 ceph-mon[103124]: pgmap v6445: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 174 KiB/s wr, 14 op/s 2023-09-10T02:35:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:52 smithi170 ceph-mon[102832]: pgmap v6446: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 206 KiB/s wr, 19 op/s 2023-09-10T02:35:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11783.800840 secs 2023-09-10T02:35:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:52 smithi082 ceph-mon[98694]: pgmap v6446: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 206 KiB/s wr, 19 op/s 2023-09-10T02:35:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11783.800840 secs 2023-09-10T02:35:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:52 smithi098 ceph-mon[103124]: pgmap v6446: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 206 KiB/s wr, 19 op/s 2023-09-10T02:35:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11783.800840 secs 2023-09-10T02:35:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11785.190025 secs 2023-09-10T02:35:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11785.190025 secs 2023-09-10T02:35:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11785.190025 secs 2023-09-10T02:35:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11785.901699 secs 2023-09-10T02:35:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:54 smithi170 ceph-mon[102832]: pgmap v6447: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 102 KiB/s wr, 12 op/s 2023-09-10T02:35:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11785.901699 secs 2023-09-10T02:35:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:54 smithi082 ceph-mon[98694]: pgmap v6447: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 102 KiB/s wr, 12 op/s 2023-09-10T02:35:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11785.901699 secs 2023-09-10T02:35:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:54 smithi098 ceph-mon[103124]: pgmap v6447: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 102 KiB/s wr, 12 op/s 2023-09-10T02:35:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:56 smithi170 ceph-mon[102832]: pgmap v6448: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 141 KiB/s wr, 17 op/s 2023-09-10T02:35:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:56 smithi082 ceph-mon[98694]: pgmap v6448: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 141 KiB/s wr, 17 op/s 2023-09-10T02:35:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:56 smithi098 ceph-mon[103124]: pgmap v6448: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 141 KiB/s wr, 17 op/s 2023-09-10T02:35:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11788.800897 secs 2023-09-10T02:35:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11788.800897 secs 2023-09-10T02:35:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11788.800897 secs 2023-09-10T02:35:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:58 smithi170 ceph-mon[102832]: pgmap v6449: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 72 KiB/s wr, 10 op/s 2023-09-10T02:35:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11790.190123 secs 2023-09-10T02:35:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:58 smithi082 ceph-mon[98694]: pgmap v6449: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 72 KiB/s wr, 10 op/s 2023-09-10T02:35:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11790.190123 secs 2023-09-10T02:35:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:58 smithi098 ceph-mon[103124]: pgmap v6449: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 72 KiB/s wr, 10 op/s 2023-09-10T02:35:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11790.190123 secs 2023-09-10T02:36:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:35:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11790.901766 secs 2023-09-10T02:36:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:35:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11790.901766 secs 2023-09-10T02:36:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:35:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11790.901766 secs 2023-09-10T02:36:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:00 smithi170 ceph-mon[102832]: pgmap v6450: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 72 KiB/s wr, 10 op/s 2023-09-10T02:36:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:00 smithi082 ceph-mon[98694]: pgmap v6450: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 72 KiB/s wr, 10 op/s 2023-09-10T02:36:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:00 smithi098 ceph-mon[103124]: pgmap v6450: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 72 KiB/s wr, 10 op/s 2023-09-10T02:36:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:02 smithi170 ceph-mon[102832]: pgmap v6451: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 90 KiB/s wr, 15 op/s 2023-09-10T02:36:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11793.800957 secs 2023-09-10T02:36:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:02 smithi082 ceph-mon[98694]: pgmap v6451: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 90 KiB/s wr, 15 op/s 2023-09-10T02:36:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11793.800957 secs 2023-09-10T02:36:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:02 smithi098 ceph-mon[103124]: pgmap v6451: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 90 KiB/s wr, 15 op/s 2023-09-10T02:36:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11793.800957 secs 2023-09-10T02:36:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11795.190179 secs 2023-09-10T02:36:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11795.190179 secs 2023-09-10T02:36:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11795.190179 secs 2023-09-10T02:36:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11795.901829 secs 2023-09-10T02:36:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:04 smithi170 ceph-mon[102832]: pgmap v6452: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 10 op/s 2023-09-10T02:36:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11795.901829 secs 2023-09-10T02:36:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:04 smithi082 ceph-mon[98694]: pgmap v6452: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 10 op/s 2023-09-10T02:36:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11795.901829 secs 2023-09-10T02:36:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:04 smithi098 ceph-mon[103124]: pgmap v6452: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 10 op/s 2023-09-10T02:36:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:06 smithi170 ceph-mon[102832]: pgmap v6453: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 104 KiB/s wr, 29 op/s 2023-09-10T02:36:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:06 smithi082 ceph-mon[98694]: pgmap v6453: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 104 KiB/s wr, 29 op/s 2023-09-10T02:36:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:06 smithi098 ceph-mon[103124]: pgmap v6453: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 104 KiB/s wr, 29 op/s 2023-09-10T02:36:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11798.801016 secs 2023-09-10T02:36:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11798.801016 secs 2023-09-10T02:36:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11798.801016 secs 2023-09-10T02:36:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:08 smithi170 ceph-mon[102832]: pgmap v6454: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 64 KiB/s wr, 24 op/s 2023-09-10T02:36:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:08 smithi170 ceph-mon[102832]: osdmap e471: 12 total, 12 up, 12 in 2023-09-10T02:36:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11800.190264 secs 2023-09-10T02:36:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:08 smithi082 ceph-mon[98694]: pgmap v6454: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 64 KiB/s wr, 24 op/s 2023-09-10T02:36:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:08 smithi082 ceph-mon[98694]: osdmap e471: 12 total, 12 up, 12 in 2023-09-10T02:36:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11800.190264 secs 2023-09-10T02:36:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:08 smithi098 ceph-mon[103124]: pgmap v6454: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 64 KiB/s wr, 24 op/s 2023-09-10T02:36:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:08 smithi098 ceph-mon[103124]: osdmap e471: 12 total, 12 up, 12 in 2023-09-10T02:36:09.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11800.190264 secs 2023-09-10T02:36:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11800.901895 secs 2023-09-10T02:36:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11800.901895 secs 2023-09-10T02:36:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11800.901895 secs 2023-09-10T02:36:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:10 smithi170 ceph-mon[102832]: pgmap v6456: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 77 KiB/s wr, 29 op/s 2023-09-10T02:36:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:10 smithi082 ceph-mon[98694]: pgmap v6456: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 77 KiB/s wr, 29 op/s 2023-09-10T02:36:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:10 smithi098 ceph-mon[103124]: pgmap v6456: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 77 KiB/s wr, 29 op/s 2023-09-10T02:36:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:12 smithi170 ceph-mon[102832]: pgmap v6457: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 56 KiB/s wr, 29 op/s 2023-09-10T02:36:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11803.801092 secs 2023-09-10T02:36:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:12 smithi082 ceph-mon[98694]: pgmap v6457: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 56 KiB/s wr, 29 op/s 2023-09-10T02:36:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11803.801092 secs 2023-09-10T02:36:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:12 smithi098 ceph-mon[103124]: pgmap v6457: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 56 KiB/s wr, 29 op/s 2023-09-10T02:36:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11803.801092 secs 2023-09-10T02:36:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11805.190322 secs 2023-09-10T02:36:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11805.190322 secs 2023-09-10T02:36:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11805.190322 secs 2023-09-10T02:36:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11805.901960 secs 2023-09-10T02:36:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:14 smithi170 ceph-mon[102832]: pgmap v6458: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 56 KiB/s wr, 29 op/s 2023-09-10T02:36:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11805.901960 secs 2023-09-10T02:36:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:14 smithi082 ceph-mon[98694]: pgmap v6458: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 56 KiB/s wr, 29 op/s 2023-09-10T02:36:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11805.901960 secs 2023-09-10T02:36:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:14 smithi098 ceph-mon[103124]: pgmap v6458: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 56 KiB/s wr, 29 op/s 2023-09-10T02:36:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:16 smithi170 ceph-mon[102832]: pgmap v6459: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:36:17.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:16 smithi170 ceph-mon[102832]: osdmap e472: 12 total, 12 up, 12 in 2023-09-10T02:36:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:16 smithi082 ceph-mon[98694]: pgmap v6459: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:36:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:16 smithi082 ceph-mon[98694]: osdmap e472: 12 total, 12 up, 12 in 2023-09-10T02:36:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:16 smithi098 ceph-mon[103124]: pgmap v6459: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:36:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:16 smithi098 ceph-mon[103124]: osdmap e472: 12 total, 12 up, 12 in 2023-09-10T02:36:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11808.801151 secs 2023-09-10T02:36:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11808.801151 secs 2023-09-10T02:36:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11808.801151 secs 2023-09-10T02:36:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:18 smithi170 ceph-mon[102832]: pgmap v6461: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:36:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11810.190382 secs 2023-09-10T02:36:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:18 smithi082 ceph-mon[98694]: pgmap v6461: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:36:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11810.190382 secs 2023-09-10T02:36:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:18 smithi098 ceph-mon[103124]: pgmap v6461: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:36:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11810.190382 secs 2023-09-10T02:36:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11810.902024 secs 2023-09-10T02:36:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11810.902024 secs 2023-09-10T02:36:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11810.902024 secs 2023-09-10T02:36:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:20 smithi170 ceph-mon[102832]: pgmap v6462: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:36:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:20 smithi082 ceph-mon[98694]: pgmap v6462: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:36:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:20 smithi098 ceph-mon[103124]: pgmap v6462: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:36:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:22 smithi170 ceph-mon[102832]: pgmap v6463: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T02:36:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:36:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:22 smithi082 ceph-mon[98694]: pgmap v6463: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T02:36:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:36:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:22 smithi098 ceph-mon[103124]: pgmap v6463: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T02:36:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:36:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:36:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:36:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:36:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11815.190438 secs 2023-09-10T02:36:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:36:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:36:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:36:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11815.190438 secs 2023-09-10T02:36:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:36:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:36:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:36:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11815.190438 secs 2023-09-10T02:36:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11815.902091 secs 2023-09-10T02:36:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:24 smithi170 ceph-mon[102832]: pgmap v6464: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T02:36:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11815.902091 secs 2023-09-10T02:36:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:24 smithi082 ceph-mon[98694]: pgmap v6464: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T02:36:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11815.902091 secs 2023-09-10T02:36:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:24 smithi098 ceph-mon[103124]: pgmap v6464: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T02:36:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:26 smithi170 ceph-mon[102832]: pgmap v6465: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T02:36:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:26 smithi082 ceph-mon[98694]: pgmap v6465: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T02:36:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:26 smithi098 ceph-mon[103124]: pgmap v6465: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T02:36:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:28 smithi170 ceph-mon[102832]: pgmap v6466: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T02:36:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11820.190521 secs 2023-09-10T02:36:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:28 smithi082 ceph-mon[98694]: pgmap v6466: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T02:36:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11820.190521 secs 2023-09-10T02:36:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:28 smithi098 ceph-mon[103124]: pgmap v6466: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T02:36:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11820.190521 secs 2023-09-10T02:36:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11820.902157 secs 2023-09-10T02:36:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11820.902157 secs 2023-09-10T02:36:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11820.902157 secs 2023-09-10T02:36:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:30 smithi170 ceph-mon[102832]: pgmap v6467: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T02:36:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:30 smithi082 ceph-mon[98694]: pgmap v6467: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T02:36:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:30 smithi098 ceph-mon[103124]: pgmap v6467: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T02:36:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:32 smithi170 ceph-mon[102832]: pgmap v6468: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 141 KiB/s wr, 17 op/s 2023-09-10T02:36:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:32 smithi082 ceph-mon[98694]: pgmap v6468: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 141 KiB/s wr, 17 op/s 2023-09-10T02:36:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:32 smithi098 ceph-mon[103124]: pgmap v6468: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 141 KiB/s wr, 17 op/s 2023-09-10T02:36:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11825.190580 secs 2023-09-10T02:36:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11825.190580 secs 2023-09-10T02:36:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11825.190580 secs 2023-09-10T02:36:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11825.902221 secs 2023-09-10T02:36:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:34 smithi170 ceph-mon[102832]: pgmap v6469: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T02:36:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11825.902221 secs 2023-09-10T02:36:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:34 smithi082 ceph-mon[98694]: pgmap v6469: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T02:36:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11825.902221 secs 2023-09-10T02:36:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:34 smithi098 ceph-mon[103124]: pgmap v6469: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T02:36:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:36 smithi170 ceph-mon[102832]: pgmap v6470: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 178 KiB/s wr, 18 op/s 2023-09-10T02:36:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:36 smithi082 ceph-mon[98694]: pgmap v6470: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 178 KiB/s wr, 18 op/s 2023-09-10T02:36:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:36 smithi098 ceph-mon[103124]: pgmap v6470: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 178 KiB/s wr, 18 op/s 2023-09-10T02:36:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:38 smithi170 ceph-mon[102832]: pgmap v6471: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T02:36:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11830.190637 secs 2023-09-10T02:36:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:38 smithi082 ceph-mon[98694]: pgmap v6471: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T02:36:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11830.190637 secs 2023-09-10T02:36:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:38 smithi098 ceph-mon[103124]: pgmap v6471: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T02:36:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11830.190637 secs 2023-09-10T02:36:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11830.902291 secs 2023-09-10T02:36:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11830.902291 secs 2023-09-10T02:36:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11830.902291 secs 2023-09-10T02:36:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:40 smithi170 ceph-mon[102832]: pgmap v6472: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T02:36:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:40 smithi082 ceph-mon[98694]: pgmap v6472: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T02:36:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:40 smithi098 ceph-mon[103124]: pgmap v6472: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T02:36:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:42 smithi170 ceph-mon[102832]: pgmap v6473: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 214 KiB/s wr, 19 op/s 2023-09-10T02:36:43.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:42 smithi082 ceph-mon[98694]: pgmap v6473: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 214 KiB/s wr, 19 op/s 2023-09-10T02:36:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:42 smithi098 ceph-mon[103124]: pgmap v6473: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 214 KiB/s wr, 19 op/s 2023-09-10T02:36:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11835.190693 secs 2023-09-10T02:36:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11835.190693 secs 2023-09-10T02:36:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11835.190693 secs 2023-09-10T02:36:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11835.902355 secs 2023-09-10T02:36:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:44 smithi170 ceph-mon[102832]: pgmap v6474: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T02:36:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11835.902355 secs 2023-09-10T02:36:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:44 smithi082 ceph-mon[98694]: pgmap v6474: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T02:36:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11835.902355 secs 2023-09-10T02:36:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:44 smithi098 ceph-mon[103124]: pgmap v6474: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T02:36:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:46 smithi170 ceph-mon[102832]: pgmap v6475: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 217 KiB/s wr, 18 op/s 2023-09-10T02:36:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:46 smithi082 ceph-mon[98694]: pgmap v6475: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 217 KiB/s wr, 18 op/s 2023-09-10T02:36:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:46 smithi098 ceph-mon[103124]: pgmap v6475: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 217 KiB/s wr, 18 op/s 2023-09-10T02:36:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11838.801573 secs 2023-09-10T02:36:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11838.801573 secs 2023-09-10T02:36:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11838.801573 secs 2023-09-10T02:36:49.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:48 smithi170 ceph-mon[102832]: pgmap v6476: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T02:36:49.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11840.190753 secs 2023-09-10T02:36:49.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:36:49.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:36:49.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:36:49.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:36:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:48 smithi082 ceph-mon[98694]: pgmap v6476: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T02:36:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11840.190753 secs 2023-09-10T02:36:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:36:49.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:36:49.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:36:49.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:36:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:48 smithi098 ceph-mon[103124]: pgmap v6476: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T02:36:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11840.190753 secs 2023-09-10T02:36:49.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:36:49.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:36:49.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:36:49.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:36:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11840.902420 secs 2023-09-10T02:36:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11840.902420 secs 2023-09-10T02:36:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11840.902420 secs 2023-09-10T02:36:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:50 smithi170 ceph-mon[102832]: pgmap v6477: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T02:36:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:50 smithi082 ceph-mon[98694]: pgmap v6477: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T02:36:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:50 smithi098 ceph-mon[103124]: pgmap v6477: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T02:36:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:36:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:36:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:36:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:36:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:36:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:36:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:36:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:36:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:36:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:52 smithi170 ceph-mon[102832]: pgmap v6478: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 211 KiB/s wr, 17 op/s 2023-09-10T02:36:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11843.801662 secs 2023-09-10T02:36:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:52 smithi082 ceph-mon[98694]: pgmap v6478: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 211 KiB/s wr, 17 op/s 2023-09-10T02:36:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11843.801662 secs 2023-09-10T02:36:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:52 smithi098 ceph-mon[103124]: pgmap v6478: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 211 KiB/s wr, 17 op/s 2023-09-10T02:36:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11843.801662 secs 2023-09-10T02:36:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11845.190839 secs 2023-09-10T02:36:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11845.190839 secs 2023-09-10T02:36:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11845.190839 secs 2023-09-10T02:36:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11845.902488 secs 2023-09-10T02:36:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:54 smithi170 ceph-mon[102832]: pgmap v6479: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 143 KiB/s wr, 10 op/s 2023-09-10T02:36:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11845.902488 secs 2023-09-10T02:36:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:54 smithi082 ceph-mon[98694]: pgmap v6479: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 143 KiB/s wr, 10 op/s 2023-09-10T02:36:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11845.902488 secs 2023-09-10T02:36:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:54 smithi098 ceph-mon[103124]: pgmap v6479: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 143 KiB/s wr, 10 op/s 2023-09-10T02:36:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:56 smithi170 ceph-mon[102832]: pgmap v6480: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 197 KiB/s wr, 16 op/s 2023-09-10T02:36:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:56 smithi082 ceph-mon[98694]: pgmap v6480: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 197 KiB/s wr, 16 op/s 2023-09-10T02:36:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:56 smithi098 ceph-mon[103124]: pgmap v6480: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 197 KiB/s wr, 16 op/s 2023-09-10T02:36:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11848.801720 secs 2023-09-10T02:36:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11848.801720 secs 2023-09-10T02:36:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11848.801720 secs 2023-09-10T02:36:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:58 smithi170 ceph-mon[102832]: pgmap v6481: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 117 KiB/s wr, 10 op/s 2023-09-10T02:36:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11850.190899 secs 2023-09-10T02:36:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:58 smithi082 ceph-mon[98694]: pgmap v6481: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 117 KiB/s wr, 10 op/s 2023-09-10T02:36:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11850.190899 secs 2023-09-10T02:36:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:58 smithi098 ceph-mon[103124]: pgmap v6481: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 117 KiB/s wr, 10 op/s 2023-09-10T02:36:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11850.190899 secs 2023-09-10T02:37:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:36:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11850.902555 secs 2023-09-10T02:37:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:36:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11850.902555 secs 2023-09-10T02:37:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:36:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11850.902555 secs 2023-09-10T02:37:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:00 smithi170 ceph-mon[102832]: pgmap v6482: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 117 KiB/s wr, 10 op/s 2023-09-10T02:37:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:00 smithi082 ceph-mon[98694]: pgmap v6482: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 117 KiB/s wr, 10 op/s 2023-09-10T02:37:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:00 smithi098 ceph-mon[103124]: pgmap v6482: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 117 KiB/s wr, 10 op/s 2023-09-10T02:37:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:02 smithi170 ceph-mon[102832]: pgmap v6483: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 148 KiB/s wr, 16 op/s 2023-09-10T02:37:03.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11853.801775 secs 2023-09-10T02:37:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:02 smithi082 ceph-mon[98694]: pgmap v6483: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 148 KiB/s wr, 16 op/s 2023-09-10T02:37:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11853.801775 secs 2023-09-10T02:37:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:02 smithi098 ceph-mon[103124]: pgmap v6483: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 148 KiB/s wr, 16 op/s 2023-09-10T02:37:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11853.801775 secs 2023-09-10T02:37:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11855.190957 secs 2023-09-10T02:37:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11855.190957 secs 2023-09-10T02:37:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11855.190957 secs 2023-09-10T02:37:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11855.902621 secs 2023-09-10T02:37:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:04 smithi170 ceph-mon[102832]: pgmap v6484: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T02:37:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11855.902621 secs 2023-09-10T02:37:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:04 smithi082 ceph-mon[98694]: pgmap v6484: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T02:37:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11855.902621 secs 2023-09-10T02:37:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:04 smithi098 ceph-mon[103124]: pgmap v6484: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T02:37:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:06 smithi170 ceph-mon[102832]: pgmap v6485: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 131 KiB/s wr, 30 op/s 2023-09-10T02:37:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:06 smithi082 ceph-mon[98694]: pgmap v6485: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 131 KiB/s wr, 30 op/s 2023-09-10T02:37:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:06 smithi098 ceph-mon[103124]: pgmap v6485: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 131 KiB/s wr, 30 op/s 2023-09-10T02:37:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11858.801835 secs 2023-09-10T02:37:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11858.801835 secs 2023-09-10T02:37:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11858.801835 secs 2023-09-10T02:37:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:08 smithi170 ceph-mon[102832]: pgmap v6486: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 77 KiB/s wr, 25 op/s 2023-09-10T02:37:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:08 smithi170 ceph-mon[102832]: osdmap e473: 12 total, 12 up, 12 in 2023-09-10T02:37:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11860.191046 secs 2023-09-10T02:37:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:08 smithi082 ceph-mon[98694]: pgmap v6486: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 77 KiB/s wr, 25 op/s 2023-09-10T02:37:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:08 smithi082 ceph-mon[98694]: osdmap e473: 12 total, 12 up, 12 in 2023-09-10T02:37:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11860.191046 secs 2023-09-10T02:37:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:08 smithi098 ceph-mon[103124]: pgmap v6486: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 77 KiB/s wr, 25 op/s 2023-09-10T02:37:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:08 smithi098 ceph-mon[103124]: osdmap e473: 12 total, 12 up, 12 in 2023-09-10T02:37:09.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11860.191046 secs 2023-09-10T02:37:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11860.902688 secs 2023-09-10T02:37:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11860.902688 secs 2023-09-10T02:37:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11860.902688 secs 2023-09-10T02:37:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:10 smithi170 ceph-mon[102832]: pgmap v6488: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 93 KiB/s wr, 30 op/s 2023-09-10T02:37:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:10 smithi082 ceph-mon[98694]: pgmap v6488: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 93 KiB/s wr, 30 op/s 2023-09-10T02:37:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:10 smithi098 ceph-mon[103124]: pgmap v6488: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 93 KiB/s wr, 30 op/s 2023-09-10T02:37:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:12 smithi170 ceph-mon[102832]: pgmap v6489: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T02:37:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11863.801926 secs 2023-09-10T02:37:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:12 smithi082 ceph-mon[98694]: pgmap v6489: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T02:37:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11863.801926 secs 2023-09-10T02:37:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:12 smithi098 ceph-mon[103124]: pgmap v6489: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T02:37:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11863.801926 secs 2023-09-10T02:37:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11865.191103 secs 2023-09-10T02:37:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11865.191103 secs 2023-09-10T02:37:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11865.191103 secs 2023-09-10T02:37:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11865.902754 secs 2023-09-10T02:37:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:14 smithi170 ceph-mon[102832]: pgmap v6490: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T02:37:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11865.902754 secs 2023-09-10T02:37:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:14 smithi082 ceph-mon[98694]: pgmap v6490: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T02:37:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11865.902754 secs 2023-09-10T02:37:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:14 smithi098 ceph-mon[103124]: pgmap v6490: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T02:37:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:16 smithi170 ceph-mon[102832]: pgmap v6491: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T02:37:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:16 smithi170 ceph-mon[102832]: osdmap e474: 12 total, 12 up, 12 in 2023-09-10T02:37:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:16 smithi082 ceph-mon[98694]: pgmap v6491: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T02:37:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:16 smithi082 ceph-mon[98694]: osdmap e474: 12 total, 12 up, 12 in 2023-09-10T02:37:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:16 smithi098 ceph-mon[103124]: pgmap v6491: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T02:37:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:16 smithi098 ceph-mon[103124]: osdmap e474: 12 total, 12 up, 12 in 2023-09-10T02:37:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11868.801985 secs 2023-09-10T02:37:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11868.801985 secs 2023-09-10T02:37:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11868.801985 secs 2023-09-10T02:37:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:18 smithi170 ceph-mon[102832]: pgmap v6493: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T02:37:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11870.191188 secs 2023-09-10T02:37:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:18 smithi082 ceph-mon[98694]: pgmap v6493: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T02:37:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11870.191188 secs 2023-09-10T02:37:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:18 smithi098 ceph-mon[103124]: pgmap v6493: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T02:37:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11870.191188 secs 2023-09-10T02:37:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11870.902821 secs 2023-09-10T02:37:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11870.902821 secs 2023-09-10T02:37:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11870.902821 secs 2023-09-10T02:37:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:20 smithi170 ceph-mon[102832]: pgmap v6494: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T02:37:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:20 smithi082 ceph-mon[98694]: pgmap v6494: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T02:37:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:20 smithi098 ceph-mon[103124]: pgmap v6494: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T02:37:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:22 smithi170 ceph-mon[102832]: pgmap v6495: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:37:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:37:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:22 smithi082 ceph-mon[98694]: pgmap v6495: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:37:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:37:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:22 smithi098 ceph-mon[103124]: pgmap v6495: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:37:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:37:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:37:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:37:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:37:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11875.191245 secs 2023-09-10T02:37:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:37:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:37:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:37:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11875.191245 secs 2023-09-10T02:37:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:37:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:37:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:37:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11875.191245 secs 2023-09-10T02:37:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11875.902885 secs 2023-09-10T02:37:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:24 smithi170 ceph-mon[102832]: pgmap v6496: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:37:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11875.902885 secs 2023-09-10T02:37:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:24 smithi082 ceph-mon[98694]: pgmap v6496: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:37:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11875.902885 secs 2023-09-10T02:37:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:24 smithi098 ceph-mon[103124]: pgmap v6496: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:37:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:26 smithi170 ceph-mon[102832]: pgmap v6497: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T02:37:27.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:26 smithi082 ceph-mon[98694]: pgmap v6497: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T02:37:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:26 smithi098 ceph-mon[103124]: pgmap v6497: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T02:37:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:28 smithi170 ceph-mon[102832]: pgmap v6498: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 10 op/s 2023-09-10T02:37:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11880.191328 secs 2023-09-10T02:37:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:28 smithi082 ceph-mon[98694]: pgmap v6498: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 10 op/s 2023-09-10T02:37:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11880.191328 secs 2023-09-10T02:37:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:28 smithi098 ceph-mon[103124]: pgmap v6498: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 10 op/s 2023-09-10T02:37:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11880.191328 secs 2023-09-10T02:37:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11880.902952 secs 2023-09-10T02:37:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11880.902952 secs 2023-09-10T02:37:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11880.902952 secs 2023-09-10T02:37:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:30 smithi170 ceph-mon[102832]: pgmap v6499: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T02:37:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:30 smithi082 ceph-mon[98694]: pgmap v6499: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T02:37:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:30 smithi098 ceph-mon[103124]: pgmap v6499: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T02:37:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:32 smithi170 ceph-mon[102832]: pgmap v6500: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 144 KiB/s wr, 15 op/s 2023-09-10T02:37:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:32 smithi082 ceph-mon[98694]: pgmap v6500: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 144 KiB/s wr, 15 op/s 2023-09-10T02:37:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:32 smithi098 ceph-mon[103124]: pgmap v6500: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 144 KiB/s wr, 15 op/s 2023-09-10T02:37:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11885.191387 secs 2023-09-10T02:37:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11885.191387 secs 2023-09-10T02:37:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11885.191387 secs 2023-09-10T02:37:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11885.903017 secs 2023-09-10T02:37:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:34 smithi170 ceph-mon[102832]: pgmap v6501: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 11 op/s 2023-09-10T02:37:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11885.903017 secs 2023-09-10T02:37:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:34 smithi082 ceph-mon[98694]: pgmap v6501: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 11 op/s 2023-09-10T02:37:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11885.903017 secs 2023-09-10T02:37:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:34 smithi098 ceph-mon[103124]: pgmap v6501: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 11 op/s 2023-09-10T02:37:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:36 smithi170 ceph-mon[102832]: pgmap v6502: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 221 KiB/s wr, 17 op/s 2023-09-10T02:37:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:36 smithi082 ceph-mon[98694]: pgmap v6502: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 221 KiB/s wr, 17 op/s 2023-09-10T02:37:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:36 smithi098 ceph-mon[103124]: pgmap v6502: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 221 KiB/s wr, 17 op/s 2023-09-10T02:37:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:38 smithi170 ceph-mon[102832]: pgmap v6503: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T02:37:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11890.191469 secs 2023-09-10T02:37:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:38 smithi082 ceph-mon[98694]: pgmap v6503: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T02:37:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11890.191469 secs 2023-09-10T02:37:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:38 smithi098 ceph-mon[103124]: pgmap v6503: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T02:37:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11890.191469 secs 2023-09-10T02:37:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11890.903081 secs 2023-09-10T02:37:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11890.903081 secs 2023-09-10T02:37:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11890.903081 secs 2023-09-10T02:37:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:40 smithi170 ceph-mon[102832]: pgmap v6504: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T02:37:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:40 smithi082 ceph-mon[98694]: pgmap v6504: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T02:37:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:40 smithi098 ceph-mon[103124]: pgmap v6504: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T02:37:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:42 smithi170 ceph-mon[102832]: pgmap v6505: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 267 KiB/s wr, 19 op/s 2023-09-10T02:37:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11893.802394 secs 2023-09-10T02:37:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:42 smithi082 ceph-mon[98694]: pgmap v6505: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 267 KiB/s wr, 19 op/s 2023-09-10T02:37:43.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11893.802394 secs 2023-09-10T02:37:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:42 smithi098 ceph-mon[103124]: pgmap v6505: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 267 KiB/s wr, 19 op/s 2023-09-10T02:37:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11893.802394 secs 2023-09-10T02:37:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:37:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:37:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:37:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11895.191493 secs 2023-09-10T02:37:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:37:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:37:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:37:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11895.191493 secs 2023-09-10T02:37:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:37:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:37:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:37:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11895.191493 secs 2023-09-10T02:37:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11895.903146 secs 2023-09-10T02:37:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:44 smithi170 ceph-mon[102832]: pgmap v6506: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 194 KiB/s wr, 13 op/s 2023-09-10T02:37:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11895.903146 secs 2023-09-10T02:37:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:44 smithi082 ceph-mon[98694]: pgmap v6506: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 194 KiB/s wr, 13 op/s 2023-09-10T02:37:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11895.903146 secs 2023-09-10T02:37:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:44 smithi098 ceph-mon[103124]: pgmap v6506: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 194 KiB/s wr, 13 op/s 2023-09-10T02:37:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:46 smithi170 ceph-mon[102832]: pgmap v6507: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 265 KiB/s wr, 19 op/s 2023-09-10T02:37:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:46 smithi082 ceph-mon[98694]: pgmap v6507: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 265 KiB/s wr, 19 op/s 2023-09-10T02:37:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:46 smithi098 ceph-mon[103124]: pgmap v6507: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 265 KiB/s wr, 19 op/s 2023-09-10T02:37:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11898.802461 secs 2023-09-10T02:37:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11898.802461 secs 2023-09-10T02:37:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11898.802461 secs 2023-09-10T02:37:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:48 smithi170 ceph-mon[102832]: pgmap v6508: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 188 KiB/s wr, 13 op/s 2023-09-10T02:37:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11900.191579 secs 2023-09-10T02:37:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:48 smithi082 ceph-mon[98694]: pgmap v6508: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 188 KiB/s wr, 13 op/s 2023-09-10T02:37:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11900.191579 secs 2023-09-10T02:37:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:48 smithi098 ceph-mon[103124]: pgmap v6508: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 188 KiB/s wr, 13 op/s 2023-09-10T02:37:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11900.191579 secs 2023-09-10T02:37:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11900.903209 secs 2023-09-10T02:37:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:37:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:37:50.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:37:50.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:37:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11900.903209 secs 2023-09-10T02:37:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:37:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:37:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:37:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:37:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11900.903209 secs 2023-09-10T02:37:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:37:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:37:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:37:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:37:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:51 smithi170 ceph-mon[102832]: pgmap v6509: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 188 KiB/s wr, 13 op/s 2023-09-10T02:37:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:51 smithi082 ceph-mon[98694]: pgmap v6509: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 188 KiB/s wr, 13 op/s 2023-09-10T02:37:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:51 smithi098 ceph-mon[103124]: pgmap v6509: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 188 KiB/s wr, 13 op/s 2023-09-10T02:37:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:52 smithi170 ceph-mon[102832]: pgmap v6510: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 252 KiB/s wr, 18 op/s 2023-09-10T02:37:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11903.802520 secs 2023-09-10T02:37:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:52 smithi082 ceph-mon[98694]: pgmap v6510: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 252 KiB/s wr, 18 op/s 2023-09-10T02:37:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11903.802520 secs 2023-09-10T02:37:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:52 smithi098 ceph-mon[103124]: pgmap v6510: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 252 KiB/s wr, 18 op/s 2023-09-10T02:37:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11903.802520 secs 2023-09-10T02:37:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11905.191651 secs 2023-09-10T02:37:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11905.191651 secs 2023-09-10T02:37:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11905.191651 secs 2023-09-10T02:37:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11905.903278 secs 2023-09-10T02:37:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:54 smithi170 ceph-mon[102832]: pgmap v6511: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T02:37:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11905.903278 secs 2023-09-10T02:37:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:54 smithi082 ceph-mon[98694]: pgmap v6511: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T02:37:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11905.903278 secs 2023-09-10T02:37:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:54 smithi098 ceph-mon[103124]: pgmap v6511: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T02:37:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:56 smithi170 ceph-mon[102832]: pgmap v6512: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 144 KiB/s wr, 16 op/s 2023-09-10T02:37:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:56 smithi082 ceph-mon[98694]: pgmap v6512: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 144 KiB/s wr, 16 op/s 2023-09-10T02:37:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:56 smithi098 ceph-mon[103124]: pgmap v6512: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 144 KiB/s wr, 16 op/s 2023-09-10T02:37:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11908.802636 secs 2023-09-10T02:37:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11908.802636 secs 2023-09-10T02:37:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11908.802636 secs 2023-09-10T02:37:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:58 smithi170 ceph-mon[102832]: pgmap v6513: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 73 KiB/s wr, 10 op/s 2023-09-10T02:37:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11910.191710 secs 2023-09-10T02:37:59.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:58 smithi082 ceph-mon[98694]: pgmap v6513: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 73 KiB/s wr, 10 op/s 2023-09-10T02:37:59.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11910.191710 secs 2023-09-10T02:37:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:58 smithi098 ceph-mon[103124]: pgmap v6513: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 73 KiB/s wr, 10 op/s 2023-09-10T02:37:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11910.191710 secs 2023-09-10T02:38:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:37:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11910.903365 secs 2023-09-10T02:38:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:37:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11910.903365 secs 2023-09-10T02:38:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:37:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11910.903365 secs 2023-09-10T02:38:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:00 smithi170 ceph-mon[102832]: pgmap v6514: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 73 KiB/s wr, 10 op/s 2023-09-10T02:38:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:00 smithi082 ceph-mon[98694]: pgmap v6514: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 73 KiB/s wr, 10 op/s 2023-09-10T02:38:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:00 smithi098 ceph-mon[103124]: pgmap v6514: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 73 KiB/s wr, 10 op/s 2023-09-10T02:38:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:02 smithi170 ceph-mon[102832]: pgmap v6515: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 140 KiB/s wr, 30 op/s 2023-09-10T02:38:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11913.802713 secs 2023-09-10T02:38:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:02 smithi082 ceph-mon[98694]: pgmap v6515: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 140 KiB/s wr, 30 op/s 2023-09-10T02:38:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11913.802713 secs 2023-09-10T02:38:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:02 smithi098 ceph-mon[103124]: pgmap v6515: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 140 KiB/s wr, 30 op/s 2023-09-10T02:38:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11913.802713 secs 2023-09-10T02:38:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11915.191771 secs 2023-09-10T02:38:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11915.191771 secs 2023-09-10T02:38:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11915.191771 secs 2023-09-10T02:38:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11915.903431 secs 2023-09-10T02:38:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:04 smithi170 ceph-mon[102832]: pgmap v6516: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 24 op/s 2023-09-10T02:38:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11915.903431 secs 2023-09-10T02:38:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:04 smithi082 ceph-mon[98694]: pgmap v6516: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 24 op/s 2023-09-10T02:38:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11915.903431 secs 2023-09-10T02:38:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:04 smithi098 ceph-mon[103124]: pgmap v6516: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 24 op/s 2023-09-10T02:38:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:06 smithi170 ceph-mon[102832]: pgmap v6517: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T02:38:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:06 smithi082 ceph-mon[98694]: pgmap v6517: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T02:38:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:06 smithi098 ceph-mon[103124]: pgmap v6517: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 78 KiB/s wr, 29 op/s 2023-09-10T02:38:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11918.802772 secs 2023-09-10T02:38:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11918.802772 secs 2023-09-10T02:38:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11918.802772 secs 2023-09-10T02:38:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:08 smithi170 ceph-mon[102832]: pgmap v6518: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 68 KiB/s wr, 24 op/s 2023-09-10T02:38:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:08 smithi170 ceph-mon[102832]: osdmap e475: 12 total, 12 up, 12 in 2023-09-10T02:38:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11920.191862 secs 2023-09-10T02:38:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:08 smithi082 ceph-mon[98694]: pgmap v6518: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 68 KiB/s wr, 24 op/s 2023-09-10T02:38:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:08 smithi082 ceph-mon[98694]: osdmap e475: 12 total, 12 up, 12 in 2023-09-10T02:38:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11920.191862 secs 2023-09-10T02:38:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:08 smithi098 ceph-mon[103124]: pgmap v6518: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 68 KiB/s wr, 24 op/s 2023-09-10T02:38:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:08 smithi098 ceph-mon[103124]: osdmap e475: 12 total, 12 up, 12 in 2023-09-10T02:38:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11920.191862 secs 2023-09-10T02:38:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11920.903494 secs 2023-09-10T02:38:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11920.903494 secs 2023-09-10T02:38:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11920.903494 secs 2023-09-10T02:38:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:10 smithi170 ceph-mon[102832]: pgmap v6520: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T02:38:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:10 smithi082 ceph-mon[98694]: pgmap v6520: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T02:38:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:10 smithi098 ceph-mon[103124]: pgmap v6520: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 82 KiB/s wr, 29 op/s 2023-09-10T02:38:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:12 smithi170 ceph-mon[102832]: pgmap v6521: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2023-09-10T02:38:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11923.802831 secs 2023-09-10T02:38:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:12 smithi082 ceph-mon[98694]: pgmap v6521: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2023-09-10T02:38:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11923.802831 secs 2023-09-10T02:38:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:12 smithi098 ceph-mon[103124]: pgmap v6521: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2023-09-10T02:38:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11923.802831 secs 2023-09-10T02:38:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11925.191918 secs 2023-09-10T02:38:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11925.191918 secs 2023-09-10T02:38:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11925.191918 secs 2023-09-10T02:38:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11925.903560 secs 2023-09-10T02:38:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:14 smithi170 ceph-mon[102832]: pgmap v6522: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2023-09-10T02:38:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11925.903560 secs 2023-09-10T02:38:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:14 smithi082 ceph-mon[98694]: pgmap v6522: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2023-09-10T02:38:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11925.903560 secs 2023-09-10T02:38:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:14 smithi098 ceph-mon[103124]: pgmap v6522: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2023-09-10T02:38:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:16 smithi170 ceph-mon[102832]: pgmap v6523: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:38:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:16 smithi170 ceph-mon[102832]: osdmap e476: 12 total, 12 up, 12 in 2023-09-10T02:38:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:16 smithi082 ceph-mon[98694]: pgmap v6523: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:38:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:16 smithi082 ceph-mon[98694]: osdmap e476: 12 total, 12 up, 12 in 2023-09-10T02:38:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:16 smithi098 ceph-mon[103124]: pgmap v6523: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:38:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:16 smithi098 ceph-mon[103124]: osdmap e476: 12 total, 12 up, 12 in 2023-09-10T02:38:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11928.802907 secs 2023-09-10T02:38:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11928.802907 secs 2023-09-10T02:38:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11928.802907 secs 2023-09-10T02:38:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:18 smithi170 ceph-mon[102832]: pgmap v6525: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:38:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11930.191976 secs 2023-09-10T02:38:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:18 smithi082 ceph-mon[98694]: pgmap v6525: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:38:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11930.191976 secs 2023-09-10T02:38:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:18 smithi098 ceph-mon[103124]: pgmap v6525: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:38:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11930.191976 secs 2023-09-10T02:38:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11930.903627 secs 2023-09-10T02:38:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11930.903627 secs 2023-09-10T02:38:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11930.903627 secs 2023-09-10T02:38:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:20 smithi170 ceph-mon[102832]: pgmap v6526: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:38:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:20 smithi082 ceph-mon[98694]: pgmap v6526: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:38:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:20 smithi098 ceph-mon[103124]: pgmap v6526: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:38:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:22 smithi170 ceph-mon[102832]: pgmap v6527: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:38:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:38:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:22 smithi082 ceph-mon[98694]: pgmap v6527: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:38:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:38:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:22 smithi098 ceph-mon[103124]: pgmap v6527: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:38:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:38:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:38:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:38:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:38:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11935.192034 secs 2023-09-10T02:38:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:38:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:38:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:38:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11935.192034 secs 2023-09-10T02:38:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:38:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:38:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:38:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11935.192034 secs 2023-09-10T02:38:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11935.903695 secs 2023-09-10T02:38:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:24 smithi170 ceph-mon[102832]: pgmap v6528: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:38:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11935.903695 secs 2023-09-10T02:38:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:24 smithi082 ceph-mon[98694]: pgmap v6528: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:38:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11935.903695 secs 2023-09-10T02:38:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:24 smithi098 ceph-mon[103124]: pgmap v6528: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T02:38:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:26 smithi170 ceph-mon[102832]: pgmap v6529: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T02:38:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:26 smithi082 ceph-mon[98694]: pgmap v6529: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T02:38:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:26 smithi098 ceph-mon[103124]: pgmap v6529: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T02:38:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:28 smithi170 ceph-mon[102832]: pgmap v6530: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 47 KiB/s wr, 10 op/s 2023-09-10T02:38:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11940.192094 secs 2023-09-10T02:38:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:28 smithi082 ceph-mon[98694]: pgmap v6530: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 47 KiB/s wr, 10 op/s 2023-09-10T02:38:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11940.192094 secs 2023-09-10T02:38:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:28 smithi098 ceph-mon[103124]: pgmap v6530: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 47 KiB/s wr, 10 op/s 2023-09-10T02:38:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11940.192094 secs 2023-09-10T02:38:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11940.903762 secs 2023-09-10T02:38:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11940.903762 secs 2023-09-10T02:38:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11940.903762 secs 2023-09-10T02:38:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:30 smithi170 ceph-mon[102832]: pgmap v6531: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 KiB/s wr, 9 op/s 2023-09-10T02:38:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:30 smithi082 ceph-mon[98694]: pgmap v6531: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 KiB/s wr, 9 op/s 2023-09-10T02:38:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:30 smithi098 ceph-mon[103124]: pgmap v6531: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 KiB/s wr, 9 op/s 2023-09-10T02:38:33.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:32 smithi170 ceph-mon[102832]: pgmap v6532: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 159 KiB/s wr, 17 op/s 2023-09-10T02:38:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:32 smithi082 ceph-mon[98694]: pgmap v6532: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 159 KiB/s wr, 17 op/s 2023-09-10T02:38:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:32 smithi098 ceph-mon[103124]: pgmap v6532: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 159 KiB/s wr, 17 op/s 2023-09-10T02:38:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11945.192176 secs 2023-09-10T02:38:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11945.192176 secs 2023-09-10T02:38:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11945.192176 secs 2023-09-10T02:38:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11945.903828 secs 2023-09-10T02:38:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:34 smithi170 ceph-mon[102832]: pgmap v6533: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T02:38:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11945.903828 secs 2023-09-10T02:38:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:34 smithi082 ceph-mon[98694]: pgmap v6533: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T02:38:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11945.903828 secs 2023-09-10T02:38:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:34 smithi098 ceph-mon[103124]: pgmap v6533: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T02:38:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:36 smithi170 ceph-mon[102832]: pgmap v6534: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 243 KiB/s wr, 17 op/s 2023-09-10T02:38:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:36 smithi082 ceph-mon[98694]: pgmap v6534: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 243 KiB/s wr, 17 op/s 2023-09-10T02:38:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:36 smithi098 ceph-mon[103124]: pgmap v6534: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 243 KiB/s wr, 17 op/s 2023-09-10T02:38:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:38 smithi170 ceph-mon[102832]: pgmap v6535: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 200 KiB/s wr, 12 op/s 2023-09-10T02:38:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11950.192233 secs 2023-09-10T02:38:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:38 smithi082 ceph-mon[98694]: pgmap v6535: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 200 KiB/s wr, 12 op/s 2023-09-10T02:38:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11950.192233 secs 2023-09-10T02:38:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:38 smithi098 ceph-mon[103124]: pgmap v6535: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 200 KiB/s wr, 12 op/s 2023-09-10T02:38:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11950.192233 secs 2023-09-10T02:38:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11950.903894 secs 2023-09-10T02:38:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11950.903894 secs 2023-09-10T02:38:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11950.903894 secs 2023-09-10T02:38:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:40 smithi170 ceph-mon[102832]: pgmap v6536: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 200 KiB/s wr, 12 op/s 2023-09-10T02:38:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:40 smithi082 ceph-mon[98694]: pgmap v6536: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 200 KiB/s wr, 12 op/s 2023-09-10T02:38:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:40 smithi098 ceph-mon[103124]: pgmap v6536: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 200 KiB/s wr, 12 op/s 2023-09-10T02:38:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:42 smithi170 ceph-mon[102832]: pgmap v6537: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 245 KiB/s wr, 18 op/s 2023-09-10T02:38:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:42 smithi082 ceph-mon[98694]: pgmap v6537: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 245 KiB/s wr, 18 op/s 2023-09-10T02:38:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:42 smithi098 ceph-mon[103124]: pgmap v6537: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 245 KiB/s wr, 18 op/s 2023-09-10T02:38:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11955.192316 secs 2023-09-10T02:38:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11955.192316 secs 2023-09-10T02:38:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11955.192316 secs 2023-09-10T02:38:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11955.903959 secs 2023-09-10T02:38:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:44 smithi170 ceph-mon[102832]: pgmap v6538: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 130 KiB/s wr, 11 op/s 2023-09-10T02:38:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11955.903959 secs 2023-09-10T02:38:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:44 smithi082 ceph-mon[98694]: pgmap v6538: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 130 KiB/s wr, 11 op/s 2023-09-10T02:38:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11955.903959 secs 2023-09-10T02:38:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:44 smithi098 ceph-mon[103124]: pgmap v6538: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 130 KiB/s wr, 11 op/s 2023-09-10T02:38:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:46 smithi170 ceph-mon[102832]: pgmap v6539: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 255 KiB/s wr, 17 op/s 2023-09-10T02:38:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:46 smithi082 ceph-mon[98694]: pgmap v6539: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 255 KiB/s wr, 17 op/s 2023-09-10T02:38:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:46 smithi098 ceph-mon[103124]: pgmap v6539: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 255 KiB/s wr, 17 op/s 2023-09-10T02:38:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11958.803396 secs 2023-09-10T02:38:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11958.803396 secs 2023-09-10T02:38:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11958.803396 secs 2023-09-10T02:38:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:48 smithi170 ceph-mon[102832]: pgmap v6540: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 171 KiB/s wr, 11 op/s 2023-09-10T02:38:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11960.192402 secs 2023-09-10T02:38:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:48 smithi082 ceph-mon[98694]: pgmap v6540: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 171 KiB/s wr, 11 op/s 2023-09-10T02:38:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11960.192402 secs 2023-09-10T02:38:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:48 smithi098 ceph-mon[103124]: pgmap v6540: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 171 KiB/s wr, 11 op/s 2023-09-10T02:38:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11960.192402 secs 2023-09-10T02:38:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11960.904024 secs 2023-09-10T02:38:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:38:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11960.904024 secs 2023-09-10T02:38:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:38:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11960.904024 secs 2023-09-10T02:38:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:38:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:50 smithi170 ceph-mon[102832]: pgmap v6541: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 171 KiB/s wr, 11 op/s 2023-09-10T02:38:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:50 smithi082 ceph-mon[98694]: pgmap v6541: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 171 KiB/s wr, 11 op/s 2023-09-10T02:38:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:50 smithi098 ceph-mon[103124]: pgmap v6541: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 171 KiB/s wr, 11 op/s 2023-09-10T02:38:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:38:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:38:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:38:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:38:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:38:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:38:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:38:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:38:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:38:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:53 smithi082 ceph-mon[98694]: pgmap v6542: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 190 KiB/s wr, 16 op/s 2023-09-10T02:38:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11963.803459 secs 2023-09-10T02:38:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:38:53.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:38:53.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:38:53.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:38:53.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:53.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:53.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:53.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:53.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:53.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:53.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:53.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:53.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11965.192457 secs 2023-09-10T02:38:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:53 smithi098 ceph-mon[103124]: pgmap v6542: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 190 KiB/s wr, 16 op/s 2023-09-10T02:38:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11963.803459 secs 2023-09-10T02:38:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:38:53.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:38:53.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:38:53.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:38:53.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:53.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:53.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:53.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:53.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:53.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:53.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:53.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:53.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11965.192457 secs 2023-09-10T02:38:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:53 smithi170 ceph-mon[102832]: pgmap v6542: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 190 KiB/s wr, 16 op/s 2023-09-10T02:38:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11963.803459 secs 2023-09-10T02:38:53.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:38:53.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:38:53.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:38:53.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:38:53.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:53.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:53.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:53.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:53.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:53.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:53.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:53.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:53.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11965.192457 secs 2023-09-10T02:38:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:54 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 170.6M 2023-09-10T02:38:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:54 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 178901708: error parsing value: Value '178901708' is below minimum 939524096 2023-09-10T02:38:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:54 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T02:38:54.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:54 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T02:38:54.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:38:54.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:38:54.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11965.904095 secs 2023-09-10T02:38:54.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:54.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:54.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:54.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:54.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:38:54.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:38:54.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:38:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:54 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 170.6M 2023-09-10T02:38:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:54 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 178901708: error parsing value: Value '178901708' is below minimum 939524096 2023-09-10T02:38:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:54 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T02:38:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:54 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T02:38:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:38:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:38:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11965.904095 secs 2023-09-10T02:38:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:54.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:54.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:54.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:54.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:38:54.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:38:54.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:38:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:54 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 170.6M 2023-09-10T02:38:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:54 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 178901708: error parsing value: Value '178901708' is below minimum 939524096 2023-09-10T02:38:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:54 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T02:38:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:54 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T02:38:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:38:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:38:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11965.904095 secs 2023-09-10T02:38:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:54.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:54.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:54.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:38:54.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:38:54.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:38:54.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:38:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:55 smithi082 ceph-mon[98694]: pgmap v6543: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T02:38:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:55 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T02:38:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:55 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T02:38:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:55 smithi098 ceph-mon[103124]: pgmap v6543: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T02:38:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:55 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T02:38:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:55 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T02:38:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:55 smithi170 ceph-mon[102832]: pgmap v6543: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T02:38:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:55 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T02:38:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:55 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T02:38:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:57 smithi082 ceph-mon[98694]: pgmap v6544: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 224 KiB/s wr, 15 op/s 2023-09-10T02:38:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:57 smithi098 ceph-mon[103124]: pgmap v6544: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 224 KiB/s wr, 15 op/s 2023-09-10T02:38:57.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:57 smithi170 ceph-mon[102832]: pgmap v6544: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 224 KiB/s wr, 15 op/s 2023-09-10T02:38:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11968.803634 secs 2023-09-10T02:38:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11970.192496 secs 2023-09-10T02:38:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11968.803634 secs 2023-09-10T02:38:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11970.192496 secs 2023-09-10T02:38:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11968.803634 secs 2023-09-10T02:38:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11970.192496 secs 2023-09-10T02:38:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:59 smithi082 ceph-mon[98694]: pgmap v6545: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T02:38:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:38:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11970.904162 secs 2023-09-10T02:38:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:59 smithi098 ceph-mon[103124]: pgmap v6545: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T02:38:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:38:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11970.904162 secs 2023-09-10T02:38:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:59 smithi170 ceph-mon[102832]: pgmap v6545: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T02:38:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:38:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11970.904162 secs 2023-09-10T02:39:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:01 smithi082 ceph-mon[98694]: pgmap v6546: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T02:39:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:01 smithi098 ceph-mon[103124]: pgmap v6546: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T02:39:01.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:01 smithi170 ceph-mon[102832]: pgmap v6546: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T02:39:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:02 smithi170 ceph-mon[102832]: pgmap v6547: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 153 KiB/s wr, 20 op/s 2023-09-10T02:39:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11973.803695 secs 2023-09-10T02:39:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:02 smithi082 ceph-mon[98694]: pgmap v6547: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 153 KiB/s wr, 20 op/s 2023-09-10T02:39:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11973.803695 secs 2023-09-10T02:39:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:02 smithi098 ceph-mon[103124]: pgmap v6547: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 153 KiB/s wr, 20 op/s 2023-09-10T02:39:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11973.803695 secs 2023-09-10T02:39:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11975.192553 secs 2023-09-10T02:39:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11975.192553 secs 2023-09-10T02:39:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11975.192553 secs 2023-09-10T02:39:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11975.904227 secs 2023-09-10T02:39:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:04 smithi170 ceph-mon[102832]: pgmap v6548: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 135 KiB/s wr, 15 op/s 2023-09-10T02:39:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11975.904227 secs 2023-09-10T02:39:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:04 smithi082 ceph-mon[98694]: pgmap v6548: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 135 KiB/s wr, 15 op/s 2023-09-10T02:39:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11975.904227 secs 2023-09-10T02:39:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:04 smithi098 ceph-mon[103124]: pgmap v6548: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 135 KiB/s wr, 15 op/s 2023-09-10T02:39:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:06 smithi170 ceph-mon[102832]: pgmap v6549: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 158 KiB/s wr, 30 op/s 2023-09-10T02:39:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:06 smithi082 ceph-mon[98694]: pgmap v6549: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 158 KiB/s wr, 30 op/s 2023-09-10T02:39:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:06 smithi098 ceph-mon[103124]: pgmap v6549: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 158 KiB/s wr, 30 op/s 2023-09-10T02:39:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11978.803754 secs 2023-09-10T02:39:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11978.803754 secs 2023-09-10T02:39:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11978.803754 secs 2023-09-10T02:39:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:08 smithi170 ceph-mon[102832]: pgmap v6550: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 78 KiB/s wr, 25 op/s 2023-09-10T02:39:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:08 smithi170 ceph-mon[102832]: osdmap e477: 12 total, 12 up, 12 in 2023-09-10T02:39:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11980.192616 secs 2023-09-10T02:39:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:08 smithi082 ceph-mon[98694]: pgmap v6550: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 78 KiB/s wr, 25 op/s 2023-09-10T02:39:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:08 smithi082 ceph-mon[98694]: osdmap e477: 12 total, 12 up, 12 in 2023-09-10T02:39:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11980.192616 secs 2023-09-10T02:39:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:08 smithi098 ceph-mon[103124]: pgmap v6550: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 78 KiB/s wr, 25 op/s 2023-09-10T02:39:09.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:08 smithi098 ceph-mon[103124]: osdmap e477: 12 total, 12 up, 12 in 2023-09-10T02:39:09.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11980.192616 secs 2023-09-10T02:39:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11980.904293 secs 2023-09-10T02:39:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11980.904293 secs 2023-09-10T02:39:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11980.904293 secs 2023-09-10T02:39:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:10 smithi170 ceph-mon[102832]: pgmap v6552: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 93 KiB/s wr, 31 op/s 2023-09-10T02:39:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:10 smithi082 ceph-mon[98694]: pgmap v6552: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 93 KiB/s wr, 31 op/s 2023-09-10T02:39:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:10 smithi098 ceph-mon[103124]: pgmap v6552: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 93 KiB/s wr, 31 op/s 2023-09-10T02:39:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:12 smithi170 ceph-mon[102832]: pgmap v6553: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 29 KiB/s wr, 24 op/s 2023-09-10T02:39:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11983.803821 secs 2023-09-10T02:39:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:12 smithi082 ceph-mon[98694]: pgmap v6553: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 29 KiB/s wr, 24 op/s 2023-09-10T02:39:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11983.803821 secs 2023-09-10T02:39:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:12 smithi098 ceph-mon[103124]: pgmap v6553: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 29 KiB/s wr, 24 op/s 2023-09-10T02:39:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11983.803821 secs 2023-09-10T02:39:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11985.192674 secs 2023-09-10T02:39:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11985.192674 secs 2023-09-10T02:39:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11985.192674 secs 2023-09-10T02:39:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11985.904360 secs 2023-09-10T02:39:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:14 smithi170 ceph-mon[102832]: pgmap v6554: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 29 KiB/s wr, 24 op/s 2023-09-10T02:39:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11985.904360 secs 2023-09-10T02:39:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:14 smithi082 ceph-mon[98694]: pgmap v6554: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 29 KiB/s wr, 24 op/s 2023-09-10T02:39:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11985.904360 secs 2023-09-10T02:39:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:14 smithi098 ceph-mon[103124]: pgmap v6554: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 29 KiB/s wr, 24 op/s 2023-09-10T02:39:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:16 smithi170 ceph-mon[102832]: pgmap v6555: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T02:39:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:16 smithi170 ceph-mon[102832]: osdmap e478: 12 total, 12 up, 12 in 2023-09-10T02:39:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:16 smithi082 ceph-mon[98694]: pgmap v6555: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T02:39:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:16 smithi082 ceph-mon[98694]: osdmap e478: 12 total, 12 up, 12 in 2023-09-10T02:39:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:16 smithi098 ceph-mon[103124]: pgmap v6555: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T02:39:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:16 smithi098 ceph-mon[103124]: osdmap e478: 12 total, 12 up, 12 in 2023-09-10T02:39:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 11988.803879 secs 2023-09-10T02:39:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 11988.803879 secs 2023-09-10T02:39:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 11988.803879 secs 2023-09-10T02:39:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:18 smithi170 ceph-mon[102832]: pgmap v6557: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T02:39:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11990.192732 secs 2023-09-10T02:39:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:18 smithi082 ceph-mon[98694]: pgmap v6557: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T02:39:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11990.192732 secs 2023-09-10T02:39:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:18 smithi098 ceph-mon[103124]: pgmap v6557: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T02:39:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11990.192732 secs 2023-09-10T02:39:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11990.904426 secs 2023-09-10T02:39:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11990.904426 secs 2023-09-10T02:39:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11990.904426 secs 2023-09-10T02:39:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:20 smithi170 ceph-mon[102832]: pgmap v6558: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T02:39:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:20 smithi082 ceph-mon[98694]: pgmap v6558: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T02:39:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:20 smithi098 ceph-mon[103124]: pgmap v6558: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T02:39:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:22 smithi170 ceph-mon[102832]: pgmap v6559: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T02:39:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:39:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:22 smithi082 ceph-mon[98694]: pgmap v6559: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T02:39:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:39:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:22 smithi098 ceph-mon[103124]: pgmap v6559: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T02:39:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:39:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:39:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:39:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:39:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11995.192789 secs 2023-09-10T02:39:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:39:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:39:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:39:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11995.192789 secs 2023-09-10T02:39:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:39:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:39:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:39:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11995.192789 secs 2023-09-10T02:39:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 11995.904512 secs 2023-09-10T02:39:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:24 smithi170 ceph-mon[102832]: pgmap v6560: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T02:39:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 11995.904512 secs 2023-09-10T02:39:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:24 smithi082 ceph-mon[98694]: pgmap v6560: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T02:39:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 11995.904512 secs 2023-09-10T02:39:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:24 smithi098 ceph-mon[103124]: pgmap v6560: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T02:39:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:26 smithi170 ceph-mon[102832]: pgmap v6561: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 85 KiB/s wr, 12 op/s 2023-09-10T02:39:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:26 smithi082 ceph-mon[98694]: pgmap v6561: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 85 KiB/s wr, 12 op/s 2023-09-10T02:39:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:26 smithi098 ceph-mon[103124]: pgmap v6561: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 85 KiB/s wr, 12 op/s 2023-09-10T02:39:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:28 smithi170 ceph-mon[102832]: pgmap v6562: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T02:39:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12000.192846 secs 2023-09-10T02:39:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:28 smithi082 ceph-mon[98694]: pgmap v6562: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T02:39:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12000.192846 secs 2023-09-10T02:39:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:28 smithi098 ceph-mon[103124]: pgmap v6562: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T02:39:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12000.192846 secs 2023-09-10T02:39:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12000.904581 secs 2023-09-10T02:39:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12000.904581 secs 2023-09-10T02:39:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12000.904581 secs 2023-09-10T02:39:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:30 smithi170 ceph-mon[102832]: pgmap v6563: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 71 KiB/s wr, 10 op/s 2023-09-10T02:39:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:30 smithi082 ceph-mon[98694]: pgmap v6563: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 71 KiB/s wr, 10 op/s 2023-09-10T02:39:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:30 smithi098 ceph-mon[103124]: pgmap v6563: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 71 KiB/s wr, 10 op/s 2023-09-10T02:39:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:32 smithi170 ceph-mon[102832]: pgmap v6564: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 187 KiB/s wr, 16 op/s 2023-09-10T02:39:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:32 smithi082 ceph-mon[98694]: pgmap v6564: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 187 KiB/s wr, 16 op/s 2023-09-10T02:39:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:32 smithi098 ceph-mon[103124]: pgmap v6564: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 187 KiB/s wr, 16 op/s 2023-09-10T02:39:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12005.192915 secs 2023-09-10T02:39:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12005.192915 secs 2023-09-10T02:39:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12005.192915 secs 2023-09-10T02:39:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12005.904651 secs 2023-09-10T02:39:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:34 smithi170 ceph-mon[102832]: pgmap v6565: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 186 KiB/s wr, 12 op/s 2023-09-10T02:39:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12005.904651 secs 2023-09-10T02:39:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:34 smithi082 ceph-mon[98694]: pgmap v6565: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 186 KiB/s wr, 12 op/s 2023-09-10T02:39:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12005.904651 secs 2023-09-10T02:39:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:34 smithi098 ceph-mon[103124]: pgmap v6565: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 186 KiB/s wr, 12 op/s 2023-09-10T02:39:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:36 smithi170 ceph-mon[102832]: pgmap v6566: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 274 KiB/s wr, 18 op/s 2023-09-10T02:39:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:36 smithi082 ceph-mon[98694]: pgmap v6566: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 274 KiB/s wr, 18 op/s 2023-09-10T02:39:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:36 smithi098 ceph-mon[103124]: pgmap v6566: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 274 KiB/s wr, 18 op/s 2023-09-10T02:39:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:38 smithi170 ceph-mon[102832]: pgmap v6567: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 203 KiB/s wr, 12 op/s 2023-09-10T02:39:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12010.192974 secs 2023-09-10T02:39:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:38 smithi082 ceph-mon[98694]: pgmap v6567: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 203 KiB/s wr, 12 op/s 2023-09-10T02:39:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12010.192974 secs 2023-09-10T02:39:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:38 smithi098 ceph-mon[103124]: pgmap v6567: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 203 KiB/s wr, 12 op/s 2023-09-10T02:39:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12010.192974 secs 2023-09-10T02:39:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12010.904717 secs 2023-09-10T02:39:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12010.904717 secs 2023-09-10T02:39:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12010.904717 secs 2023-09-10T02:39:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:40 smithi170 ceph-mon[102832]: pgmap v6568: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 203 KiB/s wr, 12 op/s 2023-09-10T02:39:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:40 smithi082 ceph-mon[98694]: pgmap v6568: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 203 KiB/s wr, 12 op/s 2023-09-10T02:39:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:40 smithi098 ceph-mon[103124]: pgmap v6568: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 203 KiB/s wr, 12 op/s 2023-09-10T02:39:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:42 smithi170 ceph-mon[102832]: pgmap v6569: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 340 KiB/s wr, 18 op/s 2023-09-10T02:39:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12013.804239 secs 2023-09-10T02:39:43.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:42 smithi082 ceph-mon[98694]: pgmap v6569: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 340 KiB/s wr, 18 op/s 2023-09-10T02:39:43.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12013.804239 secs 2023-09-10T02:39:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:42 smithi098 ceph-mon[103124]: pgmap v6569: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 340 KiB/s wr, 18 op/s 2023-09-10T02:39:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12013.804239 secs 2023-09-10T02:39:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:39:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:39:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:39:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12015.193089 secs 2023-09-10T02:39:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:39:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:39:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:39:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12015.193089 secs 2023-09-10T02:39:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:39:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:39:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:39:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12015.193089 secs 2023-09-10T02:39:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12015.904784 secs 2023-09-10T02:39:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:44 smithi170 ceph-mon[102832]: pgmap v6570: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 225 KiB/s wr, 12 op/s 2023-09-10T02:39:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12015.904784 secs 2023-09-10T02:39:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:44 smithi082 ceph-mon[98694]: pgmap v6570: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 225 KiB/s wr, 12 op/s 2023-09-10T02:39:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12015.904784 secs 2023-09-10T02:39:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:44 smithi098 ceph-mon[103124]: pgmap v6570: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 225 KiB/s wr, 12 op/s 2023-09-10T02:39:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:46 smithi170 ceph-mon[102832]: pgmap v6571: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 259 KiB/s wr, 17 op/s 2023-09-10T02:39:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:46 smithi082 ceph-mon[98694]: pgmap v6571: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 259 KiB/s wr, 17 op/s 2023-09-10T02:39:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:46 smithi098 ceph-mon[103124]: pgmap v6571: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 259 KiB/s wr, 17 op/s 2023-09-10T02:39:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12018.804301 secs 2023-09-10T02:39:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12018.804301 secs 2023-09-10T02:39:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12018.804301 secs 2023-09-10T02:39:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:48 smithi170 ceph-mon[102832]: pgmap v6572: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 171 KiB/s wr, 11 op/s 2023-09-10T02:39:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12020.193147 secs 2023-09-10T02:39:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:48 smithi082 ceph-mon[98694]: pgmap v6572: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 171 KiB/s wr, 11 op/s 2023-09-10T02:39:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12020.193147 secs 2023-09-10T02:39:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:48 smithi098 ceph-mon[103124]: pgmap v6572: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 171 KiB/s wr, 11 op/s 2023-09-10T02:39:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12020.193147 secs 2023-09-10T02:39:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12020.904851 secs 2023-09-10T02:39:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12020.904851 secs 2023-09-10T02:39:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12020.904851 secs 2023-09-10T02:39:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:50 smithi170 ceph-mon[102832]: pgmap v6573: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 171 KiB/s wr, 11 op/s 2023-09-10T02:39:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:50 smithi082 ceph-mon[98694]: pgmap v6573: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 171 KiB/s wr, 11 op/s 2023-09-10T02:39:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:50 smithi098 ceph-mon[103124]: pgmap v6573: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 171 KiB/s wr, 11 op/s 2023-09-10T02:39:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:52 smithi170 ceph-mon[102832]: pgmap v6574: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 240 KiB/s wr, 16 op/s 2023-09-10T02:39:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12023.804362 secs 2023-09-10T02:39:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:52 smithi082 ceph-mon[98694]: pgmap v6574: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 240 KiB/s wr, 16 op/s 2023-09-10T02:39:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12023.804362 secs 2023-09-10T02:39:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:52 smithi098 ceph-mon[103124]: pgmap v6574: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 240 KiB/s wr, 16 op/s 2023-09-10T02:39:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12023.804362 secs 2023-09-10T02:39:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12025.193206 secs 2023-09-10T02:39:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12025.193206 secs 2023-09-10T02:39:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12025.193206 secs 2023-09-10T02:39:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12025.904918 secs 2023-09-10T02:39:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:54 smithi170 ceph-mon[102832]: pgmap v6575: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 9 op/s 2023-09-10T02:39:55.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:39:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12025.904918 secs 2023-09-10T02:39:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:54 smithi082 ceph-mon[98694]: pgmap v6575: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 9 op/s 2023-09-10T02:39:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:39:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12025.904918 secs 2023-09-10T02:39:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:54 smithi098 ceph-mon[103124]: pgmap v6575: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 9 op/s 2023-09-10T02:39:55.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:39:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:39:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:39:56.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:39:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:39:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:39:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:39:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:39:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:39:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:39:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:57 smithi170 ceph-mon[102832]: pgmap v6576: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 135 KiB/s wr, 14 op/s 2023-09-10T02:39:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:57 smithi082 ceph-mon[98694]: pgmap v6576: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 135 KiB/s wr, 14 op/s 2023-09-10T02:39:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:57 smithi098 ceph-mon[103124]: pgmap v6576: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 135 KiB/s wr, 14 op/s 2023-09-10T02:39:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12028.804422 secs 2023-09-10T02:39:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12028.804422 secs 2023-09-10T02:39:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12028.804422 secs 2023-09-10T02:39:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:59 smithi170 ceph-mon[102832]: pgmap v6577: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T02:39:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:39:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12030.193265 secs 2023-09-10T02:39:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:59 smithi082 ceph-mon[98694]: pgmap v6577: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T02:39:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:39:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12030.193265 secs 2023-09-10T02:39:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:59 smithi098 ceph-mon[103124]: pgmap v6577: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T02:39:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:39:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12030.193265 secs 2023-09-10T02:40:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12030.904986 secs 2023-09-10T02:40:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T02:40:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12030.904986 secs 2023-09-10T02:40:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T02:40:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12030.904986 secs 2023-09-10T02:40:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T02:40:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:01 smithi170 ceph-mon[102832]: pgmap v6578: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T02:40:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:01 smithi082 ceph-mon[98694]: pgmap v6578: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T02:40:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:01 smithi098 ceph-mon[103124]: pgmap v6578: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T02:40:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:02 smithi170 ceph-mon[102832]: pgmap v6579: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 130 KiB/s wr, 14 op/s 2023-09-10T02:40:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12033.804481 secs 2023-09-10T02:40:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:02 smithi082 ceph-mon[98694]: pgmap v6579: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 130 KiB/s wr, 14 op/s 2023-09-10T02:40:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12033.804481 secs 2023-09-10T02:40:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:02 smithi098 ceph-mon[103124]: pgmap v6579: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 130 KiB/s wr, 14 op/s 2023-09-10T02:40:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12033.804481 secs 2023-09-10T02:40:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12035.193350 secs 2023-09-10T02:40:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12035.193350 secs 2023-09-10T02:40:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12035.193350 secs 2023-09-10T02:40:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12035.905052 secs 2023-09-10T02:40:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:04 smithi170 ceph-mon[102832]: pgmap v6580: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T02:40:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12035.905052 secs 2023-09-10T02:40:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:04 smithi082 ceph-mon[98694]: pgmap v6580: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T02:40:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12035.905052 secs 2023-09-10T02:40:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:04 smithi098 ceph-mon[103124]: pgmap v6580: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T02:40:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:06 smithi170 ceph-mon[102832]: pgmap v6581: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 106 KiB/s wr, 29 op/s 2023-09-10T02:40:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:06 smithi082 ceph-mon[98694]: pgmap v6581: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 106 KiB/s wr, 29 op/s 2023-09-10T02:40:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:06 smithi098 ceph-mon[103124]: pgmap v6581: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 106 KiB/s wr, 29 op/s 2023-09-10T02:40:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12038.804540 secs 2023-09-10T02:40:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12038.804540 secs 2023-09-10T02:40:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12038.804540 secs 2023-09-10T02:40:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:08 smithi170 ceph-mon[102832]: pgmap v6582: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 74 KiB/s wr, 24 op/s 2023-09-10T02:40:09.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:08 smithi170 ceph-mon[102832]: osdmap e479: 12 total, 12 up, 12 in 2023-09-10T02:40:09.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12040.193408 secs 2023-09-10T02:40:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:08 smithi082 ceph-mon[98694]: pgmap v6582: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 74 KiB/s wr, 24 op/s 2023-09-10T02:40:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:08 smithi082 ceph-mon[98694]: osdmap e479: 12 total, 12 up, 12 in 2023-09-10T02:40:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12040.193408 secs 2023-09-10T02:40:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:08 smithi098 ceph-mon[103124]: pgmap v6582: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 74 KiB/s wr, 24 op/s 2023-09-10T02:40:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:08 smithi098 ceph-mon[103124]: osdmap e479: 12 total, 12 up, 12 in 2023-09-10T02:40:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12040.193408 secs 2023-09-10T02:40:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12040.905118 secs 2023-09-10T02:40:10.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12040.905118 secs 2023-09-10T02:40:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12040.905118 secs 2023-09-10T02:40:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:10 smithi170 ceph-mon[102832]: pgmap v6584: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 89 KiB/s wr, 29 op/s 2023-09-10T02:40:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:10 smithi082 ceph-mon[98694]: pgmap v6584: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 89 KiB/s wr, 29 op/s 2023-09-10T02:40:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:10 smithi098 ceph-mon[103124]: pgmap v6584: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 89 KiB/s wr, 29 op/s 2023-09-10T02:40:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:12 smithi170 ceph-mon[102832]: pgmap v6585: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T02:40:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12043.804602 secs 2023-09-10T02:40:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:12 smithi082 ceph-mon[98694]: pgmap v6585: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T02:40:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12043.804602 secs 2023-09-10T02:40:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:12 smithi098 ceph-mon[103124]: pgmap v6585: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T02:40:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12043.804602 secs 2023-09-10T02:40:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12045.193465 secs 2023-09-10T02:40:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12045.193465 secs 2023-09-10T02:40:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12045.193465 secs 2023-09-10T02:40:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12045.905183 secs 2023-09-10T02:40:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:14 smithi170 ceph-mon[102832]: pgmap v6586: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T02:40:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12045.905183 secs 2023-09-10T02:40:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:14 smithi082 ceph-mon[98694]: pgmap v6586: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T02:40:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12045.905183 secs 2023-09-10T02:40:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:14 smithi098 ceph-mon[103124]: pgmap v6586: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T02:40:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:16 smithi170 ceph-mon[102832]: pgmap v6587: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T02:40:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:16 smithi170 ceph-mon[102832]: osdmap e480: 12 total, 12 up, 12 in 2023-09-10T02:40:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:16 smithi082 ceph-mon[98694]: pgmap v6587: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T02:40:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:16 smithi082 ceph-mon[98694]: osdmap e480: 12 total, 12 up, 12 in 2023-09-10T02:40:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:16 smithi098 ceph-mon[103124]: pgmap v6587: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T02:40:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:16 smithi098 ceph-mon[103124]: osdmap e480: 12 total, 12 up, 12 in 2023-09-10T02:40:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12048.804662 secs 2023-09-10T02:40:18.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12048.804662 secs 2023-09-10T02:40:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12048.804662 secs 2023-09-10T02:40:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:18 smithi170 ceph-mon[102832]: pgmap v6589: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T02:40:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12050.193525 secs 2023-09-10T02:40:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:18 smithi082 ceph-mon[98694]: pgmap v6589: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T02:40:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12050.193525 secs 2023-09-10T02:40:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:18 smithi098 ceph-mon[103124]: pgmap v6589: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T02:40:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12050.193525 secs 2023-09-10T02:40:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12050.905249 secs 2023-09-10T02:40:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12050.905249 secs 2023-09-10T02:40:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12050.905249 secs 2023-09-10T02:40:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:20 smithi170 ceph-mon[102832]: pgmap v6590: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T02:40:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:20 smithi082 ceph-mon[98694]: pgmap v6590: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T02:40:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:20 smithi098 ceph-mon[103124]: pgmap v6590: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 11 op/s 2023-09-10T02:40:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:22 smithi170 ceph-mon[102832]: pgmap v6591: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T02:40:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:40:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:22 smithi082 ceph-mon[98694]: pgmap v6591: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T02:40:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:40:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:22 smithi098 ceph-mon[103124]: pgmap v6591: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T02:40:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:40:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:40:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:40:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:40:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12055.193585 secs 2023-09-10T02:40:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:40:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:40:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:40:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12055.193585 secs 2023-09-10T02:40:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:40:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:40:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:40:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12055.193585 secs 2023-09-10T02:40:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12055.905313 secs 2023-09-10T02:40:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:24 smithi170 ceph-mon[102832]: pgmap v6592: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T02:40:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12055.905313 secs 2023-09-10T02:40:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:24 smithi082 ceph-mon[98694]: pgmap v6592: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T02:40:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12055.905313 secs 2023-09-10T02:40:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:24 smithi098 ceph-mon[103124]: pgmap v6592: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T02:40:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:26 smithi170 ceph-mon[102832]: pgmap v6593: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-10T02:40:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:26 smithi082 ceph-mon[98694]: pgmap v6593: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-10T02:40:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:26 smithi098 ceph-mon[103124]: pgmap v6593: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-10T02:40:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:28 smithi170 ceph-mon[102832]: pgmap v6594: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T02:40:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12060.193644 secs 2023-09-10T02:40:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:28 smithi082 ceph-mon[98694]: pgmap v6594: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T02:40:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12060.193644 secs 2023-09-10T02:40:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:28 smithi098 ceph-mon[103124]: pgmap v6594: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T02:40:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12060.193644 secs 2023-09-10T02:40:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12060.905378 secs 2023-09-10T02:40:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12060.905378 secs 2023-09-10T02:40:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12060.905378 secs 2023-09-10T02:40:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:30 smithi170 ceph-mon[102832]: pgmap v6595: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 69 KiB/s wr, 10 op/s 2023-09-10T02:40:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:30 smithi082 ceph-mon[98694]: pgmap v6595: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 69 KiB/s wr, 10 op/s 2023-09-10T02:40:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:30 smithi098 ceph-mon[103124]: pgmap v6595: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 69 KiB/s wr, 10 op/s 2023-09-10T02:40:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:32 smithi170 ceph-mon[102832]: pgmap v6596: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 139 KiB/s wr, 15 op/s 2023-09-10T02:40:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:32 smithi082 ceph-mon[98694]: pgmap v6596: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 139 KiB/s wr, 15 op/s 2023-09-10T02:40:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:32 smithi098 ceph-mon[103124]: pgmap v6596: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 139 KiB/s wr, 15 op/s 2023-09-10T02:40:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12065.193746 secs 2023-09-10T02:40:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12065.193746 secs 2023-09-10T02:40:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12065.193746 secs 2023-09-10T02:40:35.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12065.905442 secs 2023-09-10T02:40:35.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:34 smithi098 ceph-mon[103124]: pgmap v6597: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T02:40:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12065.905442 secs 2023-09-10T02:40:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:34 smithi170 ceph-mon[102832]: pgmap v6597: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T02:40:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12065.905442 secs 2023-09-10T02:40:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:34 smithi082 ceph-mon[98694]: pgmap v6597: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T02:40:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:36 smithi170 ceph-mon[102832]: pgmap v6598: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 125 KiB/s wr, 17 op/s 2023-09-10T02:40:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:36 smithi082 ceph-mon[98694]: pgmap v6598: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 125 KiB/s wr, 17 op/s 2023-09-10T02:40:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:36 smithi098 ceph-mon[103124]: pgmap v6598: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 125 KiB/s wr, 17 op/s 2023-09-10T02:40:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:38 smithi170 ceph-mon[102832]: pgmap v6599: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T02:40:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12070.193822 secs 2023-09-10T02:40:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:38 smithi082 ceph-mon[98694]: pgmap v6599: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T02:40:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12070.193822 secs 2023-09-10T02:40:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:38 smithi098 ceph-mon[103124]: pgmap v6599: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T02:40:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12070.193822 secs 2023-09-10T02:40:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12070.905487 secs 2023-09-10T02:40:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12070.905487 secs 2023-09-10T02:40:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12070.905487 secs 2023-09-10T02:40:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:40 smithi170 ceph-mon[102832]: pgmap v6600: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T02:40:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:40 smithi082 ceph-mon[98694]: pgmap v6600: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T02:40:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:40 smithi098 ceph-mon[103124]: pgmap v6600: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T02:40:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:42 smithi170 ceph-mon[102832]: pgmap v6601: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 221 KiB/s wr, 18 op/s 2023-09-10T02:40:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:42 smithi082 ceph-mon[98694]: pgmap v6601: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 221 KiB/s wr, 18 op/s 2023-09-10T02:40:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:42 smithi098 ceph-mon[103124]: pgmap v6601: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 221 KiB/s wr, 18 op/s 2023-09-10T02:40:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12075.193879 secs 2023-09-10T02:40:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12075.193879 secs 2023-09-10T02:40:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12075.193879 secs 2023-09-10T02:40:45.158 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12075.905553 secs 2023-09-10T02:40:45.158 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:44 smithi170 ceph-mon[102832]: pgmap v6602: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T02:40:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12075.905553 secs 2023-09-10T02:40:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:44 smithi082 ceph-mon[98694]: pgmap v6602: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T02:40:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12075.905553 secs 2023-09-10T02:40:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:44 smithi098 ceph-mon[103124]: pgmap v6602: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T02:40:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:46 smithi170 ceph-mon[102832]: pgmap v6603: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 198 KiB/s wr, 17 op/s 2023-09-10T02:40:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:46 smithi082 ceph-mon[98694]: pgmap v6603: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 198 KiB/s wr, 17 op/s 2023-09-10T02:40:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:46 smithi098 ceph-mon[103124]: pgmap v6603: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 198 KiB/s wr, 17 op/s 2023-09-10T02:40:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:48 smithi170 ceph-mon[102832]: pgmap v6604: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 176 KiB/s wr, 11 op/s 2023-09-10T02:40:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12080.193935 secs 2023-09-10T02:40:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:48 smithi082 ceph-mon[98694]: pgmap v6604: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 176 KiB/s wr, 11 op/s 2023-09-10T02:40:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12080.193935 secs 2023-09-10T02:40:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:48 smithi098 ceph-mon[103124]: pgmap v6604: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 176 KiB/s wr, 11 op/s 2023-09-10T02:40:49.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12080.193935 secs 2023-09-10T02:40:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12080.905615 secs 2023-09-10T02:40:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12080.905615 secs 2023-09-10T02:40:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12080.905615 secs 2023-09-10T02:40:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:50 smithi170 ceph-mon[102832]: pgmap v6605: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 176 KiB/s wr, 11 op/s 2023-09-10T02:40:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:50 smithi082 ceph-mon[98694]: pgmap v6605: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 176 KiB/s wr, 11 op/s 2023-09-10T02:40:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:50 smithi098 ceph-mon[103124]: pgmap v6605: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 176 KiB/s wr, 11 op/s 2023-09-10T02:40:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:52 smithi170 ceph-mon[102832]: pgmap v6606: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 340 KiB/s wr, 17 op/s 2023-09-10T02:40:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12083.805106 secs 2023-09-10T02:40:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:52 smithi082 ceph-mon[98694]: pgmap v6606: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 340 KiB/s wr, 17 op/s 2023-09-10T02:40:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12083.805106 secs 2023-09-10T02:40:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:52 smithi098 ceph-mon[103124]: pgmap v6606: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 340 KiB/s wr, 17 op/s 2023-09-10T02:40:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12083.805106 secs 2023-09-10T02:40:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12085.193992 secs 2023-09-10T02:40:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12085.193992 secs 2023-09-10T02:40:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12085.193992 secs 2023-09-10T02:40:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12085.905679 secs 2023-09-10T02:40:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:54 smithi170 ceph-mon[102832]: pgmap v6607: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 212 KiB/s wr, 11 op/s 2023-09-10T02:40:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12085.905679 secs 2023-09-10T02:40:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:54 smithi082 ceph-mon[98694]: pgmap v6607: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 212 KiB/s wr, 11 op/s 2023-09-10T02:40:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12085.905679 secs 2023-09-10T02:40:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:54 smithi098 ceph-mon[103124]: pgmap v6607: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 212 KiB/s wr, 11 op/s 2023-09-10T02:40:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:40:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:40:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:40:56.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:40:56.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:40:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:40:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:40:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:40:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:40:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:40:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:40:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:40:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:40:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:40:56.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:40:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:57 smithi170 ceph-mon[102832]: pgmap v6608: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 244 KiB/s wr, 16 op/s 2023-09-10T02:40:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:40:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:40:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:40:57.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:40:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:57 smithi082 ceph-mon[98694]: pgmap v6608: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 244 KiB/s wr, 16 op/s 2023-09-10T02:40:57.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:40:57.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:40:57.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:40:57.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:40:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:57 smithi098 ceph-mon[103124]: pgmap v6608: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 244 KiB/s wr, 16 op/s 2023-09-10T02:40:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:40:57.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:40:57.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:40:57.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:40:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12088.805168 secs 2023-09-10T02:40:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12088.805168 secs 2023-09-10T02:40:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12088.805168 secs 2023-09-10T02:40:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:59 smithi170 ceph-mon[102832]: pgmap v6609: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 11 op/s 2023-09-10T02:40:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:40:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12090.194064 secs 2023-09-10T02:40:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:59 smithi082 ceph-mon[98694]: pgmap v6609: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 11 op/s 2023-09-10T02:40:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:40:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12090.194064 secs 2023-09-10T02:40:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:59 smithi098 ceph-mon[103124]: pgmap v6609: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 11 op/s 2023-09-10T02:40:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:40:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12090.194064 secs 2023-09-10T02:41:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12090.905744 secs 2023-09-10T02:41:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12090.905744 secs 2023-09-10T02:41:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12090.905744 secs 2023-09-10T02:41:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:01 smithi170 ceph-mon[102832]: pgmap v6610: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 11 op/s 2023-09-10T02:41:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:01 smithi082 ceph-mon[98694]: pgmap v6610: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 11 op/s 2023-09-10T02:41:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:01 smithi098 ceph-mon[103124]: pgmap v6610: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 11 op/s 2023-09-10T02:41:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:02 smithi170 ceph-mon[102832]: pgmap v6611: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 269 KiB/s wr, 16 op/s 2023-09-10T02:41:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12093.805294 secs 2023-09-10T02:41:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:02 smithi082 ceph-mon[98694]: pgmap v6611: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 269 KiB/s wr, 16 op/s 2023-09-10T02:41:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12093.805294 secs 2023-09-10T02:41:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:02 smithi098 ceph-mon[103124]: pgmap v6611: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 269 KiB/s wr, 16 op/s 2023-09-10T02:41:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12093.805294 secs 2023-09-10T02:41:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12095.194147 secs 2023-09-10T02:41:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12095.194147 secs 2023-09-10T02:41:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12095.194147 secs 2023-09-10T02:41:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12095.905807 secs 2023-09-10T02:41:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:04 smithi170 ceph-mon[102832]: pgmap v6612: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T02:41:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12095.905807 secs 2023-09-10T02:41:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:04 smithi082 ceph-mon[98694]: pgmap v6612: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T02:41:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12095.905807 secs 2023-09-10T02:41:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:04 smithi098 ceph-mon[103124]: pgmap v6612: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T02:41:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:06 smithi170 ceph-mon[102832]: pgmap v6613: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 173 KiB/s wr, 30 op/s 2023-09-10T02:41:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:06 smithi082 ceph-mon[98694]: pgmap v6613: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 173 KiB/s wr, 30 op/s 2023-09-10T02:41:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:06 smithi098 ceph-mon[103124]: pgmap v6613: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 173 KiB/s wr, 30 op/s 2023-09-10T02:41:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12098.805387 secs 2023-09-10T02:41:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12098.805387 secs 2023-09-10T02:41:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12098.805387 secs 2023-09-10T02:41:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:08 smithi170 ceph-mon[102832]: pgmap v6614: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 141 KiB/s wr, 24 op/s 2023-09-10T02:41:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:08 smithi170 ceph-mon[102832]: osdmap e481: 12 total, 12 up, 12 in 2023-09-10T02:41:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12100.194392 secs 2023-09-10T02:41:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:08 smithi082 ceph-mon[98694]: pgmap v6614: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 141 KiB/s wr, 24 op/s 2023-09-10T02:41:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:08 smithi082 ceph-mon[98694]: osdmap e481: 12 total, 12 up, 12 in 2023-09-10T02:41:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12100.194392 secs 2023-09-10T02:41:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:08 smithi098 ceph-mon[103124]: pgmap v6614: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 141 KiB/s wr, 24 op/s 2023-09-10T02:41:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:08 smithi098 ceph-mon[103124]: osdmap e481: 12 total, 12 up, 12 in 2023-09-10T02:41:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12100.194392 secs 2023-09-10T02:41:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12100.905873 secs 2023-09-10T02:41:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12100.905873 secs 2023-09-10T02:41:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12100.905873 secs 2023-09-10T02:41:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:10 smithi082 ceph-mon[98694]: pgmap v6616: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 169 KiB/s wr, 29 op/s 2023-09-10T02:41:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:10 smithi098 ceph-mon[103124]: pgmap v6616: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 169 KiB/s wr, 29 op/s 2023-09-10T02:41:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:10 smithi170 ceph-mon[102832]: pgmap v6616: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 169 KiB/s wr, 29 op/s 2023-09-10T02:41:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:12 smithi170 ceph-mon[102832]: pgmap v6617: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 94 KiB/s wr, 30 op/s 2023-09-10T02:41:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12103.805463 secs 2023-09-10T02:41:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:12 smithi082 ceph-mon[98694]: pgmap v6617: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 94 KiB/s wr, 30 op/s 2023-09-10T02:41:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12103.805463 secs 2023-09-10T02:41:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:12 smithi098 ceph-mon[103124]: pgmap v6617: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 94 KiB/s wr, 30 op/s 2023-09-10T02:41:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12103.805463 secs 2023-09-10T02:41:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12105.194475 secs 2023-09-10T02:41:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12105.194475 secs 2023-09-10T02:41:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12105.194475 secs 2023-09-10T02:41:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12105.905939 secs 2023-09-10T02:41:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:14 smithi170 ceph-mon[102832]: pgmap v6618: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 94 KiB/s wr, 30 op/s 2023-09-10T02:41:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12105.905939 secs 2023-09-10T02:41:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:14 smithi082 ceph-mon[98694]: pgmap v6618: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 94 KiB/s wr, 30 op/s 2023-09-10T02:41:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12105.905939 secs 2023-09-10T02:41:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:14 smithi098 ceph-mon[103124]: pgmap v6618: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 94 KiB/s wr, 30 op/s 2023-09-10T02:41:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:16 smithi170 ceph-mon[102832]: pgmap v6619: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 13 op/s 2023-09-10T02:41:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:16 smithi170 ceph-mon[102832]: osdmap e482: 12 total, 12 up, 12 in 2023-09-10T02:41:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:16 smithi082 ceph-mon[98694]: pgmap v6619: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 13 op/s 2023-09-10T02:41:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:16 smithi082 ceph-mon[98694]: osdmap e482: 12 total, 12 up, 12 in 2023-09-10T02:41:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:16 smithi098 ceph-mon[103124]: pgmap v6619: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 13 op/s 2023-09-10T02:41:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:16 smithi098 ceph-mon[103124]: osdmap e482: 12 total, 12 up, 12 in 2023-09-10T02:41:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12108.805523 secs 2023-09-10T02:41:18.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12108.805523 secs 2023-09-10T02:41:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12108.805523 secs 2023-09-10T02:41:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:18 smithi170 ceph-mon[102832]: pgmap v6621: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 13 op/s 2023-09-10T02:41:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12110.194500 secs 2023-09-10T02:41:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:18 smithi082 ceph-mon[98694]: pgmap v6621: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 13 op/s 2023-09-10T02:41:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12110.194500 secs 2023-09-10T02:41:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:18 smithi098 ceph-mon[103124]: pgmap v6621: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 13 op/s 2023-09-10T02:41:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12110.194500 secs 2023-09-10T02:41:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12110.906009 secs 2023-09-10T02:41:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12110.906009 secs 2023-09-10T02:41:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12110.906009 secs 2023-09-10T02:41:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:20 smithi082 ceph-mon[98694]: pgmap v6622: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 13 op/s 2023-09-10T02:41:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:20 smithi098 ceph-mon[103124]: pgmap v6622: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 13 op/s 2023-09-10T02:41:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:20 smithi170 ceph-mon[102832]: pgmap v6622: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 13 op/s 2023-09-10T02:41:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:22 smithi170 ceph-mon[102832]: pgmap v6623: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 12 op/s 2023-09-10T02:41:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12113.805586 secs 2023-09-10T02:41:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:22 smithi082 ceph-mon[98694]: pgmap v6623: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 12 op/s 2023-09-10T02:41:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12113.805586 secs 2023-09-10T02:41:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:22 smithi098 ceph-mon[103124]: pgmap v6623: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 12 op/s 2023-09-10T02:41:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12113.805586 secs 2023-09-10T02:41:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12115.194559 secs 2023-09-10T02:41:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12115.194559 secs 2023-09-10T02:41:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12115.194559 secs 2023-09-10T02:41:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12115.906077 secs 2023-09-10T02:41:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:24 smithi170 ceph-mon[102832]: pgmap v6624: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 12 op/s 2023-09-10T02:41:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12115.906077 secs 2023-09-10T02:41:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:24 smithi082 ceph-mon[98694]: pgmap v6624: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 12 op/s 2023-09-10T02:41:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12115.906077 secs 2023-09-10T02:41:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:24 smithi098 ceph-mon[103124]: pgmap v6624: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 12 op/s 2023-09-10T02:41:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:26 smithi170 ceph-mon[102832]: pgmap v6625: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 195 KiB/s wr, 12 op/s 2023-09-10T02:41:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:26 smithi082 ceph-mon[98694]: pgmap v6625: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 195 KiB/s wr, 12 op/s 2023-09-10T02:41:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:26 smithi098 ceph-mon[103124]: pgmap v6625: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 195 KiB/s wr, 12 op/s 2023-09-10T02:41:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:28 smithi170 ceph-mon[102832]: pgmap v6626: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T02:41:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12120.194643 secs 2023-09-10T02:41:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:28 smithi082 ceph-mon[98694]: pgmap v6626: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T02:41:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12120.194643 secs 2023-09-10T02:41:29.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:28 smithi098 ceph-mon[103124]: pgmap v6626: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T02:41:29.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12120.194643 secs 2023-09-10T02:41:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12120.906142 secs 2023-09-10T02:41:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12120.906142 secs 2023-09-10T02:41:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12120.906142 secs 2023-09-10T02:41:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:30 smithi170 ceph-mon[102832]: pgmap v6627: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 163 KiB/s wr, 10 op/s 2023-09-10T02:41:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:30 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:41:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:30 smithi082 ceph-mon[98694]: pgmap v6627: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 163 KiB/s wr, 10 op/s 2023-09-10T02:41:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:30 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:41:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:30 smithi098 ceph-mon[103124]: pgmap v6627: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 163 KiB/s wr, 10 op/s 2023-09-10T02:41:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:30 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:41:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:31 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:41:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:41:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:41:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:31 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:41:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:41:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:41:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:31 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:41:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:41:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:41:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:32 smithi170 ceph-mon[102832]: pgmap v6628: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 252 KiB/s wr, 17 op/s 2023-09-10T02:41:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:32 smithi082 ceph-mon[98694]: pgmap v6628: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 252 KiB/s wr, 17 op/s 2023-09-10T02:41:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:32 smithi098 ceph-mon[103124]: pgmap v6628: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 252 KiB/s wr, 17 op/s 2023-09-10T02:41:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12125.194701 secs 2023-09-10T02:41:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12125.194701 secs 2023-09-10T02:41:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12125.194701 secs 2023-09-10T02:41:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12125.906208 secs 2023-09-10T02:41:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:34 smithi170 ceph-mon[102832]: pgmap v6629: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 217 KiB/s wr, 12 op/s 2023-09-10T02:41:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12125.906208 secs 2023-09-10T02:41:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:34 smithi082 ceph-mon[98694]: pgmap v6629: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 217 KiB/s wr, 12 op/s 2023-09-10T02:41:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12125.906208 secs 2023-09-10T02:41:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:34 smithi098 ceph-mon[103124]: pgmap v6629: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 217 KiB/s wr, 12 op/s 2023-09-10T02:41:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:36 smithi170 ceph-mon[102832]: pgmap v6630: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 298 KiB/s wr, 17 op/s 2023-09-10T02:41:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:36 smithi082 ceph-mon[98694]: pgmap v6630: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 298 KiB/s wr, 17 op/s 2023-09-10T02:41:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:36 smithi098 ceph-mon[103124]: pgmap v6630: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 298 KiB/s wr, 17 op/s 2023-09-10T02:41:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:38 smithi170 ceph-mon[102832]: pgmap v6631: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 170 KiB/s wr, 12 op/s 2023-09-10T02:41:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12130.194760 secs 2023-09-10T02:41:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:38 smithi082 ceph-mon[98694]: pgmap v6631: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 170 KiB/s wr, 12 op/s 2023-09-10T02:41:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12130.194760 secs 2023-09-10T02:41:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:38 smithi098 ceph-mon[103124]: pgmap v6631: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 170 KiB/s wr, 12 op/s 2023-09-10T02:41:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12130.194760 secs 2023-09-10T02:41:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12130.906275 secs 2023-09-10T02:41:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12130.906275 secs 2023-09-10T02:41:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12130.906275 secs 2023-09-10T02:41:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:40 smithi170 ceph-mon[102832]: pgmap v6632: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 170 KiB/s wr, 12 op/s 2023-09-10T02:41:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:40 smithi082 ceph-mon[98694]: pgmap v6632: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 170 KiB/s wr, 12 op/s 2023-09-10T02:41:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:40 smithi098 ceph-mon[103124]: pgmap v6632: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 170 KiB/s wr, 12 op/s 2023-09-10T02:41:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:42 smithi170 ceph-mon[102832]: pgmap v6633: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 300 KiB/s wr, 18 op/s 2023-09-10T02:41:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:42 smithi082 ceph-mon[98694]: pgmap v6633: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 300 KiB/s wr, 18 op/s 2023-09-10T02:41:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:42 smithi098 ceph-mon[103124]: pgmap v6633: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 300 KiB/s wr, 18 op/s 2023-09-10T02:41:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12135.194818 secs 2023-09-10T02:41:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12135.194818 secs 2023-09-10T02:41:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12135.194818 secs 2023-09-10T02:41:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12135.906340 secs 2023-09-10T02:41:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:44 smithi170 ceph-mon[102832]: pgmap v6634: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 212 KiB/s wr, 11 op/s 2023-09-10T02:41:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12135.906340 secs 2023-09-10T02:41:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:44 smithi082 ceph-mon[98694]: pgmap v6634: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 212 KiB/s wr, 11 op/s 2023-09-10T02:41:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12135.906340 secs 2023-09-10T02:41:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:44 smithi098 ceph-mon[103124]: pgmap v6634: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 212 KiB/s wr, 11 op/s 2023-09-10T02:41:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:46 smithi170 ceph-mon[102832]: pgmap v6635: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 266 KiB/s wr, 17 op/s 2023-09-10T02:41:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:46 smithi082 ceph-mon[98694]: pgmap v6635: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 266 KiB/s wr, 17 op/s 2023-09-10T02:41:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:46 smithi098 ceph-mon[103124]: pgmap v6635: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 266 KiB/s wr, 17 op/s 2023-09-10T02:41:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12138.805912 secs 2023-09-10T02:41:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12138.805912 secs 2023-09-10T02:41:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12138.805912 secs 2023-09-10T02:41:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:48 smithi082 ceph-mon[98694]: pgmap v6636: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T02:41:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12140.194880 secs 2023-09-10T02:41:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:48 smithi098 ceph-mon[103124]: pgmap v6636: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T02:41:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12140.194880 secs 2023-09-10T02:41:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:48 smithi170 ceph-mon[102832]: pgmap v6636: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T02:41:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12140.194880 secs 2023-09-10T02:41:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12140.906428 secs 2023-09-10T02:41:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12140.906428 secs 2023-09-10T02:41:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12140.906428 secs 2023-09-10T02:41:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:50 smithi082 ceph-mon[98694]: pgmap v6637: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T02:41:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:50 smithi098 ceph-mon[103124]: pgmap v6637: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T02:41:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:50 smithi170 ceph-mon[102832]: pgmap v6637: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T02:41:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:41:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:41:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:41:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:41:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:41:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:41:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:41:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:41:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:41:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:52 smithi082 ceph-mon[98694]: pgmap v6638: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 230 KiB/s wr, 17 op/s 2023-09-10T02:41:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12143.806012 secs 2023-09-10T02:41:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:52 smithi098 ceph-mon[103124]: pgmap v6638: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 230 KiB/s wr, 17 op/s 2023-09-10T02:41:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12143.806012 secs 2023-09-10T02:41:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:52 smithi170 ceph-mon[102832]: pgmap v6638: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 230 KiB/s wr, 17 op/s 2023-09-10T02:41:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12143.806012 secs 2023-09-10T02:41:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12145.194939 secs 2023-09-10T02:41:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12145.194939 secs 2023-09-10T02:41:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12145.194939 secs 2023-09-10T02:41:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12145.906487 secs 2023-09-10T02:41:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:54 smithi082 ceph-mon[98694]: pgmap v6639: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T02:41:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12145.906487 secs 2023-09-10T02:41:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:54 smithi098 ceph-mon[103124]: pgmap v6639: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T02:41:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12145.906487 secs 2023-09-10T02:41:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:54 smithi170 ceph-mon[102832]: pgmap v6639: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T02:41:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:56 smithi098 ceph-mon[103124]: pgmap v6640: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 173 KiB/s wr, 16 op/s 2023-09-10T02:41:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:41:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:56 smithi170 ceph-mon[102832]: pgmap v6640: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 173 KiB/s wr, 16 op/s 2023-09-10T02:41:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:41:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:56 smithi082 ceph-mon[98694]: pgmap v6640: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 173 KiB/s wr, 16 op/s 2023-09-10T02:41:57.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:41:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12148.806106 secs 2023-09-10T02:41:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:41:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:41:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:41:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:41:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:41:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:41:58.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:41:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12148.806106 secs 2023-09-10T02:41:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:41:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:41:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:41:58.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:41:58.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:41:58.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:41:58.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:41:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12148.806106 secs 2023-09-10T02:41:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:41:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:41:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:41:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:41:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:41:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:41:58.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:41:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:59 smithi170 ceph-mon[102832]: pgmap v6641: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T02:41:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12150.194997 secs 2023-09-10T02:41:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:41:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12150.906555 secs 2023-09-10T02:41:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:59 smithi082 ceph-mon[98694]: pgmap v6641: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T02:41:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12150.194997 secs 2023-09-10T02:41:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:41:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12150.906555 secs 2023-09-10T02:41:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:59 smithi098 ceph-mon[103124]: pgmap v6641: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T02:41:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12150.194997 secs 2023-09-10T02:41:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:41:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12150.906555 secs 2023-09-10T02:42:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:01 smithi170 ceph-mon[102832]: pgmap v6642: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T02:42:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:01 smithi082 ceph-mon[98694]: pgmap v6642: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T02:42:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:01 smithi098 ceph-mon[103124]: pgmap v6642: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T02:42:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:02 smithi170 ceph-mon[102832]: pgmap v6643: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T02:42:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12153.806168 secs 2023-09-10T02:42:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:02 smithi082 ceph-mon[98694]: pgmap v6643: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T02:42:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12153.806168 secs 2023-09-10T02:42:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:02 smithi098 ceph-mon[103124]: pgmap v6643: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T02:42:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12153.806168 secs 2023-09-10T02:42:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12155.195056 secs 2023-09-10T02:42:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12155.195056 secs 2023-09-10T02:42:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12155.195056 secs 2023-09-10T02:42:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12155.906616 secs 2023-09-10T02:42:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:04 smithi170 ceph-mon[102832]: pgmap v6644: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T02:42:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12155.906616 secs 2023-09-10T02:42:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:04 smithi082 ceph-mon[98694]: pgmap v6644: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T02:42:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12155.906616 secs 2023-09-10T02:42:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:04 smithi098 ceph-mon[103124]: pgmap v6644: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T02:42:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:06 smithi082 ceph-mon[98694]: pgmap v6645: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 161 KiB/s wr, 30 op/s 2023-09-10T02:42:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:06 smithi098 ceph-mon[103124]: pgmap v6645: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 161 KiB/s wr, 30 op/s 2023-09-10T02:42:07.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:06 smithi170 ceph-mon[102832]: pgmap v6645: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 161 KiB/s wr, 30 op/s 2023-09-10T02:42:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12158.806270 secs 2023-09-10T02:42:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12158.806270 secs 2023-09-10T02:42:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12158.806270 secs 2023-09-10T02:42:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:08 smithi082 ceph-mon[98694]: pgmap v6646: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 88 KiB/s wr, 24 op/s 2023-09-10T02:42:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12160.195125 secs 2023-09-10T02:42:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:08 smithi082 ceph-mon[98694]: osdmap e483: 12 total, 12 up, 12 in 2023-09-10T02:42:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:08 smithi098 ceph-mon[103124]: pgmap v6646: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 88 KiB/s wr, 24 op/s 2023-09-10T02:42:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12160.195125 secs 2023-09-10T02:42:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:08 smithi098 ceph-mon[103124]: osdmap e483: 12 total, 12 up, 12 in 2023-09-10T02:42:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:08 smithi170 ceph-mon[102832]: pgmap v6646: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 88 KiB/s wr, 24 op/s 2023-09-10T02:42:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12160.195125 secs 2023-09-10T02:42:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:08 smithi170 ceph-mon[102832]: osdmap e483: 12 total, 12 up, 12 in 2023-09-10T02:42:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12160.906687 secs 2023-09-10T02:42:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12160.906687 secs 2023-09-10T02:42:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12160.906687 secs 2023-09-10T02:42:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:10 smithi082 ceph-mon[98694]: pgmap v6648: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 106 KiB/s wr, 29 op/s 2023-09-10T02:42:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:10 smithi098 ceph-mon[103124]: pgmap v6648: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 106 KiB/s wr, 29 op/s 2023-09-10T02:42:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:10 smithi170 ceph-mon[102832]: pgmap v6648: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 106 KiB/s wr, 29 op/s 2023-09-10T02:42:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:12 smithi170 ceph-mon[102832]: pgmap v6649: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T02:42:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12163.806359 secs 2023-09-10T02:42:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:12 smithi082 ceph-mon[98694]: pgmap v6649: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T02:42:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12163.806359 secs 2023-09-10T02:42:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:12 smithi098 ceph-mon[103124]: pgmap v6649: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T02:42:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12163.806359 secs 2023-09-10T02:42:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12165.195186 secs 2023-09-10T02:42:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12165.195186 secs 2023-09-10T02:42:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12165.195186 secs 2023-09-10T02:42:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12165.906747 secs 2023-09-10T02:42:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:14 smithi170 ceph-mon[102832]: pgmap v6650: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T02:42:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12165.906747 secs 2023-09-10T02:42:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:14 smithi082 ceph-mon[98694]: pgmap v6650: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T02:42:15.258 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12165.906747 secs 2023-09-10T02:42:15.258 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:14 smithi098 ceph-mon[103124]: pgmap v6650: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T02:42:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:16 smithi170 ceph-mon[102832]: pgmap v6651: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T02:42:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:16 smithi170 ceph-mon[102832]: osdmap e484: 12 total, 12 up, 12 in 2023-09-10T02:42:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:16 smithi082 ceph-mon[98694]: pgmap v6651: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T02:42:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:16 smithi082 ceph-mon[98694]: osdmap e484: 12 total, 12 up, 12 in 2023-09-10T02:42:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:16 smithi098 ceph-mon[103124]: pgmap v6651: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T02:42:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:16 smithi098 ceph-mon[103124]: osdmap e484: 12 total, 12 up, 12 in 2023-09-10T02:42:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12168.806433 secs 2023-09-10T02:42:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12168.806433 secs 2023-09-10T02:42:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12168.806433 secs 2023-09-10T02:42:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:18 smithi170 ceph-mon[102832]: pgmap v6653: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 12 op/s 2023-09-10T02:42:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12170.195243 secs 2023-09-10T02:42:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:18 smithi082 ceph-mon[98694]: pgmap v6653: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 12 op/s 2023-09-10T02:42:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12170.195243 secs 2023-09-10T02:42:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:18 smithi098 ceph-mon[103124]: pgmap v6653: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 12 op/s 2023-09-10T02:42:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12170.195243 secs 2023-09-10T02:42:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12170.906816 secs 2023-09-10T02:42:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12170.906816 secs 2023-09-10T02:42:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12170.906816 secs 2023-09-10T02:42:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:20 smithi170 ceph-mon[102832]: pgmap v6654: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T02:42:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:20 smithi082 ceph-mon[98694]: pgmap v6654: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T02:42:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:20 smithi098 ceph-mon[103124]: pgmap v6654: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T02:42:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:22 smithi170 ceph-mon[102832]: pgmap v6655: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T02:42:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:42:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:22 smithi082 ceph-mon[98694]: pgmap v6655: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T02:42:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:42:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:22 smithi098 ceph-mon[103124]: pgmap v6655: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T02:42:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:42:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:42:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:42:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:42:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12175.195326 secs 2023-09-10T02:42:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:42:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:42:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:42:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12175.195326 secs 2023-09-10T02:42:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:42:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:42:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:42:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12175.195326 secs 2023-09-10T02:42:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12175.906881 secs 2023-09-10T02:42:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:24 smithi170 ceph-mon[102832]: pgmap v6656: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T02:42:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12175.906881 secs 2023-09-10T02:42:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:24 smithi082 ceph-mon[98694]: pgmap v6656: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T02:42:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12175.906881 secs 2023-09-10T02:42:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:24 smithi098 ceph-mon[103124]: pgmap v6656: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T02:42:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:26 smithi170 ceph-mon[102832]: pgmap v6657: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 89 KiB/s wr, 12 op/s 2023-09-10T02:42:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:26 smithi082 ceph-mon[98694]: pgmap v6657: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 89 KiB/s wr, 12 op/s 2023-09-10T02:42:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:26 smithi098 ceph-mon[103124]: pgmap v6657: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 89 KiB/s wr, 12 op/s 2023-09-10T02:42:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:28 smithi170 ceph-mon[102832]: pgmap v6658: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T02:42:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12180.195386 secs 2023-09-10T02:42:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:28 smithi082 ceph-mon[98694]: pgmap v6658: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T02:42:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12180.195386 secs 2023-09-10T02:42:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:28 smithi098 ceph-mon[103124]: pgmap v6658: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T02:42:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12180.195386 secs 2023-09-10T02:42:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12180.906945 secs 2023-09-10T02:42:30.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12180.906945 secs 2023-09-10T02:42:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12180.906945 secs 2023-09-10T02:42:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:30 smithi170 ceph-mon[102832]: pgmap v6659: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T02:42:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:30 smithi082 ceph-mon[98694]: pgmap v6659: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T02:42:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:30 smithi098 ceph-mon[103124]: pgmap v6659: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 10 op/s 2023-09-10T02:42:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:32 smithi170 ceph-mon[102832]: pgmap v6660: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 167 KiB/s wr, 16 op/s 2023-09-10T02:42:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:32 smithi082 ceph-mon[98694]: pgmap v6660: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 167 KiB/s wr, 16 op/s 2023-09-10T02:42:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:32 smithi098 ceph-mon[103124]: pgmap v6660: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 167 KiB/s wr, 16 op/s 2023-09-10T02:42:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12185.195494 secs 2023-09-10T02:42:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12185.195494 secs 2023-09-10T02:42:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12185.195494 secs 2023-09-10T02:42:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12185.907008 secs 2023-09-10T02:42:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:34 smithi170 ceph-mon[102832]: pgmap v6661: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 132 KiB/s wr, 12 op/s 2023-09-10T02:42:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12185.907008 secs 2023-09-10T02:42:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:34 smithi082 ceph-mon[98694]: pgmap v6661: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 132 KiB/s wr, 12 op/s 2023-09-10T02:42:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12185.907008 secs 2023-09-10T02:42:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:34 smithi098 ceph-mon[103124]: pgmap v6661: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 132 KiB/s wr, 12 op/s 2023-09-10T02:42:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:36 smithi170 ceph-mon[102832]: pgmap v6662: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 259 KiB/s wr, 18 op/s 2023-09-10T02:42:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:36 smithi082 ceph-mon[98694]: pgmap v6662: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 259 KiB/s wr, 18 op/s 2023-09-10T02:42:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:36 smithi098 ceph-mon[103124]: pgmap v6662: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 259 KiB/s wr, 18 op/s 2023-09-10T02:42:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:38 smithi170 ceph-mon[102832]: pgmap v6663: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 220 KiB/s wr, 12 op/s 2023-09-10T02:42:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12190.195553 secs 2023-09-10T02:42:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:38 smithi082 ceph-mon[98694]: pgmap v6663: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 220 KiB/s wr, 12 op/s 2023-09-10T02:42:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12190.195553 secs 2023-09-10T02:42:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:38 smithi098 ceph-mon[103124]: pgmap v6663: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 220 KiB/s wr, 12 op/s 2023-09-10T02:42:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12190.195553 secs 2023-09-10T02:42:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12190.907072 secs 2023-09-10T02:42:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12190.907072 secs 2023-09-10T02:42:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12190.907072 secs 2023-09-10T02:42:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:40 smithi170 ceph-mon[102832]: pgmap v6664: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 220 KiB/s wr, 12 op/s 2023-09-10T02:42:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:40 smithi082 ceph-mon[98694]: pgmap v6664: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 220 KiB/s wr, 12 op/s 2023-09-10T02:42:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:40 smithi098 ceph-mon[103124]: pgmap v6664: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 220 KiB/s wr, 12 op/s 2023-09-10T02:42:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:42 smithi170 ceph-mon[102832]: pgmap v6665: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 323 KiB/s wr, 19 op/s 2023-09-10T02:42:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12193.806776 secs 2023-09-10T02:42:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:42 smithi082 ceph-mon[98694]: pgmap v6665: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 323 KiB/s wr, 19 op/s 2023-09-10T02:42:43.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12193.806776 secs 2023-09-10T02:42:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:42 smithi098 ceph-mon[103124]: pgmap v6665: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 323 KiB/s wr, 19 op/s 2023-09-10T02:42:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12193.806776 secs 2023-09-10T02:42:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:42:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:42:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:42:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12195.195639 secs 2023-09-10T02:42:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:42:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:42:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:42:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12195.195639 secs 2023-09-10T02:42:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:42:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:42:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:42:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12195.195639 secs 2023-09-10T02:42:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12195.907136 secs 2023-09-10T02:42:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:44 smithi170 ceph-mon[102832]: pgmap v6666: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 231 KiB/s wr, 13 op/s 2023-09-10T02:42:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12195.907136 secs 2023-09-10T02:42:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:44 smithi082 ceph-mon[98694]: pgmap v6666: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 231 KiB/s wr, 13 op/s 2023-09-10T02:42:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12195.907136 secs 2023-09-10T02:42:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:44 smithi098 ceph-mon[103124]: pgmap v6666: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 231 KiB/s wr, 13 op/s 2023-09-10T02:42:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:46 smithi170 ceph-mon[102832]: pgmap v6667: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 253 KiB/s wr, 18 op/s 2023-09-10T02:42:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:46 smithi082 ceph-mon[98694]: pgmap v6667: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 253 KiB/s wr, 18 op/s 2023-09-10T02:42:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:46 smithi098 ceph-mon[103124]: pgmap v6667: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 253 KiB/s wr, 18 op/s 2023-09-10T02:42:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12198.806838 secs 2023-09-10T02:42:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12198.806838 secs 2023-09-10T02:42:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12198.806838 secs 2023-09-10T02:42:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:48 smithi170 ceph-mon[102832]: pgmap v6668: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T02:42:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12200.195700 secs 2023-09-10T02:42:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:48 smithi082 ceph-mon[98694]: pgmap v6668: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T02:42:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12200.195700 secs 2023-09-10T02:42:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:48 smithi098 ceph-mon[103124]: pgmap v6668: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T02:42:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12200.195700 secs 2023-09-10T02:42:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12200.907211 secs 2023-09-10T02:42:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12200.907211 secs 2023-09-10T02:42:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12200.907211 secs 2023-09-10T02:42:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:50 smithi170 ceph-mon[102832]: pgmap v6669: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T02:42:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:50 smithi082 ceph-mon[98694]: pgmap v6669: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T02:42:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:50 smithi098 ceph-mon[103124]: pgmap v6669: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T02:42:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:52 smithi170 ceph-mon[102832]: pgmap v6670: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 196 KiB/s wr, 17 op/s 2023-09-10T02:42:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12203.806902 secs 2023-09-10T02:42:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:52 smithi082 ceph-mon[98694]: pgmap v6670: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 196 KiB/s wr, 17 op/s 2023-09-10T02:42:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12203.806902 secs 2023-09-10T02:42:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:52 smithi098 ceph-mon[103124]: pgmap v6670: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 196 KiB/s wr, 17 op/s 2023-09-10T02:42:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12203.806902 secs 2023-09-10T02:42:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12205.195757 secs 2023-09-10T02:42:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12205.195757 secs 2023-09-10T02:42:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12205.195757 secs 2023-09-10T02:42:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12205.907275 secs 2023-09-10T02:42:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:54 smithi170 ceph-mon[102832]: pgmap v6671: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T02:42:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12205.907275 secs 2023-09-10T02:42:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:54 smithi082 ceph-mon[98694]: pgmap v6671: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T02:42:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12205.907275 secs 2023-09-10T02:42:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:54 smithi098 ceph-mon[103124]: pgmap v6671: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T02:42:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:56 smithi170 ceph-mon[102832]: pgmap v6672: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 92 KiB/s wr, 14 op/s 2023-09-10T02:42:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:56 smithi082 ceph-mon[98694]: pgmap v6672: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 92 KiB/s wr, 14 op/s 2023-09-10T02:42:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:56 smithi098 ceph-mon[103124]: pgmap v6672: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 92 KiB/s wr, 14 op/s 2023-09-10T02:42:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12208.806961 secs 2023-09-10T02:42:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12208.806961 secs 2023-09-10T02:42:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12208.806961 secs 2023-09-10T02:42:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:58 smithi170 ceph-mon[102832]: pgmap v6673: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T02:42:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:42:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12210.195816 secs 2023-09-10T02:42:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:42:59.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:42:59.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:42:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:58 smithi082 ceph-mon[98694]: pgmap v6673: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T02:42:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:42:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12210.195816 secs 2023-09-10T02:42:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:42:59.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:42:59.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:42:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:58 smithi098 ceph-mon[103124]: pgmap v6673: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T02:42:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:42:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12210.195816 secs 2023-09-10T02:42:59.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:42:59.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:42:59.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:43:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:42:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12210.907344 secs 2023-09-10T02:43:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:42:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12210.907344 secs 2023-09-10T02:43:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:42:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12210.907344 secs 2023-09-10T02:43:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:00 smithi170 ceph-mon[102832]: pgmap v6674: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T02:43:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:00 smithi082 ceph-mon[98694]: pgmap v6674: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T02:43:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:00 smithi098 ceph-mon[103124]: pgmap v6674: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T02:43:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:02 smithi170 ceph-mon[102832]: pgmap v6675: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 96 KiB/s wr, 14 op/s 2023-09-10T02:43:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12213.807056 secs 2023-09-10T02:43:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:02 smithi082 ceph-mon[98694]: pgmap v6675: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 96 KiB/s wr, 14 op/s 2023-09-10T02:43:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12213.807056 secs 2023-09-10T02:43:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:02 smithi098 ceph-mon[103124]: pgmap v6675: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 96 KiB/s wr, 14 op/s 2023-09-10T02:43:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12213.807056 secs 2023-09-10T02:43:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12215.195904 secs 2023-09-10T02:43:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12215.195904 secs 2023-09-10T02:43:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12215.195904 secs 2023-09-10T02:43:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12215.907411 secs 2023-09-10T02:43:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:04 smithi170 ceph-mon[102832]: pgmap v6676: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 26 KiB/s wr, 9 op/s 2023-09-10T02:43:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12215.907411 secs 2023-09-10T02:43:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:04 smithi082 ceph-mon[98694]: pgmap v6676: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 26 KiB/s wr, 9 op/s 2023-09-10T02:43:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12215.907411 secs 2023-09-10T02:43:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:04 smithi098 ceph-mon[103124]: pgmap v6676: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 26 KiB/s wr, 9 op/s 2023-09-10T02:43:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:06 smithi170 ceph-mon[102832]: pgmap v6677: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T02:43:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:06 smithi082 ceph-mon[98694]: pgmap v6677: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T02:43:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:06 smithi098 ceph-mon[103124]: pgmap v6677: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 81 KiB/s wr, 29 op/s 2023-09-10T02:43:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12218.807117 secs 2023-09-10T02:43:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12218.807117 secs 2023-09-10T02:43:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12218.807117 secs 2023-09-10T02:43:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:08 smithi170 ceph-mon[102832]: pgmap v6678: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 81 KiB/s wr, 24 op/s 2023-09-10T02:43:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:08 smithi170 ceph-mon[102832]: osdmap e485: 12 total, 12 up, 12 in 2023-09-10T02:43:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12220.195980 secs 2023-09-10T02:43:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:08 smithi082 ceph-mon[98694]: pgmap v6678: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 81 KiB/s wr, 24 op/s 2023-09-10T02:43:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:08 smithi082 ceph-mon[98694]: osdmap e485: 12 total, 12 up, 12 in 2023-09-10T02:43:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12220.195980 secs 2023-09-10T02:43:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:08 smithi098 ceph-mon[103124]: pgmap v6678: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 81 KiB/s wr, 24 op/s 2023-09-10T02:43:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:08 smithi098 ceph-mon[103124]: osdmap e485: 12 total, 12 up, 12 in 2023-09-10T02:43:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12220.195980 secs 2023-09-10T02:43:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12220.907482 secs 2023-09-10T02:43:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12220.907482 secs 2023-09-10T02:43:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12220.907482 secs 2023-09-10T02:43:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:10 smithi170 ceph-mon[102832]: pgmap v6680: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T02:43:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:10 smithi082 ceph-mon[98694]: pgmap v6680: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T02:43:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:10 smithi098 ceph-mon[103124]: pgmap v6680: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T02:43:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:12 smithi170 ceph-mon[102832]: pgmap v6681: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T02:43:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12223.807184 secs 2023-09-10T02:43:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:12 smithi082 ceph-mon[98694]: pgmap v6681: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T02:43:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12223.807184 secs 2023-09-10T02:43:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:12 smithi098 ceph-mon[103124]: pgmap v6681: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T02:43:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12223.807184 secs 2023-09-10T02:43:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12225.196067 secs 2023-09-10T02:43:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12225.196067 secs 2023-09-10T02:43:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12225.196067 secs 2023-09-10T02:43:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12225.907545 secs 2023-09-10T02:43:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:14 smithi170 ceph-mon[102832]: pgmap v6682: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T02:43:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12225.907545 secs 2023-09-10T02:43:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:14 smithi082 ceph-mon[98694]: pgmap v6682: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T02:43:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12225.907545 secs 2023-09-10T02:43:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:14 smithi098 ceph-mon[103124]: pgmap v6682: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T02:43:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:16 smithi170 ceph-mon[102832]: pgmap v6683: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T02:43:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:16 smithi170 ceph-mon[102832]: osdmap e486: 12 total, 12 up, 12 in 2023-09-10T02:43:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:16 smithi082 ceph-mon[98694]: pgmap v6683: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T02:43:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:16 smithi082 ceph-mon[98694]: osdmap e486: 12 total, 12 up, 12 in 2023-09-10T02:43:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:16 smithi098 ceph-mon[103124]: pgmap v6683: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T02:43:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:16 smithi098 ceph-mon[103124]: osdmap e486: 12 total, 12 up, 12 in 2023-09-10T02:43:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12228.807286 secs 2023-09-10T02:43:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12228.807286 secs 2023-09-10T02:43:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12228.807286 secs 2023-09-10T02:43:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:18 smithi170 ceph-mon[102832]: pgmap v6685: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T02:43:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12230.196126 secs 2023-09-10T02:43:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:18 smithi082 ceph-mon[98694]: pgmap v6685: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T02:43:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12230.196126 secs 2023-09-10T02:43:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:18 smithi098 ceph-mon[103124]: pgmap v6685: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T02:43:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12230.196126 secs 2023-09-10T02:43:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12230.907610 secs 2023-09-10T02:43:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12230.907610 secs 2023-09-10T02:43:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12230.907610 secs 2023-09-10T02:43:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:20 smithi170 ceph-mon[102832]: pgmap v6686: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T02:43:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:20 smithi082 ceph-mon[98694]: pgmap v6686: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T02:43:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:20 smithi098 ceph-mon[103124]: pgmap v6686: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T02:43:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:22 smithi170 ceph-mon[102832]: pgmap v6687: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T02:43:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12233.807346 secs 2023-09-10T02:43:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:22 smithi082 ceph-mon[98694]: pgmap v6687: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T02:43:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12233.807346 secs 2023-09-10T02:43:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:22 smithi098 ceph-mon[103124]: pgmap v6687: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T02:43:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12233.807346 secs 2023-09-10T02:43:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12235.196187 secs 2023-09-10T02:43:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12235.196187 secs 2023-09-10T02:43:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12235.196187 secs 2023-09-10T02:43:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12235.907677 secs 2023-09-10T02:43:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:24 smithi170 ceph-mon[102832]: pgmap v6688: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T02:43:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12235.907677 secs 2023-09-10T02:43:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:24 smithi082 ceph-mon[98694]: pgmap v6688: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T02:43:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12235.907677 secs 2023-09-10T02:43:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:24 smithi098 ceph-mon[103124]: pgmap v6688: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T02:43:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:26 smithi170 ceph-mon[102832]: pgmap v6689: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 12 op/s 2023-09-10T02:43:27.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:26 smithi082 ceph-mon[98694]: pgmap v6689: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 12 op/s 2023-09-10T02:43:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:26 smithi098 ceph-mon[103124]: pgmap v6689: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 12 op/s 2023-09-10T02:43:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:28 smithi170 ceph-mon[102832]: pgmap v6690: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T02:43:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12240.196247 secs 2023-09-10T02:43:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:28 smithi082 ceph-mon[98694]: pgmap v6690: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T02:43:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12240.196247 secs 2023-09-10T02:43:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:28 smithi098 ceph-mon[103124]: pgmap v6690: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T02:43:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12240.196247 secs 2023-09-10T02:43:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12240.907743 secs 2023-09-10T02:43:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12240.907743 secs 2023-09-10T02:43:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12240.907743 secs 2023-09-10T02:43:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:30 smithi170 ceph-mon[102832]: pgmap v6691: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 10 op/s 2023-09-10T02:43:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:30 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:43:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:30 smithi082 ceph-mon[98694]: pgmap v6691: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 10 op/s 2023-09-10T02:43:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:30 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:43:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:30 smithi098 ceph-mon[103124]: pgmap v6691: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 10 op/s 2023-09-10T02:43:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:30 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:43:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:31 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:43:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:43:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:43:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:31 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:43:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:43:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:43:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:31 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:43:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:43:32.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:43:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:32 smithi170 ceph-mon[102832]: pgmap v6692: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 142 KiB/s wr, 16 op/s 2023-09-10T02:43:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:32 smithi082 ceph-mon[98694]: pgmap v6692: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 142 KiB/s wr, 16 op/s 2023-09-10T02:43:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:32 smithi098 ceph-mon[103124]: pgmap v6692: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 142 KiB/s wr, 16 op/s 2023-09-10T02:43:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12245.196334 secs 2023-09-10T02:43:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12245.196334 secs 2023-09-10T02:43:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12245.196334 secs 2023-09-10T02:43:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12245.907807 secs 2023-09-10T02:43:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:34 smithi170 ceph-mon[102832]: pgmap v6693: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 107 KiB/s wr, 12 op/s 2023-09-10T02:43:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12245.907807 secs 2023-09-10T02:43:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:34 smithi082 ceph-mon[98694]: pgmap v6693: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 107 KiB/s wr, 12 op/s 2023-09-10T02:43:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12245.907807 secs 2023-09-10T02:43:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:34 smithi098 ceph-mon[103124]: pgmap v6693: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 107 KiB/s wr, 12 op/s 2023-09-10T02:43:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:36 smithi082 ceph-mon[98694]: pgmap v6694: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 188 KiB/s wr, 18 op/s 2023-09-10T02:43:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:36 smithi098 ceph-mon[103124]: pgmap v6694: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 188 KiB/s wr, 18 op/s 2023-09-10T02:43:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:36 smithi170 ceph-mon[102832]: pgmap v6694: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 188 KiB/s wr, 18 op/s 2023-09-10T02:43:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:38 smithi082 ceph-mon[98694]: pgmap v6695: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T02:43:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12250.196393 secs 2023-09-10T02:43:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:38 smithi098 ceph-mon[103124]: pgmap v6695: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T02:43:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12250.196393 secs 2023-09-10T02:43:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:38 smithi170 ceph-mon[102832]: pgmap v6695: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T02:43:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12250.196393 secs 2023-09-10T02:43:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12250.907873 secs 2023-09-10T02:43:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12250.907873 secs 2023-09-10T02:43:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12250.907873 secs 2023-09-10T02:43:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:40 smithi082 ceph-mon[98694]: pgmap v6696: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T02:43:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:40 smithi098 ceph-mon[103124]: pgmap v6696: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T02:43:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:40 smithi170 ceph-mon[102832]: pgmap v6696: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T02:43:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:42 smithi170 ceph-mon[102832]: pgmap v6697: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 221 KiB/s wr, 18 op/s 2023-09-10T02:43:43.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:42 smithi082 ceph-mon[98694]: pgmap v6697: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 221 KiB/s wr, 18 op/s 2023-09-10T02:43:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:42 smithi098 ceph-mon[103124]: pgmap v6697: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 221 KiB/s wr, 18 op/s 2023-09-10T02:43:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12255.196455 secs 2023-09-10T02:43:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12255.196455 secs 2023-09-10T02:43:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12255.196455 secs 2023-09-10T02:43:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12255.907940 secs 2023-09-10T02:43:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:44 smithi170 ceph-mon[102832]: pgmap v6698: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 144 KiB/s wr, 12 op/s 2023-09-10T02:43:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12255.907940 secs 2023-09-10T02:43:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:44 smithi082 ceph-mon[98694]: pgmap v6698: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 144 KiB/s wr, 12 op/s 2023-09-10T02:43:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12255.907940 secs 2023-09-10T02:43:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:44 smithi098 ceph-mon[103124]: pgmap v6698: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 144 KiB/s wr, 12 op/s 2023-09-10T02:43:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:46 smithi170 ceph-mon[102832]: pgmap v6699: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 239 KiB/s wr, 18 op/s 2023-09-10T02:43:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:46 smithi082 ceph-mon[98694]: pgmap v6699: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 239 KiB/s wr, 18 op/s 2023-09-10T02:43:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:46 smithi098 ceph-mon[103124]: pgmap v6699: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 239 KiB/s wr, 18 op/s 2023-09-10T02:43:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12258.807708 secs 2023-09-10T02:43:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12258.807708 secs 2023-09-10T02:43:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12258.807708 secs 2023-09-10T02:43:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:48 smithi170 ceph-mon[102832]: pgmap v6700: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 157 KiB/s wr, 12 op/s 2023-09-10T02:43:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12260.196499 secs 2023-09-10T02:43:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:48 smithi082 ceph-mon[98694]: pgmap v6700: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 157 KiB/s wr, 12 op/s 2023-09-10T02:43:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12260.196499 secs 2023-09-10T02:43:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:48 smithi098 ceph-mon[103124]: pgmap v6700: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 157 KiB/s wr, 12 op/s 2023-09-10T02:43:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12260.196499 secs 2023-09-10T02:43:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12260.908003 secs 2023-09-10T02:43:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12260.908003 secs 2023-09-10T02:43:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12260.908003 secs 2023-09-10T02:43:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:50 smithi082 ceph-mon[98694]: pgmap v6701: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 157 KiB/s wr, 12 op/s 2023-09-10T02:43:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:50 smithi098 ceph-mon[103124]: pgmap v6701: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 157 KiB/s wr, 12 op/s 2023-09-10T02:43:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:50 smithi170 ceph-mon[102832]: pgmap v6701: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 157 KiB/s wr, 12 op/s 2023-09-10T02:43:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:43:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:43:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:43:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:43:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:43:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:43:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:43:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:43:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:43:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:52 smithi082 ceph-mon[98694]: pgmap v6702: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 228 KiB/s wr, 17 op/s 2023-09-10T02:43:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12263.807806 secs 2023-09-10T02:43:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:52 smithi098 ceph-mon[103124]: pgmap v6702: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 228 KiB/s wr, 17 op/s 2023-09-10T02:43:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12263.807806 secs 2023-09-10T02:43:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:52 smithi170 ceph-mon[102832]: pgmap v6702: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 228 KiB/s wr, 17 op/s 2023-09-10T02:43:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12263.807806 secs 2023-09-10T02:43:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12265.196560 secs 2023-09-10T02:43:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12265.196560 secs 2023-09-10T02:43:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12265.196560 secs 2023-09-10T02:43:55.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12265.908071 secs 2023-09-10T02:43:55.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:54 smithi098 ceph-mon[103124]: pgmap v6703: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T02:43:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12265.908071 secs 2023-09-10T02:43:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:54 smithi170 ceph-mon[102832]: pgmap v6703: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T02:43:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12265.908071 secs 2023-09-10T02:43:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:54 smithi082 ceph-mon[98694]: pgmap v6703: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T02:43:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:56 smithi098 ceph-mon[103124]: pgmap v6704: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 212 KiB/s wr, 16 op/s 2023-09-10T02:43:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:56 smithi170 ceph-mon[102832]: pgmap v6704: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 212 KiB/s wr, 16 op/s 2023-09-10T02:43:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:56 smithi082 ceph-mon[98694]: pgmap v6704: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 212 KiB/s wr, 16 op/s 2023-09-10T02:43:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12268.807868 secs 2023-09-10T02:43:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12268.807868 secs 2023-09-10T02:43:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12268.807868 secs 2023-09-10T02:43:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:58 smithi098 ceph-mon[103124]: pgmap v6705: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T02:43:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12270.196617 secs 2023-09-10T02:43:59.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:43:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:58 smithi170 ceph-mon[102832]: pgmap v6705: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T02:43:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12270.196617 secs 2023-09-10T02:43:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:43:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:58 smithi082 ceph-mon[98694]: pgmap v6705: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T02:43:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12270.196617 secs 2023-09-10T02:43:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:44:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12270.908137 secs 2023-09-10T02:44:00.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:44:00.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:44:00.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:43:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:44:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12270.908137 secs 2023-09-10T02:44:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:44:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:44:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:43:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:44:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12270.908137 secs 2023-09-10T02:44:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:44:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:44:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:43:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:44:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:01 smithi082 ceph-mon[98694]: pgmap v6706: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T02:44:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:01 smithi098 ceph-mon[103124]: pgmap v6706: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T02:44:01.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:01 smithi170 ceph-mon[102832]: pgmap v6706: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T02:44:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:02 smithi170 ceph-mon[102832]: pgmap v6707: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 151 KiB/s wr, 16 op/s 2023-09-10T02:44:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12273.807929 secs 2023-09-10T02:44:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:02 smithi082 ceph-mon[98694]: pgmap v6707: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 151 KiB/s wr, 16 op/s 2023-09-10T02:44:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12273.807929 secs 2023-09-10T02:44:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:02 smithi098 ceph-mon[103124]: pgmap v6707: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 151 KiB/s wr, 16 op/s 2023-09-10T02:44:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12273.807929 secs 2023-09-10T02:44:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12275.196674 secs 2023-09-10T02:44:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12275.196674 secs 2023-09-10T02:44:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12275.196674 secs 2023-09-10T02:44:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12275.908202 secs 2023-09-10T02:44:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:04 smithi170 ceph-mon[102832]: pgmap v6708: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T02:44:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12275.908202 secs 2023-09-10T02:44:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:04 smithi082 ceph-mon[98694]: pgmap v6708: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T02:44:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12275.908202 secs 2023-09-10T02:44:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:04 smithi098 ceph-mon[103124]: pgmap v6708: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T02:44:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:06 smithi170 ceph-mon[102832]: pgmap v6709: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 126 KiB/s wr, 29 op/s 2023-09-10T02:44:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:06 smithi082 ceph-mon[98694]: pgmap v6709: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 126 KiB/s wr, 29 op/s 2023-09-10T02:44:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:06 smithi098 ceph-mon[103124]: pgmap v6709: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 126 KiB/s wr, 29 op/s 2023-09-10T02:44:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12278.807991 secs 2023-09-10T02:44:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12278.807991 secs 2023-09-10T02:44:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12278.807991 secs 2023-09-10T02:44:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:08 smithi082 ceph-mon[98694]: pgmap v6710: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 79 KiB/s wr, 24 op/s 2023-09-10T02:44:09.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:08 smithi082 ceph-mon[98694]: osdmap e487: 12 total, 12 up, 12 in 2023-09-10T02:44:09.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12280.196877 secs 2023-09-10T02:44:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:08 smithi098 ceph-mon[103124]: pgmap v6710: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 79 KiB/s wr, 24 op/s 2023-09-10T02:44:09.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:08 smithi098 ceph-mon[103124]: osdmap e487: 12 total, 12 up, 12 in 2023-09-10T02:44:09.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12280.196877 secs 2023-09-10T02:44:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:08 smithi170 ceph-mon[102832]: pgmap v6710: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 79 KiB/s wr, 24 op/s 2023-09-10T02:44:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:08 smithi170 ceph-mon[102832]: osdmap e487: 12 total, 12 up, 12 in 2023-09-10T02:44:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12280.196877 secs 2023-09-10T02:44:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12280.908271 secs 2023-09-10T02:44:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12280.908271 secs 2023-09-10T02:44:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12280.908271 secs 2023-09-10T02:44:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:10 smithi082 ceph-mon[98694]: pgmap v6712: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T02:44:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:10 smithi098 ceph-mon[103124]: pgmap v6712: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T02:44:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:10 smithi170 ceph-mon[102832]: pgmap v6712: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 29 op/s 2023-09-10T02:44:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:12 smithi170 ceph-mon[102832]: pgmap v6713: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 28 op/s 2023-09-10T02:44:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12283.808052 secs 2023-09-10T02:44:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:12 smithi082 ceph-mon[98694]: pgmap v6713: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 28 op/s 2023-09-10T02:44:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12283.808052 secs 2023-09-10T02:44:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:12 smithi098 ceph-mon[103124]: pgmap v6713: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 28 op/s 2023-09-10T02:44:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12283.808052 secs 2023-09-10T02:44:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12285.196934 secs 2023-09-10T02:44:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12285.196934 secs 2023-09-10T02:44:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12285.196934 secs 2023-09-10T02:44:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12285.908392 secs 2023-09-10T02:44:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:14 smithi098 ceph-mon[103124]: pgmap v6714: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 28 op/s 2023-09-10T02:44:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12285.908392 secs 2023-09-10T02:44:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:14 smithi170 ceph-mon[102832]: pgmap v6714: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 28 op/s 2023-09-10T02:44:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12285.908392 secs 2023-09-10T02:44:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:14 smithi082 ceph-mon[98694]: pgmap v6714: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 28 op/s 2023-09-10T02:44:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:16 smithi098 ceph-mon[103124]: pgmap v6715: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 12 op/s 2023-09-10T02:44:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:16 smithi098 ceph-mon[103124]: osdmap e488: 12 total, 12 up, 12 in 2023-09-10T02:44:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:16 smithi170 ceph-mon[102832]: pgmap v6715: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 12 op/s 2023-09-10T02:44:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:16 smithi170 ceph-mon[102832]: osdmap e488: 12 total, 12 up, 12 in 2023-09-10T02:44:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:16 smithi082 ceph-mon[98694]: pgmap v6715: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 12 op/s 2023-09-10T02:44:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:16 smithi082 ceph-mon[98694]: osdmap e488: 12 total, 12 up, 12 in 2023-09-10T02:44:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12288.808147 secs 2023-09-10T02:44:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12288.808147 secs 2023-09-10T02:44:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12288.808147 secs 2023-09-10T02:44:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:18 smithi098 ceph-mon[103124]: pgmap v6717: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 67 KiB/s wr, 12 op/s 2023-09-10T02:44:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12290.197010 secs 2023-09-10T02:44:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:18 smithi170 ceph-mon[102832]: pgmap v6717: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 67 KiB/s wr, 12 op/s 2023-09-10T02:44:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12290.197010 secs 2023-09-10T02:44:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:18 smithi082 ceph-mon[98694]: pgmap v6717: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 67 KiB/s wr, 12 op/s 2023-09-10T02:44:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12290.197010 secs 2023-09-10T02:44:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12290.908486 secs 2023-09-10T02:44:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12290.908486 secs 2023-09-10T02:44:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12290.908486 secs 2023-09-10T02:44:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:20 smithi098 ceph-mon[103124]: pgmap v6718: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 12 op/s 2023-09-10T02:44:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:20 smithi170 ceph-mon[102832]: pgmap v6718: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 12 op/s 2023-09-10T02:44:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:20 smithi082 ceph-mon[98694]: pgmap v6718: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 12 op/s 2023-09-10T02:44:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:22 smithi170 ceph-mon[102832]: pgmap v6719: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T02:44:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12293.808214 secs 2023-09-10T02:44:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:22 smithi082 ceph-mon[98694]: pgmap v6719: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T02:44:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12293.808214 secs 2023-09-10T02:44:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:22 smithi098 ceph-mon[103124]: pgmap v6719: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T02:44:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12293.808214 secs 2023-09-10T02:44:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12295.197069 secs 2023-09-10T02:44:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12295.197069 secs 2023-09-10T02:44:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12295.197069 secs 2023-09-10T02:44:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12295.908553 secs 2023-09-10T02:44:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:24 smithi170 ceph-mon[102832]: pgmap v6720: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T02:44:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12295.908553 secs 2023-09-10T02:44:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:24 smithi082 ceph-mon[98694]: pgmap v6720: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T02:44:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12295.908553 secs 2023-09-10T02:44:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:24 smithi098 ceph-mon[103124]: pgmap v6720: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T02:44:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:26 smithi170 ceph-mon[102832]: pgmap v6721: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T02:44:27.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:26 smithi082 ceph-mon[98694]: pgmap v6721: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T02:44:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:26 smithi098 ceph-mon[103124]: pgmap v6721: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T02:44:28.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12298.808272 secs 2023-09-10T02:44:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12298.808272 secs 2023-09-10T02:44:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12298.808272 secs 2023-09-10T02:44:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:28 smithi082 ceph-mon[98694]: pgmap v6722: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 10 op/s 2023-09-10T02:44:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12300.197129 secs 2023-09-10T02:44:29.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:28 smithi098 ceph-mon[103124]: pgmap v6722: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 10 op/s 2023-09-10T02:44:29.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12300.197129 secs 2023-09-10T02:44:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:28 smithi170 ceph-mon[102832]: pgmap v6722: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 10 op/s 2023-09-10T02:44:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12300.197129 secs 2023-09-10T02:44:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12300.908618 secs 2023-09-10T02:44:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12300.908618 secs 2023-09-10T02:44:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12300.908618 secs 2023-09-10T02:44:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:30 smithi082 ceph-mon[98694]: pgmap v6723: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T02:44:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:30 smithi098 ceph-mon[103124]: pgmap v6723: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T02:44:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:30 smithi170 ceph-mon[102832]: pgmap v6723: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T02:44:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:32 smithi170 ceph-mon[102832]: pgmap v6724: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 153 KiB/s wr, 15 op/s 2023-09-10T02:44:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:32 smithi082 ceph-mon[98694]: pgmap v6724: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 153 KiB/s wr, 15 op/s 2023-09-10T02:44:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:32 smithi098 ceph-mon[103124]: pgmap v6724: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 153 KiB/s wr, 15 op/s 2023-09-10T02:44:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12305.197189 secs 2023-09-10T02:44:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12305.197189 secs 2023-09-10T02:44:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12305.197189 secs 2023-09-10T02:44:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12305.908682 secs 2023-09-10T02:44:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:34 smithi170 ceph-mon[102832]: pgmap v6725: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T02:44:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:34 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:44:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12305.908682 secs 2023-09-10T02:44:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:34 smithi082 ceph-mon[98694]: pgmap v6725: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T02:44:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:34 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:44:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12305.908682 secs 2023-09-10T02:44:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:34 smithi098 ceph-mon[103124]: pgmap v6725: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T02:44:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:34 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:44:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:35 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:44:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:35 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:44:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:35 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:44:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:35 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:44:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:35 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:44:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:35 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:44:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:35 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:44:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:35 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:44:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:35 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:44:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:36 smithi082 ceph-mon[98694]: pgmap v6726: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 266 KiB/s wr, 17 op/s 2023-09-10T02:44:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:36 smithi098 ceph-mon[103124]: pgmap v6726: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 266 KiB/s wr, 17 op/s 2023-09-10T02:44:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:36 smithi170 ceph-mon[102832]: pgmap v6726: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 266 KiB/s wr, 17 op/s 2023-09-10T02:44:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:38 smithi082 ceph-mon[98694]: pgmap v6727: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T02:44:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12310.197249 secs 2023-09-10T02:44:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:38 smithi098 ceph-mon[103124]: pgmap v6727: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T02:44:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12310.197249 secs 2023-09-10T02:44:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:38 smithi170 ceph-mon[102832]: pgmap v6727: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T02:44:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12310.197249 secs 2023-09-10T02:44:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12310.908772 secs 2023-09-10T02:44:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12310.908772 secs 2023-09-10T02:44:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12310.908772 secs 2023-09-10T02:44:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:40 smithi082 ceph-mon[98694]: pgmap v6728: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T02:44:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:40 smithi098 ceph-mon[103124]: pgmap v6728: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T02:44:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:40 smithi170 ceph-mon[102832]: pgmap v6728: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T02:44:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:42 smithi170 ceph-mon[102832]: pgmap v6729: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 293 KiB/s wr, 18 op/s 2023-09-10T02:44:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12313.808500 secs 2023-09-10T02:44:43.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:42 smithi082 ceph-mon[98694]: pgmap v6729: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 293 KiB/s wr, 18 op/s 2023-09-10T02:44:43.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12313.808500 secs 2023-09-10T02:44:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:42 smithi098 ceph-mon[103124]: pgmap v6729: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 293 KiB/s wr, 18 op/s 2023-09-10T02:44:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12313.808500 secs 2023-09-10T02:44:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:44:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:44:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:44:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12315.197333 secs 2023-09-10T02:44:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:44:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:44:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:44:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12315.197333 secs 2023-09-10T02:44:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:44:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:44:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:44:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12315.197333 secs 2023-09-10T02:44:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12315.908842 secs 2023-09-10T02:44:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:44 smithi098 ceph-mon[103124]: pgmap v6730: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 216 KiB/s wr, 13 op/s 2023-09-10T02:44:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12315.908842 secs 2023-09-10T02:44:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:44 smithi170 ceph-mon[102832]: pgmap v6730: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 216 KiB/s wr, 13 op/s 2023-09-10T02:44:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12315.908842 secs 2023-09-10T02:44:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:44 smithi082 ceph-mon[98694]: pgmap v6730: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 216 KiB/s wr, 13 op/s 2023-09-10T02:44:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:46 smithi098 ceph-mon[103124]: pgmap v6731: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 273 KiB/s wr, 18 op/s 2023-09-10T02:44:47.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:46 smithi170 ceph-mon[102832]: pgmap v6731: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 273 KiB/s wr, 18 op/s 2023-09-10T02:44:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:46 smithi082 ceph-mon[98694]: pgmap v6731: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 273 KiB/s wr, 18 op/s 2023-09-10T02:44:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12318.808559 secs 2023-09-10T02:44:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12318.808559 secs 2023-09-10T02:44:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12318.808559 secs 2023-09-10T02:44:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:48 smithi098 ceph-mon[103124]: pgmap v6732: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T02:44:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12320.197391 secs 2023-09-10T02:44:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:48 smithi170 ceph-mon[102832]: pgmap v6732: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T02:44:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12320.197391 secs 2023-09-10T02:44:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:48 smithi082 ceph-mon[98694]: pgmap v6732: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T02:44:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12320.197391 secs 2023-09-10T02:44:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12320.908908 secs 2023-09-10T02:44:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12320.908908 secs 2023-09-10T02:44:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12320.908908 secs 2023-09-10T02:44:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:50 smithi098 ceph-mon[103124]: pgmap v6733: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T02:44:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:50 smithi170 ceph-mon[102832]: pgmap v6733: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T02:44:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:50 smithi082 ceph-mon[98694]: pgmap v6733: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T02:44:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:52 smithi170 ceph-mon[102832]: pgmap v6734: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 223 KiB/s wr, 18 op/s 2023-09-10T02:44:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12323.808621 secs 2023-09-10T02:44:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:52 smithi082 ceph-mon[98694]: pgmap v6734: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 223 KiB/s wr, 18 op/s 2023-09-10T02:44:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12323.808621 secs 2023-09-10T02:44:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:52 smithi098 ceph-mon[103124]: pgmap v6734: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 223 KiB/s wr, 18 op/s 2023-09-10T02:44:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12323.808621 secs 2023-09-10T02:44:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12325.197450 secs 2023-09-10T02:44:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12325.197450 secs 2023-09-10T02:44:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12325.197450 secs 2023-09-10T02:44:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12325.908973 secs 2023-09-10T02:44:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:54 smithi082 ceph-mon[98694]: pgmap v6735: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T02:44:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12325.908973 secs 2023-09-10T02:44:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:54 smithi098 ceph-mon[103124]: pgmap v6735: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T02:44:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12325.908973 secs 2023-09-10T02:44:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:54 smithi170 ceph-mon[102832]: pgmap v6735: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T02:44:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:56 smithi082 ceph-mon[98694]: pgmap v6736: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 153 KiB/s wr, 15 op/s 2023-09-10T02:44:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:56 smithi098 ceph-mon[103124]: pgmap v6736: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 153 KiB/s wr, 15 op/s 2023-09-10T02:44:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:56 smithi170 ceph-mon[102832]: pgmap v6736: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 153 KiB/s wr, 15 op/s 2023-09-10T02:44:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12328.808682 secs 2023-09-10T02:44:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12328.808682 secs 2023-09-10T02:44:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12328.808682 secs 2023-09-10T02:44:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:58 smithi082 ceph-mon[98694]: pgmap v6737: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T02:44:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12330.197525 secs 2023-09-10T02:44:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:58 smithi098 ceph-mon[103124]: pgmap v6737: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T02:44:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12330.197525 secs 2023-09-10T02:44:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:58 smithi170 ceph-mon[102832]: pgmap v6737: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T02:44:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12330.197525 secs 2023-09-10T02:45:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12330.909039 secs 2023-09-10T02:45:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:45:00.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:45:00.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:45:00.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:44:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:45:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12330.909039 secs 2023-09-10T02:45:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:45:00.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:45:00.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:45:00.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:44:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:45:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12330.909039 secs 2023-09-10T02:45:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:45:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:45:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:45:00.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:44:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:45:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:00 smithi082 ceph-mon[98694]: pgmap v6738: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T02:45:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:00 smithi098 ceph-mon[103124]: pgmap v6738: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T02:45:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:00 smithi170 ceph-mon[102832]: pgmap v6738: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T02:45:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:02 smithi170 ceph-mon[102832]: pgmap v6739: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 125 KiB/s wr, 15 op/s 2023-09-10T02:45:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12333.831743 secs 2023-09-10T02:45:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:02 smithi082 ceph-mon[98694]: pgmap v6739: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 125 KiB/s wr, 15 op/s 2023-09-10T02:45:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12333.831743 secs 2023-09-10T02:45:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:02 smithi098 ceph-mon[103124]: pgmap v6739: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 125 KiB/s wr, 15 op/s 2023-09-10T02:45:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12333.831743 secs 2023-09-10T02:45:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12335.197583 secs 2023-09-10T02:45:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12335.197583 secs 2023-09-10T02:45:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12335.197583 secs 2023-09-10T02:45:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12335.909103 secs 2023-09-10T02:45:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:04 smithi082 ceph-mon[98694]: pgmap v6740: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 9 op/s 2023-09-10T02:45:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12335.909103 secs 2023-09-10T02:45:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:04 smithi098 ceph-mon[103124]: pgmap v6740: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 9 op/s 2023-09-10T02:45:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12335.909103 secs 2023-09-10T02:45:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:04 smithi170 ceph-mon[102832]: pgmap v6740: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 9 op/s 2023-09-10T02:45:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:06 smithi082 ceph-mon[98694]: pgmap v6741: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 118 KiB/s wr, 29 op/s 2023-09-10T02:45:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:06 smithi098 ceph-mon[103124]: pgmap v6741: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 118 KiB/s wr, 29 op/s 2023-09-10T02:45:07.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:06 smithi170 ceph-mon[102832]: pgmap v6741: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 118 KiB/s wr, 29 op/s 2023-09-10T02:45:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12338.831777 secs 2023-09-10T02:45:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12338.831777 secs 2023-09-10T02:45:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12338.831777 secs 2023-09-10T02:45:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:08 smithi082 ceph-mon[98694]: pgmap v6742: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 85 KiB/s wr, 24 op/s 2023-09-10T02:45:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12340.197642 secs 2023-09-10T02:45:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:08 smithi082 ceph-mon[98694]: osdmap e489: 12 total, 12 up, 12 in 2023-09-10T02:45:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:08 smithi098 ceph-mon[103124]: pgmap v6742: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 85 KiB/s wr, 24 op/s 2023-09-10T02:45:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12340.197642 secs 2023-09-10T02:45:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:08 smithi098 ceph-mon[103124]: osdmap e489: 12 total, 12 up, 12 in 2023-09-10T02:45:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:08 smithi170 ceph-mon[102832]: pgmap v6742: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 85 KiB/s wr, 24 op/s 2023-09-10T02:45:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12340.197642 secs 2023-09-10T02:45:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:08 smithi170 ceph-mon[102832]: osdmap e489: 12 total, 12 up, 12 in 2023-09-10T02:45:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12340.909170 secs 2023-09-10T02:45:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12340.909170 secs 2023-09-10T02:45:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12340.909170 secs 2023-09-10T02:45:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:10 smithi082 ceph-mon[98694]: pgmap v6744: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T02:45:11.831 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:10 smithi170 ceph-mon[102832]: pgmap v6744: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T02:45:11.832 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:10 smithi098 ceph-mon[103124]: pgmap v6744: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T02:45:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:12 smithi170 ceph-mon[102832]: pgmap v6745: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T02:45:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12343.831836 secs 2023-09-10T02:45:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:12 smithi082 ceph-mon[98694]: pgmap v6745: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T02:45:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12343.831836 secs 2023-09-10T02:45:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:12 smithi098 ceph-mon[103124]: pgmap v6745: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T02:45:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12343.831836 secs 2023-09-10T02:45:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12345.197700 secs 2023-09-10T02:45:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12345.197700 secs 2023-09-10T02:45:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12345.197700 secs 2023-09-10T02:45:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12345.909234 secs 2023-09-10T02:45:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:14 smithi082 ceph-mon[98694]: pgmap v6746: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T02:45:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12345.909234 secs 2023-09-10T02:45:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:14 smithi098 ceph-mon[103124]: pgmap v6746: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T02:45:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12345.909234 secs 2023-09-10T02:45:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:14 smithi170 ceph-mon[102832]: pgmap v6746: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T02:45:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:16 smithi082 ceph-mon[98694]: pgmap v6747: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T02:45:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:16 smithi082 ceph-mon[98694]: osdmap e490: 12 total, 12 up, 12 in 2023-09-10T02:45:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:16 smithi098 ceph-mon[103124]: pgmap v6747: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T02:45:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:16 smithi098 ceph-mon[103124]: osdmap e490: 12 total, 12 up, 12 in 2023-09-10T02:45:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:16 smithi170 ceph-mon[102832]: pgmap v6747: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T02:45:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:16 smithi170 ceph-mon[102832]: osdmap e490: 12 total, 12 up, 12 in 2023-09-10T02:45:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12348.831973 secs 2023-09-10T02:45:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12348.831973 secs 2023-09-10T02:45:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12348.831973 secs 2023-09-10T02:45:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:18 smithi082 ceph-mon[98694]: pgmap v6749: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T02:45:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12350.197757 secs 2023-09-10T02:45:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:18 smithi098 ceph-mon[103124]: pgmap v6749: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T02:45:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12350.197757 secs 2023-09-10T02:45:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:18 smithi170 ceph-mon[102832]: pgmap v6749: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T02:45:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12350.197757 secs 2023-09-10T02:45:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12350.909300 secs 2023-09-10T02:45:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12350.909300 secs 2023-09-10T02:45:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12350.909300 secs 2023-09-10T02:45:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:20 smithi082 ceph-mon[98694]: pgmap v6750: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T02:45:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:20 smithi098 ceph-mon[103124]: pgmap v6750: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T02:45:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:20 smithi170 ceph-mon[102832]: pgmap v6750: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 11 op/s 2023-09-10T02:45:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:22 smithi082 ceph-mon[98694]: pgmap v6751: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T02:45:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:45:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:22 smithi098 ceph-mon[103124]: pgmap v6751: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T02:45:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:45:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:22 smithi170 ceph-mon[102832]: pgmap v6751: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T02:45:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:45:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:45:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12355.197816 secs 2023-09-10T02:45:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:45:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:45:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:45:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12355.197816 secs 2023-09-10T02:45:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:45:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:45:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:45:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12355.197816 secs 2023-09-10T02:45:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:45:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:45:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12355.909363 secs 2023-09-10T02:45:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:24 smithi082 ceph-mon[98694]: pgmap v6752: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T02:45:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12355.909363 secs 2023-09-10T02:45:25.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:24 smithi098 ceph-mon[103124]: pgmap v6752: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T02:45:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12355.909363 secs 2023-09-10T02:45:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:24 smithi170 ceph-mon[102832]: pgmap v6752: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T02:45:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:26 smithi082 ceph-mon[98694]: pgmap v6753: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T02:45:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:26 smithi098 ceph-mon[103124]: pgmap v6753: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T02:45:27.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:26 smithi170 ceph-mon[102832]: pgmap v6753: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T02:45:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12358.832269 secs 2023-09-10T02:45:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12358.832269 secs 2023-09-10T02:45:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12358.832269 secs 2023-09-10T02:45:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:28 smithi082 ceph-mon[98694]: pgmap v6754: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T02:45:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12360.197873 secs 2023-09-10T02:45:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:28 smithi098 ceph-mon[103124]: pgmap v6754: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T02:45:29.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12360.197873 secs 2023-09-10T02:45:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:28 smithi170 ceph-mon[102832]: pgmap v6754: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T02:45:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12360.197873 secs 2023-09-10T02:45:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12360.909428 secs 2023-09-10T02:45:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12360.909428 secs 2023-09-10T02:45:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12360.909428 secs 2023-09-10T02:45:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:30 smithi098 ceph-mon[103124]: pgmap v6755: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T02:45:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:30 smithi170 ceph-mon[102832]: pgmap v6755: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T02:45:31.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:30 smithi082 ceph-mon[98694]: pgmap v6755: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T02:45:32.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:31 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:45:32.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:45:32.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:45:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:31 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:45:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:45:32.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:45:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:31 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:45:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:45:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:45:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:32 smithi098 ceph-mon[103124]: pgmap v6756: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 161 KiB/s wr, 16 op/s 2023-09-10T02:45:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12363.832332 secs 2023-09-10T02:45:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:32 smithi170 ceph-mon[102832]: pgmap v6756: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 161 KiB/s wr, 16 op/s 2023-09-10T02:45:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12363.832332 secs 2023-09-10T02:45:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:32 smithi082 ceph-mon[98694]: pgmap v6756: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 161 KiB/s wr, 16 op/s 2023-09-10T02:45:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12363.832332 secs 2023-09-10T02:45:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12365.197933 secs 2023-09-10T02:45:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12365.197933 secs 2023-09-10T02:45:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12365.197933 secs 2023-09-10T02:45:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12365.909491 secs 2023-09-10T02:45:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:34 smithi098 ceph-mon[103124]: pgmap v6757: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T02:45:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12365.909491 secs 2023-09-10T02:45:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:34 smithi170 ceph-mon[102832]: pgmap v6757: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T02:45:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12365.909491 secs 2023-09-10T02:45:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:34 smithi082 ceph-mon[98694]: pgmap v6757: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T02:45:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:36 smithi098 ceph-mon[103124]: pgmap v6758: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 192 KiB/s wr, 17 op/s 2023-09-10T02:45:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:36 smithi170 ceph-mon[102832]: pgmap v6758: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 192 KiB/s wr, 17 op/s 2023-09-10T02:45:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:36 smithi082 ceph-mon[98694]: pgmap v6758: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 192 KiB/s wr, 17 op/s 2023-09-10T02:45:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:38 smithi098 ceph-mon[103124]: pgmap v6759: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 161 KiB/s wr, 12 op/s 2023-09-10T02:45:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12370.198001 secs 2023-09-10T02:45:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:38 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:45:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:38 smithi170 ceph-mon[102832]: pgmap v6759: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 161 KiB/s wr, 12 op/s 2023-09-10T02:45:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12370.198001 secs 2023-09-10T02:45:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:38 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:45:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:38 smithi082 ceph-mon[98694]: pgmap v6759: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 161 KiB/s wr, 12 op/s 2023-09-10T02:45:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12370.198001 secs 2023-09-10T02:45:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:38 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:45:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12370.909558 secs 2023-09-10T02:45:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:39 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:45:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:39 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:45:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:39 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:45:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12370.909558 secs 2023-09-10T02:45:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:39 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:45:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:39 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:45:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:39 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:45:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12370.909558 secs 2023-09-10T02:45:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:39 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:45:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:39 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:45:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:39 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:45:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:40 smithi098 ceph-mon[103124]: pgmap v6760: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 161 KiB/s wr, 12 op/s 2023-09-10T02:45:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:40 smithi170 ceph-mon[102832]: pgmap v6760: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 161 KiB/s wr, 12 op/s 2023-09-10T02:45:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:40 smithi082 ceph-mon[98694]: pgmap v6760: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 161 KiB/s wr, 12 op/s 2023-09-10T02:45:43.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:42 smithi082 ceph-mon[98694]: pgmap v6761: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 245 KiB/s wr, 18 op/s 2023-09-10T02:45:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:42 smithi098 ceph-mon[103124]: pgmap v6761: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 245 KiB/s wr, 18 op/s 2023-09-10T02:45:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:42 smithi170 ceph-mon[102832]: pgmap v6761: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 245 KiB/s wr, 18 op/s 2023-09-10T02:45:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12375.198059 secs 2023-09-10T02:45:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12375.198059 secs 2023-09-10T02:45:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12375.198059 secs 2023-09-10T02:45:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12375.909621 secs 2023-09-10T02:45:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:44 smithi098 ceph-mon[103124]: pgmap v6762: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T02:45:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12375.909621 secs 2023-09-10T02:45:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:44 smithi170 ceph-mon[102832]: pgmap v6762: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T02:45:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12375.909621 secs 2023-09-10T02:45:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:44 smithi082 ceph-mon[98694]: pgmap v6762: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T02:45:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:46 smithi098 ceph-mon[103124]: pgmap v6763: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 226 KiB/s wr, 17 op/s 2023-09-10T02:45:47.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:46 smithi170 ceph-mon[102832]: pgmap v6763: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 226 KiB/s wr, 17 op/s 2023-09-10T02:45:47.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:46 smithi082 ceph-mon[98694]: pgmap v6763: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 226 KiB/s wr, 17 op/s 2023-09-10T02:45:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12378.832658 secs 2023-09-10T02:45:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12378.832658 secs 2023-09-10T02:45:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12378.832658 secs 2023-09-10T02:45:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12380.198116 secs 2023-09-10T02:45:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:48 smithi098 ceph-mon[103124]: pgmap v6764: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 160 KiB/s wr, 11 op/s 2023-09-10T02:45:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12380.198116 secs 2023-09-10T02:45:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:48 smithi170 ceph-mon[102832]: pgmap v6764: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 160 KiB/s wr, 11 op/s 2023-09-10T02:45:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12380.198116 secs 2023-09-10T02:45:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:48 smithi082 ceph-mon[98694]: pgmap v6764: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 160 KiB/s wr, 11 op/s 2023-09-10T02:45:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12380.909683 secs 2023-09-10T02:45:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12380.909683 secs 2023-09-10T02:45:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12380.909683 secs 2023-09-10T02:45:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:51 smithi170 ceph-mon[102832]: pgmap v6765: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 160 KiB/s wr, 11 op/s 2023-09-10T02:45:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:51 smithi082 ceph-mon[98694]: pgmap v6765: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 160 KiB/s wr, 11 op/s 2023-09-10T02:45:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:51 smithi098 ceph-mon[103124]: pgmap v6765: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 160 KiB/s wr, 11 op/s 2023-09-10T02:45:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:45:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:45:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:45:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:45:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:45:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:45:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:45:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:45:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:45:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:53 smithi170 ceph-mon[102832]: pgmap v6766: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 232 KiB/s wr, 17 op/s 2023-09-10T02:45:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12383.832722 secs 2023-09-10T02:45:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:53 smithi082 ceph-mon[98694]: pgmap v6766: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 232 KiB/s wr, 17 op/s 2023-09-10T02:45:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12383.832722 secs 2023-09-10T02:45:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:53 smithi098 ceph-mon[103124]: pgmap v6766: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 232 KiB/s wr, 17 op/s 2023-09-10T02:45:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12383.832722 secs 2023-09-10T02:45:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12385.198175 secs 2023-09-10T02:45:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12385.198175 secs 2023-09-10T02:45:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12385.198175 secs 2023-09-10T02:45:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12385.909749 secs 2023-09-10T02:45:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:55 smithi170 ceph-mon[102832]: pgmap v6767: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T02:45:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12385.909749 secs 2023-09-10T02:45:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:55 smithi082 ceph-mon[98694]: pgmap v6767: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T02:45:55.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12385.909749 secs 2023-09-10T02:45:55.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:55 smithi098 ceph-mon[103124]: pgmap v6767: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T02:45:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:57 smithi170 ceph-mon[102832]: pgmap v6768: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 196 KiB/s wr, 16 op/s 2023-09-10T02:45:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:57 smithi082 ceph-mon[98694]: pgmap v6768: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 196 KiB/s wr, 16 op/s 2023-09-10T02:45:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:57 smithi098 ceph-mon[103124]: pgmap v6768: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 196 KiB/s wr, 16 op/s 2023-09-10T02:45:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12388.832815 secs 2023-09-10T02:45:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12388.832815 secs 2023-09-10T02:45:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12388.832815 secs 2023-09-10T02:45:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12390.198234 secs 2023-09-10T02:45:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:45:59 smithi170 ceph-mon[102832]: pgmap v6769: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T02:45:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12390.198234 secs 2023-09-10T02:45:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:45:59 smithi082 ceph-mon[98694]: pgmap v6769: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T02:45:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12390.198234 secs 2023-09-10T02:45:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:45:59 smithi098 ceph-mon[103124]: pgmap v6769: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T02:46:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12390.909817 secs 2023-09-10T02:46:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12390.909817 secs 2023-09-10T02:46:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12390.909817 secs 2023-09-10T02:46:01.405 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:01 smithi082 ceph-mon[98694]: pgmap v6770: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T02:46:01.406 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:46:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:01 smithi170 ceph-mon[102832]: pgmap v6770: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T02:46:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:46:01.497 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:01 smithi098 ceph-mon[103124]: pgmap v6770: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T02:46:01.498 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:46:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:02 smithi082 ceph-mon[98694]: pgmap v6771: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 160 KiB/s wr, 17 op/s 2023-09-10T02:46:03.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12393.832893 secs 2023-09-10T02:46:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:02 smithi098 ceph-mon[103124]: pgmap v6771: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 160 KiB/s wr, 17 op/s 2023-09-10T02:46:03.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12393.832893 secs 2023-09-10T02:46:03.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:02 smithi170 ceph-mon[102832]: pgmap v6771: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 160 KiB/s wr, 17 op/s 2023-09-10T02:46:03.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12393.832893 secs 2023-09-10T02:46:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12395.198296 secs 2023-09-10T02:46:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:46:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:46:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12395.198296 secs 2023-09-10T02:46:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:46:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:46:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12395.198296 secs 2023-09-10T02:46:04.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:46:04.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:46:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12395.909882 secs 2023-09-10T02:46:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:04 smithi082 ceph-mon[98694]: pgmap v6772: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T02:46:05.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:46:05.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:46:05.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:46:05.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:46:05.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:46:05.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:46:05.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:46:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12395.909882 secs 2023-09-10T02:46:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:04 smithi098 ceph-mon[103124]: pgmap v6772: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T02:46:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:46:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:46:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:46:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:46:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:46:05.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:46:05.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:46:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12395.909882 secs 2023-09-10T02:46:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:04 smithi170 ceph-mon[102832]: pgmap v6772: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T02:46:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:46:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:46:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:46:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:46:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:46:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:46:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:46:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:06 smithi082 ceph-mon[98694]: pgmap v6773: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 137 KiB/s wr, 31 op/s 2023-09-10T02:46:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:06 smithi098 ceph-mon[103124]: pgmap v6773: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 137 KiB/s wr, 31 op/s 2023-09-10T02:46:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:06 smithi170 ceph-mon[102832]: pgmap v6773: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 137 KiB/s wr, 31 op/s 2023-09-10T02:46:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12398.832954 secs 2023-09-10T02:46:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12398.832954 secs 2023-09-10T02:46:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12398.832954 secs 2023-09-10T02:46:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12400.198379 secs 2023-09-10T02:46:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:08 smithi082 ceph-mon[98694]: pgmap v6774: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 90 KiB/s wr, 26 op/s 2023-09-10T02:46:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:08 smithi082 ceph-mon[98694]: osdmap e491: 12 total, 12 up, 12 in 2023-09-10T02:46:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12400.198379 secs 2023-09-10T02:46:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:08 smithi098 ceph-mon[103124]: pgmap v6774: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 90 KiB/s wr, 26 op/s 2023-09-10T02:46:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:08 smithi098 ceph-mon[103124]: osdmap e491: 12 total, 12 up, 12 in 2023-09-10T02:46:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12400.198379 secs 2023-09-10T02:46:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:08 smithi170 ceph-mon[102832]: pgmap v6774: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 90 KiB/s wr, 26 op/s 2023-09-10T02:46:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:08 smithi170 ceph-mon[102832]: osdmap e491: 12 total, 12 up, 12 in 2023-09-10T02:46:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12400.909948 secs 2023-09-10T02:46:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12400.909948 secs 2023-09-10T02:46:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12400.909948 secs 2023-09-10T02:46:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:10 smithi098 ceph-mon[103124]: pgmap v6776: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 107 KiB/s wr, 31 op/s 2023-09-10T02:46:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:10 smithi170 ceph-mon[102832]: pgmap v6776: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 107 KiB/s wr, 31 op/s 2023-09-10T02:46:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:10 smithi082 ceph-mon[98694]: pgmap v6776: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 107 KiB/s wr, 31 op/s 2023-09-10T02:46:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:12 smithi082 ceph-mon[98694]: pgmap v6777: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 84 KiB/s wr, 30 op/s 2023-09-10T02:46:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12403.833014 secs 2023-09-10T02:46:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:12 smithi098 ceph-mon[103124]: pgmap v6777: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 84 KiB/s wr, 30 op/s 2023-09-10T02:46:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12403.833014 secs 2023-09-10T02:46:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:12 smithi170 ceph-mon[102832]: pgmap v6777: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 84 KiB/s wr, 30 op/s 2023-09-10T02:46:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12403.833014 secs 2023-09-10T02:46:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12405.198434 secs 2023-09-10T02:46:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12405.198434 secs 2023-09-10T02:46:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12405.198434 secs 2023-09-10T02:46:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12405.910014 secs 2023-09-10T02:46:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:14 smithi082 ceph-mon[98694]: pgmap v6778: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 84 KiB/s wr, 30 op/s 2023-09-10T02:46:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12405.910014 secs 2023-09-10T02:46:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:14 smithi098 ceph-mon[103124]: pgmap v6778: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 84 KiB/s wr, 30 op/s 2023-09-10T02:46:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12405.910014 secs 2023-09-10T02:46:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:14 smithi170 ceph-mon[102832]: pgmap v6778: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 84 KiB/s wr, 30 op/s 2023-09-10T02:46:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:16 smithi098 ceph-mon[103124]: pgmap v6779: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-10T02:46:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:16 smithi098 ceph-mon[103124]: osdmap e492: 12 total, 12 up, 12 in 2023-09-10T02:46:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:16 smithi170 ceph-mon[102832]: pgmap v6779: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-10T02:46:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:16 smithi170 ceph-mon[102832]: osdmap e492: 12 total, 12 up, 12 in 2023-09-10T02:46:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:16 smithi082 ceph-mon[98694]: pgmap v6779: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-10T02:46:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:16 smithi082 ceph-mon[98694]: osdmap e492: 12 total, 12 up, 12 in 2023-09-10T02:46:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12408.833073 secs 2023-09-10T02:46:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12408.833073 secs 2023-09-10T02:46:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12408.833073 secs 2023-09-10T02:46:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12410.198510 secs 2023-09-10T02:46:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:18 smithi098 ceph-mon[103124]: pgmap v6781: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 13 op/s 2023-09-10T02:46:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12410.198510 secs 2023-09-10T02:46:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:18 smithi170 ceph-mon[102832]: pgmap v6781: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 13 op/s 2023-09-10T02:46:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12410.198510 secs 2023-09-10T02:46:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:18 smithi082 ceph-mon[98694]: pgmap v6781: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 13 op/s 2023-09-10T02:46:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12410.910078 secs 2023-09-10T02:46:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12410.910078 secs 2023-09-10T02:46:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12410.910078 secs 2023-09-10T02:46:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:20 smithi098 ceph-mon[103124]: pgmap v6782: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-10T02:46:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:20 smithi170 ceph-mon[102832]: pgmap v6782: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-10T02:46:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:20 smithi082 ceph-mon[98694]: pgmap v6782: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-10T02:46:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:22 smithi082 ceph-mon[98694]: pgmap v6783: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 KiB/s wr, 12 op/s 2023-09-10T02:46:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:46:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:22 smithi098 ceph-mon[103124]: pgmap v6783: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 KiB/s wr, 12 op/s 2023-09-10T02:46:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:46:23.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:22 smithi170 ceph-mon[102832]: pgmap v6783: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 KiB/s wr, 12 op/s 2023-09-10T02:46:23.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:46:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12415.198597 secs 2023-09-10T02:46:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:46:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:46:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:46:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12415.198597 secs 2023-09-10T02:46:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:46:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:46:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:46:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12415.198597 secs 2023-09-10T02:46:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:46:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:46:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:46:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12415.910143 secs 2023-09-10T02:46:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:24 smithi082 ceph-mon[98694]: pgmap v6784: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 KiB/s wr, 12 op/s 2023-09-10T02:46:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12415.910143 secs 2023-09-10T02:46:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:24 smithi098 ceph-mon[103124]: pgmap v6784: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 KiB/s wr, 12 op/s 2023-09-10T02:46:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12415.910143 secs 2023-09-10T02:46:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:24 smithi170 ceph-mon[102832]: pgmap v6784: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 KiB/s wr, 12 op/s 2023-09-10T02:46:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:26 smithi098 ceph-mon[103124]: pgmap v6785: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T02:46:27.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:26 smithi170 ceph-mon[102832]: pgmap v6785: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T02:46:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:26 smithi082 ceph-mon[98694]: pgmap v6785: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T02:46:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12420.198675 secs 2023-09-10T02:46:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:28 smithi098 ceph-mon[103124]: pgmap v6786: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T02:46:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12420.198675 secs 2023-09-10T02:46:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:28 smithi170 ceph-mon[102832]: pgmap v6786: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T02:46:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12420.198675 secs 2023-09-10T02:46:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:28 smithi082 ceph-mon[98694]: pgmap v6786: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T02:46:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12420.910210 secs 2023-09-10T02:46:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12420.910210 secs 2023-09-10T02:46:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12420.910210 secs 2023-09-10T02:46:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:30 smithi098 ceph-mon[103124]: pgmap v6787: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 77 KiB/s wr, 10 op/s 2023-09-10T02:46:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:30 smithi170 ceph-mon[102832]: pgmap v6787: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 77 KiB/s wr, 10 op/s 2023-09-10T02:46:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:30 smithi082 ceph-mon[98694]: pgmap v6787: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 77 KiB/s wr, 10 op/s 2023-09-10T02:46:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:32 smithi082 ceph-mon[98694]: pgmap v6788: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 155 KiB/s wr, 16 op/s 2023-09-10T02:46:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12423.833258 secs 2023-09-10T02:46:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:32 smithi098 ceph-mon[103124]: pgmap v6788: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 155 KiB/s wr, 16 op/s 2023-09-10T02:46:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12423.833258 secs 2023-09-10T02:46:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:32 smithi170 ceph-mon[102832]: pgmap v6788: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 155 KiB/s wr, 16 op/s 2023-09-10T02:46:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12423.833258 secs 2023-09-10T02:46:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12425.198759 secs 2023-09-10T02:46:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12425.198759 secs 2023-09-10T02:46:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12425.198759 secs 2023-09-10T02:46:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12425.910275 secs 2023-09-10T02:46:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:34 smithi082 ceph-mon[98694]: pgmap v6789: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T02:46:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12425.910275 secs 2023-09-10T02:46:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:34 smithi098 ceph-mon[103124]: pgmap v6789: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T02:46:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12425.910275 secs 2023-09-10T02:46:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:34 smithi170 ceph-mon[102832]: pgmap v6789: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T02:46:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:35 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:46:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:35 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:46:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:35 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:46:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:35 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:46:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:35 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:46:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:35 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:46:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:35 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:46:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:35 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:46:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:35 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:46:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:36 smithi098 ceph-mon[103124]: pgmap v6790: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 239 KiB/s wr, 18 op/s 2023-09-10T02:46:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:36 smithi170 ceph-mon[102832]: pgmap v6790: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 239 KiB/s wr, 18 op/s 2023-09-10T02:46:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:36 smithi082 ceph-mon[98694]: pgmap v6790: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 239 KiB/s wr, 18 op/s 2023-09-10T02:46:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12430.198818 secs 2023-09-10T02:46:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:38 smithi098 ceph-mon[103124]: pgmap v6791: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 162 KiB/s wr, 12 op/s 2023-09-10T02:46:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:38 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:46:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12430.198818 secs 2023-09-10T02:46:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:38 smithi170 ceph-mon[102832]: pgmap v6791: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 162 KiB/s wr, 12 op/s 2023-09-10T02:46:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:38 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:46:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12430.198818 secs 2023-09-10T02:46:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:38 smithi082 ceph-mon[98694]: pgmap v6791: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 162 KiB/s wr, 12 op/s 2023-09-10T02:46:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:38 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:46:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12430.910340 secs 2023-09-10T02:46:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:39 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:46:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:39 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:46:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12430.910340 secs 2023-09-10T02:46:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:39 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:46:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:39 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:46:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12430.910340 secs 2023-09-10T02:46:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:39 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:46:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:39 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:46:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:40 smithi098 ceph-mon[103124]: pgmap v6792: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 162 KiB/s wr, 12 op/s 2023-09-10T02:46:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:40 smithi170 ceph-mon[102832]: pgmap v6792: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 162 KiB/s wr, 12 op/s 2023-09-10T02:46:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:40 smithi082 ceph-mon[98694]: pgmap v6792: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 162 KiB/s wr, 12 op/s 2023-09-10T02:46:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:41 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:46:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:41 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:46:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:41 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:46:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:42 smithi098 ceph-mon[103124]: pgmap v6793: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 278 KiB/s wr, 18 op/s 2023-09-10T02:46:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12433.833402 secs 2023-09-10T02:46:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:42 smithi170 ceph-mon[102832]: pgmap v6793: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 278 KiB/s wr, 18 op/s 2023-09-10T02:46:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12433.833402 secs 2023-09-10T02:46:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:42 smithi082 ceph-mon[98694]: pgmap v6793: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 278 KiB/s wr, 18 op/s 2023-09-10T02:46:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12433.833402 secs 2023-09-10T02:46:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12435.198874 secs 2023-09-10T02:46:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:46:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:46:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12435.198874 secs 2023-09-10T02:46:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:46:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:46:44.470 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12435.198874 secs 2023-09-10T02:46:44.470 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:46:44.470 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:46:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12435.910405 secs 2023-09-10T02:46:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:44 smithi098 ceph-mon[103124]: pgmap v6794: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 199 KiB/s wr, 12 op/s 2023-09-10T02:46:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12435.910405 secs 2023-09-10T02:46:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:44 smithi170 ceph-mon[102832]: pgmap v6794: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 199 KiB/s wr, 12 op/s 2023-09-10T02:46:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12435.910405 secs 2023-09-10T02:46:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:44 smithi082 ceph-mon[98694]: pgmap v6794: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 199 KiB/s wr, 12 op/s 2023-09-10T02:46:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:47 smithi098 ceph-mon[103124]: pgmap v6795: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 333 KiB/s wr, 18 op/s 2023-09-10T02:46:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:47 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:46:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:47 smithi170 ceph-mon[102832]: pgmap v6795: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 333 KiB/s wr, 18 op/s 2023-09-10T02:46:47.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:47 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:46:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:47 smithi082 ceph-mon[98694]: pgmap v6795: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 333 KiB/s wr, 18 op/s 2023-09-10T02:46:47.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:47 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:46:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12438.833466 secs 2023-09-10T02:46:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12438.833466 secs 2023-09-10T02:46:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12438.833466 secs 2023-09-10T02:46:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12440.198933 secs 2023-09-10T02:46:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:49 smithi170 ceph-mon[102832]: pgmap v6796: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 249 KiB/s wr, 11 op/s 2023-09-10T02:46:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12440.198933 secs 2023-09-10T02:46:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:49 smithi082 ceph-mon[98694]: pgmap v6796: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 249 KiB/s wr, 11 op/s 2023-09-10T02:46:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12440.198933 secs 2023-09-10T02:46:49.522 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:49 smithi098 ceph-mon[103124]: pgmap v6796: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 249 KiB/s wr, 11 op/s 2023-09-10T02:46:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12440.910474 secs 2023-09-10T02:46:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12440.910474 secs 2023-09-10T02:46:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12440.910474 secs 2023-09-10T02:46:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:51 smithi170 ceph-mon[102832]: pgmap v6797: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 249 KiB/s wr, 11 op/s 2023-09-10T02:46:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:51 smithi082 ceph-mon[98694]: pgmap v6797: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 249 KiB/s wr, 11 op/s 2023-09-10T02:46:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:51 smithi098 ceph-mon[103124]: pgmap v6797: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 249 KiB/s wr, 11 op/s 2023-09-10T02:46:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:52 smithi082 ceph-mon[98694]: pgmap v6798: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 322 KiB/s wr, 17 op/s 2023-09-10T02:46:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12443.833526 secs 2023-09-10T02:46:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:52 smithi098 ceph-mon[103124]: pgmap v6798: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 322 KiB/s wr, 17 op/s 2023-09-10T02:46:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12443.833526 secs 2023-09-10T02:46:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:52 smithi170 ceph-mon[102832]: pgmap v6798: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 322 KiB/s wr, 17 op/s 2023-09-10T02:46:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12443.833526 secs 2023-09-10T02:46:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12445.198990 secs 2023-09-10T02:46:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12445.198990 secs 2023-09-10T02:46:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12445.198990 secs 2023-09-10T02:46:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12445.910538 secs 2023-09-10T02:46:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:54 smithi098 ceph-mon[103124]: pgmap v6799: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 207 KiB/s wr, 11 op/s 2023-09-10T02:46:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12445.910538 secs 2023-09-10T02:46:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:54 smithi170 ceph-mon[102832]: pgmap v6799: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 207 KiB/s wr, 11 op/s 2023-09-10T02:46:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12445.910538 secs 2023-09-10T02:46:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:54 smithi082 ceph-mon[98694]: pgmap v6799: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 207 KiB/s wr, 11 op/s 2023-09-10T02:46:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:56 smithi098 ceph-mon[103124]: pgmap v6800: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 207 KiB/s wr, 15 op/s 2023-09-10T02:46:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:56 smithi170 ceph-mon[102832]: pgmap v6800: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 207 KiB/s wr, 15 op/s 2023-09-10T02:46:57.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:56 smithi082 ceph-mon[98694]: pgmap v6800: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 207 KiB/s wr, 15 op/s 2023-09-10T02:46:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12448.833614 secs 2023-09-10T02:46:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12448.833614 secs 2023-09-10T02:46:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12448.833614 secs 2023-09-10T02:46:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12450.199047 secs 2023-09-10T02:46:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:58 smithi098 ceph-mon[103124]: pgmap v6801: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 9 op/s 2023-09-10T02:46:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12450.199047 secs 2023-09-10T02:46:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:58 smithi170 ceph-mon[102832]: pgmap v6801: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 9 op/s 2023-09-10T02:46:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12450.199047 secs 2023-09-10T02:46:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:58 smithi082 ceph-mon[98694]: pgmap v6801: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 9 op/s 2023-09-10T02:47:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:46:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12450.910605 secs 2023-09-10T02:47:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:46:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12450.910605 secs 2023-09-10T02:47:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:46:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12450.910605 secs 2023-09-10T02:47:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:00 smithi098 ceph-mon[103124]: pgmap v6802: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 9 op/s 2023-09-10T02:47:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:00 smithi170 ceph-mon[102832]: pgmap v6802: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 9 op/s 2023-09-10T02:47:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:00 smithi082 ceph-mon[98694]: pgmap v6802: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 9 op/s 2023-09-10T02:47:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:02 smithi098 ceph-mon[103124]: pgmap v6803: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 142 KiB/s wr, 29 op/s 2023-09-10T02:47:03.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12453.833699 secs 2023-09-10T02:47:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:02 smithi170 ceph-mon[102832]: pgmap v6803: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 142 KiB/s wr, 29 op/s 2023-09-10T02:47:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12453.833699 secs 2023-09-10T02:47:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:02 smithi082 ceph-mon[98694]: pgmap v6803: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 142 KiB/s wr, 29 op/s 2023-09-10T02:47:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12453.833699 secs 2023-09-10T02:47:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12455.199133 secs 2023-09-10T02:47:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12455.199133 secs 2023-09-10T02:47:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12455.199133 secs 2023-09-10T02:47:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12455.910673 secs 2023-09-10T02:47:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:04 smithi098 ceph-mon[103124]: pgmap v6804: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 69 KiB/s wr, 23 op/s 2023-09-10T02:47:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:47:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12455.910673 secs 2023-09-10T02:47:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:04 smithi170 ceph-mon[102832]: pgmap v6804: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 69 KiB/s wr, 23 op/s 2023-09-10T02:47:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:47:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12455.910673 secs 2023-09-10T02:47:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:04 smithi082 ceph-mon[98694]: pgmap v6804: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 69 KiB/s wr, 23 op/s 2023-09-10T02:47:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:47:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:47:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:47:06.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:47:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:47:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:47:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:47:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:47:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:47:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:47:07.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:07 smithi170 ceph-mon[102832]: pgmap v6805: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 75 KiB/s wr, 29 op/s 2023-09-10T02:47:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:07 smithi082 ceph-mon[98694]: pgmap v6805: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 75 KiB/s wr, 29 op/s 2023-09-10T02:47:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:07 smithi098 ceph-mon[103124]: pgmap v6805: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 75 KiB/s wr, 29 op/s 2023-09-10T02:47:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12458.833759 secs 2023-09-10T02:47:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12458.833759 secs 2023-09-10T02:47:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12458.833759 secs 2023-09-10T02:47:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12460.199220 secs 2023-09-10T02:47:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:09 smithi170 ceph-mon[102832]: pgmap v6806: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 75 KiB/s wr, 25 op/s 2023-09-10T02:47:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:09 smithi170 ceph-mon[102832]: osdmap e493: 12 total, 12 up, 12 in 2023-09-10T02:47:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12460.199220 secs 2023-09-10T02:47:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:09 smithi082 ceph-mon[98694]: pgmap v6806: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 75 KiB/s wr, 25 op/s 2023-09-10T02:47:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:09 smithi082 ceph-mon[98694]: osdmap e493: 12 total, 12 up, 12 in 2023-09-10T02:47:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12460.199220 secs 2023-09-10T02:47:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:09 smithi098 ceph-mon[103124]: pgmap v6806: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 75 KiB/s wr, 25 op/s 2023-09-10T02:47:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:09 smithi098 ceph-mon[103124]: osdmap e493: 12 total, 12 up, 12 in 2023-09-10T02:47:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12460.910736 secs 2023-09-10T02:47:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12460.910736 secs 2023-09-10T02:47:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12460.910736 secs 2023-09-10T02:47:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:11 smithi170 ceph-mon[102832]: pgmap v6808: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 90 KiB/s wr, 30 op/s 2023-09-10T02:47:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:11 smithi082 ceph-mon[98694]: pgmap v6808: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 90 KiB/s wr, 30 op/s 2023-09-10T02:47:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:11 smithi098 ceph-mon[103124]: pgmap v6808: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 90 KiB/s wr, 30 op/s 2023-09-10T02:47:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:12 smithi098 ceph-mon[103124]: pgmap v6809: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 8.4 KiB/s wr, 13 op/s 2023-09-10T02:47:13.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12463.833821 secs 2023-09-10T02:47:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:12 smithi170 ceph-mon[102832]: pgmap v6809: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 8.4 KiB/s wr, 13 op/s 2023-09-10T02:47:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12463.833821 secs 2023-09-10T02:47:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:12 smithi082 ceph-mon[98694]: pgmap v6809: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 8.4 KiB/s wr, 13 op/s 2023-09-10T02:47:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12463.833821 secs 2023-09-10T02:47:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12465.199287 secs 2023-09-10T02:47:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12465.199287 secs 2023-09-10T02:47:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12465.199287 secs 2023-09-10T02:47:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12465.910799 secs 2023-09-10T02:47:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:14 smithi098 ceph-mon[103124]: pgmap v6810: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 8.4 KiB/s wr, 13 op/s 2023-09-10T02:47:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12465.910799 secs 2023-09-10T02:47:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:14 smithi170 ceph-mon[102832]: pgmap v6810: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 8.4 KiB/s wr, 13 op/s 2023-09-10T02:47:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12465.910799 secs 2023-09-10T02:47:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:14 smithi082 ceph-mon[98694]: pgmap v6810: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 8.4 KiB/s wr, 13 op/s 2023-09-10T02:47:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:16 smithi098 ceph-mon[103124]: pgmap v6811: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 KiB/s wr, 12 op/s 2023-09-10T02:47:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:16 smithi098 ceph-mon[103124]: osdmap e494: 12 total, 12 up, 12 in 2023-09-10T02:47:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:16 smithi170 ceph-mon[102832]: pgmap v6811: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 KiB/s wr, 12 op/s 2023-09-10T02:47:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:16 smithi170 ceph-mon[102832]: osdmap e494: 12 total, 12 up, 12 in 2023-09-10T02:47:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:16 smithi082 ceph-mon[98694]: pgmap v6811: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 KiB/s wr, 12 op/s 2023-09-10T02:47:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:16 smithi082 ceph-mon[98694]: osdmap e494: 12 total, 12 up, 12 in 2023-09-10T02:47:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12468.833881 secs 2023-09-10T02:47:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12468.833881 secs 2023-09-10T02:47:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12468.833881 secs 2023-09-10T02:47:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12470.199362 secs 2023-09-10T02:47:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:18 smithi098 ceph-mon[103124]: pgmap v6813: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T02:47:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12470.199362 secs 2023-09-10T02:47:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:18 smithi170 ceph-mon[102832]: pgmap v6813: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T02:47:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12470.199362 secs 2023-09-10T02:47:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:18 smithi082 ceph-mon[98694]: pgmap v6813: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 12 op/s 2023-09-10T02:47:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12470.910863 secs 2023-09-10T02:47:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12470.910863 secs 2023-09-10T02:47:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12470.910863 secs 2023-09-10T02:47:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:20 smithi098 ceph-mon[103124]: pgmap v6814: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 KiB/s wr, 12 op/s 2023-09-10T02:47:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:20 smithi170 ceph-mon[102832]: pgmap v6814: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 KiB/s wr, 12 op/s 2023-09-10T02:47:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:20 smithi082 ceph-mon[98694]: pgmap v6814: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 KiB/s wr, 12 op/s 2023-09-10T02:47:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:22 smithi098 ceph-mon[103124]: pgmap v6815: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 KiB/s wr, 12 op/s 2023-09-10T02:47:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:47:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:22 smithi170 ceph-mon[102832]: pgmap v6815: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 KiB/s wr, 12 op/s 2023-09-10T02:47:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:47:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:22 smithi082 ceph-mon[98694]: pgmap v6815: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 KiB/s wr, 12 op/s 2023-09-10T02:47:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:47:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12475.199418 secs 2023-09-10T02:47:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:47:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:47:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:47:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12475.199418 secs 2023-09-10T02:47:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:47:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:47:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:47:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12475.199418 secs 2023-09-10T02:47:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:47:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:47:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:47:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12475.910931 secs 2023-09-10T02:47:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:24 smithi098 ceph-mon[103124]: pgmap v6816: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 KiB/s wr, 12 op/s 2023-09-10T02:47:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12475.910931 secs 2023-09-10T02:47:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:24 smithi170 ceph-mon[102832]: pgmap v6816: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 KiB/s wr, 12 op/s 2023-09-10T02:47:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12475.910931 secs 2023-09-10T02:47:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:24 smithi082 ceph-mon[98694]: pgmap v6816: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 KiB/s wr, 12 op/s 2023-09-10T02:47:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:26 smithi098 ceph-mon[103124]: pgmap v6817: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 13 op/s 2023-09-10T02:47:27.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:26 smithi170 ceph-mon[102832]: pgmap v6817: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 13 op/s 2023-09-10T02:47:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:26 smithi082 ceph-mon[98694]: pgmap v6817: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 13 op/s 2023-09-10T02:47:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12480.199497 secs 2023-09-10T02:47:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:28 smithi098 ceph-mon[103124]: pgmap v6818: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T02:47:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12480.199497 secs 2023-09-10T02:47:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:28 smithi170 ceph-mon[102832]: pgmap v6818: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T02:47:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12480.199497 secs 2023-09-10T02:47:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:28 smithi082 ceph-mon[98694]: pgmap v6818: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T02:47:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12480.910997 secs 2023-09-10T02:47:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12480.910997 secs 2023-09-10T02:47:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12480.910997 secs 2023-09-10T02:47:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:30 smithi098 ceph-mon[103124]: pgmap v6819: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 68 KiB/s wr, 10 op/s 2023-09-10T02:47:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:30 smithi170 ceph-mon[102832]: pgmap v6819: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 68 KiB/s wr, 10 op/s 2023-09-10T02:47:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:30 smithi082 ceph-mon[98694]: pgmap v6819: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 68 KiB/s wr, 10 op/s 2023-09-10T02:47:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:32 smithi098 ceph-mon[103124]: pgmap v6820: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 155 KiB/s wr, 17 op/s 2023-09-10T02:47:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12483.834091 secs 2023-09-10T02:47:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:32 smithi170 ceph-mon[102832]: pgmap v6820: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 155 KiB/s wr, 17 op/s 2023-09-10T02:47:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12483.834091 secs 2023-09-10T02:47:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:32 smithi082 ceph-mon[98694]: pgmap v6820: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 155 KiB/s wr, 17 op/s 2023-09-10T02:47:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12483.834091 secs 2023-09-10T02:47:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12485.199556 secs 2023-09-10T02:47:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12485.199556 secs 2023-09-10T02:47:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12485.199556 secs 2023-09-10T02:47:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12485.911061 secs 2023-09-10T02:47:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:34 smithi098 ceph-mon[103124]: pgmap v6821: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T02:47:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12485.911061 secs 2023-09-10T02:47:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:34 smithi170 ceph-mon[102832]: pgmap v6821: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T02:47:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12485.911061 secs 2023-09-10T02:47:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:34 smithi082 ceph-mon[98694]: pgmap v6821: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T02:47:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:35 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:47:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:35 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:47:36.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:35 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:47:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:35 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:47:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:35 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:47:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:35 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:47:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:35 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:47:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:35 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:47:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:35 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:47:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:36 smithi098 ceph-mon[103124]: pgmap v6822: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 231 KiB/s wr, 17 op/s 2023-09-10T02:47:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:36 smithi170 ceph-mon[102832]: pgmap v6822: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 231 KiB/s wr, 17 op/s 2023-09-10T02:47:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:36 smithi082 ceph-mon[98694]: pgmap v6822: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 231 KiB/s wr, 17 op/s 2023-09-10T02:47:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12488.834152 secs 2023-09-10T02:47:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12488.834152 secs 2023-09-10T02:47:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12488.834152 secs 2023-09-10T02:47:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12490.199612 secs 2023-09-10T02:47:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:38 smithi098 ceph-mon[103124]: pgmap v6823: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T02:47:39.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12490.199612 secs 2023-09-10T02:47:39.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:38 smithi170 ceph-mon[102832]: pgmap v6823: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T02:47:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12490.199612 secs 2023-09-10T02:47:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:38 smithi082 ceph-mon[98694]: pgmap v6823: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T02:47:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12490.911126 secs 2023-09-10T02:47:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12490.911126 secs 2023-09-10T02:47:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12490.911126 secs 2023-09-10T02:47:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:41 smithi098 ceph-mon[103124]: pgmap v6824: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T02:47:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:41 smithi170 ceph-mon[102832]: pgmap v6824: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T02:47:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:41 smithi082 ceph-mon[98694]: pgmap v6824: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T02:47:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:42 smithi098 ceph-mon[103124]: pgmap v6825: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 256 KiB/s wr, 17 op/s 2023-09-10T02:47:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12493.834261 secs 2023-09-10T02:47:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:42 smithi170 ceph-mon[102832]: pgmap v6825: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 256 KiB/s wr, 17 op/s 2023-09-10T02:47:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12493.834261 secs 2023-09-10T02:47:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:42 smithi082 ceph-mon[98694]: pgmap v6825: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 256 KiB/s wr, 17 op/s 2023-09-10T02:47:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12493.834261 secs 2023-09-10T02:47:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12495.199670 secs 2023-09-10T02:47:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12495.199670 secs 2023-09-10T02:47:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12495.199670 secs 2023-09-10T02:47:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12495.911192 secs 2023-09-10T02:47:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:44 smithi098 ceph-mon[103124]: pgmap v6826: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 169 KiB/s wr, 11 op/s 2023-09-10T02:47:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12495.911192 secs 2023-09-10T02:47:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:44 smithi170 ceph-mon[102832]: pgmap v6826: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 169 KiB/s wr, 11 op/s 2023-09-10T02:47:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12495.911192 secs 2023-09-10T02:47:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:44 smithi082 ceph-mon[98694]: pgmap v6826: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 169 KiB/s wr, 11 op/s 2023-09-10T02:47:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:46 smithi098 ceph-mon[103124]: pgmap v6827: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 238 KiB/s wr, 17 op/s 2023-09-10T02:47:47.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:46 smithi170 ceph-mon[102832]: pgmap v6827: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 238 KiB/s wr, 17 op/s 2023-09-10T02:47:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:46 smithi082 ceph-mon[98694]: pgmap v6827: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 238 KiB/s wr, 17 op/s 2023-09-10T02:47:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12498.834320 secs 2023-09-10T02:47:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12498.834320 secs 2023-09-10T02:47:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12498.834320 secs 2023-09-10T02:47:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12500.199732 secs 2023-09-10T02:47:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:48 smithi098 ceph-mon[103124]: pgmap v6828: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T02:47:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12500.199732 secs 2023-09-10T02:47:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:48 smithi170 ceph-mon[102832]: pgmap v6828: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T02:47:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12500.199732 secs 2023-09-10T02:47:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:48 smithi082 ceph-mon[98694]: pgmap v6828: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T02:47:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12500.911274 secs 2023-09-10T02:47:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12500.911274 secs 2023-09-10T02:47:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12500.911274 secs 2023-09-10T02:47:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:50 smithi098 ceph-mon[103124]: pgmap v6829: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T02:47:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:50 smithi170 ceph-mon[102832]: pgmap v6829: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T02:47:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:50 smithi082 ceph-mon[98694]: pgmap v6829: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T02:47:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:52 smithi098 ceph-mon[103124]: pgmap v6830: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 232 KiB/s wr, 17 op/s 2023-09-10T02:47:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12503.834381 secs 2023-09-10T02:47:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:52 smithi170 ceph-mon[102832]: pgmap v6830: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 232 KiB/s wr, 17 op/s 2023-09-10T02:47:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12503.834381 secs 2023-09-10T02:47:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:52 smithi082 ceph-mon[98694]: pgmap v6830: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 232 KiB/s wr, 17 op/s 2023-09-10T02:47:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12503.834381 secs 2023-09-10T02:47:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12505.199788 secs 2023-09-10T02:47:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12505.199788 secs 2023-09-10T02:47:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12505.199788 secs 2023-09-10T02:47:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12505.911342 secs 2023-09-10T02:47:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:54 smithi098 ceph-mon[103124]: pgmap v6831: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T02:47:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12505.911342 secs 2023-09-10T02:47:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:54 smithi170 ceph-mon[102832]: pgmap v6831: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T02:47:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12505.911342 secs 2023-09-10T02:47:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:54 smithi082 ceph-mon[98694]: pgmap v6831: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T02:47:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:56 smithi098 ceph-mon[103124]: pgmap v6832: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 187 KiB/s wr, 17 op/s 2023-09-10T02:47:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:56 smithi170 ceph-mon[102832]: pgmap v6832: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 187 KiB/s wr, 17 op/s 2023-09-10T02:47:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:56 smithi082 ceph-mon[98694]: pgmap v6832: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 187 KiB/s wr, 17 op/s 2023-09-10T02:47:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12508.834440 secs 2023-09-10T02:47:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12508.834440 secs 2023-09-10T02:47:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12508.834440 secs 2023-09-10T02:47:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12510.199846 secs 2023-09-10T02:47:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:58 smithi098 ceph-mon[103124]: pgmap v6833: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T02:47:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12510.199846 secs 2023-09-10T02:47:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:58 smithi170 ceph-mon[102832]: pgmap v6833: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T02:47:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12510.199846 secs 2023-09-10T02:47:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:58 smithi082 ceph-mon[98694]: pgmap v6833: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T02:48:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:47:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12510.911408 secs 2023-09-10T02:48:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:47:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12510.911408 secs 2023-09-10T02:48:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:47:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12510.911408 secs 2023-09-10T02:48:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:00 smithi098 ceph-mon[103124]: pgmap v6834: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T02:48:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:00 smithi170 ceph-mon[102832]: pgmap v6834: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T02:48:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:00 smithi082 ceph-mon[98694]: pgmap v6834: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T02:48:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:02 smithi098 ceph-mon[103124]: pgmap v6835: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 170 KiB/s wr, 19 op/s 2023-09-10T02:48:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12513.834502 secs 2023-09-10T02:48:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:02 smithi170 ceph-mon[102832]: pgmap v6835: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 170 KiB/s wr, 19 op/s 2023-09-10T02:48:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12513.834502 secs 2023-09-10T02:48:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:02 smithi082 ceph-mon[98694]: pgmap v6835: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 170 KiB/s wr, 19 op/s 2023-09-10T02:48:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12513.834502 secs 2023-09-10T02:48:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12515.199894 secs 2023-09-10T02:48:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12515.199894 secs 2023-09-10T02:48:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12515.199894 secs 2023-09-10T02:48:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12515.911475 secs 2023-09-10T02:48:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:04 smithi098 ceph-mon[103124]: pgmap v6836: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 100 KiB/s wr, 13 op/s 2023-09-10T02:48:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12515.911475 secs 2023-09-10T02:48:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:04 smithi170 ceph-mon[102832]: pgmap v6836: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 100 KiB/s wr, 13 op/s 2023-09-10T02:48:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12515.911475 secs 2023-09-10T02:48:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:04 smithi082 ceph-mon[98694]: pgmap v6836: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 100 KiB/s wr, 13 op/s 2023-09-10T02:48:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:48:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:48:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:48:06.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:48:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:48:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:48:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:48:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:48:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:48:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:48:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:48:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:48:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:06 smithi098 ceph-mon[103124]: pgmap v6837: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 138 KiB/s wr, 30 op/s 2023-09-10T02:48:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:06 smithi170 ceph-mon[102832]: pgmap v6837: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 138 KiB/s wr, 30 op/s 2023-09-10T02:48:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:06 smithi082 ceph-mon[98694]: pgmap v6837: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 138 KiB/s wr, 30 op/s 2023-09-10T02:48:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12518.834562 secs 2023-09-10T02:48:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12518.834562 secs 2023-09-10T02:48:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12518.834562 secs 2023-09-10T02:48:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12520.199957 secs 2023-09-10T02:48:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:08 smithi098 ceph-mon[103124]: pgmap v6838: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 89 KiB/s wr, 25 op/s 2023-09-10T02:48:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:08 smithi098 ceph-mon[103124]: osdmap e495: 12 total, 12 up, 12 in 2023-09-10T02:48:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12520.199957 secs 2023-09-10T02:48:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:08 smithi170 ceph-mon[102832]: pgmap v6838: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 89 KiB/s wr, 25 op/s 2023-09-10T02:48:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:08 smithi170 ceph-mon[102832]: osdmap e495: 12 total, 12 up, 12 in 2023-09-10T02:48:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12520.199957 secs 2023-09-10T02:48:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:08 smithi082 ceph-mon[98694]: pgmap v6838: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 89 KiB/s wr, 25 op/s 2023-09-10T02:48:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:08 smithi082 ceph-mon[98694]: osdmap e495: 12 total, 12 up, 12 in 2023-09-10T02:48:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12520.911540 secs 2023-09-10T02:48:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12520.911540 secs 2023-09-10T02:48:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12520.911540 secs 2023-09-10T02:48:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:10 smithi098 ceph-mon[103124]: pgmap v6840: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 107 KiB/s wr, 30 op/s 2023-09-10T02:48:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:10 smithi170 ceph-mon[102832]: pgmap v6840: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 107 KiB/s wr, 30 op/s 2023-09-10T02:48:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:10 smithi082 ceph-mon[98694]: pgmap v6840: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 107 KiB/s wr, 30 op/s 2023-09-10T02:48:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:12 smithi098 ceph-mon[103124]: pgmap v6841: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 75 KiB/s wr, 25 op/s 2023-09-10T02:48:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12523.834621 secs 2023-09-10T02:48:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:12 smithi170 ceph-mon[102832]: pgmap v6841: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 75 KiB/s wr, 25 op/s 2023-09-10T02:48:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12523.834621 secs 2023-09-10T02:48:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:12 smithi082 ceph-mon[98694]: pgmap v6841: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 75 KiB/s wr, 25 op/s 2023-09-10T02:48:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12523.834621 secs 2023-09-10T02:48:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12525.200014 secs 2023-09-10T02:48:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12525.200014 secs 2023-09-10T02:48:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12525.200014 secs 2023-09-10T02:48:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12525.911605 secs 2023-09-10T02:48:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:14 smithi098 ceph-mon[103124]: pgmap v6842: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 75 KiB/s wr, 25 op/s 2023-09-10T02:48:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12525.911605 secs 2023-09-10T02:48:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:14 smithi170 ceph-mon[102832]: pgmap v6842: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 75 KiB/s wr, 25 op/s 2023-09-10T02:48:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12525.911605 secs 2023-09-10T02:48:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:14 smithi082 ceph-mon[98694]: pgmap v6842: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 75 KiB/s wr, 25 op/s 2023-09-10T02:48:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:16 smithi098 ceph-mon[103124]: pgmap v6843: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T02:48:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:16 smithi098 ceph-mon[103124]: osdmap e496: 12 total, 12 up, 12 in 2023-09-10T02:48:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:16 smithi170 ceph-mon[102832]: pgmap v6843: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T02:48:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:16 smithi170 ceph-mon[102832]: osdmap e496: 12 total, 12 up, 12 in 2023-09-10T02:48:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:16 smithi082 ceph-mon[98694]: pgmap v6843: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T02:48:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:16 smithi082 ceph-mon[98694]: osdmap e496: 12 total, 12 up, 12 in 2023-09-10T02:48:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12528.834681 secs 2023-09-10T02:48:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12528.834681 secs 2023-09-10T02:48:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12528.834681 secs 2023-09-10T02:48:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12530.200070 secs 2023-09-10T02:48:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:18 smithi098 ceph-mon[103124]: pgmap v6845: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T02:48:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12530.200070 secs 2023-09-10T02:48:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:18 smithi170 ceph-mon[102832]: pgmap v6845: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T02:48:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12530.200070 secs 2023-09-10T02:48:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:18 smithi082 ceph-mon[98694]: pgmap v6845: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T02:48:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12530.911667 secs 2023-09-10T02:48:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12530.911667 secs 2023-09-10T02:48:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12530.911667 secs 2023-09-10T02:48:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:21 smithi098 ceph-mon[103124]: pgmap v6846: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T02:48:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:21 smithi170 ceph-mon[102832]: pgmap v6846: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T02:48:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:21 smithi082 ceph-mon[98694]: pgmap v6846: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T02:48:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:22 smithi098 ceph-mon[103124]: pgmap v6847: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T02:48:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:48:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:22 smithi170 ceph-mon[102832]: pgmap v6847: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T02:48:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:48:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:22 smithi082 ceph-mon[98694]: pgmap v6847: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T02:48:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:48:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12535.200152 secs 2023-09-10T02:48:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:48:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:48:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:48:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12535.200152 secs 2023-09-10T02:48:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:48:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:48:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:48:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12535.200152 secs 2023-09-10T02:48:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:48:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:48:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:48:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12535.911730 secs 2023-09-10T02:48:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:24 smithi098 ceph-mon[103124]: pgmap v6848: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T02:48:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12535.911730 secs 2023-09-10T02:48:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:24 smithi170 ceph-mon[102832]: pgmap v6848: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T02:48:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12535.911730 secs 2023-09-10T02:48:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:24 smithi082 ceph-mon[98694]: pgmap v6848: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 11 op/s 2023-09-10T02:48:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:26 smithi098 ceph-mon[103124]: pgmap v6849: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T02:48:27.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:26 smithi170 ceph-mon[102832]: pgmap v6849: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T02:48:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:26 smithi082 ceph-mon[98694]: pgmap v6849: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T02:48:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12540.200213 secs 2023-09-10T02:48:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:28 smithi098 ceph-mon[103124]: pgmap v6850: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T02:48:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12540.200213 secs 2023-09-10T02:48:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:28 smithi170 ceph-mon[102832]: pgmap v6850: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T02:48:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12540.200213 secs 2023-09-10T02:48:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:28 smithi082 ceph-mon[98694]: pgmap v6850: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T02:48:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12540.911803 secs 2023-09-10T02:48:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12540.911803 secs 2023-09-10T02:48:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12540.911803 secs 2023-09-10T02:48:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:31 smithi170 ceph-mon[102832]: pgmap v6851: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 101 KiB/s wr, 10 op/s 2023-09-10T02:48:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:31 smithi082 ceph-mon[98694]: pgmap v6851: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 101 KiB/s wr, 10 op/s 2023-09-10T02:48:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:31 smithi098 ceph-mon[103124]: pgmap v6851: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 101 KiB/s wr, 10 op/s 2023-09-10T02:48:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:32 smithi098 ceph-mon[103124]: pgmap v6852: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 211 KiB/s wr, 17 op/s 2023-09-10T02:48:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12543.834955 secs 2023-09-10T02:48:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:32 smithi170 ceph-mon[102832]: pgmap v6852: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 211 KiB/s wr, 17 op/s 2023-09-10T02:48:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12543.834955 secs 2023-09-10T02:48:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:32 smithi082 ceph-mon[98694]: pgmap v6852: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 211 KiB/s wr, 17 op/s 2023-09-10T02:48:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12543.834955 secs 2023-09-10T02:48:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12545.200294 secs 2023-09-10T02:48:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12545.200294 secs 2023-09-10T02:48:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12545.200294 secs 2023-09-10T02:48:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12545.911868 secs 2023-09-10T02:48:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:34 smithi098 ceph-mon[103124]: pgmap v6853: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 174 KiB/s wr, 12 op/s 2023-09-10T02:48:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12545.911868 secs 2023-09-10T02:48:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:34 smithi170 ceph-mon[102832]: pgmap v6853: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 174 KiB/s wr, 12 op/s 2023-09-10T02:48:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12545.911868 secs 2023-09-10T02:48:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:34 smithi082 ceph-mon[98694]: pgmap v6853: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 174 KiB/s wr, 12 op/s 2023-09-10T02:48:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:35 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:48:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:35 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:48:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:35 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:48:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:35 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:48:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:35 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:48:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:35 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:48:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:35 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:48:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:35 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:48:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:35 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:48:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:37 smithi170 ceph-mon[102832]: pgmap v6854: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 272 KiB/s wr, 19 op/s 2023-09-10T02:48:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:37 smithi082 ceph-mon[98694]: pgmap v6854: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 272 KiB/s wr, 19 op/s 2023-09-10T02:48:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:37 smithi098 ceph-mon[103124]: pgmap v6854: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 272 KiB/s wr, 19 op/s 2023-09-10T02:48:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12548.835015 secs 2023-09-10T02:48:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12548.835015 secs 2023-09-10T02:48:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12548.835015 secs 2023-09-10T02:48:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12550.200360 secs 2023-09-10T02:48:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:39 smithi170 ceph-mon[102832]: pgmap v6855: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 208 KiB/s wr, 12 op/s 2023-09-10T02:48:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12550.200360 secs 2023-09-10T02:48:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:39 smithi082 ceph-mon[98694]: pgmap v6855: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 208 KiB/s wr, 12 op/s 2023-09-10T02:48:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12550.200360 secs 2023-09-10T02:48:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:39 smithi098 ceph-mon[103124]: pgmap v6855: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 208 KiB/s wr, 12 op/s 2023-09-10T02:48:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12550.911935 secs 2023-09-10T02:48:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12550.911935 secs 2023-09-10T02:48:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12550.911935 secs 2023-09-10T02:48:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:41 smithi170 ceph-mon[102832]: pgmap v6856: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 208 KiB/s wr, 12 op/s 2023-09-10T02:48:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:41 smithi082 ceph-mon[98694]: pgmap v6856: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 208 KiB/s wr, 12 op/s 2023-09-10T02:48:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:41 smithi098 ceph-mon[103124]: pgmap v6856: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 208 KiB/s wr, 12 op/s 2023-09-10T02:48:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:42 smithi098 ceph-mon[103124]: pgmap v6857: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 373 KiB/s wr, 19 op/s 2023-09-10T02:48:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12553.835077 secs 2023-09-10T02:48:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:42 smithi170 ceph-mon[102832]: pgmap v6857: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 373 KiB/s wr, 19 op/s 2023-09-10T02:48:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12553.835077 secs 2023-09-10T02:48:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:42 smithi082 ceph-mon[98694]: pgmap v6857: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 373 KiB/s wr, 19 op/s 2023-09-10T02:48:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12553.835077 secs 2023-09-10T02:48:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12555.200414 secs 2023-09-10T02:48:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12555.200414 secs 2023-09-10T02:48:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12555.200414 secs 2023-09-10T02:48:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12555.911997 secs 2023-09-10T02:48:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:44 smithi098 ceph-mon[103124]: pgmap v6858: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 263 KiB/s wr, 12 op/s 2023-09-10T02:48:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12555.911997 secs 2023-09-10T02:48:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:44 smithi170 ceph-mon[102832]: pgmap v6858: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 263 KiB/s wr, 12 op/s 2023-09-10T02:48:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12555.911997 secs 2023-09-10T02:48:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:44 smithi082 ceph-mon[98694]: pgmap v6858: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 263 KiB/s wr, 12 op/s 2023-09-10T02:48:47.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:47 smithi170 ceph-mon[102832]: pgmap v6859: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 285 KiB/s wr, 18 op/s 2023-09-10T02:48:47.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:47 smithi082 ceph-mon[98694]: pgmap v6859: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 285 KiB/s wr, 18 op/s 2023-09-10T02:48:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:47 smithi098 ceph-mon[103124]: pgmap v6859: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 285 KiB/s wr, 18 op/s 2023-09-10T02:48:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12558.835147 secs 2023-09-10T02:48:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12558.835147 secs 2023-09-10T02:48:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12558.835147 secs 2023-09-10T02:48:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12560.200497 secs 2023-09-10T02:48:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:49 smithi170 ceph-mon[102832]: pgmap v6860: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 187 KiB/s wr, 11 op/s 2023-09-10T02:48:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12560.200497 secs 2023-09-10T02:48:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:49 smithi082 ceph-mon[98694]: pgmap v6860: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 187 KiB/s wr, 11 op/s 2023-09-10T02:48:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12560.200497 secs 2023-09-10T02:48:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:49 smithi098 ceph-mon[103124]: pgmap v6860: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 187 KiB/s wr, 11 op/s 2023-09-10T02:48:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12560.912061 secs 2023-09-10T02:48:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12560.912061 secs 2023-09-10T02:48:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12560.912061 secs 2023-09-10T02:48:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:51 smithi170 ceph-mon[102832]: pgmap v6861: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 187 KiB/s wr, 11 op/s 2023-09-10T02:48:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:51 smithi082 ceph-mon[98694]: pgmap v6861: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 187 KiB/s wr, 11 op/s 2023-09-10T02:48:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:51 smithi098 ceph-mon[103124]: pgmap v6861: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 187 KiB/s wr, 11 op/s 2023-09-10T02:48:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:52 smithi098 ceph-mon[103124]: pgmap v6862: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 262 KiB/s wr, 18 op/s 2023-09-10T02:48:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12563.835206 secs 2023-09-10T02:48:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:52 smithi170 ceph-mon[102832]: pgmap v6862: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 262 KiB/s wr, 18 op/s 2023-09-10T02:48:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12563.835206 secs 2023-09-10T02:48:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:52 smithi082 ceph-mon[98694]: pgmap v6862: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 262 KiB/s wr, 18 op/s 2023-09-10T02:48:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12563.835206 secs 2023-09-10T02:48:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12565.200554 secs 2023-09-10T02:48:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12565.200554 secs 2023-09-10T02:48:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12565.200554 secs 2023-09-10T02:48:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12565.912127 secs 2023-09-10T02:48:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:54 smithi098 ceph-mon[103124]: pgmap v6863: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 97 KiB/s wr, 11 op/s 2023-09-10T02:48:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12565.912127 secs 2023-09-10T02:48:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:54 smithi170 ceph-mon[102832]: pgmap v6863: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 97 KiB/s wr, 11 op/s 2023-09-10T02:48:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12565.912127 secs 2023-09-10T02:48:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:54 smithi082 ceph-mon[98694]: pgmap v6863: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 97 KiB/s wr, 11 op/s 2023-09-10T02:48:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:57 smithi170 ceph-mon[102832]: pgmap v6864: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 97 KiB/s wr, 16 op/s 2023-09-10T02:48:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:57 smithi082 ceph-mon[98694]: pgmap v6864: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 97 KiB/s wr, 16 op/s 2023-09-10T02:48:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:57 smithi098 ceph-mon[103124]: pgmap v6864: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 97 KiB/s wr, 16 op/s 2023-09-10T02:48:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12568.835269 secs 2023-09-10T02:48:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12568.835269 secs 2023-09-10T02:48:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12568.835269 secs 2023-09-10T02:48:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12570.200634 secs 2023-09-10T02:48:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:48:59 smithi170 ceph-mon[102832]: pgmap v6865: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T02:48:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12570.200634 secs 2023-09-10T02:48:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:48:59 smithi082 ceph-mon[98694]: pgmap v6865: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T02:48:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12570.200634 secs 2023-09-10T02:48:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:48:59 smithi098 ceph-mon[103124]: pgmap v6865: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T02:49:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12570.912191 secs 2023-09-10T02:49:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12570.912191 secs 2023-09-10T02:49:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12570.912191 secs 2023-09-10T02:49:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:01 smithi170 ceph-mon[102832]: pgmap v6866: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T02:49:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:01 smithi082 ceph-mon[98694]: pgmap v6866: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T02:49:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:01 smithi098 ceph-mon[103124]: pgmap v6866: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T02:49:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:02 smithi098 ceph-mon[103124]: pgmap v6867: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 105 KiB/s wr, 16 op/s 2023-09-10T02:49:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12573.835330 secs 2023-09-10T02:49:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:02 smithi170 ceph-mon[102832]: pgmap v6867: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 105 KiB/s wr, 16 op/s 2023-09-10T02:49:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12573.835330 secs 2023-09-10T02:49:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:02 smithi082 ceph-mon[98694]: pgmap v6867: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 105 KiB/s wr, 16 op/s 2023-09-10T02:49:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12573.835330 secs 2023-09-10T02:49:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12575.200690 secs 2023-09-10T02:49:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12575.200690 secs 2023-09-10T02:49:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12575.200690 secs 2023-09-10T02:49:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12575.912257 secs 2023-09-10T02:49:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:05 smithi170 ceph-mon[102832]: pgmap v6868: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 10 op/s 2023-09-10T02:49:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12575.912257 secs 2023-09-10T02:49:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:05 smithi082 ceph-mon[98694]: pgmap v6868: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 10 op/s 2023-09-10T02:49:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12575.912257 secs 2023-09-10T02:49:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:05 smithi098 ceph-mon[103124]: pgmap v6868: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 10 op/s 2023-09-10T02:49:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:49:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:49:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:49:07.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:07 smithi170 ceph-mon[102832]: pgmap v6869: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T02:49:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:07 smithi082 ceph-mon[98694]: pgmap v6869: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T02:49:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:07 smithi098 ceph-mon[103124]: pgmap v6869: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T02:49:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12578.835393 secs 2023-09-10T02:49:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12578.835393 secs 2023-09-10T02:49:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12578.835393 secs 2023-09-10T02:49:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12580.200745 secs 2023-09-10T02:49:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:09 smithi170 ceph-mon[102832]: pgmap v6870: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 75 KiB/s wr, 24 op/s 2023-09-10T02:49:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:09 smithi170 ceph-mon[102832]: osdmap e497: 12 total, 12 up, 12 in 2023-09-10T02:49:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12580.200745 secs 2023-09-10T02:49:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:09 smithi082 ceph-mon[98694]: pgmap v6870: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 75 KiB/s wr, 24 op/s 2023-09-10T02:49:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:09 smithi082 ceph-mon[98694]: osdmap e497: 12 total, 12 up, 12 in 2023-09-10T02:49:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:09.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:09.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12580.200745 secs 2023-09-10T02:49:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:09 smithi098 ceph-mon[103124]: pgmap v6870: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 75 KiB/s wr, 24 op/s 2023-09-10T02:49:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:09 smithi098 ceph-mon[103124]: osdmap e497: 12 total, 12 up, 12 in 2023-09-10T02:49:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12580.912324 secs 2023-09-10T02:49:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:10 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 158.0M 2023-09-10T02:49:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:10 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 165748371: error parsing value: Value '165748371' is below minimum 939524096 2023-09-10T02:49:10.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:10.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:10.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:10.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:10.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:49:10.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:49:10.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12580.912324 secs 2023-09-10T02:49:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:10 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 158.0M 2023-09-10T02:49:10.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:10 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 165748371: error parsing value: Value '165748371' is below minimum 939524096 2023-09-10T02:49:10.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:10.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:10.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:10.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:10.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:49:10.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:49:10.462 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12580.912324 secs 2023-09-10T02:49:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:10 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 158.0M 2023-09-10T02:49:10.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:10 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 165748371: error parsing value: Value '165748371' is below minimum 939524096 2023-09-10T02:49:10.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:10.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:10.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:10.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:10.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:49:10.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:49:10.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:49:10.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:49:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:11 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T02:49:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:11 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T02:49:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:11 smithi170 ceph-mon[102832]: pgmap v6872: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T02:49:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:11 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T02:49:11.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:11 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T02:49:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:11 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T02:49:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:11 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T02:49:11.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:11 smithi082 ceph-mon[98694]: pgmap v6872: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T02:49:11.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:11 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T02:49:11.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:11 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T02:49:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:11 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T02:49:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:11 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T02:49:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:11 smithi098 ceph-mon[103124]: pgmap v6872: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 90 KiB/s wr, 29 op/s 2023-09-10T02:49:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:11 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T02:49:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:11 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T02:49:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:12 smithi098 ceph-mon[103124]: pgmap v6873: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 29 op/s 2023-09-10T02:49:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12583.835452 secs 2023-09-10T02:49:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:12 smithi170 ceph-mon[102832]: pgmap v6873: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 29 op/s 2023-09-10T02:49:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12583.835452 secs 2023-09-10T02:49:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:12 smithi082 ceph-mon[98694]: pgmap v6873: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 29 op/s 2023-09-10T02:49:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12583.835452 secs 2023-09-10T02:49:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12585.200803 secs 2023-09-10T02:49:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12585.200803 secs 2023-09-10T02:49:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12585.200803 secs 2023-09-10T02:49:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12585.912388 secs 2023-09-10T02:49:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:15 smithi098 ceph-mon[103124]: pgmap v6874: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 29 op/s 2023-09-10T02:49:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12585.912388 secs 2023-09-10T02:49:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:15 smithi170 ceph-mon[102832]: pgmap v6874: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 29 op/s 2023-09-10T02:49:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12585.912388 secs 2023-09-10T02:49:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:15 smithi082 ceph-mon[98694]: pgmap v6874: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 29 op/s 2023-09-10T02:49:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:17 smithi170 ceph-mon[102832]: pgmap v6875: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T02:49:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:17 smithi170 ceph-mon[102832]: osdmap e498: 12 total, 12 up, 12 in 2023-09-10T02:49:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:17 smithi082 ceph-mon[98694]: pgmap v6875: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T02:49:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:17 smithi082 ceph-mon[98694]: osdmap e498: 12 total, 12 up, 12 in 2023-09-10T02:49:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:17 smithi098 ceph-mon[103124]: pgmap v6875: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T02:49:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:17 smithi098 ceph-mon[103124]: osdmap e498: 12 total, 12 up, 12 in 2023-09-10T02:49:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12588.835511 secs 2023-09-10T02:49:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12588.835511 secs 2023-09-10T02:49:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12588.835511 secs 2023-09-10T02:49:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12590.200886 secs 2023-09-10T02:49:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:19 smithi170 ceph-mon[102832]: pgmap v6877: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T02:49:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12590.200886 secs 2023-09-10T02:49:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:19 smithi082 ceph-mon[98694]: pgmap v6877: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T02:49:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12590.200886 secs 2023-09-10T02:49:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:19 smithi098 ceph-mon[103124]: pgmap v6877: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T02:49:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12590.912455 secs 2023-09-10T02:49:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12590.912455 secs 2023-09-10T02:49:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12590.912455 secs 2023-09-10T02:49:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:21 smithi170 ceph-mon[102832]: pgmap v6878: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T02:49:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:21 smithi082 ceph-mon[98694]: pgmap v6878: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T02:49:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:21 smithi098 ceph-mon[103124]: pgmap v6878: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T02:49:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:23 smithi170 ceph-mon[102832]: pgmap v6879: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T02:49:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:49:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:23 smithi082 ceph-mon[98694]: pgmap v6879: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T02:49:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:49:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:23 smithi098 ceph-mon[103124]: pgmap v6879: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T02:49:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:49:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12595.200937 secs 2023-09-10T02:49:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:49:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:49:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:49:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12595.200937 secs 2023-09-10T02:49:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:49:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:49:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:49:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12595.200937 secs 2023-09-10T02:49:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:49:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:49:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:49:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12595.912489 secs 2023-09-10T02:49:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:25 smithi170 ceph-mon[102832]: pgmap v6880: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T02:49:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12595.912489 secs 2023-09-10T02:49:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:25 smithi082 ceph-mon[98694]: pgmap v6880: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T02:49:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12595.912489 secs 2023-09-10T02:49:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:25 smithi098 ceph-mon[103124]: pgmap v6880: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T02:49:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:27 smithi170 ceph-mon[102832]: pgmap v6881: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 12 op/s 2023-09-10T02:49:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:27 smithi082 ceph-mon[98694]: pgmap v6881: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 12 op/s 2023-09-10T02:49:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:27 smithi098 ceph-mon[103124]: pgmap v6881: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 12 op/s 2023-09-10T02:49:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12600.201000 secs 2023-09-10T02:49:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:29 smithi170 ceph-mon[102832]: pgmap v6882: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T02:49:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12600.201000 secs 2023-09-10T02:49:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:29 smithi082 ceph-mon[98694]: pgmap v6882: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T02:49:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12600.201000 secs 2023-09-10T02:49:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:29 smithi098 ceph-mon[103124]: pgmap v6882: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 11 op/s 2023-09-10T02:49:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12600.912551 secs 2023-09-10T02:49:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12600.912551 secs 2023-09-10T02:49:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12600.912551 secs 2023-09-10T02:49:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:31 smithi170 ceph-mon[102832]: pgmap v6883: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 10 op/s 2023-09-10T02:49:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:31 smithi082 ceph-mon[98694]: pgmap v6883: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 10 op/s 2023-09-10T02:49:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:31 smithi098 ceph-mon[103124]: pgmap v6883: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 10 op/s 2023-09-10T02:49:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:33 smithi098 ceph-mon[103124]: pgmap v6884: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 158 KiB/s wr, 16 op/s 2023-09-10T02:49:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12603.835857 secs 2023-09-10T02:49:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:33 smithi170 ceph-mon[102832]: pgmap v6884: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 158 KiB/s wr, 16 op/s 2023-09-10T02:49:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12603.835857 secs 2023-09-10T02:49:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:33 smithi082 ceph-mon[98694]: pgmap v6884: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 158 KiB/s wr, 16 op/s 2023-09-10T02:49:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12603.835857 secs 2023-09-10T02:49:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12605.201057 secs 2023-09-10T02:49:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12605.201057 secs 2023-09-10T02:49:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12605.201057 secs 2023-09-10T02:49:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12605.912615 secs 2023-09-10T02:49:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:35 smithi170 ceph-mon[102832]: pgmap v6885: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T02:49:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12605.912615 secs 2023-09-10T02:49:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:35 smithi082 ceph-mon[98694]: pgmap v6885: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T02:49:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12605.912615 secs 2023-09-10T02:49:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:35 smithi098 ceph-mon[103124]: pgmap v6885: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T02:49:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:36 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:49:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:49:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:49:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:36 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:49:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:49:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:49:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:36 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:49:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:49:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:49:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:37 smithi170 ceph-mon[102832]: pgmap v6886: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 235 KiB/s wr, 18 op/s 2023-09-10T02:49:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:37 smithi082 ceph-mon[98694]: pgmap v6886: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 235 KiB/s wr, 18 op/s 2023-09-10T02:49:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:37 smithi098 ceph-mon[103124]: pgmap v6886: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 235 KiB/s wr, 18 op/s 2023-09-10T02:49:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12608.835920 secs 2023-09-10T02:49:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12608.835920 secs 2023-09-10T02:49:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12608.835920 secs 2023-09-10T02:49:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12610.201140 secs 2023-09-10T02:49:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:39 smithi170 ceph-mon[102832]: pgmap v6887: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 166 KiB/s wr, 12 op/s 2023-09-10T02:49:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12610.201140 secs 2023-09-10T02:49:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:39 smithi082 ceph-mon[98694]: pgmap v6887: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 166 KiB/s wr, 12 op/s 2023-09-10T02:49:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12610.201140 secs 2023-09-10T02:49:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:39 smithi098 ceph-mon[103124]: pgmap v6887: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 166 KiB/s wr, 12 op/s 2023-09-10T02:49:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12610.912682 secs 2023-09-10T02:49:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12610.912682 secs 2023-09-10T02:49:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12610.912682 secs 2023-09-10T02:49:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:41 smithi170 ceph-mon[102832]: pgmap v6888: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 166 KiB/s wr, 12 op/s 2023-09-10T02:49:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:41 smithi082 ceph-mon[98694]: pgmap v6888: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 166 KiB/s wr, 12 op/s 2023-09-10T02:49:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:41 smithi098 ceph-mon[103124]: pgmap v6888: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 166 KiB/s wr, 12 op/s 2023-09-10T02:49:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:43 smithi170 ceph-mon[102832]: pgmap v6889: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 232 KiB/s wr, 18 op/s 2023-09-10T02:49:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12613.835981 secs 2023-09-10T02:49:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:43 smithi082 ceph-mon[98694]: pgmap v6889: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 232 KiB/s wr, 18 op/s 2023-09-10T02:49:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12613.835981 secs 2023-09-10T02:49:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:43 smithi098 ceph-mon[103124]: pgmap v6889: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 232 KiB/s wr, 18 op/s 2023-09-10T02:49:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12613.835981 secs 2023-09-10T02:49:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12615.201222 secs 2023-09-10T02:49:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12615.201222 secs 2023-09-10T02:49:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12615.201222 secs 2023-09-10T02:49:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12615.912747 secs 2023-09-10T02:49:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:45 smithi170 ceph-mon[102832]: pgmap v6890: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 143 KiB/s wr, 12 op/s 2023-09-10T02:49:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12615.912747 secs 2023-09-10T02:49:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:45 smithi082 ceph-mon[98694]: pgmap v6890: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 143 KiB/s wr, 12 op/s 2023-09-10T02:49:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12615.912747 secs 2023-09-10T02:49:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:45 smithi098 ceph-mon[103124]: pgmap v6890: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 143 KiB/s wr, 12 op/s 2023-09-10T02:49:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:47 smithi170 ceph-mon[102832]: pgmap v6891: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 242 KiB/s wr, 17 op/s 2023-09-10T02:49:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:47 smithi082 ceph-mon[98694]: pgmap v6891: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 242 KiB/s wr, 17 op/s 2023-09-10T02:49:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:47 smithi098 ceph-mon[103124]: pgmap v6891: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 242 KiB/s wr, 17 op/s 2023-09-10T02:49:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12618.836048 secs 2023-09-10T02:49:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12618.836048 secs 2023-09-10T02:49:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12618.836048 secs 2023-09-10T02:49:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12620.201307 secs 2023-09-10T02:49:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:49 smithi170 ceph-mon[102832]: pgmap v6892: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T02:49:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12620.201307 secs 2023-09-10T02:49:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:49 smithi082 ceph-mon[98694]: pgmap v6892: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T02:49:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12620.201307 secs 2023-09-10T02:49:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:49 smithi098 ceph-mon[103124]: pgmap v6892: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T02:49:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12620.912810 secs 2023-09-10T02:49:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12620.912810 secs 2023-09-10T02:49:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12620.912810 secs 2023-09-10T02:49:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:51 smithi170 ceph-mon[102832]: pgmap v6893: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T02:49:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:51 smithi082 ceph-mon[98694]: pgmap v6893: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T02:49:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:51 smithi098 ceph-mon[103124]: pgmap v6893: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T02:49:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:53 smithi170 ceph-mon[102832]: pgmap v6894: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 236 KiB/s wr, 16 op/s 2023-09-10T02:49:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12623.836108 secs 2023-09-10T02:49:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:53 smithi082 ceph-mon[98694]: pgmap v6894: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 236 KiB/s wr, 16 op/s 2023-09-10T02:49:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12623.836108 secs 2023-09-10T02:49:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:53 smithi098 ceph-mon[103124]: pgmap v6894: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 236 KiB/s wr, 16 op/s 2023-09-10T02:49:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12623.836108 secs 2023-09-10T02:49:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12625.201363 secs 2023-09-10T02:49:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12625.201363 secs 2023-09-10T02:49:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12625.201363 secs 2023-09-10T02:49:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12625.912874 secs 2023-09-10T02:49:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:55 smithi170 ceph-mon[102832]: pgmap v6895: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 170 KiB/s wr, 11 op/s 2023-09-10T02:49:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12625.912874 secs 2023-09-10T02:49:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:55 smithi082 ceph-mon[98694]: pgmap v6895: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 170 KiB/s wr, 11 op/s 2023-09-10T02:49:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12625.912874 secs 2023-09-10T02:49:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:55 smithi098 ceph-mon[103124]: pgmap v6895: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 170 KiB/s wr, 11 op/s 2023-09-10T02:49:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:57 smithi170 ceph-mon[102832]: pgmap v6896: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 219 KiB/s wr, 15 op/s 2023-09-10T02:49:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:57 smithi082 ceph-mon[98694]: pgmap v6896: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 219 KiB/s wr, 15 op/s 2023-09-10T02:49:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:57 smithi098 ceph-mon[103124]: pgmap v6896: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 219 KiB/s wr, 15 op/s 2023-09-10T02:49:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12628.836168 secs 2023-09-10T02:49:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12628.836168 secs 2023-09-10T02:49:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12628.836168 secs 2023-09-10T02:49:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12630.201450 secs 2023-09-10T02:49:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:49:59 smithi170 ceph-mon[102832]: pgmap v6897: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 9 op/s 2023-09-10T02:49:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12630.201450 secs 2023-09-10T02:49:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:49:59 smithi082 ceph-mon[98694]: pgmap v6897: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 9 op/s 2023-09-10T02:49:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12630.201450 secs 2023-09-10T02:49:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:49:59 smithi098 ceph-mon[103124]: pgmap v6897: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 9 op/s 2023-09-10T02:50:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12630.912939 secs 2023-09-10T02:50:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T02:50:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12630.912939 secs 2023-09-10T02:50:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T02:50:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12630.912939 secs 2023-09-10T02:50:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T02:50:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:01 smithi170 ceph-mon[102832]: pgmap v6898: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 9 op/s 2023-09-10T02:50:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:01 smithi082 ceph-mon[98694]: pgmap v6898: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 9 op/s 2023-09-10T02:50:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:01 smithi098 ceph-mon[103124]: pgmap v6898: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 9 op/s 2023-09-10T02:50:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:03 smithi170 ceph-mon[102832]: pgmap v6899: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 15 op/s 2023-09-10T02:50:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12633.836229 secs 2023-09-10T02:50:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:03 smithi082 ceph-mon[98694]: pgmap v6899: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 15 op/s 2023-09-10T02:50:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12633.836229 secs 2023-09-10T02:50:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:03 smithi098 ceph-mon[103124]: pgmap v6899: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 15 op/s 2023-09-10T02:50:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12633.836229 secs 2023-09-10T02:50:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12635.201534 secs 2023-09-10T02:50:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12635.201534 secs 2023-09-10T02:50:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12635.201534 secs 2023-09-10T02:50:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12635.913004 secs 2023-09-10T02:50:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:05 smithi170 ceph-mon[102832]: pgmap v6900: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 9 op/s 2023-09-10T02:50:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12635.913004 secs 2023-09-10T02:50:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:05 smithi082 ceph-mon[98694]: pgmap v6900: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 9 op/s 2023-09-10T02:50:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12635.913004 secs 2023-09-10T02:50:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:05 smithi098 ceph-mon[103124]: pgmap v6900: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 9 op/s 2023-09-10T02:50:07.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:07 smithi170 ceph-mon[102832]: pgmap v6901: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 139 KiB/s wr, 29 op/s 2023-09-10T02:50:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:07 smithi082 ceph-mon[98694]: pgmap v6901: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 139 KiB/s wr, 29 op/s 2023-09-10T02:50:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:07 smithi098 ceph-mon[103124]: pgmap v6901: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 139 KiB/s wr, 29 op/s 2023-09-10T02:50:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12638.836322 secs 2023-09-10T02:50:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12638.836322 secs 2023-09-10T02:50:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12638.836322 secs 2023-09-10T02:50:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12640.201592 secs 2023-09-10T02:50:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:09 smithi170 ceph-mon[102832]: pgmap v6902: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 90 KiB/s wr, 24 op/s 2023-09-10T02:50:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:09 smithi170 ceph-mon[102832]: osdmap e499: 12 total, 12 up, 12 in 2023-09-10T02:50:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12640.201592 secs 2023-09-10T02:50:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:09 smithi082 ceph-mon[98694]: pgmap v6902: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 90 KiB/s wr, 24 op/s 2023-09-10T02:50:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:09 smithi082 ceph-mon[98694]: osdmap e499: 12 total, 12 up, 12 in 2023-09-10T02:50:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12640.201592 secs 2023-09-10T02:50:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:09 smithi098 ceph-mon[103124]: pgmap v6902: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 90 KiB/s wr, 24 op/s 2023-09-10T02:50:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:09 smithi098 ceph-mon[103124]: osdmap e499: 12 total, 12 up, 12 in 2023-09-10T02:50:10.401 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12640.913066 secs 2023-09-10T02:50:10.402 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12640.913066 secs 2023-09-10T02:50:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12640.913066 secs 2023-09-10T02:50:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:11 smithi170 ceph-mon[102832]: pgmap v6904: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 108 KiB/s wr, 29 op/s 2023-09-10T02:50:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:50:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:50:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:50:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:50:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:11 smithi082 ceph-mon[98694]: pgmap v6904: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 108 KiB/s wr, 29 op/s 2023-09-10T02:50:11.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:50:11.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:50:11.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:50:11.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:50:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:11 smithi098 ceph-mon[103124]: pgmap v6904: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 108 KiB/s wr, 29 op/s 2023-09-10T02:50:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:50:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:50:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:50:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:50:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:13 smithi170 ceph-mon[102832]: pgmap v6905: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T02:50:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12643.836382 secs 2023-09-10T02:50:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:13 smithi082 ceph-mon[98694]: pgmap v6905: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T02:50:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12643.836382 secs 2023-09-10T02:50:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:13 smithi098 ceph-mon[103124]: pgmap v6905: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T02:50:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12643.836382 secs 2023-09-10T02:50:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12645.201647 secs 2023-09-10T02:50:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12645.201647 secs 2023-09-10T02:50:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12645.201647 secs 2023-09-10T02:50:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12645.913130 secs 2023-09-10T02:50:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:15 smithi170 ceph-mon[102832]: pgmap v6906: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T02:50:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12645.913130 secs 2023-09-10T02:50:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:15 smithi082 ceph-mon[98694]: pgmap v6906: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T02:50:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12645.913130 secs 2023-09-10T02:50:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:15 smithi098 ceph-mon[103124]: pgmap v6906: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T02:50:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:17 smithi170 ceph-mon[102832]: pgmap v6907: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-09-10T02:50:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:17 smithi170 ceph-mon[102832]: osdmap e500: 12 total, 12 up, 12 in 2023-09-10T02:50:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:17 smithi082 ceph-mon[98694]: pgmap v6907: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-09-10T02:50:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:17 smithi082 ceph-mon[98694]: osdmap e500: 12 total, 12 up, 12 in 2023-09-10T02:50:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:17 smithi098 ceph-mon[103124]: pgmap v6907: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-09-10T02:50:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:17 smithi098 ceph-mon[103124]: osdmap e500: 12 total, 12 up, 12 in 2023-09-10T02:50:18.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12648.836518 secs 2023-09-10T02:50:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12648.836518 secs 2023-09-10T02:50:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12648.836518 secs 2023-09-10T02:50:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12650.201731 secs 2023-09-10T02:50:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:19 smithi170 ceph-mon[102832]: pgmap v6909: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-09-10T02:50:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12650.201731 secs 2023-09-10T02:50:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:19 smithi082 ceph-mon[98694]: pgmap v6909: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-09-10T02:50:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12650.201731 secs 2023-09-10T02:50:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:19 smithi098 ceph-mon[103124]: pgmap v6909: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-09-10T02:50:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12650.913195 secs 2023-09-10T02:50:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12650.913195 secs 2023-09-10T02:50:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12650.913195 secs 2023-09-10T02:50:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:21 smithi170 ceph-mon[102832]: pgmap v6910: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-09-10T02:50:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:21 smithi082 ceph-mon[98694]: pgmap v6910: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-09-10T02:50:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:21 smithi098 ceph-mon[103124]: pgmap v6910: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-09-10T02:50:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:23 smithi170 ceph-mon[102832]: pgmap v6911: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T02:50:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:50:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:23 smithi082 ceph-mon[98694]: pgmap v6911: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T02:50:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:50:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:23 smithi098 ceph-mon[103124]: pgmap v6911: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T02:50:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:50:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12655.201788 secs 2023-09-10T02:50:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:50:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:50:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:50:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12655.201788 secs 2023-09-10T02:50:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:50:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:50:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:50:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12655.201788 secs 2023-09-10T02:50:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:50:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:50:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:50:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12655.913259 secs 2023-09-10T02:50:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:25 smithi170 ceph-mon[102832]: pgmap v6912: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T02:50:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12655.913259 secs 2023-09-10T02:50:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:25 smithi082 ceph-mon[98694]: pgmap v6912: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T02:50:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12655.913259 secs 2023-09-10T02:50:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:25 smithi098 ceph-mon[103124]: pgmap v6912: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T02:50:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:27 smithi170 ceph-mon[102832]: pgmap v6913: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T02:50:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:27 smithi082 ceph-mon[98694]: pgmap v6913: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T02:50:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:27 smithi098 ceph-mon[103124]: pgmap v6913: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T02:50:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12660.201846 secs 2023-09-10T02:50:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:29 smithi170 ceph-mon[102832]: pgmap v6914: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T02:50:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12660.201846 secs 2023-09-10T02:50:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:29 smithi082 ceph-mon[98694]: pgmap v6914: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T02:50:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12660.201846 secs 2023-09-10T02:50:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:29 smithi098 ceph-mon[103124]: pgmap v6914: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T02:50:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12660.913320 secs 2023-09-10T02:50:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12660.913320 secs 2023-09-10T02:50:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12660.913320 secs 2023-09-10T02:50:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:31 smithi170 ceph-mon[102832]: pgmap v6915: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 10 op/s 2023-09-10T02:50:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:31 smithi082 ceph-mon[98694]: pgmap v6915: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 10 op/s 2023-09-10T02:50:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:31 smithi098 ceph-mon[103124]: pgmap v6915: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 10 op/s 2023-09-10T02:50:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:33 smithi170 ceph-mon[102832]: pgmap v6916: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 214 KiB/s wr, 16 op/s 2023-09-10T02:50:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12663.836699 secs 2023-09-10T02:50:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:33 smithi082 ceph-mon[98694]: pgmap v6916: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 214 KiB/s wr, 16 op/s 2023-09-10T02:50:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12663.836699 secs 2023-09-10T02:50:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:33 smithi098 ceph-mon[103124]: pgmap v6916: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 214 KiB/s wr, 16 op/s 2023-09-10T02:50:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12663.836699 secs 2023-09-10T02:50:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12665.201903 secs 2023-09-10T02:50:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12665.201903 secs 2023-09-10T02:50:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12665.201903 secs 2023-09-10T02:50:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12665.913382 secs 2023-09-10T02:50:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:35 smithi170 ceph-mon[102832]: pgmap v6917: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T02:50:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12665.913382 secs 2023-09-10T02:50:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:35 smithi082 ceph-mon[98694]: pgmap v6917: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T02:50:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12665.913382 secs 2023-09-10T02:50:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:35 smithi098 ceph-mon[103124]: pgmap v6917: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T02:50:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:36 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:50:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:50:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:50:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:36 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:50:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:50:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:50:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:36 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:50:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:50:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:50:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:37 smithi170 ceph-mon[102832]: pgmap v6918: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 277 KiB/s wr, 17 op/s 2023-09-10T02:50:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:37 smithi082 ceph-mon[98694]: pgmap v6918: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 277 KiB/s wr, 17 op/s 2023-09-10T02:50:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:37 smithi098 ceph-mon[103124]: pgmap v6918: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 277 KiB/s wr, 17 op/s 2023-09-10T02:50:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12668.836846 secs 2023-09-10T02:50:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12668.836846 secs 2023-09-10T02:50:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12668.836846 secs 2023-09-10T02:50:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12670.201961 secs 2023-09-10T02:50:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:39 smithi170 ceph-mon[102832]: pgmap v6919: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 235 KiB/s wr, 12 op/s 2023-09-10T02:50:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12670.201961 secs 2023-09-10T02:50:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:39 smithi082 ceph-mon[98694]: pgmap v6919: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 235 KiB/s wr, 12 op/s 2023-09-10T02:50:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12670.201961 secs 2023-09-10T02:50:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:39 smithi098 ceph-mon[103124]: pgmap v6919: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 235 KiB/s wr, 12 op/s 2023-09-10T02:50:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12670.913447 secs 2023-09-10T02:50:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12670.913447 secs 2023-09-10T02:50:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12670.913447 secs 2023-09-10T02:50:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:41 smithi170 ceph-mon[102832]: pgmap v6920: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 235 KiB/s wr, 12 op/s 2023-09-10T02:50:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:41 smithi082 ceph-mon[98694]: pgmap v6920: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 235 KiB/s wr, 12 op/s 2023-09-10T02:50:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:41 smithi098 ceph-mon[103124]: pgmap v6920: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 235 KiB/s wr, 12 op/s 2023-09-10T02:50:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:43 smithi170 ceph-mon[102832]: pgmap v6921: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 308 KiB/s wr, 18 op/s 2023-09-10T02:50:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12673.836905 secs 2023-09-10T02:50:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:43 smithi082 ceph-mon[98694]: pgmap v6921: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 308 KiB/s wr, 18 op/s 2023-09-10T02:50:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12673.836905 secs 2023-09-10T02:50:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:43 smithi098 ceph-mon[103124]: pgmap v6921: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 308 KiB/s wr, 18 op/s 2023-09-10T02:50:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12673.836905 secs 2023-09-10T02:50:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12675.202021 secs 2023-09-10T02:50:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12675.202021 secs 2023-09-10T02:50:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12675.202021 secs 2023-09-10T02:50:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12675.913491 secs 2023-09-10T02:50:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:45 smithi170 ceph-mon[102832]: pgmap v6922: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 173 KiB/s wr, 12 op/s 2023-09-10T02:50:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12675.913491 secs 2023-09-10T02:50:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:45 smithi082 ceph-mon[98694]: pgmap v6922: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 173 KiB/s wr, 12 op/s 2023-09-10T02:50:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12675.913491 secs 2023-09-10T02:50:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:45 smithi098 ceph-mon[103124]: pgmap v6922: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 173 KiB/s wr, 12 op/s 2023-09-10T02:50:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:47 smithi170 ceph-mon[102832]: pgmap v6923: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 237 KiB/s wr, 18 op/s 2023-09-10T02:50:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:47 smithi082 ceph-mon[98694]: pgmap v6923: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 237 KiB/s wr, 18 op/s 2023-09-10T02:50:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:47 smithi098 ceph-mon[103124]: pgmap v6923: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 237 KiB/s wr, 18 op/s 2023-09-10T02:50:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12678.836986 secs 2023-09-10T02:50:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12678.836986 secs 2023-09-10T02:50:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12678.836986 secs 2023-09-10T02:50:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12680.202078 secs 2023-09-10T02:50:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:49 smithi170 ceph-mon[102832]: pgmap v6924: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T02:50:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12680.202078 secs 2023-09-10T02:50:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:49 smithi082 ceph-mon[98694]: pgmap v6924: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T02:50:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12680.202078 secs 2023-09-10T02:50:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:49 smithi098 ceph-mon[103124]: pgmap v6924: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T02:50:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12680.913547 secs 2023-09-10T02:50:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12680.913547 secs 2023-09-10T02:50:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12680.913547 secs 2023-09-10T02:50:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:51 smithi170 ceph-mon[102832]: pgmap v6925: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T02:50:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:51 smithi082 ceph-mon[98694]: pgmap v6925: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T02:50:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:51 smithi098 ceph-mon[103124]: pgmap v6925: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T02:50:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:53 smithi170 ceph-mon[102832]: pgmap v6926: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 179 KiB/s wr, 18 op/s 2023-09-10T02:50:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12683.837106 secs 2023-09-10T02:50:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:53 smithi082 ceph-mon[98694]: pgmap v6926: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 179 KiB/s wr, 18 op/s 2023-09-10T02:50:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12683.837106 secs 2023-09-10T02:50:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:53 smithi098 ceph-mon[103124]: pgmap v6926: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 179 KiB/s wr, 18 op/s 2023-09-10T02:50:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12683.837106 secs 2023-09-10T02:50:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12685.202135 secs 2023-09-10T02:50:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12685.202135 secs 2023-09-10T02:50:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12685.202135 secs 2023-09-10T02:50:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12685.913610 secs 2023-09-10T02:50:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:55 smithi170 ceph-mon[102832]: pgmap v6927: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T02:50:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12685.913610 secs 2023-09-10T02:50:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:55 smithi082 ceph-mon[98694]: pgmap v6927: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T02:50:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12685.913610 secs 2023-09-10T02:50:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:55 smithi098 ceph-mon[103124]: pgmap v6927: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T02:50:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:57 smithi170 ceph-mon[102832]: pgmap v6928: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 140 KiB/s wr, 16 op/s 2023-09-10T02:50:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:57 smithi082 ceph-mon[98694]: pgmap v6928: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 140 KiB/s wr, 16 op/s 2023-09-10T02:50:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:57 smithi098 ceph-mon[103124]: pgmap v6928: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 140 KiB/s wr, 16 op/s 2023-09-10T02:50:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12688.837167 secs 2023-09-10T02:50:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12688.837167 secs 2023-09-10T02:50:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12688.837167 secs 2023-09-10T02:50:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12690.202193 secs 2023-09-10T02:50:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:50:59 smithi170 ceph-mon[102832]: pgmap v6929: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T02:50:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12690.202193 secs 2023-09-10T02:50:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:50:59 smithi082 ceph-mon[98694]: pgmap v6929: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T02:50:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12690.202193 secs 2023-09-10T02:50:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:50:59 smithi098 ceph-mon[103124]: pgmap v6929: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T02:51:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12690.913673 secs 2023-09-10T02:51:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12690.913673 secs 2023-09-10T02:51:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12690.913673 secs 2023-09-10T02:51:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:01 smithi170 ceph-mon[102832]: pgmap v6930: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T02:51:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:01 smithi082 ceph-mon[98694]: pgmap v6930: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T02:51:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:01 smithi098 ceph-mon[103124]: pgmap v6930: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T02:51:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:03 smithi170 ceph-mon[102832]: pgmap v6931: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 103 KiB/s wr, 14 op/s 2023-09-10T02:51:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12693.837287 secs 2023-09-10T02:51:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:03 smithi082 ceph-mon[98694]: pgmap v6931: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 103 KiB/s wr, 14 op/s 2023-09-10T02:51:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12693.837287 secs 2023-09-10T02:51:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:03 smithi098 ceph-mon[103124]: pgmap v6931: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 103 KiB/s wr, 14 op/s 2023-09-10T02:51:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12693.837287 secs 2023-09-10T02:51:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12695.202249 secs 2023-09-10T02:51:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12695.202249 secs 2023-09-10T02:51:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12695.202249 secs 2023-09-10T02:51:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12695.913759 secs 2023-09-10T02:51:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:05 smithi170 ceph-mon[102832]: pgmap v6932: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T02:51:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12695.913759 secs 2023-09-10T02:51:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:05 smithi082 ceph-mon[98694]: pgmap v6932: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T02:51:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12695.913759 secs 2023-09-10T02:51:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:05 smithi098 ceph-mon[103124]: pgmap v6932: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T02:51:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:07 smithi170 ceph-mon[102832]: pgmap v6933: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 107 KiB/s wr, 28 op/s 2023-09-10T02:51:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:07 smithi082 ceph-mon[98694]: pgmap v6933: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 107 KiB/s wr, 28 op/s 2023-09-10T02:51:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:07 smithi098 ceph-mon[103124]: pgmap v6933: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 107 KiB/s wr, 28 op/s 2023-09-10T02:51:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12698.837349 secs 2023-09-10T02:51:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12698.837349 secs 2023-09-10T02:51:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12698.837349 secs 2023-09-10T02:51:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12700.202308 secs 2023-09-10T02:51:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:09 smithi170 ceph-mon[102832]: pgmap v6934: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 72 KiB/s wr, 24 op/s 2023-09-10T02:51:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:09 smithi170 ceph-mon[102832]: osdmap e501: 12 total, 12 up, 12 in 2023-09-10T02:51:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12700.202308 secs 2023-09-10T02:51:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:09 smithi082 ceph-mon[98694]: pgmap v6934: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 72 KiB/s wr, 24 op/s 2023-09-10T02:51:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:09 smithi082 ceph-mon[98694]: osdmap e501: 12 total, 12 up, 12 in 2023-09-10T02:51:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12700.202308 secs 2023-09-10T02:51:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:09 smithi098 ceph-mon[103124]: pgmap v6934: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 72 KiB/s wr, 24 op/s 2023-09-10T02:51:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:09 smithi098 ceph-mon[103124]: osdmap e501: 12 total, 12 up, 12 in 2023-09-10T02:51:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12700.913822 secs 2023-09-10T02:51:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12700.913822 secs 2023-09-10T02:51:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12700.913822 secs 2023-09-10T02:51:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:11 smithi170 ceph-mon[102832]: pgmap v6936: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 86 KiB/s wr, 28 op/s 2023-09-10T02:51:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:51:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:11 smithi082 ceph-mon[98694]: pgmap v6936: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 86 KiB/s wr, 28 op/s 2023-09-10T02:51:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:51:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:11 smithi098 ceph-mon[103124]: pgmap v6936: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 86 KiB/s wr, 28 op/s 2023-09-10T02:51:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:51:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:51:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:51:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:51:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:51:12.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:51:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:51:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:51:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:51:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:51:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:51:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:51:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:51:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:51:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:51:12.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:51:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:13 smithi170 ceph-mon[102832]: pgmap v6937: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 56 KiB/s wr, 29 op/s 2023-09-10T02:51:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12703.837415 secs 2023-09-10T02:51:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12705.202366 secs 2023-09-10T02:51:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:13 smithi082 ceph-mon[98694]: pgmap v6937: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 56 KiB/s wr, 29 op/s 2023-09-10T02:51:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12703.837415 secs 2023-09-10T02:51:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12705.202366 secs 2023-09-10T02:51:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:13 smithi098 ceph-mon[103124]: pgmap v6937: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 56 KiB/s wr, 29 op/s 2023-09-10T02:51:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12703.837415 secs 2023-09-10T02:51:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12705.202366 secs 2023-09-10T02:51:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12705.913884 secs 2023-09-10T02:51:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12705.913884 secs 2023-09-10T02:51:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12705.913884 secs 2023-09-10T02:51:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:15 smithi170 ceph-mon[102832]: pgmap v6938: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 56 KiB/s wr, 29 op/s 2023-09-10T02:51:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:15 smithi082 ceph-mon[98694]: pgmap v6938: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 56 KiB/s wr, 29 op/s 2023-09-10T02:51:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:15 smithi098 ceph-mon[103124]: pgmap v6938: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 56 KiB/s wr, 29 op/s 2023-09-10T02:51:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:17 smithi170 ceph-mon[102832]: pgmap v6939: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T02:51:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:17 smithi170 ceph-mon[102832]: osdmap e502: 12 total, 12 up, 12 in 2023-09-10T02:51:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:17 smithi082 ceph-mon[98694]: pgmap v6939: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T02:51:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:17 smithi082 ceph-mon[98694]: osdmap e502: 12 total, 12 up, 12 in 2023-09-10T02:51:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:17 smithi098 ceph-mon[103124]: pgmap v6939: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T02:51:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:17 smithi098 ceph-mon[103124]: osdmap e502: 12 total, 12 up, 12 in 2023-09-10T02:51:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12708.837472 secs 2023-09-10T02:51:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12710.202426 secs 2023-09-10T02:51:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12708.837472 secs 2023-09-10T02:51:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12710.202426 secs 2023-09-10T02:51:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12708.837472 secs 2023-09-10T02:51:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12710.202426 secs 2023-09-10T02:51:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:19 smithi170 ceph-mon[102832]: pgmap v6941: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T02:51:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12710.913943 secs 2023-09-10T02:51:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:19 smithi082 ceph-mon[98694]: pgmap v6941: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T02:51:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12710.913943 secs 2023-09-10T02:51:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:19 smithi098 ceph-mon[103124]: pgmap v6941: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T02:51:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12710.913943 secs 2023-09-10T02:51:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:21 smithi170 ceph-mon[102832]: pgmap v6942: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T02:51:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:21 smithi082 ceph-mon[98694]: pgmap v6942: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T02:51:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:21 smithi098 ceph-mon[103124]: pgmap v6942: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T02:51:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:23 smithi170 ceph-mon[102832]: pgmap v6943: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T02:51:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:51:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:23 smithi082 ceph-mon[98694]: pgmap v6943: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T02:51:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:51:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:23 smithi098 ceph-mon[103124]: pgmap v6943: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T02:51:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:51:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12715.202492 secs 2023-09-10T02:51:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:51:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:51:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:51:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12715.202492 secs 2023-09-10T02:51:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:51:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:51:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:51:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12715.202492 secs 2023-09-10T02:51:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:51:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:51:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:51:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12715.914006 secs 2023-09-10T02:51:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:25 smithi170 ceph-mon[102832]: pgmap v6944: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T02:51:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12715.914006 secs 2023-09-10T02:51:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:25 smithi082 ceph-mon[98694]: pgmap v6944: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T02:51:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12715.914006 secs 2023-09-10T02:51:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:25 smithi098 ceph-mon[103124]: pgmap v6944: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T02:51:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:27 smithi170 ceph-mon[102832]: pgmap v6945: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T02:51:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:27 smithi082 ceph-mon[98694]: pgmap v6945: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T02:51:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:27 smithi098 ceph-mon[103124]: pgmap v6945: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T02:51:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12720.202576 secs 2023-09-10T02:51:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:29 smithi170 ceph-mon[102832]: pgmap v6946: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T02:51:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12720.202576 secs 2023-09-10T02:51:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:29 smithi082 ceph-mon[98694]: pgmap v6946: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T02:51:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12720.202576 secs 2023-09-10T02:51:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:29 smithi098 ceph-mon[103124]: pgmap v6946: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T02:51:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12720.914070 secs 2023-09-10T02:51:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12720.914070 secs 2023-09-10T02:51:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12720.914070 secs 2023-09-10T02:51:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:31 smithi170 ceph-mon[102832]: pgmap v6947: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T02:51:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:31 smithi082 ceph-mon[98694]: pgmap v6947: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T02:51:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:31 smithi098 ceph-mon[103124]: pgmap v6947: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T02:51:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:33 smithi082 ceph-mon[98694]: pgmap v6948: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 165 KiB/s wr, 17 op/s 2023-09-10T02:51:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:33 smithi098 ceph-mon[103124]: pgmap v6948: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 165 KiB/s wr, 17 op/s 2023-09-10T02:51:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:33 smithi170 ceph-mon[102832]: pgmap v6948: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 165 KiB/s wr, 17 op/s 2023-09-10T02:51:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12725.202659 secs 2023-09-10T02:51:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12725.202659 secs 2023-09-10T02:51:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12725.202659 secs 2023-09-10T02:51:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12725.914135 secs 2023-09-10T02:51:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:35 smithi170 ceph-mon[102832]: pgmap v6949: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T02:51:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12725.914135 secs 2023-09-10T02:51:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:35 smithi082 ceph-mon[98694]: pgmap v6949: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T02:51:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12725.914135 secs 2023-09-10T02:51:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:35 smithi098 ceph-mon[103124]: pgmap v6949: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T02:51:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:37 smithi170 ceph-mon[102832]: pgmap v6950: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 200 KiB/s wr, 18 op/s 2023-09-10T02:51:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:37 smithi082 ceph-mon[98694]: pgmap v6950: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 200 KiB/s wr, 18 op/s 2023-09-10T02:51:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:37 smithi098 ceph-mon[103124]: pgmap v6950: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 200 KiB/s wr, 18 op/s 2023-09-10T02:51:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12730.202718 secs 2023-09-10T02:51:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:39 smithi170 ceph-mon[102832]: pgmap v6951: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 168 KiB/s wr, 13 op/s 2023-09-10T02:51:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12730.202718 secs 2023-09-10T02:51:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:39 smithi082 ceph-mon[98694]: pgmap v6951: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 168 KiB/s wr, 13 op/s 2023-09-10T02:51:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12730.202718 secs 2023-09-10T02:51:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:39 smithi098 ceph-mon[103124]: pgmap v6951: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 168 KiB/s wr, 13 op/s 2023-09-10T02:51:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12730.914195 secs 2023-09-10T02:51:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12730.914195 secs 2023-09-10T02:51:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12730.914195 secs 2023-09-10T02:51:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:41 smithi170 ceph-mon[102832]: pgmap v6952: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 168 KiB/s wr, 13 op/s 2023-09-10T02:51:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:41 smithi082 ceph-mon[98694]: pgmap v6952: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 168 KiB/s wr, 13 op/s 2023-09-10T02:51:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:41 smithi098 ceph-mon[103124]: pgmap v6952: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 168 KiB/s wr, 13 op/s 2023-09-10T02:51:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:43 smithi170 ceph-mon[102832]: pgmap v6953: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 236 KiB/s wr, 20 op/s 2023-09-10T02:51:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12733.837889 secs 2023-09-10T02:51:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:43 smithi082 ceph-mon[98694]: pgmap v6953: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 236 KiB/s wr, 20 op/s 2023-09-10T02:51:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12733.837889 secs 2023-09-10T02:51:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:43 smithi098 ceph-mon[103124]: pgmap v6953: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 236 KiB/s wr, 20 op/s 2023-09-10T02:51:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12733.837889 secs 2023-09-10T02:51:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12735.202773 secs 2023-09-10T02:51:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:51:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:51:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:51:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12735.202773 secs 2023-09-10T02:51:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:51:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:51:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:51:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12735.202773 secs 2023-09-10T02:51:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:51:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:51:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:51:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12735.914255 secs 2023-09-10T02:51:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:45 smithi170 ceph-mon[102832]: pgmap v6954: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 142 KiB/s wr, 13 op/s 2023-09-10T02:51:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12735.914255 secs 2023-09-10T02:51:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:45 smithi082 ceph-mon[98694]: pgmap v6954: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 142 KiB/s wr, 13 op/s 2023-09-10T02:51:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12735.914255 secs 2023-09-10T02:51:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:45 smithi098 ceph-mon[103124]: pgmap v6954: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 142 KiB/s wr, 13 op/s 2023-09-10T02:51:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:47 smithi170 ceph-mon[102832]: pgmap v6955: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 241 KiB/s wr, 19 op/s 2023-09-10T02:51:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:47 smithi082 ceph-mon[98694]: pgmap v6955: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 241 KiB/s wr, 19 op/s 2023-09-10T02:51:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:47 smithi098 ceph-mon[103124]: pgmap v6955: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 241 KiB/s wr, 19 op/s 2023-09-10T02:51:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12738.837948 secs 2023-09-10T02:51:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12738.837948 secs 2023-09-10T02:51:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12738.837948 secs 2023-09-10T02:51:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12740.202857 secs 2023-09-10T02:51:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:49 smithi170 ceph-mon[102832]: pgmap v6956: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 12 op/s 2023-09-10T02:51:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12740.202857 secs 2023-09-10T02:51:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:49 smithi082 ceph-mon[98694]: pgmap v6956: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 12 op/s 2023-09-10T02:51:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12740.202857 secs 2023-09-10T02:51:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:49 smithi098 ceph-mon[103124]: pgmap v6956: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 12 op/s 2023-09-10T02:51:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12740.914319 secs 2023-09-10T02:51:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12740.914319 secs 2023-09-10T02:51:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12740.914319 secs 2023-09-10T02:51:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:51 smithi170 ceph-mon[102832]: pgmap v6957: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 12 op/s 2023-09-10T02:51:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:51 smithi082 ceph-mon[98694]: pgmap v6957: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 12 op/s 2023-09-10T02:51:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:51 smithi098 ceph-mon[103124]: pgmap v6957: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 12 op/s 2023-09-10T02:51:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:53 smithi170 ceph-mon[102832]: pgmap v6958: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 243 KiB/s wr, 17 op/s 2023-09-10T02:51:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12743.838009 secs 2023-09-10T02:51:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:53 smithi082 ceph-mon[98694]: pgmap v6958: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 243 KiB/s wr, 17 op/s 2023-09-10T02:51:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12743.838009 secs 2023-09-10T02:51:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:53 smithi098 ceph-mon[103124]: pgmap v6958: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 243 KiB/s wr, 17 op/s 2023-09-10T02:51:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12743.838009 secs 2023-09-10T02:51:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12745.202913 secs 2023-09-10T02:51:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12745.202913 secs 2023-09-10T02:51:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12745.202913 secs 2023-09-10T02:51:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12745.914381 secs 2023-09-10T02:51:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:55 smithi170 ceph-mon[102832]: pgmap v6959: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 176 KiB/s wr, 11 op/s 2023-09-10T02:51:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12745.914381 secs 2023-09-10T02:51:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:55 smithi082 ceph-mon[98694]: pgmap v6959: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 176 KiB/s wr, 11 op/s 2023-09-10T02:51:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12745.914381 secs 2023-09-10T02:51:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:55 smithi098 ceph-mon[103124]: pgmap v6959: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 176 KiB/s wr, 11 op/s 2023-09-10T02:51:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:57 smithi170 ceph-mon[102832]: pgmap v6960: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 226 KiB/s wr, 15 op/s 2023-09-10T02:51:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:57 smithi082 ceph-mon[98694]: pgmap v6960: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 226 KiB/s wr, 15 op/s 2023-09-10T02:51:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:57 smithi098 ceph-mon[103124]: pgmap v6960: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 226 KiB/s wr, 15 op/s 2023-09-10T02:51:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12748.838127 secs 2023-09-10T02:51:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12748.838127 secs 2023-09-10T02:51:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12748.838127 secs 2023-09-10T02:51:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12750.202970 secs 2023-09-10T02:51:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:51:59 smithi170 ceph-mon[102832]: pgmap v6961: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 9 op/s 2023-09-10T02:51:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12750.202970 secs 2023-09-10T02:51:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:51:59 smithi082 ceph-mon[98694]: pgmap v6961: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 9 op/s 2023-09-10T02:51:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12750.202970 secs 2023-09-10T02:51:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:51:59 smithi098 ceph-mon[103124]: pgmap v6961: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 9 op/s 2023-09-10T02:52:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12750.914444 secs 2023-09-10T02:52:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12750.914444 secs 2023-09-10T02:52:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12750.914444 secs 2023-09-10T02:52:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:01 smithi170 ceph-mon[102832]: pgmap v6962: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 9 op/s 2023-09-10T02:52:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:01 smithi082 ceph-mon[98694]: pgmap v6962: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 9 op/s 2023-09-10T02:52:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:01 smithi098 ceph-mon[103124]: pgmap v6962: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 9 op/s 2023-09-10T02:52:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:03 smithi170 ceph-mon[102832]: pgmap v6963: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 161 KiB/s wr, 15 op/s 2023-09-10T02:52:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12753.838241 secs 2023-09-10T02:52:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:03 smithi082 ceph-mon[98694]: pgmap v6963: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 161 KiB/s wr, 15 op/s 2023-09-10T02:52:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12753.838241 secs 2023-09-10T02:52:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:03 smithi098 ceph-mon[103124]: pgmap v6963: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 161 KiB/s wr, 15 op/s 2023-09-10T02:52:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12753.838241 secs 2023-09-10T02:52:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12755.203052 secs 2023-09-10T02:52:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12755.203052 secs 2023-09-10T02:52:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12755.203052 secs 2023-09-10T02:52:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12755.914487 secs 2023-09-10T02:52:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:05 smithi170 ceph-mon[102832]: pgmap v6964: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T02:52:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12755.914487 secs 2023-09-10T02:52:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:05 smithi082 ceph-mon[98694]: pgmap v6964: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T02:52:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12755.914487 secs 2023-09-10T02:52:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:05 smithi098 ceph-mon[103124]: pgmap v6964: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T02:52:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:07 smithi170 ceph-mon[102832]: pgmap v6965: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 140 KiB/s wr, 30 op/s 2023-09-10T02:52:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:07 smithi082 ceph-mon[98694]: pgmap v6965: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 140 KiB/s wr, 30 op/s 2023-09-10T02:52:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:07 smithi098 ceph-mon[103124]: pgmap v6965: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 140 KiB/s wr, 30 op/s 2023-09-10T02:52:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12758.838303 secs 2023-09-10T02:52:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12758.838303 secs 2023-09-10T02:52:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12758.838303 secs 2023-09-10T02:52:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12760.203109 secs 2023-09-10T02:52:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:09 smithi170 ceph-mon[102832]: pgmap v6966: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 90 KiB/s wr, 25 op/s 2023-09-10T02:52:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:09 smithi170 ceph-mon[102832]: osdmap e503: 12 total, 12 up, 12 in 2023-09-10T02:52:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12760.203109 secs 2023-09-10T02:52:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:09 smithi082 ceph-mon[98694]: pgmap v6966: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 90 KiB/s wr, 25 op/s 2023-09-10T02:52:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:09 smithi082 ceph-mon[98694]: osdmap e503: 12 total, 12 up, 12 in 2023-09-10T02:52:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12760.203109 secs 2023-09-10T02:52:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:09 smithi098 ceph-mon[103124]: pgmap v6966: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 90 KiB/s wr, 25 op/s 2023-09-10T02:52:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:09 smithi098 ceph-mon[103124]: osdmap e503: 12 total, 12 up, 12 in 2023-09-10T02:52:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12760.914550 secs 2023-09-10T02:52:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12760.914550 secs 2023-09-10T02:52:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12760.914550 secs 2023-09-10T02:52:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:11 smithi170 ceph-mon[102832]: pgmap v6968: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 108 KiB/s wr, 31 op/s 2023-09-10T02:52:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:11 smithi082 ceph-mon[98694]: pgmap v6968: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 108 KiB/s wr, 31 op/s 2023-09-10T02:52:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:11 smithi098 ceph-mon[103124]: pgmap v6968: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 108 KiB/s wr, 31 op/s 2023-09-10T02:52:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:52:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:52:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:52:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:13 smithi082 ceph-mon[98694]: pgmap v6969: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 92 KiB/s wr, 30 op/s 2023-09-10T02:52:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12763.838431 secs 2023-09-10T02:52:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:52:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:52:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:52:13.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:52:13.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12765.203199 secs 2023-09-10T02:52:13.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:52:13.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:52:13.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:52:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:13 smithi098 ceph-mon[103124]: pgmap v6969: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 92 KiB/s wr, 30 op/s 2023-09-10T02:52:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12763.838431 secs 2023-09-10T02:52:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:52:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:52:13.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:52:13.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:52:13.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12765.203199 secs 2023-09-10T02:52:13.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:52:13.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:52:13.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:52:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:13 smithi170 ceph-mon[102832]: pgmap v6969: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 92 KiB/s wr, 30 op/s 2023-09-10T02:52:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12763.838431 secs 2023-09-10T02:52:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:52:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:52:13.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:52:13.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:52:13.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12765.203199 secs 2023-09-10T02:52:13.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:52:13.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:52:13.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:52:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12765.914611 secs 2023-09-10T02:52:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12765.914611 secs 2023-09-10T02:52:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12765.914611 secs 2023-09-10T02:52:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:15 smithi082 ceph-mon[98694]: pgmap v6970: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 92 KiB/s wr, 30 op/s 2023-09-10T02:52:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:15 smithi098 ceph-mon[103124]: pgmap v6970: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 92 KiB/s wr, 30 op/s 2023-09-10T02:52:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:15 smithi170 ceph-mon[102832]: pgmap v6970: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 92 KiB/s wr, 30 op/s 2023-09-10T02:52:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:17 smithi098 ceph-mon[103124]: pgmap v6971: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 13 op/s 2023-09-10T02:52:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:17 smithi098 ceph-mon[103124]: osdmap e504: 12 total, 12 up, 12 in 2023-09-10T02:52:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:17 smithi170 ceph-mon[102832]: pgmap v6971: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 13 op/s 2023-09-10T02:52:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:17 smithi170 ceph-mon[102832]: osdmap e504: 12 total, 12 up, 12 in 2023-09-10T02:52:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:17 smithi082 ceph-mon[98694]: pgmap v6971: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 13 op/s 2023-09-10T02:52:17.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:17 smithi082 ceph-mon[98694]: osdmap e504: 12 total, 12 up, 12 in 2023-09-10T02:52:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12768.838522 secs 2023-09-10T02:52:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12770.203261 secs 2023-09-10T02:52:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12768.838522 secs 2023-09-10T02:52:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12770.203261 secs 2023-09-10T02:52:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12768.838522 secs 2023-09-10T02:52:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12770.203261 secs 2023-09-10T02:52:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:19 smithi098 ceph-mon[103124]: pgmap v6973: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T02:52:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12770.914676 secs 2023-09-10T02:52:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:19 smithi170 ceph-mon[102832]: pgmap v6973: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T02:52:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12770.914676 secs 2023-09-10T02:52:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:19 smithi082 ceph-mon[98694]: pgmap v6973: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T02:52:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12770.914676 secs 2023-09-10T02:52:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:21 smithi098 ceph-mon[103124]: pgmap v6974: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 13 op/s 2023-09-10T02:52:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:21 smithi170 ceph-mon[102832]: pgmap v6974: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 13 op/s 2023-09-10T02:52:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:21 smithi082 ceph-mon[98694]: pgmap v6974: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 13 op/s 2023-09-10T02:52:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:23 smithi170 ceph-mon[102832]: pgmap v6975: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 KiB/s wr, 12 op/s 2023-09-10T02:52:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:52:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:23 smithi082 ceph-mon[98694]: pgmap v6975: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 KiB/s wr, 12 op/s 2023-09-10T02:52:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:52:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:23 smithi098 ceph-mon[103124]: pgmap v6975: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 KiB/s wr, 12 op/s 2023-09-10T02:52:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:52:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12775.203316 secs 2023-09-10T02:52:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:52:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:52:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:52:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12775.203316 secs 2023-09-10T02:52:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:52:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:52:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:52:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12775.203316 secs 2023-09-10T02:52:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:52:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:52:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:52:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12775.914736 secs 2023-09-10T02:52:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:25 smithi170 ceph-mon[102832]: pgmap v6976: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 KiB/s wr, 12 op/s 2023-09-10T02:52:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12775.914736 secs 2023-09-10T02:52:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:25 smithi082 ceph-mon[98694]: pgmap v6976: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 KiB/s wr, 12 op/s 2023-09-10T02:52:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12775.914736 secs 2023-09-10T02:52:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:25 smithi098 ceph-mon[103124]: pgmap v6976: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 KiB/s wr, 12 op/s 2023-09-10T02:52:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:27 smithi170 ceph-mon[102832]: pgmap v6977: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T02:52:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:27 smithi082 ceph-mon[98694]: pgmap v6977: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T02:52:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:27 smithi098 ceph-mon[103124]: pgmap v6977: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T02:52:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12780.203375 secs 2023-09-10T02:52:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:29 smithi170 ceph-mon[102832]: pgmap v6978: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T02:52:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12780.203375 secs 2023-09-10T02:52:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:29 smithi082 ceph-mon[98694]: pgmap v6978: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T02:52:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12780.203375 secs 2023-09-10T02:52:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:29 smithi098 ceph-mon[103124]: pgmap v6978: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 11 op/s 2023-09-10T02:52:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12780.914799 secs 2023-09-10T02:52:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12780.914799 secs 2023-09-10T02:52:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12780.914799 secs 2023-09-10T02:52:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:31 smithi170 ceph-mon[102832]: pgmap v6979: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T02:52:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:31 smithi082 ceph-mon[98694]: pgmap v6979: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T02:52:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:31 smithi098 ceph-mon[103124]: pgmap v6979: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T02:52:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:33 smithi170 ceph-mon[102832]: pgmap v6980: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 163 KiB/s wr, 16 op/s 2023-09-10T02:52:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12783.838717 secs 2023-09-10T02:52:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:33 smithi082 ceph-mon[98694]: pgmap v6980: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 163 KiB/s wr, 16 op/s 2023-09-10T02:52:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12783.838717 secs 2023-09-10T02:52:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:33 smithi098 ceph-mon[103124]: pgmap v6980: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 163 KiB/s wr, 16 op/s 2023-09-10T02:52:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12783.838717 secs 2023-09-10T02:52:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12785.203433 secs 2023-09-10T02:52:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12785.203433 secs 2023-09-10T02:52:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12785.203433 secs 2023-09-10T02:52:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12785.914862 secs 2023-09-10T02:52:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:35 smithi170 ceph-mon[102832]: pgmap v6981: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-09-10T02:52:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12785.914862 secs 2023-09-10T02:52:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:35 smithi082 ceph-mon[98694]: pgmap v6981: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-09-10T02:52:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12785.914862 secs 2023-09-10T02:52:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:35 smithi098 ceph-mon[103124]: pgmap v6981: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-09-10T02:52:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:36 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:52:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:52:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:52:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:36 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:52:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:52:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:52:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:36 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:52:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:52:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:52:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:37 smithi170 ceph-mon[102832]: pgmap v6982: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 250 KiB/s wr, 17 op/s 2023-09-10T02:52:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:37 smithi082 ceph-mon[98694]: pgmap v6982: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 250 KiB/s wr, 17 op/s 2023-09-10T02:52:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:37 smithi098 ceph-mon[103124]: pgmap v6982: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 250 KiB/s wr, 17 op/s 2023-09-10T02:52:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12790.203493 secs 2023-09-10T02:52:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:39 smithi170 ceph-mon[102832]: pgmap v6983: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 170 KiB/s wr, 12 op/s 2023-09-10T02:52:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:52:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12790.203493 secs 2023-09-10T02:52:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:39 smithi082 ceph-mon[98694]: pgmap v6983: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 170 KiB/s wr, 12 op/s 2023-09-10T02:52:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:52:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12790.203493 secs 2023-09-10T02:52:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:39 smithi098 ceph-mon[103124]: pgmap v6983: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 170 KiB/s wr, 12 op/s 2023-09-10T02:52:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:52:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12790.914923 secs 2023-09-10T02:52:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:52:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:52:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12790.914923 secs 2023-09-10T02:52:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:52:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:52:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12790.914923 secs 2023-09-10T02:52:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:52:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:52:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:41 smithi170 ceph-mon[102832]: pgmap v6984: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 170 KiB/s wr, 12 op/s 2023-09-10T02:52:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:41 smithi082 ceph-mon[98694]: pgmap v6984: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 170 KiB/s wr, 12 op/s 2023-09-10T02:52:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:41 smithi098 ceph-mon[103124]: pgmap v6984: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 170 KiB/s wr, 12 op/s 2023-09-10T02:52:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:42 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:52:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:42 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:52:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:42 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:52:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:43 smithi170 ceph-mon[102832]: pgmap v6985: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 292 KiB/s wr, 19 op/s 2023-09-10T02:52:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12793.838850 secs 2023-09-10T02:52:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:43 smithi082 ceph-mon[98694]: pgmap v6985: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 292 KiB/s wr, 19 op/s 2023-09-10T02:52:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12793.838850 secs 2023-09-10T02:52:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:43 smithi098 ceph-mon[103124]: pgmap v6985: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 292 KiB/s wr, 19 op/s 2023-09-10T02:52:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12793.838850 secs 2023-09-10T02:52:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12795.203553 secs 2023-09-10T02:52:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:52:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:52:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12795.203553 secs 2023-09-10T02:52:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:52:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:52:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12795.203553 secs 2023-09-10T02:52:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:52:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:52:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12795.914985 secs 2023-09-10T02:52:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:45 smithi170 ceph-mon[102832]: pgmap v6986: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 209 KiB/s wr, 12 op/s 2023-09-10T02:52:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12795.914985 secs 2023-09-10T02:52:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:45 smithi082 ceph-mon[98694]: pgmap v6986: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 209 KiB/s wr, 12 op/s 2023-09-10T02:52:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12795.914985 secs 2023-09-10T02:52:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:45 smithi098 ceph-mon[103124]: pgmap v6986: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 209 KiB/s wr, 12 op/s 2023-09-10T02:52:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:47 smithi170 ceph-mon[102832]: pgmap v6987: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 299 KiB/s wr, 18 op/s 2023-09-10T02:52:47.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:47 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:52:47.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:47 smithi082 ceph-mon[98694]: pgmap v6987: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 299 KiB/s wr, 18 op/s 2023-09-10T02:52:47.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:47 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:52:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:47 smithi098 ceph-mon[103124]: pgmap v6987: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 299 KiB/s wr, 18 op/s 2023-09-10T02:52:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:47 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:52:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12798.838911 secs 2023-09-10T02:52:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12798.838911 secs 2023-09-10T02:52:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12798.838911 secs 2023-09-10T02:52:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12800.203610 secs 2023-09-10T02:52:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:49 smithi170 ceph-mon[102832]: pgmap v6988: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 212 KiB/s wr, 13 op/s 2023-09-10T02:52:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12800.203610 secs 2023-09-10T02:52:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:49 smithi082 ceph-mon[98694]: pgmap v6988: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 212 KiB/s wr, 13 op/s 2023-09-10T02:52:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12800.203610 secs 2023-09-10T02:52:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:49 smithi098 ceph-mon[103124]: pgmap v6988: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 212 KiB/s wr, 13 op/s 2023-09-10T02:52:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12800.915049 secs 2023-09-10T02:52:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12800.915049 secs 2023-09-10T02:52:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12800.915049 secs 2023-09-10T02:52:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:51 smithi170 ceph-mon[102832]: pgmap v6989: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 212 KiB/s wr, 13 op/s 2023-09-10T02:52:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:51 smithi082 ceph-mon[98694]: pgmap v6989: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 212 KiB/s wr, 13 op/s 2023-09-10T02:52:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:51 smithi098 ceph-mon[103124]: pgmap v6989: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 212 KiB/s wr, 13 op/s 2023-09-10T02:52:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:53 smithi170 ceph-mon[102832]: pgmap v6990: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 254 KiB/s wr, 18 op/s 2023-09-10T02:52:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12803.839005 secs 2023-09-10T02:52:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:53 smithi082 ceph-mon[98694]: pgmap v6990: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 254 KiB/s wr, 18 op/s 2023-09-10T02:52:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12803.839005 secs 2023-09-10T02:52:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:53 smithi098 ceph-mon[103124]: pgmap v6990: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 254 KiB/s wr, 18 op/s 2023-09-10T02:52:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12803.839005 secs 2023-09-10T02:52:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12805.203672 secs 2023-09-10T02:52:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12805.203672 secs 2023-09-10T02:52:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12805.203672 secs 2023-09-10T02:52:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12805.915109 secs 2023-09-10T02:52:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:55 smithi170 ceph-mon[102832]: pgmap v6991: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T02:52:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12805.915109 secs 2023-09-10T02:52:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:55 smithi082 ceph-mon[98694]: pgmap v6991: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T02:52:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12805.915109 secs 2023-09-10T02:52:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:55 smithi098 ceph-mon[103124]: pgmap v6991: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T02:52:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:57 smithi170 ceph-mon[102832]: pgmap v6992: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 133 KiB/s wr, 16 op/s 2023-09-10T02:52:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:57 smithi082 ceph-mon[98694]: pgmap v6992: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 133 KiB/s wr, 16 op/s 2023-09-10T02:52:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:57 smithi098 ceph-mon[103124]: pgmap v6992: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 133 KiB/s wr, 16 op/s 2023-09-10T02:52:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12808.839154 secs 2023-09-10T02:52:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12808.839154 secs 2023-09-10T02:52:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12808.839154 secs 2023-09-10T02:52:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12810.203731 secs 2023-09-10T02:52:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:52:59 smithi170 ceph-mon[102832]: pgmap v6993: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-10T02:52:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12810.203731 secs 2023-09-10T02:52:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:52:59 smithi082 ceph-mon[98694]: pgmap v6993: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-10T02:52:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12810.203731 secs 2023-09-10T02:52:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:52:59 smithi098 ceph-mon[103124]: pgmap v6993: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-10T02:53:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12810.915172 secs 2023-09-10T02:53:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12810.915172 secs 2023-09-10T02:53:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12810.915172 secs 2023-09-10T02:53:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:01 smithi170 ceph-mon[102832]: pgmap v6994: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-10T02:53:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:01 smithi082 ceph-mon[98694]: pgmap v6994: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-10T02:53:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:01 smithi098 ceph-mon[103124]: pgmap v6994: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 10 op/s 2023-09-10T02:53:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:03 smithi170 ceph-mon[102832]: pgmap v6995: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 15 op/s 2023-09-10T02:53:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12813.839216 secs 2023-09-10T02:53:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:03 smithi082 ceph-mon[98694]: pgmap v6995: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 15 op/s 2023-09-10T02:53:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12813.839216 secs 2023-09-10T02:53:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:03 smithi098 ceph-mon[103124]: pgmap v6995: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 15 op/s 2023-09-10T02:53:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12813.839216 secs 2023-09-10T02:53:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12815.203790 secs 2023-09-10T02:53:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12815.203790 secs 2023-09-10T02:53:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12815.203790 secs 2023-09-10T02:53:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12815.915233 secs 2023-09-10T02:53:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:05 smithi170 ceph-mon[102832]: pgmap v6996: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 9 op/s 2023-09-10T02:53:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12815.915233 secs 2023-09-10T02:53:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:05 smithi082 ceph-mon[98694]: pgmap v6996: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 9 op/s 2023-09-10T02:53:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12815.915233 secs 2023-09-10T02:53:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:05 smithi098 ceph-mon[103124]: pgmap v6996: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 9 op/s 2023-09-10T02:53:07.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:07 smithi170 ceph-mon[102832]: pgmap v6997: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T02:53:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:07 smithi082 ceph-mon[98694]: pgmap v6997: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T02:53:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:07 smithi098 ceph-mon[103124]: pgmap v6997: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T02:53:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12818.839275 secs 2023-09-10T02:53:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12818.839275 secs 2023-09-10T02:53:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12818.839275 secs 2023-09-10T02:53:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12820.203848 secs 2023-09-10T02:53:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:09 smithi170 ceph-mon[102832]: pgmap v6998: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 76 KiB/s wr, 24 op/s 2023-09-10T02:53:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:09 smithi170 ceph-mon[102832]: osdmap e505: 12 total, 12 up, 12 in 2023-09-10T02:53:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12820.203848 secs 2023-09-10T02:53:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:09 smithi082 ceph-mon[98694]: pgmap v6998: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 76 KiB/s wr, 24 op/s 2023-09-10T02:53:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:09 smithi082 ceph-mon[98694]: osdmap e505: 12 total, 12 up, 12 in 2023-09-10T02:53:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12820.203848 secs 2023-09-10T02:53:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:09 smithi098 ceph-mon[103124]: pgmap v6998: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 76 KiB/s wr, 24 op/s 2023-09-10T02:53:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:09 smithi098 ceph-mon[103124]: osdmap e505: 12 total, 12 up, 12 in 2023-09-10T02:53:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12820.915295 secs 2023-09-10T02:53:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12820.915295 secs 2023-09-10T02:53:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12820.915295 secs 2023-09-10T02:53:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:11 smithi170 ceph-mon[102832]: pgmap v7000: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T02:53:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:11 smithi082 ceph-mon[98694]: pgmap v7000: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T02:53:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:11 smithi098 ceph-mon[103124]: pgmap v7000: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T02:53:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:13 smithi170 ceph-mon[102832]: pgmap v7001: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 28 op/s 2023-09-10T02:53:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12823.839371 secs 2023-09-10T02:53:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:13 smithi082 ceph-mon[98694]: pgmap v7001: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 28 op/s 2023-09-10T02:53:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12823.839371 secs 2023-09-10T02:53:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:13 smithi098 ceph-mon[103124]: pgmap v7001: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 28 op/s 2023-09-10T02:53:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12823.839371 secs 2023-09-10T02:53:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12825.203933 secs 2023-09-10T02:53:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:53:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12825.203933 secs 2023-09-10T02:53:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:53:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12825.203933 secs 2023-09-10T02:53:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:53:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12825.915359 secs 2023-09-10T02:53:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:15 smithi170 ceph-mon[102832]: pgmap v7002: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 28 op/s 2023-09-10T02:53:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:53:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:53:15.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:53:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12825.915359 secs 2023-09-10T02:53:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:15 smithi082 ceph-mon[98694]: pgmap v7002: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 28 op/s 2023-09-10T02:53:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:53:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:53:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:53:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12825.915359 secs 2023-09-10T02:53:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:15 smithi098 ceph-mon[103124]: pgmap v7002: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 28 op/s 2023-09-10T02:53:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:53:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:53:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:53:17.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:17 smithi170 ceph-mon[102832]: pgmap v7003: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T02:53:17.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:17 smithi170 ceph-mon[102832]: osdmap e506: 12 total, 12 up, 12 in 2023-09-10T02:53:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:17 smithi082 ceph-mon[98694]: pgmap v7003: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T02:53:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:17 smithi082 ceph-mon[98694]: osdmap e506: 12 total, 12 up, 12 in 2023-09-10T02:53:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:17 smithi098 ceph-mon[103124]: pgmap v7003: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T02:53:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:17 smithi098 ceph-mon[103124]: osdmap e506: 12 total, 12 up, 12 in 2023-09-10T02:53:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12828.839429 secs 2023-09-10T02:53:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12828.839429 secs 2023-09-10T02:53:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12828.839429 secs 2023-09-10T02:53:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12830.203991 secs 2023-09-10T02:53:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:19 smithi170 ceph-mon[102832]: pgmap v7005: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T02:53:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12830.203991 secs 2023-09-10T02:53:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:19 smithi082 ceph-mon[98694]: pgmap v7005: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T02:53:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12830.203991 secs 2023-09-10T02:53:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:19 smithi098 ceph-mon[103124]: pgmap v7005: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T02:53:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12830.915425 secs 2023-09-10T02:53:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12830.915425 secs 2023-09-10T02:53:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12830.915425 secs 2023-09-10T02:53:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:21 smithi170 ceph-mon[102832]: pgmap v7006: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T02:53:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:21 smithi082 ceph-mon[98694]: pgmap v7006: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T02:53:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:21 smithi098 ceph-mon[103124]: pgmap v7006: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T02:53:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:23 smithi170 ceph-mon[102832]: pgmap v7007: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T02:53:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:53:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:23 smithi082 ceph-mon[98694]: pgmap v7007: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T02:53:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:53:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:23 smithi098 ceph-mon[103124]: pgmap v7007: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T02:53:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:53:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12835.204048 secs 2023-09-10T02:53:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:53:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:53:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:53:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12835.204048 secs 2023-09-10T02:53:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:53:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:53:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:53:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12835.204048 secs 2023-09-10T02:53:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:53:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:53:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:53:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12835.915493 secs 2023-09-10T02:53:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:25 smithi170 ceph-mon[102832]: pgmap v7008: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T02:53:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12835.915493 secs 2023-09-10T02:53:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:25 smithi082 ceph-mon[98694]: pgmap v7008: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T02:53:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12835.915493 secs 2023-09-10T02:53:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:25 smithi098 ceph-mon[103124]: pgmap v7008: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T02:53:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:27 smithi170 ceph-mon[102832]: pgmap v7009: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T02:53:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:27 smithi082 ceph-mon[98694]: pgmap v7009: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T02:53:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:27 smithi098 ceph-mon[103124]: pgmap v7009: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T02:53:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12840.204145 secs 2023-09-10T02:53:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:29 smithi170 ceph-mon[102832]: pgmap v7010: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T02:53:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12840.204145 secs 2023-09-10T02:53:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:29 smithi082 ceph-mon[98694]: pgmap v7010: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T02:53:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12840.204145 secs 2023-09-10T02:53:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:29 smithi098 ceph-mon[103124]: pgmap v7010: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T02:53:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12840.915560 secs 2023-09-10T02:53:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12840.915560 secs 2023-09-10T02:53:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12840.915560 secs 2023-09-10T02:53:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:31 smithi170 ceph-mon[102832]: pgmap v7011: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 9 op/s 2023-09-10T02:53:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:31 smithi082 ceph-mon[98694]: pgmap v7011: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 9 op/s 2023-09-10T02:53:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:31 smithi098 ceph-mon[103124]: pgmap v7011: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 9 op/s 2023-09-10T02:53:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:33 smithi170 ceph-mon[102832]: pgmap v7012: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 130 KiB/s wr, 15 op/s 2023-09-10T02:53:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:33 smithi082 ceph-mon[98694]: pgmap v7012: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 130 KiB/s wr, 15 op/s 2023-09-10T02:53:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:33 smithi098 ceph-mon[103124]: pgmap v7012: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 130 KiB/s wr, 15 op/s 2023-09-10T02:53:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12845.204228 secs 2023-09-10T02:53:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12845.204228 secs 2023-09-10T02:53:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12845.204228 secs 2023-09-10T02:53:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12845.915624 secs 2023-09-10T02:53:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:35 smithi170 ceph-mon[102832]: pgmap v7013: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 130 KiB/s wr, 10 op/s 2023-09-10T02:53:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12845.915624 secs 2023-09-10T02:53:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:35 smithi082 ceph-mon[98694]: pgmap v7013: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 130 KiB/s wr, 10 op/s 2023-09-10T02:53:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12845.915624 secs 2023-09-10T02:53:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:35 smithi098 ceph-mon[103124]: pgmap v7013: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 130 KiB/s wr, 10 op/s 2023-09-10T02:53:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:37 smithi170 ceph-mon[102832]: pgmap v7014: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 188 KiB/s wr, 17 op/s 2023-09-10T02:53:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:37 smithi082 ceph-mon[98694]: pgmap v7014: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 188 KiB/s wr, 17 op/s 2023-09-10T02:53:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:37 smithi098 ceph-mon[103124]: pgmap v7014: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 188 KiB/s wr, 17 op/s 2023-09-10T02:53:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12848.839774 secs 2023-09-10T02:53:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12848.839774 secs 2023-09-10T02:53:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12848.839774 secs 2023-09-10T02:53:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12850.204290 secs 2023-09-10T02:53:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:39 smithi170 ceph-mon[102832]: pgmap v7015: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 136 KiB/s wr, 12 op/s 2023-09-10T02:53:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12850.204290 secs 2023-09-10T02:53:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:39 smithi082 ceph-mon[98694]: pgmap v7015: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 136 KiB/s wr, 12 op/s 2023-09-10T02:53:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12850.204290 secs 2023-09-10T02:53:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:39 smithi098 ceph-mon[103124]: pgmap v7015: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 136 KiB/s wr, 12 op/s 2023-09-10T02:53:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12850.915689 secs 2023-09-10T02:53:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:53:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:53:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:53:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12850.915689 secs 2023-09-10T02:53:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:53:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:53:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:53:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12850.915689 secs 2023-09-10T02:53:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:53:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:53:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:53:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:41 smithi170 ceph-mon[102832]: pgmap v7016: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 136 KiB/s wr, 12 op/s 2023-09-10T02:53:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:41 smithi082 ceph-mon[98694]: pgmap v7016: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 136 KiB/s wr, 12 op/s 2023-09-10T02:53:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:41 smithi098 ceph-mon[103124]: pgmap v7016: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 136 KiB/s wr, 12 op/s 2023-09-10T02:53:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:43 smithi170 ceph-mon[102832]: pgmap v7017: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 250 KiB/s wr, 18 op/s 2023-09-10T02:53:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:53:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:43 smithi082 ceph-mon[98694]: pgmap v7017: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 250 KiB/s wr, 18 op/s 2023-09-10T02:53:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:53:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:43 smithi098 ceph-mon[103124]: pgmap v7017: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 250 KiB/s wr, 18 op/s 2023-09-10T02:53:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:53:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12855.204368 secs 2023-09-10T02:53:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:53:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:53:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12855.204368 secs 2023-09-10T02:53:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:53:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:53:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12855.204368 secs 2023-09-10T02:53:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:53:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:53:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12855.915756 secs 2023-09-10T02:53:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:45 smithi170 ceph-mon[102832]: pgmap v7018: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 171 KiB/s wr, 13 op/s 2023-09-10T02:53:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12855.915756 secs 2023-09-10T02:53:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:45 smithi082 ceph-mon[98694]: pgmap v7018: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 171 KiB/s wr, 13 op/s 2023-09-10T02:53:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12855.915756 secs 2023-09-10T02:53:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:45 smithi098 ceph-mon[103124]: pgmap v7018: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 171 KiB/s wr, 13 op/s 2023-09-10T02:53:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:47 smithi170 ceph-mon[102832]: pgmap v7019: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 189 KiB/s wr, 18 op/s 2023-09-10T02:53:47.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:47 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:53:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:47 smithi082 ceph-mon[98694]: pgmap v7019: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 189 KiB/s wr, 18 op/s 2023-09-10T02:53:47.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:47 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:53:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:47 smithi098 ceph-mon[103124]: pgmap v7019: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 189 KiB/s wr, 18 op/s 2023-09-10T02:53:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:47 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:53:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12860.204458 secs 2023-09-10T02:53:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:49 smithi170 ceph-mon[102832]: pgmap v7020: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 131 KiB/s wr, 11 op/s 2023-09-10T02:53:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12860.204458 secs 2023-09-10T02:53:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:49 smithi082 ceph-mon[98694]: pgmap v7020: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 131 KiB/s wr, 11 op/s 2023-09-10T02:53:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12860.204458 secs 2023-09-10T02:53:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:49 smithi098 ceph-mon[103124]: pgmap v7020: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 131 KiB/s wr, 11 op/s 2023-09-10T02:53:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12860.915820 secs 2023-09-10T02:53:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12860.915820 secs 2023-09-10T02:53:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12860.915820 secs 2023-09-10T02:53:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:51 smithi170 ceph-mon[102832]: pgmap v7021: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 131 KiB/s wr, 11 op/s 2023-09-10T02:53:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:51 smithi082 ceph-mon[98694]: pgmap v7021: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 131 KiB/s wr, 11 op/s 2023-09-10T02:53:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:51 smithi098 ceph-mon[103124]: pgmap v7021: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 131 KiB/s wr, 11 op/s 2023-09-10T02:53:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:53 smithi170 ceph-mon[102832]: pgmap v7022: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 377 KiB/s wr, 18 op/s 2023-09-10T02:53:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12863.840004 secs 2023-09-10T02:53:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:53 smithi082 ceph-mon[98694]: pgmap v7022: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 377 KiB/s wr, 18 op/s 2023-09-10T02:53:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12863.840004 secs 2023-09-10T02:53:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:53 smithi098 ceph-mon[103124]: pgmap v7022: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 377 KiB/s wr, 18 op/s 2023-09-10T02:53:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12863.840004 secs 2023-09-10T02:53:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12865.204525 secs 2023-09-10T02:53:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12865.204525 secs 2023-09-10T02:53:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12865.204525 secs 2023-09-10T02:53:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12865.915888 secs 2023-09-10T02:53:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:55 smithi170 ceph-mon[102832]: pgmap v7023: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T02:53:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12865.915888 secs 2023-09-10T02:53:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:55 smithi082 ceph-mon[98694]: pgmap v7023: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T02:53:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12865.915888 secs 2023-09-10T02:53:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:55 smithi098 ceph-mon[103124]: pgmap v7023: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T02:53:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:53:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:53:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:53:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:53:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:53:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:53:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:53:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:53:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:53:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:57 smithi170 ceph-mon[102832]: pgmap v7024: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 265 KiB/s wr, 17 op/s 2023-09-10T02:53:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:57 smithi082 ceph-mon[98694]: pgmap v7024: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 265 KiB/s wr, 17 op/s 2023-09-10T02:53:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:57 smithi098 ceph-mon[103124]: pgmap v7024: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 265 KiB/s wr, 17 op/s 2023-09-10T02:53:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12868.840106 secs 2023-09-10T02:53:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12868.840106 secs 2023-09-10T02:53:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12868.840106 secs 2023-09-10T02:53:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12870.204607 secs 2023-09-10T02:53:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:53:59 smithi170 ceph-mon[102832]: pgmap v7025: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 247 KiB/s wr, 12 op/s 2023-09-10T02:53:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12870.204607 secs 2023-09-10T02:53:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:53:59 smithi082 ceph-mon[98694]: pgmap v7025: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 247 KiB/s wr, 12 op/s 2023-09-10T02:53:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12870.204607 secs 2023-09-10T02:53:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:53:59 smithi098 ceph-mon[103124]: pgmap v7025: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 247 KiB/s wr, 12 op/s 2023-09-10T02:54:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12870.915953 secs 2023-09-10T02:54:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12870.915953 secs 2023-09-10T02:54:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12870.915953 secs 2023-09-10T02:54:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:01 smithi170 ceph-mon[102832]: pgmap v7026: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 247 KiB/s wr, 12 op/s 2023-09-10T02:54:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:01 smithi082 ceph-mon[98694]: pgmap v7026: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 247 KiB/s wr, 12 op/s 2023-09-10T02:54:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:01 smithi098 ceph-mon[103124]: pgmap v7026: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 247 KiB/s wr, 12 op/s 2023-09-10T02:54:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:03 smithi170 ceph-mon[102832]: pgmap v7027: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 337 KiB/s wr, 17 op/s 2023-09-10T02:54:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12873.840178 secs 2023-09-10T02:54:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:03 smithi082 ceph-mon[98694]: pgmap v7027: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 337 KiB/s wr, 17 op/s 2023-09-10T02:54:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12873.840178 secs 2023-09-10T02:54:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:03 smithi098 ceph-mon[103124]: pgmap v7027: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 337 KiB/s wr, 17 op/s 2023-09-10T02:54:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12873.840178 secs 2023-09-10T02:54:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12875.204689 secs 2023-09-10T02:54:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12875.204689 secs 2023-09-10T02:54:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12875.204689 secs 2023-09-10T02:54:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12875.916018 secs 2023-09-10T02:54:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:05 smithi170 ceph-mon[102832]: pgmap v7028: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T02:54:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12875.916018 secs 2023-09-10T02:54:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:05 smithi082 ceph-mon[98694]: pgmap v7028: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T02:54:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12875.916018 secs 2023-09-10T02:54:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:05 smithi098 ceph-mon[103124]: pgmap v7028: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T02:54:07.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:07 smithi170 ceph-mon[102832]: pgmap v7029: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 136 KiB/s wr, 29 op/s 2023-09-10T02:54:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:07 smithi082 ceph-mon[98694]: pgmap v7029: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 136 KiB/s wr, 29 op/s 2023-09-10T02:54:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:07 smithi098 ceph-mon[103124]: pgmap v7029: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 136 KiB/s wr, 29 op/s 2023-09-10T02:54:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12878.840280 secs 2023-09-10T02:54:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12878.840280 secs 2023-09-10T02:54:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12878.840280 secs 2023-09-10T02:54:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12880.204776 secs 2023-09-10T02:54:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:09 smithi170 ceph-mon[102832]: pgmap v7030: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 135 KiB/s wr, 24 op/s 2023-09-10T02:54:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:09 smithi170 ceph-mon[102832]: osdmap e507: 12 total, 12 up, 12 in 2023-09-10T02:54:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12880.204776 secs 2023-09-10T02:54:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:09 smithi082 ceph-mon[98694]: pgmap v7030: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 135 KiB/s wr, 24 op/s 2023-09-10T02:54:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:09 smithi082 ceph-mon[98694]: osdmap e507: 12 total, 12 up, 12 in 2023-09-10T02:54:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12880.204776 secs 2023-09-10T02:54:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:09 smithi098 ceph-mon[103124]: pgmap v7030: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 135 KiB/s wr, 24 op/s 2023-09-10T02:54:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:09 smithi098 ceph-mon[103124]: osdmap e507: 12 total, 12 up, 12 in 2023-09-10T02:54:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12880.916086 secs 2023-09-10T02:54:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12880.916086 secs 2023-09-10T02:54:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12880.916086 secs 2023-09-10T02:54:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:11 smithi170 ceph-mon[102832]: pgmap v7032: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 163 KiB/s wr, 29 op/s 2023-09-10T02:54:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:11 smithi082 ceph-mon[98694]: pgmap v7032: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 163 KiB/s wr, 29 op/s 2023-09-10T02:54:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:11 smithi098 ceph-mon[103124]: pgmap v7032: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 163 KiB/s wr, 29 op/s 2023-09-10T02:54:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:13 smithi170 ceph-mon[102832]: pgmap v7033: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 103 KiB/s wr, 29 op/s 2023-09-10T02:54:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12883.840340 secs 2023-09-10T02:54:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:13 smithi082 ceph-mon[98694]: pgmap v7033: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 103 KiB/s wr, 29 op/s 2023-09-10T02:54:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12883.840340 secs 2023-09-10T02:54:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:13 smithi098 ceph-mon[103124]: pgmap v7033: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 103 KiB/s wr, 29 op/s 2023-09-10T02:54:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12883.840340 secs 2023-09-10T02:54:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12885.204858 secs 2023-09-10T02:54:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12885.204858 secs 2023-09-10T02:54:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12885.204858 secs 2023-09-10T02:54:15.145 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12885.916152 secs 2023-09-10T02:54:15.145 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:15 smithi098 ceph-mon[103124]: pgmap v7034: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 103 KiB/s wr, 29 op/s 2023-09-10T02:54:15.145 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:54:15.146 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:54:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12885.916152 secs 2023-09-10T02:54:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:15 smithi170 ceph-mon[102832]: pgmap v7034: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 103 KiB/s wr, 29 op/s 2023-09-10T02:54:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:54:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:54:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:54:15.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:54:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12885.916152 secs 2023-09-10T02:54:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:15 smithi082 ceph-mon[98694]: pgmap v7034: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 103 KiB/s wr, 29 op/s 2023-09-10T02:54:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:54:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:54:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:54:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:54:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:54:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:54:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:17 smithi170 ceph-mon[102832]: pgmap v7035: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 12 op/s 2023-09-10T02:54:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:17 smithi170 ceph-mon[102832]: osdmap e508: 12 total, 12 up, 12 in 2023-09-10T02:54:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:17 smithi082 ceph-mon[98694]: pgmap v7035: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 12 op/s 2023-09-10T02:54:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:17 smithi082 ceph-mon[98694]: osdmap e508: 12 total, 12 up, 12 in 2023-09-10T02:54:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:17 smithi098 ceph-mon[103124]: pgmap v7035: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 12 op/s 2023-09-10T02:54:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:17 smithi098 ceph-mon[103124]: osdmap e508: 12 total, 12 up, 12 in 2023-09-10T02:54:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12888.840401 secs 2023-09-10T02:54:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12888.840401 secs 2023-09-10T02:54:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12888.840401 secs 2023-09-10T02:54:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12890.204947 secs 2023-09-10T02:54:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:19 smithi170 ceph-mon[102832]: pgmap v7037: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 12 op/s 2023-09-10T02:54:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12890.204947 secs 2023-09-10T02:54:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:19 smithi082 ceph-mon[98694]: pgmap v7037: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 12 op/s 2023-09-10T02:54:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12890.204947 secs 2023-09-10T02:54:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:19 smithi098 ceph-mon[103124]: pgmap v7037: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 12 op/s 2023-09-10T02:54:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12890.916224 secs 2023-09-10T02:54:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12890.916224 secs 2023-09-10T02:54:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12890.916224 secs 2023-09-10T02:54:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:21 smithi170 ceph-mon[102832]: pgmap v7038: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 12 op/s 2023-09-10T02:54:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:21 smithi082 ceph-mon[98694]: pgmap v7038: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 12 op/s 2023-09-10T02:54:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:21 smithi098 ceph-mon[103124]: pgmap v7038: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 12 op/s 2023-09-10T02:54:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:23 smithi170 ceph-mon[102832]: pgmap v7039: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T02:54:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:54:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:23 smithi082 ceph-mon[98694]: pgmap v7039: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T02:54:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:54:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:23 smithi098 ceph-mon[103124]: pgmap v7039: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T02:54:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:54:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12895.205004 secs 2023-09-10T02:54:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:54:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:54:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:54:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12895.205004 secs 2023-09-10T02:54:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:54:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:54:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:54:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12895.205004 secs 2023-09-10T02:54:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:54:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:54:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:54:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12895.916289 secs 2023-09-10T02:54:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:25 smithi170 ceph-mon[102832]: pgmap v7040: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T02:54:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12895.916289 secs 2023-09-10T02:54:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:25 smithi082 ceph-mon[98694]: pgmap v7040: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T02:54:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12895.916289 secs 2023-09-10T02:54:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:25 smithi098 ceph-mon[103124]: pgmap v7040: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T02:54:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:27 smithi170 ceph-mon[102832]: pgmap v7041: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 135 KiB/s wr, 12 op/s 2023-09-10T02:54:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:27 smithi082 ceph-mon[98694]: pgmap v7041: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 135 KiB/s wr, 12 op/s 2023-09-10T02:54:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:27 smithi098 ceph-mon[103124]: pgmap v7041: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 135 KiB/s wr, 12 op/s 2023-09-10T02:54:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12900.205064 secs 2023-09-10T02:54:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:29 smithi170 ceph-mon[102832]: pgmap v7042: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T02:54:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12900.205064 secs 2023-09-10T02:54:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:29 smithi082 ceph-mon[98694]: pgmap v7042: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T02:54:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12900.205064 secs 2023-09-10T02:54:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:29 smithi098 ceph-mon[103124]: pgmap v7042: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T02:54:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12900.916354 secs 2023-09-10T02:54:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12900.916354 secs 2023-09-10T02:54:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12900.916354 secs 2023-09-10T02:54:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:31 smithi170 ceph-mon[102832]: pgmap v7043: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 112 KiB/s wr, 10 op/s 2023-09-10T02:54:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:31 smithi082 ceph-mon[98694]: pgmap v7043: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 112 KiB/s wr, 10 op/s 2023-09-10T02:54:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:31 smithi098 ceph-mon[103124]: pgmap v7043: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 112 KiB/s wr, 10 op/s 2023-09-10T02:54:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:33 smithi170 ceph-mon[102832]: pgmap v7044: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 280 KiB/s wr, 17 op/s 2023-09-10T02:54:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12903.840689 secs 2023-09-10T02:54:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:33 smithi082 ceph-mon[98694]: pgmap v7044: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 280 KiB/s wr, 17 op/s 2023-09-10T02:54:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12903.840689 secs 2023-09-10T02:54:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:33 smithi098 ceph-mon[103124]: pgmap v7044: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 280 KiB/s wr, 17 op/s 2023-09-10T02:54:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12903.840689 secs 2023-09-10T02:54:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12905.205122 secs 2023-09-10T02:54:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12905.205122 secs 2023-09-10T02:54:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12905.205122 secs 2023-09-10T02:54:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12905.916418 secs 2023-09-10T02:54:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:35 smithi170 ceph-mon[102832]: pgmap v7045: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 241 KiB/s wr, 11 op/s 2023-09-10T02:54:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12905.916418 secs 2023-09-10T02:54:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:35 smithi082 ceph-mon[98694]: pgmap v7045: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 241 KiB/s wr, 11 op/s 2023-09-10T02:54:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12905.916418 secs 2023-09-10T02:54:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:35 smithi098 ceph-mon[103124]: pgmap v7045: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 241 KiB/s wr, 11 op/s 2023-09-10T02:54:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:36 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:54:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:54:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:54:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:36 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:54:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:54:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:54:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:36 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:54:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:54:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:54:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:37 smithi170 ceph-mon[102832]: pgmap v7046: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 255 KiB/s wr, 16 op/s 2023-09-10T02:54:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:37 smithi082 ceph-mon[98694]: pgmap v7046: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 255 KiB/s wr, 16 op/s 2023-09-10T02:54:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:37 smithi098 ceph-mon[103124]: pgmap v7046: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 255 KiB/s wr, 16 op/s 2023-09-10T02:54:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12908.840744 secs 2023-09-10T02:54:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12908.840744 secs 2023-09-10T02:54:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12908.840744 secs 2023-09-10T02:54:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12910.205181 secs 2023-09-10T02:54:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:39 smithi170 ceph-mon[102832]: pgmap v7047: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 181 KiB/s wr, 11 op/s 2023-09-10T02:54:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12910.205181 secs 2023-09-10T02:54:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:39 smithi082 ceph-mon[98694]: pgmap v7047: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 181 KiB/s wr, 11 op/s 2023-09-10T02:54:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12910.205181 secs 2023-09-10T02:54:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:39 smithi098 ceph-mon[103124]: pgmap v7047: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 181 KiB/s wr, 11 op/s 2023-09-10T02:54:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12910.916488 secs 2023-09-10T02:54:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12910.916488 secs 2023-09-10T02:54:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12910.916488 secs 2023-09-10T02:54:41.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:41 smithi170 ceph-mon[102832]: pgmap v7048: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 181 KiB/s wr, 11 op/s 2023-09-10T02:54:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:41 smithi082 ceph-mon[98694]: pgmap v7048: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 181 KiB/s wr, 11 op/s 2023-09-10T02:54:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:41 smithi098 ceph-mon[103124]: pgmap v7048: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 181 KiB/s wr, 11 op/s 2023-09-10T02:54:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:43 smithi170 ceph-mon[102832]: pgmap v7049: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 363 KiB/s wr, 18 op/s 2023-09-10T02:54:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:54:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:43 smithi082 ceph-mon[98694]: pgmap v7049: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 363 KiB/s wr, 18 op/s 2023-09-10T02:54:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:54:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:43 smithi098 ceph-mon[103124]: pgmap v7049: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 363 KiB/s wr, 18 op/s 2023-09-10T02:54:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:54:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12915.205262 secs 2023-09-10T02:54:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:44 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:54:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:54:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:54:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12915.205262 secs 2023-09-10T02:54:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:44 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:54:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:54:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:54:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12915.205262 secs 2023-09-10T02:54:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:44 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:54:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:54:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:54:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12915.916553 secs 2023-09-10T02:54:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:45 smithi170 ceph-mon[102832]: pgmap v7050: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 195 KiB/s wr, 11 op/s 2023-09-10T02:54:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12915.916553 secs 2023-09-10T02:54:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:45 smithi082 ceph-mon[98694]: pgmap v7050: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 195 KiB/s wr, 11 op/s 2023-09-10T02:54:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12915.916553 secs 2023-09-10T02:54:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:45 smithi098 ceph-mon[103124]: pgmap v7050: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 195 KiB/s wr, 11 op/s 2023-09-10T02:54:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:47 smithi170 ceph-mon[102832]: pgmap v7051: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 325 KiB/s wr, 17 op/s 2023-09-10T02:54:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:47 smithi082 ceph-mon[98694]: pgmap v7051: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 325 KiB/s wr, 17 op/s 2023-09-10T02:54:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:47 smithi098 ceph-mon[103124]: pgmap v7051: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 325 KiB/s wr, 17 op/s 2023-09-10T02:54:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12918.840862 secs 2023-09-10T02:54:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12918.840862 secs 2023-09-10T02:54:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12918.840862 secs 2023-09-10T02:54:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12920.205321 secs 2023-09-10T02:54:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:49 smithi170 ceph-mon[102832]: pgmap v7052: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 311 KiB/s wr, 12 op/s 2023-09-10T02:54:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12920.205321 secs 2023-09-10T02:54:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:49 smithi082 ceph-mon[98694]: pgmap v7052: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 311 KiB/s wr, 12 op/s 2023-09-10T02:54:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12920.205321 secs 2023-09-10T02:54:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:49 smithi098 ceph-mon[103124]: pgmap v7052: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 311 KiB/s wr, 12 op/s 2023-09-10T02:54:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12920.916618 secs 2023-09-10T02:54:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12920.916618 secs 2023-09-10T02:54:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12920.916618 secs 2023-09-10T02:54:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:51 smithi170 ceph-mon[102832]: pgmap v7053: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 311 KiB/s wr, 12 op/s 2023-09-10T02:54:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:51 smithi082 ceph-mon[98694]: pgmap v7053: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 311 KiB/s wr, 12 op/s 2023-09-10T02:54:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:51 smithi098 ceph-mon[103124]: pgmap v7053: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 311 KiB/s wr, 12 op/s 2023-09-10T02:54:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:54:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:54:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:54:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:54:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:54:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:54:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:54:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:54:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:54:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:53 smithi170 ceph-mon[102832]: pgmap v7054: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 392 KiB/s wr, 17 op/s 2023-09-10T02:54:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12923.840924 secs 2023-09-10T02:54:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:53 smithi082 ceph-mon[98694]: pgmap v7054: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 392 KiB/s wr, 17 op/s 2023-09-10T02:54:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12923.840924 secs 2023-09-10T02:54:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:53 smithi098 ceph-mon[103124]: pgmap v7054: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 392 KiB/s wr, 17 op/s 2023-09-10T02:54:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12923.840924 secs 2023-09-10T02:54:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12925.205376 secs 2023-09-10T02:54:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12925.205376 secs 2023-09-10T02:54:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12925.205376 secs 2023-09-10T02:54:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12925.916682 secs 2023-09-10T02:54:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:55 smithi170 ceph-mon[102832]: pgmap v7055: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 211 KiB/s wr, 11 op/s 2023-09-10T02:54:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12925.916682 secs 2023-09-10T02:54:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:55 smithi082 ceph-mon[98694]: pgmap v7055: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 211 KiB/s wr, 11 op/s 2023-09-10T02:54:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12925.916682 secs 2023-09-10T02:54:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:55 smithi098 ceph-mon[103124]: pgmap v7055: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 211 KiB/s wr, 11 op/s 2023-09-10T02:54:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:57 smithi170 ceph-mon[102832]: pgmap v7056: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 211 KiB/s wr, 15 op/s 2023-09-10T02:54:57.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:57 smithi082 ceph-mon[98694]: pgmap v7056: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 211 KiB/s wr, 15 op/s 2023-09-10T02:54:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:57 smithi098 ceph-mon[103124]: pgmap v7056: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 211 KiB/s wr, 15 op/s 2023-09-10T02:54:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12928.840982 secs 2023-09-10T02:54:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12928.840982 secs 2023-09-10T02:54:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12928.840982 secs 2023-09-10T02:54:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12930.205458 secs 2023-09-10T02:54:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:54:59 smithi170 ceph-mon[102832]: pgmap v7057: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 9 op/s 2023-09-10T02:54:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12930.205458 secs 2023-09-10T02:54:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:54:59 smithi082 ceph-mon[98694]: pgmap v7057: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 9 op/s 2023-09-10T02:54:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12930.205458 secs 2023-09-10T02:54:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:54:59 smithi098 ceph-mon[103124]: pgmap v7057: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 9 op/s 2023-09-10T02:55:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12930.916747 secs 2023-09-10T02:55:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12930.916747 secs 2023-09-10T02:55:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12930.916747 secs 2023-09-10T02:55:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:01 smithi170 ceph-mon[102832]: pgmap v7058: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 9 op/s 2023-09-10T02:55:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:01 smithi082 ceph-mon[98694]: pgmap v7058: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 9 op/s 2023-09-10T02:55:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:01 smithi098 ceph-mon[103124]: pgmap v7058: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 9 op/s 2023-09-10T02:55:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:03 smithi170 ceph-mon[102832]: pgmap v7059: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 14 op/s 2023-09-10T02:55:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12933.841044 secs 2023-09-10T02:55:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:03 smithi082 ceph-mon[98694]: pgmap v7059: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 14 op/s 2023-09-10T02:55:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12933.841044 secs 2023-09-10T02:55:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:03 smithi098 ceph-mon[103124]: pgmap v7059: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 14 op/s 2023-09-10T02:55:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12933.841044 secs 2023-09-10T02:55:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12935.205540 secs 2023-09-10T02:55:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12935.205540 secs 2023-09-10T02:55:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12935.205540 secs 2023-09-10T02:55:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12935.916817 secs 2023-09-10T02:55:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:05 smithi170 ceph-mon[102832]: pgmap v7060: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 36 KiB/s wr, 9 op/s 2023-09-10T02:55:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12935.916817 secs 2023-09-10T02:55:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:05 smithi082 ceph-mon[98694]: pgmap v7060: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 36 KiB/s wr, 9 op/s 2023-09-10T02:55:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12935.916817 secs 2023-09-10T02:55:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:05 smithi098 ceph-mon[103124]: pgmap v7060: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 36 KiB/s wr, 9 op/s 2023-09-10T02:55:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:07 smithi170 ceph-mon[102832]: pgmap v7061: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T02:55:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:07 smithi082 ceph-mon[98694]: pgmap v7061: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T02:55:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:07 smithi098 ceph-mon[103124]: pgmap v7061: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 91 KiB/s wr, 29 op/s 2023-09-10T02:55:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12938.841103 secs 2023-09-10T02:55:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:08 smithi170 ceph-mon[102832]: osdmap e509: 12 total, 12 up, 12 in 2023-09-10T02:55:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12938.841103 secs 2023-09-10T02:55:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:08 smithi082 ceph-mon[98694]: osdmap e509: 12 total, 12 up, 12 in 2023-09-10T02:55:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12938.841103 secs 2023-09-10T02:55:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:08 smithi098 ceph-mon[103124]: osdmap e509: 12 total, 12 up, 12 in 2023-09-10T02:55:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12940.205626 secs 2023-09-10T02:55:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:09 smithi170 ceph-mon[102832]: pgmap v7063: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 110 KiB/s wr, 29 op/s 2023-09-10T02:55:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12940.205626 secs 2023-09-10T02:55:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:09 smithi082 ceph-mon[98694]: pgmap v7063: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 110 KiB/s wr, 29 op/s 2023-09-10T02:55:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12940.205626 secs 2023-09-10T02:55:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:09 smithi098 ceph-mon[103124]: pgmap v7063: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 110 KiB/s wr, 29 op/s 2023-09-10T02:55:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12940.916882 secs 2023-09-10T02:55:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12940.916882 secs 2023-09-10T02:55:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12940.916882 secs 2023-09-10T02:55:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:11 smithi170 ceph-mon[102832]: pgmap v7064: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 110 KiB/s wr, 29 op/s 2023-09-10T02:55:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:11 smithi082 ceph-mon[98694]: pgmap v7064: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 110 KiB/s wr, 29 op/s 2023-09-10T02:55:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:11 smithi098 ceph-mon[103124]: pgmap v7064: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 110 KiB/s wr, 29 op/s 2023-09-10T02:55:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:13 smithi170 ceph-mon[102832]: pgmap v7065: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 144 KiB/s wr, 29 op/s 2023-09-10T02:55:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12943.841167 secs 2023-09-10T02:55:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:13 smithi082 ceph-mon[98694]: pgmap v7065: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 144 KiB/s wr, 29 op/s 2023-09-10T02:55:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12943.841167 secs 2023-09-10T02:55:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:13 smithi098 ceph-mon[103124]: pgmap v7065: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 144 KiB/s wr, 29 op/s 2023-09-10T02:55:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12943.841167 secs 2023-09-10T02:55:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12945.205711 secs 2023-09-10T02:55:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12945.205711 secs 2023-09-10T02:55:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12945.205711 secs 2023-09-10T02:55:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12945.916947 secs 2023-09-10T02:55:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:15 smithi170 ceph-mon[102832]: pgmap v7066: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 144 KiB/s wr, 29 op/s 2023-09-10T02:55:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:55:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12945.916947 secs 2023-09-10T02:55:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:15 smithi082 ceph-mon[98694]: pgmap v7066: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 144 KiB/s wr, 29 op/s 2023-09-10T02:55:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:55:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12945.916947 secs 2023-09-10T02:55:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:15 smithi098 ceph-mon[103124]: pgmap v7066: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 144 KiB/s wr, 29 op/s 2023-09-10T02:55:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:55:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:55:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:55:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:55:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:55:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:55:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:55:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:55:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:55:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:55:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:17 smithi170 ceph-mon[102832]: pgmap v7067: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T02:55:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:17 smithi170 ceph-mon[102832]: osdmap e510: 12 total, 12 up, 12 in 2023-09-10T02:55:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:17 smithi082 ceph-mon[98694]: pgmap v7067: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T02:55:17.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:17 smithi082 ceph-mon[98694]: osdmap e510: 12 total, 12 up, 12 in 2023-09-10T02:55:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:17 smithi098 ceph-mon[103124]: pgmap v7067: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T02:55:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:17 smithi098 ceph-mon[103124]: osdmap e510: 12 total, 12 up, 12 in 2023-09-10T02:55:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12948.841225 secs 2023-09-10T02:55:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12950.205768 secs 2023-09-10T02:55:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12948.841225 secs 2023-09-10T02:55:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12950.205768 secs 2023-09-10T02:55:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12948.841225 secs 2023-09-10T02:55:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12950.205768 secs 2023-09-10T02:55:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:19 smithi170 ceph-mon[102832]: pgmap v7069: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T02:55:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12950.917012 secs 2023-09-10T02:55:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:19 smithi082 ceph-mon[98694]: pgmap v7069: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T02:55:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12950.917012 secs 2023-09-10T02:55:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:19 smithi098 ceph-mon[103124]: pgmap v7069: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T02:55:20.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12950.917012 secs 2023-09-10T02:55:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:21 smithi170 ceph-mon[102832]: pgmap v7070: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T02:55:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:21 smithi082 ceph-mon[98694]: pgmap v7070: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T02:55:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:21 smithi098 ceph-mon[103124]: pgmap v7070: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T02:55:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:23 smithi170 ceph-mon[102832]: pgmap v7071: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T02:55:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:55:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:23 smithi082 ceph-mon[98694]: pgmap v7071: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T02:55:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:55:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:23 smithi098 ceph-mon[103124]: pgmap v7071: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T02:55:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:55:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12955.205824 secs 2023-09-10T02:55:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:55:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:55:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:55:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12955.205824 secs 2023-09-10T02:55:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:55:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:55:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:55:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12955.205824 secs 2023-09-10T02:55:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:55:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:55:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:55:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12955.917079 secs 2023-09-10T02:55:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:25 smithi170 ceph-mon[102832]: pgmap v7072: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T02:55:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12955.917079 secs 2023-09-10T02:55:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:25 smithi082 ceph-mon[98694]: pgmap v7072: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T02:55:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12955.917079 secs 2023-09-10T02:55:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:25 smithi098 ceph-mon[103124]: pgmap v7072: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 47 KiB/s wr, 11 op/s 2023-09-10T02:55:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:27 smithi170 ceph-mon[102832]: pgmap v7073: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T02:55:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:27 smithi082 ceph-mon[98694]: pgmap v7073: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T02:55:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:27 smithi098 ceph-mon[103124]: pgmap v7073: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T02:55:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12960.205884 secs 2023-09-10T02:55:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:29 smithi170 ceph-mon[102832]: pgmap v7074: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T02:55:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12960.205884 secs 2023-09-10T02:55:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:29 smithi082 ceph-mon[98694]: pgmap v7074: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T02:55:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12960.205884 secs 2023-09-10T02:55:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:29 smithi098 ceph-mon[103124]: pgmap v7074: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T02:55:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12960.917144 secs 2023-09-10T02:55:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12960.917144 secs 2023-09-10T02:55:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12960.917144 secs 2023-09-10T02:55:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:31 smithi170 ceph-mon[102832]: pgmap v7075: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T02:55:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:31 smithi082 ceph-mon[98694]: pgmap v7075: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T02:55:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:31 smithi098 ceph-mon[103124]: pgmap v7075: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T02:55:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:33 smithi170 ceph-mon[102832]: pgmap v7076: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 138 KiB/s wr, 15 op/s 2023-09-10T02:55:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:33 smithi082 ceph-mon[98694]: pgmap v7076: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 138 KiB/s wr, 15 op/s 2023-09-10T02:55:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:33 smithi098 ceph-mon[103124]: pgmap v7076: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 138 KiB/s wr, 15 op/s 2023-09-10T02:55:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12965.205940 secs 2023-09-10T02:55:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12965.205940 secs 2023-09-10T02:55:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12965.205940 secs 2023-09-10T02:55:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12965.917208 secs 2023-09-10T02:55:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:35 smithi170 ceph-mon[102832]: pgmap v7077: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T02:55:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12965.917208 secs 2023-09-10T02:55:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:35 smithi082 ceph-mon[98694]: pgmap v7077: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T02:55:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12965.917208 secs 2023-09-10T02:55:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:35 smithi098 ceph-mon[103124]: pgmap v7077: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T02:55:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:37 smithi170 ceph-mon[102832]: pgmap v7078: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 129 KiB/s wr, 17 op/s 2023-09-10T02:55:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:37 smithi082 ceph-mon[98694]: pgmap v7078: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 129 KiB/s wr, 17 op/s 2023-09-10T02:55:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:37 smithi098 ceph-mon[103124]: pgmap v7078: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 129 KiB/s wr, 17 op/s 2023-09-10T02:55:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12968.841549 secs 2023-09-10T02:55:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12968.841549 secs 2023-09-10T02:55:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12968.841549 secs 2023-09-10T02:55:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12970.206020 secs 2023-09-10T02:55:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:39 smithi082 ceph-mon[98694]: pgmap v7079: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T02:55:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12970.206020 secs 2023-09-10T02:55:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:39 smithi098 ceph-mon[103124]: pgmap v7079: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T02:55:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12970.206020 secs 2023-09-10T02:55:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:39 smithi170 ceph-mon[102832]: pgmap v7079: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T02:55:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12970.917275 secs 2023-09-10T02:55:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:55:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:55:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:55:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12970.917275 secs 2023-09-10T02:55:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:55:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:55:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:55:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12970.917275 secs 2023-09-10T02:55:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:55:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:55:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:55:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:41 smithi082 ceph-mon[98694]: pgmap v7080: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T02:55:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:41 smithi098 ceph-mon[103124]: pgmap v7080: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T02:55:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:41 smithi170 ceph-mon[102832]: pgmap v7080: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T02:55:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:43 smithi170 ceph-mon[102832]: pgmap v7081: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 265 KiB/s wr, 19 op/s 2023-09-10T02:55:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:55:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:43 smithi082 ceph-mon[98694]: pgmap v7081: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 265 KiB/s wr, 19 op/s 2023-09-10T02:55:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:55:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:43 smithi098 ceph-mon[103124]: pgmap v7081: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 265 KiB/s wr, 19 op/s 2023-09-10T02:55:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:55:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12975.206076 secs 2023-09-10T02:55:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:55:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:55:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12975.206076 secs 2023-09-10T02:55:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:55:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:55:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12975.206076 secs 2023-09-10T02:55:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:55:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:55:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12975.917341 secs 2023-09-10T02:55:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:45 smithi170 ceph-mon[102832]: pgmap v7082: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 13 op/s 2023-09-10T02:55:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12975.917341 secs 2023-09-10T02:55:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:45 smithi082 ceph-mon[98694]: pgmap v7082: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 13 op/s 2023-09-10T02:55:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12975.917341 secs 2023-09-10T02:55:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:45 smithi098 ceph-mon[103124]: pgmap v7082: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 13 op/s 2023-09-10T02:55:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:47 smithi170 ceph-mon[102832]: pgmap v7083: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 203 KiB/s wr, 18 op/s 2023-09-10T02:55:47.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:47 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:55:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:47 smithi082 ceph-mon[98694]: pgmap v7083: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 203 KiB/s wr, 18 op/s 2023-09-10T02:55:47.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:47 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:55:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:47 smithi098 ceph-mon[103124]: pgmap v7083: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 203 KiB/s wr, 18 op/s 2023-09-10T02:55:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:47 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:55:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12980.206158 secs 2023-09-10T02:55:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:49 smithi170 ceph-mon[102832]: pgmap v7084: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T02:55:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12980.206158 secs 2023-09-10T02:55:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:49 smithi082 ceph-mon[98694]: pgmap v7084: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T02:55:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12980.206158 secs 2023-09-10T02:55:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:49 smithi098 ceph-mon[103124]: pgmap v7084: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T02:55:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12980.917408 secs 2023-09-10T02:55:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12980.917408 secs 2023-09-10T02:55:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12980.917408 secs 2023-09-10T02:55:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:51 smithi170 ceph-mon[102832]: pgmap v7085: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T02:55:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:51 smithi082 ceph-mon[98694]: pgmap v7085: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T02:55:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:51 smithi098 ceph-mon[103124]: pgmap v7085: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T02:55:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:53 smithi170 ceph-mon[102832]: pgmap v7086: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 403 KiB/s wr, 18 op/s 2023-09-10T02:55:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12983.841860 secs 2023-09-10T02:55:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:53 smithi082 ceph-mon[98694]: pgmap v7086: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 403 KiB/s wr, 18 op/s 2023-09-10T02:55:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12983.841860 secs 2023-09-10T02:55:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:53 smithi098 ceph-mon[103124]: pgmap v7086: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 403 KiB/s wr, 18 op/s 2023-09-10T02:55:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12983.841860 secs 2023-09-10T02:55:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12985.206244 secs 2023-09-10T02:55:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12985.206244 secs 2023-09-10T02:55:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12985.206244 secs 2023-09-10T02:55:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12985.917480 secs 2023-09-10T02:55:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:55 smithi170 ceph-mon[102832]: pgmap v7087: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 247 KiB/s wr, 11 op/s 2023-09-10T02:55:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12985.917480 secs 2023-09-10T02:55:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:55 smithi082 ceph-mon[98694]: pgmap v7087: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 247 KiB/s wr, 11 op/s 2023-09-10T02:55:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12985.917480 secs 2023-09-10T02:55:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:55 smithi098 ceph-mon[103124]: pgmap v7087: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 247 KiB/s wr, 11 op/s 2023-09-10T02:55:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:55:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:55:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:55:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:55:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:55:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:55:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:55:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:55:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:55:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:57 smithi170 ceph-mon[102832]: pgmap v7088: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 248 KiB/s wr, 17 op/s 2023-09-10T02:55:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:57 smithi082 ceph-mon[98694]: pgmap v7088: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 248 KiB/s wr, 17 op/s 2023-09-10T02:55:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:57 smithi098 ceph-mon[103124]: pgmap v7088: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 248 KiB/s wr, 17 op/s 2023-09-10T02:55:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12988.841918 secs 2023-09-10T02:55:58.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12988.841918 secs 2023-09-10T02:55:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12988.841918 secs 2023-09-10T02:55:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12990.206302 secs 2023-09-10T02:55:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:55:59 smithi082 ceph-mon[98694]: pgmap v7089: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 12 op/s 2023-09-10T02:55:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12990.206302 secs 2023-09-10T02:55:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:55:59 smithi098 ceph-mon[103124]: pgmap v7089: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 12 op/s 2023-09-10T02:55:59.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12990.206302 secs 2023-09-10T02:55:59.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:55:59 smithi170 ceph-mon[102832]: pgmap v7089: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 12 op/s 2023-09-10T02:56:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12990.917545 secs 2023-09-10T02:56:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12990.917545 secs 2023-09-10T02:56:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12990.917545 secs 2023-09-10T02:56:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:01 smithi082 ceph-mon[98694]: pgmap v7090: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 12 op/s 2023-09-10T02:56:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:01 smithi098 ceph-mon[103124]: pgmap v7090: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 12 op/s 2023-09-10T02:56:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:01 smithi170 ceph-mon[102832]: pgmap v7090: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 12 op/s 2023-09-10T02:56:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:03 smithi170 ceph-mon[102832]: pgmap v7091: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 366 KiB/s wr, 32 op/s 2023-09-10T02:56:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12993.841978 secs 2023-09-10T02:56:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:03 smithi082 ceph-mon[98694]: pgmap v7091: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 366 KiB/s wr, 32 op/s 2023-09-10T02:56:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12993.841978 secs 2023-09-10T02:56:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:03 smithi098 ceph-mon[103124]: pgmap v7091: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 366 KiB/s wr, 32 op/s 2023-09-10T02:56:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12993.841978 secs 2023-09-10T02:56:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12995.206359 secs 2023-09-10T02:56:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12995.206359 secs 2023-09-10T02:56:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12995.206359 secs 2023-09-10T02:56:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 12995.917611 secs 2023-09-10T02:56:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:05 smithi170 ceph-mon[102832]: pgmap v7092: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 137 KiB/s wr, 25 op/s 2023-09-10T02:56:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 12995.917611 secs 2023-09-10T02:56:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:05 smithi082 ceph-mon[98694]: pgmap v7092: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 137 KiB/s wr, 25 op/s 2023-09-10T02:56:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 12995.917611 secs 2023-09-10T02:56:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:05 smithi098 ceph-mon[103124]: pgmap v7092: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 137 KiB/s wr, 25 op/s 2023-09-10T02:56:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:07 smithi170 ceph-mon[102832]: pgmap v7093: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 137 KiB/s wr, 30 op/s 2023-09-10T02:56:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:07 smithi082 ceph-mon[98694]: pgmap v7093: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 137 KiB/s wr, 30 op/s 2023-09-10T02:56:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:07 smithi098 ceph-mon[103124]: pgmap v7093: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 137 KiB/s wr, 30 op/s 2023-09-10T02:56:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 12998.842038 secs 2023-09-10T02:56:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:08 smithi170 ceph-mon[102832]: osdmap e511: 12 total, 12 up, 12 in 2023-09-10T02:56:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 12998.842038 secs 2023-09-10T02:56:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:08 smithi082 ceph-mon[98694]: osdmap e511: 12 total, 12 up, 12 in 2023-09-10T02:56:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 12998.842038 secs 2023-09-10T02:56:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:08 smithi098 ceph-mon[103124]: osdmap e511: 12 total, 12 up, 12 in 2023-09-10T02:56:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13000.206436 secs 2023-09-10T02:56:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:09 smithi170 ceph-mon[102832]: pgmap v7095: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 164 KiB/s wr, 29 op/s 2023-09-10T02:56:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13000.206436 secs 2023-09-10T02:56:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:09 smithi082 ceph-mon[98694]: pgmap v7095: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 164 KiB/s wr, 29 op/s 2023-09-10T02:56:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13000.206436 secs 2023-09-10T02:56:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:09 smithi098 ceph-mon[103124]: pgmap v7095: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 164 KiB/s wr, 29 op/s 2023-09-10T02:56:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13000.917676 secs 2023-09-10T02:56:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13000.917676 secs 2023-09-10T02:56:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13000.917676 secs 2023-09-10T02:56:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:11 smithi082 ceph-mon[98694]: pgmap v7096: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 164 KiB/s wr, 29 op/s 2023-09-10T02:56:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:11 smithi098 ceph-mon[103124]: pgmap v7096: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 164 KiB/s wr, 29 op/s 2023-09-10T02:56:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:11 smithi170 ceph-mon[102832]: pgmap v7096: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 164 KiB/s wr, 29 op/s 2023-09-10T02:56:13.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:13 smithi170 ceph-mon[102832]: pgmap v7097: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T02:56:13.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13003.842147 secs 2023-09-10T02:56:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:13 smithi082 ceph-mon[98694]: pgmap v7097: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T02:56:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13003.842147 secs 2023-09-10T02:56:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:13 smithi098 ceph-mon[103124]: pgmap v7097: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T02:56:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13003.842147 secs 2023-09-10T02:56:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13005.206518 secs 2023-09-10T02:56:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13005.206518 secs 2023-09-10T02:56:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13005.206518 secs 2023-09-10T02:56:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13005.917741 secs 2023-09-10T02:56:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:15 smithi170 ceph-mon[102832]: pgmap v7098: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T02:56:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13005.917741 secs 2023-09-10T02:56:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:15 smithi082 ceph-mon[98694]: pgmap v7098: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T02:56:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13005.917741 secs 2023-09-10T02:56:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:15 smithi098 ceph-mon[103124]: pgmap v7098: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 10 op/s 2023-09-10T02:56:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:56:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:56:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:56:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:17 smithi170 ceph-mon[102832]: pgmap v7099: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:56:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:56:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:56:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:56:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:17 smithi170 ceph-mon[102832]: osdmap e512: 12 total, 12 up, 12 in 2023-09-10T02:56:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:17 smithi082 ceph-mon[98694]: pgmap v7099: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:56:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:56:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:56:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:56:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:17 smithi082 ceph-mon[98694]: osdmap e512: 12 total, 12 up, 12 in 2023-09-10T02:56:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:17 smithi098 ceph-mon[103124]: pgmap v7099: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:56:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:56:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:56:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:56:17.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:17 smithi098 ceph-mon[103124]: osdmap e512: 12 total, 12 up, 12 in 2023-09-10T02:56:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13008.842270 secs 2023-09-10T02:56:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13008.842270 secs 2023-09-10T02:56:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13008.842270 secs 2023-09-10T02:56:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13010.206581 secs 2023-09-10T02:56:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:19 smithi082 ceph-mon[98694]: pgmap v7101: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:56:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13010.206581 secs 2023-09-10T02:56:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:19 smithi098 ceph-mon[103124]: pgmap v7101: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:56:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13010.206581 secs 2023-09-10T02:56:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:19 smithi170 ceph-mon[102832]: pgmap v7101: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:56:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13010.917807 secs 2023-09-10T02:56:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13010.917807 secs 2023-09-10T02:56:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13010.917807 secs 2023-09-10T02:56:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:21 smithi082 ceph-mon[98694]: pgmap v7102: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:56:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:21 smithi098 ceph-mon[103124]: pgmap v7102: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:56:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:21 smithi170 ceph-mon[102832]: pgmap v7102: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:56:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:23 smithi170 ceph-mon[102832]: pgmap v7103: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T02:56:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:56:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:23 smithi082 ceph-mon[98694]: pgmap v7103: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T02:56:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:56:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:23 smithi098 ceph-mon[103124]: pgmap v7103: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T02:56:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:56:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13015.206660 secs 2023-09-10T02:56:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:56:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:56:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:56:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13015.206660 secs 2023-09-10T02:56:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:56:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:56:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:56:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13015.206660 secs 2023-09-10T02:56:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:56:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:56:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:56:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13015.917875 secs 2023-09-10T02:56:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:25 smithi170 ceph-mon[102832]: pgmap v7104: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T02:56:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13015.917875 secs 2023-09-10T02:56:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:25 smithi082 ceph-mon[98694]: pgmap v7104: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T02:56:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13015.917875 secs 2023-09-10T02:56:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:25 smithi098 ceph-mon[103124]: pgmap v7104: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T02:56:27.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:27 smithi170 ceph-mon[102832]: pgmap v7105: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 137 KiB/s wr, 11 op/s 2023-09-10T02:56:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:27 smithi082 ceph-mon[98694]: pgmap v7105: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 137 KiB/s wr, 11 op/s 2023-09-10T02:56:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:27 smithi098 ceph-mon[103124]: pgmap v7105: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 137 KiB/s wr, 11 op/s 2023-09-10T02:56:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13020.206745 secs 2023-09-10T02:56:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:29 smithi170 ceph-mon[102832]: pgmap v7106: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T02:56:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13020.206745 secs 2023-09-10T02:56:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:29 smithi082 ceph-mon[98694]: pgmap v7106: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T02:56:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13020.206745 secs 2023-09-10T02:56:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:29 smithi098 ceph-mon[103124]: pgmap v7106: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T02:56:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13020.917939 secs 2023-09-10T02:56:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13020.917939 secs 2023-09-10T02:56:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13020.917939 secs 2023-09-10T02:56:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:31 smithi170 ceph-mon[102832]: pgmap v7107: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 114 KiB/s wr, 9 op/s 2023-09-10T02:56:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:31 smithi082 ceph-mon[98694]: pgmap v7107: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 114 KiB/s wr, 9 op/s 2023-09-10T02:56:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:31 smithi098 ceph-mon[103124]: pgmap v7107: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 114 KiB/s wr, 9 op/s 2023-09-10T02:56:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:33 smithi170 ceph-mon[102832]: pgmap v7108: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 284 KiB/s wr, 17 op/s 2023-09-10T02:56:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13023.842536 secs 2023-09-10T02:56:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:33 smithi082 ceph-mon[98694]: pgmap v7108: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 284 KiB/s wr, 17 op/s 2023-09-10T02:56:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13023.842536 secs 2023-09-10T02:56:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:33 smithi098 ceph-mon[103124]: pgmap v7108: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 284 KiB/s wr, 17 op/s 2023-09-10T02:56:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13023.842536 secs 2023-09-10T02:56:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13025.206801 secs 2023-09-10T02:56:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13025.206801 secs 2023-09-10T02:56:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13025.206801 secs 2023-09-10T02:56:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13025.918006 secs 2023-09-10T02:56:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:35 smithi170 ceph-mon[102832]: pgmap v7109: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 245 KiB/s wr, 12 op/s 2023-09-10T02:56:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13025.918006 secs 2023-09-10T02:56:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:35 smithi082 ceph-mon[98694]: pgmap v7109: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 245 KiB/s wr, 12 op/s 2023-09-10T02:56:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13025.918006 secs 2023-09-10T02:56:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:35 smithi098 ceph-mon[103124]: pgmap v7109: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 245 KiB/s wr, 12 op/s 2023-09-10T02:56:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:36 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:56:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:56:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:56:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:36 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:56:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:56:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:56:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:36 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:56:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:56:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:56:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:37 smithi082 ceph-mon[98694]: pgmap v7110: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 259 KiB/s wr, 17 op/s 2023-09-10T02:56:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:37 smithi098 ceph-mon[103124]: pgmap v7110: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 259 KiB/s wr, 17 op/s 2023-09-10T02:56:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:37 smithi170 ceph-mon[102832]: pgmap v7110: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 259 KiB/s wr, 17 op/s 2023-09-10T02:56:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13030.206859 secs 2023-09-10T02:56:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:39 smithi082 ceph-mon[98694]: pgmap v7111: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 184 KiB/s wr, 12 op/s 2023-09-10T02:56:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:56:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13030.206859 secs 2023-09-10T02:56:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:39 smithi098 ceph-mon[103124]: pgmap v7111: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 184 KiB/s wr, 12 op/s 2023-09-10T02:56:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:56:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13030.206859 secs 2023-09-10T02:56:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:39 smithi170 ceph-mon[102832]: pgmap v7111: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 184 KiB/s wr, 12 op/s 2023-09-10T02:56:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:56:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13030.918074 secs 2023-09-10T02:56:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:56:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:56:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13030.918074 secs 2023-09-10T02:56:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:56:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:56:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13030.918074 secs 2023-09-10T02:56:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:56:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:56:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:41 smithi082 ceph-mon[98694]: pgmap v7112: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 184 KiB/s wr, 12 op/s 2023-09-10T02:56:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:41 smithi098 ceph-mon[103124]: pgmap v7112: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 184 KiB/s wr, 12 op/s 2023-09-10T02:56:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:41 smithi170 ceph-mon[102832]: pgmap v7112: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 184 KiB/s wr, 12 op/s 2023-09-10T02:56:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:42 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:56:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:42 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:56:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:42 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:56:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:43 smithi082 ceph-mon[98694]: pgmap v7113: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 325 KiB/s wr, 18 op/s 2023-09-10T02:56:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:43 smithi098 ceph-mon[103124]: pgmap v7113: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 325 KiB/s wr, 18 op/s 2023-09-10T02:56:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:43 smithi170 ceph-mon[102832]: pgmap v7113: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 325 KiB/s wr, 18 op/s 2023-09-10T02:56:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13035.206917 secs 2023-09-10T02:56:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13035.918141 secs 2023-09-10T02:56:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13035.206917 secs 2023-09-10T02:56:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13035.918141 secs 2023-09-10T02:56:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13035.206917 secs 2023-09-10T02:56:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13035.918141 secs 2023-09-10T02:56:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:45 smithi098 ceph-mon[103124]: pgmap v7114: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T02:56:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:45 smithi170 ceph-mon[102832]: pgmap v7114: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T02:56:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:45 smithi082 ceph-mon[98694]: pgmap v7114: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T02:56:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:47 smithi098 ceph-mon[103124]: pgmap v7115: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 299 KiB/s wr, 18 op/s 2023-09-10T02:56:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:47 smithi170 ceph-mon[102832]: pgmap v7115: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 299 KiB/s wr, 18 op/s 2023-09-10T02:56:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:47 smithi082 ceph-mon[98694]: pgmap v7115: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 299 KiB/s wr, 18 op/s 2023-09-10T02:56:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13038.842775 secs 2023-09-10T02:56:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13038.842775 secs 2023-09-10T02:56:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13038.842775 secs 2023-09-10T02:56:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13040.207001 secs 2023-09-10T02:56:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:49 smithi098 ceph-mon[103124]: pgmap v7116: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 285 KiB/s wr, 13 op/s 2023-09-10T02:56:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13040.918207 secs 2023-09-10T02:56:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13040.207001 secs 2023-09-10T02:56:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:49 smithi170 ceph-mon[102832]: pgmap v7116: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 285 KiB/s wr, 13 op/s 2023-09-10T02:56:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13040.918207 secs 2023-09-10T02:56:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13040.207001 secs 2023-09-10T02:56:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:49 smithi082 ceph-mon[98694]: pgmap v7116: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 285 KiB/s wr, 13 op/s 2023-09-10T02:56:49.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13040.918207 secs 2023-09-10T02:56:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:51 smithi098 ceph-mon[103124]: pgmap v7117: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 285 KiB/s wr, 13 op/s 2023-09-10T02:56:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:51 smithi170 ceph-mon[102832]: pgmap v7117: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 285 KiB/s wr, 13 op/s 2023-09-10T02:56:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:51 smithi082 ceph-mon[98694]: pgmap v7117: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 285 KiB/s wr, 13 op/s 2023-09-10T02:56:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:56:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:56:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:56:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:56:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:56:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:56:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:56:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:56:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:56:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:53 smithi098 ceph-mon[103124]: pgmap v7118: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 307 KiB/s wr, 18 op/s 2023-09-10T02:56:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13043.842834 secs 2023-09-10T02:56:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:53 smithi170 ceph-mon[102832]: pgmap v7118: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 307 KiB/s wr, 18 op/s 2023-09-10T02:56:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13043.842834 secs 2023-09-10T02:56:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:53 smithi082 ceph-mon[98694]: pgmap v7118: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 307 KiB/s wr, 18 op/s 2023-09-10T02:56:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13043.842834 secs 2023-09-10T02:56:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13045.207056 secs 2023-09-10T02:56:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13045.918269 secs 2023-09-10T02:56:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13045.207056 secs 2023-09-10T02:56:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13045.918269 secs 2023-09-10T02:56:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13045.207056 secs 2023-09-10T02:56:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13045.918269 secs 2023-09-10T02:56:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:55 smithi098 ceph-mon[103124]: pgmap v7119: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 166 KiB/s wr, 11 op/s 2023-09-10T02:56:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:55 smithi170 ceph-mon[102832]: pgmap v7119: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 166 KiB/s wr, 11 op/s 2023-09-10T02:56:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:55 smithi082 ceph-mon[98694]: pgmap v7119: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 166 KiB/s wr, 11 op/s 2023-09-10T02:56:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:57 smithi098 ceph-mon[103124]: pgmap v7120: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 252 KiB/s wr, 17 op/s 2023-09-10T02:56:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:57 smithi170 ceph-mon[102832]: pgmap v7120: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 252 KiB/s wr, 17 op/s 2023-09-10T02:56:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:57 smithi082 ceph-mon[98694]: pgmap v7120: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 252 KiB/s wr, 17 op/s 2023-09-10T02:56:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13048.842896 secs 2023-09-10T02:56:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13048.842896 secs 2023-09-10T02:56:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13048.842896 secs 2023-09-10T02:56:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13050.207115 secs 2023-09-10T02:56:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:59 smithi098 ceph-mon[103124]: pgmap v7121: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 10 op/s 2023-09-10T02:56:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:56:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13050.918334 secs 2023-09-10T02:56:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13050.207115 secs 2023-09-10T02:56:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:59 smithi170 ceph-mon[102832]: pgmap v7121: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 10 op/s 2023-09-10T02:56:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:56:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13050.918334 secs 2023-09-10T02:56:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13050.207115 secs 2023-09-10T02:56:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:59 smithi082 ceph-mon[98694]: pgmap v7121: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 10 op/s 2023-09-10T02:56:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:56:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13050.918334 secs 2023-09-10T02:57:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:01 smithi170 ceph-mon[102832]: pgmap v7122: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 10 op/s 2023-09-10T02:57:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:01 smithi082 ceph-mon[98694]: pgmap v7122: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 10 op/s 2023-09-10T02:57:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:01 smithi098 ceph-mon[103124]: pgmap v7122: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 10 op/s 2023-09-10T02:57:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:03 smithi170 ceph-mon[102832]: pgmap v7123: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 167 KiB/s wr, 24 op/s 2023-09-10T02:57:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13053.842957 secs 2023-09-10T02:57:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:03 smithi082 ceph-mon[98694]: pgmap v7123: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 167 KiB/s wr, 24 op/s 2023-09-10T02:57:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13053.842957 secs 2023-09-10T02:57:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:03 smithi098 ceph-mon[103124]: pgmap v7123: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 167 KiB/s wr, 24 op/s 2023-09-10T02:57:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13053.842957 secs 2023-09-10T02:57:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13055.207170 secs 2023-09-10T02:57:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13055.207170 secs 2023-09-10T02:57:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13055.207170 secs 2023-09-10T02:57:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13055.918419 secs 2023-09-10T02:57:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:05 smithi170 ceph-mon[102832]: pgmap v7124: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 145 KiB/s wr, 18 op/s 2023-09-10T02:57:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13055.918419 secs 2023-09-10T02:57:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:05 smithi082 ceph-mon[98694]: pgmap v7124: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 145 KiB/s wr, 18 op/s 2023-09-10T02:57:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13055.918419 secs 2023-09-10T02:57:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:05 smithi098 ceph-mon[103124]: pgmap v7124: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 145 KiB/s wr, 18 op/s 2023-09-10T02:57:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:07 smithi170 ceph-mon[102832]: pgmap v7125: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 178 KiB/s wr, 30 op/s 2023-09-10T02:57:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:07 smithi082 ceph-mon[98694]: pgmap v7125: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 178 KiB/s wr, 30 op/s 2023-09-10T02:57:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:07 smithi098 ceph-mon[103124]: pgmap v7125: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 178 KiB/s wr, 30 op/s 2023-09-10T02:57:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13058.843057 secs 2023-09-10T02:57:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13058.843057 secs 2023-09-10T02:57:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13058.843057 secs 2023-09-10T02:57:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13060.207230 secs 2023-09-10T02:57:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:09 smithi170 ceph-mon[102832]: pgmap v7126: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 91 KiB/s wr, 25 op/s 2023-09-10T02:57:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:09 smithi170 ceph-mon[102832]: osdmap e513: 12 total, 12 up, 12 in 2023-09-10T02:57:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13060.207230 secs 2023-09-10T02:57:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:09 smithi082 ceph-mon[98694]: pgmap v7126: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 91 KiB/s wr, 25 op/s 2023-09-10T02:57:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:09 smithi082 ceph-mon[98694]: osdmap e513: 12 total, 12 up, 12 in 2023-09-10T02:57:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13060.207230 secs 2023-09-10T02:57:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:09 smithi098 ceph-mon[103124]: pgmap v7126: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 91 KiB/s wr, 25 op/s 2023-09-10T02:57:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:09 smithi098 ceph-mon[103124]: osdmap e513: 12 total, 12 up, 12 in 2023-09-10T02:57:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13060.918489 secs 2023-09-10T02:57:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13060.918489 secs 2023-09-10T02:57:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13060.918489 secs 2023-09-10T02:57:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:11 smithi170 ceph-mon[102832]: pgmap v7128: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 110 KiB/s wr, 30 op/s 2023-09-10T02:57:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:11 smithi082 ceph-mon[98694]: pgmap v7128: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 110 KiB/s wr, 30 op/s 2023-09-10T02:57:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:11 smithi098 ceph-mon[103124]: pgmap v7128: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 110 KiB/s wr, 30 op/s 2023-09-10T02:57:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:13 smithi170 ceph-mon[102832]: pgmap v7129: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 39 KiB/s wr, 19 op/s 2023-09-10T02:57:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13063.843119 secs 2023-09-10T02:57:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:13 smithi082 ceph-mon[98694]: pgmap v7129: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 39 KiB/s wr, 19 op/s 2023-09-10T02:57:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13063.843119 secs 2023-09-10T02:57:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:13 smithi098 ceph-mon[103124]: pgmap v7129: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 39 KiB/s wr, 19 op/s 2023-09-10T02:57:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13063.843119 secs 2023-09-10T02:57:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13065.207290 secs 2023-09-10T02:57:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13065.207290 secs 2023-09-10T02:57:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13065.207290 secs 2023-09-10T02:57:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13065.918555 secs 2023-09-10T02:57:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:15 smithi170 ceph-mon[102832]: pgmap v7130: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 39 KiB/s wr, 19 op/s 2023-09-10T02:57:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13065.918555 secs 2023-09-10T02:57:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:15 smithi082 ceph-mon[98694]: pgmap v7130: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 39 KiB/s wr, 19 op/s 2023-09-10T02:57:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13065.918555 secs 2023-09-10T02:57:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:15 smithi098 ceph-mon[103124]: pgmap v7130: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 39 KiB/s wr, 19 op/s 2023-09-10T02:57:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:17 smithi170 ceph-mon[102832]: pgmap v7131: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T02:57:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:57:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:17 smithi170 ceph-mon[102832]: osdmap e514: 12 total, 12 up, 12 in 2023-09-10T02:57:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:17 smithi082 ceph-mon[98694]: pgmap v7131: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T02:57:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:57:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:17 smithi082 ceph-mon[98694]: osdmap e514: 12 total, 12 up, 12 in 2023-09-10T02:57:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:17 smithi098 ceph-mon[103124]: pgmap v7131: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T02:57:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:57:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:17 smithi098 ceph-mon[103124]: osdmap e514: 12 total, 12 up, 12 in 2023-09-10T02:57:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13068.843237 secs 2023-09-10T02:57:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:57:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:57:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:57:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13068.843237 secs 2023-09-10T02:57:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:57:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:57:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:57:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13068.843237 secs 2023-09-10T02:57:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:57:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:57:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:57:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13070.207373 secs 2023-09-10T02:57:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:19 smithi170 ceph-mon[102832]: pgmap v7133: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 12 op/s 2023-09-10T02:57:19.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13070.918623 secs 2023-09-10T02:57:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13070.207373 secs 2023-09-10T02:57:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:19 smithi082 ceph-mon[98694]: pgmap v7133: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 12 op/s 2023-09-10T02:57:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13070.918623 secs 2023-09-10T02:57:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13070.207373 secs 2023-09-10T02:57:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:19 smithi098 ceph-mon[103124]: pgmap v7133: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 12 op/s 2023-09-10T02:57:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13070.918623 secs 2023-09-10T02:57:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:21 smithi170 ceph-mon[102832]: pgmap v7134: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T02:57:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:21 smithi082 ceph-mon[98694]: pgmap v7134: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T02:57:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:21 smithi098 ceph-mon[103124]: pgmap v7134: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 11 op/s 2023-09-10T02:57:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:23 smithi170 ceph-mon[102832]: pgmap v7135: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T02:57:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:57:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:23 smithi082 ceph-mon[98694]: pgmap v7135: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T02:57:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:57:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:23 smithi098 ceph-mon[103124]: pgmap v7135: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T02:57:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:57:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13075.207428 secs 2023-09-10T02:57:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:57:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:57:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:57:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13075.207428 secs 2023-09-10T02:57:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:57:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:57:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:57:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13075.207428 secs 2023-09-10T02:57:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:57:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:57:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:57:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13075.918689 secs 2023-09-10T02:57:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:25 smithi082 ceph-mon[98694]: pgmap v7136: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T02:57:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13075.918689 secs 2023-09-10T02:57:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:25 smithi098 ceph-mon[103124]: pgmap v7136: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T02:57:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13075.918689 secs 2023-09-10T02:57:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:25 smithi170 ceph-mon[102832]: pgmap v7136: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T02:57:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:27 smithi082 ceph-mon[98694]: pgmap v7137: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T02:57:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:27 smithi098 ceph-mon[103124]: pgmap v7137: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T02:57:27.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:27 smithi170 ceph-mon[102832]: pgmap v7137: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T02:57:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13080.207511 secs 2023-09-10T02:57:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:29 smithi082 ceph-mon[98694]: pgmap v7138: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T02:57:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13080.207511 secs 2023-09-10T02:57:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:29 smithi098 ceph-mon[103124]: pgmap v7138: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T02:57:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13080.207511 secs 2023-09-10T02:57:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:29 smithi170 ceph-mon[102832]: pgmap v7138: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T02:57:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13080.918754 secs 2023-09-10T02:57:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13080.918754 secs 2023-09-10T02:57:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13080.918754 secs 2023-09-10T02:57:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:31 smithi098 ceph-mon[103124]: pgmap v7139: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T02:57:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:31 smithi170 ceph-mon[102832]: pgmap v7139: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T02:57:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:31 smithi082 ceph-mon[98694]: pgmap v7139: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T02:57:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:33 smithi170 ceph-mon[102832]: pgmap v7140: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 140 KiB/s wr, 15 op/s 2023-09-10T02:57:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:33 smithi082 ceph-mon[98694]: pgmap v7140: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 140 KiB/s wr, 15 op/s 2023-09-10T02:57:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:33 smithi098 ceph-mon[103124]: pgmap v7140: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 140 KiB/s wr, 15 op/s 2023-09-10T02:57:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13085.207567 secs 2023-09-10T02:57:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13085.207567 secs 2023-09-10T02:57:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13085.207567 secs 2023-09-10T02:57:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13085.918817 secs 2023-09-10T02:57:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:35 smithi170 ceph-mon[102832]: pgmap v7141: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 104 KiB/s wr, 10 op/s 2023-09-10T02:57:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13085.918817 secs 2023-09-10T02:57:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:35 smithi082 ceph-mon[98694]: pgmap v7141: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 104 KiB/s wr, 10 op/s 2023-09-10T02:57:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13085.918817 secs 2023-09-10T02:57:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:35 smithi098 ceph-mon[103124]: pgmap v7141: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 104 KiB/s wr, 10 op/s 2023-09-10T02:57:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:37 smithi170 ceph-mon[102832]: pgmap v7142: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 173 KiB/s wr, 17 op/s 2023-09-10T02:57:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:37 smithi082 ceph-mon[98694]: pgmap v7142: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 173 KiB/s wr, 17 op/s 2023-09-10T02:57:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:37 smithi098 ceph-mon[103124]: pgmap v7142: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 173 KiB/s wr, 17 op/s 2023-09-10T02:57:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13088.843491 secs 2023-09-10T02:57:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13088.843491 secs 2023-09-10T02:57:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13088.843491 secs 2023-09-10T02:57:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13090.207650 secs 2023-09-10T02:57:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:39 smithi170 ceph-mon[102832]: pgmap v7143: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T02:57:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13090.207650 secs 2023-09-10T02:57:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:39 smithi082 ceph-mon[98694]: pgmap v7143: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T02:57:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13090.207650 secs 2023-09-10T02:57:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:39 smithi098 ceph-mon[103124]: pgmap v7143: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T02:57:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13090.918885 secs 2023-09-10T02:57:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:57:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:57:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:57:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13090.918885 secs 2023-09-10T02:57:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:57:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:57:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:57:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13090.918885 secs 2023-09-10T02:57:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:57:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:57:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:57:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:41 smithi170 ceph-mon[102832]: pgmap v7144: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T02:57:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:41 smithi082 ceph-mon[98694]: pgmap v7144: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T02:57:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:41 smithi098 ceph-mon[103124]: pgmap v7144: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T02:57:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:43 smithi170 ceph-mon[102832]: pgmap v7145: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 268 KiB/s wr, 18 op/s 2023-09-10T02:57:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:57:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:43 smithi082 ceph-mon[98694]: pgmap v7145: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 268 KiB/s wr, 18 op/s 2023-09-10T02:57:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:57:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:43 smithi098 ceph-mon[103124]: pgmap v7145: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 268 KiB/s wr, 18 op/s 2023-09-10T02:57:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:57:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13095.207706 secs 2023-09-10T02:57:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:57:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:57:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13095.207706 secs 2023-09-10T02:57:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:57:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:57:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13095.207706 secs 2023-09-10T02:57:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:57:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:57:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13095.918952 secs 2023-09-10T02:57:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:45 smithi170 ceph-mon[102832]: pgmap v7146: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 187 KiB/s wr, 12 op/s 2023-09-10T02:57:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13095.918952 secs 2023-09-10T02:57:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:45 smithi082 ceph-mon[98694]: pgmap v7146: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 187 KiB/s wr, 12 op/s 2023-09-10T02:57:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13095.918952 secs 2023-09-10T02:57:45.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:45 smithi098 ceph-mon[103124]: pgmap v7146: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 187 KiB/s wr, 12 op/s 2023-09-10T02:57:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:47 smithi170 ceph-mon[102832]: pgmap v7147: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 241 KiB/s wr, 18 op/s 2023-09-10T02:57:47.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:47 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:57:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:47 smithi082 ceph-mon[98694]: pgmap v7147: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 241 KiB/s wr, 18 op/s 2023-09-10T02:57:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:47 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:57:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:47 smithi098 ceph-mon[103124]: pgmap v7147: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 241 KiB/s wr, 18 op/s 2023-09-10T02:57:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:47 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:57:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13100.207774 secs 2023-09-10T02:57:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:49 smithi170 ceph-mon[102832]: pgmap v7148: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 171 KiB/s wr, 11 op/s 2023-09-10T02:57:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13100.207774 secs 2023-09-10T02:57:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:49 smithi082 ceph-mon[98694]: pgmap v7148: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 171 KiB/s wr, 11 op/s 2023-09-10T02:57:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13100.207774 secs 2023-09-10T02:57:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:49 smithi098 ceph-mon[103124]: pgmap v7148: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 171 KiB/s wr, 11 op/s 2023-09-10T02:57:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13100.919019 secs 2023-09-10T02:57:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13100.919019 secs 2023-09-10T02:57:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13100.919019 secs 2023-09-10T02:57:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:51 smithi170 ceph-mon[102832]: pgmap v7149: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 171 KiB/s wr, 11 op/s 2023-09-10T02:57:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:51 smithi082 ceph-mon[98694]: pgmap v7149: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 171 KiB/s wr, 11 op/s 2023-09-10T02:57:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:51 smithi098 ceph-mon[103124]: pgmap v7149: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 171 KiB/s wr, 11 op/s 2023-09-10T02:57:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13103.843671 secs 2023-09-10T02:57:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:53 smithi170 ceph-mon[102832]: pgmap v7150: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 359 KiB/s wr, 18 op/s 2023-09-10T02:57:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13103.843671 secs 2023-09-10T02:57:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:53 smithi082 ceph-mon[98694]: pgmap v7150: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 359 KiB/s wr, 18 op/s 2023-09-10T02:57:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13103.843671 secs 2023-09-10T02:57:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:53 smithi098 ceph-mon[103124]: pgmap v7150: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 359 KiB/s wr, 18 op/s 2023-09-10T02:57:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13105.207831 secs 2023-09-10T02:57:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13105.207831 secs 2023-09-10T02:57:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13105.207831 secs 2023-09-10T02:57:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13105.919084 secs 2023-09-10T02:57:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:55 smithi170 ceph-mon[102832]: pgmap v7151: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 241 KiB/s wr, 11 op/s 2023-09-10T02:57:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13105.919084 secs 2023-09-10T02:57:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:55 smithi082 ceph-mon[98694]: pgmap v7151: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 241 KiB/s wr, 11 op/s 2023-09-10T02:57:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13105.919084 secs 2023-09-10T02:57:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:55 smithi098 ceph-mon[103124]: pgmap v7151: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 241 KiB/s wr, 11 op/s 2023-09-10T02:57:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:57:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:57:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:57:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:57:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:57:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:57:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:57:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:57:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:57:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:57 smithi170 ceph-mon[102832]: pgmap v7152: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 242 KiB/s wr, 17 op/s 2023-09-10T02:57:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:57 smithi082 ceph-mon[98694]: pgmap v7152: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 242 KiB/s wr, 17 op/s 2023-09-10T02:57:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:57 smithi098 ceph-mon[103124]: pgmap v7152: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 242 KiB/s wr, 17 op/s 2023-09-10T02:57:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13108.843733 secs 2023-09-10T02:57:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13108.843733 secs 2023-09-10T02:57:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13108.843733 secs 2023-09-10T02:57:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13110.207888 secs 2023-09-10T02:57:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:57:59 smithi170 ceph-mon[102832]: pgmap v7153: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 189 KiB/s wr, 12 op/s 2023-09-10T02:57:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13110.207888 secs 2023-09-10T02:57:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:57:59 smithi082 ceph-mon[98694]: pgmap v7153: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 189 KiB/s wr, 12 op/s 2023-09-10T02:57:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13110.207888 secs 2023-09-10T02:57:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:57:59 smithi098 ceph-mon[103124]: pgmap v7153: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 189 KiB/s wr, 12 op/s 2023-09-10T02:58:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13110.919176 secs 2023-09-10T02:58:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13110.919176 secs 2023-09-10T02:58:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13110.919176 secs 2023-09-10T02:58:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:01 smithi170 ceph-mon[102832]: pgmap v7154: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 189 KiB/s wr, 12 op/s 2023-09-10T02:58:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:01 smithi082 ceph-mon[98694]: pgmap v7154: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 189 KiB/s wr, 12 op/s 2023-09-10T02:58:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:01 smithi098 ceph-mon[103124]: pgmap v7154: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 189 KiB/s wr, 12 op/s 2023-09-10T02:58:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13113.843779 secs 2023-09-10T02:58:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:03 smithi170 ceph-mon[102832]: pgmap v7155: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 281 KiB/s wr, 17 op/s 2023-09-10T02:58:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13113.843779 secs 2023-09-10T02:58:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:03 smithi082 ceph-mon[98694]: pgmap v7155: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 281 KiB/s wr, 17 op/s 2023-09-10T02:58:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13113.843779 secs 2023-09-10T02:58:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:03 smithi098 ceph-mon[103124]: pgmap v7155: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 281 KiB/s wr, 17 op/s 2023-09-10T02:58:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13115.207944 secs 2023-09-10T02:58:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13115.207944 secs 2023-09-10T02:58:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13115.207944 secs 2023-09-10T02:58:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13115.919239 secs 2023-09-10T02:58:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:05 smithi170 ceph-mon[102832]: pgmap v7156: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T02:58:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13115.919239 secs 2023-09-10T02:58:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:05 smithi082 ceph-mon[98694]: pgmap v7156: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T02:58:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13115.919239 secs 2023-09-10T02:58:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:05 smithi098 ceph-mon[103124]: pgmap v7156: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T02:58:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:07 smithi170 ceph-mon[102832]: pgmap v7157: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 138 KiB/s wr, 30 op/s 2023-09-10T02:58:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:07 smithi082 ceph-mon[98694]: pgmap v7157: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 138 KiB/s wr, 30 op/s 2023-09-10T02:58:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:07 smithi098 ceph-mon[103124]: pgmap v7157: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 138 KiB/s wr, 30 op/s 2023-09-10T02:58:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13118.843845 secs 2023-09-10T02:58:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:08 smithi170 ceph-mon[102832]: osdmap e515: 12 total, 12 up, 12 in 2023-09-10T02:58:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13118.843845 secs 2023-09-10T02:58:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:08 smithi082 ceph-mon[98694]: osdmap e515: 12 total, 12 up, 12 in 2023-09-10T02:58:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13118.843845 secs 2023-09-10T02:58:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:08 smithi098 ceph-mon[103124]: osdmap e515: 12 total, 12 up, 12 in 2023-09-10T02:58:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13120.208003 secs 2023-09-10T02:58:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:09 smithi170 ceph-mon[102832]: pgmap v7159: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 165 KiB/s wr, 29 op/s 2023-09-10T02:58:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13120.208003 secs 2023-09-10T02:58:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:09 smithi082 ceph-mon[98694]: pgmap v7159: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 165 KiB/s wr, 29 op/s 2023-09-10T02:58:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13120.208003 secs 2023-09-10T02:58:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:09 smithi098 ceph-mon[103124]: pgmap v7159: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 165 KiB/s wr, 29 op/s 2023-09-10T02:58:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13120.919308 secs 2023-09-10T02:58:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13120.919308 secs 2023-09-10T02:58:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13120.919308 secs 2023-09-10T02:58:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:11 smithi170 ceph-mon[102832]: pgmap v7160: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 165 KiB/s wr, 29 op/s 2023-09-10T02:58:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:11 smithi082 ceph-mon[98694]: pgmap v7160: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 165 KiB/s wr, 29 op/s 2023-09-10T02:58:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:11 smithi098 ceph-mon[103124]: pgmap v7160: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 165 KiB/s wr, 29 op/s 2023-09-10T02:58:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13123.843908 secs 2023-09-10T02:58:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:13 smithi170 ceph-mon[102832]: pgmap v7161: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 92 KiB/s wr, 28 op/s 2023-09-10T02:58:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13123.843908 secs 2023-09-10T02:58:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:13 smithi082 ceph-mon[98694]: pgmap v7161: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 92 KiB/s wr, 28 op/s 2023-09-10T02:58:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13123.843908 secs 2023-09-10T02:58:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:13 smithi098 ceph-mon[103124]: pgmap v7161: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 92 KiB/s wr, 28 op/s 2023-09-10T02:58:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13125.208061 secs 2023-09-10T02:58:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13125.208061 secs 2023-09-10T02:58:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13125.208061 secs 2023-09-10T02:58:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13125.919372 secs 2023-09-10T02:58:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:15 smithi170 ceph-mon[102832]: pgmap v7162: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 92 KiB/s wr, 28 op/s 2023-09-10T02:58:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13125.919372 secs 2023-09-10T02:58:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:15 smithi082 ceph-mon[98694]: pgmap v7162: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 92 KiB/s wr, 28 op/s 2023-09-10T02:58:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13125.919372 secs 2023-09-10T02:58:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:15 smithi098 ceph-mon[103124]: pgmap v7162: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 92 KiB/s wr, 28 op/s 2023-09-10T02:58:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:17 smithi098 ceph-mon[103124]: pgmap v7163: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:58:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:17 smithi098 ceph-mon[103124]: osdmap e516: 12 total, 12 up, 12 in 2023-09-10T02:58:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:17 smithi170 ceph-mon[102832]: pgmap v7163: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:58:17.709 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:17 smithi170 ceph-mon[102832]: osdmap e516: 12 total, 12 up, 12 in 2023-09-10T02:58:17.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:17 smithi082 ceph-mon[98694]: pgmap v7163: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:58:17.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:17 smithi082 ceph-mon[98694]: osdmap e516: 12 total, 12 up, 12 in 2023-09-10T02:58:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13128.843991 secs 2023-09-10T02:58:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:58:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:58:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:58:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:58:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13128.843991 secs 2023-09-10T02:58:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:58:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:58:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:58:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:58:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13128.843991 secs 2023-09-10T02:58:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:58:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:58:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:58:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:58:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13130.208119 secs 2023-09-10T02:58:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:19 smithi098 ceph-mon[103124]: pgmap v7165: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:58:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13130.919435 secs 2023-09-10T02:58:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13130.208119 secs 2023-09-10T02:58:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:19 smithi170 ceph-mon[102832]: pgmap v7165: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:58:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13130.919435 secs 2023-09-10T02:58:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13130.208119 secs 2023-09-10T02:58:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:19 smithi082 ceph-mon[98694]: pgmap v7165: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:58:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13130.919435 secs 2023-09-10T02:58:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:21 smithi098 ceph-mon[103124]: pgmap v7166: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:58:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:21 smithi170 ceph-mon[102832]: pgmap v7166: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:58:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:21 smithi082 ceph-mon[98694]: pgmap v7166: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 11 op/s 2023-09-10T02:58:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:23 smithi170 ceph-mon[102832]: pgmap v7167: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T02:58:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:58:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:23 smithi082 ceph-mon[98694]: pgmap v7167: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T02:58:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:58:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:23 smithi098 ceph-mon[103124]: pgmap v7167: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T02:58:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:58:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13135.208178 secs 2023-09-10T02:58:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:58:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:58:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:58:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13135.208178 secs 2023-09-10T02:58:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:58:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:58:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:58:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13135.208178 secs 2023-09-10T02:58:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:58:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:58:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:58:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13135.919490 secs 2023-09-10T02:58:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:25 smithi170 ceph-mon[102832]: pgmap v7168: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T02:58:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13135.919490 secs 2023-09-10T02:58:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:25 smithi082 ceph-mon[98694]: pgmap v7168: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T02:58:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13135.919490 secs 2023-09-10T02:58:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:25 smithi098 ceph-mon[103124]: pgmap v7168: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T02:58:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:27 smithi170 ceph-mon[102832]: pgmap v7169: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T02:58:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:27 smithi082 ceph-mon[98694]: pgmap v7169: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T02:58:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:27 smithi098 ceph-mon[103124]: pgmap v7169: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T02:58:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13140.208241 secs 2023-09-10T02:58:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:29 smithi170 ceph-mon[102832]: pgmap v7170: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T02:58:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13140.208241 secs 2023-09-10T02:58:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:29 smithi082 ceph-mon[98694]: pgmap v7170: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T02:58:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13140.208241 secs 2023-09-10T02:58:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:29 smithi098 ceph-mon[103124]: pgmap v7170: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T02:58:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13140.919555 secs 2023-09-10T02:58:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13140.919555 secs 2023-09-10T02:58:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13140.919555 secs 2023-09-10T02:58:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:31 smithi170 ceph-mon[102832]: pgmap v7171: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 9 op/s 2023-09-10T02:58:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:31 smithi082 ceph-mon[98694]: pgmap v7171: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 9 op/s 2023-09-10T02:58:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:31 smithi098 ceph-mon[103124]: pgmap v7171: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 9 op/s 2023-09-10T02:58:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:33 smithi170 ceph-mon[102832]: pgmap v7172: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 288 KiB/s wr, 16 op/s 2023-09-10T02:58:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:33 smithi082 ceph-mon[98694]: pgmap v7172: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 288 KiB/s wr, 16 op/s 2023-09-10T02:58:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:33 smithi098 ceph-mon[103124]: pgmap v7172: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 288 KiB/s wr, 16 op/s 2023-09-10T02:58:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13145.208298 secs 2023-09-10T02:58:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13145.208298 secs 2023-09-10T02:58:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13145.208298 secs 2023-09-10T02:58:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13145.919621 secs 2023-09-10T02:58:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:35 smithi170 ceph-mon[102832]: pgmap v7173: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 248 KiB/s wr, 11 op/s 2023-09-10T02:58:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13145.919621 secs 2023-09-10T02:58:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:35 smithi082 ceph-mon[98694]: pgmap v7173: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 248 KiB/s wr, 11 op/s 2023-09-10T02:58:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13145.919621 secs 2023-09-10T02:58:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:35 smithi098 ceph-mon[103124]: pgmap v7173: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 248 KiB/s wr, 11 op/s 2023-09-10T02:58:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:37 smithi098 ceph-mon[103124]: pgmap v7174: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 341 KiB/s wr, 17 op/s 2023-09-10T02:58:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:37 smithi170 ceph-mon[102832]: pgmap v7174: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 341 KiB/s wr, 17 op/s 2023-09-10T02:58:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:37 smithi082 ceph-mon[98694]: pgmap v7174: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 341 KiB/s wr, 17 op/s 2023-09-10T02:58:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13148.844326 secs 2023-09-10T02:58:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13148.844326 secs 2023-09-10T02:58:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13148.844326 secs 2023-09-10T02:58:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13150.208358 secs 2023-09-10T02:58:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:39 smithi098 ceph-mon[103124]: pgmap v7175: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 301 KiB/s wr, 12 op/s 2023-09-10T02:58:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13150.919685 secs 2023-09-10T02:58:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13150.208358 secs 2023-09-10T02:58:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:39 smithi170 ceph-mon[102832]: pgmap v7175: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 301 KiB/s wr, 12 op/s 2023-09-10T02:58:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13150.919685 secs 2023-09-10T02:58:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13150.208358 secs 2023-09-10T02:58:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:39 smithi082 ceph-mon[98694]: pgmap v7175: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 301 KiB/s wr, 12 op/s 2023-09-10T02:58:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13150.919685 secs 2023-09-10T02:58:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:58:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:58:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:58:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:58:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:58:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:58:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:58:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:58:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:58:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:41 smithi098 ceph-mon[103124]: pgmap v7176: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 301 KiB/s wr, 12 op/s 2023-09-10T02:58:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:41 smithi170 ceph-mon[102832]: pgmap v7176: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 301 KiB/s wr, 12 op/s 2023-09-10T02:58:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:41 smithi082 ceph-mon[98694]: pgmap v7176: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 301 KiB/s wr, 12 op/s 2023-09-10T02:58:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:43 smithi170 ceph-mon[102832]: pgmap v7177: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 451 KiB/s wr, 18 op/s 2023-09-10T02:58:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:58:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:43 smithi082 ceph-mon[98694]: pgmap v7177: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 451 KiB/s wr, 18 op/s 2023-09-10T02:58:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:58:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:43 smithi098 ceph-mon[103124]: pgmap v7177: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 451 KiB/s wr, 18 op/s 2023-09-10T02:58:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:58:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13155.208415 secs 2023-09-10T02:58:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:58:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:58:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13155.208415 secs 2023-09-10T02:58:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:58:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:58:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13155.208415 secs 2023-09-10T02:58:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:58:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:58:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13155.919752 secs 2023-09-10T02:58:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:45 smithi170 ceph-mon[102832]: pgmap v7178: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 244 KiB/s wr, 11 op/s 2023-09-10T02:58:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13155.919752 secs 2023-09-10T02:58:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:45 smithi082 ceph-mon[98694]: pgmap v7178: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 244 KiB/s wr, 11 op/s 2023-09-10T02:58:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13155.919752 secs 2023-09-10T02:58:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:45 smithi098 ceph-mon[103124]: pgmap v7178: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 244 KiB/s wr, 11 op/s 2023-09-10T02:58:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:47 smithi098 ceph-mon[103124]: pgmap v7179: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 304 KiB/s wr, 17 op/s 2023-09-10T02:58:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:47 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:58:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:47 smithi170 ceph-mon[102832]: pgmap v7179: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 304 KiB/s wr, 17 op/s 2023-09-10T02:58:47.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:47 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:58:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:47 smithi082 ceph-mon[98694]: pgmap v7179: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 304 KiB/s wr, 17 op/s 2023-09-10T02:58:47.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:47 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:58:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13158.844463 secs 2023-09-10T02:58:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13158.844463 secs 2023-09-10T02:58:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13158.844463 secs 2023-09-10T02:58:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13160.208473 secs 2023-09-10T02:58:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:49 smithi098 ceph-mon[103124]: pgmap v7180: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T02:58:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13160.919819 secs 2023-09-10T02:58:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13160.208473 secs 2023-09-10T02:58:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:49 smithi170 ceph-mon[102832]: pgmap v7180: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T02:58:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13160.919819 secs 2023-09-10T02:58:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13160.208473 secs 2023-09-10T02:58:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:49 smithi082 ceph-mon[98694]: pgmap v7180: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T02:58:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13160.919819 secs 2023-09-10T02:58:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:51 smithi098 ceph-mon[103124]: pgmap v7181: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T02:58:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:51 smithi170 ceph-mon[102832]: pgmap v7181: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T02:58:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:51 smithi082 ceph-mon[98694]: pgmap v7181: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T02:58:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:58:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:58:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:58:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:58:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:58:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:58:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:58:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:58:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:58:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13163.844542 secs 2023-09-10T02:58:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:53 smithi098 ceph-mon[103124]: pgmap v7182: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 232 KiB/s wr, 16 op/s 2023-09-10T02:58:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13163.844542 secs 2023-09-10T02:58:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:53 smithi170 ceph-mon[102832]: pgmap v7182: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 232 KiB/s wr, 16 op/s 2023-09-10T02:58:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13163.844542 secs 2023-09-10T02:58:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:53 smithi082 ceph-mon[98694]: pgmap v7182: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 232 KiB/s wr, 16 op/s 2023-09-10T02:58:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13165.208499 secs 2023-09-10T02:58:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13165.919883 secs 2023-09-10T02:58:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13165.208499 secs 2023-09-10T02:58:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13165.919883 secs 2023-09-10T02:58:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13165.208499 secs 2023-09-10T02:58:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13165.919883 secs 2023-09-10T02:58:55.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:55 smithi098 ceph-mon[103124]: pgmap v7183: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T02:58:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:55 smithi170 ceph-mon[102832]: pgmap v7183: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T02:58:55.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:55 smithi082 ceph-mon[98694]: pgmap v7183: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T02:58:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:57 smithi098 ceph-mon[103124]: pgmap v7184: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 165 KiB/s wr, 16 op/s 2023-09-10T02:58:57.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:57 smithi170 ceph-mon[102832]: pgmap v7184: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 165 KiB/s wr, 16 op/s 2023-09-10T02:58:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:57 smithi082 ceph-mon[98694]: pgmap v7184: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 165 KiB/s wr, 16 op/s 2023-09-10T02:58:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13168.844601 secs 2023-09-10T02:58:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13168.844601 secs 2023-09-10T02:58:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13168.844601 secs 2023-09-10T02:58:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13170.208517 secs 2023-09-10T02:58:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:59 smithi098 ceph-mon[103124]: pgmap v7185: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T02:58:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:58:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13170.919949 secs 2023-09-10T02:58:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13170.208517 secs 2023-09-10T02:58:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:59 smithi170 ceph-mon[102832]: pgmap v7185: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T02:58:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:58:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13170.919949 secs 2023-09-10T02:58:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13170.208517 secs 2023-09-10T02:58:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:59 smithi082 ceph-mon[98694]: pgmap v7185: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T02:58:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:58:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13170.919949 secs 2023-09-10T02:59:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:01 smithi170 ceph-mon[102832]: pgmap v7186: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T02:59:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:01 smithi082 ceph-mon[98694]: pgmap v7186: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T02:59:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:01 smithi098 ceph-mon[103124]: pgmap v7186: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T02:59:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13173.844685 secs 2023-09-10T02:59:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:03 smithi170 ceph-mon[102832]: pgmap v7187: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 141 KiB/s wr, 15 op/s 2023-09-10T02:59:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13173.844685 secs 2023-09-10T02:59:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:03 smithi082 ceph-mon[98694]: pgmap v7187: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 141 KiB/s wr, 15 op/s 2023-09-10T02:59:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13173.844685 secs 2023-09-10T02:59:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:03 smithi098 ceph-mon[103124]: pgmap v7187: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 141 KiB/s wr, 15 op/s 2023-09-10T02:59:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13175.208576 secs 2023-09-10T02:59:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13175.208576 secs 2023-09-10T02:59:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13175.208576 secs 2023-09-10T02:59:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13175.920014 secs 2023-09-10T02:59:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:05 smithi170 ceph-mon[102832]: pgmap v7188: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T02:59:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13175.920014 secs 2023-09-10T02:59:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:05 smithi082 ceph-mon[98694]: pgmap v7188: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T02:59:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13175.920014 secs 2023-09-10T02:59:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:05 smithi098 ceph-mon[103124]: pgmap v7188: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T02:59:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:07 smithi098 ceph-mon[103124]: pgmap v7189: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 175 KiB/s wr, 30 op/s 2023-09-10T02:59:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:07 smithi170 ceph-mon[102832]: pgmap v7189: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 175 KiB/s wr, 30 op/s 2023-09-10T02:59:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:07 smithi082 ceph-mon[98694]: pgmap v7189: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 175 KiB/s wr, 30 op/s 2023-09-10T02:59:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13178.844814 secs 2023-09-10T02:59:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:08 smithi098 ceph-mon[103124]: osdmap e517: 12 total, 12 up, 12 in 2023-09-10T02:59:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13178.844814 secs 2023-09-10T02:59:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:08 smithi170 ceph-mon[102832]: osdmap e517: 12 total, 12 up, 12 in 2023-09-10T02:59:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13178.844814 secs 2023-09-10T02:59:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:08 smithi082 ceph-mon[98694]: osdmap e517: 12 total, 12 up, 12 in 2023-09-10T02:59:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13180.208632 secs 2023-09-10T02:59:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:09 smithi098 ceph-mon[103124]: pgmap v7191: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 111 KiB/s wr, 30 op/s 2023-09-10T02:59:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13180.920077 secs 2023-09-10T02:59:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13180.208632 secs 2023-09-10T02:59:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:09 smithi170 ceph-mon[102832]: pgmap v7191: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 111 KiB/s wr, 30 op/s 2023-09-10T02:59:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13180.920077 secs 2023-09-10T02:59:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13180.208632 secs 2023-09-10T02:59:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:09 smithi082 ceph-mon[98694]: pgmap v7191: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 111 KiB/s wr, 30 op/s 2023-09-10T02:59:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13180.920077 secs 2023-09-10T02:59:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:11 smithi098 ceph-mon[103124]: pgmap v7192: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 111 KiB/s wr, 30 op/s 2023-09-10T02:59:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:11 smithi170 ceph-mon[102832]: pgmap v7192: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 111 KiB/s wr, 30 op/s 2023-09-10T02:59:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:11 smithi082 ceph-mon[98694]: pgmap v7192: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 111 KiB/s wr, 30 op/s 2023-09-10T02:59:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13183.844938 secs 2023-09-10T02:59:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:13 smithi170 ceph-mon[102832]: pgmap v7193: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T02:59:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13183.844938 secs 2023-09-10T02:59:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:13 smithi082 ceph-mon[98694]: pgmap v7193: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T02:59:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13183.844938 secs 2023-09-10T02:59:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:13 smithi098 ceph-mon[103124]: pgmap v7193: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T02:59:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13185.208692 secs 2023-09-10T02:59:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13185.208692 secs 2023-09-10T02:59:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13185.208692 secs 2023-09-10T02:59:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13185.920147 secs 2023-09-10T02:59:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:15 smithi170 ceph-mon[102832]: pgmap v7194: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T02:59:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13185.920147 secs 2023-09-10T02:59:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:15 smithi082 ceph-mon[98694]: pgmap v7194: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T02:59:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13185.920147 secs 2023-09-10T02:59:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:15 smithi098 ceph-mon[103124]: pgmap v7194: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T02:59:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:17 smithi098 ceph-mon[103124]: pgmap v7195: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T02:59:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:17 smithi098 ceph-mon[103124]: osdmap e518: 12 total, 12 up, 12 in 2023-09-10T02:59:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:17 smithi170 ceph-mon[102832]: pgmap v7195: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T02:59:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:17 smithi170 ceph-mon[102832]: osdmap e518: 12 total, 12 up, 12 in 2023-09-10T02:59:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:17 smithi082 ceph-mon[98694]: pgmap v7195: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T02:59:17.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:17 smithi082 ceph-mon[98694]: osdmap e518: 12 total, 12 up, 12 in 2023-09-10T02:59:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13188.845060 secs 2023-09-10T02:59:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:59:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13188.845060 secs 2023-09-10T02:59:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:59:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13188.845060 secs 2023-09-10T02:59:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T02:59:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13190.208754 secs 2023-09-10T02:59:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:19 smithi098 ceph-mon[103124]: pgmap v7197: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T02:59:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13190.920211 secs 2023-09-10T02:59:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13190.208754 secs 2023-09-10T02:59:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:19 smithi170 ceph-mon[102832]: pgmap v7197: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T02:59:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13190.920211 secs 2023-09-10T02:59:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13190.208754 secs 2023-09-10T02:59:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:19 smithi082 ceph-mon[98694]: pgmap v7197: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T02:59:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13190.920211 secs 2023-09-10T02:59:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:21 smithi098 ceph-mon[103124]: pgmap v7198: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T02:59:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:59:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:59:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:59:21.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:59:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:21 smithi170 ceph-mon[102832]: pgmap v7198: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T02:59:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:59:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:59:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:59:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:59:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:21 smithi082 ceph-mon[98694]: pgmap v7198: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T02:59:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:59:21.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:59:21.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:59:21.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:59:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:22 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 145.0M 2023-09-10T02:59:22.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:22 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 152058163: error parsing value: Value '152058163' is below minimum 939524096 2023-09-10T02:59:22.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:59:22.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:59:22.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:59:22.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:59:22.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:59:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:22 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 145.0M 2023-09-10T02:59:22.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:22 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 152058163: error parsing value: Value '152058163' is below minimum 939524096 2023-09-10T02:59:22.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:59:22.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:59:22.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:59:22.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:59:22.682 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:59:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:22 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 145.0M 2023-09-10T02:59:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:22 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 152058163: error parsing value: Value '152058163' is below minimum 939524096 2023-09-10T02:59:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:59:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:59:22.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T02:59:22.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T02:59:22.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T02:59:22.711 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T02:59:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:23 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T02:59:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:23 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T02:59:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:23 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T02:59:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:23 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T02:59:23.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:23 smithi098 ceph-mon[103124]: pgmap v7199: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T02:59:23.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:59:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:23 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T02:59:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:23 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T02:59:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:23 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T02:59:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:23 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T02:59:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:23 smithi170 ceph-mon[102832]: pgmap v7199: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T02:59:23.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:59:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:23 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T02:59:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:23 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T02:59:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:23 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T02:59:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:23 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T02:59:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:23 smithi082 ceph-mon[98694]: pgmap v7199: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T02:59:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:59:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13195.208837 secs 2023-09-10T02:59:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:59:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:59:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:59:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13195.920275 secs 2023-09-10T02:59:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13195.208837 secs 2023-09-10T02:59:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:59:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:59:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:59:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13195.920275 secs 2023-09-10T02:59:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13195.208837 secs 2023-09-10T02:59:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:59:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:59:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:59:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13195.920275 secs 2023-09-10T02:59:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:25 smithi170 ceph-mon[102832]: pgmap v7200: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T02:59:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:25 smithi082 ceph-mon[98694]: pgmap v7200: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T02:59:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:25 smithi098 ceph-mon[103124]: pgmap v7200: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T02:59:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:27 smithi170 ceph-mon[102832]: pgmap v7201: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T02:59:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:27 smithi082 ceph-mon[98694]: pgmap v7201: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T02:59:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:27 smithi098 ceph-mon[103124]: pgmap v7201: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T02:59:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13200.208895 secs 2023-09-10T02:59:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:29 smithi170 ceph-mon[102832]: pgmap v7202: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T02:59:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13200.920341 secs 2023-09-10T02:59:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13200.208895 secs 2023-09-10T02:59:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:29 smithi082 ceph-mon[98694]: pgmap v7202: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T02:59:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13200.920341 secs 2023-09-10T02:59:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13200.208895 secs 2023-09-10T02:59:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:29 smithi098 ceph-mon[103124]: pgmap v7202: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T02:59:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13200.920341 secs 2023-09-10T02:59:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:31 smithi170 ceph-mon[102832]: pgmap v7203: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 9 op/s 2023-09-10T02:59:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:31 smithi082 ceph-mon[98694]: pgmap v7203: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 9 op/s 2023-09-10T02:59:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:31 smithi098 ceph-mon[103124]: pgmap v7203: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 9 op/s 2023-09-10T02:59:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:33 smithi170 ceph-mon[102832]: pgmap v7204: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 147 KiB/s wr, 14 op/s 2023-09-10T02:59:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:33 smithi082 ceph-mon[98694]: pgmap v7204: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 147 KiB/s wr, 14 op/s 2023-09-10T02:59:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:33 smithi098 ceph-mon[103124]: pgmap v7204: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 147 KiB/s wr, 14 op/s 2023-09-10T02:59:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13205.208956 secs 2023-09-10T02:59:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13205.208956 secs 2023-09-10T02:59:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13205.208956 secs 2023-09-10T02:59:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13205.920406 secs 2023-09-10T02:59:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:35 smithi082 ceph-mon[98694]: pgmap v7205: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 10 op/s 2023-09-10T02:59:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13205.920406 secs 2023-09-10T02:59:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:35 smithi098 ceph-mon[103124]: pgmap v7205: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 10 op/s 2023-09-10T02:59:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13205.920406 secs 2023-09-10T02:59:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:35 smithi170 ceph-mon[102832]: pgmap v7205: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 10 op/s 2023-09-10T02:59:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:37 smithi098 ceph-mon[103124]: pgmap v7206: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 208 KiB/s wr, 16 op/s 2023-09-10T02:59:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:37 smithi170 ceph-mon[102832]: pgmap v7206: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 208 KiB/s wr, 16 op/s 2023-09-10T02:59:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:37 smithi082 ceph-mon[98694]: pgmap v7206: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 208 KiB/s wr, 16 op/s 2023-09-10T02:59:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13208.845448 secs 2023-09-10T02:59:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13208.845448 secs 2023-09-10T02:59:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13208.845448 secs 2023-09-10T02:59:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13210.209017 secs 2023-09-10T02:59:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:39 smithi170 ceph-mon[102832]: pgmap v7207: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T02:59:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13210.920480 secs 2023-09-10T02:59:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13210.209017 secs 2023-09-10T02:59:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:39 smithi082 ceph-mon[98694]: pgmap v7207: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T02:59:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13210.920480 secs 2023-09-10T02:59:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13210.209017 secs 2023-09-10T02:59:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:39 smithi098 ceph-mon[103124]: pgmap v7207: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T02:59:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13210.920480 secs 2023-09-10T02:59:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:59:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:59:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:59:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:59:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:59:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:59:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:59:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:59:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:59:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:41 smithi170 ceph-mon[102832]: pgmap v7208: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T02:59:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:41 smithi082 ceph-mon[98694]: pgmap v7208: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T02:59:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:41 smithi098 ceph-mon[103124]: pgmap v7208: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T02:59:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:43 smithi082 ceph-mon[98694]: pgmap v7209: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 272 KiB/s wr, 18 op/s 2023-09-10T02:59:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:59:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:43 smithi098 ceph-mon[103124]: pgmap v7209: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 272 KiB/s wr, 18 op/s 2023-09-10T02:59:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:59:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:43 smithi170 ceph-mon[102832]: pgmap v7209: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 272 KiB/s wr, 18 op/s 2023-09-10T02:59:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T02:59:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13215.209104 secs 2023-09-10T02:59:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:59:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:59:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13215.209104 secs 2023-09-10T02:59:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:59:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:59:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13215.209104 secs 2023-09-10T02:59:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:59:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:59:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13215.920545 secs 2023-09-10T02:59:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:45 smithi082 ceph-mon[98694]: pgmap v7210: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 185 KiB/s wr, 12 op/s 2023-09-10T02:59:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13215.920545 secs 2023-09-10T02:59:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:45 smithi098 ceph-mon[103124]: pgmap v7210: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 185 KiB/s wr, 12 op/s 2023-09-10T02:59:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13215.920545 secs 2023-09-10T02:59:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:45 smithi170 ceph-mon[102832]: pgmap v7210: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 185 KiB/s wr, 12 op/s 2023-09-10T02:59:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:47 smithi082 ceph-mon[98694]: pgmap v7211: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 244 KiB/s wr, 18 op/s 2023-09-10T02:59:47.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:47 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:59:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:47 smithi098 ceph-mon[103124]: pgmap v7211: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 244 KiB/s wr, 18 op/s 2023-09-10T02:59:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:47 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:59:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:47 smithi170 ceph-mon[102832]: pgmap v7211: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 244 KiB/s wr, 18 op/s 2023-09-10T02:59:47.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:47 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:59:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13220.209197 secs 2023-09-10T02:59:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:49 smithi082 ceph-mon[98694]: pgmap v7212: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T02:59:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13220.209197 secs 2023-09-10T02:59:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:49 smithi098 ceph-mon[103124]: pgmap v7212: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T02:59:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13220.209197 secs 2023-09-10T02:59:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:49 smithi170 ceph-mon[102832]: pgmap v7212: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T02:59:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13220.920612 secs 2023-09-10T02:59:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13220.920612 secs 2023-09-10T02:59:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13220.920612 secs 2023-09-10T02:59:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:51 smithi098 ceph-mon[103124]: pgmap v7213: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T02:59:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:51 smithi170 ceph-mon[102832]: pgmap v7213: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T02:59:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:51 smithi082 ceph-mon[98694]: pgmap v7213: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T02:59:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13223.845658 secs 2023-09-10T02:59:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:53 smithi082 ceph-mon[98694]: pgmap v7214: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 298 KiB/s wr, 18 op/s 2023-09-10T02:59:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13223.845658 secs 2023-09-10T02:59:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:53 smithi098 ceph-mon[103124]: pgmap v7214: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 298 KiB/s wr, 18 op/s 2023-09-10T02:59:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13223.845658 secs 2023-09-10T02:59:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:53 smithi170 ceph-mon[102832]: pgmap v7214: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 298 KiB/s wr, 18 op/s 2023-09-10T02:59:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13225.209256 secs 2023-09-10T02:59:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13225.209256 secs 2023-09-10T02:59:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13225.209256 secs 2023-09-10T02:59:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13225.920676 secs 2023-09-10T02:59:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:55 smithi082 ceph-mon[98694]: pgmap v7215: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 174 KiB/s wr, 12 op/s 2023-09-10T02:59:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13225.920676 secs 2023-09-10T02:59:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:55 smithi098 ceph-mon[103124]: pgmap v7215: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 174 KiB/s wr, 12 op/s 2023-09-10T02:59:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13225.920676 secs 2023-09-10T02:59:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:55 smithi170 ceph-mon[102832]: pgmap v7215: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 174 KiB/s wr, 12 op/s 2023-09-10T02:59:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:59:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:59:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:59:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:59:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:59:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:59:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T02:59:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T02:59:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T02:59:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:57 smithi082 ceph-mon[98694]: pgmap v7216: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 248 KiB/s wr, 18 op/s 2023-09-10T02:59:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:57 smithi098 ceph-mon[103124]: pgmap v7216: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 248 KiB/s wr, 18 op/s 2023-09-10T02:59:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:57 smithi170 ceph-mon[102832]: pgmap v7216: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 248 KiB/s wr, 18 op/s 2023-09-10T02:59:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13228.845715 secs 2023-09-10T02:59:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13228.845715 secs 2023-09-10T02:59:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13228.845715 secs 2023-09-10T02:59:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13230.209315 secs 2023-09-10T02:59:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 02:59:59 smithi082 ceph-mon[98694]: pgmap v7217: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 189 KiB/s wr, 12 op/s 2023-09-10T02:59:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13230.209315 secs 2023-09-10T02:59:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 02:59:59 smithi098 ceph-mon[103124]: pgmap v7217: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 189 KiB/s wr, 12 op/s 2023-09-10T02:59:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13230.209315 secs 2023-09-10T02:59:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 02:59:59 smithi170 ceph-mon[102832]: pgmap v7217: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 189 KiB/s wr, 12 op/s 2023-09-10T03:00:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13230.920747 secs 2023-09-10T03:00:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T03:00:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13230.920747 secs 2023-09-10T03:00:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T03:00:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13230.920747 secs 2023-09-10T03:00:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T03:00:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:01 smithi098 ceph-mon[103124]: pgmap v7218: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 189 KiB/s wr, 12 op/s 2023-09-10T03:00:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:01 smithi170 ceph-mon[102832]: pgmap v7218: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 189 KiB/s wr, 12 op/s 2023-09-10T03:00:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:01 smithi082 ceph-mon[98694]: pgmap v7218: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 189 KiB/s wr, 12 op/s 2023-09-10T03:00:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13233.845832 secs 2023-09-10T03:00:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:03 smithi082 ceph-mon[98694]: pgmap v7219: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 252 KiB/s wr, 18 op/s 2023-09-10T03:00:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13233.845832 secs 2023-09-10T03:00:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:03 smithi098 ceph-mon[103124]: pgmap v7219: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 252 KiB/s wr, 18 op/s 2023-09-10T03:00:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13233.845832 secs 2023-09-10T03:00:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:03 smithi170 ceph-mon[102832]: pgmap v7219: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 252 KiB/s wr, 18 op/s 2023-09-10T03:00:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13235.209372 secs 2023-09-10T03:00:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13235.209372 secs 2023-09-10T03:00:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13235.209372 secs 2023-09-10T03:00:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13235.920816 secs 2023-09-10T03:00:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:05 smithi082 ceph-mon[98694]: pgmap v7220: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 11 op/s 2023-09-10T03:00:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13235.920816 secs 2023-09-10T03:00:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:05 smithi098 ceph-mon[103124]: pgmap v7220: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 11 op/s 2023-09-10T03:00:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13235.920816 secs 2023-09-10T03:00:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:05 smithi170 ceph-mon[102832]: pgmap v7220: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 11 op/s 2023-09-10T03:00:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:07 smithi098 ceph-mon[103124]: pgmap v7221: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 209 KiB/s wr, 31 op/s 2023-09-10T03:00:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:07 smithi170 ceph-mon[102832]: pgmap v7221: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 209 KiB/s wr, 31 op/s 2023-09-10T03:00:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:07 smithi082 ceph-mon[98694]: pgmap v7221: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 209 KiB/s wr, 31 op/s 2023-09-10T03:00:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13238.845891 secs 2023-09-10T03:00:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:08 smithi170 ceph-mon[102832]: osdmap e519: 12 total, 12 up, 12 in 2023-09-10T03:00:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13238.845891 secs 2023-09-10T03:00:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:08 smithi082 ceph-mon[98694]: osdmap e519: 12 total, 12 up, 12 in 2023-09-10T03:00:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13238.845891 secs 2023-09-10T03:00:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:08 smithi098 ceph-mon[103124]: osdmap e519: 12 total, 12 up, 12 in 2023-09-10T03:00:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13240.209456 secs 2023-09-10T03:00:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:09 smithi170 ceph-mon[102832]: pgmap v7223: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 162 KiB/s wr, 30 op/s 2023-09-10T03:00:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13240.920880 secs 2023-09-10T03:00:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13240.209456 secs 2023-09-10T03:00:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:09 smithi082 ceph-mon[98694]: pgmap v7223: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 162 KiB/s wr, 30 op/s 2023-09-10T03:00:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13240.920880 secs 2023-09-10T03:00:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13240.209456 secs 2023-09-10T03:00:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:09 smithi098 ceph-mon[103124]: pgmap v7223: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 162 KiB/s wr, 30 op/s 2023-09-10T03:00:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13240.920880 secs 2023-09-10T03:00:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:11 smithi170 ceph-mon[102832]: pgmap v7224: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 162 KiB/s wr, 30 op/s 2023-09-10T03:00:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:11 smithi082 ceph-mon[98694]: pgmap v7224: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 162 KiB/s wr, 30 op/s 2023-09-10T03:00:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:11 smithi098 ceph-mon[103124]: pgmap v7224: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 162 KiB/s wr, 30 op/s 2023-09-10T03:00:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13243.846001 secs 2023-09-10T03:00:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:13 smithi082 ceph-mon[98694]: pgmap v7225: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T03:00:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13243.846001 secs 2023-09-10T03:00:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:13 smithi098 ceph-mon[103124]: pgmap v7225: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T03:00:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13243.846001 secs 2023-09-10T03:00:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:13 smithi170 ceph-mon[102832]: pgmap v7225: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T03:00:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13245.209518 secs 2023-09-10T03:00:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13245.209518 secs 2023-09-10T03:00:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13245.209518 secs 2023-09-10T03:00:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13245.920944 secs 2023-09-10T03:00:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:15 smithi082 ceph-mon[98694]: pgmap v7226: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T03:00:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13245.920944 secs 2023-09-10T03:00:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:15 smithi098 ceph-mon[103124]: pgmap v7226: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T03:00:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13245.920944 secs 2023-09-10T03:00:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:15 smithi170 ceph-mon[102832]: pgmap v7226: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T03:00:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:17 smithi098 ceph-mon[103124]: pgmap v7227: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2023-09-10T03:00:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:17 smithi098 ceph-mon[103124]: osdmap e520: 12 total, 12 up, 12 in 2023-09-10T03:00:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:17 smithi170 ceph-mon[102832]: pgmap v7227: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2023-09-10T03:00:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:17 smithi170 ceph-mon[102832]: osdmap e520: 12 total, 12 up, 12 in 2023-09-10T03:00:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:17 smithi082 ceph-mon[98694]: pgmap v7227: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2023-09-10T03:00:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:17 smithi082 ceph-mon[98694]: osdmap e520: 12 total, 12 up, 12 in 2023-09-10T03:00:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13248.846061 secs 2023-09-10T03:00:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13248.846061 secs 2023-09-10T03:00:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13248.846061 secs 2023-09-10T03:00:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13250.209599 secs 2023-09-10T03:00:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:19 smithi170 ceph-mon[102832]: pgmap v7229: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2023-09-10T03:00:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13250.921012 secs 2023-09-10T03:00:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13250.209599 secs 2023-09-10T03:00:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:19 smithi082 ceph-mon[98694]: pgmap v7229: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2023-09-10T03:00:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13250.921012 secs 2023-09-10T03:00:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13250.209599 secs 2023-09-10T03:00:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:19 smithi098 ceph-mon[103124]: pgmap v7229: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2023-09-10T03:00:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13250.921012 secs 2023-09-10T03:00:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:21 smithi170 ceph-mon[102832]: pgmap v7230: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2023-09-10T03:00:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:21 smithi082 ceph-mon[98694]: pgmap v7230: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2023-09-10T03:00:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:21 smithi098 ceph-mon[103124]: pgmap v7230: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2023-09-10T03:00:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:00:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:23 smithi082 ceph-mon[98694]: pgmap v7231: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 12 op/s 2023-09-10T03:00:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:00:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:00:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:00:23.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:00:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:00:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:23 smithi098 ceph-mon[103124]: pgmap v7231: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 12 op/s 2023-09-10T03:00:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:00:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:00:23.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:00:23.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:00:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:00:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:23 smithi170 ceph-mon[102832]: pgmap v7231: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 12 op/s 2023-09-10T03:00:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:00:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:00:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:00:23.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:00:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13255.209656 secs 2023-09-10T03:00:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:00:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:00:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:00:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13255.209656 secs 2023-09-10T03:00:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:00:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:00:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:00:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13255.209656 secs 2023-09-10T03:00:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:00:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:00:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:00:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13255.921079 secs 2023-09-10T03:00:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:25 smithi082 ceph-mon[98694]: pgmap v7232: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 12 op/s 2023-09-10T03:00:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13255.921079 secs 2023-09-10T03:00:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:25 smithi098 ceph-mon[103124]: pgmap v7232: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 12 op/s 2023-09-10T03:00:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13255.921079 secs 2023-09-10T03:00:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:25 smithi170 ceph-mon[102832]: pgmap v7232: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 12 op/s 2023-09-10T03:00:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:27 smithi082 ceph-mon[98694]: pgmap v7233: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 11 op/s 2023-09-10T03:00:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:27 smithi098 ceph-mon[103124]: pgmap v7233: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 11 op/s 2023-09-10T03:00:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:27 smithi170 ceph-mon[102832]: pgmap v7233: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 11 op/s 2023-09-10T03:00:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13260.209715 secs 2023-09-10T03:00:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:29 smithi082 ceph-mon[98694]: pgmap v7234: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T03:00:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13260.209715 secs 2023-09-10T03:00:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:29 smithi098 ceph-mon[103124]: pgmap v7234: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T03:00:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13260.209715 secs 2023-09-10T03:00:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:29 smithi170 ceph-mon[102832]: pgmap v7234: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T03:00:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13260.921145 secs 2023-09-10T03:00:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13260.921145 secs 2023-09-10T03:00:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13260.921145 secs 2023-09-10T03:00:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:31 smithi098 ceph-mon[103124]: pgmap v7235: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T03:00:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:31 smithi170 ceph-mon[102832]: pgmap v7235: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T03:00:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:31 smithi082 ceph-mon[98694]: pgmap v7235: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T03:00:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:33 smithi082 ceph-mon[98694]: pgmap v7236: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 292 KiB/s wr, 16 op/s 2023-09-10T03:00:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:33 smithi098 ceph-mon[103124]: pgmap v7236: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 292 KiB/s wr, 16 op/s 2023-09-10T03:00:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:33 smithi170 ceph-mon[102832]: pgmap v7236: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 292 KiB/s wr, 16 op/s 2023-09-10T03:00:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13265.209775 secs 2023-09-10T03:00:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13265.209775 secs 2023-09-10T03:00:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13265.209775 secs 2023-09-10T03:00:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13265.921210 secs 2023-09-10T03:00:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:35 smithi082 ceph-mon[98694]: pgmap v7237: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 252 KiB/s wr, 11 op/s 2023-09-10T03:00:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13265.921210 secs 2023-09-10T03:00:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:35 smithi098 ceph-mon[103124]: pgmap v7237: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 252 KiB/s wr, 11 op/s 2023-09-10T03:00:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13265.921210 secs 2023-09-10T03:00:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:35 smithi170 ceph-mon[102832]: pgmap v7237: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 252 KiB/s wr, 11 op/s 2023-09-10T03:00:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:37 smithi098 ceph-mon[103124]: pgmap v7238: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 266 KiB/s wr, 17 op/s 2023-09-10T03:00:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:37 smithi170 ceph-mon[102832]: pgmap v7238: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 266 KiB/s wr, 17 op/s 2023-09-10T03:00:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:37 smithi082 ceph-mon[98694]: pgmap v7238: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 266 KiB/s wr, 17 op/s 2023-09-10T03:00:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13268.846352 secs 2023-09-10T03:00:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13268.846352 secs 2023-09-10T03:00:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13268.846352 secs 2023-09-10T03:00:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13270.209852 secs 2023-09-10T03:00:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:39 smithi098 ceph-mon[103124]: pgmap v7239: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 226 KiB/s wr, 12 op/s 2023-09-10T03:00:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13270.209852 secs 2023-09-10T03:00:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:39 smithi170 ceph-mon[102832]: pgmap v7239: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 226 KiB/s wr, 12 op/s 2023-09-10T03:00:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13270.209852 secs 2023-09-10T03:00:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:39 smithi082 ceph-mon[98694]: pgmap v7239: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 226 KiB/s wr, 12 op/s 2023-09-10T03:00:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13270.921278 secs 2023-09-10T03:00:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:00:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:00:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:00:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13270.921278 secs 2023-09-10T03:00:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:00:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:00:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:00:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13270.921278 secs 2023-09-10T03:00:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:00:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:00:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:00:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:41 smithi098 ceph-mon[103124]: pgmap v7240: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 226 KiB/s wr, 12 op/s 2023-09-10T03:00:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:41 smithi170 ceph-mon[102832]: pgmap v7240: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 226 KiB/s wr, 12 op/s 2023-09-10T03:00:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:41 smithi082 ceph-mon[98694]: pgmap v7240: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 226 KiB/s wr, 12 op/s 2023-09-10T03:00:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13273.846419 secs 2023-09-10T03:00:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:43 smithi082 ceph-mon[98694]: pgmap v7241: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 371 KiB/s wr, 19 op/s 2023-09-10T03:00:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13273.846419 secs 2023-09-10T03:00:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:43 smithi098 ceph-mon[103124]: pgmap v7241: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 371 KiB/s wr, 19 op/s 2023-09-10T03:00:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13273.846419 secs 2023-09-10T03:00:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:43 smithi170 ceph-mon[102832]: pgmap v7241: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 371 KiB/s wr, 19 op/s 2023-09-10T03:00:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13275.209936 secs 2023-09-10T03:00:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13275.209936 secs 2023-09-10T03:00:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13275.209936 secs 2023-09-10T03:00:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13275.921344 secs 2023-09-10T03:00:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:45 smithi082 ceph-mon[98694]: pgmap v7242: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T03:00:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13275.921344 secs 2023-09-10T03:00:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:45 smithi098 ceph-mon[103124]: pgmap v7242: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T03:00:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13275.921344 secs 2023-09-10T03:00:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:45 smithi170 ceph-mon[102832]: pgmap v7242: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T03:00:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:47 smithi170 ceph-mon[102832]: pgmap v7243: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 309 KiB/s wr, 18 op/s 2023-09-10T03:00:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:47 smithi082 ceph-mon[98694]: pgmap v7243: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 309 KiB/s wr, 18 op/s 2023-09-10T03:00:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:47 smithi098 ceph-mon[103124]: pgmap v7243: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 309 KiB/s wr, 18 op/s 2023-09-10T03:00:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13278.846479 secs 2023-09-10T03:00:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13278.846479 secs 2023-09-10T03:00:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13278.846479 secs 2023-09-10T03:00:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13280.209994 secs 2023-09-10T03:00:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:49 smithi170 ceph-mon[102832]: pgmap v7244: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T03:00:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13280.921413 secs 2023-09-10T03:00:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13280.209994 secs 2023-09-10T03:00:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:49 smithi082 ceph-mon[98694]: pgmap v7244: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T03:00:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13280.921413 secs 2023-09-10T03:00:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13280.209994 secs 2023-09-10T03:00:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:49 smithi098 ceph-mon[103124]: pgmap v7244: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T03:00:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13280.921413 secs 2023-09-10T03:00:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:51 smithi170 ceph-mon[102832]: pgmap v7245: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T03:00:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:51 smithi082 ceph-mon[98694]: pgmap v7245: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T03:00:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:51 smithi098 ceph-mon[103124]: pgmap v7245: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T03:00:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13283.846539 secs 2023-09-10T03:00:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:53 smithi082 ceph-mon[98694]: pgmap v7246: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 328 KiB/s wr, 17 op/s 2023-09-10T03:00:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13283.846539 secs 2023-09-10T03:00:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:53 smithi098 ceph-mon[103124]: pgmap v7246: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 328 KiB/s wr, 17 op/s 2023-09-10T03:00:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13283.846539 secs 2023-09-10T03:00:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:53 smithi170 ceph-mon[102832]: pgmap v7246: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 328 KiB/s wr, 17 op/s 2023-09-10T03:00:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13285.210051 secs 2023-09-10T03:00:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13285.210051 secs 2023-09-10T03:00:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13285.210051 secs 2023-09-10T03:00:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13285.921480 secs 2023-09-10T03:00:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:55 smithi098 ceph-mon[103124]: pgmap v7247: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T03:00:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13285.921480 secs 2023-09-10T03:00:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:55 smithi170 ceph-mon[102832]: pgmap v7247: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T03:00:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13285.921480 secs 2023-09-10T03:00:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:55 smithi082 ceph-mon[98694]: pgmap v7247: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T03:00:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:57 smithi098 ceph-mon[103124]: pgmap v7248: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 286 KiB/s wr, 16 op/s 2023-09-10T03:00:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:57 smithi170 ceph-mon[102832]: pgmap v7248: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 286 KiB/s wr, 16 op/s 2023-09-10T03:00:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:57 smithi082 ceph-mon[98694]: pgmap v7248: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 286 KiB/s wr, 16 op/s 2023-09-10T03:00:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13288.846608 secs 2023-09-10T03:00:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13288.846608 secs 2023-09-10T03:00:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13288.846608 secs 2023-09-10T03:00:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13290.210109 secs 2023-09-10T03:00:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:59 smithi098 ceph-mon[103124]: pgmap v7249: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 138 KiB/s wr, 10 op/s 2023-09-10T03:00:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:00:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13290.921546 secs 2023-09-10T03:00:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13290.210109 secs 2023-09-10T03:00:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:59 smithi170 ceph-mon[102832]: pgmap v7249: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 138 KiB/s wr, 10 op/s 2023-09-10T03:00:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:00:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13290.921546 secs 2023-09-10T03:00:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13290.210109 secs 2023-09-10T03:00:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:59 smithi082 ceph-mon[98694]: pgmap v7249: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 138 KiB/s wr, 10 op/s 2023-09-10T03:00:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:00:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13290.921546 secs 2023-09-10T03:01:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:01 smithi098 ceph-mon[103124]: pgmap v7250: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T03:01:01.614 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:01 smithi082 ceph-mon[98694]: pgmap v7250: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T03:01:02.034 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:01 smithi170 ceph-mon[102832]: pgmap v7250: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T03:01:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13293.846668 secs 2023-09-10T03:01:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:03 smithi082 ceph-mon[98694]: pgmap v7251: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 175 KiB/s wr, 15 op/s 2023-09-10T03:01:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13293.846668 secs 2023-09-10T03:01:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:03 smithi098 ceph-mon[103124]: pgmap v7251: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 175 KiB/s wr, 15 op/s 2023-09-10T03:01:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13293.846668 secs 2023-09-10T03:01:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:03 smithi170 ceph-mon[102832]: pgmap v7251: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 175 KiB/s wr, 15 op/s 2023-09-10T03:01:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13295.210175 secs 2023-09-10T03:01:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13295.210175 secs 2023-09-10T03:01:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13295.210175 secs 2023-09-10T03:01:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13295.921611 secs 2023-09-10T03:01:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:05 smithi098 ceph-mon[103124]: pgmap v7252: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 141 KiB/s wr, 10 op/s 2023-09-10T03:01:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13295.921611 secs 2023-09-10T03:01:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:05 smithi170 ceph-mon[102832]: pgmap v7252: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 141 KiB/s wr, 10 op/s 2023-09-10T03:01:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13295.921611 secs 2023-09-10T03:01:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:05 smithi082 ceph-mon[98694]: pgmap v7252: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 141 KiB/s wr, 10 op/s 2023-09-10T03:01:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:07 smithi098 ceph-mon[103124]: pgmap v7253: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 197 KiB/s wr, 30 op/s 2023-09-10T03:01:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:07 smithi170 ceph-mon[102832]: pgmap v7253: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 197 KiB/s wr, 30 op/s 2023-09-10T03:01:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:07 smithi082 ceph-mon[98694]: pgmap v7253: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 197 KiB/s wr, 30 op/s 2023-09-10T03:01:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13298.846775 secs 2023-09-10T03:01:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:08 smithi098 ceph-mon[103124]: osdmap e521: 12 total, 12 up, 12 in 2023-09-10T03:01:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13298.846775 secs 2023-09-10T03:01:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:08 smithi170 ceph-mon[102832]: osdmap e521: 12 total, 12 up, 12 in 2023-09-10T03:01:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13298.846775 secs 2023-09-10T03:01:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:08 smithi082 ceph-mon[98694]: osdmap e521: 12 total, 12 up, 12 in 2023-09-10T03:01:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13300.210234 secs 2023-09-10T03:01:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:09 smithi098 ceph-mon[103124]: pgmap v7255: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 111 KiB/s wr, 30 op/s 2023-09-10T03:01:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13300.921679 secs 2023-09-10T03:01:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13300.210234 secs 2023-09-10T03:01:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:09 smithi170 ceph-mon[102832]: pgmap v7255: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 111 KiB/s wr, 30 op/s 2023-09-10T03:01:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13300.921679 secs 2023-09-10T03:01:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13300.210234 secs 2023-09-10T03:01:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:09 smithi082 ceph-mon[98694]: pgmap v7255: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 111 KiB/s wr, 30 op/s 2023-09-10T03:01:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13300.921679 secs 2023-09-10T03:01:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:11 smithi098 ceph-mon[103124]: pgmap v7256: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 112 KiB/s wr, 30 op/s 2023-09-10T03:01:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:11 smithi170 ceph-mon[102832]: pgmap v7256: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 112 KiB/s wr, 30 op/s 2023-09-10T03:01:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:11 smithi082 ceph-mon[98694]: pgmap v7256: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 112 KiB/s wr, 30 op/s 2023-09-10T03:01:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13303.846883 secs 2023-09-10T03:01:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:13 smithi098 ceph-mon[103124]: pgmap v7257: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 68 KiB/s wr, 29 op/s 2023-09-10T03:01:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13303.846883 secs 2023-09-10T03:01:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:13 smithi170 ceph-mon[102832]: pgmap v7257: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 68 KiB/s wr, 29 op/s 2023-09-10T03:01:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13303.846883 secs 2023-09-10T03:01:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:13 smithi082 ceph-mon[98694]: pgmap v7257: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 68 KiB/s wr, 29 op/s 2023-09-10T03:01:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13305.210315 secs 2023-09-10T03:01:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13305.210315 secs 2023-09-10T03:01:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13305.210315 secs 2023-09-10T03:01:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13305.921740 secs 2023-09-10T03:01:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:15 smithi098 ceph-mon[103124]: pgmap v7258: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 68 KiB/s wr, 29 op/s 2023-09-10T03:01:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13305.921740 secs 2023-09-10T03:01:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:15 smithi170 ceph-mon[102832]: pgmap v7258: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 68 KiB/s wr, 29 op/s 2023-09-10T03:01:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13305.921740 secs 2023-09-10T03:01:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:15 smithi082 ceph-mon[98694]: pgmap v7258: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 68 KiB/s wr, 29 op/s 2023-09-10T03:01:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:17 smithi098 ceph-mon[103124]: pgmap v7259: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T03:01:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:17 smithi098 ceph-mon[103124]: osdmap e522: 12 total, 12 up, 12 in 2023-09-10T03:01:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:17 smithi170 ceph-mon[102832]: pgmap v7259: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T03:01:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:17 smithi170 ceph-mon[102832]: osdmap e522: 12 total, 12 up, 12 in 2023-09-10T03:01:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:17 smithi082 ceph-mon[98694]: pgmap v7259: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T03:01:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:17 smithi082 ceph-mon[98694]: osdmap e522: 12 total, 12 up, 12 in 2023-09-10T03:01:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13308.847003 secs 2023-09-10T03:01:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13308.847003 secs 2023-09-10T03:01:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13308.847003 secs 2023-09-10T03:01:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13310.210373 secs 2023-09-10T03:01:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:19 smithi098 ceph-mon[103124]: pgmap v7261: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T03:01:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13310.210373 secs 2023-09-10T03:01:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:19 smithi170 ceph-mon[102832]: pgmap v7261: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T03:01:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13310.210373 secs 2023-09-10T03:01:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:19 smithi082 ceph-mon[98694]: pgmap v7261: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T03:01:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13310.921807 secs 2023-09-10T03:01:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13310.921807 secs 2023-09-10T03:01:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13310.921807 secs 2023-09-10T03:01:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:21 smithi098 ceph-mon[103124]: pgmap v7262: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T03:01:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:21 smithi170 ceph-mon[102832]: pgmap v7262: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T03:01:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:21 smithi082 ceph-mon[98694]: pgmap v7262: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T03:01:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:23 smithi098 ceph-mon[103124]: pgmap v7263: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T03:01:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:01:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:23 smithi170 ceph-mon[102832]: pgmap v7263: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T03:01:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:01:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:23 smithi082 ceph-mon[98694]: pgmap v7263: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T03:01:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:01:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13315.210452 secs 2023-09-10T03:01:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:01:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:01:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:01:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:01:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:01:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:01:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:01:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:01:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:01:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13315.210452 secs 2023-09-10T03:01:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:01:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:01:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:01:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:01:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:01:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:01:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:01:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:01:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:01:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13315.210452 secs 2023-09-10T03:01:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:01:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:01:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:01:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:01:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:01:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:01:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:01:24.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:01:24.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:01:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13315.921876 secs 2023-09-10T03:01:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:25 smithi098 ceph-mon[103124]: pgmap v7264: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T03:01:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13315.921876 secs 2023-09-10T03:01:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:25 smithi170 ceph-mon[102832]: pgmap v7264: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T03:01:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13315.921876 secs 2023-09-10T03:01:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:25 smithi082 ceph-mon[98694]: pgmap v7264: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 49 KiB/s wr, 11 op/s 2023-09-10T03:01:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:27 smithi098 ceph-mon[103124]: pgmap v7265: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T03:01:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:27 smithi170 ceph-mon[102832]: pgmap v7265: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T03:01:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:27 smithi082 ceph-mon[98694]: pgmap v7265: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T03:01:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13320.210496 secs 2023-09-10T03:01:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:29 smithi098 ceph-mon[103124]: pgmap v7266: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T03:01:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13320.210496 secs 2023-09-10T03:01:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:29 smithi170 ceph-mon[102832]: pgmap v7266: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T03:01:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13320.210496 secs 2023-09-10T03:01:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:29 smithi082 ceph-mon[98694]: pgmap v7266: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T03:01:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13320.921941 secs 2023-09-10T03:01:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13320.921941 secs 2023-09-10T03:01:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13320.921941 secs 2023-09-10T03:01:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:31 smithi098 ceph-mon[103124]: pgmap v7267: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 10 op/s 2023-09-10T03:01:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:31 smithi170 ceph-mon[102832]: pgmap v7267: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 10 op/s 2023-09-10T03:01:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:31 smithi082 ceph-mon[98694]: pgmap v7267: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 10 op/s 2023-09-10T03:01:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:33 smithi098 ceph-mon[103124]: pgmap v7268: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 149 KiB/s wr, 14 op/s 2023-09-10T03:01:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:33 smithi170 ceph-mon[102832]: pgmap v7268: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 149 KiB/s wr, 14 op/s 2023-09-10T03:01:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:33 smithi082 ceph-mon[98694]: pgmap v7268: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 149 KiB/s wr, 14 op/s 2023-09-10T03:01:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13325.210554 secs 2023-09-10T03:01:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13325.210554 secs 2023-09-10T03:01:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13325.210554 secs 2023-09-10T03:01:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13325.922012 secs 2023-09-10T03:01:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:35 smithi098 ceph-mon[103124]: pgmap v7269: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 10 op/s 2023-09-10T03:01:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13325.922012 secs 2023-09-10T03:01:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:35 smithi170 ceph-mon[102832]: pgmap v7269: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 10 op/s 2023-09-10T03:01:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13325.922012 secs 2023-09-10T03:01:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:35 smithi082 ceph-mon[98694]: pgmap v7269: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 10 op/s 2023-09-10T03:01:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:37 smithi098 ceph-mon[103124]: pgmap v7270: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 211 KiB/s wr, 17 op/s 2023-09-10T03:01:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:37 smithi170 ceph-mon[102832]: pgmap v7270: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 211 KiB/s wr, 17 op/s 2023-09-10T03:01:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:37 smithi082 ceph-mon[98694]: pgmap v7270: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 211 KiB/s wr, 17 op/s 2023-09-10T03:01:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13328.847313 secs 2023-09-10T03:01:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13328.847313 secs 2023-09-10T03:01:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13328.847313 secs 2023-09-10T03:01:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13330.210613 secs 2023-09-10T03:01:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:39 smithi098 ceph-mon[103124]: pgmap v7271: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T03:01:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13330.922072 secs 2023-09-10T03:01:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13330.210613 secs 2023-09-10T03:01:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:39 smithi170 ceph-mon[102832]: pgmap v7271: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T03:01:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13330.922072 secs 2023-09-10T03:01:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13330.210613 secs 2023-09-10T03:01:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:39 smithi082 ceph-mon[98694]: pgmap v7271: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T03:01:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13330.922072 secs 2023-09-10T03:01:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:01:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:01:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:01:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:01:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:01:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:01:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:01:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:01:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:01:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:41 smithi098 ceph-mon[103124]: pgmap v7272: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 266 KiB/s wr, 16 op/s 2023-09-10T03:01:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:41 smithi170 ceph-mon[102832]: pgmap v7272: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 266 KiB/s wr, 16 op/s 2023-09-10T03:01:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:41 smithi082 ceph-mon[98694]: pgmap v7272: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 266 KiB/s wr, 16 op/s 2023-09-10T03:01:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13333.847417 secs 2023-09-10T03:01:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:43 smithi098 ceph-mon[103124]: pgmap v7273: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 277 KiB/s wr, 18 op/s 2023-09-10T03:01:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13333.847417 secs 2023-09-10T03:01:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:43 smithi170 ceph-mon[102832]: pgmap v7273: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 277 KiB/s wr, 18 op/s 2023-09-10T03:01:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13333.847417 secs 2023-09-10T03:01:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:43 smithi082 ceph-mon[98694]: pgmap v7273: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 277 KiB/s wr, 18 op/s 2023-09-10T03:01:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13335.210669 secs 2023-09-10T03:01:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13335.210669 secs 2023-09-10T03:01:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13335.210669 secs 2023-09-10T03:01:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13335.922139 secs 2023-09-10T03:01:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:45 smithi098 ceph-mon[103124]: pgmap v7274: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 189 KiB/s wr, 13 op/s 2023-09-10T03:01:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13335.922139 secs 2023-09-10T03:01:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:45 smithi170 ceph-mon[102832]: pgmap v7274: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 189 KiB/s wr, 13 op/s 2023-09-10T03:01:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13335.922139 secs 2023-09-10T03:01:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:45 smithi082 ceph-mon[98694]: pgmap v7274: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 189 KiB/s wr, 13 op/s 2023-09-10T03:01:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:47 smithi098 ceph-mon[103124]: pgmap v7275: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 239 KiB/s wr, 18 op/s 2023-09-10T03:01:47.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:47 smithi170 ceph-mon[102832]: pgmap v7275: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 239 KiB/s wr, 18 op/s 2023-09-10T03:01:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:47 smithi082 ceph-mon[98694]: pgmap v7275: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 239 KiB/s wr, 18 op/s 2023-09-10T03:01:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13338.847484 secs 2023-09-10T03:01:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13338.847484 secs 2023-09-10T03:01:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13338.847484 secs 2023-09-10T03:01:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13340.210762 secs 2023-09-10T03:01:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:49 smithi098 ceph-mon[103124]: pgmap v7276: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T03:01:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13340.922205 secs 2023-09-10T03:01:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13340.210762 secs 2023-09-10T03:01:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:49 smithi170 ceph-mon[102832]: pgmap v7276: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T03:01:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13340.922205 secs 2023-09-10T03:01:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13340.210762 secs 2023-09-10T03:01:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:49 smithi082 ceph-mon[98694]: pgmap v7276: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T03:01:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13340.922205 secs 2023-09-10T03:01:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:51 smithi098 ceph-mon[103124]: pgmap v7277: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 342 KiB/s wr, 15 op/s 2023-09-10T03:01:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:51 smithi170 ceph-mon[102832]: pgmap v7277: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 342 KiB/s wr, 15 op/s 2023-09-10T03:01:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:51 smithi082 ceph-mon[98694]: pgmap v7277: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 342 KiB/s wr, 15 op/s 2023-09-10T03:01:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13343.847564 secs 2023-09-10T03:01:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:53 smithi098 ceph-mon[103124]: pgmap v7278: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 246 KiB/s wr, 15 op/s 2023-09-10T03:01:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13343.847564 secs 2023-09-10T03:01:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:53 smithi170 ceph-mon[102832]: pgmap v7278: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 246 KiB/s wr, 15 op/s 2023-09-10T03:01:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13343.847564 secs 2023-09-10T03:01:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:53 smithi082 ceph-mon[98694]: pgmap v7278: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 246 KiB/s wr, 15 op/s 2023-09-10T03:01:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13345.210816 secs 2023-09-10T03:01:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13345.210816 secs 2023-09-10T03:01:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13345.210816 secs 2023-09-10T03:01:55.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13345.922269 secs 2023-09-10T03:01:55.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:55 smithi098 ceph-mon[103124]: pgmap v7279: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 236 KiB/s wr, 12 op/s 2023-09-10T03:01:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13345.922269 secs 2023-09-10T03:01:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:55 smithi170 ceph-mon[102832]: pgmap v7279: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 236 KiB/s wr, 12 op/s 2023-09-10T03:01:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13345.922269 secs 2023-09-10T03:01:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:55 smithi082 ceph-mon[98694]: pgmap v7279: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 236 KiB/s wr, 12 op/s 2023-09-10T03:01:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:57 smithi098 ceph-mon[103124]: pgmap v7280: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 267 KiB/s wr, 18 op/s 2023-09-10T03:01:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:57 smithi170 ceph-mon[102832]: pgmap v7280: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 267 KiB/s wr, 18 op/s 2023-09-10T03:01:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:57 smithi082 ceph-mon[98694]: pgmap v7280: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 267 KiB/s wr, 18 op/s 2023-09-10T03:01:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13348.847622 secs 2023-09-10T03:01:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13348.847622 secs 2023-09-10T03:01:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13348.847622 secs 2023-09-10T03:01:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13350.210876 secs 2023-09-10T03:01:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:59 smithi098 ceph-mon[103124]: pgmap v7281: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 217 KiB/s wr, 13 op/s 2023-09-10T03:01:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:01:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13350.922336 secs 2023-09-10T03:01:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13350.210876 secs 2023-09-10T03:01:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:59 smithi170 ceph-mon[102832]: pgmap v7281: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 217 KiB/s wr, 13 op/s 2023-09-10T03:01:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:01:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13350.922336 secs 2023-09-10T03:01:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13350.210876 secs 2023-09-10T03:01:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:59 smithi082 ceph-mon[98694]: pgmap v7281: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 217 KiB/s wr, 13 op/s 2023-09-10T03:01:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:01:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13350.922336 secs 2023-09-10T03:02:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:01 smithi098 ceph-mon[103124]: pgmap v7282: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 271 KiB/s wr, 15 op/s 2023-09-10T03:02:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:01 smithi170 ceph-mon[102832]: pgmap v7282: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 271 KiB/s wr, 15 op/s 2023-09-10T03:02:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:01 smithi082 ceph-mon[98694]: pgmap v7282: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 271 KiB/s wr, 15 op/s 2023-09-10T03:02:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13353.847683 secs 2023-09-10T03:02:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:03 smithi098 ceph-mon[103124]: pgmap v7283: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 14 op/s 2023-09-10T03:02:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13353.847683 secs 2023-09-10T03:02:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:03 smithi170 ceph-mon[102832]: pgmap v7283: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 14 op/s 2023-09-10T03:02:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13353.847683 secs 2023-09-10T03:02:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:03 smithi082 ceph-mon[98694]: pgmap v7283: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 14 op/s 2023-09-10T03:02:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13355.210959 secs 2023-09-10T03:02:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13355.210959 secs 2023-09-10T03:02:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13355.210959 secs 2023-09-10T03:02:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13355.922401 secs 2023-09-10T03:02:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:05 smithi098 ceph-mon[103124]: pgmap v7284: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T03:02:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13355.922401 secs 2023-09-10T03:02:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:05 smithi170 ceph-mon[102832]: pgmap v7284: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T03:02:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13355.922401 secs 2023-09-10T03:02:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:05 smithi082 ceph-mon[98694]: pgmap v7284: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T03:02:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:07 smithi170 ceph-mon[102832]: pgmap v7285: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 168 KiB/s wr, 30 op/s 2023-09-10T03:02:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:07 smithi082 ceph-mon[98694]: pgmap v7285: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 168 KiB/s wr, 30 op/s 2023-09-10T03:02:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:07 smithi098 ceph-mon[103124]: pgmap v7285: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 168 KiB/s wr, 30 op/s 2023-09-10T03:02:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13358.847765 secs 2023-09-10T03:02:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:08 smithi170 ceph-mon[102832]: osdmap e523: 12 total, 12 up, 12 in 2023-09-10T03:02:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13358.847765 secs 2023-09-10T03:02:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:08 smithi082 ceph-mon[98694]: osdmap e523: 12 total, 12 up, 12 in 2023-09-10T03:02:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13358.847765 secs 2023-09-10T03:02:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:08 smithi098 ceph-mon[103124]: osdmap e523: 12 total, 12 up, 12 in 2023-09-10T03:02:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13360.211018 secs 2023-09-10T03:02:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:09 smithi170 ceph-mon[102832]: pgmap v7287: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 163 KiB/s wr, 30 op/s 2023-09-10T03:02:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13360.922464 secs 2023-09-10T03:02:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13360.211018 secs 2023-09-10T03:02:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:09 smithi082 ceph-mon[98694]: pgmap v7287: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 163 KiB/s wr, 30 op/s 2023-09-10T03:02:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13360.922464 secs 2023-09-10T03:02:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13360.211018 secs 2023-09-10T03:02:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:09 smithi098 ceph-mon[103124]: pgmap v7287: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 163 KiB/s wr, 30 op/s 2023-09-10T03:02:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13360.922464 secs 2023-09-10T03:02:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:11 smithi170 ceph-mon[102832]: pgmap v7288: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 99 KiB/s wr, 29 op/s 2023-09-10T03:02:11.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:11 smithi082 ceph-mon[98694]: pgmap v7288: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 99 KiB/s wr, 29 op/s 2023-09-10T03:02:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:11 smithi098 ceph-mon[103124]: pgmap v7288: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 99 KiB/s wr, 29 op/s 2023-09-10T03:02:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13363.847776 secs 2023-09-10T03:02:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:13 smithi098 ceph-mon[103124]: pgmap v7289: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T03:02:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13363.847776 secs 2023-09-10T03:02:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:13 smithi170 ceph-mon[102832]: pgmap v7289: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T03:02:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13363.847776 secs 2023-09-10T03:02:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:13 smithi082 ceph-mon[98694]: pgmap v7289: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T03:02:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13365.211077 secs 2023-09-10T03:02:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13365.211077 secs 2023-09-10T03:02:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13365.211077 secs 2023-09-10T03:02:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13365.922484 secs 2023-09-10T03:02:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:15 smithi098 ceph-mon[103124]: pgmap v7290: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T03:02:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13365.922484 secs 2023-09-10T03:02:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:15 smithi170 ceph-mon[102832]: pgmap v7290: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T03:02:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13365.922484 secs 2023-09-10T03:02:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:15 smithi082 ceph-mon[98694]: pgmap v7290: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 87 KiB/s wr, 29 op/s 2023-09-10T03:02:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:17 smithi170 ceph-mon[102832]: pgmap v7291: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-09-10T03:02:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:17 smithi170 ceph-mon[102832]: osdmap e524: 12 total, 12 up, 12 in 2023-09-10T03:02:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:17 smithi082 ceph-mon[98694]: pgmap v7291: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-09-10T03:02:17.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:17 smithi082 ceph-mon[98694]: osdmap e524: 12 total, 12 up, 12 in 2023-09-10T03:02:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:17 smithi098 ceph-mon[103124]: pgmap v7291: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-09-10T03:02:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:17 smithi098 ceph-mon[103124]: osdmap e524: 12 total, 12 up, 12 in 2023-09-10T03:02:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13368.847836 secs 2023-09-10T03:02:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13368.847836 secs 2023-09-10T03:02:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13368.847836 secs 2023-09-10T03:02:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13370.211155 secs 2023-09-10T03:02:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:19 smithi170 ceph-mon[102832]: pgmap v7293: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-09-10T03:02:19.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13370.922549 secs 2023-09-10T03:02:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13370.211155 secs 2023-09-10T03:02:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:19 smithi082 ceph-mon[98694]: pgmap v7293: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-09-10T03:02:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13370.922549 secs 2023-09-10T03:02:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13370.211155 secs 2023-09-10T03:02:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:19 smithi098 ceph-mon[103124]: pgmap v7293: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-09-10T03:02:19.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13370.922549 secs 2023-09-10T03:02:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:21 smithi170 ceph-mon[102832]: pgmap v7294: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 12 op/s 2023-09-10T03:02:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:21 smithi082 ceph-mon[98694]: pgmap v7294: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 12 op/s 2023-09-10T03:02:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:21 smithi098 ceph-mon[103124]: pgmap v7294: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 12 op/s 2023-09-10T03:02:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:23 smithi098 ceph-mon[103124]: pgmap v7295: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T03:02:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:02:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:23 smithi170 ceph-mon[102832]: pgmap v7295: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T03:02:23.740 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:02:23.741 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:23 smithi082 ceph-mon[98694]: pgmap v7295: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T03:02:23.741 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:02:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13375.211212 secs 2023-09-10T03:02:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:02:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:02:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:02:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13375.211212 secs 2023-09-10T03:02:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:02:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:02:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:02:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13375.211212 secs 2023-09-10T03:02:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:02:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:02:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:02:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13375.922614 secs 2023-09-10T03:02:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:25 smithi098 ceph-mon[103124]: pgmap v7296: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T03:02:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:02:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:02:25.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:02:25.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:02:25.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:02:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13375.922614 secs 2023-09-10T03:02:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:25 smithi170 ceph-mon[102832]: pgmap v7296: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T03:02:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:02:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:02:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:02:25.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:02:25.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:02:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13375.922614 secs 2023-09-10T03:02:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:25 smithi082 ceph-mon[98694]: pgmap v7296: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T03:02:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:02:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:02:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:02:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:02:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:02:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:02:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:02:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:02:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:02:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:02:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:02:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:02:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:02:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:02:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:27 smithi170 ceph-mon[102832]: pgmap v7297: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T03:02:27.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:27 smithi082 ceph-mon[98694]: pgmap v7297: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T03:02:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:27 smithi098 ceph-mon[103124]: pgmap v7297: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T03:02:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13380.211270 secs 2023-09-10T03:02:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:29 smithi170 ceph-mon[102832]: pgmap v7298: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T03:02:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13380.922678 secs 2023-09-10T03:02:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13380.211270 secs 2023-09-10T03:02:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:29 smithi082 ceph-mon[98694]: pgmap v7298: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T03:02:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13380.922678 secs 2023-09-10T03:02:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13380.211270 secs 2023-09-10T03:02:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:29 smithi098 ceph-mon[103124]: pgmap v7298: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T03:02:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13380.922678 secs 2023-09-10T03:02:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:31 smithi170 ceph-mon[102832]: pgmap v7299: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 276 KiB/s wr, 14 op/s 2023-09-10T03:02:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:31 smithi082 ceph-mon[98694]: pgmap v7299: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 276 KiB/s wr, 14 op/s 2023-09-10T03:02:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:31 smithi098 ceph-mon[103124]: pgmap v7299: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 276 KiB/s wr, 14 op/s 2023-09-10T03:02:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:33 smithi098 ceph-mon[103124]: pgmap v7300: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 256 KiB/s wr, 14 op/s 2023-09-10T03:02:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:33 smithi170 ceph-mon[102832]: pgmap v7300: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 256 KiB/s wr, 14 op/s 2023-09-10T03:02:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:33 smithi082 ceph-mon[98694]: pgmap v7300: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 256 KiB/s wr, 14 op/s 2023-09-10T03:02:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13385.211328 secs 2023-09-10T03:02:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13385.211328 secs 2023-09-10T03:02:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13385.211328 secs 2023-09-10T03:02:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13385.922742 secs 2023-09-10T03:02:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:35 smithi098 ceph-mon[103124]: pgmap v7301: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 255 KiB/s wr, 11 op/s 2023-09-10T03:02:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13385.922742 secs 2023-09-10T03:02:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:35 smithi170 ceph-mon[102832]: pgmap v7301: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 255 KiB/s wr, 11 op/s 2023-09-10T03:02:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13385.922742 secs 2023-09-10T03:02:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:35 smithi082 ceph-mon[98694]: pgmap v7301: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 255 KiB/s wr, 11 op/s 2023-09-10T03:02:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:37 smithi098 ceph-mon[103124]: pgmap v7302: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 270 KiB/s wr, 16 op/s 2023-09-10T03:02:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:37 smithi170 ceph-mon[102832]: pgmap v7302: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 270 KiB/s wr, 16 op/s 2023-09-10T03:02:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:37 smithi082 ceph-mon[98694]: pgmap v7302: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 270 KiB/s wr, 16 op/s 2023-09-10T03:02:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13388.848081 secs 2023-09-10T03:02:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13388.848081 secs 2023-09-10T03:02:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13388.848081 secs 2023-09-10T03:02:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13390.211413 secs 2023-09-10T03:02:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:39 smithi170 ceph-mon[102832]: pgmap v7303: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 229 KiB/s wr, 12 op/s 2023-09-10T03:02:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13390.922807 secs 2023-09-10T03:02:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13390.211413 secs 2023-09-10T03:02:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:39 smithi082 ceph-mon[98694]: pgmap v7303: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 229 KiB/s wr, 12 op/s 2023-09-10T03:02:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13390.922807 secs 2023-09-10T03:02:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13390.211413 secs 2023-09-10T03:02:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:39 smithi098 ceph-mon[103124]: pgmap v7303: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 229 KiB/s wr, 12 op/s 2023-09-10T03:02:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13390.922807 secs 2023-09-10T03:02:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:02:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:02:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:02:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:02:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:02:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:02:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:02:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:02:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:02:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:41 smithi170 ceph-mon[102832]: pgmap v7304: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 365 KiB/s wr, 15 op/s 2023-09-10T03:02:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:41 smithi082 ceph-mon[98694]: pgmap v7304: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 365 KiB/s wr, 15 op/s 2023-09-10T03:02:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:41 smithi098 ceph-mon[103124]: pgmap v7304: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 365 KiB/s wr, 15 op/s 2023-09-10T03:02:43.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13393.848143 secs 2023-09-10T03:02:43.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:43 smithi098 ceph-mon[103124]: pgmap v7305: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 183 KiB/s wr, 13 op/s 2023-09-10T03:02:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13393.848143 secs 2023-09-10T03:02:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:43 smithi170 ceph-mon[102832]: pgmap v7305: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 183 KiB/s wr, 13 op/s 2023-09-10T03:02:43.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13393.848143 secs 2023-09-10T03:02:43.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:43 smithi082 ceph-mon[98694]: pgmap v7305: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 183 KiB/s wr, 13 op/s 2023-09-10T03:02:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13395.211467 secs 2023-09-10T03:02:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13395.922871 secs 2023-09-10T03:02:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13395.211467 secs 2023-09-10T03:02:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13395.922871 secs 2023-09-10T03:02:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13395.211467 secs 2023-09-10T03:02:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13395.922871 secs 2023-09-10T03:02:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:45 smithi098 ceph-mon[103124]: pgmap v7306: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T03:02:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:45 smithi170 ceph-mon[102832]: pgmap v7306: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T03:02:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:45 smithi082 ceph-mon[98694]: pgmap v7306: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T03:02:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:47 smithi098 ceph-mon[103124]: pgmap v7307: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 345 KiB/s wr, 17 op/s 2023-09-10T03:02:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:47 smithi170 ceph-mon[102832]: pgmap v7307: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 345 KiB/s wr, 17 op/s 2023-09-10T03:02:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:47 smithi082 ceph-mon[98694]: pgmap v7307: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 345 KiB/s wr, 17 op/s 2023-09-10T03:02:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13398.848205 secs 2023-09-10T03:02:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13398.848205 secs 2023-09-10T03:02:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13398.848205 secs 2023-09-10T03:02:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13400.211526 secs 2023-09-10T03:02:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:49 smithi098 ceph-mon[103124]: pgmap v7308: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 330 KiB/s wr, 12 op/s 2023-09-10T03:02:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13400.922935 secs 2023-09-10T03:02:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13400.211526 secs 2023-09-10T03:02:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:49 smithi170 ceph-mon[102832]: pgmap v7308: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 330 KiB/s wr, 12 op/s 2023-09-10T03:02:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13400.922935 secs 2023-09-10T03:02:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13400.211526 secs 2023-09-10T03:02:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:49 smithi082 ceph-mon[98694]: pgmap v7308: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 330 KiB/s wr, 12 op/s 2023-09-10T03:02:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13400.922935 secs 2023-09-10T03:02:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:51 smithi170 ceph-mon[102832]: pgmap v7309: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 361 KiB/s wr, 15 op/s 2023-09-10T03:02:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:51 smithi082 ceph-mon[98694]: pgmap v7309: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 361 KiB/s wr, 15 op/s 2023-09-10T03:02:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:51 smithi098 ceph-mon[103124]: pgmap v7309: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 361 KiB/s wr, 15 op/s 2023-09-10T03:02:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13403.848267 secs 2023-09-10T03:02:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:53 smithi098 ceph-mon[103124]: pgmap v7310: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 225 KiB/s wr, 13 op/s 2023-09-10T03:02:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13403.848267 secs 2023-09-10T03:02:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:53 smithi170 ceph-mon[102832]: pgmap v7310: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 225 KiB/s wr, 13 op/s 2023-09-10T03:02:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13403.848267 secs 2023-09-10T03:02:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:53 smithi082 ceph-mon[98694]: pgmap v7310: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 225 KiB/s wr, 13 op/s 2023-09-10T03:02:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13405.211583 secs 2023-09-10T03:02:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13405.923002 secs 2023-09-10T03:02:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13405.211583 secs 2023-09-10T03:02:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13405.923002 secs 2023-09-10T03:02:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13405.211583 secs 2023-09-10T03:02:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13405.923002 secs 2023-09-10T03:02:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:55 smithi098 ceph-mon[103124]: pgmap v7311: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 214 KiB/s wr, 11 op/s 2023-09-10T03:02:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:55 smithi170 ceph-mon[102832]: pgmap v7311: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 214 KiB/s wr, 11 op/s 2023-09-10T03:02:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:55 smithi082 ceph-mon[98694]: pgmap v7311: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 214 KiB/s wr, 11 op/s 2023-09-10T03:02:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:57 smithi098 ceph-mon[103124]: pgmap v7312: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 268 KiB/s wr, 15 op/s 2023-09-10T03:02:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:57 smithi170 ceph-mon[102832]: pgmap v7312: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 268 KiB/s wr, 15 op/s 2023-09-10T03:02:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:57 smithi082 ceph-mon[98694]: pgmap v7312: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 268 KiB/s wr, 15 op/s 2023-09-10T03:02:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13408.848386 secs 2023-09-10T03:02:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13408.848386 secs 2023-09-10T03:02:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13408.848386 secs 2023-09-10T03:02:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13410.211640 secs 2023-09-10T03:02:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:59 smithi170 ceph-mon[102832]: pgmap v7313: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 9 op/s 2023-09-10T03:02:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:02:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13410.923068 secs 2023-09-10T03:02:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13410.211640 secs 2023-09-10T03:02:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:59 smithi082 ceph-mon[98694]: pgmap v7313: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 9 op/s 2023-09-10T03:02:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:02:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13410.923068 secs 2023-09-10T03:02:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13410.211640 secs 2023-09-10T03:02:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:59 smithi098 ceph-mon[103124]: pgmap v7313: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 9 op/s 2023-09-10T03:02:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:02:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13410.923068 secs 2023-09-10T03:03:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:01 smithi170 ceph-mon[102832]: pgmap v7314: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T03:03:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:01 smithi082 ceph-mon[98694]: pgmap v7314: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T03:03:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:01 smithi098 ceph-mon[103124]: pgmap v7314: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T03:03:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13413.848473 secs 2023-09-10T03:03:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:03 smithi098 ceph-mon[103124]: pgmap v7315: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T03:03:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13413.848473 secs 2023-09-10T03:03:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:03 smithi170 ceph-mon[102832]: pgmap v7315: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T03:03:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13413.848473 secs 2023-09-10T03:03:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:03 smithi082 ceph-mon[98694]: pgmap v7315: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T03:03:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13415.211697 secs 2023-09-10T03:03:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13415.923136 secs 2023-09-10T03:03:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13415.211697 secs 2023-09-10T03:03:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13415.923136 secs 2023-09-10T03:03:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13415.211697 secs 2023-09-10T03:03:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13415.923136 secs 2023-09-10T03:03:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:05 smithi098 ceph-mon[103124]: pgmap v7316: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 9 op/s 2023-09-10T03:03:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:05 smithi170 ceph-mon[102832]: pgmap v7316: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 9 op/s 2023-09-10T03:03:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:05 smithi082 ceph-mon[98694]: pgmap v7316: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 9 op/s 2023-09-10T03:03:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:07 smithi098 ceph-mon[103124]: pgmap v7317: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 147 KiB/s wr, 29 op/s 2023-09-10T03:03:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:07 smithi170 ceph-mon[102832]: pgmap v7317: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 147 KiB/s wr, 29 op/s 2023-09-10T03:03:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:07 smithi082 ceph-mon[98694]: pgmap v7317: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 147 KiB/s wr, 29 op/s 2023-09-10T03:03:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13418.848533 secs 2023-09-10T03:03:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:08 smithi170 ceph-mon[102832]: osdmap e525: 12 total, 12 up, 12 in 2023-09-10T03:03:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13418.848533 secs 2023-09-10T03:03:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:08 smithi082 ceph-mon[98694]: osdmap e525: 12 total, 12 up, 12 in 2023-09-10T03:03:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13418.848533 secs 2023-09-10T03:03:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:08 smithi098 ceph-mon[103124]: osdmap e525: 12 total, 12 up, 12 in 2023-09-10T03:03:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13420.211757 secs 2023-09-10T03:03:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:09 smithi170 ceph-mon[102832]: pgmap v7319: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 112 KiB/s wr, 29 op/s 2023-09-10T03:03:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13420.923200 secs 2023-09-10T03:03:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13420.211757 secs 2023-09-10T03:03:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:09 smithi082 ceph-mon[98694]: pgmap v7319: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 112 KiB/s wr, 29 op/s 2023-09-10T03:03:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13420.923200 secs 2023-09-10T03:03:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13420.211757 secs 2023-09-10T03:03:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:09 smithi098 ceph-mon[103124]: pgmap v7319: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 112 KiB/s wr, 29 op/s 2023-09-10T03:03:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13420.923200 secs 2023-09-10T03:03:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:11 smithi170 ceph-mon[102832]: pgmap v7320: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 111 KiB/s wr, 29 op/s 2023-09-10T03:03:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:11 smithi082 ceph-mon[98694]: pgmap v7320: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 111 KiB/s wr, 29 op/s 2023-09-10T03:03:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:11 smithi098 ceph-mon[103124]: pgmap v7320: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 111 KiB/s wr, 29 op/s 2023-09-10T03:03:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13423.848595 secs 2023-09-10T03:03:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:13 smithi098 ceph-mon[103124]: pgmap v7321: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T03:03:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13423.848595 secs 2023-09-10T03:03:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:13 smithi170 ceph-mon[102832]: pgmap v7321: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T03:03:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13423.848595 secs 2023-09-10T03:03:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:13 smithi082 ceph-mon[98694]: pgmap v7321: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T03:03:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13425.211816 secs 2023-09-10T03:03:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13425.923266 secs 2023-09-10T03:03:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13425.211816 secs 2023-09-10T03:03:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13425.923266 secs 2023-09-10T03:03:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13425.211816 secs 2023-09-10T03:03:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13425.923266 secs 2023-09-10T03:03:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:15 smithi098 ceph-mon[103124]: pgmap v7322: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T03:03:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:15 smithi170 ceph-mon[102832]: pgmap v7322: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T03:03:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:15 smithi082 ceph-mon[98694]: pgmap v7322: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T03:03:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:17 smithi170 ceph-mon[102832]: pgmap v7323: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T03:03:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:17 smithi170 ceph-mon[102832]: osdmap e526: 12 total, 12 up, 12 in 2023-09-10T03:03:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:17 smithi082 ceph-mon[98694]: pgmap v7323: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T03:03:17.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:17 smithi082 ceph-mon[98694]: osdmap e526: 12 total, 12 up, 12 in 2023-09-10T03:03:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:17 smithi098 ceph-mon[103124]: pgmap v7323: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T03:03:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:17 smithi098 ceph-mon[103124]: osdmap e526: 12 total, 12 up, 12 in 2023-09-10T03:03:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13428.848652 secs 2023-09-10T03:03:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13428.848652 secs 2023-09-10T03:03:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13428.848652 secs 2023-09-10T03:03:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13430.211900 secs 2023-09-10T03:03:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:19 smithi170 ceph-mon[102832]: pgmap v7325: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T03:03:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13430.923330 secs 2023-09-10T03:03:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13430.211900 secs 2023-09-10T03:03:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:19 smithi082 ceph-mon[98694]: pgmap v7325: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T03:03:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13430.923330 secs 2023-09-10T03:03:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13430.211900 secs 2023-09-10T03:03:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:19 smithi098 ceph-mon[103124]: pgmap v7325: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T03:03:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13430.923330 secs 2023-09-10T03:03:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:21 smithi170 ceph-mon[102832]: pgmap v7326: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T03:03:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:21 smithi082 ceph-mon[98694]: pgmap v7326: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T03:03:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:21 smithi098 ceph-mon[103124]: pgmap v7326: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T03:03:23.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:23 smithi098 ceph-mon[103124]: pgmap v7327: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T03:03:23.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:03:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:23 smithi170 ceph-mon[102832]: pgmap v7327: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T03:03:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:03:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:23 smithi082 ceph-mon[98694]: pgmap v7327: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T03:03:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:03:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13435.211956 secs 2023-09-10T03:03:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:03:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:03:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:03:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13435.923397 secs 2023-09-10T03:03:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13435.211956 secs 2023-09-10T03:03:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:03:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:03:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:03:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13435.923397 secs 2023-09-10T03:03:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13435.211956 secs 2023-09-10T03:03:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:03:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:03:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:03:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13435.923397 secs 2023-09-10T03:03:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:25 smithi098 ceph-mon[103124]: pgmap v7328: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T03:03:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:25 smithi170 ceph-mon[102832]: pgmap v7328: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T03:03:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:25 smithi082 ceph-mon[98694]: pgmap v7328: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 11 op/s 2023-09-10T03:03:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:03:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:03:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:03:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:03:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:03:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:03:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:03:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:03:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:03:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:03:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:03:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:03:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:27 smithi098 ceph-mon[103124]: pgmap v7329: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T03:03:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:27 smithi170 ceph-mon[102832]: pgmap v7329: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T03:03:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:27 smithi082 ceph-mon[98694]: pgmap v7329: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T03:03:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13440.212015 secs 2023-09-10T03:03:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:29 smithi170 ceph-mon[102832]: pgmap v7330: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T03:03:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13440.923462 secs 2023-09-10T03:03:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13440.212015 secs 2023-09-10T03:03:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:29 smithi082 ceph-mon[98694]: pgmap v7330: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T03:03:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13440.923462 secs 2023-09-10T03:03:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13440.212015 secs 2023-09-10T03:03:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:29 smithi098 ceph-mon[103124]: pgmap v7330: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T03:03:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13440.923462 secs 2023-09-10T03:03:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:31 smithi170 ceph-mon[102832]: pgmap v7331: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 13 op/s 2023-09-10T03:03:31.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:31 smithi082 ceph-mon[98694]: pgmap v7331: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 13 op/s 2023-09-10T03:03:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:31 smithi098 ceph-mon[103124]: pgmap v7331: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 13 op/s 2023-09-10T03:03:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:33 smithi098 ceph-mon[103124]: pgmap v7332: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 12 op/s 2023-09-10T03:03:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:33 smithi170 ceph-mon[102832]: pgmap v7332: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 12 op/s 2023-09-10T03:03:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:33 smithi082 ceph-mon[98694]: pgmap v7332: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 12 op/s 2023-09-10T03:03:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13445.212070 secs 2023-09-10T03:03:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13445.923528 secs 2023-09-10T03:03:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13445.212070 secs 2023-09-10T03:03:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13445.923528 secs 2023-09-10T03:03:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13445.212070 secs 2023-09-10T03:03:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13445.923528 secs 2023-09-10T03:03:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:35 smithi170 ceph-mon[102832]: pgmap v7333: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T03:03:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:35 smithi082 ceph-mon[98694]: pgmap v7333: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T03:03:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:35 smithi098 ceph-mon[103124]: pgmap v7333: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T03:03:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:37 smithi170 ceph-mon[102832]: pgmap v7334: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 182 KiB/s wr, 17 op/s 2023-09-10T03:03:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:37 smithi082 ceph-mon[98694]: pgmap v7334: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 182 KiB/s wr, 17 op/s 2023-09-10T03:03:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:37 smithi098 ceph-mon[103124]: pgmap v7334: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 182 KiB/s wr, 17 op/s 2023-09-10T03:03:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13450.212126 secs 2023-09-10T03:03:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:39 smithi170 ceph-mon[102832]: pgmap v7335: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T03:03:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13450.923593 secs 2023-09-10T03:03:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13450.212126 secs 2023-09-10T03:03:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:39 smithi082 ceph-mon[98694]: pgmap v7335: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T03:03:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13450.923593 secs 2023-09-10T03:03:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13450.212126 secs 2023-09-10T03:03:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:39 smithi098 ceph-mon[103124]: pgmap v7335: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T03:03:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13450.923593 secs 2023-09-10T03:03:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:41 smithi170 ceph-mon[102832]: pgmap v7336: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 269 KiB/s wr, 16 op/s 2023-09-10T03:03:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:41 smithi082 ceph-mon[98694]: pgmap v7336: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 269 KiB/s wr, 16 op/s 2023-09-10T03:03:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:41 smithi098 ceph-mon[103124]: pgmap v7336: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 269 KiB/s wr, 16 op/s 2023-09-10T03:03:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:43 smithi098 ceph-mon[103124]: pgmap v7337: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 201 KiB/s wr, 15 op/s 2023-09-10T03:03:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:43 smithi170 ceph-mon[102832]: pgmap v7337: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 201 KiB/s wr, 15 op/s 2023-09-10T03:03:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:43 smithi082 ceph-mon[98694]: pgmap v7337: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 201 KiB/s wr, 15 op/s 2023-09-10T03:03:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13455.212207 secs 2023-09-10T03:03:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13455.923656 secs 2023-09-10T03:03:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13455.212207 secs 2023-09-10T03:03:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13455.923656 secs 2023-09-10T03:03:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13455.212207 secs 2023-09-10T03:03:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13455.923656 secs 2023-09-10T03:03:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:45 smithi170 ceph-mon[102832]: pgmap v7338: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 190 KiB/s wr, 12 op/s 2023-09-10T03:03:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:45 smithi082 ceph-mon[98694]: pgmap v7338: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 190 KiB/s wr, 12 op/s 2023-09-10T03:03:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:45 smithi098 ceph-mon[103124]: pgmap v7338: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 190 KiB/s wr, 12 op/s 2023-09-10T03:03:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:47 smithi170 ceph-mon[102832]: pgmap v7339: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 241 KiB/s wr, 18 op/s 2023-09-10T03:03:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:47 smithi082 ceph-mon[98694]: pgmap v7339: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 241 KiB/s wr, 18 op/s 2023-09-10T03:03:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:47 smithi098 ceph-mon[103124]: pgmap v7339: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 241 KiB/s wr, 18 op/s 2023-09-10T03:03:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13460.212294 secs 2023-09-10T03:03:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:49 smithi170 ceph-mon[102832]: pgmap v7340: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 211 KiB/s wr, 11 op/s 2023-09-10T03:03:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13460.923724 secs 2023-09-10T03:03:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13460.212294 secs 2023-09-10T03:03:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:49 smithi082 ceph-mon[98694]: pgmap v7340: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 211 KiB/s wr, 11 op/s 2023-09-10T03:03:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13460.923724 secs 2023-09-10T03:03:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13460.212294 secs 2023-09-10T03:03:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:49 smithi098 ceph-mon[103124]: pgmap v7340: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 211 KiB/s wr, 11 op/s 2023-09-10T03:03:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13460.923724 secs 2023-09-10T03:03:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:51 smithi170 ceph-mon[102832]: pgmap v7341: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 415 KiB/s wr, 16 op/s 2023-09-10T03:03:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:51 smithi082 ceph-mon[98694]: pgmap v7341: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 415 KiB/s wr, 16 op/s 2023-09-10T03:03:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:51 smithi098 ceph-mon[103124]: pgmap v7341: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 415 KiB/s wr, 16 op/s 2023-09-10T03:03:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13463.849305 secs 2023-09-10T03:03:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:53 smithi098 ceph-mon[103124]: pgmap v7342: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 286 KiB/s wr, 14 op/s 2023-09-10T03:03:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13463.849305 secs 2023-09-10T03:03:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:53 smithi170 ceph-mon[102832]: pgmap v7342: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 286 KiB/s wr, 14 op/s 2023-09-10T03:03:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13463.849305 secs 2023-09-10T03:03:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:53 smithi082 ceph-mon[98694]: pgmap v7342: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 286 KiB/s wr, 14 op/s 2023-09-10T03:03:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13465.212353 secs 2023-09-10T03:03:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13465.923790 secs 2023-09-10T03:03:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13465.212353 secs 2023-09-10T03:03:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13465.923790 secs 2023-09-10T03:03:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13465.212353 secs 2023-09-10T03:03:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13465.923790 secs 2023-09-10T03:03:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:55 smithi170 ceph-mon[102832]: pgmap v7343: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 276 KiB/s wr, 11 op/s 2023-09-10T03:03:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:55 smithi082 ceph-mon[98694]: pgmap v7343: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 276 KiB/s wr, 11 op/s 2023-09-10T03:03:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:55 smithi098 ceph-mon[103124]: pgmap v7343: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 276 KiB/s wr, 11 op/s 2023-09-10T03:03:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:03:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:03:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:03:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:03:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:03:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:03:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:03:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:03:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:03:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:57 smithi170 ceph-mon[102832]: pgmap v7344: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 308 KiB/s wr, 17 op/s 2023-09-10T03:03:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:57 smithi082 ceph-mon[98694]: pgmap v7344: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 308 KiB/s wr, 17 op/s 2023-09-10T03:03:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:57 smithi098 ceph-mon[103124]: pgmap v7344: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 308 KiB/s wr, 17 op/s 2023-09-10T03:03:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13468.849394 secs 2023-09-10T03:03:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13468.849394 secs 2023-09-10T03:03:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13468.849394 secs 2023-09-10T03:03:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13470.212411 secs 2023-09-10T03:03:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:59 smithi170 ceph-mon[102832]: pgmap v7345: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 257 KiB/s wr, 11 op/s 2023-09-10T03:03:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:03:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13470.923855 secs 2023-09-10T03:03:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13470.212411 secs 2023-09-10T03:03:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:59 smithi082 ceph-mon[98694]: pgmap v7345: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 257 KiB/s wr, 11 op/s 2023-09-10T03:03:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:03:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13470.923855 secs 2023-09-10T03:03:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13470.212411 secs 2023-09-10T03:03:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:59 smithi098 ceph-mon[103124]: pgmap v7345: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 257 KiB/s wr, 11 op/s 2023-09-10T03:03:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:03:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13470.923855 secs 2023-09-10T03:04:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:01 smithi170 ceph-mon[102832]: pgmap v7346: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 312 KiB/s wr, 15 op/s 2023-09-10T03:04:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:01 smithi082 ceph-mon[98694]: pgmap v7346: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 312 KiB/s wr, 15 op/s 2023-09-10T03:04:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:01 smithi098 ceph-mon[103124]: pgmap v7346: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 312 KiB/s wr, 15 op/s 2023-09-10T03:04:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13473.849501 secs 2023-09-10T03:04:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:03 smithi170 ceph-mon[102832]: pgmap v7347: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 162 KiB/s wr, 27 op/s 2023-09-10T03:04:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13473.849501 secs 2023-09-10T03:04:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:03 smithi082 ceph-mon[98694]: pgmap v7347: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 162 KiB/s wr, 27 op/s 2023-09-10T03:04:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13473.849501 secs 2023-09-10T03:04:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:03 smithi098 ceph-mon[103124]: pgmap v7347: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 162 KiB/s wr, 27 op/s 2023-09-10T03:04:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13475.212493 secs 2023-09-10T03:04:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13475.923920 secs 2023-09-10T03:04:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13475.212493 secs 2023-09-10T03:04:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13475.923920 secs 2023-09-10T03:04:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13475.212493 secs 2023-09-10T03:04:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13475.923920 secs 2023-09-10T03:04:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:05 smithi170 ceph-mon[102832]: pgmap v7348: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 141 KiB/s wr, 24 op/s 2023-09-10T03:04:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:05 smithi082 ceph-mon[98694]: pgmap v7348: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 141 KiB/s wr, 24 op/s 2023-09-10T03:04:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:05 smithi098 ceph-mon[103124]: pgmap v7348: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 141 KiB/s wr, 24 op/s 2023-09-10T03:04:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:07 smithi170 ceph-mon[102832]: pgmap v7349: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 169 KiB/s wr, 29 op/s 2023-09-10T03:04:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:07 smithi082 ceph-mon[98694]: pgmap v7349: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 169 KiB/s wr, 29 op/s 2023-09-10T03:04:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:07 smithi098 ceph-mon[103124]: pgmap v7349: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 169 KiB/s wr, 29 op/s 2023-09-10T03:04:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13478.849558 secs 2023-09-10T03:04:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:08 smithi170 ceph-mon[102832]: osdmap e527: 12 total, 12 up, 12 in 2023-09-10T03:04:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13478.849558 secs 2023-09-10T03:04:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:08 smithi082 ceph-mon[98694]: osdmap e527: 12 total, 12 up, 12 in 2023-09-10T03:04:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13478.849558 secs 2023-09-10T03:04:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:08 smithi098 ceph-mon[103124]: osdmap e527: 12 total, 12 up, 12 in 2023-09-10T03:04:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13480.212554 secs 2023-09-10T03:04:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:09 smithi170 ceph-mon[102832]: pgmap v7351: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 164 KiB/s wr, 29 op/s 2023-09-10T03:04:09.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13480.923986 secs 2023-09-10T03:04:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13480.212554 secs 2023-09-10T03:04:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:09 smithi082 ceph-mon[98694]: pgmap v7351: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 164 KiB/s wr, 29 op/s 2023-09-10T03:04:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13480.923986 secs 2023-09-10T03:04:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13480.212554 secs 2023-09-10T03:04:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:09 smithi098 ceph-mon[103124]: pgmap v7351: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 164 KiB/s wr, 29 op/s 2023-09-10T03:04:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13480.923986 secs 2023-09-10T03:04:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:11 smithi170 ceph-mon[102832]: pgmap v7352: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 98 KiB/s wr, 28 op/s 2023-09-10T03:04:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:11 smithi082 ceph-mon[98694]: pgmap v7352: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 98 KiB/s wr, 28 op/s 2023-09-10T03:04:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:11 smithi098 ceph-mon[103124]: pgmap v7352: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 98 KiB/s wr, 28 op/s 2023-09-10T03:04:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13483.849618 secs 2023-09-10T03:04:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:13 smithi170 ceph-mon[102832]: pgmap v7353: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T03:04:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13483.849618 secs 2023-09-10T03:04:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:13 smithi082 ceph-mon[98694]: pgmap v7353: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T03:04:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13483.849618 secs 2023-09-10T03:04:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:13 smithi098 ceph-mon[103124]: pgmap v7353: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T03:04:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13485.212610 secs 2023-09-10T03:04:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13485.924053 secs 2023-09-10T03:04:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13485.212610 secs 2023-09-10T03:04:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13485.924053 secs 2023-09-10T03:04:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13485.212610 secs 2023-09-10T03:04:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13485.924053 secs 2023-09-10T03:04:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:15 smithi170 ceph-mon[102832]: pgmap v7354: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T03:04:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:15 smithi082 ceph-mon[98694]: pgmap v7354: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T03:04:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:15 smithi098 ceph-mon[103124]: pgmap v7354: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 KiB/s wr, 11 op/s 2023-09-10T03:04:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:17 smithi170 ceph-mon[102832]: pgmap v7355: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T03:04:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:17 smithi170 ceph-mon[102832]: osdmap e528: 12 total, 12 up, 12 in 2023-09-10T03:04:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:17 smithi082 ceph-mon[98694]: pgmap v7355: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T03:04:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:17 smithi082 ceph-mon[98694]: osdmap e528: 12 total, 12 up, 12 in 2023-09-10T03:04:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:17 smithi098 ceph-mon[103124]: pgmap v7355: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T03:04:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:17 smithi098 ceph-mon[103124]: osdmap e528: 12 total, 12 up, 12 in 2023-09-10T03:04:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13488.849693 secs 2023-09-10T03:04:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13488.849693 secs 2023-09-10T03:04:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13488.849693 secs 2023-09-10T03:04:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13490.212670 secs 2023-09-10T03:04:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:19 smithi170 ceph-mon[102832]: pgmap v7357: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T03:04:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13490.924120 secs 2023-09-10T03:04:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13490.212670 secs 2023-09-10T03:04:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:19 smithi082 ceph-mon[98694]: pgmap v7357: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T03:04:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13490.924120 secs 2023-09-10T03:04:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13490.212670 secs 2023-09-10T03:04:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:19 smithi098 ceph-mon[103124]: pgmap v7357: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T03:04:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13490.924120 secs 2023-09-10T03:04:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:21 smithi170 ceph-mon[102832]: pgmap v7358: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T03:04:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:21 smithi082 ceph-mon[98694]: pgmap v7358: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T03:04:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:21 smithi098 ceph-mon[103124]: pgmap v7358: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T03:04:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:23 smithi170 ceph-mon[102832]: pgmap v7359: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:04:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:04:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:23 smithi082 ceph-mon[98694]: pgmap v7359: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:04:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:04:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:23 smithi098 ceph-mon[103124]: pgmap v7359: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:04:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:04:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13495.212727 secs 2023-09-10T03:04:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:04:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:04:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:04:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13495.924187 secs 2023-09-10T03:04:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13495.212727 secs 2023-09-10T03:04:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:04:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:04:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:04:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13495.924187 secs 2023-09-10T03:04:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13495.212727 secs 2023-09-10T03:04:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:04:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:04:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:04:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13495.924187 secs 2023-09-10T03:04:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:25 smithi170 ceph-mon[102832]: pgmap v7360: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:04:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:25 smithi082 ceph-mon[98694]: pgmap v7360: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:04:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:25 smithi098 ceph-mon[103124]: pgmap v7360: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:04:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:04:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:04:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:04:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:27 smithi170 ceph-mon[102832]: pgmap v7361: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 11 op/s 2023-09-10T03:04:27.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:04:27.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:04:27.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:04:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:27 smithi082 ceph-mon[98694]: pgmap v7361: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 11 op/s 2023-09-10T03:04:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:04:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:04:27.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:04:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:27 smithi098 ceph-mon[103124]: pgmap v7361: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 11 op/s 2023-09-10T03:04:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:04:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:04:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:04:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13500.212820 secs 2023-09-10T03:04:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:29 smithi170 ceph-mon[102832]: pgmap v7362: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T03:04:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13500.924252 secs 2023-09-10T03:04:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13500.212820 secs 2023-09-10T03:04:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:29 smithi082 ceph-mon[98694]: pgmap v7362: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T03:04:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13500.924252 secs 2023-09-10T03:04:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13500.212820 secs 2023-09-10T03:04:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:29 smithi098 ceph-mon[103124]: pgmap v7362: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T03:04:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13500.924252 secs 2023-09-10T03:04:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:31 smithi170 ceph-mon[102832]: pgmap v7363: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 281 KiB/s wr, 14 op/s 2023-09-10T03:04:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:31 smithi082 ceph-mon[98694]: pgmap v7363: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 281 KiB/s wr, 14 op/s 2023-09-10T03:04:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:31 smithi098 ceph-mon[103124]: pgmap v7363: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 281 KiB/s wr, 14 op/s 2023-09-10T03:04:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:33 smithi170 ceph-mon[102832]: pgmap v7364: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 260 KiB/s wr, 14 op/s 2023-09-10T03:04:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:33 smithi082 ceph-mon[98694]: pgmap v7364: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 260 KiB/s wr, 14 op/s 2023-09-10T03:04:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:33 smithi098 ceph-mon[103124]: pgmap v7364: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 260 KiB/s wr, 14 op/s 2023-09-10T03:04:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13505.212872 secs 2023-09-10T03:04:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13505.924316 secs 2023-09-10T03:04:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13505.212872 secs 2023-09-10T03:04:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13505.924316 secs 2023-09-10T03:04:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13505.212872 secs 2023-09-10T03:04:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13505.924316 secs 2023-09-10T03:04:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:35 smithi170 ceph-mon[102832]: pgmap v7365: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 259 KiB/s wr, 12 op/s 2023-09-10T03:04:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:35 smithi082 ceph-mon[98694]: pgmap v7365: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 259 KiB/s wr, 12 op/s 2023-09-10T03:04:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:35 smithi098 ceph-mon[103124]: pgmap v7365: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 259 KiB/s wr, 12 op/s 2023-09-10T03:04:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:37 smithi170 ceph-mon[102832]: pgmap v7366: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 274 KiB/s wr, 17 op/s 2023-09-10T03:04:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:37 smithi082 ceph-mon[98694]: pgmap v7366: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 274 KiB/s wr, 17 op/s 2023-09-10T03:04:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:37 smithi098 ceph-mon[103124]: pgmap v7366: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 274 KiB/s wr, 17 op/s 2023-09-10T03:04:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13510.212931 secs 2023-09-10T03:04:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:39 smithi170 ceph-mon[102832]: pgmap v7367: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 232 KiB/s wr, 12 op/s 2023-09-10T03:04:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13510.924381 secs 2023-09-10T03:04:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13510.212931 secs 2023-09-10T03:04:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:39 smithi082 ceph-mon[98694]: pgmap v7367: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 232 KiB/s wr, 12 op/s 2023-09-10T03:04:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13510.924381 secs 2023-09-10T03:04:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13510.212931 secs 2023-09-10T03:04:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:39 smithi098 ceph-mon[103124]: pgmap v7367: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 232 KiB/s wr, 12 op/s 2023-09-10T03:04:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13510.924381 secs 2023-09-10T03:04:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:41 smithi170 ceph-mon[102832]: pgmap v7368: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 371 KiB/s wr, 15 op/s 2023-09-10T03:04:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:41 smithi082 ceph-mon[98694]: pgmap v7368: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 371 KiB/s wr, 15 op/s 2023-09-10T03:04:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:41 smithi098 ceph-mon[103124]: pgmap v7368: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 371 KiB/s wr, 15 op/s 2023-09-10T03:04:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:43 smithi170 ceph-mon[102832]: pgmap v7369: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 13 op/s 2023-09-10T03:04:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:43 smithi082 ceph-mon[98694]: pgmap v7369: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 13 op/s 2023-09-10T03:04:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:43 smithi098 ceph-mon[103124]: pgmap v7369: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 13 op/s 2023-09-10T03:04:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13515.212987 secs 2023-09-10T03:04:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13515.924449 secs 2023-09-10T03:04:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13515.212987 secs 2023-09-10T03:04:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13515.924449 secs 2023-09-10T03:04:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13515.212987 secs 2023-09-10T03:04:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13515.924449 secs 2023-09-10T03:04:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:45 smithi170 ceph-mon[102832]: pgmap v7370: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 10 op/s 2023-09-10T03:04:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:45 smithi082 ceph-mon[98694]: pgmap v7370: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 10 op/s 2023-09-10T03:04:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:45 smithi098 ceph-mon[103124]: pgmap v7370: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 10 op/s 2023-09-10T03:04:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:47 smithi170 ceph-mon[102832]: pgmap v7371: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 318 KiB/s wr, 17 op/s 2023-09-10T03:04:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:47 smithi082 ceph-mon[98694]: pgmap v7371: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 318 KiB/s wr, 17 op/s 2023-09-10T03:04:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:47 smithi098 ceph-mon[103124]: pgmap v7371: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 318 KiB/s wr, 17 op/s 2023-09-10T03:04:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13518.850240 secs 2023-09-10T03:04:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13518.850240 secs 2023-09-10T03:04:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13518.850240 secs 2023-09-10T03:04:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13520.213044 secs 2023-09-10T03:04:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:49 smithi170 ceph-mon[102832]: pgmap v7372: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 302 KiB/s wr, 12 op/s 2023-09-10T03:04:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13520.924488 secs 2023-09-10T03:04:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13520.213044 secs 2023-09-10T03:04:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:49 smithi082 ceph-mon[98694]: pgmap v7372: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 302 KiB/s wr, 12 op/s 2023-09-10T03:04:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13520.924488 secs 2023-09-10T03:04:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13520.213044 secs 2023-09-10T03:04:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:49 smithi098 ceph-mon[103124]: pgmap v7372: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 302 KiB/s wr, 12 op/s 2023-09-10T03:04:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13520.924488 secs 2023-09-10T03:04:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:51 smithi170 ceph-mon[102832]: pgmap v7373: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 303 KiB/s wr, 14 op/s 2023-09-10T03:04:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:51 smithi082 ceph-mon[98694]: pgmap v7373: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 303 KiB/s wr, 14 op/s 2023-09-10T03:04:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:51 smithi098 ceph-mon[103124]: pgmap v7373: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 303 KiB/s wr, 14 op/s 2023-09-10T03:04:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:04:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:04:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:04:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:04:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:04:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:04:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:04:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:04:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:04:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13523.850299 secs 2023-09-10T03:04:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:53 smithi170 ceph-mon[102832]: pgmap v7374: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 187 KiB/s wr, 13 op/s 2023-09-10T03:04:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13523.850299 secs 2023-09-10T03:04:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:53 smithi082 ceph-mon[98694]: pgmap v7374: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 187 KiB/s wr, 13 op/s 2023-09-10T03:04:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13523.850299 secs 2023-09-10T03:04:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:53 smithi098 ceph-mon[103124]: pgmap v7374: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 187 KiB/s wr, 13 op/s 2023-09-10T03:04:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13525.213101 secs 2023-09-10T03:04:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13525.924552 secs 2023-09-10T03:04:54.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13525.213101 secs 2023-09-10T03:04:54.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13525.924552 secs 2023-09-10T03:04:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13525.213101 secs 2023-09-10T03:04:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13525.924552 secs 2023-09-10T03:04:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:55 smithi170 ceph-mon[102832]: pgmap v7375: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 176 KiB/s wr, 11 op/s 2023-09-10T03:04:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:55 smithi082 ceph-mon[98694]: pgmap v7375: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 176 KiB/s wr, 11 op/s 2023-09-10T03:04:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:55 smithi098 ceph-mon[103124]: pgmap v7375: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 176 KiB/s wr, 11 op/s 2023-09-10T03:04:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:57 smithi170 ceph-mon[102832]: pgmap v7376: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 268 KiB/s wr, 16 op/s 2023-09-10T03:04:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:57 smithi082 ceph-mon[98694]: pgmap v7376: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 268 KiB/s wr, 16 op/s 2023-09-10T03:04:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:57 smithi098 ceph-mon[103124]: pgmap v7376: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 268 KiB/s wr, 16 op/s 2023-09-10T03:04:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13528.850377 secs 2023-09-10T03:04:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13528.850377 secs 2023-09-10T03:04:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13528.850377 secs 2023-09-10T03:04:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13530.213158 secs 2023-09-10T03:04:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:59 smithi170 ceph-mon[102832]: pgmap v7377: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 9 op/s 2023-09-10T03:04:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:04:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13530.924618 secs 2023-09-10T03:04:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13530.213158 secs 2023-09-10T03:04:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:59 smithi082 ceph-mon[98694]: pgmap v7377: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 9 op/s 2023-09-10T03:04:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:04:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13530.924618 secs 2023-09-10T03:04:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13530.213158 secs 2023-09-10T03:04:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:59 smithi098 ceph-mon[103124]: pgmap v7377: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 9 op/s 2023-09-10T03:04:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:04:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13530.924618 secs 2023-09-10T03:05:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:01 smithi170 ceph-mon[102832]: pgmap v7378: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T03:05:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:01 smithi082 ceph-mon[98694]: pgmap v7378: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T03:05:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:01 smithi098 ceph-mon[103124]: pgmap v7378: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T03:05:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13533.850438 secs 2023-09-10T03:05:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:03 smithi170 ceph-mon[102832]: pgmap v7379: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 197 KiB/s wr, 27 op/s 2023-09-10T03:05:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13533.850438 secs 2023-09-10T03:05:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:03 smithi082 ceph-mon[98694]: pgmap v7379: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 197 KiB/s wr, 27 op/s 2023-09-10T03:05:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13533.850438 secs 2023-09-10T03:05:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:03 smithi098 ceph-mon[103124]: pgmap v7379: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 197 KiB/s wr, 27 op/s 2023-09-10T03:05:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13535.213215 secs 2023-09-10T03:05:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13535.924684 secs 2023-09-10T03:05:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13535.213215 secs 2023-09-10T03:05:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13535.924684 secs 2023-09-10T03:05:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13535.213215 secs 2023-09-10T03:05:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13535.924684 secs 2023-09-10T03:05:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:05 smithi170 ceph-mon[102832]: pgmap v7380: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 174 KiB/s wr, 24 op/s 2023-09-10T03:05:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:05 smithi082 ceph-mon[98694]: pgmap v7380: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 174 KiB/s wr, 24 op/s 2023-09-10T03:05:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:05 smithi098 ceph-mon[103124]: pgmap v7380: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 174 KiB/s wr, 24 op/s 2023-09-10T03:05:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:07 smithi170 ceph-mon[102832]: pgmap v7381: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 185 KiB/s wr, 29 op/s 2023-09-10T03:05:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:07 smithi082 ceph-mon[98694]: pgmap v7381: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 185 KiB/s wr, 29 op/s 2023-09-10T03:05:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:07 smithi098 ceph-mon[103124]: pgmap v7381: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 185 KiB/s wr, 29 op/s 2023-09-10T03:05:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13538.850498 secs 2023-09-10T03:05:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:08 smithi170 ceph-mon[102832]: osdmap e529: 12 total, 12 up, 12 in 2023-09-10T03:05:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13538.850498 secs 2023-09-10T03:05:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:08 smithi082 ceph-mon[98694]: osdmap e529: 12 total, 12 up, 12 in 2023-09-10T03:05:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13538.850498 secs 2023-09-10T03:05:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:08 smithi098 ceph-mon[103124]: osdmap e529: 12 total, 12 up, 12 in 2023-09-10T03:05:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13540.213272 secs 2023-09-10T03:05:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:09 smithi170 ceph-mon[102832]: pgmap v7383: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 112 KiB/s wr, 29 op/s 2023-09-10T03:05:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13540.924757 secs 2023-09-10T03:05:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13540.213272 secs 2023-09-10T03:05:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:09 smithi082 ceph-mon[98694]: pgmap v7383: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 112 KiB/s wr, 29 op/s 2023-09-10T03:05:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13540.924757 secs 2023-09-10T03:05:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13540.213272 secs 2023-09-10T03:05:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:09 smithi098 ceph-mon[103124]: pgmap v7383: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 112 KiB/s wr, 29 op/s 2023-09-10T03:05:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13540.924757 secs 2023-09-10T03:05:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:11 smithi170 ceph-mon[102832]: pgmap v7384: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 112 KiB/s wr, 29 op/s 2023-09-10T03:05:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:11 smithi082 ceph-mon[98694]: pgmap v7384: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 112 KiB/s wr, 29 op/s 2023-09-10T03:05:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:11 smithi098 ceph-mon[103124]: pgmap v7384: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 112 KiB/s wr, 29 op/s 2023-09-10T03:05:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13543.850559 secs 2023-09-10T03:05:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:13 smithi170 ceph-mon[102832]: pgmap v7385: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T03:05:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13543.850559 secs 2023-09-10T03:05:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:13 smithi082 ceph-mon[98694]: pgmap v7385: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T03:05:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13543.850559 secs 2023-09-10T03:05:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:13 smithi098 ceph-mon[103124]: pgmap v7385: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T03:05:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13545.213332 secs 2023-09-10T03:05:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13545.924823 secs 2023-09-10T03:05:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13545.213332 secs 2023-09-10T03:05:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13545.924823 secs 2023-09-10T03:05:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13545.213332 secs 2023-09-10T03:05:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13545.924823 secs 2023-09-10T03:05:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:15 smithi170 ceph-mon[102832]: pgmap v7386: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T03:05:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:15 smithi082 ceph-mon[98694]: pgmap v7386: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T03:05:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:15 smithi098 ceph-mon[103124]: pgmap v7386: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T03:05:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:17 smithi170 ceph-mon[102832]: pgmap v7387: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:05:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:17 smithi170 ceph-mon[102832]: osdmap e530: 12 total, 12 up, 12 in 2023-09-10T03:05:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:17 smithi082 ceph-mon[98694]: pgmap v7387: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:05:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:17 smithi082 ceph-mon[98694]: osdmap e530: 12 total, 12 up, 12 in 2023-09-10T03:05:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:17 smithi098 ceph-mon[103124]: pgmap v7387: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:05:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:17 smithi098 ceph-mon[103124]: osdmap e530: 12 total, 12 up, 12 in 2023-09-10T03:05:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13548.850621 secs 2023-09-10T03:05:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13548.850621 secs 2023-09-10T03:05:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13548.850621 secs 2023-09-10T03:05:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13550.213432 secs 2023-09-10T03:05:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:19 smithi170 ceph-mon[102832]: pgmap v7389: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:05:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13550.924888 secs 2023-09-10T03:05:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13550.213432 secs 2023-09-10T03:05:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:19 smithi082 ceph-mon[98694]: pgmap v7389: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:05:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13550.924888 secs 2023-09-10T03:05:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13550.213432 secs 2023-09-10T03:05:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:19 smithi098 ceph-mon[103124]: pgmap v7389: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:05:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13550.924888 secs 2023-09-10T03:05:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:21 smithi170 ceph-mon[102832]: pgmap v7390: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:05:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:21 smithi082 ceph-mon[98694]: pgmap v7390: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:05:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:21 smithi098 ceph-mon[103124]: pgmap v7390: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:05:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:23 smithi170 ceph-mon[102832]: pgmap v7391: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:05:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:05:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:23 smithi082 ceph-mon[98694]: pgmap v7391: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:05:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:05:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:23 smithi098 ceph-mon[103124]: pgmap v7391: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:05:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:05:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13555.213521 secs 2023-09-10T03:05:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:05:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:05:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:05:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13555.924955 secs 2023-09-10T03:05:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13555.213521 secs 2023-09-10T03:05:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:05:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:05:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:05:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13555.924955 secs 2023-09-10T03:05:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13555.213521 secs 2023-09-10T03:05:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:05:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:05:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:05:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13555.924955 secs 2023-09-10T03:05:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:25 smithi170 ceph-mon[102832]: pgmap v7392: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:05:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:25 smithi082 ceph-mon[98694]: pgmap v7392: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:05:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:25 smithi098 ceph-mon[103124]: pgmap v7392: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:05:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:27 smithi170 ceph-mon[102832]: pgmap v7393: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T03:05:27.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:05:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:27 smithi082 ceph-mon[98694]: pgmap v7393: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T03:05:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:05:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:27 smithi098 ceph-mon[103124]: pgmap v7393: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T03:05:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:05:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:05:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:05:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:05:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:05:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:05:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:05:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:05:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:05:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:05:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13560.213587 secs 2023-09-10T03:05:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:29 smithi170 ceph-mon[102832]: pgmap v7394: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T03:05:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13560.925022 secs 2023-09-10T03:05:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13560.213587 secs 2023-09-10T03:05:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:29 smithi082 ceph-mon[98694]: pgmap v7394: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T03:05:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13560.925022 secs 2023-09-10T03:05:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13560.213587 secs 2023-09-10T03:05:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:29 smithi098 ceph-mon[103124]: pgmap v7394: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T03:05:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13560.925022 secs 2023-09-10T03:05:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:31 smithi170 ceph-mon[102832]: pgmap v7395: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 143 KiB/s wr, 13 op/s 2023-09-10T03:05:31.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:31 smithi082 ceph-mon[98694]: pgmap v7395: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 143 KiB/s wr, 13 op/s 2023-09-10T03:05:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:31 smithi098 ceph-mon[103124]: pgmap v7395: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 143 KiB/s wr, 13 op/s 2023-09-10T03:05:33.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:33 smithi170 ceph-mon[102832]: pgmap v7396: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 154 KiB/s wr, 13 op/s 2023-09-10T03:05:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:33 smithi082 ceph-mon[98694]: pgmap v7396: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 154 KiB/s wr, 13 op/s 2023-09-10T03:05:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:33 smithi098 ceph-mon[103124]: pgmap v7396: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 154 KiB/s wr, 13 op/s 2023-09-10T03:05:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13565.213643 secs 2023-09-10T03:05:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13565.925087 secs 2023-09-10T03:05:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13565.213643 secs 2023-09-10T03:05:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13565.925087 secs 2023-09-10T03:05:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13565.213643 secs 2023-09-10T03:05:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13565.925087 secs 2023-09-10T03:05:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:35 smithi170 ceph-mon[102832]: pgmap v7397: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T03:05:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:35 smithi082 ceph-mon[98694]: pgmap v7397: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T03:05:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:35 smithi098 ceph-mon[103124]: pgmap v7397: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T03:05:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:37 smithi170 ceph-mon[102832]: pgmap v7398: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 217 KiB/s wr, 18 op/s 2023-09-10T03:05:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:37 smithi082 ceph-mon[98694]: pgmap v7398: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 217 KiB/s wr, 18 op/s 2023-09-10T03:05:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:37 smithi098 ceph-mon[103124]: pgmap v7398: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 217 KiB/s wr, 18 op/s 2023-09-10T03:05:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13570.213700 secs 2023-09-10T03:05:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:39 smithi170 ceph-mon[102832]: pgmap v7399: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 134 KiB/s wr, 12 op/s 2023-09-10T03:05:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13570.925150 secs 2023-09-10T03:05:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13570.213700 secs 2023-09-10T03:05:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:39 smithi082 ceph-mon[98694]: pgmap v7399: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 134 KiB/s wr, 12 op/s 2023-09-10T03:05:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13570.925150 secs 2023-09-10T03:05:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13570.213700 secs 2023-09-10T03:05:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:39 smithi098 ceph-mon[103124]: pgmap v7399: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 134 KiB/s wr, 12 op/s 2023-09-10T03:05:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13570.925150 secs 2023-09-10T03:05:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:41 smithi170 ceph-mon[102832]: pgmap v7400: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 253 KiB/s wr, 16 op/s 2023-09-10T03:05:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:41 smithi082 ceph-mon[98694]: pgmap v7400: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 253 KiB/s wr, 16 op/s 2023-09-10T03:05:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:41 smithi098 ceph-mon[103124]: pgmap v7400: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 253 KiB/s wr, 16 op/s 2023-09-10T03:05:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:43 smithi170 ceph-mon[102832]: pgmap v7401: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 204 KiB/s wr, 15 op/s 2023-09-10T03:05:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:43 smithi082 ceph-mon[98694]: pgmap v7401: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 204 KiB/s wr, 15 op/s 2023-09-10T03:05:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:43 smithi098 ceph-mon[103124]: pgmap v7401: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 204 KiB/s wr, 15 op/s 2023-09-10T03:05:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13575.213764 secs 2023-09-10T03:05:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13575.925217 secs 2023-09-10T03:05:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13575.213764 secs 2023-09-10T03:05:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13575.925217 secs 2023-09-10T03:05:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13575.213764 secs 2023-09-10T03:05:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13575.925217 secs 2023-09-10T03:05:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:45 smithi170 ceph-mon[102832]: pgmap v7402: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 193 KiB/s wr, 12 op/s 2023-09-10T03:05:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:45 smithi082 ceph-mon[98694]: pgmap v7402: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 193 KiB/s wr, 12 op/s 2023-09-10T03:05:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:45 smithi098 ceph-mon[103124]: pgmap v7402: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 193 KiB/s wr, 12 op/s 2023-09-10T03:05:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:47 smithi170 ceph-mon[102832]: pgmap v7403: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 245 KiB/s wr, 18 op/s 2023-09-10T03:05:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:47 smithi082 ceph-mon[98694]: pgmap v7403: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 245 KiB/s wr, 18 op/s 2023-09-10T03:05:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:47 smithi098 ceph-mon[103124]: pgmap v7403: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 245 KiB/s wr, 18 op/s 2023-09-10T03:05:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13580.213821 secs 2023-09-10T03:05:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:49 smithi170 ceph-mon[102832]: pgmap v7404: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 182 KiB/s wr, 11 op/s 2023-09-10T03:05:49.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13580.925281 secs 2023-09-10T03:05:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13580.213821 secs 2023-09-10T03:05:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:49 smithi082 ceph-mon[98694]: pgmap v7404: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 182 KiB/s wr, 11 op/s 2023-09-10T03:05:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13580.925281 secs 2023-09-10T03:05:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13580.213821 secs 2023-09-10T03:05:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:49 smithi098 ceph-mon[103124]: pgmap v7404: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 182 KiB/s wr, 11 op/s 2023-09-10T03:05:49.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13580.925281 secs 2023-09-10T03:05:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:51 smithi170 ceph-mon[102832]: pgmap v7405: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 280 KiB/s wr, 15 op/s 2023-09-10T03:05:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:51 smithi082 ceph-mon[98694]: pgmap v7405: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 280 KiB/s wr, 15 op/s 2023-09-10T03:05:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:51 smithi098 ceph-mon[103124]: pgmap v7405: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 280 KiB/s wr, 15 op/s 2023-09-10T03:05:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13583.851152 secs 2023-09-10T03:05:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:53 smithi170 ceph-mon[102832]: pgmap v7406: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 182 KiB/s wr, 14 op/s 2023-09-10T03:05:53.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13583.851152 secs 2023-09-10T03:05:53.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:53 smithi082 ceph-mon[98694]: pgmap v7406: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 182 KiB/s wr, 14 op/s 2023-09-10T03:05:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13583.851152 secs 2023-09-10T03:05:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:53 smithi098 ceph-mon[103124]: pgmap v7406: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 182 KiB/s wr, 14 op/s 2023-09-10T03:05:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13585.213881 secs 2023-09-10T03:05:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13585.925346 secs 2023-09-10T03:05:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13585.213881 secs 2023-09-10T03:05:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13585.925346 secs 2023-09-10T03:05:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13585.213881 secs 2023-09-10T03:05:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13585.925346 secs 2023-09-10T03:05:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:55 smithi170 ceph-mon[102832]: pgmap v7407: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 171 KiB/s wr, 12 op/s 2023-09-10T03:05:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:55 smithi082 ceph-mon[98694]: pgmap v7407: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 171 KiB/s wr, 12 op/s 2023-09-10T03:05:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:55 smithi098 ceph-mon[103124]: pgmap v7407: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 171 KiB/s wr, 12 op/s 2023-09-10T03:05:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:05:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:05:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:05:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:05:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:05:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:05:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:05:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:05:56.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:05:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:57 smithi170 ceph-mon[102832]: pgmap v7408: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 243 KiB/s wr, 17 op/s 2023-09-10T03:05:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:57 smithi082 ceph-mon[98694]: pgmap v7408: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 243 KiB/s wr, 17 op/s 2023-09-10T03:05:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:57 smithi098 ceph-mon[103124]: pgmap v7408: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 243 KiB/s wr, 17 op/s 2023-09-10T03:05:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13588.851242 secs 2023-09-10T03:05:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13588.851242 secs 2023-09-10T03:05:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13588.851242 secs 2023-09-10T03:05:59.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13590.213942 secs 2023-09-10T03:05:59.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:59 smithi170 ceph-mon[102832]: pgmap v7409: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T03:05:59.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:05:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13590.925411 secs 2023-09-10T03:05:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13590.213942 secs 2023-09-10T03:05:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:59 smithi082 ceph-mon[98694]: pgmap v7409: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T03:05:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:05:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13590.925411 secs 2023-09-10T03:05:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13590.213942 secs 2023-09-10T03:05:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:59 smithi098 ceph-mon[103124]: pgmap v7409: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T03:05:59.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:05:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13590.925411 secs 2023-09-10T03:06:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:01 smithi170 ceph-mon[102832]: pgmap v7410: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 246 KiB/s wr, 15 op/s 2023-09-10T03:06:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:01 smithi082 ceph-mon[98694]: pgmap v7410: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 246 KiB/s wr, 15 op/s 2023-09-10T03:06:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:01 smithi098 ceph-mon[103124]: pgmap v7410: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 246 KiB/s wr, 15 op/s 2023-09-10T03:06:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13593.851313 secs 2023-09-10T03:06:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:03 smithi170 ceph-mon[102832]: pgmap v7411: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 179 KiB/s wr, 24 op/s 2023-09-10T03:06:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13593.851313 secs 2023-09-10T03:06:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:03 smithi082 ceph-mon[98694]: pgmap v7411: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 179 KiB/s wr, 24 op/s 2023-09-10T03:06:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13593.851313 secs 2023-09-10T03:06:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:03 smithi098 ceph-mon[103124]: pgmap v7411: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 179 KiB/s wr, 24 op/s 2023-09-10T03:06:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13595.213998 secs 2023-09-10T03:06:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13595.925483 secs 2023-09-10T03:06:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13595.213998 secs 2023-09-10T03:06:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13595.925483 secs 2023-09-10T03:06:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13595.213998 secs 2023-09-10T03:06:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13595.925483 secs 2023-09-10T03:06:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:05 smithi170 ceph-mon[102832]: pgmap v7412: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 159 KiB/s wr, 21 op/s 2023-09-10T03:06:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:05 smithi082 ceph-mon[98694]: pgmap v7412: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 159 KiB/s wr, 21 op/s 2023-09-10T03:06:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:05 smithi098 ceph-mon[103124]: pgmap v7412: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 159 KiB/s wr, 21 op/s 2023-09-10T03:06:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:07 smithi170 ceph-mon[102832]: pgmap v7413: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 210 KiB/s wr, 29 op/s 2023-09-10T03:06:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:07 smithi082 ceph-mon[98694]: pgmap v7413: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 210 KiB/s wr, 29 op/s 2023-09-10T03:06:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:07 smithi098 ceph-mon[103124]: pgmap v7413: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 210 KiB/s wr, 29 op/s 2023-09-10T03:06:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13598.851374 secs 2023-09-10T03:06:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:08 smithi170 ceph-mon[102832]: osdmap e531: 12 total, 12 up, 12 in 2023-09-10T03:06:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13598.851374 secs 2023-09-10T03:06:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:08 smithi082 ceph-mon[98694]: osdmap e531: 12 total, 12 up, 12 in 2023-09-10T03:06:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13598.851374 secs 2023-09-10T03:06:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:08 smithi098 ceph-mon[103124]: osdmap e531: 12 total, 12 up, 12 in 2023-09-10T03:06:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13600.214056 secs 2023-09-10T03:06:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:09 smithi170 ceph-mon[102832]: pgmap v7415: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 166 KiB/s wr, 29 op/s 2023-09-10T03:06:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13600.925550 secs 2023-09-10T03:06:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13600.214056 secs 2023-09-10T03:06:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:09 smithi082 ceph-mon[98694]: pgmap v7415: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 166 KiB/s wr, 29 op/s 2023-09-10T03:06:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13600.925550 secs 2023-09-10T03:06:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13600.214056 secs 2023-09-10T03:06:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:09 smithi098 ceph-mon[103124]: pgmap v7415: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 166 KiB/s wr, 29 op/s 2023-09-10T03:06:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13600.925550 secs 2023-09-10T03:06:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:11 smithi170 ceph-mon[102832]: pgmap v7416: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T03:06:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:11 smithi082 ceph-mon[98694]: pgmap v7416: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T03:06:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:11 smithi098 ceph-mon[103124]: pgmap v7416: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T03:06:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13603.851447 secs 2023-09-10T03:06:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:13 smithi170 ceph-mon[102832]: pgmap v7417: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 75 KiB/s wr, 15 op/s 2023-09-10T03:06:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13603.851447 secs 2023-09-10T03:06:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:13 smithi082 ceph-mon[98694]: pgmap v7417: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 75 KiB/s wr, 15 op/s 2023-09-10T03:06:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13603.851447 secs 2023-09-10T03:06:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:13 smithi098 ceph-mon[103124]: pgmap v7417: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 75 KiB/s wr, 15 op/s 2023-09-10T03:06:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13605.214113 secs 2023-09-10T03:06:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13605.925617 secs 2023-09-10T03:06:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13605.214113 secs 2023-09-10T03:06:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13605.925617 secs 2023-09-10T03:06:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13605.214113 secs 2023-09-10T03:06:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13605.925617 secs 2023-09-10T03:06:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:15 smithi170 ceph-mon[102832]: pgmap v7418: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 75 KiB/s wr, 15 op/s 2023-09-10T03:06:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:15 smithi082 ceph-mon[98694]: pgmap v7418: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 75 KiB/s wr, 15 op/s 2023-09-10T03:06:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:15 smithi098 ceph-mon[103124]: pgmap v7418: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 75 KiB/s wr, 15 op/s 2023-09-10T03:06:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:17 smithi170 ceph-mon[102832]: pgmap v7419: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-09-10T03:06:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:17 smithi170 ceph-mon[102832]: osdmap e532: 12 total, 12 up, 12 in 2023-09-10T03:06:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:17 smithi082 ceph-mon[98694]: pgmap v7419: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-09-10T03:06:17.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:17 smithi082 ceph-mon[98694]: osdmap e532: 12 total, 12 up, 12 in 2023-09-10T03:06:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:17 smithi098 ceph-mon[103124]: pgmap v7419: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-09-10T03:06:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:17 smithi098 ceph-mon[103124]: osdmap e532: 12 total, 12 up, 12 in 2023-09-10T03:06:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13608.851535 secs 2023-09-10T03:06:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13608.851535 secs 2023-09-10T03:06:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13608.851535 secs 2023-09-10T03:06:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13610.214171 secs 2023-09-10T03:06:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:19 smithi170 ceph-mon[102832]: pgmap v7421: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-09-10T03:06:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13610.925684 secs 2023-09-10T03:06:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13610.214171 secs 2023-09-10T03:06:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:19 smithi082 ceph-mon[98694]: pgmap v7421: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-09-10T03:06:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13610.925684 secs 2023-09-10T03:06:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13610.214171 secs 2023-09-10T03:06:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:19 smithi098 ceph-mon[103124]: pgmap v7421: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-09-10T03:06:19.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13610.925684 secs 2023-09-10T03:06:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:21 smithi170 ceph-mon[102832]: pgmap v7422: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T03:06:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:21 smithi082 ceph-mon[98694]: pgmap v7422: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T03:06:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:21 smithi098 ceph-mon[103124]: pgmap v7422: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T03:06:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:23 smithi170 ceph-mon[102832]: pgmap v7423: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T03:06:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:06:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:23 smithi082 ceph-mon[98694]: pgmap v7423: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T03:06:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:06:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:23 smithi098 ceph-mon[103124]: pgmap v7423: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T03:06:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:06:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13615.214322 secs 2023-09-10T03:06:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:06:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:06:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:06:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13615.925749 secs 2023-09-10T03:06:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13615.214322 secs 2023-09-10T03:06:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:06:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:06:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:06:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13615.925749 secs 2023-09-10T03:06:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13615.214322 secs 2023-09-10T03:06:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:06:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:06:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:06:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13615.925749 secs 2023-09-10T03:06:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:25 smithi170 ceph-mon[102832]: pgmap v7424: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T03:06:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:25 smithi082 ceph-mon[98694]: pgmap v7424: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T03:06:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:25 smithi098 ceph-mon[103124]: pgmap v7424: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T03:06:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:27 smithi170 ceph-mon[102832]: pgmap v7425: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T03:06:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:27 smithi082 ceph-mon[98694]: pgmap v7425: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T03:06:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:27 smithi098 ceph-mon[103124]: pgmap v7425: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T03:06:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:06:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:06:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:06:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:06:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:06:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:06:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:06:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:06:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:06:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:06:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:06:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:06:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13620.214380 secs 2023-09-10T03:06:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:29 smithi170 ceph-mon[102832]: pgmap v7426: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 10 op/s 2023-09-10T03:06:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13620.925814 secs 2023-09-10T03:06:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13620.214380 secs 2023-09-10T03:06:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:29 smithi082 ceph-mon[98694]: pgmap v7426: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 10 op/s 2023-09-10T03:06:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13620.925814 secs 2023-09-10T03:06:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13620.214380 secs 2023-09-10T03:06:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:29 smithi098 ceph-mon[103124]: pgmap v7426: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 10 op/s 2023-09-10T03:06:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13620.925814 secs 2023-09-10T03:06:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:31 smithi170 ceph-mon[102832]: pgmap v7427: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T03:06:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:31 smithi082 ceph-mon[98694]: pgmap v7427: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T03:06:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:31 smithi098 ceph-mon[103124]: pgmap v7427: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T03:06:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:33 smithi170 ceph-mon[102832]: pgmap v7428: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 263 KiB/s wr, 13 op/s 2023-09-10T03:06:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:33 smithi082 ceph-mon[98694]: pgmap v7428: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 263 KiB/s wr, 13 op/s 2023-09-10T03:06:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:33 smithi098 ceph-mon[103124]: pgmap v7428: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 263 KiB/s wr, 13 op/s 2023-09-10T03:06:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13625.214436 secs 2023-09-10T03:06:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13625.925882 secs 2023-09-10T03:06:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13625.214436 secs 2023-09-10T03:06:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13625.925882 secs 2023-09-10T03:06:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13625.214436 secs 2023-09-10T03:06:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13625.925882 secs 2023-09-10T03:06:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:35 smithi170 ceph-mon[102832]: pgmap v7429: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 263 KiB/s wr, 11 op/s 2023-09-10T03:06:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:35 smithi082 ceph-mon[98694]: pgmap v7429: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 263 KiB/s wr, 11 op/s 2023-09-10T03:06:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:35 smithi098 ceph-mon[103124]: pgmap v7429: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 263 KiB/s wr, 11 op/s 2023-09-10T03:06:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:37 smithi170 ceph-mon[102832]: pgmap v7430: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 316 KiB/s wr, 17 op/s 2023-09-10T03:06:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:37 smithi082 ceph-mon[98694]: pgmap v7430: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 316 KiB/s wr, 17 op/s 2023-09-10T03:06:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:37 smithi098 ceph-mon[103124]: pgmap v7430: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 316 KiB/s wr, 17 op/s 2023-09-10T03:06:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13630.214496 secs 2023-09-10T03:06:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:39 smithi170 ceph-mon[102832]: pgmap v7431: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 273 KiB/s wr, 12 op/s 2023-09-10T03:06:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13630.925946 secs 2023-09-10T03:06:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13630.214496 secs 2023-09-10T03:06:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:39 smithi082 ceph-mon[98694]: pgmap v7431: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 273 KiB/s wr, 12 op/s 2023-09-10T03:06:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13630.925946 secs 2023-09-10T03:06:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13630.214496 secs 2023-09-10T03:06:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:39 smithi098 ceph-mon[103124]: pgmap v7431: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 273 KiB/s wr, 12 op/s 2023-09-10T03:06:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13630.925946 secs 2023-09-10T03:06:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:41 smithi170 ceph-mon[102832]: pgmap v7432: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 282 KiB/s wr, 15 op/s 2023-09-10T03:06:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:41 smithi082 ceph-mon[98694]: pgmap v7432: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 282 KiB/s wr, 15 op/s 2023-09-10T03:06:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:41 smithi098 ceph-mon[103124]: pgmap v7432: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 282 KiB/s wr, 15 op/s 2023-09-10T03:06:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:43 smithi170 ceph-mon[102832]: pgmap v7433: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 337 KiB/s wr, 15 op/s 2023-09-10T03:06:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:43 smithi082 ceph-mon[98694]: pgmap v7433: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 337 KiB/s wr, 15 op/s 2023-09-10T03:06:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:43 smithi098 ceph-mon[103124]: pgmap v7433: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 337 KiB/s wr, 15 op/s 2023-09-10T03:06:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13635.214553 secs 2023-09-10T03:06:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:44 smithi170 ceph-mon[102832]: Health check failed: 1 MDSs behind on trimming (MDS_TRIM) 2023-09-10T03:06:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:06:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:06:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13635.926013 secs 2023-09-10T03:06:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13635.214553 secs 2023-09-10T03:06:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:44 smithi082 ceph-mon[98694]: Health check failed: 1 MDSs behind on trimming (MDS_TRIM) 2023-09-10T03:06:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:06:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:06:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13635.926013 secs 2023-09-10T03:06:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13635.214553 secs 2023-09-10T03:06:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:44 smithi098 ceph-mon[103124]: Health check failed: 1 MDSs behind on trimming (MDS_TRIM) 2023-09-10T03:06:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:06:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:06:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13635.926013 secs 2023-09-10T03:06:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:45 smithi170 ceph-mon[102832]: pgmap v7434: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 182 KiB/s wr, 11 op/s 2023-09-10T03:06:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:45 smithi082 ceph-mon[98694]: pgmap v7434: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 182 KiB/s wr, 11 op/s 2023-09-10T03:06:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:45 smithi098 ceph-mon[103124]: pgmap v7434: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 182 KiB/s wr, 11 op/s 2023-09-10T03:06:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:06:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:06:46.717 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:06:46.718 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:06:46.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:06:46.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:06:47.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:47 smithi170 ceph-mon[102832]: pgmap v7435: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 323 KiB/s wr, 18 op/s 2023-09-10T03:06:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:47 smithi082 ceph-mon[98694]: pgmap v7435: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 323 KiB/s wr, 18 op/s 2023-09-10T03:06:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:47 smithi098 ceph-mon[103124]: pgmap v7435: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 323 KiB/s wr, 18 op/s 2023-09-10T03:06:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13638.852013 secs 2023-09-10T03:06:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13638.852013 secs 2023-09-10T03:06:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13638.852013 secs 2023-09-10T03:06:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13640.214641 secs 2023-09-10T03:06:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:49 smithi170 ceph-mon[102832]: pgmap v7436: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 270 KiB/s wr, 13 op/s 2023-09-10T03:06:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13640.926081 secs 2023-09-10T03:06:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13640.214641 secs 2023-09-10T03:06:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:49 smithi082 ceph-mon[98694]: pgmap v7436: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 270 KiB/s wr, 13 op/s 2023-09-10T03:06:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13640.926081 secs 2023-09-10T03:06:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13640.214641 secs 2023-09-10T03:06:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:49 smithi098 ceph-mon[103124]: pgmap v7436: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 270 KiB/s wr, 13 op/s 2023-09-10T03:06:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13640.926081 secs 2023-09-10T03:06:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:51 smithi170 ceph-mon[102832]: pgmap v7437: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 293 KiB/s wr, 16 op/s 2023-09-10T03:06:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:51 smithi082 ceph-mon[98694]: pgmap v7437: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 293 KiB/s wr, 16 op/s 2023-09-10T03:06:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:51 smithi098 ceph-mon[103124]: pgmap v7437: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 293 KiB/s wr, 16 op/s 2023-09-10T03:06:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:06:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:06:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:06:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:52 smithi170 ceph-mon[102832]: Health check update: 2 MDSs behind on trimming (MDS_TRIM) 2023-09-10T03:06:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:06:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:06:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:06:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:52 smithi082 ceph-mon[98694]: Health check update: 2 MDSs behind on trimming (MDS_TRIM) 2023-09-10T03:06:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:06:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:06:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:06:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:52 smithi098 ceph-mon[103124]: Health check update: 2 MDSs behind on trimming (MDS_TRIM) 2023-09-10T03:06:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13643.852073 secs 2023-09-10T03:06:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:53 smithi170 ceph-mon[102832]: pgmap v7438: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 285 KiB/s wr, 15 op/s 2023-09-10T03:06:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13643.852073 secs 2023-09-10T03:06:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:53 smithi082 ceph-mon[98694]: pgmap v7438: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 285 KiB/s wr, 15 op/s 2023-09-10T03:06:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13643.852073 secs 2023-09-10T03:06:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:53 smithi098 ceph-mon[103124]: pgmap v7438: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 285 KiB/s wr, 15 op/s 2023-09-10T03:06:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13645.214722 secs 2023-09-10T03:06:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13645.926145 secs 2023-09-10T03:06:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13645.214722 secs 2023-09-10T03:06:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13645.926145 secs 2023-09-10T03:06:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13645.214722 secs 2023-09-10T03:06:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13645.926145 secs 2023-09-10T03:06:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:55 smithi170 ceph-mon[102832]: pgmap v7439: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 164 KiB/s wr, 11 op/s 2023-09-10T03:06:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:55 smithi082 ceph-mon[98694]: pgmap v7439: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 164 KiB/s wr, 11 op/s 2023-09-10T03:06:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:55 smithi098 ceph-mon[103124]: pgmap v7439: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 164 KiB/s wr, 11 op/s 2023-09-10T03:06:57.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:57 smithi170 ceph-mon[102832]: pgmap v7440: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 252 KiB/s wr, 16 op/s 2023-09-10T03:06:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:57 smithi082 ceph-mon[98694]: pgmap v7440: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 252 KiB/s wr, 16 op/s 2023-09-10T03:06:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:57 smithi098 ceph-mon[103124]: pgmap v7440: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 252 KiB/s wr, 16 op/s 2023-09-10T03:06:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13648.852132 secs 2023-09-10T03:06:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13648.852132 secs 2023-09-10T03:06:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13648.852132 secs 2023-09-10T03:06:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13650.214780 secs 2023-09-10T03:06:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:59 smithi170 ceph-mon[102832]: pgmap v7441: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 9 op/s 2023-09-10T03:06:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:06:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13650.926215 secs 2023-09-10T03:06:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13650.214780 secs 2023-09-10T03:06:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:59 smithi082 ceph-mon[98694]: pgmap v7441: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 9 op/s 2023-09-10T03:06:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:06:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13650.926215 secs 2023-09-10T03:06:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13650.214780 secs 2023-09-10T03:06:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:59 smithi098 ceph-mon[103124]: pgmap v7441: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 9 op/s 2023-09-10T03:06:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:06:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13650.926215 secs 2023-09-10T03:07:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:01 smithi170 ceph-mon[102832]: pgmap v7442: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 149 KiB/s wr, 13 op/s 2023-09-10T03:07:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:01 smithi082 ceph-mon[98694]: pgmap v7442: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 149 KiB/s wr, 13 op/s 2023-09-10T03:07:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:01 smithi098 ceph-mon[103124]: pgmap v7442: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 149 KiB/s wr, 13 op/s 2023-09-10T03:07:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13653.852237 secs 2023-09-10T03:07:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:03 smithi170 ceph-mon[102832]: pgmap v7443: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T03:07:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13653.852237 secs 2023-09-10T03:07:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:03 smithi082 ceph-mon[98694]: pgmap v7443: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T03:07:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13653.852237 secs 2023-09-10T03:07:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:03 smithi098 ceph-mon[103124]: pgmap v7443: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T03:07:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13655.214837 secs 2023-09-10T03:07:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13655.926279 secs 2023-09-10T03:07:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13655.214837 secs 2023-09-10T03:07:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13655.926279 secs 2023-09-10T03:07:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13655.214837 secs 2023-09-10T03:07:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13655.926279 secs 2023-09-10T03:07:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:05 smithi170 ceph-mon[102832]: pgmap v7444: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 10 op/s 2023-09-10T03:07:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:05 smithi082 ceph-mon[98694]: pgmap v7444: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 10 op/s 2023-09-10T03:07:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:05 smithi098 ceph-mon[103124]: pgmap v7444: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 10 op/s 2023-09-10T03:07:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:07 smithi170 ceph-mon[102832]: pgmap v7445: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 182 KiB/s wr, 29 op/s 2023-09-10T03:07:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:07 smithi082 ceph-mon[98694]: pgmap v7445: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 182 KiB/s wr, 29 op/s 2023-09-10T03:07:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:07 smithi098 ceph-mon[103124]: pgmap v7445: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 182 KiB/s wr, 29 op/s 2023-09-10T03:07:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13658.852342 secs 2023-09-10T03:07:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:08 smithi170 ceph-mon[102832]: osdmap e533: 12 total, 12 up, 12 in 2023-09-10T03:07:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13658.852342 secs 2023-09-10T03:07:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:08 smithi082 ceph-mon[98694]: osdmap e533: 12 total, 12 up, 12 in 2023-09-10T03:07:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13658.852342 secs 2023-09-10T03:07:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:08 smithi098 ceph-mon[103124]: osdmap e533: 12 total, 12 up, 12 in 2023-09-10T03:07:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13660.214941 secs 2023-09-10T03:07:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:09 smithi170 ceph-mon[102832]: pgmap v7447: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 113 KiB/s wr, 29 op/s 2023-09-10T03:07:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13660.926345 secs 2023-09-10T03:07:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13660.214941 secs 2023-09-10T03:07:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:09 smithi082 ceph-mon[98694]: pgmap v7447: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 113 KiB/s wr, 29 op/s 2023-09-10T03:07:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13660.926345 secs 2023-09-10T03:07:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13660.214941 secs 2023-09-10T03:07:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:09 smithi098 ceph-mon[103124]: pgmap v7447: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 113 KiB/s wr, 29 op/s 2023-09-10T03:07:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13660.926345 secs 2023-09-10T03:07:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:11 smithi170 ceph-mon[102832]: pgmap v7448: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T03:07:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:11 smithi082 ceph-mon[98694]: pgmap v7448: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T03:07:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:11 smithi098 ceph-mon[103124]: pgmap v7448: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 67 KiB/s wr, 29 op/s 2023-09-10T03:07:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13663.852482 secs 2023-09-10T03:07:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:13 smithi170 ceph-mon[102832]: pgmap v7449: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 28 op/s 2023-09-10T03:07:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13663.852482 secs 2023-09-10T03:07:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:13 smithi082 ceph-mon[98694]: pgmap v7449: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 28 op/s 2023-09-10T03:07:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13663.852482 secs 2023-09-10T03:07:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:13 smithi098 ceph-mon[103124]: pgmap v7449: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 28 op/s 2023-09-10T03:07:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13665.214999 secs 2023-09-10T03:07:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13665.926411 secs 2023-09-10T03:07:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13665.214999 secs 2023-09-10T03:07:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13665.926411 secs 2023-09-10T03:07:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13665.214999 secs 2023-09-10T03:07:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13665.926411 secs 2023-09-10T03:07:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:15 smithi170 ceph-mon[102832]: pgmap v7450: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 28 op/s 2023-09-10T03:07:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:15 smithi082 ceph-mon[98694]: pgmap v7450: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 28 op/s 2023-09-10T03:07:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:15 smithi098 ceph-mon[103124]: pgmap v7450: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 67 KiB/s wr, 28 op/s 2023-09-10T03:07:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:17 smithi170 ceph-mon[102832]: pgmap v7451: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T03:07:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:17 smithi170 ceph-mon[102832]: osdmap e534: 12 total, 12 up, 12 in 2023-09-10T03:07:17.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:17 smithi082 ceph-mon[98694]: pgmap v7451: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T03:07:17.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:17 smithi082 ceph-mon[98694]: osdmap e534: 12 total, 12 up, 12 in 2023-09-10T03:07:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:17 smithi098 ceph-mon[103124]: pgmap v7451: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T03:07:17.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:17 smithi098 ceph-mon[103124]: osdmap e534: 12 total, 12 up, 12 in 2023-09-10T03:07:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13668.852544 secs 2023-09-10T03:07:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13668.852544 secs 2023-09-10T03:07:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13668.852544 secs 2023-09-10T03:07:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13670.215083 secs 2023-09-10T03:07:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:19 smithi170 ceph-mon[102832]: pgmap v7453: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T03:07:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13670.926483 secs 2023-09-10T03:07:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13670.215083 secs 2023-09-10T03:07:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:19 smithi082 ceph-mon[98694]: pgmap v7453: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T03:07:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13670.926483 secs 2023-09-10T03:07:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13670.215083 secs 2023-09-10T03:07:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:19 smithi098 ceph-mon[103124]: pgmap v7453: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T03:07:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13670.926483 secs 2023-09-10T03:07:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:21 smithi170 ceph-mon[102832]: pgmap v7454: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:07:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:21 smithi082 ceph-mon[98694]: pgmap v7454: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:07:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:21 smithi098 ceph-mon[103124]: pgmap v7454: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:07:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:23 smithi170 ceph-mon[102832]: pgmap v7455: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:07:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:07:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:23 smithi082 ceph-mon[98694]: pgmap v7455: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:07:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:07:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:23 smithi098 ceph-mon[103124]: pgmap v7455: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:07:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:07:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13675.215141 secs 2023-09-10T03:07:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:07:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:07:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:07:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13675.926548 secs 2023-09-10T03:07:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13675.215141 secs 2023-09-10T03:07:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:07:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:07:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:07:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13675.926548 secs 2023-09-10T03:07:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13675.215141 secs 2023-09-10T03:07:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:07:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:07:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:07:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13675.926548 secs 2023-09-10T03:07:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:25 smithi170 ceph-mon[102832]: pgmap v7456: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:07:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:25 smithi082 ceph-mon[98694]: pgmap v7456: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:07:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:25 smithi098 ceph-mon[103124]: pgmap v7456: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 51 KiB/s wr, 11 op/s 2023-09-10T03:07:27.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:27 smithi170 ceph-mon[102832]: pgmap v7457: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T03:07:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:27 smithi082 ceph-mon[98694]: pgmap v7457: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T03:07:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:27 smithi098 ceph-mon[103124]: pgmap v7457: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T03:07:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13680.215201 secs 2023-09-10T03:07:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:29 smithi170 ceph-mon[102832]: pgmap v7458: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 86 KiB/s wr, 10 op/s 2023-09-10T03:07:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:07:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13680.926612 secs 2023-09-10T03:07:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:07:29.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:07:29.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:07:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13680.215201 secs 2023-09-10T03:07:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:29 smithi082 ceph-mon[98694]: pgmap v7458: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 86 KiB/s wr, 10 op/s 2023-09-10T03:07:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:07:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13680.926612 secs 2023-09-10T03:07:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:07:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:07:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:07:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13680.215201 secs 2023-09-10T03:07:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:29 smithi098 ceph-mon[103124]: pgmap v7458: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 86 KiB/s wr, 10 op/s 2023-09-10T03:07:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:07:29.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13680.926612 secs 2023-09-10T03:07:29.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:07:29.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:07:29.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:07:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:31 smithi170 ceph-mon[102832]: pgmap v7459: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 150 KiB/s wr, 14 op/s 2023-09-10T03:07:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:31 smithi082 ceph-mon[98694]: pgmap v7459: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 150 KiB/s wr, 14 op/s 2023-09-10T03:07:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:31 smithi098 ceph-mon[103124]: pgmap v7459: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 150 KiB/s wr, 14 op/s 2023-09-10T03:07:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:33 smithi170 ceph-mon[102832]: pgmap v7460: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T03:07:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:33 smithi082 ceph-mon[98694]: pgmap v7460: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T03:07:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:33 smithi098 ceph-mon[103124]: pgmap v7460: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T03:07:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13685.215283 secs 2023-09-10T03:07:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:07:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:07:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13685.926678 secs 2023-09-10T03:07:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13685.215283 secs 2023-09-10T03:07:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:07:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:07:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13685.926678 secs 2023-09-10T03:07:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13685.215283 secs 2023-09-10T03:07:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:07:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:07:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13685.926678 secs 2023-09-10T03:07:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:35 smithi170 ceph-mon[102832]: pgmap v7461: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T03:07:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:35 smithi082 ceph-mon[98694]: pgmap v7461: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T03:07:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:35 smithi098 ceph-mon[103124]: pgmap v7461: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T03:07:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:07:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:07:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:07:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:07:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:07:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:07:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:37 smithi170 ceph-mon[102832]: pgmap v7462: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 182 KiB/s wr, 17 op/s 2023-09-10T03:07:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:37 smithi082 ceph-mon[98694]: pgmap v7462: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 182 KiB/s wr, 17 op/s 2023-09-10T03:07:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:37 smithi098 ceph-mon[103124]: pgmap v7462: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 182 KiB/s wr, 17 op/s 2023-09-10T03:07:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13690.215359 secs 2023-09-10T03:07:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:39 smithi170 ceph-mon[102832]: pgmap v7463: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T03:07:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13690.926743 secs 2023-09-10T03:07:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13690.215359 secs 2023-09-10T03:07:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:39 smithi082 ceph-mon[98694]: pgmap v7463: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T03:07:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13690.926743 secs 2023-09-10T03:07:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13690.215359 secs 2023-09-10T03:07:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:39 smithi098 ceph-mon[103124]: pgmap v7463: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T03:07:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13690.926743 secs 2023-09-10T03:07:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:41 smithi170 ceph-mon[102832]: pgmap v7464: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 272 KiB/s wr, 17 op/s 2023-09-10T03:07:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:41 smithi082 ceph-mon[98694]: pgmap v7464: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 272 KiB/s wr, 17 op/s 2023-09-10T03:07:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:41 smithi098 ceph-mon[103124]: pgmap v7464: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 272 KiB/s wr, 17 op/s 2023-09-10T03:07:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:43 smithi170 ceph-mon[102832]: pgmap v7465: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 201 KiB/s wr, 14 op/s 2023-09-10T03:07:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:43 smithi082 ceph-mon[98694]: pgmap v7465: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 201 KiB/s wr, 14 op/s 2023-09-10T03:07:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:43 smithi098 ceph-mon[103124]: pgmap v7465: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 201 KiB/s wr, 14 op/s 2023-09-10T03:07:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13695.215416 secs 2023-09-10T03:07:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13695.926809 secs 2023-09-10T03:07:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13695.215416 secs 2023-09-10T03:07:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13695.926809 secs 2023-09-10T03:07:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13695.215416 secs 2023-09-10T03:07:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13695.926809 secs 2023-09-10T03:07:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:45 smithi170 ceph-mon[102832]: pgmap v7466: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T03:07:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:45 smithi082 ceph-mon[98694]: pgmap v7466: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T03:07:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:45 smithi098 ceph-mon[103124]: pgmap v7466: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T03:07:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:47 smithi170 ceph-mon[102832]: pgmap v7467: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 254 KiB/s wr, 18 op/s 2023-09-10T03:07:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:47 smithi082 ceph-mon[98694]: pgmap v7467: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 254 KiB/s wr, 18 op/s 2023-09-10T03:07:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:47 smithi098 ceph-mon[103124]: pgmap v7467: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 254 KiB/s wr, 18 op/s 2023-09-10T03:07:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13700.215472 secs 2023-09-10T03:07:49.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:49 smithi170 ceph-mon[102832]: pgmap v7468: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 222 KiB/s wr, 11 op/s 2023-09-10T03:07:49.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13700.926875 secs 2023-09-10T03:07:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13700.215472 secs 2023-09-10T03:07:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:49 smithi082 ceph-mon[98694]: pgmap v7468: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 222 KiB/s wr, 11 op/s 2023-09-10T03:07:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13700.926875 secs 2023-09-10T03:07:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13700.215472 secs 2023-09-10T03:07:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:49 smithi098 ceph-mon[103124]: pgmap v7468: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 222 KiB/s wr, 11 op/s 2023-09-10T03:07:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13700.926875 secs 2023-09-10T03:07:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:07:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:07:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:07:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:07:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:07:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:07:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:51 smithi170 ceph-mon[102832]: pgmap v7469: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 362 KiB/s wr, 17 op/s 2023-09-10T03:07:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:51 smithi082 ceph-mon[98694]: pgmap v7469: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 362 KiB/s wr, 17 op/s 2023-09-10T03:07:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:51 smithi098 ceph-mon[103124]: pgmap v7469: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 362 KiB/s wr, 17 op/s 2023-09-10T03:07:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:07:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:07:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:07:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:07:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:07:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:07:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13703.853133 secs 2023-09-10T03:07:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:53 smithi170 ceph-mon[102832]: pgmap v7470: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 14 op/s 2023-09-10T03:07:53.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13703.853133 secs 2023-09-10T03:07:53.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:53 smithi082 ceph-mon[98694]: pgmap v7470: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 14 op/s 2023-09-10T03:07:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13703.853133 secs 2023-09-10T03:07:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:53 smithi098 ceph-mon[103124]: pgmap v7470: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 14 op/s 2023-09-10T03:07:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13705.215490 secs 2023-09-10T03:07:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13705.926939 secs 2023-09-10T03:07:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13705.215490 secs 2023-09-10T03:07:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13705.926939 secs 2023-09-10T03:07:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13705.215490 secs 2023-09-10T03:07:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13705.926939 secs 2023-09-10T03:07:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:55 smithi170 ceph-mon[102832]: pgmap v7471: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 12 op/s 2023-09-10T03:07:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:55 smithi082 ceph-mon[98694]: pgmap v7471: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 12 op/s 2023-09-10T03:07:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:55 smithi098 ceph-mon[103124]: pgmap v7471: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 12 op/s 2023-09-10T03:07:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:07:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:07:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:07:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:07:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:07:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:07:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:07:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:07:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:07:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:57 smithi170 ceph-mon[102832]: pgmap v7472: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 233 KiB/s wr, 18 op/s 2023-09-10T03:07:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:57 smithi082 ceph-mon[98694]: pgmap v7472: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 233 KiB/s wr, 18 op/s 2023-09-10T03:07:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:57 smithi098 ceph-mon[103124]: pgmap v7472: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 233 KiB/s wr, 18 op/s 2023-09-10T03:07:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13708.853203 secs 2023-09-10T03:07:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13708.853203 secs 2023-09-10T03:07:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13708.853203 secs 2023-09-10T03:07:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13710.215549 secs 2023-09-10T03:07:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:59 smithi082 ceph-mon[98694]: pgmap v7473: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 179 KiB/s wr, 12 op/s 2023-09-10T03:07:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:07:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13710.927003 secs 2023-09-10T03:07:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13710.215549 secs 2023-09-10T03:07:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:59 smithi098 ceph-mon[103124]: pgmap v7473: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 179 KiB/s wr, 12 op/s 2023-09-10T03:07:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:07:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13710.927003 secs 2023-09-10T03:07:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13710.215549 secs 2023-09-10T03:07:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:59 smithi170 ceph-mon[102832]: pgmap v7473: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 179 KiB/s wr, 12 op/s 2023-09-10T03:07:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:07:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13710.927003 secs 2023-09-10T03:08:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:01 smithi082 ceph-mon[98694]: pgmap v7474: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 278 KiB/s wr, 16 op/s 2023-09-10T03:08:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:01 smithi098 ceph-mon[103124]: pgmap v7474: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 278 KiB/s wr, 16 op/s 2023-09-10T03:08:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:01 smithi170 ceph-mon[102832]: pgmap v7474: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 278 KiB/s wr, 16 op/s 2023-09-10T03:08:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13713.853263 secs 2023-09-10T03:08:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:03 smithi170 ceph-mon[102832]: pgmap v7475: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 12 op/s 2023-09-10T03:08:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13713.853263 secs 2023-09-10T03:08:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:03 smithi082 ceph-mon[98694]: pgmap v7475: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 12 op/s 2023-09-10T03:08:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13713.853263 secs 2023-09-10T03:08:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:03 smithi098 ceph-mon[103124]: pgmap v7475: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 12 op/s 2023-09-10T03:08:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13715.215603 secs 2023-09-10T03:08:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13715.927070 secs 2023-09-10T03:08:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13715.215603 secs 2023-09-10T03:08:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13715.927070 secs 2023-09-10T03:08:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13715.215603 secs 2023-09-10T03:08:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13715.927070 secs 2023-09-10T03:08:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:05 smithi170 ceph-mon[102832]: pgmap v7476: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 10 op/s 2023-09-10T03:08:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:05 smithi082 ceph-mon[98694]: pgmap v7476: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 10 op/s 2023-09-10T03:08:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:05 smithi098 ceph-mon[103124]: pgmap v7476: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 10 op/s 2023-09-10T03:08:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:07 smithi170 ceph-mon[102832]: pgmap v7477: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 184 KiB/s wr, 30 op/s 2023-09-10T03:08:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:07 smithi082 ceph-mon[98694]: pgmap v7477: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 184 KiB/s wr, 30 op/s 2023-09-10T03:08:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:07 smithi098 ceph-mon[103124]: pgmap v7477: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 184 KiB/s wr, 30 op/s 2023-09-10T03:08:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13718.853324 secs 2023-09-10T03:08:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:08 smithi170 ceph-mon[102832]: osdmap e535: 12 total, 12 up, 12 in 2023-09-10T03:08:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13718.853324 secs 2023-09-10T03:08:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:08 smithi082 ceph-mon[98694]: osdmap e535: 12 total, 12 up, 12 in 2023-09-10T03:08:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13718.853324 secs 2023-09-10T03:08:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:08 smithi098 ceph-mon[103124]: osdmap e535: 12 total, 12 up, 12 in 2023-09-10T03:08:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13720.215663 secs 2023-09-10T03:08:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:09 smithi170 ceph-mon[102832]: pgmap v7479: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 173 KiB/s wr, 29 op/s 2023-09-10T03:08:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13720.927136 secs 2023-09-10T03:08:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13720.215663 secs 2023-09-10T03:08:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:09 smithi082 ceph-mon[98694]: pgmap v7479: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 173 KiB/s wr, 29 op/s 2023-09-10T03:08:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13720.927136 secs 2023-09-10T03:08:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13720.215663 secs 2023-09-10T03:08:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:09 smithi098 ceph-mon[103124]: pgmap v7479: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 173 KiB/s wr, 29 op/s 2023-09-10T03:08:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13720.927136 secs 2023-09-10T03:08:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:11 smithi170 ceph-mon[102832]: pgmap v7480: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 28 op/s 2023-09-10T03:08:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:11 smithi082 ceph-mon[98694]: pgmap v7480: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 28 op/s 2023-09-10T03:08:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:11 smithi098 ceph-mon[103124]: pgmap v7480: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 28 op/s 2023-09-10T03:08:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13723.853384 secs 2023-09-10T03:08:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:13 smithi170 ceph-mon[102832]: pgmap v7481: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 28 op/s 2023-09-10T03:08:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13723.853384 secs 2023-09-10T03:08:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:13 smithi082 ceph-mon[98694]: pgmap v7481: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 28 op/s 2023-09-10T03:08:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13723.853384 secs 2023-09-10T03:08:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:13 smithi098 ceph-mon[103124]: pgmap v7481: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 28 op/s 2023-09-10T03:08:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13725.215769 secs 2023-09-10T03:08:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13725.927201 secs 2023-09-10T03:08:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13725.215769 secs 2023-09-10T03:08:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13725.927201 secs 2023-09-10T03:08:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13725.215769 secs 2023-09-10T03:08:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13725.927201 secs 2023-09-10T03:08:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:15 smithi170 ceph-mon[102832]: pgmap v7482: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 28 op/s 2023-09-10T03:08:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:15 smithi082 ceph-mon[98694]: pgmap v7482: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 28 op/s 2023-09-10T03:08:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:15 smithi098 ceph-mon[103124]: pgmap v7482: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 95 KiB/s wr, 28 op/s 2023-09-10T03:08:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:17 smithi170 ceph-mon[102832]: pgmap v7483: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T03:08:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:17 smithi170 ceph-mon[102832]: osdmap e536: 12 total, 12 up, 12 in 2023-09-10T03:08:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:17 smithi082 ceph-mon[98694]: pgmap v7483: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T03:08:17.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:17 smithi082 ceph-mon[98694]: osdmap e536: 12 total, 12 up, 12 in 2023-09-10T03:08:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:17 smithi098 ceph-mon[103124]: pgmap v7483: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T03:08:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:17 smithi098 ceph-mon[103124]: osdmap e536: 12 total, 12 up, 12 in 2023-09-10T03:08:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13728.853407 secs 2023-09-10T03:08:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13728.853407 secs 2023-09-10T03:08:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13728.853407 secs 2023-09-10T03:08:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13730.215830 secs 2023-09-10T03:08:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:19 smithi170 ceph-mon[102832]: pgmap v7485: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T03:08:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13730.927267 secs 2023-09-10T03:08:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13730.215830 secs 2023-09-10T03:08:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:19 smithi082 ceph-mon[98694]: pgmap v7485: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T03:08:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13730.927267 secs 2023-09-10T03:08:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13730.215830 secs 2023-09-10T03:08:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:19 smithi098 ceph-mon[103124]: pgmap v7485: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 11 op/s 2023-09-10T03:08:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13730.927267 secs 2023-09-10T03:08:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:21 smithi170 ceph-mon[102832]: pgmap v7486: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T03:08:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:21 smithi082 ceph-mon[98694]: pgmap v7486: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T03:08:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:21 smithi098 ceph-mon[103124]: pgmap v7486: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2023-09-10T03:08:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:23 smithi170 ceph-mon[102832]: pgmap v7487: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 12 op/s 2023-09-10T03:08:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:08:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:23 smithi082 ceph-mon[98694]: pgmap v7487: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 12 op/s 2023-09-10T03:08:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:08:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:23 smithi098 ceph-mon[103124]: pgmap v7487: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 12 op/s 2023-09-10T03:08:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:08:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13735.215917 secs 2023-09-10T03:08:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:08:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:08:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:08:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13735.927332 secs 2023-09-10T03:08:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13735.215917 secs 2023-09-10T03:08:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:08:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:08:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:08:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13735.927332 secs 2023-09-10T03:08:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13735.215917 secs 2023-09-10T03:08:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:08:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:08:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:08:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13735.927332 secs 2023-09-10T03:08:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:25 smithi170 ceph-mon[102832]: pgmap v7488: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 12 op/s 2023-09-10T03:08:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:25 smithi082 ceph-mon[98694]: pgmap v7488: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 12 op/s 2023-09-10T03:08:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:25 smithi098 ceph-mon[103124]: pgmap v7488: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 12 op/s 2023-09-10T03:08:27.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:27 smithi170 ceph-mon[102832]: pgmap v7489: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 135 KiB/s wr, 12 op/s 2023-09-10T03:08:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:27 smithi082 ceph-mon[98694]: pgmap v7489: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 135 KiB/s wr, 12 op/s 2023-09-10T03:08:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:27 smithi098 ceph-mon[103124]: pgmap v7489: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 135 KiB/s wr, 12 op/s 2023-09-10T03:08:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13740.215997 secs 2023-09-10T03:08:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:29 smithi170 ceph-mon[102832]: pgmap v7490: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T03:08:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13740.927396 secs 2023-09-10T03:08:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:08:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13740.215997 secs 2023-09-10T03:08:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:29 smithi082 ceph-mon[98694]: pgmap v7490: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T03:08:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13740.927396 secs 2023-09-10T03:08:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:08:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13740.215997 secs 2023-09-10T03:08:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:29 smithi098 ceph-mon[103124]: pgmap v7490: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T03:08:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13740.927396 secs 2023-09-10T03:08:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:08:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:08:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:08:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:08:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:08:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:08:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:08:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:08:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:08:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:08:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:08:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:08:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:08:30.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:08:30.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:08:30.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:08:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:31 smithi170 ceph-mon[102832]: pgmap v7491: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 309 KiB/s wr, 15 op/s 2023-09-10T03:08:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:31 smithi082 ceph-mon[98694]: pgmap v7491: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 309 KiB/s wr, 15 op/s 2023-09-10T03:08:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:31 smithi098 ceph-mon[103124]: pgmap v7491: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 309 KiB/s wr, 15 op/s 2023-09-10T03:08:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:08:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:08:33.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:08:33.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:08:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:08:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:08:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:33 smithi170 ceph-mon[102832]: pgmap v7492: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 267 KiB/s wr, 13 op/s 2023-09-10T03:08:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13745.216084 secs 2023-09-10T03:08:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:33 smithi082 ceph-mon[98694]: pgmap v7492: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 267 KiB/s wr, 13 op/s 2023-09-10T03:08:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13745.216084 secs 2023-09-10T03:08:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:33 smithi098 ceph-mon[103124]: pgmap v7492: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 267 KiB/s wr, 13 op/s 2023-09-10T03:08:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13745.216084 secs 2023-09-10T03:08:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13745.927460 secs 2023-09-10T03:08:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13745.927460 secs 2023-09-10T03:08:35.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13745.927460 secs 2023-09-10T03:08:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:35 smithi170 ceph-mon[102832]: pgmap v7493: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 266 KiB/s wr, 11 op/s 2023-09-10T03:08:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:35 smithi082 ceph-mon[98694]: pgmap v7493: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 266 KiB/s wr, 11 op/s 2023-09-10T03:08:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:35 smithi098 ceph-mon[103124]: pgmap v7493: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 266 KiB/s wr, 11 op/s 2023-09-10T03:08:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:37 smithi170 ceph-mon[102832]: pgmap v7494: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 282 KiB/s wr, 17 op/s 2023-09-10T03:08:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:37 smithi082 ceph-mon[98694]: pgmap v7494: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 282 KiB/s wr, 17 op/s 2023-09-10T03:08:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:37 smithi098 ceph-mon[103124]: pgmap v7494: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 282 KiB/s wr, 17 op/s 2023-09-10T03:08:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13750.216143 secs 2023-09-10T03:08:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13750.216143 secs 2023-09-10T03:08:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13750.216143 secs 2023-09-10T03:08:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:39 smithi170 ceph-mon[102832]: pgmap v7495: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 212 KiB/s wr, 12 op/s 2023-09-10T03:08:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13750.927485 secs 2023-09-10T03:08:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:39 smithi082 ceph-mon[98694]: pgmap v7495: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 212 KiB/s wr, 12 op/s 2023-09-10T03:08:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13750.927485 secs 2023-09-10T03:08:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:39 smithi098 ceph-mon[103124]: pgmap v7495: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 212 KiB/s wr, 12 op/s 2023-09-10T03:08:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13750.927485 secs 2023-09-10T03:08:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:41 smithi170 ceph-mon[102832]: pgmap v7496: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 381 KiB/s wr, 17 op/s 2023-09-10T03:08:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:41 smithi082 ceph-mon[98694]: pgmap v7496: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 381 KiB/s wr, 17 op/s 2023-09-10T03:08:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:41 smithi098 ceph-mon[103124]: pgmap v7496: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 381 KiB/s wr, 17 op/s 2023-09-10T03:08:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:43 smithi170 ceph-mon[102832]: pgmap v7497: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 185 KiB/s wr, 13 op/s 2023-09-10T03:08:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:43 smithi082 ceph-mon[98694]: pgmap v7497: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 185 KiB/s wr, 13 op/s 2023-09-10T03:08:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:43 smithi098 ceph-mon[103124]: pgmap v7497: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 185 KiB/s wr, 13 op/s 2023-09-10T03:08:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13755.216202 secs 2023-09-10T03:08:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:08:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:08:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13755.927551 secs 2023-09-10T03:08:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13755.216202 secs 2023-09-10T03:08:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:08:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:08:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13755.927551 secs 2023-09-10T03:08:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13755.216202 secs 2023-09-10T03:08:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:08:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:08:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13755.927551 secs 2023-09-10T03:08:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:45 smithi170 ceph-mon[102832]: pgmap v7498: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T03:08:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:45 smithi082 ceph-mon[98694]: pgmap v7498: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T03:08:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:45 smithi098 ceph-mon[103124]: pgmap v7498: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T03:08:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:08:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:08:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:08:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:08:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:08:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:08:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:47 smithi170 ceph-mon[102832]: pgmap v7499: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 326 KiB/s wr, 17 op/s 2023-09-10T03:08:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:47 smithi082 ceph-mon[98694]: pgmap v7499: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 326 KiB/s wr, 17 op/s 2023-09-10T03:08:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:47 smithi098 ceph-mon[103124]: pgmap v7499: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 326 KiB/s wr, 17 op/s 2023-09-10T03:08:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13758.853876 secs 2023-09-10T03:08:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13758.853876 secs 2023-09-10T03:08:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13758.853876 secs 2023-09-10T03:08:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13760.216290 secs 2023-09-10T03:08:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:49 smithi170 ceph-mon[102832]: pgmap v7500: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 311 KiB/s wr, 12 op/s 2023-09-10T03:08:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13760.927616 secs 2023-09-10T03:08:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13760.216290 secs 2023-09-10T03:08:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:49 smithi082 ceph-mon[98694]: pgmap v7500: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 311 KiB/s wr, 12 op/s 2023-09-10T03:08:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13760.927616 secs 2023-09-10T03:08:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13760.216290 secs 2023-09-10T03:08:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:49 smithi098 ceph-mon[103124]: pgmap v7500: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 311 KiB/s wr, 12 op/s 2023-09-10T03:08:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13760.927616 secs 2023-09-10T03:08:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:51 smithi170 ceph-mon[102832]: pgmap v7501: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 349 KiB/s wr, 16 op/s 2023-09-10T03:08:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:51 smithi082 ceph-mon[98694]: pgmap v7501: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 349 KiB/s wr, 16 op/s 2023-09-10T03:08:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:51 smithi098 ceph-mon[103124]: pgmap v7501: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 349 KiB/s wr, 16 op/s 2023-09-10T03:08:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:08:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:08:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:08:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:08:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:08:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:08:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:08:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:08:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:08:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13763.853951 secs 2023-09-10T03:08:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:53 smithi170 ceph-mon[102832]: pgmap v7502: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 180 KiB/s wr, 12 op/s 2023-09-10T03:08:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13763.853951 secs 2023-09-10T03:08:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:53 smithi082 ceph-mon[98694]: pgmap v7502: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 180 KiB/s wr, 12 op/s 2023-09-10T03:08:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13763.853951 secs 2023-09-10T03:08:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:53 smithi098 ceph-mon[103124]: pgmap v7502: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 180 KiB/s wr, 12 op/s 2023-09-10T03:08:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13765.216374 secs 2023-09-10T03:08:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13765.927682 secs 2023-09-10T03:08:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13765.216374 secs 2023-09-10T03:08:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13765.927682 secs 2023-09-10T03:08:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13765.216374 secs 2023-09-10T03:08:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13765.927682 secs 2023-09-10T03:08:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:55 smithi170 ceph-mon[102832]: pgmap v7503: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 180 KiB/s wr, 11 op/s 2023-09-10T03:08:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:55 smithi082 ceph-mon[98694]: pgmap v7503: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 180 KiB/s wr, 11 op/s 2023-09-10T03:08:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:55 smithi098 ceph-mon[103124]: pgmap v7503: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 180 KiB/s wr, 11 op/s 2023-09-10T03:08:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:57 smithi170 ceph-mon[102832]: pgmap v7504: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 269 KiB/s wr, 16 op/s 2023-09-10T03:08:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:57 smithi082 ceph-mon[98694]: pgmap v7504: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 269 KiB/s wr, 16 op/s 2023-09-10T03:08:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:57 smithi098 ceph-mon[103124]: pgmap v7504: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 269 KiB/s wr, 16 op/s 2023-09-10T03:08:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13768.854013 secs 2023-09-10T03:08:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13768.854013 secs 2023-09-10T03:08:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13768.854013 secs 2023-09-10T03:08:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13770.216431 secs 2023-09-10T03:08:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:59 smithi170 ceph-mon[102832]: pgmap v7505: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 9 op/s 2023-09-10T03:08:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:08:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13770.927745 secs 2023-09-10T03:08:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13770.216431 secs 2023-09-10T03:08:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:59 smithi082 ceph-mon[98694]: pgmap v7505: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 9 op/s 2023-09-10T03:08:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:08:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13770.927745 secs 2023-09-10T03:08:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13770.216431 secs 2023-09-10T03:08:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:59 smithi098 ceph-mon[103124]: pgmap v7505: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 9 op/s 2023-09-10T03:08:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:08:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13770.927745 secs 2023-09-10T03:09:01.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:01 smithi170 ceph-mon[102832]: pgmap v7506: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 166 KiB/s wr, 14 op/s 2023-09-10T03:09:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:01 smithi082 ceph-mon[98694]: pgmap v7506: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 166 KiB/s wr, 14 op/s 2023-09-10T03:09:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:01 smithi098 ceph-mon[103124]: pgmap v7506: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 166 KiB/s wr, 14 op/s 2023-09-10T03:09:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13773.854071 secs 2023-09-10T03:09:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:03 smithi170 ceph-mon[102832]: pgmap v7507: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T03:09:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13773.854071 secs 2023-09-10T03:09:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:03 smithi082 ceph-mon[98694]: pgmap v7507: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T03:09:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13773.854071 secs 2023-09-10T03:09:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:03 smithi098 ceph-mon[103124]: pgmap v7507: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T03:09:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13775.216526 secs 2023-09-10T03:09:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13775.927814 secs 2023-09-10T03:09:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13775.216526 secs 2023-09-10T03:09:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13775.927814 secs 2023-09-10T03:09:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13775.216526 secs 2023-09-10T03:09:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13775.927814 secs 2023-09-10T03:09:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:05 smithi170 ceph-mon[102832]: pgmap v7508: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 128 KiB/s wr, 10 op/s 2023-09-10T03:09:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:05 smithi082 ceph-mon[98694]: pgmap v7508: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 128 KiB/s wr, 10 op/s 2023-09-10T03:09:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:05 smithi098 ceph-mon[103124]: pgmap v7508: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 128 KiB/s wr, 10 op/s 2023-09-10T03:09:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:07 smithi170 ceph-mon[102832]: pgmap v7509: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 184 KiB/s wr, 29 op/s 2023-09-10T03:09:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:07 smithi082 ceph-mon[98694]: pgmap v7509: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 184 KiB/s wr, 29 op/s 2023-09-10T03:09:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:07 smithi098 ceph-mon[103124]: pgmap v7509: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 184 KiB/s wr, 29 op/s 2023-09-10T03:09:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13778.854130 secs 2023-09-10T03:09:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:08 smithi170 ceph-mon[102832]: osdmap e537: 12 total, 12 up, 12 in 2023-09-10T03:09:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13778.854130 secs 2023-09-10T03:09:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:08 smithi082 ceph-mon[98694]: osdmap e537: 12 total, 12 up, 12 in 2023-09-10T03:09:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13778.854130 secs 2023-09-10T03:09:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:08 smithi098 ceph-mon[103124]: osdmap e537: 12 total, 12 up, 12 in 2023-09-10T03:09:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13780.216585 secs 2023-09-10T03:09:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:09 smithi170 ceph-mon[102832]: pgmap v7511: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T03:09:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13780.927877 secs 2023-09-10T03:09:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13780.216585 secs 2023-09-10T03:09:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:09 smithi082 ceph-mon[98694]: pgmap v7511: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T03:09:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13780.927877 secs 2023-09-10T03:09:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13780.216585 secs 2023-09-10T03:09:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:09 smithi098 ceph-mon[103124]: pgmap v7511: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T03:09:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13780.927877 secs 2023-09-10T03:09:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:11 smithi170 ceph-mon[102832]: pgmap v7512: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 96 KiB/s wr, 30 op/s 2023-09-10T03:09:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:11 smithi082 ceph-mon[98694]: pgmap v7512: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 96 KiB/s wr, 30 op/s 2023-09-10T03:09:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:11 smithi098 ceph-mon[103124]: pgmap v7512: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 96 KiB/s wr, 30 op/s 2023-09-10T03:09:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13783.854245 secs 2023-09-10T03:09:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:13 smithi170 ceph-mon[102832]: pgmap v7513: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T03:09:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13783.854245 secs 2023-09-10T03:09:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:13 smithi082 ceph-mon[98694]: pgmap v7513: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T03:09:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13783.854245 secs 2023-09-10T03:09:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:13 smithi098 ceph-mon[103124]: pgmap v7513: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T03:09:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13785.216667 secs 2023-09-10T03:09:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13785.927942 secs 2023-09-10T03:09:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13785.216667 secs 2023-09-10T03:09:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13785.927942 secs 2023-09-10T03:09:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13785.216667 secs 2023-09-10T03:09:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13785.927942 secs 2023-09-10T03:09:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:15 smithi170 ceph-mon[102832]: pgmap v7514: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T03:09:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:15 smithi082 ceph-mon[98694]: pgmap v7514: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T03:09:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:15 smithi098 ceph-mon[103124]: pgmap v7514: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 96 KiB/s wr, 29 op/s 2023-09-10T03:09:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:17 smithi170 ceph-mon[102832]: pgmap v7515: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T03:09:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:17 smithi170 ceph-mon[102832]: osdmap e538: 12 total, 12 up, 12 in 2023-09-10T03:09:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:17 smithi082 ceph-mon[98694]: pgmap v7515: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T03:09:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:17 smithi082 ceph-mon[98694]: osdmap e538: 12 total, 12 up, 12 in 2023-09-10T03:09:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:17 smithi098 ceph-mon[103124]: pgmap v7515: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T03:09:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:17 smithi098 ceph-mon[103124]: osdmap e538: 12 total, 12 up, 12 in 2023-09-10T03:09:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13788.854323 secs 2023-09-10T03:09:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13788.854323 secs 2023-09-10T03:09:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13788.854323 secs 2023-09-10T03:09:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13790.216724 secs 2023-09-10T03:09:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:19 smithi170 ceph-mon[102832]: pgmap v7517: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T03:09:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13790.928008 secs 2023-09-10T03:09:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13790.216724 secs 2023-09-10T03:09:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:19 smithi082 ceph-mon[98694]: pgmap v7517: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T03:09:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13790.928008 secs 2023-09-10T03:09:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13790.216724 secs 2023-09-10T03:09:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:19 smithi098 ceph-mon[103124]: pgmap v7517: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T03:09:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13790.928008 secs 2023-09-10T03:09:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:21 smithi170 ceph-mon[102832]: pgmap v7518: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 12 op/s 2023-09-10T03:09:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:21 smithi082 ceph-mon[98694]: pgmap v7518: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 12 op/s 2023-09-10T03:09:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:21 smithi098 ceph-mon[103124]: pgmap v7518: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 12 op/s 2023-09-10T03:09:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:23 smithi170 ceph-mon[102832]: pgmap v7519: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 12 op/s 2023-09-10T03:09:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:09:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:23 smithi082 ceph-mon[98694]: pgmap v7519: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 12 op/s 2023-09-10T03:09:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:09:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:23 smithi098 ceph-mon[103124]: pgmap v7519: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 12 op/s 2023-09-10T03:09:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:09:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13795.216807 secs 2023-09-10T03:09:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:09:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:09:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:09:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13795.928073 secs 2023-09-10T03:09:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13795.216807 secs 2023-09-10T03:09:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:09:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:09:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:09:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13795.928073 secs 2023-09-10T03:09:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13795.216807 secs 2023-09-10T03:09:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:09:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:09:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:09:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13795.928073 secs 2023-09-10T03:09:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:25 smithi170 ceph-mon[102832]: pgmap v7520: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 12 op/s 2023-09-10T03:09:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:25 smithi082 ceph-mon[98694]: pgmap v7520: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 12 op/s 2023-09-10T03:09:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:25 smithi098 ceph-mon[103124]: pgmap v7520: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 12 op/s 2023-09-10T03:09:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:26 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:09:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:09:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:26 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:09:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:09:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:26 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:09:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:09:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:27 smithi170 ceph-mon[102832]: pgmap v7521: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T03:09:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:27 smithi082 ceph-mon[98694]: pgmap v7521: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T03:09:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:27 smithi098 ceph-mon[103124]: pgmap v7521: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T03:09:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13800.216869 secs 2023-09-10T03:09:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:29 smithi170 ceph-mon[102832]: pgmap v7522: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 10 op/s 2023-09-10T03:09:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13800.928139 secs 2023-09-10T03:09:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13800.216869 secs 2023-09-10T03:09:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:29 smithi082 ceph-mon[98694]: pgmap v7522: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 10 op/s 2023-09-10T03:09:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13800.928139 secs 2023-09-10T03:09:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13800.216869 secs 2023-09-10T03:09:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:29 smithi098 ceph-mon[103124]: pgmap v7522: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 10 op/s 2023-09-10T03:09:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13800.928139 secs 2023-09-10T03:09:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:09:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:09:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:09:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:31 smithi170 ceph-mon[102832]: pgmap v7523: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 14 op/s 2023-09-10T03:09:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:31 smithi082 ceph-mon[98694]: pgmap v7523: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 14 op/s 2023-09-10T03:09:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:31 smithi098 ceph-mon[103124]: pgmap v7523: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 14 op/s 2023-09-10T03:09:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:33 smithi170 ceph-mon[102832]: pgmap v7524: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 10 op/s 2023-09-10T03:09:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:09:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:09:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:09:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:09:33.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13805.216930 secs 2023-09-10T03:09:33.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:33.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:33.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:33.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:33.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:33.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:33.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:33.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:33 smithi082 ceph-mon[98694]: pgmap v7524: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 10 op/s 2023-09-10T03:09:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:09:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:09:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:09:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:09:33.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13805.216930 secs 2023-09-10T03:09:33.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:33.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:33.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:33.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:33.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:33.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:33.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:33.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:33 smithi098 ceph-mon[103124]: pgmap v7524: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 10 op/s 2023-09-10T03:09:34.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:09:34.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:09:34.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:09:34.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:09:34.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13805.216930 secs 2023-09-10T03:09:34.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:34.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:34.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:34.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:34.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:34.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:34.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:34.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:34 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 132.9M 2023-09-10T03:09:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:34 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 139441696: error parsing value: Value '139441696' is below minimum 939524096 2023-09-10T03:09:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:34 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T03:09:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:34 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T03:09:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13805.928203 secs 2023-09-10T03:09:34.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:09:34.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:09:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:34 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 132.9M 2023-09-10T03:09:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:34 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 139441696: error parsing value: Value '139441696' is below minimum 939524096 2023-09-10T03:09:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:34 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T03:09:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:34 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T03:09:34.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13805.928203 secs 2023-09-10T03:09:34.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:09:34.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:09:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:34 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 132.9M 2023-09-10T03:09:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:34 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 139441696: error parsing value: Value '139441696' is below minimum 939524096 2023-09-10T03:09:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:34 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T03:09:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:34 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T03:09:35.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13805.928203 secs 2023-09-10T03:09:35.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:09:35.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:09:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:35 smithi170 ceph-mon[102832]: pgmap v7525: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 158 KiB/s wr, 10 op/s 2023-09-10T03:09:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:35.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:09:35.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:09:35.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:09:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:35 smithi082 ceph-mon[98694]: pgmap v7525: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 158 KiB/s wr, 10 op/s 2023-09-10T03:09:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:35.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:35.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:35.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:09:35.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:09:35.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:09:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:35 smithi098 ceph-mon[103124]: pgmap v7525: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 158 KiB/s wr, 10 op/s 2023-09-10T03:09:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:09:36.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:09:36.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:09:36.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:09:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:36 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T03:09:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:36 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T03:09:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:36 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T03:09:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:36 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T03:09:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:36 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T03:09:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:36 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T03:09:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:37 smithi170 ceph-mon[102832]: pgmap v7526: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 223 KiB/s wr, 16 op/s 2023-09-10T03:09:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:37 smithi082 ceph-mon[98694]: pgmap v7526: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 223 KiB/s wr, 16 op/s 2023-09-10T03:09:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:37 smithi098 ceph-mon[103124]: pgmap v7526: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 223 KiB/s wr, 16 op/s 2023-09-10T03:09:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13810.217016 secs 2023-09-10T03:09:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13810.217016 secs 2023-09-10T03:09:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13810.217016 secs 2023-09-10T03:09:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:39 smithi170 ceph-mon[102832]: pgmap v7527: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T03:09:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13810.928267 secs 2023-09-10T03:09:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:39 smithi082 ceph-mon[98694]: pgmap v7527: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T03:09:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13810.928267 secs 2023-09-10T03:09:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:39 smithi098 ceph-mon[103124]: pgmap v7527: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T03:09:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13810.928267 secs 2023-09-10T03:09:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:41 smithi170 ceph-mon[102832]: pgmap v7528: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 304 KiB/s wr, 17 op/s 2023-09-10T03:09:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:41 smithi082 ceph-mon[98694]: pgmap v7528: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 304 KiB/s wr, 17 op/s 2023-09-10T03:09:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:41 smithi098 ceph-mon[103124]: pgmap v7528: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 304 KiB/s wr, 17 op/s 2023-09-10T03:09:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:43 smithi170 ceph-mon[102832]: pgmap v7529: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 211 KiB/s wr, 13 op/s 2023-09-10T03:09:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:43 smithi082 ceph-mon[98694]: pgmap v7529: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 211 KiB/s wr, 13 op/s 2023-09-10T03:09:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:43 smithi098 ceph-mon[103124]: pgmap v7529: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 211 KiB/s wr, 13 op/s 2023-09-10T03:09:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13815.217074 secs 2023-09-10T03:09:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13815.928331 secs 2023-09-10T03:09:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13815.217074 secs 2023-09-10T03:09:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13815.928331 secs 2023-09-10T03:09:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13815.217074 secs 2023-09-10T03:09:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13815.928331 secs 2023-09-10T03:09:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:45 smithi170 ceph-mon[102832]: pgmap v7530: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 211 KiB/s wr, 12 op/s 2023-09-10T03:09:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:45 smithi082 ceph-mon[98694]: pgmap v7530: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 211 KiB/s wr, 12 op/s 2023-09-10T03:09:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:45 smithi098 ceph-mon[103124]: pgmap v7530: 97 pgs: 97 active+clean; 9.9 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 211 KiB/s wr, 12 op/s 2023-09-10T03:09:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:47 smithi170 ceph-mon[102832]: pgmap v7531: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 253 KiB/s wr, 18 op/s 2023-09-10T03:09:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:47 smithi082 ceph-mon[98694]: pgmap v7531: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 253 KiB/s wr, 18 op/s 2023-09-10T03:09:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:47 smithi098 ceph-mon[103124]: pgmap v7531: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 253 KiB/s wr, 18 op/s 2023-09-10T03:09:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13820.217174 secs 2023-09-10T03:09:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:49 smithi170 ceph-mon[102832]: pgmap v7532: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 188 KiB/s wr, 12 op/s 2023-09-10T03:09:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13820.928399 secs 2023-09-10T03:09:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13820.217174 secs 2023-09-10T03:09:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:49 smithi082 ceph-mon[98694]: pgmap v7532: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 188 KiB/s wr, 12 op/s 2023-09-10T03:09:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13820.928399 secs 2023-09-10T03:09:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13820.217174 secs 2023-09-10T03:09:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:49 smithi098 ceph-mon[103124]: pgmap v7532: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 188 KiB/s wr, 12 op/s 2023-09-10T03:09:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13820.928399 secs 2023-09-10T03:09:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:09:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:09:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:09:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:09:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:09:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:09:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:51 smithi082 ceph-mon[98694]: pgmap v7533: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 378 KiB/s wr, 18 op/s 2023-09-10T03:09:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:51 smithi098 ceph-mon[103124]: pgmap v7533: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 378 KiB/s wr, 18 op/s 2023-09-10T03:09:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:51 smithi170 ceph-mon[102832]: pgmap v7533: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 378 KiB/s wr, 18 op/s 2023-09-10T03:09:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:09:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:09:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:09:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:09:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:09:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:09:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13823.854747 secs 2023-09-10T03:09:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:53 smithi082 ceph-mon[98694]: pgmap v7534: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 232 KiB/s wr, 12 op/s 2023-09-10T03:09:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13823.854747 secs 2023-09-10T03:09:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:53 smithi098 ceph-mon[103124]: pgmap v7534: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 232 KiB/s wr, 12 op/s 2023-09-10T03:09:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13823.854747 secs 2023-09-10T03:09:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:53 smithi170 ceph-mon[102832]: pgmap v7534: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 232 KiB/s wr, 12 op/s 2023-09-10T03:09:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13825.217230 secs 2023-09-10T03:09:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13825.928464 secs 2023-09-10T03:09:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13825.217230 secs 2023-09-10T03:09:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13825.928464 secs 2023-09-10T03:09:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13825.217230 secs 2023-09-10T03:09:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13825.928464 secs 2023-09-10T03:09:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:55 smithi082 ceph-mon[98694]: pgmap v7535: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 232 KiB/s wr, 12 op/s 2023-09-10T03:09:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:55 smithi098 ceph-mon[103124]: pgmap v7535: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 232 KiB/s wr, 12 op/s 2023-09-10T03:09:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:55 smithi170 ceph-mon[102832]: pgmap v7535: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 232 KiB/s wr, 12 op/s 2023-09-10T03:09:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:09:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:09:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:09:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:09:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:09:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:09:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:09:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:09:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:09:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:57 smithi082 ceph-mon[98694]: pgmap v7536: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 271 KiB/s wr, 17 op/s 2023-09-10T03:09:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:57 smithi098 ceph-mon[103124]: pgmap v7536: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 271 KiB/s wr, 17 op/s 2023-09-10T03:09:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:57 smithi170 ceph-mon[102832]: pgmap v7536: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 271 KiB/s wr, 17 op/s 2023-09-10T03:09:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13828.854776 secs 2023-09-10T03:09:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13828.854776 secs 2023-09-10T03:09:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13828.854776 secs 2023-09-10T03:09:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13830.217288 secs 2023-09-10T03:09:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:59 smithi098 ceph-mon[103124]: pgmap v7537: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 229 KiB/s wr, 11 op/s 2023-09-10T03:09:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:09:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13830.928489 secs 2023-09-10T03:09:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13830.217288 secs 2023-09-10T03:09:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:59 smithi170 ceph-mon[102832]: pgmap v7537: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 229 KiB/s wr, 11 op/s 2023-09-10T03:09:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:09:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13830.928489 secs 2023-09-10T03:09:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13830.217288 secs 2023-09-10T03:09:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:59 smithi082 ceph-mon[98694]: pgmap v7537: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 229 KiB/s wr, 11 op/s 2023-09-10T03:09:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:09:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13830.928489 secs 2023-09-10T03:10:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:00 smithi098 ceph-mon[103124]: Health detail: HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T03:10:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:00 smithi098 ceph-mon[103124]: [WRN] MDS_SLOW_REQUEST: 3 MDSs report slow requests 2023-09-10T03:10:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:00 smithi098 ceph-mon[103124]: mds.l(mds.4): 3 slow requests are blocked > 180 secs 2023-09-10T03:10:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:00 smithi098 ceph-mon[103124]: mds.h(mds.1): 3 slow requests are blocked > 180 secs 2023-09-10T03:10:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:00 smithi098 ceph-mon[103124]: mds.b(mds.0): 1 slow requests are blocked > 180 secs 2023-09-10T03:10:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:00 smithi098 ceph-mon[103124]: [WRN] MDS_TRIM: 2 MDSs behind on trimming 2023-09-10T03:10:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:00 smithi098 ceph-mon[103124]: mds.e(mds.0): Behind on trimming (263/128) max_segments: 128, num_segments: 263 2023-09-10T03:10:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:00 smithi098 ceph-mon[103124]: mds.b(mds.0): Behind on trimming (263/128) max_segments: 128, num_segments: 263 2023-09-10T03:10:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:00 smithi170 ceph-mon[102832]: Health detail: HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T03:10:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:00 smithi170 ceph-mon[102832]: [WRN] MDS_SLOW_REQUEST: 3 MDSs report slow requests 2023-09-10T03:10:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:00 smithi170 ceph-mon[102832]: mds.l(mds.4): 3 slow requests are blocked > 180 secs 2023-09-10T03:10:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:00 smithi170 ceph-mon[102832]: mds.h(mds.1): 3 slow requests are blocked > 180 secs 2023-09-10T03:10:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:00 smithi170 ceph-mon[102832]: mds.b(mds.0): 1 slow requests are blocked > 180 secs 2023-09-10T03:10:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:00 smithi170 ceph-mon[102832]: [WRN] MDS_TRIM: 2 MDSs behind on trimming 2023-09-10T03:10:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:00 smithi170 ceph-mon[102832]: mds.e(mds.0): Behind on trimming (263/128) max_segments: 128, num_segments: 263 2023-09-10T03:10:00.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:00 smithi170 ceph-mon[102832]: mds.b(mds.0): Behind on trimming (263/128) max_segments: 128, num_segments: 263 2023-09-10T03:10:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:00 smithi082 ceph-mon[98694]: Health detail: HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T03:10:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:00 smithi082 ceph-mon[98694]: [WRN] MDS_SLOW_REQUEST: 3 MDSs report slow requests 2023-09-10T03:10:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:00 smithi082 ceph-mon[98694]: mds.l(mds.4): 3 slow requests are blocked > 180 secs 2023-09-10T03:10:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:00 smithi082 ceph-mon[98694]: mds.h(mds.1): 3 slow requests are blocked > 180 secs 2023-09-10T03:10:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:00 smithi082 ceph-mon[98694]: mds.b(mds.0): 1 slow requests are blocked > 180 secs 2023-09-10T03:10:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:00 smithi082 ceph-mon[98694]: [WRN] MDS_TRIM: 2 MDSs behind on trimming 2023-09-10T03:10:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:00 smithi082 ceph-mon[98694]: mds.e(mds.0): Behind on trimming (263/128) max_segments: 128, num_segments: 263 2023-09-10T03:10:00.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:00 smithi082 ceph-mon[98694]: mds.b(mds.0): Behind on trimming (263/128) max_segments: 128, num_segments: 263 2023-09-10T03:10:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:01 smithi098 ceph-mon[103124]: pgmap v7538: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 296 KiB/s wr, 16 op/s 2023-09-10T03:10:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:01 smithi170 ceph-mon[102832]: pgmap v7538: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 296 KiB/s wr, 16 op/s 2023-09-10T03:10:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:01 smithi082 ceph-mon[98694]: pgmap v7538: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 296 KiB/s wr, 16 op/s 2023-09-10T03:10:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13833.854863 secs 2023-09-10T03:10:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:03 smithi170 ceph-mon[102832]: pgmap v7539: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T03:10:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13833.854863 secs 2023-09-10T03:10:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:03 smithi082 ceph-mon[98694]: pgmap v7539: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T03:10:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13833.854863 secs 2023-09-10T03:10:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:03 smithi098 ceph-mon[103124]: pgmap v7539: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T03:10:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13835.217368 secs 2023-09-10T03:10:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13835.928553 secs 2023-09-10T03:10:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13835.217368 secs 2023-09-10T03:10:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13835.928553 secs 2023-09-10T03:10:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13835.217368 secs 2023-09-10T03:10:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13835.928553 secs 2023-09-10T03:10:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:05 smithi170 ceph-mon[102832]: pgmap v7540: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 106 KiB/s wr, 10 op/s 2023-09-10T03:10:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:05 smithi082 ceph-mon[98694]: pgmap v7540: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 106 KiB/s wr, 10 op/s 2023-09-10T03:10:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:05 smithi098 ceph-mon[103124]: pgmap v7540: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 106 KiB/s wr, 10 op/s 2023-09-10T03:10:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:07 smithi170 ceph-mon[102832]: pgmap v7541: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 179 KiB/s wr, 29 op/s 2023-09-10T03:10:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:07 smithi082 ceph-mon[98694]: pgmap v7541: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 179 KiB/s wr, 29 op/s 2023-09-10T03:10:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:07 smithi098 ceph-mon[103124]: pgmap v7541: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 179 KiB/s wr, 29 op/s 2023-09-10T03:10:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13838.854927 secs 2023-09-10T03:10:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:08 smithi170 ceph-mon[102832]: osdmap e539: 12 total, 12 up, 12 in 2023-09-10T03:10:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13838.854927 secs 2023-09-10T03:10:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:08 smithi082 ceph-mon[98694]: osdmap e539: 12 total, 12 up, 12 in 2023-09-10T03:10:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13838.854927 secs 2023-09-10T03:10:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:08 smithi098 ceph-mon[103124]: osdmap e539: 12 total, 12 up, 12 in 2023-09-10T03:10:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13840.217458 secs 2023-09-10T03:10:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:09 smithi082 ceph-mon[98694]: pgmap v7543: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 169 KiB/s wr, 29 op/s 2023-09-10T03:10:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13840.928617 secs 2023-09-10T03:10:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13840.217458 secs 2023-09-10T03:10:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:09 smithi098 ceph-mon[103124]: pgmap v7543: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 169 KiB/s wr, 29 op/s 2023-09-10T03:10:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13840.928617 secs 2023-09-10T03:10:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13840.217458 secs 2023-09-10T03:10:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:09 smithi170 ceph-mon[102832]: pgmap v7543: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 169 KiB/s wr, 29 op/s 2023-09-10T03:10:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13840.928617 secs 2023-09-10T03:10:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:11 smithi082 ceph-mon[98694]: pgmap v7544: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T03:10:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:11 smithi098 ceph-mon[103124]: pgmap v7544: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T03:10:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:11 smithi170 ceph-mon[102832]: pgmap v7544: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T03:10:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13843.854985 secs 2023-09-10T03:10:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:13 smithi170 ceph-mon[102832]: pgmap v7545: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T03:10:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13843.854985 secs 2023-09-10T03:10:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:13 smithi082 ceph-mon[98694]: pgmap v7545: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T03:10:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13843.854985 secs 2023-09-10T03:10:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:13 smithi098 ceph-mon[103124]: pgmap v7545: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T03:10:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13845.217497 secs 2023-09-10T03:10:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13845.928684 secs 2023-09-10T03:10:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13845.217497 secs 2023-09-10T03:10:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13845.928684 secs 2023-09-10T03:10:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13845.217497 secs 2023-09-10T03:10:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13845.928684 secs 2023-09-10T03:10:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:15 smithi170 ceph-mon[102832]: pgmap v7546: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T03:10:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:15 smithi082 ceph-mon[98694]: pgmap v7546: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T03:10:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:15 smithi098 ceph-mon[103124]: pgmap v7546: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T03:10:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:17 smithi170 ceph-mon[102832]: pgmap v7547: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T03:10:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:17 smithi170 ceph-mon[102832]: osdmap e540: 12 total, 12 up, 12 in 2023-09-10T03:10:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:17 smithi082 ceph-mon[98694]: pgmap v7547: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T03:10:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:17 smithi082 ceph-mon[98694]: osdmap e540: 12 total, 12 up, 12 in 2023-09-10T03:10:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:17 smithi098 ceph-mon[103124]: pgmap v7547: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T03:10:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:17 smithi098 ceph-mon[103124]: osdmap e540: 12 total, 12 up, 12 in 2023-09-10T03:10:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13848.855069 secs 2023-09-10T03:10:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13848.855069 secs 2023-09-10T03:10:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13848.855069 secs 2023-09-10T03:10:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13850.217555 secs 2023-09-10T03:10:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:19 smithi082 ceph-mon[98694]: pgmap v7549: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T03:10:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13850.928750 secs 2023-09-10T03:10:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13850.217555 secs 2023-09-10T03:10:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:19 smithi098 ceph-mon[103124]: pgmap v7549: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T03:10:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13850.928750 secs 2023-09-10T03:10:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13850.217555 secs 2023-09-10T03:10:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:19 smithi170 ceph-mon[102832]: pgmap v7549: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T03:10:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13850.928750 secs 2023-09-10T03:10:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:21 smithi082 ceph-mon[98694]: pgmap v7550: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 12 op/s 2023-09-10T03:10:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:21 smithi098 ceph-mon[103124]: pgmap v7550: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 12 op/s 2023-09-10T03:10:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:21 smithi170 ceph-mon[102832]: pgmap v7550: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 12 op/s 2023-09-10T03:10:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:23 smithi170 ceph-mon[102832]: pgmap v7551: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 12 op/s 2023-09-10T03:10:23.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:10:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:23 smithi082 ceph-mon[98694]: pgmap v7551: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 12 op/s 2023-09-10T03:10:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:10:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:23 smithi098 ceph-mon[103124]: pgmap v7551: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 12 op/s 2023-09-10T03:10:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:10:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13855.217613 secs 2023-09-10T03:10:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:10:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:10:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:10:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13855.928814 secs 2023-09-10T03:10:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13855.217613 secs 2023-09-10T03:10:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:10:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:10:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:10:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13855.928814 secs 2023-09-10T03:10:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13855.217613 secs 2023-09-10T03:10:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:10:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:10:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:10:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13855.928814 secs 2023-09-10T03:10:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:25 smithi170 ceph-mon[102832]: pgmap v7552: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 12 op/s 2023-09-10T03:10:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:25 smithi082 ceph-mon[98694]: pgmap v7552: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 12 op/s 2023-09-10T03:10:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:25 smithi098 ceph-mon[103124]: pgmap v7552: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 12 op/s 2023-09-10T03:10:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:27 smithi170 ceph-mon[102832]: pgmap v7553: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 152 KiB/s wr, 13 op/s 2023-09-10T03:10:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:27 smithi082 ceph-mon[98694]: pgmap v7553: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 152 KiB/s wr, 13 op/s 2023-09-10T03:10:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:27 smithi098 ceph-mon[103124]: pgmap v7553: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 152 KiB/s wr, 13 op/s 2023-09-10T03:10:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13860.217684 secs 2023-09-10T03:10:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:29 smithi082 ceph-mon[98694]: pgmap v7554: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T03:10:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13860.928880 secs 2023-09-10T03:10:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13860.217684 secs 2023-09-10T03:10:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:29 smithi098 ceph-mon[103124]: pgmap v7554: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T03:10:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13860.928880 secs 2023-09-10T03:10:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13860.217684 secs 2023-09-10T03:10:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:29 smithi170 ceph-mon[102832]: pgmap v7554: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T03:10:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13860.928880 secs 2023-09-10T03:10:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:10:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:10:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:10:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:10:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:10:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:10:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:31 smithi082 ceph-mon[98694]: pgmap v7555: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 314 KiB/s wr, 17 op/s 2023-09-10T03:10:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:31 smithi098 ceph-mon[103124]: pgmap v7555: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 314 KiB/s wr, 17 op/s 2023-09-10T03:10:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:31 smithi170 ceph-mon[102832]: pgmap v7555: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 314 KiB/s wr, 17 op/s 2023-09-10T03:10:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:33 smithi170 ceph-mon[102832]: pgmap v7556: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T03:10:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:33 smithi082 ceph-mon[98694]: pgmap v7556: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T03:10:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:33 smithi098 ceph-mon[103124]: pgmap v7556: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T03:10:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13865.217743 secs 2023-09-10T03:10:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13865.928946 secs 2023-09-10T03:10:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13865.217743 secs 2023-09-10T03:10:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13865.928946 secs 2023-09-10T03:10:34.708 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13865.217743 secs 2023-09-10T03:10:34.709 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13865.928946 secs 2023-09-10T03:10:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:35 smithi170 ceph-mon[102832]: pgmap v7557: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T03:10:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:10:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:35 smithi082 ceph-mon[98694]: pgmap v7557: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T03:10:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:10:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:35 smithi098 ceph-mon[103124]: pgmap v7557: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T03:10:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:10:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:10:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:10:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:10:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:10:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:10:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:10:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:10:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:10:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:10:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:37 smithi170 ceph-mon[102832]: pgmap v7558: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 329 KiB/s wr, 17 op/s 2023-09-10T03:10:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:10:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:10:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:37 smithi082 ceph-mon[98694]: pgmap v7558: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 329 KiB/s wr, 17 op/s 2023-09-10T03:10:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:10:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:10:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:37 smithi098 ceph-mon[103124]: pgmap v7558: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 329 KiB/s wr, 17 op/s 2023-09-10T03:10:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:10:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:10:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13870.217802 secs 2023-09-10T03:10:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13870.217802 secs 2023-09-10T03:10:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13870.217802 secs 2023-09-10T03:10:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:39 smithi170 ceph-mon[102832]: pgmap v7559: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 246 KiB/s wr, 11 op/s 2023-09-10T03:10:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13870.929015 secs 2023-09-10T03:10:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:39 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:10:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:39 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:10:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:39 smithi082 ceph-mon[98694]: pgmap v7559: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 246 KiB/s wr, 11 op/s 2023-09-10T03:10:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13870.929015 secs 2023-09-10T03:10:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:39 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:10:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:39 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:10:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:39 smithi098 ceph-mon[103124]: pgmap v7559: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 246 KiB/s wr, 11 op/s 2023-09-10T03:10:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13870.929015 secs 2023-09-10T03:10:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:39 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:10:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:39 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:10:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:41 smithi170 ceph-mon[102832]: pgmap v7560: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 359 KiB/s wr, 17 op/s 2023-09-10T03:10:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:41 smithi082 ceph-mon[98694]: pgmap v7560: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 359 KiB/s wr, 17 op/s 2023-09-10T03:10:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:41 smithi098 ceph-mon[103124]: pgmap v7560: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 359 KiB/s wr, 17 op/s 2023-09-10T03:10:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:43 smithi170 ceph-mon[102832]: pgmap v7561: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 172 KiB/s wr, 12 op/s 2023-09-10T03:10:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:43 smithi082 ceph-mon[98694]: pgmap v7561: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 172 KiB/s wr, 12 op/s 2023-09-10T03:10:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:43 smithi098 ceph-mon[103124]: pgmap v7561: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 172 KiB/s wr, 12 op/s 2023-09-10T03:10:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13875.217861 secs 2023-09-10T03:10:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13875.929082 secs 2023-09-10T03:10:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13875.217861 secs 2023-09-10T03:10:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13875.929082 secs 2023-09-10T03:10:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13875.217861 secs 2023-09-10T03:10:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13875.929082 secs 2023-09-10T03:10:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:45 smithi170 ceph-mon[102832]: pgmap v7562: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 172 KiB/s wr, 11 op/s 2023-09-10T03:10:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:45 smithi082 ceph-mon[98694]: pgmap v7562: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 172 KiB/s wr, 11 op/s 2023-09-10T03:10:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:45 smithi098 ceph-mon[103124]: pgmap v7562: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 172 KiB/s wr, 11 op/s 2023-09-10T03:10:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:47 smithi170 ceph-mon[102832]: pgmap v7563: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 365 KiB/s wr, 17 op/s 2023-09-10T03:10:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:47 smithi082 ceph-mon[98694]: pgmap v7563: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 365 KiB/s wr, 17 op/s 2023-09-10T03:10:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:47 smithi098 ceph-mon[103124]: pgmap v7563: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 365 KiB/s wr, 17 op/s 2023-09-10T03:10:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13878.855453 secs 2023-09-10T03:10:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13878.855453 secs 2023-09-10T03:10:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13878.855453 secs 2023-09-10T03:10:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13880.217918 secs 2023-09-10T03:10:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:49 smithi170 ceph-mon[102832]: pgmap v7564: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 306 KiB/s wr, 12 op/s 2023-09-10T03:10:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13880.929147 secs 2023-09-10T03:10:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13880.217918 secs 2023-09-10T03:10:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:49 smithi082 ceph-mon[98694]: pgmap v7564: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 306 KiB/s wr, 12 op/s 2023-09-10T03:10:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13880.929147 secs 2023-09-10T03:10:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13880.217918 secs 2023-09-10T03:10:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:49 smithi098 ceph-mon[103124]: pgmap v7564: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 306 KiB/s wr, 12 op/s 2023-09-10T03:10:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13880.929147 secs 2023-09-10T03:10:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:51 smithi170 ceph-mon[102832]: pgmap v7565: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 340 KiB/s wr, 16 op/s 2023-09-10T03:10:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:51 smithi082 ceph-mon[98694]: pgmap v7565: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 340 KiB/s wr, 16 op/s 2023-09-10T03:10:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:51 smithi098 ceph-mon[103124]: pgmap v7565: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 340 KiB/s wr, 16 op/s 2023-09-10T03:10:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:10:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:10:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:10:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:10:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:10:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:10:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:10:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:10:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:10:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13883.855514 secs 2023-09-10T03:10:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:53 smithi170 ceph-mon[102832]: pgmap v7566: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T03:10:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13883.855514 secs 2023-09-10T03:10:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:53 smithi082 ceph-mon[98694]: pgmap v7566: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T03:10:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13883.855514 secs 2023-09-10T03:10:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:53 smithi098 ceph-mon[103124]: pgmap v7566: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T03:10:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13885.217974 secs 2023-09-10T03:10:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:10:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:10:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13885.929214 secs 2023-09-10T03:10:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13885.217974 secs 2023-09-10T03:10:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:10:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:10:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13885.929214 secs 2023-09-10T03:10:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13885.217974 secs 2023-09-10T03:10:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:10:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:10:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13885.929214 secs 2023-09-10T03:10:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:55 smithi170 ceph-mon[102832]: pgmap v7567: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 10 op/s 2023-09-10T03:10:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:55 smithi082 ceph-mon[98694]: pgmap v7567: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 10 op/s 2023-09-10T03:10:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:55 smithi098 ceph-mon[103124]: pgmap v7567: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 10 op/s 2023-09-10T03:10:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:10:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:10:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:10:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:10:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:10:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:10:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:57 smithi170 ceph-mon[102832]: pgmap v7568: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 284 KiB/s wr, 16 op/s 2023-09-10T03:10:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:57 smithi082 ceph-mon[98694]: pgmap v7568: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 284 KiB/s wr, 16 op/s 2023-09-10T03:10:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:57 smithi098 ceph-mon[103124]: pgmap v7568: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 284 KiB/s wr, 16 op/s 2023-09-10T03:10:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13888.855573 secs 2023-09-10T03:10:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13888.855573 secs 2023-09-10T03:10:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13888.855573 secs 2023-09-10T03:10:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13890.218032 secs 2023-09-10T03:10:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:59 smithi082 ceph-mon[98694]: pgmap v7569: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 91 KiB/s wr, 9 op/s 2023-09-10T03:10:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:10:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13890.929282 secs 2023-09-10T03:10:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13890.218032 secs 2023-09-10T03:10:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:59 smithi098 ceph-mon[103124]: pgmap v7569: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 91 KiB/s wr, 9 op/s 2023-09-10T03:10:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:10:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13890.929282 secs 2023-09-10T03:10:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13890.218032 secs 2023-09-10T03:10:59.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:59 smithi170 ceph-mon[102832]: pgmap v7569: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 91 KiB/s wr, 9 op/s 2023-09-10T03:10:59.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:10:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13890.929282 secs 2023-09-10T03:11:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:01 smithi082 ceph-mon[98694]: pgmap v7570: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 130 KiB/s wr, 14 op/s 2023-09-10T03:11:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:01 smithi098 ceph-mon[103124]: pgmap v7570: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 130 KiB/s wr, 14 op/s 2023-09-10T03:11:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:01 smithi170 ceph-mon[102832]: pgmap v7570: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 130 KiB/s wr, 14 op/s 2023-09-10T03:11:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13893.855636 secs 2023-09-10T03:11:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:03 smithi170 ceph-mon[102832]: pgmap v7571: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T03:11:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13893.855636 secs 2023-09-10T03:11:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:03 smithi082 ceph-mon[98694]: pgmap v7571: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T03:11:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13893.855636 secs 2023-09-10T03:11:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:03 smithi098 ceph-mon[103124]: pgmap v7571: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T03:11:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13895.218090 secs 2023-09-10T03:11:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13895.929348 secs 2023-09-10T03:11:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13895.218090 secs 2023-09-10T03:11:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13895.929348 secs 2023-09-10T03:11:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13895.218090 secs 2023-09-10T03:11:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13895.929348 secs 2023-09-10T03:11:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:05 smithi170 ceph-mon[102832]: pgmap v7572: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T03:11:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:05 smithi082 ceph-mon[98694]: pgmap v7572: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T03:11:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:05 smithi098 ceph-mon[103124]: pgmap v7572: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T03:11:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:07 smithi170 ceph-mon[102832]: pgmap v7573: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 152 KiB/s wr, 30 op/s 2023-09-10T03:11:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:07 smithi082 ceph-mon[98694]: pgmap v7573: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 152 KiB/s wr, 30 op/s 2023-09-10T03:11:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:07 smithi098 ceph-mon[103124]: pgmap v7573: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 152 KiB/s wr, 30 op/s 2023-09-10T03:11:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13898.855695 secs 2023-09-10T03:11:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:08 smithi170 ceph-mon[102832]: osdmap e541: 12 total, 12 up, 12 in 2023-09-10T03:11:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13898.855695 secs 2023-09-10T03:11:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:08 smithi082 ceph-mon[98694]: osdmap e541: 12 total, 12 up, 12 in 2023-09-10T03:11:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13898.855695 secs 2023-09-10T03:11:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:08 smithi098 ceph-mon[103124]: osdmap e541: 12 total, 12 up, 12 in 2023-09-10T03:11:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13900.218173 secs 2023-09-10T03:11:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:09 smithi170 ceph-mon[102832]: pgmap v7575: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 114 KiB/s wr, 30 op/s 2023-09-10T03:11:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13900.929415 secs 2023-09-10T03:11:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13900.218173 secs 2023-09-10T03:11:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:09 smithi082 ceph-mon[98694]: pgmap v7575: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 114 KiB/s wr, 30 op/s 2023-09-10T03:11:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13900.929415 secs 2023-09-10T03:11:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13900.218173 secs 2023-09-10T03:11:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:09 smithi098 ceph-mon[103124]: pgmap v7575: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 114 KiB/s wr, 30 op/s 2023-09-10T03:11:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13900.929415 secs 2023-09-10T03:11:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:11 smithi170 ceph-mon[102832]: pgmap v7576: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T03:11:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:11 smithi082 ceph-mon[98694]: pgmap v7576: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T03:11:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:11 smithi098 ceph-mon[103124]: pgmap v7576: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T03:11:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13903.855797 secs 2023-09-10T03:11:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:13 smithi170 ceph-mon[102832]: pgmap v7577: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T03:11:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13903.855797 secs 2023-09-10T03:11:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:13 smithi082 ceph-mon[98694]: pgmap v7577: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T03:11:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13903.855797 secs 2023-09-10T03:11:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:13 smithi098 ceph-mon[103124]: pgmap v7577: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T03:11:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13905.218234 secs 2023-09-10T03:11:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13905.929485 secs 2023-09-10T03:11:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13905.218234 secs 2023-09-10T03:11:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13905.929485 secs 2023-09-10T03:11:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13905.218234 secs 2023-09-10T03:11:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13905.929485 secs 2023-09-10T03:11:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:15 smithi170 ceph-mon[102832]: pgmap v7578: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T03:11:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:15 smithi082 ceph-mon[98694]: pgmap v7578: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T03:11:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:15 smithi098 ceph-mon[103124]: pgmap v7578: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 102 KiB/s wr, 29 op/s 2023-09-10T03:11:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:17 smithi170 ceph-mon[102832]: pgmap v7579: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T03:11:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:17 smithi170 ceph-mon[102832]: osdmap e542: 12 total, 12 up, 12 in 2023-09-10T03:11:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:17 smithi082 ceph-mon[98694]: pgmap v7579: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T03:11:17.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:17 smithi082 ceph-mon[98694]: osdmap e542: 12 total, 12 up, 12 in 2023-09-10T03:11:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:17 smithi098 ceph-mon[103124]: pgmap v7579: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T03:11:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:17 smithi098 ceph-mon[103124]: osdmap e542: 12 total, 12 up, 12 in 2023-09-10T03:11:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13908.855858 secs 2023-09-10T03:11:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13908.855858 secs 2023-09-10T03:11:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13908.855858 secs 2023-09-10T03:11:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13910.218286 secs 2023-09-10T03:11:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:19 smithi170 ceph-mon[102832]: pgmap v7581: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T03:11:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13910.929550 secs 2023-09-10T03:11:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13910.218286 secs 2023-09-10T03:11:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:19 smithi082 ceph-mon[98694]: pgmap v7581: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T03:11:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13910.929550 secs 2023-09-10T03:11:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13910.218286 secs 2023-09-10T03:11:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:19 smithi098 ceph-mon[103124]: pgmap v7581: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T03:11:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13910.929550 secs 2023-09-10T03:11:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:21 smithi170 ceph-mon[102832]: pgmap v7582: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T03:11:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:21 smithi082 ceph-mon[98694]: pgmap v7582: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T03:11:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:21 smithi098 ceph-mon[103124]: pgmap v7582: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T03:11:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13913.855920 secs 2023-09-10T03:11:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:23 smithi170 ceph-mon[102832]: pgmap v7583: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T03:11:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13913.855920 secs 2023-09-10T03:11:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:23 smithi082 ceph-mon[98694]: pgmap v7583: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T03:11:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13913.855920 secs 2023-09-10T03:11:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:23 smithi098 ceph-mon[103124]: pgmap v7583: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T03:11:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13915.218342 secs 2023-09-10T03:11:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13915.929614 secs 2023-09-10T03:11:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13915.218342 secs 2023-09-10T03:11:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13915.929614 secs 2023-09-10T03:11:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13915.218342 secs 2023-09-10T03:11:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13915.929614 secs 2023-09-10T03:11:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:25 smithi170 ceph-mon[102832]: pgmap v7584: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T03:11:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:25 smithi082 ceph-mon[98694]: pgmap v7584: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T03:11:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:25 smithi098 ceph-mon[103124]: pgmap v7584: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T03:11:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:27 smithi170 ceph-mon[102832]: pgmap v7585: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T03:11:27.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:27 smithi082 ceph-mon[98694]: pgmap v7585: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T03:11:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:27 smithi098 ceph-mon[103124]: pgmap v7585: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T03:11:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13920.218401 secs 2023-09-10T03:11:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:29 smithi170 ceph-mon[102832]: pgmap v7586: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 10 op/s 2023-09-10T03:11:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13920.929677 secs 2023-09-10T03:11:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13920.218401 secs 2023-09-10T03:11:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:29 smithi082 ceph-mon[98694]: pgmap v7586: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 10 op/s 2023-09-10T03:11:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13920.929677 secs 2023-09-10T03:11:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13920.218401 secs 2023-09-10T03:11:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:29 smithi098 ceph-mon[103124]: pgmap v7586: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 10 op/s 2023-09-10T03:11:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13920.929677 secs 2023-09-10T03:11:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:31 smithi170 ceph-mon[102832]: pgmap v7587: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 15 op/s 2023-09-10T03:11:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:11:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:31 smithi082 ceph-mon[98694]: pgmap v7587: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 15 op/s 2023-09-10T03:11:31.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:11:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:31 smithi098 ceph-mon[103124]: pgmap v7587: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 15 op/s 2023-09-10T03:11:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:11:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:11:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:11:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:11:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:11:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:11:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:11:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:11:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:11:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:11:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:33 smithi170 ceph-mon[102832]: pgmap v7588: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T03:11:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:33 smithi082 ceph-mon[98694]: pgmap v7588: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T03:11:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:33 smithi098 ceph-mon[103124]: pgmap v7588: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T03:11:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13925.218458 secs 2023-09-10T03:11:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13925.929779 secs 2023-09-10T03:11:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13925.218458 secs 2023-09-10T03:11:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13925.929779 secs 2023-09-10T03:11:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13925.218458 secs 2023-09-10T03:11:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13925.929779 secs 2023-09-10T03:11:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:35 smithi170 ceph-mon[102832]: pgmap v7589: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T03:11:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:35 smithi082 ceph-mon[98694]: pgmap v7589: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T03:11:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:35 smithi098 ceph-mon[103124]: pgmap v7589: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T03:11:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:11:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:11:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:11:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:11:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:11:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:11:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:11:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:11:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:11:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:37 smithi170 ceph-mon[102832]: pgmap v7590: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 187 KiB/s wr, 17 op/s 2023-09-10T03:11:37.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:11:37.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:11:37.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:11:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:37 smithi082 ceph-mon[98694]: pgmap v7590: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 187 KiB/s wr, 17 op/s 2023-09-10T03:11:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:11:37.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:11:37.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:11:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:37 smithi098 ceph-mon[103124]: pgmap v7590: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 187 KiB/s wr, 17 op/s 2023-09-10T03:11:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:11:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:11:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:11:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:11:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:11:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:11:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:11:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:11:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:11:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13930.218540 secs 2023-09-10T03:11:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:39 smithi170 ceph-mon[102832]: pgmap v7591: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T03:11:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13930.929846 secs 2023-09-10T03:11:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13930.218540 secs 2023-09-10T03:11:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:39 smithi082 ceph-mon[98694]: pgmap v7591: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T03:11:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13930.929846 secs 2023-09-10T03:11:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13930.218540 secs 2023-09-10T03:11:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:39 smithi098 ceph-mon[103124]: pgmap v7591: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T03:11:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13930.929846 secs 2023-09-10T03:11:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:11:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:11:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:11:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:11:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:11:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:11:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:41 smithi082 ceph-mon[98694]: pgmap v7592: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 296 KiB/s wr, 18 op/s 2023-09-10T03:11:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:41 smithi098 ceph-mon[103124]: pgmap v7592: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 296 KiB/s wr, 18 op/s 2023-09-10T03:11:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:41 smithi170 ceph-mon[102832]: pgmap v7592: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 296 KiB/s wr, 18 op/s 2023-09-10T03:11:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:43 smithi170 ceph-mon[102832]: pgmap v7593: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 207 KiB/s wr, 13 op/s 2023-09-10T03:11:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:43 smithi082 ceph-mon[98694]: pgmap v7593: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 207 KiB/s wr, 13 op/s 2023-09-10T03:11:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:43 smithi098 ceph-mon[103124]: pgmap v7593: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 207 KiB/s wr, 13 op/s 2023-09-10T03:11:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13935.218597 secs 2023-09-10T03:11:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13935.929910 secs 2023-09-10T03:11:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13935.218597 secs 2023-09-10T03:11:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13935.929910 secs 2023-09-10T03:11:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13935.218597 secs 2023-09-10T03:11:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13935.929910 secs 2023-09-10T03:11:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:45 smithi170 ceph-mon[102832]: pgmap v7594: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 207 KiB/s wr, 13 op/s 2023-09-10T03:11:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:45 smithi082 ceph-mon[98694]: pgmap v7594: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 207 KiB/s wr, 13 op/s 2023-09-10T03:11:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:45 smithi098 ceph-mon[103124]: pgmap v7594: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 207 KiB/s wr, 13 op/s 2023-09-10T03:11:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:47 smithi082 ceph-mon[98694]: pgmap v7595: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 312 KiB/s wr, 18 op/s 2023-09-10T03:11:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:47 smithi098 ceph-mon[103124]: pgmap v7595: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 312 KiB/s wr, 18 op/s 2023-09-10T03:11:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:47 smithi170 ceph-mon[102832]: pgmap v7595: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 312 KiB/s wr, 18 op/s 2023-09-10T03:11:49.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13940.218654 secs 2023-09-10T03:11:49.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:49 smithi082 ceph-mon[98694]: pgmap v7596: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 280 KiB/s wr, 12 op/s 2023-09-10T03:11:49.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13940.929973 secs 2023-09-10T03:11:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13940.218654 secs 2023-09-10T03:11:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:49 smithi098 ceph-mon[103124]: pgmap v7596: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 280 KiB/s wr, 12 op/s 2023-09-10T03:11:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13940.929973 secs 2023-09-10T03:11:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13940.218654 secs 2023-09-10T03:11:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:49 smithi170 ceph-mon[102832]: pgmap v7596: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 280 KiB/s wr, 12 op/s 2023-09-10T03:11:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13940.929973 secs 2023-09-10T03:11:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:51 smithi082 ceph-mon[98694]: pgmap v7597: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 452 KiB/s wr, 19 op/s 2023-09-10T03:11:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:51 smithi098 ceph-mon[103124]: pgmap v7597: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 452 KiB/s wr, 19 op/s 2023-09-10T03:11:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:51 smithi170 ceph-mon[102832]: pgmap v7597: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 452 KiB/s wr, 19 op/s 2023-09-10T03:11:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13943.856379 secs 2023-09-10T03:11:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:53 smithi170 ceph-mon[102832]: pgmap v7598: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 13 op/s 2023-09-10T03:11:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13943.856379 secs 2023-09-10T03:11:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:53 smithi082 ceph-mon[98694]: pgmap v7598: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 13 op/s 2023-09-10T03:11:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13943.856379 secs 2023-09-10T03:11:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:53 smithi098 ceph-mon[103124]: pgmap v7598: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 13 op/s 2023-09-10T03:11:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13945.218714 secs 2023-09-10T03:11:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:11:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:11:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13945.930042 secs 2023-09-10T03:11:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13945.218714 secs 2023-09-10T03:11:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:11:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:11:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13945.930042 secs 2023-09-10T03:11:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13945.218714 secs 2023-09-10T03:11:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:11:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:11:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13945.930042 secs 2023-09-10T03:11:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:55 smithi170 ceph-mon[102832]: pgmap v7599: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T03:11:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:55 smithi082 ceph-mon[98694]: pgmap v7599: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T03:11:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:55 smithi098 ceph-mon[103124]: pgmap v7599: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T03:11:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:11:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:11:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:11:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:11:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:11:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:11:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:11:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:11:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:11:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:57 smithi170 ceph-mon[102832]: pgmap v7600: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 313 KiB/s wr, 19 op/s 2023-09-10T03:11:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:57 smithi082 ceph-mon[98694]: pgmap v7600: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 313 KiB/s wr, 19 op/s 2023-09-10T03:11:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:57 smithi098 ceph-mon[103124]: pgmap v7600: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 313 KiB/s wr, 19 op/s 2023-09-10T03:11:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13948.856440 secs 2023-09-10T03:11:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13948.856440 secs 2023-09-10T03:11:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13948.856440 secs 2023-09-10T03:11:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13950.218773 secs 2023-09-10T03:11:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:59 smithi170 ceph-mon[102832]: pgmap v7601: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 208 KiB/s wr, 13 op/s 2023-09-10T03:11:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:11:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13950.930105 secs 2023-09-10T03:11:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13950.218773 secs 2023-09-10T03:11:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:59 smithi082 ceph-mon[98694]: pgmap v7601: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 208 KiB/s wr, 13 op/s 2023-09-10T03:11:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:11:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13950.930105 secs 2023-09-10T03:11:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13950.218773 secs 2023-09-10T03:11:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:59 smithi098 ceph-mon[103124]: pgmap v7601: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 208 KiB/s wr, 13 op/s 2023-09-10T03:11:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:11:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13950.930105 secs 2023-09-10T03:12:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:01 smithi170 ceph-mon[102832]: pgmap v7602: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 309 KiB/s wr, 18 op/s 2023-09-10T03:12:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:01 smithi082 ceph-mon[98694]: pgmap v7602: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 309 KiB/s wr, 18 op/s 2023-09-10T03:12:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:01 smithi098 ceph-mon[103124]: pgmap v7602: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 309 KiB/s wr, 18 op/s 2023-09-10T03:12:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13953.856570 secs 2023-09-10T03:12:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:03 smithi170 ceph-mon[102832]: pgmap v7603: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T03:12:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13953.856570 secs 2023-09-10T03:12:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:03 smithi082 ceph-mon[98694]: pgmap v7603: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T03:12:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13953.856570 secs 2023-09-10T03:12:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:03 smithi098 ceph-mon[103124]: pgmap v7603: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T03:12:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13955.218830 secs 2023-09-10T03:12:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13955.930171 secs 2023-09-10T03:12:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13955.218830 secs 2023-09-10T03:12:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13955.930171 secs 2023-09-10T03:12:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13955.218830 secs 2023-09-10T03:12:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13955.930171 secs 2023-09-10T03:12:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:05 smithi170 ceph-mon[102832]: pgmap v7604: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 11 op/s 2023-09-10T03:12:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:05 smithi082 ceph-mon[98694]: pgmap v7604: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 11 op/s 2023-09-10T03:12:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:05 smithi098 ceph-mon[103124]: pgmap v7604: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 11 op/s 2023-09-10T03:12:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:07 smithi170 ceph-mon[102832]: pgmap v7605: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 182 KiB/s wr, 30 op/s 2023-09-10T03:12:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:07 smithi082 ceph-mon[98694]: pgmap v7605: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 182 KiB/s wr, 30 op/s 2023-09-10T03:12:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:07 smithi098 ceph-mon[103124]: pgmap v7605: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 182 KiB/s wr, 30 op/s 2023-09-10T03:12:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13958.856666 secs 2023-09-10T03:12:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:08 smithi170 ceph-mon[102832]: osdmap e543: 12 total, 12 up, 12 in 2023-09-10T03:12:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13958.856666 secs 2023-09-10T03:12:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:08 smithi082 ceph-mon[98694]: osdmap e543: 12 total, 12 up, 12 in 2023-09-10T03:12:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13958.856666 secs 2023-09-10T03:12:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:08 smithi098 ceph-mon[103124]: osdmap e543: 12 total, 12 up, 12 in 2023-09-10T03:12:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13960.218888 secs 2023-09-10T03:12:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:09 smithi170 ceph-mon[102832]: pgmap v7607: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 176 KiB/s wr, 29 op/s 2023-09-10T03:12:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13960.930235 secs 2023-09-10T03:12:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13960.218888 secs 2023-09-10T03:12:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:09 smithi082 ceph-mon[98694]: pgmap v7607: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 176 KiB/s wr, 29 op/s 2023-09-10T03:12:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13960.930235 secs 2023-09-10T03:12:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13960.218888 secs 2023-09-10T03:12:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:09 smithi098 ceph-mon[103124]: pgmap v7607: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 176 KiB/s wr, 29 op/s 2023-09-10T03:12:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13960.930235 secs 2023-09-10T03:12:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:11 smithi170 ceph-mon[102832]: pgmap v7608: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T03:12:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:11 smithi082 ceph-mon[98694]: pgmap v7608: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T03:12:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:11 smithi098 ceph-mon[103124]: pgmap v7608: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T03:12:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13963.856726 secs 2023-09-10T03:12:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:13 smithi170 ceph-mon[102832]: pgmap v7609: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T03:12:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13963.856726 secs 2023-09-10T03:12:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:13 smithi082 ceph-mon[98694]: pgmap v7609: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T03:12:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13963.856726 secs 2023-09-10T03:12:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:13 smithi098 ceph-mon[103124]: pgmap v7609: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T03:12:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13965.218950 secs 2023-09-10T03:12:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13965.930302 secs 2023-09-10T03:12:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13965.218950 secs 2023-09-10T03:12:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13965.930302 secs 2023-09-10T03:12:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13965.218950 secs 2023-09-10T03:12:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13965.930302 secs 2023-09-10T03:12:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:15 smithi170 ceph-mon[102832]: pgmap v7610: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T03:12:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:15 smithi082 ceph-mon[98694]: pgmap v7610: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T03:12:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:15 smithi098 ceph-mon[103124]: pgmap v7610: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T03:12:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:17 smithi098 ceph-mon[103124]: pgmap v7611: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-10T03:12:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:17 smithi098 ceph-mon[103124]: osdmap e544: 12 total, 12 up, 12 in 2023-09-10T03:12:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:17 smithi170 ceph-mon[102832]: pgmap v7611: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-10T03:12:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:17 smithi170 ceph-mon[102832]: osdmap e544: 12 total, 12 up, 12 in 2023-09-10T03:12:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:17 smithi082 ceph-mon[98694]: pgmap v7611: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-10T03:12:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:17 smithi082 ceph-mon[98694]: osdmap e544: 12 total, 12 up, 12 in 2023-09-10T03:12:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13968.856803 secs 2023-09-10T03:12:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13968.856803 secs 2023-09-10T03:12:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13968.856803 secs 2023-09-10T03:12:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13970.219048 secs 2023-09-10T03:12:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:19 smithi098 ceph-mon[103124]: pgmap v7613: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-10T03:12:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13970.930369 secs 2023-09-10T03:12:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13970.219048 secs 2023-09-10T03:12:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:19 smithi170 ceph-mon[102832]: pgmap v7613: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-10T03:12:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13970.930369 secs 2023-09-10T03:12:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13970.219048 secs 2023-09-10T03:12:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:19 smithi082 ceph-mon[98694]: pgmap v7613: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-10T03:12:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13970.930369 secs 2023-09-10T03:12:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:21 smithi098 ceph-mon[103124]: pgmap v7614: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T03:12:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:21 smithi170 ceph-mon[102832]: pgmap v7614: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T03:12:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:21 smithi082 ceph-mon[98694]: pgmap v7614: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T03:12:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13973.856866 secs 2023-09-10T03:12:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:23 smithi170 ceph-mon[102832]: pgmap v7615: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T03:12:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13973.856866 secs 2023-09-10T03:12:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:23 smithi082 ceph-mon[98694]: pgmap v7615: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T03:12:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13973.856866 secs 2023-09-10T03:12:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:23 smithi098 ceph-mon[103124]: pgmap v7615: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T03:12:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13975.219102 secs 2023-09-10T03:12:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13975.930434 secs 2023-09-10T03:12:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13975.219102 secs 2023-09-10T03:12:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13975.930434 secs 2023-09-10T03:12:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13975.219102 secs 2023-09-10T03:12:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13975.930434 secs 2023-09-10T03:12:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:25 smithi170 ceph-mon[102832]: pgmap v7616: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T03:12:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:25 smithi082 ceph-mon[98694]: pgmap v7616: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T03:12:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:25 smithi098 ceph-mon[103124]: pgmap v7616: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T03:12:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:27 smithi170 ceph-mon[102832]: pgmap v7617: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 164 KiB/s wr, 12 op/s 2023-09-10T03:12:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:27 smithi082 ceph-mon[98694]: pgmap v7617: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 164 KiB/s wr, 12 op/s 2023-09-10T03:12:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:27 smithi098 ceph-mon[103124]: pgmap v7617: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 164 KiB/s wr, 12 op/s 2023-09-10T03:12:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13980.219161 secs 2023-09-10T03:12:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:29 smithi170 ceph-mon[102832]: pgmap v7618: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T03:12:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13980.930489 secs 2023-09-10T03:12:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13980.219161 secs 2023-09-10T03:12:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:29 smithi082 ceph-mon[98694]: pgmap v7618: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T03:12:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13980.930489 secs 2023-09-10T03:12:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13980.219161 secs 2023-09-10T03:12:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:29 smithi098 ceph-mon[103124]: pgmap v7618: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T03:12:29.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13980.930489 secs 2023-09-10T03:12:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:12:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:12:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:12:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:12:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:12:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:12:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:31 smithi082 ceph-mon[98694]: pgmap v7619: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 318 KiB/s wr, 17 op/s 2023-09-10T03:12:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:12:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:31 smithi098 ceph-mon[103124]: pgmap v7619: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 318 KiB/s wr, 17 op/s 2023-09-10T03:12:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:12:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:31 smithi170 ceph-mon[102832]: pgmap v7619: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 318 KiB/s wr, 17 op/s 2023-09-10T03:12:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:12:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:12:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:12:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:12:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:12:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:12:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:12:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:12:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:12:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:12:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:33 smithi082 ceph-mon[98694]: pgmap v7620: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 274 KiB/s wr, 12 op/s 2023-09-10T03:12:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:33 smithi098 ceph-mon[103124]: pgmap v7620: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 274 KiB/s wr, 12 op/s 2023-09-10T03:12:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:33 smithi170 ceph-mon[102832]: pgmap v7620: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 274 KiB/s wr, 12 op/s 2023-09-10T03:12:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13985.219226 secs 2023-09-10T03:12:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13985.930565 secs 2023-09-10T03:12:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13985.219226 secs 2023-09-10T03:12:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13985.930565 secs 2023-09-10T03:12:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13985.219226 secs 2023-09-10T03:12:34.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13985.930565 secs 2023-09-10T03:12:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:35 smithi082 ceph-mon[98694]: pgmap v7621: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 274 KiB/s wr, 11 op/s 2023-09-10T03:12:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:35 smithi098 ceph-mon[103124]: pgmap v7621: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 274 KiB/s wr, 11 op/s 2023-09-10T03:12:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:35 smithi170 ceph-mon[102832]: pgmap v7621: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 274 KiB/s wr, 11 op/s 2023-09-10T03:12:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:37 smithi082 ceph-mon[98694]: pgmap v7622: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 378 KiB/s wr, 17 op/s 2023-09-10T03:12:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:12:37.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:12:37.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:12:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:37 smithi098 ceph-mon[103124]: pgmap v7622: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 378 KiB/s wr, 17 op/s 2023-09-10T03:12:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:12:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:12:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:12:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:37 smithi170 ceph-mon[102832]: pgmap v7622: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 378 KiB/s wr, 17 op/s 2023-09-10T03:12:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:12:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:12:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:12:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:12:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:12:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:12:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:12:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:12:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:12:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:12:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:12:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:12:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:12:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:12:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:12:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:12:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:12:38.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:12:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13990.219315 secs 2023-09-10T03:12:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:39 smithi098 ceph-mon[103124]: pgmap v7623: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T03:12:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13990.930624 secs 2023-09-10T03:12:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13990.219315 secs 2023-09-10T03:12:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:39 smithi170 ceph-mon[102832]: pgmap v7623: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T03:12:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13990.930624 secs 2023-09-10T03:12:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13990.219315 secs 2023-09-10T03:12:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:39 smithi082 ceph-mon[98694]: pgmap v7623: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T03:12:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13990.930624 secs 2023-09-10T03:12:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:41 smithi098 ceph-mon[103124]: pgmap v7624: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 411 KiB/s wr, 18 op/s 2023-09-10T03:12:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:41 smithi170 ceph-mon[102832]: pgmap v7624: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 411 KiB/s wr, 18 op/s 2023-09-10T03:12:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:41 smithi082 ceph-mon[98694]: pgmap v7624: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 411 KiB/s wr, 18 op/s 2023-09-10T03:12:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:43 smithi170 ceph-mon[102832]: pgmap v7625: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T03:12:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:43 smithi082 ceph-mon[98694]: pgmap v7625: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T03:12:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:43 smithi098 ceph-mon[103124]: pgmap v7625: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T03:12:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13995.219398 secs 2023-09-10T03:12:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:12:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:12:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 13995.930690 secs 2023-09-10T03:12:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13995.219398 secs 2023-09-10T03:12:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:12:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:12:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 13995.930690 secs 2023-09-10T03:12:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13995.219398 secs 2023-09-10T03:12:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:12:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:12:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 13995.930690 secs 2023-09-10T03:12:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:45 smithi170 ceph-mon[102832]: pgmap v7626: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T03:12:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:45 smithi082 ceph-mon[98694]: pgmap v7626: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T03:12:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:45 smithi098 ceph-mon[103124]: pgmap v7626: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T03:12:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:12:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:12:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:12:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:12:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:12:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:12:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:47 smithi170 ceph-mon[102832]: pgmap v7627: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 336 KiB/s wr, 17 op/s 2023-09-10T03:12:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:47 smithi082 ceph-mon[98694]: pgmap v7627: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 336 KiB/s wr, 17 op/s 2023-09-10T03:12:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:47 smithi098 ceph-mon[103124]: pgmap v7627: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 336 KiB/s wr, 17 op/s 2023-09-10T03:12:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 13998.857240 secs 2023-09-10T03:12:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 13998.857240 secs 2023-09-10T03:12:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 13998.857240 secs 2023-09-10T03:12:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14000.219457 secs 2023-09-10T03:12:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:49 smithi170 ceph-mon[102832]: pgmap v7628: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 232 KiB/s wr, 11 op/s 2023-09-10T03:12:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14000.930756 secs 2023-09-10T03:12:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14000.219457 secs 2023-09-10T03:12:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:49 smithi082 ceph-mon[98694]: pgmap v7628: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 232 KiB/s wr, 11 op/s 2023-09-10T03:12:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14000.930756 secs 2023-09-10T03:12:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14000.219457 secs 2023-09-10T03:12:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:49 smithi098 ceph-mon[103124]: pgmap v7628: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 232 KiB/s wr, 11 op/s 2023-09-10T03:12:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14000.930756 secs 2023-09-10T03:12:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:51 smithi082 ceph-mon[98694]: pgmap v7629: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 256 KiB/s wr, 16 op/s 2023-09-10T03:12:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:51 smithi098 ceph-mon[103124]: pgmap v7629: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 256 KiB/s wr, 16 op/s 2023-09-10T03:12:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:51 smithi170 ceph-mon[102832]: pgmap v7629: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 256 KiB/s wr, 16 op/s 2023-09-10T03:12:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:12:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:12:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:12:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:12:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:12:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:12:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:12:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:12:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:12:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14003.857302 secs 2023-09-10T03:12:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:53 smithi082 ceph-mon[98694]: pgmap v7630: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 129 KiB/s wr, 10 op/s 2023-09-10T03:12:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14003.857302 secs 2023-09-10T03:12:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:53 smithi098 ceph-mon[103124]: pgmap v7630: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 129 KiB/s wr, 10 op/s 2023-09-10T03:12:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14003.857302 secs 2023-09-10T03:12:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:53 smithi170 ceph-mon[102832]: pgmap v7630: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 129 KiB/s wr, 10 op/s 2023-09-10T03:12:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14005.219493 secs 2023-09-10T03:12:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14005.930820 secs 2023-09-10T03:12:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14005.219493 secs 2023-09-10T03:12:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14005.930820 secs 2023-09-10T03:12:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14005.219493 secs 2023-09-10T03:12:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14005.930820 secs 2023-09-10T03:12:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:55 smithi082 ceph-mon[98694]: pgmap v7631: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 129 KiB/s wr, 10 op/s 2023-09-10T03:12:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:55 smithi098 ceph-mon[103124]: pgmap v7631: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 129 KiB/s wr, 10 op/s 2023-09-10T03:12:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:55 smithi170 ceph-mon[102832]: pgmap v7631: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 129 KiB/s wr, 10 op/s 2023-09-10T03:12:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:57 smithi082 ceph-mon[98694]: pgmap v7632: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 226 KiB/s wr, 15 op/s 2023-09-10T03:12:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:57 smithi098 ceph-mon[103124]: pgmap v7632: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 226 KiB/s wr, 15 op/s 2023-09-10T03:12:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:57 smithi170 ceph-mon[102832]: pgmap v7632: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 226 KiB/s wr, 15 op/s 2023-09-10T03:12:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14008.857417 secs 2023-09-10T03:12:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14008.857417 secs 2023-09-10T03:12:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14008.857417 secs 2023-09-10T03:12:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14010.219554 secs 2023-09-10T03:12:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:59 smithi098 ceph-mon[103124]: pgmap v7633: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 9 op/s 2023-09-10T03:12:59.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:12:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14010.930890 secs 2023-09-10T03:12:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14010.219554 secs 2023-09-10T03:12:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:59 smithi170 ceph-mon[102832]: pgmap v7633: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 9 op/s 2023-09-10T03:12:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:12:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14010.930890 secs 2023-09-10T03:12:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14010.219554 secs 2023-09-10T03:12:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:59 smithi082 ceph-mon[98694]: pgmap v7633: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 9 op/s 2023-09-10T03:12:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:12:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14010.930890 secs 2023-09-10T03:13:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:01 smithi098 ceph-mon[103124]: pgmap v7634: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 205 KiB/s wr, 29 op/s 2023-09-10T03:13:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:01 smithi170 ceph-mon[102832]: pgmap v7634: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 205 KiB/s wr, 29 op/s 2023-09-10T03:13:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:01 smithi082 ceph-mon[98694]: pgmap v7634: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 205 KiB/s wr, 29 op/s 2023-09-10T03:13:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14013.857500 secs 2023-09-10T03:13:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:03 smithi170 ceph-mon[102832]: pgmap v7635: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 181 KiB/s wr, 24 op/s 2023-09-10T03:13:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14013.857500 secs 2023-09-10T03:13:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:03 smithi082 ceph-mon[98694]: pgmap v7635: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 181 KiB/s wr, 24 op/s 2023-09-10T03:13:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14013.857500 secs 2023-09-10T03:13:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:03 smithi098 ceph-mon[103124]: pgmap v7635: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 181 KiB/s wr, 24 op/s 2023-09-10T03:13:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14015.219631 secs 2023-09-10T03:13:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14015.930955 secs 2023-09-10T03:13:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14015.219631 secs 2023-09-10T03:13:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14015.930955 secs 2023-09-10T03:13:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14015.219631 secs 2023-09-10T03:13:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14015.930955 secs 2023-09-10T03:13:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:05 smithi170 ceph-mon[102832]: pgmap v7636: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 181 KiB/s wr, 24 op/s 2023-09-10T03:13:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:05 smithi082 ceph-mon[98694]: pgmap v7636: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 181 KiB/s wr, 24 op/s 2023-09-10T03:13:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:05 smithi098 ceph-mon[103124]: pgmap v7636: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 181 KiB/s wr, 24 op/s 2023-09-10T03:13:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:07 smithi098 ceph-mon[103124]: pgmap v7637: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 193 KiB/s wr, 29 op/s 2023-09-10T03:13:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:07 smithi170 ceph-mon[102832]: pgmap v7637: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 193 KiB/s wr, 29 op/s 2023-09-10T03:13:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:07 smithi082 ceph-mon[98694]: pgmap v7637: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 193 KiB/s wr, 29 op/s 2023-09-10T03:13:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14018.857558 secs 2023-09-10T03:13:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:08 smithi098 ceph-mon[103124]: osdmap e545: 12 total, 12 up, 12 in 2023-09-10T03:13:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14018.857558 secs 2023-09-10T03:13:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:08 smithi170 ceph-mon[102832]: osdmap e545: 12 total, 12 up, 12 in 2023-09-10T03:13:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14018.857558 secs 2023-09-10T03:13:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:08 smithi082 ceph-mon[98694]: osdmap e545: 12 total, 12 up, 12 in 2023-09-10T03:13:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14020.219689 secs 2023-09-10T03:13:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:09 smithi098 ceph-mon[103124]: pgmap v7639: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 115 KiB/s wr, 29 op/s 2023-09-10T03:13:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14020.931019 secs 2023-09-10T03:13:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14020.219689 secs 2023-09-10T03:13:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:09 smithi170 ceph-mon[102832]: pgmap v7639: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 115 KiB/s wr, 29 op/s 2023-09-10T03:13:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14020.931019 secs 2023-09-10T03:13:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14020.219689 secs 2023-09-10T03:13:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:09 smithi082 ceph-mon[98694]: pgmap v7639: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 115 KiB/s wr, 29 op/s 2023-09-10T03:13:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14020.931019 secs 2023-09-10T03:13:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:11 smithi098 ceph-mon[103124]: pgmap v7640: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-09-10T03:13:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:11 smithi170 ceph-mon[102832]: pgmap v7640: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-09-10T03:13:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:11 smithi082 ceph-mon[98694]: pgmap v7640: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-09-10T03:13:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14023.857664 secs 2023-09-10T03:13:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:13 smithi170 ceph-mon[102832]: osdmap e546: 12 total, 12 up, 12 in 2023-09-10T03:13:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:13 smithi170 ceph-mon[102832]: pgmap v7642: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 15 op/s 2023-09-10T03:13:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14023.857664 secs 2023-09-10T03:13:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:13 smithi082 ceph-mon[98694]: osdmap e546: 12 total, 12 up, 12 in 2023-09-10T03:13:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:13 smithi082 ceph-mon[98694]: pgmap v7642: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 15 op/s 2023-09-10T03:13:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14023.857664 secs 2023-09-10T03:13:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:13 smithi098 ceph-mon[103124]: osdmap e546: 12 total, 12 up, 12 in 2023-09-10T03:13:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:13 smithi098 ceph-mon[103124]: pgmap v7642: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 15 op/s 2023-09-10T03:13:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14025.219745 secs 2023-09-10T03:13:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14025.931081 secs 2023-09-10T03:13:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14025.219745 secs 2023-09-10T03:13:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14025.931081 secs 2023-09-10T03:13:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14025.219745 secs 2023-09-10T03:13:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14025.931081 secs 2023-09-10T03:13:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:15 smithi170 ceph-mon[102832]: pgmap v7643: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T03:13:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:15 smithi082 ceph-mon[98694]: pgmap v7643: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T03:13:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:15 smithi098 ceph-mon[103124]: pgmap v7643: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T03:13:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:17 smithi082 ceph-mon[98694]: pgmap v7644: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 13 op/s 2023-09-10T03:13:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:17 smithi098 ceph-mon[103124]: pgmap v7644: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 13 op/s 2023-09-10T03:13:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:17 smithi170 ceph-mon[102832]: pgmap v7644: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 13 op/s 2023-09-10T03:13:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14028.857756 secs 2023-09-10T03:13:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14028.857756 secs 2023-09-10T03:13:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14028.857756 secs 2023-09-10T03:13:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14030.219803 secs 2023-09-10T03:13:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:19 smithi082 ceph-mon[98694]: pgmap v7645: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 11 op/s 2023-09-10T03:13:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14030.931148 secs 2023-09-10T03:13:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14030.219803 secs 2023-09-10T03:13:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:19 smithi098 ceph-mon[103124]: pgmap v7645: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 11 op/s 2023-09-10T03:13:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14030.931148 secs 2023-09-10T03:13:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14030.219803 secs 2023-09-10T03:13:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:19 smithi170 ceph-mon[102832]: pgmap v7645: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 11 op/s 2023-09-10T03:13:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14030.931148 secs 2023-09-10T03:13:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:21 smithi082 ceph-mon[98694]: pgmap v7646: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T03:13:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:21 smithi098 ceph-mon[103124]: pgmap v7646: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T03:13:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:21 smithi170 ceph-mon[102832]: pgmap v7646: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T03:13:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:23 smithi170 ceph-mon[102832]: pgmap v7647: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T03:13:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:13:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:23 smithi082 ceph-mon[98694]: pgmap v7647: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T03:13:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:13:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:23 smithi098 ceph-mon[103124]: pgmap v7647: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 11 op/s 2023-09-10T03:13:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:13:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14035.219861 secs 2023-09-10T03:13:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:13:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:13:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:13:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14035.931214 secs 2023-09-10T03:13:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14035.219861 secs 2023-09-10T03:13:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:13:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:13:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:13:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14035.931214 secs 2023-09-10T03:13:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14035.219861 secs 2023-09-10T03:13:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:13:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:13:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:13:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14035.931214 secs 2023-09-10T03:13:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:25 smithi170 ceph-mon[102832]: pgmap v7648: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 9 op/s 2023-09-10T03:13:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:25 smithi082 ceph-mon[98694]: pgmap v7648: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 9 op/s 2023-09-10T03:13:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:25 smithi098 ceph-mon[103124]: pgmap v7648: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 9 op/s 2023-09-10T03:13:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:27 smithi098 ceph-mon[103124]: pgmap v7649: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 15 op/s 2023-09-10T03:13:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:27 smithi170 ceph-mon[102832]: pgmap v7649: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 15 op/s 2023-09-10T03:13:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:27 smithi082 ceph-mon[98694]: pgmap v7649: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 15 op/s 2023-09-10T03:13:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14038.857887 secs 2023-09-10T03:13:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14038.857887 secs 2023-09-10T03:13:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14038.857887 secs 2023-09-10T03:13:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14040.219919 secs 2023-09-10T03:13:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:29 smithi098 ceph-mon[103124]: pgmap v7650: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T03:13:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14040.931278 secs 2023-09-10T03:13:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14040.219919 secs 2023-09-10T03:13:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:29 smithi170 ceph-mon[102832]: pgmap v7650: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T03:13:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14040.931278 secs 2023-09-10T03:13:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14040.219919 secs 2023-09-10T03:13:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:29 smithi082 ceph-mon[98694]: pgmap v7650: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T03:13:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14040.931278 secs 2023-09-10T03:13:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:13:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:13:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:13:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:13:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:13:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:13:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:31 smithi098 ceph-mon[103124]: pgmap v7651: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 157 KiB/s wr, 16 op/s 2023-09-10T03:13:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:31 smithi170 ceph-mon[102832]: pgmap v7651: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 157 KiB/s wr, 16 op/s 2023-09-10T03:13:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:31 smithi082 ceph-mon[98694]: pgmap v7651: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 157 KiB/s wr, 16 op/s 2023-09-10T03:13:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:32 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:13:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:13:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:13:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:32 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:13:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:13:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:13:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:32 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:13:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:13:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:13:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14043.857949 secs 2023-09-10T03:13:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:33 smithi170 ceph-mon[102832]: pgmap v7652: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 157 KiB/s wr, 11 op/s 2023-09-10T03:13:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14043.857949 secs 2023-09-10T03:13:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:33 smithi082 ceph-mon[98694]: pgmap v7652: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 157 KiB/s wr, 11 op/s 2023-09-10T03:13:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14043.857949 secs 2023-09-10T03:13:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:33 smithi098 ceph-mon[103124]: pgmap v7652: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 157 KiB/s wr, 11 op/s 2023-09-10T03:13:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14045.219974 secs 2023-09-10T03:13:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14045.931343 secs 2023-09-10T03:13:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14045.219974 secs 2023-09-10T03:13:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14045.931343 secs 2023-09-10T03:13:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14045.219974 secs 2023-09-10T03:13:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14045.931343 secs 2023-09-10T03:13:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:35 smithi170 ceph-mon[102832]: pgmap v7653: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 157 KiB/s wr, 11 op/s 2023-09-10T03:13:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:35 smithi082 ceph-mon[98694]: pgmap v7653: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 157 KiB/s wr, 11 op/s 2023-09-10T03:13:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:35 smithi098 ceph-mon[103124]: pgmap v7653: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 157 KiB/s wr, 11 op/s 2023-09-10T03:13:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:37 smithi170 ceph-mon[102832]: pgmap v7654: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 276 KiB/s wr, 18 op/s 2023-09-10T03:13:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:37 smithi082 ceph-mon[98694]: pgmap v7654: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 276 KiB/s wr, 18 op/s 2023-09-10T03:13:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:37 smithi098 ceph-mon[103124]: pgmap v7654: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 276 KiB/s wr, 18 op/s 2023-09-10T03:13:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:13:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:13:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:13:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14050.220034 secs 2023-09-10T03:13:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:39 smithi170 ceph-mon[102832]: pgmap v7655: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 209 KiB/s wr, 12 op/s 2023-09-10T03:13:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14050.931412 secs 2023-09-10T03:13:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:13:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:13:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:13:39.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:13:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14050.220034 secs 2023-09-10T03:13:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:39 smithi082 ceph-mon[98694]: pgmap v7655: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 209 KiB/s wr, 12 op/s 2023-09-10T03:13:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14050.931412 secs 2023-09-10T03:13:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:13:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:13:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:13:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:13:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14050.220034 secs 2023-09-10T03:13:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:39 smithi098 ceph-mon[103124]: pgmap v7655: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 209 KiB/s wr, 12 op/s 2023-09-10T03:13:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14050.931412 secs 2023-09-10T03:13:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:13:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:13:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:13:40.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:13:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:13:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:13:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:13:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:13:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:13:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:13:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:13:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:13:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:13:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:41 smithi170 ceph-mon[102832]: pgmap v7656: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 300 KiB/s wr, 18 op/s 2023-09-10T03:13:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:41 smithi082 ceph-mon[98694]: pgmap v7656: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 300 KiB/s wr, 18 op/s 2023-09-10T03:13:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:41 smithi098 ceph-mon[103124]: pgmap v7656: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 300 KiB/s wr, 18 op/s 2023-09-10T03:13:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:43 smithi170 ceph-mon[102832]: pgmap v7657: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 209 KiB/s wr, 12 op/s 2023-09-10T03:13:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:43 smithi082 ceph-mon[98694]: pgmap v7657: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 209 KiB/s wr, 12 op/s 2023-09-10T03:13:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:43 smithi098 ceph-mon[103124]: pgmap v7657: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 209 KiB/s wr, 12 op/s 2023-09-10T03:13:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14055.220089 secs 2023-09-10T03:13:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14055.931481 secs 2023-09-10T03:13:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14055.220089 secs 2023-09-10T03:13:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14055.931481 secs 2023-09-10T03:13:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14055.220089 secs 2023-09-10T03:13:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14055.931481 secs 2023-09-10T03:13:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:45 smithi082 ceph-mon[98694]: pgmap v7658: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 209 KiB/s wr, 12 op/s 2023-09-10T03:13:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:45 smithi098 ceph-mon[103124]: pgmap v7658: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 209 KiB/s wr, 12 op/s 2023-09-10T03:13:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:45 smithi170 ceph-mon[102832]: pgmap v7658: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 209 KiB/s wr, 12 op/s 2023-09-10T03:13:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:47 smithi098 ceph-mon[103124]: pgmap v7659: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 311 KiB/s wr, 18 op/s 2023-09-10T03:13:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:47 smithi170 ceph-mon[102832]: pgmap v7659: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 311 KiB/s wr, 18 op/s 2023-09-10T03:13:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:47 smithi082 ceph-mon[98694]: pgmap v7659: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 311 KiB/s wr, 18 op/s 2023-09-10T03:13:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14060.220170 secs 2023-09-10T03:13:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:49 smithi098 ceph-mon[103124]: pgmap v7660: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T03:13:49.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14060.931547 secs 2023-09-10T03:13:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14060.220170 secs 2023-09-10T03:13:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:49 smithi170 ceph-mon[102832]: pgmap v7660: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T03:13:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14060.931547 secs 2023-09-10T03:13:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14060.220170 secs 2023-09-10T03:13:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:49 smithi082 ceph-mon[98694]: pgmap v7660: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T03:13:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14060.931547 secs 2023-09-10T03:13:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:13:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:13:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:13:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:13:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:13:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:13:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:51 smithi170 ceph-mon[102832]: pgmap v7661: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 361 KiB/s wr, 18 op/s 2023-09-10T03:13:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:51 smithi082 ceph-mon[98694]: pgmap v7661: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 361 KiB/s wr, 18 op/s 2023-09-10T03:13:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:51 smithi098 ceph-mon[103124]: pgmap v7661: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 361 KiB/s wr, 18 op/s 2023-09-10T03:13:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:13:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:13:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:13:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:13:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:13:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:13:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14063.858333 secs 2023-09-10T03:13:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:53 smithi170 ceph-mon[102832]: pgmap v7662: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 271 KiB/s wr, 12 op/s 2023-09-10T03:13:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14063.858333 secs 2023-09-10T03:13:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:53 smithi082 ceph-mon[98694]: pgmap v7662: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 271 KiB/s wr, 12 op/s 2023-09-10T03:13:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14063.858333 secs 2023-09-10T03:13:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:53 smithi098 ceph-mon[103124]: pgmap v7662: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 271 KiB/s wr, 12 op/s 2023-09-10T03:13:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14065.220239 secs 2023-09-10T03:13:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14065.931613 secs 2023-09-10T03:13:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14065.220239 secs 2023-09-10T03:13:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14065.931613 secs 2023-09-10T03:13:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14065.220239 secs 2023-09-10T03:13:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14065.931613 secs 2023-09-10T03:13:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:55 smithi170 ceph-mon[102832]: pgmap v7663: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 271 KiB/s wr, 12 op/s 2023-09-10T03:13:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:55 smithi082 ceph-mon[98694]: pgmap v7663: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 271 KiB/s wr, 12 op/s 2023-09-10T03:13:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:55 smithi098 ceph-mon[103124]: pgmap v7663: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 271 KiB/s wr, 12 op/s 2023-09-10T03:13:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:13:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:13:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:13:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:13:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:13:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:13:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:13:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:13:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:13:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:57 smithi170 ceph-mon[102832]: pgmap v7664: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 271 KiB/s wr, 17 op/s 2023-09-10T03:13:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:57 smithi082 ceph-mon[98694]: pgmap v7664: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 271 KiB/s wr, 17 op/s 2023-09-10T03:13:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:57 smithi098 ceph-mon[103124]: pgmap v7664: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 271 KiB/s wr, 17 op/s 2023-09-10T03:13:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14068.858393 secs 2023-09-10T03:13:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14068.858393 secs 2023-09-10T03:13:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14068.858393 secs 2023-09-10T03:13:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14070.220298 secs 2023-09-10T03:13:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:59 smithi170 ceph-mon[102832]: pgmap v7665: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 169 KiB/s wr, 11 op/s 2023-09-10T03:13:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:13:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14070.931681 secs 2023-09-10T03:13:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14070.220298 secs 2023-09-10T03:13:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:59 smithi082 ceph-mon[98694]: pgmap v7665: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 169 KiB/s wr, 11 op/s 2023-09-10T03:13:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:13:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14070.931681 secs 2023-09-10T03:14:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14070.220298 secs 2023-09-10T03:14:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:59 smithi098 ceph-mon[103124]: pgmap v7665: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 169 KiB/s wr, 11 op/s 2023-09-10T03:14:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:13:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14070.931681 secs 2023-09-10T03:14:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:01 smithi170 ceph-mon[102832]: pgmap v7666: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 272 KiB/s wr, 18 op/s 2023-09-10T03:14:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:01 smithi082 ceph-mon[98694]: pgmap v7666: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 272 KiB/s wr, 18 op/s 2023-09-10T03:14:02.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:01 smithi098 ceph-mon[103124]: pgmap v7666: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 272 KiB/s wr, 18 op/s 2023-09-10T03:14:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14073.858500 secs 2023-09-10T03:14:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:03 smithi082 ceph-mon[98694]: pgmap v7667: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T03:14:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14073.858500 secs 2023-09-10T03:14:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:03 smithi098 ceph-mon[103124]: pgmap v7667: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T03:14:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14073.858500 secs 2023-09-10T03:14:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:03 smithi170 ceph-mon[102832]: pgmap v7667: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T03:14:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14075.220354 secs 2023-09-10T03:14:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14075.931745 secs 2023-09-10T03:14:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14075.220354 secs 2023-09-10T03:14:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14075.931745 secs 2023-09-10T03:14:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14075.220354 secs 2023-09-10T03:14:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14075.931745 secs 2023-09-10T03:14:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:05 smithi082 ceph-mon[98694]: pgmap v7668: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T03:14:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:05 smithi098 ceph-mon[103124]: pgmap v7668: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T03:14:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:05 smithi170 ceph-mon[102832]: pgmap v7668: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T03:14:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:07 smithi098 ceph-mon[103124]: pgmap v7669: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 149 KiB/s wr, 30 op/s 2023-09-10T03:14:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:07 smithi170 ceph-mon[102832]: pgmap v7669: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 149 KiB/s wr, 30 op/s 2023-09-10T03:14:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:07 smithi082 ceph-mon[98694]: pgmap v7669: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 149 KiB/s wr, 30 op/s 2023-09-10T03:14:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14078.858561 secs 2023-09-10T03:14:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:08 smithi098 ceph-mon[103124]: osdmap e547: 12 total, 12 up, 12 in 2023-09-10T03:14:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14078.858561 secs 2023-09-10T03:14:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:08 smithi170 ceph-mon[102832]: osdmap e547: 12 total, 12 up, 12 in 2023-09-10T03:14:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14078.858561 secs 2023-09-10T03:14:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:08 smithi082 ceph-mon[98694]: osdmap e547: 12 total, 12 up, 12 in 2023-09-10T03:14:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14080.220437 secs 2023-09-10T03:14:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:09 smithi098 ceph-mon[103124]: pgmap v7671: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 178 KiB/s wr, 29 op/s 2023-09-10T03:14:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14080.931815 secs 2023-09-10T03:14:09.761 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14080.220437 secs 2023-09-10T03:14:09.761 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:09 smithi082 ceph-mon[98694]: pgmap v7671: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 178 KiB/s wr, 29 op/s 2023-09-10T03:14:09.761 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14080.931815 secs 2023-09-10T03:14:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14080.220437 secs 2023-09-10T03:14:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:09 smithi170 ceph-mon[102832]: pgmap v7671: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 178 KiB/s wr, 29 op/s 2023-09-10T03:14:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14080.931815 secs 2023-09-10T03:14:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:11 smithi170 ceph-mon[102832]: pgmap v7672: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 97 KiB/s wr, 26 op/s 2023-09-10T03:14:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:11 smithi082 ceph-mon[98694]: pgmap v7672: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 97 KiB/s wr, 26 op/s 2023-09-10T03:14:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:11 smithi098 ceph-mon[103124]: pgmap v7672: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 97 KiB/s wr, 26 op/s 2023-09-10T03:14:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14083.858621 secs 2023-09-10T03:14:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:13 smithi170 ceph-mon[102832]: osdmap e548: 12 total, 12 up, 12 in 2023-09-10T03:14:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:13 smithi170 ceph-mon[102832]: pgmap v7674: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 121 KiB/s wr, 33 op/s 2023-09-10T03:14:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14083.858621 secs 2023-09-10T03:14:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:13 smithi082 ceph-mon[98694]: osdmap e548: 12 total, 12 up, 12 in 2023-09-10T03:14:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:13 smithi082 ceph-mon[98694]: pgmap v7674: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 121 KiB/s wr, 33 op/s 2023-09-10T03:14:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14083.858621 secs 2023-09-10T03:14:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:13 smithi098 ceph-mon[103124]: osdmap e548: 12 total, 12 up, 12 in 2023-09-10T03:14:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:13 smithi098 ceph-mon[103124]: pgmap v7674: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 121 KiB/s wr, 33 op/s 2023-09-10T03:14:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14085.220489 secs 2023-09-10T03:14:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14085.931877 secs 2023-09-10T03:14:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14085.220489 secs 2023-09-10T03:14:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14085.931877 secs 2023-09-10T03:14:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14085.220489 secs 2023-09-10T03:14:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14085.931877 secs 2023-09-10T03:14:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:15 smithi082 ceph-mon[98694]: pgmap v7675: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 6 op/s 2023-09-10T03:14:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:15 smithi098 ceph-mon[103124]: pgmap v7675: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 6 op/s 2023-09-10T03:14:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:15 smithi170 ceph-mon[102832]: pgmap v7675: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 6 op/s 2023-09-10T03:14:17.697 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:17 smithi082 ceph-mon[98694]: pgmap v7676: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 12 op/s 2023-09-10T03:14:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:17 smithi098 ceph-mon[103124]: pgmap v7676: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 12 op/s 2023-09-10T03:14:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:17 smithi170 ceph-mon[102832]: pgmap v7676: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 12 op/s 2023-09-10T03:14:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14088.858711 secs 2023-09-10T03:14:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14088.858711 secs 2023-09-10T03:14:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14088.858711 secs 2023-09-10T03:14:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14090.220572 secs 2023-09-10T03:14:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:19 smithi082 ceph-mon[98694]: pgmap v7677: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T03:14:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14090.931946 secs 2023-09-10T03:14:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14090.220572 secs 2023-09-10T03:14:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:19 smithi098 ceph-mon[103124]: pgmap v7677: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T03:14:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14090.931946 secs 2023-09-10T03:14:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14090.220572 secs 2023-09-10T03:14:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:19 smithi170 ceph-mon[102832]: pgmap v7677: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 11 op/s 2023-09-10T03:14:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14090.931946 secs 2023-09-10T03:14:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:21 smithi082 ceph-mon[98694]: pgmap v7678: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 12 op/s 2023-09-10T03:14:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:21 smithi098 ceph-mon[103124]: pgmap v7678: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 12 op/s 2023-09-10T03:14:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:21 smithi170 ceph-mon[102832]: pgmap v7678: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 12 op/s 2023-09-10T03:14:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:23 smithi082 ceph-mon[98694]: pgmap v7679: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 12 op/s 2023-09-10T03:14:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:14:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:23 smithi098 ceph-mon[103124]: pgmap v7679: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 12 op/s 2023-09-10T03:14:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:14:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:23 smithi170 ceph-mon[102832]: pgmap v7679: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 12 op/s 2023-09-10T03:14:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:14:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14095.220654 secs 2023-09-10T03:14:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:14:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:14:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:14:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14095.932012 secs 2023-09-10T03:14:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14095.220654 secs 2023-09-10T03:14:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:14:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:14:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:14:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14095.932012 secs 2023-09-10T03:14:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14095.220654 secs 2023-09-10T03:14:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:14:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:14:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:14:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14095.932012 secs 2023-09-10T03:14:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:25 smithi082 ceph-mon[98694]: pgmap v7680: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 10 op/s 2023-09-10T03:14:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:25 smithi098 ceph-mon[103124]: pgmap v7680: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 10 op/s 2023-09-10T03:14:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:25 smithi170 ceph-mon[102832]: pgmap v7680: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 10 op/s 2023-09-10T03:14:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:27 smithi098 ceph-mon[103124]: pgmap v7681: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 91 KiB/s wr, 15 op/s 2023-09-10T03:14:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:27 smithi170 ceph-mon[102832]: pgmap v7681: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 91 KiB/s wr, 15 op/s 2023-09-10T03:14:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:27 smithi082 ceph-mon[98694]: pgmap v7681: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 91 KiB/s wr, 15 op/s 2023-09-10T03:14:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14098.889807 secs 2023-09-10T03:14:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14098.889807 secs 2023-09-10T03:14:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14098.889807 secs 2023-09-10T03:14:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14100.220713 secs 2023-09-10T03:14:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:29 smithi098 ceph-mon[103124]: pgmap v7682: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T03:14:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14100.932077 secs 2023-09-10T03:14:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14100.220713 secs 2023-09-10T03:14:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:29 smithi170 ceph-mon[102832]: pgmap v7682: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T03:14:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14100.932077 secs 2023-09-10T03:14:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14100.220713 secs 2023-09-10T03:14:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:29 smithi082 ceph-mon[98694]: pgmap v7682: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T03:14:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14100.932077 secs 2023-09-10T03:14:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:14:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:14:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:14:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:14:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:14:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:14:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:31 smithi170 ceph-mon[102832]: pgmap v7683: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 322 KiB/s wr, 17 op/s 2023-09-10T03:14:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:31 smithi082 ceph-mon[98694]: pgmap v7683: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 322 KiB/s wr, 17 op/s 2023-09-10T03:14:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:31 smithi098 ceph-mon[103124]: pgmap v7683: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 322 KiB/s wr, 17 op/s 2023-09-10T03:14:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:32 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:14:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:14:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:14:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:32 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:14:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:14:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:14:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:32 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:14:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:14:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:14:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14103.889868 secs 2023-09-10T03:14:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:33 smithi170 ceph-mon[102832]: pgmap v7684: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T03:14:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14103.889868 secs 2023-09-10T03:14:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:33 smithi082 ceph-mon[98694]: pgmap v7684: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T03:14:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14103.889868 secs 2023-09-10T03:14:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:33 smithi098 ceph-mon[103124]: pgmap v7684: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T03:14:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14105.220769 secs 2023-09-10T03:14:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14105.932142 secs 2023-09-10T03:14:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14105.220769 secs 2023-09-10T03:14:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14105.932142 secs 2023-09-10T03:14:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14105.220769 secs 2023-09-10T03:14:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14105.932142 secs 2023-09-10T03:14:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:35 smithi170 ceph-mon[102832]: pgmap v7685: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T03:14:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:35 smithi082 ceph-mon[98694]: pgmap v7685: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T03:14:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:35 smithi098 ceph-mon[103124]: pgmap v7685: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T03:14:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:37 smithi170 ceph-mon[102832]: pgmap v7686: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 294 KiB/s wr, 17 op/s 2023-09-10T03:14:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:14:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:14:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:37 smithi082 ceph-mon[98694]: pgmap v7686: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 294 KiB/s wr, 17 op/s 2023-09-10T03:14:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:14:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:14:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:37 smithi098 ceph-mon[103124]: pgmap v7686: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 294 KiB/s wr, 17 op/s 2023-09-10T03:14:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:14:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:14:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14110.220837 secs 2023-09-10T03:14:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:39 smithi170 ceph-mon[102832]: pgmap v7687: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 249 KiB/s wr, 11 op/s 2023-09-10T03:14:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14110.932205 secs 2023-09-10T03:14:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:14:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:14:39.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:14:39.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:14:39.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:14:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14110.220837 secs 2023-09-10T03:14:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:39 smithi082 ceph-mon[98694]: pgmap v7687: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 249 KiB/s wr, 11 op/s 2023-09-10T03:14:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14110.932205 secs 2023-09-10T03:14:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:14:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:14:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:14:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:14:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:14:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14110.220837 secs 2023-09-10T03:14:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:39 smithi098 ceph-mon[103124]: pgmap v7687: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 249 KiB/s wr, 11 op/s 2023-09-10T03:14:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14110.932205 secs 2023-09-10T03:14:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:14:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:14:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:14:40.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:14:40.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:14:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:14:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:14:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:14:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:14:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:14:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:14:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:14:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:14:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:14:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:41 smithi170 ceph-mon[102832]: pgmap v7688: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 409 KiB/s wr, 18 op/s 2023-09-10T03:14:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:41 smithi082 ceph-mon[98694]: pgmap v7688: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 409 KiB/s wr, 18 op/s 2023-09-10T03:14:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:41 smithi098 ceph-mon[103124]: pgmap v7688: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 409 KiB/s wr, 18 op/s 2023-09-10T03:14:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:43 smithi082 ceph-mon[98694]: pgmap v7689: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T03:14:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:43 smithi098 ceph-mon[103124]: pgmap v7689: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T03:14:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:43 smithi170 ceph-mon[102832]: pgmap v7689: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T03:14:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14115.220917 secs 2023-09-10T03:14:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14115.932287 secs 2023-09-10T03:14:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14115.220917 secs 2023-09-10T03:14:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14115.932287 secs 2023-09-10T03:14:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14115.220917 secs 2023-09-10T03:14:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14115.932287 secs 2023-09-10T03:14:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:45 smithi082 ceph-mon[98694]: pgmap v7690: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T03:14:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:45 smithi098 ceph-mon[103124]: pgmap v7690: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T03:14:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:45 smithi170 ceph-mon[102832]: pgmap v7690: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T03:14:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:47 smithi170 ceph-mon[102832]: pgmap v7691: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 341 KiB/s wr, 17 op/s 2023-09-10T03:14:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:47 smithi082 ceph-mon[98694]: pgmap v7691: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 341 KiB/s wr, 17 op/s 2023-09-10T03:14:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:47 smithi098 ceph-mon[103124]: pgmap v7691: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 341 KiB/s wr, 17 op/s 2023-09-10T03:14:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14118.890053 secs 2023-09-10T03:14:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14118.890053 secs 2023-09-10T03:14:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14118.890053 secs 2023-09-10T03:14:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14120.221005 secs 2023-09-10T03:14:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:49 smithi170 ceph-mon[102832]: pgmap v7692: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 324 KiB/s wr, 12 op/s 2023-09-10T03:14:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14120.932349 secs 2023-09-10T03:14:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14120.221005 secs 2023-09-10T03:14:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:49 smithi082 ceph-mon[98694]: pgmap v7692: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 324 KiB/s wr, 12 op/s 2023-09-10T03:14:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14120.932349 secs 2023-09-10T03:14:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14120.221005 secs 2023-09-10T03:14:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:49 smithi098 ceph-mon[103124]: pgmap v7692: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 324 KiB/s wr, 12 op/s 2023-09-10T03:14:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14120.932349 secs 2023-09-10T03:14:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:51 smithi170 ceph-mon[102832]: pgmap v7693: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 349 KiB/s wr, 17 op/s 2023-09-10T03:14:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:51 smithi082 ceph-mon[98694]: pgmap v7693: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 349 KiB/s wr, 17 op/s 2023-09-10T03:14:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:51 smithi098 ceph-mon[103124]: pgmap v7693: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 349 KiB/s wr, 17 op/s 2023-09-10T03:14:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:14:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:14:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:14:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:14:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:14:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:14:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:14:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:14:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:14:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14123.890113 secs 2023-09-10T03:14:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:53 smithi170 ceph-mon[102832]: pgmap v7694: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 188 KiB/s wr, 11 op/s 2023-09-10T03:14:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14123.890113 secs 2023-09-10T03:14:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:53 smithi082 ceph-mon[98694]: pgmap v7694: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 188 KiB/s wr, 11 op/s 2023-09-10T03:14:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14123.890113 secs 2023-09-10T03:14:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:53 smithi098 ceph-mon[103124]: pgmap v7694: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 188 KiB/s wr, 11 op/s 2023-09-10T03:14:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14125.221060 secs 2023-09-10T03:14:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14125.932414 secs 2023-09-10T03:14:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14125.221060 secs 2023-09-10T03:14:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14125.932414 secs 2023-09-10T03:14:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14125.221060 secs 2023-09-10T03:14:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14125.932414 secs 2023-09-10T03:14:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:55 smithi170 ceph-mon[102832]: pgmap v7695: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 188 KiB/s wr, 11 op/s 2023-09-10T03:14:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:55 smithi082 ceph-mon[98694]: pgmap v7695: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 188 KiB/s wr, 11 op/s 2023-09-10T03:14:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:55 smithi098 ceph-mon[103124]: pgmap v7695: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 188 KiB/s wr, 11 op/s 2023-09-10T03:14:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:57 smithi170 ceph-mon[102832]: pgmap v7696: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 281 KiB/s wr, 16 op/s 2023-09-10T03:14:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:57 smithi082 ceph-mon[98694]: pgmap v7696: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 281 KiB/s wr, 16 op/s 2023-09-10T03:14:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:57 smithi098 ceph-mon[103124]: pgmap v7696: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 281 KiB/s wr, 16 op/s 2023-09-10T03:14:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14128.890197 secs 2023-09-10T03:14:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14128.890197 secs 2023-09-10T03:14:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14128.890197 secs 2023-09-10T03:14:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14130.221118 secs 2023-09-10T03:14:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:59 smithi170 ceph-mon[102832]: pgmap v7697: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 10 op/s 2023-09-10T03:14:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:14:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14130.932481 secs 2023-09-10T03:14:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14130.221118 secs 2023-09-10T03:14:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:59 smithi082 ceph-mon[98694]: pgmap v7697: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 10 op/s 2023-09-10T03:14:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:14:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14130.932481 secs 2023-09-10T03:15:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14130.221118 secs 2023-09-10T03:15:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:59 smithi098 ceph-mon[103124]: pgmap v7697: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 10 op/s 2023-09-10T03:15:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:14:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14130.932481 secs 2023-09-10T03:15:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:01 smithi170 ceph-mon[102832]: pgmap v7698: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 157 KiB/s wr, 15 op/s 2023-09-10T03:15:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:01 smithi082 ceph-mon[98694]: pgmap v7698: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 157 KiB/s wr, 15 op/s 2023-09-10T03:15:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:01 smithi098 ceph-mon[103124]: pgmap v7698: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 157 KiB/s wr, 15 op/s 2023-09-10T03:15:03.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14133.890259 secs 2023-09-10T03:15:03.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:03 smithi082 ceph-mon[98694]: pgmap v7699: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 10 op/s 2023-09-10T03:15:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14133.890259 secs 2023-09-10T03:15:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:03 smithi098 ceph-mon[103124]: pgmap v7699: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 10 op/s 2023-09-10T03:15:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14133.890259 secs 2023-09-10T03:15:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:03 smithi170 ceph-mon[102832]: pgmap v7699: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 10 op/s 2023-09-10T03:15:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14135.221196 secs 2023-09-10T03:15:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:15:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:15:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14135.932549 secs 2023-09-10T03:15:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14135.221196 secs 2023-09-10T03:15:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:15:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:15:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14135.932549 secs 2023-09-10T03:15:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14135.221196 secs 2023-09-10T03:15:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:15:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:15:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14135.932549 secs 2023-09-10T03:15:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:05 smithi082 ceph-mon[98694]: pgmap v7700: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 10 op/s 2023-09-10T03:15:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:05 smithi098 ceph-mon[103124]: pgmap v7700: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 10 op/s 2023-09-10T03:15:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:05 smithi170 ceph-mon[102832]: pgmap v7700: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 10 op/s 2023-09-10T03:15:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:15:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:15:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:15:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:15:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:15:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:15:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:07 smithi098 ceph-mon[103124]: pgmap v7701: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 189 KiB/s wr, 30 op/s 2023-09-10T03:15:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:07 smithi170 ceph-mon[102832]: pgmap v7701: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 189 KiB/s wr, 30 op/s 2023-09-10T03:15:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:07 smithi082 ceph-mon[98694]: pgmap v7701: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 189 KiB/s wr, 30 op/s 2023-09-10T03:15:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14138.890318 secs 2023-09-10T03:15:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:08 smithi098 ceph-mon[103124]: osdmap e549: 12 total, 12 up, 12 in 2023-09-10T03:15:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14138.890318 secs 2023-09-10T03:15:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:08 smithi170 ceph-mon[102832]: osdmap e549: 12 total, 12 up, 12 in 2023-09-10T03:15:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14138.890318 secs 2023-09-10T03:15:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:08 smithi082 ceph-mon[98694]: osdmap e549: 12 total, 12 up, 12 in 2023-09-10T03:15:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14140.221255 secs 2023-09-10T03:15:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:09 smithi098 ceph-mon[103124]: pgmap v7703: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 116 KiB/s wr, 30 op/s 2023-09-10T03:15:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14140.932614 secs 2023-09-10T03:15:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14140.221255 secs 2023-09-10T03:15:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:09 smithi170 ceph-mon[102832]: pgmap v7703: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 116 KiB/s wr, 30 op/s 2023-09-10T03:15:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14140.932614 secs 2023-09-10T03:15:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14140.221255 secs 2023-09-10T03:15:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:09 smithi082 ceph-mon[98694]: pgmap v7703: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 116 KiB/s wr, 30 op/s 2023-09-10T03:15:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14140.932614 secs 2023-09-10T03:15:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:11 smithi098 ceph-mon[103124]: pgmap v7704: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 68 KiB/s wr, 29 op/s 2023-09-10T03:15:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:11 smithi170 ceph-mon[102832]: pgmap v7704: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 68 KiB/s wr, 29 op/s 2023-09-10T03:15:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:11 smithi082 ceph-mon[98694]: pgmap v7704: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 68 KiB/s wr, 29 op/s 2023-09-10T03:15:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14143.890396 secs 2023-09-10T03:15:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:13 smithi082 ceph-mon[98694]: osdmap e550: 12 total, 12 up, 12 in 2023-09-10T03:15:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:13 smithi082 ceph-mon[98694]: pgmap v7706: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 85 KiB/s wr, 37 op/s 2023-09-10T03:15:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14143.890396 secs 2023-09-10T03:15:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:13 smithi098 ceph-mon[103124]: osdmap e550: 12 total, 12 up, 12 in 2023-09-10T03:15:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:13 smithi098 ceph-mon[103124]: pgmap v7706: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 85 KiB/s wr, 37 op/s 2023-09-10T03:15:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14143.890396 secs 2023-09-10T03:15:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:13 smithi170 ceph-mon[102832]: osdmap e550: 12 total, 12 up, 12 in 2023-09-10T03:15:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:13 smithi170 ceph-mon[102832]: pgmap v7706: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 85 KiB/s wr, 37 op/s 2023-09-10T03:15:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14145.221313 secs 2023-09-10T03:15:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14145.932680 secs 2023-09-10T03:15:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14145.221313 secs 2023-09-10T03:15:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14145.932680 secs 2023-09-10T03:15:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14145.221313 secs 2023-09-10T03:15:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14145.932680 secs 2023-09-10T03:15:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:15 smithi098 ceph-mon[103124]: pgmap v7707: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T03:15:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:15 smithi170 ceph-mon[102832]: pgmap v7707: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T03:15:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:15 smithi082 ceph-mon[98694]: pgmap v7707: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T03:15:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:17 smithi098 ceph-mon[103124]: pgmap v7708: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 13 op/s 2023-09-10T03:15:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:17 smithi170 ceph-mon[102832]: pgmap v7708: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 13 op/s 2023-09-10T03:15:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:17 smithi082 ceph-mon[98694]: pgmap v7708: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 13 op/s 2023-09-10T03:15:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14148.890427 secs 2023-09-10T03:15:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14148.890427 secs 2023-09-10T03:15:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14148.890427 secs 2023-09-10T03:15:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14150.221402 secs 2023-09-10T03:15:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:19 smithi098 ceph-mon[103124]: pgmap v7709: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 54 KiB/s wr, 11 op/s 2023-09-10T03:15:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14150.932747 secs 2023-09-10T03:15:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14150.221402 secs 2023-09-10T03:15:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:19 smithi170 ceph-mon[102832]: pgmap v7709: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 54 KiB/s wr, 11 op/s 2023-09-10T03:15:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14150.932747 secs 2023-09-10T03:15:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14150.221402 secs 2023-09-10T03:15:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:19 smithi082 ceph-mon[98694]: pgmap v7709: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 54 KiB/s wr, 11 op/s 2023-09-10T03:15:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14150.932747 secs 2023-09-10T03:15:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:21 smithi098 ceph-mon[103124]: pgmap v7710: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 54 KiB/s wr, 11 op/s 2023-09-10T03:15:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:21 smithi170 ceph-mon[102832]: pgmap v7710: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 54 KiB/s wr, 11 op/s 2023-09-10T03:15:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:21 smithi082 ceph-mon[98694]: pgmap v7710: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 54 KiB/s wr, 11 op/s 2023-09-10T03:15:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:23 smithi082 ceph-mon[98694]: pgmap v7711: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 54 KiB/s wr, 11 op/s 2023-09-10T03:15:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:15:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:23 smithi098 ceph-mon[103124]: pgmap v7711: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 54 KiB/s wr, 11 op/s 2023-09-10T03:15:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:15:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:23 smithi170 ceph-mon[102832]: pgmap v7711: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 54 KiB/s wr, 11 op/s 2023-09-10T03:15:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:15:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14155.221457 secs 2023-09-10T03:15:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:15:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:15:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:15:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14155.932811 secs 2023-09-10T03:15:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14155.221457 secs 2023-09-10T03:15:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:15:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:15:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:15:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14155.932811 secs 2023-09-10T03:15:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14155.221457 secs 2023-09-10T03:15:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:15:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:15:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:15:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14155.932811 secs 2023-09-10T03:15:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:25 smithi082 ceph-mon[98694]: pgmap v7712: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 9 op/s 2023-09-10T03:15:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:25 smithi098 ceph-mon[103124]: pgmap v7712: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 9 op/s 2023-09-10T03:15:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:25 smithi170 ceph-mon[102832]: pgmap v7712: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 45 KiB/s wr, 9 op/s 2023-09-10T03:15:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:27 smithi170 ceph-mon[102832]: pgmap v7713: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 112 KiB/s wr, 14 op/s 2023-09-10T03:15:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:27 smithi082 ceph-mon[98694]: pgmap v7713: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 112 KiB/s wr, 14 op/s 2023-09-10T03:15:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:27 smithi098 ceph-mon[103124]: pgmap v7713: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 112 KiB/s wr, 14 op/s 2023-09-10T03:15:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14160.221517 secs 2023-09-10T03:15:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:29 smithi170 ceph-mon[102832]: pgmap v7714: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 68 KiB/s wr, 9 op/s 2023-09-10T03:15:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14160.932876 secs 2023-09-10T03:15:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14160.221517 secs 2023-09-10T03:15:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:29 smithi082 ceph-mon[98694]: pgmap v7714: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 68 KiB/s wr, 9 op/s 2023-09-10T03:15:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14160.932876 secs 2023-09-10T03:15:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14160.221517 secs 2023-09-10T03:15:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:29 smithi098 ceph-mon[103124]: pgmap v7714: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 68 KiB/s wr, 9 op/s 2023-09-10T03:15:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14160.932876 secs 2023-09-10T03:15:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:31 smithi170 ceph-mon[102832]: pgmap v7715: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 165 KiB/s wr, 15 op/s 2023-09-10T03:15:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:31 smithi082 ceph-mon[98694]: pgmap v7715: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 165 KiB/s wr, 15 op/s 2023-09-10T03:15:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:31 smithi098 ceph-mon[103124]: pgmap v7715: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 165 KiB/s wr, 15 op/s 2023-09-10T03:15:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:33 smithi082 ceph-mon[98694]: pgmap v7716: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 10 op/s 2023-09-10T03:15:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:33 smithi098 ceph-mon[103124]: pgmap v7716: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 10 op/s 2023-09-10T03:15:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:33 smithi170 ceph-mon[102832]: pgmap v7716: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 10 op/s 2023-09-10T03:15:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14165.221574 secs 2023-09-10T03:15:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14165.932945 secs 2023-09-10T03:15:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14165.221574 secs 2023-09-10T03:15:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14165.932945 secs 2023-09-10T03:15:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14165.221574 secs 2023-09-10T03:15:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14165.932945 secs 2023-09-10T03:15:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:35 smithi098 ceph-mon[103124]: pgmap v7717: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 10 op/s 2023-09-10T03:15:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:35 smithi170 ceph-mon[102832]: pgmap v7717: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 10 op/s 2023-09-10T03:15:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:35 smithi082 ceph-mon[98694]: pgmap v7717: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 10 op/s 2023-09-10T03:15:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:37 smithi170 ceph-mon[102832]: pgmap v7718: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 197 KiB/s wr, 16 op/s 2023-09-10T03:15:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:37 smithi082 ceph-mon[98694]: pgmap v7718: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 197 KiB/s wr, 16 op/s 2023-09-10T03:15:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:37 smithi098 ceph-mon[103124]: pgmap v7718: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 197 KiB/s wr, 16 op/s 2023-09-10T03:15:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14168.890765 secs 2023-09-10T03:15:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:15:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:15:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14168.890765 secs 2023-09-10T03:15:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:15:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:15:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14168.890765 secs 2023-09-10T03:15:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:15:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:15:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14170.221659 secs 2023-09-10T03:15:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:39 smithi170 ceph-mon[102832]: pgmap v7719: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 130 KiB/s wr, 11 op/s 2023-09-10T03:15:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14170.933011 secs 2023-09-10T03:15:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14170.221659 secs 2023-09-10T03:15:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:39 smithi082 ceph-mon[98694]: pgmap v7719: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 130 KiB/s wr, 11 op/s 2023-09-10T03:15:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14170.933011 secs 2023-09-10T03:15:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14170.221659 secs 2023-09-10T03:15:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:39 smithi098 ceph-mon[103124]: pgmap v7719: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 130 KiB/s wr, 11 op/s 2023-09-10T03:15:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14170.933011 secs 2023-09-10T03:15:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:15:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:15:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:15:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:15:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:15:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:15:40.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:15:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:15:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:15:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:15:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:15:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:15:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:15:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:15:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:15:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:15:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:15:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:15:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:15:41.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:15:41.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:15:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:41 smithi170 ceph-mon[102832]: pgmap v7720: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 304 KiB/s wr, 18 op/s 2023-09-10T03:15:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:41 smithi082 ceph-mon[98694]: pgmap v7720: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 304 KiB/s wr, 18 op/s 2023-09-10T03:15:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:41 smithi098 ceph-mon[103124]: pgmap v7720: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 304 KiB/s wr, 18 op/s 2023-09-10T03:15:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14173.890825 secs 2023-09-10T03:15:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:43 smithi098 ceph-mon[103124]: pgmap v7721: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 207 KiB/s wr, 12 op/s 2023-09-10T03:15:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14173.890825 secs 2023-09-10T03:15:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:43 smithi170 ceph-mon[102832]: pgmap v7721: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 207 KiB/s wr, 12 op/s 2023-09-10T03:15:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14173.890825 secs 2023-09-10T03:15:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:43 smithi082 ceph-mon[98694]: pgmap v7721: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 207 KiB/s wr, 12 op/s 2023-09-10T03:15:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14175.221733 secs 2023-09-10T03:15:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14175.933075 secs 2023-09-10T03:15:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14175.221733 secs 2023-09-10T03:15:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14175.933075 secs 2023-09-10T03:15:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14175.221733 secs 2023-09-10T03:15:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14175.933075 secs 2023-09-10T03:15:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:45 smithi098 ceph-mon[103124]: pgmap v7722: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 207 KiB/s wr, 12 op/s 2023-09-10T03:15:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:45 smithi170 ceph-mon[102832]: pgmap v7722: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 207 KiB/s wr, 12 op/s 2023-09-10T03:15:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:45 smithi082 ceph-mon[98694]: pgmap v7722: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 207 KiB/s wr, 12 op/s 2023-09-10T03:15:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:47 smithi170 ceph-mon[102832]: pgmap v7723: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 274 KiB/s wr, 18 op/s 2023-09-10T03:15:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:47 smithi082 ceph-mon[98694]: pgmap v7723: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 274 KiB/s wr, 18 op/s 2023-09-10T03:15:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:47 smithi098 ceph-mon[103124]: pgmap v7723: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 274 KiB/s wr, 18 op/s 2023-09-10T03:15:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14180.221819 secs 2023-09-10T03:15:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:49 smithi170 ceph-mon[102832]: pgmap v7724: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 241 KiB/s wr, 12 op/s 2023-09-10T03:15:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14180.933139 secs 2023-09-10T03:15:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14180.221819 secs 2023-09-10T03:15:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:49 smithi082 ceph-mon[98694]: pgmap v7724: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 241 KiB/s wr, 12 op/s 2023-09-10T03:15:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14180.933139 secs 2023-09-10T03:15:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14180.221819 secs 2023-09-10T03:15:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:49 smithi098 ceph-mon[103124]: pgmap v7724: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 241 KiB/s wr, 12 op/s 2023-09-10T03:15:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14180.933139 secs 2023-09-10T03:15:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:51 smithi170 ceph-mon[102832]: pgmap v7725: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 368 KiB/s wr, 19 op/s 2023-09-10T03:15:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:51 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:15:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:51 smithi082 ceph-mon[98694]: pgmap v7725: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 368 KiB/s wr, 19 op/s 2023-09-10T03:15:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:51 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:15:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:51 smithi098 ceph-mon[103124]: pgmap v7725: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 368 KiB/s wr, 19 op/s 2023-09-10T03:15:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:51 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:15:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:52 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:15:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:15:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:15:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:52 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:15:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:15:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:15:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:52 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:15:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:15:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:15:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14183.890948 secs 2023-09-10T03:15:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:53 smithi170 ceph-mon[102832]: pgmap v7726: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 12 op/s 2023-09-10T03:15:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14183.890948 secs 2023-09-10T03:15:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:53 smithi082 ceph-mon[98694]: pgmap v7726: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 12 op/s 2023-09-10T03:15:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14183.890948 secs 2023-09-10T03:15:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:53 smithi098 ceph-mon[103124]: pgmap v7726: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 12 op/s 2023-09-10T03:15:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14185.221901 secs 2023-09-10T03:15:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14185.933206 secs 2023-09-10T03:15:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14185.221901 secs 2023-09-10T03:15:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14185.933206 secs 2023-09-10T03:15:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14185.221901 secs 2023-09-10T03:15:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14185.933206 secs 2023-09-10T03:15:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:55 smithi170 ceph-mon[102832]: pgmap v7727: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 12 op/s 2023-09-10T03:15:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:55 smithi082 ceph-mon[98694]: pgmap v7727: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 12 op/s 2023-09-10T03:15:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:55 smithi098 ceph-mon[103124]: pgmap v7727: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 12 op/s 2023-09-10T03:15:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:15:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:15:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:15:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:15:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:15:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:15:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:57 smithi170 ceph-mon[102832]: pgmap v7728: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 17 op/s 2023-09-10T03:15:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:15:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:15:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:57 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:15:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:57 smithi082 ceph-mon[98694]: pgmap v7728: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 17 op/s 2023-09-10T03:15:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:15:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:15:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:57 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:15:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:57 smithi098 ceph-mon[103124]: pgmap v7728: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 17 op/s 2023-09-10T03:15:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:15:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:15:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:57 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:15:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14188.891067 secs 2023-09-10T03:15:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14188.891067 secs 2023-09-10T03:15:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14188.891067 secs 2023-09-10T03:15:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14190.221960 secs 2023-09-10T03:15:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:59 smithi170 ceph-mon[102832]: pgmap v7729: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 210 KiB/s wr, 12 op/s 2023-09-10T03:15:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:15:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14190.933280 secs 2023-09-10T03:15:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14190.221960 secs 2023-09-10T03:15:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:59 smithi082 ceph-mon[98694]: pgmap v7729: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 210 KiB/s wr, 12 op/s 2023-09-10T03:15:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:15:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14190.933280 secs 2023-09-10T03:16:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14190.221960 secs 2023-09-10T03:16:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:59 smithi098 ceph-mon[103124]: pgmap v7729: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 210 KiB/s wr, 12 op/s 2023-09-10T03:16:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:15:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14190.933280 secs 2023-09-10T03:16:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:16:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:16:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:16:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:16:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:16:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:16:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:01 smithi170 ceph-mon[102832]: pgmap v7730: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 279 KiB/s wr, 17 op/s 2023-09-10T03:16:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:01 smithi082 ceph-mon[98694]: pgmap v7730: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 279 KiB/s wr, 17 op/s 2023-09-10T03:16:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:01 smithi098 ceph-mon[103124]: pgmap v7730: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 279 KiB/s wr, 17 op/s 2023-09-10T03:16:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14193.891175 secs 2023-09-10T03:16:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:03 smithi098 ceph-mon[103124]: pgmap v7731: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T03:16:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14193.891175 secs 2023-09-10T03:16:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:03 smithi170 ceph-mon[102832]: pgmap v7731: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T03:16:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14193.891175 secs 2023-09-10T03:16:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:03 smithi082 ceph-mon[98694]: pgmap v7731: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T03:16:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14195.222017 secs 2023-09-10T03:16:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14195.933345 secs 2023-09-10T03:16:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14195.222017 secs 2023-09-10T03:16:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14195.933345 secs 2023-09-10T03:16:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14195.222017 secs 2023-09-10T03:16:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14195.933345 secs 2023-09-10T03:16:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:05 smithi098 ceph-mon[103124]: pgmap v7732: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T03:16:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:05 smithi170 ceph-mon[102832]: pgmap v7732: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T03:16:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:05 smithi082 ceph-mon[98694]: pgmap v7732: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T03:16:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:07 smithi170 ceph-mon[102832]: pgmap v7733: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 227 KiB/s wr, 31 op/s 2023-09-10T03:16:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:07 smithi082 ceph-mon[98694]: pgmap v7733: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 227 KiB/s wr, 31 op/s 2023-09-10T03:16:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:07 smithi098 ceph-mon[103124]: pgmap v7733: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 227 KiB/s wr, 31 op/s 2023-09-10T03:16:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14198.891237 secs 2023-09-10T03:16:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:08 smithi170 ceph-mon[102832]: osdmap e551: 12 total, 12 up, 12 in 2023-09-10T03:16:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14198.891237 secs 2023-09-10T03:16:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:08 smithi082 ceph-mon[98694]: osdmap e551: 12 total, 12 up, 12 in 2023-09-10T03:16:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14198.891237 secs 2023-09-10T03:16:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:08 smithi098 ceph-mon[103124]: osdmap e551: 12 total, 12 up, 12 in 2023-09-10T03:16:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14200.222075 secs 2023-09-10T03:16:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:09 smithi170 ceph-mon[102832]: pgmap v7735: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 173 KiB/s wr, 30 op/s 2023-09-10T03:16:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14200.933411 secs 2023-09-10T03:16:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14200.222075 secs 2023-09-10T03:16:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:09 smithi082 ceph-mon[98694]: pgmap v7735: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 173 KiB/s wr, 30 op/s 2023-09-10T03:16:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14200.933411 secs 2023-09-10T03:16:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14200.222075 secs 2023-09-10T03:16:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:09 smithi098 ceph-mon[103124]: pgmap v7735: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 173 KiB/s wr, 30 op/s 2023-09-10T03:16:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14200.933411 secs 2023-09-10T03:16:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:11 smithi170 ceph-mon[102832]: pgmap v7736: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 104 KiB/s wr, 31 op/s 2023-09-10T03:16:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:11 smithi082 ceph-mon[98694]: pgmap v7736: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 104 KiB/s wr, 31 op/s 2023-09-10T03:16:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:11 smithi098 ceph-mon[103124]: pgmap v7736: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 104 KiB/s wr, 31 op/s 2023-09-10T03:16:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14203.891298 secs 2023-09-10T03:16:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:13 smithi098 ceph-mon[103124]: osdmap e552: 12 total, 12 up, 12 in 2023-09-10T03:16:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:13 smithi098 ceph-mon[103124]: pgmap v7738: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 130 KiB/s wr, 39 op/s 2023-09-10T03:16:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14203.891298 secs 2023-09-10T03:16:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:13 smithi170 ceph-mon[102832]: osdmap e552: 12 total, 12 up, 12 in 2023-09-10T03:16:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:13 smithi170 ceph-mon[102832]: pgmap v7738: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 130 KiB/s wr, 39 op/s 2023-09-10T03:16:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14203.891298 secs 2023-09-10T03:16:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:13 smithi082 ceph-mon[98694]: osdmap e552: 12 total, 12 up, 12 in 2023-09-10T03:16:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:13 smithi082 ceph-mon[98694]: pgmap v7738: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 130 KiB/s wr, 39 op/s 2023-09-10T03:16:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14205.222133 secs 2023-09-10T03:16:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14205.933481 secs 2023-09-10T03:16:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14205.222133 secs 2023-09-10T03:16:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14205.933481 secs 2023-09-10T03:16:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14205.222133 secs 2023-09-10T03:16:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14205.933481 secs 2023-09-10T03:16:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:15 smithi098 ceph-mon[103124]: pgmap v7739: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 8 op/s 2023-09-10T03:16:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:15 smithi170 ceph-mon[102832]: pgmap v7739: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 8 op/s 2023-09-10T03:16:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:15 smithi082 ceph-mon[98694]: pgmap v7739: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 8 op/s 2023-09-10T03:16:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:17 smithi170 ceph-mon[102832]: pgmap v7740: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 15 op/s 2023-09-10T03:16:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:17 smithi082 ceph-mon[98694]: pgmap v7740: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 15 op/s 2023-09-10T03:16:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:17 smithi098 ceph-mon[103124]: pgmap v7740: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 15 op/s 2023-09-10T03:16:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14208.891356 secs 2023-09-10T03:16:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14208.891356 secs 2023-09-10T03:16:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14208.891356 secs 2023-09-10T03:16:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14210.222217 secs 2023-09-10T03:16:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:19 smithi170 ceph-mon[102832]: pgmap v7741: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 14 op/s 2023-09-10T03:16:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14210.933548 secs 2023-09-10T03:16:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14210.222217 secs 2023-09-10T03:16:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:19 smithi082 ceph-mon[98694]: pgmap v7741: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 14 op/s 2023-09-10T03:16:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14210.933548 secs 2023-09-10T03:16:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14210.222217 secs 2023-09-10T03:16:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:19 smithi098 ceph-mon[103124]: pgmap v7741: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 14 op/s 2023-09-10T03:16:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14210.933548 secs 2023-09-10T03:16:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:21 smithi170 ceph-mon[102832]: pgmap v7742: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 12 op/s 2023-09-10T03:16:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:21 smithi082 ceph-mon[98694]: pgmap v7742: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 12 op/s 2023-09-10T03:16:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:21 smithi098 ceph-mon[103124]: pgmap v7742: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 12 op/s 2023-09-10T03:16:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:23 smithi098 ceph-mon[103124]: pgmap v7743: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 12 op/s 2023-09-10T03:16:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:16:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:23 smithi170 ceph-mon[102832]: pgmap v7743: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 12 op/s 2023-09-10T03:16:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:16:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:23 smithi082 ceph-mon[98694]: pgmap v7743: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 12 op/s 2023-09-10T03:16:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:16:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14215.222272 secs 2023-09-10T03:16:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:16:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:16:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:16:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14215.933613 secs 2023-09-10T03:16:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14215.222272 secs 2023-09-10T03:16:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:16:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:16:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:16:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14215.933613 secs 2023-09-10T03:16:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14215.222272 secs 2023-09-10T03:16:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:16:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:16:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:16:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14215.933613 secs 2023-09-10T03:16:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:25 smithi098 ceph-mon[103124]: pgmap v7744: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T03:16:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:25 smithi170 ceph-mon[102832]: pgmap v7744: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T03:16:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:25 smithi082 ceph-mon[98694]: pgmap v7744: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T03:16:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:27 smithi170 ceph-mon[102832]: pgmap v7745: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 128 KiB/s wr, 15 op/s 2023-09-10T03:16:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:27 smithi082 ceph-mon[98694]: pgmap v7745: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 128 KiB/s wr, 15 op/s 2023-09-10T03:16:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:27 smithi098 ceph-mon[103124]: pgmap v7745: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 128 KiB/s wr, 15 op/s 2023-09-10T03:16:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14220.222330 secs 2023-09-10T03:16:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:29 smithi170 ceph-mon[102832]: pgmap v7746: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 10 op/s 2023-09-10T03:16:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14220.933678 secs 2023-09-10T03:16:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14220.222330 secs 2023-09-10T03:16:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:29 smithi082 ceph-mon[98694]: pgmap v7746: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 10 op/s 2023-09-10T03:16:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14220.933678 secs 2023-09-10T03:16:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14220.222330 secs 2023-09-10T03:16:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:29 smithi098 ceph-mon[103124]: pgmap v7746: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 10 op/s 2023-09-10T03:16:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14220.933678 secs 2023-09-10T03:16:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:31 smithi170 ceph-mon[102832]: pgmap v7747: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 256 KiB/s wr, 17 op/s 2023-09-10T03:16:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:31 smithi082 ceph-mon[98694]: pgmap v7747: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 256 KiB/s wr, 17 op/s 2023-09-10T03:16:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:31 smithi098 ceph-mon[103124]: pgmap v7747: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 256 KiB/s wr, 17 op/s 2023-09-10T03:16:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14223.891559 secs 2023-09-10T03:16:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:33 smithi098 ceph-mon[103124]: pgmap v7748: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 212 KiB/s wr, 12 op/s 2023-09-10T03:16:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14223.891559 secs 2023-09-10T03:16:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:33 smithi170 ceph-mon[102832]: pgmap v7748: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 212 KiB/s wr, 12 op/s 2023-09-10T03:16:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14223.891559 secs 2023-09-10T03:16:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:33 smithi082 ceph-mon[98694]: pgmap v7748: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 212 KiB/s wr, 12 op/s 2023-09-10T03:16:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14225.222401 secs 2023-09-10T03:16:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:16:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:16:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14225.933743 secs 2023-09-10T03:16:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14225.222401 secs 2023-09-10T03:16:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:16:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:16:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14225.933743 secs 2023-09-10T03:16:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14225.222401 secs 2023-09-10T03:16:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:16:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:16:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14225.933743 secs 2023-09-10T03:16:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:35 smithi098 ceph-mon[103124]: pgmap v7749: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 212 KiB/s wr, 12 op/s 2023-09-10T03:16:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:35 smithi170 ceph-mon[102832]: pgmap v7749: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 212 KiB/s wr, 12 op/s 2023-09-10T03:16:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:35 smithi082 ceph-mon[98694]: pgmap v7749: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 212 KiB/s wr, 12 op/s 2023-09-10T03:16:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:36 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:16:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:16:36.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:16:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:36 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:16:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:16:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:16:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:36 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:16:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:16:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:16:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:37 smithi098 ceph-mon[103124]: pgmap v7750: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 229 KiB/s wr, 17 op/s 2023-09-10T03:16:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:37 smithi170 ceph-mon[102832]: pgmap v7750: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 229 KiB/s wr, 17 op/s 2023-09-10T03:16:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:37 smithi082 ceph-mon[98694]: pgmap v7750: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 229 KiB/s wr, 17 op/s 2023-09-10T03:16:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14228.891642 secs 2023-09-10T03:16:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14228.891642 secs 2023-09-10T03:16:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14228.891642 secs 2023-09-10T03:16:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14230.222459 secs 2023-09-10T03:16:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:39 smithi098 ceph-mon[103124]: pgmap v7751: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 175 KiB/s wr, 12 op/s 2023-09-10T03:16:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14230.933808 secs 2023-09-10T03:16:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14230.222459 secs 2023-09-10T03:16:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:39 smithi170 ceph-mon[102832]: pgmap v7751: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 175 KiB/s wr, 12 op/s 2023-09-10T03:16:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14230.933808 secs 2023-09-10T03:16:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14230.222459 secs 2023-09-10T03:16:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:39 smithi082 ceph-mon[98694]: pgmap v7751: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 175 KiB/s wr, 12 op/s 2023-09-10T03:16:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14230.933808 secs 2023-09-10T03:16:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:16:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:16:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:16:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:41 smithi098 ceph-mon[103124]: pgmap v7752: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 415 KiB/s wr, 19 op/s 2023-09-10T03:16:41.763 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:41 smithi082 ceph-mon[98694]: pgmap v7752: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 415 KiB/s wr, 19 op/s 2023-09-10T03:16:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:41 smithi170 ceph-mon[102832]: pgmap v7752: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 415 KiB/s wr, 19 op/s 2023-09-10T03:16:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:43 smithi098 ceph-mon[103124]: pgmap v7753: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 256 KiB/s wr, 12 op/s 2023-09-10T03:16:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:16:43.786 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:43 smithi082 ceph-mon[98694]: pgmap v7753: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 256 KiB/s wr, 12 op/s 2023-09-10T03:16:43.786 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:16:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:43 smithi170 ceph-mon[102832]: pgmap v7753: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 256 KiB/s wr, 12 op/s 2023-09-10T03:16:43.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:16:44.617 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14235.222503 secs 2023-09-10T03:16:44.617 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:44 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:16:44.618 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:16:44.618 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:16:44.618 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14235.933875 secs 2023-09-10T03:16:44.618 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:16:44.619 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:16:44.771 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14235.222503 secs 2023-09-10T03:16:44.771 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:44 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:16:44.771 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:16:44.772 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:16:44.772 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14235.933875 secs 2023-09-10T03:16:44.772 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:16:44.772 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:16:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14235.222503 secs 2023-09-10T03:16:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:44 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:16:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:16:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:16:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14235.933875 secs 2023-09-10T03:16:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:16:44.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:16:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:45 smithi170 ceph-mon[102832]: pgmap v7754: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 256 KiB/s wr, 12 op/s 2023-09-10T03:16:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:16:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:16:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:16:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:16:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:16:45.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:16:45.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:16:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:45 smithi082 ceph-mon[98694]: pgmap v7754: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 256 KiB/s wr, 12 op/s 2023-09-10T03:16:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:16:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:16:45.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:16:45.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:16:45.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:16:45.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:16:45.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:16:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:45 smithi098 ceph-mon[103124]: pgmap v7754: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 256 KiB/s wr, 12 op/s 2023-09-10T03:16:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:16:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:16:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:16:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:16:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:16:46.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:16:46.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:16:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:47 smithi170 ceph-mon[102832]: pgmap v7755: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 289 KiB/s wr, 17 op/s 2023-09-10T03:16:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:47 smithi082 ceph-mon[98694]: pgmap v7755: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 289 KiB/s wr, 17 op/s 2023-09-10T03:16:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:47 smithi098 ceph-mon[103124]: pgmap v7755: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 289 KiB/s wr, 17 op/s 2023-09-10T03:16:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14240.222562 secs 2023-09-10T03:16:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14240.222562 secs 2023-09-10T03:16:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14240.222562 secs 2023-09-10T03:16:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:49 smithi170 ceph-mon[102832]: pgmap v7756: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T03:16:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14240.933942 secs 2023-09-10T03:16:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:49 smithi082 ceph-mon[98694]: pgmap v7756: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T03:16:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14240.933942 secs 2023-09-10T03:16:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:49 smithi098 ceph-mon[103124]: pgmap v7756: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T03:16:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14240.933942 secs 2023-09-10T03:16:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:51 smithi170 ceph-mon[102832]: pgmap v7757: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 354 KiB/s wr, 17 op/s 2023-09-10T03:16:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:51 smithi082 ceph-mon[98694]: pgmap v7757: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 354 KiB/s wr, 17 op/s 2023-09-10T03:16:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:51 smithi098 ceph-mon[103124]: pgmap v7757: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 354 KiB/s wr, 17 op/s 2023-09-10T03:16:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14243.891863 secs 2023-09-10T03:16:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:53 smithi098 ceph-mon[103124]: pgmap v7758: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 10 op/s 2023-09-10T03:16:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14243.891863 secs 2023-09-10T03:16:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:53 smithi170 ceph-mon[102832]: pgmap v7758: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 10 op/s 2023-09-10T03:16:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14243.891863 secs 2023-09-10T03:16:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:53 smithi082 ceph-mon[98694]: pgmap v7758: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 10 op/s 2023-09-10T03:16:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14245.222618 secs 2023-09-10T03:16:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:16:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:16:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14245.934007 secs 2023-09-10T03:16:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14245.222618 secs 2023-09-10T03:16:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:16:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:16:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14245.934007 secs 2023-09-10T03:16:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14245.222618 secs 2023-09-10T03:16:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:16:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:16:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14245.934007 secs 2023-09-10T03:16:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:55 smithi098 ceph-mon[103124]: pgmap v7759: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 10 op/s 2023-09-10T03:16:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:55 smithi170 ceph-mon[102832]: pgmap v7759: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 10 op/s 2023-09-10T03:16:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:55 smithi082 ceph-mon[98694]: pgmap v7759: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 10 op/s 2023-09-10T03:16:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:16:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:16:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:16:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:16:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:16:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:16:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:16:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:16:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:16:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:57 smithi098 ceph-mon[103124]: pgmap v7760: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 224 KiB/s wr, 16 op/s 2023-09-10T03:16:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:57 smithi170 ceph-mon[102832]: pgmap v7760: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 224 KiB/s wr, 16 op/s 2023-09-10T03:16:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:57 smithi082 ceph-mon[98694]: pgmap v7760: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 224 KiB/s wr, 16 op/s 2023-09-10T03:16:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14248.891920 secs 2023-09-10T03:16:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14248.891920 secs 2023-09-10T03:16:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14248.891920 secs 2023-09-10T03:16:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14250.222679 secs 2023-09-10T03:16:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:59 smithi098 ceph-mon[103124]: pgmap v7761: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T03:16:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:16:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14250.934074 secs 2023-09-10T03:16:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14250.222679 secs 2023-09-10T03:16:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:59 smithi170 ceph-mon[102832]: pgmap v7761: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T03:16:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:16:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14250.934074 secs 2023-09-10T03:16:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14250.222679 secs 2023-09-10T03:16:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:59 smithi082 ceph-mon[98694]: pgmap v7761: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T03:16:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:16:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14250.934074 secs 2023-09-10T03:17:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:01 smithi098 ceph-mon[103124]: pgmap v7762: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 232 KiB/s wr, 17 op/s 2023-09-10T03:17:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:01 smithi170 ceph-mon[102832]: pgmap v7762: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 232 KiB/s wr, 17 op/s 2023-09-10T03:17:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:01 smithi082 ceph-mon[98694]: pgmap v7762: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 232 KiB/s wr, 17 op/s 2023-09-10T03:17:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14253.892010 secs 2023-09-10T03:17:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:03 smithi098 ceph-mon[103124]: pgmap v7763: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T03:17:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14253.892010 secs 2023-09-10T03:17:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:03 smithi170 ceph-mon[102832]: pgmap v7763: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T03:17:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14253.892010 secs 2023-09-10T03:17:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:03 smithi082 ceph-mon[98694]: pgmap v7763: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T03:17:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14255.222759 secs 2023-09-10T03:17:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14255.934137 secs 2023-09-10T03:17:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14255.222759 secs 2023-09-10T03:17:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14255.934137 secs 2023-09-10T03:17:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14255.222759 secs 2023-09-10T03:17:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14255.934137 secs 2023-09-10T03:17:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:05 smithi098 ceph-mon[103124]: pgmap v7764: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T03:17:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:05 smithi170 ceph-mon[102832]: pgmap v7764: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T03:17:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:05 smithi082 ceph-mon[98694]: pgmap v7764: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T03:17:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:07 smithi098 ceph-mon[103124]: pgmap v7765: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 206 KiB/s wr, 30 op/s 2023-09-10T03:17:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:07 smithi170 ceph-mon[102832]: pgmap v7765: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 206 KiB/s wr, 30 op/s 2023-09-10T03:17:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:07 smithi082 ceph-mon[98694]: pgmap v7765: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 206 KiB/s wr, 30 op/s 2023-09-10T03:17:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14258.892069 secs 2023-09-10T03:17:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:08 smithi170 ceph-mon[102832]: osdmap e553: 12 total, 12 up, 12 in 2023-09-10T03:17:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14258.892069 secs 2023-09-10T03:17:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:08 smithi082 ceph-mon[98694]: osdmap e553: 12 total, 12 up, 12 in 2023-09-10T03:17:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14258.892069 secs 2023-09-10T03:17:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:08 smithi098 ceph-mon[103124]: osdmap e553: 12 total, 12 up, 12 in 2023-09-10T03:17:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14260.222818 secs 2023-09-10T03:17:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:09 smithi170 ceph-mon[102832]: pgmap v7767: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 116 KiB/s wr, 30 op/s 2023-09-10T03:17:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14260.934201 secs 2023-09-10T03:17:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14260.222818 secs 2023-09-10T03:17:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:09 smithi082 ceph-mon[98694]: pgmap v7767: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 116 KiB/s wr, 30 op/s 2023-09-10T03:17:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14260.934201 secs 2023-09-10T03:17:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14260.222818 secs 2023-09-10T03:17:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:09 smithi098 ceph-mon[103124]: pgmap v7767: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 116 KiB/s wr, 30 op/s 2023-09-10T03:17:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14260.934201 secs 2023-09-10T03:17:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:11 smithi170 ceph-mon[102832]: pgmap v7768: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T03:17:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:11 smithi082 ceph-mon[98694]: pgmap v7768: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T03:17:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:11 smithi098 ceph-mon[103124]: pgmap v7768: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 97 KiB/s wr, 29 op/s 2023-09-10T03:17:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14263.892145 secs 2023-09-10T03:17:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:13 smithi098 ceph-mon[103124]: osdmap e554: 12 total, 12 up, 12 in 2023-09-10T03:17:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:13 smithi098 ceph-mon[103124]: pgmap v7770: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 121 KiB/s wr, 36 op/s 2023-09-10T03:17:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14263.892145 secs 2023-09-10T03:17:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:13 smithi170 ceph-mon[102832]: osdmap e554: 12 total, 12 up, 12 in 2023-09-10T03:17:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:13 smithi170 ceph-mon[102832]: pgmap v7770: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 121 KiB/s wr, 36 op/s 2023-09-10T03:17:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14263.892145 secs 2023-09-10T03:17:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:13 smithi082 ceph-mon[98694]: osdmap e554: 12 total, 12 up, 12 in 2023-09-10T03:17:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:13 smithi082 ceph-mon[98694]: pgmap v7770: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 121 KiB/s wr, 36 op/s 2023-09-10T03:17:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14265.222897 secs 2023-09-10T03:17:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14265.934290 secs 2023-09-10T03:17:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14265.222897 secs 2023-09-10T03:17:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14265.934290 secs 2023-09-10T03:17:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14265.222897 secs 2023-09-10T03:17:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14265.934290 secs 2023-09-10T03:17:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:15 smithi098 ceph-mon[103124]: pgmap v7771: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 7 op/s 2023-09-10T03:17:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:15 smithi170 ceph-mon[102832]: pgmap v7771: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 7 op/s 2023-09-10T03:17:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:15 smithi082 ceph-mon[98694]: pgmap v7771: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 7 op/s 2023-09-10T03:17:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:17 smithi098 ceph-mon[103124]: pgmap v7772: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 13 op/s 2023-09-10T03:17:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:17 smithi170 ceph-mon[102832]: pgmap v7772: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 13 op/s 2023-09-10T03:17:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:17 smithi082 ceph-mon[98694]: pgmap v7772: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 13 op/s 2023-09-10T03:17:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14268.892241 secs 2023-09-10T03:17:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14268.892241 secs 2023-09-10T03:17:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14268.892241 secs 2023-09-10T03:17:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14270.222982 secs 2023-09-10T03:17:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:19 smithi170 ceph-mon[102832]: pgmap v7773: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T03:17:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14270.934357 secs 2023-09-10T03:17:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14270.222982 secs 2023-09-10T03:17:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:19 smithi082 ceph-mon[98694]: pgmap v7773: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T03:17:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14270.934357 secs 2023-09-10T03:17:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14270.222982 secs 2023-09-10T03:17:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:19 smithi098 ceph-mon[103124]: pgmap v7773: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T03:17:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14270.934357 secs 2023-09-10T03:17:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:21 smithi170 ceph-mon[102832]: pgmap v7774: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-10T03:17:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:21 smithi082 ceph-mon[98694]: pgmap v7774: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-10T03:17:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:21 smithi098 ceph-mon[103124]: pgmap v7774: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-10T03:17:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:23 smithi098 ceph-mon[103124]: pgmap v7775: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-10T03:17:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:17:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:23 smithi170 ceph-mon[102832]: pgmap v7775: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-10T03:17:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:17:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:23 smithi082 ceph-mon[98694]: pgmap v7775: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 11 op/s 2023-09-10T03:17:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:17:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14275.223076 secs 2023-09-10T03:17:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:17:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:17:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:17:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14275.934422 secs 2023-09-10T03:17:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14275.223076 secs 2023-09-10T03:17:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:17:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:17:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:17:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14275.934422 secs 2023-09-10T03:17:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14275.223076 secs 2023-09-10T03:17:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:17:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:17:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:17:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14275.934422 secs 2023-09-10T03:17:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:25 smithi098 ceph-mon[103124]: pgmap v7776: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 9 op/s 2023-09-10T03:17:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:25 smithi170 ceph-mon[102832]: pgmap v7776: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 9 op/s 2023-09-10T03:17:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:25 smithi082 ceph-mon[98694]: pgmap v7776: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 9 op/s 2023-09-10T03:17:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:27 smithi170 ceph-mon[102832]: pgmap v7777: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 114 KiB/s wr, 14 op/s 2023-09-10T03:17:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:27 smithi082 ceph-mon[98694]: pgmap v7777: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 114 KiB/s wr, 14 op/s 2023-09-10T03:17:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:27 smithi098 ceph-mon[103124]: pgmap v7777: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 114 KiB/s wr, 14 op/s 2023-09-10T03:17:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14280.223135 secs 2023-09-10T03:17:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:29 smithi170 ceph-mon[102832]: pgmap v7778: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 9 op/s 2023-09-10T03:17:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14280.934489 secs 2023-09-10T03:17:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14280.223135 secs 2023-09-10T03:17:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:29 smithi082 ceph-mon[98694]: pgmap v7778: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 9 op/s 2023-09-10T03:17:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14280.934489 secs 2023-09-10T03:17:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14280.223135 secs 2023-09-10T03:17:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:29 smithi098 ceph-mon[103124]: pgmap v7778: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 9 op/s 2023-09-10T03:17:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14280.934489 secs 2023-09-10T03:17:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:31 smithi170 ceph-mon[102832]: pgmap v7779: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 161 KiB/s wr, 15 op/s 2023-09-10T03:17:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:31 smithi082 ceph-mon[98694]: pgmap v7779: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 161 KiB/s wr, 15 op/s 2023-09-10T03:17:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:31 smithi098 ceph-mon[103124]: pgmap v7779: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 161 KiB/s wr, 15 op/s 2023-09-10T03:17:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:33 smithi098 ceph-mon[103124]: pgmap v7780: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 161 KiB/s wr, 10 op/s 2023-09-10T03:17:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:33 smithi170 ceph-mon[102832]: pgmap v7780: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 161 KiB/s wr, 10 op/s 2023-09-10T03:17:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:33 smithi082 ceph-mon[98694]: pgmap v7780: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 161 KiB/s wr, 10 op/s 2023-09-10T03:17:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14285.223190 secs 2023-09-10T03:17:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:17:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:17:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14285.934561 secs 2023-09-10T03:17:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14285.223190 secs 2023-09-10T03:17:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:17:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:17:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14285.934561 secs 2023-09-10T03:17:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14285.223190 secs 2023-09-10T03:17:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:17:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:17:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14285.934561 secs 2023-09-10T03:17:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:35 smithi098 ceph-mon[103124]: pgmap v7781: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 161 KiB/s wr, 10 op/s 2023-09-10T03:17:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:35 smithi170 ceph-mon[102832]: pgmap v7781: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 161 KiB/s wr, 10 op/s 2023-09-10T03:17:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:35 smithi082 ceph-mon[98694]: pgmap v7781: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 161 KiB/s wr, 10 op/s 2023-09-10T03:17:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:17:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:17:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:17:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:17:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:17:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:17:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:37 smithi170 ceph-mon[102832]: pgmap v7782: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 230 KiB/s wr, 17 op/s 2023-09-10T03:17:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:37 smithi082 ceph-mon[98694]: pgmap v7782: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 230 KiB/s wr, 17 op/s 2023-09-10T03:17:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:37 smithi098 ceph-mon[103124]: pgmap v7782: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 230 KiB/s wr, 17 op/s 2023-09-10T03:17:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14288.892490 secs 2023-09-10T03:17:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14288.892490 secs 2023-09-10T03:17:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14288.892490 secs 2023-09-10T03:17:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14290.223249 secs 2023-09-10T03:17:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:39 smithi170 ceph-mon[102832]: pgmap v7783: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 162 KiB/s wr, 12 op/s 2023-09-10T03:17:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14290.934620 secs 2023-09-10T03:17:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14290.223249 secs 2023-09-10T03:17:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:39 smithi082 ceph-mon[98694]: pgmap v7783: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 162 KiB/s wr, 12 op/s 2023-09-10T03:17:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14290.934620 secs 2023-09-10T03:17:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14290.223249 secs 2023-09-10T03:17:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:39 smithi098 ceph-mon[103124]: pgmap v7783: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 162 KiB/s wr, 12 op/s 2023-09-10T03:17:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14290.934620 secs 2023-09-10T03:17:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:17:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:17:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:17:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:17:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:17:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:17:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:17:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:17:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:17:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:41 smithi170 ceph-mon[102832]: pgmap v7784: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 308 KiB/s wr, 19 op/s 2023-09-10T03:17:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:41 smithi082 ceph-mon[98694]: pgmap v7784: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 308 KiB/s wr, 19 op/s 2023-09-10T03:17:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:41 smithi098 ceph-mon[103124]: pgmap v7784: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 308 KiB/s wr, 19 op/s 2023-09-10T03:17:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14293.892553 secs 2023-09-10T03:17:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:43 smithi098 ceph-mon[103124]: pgmap v7785: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 215 KiB/s wr, 13 op/s 2023-09-10T03:17:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14293.892553 secs 2023-09-10T03:17:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:43 smithi170 ceph-mon[102832]: pgmap v7785: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 215 KiB/s wr, 13 op/s 2023-09-10T03:17:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14293.892553 secs 2023-09-10T03:17:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:43 smithi082 ceph-mon[98694]: pgmap v7785: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 215 KiB/s wr, 13 op/s 2023-09-10T03:17:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14295.223303 secs 2023-09-10T03:17:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14295.934685 secs 2023-09-10T03:17:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14295.223303 secs 2023-09-10T03:17:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14295.934685 secs 2023-09-10T03:17:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14295.223303 secs 2023-09-10T03:17:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14295.934685 secs 2023-09-10T03:17:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:45 smithi098 ceph-mon[103124]: pgmap v7786: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 215 KiB/s wr, 13 op/s 2023-09-10T03:17:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:17:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:45 smithi170 ceph-mon[102832]: pgmap v7786: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 215 KiB/s wr, 13 op/s 2023-09-10T03:17:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:17:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:45 smithi082 ceph-mon[98694]: pgmap v7786: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 215 KiB/s wr, 13 op/s 2023-09-10T03:17:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:17:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:17:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:17:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:17:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:17:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:17:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:17:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:17:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:17:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:17:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:47 smithi170 ceph-mon[102832]: pgmap v7787: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 237 KiB/s wr, 18 op/s 2023-09-10T03:17:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:47 smithi082 ceph-mon[98694]: pgmap v7787: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 237 KiB/s wr, 18 op/s 2023-09-10T03:17:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:47 smithi098 ceph-mon[103124]: pgmap v7787: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 237 KiB/s wr, 18 op/s 2023-09-10T03:17:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14298.892637 secs 2023-09-10T03:17:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14298.892637 secs 2023-09-10T03:17:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14298.892637 secs 2023-09-10T03:17:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14300.223360 secs 2023-09-10T03:17:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:49 smithi170 ceph-mon[102832]: pgmap v7788: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 168 KiB/s wr, 11 op/s 2023-09-10T03:17:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14300.934749 secs 2023-09-10T03:17:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14300.223360 secs 2023-09-10T03:17:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:49 smithi082 ceph-mon[98694]: pgmap v7788: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 168 KiB/s wr, 11 op/s 2023-09-10T03:17:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14300.934749 secs 2023-09-10T03:17:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14300.223360 secs 2023-09-10T03:17:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:49 smithi098 ceph-mon[103124]: pgmap v7788: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 168 KiB/s wr, 11 op/s 2023-09-10T03:17:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14300.934749 secs 2023-09-10T03:17:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:51 smithi170 ceph-mon[102832]: pgmap v7789: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 346 KiB/s wr, 17 op/s 2023-09-10T03:17:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:51 smithi082 ceph-mon[98694]: pgmap v7789: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 346 KiB/s wr, 17 op/s 2023-09-10T03:17:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:51 smithi098 ceph-mon[103124]: pgmap v7789: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 346 KiB/s wr, 17 op/s 2023-09-10T03:17:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14303.892699 secs 2023-09-10T03:17:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:53 smithi098 ceph-mon[103124]: pgmap v7790: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 200 KiB/s wr, 11 op/s 2023-09-10T03:17:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14303.892699 secs 2023-09-10T03:17:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:53 smithi170 ceph-mon[102832]: pgmap v7790: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 200 KiB/s wr, 11 op/s 2023-09-10T03:17:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14303.892699 secs 2023-09-10T03:17:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:53 smithi082 ceph-mon[98694]: pgmap v7790: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 200 KiB/s wr, 11 op/s 2023-09-10T03:17:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14305.223433 secs 2023-09-10T03:17:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:17:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:17:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14305.934812 secs 2023-09-10T03:17:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14305.223433 secs 2023-09-10T03:17:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:17:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:17:54.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14305.934812 secs 2023-09-10T03:17:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14305.223433 secs 2023-09-10T03:17:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:17:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:17:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14305.934812 secs 2023-09-10T03:17:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:55 smithi098 ceph-mon[103124]: pgmap v7791: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 200 KiB/s wr, 11 op/s 2023-09-10T03:17:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:55 smithi170 ceph-mon[102832]: pgmap v7791: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 200 KiB/s wr, 11 op/s 2023-09-10T03:17:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:55 smithi082 ceph-mon[98694]: pgmap v7791: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 200 KiB/s wr, 11 op/s 2023-09-10T03:17:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:17:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:17:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:17:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:17:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:17:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:17:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:57 smithi170 ceph-mon[102832]: pgmap v7792: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 200 KiB/s wr, 16 op/s 2023-09-10T03:17:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:57 smithi082 ceph-mon[98694]: pgmap v7792: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 200 KiB/s wr, 16 op/s 2023-09-10T03:17:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:57 smithi098 ceph-mon[103124]: pgmap v7792: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 200 KiB/s wr, 16 op/s 2023-09-10T03:17:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14308.892758 secs 2023-09-10T03:17:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14308.892758 secs 2023-09-10T03:17:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14308.892758 secs 2023-09-10T03:17:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14310.223494 secs 2023-09-10T03:17:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:59 smithi170 ceph-mon[102832]: pgmap v7793: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 179 KiB/s wr, 11 op/s 2023-09-10T03:17:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:17:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14310.934876 secs 2023-09-10T03:17:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14310.223494 secs 2023-09-10T03:17:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:59 smithi082 ceph-mon[98694]: pgmap v7793: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 179 KiB/s wr, 11 op/s 2023-09-10T03:17:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:17:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14310.934876 secs 2023-09-10T03:18:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14310.223494 secs 2023-09-10T03:18:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:59 smithi098 ceph-mon[103124]: pgmap v7793: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 179 KiB/s wr, 11 op/s 2023-09-10T03:18:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:17:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14310.934876 secs 2023-09-10T03:18:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:01 smithi170 ceph-mon[102832]: pgmap v7794: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 284 KiB/s wr, 17 op/s 2023-09-10T03:18:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:01 smithi082 ceph-mon[98694]: pgmap v7794: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 284 KiB/s wr, 17 op/s 2023-09-10T03:18:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:01 smithi098 ceph-mon[103124]: pgmap v7794: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 284 KiB/s wr, 17 op/s 2023-09-10T03:18:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14313.892777 secs 2023-09-10T03:18:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:03 smithi098 ceph-mon[103124]: pgmap v7795: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 106 KiB/s wr, 10 op/s 2023-09-10T03:18:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14313.892777 secs 2023-09-10T03:18:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:03 smithi170 ceph-mon[102832]: pgmap v7795: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 106 KiB/s wr, 10 op/s 2023-09-10T03:18:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14313.892777 secs 2023-09-10T03:18:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:03 smithi082 ceph-mon[98694]: pgmap v7795: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 106 KiB/s wr, 10 op/s 2023-09-10T03:18:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14315.223576 secs 2023-09-10T03:18:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14315.934943 secs 2023-09-10T03:18:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14315.223576 secs 2023-09-10T03:18:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14315.934943 secs 2023-09-10T03:18:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14315.223576 secs 2023-09-10T03:18:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14315.934943 secs 2023-09-10T03:18:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:05 smithi170 ceph-mon[102832]: pgmap v7796: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 106 KiB/s wr, 10 op/s 2023-09-10T03:18:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:05 smithi082 ceph-mon[98694]: pgmap v7796: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 106 KiB/s wr, 10 op/s 2023-09-10T03:18:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:05 smithi098 ceph-mon[103124]: pgmap v7796: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 106 KiB/s wr, 10 op/s 2023-09-10T03:18:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:07 smithi170 ceph-mon[102832]: pgmap v7797: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 151 KiB/s wr, 29 op/s 2023-09-10T03:18:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:07 smithi082 ceph-mon[98694]: pgmap v7797: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 151 KiB/s wr, 29 op/s 2023-09-10T03:18:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:07 smithi098 ceph-mon[103124]: pgmap v7797: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 151 KiB/s wr, 29 op/s 2023-09-10T03:18:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14318.892836 secs 2023-09-10T03:18:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:08 smithi170 ceph-mon[102832]: osdmap e555: 12 total, 12 up, 12 in 2023-09-10T03:18:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14318.892836 secs 2023-09-10T03:18:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:08 smithi082 ceph-mon[98694]: osdmap e555: 12 total, 12 up, 12 in 2023-09-10T03:18:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14318.892836 secs 2023-09-10T03:18:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:08 smithi098 ceph-mon[103124]: osdmap e555: 12 total, 12 up, 12 in 2023-09-10T03:18:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14320.223642 secs 2023-09-10T03:18:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:09 smithi170 ceph-mon[102832]: pgmap v7799: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 181 KiB/s wr, 29 op/s 2023-09-10T03:18:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14320.935006 secs 2023-09-10T03:18:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14320.223642 secs 2023-09-10T03:18:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:09 smithi082 ceph-mon[98694]: pgmap v7799: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 181 KiB/s wr, 29 op/s 2023-09-10T03:18:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14320.935006 secs 2023-09-10T03:18:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14320.223642 secs 2023-09-10T03:18:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:09 smithi098 ceph-mon[103124]: pgmap v7799: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 181 KiB/s wr, 29 op/s 2023-09-10T03:18:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14320.935006 secs 2023-09-10T03:18:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:11 smithi170 ceph-mon[102832]: pgmap v7800: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 110 KiB/s wr, 29 op/s 2023-09-10T03:18:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:11 smithi082 ceph-mon[98694]: pgmap v7800: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 110 KiB/s wr, 29 op/s 2023-09-10T03:18:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:11 smithi098 ceph-mon[103124]: pgmap v7800: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 110 KiB/s wr, 29 op/s 2023-09-10T03:18:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14323.892899 secs 2023-09-10T03:18:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:13 smithi098 ceph-mon[103124]: osdmap e556: 12 total, 12 up, 12 in 2023-09-10T03:18:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:13 smithi098 ceph-mon[103124]: pgmap v7802: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 138 KiB/s wr, 36 op/s 2023-09-10T03:18:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14323.892899 secs 2023-09-10T03:18:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:13 smithi170 ceph-mon[102832]: osdmap e556: 12 total, 12 up, 12 in 2023-09-10T03:18:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:13 smithi170 ceph-mon[102832]: pgmap v7802: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 138 KiB/s wr, 36 op/s 2023-09-10T03:18:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14323.892899 secs 2023-09-10T03:18:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:13 smithi082 ceph-mon[98694]: osdmap e556: 12 total, 12 up, 12 in 2023-09-10T03:18:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:13 smithi082 ceph-mon[98694]: pgmap v7802: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 138 KiB/s wr, 36 op/s 2023-09-10T03:18:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14325.223697 secs 2023-09-10T03:18:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14325.935071 secs 2023-09-10T03:18:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14325.223697 secs 2023-09-10T03:18:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14325.935071 secs 2023-09-10T03:18:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14325.223697 secs 2023-09-10T03:18:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14325.935071 secs 2023-09-10T03:18:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:15 smithi170 ceph-mon[102832]: pgmap v7803: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 70 KiB/s wr, 7 op/s 2023-09-10T03:18:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:15 smithi082 ceph-mon[98694]: pgmap v7803: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 70 KiB/s wr, 7 op/s 2023-09-10T03:18:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:15 smithi098 ceph-mon[103124]: pgmap v7803: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 70 KiB/s wr, 7 op/s 2023-09-10T03:18:17.787 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:17 smithi170 ceph-mon[102832]: pgmap v7804: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 63 KiB/s wr, 13 op/s 2023-09-10T03:18:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:17 smithi082 ceph-mon[98694]: pgmap v7804: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 63 KiB/s wr, 13 op/s 2023-09-10T03:18:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:17 smithi098 ceph-mon[103124]: pgmap v7804: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 63 KiB/s wr, 13 op/s 2023-09-10T03:18:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14328.892955 secs 2023-09-10T03:18:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14328.892955 secs 2023-09-10T03:18:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14328.892955 secs 2023-09-10T03:18:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14330.223778 secs 2023-09-10T03:18:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:19 smithi170 ceph-mon[102832]: pgmap v7805: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 12 op/s 2023-09-10T03:18:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14330.935134 secs 2023-09-10T03:18:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14330.223778 secs 2023-09-10T03:18:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:19 smithi082 ceph-mon[98694]: pgmap v7805: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 12 op/s 2023-09-10T03:18:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14330.935134 secs 2023-09-10T03:18:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14330.223778 secs 2023-09-10T03:18:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:19 smithi098 ceph-mon[103124]: pgmap v7805: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 12 op/s 2023-09-10T03:18:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14330.935134 secs 2023-09-10T03:18:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:21 smithi170 ceph-mon[102832]: pgmap v7806: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 11 op/s 2023-09-10T03:18:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:21 smithi082 ceph-mon[98694]: pgmap v7806: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 11 op/s 2023-09-10T03:18:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:21 smithi098 ceph-mon[103124]: pgmap v7806: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 11 op/s 2023-09-10T03:18:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:23 smithi170 ceph-mon[102832]: pgmap v7807: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 11 op/s 2023-09-10T03:18:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:18:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:23 smithi082 ceph-mon[98694]: pgmap v7807: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 11 op/s 2023-09-10T03:18:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:18:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:23 smithi098 ceph-mon[103124]: pgmap v7807: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 11 op/s 2023-09-10T03:18:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:18:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14335.223837 secs 2023-09-10T03:18:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:18:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:18:24.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:18:24.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14335.935200 secs 2023-09-10T03:18:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14335.223837 secs 2023-09-10T03:18:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:18:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:18:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:18:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14335.935200 secs 2023-09-10T03:18:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14335.223837 secs 2023-09-10T03:18:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:18:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:18:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:18:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14335.935200 secs 2023-09-10T03:18:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:25 smithi170 ceph-mon[102832]: pgmap v7808: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 9 op/s 2023-09-10T03:18:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:25 smithi082 ceph-mon[98694]: pgmap v7808: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 9 op/s 2023-09-10T03:18:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:25 smithi098 ceph-mon[103124]: pgmap v7808: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 9 op/s 2023-09-10T03:18:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:27 smithi170 ceph-mon[102832]: pgmap v7809: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 14 op/s 2023-09-10T03:18:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:27 smithi082 ceph-mon[98694]: pgmap v7809: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 14 op/s 2023-09-10T03:18:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:27 smithi098 ceph-mon[103124]: pgmap v7809: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 14 op/s 2023-09-10T03:18:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14340.223894 secs 2023-09-10T03:18:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:29 smithi170 ceph-mon[102832]: pgmap v7810: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 9 op/s 2023-09-10T03:18:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:29 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:18:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:29 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:18:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14340.935286 secs 2023-09-10T03:18:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14340.223894 secs 2023-09-10T03:18:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:29 smithi082 ceph-mon[98694]: pgmap v7810: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 9 op/s 2023-09-10T03:18:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:29 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:18:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:29 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:18:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14340.935286 secs 2023-09-10T03:18:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14340.223894 secs 2023-09-10T03:18:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:29 smithi098 ceph-mon[103124]: pgmap v7810: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 9 op/s 2023-09-10T03:18:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:29 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:18:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:29 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:18:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14340.935286 secs 2023-09-10T03:18:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:31 smithi170 ceph-mon[102832]: pgmap v7811: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 331 KiB/s wr, 16 op/s 2023-09-10T03:18:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:31 smithi082 ceph-mon[98694]: pgmap v7811: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 331 KiB/s wr, 16 op/s 2023-09-10T03:18:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:31 smithi098 ceph-mon[103124]: pgmap v7811: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 331 KiB/s wr, 16 op/s 2023-09-10T03:18:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:18:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:18:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:18:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:18:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:18:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:18:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:33 smithi170 ceph-mon[102832]: pgmap v7812: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 285 KiB/s wr, 11 op/s 2023-09-10T03:18:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:33 smithi082 ceph-mon[98694]: pgmap v7812: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 285 KiB/s wr, 11 op/s 2023-09-10T03:18:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:33 smithi098 ceph-mon[103124]: pgmap v7812: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 285 KiB/s wr, 11 op/s 2023-09-10T03:18:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14345.223950 secs 2023-09-10T03:18:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14345.935350 secs 2023-09-10T03:18:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14345.223950 secs 2023-09-10T03:18:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14345.935350 secs 2023-09-10T03:18:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14345.223950 secs 2023-09-10T03:18:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14345.935350 secs 2023-09-10T03:18:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:35 smithi170 ceph-mon[102832]: pgmap v7813: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 285 KiB/s wr, 11 op/s 2023-09-10T03:18:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:35 smithi082 ceph-mon[98694]: pgmap v7813: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 285 KiB/s wr, 11 op/s 2023-09-10T03:18:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:35 smithi098 ceph-mon[103124]: pgmap v7813: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 285 KiB/s wr, 11 op/s 2023-09-10T03:18:37.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:37 smithi170 ceph-mon[102832]: pgmap v7814: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 302 KiB/s wr, 16 op/s 2023-09-10T03:18:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:37 smithi082 ceph-mon[98694]: pgmap v7814: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 302 KiB/s wr, 16 op/s 2023-09-10T03:18:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:37 smithi098 ceph-mon[103124]: pgmap v7814: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 302 KiB/s wr, 16 op/s 2023-09-10T03:18:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14350.224009 secs 2023-09-10T03:18:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:39 smithi170 ceph-mon[102832]: pgmap v7815: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 255 KiB/s wr, 12 op/s 2023-09-10T03:18:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14350.935416 secs 2023-09-10T03:18:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14350.224009 secs 2023-09-10T03:18:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:39 smithi082 ceph-mon[98694]: pgmap v7815: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 255 KiB/s wr, 12 op/s 2023-09-10T03:18:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14350.935416 secs 2023-09-10T03:18:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14350.224009 secs 2023-09-10T03:18:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:39 smithi098 ceph-mon[103124]: pgmap v7815: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 255 KiB/s wr, 12 op/s 2023-09-10T03:18:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14350.935416 secs 2023-09-10T03:18:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:41 smithi170 ceph-mon[102832]: pgmap v7816: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 559 KiB/s wr, 19 op/s 2023-09-10T03:18:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:41 smithi082 ceph-mon[98694]: pgmap v7816: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 559 KiB/s wr, 19 op/s 2023-09-10T03:18:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:41 smithi098 ceph-mon[103124]: pgmap v7816: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 559 KiB/s wr, 19 op/s 2023-09-10T03:18:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14353.893314 secs 2023-09-10T03:18:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:43 smithi170 ceph-mon[102832]: pgmap v7817: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 321 KiB/s wr, 12 op/s 2023-09-10T03:18:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14353.893314 secs 2023-09-10T03:18:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:43 smithi082 ceph-mon[98694]: pgmap v7817: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 321 KiB/s wr, 12 op/s 2023-09-10T03:18:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14353.893314 secs 2023-09-10T03:18:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:43 smithi098 ceph-mon[103124]: pgmap v7817: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 321 KiB/s wr, 12 op/s 2023-09-10T03:18:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14355.224091 secs 2023-09-10T03:18:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:18:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:18:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:18:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14355.935488 secs 2023-09-10T03:18:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14355.224091 secs 2023-09-10T03:18:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:18:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:18:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:18:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14355.935488 secs 2023-09-10T03:18:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14355.224091 secs 2023-09-10T03:18:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:18:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:18:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:18:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14355.935488 secs 2023-09-10T03:18:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:45 smithi170 ceph-mon[102832]: pgmap v7818: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 321 KiB/s wr, 12 op/s 2023-09-10T03:18:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:45 smithi082 ceph-mon[98694]: pgmap v7818: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 321 KiB/s wr, 12 op/s 2023-09-10T03:18:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:45 smithi098 ceph-mon[103124]: pgmap v7818: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 321 KiB/s wr, 12 op/s 2023-09-10T03:18:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:18:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:18:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:18:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:18:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:18:46.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:18:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:18:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:18:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:18:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:18:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:18:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:18:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:18:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:18:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:18:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:18:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:18:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:18:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:47 smithi170 ceph-mon[102832]: pgmap v7819: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 421 KiB/s wr, 17 op/s 2023-09-10T03:18:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:47 smithi082 ceph-mon[98694]: pgmap v7819: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 421 KiB/s wr, 17 op/s 2023-09-10T03:18:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:47 smithi098 ceph-mon[103124]: pgmap v7819: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 421 KiB/s wr, 17 op/s 2023-09-10T03:18:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14358.893400 secs 2023-09-10T03:18:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14358.893400 secs 2023-09-10T03:18:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14358.893400 secs 2023-09-10T03:18:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14360.224184 secs 2023-09-10T03:18:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:49 smithi170 ceph-mon[102832]: pgmap v7820: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 404 KiB/s wr, 12 op/s 2023-09-10T03:18:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14360.935554 secs 2023-09-10T03:18:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14360.224184 secs 2023-09-10T03:18:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:49 smithi082 ceph-mon[98694]: pgmap v7820: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 404 KiB/s wr, 12 op/s 2023-09-10T03:18:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14360.935554 secs 2023-09-10T03:18:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14360.224184 secs 2023-09-10T03:18:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:49 smithi098 ceph-mon[103124]: pgmap v7820: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 404 KiB/s wr, 12 op/s 2023-09-10T03:18:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14360.935554 secs 2023-09-10T03:18:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:51 smithi170 ceph-mon[102832]: pgmap v7821: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 499 KiB/s wr, 18 op/s 2023-09-10T03:18:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:51 smithi082 ceph-mon[98694]: pgmap v7821: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 499 KiB/s wr, 18 op/s 2023-09-10T03:18:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:51 smithi098 ceph-mon[103124]: pgmap v7821: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 499 KiB/s wr, 18 op/s 2023-09-10T03:18:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14363.893494 secs 2023-09-10T03:18:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:53 smithi170 ceph-mon[102832]: pgmap v7822: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 10 op/s 2023-09-10T03:18:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14363.893494 secs 2023-09-10T03:18:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:53 smithi082 ceph-mon[98694]: pgmap v7822: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 10 op/s 2023-09-10T03:18:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14363.893494 secs 2023-09-10T03:18:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:53 smithi098 ceph-mon[103124]: pgmap v7822: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 10 op/s 2023-09-10T03:18:54.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14365.224240 secs 2023-09-10T03:18:54.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14365.935622 secs 2023-09-10T03:18:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14365.224240 secs 2023-09-10T03:18:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14365.935622 secs 2023-09-10T03:18:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14365.224240 secs 2023-09-10T03:18:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14365.935622 secs 2023-09-10T03:18:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:55 smithi170 ceph-mon[102832]: pgmap v7823: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 10 op/s 2023-09-10T03:18:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:55 smithi082 ceph-mon[98694]: pgmap v7823: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 10 op/s 2023-09-10T03:18:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:55 smithi098 ceph-mon[103124]: pgmap v7823: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 10 op/s 2023-09-10T03:18:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:57 smithi170 ceph-mon[102832]: pgmap v7824: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 226 KiB/s wr, 15 op/s 2023-09-10T03:18:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:57 smithi082 ceph-mon[98694]: pgmap v7824: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 226 KiB/s wr, 15 op/s 2023-09-10T03:18:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:57 smithi098 ceph-mon[103124]: pgmap v7824: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 226 KiB/s wr, 15 op/s 2023-09-10T03:18:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14368.893553 secs 2023-09-10T03:18:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14368.893553 secs 2023-09-10T03:18:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14368.893553 secs 2023-09-10T03:18:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14370.224323 secs 2023-09-10T03:18:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:59 smithi170 ceph-mon[102832]: pgmap v7825: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 9 op/s 2023-09-10T03:18:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:18:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14370.935687 secs 2023-09-10T03:18:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14370.224323 secs 2023-09-10T03:18:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:59 smithi082 ceph-mon[98694]: pgmap v7825: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 9 op/s 2023-09-10T03:18:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:18:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14370.935687 secs 2023-09-10T03:19:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14370.224323 secs 2023-09-10T03:19:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:59 smithi098 ceph-mon[103124]: pgmap v7825: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 9 op/s 2023-09-10T03:19:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:18:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14370.935687 secs 2023-09-10T03:19:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:01 smithi170 ceph-mon[102832]: pgmap v7826: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 161 KiB/s wr, 14 op/s 2023-09-10T03:19:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:01 smithi082 ceph-mon[98694]: pgmap v7826: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 161 KiB/s wr, 14 op/s 2023-09-10T03:19:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:01 smithi098 ceph-mon[103124]: pgmap v7826: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 161 KiB/s wr, 14 op/s 2023-09-10T03:19:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14373.893615 secs 2023-09-10T03:19:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:03 smithi170 ceph-mon[102832]: pgmap v7827: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 9 op/s 2023-09-10T03:19:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14373.893615 secs 2023-09-10T03:19:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:03 smithi082 ceph-mon[98694]: pgmap v7827: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 9 op/s 2023-09-10T03:19:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14373.893615 secs 2023-09-10T03:19:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:03 smithi098 ceph-mon[103124]: pgmap v7827: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 9 op/s 2023-09-10T03:19:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14375.224377 secs 2023-09-10T03:19:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14375.935751 secs 2023-09-10T03:19:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14375.224377 secs 2023-09-10T03:19:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14375.935751 secs 2023-09-10T03:19:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14375.224377 secs 2023-09-10T03:19:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14375.935751 secs 2023-09-10T03:19:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:05 smithi170 ceph-mon[102832]: pgmap v7828: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 9 op/s 2023-09-10T03:19:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:05 smithi082 ceph-mon[98694]: pgmap v7828: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 9 op/s 2023-09-10T03:19:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:05 smithi098 ceph-mon[103124]: pgmap v7828: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 9 op/s 2023-09-10T03:19:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:07 smithi170 ceph-mon[102832]: pgmap v7829: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 122 KiB/s wr, 29 op/s 2023-09-10T03:19:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:07 smithi082 ceph-mon[98694]: pgmap v7829: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 122 KiB/s wr, 29 op/s 2023-09-10T03:19:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:07 smithi098 ceph-mon[103124]: pgmap v7829: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 122 KiB/s wr, 29 op/s 2023-09-10T03:19:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14378.893724 secs 2023-09-10T03:19:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:08 smithi170 ceph-mon[102832]: osdmap e557: 12 total, 12 up, 12 in 2023-09-10T03:19:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14378.893724 secs 2023-09-10T03:19:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:08 smithi082 ceph-mon[98694]: osdmap e557: 12 total, 12 up, 12 in 2023-09-10T03:19:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14378.893724 secs 2023-09-10T03:19:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:08 smithi098 ceph-mon[103124]: osdmap e557: 12 total, 12 up, 12 in 2023-09-10T03:19:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14380.224433 secs 2023-09-10T03:19:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:09 smithi170 ceph-mon[102832]: pgmap v7831: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 110 KiB/s wr, 29 op/s 2023-09-10T03:19:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14380.935815 secs 2023-09-10T03:19:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14380.224433 secs 2023-09-10T03:19:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:09 smithi082 ceph-mon[98694]: pgmap v7831: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 110 KiB/s wr, 29 op/s 2023-09-10T03:19:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14380.935815 secs 2023-09-10T03:19:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14380.224433 secs 2023-09-10T03:19:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:09 smithi098 ceph-mon[103124]: pgmap v7831: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 110 KiB/s wr, 29 op/s 2023-09-10T03:19:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14380.935815 secs 2023-09-10T03:19:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:11 smithi170 ceph-mon[102832]: pgmap v7832: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-10T03:19:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:11 smithi082 ceph-mon[98694]: pgmap v7832: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-10T03:19:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:11 smithi098 ceph-mon[103124]: pgmap v7832: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-10T03:19:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14383.893799 secs 2023-09-10T03:19:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:13 smithi170 ceph-mon[102832]: osdmap e558: 12 total, 12 up, 12 in 2023-09-10T03:19:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:13 smithi170 ceph-mon[102832]: pgmap v7834: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 86 KiB/s wr, 37 op/s 2023-09-10T03:19:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14383.893799 secs 2023-09-10T03:19:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:13 smithi082 ceph-mon[98694]: osdmap e558: 12 total, 12 up, 12 in 2023-09-10T03:19:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:13 smithi082 ceph-mon[98694]: pgmap v7834: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 86 KiB/s wr, 37 op/s 2023-09-10T03:19:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14383.893799 secs 2023-09-10T03:19:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:13 smithi098 ceph-mon[103124]: osdmap e558: 12 total, 12 up, 12 in 2023-09-10T03:19:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:13 smithi098 ceph-mon[103124]: pgmap v7834: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 86 KiB/s wr, 37 op/s 2023-09-10T03:19:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14385.224512 secs 2023-09-10T03:19:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14385.935879 secs 2023-09-10T03:19:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14385.224512 secs 2023-09-10T03:19:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14385.935879 secs 2023-09-10T03:19:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14385.224512 secs 2023-09-10T03:19:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14385.935879 secs 2023-09-10T03:19:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:15 smithi170 ceph-mon[102832]: pgmap v7835: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2023-09-10T03:19:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:15 smithi082 ceph-mon[98694]: pgmap v7835: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2023-09-10T03:19:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:15 smithi098 ceph-mon[103124]: pgmap v7835: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2023-09-10T03:19:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:17 smithi170 ceph-mon[102832]: pgmap v7836: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 62 KiB/s wr, 13 op/s 2023-09-10T03:19:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:17 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:19:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:17 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:19:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:17 smithi082 ceph-mon[98694]: pgmap v7836: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 62 KiB/s wr, 13 op/s 2023-09-10T03:19:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:17 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:19:17.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:17 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:19:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:17 smithi098 ceph-mon[103124]: pgmap v7836: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 62 KiB/s wr, 13 op/s 2023-09-10T03:19:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:17 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:19:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:17 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:19:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14388.893860 secs 2023-09-10T03:19:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14388.893860 secs 2023-09-10T03:19:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14388.893860 secs 2023-09-10T03:19:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14390.224571 secs 2023-09-10T03:19:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:19 smithi170 ceph-mon[102832]: pgmap v7837: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 12 op/s 2023-09-10T03:19:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14390.935948 secs 2023-09-10T03:19:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14390.224571 secs 2023-09-10T03:19:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:19 smithi082 ceph-mon[98694]: pgmap v7837: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 12 op/s 2023-09-10T03:19:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14390.935948 secs 2023-09-10T03:19:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14390.224571 secs 2023-09-10T03:19:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:19 smithi098 ceph-mon[103124]: pgmap v7837: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 12 op/s 2023-09-10T03:19:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14390.935948 secs 2023-09-10T03:19:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:19:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:19:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:19:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:19:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:19:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:19:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:21 smithi170 ceph-mon[102832]: pgmap v7838: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T03:19:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:21 smithi082 ceph-mon[98694]: pgmap v7838: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T03:19:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:21 smithi098 ceph-mon[103124]: pgmap v7838: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T03:19:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:23 smithi170 ceph-mon[102832]: pgmap v7839: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T03:19:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:19:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:23 smithi082 ceph-mon[98694]: pgmap v7839: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T03:19:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:19:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:23 smithi098 ceph-mon[103124]: pgmap v7839: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T03:19:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:19:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14395.224630 secs 2023-09-10T03:19:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:19:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:19:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:19:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14395.936011 secs 2023-09-10T03:19:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14395.224630 secs 2023-09-10T03:19:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:19:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:19:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:19:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14395.936011 secs 2023-09-10T03:19:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14395.224630 secs 2023-09-10T03:19:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:19:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:19:25.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:19:25.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14395.936011 secs 2023-09-10T03:19:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:25 smithi170 ceph-mon[102832]: pgmap v7840: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T03:19:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:25 smithi082 ceph-mon[98694]: pgmap v7840: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T03:19:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:25 smithi098 ceph-mon[103124]: pgmap v7840: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T03:19:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:27 smithi170 ceph-mon[102832]: pgmap v7841: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 145 KiB/s wr, 15 op/s 2023-09-10T03:19:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:27 smithi082 ceph-mon[98694]: pgmap v7841: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 145 KiB/s wr, 15 op/s 2023-09-10T03:19:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:27 smithi098 ceph-mon[103124]: pgmap v7841: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 145 KiB/s wr, 15 op/s 2023-09-10T03:19:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14400.224690 secs 2023-09-10T03:19:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:29 smithi170 ceph-mon[102832]: pgmap v7842: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 10 op/s 2023-09-10T03:19:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14400.936077 secs 2023-09-10T03:19:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14400.224690 secs 2023-09-10T03:19:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:29 smithi082 ceph-mon[98694]: pgmap v7842: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 10 op/s 2023-09-10T03:19:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14400.936077 secs 2023-09-10T03:19:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14400.224690 secs 2023-09-10T03:19:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:29 smithi098 ceph-mon[103124]: pgmap v7842: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 10 op/s 2023-09-10T03:19:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14400.936077 secs 2023-09-10T03:19:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:31 smithi170 ceph-mon[102832]: pgmap v7843: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 194 KiB/s wr, 16 op/s 2023-09-10T03:19:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:31 smithi082 ceph-mon[98694]: pgmap v7843: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 194 KiB/s wr, 16 op/s 2023-09-10T03:19:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:31 smithi098 ceph-mon[103124]: pgmap v7843: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 194 KiB/s wr, 16 op/s 2023-09-10T03:19:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:33 smithi170 ceph-mon[102832]: pgmap v7844: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T03:19:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:33 smithi082 ceph-mon[98694]: pgmap v7844: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T03:19:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:33 smithi098 ceph-mon[103124]: pgmap v7844: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T03:19:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14405.224744 secs 2023-09-10T03:19:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14405.936144 secs 2023-09-10T03:19:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14405.224744 secs 2023-09-10T03:19:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14405.936144 secs 2023-09-10T03:19:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14405.224744 secs 2023-09-10T03:19:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14405.936144 secs 2023-09-10T03:19:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:35 smithi170 ceph-mon[102832]: pgmap v7845: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T03:19:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:35 smithi082 ceph-mon[98694]: pgmap v7845: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T03:19:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:35 smithi098 ceph-mon[103124]: pgmap v7845: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T03:19:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:19:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:19:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:19:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:19:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:19:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:19:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:37 smithi170 ceph-mon[102832]: pgmap v7846: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 304 KiB/s wr, 17 op/s 2023-09-10T03:19:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:19:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:19:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:37 smithi082 ceph-mon[98694]: pgmap v7846: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 304 KiB/s wr, 17 op/s 2023-09-10T03:19:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:19:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:19:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:37 smithi098 ceph-mon[103124]: pgmap v7846: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 304 KiB/s wr, 17 op/s 2023-09-10T03:19:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:19:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:19:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14410.224807 secs 2023-09-10T03:19:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:39 smithi170 ceph-mon[102832]: pgmap v7847: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 248 KiB/s wr, 12 op/s 2023-09-10T03:19:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14410.936208 secs 2023-09-10T03:19:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14410.224807 secs 2023-09-10T03:19:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:39 smithi082 ceph-mon[98694]: pgmap v7847: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 248 KiB/s wr, 12 op/s 2023-09-10T03:19:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14410.936208 secs 2023-09-10T03:19:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14410.224807 secs 2023-09-10T03:19:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:39 smithi098 ceph-mon[103124]: pgmap v7847: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 248 KiB/s wr, 12 op/s 2023-09-10T03:19:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14410.936208 secs 2023-09-10T03:19:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:41 smithi170 ceph-mon[102832]: pgmap v7848: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 413 KiB/s wr, 18 op/s 2023-09-10T03:19:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:41 smithi082 ceph-mon[98694]: pgmap v7848: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 413 KiB/s wr, 18 op/s 2023-09-10T03:19:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:41 smithi098 ceph-mon[103124]: pgmap v7848: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 413 KiB/s wr, 18 op/s 2023-09-10T03:19:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:43 smithi170 ceph-mon[102832]: pgmap v7849: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 319 KiB/s wr, 12 op/s 2023-09-10T03:19:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:43 smithi082 ceph-mon[98694]: pgmap v7849: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 319 KiB/s wr, 12 op/s 2023-09-10T03:19:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:43 smithi098 ceph-mon[103124]: pgmap v7849: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 319 KiB/s wr, 12 op/s 2023-09-10T03:19:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14415.224888 secs 2023-09-10T03:19:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14415.936275 secs 2023-09-10T03:19:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14415.224888 secs 2023-09-10T03:19:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14415.936275 secs 2023-09-10T03:19:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14415.224888 secs 2023-09-10T03:19:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14415.936275 secs 2023-09-10T03:19:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:45 smithi170 ceph-mon[102832]: pgmap v7850: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 319 KiB/s wr, 12 op/s 2023-09-10T03:19:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:45 smithi082 ceph-mon[98694]: pgmap v7850: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 319 KiB/s wr, 12 op/s 2023-09-10T03:19:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:45 smithi098 ceph-mon[103124]: pgmap v7850: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 319 KiB/s wr, 12 op/s 2023-09-10T03:19:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:47 smithi170 ceph-mon[102832]: pgmap v7851: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 460 KiB/s wr, 18 op/s 2023-09-10T03:19:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:19:47.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:47 smithi082 ceph-mon[98694]: pgmap v7851: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 460 KiB/s wr, 18 op/s 2023-09-10T03:19:47.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:19:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:47 smithi098 ceph-mon[103124]: pgmap v7851: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 460 KiB/s wr, 18 op/s 2023-09-10T03:19:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:19:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14418.894375 secs 2023-09-10T03:19:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14418.894375 secs 2023-09-10T03:19:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14418.894375 secs 2023-09-10T03:19:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14420.224949 secs 2023-09-10T03:19:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:49 smithi170 ceph-mon[102832]: pgmap v7852: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 306 KiB/s wr, 11 op/s 2023-09-10T03:19:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:19:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:19:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14420.936342 secs 2023-09-10T03:19:49.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:49.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:49.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:49.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14420.224949 secs 2023-09-10T03:19:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:49 smithi082 ceph-mon[98694]: pgmap v7852: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 306 KiB/s wr, 11 op/s 2023-09-10T03:19:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:19:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:19:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14420.936342 secs 2023-09-10T03:19:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:49.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:49.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14420.224949 secs 2023-09-10T03:19:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:49 smithi098 ceph-mon[103124]: pgmap v7852: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 306 KiB/s wr, 11 op/s 2023-09-10T03:19:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:19:50.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:19:50.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14420.936342 secs 2023-09-10T03:19:50.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:50.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:50.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:50.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:50.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:50.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:50.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:50.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:50.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:50.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:50.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:50.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:50.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:50.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:51.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:51.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:51.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:51.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:51.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:51.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:51.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:51.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:51.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:51.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:51.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:51.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:51.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:51.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:51.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:51.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:51.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:51.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:51.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:51 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T03:19:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:51 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T03:19:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:51 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 119.9M 2023-09-10T03:19:52.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:51 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 125751488: error parsing value: Value '125751488' is below minimum 939524096 2023-09-10T03:19:52.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:51 smithi098 ceph-mon[103124]: pgmap v7853: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 433 KiB/s wr, 18 op/s 2023-09-10T03:19:52.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:52.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:52.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:52.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:52.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:52.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:52.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:51 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T03:19:52.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:51 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T03:19:52.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:19:52.008 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:19:52.008 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:51 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T03:19:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:51 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T03:19:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:51 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 119.9M 2023-09-10T03:19:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:51 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 125751488: error parsing value: Value '125751488' is below minimum 939524096 2023-09-10T03:19:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:51 smithi170 ceph-mon[102832]: pgmap v7853: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 433 KiB/s wr, 18 op/s 2023-09-10T03:19:52.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:52.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:52.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:52.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:52.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:52.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:52.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:51 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T03:19:52.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:51 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T03:19:52.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:19:52.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:19:52.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:51 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T03:19:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:51 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T03:19:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:51 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 119.9M 2023-09-10T03:19:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:51 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 125751488: error parsing value: Value '125751488' is below minimum 939524096 2023-09-10T03:19:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:51 smithi082 ceph-mon[98694]: pgmap v7853: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 433 KiB/s wr, 18 op/s 2023-09-10T03:19:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:52.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:52.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:52.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:19:52.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:51 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T03:19:52.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:51 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T03:19:52.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:19:52.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:19:52.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:19:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:19:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:19:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:19:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:19:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:19:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:19:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:19:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:19:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:19:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14423.894446 secs 2023-09-10T03:19:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:53 smithi098 ceph-mon[103124]: pgmap v7854: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T03:19:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14425.225053 secs 2023-09-10T03:19:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14423.894446 secs 2023-09-10T03:19:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:53 smithi170 ceph-mon[102832]: pgmap v7854: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T03:19:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14425.225053 secs 2023-09-10T03:19:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14423.894446 secs 2023-09-10T03:19:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:53 smithi082 ceph-mon[98694]: pgmap v7854: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T03:19:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14425.225053 secs 2023-09-10T03:19:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14425.936405 secs 2023-09-10T03:19:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14425.936405 secs 2023-09-10T03:19:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14425.936405 secs 2023-09-10T03:19:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:55 smithi098 ceph-mon[103124]: pgmap v7855: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T03:19:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:55 smithi170 ceph-mon[102832]: pgmap v7855: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T03:19:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:55 smithi082 ceph-mon[98694]: pgmap v7855: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T03:19:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:57 smithi098 ceph-mon[103124]: pgmap v7856: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 364 KiB/s wr, 17 op/s 2023-09-10T03:19:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:57 smithi170 ceph-mon[102832]: pgmap v7856: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 364 KiB/s wr, 17 op/s 2023-09-10T03:19:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:57 smithi082 ceph-mon[98694]: pgmap v7856: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 364 KiB/s wr, 17 op/s 2023-09-10T03:19:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14428.894502 secs 2023-09-10T03:19:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14430.225111 secs 2023-09-10T03:19:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14428.894502 secs 2023-09-10T03:19:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14430.225111 secs 2023-09-10T03:19:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14428.894502 secs 2023-09-10T03:19:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14430.225111 secs 2023-09-10T03:20:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:59 smithi170 ceph-mon[102832]: pgmap v7857: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T03:20:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:19:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14430.936474 secs 2023-09-10T03:20:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:59 smithi082 ceph-mon[98694]: pgmap v7857: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T03:20:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:19:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14430.936474 secs 2023-09-10T03:20:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:59 smithi098 ceph-mon[103124]: pgmap v7857: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T03:20:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:19:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14430.936474 secs 2023-09-10T03:20:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T03:20:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T03:20:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T03:20:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:01 smithi170 ceph-mon[102832]: pgmap v7858: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 16 op/s 2023-09-10T03:20:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:01 smithi082 ceph-mon[98694]: pgmap v7858: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 16 op/s 2023-09-10T03:20:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:01 smithi098 ceph-mon[103124]: pgmap v7858: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 16 op/s 2023-09-10T03:20:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14433.894672 secs 2023-09-10T03:20:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:03 smithi170 ceph-mon[102832]: pgmap v7859: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T03:20:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14433.894672 secs 2023-09-10T03:20:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:03 smithi082 ceph-mon[98694]: pgmap v7859: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T03:20:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14433.894672 secs 2023-09-10T03:20:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:03 smithi098 ceph-mon[103124]: pgmap v7859: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T03:20:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14435.225166 secs 2023-09-10T03:20:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14435.936491 secs 2023-09-10T03:20:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14435.225166 secs 2023-09-10T03:20:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14435.936491 secs 2023-09-10T03:20:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14435.225166 secs 2023-09-10T03:20:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14435.936491 secs 2023-09-10T03:20:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:05 smithi170 ceph-mon[102832]: pgmap v7860: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T03:20:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:05 smithi082 ceph-mon[98694]: pgmap v7860: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T03:20:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:05 smithi098 ceph-mon[103124]: pgmap v7860: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T03:20:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:07 smithi170 ceph-mon[102832]: pgmap v7861: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 183 KiB/s wr, 29 op/s 2023-09-10T03:20:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:07 smithi082 ceph-mon[98694]: pgmap v7861: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 183 KiB/s wr, 29 op/s 2023-09-10T03:20:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:07 smithi098 ceph-mon[103124]: pgmap v7861: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 183 KiB/s wr, 29 op/s 2023-09-10T03:20:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14438.894799 secs 2023-09-10T03:20:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:08 smithi170 ceph-mon[102832]: osdmap e559: 12 total, 12 up, 12 in 2023-09-10T03:20:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14438.894799 secs 2023-09-10T03:20:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:08 smithi082 ceph-mon[98694]: osdmap e559: 12 total, 12 up, 12 in 2023-09-10T03:20:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14438.894799 secs 2023-09-10T03:20:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:08 smithi098 ceph-mon[103124]: osdmap e559: 12 total, 12 up, 12 in 2023-09-10T03:20:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14440.225250 secs 2023-09-10T03:20:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:09 smithi170 ceph-mon[102832]: pgmap v7863: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 104 KiB/s wr, 29 op/s 2023-09-10T03:20:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14440.936555 secs 2023-09-10T03:20:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14440.225250 secs 2023-09-10T03:20:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:09 smithi082 ceph-mon[98694]: pgmap v7863: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 104 KiB/s wr, 29 op/s 2023-09-10T03:20:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14440.936555 secs 2023-09-10T03:20:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14440.225250 secs 2023-09-10T03:20:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:09 smithi098 ceph-mon[103124]: pgmap v7863: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 104 KiB/s wr, 29 op/s 2023-09-10T03:20:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14440.936555 secs 2023-09-10T03:20:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:11 smithi170 ceph-mon[102832]: pgmap v7864: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 68 KiB/s wr, 29 op/s 2023-09-10T03:20:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:11 smithi082 ceph-mon[98694]: pgmap v7864: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 68 KiB/s wr, 29 op/s 2023-09-10T03:20:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:11 smithi098 ceph-mon[103124]: pgmap v7864: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 68 KiB/s wr, 29 op/s 2023-09-10T03:20:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14443.894889 secs 2023-09-10T03:20:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:13 smithi170 ceph-mon[102832]: osdmap e560: 12 total, 12 up, 12 in 2023-09-10T03:20:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:13 smithi170 ceph-mon[102832]: pgmap v7866: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 85 KiB/s wr, 36 op/s 2023-09-10T03:20:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14443.894889 secs 2023-09-10T03:20:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:13 smithi082 ceph-mon[98694]: osdmap e560: 12 total, 12 up, 12 in 2023-09-10T03:20:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:13 smithi082 ceph-mon[98694]: pgmap v7866: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 85 KiB/s wr, 36 op/s 2023-09-10T03:20:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14443.894889 secs 2023-09-10T03:20:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:13 smithi098 ceph-mon[103124]: osdmap e560: 12 total, 12 up, 12 in 2023-09-10T03:20:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:13 smithi098 ceph-mon[103124]: pgmap v7866: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 85 KiB/s wr, 36 op/s 2023-09-10T03:20:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14445.225305 secs 2023-09-10T03:20:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14445.936618 secs 2023-09-10T03:20:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14445.225305 secs 2023-09-10T03:20:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14445.936618 secs 2023-09-10T03:20:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14445.225305 secs 2023-09-10T03:20:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14445.936618 secs 2023-09-10T03:20:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:15 smithi170 ceph-mon[102832]: pgmap v7867: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 7 op/s 2023-09-10T03:20:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:15 smithi082 ceph-mon[98694]: pgmap v7867: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 7 op/s 2023-09-10T03:20:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:15 smithi098 ceph-mon[103124]: pgmap v7867: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 7 op/s 2023-09-10T03:20:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:17 smithi170 ceph-mon[102832]: pgmap v7868: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 13 op/s 2023-09-10T03:20:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:17 smithi082 ceph-mon[98694]: pgmap v7868: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 13 op/s 2023-09-10T03:20:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:17 smithi098 ceph-mon[103124]: pgmap v7868: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 13 op/s 2023-09-10T03:20:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14448.894948 secs 2023-09-10T03:20:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14448.894948 secs 2023-09-10T03:20:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14448.894948 secs 2023-09-10T03:20:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14450.225364 secs 2023-09-10T03:20:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:19 smithi170 ceph-mon[102832]: pgmap v7869: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T03:20:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14450.936690 secs 2023-09-10T03:20:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14450.225364 secs 2023-09-10T03:20:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:19 smithi082 ceph-mon[98694]: pgmap v7869: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T03:20:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14450.936690 secs 2023-09-10T03:20:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14450.225364 secs 2023-09-10T03:20:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:19 smithi098 ceph-mon[103124]: pgmap v7869: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T03:20:20.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14450.936690 secs 2023-09-10T03:20:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:21 smithi170 ceph-mon[102832]: pgmap v7870: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 11 op/s 2023-09-10T03:20:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:21 smithi082 ceph-mon[98694]: pgmap v7870: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 11 op/s 2023-09-10T03:20:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:21 smithi098 ceph-mon[103124]: pgmap v7870: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 11 op/s 2023-09-10T03:20:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:23 smithi170 ceph-mon[102832]: pgmap v7871: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 11 op/s 2023-09-10T03:20:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:20:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:23 smithi082 ceph-mon[98694]: pgmap v7871: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 11 op/s 2023-09-10T03:20:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:20:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:23 smithi098 ceph-mon[103124]: pgmap v7871: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 11 op/s 2023-09-10T03:20:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:20:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14455.225446 secs 2023-09-10T03:20:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:20:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:20:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:20:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14455.936751 secs 2023-09-10T03:20:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14455.225446 secs 2023-09-10T03:20:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:20:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:20:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:20:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14455.936751 secs 2023-09-10T03:20:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14455.225446 secs 2023-09-10T03:20:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:20:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:20:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:20:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14455.936751 secs 2023-09-10T03:20:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:25 smithi170 ceph-mon[102832]: pgmap v7872: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 9 op/s 2023-09-10T03:20:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:25 smithi082 ceph-mon[98694]: pgmap v7872: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 9 op/s 2023-09-10T03:20:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:25 smithi098 ceph-mon[103124]: pgmap v7872: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 9 op/s 2023-09-10T03:20:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:27 smithi170 ceph-mon[102832]: pgmap v7873: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 136 KiB/s wr, 14 op/s 2023-09-10T03:20:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:27 smithi082 ceph-mon[98694]: pgmap v7873: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 136 KiB/s wr, 14 op/s 2023-09-10T03:20:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:27 smithi098 ceph-mon[103124]: pgmap v7873: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 136 KiB/s wr, 14 op/s 2023-09-10T03:20:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14460.225497 secs 2023-09-10T03:20:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:29 smithi170 ceph-mon[102832]: pgmap v7874: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 90 KiB/s wr, 9 op/s 2023-09-10T03:20:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14460.936816 secs 2023-09-10T03:20:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14460.225497 secs 2023-09-10T03:20:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:29 smithi082 ceph-mon[98694]: pgmap v7874: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 90 KiB/s wr, 9 op/s 2023-09-10T03:20:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14460.936816 secs 2023-09-10T03:20:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14460.225497 secs 2023-09-10T03:20:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:29 smithi098 ceph-mon[103124]: pgmap v7874: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 90 KiB/s wr, 9 op/s 2023-09-10T03:20:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14460.936816 secs 2023-09-10T03:20:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:31 smithi170 ceph-mon[102832]: pgmap v7875: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 174 KiB/s wr, 15 op/s 2023-09-10T03:20:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:31 smithi082 ceph-mon[98694]: pgmap v7875: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 174 KiB/s wr, 15 op/s 2023-09-10T03:20:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:31 smithi098 ceph-mon[103124]: pgmap v7875: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 174 KiB/s wr, 15 op/s 2023-09-10T03:20:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:33 smithi170 ceph-mon[102832]: pgmap v7876: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T03:20:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:20:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:20:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14465.225509 secs 2023-09-10T03:20:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:33 smithi082 ceph-mon[98694]: pgmap v7876: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T03:20:34.019 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:20:34.019 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:20:34.020 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14465.225509 secs 2023-09-10T03:20:34.020 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:33 smithi098 ceph-mon[103124]: pgmap v7876: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T03:20:34.020 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:20:34.021 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:20:34.021 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14465.225509 secs 2023-09-10T03:20:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14465.936880 secs 2023-09-10T03:20:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14465.936880 secs 2023-09-10T03:20:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14465.936880 secs 2023-09-10T03:20:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:35 smithi170 ceph-mon[102832]: pgmap v7877: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T03:20:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:35 smithi082 ceph-mon[98694]: pgmap v7877: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T03:20:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:35 smithi098 ceph-mon[103124]: pgmap v7877: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T03:20:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:20:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:20:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:20:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:20:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:20:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:20:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:37 smithi170 ceph-mon[102832]: pgmap v7878: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 208 KiB/s wr, 17 op/s 2023-09-10T03:20:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:37 smithi082 ceph-mon[98694]: pgmap v7878: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 208 KiB/s wr, 17 op/s 2023-09-10T03:20:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:37 smithi098 ceph-mon[103124]: pgmap v7878: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 208 KiB/s wr, 17 op/s 2023-09-10T03:20:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14470.225592 secs 2023-09-10T03:20:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14470.225592 secs 2023-09-10T03:20:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14470.225592 secs 2023-09-10T03:20:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:39 smithi170 ceph-mon[102832]: pgmap v7879: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T03:20:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14470.936942 secs 2023-09-10T03:20:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:39 smithi082 ceph-mon[98694]: pgmap v7879: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T03:20:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14470.936942 secs 2023-09-10T03:20:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:39 smithi098 ceph-mon[103124]: pgmap v7879: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T03:20:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14470.936942 secs 2023-09-10T03:20:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:41 smithi170 ceph-mon[102832]: pgmap v7880: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 298 KiB/s wr, 19 op/s 2023-09-10T03:20:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:41 smithi082 ceph-mon[98694]: pgmap v7880: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 298 KiB/s wr, 19 op/s 2023-09-10T03:20:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:41 smithi098 ceph-mon[103124]: pgmap v7880: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 298 KiB/s wr, 19 op/s 2023-09-10T03:20:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:43 smithi170 ceph-mon[102832]: pgmap v7881: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 214 KiB/s wr, 13 op/s 2023-09-10T03:20:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:43 smithi082 ceph-mon[98694]: pgmap v7881: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 214 KiB/s wr, 13 op/s 2023-09-10T03:20:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:43 smithi098 ceph-mon[103124]: pgmap v7881: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 214 KiB/s wr, 13 op/s 2023-09-10T03:20:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14475.225667 secs 2023-09-10T03:20:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14475.937005 secs 2023-09-10T03:20:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14475.225667 secs 2023-09-10T03:20:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14475.937005 secs 2023-09-10T03:20:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14475.225667 secs 2023-09-10T03:20:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14475.937005 secs 2023-09-10T03:20:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:45 smithi170 ceph-mon[102832]: pgmap v7882: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 214 KiB/s wr, 13 op/s 2023-09-10T03:20:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:45 smithi082 ceph-mon[98694]: pgmap v7882: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 214 KiB/s wr, 13 op/s 2023-09-10T03:20:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:45 smithi098 ceph-mon[103124]: pgmap v7882: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 214 KiB/s wr, 13 op/s 2023-09-10T03:20:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:47 smithi170 ceph-mon[102832]: pgmap v7883: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 371 KiB/s wr, 19 op/s 2023-09-10T03:20:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:47 smithi082 ceph-mon[98694]: pgmap v7883: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 371 KiB/s wr, 19 op/s 2023-09-10T03:20:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:47 smithi098 ceph-mon[103124]: pgmap v7883: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 371 KiB/s wr, 19 op/s 2023-09-10T03:20:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14478.895398 secs 2023-09-10T03:20:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14480.225722 secs 2023-09-10T03:20:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14478.895398 secs 2023-09-10T03:20:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14480.225722 secs 2023-09-10T03:20:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14478.895398 secs 2023-09-10T03:20:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14480.225722 secs 2023-09-10T03:20:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:49 smithi170 ceph-mon[102832]: pgmap v7884: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 337 KiB/s wr, 12 op/s 2023-09-10T03:20:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:20:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:20:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14480.937068 secs 2023-09-10T03:20:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:49 smithi082 ceph-mon[98694]: pgmap v7884: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 337 KiB/s wr, 12 op/s 2023-09-10T03:20:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:20:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:20:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14480.937068 secs 2023-09-10T03:20:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:49 smithi098 ceph-mon[103124]: pgmap v7884: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 337 KiB/s wr, 12 op/s 2023-09-10T03:20:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:20:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:20:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14480.937068 secs 2023-09-10T03:20:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:51 smithi170 ceph-mon[102832]: pgmap v7885: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 439 KiB/s wr, 19 op/s 2023-09-10T03:20:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:20:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:20:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:20:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:20:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:51 smithi082 ceph-mon[98694]: pgmap v7885: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 439 KiB/s wr, 19 op/s 2023-09-10T03:20:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:20:51.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:20:51.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:20:51.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:20:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:51 smithi098 ceph-mon[103124]: pgmap v7885: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 439 KiB/s wr, 19 op/s 2023-09-10T03:20:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:20:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:20:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:20:52.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:20:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:20:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:20:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:20:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:20:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:20:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:20:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:20:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:20:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:20:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14483.895491 secs 2023-09-10T03:20:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:53 smithi170 ceph-mon[102832]: pgmap v7886: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T03:20:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14485.225794 secs 2023-09-10T03:20:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14483.895491 secs 2023-09-10T03:20:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:53 smithi082 ceph-mon[98694]: pgmap v7886: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T03:20:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14485.225794 secs 2023-09-10T03:20:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14483.895491 secs 2023-09-10T03:20:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:53 smithi098 ceph-mon[103124]: pgmap v7886: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T03:20:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14485.225794 secs 2023-09-10T03:20:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14485.937133 secs 2023-09-10T03:20:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14485.937133 secs 2023-09-10T03:20:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14485.937133 secs 2023-09-10T03:20:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:55 smithi170 ceph-mon[102832]: pgmap v7887: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T03:20:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:55 smithi082 ceph-mon[98694]: pgmap v7887: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T03:20:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:55 smithi098 ceph-mon[103124]: pgmap v7887: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T03:20:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:57 smithi170 ceph-mon[102832]: pgmap v7888: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 334 KiB/s wr, 18 op/s 2023-09-10T03:20:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:57 smithi082 ceph-mon[98694]: pgmap v7888: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 334 KiB/s wr, 18 op/s 2023-09-10T03:20:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:57 smithi098 ceph-mon[103124]: pgmap v7888: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 334 KiB/s wr, 18 op/s 2023-09-10T03:20:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14488.895553 secs 2023-09-10T03:20:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14490.225872 secs 2023-09-10T03:20:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14488.895553 secs 2023-09-10T03:20:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14490.225872 secs 2023-09-10T03:20:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14488.895553 secs 2023-09-10T03:20:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14490.225872 secs 2023-09-10T03:20:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:59 smithi170 ceph-mon[102832]: pgmap v7889: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 176 KiB/s wr, 12 op/s 2023-09-10T03:20:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:20:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14490.937200 secs 2023-09-10T03:20:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:59 smithi082 ceph-mon[98694]: pgmap v7889: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 176 KiB/s wr, 12 op/s 2023-09-10T03:20:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:20:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14490.937200 secs 2023-09-10T03:21:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:59 smithi098 ceph-mon[103124]: pgmap v7889: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 176 KiB/s wr, 12 op/s 2023-09-10T03:21:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:20:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14490.937200 secs 2023-09-10T03:21:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:01 smithi170 ceph-mon[102832]: pgmap v7890: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 243 KiB/s wr, 33 op/s 2023-09-10T03:21:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:01 smithi082 ceph-mon[98694]: pgmap v7890: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 243 KiB/s wr, 33 op/s 2023-09-10T03:21:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:01 smithi098 ceph-mon[103124]: pgmap v7890: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 243 KiB/s wr, 33 op/s 2023-09-10T03:21:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14493.895617 secs 2023-09-10T03:21:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:03 smithi170 ceph-mon[102832]: pgmap v7891: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 141 KiB/s wr, 26 op/s 2023-09-10T03:21:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14493.895617 secs 2023-09-10T03:21:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:03 smithi082 ceph-mon[98694]: pgmap v7891: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 141 KiB/s wr, 26 op/s 2023-09-10T03:21:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14493.895617 secs 2023-09-10T03:21:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:03 smithi098 ceph-mon[103124]: pgmap v7891: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 141 KiB/s wr, 26 op/s 2023-09-10T03:21:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14495.225928 secs 2023-09-10T03:21:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14495.937266 secs 2023-09-10T03:21:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14495.225928 secs 2023-09-10T03:21:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14495.937266 secs 2023-09-10T03:21:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14495.225928 secs 2023-09-10T03:21:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14495.937266 secs 2023-09-10T03:21:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:05 smithi170 ceph-mon[102832]: pgmap v7892: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 141 KiB/s wr, 26 op/s 2023-09-10T03:21:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:05 smithi082 ceph-mon[98694]: pgmap v7892: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 141 KiB/s wr, 26 op/s 2023-09-10T03:21:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:05 smithi098 ceph-mon[103124]: pgmap v7892: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 141 KiB/s wr, 26 op/s 2023-09-10T03:21:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:07 smithi170 ceph-mon[102832]: pgmap v7893: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 177 KiB/s wr, 31 op/s 2023-09-10T03:21:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:07 smithi082 ceph-mon[98694]: pgmap v7893: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 177 KiB/s wr, 31 op/s 2023-09-10T03:21:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:07 smithi098 ceph-mon[103124]: pgmap v7893: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 177 KiB/s wr, 31 op/s 2023-09-10T03:21:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14498.895675 secs 2023-09-10T03:21:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:08 smithi170 ceph-mon[102832]: osdmap e561: 12 total, 12 up, 12 in 2023-09-10T03:21:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14500.225997 secs 2023-09-10T03:21:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14498.895675 secs 2023-09-10T03:21:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:08 smithi082 ceph-mon[98694]: osdmap e561: 12 total, 12 up, 12 in 2023-09-10T03:21:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14500.225997 secs 2023-09-10T03:21:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14498.895675 secs 2023-09-10T03:21:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:08 smithi098 ceph-mon[103124]: osdmap e561: 12 total, 12 up, 12 in 2023-09-10T03:21:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14500.225997 secs 2023-09-10T03:21:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:09 smithi170 ceph-mon[102832]: pgmap v7895: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T03:21:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14500.937330 secs 2023-09-10T03:21:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:09 smithi082 ceph-mon[98694]: pgmap v7895: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T03:21:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14500.937330 secs 2023-09-10T03:21:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:09 smithi098 ceph-mon[103124]: pgmap v7895: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T03:21:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14500.937330 secs 2023-09-10T03:21:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:11 smithi170 ceph-mon[102832]: pgmap v7896: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T03:21:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:11 smithi082 ceph-mon[98694]: pgmap v7896: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T03:21:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:11 smithi098 ceph-mon[103124]: pgmap v7896: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 45 KiB/s wr, 11 op/s 2023-09-10T03:21:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14503.895737 secs 2023-09-10T03:21:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:13 smithi170 ceph-mon[102832]: osdmap e562: 12 total, 12 up, 12 in 2023-09-10T03:21:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:13 smithi170 ceph-mon[102832]: pgmap v7898: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 56 KiB/s wr, 13 op/s 2023-09-10T03:21:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14503.895737 secs 2023-09-10T03:21:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:13 smithi082 ceph-mon[98694]: osdmap e562: 12 total, 12 up, 12 in 2023-09-10T03:21:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:13 smithi082 ceph-mon[98694]: pgmap v7898: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 56 KiB/s wr, 13 op/s 2023-09-10T03:21:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14503.895737 secs 2023-09-10T03:21:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:13 smithi098 ceph-mon[103124]: osdmap e562: 12 total, 12 up, 12 in 2023-09-10T03:21:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:13 smithi098 ceph-mon[103124]: pgmap v7898: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 56 KiB/s wr, 13 op/s 2023-09-10T03:21:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14505.226085 secs 2023-09-10T03:21:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14505.937395 secs 2023-09-10T03:21:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14505.226085 secs 2023-09-10T03:21:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14505.937395 secs 2023-09-10T03:21:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14505.226085 secs 2023-09-10T03:21:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14505.937395 secs 2023-09-10T03:21:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:15 smithi170 ceph-mon[102832]: pgmap v7899: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2023-09-10T03:21:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:15 smithi082 ceph-mon[98694]: pgmap v7899: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2023-09-10T03:21:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:15 smithi098 ceph-mon[103124]: pgmap v7899: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2023-09-10T03:21:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:17 smithi170 ceph-mon[102832]: pgmap v7900: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2023-09-10T03:21:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:17 smithi082 ceph-mon[98694]: pgmap v7900: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2023-09-10T03:21:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:17 smithi098 ceph-mon[103124]: pgmap v7900: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2023-09-10T03:21:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14508.895786 secs 2023-09-10T03:21:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14510.226143 secs 2023-09-10T03:21:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14508.895786 secs 2023-09-10T03:21:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14510.226143 secs 2023-09-10T03:21:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14508.895786 secs 2023-09-10T03:21:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14510.226143 secs 2023-09-10T03:21:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:19 smithi170 ceph-mon[102832]: pgmap v7901: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2023-09-10T03:21:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14510.937459 secs 2023-09-10T03:21:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:19 smithi082 ceph-mon[98694]: pgmap v7901: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2023-09-10T03:21:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14510.937459 secs 2023-09-10T03:21:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:19 smithi098 ceph-mon[103124]: pgmap v7901: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2023-09-10T03:21:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14510.937459 secs 2023-09-10T03:21:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:21 smithi170 ceph-mon[102832]: pgmap v7902: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 11 op/s 2023-09-10T03:21:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:21 smithi082 ceph-mon[98694]: pgmap v7902: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 11 op/s 2023-09-10T03:21:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:21 smithi098 ceph-mon[103124]: pgmap v7902: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 11 op/s 2023-09-10T03:21:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:23 smithi170 ceph-mon[102832]: pgmap v7903: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 11 op/s 2023-09-10T03:21:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:21:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:23 smithi082 ceph-mon[98694]: pgmap v7903: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 11 op/s 2023-09-10T03:21:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:21:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:23 smithi098 ceph-mon[103124]: pgmap v7903: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 11 op/s 2023-09-10T03:21:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:21:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14515.226198 secs 2023-09-10T03:21:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:21:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:21:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:21:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14515.937487 secs 2023-09-10T03:21:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14515.226198 secs 2023-09-10T03:21:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:21:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:21:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:21:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14515.937487 secs 2023-09-10T03:21:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14515.226198 secs 2023-09-10T03:21:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:21:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:21:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:21:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14515.937487 secs 2023-09-10T03:21:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:25 smithi170 ceph-mon[102832]: pgmap v7904: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 9 op/s 2023-09-10T03:21:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:25 smithi082 ceph-mon[98694]: pgmap v7904: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 9 op/s 2023-09-10T03:21:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:25 smithi098 ceph-mon[103124]: pgmap v7904: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 9 op/s 2023-09-10T03:21:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:27 smithi170 ceph-mon[102832]: pgmap v7905: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 150 KiB/s wr, 15 op/s 2023-09-10T03:21:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:27 smithi082 ceph-mon[98694]: pgmap v7905: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 150 KiB/s wr, 15 op/s 2023-09-10T03:21:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:27 smithi098 ceph-mon[103124]: pgmap v7905: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 150 KiB/s wr, 15 op/s 2023-09-10T03:21:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14520.226283 secs 2023-09-10T03:21:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14520.226283 secs 2023-09-10T03:21:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14520.226283 secs 2023-09-10T03:21:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:29 smithi170 ceph-mon[102832]: pgmap v7906: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 149 KiB/s wr, 10 op/s 2023-09-10T03:21:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:29 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:21:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:29 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:21:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14520.937551 secs 2023-09-10T03:21:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:29 smithi082 ceph-mon[98694]: pgmap v7906: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 149 KiB/s wr, 10 op/s 2023-09-10T03:21:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:29 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:21:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:29 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:21:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14520.937551 secs 2023-09-10T03:21:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:29 smithi098 ceph-mon[103124]: pgmap v7906: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 149 KiB/s wr, 10 op/s 2023-09-10T03:21:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:29 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:21:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:29 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:21:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14520.937551 secs 2023-09-10T03:21:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:31 smithi170 ceph-mon[102832]: pgmap v7907: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 266 KiB/s wr, 17 op/s 2023-09-10T03:21:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:31 smithi082 ceph-mon[98694]: pgmap v7907: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 266 KiB/s wr, 17 op/s 2023-09-10T03:21:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:31 smithi098 ceph-mon[103124]: pgmap v7907: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 266 KiB/s wr, 17 op/s 2023-09-10T03:21:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:33 smithi170 ceph-mon[102832]: pgmap v7908: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 220 KiB/s wr, 13 op/s 2023-09-10T03:21:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:33 smithi082 ceph-mon[98694]: pgmap v7908: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 220 KiB/s wr, 13 op/s 2023-09-10T03:21:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:33 smithi098 ceph-mon[103124]: pgmap v7908: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 220 KiB/s wr, 13 op/s 2023-09-10T03:21:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14525.226339 secs 2023-09-10T03:21:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14525.937618 secs 2023-09-10T03:21:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14525.226339 secs 2023-09-10T03:21:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14525.937618 secs 2023-09-10T03:21:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14525.226339 secs 2023-09-10T03:21:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14525.937618 secs 2023-09-10T03:21:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:35 smithi170 ceph-mon[102832]: pgmap v7909: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 220 KiB/s wr, 13 op/s 2023-09-10T03:21:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:35 smithi082 ceph-mon[98694]: pgmap v7909: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 220 KiB/s wr, 13 op/s 2023-09-10T03:21:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:35 smithi098 ceph-mon[103124]: pgmap v7909: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 220 KiB/s wr, 13 op/s 2023-09-10T03:21:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:37 smithi170 ceph-mon[102832]: pgmap v7910: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 280 KiB/s wr, 18 op/s 2023-09-10T03:21:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:21:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:21:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:37 smithi082 ceph-mon[98694]: pgmap v7910: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 280 KiB/s wr, 18 op/s 2023-09-10T03:21:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:21:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:21:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:37 smithi098 ceph-mon[103124]: pgmap v7910: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 280 KiB/s wr, 18 op/s 2023-09-10T03:21:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:21:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:21:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14530.226396 secs 2023-09-10T03:21:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:39 smithi170 ceph-mon[102832]: pgmap v7911: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 177 KiB/s wr, 12 op/s 2023-09-10T03:21:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14530.937682 secs 2023-09-10T03:21:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14530.226396 secs 2023-09-10T03:21:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:39 smithi082 ceph-mon[98694]: pgmap v7911: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 177 KiB/s wr, 12 op/s 2023-09-10T03:21:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14530.937682 secs 2023-09-10T03:21:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14530.226396 secs 2023-09-10T03:21:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:39 smithi098 ceph-mon[103124]: pgmap v7911: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 177 KiB/s wr, 12 op/s 2023-09-10T03:21:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14530.937682 secs 2023-09-10T03:21:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:21:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:21:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:21:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:21:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:21:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:21:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:41 smithi170 ceph-mon[102832]: pgmap v7912: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 382 KiB/s wr, 18 op/s 2023-09-10T03:21:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:41 smithi082 ceph-mon[98694]: pgmap v7912: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 382 KiB/s wr, 18 op/s 2023-09-10T03:21:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:41 smithi098 ceph-mon[103124]: pgmap v7912: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 382 KiB/s wr, 18 op/s 2023-09-10T03:21:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:43 smithi170 ceph-mon[102832]: pgmap v7913: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T03:21:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:43 smithi082 ceph-mon[98694]: pgmap v7913: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T03:21:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:43 smithi098 ceph-mon[103124]: pgmap v7913: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T03:21:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14535.226453 secs 2023-09-10T03:21:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14535.937746 secs 2023-09-10T03:21:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14535.226453 secs 2023-09-10T03:21:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14535.937746 secs 2023-09-10T03:21:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14535.226453 secs 2023-09-10T03:21:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14535.937746 secs 2023-09-10T03:21:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:45 smithi170 ceph-mon[102832]: pgmap v7914: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T03:21:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:45 smithi082 ceph-mon[98694]: pgmap v7914: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T03:21:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:45 smithi098 ceph-mon[103124]: pgmap v7914: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T03:21:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:47 smithi170 ceph-mon[102832]: pgmap v7915: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 300 KiB/s wr, 17 op/s 2023-09-10T03:21:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:47 smithi082 ceph-mon[98694]: pgmap v7915: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 300 KiB/s wr, 17 op/s 2023-09-10T03:21:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:47 smithi098 ceph-mon[103124]: pgmap v7915: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 300 KiB/s wr, 17 op/s 2023-09-10T03:21:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14540.226492 secs 2023-09-10T03:21:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14540.226492 secs 2023-09-10T03:21:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14540.226492 secs 2023-09-10T03:21:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:49 smithi170 ceph-mon[102832]: pgmap v7916: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 240 KiB/s wr, 11 op/s 2023-09-10T03:21:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14540.937810 secs 2023-09-10T03:21:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:49 smithi082 ceph-mon[98694]: pgmap v7916: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 240 KiB/s wr, 11 op/s 2023-09-10T03:21:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14540.937810 secs 2023-09-10T03:21:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:49 smithi098 ceph-mon[103124]: pgmap v7916: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 240 KiB/s wr, 11 op/s 2023-09-10T03:21:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14540.937810 secs 2023-09-10T03:21:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:51 smithi170 ceph-mon[102832]: pgmap v7917: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 397 KiB/s wr, 17 op/s 2023-09-10T03:21:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:51 smithi082 ceph-mon[98694]: pgmap v7917: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 397 KiB/s wr, 17 op/s 2023-09-10T03:21:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:51 smithi098 ceph-mon[103124]: pgmap v7917: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 397 KiB/s wr, 17 op/s 2023-09-10T03:21:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:21:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:21:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:21:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:21:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:21:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:21:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:21:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:21:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:21:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:21:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:21:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:21:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:21:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:21:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:21:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:21:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:21:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:21:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14543.896347 secs 2023-09-10T03:21:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:53 smithi170 ceph-mon[102832]: pgmap v7918: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T03:21:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14545.226548 secs 2023-09-10T03:21:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14543.896347 secs 2023-09-10T03:21:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:53 smithi082 ceph-mon[98694]: pgmap v7918: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T03:21:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14545.226548 secs 2023-09-10T03:21:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14543.896347 secs 2023-09-10T03:21:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:53 smithi098 ceph-mon[103124]: pgmap v7918: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T03:21:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14545.226548 secs 2023-09-10T03:21:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14545.937878 secs 2023-09-10T03:21:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14545.937878 secs 2023-09-10T03:21:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14545.937878 secs 2023-09-10T03:21:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:55 smithi170 ceph-mon[102832]: pgmap v7919: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T03:21:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:55 smithi082 ceph-mon[98694]: pgmap v7919: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T03:21:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:55 smithi098 ceph-mon[103124]: pgmap v7919: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T03:21:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:21:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:21:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:21:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:21:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:21:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:21:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:21:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:21:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:21:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:57 smithi170 ceph-mon[102832]: pgmap v7920: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 289 KiB/s wr, 17 op/s 2023-09-10T03:21:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:57 smithi082 ceph-mon[98694]: pgmap v7920: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 289 KiB/s wr, 17 op/s 2023-09-10T03:21:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:57 smithi098 ceph-mon[103124]: pgmap v7920: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 289 KiB/s wr, 17 op/s 2023-09-10T03:21:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14548.896407 secs 2023-09-10T03:21:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14550.226606 secs 2023-09-10T03:21:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14548.896407 secs 2023-09-10T03:21:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14550.226606 secs 2023-09-10T03:21:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14548.896407 secs 2023-09-10T03:21:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14550.226606 secs 2023-09-10T03:21:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:59 smithi170 ceph-mon[102832]: pgmap v7921: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 254 KiB/s wr, 11 op/s 2023-09-10T03:21:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:21:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14550.937942 secs 2023-09-10T03:21:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:59 smithi082 ceph-mon[98694]: pgmap v7921: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 254 KiB/s wr, 11 op/s 2023-09-10T03:21:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:21:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14550.937942 secs 2023-09-10T03:22:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:59 smithi098 ceph-mon[103124]: pgmap v7921: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 254 KiB/s wr, 11 op/s 2023-09-10T03:22:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:21:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14550.937942 secs 2023-09-10T03:22:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:01 smithi170 ceph-mon[102832]: pgmap v7922: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 319 KiB/s wr, 26 op/s 2023-09-10T03:22:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:01 smithi082 ceph-mon[98694]: pgmap v7922: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 319 KiB/s wr, 26 op/s 2023-09-10T03:22:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:01 smithi098 ceph-mon[103124]: pgmap v7922: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 319 KiB/s wr, 26 op/s 2023-09-10T03:22:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14553.896467 secs 2023-09-10T03:22:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:03 smithi170 ceph-mon[102832]: pgmap v7923: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 163 KiB/s wr, 20 op/s 2023-09-10T03:22:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14553.896467 secs 2023-09-10T03:22:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:03 smithi082 ceph-mon[98694]: pgmap v7923: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 163 KiB/s wr, 20 op/s 2023-09-10T03:22:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14553.896467 secs 2023-09-10T03:22:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:03 smithi098 ceph-mon[103124]: pgmap v7923: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 163 KiB/s wr, 20 op/s 2023-09-10T03:22:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14555.226662 secs 2023-09-10T03:22:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:22:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:22:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14555.938007 secs 2023-09-10T03:22:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14555.226662 secs 2023-09-10T03:22:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:22:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:22:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14555.938007 secs 2023-09-10T03:22:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14555.226662 secs 2023-09-10T03:22:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:22:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:22:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14555.938007 secs 2023-09-10T03:22:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:05 smithi170 ceph-mon[102832]: pgmap v7924: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 163 KiB/s wr, 20 op/s 2023-09-10T03:22:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:22:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:22:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:05 smithi082 ceph-mon[98694]: pgmap v7924: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 163 KiB/s wr, 20 op/s 2023-09-10T03:22:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:22:05.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:22:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:05 smithi098 ceph-mon[103124]: pgmap v7924: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 163 KiB/s wr, 20 op/s 2023-09-10T03:22:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:22:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:22:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:07 smithi170 ceph-mon[102832]: pgmap v7925: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 197 KiB/s wr, 31 op/s 2023-09-10T03:22:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:07 smithi082 ceph-mon[98694]: pgmap v7925: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 197 KiB/s wr, 31 op/s 2023-09-10T03:22:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:07 smithi098 ceph-mon[103124]: pgmap v7925: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 197 KiB/s wr, 31 op/s 2023-09-10T03:22:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14558.896528 secs 2023-09-10T03:22:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:08 smithi170 ceph-mon[102832]: osdmap e563: 12 total, 12 up, 12 in 2023-09-10T03:22:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14558.896528 secs 2023-09-10T03:22:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:08 smithi082 ceph-mon[98694]: osdmap e563: 12 total, 12 up, 12 in 2023-09-10T03:22:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14558.896528 secs 2023-09-10T03:22:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:08 smithi098 ceph-mon[103124]: osdmap e563: 12 total, 12 up, 12 in 2023-09-10T03:22:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14560.226725 secs 2023-09-10T03:22:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:09 smithi170 ceph-mon[102832]: pgmap v7927: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 119 KiB/s wr, 30 op/s 2023-09-10T03:22:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14560.938071 secs 2023-09-10T03:22:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14560.226725 secs 2023-09-10T03:22:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:09 smithi082 ceph-mon[98694]: pgmap v7927: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 119 KiB/s wr, 30 op/s 2023-09-10T03:22:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14560.938071 secs 2023-09-10T03:22:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14560.226725 secs 2023-09-10T03:22:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:09 smithi098 ceph-mon[103124]: pgmap v7927: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 119 KiB/s wr, 30 op/s 2023-09-10T03:22:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14560.938071 secs 2023-09-10T03:22:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:11 smithi170 ceph-mon[102832]: pgmap v7928: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 41 KiB/s wr, 18 op/s 2023-09-10T03:22:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:11 smithi082 ceph-mon[98694]: pgmap v7928: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 41 KiB/s wr, 18 op/s 2023-09-10T03:22:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:11 smithi098 ceph-mon[103124]: pgmap v7928: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 41 KiB/s wr, 18 op/s 2023-09-10T03:22:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14563.896611 secs 2023-09-10T03:22:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:13 smithi170 ceph-mon[102832]: osdmap e564: 12 total, 12 up, 12 in 2023-09-10T03:22:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:13 smithi170 ceph-mon[102832]: pgmap v7930: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 52 KiB/s wr, 22 op/s 2023-09-10T03:22:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14563.896611 secs 2023-09-10T03:22:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:13 smithi082 ceph-mon[98694]: osdmap e564: 12 total, 12 up, 12 in 2023-09-10T03:22:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:13 smithi082 ceph-mon[98694]: pgmap v7930: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 52 KiB/s wr, 22 op/s 2023-09-10T03:22:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14563.896611 secs 2023-09-10T03:22:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:13 smithi098 ceph-mon[103124]: osdmap e564: 12 total, 12 up, 12 in 2023-09-10T03:22:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:13 smithi098 ceph-mon[103124]: pgmap v7930: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 52 KiB/s wr, 22 op/s 2023-09-10T03:22:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14565.226826 secs 2023-09-10T03:22:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14565.938137 secs 2023-09-10T03:22:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14565.226826 secs 2023-09-10T03:22:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14565.938137 secs 2023-09-10T03:22:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14565.226826 secs 2023-09-10T03:22:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14565.938137 secs 2023-09-10T03:22:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:15 smithi170 ceph-mon[102832]: pgmap v7931: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:22:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:15 smithi082 ceph-mon[98694]: pgmap v7931: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:22:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:15 smithi098 ceph-mon[103124]: pgmap v7931: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:22:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:17 smithi170 ceph-mon[102832]: pgmap v7932: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 63 KiB/s wr, 13 op/s 2023-09-10T03:22:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:17 smithi082 ceph-mon[98694]: pgmap v7932: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 63 KiB/s wr, 13 op/s 2023-09-10T03:22:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:17 smithi098 ceph-mon[103124]: pgmap v7932: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 63 KiB/s wr, 13 op/s 2023-09-10T03:22:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14568.896764 secs 2023-09-10T03:22:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14570.226895 secs 2023-09-10T03:22:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14568.896764 secs 2023-09-10T03:22:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14570.226895 secs 2023-09-10T03:22:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14568.896764 secs 2023-09-10T03:22:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14570.226895 secs 2023-09-10T03:22:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:19 smithi170 ceph-mon[102832]: pgmap v7933: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T03:22:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14570.938202 secs 2023-09-10T03:22:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:19 smithi082 ceph-mon[98694]: pgmap v7933: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T03:22:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14570.938202 secs 2023-09-10T03:22:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:19 smithi098 ceph-mon[103124]: pgmap v7933: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T03:22:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14570.938202 secs 2023-09-10T03:22:21.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:21 smithi170 ceph-mon[102832]: pgmap v7934: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 KiB/s wr, 11 op/s 2023-09-10T03:22:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:21 smithi082 ceph-mon[98694]: pgmap v7934: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 KiB/s wr, 11 op/s 2023-09-10T03:22:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:21 smithi098 ceph-mon[103124]: pgmap v7934: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 KiB/s wr, 11 op/s 2023-09-10T03:22:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:23 smithi170 ceph-mon[102832]: pgmap v7935: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 KiB/s wr, 11 op/s 2023-09-10T03:22:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:22:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:23 smithi082 ceph-mon[98694]: pgmap v7935: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 KiB/s wr, 11 op/s 2023-09-10T03:22:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:22:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:23 smithi098 ceph-mon[103124]: pgmap v7935: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 KiB/s wr, 11 op/s 2023-09-10T03:22:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:22:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14575.226951 secs 2023-09-10T03:22:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:22:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:22:24.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:22:24.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14575.938270 secs 2023-09-10T03:22:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14575.226951 secs 2023-09-10T03:22:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:22:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:22:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:22:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14575.938270 secs 2023-09-10T03:22:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14575.226951 secs 2023-09-10T03:22:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:22:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:22:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:22:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14575.938270 secs 2023-09-10T03:22:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:25 smithi170 ceph-mon[102832]: pgmap v7936: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 9 op/s 2023-09-10T03:22:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:25 smithi082 ceph-mon[98694]: pgmap v7936: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 9 op/s 2023-09-10T03:22:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:25 smithi098 ceph-mon[103124]: pgmap v7936: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 9 op/s 2023-09-10T03:22:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:27 smithi170 ceph-mon[102832]: pgmap v7937: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 118 KiB/s wr, 14 op/s 2023-09-10T03:22:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:27 smithi082 ceph-mon[98694]: pgmap v7937: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 118 KiB/s wr, 14 op/s 2023-09-10T03:22:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:27 smithi098 ceph-mon[103124]: pgmap v7937: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 118 KiB/s wr, 14 op/s 2023-09-10T03:22:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14580.227027 secs 2023-09-10T03:22:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14580.227027 secs 2023-09-10T03:22:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14580.227027 secs 2023-09-10T03:22:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:29 smithi170 ceph-mon[102832]: pgmap v7938: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T03:22:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14580.938333 secs 2023-09-10T03:22:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:29 smithi082 ceph-mon[98694]: pgmap v7938: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T03:22:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14580.938333 secs 2023-09-10T03:22:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:29 smithi098 ceph-mon[103124]: pgmap v7938: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T03:22:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14580.938333 secs 2023-09-10T03:22:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:31 smithi170 ceph-mon[102832]: pgmap v7939: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 167 KiB/s wr, 14 op/s 2023-09-10T03:22:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:31 smithi082 ceph-mon[98694]: pgmap v7939: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 167 KiB/s wr, 14 op/s 2023-09-10T03:22:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:31 smithi098 ceph-mon[103124]: pgmap v7939: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 167 KiB/s wr, 14 op/s 2023-09-10T03:22:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:33 smithi170 ceph-mon[102832]: pgmap v7940: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 167 KiB/s wr, 10 op/s 2023-09-10T03:22:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:33 smithi082 ceph-mon[98694]: pgmap v7940: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 167 KiB/s wr, 10 op/s 2023-09-10T03:22:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:33 smithi098 ceph-mon[103124]: pgmap v7940: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 167 KiB/s wr, 10 op/s 2023-09-10T03:22:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14585.227082 secs 2023-09-10T03:22:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14585.938396 secs 2023-09-10T03:22:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14585.227082 secs 2023-09-10T03:22:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14585.938396 secs 2023-09-10T03:22:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14585.227082 secs 2023-09-10T03:22:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14585.938396 secs 2023-09-10T03:22:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:35 smithi170 ceph-mon[102832]: pgmap v7941: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 167 KiB/s wr, 10 op/s 2023-09-10T03:22:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:35 smithi082 ceph-mon[98694]: pgmap v7941: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 167 KiB/s wr, 10 op/s 2023-09-10T03:22:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:35 smithi098 ceph-mon[103124]: pgmap v7941: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 167 KiB/s wr, 10 op/s 2023-09-10T03:22:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:37 smithi170 ceph-mon[102832]: pgmap v7942: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 239 KiB/s wr, 16 op/s 2023-09-10T03:22:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:22:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:22:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:37 smithi082 ceph-mon[98694]: pgmap v7942: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 239 KiB/s wr, 16 op/s 2023-09-10T03:22:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:22:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:22:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:37 smithi098 ceph-mon[103124]: pgmap v7942: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 239 KiB/s wr, 16 op/s 2023-09-10T03:22:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:22:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:22:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14590.227181 secs 2023-09-10T03:22:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:39 smithi170 ceph-mon[102832]: pgmap v7943: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 168 KiB/s wr, 11 op/s 2023-09-10T03:22:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14590.938463 secs 2023-09-10T03:22:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14590.227181 secs 2023-09-10T03:22:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:39 smithi082 ceph-mon[98694]: pgmap v7943: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 168 KiB/s wr, 11 op/s 2023-09-10T03:22:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14590.938463 secs 2023-09-10T03:22:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14590.227181 secs 2023-09-10T03:22:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:39 smithi098 ceph-mon[103124]: pgmap v7943: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 168 KiB/s wr, 11 op/s 2023-09-10T03:22:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14590.938463 secs 2023-09-10T03:22:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:22:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:22:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:22:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:22:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:22:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:22:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:41 smithi170 ceph-mon[102832]: pgmap v7944: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 320 KiB/s wr, 17 op/s 2023-09-10T03:22:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:41 smithi082 ceph-mon[98694]: pgmap v7944: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 320 KiB/s wr, 17 op/s 2023-09-10T03:22:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:41 smithi098 ceph-mon[103124]: pgmap v7944: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 320 KiB/s wr, 17 op/s 2023-09-10T03:22:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:43 smithi170 ceph-mon[102832]: pgmap v7945: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 223 KiB/s wr, 12 op/s 2023-09-10T03:22:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:43 smithi082 ceph-mon[98694]: pgmap v7945: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 223 KiB/s wr, 12 op/s 2023-09-10T03:22:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:43 smithi098 ceph-mon[103124]: pgmap v7945: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 223 KiB/s wr, 12 op/s 2023-09-10T03:22:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14595.227236 secs 2023-09-10T03:22:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14595.938485 secs 2023-09-10T03:22:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14595.227236 secs 2023-09-10T03:22:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14595.938485 secs 2023-09-10T03:22:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14595.227236 secs 2023-09-10T03:22:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14595.938485 secs 2023-09-10T03:22:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:45 smithi170 ceph-mon[102832]: pgmap v7946: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 223 KiB/s wr, 12 op/s 2023-09-10T03:22:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:45 smithi082 ceph-mon[98694]: pgmap v7946: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 223 KiB/s wr, 12 op/s 2023-09-10T03:22:46.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:45 smithi098 ceph-mon[103124]: pgmap v7946: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 223 KiB/s wr, 12 op/s 2023-09-10T03:22:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:47 smithi170 ceph-mon[102832]: pgmap v7947: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 246 KiB/s wr, 18 op/s 2023-09-10T03:22:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:47 smithi082 ceph-mon[98694]: pgmap v7947: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 246 KiB/s wr, 18 op/s 2023-09-10T03:22:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:47 smithi098 ceph-mon[103124]: pgmap v7947: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 246 KiB/s wr, 18 op/s 2023-09-10T03:22:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14600.227294 secs 2023-09-10T03:22:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:49 smithi170 ceph-mon[102832]: pgmap v7948: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 12 op/s 2023-09-10T03:22:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14600.938550 secs 2023-09-10T03:22:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14600.227294 secs 2023-09-10T03:22:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:49 smithi082 ceph-mon[98694]: pgmap v7948: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 12 op/s 2023-09-10T03:22:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14600.938550 secs 2023-09-10T03:22:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14600.227294 secs 2023-09-10T03:22:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:49 smithi098 ceph-mon[103124]: pgmap v7948: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 12 op/s 2023-09-10T03:22:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14600.938550 secs 2023-09-10T03:22:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:51 smithi170 ceph-mon[102832]: pgmap v7949: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 360 KiB/s wr, 18 op/s 2023-09-10T03:22:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:51 smithi082 ceph-mon[98694]: pgmap v7949: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 360 KiB/s wr, 18 op/s 2023-09-10T03:22:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:51 smithi098 ceph-mon[103124]: pgmap v7949: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 360 KiB/s wr, 18 op/s 2023-09-10T03:22:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14603.897249 secs 2023-09-10T03:22:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:53 smithi170 ceph-mon[102832]: pgmap v7950: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 208 KiB/s wr, 11 op/s 2023-09-10T03:22:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:22:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:22:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:22:53.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:22:53.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:22:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14603.897249 secs 2023-09-10T03:22:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:53 smithi082 ceph-mon[98694]: pgmap v7950: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 208 KiB/s wr, 11 op/s 2023-09-10T03:22:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:22:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:22:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:22:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:22:53.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:22:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14603.897249 secs 2023-09-10T03:22:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:53 smithi098 ceph-mon[103124]: pgmap v7950: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 208 KiB/s wr, 11 op/s 2023-09-10T03:22:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:22:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:22:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:22:54.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:22:54.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:22:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14605.227351 secs 2023-09-10T03:22:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14605.938618 secs 2023-09-10T03:22:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:22:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:22:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:22:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14605.227351 secs 2023-09-10T03:22:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14605.938618 secs 2023-09-10T03:22:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:22:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:22:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:22:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14605.227351 secs 2023-09-10T03:22:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14605.938618 secs 2023-09-10T03:22:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:22:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:22:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:22:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:55 smithi170 ceph-mon[102832]: pgmap v7951: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 208 KiB/s wr, 11 op/s 2023-09-10T03:22:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:55 smithi082 ceph-mon[98694]: pgmap v7951: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 208 KiB/s wr, 11 op/s 2023-09-10T03:22:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:55 smithi098 ceph-mon[103124]: pgmap v7951: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 208 KiB/s wr, 11 op/s 2023-09-10T03:22:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:22:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:22:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:22:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:22:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:22:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:22:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:22:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:22:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:22:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:57 smithi170 ceph-mon[102832]: pgmap v7952: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 253 KiB/s wr, 17 op/s 2023-09-10T03:22:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:22:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:22:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:57 smithi082 ceph-mon[98694]: pgmap v7952: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 253 KiB/s wr, 17 op/s 2023-09-10T03:22:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:22:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:22:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:57 smithi098 ceph-mon[103124]: pgmap v7952: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 253 KiB/s wr, 17 op/s 2023-09-10T03:22:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:22:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:22:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14608.897322 secs 2023-09-10T03:22:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14608.897322 secs 2023-09-10T03:22:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14608.897322 secs 2023-09-10T03:22:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14610.227425 secs 2023-09-10T03:22:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:59 smithi170 ceph-mon[102832]: pgmap v7953: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T03:22:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:22:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14610.938686 secs 2023-09-10T03:22:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14610.227425 secs 2023-09-10T03:22:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:59 smithi082 ceph-mon[98694]: pgmap v7953: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T03:22:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:22:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14610.938686 secs 2023-09-10T03:23:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14610.227425 secs 2023-09-10T03:23:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:59 smithi098 ceph-mon[103124]: pgmap v7953: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T03:23:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:22:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14610.938686 secs 2023-09-10T03:23:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:23:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:23:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:23:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:23:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:23:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:23:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:01 smithi170 ceph-mon[102832]: pgmap v7954: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 339 KiB/s wr, 17 op/s 2023-09-10T03:23:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:01 smithi082 ceph-mon[98694]: pgmap v7954: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 339 KiB/s wr, 17 op/s 2023-09-10T03:23:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:01 smithi098 ceph-mon[103124]: pgmap v7954: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 339 KiB/s wr, 17 op/s 2023-09-10T03:23:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14613.897454 secs 2023-09-10T03:23:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:03 smithi170 ceph-mon[102832]: pgmap v7955: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T03:23:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14613.897454 secs 2023-09-10T03:23:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:03 smithi082 ceph-mon[98694]: pgmap v7955: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T03:23:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14613.897454 secs 2023-09-10T03:23:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:03 smithi098 ceph-mon[103124]: pgmap v7955: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T03:23:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14615.227479 secs 2023-09-10T03:23:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14615.938752 secs 2023-09-10T03:23:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14615.227479 secs 2023-09-10T03:23:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14615.938752 secs 2023-09-10T03:23:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14615.227479 secs 2023-09-10T03:23:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14615.938752 secs 2023-09-10T03:23:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:05 smithi170 ceph-mon[102832]: pgmap v7956: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T03:23:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:05 smithi082 ceph-mon[98694]: pgmap v7956: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T03:23:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:05 smithi098 ceph-mon[103124]: pgmap v7956: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T03:23:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:07 smithi170 ceph-mon[102832]: pgmap v7957: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 199 KiB/s wr, 31 op/s 2023-09-10T03:23:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:07 smithi082 ceph-mon[98694]: pgmap v7957: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 199 KiB/s wr, 31 op/s 2023-09-10T03:23:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:07 smithi098 ceph-mon[103124]: pgmap v7957: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 199 KiB/s wr, 31 op/s 2023-09-10T03:23:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14618.897513 secs 2023-09-10T03:23:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:08 smithi170 ceph-mon[102832]: osdmap e565: 12 total, 12 up, 12 in 2023-09-10T03:23:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14620.227541 secs 2023-09-10T03:23:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14618.897513 secs 2023-09-10T03:23:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:08 smithi082 ceph-mon[98694]: osdmap e565: 12 total, 12 up, 12 in 2023-09-10T03:23:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14620.227541 secs 2023-09-10T03:23:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14618.897513 secs 2023-09-10T03:23:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:08 smithi098 ceph-mon[103124]: osdmap e565: 12 total, 12 up, 12 in 2023-09-10T03:23:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14620.227541 secs 2023-09-10T03:23:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:09 smithi170 ceph-mon[102832]: pgmap v7959: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 186 KiB/s wr, 31 op/s 2023-09-10T03:23:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14620.938816 secs 2023-09-10T03:23:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:09 smithi082 ceph-mon[98694]: pgmap v7959: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 186 KiB/s wr, 31 op/s 2023-09-10T03:23:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14620.938816 secs 2023-09-10T03:23:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:09 smithi098 ceph-mon[103124]: pgmap v7959: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 186 KiB/s wr, 31 op/s 2023-09-10T03:23:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14620.938816 secs 2023-09-10T03:23:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:11 smithi170 ceph-mon[102832]: pgmap v7960: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T03:23:11.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:11 smithi082 ceph-mon[98694]: pgmap v7960: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T03:23:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:11 smithi098 ceph-mon[103124]: pgmap v7960: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 100 KiB/s wr, 29 op/s 2023-09-10T03:23:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14623.897571 secs 2023-09-10T03:23:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:13 smithi170 ceph-mon[102832]: osdmap e566: 12 total, 12 up, 12 in 2023-09-10T03:23:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:13 smithi170 ceph-mon[102832]: pgmap v7962: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 124 KiB/s wr, 37 op/s 2023-09-10T03:23:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14623.897571 secs 2023-09-10T03:23:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:13 smithi082 ceph-mon[98694]: osdmap e566: 12 total, 12 up, 12 in 2023-09-10T03:23:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:13 smithi082 ceph-mon[98694]: pgmap v7962: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 124 KiB/s wr, 37 op/s 2023-09-10T03:23:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14623.897571 secs 2023-09-10T03:23:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:13 smithi098 ceph-mon[103124]: osdmap e566: 12 total, 12 up, 12 in 2023-09-10T03:23:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:13 smithi098 ceph-mon[103124]: pgmap v7962: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 124 KiB/s wr, 37 op/s 2023-09-10T03:23:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14625.227623 secs 2023-09-10T03:23:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14625.938882 secs 2023-09-10T03:23:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14625.227623 secs 2023-09-10T03:23:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14625.938882 secs 2023-09-10T03:23:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14625.227623 secs 2023-09-10T03:23:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14625.938882 secs 2023-09-10T03:23:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:15 smithi170 ceph-mon[102832]: pgmap v7963: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 7 op/s 2023-09-10T03:23:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:15 smithi082 ceph-mon[98694]: pgmap v7963: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 7 op/s 2023-09-10T03:23:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:15 smithi098 ceph-mon[103124]: pgmap v7963: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 7 op/s 2023-09-10T03:23:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:17 smithi170 ceph-mon[102832]: pgmap v7964: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 14 op/s 2023-09-10T03:23:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:17 smithi082 ceph-mon[98694]: pgmap v7964: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 14 op/s 2023-09-10T03:23:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:17 smithi098 ceph-mon[103124]: pgmap v7964: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 14 op/s 2023-09-10T03:23:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14628.897630 secs 2023-09-10T03:23:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14630.227683 secs 2023-09-10T03:23:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14628.897630 secs 2023-09-10T03:23:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14630.227683 secs 2023-09-10T03:23:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14628.897630 secs 2023-09-10T03:23:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14630.227683 secs 2023-09-10T03:23:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:19 smithi170 ceph-mon[102832]: pgmap v7965: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T03:23:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14630.938949 secs 2023-09-10T03:23:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:19 smithi082 ceph-mon[98694]: pgmap v7965: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T03:23:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14630.938949 secs 2023-09-10T03:23:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:19 smithi098 ceph-mon[103124]: pgmap v7965: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T03:23:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14630.938949 secs 2023-09-10T03:23:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:21 smithi170 ceph-mon[102832]: pgmap v7966: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T03:23:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:21 smithi082 ceph-mon[98694]: pgmap v7966: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T03:23:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:21 smithi098 ceph-mon[103124]: pgmap v7966: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T03:23:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:23 smithi170 ceph-mon[102832]: pgmap v7967: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T03:23:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:23:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:23 smithi082 ceph-mon[98694]: pgmap v7967: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T03:23:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:23:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:23 smithi098 ceph-mon[103124]: pgmap v7967: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T03:23:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:23:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14635.227742 secs 2023-09-10T03:23:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:23:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:23:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:23:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14635.939016 secs 2023-09-10T03:23:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14635.227742 secs 2023-09-10T03:23:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:23:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:23:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:23:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14635.939016 secs 2023-09-10T03:23:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14635.227742 secs 2023-09-10T03:23:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:23:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:23:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:23:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14635.939016 secs 2023-09-10T03:23:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:25 smithi170 ceph-mon[102832]: pgmap v7968: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 9 op/s 2023-09-10T03:23:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:25 smithi082 ceph-mon[98694]: pgmap v7968: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 9 op/s 2023-09-10T03:23:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:25 smithi098 ceph-mon[103124]: pgmap v7968: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 9 op/s 2023-09-10T03:23:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:27 smithi170 ceph-mon[102832]: pgmap v7969: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 103 KiB/s wr, 14 op/s 2023-09-10T03:23:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:27 smithi082 ceph-mon[98694]: pgmap v7969: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 103 KiB/s wr, 14 op/s 2023-09-10T03:23:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:27 smithi098 ceph-mon[103124]: pgmap v7969: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 103 KiB/s wr, 14 op/s 2023-09-10T03:23:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14640.227801 secs 2023-09-10T03:23:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14640.227801 secs 2023-09-10T03:23:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14640.227801 secs 2023-09-10T03:23:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:29 smithi170 ceph-mon[102832]: pgmap v7970: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 9 op/s 2023-09-10T03:23:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14640.939105 secs 2023-09-10T03:23:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:29 smithi082 ceph-mon[98694]: pgmap v7970: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 9 op/s 2023-09-10T03:23:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14640.939105 secs 2023-09-10T03:23:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:29 smithi098 ceph-mon[103124]: pgmap v7970: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 9 op/s 2023-09-10T03:23:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14640.939105 secs 2023-09-10T03:23:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:31 smithi170 ceph-mon[102832]: pgmap v7971: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 223 KiB/s wr, 16 op/s 2023-09-10T03:23:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:31 smithi082 ceph-mon[98694]: pgmap v7971: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 223 KiB/s wr, 16 op/s 2023-09-10T03:23:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:31 smithi098 ceph-mon[103124]: pgmap v7971: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 223 KiB/s wr, 16 op/s 2023-09-10T03:23:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:33 smithi170 ceph-mon[102832]: pgmap v7972: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 223 KiB/s wr, 12 op/s 2023-09-10T03:23:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:23:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:23:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14645.227871 secs 2023-09-10T03:23:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:33 smithi082 ceph-mon[98694]: pgmap v7972: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 223 KiB/s wr, 12 op/s 2023-09-10T03:23:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:23:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:23:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14645.227871 secs 2023-09-10T03:23:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:33 smithi098 ceph-mon[103124]: pgmap v7972: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 223 KiB/s wr, 12 op/s 2023-09-10T03:23:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:23:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:23:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14645.227871 secs 2023-09-10T03:23:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14645.939174 secs 2023-09-10T03:23:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14645.939174 secs 2023-09-10T03:23:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14645.939174 secs 2023-09-10T03:23:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:35 smithi170 ceph-mon[102832]: pgmap v7973: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 223 KiB/s wr, 12 op/s 2023-09-10T03:23:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:35 smithi082 ceph-mon[98694]: pgmap v7973: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 223 KiB/s wr, 12 op/s 2023-09-10T03:23:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:35 smithi098 ceph-mon[103124]: pgmap v7973: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 223 KiB/s wr, 12 op/s 2023-09-10T03:23:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:23:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:23:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:23:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:23:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:23:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:23:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:37 smithi082 ceph-mon[98694]: pgmap v7974: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 241 KiB/s wr, 17 op/s 2023-09-10T03:23:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:37 smithi098 ceph-mon[103124]: pgmap v7974: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 241 KiB/s wr, 17 op/s 2023-09-10T03:23:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:37 smithi170 ceph-mon[102832]: pgmap v7974: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 241 KiB/s wr, 17 op/s 2023-09-10T03:23:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14650.227928 secs 2023-09-10T03:23:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14650.227928 secs 2023-09-10T03:23:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14650.227928 secs 2023-09-10T03:23:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:39 smithi082 ceph-mon[98694]: pgmap v7975: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 185 KiB/s wr, 12 op/s 2023-09-10T03:23:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14650.939240 secs 2023-09-10T03:23:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:39 smithi098 ceph-mon[103124]: pgmap v7975: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 185 KiB/s wr, 12 op/s 2023-09-10T03:23:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14650.939240 secs 2023-09-10T03:23:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:39 smithi170 ceph-mon[102832]: pgmap v7975: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 185 KiB/s wr, 12 op/s 2023-09-10T03:23:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14650.939240 secs 2023-09-10T03:23:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:41 smithi082 ceph-mon[98694]: pgmap v7976: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 436 KiB/s wr, 19 op/s 2023-09-10T03:23:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:41 smithi098 ceph-mon[103124]: pgmap v7976: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 436 KiB/s wr, 19 op/s 2023-09-10T03:23:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:41 smithi170 ceph-mon[102832]: pgmap v7976: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 436 KiB/s wr, 19 op/s 2023-09-10T03:23:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:43 smithi170 ceph-mon[102832]: pgmap v7977: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T03:23:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:43 smithi082 ceph-mon[98694]: pgmap v7977: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T03:23:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:43 smithi098 ceph-mon[103124]: pgmap v7977: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T03:23:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14655.228009 secs 2023-09-10T03:23:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14655.939305 secs 2023-09-10T03:23:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14655.228009 secs 2023-09-10T03:23:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14655.939305 secs 2023-09-10T03:23:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14655.228009 secs 2023-09-10T03:23:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14655.939305 secs 2023-09-10T03:23:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:45 smithi170 ceph-mon[102832]: pgmap v7978: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T03:23:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:45 smithi082 ceph-mon[98694]: pgmap v7978: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T03:23:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:45 smithi098 ceph-mon[103124]: pgmap v7978: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T03:23:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:47 smithi170 ceph-mon[102832]: pgmap v7979: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 414 KiB/s wr, 18 op/s 2023-09-10T03:23:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:47 smithi082 ceph-mon[98694]: pgmap v7979: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 414 KiB/s wr, 18 op/s 2023-09-10T03:23:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:47 smithi098 ceph-mon[103124]: pgmap v7979: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 414 KiB/s wr, 18 op/s 2023-09-10T03:23:48.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14660.228092 secs 2023-09-10T03:23:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14660.228092 secs 2023-09-10T03:23:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14660.228092 secs 2023-09-10T03:23:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:49 smithi170 ceph-mon[102832]: pgmap v7980: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 396 KiB/s wr, 12 op/s 2023-09-10T03:23:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:23:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:23:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14660.939368 secs 2023-09-10T03:23:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:49 smithi082 ceph-mon[98694]: pgmap v7980: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 396 KiB/s wr, 12 op/s 2023-09-10T03:23:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:23:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:23:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14660.939368 secs 2023-09-10T03:23:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:49 smithi098 ceph-mon[103124]: pgmap v7980: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 396 KiB/s wr, 12 op/s 2023-09-10T03:23:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:23:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:23:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14660.939368 secs 2023-09-10T03:23:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:51 smithi170 ceph-mon[102832]: pgmap v7981: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 444 KiB/s wr, 18 op/s 2023-09-10T03:23:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:51 smithi082 ceph-mon[98694]: pgmap v7981: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 444 KiB/s wr, 18 op/s 2023-09-10T03:23:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:51 smithi098 ceph-mon[103124]: pgmap v7981: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 444 KiB/s wr, 18 op/s 2023-09-10T03:23:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:23:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:23:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:23:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:23:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:23:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:23:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14663.898108 secs 2023-09-10T03:23:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:53 smithi082 ceph-mon[98694]: pgmap v7982: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T03:23:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14665.228151 secs 2023-09-10T03:23:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14663.898108 secs 2023-09-10T03:23:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:53 smithi098 ceph-mon[103124]: pgmap v7982: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T03:23:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14665.228151 secs 2023-09-10T03:23:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14663.898108 secs 2023-09-10T03:23:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:53 smithi170 ceph-mon[102832]: pgmap v7982: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T03:23:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14665.228151 secs 2023-09-10T03:23:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14665.939438 secs 2023-09-10T03:23:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:23:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:23:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:23:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:23:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14665.939438 secs 2023-09-10T03:23:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:23:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:23:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:23:55.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:23:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14665.939438 secs 2023-09-10T03:23:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:23:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:23:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:23:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:23:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:55 smithi082 ceph-mon[98694]: pgmap v7983: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T03:23:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:55 smithi098 ceph-mon[103124]: pgmap v7983: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T03:23:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:55 smithi170 ceph-mon[102832]: pgmap v7983: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T03:23:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:23:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:23:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:23:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:23:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:23:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:23:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:23:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:23:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:23:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:57 smithi082 ceph-mon[98694]: pgmap v7984: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 292 KiB/s wr, 17 op/s 2023-09-10T03:23:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:57 smithi098 ceph-mon[103124]: pgmap v7984: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 292 KiB/s wr, 17 op/s 2023-09-10T03:23:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:57 smithi170 ceph-mon[102832]: pgmap v7984: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 292 KiB/s wr, 17 op/s 2023-09-10T03:23:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14668.898204 secs 2023-09-10T03:23:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14670.228210 secs 2023-09-10T03:23:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14668.898204 secs 2023-09-10T03:23:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14670.228210 secs 2023-09-10T03:23:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14668.898204 secs 2023-09-10T03:23:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14670.228210 secs 2023-09-10T03:23:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:59 smithi082 ceph-mon[98694]: pgmap v7985: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T03:23:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:23:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14670.939490 secs 2023-09-10T03:24:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:59 smithi098 ceph-mon[103124]: pgmap v7985: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T03:24:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:23:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14670.939490 secs 2023-09-10T03:24:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:59 smithi170 ceph-mon[102832]: pgmap v7985: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T03:24:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:23:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14670.939490 secs 2023-09-10T03:24:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:01 smithi082 ceph-mon[98694]: pgmap v7986: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 190 KiB/s wr, 16 op/s 2023-09-10T03:24:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:01 smithi098 ceph-mon[103124]: pgmap v7986: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 190 KiB/s wr, 16 op/s 2023-09-10T03:24:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:01 smithi170 ceph-mon[102832]: pgmap v7986: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 190 KiB/s wr, 16 op/s 2023-09-10T03:24:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14673.898266 secs 2023-09-10T03:24:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:03 smithi170 ceph-mon[102832]: pgmap v7987: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 10 op/s 2023-09-10T03:24:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14673.898266 secs 2023-09-10T03:24:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:03 smithi082 ceph-mon[98694]: pgmap v7987: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 10 op/s 2023-09-10T03:24:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14673.898266 secs 2023-09-10T03:24:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:03 smithi098 ceph-mon[103124]: pgmap v7987: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 10 op/s 2023-09-10T03:24:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14675.228271 secs 2023-09-10T03:24:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14675.939558 secs 2023-09-10T03:24:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14675.228271 secs 2023-09-10T03:24:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14675.939558 secs 2023-09-10T03:24:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14675.228271 secs 2023-09-10T03:24:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14675.939558 secs 2023-09-10T03:24:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:05 smithi170 ceph-mon[102832]: pgmap v7988: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 10 op/s 2023-09-10T03:24:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:05 smithi082 ceph-mon[98694]: pgmap v7988: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 10 op/s 2023-09-10T03:24:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:05 smithi098 ceph-mon[103124]: pgmap v7988: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 10 op/s 2023-09-10T03:24:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:07 smithi170 ceph-mon[102832]: pgmap v7989: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 199 KiB/s wr, 30 op/s 2023-09-10T03:24:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:07 smithi082 ceph-mon[98694]: pgmap v7989: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 199 KiB/s wr, 30 op/s 2023-09-10T03:24:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:07 smithi098 ceph-mon[103124]: pgmap v7989: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 199 KiB/s wr, 30 op/s 2023-09-10T03:24:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14678.898325 secs 2023-09-10T03:24:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:08 smithi082 ceph-mon[98694]: osdmap e567: 12 total, 12 up, 12 in 2023-09-10T03:24:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14680.228354 secs 2023-09-10T03:24:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14678.898325 secs 2023-09-10T03:24:09.076 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:08 smithi098 ceph-mon[103124]: osdmap e567: 12 total, 12 up, 12 in 2023-09-10T03:24:09.077 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14680.228354 secs 2023-09-10T03:24:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14678.898325 secs 2023-09-10T03:24:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:08 smithi170 ceph-mon[102832]: osdmap e567: 12 total, 12 up, 12 in 2023-09-10T03:24:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14680.228354 secs 2023-09-10T03:24:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:09 smithi082 ceph-mon[98694]: pgmap v7991: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 120 KiB/s wr, 30 op/s 2023-09-10T03:24:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14680.939621 secs 2023-09-10T03:24:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:09 smithi098 ceph-mon[103124]: pgmap v7991: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 120 KiB/s wr, 30 op/s 2023-09-10T03:24:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14680.939621 secs 2023-09-10T03:24:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:09 smithi170 ceph-mon[102832]: pgmap v7991: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 120 KiB/s wr, 30 op/s 2023-09-10T03:24:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14680.939621 secs 2023-09-10T03:24:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:11 smithi082 ceph-mon[98694]: pgmap v7992: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-10T03:24:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:11 smithi098 ceph-mon[103124]: pgmap v7992: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-10T03:24:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:11 smithi170 ceph-mon[102832]: pgmap v7992: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-10T03:24:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14683.898385 secs 2023-09-10T03:24:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:13 smithi170 ceph-mon[102832]: osdmap e568: 12 total, 12 up, 12 in 2023-09-10T03:24:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:13 smithi170 ceph-mon[102832]: pgmap v7994: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 87 KiB/s wr, 37 op/s 2023-09-10T03:24:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14683.898385 secs 2023-09-10T03:24:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:13 smithi082 ceph-mon[98694]: osdmap e568: 12 total, 12 up, 12 in 2023-09-10T03:24:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:13 smithi082 ceph-mon[98694]: pgmap v7994: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 87 KiB/s wr, 37 op/s 2023-09-10T03:24:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14683.898385 secs 2023-09-10T03:24:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:13 smithi098 ceph-mon[103124]: osdmap e568: 12 total, 12 up, 12 in 2023-09-10T03:24:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:13 smithi098 ceph-mon[103124]: pgmap v7994: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 87 KiB/s wr, 37 op/s 2023-09-10T03:24:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14685.228412 secs 2023-09-10T03:24:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14685.939689 secs 2023-09-10T03:24:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14685.228412 secs 2023-09-10T03:24:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14685.939689 secs 2023-09-10T03:24:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14685.228412 secs 2023-09-10T03:24:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14685.939689 secs 2023-09-10T03:24:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:15 smithi170 ceph-mon[102832]: pgmap v7995: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-09-10T03:24:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:15 smithi082 ceph-mon[98694]: pgmap v7995: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-09-10T03:24:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:15 smithi098 ceph-mon[103124]: pgmap v7995: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-09-10T03:24:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:17 smithi170 ceph-mon[102832]: pgmap v7996: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 13 op/s 2023-09-10T03:24:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:17 smithi082 ceph-mon[98694]: pgmap v7996: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 13 op/s 2023-09-10T03:24:18.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:17 smithi098 ceph-mon[103124]: pgmap v7996: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 13 op/s 2023-09-10T03:24:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14688.898445 secs 2023-09-10T03:24:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14690.228496 secs 2023-09-10T03:24:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14688.898445 secs 2023-09-10T03:24:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14690.228496 secs 2023-09-10T03:24:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14688.898445 secs 2023-09-10T03:24:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14690.228496 secs 2023-09-10T03:24:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:19 smithi170 ceph-mon[102832]: pgmap v7997: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T03:24:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14690.939755 secs 2023-09-10T03:24:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:19 smithi082 ceph-mon[98694]: pgmap v7997: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T03:24:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14690.939755 secs 2023-09-10T03:24:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:19 smithi098 ceph-mon[103124]: pgmap v7997: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T03:24:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14690.939755 secs 2023-09-10T03:24:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:21 smithi082 ceph-mon[98694]: pgmap v7998: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T03:24:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:21 smithi098 ceph-mon[103124]: pgmap v7998: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T03:24:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:21 smithi170 ceph-mon[102832]: pgmap v7998: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T03:24:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:23 smithi170 ceph-mon[102832]: pgmap v7999: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 109 KiB/s wr, 11 op/s 2023-09-10T03:24:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:24:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:23 smithi082 ceph-mon[98694]: pgmap v7999: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 109 KiB/s wr, 11 op/s 2023-09-10T03:24:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:24:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:23 smithi098 ceph-mon[103124]: pgmap v7999: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 109 KiB/s wr, 11 op/s 2023-09-10T03:24:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:24:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14695.228551 secs 2023-09-10T03:24:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:24:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:24:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:24:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14695.939821 secs 2023-09-10T03:24:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14695.228551 secs 2023-09-10T03:24:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:24:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:24:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:24:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14695.939821 secs 2023-09-10T03:24:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14695.228551 secs 2023-09-10T03:24:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:24:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:24:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:24:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14695.939821 secs 2023-09-10T03:24:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:25 smithi170 ceph-mon[102832]: pgmap v8000: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 9 op/s 2023-09-10T03:24:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:25 smithi082 ceph-mon[98694]: pgmap v8000: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 9 op/s 2023-09-10T03:24:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:25 smithi098 ceph-mon[103124]: pgmap v8000: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 9 op/s 2023-09-10T03:24:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:27 smithi170 ceph-mon[102832]: pgmap v8001: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 150 KiB/s wr, 15 op/s 2023-09-10T03:24:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:27 smithi082 ceph-mon[98694]: pgmap v8001: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 150 KiB/s wr, 15 op/s 2023-09-10T03:24:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:27 smithi098 ceph-mon[103124]: pgmap v8001: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 150 KiB/s wr, 15 op/s 2023-09-10T03:24:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14700.228608 secs 2023-09-10T03:24:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14700.228608 secs 2023-09-10T03:24:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14700.228608 secs 2023-09-10T03:24:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:29 smithi082 ceph-mon[98694]: pgmap v8002: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 10 op/s 2023-09-10T03:24:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14700.939885 secs 2023-09-10T03:24:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:29 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:24:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:29 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:24:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:29 smithi098 ceph-mon[103124]: pgmap v8002: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 10 op/s 2023-09-10T03:24:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14700.939885 secs 2023-09-10T03:24:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:29 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:24:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:29 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:24:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:29 smithi170 ceph-mon[102832]: pgmap v8002: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 10 op/s 2023-09-10T03:24:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14700.939885 secs 2023-09-10T03:24:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:29 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:24:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:29 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:24:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:31 smithi082 ceph-mon[98694]: pgmap v8003: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 201 KiB/s wr, 16 op/s 2023-09-10T03:24:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:31 smithi098 ceph-mon[103124]: pgmap v8003: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 201 KiB/s wr, 16 op/s 2023-09-10T03:24:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:31 smithi170 ceph-mon[102832]: pgmap v8003: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 201 KiB/s wr, 16 op/s 2023-09-10T03:24:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:33 smithi170 ceph-mon[102832]: pgmap v8004: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 156 KiB/s wr, 11 op/s 2023-09-10T03:24:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:33 smithi082 ceph-mon[98694]: pgmap v8004: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 156 KiB/s wr, 11 op/s 2023-09-10T03:24:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:33 smithi098 ceph-mon[103124]: pgmap v8004: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 156 KiB/s wr, 11 op/s 2023-09-10T03:24:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14705.228663 secs 2023-09-10T03:24:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14705.939950 secs 2023-09-10T03:24:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14705.228663 secs 2023-09-10T03:24:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14705.939950 secs 2023-09-10T03:24:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14705.228663 secs 2023-09-10T03:24:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14705.939950 secs 2023-09-10T03:24:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:35 smithi170 ceph-mon[102832]: pgmap v8005: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 156 KiB/s wr, 11 op/s 2023-09-10T03:24:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:35 smithi082 ceph-mon[98694]: pgmap v8005: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 156 KiB/s wr, 11 op/s 2023-09-10T03:24:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:35 smithi098 ceph-mon[103124]: pgmap v8005: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 156 KiB/s wr, 11 op/s 2023-09-10T03:24:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:37 smithi170 ceph-mon[102832]: pgmap v8006: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 316 KiB/s wr, 17 op/s 2023-09-10T03:24:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:24:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:24:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:37 smithi082 ceph-mon[98694]: pgmap v8006: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 316 KiB/s wr, 17 op/s 2023-09-10T03:24:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:24:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:24:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:37 smithi098 ceph-mon[103124]: pgmap v8006: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 316 KiB/s wr, 17 op/s 2023-09-10T03:24:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:24:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:24:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14710.228722 secs 2023-09-10T03:24:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:39 smithi170 ceph-mon[102832]: pgmap v8007: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 257 KiB/s wr, 12 op/s 2023-09-10T03:24:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14710.940017 secs 2023-09-10T03:24:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14710.228722 secs 2023-09-10T03:24:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:39 smithi082 ceph-mon[98694]: pgmap v8007: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 257 KiB/s wr, 12 op/s 2023-09-10T03:24:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14710.940017 secs 2023-09-10T03:24:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14710.228722 secs 2023-09-10T03:24:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:39 smithi098 ceph-mon[103124]: pgmap v8007: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 257 KiB/s wr, 12 op/s 2023-09-10T03:24:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14710.940017 secs 2023-09-10T03:24:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:24:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:24:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:24:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:24:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:24:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:24:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:41 smithi170 ceph-mon[102832]: pgmap v8008: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 428 KiB/s wr, 18 op/s 2023-09-10T03:24:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:41 smithi082 ceph-mon[98694]: pgmap v8008: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 428 KiB/s wr, 18 op/s 2023-09-10T03:24:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:41 smithi098 ceph-mon[103124]: pgmap v8008: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 428 KiB/s wr, 18 op/s 2023-09-10T03:24:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:43 smithi170 ceph-mon[102832]: pgmap v8009: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 331 KiB/s wr, 13 op/s 2023-09-10T03:24:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:43 smithi082 ceph-mon[98694]: pgmap v8009: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 331 KiB/s wr, 13 op/s 2023-09-10T03:24:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:43 smithi098 ceph-mon[103124]: pgmap v8009: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 331 KiB/s wr, 13 op/s 2023-09-10T03:24:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14715.228777 secs 2023-09-10T03:24:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14715.940083 secs 2023-09-10T03:24:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14715.228777 secs 2023-09-10T03:24:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14715.940083 secs 2023-09-10T03:24:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14715.228777 secs 2023-09-10T03:24:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14715.940083 secs 2023-09-10T03:24:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:45 smithi170 ceph-mon[102832]: pgmap v8010: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 331 KiB/s wr, 13 op/s 2023-09-10T03:24:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:45 smithi082 ceph-mon[98694]: pgmap v8010: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 331 KiB/s wr, 13 op/s 2023-09-10T03:24:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:45 smithi098 ceph-mon[103124]: pgmap v8010: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 331 KiB/s wr, 13 op/s 2023-09-10T03:24:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:47 smithi082 ceph-mon[98694]: pgmap v8011: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 366 KiB/s wr, 18 op/s 2023-09-10T03:24:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:47 smithi098 ceph-mon[103124]: pgmap v8011: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 366 KiB/s wr, 18 op/s 2023-09-10T03:24:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:47 smithi170 ceph-mon[102832]: pgmap v8011: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 366 KiB/s wr, 18 op/s 2023-09-10T03:24:48.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14720.228834 secs 2023-09-10T03:24:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14720.228834 secs 2023-09-10T03:24:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14720.228834 secs 2023-09-10T03:24:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:49 smithi082 ceph-mon[98694]: pgmap v8012: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 207 KiB/s wr, 12 op/s 2023-09-10T03:24:50.058 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14720.940150 secs 2023-09-10T03:24:50.059 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:49 smithi098 ceph-mon[103124]: pgmap v8012: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 207 KiB/s wr, 12 op/s 2023-09-10T03:24:50.059 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14720.940150 secs 2023-09-10T03:24:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:49 smithi170 ceph-mon[102832]: pgmap v8012: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 207 KiB/s wr, 12 op/s 2023-09-10T03:24:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14720.940150 secs 2023-09-10T03:24:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:51 smithi082 ceph-mon[98694]: pgmap v8013: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 18 op/s 2023-09-10T03:24:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:51 smithi098 ceph-mon[103124]: pgmap v8013: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 18 op/s 2023-09-10T03:24:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:51 smithi170 ceph-mon[102832]: pgmap v8013: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 18 op/s 2023-09-10T03:24:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14723.898947 secs 2023-09-10T03:24:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:53 smithi170 ceph-mon[102832]: pgmap v8014: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T03:24:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14723.898947 secs 2023-09-10T03:24:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:53 smithi082 ceph-mon[98694]: pgmap v8014: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T03:24:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14723.898947 secs 2023-09-10T03:24:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:53 smithi098 ceph-mon[103124]: pgmap v8014: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T03:24:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14725.228889 secs 2023-09-10T03:24:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14725.940214 secs 2023-09-10T03:24:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14725.228889 secs 2023-09-10T03:24:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14725.940214 secs 2023-09-10T03:24:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14725.228889 secs 2023-09-10T03:24:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14725.940214 secs 2023-09-10T03:24:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:55 smithi170 ceph-mon[102832]: pgmap v8015: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T03:24:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:24:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:24:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:24:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:24:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:55 smithi082 ceph-mon[98694]: pgmap v8015: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T03:24:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:24:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:24:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:24:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:24:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:55 smithi098 ceph-mon[103124]: pgmap v8015: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T03:24:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:24:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:24:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:24:56.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:24:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:24:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:24:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:24:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:24:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:24:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:24:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:24:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:24:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:24:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:57 smithi170 ceph-mon[102832]: pgmap v8016: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 227 KiB/s wr, 16 op/s 2023-09-10T03:24:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:57 smithi082 ceph-mon[98694]: pgmap v8016: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 227 KiB/s wr, 16 op/s 2023-09-10T03:24:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:57 smithi098 ceph-mon[103124]: pgmap v8016: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 227 KiB/s wr, 16 op/s 2023-09-10T03:24:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14728.899008 secs 2023-09-10T03:24:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14728.899008 secs 2023-09-10T03:24:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14728.899008 secs 2023-09-10T03:24:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14730.228943 secs 2023-09-10T03:24:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:59 smithi170 ceph-mon[102832]: pgmap v8017: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 192 KiB/s wr, 11 op/s 2023-09-10T03:24:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:24:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14730.940283 secs 2023-09-10T03:24:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14730.228943 secs 2023-09-10T03:24:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:59 smithi082 ceph-mon[98694]: pgmap v8017: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 192 KiB/s wr, 11 op/s 2023-09-10T03:24:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:24:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14730.940283 secs 2023-09-10T03:25:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14730.228943 secs 2023-09-10T03:25:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:59 smithi098 ceph-mon[103124]: pgmap v8017: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 192 KiB/s wr, 11 op/s 2023-09-10T03:25:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:24:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14730.940283 secs 2023-09-10T03:25:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:01 smithi170 ceph-mon[102832]: pgmap v8018: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 228 KiB/s wr, 16 op/s 2023-09-10T03:25:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:01 smithi082 ceph-mon[98694]: pgmap v8018: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 228 KiB/s wr, 16 op/s 2023-09-10T03:25:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:01 smithi098 ceph-mon[103124]: pgmap v8018: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 228 KiB/s wr, 16 op/s 2023-09-10T03:25:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14733.899128 secs 2023-09-10T03:25:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:03 smithi170 ceph-mon[102832]: pgmap v8019: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T03:25:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14733.899128 secs 2023-09-10T03:25:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:03 smithi082 ceph-mon[98694]: pgmap v8019: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T03:25:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14733.899128 secs 2023-09-10T03:25:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:03 smithi098 ceph-mon[103124]: pgmap v8019: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T03:25:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14735.228998 secs 2023-09-10T03:25:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14735.940348 secs 2023-09-10T03:25:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14735.228998 secs 2023-09-10T03:25:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14735.940348 secs 2023-09-10T03:25:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14735.228998 secs 2023-09-10T03:25:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14735.940348 secs 2023-09-10T03:25:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:05 smithi170 ceph-mon[102832]: pgmap v8020: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T03:25:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:05 smithi082 ceph-mon[98694]: pgmap v8020: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T03:25:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:05 smithi098 ceph-mon[103124]: pgmap v8020: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T03:25:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:07 smithi082 ceph-mon[98694]: pgmap v8021: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 188 KiB/s wr, 30 op/s 2023-09-10T03:25:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:07 smithi098 ceph-mon[103124]: pgmap v8021: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 188 KiB/s wr, 30 op/s 2023-09-10T03:25:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:07 smithi170 ceph-mon[102832]: pgmap v8021: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 188 KiB/s wr, 30 op/s 2023-09-10T03:25:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14738.899186 secs 2023-09-10T03:25:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:08 smithi082 ceph-mon[98694]: osdmap e569: 12 total, 12 up, 12 in 2023-09-10T03:25:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14740.229079 secs 2023-09-10T03:25:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14738.899186 secs 2023-09-10T03:25:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:08 smithi098 ceph-mon[103124]: osdmap e569: 12 total, 12 up, 12 in 2023-09-10T03:25:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14740.229079 secs 2023-09-10T03:25:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14738.899186 secs 2023-09-10T03:25:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:08 smithi170 ceph-mon[102832]: osdmap e569: 12 total, 12 up, 12 in 2023-09-10T03:25:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14740.229079 secs 2023-09-10T03:25:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:09 smithi082 ceph-mon[98694]: pgmap v8023: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 99 KiB/s wr, 30 op/s 2023-09-10T03:25:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14740.940412 secs 2023-09-10T03:25:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:09 smithi098 ceph-mon[103124]: pgmap v8023: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 99 KiB/s wr, 30 op/s 2023-09-10T03:25:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14740.940412 secs 2023-09-10T03:25:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:09 smithi170 ceph-mon[102832]: pgmap v8023: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 99 KiB/s wr, 30 op/s 2023-09-10T03:25:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14740.940412 secs 2023-09-10T03:25:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:11 smithi082 ceph-mon[98694]: pgmap v8024: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 70 KiB/s wr, 30 op/s 2023-09-10T03:25:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:11 smithi098 ceph-mon[103124]: pgmap v8024: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 70 KiB/s wr, 30 op/s 2023-09-10T03:25:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:11 smithi170 ceph-mon[102832]: pgmap v8024: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 70 KiB/s wr, 30 op/s 2023-09-10T03:25:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14743.899252 secs 2023-09-10T03:25:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:13 smithi170 ceph-mon[102832]: osdmap e570: 12 total, 12 up, 12 in 2023-09-10T03:25:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:13 smithi170 ceph-mon[102832]: pgmap v8026: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 87 KiB/s wr, 37 op/s 2023-09-10T03:25:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14743.899252 secs 2023-09-10T03:25:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:13 smithi082 ceph-mon[98694]: osdmap e570: 12 total, 12 up, 12 in 2023-09-10T03:25:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:13 smithi082 ceph-mon[98694]: pgmap v8026: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 87 KiB/s wr, 37 op/s 2023-09-10T03:25:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14743.899252 secs 2023-09-10T03:25:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:13 smithi098 ceph-mon[103124]: osdmap e570: 12 total, 12 up, 12 in 2023-09-10T03:25:14.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:13 smithi098 ceph-mon[103124]: pgmap v8026: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 87 KiB/s wr, 37 op/s 2023-09-10T03:25:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14745.229162 secs 2023-09-10T03:25:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14745.940502 secs 2023-09-10T03:25:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14745.229162 secs 2023-09-10T03:25:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14745.940502 secs 2023-09-10T03:25:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14745.229162 secs 2023-09-10T03:25:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14745.940502 secs 2023-09-10T03:25:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:15 smithi170 ceph-mon[102832]: pgmap v8027: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 7 op/s 2023-09-10T03:25:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:15 smithi082 ceph-mon[98694]: pgmap v8027: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 7 op/s 2023-09-10T03:25:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:15 smithi098 ceph-mon[103124]: pgmap v8027: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 7 op/s 2023-09-10T03:25:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:17 smithi082 ceph-mon[98694]: pgmap v8028: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 14 op/s 2023-09-10T03:25:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:17 smithi098 ceph-mon[103124]: pgmap v8028: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 14 op/s 2023-09-10T03:25:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:17 smithi170 ceph-mon[102832]: pgmap v8028: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 14 op/s 2023-09-10T03:25:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14748.899323 secs 2023-09-10T03:25:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14750.229243 secs 2023-09-10T03:25:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14748.899323 secs 2023-09-10T03:25:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14750.229243 secs 2023-09-10T03:25:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14748.899323 secs 2023-09-10T03:25:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14750.229243 secs 2023-09-10T03:25:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:19 smithi082 ceph-mon[98694]: pgmap v8029: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-10T03:25:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14750.940567 secs 2023-09-10T03:25:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:19 smithi098 ceph-mon[103124]: pgmap v8029: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-10T03:25:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14750.940567 secs 2023-09-10T03:25:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:19 smithi170 ceph-mon[102832]: pgmap v8029: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-10T03:25:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14750.940567 secs 2023-09-10T03:25:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:21 smithi082 ceph-mon[98694]: pgmap v8030: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 12 op/s 2023-09-10T03:25:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:21 smithi098 ceph-mon[103124]: pgmap v8030: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 12 op/s 2023-09-10T03:25:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:21 smithi170 ceph-mon[102832]: pgmap v8030: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 12 op/s 2023-09-10T03:25:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:23 smithi170 ceph-mon[102832]: pgmap v8031: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 12 op/s 2023-09-10T03:25:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:25:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:23 smithi082 ceph-mon[98694]: pgmap v8031: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 12 op/s 2023-09-10T03:25:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:25:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:23 smithi098 ceph-mon[103124]: pgmap v8031: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 12 op/s 2023-09-10T03:25:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:25:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14755.229297 secs 2023-09-10T03:25:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:25:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:25:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:25:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14755.940628 secs 2023-09-10T03:25:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14755.229297 secs 2023-09-10T03:25:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:25:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:25:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:25:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14755.940628 secs 2023-09-10T03:25:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14755.229297 secs 2023-09-10T03:25:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:25:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:25:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:25:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14755.940628 secs 2023-09-10T03:25:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:25 smithi170 ceph-mon[102832]: pgmap v8032: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T03:25:25.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:25 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:25:25.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:25 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:25:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:25 smithi082 ceph-mon[98694]: pgmap v8032: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T03:25:25.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:25 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:25:25.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:25 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:25:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:25 smithi098 ceph-mon[103124]: pgmap v8032: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 10 op/s 2023-09-10T03:25:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:25 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:25:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:25 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:25:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:27 smithi170 ceph-mon[102832]: pgmap v8033: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 120 KiB/s wr, 15 op/s 2023-09-10T03:25:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:27 smithi082 ceph-mon[98694]: pgmap v8033: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 120 KiB/s wr, 15 op/s 2023-09-10T03:25:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:27 smithi098 ceph-mon[103124]: pgmap v8033: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 120 KiB/s wr, 15 op/s 2023-09-10T03:25:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14760.229376 secs 2023-09-10T03:25:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14760.229376 secs 2023-09-10T03:25:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14760.229376 secs 2023-09-10T03:25:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:29 smithi170 ceph-mon[102832]: pgmap v8034: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 9 op/s 2023-09-10T03:25:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14760.940697 secs 2023-09-10T03:25:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:29 smithi082 ceph-mon[98694]: pgmap v8034: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 9 op/s 2023-09-10T03:25:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14760.940697 secs 2023-09-10T03:25:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:29 smithi098 ceph-mon[103124]: pgmap v8034: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 9 op/s 2023-09-10T03:25:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14760.940697 secs 2023-09-10T03:25:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:31 smithi170 ceph-mon[102832]: pgmap v8035: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 177 KiB/s wr, 15 op/s 2023-09-10T03:25:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:31 smithi082 ceph-mon[98694]: pgmap v8035: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 177 KiB/s wr, 15 op/s 2023-09-10T03:25:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:31 smithi098 ceph-mon[103124]: pgmap v8035: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 177 KiB/s wr, 15 op/s 2023-09-10T03:25:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:33 smithi170 ceph-mon[102832]: pgmap v8036: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 10 op/s 2023-09-10T03:25:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:33 smithi082 ceph-mon[98694]: pgmap v8036: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 10 op/s 2023-09-10T03:25:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:33 smithi098 ceph-mon[103124]: pgmap v8036: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 10 op/s 2023-09-10T03:25:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14765.229463 secs 2023-09-10T03:25:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14765.940762 secs 2023-09-10T03:25:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14765.229463 secs 2023-09-10T03:25:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14765.940762 secs 2023-09-10T03:25:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14765.229463 secs 2023-09-10T03:25:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14765.940762 secs 2023-09-10T03:25:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:35 smithi170 ceph-mon[102832]: pgmap v8037: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 10 op/s 2023-09-10T03:25:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:35 smithi082 ceph-mon[98694]: pgmap v8037: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 10 op/s 2023-09-10T03:25:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:35 smithi098 ceph-mon[103124]: pgmap v8037: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 10 op/s 2023-09-10T03:25:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:37 smithi082 ceph-mon[98694]: pgmap v8038: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 213 KiB/s wr, 16 op/s 2023-09-10T03:25:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:37 smithi098 ceph-mon[103124]: pgmap v8038: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 213 KiB/s wr, 16 op/s 2023-09-10T03:25:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:37 smithi170 ceph-mon[102832]: pgmap v8038: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 213 KiB/s wr, 16 op/s 2023-09-10T03:25:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14770.229492 secs 2023-09-10T03:25:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14770.229492 secs 2023-09-10T03:25:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14770.229492 secs 2023-09-10T03:25:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:39 smithi082 ceph-mon[98694]: pgmap v8039: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T03:25:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14770.940828 secs 2023-09-10T03:25:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:39 smithi098 ceph-mon[103124]: pgmap v8039: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T03:25:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14770.940828 secs 2023-09-10T03:25:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:39 smithi170 ceph-mon[102832]: pgmap v8039: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T03:25:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14770.940828 secs 2023-09-10T03:25:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:41 smithi082 ceph-mon[98694]: pgmap v8040: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 327 KiB/s wr, 18 op/s 2023-09-10T03:25:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:41 smithi098 ceph-mon[103124]: pgmap v8040: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 327 KiB/s wr, 18 op/s 2023-09-10T03:25:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:41 smithi170 ceph-mon[102832]: pgmap v8040: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 327 KiB/s wr, 18 op/s 2023-09-10T03:25:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:43 smithi170 ceph-mon[102832]: pgmap v8041: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 221 KiB/s wr, 12 op/s 2023-09-10T03:25:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:43 smithi082 ceph-mon[98694]: pgmap v8041: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 221 KiB/s wr, 12 op/s 2023-09-10T03:25:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:43 smithi098 ceph-mon[103124]: pgmap v8041: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 221 KiB/s wr, 12 op/s 2023-09-10T03:25:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14775.229517 secs 2023-09-10T03:25:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:25:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:25:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14775.940892 secs 2023-09-10T03:25:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14775.229517 secs 2023-09-10T03:25:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:25:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:25:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14775.940892 secs 2023-09-10T03:25:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14775.229517 secs 2023-09-10T03:25:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:25:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:25:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14775.940892 secs 2023-09-10T03:25:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:25:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:25:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:45 smithi170 ceph-mon[102832]: pgmap v8042: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 221 KiB/s wr, 12 op/s 2023-09-10T03:25:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:25:45.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:25:45.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:45 smithi082 ceph-mon[98694]: pgmap v8042: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 221 KiB/s wr, 12 op/s 2023-09-10T03:25:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:25:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:25:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:45 smithi098 ceph-mon[103124]: pgmap v8042: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 221 KiB/s wr, 12 op/s 2023-09-10T03:25:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:47 smithi170 ceph-mon[102832]: pgmap v8043: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 282 KiB/s wr, 18 op/s 2023-09-10T03:25:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:47 smithi082 ceph-mon[98694]: pgmap v8043: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 282 KiB/s wr, 18 op/s 2023-09-10T03:25:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:47 smithi098 ceph-mon[103124]: pgmap v8043: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 282 KiB/s wr, 18 op/s 2023-09-10T03:25:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14780.229575 secs 2023-09-10T03:25:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14780.229575 secs 2023-09-10T03:25:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14780.229575 secs 2023-09-10T03:25:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:49 smithi170 ceph-mon[102832]: pgmap v8044: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 246 KiB/s wr, 12 op/s 2023-09-10T03:25:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14780.940957 secs 2023-09-10T03:25:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:49 smithi082 ceph-mon[98694]: pgmap v8044: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 246 KiB/s wr, 12 op/s 2023-09-10T03:25:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14780.940957 secs 2023-09-10T03:25:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:49 smithi098 ceph-mon[103124]: pgmap v8044: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 246 KiB/s wr, 12 op/s 2023-09-10T03:25:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14780.940957 secs 2023-09-10T03:25:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:51 smithi170 ceph-mon[102832]: pgmap v8045: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 462 KiB/s wr, 18 op/s 2023-09-10T03:25:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:51 smithi082 ceph-mon[98694]: pgmap v8045: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 462 KiB/s wr, 18 op/s 2023-09-10T03:25:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:51 smithi098 ceph-mon[103124]: pgmap v8045: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 462 KiB/s wr, 18 op/s 2023-09-10T03:25:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14783.899901 secs 2023-09-10T03:25:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:53 smithi170 ceph-mon[102832]: pgmap v8046: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 276 KiB/s wr, 11 op/s 2023-09-10T03:25:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14783.899901 secs 2023-09-10T03:25:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:53 smithi082 ceph-mon[98694]: pgmap v8046: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 276 KiB/s wr, 11 op/s 2023-09-10T03:25:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14783.899901 secs 2023-09-10T03:25:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:53 smithi098 ceph-mon[103124]: pgmap v8046: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 276 KiB/s wr, 11 op/s 2023-09-10T03:25:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14785.229633 secs 2023-09-10T03:25:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14785.941022 secs 2023-09-10T03:25:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14785.229633 secs 2023-09-10T03:25:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14785.941022 secs 2023-09-10T03:25:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14785.229633 secs 2023-09-10T03:25:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14785.941022 secs 2023-09-10T03:25:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:55 smithi170 ceph-mon[102832]: pgmap v8047: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 276 KiB/s wr, 11 op/s 2023-09-10T03:25:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:25:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:55 smithi082 ceph-mon[98694]: pgmap v8047: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 276 KiB/s wr, 11 op/s 2023-09-10T03:25:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:25:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:55 smithi098 ceph-mon[103124]: pgmap v8047: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 276 KiB/s wr, 11 op/s 2023-09-10T03:25:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:25:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:25:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:25:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:25:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:25:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:25:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:25:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:25:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:25:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:25:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:25:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:25:56.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:25:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:25:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:25:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:25:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:25:57.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:25:57.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:25:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:57 smithi170 ceph-mon[102832]: pgmap v8048: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 314 KiB/s wr, 17 op/s 2023-09-10T03:25:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:25:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:25:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:57 smithi082 ceph-mon[98694]: pgmap v8048: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 314 KiB/s wr, 17 op/s 2023-09-10T03:25:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:25:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:25:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:57 smithi098 ceph-mon[103124]: pgmap v8048: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 314 KiB/s wr, 17 op/s 2023-09-10T03:25:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:25:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:25:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14788.899978 secs 2023-09-10T03:25:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14790.229688 secs 2023-09-10T03:25:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14788.899978 secs 2023-09-10T03:25:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14790.229688 secs 2023-09-10T03:25:59.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14788.899978 secs 2023-09-10T03:25:59.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14790.229688 secs 2023-09-10T03:25:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:59 smithi170 ceph-mon[102832]: pgmap v8049: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 253 KiB/s wr, 11 op/s 2023-09-10T03:25:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:25:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14790.941086 secs 2023-09-10T03:25:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:59 smithi082 ceph-mon[98694]: pgmap v8049: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 253 KiB/s wr, 11 op/s 2023-09-10T03:25:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:25:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14790.941086 secs 2023-09-10T03:26:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:59 smithi098 ceph-mon[103124]: pgmap v8049: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 253 KiB/s wr, 11 op/s 2023-09-10T03:26:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:25:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14790.941086 secs 2023-09-10T03:26:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:26:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:26:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:26:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:26:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:26:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:26:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:01 smithi170 ceph-mon[102832]: pgmap v8050: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 327 KiB/s wr, 17 op/s 2023-09-10T03:26:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:01 smithi082 ceph-mon[98694]: pgmap v8050: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 327 KiB/s wr, 17 op/s 2023-09-10T03:26:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:01 smithi098 ceph-mon[103124]: pgmap v8050: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 327 KiB/s wr, 17 op/s 2023-09-10T03:26:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14793.900037 secs 2023-09-10T03:26:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:03 smithi170 ceph-mon[102832]: pgmap v8051: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T03:26:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14793.900037 secs 2023-09-10T03:26:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:03 smithi082 ceph-mon[98694]: pgmap v8051: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T03:26:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14793.900037 secs 2023-09-10T03:26:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:03 smithi098 ceph-mon[103124]: pgmap v8051: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T03:26:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14795.229746 secs 2023-09-10T03:26:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14795.941152 secs 2023-09-10T03:26:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14795.229746 secs 2023-09-10T03:26:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14795.941152 secs 2023-09-10T03:26:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14795.229746 secs 2023-09-10T03:26:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14795.941152 secs 2023-09-10T03:26:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:05 smithi170 ceph-mon[102832]: pgmap v8052: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T03:26:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:05 smithi082 ceph-mon[98694]: pgmap v8052: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T03:26:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:05 smithi098 ceph-mon[103124]: pgmap v8052: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T03:26:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:07 smithi170 ceph-mon[102832]: pgmap v8053: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 188 KiB/s wr, 30 op/s 2023-09-10T03:26:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:07 smithi082 ceph-mon[98694]: pgmap v8053: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 188 KiB/s wr, 30 op/s 2023-09-10T03:26:08.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:07 smithi098 ceph-mon[103124]: pgmap v8053: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 188 KiB/s wr, 30 op/s 2023-09-10T03:26:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14798.900135 secs 2023-09-10T03:26:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:08 smithi170 ceph-mon[102832]: osdmap e571: 12 total, 12 up, 12 in 2023-09-10T03:26:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14800.229807 secs 2023-09-10T03:26:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14798.900135 secs 2023-09-10T03:26:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:08 smithi082 ceph-mon[98694]: osdmap e571: 12 total, 12 up, 12 in 2023-09-10T03:26:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14800.229807 secs 2023-09-10T03:26:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14798.900135 secs 2023-09-10T03:26:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:08 smithi098 ceph-mon[103124]: osdmap e571: 12 total, 12 up, 12 in 2023-09-10T03:26:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14800.229807 secs 2023-09-10T03:26:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:09 smithi170 ceph-mon[102832]: pgmap v8055: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 181 KiB/s wr, 30 op/s 2023-09-10T03:26:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14800.941217 secs 2023-09-10T03:26:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:09 smithi082 ceph-mon[98694]: pgmap v8055: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 181 KiB/s wr, 30 op/s 2023-09-10T03:26:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14800.941217 secs 2023-09-10T03:26:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:09 smithi098 ceph-mon[103124]: pgmap v8055: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 181 KiB/s wr, 30 op/s 2023-09-10T03:26:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14800.941217 secs 2023-09-10T03:26:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:11 smithi082 ceph-mon[98694]: pgmap v8056: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 107 KiB/s wr, 30 op/s 2023-09-10T03:26:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:11 smithi098 ceph-mon[103124]: pgmap v8056: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 107 KiB/s wr, 30 op/s 2023-09-10T03:26:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:11 smithi170 ceph-mon[102832]: pgmap v8056: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 107 KiB/s wr, 30 op/s 2023-09-10T03:26:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14803.900195 secs 2023-09-10T03:26:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:13 smithi170 ceph-mon[102832]: osdmap e572: 12 total, 12 up, 12 in 2023-09-10T03:26:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:13 smithi170 ceph-mon[102832]: pgmap v8058: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 134 KiB/s wr, 37 op/s 2023-09-10T03:26:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14805.229881 secs 2023-09-10T03:26:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14803.900195 secs 2023-09-10T03:26:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:13 smithi082 ceph-mon[98694]: osdmap e572: 12 total, 12 up, 12 in 2023-09-10T03:26:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:13 smithi082 ceph-mon[98694]: pgmap v8058: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 134 KiB/s wr, 37 op/s 2023-09-10T03:26:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14805.229881 secs 2023-09-10T03:26:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14803.900195 secs 2023-09-10T03:26:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:13 smithi098 ceph-mon[103124]: osdmap e572: 12 total, 12 up, 12 in 2023-09-10T03:26:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:13 smithi098 ceph-mon[103124]: pgmap v8058: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 134 KiB/s wr, 37 op/s 2023-09-10T03:26:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14805.229881 secs 2023-09-10T03:26:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14805.941284 secs 2023-09-10T03:26:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14805.941284 secs 2023-09-10T03:26:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14805.941284 secs 2023-09-10T03:26:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:15 smithi170 ceph-mon[102832]: pgmap v8059: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 8 op/s 2023-09-10T03:26:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:15 smithi082 ceph-mon[98694]: pgmap v8059: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 8 op/s 2023-09-10T03:26:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:15 smithi098 ceph-mon[103124]: pgmap v8059: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 8 op/s 2023-09-10T03:26:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:17 smithi170 ceph-mon[102832]: pgmap v8060: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-09-10T03:26:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:17 smithi082 ceph-mon[98694]: pgmap v8060: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-09-10T03:26:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:17 smithi098 ceph-mon[103124]: pgmap v8060: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-09-10T03:26:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14808.900254 secs 2023-09-10T03:26:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14810.229940 secs 2023-09-10T03:26:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14808.900254 secs 2023-09-10T03:26:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14810.229940 secs 2023-09-10T03:26:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14808.900254 secs 2023-09-10T03:26:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14810.229940 secs 2023-09-10T03:26:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:19 smithi170 ceph-mon[102832]: pgmap v8061: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2023-09-10T03:26:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14810.941350 secs 2023-09-10T03:26:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:19 smithi082 ceph-mon[98694]: pgmap v8061: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2023-09-10T03:26:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14810.941350 secs 2023-09-10T03:26:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:19 smithi098 ceph-mon[103124]: pgmap v8061: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2023-09-10T03:26:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14810.941350 secs 2023-09-10T03:26:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:21 smithi170 ceph-mon[102832]: pgmap v8062: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T03:26:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:21 smithi082 ceph-mon[98694]: pgmap v8062: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T03:26:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:21 smithi098 ceph-mon[103124]: pgmap v8062: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T03:26:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14813.900373 secs 2023-09-10T03:26:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:23 smithi170 ceph-mon[102832]: pgmap v8063: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T03:26:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14815.230013 secs 2023-09-10T03:26:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14813.900373 secs 2023-09-10T03:26:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:23 smithi082 ceph-mon[98694]: pgmap v8063: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T03:26:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14815.230013 secs 2023-09-10T03:26:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14813.900373 secs 2023-09-10T03:26:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:23 smithi098 ceph-mon[103124]: pgmap v8063: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T03:26:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14815.230013 secs 2023-09-10T03:26:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14815.941414 secs 2023-09-10T03:26:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14815.941414 secs 2023-09-10T03:26:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14815.941414 secs 2023-09-10T03:26:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:25 smithi170 ceph-mon[102832]: pgmap v8064: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 9 op/s 2023-09-10T03:26:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:25 smithi082 ceph-mon[98694]: pgmap v8064: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 9 op/s 2023-09-10T03:26:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:25 smithi098 ceph-mon[103124]: pgmap v8064: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 9 op/s 2023-09-10T03:26:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:27 smithi082 ceph-mon[98694]: pgmap v8065: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 105 KiB/s wr, 15 op/s 2023-09-10T03:26:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:27 smithi098 ceph-mon[103124]: pgmap v8065: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 105 KiB/s wr, 15 op/s 2023-09-10T03:26:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:27 smithi170 ceph-mon[102832]: pgmap v8065: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 105 KiB/s wr, 15 op/s 2023-09-10T03:26:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14820.230072 secs 2023-09-10T03:26:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14820.230072 secs 2023-09-10T03:26:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14820.230072 secs 2023-09-10T03:26:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:29 smithi098 ceph-mon[103124]: pgmap v8066: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 9 op/s 2023-09-10T03:26:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14820.941484 secs 2023-09-10T03:26:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:29 smithi170 ceph-mon[102832]: pgmap v8066: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 9 op/s 2023-09-10T03:26:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14820.941484 secs 2023-09-10T03:26:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:29 smithi082 ceph-mon[98694]: pgmap v8066: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 9 op/s 2023-09-10T03:26:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14820.941484 secs 2023-09-10T03:26:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:31 smithi098 ceph-mon[103124]: pgmap v8067: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 283 KiB/s wr, 16 op/s 2023-09-10T03:26:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:26:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:31 smithi170 ceph-mon[102832]: pgmap v8067: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 283 KiB/s wr, 16 op/s 2023-09-10T03:26:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:26:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:31 smithi082 ceph-mon[98694]: pgmap v8067: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 283 KiB/s wr, 16 op/s 2023-09-10T03:26:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:26:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:26:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:26:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:26:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:26:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:26:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:26:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:26:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:26:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:26:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:33 smithi098 ceph-mon[103124]: pgmap v8068: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 236 KiB/s wr, 12 op/s 2023-09-10T03:26:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14825.230129 secs 2023-09-10T03:26:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:33 smithi170 ceph-mon[102832]: pgmap v8068: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 236 KiB/s wr, 12 op/s 2023-09-10T03:26:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14825.230129 secs 2023-09-10T03:26:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:33 smithi082 ceph-mon[98694]: pgmap v8068: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 236 KiB/s wr, 12 op/s 2023-09-10T03:26:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14825.230129 secs 2023-09-10T03:26:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14825.941548 secs 2023-09-10T03:26:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14825.941548 secs 2023-09-10T03:26:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14825.941548 secs 2023-09-10T03:26:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:35 smithi098 ceph-mon[103124]: pgmap v8069: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 236 KiB/s wr, 12 op/s 2023-09-10T03:26:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:35 smithi170 ceph-mon[102832]: pgmap v8069: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 236 KiB/s wr, 12 op/s 2023-09-10T03:26:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:35 smithi082 ceph-mon[98694]: pgmap v8069: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 236 KiB/s wr, 12 op/s 2023-09-10T03:26:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:26:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:26:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:26:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:26:37.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:26:37.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:26:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:37 smithi098 ceph-mon[103124]: pgmap v8070: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 245 KiB/s wr, 17 op/s 2023-09-10T03:26:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:26:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:26:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:37 smithi170 ceph-mon[102832]: pgmap v8070: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 245 KiB/s wr, 17 op/s 2023-09-10T03:26:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:26:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:26:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:37 smithi082 ceph-mon[98694]: pgmap v8070: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 245 KiB/s wr, 17 op/s 2023-09-10T03:26:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:26:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:26:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14830.230214 secs 2023-09-10T03:26:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14830.230214 secs 2023-09-10T03:26:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14830.230214 secs 2023-09-10T03:26:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:39 smithi098 ceph-mon[103124]: pgmap v8071: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 188 KiB/s wr, 12 op/s 2023-09-10T03:26:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14830.941617 secs 2023-09-10T03:26:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:39 smithi170 ceph-mon[102832]: pgmap v8071: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 188 KiB/s wr, 12 op/s 2023-09-10T03:26:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14830.941617 secs 2023-09-10T03:26:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:39 smithi082 ceph-mon[98694]: pgmap v8071: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 188 KiB/s wr, 12 op/s 2023-09-10T03:26:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14830.941617 secs 2023-09-10T03:26:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:41 smithi098 ceph-mon[103124]: pgmap v8072: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 444 KiB/s wr, 19 op/s 2023-09-10T03:26:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:41 smithi170 ceph-mon[102832]: pgmap v8072: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 444 KiB/s wr, 19 op/s 2023-09-10T03:26:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:41 smithi082 ceph-mon[98694]: pgmap v8072: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 444 KiB/s wr, 19 op/s 2023-09-10T03:26:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:43 smithi170 ceph-mon[102832]: pgmap v8073: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 265 KiB/s wr, 12 op/s 2023-09-10T03:26:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14835.230273 secs 2023-09-10T03:26:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:43 smithi082 ceph-mon[98694]: pgmap v8073: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 265 KiB/s wr, 12 op/s 2023-09-10T03:26:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14835.230273 secs 2023-09-10T03:26:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:43 smithi098 ceph-mon[103124]: pgmap v8073: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 265 KiB/s wr, 12 op/s 2023-09-10T03:26:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14835.230273 secs 2023-09-10T03:26:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14835.941681 secs 2023-09-10T03:26:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14835.941681 secs 2023-09-10T03:26:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14835.941681 secs 2023-09-10T03:26:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:45 smithi170 ceph-mon[102832]: pgmap v8074: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 265 KiB/s wr, 12 op/s 2023-09-10T03:26:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:45 smithi082 ceph-mon[98694]: pgmap v8074: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 265 KiB/s wr, 12 op/s 2023-09-10T03:26:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:45 smithi098 ceph-mon[103124]: pgmap v8074: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 265 KiB/s wr, 12 op/s 2023-09-10T03:26:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:47 smithi098 ceph-mon[103124]: pgmap v8075: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 300 KiB/s wr, 17 op/s 2023-09-10T03:26:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:47 smithi170 ceph-mon[102832]: pgmap v8075: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 300 KiB/s wr, 17 op/s 2023-09-10T03:26:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:47 smithi082 ceph-mon[98694]: pgmap v8075: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 300 KiB/s wr, 17 op/s 2023-09-10T03:26:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14840.230330 secs 2023-09-10T03:26:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14840.230330 secs 2023-09-10T03:26:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14840.230330 secs 2023-09-10T03:26:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:49 smithi098 ceph-mon[103124]: pgmap v8076: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 291 KiB/s wr, 12 op/s 2023-09-10T03:26:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14840.941745 secs 2023-09-10T03:26:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:49 smithi170 ceph-mon[102832]: pgmap v8076: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 291 KiB/s wr, 12 op/s 2023-09-10T03:26:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14840.941745 secs 2023-09-10T03:26:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:49 smithi082 ceph-mon[98694]: pgmap v8076: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 291 KiB/s wr, 12 op/s 2023-09-10T03:26:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14840.941745 secs 2023-09-10T03:26:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:51 smithi098 ceph-mon[103124]: pgmap v8077: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 378 KiB/s wr, 17 op/s 2023-09-10T03:26:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:51 smithi170 ceph-mon[102832]: pgmap v8077: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 378 KiB/s wr, 17 op/s 2023-09-10T03:26:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:51 smithi082 ceph-mon[98694]: pgmap v8077: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 378 KiB/s wr, 17 op/s 2023-09-10T03:26:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14843.900740 secs 2023-09-10T03:26:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:53 smithi098 ceph-mon[103124]: pgmap v8078: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T03:26:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:26:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:26:54.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14845.230412 secs 2023-09-10T03:26:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14843.900740 secs 2023-09-10T03:26:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:53 smithi170 ceph-mon[102832]: pgmap v8078: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T03:26:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:26:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:26:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14845.230412 secs 2023-09-10T03:26:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14843.900740 secs 2023-09-10T03:26:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:53 smithi082 ceph-mon[98694]: pgmap v8078: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T03:26:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:26:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:26:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14845.230412 secs 2023-09-10T03:26:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14845.941810 secs 2023-09-10T03:26:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14845.941810 secs 2023-09-10T03:26:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14845.941810 secs 2023-09-10T03:26:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:55 smithi098 ceph-mon[103124]: pgmap v8079: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T03:26:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:55 smithi170 ceph-mon[102832]: pgmap v8079: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T03:26:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:55 smithi082 ceph-mon[98694]: pgmap v8079: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T03:26:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:26:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:26:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:26:57.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:26:57.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:26:57.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:26:57.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:26:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:26:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:26:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:26:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:26:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:26:57.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:26:57.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:26:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:26:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:26:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:26:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:26:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:26:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:26:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:26:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:57 smithi098 ceph-mon[103124]: pgmap v8080: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 260 KiB/s wr, 17 op/s 2023-09-10T03:26:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:57 smithi170 ceph-mon[102832]: pgmap v8080: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 260 KiB/s wr, 17 op/s 2023-09-10T03:26:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:57 smithi082 ceph-mon[98694]: pgmap v8080: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 260 KiB/s wr, 17 op/s 2023-09-10T03:26:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14848.900776 secs 2023-09-10T03:26:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14850.230494 secs 2023-09-10T03:26:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14848.900776 secs 2023-09-10T03:26:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14850.230494 secs 2023-09-10T03:26:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14848.900776 secs 2023-09-10T03:26:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14850.230494 secs 2023-09-10T03:27:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:59 smithi170 ceph-mon[102832]: pgmap v8081: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 225 KiB/s wr, 11 op/s 2023-09-10T03:27:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:26:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14850.941873 secs 2023-09-10T03:27:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:59 smithi082 ceph-mon[98694]: pgmap v8081: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 225 KiB/s wr, 11 op/s 2023-09-10T03:27:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:26:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14850.941873 secs 2023-09-10T03:27:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:59 smithi098 ceph-mon[103124]: pgmap v8081: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 225 KiB/s wr, 11 op/s 2023-09-10T03:27:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:26:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14850.941873 secs 2023-09-10T03:27:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:01 smithi170 ceph-mon[102832]: pgmap v8082: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 269 KiB/s wr, 16 op/s 2023-09-10T03:27:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:01 smithi082 ceph-mon[98694]: pgmap v8082: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 269 KiB/s wr, 16 op/s 2023-09-10T03:27:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:01 smithi098 ceph-mon[103124]: pgmap v8082: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 269 KiB/s wr, 16 op/s 2023-09-10T03:27:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14853.900890 secs 2023-09-10T03:27:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:03 smithi170 ceph-mon[102832]: pgmap v8083: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 182 KiB/s wr, 11 op/s 2023-09-10T03:27:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14855.230554 secs 2023-09-10T03:27:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14853.900890 secs 2023-09-10T03:27:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:03 smithi082 ceph-mon[98694]: pgmap v8083: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 182 KiB/s wr, 11 op/s 2023-09-10T03:27:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14855.230554 secs 2023-09-10T03:27:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14853.900890 secs 2023-09-10T03:27:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:03 smithi098 ceph-mon[103124]: pgmap v8083: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 182 KiB/s wr, 11 op/s 2023-09-10T03:27:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14855.230554 secs 2023-09-10T03:27:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14855.941939 secs 2023-09-10T03:27:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14855.941939 secs 2023-09-10T03:27:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14855.941939 secs 2023-09-10T03:27:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:05 smithi170 ceph-mon[102832]: pgmap v8084: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 182 KiB/s wr, 11 op/s 2023-09-10T03:27:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:05 smithi082 ceph-mon[98694]: pgmap v8084: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 182 KiB/s wr, 11 op/s 2023-09-10T03:27:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:05 smithi098 ceph-mon[103124]: pgmap v8084: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 182 KiB/s wr, 11 op/s 2023-09-10T03:27:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:07 smithi098 ceph-mon[103124]: pgmap v8085: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 239 KiB/s wr, 30 op/s 2023-09-10T03:27:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:07 smithi170 ceph-mon[102832]: pgmap v8085: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 239 KiB/s wr, 30 op/s 2023-09-10T03:27:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:07 smithi082 ceph-mon[98694]: pgmap v8085: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 239 KiB/s wr, 30 op/s 2023-09-10T03:27:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14858.900950 secs 2023-09-10T03:27:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:08 smithi098 ceph-mon[103124]: osdmap e573: 12 total, 12 up, 12 in 2023-09-10T03:27:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14860.230616 secs 2023-09-10T03:27:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14858.900950 secs 2023-09-10T03:27:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:08 smithi170 ceph-mon[102832]: osdmap e573: 12 total, 12 up, 12 in 2023-09-10T03:27:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14860.230616 secs 2023-09-10T03:27:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14858.900950 secs 2023-09-10T03:27:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:08 smithi082 ceph-mon[98694]: osdmap e573: 12 total, 12 up, 12 in 2023-09-10T03:27:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14860.230616 secs 2023-09-10T03:27:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:09 smithi098 ceph-mon[103124]: pgmap v8087: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 121 KiB/s wr, 30 op/s 2023-09-10T03:27:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14860.942006 secs 2023-09-10T03:27:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:09 smithi170 ceph-mon[102832]: pgmap v8087: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 121 KiB/s wr, 30 op/s 2023-09-10T03:27:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14860.942006 secs 2023-09-10T03:27:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:09 smithi082 ceph-mon[98694]: pgmap v8087: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 121 KiB/s wr, 30 op/s 2023-09-10T03:27:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14860.942006 secs 2023-09-10T03:27:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:11 smithi098 ceph-mon[103124]: pgmap v8088: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 126 KiB/s wr, 29 op/s 2023-09-10T03:27:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:11 smithi170 ceph-mon[102832]: pgmap v8088: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 126 KiB/s wr, 29 op/s 2023-09-10T03:27:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:11 smithi082 ceph-mon[98694]: pgmap v8088: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 126 KiB/s wr, 29 op/s 2023-09-10T03:27:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14863.901066 secs 2023-09-10T03:27:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:13 smithi170 ceph-mon[102832]: osdmap e574: 12 total, 12 up, 12 in 2023-09-10T03:27:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:13 smithi170 ceph-mon[102832]: pgmap v8090: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 157 KiB/s wr, 36 op/s 2023-09-10T03:27:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14865.230673 secs 2023-09-10T03:27:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14863.901066 secs 2023-09-10T03:27:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:13 smithi082 ceph-mon[98694]: osdmap e574: 12 total, 12 up, 12 in 2023-09-10T03:27:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:13 smithi082 ceph-mon[98694]: pgmap v8090: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 157 KiB/s wr, 36 op/s 2023-09-10T03:27:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14865.230673 secs 2023-09-10T03:27:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14863.901066 secs 2023-09-10T03:27:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:13 smithi098 ceph-mon[103124]: osdmap e574: 12 total, 12 up, 12 in 2023-09-10T03:27:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:13 smithi098 ceph-mon[103124]: pgmap v8090: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 157 KiB/s wr, 36 op/s 2023-09-10T03:27:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14865.230673 secs 2023-09-10T03:27:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14865.942094 secs 2023-09-10T03:27:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14865.942094 secs 2023-09-10T03:27:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14865.942094 secs 2023-09-10T03:27:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:15 smithi170 ceph-mon[102832]: pgmap v8091: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 73 KiB/s wr, 7 op/s 2023-09-10T03:27:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:15 smithi082 ceph-mon[98694]: pgmap v8091: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 73 KiB/s wr, 7 op/s 2023-09-10T03:27:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:15 smithi098 ceph-mon[103124]: pgmap v8091: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 73 KiB/s wr, 7 op/s 2023-09-10T03:27:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:17 smithi170 ceph-mon[102832]: pgmap v8092: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 13 op/s 2023-09-10T03:27:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:17 smithi082 ceph-mon[98694]: pgmap v8092: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 13 op/s 2023-09-10T03:27:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:17 smithi098 ceph-mon[103124]: pgmap v8092: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 13 op/s 2023-09-10T03:27:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14868.901160 secs 2023-09-10T03:27:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14870.230755 secs 2023-09-10T03:27:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14868.901160 secs 2023-09-10T03:27:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14870.230755 secs 2023-09-10T03:27:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14868.901160 secs 2023-09-10T03:27:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14870.230755 secs 2023-09-10T03:27:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:19 smithi082 ceph-mon[98694]: pgmap v8093: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T03:27:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14870.942160 secs 2023-09-10T03:27:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:19 smithi098 ceph-mon[103124]: pgmap v8093: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T03:27:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14870.942160 secs 2023-09-10T03:27:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:19 smithi170 ceph-mon[102832]: pgmap v8093: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T03:27:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14870.942160 secs 2023-09-10T03:27:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:21 smithi082 ceph-mon[98694]: pgmap v8094: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T03:27:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:21 smithi098 ceph-mon[103124]: pgmap v8094: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T03:27:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:21 smithi170 ceph-mon[102832]: pgmap v8094: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T03:27:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:23 smithi170 ceph-mon[102832]: pgmap v8095: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T03:27:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:27:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14875.230814 secs 2023-09-10T03:27:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:23 smithi082 ceph-mon[98694]: pgmap v8095: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T03:27:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:27:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14875.230814 secs 2023-09-10T03:27:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:23 smithi098 ceph-mon[103124]: pgmap v8095: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 11 op/s 2023-09-10T03:27:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:27:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14875.230814 secs 2023-09-10T03:27:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:27:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:27:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:27:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14875.942225 secs 2023-09-10T03:27:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:27:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:27:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:27:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14875.942225 secs 2023-09-10T03:27:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:27:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:27:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:27:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14875.942225 secs 2023-09-10T03:27:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:25 smithi170 ceph-mon[102832]: pgmap v8096: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T03:27:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:25 smithi082 ceph-mon[98694]: pgmap v8096: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T03:27:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:25 smithi098 ceph-mon[103124]: pgmap v8096: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T03:27:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:27 smithi098 ceph-mon[103124]: pgmap v8097: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 122 KiB/s wr, 14 op/s 2023-09-10T03:27:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:27 smithi170 ceph-mon[102832]: pgmap v8097: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 122 KiB/s wr, 14 op/s 2023-09-10T03:27:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:27 smithi082 ceph-mon[98694]: pgmap v8097: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 122 KiB/s wr, 14 op/s 2023-09-10T03:27:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14878.901330 secs 2023-09-10T03:27:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14880.230872 secs 2023-09-10T03:27:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14878.901330 secs 2023-09-10T03:27:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14880.230872 secs 2023-09-10T03:27:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14878.901330 secs 2023-09-10T03:27:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14880.230872 secs 2023-09-10T03:27:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:29 smithi098 ceph-mon[103124]: pgmap v8098: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T03:27:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14880.942290 secs 2023-09-10T03:27:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:29 smithi170 ceph-mon[102832]: pgmap v8098: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T03:27:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14880.942290 secs 2023-09-10T03:27:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:29 smithi082 ceph-mon[98694]: pgmap v8098: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T03:27:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14880.942290 secs 2023-09-10T03:27:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:31 smithi098 ceph-mon[103124]: pgmap v8099: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 184 KiB/s wr, 16 op/s 2023-09-10T03:27:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:31 smithi170 ceph-mon[102832]: pgmap v8099: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 184 KiB/s wr, 16 op/s 2023-09-10T03:27:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:31 smithi082 ceph-mon[98694]: pgmap v8099: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 184 KiB/s wr, 16 op/s 2023-09-10T03:27:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:32 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:27:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:27:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:27:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:32 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:27:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:27:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:27:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:32 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:27:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:27:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:27:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14883.901391 secs 2023-09-10T03:27:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:33 smithi098 ceph-mon[103124]: pgmap v8100: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T03:27:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:27:34.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:27:34.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14885.230957 secs 2023-09-10T03:27:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14883.901391 secs 2023-09-10T03:27:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:33 smithi170 ceph-mon[102832]: pgmap v8100: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T03:27:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:27:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:27:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14885.230957 secs 2023-09-10T03:27:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14883.901391 secs 2023-09-10T03:27:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:33 smithi082 ceph-mon[98694]: pgmap v8100: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T03:27:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:27:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:27:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14885.230957 secs 2023-09-10T03:27:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14885.942359 secs 2023-09-10T03:27:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14885.942359 secs 2023-09-10T03:27:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14885.942359 secs 2023-09-10T03:27:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:35 smithi170 ceph-mon[102832]: pgmap v8101: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T03:27:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:35 smithi082 ceph-mon[98694]: pgmap v8101: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T03:27:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:35 smithi098 ceph-mon[103124]: pgmap v8101: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T03:27:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:37 smithi170 ceph-mon[102832]: pgmap v8102: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 306 KiB/s wr, 17 op/s 2023-09-10T03:27:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:37 smithi082 ceph-mon[98694]: pgmap v8102: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 306 KiB/s wr, 17 op/s 2023-09-10T03:27:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:37 smithi098 ceph-mon[103124]: pgmap v8102: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 306 KiB/s wr, 17 op/s 2023-09-10T03:27:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14890.231017 secs 2023-09-10T03:27:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14890.231017 secs 2023-09-10T03:27:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14890.231017 secs 2023-09-10T03:27:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:39 smithi170 ceph-mon[102832]: pgmap v8103: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 245 KiB/s wr, 12 op/s 2023-09-10T03:27:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14890.942424 secs 2023-09-10T03:27:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:27:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:39 smithi082 ceph-mon[98694]: pgmap v8103: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 245 KiB/s wr, 12 op/s 2023-09-10T03:27:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14890.942424 secs 2023-09-10T03:27:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:27:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:39 smithi098 ceph-mon[103124]: pgmap v8103: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 245 KiB/s wr, 12 op/s 2023-09-10T03:27:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14890.942424 secs 2023-09-10T03:27:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:27:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:27:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:27:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:27:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:27:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:27:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:27:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:27:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:27:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:27:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:41 smithi170 ceph-mon[102832]: pgmap v8104: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 404 KiB/s wr, 18 op/s 2023-09-10T03:27:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:41 smithi082 ceph-mon[98694]: pgmap v8104: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 404 KiB/s wr, 18 op/s 2023-09-10T03:27:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:41 smithi098 ceph-mon[103124]: pgmap v8104: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 404 KiB/s wr, 18 op/s 2023-09-10T03:27:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:43 smithi170 ceph-mon[102832]: pgmap v8105: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 342 KiB/s wr, 12 op/s 2023-09-10T03:27:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14895.231099 secs 2023-09-10T03:27:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:43 smithi082 ceph-mon[98694]: pgmap v8105: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 342 KiB/s wr, 12 op/s 2023-09-10T03:27:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14895.231099 secs 2023-09-10T03:27:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:43 smithi098 ceph-mon[103124]: pgmap v8105: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 342 KiB/s wr, 12 op/s 2023-09-10T03:27:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14895.231099 secs 2023-09-10T03:27:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:27:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:27:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14895.942487 secs 2023-09-10T03:27:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:27:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:27:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14895.942487 secs 2023-09-10T03:27:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:27:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:27:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14895.942487 secs 2023-09-10T03:27:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:45 smithi170 ceph-mon[102832]: pgmap v8106: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 342 KiB/s wr, 12 op/s 2023-09-10T03:27:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:45 smithi082 ceph-mon[98694]: pgmap v8106: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 342 KiB/s wr, 12 op/s 2023-09-10T03:27:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:45 smithi098 ceph-mon[103124]: pgmap v8106: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 342 KiB/s wr, 12 op/s 2023-09-10T03:27:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:47 smithi098 ceph-mon[103124]: pgmap v8107: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 394 KiB/s wr, 18 op/s 2023-09-10T03:27:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:47 smithi170 ceph-mon[102832]: pgmap v8107: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 394 KiB/s wr, 18 op/s 2023-09-10T03:27:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:47 smithi082 ceph-mon[98694]: pgmap v8107: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 394 KiB/s wr, 18 op/s 2023-09-10T03:27:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14898.901649 secs 2023-09-10T03:27:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14900.231157 secs 2023-09-10T03:27:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14898.901649 secs 2023-09-10T03:27:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14900.231157 secs 2023-09-10T03:27:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14898.901649 secs 2023-09-10T03:27:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14900.231157 secs 2023-09-10T03:27:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:49 smithi098 ceph-mon[103124]: pgmap v8108: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 210 KiB/s wr, 12 op/s 2023-09-10T03:27:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14900.942554 secs 2023-09-10T03:27:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:27:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:27:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:49 smithi170 ceph-mon[102832]: pgmap v8108: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 210 KiB/s wr, 12 op/s 2023-09-10T03:27:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14900.942554 secs 2023-09-10T03:27:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:27:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:27:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:49 smithi082 ceph-mon[98694]: pgmap v8108: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 210 KiB/s wr, 12 op/s 2023-09-10T03:27:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14900.942554 secs 2023-09-10T03:27:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:27:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:27:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:51 smithi098 ceph-mon[103124]: pgmap v8109: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 370 KiB/s wr, 17 op/s 2023-09-10T03:27:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:51 smithi170 ceph-mon[102832]: pgmap v8109: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 370 KiB/s wr, 17 op/s 2023-09-10T03:27:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:51 smithi082 ceph-mon[98694]: pgmap v8109: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 370 KiB/s wr, 17 op/s 2023-09-10T03:27:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:27:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:27:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:27:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:27:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:27:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:27:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:27:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:27:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:27:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14903.901757 secs 2023-09-10T03:27:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:53 smithi098 ceph-mon[103124]: pgmap v8110: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 212 KiB/s wr, 11 op/s 2023-09-10T03:27:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14905.231216 secs 2023-09-10T03:27:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14903.901757 secs 2023-09-10T03:27:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:53 smithi170 ceph-mon[102832]: pgmap v8110: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 212 KiB/s wr, 11 op/s 2023-09-10T03:27:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14905.231216 secs 2023-09-10T03:27:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14903.901757 secs 2023-09-10T03:27:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:53 smithi082 ceph-mon[98694]: pgmap v8110: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 212 KiB/s wr, 11 op/s 2023-09-10T03:27:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14905.231216 secs 2023-09-10T03:27:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14905.942618 secs 2023-09-10T03:27:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14905.942618 secs 2023-09-10T03:27:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14905.942618 secs 2023-09-10T03:27:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:55 smithi170 ceph-mon[102832]: pgmap v8111: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 212 KiB/s wr, 11 op/s 2023-09-10T03:27:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:55 smithi082 ceph-mon[98694]: pgmap v8111: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 212 KiB/s wr, 11 op/s 2023-09-10T03:27:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:55 smithi098 ceph-mon[103124]: pgmap v8111: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 212 KiB/s wr, 11 op/s 2023-09-10T03:27:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:57 smithi170 ceph-mon[102832]: pgmap v8112: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 318 KiB/s wr, 16 op/s 2023-09-10T03:27:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:27:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:27:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:27:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:27:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:57 smithi082 ceph-mon[98694]: pgmap v8112: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 318 KiB/s wr, 16 op/s 2023-09-10T03:27:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:27:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:27:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:27:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:27:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:57 smithi098 ceph-mon[103124]: pgmap v8112: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 318 KiB/s wr, 16 op/s 2023-09-10T03:27:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:27:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:27:58.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:27:58.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:27:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14908.901806 secs 2023-09-10T03:27:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14910.231278 secs 2023-09-10T03:27:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14908.901806 secs 2023-09-10T03:27:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14910.231278 secs 2023-09-10T03:27:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14908.901806 secs 2023-09-10T03:27:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14910.231278 secs 2023-09-10T03:28:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:59 smithi170 ceph-mon[102832]: pgmap v8113: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 266 KiB/s wr, 10 op/s 2023-09-10T03:28:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:27:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14910.942685 secs 2023-09-10T03:28:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:59 smithi082 ceph-mon[98694]: pgmap v8113: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 266 KiB/s wr, 10 op/s 2023-09-10T03:28:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:27:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14910.942685 secs 2023-09-10T03:28:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:59 smithi098 ceph-mon[103124]: pgmap v8113: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 266 KiB/s wr, 10 op/s 2023-09-10T03:28:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:27:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14910.942685 secs 2023-09-10T03:28:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:01 smithi082 ceph-mon[98694]: pgmap v8114: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 310 KiB/s wr, 16 op/s 2023-09-10T03:28:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:01 smithi170 ceph-mon[102832]: pgmap v8114: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 310 KiB/s wr, 16 op/s 2023-09-10T03:28:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:01 smithi098 ceph-mon[103124]: pgmap v8114: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 310 KiB/s wr, 16 op/s 2023-09-10T03:28:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14913.901868 secs 2023-09-10T03:28:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:03 smithi170 ceph-mon[102832]: pgmap v8115: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 10 op/s 2023-09-10T03:28:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14915.231339 secs 2023-09-10T03:28:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14913.901868 secs 2023-09-10T03:28:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:03 smithi082 ceph-mon[98694]: pgmap v8115: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 10 op/s 2023-09-10T03:28:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14915.231339 secs 2023-09-10T03:28:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14913.901868 secs 2023-09-10T03:28:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:03 smithi098 ceph-mon[103124]: pgmap v8115: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 10 op/s 2023-09-10T03:28:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14915.231339 secs 2023-09-10T03:28:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:28:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:28:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14915.942748 secs 2023-09-10T03:28:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:28:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:28:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14915.942748 secs 2023-09-10T03:28:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:28:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:28:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14915.942748 secs 2023-09-10T03:28:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:05 smithi082 ceph-mon[98694]: pgmap v8116: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 10 op/s 2023-09-10T03:28:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:05 smithi098 ceph-mon[103124]: pgmap v8116: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 10 op/s 2023-09-10T03:28:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:05 smithi170 ceph-mon[102832]: pgmap v8116: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 10 op/s 2023-09-10T03:28:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:07 smithi098 ceph-mon[103124]: pgmap v8117: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 207 KiB/s wr, 30 op/s 2023-09-10T03:28:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:07 smithi170 ceph-mon[102832]: pgmap v8117: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 207 KiB/s wr, 30 op/s 2023-09-10T03:28:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:07 smithi082 ceph-mon[98694]: pgmap v8117: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 207 KiB/s wr, 30 op/s 2023-09-10T03:28:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14918.901977 secs 2023-09-10T03:28:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:08 smithi098 ceph-mon[103124]: osdmap e575: 12 total, 12 up, 12 in 2023-09-10T03:28:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14920.231400 secs 2023-09-10T03:28:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14918.901977 secs 2023-09-10T03:28:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:08 smithi170 ceph-mon[102832]: osdmap e575: 12 total, 12 up, 12 in 2023-09-10T03:28:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14920.231400 secs 2023-09-10T03:28:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14918.901977 secs 2023-09-10T03:28:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:08 smithi082 ceph-mon[98694]: osdmap e575: 12 total, 12 up, 12 in 2023-09-10T03:28:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14920.231400 secs 2023-09-10T03:28:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:09 smithi098 ceph-mon[103124]: pgmap v8119: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 122 KiB/s wr, 30 op/s 2023-09-10T03:28:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14920.942812 secs 2023-09-10T03:28:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:09 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:28:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:09 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:28:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:09 smithi170 ceph-mon[102832]: pgmap v8119: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 122 KiB/s wr, 30 op/s 2023-09-10T03:28:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14920.942812 secs 2023-09-10T03:28:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:09 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:28:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:09 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:28:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:09 smithi082 ceph-mon[98694]: pgmap v8119: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 122 KiB/s wr, 30 op/s 2023-09-10T03:28:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14920.942812 secs 2023-09-10T03:28:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:09 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:28:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:09 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:28:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:11 smithi170 ceph-mon[102832]: pgmap v8120: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-10T03:28:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:11 smithi082 ceph-mon[98694]: pgmap v8120: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-10T03:28:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:11 smithi098 ceph-mon[103124]: pgmap v8120: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-10T03:28:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14923.902099 secs 2023-09-10T03:28:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:13 smithi170 ceph-mon[102832]: osdmap e576: 12 total, 12 up, 12 in 2023-09-10T03:28:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:13 smithi170 ceph-mon[102832]: pgmap v8122: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 87 KiB/s wr, 37 op/s 2023-09-10T03:28:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14925.231494 secs 2023-09-10T03:28:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14923.902099 secs 2023-09-10T03:28:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:13 smithi082 ceph-mon[98694]: osdmap e576: 12 total, 12 up, 12 in 2023-09-10T03:28:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:13 smithi082 ceph-mon[98694]: pgmap v8122: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 87 KiB/s wr, 37 op/s 2023-09-10T03:28:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14925.231494 secs 2023-09-10T03:28:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14923.902099 secs 2023-09-10T03:28:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:13 smithi098 ceph-mon[103124]: osdmap e576: 12 total, 12 up, 12 in 2023-09-10T03:28:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:13 smithi098 ceph-mon[103124]: pgmap v8122: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 87 KiB/s wr, 37 op/s 2023-09-10T03:28:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14925.231494 secs 2023-09-10T03:28:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14925.942875 secs 2023-09-10T03:28:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14925.942875 secs 2023-09-10T03:28:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14925.942875 secs 2023-09-10T03:28:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:15 smithi082 ceph-mon[98694]: pgmap v8123: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:28:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:15 smithi098 ceph-mon[103124]: pgmap v8123: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:28:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:15 smithi170 ceph-mon[102832]: pgmap v8123: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:28:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:17 smithi082 ceph-mon[98694]: pgmap v8124: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 13 op/s 2023-09-10T03:28:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:17 smithi098 ceph-mon[103124]: pgmap v8124: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 13 op/s 2023-09-10T03:28:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:17 smithi170 ceph-mon[102832]: pgmap v8124: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 13 op/s 2023-09-10T03:28:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14928.902213 secs 2023-09-10T03:28:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14930.231552 secs 2023-09-10T03:28:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14928.902213 secs 2023-09-10T03:28:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14930.231552 secs 2023-09-10T03:28:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14928.902213 secs 2023-09-10T03:28:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14930.231552 secs 2023-09-10T03:28:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14930.942938 secs 2023-09-10T03:28:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:19 smithi082 ceph-mon[98694]: pgmap v8125: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T03:28:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14930.942938 secs 2023-09-10T03:28:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:19 smithi098 ceph-mon[103124]: pgmap v8125: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T03:28:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14930.942938 secs 2023-09-10T03:28:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:19 smithi170 ceph-mon[102832]: pgmap v8125: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T03:28:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:21 smithi082 ceph-mon[98694]: pgmap v8126: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T03:28:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:21 smithi098 ceph-mon[103124]: pgmap v8126: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T03:28:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:21 smithi170 ceph-mon[102832]: pgmap v8126: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T03:28:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:23 smithi082 ceph-mon[98694]: pgmap v8127: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T03:28:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:28:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14935.231610 secs 2023-09-10T03:28:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:23 smithi098 ceph-mon[103124]: pgmap v8127: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T03:28:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:28:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14935.231610 secs 2023-09-10T03:28:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:23 smithi170 ceph-mon[102832]: pgmap v8127: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T03:28:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:28:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14935.231610 secs 2023-09-10T03:28:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:28:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14935.943006 secs 2023-09-10T03:28:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:28:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:28:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:28:25.049 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14935.943006 secs 2023-09-10T03:28:25.049 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:28:25.049 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:28:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:28:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14935.943006 secs 2023-09-10T03:28:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:28:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:28:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:25 smithi082 ceph-mon[98694]: pgmap v8128: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 49 KiB/s wr, 9 op/s 2023-09-10T03:28:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:25 smithi098 ceph-mon[103124]: pgmap v8128: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 49 KiB/s wr, 9 op/s 2023-09-10T03:28:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:25 smithi170 ceph-mon[102832]: pgmap v8128: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 49 KiB/s wr, 9 op/s 2023-09-10T03:28:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:27 smithi098 ceph-mon[103124]: pgmap v8129: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 122 KiB/s wr, 14 op/s 2023-09-10T03:28:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:27 smithi170 ceph-mon[102832]: pgmap v8129: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 122 KiB/s wr, 14 op/s 2023-09-10T03:28:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:27 smithi082 ceph-mon[98694]: pgmap v8129: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 122 KiB/s wr, 14 op/s 2023-09-10T03:28:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14940.231670 secs 2023-09-10T03:28:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14940.231670 secs 2023-09-10T03:28:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14940.231670 secs 2023-09-10T03:28:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14940.943078 secs 2023-09-10T03:28:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:29 smithi098 ceph-mon[103124]: pgmap v8130: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 9 op/s 2023-09-10T03:28:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14940.943078 secs 2023-09-10T03:28:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:29 smithi170 ceph-mon[102832]: pgmap v8130: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 9 op/s 2023-09-10T03:28:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14940.943078 secs 2023-09-10T03:28:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:29 smithi082 ceph-mon[98694]: pgmap v8130: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 9 op/s 2023-09-10T03:28:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:31 smithi170 ceph-mon[102832]: pgmap v8131: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 15 op/s 2023-09-10T03:28:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:31 smithi082 ceph-mon[98694]: pgmap v8131: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 15 op/s 2023-09-10T03:28:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:31 smithi098 ceph-mon[103124]: pgmap v8131: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 15 op/s 2023-09-10T03:28:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14943.902443 secs 2023-09-10T03:28:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:33 smithi082 ceph-mon[98694]: pgmap v8132: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 10 op/s 2023-09-10T03:28:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14945.231727 secs 2023-09-10T03:28:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14943.902443 secs 2023-09-10T03:28:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:33 smithi098 ceph-mon[103124]: pgmap v8132: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 10 op/s 2023-09-10T03:28:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14945.231727 secs 2023-09-10T03:28:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14943.902443 secs 2023-09-10T03:28:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:33 smithi170 ceph-mon[102832]: pgmap v8132: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 10 op/s 2023-09-10T03:28:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14945.231727 secs 2023-09-10T03:28:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14945.943144 secs 2023-09-10T03:28:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14945.943144 secs 2023-09-10T03:28:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14945.943144 secs 2023-09-10T03:28:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:35 smithi082 ceph-mon[98694]: pgmap v8133: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 10 op/s 2023-09-10T03:28:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:35 smithi098 ceph-mon[103124]: pgmap v8133: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 10 op/s 2023-09-10T03:28:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:35 smithi170 ceph-mon[102832]: pgmap v8133: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 10 op/s 2023-09-10T03:28:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:36 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:28:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:28:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:28:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:36 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:28:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:28:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:28:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:36 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:28:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:28:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:28:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:37 smithi082 ceph-mon[98694]: pgmap v8134: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 223 KiB/s wr, 16 op/s 2023-09-10T03:28:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:28:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:28:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:37 smithi098 ceph-mon[103124]: pgmap v8134: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 223 KiB/s wr, 16 op/s 2023-09-10T03:28:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:28:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:28:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:37 smithi170 ceph-mon[102832]: pgmap v8134: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 223 KiB/s wr, 16 op/s 2023-09-10T03:28:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:28:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:28:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14950.231811 secs 2023-09-10T03:28:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14950.231811 secs 2023-09-10T03:28:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14950.231811 secs 2023-09-10T03:28:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14950.943209 secs 2023-09-10T03:28:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:39 smithi082 ceph-mon[98694]: pgmap v8135: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T03:28:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:28:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14950.943209 secs 2023-09-10T03:28:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:39 smithi098 ceph-mon[103124]: pgmap v8135: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T03:28:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:28:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14950.943209 secs 2023-09-10T03:28:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:39 smithi170 ceph-mon[102832]: pgmap v8135: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T03:28:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:28:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:28:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:28:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:28:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:28:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:28:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:28:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:41 smithi098 ceph-mon[103124]: pgmap v8136: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 332 KiB/s wr, 18 op/s 2023-09-10T03:28:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:41 smithi170 ceph-mon[102832]: pgmap v8136: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 332 KiB/s wr, 18 op/s 2023-09-10T03:28:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:41 smithi082 ceph-mon[98694]: pgmap v8136: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 332 KiB/s wr, 18 op/s 2023-09-10T03:28:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:42 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:28:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:42 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:28:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:42 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:28:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:43 smithi098 ceph-mon[103124]: pgmap v8137: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 226 KiB/s wr, 12 op/s 2023-09-10T03:28:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14955.231869 secs 2023-09-10T03:28:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:43 smithi170 ceph-mon[102832]: pgmap v8137: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 226 KiB/s wr, 12 op/s 2023-09-10T03:28:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14955.231869 secs 2023-09-10T03:28:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:43 smithi082 ceph-mon[98694]: pgmap v8137: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 226 KiB/s wr, 12 op/s 2023-09-10T03:28:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14955.231869 secs 2023-09-10T03:28:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14955.943277 secs 2023-09-10T03:28:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14955.943277 secs 2023-09-10T03:28:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14955.943277 secs 2023-09-10T03:28:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:45 smithi098 ceph-mon[103124]: pgmap v8138: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 226 KiB/s wr, 12 op/s 2023-09-10T03:28:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:45 smithi170 ceph-mon[102832]: pgmap v8138: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 226 KiB/s wr, 12 op/s 2023-09-10T03:28:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:45 smithi082 ceph-mon[98694]: pgmap v8138: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 226 KiB/s wr, 12 op/s 2023-09-10T03:28:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:47 smithi098 ceph-mon[103124]: pgmap v8139: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 353 KiB/s wr, 17 op/s 2023-09-10T03:28:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:47 smithi170 ceph-mon[102832]: pgmap v8139: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 353 KiB/s wr, 17 op/s 2023-09-10T03:28:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:47 smithi082 ceph-mon[98694]: pgmap v8139: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 353 KiB/s wr, 17 op/s 2023-09-10T03:28:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14960.231948 secs 2023-09-10T03:28:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14960.231948 secs 2023-09-10T03:28:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14960.231948 secs 2023-09-10T03:28:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14960.943340 secs 2023-09-10T03:28:50.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:49 smithi098 ceph-mon[103124]: pgmap v8140: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 309 KiB/s wr, 11 op/s 2023-09-10T03:28:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14960.943340 secs 2023-09-10T03:28:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:49 smithi170 ceph-mon[102832]: pgmap v8140: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 309 KiB/s wr, 11 op/s 2023-09-10T03:28:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14960.943340 secs 2023-09-10T03:28:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:49 smithi082 ceph-mon[98694]: pgmap v8140: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 309 KiB/s wr, 11 op/s 2023-09-10T03:28:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:51 smithi098 ceph-mon[103124]: pgmap v8141: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 393 KiB/s wr, 17 op/s 2023-09-10T03:28:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:51 smithi170 ceph-mon[102832]: pgmap v8141: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 393 KiB/s wr, 17 op/s 2023-09-10T03:28:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:51 smithi082 ceph-mon[98694]: pgmap v8141: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 393 KiB/s wr, 17 op/s 2023-09-10T03:28:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14963.902680 secs 2023-09-10T03:28:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:53 smithi082 ceph-mon[98694]: pgmap v8142: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 211 KiB/s wr, 10 op/s 2023-09-10T03:28:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14965.232016 secs 2023-09-10T03:28:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14963.902680 secs 2023-09-10T03:28:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:53 smithi098 ceph-mon[103124]: pgmap v8142: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 211 KiB/s wr, 10 op/s 2023-09-10T03:28:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14965.232016 secs 2023-09-10T03:28:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14963.902680 secs 2023-09-10T03:28:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:53 smithi170 ceph-mon[102832]: pgmap v8142: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 211 KiB/s wr, 10 op/s 2023-09-10T03:28:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14965.232016 secs 2023-09-10T03:28:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14965.943431 secs 2023-09-10T03:28:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14965.943431 secs 2023-09-10T03:28:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14965.943431 secs 2023-09-10T03:28:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:55 smithi082 ceph-mon[98694]: pgmap v8143: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 211 KiB/s wr, 10 op/s 2023-09-10T03:28:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:55 smithi098 ceph-mon[103124]: pgmap v8143: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 211 KiB/s wr, 10 op/s 2023-09-10T03:28:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:55 smithi170 ceph-mon[102832]: pgmap v8143: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 211 KiB/s wr, 10 op/s 2023-09-10T03:28:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:28:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:28:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:28:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:28:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:28:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:28:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:28:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:28:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:28:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:57 smithi082 ceph-mon[98694]: pgmap v8144: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 250 KiB/s wr, 16 op/s 2023-09-10T03:28:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:28:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:28:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:28:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:57 smithi098 ceph-mon[103124]: pgmap v8144: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 250 KiB/s wr, 16 op/s 2023-09-10T03:28:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:28:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:28:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:28:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:57 smithi170 ceph-mon[102832]: pgmap v8144: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 250 KiB/s wr, 16 op/s 2023-09-10T03:28:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:28:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:28:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:28:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14968.902742 secs 2023-09-10T03:28:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14970.232074 secs 2023-09-10T03:28:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:28:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:28:58.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:28:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14968.902742 secs 2023-09-10T03:28:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14970.232074 secs 2023-09-10T03:28:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:28:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:28:59.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:28:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14968.902742 secs 2023-09-10T03:28:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14970.232074 secs 2023-09-10T03:28:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:28:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:28:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:28:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14970.943491 secs 2023-09-10T03:28:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:28:59 smithi082 ceph-mon[98694]: pgmap v8145: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T03:29:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14970.943491 secs 2023-09-10T03:29:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:28:59 smithi098 ceph-mon[103124]: pgmap v8145: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T03:29:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14970.943491 secs 2023-09-10T03:29:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:28:59 smithi170 ceph-mon[102832]: pgmap v8145: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T03:29:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:29:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:29:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:29:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:29:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:29:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:29:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:01 smithi098 ceph-mon[103124]: pgmap v8146: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 232 KiB/s wr, 30 op/s 2023-09-10T03:29:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:01 smithi170 ceph-mon[102832]: pgmap v8146: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 232 KiB/s wr, 30 op/s 2023-09-10T03:29:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:01 smithi082 ceph-mon[98694]: pgmap v8146: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 232 KiB/s wr, 30 op/s 2023-09-10T03:29:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14973.902823 secs 2023-09-10T03:29:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:03 smithi082 ceph-mon[98694]: pgmap v8147: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 148 KiB/s wr, 25 op/s 2023-09-10T03:29:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14975.232137 secs 2023-09-10T03:29:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14973.902823 secs 2023-09-10T03:29:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:03 smithi098 ceph-mon[103124]: pgmap v8147: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 148 KiB/s wr, 25 op/s 2023-09-10T03:29:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14975.232137 secs 2023-09-10T03:29:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14973.902823 secs 2023-09-10T03:29:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:03 smithi170 ceph-mon[102832]: pgmap v8147: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 148 KiB/s wr, 25 op/s 2023-09-10T03:29:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14975.232137 secs 2023-09-10T03:29:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14975.943557 secs 2023-09-10T03:29:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14975.943557 secs 2023-09-10T03:29:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14975.943557 secs 2023-09-10T03:29:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:05 smithi082 ceph-mon[98694]: pgmap v8148: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 148 KiB/s wr, 25 op/s 2023-09-10T03:29:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:05 smithi098 ceph-mon[103124]: pgmap v8148: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 148 KiB/s wr, 25 op/s 2023-09-10T03:29:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:05 smithi170 ceph-mon[102832]: pgmap v8148: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 148 KiB/s wr, 25 op/s 2023-09-10T03:29:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:07 smithi170 ceph-mon[102832]: pgmap v8149: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 192 KiB/s wr, 31 op/s 2023-09-10T03:29:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:07 smithi082 ceph-mon[98694]: pgmap v8149: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 192 KiB/s wr, 31 op/s 2023-09-10T03:29:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:07 smithi098 ceph-mon[103124]: pgmap v8149: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 192 KiB/s wr, 31 op/s 2023-09-10T03:29:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14978.902880 secs 2023-09-10T03:29:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:08 smithi170 ceph-mon[102832]: osdmap e577: 12 total, 12 up, 12 in 2023-09-10T03:29:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14980.232194 secs 2023-09-10T03:29:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14978.902880 secs 2023-09-10T03:29:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:08 smithi082 ceph-mon[98694]: osdmap e577: 12 total, 12 up, 12 in 2023-09-10T03:29:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14980.232194 secs 2023-09-10T03:29:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14978.902880 secs 2023-09-10T03:29:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:08 smithi098 ceph-mon[103124]: osdmap e577: 12 total, 12 up, 12 in 2023-09-10T03:29:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14980.232194 secs 2023-09-10T03:29:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14980.943622 secs 2023-09-10T03:29:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:09 smithi170 ceph-mon[102832]: pgmap v8151: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 184 KiB/s wr, 30 op/s 2023-09-10T03:29:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14980.943622 secs 2023-09-10T03:29:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:09 smithi082 ceph-mon[98694]: pgmap v8151: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 184 KiB/s wr, 30 op/s 2023-09-10T03:29:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14980.943622 secs 2023-09-10T03:29:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:09 smithi098 ceph-mon[103124]: pgmap v8151: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 184 KiB/s wr, 30 op/s 2023-09-10T03:29:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:11 smithi170 ceph-mon[102832]: pgmap v8152: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 53 KiB/s wr, 13 op/s 2023-09-10T03:29:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:11 smithi082 ceph-mon[98694]: pgmap v8152: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 53 KiB/s wr, 13 op/s 2023-09-10T03:29:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:11 smithi098 ceph-mon[103124]: pgmap v8152: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 53 KiB/s wr, 13 op/s 2023-09-10T03:29:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14983.902961 secs 2023-09-10T03:29:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:13 smithi082 ceph-mon[98694]: osdmap e578: 12 total, 12 up, 12 in 2023-09-10T03:29:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:13 smithi082 ceph-mon[98694]: pgmap v8154: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 67 KiB/s wr, 16 op/s 2023-09-10T03:29:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14985.232276 secs 2023-09-10T03:29:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14983.902961 secs 2023-09-10T03:29:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:13 smithi098 ceph-mon[103124]: osdmap e578: 12 total, 12 up, 12 in 2023-09-10T03:29:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:13 smithi098 ceph-mon[103124]: pgmap v8154: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 67 KiB/s wr, 16 op/s 2023-09-10T03:29:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14985.232276 secs 2023-09-10T03:29:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14983.902961 secs 2023-09-10T03:29:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:13 smithi170 ceph-mon[102832]: osdmap e578: 12 total, 12 up, 12 in 2023-09-10T03:29:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:13 smithi170 ceph-mon[102832]: pgmap v8154: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 67 KiB/s wr, 16 op/s 2023-09-10T03:29:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14985.232276 secs 2023-09-10T03:29:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14985.943688 secs 2023-09-10T03:29:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14985.943688 secs 2023-09-10T03:29:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14985.943688 secs 2023-09-10T03:29:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:15 smithi098 ceph-mon[103124]: pgmap v8155: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:29:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:15 smithi170 ceph-mon[102832]: pgmap v8155: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:29:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:15 smithi082 ceph-mon[98694]: pgmap v8155: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:29:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:17 smithi098 ceph-mon[103124]: pgmap v8156: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 13 op/s 2023-09-10T03:29:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:17 smithi170 ceph-mon[102832]: pgmap v8156: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 13 op/s 2023-09-10T03:29:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:17 smithi082 ceph-mon[98694]: pgmap v8156: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 13 op/s 2023-09-10T03:29:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14988.903021 secs 2023-09-10T03:29:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14990.232333 secs 2023-09-10T03:29:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14988.903021 secs 2023-09-10T03:29:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14990.232333 secs 2023-09-10T03:29:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14988.903021 secs 2023-09-10T03:29:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14990.232333 secs 2023-09-10T03:29:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14990.943756 secs 2023-09-10T03:29:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:19 smithi098 ceph-mon[103124]: pgmap v8157: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 12 op/s 2023-09-10T03:29:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14990.943756 secs 2023-09-10T03:29:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:19 smithi170 ceph-mon[102832]: pgmap v8157: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 12 op/s 2023-09-10T03:29:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14990.943756 secs 2023-09-10T03:29:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:19 smithi082 ceph-mon[98694]: pgmap v8157: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 12 op/s 2023-09-10T03:29:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:21 smithi098 ceph-mon[103124]: pgmap v8158: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T03:29:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:21 smithi170 ceph-mon[102832]: pgmap v8158: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T03:29:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:21 smithi082 ceph-mon[98694]: pgmap v8158: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T03:29:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:23 smithi082 ceph-mon[98694]: pgmap v8159: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T03:29:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:29:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14995.232392 secs 2023-09-10T03:29:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:23 smithi098 ceph-mon[103124]: pgmap v8159: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T03:29:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:29:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14995.232392 secs 2023-09-10T03:29:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:23 smithi170 ceph-mon[102832]: pgmap v8159: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T03:29:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:29:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14995.232392 secs 2023-09-10T03:29:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 14995.943820 secs 2023-09-10T03:29:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:29:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:29:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:29:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 14995.943820 secs 2023-09-10T03:29:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:29:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:29:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:29:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 14995.943820 secs 2023-09-10T03:29:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:29:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:29:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:29:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:25 smithi098 ceph-mon[103124]: pgmap v8160: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 9 op/s 2023-09-10T03:29:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:25 smithi170 ceph-mon[102832]: pgmap v8160: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 9 op/s 2023-09-10T03:29:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:25 smithi082 ceph-mon[98694]: pgmap v8160: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 9 op/s 2023-09-10T03:29:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:27 smithi098 ceph-mon[103124]: pgmap v8161: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 107 KiB/s wr, 14 op/s 2023-09-10T03:29:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:27 smithi170 ceph-mon[102832]: pgmap v8161: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 107 KiB/s wr, 14 op/s 2023-09-10T03:29:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:27 smithi082 ceph-mon[98694]: pgmap v8161: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 107 KiB/s wr, 14 op/s 2023-09-10T03:29:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 14998.903189 secs 2023-09-10T03:29:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15000.232480 secs 2023-09-10T03:29:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 14998.903189 secs 2023-09-10T03:29:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15000.232480 secs 2023-09-10T03:29:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 14998.903189 secs 2023-09-10T03:29:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15000.232480 secs 2023-09-10T03:29:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15000.943885 secs 2023-09-10T03:29:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:29 smithi098 ceph-mon[103124]: pgmap v8162: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T03:29:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15000.943885 secs 2023-09-10T03:29:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:29 smithi170 ceph-mon[102832]: pgmap v8162: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T03:29:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15000.943885 secs 2023-09-10T03:29:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:29 smithi082 ceph-mon[98694]: pgmap v8162: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T03:29:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:31 smithi098 ceph-mon[103124]: pgmap v8163: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 232 KiB/s wr, 16 op/s 2023-09-10T03:29:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:31 smithi170 ceph-mon[102832]: pgmap v8163: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 232 KiB/s wr, 16 op/s 2023-09-10T03:29:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:31 smithi082 ceph-mon[98694]: pgmap v8163: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 232 KiB/s wr, 16 op/s 2023-09-10T03:29:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:32 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:29:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:29:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:29:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:32 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:29:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:29:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:29:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:32 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:29:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:29:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:29:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:33 smithi098 ceph-mon[103124]: pgmap v8164: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 232 KiB/s wr, 12 op/s 2023-09-10T03:29:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15005.232563 secs 2023-09-10T03:29:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:33 smithi170 ceph-mon[102832]: pgmap v8164: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 232 KiB/s wr, 12 op/s 2023-09-10T03:29:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15005.232563 secs 2023-09-10T03:29:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:33 smithi082 ceph-mon[98694]: pgmap v8164: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 232 KiB/s wr, 12 op/s 2023-09-10T03:29:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15005.232563 secs 2023-09-10T03:29:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15005.943950 secs 2023-09-10T03:29:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15005.943950 secs 2023-09-10T03:29:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15005.943950 secs 2023-09-10T03:29:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:35 smithi098 ceph-mon[103124]: pgmap v8165: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 232 KiB/s wr, 12 op/s 2023-09-10T03:29:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:29:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:35 smithi170 ceph-mon[102832]: pgmap v8165: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 232 KiB/s wr, 12 op/s 2023-09-10T03:29:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:29:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:35 smithi082 ceph-mon[98694]: pgmap v8165: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 232 KiB/s wr, 12 op/s 2023-09-10T03:29:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:29:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:29:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:29:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:29:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:29:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:29:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:29:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:37 smithi170 ceph-mon[102832]: pgmap v8166: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 251 KiB/s wr, 18 op/s 2023-09-10T03:29:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:29:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:29:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:37 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:29:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:37 smithi082 ceph-mon[98694]: pgmap v8166: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 251 KiB/s wr, 18 op/s 2023-09-10T03:29:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:29:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:29:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:37 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:29:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:37 smithi098 ceph-mon[103124]: pgmap v8166: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 251 KiB/s wr, 18 op/s 2023-09-10T03:29:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:29:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:29:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:37 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:29:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15010.232615 secs 2023-09-10T03:29:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15010.232615 secs 2023-09-10T03:29:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15010.232615 secs 2023-09-10T03:29:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15010.944015 secs 2023-09-10T03:29:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:39 smithi170 ceph-mon[102832]: pgmap v8167: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 193 KiB/s wr, 13 op/s 2023-09-10T03:29:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15010.944015 secs 2023-09-10T03:29:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:39 smithi082 ceph-mon[98694]: pgmap v8167: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 193 KiB/s wr, 13 op/s 2023-09-10T03:29:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15010.944015 secs 2023-09-10T03:29:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:39 smithi098 ceph-mon[103124]: pgmap v8167: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 193 KiB/s wr, 13 op/s 2023-09-10T03:29:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:29:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:29:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:29:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:29:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:29:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:29:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:41 smithi170 ceph-mon[102832]: pgmap v8168: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 453 KiB/s wr, 20 op/s 2023-09-10T03:29:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:41 smithi082 ceph-mon[98694]: pgmap v8168: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 453 KiB/s wr, 20 op/s 2023-09-10T03:29:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:41 smithi098 ceph-mon[103124]: pgmap v8168: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 453 KiB/s wr, 20 op/s 2023-09-10T03:29:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:43 smithi082 ceph-mon[98694]: pgmap v8169: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T03:29:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15015.232672 secs 2023-09-10T03:29:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:43 smithi098 ceph-mon[103124]: pgmap v8169: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T03:29:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15015.232672 secs 2023-09-10T03:29:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:43 smithi170 ceph-mon[102832]: pgmap v8169: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T03:29:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15015.232672 secs 2023-09-10T03:29:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15015.944078 secs 2023-09-10T03:29:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15015.944078 secs 2023-09-10T03:29:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15015.944078 secs 2023-09-10T03:29:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:45 smithi098 ceph-mon[103124]: pgmap v8170: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T03:29:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:45 smithi170 ceph-mon[102832]: pgmap v8170: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T03:29:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:45 smithi082 ceph-mon[98694]: pgmap v8170: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T03:29:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:47 smithi098 ceph-mon[103124]: pgmap v8171: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 353 KiB/s wr, 18 op/s 2023-09-10T03:29:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:47 smithi170 ceph-mon[102832]: pgmap v8171: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 353 KiB/s wr, 18 op/s 2023-09-10T03:29:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:47 smithi082 ceph-mon[98694]: pgmap v8171: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 353 KiB/s wr, 18 op/s 2023-09-10T03:29:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15020.232731 secs 2023-09-10T03:29:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15020.232731 secs 2023-09-10T03:29:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15020.232731 secs 2023-09-10T03:29:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15020.944143 secs 2023-09-10T03:29:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:49 smithi098 ceph-mon[103124]: pgmap v8172: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 333 KiB/s wr, 12 op/s 2023-09-10T03:29:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15020.944143 secs 2023-09-10T03:29:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:49 smithi170 ceph-mon[102832]: pgmap v8172: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 333 KiB/s wr, 12 op/s 2023-09-10T03:29:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15020.944143 secs 2023-09-10T03:29:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:49 smithi082 ceph-mon[98694]: pgmap v8172: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 333 KiB/s wr, 12 op/s 2023-09-10T03:29:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:51 smithi098 ceph-mon[103124]: pgmap v8173: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 384 KiB/s wr, 18 op/s 2023-09-10T03:29:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:51 smithi170 ceph-mon[102832]: pgmap v8173: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 384 KiB/s wr, 18 op/s 2023-09-10T03:29:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:51 smithi082 ceph-mon[98694]: pgmap v8173: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 384 KiB/s wr, 18 op/s 2023-09-10T03:29:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15023.903601 secs 2023-09-10T03:29:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:53 smithi098 ceph-mon[103124]: pgmap v8174: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T03:29:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:29:54.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:29:54.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15025.232789 secs 2023-09-10T03:29:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15023.903601 secs 2023-09-10T03:29:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:53 smithi170 ceph-mon[102832]: pgmap v8174: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T03:29:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:29:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:29:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15025.232789 secs 2023-09-10T03:29:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15023.903601 secs 2023-09-10T03:29:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:53 smithi082 ceph-mon[98694]: pgmap v8174: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T03:29:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:29:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:29:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15025.232789 secs 2023-09-10T03:29:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15025.944213 secs 2023-09-10T03:29:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15025.944213 secs 2023-09-10T03:29:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15025.944213 secs 2023-09-10T03:29:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:55 smithi098 ceph-mon[103124]: pgmap v8175: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T03:29:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:55 smithi170 ceph-mon[102832]: pgmap v8175: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T03:29:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:55 smithi082 ceph-mon[98694]: pgmap v8175: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T03:29:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:29:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:29:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:29:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:29:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:29:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:29:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:29:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:29:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:29:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:57 smithi098 ceph-mon[103124]: pgmap v8176: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 227 KiB/s wr, 17 op/s 2023-09-10T03:29:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:57 smithi170 ceph-mon[102832]: pgmap v8176: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 227 KiB/s wr, 17 op/s 2023-09-10T03:29:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:57 smithi082 ceph-mon[98694]: pgmap v8176: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 227 KiB/s wr, 17 op/s 2023-09-10T03:29:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15028.903729 secs 2023-09-10T03:29:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15030.232871 secs 2023-09-10T03:29:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:29:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15028.903729 secs 2023-09-10T03:29:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15030.232871 secs 2023-09-10T03:29:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:29:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15028.903729 secs 2023-09-10T03:29:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15030.232871 secs 2023-09-10T03:29:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:30:00.039 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15030.944277 secs 2023-09-10T03:30:00.039 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:29:59 smithi098 ceph-mon[103124]: pgmap v8177: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T03:30:00.043 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15030.944277 secs 2023-09-10T03:30:00.043 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:29:59 smithi082 ceph-mon[98694]: pgmap v8177: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T03:30:00.078 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15030.944277 secs 2023-09-10T03:30:00.078 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:29:59 smithi170 ceph-mon[102832]: pgmap v8177: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T03:30:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T03:30:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T03:30:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T03:30:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:01 smithi098 ceph-mon[103124]: pgmap v8178: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 208 KiB/s wr, 20 op/s 2023-09-10T03:30:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:30:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:30:02.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:30:02.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:30:02.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:30:02.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:30:02.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:02.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:02.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:02.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:02.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:02.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:02.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:02.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:01 smithi170 ceph-mon[102832]: pgmap v8178: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 208 KiB/s wr, 20 op/s 2023-09-10T03:30:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:30:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:30:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:30:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:30:02.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:30:02.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:30:02.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:02.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:02.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:02.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:02.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:02.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:02.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:02.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:01 smithi082 ceph-mon[98694]: pgmap v8178: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 208 KiB/s wr, 20 op/s 2023-09-10T03:30:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:30:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:30:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:30:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:30:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:30:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:30:02.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:02.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:02.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:02.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:02.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:02.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:02.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:02.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:02 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 107.1M 2023-09-10T03:30:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:02 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 112329715: error parsing value: Value '112329715' is below minimum 939524096 2023-09-10T03:30:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:02 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T03:30:03.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:02 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T03:30:03.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:03.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:03.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:03.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:03.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:02 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T03:30:03.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:02 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T03:30:03.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:30:03.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:30:03.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:30:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:02 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 107.1M 2023-09-10T03:30:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:02 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 112329715: error parsing value: Value '112329715' is below minimum 939524096 2023-09-10T03:30:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:02 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T03:30:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:02 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T03:30:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:03.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:03.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:03.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:03.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:02 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T03:30:03.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:02 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T03:30:03.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:30:03.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:30:03.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:30:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:02 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 107.1M 2023-09-10T03:30:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:02 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 112329715: error parsing value: Value '112329715' is below minimum 939524096 2023-09-10T03:30:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:02 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T03:30:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:02 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T03:30:03.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:03.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:03.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:03.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:30:03.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:02 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T03:30:03.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:02 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T03:30:03.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:30:03.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:30:03.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:30:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15033.903778 secs 2023-09-10T03:30:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:03 smithi098 ceph-mon[103124]: pgmap v8179: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 158 KiB/s wr, 14 op/s 2023-09-10T03:30:04.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15035.232933 secs 2023-09-10T03:30:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15033.903778 secs 2023-09-10T03:30:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:03 smithi170 ceph-mon[102832]: pgmap v8179: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 158 KiB/s wr, 14 op/s 2023-09-10T03:30:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15035.232933 secs 2023-09-10T03:30:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15033.903778 secs 2023-09-10T03:30:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:03 smithi082 ceph-mon[98694]: pgmap v8179: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 158 KiB/s wr, 14 op/s 2023-09-10T03:30:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15035.232933 secs 2023-09-10T03:30:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15035.944343 secs 2023-09-10T03:30:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15035.944343 secs 2023-09-10T03:30:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15035.944343 secs 2023-09-10T03:30:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:05 smithi098 ceph-mon[103124]: pgmap v8180: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 158 KiB/s wr, 14 op/s 2023-09-10T03:30:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:05 smithi170 ceph-mon[102832]: pgmap v8180: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 158 KiB/s wr, 14 op/s 2023-09-10T03:30:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:05 smithi082 ceph-mon[98694]: pgmap v8180: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 158 KiB/s wr, 14 op/s 2023-09-10T03:30:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:08 smithi170 ceph-mon[102832]: pgmap v8181: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 204 KiB/s wr, 30 op/s 2023-09-10T03:30:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:08 smithi082 ceph-mon[98694]: pgmap v8181: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 204 KiB/s wr, 30 op/s 2023-09-10T03:30:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:08 smithi098 ceph-mon[103124]: pgmap v8181: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 204 KiB/s wr, 30 op/s 2023-09-10T03:30:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15038.903837 secs 2023-09-10T03:30:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15040.232998 secs 2023-09-10T03:30:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:09 smithi170 ceph-mon[102832]: osdmap e579: 12 total, 12 up, 12 in 2023-09-10T03:30:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15038.903837 secs 2023-09-10T03:30:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15040.232998 secs 2023-09-10T03:30:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:09 smithi082 ceph-mon[98694]: osdmap e579: 12 total, 12 up, 12 in 2023-09-10T03:30:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15038.903837 secs 2023-09-10T03:30:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15040.232998 secs 2023-09-10T03:30:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:09 smithi098 ceph-mon[103124]: osdmap e579: 12 total, 12 up, 12 in 2023-09-10T03:30:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15040.944407 secs 2023-09-10T03:30:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:10 smithi170 ceph-mon[102832]: pgmap v8183: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 122 KiB/s wr, 30 op/s 2023-09-10T03:30:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15040.944407 secs 2023-09-10T03:30:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:10 smithi082 ceph-mon[98694]: pgmap v8183: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 122 KiB/s wr, 30 op/s 2023-09-10T03:30:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15040.944407 secs 2023-09-10T03:30:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:10 smithi098 ceph-mon[103124]: pgmap v8183: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 122 KiB/s wr, 30 op/s 2023-09-10T03:30:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:12 smithi170 ceph-mon[102832]: pgmap v8184: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 56 KiB/s wr, 25 op/s 2023-09-10T03:30:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:12 smithi082 ceph-mon[98694]: pgmap v8184: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 56 KiB/s wr, 25 op/s 2023-09-10T03:30:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:12 smithi098 ceph-mon[103124]: pgmap v8184: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 56 KiB/s wr, 25 op/s 2023-09-10T03:30:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15043.903898 secs 2023-09-10T03:30:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:13 smithi098 ceph-mon[103124]: osdmap e580: 12 total, 12 up, 12 in 2023-09-10T03:30:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:13 smithi098 ceph-mon[103124]: pgmap v8186: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 70 KiB/s wr, 31 op/s 2023-09-10T03:30:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15045.233060 secs 2023-09-10T03:30:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15043.903898 secs 2023-09-10T03:30:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:13 smithi170 ceph-mon[102832]: osdmap e580: 12 total, 12 up, 12 in 2023-09-10T03:30:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:13 smithi170 ceph-mon[102832]: pgmap v8186: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 70 KiB/s wr, 31 op/s 2023-09-10T03:30:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15045.233060 secs 2023-09-10T03:30:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15043.903898 secs 2023-09-10T03:30:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:13 smithi082 ceph-mon[98694]: osdmap e580: 12 total, 12 up, 12 in 2023-09-10T03:30:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:13 smithi082 ceph-mon[98694]: pgmap v8186: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 70 KiB/s wr, 31 op/s 2023-09-10T03:30:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15045.233060 secs 2023-09-10T03:30:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15045.944476 secs 2023-09-10T03:30:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15045.944476 secs 2023-09-10T03:30:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15045.944476 secs 2023-09-10T03:30:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:15 smithi098 ceph-mon[103124]: pgmap v8187: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-09-10T03:30:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:15 smithi170 ceph-mon[102832]: pgmap v8187: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-09-10T03:30:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:15 smithi082 ceph-mon[98694]: pgmap v8187: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-09-10T03:30:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:17 smithi098 ceph-mon[103124]: pgmap v8188: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 13 op/s 2023-09-10T03:30:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:17 smithi170 ceph-mon[102832]: pgmap v8188: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 13 op/s 2023-09-10T03:30:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:17 smithi082 ceph-mon[98694]: pgmap v8188: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 13 op/s 2023-09-10T03:30:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15048.904010 secs 2023-09-10T03:30:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15050.233123 secs 2023-09-10T03:30:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15048.904010 secs 2023-09-10T03:30:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15050.233123 secs 2023-09-10T03:30:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15048.904010 secs 2023-09-10T03:30:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15050.233123 secs 2023-09-10T03:30:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15050.944541 secs 2023-09-10T03:30:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:19 smithi098 ceph-mon[103124]: pgmap v8189: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T03:30:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15050.944541 secs 2023-09-10T03:30:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:19 smithi170 ceph-mon[102832]: pgmap v8189: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T03:30:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15050.944541 secs 2023-09-10T03:30:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:19 smithi082 ceph-mon[98694]: pgmap v8189: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 59 KiB/s wr, 11 op/s 2023-09-10T03:30:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:21 smithi098 ceph-mon[103124]: pgmap v8190: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T03:30:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:21 smithi170 ceph-mon[102832]: pgmap v8190: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T03:30:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:21 smithi082 ceph-mon[98694]: pgmap v8190: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T03:30:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:23 smithi098 ceph-mon[103124]: pgmap v8191: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T03:30:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:30:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15055.233180 secs 2023-09-10T03:30:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:23 smithi170 ceph-mon[102832]: pgmap v8191: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T03:30:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:30:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15055.233180 secs 2023-09-10T03:30:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:23 smithi082 ceph-mon[98694]: pgmap v8191: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T03:30:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:30:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15055.233180 secs 2023-09-10T03:30:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15055.944607 secs 2023-09-10T03:30:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:30:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:30:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:30:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15055.944607 secs 2023-09-10T03:30:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:30:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:30:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:30:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15055.944607 secs 2023-09-10T03:30:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:30:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:30:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:30:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:25 smithi098 ceph-mon[103124]: pgmap v8192: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 95 KiB/s wr, 9 op/s 2023-09-10T03:30:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:25 smithi170 ceph-mon[102832]: pgmap v8192: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 95 KiB/s wr, 9 op/s 2023-09-10T03:30:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:25 smithi082 ceph-mon[98694]: pgmap v8192: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 95 KiB/s wr, 9 op/s 2023-09-10T03:30:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:27 smithi170 ceph-mon[102832]: pgmap v8193: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 155 KiB/s wr, 14 op/s 2023-09-10T03:30:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:27 smithi082 ceph-mon[98694]: pgmap v8193: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 155 KiB/s wr, 14 op/s 2023-09-10T03:30:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:27 smithi098 ceph-mon[103124]: pgmap v8193: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 155 KiB/s wr, 14 op/s 2023-09-10T03:30:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15060.233265 secs 2023-09-10T03:30:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15060.233265 secs 2023-09-10T03:30:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15060.233265 secs 2023-09-10T03:30:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15060.944670 secs 2023-09-10T03:30:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:29 smithi170 ceph-mon[102832]: pgmap v8194: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 107 KiB/s wr, 9 op/s 2023-09-10T03:30:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15060.944670 secs 2023-09-10T03:30:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:29 smithi082 ceph-mon[98694]: pgmap v8194: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 107 KiB/s wr, 9 op/s 2023-09-10T03:30:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15060.944670 secs 2023-09-10T03:30:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:29 smithi098 ceph-mon[103124]: pgmap v8194: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 107 KiB/s wr, 9 op/s 2023-09-10T03:30:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:31 smithi170 ceph-mon[102832]: pgmap v8195: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 208 KiB/s wr, 15 op/s 2023-09-10T03:30:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:31 smithi082 ceph-mon[98694]: pgmap v8195: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 208 KiB/s wr, 15 op/s 2023-09-10T03:30:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:31 smithi098 ceph-mon[103124]: pgmap v8195: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 208 KiB/s wr, 15 op/s 2023-09-10T03:30:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15063.904203 secs 2023-09-10T03:30:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:33 smithi170 ceph-mon[102832]: pgmap v8196: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T03:30:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:30:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:30:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15065.233353 secs 2023-09-10T03:30:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15063.904203 secs 2023-09-10T03:30:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:33 smithi082 ceph-mon[98694]: pgmap v8196: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T03:30:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:30:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:30:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15065.233353 secs 2023-09-10T03:30:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15063.904203 secs 2023-09-10T03:30:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:33 smithi098 ceph-mon[103124]: pgmap v8196: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T03:30:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:30:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:30:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15065.233353 secs 2023-09-10T03:30:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15065.944734 secs 2023-09-10T03:30:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15065.944734 secs 2023-09-10T03:30:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15065.944734 secs 2023-09-10T03:30:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:35 smithi170 ceph-mon[102832]: pgmap v8197: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T03:30:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:35 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:30:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:35 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:30:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:35 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:30:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:35 smithi082 ceph-mon[98694]: pgmap v8197: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T03:30:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:35 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:30:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:35 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:30:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:35 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:30:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:35 smithi098 ceph-mon[103124]: pgmap v8197: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T03:30:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:35 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:30:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:35 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:30:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:35 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:30:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:37 smithi170 ceph-mon[102832]: pgmap v8198: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 327 KiB/s wr, 16 op/s 2023-09-10T03:30:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:37 smithi082 ceph-mon[98694]: pgmap v8198: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 327 KiB/s wr, 16 op/s 2023-09-10T03:30:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:37 smithi098 ceph-mon[103124]: pgmap v8198: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 327 KiB/s wr, 16 op/s 2023-09-10T03:30:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15068.904281 secs 2023-09-10T03:30:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15070.233411 secs 2023-09-10T03:30:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15068.904281 secs 2023-09-10T03:30:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15070.233411 secs 2023-09-10T03:30:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15068.904281 secs 2023-09-10T03:30:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15070.233411 secs 2023-09-10T03:30:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15070.944800 secs 2023-09-10T03:30:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:39 smithi170 ceph-mon[102832]: pgmap v8199: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 266 KiB/s wr, 11 op/s 2023-09-10T03:30:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15070.944800 secs 2023-09-10T03:30:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:39 smithi082 ceph-mon[98694]: pgmap v8199: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 266 KiB/s wr, 11 op/s 2023-09-10T03:30:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15070.944800 secs 2023-09-10T03:30:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:39 smithi098 ceph-mon[103124]: pgmap v8199: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 266 KiB/s wr, 11 op/s 2023-09-10T03:30:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:41 smithi170 ceph-mon[102832]: pgmap v8200: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 444 KiB/s wr, 18 op/s 2023-09-10T03:30:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:41 smithi082 ceph-mon[98694]: pgmap v8200: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 444 KiB/s wr, 18 op/s 2023-09-10T03:30:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:41 smithi098 ceph-mon[103124]: pgmap v8200: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 444 KiB/s wr, 18 op/s 2023-09-10T03:30:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:43 smithi098 ceph-mon[103124]: pgmap v8201: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 342 KiB/s wr, 12 op/s 2023-09-10T03:30:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:30:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15075.233498 secs 2023-09-10T03:30:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:43 smithi170 ceph-mon[102832]: pgmap v8201: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 342 KiB/s wr, 12 op/s 2023-09-10T03:30:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:30:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15075.233498 secs 2023-09-10T03:30:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:43 smithi082 ceph-mon[98694]: pgmap v8201: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 342 KiB/s wr, 12 op/s 2023-09-10T03:30:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:30:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15075.233498 secs 2023-09-10T03:30:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15075.944865 secs 2023-09-10T03:30:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:44 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:30:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:30:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:30:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15075.944865 secs 2023-09-10T03:30:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:44 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:30:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:30:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:30:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15075.944865 secs 2023-09-10T03:30:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:44 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:30:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:30:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:30:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:45 smithi098 ceph-mon[103124]: pgmap v8202: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 342 KiB/s wr, 12 op/s 2023-09-10T03:30:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:45 smithi170 ceph-mon[102832]: pgmap v8202: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 342 KiB/s wr, 12 op/s 2023-09-10T03:30:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:45 smithi082 ceph-mon[98694]: pgmap v8202: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 342 KiB/s wr, 12 op/s 2023-09-10T03:30:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:47 smithi170 ceph-mon[102832]: pgmap v8203: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 379 KiB/s wr, 18 op/s 2023-09-10T03:30:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:47 smithi082 ceph-mon[98694]: pgmap v8203: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 379 KiB/s wr, 18 op/s 2023-09-10T03:30:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:47 smithi098 ceph-mon[103124]: pgmap v8203: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 379 KiB/s wr, 18 op/s 2023-09-10T03:30:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15080.233556 secs 2023-09-10T03:30:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15080.233556 secs 2023-09-10T03:30:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15080.233556 secs 2023-09-10T03:30:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15080.944954 secs 2023-09-10T03:30:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:49 smithi170 ceph-mon[102832]: pgmap v8204: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 214 KiB/s wr, 12 op/s 2023-09-10T03:30:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:30:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:30:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15080.944954 secs 2023-09-10T03:30:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:49 smithi082 ceph-mon[98694]: pgmap v8204: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 214 KiB/s wr, 12 op/s 2023-09-10T03:30:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:30:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:30:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15080.944954 secs 2023-09-10T03:30:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:49 smithi098 ceph-mon[103124]: pgmap v8204: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 214 KiB/s wr, 12 op/s 2023-09-10T03:30:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:30:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:30:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:51 smithi170 ceph-mon[102832]: pgmap v8205: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 303 KiB/s wr, 18 op/s 2023-09-10T03:30:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:51 smithi082 ceph-mon[98694]: pgmap v8205: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 303 KiB/s wr, 18 op/s 2023-09-10T03:30:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:51 smithi098 ceph-mon[103124]: pgmap v8205: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 303 KiB/s wr, 18 op/s 2023-09-10T03:30:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15083.904555 secs 2023-09-10T03:30:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:53 smithi098 ceph-mon[103124]: pgmap v8206: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T03:30:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15085.233617 secs 2023-09-10T03:30:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15083.904555 secs 2023-09-10T03:30:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:53 smithi170 ceph-mon[102832]: pgmap v8206: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T03:30:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15085.233617 secs 2023-09-10T03:30:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15083.904555 secs 2023-09-10T03:30:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:53 smithi082 ceph-mon[98694]: pgmap v8206: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T03:30:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15085.233617 secs 2023-09-10T03:30:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15085.945019 secs 2023-09-10T03:30:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15085.945019 secs 2023-09-10T03:30:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15085.945019 secs 2023-09-10T03:30:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:55 smithi098 ceph-mon[103124]: pgmap v8207: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T03:30:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:55 smithi170 ceph-mon[102832]: pgmap v8207: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T03:30:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:55 smithi082 ceph-mon[98694]: pgmap v8207: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T03:30:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:30:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:30:57.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:30:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:30:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:30:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:30:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:30:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:30:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:30:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:57 smithi098 ceph-mon[103124]: pgmap v8208: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 228 KiB/s wr, 16 op/s 2023-09-10T03:30:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:57 smithi170 ceph-mon[102832]: pgmap v8208: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 228 KiB/s wr, 16 op/s 2023-09-10T03:30:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:57 smithi082 ceph-mon[98694]: pgmap v8208: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 228 KiB/s wr, 16 op/s 2023-09-10T03:30:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15088.904649 secs 2023-09-10T03:30:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15090.233676 secs 2023-09-10T03:30:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15088.904649 secs 2023-09-10T03:30:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15090.233676 secs 2023-09-10T03:30:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15088.904649 secs 2023-09-10T03:30:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15090.233676 secs 2023-09-10T03:31:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15090.945083 secs 2023-09-10T03:31:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:30:59 smithi098 ceph-mon[103124]: pgmap v8209: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 10 op/s 2023-09-10T03:31:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15090.945083 secs 2023-09-10T03:31:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:30:59 smithi170 ceph-mon[102832]: pgmap v8209: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 10 op/s 2023-09-10T03:31:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15090.945083 secs 2023-09-10T03:31:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:30:59 smithi082 ceph-mon[98694]: pgmap v8209: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 10 op/s 2023-09-10T03:31:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:01 smithi098 ceph-mon[103124]: pgmap v8210: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 236 KiB/s wr, 15 op/s 2023-09-10T03:31:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:01 smithi170 ceph-mon[102832]: pgmap v8210: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 236 KiB/s wr, 15 op/s 2023-09-10T03:31:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:01 smithi082 ceph-mon[98694]: pgmap v8210: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 236 KiB/s wr, 15 op/s 2023-09-10T03:31:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:31:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:31:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:31:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15093.904756 secs 2023-09-10T03:31:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:03 smithi098 ceph-mon[103124]: pgmap v8211: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 10 op/s 2023-09-10T03:31:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:31:04.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:31:04.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:31:04.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15095.233737 secs 2023-09-10T03:31:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15093.904756 secs 2023-09-10T03:31:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:03 smithi170 ceph-mon[102832]: pgmap v8211: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 10 op/s 2023-09-10T03:31:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:31:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:31:04.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:31:04.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15095.233737 secs 2023-09-10T03:31:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15093.904756 secs 2023-09-10T03:31:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:03 smithi082 ceph-mon[98694]: pgmap v8211: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 10 op/s 2023-09-10T03:31:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:31:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:31:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:31:04.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15095.233737 secs 2023-09-10T03:31:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15095.945151 secs 2023-09-10T03:31:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15095.945151 secs 2023-09-10T03:31:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15095.945151 secs 2023-09-10T03:31:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:05 smithi098 ceph-mon[103124]: pgmap v8212: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 10 op/s 2023-09-10T03:31:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:05 smithi170 ceph-mon[102832]: pgmap v8212: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 10 op/s 2023-09-10T03:31:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:05 smithi082 ceph-mon[98694]: pgmap v8212: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 10 op/s 2023-09-10T03:31:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:07 smithi170 ceph-mon[102832]: pgmap v8213: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 204 KiB/s wr, 30 op/s 2023-09-10T03:31:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:07 smithi082 ceph-mon[98694]: pgmap v8213: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 204 KiB/s wr, 30 op/s 2023-09-10T03:31:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:07 smithi098 ceph-mon[103124]: pgmap v8213: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 204 KiB/s wr, 30 op/s 2023-09-10T03:31:09.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15098.904772 secs 2023-09-10T03:31:09.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:08 smithi170 ceph-mon[102832]: osdmap e581: 12 total, 12 up, 12 in 2023-09-10T03:31:09.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15100.233812 secs 2023-09-10T03:31:09.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15098.904772 secs 2023-09-10T03:31:09.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:08 smithi082 ceph-mon[98694]: osdmap e581: 12 total, 12 up, 12 in 2023-09-10T03:31:09.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15100.233812 secs 2023-09-10T03:31:09.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15098.904772 secs 2023-09-10T03:31:09.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:08 smithi098 ceph-mon[103124]: osdmap e581: 12 total, 12 up, 12 in 2023-09-10T03:31:09.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15100.233812 secs 2023-09-10T03:31:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15100.945218 secs 2023-09-10T03:31:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:09 smithi170 ceph-mon[102832]: pgmap v8215: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 122 KiB/s wr, 29 op/s 2023-09-10T03:31:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15100.945218 secs 2023-09-10T03:31:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:09 smithi082 ceph-mon[98694]: pgmap v8215: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 122 KiB/s wr, 29 op/s 2023-09-10T03:31:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15100.945218 secs 2023-09-10T03:31:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:09 smithi098 ceph-mon[103124]: pgmap v8215: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 122 KiB/s wr, 29 op/s 2023-09-10T03:31:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:11 smithi170 ceph-mon[102832]: pgmap v8216: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-10T03:31:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:11 smithi082 ceph-mon[98694]: pgmap v8216: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-10T03:31:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:11 smithi098 ceph-mon[103124]: pgmap v8216: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-10T03:31:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15103.904831 secs 2023-09-10T03:31:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:13 smithi098 ceph-mon[103124]: osdmap e582: 12 total, 12 up, 12 in 2023-09-10T03:31:14.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:13 smithi098 ceph-mon[103124]: pgmap v8218: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 87 KiB/s wr, 36 op/s 2023-09-10T03:31:14.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15105.233870 secs 2023-09-10T03:31:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15103.904831 secs 2023-09-10T03:31:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:13 smithi170 ceph-mon[102832]: osdmap e582: 12 total, 12 up, 12 in 2023-09-10T03:31:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:13 smithi170 ceph-mon[102832]: pgmap v8218: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 87 KiB/s wr, 36 op/s 2023-09-10T03:31:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15105.233870 secs 2023-09-10T03:31:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15103.904831 secs 2023-09-10T03:31:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:13 smithi082 ceph-mon[98694]: osdmap e582: 12 total, 12 up, 12 in 2023-09-10T03:31:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:13 smithi082 ceph-mon[98694]: pgmap v8218: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 87 KiB/s wr, 36 op/s 2023-09-10T03:31:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15105.233870 secs 2023-09-10T03:31:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15105.945286 secs 2023-09-10T03:31:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15105.945286 secs 2023-09-10T03:31:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15105.945286 secs 2023-09-10T03:31:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:15 smithi098 ceph-mon[103124]: pgmap v8219: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:31:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:15 smithi170 ceph-mon[102832]: pgmap v8219: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:31:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:15 smithi082 ceph-mon[98694]: pgmap v8219: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:31:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:17 smithi170 ceph-mon[102832]: pgmap v8220: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 13 op/s 2023-09-10T03:31:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:17 smithi082 ceph-mon[98694]: pgmap v8220: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 13 op/s 2023-09-10T03:31:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:17 smithi098 ceph-mon[103124]: pgmap v8220: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 13 op/s 2023-09-10T03:31:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15108.904888 secs 2023-09-10T03:31:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15110.233927 secs 2023-09-10T03:31:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15108.904888 secs 2023-09-10T03:31:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15110.233927 secs 2023-09-10T03:31:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15108.904888 secs 2023-09-10T03:31:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15110.233927 secs 2023-09-10T03:31:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15110.945376 secs 2023-09-10T03:31:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:19 smithi170 ceph-mon[102832]: pgmap v8221: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 11 op/s 2023-09-10T03:31:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15110.945376 secs 2023-09-10T03:31:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:19 smithi082 ceph-mon[98694]: pgmap v8221: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 11 op/s 2023-09-10T03:31:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15110.945376 secs 2023-09-10T03:31:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:19 smithi098 ceph-mon[103124]: pgmap v8221: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 11 op/s 2023-09-10T03:31:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:21 smithi170 ceph-mon[102832]: pgmap v8222: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 11 op/s 2023-09-10T03:31:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:21 smithi082 ceph-mon[98694]: pgmap v8222: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 11 op/s 2023-09-10T03:31:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:21 smithi098 ceph-mon[103124]: pgmap v8222: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 11 op/s 2023-09-10T03:31:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:23 smithi098 ceph-mon[103124]: pgmap v8223: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 11 op/s 2023-09-10T03:31:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:31:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15115.233987 secs 2023-09-10T03:31:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:23 smithi170 ceph-mon[102832]: pgmap v8223: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 11 op/s 2023-09-10T03:31:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:31:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15115.233987 secs 2023-09-10T03:31:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:23 smithi082 ceph-mon[98694]: pgmap v8223: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 11 op/s 2023-09-10T03:31:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:31:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15115.233987 secs 2023-09-10T03:31:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15115.945440 secs 2023-09-10T03:31:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:31:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:31:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:31:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15115.945440 secs 2023-09-10T03:31:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:31:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:31:25.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:31:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15115.945440 secs 2023-09-10T03:31:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:31:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:31:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:31:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:25 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:31:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:25 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:31:26.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:25 smithi098 ceph-mon[103124]: pgmap v8224: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 9 op/s 2023-09-10T03:31:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:25 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:31:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:25 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:31:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:25 smithi170 ceph-mon[102832]: pgmap v8224: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 9 op/s 2023-09-10T03:31:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:25 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:31:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:25 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:31:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:25 smithi082 ceph-mon[98694]: pgmap v8224: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 9 op/s 2023-09-10T03:31:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:27 smithi170 ceph-mon[102832]: pgmap v8225: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 125 KiB/s wr, 15 op/s 2023-09-10T03:31:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:27 smithi082 ceph-mon[98694]: pgmap v8225: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 125 KiB/s wr, 15 op/s 2023-09-10T03:31:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:27 smithi098 ceph-mon[103124]: pgmap v8225: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 125 KiB/s wr, 15 op/s 2023-09-10T03:31:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15120.234058 secs 2023-09-10T03:31:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15120.234058 secs 2023-09-10T03:31:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15120.234058 secs 2023-09-10T03:31:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15120.945490 secs 2023-09-10T03:31:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:29 smithi170 ceph-mon[102832]: pgmap v8226: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T03:31:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15120.945490 secs 2023-09-10T03:31:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:29 smithi082 ceph-mon[98694]: pgmap v8226: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T03:31:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15120.945490 secs 2023-09-10T03:31:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:29 smithi098 ceph-mon[103124]: pgmap v8226: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T03:31:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:31 smithi170 ceph-mon[102832]: pgmap v8227: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 183 KiB/s wr, 15 op/s 2023-09-10T03:31:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:31 smithi082 ceph-mon[98694]: pgmap v8227: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 183 KiB/s wr, 15 op/s 2023-09-10T03:31:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:31 smithi098 ceph-mon[103124]: pgmap v8227: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 183 KiB/s wr, 15 op/s 2023-09-10T03:31:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:33 smithi098 ceph-mon[103124]: pgmap v8228: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T03:31:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15125.234144 secs 2023-09-10T03:31:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:33 smithi170 ceph-mon[102832]: pgmap v8228: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T03:31:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15125.234144 secs 2023-09-10T03:31:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:33 smithi082 ceph-mon[98694]: pgmap v8228: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T03:31:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15125.234144 secs 2023-09-10T03:31:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15125.945557 secs 2023-09-10T03:31:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15125.945557 secs 2023-09-10T03:31:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15125.945557 secs 2023-09-10T03:31:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:35 smithi098 ceph-mon[103124]: pgmap v8229: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T03:31:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:35 smithi170 ceph-mon[102832]: pgmap v8229: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T03:31:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:35 smithi082 ceph-mon[98694]: pgmap v8229: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T03:31:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:37 smithi098 ceph-mon[103124]: pgmap v8230: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 219 KiB/s wr, 17 op/s 2023-09-10T03:31:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:37 smithi170 ceph-mon[102832]: pgmap v8230: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 219 KiB/s wr, 17 op/s 2023-09-10T03:31:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:37 smithi082 ceph-mon[98694]: pgmap v8230: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 219 KiB/s wr, 17 op/s 2023-09-10T03:31:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15128.905199 secs 2023-09-10T03:31:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15130.234202 secs 2023-09-10T03:31:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15128.905199 secs 2023-09-10T03:31:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15130.234202 secs 2023-09-10T03:31:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15128.905199 secs 2023-09-10T03:31:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15130.234202 secs 2023-09-10T03:31:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15130.945624 secs 2023-09-10T03:31:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:39 smithi170 ceph-mon[102832]: pgmap v8231: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 144 KiB/s wr, 12 op/s 2023-09-10T03:31:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15130.945624 secs 2023-09-10T03:31:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:39 smithi082 ceph-mon[98694]: pgmap v8231: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 144 KiB/s wr, 12 op/s 2023-09-10T03:31:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15130.945624 secs 2023-09-10T03:31:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:39 smithi098 ceph-mon[103124]: pgmap v8231: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 144 KiB/s wr, 12 op/s 2023-09-10T03:31:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:31:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:31:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:31:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:31:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:31:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:31:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:31:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:31:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:31:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:41 smithi170 ceph-mon[102832]: pgmap v8232: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 341 KiB/s wr, 18 op/s 2023-09-10T03:31:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:41 smithi082 ceph-mon[98694]: pgmap v8232: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 341 KiB/s wr, 18 op/s 2023-09-10T03:31:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:41 smithi098 ceph-mon[103124]: pgmap v8232: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 341 KiB/s wr, 18 op/s 2023-09-10T03:31:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15133.905261 secs 2023-09-10T03:31:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:43 smithi098 ceph-mon[103124]: pgmap v8233: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 233 KiB/s wr, 12 op/s 2023-09-10T03:31:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15135.234310 secs 2023-09-10T03:31:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15133.905261 secs 2023-09-10T03:31:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:43 smithi170 ceph-mon[102832]: pgmap v8233: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 233 KiB/s wr, 12 op/s 2023-09-10T03:31:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15135.234310 secs 2023-09-10T03:31:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15133.905261 secs 2023-09-10T03:31:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:43 smithi082 ceph-mon[98694]: pgmap v8233: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 233 KiB/s wr, 12 op/s 2023-09-10T03:31:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15135.234310 secs 2023-09-10T03:31:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15135.945689 secs 2023-09-10T03:31:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15135.945689 secs 2023-09-10T03:31:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15135.945689 secs 2023-09-10T03:31:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:45 smithi098 ceph-mon[103124]: pgmap v8234: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 233 KiB/s wr, 12 op/s 2023-09-10T03:31:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:45 smithi170 ceph-mon[102832]: pgmap v8234: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 233 KiB/s wr, 12 op/s 2023-09-10T03:31:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:45 smithi082 ceph-mon[98694]: pgmap v8234: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 233 KiB/s wr, 12 op/s 2023-09-10T03:31:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:47 smithi170 ceph-mon[102832]: pgmap v8235: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 292 KiB/s wr, 18 op/s 2023-09-10T03:31:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:47 smithi082 ceph-mon[98694]: pgmap v8235: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 292 KiB/s wr, 18 op/s 2023-09-10T03:31:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:47 smithi098 ceph-mon[103124]: pgmap v8235: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 292 KiB/s wr, 18 op/s 2023-09-10T03:31:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15138.905323 secs 2023-09-10T03:31:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15140.234369 secs 2023-09-10T03:31:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15138.905323 secs 2023-09-10T03:31:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15140.234369 secs 2023-09-10T03:31:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15138.905323 secs 2023-09-10T03:31:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15140.234369 secs 2023-09-10T03:31:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15140.945753 secs 2023-09-10T03:31:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:49 smithi170 ceph-mon[102832]: pgmap v8236: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 255 KiB/s wr, 12 op/s 2023-09-10T03:31:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:31:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:31:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15140.945753 secs 2023-09-10T03:31:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:49 smithi082 ceph-mon[98694]: pgmap v8236: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 255 KiB/s wr, 12 op/s 2023-09-10T03:31:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:31:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:31:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15140.945753 secs 2023-09-10T03:31:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:49 smithi098 ceph-mon[103124]: pgmap v8236: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 255 KiB/s wr, 12 op/s 2023-09-10T03:31:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:31:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:31:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:51 smithi170 ceph-mon[102832]: pgmap v8237: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 396 KiB/s wr, 18 op/s 2023-09-10T03:31:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:31:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:31:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:51 smithi082 ceph-mon[98694]: pgmap v8237: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 396 KiB/s wr, 18 op/s 2023-09-10T03:31:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:31:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:31:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:51 smithi098 ceph-mon[103124]: pgmap v8237: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 396 KiB/s wr, 18 op/s 2023-09-10T03:31:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:31:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:31:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15143.905382 secs 2023-09-10T03:31:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:53 smithi098 ceph-mon[103124]: pgmap v8238: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 199 KiB/s wr, 12 op/s 2023-09-10T03:31:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15145.234428 secs 2023-09-10T03:31:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15143.905382 secs 2023-09-10T03:31:54.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:53 smithi170 ceph-mon[102832]: pgmap v8238: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 199 KiB/s wr, 12 op/s 2023-09-10T03:31:54.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15145.234428 secs 2023-09-10T03:31:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15143.905382 secs 2023-09-10T03:31:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:53 smithi082 ceph-mon[98694]: pgmap v8238: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 199 KiB/s wr, 12 op/s 2023-09-10T03:31:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15145.234428 secs 2023-09-10T03:31:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15145.945818 secs 2023-09-10T03:31:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15145.945818 secs 2023-09-10T03:31:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15145.945818 secs 2023-09-10T03:31:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:55 smithi098 ceph-mon[103124]: pgmap v8239: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 199 KiB/s wr, 12 op/s 2023-09-10T03:31:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:55 smithi170 ceph-mon[102832]: pgmap v8239: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 199 KiB/s wr, 12 op/s 2023-09-10T03:31:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:55 smithi082 ceph-mon[98694]: pgmap v8239: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 199 KiB/s wr, 12 op/s 2023-09-10T03:31:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:57 smithi170 ceph-mon[102832]: pgmap v8240: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 284 KiB/s wr, 17 op/s 2023-09-10T03:31:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:57 smithi082 ceph-mon[98694]: pgmap v8240: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 284 KiB/s wr, 17 op/s 2023-09-10T03:31:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:57 smithi098 ceph-mon[103124]: pgmap v8240: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 284 KiB/s wr, 17 op/s 2023-09-10T03:31:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15148.905445 secs 2023-09-10T03:31:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15150.234494 secs 2023-09-10T03:31:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15148.905445 secs 2023-09-10T03:31:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15150.234494 secs 2023-09-10T03:31:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15148.905445 secs 2023-09-10T03:31:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15150.234494 secs 2023-09-10T03:32:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15150.945887 secs 2023-09-10T03:32:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:31:59 smithi170 ceph-mon[102832]: pgmap v8241: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 226 KiB/s wr, 11 op/s 2023-09-10T03:32:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15150.945887 secs 2023-09-10T03:32:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:31:59 smithi082 ceph-mon[98694]: pgmap v8241: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 226 KiB/s wr, 11 op/s 2023-09-10T03:32:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15150.945887 secs 2023-09-10T03:32:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:31:59 smithi098 ceph-mon[103124]: pgmap v8241: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 226 KiB/s wr, 11 op/s 2023-09-10T03:32:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:01 smithi170 ceph-mon[102832]: pgmap v8242: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 302 KiB/s wr, 16 op/s 2023-09-10T03:32:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:01 smithi082 ceph-mon[98694]: pgmap v8242: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 302 KiB/s wr, 16 op/s 2023-09-10T03:32:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:01 smithi098 ceph-mon[103124]: pgmap v8242: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 302 KiB/s wr, 16 op/s 2023-09-10T03:32:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15153.905546 secs 2023-09-10T03:32:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:03 smithi098 ceph-mon[103124]: pgmap v8243: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T03:32:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:32:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15155.234554 secs 2023-09-10T03:32:04.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:32:04.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:32:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15153.905546 secs 2023-09-10T03:32:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:03 smithi170 ceph-mon[102832]: pgmap v8243: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T03:32:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:32:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15155.234554 secs 2023-09-10T03:32:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:32:04.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:32:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15153.905546 secs 2023-09-10T03:32:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:03 smithi082 ceph-mon[98694]: pgmap v8243: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T03:32:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:32:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15155.234554 secs 2023-09-10T03:32:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:32:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:32:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15155.945954 secs 2023-09-10T03:32:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:32:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:32:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:32:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15155.945954 secs 2023-09-10T03:32:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:32:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:32:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:32:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15155.945954 secs 2023-09-10T03:32:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:32:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:32:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:32:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:05 smithi098 ceph-mon[103124]: pgmap v8244: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T03:32:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:05 smithi170 ceph-mon[102832]: pgmap v8244: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T03:32:06.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:05 smithi082 ceph-mon[98694]: pgmap v8244: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T03:32:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:07 smithi170 ceph-mon[102832]: pgmap v8245: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 252 KiB/s wr, 30 op/s 2023-09-10T03:32:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:07 smithi082 ceph-mon[98694]: pgmap v8245: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 252 KiB/s wr, 30 op/s 2023-09-10T03:32:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:07 smithi098 ceph-mon[103124]: pgmap v8245: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 252 KiB/s wr, 30 op/s 2023-09-10T03:32:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15158.905606 secs 2023-09-10T03:32:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:08 smithi170 ceph-mon[102832]: osdmap e583: 12 total, 12 up, 12 in 2023-09-10T03:32:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15160.234613 secs 2023-09-10T03:32:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15158.905606 secs 2023-09-10T03:32:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:08 smithi082 ceph-mon[98694]: osdmap e583: 12 total, 12 up, 12 in 2023-09-10T03:32:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15160.234613 secs 2023-09-10T03:32:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15158.905606 secs 2023-09-10T03:32:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:08 smithi098 ceph-mon[103124]: osdmap e583: 12 total, 12 up, 12 in 2023-09-10T03:32:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15160.234613 secs 2023-09-10T03:32:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15160.946020 secs 2023-09-10T03:32:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:09 smithi170 ceph-mon[102832]: pgmap v8247: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 200 KiB/s wr, 30 op/s 2023-09-10T03:32:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15160.946020 secs 2023-09-10T03:32:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:09 smithi082 ceph-mon[98694]: pgmap v8247: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 200 KiB/s wr, 30 op/s 2023-09-10T03:32:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15160.946020 secs 2023-09-10T03:32:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:09 smithi098 ceph-mon[103124]: pgmap v8247: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 200 KiB/s wr, 30 op/s 2023-09-10T03:32:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:11 smithi170 ceph-mon[102832]: pgmap v8248: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 30 op/s 2023-09-10T03:32:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:11 smithi082 ceph-mon[98694]: pgmap v8248: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 30 op/s 2023-09-10T03:32:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:11 smithi098 ceph-mon[103124]: pgmap v8248: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 30 op/s 2023-09-10T03:32:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15163.905707 secs 2023-09-10T03:32:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:13 smithi098 ceph-mon[103124]: osdmap e584: 12 total, 12 up, 12 in 2023-09-10T03:32:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:13 smithi098 ceph-mon[103124]: pgmap v8250: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 136 KiB/s wr, 37 op/s 2023-09-10T03:32:14.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:32:14.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:32:14.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15165.234675 secs 2023-09-10T03:32:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15163.905707 secs 2023-09-10T03:32:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:13 smithi170 ceph-mon[102832]: osdmap e584: 12 total, 12 up, 12 in 2023-09-10T03:32:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:13 smithi170 ceph-mon[102832]: pgmap v8250: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 136 KiB/s wr, 37 op/s 2023-09-10T03:32:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:32:14.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:32:14.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15165.234675 secs 2023-09-10T03:32:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15163.905707 secs 2023-09-10T03:32:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:13 smithi082 ceph-mon[98694]: osdmap e584: 12 total, 12 up, 12 in 2023-09-10T03:32:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:13 smithi082 ceph-mon[98694]: pgmap v8250: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 136 KiB/s wr, 37 op/s 2023-09-10T03:32:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:32:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:32:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15165.234675 secs 2023-09-10T03:32:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15165.946083 secs 2023-09-10T03:32:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15165.946083 secs 2023-09-10T03:32:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15165.946083 secs 2023-09-10T03:32:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:15 smithi170 ceph-mon[102832]: pgmap v8251: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T03:32:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:15 smithi082 ceph-mon[98694]: pgmap v8251: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T03:32:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:15 smithi098 ceph-mon[103124]: pgmap v8251: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T03:32:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:32:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:32:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:32:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:32:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:32:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:32:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:17 smithi170 ceph-mon[102832]: pgmap v8252: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 67 KiB/s wr, 13 op/s 2023-09-10T03:32:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:17 smithi082 ceph-mon[98694]: pgmap v8252: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 67 KiB/s wr, 13 op/s 2023-09-10T03:32:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:17 smithi098 ceph-mon[103124]: pgmap v8252: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 67 KiB/s wr, 13 op/s 2023-09-10T03:32:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15168.905774 secs 2023-09-10T03:32:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15170.234735 secs 2023-09-10T03:32:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15168.905774 secs 2023-09-10T03:32:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15170.234735 secs 2023-09-10T03:32:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15168.905774 secs 2023-09-10T03:32:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15170.234735 secs 2023-09-10T03:32:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15170.946149 secs 2023-09-10T03:32:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:19 smithi170 ceph-mon[102832]: pgmap v8253: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 11 op/s 2023-09-10T03:32:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15170.946149 secs 2023-09-10T03:32:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:19 smithi082 ceph-mon[98694]: pgmap v8253: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 11 op/s 2023-09-10T03:32:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15170.946149 secs 2023-09-10T03:32:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:19 smithi098 ceph-mon[103124]: pgmap v8253: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 11 op/s 2023-09-10T03:32:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:21 smithi170 ceph-mon[102832]: pgmap v8254: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 11 op/s 2023-09-10T03:32:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:21 smithi082 ceph-mon[98694]: pgmap v8254: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 11 op/s 2023-09-10T03:32:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:21 smithi098 ceph-mon[103124]: pgmap v8254: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 11 op/s 2023-09-10T03:32:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:23 smithi098 ceph-mon[103124]: pgmap v8255: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 11 op/s 2023-09-10T03:32:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:32:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15175.234836 secs 2023-09-10T03:32:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:23 smithi170 ceph-mon[102832]: pgmap v8255: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 11 op/s 2023-09-10T03:32:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:32:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15175.234836 secs 2023-09-10T03:32:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:23 smithi082 ceph-mon[98694]: pgmap v8255: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 11 op/s 2023-09-10T03:32:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:32:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15175.234836 secs 2023-09-10T03:32:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15175.946215 secs 2023-09-10T03:32:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:32:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:32:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:32:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15175.946215 secs 2023-09-10T03:32:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:32:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:32:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:32:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15175.946215 secs 2023-09-10T03:32:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:32:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:32:25.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:32:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:25 smithi170 ceph-mon[102832]: pgmap v8256: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 9 op/s 2023-09-10T03:32:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:25 smithi082 ceph-mon[98694]: pgmap v8256: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 9 op/s 2023-09-10T03:32:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:25 smithi098 ceph-mon[103124]: pgmap v8256: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 9 op/s 2023-09-10T03:32:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:27 smithi170 ceph-mon[102832]: pgmap v8257: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 14 op/s 2023-09-10T03:32:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:27 smithi082 ceph-mon[98694]: pgmap v8257: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 14 op/s 2023-09-10T03:32:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:27 smithi098 ceph-mon[103124]: pgmap v8257: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 14 op/s 2023-09-10T03:32:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15180.234917 secs 2023-09-10T03:32:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15180.234917 secs 2023-09-10T03:32:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15180.234917 secs 2023-09-10T03:32:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15180.946282 secs 2023-09-10T03:32:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:29 smithi170 ceph-mon[102832]: pgmap v8258: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T03:32:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15180.946282 secs 2023-09-10T03:32:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:29 smithi082 ceph-mon[98694]: pgmap v8258: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T03:32:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15180.946282 secs 2023-09-10T03:32:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:29 smithi098 ceph-mon[103124]: pgmap v8258: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T03:32:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:31 smithi170 ceph-mon[102832]: pgmap v8259: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 235 KiB/s wr, 15 op/s 2023-09-10T03:32:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:31 smithi082 ceph-mon[98694]: pgmap v8259: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 235 KiB/s wr, 15 op/s 2023-09-10T03:32:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:31 smithi098 ceph-mon[103124]: pgmap v8259: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 235 KiB/s wr, 15 op/s 2023-09-10T03:32:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:33 smithi098 ceph-mon[103124]: pgmap v8260: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 235 KiB/s wr, 11 op/s 2023-09-10T03:32:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15185.234996 secs 2023-09-10T03:32:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:33 smithi170 ceph-mon[102832]: pgmap v8260: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 235 KiB/s wr, 11 op/s 2023-09-10T03:32:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15185.234996 secs 2023-09-10T03:32:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:33 smithi082 ceph-mon[98694]: pgmap v8260: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 235 KiB/s wr, 11 op/s 2023-09-10T03:32:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15185.234996 secs 2023-09-10T03:32:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15185.946347 secs 2023-09-10T03:32:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15185.946347 secs 2023-09-10T03:32:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15185.946347 secs 2023-09-10T03:32:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:35 smithi170 ceph-mon[102832]: pgmap v8261: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 235 KiB/s wr, 11 op/s 2023-09-10T03:32:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:35 smithi082 ceph-mon[98694]: pgmap v8261: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 235 KiB/s wr, 11 op/s 2023-09-10T03:32:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:35 smithi098 ceph-mon[103124]: pgmap v8261: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 235 KiB/s wr, 11 op/s 2023-09-10T03:32:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:37 smithi170 ceph-mon[102832]: pgmap v8262: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 254 KiB/s wr, 16 op/s 2023-09-10T03:32:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:32:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:32:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:37 smithi082 ceph-mon[98694]: pgmap v8262: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 254 KiB/s wr, 16 op/s 2023-09-10T03:32:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:32:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:32:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:37 smithi098 ceph-mon[103124]: pgmap v8262: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 254 KiB/s wr, 16 op/s 2023-09-10T03:32:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:32:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:32:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15190.235053 secs 2023-09-10T03:32:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15190.235053 secs 2023-09-10T03:32:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15190.235053 secs 2023-09-10T03:32:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15190.946415 secs 2023-09-10T03:32:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:39 smithi170 ceph-mon[102832]: pgmap v8263: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 156 KiB/s wr, 11 op/s 2023-09-10T03:32:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15190.946415 secs 2023-09-10T03:32:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:39 smithi082 ceph-mon[98694]: pgmap v8263: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 156 KiB/s wr, 11 op/s 2023-09-10T03:32:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15190.946415 secs 2023-09-10T03:32:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:39 smithi098 ceph-mon[103124]: pgmap v8263: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 156 KiB/s wr, 11 op/s 2023-09-10T03:32:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:32:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:32:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:32:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:32:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:32:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:32:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:41 smithi170 ceph-mon[102832]: pgmap v8264: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 421 KiB/s wr, 18 op/s 2023-09-10T03:32:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:41 smithi082 ceph-mon[98694]: pgmap v8264: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 421 KiB/s wr, 18 op/s 2023-09-10T03:32:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:41 smithi098 ceph-mon[103124]: pgmap v8264: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 421 KiB/s wr, 18 op/s 2023-09-10T03:32:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15194.024389 secs 2023-09-10T03:32:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:43 smithi170 ceph-mon[102832]: pgmap v8265: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 284 KiB/s wr, 11 op/s 2023-09-10T03:32:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15195.235137 secs 2023-09-10T03:32:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15194.024389 secs 2023-09-10T03:32:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:43 smithi082 ceph-mon[98694]: pgmap v8265: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 284 KiB/s wr, 11 op/s 2023-09-10T03:32:44.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15195.235137 secs 2023-09-10T03:32:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15194.024389 secs 2023-09-10T03:32:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:43 smithi098 ceph-mon[103124]: pgmap v8265: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 284 KiB/s wr, 11 op/s 2023-09-10T03:32:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15195.235137 secs 2023-09-10T03:32:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15195.946482 secs 2023-09-10T03:32:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:32:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:32:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:32:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15195.946482 secs 2023-09-10T03:32:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:32:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:32:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:32:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15195.946482 secs 2023-09-10T03:32:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:32:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:32:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:32:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:45 smithi170 ceph-mon[102832]: pgmap v8266: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 284 KiB/s wr, 11 op/s 2023-09-10T03:32:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:45 smithi082 ceph-mon[98694]: pgmap v8266: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 284 KiB/s wr, 11 op/s 2023-09-10T03:32:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:45 smithi098 ceph-mon[103124]: pgmap v8266: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 284 KiB/s wr, 11 op/s 2023-09-10T03:32:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:47 smithi170 ceph-mon[102832]: pgmap v8267: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 347 KiB/s wr, 17 op/s 2023-09-10T03:32:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15199.024510 secs 2023-09-10T03:32:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:47 smithi082 ceph-mon[98694]: pgmap v8267: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 347 KiB/s wr, 17 op/s 2023-09-10T03:32:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15199.024510 secs 2023-09-10T03:32:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:47 smithi098 ceph-mon[103124]: pgmap v8267: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 347 KiB/s wr, 17 op/s 2023-09-10T03:32:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15199.024510 secs 2023-09-10T03:32:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15200.235195 secs 2023-09-10T03:32:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15200.235195 secs 2023-09-10T03:32:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15200.235195 secs 2023-09-10T03:32:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15200.946548 secs 2023-09-10T03:32:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:49 smithi170 ceph-mon[102832]: pgmap v8268: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 328 KiB/s wr, 11 op/s 2023-09-10T03:32:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15200.946548 secs 2023-09-10T03:32:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:49 smithi082 ceph-mon[98694]: pgmap v8268: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 328 KiB/s wr, 11 op/s 2023-09-10T03:32:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15200.946548 secs 2023-09-10T03:32:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:49 smithi098 ceph-mon[103124]: pgmap v8268: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 328 KiB/s wr, 11 op/s 2023-09-10T03:32:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:51 smithi170 ceph-mon[102832]: pgmap v8269: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 370 KiB/s wr, 17 op/s 2023-09-10T03:32:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:51 smithi082 ceph-mon[98694]: pgmap v8269: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 370 KiB/s wr, 17 op/s 2023-09-10T03:32:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:51 smithi098 ceph-mon[103124]: pgmap v8269: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 370 KiB/s wr, 17 op/s 2023-09-10T03:32:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15204.024573 secs 2023-09-10T03:32:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15204.024573 secs 2023-09-10T03:32:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15204.024573 secs 2023-09-10T03:32:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:53 smithi170 ceph-mon[102832]: pgmap v8270: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T03:32:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:32:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:32:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15205.235255 secs 2023-09-10T03:32:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:53 smithi082 ceph-mon[98694]: pgmap v8270: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T03:32:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:32:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:32:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15205.235255 secs 2023-09-10T03:32:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:53 smithi098 ceph-mon[103124]: pgmap v8270: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T03:32:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:32:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:32:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15205.235255 secs 2023-09-10T03:32:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15205.946613 secs 2023-09-10T03:32:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15205.946613 secs 2023-09-10T03:32:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15205.946613 secs 2023-09-10T03:32:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:55 smithi170 ceph-mon[102832]: pgmap v8271: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T03:32:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:32:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:32:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:55 smithi082 ceph-mon[98694]: pgmap v8271: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T03:32:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:32:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:32:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:55 smithi098 ceph-mon[103124]: pgmap v8271: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T03:32:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:32:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:32:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:57 smithi170 ceph-mon[102832]: pgmap v8272: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 254 KiB/s wr, 16 op/s 2023-09-10T03:32:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15209.024634 secs 2023-09-10T03:32:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:57 smithi082 ceph-mon[98694]: pgmap v8272: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 254 KiB/s wr, 16 op/s 2023-09-10T03:32:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15209.024634 secs 2023-09-10T03:32:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:57 smithi098 ceph-mon[103124]: pgmap v8272: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 254 KiB/s wr, 16 op/s 2023-09-10T03:32:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15209.024634 secs 2023-09-10T03:32:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15210.235315 secs 2023-09-10T03:32:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15210.235315 secs 2023-09-10T03:32:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15210.235315 secs 2023-09-10T03:33:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15210.946680 secs 2023-09-10T03:33:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:32:59 smithi170 ceph-mon[102832]: pgmap v8273: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T03:33:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15210.946680 secs 2023-09-10T03:33:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:32:59 smithi082 ceph-mon[98694]: pgmap v8273: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T03:33:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15210.946680 secs 2023-09-10T03:33:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:32:59 smithi098 ceph-mon[103124]: pgmap v8273: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T03:33:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:01 smithi170 ceph-mon[102832]: pgmap v8274: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 236 KiB/s wr, 16 op/s 2023-09-10T03:33:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:01 smithi082 ceph-mon[98694]: pgmap v8274: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 236 KiB/s wr, 16 op/s 2023-09-10T03:33:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:01 smithi098 ceph-mon[103124]: pgmap v8274: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 236 KiB/s wr, 16 op/s 2023-09-10T03:33:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15214.024731 secs 2023-09-10T03:33:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15214.024731 secs 2023-09-10T03:33:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15214.024731 secs 2023-09-10T03:33:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:03 smithi170 ceph-mon[102832]: pgmap v8275: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T03:33:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15215.235379 secs 2023-09-10T03:33:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:03 smithi082 ceph-mon[98694]: pgmap v8275: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T03:33:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15215.235379 secs 2023-09-10T03:33:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:03 smithi098 ceph-mon[103124]: pgmap v8275: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T03:33:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15215.235379 secs 2023-09-10T03:33:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15215.946749 secs 2023-09-10T03:33:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:33:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:33:05.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:33:05.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:33:05.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:33:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15215.946749 secs 2023-09-10T03:33:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:33:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:33:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:33:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:33:05.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:33:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15215.946749 secs 2023-09-10T03:33:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:33:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:33:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:33:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:33:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:33:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:05 smithi170 ceph-mon[102832]: pgmap v8276: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T03:33:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:33:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:33:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:33:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:05 smithi082 ceph-mon[98694]: pgmap v8276: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T03:33:06.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:33:06.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:33:06.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:33:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:05 smithi098 ceph-mon[103124]: pgmap v8276: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T03:33:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:33:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:33:06.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:33:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:07 smithi170 ceph-mon[102832]: pgmap v8277: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 251 KiB/s wr, 30 op/s 2023-09-10T03:33:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15219.024791 secs 2023-09-10T03:33:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:07 smithi082 ceph-mon[98694]: pgmap v8277: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 251 KiB/s wr, 30 op/s 2023-09-10T03:33:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15219.024791 secs 2023-09-10T03:33:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:07 smithi098 ceph-mon[103124]: pgmap v8277: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 251 KiB/s wr, 30 op/s 2023-09-10T03:33:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15219.024791 secs 2023-09-10T03:33:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:08 smithi170 ceph-mon[102832]: osdmap e585: 12 total, 12 up, 12 in 2023-09-10T03:33:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15220.235437 secs 2023-09-10T03:33:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:08 smithi082 ceph-mon[98694]: osdmap e585: 12 total, 12 up, 12 in 2023-09-10T03:33:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15220.235437 secs 2023-09-10T03:33:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:08 smithi098 ceph-mon[103124]: osdmap e585: 12 total, 12 up, 12 in 2023-09-10T03:33:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15220.235437 secs 2023-09-10T03:33:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15220.946814 secs 2023-09-10T03:33:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:09 smithi170 ceph-mon[102832]: pgmap v8279: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T03:33:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15220.946814 secs 2023-09-10T03:33:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:09 smithi082 ceph-mon[98694]: pgmap v8279: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T03:33:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15220.946814 secs 2023-09-10T03:33:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:09 smithi098 ceph-mon[103124]: pgmap v8279: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T03:33:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:11 smithi170 ceph-mon[102832]: pgmap v8280: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T03:33:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:11 smithi082 ceph-mon[98694]: pgmap v8280: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T03:33:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:11 smithi098 ceph-mon[103124]: pgmap v8280: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T03:33:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15224.024914 secs 2023-09-10T03:33:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:12 smithi170 ceph-mon[102832]: osdmap e586: 12 total, 12 up, 12 in 2023-09-10T03:33:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15224.024914 secs 2023-09-10T03:33:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:12 smithi082 ceph-mon[98694]: osdmap e586: 12 total, 12 up, 12 in 2023-09-10T03:33:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15224.024914 secs 2023-09-10T03:33:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:12 smithi098 ceph-mon[103124]: osdmap e586: 12 total, 12 up, 12 in 2023-09-10T03:33:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:13 smithi170 ceph-mon[102832]: pgmap v8282: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 161 KiB/s wr, 37 op/s 2023-09-10T03:33:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15225.235528 secs 2023-09-10T03:33:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:13 smithi082 ceph-mon[98694]: pgmap v8282: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 161 KiB/s wr, 37 op/s 2023-09-10T03:33:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15225.235528 secs 2023-09-10T03:33:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:13 smithi098 ceph-mon[103124]: pgmap v8282: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 161 KiB/s wr, 37 op/s 2023-09-10T03:33:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15225.235528 secs 2023-09-10T03:33:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15225.946881 secs 2023-09-10T03:33:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15225.946881 secs 2023-09-10T03:33:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15225.946881 secs 2023-09-10T03:33:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:15 smithi170 ceph-mon[102832]: pgmap v8283: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 75 KiB/s wr, 7 op/s 2023-09-10T03:33:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:15 smithi082 ceph-mon[98694]: pgmap v8283: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 75 KiB/s wr, 7 op/s 2023-09-10T03:33:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:15 smithi098 ceph-mon[103124]: pgmap v8283: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 75 KiB/s wr, 7 op/s 2023-09-10T03:33:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:17 smithi170 ceph-mon[102832]: pgmap v8284: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 13 op/s 2023-09-10T03:33:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15229.025026 secs 2023-09-10T03:33:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:17 smithi082 ceph-mon[98694]: pgmap v8284: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 13 op/s 2023-09-10T03:33:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15229.025026 secs 2023-09-10T03:33:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:17 smithi098 ceph-mon[103124]: pgmap v8284: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 13 op/s 2023-09-10T03:33:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15229.025026 secs 2023-09-10T03:33:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15230.235590 secs 2023-09-10T03:33:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15230.235590 secs 2023-09-10T03:33:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15230.235590 secs 2023-09-10T03:33:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15230.946947 secs 2023-09-10T03:33:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:19 smithi170 ceph-mon[102832]: pgmap v8285: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T03:33:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15230.946947 secs 2023-09-10T03:33:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:19 smithi082 ceph-mon[98694]: pgmap v8285: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T03:33:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15230.946947 secs 2023-09-10T03:33:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:19 smithi098 ceph-mon[103124]: pgmap v8285: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T03:33:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:21 smithi170 ceph-mon[102832]: pgmap v8286: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T03:33:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:21 smithi082 ceph-mon[98694]: pgmap v8286: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T03:33:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:21 smithi098 ceph-mon[103124]: pgmap v8286: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T03:33:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:23 smithi170 ceph-mon[102832]: pgmap v8287: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T03:33:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:33:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15235.235648 secs 2023-09-10T03:33:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:23 smithi082 ceph-mon[98694]: pgmap v8287: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T03:33:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:33:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15235.235648 secs 2023-09-10T03:33:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:23 smithi098 ceph-mon[103124]: pgmap v8287: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 76 KiB/s wr, 11 op/s 2023-09-10T03:33:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:33:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15235.235648 secs 2023-09-10T03:33:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15235.947014 secs 2023-09-10T03:33:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:33:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:33:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:33:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15235.947014 secs 2023-09-10T03:33:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:33:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:33:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:33:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15235.947014 secs 2023-09-10T03:33:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:33:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:33:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:33:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:25 smithi170 ceph-mon[102832]: pgmap v8288: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 9 op/s 2023-09-10T03:33:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:25 smithi082 ceph-mon[98694]: pgmap v8288: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 9 op/s 2023-09-10T03:33:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:25 smithi098 ceph-mon[103124]: pgmap v8288: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 9 op/s 2023-09-10T03:33:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:27 smithi170 ceph-mon[102832]: pgmap v8289: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 127 KiB/s wr, 14 op/s 2023-09-10T03:33:28.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:27 smithi082 ceph-mon[98694]: pgmap v8289: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 127 KiB/s wr, 14 op/s 2023-09-10T03:33:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:27 smithi098 ceph-mon[103124]: pgmap v8289: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 127 KiB/s wr, 14 op/s 2023-09-10T03:33:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15240.235708 secs 2023-09-10T03:33:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15240.235708 secs 2023-09-10T03:33:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15240.235708 secs 2023-09-10T03:33:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15240.947077 secs 2023-09-10T03:33:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:29 smithi170 ceph-mon[102832]: pgmap v8290: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 9 op/s 2023-09-10T03:33:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15240.947077 secs 2023-09-10T03:33:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:29 smithi082 ceph-mon[98694]: pgmap v8290: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 9 op/s 2023-09-10T03:33:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15240.947077 secs 2023-09-10T03:33:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:29 smithi098 ceph-mon[103124]: pgmap v8290: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 9 op/s 2023-09-10T03:33:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:31 smithi170 ceph-mon[102832]: pgmap v8291: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 191 KiB/s wr, 15 op/s 2023-09-10T03:33:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:31 smithi082 ceph-mon[98694]: pgmap v8291: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 191 KiB/s wr, 15 op/s 2023-09-10T03:33:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:31 smithi098 ceph-mon[103124]: pgmap v8291: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 191 KiB/s wr, 15 op/s 2023-09-10T03:33:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:33 smithi170 ceph-mon[102832]: pgmap v8292: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 10 op/s 2023-09-10T03:33:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:33:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:33:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15245.235766 secs 2023-09-10T03:33:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:33 smithi082 ceph-mon[98694]: pgmap v8292: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 10 op/s 2023-09-10T03:33:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:33:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:33:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15245.235766 secs 2023-09-10T03:33:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:33 smithi098 ceph-mon[103124]: pgmap v8292: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 10 op/s 2023-09-10T03:33:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:33:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:33:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15245.235766 secs 2023-09-10T03:33:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15245.947144 secs 2023-09-10T03:33:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15245.947144 secs 2023-09-10T03:33:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15245.947144 secs 2023-09-10T03:33:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:35 smithi170 ceph-mon[102832]: pgmap v8293: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 10 op/s 2023-09-10T03:33:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:35 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:33:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:35 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:33:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:35 smithi082 ceph-mon[98694]: pgmap v8293: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 10 op/s 2023-09-10T03:33:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:35 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:33:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:35 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:33:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:35 smithi098 ceph-mon[103124]: pgmap v8293: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 10 op/s 2023-09-10T03:33:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:35 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:33:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:35 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:33:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:37 smithi170 ceph-mon[102832]: pgmap v8294: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 318 KiB/s wr, 38 op/s 2023-09-10T03:33:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15249.025348 secs 2023-09-10T03:33:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:37 smithi082 ceph-mon[98694]: pgmap v8294: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 318 KiB/s wr, 38 op/s 2023-09-10T03:33:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15249.025348 secs 2023-09-10T03:33:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:37 smithi098 ceph-mon[103124]: pgmap v8294: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 318 KiB/s wr, 38 op/s 2023-09-10T03:33:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15249.025348 secs 2023-09-10T03:33:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15250.235823 secs 2023-09-10T03:33:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15250.235823 secs 2023-09-10T03:33:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15250.235823 secs 2023-09-10T03:33:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15250.947208 secs 2023-09-10T03:33:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:39 smithi170 ceph-mon[102832]: pgmap v8295: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 254 KiB/s wr, 33 op/s 2023-09-10T03:33:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:39 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:33:40.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:39 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:33:40.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:39 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:33:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15250.947208 secs 2023-09-10T03:33:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:39 smithi082 ceph-mon[98694]: pgmap v8295: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 254 KiB/s wr, 33 op/s 2023-09-10T03:33:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:39 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:33:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:39 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:33:40.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:39 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:33:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15250.947208 secs 2023-09-10T03:33:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:39 smithi098 ceph-mon[103124]: pgmap v8295: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 254 KiB/s wr, 33 op/s 2023-09-10T03:33:40.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:39 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:33:40.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:39 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:33:40.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:39 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:33:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:41 smithi170 ceph-mon[102832]: pgmap v8296: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 422 KiB/s wr, 40 op/s 2023-09-10T03:33:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:41 smithi082 ceph-mon[98694]: pgmap v8296: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 422 KiB/s wr, 40 op/s 2023-09-10T03:33:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:41 smithi098 ceph-mon[103124]: pgmap v8296: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 422 KiB/s wr, 40 op/s 2023-09-10T03:33:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15254.025430 secs 2023-09-10T03:33:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15254.025430 secs 2023-09-10T03:33:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15254.025430 secs 2023-09-10T03:33:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:43 smithi170 ceph-mon[102832]: pgmap v8297: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 358 KiB/s wr, 35 op/s 2023-09-10T03:33:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15255.235881 secs 2023-09-10T03:33:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:43 smithi082 ceph-mon[98694]: pgmap v8297: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 358 KiB/s wr, 35 op/s 2023-09-10T03:33:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15255.235881 secs 2023-09-10T03:33:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:43 smithi098 ceph-mon[103124]: pgmap v8297: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 358 KiB/s wr, 35 op/s 2023-09-10T03:33:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15255.235881 secs 2023-09-10T03:33:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15255.947276 secs 2023-09-10T03:33:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15255.947276 secs 2023-09-10T03:33:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15255.947276 secs 2023-09-10T03:33:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:45 smithi170 ceph-mon[102832]: pgmap v8298: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 358 KiB/s wr, 35 op/s 2023-09-10T03:33:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:45 smithi082 ceph-mon[98694]: pgmap v8298: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 358 KiB/s wr, 35 op/s 2023-09-10T03:33:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:45 smithi098 ceph-mon[103124]: pgmap v8298: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 358 KiB/s wr, 35 op/s 2023-09-10T03:33:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:47 smithi170 ceph-mon[102832]: pgmap v8299: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 409 KiB/s wr, 41 op/s 2023-09-10T03:33:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:47 smithi082 ceph-mon[98694]: pgmap v8299: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 409 KiB/s wr, 41 op/s 2023-09-10T03:33:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:47 smithi098 ceph-mon[103124]: pgmap v8299: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 409 KiB/s wr, 41 op/s 2023-09-10T03:33:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15260.235969 secs 2023-09-10T03:33:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15260.235969 secs 2023-09-10T03:33:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15260.235969 secs 2023-09-10T03:33:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15260.947338 secs 2023-09-10T03:33:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:49 smithi170 ceph-mon[102832]: pgmap v8300: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 219 KiB/s wr, 12 op/s 2023-09-10T03:33:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:33:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:33:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15260.947338 secs 2023-09-10T03:33:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:49 smithi082 ceph-mon[98694]: pgmap v8300: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 219 KiB/s wr, 12 op/s 2023-09-10T03:33:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:33:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:33:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15260.947338 secs 2023-09-10T03:33:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:49 smithi098 ceph-mon[103124]: pgmap v8300: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 219 KiB/s wr, 12 op/s 2023-09-10T03:33:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:33:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:33:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:50 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:33:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:50 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:33:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:50 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:33:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:51 smithi170 ceph-mon[102832]: pgmap v8301: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 299 KiB/s wr, 19 op/s 2023-09-10T03:33:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:51 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:33:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:33:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:33:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:51 smithi082 ceph-mon[98694]: pgmap v8301: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 299 KiB/s wr, 19 op/s 2023-09-10T03:33:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:51 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:33:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:33:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:33:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:51 smithi098 ceph-mon[103124]: pgmap v8301: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 299 KiB/s wr, 19 op/s 2023-09-10T03:33:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:51 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:33:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:33:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:33:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15264.025556 secs 2023-09-10T03:33:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15264.025556 secs 2023-09-10T03:33:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15264.025556 secs 2023-09-10T03:33:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:53 smithi170 ceph-mon[102832]: pgmap v8302: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T03:33:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15265.236027 secs 2023-09-10T03:33:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:53 smithi082 ceph-mon[98694]: pgmap v8302: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T03:33:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15265.236027 secs 2023-09-10T03:33:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:53 smithi098 ceph-mon[103124]: pgmap v8302: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T03:33:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15265.236027 secs 2023-09-10T03:33:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15265.947403 secs 2023-09-10T03:33:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15265.947403 secs 2023-09-10T03:33:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15265.947403 secs 2023-09-10T03:33:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:55 smithi170 ceph-mon[102832]: pgmap v8303: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T03:33:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:33:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:33:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:55 smithi082 ceph-mon[98694]: pgmap v8303: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T03:33:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:33:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:33:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:55 smithi098 ceph-mon[103124]: pgmap v8303: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T03:33:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:33:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:33:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:57 smithi170 ceph-mon[102832]: pgmap v8304: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 242 KiB/s wr, 18 op/s 2023-09-10T03:33:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15269.025616 secs 2023-09-10T03:33:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:57 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:33:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:57 smithi082 ceph-mon[98694]: pgmap v8304: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 242 KiB/s wr, 18 op/s 2023-09-10T03:33:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15269.025616 secs 2023-09-10T03:33:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:57 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:33:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:57 smithi098 ceph-mon[103124]: pgmap v8304: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 242 KiB/s wr, 18 op/s 2023-09-10T03:33:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15269.025616 secs 2023-09-10T03:33:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:57 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:33:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15270.236086 secs 2023-09-10T03:33:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15270.236086 secs 2023-09-10T03:33:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15270.236086 secs 2023-09-10T03:34:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15270.947475 secs 2023-09-10T03:34:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:33:59 smithi170 ceph-mon[102832]: pgmap v8305: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 190 KiB/s wr, 12 op/s 2023-09-10T03:34:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15270.947475 secs 2023-09-10T03:34:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:33:59 smithi082 ceph-mon[98694]: pgmap v8305: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 190 KiB/s wr, 12 op/s 2023-09-10T03:34:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15270.947475 secs 2023-09-10T03:34:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:33:59 smithi098 ceph-mon[103124]: pgmap v8305: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 190 KiB/s wr, 12 op/s 2023-09-10T03:34:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:01 smithi170 ceph-mon[102832]: pgmap v8306: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 235 KiB/s wr, 17 op/s 2023-09-10T03:34:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:01 smithi082 ceph-mon[98694]: pgmap v8306: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 235 KiB/s wr, 17 op/s 2023-09-10T03:34:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:01 smithi098 ceph-mon[103124]: pgmap v8306: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 235 KiB/s wr, 17 op/s 2023-09-10T03:34:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15274.025678 secs 2023-09-10T03:34:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15274.025678 secs 2023-09-10T03:34:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15274.025678 secs 2023-09-10T03:34:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:03 smithi170 ceph-mon[102832]: pgmap v8307: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T03:34:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15275.236145 secs 2023-09-10T03:34:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:03 smithi082 ceph-mon[98694]: pgmap v8307: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T03:34:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15275.236145 secs 2023-09-10T03:34:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:03 smithi098 ceph-mon[103124]: pgmap v8307: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T03:34:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15275.236145 secs 2023-09-10T03:34:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15275.947484 secs 2023-09-10T03:34:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15275.947484 secs 2023-09-10T03:34:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15275.947484 secs 2023-09-10T03:34:06.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:05 smithi082 ceph-mon[98694]: pgmap v8308: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T03:34:06.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:34:06.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:34:06.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:34:06.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:34:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:05 smithi098 ceph-mon[103124]: pgmap v8308: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T03:34:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:34:06.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:34:06.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:34:06.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:34:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:05 smithi170 ceph-mon[102832]: pgmap v8308: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T03:34:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:34:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:34:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:34:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:34:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:07 smithi082 ceph-mon[98694]: pgmap v8309: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 201 KiB/s wr, 30 op/s 2023-09-10T03:34:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15279.025749 secs 2023-09-10T03:34:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:07 smithi098 ceph-mon[103124]: pgmap v8309: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 201 KiB/s wr, 30 op/s 2023-09-10T03:34:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15279.025749 secs 2023-09-10T03:34:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:07 smithi170 ceph-mon[102832]: pgmap v8309: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 201 KiB/s wr, 30 op/s 2023-09-10T03:34:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15279.025749 secs 2023-09-10T03:34:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:08 smithi082 ceph-mon[98694]: osdmap e587: 12 total, 12 up, 12 in 2023-09-10T03:34:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15280.236226 secs 2023-09-10T03:34:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:08 smithi098 ceph-mon[103124]: osdmap e587: 12 total, 12 up, 12 in 2023-09-10T03:34:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15280.236226 secs 2023-09-10T03:34:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:08 smithi170 ceph-mon[102832]: osdmap e587: 12 total, 12 up, 12 in 2023-09-10T03:34:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15280.236226 secs 2023-09-10T03:34:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15280.947551 secs 2023-09-10T03:34:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:09 smithi082 ceph-mon[98694]: pgmap v8311: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T03:34:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15280.947551 secs 2023-09-10T03:34:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:09 smithi098 ceph-mon[103124]: pgmap v8311: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T03:34:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15280.947551 secs 2023-09-10T03:34:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:09 smithi170 ceph-mon[102832]: pgmap v8311: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T03:34:12.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:11 smithi082 ceph-mon[98694]: pgmap v8312: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 28 op/s 2023-09-10T03:34:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:11 smithi098 ceph-mon[103124]: pgmap v8312: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 28 op/s 2023-09-10T03:34:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:11 smithi170 ceph-mon[102832]: pgmap v8312: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 28 op/s 2023-09-10T03:34:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15284.025779 secs 2023-09-10T03:34:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:12 smithi082 ceph-mon[98694]: osdmap e588: 12 total, 12 up, 12 in 2023-09-10T03:34:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15284.025779 secs 2023-09-10T03:34:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:12 smithi098 ceph-mon[103124]: osdmap e588: 12 total, 12 up, 12 in 2023-09-10T03:34:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15284.025779 secs 2023-09-10T03:34:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:12 smithi170 ceph-mon[102832]: osdmap e588: 12 total, 12 up, 12 in 2023-09-10T03:34:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:13 smithi082 ceph-mon[98694]: pgmap v8314: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 69 KiB/s wr, 35 op/s 2023-09-10T03:34:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15285.236310 secs 2023-09-10T03:34:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:13 smithi098 ceph-mon[103124]: pgmap v8314: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 69 KiB/s wr, 35 op/s 2023-09-10T03:34:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15285.236310 secs 2023-09-10T03:34:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:13 smithi170 ceph-mon[102832]: pgmap v8314: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 69 KiB/s wr, 35 op/s 2023-09-10T03:34:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15285.236310 secs 2023-09-10T03:34:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15285.947615 secs 2023-09-10T03:34:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15285.947615 secs 2023-09-10T03:34:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15285.947615 secs 2023-09-10T03:34:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:15 smithi082 ceph-mon[98694]: pgmap v8315: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T03:34:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:15 smithi098 ceph-mon[103124]: pgmap v8315: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T03:34:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:15 smithi170 ceph-mon[102832]: pgmap v8315: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T03:34:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:17 smithi082 ceph-mon[98694]: pgmap v8316: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 69 KiB/s wr, 13 op/s 2023-09-10T03:34:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15289.025843 secs 2023-09-10T03:34:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:17 smithi098 ceph-mon[103124]: pgmap v8316: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 69 KiB/s wr, 13 op/s 2023-09-10T03:34:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15289.025843 secs 2023-09-10T03:34:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:17 smithi170 ceph-mon[102832]: pgmap v8316: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 69 KiB/s wr, 13 op/s 2023-09-10T03:34:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15289.025843 secs 2023-09-10T03:34:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15290.236390 secs 2023-09-10T03:34:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15290.236390 secs 2023-09-10T03:34:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15290.236390 secs 2023-09-10T03:34:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15290.947678 secs 2023-09-10T03:34:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:19 smithi098 ceph-mon[103124]: pgmap v8317: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T03:34:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15290.947678 secs 2023-09-10T03:34:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:19 smithi170 ceph-mon[102832]: pgmap v8317: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T03:34:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15290.947678 secs 2023-09-10T03:34:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:19 smithi082 ceph-mon[98694]: pgmap v8317: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T03:34:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:21 smithi098 ceph-mon[103124]: pgmap v8318: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T03:34:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:21 smithi170 ceph-mon[102832]: pgmap v8318: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T03:34:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:21 smithi082 ceph-mon[98694]: pgmap v8318: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T03:34:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:23 smithi170 ceph-mon[102832]: pgmap v8319: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T03:34:24.207 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:34:24.207 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15295.236453 secs 2023-09-10T03:34:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:23 smithi082 ceph-mon[98694]: pgmap v8319: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T03:34:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:34:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15295.236453 secs 2023-09-10T03:34:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:23 smithi098 ceph-mon[103124]: pgmap v8319: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T03:34:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:34:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15295.236453 secs 2023-09-10T03:34:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15295.947746 secs 2023-09-10T03:34:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:34:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:34:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:34:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15295.947746 secs 2023-09-10T03:34:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:34:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:34:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:34:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15295.947746 secs 2023-09-10T03:34:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:34:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:34:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:34:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:25 smithi170 ceph-mon[102832]: pgmap v8320: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 9 op/s 2023-09-10T03:34:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:25 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:34:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:25 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:34:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:25 smithi082 ceph-mon[98694]: pgmap v8320: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 9 op/s 2023-09-10T03:34:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:25 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:34:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:25 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:34:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:25 smithi098 ceph-mon[103124]: pgmap v8320: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 9 op/s 2023-09-10T03:34:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:25 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:34:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:25 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:34:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:27 smithi170 ceph-mon[102832]: pgmap v8321: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 166 KiB/s wr, 15 op/s 2023-09-10T03:34:28.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:27 smithi082 ceph-mon[98694]: pgmap v8321: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 166 KiB/s wr, 15 op/s 2023-09-10T03:34:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:27 smithi098 ceph-mon[103124]: pgmap v8321: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 166 KiB/s wr, 15 op/s 2023-09-10T03:34:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15300.236497 secs 2023-09-10T03:34:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15300.236497 secs 2023-09-10T03:34:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15300.236497 secs 2023-09-10T03:34:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15300.947811 secs 2023-09-10T03:34:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:29 smithi170 ceph-mon[102832]: pgmap v8322: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 9 op/s 2023-09-10T03:34:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15300.947811 secs 2023-09-10T03:34:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:29 smithi082 ceph-mon[98694]: pgmap v8322: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 9 op/s 2023-09-10T03:34:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15300.947811 secs 2023-09-10T03:34:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:29 smithi098 ceph-mon[103124]: pgmap v8322: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 9 op/s 2023-09-10T03:34:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:31 smithi170 ceph-mon[102832]: pgmap v8323: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 16 op/s 2023-09-10T03:34:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:31 smithi082 ceph-mon[98694]: pgmap v8323: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 16 op/s 2023-09-10T03:34:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:31 smithi098 ceph-mon[103124]: pgmap v8323: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 16 op/s 2023-09-10T03:34:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:33 smithi170 ceph-mon[102832]: pgmap v8324: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 11 op/s 2023-09-10T03:34:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15305.236559 secs 2023-09-10T03:34:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:33 smithi082 ceph-mon[98694]: pgmap v8324: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 11 op/s 2023-09-10T03:34:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15305.236559 secs 2023-09-10T03:34:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:33 smithi098 ceph-mon[103124]: pgmap v8324: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 11 op/s 2023-09-10T03:34:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15305.236559 secs 2023-09-10T03:34:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15305.947876 secs 2023-09-10T03:34:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15305.947876 secs 2023-09-10T03:34:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15305.947876 secs 2023-09-10T03:34:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:35 smithi170 ceph-mon[102832]: pgmap v8325: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 11 op/s 2023-09-10T03:34:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:35 smithi082 ceph-mon[98694]: pgmap v8325: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 11 op/s 2023-09-10T03:34:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:35 smithi098 ceph-mon[103124]: pgmap v8325: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 11 op/s 2023-09-10T03:34:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:37 smithi170 ceph-mon[102832]: pgmap v8326: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 217 KiB/s wr, 17 op/s 2023-09-10T03:34:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:37 smithi082 ceph-mon[98694]: pgmap v8326: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 217 KiB/s wr, 17 op/s 2023-09-10T03:34:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:37 smithi098 ceph-mon[103124]: pgmap v8326: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 217 KiB/s wr, 17 op/s 2023-09-10T03:34:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15310.236616 secs 2023-09-10T03:34:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15310.236616 secs 2023-09-10T03:34:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15310.236616 secs 2023-09-10T03:34:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15310.947941 secs 2023-09-10T03:34:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:39 smithi170 ceph-mon[102832]: pgmap v8327: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 102 KiB/s wr, 12 op/s 2023-09-10T03:34:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15310.947941 secs 2023-09-10T03:34:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:39 smithi082 ceph-mon[98694]: pgmap v8327: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 102 KiB/s wr, 12 op/s 2023-09-10T03:34:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15310.947941 secs 2023-09-10T03:34:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:39 smithi098 ceph-mon[103124]: pgmap v8327: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 102 KiB/s wr, 12 op/s 2023-09-10T03:34:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:41 smithi170 ceph-mon[102832]: pgmap v8328: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 306 KiB/s wr, 18 op/s 2023-09-10T03:34:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:41 smithi082 ceph-mon[98694]: pgmap v8328: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 306 KiB/s wr, 18 op/s 2023-09-10T03:34:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:41 smithi098 ceph-mon[103124]: pgmap v8328: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 306 KiB/s wr, 18 op/s 2023-09-10T03:34:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15314.026271 secs 2023-09-10T03:34:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15314.026271 secs 2023-09-10T03:34:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15314.026271 secs 2023-09-10T03:34:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:43 smithi170 ceph-mon[102832]: pgmap v8329: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 241 KiB/s wr, 12 op/s 2023-09-10T03:34:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15315.236675 secs 2023-09-10T03:34:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:34:44.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:34:44.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:34:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:43 smithi082 ceph-mon[98694]: pgmap v8329: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 241 KiB/s wr, 12 op/s 2023-09-10T03:34:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15315.236675 secs 2023-09-10T03:34:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:34:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:34:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:34:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:43 smithi098 ceph-mon[103124]: pgmap v8329: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 241 KiB/s wr, 12 op/s 2023-09-10T03:34:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15315.236675 secs 2023-09-10T03:34:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:34:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:34:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:34:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15315.948006 secs 2023-09-10T03:34:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15315.948006 secs 2023-09-10T03:34:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15315.948006 secs 2023-09-10T03:34:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:45 smithi170 ceph-mon[102832]: pgmap v8330: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 241 KiB/s wr, 12 op/s 2023-09-10T03:34:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:45 smithi082 ceph-mon[98694]: pgmap v8330: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 241 KiB/s wr, 12 op/s 2023-09-10T03:34:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:45 smithi098 ceph-mon[103124]: pgmap v8330: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 241 KiB/s wr, 12 op/s 2023-09-10T03:34:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:47 smithi170 ceph-mon[102832]: pgmap v8331: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 264 KiB/s wr, 18 op/s 2023-09-10T03:34:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15319.026333 secs 2023-09-10T03:34:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:47 smithi082 ceph-mon[98694]: pgmap v8331: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 264 KiB/s wr, 18 op/s 2023-09-10T03:34:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15319.026333 secs 2023-09-10T03:34:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:47 smithi098 ceph-mon[103124]: pgmap v8331: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 264 KiB/s wr, 18 op/s 2023-09-10T03:34:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15319.026333 secs 2023-09-10T03:34:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15320.236722 secs 2023-09-10T03:34:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15320.236722 secs 2023-09-10T03:34:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15320.236722 secs 2023-09-10T03:34:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15320.948069 secs 2023-09-10T03:34:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:49 smithi170 ceph-mon[102832]: pgmap v8332: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 227 KiB/s wr, 12 op/s 2023-09-10T03:34:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:34:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:34:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15320.948069 secs 2023-09-10T03:34:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:49 smithi082 ceph-mon[98694]: pgmap v8332: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 227 KiB/s wr, 12 op/s 2023-09-10T03:34:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:34:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:34:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15320.948069 secs 2023-09-10T03:34:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:49 smithi098 ceph-mon[103124]: pgmap v8332: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 227 KiB/s wr, 12 op/s 2023-09-10T03:34:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:34:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:34:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:51 smithi170 ceph-mon[102832]: pgmap v8333: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 504 KiB/s wr, 18 op/s 2023-09-10T03:34:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:34:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:34:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:51 smithi082 ceph-mon[98694]: pgmap v8333: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 504 KiB/s wr, 18 op/s 2023-09-10T03:34:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:34:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:34:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:51 smithi098 ceph-mon[103124]: pgmap v8333: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 504 KiB/s wr, 18 op/s 2023-09-10T03:34:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:34:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:34:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15324.026392 secs 2023-09-10T03:34:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15324.026392 secs 2023-09-10T03:34:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15324.026392 secs 2023-09-10T03:34:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:53 smithi170 ceph-mon[102832]: pgmap v8334: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 300 KiB/s wr, 12 op/s 2023-09-10T03:34:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15325.236805 secs 2023-09-10T03:34:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:53 smithi082 ceph-mon[98694]: pgmap v8334: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 300 KiB/s wr, 12 op/s 2023-09-10T03:34:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15325.236805 secs 2023-09-10T03:34:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:53 smithi098 ceph-mon[103124]: pgmap v8334: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 300 KiB/s wr, 12 op/s 2023-09-10T03:34:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15325.236805 secs 2023-09-10T03:34:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15325.948133 secs 2023-09-10T03:34:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15325.948133 secs 2023-09-10T03:34:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15325.948133 secs 2023-09-10T03:34:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:55 smithi170 ceph-mon[102832]: pgmap v8335: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 300 KiB/s wr, 12 op/s 2023-09-10T03:34:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:55 smithi082 ceph-mon[98694]: pgmap v8335: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 300 KiB/s wr, 12 op/s 2023-09-10T03:34:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:55 smithi098 ceph-mon[103124]: pgmap v8335: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 300 KiB/s wr, 12 op/s 2023-09-10T03:34:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:57 smithi170 ceph-mon[102832]: pgmap v8336: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 340 KiB/s wr, 17 op/s 2023-09-10T03:34:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:34:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:34:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15329.026500 secs 2023-09-10T03:34:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:57 smithi082 ceph-mon[98694]: pgmap v8336: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 340 KiB/s wr, 17 op/s 2023-09-10T03:34:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:34:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:34:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15329.026500 secs 2023-09-10T03:34:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:57 smithi098 ceph-mon[103124]: pgmap v8336: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 340 KiB/s wr, 17 op/s 2023-09-10T03:34:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:34:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:34:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15329.026500 secs 2023-09-10T03:34:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15330.236863 secs 2023-09-10T03:34:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15330.236863 secs 2023-09-10T03:34:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15330.236863 secs 2023-09-10T03:35:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15330.948202 secs 2023-09-10T03:35:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:59 smithi170 ceph-mon[102832]: pgmap v8337: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 318 KiB/s wr, 11 op/s 2023-09-10T03:35:00.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:59 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:35:00.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:34:59 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:35:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15330.948202 secs 2023-09-10T03:35:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:59 smithi082 ceph-mon[98694]: pgmap v8337: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 318 KiB/s wr, 11 op/s 2023-09-10T03:35:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:59 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:35:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:34:59 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:35:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15330.948202 secs 2023-09-10T03:35:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:59 smithi098 ceph-mon[103124]: pgmap v8337: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 318 KiB/s wr, 11 op/s 2023-09-10T03:35:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:59 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:35:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:34:59 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:35:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:01 smithi170 ceph-mon[102832]: pgmap v8338: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 435 KiB/s wr, 18 op/s 2023-09-10T03:35:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:01 smithi082 ceph-mon[98694]: pgmap v8338: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 435 KiB/s wr, 18 op/s 2023-09-10T03:35:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:01 smithi098 ceph-mon[103124]: pgmap v8338: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 435 KiB/s wr, 18 op/s 2023-09-10T03:35:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15334.026593 secs 2023-09-10T03:35:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15334.026593 secs 2023-09-10T03:35:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15334.026593 secs 2023-09-10T03:35:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:03 smithi170 ceph-mon[102832]: pgmap v8339: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T03:35:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15335.236925 secs 2023-09-10T03:35:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:03 smithi082 ceph-mon[98694]: pgmap v8339: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T03:35:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15335.236925 secs 2023-09-10T03:35:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:03 smithi098 ceph-mon[103124]: pgmap v8339: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T03:35:04.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15335.236925 secs 2023-09-10T03:35:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15335.948266 secs 2023-09-10T03:35:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15335.948266 secs 2023-09-10T03:35:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15335.948266 secs 2023-09-10T03:35:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:05 smithi170 ceph-mon[102832]: pgmap v8340: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T03:35:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:05 smithi082 ceph-mon[98694]: pgmap v8340: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T03:35:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:05 smithi098 ceph-mon[103124]: pgmap v8340: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T03:35:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:35:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:35:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:35:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:35:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:35:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:35:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:35:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:35:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:35:07.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:35:07.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:35:07.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:35:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:07 smithi082 ceph-mon[98694]: pgmap v8341: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 215 KiB/s wr, 31 op/s 2023-09-10T03:35:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15339.026700 secs 2023-09-10T03:35:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:07 smithi098 ceph-mon[103124]: pgmap v8341: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 215 KiB/s wr, 31 op/s 2023-09-10T03:35:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15339.026700 secs 2023-09-10T03:35:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:07 smithi170 ceph-mon[102832]: pgmap v8341: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 215 KiB/s wr, 31 op/s 2023-09-10T03:35:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15339.026700 secs 2023-09-10T03:35:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:08 smithi082 ceph-mon[98694]: osdmap e589: 12 total, 12 up, 12 in 2023-09-10T03:35:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15340.237013 secs 2023-09-10T03:35:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:08 smithi098 ceph-mon[103124]: osdmap e589: 12 total, 12 up, 12 in 2023-09-10T03:35:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15340.237013 secs 2023-09-10T03:35:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:08 smithi170 ceph-mon[102832]: osdmap e589: 12 total, 12 up, 12 in 2023-09-10T03:35:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15340.237013 secs 2023-09-10T03:35:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15340.948330 secs 2023-09-10T03:35:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:09 smithi082 ceph-mon[98694]: pgmap v8343: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 210 KiB/s wr, 31 op/s 2023-09-10T03:35:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15340.948330 secs 2023-09-10T03:35:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:09 smithi098 ceph-mon[103124]: pgmap v8343: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 210 KiB/s wr, 31 op/s 2023-09-10T03:35:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15340.948330 secs 2023-09-10T03:35:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:09 smithi170 ceph-mon[102832]: pgmap v8343: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 210 KiB/s wr, 31 op/s 2023-09-10T03:35:12.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:11 smithi082 ceph-mon[98694]: pgmap v8344: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 117 KiB/s wr, 30 op/s 2023-09-10T03:35:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:11 smithi098 ceph-mon[103124]: pgmap v8344: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 117 KiB/s wr, 30 op/s 2023-09-10T03:35:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:11 smithi170 ceph-mon[102832]: pgmap v8344: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 117 KiB/s wr, 30 op/s 2023-09-10T03:35:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15344.026787 secs 2023-09-10T03:35:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:12 smithi082 ceph-mon[98694]: osdmap e590: 12 total, 12 up, 12 in 2023-09-10T03:35:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15344.026787 secs 2023-09-10T03:35:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:12 smithi098 ceph-mon[103124]: osdmap e590: 12 total, 12 up, 12 in 2023-09-10T03:35:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15344.026787 secs 2023-09-10T03:35:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:12 smithi170 ceph-mon[102832]: osdmap e590: 12 total, 12 up, 12 in 2023-09-10T03:35:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:13 smithi082 ceph-mon[98694]: pgmap v8346: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 146 KiB/s wr, 37 op/s 2023-09-10T03:35:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15345.237071 secs 2023-09-10T03:35:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:13 smithi098 ceph-mon[103124]: pgmap v8346: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 146 KiB/s wr, 37 op/s 2023-09-10T03:35:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15345.237071 secs 2023-09-10T03:35:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:13 smithi170 ceph-mon[102832]: pgmap v8346: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 146 KiB/s wr, 37 op/s 2023-09-10T03:35:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15345.237071 secs 2023-09-10T03:35:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15345.948398 secs 2023-09-10T03:35:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15345.948398 secs 2023-09-10T03:35:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15345.948398 secs 2023-09-10T03:35:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:15 smithi098 ceph-mon[103124]: pgmap v8347: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 7 op/s 2023-09-10T03:35:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:15 smithi170 ceph-mon[102832]: pgmap v8347: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 7 op/s 2023-09-10T03:35:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:15 smithi082 ceph-mon[98694]: pgmap v8347: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 7 op/s 2023-09-10T03:35:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:17 smithi098 ceph-mon[103124]: pgmap v8348: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 14 op/s 2023-09-10T03:35:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15349.026846 secs 2023-09-10T03:35:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:17 smithi170 ceph-mon[102832]: pgmap v8348: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 14 op/s 2023-09-10T03:35:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15349.026846 secs 2023-09-10T03:35:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:17 smithi082 ceph-mon[98694]: pgmap v8348: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 14 op/s 2023-09-10T03:35:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15349.026846 secs 2023-09-10T03:35:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15350.237158 secs 2023-09-10T03:35:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15350.237158 secs 2023-09-10T03:35:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15350.237158 secs 2023-09-10T03:35:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15350.948463 secs 2023-09-10T03:35:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:19 smithi098 ceph-mon[103124]: pgmap v8349: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T03:35:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15350.948463 secs 2023-09-10T03:35:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:19 smithi170 ceph-mon[102832]: pgmap v8349: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T03:35:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15350.948463 secs 2023-09-10T03:35:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:19 smithi082 ceph-mon[98694]: pgmap v8349: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T03:35:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:21 smithi098 ceph-mon[103124]: pgmap v8350: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T03:35:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:21 smithi170 ceph-mon[102832]: pgmap v8350: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T03:35:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:21 smithi082 ceph-mon[98694]: pgmap v8350: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T03:35:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:35:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:23 smithi170 ceph-mon[102832]: pgmap v8351: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T03:35:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15355.237218 secs 2023-09-10T03:35:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:35:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:23 smithi082 ceph-mon[98694]: pgmap v8351: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T03:35:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15355.237218 secs 2023-09-10T03:35:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:35:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:23 smithi098 ceph-mon[103124]: pgmap v8351: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 11 op/s 2023-09-10T03:35:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15355.237218 secs 2023-09-10T03:35:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15355.948490 secs 2023-09-10T03:35:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:35:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:35:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:35:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15355.948490 secs 2023-09-10T03:35:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:35:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:35:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:35:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15355.948490 secs 2023-09-10T03:35:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:35:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:35:25.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:35:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:25 smithi170 ceph-mon[102832]: pgmap v8352: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 9 op/s 2023-09-10T03:35:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:25 smithi082 ceph-mon[98694]: pgmap v8352: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 9 op/s 2023-09-10T03:35:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:25 smithi098 ceph-mon[103124]: pgmap v8352: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 51 KiB/s wr, 9 op/s 2023-09-10T03:35:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:27 smithi170 ceph-mon[102832]: pgmap v8353: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 14 op/s 2023-09-10T03:35:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:27 smithi082 ceph-mon[98694]: pgmap v8353: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 14 op/s 2023-09-10T03:35:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:27 smithi098 ceph-mon[103124]: pgmap v8353: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 14 op/s 2023-09-10T03:35:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15360.237275 secs 2023-09-10T03:35:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15360.237275 secs 2023-09-10T03:35:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15360.237275 secs 2023-09-10T03:35:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15360.948555 secs 2023-09-10T03:35:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:29 smithi170 ceph-mon[102832]: pgmap v8354: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 9 op/s 2023-09-10T03:35:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15360.948555 secs 2023-09-10T03:35:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:29 smithi082 ceph-mon[98694]: pgmap v8354: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 9 op/s 2023-09-10T03:35:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15360.948555 secs 2023-09-10T03:35:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:29 smithi098 ceph-mon[103124]: pgmap v8354: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 9 op/s 2023-09-10T03:35:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:31 smithi170 ceph-mon[102832]: pgmap v8355: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 239 KiB/s wr, 17 op/s 2023-09-10T03:35:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:31 smithi082 ceph-mon[98694]: pgmap v8355: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 239 KiB/s wr, 17 op/s 2023-09-10T03:35:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:31 smithi098 ceph-mon[103124]: pgmap v8355: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 239 KiB/s wr, 17 op/s 2023-09-10T03:35:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:33 smithi170 ceph-mon[102832]: pgmap v8356: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 239 KiB/s wr, 12 op/s 2023-09-10T03:35:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:35:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:35:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15365.237333 secs 2023-09-10T03:35:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:33 smithi082 ceph-mon[98694]: pgmap v8356: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 239 KiB/s wr, 12 op/s 2023-09-10T03:35:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:35:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:35:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15365.237333 secs 2023-09-10T03:35:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:33 smithi098 ceph-mon[103124]: pgmap v8356: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 239 KiB/s wr, 12 op/s 2023-09-10T03:35:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:35:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:35:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15365.237333 secs 2023-09-10T03:35:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15365.948620 secs 2023-09-10T03:35:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15365.948620 secs 2023-09-10T03:35:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15365.948620 secs 2023-09-10T03:35:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:35 smithi170 ceph-mon[102832]: pgmap v8357: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 239 KiB/s wr, 12 op/s 2023-09-10T03:35:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:35 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:35:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:35 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:35:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:35 smithi082 ceph-mon[98694]: pgmap v8357: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 239 KiB/s wr, 12 op/s 2023-09-10T03:35:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:35 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:35:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:35 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:35:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:35 smithi098 ceph-mon[103124]: pgmap v8357: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 239 KiB/s wr, 12 op/s 2023-09-10T03:35:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:35 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:35:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:35 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:35:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:37 smithi170 ceph-mon[102832]: pgmap v8358: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 258 KiB/s wr, 18 op/s 2023-09-10T03:35:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:37 smithi082 ceph-mon[98694]: pgmap v8358: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 258 KiB/s wr, 18 op/s 2023-09-10T03:35:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:37 smithi098 ceph-mon[103124]: pgmap v8358: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 258 KiB/s wr, 18 op/s 2023-09-10T03:35:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15370.237413 secs 2023-09-10T03:35:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15370.237413 secs 2023-09-10T03:35:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15370.237413 secs 2023-09-10T03:35:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15370.948687 secs 2023-09-10T03:35:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:39 smithi170 ceph-mon[102832]: pgmap v8359: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 12 op/s 2023-09-10T03:35:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15370.948687 secs 2023-09-10T03:35:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:39 smithi082 ceph-mon[98694]: pgmap v8359: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 12 op/s 2023-09-10T03:35:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15370.948687 secs 2023-09-10T03:35:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:39 smithi098 ceph-mon[103124]: pgmap v8359: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 12 op/s 2023-09-10T03:35:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:41 smithi170 ceph-mon[102832]: pgmap v8360: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 428 KiB/s wr, 19 op/s 2023-09-10T03:35:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:41 smithi082 ceph-mon[98694]: pgmap v8360: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 428 KiB/s wr, 19 op/s 2023-09-10T03:35:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:41 smithi098 ceph-mon[103124]: pgmap v8360: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 428 KiB/s wr, 19 op/s 2023-09-10T03:35:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:43 smithi170 ceph-mon[102832]: pgmap v8361: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 289 KiB/s wr, 11 op/s 2023-09-10T03:35:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15375.237473 secs 2023-09-10T03:35:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:43 smithi082 ceph-mon[98694]: pgmap v8361: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 289 KiB/s wr, 11 op/s 2023-09-10T03:35:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15375.237473 secs 2023-09-10T03:35:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:43 smithi098 ceph-mon[103124]: pgmap v8361: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 289 KiB/s wr, 11 op/s 2023-09-10T03:35:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15375.237473 secs 2023-09-10T03:35:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15375.948752 secs 2023-09-10T03:35:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15375.948752 secs 2023-09-10T03:35:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15375.948752 secs 2023-09-10T03:35:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:45 smithi170 ceph-mon[102832]: pgmap v8362: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 289 KiB/s wr, 11 op/s 2023-09-10T03:35:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:45 smithi082 ceph-mon[98694]: pgmap v8362: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 289 KiB/s wr, 11 op/s 2023-09-10T03:35:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:45 smithi098 ceph-mon[103124]: pgmap v8362: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 289 KiB/s wr, 11 op/s 2023-09-10T03:35:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:47 smithi170 ceph-mon[102832]: pgmap v8363: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 366 KiB/s wr, 17 op/s 2023-09-10T03:35:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:47 smithi082 ceph-mon[98694]: pgmap v8363: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 366 KiB/s wr, 17 op/s 2023-09-10T03:35:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:47 smithi098 ceph-mon[103124]: pgmap v8363: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 366 KiB/s wr, 17 op/s 2023-09-10T03:35:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15380.237495 secs 2023-09-10T03:35:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15380.237495 secs 2023-09-10T03:35:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15380.237495 secs 2023-09-10T03:35:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15380.948816 secs 2023-09-10T03:35:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:49 smithi170 ceph-mon[102832]: pgmap v8364: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 346 KiB/s wr, 12 op/s 2023-09-10T03:35:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:35:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:35:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15380.948816 secs 2023-09-10T03:35:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:49 smithi082 ceph-mon[98694]: pgmap v8364: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 346 KiB/s wr, 12 op/s 2023-09-10T03:35:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:35:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:35:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15380.948816 secs 2023-09-10T03:35:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:49 smithi098 ceph-mon[103124]: pgmap v8364: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 346 KiB/s wr, 12 op/s 2023-09-10T03:35:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:35:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:35:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:51 smithi170 ceph-mon[102832]: pgmap v8365: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 398 KiB/s wr, 18 op/s 2023-09-10T03:35:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:35:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:35:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:51 smithi082 ceph-mon[98694]: pgmap v8365: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 398 KiB/s wr, 18 op/s 2023-09-10T03:35:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:35:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:35:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:51 smithi098 ceph-mon[103124]: pgmap v8365: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 398 KiB/s wr, 18 op/s 2023-09-10T03:35:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:35:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:35:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15384.027369 secs 2023-09-10T03:35:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15384.027369 secs 2023-09-10T03:35:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15384.027369 secs 2023-09-10T03:35:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:53 smithi170 ceph-mon[102832]: pgmap v8366: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T03:35:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15385.237553 secs 2023-09-10T03:35:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:53 smithi082 ceph-mon[98694]: pgmap v8366: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T03:35:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15385.237553 secs 2023-09-10T03:35:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:53 smithi098 ceph-mon[103124]: pgmap v8366: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T03:35:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15385.237553 secs 2023-09-10T03:35:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15385.948882 secs 2023-09-10T03:35:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15385.948882 secs 2023-09-10T03:35:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15385.948882 secs 2023-09-10T03:35:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:55 smithi170 ceph-mon[102832]: pgmap v8367: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T03:35:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:35:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:35:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:35:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:55 smithi082 ceph-mon[98694]: pgmap v8367: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T03:35:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:35:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:35:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:35:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:55 smithi098 ceph-mon[103124]: pgmap v8367: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T03:35:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:35:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:35:56.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:35:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:57 smithi170 ceph-mon[102832]: pgmap v8368: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 235 KiB/s wr, 17 op/s 2023-09-10T03:35:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15389.027430 secs 2023-09-10T03:35:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:57 smithi082 ceph-mon[98694]: pgmap v8368: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 235 KiB/s wr, 17 op/s 2023-09-10T03:35:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15389.027430 secs 2023-09-10T03:35:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:57 smithi098 ceph-mon[103124]: pgmap v8368: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 235 KiB/s wr, 17 op/s 2023-09-10T03:35:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15389.027430 secs 2023-09-10T03:35:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15390.237611 secs 2023-09-10T03:35:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15390.237611 secs 2023-09-10T03:35:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15390.237611 secs 2023-09-10T03:36:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15390.948949 secs 2023-09-10T03:36:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:35:59 smithi170 ceph-mon[102832]: pgmap v8369: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T03:36:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15390.948949 secs 2023-09-10T03:36:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:35:59 smithi082 ceph-mon[98694]: pgmap v8369: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T03:36:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15390.948949 secs 2023-09-10T03:36:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:35:59 smithi098 ceph-mon[103124]: pgmap v8369: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T03:36:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:01 smithi170 ceph-mon[102832]: pgmap v8370: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 204 KiB/s wr, 16 op/s 2023-09-10T03:36:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:01 smithi082 ceph-mon[98694]: pgmap v8370: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 204 KiB/s wr, 16 op/s 2023-09-10T03:36:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:01 smithi098 ceph-mon[103124]: pgmap v8370: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 204 KiB/s wr, 16 op/s 2023-09-10T03:36:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15394.027492 secs 2023-09-10T03:36:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15394.027492 secs 2023-09-10T03:36:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15394.027492 secs 2023-09-10T03:36:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:03 smithi170 ceph-mon[102832]: pgmap v8371: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T03:36:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15395.237667 secs 2023-09-10T03:36:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:03 smithi082 ceph-mon[98694]: pgmap v8371: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T03:36:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15395.237667 secs 2023-09-10T03:36:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:03 smithi098 ceph-mon[103124]: pgmap v8371: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T03:36:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15395.237667 secs 2023-09-10T03:36:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15395.949014 secs 2023-09-10T03:36:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15395.949014 secs 2023-09-10T03:36:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15395.949014 secs 2023-09-10T03:36:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:05 smithi082 ceph-mon[98694]: pgmap v8372: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T03:36:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:05 smithi098 ceph-mon[103124]: pgmap v8372: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T03:36:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:05 smithi170 ceph-mon[102832]: pgmap v8372: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T03:36:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:36:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:36:07.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:36:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:07 smithi082 ceph-mon[98694]: pgmap v8373: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 209 KiB/s wr, 30 op/s 2023-09-10T03:36:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:36:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:36:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:36:08.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15399.027595 secs 2023-09-10T03:36:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:07 smithi098 ceph-mon[103124]: pgmap v8373: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 209 KiB/s wr, 30 op/s 2023-09-10T03:36:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:36:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:36:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:36:08.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15399.027595 secs 2023-09-10T03:36:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:07 smithi170 ceph-mon[102832]: pgmap v8373: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 209 KiB/s wr, 30 op/s 2023-09-10T03:36:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:36:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:36:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:36:08.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15399.027595 secs 2023-09-10T03:36:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15400.237724 secs 2023-09-10T03:36:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:09 smithi170 ceph-mon[102832]: osdmap e591: 12 total, 12 up, 12 in 2023-09-10T03:36:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:09 smithi170 ceph-mon[102832]: pgmap v8375: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 124 KiB/s wr, 30 op/s 2023-09-10T03:36:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15400.237724 secs 2023-09-10T03:36:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:09 smithi082 ceph-mon[98694]: osdmap e591: 12 total, 12 up, 12 in 2023-09-10T03:36:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:09 smithi082 ceph-mon[98694]: pgmap v8375: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 124 KiB/s wr, 30 op/s 2023-09-10T03:36:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15400.237724 secs 2023-09-10T03:36:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:09 smithi098 ceph-mon[103124]: osdmap e591: 12 total, 12 up, 12 in 2023-09-10T03:36:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:09 smithi098 ceph-mon[103124]: pgmap v8375: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 124 KiB/s wr, 30 op/s 2023-09-10T03:36:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15400.949079 secs 2023-09-10T03:36:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15400.949079 secs 2023-09-10T03:36:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15400.949079 secs 2023-09-10T03:36:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:11 smithi170 ceph-mon[102832]: pgmap v8376: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 129 KiB/s wr, 29 op/s 2023-09-10T03:36:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:11 smithi082 ceph-mon[98694]: pgmap v8376: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 129 KiB/s wr, 29 op/s 2023-09-10T03:36:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:11 smithi098 ceph-mon[103124]: pgmap v8376: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 129 KiB/s wr, 29 op/s 2023-09-10T03:36:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15404.027708 secs 2023-09-10T03:36:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15404.027708 secs 2023-09-10T03:36:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15404.027708 secs 2023-09-10T03:36:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:13 smithi170 ceph-mon[102832]: osdmap e592: 12 total, 12 up, 12 in 2023-09-10T03:36:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:13 smithi170 ceph-mon[102832]: pgmap v8378: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 162 KiB/s wr, 36 op/s 2023-09-10T03:36:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15405.237783 secs 2023-09-10T03:36:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:13 smithi082 ceph-mon[98694]: osdmap e592: 12 total, 12 up, 12 in 2023-09-10T03:36:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:13 smithi082 ceph-mon[98694]: pgmap v8378: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 162 KiB/s wr, 36 op/s 2023-09-10T03:36:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15405.237783 secs 2023-09-10T03:36:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:13 smithi098 ceph-mon[103124]: osdmap e592: 12 total, 12 up, 12 in 2023-09-10T03:36:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:13 smithi098 ceph-mon[103124]: pgmap v8378: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 162 KiB/s wr, 36 op/s 2023-09-10T03:36:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15405.237783 secs 2023-09-10T03:36:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15405.949143 secs 2023-09-10T03:36:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15405.949143 secs 2023-09-10T03:36:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15405.949143 secs 2023-09-10T03:36:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:15 smithi170 ceph-mon[102832]: pgmap v8379: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 77 KiB/s wr, 7 op/s 2023-09-10T03:36:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:15 smithi082 ceph-mon[98694]: pgmap v8379: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 77 KiB/s wr, 7 op/s 2023-09-10T03:36:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:15 smithi098 ceph-mon[103124]: pgmap v8379: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 77 KiB/s wr, 7 op/s 2023-09-10T03:36:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:17 smithi170 ceph-mon[102832]: pgmap v8380: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T03:36:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15409.027772 secs 2023-09-10T03:36:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:17 smithi082 ceph-mon[98694]: pgmap v8380: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T03:36:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15409.027772 secs 2023-09-10T03:36:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:17 smithi098 ceph-mon[103124]: pgmap v8380: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T03:36:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15409.027772 secs 2023-09-10T03:36:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15410.237842 secs 2023-09-10T03:36:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15410.237842 secs 2023-09-10T03:36:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15410.237842 secs 2023-09-10T03:36:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15410.949209 secs 2023-09-10T03:36:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:19 smithi170 ceph-mon[102832]: pgmap v8381: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T03:36:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15410.949209 secs 2023-09-10T03:36:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:19 smithi082 ceph-mon[98694]: pgmap v8381: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T03:36:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15410.949209 secs 2023-09-10T03:36:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:19 smithi098 ceph-mon[103124]: pgmap v8381: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T03:36:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:21 smithi170 ceph-mon[102832]: pgmap v8382: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T03:36:22.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:21 smithi082 ceph-mon[98694]: pgmap v8382: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T03:36:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:21 smithi098 ceph-mon[103124]: pgmap v8382: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T03:36:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:36:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:36:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:36:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:23 smithi170 ceph-mon[102832]: pgmap v8383: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T03:36:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:36:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:36:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:36:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15415.237930 secs 2023-09-10T03:36:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:23 smithi082 ceph-mon[98694]: pgmap v8383: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T03:36:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:36:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:36:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:36:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15415.237930 secs 2023-09-10T03:36:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:23 smithi098 ceph-mon[103124]: pgmap v8383: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T03:36:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:36:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:36:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:36:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15415.237930 secs 2023-09-10T03:36:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15415.949277 secs 2023-09-10T03:36:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15415.949277 secs 2023-09-10T03:36:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15415.949277 secs 2023-09-10T03:36:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:25 smithi170 ceph-mon[102832]: pgmap v8384: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 9 op/s 2023-09-10T03:36:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:25 smithi082 ceph-mon[98694]: pgmap v8384: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 9 op/s 2023-09-10T03:36:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:25 smithi098 ceph-mon[103124]: pgmap v8384: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 9 op/s 2023-09-10T03:36:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:27 smithi170 ceph-mon[102832]: pgmap v8385: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 129 KiB/s wr, 14 op/s 2023-09-10T03:36:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:27 smithi082 ceph-mon[98694]: pgmap v8385: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 129 KiB/s wr, 14 op/s 2023-09-10T03:36:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:27 smithi098 ceph-mon[103124]: pgmap v8385: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 129 KiB/s wr, 14 op/s 2023-09-10T03:36:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15420.237989 secs 2023-09-10T03:36:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15420.237989 secs 2023-09-10T03:36:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15420.237989 secs 2023-09-10T03:36:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15420.949342 secs 2023-09-10T03:36:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:29 smithi170 ceph-mon[102832]: pgmap v8386: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 129 KiB/s wr, 10 op/s 2023-09-10T03:36:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:29 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:36:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:29 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:36:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15420.949342 secs 2023-09-10T03:36:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:29 smithi082 ceph-mon[98694]: pgmap v8386: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 129 KiB/s wr, 10 op/s 2023-09-10T03:36:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:29 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:36:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:29 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:36:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15420.949342 secs 2023-09-10T03:36:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:29 smithi098 ceph-mon[103124]: pgmap v8386: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 129 KiB/s wr, 10 op/s 2023-09-10T03:36:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:29 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:36:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:29 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:36:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:31 smithi170 ceph-mon[102832]: pgmap v8387: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 194 KiB/s wr, 16 op/s 2023-09-10T03:36:32.243 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:31 smithi082 ceph-mon[98694]: pgmap v8387: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 194 KiB/s wr, 16 op/s 2023-09-10T03:36:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:31 smithi098 ceph-mon[103124]: pgmap v8387: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 194 KiB/s wr, 16 op/s 2023-09-10T03:36:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:33 smithi170 ceph-mon[102832]: pgmap v8388: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 130 KiB/s wr, 11 op/s 2023-09-10T03:36:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15425.238075 secs 2023-09-10T03:36:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:33 smithi082 ceph-mon[98694]: pgmap v8388: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 130 KiB/s wr, 11 op/s 2023-09-10T03:36:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15425.238075 secs 2023-09-10T03:36:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:33 smithi098 ceph-mon[103124]: pgmap v8388: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 130 KiB/s wr, 11 op/s 2023-09-10T03:36:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15425.238075 secs 2023-09-10T03:36:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15425.949407 secs 2023-09-10T03:36:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15425.949407 secs 2023-09-10T03:36:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15425.949407 secs 2023-09-10T03:36:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:35 smithi170 ceph-mon[102832]: pgmap v8389: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 130 KiB/s wr, 11 op/s 2023-09-10T03:36:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:35 smithi082 ceph-mon[98694]: pgmap v8389: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 130 KiB/s wr, 11 op/s 2023-09-10T03:36:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:35 smithi098 ceph-mon[103124]: pgmap v8389: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 130 KiB/s wr, 11 op/s 2023-09-10T03:36:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:37 smithi170 ceph-mon[102832]: pgmap v8390: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 323 KiB/s wr, 17 op/s 2023-09-10T03:36:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:36:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:36:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:37 smithi082 ceph-mon[98694]: pgmap v8390: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 323 KiB/s wr, 17 op/s 2023-09-10T03:36:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:36:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:36:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:37 smithi098 ceph-mon[103124]: pgmap v8390: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 323 KiB/s wr, 17 op/s 2023-09-10T03:36:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:36:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:36:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15430.238152 secs 2023-09-10T03:36:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15430.238152 secs 2023-09-10T03:36:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15430.238152 secs 2023-09-10T03:36:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15430.949476 secs 2023-09-10T03:36:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:39 smithi170 ceph-mon[102832]: pgmap v8391: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 258 KiB/s wr, 12 op/s 2023-09-10T03:36:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:39 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:36:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:39 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:36:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15430.949476 secs 2023-09-10T03:36:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:39 smithi082 ceph-mon[98694]: pgmap v8391: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 258 KiB/s wr, 12 op/s 2023-09-10T03:36:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:39 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:36:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:39 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:36:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15430.949476 secs 2023-09-10T03:36:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:39 smithi098 ceph-mon[103124]: pgmap v8391: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 258 KiB/s wr, 12 op/s 2023-09-10T03:36:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:39 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:36:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:39 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:36:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:41 smithi170 ceph-mon[102832]: pgmap v8392: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 426 KiB/s wr, 19 op/s 2023-09-10T03:36:42.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:41 smithi082 ceph-mon[98694]: pgmap v8392: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 426 KiB/s wr, 19 op/s 2023-09-10T03:36:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:41 smithi098 ceph-mon[103124]: pgmap v8392: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 426 KiB/s wr, 19 op/s 2023-09-10T03:36:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:43 smithi170 ceph-mon[102832]: pgmap v8393: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 361 KiB/s wr, 13 op/s 2023-09-10T03:36:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15435.238216 secs 2023-09-10T03:36:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:43 smithi082 ceph-mon[98694]: pgmap v8393: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 361 KiB/s wr, 13 op/s 2023-09-10T03:36:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15435.238216 secs 2023-09-10T03:36:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:43 smithi098 ceph-mon[103124]: pgmap v8393: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 361 KiB/s wr, 13 op/s 2023-09-10T03:36:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15435.238216 secs 2023-09-10T03:36:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15435.949487 secs 2023-09-10T03:36:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15435.949487 secs 2023-09-10T03:36:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15435.949487 secs 2023-09-10T03:36:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:45 smithi170 ceph-mon[102832]: pgmap v8394: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 361 KiB/s wr, 13 op/s 2023-09-10T03:36:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:45 smithi082 ceph-mon[98694]: pgmap v8394: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 361 KiB/s wr, 13 op/s 2023-09-10T03:36:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:45 smithi098 ceph-mon[103124]: pgmap v8394: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 361 KiB/s wr, 13 op/s 2023-09-10T03:36:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:47 smithi170 ceph-mon[102832]: pgmap v8395: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 415 KiB/s wr, 18 op/s 2023-09-10T03:36:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15439.028168 secs 2023-09-10T03:36:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:47 smithi082 ceph-mon[98694]: pgmap v8395: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 415 KiB/s wr, 18 op/s 2023-09-10T03:36:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15439.028168 secs 2023-09-10T03:36:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:47 smithi098 ceph-mon[103124]: pgmap v8395: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 415 KiB/s wr, 18 op/s 2023-09-10T03:36:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15439.028168 secs 2023-09-10T03:36:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15440.238300 secs 2023-09-10T03:36:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15440.238300 secs 2023-09-10T03:36:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15440.238300 secs 2023-09-10T03:36:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15440.949550 secs 2023-09-10T03:36:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:49 smithi170 ceph-mon[102832]: pgmap v8396: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 222 KiB/s wr, 12 op/s 2023-09-10T03:36:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15440.949550 secs 2023-09-10T03:36:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:49 smithi082 ceph-mon[98694]: pgmap v8396: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 222 KiB/s wr, 12 op/s 2023-09-10T03:36:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15440.949550 secs 2023-09-10T03:36:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:49 smithi098 ceph-mon[103124]: pgmap v8396: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 222 KiB/s wr, 12 op/s 2023-09-10T03:36:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:50 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:36:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:36:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:36:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:50 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:36:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:36:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:36:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:50 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:36:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:36:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:36:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:51 smithi082 ceph-mon[98694]: pgmap v8397: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 391 KiB/s wr, 18 op/s 2023-09-10T03:36:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:51 smithi098 ceph-mon[103124]: pgmap v8397: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 391 KiB/s wr, 18 op/s 2023-09-10T03:36:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:51 smithi170 ceph-mon[102832]: pgmap v8397: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 391 KiB/s wr, 18 op/s 2023-09-10T03:36:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15444.028234 secs 2023-09-10T03:36:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15444.028234 secs 2023-09-10T03:36:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15444.028234 secs 2023-09-10T03:36:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:53 smithi098 ceph-mon[103124]: pgmap v8398: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T03:36:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:36:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:36:54.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15445.238359 secs 2023-09-10T03:36:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:53 smithi170 ceph-mon[102832]: pgmap v8398: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T03:36:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:36:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:36:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15445.238359 secs 2023-09-10T03:36:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:53 smithi082 ceph-mon[98694]: pgmap v8398: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T03:36:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:36:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:36:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15445.238359 secs 2023-09-10T03:36:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15445.949615 secs 2023-09-10T03:36:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15445.949615 secs 2023-09-10T03:36:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15445.949615 secs 2023-09-10T03:36:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:55 smithi098 ceph-mon[103124]: pgmap v8399: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T03:36:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:36:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:36:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:55 smithi170 ceph-mon[102832]: pgmap v8399: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T03:36:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:36:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:36:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:55 smithi082 ceph-mon[98694]: pgmap v8399: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T03:36:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:36:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:36:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:57 smithi098 ceph-mon[103124]: pgmap v8400: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 335 KiB/s wr, 17 op/s 2023-09-10T03:36:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15449.028308 secs 2023-09-10T03:36:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:57 smithi170 ceph-mon[102832]: pgmap v8400: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 335 KiB/s wr, 17 op/s 2023-09-10T03:36:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15449.028308 secs 2023-09-10T03:36:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:57 smithi082 ceph-mon[98694]: pgmap v8400: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 335 KiB/s wr, 17 op/s 2023-09-10T03:36:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15449.028308 secs 2023-09-10T03:36:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15450.238445 secs 2023-09-10T03:36:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15450.238445 secs 2023-09-10T03:36:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15450.238445 secs 2023-09-10T03:37:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15450.949683 secs 2023-09-10T03:37:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:36:59 smithi098 ceph-mon[103124]: pgmap v8401: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 281 KiB/s wr, 11 op/s 2023-09-10T03:37:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15450.949683 secs 2023-09-10T03:37:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:36:59 smithi170 ceph-mon[102832]: pgmap v8401: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 281 KiB/s wr, 11 op/s 2023-09-10T03:37:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15450.949683 secs 2023-09-10T03:37:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:36:59 smithi082 ceph-mon[98694]: pgmap v8401: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 281 KiB/s wr, 11 op/s 2023-09-10T03:37:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:01 smithi098 ceph-mon[103124]: pgmap v8402: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 371 KiB/s wr, 31 op/s 2023-09-10T03:37:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:01 smithi170 ceph-mon[102832]: pgmap v8402: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 371 KiB/s wr, 31 op/s 2023-09-10T03:37:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:01 smithi082 ceph-mon[98694]: pgmap v8402: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 371 KiB/s wr, 31 op/s 2023-09-10T03:37:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15454.028367 secs 2023-09-10T03:37:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15454.028367 secs 2023-09-10T03:37:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15454.028367 secs 2023-09-10T03:37:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:03 smithi098 ceph-mon[103124]: pgmap v8403: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 203 KiB/s wr, 25 op/s 2023-09-10T03:37:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15455.238528 secs 2023-09-10T03:37:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:03 smithi170 ceph-mon[102832]: pgmap v8403: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 203 KiB/s wr, 25 op/s 2023-09-10T03:37:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15455.238528 secs 2023-09-10T03:37:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:03 smithi082 ceph-mon[98694]: pgmap v8403: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 203 KiB/s wr, 25 op/s 2023-09-10T03:37:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15455.238528 secs 2023-09-10T03:37:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15455.949747 secs 2023-09-10T03:37:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15455.949747 secs 2023-09-10T03:37:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15455.949747 secs 2023-09-10T03:37:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:05 smithi098 ceph-mon[103124]: pgmap v8404: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 203 KiB/s wr, 25 op/s 2023-09-10T03:37:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:05 smithi170 ceph-mon[102832]: pgmap v8404: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 203 KiB/s wr, 25 op/s 2023-09-10T03:37:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:05 smithi082 ceph-mon[98694]: pgmap v8404: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 203 KiB/s wr, 25 op/s 2023-09-10T03:37:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:07 smithi098 ceph-mon[103124]: pgmap v8405: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 215 KiB/s wr, 31 op/s 2023-09-10T03:37:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15459.028428 secs 2023-09-10T03:37:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:37:08.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:37:08.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:37:08.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:37:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:07 smithi170 ceph-mon[102832]: pgmap v8405: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 215 KiB/s wr, 31 op/s 2023-09-10T03:37:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15459.028428 secs 2023-09-10T03:37:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:37:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:37:08.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:37:08.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:37:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:07 smithi082 ceph-mon[98694]: pgmap v8405: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 215 KiB/s wr, 31 op/s 2023-09-10T03:37:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15459.028428 secs 2023-09-10T03:37:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:37:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:37:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:37:08.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:37:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15460.238586 secs 2023-09-10T03:37:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:09 smithi098 ceph-mon[103124]: osdmap e593: 12 total, 12 up, 12 in 2023-09-10T03:37:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15460.238586 secs 2023-09-10T03:37:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:09 smithi170 ceph-mon[102832]: osdmap e593: 12 total, 12 up, 12 in 2023-09-10T03:37:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15460.238586 secs 2023-09-10T03:37:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:09 smithi082 ceph-mon[98694]: osdmap e593: 12 total, 12 up, 12 in 2023-09-10T03:37:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15460.949813 secs 2023-09-10T03:37:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:10 smithi098 ceph-mon[103124]: pgmap v8407: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 124 KiB/s wr, 30 op/s 2023-09-10T03:37:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15460.949813 secs 2023-09-10T03:37:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:10 smithi170 ceph-mon[102832]: pgmap v8407: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 124 KiB/s wr, 30 op/s 2023-09-10T03:37:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15460.949813 secs 2023-09-10T03:37:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:10 smithi082 ceph-mon[98694]: pgmap v8407: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 124 KiB/s wr, 30 op/s 2023-09-10T03:37:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:12 smithi170 ceph-mon[102832]: pgmap v8408: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 12 op/s 2023-09-10T03:37:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:12 smithi082 ceph-mon[98694]: pgmap v8408: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 12 op/s 2023-09-10T03:37:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:12 smithi098 ceph-mon[103124]: pgmap v8408: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 12 op/s 2023-09-10T03:37:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15464.028511 secs 2023-09-10T03:37:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:13 smithi170 ceph-mon[102832]: osdmap e594: 12 total, 12 up, 12 in 2023-09-10T03:37:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15464.028511 secs 2023-09-10T03:37:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:13 smithi082 ceph-mon[98694]: osdmap e594: 12 total, 12 up, 12 in 2023-09-10T03:37:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15464.028511 secs 2023-09-10T03:37:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:13 smithi098 ceph-mon[103124]: osdmap e594: 12 total, 12 up, 12 in 2023-09-10T03:37:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:14 smithi170 ceph-mon[102832]: pgmap v8410: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 KiB/s wr, 15 op/s 2023-09-10T03:37:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15465.238645 secs 2023-09-10T03:37:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:14 smithi082 ceph-mon[98694]: pgmap v8410: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 KiB/s wr, 15 op/s 2023-09-10T03:37:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15465.238645 secs 2023-09-10T03:37:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:14 smithi098 ceph-mon[103124]: pgmap v8410: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 20 KiB/s wr, 15 op/s 2023-09-10T03:37:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15465.238645 secs 2023-09-10T03:37:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15465.949878 secs 2023-09-10T03:37:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15465.949878 secs 2023-09-10T03:37:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15465.949878 secs 2023-09-10T03:37:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:16 smithi170 ceph-mon[102832]: pgmap v8411: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:37:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:16 smithi082 ceph-mon[98694]: pgmap v8411: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:37:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:16 smithi098 ceph-mon[103124]: pgmap v8411: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:37:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:18 smithi170 ceph-mon[102832]: pgmap v8412: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 70 KiB/s wr, 13 op/s 2023-09-10T03:37:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15469.028596 secs 2023-09-10T03:37:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:18 smithi082 ceph-mon[98694]: pgmap v8412: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 70 KiB/s wr, 13 op/s 2023-09-10T03:37:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15469.028596 secs 2023-09-10T03:37:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:18 smithi098 ceph-mon[103124]: pgmap v8412: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 70 KiB/s wr, 13 op/s 2023-09-10T03:37:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15469.028596 secs 2023-09-10T03:37:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15470.238709 secs 2023-09-10T03:37:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15470.238709 secs 2023-09-10T03:37:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15470.238709 secs 2023-09-10T03:37:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15470.949962 secs 2023-09-10T03:37:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:20 smithi170 ceph-mon[102832]: pgmap v8413: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T03:37:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15470.949962 secs 2023-09-10T03:37:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:20 smithi082 ceph-mon[98694]: pgmap v8413: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T03:37:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15470.949962 secs 2023-09-10T03:37:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:20 smithi098 ceph-mon[103124]: pgmap v8413: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T03:37:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:22 smithi170 ceph-mon[102832]: pgmap v8414: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T03:37:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:22 smithi082 ceph-mon[98694]: pgmap v8414: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T03:37:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:22 smithi098 ceph-mon[103124]: pgmap v8414: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T03:37:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:37:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:37:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:37:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:24 smithi170 ceph-mon[102832]: pgmap v8415: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T03:37:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15475.238768 secs 2023-09-10T03:37:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:37:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:37:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:37:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:24 smithi082 ceph-mon[98694]: pgmap v8415: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T03:37:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15475.238768 secs 2023-09-10T03:37:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:37:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:37:24.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:37:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:24 smithi098 ceph-mon[103124]: pgmap v8415: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T03:37:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15475.238768 secs 2023-09-10T03:37:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:37:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:37:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:37:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15475.950026 secs 2023-09-10T03:37:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15475.950026 secs 2023-09-10T03:37:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15475.950026 secs 2023-09-10T03:37:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:26 smithi170 ceph-mon[102832]: pgmap v8416: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 9 op/s 2023-09-10T03:37:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:26 smithi082 ceph-mon[98694]: pgmap v8416: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 9 op/s 2023-09-10T03:37:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:26 smithi098 ceph-mon[103124]: pgmap v8416: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 9 op/s 2023-09-10T03:37:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:28 smithi170 ceph-mon[102832]: pgmap v8417: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 169 KiB/s wr, 14 op/s 2023-09-10T03:37:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:28 smithi082 ceph-mon[98694]: pgmap v8417: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 169 KiB/s wr, 14 op/s 2023-09-10T03:37:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:28 smithi098 ceph-mon[103124]: pgmap v8417: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 169 KiB/s wr, 14 op/s 2023-09-10T03:37:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15480.238843 secs 2023-09-10T03:37:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15480.238843 secs 2023-09-10T03:37:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15480.238843 secs 2023-09-10T03:37:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15480.950091 secs 2023-09-10T03:37:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:30 smithi170 ceph-mon[102832]: pgmap v8418: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 118 KiB/s wr, 9 op/s 2023-09-10T03:37:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15480.950091 secs 2023-09-10T03:37:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:30 smithi082 ceph-mon[98694]: pgmap v8418: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 118 KiB/s wr, 9 op/s 2023-09-10T03:37:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15480.950091 secs 2023-09-10T03:37:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:30 smithi098 ceph-mon[103124]: pgmap v8418: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 118 KiB/s wr, 9 op/s 2023-09-10T03:37:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:32 smithi170 ceph-mon[102832]: pgmap v8419: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 198 KiB/s wr, 15 op/s 2023-09-10T03:37:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:32 smithi082 ceph-mon[98694]: pgmap v8419: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 198 KiB/s wr, 15 op/s 2023-09-10T03:37:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:32 smithi098 ceph-mon[103124]: pgmap v8419: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 198 KiB/s wr, 15 op/s 2023-09-10T03:37:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:33 smithi170 ceph-mon[102832]: pgmap v8420: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 198 KiB/s wr, 10 op/s 2023-09-10T03:37:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15485.238902 secs 2023-09-10T03:37:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:33 smithi082 ceph-mon[98694]: pgmap v8420: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 198 KiB/s wr, 10 op/s 2023-09-10T03:37:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15485.238902 secs 2023-09-10T03:37:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:33 smithi098 ceph-mon[103124]: pgmap v8420: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 198 KiB/s wr, 10 op/s 2023-09-10T03:37:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15485.238902 secs 2023-09-10T03:37:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15485.950156 secs 2023-09-10T03:37:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15485.950156 secs 2023-09-10T03:37:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15485.950156 secs 2023-09-10T03:37:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:35 smithi170 ceph-mon[102832]: pgmap v8421: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 198 KiB/s wr, 10 op/s 2023-09-10T03:37:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:35 smithi082 ceph-mon[98694]: pgmap v8421: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 198 KiB/s wr, 10 op/s 2023-09-10T03:37:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:35 smithi098 ceph-mon[103124]: pgmap v8421: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 198 KiB/s wr, 10 op/s 2023-09-10T03:37:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:37:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:37:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:37 smithi170 ceph-mon[102832]: pgmap v8422: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 221 KiB/s wr, 17 op/s 2023-09-10T03:37:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:37:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:37:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:37 smithi082 ceph-mon[98694]: pgmap v8422: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 221 KiB/s wr, 17 op/s 2023-09-10T03:37:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:37:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:37:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:37 smithi098 ceph-mon[103124]: pgmap v8422: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 221 KiB/s wr, 17 op/s 2023-09-10T03:37:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15490.238984 secs 2023-09-10T03:37:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15490.238984 secs 2023-09-10T03:37:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15490.238984 secs 2023-09-10T03:37:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15490.950220 secs 2023-09-10T03:37:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:39 smithi170 ceph-mon[102832]: pgmap v8423: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T03:37:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:39 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:37:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:39 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:37:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15490.950220 secs 2023-09-10T03:37:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:39 smithi082 ceph-mon[98694]: pgmap v8423: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T03:37:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:39 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:37:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:39 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:37:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15490.950220 secs 2023-09-10T03:37:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:39 smithi098 ceph-mon[103124]: pgmap v8423: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T03:37:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:39 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:37:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:39 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:37:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:41 smithi170 ceph-mon[102832]: pgmap v8424: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 310 KiB/s wr, 19 op/s 2023-09-10T03:37:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:41 smithi082 ceph-mon[98694]: pgmap v8424: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 310 KiB/s wr, 19 op/s 2023-09-10T03:37:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:41 smithi098 ceph-mon[103124]: pgmap v8424: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 310 KiB/s wr, 19 op/s 2023-09-10T03:37:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:43 smithi170 ceph-mon[102832]: pgmap v8425: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 229 KiB/s wr, 13 op/s 2023-09-10T03:37:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15495.239044 secs 2023-09-10T03:37:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:43 smithi082 ceph-mon[98694]: pgmap v8425: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 229 KiB/s wr, 13 op/s 2023-09-10T03:37:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15495.239044 secs 2023-09-10T03:37:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:43 smithi098 ceph-mon[103124]: pgmap v8425: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 229 KiB/s wr, 13 op/s 2023-09-10T03:37:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15495.239044 secs 2023-09-10T03:37:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15495.950286 secs 2023-09-10T03:37:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15495.950286 secs 2023-09-10T03:37:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15495.950286 secs 2023-09-10T03:37:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:45 smithi170 ceph-mon[102832]: pgmap v8426: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 229 KiB/s wr, 13 op/s 2023-09-10T03:37:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:45 smithi082 ceph-mon[98694]: pgmap v8426: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 229 KiB/s wr, 13 op/s 2023-09-10T03:37:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:45 smithi098 ceph-mon[103124]: pgmap v8426: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 229 KiB/s wr, 13 op/s 2023-09-10T03:37:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:47 smithi170 ceph-mon[102832]: pgmap v8427: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 254 KiB/s wr, 19 op/s 2023-09-10T03:37:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:47 smithi082 ceph-mon[98694]: pgmap v8427: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 254 KiB/s wr, 19 op/s 2023-09-10T03:37:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:47 smithi098 ceph-mon[103124]: pgmap v8427: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 254 KiB/s wr, 19 op/s 2023-09-10T03:37:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15500.239104 secs 2023-09-10T03:37:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15500.239104 secs 2023-09-10T03:37:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15500.239104 secs 2023-09-10T03:37:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15500.950350 secs 2023-09-10T03:37:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:49 smithi170 ceph-mon[102832]: pgmap v8428: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 231 KiB/s wr, 12 op/s 2023-09-10T03:37:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15500.950350 secs 2023-09-10T03:37:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:49 smithi082 ceph-mon[98694]: pgmap v8428: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 231 KiB/s wr, 12 op/s 2023-09-10T03:37:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15500.950350 secs 2023-09-10T03:37:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:49 smithi098 ceph-mon[103124]: pgmap v8428: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 231 KiB/s wr, 12 op/s 2023-09-10T03:37:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:51 smithi170 ceph-mon[102832]: pgmap v8429: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 434 KiB/s wr, 18 op/s 2023-09-10T03:37:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:37:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:37:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:51 smithi082 ceph-mon[98694]: pgmap v8429: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 434 KiB/s wr, 18 op/s 2023-09-10T03:37:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:37:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:37:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:51 smithi098 ceph-mon[103124]: pgmap v8429: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 434 KiB/s wr, 18 op/s 2023-09-10T03:37:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:37:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:37:53.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15504.029119 secs 2023-09-10T03:37:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15504.029119 secs 2023-09-10T03:37:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15504.029119 secs 2023-09-10T03:37:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:53 smithi082 ceph-mon[98694]: pgmap v8430: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 228 KiB/s wr, 12 op/s 2023-09-10T03:37:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:37:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:37:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15505.239164 secs 2023-09-10T03:37:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:53 smithi098 ceph-mon[103124]: pgmap v8430: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 228 KiB/s wr, 12 op/s 2023-09-10T03:37:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:37:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:37:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15505.239164 secs 2023-09-10T03:37:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:53 smithi170 ceph-mon[102832]: pgmap v8430: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 228 KiB/s wr, 12 op/s 2023-09-10T03:37:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:37:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:37:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15505.239164 secs 2023-09-10T03:37:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15505.950413 secs 2023-09-10T03:37:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15505.950413 secs 2023-09-10T03:37:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15505.950413 secs 2023-09-10T03:37:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:55 smithi082 ceph-mon[98694]: pgmap v8431: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 228 KiB/s wr, 12 op/s 2023-09-10T03:37:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:37:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:37:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:37:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:55 smithi098 ceph-mon[103124]: pgmap v8431: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 228 KiB/s wr, 12 op/s 2023-09-10T03:37:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:37:56.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:37:56.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:37:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:55 smithi170 ceph-mon[102832]: pgmap v8431: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 228 KiB/s wr, 12 op/s 2023-09-10T03:37:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:37:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:37:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:37:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:57 smithi082 ceph-mon[98694]: pgmap v8432: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 229 KiB/s wr, 17 op/s 2023-09-10T03:37:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15509.029177 secs 2023-09-10T03:37:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:57 smithi098 ceph-mon[103124]: pgmap v8432: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 229 KiB/s wr, 17 op/s 2023-09-10T03:37:58.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15509.029177 secs 2023-09-10T03:37:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:57 smithi170 ceph-mon[102832]: pgmap v8432: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 229 KiB/s wr, 17 op/s 2023-09-10T03:37:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15509.029177 secs 2023-09-10T03:37:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15510.239219 secs 2023-09-10T03:37:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15510.239219 secs 2023-09-10T03:37:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15510.239219 secs 2023-09-10T03:38:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15510.950487 secs 2023-09-10T03:38:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:37:59 smithi082 ceph-mon[98694]: pgmap v8433: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 204 KiB/s wr, 12 op/s 2023-09-10T03:38:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15510.950487 secs 2023-09-10T03:38:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:37:59 smithi098 ceph-mon[103124]: pgmap v8433: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 204 KiB/s wr, 12 op/s 2023-09-10T03:38:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15510.950487 secs 2023-09-10T03:38:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:37:59 smithi170 ceph-mon[102832]: pgmap v8433: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 204 KiB/s wr, 12 op/s 2023-09-10T03:38:02.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:01 smithi082 ceph-mon[98694]: pgmap v8434: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 345 KiB/s wr, 24 op/s 2023-09-10T03:38:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:01 smithi098 ceph-mon[103124]: pgmap v8434: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 345 KiB/s wr, 24 op/s 2023-09-10T03:38:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:01 smithi170 ceph-mon[102832]: pgmap v8434: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 345 KiB/s wr, 24 op/s 2023-09-10T03:38:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15514.029236 secs 2023-09-10T03:38:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15514.029236 secs 2023-09-10T03:38:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15514.029236 secs 2023-09-10T03:38:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:03 smithi098 ceph-mon[103124]: pgmap v8435: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 142 KiB/s wr, 18 op/s 2023-09-10T03:38:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15515.239303 secs 2023-09-10T03:38:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:03 smithi170 ceph-mon[102832]: pgmap v8435: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 142 KiB/s wr, 18 op/s 2023-09-10T03:38:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15515.239303 secs 2023-09-10T03:38:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:03 smithi082 ceph-mon[98694]: pgmap v8435: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 142 KiB/s wr, 18 op/s 2023-09-10T03:38:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15515.239303 secs 2023-09-10T03:38:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15515.950555 secs 2023-09-10T03:38:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15515.950555 secs 2023-09-10T03:38:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15515.950555 secs 2023-09-10T03:38:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:05 smithi098 ceph-mon[103124]: pgmap v8436: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 142 KiB/s wr, 18 op/s 2023-09-10T03:38:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:05 smithi170 ceph-mon[102832]: pgmap v8436: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 142 KiB/s wr, 18 op/s 2023-09-10T03:38:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:05 smithi082 ceph-mon[98694]: pgmap v8436: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 142 KiB/s wr, 18 op/s 2023-09-10T03:38:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:07 smithi098 ceph-mon[103124]: pgmap v8437: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 165 KiB/s wr, 30 op/s 2023-09-10T03:38:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15519.029343 secs 2023-09-10T03:38:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:07 smithi170 ceph-mon[102832]: pgmap v8437: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 165 KiB/s wr, 30 op/s 2023-09-10T03:38:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15519.029343 secs 2023-09-10T03:38:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:07 smithi082 ceph-mon[98694]: pgmap v8437: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 165 KiB/s wr, 30 op/s 2023-09-10T03:38:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15519.029343 secs 2023-09-10T03:38:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:08 smithi098 ceph-mon[103124]: osdmap e595: 12 total, 12 up, 12 in 2023-09-10T03:38:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15520.239369 secs 2023-09-10T03:38:09.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:38:09.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:38:09.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:38:09.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:38:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:08 smithi170 ceph-mon[102832]: osdmap e595: 12 total, 12 up, 12 in 2023-09-10T03:38:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15520.239369 secs 2023-09-10T03:38:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:38:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:38:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:38:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:38:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:08 smithi082 ceph-mon[98694]: osdmap e595: 12 total, 12 up, 12 in 2023-09-10T03:38:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15520.239369 secs 2023-09-10T03:38:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:38:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:38:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:38:09.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:38:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15520.950620 secs 2023-09-10T03:38:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:09 smithi098 ceph-mon[103124]: pgmap v8439: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 197 KiB/s wr, 29 op/s 2023-09-10T03:38:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15520.950620 secs 2023-09-10T03:38:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:09 smithi170 ceph-mon[102832]: pgmap v8439: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 197 KiB/s wr, 29 op/s 2023-09-10T03:38:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15520.950620 secs 2023-09-10T03:38:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:09 smithi082 ceph-mon[98694]: pgmap v8439: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 197 KiB/s wr, 29 op/s 2023-09-10T03:38:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:11 smithi098 ceph-mon[103124]: pgmap v8440: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 77 KiB/s wr, 20 op/s 2023-09-10T03:38:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:11 smithi170 ceph-mon[102832]: pgmap v8440: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 77 KiB/s wr, 20 op/s 2023-09-10T03:38:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:11 smithi082 ceph-mon[98694]: pgmap v8440: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 77 KiB/s wr, 20 op/s 2023-09-10T03:38:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15524.029422 secs 2023-09-10T03:38:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:12 smithi098 ceph-mon[103124]: osdmap e596: 12 total, 12 up, 12 in 2023-09-10T03:38:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15524.029422 secs 2023-09-10T03:38:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:12 smithi170 ceph-mon[102832]: osdmap e596: 12 total, 12 up, 12 in 2023-09-10T03:38:13.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15524.029422 secs 2023-09-10T03:38:13.462 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:12 smithi082 ceph-mon[98694]: osdmap e596: 12 total, 12 up, 12 in 2023-09-10T03:38:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:13 smithi098 ceph-mon[103124]: pgmap v8442: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 96 KiB/s wr, 25 op/s 2023-09-10T03:38:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15525.239426 secs 2023-09-10T03:38:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:13 smithi170 ceph-mon[102832]: pgmap v8442: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 96 KiB/s wr, 25 op/s 2023-09-10T03:38:14.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15525.239426 secs 2023-09-10T03:38:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:13 smithi082 ceph-mon[98694]: pgmap v8442: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 96 KiB/s wr, 25 op/s 2023-09-10T03:38:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15525.239426 secs 2023-09-10T03:38:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15525.950685 secs 2023-09-10T03:38:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15525.950685 secs 2023-09-10T03:38:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15525.950685 secs 2023-09-10T03:38:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:15 smithi098 ceph-mon[103124]: pgmap v8443: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 7 op/s 2023-09-10T03:38:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:15 smithi170 ceph-mon[102832]: pgmap v8443: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 7 op/s 2023-09-10T03:38:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:15 smithi082 ceph-mon[98694]: pgmap v8443: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 7 op/s 2023-09-10T03:38:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:17 smithi098 ceph-mon[103124]: pgmap v8444: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 13 op/s 2023-09-10T03:38:18.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15529.029488 secs 2023-09-10T03:38:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:17 smithi170 ceph-mon[102832]: pgmap v8444: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 13 op/s 2023-09-10T03:38:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15529.029488 secs 2023-09-10T03:38:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:17 smithi082 ceph-mon[98694]: pgmap v8444: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 13 op/s 2023-09-10T03:38:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15529.029488 secs 2023-09-10T03:38:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15530.239514 secs 2023-09-10T03:38:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15530.239514 secs 2023-09-10T03:38:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15530.239514 secs 2023-09-10T03:38:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15530.950751 secs 2023-09-10T03:38:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:20 smithi098 ceph-mon[103124]: pgmap v8445: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T03:38:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15530.950751 secs 2023-09-10T03:38:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:20 smithi170 ceph-mon[102832]: pgmap v8445: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T03:38:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15530.950751 secs 2023-09-10T03:38:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:20 smithi082 ceph-mon[98694]: pgmap v8445: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T03:38:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:22 smithi098 ceph-mon[103124]: pgmap v8446: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T03:38:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:22 smithi170 ceph-mon[102832]: pgmap v8446: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T03:38:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:22 smithi082 ceph-mon[98694]: pgmap v8446: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T03:38:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:38:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:38:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:38:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:24 smithi170 ceph-mon[102832]: pgmap v8447: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T03:38:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15535.239575 secs 2023-09-10T03:38:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:38:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:38:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:38:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:24 smithi082 ceph-mon[98694]: pgmap v8447: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T03:38:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15535.239575 secs 2023-09-10T03:38:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:38:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:38:24.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:38:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:24 smithi098 ceph-mon[103124]: pgmap v8447: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 11 op/s 2023-09-10T03:38:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15535.239575 secs 2023-09-10T03:38:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:38:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:38:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:38:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15535.950817 secs 2023-09-10T03:38:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15535.950817 secs 2023-09-10T03:38:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15535.950817 secs 2023-09-10T03:38:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:26 smithi170 ceph-mon[102832]: pgmap v8448: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 9 op/s 2023-09-10T03:38:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:26 smithi082 ceph-mon[98694]: pgmap v8448: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 9 op/s 2023-09-10T03:38:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:26 smithi098 ceph-mon[103124]: pgmap v8448: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 9 op/s 2023-09-10T03:38:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:28 smithi170 ceph-mon[102832]: pgmap v8449: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 154 KiB/s wr, 15 op/s 2023-09-10T03:38:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:28 smithi082 ceph-mon[98694]: pgmap v8449: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 154 KiB/s wr, 15 op/s 2023-09-10T03:38:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:28 smithi098 ceph-mon[103124]: pgmap v8449: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 154 KiB/s wr, 15 op/s 2023-09-10T03:38:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15540.239662 secs 2023-09-10T03:38:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15540.239662 secs 2023-09-10T03:38:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15540.239662 secs 2023-09-10T03:38:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15540.950884 secs 2023-09-10T03:38:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:30 smithi170 ceph-mon[102832]: pgmap v8450: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 102 KiB/s wr, 10 op/s 2023-09-10T03:38:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:38:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:38:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15540.950884 secs 2023-09-10T03:38:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:30 smithi082 ceph-mon[98694]: pgmap v8450: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 102 KiB/s wr, 10 op/s 2023-09-10T03:38:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:38:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:38:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15540.950884 secs 2023-09-10T03:38:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:30 smithi098 ceph-mon[103124]: pgmap v8450: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 102 KiB/s wr, 10 op/s 2023-09-10T03:38:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:38:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:38:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:32 smithi170 ceph-mon[102832]: pgmap v8451: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 243 KiB/s wr, 16 op/s 2023-09-10T03:38:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:32 smithi082 ceph-mon[98694]: pgmap v8451: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 243 KiB/s wr, 16 op/s 2023-09-10T03:38:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:32 smithi098 ceph-mon[103124]: pgmap v8451: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 243 KiB/s wr, 16 op/s 2023-09-10T03:38:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:33 smithi170 ceph-mon[102832]: pgmap v8452: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 243 KiB/s wr, 11 op/s 2023-09-10T03:38:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15545.239728 secs 2023-09-10T03:38:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:33 smithi082 ceph-mon[98694]: pgmap v8452: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 243 KiB/s wr, 11 op/s 2023-09-10T03:38:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15545.239728 secs 2023-09-10T03:38:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:33 smithi098 ceph-mon[103124]: pgmap v8452: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 243 KiB/s wr, 11 op/s 2023-09-10T03:38:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15545.239728 secs 2023-09-10T03:38:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15545.950951 secs 2023-09-10T03:38:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15545.950951 secs 2023-09-10T03:38:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15545.950951 secs 2023-09-10T03:38:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:35 smithi170 ceph-mon[102832]: pgmap v8453: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 243 KiB/s wr, 11 op/s 2023-09-10T03:38:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:35 smithi082 ceph-mon[98694]: pgmap v8453: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 243 KiB/s wr, 11 op/s 2023-09-10T03:38:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:35 smithi098 ceph-mon[103124]: pgmap v8453: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 243 KiB/s wr, 11 op/s 2023-09-10T03:38:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:37 smithi170 ceph-mon[102832]: pgmap v8454: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 263 KiB/s wr, 17 op/s 2023-09-10T03:38:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:38:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:38:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:37 smithi082 ceph-mon[98694]: pgmap v8454: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 263 KiB/s wr, 17 op/s 2023-09-10T03:38:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:38:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:38:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:37 smithi098 ceph-mon[103124]: pgmap v8454: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 263 KiB/s wr, 17 op/s 2023-09-10T03:38:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:38:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:38:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15550.239785 secs 2023-09-10T03:38:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15550.239785 secs 2023-09-10T03:38:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15550.239785 secs 2023-09-10T03:38:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15550.951014 secs 2023-09-10T03:38:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:39 smithi170 ceph-mon[102832]: pgmap v8455: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 161 KiB/s wr, 11 op/s 2023-09-10T03:38:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:39 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:38:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:39 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:38:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15550.951014 secs 2023-09-10T03:38:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:39 smithi082 ceph-mon[98694]: pgmap v8455: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 161 KiB/s wr, 11 op/s 2023-09-10T03:38:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:39 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:38:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:39 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:38:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15550.951014 secs 2023-09-10T03:38:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:39 smithi098 ceph-mon[103124]: pgmap v8455: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 161 KiB/s wr, 11 op/s 2023-09-10T03:38:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:39 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:38:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:39 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:38:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:41 smithi170 ceph-mon[102832]: pgmap v8456: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 436 KiB/s wr, 18 op/s 2023-09-10T03:38:42.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:41 smithi082 ceph-mon[98694]: pgmap v8456: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 436 KiB/s wr, 18 op/s 2023-09-10T03:38:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:41 smithi098 ceph-mon[103124]: pgmap v8456: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 436 KiB/s wr, 18 op/s 2023-09-10T03:38:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:43 smithi170 ceph-mon[102832]: pgmap v8457: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T03:38:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15555.239855 secs 2023-09-10T03:38:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:43 smithi082 ceph-mon[98694]: pgmap v8457: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T03:38:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15555.239855 secs 2023-09-10T03:38:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:43 smithi098 ceph-mon[103124]: pgmap v8457: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T03:38:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15555.239855 secs 2023-09-10T03:38:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15555.951078 secs 2023-09-10T03:38:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15555.951078 secs 2023-09-10T03:38:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15555.951078 secs 2023-09-10T03:38:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:45 smithi170 ceph-mon[102832]: pgmap v8458: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T03:38:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:45 smithi082 ceph-mon[98694]: pgmap v8458: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T03:38:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:45 smithi098 ceph-mon[103124]: pgmap v8458: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T03:38:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:47 smithi082 ceph-mon[98694]: pgmap v8459: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 332 KiB/s wr, 17 op/s 2023-09-10T03:38:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:47 smithi098 ceph-mon[103124]: pgmap v8459: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 332 KiB/s wr, 17 op/s 2023-09-10T03:38:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:47 smithi170 ceph-mon[102832]: pgmap v8459: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 332 KiB/s wr, 17 op/s 2023-09-10T03:38:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15560.239914 secs 2023-09-10T03:38:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15560.239914 secs 2023-09-10T03:38:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15560.239914 secs 2023-09-10T03:38:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15560.951141 secs 2023-09-10T03:38:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:49 smithi082 ceph-mon[98694]: pgmap v8460: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 312 KiB/s wr, 12 op/s 2023-09-10T03:38:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15560.951141 secs 2023-09-10T03:38:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:49 smithi098 ceph-mon[103124]: pgmap v8460: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 312 KiB/s wr, 12 op/s 2023-09-10T03:38:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15560.951141 secs 2023-09-10T03:38:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:49 smithi170 ceph-mon[102832]: pgmap v8460: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 312 KiB/s wr, 12 op/s 2023-09-10T03:38:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:51 smithi082 ceph-mon[98694]: pgmap v8461: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 469 KiB/s wr, 17 op/s 2023-09-10T03:38:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:51 smithi098 ceph-mon[103124]: pgmap v8461: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 469 KiB/s wr, 17 op/s 2023-09-10T03:38:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:51 smithi170 ceph-mon[102832]: pgmap v8461: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 469 KiB/s wr, 17 op/s 2023-09-10T03:38:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15564.030065 secs 2023-09-10T03:38:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15564.030065 secs 2023-09-10T03:38:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15564.030065 secs 2023-09-10T03:38:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:53 smithi082 ceph-mon[98694]: pgmap v8462: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 195 KiB/s wr, 11 op/s 2023-09-10T03:38:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15565.239971 secs 2023-09-10T03:38:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:53 smithi098 ceph-mon[103124]: pgmap v8462: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 195 KiB/s wr, 11 op/s 2023-09-10T03:38:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15565.239971 secs 2023-09-10T03:38:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:53 smithi170 ceph-mon[102832]: pgmap v8462: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 195 KiB/s wr, 11 op/s 2023-09-10T03:38:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15565.239971 secs 2023-09-10T03:38:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15565.951210 secs 2023-09-10T03:38:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15565.951210 secs 2023-09-10T03:38:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15565.951210 secs 2023-09-10T03:38:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:55 smithi082 ceph-mon[98694]: pgmap v8463: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 195 KiB/s wr, 11 op/s 2023-09-10T03:38:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:38:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:38:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:38:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:55 smithi098 ceph-mon[103124]: pgmap v8463: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 195 KiB/s wr, 11 op/s 2023-09-10T03:38:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:38:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:38:56.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:38:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:55 smithi170 ceph-mon[102832]: pgmap v8463: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 195 KiB/s wr, 11 op/s 2023-09-10T03:38:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:38:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:38:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:38:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:57 smithi082 ceph-mon[98694]: pgmap v8464: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 303 KiB/s wr, 16 op/s 2023-09-10T03:38:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15569.030125 secs 2023-09-10T03:38:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:57 smithi098 ceph-mon[103124]: pgmap v8464: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 303 KiB/s wr, 16 op/s 2023-09-10T03:38:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15569.030125 secs 2023-09-10T03:38:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:57 smithi170 ceph-mon[102832]: pgmap v8464: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 303 KiB/s wr, 16 op/s 2023-09-10T03:38:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15569.030125 secs 2023-09-10T03:38:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15570.240033 secs 2023-09-10T03:38:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15570.240033 secs 2023-09-10T03:38:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15570.240033 secs 2023-09-10T03:39:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15570.951275 secs 2023-09-10T03:39:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:38:59 smithi098 ceph-mon[103124]: pgmap v8465: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 265 KiB/s wr, 10 op/s 2023-09-10T03:39:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15570.951275 secs 2023-09-10T03:39:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:38:59 smithi170 ceph-mon[102832]: pgmap v8465: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 265 KiB/s wr, 10 op/s 2023-09-10T03:39:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15570.951275 secs 2023-09-10T03:39:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:38:59 smithi082 ceph-mon[98694]: pgmap v8465: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 265 KiB/s wr, 10 op/s 2023-09-10T03:39:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:01 smithi098 ceph-mon[103124]: pgmap v8466: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 312 KiB/s wr, 16 op/s 2023-09-10T03:39:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:01 smithi170 ceph-mon[102832]: pgmap v8466: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 312 KiB/s wr, 16 op/s 2023-09-10T03:39:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:01 smithi082 ceph-mon[98694]: pgmap v8466: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 312 KiB/s wr, 16 op/s 2023-09-10T03:39:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15574.030186 secs 2023-09-10T03:39:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15574.030186 secs 2023-09-10T03:39:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15574.030186 secs 2023-09-10T03:39:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:03 smithi098 ceph-mon[103124]: pgmap v8467: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T03:39:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:03 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:39:04.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:03 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:39:04.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15575.240093 secs 2023-09-10T03:39:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:03 smithi170 ceph-mon[102832]: pgmap v8467: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T03:39:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:03 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:39:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:03 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:39:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15575.240093 secs 2023-09-10T03:39:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:03 smithi082 ceph-mon[98694]: pgmap v8467: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T03:39:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:03 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:39:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:03 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:39:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15575.240093 secs 2023-09-10T03:39:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15575.951338 secs 2023-09-10T03:39:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15575.951338 secs 2023-09-10T03:39:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15575.951338 secs 2023-09-10T03:39:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:05 smithi098 ceph-mon[103124]: pgmap v8468: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T03:39:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:39:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:39:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:05 smithi170 ceph-mon[102832]: pgmap v8468: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T03:39:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:39:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:39:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:05 smithi082 ceph-mon[98694]: pgmap v8468: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T03:39:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:39:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:39:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:07 smithi098 ceph-mon[103124]: pgmap v8469: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 212 KiB/s wr, 30 op/s 2023-09-10T03:39:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15579.030251 secs 2023-09-10T03:39:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:07 smithi170 ceph-mon[102832]: pgmap v8469: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 212 KiB/s wr, 30 op/s 2023-09-10T03:39:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15579.030251 secs 2023-09-10T03:39:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:07 smithi082 ceph-mon[98694]: pgmap v8469: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 212 KiB/s wr, 30 op/s 2023-09-10T03:39:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15579.030251 secs 2023-09-10T03:39:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15580.240150 secs 2023-09-10T03:39:09.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:08 smithi098 ceph-mon[103124]: osdmap e597: 12 total, 12 up, 12 in 2023-09-10T03:39:09.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:39:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15580.240150 secs 2023-09-10T03:39:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:08 smithi170 ceph-mon[102832]: osdmap e597: 12 total, 12 up, 12 in 2023-09-10T03:39:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:39:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15580.240150 secs 2023-09-10T03:39:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:08 smithi082 ceph-mon[98694]: osdmap e597: 12 total, 12 up, 12 in 2023-09-10T03:39:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:39:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15580.951403 secs 2023-09-10T03:39:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:09 smithi098 ceph-mon[103124]: pgmap v8471: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 124 KiB/s wr, 30 op/s 2023-09-10T03:39:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:39:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:39:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:39:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15580.951403 secs 2023-09-10T03:39:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:09 smithi170 ceph-mon[102832]: pgmap v8471: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 124 KiB/s wr, 30 op/s 2023-09-10T03:39:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:39:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:39:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:39:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15580.951403 secs 2023-09-10T03:39:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:09 smithi082 ceph-mon[98694]: pgmap v8471: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 124 KiB/s wr, 30 op/s 2023-09-10T03:39:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:39:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:39:10.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:39:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:11 smithi098 ceph-mon[103124]: pgmap v8472: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 150 KiB/s wr, 29 op/s 2023-09-10T03:39:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:11 smithi170 ceph-mon[102832]: pgmap v8472: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 150 KiB/s wr, 29 op/s 2023-09-10T03:39:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:11 smithi082 ceph-mon[98694]: pgmap v8472: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 150 KiB/s wr, 29 op/s 2023-09-10T03:39:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15584.030331 secs 2023-09-10T03:39:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:13 smithi098 ceph-mon[103124]: osdmap e598: 12 total, 12 up, 12 in 2023-09-10T03:39:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15584.030331 secs 2023-09-10T03:39:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:13 smithi170 ceph-mon[102832]: osdmap e598: 12 total, 12 up, 12 in 2023-09-10T03:39:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15584.030331 secs 2023-09-10T03:39:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:13 smithi082 ceph-mon[98694]: osdmap e598: 12 total, 12 up, 12 in 2023-09-10T03:39:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:14 smithi098 ceph-mon[103124]: pgmap v8474: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 188 KiB/s wr, 37 op/s 2023-09-10T03:39:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15585.240212 secs 2023-09-10T03:39:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:14 smithi170 ceph-mon[102832]: pgmap v8474: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 188 KiB/s wr, 37 op/s 2023-09-10T03:39:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15585.240212 secs 2023-09-10T03:39:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:14 smithi082 ceph-mon[98694]: pgmap v8474: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 188 KiB/s wr, 37 op/s 2023-09-10T03:39:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15585.240212 secs 2023-09-10T03:39:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15585.951492 secs 2023-09-10T03:39:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15585.951492 secs 2023-09-10T03:39:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15585.951492 secs 2023-09-10T03:39:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:16 smithi170 ceph-mon[102832]: pgmap v8475: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 103 KiB/s wr, 7 op/s 2023-09-10T03:39:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:16 smithi082 ceph-mon[98694]: pgmap v8475: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 103 KiB/s wr, 7 op/s 2023-09-10T03:39:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:16 smithi098 ceph-mon[103124]: pgmap v8475: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 103 KiB/s wr, 7 op/s 2023-09-10T03:39:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:18 smithi170 ceph-mon[102832]: pgmap v8476: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 13 op/s 2023-09-10T03:39:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15589.030392 secs 2023-09-10T03:39:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:18 smithi082 ceph-mon[98694]: pgmap v8476: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 13 op/s 2023-09-10T03:39:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15589.030392 secs 2023-09-10T03:39:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:18 smithi098 ceph-mon[103124]: pgmap v8476: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 13 op/s 2023-09-10T03:39:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15589.030392 secs 2023-09-10T03:39:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15590.240296 secs 2023-09-10T03:39:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15590.240296 secs 2023-09-10T03:39:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15590.240296 secs 2023-09-10T03:39:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15590.951559 secs 2023-09-10T03:39:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:20 smithi170 ceph-mon[102832]: pgmap v8477: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T03:39:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15590.951559 secs 2023-09-10T03:39:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:20 smithi082 ceph-mon[98694]: pgmap v8477: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T03:39:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15590.951559 secs 2023-09-10T03:39:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:20 smithi098 ceph-mon[103124]: pgmap v8477: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T03:39:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:22 smithi170 ceph-mon[102832]: pgmap v8478: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T03:39:22.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:22 smithi082 ceph-mon[98694]: pgmap v8478: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T03:39:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:22 smithi098 ceph-mon[103124]: pgmap v8478: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T03:39:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:39:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:39:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:39:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:24 smithi170 ceph-mon[102832]: pgmap v8479: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T03:39:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15595.240397 secs 2023-09-10T03:39:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:39:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:39:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:39:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:24 smithi082 ceph-mon[98694]: pgmap v8479: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T03:39:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15595.240397 secs 2023-09-10T03:39:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:39:24.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:39:24.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:39:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:24 smithi098 ceph-mon[103124]: pgmap v8479: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T03:39:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15595.240397 secs 2023-09-10T03:39:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:39:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:39:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:39:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15595.951622 secs 2023-09-10T03:39:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15595.951622 secs 2023-09-10T03:39:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15595.951622 secs 2023-09-10T03:39:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:26 smithi170 ceph-mon[102832]: pgmap v8480: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 9 op/s 2023-09-10T03:39:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:26 smithi082 ceph-mon[98694]: pgmap v8480: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 9 op/s 2023-09-10T03:39:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:26 smithi098 ceph-mon[103124]: pgmap v8480: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 9 op/s 2023-09-10T03:39:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:28 smithi170 ceph-mon[102832]: pgmap v8481: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 132 KiB/s wr, 14 op/s 2023-09-10T03:39:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:28 smithi082 ceph-mon[98694]: pgmap v8481: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 132 KiB/s wr, 14 op/s 2023-09-10T03:39:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:28 smithi098 ceph-mon[103124]: pgmap v8481: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 132 KiB/s wr, 14 op/s 2023-09-10T03:39:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15600.240451 secs 2023-09-10T03:39:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15600.240451 secs 2023-09-10T03:39:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15600.240451 secs 2023-09-10T03:39:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15600.951686 secs 2023-09-10T03:39:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:30 smithi170 ceph-mon[102832]: pgmap v8482: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-09-10T03:39:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15600.951686 secs 2023-09-10T03:39:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:30 smithi082 ceph-mon[98694]: pgmap v8482: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-09-10T03:39:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15600.951686 secs 2023-09-10T03:39:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:30 smithi098 ceph-mon[103124]: pgmap v8482: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-09-10T03:39:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:32 smithi170 ceph-mon[102832]: pgmap v8483: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 197 KiB/s wr, 15 op/s 2023-09-10T03:39:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:32 smithi082 ceph-mon[98694]: pgmap v8483: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 197 KiB/s wr, 15 op/s 2023-09-10T03:39:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:32 smithi098 ceph-mon[103124]: pgmap v8483: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 197 KiB/s wr, 15 op/s 2023-09-10T03:39:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:34 smithi170 ceph-mon[102832]: pgmap v8484: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 10 op/s 2023-09-10T03:39:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15605.240510 secs 2023-09-10T03:39:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:39:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:39:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:34 smithi082 ceph-mon[98694]: pgmap v8484: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 10 op/s 2023-09-10T03:39:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15605.240510 secs 2023-09-10T03:39:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:39:34.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:39:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:34 smithi098 ceph-mon[103124]: pgmap v8484: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 10 op/s 2023-09-10T03:39:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15605.240510 secs 2023-09-10T03:39:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:39:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:39:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15605.951750 secs 2023-09-10T03:39:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15605.951750 secs 2023-09-10T03:39:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15605.951750 secs 2023-09-10T03:39:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:36 smithi170 ceph-mon[102832]: pgmap v8485: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 10 op/s 2023-09-10T03:39:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:39:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:39:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:36 smithi082 ceph-mon[98694]: pgmap v8485: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 10 op/s 2023-09-10T03:39:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:39:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:39:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:36 smithi098 ceph-mon[103124]: pgmap v8485: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 10 op/s 2023-09-10T03:39:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:39:36.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:39:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:38 smithi170 ceph-mon[102832]: pgmap v8486: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 329 KiB/s wr, 34 op/s 2023-09-10T03:39:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:38 smithi082 ceph-mon[98694]: pgmap v8486: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 329 KiB/s wr, 34 op/s 2023-09-10T03:39:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:38 smithi098 ceph-mon[103124]: pgmap v8486: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 329 KiB/s wr, 34 op/s 2023-09-10T03:39:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15610.240566 secs 2023-09-10T03:39:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15610.240566 secs 2023-09-10T03:39:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15610.240566 secs 2023-09-10T03:39:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15610.951812 secs 2023-09-10T03:39:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:40 smithi170 ceph-mon[102832]: pgmap v8487: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 263 KiB/s wr, 29 op/s 2023-09-10T03:39:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15610.951812 secs 2023-09-10T03:39:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:40 smithi082 ceph-mon[98694]: pgmap v8487: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 263 KiB/s wr, 29 op/s 2023-09-10T03:39:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15610.951812 secs 2023-09-10T03:39:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:40 smithi098 ceph-mon[103124]: pgmap v8487: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 263 KiB/s wr, 29 op/s 2023-09-10T03:39:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:42 smithi170 ceph-mon[102832]: pgmap v8488: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 433 KiB/s wr, 36 op/s 2023-09-10T03:39:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:42 smithi082 ceph-mon[98694]: pgmap v8488: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 433 KiB/s wr, 36 op/s 2023-09-10T03:39:42.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:42 smithi098 ceph-mon[103124]: pgmap v8488: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 433 KiB/s wr, 36 op/s 2023-09-10T03:39:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:43 smithi170 ceph-mon[102832]: pgmap v8489: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 367 KiB/s wr, 30 op/s 2023-09-10T03:39:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15615.240629 secs 2023-09-10T03:39:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:43 smithi082 ceph-mon[98694]: pgmap v8489: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 367 KiB/s wr, 30 op/s 2023-09-10T03:39:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15615.240629 secs 2023-09-10T03:39:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:43 smithi098 ceph-mon[103124]: pgmap v8489: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 367 KiB/s wr, 30 op/s 2023-09-10T03:39:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15615.240629 secs 2023-09-10T03:39:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15615.951878 secs 2023-09-10T03:39:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15615.951878 secs 2023-09-10T03:39:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15615.951878 secs 2023-09-10T03:39:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:45 smithi170 ceph-mon[102832]: pgmap v8490: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 367 KiB/s wr, 30 op/s 2023-09-10T03:39:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:45 smithi082 ceph-mon[98694]: pgmap v8490: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 367 KiB/s wr, 30 op/s 2023-09-10T03:39:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:45 smithi098 ceph-mon[103124]: pgmap v8490: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 367 KiB/s wr, 30 op/s 2023-09-10T03:39:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:47 smithi082 ceph-mon[98694]: pgmap v8491: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 422 KiB/s wr, 36 op/s 2023-09-10T03:39:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:47 smithi098 ceph-mon[103124]: pgmap v8491: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 422 KiB/s wr, 36 op/s 2023-09-10T03:39:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:47 smithi170 ceph-mon[102832]: pgmap v8491: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 422 KiB/s wr, 36 op/s 2023-09-10T03:39:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15620.240690 secs 2023-09-10T03:39:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15620.240690 secs 2023-09-10T03:39:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15620.240690 secs 2023-09-10T03:39:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15620.951942 secs 2023-09-10T03:39:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:49 smithi082 ceph-mon[98694]: pgmap v8492: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 225 KiB/s wr, 12 op/s 2023-09-10T03:39:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:39:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:39:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15620.951942 secs 2023-09-10T03:39:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:49 smithi098 ceph-mon[103124]: pgmap v8492: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 225 KiB/s wr, 12 op/s 2023-09-10T03:39:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:39:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:39:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15620.951942 secs 2023-09-10T03:39:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:49 smithi170 ceph-mon[102832]: pgmap v8492: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 225 KiB/s wr, 12 op/s 2023-09-10T03:39:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:39:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:39:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:51 smithi082 ceph-mon[98694]: pgmap v8493: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 308 KiB/s wr, 18 op/s 2023-09-10T03:39:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:39:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:39:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:51 smithi098 ceph-mon[103124]: pgmap v8493: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 308 KiB/s wr, 18 op/s 2023-09-10T03:39:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:39:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:39:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:51 smithi170 ceph-mon[102832]: pgmap v8493: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 308 KiB/s wr, 18 op/s 2023-09-10T03:39:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:39:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:39:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15624.030975 secs 2023-09-10T03:39:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15624.030975 secs 2023-09-10T03:39:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15624.030975 secs 2023-09-10T03:39:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:53 smithi098 ceph-mon[103124]: pgmap v8494: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T03:39:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15625.240752 secs 2023-09-10T03:39:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:53 smithi170 ceph-mon[102832]: pgmap v8494: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T03:39:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15625.240752 secs 2023-09-10T03:39:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:53 smithi082 ceph-mon[98694]: pgmap v8494: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T03:39:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15625.240752 secs 2023-09-10T03:39:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15625.952007 secs 2023-09-10T03:39:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15625.952007 secs 2023-09-10T03:39:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15625.952007 secs 2023-09-10T03:39:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:55 smithi098 ceph-mon[103124]: pgmap v8495: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T03:39:56.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:39:56.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:39:56.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:39:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:55 smithi170 ceph-mon[102832]: pgmap v8495: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T03:39:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:39:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:39:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:39:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:55 smithi082 ceph-mon[98694]: pgmap v8495: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T03:39:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:39:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:39:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:39:58.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:57 smithi098 ceph-mon[103124]: pgmap v8496: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 252 KiB/s wr, 17 op/s 2023-09-10T03:39:58.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15629.031032 secs 2023-09-10T03:39:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:57 smithi170 ceph-mon[102832]: pgmap v8496: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 252 KiB/s wr, 17 op/s 2023-09-10T03:39:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15629.031032 secs 2023-09-10T03:39:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:57 smithi082 ceph-mon[98694]: pgmap v8496: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 252 KiB/s wr, 17 op/s 2023-09-10T03:39:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15629.031032 secs 2023-09-10T03:39:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15630.240813 secs 2023-09-10T03:39:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15630.240813 secs 2023-09-10T03:39:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15630.240813 secs 2023-09-10T03:40:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15630.952074 secs 2023-09-10T03:40:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:39:59 smithi098 ceph-mon[103124]: pgmap v8497: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 197 KiB/s wr, 11 op/s 2023-09-10T03:40:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15630.952074 secs 2023-09-10T03:40:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:39:59 smithi170 ceph-mon[102832]: pgmap v8497: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 197 KiB/s wr, 11 op/s 2023-09-10T03:40:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15630.952074 secs 2023-09-10T03:40:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:39:59 smithi082 ceph-mon[98694]: pgmap v8497: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 197 KiB/s wr, 11 op/s 2023-09-10T03:40:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T03:40:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T03:40:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T03:40:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:01 smithi098 ceph-mon[103124]: pgmap v8498: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 243 KiB/s wr, 16 op/s 2023-09-10T03:40:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:01 smithi170 ceph-mon[102832]: pgmap v8498: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 243 KiB/s wr, 16 op/s 2023-09-10T03:40:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:01 smithi082 ceph-mon[98694]: pgmap v8498: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 243 KiB/s wr, 16 op/s 2023-09-10T03:40:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15634.031094 secs 2023-09-10T03:40:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15634.031094 secs 2023-09-10T03:40:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15634.031094 secs 2023-09-10T03:40:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:03 smithi098 ceph-mon[103124]: pgmap v8499: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 10 op/s 2023-09-10T03:40:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15635.240873 secs 2023-09-10T03:40:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:03 smithi170 ceph-mon[102832]: pgmap v8499: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 10 op/s 2023-09-10T03:40:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15635.240873 secs 2023-09-10T03:40:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:03 smithi082 ceph-mon[98694]: pgmap v8499: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 10 op/s 2023-09-10T03:40:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15635.240873 secs 2023-09-10T03:40:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15635.952139 secs 2023-09-10T03:40:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15635.952139 secs 2023-09-10T03:40:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15635.952139 secs 2023-09-10T03:40:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:05 smithi098 ceph-mon[103124]: pgmap v8500: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 10 op/s 2023-09-10T03:40:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:05 smithi170 ceph-mon[102832]: pgmap v8500: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 10 op/s 2023-09-10T03:40:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:05 smithi082 ceph-mon[98694]: pgmap v8500: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 10 op/s 2023-09-10T03:40:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:08 smithi170 ceph-mon[102832]: pgmap v8501: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 206 KiB/s wr, 29 op/s 2023-09-10T03:40:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15639.031176 secs 2023-09-10T03:40:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:08 smithi082 ceph-mon[98694]: pgmap v8501: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 206 KiB/s wr, 29 op/s 2023-09-10T03:40:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15639.031176 secs 2023-09-10T03:40:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:08 smithi098 ceph-mon[103124]: pgmap v8501: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 206 KiB/s wr, 29 op/s 2023-09-10T03:40:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15639.031176 secs 2023-09-10T03:40:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15640.240928 secs 2023-09-10T03:40:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:09 smithi170 ceph-mon[102832]: osdmap e599: 12 total, 12 up, 12 in 2023-09-10T03:40:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15640.240928 secs 2023-09-10T03:40:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:09 smithi082 ceph-mon[98694]: osdmap e599: 12 total, 12 up, 12 in 2023-09-10T03:40:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15640.240928 secs 2023-09-10T03:40:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:09 smithi098 ceph-mon[103124]: osdmap e599: 12 total, 12 up, 12 in 2023-09-10T03:40:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15640.952203 secs 2023-09-10T03:40:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:10 smithi170 ceph-mon[102832]: pgmap v8503: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 110 KiB/s wr, 29 op/s 2023-09-10T03:40:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:40:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15640.952203 secs 2023-09-10T03:40:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:10 smithi082 ceph-mon[98694]: pgmap v8503: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 110 KiB/s wr, 29 op/s 2023-09-10T03:40:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:40:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15640.952203 secs 2023-09-10T03:40:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:10 smithi098 ceph-mon[103124]: pgmap v8503: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 110 KiB/s wr, 29 op/s 2023-09-10T03:40:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:40:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:12 smithi170 ceph-mon[102832]: pgmap v8504: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 28 op/s 2023-09-10T03:40:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:12 smithi082 ceph-mon[98694]: pgmap v8504: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 28 op/s 2023-09-10T03:40:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:12 smithi098 ceph-mon[103124]: pgmap v8504: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 28 op/s 2023-09-10T03:40:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:40:13.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:40:13.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:40:13.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:40:13.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15644.031235 secs 2023-09-10T03:40:13.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:13.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:13.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:13.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:13.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:13 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T03:40:13.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:13 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T03:40:13.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:13.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:13.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:13.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:13.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:13 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 96589k 2023-09-10T03:40:13.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:13 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 98907943: error parsing value: Value '98907943' is below minimum 939524096 2023-09-10T03:40:13.682 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:13 smithi170 ceph-mon[102832]: osdmap e600: 12 total, 12 up, 12 in 2023-09-10T03:40:13.682 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:13 smithi170 ceph-mon[102832]: pgmap v8506: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 69 KiB/s wr, 35 op/s 2023-09-10T03:40:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:40:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:40:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:40:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:40:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15644.031235 secs 2023-09-10T03:40:13.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:13.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:13.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:13.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:13.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:13 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T03:40:13.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:13 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T03:40:13.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:13.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:13.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:13.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:13.711 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:13 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 96589k 2023-09-10T03:40:13.711 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:13 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 98907943: error parsing value: Value '98907943' is below minimum 939524096 2023-09-10T03:40:13.711 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:13 smithi082 ceph-mon[98694]: osdmap e600: 12 total, 12 up, 12 in 2023-09-10T03:40:13.711 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:13 smithi082 ceph-mon[98694]: pgmap v8506: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 69 KiB/s wr, 35 op/s 2023-09-10T03:40:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:40:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:40:13.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:40:13.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:40:13.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15644.031235 secs 2023-09-10T03:40:13.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:13.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:13.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:13.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:13.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:13 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T03:40:13.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:13 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T03:40:13.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:13.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:13.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:13.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:13.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:13 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 96589k 2023-09-10T03:40:13.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:13 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 98907943: error parsing value: Value '98907943' is below minimum 939524096 2023-09-10T03:40:13.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:13 smithi098 ceph-mon[103124]: osdmap e600: 12 total, 12 up, 12 in 2023-09-10T03:40:13.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:13 smithi098 ceph-mon[103124]: pgmap v8506: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 69 KiB/s wr, 35 op/s 2023-09-10T03:40:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15645.241013 secs 2023-09-10T03:40:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:40:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:40:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:40:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:40:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:40:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15645.241013 secs 2023-09-10T03:40:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:40:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:40:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:14.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:14.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:14.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:40:14.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:40:14.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:40:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15645.241013 secs 2023-09-10T03:40:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:40:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:40:14.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:14.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:14.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:14.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:40:14.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:40:14.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:40:14.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:40:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15645.952268 secs 2023-09-10T03:40:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:15 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T03:40:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:15 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T03:40:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:15 smithi170 ceph-mon[102832]: pgmap v8507: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:40:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15645.952268 secs 2023-09-10T03:40:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:15 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T03:40:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:15 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T03:40:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:15 smithi082 ceph-mon[98694]: pgmap v8507: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:40:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15645.952268 secs 2023-09-10T03:40:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:15 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T03:40:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:15 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T03:40:15.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:15 smithi098 ceph-mon[103124]: pgmap v8507: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:40:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:17 smithi082 ceph-mon[98694]: pgmap v8508: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 13 op/s 2023-09-10T03:40:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15649.031331 secs 2023-09-10T03:40:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:17 smithi098 ceph-mon[103124]: pgmap v8508: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 13 op/s 2023-09-10T03:40:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15649.031331 secs 2023-09-10T03:40:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:17 smithi170 ceph-mon[102832]: pgmap v8508: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 13 op/s 2023-09-10T03:40:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15649.031331 secs 2023-09-10T03:40:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15650.241075 secs 2023-09-10T03:40:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15650.241075 secs 2023-09-10T03:40:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15650.241075 secs 2023-09-10T03:40:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15650.952333 secs 2023-09-10T03:40:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:19 smithi098 ceph-mon[103124]: pgmap v8509: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T03:40:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15650.952333 secs 2023-09-10T03:40:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:19 smithi170 ceph-mon[102832]: pgmap v8509: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T03:40:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15650.952333 secs 2023-09-10T03:40:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:19 smithi082 ceph-mon[98694]: pgmap v8509: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T03:40:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:21 smithi098 ceph-mon[103124]: pgmap v8510: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T03:40:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:21 smithi170 ceph-mon[102832]: pgmap v8510: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T03:40:22.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:21 smithi082 ceph-mon[98694]: pgmap v8510: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T03:40:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:40:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:40:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:40:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:23 smithi098 ceph-mon[103124]: pgmap v8511: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T03:40:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:40:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:40:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:40:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15655.241159 secs 2023-09-10T03:40:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:23 smithi170 ceph-mon[102832]: pgmap v8511: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T03:40:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:40:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:40:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:40:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15655.241159 secs 2023-09-10T03:40:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:23 smithi082 ceph-mon[98694]: pgmap v8511: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T03:40:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:40:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:40:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:40:24.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15655.241159 secs 2023-09-10T03:40:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15655.952399 secs 2023-09-10T03:40:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15655.952399 secs 2023-09-10T03:40:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15655.952399 secs 2023-09-10T03:40:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:25 smithi098 ceph-mon[103124]: pgmap v8512: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 9 op/s 2023-09-10T03:40:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:25 smithi170 ceph-mon[102832]: pgmap v8512: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 9 op/s 2023-09-10T03:40:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:25 smithi082 ceph-mon[98694]: pgmap v8512: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 9 op/s 2023-09-10T03:40:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:27 smithi098 ceph-mon[103124]: pgmap v8513: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 172 KiB/s wr, 32 op/s 2023-09-10T03:40:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:27 smithi170 ceph-mon[102832]: pgmap v8513: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 172 KiB/s wr, 32 op/s 2023-09-10T03:40:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:27 smithi082 ceph-mon[98694]: pgmap v8513: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 172 KiB/s wr, 32 op/s 2023-09-10T03:40:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15660.241218 secs 2023-09-10T03:40:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:28 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): Behind on trimming (333/128) 2023-09-10T03:40:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15660.241218 secs 2023-09-10T03:40:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:28 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): Behind on trimming (333/128) 2023-09-10T03:40:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15660.241218 secs 2023-09-10T03:40:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:28 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): Behind on trimming (333/128) 2023-09-10T03:40:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15660.952465 secs 2023-09-10T03:40:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:29 smithi098 ceph-mon[103124]: pgmap v8514: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 26 op/s 2023-09-10T03:40:30.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:29 smithi098 ceph-mon[103124]: Health check update: 1 MDSs behind on trimming (MDS_TRIM) 2023-09-10T03:40:30.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:29 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:40:30.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:29 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:40:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15660.952465 secs 2023-09-10T03:40:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:29 smithi170 ceph-mon[102832]: pgmap v8514: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 26 op/s 2023-09-10T03:40:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:29 smithi170 ceph-mon[102832]: Health check update: 1 MDSs behind on trimming (MDS_TRIM) 2023-09-10T03:40:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:29 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:40:30.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:29 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:40:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15660.952465 secs 2023-09-10T03:40:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:29 smithi082 ceph-mon[98694]: pgmap v8514: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 26 op/s 2023-09-10T03:40:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:29 smithi082 ceph-mon[98694]: Health check update: 1 MDSs behind on trimming (MDS_TRIM) 2023-09-10T03:40:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:29 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T03:40:30.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:29 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:40:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:30 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): Behind on trimming (334/128) 2023-09-10T03:40:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:30 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): Behind on trimming (334/128) 2023-09-10T03:40:31.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:30 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): Behind on trimming (334/128) 2023-09-10T03:40:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:31 smithi098 ceph-mon[103124]: pgmap v8515: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 187 KiB/s wr, 32 op/s 2023-09-10T03:40:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:31 smithi098 ceph-mon[103124]: Health check cleared: MDS_TRIM (was: 1 MDSs behind on trimming) 2023-09-10T03:40:32.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:40:32.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:40:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:31 smithi170 ceph-mon[102832]: pgmap v8515: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 187 KiB/s wr, 32 op/s 2023-09-10T03:40:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:31 smithi170 ceph-mon[102832]: Health check cleared: MDS_TRIM (was: 1 MDSs behind on trimming) 2023-09-10T03:40:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:40:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:40:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:31 smithi082 ceph-mon[98694]: pgmap v8515: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 187 KiB/s wr, 32 op/s 2023-09-10T03:40:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:31 smithi082 ceph-mon[98694]: Health check cleared: MDS_TRIM (was: 1 MDSs behind on trimming) 2023-09-10T03:40:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:40:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:40:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:33 smithi170 ceph-mon[102832]: pgmap v8516: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 186 KiB/s wr, 27 op/s 2023-09-10T03:40:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15665.241278 secs 2023-09-10T03:40:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:33 smithi082 ceph-mon[98694]: pgmap v8516: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 186 KiB/s wr, 27 op/s 2023-09-10T03:40:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15665.241278 secs 2023-09-10T03:40:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:33 smithi098 ceph-mon[103124]: pgmap v8516: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 186 KiB/s wr, 27 op/s 2023-09-10T03:40:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15665.241278 secs 2023-09-10T03:40:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15665.952488 secs 2023-09-10T03:40:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15665.952488 secs 2023-09-10T03:40:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15665.952488 secs 2023-09-10T03:40:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:35 smithi170 ceph-mon[102832]: pgmap v8517: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 186 KiB/s wr, 27 op/s 2023-09-10T03:40:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:35 smithi082 ceph-mon[98694]: pgmap v8517: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 186 KiB/s wr, 27 op/s 2023-09-10T03:40:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:35 smithi098 ceph-mon[103124]: pgmap v8517: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 186 KiB/s wr, 27 op/s 2023-09-10T03:40:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:37 smithi098 ceph-mon[103124]: pgmap v8518: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 225 KiB/s wr, 33 op/s 2023-09-10T03:40:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:37 smithi170 ceph-mon[102832]: pgmap v8518: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 225 KiB/s wr, 33 op/s 2023-09-10T03:40:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:37 smithi082 ceph-mon[98694]: pgmap v8518: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 225 KiB/s wr, 33 op/s 2023-09-10T03:40:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15670.241335 secs 2023-09-10T03:40:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15670.241335 secs 2023-09-10T03:40:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15670.241335 secs 2023-09-10T03:40:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15670.952555 secs 2023-09-10T03:40:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:39 smithi098 ceph-mon[103124]: pgmap v8519: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T03:40:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15670.952555 secs 2023-09-10T03:40:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:39 smithi170 ceph-mon[102832]: pgmap v8519: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T03:40:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15670.952555 secs 2023-09-10T03:40:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:39 smithi082 ceph-mon[98694]: pgmap v8519: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T03:40:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:41 smithi098 ceph-mon[103124]: pgmap v8520: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 329 KiB/s wr, 17 op/s 2023-09-10T03:40:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:41 smithi170 ceph-mon[102832]: pgmap v8520: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 329 KiB/s wr, 17 op/s 2023-09-10T03:40:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:41 smithi082 ceph-mon[98694]: pgmap v8520: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 329 KiB/s wr, 17 op/s 2023-09-10T03:40:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:43 smithi170 ceph-mon[102832]: pgmap v8521: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T03:40:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15675.241420 secs 2023-09-10T03:40:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:43 smithi082 ceph-mon[98694]: pgmap v8521: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T03:40:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15675.241420 secs 2023-09-10T03:40:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:43 smithi098 ceph-mon[103124]: pgmap v8521: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T03:40:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15675.241420 secs 2023-09-10T03:40:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15675.952620 secs 2023-09-10T03:40:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15675.952620 secs 2023-09-10T03:40:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15675.952620 secs 2023-09-10T03:40:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:45 smithi170 ceph-mon[102832]: pgmap v8522: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T03:40:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:45 smithi082 ceph-mon[98694]: pgmap v8522: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T03:40:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:45 smithi098 ceph-mon[103124]: pgmap v8522: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T03:40:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:47 smithi098 ceph-mon[103124]: pgmap v8523: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 328 KiB/s wr, 18 op/s 2023-09-10T03:40:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:47 smithi170 ceph-mon[102832]: pgmap v8523: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 328 KiB/s wr, 18 op/s 2023-09-10T03:40:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:47 smithi082 ceph-mon[98694]: pgmap v8523: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 328 KiB/s wr, 18 op/s 2023-09-10T03:40:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15680.241479 secs 2023-09-10T03:40:49.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15680.241479 secs 2023-09-10T03:40:49.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15680.241479 secs 2023-09-10T03:40:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15680.952684 secs 2023-09-10T03:40:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:49 smithi098 ceph-mon[103124]: pgmap v8524: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 289 KiB/s wr, 12 op/s 2023-09-10T03:40:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15680.952684 secs 2023-09-10T03:40:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:49 smithi170 ceph-mon[102832]: pgmap v8524: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 289 KiB/s wr, 12 op/s 2023-09-10T03:40:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15680.952684 secs 2023-09-10T03:40:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:49 smithi082 ceph-mon[98694]: pgmap v8524: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 289 KiB/s wr, 12 op/s 2023-09-10T03:40:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:51 smithi098 ceph-mon[103124]: pgmap v8525: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 441 KiB/s wr, 19 op/s 2023-09-10T03:40:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:51 smithi170 ceph-mon[102832]: pgmap v8525: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 441 KiB/s wr, 19 op/s 2023-09-10T03:40:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:51 smithi082 ceph-mon[98694]: pgmap v8525: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 441 KiB/s wr, 19 op/s 2023-09-10T03:40:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15684.031934 secs 2023-09-10T03:40:53.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15684.031934 secs 2023-09-10T03:40:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15684.031934 secs 2023-09-10T03:40:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:53 smithi098 ceph-mon[103124]: pgmap v8526: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 217 KiB/s wr, 13 op/s 2023-09-10T03:40:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15685.241494 secs 2023-09-10T03:40:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:53 smithi170 ceph-mon[102832]: pgmap v8526: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 217 KiB/s wr, 13 op/s 2023-09-10T03:40:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15685.241494 secs 2023-09-10T03:40:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:53 smithi082 ceph-mon[98694]: pgmap v8526: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 217 KiB/s wr, 13 op/s 2023-09-10T03:40:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15685.241494 secs 2023-09-10T03:40:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15685.952752 secs 2023-09-10T03:40:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15685.952752 secs 2023-09-10T03:40:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15685.952752 secs 2023-09-10T03:40:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:55 smithi098 ceph-mon[103124]: pgmap v8527: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 217 KiB/s wr, 13 op/s 2023-09-10T03:40:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:40:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:40:56.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:40:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:55 smithi170 ceph-mon[102832]: pgmap v8527: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 217 KiB/s wr, 13 op/s 2023-09-10T03:40:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:40:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:40:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:40:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:55 smithi082 ceph-mon[98694]: pgmap v8527: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 217 KiB/s wr, 13 op/s 2023-09-10T03:40:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:40:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:40:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:40:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:57 smithi098 ceph-mon[103124]: pgmap v8528: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 266 KiB/s wr, 18 op/s 2023-09-10T03:40:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15689.031993 secs 2023-09-10T03:40:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:57 smithi170 ceph-mon[102832]: pgmap v8528: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 266 KiB/s wr, 18 op/s 2023-09-10T03:40:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15689.031993 secs 2023-09-10T03:40:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:57 smithi082 ceph-mon[98694]: pgmap v8528: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 266 KiB/s wr, 18 op/s 2023-09-10T03:40:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15689.031993 secs 2023-09-10T03:40:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15690.241566 secs 2023-09-10T03:40:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15690.241566 secs 2023-09-10T03:40:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15690.241566 secs 2023-09-10T03:41:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15690.952820 secs 2023-09-10T03:41:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:40:59 smithi098 ceph-mon[103124]: pgmap v8529: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T03:41:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15690.952820 secs 2023-09-10T03:41:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:40:59 smithi170 ceph-mon[102832]: pgmap v8529: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T03:41:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15690.952820 secs 2023-09-10T03:41:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:40:59 smithi082 ceph-mon[98694]: pgmap v8529: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T03:41:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:01 smithi098 ceph-mon[103124]: pgmap v8530: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 322 KiB/s wr, 18 op/s 2023-09-10T03:41:02.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:01 smithi170 ceph-mon[102832]: pgmap v8530: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 322 KiB/s wr, 18 op/s 2023-09-10T03:41:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:01 smithi082 ceph-mon[98694]: pgmap v8530: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 322 KiB/s wr, 18 op/s 2023-09-10T03:41:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15694.032104 secs 2023-09-10T03:41:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15694.032104 secs 2023-09-10T03:41:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15694.032104 secs 2023-09-10T03:41:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:04 smithi098 ceph-mon[103124]: pgmap v8531: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 170 KiB/s wr, 11 op/s 2023-09-10T03:41:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15695.241626 secs 2023-09-10T03:41:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:04 smithi170 ceph-mon[102832]: pgmap v8531: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 170 KiB/s wr, 11 op/s 2023-09-10T03:41:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15695.241626 secs 2023-09-10T03:41:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:03 smithi082 ceph-mon[98694]: pgmap v8531: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 170 KiB/s wr, 11 op/s 2023-09-10T03:41:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15695.241626 secs 2023-09-10T03:41:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15695.952882 secs 2023-09-10T03:41:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15695.952882 secs 2023-09-10T03:41:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15695.952882 secs 2023-09-10T03:41:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:06 smithi170 ceph-mon[102832]: pgmap v8532: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 170 KiB/s wr, 11 op/s 2023-09-10T03:41:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:06 smithi082 ceph-mon[98694]: pgmap v8532: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 170 KiB/s wr, 11 op/s 2023-09-10T03:41:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:06 smithi098 ceph-mon[103124]: pgmap v8532: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 170 KiB/s wr, 11 op/s 2023-09-10T03:41:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:08 smithi170 ceph-mon[102832]: pgmap v8533: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 216 KiB/s wr, 31 op/s 2023-09-10T03:41:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15699.032169 secs 2023-09-10T03:41:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:08 smithi082 ceph-mon[98694]: pgmap v8533: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 216 KiB/s wr, 31 op/s 2023-09-10T03:41:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15699.032169 secs 2023-09-10T03:41:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:08 smithi098 ceph-mon[103124]: pgmap v8533: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 216 KiB/s wr, 31 op/s 2023-09-10T03:41:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15699.032169 secs 2023-09-10T03:41:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15700.241686 secs 2023-09-10T03:41:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:09 smithi170 ceph-mon[102832]: osdmap e601: 12 total, 12 up, 12 in 2023-09-10T03:41:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15700.241686 secs 2023-09-10T03:41:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:09 smithi082 ceph-mon[98694]: osdmap e601: 12 total, 12 up, 12 in 2023-09-10T03:41:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15700.241686 secs 2023-09-10T03:41:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:09 smithi098 ceph-mon[103124]: osdmap e601: 12 total, 12 up, 12 in 2023-09-10T03:41:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15700.952947 secs 2023-09-10T03:41:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:10 smithi170 ceph-mon[102832]: pgmap v8535: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 200 KiB/s wr, 30 op/s 2023-09-10T03:41:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15700.952947 secs 2023-09-10T03:41:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:10 smithi082 ceph-mon[98694]: pgmap v8535: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 200 KiB/s wr, 30 op/s 2023-09-10T03:41:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15700.952947 secs 2023-09-10T03:41:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:10 smithi098 ceph-mon[103124]: pgmap v8535: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 200 KiB/s wr, 30 op/s 2023-09-10T03:41:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:12 smithi170 ceph-mon[102832]: pgmap v8536: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 105 KiB/s wr, 30 op/s 2023-09-10T03:41:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:12 smithi082 ceph-mon[98694]: pgmap v8536: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 105 KiB/s wr, 30 op/s 2023-09-10T03:41:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:12 smithi098 ceph-mon[103124]: pgmap v8536: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 105 KiB/s wr, 30 op/s 2023-09-10T03:41:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15704.032223 secs 2023-09-10T03:41:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:13 smithi170 ceph-mon[102832]: osdmap e602: 12 total, 12 up, 12 in 2023-09-10T03:41:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15704.032223 secs 2023-09-10T03:41:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:13 smithi082 ceph-mon[98694]: osdmap e602: 12 total, 12 up, 12 in 2023-09-10T03:41:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15704.032223 secs 2023-09-10T03:41:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:13 smithi098 ceph-mon[103124]: osdmap e602: 12 total, 12 up, 12 in 2023-09-10T03:41:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:14 smithi170 ceph-mon[102832]: pgmap v8538: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 131 KiB/s wr, 37 op/s 2023-09-10T03:41:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15705.241744 secs 2023-09-10T03:41:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:14 smithi082 ceph-mon[98694]: pgmap v8538: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 131 KiB/s wr, 37 op/s 2023-09-10T03:41:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15705.241744 secs 2023-09-10T03:41:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:14 smithi098 ceph-mon[103124]: pgmap v8538: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 131 KiB/s wr, 37 op/s 2023-09-10T03:41:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15705.241744 secs 2023-09-10T03:41:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15705.953013 secs 2023-09-10T03:41:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:41:15.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:41:15.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:41:15.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:41:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15705.953013 secs 2023-09-10T03:41:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:41:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:41:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:41:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:41:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15705.953013 secs 2023-09-10T03:41:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:41:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:41:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:41:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:41:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:16 smithi170 ceph-mon[102832]: pgmap v8539: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 7 op/s 2023-09-10T03:41:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:16 smithi082 ceph-mon[98694]: pgmap v8539: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 7 op/s 2023-09-10T03:41:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:16 smithi098 ceph-mon[103124]: pgmap v8539: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 7 op/s 2023-09-10T03:41:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:18 smithi170 ceph-mon[102832]: pgmap v8540: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 14 op/s 2023-09-10T03:41:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15709.032285 secs 2023-09-10T03:41:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:18 smithi082 ceph-mon[98694]: pgmap v8540: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 14 op/s 2023-09-10T03:41:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15709.032285 secs 2023-09-10T03:41:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:18 smithi098 ceph-mon[103124]: pgmap v8540: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 14 op/s 2023-09-10T03:41:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15709.032285 secs 2023-09-10T03:41:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15710.241802 secs 2023-09-10T03:41:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15710.241802 secs 2023-09-10T03:41:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15710.241802 secs 2023-09-10T03:41:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15710.953077 secs 2023-09-10T03:41:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:20 smithi170 ceph-mon[102832]: pgmap v8541: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T03:41:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15710.953077 secs 2023-09-10T03:41:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:20 smithi082 ceph-mon[98694]: pgmap v8541: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T03:41:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15710.953077 secs 2023-09-10T03:41:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:20 smithi098 ceph-mon[103124]: pgmap v8541: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T03:41:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:22 smithi170 ceph-mon[102832]: pgmap v8542: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T03:41:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:22 smithi082 ceph-mon[98694]: pgmap v8542: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T03:41:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:22 smithi098 ceph-mon[103124]: pgmap v8542: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T03:41:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:41:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:41:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:41:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:24 smithi170 ceph-mon[102832]: pgmap v8543: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T03:41:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15715.241872 secs 2023-09-10T03:41:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:41:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:41:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:41:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:24 smithi082 ceph-mon[98694]: pgmap v8543: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T03:41:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15715.241872 secs 2023-09-10T03:41:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:41:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:41:24.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:41:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:24 smithi098 ceph-mon[103124]: pgmap v8543: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 11 op/s 2023-09-10T03:41:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15715.241872 secs 2023-09-10T03:41:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:41:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:41:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:41:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15715.953141 secs 2023-09-10T03:41:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15715.953141 secs 2023-09-10T03:41:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15715.953141 secs 2023-09-10T03:41:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:26 smithi170 ceph-mon[102832]: pgmap v8544: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 53 KiB/s wr, 9 op/s 2023-09-10T03:41:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:26 smithi082 ceph-mon[98694]: pgmap v8544: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 53 KiB/s wr, 9 op/s 2023-09-10T03:41:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:26 smithi098 ceph-mon[103124]: pgmap v8544: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 53 KiB/s wr, 9 op/s 2023-09-10T03:41:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:28 smithi170 ceph-mon[102832]: pgmap v8545: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 156 KiB/s wr, 14 op/s 2023-09-10T03:41:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:28 smithi082 ceph-mon[98694]: pgmap v8545: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 156 KiB/s wr, 14 op/s 2023-09-10T03:41:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:28 smithi098 ceph-mon[103124]: pgmap v8545: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 156 KiB/s wr, 14 op/s 2023-09-10T03:41:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15720.241928 secs 2023-09-10T03:41:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15720.241928 secs 2023-09-10T03:41:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15720.241928 secs 2023-09-10T03:41:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15720.953207 secs 2023-09-10T03:41:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:30 smithi170 ceph-mon[102832]: pgmap v8546: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 9 op/s 2023-09-10T03:41:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15720.953207 secs 2023-09-10T03:41:30.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:30 smithi082 ceph-mon[98694]: pgmap v8546: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 9 op/s 2023-09-10T03:41:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15720.953207 secs 2023-09-10T03:41:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:30 smithi098 ceph-mon[103124]: pgmap v8546: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 9 op/s 2023-09-10T03:41:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:32 smithi170 ceph-mon[102832]: pgmap v8547: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 266 KiB/s wr, 16 op/s 2023-09-10T03:41:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:32 smithi082 ceph-mon[98694]: pgmap v8547: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 266 KiB/s wr, 16 op/s 2023-09-10T03:41:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:32 smithi098 ceph-mon[103124]: pgmap v8547: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 266 KiB/s wr, 16 op/s 2023-09-10T03:41:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:33 smithi170 ceph-mon[102832]: pgmap v8548: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 266 KiB/s wr, 12 op/s 2023-09-10T03:41:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15725.242011 secs 2023-09-10T03:41:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:33 smithi082 ceph-mon[98694]: pgmap v8548: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 266 KiB/s wr, 12 op/s 2023-09-10T03:41:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15725.242011 secs 2023-09-10T03:41:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:33 smithi098 ceph-mon[103124]: pgmap v8548: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 266 KiB/s wr, 12 op/s 2023-09-10T03:41:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15725.242011 secs 2023-09-10T03:41:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15725.953274 secs 2023-09-10T03:41:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15725.953274 secs 2023-09-10T03:41:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15725.953274 secs 2023-09-10T03:41:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:35 smithi170 ceph-mon[102832]: pgmap v8549: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 266 KiB/s wr, 12 op/s 2023-09-10T03:41:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:35 smithi082 ceph-mon[98694]: pgmap v8549: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 266 KiB/s wr, 12 op/s 2023-09-10T03:41:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:35 smithi098 ceph-mon[103124]: pgmap v8549: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 266 KiB/s wr, 12 op/s 2023-09-10T03:41:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:37 smithi098 ceph-mon[103124]: pgmap v8550: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 268 KiB/s wr, 17 op/s 2023-09-10T03:41:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:37 smithi170 ceph-mon[102832]: pgmap v8550: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 268 KiB/s wr, 17 op/s 2023-09-10T03:41:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:37 smithi082 ceph-mon[98694]: pgmap v8550: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 268 KiB/s wr, 17 op/s 2023-09-10T03:41:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15730.242070 secs 2023-09-10T03:41:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15730.242070 secs 2023-09-10T03:41:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15730.242070 secs 2023-09-10T03:41:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15730.953338 secs 2023-09-10T03:41:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:39 smithi098 ceph-mon[103124]: pgmap v8551: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 164 KiB/s wr, 12 op/s 2023-09-10T03:41:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15730.953338 secs 2023-09-10T03:41:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:39 smithi170 ceph-mon[102832]: pgmap v8551: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 164 KiB/s wr, 12 op/s 2023-09-10T03:41:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15730.953338 secs 2023-09-10T03:41:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:39 smithi082 ceph-mon[98694]: pgmap v8551: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 164 KiB/s wr, 12 op/s 2023-09-10T03:41:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:41 smithi098 ceph-mon[103124]: pgmap v8552: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 443 KiB/s wr, 19 op/s 2023-09-10T03:41:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:41 smithi170 ceph-mon[102832]: pgmap v8552: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 443 KiB/s wr, 19 op/s 2023-09-10T03:41:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:41 smithi082 ceph-mon[98694]: pgmap v8552: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 443 KiB/s wr, 19 op/s 2023-09-10T03:41:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:43 smithi170 ceph-mon[102832]: pgmap v8553: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 280 KiB/s wr, 12 op/s 2023-09-10T03:41:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15735.242135 secs 2023-09-10T03:41:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:43 smithi082 ceph-mon[98694]: pgmap v8553: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 280 KiB/s wr, 12 op/s 2023-09-10T03:41:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15735.242135 secs 2023-09-10T03:41:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:43 smithi098 ceph-mon[103124]: pgmap v8553: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 280 KiB/s wr, 12 op/s 2023-09-10T03:41:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15735.242135 secs 2023-09-10T03:41:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15735.953401 secs 2023-09-10T03:41:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15735.953401 secs 2023-09-10T03:41:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15735.953401 secs 2023-09-10T03:41:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:45 smithi170 ceph-mon[102832]: pgmap v8554: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 280 KiB/s wr, 12 op/s 2023-09-10T03:41:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:45 smithi082 ceph-mon[98694]: pgmap v8554: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 280 KiB/s wr, 12 op/s 2023-09-10T03:41:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:45 smithi098 ceph-mon[103124]: pgmap v8554: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 280 KiB/s wr, 12 op/s 2023-09-10T03:41:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:47 smithi098 ceph-mon[103124]: pgmap v8555: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 360 KiB/s wr, 18 op/s 2023-09-10T03:41:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:47 smithi170 ceph-mon[102832]: pgmap v8555: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 360 KiB/s wr, 18 op/s 2023-09-10T03:41:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:47 smithi082 ceph-mon[98694]: pgmap v8555: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 360 KiB/s wr, 18 op/s 2023-09-10T03:41:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15740.242197 secs 2023-09-10T03:41:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15740.242197 secs 2023-09-10T03:41:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15740.242197 secs 2023-09-10T03:41:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15740.953473 secs 2023-09-10T03:41:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:49 smithi098 ceph-mon[103124]: pgmap v8556: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 359 KiB/s wr, 12 op/s 2023-09-10T03:41:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15740.953473 secs 2023-09-10T03:41:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:49 smithi170 ceph-mon[102832]: pgmap v8556: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 359 KiB/s wr, 12 op/s 2023-09-10T03:41:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15740.953473 secs 2023-09-10T03:41:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:49 smithi082 ceph-mon[98694]: pgmap v8556: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 359 KiB/s wr, 12 op/s 2023-09-10T03:41:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:51 smithi098 ceph-mon[103124]: pgmap v8557: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 412 KiB/s wr, 18 op/s 2023-09-10T03:41:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:51 smithi170 ceph-mon[102832]: pgmap v8557: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 412 KiB/s wr, 18 op/s 2023-09-10T03:41:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:51 smithi082 ceph-mon[98694]: pgmap v8557: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 412 KiB/s wr, 18 op/s 2023-09-10T03:41:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15744.032850 secs 2023-09-10T03:41:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15744.032850 secs 2023-09-10T03:41:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15744.032850 secs 2023-09-10T03:41:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:53 smithi098 ceph-mon[103124]: pgmap v8558: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T03:41:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15745.242255 secs 2023-09-10T03:41:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:53 smithi170 ceph-mon[102832]: pgmap v8558: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T03:41:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15745.242255 secs 2023-09-10T03:41:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:53 smithi082 ceph-mon[98694]: pgmap v8558: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T03:41:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15745.242255 secs 2023-09-10T03:41:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15745.953486 secs 2023-09-10T03:41:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15745.953486 secs 2023-09-10T03:41:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15745.953486 secs 2023-09-10T03:41:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:55 smithi098 ceph-mon[103124]: pgmap v8559: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T03:41:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:41:56.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:41:56.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:41:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:55 smithi170 ceph-mon[102832]: pgmap v8559: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T03:41:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:41:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:41:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:41:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:55 smithi082 ceph-mon[98694]: pgmap v8559: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T03:41:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:41:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:41:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:41:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:58 smithi170 ceph-mon[102832]: pgmap v8560: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 243 KiB/s wr, 17 op/s 2023-09-10T03:41:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15749.032965 secs 2023-09-10T03:41:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:58 smithi082 ceph-mon[98694]: pgmap v8560: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 243 KiB/s wr, 17 op/s 2023-09-10T03:41:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15749.032965 secs 2023-09-10T03:41:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:58 smithi098 ceph-mon[103124]: pgmap v8560: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 243 KiB/s wr, 17 op/s 2023-09-10T03:41:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15749.032965 secs 2023-09-10T03:41:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:41:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15750.242361 secs 2023-09-10T03:41:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:41:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15750.242361 secs 2023-09-10T03:41:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:41:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15750.242361 secs 2023-09-10T03:42:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15750.953550 secs 2023-09-10T03:42:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:00 smithi170 ceph-mon[102832]: pgmap v8561: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-09-10T03:42:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15750.953550 secs 2023-09-10T03:42:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:00 smithi082 ceph-mon[98694]: pgmap v8561: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-09-10T03:42:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15750.953550 secs 2023-09-10T03:42:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:00 smithi098 ceph-mon[103124]: pgmap v8561: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-09-10T03:42:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:02 smithi170 ceph-mon[102832]: pgmap v8562: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 215 KiB/s wr, 17 op/s 2023-09-10T03:42:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:02 smithi082 ceph-mon[98694]: pgmap v8562: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 215 KiB/s wr, 17 op/s 2023-09-10T03:42:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:02 smithi098 ceph-mon[103124]: pgmap v8562: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 215 KiB/s wr, 17 op/s 2023-09-10T03:42:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15754.033026 secs 2023-09-10T03:42:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15754.033026 secs 2023-09-10T03:42:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15754.033026 secs 2023-09-10T03:42:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:04 smithi170 ceph-mon[102832]: pgmap v8563: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T03:42:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15755.242420 secs 2023-09-10T03:42:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:04 smithi082 ceph-mon[98694]: pgmap v8563: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T03:42:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15755.242420 secs 2023-09-10T03:42:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:04 smithi098 ceph-mon[103124]: pgmap v8563: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T03:42:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15755.242420 secs 2023-09-10T03:42:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15755.953614 secs 2023-09-10T03:42:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15755.953614 secs 2023-09-10T03:42:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15755.953614 secs 2023-09-10T03:42:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:06 smithi170 ceph-mon[102832]: pgmap v8564: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T03:42:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:06 smithi082 ceph-mon[98694]: pgmap v8564: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T03:42:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:06 smithi098 ceph-mon[103124]: pgmap v8564: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T03:42:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:08 smithi170 ceph-mon[102832]: pgmap v8565: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 214 KiB/s wr, 31 op/s 2023-09-10T03:42:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15759.033106 secs 2023-09-10T03:42:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:08 smithi082 ceph-mon[98694]: pgmap v8565: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 214 KiB/s wr, 31 op/s 2023-09-10T03:42:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15759.033106 secs 2023-09-10T03:42:08.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:08 smithi098 ceph-mon[103124]: pgmap v8565: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 214 KiB/s wr, 31 op/s 2023-09-10T03:42:08.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15759.033106 secs 2023-09-10T03:42:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15760.242480 secs 2023-09-10T03:42:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:09 smithi170 ceph-mon[102832]: osdmap e603: 12 total, 12 up, 12 in 2023-09-10T03:42:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15760.242480 secs 2023-09-10T03:42:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:09 smithi082 ceph-mon[98694]: osdmap e603: 12 total, 12 up, 12 in 2023-09-10T03:42:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15760.242480 secs 2023-09-10T03:42:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:09 smithi098 ceph-mon[103124]: osdmap e603: 12 total, 12 up, 12 in 2023-09-10T03:42:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15760.953677 secs 2023-09-10T03:42:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:10 smithi170 ceph-mon[102832]: pgmap v8567: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 126 KiB/s wr, 31 op/s 2023-09-10T03:42:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15760.953677 secs 2023-09-10T03:42:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:10 smithi082 ceph-mon[98694]: pgmap v8567: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 126 KiB/s wr, 31 op/s 2023-09-10T03:42:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15760.953677 secs 2023-09-10T03:42:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:10 smithi098 ceph-mon[103124]: pgmap v8567: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 126 KiB/s wr, 31 op/s 2023-09-10T03:42:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:12 smithi170 ceph-mon[102832]: pgmap v8568: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 127 KiB/s wr, 30 op/s 2023-09-10T03:42:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:12 smithi082 ceph-mon[98694]: pgmap v8568: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 127 KiB/s wr, 30 op/s 2023-09-10T03:42:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:12 smithi098 ceph-mon[103124]: pgmap v8568: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 127 KiB/s wr, 30 op/s 2023-09-10T03:42:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15764.033257 secs 2023-09-10T03:42:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:13 smithi170 ceph-mon[102832]: osdmap e604: 12 total, 12 up, 12 in 2023-09-10T03:42:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15764.033257 secs 2023-09-10T03:42:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:13 smithi082 ceph-mon[98694]: osdmap e604: 12 total, 12 up, 12 in 2023-09-10T03:42:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15764.033257 secs 2023-09-10T03:42:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:13 smithi098 ceph-mon[103124]: osdmap e604: 12 total, 12 up, 12 in 2023-09-10T03:42:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:14 smithi170 ceph-mon[102832]: pgmap v8570: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 158 KiB/s wr, 38 op/s 2023-09-10T03:42:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15765.242544 secs 2023-09-10T03:42:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:14 smithi082 ceph-mon[98694]: pgmap v8570: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 158 KiB/s wr, 38 op/s 2023-09-10T03:42:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15765.242544 secs 2023-09-10T03:42:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:14 smithi098 ceph-mon[103124]: pgmap v8570: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 158 KiB/s wr, 38 op/s 2023-09-10T03:42:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15765.242544 secs 2023-09-10T03:42:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15765.953740 secs 2023-09-10T03:42:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15765.953740 secs 2023-09-10T03:42:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15765.953740 secs 2023-09-10T03:42:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:16 smithi170 ceph-mon[102832]: pgmap v8571: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 79 KiB/s wr, 8 op/s 2023-09-10T03:42:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:42:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:42:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:42:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:16 smithi082 ceph-mon[98694]: pgmap v8571: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 79 KiB/s wr, 8 op/s 2023-09-10T03:42:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:42:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:42:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:42:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:16 smithi098 ceph-mon[103124]: pgmap v8571: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 79 KiB/s wr, 8 op/s 2023-09-10T03:42:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:42:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:42:16.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:42:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:42:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:42:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:42:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:42:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:42:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:42:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:42:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:42:17.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:42:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:18 smithi170 ceph-mon[102832]: pgmap v8572: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 14 op/s 2023-09-10T03:42:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15769.033365 secs 2023-09-10T03:42:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:18 smithi082 ceph-mon[98694]: pgmap v8572: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 14 op/s 2023-09-10T03:42:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15769.033365 secs 2023-09-10T03:42:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:18 smithi098 ceph-mon[103124]: pgmap v8572: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 14 op/s 2023-09-10T03:42:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15769.033365 secs 2023-09-10T03:42:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15770.242580 secs 2023-09-10T03:42:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15770.242580 secs 2023-09-10T03:42:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15770.242580 secs 2023-09-10T03:42:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15770.953803 secs 2023-09-10T03:42:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:20 smithi170 ceph-mon[102832]: pgmap v8573: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T03:42:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15770.953803 secs 2023-09-10T03:42:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:20 smithi082 ceph-mon[98694]: pgmap v8573: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T03:42:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15770.953803 secs 2023-09-10T03:42:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:20 smithi098 ceph-mon[103124]: pgmap v8573: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T03:42:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:22 smithi170 ceph-mon[102832]: pgmap v8574: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T03:42:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:22 smithi082 ceph-mon[98694]: pgmap v8574: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T03:42:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:22 smithi098 ceph-mon[103124]: pgmap v8574: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T03:42:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:42:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:42:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:42:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:24 smithi170 ceph-mon[102832]: pgmap v8575: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T03:42:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15775.242666 secs 2023-09-10T03:42:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:42:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:42:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:42:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:24 smithi082 ceph-mon[98694]: pgmap v8575: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T03:42:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15775.242666 secs 2023-09-10T03:42:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:42:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:42:24.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:42:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:24 smithi098 ceph-mon[103124]: pgmap v8575: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T03:42:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15775.242666 secs 2023-09-10T03:42:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:42:24.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:42:24.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:42:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15775.953870 secs 2023-09-10T03:42:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15775.953870 secs 2023-09-10T03:42:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15775.953870 secs 2023-09-10T03:42:26.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:26 smithi170 ceph-mon[102832]: pgmap v8576: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 9 op/s 2023-09-10T03:42:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:26 smithi082 ceph-mon[98694]: pgmap v8576: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 9 op/s 2023-09-10T03:42:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:26 smithi098 ceph-mon[103124]: pgmap v8576: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 9 op/s 2023-09-10T03:42:27.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:27 smithi170 ceph-mon[102832]: pgmap v8577: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 134 KiB/s wr, 15 op/s 2023-09-10T03:42:27.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:27 smithi082 ceph-mon[98694]: pgmap v8577: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 134 KiB/s wr, 15 op/s 2023-09-10T03:42:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:27 smithi098 ceph-mon[103124]: pgmap v8577: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 134 KiB/s wr, 15 op/s 2023-09-10T03:42:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15780.242724 secs 2023-09-10T03:42:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15780.242724 secs 2023-09-10T03:42:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15780.242724 secs 2023-09-10T03:42:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15780.953934 secs 2023-09-10T03:42:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:29 smithi082 ceph-mon[98694]: pgmap v8578: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 10 op/s 2023-09-10T03:42:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15780.953934 secs 2023-09-10T03:42:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:29 smithi098 ceph-mon[103124]: pgmap v8578: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 10 op/s 2023-09-10T03:42:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15780.953934 secs 2023-09-10T03:42:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:29 smithi170 ceph-mon[102832]: pgmap v8578: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 10 op/s 2023-09-10T03:42:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:31 smithi098 ceph-mon[103124]: pgmap v8579: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 201 KiB/s wr, 16 op/s 2023-09-10T03:42:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:31 smithi170 ceph-mon[102832]: pgmap v8579: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 201 KiB/s wr, 16 op/s 2023-09-10T03:42:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:31 smithi082 ceph-mon[98694]: pgmap v8579: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 201 KiB/s wr, 16 op/s 2023-09-10T03:42:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:33 smithi170 ceph-mon[102832]: pgmap v8580: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T03:42:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15785.242782 secs 2023-09-10T03:42:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:33 smithi082 ceph-mon[98694]: pgmap v8580: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T03:42:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15785.242782 secs 2023-09-10T03:42:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:33 smithi098 ceph-mon[103124]: pgmap v8580: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T03:42:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15785.242782 secs 2023-09-10T03:42:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15785.953997 secs 2023-09-10T03:42:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15785.953997 secs 2023-09-10T03:42:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15785.953997 secs 2023-09-10T03:42:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:35 smithi082 ceph-mon[98694]: pgmap v8581: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T03:42:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:35 smithi098 ceph-mon[103124]: pgmap v8581: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T03:42:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:35 smithi170 ceph-mon[102832]: pgmap v8581: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T03:42:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:37 smithi098 ceph-mon[103124]: pgmap v8582: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 335 KiB/s wr, 17 op/s 2023-09-10T03:42:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:37 smithi170 ceph-mon[102832]: pgmap v8582: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 335 KiB/s wr, 17 op/s 2023-09-10T03:42:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:37 smithi082 ceph-mon[98694]: pgmap v8582: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 335 KiB/s wr, 17 op/s 2023-09-10T03:42:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15790.242840 secs 2023-09-10T03:42:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15790.242840 secs 2023-09-10T03:42:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15790.242840 secs 2023-09-10T03:42:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15790.954061 secs 2023-09-10T03:42:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:40 smithi170 ceph-mon[102832]: pgmap v8583: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 267 KiB/s wr, 12 op/s 2023-09-10T03:42:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15790.954061 secs 2023-09-10T03:42:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:40 smithi082 ceph-mon[98694]: pgmap v8583: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 267 KiB/s wr, 12 op/s 2023-09-10T03:42:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15790.954061 secs 2023-09-10T03:42:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:40 smithi098 ceph-mon[103124]: pgmap v8583: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 267 KiB/s wr, 12 op/s 2023-09-10T03:42:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:42 smithi170 ceph-mon[102832]: pgmap v8584: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 441 KiB/s wr, 19 op/s 2023-09-10T03:42:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:42 smithi082 ceph-mon[98694]: pgmap v8584: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 441 KiB/s wr, 19 op/s 2023-09-10T03:42:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:42 smithi098 ceph-mon[103124]: pgmap v8584: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 441 KiB/s wr, 19 op/s 2023-09-10T03:42:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:43 smithi082 ceph-mon[98694]: pgmap v8585: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 373 KiB/s wr, 13 op/s 2023-09-10T03:42:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15795.242896 secs 2023-09-10T03:42:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:43 smithi098 ceph-mon[103124]: pgmap v8585: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 373 KiB/s wr, 13 op/s 2023-09-10T03:42:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15795.242896 secs 2023-09-10T03:42:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:43 smithi170 ceph-mon[102832]: pgmap v8585: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 373 KiB/s wr, 13 op/s 2023-09-10T03:42:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15795.242896 secs 2023-09-10T03:42:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15795.954126 secs 2023-09-10T03:42:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15795.954126 secs 2023-09-10T03:42:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15795.954126 secs 2023-09-10T03:42:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:45 smithi082 ceph-mon[98694]: pgmap v8586: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 373 KiB/s wr, 13 op/s 2023-09-10T03:42:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:45 smithi098 ceph-mon[103124]: pgmap v8586: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 373 KiB/s wr, 13 op/s 2023-09-10T03:42:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:45 smithi170 ceph-mon[102832]: pgmap v8586: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 373 KiB/s wr, 13 op/s 2023-09-10T03:42:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:48 smithi098 ceph-mon[103124]: pgmap v8587: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 429 KiB/s wr, 19 op/s 2023-09-10T03:42:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15799.033779 secs 2023-09-10T03:42:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:47 smithi170 ceph-mon[102832]: pgmap v8587: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 429 KiB/s wr, 19 op/s 2023-09-10T03:42:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15799.033779 secs 2023-09-10T03:42:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:47 smithi082 ceph-mon[98694]: pgmap v8587: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 429 KiB/s wr, 19 op/s 2023-09-10T03:42:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15799.033779 secs 2023-09-10T03:42:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15800.242976 secs 2023-09-10T03:42:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15800.242976 secs 2023-09-10T03:42:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15800.242976 secs 2023-09-10T03:42:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15800.954190 secs 2023-09-10T03:42:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:50 smithi170 ceph-mon[102832]: pgmap v8588: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 229 KiB/s wr, 12 op/s 2023-09-10T03:42:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15800.954190 secs 2023-09-10T03:42:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:50 smithi082 ceph-mon[98694]: pgmap v8588: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 229 KiB/s wr, 12 op/s 2023-09-10T03:42:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15800.954190 secs 2023-09-10T03:42:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:50 smithi098 ceph-mon[103124]: pgmap v8588: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 229 KiB/s wr, 12 op/s 2023-09-10T03:42:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:42:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:42:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:42:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:42:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:42:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:42:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:42:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:42:51.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:42:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:52 smithi170 ceph-mon[102832]: pgmap v8589: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 404 KiB/s wr, 18 op/s 2023-09-10T03:42:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:52 smithi082 ceph-mon[98694]: pgmap v8589: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 404 KiB/s wr, 18 op/s 2023-09-10T03:42:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:52 smithi098 ceph-mon[103124]: pgmap v8589: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 404 KiB/s wr, 18 op/s 2023-09-10T03:42:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15804.033857 secs 2023-09-10T03:42:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15804.033857 secs 2023-09-10T03:42:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15804.033857 secs 2023-09-10T03:42:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:54 smithi170 ceph-mon[102832]: pgmap v8590: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T03:42:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15805.243033 secs 2023-09-10T03:42:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:54 smithi082 ceph-mon[98694]: pgmap v8590: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T03:42:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15805.243033 secs 2023-09-10T03:42:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:54 smithi098 ceph-mon[103124]: pgmap v8590: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T03:42:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15805.243033 secs 2023-09-10T03:42:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15805.954249 secs 2023-09-10T03:42:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15805.954249 secs 2023-09-10T03:42:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15805.954249 secs 2023-09-10T03:42:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:56 smithi170 ceph-mon[102832]: pgmap v8591: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T03:42:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:56 smithi082 ceph-mon[98694]: pgmap v8591: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T03:42:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:56 smithi098 ceph-mon[103124]: pgmap v8591: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T03:42:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:58 smithi170 ceph-mon[102832]: pgmap v8592: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 346 KiB/s wr, 17 op/s 2023-09-10T03:42:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15809.033961 secs 2023-09-10T03:42:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:58 smithi082 ceph-mon[98694]: pgmap v8592: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 346 KiB/s wr, 17 op/s 2023-09-10T03:42:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15809.033961 secs 2023-09-10T03:42:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:58 smithi098 ceph-mon[103124]: pgmap v8592: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 346 KiB/s wr, 17 op/s 2023-09-10T03:42:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15809.033961 secs 2023-09-10T03:42:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:42:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15810.243101 secs 2023-09-10T03:42:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:42:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15810.243101 secs 2023-09-10T03:42:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:42:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15810.243101 secs 2023-09-10T03:43:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15810.954316 secs 2023-09-10T03:43:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:00 smithi170 ceph-mon[102832]: pgmap v8593: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 290 KiB/s wr, 11 op/s 2023-09-10T03:43:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15810.954316 secs 2023-09-10T03:43:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:00 smithi082 ceph-mon[98694]: pgmap v8593: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 290 KiB/s wr, 11 op/s 2023-09-10T03:43:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15810.954316 secs 2023-09-10T03:43:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:00 smithi098 ceph-mon[103124]: pgmap v8593: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 290 KiB/s wr, 11 op/s 2023-09-10T03:43:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:02 smithi170 ceph-mon[102832]: pgmap v8594: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 337 KiB/s wr, 16 op/s 2023-09-10T03:43:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:02 smithi082 ceph-mon[98694]: pgmap v8594: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 337 KiB/s wr, 16 op/s 2023-09-10T03:43:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:02 smithi098 ceph-mon[103124]: pgmap v8594: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 337 KiB/s wr, 16 op/s 2023-09-10T03:43:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15814.034021 secs 2023-09-10T03:43:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15814.034021 secs 2023-09-10T03:43:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15814.034021 secs 2023-09-10T03:43:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:04 smithi170 ceph-mon[102832]: pgmap v8595: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-09-10T03:43:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15815.243158 secs 2023-09-10T03:43:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:04 smithi082 ceph-mon[98694]: pgmap v8595: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-09-10T03:43:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15815.243158 secs 2023-09-10T03:43:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:04 smithi098 ceph-mon[103124]: pgmap v8595: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-09-10T03:43:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15815.243158 secs 2023-09-10T03:43:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15815.954378 secs 2023-09-10T03:43:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15815.954378 secs 2023-09-10T03:43:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15815.954378 secs 2023-09-10T03:43:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:06 smithi170 ceph-mon[102832]: pgmap v8596: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-09-10T03:43:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:06 smithi082 ceph-mon[98694]: pgmap v8596: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-09-10T03:43:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:06 smithi098 ceph-mon[103124]: pgmap v8596: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-09-10T03:43:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:08 smithi170 ceph-mon[102832]: pgmap v8597: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 220 KiB/s wr, 31 op/s 2023-09-10T03:43:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15819.034092 secs 2023-09-10T03:43:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:08 smithi082 ceph-mon[98694]: pgmap v8597: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 220 KiB/s wr, 31 op/s 2023-09-10T03:43:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15819.034092 secs 2023-09-10T03:43:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:08 smithi098 ceph-mon[103124]: pgmap v8597: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 220 KiB/s wr, 31 op/s 2023-09-10T03:43:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15819.034092 secs 2023-09-10T03:43:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15820.243216 secs 2023-09-10T03:43:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:09 smithi170 ceph-mon[102832]: osdmap e605: 12 total, 12 up, 12 in 2023-09-10T03:43:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15820.243216 secs 2023-09-10T03:43:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:09 smithi082 ceph-mon[98694]: osdmap e605: 12 total, 12 up, 12 in 2023-09-10T03:43:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15820.243216 secs 2023-09-10T03:43:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:09 smithi098 ceph-mon[103124]: osdmap e605: 12 total, 12 up, 12 in 2023-09-10T03:43:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15820.954443 secs 2023-09-10T03:43:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:10 smithi170 ceph-mon[102832]: pgmap v8599: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 125 KiB/s wr, 30 op/s 2023-09-10T03:43:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15820.954443 secs 2023-09-10T03:43:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:10 smithi082 ceph-mon[98694]: pgmap v8599: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 125 KiB/s wr, 30 op/s 2023-09-10T03:43:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15820.954443 secs 2023-09-10T03:43:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:10 smithi098 ceph-mon[103124]: pgmap v8599: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 125 KiB/s wr, 30 op/s 2023-09-10T03:43:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:12 smithi170 ceph-mon[102832]: pgmap v8600: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-10T03:43:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:12 smithi082 ceph-mon[98694]: pgmap v8600: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-10T03:43:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:12 smithi098 ceph-mon[103124]: pgmap v8600: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 69 KiB/s wr, 29 op/s 2023-09-10T03:43:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15824.034216 secs 2023-09-10T03:43:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:13 smithi170 ceph-mon[102832]: osdmap e606: 12 total, 12 up, 12 in 2023-09-10T03:43:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15824.034216 secs 2023-09-10T03:43:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:13 smithi082 ceph-mon[98694]: osdmap e606: 12 total, 12 up, 12 in 2023-09-10T03:43:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15824.034216 secs 2023-09-10T03:43:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:13 smithi098 ceph-mon[103124]: osdmap e606: 12 total, 12 up, 12 in 2023-09-10T03:43:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:14 smithi170 ceph-mon[102832]: pgmap v8602: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 87 KiB/s wr, 36 op/s 2023-09-10T03:43:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15825.243306 secs 2023-09-10T03:43:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:14 smithi082 ceph-mon[98694]: pgmap v8602: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 87 KiB/s wr, 36 op/s 2023-09-10T03:43:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15825.243306 secs 2023-09-10T03:43:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:14 smithi098 ceph-mon[103124]: pgmap v8602: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 87 KiB/s wr, 36 op/s 2023-09-10T03:43:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15825.243306 secs 2023-09-10T03:43:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15825.954485 secs 2023-09-10T03:43:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15825.954485 secs 2023-09-10T03:43:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15825.954485 secs 2023-09-10T03:43:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:16 smithi170 ceph-mon[102832]: pgmap v8603: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:43:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:16 smithi082 ceph-mon[98694]: pgmap v8603: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:43:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:16 smithi098 ceph-mon[103124]: pgmap v8603: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:43:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:43:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:43:17.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:43:17.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:43:17.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:43:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:43:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:43:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:43:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:43:17.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:43:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:43:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:43:17.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:43:17.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:43:17.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:43:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:18 smithi170 ceph-mon[102832]: pgmap v8604: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T03:43:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:43:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:43:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15829.034300 secs 2023-09-10T03:43:18.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:43:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:18 smithi082 ceph-mon[98694]: pgmap v8604: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T03:43:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:43:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:43:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15829.034300 secs 2023-09-10T03:43:18.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:43:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:18 smithi098 ceph-mon[103124]: pgmap v8604: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T03:43:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:43:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:43:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15829.034300 secs 2023-09-10T03:43:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:43:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15830.243374 secs 2023-09-10T03:43:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15830.243374 secs 2023-09-10T03:43:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15830.243374 secs 2023-09-10T03:43:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15830.954547 secs 2023-09-10T03:43:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:20 smithi170 ceph-mon[102832]: pgmap v8605: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T03:43:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15830.954547 secs 2023-09-10T03:43:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:20 smithi082 ceph-mon[98694]: pgmap v8605: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T03:43:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15830.954547 secs 2023-09-10T03:43:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:20 smithi098 ceph-mon[103124]: pgmap v8605: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T03:43:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:22 smithi170 ceph-mon[102832]: pgmap v8606: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T03:43:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:22 smithi082 ceph-mon[98694]: pgmap v8606: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T03:43:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:22 smithi098 ceph-mon[103124]: pgmap v8606: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T03:43:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:43:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:43:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:43:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:24 smithi170 ceph-mon[102832]: pgmap v8607: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T03:43:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15835.243435 secs 2023-09-10T03:43:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:43:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:43:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:43:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:24 smithi082 ceph-mon[98694]: pgmap v8607: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T03:43:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15835.243435 secs 2023-09-10T03:43:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:43:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:43:24.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:43:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:24 smithi098 ceph-mon[103124]: pgmap v8607: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T03:43:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15835.243435 secs 2023-09-10T03:43:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:43:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:43:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:43:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15835.954608 secs 2023-09-10T03:43:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15835.954608 secs 2023-09-10T03:43:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15835.954608 secs 2023-09-10T03:43:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:26 smithi170 ceph-mon[102832]: pgmap v8608: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 9 op/s 2023-09-10T03:43:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:26 smithi082 ceph-mon[98694]: pgmap v8608: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 9 op/s 2023-09-10T03:43:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:26 smithi098 ceph-mon[103124]: pgmap v8608: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 9 op/s 2023-09-10T03:43:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:28 smithi170 ceph-mon[102832]: pgmap v8609: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 175 KiB/s wr, 14 op/s 2023-09-10T03:43:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:28 smithi082 ceph-mon[98694]: pgmap v8609: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 175 KiB/s wr, 14 op/s 2023-09-10T03:43:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:28 smithi098 ceph-mon[103124]: pgmap v8609: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 175 KiB/s wr, 14 op/s 2023-09-10T03:43:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15840.243493 secs 2023-09-10T03:43:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15840.243493 secs 2023-09-10T03:43:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15840.243493 secs 2023-09-10T03:43:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15840.954673 secs 2023-09-10T03:43:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:30 smithi170 ceph-mon[102832]: pgmap v8610: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 9 op/s 2023-09-10T03:43:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15840.954673 secs 2023-09-10T03:43:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:30 smithi082 ceph-mon[98694]: pgmap v8610: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 9 op/s 2023-09-10T03:43:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15840.954673 secs 2023-09-10T03:43:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:30 smithi098 ceph-mon[103124]: pgmap v8610: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 9 op/s 2023-09-10T03:43:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:32 smithi170 ceph-mon[102832]: pgmap v8611: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 205 KiB/s wr, 15 op/s 2023-09-10T03:43:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:32 smithi082 ceph-mon[98694]: pgmap v8611: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 205 KiB/s wr, 15 op/s 2023-09-10T03:43:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:32 smithi098 ceph-mon[103124]: pgmap v8611: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 205 KiB/s wr, 15 op/s 2023-09-10T03:43:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:33 smithi082 ceph-mon[98694]: pgmap v8612: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 205 KiB/s wr, 10 op/s 2023-09-10T03:43:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15845.243551 secs 2023-09-10T03:43:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:33 smithi098 ceph-mon[103124]: pgmap v8612: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 205 KiB/s wr, 10 op/s 2023-09-10T03:43:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15845.243551 secs 2023-09-10T03:43:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:33 smithi170 ceph-mon[102832]: pgmap v8612: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 205 KiB/s wr, 10 op/s 2023-09-10T03:43:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15845.243551 secs 2023-09-10T03:43:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15845.954737 secs 2023-09-10T03:43:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15845.954737 secs 2023-09-10T03:43:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15845.954737 secs 2023-09-10T03:43:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:35 smithi082 ceph-mon[98694]: pgmap v8613: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 205 KiB/s wr, 10 op/s 2023-09-10T03:43:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:35 smithi098 ceph-mon[103124]: pgmap v8613: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 205 KiB/s wr, 10 op/s 2023-09-10T03:43:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:35 smithi170 ceph-mon[102832]: pgmap v8613: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 205 KiB/s wr, 10 op/s 2023-09-10T03:43:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:38 smithi170 ceph-mon[102832]: pgmap v8614: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 228 KiB/s wr, 17 op/s 2023-09-10T03:43:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:38 smithi082 ceph-mon[98694]: pgmap v8614: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 228 KiB/s wr, 17 op/s 2023-09-10T03:43:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:38 smithi098 ceph-mon[103124]: pgmap v8614: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 228 KiB/s wr, 17 op/s 2023-09-10T03:43:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15850.243609 secs 2023-09-10T03:43:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15850.243609 secs 2023-09-10T03:43:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15850.243609 secs 2023-09-10T03:43:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15850.954801 secs 2023-09-10T03:43:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:40 smithi170 ceph-mon[102832]: pgmap v8615: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 107 KiB/s wr, 12 op/s 2023-09-10T03:43:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15850.954801 secs 2023-09-10T03:43:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:40 smithi082 ceph-mon[98694]: pgmap v8615: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 107 KiB/s wr, 12 op/s 2023-09-10T03:43:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15850.954801 secs 2023-09-10T03:43:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:40 smithi098 ceph-mon[103124]: pgmap v8615: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 107 KiB/s wr, 12 op/s 2023-09-10T03:43:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:42 smithi170 ceph-mon[102832]: pgmap v8616: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 320 KiB/s wr, 19 op/s 2023-09-10T03:43:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:42 smithi082 ceph-mon[98694]: pgmap v8616: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 320 KiB/s wr, 19 op/s 2023-09-10T03:43:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:42 smithi098 ceph-mon[103124]: pgmap v8616: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 320 KiB/s wr, 19 op/s 2023-09-10T03:43:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:43 smithi082 ceph-mon[98694]: pgmap v8617: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 237 KiB/s wr, 13 op/s 2023-09-10T03:43:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15855.243670 secs 2023-09-10T03:43:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:43 smithi098 ceph-mon[103124]: pgmap v8617: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 237 KiB/s wr, 13 op/s 2023-09-10T03:43:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15855.243670 secs 2023-09-10T03:43:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:43 smithi170 ceph-mon[102832]: pgmap v8617: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 237 KiB/s wr, 13 op/s 2023-09-10T03:43:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15855.243670 secs 2023-09-10T03:43:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15855.954864 secs 2023-09-10T03:43:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15855.954864 secs 2023-09-10T03:43:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15855.954864 secs 2023-09-10T03:43:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:45 smithi082 ceph-mon[98694]: pgmap v8618: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 237 KiB/s wr, 13 op/s 2023-09-10T03:43:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:45 smithi098 ceph-mon[103124]: pgmap v8618: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 237 KiB/s wr, 13 op/s 2023-09-10T03:43:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:45 smithi170 ceph-mon[102832]: pgmap v8618: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 237 KiB/s wr, 13 op/s 2023-09-10T03:43:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:48 smithi170 ceph-mon[102832]: pgmap v8619: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 263 KiB/s wr, 18 op/s 2023-09-10T03:43:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:48 smithi082 ceph-mon[98694]: pgmap v8619: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 263 KiB/s wr, 18 op/s 2023-09-10T03:43:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:48 smithi098 ceph-mon[103124]: pgmap v8619: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 263 KiB/s wr, 18 op/s 2023-09-10T03:43:49.029 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15860.243729 secs 2023-09-10T03:43:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15860.243729 secs 2023-09-10T03:43:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15860.243729 secs 2023-09-10T03:43:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15860.954925 secs 2023-09-10T03:43:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:50 smithi170 ceph-mon[102832]: pgmap v8620: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 239 KiB/s wr, 11 op/s 2023-09-10T03:43:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15860.954925 secs 2023-09-10T03:43:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:50 smithi082 ceph-mon[98694]: pgmap v8620: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 239 KiB/s wr, 11 op/s 2023-09-10T03:43:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15860.954925 secs 2023-09-10T03:43:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:50 smithi098 ceph-mon[103124]: pgmap v8620: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 239 KiB/s wr, 11 op/s 2023-09-10T03:43:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:52 smithi170 ceph-mon[102832]: pgmap v8621: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 538 KiB/s wr, 18 op/s 2023-09-10T03:43:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:52 smithi082 ceph-mon[98694]: pgmap v8621: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 538 KiB/s wr, 18 op/s 2023-09-10T03:43:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:52 smithi098 ceph-mon[103124]: pgmap v8621: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 538 KiB/s wr, 18 op/s 2023-09-10T03:43:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15864.034839 secs 2023-09-10T03:43:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15864.034839 secs 2023-09-10T03:43:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15864.034839 secs 2023-09-10T03:43:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:54 smithi170 ceph-mon[102832]: pgmap v8622: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 325 KiB/s wr, 11 op/s 2023-09-10T03:43:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15865.243788 secs 2023-09-10T03:43:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:54 smithi082 ceph-mon[98694]: pgmap v8622: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 325 KiB/s wr, 11 op/s 2023-09-10T03:43:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15865.243788 secs 2023-09-10T03:43:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:54 smithi098 ceph-mon[103124]: pgmap v8622: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 325 KiB/s wr, 11 op/s 2023-09-10T03:43:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15865.243788 secs 2023-09-10T03:43:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15865.954985 secs 2023-09-10T03:43:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15865.954985 secs 2023-09-10T03:43:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15865.954985 secs 2023-09-10T03:43:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:56 smithi170 ceph-mon[102832]: pgmap v8623: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 325 KiB/s wr, 11 op/s 2023-09-10T03:43:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:43:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:43:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:43:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:56 smithi082 ceph-mon[98694]: pgmap v8623: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 325 KiB/s wr, 11 op/s 2023-09-10T03:43:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:43:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:43:56.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:43:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:56 smithi098 ceph-mon[103124]: pgmap v8623: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 325 KiB/s wr, 11 op/s 2023-09-10T03:43:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:43:56.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:43:56.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:43:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:58 smithi170 ceph-mon[102832]: pgmap v8624: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 369 KiB/s wr, 18 op/s 2023-09-10T03:43:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15869.034900 secs 2023-09-10T03:43:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:58 smithi082 ceph-mon[98694]: pgmap v8624: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 369 KiB/s wr, 18 op/s 2023-09-10T03:43:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15869.034900 secs 2023-09-10T03:43:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:58 smithi098 ceph-mon[103124]: pgmap v8624: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 369 KiB/s wr, 18 op/s 2023-09-10T03:43:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15869.034900 secs 2023-09-10T03:43:59.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:43:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15870.243850 secs 2023-09-10T03:43:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:43:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15870.243850 secs 2023-09-10T03:43:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:43:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15870.243850 secs 2023-09-10T03:44:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15870.955045 secs 2023-09-10T03:44:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:00 smithi170 ceph-mon[102832]: pgmap v8625: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 343 KiB/s wr, 13 op/s 2023-09-10T03:44:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15870.955045 secs 2023-09-10T03:44:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:00 smithi082 ceph-mon[98694]: pgmap v8625: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 343 KiB/s wr, 13 op/s 2023-09-10T03:44:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15870.955045 secs 2023-09-10T03:44:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:00 smithi098 ceph-mon[103124]: pgmap v8625: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 343 KiB/s wr, 13 op/s 2023-09-10T03:44:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:02 smithi170 ceph-mon[102832]: pgmap v8626: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 466 KiB/s wr, 18 op/s 2023-09-10T03:44:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:02 smithi082 ceph-mon[98694]: pgmap v8626: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 466 KiB/s wr, 18 op/s 2023-09-10T03:44:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:02 smithi098 ceph-mon[103124]: pgmap v8626: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 466 KiB/s wr, 18 op/s 2023-09-10T03:44:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15874.034958 secs 2023-09-10T03:44:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15874.034958 secs 2023-09-10T03:44:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15874.034958 secs 2023-09-10T03:44:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:04 smithi170 ceph-mon[102832]: pgmap v8627: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T03:44:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15875.243909 secs 2023-09-10T03:44:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:04 smithi082 ceph-mon[98694]: pgmap v8627: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T03:44:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15875.243909 secs 2023-09-10T03:44:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:04 smithi098 ceph-mon[103124]: pgmap v8627: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T03:44:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15875.243909 secs 2023-09-10T03:44:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15875.955108 secs 2023-09-10T03:44:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15875.955108 secs 2023-09-10T03:44:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15875.955108 secs 2023-09-10T03:44:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:06 smithi170 ceph-mon[102832]: pgmap v8628: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T03:44:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:06 smithi082 ceph-mon[98694]: pgmap v8628: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T03:44:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:06 smithi098 ceph-mon[103124]: pgmap v8628: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T03:44:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:08 smithi170 ceph-mon[102832]: pgmap v8629: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 213 KiB/s wr, 30 op/s 2023-09-10T03:44:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15879.035018 secs 2023-09-10T03:44:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:08 smithi082 ceph-mon[98694]: pgmap v8629: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 213 KiB/s wr, 30 op/s 2023-09-10T03:44:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15879.035018 secs 2023-09-10T03:44:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:08 smithi098 ceph-mon[103124]: pgmap v8629: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 213 KiB/s wr, 30 op/s 2023-09-10T03:44:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15879.035018 secs 2023-09-10T03:44:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15880.243971 secs 2023-09-10T03:44:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:09 smithi170 ceph-mon[102832]: osdmap e607: 12 total, 12 up, 12 in 2023-09-10T03:44:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15880.243971 secs 2023-09-10T03:44:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:09 smithi082 ceph-mon[98694]: osdmap e607: 12 total, 12 up, 12 in 2023-09-10T03:44:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15880.243971 secs 2023-09-10T03:44:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:09 smithi098 ceph-mon[103124]: osdmap e607: 12 total, 12 up, 12 in 2023-09-10T03:44:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15880.955170 secs 2023-09-10T03:44:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:10 smithi170 ceph-mon[102832]: pgmap v8631: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 203 KiB/s wr, 29 op/s 2023-09-10T03:44:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15880.955170 secs 2023-09-10T03:44:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:10 smithi082 ceph-mon[98694]: pgmap v8631: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 203 KiB/s wr, 29 op/s 2023-09-10T03:44:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15880.955170 secs 2023-09-10T03:44:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:10 smithi098 ceph-mon[103124]: pgmap v8631: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 203 KiB/s wr, 29 op/s 2023-09-10T03:44:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:12 smithi170 ceph-mon[102832]: pgmap v8632: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 106 KiB/s wr, 29 op/s 2023-09-10T03:44:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:12 smithi082 ceph-mon[98694]: pgmap v8632: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 106 KiB/s wr, 29 op/s 2023-09-10T03:44:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:12 smithi098 ceph-mon[103124]: pgmap v8632: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 106 KiB/s wr, 29 op/s 2023-09-10T03:44:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15884.035117 secs 2023-09-10T03:44:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:13 smithi170 ceph-mon[102832]: osdmap e608: 12 total, 12 up, 12 in 2023-09-10T03:44:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15884.035117 secs 2023-09-10T03:44:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:13 smithi082 ceph-mon[98694]: osdmap e608: 12 total, 12 up, 12 in 2023-09-10T03:44:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15884.035117 secs 2023-09-10T03:44:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:13 smithi098 ceph-mon[103124]: osdmap e608: 12 total, 12 up, 12 in 2023-09-10T03:44:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:14 smithi170 ceph-mon[102832]: pgmap v8634: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 132 KiB/s wr, 36 op/s 2023-09-10T03:44:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15885.244052 secs 2023-09-10T03:44:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:14 smithi082 ceph-mon[98694]: pgmap v8634: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 132 KiB/s wr, 36 op/s 2023-09-10T03:44:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15885.244052 secs 2023-09-10T03:44:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:14 smithi098 ceph-mon[103124]: pgmap v8634: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 132 KiB/s wr, 36 op/s 2023-09-10T03:44:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15885.244052 secs 2023-09-10T03:44:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15885.955231 secs 2023-09-10T03:44:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15885.955231 secs 2023-09-10T03:44:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15885.955231 secs 2023-09-10T03:44:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:16 smithi170 ceph-mon[102832]: pgmap v8635: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 63 KiB/s wr, 7 op/s 2023-09-10T03:44:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:16 smithi082 ceph-mon[98694]: pgmap v8635: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 63 KiB/s wr, 7 op/s 2023-09-10T03:44:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:16 smithi098 ceph-mon[103124]: pgmap v8635: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 63 KiB/s wr, 7 op/s 2023-09-10T03:44:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:18 smithi170 ceph-mon[102832]: pgmap v8636: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 14 op/s 2023-09-10T03:44:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15889.035212 secs 2023-09-10T03:44:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:44:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:44:18.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:44:18.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:44:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:18 smithi082 ceph-mon[98694]: pgmap v8636: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 14 op/s 2023-09-10T03:44:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15889.035212 secs 2023-09-10T03:44:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:44:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:44:18.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:44:18.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:44:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:18 smithi098 ceph-mon[103124]: pgmap v8636: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 14 op/s 2023-09-10T03:44:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15889.035212 secs 2023-09-10T03:44:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:44:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:44:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:44:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:44:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15890.244114 secs 2023-09-10T03:44:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15890.244114 secs 2023-09-10T03:44:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15890.244114 secs 2023-09-10T03:44:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15890.955296 secs 2023-09-10T03:44:20.495 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:20 smithi170 ceph-mon[102832]: pgmap v8637: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 114 KiB/s wr, 13 op/s 2023-09-10T03:44:20.495 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15890.955296 secs 2023-09-10T03:44:20.496 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:20 smithi082 ceph-mon[98694]: pgmap v8637: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 114 KiB/s wr, 13 op/s 2023-09-10T03:44:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15890.955296 secs 2023-09-10T03:44:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:20 smithi098 ceph-mon[103124]: pgmap v8637: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 114 KiB/s wr, 13 op/s 2023-09-10T03:44:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:22 smithi170 ceph-mon[102832]: pgmap v8638: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T03:44:22.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:22 smithi082 ceph-mon[98694]: pgmap v8638: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T03:44:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:22 smithi098 ceph-mon[103124]: pgmap v8638: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T03:44:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:44:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:44:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:44:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:24 smithi170 ceph-mon[102832]: pgmap v8639: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T03:44:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15895.244176 secs 2023-09-10T03:44:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:44:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:44:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:44:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:24 smithi082 ceph-mon[98694]: pgmap v8639: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T03:44:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15895.244176 secs 2023-09-10T03:44:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:44:24.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:44:24.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:44:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:24 smithi098 ceph-mon[103124]: pgmap v8639: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T03:44:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15895.244176 secs 2023-09-10T03:44:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:44:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:44:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:44:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15895.955357 secs 2023-09-10T03:44:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15895.955357 secs 2023-09-10T03:44:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15895.955357 secs 2023-09-10T03:44:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:26 smithi170 ceph-mon[102832]: pgmap v8640: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-10T03:44:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:26 smithi082 ceph-mon[98694]: pgmap v8640: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-10T03:44:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:26 smithi098 ceph-mon[103124]: pgmap v8640: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 10 op/s 2023-09-10T03:44:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:28 smithi170 ceph-mon[102832]: pgmap v8641: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 159 KiB/s wr, 15 op/s 2023-09-10T03:44:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:28 smithi082 ceph-mon[98694]: pgmap v8641: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 159 KiB/s wr, 15 op/s 2023-09-10T03:44:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:28 smithi098 ceph-mon[103124]: pgmap v8641: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 159 KiB/s wr, 15 op/s 2023-09-10T03:44:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15900.244257 secs 2023-09-10T03:44:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15900.244257 secs 2023-09-10T03:44:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15900.244257 secs 2023-09-10T03:44:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15900.955420 secs 2023-09-10T03:44:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:30 smithi170 ceph-mon[102832]: pgmap v8642: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 106 KiB/s wr, 10 op/s 2023-09-10T03:44:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15900.955420 secs 2023-09-10T03:44:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:30 smithi082 ceph-mon[98694]: pgmap v8642: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 106 KiB/s wr, 10 op/s 2023-09-10T03:44:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15900.955420 secs 2023-09-10T03:44:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:30 smithi098 ceph-mon[103124]: pgmap v8642: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 106 KiB/s wr, 10 op/s 2023-09-10T03:44:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:32 smithi170 ceph-mon[102832]: pgmap v8643: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 251 KiB/s wr, 16 op/s 2023-09-10T03:44:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:32 smithi082 ceph-mon[98694]: pgmap v8643: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 251 KiB/s wr, 16 op/s 2023-09-10T03:44:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:32 smithi098 ceph-mon[103124]: pgmap v8643: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 251 KiB/s wr, 16 op/s 2023-09-10T03:44:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:33 smithi098 ceph-mon[103124]: pgmap v8644: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 251 KiB/s wr, 12 op/s 2023-09-10T03:44:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15905.244316 secs 2023-09-10T03:44:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:33 smithi170 ceph-mon[102832]: pgmap v8644: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 251 KiB/s wr, 12 op/s 2023-09-10T03:44:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15905.244316 secs 2023-09-10T03:44:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:33 smithi082 ceph-mon[98694]: pgmap v8644: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 251 KiB/s wr, 12 op/s 2023-09-10T03:44:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15905.244316 secs 2023-09-10T03:44:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15905.955485 secs 2023-09-10T03:44:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15905.955485 secs 2023-09-10T03:44:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15905.955485 secs 2023-09-10T03:44:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:35 smithi098 ceph-mon[103124]: pgmap v8645: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 251 KiB/s wr, 12 op/s 2023-09-10T03:44:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:35 smithi170 ceph-mon[102832]: pgmap v8645: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 251 KiB/s wr, 12 op/s 2023-09-10T03:44:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:35 smithi082 ceph-mon[98694]: pgmap v8645: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 251 KiB/s wr, 12 op/s 2023-09-10T03:44:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:38 smithi170 ceph-mon[102832]: pgmap v8646: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 292 KiB/s wr, 17 op/s 2023-09-10T03:44:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:38 smithi082 ceph-mon[98694]: pgmap v8646: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 292 KiB/s wr, 17 op/s 2023-09-10T03:44:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:38 smithi098 ceph-mon[103124]: pgmap v8646: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 292 KiB/s wr, 17 op/s 2023-09-10T03:44:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15910.244375 secs 2023-09-10T03:44:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15910.244375 secs 2023-09-10T03:44:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15910.244375 secs 2023-09-10T03:44:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15910.955548 secs 2023-09-10T03:44:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:40 smithi170 ceph-mon[102832]: pgmap v8647: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T03:44:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15910.955548 secs 2023-09-10T03:44:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:40 smithi082 ceph-mon[98694]: pgmap v8647: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T03:44:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15910.955548 secs 2023-09-10T03:44:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:40 smithi098 ceph-mon[103124]: pgmap v8647: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T03:44:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:42 smithi170 ceph-mon[102832]: pgmap v8648: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 450 KiB/s wr, 18 op/s 2023-09-10T03:44:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:42 smithi082 ceph-mon[98694]: pgmap v8648: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 450 KiB/s wr, 18 op/s 2023-09-10T03:44:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:42 smithi098 ceph-mon[103124]: pgmap v8648: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 450 KiB/s wr, 18 op/s 2023-09-10T03:44:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:43 smithi098 ceph-mon[103124]: pgmap v8649: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 304 KiB/s wr, 11 op/s 2023-09-10T03:44:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15915.244461 secs 2023-09-10T03:44:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:43 smithi170 ceph-mon[102832]: pgmap v8649: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 304 KiB/s wr, 11 op/s 2023-09-10T03:44:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15915.244461 secs 2023-09-10T03:44:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:43 smithi082 ceph-mon[98694]: pgmap v8649: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 304 KiB/s wr, 11 op/s 2023-09-10T03:44:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15915.244461 secs 2023-09-10T03:44:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15915.955608 secs 2023-09-10T03:44:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15915.955608 secs 2023-09-10T03:44:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15915.955608 secs 2023-09-10T03:44:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:45 smithi098 ceph-mon[103124]: pgmap v8650: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 304 KiB/s wr, 11 op/s 2023-09-10T03:44:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:45 smithi170 ceph-mon[102832]: pgmap v8650: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 304 KiB/s wr, 11 op/s 2023-09-10T03:44:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:45 smithi082 ceph-mon[98694]: pgmap v8650: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 304 KiB/s wr, 11 op/s 2023-09-10T03:44:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:48 smithi170 ceph-mon[102832]: pgmap v8651: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 343 KiB/s wr, 16 op/s 2023-09-10T03:44:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:48 smithi082 ceph-mon[98694]: pgmap v8651: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 343 KiB/s wr, 16 op/s 2023-09-10T03:44:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:48 smithi098 ceph-mon[103124]: pgmap v8651: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 343 KiB/s wr, 16 op/s 2023-09-10T03:44:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15920.244549 secs 2023-09-10T03:44:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15920.244549 secs 2023-09-10T03:44:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15920.244549 secs 2023-09-10T03:44:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15920.955671 secs 2023-09-10T03:44:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:50 smithi170 ceph-mon[102832]: pgmap v8652: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 302 KiB/s wr, 11 op/s 2023-09-10T03:44:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15920.955671 secs 2023-09-10T03:44:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:50 smithi082 ceph-mon[98694]: pgmap v8652: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 302 KiB/s wr, 11 op/s 2023-09-10T03:44:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15920.955671 secs 2023-09-10T03:44:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:50 smithi098 ceph-mon[103124]: pgmap v8652: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 302 KiB/s wr, 11 op/s 2023-09-10T03:44:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:52 smithi170 ceph-mon[102832]: pgmap v8653: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 398 KiB/s wr, 17 op/s 2023-09-10T03:44:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:52 smithi082 ceph-mon[98694]: pgmap v8653: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 398 KiB/s wr, 17 op/s 2023-09-10T03:44:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:52 smithi098 ceph-mon[103124]: pgmap v8653: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 398 KiB/s wr, 17 op/s 2023-09-10T03:44:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15924.035723 secs 2023-09-10T03:44:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15924.035723 secs 2023-09-10T03:44:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15924.035723 secs 2023-09-10T03:44:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:54 smithi170 ceph-mon[102832]: pgmap v8654: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T03:44:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15925.244631 secs 2023-09-10T03:44:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:54 smithi082 ceph-mon[98694]: pgmap v8654: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T03:44:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15925.244631 secs 2023-09-10T03:44:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:54 smithi098 ceph-mon[103124]: pgmap v8654: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T03:44:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15925.244631 secs 2023-09-10T03:44:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15925.955732 secs 2023-09-10T03:44:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15925.955732 secs 2023-09-10T03:44:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15925.955732 secs 2023-09-10T03:44:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:56 smithi170 ceph-mon[102832]: pgmap v8655: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T03:44:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:44:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:44:56.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:44:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:56 smithi082 ceph-mon[98694]: pgmap v8655: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T03:44:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:44:56.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:44:56.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:44:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:56 smithi098 ceph-mon[103124]: pgmap v8655: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T03:44:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:44:56.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:44:56.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:44:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:58 smithi170 ceph-mon[102832]: pgmap v8656: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 253 KiB/s wr, 16 op/s 2023-09-10T03:44:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15929.035853 secs 2023-09-10T03:44:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:58 smithi082 ceph-mon[98694]: pgmap v8656: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 253 KiB/s wr, 16 op/s 2023-09-10T03:44:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15929.035853 secs 2023-09-10T03:44:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:58 smithi098 ceph-mon[103124]: pgmap v8656: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 253 KiB/s wr, 16 op/s 2023-09-10T03:44:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15929.035853 secs 2023-09-10T03:44:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:44:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15930.244719 secs 2023-09-10T03:44:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:44:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15930.244719 secs 2023-09-10T03:44:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:44:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15930.244719 secs 2023-09-10T03:45:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15930.955795 secs 2023-09-10T03:45:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:00 smithi170 ceph-mon[102832]: pgmap v8657: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 214 KiB/s wr, 11 op/s 2023-09-10T03:45:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15930.955795 secs 2023-09-10T03:45:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:00 smithi082 ceph-mon[98694]: pgmap v8657: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 214 KiB/s wr, 11 op/s 2023-09-10T03:45:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15930.955795 secs 2023-09-10T03:45:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:00 smithi098 ceph-mon[103124]: pgmap v8657: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 214 KiB/s wr, 11 op/s 2023-09-10T03:45:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:02 smithi170 ceph-mon[102832]: pgmap v8658: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 295 KiB/s wr, 30 op/s 2023-09-10T03:45:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:02 smithi082 ceph-mon[98694]: pgmap v8658: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 295 KiB/s wr, 30 op/s 2023-09-10T03:45:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:02 smithi098 ceph-mon[103124]: pgmap v8658: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 295 KiB/s wr, 30 op/s 2023-09-10T03:45:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15934.035912 secs 2023-09-10T03:45:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15934.035912 secs 2023-09-10T03:45:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15934.035912 secs 2023-09-10T03:45:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15935.244782 secs 2023-09-10T03:45:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:04 smithi170 ceph-mon[102832]: pgmap v8659: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 199 KiB/s wr, 24 op/s 2023-09-10T03:45:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15935.244782 secs 2023-09-10T03:45:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:04 smithi082 ceph-mon[98694]: pgmap v8659: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 199 KiB/s wr, 24 op/s 2023-09-10T03:45:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15935.244782 secs 2023-09-10T03:45:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:04 smithi098 ceph-mon[103124]: pgmap v8659: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 199 KiB/s wr, 24 op/s 2023-09-10T03:45:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15935.955860 secs 2023-09-10T03:45:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15935.955860 secs 2023-09-10T03:45:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15935.955860 secs 2023-09-10T03:45:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:06 smithi170 ceph-mon[102832]: pgmap v8660: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 199 KiB/s wr, 24 op/s 2023-09-10T03:45:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:06 smithi082 ceph-mon[98694]: pgmap v8660: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 199 KiB/s wr, 24 op/s 2023-09-10T03:45:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:06 smithi098 ceph-mon[103124]: pgmap v8660: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 199 KiB/s wr, 24 op/s 2023-09-10T03:45:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:08 smithi170 ceph-mon[102832]: pgmap v8661: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 223 KiB/s wr, 30 op/s 2023-09-10T03:45:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15939.035971 secs 2023-09-10T03:45:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:08 smithi082 ceph-mon[98694]: pgmap v8661: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 223 KiB/s wr, 30 op/s 2023-09-10T03:45:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15939.035971 secs 2023-09-10T03:45:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:08 smithi098 ceph-mon[103124]: pgmap v8661: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 223 KiB/s wr, 30 op/s 2023-09-10T03:45:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15939.035971 secs 2023-09-10T03:45:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15940.244847 secs 2023-09-10T03:45:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:09 smithi170 ceph-mon[102832]: osdmap e609: 12 total, 12 up, 12 in 2023-09-10T03:45:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15940.244847 secs 2023-09-10T03:45:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:09 smithi082 ceph-mon[98694]: osdmap e609: 12 total, 12 up, 12 in 2023-09-10T03:45:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15940.244847 secs 2023-09-10T03:45:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:09 smithi098 ceph-mon[103124]: osdmap e609: 12 total, 12 up, 12 in 2023-09-10T03:45:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15940.955920 secs 2023-09-10T03:45:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:10 smithi170 ceph-mon[102832]: pgmap v8663: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 126 KiB/s wr, 29 op/s 2023-09-10T03:45:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15940.955920 secs 2023-09-10T03:45:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:10 smithi082 ceph-mon[98694]: pgmap v8663: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 126 KiB/s wr, 29 op/s 2023-09-10T03:45:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15940.955920 secs 2023-09-10T03:45:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:10 smithi098 ceph-mon[103124]: pgmap v8663: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 126 KiB/s wr, 29 op/s 2023-09-10T03:45:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:12 smithi170 ceph-mon[102832]: pgmap v8664: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 93 KiB/s wr, 13 op/s 2023-09-10T03:45:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:12 smithi082 ceph-mon[98694]: pgmap v8664: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 93 KiB/s wr, 13 op/s 2023-09-10T03:45:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:12 smithi098 ceph-mon[103124]: pgmap v8664: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 93 KiB/s wr, 13 op/s 2023-09-10T03:45:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15944.036039 secs 2023-09-10T03:45:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:13 smithi170 ceph-mon[102832]: osdmap e610: 12 total, 12 up, 12 in 2023-09-10T03:45:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15944.036039 secs 2023-09-10T03:45:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:13 smithi082 ceph-mon[98694]: osdmap e610: 12 total, 12 up, 12 in 2023-09-10T03:45:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15944.036039 secs 2023-09-10T03:45:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:13 smithi098 ceph-mon[103124]: osdmap e610: 12 total, 12 up, 12 in 2023-09-10T03:45:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15945.244906 secs 2023-09-10T03:45:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:14 smithi170 ceph-mon[102832]: pgmap v8666: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 116 KiB/s wr, 16 op/s 2023-09-10T03:45:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15945.244906 secs 2023-09-10T03:45:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:14 smithi082 ceph-mon[98694]: pgmap v8666: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 116 KiB/s wr, 16 op/s 2023-09-10T03:45:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15945.244906 secs 2023-09-10T03:45:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:14 smithi098 ceph-mon[103124]: pgmap v8666: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 116 KiB/s wr, 16 op/s 2023-09-10T03:45:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15945.955984 secs 2023-09-10T03:45:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15945.955984 secs 2023-09-10T03:45:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15945.955984 secs 2023-09-10T03:45:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:16 smithi170 ceph-mon[102832]: pgmap v8667: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 80 KiB/s wr, 7 op/s 2023-09-10T03:45:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:16 smithi082 ceph-mon[98694]: pgmap v8667: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 80 KiB/s wr, 7 op/s 2023-09-10T03:45:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:16 smithi098 ceph-mon[103124]: pgmap v8667: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 80 KiB/s wr, 7 op/s 2023-09-10T03:45:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:18 smithi170 ceph-mon[102832]: pgmap v8668: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T03:45:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15949.036141 secs 2023-09-10T03:45:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:18 smithi082 ceph-mon[98694]: pgmap v8668: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T03:45:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15949.036141 secs 2023-09-10T03:45:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:18 smithi098 ceph-mon[103124]: pgmap v8668: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T03:45:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15949.036141 secs 2023-09-10T03:45:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15950.245010 secs 2023-09-10T03:45:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:45:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:45:19.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:45:19.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:45:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15950.245010 secs 2023-09-10T03:45:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:45:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:45:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:45:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:45:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15950.245010 secs 2023-09-10T03:45:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:45:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:45:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:45:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:45:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15950.956047 secs 2023-09-10T03:45:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:20 smithi170 ceph-mon[102832]: pgmap v8669: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T03:45:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15950.956047 secs 2023-09-10T03:45:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:20 smithi082 ceph-mon[98694]: pgmap v8669: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T03:45:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15950.956047 secs 2023-09-10T03:45:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:20 smithi098 ceph-mon[103124]: pgmap v8669: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T03:45:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:22 smithi170 ceph-mon[102832]: pgmap v8670: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T03:45:22.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:22 smithi082 ceph-mon[98694]: pgmap v8670: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T03:45:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:22 smithi098 ceph-mon[103124]: pgmap v8670: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T03:45:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:45:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:45:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:45:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15955.245069 secs 2023-09-10T03:45:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:24 smithi170 ceph-mon[102832]: pgmap v8671: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T03:45:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:45:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:45:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:45:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15955.245069 secs 2023-09-10T03:45:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:24 smithi082 ceph-mon[98694]: pgmap v8671: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T03:45:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:45:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:45:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:45:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15955.245069 secs 2023-09-10T03:45:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:24 smithi098 ceph-mon[103124]: pgmap v8671: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T03:45:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:45:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:45:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:45:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15955.956109 secs 2023-09-10T03:45:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15955.956109 secs 2023-09-10T03:45:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15955.956109 secs 2023-09-10T03:45:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:26 smithi170 ceph-mon[102832]: pgmap v8672: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 9 op/s 2023-09-10T03:45:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:26 smithi082 ceph-mon[98694]: pgmap v8672: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 9 op/s 2023-09-10T03:45:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:26 smithi098 ceph-mon[103124]: pgmap v8672: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 9 op/s 2023-09-10T03:45:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:28 smithi170 ceph-mon[102832]: pgmap v8673: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 137 KiB/s wr, 15 op/s 2023-09-10T03:45:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:28 smithi082 ceph-mon[98694]: pgmap v8673: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 137 KiB/s wr, 15 op/s 2023-09-10T03:45:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:28 smithi098 ceph-mon[103124]: pgmap v8673: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 137 KiB/s wr, 15 op/s 2023-09-10T03:45:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15960.245142 secs 2023-09-10T03:45:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15960.245142 secs 2023-09-10T03:45:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15960.245142 secs 2023-09-10T03:45:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15960.956172 secs 2023-09-10T03:45:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:30 smithi170 ceph-mon[102832]: pgmap v8674: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 136 KiB/s wr, 10 op/s 2023-09-10T03:45:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15960.956172 secs 2023-09-10T03:45:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:30 smithi082 ceph-mon[98694]: pgmap v8674: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 136 KiB/s wr, 10 op/s 2023-09-10T03:45:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15960.956172 secs 2023-09-10T03:45:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:30 smithi098 ceph-mon[103124]: pgmap v8674: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 136 KiB/s wr, 10 op/s 2023-09-10T03:45:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:32 smithi170 ceph-mon[102832]: pgmap v8675: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 204 KiB/s wr, 15 op/s 2023-09-10T03:45:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:32 smithi082 ceph-mon[98694]: pgmap v8675: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 204 KiB/s wr, 15 op/s 2023-09-10T03:45:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:32 smithi098 ceph-mon[103124]: pgmap v8675: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 204 KiB/s wr, 15 op/s 2023-09-10T03:45:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15965.245203 secs 2023-09-10T03:45:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:33 smithi098 ceph-mon[103124]: pgmap v8676: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 137 KiB/s wr, 11 op/s 2023-09-10T03:45:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15965.245203 secs 2023-09-10T03:45:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:33 smithi170 ceph-mon[102832]: pgmap v8676: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 137 KiB/s wr, 11 op/s 2023-09-10T03:45:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15965.245203 secs 2023-09-10T03:45:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:33 smithi082 ceph-mon[98694]: pgmap v8676: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 137 KiB/s wr, 11 op/s 2023-09-10T03:45:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15965.956236 secs 2023-09-10T03:45:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15965.956236 secs 2023-09-10T03:45:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15965.956236 secs 2023-09-10T03:45:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:35 smithi098 ceph-mon[103124]: pgmap v8677: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 137 KiB/s wr, 11 op/s 2023-09-10T03:45:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:35 smithi170 ceph-mon[102832]: pgmap v8677: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 137 KiB/s wr, 11 op/s 2023-09-10T03:45:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:35 smithi082 ceph-mon[98694]: pgmap v8677: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 137 KiB/s wr, 11 op/s 2023-09-10T03:45:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:38 smithi170 ceph-mon[102832]: pgmap v8678: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 340 KiB/s wr, 17 op/s 2023-09-10T03:45:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:38 smithi082 ceph-mon[98694]: pgmap v8678: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 340 KiB/s wr, 17 op/s 2023-09-10T03:45:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:38 smithi098 ceph-mon[103124]: pgmap v8678: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 340 KiB/s wr, 17 op/s 2023-09-10T03:45:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15970.245263 secs 2023-09-10T03:45:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15970.245263 secs 2023-09-10T03:45:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15970.245263 secs 2023-09-10T03:45:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15970.956299 secs 2023-09-10T03:45:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:40 smithi170 ceph-mon[102832]: pgmap v8679: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 271 KiB/s wr, 11 op/s 2023-09-10T03:45:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15970.956299 secs 2023-09-10T03:45:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:40 smithi082 ceph-mon[98694]: pgmap v8679: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 271 KiB/s wr, 11 op/s 2023-09-10T03:45:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15970.956299 secs 2023-09-10T03:45:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:40 smithi098 ceph-mon[103124]: pgmap v8679: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 271 KiB/s wr, 11 op/s 2023-09-10T03:45:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:42 smithi170 ceph-mon[102832]: pgmap v8680: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 448 KiB/s wr, 18 op/s 2023-09-10T03:45:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:42 smithi082 ceph-mon[98694]: pgmap v8680: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 448 KiB/s wr, 18 op/s 2023-09-10T03:45:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:42 smithi098 ceph-mon[103124]: pgmap v8680: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 448 KiB/s wr, 18 op/s 2023-09-10T03:45:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15975.245321 secs 2023-09-10T03:45:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:43 smithi098 ceph-mon[103124]: pgmap v8681: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 380 KiB/s wr, 12 op/s 2023-09-10T03:45:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15975.245321 secs 2023-09-10T03:45:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:43 smithi170 ceph-mon[102832]: pgmap v8681: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 380 KiB/s wr, 12 op/s 2023-09-10T03:45:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15975.245321 secs 2023-09-10T03:45:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:43 smithi082 ceph-mon[98694]: pgmap v8681: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 380 KiB/s wr, 12 op/s 2023-09-10T03:45:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15975.956366 secs 2023-09-10T03:45:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15975.956366 secs 2023-09-10T03:45:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15975.956366 secs 2023-09-10T03:45:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:45 smithi098 ceph-mon[103124]: pgmap v8682: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 380 KiB/s wr, 12 op/s 2023-09-10T03:45:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:45 smithi170 ceph-mon[102832]: pgmap v8682: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 380 KiB/s wr, 12 op/s 2023-09-10T03:45:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:45 smithi082 ceph-mon[98694]: pgmap v8682: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 380 KiB/s wr, 12 op/s 2023-09-10T03:45:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:48 smithi170 ceph-mon[102832]: pgmap v8683: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 437 KiB/s wr, 19 op/s 2023-09-10T03:45:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15979.036622 secs 2023-09-10T03:45:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:48 smithi082 ceph-mon[98694]: pgmap v8683: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 437 KiB/s wr, 19 op/s 2023-09-10T03:45:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15979.036622 secs 2023-09-10T03:45:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:48 smithi098 ceph-mon[103124]: pgmap v8683: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 437 KiB/s wr, 19 op/s 2023-09-10T03:45:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15979.036622 secs 2023-09-10T03:45:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15980.245381 secs 2023-09-10T03:45:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15980.245381 secs 2023-09-10T03:45:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15980.245381 secs 2023-09-10T03:45:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15980.956430 secs 2023-09-10T03:45:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:50 smithi170 ceph-mon[102832]: pgmap v8684: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 233 KiB/s wr, 12 op/s 2023-09-10T03:45:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15980.956430 secs 2023-09-10T03:45:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:50 smithi082 ceph-mon[98694]: pgmap v8684: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 233 KiB/s wr, 12 op/s 2023-09-10T03:45:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15980.956430 secs 2023-09-10T03:45:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:50 smithi098 ceph-mon[103124]: pgmap v8684: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 233 KiB/s wr, 12 op/s 2023-09-10T03:45:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:45:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:45:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:45:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:45:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:45:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:45:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:45:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:45:51.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:45:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:52 smithi170 ceph-mon[102832]: pgmap v8685: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 411 KiB/s wr, 18 op/s 2023-09-10T03:45:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:52 smithi082 ceph-mon[98694]: pgmap v8685: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 411 KiB/s wr, 18 op/s 2023-09-10T03:45:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:52 smithi098 ceph-mon[103124]: pgmap v8685: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 411 KiB/s wr, 18 op/s 2023-09-10T03:45:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15984.036712 secs 2023-09-10T03:45:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15984.036712 secs 2023-09-10T03:45:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15984.036712 secs 2023-09-10T03:45:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15985.245440 secs 2023-09-10T03:45:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:54 smithi170 ceph-mon[102832]: pgmap v8686: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 235 KiB/s wr, 11 op/s 2023-09-10T03:45:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15985.245440 secs 2023-09-10T03:45:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:54 smithi082 ceph-mon[98694]: pgmap v8686: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 235 KiB/s wr, 11 op/s 2023-09-10T03:45:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15985.245440 secs 2023-09-10T03:45:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:54 smithi098 ceph-mon[103124]: pgmap v8686: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 235 KiB/s wr, 11 op/s 2023-09-10T03:45:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15985.956491 secs 2023-09-10T03:45:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15985.956491 secs 2023-09-10T03:45:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15985.956491 secs 2023-09-10T03:45:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:56 smithi170 ceph-mon[102832]: pgmap v8687: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 235 KiB/s wr, 11 op/s 2023-09-10T03:45:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:56 smithi082 ceph-mon[98694]: pgmap v8687: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 235 KiB/s wr, 11 op/s 2023-09-10T03:45:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:56 smithi098 ceph-mon[103124]: pgmap v8687: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 235 KiB/s wr, 11 op/s 2023-09-10T03:45:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:58 smithi170 ceph-mon[102832]: pgmap v8688: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 353 KiB/s wr, 17 op/s 2023-09-10T03:45:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15989.036778 secs 2023-09-10T03:45:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:58 smithi082 ceph-mon[98694]: pgmap v8688: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 353 KiB/s wr, 17 op/s 2023-09-10T03:45:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15989.036778 secs 2023-09-10T03:45:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:58 smithi098 ceph-mon[103124]: pgmap v8688: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 353 KiB/s wr, 17 op/s 2023-09-10T03:45:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15989.036778 secs 2023-09-10T03:45:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:45:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15990.245493 secs 2023-09-10T03:45:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:45:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15990.245493 secs 2023-09-10T03:45:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:45:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15990.245493 secs 2023-09-10T03:46:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15990.956558 secs 2023-09-10T03:46:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:00 smithi170 ceph-mon[102832]: pgmap v8689: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T03:46:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15990.956558 secs 2023-09-10T03:46:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:00 smithi082 ceph-mon[98694]: pgmap v8689: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T03:46:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15990.956558 secs 2023-09-10T03:46:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:00 smithi098 ceph-mon[103124]: pgmap v8689: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T03:46:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:02 smithi170 ceph-mon[102832]: pgmap v8690: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 354 KiB/s wr, 26 op/s 2023-09-10T03:46:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:02 smithi082 ceph-mon[98694]: pgmap v8690: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 354 KiB/s wr, 26 op/s 2023-09-10T03:46:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:02 smithi098 ceph-mon[103124]: pgmap v8690: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 354 KiB/s wr, 26 op/s 2023-09-10T03:46:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15994.036912 secs 2023-09-10T03:46:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15994.036912 secs 2023-09-10T03:46:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15994.036912 secs 2023-09-10T03:46:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15995.245552 secs 2023-09-10T03:46:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:04 smithi170 ceph-mon[102832]: pgmap v8691: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 176 KiB/s wr, 21 op/s 2023-09-10T03:46:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15995.245552 secs 2023-09-10T03:46:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:04 smithi082 ceph-mon[98694]: pgmap v8691: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 176 KiB/s wr, 21 op/s 2023-09-10T03:46:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15995.245552 secs 2023-09-10T03:46:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:04 smithi098 ceph-mon[103124]: pgmap v8691: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 176 KiB/s wr, 21 op/s 2023-09-10T03:46:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 15995.956624 secs 2023-09-10T03:46:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 15995.956624 secs 2023-09-10T03:46:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 15995.956624 secs 2023-09-10T03:46:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:06 smithi170 ceph-mon[102832]: pgmap v8692: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 176 KiB/s wr, 21 op/s 2023-09-10T03:46:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:06 smithi082 ceph-mon[98694]: pgmap v8692: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 176 KiB/s wr, 21 op/s 2023-09-10T03:46:06.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:06 smithi098 ceph-mon[103124]: pgmap v8692: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 176 KiB/s wr, 21 op/s 2023-09-10T03:46:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:08 smithi170 ceph-mon[102832]: pgmap v8693: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 223 KiB/s wr, 30 op/s 2023-09-10T03:46:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 15999.036974 secs 2023-09-10T03:46:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:08 smithi082 ceph-mon[98694]: pgmap v8693: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 223 KiB/s wr, 30 op/s 2023-09-10T03:46:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 15999.036974 secs 2023-09-10T03:46:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:08 smithi098 ceph-mon[103124]: pgmap v8693: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 223 KiB/s wr, 30 op/s 2023-09-10T03:46:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 15999.036974 secs 2023-09-10T03:46:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16000.245610 secs 2023-09-10T03:46:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:09 smithi170 ceph-mon[102832]: osdmap e611: 12 total, 12 up, 12 in 2023-09-10T03:46:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16000.245610 secs 2023-09-10T03:46:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:09 smithi082 ceph-mon[98694]: osdmap e611: 12 total, 12 up, 12 in 2023-09-10T03:46:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16000.245610 secs 2023-09-10T03:46:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:09 smithi098 ceph-mon[103124]: osdmap e611: 12 total, 12 up, 12 in 2023-09-10T03:46:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16000.956689 secs 2023-09-10T03:46:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:10 smithi170 ceph-mon[102832]: pgmap v8695: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 126 KiB/s wr, 30 op/s 2023-09-10T03:46:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16000.956689 secs 2023-09-10T03:46:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:10 smithi082 ceph-mon[98694]: pgmap v8695: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 126 KiB/s wr, 30 op/s 2023-09-10T03:46:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16000.956689 secs 2023-09-10T03:46:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:10 smithi098 ceph-mon[103124]: pgmap v8695: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 126 KiB/s wr, 30 op/s 2023-09-10T03:46:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:12 smithi170 ceph-mon[102832]: pgmap v8696: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 57 KiB/s wr, 16 op/s 2023-09-10T03:46:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:12 smithi082 ceph-mon[98694]: pgmap v8696: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 57 KiB/s wr, 16 op/s 2023-09-10T03:46:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:12 smithi098 ceph-mon[103124]: pgmap v8696: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 57 KiB/s wr, 16 op/s 2023-09-10T03:46:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16004.037036 secs 2023-09-10T03:46:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:13 smithi170 ceph-mon[102832]: osdmap e612: 12 total, 12 up, 12 in 2023-09-10T03:46:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16004.037036 secs 2023-09-10T03:46:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:13 smithi082 ceph-mon[98694]: osdmap e612: 12 total, 12 up, 12 in 2023-09-10T03:46:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16004.037036 secs 2023-09-10T03:46:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:13 smithi098 ceph-mon[103124]: osdmap e612: 12 total, 12 up, 12 in 2023-09-10T03:46:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16005.245664 secs 2023-09-10T03:46:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:14 smithi170 ceph-mon[102832]: pgmap v8698: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 71 KiB/s wr, 21 op/s 2023-09-10T03:46:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16005.245664 secs 2023-09-10T03:46:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:14 smithi082 ceph-mon[98694]: pgmap v8698: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 71 KiB/s wr, 21 op/s 2023-09-10T03:46:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16005.245664 secs 2023-09-10T03:46:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:14 smithi098 ceph-mon[103124]: pgmap v8698: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 71 KiB/s wr, 21 op/s 2023-09-10T03:46:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16005.956758 secs 2023-09-10T03:46:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16005.956758 secs 2023-09-10T03:46:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16005.956758 secs 2023-09-10T03:46:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:16 smithi170 ceph-mon[102832]: pgmap v8699: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:46:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:16 smithi082 ceph-mon[98694]: pgmap v8699: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:46:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:16 smithi098 ceph-mon[103124]: pgmap v8699: 97 pgs: 97 active+clean; 9.4 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:46:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:18 smithi170 ceph-mon[102832]: pgmap v8700: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 13 op/s 2023-09-10T03:46:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16009.037144 secs 2023-09-10T03:46:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:18 smithi082 ceph-mon[98694]: pgmap v8700: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 13 op/s 2023-09-10T03:46:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16009.037144 secs 2023-09-10T03:46:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:18 smithi098 ceph-mon[103124]: pgmap v8700: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 13 op/s 2023-09-10T03:46:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16009.037144 secs 2023-09-10T03:46:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16010.245722 secs 2023-09-10T03:46:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:46:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16010.245722 secs 2023-09-10T03:46:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:46:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16010.245722 secs 2023-09-10T03:46:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:46:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16010.956822 secs 2023-09-10T03:46:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:20 smithi082 ceph-mon[98694]: pgmap v8701: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T03:46:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:46:20.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:46:20.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:46:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16010.956822 secs 2023-09-10T03:46:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:20 smithi098 ceph-mon[103124]: pgmap v8701: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T03:46:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:46:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:46:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:46:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16010.956822 secs 2023-09-10T03:46:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:20 smithi170 ceph-mon[102832]: pgmap v8701: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T03:46:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:46:20.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:46:20.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:46:22.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:22 smithi082 ceph-mon[98694]: pgmap v8702: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T03:46:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:22 smithi098 ceph-mon[103124]: pgmap v8702: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T03:46:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:22 smithi170 ceph-mon[102832]: pgmap v8702: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T03:46:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:46:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:46:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:46:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16015.245803 secs 2023-09-10T03:46:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:24 smithi082 ceph-mon[98694]: pgmap v8703: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T03:46:24.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:46:24.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:46:24.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:46:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16015.245803 secs 2023-09-10T03:46:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:24 smithi098 ceph-mon[103124]: pgmap v8703: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T03:46:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:46:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:46:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:46:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16015.245803 secs 2023-09-10T03:46:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:24 smithi170 ceph-mon[102832]: pgmap v8703: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T03:46:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:46:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:46:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:46:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16015.956888 secs 2023-09-10T03:46:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16015.956888 secs 2023-09-10T03:46:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16015.956888 secs 2023-09-10T03:46:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:26 smithi082 ceph-mon[98694]: pgmap v8704: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 9 op/s 2023-09-10T03:46:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:26 smithi098 ceph-mon[103124]: pgmap v8704: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 9 op/s 2023-09-10T03:46:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:26 smithi170 ceph-mon[102832]: pgmap v8704: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 9 op/s 2023-09-10T03:46:27.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:27 smithi082 ceph-mon[98694]: pgmap v8705: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 177 KiB/s wr, 14 op/s 2023-09-10T03:46:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:27 smithi098 ceph-mon[103124]: pgmap v8705: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 177 KiB/s wr, 14 op/s 2023-09-10T03:46:27.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:27 smithi170 ceph-mon[102832]: pgmap v8705: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 177 KiB/s wr, 14 op/s 2023-09-10T03:46:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16020.245864 secs 2023-09-10T03:46:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16020.245864 secs 2023-09-10T03:46:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16020.245864 secs 2023-09-10T03:46:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16020.956955 secs 2023-09-10T03:46:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:29 smithi098 ceph-mon[103124]: pgmap v8706: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 9 op/s 2023-09-10T03:46:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16020.956955 secs 2023-09-10T03:46:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:29 smithi170 ceph-mon[102832]: pgmap v8706: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 9 op/s 2023-09-10T03:46:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16020.956955 secs 2023-09-10T03:46:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:29 smithi082 ceph-mon[98694]: pgmap v8706: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 9 op/s 2023-09-10T03:46:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:32 smithi170 ceph-mon[102832]: pgmap v8707: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 193 KiB/s wr, 15 op/s 2023-09-10T03:46:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:32 smithi082 ceph-mon[98694]: pgmap v8707: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 193 KiB/s wr, 15 op/s 2023-09-10T03:46:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:32 smithi098 ceph-mon[103124]: pgmap v8707: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 193 KiB/s wr, 15 op/s 2023-09-10T03:46:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16025.245922 secs 2023-09-10T03:46:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:33 smithi098 ceph-mon[103124]: pgmap v8708: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T03:46:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16025.245922 secs 2023-09-10T03:46:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:33 smithi170 ceph-mon[102832]: pgmap v8708: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T03:46:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16025.245922 secs 2023-09-10T03:46:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:33 smithi082 ceph-mon[98694]: pgmap v8708: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T03:46:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16025.957021 secs 2023-09-10T03:46:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16025.957021 secs 2023-09-10T03:46:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16025.957021 secs 2023-09-10T03:46:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:36 smithi170 ceph-mon[102832]: pgmap v8709: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T03:46:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:36 smithi082 ceph-mon[98694]: pgmap v8709: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T03:46:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:36 smithi098 ceph-mon[103124]: pgmap v8709: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T03:46:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:38 smithi170 ceph-mon[102832]: pgmap v8710: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 233 KiB/s wr, 17 op/s 2023-09-10T03:46:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:38 smithi082 ceph-mon[98694]: pgmap v8710: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 233 KiB/s wr, 17 op/s 2023-09-10T03:46:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:38 smithi098 ceph-mon[103124]: pgmap v8710: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 233 KiB/s wr, 17 op/s 2023-09-10T03:46:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16030.246039 secs 2023-09-10T03:46:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16030.246039 secs 2023-09-10T03:46:39.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16030.246039 secs 2023-09-10T03:46:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16030.957090 secs 2023-09-10T03:46:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:40 smithi170 ceph-mon[102832]: pgmap v8711: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T03:46:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16030.957090 secs 2023-09-10T03:46:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:40 smithi082 ceph-mon[98694]: pgmap v8711: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T03:46:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16030.957090 secs 2023-09-10T03:46:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:40 smithi098 ceph-mon[103124]: pgmap v8711: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T03:46:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:42 smithi170 ceph-mon[102832]: pgmap v8712: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 325 KiB/s wr, 19 op/s 2023-09-10T03:46:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:42 smithi082 ceph-mon[98694]: pgmap v8712: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 325 KiB/s wr, 19 op/s 2023-09-10T03:46:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:42 smithi098 ceph-mon[103124]: pgmap v8712: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 325 KiB/s wr, 19 op/s 2023-09-10T03:46:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16035.246129 secs 2023-09-10T03:46:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:44 smithi098 ceph-mon[103124]: pgmap v8713: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 256 KiB/s wr, 13 op/s 2023-09-10T03:46:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16035.246129 secs 2023-09-10T03:46:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:44 smithi170 ceph-mon[102832]: pgmap v8713: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 256 KiB/s wr, 13 op/s 2023-09-10T03:46:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16035.246129 secs 2023-09-10T03:46:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:44 smithi082 ceph-mon[98694]: pgmap v8713: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 256 KiB/s wr, 13 op/s 2023-09-10T03:46:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16035.957154 secs 2023-09-10T03:46:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16035.957154 secs 2023-09-10T03:46:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16035.957154 secs 2023-09-10T03:46:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:46 smithi170 ceph-mon[102832]: pgmap v8714: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 256 KiB/s wr, 13 op/s 2023-09-10T03:46:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:46 smithi082 ceph-mon[98694]: pgmap v8714: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 256 KiB/s wr, 13 op/s 2023-09-10T03:46:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:46 smithi098 ceph-mon[103124]: pgmap v8714: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 256 KiB/s wr, 13 op/s 2023-09-10T03:46:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:48 smithi170 ceph-mon[102832]: pgmap v8715: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 368 KiB/s wr, 19 op/s 2023-09-10T03:46:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:48 smithi082 ceph-mon[98694]: pgmap v8715: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 368 KiB/s wr, 19 op/s 2023-09-10T03:46:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:48 smithi098 ceph-mon[103124]: pgmap v8715: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 368 KiB/s wr, 19 op/s 2023-09-10T03:46:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16040.246191 secs 2023-09-10T03:46:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16040.246191 secs 2023-09-10T03:46:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16040.246191 secs 2023-09-10T03:46:50.350 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16040.957221 secs 2023-09-10T03:46:50.351 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:50 smithi170 ceph-mon[102832]: pgmap v8716: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 328 KiB/s wr, 12 op/s 2023-09-10T03:46:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16040.957221 secs 2023-09-10T03:46:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:50 smithi082 ceph-mon[98694]: pgmap v8716: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 328 KiB/s wr, 12 op/s 2023-09-10T03:46:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16040.957221 secs 2023-09-10T03:46:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:50 smithi098 ceph-mon[103124]: pgmap v8716: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 328 KiB/s wr, 12 op/s 2023-09-10T03:46:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:52 smithi170 ceph-mon[102832]: pgmap v8717: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 497 KiB/s wr, 19 op/s 2023-09-10T03:46:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:52 smithi082 ceph-mon[98694]: pgmap v8717: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 497 KiB/s wr, 19 op/s 2023-09-10T03:46:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:52 smithi098 ceph-mon[103124]: pgmap v8717: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 497 KiB/s wr, 19 op/s 2023-09-10T03:46:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16044.037767 secs 2023-09-10T03:46:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16044.037767 secs 2023-09-10T03:46:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16044.037767 secs 2023-09-10T03:46:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16045.246280 secs 2023-09-10T03:46:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:54 smithi170 ceph-mon[102832]: pgmap v8718: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T03:46:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16045.246280 secs 2023-09-10T03:46:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:54 smithi082 ceph-mon[98694]: pgmap v8718: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T03:46:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16045.246280 secs 2023-09-10T03:46:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:54 smithi098 ceph-mon[103124]: pgmap v8718: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T03:46:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16045.957288 secs 2023-09-10T03:46:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16045.957288 secs 2023-09-10T03:46:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16045.957288 secs 2023-09-10T03:46:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:56 smithi170 ceph-mon[102832]: pgmap v8719: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T03:46:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:46:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:46:56.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:46:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:56 smithi082 ceph-mon[98694]: pgmap v8719: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T03:46:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:46:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:46:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:46:56.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:56 smithi098 ceph-mon[103124]: pgmap v8719: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T03:46:56.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:46:56.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:46:56.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:46:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:58 smithi170 ceph-mon[102832]: pgmap v8720: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 332 KiB/s wr, 17 op/s 2023-09-10T03:46:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16049.037825 secs 2023-09-10T03:46:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:58 smithi082 ceph-mon[98694]: pgmap v8720: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 332 KiB/s wr, 17 op/s 2023-09-10T03:46:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16049.037825 secs 2023-09-10T03:46:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:58 smithi098 ceph-mon[103124]: pgmap v8720: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 332 KiB/s wr, 17 op/s 2023-09-10T03:46:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16049.037825 secs 2023-09-10T03:46:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:46:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16050.246340 secs 2023-09-10T03:46:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:46:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16050.246340 secs 2023-09-10T03:46:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:46:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16050.246340 secs 2023-09-10T03:47:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16050.957355 secs 2023-09-10T03:47:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:00 smithi170 ceph-mon[102832]: pgmap v8721: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 220 KiB/s wr, 12 op/s 2023-09-10T03:47:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16050.957355 secs 2023-09-10T03:47:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:00 smithi082 ceph-mon[98694]: pgmap v8721: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 220 KiB/s wr, 12 op/s 2023-09-10T03:47:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16050.957355 secs 2023-09-10T03:47:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:00 smithi098 ceph-mon[103124]: pgmap v8721: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 220 KiB/s wr, 12 op/s 2023-09-10T03:47:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:02 smithi170 ceph-mon[102832]: pgmap v8722: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 344 KiB/s wr, 18 op/s 2023-09-10T03:47:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:02 smithi082 ceph-mon[98694]: pgmap v8722: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 344 KiB/s wr, 18 op/s 2023-09-10T03:47:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:02 smithi098 ceph-mon[103124]: pgmap v8722: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 344 KiB/s wr, 18 op/s 2023-09-10T03:47:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16054.037883 secs 2023-09-10T03:47:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16054.037883 secs 2023-09-10T03:47:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16054.037883 secs 2023-09-10T03:47:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16055.246399 secs 2023-09-10T03:47:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:04 smithi170 ceph-mon[102832]: pgmap v8723: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 175 KiB/s wr, 11 op/s 2023-09-10T03:47:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16055.246399 secs 2023-09-10T03:47:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:04 smithi082 ceph-mon[98694]: pgmap v8723: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 175 KiB/s wr, 11 op/s 2023-09-10T03:47:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16055.246399 secs 2023-09-10T03:47:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:04 smithi098 ceph-mon[103124]: pgmap v8723: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 175 KiB/s wr, 11 op/s 2023-09-10T03:47:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16055.957420 secs 2023-09-10T03:47:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16055.957420 secs 2023-09-10T03:47:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16055.957420 secs 2023-09-10T03:47:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:06 smithi170 ceph-mon[102832]: pgmap v8724: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 175 KiB/s wr, 11 op/s 2023-09-10T03:47:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:06 smithi082 ceph-mon[98694]: pgmap v8724: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 175 KiB/s wr, 11 op/s 2023-09-10T03:47:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:06 smithi098 ceph-mon[103124]: pgmap v8724: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 175 KiB/s wr, 11 op/s 2023-09-10T03:47:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:08 smithi170 ceph-mon[102832]: pgmap v8725: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 233 KiB/s wr, 30 op/s 2023-09-10T03:47:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16059.037943 secs 2023-09-10T03:47:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:08 smithi082 ceph-mon[98694]: pgmap v8725: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 233 KiB/s wr, 30 op/s 2023-09-10T03:47:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16059.037943 secs 2023-09-10T03:47:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:08 smithi098 ceph-mon[103124]: pgmap v8725: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 233 KiB/s wr, 30 op/s 2023-09-10T03:47:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16059.037943 secs 2023-09-10T03:47:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16060.246457 secs 2023-09-10T03:47:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:09 smithi170 ceph-mon[102832]: osdmap e613: 12 total, 12 up, 12 in 2023-09-10T03:47:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16060.246457 secs 2023-09-10T03:47:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:09 smithi082 ceph-mon[98694]: osdmap e613: 12 total, 12 up, 12 in 2023-09-10T03:47:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16060.246457 secs 2023-09-10T03:47:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:09 smithi098 ceph-mon[103124]: osdmap e613: 12 total, 12 up, 12 in 2023-09-10T03:47:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16060.957490 secs 2023-09-10T03:47:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:10 smithi170 ceph-mon[102832]: pgmap v8727: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 218 KiB/s wr, 30 op/s 2023-09-10T03:47:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16060.957490 secs 2023-09-10T03:47:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:10 smithi082 ceph-mon[98694]: pgmap v8727: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 218 KiB/s wr, 30 op/s 2023-09-10T03:47:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16060.957490 secs 2023-09-10T03:47:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:10 smithi098 ceph-mon[103124]: pgmap v8727: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 218 KiB/s wr, 30 op/s 2023-09-10T03:47:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:12 smithi170 ceph-mon[102832]: pgmap v8728: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 121 KiB/s wr, 29 op/s 2023-09-10T03:47:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:12 smithi082 ceph-mon[98694]: pgmap v8728: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 121 KiB/s wr, 29 op/s 2023-09-10T03:47:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:12 smithi098 ceph-mon[103124]: pgmap v8728: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 121 KiB/s wr, 29 op/s 2023-09-10T03:47:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16064.038005 secs 2023-09-10T03:47:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:13 smithi170 ceph-mon[102832]: osdmap e614: 12 total, 12 up, 12 in 2023-09-10T03:47:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16064.038005 secs 2023-09-10T03:47:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:13 smithi082 ceph-mon[98694]: osdmap e614: 12 total, 12 up, 12 in 2023-09-10T03:47:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16064.038005 secs 2023-09-10T03:47:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:13 smithi098 ceph-mon[103124]: osdmap e614: 12 total, 12 up, 12 in 2023-09-10T03:47:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16065.246521 secs 2023-09-10T03:47:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:14 smithi170 ceph-mon[102832]: pgmap v8730: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 151 KiB/s wr, 36 op/s 2023-09-10T03:47:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16065.246521 secs 2023-09-10T03:47:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:14 smithi082 ceph-mon[98694]: pgmap v8730: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 151 KiB/s wr, 36 op/s 2023-09-10T03:47:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16065.246521 secs 2023-09-10T03:47:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:14 smithi098 ceph-mon[103124]: pgmap v8730: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 151 KiB/s wr, 36 op/s 2023-09-10T03:47:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16065.957556 secs 2023-09-10T03:47:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16065.957556 secs 2023-09-10T03:47:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16065.957556 secs 2023-09-10T03:47:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:16 smithi170 ceph-mon[102832]: pgmap v8731: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 7 op/s 2023-09-10T03:47:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:16 smithi082 ceph-mon[98694]: pgmap v8731: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 7 op/s 2023-09-10T03:47:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:16 smithi098 ceph-mon[103124]: pgmap v8731: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 7 op/s 2023-09-10T03:47:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:18 smithi170 ceph-mon[102832]: pgmap v8732: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 130 KiB/s wr, 14 op/s 2023-09-10T03:47:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16069.038095 secs 2023-09-10T03:47:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:18 smithi082 ceph-mon[98694]: pgmap v8732: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 130 KiB/s wr, 14 op/s 2023-09-10T03:47:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16069.038095 secs 2023-09-10T03:47:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:18 smithi098 ceph-mon[103124]: pgmap v8732: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 130 KiB/s wr, 14 op/s 2023-09-10T03:47:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16069.038095 secs 2023-09-10T03:47:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16070.246580 secs 2023-09-10T03:47:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16070.246580 secs 2023-09-10T03:47:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16070.246580 secs 2023-09-10T03:47:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16070.957622 secs 2023-09-10T03:47:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:20 smithi170 ceph-mon[102832]: pgmap v8733: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 13 op/s 2023-09-10T03:47:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:47:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16070.957622 secs 2023-09-10T03:47:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:20 smithi082 ceph-mon[98694]: pgmap v8733: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 13 op/s 2023-09-10T03:47:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:47:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16070.957622 secs 2023-09-10T03:47:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:20 smithi098 ceph-mon[103124]: pgmap v8733: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 13 op/s 2023-09-10T03:47:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:47:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:22 smithi170 ceph-mon[102832]: pgmap v8734: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 12 op/s 2023-09-10T03:47:22.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:22 smithi082 ceph-mon[98694]: pgmap v8734: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 12 op/s 2023-09-10T03:47:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:22 smithi098 ceph-mon[103124]: pgmap v8734: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 12 op/s 2023-09-10T03:47:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:47:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:47:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:47:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:47:23.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:47:23.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:47:23.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:47:23.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:47:23.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:47:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16075.246639 secs 2023-09-10T03:47:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:24 smithi098 ceph-mon[103124]: pgmap v8735: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 12 op/s 2023-09-10T03:47:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:47:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:47:24.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:47:24.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:47:24.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:47:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16075.246639 secs 2023-09-10T03:47:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:24 smithi170 ceph-mon[102832]: pgmap v8735: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 12 op/s 2023-09-10T03:47:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:47:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:47:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:47:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:47:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:47:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16075.246639 secs 2023-09-10T03:47:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:24 smithi082 ceph-mon[98694]: pgmap v8735: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 12 op/s 2023-09-10T03:47:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:47:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:47:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:47:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:47:24.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:47:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16075.957688 secs 2023-09-10T03:47:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16075.957688 secs 2023-09-10T03:47:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16075.957688 secs 2023-09-10T03:47:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:26 smithi170 ceph-mon[102832]: pgmap v8736: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 10 op/s 2023-09-10T03:47:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:26 smithi082 ceph-mon[98694]: pgmap v8736: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 10 op/s 2023-09-10T03:47:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:26 smithi098 ceph-mon[103124]: pgmap v8736: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 10 op/s 2023-09-10T03:47:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:28 smithi170 ceph-mon[102832]: pgmap v8737: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 161 KiB/s wr, 15 op/s 2023-09-10T03:47:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:47:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:47:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:47:28.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:47:28.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:47:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:28 smithi082 ceph-mon[98694]: pgmap v8737: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 161 KiB/s wr, 15 op/s 2023-09-10T03:47:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:47:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:47:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:47:28.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:47:28.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:47:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:28 smithi098 ceph-mon[103124]: pgmap v8737: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 161 KiB/s wr, 15 op/s 2023-09-10T03:47:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:47:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:47:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:47:28.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:47:28.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:47:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16080.246697 secs 2023-09-10T03:47:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16080.246697 secs 2023-09-10T03:47:29.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16080.246697 secs 2023-09-10T03:47:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16080.957753 secs 2023-09-10T03:47:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:30 smithi170 ceph-mon[102832]: pgmap v8738: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 107 KiB/s wr, 9 op/s 2023-09-10T03:47:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16080.957753 secs 2023-09-10T03:47:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:30 smithi082 ceph-mon[98694]: pgmap v8738: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 107 KiB/s wr, 9 op/s 2023-09-10T03:47:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16080.957753 secs 2023-09-10T03:47:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:30 smithi098 ceph-mon[103124]: pgmap v8738: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 107 KiB/s wr, 9 op/s 2023-09-10T03:47:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:32 smithi170 ceph-mon[102832]: pgmap v8739: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 255 KiB/s wr, 16 op/s 2023-09-10T03:47:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:32 smithi082 ceph-mon[98694]: pgmap v8739: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 255 KiB/s wr, 16 op/s 2023-09-10T03:47:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:32 smithi098 ceph-mon[103124]: pgmap v8739: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 255 KiB/s wr, 16 op/s 2023-09-10T03:47:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16085.246785 secs 2023-09-10T03:47:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:34 smithi170 ceph-mon[102832]: pgmap v8740: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 255 KiB/s wr, 11 op/s 2023-09-10T03:47:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16085.246785 secs 2023-09-10T03:47:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:34 smithi082 ceph-mon[98694]: pgmap v8740: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 255 KiB/s wr, 11 op/s 2023-09-10T03:47:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16085.246785 secs 2023-09-10T03:47:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:34 smithi098 ceph-mon[103124]: pgmap v8740: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 255 KiB/s wr, 11 op/s 2023-09-10T03:47:35.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16085.957821 secs 2023-09-10T03:47:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16085.957821 secs 2023-09-10T03:47:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16085.957821 secs 2023-09-10T03:47:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:36 smithi170 ceph-mon[102832]: pgmap v8741: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 255 KiB/s wr, 11 op/s 2023-09-10T03:47:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:36 smithi082 ceph-mon[98694]: pgmap v8741: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 255 KiB/s wr, 11 op/s 2023-09-10T03:47:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:36 smithi098 ceph-mon[103124]: pgmap v8741: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 255 KiB/s wr, 11 op/s 2023-09-10T03:47:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:38 smithi170 ceph-mon[102832]: pgmap v8742: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 305 KiB/s wr, 17 op/s 2023-09-10T03:47:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:38 smithi082 ceph-mon[98694]: pgmap v8742: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 305 KiB/s wr, 17 op/s 2023-09-10T03:47:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:38 smithi098 ceph-mon[103124]: pgmap v8742: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 305 KiB/s wr, 17 op/s 2023-09-10T03:47:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16090.246843 secs 2023-09-10T03:47:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16090.246843 secs 2023-09-10T03:47:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16090.246843 secs 2023-09-10T03:47:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16090.957887 secs 2023-09-10T03:47:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:40 smithi170 ceph-mon[102832]: pgmap v8743: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 198 KiB/s wr, 12 op/s 2023-09-10T03:47:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16090.957887 secs 2023-09-10T03:47:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:40 smithi082 ceph-mon[98694]: pgmap v8743: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 198 KiB/s wr, 12 op/s 2023-09-10T03:47:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16090.957887 secs 2023-09-10T03:47:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:40 smithi098 ceph-mon[103124]: pgmap v8743: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 198 KiB/s wr, 12 op/s 2023-09-10T03:47:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:42 smithi170 ceph-mon[102832]: pgmap v8744: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 458 KiB/s wr, 18 op/s 2023-09-10T03:47:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:42 smithi082 ceph-mon[98694]: pgmap v8744: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 458 KiB/s wr, 18 op/s 2023-09-10T03:47:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:42 smithi098 ceph-mon[103124]: pgmap v8744: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 458 KiB/s wr, 18 op/s 2023-09-10T03:47:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16095.246901 secs 2023-09-10T03:47:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:44 smithi170 ceph-mon[102832]: pgmap v8745: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 310 KiB/s wr, 12 op/s 2023-09-10T03:47:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16095.246901 secs 2023-09-10T03:47:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:44 smithi082 ceph-mon[98694]: pgmap v8745: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 310 KiB/s wr, 12 op/s 2023-09-10T03:47:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16095.246901 secs 2023-09-10T03:47:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:44 smithi098 ceph-mon[103124]: pgmap v8745: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 310 KiB/s wr, 12 op/s 2023-09-10T03:47:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16095.957954 secs 2023-09-10T03:47:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16095.957954 secs 2023-09-10T03:47:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16095.957954 secs 2023-09-10T03:47:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:46 smithi170 ceph-mon[102832]: pgmap v8746: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 310 KiB/s wr, 12 op/s 2023-09-10T03:47:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:46 smithi082 ceph-mon[98694]: pgmap v8746: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 310 KiB/s wr, 12 op/s 2023-09-10T03:47:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:46 smithi098 ceph-mon[103124]: pgmap v8746: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 310 KiB/s wr, 12 op/s 2023-09-10T03:47:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:48 smithi170 ceph-mon[102832]: pgmap v8747: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 392 KiB/s wr, 17 op/s 2023-09-10T03:47:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:48 smithi082 ceph-mon[98694]: pgmap v8747: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 392 KiB/s wr, 17 op/s 2023-09-10T03:47:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:48 smithi098 ceph-mon[103124]: pgmap v8747: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 392 KiB/s wr, 17 op/s 2023-09-10T03:47:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16100.246959 secs 2023-09-10T03:47:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16100.246959 secs 2023-09-10T03:47:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16100.246959 secs 2023-09-10T03:47:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16100.958024 secs 2023-09-10T03:47:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:50 smithi170 ceph-mon[102832]: pgmap v8748: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 342 KiB/s wr, 11 op/s 2023-09-10T03:47:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16100.958024 secs 2023-09-10T03:47:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:50 smithi082 ceph-mon[98694]: pgmap v8748: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 342 KiB/s wr, 11 op/s 2023-09-10T03:47:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16100.958024 secs 2023-09-10T03:47:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:50 smithi098 ceph-mon[103124]: pgmap v8748: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 342 KiB/s wr, 11 op/s 2023-09-10T03:47:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:52 smithi170 ceph-mon[102832]: pgmap v8749: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 397 KiB/s wr, 17 op/s 2023-09-10T03:47:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:52 smithi082 ceph-mon[98694]: pgmap v8749: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 397 KiB/s wr, 17 op/s 2023-09-10T03:47:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:52 smithi098 ceph-mon[103124]: pgmap v8749: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 397 KiB/s wr, 17 op/s 2023-09-10T03:47:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16104.038696 secs 2023-09-10T03:47:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16104.038696 secs 2023-09-10T03:47:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16104.038696 secs 2023-09-10T03:47:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16105.247020 secs 2023-09-10T03:47:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:54 smithi170 ceph-mon[102832]: pgmap v8750: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T03:47:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16105.247020 secs 2023-09-10T03:47:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:54 smithi082 ceph-mon[98694]: pgmap v8750: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T03:47:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16105.247020 secs 2023-09-10T03:47:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:54 smithi098 ceph-mon[103124]: pgmap v8750: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T03:47:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16105.958088 secs 2023-09-10T03:47:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16105.958088 secs 2023-09-10T03:47:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16105.958088 secs 2023-09-10T03:47:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:56 smithi170 ceph-mon[102832]: pgmap v8751: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T03:47:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:47:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:47:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:47:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:56 smithi082 ceph-mon[98694]: pgmap v8751: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T03:47:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:47:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:47:56.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:47:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:56 smithi098 ceph-mon[103124]: pgmap v8751: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T03:47:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:47:56.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:47:56.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:47:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:58 smithi170 ceph-mon[102832]: pgmap v8752: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 250 KiB/s wr, 16 op/s 2023-09-10T03:47:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16109.038803 secs 2023-09-10T03:47:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:58 smithi082 ceph-mon[98694]: pgmap v8752: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 250 KiB/s wr, 16 op/s 2023-09-10T03:47:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16109.038803 secs 2023-09-10T03:47:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:58 smithi098 ceph-mon[103124]: pgmap v8752: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 250 KiB/s wr, 16 op/s 2023-09-10T03:47:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16109.038803 secs 2023-09-10T03:47:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:47:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16110.247078 secs 2023-09-10T03:47:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:47:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16110.247078 secs 2023-09-10T03:47:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:47:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16110.247078 secs 2023-09-10T03:48:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16110.958157 secs 2023-09-10T03:48:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:00 smithi170 ceph-mon[102832]: pgmap v8753: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 168 KiB/s wr, 11 op/s 2023-09-10T03:48:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16110.958157 secs 2023-09-10T03:48:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:00 smithi082 ceph-mon[98694]: pgmap v8753: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 168 KiB/s wr, 11 op/s 2023-09-10T03:48:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16110.958157 secs 2023-09-10T03:48:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:00 smithi098 ceph-mon[103124]: pgmap v8753: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 168 KiB/s wr, 11 op/s 2023-09-10T03:48:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:02 smithi170 ceph-mon[102832]: pgmap v8754: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 217 KiB/s wr, 16 op/s 2023-09-10T03:48:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:02 smithi082 ceph-mon[98694]: pgmap v8754: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 217 KiB/s wr, 16 op/s 2023-09-10T03:48:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:02 smithi098 ceph-mon[103124]: pgmap v8754: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 217 KiB/s wr, 16 op/s 2023-09-10T03:48:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16114.038909 secs 2023-09-10T03:48:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16114.038909 secs 2023-09-10T03:48:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16114.038909 secs 2023-09-10T03:48:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16115.247163 secs 2023-09-10T03:48:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:04 smithi170 ceph-mon[102832]: pgmap v8755: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T03:48:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16115.247163 secs 2023-09-10T03:48:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:04 smithi082 ceph-mon[98694]: pgmap v8755: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T03:48:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16115.247163 secs 2023-09-10T03:48:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:04 smithi098 ceph-mon[103124]: pgmap v8755: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T03:48:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16115.958223 secs 2023-09-10T03:48:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16115.958223 secs 2023-09-10T03:48:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16115.958223 secs 2023-09-10T03:48:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:06 smithi170 ceph-mon[102832]: pgmap v8756: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T03:48:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:06 smithi082 ceph-mon[98694]: pgmap v8756: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T03:48:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:06 smithi098 ceph-mon[103124]: pgmap v8756: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T03:48:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:08 smithi170 ceph-mon[102832]: pgmap v8757: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 219 KiB/s wr, 30 op/s 2023-09-10T03:48:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16119.039008 secs 2023-09-10T03:48:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:08 smithi082 ceph-mon[98694]: pgmap v8757: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 219 KiB/s wr, 30 op/s 2023-09-10T03:48:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16119.039008 secs 2023-09-10T03:48:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:08 smithi098 ceph-mon[103124]: pgmap v8757: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 219 KiB/s wr, 30 op/s 2023-09-10T03:48:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16119.039008 secs 2023-09-10T03:48:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16120.247223 secs 2023-09-10T03:48:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:09 smithi170 ceph-mon[102832]: osdmap e615: 12 total, 12 up, 12 in 2023-09-10T03:48:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16120.247223 secs 2023-09-10T03:48:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:09 smithi082 ceph-mon[98694]: osdmap e615: 12 total, 12 up, 12 in 2023-09-10T03:48:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16120.247223 secs 2023-09-10T03:48:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:09 smithi098 ceph-mon[103124]: osdmap e615: 12 total, 12 up, 12 in 2023-09-10T03:48:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16120.958288 secs 2023-09-10T03:48:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:10 smithi170 ceph-mon[102832]: pgmap v8759: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T03:48:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16120.958288 secs 2023-09-10T03:48:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:10 smithi082 ceph-mon[98694]: pgmap v8759: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T03:48:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16120.958288 secs 2023-09-10T03:48:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:10 smithi098 ceph-mon[103124]: pgmap v8759: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 127 KiB/s wr, 29 op/s 2023-09-10T03:48:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:12 smithi170 ceph-mon[102832]: pgmap v8760: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 29 op/s 2023-09-10T03:48:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:12 smithi082 ceph-mon[98694]: pgmap v8760: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 29 op/s 2023-09-10T03:48:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:12 smithi098 ceph-mon[103124]: pgmap v8760: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 29 op/s 2023-09-10T03:48:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16124.039067 secs 2023-09-10T03:48:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:13 smithi170 ceph-mon[102832]: osdmap e616: 12 total, 12 up, 12 in 2023-09-10T03:48:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16124.039067 secs 2023-09-10T03:48:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:13 smithi082 ceph-mon[98694]: osdmap e616: 12 total, 12 up, 12 in 2023-09-10T03:48:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16124.039067 secs 2023-09-10T03:48:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:13 smithi098 ceph-mon[103124]: osdmap e616: 12 total, 12 up, 12 in 2023-09-10T03:48:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16125.247310 secs 2023-09-10T03:48:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:14 smithi170 ceph-mon[102832]: pgmap v8762: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 167 KiB/s wr, 37 op/s 2023-09-10T03:48:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16125.247310 secs 2023-09-10T03:48:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:14 smithi082 ceph-mon[98694]: pgmap v8762: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 167 KiB/s wr, 37 op/s 2023-09-10T03:48:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16125.247310 secs 2023-09-10T03:48:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:14 smithi098 ceph-mon[103124]: pgmap v8762: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 167 KiB/s wr, 37 op/s 2023-09-10T03:48:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16125.958354 secs 2023-09-10T03:48:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16125.958354 secs 2023-09-10T03:48:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16125.958354 secs 2023-09-10T03:48:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:16 smithi170 ceph-mon[102832]: pgmap v8763: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 82 KiB/s wr, 7 op/s 2023-09-10T03:48:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:16 smithi082 ceph-mon[98694]: pgmap v8763: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 82 KiB/s wr, 7 op/s 2023-09-10T03:48:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:16 smithi098 ceph-mon[103124]: pgmap v8763: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 82 KiB/s wr, 7 op/s 2023-09-10T03:48:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:18 smithi170 ceph-mon[102832]: pgmap v8764: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 13 op/s 2023-09-10T03:48:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16129.039127 secs 2023-09-10T03:48:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:18 smithi082 ceph-mon[98694]: pgmap v8764: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 13 op/s 2023-09-10T03:48:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16129.039127 secs 2023-09-10T03:48:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:18 smithi098 ceph-mon[103124]: pgmap v8764: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 13 op/s 2023-09-10T03:48:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16129.039127 secs 2023-09-10T03:48:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16130.247368 secs 2023-09-10T03:48:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16130.247368 secs 2023-09-10T03:48:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16130.247368 secs 2023-09-10T03:48:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16130.958418 secs 2023-09-10T03:48:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:20 smithi170 ceph-mon[102832]: pgmap v8765: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T03:48:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16130.958418 secs 2023-09-10T03:48:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:20 smithi082 ceph-mon[98694]: pgmap v8765: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T03:48:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16130.958418 secs 2023-09-10T03:48:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:20 smithi098 ceph-mon[103124]: pgmap v8765: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T03:48:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:22 smithi170 ceph-mon[102832]: pgmap v8766: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T03:48:22.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:22 smithi082 ceph-mon[98694]: pgmap v8766: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T03:48:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:22 smithi098 ceph-mon[103124]: pgmap v8766: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T03:48:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16134.039219 secs 2023-09-10T03:48:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16134.039219 secs 2023-09-10T03:48:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16134.039219 secs 2023-09-10T03:48:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16135.247456 secs 2023-09-10T03:48:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:24 smithi082 ceph-mon[98694]: pgmap v8767: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T03:48:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16135.247456 secs 2023-09-10T03:48:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:24 smithi098 ceph-mon[103124]: pgmap v8767: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T03:48:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16135.247456 secs 2023-09-10T03:48:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:24 smithi170 ceph-mon[102832]: pgmap v8767: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T03:48:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16135.958486 secs 2023-09-10T03:48:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16135.958486 secs 2023-09-10T03:48:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16135.958486 secs 2023-09-10T03:48:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:26 smithi082 ceph-mon[98694]: pgmap v8768: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 9 op/s 2023-09-10T03:48:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:26 smithi098 ceph-mon[103124]: pgmap v8768: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 9 op/s 2023-09-10T03:48:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:26 smithi170 ceph-mon[102832]: pgmap v8768: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 9 op/s 2023-09-10T03:48:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:28 smithi082 ceph-mon[98694]: pgmap v8769: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 14 op/s 2023-09-10T03:48:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:48:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16139.039288 secs 2023-09-10T03:48:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:48:28.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:48:28.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:48:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:28 smithi098 ceph-mon[103124]: pgmap v8769: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 14 op/s 2023-09-10T03:48:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:48:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16139.039288 secs 2023-09-10T03:48:28.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:48:28.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:48:28.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:48:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:28 smithi170 ceph-mon[102832]: pgmap v8769: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 14 op/s 2023-09-10T03:48:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:48:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16139.039288 secs 2023-09-10T03:48:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:48:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:48:28.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:48:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16140.247541 secs 2023-09-10T03:48:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16140.247541 secs 2023-09-10T03:48:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16140.247541 secs 2023-09-10T03:48:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16140.958552 secs 2023-09-10T03:48:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:30 smithi082 ceph-mon[98694]: pgmap v8770: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 10 op/s 2023-09-10T03:48:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16140.958552 secs 2023-09-10T03:48:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:30 smithi098 ceph-mon[103124]: pgmap v8770: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 10 op/s 2023-09-10T03:48:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16140.958552 secs 2023-09-10T03:48:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:30 smithi170 ceph-mon[102832]: pgmap v8770: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 10 op/s 2023-09-10T03:48:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:32 smithi082 ceph-mon[98694]: pgmap v8771: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 207 KiB/s wr, 15 op/s 2023-09-10T03:48:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:32 smithi098 ceph-mon[103124]: pgmap v8771: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 207 KiB/s wr, 15 op/s 2023-09-10T03:48:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:32 smithi170 ceph-mon[102832]: pgmap v8771: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 207 KiB/s wr, 15 op/s 2023-09-10T03:48:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16145.247599 secs 2023-09-10T03:48:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:34 smithi170 ceph-mon[102832]: pgmap v8772: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T03:48:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16145.247599 secs 2023-09-10T03:48:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:34 smithi082 ceph-mon[98694]: pgmap v8772: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T03:48:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16145.247599 secs 2023-09-10T03:48:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:34 smithi098 ceph-mon[103124]: pgmap v8772: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T03:48:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16145.958623 secs 2023-09-10T03:48:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:48:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16145.958623 secs 2023-09-10T03:48:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:48:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16145.958623 secs 2023-09-10T03:48:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:48:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:48:36.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:48:36.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:48:36.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:36 smithi170 ceph-mon[102832]: pgmap v8773: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T03:48:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:48:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:48:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:48:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:36 smithi082 ceph-mon[98694]: pgmap v8773: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T03:48:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:48:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:48:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:48:36.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:36 smithi098 ceph-mon[103124]: pgmap v8773: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T03:48:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:38 smithi170 ceph-mon[102832]: pgmap v8774: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 345 KiB/s wr, 17 op/s 2023-09-10T03:48:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:38 smithi082 ceph-mon[98694]: pgmap v8774: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 345 KiB/s wr, 17 op/s 2023-09-10T03:48:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:38 smithi098 ceph-mon[103124]: pgmap v8774: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 345 KiB/s wr, 17 op/s 2023-09-10T03:48:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16150.247658 secs 2023-09-10T03:48:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16150.247658 secs 2023-09-10T03:48:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16150.247658 secs 2023-09-10T03:48:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16150.958689 secs 2023-09-10T03:48:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:40 smithi170 ceph-mon[102832]: pgmap v8775: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 275 KiB/s wr, 12 op/s 2023-09-10T03:48:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16150.958689 secs 2023-09-10T03:48:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:40 smithi082 ceph-mon[98694]: pgmap v8775: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 275 KiB/s wr, 12 op/s 2023-09-10T03:48:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16150.958689 secs 2023-09-10T03:48:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:40 smithi098 ceph-mon[103124]: pgmap v8775: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 275 KiB/s wr, 12 op/s 2023-09-10T03:48:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:42 smithi170 ceph-mon[102832]: pgmap v8776: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 455 KiB/s wr, 19 op/s 2023-09-10T03:48:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:42 smithi082 ceph-mon[98694]: pgmap v8776: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 455 KiB/s wr, 19 op/s 2023-09-10T03:48:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:42 smithi098 ceph-mon[103124]: pgmap v8776: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 455 KiB/s wr, 19 op/s 2023-09-10T03:48:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16155.247742 secs 2023-09-10T03:48:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:44 smithi170 ceph-mon[102832]: pgmap v8777: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 386 KiB/s wr, 13 op/s 2023-09-10T03:48:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16155.247742 secs 2023-09-10T03:48:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:44 smithi082 ceph-mon[98694]: pgmap v8777: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 386 KiB/s wr, 13 op/s 2023-09-10T03:48:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16155.247742 secs 2023-09-10T03:48:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:44 smithi098 ceph-mon[103124]: pgmap v8777: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 386 KiB/s wr, 13 op/s 2023-09-10T03:48:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16155.958756 secs 2023-09-10T03:48:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16155.958756 secs 2023-09-10T03:48:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16155.958756 secs 2023-09-10T03:48:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:46 smithi170 ceph-mon[102832]: pgmap v8778: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 386 KiB/s wr, 13 op/s 2023-09-10T03:48:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:46 smithi082 ceph-mon[98694]: pgmap v8778: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 386 KiB/s wr, 13 op/s 2023-09-10T03:48:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:46 smithi098 ceph-mon[103124]: pgmap v8778: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 386 KiB/s wr, 13 op/s 2023-09-10T03:48:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:48 smithi170 ceph-mon[102832]: pgmap v8779: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 443 KiB/s wr, 18 op/s 2023-09-10T03:48:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:48 smithi082 ceph-mon[98694]: pgmap v8779: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 443 KiB/s wr, 18 op/s 2023-09-10T03:48:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:48 smithi098 ceph-mon[103124]: pgmap v8779: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 443 KiB/s wr, 18 op/s 2023-09-10T03:48:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16160.247823 secs 2023-09-10T03:48:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16160.247823 secs 2023-09-10T03:48:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16160.247823 secs 2023-09-10T03:48:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16160.958819 secs 2023-09-10T03:48:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:50 smithi170 ceph-mon[102832]: pgmap v8780: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 236 KiB/s wr, 11 op/s 2023-09-10T03:48:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16160.958819 secs 2023-09-10T03:48:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:50 smithi082 ceph-mon[98694]: pgmap v8780: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 236 KiB/s wr, 11 op/s 2023-09-10T03:48:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16160.958819 secs 2023-09-10T03:48:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:50 smithi098 ceph-mon[103124]: pgmap v8780: 97 pgs: 97 active+clean; 9.5 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 236 KiB/s wr, 11 op/s 2023-09-10T03:48:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:52 smithi170 ceph-mon[102832]: pgmap v8781: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 323 KiB/s wr, 17 op/s 2023-09-10T03:48:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:52 smithi082 ceph-mon[98694]: pgmap v8781: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 323 KiB/s wr, 17 op/s 2023-09-10T03:48:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:52 smithi098 ceph-mon[103124]: pgmap v8781: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 323 KiB/s wr, 17 op/s 2023-09-10T03:48:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16164.039658 secs 2023-09-10T03:48:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16164.039658 secs 2023-09-10T03:48:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16164.039658 secs 2023-09-10T03:48:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16165.247884 secs 2023-09-10T03:48:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:54 smithi170 ceph-mon[102832]: pgmap v8782: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T03:48:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16165.247884 secs 2023-09-10T03:48:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:54 smithi082 ceph-mon[98694]: pgmap v8782: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T03:48:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16165.247884 secs 2023-09-10T03:48:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:54 smithi098 ceph-mon[103124]: pgmap v8782: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T03:48:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16165.958889 secs 2023-09-10T03:48:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16165.958889 secs 2023-09-10T03:48:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16165.958889 secs 2023-09-10T03:48:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:56 smithi170 ceph-mon[102832]: pgmap v8783: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T03:48:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:48:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:48:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:48:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:56 smithi082 ceph-mon[98694]: pgmap v8783: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T03:48:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:48:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:48:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:48:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:56 smithi098 ceph-mon[103124]: pgmap v8783: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T03:48:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:48:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:48:56.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:48:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:58 smithi170 ceph-mon[102832]: pgmap v8784: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 16 op/s 2023-09-10T03:48:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16169.039717 secs 2023-09-10T03:48:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:58 smithi082 ceph-mon[98694]: pgmap v8784: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 16 op/s 2023-09-10T03:48:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16169.039717 secs 2023-09-10T03:48:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:58 smithi098 ceph-mon[103124]: pgmap v8784: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 16 op/s 2023-09-10T03:48:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16169.039717 secs 2023-09-10T03:48:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:48:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16170.247942 secs 2023-09-10T03:48:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:48:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16170.247942 secs 2023-09-10T03:48:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:48:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16170.247942 secs 2023-09-10T03:49:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16170.958956 secs 2023-09-10T03:49:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:00 smithi170 ceph-mon[102832]: pgmap v8785: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 226 KiB/s wr, 11 op/s 2023-09-10T03:49:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16170.958956 secs 2023-09-10T03:49:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:00 smithi082 ceph-mon[98694]: pgmap v8785: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 226 KiB/s wr, 11 op/s 2023-09-10T03:49:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16170.958956 secs 2023-09-10T03:49:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:00 smithi098 ceph-mon[103124]: pgmap v8785: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 226 KiB/s wr, 11 op/s 2023-09-10T03:49:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:02 smithi170 ceph-mon[102832]: pgmap v8786: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 275 KiB/s wr, 17 op/s 2023-09-10T03:49:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:02 smithi082 ceph-mon[98694]: pgmap v8786: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 275 KiB/s wr, 17 op/s 2023-09-10T03:49:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:02 smithi098 ceph-mon[103124]: pgmap v8786: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 275 KiB/s wr, 17 op/s 2023-09-10T03:49:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16174.039784 secs 2023-09-10T03:49:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16174.039784 secs 2023-09-10T03:49:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16174.039784 secs 2023-09-10T03:49:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16175.247999 secs 2023-09-10T03:49:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:04 smithi170 ceph-mon[102832]: pgmap v8787: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 188 KiB/s wr, 11 op/s 2023-09-10T03:49:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16175.247999 secs 2023-09-10T03:49:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:04 smithi082 ceph-mon[98694]: pgmap v8787: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 188 KiB/s wr, 11 op/s 2023-09-10T03:49:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16175.247999 secs 2023-09-10T03:49:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:04 smithi098 ceph-mon[103124]: pgmap v8787: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 188 KiB/s wr, 11 op/s 2023-09-10T03:49:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16175.959023 secs 2023-09-10T03:49:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16175.959023 secs 2023-09-10T03:49:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16175.959023 secs 2023-09-10T03:49:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:06 smithi170 ceph-mon[102832]: pgmap v8788: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 188 KiB/s wr, 11 op/s 2023-09-10T03:49:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:06 smithi082 ceph-mon[98694]: pgmap v8788: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 188 KiB/s wr, 11 op/s 2023-09-10T03:49:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:06 smithi098 ceph-mon[103124]: pgmap v8788: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 188 KiB/s wr, 11 op/s 2023-09-10T03:49:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:08 smithi170 ceph-mon[102832]: pgmap v8789: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 246 KiB/s wr, 31 op/s 2023-09-10T03:49:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16179.039842 secs 2023-09-10T03:49:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:08 smithi082 ceph-mon[98694]: pgmap v8789: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 246 KiB/s wr, 31 op/s 2023-09-10T03:49:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16179.039842 secs 2023-09-10T03:49:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:08 smithi098 ceph-mon[103124]: pgmap v8789: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 246 KiB/s wr, 31 op/s 2023-09-10T03:49:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16179.039842 secs 2023-09-10T03:49:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16180.248057 secs 2023-09-10T03:49:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:09 smithi170 ceph-mon[102832]: osdmap e617: 12 total, 12 up, 12 in 2023-09-10T03:49:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16180.248057 secs 2023-09-10T03:49:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:09 smithi082 ceph-mon[98694]: osdmap e617: 12 total, 12 up, 12 in 2023-09-10T03:49:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16180.248057 secs 2023-09-10T03:49:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:09 smithi098 ceph-mon[103124]: osdmap e617: 12 total, 12 up, 12 in 2023-09-10T03:49:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16180.959088 secs 2023-09-10T03:49:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:10 smithi170 ceph-mon[102832]: pgmap v8791: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 127 KiB/s wr, 31 op/s 2023-09-10T03:49:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16180.959088 secs 2023-09-10T03:49:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:10 smithi082 ceph-mon[98694]: pgmap v8791: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 127 KiB/s wr, 31 op/s 2023-09-10T03:49:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16180.959088 secs 2023-09-10T03:49:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:10 smithi098 ceph-mon[103124]: pgmap v8791: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 127 KiB/s wr, 31 op/s 2023-09-10T03:49:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:12 smithi170 ceph-mon[102832]: pgmap v8792: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 105 KiB/s wr, 30 op/s 2023-09-10T03:49:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:12 smithi082 ceph-mon[98694]: pgmap v8792: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 105 KiB/s wr, 30 op/s 2023-09-10T03:49:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:12 smithi098 ceph-mon[103124]: pgmap v8792: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 105 KiB/s wr, 30 op/s 2023-09-10T03:49:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16184.039964 secs 2023-09-10T03:49:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:13 smithi170 ceph-mon[102832]: osdmap e618: 12 total, 12 up, 12 in 2023-09-10T03:49:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16184.039964 secs 2023-09-10T03:49:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:13 smithi082 ceph-mon[98694]: osdmap e618: 12 total, 12 up, 12 in 2023-09-10T03:49:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16184.039964 secs 2023-09-10T03:49:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:13 smithi098 ceph-mon[103124]: osdmap e618: 12 total, 12 up, 12 in 2023-09-10T03:49:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16185.248143 secs 2023-09-10T03:49:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:14 smithi170 ceph-mon[102832]: pgmap v8794: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 131 KiB/s wr, 38 op/s 2023-09-10T03:49:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16185.248143 secs 2023-09-10T03:49:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:14 smithi082 ceph-mon[98694]: pgmap v8794: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 131 KiB/s wr, 38 op/s 2023-09-10T03:49:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16185.248143 secs 2023-09-10T03:49:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:14 smithi098 ceph-mon[103124]: pgmap v8794: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 131 KiB/s wr, 38 op/s 2023-09-10T03:49:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16185.959152 secs 2023-09-10T03:49:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16185.959152 secs 2023-09-10T03:49:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16185.959152 secs 2023-09-10T03:49:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:16 smithi170 ceph-mon[102832]: pgmap v8795: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 7 op/s 2023-09-10T03:49:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:16 smithi082 ceph-mon[98694]: pgmap v8795: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 7 op/s 2023-09-10T03:49:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:16 smithi098 ceph-mon[103124]: pgmap v8795: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 7 op/s 2023-09-10T03:49:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:18 smithi170 ceph-mon[102832]: pgmap v8796: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 13 op/s 2023-09-10T03:49:18.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16189.040023 secs 2023-09-10T03:49:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:18 smithi082 ceph-mon[98694]: pgmap v8796: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 13 op/s 2023-09-10T03:49:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16189.040023 secs 2023-09-10T03:49:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:18 smithi098 ceph-mon[103124]: pgmap v8796: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 13 op/s 2023-09-10T03:49:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16189.040023 secs 2023-09-10T03:49:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16190.248200 secs 2023-09-10T03:49:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16190.248200 secs 2023-09-10T03:49:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16190.248200 secs 2023-09-10T03:49:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16190.959242 secs 2023-09-10T03:49:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:20 smithi082 ceph-mon[98694]: pgmap v8797: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T03:49:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16190.959242 secs 2023-09-10T03:49:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:20 smithi098 ceph-mon[103124]: pgmap v8797: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T03:49:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16190.959242 secs 2023-09-10T03:49:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:20 smithi170 ceph-mon[102832]: pgmap v8797: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T03:49:22.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:22 smithi082 ceph-mon[98694]: pgmap v8798: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T03:49:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:22 smithi098 ceph-mon[103124]: pgmap v8798: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T03:49:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:22 smithi170 ceph-mon[102832]: pgmap v8798: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T03:49:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:49:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:49:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:49:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16195.248257 secs 2023-09-10T03:49:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:24 smithi082 ceph-mon[98694]: pgmap v8799: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T03:49:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:49:24.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:49:24.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:49:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16195.248257 secs 2023-09-10T03:49:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:24 smithi098 ceph-mon[103124]: pgmap v8799: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T03:49:24.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:49:24.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:49:24.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:49:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16195.248257 secs 2023-09-10T03:49:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:24 smithi170 ceph-mon[102832]: pgmap v8799: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-09-10T03:49:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:49:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:49:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:49:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16195.959308 secs 2023-09-10T03:49:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16195.959308 secs 2023-09-10T03:49:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16195.959308 secs 2023-09-10T03:49:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:26 smithi082 ceph-mon[98694]: pgmap v8800: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 55 KiB/s wr, 9 op/s 2023-09-10T03:49:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:26 smithi098 ceph-mon[103124]: pgmap v8800: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 55 KiB/s wr, 9 op/s 2023-09-10T03:49:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:26 smithi170 ceph-mon[102832]: pgmap v8800: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 55 KiB/s wr, 9 op/s 2023-09-10T03:49:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:28 smithi082 ceph-mon[98694]: pgmap v8801: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 180 KiB/s wr, 14 op/s 2023-09-10T03:49:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16199.040265 secs 2023-09-10T03:49:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:49:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:28 smithi098 ceph-mon[103124]: pgmap v8801: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 180 KiB/s wr, 14 op/s 2023-09-10T03:49:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16199.040265 secs 2023-09-10T03:49:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:49:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:28 smithi170 ceph-mon[102832]: pgmap v8801: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 180 KiB/s wr, 14 op/s 2023-09-10T03:49:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16199.040265 secs 2023-09-10T03:49:28.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:49:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16200.248315 secs 2023-09-10T03:49:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:49:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:49:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:49:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16200.248315 secs 2023-09-10T03:49:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:49:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:49:29.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:49:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16200.248315 secs 2023-09-10T03:49:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:49:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:49:29.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:49:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16200.959377 secs 2023-09-10T03:49:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:30 smithi082 ceph-mon[98694]: pgmap v8802: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 9 op/s 2023-09-10T03:49:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16200.959377 secs 2023-09-10T03:49:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:30 smithi098 ceph-mon[103124]: pgmap v8802: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 9 op/s 2023-09-10T03:49:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16200.959377 secs 2023-09-10T03:49:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:30 smithi170 ceph-mon[102832]: pgmap v8802: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 9 op/s 2023-09-10T03:49:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:31 smithi170 ceph-mon[102832]: pgmap v8803: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 195 KiB/s wr, 14 op/s 2023-09-10T03:49:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:31 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:49:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:49:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:49:31.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:31 smithi082 ceph-mon[98694]: pgmap v8803: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 195 KiB/s wr, 14 op/s 2023-09-10T03:49:31.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:31 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:49:31.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:49:31.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:49:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:31 smithi098 ceph-mon[103124]: pgmap v8803: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 195 KiB/s wr, 14 op/s 2023-09-10T03:49:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:31 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:49:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:49:31.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:49:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16204.040363 secs 2023-09-10T03:49:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16204.040363 secs 2023-09-10T03:49:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16204.040363 secs 2023-09-10T03:49:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16205.248372 secs 2023-09-10T03:49:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:34 smithi170 ceph-mon[102832]: pgmap v8804: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 10 op/s 2023-09-10T03:49:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16205.248372 secs 2023-09-10T03:49:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:34 smithi082 ceph-mon[98694]: pgmap v8804: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 10 op/s 2023-09-10T03:49:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16205.248372 secs 2023-09-10T03:49:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:34 smithi098 ceph-mon[103124]: pgmap v8804: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 10 op/s 2023-09-10T03:49:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16205.959443 secs 2023-09-10T03:49:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16205.959443 secs 2023-09-10T03:49:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16205.959443 secs 2023-09-10T03:49:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:36 smithi170 ceph-mon[102832]: pgmap v8805: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 10 op/s 2023-09-10T03:49:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:36 smithi082 ceph-mon[98694]: pgmap v8805: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 10 op/s 2023-09-10T03:49:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:36 smithi098 ceph-mon[103124]: pgmap v8805: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 10 op/s 2023-09-10T03:49:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:38 smithi170 ceph-mon[102832]: pgmap v8806: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 239 KiB/s wr, 16 op/s 2023-09-10T03:49:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:38 smithi082 ceph-mon[98694]: pgmap v8806: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 239 KiB/s wr, 16 op/s 2023-09-10T03:49:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:38 smithi098 ceph-mon[103124]: pgmap v8806: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 239 KiB/s wr, 16 op/s 2023-09-10T03:49:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16210.248428 secs 2023-09-10T03:49:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:49:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16210.248428 secs 2023-09-10T03:49:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:49:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16210.248428 secs 2023-09-10T03:49:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:49:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16210.959488 secs 2023-09-10T03:49:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:40 smithi170 ceph-mon[102832]: pgmap v8807: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T03:49:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:49:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:49:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:49:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16210.959488 secs 2023-09-10T03:49:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:40 smithi082 ceph-mon[98694]: pgmap v8807: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T03:49:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:49:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:49:40.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:49:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16210.959488 secs 2023-09-10T03:49:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:40 smithi098 ceph-mon[103124]: pgmap v8807: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T03:49:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:49:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:49:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:49:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:42 smithi170 ceph-mon[102832]: pgmap v8808: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 330 KiB/s wr, 18 op/s 2023-09-10T03:49:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:42 smithi082 ceph-mon[98694]: pgmap v8808: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 330 KiB/s wr, 18 op/s 2023-09-10T03:49:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:42 smithi098 ceph-mon[103124]: pgmap v8808: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 330 KiB/s wr, 18 op/s 2023-09-10T03:49:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16215.248492 secs 2023-09-10T03:49:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:44 smithi170 ceph-mon[102832]: pgmap v8809: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T03:49:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16215.248492 secs 2023-09-10T03:49:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:44 smithi082 ceph-mon[98694]: pgmap v8809: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T03:49:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16215.248492 secs 2023-09-10T03:49:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:44 smithi098 ceph-mon[103124]: pgmap v8809: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T03:49:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16215.959555 secs 2023-09-10T03:49:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16215.959555 secs 2023-09-10T03:49:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16215.959555 secs 2023-09-10T03:49:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:46 smithi170 ceph-mon[102832]: pgmap v8810: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T03:49:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:46 smithi082 ceph-mon[98694]: pgmap v8810: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T03:49:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:46 smithi098 ceph-mon[103124]: pgmap v8810: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T03:49:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:48 smithi170 ceph-mon[102832]: pgmap v8811: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 287 KiB/s wr, 18 op/s 2023-09-10T03:49:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:48 smithi082 ceph-mon[98694]: pgmap v8811: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 287 KiB/s wr, 18 op/s 2023-09-10T03:49:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:48 smithi098 ceph-mon[103124]: pgmap v8811: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 287 KiB/s wr, 18 op/s 2023-09-10T03:49:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16220.248547 secs 2023-09-10T03:49:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16220.248547 secs 2023-09-10T03:49:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16220.248547 secs 2023-09-10T03:49:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16220.959619 secs 2023-09-10T03:49:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:50 smithi170 ceph-mon[102832]: pgmap v8812: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 244 KiB/s wr, 12 op/s 2023-09-10T03:49:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16220.959619 secs 2023-09-10T03:49:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:50 smithi082 ceph-mon[98694]: pgmap v8812: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 244 KiB/s wr, 12 op/s 2023-09-10T03:49:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16220.959619 secs 2023-09-10T03:49:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:50 smithi098 ceph-mon[103124]: pgmap v8812: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 244 KiB/s wr, 12 op/s 2023-09-10T03:49:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:52 smithi170 ceph-mon[102832]: pgmap v8813: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 466 KiB/s wr, 19 op/s 2023-09-10T03:49:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:52 smithi082 ceph-mon[98694]: pgmap v8813: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 466 KiB/s wr, 19 op/s 2023-09-10T03:49:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:52 smithi098 ceph-mon[103124]: pgmap v8813: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 466 KiB/s wr, 19 op/s 2023-09-10T03:49:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16224.040658 secs 2023-09-10T03:49:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16224.040658 secs 2023-09-10T03:49:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16224.040658 secs 2023-09-10T03:49:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16225.248611 secs 2023-09-10T03:49:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:54 smithi170 ceph-mon[102832]: pgmap v8814: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 250 KiB/s wr, 12 op/s 2023-09-10T03:49:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16225.248611 secs 2023-09-10T03:49:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:54 smithi082 ceph-mon[98694]: pgmap v8814: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 250 KiB/s wr, 12 op/s 2023-09-10T03:49:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16225.248611 secs 2023-09-10T03:49:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:54 smithi098 ceph-mon[103124]: pgmap v8814: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 250 KiB/s wr, 12 op/s 2023-09-10T03:49:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16225.959684 secs 2023-09-10T03:49:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16225.959684 secs 2023-09-10T03:49:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16225.959684 secs 2023-09-10T03:49:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:56 smithi170 ceph-mon[102832]: pgmap v8815: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 250 KiB/s wr, 12 op/s 2023-09-10T03:49:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:49:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:49:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:49:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:56 smithi082 ceph-mon[98694]: pgmap v8815: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 250 KiB/s wr, 12 op/s 2023-09-10T03:49:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:49:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:49:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:49:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:56 smithi098 ceph-mon[103124]: pgmap v8815: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 250 KiB/s wr, 12 op/s 2023-09-10T03:49:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:49:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:49:56.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:49:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:58 smithi170 ceph-mon[102832]: pgmap v8816: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 302 KiB/s wr, 17 op/s 2023-09-10T03:49:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16229.040737 secs 2023-09-10T03:49:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:58 smithi082 ceph-mon[98694]: pgmap v8816: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 302 KiB/s wr, 17 op/s 2023-09-10T03:49:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16229.040737 secs 2023-09-10T03:49:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:58 smithi098 ceph-mon[103124]: pgmap v8816: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 302 KiB/s wr, 17 op/s 2023-09-10T03:49:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16229.040737 secs 2023-09-10T03:49:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:49:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16230.248668 secs 2023-09-10T03:49:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:49:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16230.248668 secs 2023-09-10T03:49:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:49:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16230.248668 secs 2023-09-10T03:50:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16230.959749 secs 2023-09-10T03:50:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:00 smithi170 ceph-mon[102832]: pgmap v8817: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 274 KiB/s wr, 12 op/s 2023-09-10T03:50:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:00 smithi170 ceph-mon[102832]: Health detail: HEALTH_WARN 3 MDSs report slow requests 2023-09-10T03:50:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:00 smithi170 ceph-mon[102832]: [WRN] MDS_SLOW_REQUEST: 3 MDSs report slow requests 2023-09-10T03:50:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:00 smithi170 ceph-mon[102832]: mds.l(mds.4): 3 slow requests are blocked > 180 secs 2023-09-10T03:50:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:00 smithi170 ceph-mon[102832]: mds.h(mds.1): 3 slow requests are blocked > 180 secs 2023-09-10T03:50:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:00 smithi170 ceph-mon[102832]: mds.b(mds.0): 1 slow requests are blocked > 180 secs 2023-09-10T03:50:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16230.959749 secs 2023-09-10T03:50:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:00 smithi082 ceph-mon[98694]: pgmap v8817: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 274 KiB/s wr, 12 op/s 2023-09-10T03:50:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:00 smithi082 ceph-mon[98694]: Health detail: HEALTH_WARN 3 MDSs report slow requests 2023-09-10T03:50:00.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:00 smithi082 ceph-mon[98694]: [WRN] MDS_SLOW_REQUEST: 3 MDSs report slow requests 2023-09-10T03:50:00.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:00 smithi082 ceph-mon[98694]: mds.l(mds.4): 3 slow requests are blocked > 180 secs 2023-09-10T03:50:00.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:00 smithi082 ceph-mon[98694]: mds.h(mds.1): 3 slow requests are blocked > 180 secs 2023-09-10T03:50:00.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:00 smithi082 ceph-mon[98694]: mds.b(mds.0): 1 slow requests are blocked > 180 secs 2023-09-10T03:50:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16230.959749 secs 2023-09-10T03:50:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:00 smithi098 ceph-mon[103124]: pgmap v8817: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 274 KiB/s wr, 12 op/s 2023-09-10T03:50:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:00 smithi098 ceph-mon[103124]: Health detail: HEALTH_WARN 3 MDSs report slow requests 2023-09-10T03:50:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:00 smithi098 ceph-mon[103124]: [WRN] MDS_SLOW_REQUEST: 3 MDSs report slow requests 2023-09-10T03:50:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:00 smithi098 ceph-mon[103124]: mds.l(mds.4): 3 slow requests are blocked > 180 secs 2023-09-10T03:50:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:00 smithi098 ceph-mon[103124]: mds.h(mds.1): 3 slow requests are blocked > 180 secs 2023-09-10T03:50:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:00 smithi098 ceph-mon[103124]: mds.b(mds.0): 1 slow requests are blocked > 180 secs 2023-09-10T03:50:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:02 smithi170 ceph-mon[102832]: pgmap v8818: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 401 KiB/s wr, 17 op/s 2023-09-10T03:50:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:02 smithi082 ceph-mon[98694]: pgmap v8818: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 401 KiB/s wr, 17 op/s 2023-09-10T03:50:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:02 smithi098 ceph-mon[103124]: pgmap v8818: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 401 KiB/s wr, 17 op/s 2023-09-10T03:50:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16234.040787 secs 2023-09-10T03:50:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16234.040787 secs 2023-09-10T03:50:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16234.040787 secs 2023-09-10T03:50:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16235.248730 secs 2023-09-10T03:50:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:04 smithi170 ceph-mon[102832]: pgmap v8819: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T03:50:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16235.248730 secs 2023-09-10T03:50:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:04 smithi082 ceph-mon[98694]: pgmap v8819: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T03:50:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16235.248730 secs 2023-09-10T03:50:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:04 smithi098 ceph-mon[103124]: pgmap v8819: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T03:50:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16235.959815 secs 2023-09-10T03:50:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16235.959815 secs 2023-09-10T03:50:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16235.959815 secs 2023-09-10T03:50:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:06 smithi170 ceph-mon[102832]: pgmap v8820: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T03:50:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:06 smithi082 ceph-mon[98694]: pgmap v8820: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T03:50:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:06 smithi098 ceph-mon[103124]: pgmap v8820: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T03:50:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:08 smithi170 ceph-mon[102832]: pgmap v8821: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 236 KiB/s wr, 30 op/s 2023-09-10T03:50:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16239.040847 secs 2023-09-10T03:50:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:08 smithi082 ceph-mon[98694]: pgmap v8821: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 236 KiB/s wr, 30 op/s 2023-09-10T03:50:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16239.040847 secs 2023-09-10T03:50:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:08 smithi098 ceph-mon[103124]: pgmap v8821: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 236 KiB/s wr, 30 op/s 2023-09-10T03:50:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16239.040847 secs 2023-09-10T03:50:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16240.248790 secs 2023-09-10T03:50:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:09 smithi082 ceph-mon[98694]: osdmap e619: 12 total, 12 up, 12 in 2023-09-10T03:50:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16240.248790 secs 2023-09-10T03:50:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:09 smithi098 ceph-mon[103124]: osdmap e619: 12 total, 12 up, 12 in 2023-09-10T03:50:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16240.248790 secs 2023-09-10T03:50:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:09 smithi170 ceph-mon[102832]: osdmap e619: 12 total, 12 up, 12 in 2023-09-10T03:50:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16240.959879 secs 2023-09-10T03:50:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:10 smithi082 ceph-mon[98694]: pgmap v8823: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 221 KiB/s wr, 30 op/s 2023-09-10T03:50:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16240.959879 secs 2023-09-10T03:50:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:10 smithi098 ceph-mon[103124]: pgmap v8823: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 221 KiB/s wr, 30 op/s 2023-09-10T03:50:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16240.959879 secs 2023-09-10T03:50:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:10 smithi170 ceph-mon[102832]: pgmap v8823: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 221 KiB/s wr, 30 op/s 2023-09-10T03:50:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:12 smithi082 ceph-mon[98694]: pgmap v8824: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 122 KiB/s wr, 29 op/s 2023-09-10T03:50:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:12 smithi098 ceph-mon[103124]: pgmap v8824: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 122 KiB/s wr, 29 op/s 2023-09-10T03:50:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:12 smithi170 ceph-mon[102832]: pgmap v8824: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 122 KiB/s wr, 29 op/s 2023-09-10T03:50:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16244.040911 secs 2023-09-10T03:50:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:13 smithi082 ceph-mon[98694]: osdmap e620: 12 total, 12 up, 12 in 2023-09-10T03:50:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16244.040911 secs 2023-09-10T03:50:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:13 smithi098 ceph-mon[103124]: osdmap e620: 12 total, 12 up, 12 in 2023-09-10T03:50:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16244.040911 secs 2023-09-10T03:50:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:13 smithi170 ceph-mon[102832]: osdmap e620: 12 total, 12 up, 12 in 2023-09-10T03:50:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16245.248852 secs 2023-09-10T03:50:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:14 smithi082 ceph-mon[98694]: pgmap v8826: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 152 KiB/s wr, 36 op/s 2023-09-10T03:50:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16245.248852 secs 2023-09-10T03:50:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:14 smithi098 ceph-mon[103124]: pgmap v8826: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 152 KiB/s wr, 36 op/s 2023-09-10T03:50:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16245.248852 secs 2023-09-10T03:50:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:14 smithi170 ceph-mon[102832]: pgmap v8826: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 152 KiB/s wr, 36 op/s 2023-09-10T03:50:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16245.959945 secs 2023-09-10T03:50:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16245.959945 secs 2023-09-10T03:50:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16245.959945 secs 2023-09-10T03:50:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:16 smithi082 ceph-mon[98694]: pgmap v8827: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 7 op/s 2023-09-10T03:50:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:16 smithi098 ceph-mon[103124]: pgmap v8827: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 7 op/s 2023-09-10T03:50:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:16 smithi170 ceph-mon[102832]: pgmap v8827: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 7 op/s 2023-09-10T03:50:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:18 smithi082 ceph-mon[98694]: pgmap v8828: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 14 op/s 2023-09-10T03:50:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16249.040987 secs 2023-09-10T03:50:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:18 smithi098 ceph-mon[103124]: pgmap v8828: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 14 op/s 2023-09-10T03:50:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16249.040987 secs 2023-09-10T03:50:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:18 smithi170 ceph-mon[102832]: pgmap v8828: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 14 op/s 2023-09-10T03:50:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16249.040987 secs 2023-09-10T03:50:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16250.248911 secs 2023-09-10T03:50:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16250.248911 secs 2023-09-10T03:50:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16250.248911 secs 2023-09-10T03:50:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16250.960012 secs 2023-09-10T03:50:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:20 smithi082 ceph-mon[98694]: pgmap v8829: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T03:50:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16250.960012 secs 2023-09-10T03:50:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:20 smithi098 ceph-mon[103124]: pgmap v8829: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T03:50:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16250.960012 secs 2023-09-10T03:50:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:20 smithi170 ceph-mon[102832]: pgmap v8829: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T03:50:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:22 smithi098 ceph-mon[103124]: pgmap v8830: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T03:50:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:22 smithi170 ceph-mon[102832]: pgmap v8830: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T03:50:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:22 smithi082 ceph-mon[98694]: pgmap v8830: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T03:50:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:50:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:50:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:50:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16255.248968 secs 2023-09-10T03:50:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:24 smithi098 ceph-mon[103124]: pgmap v8831: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T03:50:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:50:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:50:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:50:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16255.248968 secs 2023-09-10T03:50:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:24 smithi170 ceph-mon[102832]: pgmap v8831: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T03:50:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:50:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:50:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:50:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16255.248968 secs 2023-09-10T03:50:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:24 smithi082 ceph-mon[98694]: pgmap v8831: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 12 op/s 2023-09-10T03:50:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:50:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:50:24.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:50:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16255.960078 secs 2023-09-10T03:50:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16255.960078 secs 2023-09-10T03:50:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16255.960078 secs 2023-09-10T03:50:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:26 smithi098 ceph-mon[103124]: pgmap v8832: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T03:50:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:26 smithi170 ceph-mon[102832]: pgmap v8832: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T03:50:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:26 smithi082 ceph-mon[98694]: pgmap v8832: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 10 op/s 2023-09-10T03:50:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:28 smithi098 ceph-mon[103124]: pgmap v8833: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 164 KiB/s wr, 15 op/s 2023-09-10T03:50:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16259.041106 secs 2023-09-10T03:50:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:28 smithi170 ceph-mon[102832]: pgmap v8833: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 164 KiB/s wr, 15 op/s 2023-09-10T03:50:28.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16259.041106 secs 2023-09-10T03:50:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:28 smithi082 ceph-mon[98694]: pgmap v8833: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 164 KiB/s wr, 15 op/s 2023-09-10T03:50:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16259.041106 secs 2023-09-10T03:50:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16260.249026 secs 2023-09-10T03:50:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:50:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16260.249026 secs 2023-09-10T03:50:29.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:50:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16260.249026 secs 2023-09-10T03:50:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:50:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16260.960146 secs 2023-09-10T03:50:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:30 smithi098 ceph-mon[103124]: pgmap v8834: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T03:50:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16260.960146 secs 2023-09-10T03:50:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:30 smithi170 ceph-mon[102832]: pgmap v8834: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T03:50:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16260.960146 secs 2023-09-10T03:50:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:30 smithi082 ceph-mon[98694]: pgmap v8834: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T03:50:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:31 smithi170 ceph-mon[102832]: pgmap v8835: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 261 KiB/s wr, 17 op/s 2023-09-10T03:50:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:31 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:50:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:50:31.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:50:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:31 smithi082 ceph-mon[98694]: pgmap v8835: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 261 KiB/s wr, 17 op/s 2023-09-10T03:50:31.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:31 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:50:31.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:50:31.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:50:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:31 smithi098 ceph-mon[103124]: pgmap v8835: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 261 KiB/s wr, 17 op/s 2023-09-10T03:50:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:31 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:50:31.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:50:31.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:50:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:32.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:32.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:32.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:32.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:32.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:32.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:32.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:32.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:32.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:32.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:32.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:32.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:32.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:32.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:32.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:32.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:32.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:32.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:32.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:32.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:33 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 83744k 2023-09-10T03:50:33.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:33 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 85754605: error parsing value: Value '85754605' is below minimum 939524096 2023-09-10T03:50:33.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16264.041167 secs 2023-09-10T03:50:33.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:33.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:33.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:33.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:33.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:33.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:33.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:33 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T03:50:33.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:33 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T03:50:33.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:33.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:33.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:33 smithi098 ceph-mon[103124]: pgmap v8836: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 261 KiB/s wr, 12 op/s 2023-09-10T03:50:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:33 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 83744k 2023-09-10T03:50:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:33 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 85754605: error parsing value: Value '85754605' is below minimum 939524096 2023-09-10T03:50:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16264.041167 secs 2023-09-10T03:50:33.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:33.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:33.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:33.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:33.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:33.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:33.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:33 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T03:50:33.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:33 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T03:50:33.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:33.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:33.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:33 smithi170 ceph-mon[102832]: pgmap v8836: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 261 KiB/s wr, 12 op/s 2023-09-10T03:50:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:33 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 83744k 2023-09-10T03:50:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:33 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 85754605: error parsing value: Value '85754605' is below minimum 939524096 2023-09-10T03:50:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16264.041167 secs 2023-09-10T03:50:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:33.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:33.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:33.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:33.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:33.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:33 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T03:50:33.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:33 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T03:50:33.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:33.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:33.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:33 smithi082 ceph-mon[98694]: pgmap v8836: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 261 KiB/s wr, 12 op/s 2023-09-10T03:50:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16265.249084 secs 2023-09-10T03:50:35.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16265.960214 secs 2023-09-10T03:50:35.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:35.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:35.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:35.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:35.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:35.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:35.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:34 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T03:50:35.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:34 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T03:50:35.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:50:35.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:50:35.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16265.249084 secs 2023-09-10T03:50:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16265.960214 secs 2023-09-10T03:50:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:35.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:35.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:35.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:35.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:35.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:35.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:34 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T03:50:35.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:34 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T03:50:35.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:50:35.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:50:35.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16265.249084 secs 2023-09-10T03:50:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16265.960214 secs 2023-09-10T03:50:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:35.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:35.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:35.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:35.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T03:50:35.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:34 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T03:50:35.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:34 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T03:50:35.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:50:35.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:50:35.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:50:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:35 smithi098 ceph-mon[103124]: pgmap v8837: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 261 KiB/s wr, 12 op/s 2023-09-10T03:50:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:35 smithi170 ceph-mon[102832]: pgmap v8837: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 261 KiB/s wr, 12 op/s 2023-09-10T03:50:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:35 smithi082 ceph-mon[98694]: pgmap v8837: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 261 KiB/s wr, 12 op/s 2023-09-10T03:50:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:38 smithi170 ceph-mon[102832]: pgmap v8838: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 282 KiB/s wr, 18 op/s 2023-09-10T03:50:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:38 smithi082 ceph-mon[98694]: pgmap v8838: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 282 KiB/s wr, 18 op/s 2023-09-10T03:50:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:38 smithi098 ceph-mon[103124]: pgmap v8838: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 282 KiB/s wr, 18 op/s 2023-09-10T03:50:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16270.249156 secs 2023-09-10T03:50:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:50:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16270.249156 secs 2023-09-10T03:50:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:50:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16270.249156 secs 2023-09-10T03:50:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:50:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16270.960285 secs 2023-09-10T03:50:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:40 smithi170 ceph-mon[102832]: pgmap v8839: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 173 KiB/s wr, 12 op/s 2023-09-10T03:50:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:50:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:50:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:50:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16270.960285 secs 2023-09-10T03:50:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:40 smithi082 ceph-mon[98694]: pgmap v8839: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 173 KiB/s wr, 12 op/s 2023-09-10T03:50:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:50:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:50:40.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:50:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16270.960285 secs 2023-09-10T03:50:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:40 smithi098 ceph-mon[103124]: pgmap v8839: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 173 KiB/s wr, 12 op/s 2023-09-10T03:50:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:50:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:50:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:50:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:42 smithi170 ceph-mon[102832]: pgmap v8840: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 466 KiB/s wr, 19 op/s 2023-09-10T03:50:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:42 smithi082 ceph-mon[98694]: pgmap v8840: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 466 KiB/s wr, 19 op/s 2023-09-10T03:50:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:42 smithi098 ceph-mon[103124]: pgmap v8840: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 466 KiB/s wr, 19 op/s 2023-09-10T03:50:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16275.249214 secs 2023-09-10T03:50:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:44 smithi170 ceph-mon[102832]: pgmap v8841: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 314 KiB/s wr, 12 op/s 2023-09-10T03:50:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16275.249214 secs 2023-09-10T03:50:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:44 smithi082 ceph-mon[98694]: pgmap v8841: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 314 KiB/s wr, 12 op/s 2023-09-10T03:50:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16275.249214 secs 2023-09-10T03:50:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:44 smithi098 ceph-mon[103124]: pgmap v8841: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 314 KiB/s wr, 12 op/s 2023-09-10T03:50:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16275.960349 secs 2023-09-10T03:50:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16275.960349 secs 2023-09-10T03:50:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16275.960349 secs 2023-09-10T03:50:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:46 smithi170 ceph-mon[102832]: pgmap v8842: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 314 KiB/s wr, 12 op/s 2023-09-10T03:50:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:46 smithi082 ceph-mon[98694]: pgmap v8842: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 314 KiB/s wr, 12 op/s 2023-09-10T03:50:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:46 smithi098 ceph-mon[103124]: pgmap v8842: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 314 KiB/s wr, 12 op/s 2023-09-10T03:50:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:48 smithi170 ceph-mon[102832]: pgmap v8843: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 355 KiB/s wr, 18 op/s 2023-09-10T03:50:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:48 smithi082 ceph-mon[98694]: pgmap v8843: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 355 KiB/s wr, 18 op/s 2023-09-10T03:50:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:48 smithi098 ceph-mon[103124]: pgmap v8843: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 355 KiB/s wr, 18 op/s 2023-09-10T03:50:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16280.249271 secs 2023-09-10T03:50:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16280.249271 secs 2023-09-10T03:50:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16280.249271 secs 2023-09-10T03:50:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16280.960416 secs 2023-09-10T03:50:50.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:50 smithi170 ceph-mon[102832]: pgmap v8844: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 333 KiB/s wr, 12 op/s 2023-09-10T03:50:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16280.960416 secs 2023-09-10T03:50:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:50 smithi082 ceph-mon[98694]: pgmap v8844: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 333 KiB/s wr, 12 op/s 2023-09-10T03:50:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16280.960416 secs 2023-09-10T03:50:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:50 smithi098 ceph-mon[103124]: pgmap v8844: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 333 KiB/s wr, 12 op/s 2023-09-10T03:50:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:52 smithi170 ceph-mon[102832]: pgmap v8845: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 458 KiB/s wr, 18 op/s 2023-09-10T03:50:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:52 smithi082 ceph-mon[98694]: pgmap v8845: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 458 KiB/s wr, 18 op/s 2023-09-10T03:50:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:52 smithi098 ceph-mon[103124]: pgmap v8845: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 458 KiB/s wr, 18 op/s 2023-09-10T03:50:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16284.041529 secs 2023-09-10T03:50:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16284.041529 secs 2023-09-10T03:50:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16284.041529 secs 2023-09-10T03:50:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16285.249354 secs 2023-09-10T03:50:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:54 smithi170 ceph-mon[102832]: pgmap v8846: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T03:50:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16285.249354 secs 2023-09-10T03:50:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:54 smithi082 ceph-mon[98694]: pgmap v8846: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T03:50:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16285.249354 secs 2023-09-10T03:50:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:54 smithi098 ceph-mon[103124]: pgmap v8846: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T03:50:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16285.960486 secs 2023-09-10T03:50:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16285.960486 secs 2023-09-10T03:50:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16285.960486 secs 2023-09-10T03:50:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:56 smithi170 ceph-mon[102832]: pgmap v8847: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T03:50:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:50:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:50:56.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:50:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:56 smithi082 ceph-mon[98694]: pgmap v8847: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T03:50:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:50:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:50:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:50:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:56 smithi098 ceph-mon[103124]: pgmap v8847: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T03:50:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:50:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:50:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:50:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:58 smithi170 ceph-mon[102832]: pgmap v8848: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 261 KiB/s wr, 17 op/s 2023-09-10T03:50:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16289.041591 secs 2023-09-10T03:50:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:58 smithi082 ceph-mon[98694]: pgmap v8848: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 261 KiB/s wr, 17 op/s 2023-09-10T03:50:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16289.041591 secs 2023-09-10T03:50:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:58 smithi098 ceph-mon[103124]: pgmap v8848: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 261 KiB/s wr, 17 op/s 2023-09-10T03:50:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16289.041591 secs 2023-09-10T03:50:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:50:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16290.249445 secs 2023-09-10T03:50:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:50:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16290.249445 secs 2023-09-10T03:50:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:50:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16290.249445 secs 2023-09-10T03:51:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16290.960553 secs 2023-09-10T03:51:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:00 smithi082 ceph-mon[98694]: pgmap v8849: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 221 KiB/s wr, 11 op/s 2023-09-10T03:51:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16290.960553 secs 2023-09-10T03:51:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:00 smithi098 ceph-mon[103124]: pgmap v8849: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 221 KiB/s wr, 11 op/s 2023-09-10T03:51:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16290.960553 secs 2023-09-10T03:51:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:00 smithi170 ceph-mon[102832]: pgmap v8849: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 221 KiB/s wr, 11 op/s 2023-09-10T03:51:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:02 smithi082 ceph-mon[98694]: pgmap v8850: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 270 KiB/s wr, 16 op/s 2023-09-10T03:51:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:02 smithi098 ceph-mon[103124]: pgmap v8850: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 270 KiB/s wr, 16 op/s 2023-09-10T03:51:02.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:02 smithi170 ceph-mon[102832]: pgmap v8850: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 270 KiB/s wr, 16 op/s 2023-09-10T03:51:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16294.041655 secs 2023-09-10T03:51:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16294.041655 secs 2023-09-10T03:51:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16294.041655 secs 2023-09-10T03:51:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16295.249521 secs 2023-09-10T03:51:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:04 smithi082 ceph-mon[98694]: pgmap v8851: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 145 KiB/s wr, 10 op/s 2023-09-10T03:51:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16295.249521 secs 2023-09-10T03:51:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:04 smithi098 ceph-mon[103124]: pgmap v8851: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 145 KiB/s wr, 10 op/s 2023-09-10T03:51:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16295.249521 secs 2023-09-10T03:51:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:04 smithi170 ceph-mon[102832]: pgmap v8851: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 145 KiB/s wr, 10 op/s 2023-09-10T03:51:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16295.960619 secs 2023-09-10T03:51:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16295.960619 secs 2023-09-10T03:51:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16295.960619 secs 2023-09-10T03:51:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:06 smithi082 ceph-mon[98694]: pgmap v8852: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 145 KiB/s wr, 10 op/s 2023-09-10T03:51:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:06 smithi098 ceph-mon[103124]: pgmap v8852: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 145 KiB/s wr, 10 op/s 2023-09-10T03:51:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:06 smithi170 ceph-mon[102832]: pgmap v8852: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 145 KiB/s wr, 10 op/s 2023-09-10T03:51:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:08 smithi082 ceph-mon[98694]: pgmap v8853: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 203 KiB/s wr, 30 op/s 2023-09-10T03:51:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16299.041759 secs 2023-09-10T03:51:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:08 smithi098 ceph-mon[103124]: pgmap v8853: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 203 KiB/s wr, 30 op/s 2023-09-10T03:51:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16299.041759 secs 2023-09-10T03:51:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:08 smithi170 ceph-mon[102832]: pgmap v8853: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 203 KiB/s wr, 30 op/s 2023-09-10T03:51:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16299.041759 secs 2023-09-10T03:51:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16300.249577 secs 2023-09-10T03:51:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:09 smithi082 ceph-mon[98694]: osdmap e621: 12 total, 12 up, 12 in 2023-09-10T03:51:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16300.249577 secs 2023-09-10T03:51:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:09 smithi098 ceph-mon[103124]: osdmap e621: 12 total, 12 up, 12 in 2023-09-10T03:51:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16300.249577 secs 2023-09-10T03:51:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:09 smithi170 ceph-mon[102832]: osdmap e621: 12 total, 12 up, 12 in 2023-09-10T03:51:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16300.960685 secs 2023-09-10T03:51:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:10 smithi098 ceph-mon[103124]: pgmap v8855: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T03:51:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16300.960685 secs 2023-09-10T03:51:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:10 smithi170 ceph-mon[102832]: pgmap v8855: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T03:51:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16300.960685 secs 2023-09-10T03:51:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:10 smithi082 ceph-mon[98694]: pgmap v8855: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T03:51:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:12 smithi098 ceph-mon[103124]: pgmap v8856: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 135 KiB/s wr, 29 op/s 2023-09-10T03:51:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:12 smithi170 ceph-mon[102832]: pgmap v8856: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 135 KiB/s wr, 29 op/s 2023-09-10T03:51:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:12 smithi082 ceph-mon[98694]: pgmap v8856: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 135 KiB/s wr, 29 op/s 2023-09-10T03:51:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16304.041775 secs 2023-09-10T03:51:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:13 smithi098 ceph-mon[103124]: osdmap e622: 12 total, 12 up, 12 in 2023-09-10T03:51:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16304.041775 secs 2023-09-10T03:51:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:13 smithi170 ceph-mon[102832]: osdmap e622: 12 total, 12 up, 12 in 2023-09-10T03:51:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16304.041775 secs 2023-09-10T03:51:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:13 smithi082 ceph-mon[98694]: osdmap e622: 12 total, 12 up, 12 in 2023-09-10T03:51:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16305.249635 secs 2023-09-10T03:51:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:14 smithi098 ceph-mon[103124]: pgmap v8858: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 168 KiB/s wr, 36 op/s 2023-09-10T03:51:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16305.249635 secs 2023-09-10T03:51:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:14 smithi170 ceph-mon[102832]: pgmap v8858: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 168 KiB/s wr, 36 op/s 2023-09-10T03:51:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16305.249635 secs 2023-09-10T03:51:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:14 smithi082 ceph-mon[98694]: pgmap v8858: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 168 KiB/s wr, 36 op/s 2023-09-10T03:51:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16305.960749 secs 2023-09-10T03:51:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16305.960749 secs 2023-09-10T03:51:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16305.960749 secs 2023-09-10T03:51:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:16 smithi098 ceph-mon[103124]: pgmap v8859: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 83 KiB/s wr, 7 op/s 2023-09-10T03:51:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:16 smithi170 ceph-mon[102832]: pgmap v8859: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 83 KiB/s wr, 7 op/s 2023-09-10T03:51:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:16 smithi082 ceph-mon[98694]: pgmap v8859: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 83 KiB/s wr, 7 op/s 2023-09-10T03:51:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:18 smithi098 ceph-mon[103124]: pgmap v8860: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 13 op/s 2023-09-10T03:51:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16309.041834 secs 2023-09-10T03:51:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:18 smithi170 ceph-mon[102832]: pgmap v8860: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 13 op/s 2023-09-10T03:51:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16309.041834 secs 2023-09-10T03:51:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:18 smithi082 ceph-mon[98694]: pgmap v8860: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 13 op/s 2023-09-10T03:51:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16309.041834 secs 2023-09-10T03:51:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16310.249724 secs 2023-09-10T03:51:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16310.249724 secs 2023-09-10T03:51:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16310.249724 secs 2023-09-10T03:51:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16310.960838 secs 2023-09-10T03:51:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:20 smithi098 ceph-mon[103124]: pgmap v8861: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T03:51:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16310.960838 secs 2023-09-10T03:51:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:20 smithi170 ceph-mon[102832]: pgmap v8861: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T03:51:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16310.960838 secs 2023-09-10T03:51:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:20 smithi082 ceph-mon[98694]: pgmap v8861: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T03:51:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:22 smithi098 ceph-mon[103124]: pgmap v8862: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T03:51:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:22 smithi170 ceph-mon[102832]: pgmap v8862: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T03:51:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:22 smithi082 ceph-mon[98694]: pgmap v8862: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T03:51:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:51:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:51:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:51:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16315.249783 secs 2023-09-10T03:51:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:24 smithi098 ceph-mon[103124]: pgmap v8863: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T03:51:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:51:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:51:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:51:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16315.249783 secs 2023-09-10T03:51:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:24 smithi170 ceph-mon[102832]: pgmap v8863: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T03:51:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:51:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:51:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:51:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16315.249783 secs 2023-09-10T03:51:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:24 smithi082 ceph-mon[98694]: pgmap v8863: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T03:51:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:51:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:51:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:51:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16315.960906 secs 2023-09-10T03:51:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16315.960906 secs 2023-09-10T03:51:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16315.960906 secs 2023-09-10T03:51:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:26 smithi098 ceph-mon[103124]: pgmap v8864: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 9 op/s 2023-09-10T03:51:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:26 smithi170 ceph-mon[102832]: pgmap v8864: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 9 op/s 2023-09-10T03:51:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:26 smithi082 ceph-mon[98694]: pgmap v8864: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 9 op/s 2023-09-10T03:51:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:28 smithi098 ceph-mon[103124]: pgmap v8865: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 141 KiB/s wr, 14 op/s 2023-09-10T03:51:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:28 smithi170 ceph-mon[102832]: pgmap v8865: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 141 KiB/s wr, 14 op/s 2023-09-10T03:51:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:28 smithi082 ceph-mon[98694]: pgmap v8865: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 141 KiB/s wr, 14 op/s 2023-09-10T03:51:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16320.249835 secs 2023-09-10T03:51:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16320.249835 secs 2023-09-10T03:51:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16320.249835 secs 2023-09-10T03:51:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16320.960972 secs 2023-09-10T03:51:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:30 smithi098 ceph-mon[103124]: pgmap v8866: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 9 op/s 2023-09-10T03:51:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16320.960972 secs 2023-09-10T03:51:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:30 smithi170 ceph-mon[102832]: pgmap v8866: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 9 op/s 2023-09-10T03:51:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16320.960972 secs 2023-09-10T03:51:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:30 smithi082 ceph-mon[98694]: pgmap v8866: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 9 op/s 2023-09-10T03:51:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:31 smithi170 ceph-mon[102832]: pgmap v8867: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 211 KiB/s wr, 15 op/s 2023-09-10T03:51:31.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:31 smithi082 ceph-mon[98694]: pgmap v8867: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 211 KiB/s wr, 15 op/s 2023-09-10T03:51:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:31 smithi098 ceph-mon[103124]: pgmap v8867: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 211 KiB/s wr, 15 op/s 2023-09-10T03:51:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16324.042087 secs 2023-09-10T03:51:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16324.042087 secs 2023-09-10T03:51:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16324.042087 secs 2023-09-10T03:51:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16325.249893 secs 2023-09-10T03:51:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:34 smithi170 ceph-mon[102832]: pgmap v8868: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T03:51:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:51:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16325.249893 secs 2023-09-10T03:51:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:34 smithi082 ceph-mon[98694]: pgmap v8868: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T03:51:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:51:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16325.249893 secs 2023-09-10T03:51:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:34 smithi098 ceph-mon[103124]: pgmap v8868: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T03:51:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:51:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16325.961036 secs 2023-09-10T03:51:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:51:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:51:35.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:51:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16325.961036 secs 2023-09-10T03:51:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:51:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:51:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:51:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16325.961036 secs 2023-09-10T03:51:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:51:35.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:51:35.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:51:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:36 smithi170 ceph-mon[102832]: pgmap v8869: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T03:51:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:36 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:51:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:51:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:51:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:36 smithi082 ceph-mon[98694]: pgmap v8869: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T03:51:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:36 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:51:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:51:36.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:51:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:36 smithi098 ceph-mon[103124]: pgmap v8869: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T03:51:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:36 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:51:36.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:51:36.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:51:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:38 smithi170 ceph-mon[102832]: pgmap v8870: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 352 KiB/s wr, 17 op/s 2023-09-10T03:51:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16329.042198 secs 2023-09-10T03:51:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:38 smithi082 ceph-mon[98694]: pgmap v8870: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 352 KiB/s wr, 17 op/s 2023-09-10T03:51:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16329.042198 secs 2023-09-10T03:51:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:38 smithi098 ceph-mon[103124]: pgmap v8870: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 352 KiB/s wr, 17 op/s 2023-09-10T03:51:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16329.042198 secs 2023-09-10T03:51:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16330.249959 secs 2023-09-10T03:51:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16330.249959 secs 2023-09-10T03:51:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16330.249959 secs 2023-09-10T03:51:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16330.961101 secs 2023-09-10T03:51:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:40 smithi170 ceph-mon[102832]: pgmap v8871: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T03:51:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16330.961101 secs 2023-09-10T03:51:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:40 smithi082 ceph-mon[98694]: pgmap v8871: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T03:51:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16330.961101 secs 2023-09-10T03:51:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:40 smithi098 ceph-mon[103124]: pgmap v8871: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T03:51:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:42 smithi170 ceph-mon[102832]: pgmap v8872: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 463 KiB/s wr, 19 op/s 2023-09-10T03:51:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:42 smithi082 ceph-mon[98694]: pgmap v8872: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 463 KiB/s wr, 19 op/s 2023-09-10T03:51:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:42 smithi098 ceph-mon[103124]: pgmap v8872: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 463 KiB/s wr, 19 op/s 2023-09-10T03:51:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16334.042299 secs 2023-09-10T03:51:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16334.042299 secs 2023-09-10T03:51:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16334.042299 secs 2023-09-10T03:51:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16335.250017 secs 2023-09-10T03:51:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:44 smithi170 ceph-mon[102832]: pgmap v8873: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T03:51:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16335.250017 secs 2023-09-10T03:51:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:44 smithi082 ceph-mon[98694]: pgmap v8873: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T03:51:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16335.250017 secs 2023-09-10T03:51:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:44 smithi098 ceph-mon[103124]: pgmap v8873: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T03:51:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16335.961191 secs 2023-09-10T03:51:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16335.961191 secs 2023-09-10T03:51:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16335.961191 secs 2023-09-10T03:51:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:46 smithi082 ceph-mon[98694]: pgmap v8874: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T03:51:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:46 smithi098 ceph-mon[103124]: pgmap v8874: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T03:51:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:46 smithi170 ceph-mon[102832]: pgmap v8874: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T03:51:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:48 smithi082 ceph-mon[98694]: pgmap v8875: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 451 KiB/s wr, 18 op/s 2023-09-10T03:51:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16339.042389 secs 2023-09-10T03:51:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:48 smithi098 ceph-mon[103124]: pgmap v8875: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 451 KiB/s wr, 18 op/s 2023-09-10T03:51:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16339.042389 secs 2023-09-10T03:51:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:48 smithi170 ceph-mon[102832]: pgmap v8875: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 451 KiB/s wr, 18 op/s 2023-09-10T03:51:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16339.042389 secs 2023-09-10T03:51:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16340.250104 secs 2023-09-10T03:51:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16340.250104 secs 2023-09-10T03:51:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16340.250104 secs 2023-09-10T03:51:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16340.961257 secs 2023-09-10T03:51:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:50 smithi082 ceph-mon[98694]: pgmap v8876: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 241 KiB/s wr, 12 op/s 2023-09-10T03:51:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16340.961257 secs 2023-09-10T03:51:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:50 smithi098 ceph-mon[103124]: pgmap v8876: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 241 KiB/s wr, 12 op/s 2023-09-10T03:51:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16340.961257 secs 2023-09-10T03:51:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:50 smithi170 ceph-mon[102832]: pgmap v8876: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 241 KiB/s wr, 12 op/s 2023-09-10T03:51:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:52 smithi082 ceph-mon[98694]: pgmap v8877: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 424 KiB/s wr, 17 op/s 2023-09-10T03:51:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:52 smithi098 ceph-mon[103124]: pgmap v8877: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 424 KiB/s wr, 17 op/s 2023-09-10T03:51:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:52 smithi170 ceph-mon[102832]: pgmap v8877: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 424 KiB/s wr, 17 op/s 2023-09-10T03:51:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16344.042450 secs 2023-09-10T03:51:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16344.042450 secs 2023-09-10T03:51:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16344.042450 secs 2023-09-10T03:51:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16345.250188 secs 2023-09-10T03:51:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:54 smithi082 ceph-mon[98694]: pgmap v8878: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 242 KiB/s wr, 10 op/s 2023-09-10T03:51:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16345.250188 secs 2023-09-10T03:51:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:54 smithi098 ceph-mon[103124]: pgmap v8878: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 242 KiB/s wr, 10 op/s 2023-09-10T03:51:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16345.250188 secs 2023-09-10T03:51:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:54 smithi170 ceph-mon[102832]: pgmap v8878: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 242 KiB/s wr, 10 op/s 2023-09-10T03:51:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16345.961321 secs 2023-09-10T03:51:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16345.961321 secs 2023-09-10T03:51:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16345.961321 secs 2023-09-10T03:51:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:56 smithi082 ceph-mon[98694]: pgmap v8879: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 242 KiB/s wr, 10 op/s 2023-09-10T03:51:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:56 smithi098 ceph-mon[103124]: pgmap v8879: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 242 KiB/s wr, 10 op/s 2023-09-10T03:51:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:56 smithi170 ceph-mon[102832]: pgmap v8879: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 242 KiB/s wr, 10 op/s 2023-09-10T03:51:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:58 smithi082 ceph-mon[98694]: pgmap v8880: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 364 KiB/s wr, 16 op/s 2023-09-10T03:51:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16349.042512 secs 2023-09-10T03:51:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:58 smithi098 ceph-mon[103124]: pgmap v8880: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 364 KiB/s wr, 16 op/s 2023-09-10T03:51:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16349.042512 secs 2023-09-10T03:51:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:58 smithi170 ceph-mon[102832]: pgmap v8880: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 364 KiB/s wr, 16 op/s 2023-09-10T03:51:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16349.042512 secs 2023-09-10T03:51:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:51:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16350.250242 secs 2023-09-10T03:51:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:51:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16350.250242 secs 2023-09-10T03:51:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:51:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16350.250242 secs 2023-09-10T03:52:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16350.961389 secs 2023-09-10T03:52:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:00 smithi098 ceph-mon[103124]: pgmap v8881: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 305 KiB/s wr, 10 op/s 2023-09-10T03:52:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16350.961389 secs 2023-09-10T03:52:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:00 smithi170 ceph-mon[102832]: pgmap v8881: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 305 KiB/s wr, 10 op/s 2023-09-10T03:52:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16350.961389 secs 2023-09-10T03:52:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:00 smithi082 ceph-mon[98694]: pgmap v8881: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 305 KiB/s wr, 10 op/s 2023-09-10T03:52:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:02 smithi098 ceph-mon[103124]: pgmap v8882: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 354 KiB/s wr, 16 op/s 2023-09-10T03:52:02.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:02 smithi170 ceph-mon[102832]: pgmap v8882: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 354 KiB/s wr, 16 op/s 2023-09-10T03:52:02.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:02 smithi082 ceph-mon[98694]: pgmap v8882: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 354 KiB/s wr, 16 op/s 2023-09-10T03:52:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16354.042628 secs 2023-09-10T03:52:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16354.042628 secs 2023-09-10T03:52:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16354.042628 secs 2023-09-10T03:52:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16355.250324 secs 2023-09-10T03:52:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:04 smithi098 ceph-mon[103124]: pgmap v8883: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 171 KiB/s wr, 10 op/s 2023-09-10T03:52:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16355.250324 secs 2023-09-10T03:52:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:04 smithi170 ceph-mon[102832]: pgmap v8883: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 171 KiB/s wr, 10 op/s 2023-09-10T03:52:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16355.250324 secs 2023-09-10T03:52:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:04 smithi082 ceph-mon[98694]: pgmap v8883: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 171 KiB/s wr, 10 op/s 2023-09-10T03:52:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16355.961456 secs 2023-09-10T03:52:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16355.961456 secs 2023-09-10T03:52:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16355.961456 secs 2023-09-10T03:52:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:06 smithi098 ceph-mon[103124]: pgmap v8884: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 171 KiB/s wr, 10 op/s 2023-09-10T03:52:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:06 smithi170 ceph-mon[102832]: pgmap v8884: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 171 KiB/s wr, 10 op/s 2023-09-10T03:52:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:06 smithi082 ceph-mon[98694]: pgmap v8884: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 171 KiB/s wr, 10 op/s 2023-09-10T03:52:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:08 smithi098 ceph-mon[103124]: pgmap v8885: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 229 KiB/s wr, 30 op/s 2023-09-10T03:52:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16359.042722 secs 2023-09-10T03:52:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:08 smithi170 ceph-mon[102832]: pgmap v8885: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 229 KiB/s wr, 30 op/s 2023-09-10T03:52:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16359.042722 secs 2023-09-10T03:52:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:08 smithi082 ceph-mon[98694]: pgmap v8885: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 229 KiB/s wr, 30 op/s 2023-09-10T03:52:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16359.042722 secs 2023-09-10T03:52:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16360.250409 secs 2023-09-10T03:52:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:09 smithi098 ceph-mon[103124]: osdmap e623: 12 total, 12 up, 12 in 2023-09-10T03:52:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16360.250409 secs 2023-09-10T03:52:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:09 smithi170 ceph-mon[102832]: osdmap e623: 12 total, 12 up, 12 in 2023-09-10T03:52:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16360.250409 secs 2023-09-10T03:52:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:09 smithi082 ceph-mon[98694]: osdmap e623: 12 total, 12 up, 12 in 2023-09-10T03:52:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16360.961487 secs 2023-09-10T03:52:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:10 smithi098 ceph-mon[103124]: pgmap v8887: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T03:52:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16360.961487 secs 2023-09-10T03:52:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:10 smithi170 ceph-mon[102832]: pgmap v8887: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T03:52:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16360.961487 secs 2023-09-10T03:52:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:10 smithi082 ceph-mon[98694]: pgmap v8887: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T03:52:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:12 smithi098 ceph-mon[103124]: pgmap v8888: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-10T03:52:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:12 smithi170 ceph-mon[102832]: pgmap v8888: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-10T03:52:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:12 smithi082 ceph-mon[98694]: pgmap v8888: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-10T03:52:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16364.042773 secs 2023-09-10T03:52:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:13 smithi098 ceph-mon[103124]: osdmap e624: 12 total, 12 up, 12 in 2023-09-10T03:52:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16364.042773 secs 2023-09-10T03:52:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:13 smithi170 ceph-mon[102832]: osdmap e624: 12 total, 12 up, 12 in 2023-09-10T03:52:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16364.042773 secs 2023-09-10T03:52:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:13 smithi082 ceph-mon[98694]: osdmap e624: 12 total, 12 up, 12 in 2023-09-10T03:52:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16365.250468 secs 2023-09-10T03:52:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:14 smithi098 ceph-mon[103124]: pgmap v8890: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 87 KiB/s wr, 36 op/s 2023-09-10T03:52:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16365.250468 secs 2023-09-10T03:52:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:14 smithi170 ceph-mon[102832]: pgmap v8890: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 87 KiB/s wr, 36 op/s 2023-09-10T03:52:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16365.250468 secs 2023-09-10T03:52:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:14 smithi082 ceph-mon[98694]: pgmap v8890: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 87 KiB/s wr, 36 op/s 2023-09-10T03:52:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16365.961554 secs 2023-09-10T03:52:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16365.961554 secs 2023-09-10T03:52:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16365.961554 secs 2023-09-10T03:52:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:16 smithi098 ceph-mon[103124]: pgmap v8891: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:52:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:16 smithi170 ceph-mon[102832]: pgmap v8891: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:52:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:16 smithi082 ceph-mon[98694]: pgmap v8891: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:52:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:17 smithi170 ceph-mon[102832]: pgmap v8892: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 13 op/s 2023-09-10T03:52:17.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:17 smithi082 ceph-mon[98694]: pgmap v8892: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 13 op/s 2023-09-10T03:52:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:17 smithi098 ceph-mon[103124]: pgmap v8892: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 13 op/s 2023-09-10T03:52:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16369.042833 secs 2023-09-10T03:52:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16369.042833 secs 2023-09-10T03:52:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16369.042833 secs 2023-09-10T03:52:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16370.250526 secs 2023-09-10T03:52:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:19 smithi170 ceph-mon[102832]: pgmap v8893: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T03:52:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16370.250526 secs 2023-09-10T03:52:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:19 smithi082 ceph-mon[98694]: pgmap v8893: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T03:52:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16370.250526 secs 2023-09-10T03:52:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:19 smithi098 ceph-mon[103124]: pgmap v8893: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T03:52:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16370.961618 secs 2023-09-10T03:52:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16370.961618 secs 2023-09-10T03:52:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16370.961618 secs 2023-09-10T03:52:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:21 smithi170 ceph-mon[102832]: pgmap v8894: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T03:52:21.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:21 smithi082 ceph-mon[98694]: pgmap v8894: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T03:52:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:21 smithi098 ceph-mon[103124]: pgmap v8894: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T03:52:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:52:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:52:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:52:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:52:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:52:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:52:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16375.250583 secs 2023-09-10T03:52:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:23 smithi170 ceph-mon[102832]: pgmap v8895: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T03:52:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:52:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:52:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:52:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16375.250583 secs 2023-09-10T03:52:24.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:23 smithi082 ceph-mon[98694]: pgmap v8895: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T03:52:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:52:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:52:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:52:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16375.250583 secs 2023-09-10T03:52:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:23 smithi098 ceph-mon[103124]: pgmap v8895: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 11 op/s 2023-09-10T03:52:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16375.961682 secs 2023-09-10T03:52:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16375.961682 secs 2023-09-10T03:52:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16375.961682 secs 2023-09-10T03:52:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:26 smithi170 ceph-mon[102832]: pgmap v8896: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 9 op/s 2023-09-10T03:52:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:26 smithi082 ceph-mon[98694]: pgmap v8896: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 9 op/s 2023-09-10T03:52:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:26 smithi098 ceph-mon[103124]: pgmap v8896: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 9 op/s 2023-09-10T03:52:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:28 smithi170 ceph-mon[102832]: pgmap v8897: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T03:52:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:28 smithi082 ceph-mon[98694]: pgmap v8897: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T03:52:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:28 smithi098 ceph-mon[103124]: pgmap v8897: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T03:52:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16380.250641 secs 2023-09-10T03:52:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16380.250641 secs 2023-09-10T03:52:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16380.250641 secs 2023-09-10T03:52:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16380.961748 secs 2023-09-10T03:52:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:30 smithi082 ceph-mon[98694]: pgmap v8898: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 9 op/s 2023-09-10T03:52:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16380.961748 secs 2023-09-10T03:52:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:30 smithi098 ceph-mon[103124]: pgmap v8898: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 9 op/s 2023-09-10T03:52:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16380.961748 secs 2023-09-10T03:52:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:30 smithi170 ceph-mon[102832]: pgmap v8898: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 9 op/s 2023-09-10T03:52:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:32 smithi082 ceph-mon[98694]: pgmap v8899: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 199 KiB/s wr, 15 op/s 2023-09-10T03:52:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:32 smithi098 ceph-mon[103124]: pgmap v8899: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 199 KiB/s wr, 15 op/s 2023-09-10T03:52:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:32 smithi170 ceph-mon[102832]: pgmap v8899: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 199 KiB/s wr, 15 op/s 2023-09-10T03:52:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16385.250703 secs 2023-09-10T03:52:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:34 smithi170 ceph-mon[102832]: pgmap v8900: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 199 KiB/s wr, 10 op/s 2023-09-10T03:52:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16385.250703 secs 2023-09-10T03:52:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:34 smithi082 ceph-mon[98694]: pgmap v8900: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 199 KiB/s wr, 10 op/s 2023-09-10T03:52:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16385.250703 secs 2023-09-10T03:52:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:34 smithi098 ceph-mon[103124]: pgmap v8900: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 199 KiB/s wr, 10 op/s 2023-09-10T03:52:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16385.961811 secs 2023-09-10T03:52:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:52:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16385.961811 secs 2023-09-10T03:52:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:52:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16385.961811 secs 2023-09-10T03:52:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:52:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:52:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:52:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:36 smithi170 ceph-mon[102832]: pgmap v8901: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 199 KiB/s wr, 10 op/s 2023-09-10T03:52:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:52:36.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:52:36.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:52:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:52:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:52:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:36 smithi082 ceph-mon[98694]: pgmap v8901: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 199 KiB/s wr, 10 op/s 2023-09-10T03:52:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:52:36.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:52:36.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:52:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:52:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:52:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:36 smithi098 ceph-mon[103124]: pgmap v8901: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 199 KiB/s wr, 10 op/s 2023-09-10T03:52:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:52:36.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:52:36.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:52:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:38 smithi170 ceph-mon[102832]: pgmap v8902: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 240 KiB/s wr, 17 op/s 2023-09-10T03:52:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:38 smithi082 ceph-mon[98694]: pgmap v8902: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 240 KiB/s wr, 17 op/s 2023-09-10T03:52:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:38 smithi098 ceph-mon[103124]: pgmap v8902: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 240 KiB/s wr, 17 op/s 2023-09-10T03:52:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16390.250756 secs 2023-09-10T03:52:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16390.250756 secs 2023-09-10T03:52:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16390.250756 secs 2023-09-10T03:52:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16390.961876 secs 2023-09-10T03:52:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:40 smithi082 ceph-mon[98694]: pgmap v8903: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T03:52:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16390.961876 secs 2023-09-10T03:52:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:40 smithi098 ceph-mon[103124]: pgmap v8903: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T03:52:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16390.961876 secs 2023-09-10T03:52:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:40 smithi170 ceph-mon[102832]: pgmap v8903: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T03:52:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:42 smithi082 ceph-mon[98694]: pgmap v8904: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 335 KiB/s wr, 19 op/s 2023-09-10T03:52:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:42 smithi098 ceph-mon[103124]: pgmap v8904: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 335 KiB/s wr, 19 op/s 2023-09-10T03:52:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:42 smithi170 ceph-mon[102832]: pgmap v8904: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 335 KiB/s wr, 19 op/s 2023-09-10T03:52:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16394.043228 secs 2023-09-10T03:52:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16394.043228 secs 2023-09-10T03:52:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16394.043228 secs 2023-09-10T03:52:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16395.250815 secs 2023-09-10T03:52:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:44 smithi082 ceph-mon[98694]: pgmap v8905: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T03:52:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:52:44.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:52:44.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:52:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16395.250815 secs 2023-09-10T03:52:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:44 smithi098 ceph-mon[103124]: pgmap v8905: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T03:52:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:52:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:52:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:52:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16395.250815 secs 2023-09-10T03:52:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:44 smithi170 ceph-mon[102832]: pgmap v8905: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T03:52:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:52:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:52:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:52:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16395.961940 secs 2023-09-10T03:52:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16395.961940 secs 2023-09-10T03:52:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16395.961940 secs 2023-09-10T03:52:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:46 smithi098 ceph-mon[103124]: pgmap v8906: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T03:52:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:46 smithi170 ceph-mon[102832]: pgmap v8906: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T03:52:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:46 smithi082 ceph-mon[98694]: pgmap v8906: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T03:52:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:48 smithi098 ceph-mon[103124]: pgmap v8907: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 367 KiB/s wr, 18 op/s 2023-09-10T03:52:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16399.043296 secs 2023-09-10T03:52:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:48 smithi170 ceph-mon[102832]: pgmap v8907: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 367 KiB/s wr, 18 op/s 2023-09-10T03:52:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16399.043296 secs 2023-09-10T03:52:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:48 smithi082 ceph-mon[98694]: pgmap v8907: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 367 KiB/s wr, 18 op/s 2023-09-10T03:52:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16399.043296 secs 2023-09-10T03:52:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16400.250874 secs 2023-09-10T03:52:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16400.250874 secs 2023-09-10T03:52:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16400.250874 secs 2023-09-10T03:52:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16400.962007 secs 2023-09-10T03:52:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:50 smithi098 ceph-mon[103124]: pgmap v8908: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 327 KiB/s wr, 12 op/s 2023-09-10T03:52:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16400.962007 secs 2023-09-10T03:52:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:50 smithi170 ceph-mon[102832]: pgmap v8908: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 327 KiB/s wr, 12 op/s 2023-09-10T03:52:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16400.962007 secs 2023-09-10T03:52:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:50 smithi082 ceph-mon[98694]: pgmap v8908: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 327 KiB/s wr, 12 op/s 2023-09-10T03:52:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:52 smithi098 ceph-mon[103124]: pgmap v8909: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 553 KiB/s wr, 18 op/s 2023-09-10T03:52:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:52 smithi170 ceph-mon[102832]: pgmap v8909: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 553 KiB/s wr, 18 op/s 2023-09-10T03:52:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:52 smithi082 ceph-mon[98694]: pgmap v8909: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 553 KiB/s wr, 18 op/s 2023-09-10T03:52:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16404.043388 secs 2023-09-10T03:52:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16404.043388 secs 2023-09-10T03:52:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16404.043388 secs 2023-09-10T03:52:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16405.250940 secs 2023-09-10T03:52:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:54 smithi098 ceph-mon[103124]: pgmap v8910: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 330 KiB/s wr, 12 op/s 2023-09-10T03:52:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16405.250940 secs 2023-09-10T03:52:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:54 smithi170 ceph-mon[102832]: pgmap v8910: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 330 KiB/s wr, 12 op/s 2023-09-10T03:52:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16405.250940 secs 2023-09-10T03:52:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:54 smithi082 ceph-mon[98694]: pgmap v8910: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 330 KiB/s wr, 12 op/s 2023-09-10T03:52:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16405.962075 secs 2023-09-10T03:52:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16405.962075 secs 2023-09-10T03:52:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16405.962075 secs 2023-09-10T03:52:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:56 smithi098 ceph-mon[103124]: pgmap v8911: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 330 KiB/s wr, 12 op/s 2023-09-10T03:52:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:56 smithi170 ceph-mon[102832]: pgmap v8911: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 330 KiB/s wr, 12 op/s 2023-09-10T03:52:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:56 smithi082 ceph-mon[98694]: pgmap v8911: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 330 KiB/s wr, 12 op/s 2023-09-10T03:52:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:58 smithi098 ceph-mon[103124]: pgmap v8912: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 375 KiB/s wr, 17 op/s 2023-09-10T03:52:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16409.043449 secs 2023-09-10T03:52:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:58 smithi170 ceph-mon[102832]: pgmap v8912: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 375 KiB/s wr, 17 op/s 2023-09-10T03:52:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16409.043449 secs 2023-09-10T03:52:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:58 smithi082 ceph-mon[98694]: pgmap v8912: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 375 KiB/s wr, 17 op/s 2023-09-10T03:52:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16409.043449 secs 2023-09-10T03:52:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:52:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16410.251023 secs 2023-09-10T03:52:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:52:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16410.251023 secs 2023-09-10T03:52:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:52:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16410.251023 secs 2023-09-10T03:53:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16410.962146 secs 2023-09-10T03:53:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:00 smithi098 ceph-mon[103124]: pgmap v8913: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 272 KiB/s wr, 12 op/s 2023-09-10T03:53:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16410.962146 secs 2023-09-10T03:53:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:00 smithi170 ceph-mon[102832]: pgmap v8913: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 272 KiB/s wr, 12 op/s 2023-09-10T03:53:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16410.962146 secs 2023-09-10T03:53:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:00 smithi082 ceph-mon[98694]: pgmap v8913: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 272 KiB/s wr, 12 op/s 2023-09-10T03:53:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:02 smithi098 ceph-mon[103124]: pgmap v8914: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 366 KiB/s wr, 31 op/s 2023-09-10T03:53:02.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:02 smithi170 ceph-mon[102832]: pgmap v8914: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 366 KiB/s wr, 31 op/s 2023-09-10T03:53:02.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:02 smithi082 ceph-mon[98694]: pgmap v8914: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 366 KiB/s wr, 31 op/s 2023-09-10T03:53:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16414.043509 secs 2023-09-10T03:53:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16414.043509 secs 2023-09-10T03:53:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16414.043509 secs 2023-09-10T03:53:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16415.251127 secs 2023-09-10T03:53:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:04 smithi098 ceph-mon[103124]: pgmap v8915: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 139 KiB/s wr, 24 op/s 2023-09-10T03:53:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16415.251127 secs 2023-09-10T03:53:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:04 smithi170 ceph-mon[102832]: pgmap v8915: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 139 KiB/s wr, 24 op/s 2023-09-10T03:53:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16415.251127 secs 2023-09-10T03:53:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:04 smithi082 ceph-mon[98694]: pgmap v8915: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 139 KiB/s wr, 24 op/s 2023-09-10T03:53:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16415.962212 secs 2023-09-10T03:53:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16415.962212 secs 2023-09-10T03:53:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16415.962212 secs 2023-09-10T03:53:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:06 smithi170 ceph-mon[102832]: pgmap v8916: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 139 KiB/s wr, 24 op/s 2023-09-10T03:53:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:06 smithi082 ceph-mon[98694]: pgmap v8916: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 139 KiB/s wr, 24 op/s 2023-09-10T03:53:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:06 smithi098 ceph-mon[103124]: pgmap v8916: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 139 KiB/s wr, 24 op/s 2023-09-10T03:53:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:08 smithi170 ceph-mon[102832]: pgmap v8917: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 163 KiB/s wr, 30 op/s 2023-09-10T03:53:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16419.043588 secs 2023-09-10T03:53:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:08 smithi082 ceph-mon[98694]: pgmap v8917: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 163 KiB/s wr, 30 op/s 2023-09-10T03:53:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16419.043588 secs 2023-09-10T03:53:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:08 smithi098 ceph-mon[103124]: pgmap v8917: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 163 KiB/s wr, 30 op/s 2023-09-10T03:53:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16419.043588 secs 2023-09-10T03:53:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16420.251200 secs 2023-09-10T03:53:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:09 smithi170 ceph-mon[102832]: osdmap e625: 12 total, 12 up, 12 in 2023-09-10T03:53:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:09 smithi170 ceph-mon[102832]: pgmap v8919: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 142 KiB/s wr, 30 op/s 2023-09-10T03:53:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16420.251200 secs 2023-09-10T03:53:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:09 smithi082 ceph-mon[98694]: osdmap e625: 12 total, 12 up, 12 in 2023-09-10T03:53:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:09 smithi082 ceph-mon[98694]: pgmap v8919: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 142 KiB/s wr, 30 op/s 2023-09-10T03:53:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16420.251200 secs 2023-09-10T03:53:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:09 smithi098 ceph-mon[103124]: osdmap e625: 12 total, 12 up, 12 in 2023-09-10T03:53:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:09 smithi098 ceph-mon[103124]: pgmap v8919: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 142 KiB/s wr, 30 op/s 2023-09-10T03:53:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16420.962280 secs 2023-09-10T03:53:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16420.962280 secs 2023-09-10T03:53:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16420.962280 secs 2023-09-10T03:53:11.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:11 smithi170 ceph-mon[102832]: pgmap v8920: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T03:53:11.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:11 smithi082 ceph-mon[98694]: pgmap v8920: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T03:53:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:11 smithi098 ceph-mon[103124]: pgmap v8920: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T03:53:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16424.043650 secs 2023-09-10T03:53:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16424.043650 secs 2023-09-10T03:53:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16424.043650 secs 2023-09-10T03:53:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16425.251259 secs 2023-09-10T03:53:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:14 smithi170 ceph-mon[102832]: osdmap e626: 12 total, 12 up, 12 in 2023-09-10T03:53:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:14 smithi170 ceph-mon[102832]: pgmap v8922: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 37 KiB/s wr, 16 op/s 2023-09-10T03:53:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16425.251259 secs 2023-09-10T03:53:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:14 smithi082 ceph-mon[98694]: osdmap e626: 12 total, 12 up, 12 in 2023-09-10T03:53:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:14 smithi082 ceph-mon[98694]: pgmap v8922: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 37 KiB/s wr, 16 op/s 2023-09-10T03:53:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16425.251259 secs 2023-09-10T03:53:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:14 smithi098 ceph-mon[103124]: osdmap e626: 12 total, 12 up, 12 in 2023-09-10T03:53:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:14 smithi098 ceph-mon[103124]: pgmap v8922: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 37 KiB/s wr, 16 op/s 2023-09-10T03:53:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16425.962343 secs 2023-09-10T03:53:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16425.962343 secs 2023-09-10T03:53:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16425.962343 secs 2023-09-10T03:53:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:16 smithi170 ceph-mon[102832]: pgmap v8923: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T03:53:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:16 smithi082 ceph-mon[98694]: pgmap v8923: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T03:53:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:16 smithi098 ceph-mon[103124]: pgmap v8923: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T03:53:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:18 smithi082 ceph-mon[98694]: pgmap v8924: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 76 KiB/s wr, 14 op/s 2023-09-10T03:53:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16429.043710 secs 2023-09-10T03:53:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:18 smithi098 ceph-mon[103124]: pgmap v8924: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 76 KiB/s wr, 14 op/s 2023-09-10T03:53:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16429.043710 secs 2023-09-10T03:53:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:18 smithi170 ceph-mon[102832]: pgmap v8924: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 76 KiB/s wr, 14 op/s 2023-09-10T03:53:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16429.043710 secs 2023-09-10T03:53:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16430.251341 secs 2023-09-10T03:53:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16430.251341 secs 2023-09-10T03:53:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16430.251341 secs 2023-09-10T03:53:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16430.962411 secs 2023-09-10T03:53:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:20 smithi082 ceph-mon[98694]: pgmap v8925: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 67 KiB/s wr, 12 op/s 2023-09-10T03:53:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16430.962411 secs 2023-09-10T03:53:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:20 smithi098 ceph-mon[103124]: pgmap v8925: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 67 KiB/s wr, 12 op/s 2023-09-10T03:53:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16430.962411 secs 2023-09-10T03:53:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:20 smithi170 ceph-mon[102832]: pgmap v8925: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 67 KiB/s wr, 12 op/s 2023-09-10T03:53:22.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:22 smithi082 ceph-mon[98694]: pgmap v8926: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 12 op/s 2023-09-10T03:53:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:22 smithi098 ceph-mon[103124]: pgmap v8926: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 12 op/s 2023-09-10T03:53:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:22 smithi170 ceph-mon[102832]: pgmap v8926: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 12 op/s 2023-09-10T03:53:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:53:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:53:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:53:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16435.251423 secs 2023-09-10T03:53:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:24 smithi082 ceph-mon[98694]: pgmap v8927: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 12 op/s 2023-09-10T03:53:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:53:24.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:53:24.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:53:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16435.251423 secs 2023-09-10T03:53:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:24 smithi098 ceph-mon[103124]: pgmap v8927: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 12 op/s 2023-09-10T03:53:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:53:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:53:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:53:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16435.251423 secs 2023-09-10T03:53:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:24 smithi170 ceph-mon[102832]: pgmap v8927: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 12 op/s 2023-09-10T03:53:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:53:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:53:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:53:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16435.962475 secs 2023-09-10T03:53:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16435.962475 secs 2023-09-10T03:53:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16435.962475 secs 2023-09-10T03:53:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:26 smithi098 ceph-mon[103124]: pgmap v8928: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T03:53:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:26 smithi170 ceph-mon[102832]: pgmap v8928: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T03:53:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:26 smithi082 ceph-mon[98694]: pgmap v8928: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 10 op/s 2023-09-10T03:53:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:28 smithi098 ceph-mon[103124]: pgmap v8929: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 181 KiB/s wr, 15 op/s 2023-09-10T03:53:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:28 smithi170 ceph-mon[102832]: pgmap v8929: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 181 KiB/s wr, 15 op/s 2023-09-10T03:53:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:28 smithi082 ceph-mon[98694]: pgmap v8929: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 181 KiB/s wr, 15 op/s 2023-09-10T03:53:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16440.251506 secs 2023-09-10T03:53:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16440.251506 secs 2023-09-10T03:53:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16440.251506 secs 2023-09-10T03:53:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16440.962542 secs 2023-09-10T03:53:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:30 smithi098 ceph-mon[103124]: pgmap v8930: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 125 KiB/s wr, 10 op/s 2023-09-10T03:53:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16440.962542 secs 2023-09-10T03:53:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:30 smithi170 ceph-mon[102832]: pgmap v8930: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 125 KiB/s wr, 10 op/s 2023-09-10T03:53:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16440.962542 secs 2023-09-10T03:53:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:30 smithi082 ceph-mon[98694]: pgmap v8930: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 125 KiB/s wr, 10 op/s 2023-09-10T03:53:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:32 smithi098 ceph-mon[103124]: pgmap v8931: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 264 KiB/s wr, 17 op/s 2023-09-10T03:53:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:32 smithi170 ceph-mon[102832]: pgmap v8931: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 264 KiB/s wr, 17 op/s 2023-09-10T03:53:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:32 smithi082 ceph-mon[98694]: pgmap v8931: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 264 KiB/s wr, 17 op/s 2023-09-10T03:53:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16445.251564 secs 2023-09-10T03:53:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:34 smithi170 ceph-mon[102832]: pgmap v8932: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T03:53:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16445.251564 secs 2023-09-10T03:53:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:34 smithi082 ceph-mon[98694]: pgmap v8932: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T03:53:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16445.251564 secs 2023-09-10T03:53:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:34 smithi098 ceph-mon[103124]: pgmap v8932: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T03:53:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16445.962606 secs 2023-09-10T03:53:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16445.962606 secs 2023-09-10T03:53:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16445.962606 secs 2023-09-10T03:53:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:36 smithi170 ceph-mon[102832]: pgmap v8933: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T03:53:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:53:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:36 smithi082 ceph-mon[98694]: pgmap v8933: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T03:53:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:53:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:36 smithi098 ceph-mon[103124]: pgmap v8933: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T03:53:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:53:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:53:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:53:37.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:53:37.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:53:37.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:53:37.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:53:37.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:53:37.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:53:37.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:53:37.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:53:37.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:53:37.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:53:37.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:53:37.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:53:37.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:53:37.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:53:37.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:53:37.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:53:37.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:53:37.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:53:37.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:53:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:38 smithi098 ceph-mon[103124]: pgmap v8934: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 337 KiB/s wr, 17 op/s 2023-09-10T03:53:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:38 smithi170 ceph-mon[102832]: pgmap v8934: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 337 KiB/s wr, 17 op/s 2023-09-10T03:53:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:38 smithi082 ceph-mon[98694]: pgmap v8934: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 337 KiB/s wr, 17 op/s 2023-09-10T03:53:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16450.251650 secs 2023-09-10T03:53:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16450.251650 secs 2023-09-10T03:53:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16450.251650 secs 2023-09-10T03:53:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16450.962676 secs 2023-09-10T03:53:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:40 smithi170 ceph-mon[102832]: pgmap v8935: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 211 KiB/s wr, 11 op/s 2023-09-10T03:53:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16450.962676 secs 2023-09-10T03:53:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:40 smithi082 ceph-mon[98694]: pgmap v8935: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 211 KiB/s wr, 11 op/s 2023-09-10T03:53:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16450.962676 secs 2023-09-10T03:53:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:40 smithi098 ceph-mon[103124]: pgmap v8935: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 211 KiB/s wr, 11 op/s 2023-09-10T03:53:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:42 smithi170 ceph-mon[102832]: pgmap v8936: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 458 KiB/s wr, 18 op/s 2023-09-10T03:53:42.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:42 smithi082 ceph-mon[98694]: pgmap v8936: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 458 KiB/s wr, 18 op/s 2023-09-10T03:53:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:42 smithi098 ceph-mon[103124]: pgmap v8936: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 458 KiB/s wr, 18 op/s 2023-09-10T03:53:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16455.251708 secs 2023-09-10T03:53:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:44 smithi170 ceph-mon[102832]: pgmap v8937: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 320 KiB/s wr, 11 op/s 2023-09-10T03:53:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16455.251708 secs 2023-09-10T03:53:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:44 smithi082 ceph-mon[98694]: pgmap v8937: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 320 KiB/s wr, 11 op/s 2023-09-10T03:53:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16455.251708 secs 2023-09-10T03:53:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:44 smithi098 ceph-mon[103124]: pgmap v8937: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 320 KiB/s wr, 11 op/s 2023-09-10T03:53:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16455.962740 secs 2023-09-10T03:53:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16455.962740 secs 2023-09-10T03:53:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16455.962740 secs 2023-09-10T03:53:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:46 smithi170 ceph-mon[102832]: pgmap v8938: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 320 KiB/s wr, 11 op/s 2023-09-10T03:53:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:46 smithi082 ceph-mon[98694]: pgmap v8938: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 320 KiB/s wr, 11 op/s 2023-09-10T03:53:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:46 smithi098 ceph-mon[103124]: pgmap v8938: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 320 KiB/s wr, 11 op/s 2023-09-10T03:53:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:48 smithi082 ceph-mon[98694]: pgmap v8939: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 405 KiB/s wr, 17 op/s 2023-09-10T03:53:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16459.044112 secs 2023-09-10T03:53:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:48 smithi098 ceph-mon[103124]: pgmap v8939: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 405 KiB/s wr, 17 op/s 2023-09-10T03:53:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16459.044112 secs 2023-09-10T03:53:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:48 smithi170 ceph-mon[102832]: pgmap v8939: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 405 KiB/s wr, 17 op/s 2023-09-10T03:53:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16459.044112 secs 2023-09-10T03:53:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16460.251766 secs 2023-09-10T03:53:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16460.251766 secs 2023-09-10T03:53:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16460.251766 secs 2023-09-10T03:53:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16460.962804 secs 2023-09-10T03:53:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:50 smithi098 ceph-mon[103124]: pgmap v8940: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T03:53:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16460.962804 secs 2023-09-10T03:53:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:50 smithi170 ceph-mon[102832]: pgmap v8940: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T03:53:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16460.962804 secs 2023-09-10T03:53:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:50 smithi082 ceph-mon[98694]: pgmap v8940: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T03:53:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:53:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:53:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:53:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:53:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:53:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:53:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:53:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:53:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:53:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:52 smithi170 ceph-mon[102832]: pgmap v8941: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 389 KiB/s wr, 18 op/s 2023-09-10T03:53:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:52 smithi082 ceph-mon[98694]: pgmap v8941: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 389 KiB/s wr, 18 op/s 2023-09-10T03:53:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:52 smithi098 ceph-mon[103124]: pgmap v8941: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 389 KiB/s wr, 18 op/s 2023-09-10T03:53:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16464.044184 secs 2023-09-10T03:53:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16464.044184 secs 2023-09-10T03:53:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16464.044184 secs 2023-09-10T03:53:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16465.251843 secs 2023-09-10T03:53:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:54 smithi170 ceph-mon[102832]: pgmap v8942: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T03:53:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16465.251843 secs 2023-09-10T03:53:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:54 smithi082 ceph-mon[98694]: pgmap v8942: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T03:53:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16465.251843 secs 2023-09-10T03:53:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:54 smithi098 ceph-mon[103124]: pgmap v8942: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T03:53:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16465.962870 secs 2023-09-10T03:53:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16465.962870 secs 2023-09-10T03:53:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16465.962870 secs 2023-09-10T03:53:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:56 smithi170 ceph-mon[102832]: pgmap v8943: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T03:53:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:56 smithi082 ceph-mon[98694]: pgmap v8943: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T03:53:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:56 smithi098 ceph-mon[103124]: pgmap v8943: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T03:53:57.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:57 smithi170 ceph-mon[102832]: pgmap v8944: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 259 KiB/s wr, 17 op/s 2023-09-10T03:53:57.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:57 smithi082 ceph-mon[98694]: pgmap v8944: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 259 KiB/s wr, 17 op/s 2023-09-10T03:53:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:57 smithi098 ceph-mon[103124]: pgmap v8944: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 259 KiB/s wr, 17 op/s 2023-09-10T03:53:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16469.044307 secs 2023-09-10T03:53:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16469.044307 secs 2023-09-10T03:53:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16469.044307 secs 2023-09-10T03:53:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16470.251925 secs 2023-09-10T03:53:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:53:59 smithi170 ceph-mon[102832]: pgmap v8945: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T03:53:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16470.251925 secs 2023-09-10T03:53:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:53:59 smithi082 ceph-mon[98694]: pgmap v8945: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T03:53:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16470.251925 secs 2023-09-10T03:53:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:53:59 smithi098 ceph-mon[103124]: pgmap v8945: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T03:54:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16470.962937 secs 2023-09-10T03:54:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16470.962937 secs 2023-09-10T03:54:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16470.962937 secs 2023-09-10T03:54:01.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:01 smithi170 ceph-mon[102832]: pgmap v8946: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 235 KiB/s wr, 28 op/s 2023-09-10T03:54:01.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:01 smithi082 ceph-mon[98694]: pgmap v8946: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 235 KiB/s wr, 28 op/s 2023-09-10T03:54:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:01 smithi098 ceph-mon[103124]: pgmap v8946: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 235 KiB/s wr, 28 op/s 2023-09-10T03:54:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16474.044366 secs 2023-09-10T03:54:02.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16474.044366 secs 2023-09-10T03:54:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16474.044366 secs 2023-09-10T03:54:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16475.251984 secs 2023-09-10T03:54:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:04 smithi170 ceph-mon[102832]: pgmap v8947: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 178 KiB/s wr, 22 op/s 2023-09-10T03:54:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16475.251984 secs 2023-09-10T03:54:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:04 smithi082 ceph-mon[98694]: pgmap v8947: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 178 KiB/s wr, 22 op/s 2023-09-10T03:54:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16475.251984 secs 2023-09-10T03:54:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:04 smithi098 ceph-mon[103124]: pgmap v8947: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 178 KiB/s wr, 22 op/s 2023-09-10T03:54:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16475.963002 secs 2023-09-10T03:54:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16475.963002 secs 2023-09-10T03:54:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16475.963002 secs 2023-09-10T03:54:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:06 smithi170 ceph-mon[102832]: pgmap v8948: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 178 KiB/s wr, 22 op/s 2023-09-10T03:54:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:06 smithi082 ceph-mon[98694]: pgmap v8948: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 178 KiB/s wr, 22 op/s 2023-09-10T03:54:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:06 smithi098 ceph-mon[103124]: pgmap v8948: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 178 KiB/s wr, 22 op/s 2023-09-10T03:54:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:08 smithi082 ceph-mon[98694]: pgmap v8949: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 225 KiB/s wr, 31 op/s 2023-09-10T03:54:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16479.044426 secs 2023-09-10T03:54:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:08 smithi098 ceph-mon[103124]: pgmap v8949: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 225 KiB/s wr, 31 op/s 2023-09-10T03:54:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16479.044426 secs 2023-09-10T03:54:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:08 smithi170 ceph-mon[102832]: pgmap v8949: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 225 KiB/s wr, 31 op/s 2023-09-10T03:54:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16479.044426 secs 2023-09-10T03:54:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16480.252050 secs 2023-09-10T03:54:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:09 smithi098 ceph-mon[103124]: osdmap e627: 12 total, 12 up, 12 in 2023-09-10T03:54:09.549 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16480.252050 secs 2023-09-10T03:54:09.549 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:09 smithi082 ceph-mon[98694]: osdmap e627: 12 total, 12 up, 12 in 2023-09-10T03:54:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16480.252050 secs 2023-09-10T03:54:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:09 smithi170 ceph-mon[102832]: osdmap e627: 12 total, 12 up, 12 in 2023-09-10T03:54:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16480.963084 secs 2023-09-10T03:54:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:10 smithi082 ceph-mon[98694]: pgmap v8951: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 129 KiB/s wr, 30 op/s 2023-09-10T03:54:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16480.963084 secs 2023-09-10T03:54:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:10 smithi098 ceph-mon[103124]: pgmap v8951: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 129 KiB/s wr, 30 op/s 2023-09-10T03:54:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16480.963084 secs 2023-09-10T03:54:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:10 smithi170 ceph-mon[102832]: pgmap v8951: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 129 KiB/s wr, 30 op/s 2023-09-10T03:54:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:12 smithi098 ceph-mon[103124]: pgmap v8952: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 123 KiB/s wr, 17 op/s 2023-09-10T03:54:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:12 smithi170 ceph-mon[102832]: pgmap v8952: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 123 KiB/s wr, 17 op/s 2023-09-10T03:54:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:12 smithi082 ceph-mon[98694]: pgmap v8952: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 123 KiB/s wr, 17 op/s 2023-09-10T03:54:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16484.044488 secs 2023-09-10T03:54:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:13 smithi098 ceph-mon[103124]: osdmap e628: 12 total, 12 up, 12 in 2023-09-10T03:54:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16484.044488 secs 2023-09-10T03:54:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:13 smithi170 ceph-mon[102832]: osdmap e628: 12 total, 12 up, 12 in 2023-09-10T03:54:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16484.044488 secs 2023-09-10T03:54:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:13 smithi082 ceph-mon[98694]: osdmap e628: 12 total, 12 up, 12 in 2023-09-10T03:54:14.305 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16485.252109 secs 2023-09-10T03:54:14.306 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:14 smithi082 ceph-mon[98694]: pgmap v8954: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 154 KiB/s wr, 21 op/s 2023-09-10T03:54:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16485.252109 secs 2023-09-10T03:54:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:14 smithi098 ceph-mon[103124]: pgmap v8954: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 154 KiB/s wr, 21 op/s 2023-09-10T03:54:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16485.252109 secs 2023-09-10T03:54:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:14 smithi170 ceph-mon[102832]: pgmap v8954: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 154 KiB/s wr, 21 op/s 2023-09-10T03:54:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16485.963150 secs 2023-09-10T03:54:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16485.963150 secs 2023-09-10T03:54:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16485.963150 secs 2023-09-10T03:54:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:16 smithi098 ceph-mon[103124]: pgmap v8955: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 84 KiB/s wr, 8 op/s 2023-09-10T03:54:16.631 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:16 smithi082 ceph-mon[98694]: pgmap v8955: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 84 KiB/s wr, 8 op/s 2023-09-10T03:54:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:16 smithi170 ceph-mon[102832]: pgmap v8955: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 84 KiB/s wr, 8 op/s 2023-09-10T03:54:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:18 smithi098 ceph-mon[103124]: pgmap v8956: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 13 op/s 2023-09-10T03:54:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16489.044548 secs 2023-09-10T03:54:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:18 smithi170 ceph-mon[102832]: pgmap v8956: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 13 op/s 2023-09-10T03:54:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16489.044548 secs 2023-09-10T03:54:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:18 smithi082 ceph-mon[98694]: pgmap v8956: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 13 op/s 2023-09-10T03:54:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16489.044548 secs 2023-09-10T03:54:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16490.252168 secs 2023-09-10T03:54:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16490.252168 secs 2023-09-10T03:54:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16490.252168 secs 2023-09-10T03:54:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16490.963215 secs 2023-09-10T03:54:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:20 smithi098 ceph-mon[103124]: pgmap v8957: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-10T03:54:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16490.963215 secs 2023-09-10T03:54:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:20 smithi170 ceph-mon[102832]: pgmap v8957: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-10T03:54:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16490.963215 secs 2023-09-10T03:54:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:20 smithi082 ceph-mon[98694]: pgmap v8957: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-10T03:54:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:22 smithi098 ceph-mon[103124]: pgmap v8958: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T03:54:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:22 smithi170 ceph-mon[102832]: pgmap v8958: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T03:54:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:22 smithi082 ceph-mon[98694]: pgmap v8958: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T03:54:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:54:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:54:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:54:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16495.252227 secs 2023-09-10T03:54:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:24 smithi098 ceph-mon[103124]: pgmap v8959: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T03:54:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:54:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:54:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:54:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16495.252227 secs 2023-09-10T03:54:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:24 smithi170 ceph-mon[102832]: pgmap v8959: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T03:54:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:54:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:54:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:54:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16495.252227 secs 2023-09-10T03:54:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:24 smithi082 ceph-mon[98694]: pgmap v8959: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T03:54:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:54:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:54:24.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:54:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16495.963282 secs 2023-09-10T03:54:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16495.963282 secs 2023-09-10T03:54:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16495.963282 secs 2023-09-10T03:54:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:26 smithi082 ceph-mon[98694]: pgmap v8960: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T03:54:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:26 smithi098 ceph-mon[103124]: pgmap v8960: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T03:54:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:26 smithi170 ceph-mon[102832]: pgmap v8960: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T03:54:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:28 smithi098 ceph-mon[103124]: pgmap v8961: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 148 KiB/s wr, 14 op/s 2023-09-10T03:54:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:28 smithi170 ceph-mon[102832]: pgmap v8961: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 148 KiB/s wr, 14 op/s 2023-09-10T03:54:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:28 smithi082 ceph-mon[98694]: pgmap v8961: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 148 KiB/s wr, 14 op/s 2023-09-10T03:54:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16500.252324 secs 2023-09-10T03:54:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16500.252324 secs 2023-09-10T03:54:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16500.252324 secs 2023-09-10T03:54:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16500.963349 secs 2023-09-10T03:54:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:30 smithi170 ceph-mon[102832]: pgmap v8962: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T03:54:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16500.963349 secs 2023-09-10T03:54:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:30 smithi082 ceph-mon[98694]: pgmap v8962: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T03:54:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16500.963349 secs 2023-09-10T03:54:30.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:30 smithi098 ceph-mon[103124]: pgmap v8962: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T03:54:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:32 smithi170 ceph-mon[102832]: pgmap v8963: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 214 KiB/s wr, 16 op/s 2023-09-10T03:54:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:32 smithi082 ceph-mon[98694]: pgmap v8963: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 214 KiB/s wr, 16 op/s 2023-09-10T03:54:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:32 smithi098 ceph-mon[103124]: pgmap v8963: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 214 KiB/s wr, 16 op/s 2023-09-10T03:54:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16505.252383 secs 2023-09-10T03:54:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:34 smithi170 ceph-mon[102832]: pgmap v8964: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 143 KiB/s wr, 11 op/s 2023-09-10T03:54:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16505.252383 secs 2023-09-10T03:54:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:34 smithi082 ceph-mon[98694]: pgmap v8964: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 143 KiB/s wr, 11 op/s 2023-09-10T03:54:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16505.252383 secs 2023-09-10T03:54:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:34 smithi098 ceph-mon[103124]: pgmap v8964: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 143 KiB/s wr, 11 op/s 2023-09-10T03:54:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16505.963414 secs 2023-09-10T03:54:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16505.963414 secs 2023-09-10T03:54:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16505.963414 secs 2023-09-10T03:54:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:36 smithi170 ceph-mon[102832]: pgmap v8965: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 143 KiB/s wr, 11 op/s 2023-09-10T03:54:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:36 smithi082 ceph-mon[98694]: pgmap v8965: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 143 KiB/s wr, 11 op/s 2023-09-10T03:54:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:36 smithi098 ceph-mon[103124]: pgmap v8965: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 143 KiB/s wr, 11 op/s 2023-09-10T03:54:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:54:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:54:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:54:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:38 smithi170 ceph-mon[102832]: pgmap v8966: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 358 KiB/s wr, 17 op/s 2023-09-10T03:54:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:54:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:54:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:54:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:38 smithi082 ceph-mon[98694]: pgmap v8966: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 358 KiB/s wr, 17 op/s 2023-09-10T03:54:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:54:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:54:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:54:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:38 smithi098 ceph-mon[103124]: pgmap v8966: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 358 KiB/s wr, 17 op/s 2023-09-10T03:54:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:54:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:54:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:54:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16510.252441 secs 2023-09-10T03:54:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16510.252441 secs 2023-09-10T03:54:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16510.252441 secs 2023-09-10T03:54:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16510.963485 secs 2023-09-10T03:54:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:40 smithi170 ceph-mon[102832]: pgmap v8967: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T03:54:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16510.963485 secs 2023-09-10T03:54:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:40 smithi082 ceph-mon[98694]: pgmap v8967: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T03:54:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16510.963485 secs 2023-09-10T03:54:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:40 smithi098 ceph-mon[103124]: pgmap v8967: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T03:54:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:42 smithi170 ceph-mon[102832]: pgmap v8968: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 466 KiB/s wr, 19 op/s 2023-09-10T03:54:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:42 smithi082 ceph-mon[98694]: pgmap v8968: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 466 KiB/s wr, 19 op/s 2023-09-10T03:54:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:42 smithi098 ceph-mon[103124]: pgmap v8968: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 466 KiB/s wr, 19 op/s 2023-09-10T03:54:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16515.252494 secs 2023-09-10T03:54:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:44 smithi170 ceph-mon[102832]: pgmap v8969: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 399 KiB/s wr, 13 op/s 2023-09-10T03:54:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16515.252494 secs 2023-09-10T03:54:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:44 smithi082 ceph-mon[98694]: pgmap v8969: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 399 KiB/s wr, 13 op/s 2023-09-10T03:54:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16515.252494 secs 2023-09-10T03:54:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:44 smithi098 ceph-mon[103124]: pgmap v8969: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 399 KiB/s wr, 13 op/s 2023-09-10T03:54:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16515.963553 secs 2023-09-10T03:54:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16515.963553 secs 2023-09-10T03:54:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16515.963553 secs 2023-09-10T03:54:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:46 smithi170 ceph-mon[102832]: pgmap v8970: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 399 KiB/s wr, 13 op/s 2023-09-10T03:54:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:46 smithi082 ceph-mon[98694]: pgmap v8970: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 399 KiB/s wr, 13 op/s 2023-09-10T03:54:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:46 smithi098 ceph-mon[103124]: pgmap v8970: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 399 KiB/s wr, 13 op/s 2023-09-10T03:54:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:48 smithi098 ceph-mon[103124]: pgmap v8971: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 458 KiB/s wr, 19 op/s 2023-09-10T03:54:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:48 smithi170 ceph-mon[102832]: pgmap v8971: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 458 KiB/s wr, 19 op/s 2023-09-10T03:54:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:48 smithi082 ceph-mon[98694]: pgmap v8971: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 458 KiB/s wr, 19 op/s 2023-09-10T03:54:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16520.252553 secs 2023-09-10T03:54:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16520.252553 secs 2023-09-10T03:54:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16520.252553 secs 2023-09-10T03:54:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16520.963617 secs 2023-09-10T03:54:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:50 smithi098 ceph-mon[103124]: pgmap v8972: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 244 KiB/s wr, 13 op/s 2023-09-10T03:54:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16520.963617 secs 2023-09-10T03:54:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:50 smithi170 ceph-mon[102832]: pgmap v8972: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 244 KiB/s wr, 13 op/s 2023-09-10T03:54:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16520.963617 secs 2023-09-10T03:54:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:50 smithi082 ceph-mon[98694]: pgmap v8972: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 244 KiB/s wr, 13 op/s 2023-09-10T03:54:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:52 smithi098 ceph-mon[103124]: pgmap v8973: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 334 KiB/s wr, 18 op/s 2023-09-10T03:54:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:52 smithi170 ceph-mon[102832]: pgmap v8973: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 334 KiB/s wr, 18 op/s 2023-09-10T03:54:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:52 smithi082 ceph-mon[98694]: pgmap v8973: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 334 KiB/s wr, 18 op/s 2023-09-10T03:54:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16524.045076 secs 2023-09-10T03:54:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16524.045076 secs 2023-09-10T03:54:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16524.045076 secs 2023-09-10T03:54:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16525.252611 secs 2023-09-10T03:54:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:54 smithi098 ceph-mon[103124]: pgmap v8974: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T03:54:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16525.252611 secs 2023-09-10T03:54:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:54 smithi170 ceph-mon[102832]: pgmap v8974: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T03:54:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16525.252611 secs 2023-09-10T03:54:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:54 smithi082 ceph-mon[98694]: pgmap v8974: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T03:54:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16525.963684 secs 2023-09-10T03:54:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16525.963684 secs 2023-09-10T03:54:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16525.963684 secs 2023-09-10T03:54:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:56 smithi098 ceph-mon[103124]: pgmap v8975: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T03:54:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:54:56.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:54:56.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:54:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:56 smithi170 ceph-mon[102832]: pgmap v8975: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T03:54:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:54:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:54:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:54:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:56 smithi082 ceph-mon[98694]: pgmap v8975: 97 pgs: 97 active+clean; 9.5 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T03:54:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:54:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:54:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:54:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:58 smithi098 ceph-mon[103124]: pgmap v8976: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 288 KiB/s wr, 17 op/s 2023-09-10T03:54:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16529.045139 secs 2023-09-10T03:54:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:58 smithi170 ceph-mon[102832]: pgmap v8976: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 288 KiB/s wr, 17 op/s 2023-09-10T03:54:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16529.045139 secs 2023-09-10T03:54:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:58 smithi082 ceph-mon[98694]: pgmap v8976: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 288 KiB/s wr, 17 op/s 2023-09-10T03:54:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16529.045139 secs 2023-09-10T03:54:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:54:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16530.252669 secs 2023-09-10T03:54:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:54:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16530.252669 secs 2023-09-10T03:54:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:54:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16530.252669 secs 2023-09-10T03:55:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16530.963750 secs 2023-09-10T03:55:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:00 smithi098 ceph-mon[103124]: pgmap v8977: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 229 KiB/s wr, 11 op/s 2023-09-10T03:55:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16530.963750 secs 2023-09-10T03:55:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:00 smithi170 ceph-mon[102832]: pgmap v8977: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 229 KiB/s wr, 11 op/s 2023-09-10T03:55:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16530.963750 secs 2023-09-10T03:55:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:00 smithi082 ceph-mon[98694]: pgmap v8977: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 229 KiB/s wr, 11 op/s 2023-09-10T03:55:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:02 smithi098 ceph-mon[103124]: pgmap v8978: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 279 KiB/s wr, 18 op/s 2023-09-10T03:55:02.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:02 smithi170 ceph-mon[102832]: pgmap v8978: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 279 KiB/s wr, 18 op/s 2023-09-10T03:55:02.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:02 smithi082 ceph-mon[98694]: pgmap v8978: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 279 KiB/s wr, 18 op/s 2023-09-10T03:55:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16534.045236 secs 2023-09-10T03:55:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16534.045236 secs 2023-09-10T03:55:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16534.045236 secs 2023-09-10T03:55:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16535.252727 secs 2023-09-10T03:55:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:04 smithi098 ceph-mon[103124]: pgmap v8979: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 190 KiB/s wr, 12 op/s 2023-09-10T03:55:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16535.252727 secs 2023-09-10T03:55:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:04 smithi170 ceph-mon[102832]: pgmap v8979: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 190 KiB/s wr, 12 op/s 2023-09-10T03:55:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16535.252727 secs 2023-09-10T03:55:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:04 smithi082 ceph-mon[98694]: pgmap v8979: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 190 KiB/s wr, 12 op/s 2023-09-10T03:55:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16535.963815 secs 2023-09-10T03:55:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16535.963815 secs 2023-09-10T03:55:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16535.963815 secs 2023-09-10T03:55:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:06 smithi170 ceph-mon[102832]: pgmap v8980: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 190 KiB/s wr, 12 op/s 2023-09-10T03:55:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:06 smithi082 ceph-mon[98694]: pgmap v8980: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 190 KiB/s wr, 12 op/s 2023-09-10T03:55:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:06 smithi098 ceph-mon[103124]: pgmap v8980: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 190 KiB/s wr, 12 op/s 2023-09-10T03:55:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:08 smithi170 ceph-mon[102832]: pgmap v8981: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 247 KiB/s wr, 31 op/s 2023-09-10T03:55:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16539.045298 secs 2023-09-10T03:55:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:08 smithi082 ceph-mon[98694]: pgmap v8981: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 247 KiB/s wr, 31 op/s 2023-09-10T03:55:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16539.045298 secs 2023-09-10T03:55:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:08 smithi098 ceph-mon[103124]: pgmap v8981: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 247 KiB/s wr, 31 op/s 2023-09-10T03:55:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16539.045298 secs 2023-09-10T03:55:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16540.252785 secs 2023-09-10T03:55:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:09 smithi170 ceph-mon[102832]: osdmap e629: 12 total, 12 up, 12 in 2023-09-10T03:55:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16540.252785 secs 2023-09-10T03:55:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:09 smithi082 ceph-mon[98694]: osdmap e629: 12 total, 12 up, 12 in 2023-09-10T03:55:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16540.252785 secs 2023-09-10T03:55:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:09 smithi098 ceph-mon[103124]: osdmap e629: 12 total, 12 up, 12 in 2023-09-10T03:55:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16540.963879 secs 2023-09-10T03:55:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:10 smithi170 ceph-mon[102832]: pgmap v8983: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 129 KiB/s wr, 30 op/s 2023-09-10T03:55:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16540.963879 secs 2023-09-10T03:55:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:10 smithi082 ceph-mon[98694]: pgmap v8983: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 129 KiB/s wr, 30 op/s 2023-09-10T03:55:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16540.963879 secs 2023-09-10T03:55:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:10 smithi098 ceph-mon[103124]: pgmap v8983: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 129 KiB/s wr, 30 op/s 2023-09-10T03:55:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:12 smithi170 ceph-mon[102832]: pgmap v8984: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 69 KiB/s wr, 28 op/s 2023-09-10T03:55:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:12 smithi082 ceph-mon[98694]: pgmap v8984: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 69 KiB/s wr, 28 op/s 2023-09-10T03:55:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:12 smithi098 ceph-mon[103124]: pgmap v8984: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 69 KiB/s wr, 28 op/s 2023-09-10T03:55:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16544.045359 secs 2023-09-10T03:55:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:13 smithi170 ceph-mon[102832]: osdmap e630: 12 total, 12 up, 12 in 2023-09-10T03:55:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16544.045359 secs 2023-09-10T03:55:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:13 smithi082 ceph-mon[98694]: osdmap e630: 12 total, 12 up, 12 in 2023-09-10T03:55:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16544.045359 secs 2023-09-10T03:55:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:13 smithi098 ceph-mon[103124]: osdmap e630: 12 total, 12 up, 12 in 2023-09-10T03:55:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16545.252870 secs 2023-09-10T03:55:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:14 smithi170 ceph-mon[102832]: pgmap v8986: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 87 KiB/s wr, 35 op/s 2023-09-10T03:55:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16545.252870 secs 2023-09-10T03:55:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:14 smithi082 ceph-mon[98694]: pgmap v8986: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 87 KiB/s wr, 35 op/s 2023-09-10T03:55:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16545.252870 secs 2023-09-10T03:55:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:14 smithi098 ceph-mon[103124]: pgmap v8986: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 87 KiB/s wr, 35 op/s 2023-09-10T03:55:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16545.964024 secs 2023-09-10T03:55:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16545.964024 secs 2023-09-10T03:55:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16545.964024 secs 2023-09-10T03:55:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:16 smithi170 ceph-mon[102832]: pgmap v8987: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T03:55:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:16 smithi082 ceph-mon[98694]: pgmap v8987: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T03:55:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:16 smithi098 ceph-mon[103124]: pgmap v8987: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T03:55:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:17 smithi170 ceph-mon[102832]: pgmap v8988: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 13 op/s 2023-09-10T03:55:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:17 smithi082 ceph-mon[98694]: pgmap v8988: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 13 op/s 2023-09-10T03:55:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:17 smithi098 ceph-mon[103124]: pgmap v8988: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 13 op/s 2023-09-10T03:55:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16549.045420 secs 2023-09-10T03:55:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16549.045420 secs 2023-09-10T03:55:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16549.045420 secs 2023-09-10T03:55:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16550.252930 secs 2023-09-10T03:55:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:19 smithi170 ceph-mon[102832]: pgmap v8989: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T03:55:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16550.252930 secs 2023-09-10T03:55:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:19 smithi082 ceph-mon[98694]: pgmap v8989: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T03:55:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16550.252930 secs 2023-09-10T03:55:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:19 smithi098 ceph-mon[103124]: pgmap v8989: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T03:55:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16550.964088 secs 2023-09-10T03:55:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16550.964088 secs 2023-09-10T03:55:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16550.964088 secs 2023-09-10T03:55:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:21 smithi170 ceph-mon[102832]: pgmap v8990: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T03:55:21.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:21 smithi082 ceph-mon[98694]: pgmap v8990: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T03:55:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:21 smithi098 ceph-mon[103124]: pgmap v8990: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T03:55:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:55:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:55:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:55:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:55:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:55:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:55:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16555.252994 secs 2023-09-10T03:55:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:23 smithi170 ceph-mon[102832]: pgmap v8991: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T03:55:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:55:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:55:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:55:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16555.252994 secs 2023-09-10T03:55:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:23 smithi082 ceph-mon[98694]: pgmap v8991: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T03:55:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:55:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:55:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:55:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16555.252994 secs 2023-09-10T03:55:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:23 smithi098 ceph-mon[103124]: pgmap v8991: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 11 op/s 2023-09-10T03:55:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16555.964153 secs 2023-09-10T03:55:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16555.964153 secs 2023-09-10T03:55:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16555.964153 secs 2023-09-10T03:55:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:26 smithi170 ceph-mon[102832]: pgmap v8992: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 9 op/s 2023-09-10T03:55:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:26 smithi082 ceph-mon[98694]: pgmap v8992: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 9 op/s 2023-09-10T03:55:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:26 smithi098 ceph-mon[103124]: pgmap v8992: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 9 op/s 2023-09-10T03:55:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:28 smithi098 ceph-mon[103124]: pgmap v8993: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 186 KiB/s wr, 14 op/s 2023-09-10T03:55:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:28 smithi170 ceph-mon[102832]: pgmap v8993: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 186 KiB/s wr, 14 op/s 2023-09-10T03:55:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:28 smithi082 ceph-mon[98694]: pgmap v8993: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 186 KiB/s wr, 14 op/s 2023-09-10T03:55:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16560.253051 secs 2023-09-10T03:55:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16560.253051 secs 2023-09-10T03:55:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16560.253051 secs 2023-09-10T03:55:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16560.964219 secs 2023-09-10T03:55:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:30 smithi098 ceph-mon[103124]: pgmap v8994: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 130 KiB/s wr, 9 op/s 2023-09-10T03:55:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16560.964219 secs 2023-09-10T03:55:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:30 smithi170 ceph-mon[102832]: pgmap v8994: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 130 KiB/s wr, 9 op/s 2023-09-10T03:55:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16560.964219 secs 2023-09-10T03:55:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:30 smithi082 ceph-mon[98694]: pgmap v8994: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 130 KiB/s wr, 9 op/s 2023-09-10T03:55:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:32 smithi098 ceph-mon[103124]: pgmap v8995: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 202 KiB/s wr, 14 op/s 2023-09-10T03:55:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:32 smithi170 ceph-mon[102832]: pgmap v8995: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 202 KiB/s wr, 14 op/s 2023-09-10T03:55:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:32 smithi082 ceph-mon[98694]: pgmap v8995: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 202 KiB/s wr, 14 op/s 2023-09-10T03:55:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16565.253111 secs 2023-09-10T03:55:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:34 smithi170 ceph-mon[102832]: pgmap v8996: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 202 KiB/s wr, 10 op/s 2023-09-10T03:55:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16565.253111 secs 2023-09-10T03:55:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:34 smithi082 ceph-mon[98694]: pgmap v8996: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 202 KiB/s wr, 10 op/s 2023-09-10T03:55:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16565.253111 secs 2023-09-10T03:55:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:34 smithi098 ceph-mon[103124]: pgmap v8996: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 202 KiB/s wr, 10 op/s 2023-09-10T03:55:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16565.964284 secs 2023-09-10T03:55:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16565.964284 secs 2023-09-10T03:55:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16565.964284 secs 2023-09-10T03:55:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:36 smithi170 ceph-mon[102832]: pgmap v8997: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 202 KiB/s wr, 10 op/s 2023-09-10T03:55:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:36 smithi082 ceph-mon[98694]: pgmap v8997: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 202 KiB/s wr, 10 op/s 2023-09-10T03:55:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:36 smithi098 ceph-mon[103124]: pgmap v8997: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 202 KiB/s wr, 10 op/s 2023-09-10T03:55:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:38 smithi098 ceph-mon[103124]: pgmap v8998: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 269 KiB/s wr, 16 op/s 2023-09-10T03:55:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:55:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:38 smithi170 ceph-mon[102832]: pgmap v8998: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 269 KiB/s wr, 16 op/s 2023-09-10T03:55:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:55:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:38 smithi082 ceph-mon[98694]: pgmap v8998: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 269 KiB/s wr, 16 op/s 2023-09-10T03:55:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:55:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16570.253175 secs 2023-09-10T03:55:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:55:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:55:39.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:55:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16570.253175 secs 2023-09-10T03:55:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:55:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:55:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:55:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16570.253175 secs 2023-09-10T03:55:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:55:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:55:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:55:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16570.964349 secs 2023-09-10T03:55:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:40 smithi098 ceph-mon[103124]: pgmap v8999: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T03:55:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16570.964349 secs 2023-09-10T03:55:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:40 smithi170 ceph-mon[102832]: pgmap v8999: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T03:55:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16570.964349 secs 2023-09-10T03:55:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:40 smithi082 ceph-mon[98694]: pgmap v8999: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T03:55:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:42 smithi098 ceph-mon[103124]: pgmap v9000: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 340 KiB/s wr, 18 op/s 2023-09-10T03:55:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:42 smithi170 ceph-mon[102832]: pgmap v9000: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 340 KiB/s wr, 18 op/s 2023-09-10T03:55:42.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:42 smithi082 ceph-mon[98694]: pgmap v9000: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 340 KiB/s wr, 18 op/s 2023-09-10T03:55:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16575.253249 secs 2023-09-10T03:55:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:44 smithi170 ceph-mon[102832]: pgmap v9001: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T03:55:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16575.253249 secs 2023-09-10T03:55:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:44 smithi082 ceph-mon[98694]: pgmap v9001: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T03:55:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16575.253249 secs 2023-09-10T03:55:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:44 smithi098 ceph-mon[103124]: pgmap v9001: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T03:55:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16575.964413 secs 2023-09-10T03:55:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16575.964413 secs 2023-09-10T03:55:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16575.964413 secs 2023-09-10T03:55:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:46 smithi170 ceph-mon[102832]: pgmap v9002: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T03:55:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:46 smithi082 ceph-mon[98694]: pgmap v9002: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T03:55:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:46 smithi098 ceph-mon[103124]: pgmap v9002: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T03:55:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:48 smithi098 ceph-mon[103124]: pgmap v9003: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 297 KiB/s wr, 18 op/s 2023-09-10T03:55:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16579.045795 secs 2023-09-10T03:55:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:48 smithi170 ceph-mon[102832]: pgmap v9003: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 297 KiB/s wr, 18 op/s 2023-09-10T03:55:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16579.045795 secs 2023-09-10T03:55:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:48 smithi082 ceph-mon[98694]: pgmap v9003: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 297 KiB/s wr, 18 op/s 2023-09-10T03:55:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16579.045795 secs 2023-09-10T03:55:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16580.253332 secs 2023-09-10T03:55:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16580.253332 secs 2023-09-10T03:55:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16580.253332 secs 2023-09-10T03:55:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16580.964484 secs 2023-09-10T03:55:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:50 smithi098 ceph-mon[103124]: pgmap v9004: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T03:55:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16580.964484 secs 2023-09-10T03:55:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:50 smithi170 ceph-mon[102832]: pgmap v9004: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T03:55:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16580.964484 secs 2023-09-10T03:55:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:50 smithi082 ceph-mon[98694]: pgmap v9004: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T03:55:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:51 smithi170 ceph-mon[102832]: pgmap v9005: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 452 KiB/s wr, 18 op/s 2023-09-10T03:55:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:55:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:55:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:55:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:51 smithi082 ceph-mon[98694]: pgmap v9005: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 452 KiB/s wr, 18 op/s 2023-09-10T03:55:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:55:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:55:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:55:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:51 smithi098 ceph-mon[103124]: pgmap v9005: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 452 KiB/s wr, 18 op/s 2023-09-10T03:55:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:55:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:55:51.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:55:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16584.045856 secs 2023-09-10T03:55:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16584.045856 secs 2023-09-10T03:55:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16584.045856 secs 2023-09-10T03:55:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16585.253390 secs 2023-09-10T03:55:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:54 smithi170 ceph-mon[102832]: pgmap v9006: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 251 KiB/s wr, 12 op/s 2023-09-10T03:55:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16585.253390 secs 2023-09-10T03:55:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:54 smithi082 ceph-mon[98694]: pgmap v9006: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 251 KiB/s wr, 12 op/s 2023-09-10T03:55:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16585.253390 secs 2023-09-10T03:55:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:54 smithi098 ceph-mon[103124]: pgmap v9006: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 251 KiB/s wr, 12 op/s 2023-09-10T03:55:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16585.964551 secs 2023-09-10T03:55:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16585.964551 secs 2023-09-10T03:55:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16585.964551 secs 2023-09-10T03:55:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:56 smithi170 ceph-mon[102832]: pgmap v9007: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 251 KiB/s wr, 12 op/s 2023-09-10T03:55:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:56 smithi082 ceph-mon[98694]: pgmap v9007: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 251 KiB/s wr, 12 op/s 2023-09-10T03:55:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:56 smithi098 ceph-mon[103124]: pgmap v9007: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 251 KiB/s wr, 12 op/s 2023-09-10T03:55:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:58 smithi098 ceph-mon[103124]: pgmap v9008: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 304 KiB/s wr, 17 op/s 2023-09-10T03:55:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16589.045920 secs 2023-09-10T03:55:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:58 smithi170 ceph-mon[102832]: pgmap v9008: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 304 KiB/s wr, 17 op/s 2023-09-10T03:55:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16589.045920 secs 2023-09-10T03:55:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:58 smithi082 ceph-mon[98694]: pgmap v9008: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 304 KiB/s wr, 17 op/s 2023-09-10T03:55:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16589.045920 secs 2023-09-10T03:55:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:55:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16590.253451 secs 2023-09-10T03:55:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:55:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16590.253451 secs 2023-09-10T03:55:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:55:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16590.253451 secs 2023-09-10T03:56:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16590.964617 secs 2023-09-10T03:56:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:00 smithi098 ceph-mon[103124]: pgmap v9009: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 276 KiB/s wr, 12 op/s 2023-09-10T03:56:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16590.964617 secs 2023-09-10T03:56:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:00 smithi170 ceph-mon[102832]: pgmap v9009: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 276 KiB/s wr, 12 op/s 2023-09-10T03:56:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16590.964617 secs 2023-09-10T03:56:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:00 smithi082 ceph-mon[98694]: pgmap v9009: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 276 KiB/s wr, 12 op/s 2023-09-10T03:56:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:02 smithi098 ceph-mon[103124]: pgmap v9010: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 406 KiB/s wr, 17 op/s 2023-09-10T03:56:02.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:02 smithi170 ceph-mon[102832]: pgmap v9010: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 406 KiB/s wr, 17 op/s 2023-09-10T03:56:02.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:02 smithi082 ceph-mon[98694]: pgmap v9010: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 406 KiB/s wr, 17 op/s 2023-09-10T03:56:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16594.046026 secs 2023-09-10T03:56:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16594.046026 secs 2023-09-10T03:56:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16594.046026 secs 2023-09-10T03:56:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16595.253516 secs 2023-09-10T03:56:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:04 smithi098 ceph-mon[103124]: pgmap v9011: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 184 KiB/s wr, 10 op/s 2023-09-10T03:56:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16595.253516 secs 2023-09-10T03:56:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:04 smithi170 ceph-mon[102832]: pgmap v9011: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 184 KiB/s wr, 10 op/s 2023-09-10T03:56:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16595.253516 secs 2023-09-10T03:56:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:04 smithi082 ceph-mon[98694]: pgmap v9011: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 184 KiB/s wr, 10 op/s 2023-09-10T03:56:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16595.964680 secs 2023-09-10T03:56:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16595.964680 secs 2023-09-10T03:56:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16595.964680 secs 2023-09-10T03:56:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:06 smithi170 ceph-mon[102832]: pgmap v9012: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 184 KiB/s wr, 10 op/s 2023-09-10T03:56:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:06 smithi082 ceph-mon[98694]: pgmap v9012: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 184 KiB/s wr, 10 op/s 2023-09-10T03:56:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:06 smithi098 ceph-mon[103124]: pgmap v9012: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 184 KiB/s wr, 10 op/s 2023-09-10T03:56:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:08 smithi170 ceph-mon[102832]: pgmap v9013: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 230 KiB/s wr, 30 op/s 2023-09-10T03:56:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16599.046087 secs 2023-09-10T03:56:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:08 smithi082 ceph-mon[98694]: pgmap v9013: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 230 KiB/s wr, 30 op/s 2023-09-10T03:56:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16599.046087 secs 2023-09-10T03:56:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:08 smithi098 ceph-mon[103124]: pgmap v9013: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 230 KiB/s wr, 30 op/s 2023-09-10T03:56:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16599.046087 secs 2023-09-10T03:56:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16600.253577 secs 2023-09-10T03:56:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:09 smithi170 ceph-mon[102832]: osdmap e631: 12 total, 12 up, 12 in 2023-09-10T03:56:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16600.253577 secs 2023-09-10T03:56:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:09 smithi082 ceph-mon[98694]: osdmap e631: 12 total, 12 up, 12 in 2023-09-10T03:56:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16600.253577 secs 2023-09-10T03:56:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:09 smithi098 ceph-mon[103124]: osdmap e631: 12 total, 12 up, 12 in 2023-09-10T03:56:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16600.964769 secs 2023-09-10T03:56:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:10 smithi170 ceph-mon[102832]: pgmap v9015: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 211 KiB/s wr, 29 op/s 2023-09-10T03:56:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16600.964769 secs 2023-09-10T03:56:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:10 smithi082 ceph-mon[98694]: pgmap v9015: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 211 KiB/s wr, 29 op/s 2023-09-10T03:56:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16600.964769 secs 2023-09-10T03:56:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:10 smithi098 ceph-mon[103124]: pgmap v9015: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 211 KiB/s wr, 29 op/s 2023-09-10T03:56:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:12 smithi170 ceph-mon[102832]: pgmap v9016: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 28 op/s 2023-09-10T03:56:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:12 smithi082 ceph-mon[98694]: pgmap v9016: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 28 op/s 2023-09-10T03:56:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:12 smithi098 ceph-mon[103124]: pgmap v9016: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 28 op/s 2023-09-10T03:56:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16604.046202 secs 2023-09-10T03:56:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:13 smithi170 ceph-mon[102832]: osdmap e632: 12 total, 12 up, 12 in 2023-09-10T03:56:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16604.046202 secs 2023-09-10T03:56:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:13 smithi082 ceph-mon[98694]: osdmap e632: 12 total, 12 up, 12 in 2023-09-10T03:56:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16604.046202 secs 2023-09-10T03:56:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:13 smithi098 ceph-mon[103124]: osdmap e632: 12 total, 12 up, 12 in 2023-09-10T03:56:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16605.253635 secs 2023-09-10T03:56:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:14 smithi170 ceph-mon[102832]: pgmap v9018: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 136 KiB/s wr, 36 op/s 2023-09-10T03:56:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16605.253635 secs 2023-09-10T03:56:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:14 smithi082 ceph-mon[98694]: pgmap v9018: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 136 KiB/s wr, 36 op/s 2023-09-10T03:56:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16605.253635 secs 2023-09-10T03:56:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:14 smithi098 ceph-mon[103124]: pgmap v9018: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 136 KiB/s wr, 36 op/s 2023-09-10T03:56:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16605.964832 secs 2023-09-10T03:56:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16605.964832 secs 2023-09-10T03:56:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16605.964832 secs 2023-09-10T03:56:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:16 smithi170 ceph-mon[102832]: pgmap v9019: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 7 op/s 2023-09-10T03:56:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:16 smithi082 ceph-mon[98694]: pgmap v9019: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 7 op/s 2023-09-10T03:56:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:16 smithi098 ceph-mon[103124]: pgmap v9019: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 7 op/s 2023-09-10T03:56:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:18 smithi170 ceph-mon[102832]: pgmap v9020: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 14 op/s 2023-09-10T03:56:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16609.046263 secs 2023-09-10T03:56:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:18 smithi082 ceph-mon[98694]: pgmap v9020: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 14 op/s 2023-09-10T03:56:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16609.046263 secs 2023-09-10T03:56:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:18 smithi098 ceph-mon[103124]: pgmap v9020: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 14 op/s 2023-09-10T03:56:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16609.046263 secs 2023-09-10T03:56:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16610.253699 secs 2023-09-10T03:56:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16610.253699 secs 2023-09-10T03:56:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16610.253699 secs 2023-09-10T03:56:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16610.964900 secs 2023-09-10T03:56:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:20 smithi170 ceph-mon[102832]: pgmap v9021: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T03:56:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16610.964900 secs 2023-09-10T03:56:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:20 smithi082 ceph-mon[98694]: pgmap v9021: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T03:56:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16610.964900 secs 2023-09-10T03:56:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:20 smithi098 ceph-mon[103124]: pgmap v9021: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T03:56:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:21 smithi170 ceph-mon[102832]: pgmap v9022: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-10T03:56:21.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:21 smithi082 ceph-mon[98694]: pgmap v9022: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-10T03:56:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:21 smithi098 ceph-mon[103124]: pgmap v9022: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-10T03:56:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:56:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:56:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:56:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:56:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:56:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:56:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16615.253782 secs 2023-09-10T03:56:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:23 smithi170 ceph-mon[102832]: pgmap v9023: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-10T03:56:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:56:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:56:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:56:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16615.253782 secs 2023-09-10T03:56:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:23 smithi082 ceph-mon[98694]: pgmap v9023: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-10T03:56:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:56:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:56:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:56:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16615.253782 secs 2023-09-10T03:56:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:23 smithi098 ceph-mon[103124]: pgmap v9023: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 12 op/s 2023-09-10T03:56:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16615.964965 secs 2023-09-10T03:56:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16615.964965 secs 2023-09-10T03:56:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16615.964965 secs 2023-09-10T03:56:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:26 smithi170 ceph-mon[102832]: pgmap v9024: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 10 op/s 2023-09-10T03:56:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:26 smithi082 ceph-mon[98694]: pgmap v9024: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 10 op/s 2023-09-10T03:56:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:26 smithi098 ceph-mon[103124]: pgmap v9024: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 10 op/s 2023-09-10T03:56:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:28 smithi098 ceph-mon[103124]: pgmap v9025: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 169 KiB/s wr, 16 op/s 2023-09-10T03:56:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:28 smithi170 ceph-mon[102832]: pgmap v9025: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 169 KiB/s wr, 16 op/s 2023-09-10T03:56:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:28 smithi082 ceph-mon[98694]: pgmap v9025: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 169 KiB/s wr, 16 op/s 2023-09-10T03:56:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16620.253840 secs 2023-09-10T03:56:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16620.253840 secs 2023-09-10T03:56:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16620.253840 secs 2023-09-10T03:56:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16620.965032 secs 2023-09-10T03:56:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:30 smithi170 ceph-mon[102832]: pgmap v9026: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T03:56:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16620.965032 secs 2023-09-10T03:56:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:30 smithi082 ceph-mon[98694]: pgmap v9026: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T03:56:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16620.965032 secs 2023-09-10T03:56:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:30 smithi098 ceph-mon[103124]: pgmap v9026: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T03:56:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:32 smithi170 ceph-mon[102832]: pgmap v9027: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 267 KiB/s wr, 16 op/s 2023-09-10T03:56:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:32 smithi082 ceph-mon[98694]: pgmap v9027: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 267 KiB/s wr, 16 op/s 2023-09-10T03:56:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:32 smithi098 ceph-mon[103124]: pgmap v9027: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 267 KiB/s wr, 16 op/s 2023-09-10T03:56:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16625.253898 secs 2023-09-10T03:56:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:34 smithi170 ceph-mon[102832]: pgmap v9028: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 267 KiB/s wr, 11 op/s 2023-09-10T03:56:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16625.253898 secs 2023-09-10T03:56:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:34 smithi082 ceph-mon[98694]: pgmap v9028: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 267 KiB/s wr, 11 op/s 2023-09-10T03:56:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16625.253898 secs 2023-09-10T03:56:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:34 smithi098 ceph-mon[103124]: pgmap v9028: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 267 KiB/s wr, 11 op/s 2023-09-10T03:56:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16625.965097 secs 2023-09-10T03:56:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16625.965097 secs 2023-09-10T03:56:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16625.965097 secs 2023-09-10T03:56:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:36 smithi082 ceph-mon[98694]: pgmap v9029: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 267 KiB/s wr, 11 op/s 2023-09-10T03:56:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:36 smithi098 ceph-mon[103124]: pgmap v9029: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 267 KiB/s wr, 11 op/s 2023-09-10T03:56:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:36 smithi170 ceph-mon[102832]: pgmap v9029: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 267 KiB/s wr, 11 op/s 2023-09-10T03:56:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:38 smithi098 ceph-mon[103124]: pgmap v9030: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 289 KiB/s wr, 17 op/s 2023-09-10T03:56:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:38 smithi170 ceph-mon[102832]: pgmap v9030: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 289 KiB/s wr, 17 op/s 2023-09-10T03:56:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:38 smithi082 ceph-mon[98694]: pgmap v9030: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 289 KiB/s wr, 17 op/s 2023-09-10T03:56:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16630.253955 secs 2023-09-10T03:56:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:56:39.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:56:39.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:56:39.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:56:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16630.253955 secs 2023-09-10T03:56:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:56:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:56:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:56:39.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:56:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16630.253955 secs 2023-09-10T03:56:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:56:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:56:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:56:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:56:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16630.965168 secs 2023-09-10T03:56:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:40 smithi098 ceph-mon[103124]: pgmap v9031: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T03:56:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16630.965168 secs 2023-09-10T03:56:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:40 smithi170 ceph-mon[102832]: pgmap v9031: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T03:56:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16630.965168 secs 2023-09-10T03:56:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:40 smithi082 ceph-mon[98694]: pgmap v9031: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T03:56:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:42 smithi098 ceph-mon[103124]: pgmap v9032: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 480 KiB/s wr, 18 op/s 2023-09-10T03:56:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:42 smithi170 ceph-mon[102832]: pgmap v9032: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 480 KiB/s wr, 18 op/s 2023-09-10T03:56:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:42 smithi082 ceph-mon[98694]: pgmap v9032: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 480 KiB/s wr, 18 op/s 2023-09-10T03:56:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16635.254014 secs 2023-09-10T03:56:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:44 smithi170 ceph-mon[102832]: pgmap v9033: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 325 KiB/s wr, 12 op/s 2023-09-10T03:56:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16635.254014 secs 2023-09-10T03:56:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:44 smithi082 ceph-mon[98694]: pgmap v9033: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 325 KiB/s wr, 12 op/s 2023-09-10T03:56:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16635.254014 secs 2023-09-10T03:56:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:44 smithi098 ceph-mon[103124]: pgmap v9033: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 325 KiB/s wr, 12 op/s 2023-09-10T03:56:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16635.965248 secs 2023-09-10T03:56:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16635.965248 secs 2023-09-10T03:56:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16635.965248 secs 2023-09-10T03:56:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:46 smithi082 ceph-mon[98694]: pgmap v9034: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 325 KiB/s wr, 12 op/s 2023-09-10T03:56:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:46 smithi098 ceph-mon[103124]: pgmap v9034: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 325 KiB/s wr, 12 op/s 2023-09-10T03:56:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:46 smithi170 ceph-mon[102832]: pgmap v9034: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 325 KiB/s wr, 12 op/s 2023-09-10T03:56:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:48 smithi098 ceph-mon[103124]: pgmap v9035: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 366 KiB/s wr, 17 op/s 2023-09-10T03:56:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:48 smithi170 ceph-mon[102832]: pgmap v9035: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 366 KiB/s wr, 17 op/s 2023-09-10T03:56:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:48 smithi082 ceph-mon[98694]: pgmap v9035: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 366 KiB/s wr, 17 op/s 2023-09-10T03:56:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16640.254074 secs 2023-09-10T03:56:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16640.254074 secs 2023-09-10T03:56:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16640.254074 secs 2023-09-10T03:56:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16640.965312 secs 2023-09-10T03:56:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:50 smithi170 ceph-mon[102832]: pgmap v9036: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 344 KiB/s wr, 12 op/s 2023-09-10T03:56:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16640.965312 secs 2023-09-10T03:56:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:50 smithi082 ceph-mon[98694]: pgmap v9036: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 344 KiB/s wr, 12 op/s 2023-09-10T03:56:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16640.965312 secs 2023-09-10T03:56:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:50 smithi098 ceph-mon[103124]: pgmap v9036: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 344 KiB/s wr, 12 op/s 2023-09-10T03:56:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:52 smithi170 ceph-mon[102832]: pgmap v9037: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 446 KiB/s wr, 17 op/s 2023-09-10T03:56:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:52 smithi082 ceph-mon[98694]: pgmap v9037: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 446 KiB/s wr, 17 op/s 2023-09-10T03:56:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:52 smithi098 ceph-mon[103124]: pgmap v9037: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 446 KiB/s wr, 17 op/s 2023-09-10T03:56:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16644.046756 secs 2023-09-10T03:56:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16644.046756 secs 2023-09-10T03:56:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16644.046756 secs 2023-09-10T03:56:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16645.254131 secs 2023-09-10T03:56:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:54 smithi170 ceph-mon[102832]: pgmap v9038: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 10 op/s 2023-09-10T03:56:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16645.254131 secs 2023-09-10T03:56:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:54 smithi082 ceph-mon[98694]: pgmap v9038: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 10 op/s 2023-09-10T03:56:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16645.254131 secs 2023-09-10T03:56:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:54 smithi098 ceph-mon[103124]: pgmap v9038: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 10 op/s 2023-09-10T03:56:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16645.965379 secs 2023-09-10T03:56:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16645.965379 secs 2023-09-10T03:56:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16645.965379 secs 2023-09-10T03:56:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:56 smithi170 ceph-mon[102832]: pgmap v9039: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 10 op/s 2023-09-10T03:56:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:56:56.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:56:56.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:56:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:56 smithi082 ceph-mon[98694]: pgmap v9039: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 10 op/s 2023-09-10T03:56:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:56:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:56:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:56:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:56 smithi098 ceph-mon[103124]: pgmap v9039: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 10 op/s 2023-09-10T03:56:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:56:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:56:56.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:56:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:58 smithi170 ceph-mon[102832]: pgmap v9040: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 262 KiB/s wr, 16 op/s 2023-09-10T03:56:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16649.046805 secs 2023-09-10T03:56:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:58 smithi082 ceph-mon[98694]: pgmap v9040: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 262 KiB/s wr, 16 op/s 2023-09-10T03:56:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16649.046805 secs 2023-09-10T03:56:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:58 smithi098 ceph-mon[103124]: pgmap v9040: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 262 KiB/s wr, 16 op/s 2023-09-10T03:56:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16649.046805 secs 2023-09-10T03:56:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:56:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16650.254189 secs 2023-09-10T03:56:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:56:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16650.254189 secs 2023-09-10T03:56:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:56:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16650.254189 secs 2023-09-10T03:57:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16650.965444 secs 2023-09-10T03:57:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:00 smithi170 ceph-mon[102832]: pgmap v9041: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 221 KiB/s wr, 11 op/s 2023-09-10T03:57:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16650.965444 secs 2023-09-10T03:57:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:00 smithi082 ceph-mon[98694]: pgmap v9041: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 221 KiB/s wr, 11 op/s 2023-09-10T03:57:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16650.965444 secs 2023-09-10T03:57:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:00 smithi098 ceph-mon[103124]: pgmap v9041: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 221 KiB/s wr, 11 op/s 2023-09-10T03:57:02.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:02 smithi170 ceph-mon[102832]: pgmap v9042: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 316 KiB/s wr, 17 op/s 2023-09-10T03:57:02.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:02 smithi082 ceph-mon[98694]: pgmap v9042: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 316 KiB/s wr, 17 op/s 2023-09-10T03:57:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:02 smithi098 ceph-mon[103124]: pgmap v9042: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 316 KiB/s wr, 17 op/s 2023-09-10T03:57:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16654.046917 secs 2023-09-10T03:57:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16654.046917 secs 2023-09-10T03:57:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16654.046917 secs 2023-09-10T03:57:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16655.254247 secs 2023-09-10T03:57:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:04 smithi170 ceph-mon[102832]: pgmap v9043: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 214 KiB/s wr, 11 op/s 2023-09-10T03:57:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16655.254247 secs 2023-09-10T03:57:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:04 smithi082 ceph-mon[98694]: pgmap v9043: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 214 KiB/s wr, 11 op/s 2023-09-10T03:57:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16655.254247 secs 2023-09-10T03:57:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:04 smithi098 ceph-mon[103124]: pgmap v9043: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 214 KiB/s wr, 11 op/s 2023-09-10T03:57:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16655.965488 secs 2023-09-10T03:57:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16655.965488 secs 2023-09-10T03:57:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16655.965488 secs 2023-09-10T03:57:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:06 smithi170 ceph-mon[102832]: pgmap v9044: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 214 KiB/s wr, 11 op/s 2023-09-10T03:57:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:06 smithi082 ceph-mon[98694]: pgmap v9044: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 214 KiB/s wr, 11 op/s 2023-09-10T03:57:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:06 smithi098 ceph-mon[103124]: pgmap v9044: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 214 KiB/s wr, 11 op/s 2023-09-10T03:57:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:08 smithi170 ceph-mon[102832]: pgmap v9045: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 272 KiB/s wr, 31 op/s 2023-09-10T03:57:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16659.046998 secs 2023-09-10T03:57:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:08 smithi082 ceph-mon[98694]: pgmap v9045: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 272 KiB/s wr, 31 op/s 2023-09-10T03:57:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16659.046998 secs 2023-09-10T03:57:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:08 smithi098 ceph-mon[103124]: pgmap v9045: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 272 KiB/s wr, 31 op/s 2023-09-10T03:57:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16659.046998 secs 2023-09-10T03:57:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16660.254307 secs 2023-09-10T03:57:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:09 smithi170 ceph-mon[102832]: osdmap e633: 12 total, 12 up, 12 in 2023-09-10T03:57:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16660.254307 secs 2023-09-10T03:57:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:09 smithi082 ceph-mon[98694]: osdmap e633: 12 total, 12 up, 12 in 2023-09-10T03:57:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16660.254307 secs 2023-09-10T03:57:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:09 smithi098 ceph-mon[103124]: osdmap e633: 12 total, 12 up, 12 in 2023-09-10T03:57:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16660.965554 secs 2023-09-10T03:57:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:10 smithi170 ceph-mon[102832]: pgmap v9047: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 184 KiB/s wr, 30 op/s 2023-09-10T03:57:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16660.965554 secs 2023-09-10T03:57:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:10 smithi082 ceph-mon[98694]: pgmap v9047: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 184 KiB/s wr, 30 op/s 2023-09-10T03:57:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16660.965554 secs 2023-09-10T03:57:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:10 smithi098 ceph-mon[103124]: pgmap v9047: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 184 KiB/s wr, 30 op/s 2023-09-10T03:57:11.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:11 smithi170 ceph-mon[102832]: pgmap v9048: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 137 KiB/s wr, 29 op/s 2023-09-10T03:57:11.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:11 smithi082 ceph-mon[98694]: pgmap v9048: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 137 KiB/s wr, 29 op/s 2023-09-10T03:57:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:11 smithi098 ceph-mon[103124]: pgmap v9048: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 137 KiB/s wr, 29 op/s 2023-09-10T03:57:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16664.047110 secs 2023-09-10T03:57:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16664.047110 secs 2023-09-10T03:57:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16664.047110 secs 2023-09-10T03:57:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16665.254365 secs 2023-09-10T03:57:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:14 smithi082 ceph-mon[98694]: osdmap e634: 12 total, 12 up, 12 in 2023-09-10T03:57:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:14 smithi082 ceph-mon[98694]: pgmap v9050: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 171 KiB/s wr, 37 op/s 2023-09-10T03:57:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16665.254365 secs 2023-09-10T03:57:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:14 smithi098 ceph-mon[103124]: osdmap e634: 12 total, 12 up, 12 in 2023-09-10T03:57:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:14 smithi098 ceph-mon[103124]: pgmap v9050: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 171 KiB/s wr, 37 op/s 2023-09-10T03:57:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16665.254365 secs 2023-09-10T03:57:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:14 smithi170 ceph-mon[102832]: osdmap e634: 12 total, 12 up, 12 in 2023-09-10T03:57:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:14 smithi170 ceph-mon[102832]: pgmap v9050: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 171 KiB/s wr, 37 op/s 2023-09-10T03:57:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16665.965619 secs 2023-09-10T03:57:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16665.965619 secs 2023-09-10T03:57:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16665.965619 secs 2023-09-10T03:57:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:16 smithi082 ceph-mon[98694]: pgmap v9051: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 85 KiB/s wr, 7 op/s 2023-09-10T03:57:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:16 smithi098 ceph-mon[103124]: pgmap v9051: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 85 KiB/s wr, 7 op/s 2023-09-10T03:57:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:16 smithi170 ceph-mon[102832]: pgmap v9051: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 85 KiB/s wr, 7 op/s 2023-09-10T03:57:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:18 smithi170 ceph-mon[102832]: pgmap v9052: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 13 op/s 2023-09-10T03:57:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16669.047226 secs 2023-09-10T03:57:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:18 smithi082 ceph-mon[98694]: pgmap v9052: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 13 op/s 2023-09-10T03:57:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16669.047226 secs 2023-09-10T03:57:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:18 smithi098 ceph-mon[103124]: pgmap v9052: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 13 op/s 2023-09-10T03:57:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16669.047226 secs 2023-09-10T03:57:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16670.254428 secs 2023-09-10T03:57:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16670.254428 secs 2023-09-10T03:57:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16670.254428 secs 2023-09-10T03:57:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16670.965686 secs 2023-09-10T03:57:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:20 smithi170 ceph-mon[102832]: pgmap v9053: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T03:57:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16670.965686 secs 2023-09-10T03:57:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:20 smithi082 ceph-mon[98694]: pgmap v9053: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T03:57:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16670.965686 secs 2023-09-10T03:57:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:20 smithi098 ceph-mon[103124]: pgmap v9053: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T03:57:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:22 smithi170 ceph-mon[102832]: pgmap v9054: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T03:57:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:22 smithi082 ceph-mon[98694]: pgmap v9054: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T03:57:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:22 smithi098 ceph-mon[103124]: pgmap v9054: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T03:57:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:57:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:57:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:57:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16675.254498 secs 2023-09-10T03:57:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:24 smithi170 ceph-mon[102832]: pgmap v9055: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T03:57:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:57:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:57:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:57:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16675.254498 secs 2023-09-10T03:57:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:24 smithi082 ceph-mon[98694]: pgmap v9055: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T03:57:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:57:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:57:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:57:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16675.254498 secs 2023-09-10T03:57:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:24 smithi098 ceph-mon[103124]: pgmap v9055: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T03:57:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:57:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:57:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:57:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16675.965750 secs 2023-09-10T03:57:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16675.965750 secs 2023-09-10T03:57:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16675.965750 secs 2023-09-10T03:57:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:26 smithi170 ceph-mon[102832]: pgmap v9056: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 9 op/s 2023-09-10T03:57:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:26 smithi082 ceph-mon[98694]: pgmap v9056: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 9 op/s 2023-09-10T03:57:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:26 smithi098 ceph-mon[103124]: pgmap v9056: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 9 op/s 2023-09-10T03:57:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:28 smithi170 ceph-mon[102832]: pgmap v9057: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 146 KiB/s wr, 14 op/s 2023-09-10T03:57:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:28 smithi082 ceph-mon[98694]: pgmap v9057: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 146 KiB/s wr, 14 op/s 2023-09-10T03:57:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:28 smithi098 ceph-mon[103124]: pgmap v9057: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 146 KiB/s wr, 14 op/s 2023-09-10T03:57:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16680.254560 secs 2023-09-10T03:57:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16680.254560 secs 2023-09-10T03:57:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16680.254560 secs 2023-09-10T03:57:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16680.965815 secs 2023-09-10T03:57:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:30 smithi170 ceph-mon[102832]: pgmap v9058: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 10 op/s 2023-09-10T03:57:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16680.965815 secs 2023-09-10T03:57:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:30 smithi082 ceph-mon[98694]: pgmap v9058: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 10 op/s 2023-09-10T03:57:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16680.965815 secs 2023-09-10T03:57:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:30 smithi098 ceph-mon[103124]: pgmap v9058: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 10 op/s 2023-09-10T03:57:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:32 smithi170 ceph-mon[102832]: pgmap v9059: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 218 KiB/s wr, 15 op/s 2023-09-10T03:57:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:32 smithi082 ceph-mon[98694]: pgmap v9059: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 218 KiB/s wr, 15 op/s 2023-09-10T03:57:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:32 smithi098 ceph-mon[103124]: pgmap v9059: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 218 KiB/s wr, 15 op/s 2023-09-10T03:57:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16685.254661 secs 2023-09-10T03:57:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:34 smithi082 ceph-mon[98694]: pgmap v9060: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 10 op/s 2023-09-10T03:57:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16685.254661 secs 2023-09-10T03:57:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:34 smithi098 ceph-mon[103124]: pgmap v9060: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 10 op/s 2023-09-10T03:57:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16685.254661 secs 2023-09-10T03:57:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:34 smithi170 ceph-mon[102832]: pgmap v9060: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 10 op/s 2023-09-10T03:57:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16685.965883 secs 2023-09-10T03:57:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16685.965883 secs 2023-09-10T03:57:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16685.965883 secs 2023-09-10T03:57:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:36 smithi082 ceph-mon[98694]: pgmap v9061: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 10 op/s 2023-09-10T03:57:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:36 smithi098 ceph-mon[103124]: pgmap v9061: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 10 op/s 2023-09-10T03:57:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:36 smithi170 ceph-mon[102832]: pgmap v9061: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 10 op/s 2023-09-10T03:57:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:38 smithi170 ceph-mon[102832]: pgmap v9062: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 391 KiB/s wr, 17 op/s 2023-09-10T03:57:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:38 smithi082 ceph-mon[98694]: pgmap v9062: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 391 KiB/s wr, 17 op/s 2023-09-10T03:57:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:38 smithi098 ceph-mon[103124]: pgmap v9062: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 391 KiB/s wr, 17 op/s 2023-09-10T03:57:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16690.254720 secs 2023-09-10T03:57:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:57:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16690.254720 secs 2023-09-10T03:57:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:57:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16690.254720 secs 2023-09-10T03:57:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:57:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16690.965950 secs 2023-09-10T03:57:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:40 smithi170 ceph-mon[102832]: pgmap v9063: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 317 KiB/s wr, 12 op/s 2023-09-10T03:57:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:57:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:57:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:57:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16690.965950 secs 2023-09-10T03:57:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:40 smithi082 ceph-mon[98694]: pgmap v9063: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 317 KiB/s wr, 12 op/s 2023-09-10T03:57:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:57:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:57:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:57:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16690.965950 secs 2023-09-10T03:57:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:40 smithi098 ceph-mon[103124]: pgmap v9063: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 317 KiB/s wr, 12 op/s 2023-09-10T03:57:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:57:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:57:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:57:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:42 smithi170 ceph-mon[102832]: pgmap v9064: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 478 KiB/s wr, 18 op/s 2023-09-10T03:57:42.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:42 smithi082 ceph-mon[98694]: pgmap v9064: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 478 KiB/s wr, 18 op/s 2023-09-10T03:57:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:42 smithi098 ceph-mon[103124]: pgmap v9064: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 478 KiB/s wr, 18 op/s 2023-09-10T03:57:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16695.254778 secs 2023-09-10T03:57:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:44 smithi082 ceph-mon[98694]: pgmap v9065: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 405 KiB/s wr, 13 op/s 2023-09-10T03:57:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16695.254778 secs 2023-09-10T03:57:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:44 smithi098 ceph-mon[103124]: pgmap v9065: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 405 KiB/s wr, 13 op/s 2023-09-10T03:57:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16695.254778 secs 2023-09-10T03:57:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:44 smithi170 ceph-mon[102832]: pgmap v9065: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 405 KiB/s wr, 13 op/s 2023-09-10T03:57:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16695.966016 secs 2023-09-10T03:57:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16695.966016 secs 2023-09-10T03:57:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16695.966016 secs 2023-09-10T03:57:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:46 smithi082 ceph-mon[98694]: pgmap v9066: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 405 KiB/s wr, 13 op/s 2023-09-10T03:57:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:46 smithi098 ceph-mon[103124]: pgmap v9066: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 405 KiB/s wr, 13 op/s 2023-09-10T03:57:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:46 smithi170 ceph-mon[102832]: pgmap v9066: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 405 KiB/s wr, 13 op/s 2023-09-10T03:57:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:48 smithi170 ceph-mon[102832]: pgmap v9067: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 465 KiB/s wr, 18 op/s 2023-09-10T03:57:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:48 smithi082 ceph-mon[98694]: pgmap v9067: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 465 KiB/s wr, 18 op/s 2023-09-10T03:57:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:48 smithi098 ceph-mon[103124]: pgmap v9067: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 465 KiB/s wr, 18 op/s 2023-09-10T03:57:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16700.254836 secs 2023-09-10T03:57:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16700.254836 secs 2023-09-10T03:57:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16700.254836 secs 2023-09-10T03:57:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16700.966082 secs 2023-09-10T03:57:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:50 smithi170 ceph-mon[102832]: pgmap v9068: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 220 KiB/s wr, 12 op/s 2023-09-10T03:57:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16700.966082 secs 2023-09-10T03:57:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:50 smithi082 ceph-mon[98694]: pgmap v9068: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 220 KiB/s wr, 12 op/s 2023-09-10T03:57:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16700.966082 secs 2023-09-10T03:57:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:50 smithi098 ceph-mon[103124]: pgmap v9068: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 220 KiB/s wr, 12 op/s 2023-09-10T03:57:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:52 smithi170 ceph-mon[102832]: pgmap v9069: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 311 KiB/s wr, 17 op/s 2023-09-10T03:57:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:52 smithi082 ceph-mon[98694]: pgmap v9069: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 311 KiB/s wr, 17 op/s 2023-09-10T03:57:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:52 smithi098 ceph-mon[103124]: pgmap v9069: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 311 KiB/s wr, 17 op/s 2023-09-10T03:57:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16704.047836 secs 2023-09-10T03:57:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16704.047836 secs 2023-09-10T03:57:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16704.047836 secs 2023-09-10T03:57:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16705.254894 secs 2023-09-10T03:57:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:54 smithi170 ceph-mon[102832]: pgmap v9070: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T03:57:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16705.254894 secs 2023-09-10T03:57:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:54 smithi082 ceph-mon[98694]: pgmap v9070: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T03:57:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16705.254894 secs 2023-09-10T03:57:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:54 smithi098 ceph-mon[103124]: pgmap v9070: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T03:57:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16705.966149 secs 2023-09-10T03:57:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16705.966149 secs 2023-09-10T03:57:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16705.966149 secs 2023-09-10T03:57:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:56 smithi170 ceph-mon[102832]: pgmap v9071: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T03:57:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:57:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:57:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:57:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:56 smithi082 ceph-mon[98694]: pgmap v9071: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T03:57:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:57:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:57:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:57:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:56 smithi098 ceph-mon[103124]: pgmap v9071: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T03:57:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:57:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:57:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:57:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:58 smithi170 ceph-mon[102832]: pgmap v9072: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 16 op/s 2023-09-10T03:57:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16709.047896 secs 2023-09-10T03:57:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:58 smithi082 ceph-mon[98694]: pgmap v9072: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 16 op/s 2023-09-10T03:57:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16709.047896 secs 2023-09-10T03:57:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:58 smithi098 ceph-mon[103124]: pgmap v9072: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 16 op/s 2023-09-10T03:57:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16709.047896 secs 2023-09-10T03:57:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:57:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16710.254954 secs 2023-09-10T03:57:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:57:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16710.254954 secs 2023-09-10T03:57:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:57:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16710.254954 secs 2023-09-10T03:58:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16710.966215 secs 2023-09-10T03:58:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:00 smithi170 ceph-mon[102832]: pgmap v9073: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 217 KiB/s wr, 11 op/s 2023-09-10T03:58:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16710.966215 secs 2023-09-10T03:58:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:00 smithi082 ceph-mon[98694]: pgmap v9073: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 217 KiB/s wr, 11 op/s 2023-09-10T03:58:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16710.966215 secs 2023-09-10T03:58:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:00 smithi098 ceph-mon[103124]: pgmap v9073: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 217 KiB/s wr, 11 op/s 2023-09-10T03:58:02.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:02 smithi170 ceph-mon[102832]: pgmap v9074: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 267 KiB/s wr, 16 op/s 2023-09-10T03:58:02.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:02 smithi082 ceph-mon[98694]: pgmap v9074: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 267 KiB/s wr, 16 op/s 2023-09-10T03:58:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:02 smithi098 ceph-mon[103124]: pgmap v9074: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 267 KiB/s wr, 16 op/s 2023-09-10T03:58:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16714.048012 secs 2023-09-10T03:58:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16714.048012 secs 2023-09-10T03:58:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16714.048012 secs 2023-09-10T03:58:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16715.255012 secs 2023-09-10T03:58:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:04 smithi170 ceph-mon[102832]: pgmap v9075: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 11 op/s 2023-09-10T03:58:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16715.255012 secs 2023-09-10T03:58:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:04 smithi082 ceph-mon[98694]: pgmap v9075: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 11 op/s 2023-09-10T03:58:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16715.255012 secs 2023-09-10T03:58:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:04 smithi098 ceph-mon[103124]: pgmap v9075: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 11 op/s 2023-09-10T03:58:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16715.966278 secs 2023-09-10T03:58:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16715.966278 secs 2023-09-10T03:58:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16715.966278 secs 2023-09-10T03:58:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:06 smithi170 ceph-mon[102832]: pgmap v9076: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 11 op/s 2023-09-10T03:58:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:06 smithi082 ceph-mon[98694]: pgmap v9076: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 11 op/s 2023-09-10T03:58:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:06 smithi098 ceph-mon[103124]: pgmap v9076: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 11 op/s 2023-09-10T03:58:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:08 smithi170 ceph-mon[102832]: pgmap v9077: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 234 KiB/s wr, 31 op/s 2023-09-10T03:58:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16719.048101 secs 2023-09-10T03:58:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:08 smithi082 ceph-mon[98694]: pgmap v9077: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 234 KiB/s wr, 31 op/s 2023-09-10T03:58:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16719.048101 secs 2023-09-10T03:58:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:08 smithi098 ceph-mon[103124]: pgmap v9077: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 234 KiB/s wr, 31 op/s 2023-09-10T03:58:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16719.048101 secs 2023-09-10T03:58:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16720.255071 secs 2023-09-10T03:58:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:09 smithi170 ceph-mon[102832]: osdmap e635: 12 total, 12 up, 12 in 2023-09-10T03:58:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:09 smithi170 ceph-mon[102832]: pgmap v9079: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 130 KiB/s wr, 30 op/s 2023-09-10T03:58:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16720.255071 secs 2023-09-10T03:58:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:09 smithi082 ceph-mon[98694]: osdmap e635: 12 total, 12 up, 12 in 2023-09-10T03:58:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:09 smithi082 ceph-mon[98694]: pgmap v9079: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 130 KiB/s wr, 30 op/s 2023-09-10T03:58:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16720.255071 secs 2023-09-10T03:58:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:09 smithi098 ceph-mon[103124]: osdmap e635: 12 total, 12 up, 12 in 2023-09-10T03:58:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:09 smithi098 ceph-mon[103124]: pgmap v9079: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 130 KiB/s wr, 30 op/s 2023-09-10T03:58:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16720.966346 secs 2023-09-10T03:58:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16720.966346 secs 2023-09-10T03:58:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16720.966346 secs 2023-09-10T03:58:11.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:11 smithi170 ceph-mon[102832]: pgmap v9080: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-10T03:58:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:11 smithi082 ceph-mon[98694]: pgmap v9080: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-10T03:58:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:11 smithi098 ceph-mon[103124]: pgmap v9080: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 70 KiB/s wr, 29 op/s 2023-09-10T03:58:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16724.048200 secs 2023-09-10T03:58:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16724.048200 secs 2023-09-10T03:58:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16724.048200 secs 2023-09-10T03:58:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16725.255130 secs 2023-09-10T03:58:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:14 smithi082 ceph-mon[98694]: osdmap e636: 12 total, 12 up, 12 in 2023-09-10T03:58:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:14 smithi082 ceph-mon[98694]: pgmap v9082: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 88 KiB/s wr, 36 op/s 2023-09-10T03:58:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16725.255130 secs 2023-09-10T03:58:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:14 smithi098 ceph-mon[103124]: osdmap e636: 12 total, 12 up, 12 in 2023-09-10T03:58:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:14 smithi098 ceph-mon[103124]: pgmap v9082: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 88 KiB/s wr, 36 op/s 2023-09-10T03:58:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16725.255130 secs 2023-09-10T03:58:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:14 smithi170 ceph-mon[102832]: osdmap e636: 12 total, 12 up, 12 in 2023-09-10T03:58:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:14 smithi170 ceph-mon[102832]: pgmap v9082: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 88 KiB/s wr, 36 op/s 2023-09-10T03:58:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16725.966410 secs 2023-09-10T03:58:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16725.966410 secs 2023-09-10T03:58:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16725.966410 secs 2023-09-10T03:58:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:16 smithi098 ceph-mon[103124]: pgmap v9083: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:58:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:16 smithi170 ceph-mon[102832]: pgmap v9083: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:58:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:16 smithi082 ceph-mon[98694]: pgmap v9083: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T03:58:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:18 smithi170 ceph-mon[102832]: pgmap v9084: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 13 op/s 2023-09-10T03:58:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16729.048275 secs 2023-09-10T03:58:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:18 smithi082 ceph-mon[98694]: pgmap v9084: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 13 op/s 2023-09-10T03:58:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16729.048275 secs 2023-09-10T03:58:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:18 smithi098 ceph-mon[103124]: pgmap v9084: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 13 op/s 2023-09-10T03:58:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16729.048275 secs 2023-09-10T03:58:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16730.255209 secs 2023-09-10T03:58:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16730.255209 secs 2023-09-10T03:58:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16730.255209 secs 2023-09-10T03:58:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16730.966483 secs 2023-09-10T03:58:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:20 smithi170 ceph-mon[102832]: pgmap v9085: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T03:58:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16730.966483 secs 2023-09-10T03:58:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:20 smithi082 ceph-mon[98694]: pgmap v9085: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T03:58:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16730.966483 secs 2023-09-10T03:58:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:20 smithi098 ceph-mon[103124]: pgmap v9085: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T03:58:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:22 smithi170 ceph-mon[102832]: pgmap v9086: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T03:58:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:22 smithi082 ceph-mon[98694]: pgmap v9086: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T03:58:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:22 smithi098 ceph-mon[103124]: pgmap v9086: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T03:58:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:58:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:58:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:58:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16735.303125 secs 2023-09-10T03:58:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:24 smithi170 ceph-mon[102832]: pgmap v9087: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T03:58:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:58:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:58:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:58:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16735.303125 secs 2023-09-10T03:58:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:24 smithi082 ceph-mon[98694]: pgmap v9087: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T03:58:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:58:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:58:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:58:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16735.303125 secs 2023-09-10T03:58:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:24 smithi098 ceph-mon[103124]: pgmap v9087: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 11 op/s 2023-09-10T03:58:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:58:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:58:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:58:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16735.966547 secs 2023-09-10T03:58:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16735.966547 secs 2023-09-10T03:58:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16735.966547 secs 2023-09-10T03:58:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:26 smithi170 ceph-mon[102832]: pgmap v9088: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 9 op/s 2023-09-10T03:58:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:26 smithi082 ceph-mon[98694]: pgmap v9088: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 9 op/s 2023-09-10T03:58:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:26 smithi098 ceph-mon[103124]: pgmap v9088: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 9 op/s 2023-09-10T03:58:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:28 smithi170 ceph-mon[102832]: pgmap v9089: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 189 KiB/s wr, 14 op/s 2023-09-10T03:58:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:28 smithi082 ceph-mon[98694]: pgmap v9089: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 189 KiB/s wr, 14 op/s 2023-09-10T03:58:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:28 smithi098 ceph-mon[103124]: pgmap v9089: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 189 KiB/s wr, 14 op/s 2023-09-10T03:58:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16740.303184 secs 2023-09-10T03:58:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16740.303184 secs 2023-09-10T03:58:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16740.303184 secs 2023-09-10T03:58:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16740.966613 secs 2023-09-10T03:58:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:30 smithi170 ceph-mon[102832]: pgmap v9090: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 9 op/s 2023-09-10T03:58:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16740.966613 secs 2023-09-10T03:58:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:30 smithi082 ceph-mon[98694]: pgmap v9090: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 9 op/s 2023-09-10T03:58:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16740.966613 secs 2023-09-10T03:58:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:30 smithi098 ceph-mon[103124]: pgmap v9090: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 9 op/s 2023-09-10T03:58:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:32 smithi170 ceph-mon[102832]: pgmap v9091: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 205 KiB/s wr, 15 op/s 2023-09-10T03:58:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:32 smithi082 ceph-mon[98694]: pgmap v9091: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 205 KiB/s wr, 15 op/s 2023-09-10T03:58:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:32 smithi098 ceph-mon[103124]: pgmap v9091: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 205 KiB/s wr, 15 op/s 2023-09-10T03:58:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16745.303243 secs 2023-09-10T03:58:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:34 smithi082 ceph-mon[98694]: pgmap v9092: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 205 KiB/s wr, 11 op/s 2023-09-10T03:58:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16745.303243 secs 2023-09-10T03:58:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:34 smithi098 ceph-mon[103124]: pgmap v9092: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 205 KiB/s wr, 11 op/s 2023-09-10T03:58:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16745.303243 secs 2023-09-10T03:58:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:34 smithi170 ceph-mon[102832]: pgmap v9092: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 205 KiB/s wr, 11 op/s 2023-09-10T03:58:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16745.966677 secs 2023-09-10T03:58:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16745.966677 secs 2023-09-10T03:58:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16745.966677 secs 2023-09-10T03:58:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:36 smithi098 ceph-mon[103124]: pgmap v9093: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 205 KiB/s wr, 11 op/s 2023-09-10T03:58:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:36 smithi170 ceph-mon[102832]: pgmap v9093: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 205 KiB/s wr, 11 op/s 2023-09-10T03:58:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:36 smithi082 ceph-mon[98694]: pgmap v9093: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 205 KiB/s wr, 11 op/s 2023-09-10T03:58:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:38 smithi170 ceph-mon[102832]: pgmap v9094: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 248 KiB/s wr, 18 op/s 2023-09-10T03:58:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:38 smithi082 ceph-mon[98694]: pgmap v9094: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 248 KiB/s wr, 18 op/s 2023-09-10T03:58:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:38 smithi098 ceph-mon[103124]: pgmap v9094: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 248 KiB/s wr, 18 op/s 2023-09-10T03:58:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16750.303331 secs 2023-09-10T03:58:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16750.303331 secs 2023-09-10T03:58:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16750.303331 secs 2023-09-10T03:58:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16750.966744 secs 2023-09-10T03:58:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:40 smithi170 ceph-mon[102832]: pgmap v9095: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 116 KiB/s wr, 13 op/s 2023-09-10T03:58:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:58:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16750.966744 secs 2023-09-10T03:58:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:40 smithi082 ceph-mon[98694]: pgmap v9095: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 116 KiB/s wr, 13 op/s 2023-09-10T03:58:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:58:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16750.966744 secs 2023-09-10T03:58:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:40 smithi098 ceph-mon[103124]: pgmap v9095: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 116 KiB/s wr, 13 op/s 2023-09-10T03:58:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:58:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:58:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:58:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:58:41.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:58:41.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:58:41.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:58:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:58:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:58:41.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:58:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:42 smithi170 ceph-mon[102832]: pgmap v9096: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 346 KiB/s wr, 20 op/s 2023-09-10T03:58:42.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:42 smithi082 ceph-mon[98694]: pgmap v9096: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 346 KiB/s wr, 20 op/s 2023-09-10T03:58:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:42 smithi098 ceph-mon[103124]: pgmap v9096: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 346 KiB/s wr, 20 op/s 2023-09-10T03:58:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16754.048632 secs 2023-09-10T03:58:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16754.048632 secs 2023-09-10T03:58:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16754.048632 secs 2023-09-10T03:58:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16755.303388 secs 2023-09-10T03:58:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:44 smithi170 ceph-mon[102832]: pgmap v9097: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 273 KiB/s wr, 14 op/s 2023-09-10T03:58:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:58:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:58:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:58:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16755.303388 secs 2023-09-10T03:58:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:44 smithi082 ceph-mon[98694]: pgmap v9097: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 273 KiB/s wr, 14 op/s 2023-09-10T03:58:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:58:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:58:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:58:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16755.303388 secs 2023-09-10T03:58:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:44 smithi098 ceph-mon[103124]: pgmap v9097: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 273 KiB/s wr, 14 op/s 2023-09-10T03:58:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:58:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:58:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:58:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16755.966812 secs 2023-09-10T03:58:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16755.966812 secs 2023-09-10T03:58:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16755.966812 secs 2023-09-10T03:58:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:46 smithi170 ceph-mon[102832]: pgmap v9098: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 273 KiB/s wr, 14 op/s 2023-09-10T03:58:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:46 smithi082 ceph-mon[98694]: pgmap v9098: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 273 KiB/s wr, 14 op/s 2023-09-10T03:58:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:46 smithi098 ceph-mon[103124]: pgmap v9098: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 273 KiB/s wr, 14 op/s 2023-09-10T03:58:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:48 smithi170 ceph-mon[102832]: pgmap v9099: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 464 KiB/s wr, 19 op/s 2023-09-10T03:58:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16759.048725 secs 2023-09-10T03:58:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:48 smithi082 ceph-mon[98694]: pgmap v9099: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 464 KiB/s wr, 19 op/s 2023-09-10T03:58:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16759.048725 secs 2023-09-10T03:58:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:48 smithi098 ceph-mon[103124]: pgmap v9099: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 464 KiB/s wr, 19 op/s 2023-09-10T03:58:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16759.048725 secs 2023-09-10T03:58:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16760.303447 secs 2023-09-10T03:58:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16760.303447 secs 2023-09-10T03:58:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16760.303447 secs 2023-09-10T03:58:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16760.966875 secs 2023-09-10T03:58:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:50 smithi170 ceph-mon[102832]: pgmap v9100: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 422 KiB/s wr, 12 op/s 2023-09-10T03:58:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16760.966875 secs 2023-09-10T03:58:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:50 smithi082 ceph-mon[98694]: pgmap v9100: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 422 KiB/s wr, 12 op/s 2023-09-10T03:58:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16760.966875 secs 2023-09-10T03:58:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:50 smithi098 ceph-mon[103124]: pgmap v9100: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 422 KiB/s wr, 12 op/s 2023-09-10T03:58:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:52 smithi170 ceph-mon[102832]: pgmap v9101: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 620 KiB/s wr, 19 op/s 2023-09-10T03:58:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:52 smithi082 ceph-mon[98694]: pgmap v9101: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 620 KiB/s wr, 19 op/s 2023-09-10T03:58:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:52 smithi098 ceph-mon[103124]: pgmap v9101: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 620 KiB/s wr, 19 op/s 2023-09-10T03:58:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16764.048774 secs 2023-09-10T03:58:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16764.048774 secs 2023-09-10T03:58:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16764.048774 secs 2023-09-10T03:58:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16765.303504 secs 2023-09-10T03:58:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:54 smithi170 ceph-mon[102832]: pgmap v9102: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 390 KiB/s wr, 12 op/s 2023-09-10T03:58:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16765.303504 secs 2023-09-10T03:58:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:54 smithi082 ceph-mon[98694]: pgmap v9102: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 390 KiB/s wr, 12 op/s 2023-09-10T03:58:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16765.303504 secs 2023-09-10T03:58:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:54 smithi098 ceph-mon[103124]: pgmap v9102: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 390 KiB/s wr, 12 op/s 2023-09-10T03:58:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16765.966941 secs 2023-09-10T03:58:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16765.966941 secs 2023-09-10T03:58:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16765.966941 secs 2023-09-10T03:58:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:56 smithi170 ceph-mon[102832]: pgmap v9103: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 390 KiB/s wr, 14 op/s 2023-09-10T03:58:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:56 smithi082 ceph-mon[98694]: pgmap v9103: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 390 KiB/s wr, 14 op/s 2023-09-10T03:58:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:56 smithi098 ceph-mon[103124]: pgmap v9103: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 390 KiB/s wr, 14 op/s 2023-09-10T03:58:57.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:57 smithi170 ceph-mon[102832]: pgmap v9104: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 436 KiB/s wr, 17 op/s 2023-09-10T03:58:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:57 smithi082 ceph-mon[98694]: pgmap v9104: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 436 KiB/s wr, 17 op/s 2023-09-10T03:58:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:57 smithi098 ceph-mon[103124]: pgmap v9104: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 436 KiB/s wr, 17 op/s 2023-09-10T03:58:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16769.048835 secs 2023-09-10T03:58:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16769.048835 secs 2023-09-10T03:58:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16769.048835 secs 2023-09-10T03:58:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16770.303530 secs 2023-09-10T03:58:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:58:59 smithi170 ceph-mon[102832]: pgmap v9105: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 245 KiB/s wr, 11 op/s 2023-09-10T03:58:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16770.303530 secs 2023-09-10T03:58:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:58:59 smithi082 ceph-mon[98694]: pgmap v9105: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 245 KiB/s wr, 11 op/s 2023-09-10T03:58:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16770.303530 secs 2023-09-10T03:58:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:58:59 smithi098 ceph-mon[103124]: pgmap v9105: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 245 KiB/s wr, 11 op/s 2023-09-10T03:59:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16770.967009 secs 2023-09-10T03:59:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16770.967009 secs 2023-09-10T03:59:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16770.967009 secs 2023-09-10T03:59:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:01 smithi098 ceph-mon[103124]: pgmap v9106: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 296 KiB/s wr, 17 op/s 2023-09-10T03:59:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:01 smithi170 ceph-mon[102832]: pgmap v9106: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 296 KiB/s wr, 17 op/s 2023-09-10T03:59:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:01 smithi082 ceph-mon[98694]: pgmap v9106: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 296 KiB/s wr, 17 op/s 2023-09-10T03:59:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16774.048893 secs 2023-09-10T03:59:02.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16774.048893 secs 2023-09-10T03:59:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16774.048893 secs 2023-09-10T03:59:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16775.303616 secs 2023-09-10T03:59:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:04 smithi098 ceph-mon[103124]: pgmap v9107: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 11 op/s 2023-09-10T03:59:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16775.303616 secs 2023-09-10T03:59:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:04 smithi170 ceph-mon[102832]: pgmap v9107: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 11 op/s 2023-09-10T03:59:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16775.303616 secs 2023-09-10T03:59:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:04 smithi082 ceph-mon[98694]: pgmap v9107: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 11 op/s 2023-09-10T03:59:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16775.967098 secs 2023-09-10T03:59:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16775.967098 secs 2023-09-10T03:59:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16775.967098 secs 2023-09-10T03:59:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:06 smithi098 ceph-mon[103124]: pgmap v9108: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 119 KiB/s wr, 24 op/s 2023-09-10T03:59:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:06 smithi170 ceph-mon[102832]: pgmap v9108: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 119 KiB/s wr, 24 op/s 2023-09-10T03:59:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:06 smithi082 ceph-mon[98694]: pgmap v9108: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 119 KiB/s wr, 24 op/s 2023-09-10T03:59:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:08 smithi170 ceph-mon[102832]: pgmap v9109: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 143 KiB/s wr, 28 op/s 2023-09-10T03:59:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16779.048953 secs 2023-09-10T03:59:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:08 smithi082 ceph-mon[98694]: pgmap v9109: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 143 KiB/s wr, 28 op/s 2023-09-10T03:59:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16779.048953 secs 2023-09-10T03:59:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:08 smithi098 ceph-mon[103124]: pgmap v9109: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 143 KiB/s wr, 28 op/s 2023-09-10T03:59:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16779.048953 secs 2023-09-10T03:59:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16780.303675 secs 2023-09-10T03:59:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:09 smithi170 ceph-mon[102832]: osdmap e637: 12 total, 12 up, 12 in 2023-09-10T03:59:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16780.303675 secs 2023-09-10T03:59:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:09 smithi082 ceph-mon[98694]: osdmap e637: 12 total, 12 up, 12 in 2023-09-10T03:59:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16780.303675 secs 2023-09-10T03:59:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:09 smithi098 ceph-mon[103124]: osdmap e637: 12 total, 12 up, 12 in 2023-09-10T03:59:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16780.967165 secs 2023-09-10T03:59:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:10 smithi170 ceph-mon[102832]: pgmap v9111: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 116 KiB/s wr, 30 op/s 2023-09-10T03:59:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16780.967165 secs 2023-09-10T03:59:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:10 smithi082 ceph-mon[98694]: pgmap v9111: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 116 KiB/s wr, 30 op/s 2023-09-10T03:59:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16780.967165 secs 2023-09-10T03:59:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:10 smithi098 ceph-mon[103124]: pgmap v9111: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 116 KiB/s wr, 30 op/s 2023-09-10T03:59:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:12 smithi170 ceph-mon[102832]: pgmap v9112: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 29 op/s 2023-09-10T03:59:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:12 smithi082 ceph-mon[98694]: pgmap v9112: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 29 op/s 2023-09-10T03:59:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:12 smithi098 ceph-mon[103124]: pgmap v9112: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 29 op/s 2023-09-10T03:59:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16784.049045 secs 2023-09-10T03:59:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:13 smithi170 ceph-mon[102832]: osdmap e638: 12 total, 12 up, 12 in 2023-09-10T03:59:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16784.049045 secs 2023-09-10T03:59:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:13 smithi082 ceph-mon[98694]: osdmap e638: 12 total, 12 up, 12 in 2023-09-10T03:59:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16784.049045 secs 2023-09-10T03:59:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:13 smithi098 ceph-mon[103124]: osdmap e638: 12 total, 12 up, 12 in 2023-09-10T03:59:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16785.303735 secs 2023-09-10T03:59:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:14 smithi170 ceph-mon[102832]: pgmap v9114: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 36 KiB/s wr, 16 op/s 2023-09-10T03:59:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16785.303735 secs 2023-09-10T03:59:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:14 smithi082 ceph-mon[98694]: pgmap v9114: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 36 KiB/s wr, 16 op/s 2023-09-10T03:59:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16785.303735 secs 2023-09-10T03:59:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:14 smithi098 ceph-mon[103124]: pgmap v9114: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 36 KiB/s wr, 16 op/s 2023-09-10T03:59:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16785.967229 secs 2023-09-10T03:59:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16785.967229 secs 2023-09-10T03:59:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16785.967229 secs 2023-09-10T03:59:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:16 smithi170 ceph-mon[102832]: pgmap v9115: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 639 B/s wr, 10 op/s 2023-09-10T03:59:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:16 smithi082 ceph-mon[98694]: pgmap v9115: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 639 B/s wr, 10 op/s 2023-09-10T03:59:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:16 smithi098 ceph-mon[103124]: pgmap v9115: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 639 B/s wr, 10 op/s 2023-09-10T03:59:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:17 smithi170 ceph-mon[102832]: pgmap v9116: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 14 op/s 2023-09-10T03:59:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:17 smithi082 ceph-mon[98694]: pgmap v9116: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 14 op/s 2023-09-10T03:59:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:17 smithi098 ceph-mon[103124]: pgmap v9116: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 14 op/s 2023-09-10T03:59:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16789.049106 secs 2023-09-10T03:59:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16789.049106 secs 2023-09-10T03:59:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16789.049106 secs 2023-09-10T03:59:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16790.303797 secs 2023-09-10T03:59:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:19 smithi170 ceph-mon[102832]: pgmap v9117: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T03:59:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16790.303797 secs 2023-09-10T03:59:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:19 smithi082 ceph-mon[98694]: pgmap v9117: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T03:59:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16790.303797 secs 2023-09-10T03:59:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:19 smithi098 ceph-mon[103124]: pgmap v9117: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T03:59:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16790.967294 secs 2023-09-10T03:59:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16790.967294 secs 2023-09-10T03:59:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16790.967294 secs 2023-09-10T03:59:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:21 smithi170 ceph-mon[102832]: pgmap v9118: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T03:59:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:21 smithi082 ceph-mon[98694]: pgmap v9118: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T03:59:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:21 smithi098 ceph-mon[103124]: pgmap v9118: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T03:59:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:59:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:59:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T03:59:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:59:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:59:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:59:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16795.303855 secs 2023-09-10T03:59:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:23 smithi170 ceph-mon[102832]: pgmap v9119: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T03:59:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:59:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:59:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:59:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16795.303855 secs 2023-09-10T03:59:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:23 smithi082 ceph-mon[98694]: pgmap v9119: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T03:59:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:59:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:59:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:59:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16795.303855 secs 2023-09-10T03:59:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:23 smithi098 ceph-mon[103124]: pgmap v9119: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T03:59:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16795.967358 secs 2023-09-10T03:59:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16795.967358 secs 2023-09-10T03:59:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16795.967358 secs 2023-09-10T03:59:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:26 smithi098 ceph-mon[103124]: pgmap v9120: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T03:59:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:26 smithi170 ceph-mon[102832]: pgmap v9120: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T03:59:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:26 smithi082 ceph-mon[98694]: pgmap v9120: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 58 KiB/s wr, 12 op/s 2023-09-10T03:59:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:28 smithi170 ceph-mon[102832]: pgmap v9121: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 172 KiB/s wr, 13 op/s 2023-09-10T03:59:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:28 smithi082 ceph-mon[98694]: pgmap v9121: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 172 KiB/s wr, 13 op/s 2023-09-10T03:59:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:28 smithi098 ceph-mon[103124]: pgmap v9121: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 172 KiB/s wr, 13 op/s 2023-09-10T03:59:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16800.303913 secs 2023-09-10T03:59:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16800.303913 secs 2023-09-10T03:59:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16800.303913 secs 2023-09-10T03:59:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16800.967432 secs 2023-09-10T03:59:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:30 smithi170 ceph-mon[102832]: pgmap v9122: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 10 op/s 2023-09-10T03:59:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16800.967432 secs 2023-09-10T03:59:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:30 smithi082 ceph-mon[98694]: pgmap v9122: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 10 op/s 2023-09-10T03:59:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16800.967432 secs 2023-09-10T03:59:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:30 smithi098 ceph-mon[103124]: pgmap v9122: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 10 op/s 2023-09-10T03:59:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:32 smithi170 ceph-mon[102832]: pgmap v9123: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 272 KiB/s wr, 16 op/s 2023-09-10T03:59:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:32 smithi082 ceph-mon[98694]: pgmap v9123: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 272 KiB/s wr, 16 op/s 2023-09-10T03:59:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:32 smithi098 ceph-mon[103124]: pgmap v9123: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 272 KiB/s wr, 16 op/s 2023-09-10T03:59:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16805.303971 secs 2023-09-10T03:59:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:34 smithi098 ceph-mon[103124]: pgmap v9124: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T03:59:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16805.303971 secs 2023-09-10T03:59:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:34 smithi170 ceph-mon[102832]: pgmap v9124: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T03:59:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16805.303971 secs 2023-09-10T03:59:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:34 smithi082 ceph-mon[98694]: pgmap v9124: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T03:59:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16805.967490 secs 2023-09-10T03:59:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16805.967490 secs 2023-09-10T03:59:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16805.967490 secs 2023-09-10T03:59:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:36 smithi098 ceph-mon[103124]: pgmap v9125: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 272 KiB/s wr, 13 op/s 2023-09-10T03:59:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:36 smithi170 ceph-mon[102832]: pgmap v9125: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 272 KiB/s wr, 13 op/s 2023-09-10T03:59:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:36 smithi082 ceph-mon[98694]: pgmap v9125: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 272 KiB/s wr, 13 op/s 2023-09-10T03:59:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:38 smithi170 ceph-mon[102832]: pgmap v9126: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 299 KiB/s wr, 15 op/s 2023-09-10T03:59:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16809.049394 secs 2023-09-10T03:59:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:38 smithi082 ceph-mon[98694]: pgmap v9126: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 299 KiB/s wr, 15 op/s 2023-09-10T03:59:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16809.049394 secs 2023-09-10T03:59:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:38 smithi098 ceph-mon[103124]: pgmap v9126: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 299 KiB/s wr, 15 op/s 2023-09-10T03:59:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16809.049394 secs 2023-09-10T03:59:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16810.304055 secs 2023-09-10T03:59:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16810.304055 secs 2023-09-10T03:59:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16810.304055 secs 2023-09-10T03:59:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16810.967555 secs 2023-09-10T03:59:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:40 smithi170 ceph-mon[102832]: pgmap v9127: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T03:59:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:59:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:59:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:59:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16810.967555 secs 2023-09-10T03:59:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:40 smithi082 ceph-mon[98694]: pgmap v9127: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T03:59:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:59:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:59:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:59:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16810.967555 secs 2023-09-10T03:59:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:40 smithi098 ceph-mon[103124]: pgmap v9127: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T03:59:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T03:59:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T03:59:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T03:59:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:59:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:59:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:59:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:59:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:59:41.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:59:41.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:59:41.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:59:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T03:59:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T03:59:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T03:59:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T03:59:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:42 smithi170 ceph-mon[102832]: pgmap v9128: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 489 KiB/s wr, 18 op/s 2023-09-10T03:59:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:42 smithi082 ceph-mon[98694]: pgmap v9128: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 489 KiB/s wr, 18 op/s 2023-09-10T03:59:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:42 smithi098 ceph-mon[103124]: pgmap v9128: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 489 KiB/s wr, 18 op/s 2023-09-10T03:59:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16814.049453 secs 2023-09-10T03:59:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16814.049453 secs 2023-09-10T03:59:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16814.049453 secs 2023-09-10T03:59:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16815.304110 secs 2023-09-10T03:59:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:44 smithi170 ceph-mon[102832]: pgmap v9129: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 332 KiB/s wr, 11 op/s 2023-09-10T03:59:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16815.304110 secs 2023-09-10T03:59:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:44 smithi082 ceph-mon[98694]: pgmap v9129: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 332 KiB/s wr, 11 op/s 2023-09-10T03:59:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16815.304110 secs 2023-09-10T03:59:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:44 smithi098 ceph-mon[103124]: pgmap v9129: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 332 KiB/s wr, 11 op/s 2023-09-10T03:59:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16815.967617 secs 2023-09-10T03:59:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16815.967617 secs 2023-09-10T03:59:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16815.967617 secs 2023-09-10T03:59:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:46 smithi170 ceph-mon[102832]: pgmap v9130: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 384 KiB/s wr, 14 op/s 2023-09-10T03:59:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:46 smithi082 ceph-mon[98694]: pgmap v9130: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 384 KiB/s wr, 14 op/s 2023-09-10T03:59:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:46 smithi098 ceph-mon[103124]: pgmap v9130: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 384 KiB/s wr, 14 op/s 2023-09-10T03:59:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:48 smithi170 ceph-mon[102832]: pgmap v9131: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 420 KiB/s wr, 15 op/s 2023-09-10T03:59:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16819.049525 secs 2023-09-10T03:59:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:48 smithi082 ceph-mon[98694]: pgmap v9131: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 420 KiB/s wr, 15 op/s 2023-09-10T03:59:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16819.049525 secs 2023-09-10T03:59:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:48 smithi098 ceph-mon[103124]: pgmap v9131: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 420 KiB/s wr, 15 op/s 2023-09-10T03:59:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16819.049525 secs 2023-09-10T03:59:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16820.304205 secs 2023-09-10T03:59:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16820.304205 secs 2023-09-10T03:59:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16820.304205 secs 2023-09-10T03:59:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16820.967681 secs 2023-09-10T03:59:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:50 smithi170 ceph-mon[102832]: pgmap v9132: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 392 KiB/s wr, 12 op/s 2023-09-10T03:59:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16820.967681 secs 2023-09-10T03:59:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:50 smithi082 ceph-mon[98694]: pgmap v9132: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 392 KiB/s wr, 12 op/s 2023-09-10T03:59:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16820.967681 secs 2023-09-10T03:59:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:50 smithi098 ceph-mon[103124]: pgmap v9132: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 392 KiB/s wr, 12 op/s 2023-09-10T03:59:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:51 smithi170 ceph-mon[102832]: pgmap v9133: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 451 KiB/s wr, 17 op/s 2023-09-10T03:59:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:51 smithi082 ceph-mon[98694]: pgmap v9133: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 451 KiB/s wr, 17 op/s 2023-09-10T03:59:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:51 smithi098 ceph-mon[103124]: pgmap v9133: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 451 KiB/s wr, 17 op/s 2023-09-10T03:59:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16824.049584 secs 2023-09-10T03:59:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16824.049584 secs 2023-09-10T03:59:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16824.049584 secs 2023-09-10T03:59:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16825.304285 secs 2023-09-10T03:59:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:54 smithi098 ceph-mon[103124]: pgmap v9134: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T03:59:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16825.304285 secs 2023-09-10T03:59:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:54 smithi170 ceph-mon[102832]: pgmap v9134: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T03:59:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16825.304285 secs 2023-09-10T03:59:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:54 smithi082 ceph-mon[98694]: pgmap v9134: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T03:59:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16825.967747 secs 2023-09-10T03:59:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16825.967747 secs 2023-09-10T03:59:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16825.967747 secs 2023-09-10T03:59:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:56 smithi098 ceph-mon[103124]: pgmap v9135: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T03:59:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:56 smithi170 ceph-mon[102832]: pgmap v9135: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T03:59:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:56 smithi082 ceph-mon[98694]: pgmap v9135: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T03:59:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:58 smithi170 ceph-mon[102832]: pgmap v9136: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 214 KiB/s wr, 14 op/s 2023-09-10T03:59:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16829.049727 secs 2023-09-10T03:59:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:58 smithi082 ceph-mon[98694]: pgmap v9136: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 214 KiB/s wr, 14 op/s 2023-09-10T03:59:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16829.049727 secs 2023-09-10T03:59:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:58 smithi098 ceph-mon[103124]: pgmap v9136: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 214 KiB/s wr, 14 op/s 2023-09-10T03:59:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16829.049727 secs 2023-09-10T03:59:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 03:59:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16830.304343 secs 2023-09-10T03:59:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 03:59:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16830.304343 secs 2023-09-10T03:59:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 03:59:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16830.304343 secs 2023-09-10T04:00:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16830.967812 secs 2023-09-10T04:00:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:00 smithi170 ceph-mon[102832]: pgmap v9137: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 11 op/s 2023-09-10T04:00:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T04:00:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16830.967812 secs 2023-09-10T04:00:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:00 smithi082 ceph-mon[98694]: pgmap v9137: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 11 op/s 2023-09-10T04:00:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T04:00:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16830.967812 secs 2023-09-10T04:00:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:00 smithi098 ceph-mon[103124]: pgmap v9137: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 11 op/s 2023-09-10T04:00:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T04:00:02.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:02 smithi170 ceph-mon[102832]: pgmap v9138: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 276 KiB/s wr, 30 op/s 2023-09-10T04:00:02.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:02 smithi082 ceph-mon[98694]: pgmap v9138: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 276 KiB/s wr, 30 op/s 2023-09-10T04:00:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:02 smithi098 ceph-mon[103124]: pgmap v9138: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 276 KiB/s wr, 30 op/s 2023-09-10T04:00:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16834.049777 secs 2023-09-10T04:00:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16834.049777 secs 2023-09-10T04:00:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16834.049777 secs 2023-09-10T04:00:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16835.304431 secs 2023-09-10T04:00:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:04 smithi170 ceph-mon[102832]: pgmap v9139: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 217 KiB/s wr, 25 op/s 2023-09-10T04:00:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16835.304431 secs 2023-09-10T04:00:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:04 smithi082 ceph-mon[98694]: pgmap v9139: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 217 KiB/s wr, 25 op/s 2023-09-10T04:00:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16835.304431 secs 2023-09-10T04:00:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:04 smithi098 ceph-mon[103124]: pgmap v9139: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 217 KiB/s wr, 25 op/s 2023-09-10T04:00:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16835.967879 secs 2023-09-10T04:00:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16835.967879 secs 2023-09-10T04:00:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16835.967879 secs 2023-09-10T04:00:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:06 smithi170 ceph-mon[102832]: pgmap v9140: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 217 KiB/s wr, 26 op/s 2023-09-10T04:00:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:06 smithi082 ceph-mon[98694]: pgmap v9140: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 217 KiB/s wr, 26 op/s 2023-09-10T04:00:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:06 smithi098 ceph-mon[103124]: pgmap v9140: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 217 KiB/s wr, 26 op/s 2023-09-10T04:00:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:08 smithi170 ceph-mon[102832]: pgmap v9141: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 230 KiB/s wr, 27 op/s 2023-09-10T04:00:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16839.049837 secs 2023-09-10T04:00:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:08 smithi082 ceph-mon[98694]: pgmap v9141: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 230 KiB/s wr, 27 op/s 2023-09-10T04:00:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16839.049837 secs 2023-09-10T04:00:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:08 smithi098 ceph-mon[103124]: pgmap v9141: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 230 KiB/s wr, 27 op/s 2023-09-10T04:00:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16839.049837 secs 2023-09-10T04:00:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16840.304507 secs 2023-09-10T04:00:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:09 smithi170 ceph-mon[102832]: osdmap e639: 12 total, 12 up, 12 in 2023-09-10T04:00:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:09 smithi170 ceph-mon[102832]: pgmap v9143: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 131 KiB/s wr, 29 op/s 2023-09-10T04:00:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16840.304507 secs 2023-09-10T04:00:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:09 smithi082 ceph-mon[98694]: osdmap e639: 12 total, 12 up, 12 in 2023-09-10T04:00:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:09 smithi082 ceph-mon[98694]: pgmap v9143: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 131 KiB/s wr, 29 op/s 2023-09-10T04:00:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16840.304507 secs 2023-09-10T04:00:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:09 smithi098 ceph-mon[103124]: osdmap e639: 12 total, 12 up, 12 in 2023-09-10T04:00:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:09 smithi098 ceph-mon[103124]: pgmap v9143: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 131 KiB/s wr, 29 op/s 2023-09-10T04:00:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16840.967944 secs 2023-09-10T04:00:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16840.967944 secs 2023-09-10T04:00:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16840.967944 secs 2023-09-10T04:00:11.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:11 smithi170 ceph-mon[102832]: pgmap v9144: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 12 op/s 2023-09-10T04:00:11.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:11 smithi082 ceph-mon[98694]: pgmap v9144: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 12 op/s 2023-09-10T04:00:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:11 smithi098 ceph-mon[103124]: pgmap v9144: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 12 op/s 2023-09-10T04:00:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16844.049924 secs 2023-09-10T04:00:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16844.049924 secs 2023-09-10T04:00:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16844.049924 secs 2023-09-10T04:00:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16845.304559 secs 2023-09-10T04:00:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:14 smithi098 ceph-mon[103124]: osdmap e640: 12 total, 12 up, 12 in 2023-09-10T04:00:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:14 smithi098 ceph-mon[103124]: pgmap v9146: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T04:00:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16845.304559 secs 2023-09-10T04:00:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:14 smithi170 ceph-mon[102832]: osdmap e640: 12 total, 12 up, 12 in 2023-09-10T04:00:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:14 smithi170 ceph-mon[102832]: pgmap v9146: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T04:00:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16845.304559 secs 2023-09-10T04:00:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:14 smithi082 ceph-mon[98694]: osdmap e640: 12 total, 12 up, 12 in 2023-09-10T04:00:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:14 smithi082 ceph-mon[98694]: pgmap v9146: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T04:00:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16845.968010 secs 2023-09-10T04:00:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16845.968010 secs 2023-09-10T04:00:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16845.968010 secs 2023-09-10T04:00:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:16 smithi098 ceph-mon[103124]: pgmap v9147: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T04:00:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:16 smithi170 ceph-mon[102832]: pgmap v9147: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T04:00:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:16 smithi082 ceph-mon[98694]: pgmap v9147: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T04:00:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:18 smithi170 ceph-mon[102832]: pgmap v9148: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 13 op/s 2023-09-10T04:00:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16849.050067 secs 2023-09-10T04:00:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:18 smithi082 ceph-mon[98694]: pgmap v9148: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 13 op/s 2023-09-10T04:00:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16849.050067 secs 2023-09-10T04:00:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:18 smithi098 ceph-mon[103124]: pgmap v9148: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 13 op/s 2023-09-10T04:00:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16849.050067 secs 2023-09-10T04:00:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16850.304642 secs 2023-09-10T04:00:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16850.304642 secs 2023-09-10T04:00:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16850.304642 secs 2023-09-10T04:00:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16850.968073 secs 2023-09-10T04:00:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:20 smithi170 ceph-mon[102832]: pgmap v9149: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T04:00:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16850.968073 secs 2023-09-10T04:00:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:20 smithi082 ceph-mon[98694]: pgmap v9149: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T04:00:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16850.968073 secs 2023-09-10T04:00:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:20 smithi098 ceph-mon[103124]: pgmap v9149: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T04:00:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:22 smithi170 ceph-mon[102832]: pgmap v9150: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T04:00:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:22 smithi082 ceph-mon[98694]: pgmap v9150: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T04:00:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:22 smithi098 ceph-mon[103124]: pgmap v9150: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T04:00:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:23 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:00:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:23 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:00:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:23 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:00:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16855.304712 secs 2023-09-10T04:00:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:24 smithi170 ceph-mon[102832]: pgmap v9151: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T04:00:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:24 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:00:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:00:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:00:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16855.304712 secs 2023-09-10T04:00:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:24 smithi082 ceph-mon[98694]: pgmap v9151: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T04:00:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:24 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:00:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:00:24.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:00:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16855.304712 secs 2023-09-10T04:00:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:24 smithi098 ceph-mon[103124]: pgmap v9151: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T04:00:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:24 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:00:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:00:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:00:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16855.968139 secs 2023-09-10T04:00:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:25 smithi170 ceph-mon[102832]: pgmap v9152: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 12 op/s 2023-09-10T04:00:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16855.968139 secs 2023-09-10T04:00:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:25 smithi082 ceph-mon[98694]: pgmap v9152: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 12 op/s 2023-09-10T04:00:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16855.968139 secs 2023-09-10T04:00:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:25 smithi098 ceph-mon[103124]: pgmap v9152: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 12 op/s 2023-09-10T04:00:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:28 smithi170 ceph-mon[102832]: pgmap v9153: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 13 op/s 2023-09-10T04:00:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:28 smithi082 ceph-mon[98694]: pgmap v9153: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 13 op/s 2023-09-10T04:00:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:28 smithi098 ceph-mon[103124]: pgmap v9153: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 13 op/s 2023-09-10T04:00:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16860.304794 secs 2023-09-10T04:00:29.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16860.304794 secs 2023-09-10T04:00:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16860.304794 secs 2023-09-10T04:00:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16860.968203 secs 2023-09-10T04:00:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:30 smithi170 ceph-mon[102832]: pgmap v9154: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T04:00:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16860.968203 secs 2023-09-10T04:00:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:30 smithi082 ceph-mon[98694]: pgmap v9154: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T04:00:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16860.968203 secs 2023-09-10T04:00:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:30 smithi098 ceph-mon[103124]: pgmap v9154: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T04:00:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:32 smithi170 ceph-mon[102832]: pgmap v9155: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 222 KiB/s wr, 16 op/s 2023-09-10T04:00:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:32 smithi082 ceph-mon[98694]: pgmap v9155: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 222 KiB/s wr, 16 op/s 2023-09-10T04:00:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:32 smithi098 ceph-mon[103124]: pgmap v9155: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 222 KiB/s wr, 16 op/s 2023-09-10T04:00:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16865.304897 secs 2023-09-10T04:00:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:34 smithi098 ceph-mon[103124]: pgmap v9156: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T04:00:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16865.304897 secs 2023-09-10T04:00:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:34 smithi170 ceph-mon[102832]: pgmap v9156: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T04:00:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16865.304897 secs 2023-09-10T04:00:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:34 smithi082 ceph-mon[98694]: pgmap v9156: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T04:00:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16865.968273 secs 2023-09-10T04:00:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16865.968273 secs 2023-09-10T04:00:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16865.968273 secs 2023-09-10T04:00:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:36 smithi098 ceph-mon[103124]: pgmap v9157: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 356 KiB/s wr, 14 op/s 2023-09-10T04:00:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:36 smithi170 ceph-mon[102832]: pgmap v9157: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 356 KiB/s wr, 14 op/s 2023-09-10T04:00:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:36 smithi082 ceph-mon[98694]: pgmap v9157: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 356 KiB/s wr, 14 op/s 2023-09-10T04:00:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:38 smithi170 ceph-mon[102832]: pgmap v9158: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 294 KiB/s wr, 15 op/s 2023-09-10T04:00:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:38 smithi082 ceph-mon[98694]: pgmap v9158: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 294 KiB/s wr, 15 op/s 2023-09-10T04:00:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:38 smithi098 ceph-mon[103124]: pgmap v9158: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 294 KiB/s wr, 15 op/s 2023-09-10T04:00:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16870.304981 secs 2023-09-10T04:00:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16870.304981 secs 2023-09-10T04:00:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16870.304981 secs 2023-09-10T04:00:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16870.968341 secs 2023-09-10T04:00:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:40 smithi170 ceph-mon[102832]: pgmap v9159: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T04:00:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16870.968341 secs 2023-09-10T04:00:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:40 smithi082 ceph-mon[98694]: pgmap v9159: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T04:00:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16870.968341 secs 2023-09-10T04:00:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:40 smithi098 ceph-mon[103124]: pgmap v9159: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T04:00:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:00:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:00:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:00:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:42 smithi170 ceph-mon[102832]: pgmap v9160: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 485 KiB/s wr, 18 op/s 2023-09-10T04:00:42.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:42 smithi082 ceph-mon[98694]: pgmap v9160: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 485 KiB/s wr, 18 op/s 2023-09-10T04:00:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:42 smithi098 ceph-mon[103124]: pgmap v9160: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 485 KiB/s wr, 18 op/s 2023-09-10T04:00:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16875.305042 secs 2023-09-10T04:00:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:44 smithi098 ceph-mon[103124]: pgmap v9161: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 411 KiB/s wr, 12 op/s 2023-09-10T04:00:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:00:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:00:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:00:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:00:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16875.305042 secs 2023-09-10T04:00:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:44 smithi170 ceph-mon[102832]: pgmap v9161: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 411 KiB/s wr, 12 op/s 2023-09-10T04:00:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:00:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:00:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:00:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:00:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16875.305042 secs 2023-09-10T04:00:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:44 smithi082 ceph-mon[98694]: pgmap v9161: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 411 KiB/s wr, 12 op/s 2023-09-10T04:00:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:00:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:00:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:00:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:00:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16875.968411 secs 2023-09-10T04:00:45.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:00:45.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:00:45.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:45 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T04:00:45.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:45 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T04:00:45.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:45 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 70899k 2023-09-10T04:00:45.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:45 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 72601268: error parsing value: Value '72601268' is below minimum 939524096 2023-09-10T04:00:45.682 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.682 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.682 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.682 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.682 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:45 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T04:00:45.683 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:45 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T04:00:45.683 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:00:45.683 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:00:45.683 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:00:45.683 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:45 smithi170 ceph-mon[102832]: pgmap v9162: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 459 KiB/s wr, 16 op/s 2023-09-10T04:00:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16875.968411 secs 2023-09-10T04:00:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:00:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:00:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:45 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T04:00:45.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:45 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T04:00:45.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:45 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 70899k 2023-09-10T04:00:45.711 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:45 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 72601268: error parsing value: Value '72601268' is below minimum 939524096 2023-09-10T04:00:45.711 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.711 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.711 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.711 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.711 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:45 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T04:00:45.712 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:45 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T04:00:45.712 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:00:45.712 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:00:45.712 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:00:45.712 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:45 smithi082 ceph-mon[98694]: pgmap v9162: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 459 KiB/s wr, 16 op/s 2023-09-10T04:00:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16875.968411 secs 2023-09-10T04:00:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:00:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:00:45.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:45 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T04:00:45.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:45 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T04:00:45.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:45 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 70899k 2023-09-10T04:00:45.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:45 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 72601268: error parsing value: Value '72601268' is below minimum 939524096 2023-09-10T04:00:45.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:00:45.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:45 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T04:00:45.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:45 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T04:00:45.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:00:45.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:00:45.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:00:45.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:45 smithi098 ceph-mon[103124]: pgmap v9162: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 459 KiB/s wr, 16 op/s 2023-09-10T04:00:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:48 smithi170 ceph-mon[102832]: pgmap v9163: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 265 KiB/s wr, 15 op/s 2023-09-10T04:00:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16879.050534 secs 2023-09-10T04:00:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:48 smithi082 ceph-mon[98694]: pgmap v9163: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 265 KiB/s wr, 15 op/s 2023-09-10T04:00:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16879.050534 secs 2023-09-10T04:00:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:48 smithi098 ceph-mon[103124]: pgmap v9163: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 265 KiB/s wr, 15 op/s 2023-09-10T04:00:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16879.050534 secs 2023-09-10T04:00:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16880.305100 secs 2023-09-10T04:00:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16880.305100 secs 2023-09-10T04:00:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16880.305100 secs 2023-09-10T04:00:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16880.968482 secs 2023-09-10T04:00:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:50 smithi170 ceph-mon[102832]: pgmap v9164: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 252 KiB/s wr, 12 op/s 2023-09-10T04:00:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16880.968482 secs 2023-09-10T04:00:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:50 smithi082 ceph-mon[98694]: pgmap v9164: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 252 KiB/s wr, 12 op/s 2023-09-10T04:00:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16880.968482 secs 2023-09-10T04:00:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:50 smithi098 ceph-mon[103124]: pgmap v9164: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 252 KiB/s wr, 12 op/s 2023-09-10T04:00:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:51 smithi170 ceph-mon[102832]: pgmap v9165: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 18 op/s 2023-09-10T04:00:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:00:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:00:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:00:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:51 smithi082 ceph-mon[98694]: pgmap v9165: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 18 op/s 2023-09-10T04:00:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:00:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:00:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:00:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:51 smithi098 ceph-mon[103124]: pgmap v9165: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 18 op/s 2023-09-10T04:00:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:00:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:00:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:00:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16884.050618 secs 2023-09-10T04:00:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16884.050618 secs 2023-09-10T04:00:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16884.050618 secs 2023-09-10T04:00:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16885.305157 secs 2023-09-10T04:00:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:54 smithi098 ceph-mon[103124]: pgmap v9166: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 254 KiB/s wr, 12 op/s 2023-09-10T04:00:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16885.305157 secs 2023-09-10T04:00:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:54 smithi170 ceph-mon[102832]: pgmap v9166: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 254 KiB/s wr, 12 op/s 2023-09-10T04:00:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16885.305157 secs 2023-09-10T04:00:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:54 smithi082 ceph-mon[98694]: pgmap v9166: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 254 KiB/s wr, 12 op/s 2023-09-10T04:00:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16885.968548 secs 2023-09-10T04:00:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16885.968548 secs 2023-09-10T04:00:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16885.968548 secs 2023-09-10T04:00:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:56 smithi170 ceph-mon[102832]: pgmap v9167: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 254 KiB/s wr, 14 op/s 2023-09-10T04:00:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:56 smithi082 ceph-mon[98694]: pgmap v9167: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 254 KiB/s wr, 14 op/s 2023-09-10T04:00:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:56 smithi098 ceph-mon[103124]: pgmap v9167: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 254 KiB/s wr, 14 op/s 2023-09-10T04:00:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:58 smithi170 ceph-mon[102832]: pgmap v9168: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 334 KiB/s wr, 14 op/s 2023-09-10T04:00:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16889.050751 secs 2023-09-10T04:00:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:58 smithi082 ceph-mon[98694]: pgmap v9168: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 334 KiB/s wr, 14 op/s 2023-09-10T04:00:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16889.050751 secs 2023-09-10T04:00:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:58 smithi098 ceph-mon[103124]: pgmap v9168: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 334 KiB/s wr, 14 op/s 2023-09-10T04:00:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16889.050751 secs 2023-09-10T04:00:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:00:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16890.305215 secs 2023-09-10T04:00:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:00:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16890.305215 secs 2023-09-10T04:00:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:00:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16890.305215 secs 2023-09-10T04:01:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16890.968619 secs 2023-09-10T04:01:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:00 smithi170 ceph-mon[102832]: pgmap v9169: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 321 KiB/s wr, 11 op/s 2023-09-10T04:01:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16890.968619 secs 2023-09-10T04:01:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:00 smithi082 ceph-mon[98694]: pgmap v9169: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 321 KiB/s wr, 11 op/s 2023-09-10T04:01:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16890.968619 secs 2023-09-10T04:01:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:00 smithi098 ceph-mon[103124]: pgmap v9169: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 321 KiB/s wr, 11 op/s 2023-09-10T04:01:02.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:02 smithi170 ceph-mon[102832]: pgmap v9170: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 395 KiB/s wr, 21 op/s 2023-09-10T04:01:02.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:02 smithi082 ceph-mon[98694]: pgmap v9170: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 395 KiB/s wr, 21 op/s 2023-09-10T04:01:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:02 smithi098 ceph-mon[103124]: pgmap v9170: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 395 KiB/s wr, 21 op/s 2023-09-10T04:01:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16894.050774 secs 2023-09-10T04:01:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16894.050774 secs 2023-09-10T04:01:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16894.050774 secs 2023-09-10T04:01:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16895.305273 secs 2023-09-10T04:01:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:04 smithi170 ceph-mon[102832]: pgmap v9171: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 202 KiB/s wr, 15 op/s 2023-09-10T04:01:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16895.305273 secs 2023-09-10T04:01:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:04 smithi082 ceph-mon[98694]: pgmap v9171: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 202 KiB/s wr, 15 op/s 2023-09-10T04:01:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16895.305273 secs 2023-09-10T04:01:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:04 smithi098 ceph-mon[103124]: pgmap v9171: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 202 KiB/s wr, 15 op/s 2023-09-10T04:01:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16895.968684 secs 2023-09-10T04:01:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:05 smithi170 ceph-mon[102832]: pgmap v9172: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 213 KiB/s wr, 26 op/s 2023-09-10T04:01:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16895.968684 secs 2023-09-10T04:01:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:05 smithi082 ceph-mon[98694]: pgmap v9172: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 213 KiB/s wr, 26 op/s 2023-09-10T04:01:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16895.968684 secs 2023-09-10T04:01:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:05 smithi098 ceph-mon[103124]: pgmap v9172: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 213 KiB/s wr, 26 op/s 2023-09-10T04:01:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:08 smithi170 ceph-mon[102832]: pgmap v9173: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 237 KiB/s wr, 28 op/s 2023-09-10T04:01:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16899.050864 secs 2023-09-10T04:01:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:08 smithi082 ceph-mon[98694]: pgmap v9173: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 237 KiB/s wr, 28 op/s 2023-09-10T04:01:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16899.050864 secs 2023-09-10T04:01:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:08 smithi098 ceph-mon[103124]: pgmap v9173: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 237 KiB/s wr, 28 op/s 2023-09-10T04:01:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16899.050864 secs 2023-09-10T04:01:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16900.305343 secs 2023-09-10T04:01:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:09 smithi170 ceph-mon[102832]: osdmap e641: 12 total, 12 up, 12 in 2023-09-10T04:01:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16900.305343 secs 2023-09-10T04:01:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:09 smithi082 ceph-mon[98694]: osdmap e641: 12 total, 12 up, 12 in 2023-09-10T04:01:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16900.305343 secs 2023-09-10T04:01:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:09 smithi098 ceph-mon[103124]: osdmap e641: 12 total, 12 up, 12 in 2023-09-10T04:01:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16900.968749 secs 2023-09-10T04:01:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:10 smithi170 ceph-mon[102832]: pgmap v9175: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 131 KiB/s wr, 30 op/s 2023-09-10T04:01:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16900.968749 secs 2023-09-10T04:01:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:10 smithi082 ceph-mon[98694]: pgmap v9175: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 131 KiB/s wr, 30 op/s 2023-09-10T04:01:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16900.968749 secs 2023-09-10T04:01:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:10 smithi098 ceph-mon[103124]: pgmap v9175: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 131 KiB/s wr, 30 op/s 2023-09-10T04:01:11.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:11 smithi170 ceph-mon[102832]: pgmap v9176: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 43 KiB/s wr, 24 op/s 2023-09-10T04:01:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:11 smithi082 ceph-mon[98694]: pgmap v9176: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 43 KiB/s wr, 24 op/s 2023-09-10T04:01:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:11 smithi098 ceph-mon[103124]: pgmap v9176: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 43 KiB/s wr, 24 op/s 2023-09-10T04:01:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16904.050930 secs 2023-09-10T04:01:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16904.050930 secs 2023-09-10T04:01:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16904.050930 secs 2023-09-10T04:01:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16905.305425 secs 2023-09-10T04:01:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:14 smithi098 ceph-mon[103124]: osdmap e642: 12 total, 12 up, 12 in 2023-09-10T04:01:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:14 smithi098 ceph-mon[103124]: pgmap v9178: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 36 KiB/s wr, 12 op/s 2023-09-10T04:01:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16905.305425 secs 2023-09-10T04:01:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:14 smithi170 ceph-mon[102832]: osdmap e642: 12 total, 12 up, 12 in 2023-09-10T04:01:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:14 smithi170 ceph-mon[102832]: pgmap v9178: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 36 KiB/s wr, 12 op/s 2023-09-10T04:01:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16905.305425 secs 2023-09-10T04:01:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:14 smithi082 ceph-mon[98694]: osdmap e642: 12 total, 12 up, 12 in 2023-09-10T04:01:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:14 smithi082 ceph-mon[98694]: pgmap v9178: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 36 KiB/s wr, 12 op/s 2023-09-10T04:01:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16905.968816 secs 2023-09-10T04:01:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16905.968816 secs 2023-09-10T04:01:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16905.968816 secs 2023-09-10T04:01:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:16 smithi170 ceph-mon[102832]: pgmap v9179: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 9 op/s 2023-09-10T04:01:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:16 smithi082 ceph-mon[98694]: pgmap v9179: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 9 op/s 2023-09-10T04:01:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:16 smithi098 ceph-mon[103124]: pgmap v9179: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 9 op/s 2023-09-10T04:01:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:18 smithi170 ceph-mon[102832]: pgmap v9180: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 13 op/s 2023-09-10T04:01:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16909.050989 secs 2023-09-10T04:01:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:18 smithi082 ceph-mon[98694]: pgmap v9180: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 13 op/s 2023-09-10T04:01:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16909.050989 secs 2023-09-10T04:01:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:18 smithi098 ceph-mon[103124]: pgmap v9180: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 13 op/s 2023-09-10T04:01:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16909.050989 secs 2023-09-10T04:01:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16910.305480 secs 2023-09-10T04:01:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16910.305480 secs 2023-09-10T04:01:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16910.305480 secs 2023-09-10T04:01:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16910.968881 secs 2023-09-10T04:01:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:20 smithi170 ceph-mon[102832]: pgmap v9181: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T04:01:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16910.968881 secs 2023-09-10T04:01:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:20 smithi082 ceph-mon[98694]: pgmap v9181: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T04:01:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16910.968881 secs 2023-09-10T04:01:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:20 smithi098 ceph-mon[103124]: pgmap v9181: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T04:01:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:21 smithi170 ceph-mon[102832]: pgmap v9182: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T04:01:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:21 smithi082 ceph-mon[98694]: pgmap v9182: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T04:01:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:21 smithi098 ceph-mon[103124]: pgmap v9182: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T04:01:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:01:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:22 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:01:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:01:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:22 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:01:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:01:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:22 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:01:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:01:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:01:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16915.305569 secs 2023-09-10T04:01:24.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:23 smithi170 ceph-mon[102832]: pgmap v9183: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T04:01:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:01:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:01:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16915.305569 secs 2023-09-10T04:01:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:23 smithi082 ceph-mon[98694]: pgmap v9183: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T04:01:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:01:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:01:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16915.305569 secs 2023-09-10T04:01:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:23 smithi098 ceph-mon[103124]: pgmap v9183: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 11 op/s 2023-09-10T04:01:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16915.968948 secs 2023-09-10T04:01:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16915.968948 secs 2023-09-10T04:01:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16915.968948 secs 2023-09-10T04:01:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:26 smithi170 ceph-mon[102832]: pgmap v9184: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T04:01:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:26 smithi082 ceph-mon[98694]: pgmap v9184: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T04:01:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:26 smithi098 ceph-mon[103124]: pgmap v9184: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 58 KiB/s wr, 11 op/s 2023-09-10T04:01:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:28 smithi170 ceph-mon[102832]: pgmap v9185: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 192 KiB/s wr, 13 op/s 2023-09-10T04:01:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:28 smithi082 ceph-mon[98694]: pgmap v9185: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 192 KiB/s wr, 13 op/s 2023-09-10T04:01:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:28 smithi098 ceph-mon[103124]: pgmap v9185: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 192 KiB/s wr, 13 op/s 2023-09-10T04:01:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16920.305639 secs 2023-09-10T04:01:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16920.305639 secs 2023-09-10T04:01:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16920.305639 secs 2023-09-10T04:01:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16920.969015 secs 2023-09-10T04:01:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:30 smithi170 ceph-mon[102832]: pgmap v9186: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 10 op/s 2023-09-10T04:01:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16920.969015 secs 2023-09-10T04:01:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:30 smithi082 ceph-mon[98694]: pgmap v9186: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 10 op/s 2023-09-10T04:01:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16920.969015 secs 2023-09-10T04:01:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:30 smithi098 ceph-mon[103124]: pgmap v9186: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 10 op/s 2023-09-10T04:01:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:31 smithi170 ceph-mon[102832]: pgmap v9187: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 209 KiB/s wr, 15 op/s 2023-09-10T04:01:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:31 smithi082 ceph-mon[98694]: pgmap v9187: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 209 KiB/s wr, 15 op/s 2023-09-10T04:01:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:31 smithi098 ceph-mon[103124]: pgmap v9187: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 209 KiB/s wr, 15 op/s 2023-09-10T04:01:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16925.305698 secs 2023-09-10T04:01:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:34 smithi170 ceph-mon[102832]: pgmap v9188: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 208 KiB/s wr, 11 op/s 2023-09-10T04:01:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16925.305698 secs 2023-09-10T04:01:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:34 smithi082 ceph-mon[98694]: pgmap v9188: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 208 KiB/s wr, 11 op/s 2023-09-10T04:01:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16925.305698 secs 2023-09-10T04:01:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:34 smithi098 ceph-mon[103124]: pgmap v9188: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 208 KiB/s wr, 11 op/s 2023-09-10T04:01:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16925.969080 secs 2023-09-10T04:01:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16925.969080 secs 2023-09-10T04:01:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16925.969080 secs 2023-09-10T04:01:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:36 smithi170 ceph-mon[102832]: pgmap v9189: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 209 KiB/s wr, 13 op/s 2023-09-10T04:01:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:36 smithi082 ceph-mon[98694]: pgmap v9189: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 209 KiB/s wr, 13 op/s 2023-09-10T04:01:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:36 smithi098 ceph-mon[103124]: pgmap v9189: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 209 KiB/s wr, 13 op/s 2023-09-10T04:01:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:38 smithi170 ceph-mon[102832]: pgmap v9190: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 251 KiB/s wr, 15 op/s 2023-09-10T04:01:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:38 smithi082 ceph-mon[98694]: pgmap v9190: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 251 KiB/s wr, 15 op/s 2023-09-10T04:01:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:38 smithi098 ceph-mon[103124]: pgmap v9190: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 251 KiB/s wr, 15 op/s 2023-09-10T04:01:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16930.305756 secs 2023-09-10T04:01:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16930.305756 secs 2023-09-10T04:01:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16930.305756 secs 2023-09-10T04:01:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16930.969171 secs 2023-09-10T04:01:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:40 smithi170 ceph-mon[102832]: pgmap v9191: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T04:01:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16930.969171 secs 2023-09-10T04:01:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:40 smithi082 ceph-mon[98694]: pgmap v9191: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T04:01:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16930.969171 secs 2023-09-10T04:01:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:40 smithi098 ceph-mon[103124]: pgmap v9191: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T04:01:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:41 smithi170 ceph-mon[102832]: pgmap v9192: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 351 KiB/s wr, 19 op/s 2023-09-10T04:01:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:41 smithi082 ceph-mon[98694]: pgmap v9192: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 351 KiB/s wr, 19 op/s 2023-09-10T04:01:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:41 smithi098 ceph-mon[103124]: pgmap v9192: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 351 KiB/s wr, 19 op/s 2023-09-10T04:01:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16935.305850 secs 2023-09-10T04:01:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:44 smithi170 ceph-mon[102832]: pgmap v9193: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 277 KiB/s wr, 13 op/s 2023-09-10T04:01:44.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16935.305850 secs 2023-09-10T04:01:44.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:44 smithi082 ceph-mon[98694]: pgmap v9193: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 277 KiB/s wr, 13 op/s 2023-09-10T04:01:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16935.305850 secs 2023-09-10T04:01:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:44 smithi098 ceph-mon[103124]: pgmap v9193: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 277 KiB/s wr, 13 op/s 2023-09-10T04:01:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16935.969259 secs 2023-09-10T04:01:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:01:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16935.969259 secs 2023-09-10T04:01:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:01:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16935.969259 secs 2023-09-10T04:01:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:01:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:46 smithi170 ceph-mon[102832]: pgmap v9194: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 306 KiB/s wr, 16 op/s 2023-09-10T04:01:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:01:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:01:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:01:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:46 smithi082 ceph-mon[98694]: pgmap v9194: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 306 KiB/s wr, 16 op/s 2023-09-10T04:01:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:01:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:01:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:01:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:46 smithi098 ceph-mon[103124]: pgmap v9194: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 306 KiB/s wr, 16 op/s 2023-09-10T04:01:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:01:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:01:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:01:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:48 smithi170 ceph-mon[102832]: pgmap v9195: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 306 KiB/s wr, 16 op/s 2023-09-10T04:01:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:48 smithi082 ceph-mon[98694]: pgmap v9195: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 306 KiB/s wr, 16 op/s 2023-09-10T04:01:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:48 smithi098 ceph-mon[103124]: pgmap v9195: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 306 KiB/s wr, 16 op/s 2023-09-10T04:01:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16940.305912 secs 2023-09-10T04:01:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16940.305912 secs 2023-09-10T04:01:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16940.305912 secs 2023-09-10T04:01:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16940.969322 secs 2023-09-10T04:01:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:50 smithi170 ceph-mon[102832]: pgmap v9196: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T04:01:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16940.969322 secs 2023-09-10T04:01:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:50 smithi082 ceph-mon[98694]: pgmap v9196: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T04:01:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16940.969322 secs 2023-09-10T04:01:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:50 smithi098 ceph-mon[103124]: pgmap v9196: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T04:01:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:51 smithi170 ceph-mon[102832]: pgmap v9197: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 493 KiB/s wr, 19 op/s 2023-09-10T04:01:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:51 smithi082 ceph-mon[98694]: pgmap v9197: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 493 KiB/s wr, 19 op/s 2023-09-10T04:01:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:51 smithi098 ceph-mon[103124]: pgmap v9197: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 493 KiB/s wr, 19 op/s 2023-09-10T04:01:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16944.051541 secs 2023-09-10T04:01:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16944.051541 secs 2023-09-10T04:01:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16944.051541 secs 2023-09-10T04:01:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16945.305970 secs 2023-09-10T04:01:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:54 smithi170 ceph-mon[102832]: pgmap v9198: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 259 KiB/s wr, 12 op/s 2023-09-10T04:01:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16945.305970 secs 2023-09-10T04:01:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:54 smithi082 ceph-mon[98694]: pgmap v9198: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 259 KiB/s wr, 12 op/s 2023-09-10T04:01:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16945.305970 secs 2023-09-10T04:01:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:54 smithi098 ceph-mon[103124]: pgmap v9198: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 259 KiB/s wr, 12 op/s 2023-09-10T04:01:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16945.969386 secs 2023-09-10T04:01:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16945.969386 secs 2023-09-10T04:01:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16945.969386 secs 2023-09-10T04:01:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:01:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:01:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:01:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:56 smithi170 ceph-mon[102832]: pgmap v9199: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 304 KiB/s wr, 15 op/s 2023-09-10T04:01:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:01:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:01:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:01:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:56 smithi082 ceph-mon[98694]: pgmap v9199: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 304 KiB/s wr, 15 op/s 2023-09-10T04:01:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:01:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:01:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:01:56.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:56 smithi098 ceph-mon[103124]: pgmap v9199: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 304 KiB/s wr, 15 op/s 2023-09-10T04:01:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:58 smithi170 ceph-mon[102832]: pgmap v9200: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 276 KiB/s wr, 14 op/s 2023-09-10T04:01:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16949.051655 secs 2023-09-10T04:01:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:58 smithi082 ceph-mon[98694]: pgmap v9200: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 276 KiB/s wr, 14 op/s 2023-09-10T04:01:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16949.051655 secs 2023-09-10T04:01:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:58 smithi098 ceph-mon[103124]: pgmap v9200: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 276 KiB/s wr, 14 op/s 2023-09-10T04:01:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16949.051655 secs 2023-09-10T04:01:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:01:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16950.306027 secs 2023-09-10T04:01:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:01:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16950.306027 secs 2023-09-10T04:01:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:01:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16950.306027 secs 2023-09-10T04:02:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16950.969450 secs 2023-09-10T04:02:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:00 smithi170 ceph-mon[102832]: pgmap v9201: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 276 KiB/s wr, 11 op/s 2023-09-10T04:02:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16950.969450 secs 2023-09-10T04:02:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:00 smithi082 ceph-mon[98694]: pgmap v9201: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 276 KiB/s wr, 11 op/s 2023-09-10T04:02:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16950.969450 secs 2023-09-10T04:02:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:00 smithi098 ceph-mon[103124]: pgmap v9201: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 276 KiB/s wr, 11 op/s 2023-09-10T04:02:01.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:01 smithi170 ceph-mon[102832]: pgmap v9202: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 410 KiB/s wr, 17 op/s 2023-09-10T04:02:01.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:01 smithi082 ceph-mon[98694]: pgmap v9202: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 410 KiB/s wr, 17 op/s 2023-09-10T04:02:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:01 smithi098 ceph-mon[103124]: pgmap v9202: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 410 KiB/s wr, 17 op/s 2023-09-10T04:02:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16954.051754 secs 2023-09-10T04:02:02.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16954.051754 secs 2023-09-10T04:02:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16954.051754 secs 2023-09-10T04:02:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16955.306085 secs 2023-09-10T04:02:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:04 smithi170 ceph-mon[102832]: pgmap v9203: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 181 KiB/s wr, 11 op/s 2023-09-10T04:02:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16955.306085 secs 2023-09-10T04:02:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:04 smithi082 ceph-mon[98694]: pgmap v9203: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 181 KiB/s wr, 11 op/s 2023-09-10T04:02:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16955.306085 secs 2023-09-10T04:02:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:04 smithi098 ceph-mon[103124]: pgmap v9203: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 181 KiB/s wr, 11 op/s 2023-09-10T04:02:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16955.969483 secs 2023-09-10T04:02:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16955.969483 secs 2023-09-10T04:02:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16955.969483 secs 2023-09-10T04:02:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:06 smithi170 ceph-mon[102832]: pgmap v9204: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 203 KiB/s wr, 26 op/s 2023-09-10T04:02:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:06 smithi082 ceph-mon[98694]: pgmap v9204: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 203 KiB/s wr, 26 op/s 2023-09-10T04:02:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:06 smithi098 ceph-mon[103124]: pgmap v9204: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 203 KiB/s wr, 26 op/s 2023-09-10T04:02:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:08 smithi170 ceph-mon[102832]: pgmap v9205: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 194 KiB/s wr, 28 op/s 2023-09-10T04:02:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16959.051772 secs 2023-09-10T04:02:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:08 smithi082 ceph-mon[98694]: pgmap v9205: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 194 KiB/s wr, 28 op/s 2023-09-10T04:02:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16959.051772 secs 2023-09-10T04:02:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:08 smithi098 ceph-mon[103124]: pgmap v9205: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 194 KiB/s wr, 28 op/s 2023-09-10T04:02:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16959.051772 secs 2023-09-10T04:02:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16960.306147 secs 2023-09-10T04:02:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:09 smithi170 ceph-mon[102832]: osdmap e643: 12 total, 12 up, 12 in 2023-09-10T04:02:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:09 smithi170 ceph-mon[102832]: pgmap v9207: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 231 KiB/s wr, 31 op/s 2023-09-10T04:02:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16960.306147 secs 2023-09-10T04:02:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:09 smithi082 ceph-mon[98694]: osdmap e643: 12 total, 12 up, 12 in 2023-09-10T04:02:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:09 smithi082 ceph-mon[98694]: pgmap v9207: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 231 KiB/s wr, 31 op/s 2023-09-10T04:02:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16960.306147 secs 2023-09-10T04:02:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:09 smithi098 ceph-mon[103124]: osdmap e643: 12 total, 12 up, 12 in 2023-09-10T04:02:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:09 smithi098 ceph-mon[103124]: pgmap v9207: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 231 KiB/s wr, 31 op/s 2023-09-10T04:02:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16960.969550 secs 2023-09-10T04:02:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16960.969550 secs 2023-09-10T04:02:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16960.969550 secs 2023-09-10T04:02:11.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:11 smithi170 ceph-mon[102832]: pgmap v9208: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 127 KiB/s wr, 30 op/s 2023-09-10T04:02:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:11 smithi082 ceph-mon[98694]: pgmap v9208: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 127 KiB/s wr, 30 op/s 2023-09-10T04:02:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:11 smithi098 ceph-mon[103124]: pgmap v9208: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 127 KiB/s wr, 30 op/s 2023-09-10T04:02:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16964.051834 secs 2023-09-10T04:02:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16964.051834 secs 2023-09-10T04:02:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16964.051834 secs 2023-09-10T04:02:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16965.306202 secs 2023-09-10T04:02:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:14 smithi170 ceph-mon[102832]: osdmap e644: 12 total, 12 up, 12 in 2023-09-10T04:02:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:14 smithi170 ceph-mon[102832]: pgmap v9210: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 125 KiB/s wr, 15 op/s 2023-09-10T04:02:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16965.306202 secs 2023-09-10T04:02:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:14 smithi082 ceph-mon[98694]: osdmap e644: 12 total, 12 up, 12 in 2023-09-10T04:02:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:14 smithi082 ceph-mon[98694]: pgmap v9210: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 125 KiB/s wr, 15 op/s 2023-09-10T04:02:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16965.306202 secs 2023-09-10T04:02:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:14 smithi098 ceph-mon[103124]: osdmap e644: 12 total, 12 up, 12 in 2023-09-10T04:02:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:14 smithi098 ceph-mon[103124]: pgmap v9210: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 125 KiB/s wr, 15 op/s 2023-09-10T04:02:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16965.969615 secs 2023-09-10T04:02:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16965.969615 secs 2023-09-10T04:02:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16965.969615 secs 2023-09-10T04:02:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:16 smithi170 ceph-mon[102832]: pgmap v9211: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T04:02:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:16 smithi082 ceph-mon[98694]: pgmap v9211: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T04:02:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:16 smithi098 ceph-mon[103124]: pgmap v9211: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 12 op/s 2023-09-10T04:02:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:17 smithi170 ceph-mon[102832]: pgmap v9212: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 13 op/s 2023-09-10T04:02:17.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:17 smithi082 ceph-mon[98694]: pgmap v9212: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 13 op/s 2023-09-10T04:02:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:17 smithi098 ceph-mon[103124]: pgmap v9212: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 13 op/s 2023-09-10T04:02:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16969.051892 secs 2023-09-10T04:02:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16969.051892 secs 2023-09-10T04:02:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16969.051892 secs 2023-09-10T04:02:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16970.306260 secs 2023-09-10T04:02:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:19 smithi170 ceph-mon[102832]: pgmap v9213: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T04:02:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16970.306260 secs 2023-09-10T04:02:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:19 smithi082 ceph-mon[98694]: pgmap v9213: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T04:02:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16970.306260 secs 2023-09-10T04:02:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:19 smithi098 ceph-mon[103124]: pgmap v9213: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T04:02:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16970.969680 secs 2023-09-10T04:02:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16970.969680 secs 2023-09-10T04:02:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16970.969680 secs 2023-09-10T04:02:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:21 smithi170 ceph-mon[102832]: pgmap v9214: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 11 op/s 2023-09-10T04:02:21.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:21 smithi082 ceph-mon[98694]: pgmap v9214: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 11 op/s 2023-09-10T04:02:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:21 smithi098 ceph-mon[103124]: pgmap v9214: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 11 op/s 2023-09-10T04:02:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:02:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:02:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:02:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:02:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:02:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:02:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16975.306339 secs 2023-09-10T04:02:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:23 smithi170 ceph-mon[102832]: pgmap v9215: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T04:02:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:02:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:02:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:02:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16975.306339 secs 2023-09-10T04:02:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:23 smithi082 ceph-mon[98694]: pgmap v9215: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T04:02:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:02:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:02:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:02:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16975.306339 secs 2023-09-10T04:02:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:23 smithi098 ceph-mon[103124]: pgmap v9215: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T04:02:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16975.969746 secs 2023-09-10T04:02:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16975.969746 secs 2023-09-10T04:02:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16975.969746 secs 2023-09-10T04:02:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:26 smithi170 ceph-mon[102832]: pgmap v9216: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 13 op/s 2023-09-10T04:02:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:26 smithi082 ceph-mon[98694]: pgmap v9216: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 13 op/s 2023-09-10T04:02:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:26 smithi098 ceph-mon[103124]: pgmap v9216: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 13 op/s 2023-09-10T04:02:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:27 smithi170 ceph-mon[102832]: pgmap v9217: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T04:02:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:27 smithi082 ceph-mon[98694]: pgmap v9217: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T04:02:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:27 smithi098 ceph-mon[103124]: pgmap v9217: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T04:02:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16980.306478 secs 2023-09-10T04:02:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16980.306478 secs 2023-09-10T04:02:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16980.306478 secs 2023-09-10T04:02:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16980.969812 secs 2023-09-10T04:02:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:30 smithi170 ceph-mon[102832]: pgmap v9218: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 9 op/s 2023-09-10T04:02:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16980.969812 secs 2023-09-10T04:02:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:30 smithi082 ceph-mon[98694]: pgmap v9218: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 9 op/s 2023-09-10T04:02:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16980.969812 secs 2023-09-10T04:02:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:30 smithi098 ceph-mon[103124]: pgmap v9218: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 9 op/s 2023-09-10T04:02:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:31 smithi170 ceph-mon[102832]: pgmap v9219: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 191 KiB/s wr, 16 op/s 2023-09-10T04:02:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:31 smithi082 ceph-mon[98694]: pgmap v9219: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 191 KiB/s wr, 16 op/s 2023-09-10T04:02:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:31 smithi098 ceph-mon[103124]: pgmap v9219: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 191 KiB/s wr, 16 op/s 2023-09-10T04:02:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16985.306495 secs 2023-09-10T04:02:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:34 smithi170 ceph-mon[102832]: pgmap v9220: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 136 KiB/s wr, 11 op/s 2023-09-10T04:02:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16985.306495 secs 2023-09-10T04:02:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:34 smithi082 ceph-mon[98694]: pgmap v9220: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 136 KiB/s wr, 11 op/s 2023-09-10T04:02:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16985.306495 secs 2023-09-10T04:02:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:34 smithi098 ceph-mon[103124]: pgmap v9220: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 136 KiB/s wr, 11 op/s 2023-09-10T04:02:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16985.969874 secs 2023-09-10T04:02:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16985.969874 secs 2023-09-10T04:02:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16985.969874 secs 2023-09-10T04:02:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:36 smithi170 ceph-mon[102832]: pgmap v9221: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 335 KiB/s wr, 36 op/s 2023-09-10T04:02:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:36 smithi082 ceph-mon[98694]: pgmap v9221: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 335 KiB/s wr, 36 op/s 2023-09-10T04:02:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:36 smithi098 ceph-mon[103124]: pgmap v9221: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 335 KiB/s wr, 36 op/s 2023-09-10T04:02:37.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:37 smithi170 ceph-mon[102832]: pgmap v9222: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 333 KiB/s wr, 33 op/s 2023-09-10T04:02:37.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:37 smithi082 ceph-mon[98694]: pgmap v9222: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 333 KiB/s wr, 33 op/s 2023-09-10T04:02:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:37 smithi098 ceph-mon[103124]: pgmap v9222: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 333 KiB/s wr, 33 op/s 2023-09-10T04:02:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16990.306580 secs 2023-09-10T04:02:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16990.306580 secs 2023-09-10T04:02:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16990.306580 secs 2023-09-10T04:02:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16990.969943 secs 2023-09-10T04:02:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:40 smithi170 ceph-mon[102832]: pgmap v9223: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 273 KiB/s wr, 31 op/s 2023-09-10T04:02:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16990.969943 secs 2023-09-10T04:02:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:40 smithi082 ceph-mon[98694]: pgmap v9223: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 273 KiB/s wr, 31 op/s 2023-09-10T04:02:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16990.969943 secs 2023-09-10T04:02:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:40 smithi098 ceph-mon[103124]: pgmap v9223: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 273 KiB/s wr, 31 op/s 2023-09-10T04:02:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:41 smithi170 ceph-mon[102832]: pgmap v9224: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 532 KiB/s wr, 39 op/s 2023-09-10T04:02:41.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:41 smithi082 ceph-mon[98694]: pgmap v9224: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 532 KiB/s wr, 39 op/s 2023-09-10T04:02:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:41 smithi098 ceph-mon[103124]: pgmap v9224: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 532 KiB/s wr, 39 op/s 2023-09-10T04:02:42.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16994.052263 secs 2023-09-10T04:02:42.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16994.052263 secs 2023-09-10T04:02:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16994.052263 secs 2023-09-10T04:02:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16995.306638 secs 2023-09-10T04:02:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:44 smithi170 ceph-mon[102832]: pgmap v9225: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 457 KiB/s wr, 33 op/s 2023-09-10T04:02:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:02:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:02:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:02:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16995.306638 secs 2023-09-10T04:02:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:44 smithi082 ceph-mon[98694]: pgmap v9225: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 457 KiB/s wr, 33 op/s 2023-09-10T04:02:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:02:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:02:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:02:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16995.306638 secs 2023-09-10T04:02:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:44 smithi098 ceph-mon[103124]: pgmap v9225: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 457 KiB/s wr, 33 op/s 2023-09-10T04:02:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:02:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:02:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:02:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 16995.970008 secs 2023-09-10T04:02:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 16995.970008 secs 2023-09-10T04:02:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 16995.970008 secs 2023-09-10T04:02:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:46 smithi170 ceph-mon[102832]: pgmap v9226: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 458 KiB/s wr, 38 op/s 2023-09-10T04:02:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:02:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:02:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:02:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:46 smithi082 ceph-mon[98694]: pgmap v9226: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 458 KiB/s wr, 38 op/s 2023-09-10T04:02:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:02:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:02:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:02:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:46 smithi098 ceph-mon[103124]: pgmap v9226: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 458 KiB/s wr, 38 op/s 2023-09-10T04:02:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:02:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:02:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:02:47.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:02:47.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:02:47.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:02:47.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:02:47.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:02:47.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:02:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:02:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:02:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:02:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:48 smithi170 ceph-mon[102832]: pgmap v9227: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 259 KiB/s wr, 14 op/s 2023-09-10T04:02:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 16999.052321 secs 2023-09-10T04:02:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:48 smithi082 ceph-mon[98694]: pgmap v9227: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 259 KiB/s wr, 14 op/s 2023-09-10T04:02:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 16999.052321 secs 2023-09-10T04:02:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:48 smithi098 ceph-mon[103124]: pgmap v9227: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 259 KiB/s wr, 14 op/s 2023-09-10T04:02:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 16999.052321 secs 2023-09-10T04:02:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17000.306695 secs 2023-09-10T04:02:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17000.306695 secs 2023-09-10T04:02:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17000.306695 secs 2023-09-10T04:02:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17000.970077 secs 2023-09-10T04:02:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:50 smithi170 ceph-mon[102832]: pgmap v9228: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 259 KiB/s wr, 13 op/s 2023-09-10T04:02:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17000.970077 secs 2023-09-10T04:02:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:50 smithi082 ceph-mon[98694]: pgmap v9228: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 259 KiB/s wr, 13 op/s 2023-09-10T04:02:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17000.970077 secs 2023-09-10T04:02:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:50 smithi098 ceph-mon[103124]: pgmap v9228: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 259 KiB/s wr, 13 op/s 2023-09-10T04:02:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:51 smithi170 ceph-mon[102832]: pgmap v9229: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 393 KiB/s wr, 18 op/s 2023-09-10T04:02:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:51 smithi082 ceph-mon[98694]: pgmap v9229: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 393 KiB/s wr, 18 op/s 2023-09-10T04:02:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:51 smithi098 ceph-mon[103124]: pgmap v9229: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 393 KiB/s wr, 18 op/s 2023-09-10T04:02:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17004.052382 secs 2023-09-10T04:02:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17004.052382 secs 2023-09-10T04:02:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17004.052382 secs 2023-09-10T04:02:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17005.306754 secs 2023-09-10T04:02:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:54 smithi170 ceph-mon[102832]: pgmap v9230: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 10 op/s 2023-09-10T04:02:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17005.306754 secs 2023-09-10T04:02:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:54 smithi082 ceph-mon[98694]: pgmap v9230: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 10 op/s 2023-09-10T04:02:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17005.306754 secs 2023-09-10T04:02:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:54 smithi098 ceph-mon[103124]: pgmap v9230: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 10 op/s 2023-09-10T04:02:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17005.970141 secs 2023-09-10T04:02:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17005.970141 secs 2023-09-10T04:02:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17005.970141 secs 2023-09-10T04:02:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:56 smithi170 ceph-mon[102832]: pgmap v9231: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 173 KiB/s wr, 14 op/s 2023-09-10T04:02:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:56 smithi082 ceph-mon[98694]: pgmap v9231: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 173 KiB/s wr, 14 op/s 2023-09-10T04:02:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:56 smithi098 ceph-mon[103124]: pgmap v9231: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 173 KiB/s wr, 14 op/s 2023-09-10T04:02:57.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:57 smithi170 ceph-mon[102832]: pgmap v9232: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 173 KiB/s wr, 10 op/s 2023-09-10T04:02:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:57 smithi082 ceph-mon[98694]: pgmap v9232: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 173 KiB/s wr, 10 op/s 2023-09-10T04:02:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:57 smithi098 ceph-mon[103124]: pgmap v9232: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 173 KiB/s wr, 10 op/s 2023-09-10T04:02:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17009.052442 secs 2023-09-10T04:02:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17009.052442 secs 2023-09-10T04:02:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17009.052442 secs 2023-09-10T04:02:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17010.306811 secs 2023-09-10T04:02:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:02:59 smithi170 ceph-mon[102832]: pgmap v9233: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 173 KiB/s wr, 9 op/s 2023-09-10T04:02:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17010.306811 secs 2023-09-10T04:02:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:02:59 smithi082 ceph-mon[98694]: pgmap v9233: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 173 KiB/s wr, 9 op/s 2023-09-10T04:02:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17010.306811 secs 2023-09-10T04:02:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:02:59 smithi098 ceph-mon[103124]: pgmap v9233: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 173 KiB/s wr, 9 op/s 2023-09-10T04:03:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17010.970207 secs 2023-09-10T04:03:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17010.970207 secs 2023-09-10T04:03:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17010.970207 secs 2023-09-10T04:03:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:01 smithi082 ceph-mon[98694]: pgmap v9234: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 173 KiB/s wr, 13 op/s 2023-09-10T04:03:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:01 smithi098 ceph-mon[103124]: pgmap v9234: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 173 KiB/s wr, 13 op/s 2023-09-10T04:03:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:01 smithi170 ceph-mon[102832]: pgmap v9234: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 173 KiB/s wr, 13 op/s 2023-09-10T04:03:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17014.052520 secs 2023-09-10T04:03:02.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17014.052520 secs 2023-09-10T04:03:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17014.052520 secs 2023-09-10T04:03:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17015.306868 secs 2023-09-10T04:03:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:04 smithi170 ceph-mon[102832]: pgmap v9235: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 8 op/s 2023-09-10T04:03:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17015.306868 secs 2023-09-10T04:03:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:04 smithi082 ceph-mon[98694]: pgmap v9235: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 8 op/s 2023-09-10T04:03:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17015.306868 secs 2023-09-10T04:03:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:04 smithi098 ceph-mon[103124]: pgmap v9235: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 8 op/s 2023-09-10T04:03:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17015.970273 secs 2023-09-10T04:03:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17015.970273 secs 2023-09-10T04:03:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17015.970273 secs 2023-09-10T04:03:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:06 smithi170 ceph-mon[102832]: pgmap v9236: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 84 KiB/s wr, 26 op/s 2023-09-10T04:03:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:06 smithi082 ceph-mon[98694]: pgmap v9236: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 84 KiB/s wr, 26 op/s 2023-09-10T04:03:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:06 smithi098 ceph-mon[103124]: pgmap v9236: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 84 KiB/s wr, 26 op/s 2023-09-10T04:03:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:07 smithi170 ceph-mon[102832]: pgmap v9237: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-09-10T04:03:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:07 smithi082 ceph-mon[98694]: pgmap v9237: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-09-10T04:03:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:07 smithi098 ceph-mon[103124]: pgmap v9237: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-09-10T04:03:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17019.052580 secs 2023-09-10T04:03:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:08 smithi170 ceph-mon[102832]: osdmap e645: 12 total, 12 up, 12 in 2023-09-10T04:03:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17019.052580 secs 2023-09-10T04:03:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:08 smithi082 ceph-mon[98694]: osdmap e645: 12 total, 12 up, 12 in 2023-09-10T04:03:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17019.052580 secs 2023-09-10T04:03:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:08 smithi098 ceph-mon[103124]: osdmap e645: 12 total, 12 up, 12 in 2023-09-10T04:03:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17020.306926 secs 2023-09-10T04:03:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:09 smithi170 ceph-mon[102832]: pgmap v9239: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T04:03:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17020.306926 secs 2023-09-10T04:03:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:09 smithi082 ceph-mon[98694]: pgmap v9239: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T04:03:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17020.306926 secs 2023-09-10T04:03:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:09 smithi098 ceph-mon[103124]: pgmap v9239: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T04:03:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17020.970338 secs 2023-09-10T04:03:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17020.970338 secs 2023-09-10T04:03:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17020.970338 secs 2023-09-10T04:03:11.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:11 smithi170 ceph-mon[102832]: pgmap v9240: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T04:03:11.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:11 smithi082 ceph-mon[98694]: pgmap v9240: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T04:03:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:11 smithi098 ceph-mon[103124]: pgmap v9240: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T04:03:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17024.052641 secs 2023-09-10T04:03:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17024.052641 secs 2023-09-10T04:03:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17024.052641 secs 2023-09-10T04:03:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17025.306984 secs 2023-09-10T04:03:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:14 smithi170 ceph-mon[102832]: osdmap e646: 12 total, 12 up, 12 in 2023-09-10T04:03:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:14 smithi170 ceph-mon[102832]: pgmap v9242: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T04:03:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17025.306984 secs 2023-09-10T04:03:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:14 smithi082 ceph-mon[98694]: osdmap e646: 12 total, 12 up, 12 in 2023-09-10T04:03:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:14 smithi082 ceph-mon[98694]: pgmap v9242: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T04:03:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17025.306984 secs 2023-09-10T04:03:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:14 smithi098 ceph-mon[103124]: osdmap e646: 12 total, 12 up, 12 in 2023-09-10T04:03:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:14 smithi098 ceph-mon[103124]: pgmap v9242: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T04:03:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17025.970403 secs 2023-09-10T04:03:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17025.970403 secs 2023-09-10T04:03:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17025.970403 secs 2023-09-10T04:03:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:16 smithi170 ceph-mon[102832]: pgmap v9243: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 12 op/s 2023-09-10T04:03:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:16 smithi082 ceph-mon[98694]: pgmap v9243: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 12 op/s 2023-09-10T04:03:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:16 smithi098 ceph-mon[103124]: pgmap v9243: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 12 op/s 2023-09-10T04:03:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:17 smithi170 ceph-mon[102832]: pgmap v9244: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 413 B/s wr, 10 op/s 2023-09-10T04:03:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:17 smithi082 ceph-mon[98694]: pgmap v9244: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 413 B/s wr, 10 op/s 2023-09-10T04:03:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:17 smithi098 ceph-mon[103124]: pgmap v9244: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 413 B/s wr, 10 op/s 2023-09-10T04:03:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17029.052746 secs 2023-09-10T04:03:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17029.052746 secs 2023-09-10T04:03:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17029.052746 secs 2023-09-10T04:03:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17030.307041 secs 2023-09-10T04:03:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:19 smithi170 ceph-mon[102832]: pgmap v9245: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 409 B/s wr, 10 op/s 2023-09-10T04:03:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17030.307041 secs 2023-09-10T04:03:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:19 smithi082 ceph-mon[98694]: pgmap v9245: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 409 B/s wr, 10 op/s 2023-09-10T04:03:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17030.307041 secs 2023-09-10T04:03:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:19 smithi098 ceph-mon[103124]: pgmap v9245: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 409 B/s wr, 10 op/s 2023-09-10T04:03:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17030.970474 secs 2023-09-10T04:03:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17030.970474 secs 2023-09-10T04:03:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17030.970474 secs 2023-09-10T04:03:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:21 smithi170 ceph-mon[102832]: pgmap v9246: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-09-10T04:03:21.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:21 smithi082 ceph-mon[98694]: pgmap v9246: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-09-10T04:03:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:21 smithi098 ceph-mon[103124]: pgmap v9246: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-09-10T04:03:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:22 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:03:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:22 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:03:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:22 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:03:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:23 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:03:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:03:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:03:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17035.307102 secs 2023-09-10T04:03:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:23 smithi170 ceph-mon[102832]: pgmap v9247: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-09-10T04:03:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:23 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:03:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:03:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:03:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17035.307102 secs 2023-09-10T04:03:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:23 smithi082 ceph-mon[98694]: pgmap v9247: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-09-10T04:03:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:23 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:03:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:03:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:03:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17035.307102 secs 2023-09-10T04:03:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:23 smithi098 ceph-mon[103124]: pgmap v9247: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-09-10T04:03:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17035.970487 secs 2023-09-10T04:03:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17035.970487 secs 2023-09-10T04:03:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17035.970487 secs 2023-09-10T04:03:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:26 smithi170 ceph-mon[102832]: pgmap v9248: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 129 KiB/s wr, 14 op/s 2023-09-10T04:03:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:26 smithi082 ceph-mon[98694]: pgmap v9248: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 129 KiB/s wr, 14 op/s 2023-09-10T04:03:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:26 smithi098 ceph-mon[103124]: pgmap v9248: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 129 KiB/s wr, 14 op/s 2023-09-10T04:03:27.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:27 smithi170 ceph-mon[102832]: pgmap v9249: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T04:03:27.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:27 smithi082 ceph-mon[98694]: pgmap v9249: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T04:03:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:27 smithi098 ceph-mon[103124]: pgmap v9249: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T04:03:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17040.307187 secs 2023-09-10T04:03:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17040.307187 secs 2023-09-10T04:03:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17040.307187 secs 2023-09-10T04:03:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17040.970551 secs 2023-09-10T04:03:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:30 smithi170 ceph-mon[102832]: pgmap v9250: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 10 op/s 2023-09-10T04:03:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17040.970551 secs 2023-09-10T04:03:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:30 smithi082 ceph-mon[98694]: pgmap v9250: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 10 op/s 2023-09-10T04:03:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17040.970551 secs 2023-09-10T04:03:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:30 smithi098 ceph-mon[103124]: pgmap v9250: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 10 op/s 2023-09-10T04:03:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:31 smithi170 ceph-mon[102832]: pgmap v9251: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 176 KiB/s wr, 15 op/s 2023-09-10T04:03:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:31 smithi082 ceph-mon[98694]: pgmap v9251: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 176 KiB/s wr, 15 op/s 2023-09-10T04:03:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:31 smithi098 ceph-mon[103124]: pgmap v9251: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 176 KiB/s wr, 15 op/s 2023-09-10T04:03:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17045.307246 secs 2023-09-10T04:03:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:34 smithi170 ceph-mon[102832]: pgmap v9252: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 175 KiB/s wr, 10 op/s 2023-09-10T04:03:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17045.307246 secs 2023-09-10T04:03:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:34 smithi082 ceph-mon[98694]: pgmap v9252: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 175 KiB/s wr, 10 op/s 2023-09-10T04:03:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17045.307246 secs 2023-09-10T04:03:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:34 smithi098 ceph-mon[103124]: pgmap v9252: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 175 KiB/s wr, 10 op/s 2023-09-10T04:03:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17045.970616 secs 2023-09-10T04:03:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17045.970616 secs 2023-09-10T04:03:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17045.970616 secs 2023-09-10T04:03:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:36 smithi170 ceph-mon[102832]: pgmap v9253: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 272 KiB/s wr, 15 op/s 2023-09-10T04:03:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:36 smithi082 ceph-mon[98694]: pgmap v9253: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 272 KiB/s wr, 15 op/s 2023-09-10T04:03:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:36 smithi098 ceph-mon[103124]: pgmap v9253: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 272 KiB/s wr, 15 op/s 2023-09-10T04:03:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:37 smithi170 ceph-mon[102832]: pgmap v9254: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T04:03:37.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:37 smithi082 ceph-mon[98694]: pgmap v9254: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T04:03:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:37 smithi098 ceph-mon[103124]: pgmap v9254: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T04:03:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17050.307305 secs 2023-09-10T04:03:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17050.307305 secs 2023-09-10T04:03:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17050.307305 secs 2023-09-10T04:03:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17050.970682 secs 2023-09-10T04:03:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:40 smithi170 ceph-mon[102832]: pgmap v9255: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T04:03:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17050.970682 secs 2023-09-10T04:03:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:40 smithi082 ceph-mon[98694]: pgmap v9255: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T04:03:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17050.970682 secs 2023-09-10T04:03:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:40 smithi098 ceph-mon[103124]: pgmap v9255: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T04:03:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:41 smithi170 ceph-mon[102832]: pgmap v9256: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 267 KiB/s wr, 16 op/s 2023-09-10T04:03:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:41 smithi082 ceph-mon[98694]: pgmap v9256: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 267 KiB/s wr, 16 op/s 2023-09-10T04:03:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:41 smithi098 ceph-mon[103124]: pgmap v9256: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 267 KiB/s wr, 16 op/s 2023-09-10T04:03:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17055.307364 secs 2023-09-10T04:03:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:44 smithi170 ceph-mon[102832]: pgmap v9257: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 220 KiB/s wr, 11 op/s 2023-09-10T04:03:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17055.307364 secs 2023-09-10T04:03:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:44 smithi082 ceph-mon[98694]: pgmap v9257: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 220 KiB/s wr, 11 op/s 2023-09-10T04:03:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17055.307364 secs 2023-09-10T04:03:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:44 smithi098 ceph-mon[103124]: pgmap v9257: 97 pgs: 97 active+clean; 9.6 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 220 KiB/s wr, 11 op/s 2023-09-10T04:03:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17055.970748 secs 2023-09-10T04:03:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17055.970748 secs 2023-09-10T04:03:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17055.970748 secs 2023-09-10T04:03:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:46 smithi170 ceph-mon[102832]: pgmap v9258: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 387 KiB/s wr, 18 op/s 2023-09-10T04:03:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:46 smithi082 ceph-mon[98694]: pgmap v9258: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 387 KiB/s wr, 18 op/s 2023-09-10T04:03:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:46 smithi098 ceph-mon[103124]: pgmap v9258: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 387 KiB/s wr, 18 op/s 2023-09-10T04:03:47.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:03:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:03:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:03:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:48 smithi170 ceph-mon[102832]: pgmap v9259: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 290 KiB/s wr, 13 op/s 2023-09-10T04:03:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:03:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:03:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:03:48.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:03:48.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:03:48.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:03:48.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:03:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:48 smithi082 ceph-mon[98694]: pgmap v9259: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 290 KiB/s wr, 13 op/s 2023-09-10T04:03:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:03:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:03:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:03:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:03:48.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:03:48.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:03:48.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:03:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:48 smithi098 ceph-mon[103124]: pgmap v9259: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 290 KiB/s wr, 13 op/s 2023-09-10T04:03:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:03:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:03:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:03:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:03:48.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:03:48.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:03:48.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:03:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17060.307469 secs 2023-09-10T04:03:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17060.307469 secs 2023-09-10T04:03:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17060.307469 secs 2023-09-10T04:03:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17060.970814 secs 2023-09-10T04:03:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:50 smithi170 ceph-mon[102832]: pgmap v9260: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 290 KiB/s wr, 13 op/s 2023-09-10T04:03:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17060.970814 secs 2023-09-10T04:03:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:50 smithi082 ceph-mon[98694]: pgmap v9260: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 290 KiB/s wr, 13 op/s 2023-09-10T04:03:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17060.970814 secs 2023-09-10T04:03:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:50 smithi098 ceph-mon[103124]: pgmap v9260: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 290 KiB/s wr, 13 op/s 2023-09-10T04:03:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:51 smithi170 ceph-mon[102832]: pgmap v9261: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 370 KiB/s wr, 19 op/s 2023-09-10T04:03:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:51 smithi082 ceph-mon[98694]: pgmap v9261: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 370 KiB/s wr, 19 op/s 2023-09-10T04:03:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:51 smithi098 ceph-mon[103124]: pgmap v9261: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 370 KiB/s wr, 19 op/s 2023-09-10T04:03:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17064.060631 secs 2023-09-10T04:03:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17064.060631 secs 2023-09-10T04:03:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17064.060631 secs 2023-09-10T04:03:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17065.307547 secs 2023-09-10T04:03:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:54 smithi170 ceph-mon[102832]: pgmap v9262: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 247 KiB/s wr, 12 op/s 2023-09-10T04:03:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17065.307547 secs 2023-09-10T04:03:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:54 smithi082 ceph-mon[98694]: pgmap v9262: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 247 KiB/s wr, 12 op/s 2023-09-10T04:03:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17065.307547 secs 2023-09-10T04:03:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:54 smithi098 ceph-mon[103124]: pgmap v9262: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 247 KiB/s wr, 12 op/s 2023-09-10T04:03:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17065.970881 secs 2023-09-10T04:03:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17065.970881 secs 2023-09-10T04:03:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17065.970881 secs 2023-09-10T04:03:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:03:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:03:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:03:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:56 smithi170 ceph-mon[102832]: pgmap v9263: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 378 KiB/s wr, 18 op/s 2023-09-10T04:03:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:03:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:03:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:03:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:56 smithi082 ceph-mon[98694]: pgmap v9263: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 378 KiB/s wr, 18 op/s 2023-09-10T04:03:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:03:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:03:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:03:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:56 smithi098 ceph-mon[103124]: pgmap v9263: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 378 KiB/s wr, 18 op/s 2023-09-10T04:03:57.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:57 smithi170 ceph-mon[102832]: pgmap v9264: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T04:03:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:57 smithi082 ceph-mon[98694]: pgmap v9264: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T04:03:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:57 smithi098 ceph-mon[103124]: pgmap v9264: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T04:03:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17069.060698 secs 2023-09-10T04:03:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17069.060698 secs 2023-09-10T04:03:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17069.060698 secs 2023-09-10T04:03:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17070.307606 secs 2023-09-10T04:03:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:03:59 smithi170 ceph-mon[102832]: pgmap v9265: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T04:03:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17070.307606 secs 2023-09-10T04:03:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:03:59 smithi082 ceph-mon[98694]: pgmap v9265: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T04:03:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17070.307606 secs 2023-09-10T04:03:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:03:59 smithi098 ceph-mon[103124]: pgmap v9265: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T04:04:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17070.970943 secs 2023-09-10T04:04:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17070.970943 secs 2023-09-10T04:04:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17070.970943 secs 2023-09-10T04:04:01.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:01 smithi082 ceph-mon[98694]: pgmap v9266: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 250 KiB/s wr, 16 op/s 2023-09-10T04:04:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:01 smithi098 ceph-mon[103124]: pgmap v9266: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 250 KiB/s wr, 16 op/s 2023-09-10T04:04:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:01 smithi170 ceph-mon[102832]: pgmap v9266: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 250 KiB/s wr, 16 op/s 2023-09-10T04:04:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17074.060759 secs 2023-09-10T04:04:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17074.060759 secs 2023-09-10T04:04:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17074.060759 secs 2023-09-10T04:04:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17075.307665 secs 2023-09-10T04:04:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:04 smithi170 ceph-mon[102832]: pgmap v9267: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 170 KiB/s wr, 10 op/s 2023-09-10T04:04:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17075.307665 secs 2023-09-10T04:04:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:04 smithi082 ceph-mon[98694]: pgmap v9267: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 170 KiB/s wr, 10 op/s 2023-09-10T04:04:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17075.307665 secs 2023-09-10T04:04:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:04 smithi098 ceph-mon[103124]: pgmap v9267: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 170 KiB/s wr, 10 op/s 2023-09-10T04:04:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17075.971008 secs 2023-09-10T04:04:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17075.971008 secs 2023-09-10T04:04:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17075.971008 secs 2023-09-10T04:04:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:06 smithi170 ceph-mon[102832]: pgmap v9268: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 214 KiB/s wr, 28 op/s 2023-09-10T04:04:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:06 smithi082 ceph-mon[98694]: pgmap v9268: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 214 KiB/s wr, 28 op/s 2023-09-10T04:04:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:06 smithi098 ceph-mon[103124]: pgmap v9268: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 214 KiB/s wr, 28 op/s 2023-09-10T04:04:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:07 smithi170 ceph-mon[102832]: pgmap v9269: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 84 KiB/s wr, 25 op/s 2023-09-10T04:04:07.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:07 smithi170 ceph-mon[102832]: osdmap e647: 12 total, 12 up, 12 in 2023-09-10T04:04:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:07 smithi082 ceph-mon[98694]: pgmap v9269: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 84 KiB/s wr, 25 op/s 2023-09-10T04:04:07.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:07 smithi082 ceph-mon[98694]: osdmap e647: 12 total, 12 up, 12 in 2023-09-10T04:04:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:07 smithi098 ceph-mon[103124]: pgmap v9269: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 84 KiB/s wr, 25 op/s 2023-09-10T04:04:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:07 smithi098 ceph-mon[103124]: osdmap e647: 12 total, 12 up, 12 in 2023-09-10T04:04:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17079.060775 secs 2023-09-10T04:04:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17079.060775 secs 2023-09-10T04:04:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17079.060775 secs 2023-09-10T04:04:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17080.307723 secs 2023-09-10T04:04:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:09 smithi082 ceph-mon[98694]: pgmap v9271: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T04:04:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17080.307723 secs 2023-09-10T04:04:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:09 smithi098 ceph-mon[103124]: pgmap v9271: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T04:04:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17080.307723 secs 2023-09-10T04:04:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:09 smithi170 ceph-mon[102832]: pgmap v9271: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 101 KiB/s wr, 29 op/s 2023-09-10T04:04:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17080.971070 secs 2023-09-10T04:04:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17080.971070 secs 2023-09-10T04:04:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17080.971070 secs 2023-09-10T04:04:11.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:11 smithi082 ceph-mon[98694]: pgmap v9272: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-09-10T04:04:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:11 smithi098 ceph-mon[103124]: pgmap v9272: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-09-10T04:04:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:11 smithi170 ceph-mon[102832]: pgmap v9272: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-09-10T04:04:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17084.060854 secs 2023-09-10T04:04:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17084.060854 secs 2023-09-10T04:04:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17084.060854 secs 2023-09-10T04:04:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17085.307781 secs 2023-09-10T04:04:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:14 smithi170 ceph-mon[102832]: osdmap e648: 12 total, 12 up, 12 in 2023-09-10T04:04:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:14 smithi170 ceph-mon[102832]: pgmap v9274: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2023-09-10T04:04:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17085.307781 secs 2023-09-10T04:04:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:14 smithi082 ceph-mon[98694]: osdmap e648: 12 total, 12 up, 12 in 2023-09-10T04:04:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:14 smithi082 ceph-mon[98694]: pgmap v9274: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2023-09-10T04:04:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17085.307781 secs 2023-09-10T04:04:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:14 smithi098 ceph-mon[103124]: osdmap e648: 12 total, 12 up, 12 in 2023-09-10T04:04:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:14 smithi098 ceph-mon[103124]: pgmap v9274: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2023-09-10T04:04:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17085.971136 secs 2023-09-10T04:04:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17085.971136 secs 2023-09-10T04:04:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17085.971136 secs 2023-09-10T04:04:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:16 smithi170 ceph-mon[102832]: pgmap v9275: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 127 B/s wr, 12 op/s 2023-09-10T04:04:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:16 smithi082 ceph-mon[98694]: pgmap v9275: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 127 B/s wr, 12 op/s 2023-09-10T04:04:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:16 smithi098 ceph-mon[103124]: pgmap v9275: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 127 B/s wr, 12 op/s 2023-09-10T04:04:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:17 smithi170 ceph-mon[102832]: pgmap v9276: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 11 op/s 2023-09-10T04:04:17.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:17 smithi082 ceph-mon[98694]: pgmap v9276: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 11 op/s 2023-09-10T04:04:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:17 smithi098 ceph-mon[103124]: pgmap v9276: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 11 op/s 2023-09-10T04:04:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17089.060915 secs 2023-09-10T04:04:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17089.060915 secs 2023-09-10T04:04:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17089.060915 secs 2023-09-10T04:04:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17090.315123 secs 2023-09-10T04:04:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:19 smithi082 ceph-mon[98694]: pgmap v9277: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T04:04:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17090.315123 secs 2023-09-10T04:04:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:19 smithi098 ceph-mon[103124]: pgmap v9277: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T04:04:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17090.315123 secs 2023-09-10T04:04:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:19 smithi170 ceph-mon[102832]: pgmap v9277: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T04:04:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17090.971202 secs 2023-09-10T04:04:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17090.971202 secs 2023-09-10T04:04:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17090.971202 secs 2023-09-10T04:04:21.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:21 smithi082 ceph-mon[98694]: pgmap v9278: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 614 B/s wr, 11 op/s 2023-09-10T04:04:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:21 smithi098 ceph-mon[103124]: pgmap v9278: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 614 B/s wr, 11 op/s 2023-09-10T04:04:21.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:21 smithi170 ceph-mon[102832]: pgmap v9278: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 614 B/s wr, 11 op/s 2023-09-10T04:04:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17094.060973 secs 2023-09-10T04:04:22.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17094.060973 secs 2023-09-10T04:04:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17094.060973 secs 2023-09-10T04:04:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17095.315210 secs 2023-09-10T04:04:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:24 smithi170 ceph-mon[102832]: pgmap v9279: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 612 B/s wr, 11 op/s 2023-09-10T04:04:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17095.315210 secs 2023-09-10T04:04:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:24 smithi082 ceph-mon[98694]: pgmap v9279: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 612 B/s wr, 11 op/s 2023-09-10T04:04:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17095.315210 secs 2023-09-10T04:04:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:24 smithi098 ceph-mon[103124]: pgmap v9279: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 612 B/s wr, 11 op/s 2023-09-10T04:04:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17095.971269 secs 2023-09-10T04:04:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17095.971269 secs 2023-09-10T04:04:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17095.971269 secs 2023-09-10T04:04:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:26 smithi170 ceph-mon[102832]: pgmap v9280: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 13 op/s 2023-09-10T04:04:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:26 smithi082 ceph-mon[98694]: pgmap v9280: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 13 op/s 2023-09-10T04:04:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:26 smithi098 ceph-mon[103124]: pgmap v9280: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 13 op/s 2023-09-10T04:04:27.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:27 smithi170 ceph-mon[102832]: pgmap v9281: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T04:04:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:27 smithi082 ceph-mon[98694]: pgmap v9281: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T04:04:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:27 smithi098 ceph-mon[103124]: pgmap v9281: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T04:04:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17100.315267 secs 2023-09-10T04:04:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17100.315267 secs 2023-09-10T04:04:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17100.315267 secs 2023-09-10T04:04:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17100.971331 secs 2023-09-10T04:04:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:30 smithi170 ceph-mon[102832]: pgmap v9282: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T04:04:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17100.971331 secs 2023-09-10T04:04:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:30 smithi082 ceph-mon[98694]: pgmap v9282: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T04:04:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17100.971331 secs 2023-09-10T04:04:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:30 smithi098 ceph-mon[103124]: pgmap v9282: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T04:04:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:04:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:04:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:04:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:04:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:04:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:04:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:32 smithi170 ceph-mon[102832]: pgmap v9283: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 14 op/s 2023-09-10T04:04:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:04:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:04:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:04:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:32 smithi082 ceph-mon[98694]: pgmap v9283: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 14 op/s 2023-09-10T04:04:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:04:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:04:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:04:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:32 smithi098 ceph-mon[103124]: pgmap v9283: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 14 op/s 2023-09-10T04:04:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17105.315328 secs 2023-09-10T04:04:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:34 smithi170 ceph-mon[102832]: pgmap v9284: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 9 op/s 2023-09-10T04:04:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17105.315328 secs 2023-09-10T04:04:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:34 smithi082 ceph-mon[98694]: pgmap v9284: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 9 op/s 2023-09-10T04:04:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17105.315328 secs 2023-09-10T04:04:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:34 smithi098 ceph-mon[103124]: pgmap v9284: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 9 op/s 2023-09-10T04:04:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17105.971395 secs 2023-09-10T04:04:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17105.971395 secs 2023-09-10T04:04:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17105.971395 secs 2023-09-10T04:04:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:36 smithi170 ceph-mon[102832]: pgmap v9285: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 147 KiB/s wr, 13 op/s 2023-09-10T04:04:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:36 smithi082 ceph-mon[98694]: pgmap v9285: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 147 KiB/s wr, 13 op/s 2023-09-10T04:04:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:36 smithi098 ceph-mon[103124]: pgmap v9285: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 147 KiB/s wr, 13 op/s 2023-09-10T04:04:37.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:37 smithi170 ceph-mon[102832]: pgmap v9286: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 357 KiB/s wr, 12 op/s 2023-09-10T04:04:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:37 smithi082 ceph-mon[98694]: pgmap v9286: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 357 KiB/s wr, 12 op/s 2023-09-10T04:04:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:37 smithi098 ceph-mon[103124]: pgmap v9286: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 357 KiB/s wr, 12 op/s 2023-09-10T04:04:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17110.315384 secs 2023-09-10T04:04:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17110.315384 secs 2023-09-10T04:04:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17110.315384 secs 2023-09-10T04:04:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17110.971461 secs 2023-09-10T04:04:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:40 smithi170 ceph-mon[102832]: pgmap v9287: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 299 KiB/s wr, 11 op/s 2023-09-10T04:04:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17110.971461 secs 2023-09-10T04:04:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:40 smithi082 ceph-mon[98694]: pgmap v9287: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 299 KiB/s wr, 11 op/s 2023-09-10T04:04:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17110.971461 secs 2023-09-10T04:04:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:40 smithi098 ceph-mon[103124]: pgmap v9287: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 299 KiB/s wr, 11 op/s 2023-09-10T04:04:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:41 smithi170 ceph-mon[102832]: pgmap v9288: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 385 KiB/s wr, 17 op/s 2023-09-10T04:04:41.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:41 smithi082 ceph-mon[98694]: pgmap v9288: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 385 KiB/s wr, 17 op/s 2023-09-10T04:04:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:41 smithi098 ceph-mon[103124]: pgmap v9288: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 385 KiB/s wr, 17 op/s 2023-09-10T04:04:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17115.315469 secs 2023-09-10T04:04:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:44 smithi170 ceph-mon[102832]: pgmap v9289: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 323 KiB/s wr, 13 op/s 2023-09-10T04:04:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17115.315469 secs 2023-09-10T04:04:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:44 smithi082 ceph-mon[98694]: pgmap v9289: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 323 KiB/s wr, 13 op/s 2023-09-10T04:04:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17115.315469 secs 2023-09-10T04:04:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:44 smithi098 ceph-mon[103124]: pgmap v9289: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 323 KiB/s wr, 13 op/s 2023-09-10T04:04:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17115.971486 secs 2023-09-10T04:04:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17115.971486 secs 2023-09-10T04:04:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17115.971486 secs 2023-09-10T04:04:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:46 smithi170 ceph-mon[102832]: pgmap v9290: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 559 KiB/s wr, 19 op/s 2023-09-10T04:04:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:46 smithi082 ceph-mon[98694]: pgmap v9290: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 559 KiB/s wr, 19 op/s 2023-09-10T04:04:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:46 smithi098 ceph-mon[103124]: pgmap v9290: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 559 KiB/s wr, 19 op/s 2023-09-10T04:04:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:47 smithi170 ceph-mon[102832]: pgmap v9291: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 532 KiB/s wr, 16 op/s 2023-09-10T04:04:47.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:47 smithi082 ceph-mon[98694]: pgmap v9291: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 532 KiB/s wr, 16 op/s 2023-09-10T04:04:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:47 smithi098 ceph-mon[103124]: pgmap v9291: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 532 KiB/s wr, 16 op/s 2023-09-10T04:04:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17119.061316 secs 2023-09-10T04:04:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:04:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17119.061316 secs 2023-09-10T04:04:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:04:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17119.061316 secs 2023-09-10T04:04:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:04:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17120.315497 secs 2023-09-10T04:04:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:04:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:04:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:04:49.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:49 smithi082 ceph-mon[98694]: pgmap v9292: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 321 KiB/s wr, 14 op/s 2023-09-10T04:04:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17120.315497 secs 2023-09-10T04:04:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:04:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:04:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:04:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:49 smithi098 ceph-mon[103124]: pgmap v9292: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 321 KiB/s wr, 14 op/s 2023-09-10T04:04:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17120.315497 secs 2023-09-10T04:04:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:04:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:04:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:04:49.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:49 smithi170 ceph-mon[102832]: pgmap v9292: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 321 KiB/s wr, 14 op/s 2023-09-10T04:04:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17120.971551 secs 2023-09-10T04:04:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17120.971551 secs 2023-09-10T04:04:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17120.971551 secs 2023-09-10T04:04:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:51 smithi082 ceph-mon[98694]: pgmap v9293: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 322 KiB/s wr, 19 op/s 2023-09-10T04:04:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:51 smithi098 ceph-mon[103124]: pgmap v9293: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 322 KiB/s wr, 19 op/s 2023-09-10T04:04:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:51 smithi170 ceph-mon[102832]: pgmap v9293: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 322 KiB/s wr, 19 op/s 2023-09-10T04:04:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:04:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:04:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:04:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:04:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:04:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:04:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:04:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:04:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:04:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17124.061405 secs 2023-09-10T04:04:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:53 smithi082 ceph-mon[98694]: pgmap v9294: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 236 KiB/s wr, 12 op/s 2023-09-10T04:04:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17124.061405 secs 2023-09-10T04:04:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:53 smithi098 ceph-mon[103124]: pgmap v9294: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 236 KiB/s wr, 12 op/s 2023-09-10T04:04:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17124.061405 secs 2023-09-10T04:04:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:53 smithi170 ceph-mon[102832]: pgmap v9294: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 236 KiB/s wr, 12 op/s 2023-09-10T04:04:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17125.315583 secs 2023-09-10T04:04:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17125.315583 secs 2023-09-10T04:04:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17125.315583 secs 2023-09-10T04:04:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17125.971617 secs 2023-09-10T04:04:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:55 smithi082 ceph-mon[98694]: pgmap v9295: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 359 KiB/s wr, 17 op/s 2023-09-10T04:04:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17125.971617 secs 2023-09-10T04:04:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:55 smithi098 ceph-mon[103124]: pgmap v9295: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 359 KiB/s wr, 17 op/s 2023-09-10T04:04:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17125.971617 secs 2023-09-10T04:04:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:55 smithi170 ceph-mon[102832]: pgmap v9295: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 359 KiB/s wr, 17 op/s 2023-09-10T04:04:57.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:57 smithi082 ceph-mon[98694]: pgmap v9296: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T04:04:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:57 smithi098 ceph-mon[103124]: pgmap v9296: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T04:04:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:57 smithi170 ceph-mon[102832]: pgmap v9296: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T04:04:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17129.061515 secs 2023-09-10T04:04:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17129.061515 secs 2023-09-10T04:04:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17129.061515 secs 2023-09-10T04:04:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17130.315642 secs 2023-09-10T04:04:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:04:59 smithi082 ceph-mon[98694]: pgmap v9297: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T04:04:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17130.315642 secs 2023-09-10T04:04:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:04:59 smithi098 ceph-mon[103124]: pgmap v9297: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T04:04:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17130.315642 secs 2023-09-10T04:04:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:04:59 smithi170 ceph-mon[102832]: pgmap v9297: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T04:05:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17130.971681 secs 2023-09-10T04:05:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17130.971681 secs 2023-09-10T04:05:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17130.971681 secs 2023-09-10T04:05:01.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:01 smithi082 ceph-mon[98694]: pgmap v9298: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 163 KiB/s wr, 15 op/s 2023-09-10T04:05:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:01 smithi098 ceph-mon[103124]: pgmap v9298: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 163 KiB/s wr, 15 op/s 2023-09-10T04:05:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:01 smithi170 ceph-mon[102832]: pgmap v9298: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 163 KiB/s wr, 15 op/s 2023-09-10T04:05:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17134.061575 secs 2023-09-10T04:05:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17134.061575 secs 2023-09-10T04:05:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17134.061575 secs 2023-09-10T04:05:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17135.315724 secs 2023-09-10T04:05:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:04 smithi170 ceph-mon[102832]: pgmap v9299: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 163 KiB/s wr, 9 op/s 2023-09-10T04:05:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17135.315724 secs 2023-09-10T04:05:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:04 smithi082 ceph-mon[98694]: pgmap v9299: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 163 KiB/s wr, 9 op/s 2023-09-10T04:05:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17135.315724 secs 2023-09-10T04:05:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:04 smithi098 ceph-mon[103124]: pgmap v9299: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 163 KiB/s wr, 9 op/s 2023-09-10T04:05:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17135.971747 secs 2023-09-10T04:05:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17135.971747 secs 2023-09-10T04:05:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17135.971747 secs 2023-09-10T04:05:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:06 smithi170 ceph-mon[102832]: pgmap v9300: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 208 KiB/s wr, 28 op/s 2023-09-10T04:05:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:06 smithi082 ceph-mon[98694]: pgmap v9300: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 208 KiB/s wr, 28 op/s 2023-09-10T04:05:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:06 smithi098 ceph-mon[103124]: pgmap v9300: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 208 KiB/s wr, 28 op/s 2023-09-10T04:05:07.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:07 smithi082 ceph-mon[98694]: pgmap v9301: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 84 KiB/s wr, 24 op/s 2023-09-10T04:05:07.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:07 smithi082 ceph-mon[98694]: osdmap e649: 12 total, 12 up, 12 in 2023-09-10T04:05:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:07 smithi098 ceph-mon[103124]: pgmap v9301: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 84 KiB/s wr, 24 op/s 2023-09-10T04:05:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:07 smithi098 ceph-mon[103124]: osdmap e649: 12 total, 12 up, 12 in 2023-09-10T04:05:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:07 smithi170 ceph-mon[102832]: pgmap v9301: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 84 KiB/s wr, 24 op/s 2023-09-10T04:05:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:07 smithi170 ceph-mon[102832]: osdmap e649: 12 total, 12 up, 12 in 2023-09-10T04:05:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17139.061634 secs 2023-09-10T04:05:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17139.061634 secs 2023-09-10T04:05:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17139.061634 secs 2023-09-10T04:05:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17140.315812 secs 2023-09-10T04:05:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:09 smithi082 ceph-mon[98694]: pgmap v9303: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 101 KiB/s wr, 28 op/s 2023-09-10T04:05:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17140.315812 secs 2023-09-10T04:05:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:09 smithi098 ceph-mon[103124]: pgmap v9303: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 101 KiB/s wr, 28 op/s 2023-09-10T04:05:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17140.315812 secs 2023-09-10T04:05:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:09 smithi170 ceph-mon[102832]: pgmap v9303: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 101 KiB/s wr, 28 op/s 2023-09-10T04:05:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17140.971813 secs 2023-09-10T04:05:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17140.971813 secs 2023-09-10T04:05:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17140.971813 secs 2023-09-10T04:05:11.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:11 smithi082 ceph-mon[98694]: pgmap v9304: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T04:05:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:11 smithi098 ceph-mon[103124]: pgmap v9304: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T04:05:11.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:11 smithi170 ceph-mon[102832]: pgmap v9304: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T04:05:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17144.061692 secs 2023-09-10T04:05:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17144.061692 secs 2023-09-10T04:05:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17144.061692 secs 2023-09-10T04:05:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17145.315868 secs 2023-09-10T04:05:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:14 smithi170 ceph-mon[102832]: osdmap e650: 12 total, 12 up, 12 in 2023-09-10T04:05:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:14 smithi170 ceph-mon[102832]: pgmap v9306: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T04:05:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17145.315868 secs 2023-09-10T04:05:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:14 smithi082 ceph-mon[98694]: osdmap e650: 12 total, 12 up, 12 in 2023-09-10T04:05:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:14 smithi082 ceph-mon[98694]: pgmap v9306: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T04:05:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17145.315868 secs 2023-09-10T04:05:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:14 smithi098 ceph-mon[103124]: osdmap e650: 12 total, 12 up, 12 in 2023-09-10T04:05:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:14 smithi098 ceph-mon[103124]: pgmap v9306: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T04:05:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17145.971878 secs 2023-09-10T04:05:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17145.971878 secs 2023-09-10T04:05:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17145.971878 secs 2023-09-10T04:05:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:16 smithi170 ceph-mon[102832]: pgmap v9307: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 13 op/s 2023-09-10T04:05:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:16 smithi082 ceph-mon[98694]: pgmap v9307: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 13 op/s 2023-09-10T04:05:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:16 smithi098 ceph-mon[103124]: pgmap v9307: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 13 op/s 2023-09-10T04:05:17.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:17 smithi082 ceph-mon[98694]: pgmap v9308: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T04:05:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:17 smithi098 ceph-mon[103124]: pgmap v9308: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T04:05:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:17 smithi170 ceph-mon[102832]: pgmap v9308: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T04:05:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17149.061769 secs 2023-09-10T04:05:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17149.061769 secs 2023-09-10T04:05:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17149.061769 secs 2023-09-10T04:05:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17150.315926 secs 2023-09-10T04:05:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:19 smithi082 ceph-mon[98694]: pgmap v9309: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T04:05:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17150.315926 secs 2023-09-10T04:05:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:19 smithi098 ceph-mon[103124]: pgmap v9309: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T04:05:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17150.315926 secs 2023-09-10T04:05:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:19 smithi170 ceph-mon[102832]: pgmap v9309: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 11 op/s 2023-09-10T04:05:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17150.971944 secs 2023-09-10T04:05:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17150.971944 secs 2023-09-10T04:05:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17150.971944 secs 2023-09-10T04:05:21.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:21 smithi082 ceph-mon[98694]: pgmap v9310: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T04:05:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:21 smithi098 ceph-mon[103124]: pgmap v9310: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T04:05:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:21 smithi170 ceph-mon[102832]: pgmap v9310: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T04:05:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17154.061829 secs 2023-09-10T04:05:22.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17154.061829 secs 2023-09-10T04:05:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17154.061829 secs 2023-09-10T04:05:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17155.315983 secs 2023-09-10T04:05:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:24 smithi170 ceph-mon[102832]: pgmap v9311: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T04:05:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17155.315983 secs 2023-09-10T04:05:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:24 smithi082 ceph-mon[98694]: pgmap v9311: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T04:05:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17155.315983 secs 2023-09-10T04:05:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:24 smithi098 ceph-mon[103124]: pgmap v9311: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T04:05:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17155.972008 secs 2023-09-10T04:05:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17155.972008 secs 2023-09-10T04:05:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17155.972008 secs 2023-09-10T04:05:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:26 smithi170 ceph-mon[102832]: pgmap v9312: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 106 KiB/s wr, 13 op/s 2023-09-10T04:05:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:26 smithi082 ceph-mon[98694]: pgmap v9312: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 106 KiB/s wr, 13 op/s 2023-09-10T04:05:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:26 smithi098 ceph-mon[103124]: pgmap v9312: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 106 KiB/s wr, 13 op/s 2023-09-10T04:05:27.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:27 smithi082 ceph-mon[98694]: pgmap v9313: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T04:05:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:27 smithi098 ceph-mon[103124]: pgmap v9313: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T04:05:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:27 smithi170 ceph-mon[102832]: pgmap v9313: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T04:05:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17160.316041 secs 2023-09-10T04:05:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17160.316041 secs 2023-09-10T04:05:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17160.316041 secs 2023-09-10T04:05:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17160.972073 secs 2023-09-10T04:05:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:30 smithi170 ceph-mon[102832]: pgmap v9314: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T04:05:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17160.972073 secs 2023-09-10T04:05:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:30 smithi082 ceph-mon[98694]: pgmap v9314: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T04:05:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17160.972073 secs 2023-09-10T04:05:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:30 smithi098 ceph-mon[103124]: pgmap v9314: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T04:05:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:05:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:05:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:05:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:05:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:05:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:05:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:32 smithi170 ceph-mon[102832]: pgmap v9315: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 121 KiB/s wr, 13 op/s 2023-09-10T04:05:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:05:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:05:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:05:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:32 smithi082 ceph-mon[98694]: pgmap v9315: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 121 KiB/s wr, 13 op/s 2023-09-10T04:05:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:05:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:05:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:05:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:32 smithi098 ceph-mon[103124]: pgmap v9315: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 121 KiB/s wr, 13 op/s 2023-09-10T04:05:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17165.316113 secs 2023-09-10T04:05:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:34 smithi170 ceph-mon[102832]: pgmap v9316: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 9 op/s 2023-09-10T04:05:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17165.316113 secs 2023-09-10T04:05:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:34 smithi082 ceph-mon[98694]: pgmap v9316: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 9 op/s 2023-09-10T04:05:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17165.316113 secs 2023-09-10T04:05:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:34 smithi098 ceph-mon[103124]: pgmap v9316: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 9 op/s 2023-09-10T04:05:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17165.972139 secs 2023-09-10T04:05:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17165.972139 secs 2023-09-10T04:05:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17165.972139 secs 2023-09-10T04:05:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:36 smithi170 ceph-mon[102832]: pgmap v9317: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 185 KiB/s wr, 14 op/s 2023-09-10T04:05:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:36 smithi082 ceph-mon[98694]: pgmap v9317: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 185 KiB/s wr, 14 op/s 2023-09-10T04:05:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:36 smithi098 ceph-mon[103124]: pgmap v9317: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 185 KiB/s wr, 14 op/s 2023-09-10T04:05:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:37 smithi082 ceph-mon[98694]: pgmap v9318: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 10 op/s 2023-09-10T04:05:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:37 smithi098 ceph-mon[103124]: pgmap v9318: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 10 op/s 2023-09-10T04:05:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:37 smithi170 ceph-mon[102832]: pgmap v9318: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 10 op/s 2023-09-10T04:05:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17170.316202 secs 2023-09-10T04:05:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17170.316202 secs 2023-09-10T04:05:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17170.316202 secs 2023-09-10T04:05:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17170.972207 secs 2023-09-10T04:05:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:40 smithi170 ceph-mon[102832]: pgmap v9319: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 185 KiB/s wr, 10 op/s 2023-09-10T04:05:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17170.972207 secs 2023-09-10T04:05:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:40 smithi082 ceph-mon[98694]: pgmap v9319: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 185 KiB/s wr, 10 op/s 2023-09-10T04:05:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17170.972207 secs 2023-09-10T04:05:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:40 smithi098 ceph-mon[103124]: pgmap v9319: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 185 KiB/s wr, 10 op/s 2023-09-10T04:05:41.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:41 smithi082 ceph-mon[98694]: pgmap v9320: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 222 KiB/s wr, 16 op/s 2023-09-10T04:05:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:41 smithi098 ceph-mon[103124]: pgmap v9320: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 222 KiB/s wr, 16 op/s 2023-09-10T04:05:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:41 smithi170 ceph-mon[102832]: pgmap v9320: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 222 KiB/s wr, 16 op/s 2023-09-10T04:05:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17175.316292 secs 2023-09-10T04:05:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:44 smithi170 ceph-mon[102832]: pgmap v9321: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T04:05:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17175.316292 secs 2023-09-10T04:05:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:44 smithi082 ceph-mon[98694]: pgmap v9321: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T04:05:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17175.316292 secs 2023-09-10T04:05:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:44 smithi098 ceph-mon[103124]: pgmap v9321: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T04:05:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17175.972271 secs 2023-09-10T04:05:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17175.972271 secs 2023-09-10T04:05:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17175.972271 secs 2023-09-10T04:05:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:46 smithi170 ceph-mon[102832]: pgmap v9322: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 382 KiB/s wr, 18 op/s 2023-09-10T04:05:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:46 smithi082 ceph-mon[98694]: pgmap v9322: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 382 KiB/s wr, 18 op/s 2023-09-10T04:05:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:46 smithi098 ceph-mon[103124]: pgmap v9322: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 382 KiB/s wr, 18 op/s 2023-09-10T04:05:47.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:47 smithi082 ceph-mon[98694]: pgmap v9323: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 260 KiB/s wr, 13 op/s 2023-09-10T04:05:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:47 smithi098 ceph-mon[103124]: pgmap v9323: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 260 KiB/s wr, 13 op/s 2023-09-10T04:05:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:47 smithi170 ceph-mon[102832]: pgmap v9323: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 260 KiB/s wr, 13 op/s 2023-09-10T04:05:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17180.316350 secs 2023-09-10T04:05:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:05:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17180.316350 secs 2023-09-10T04:05:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:05:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17180.316350 secs 2023-09-10T04:05:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:05:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17180.972336 secs 2023-09-10T04:05:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:50 smithi170 ceph-mon[102832]: pgmap v9324: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 260 KiB/s wr, 13 op/s 2023-09-10T04:05:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:05:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:05:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:05:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17180.972336 secs 2023-09-10T04:05:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:50 smithi082 ceph-mon[98694]: pgmap v9324: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 260 KiB/s wr, 13 op/s 2023-09-10T04:05:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:05:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:05:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:05:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17180.972336 secs 2023-09-10T04:05:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:50 smithi098 ceph-mon[103124]: pgmap v9324: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 260 KiB/s wr, 13 op/s 2023-09-10T04:05:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:05:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:05:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:05:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:51 smithi082 ceph-mon[98694]: pgmap v9325: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 336 KiB/s wr, 18 op/s 2023-09-10T04:05:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:51 smithi098 ceph-mon[103124]: pgmap v9325: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 336 KiB/s wr, 18 op/s 2023-09-10T04:05:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:51 smithi170 ceph-mon[102832]: pgmap v9325: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 336 KiB/s wr, 18 op/s 2023-09-10T04:05:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17185.316403 secs 2023-09-10T04:05:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:54 smithi170 ceph-mon[102832]: pgmap v9326: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 299 KiB/s wr, 11 op/s 2023-09-10T04:05:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17185.316403 secs 2023-09-10T04:05:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:54 smithi082 ceph-mon[98694]: pgmap v9326: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 299 KiB/s wr, 11 op/s 2023-09-10T04:05:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17185.316403 secs 2023-09-10T04:05:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:54 smithi098 ceph-mon[103124]: pgmap v9326: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 299 KiB/s wr, 11 op/s 2023-09-10T04:05:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17185.972403 secs 2023-09-10T04:05:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17185.972403 secs 2023-09-10T04:05:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17185.972403 secs 2023-09-10T04:05:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:56 smithi170 ceph-mon[102832]: pgmap v9327: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 326 KiB/s wr, 16 op/s 2023-09-10T04:05:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:56 smithi082 ceph-mon[98694]: pgmap v9327: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 326 KiB/s wr, 16 op/s 2023-09-10T04:05:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:56 smithi098 ceph-mon[103124]: pgmap v9327: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 326 KiB/s wr, 16 op/s 2023-09-10T04:05:57.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:57 smithi082 ceph-mon[98694]: pgmap v9328: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 267 KiB/s wr, 12 op/s 2023-09-10T04:05:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:57 smithi098 ceph-mon[103124]: pgmap v9328: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 267 KiB/s wr, 12 op/s 2023-09-10T04:05:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:57 smithi170 ceph-mon[102832]: pgmap v9328: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 267 KiB/s wr, 12 op/s 2023-09-10T04:05:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17189.062296 secs 2023-09-10T04:05:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17189.062296 secs 2023-09-10T04:05:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17189.062296 secs 2023-09-10T04:05:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17190.316472 secs 2023-09-10T04:05:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:05:59 smithi082 ceph-mon[98694]: pgmap v9329: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 267 KiB/s wr, 12 op/s 2023-09-10T04:05:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17190.316472 secs 2023-09-10T04:05:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:05:59 smithi098 ceph-mon[103124]: pgmap v9329: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 267 KiB/s wr, 12 op/s 2023-09-10T04:05:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17190.316472 secs 2023-09-10T04:05:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:05:59 smithi170 ceph-mon[102832]: pgmap v9329: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 267 KiB/s wr, 12 op/s 2023-09-10T04:06:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17190.972472 secs 2023-09-10T04:06:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:06:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:06:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:06:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17190.972472 secs 2023-09-10T04:06:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:06:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:06:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:06:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17190.972472 secs 2023-09-10T04:06:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:06:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:06:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:06:01.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:01 smithi082 ceph-mon[98694]: pgmap v9330: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 365 KiB/s wr, 18 op/s 2023-09-10T04:06:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:01 smithi098 ceph-mon[103124]: pgmap v9330: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 365 KiB/s wr, 18 op/s 2023-09-10T04:06:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:01 smithi170 ceph-mon[102832]: pgmap v9330: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 365 KiB/s wr, 18 op/s 2023-09-10T04:06:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17194.062356 secs 2023-09-10T04:06:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17194.062356 secs 2023-09-10T04:06:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17194.062356 secs 2023-09-10T04:06:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17195.316517 secs 2023-09-10T04:06:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:04 smithi170 ceph-mon[102832]: pgmap v9331: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 289 KiB/s wr, 12 op/s 2023-09-10T04:06:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17195.316517 secs 2023-09-10T04:06:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:04 smithi082 ceph-mon[98694]: pgmap v9331: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 289 KiB/s wr, 12 op/s 2023-09-10T04:06:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17195.316517 secs 2023-09-10T04:06:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:04 smithi098 ceph-mon[103124]: pgmap v9331: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 289 KiB/s wr, 12 op/s 2023-09-10T04:06:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17195.972535 secs 2023-09-10T04:06:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17195.972535 secs 2023-09-10T04:06:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17195.972535 secs 2023-09-10T04:06:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:06 smithi170 ceph-mon[102832]: pgmap v9332: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 386 KiB/s wr, 30 op/s 2023-09-10T04:06:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:06 smithi082 ceph-mon[98694]: pgmap v9332: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 386 KiB/s wr, 30 op/s 2023-09-10T04:06:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:06 smithi098 ceph-mon[103124]: pgmap v9332: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 386 KiB/s wr, 30 op/s 2023-09-10T04:06:07.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:07 smithi082 ceph-mon[98694]: pgmap v9333: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 442 KiB/s wr, 27 op/s 2023-09-10T04:06:07.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:07 smithi082 ceph-mon[98694]: osdmap e651: 12 total, 12 up, 12 in 2023-09-10T04:06:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:07 smithi098 ceph-mon[103124]: pgmap v9333: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 442 KiB/s wr, 27 op/s 2023-09-10T04:06:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:07 smithi098 ceph-mon[103124]: osdmap e651: 12 total, 12 up, 12 in 2023-09-10T04:06:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:07 smithi170 ceph-mon[102832]: pgmap v9333: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 442 KiB/s wr, 27 op/s 2023-09-10T04:06:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:07 smithi170 ceph-mon[102832]: osdmap e651: 12 total, 12 up, 12 in 2023-09-10T04:06:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17199.062418 secs 2023-09-10T04:06:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17199.062418 secs 2023-09-10T04:06:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17199.062418 secs 2023-09-10T04:06:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17200.316574 secs 2023-09-10T04:06:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:09 smithi082 ceph-mon[98694]: pgmap v9335: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 334 KiB/s wr, 30 op/s 2023-09-10T04:06:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17200.316574 secs 2023-09-10T04:06:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:09 smithi098 ceph-mon[103124]: pgmap v9335: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 334 KiB/s wr, 30 op/s 2023-09-10T04:06:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17200.316574 secs 2023-09-10T04:06:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:09 smithi170 ceph-mon[102832]: pgmap v9335: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 334 KiB/s wr, 30 op/s 2023-09-10T04:06:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17200.972599 secs 2023-09-10T04:06:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17200.972599 secs 2023-09-10T04:06:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17200.972599 secs 2023-09-10T04:06:11.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:11 smithi082 ceph-mon[98694]: pgmap v9336: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 265 KiB/s wr, 29 op/s 2023-09-10T04:06:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:11 smithi098 ceph-mon[103124]: pgmap v9336: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 265 KiB/s wr, 29 op/s 2023-09-10T04:06:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:11 smithi170 ceph-mon[102832]: pgmap v9336: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 265 KiB/s wr, 29 op/s 2023-09-10T04:06:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17204.062502 secs 2023-09-10T04:06:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17204.062502 secs 2023-09-10T04:06:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17204.062502 secs 2023-09-10T04:06:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17205.316633 secs 2023-09-10T04:06:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:14 smithi170 ceph-mon[102832]: osdmap e652: 12 total, 12 up, 12 in 2023-09-10T04:06:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:14 smithi170 ceph-mon[102832]: pgmap v9338: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 186 KiB/s wr, 9 op/s 2023-09-10T04:06:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17205.316633 secs 2023-09-10T04:06:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:14 smithi082 ceph-mon[98694]: osdmap e652: 12 total, 12 up, 12 in 2023-09-10T04:06:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:14 smithi082 ceph-mon[98694]: pgmap v9338: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 186 KiB/s wr, 9 op/s 2023-09-10T04:06:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17205.316633 secs 2023-09-10T04:06:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:14 smithi098 ceph-mon[103124]: osdmap e652: 12 total, 12 up, 12 in 2023-09-10T04:06:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:14 smithi098 ceph-mon[103124]: pgmap v9338: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 186 KiB/s wr, 9 op/s 2023-09-10T04:06:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17205.972665 secs 2023-09-10T04:06:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17205.972665 secs 2023-09-10T04:06:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17205.972665 secs 2023-09-10T04:06:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:16 smithi170 ceph-mon[102832]: pgmap v9339: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 61 KiB/s wr, 15 op/s 2023-09-10T04:06:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:16 smithi082 ceph-mon[98694]: pgmap v9339: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 61 KiB/s wr, 15 op/s 2023-09-10T04:06:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:16 smithi098 ceph-mon[103124]: pgmap v9339: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 61 KiB/s wr, 15 op/s 2023-09-10T04:06:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:17 smithi082 ceph-mon[98694]: pgmap v9340: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 12 op/s 2023-09-10T04:06:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:17 smithi098 ceph-mon[103124]: pgmap v9340: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 12 op/s 2023-09-10T04:06:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:17 smithi170 ceph-mon[102832]: pgmap v9340: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 12 op/s 2023-09-10T04:06:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17209.062559 secs 2023-09-10T04:06:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17209.062559 secs 2023-09-10T04:06:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17209.062559 secs 2023-09-10T04:06:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17210.316692 secs 2023-09-10T04:06:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:19 smithi082 ceph-mon[98694]: pgmap v9341: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 12 op/s 2023-09-10T04:06:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17210.316692 secs 2023-09-10T04:06:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:19 smithi098 ceph-mon[103124]: pgmap v9341: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 12 op/s 2023-09-10T04:06:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17210.316692 secs 2023-09-10T04:06:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:19 smithi170 ceph-mon[102832]: pgmap v9341: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 12 op/s 2023-09-10T04:06:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17210.972733 secs 2023-09-10T04:06:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17210.972733 secs 2023-09-10T04:06:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17210.972733 secs 2023-09-10T04:06:21.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:21 smithi082 ceph-mon[98694]: pgmap v9342: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-10T04:06:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:21 smithi098 ceph-mon[103124]: pgmap v9342: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-10T04:06:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:21 smithi170 ceph-mon[102832]: pgmap v9342: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-10T04:06:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17214.062618 secs 2023-09-10T04:06:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17214.062618 secs 2023-09-10T04:06:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17214.062618 secs 2023-09-10T04:06:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17215.316752 secs 2023-09-10T04:06:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:24 smithi170 ceph-mon[102832]: pgmap v9343: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-10T04:06:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17215.316752 secs 2023-09-10T04:06:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:24 smithi082 ceph-mon[98694]: pgmap v9343: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-10T04:06:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17215.316752 secs 2023-09-10T04:06:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:24 smithi098 ceph-mon[103124]: pgmap v9343: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-10T04:06:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17215.972802 secs 2023-09-10T04:06:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17215.972802 secs 2023-09-10T04:06:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17215.972802 secs 2023-09-10T04:06:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:26 smithi170 ceph-mon[102832]: pgmap v9344: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 60 KiB/s wr, 14 op/s 2023-09-10T04:06:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:26 smithi082 ceph-mon[98694]: pgmap v9344: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 60 KiB/s wr, 14 op/s 2023-09-10T04:06:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:26 smithi098 ceph-mon[103124]: pgmap v9344: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 60 KiB/s wr, 14 op/s 2023-09-10T04:06:27.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:27 smithi082 ceph-mon[98694]: pgmap v9345: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 9 op/s 2023-09-10T04:06:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:27 smithi098 ceph-mon[103124]: pgmap v9345: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 9 op/s 2023-09-10T04:06:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:27 smithi170 ceph-mon[102832]: pgmap v9345: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 9 op/s 2023-09-10T04:06:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17220.316802 secs 2023-09-10T04:06:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17220.316802 secs 2023-09-10T04:06:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17220.316802 secs 2023-09-10T04:06:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17220.972868 secs 2023-09-10T04:06:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:30 smithi170 ceph-mon[102832]: pgmap v9346: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 9 op/s 2023-09-10T04:06:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17220.972868 secs 2023-09-10T04:06:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:30 smithi082 ceph-mon[98694]: pgmap v9346: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 9 op/s 2023-09-10T04:06:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17220.972868 secs 2023-09-10T04:06:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:30 smithi098 ceph-mon[103124]: pgmap v9346: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 9 op/s 2023-09-10T04:06:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:06:31.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:06:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:06:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:06:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:06:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:06:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:32 smithi170 ceph-mon[102832]: pgmap v9347: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 207 KiB/s wr, 15 op/s 2023-09-10T04:06:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:06:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:06:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:06:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:32 smithi082 ceph-mon[98694]: pgmap v9347: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 207 KiB/s wr, 15 op/s 2023-09-10T04:06:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:06:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:06:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:06:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:32 smithi098 ceph-mon[103124]: pgmap v9347: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 207 KiB/s wr, 15 op/s 2023-09-10T04:06:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17225.316890 secs 2023-09-10T04:06:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:34 smithi170 ceph-mon[102832]: pgmap v9348: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T04:06:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17225.316890 secs 2023-09-10T04:06:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:34 smithi082 ceph-mon[98694]: pgmap v9348: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T04:06:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17225.316890 secs 2023-09-10T04:06:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:34 smithi098 ceph-mon[103124]: pgmap v9348: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T04:06:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17225.972933 secs 2023-09-10T04:06:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17225.972933 secs 2023-09-10T04:06:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17225.972933 secs 2023-09-10T04:06:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:36 smithi170 ceph-mon[102832]: pgmap v9349: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 272 KiB/s wr, 16 op/s 2023-09-10T04:06:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:36 smithi082 ceph-mon[98694]: pgmap v9349: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 272 KiB/s wr, 16 op/s 2023-09-10T04:06:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:36 smithi098 ceph-mon[103124]: pgmap v9349: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 272 KiB/s wr, 16 op/s 2023-09-10T04:06:37.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:37 smithi082 ceph-mon[98694]: pgmap v9350: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 271 KiB/s wr, 11 op/s 2023-09-10T04:06:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:37 smithi098 ceph-mon[103124]: pgmap v9350: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 271 KiB/s wr, 11 op/s 2023-09-10T04:06:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:37 smithi170 ceph-mon[102832]: pgmap v9350: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 271 KiB/s wr, 11 op/s 2023-09-10T04:06:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17230.316968 secs 2023-09-10T04:06:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17230.316968 secs 2023-09-10T04:06:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17230.316968 secs 2023-09-10T04:06:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17230.973002 secs 2023-09-10T04:06:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:40 smithi170 ceph-mon[102832]: pgmap v9351: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 271 KiB/s wr, 11 op/s 2023-09-10T04:06:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17230.973002 secs 2023-09-10T04:06:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:40 smithi082 ceph-mon[98694]: pgmap v9351: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 271 KiB/s wr, 11 op/s 2023-09-10T04:06:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17230.973002 secs 2023-09-10T04:06:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:40 smithi098 ceph-mon[103124]: pgmap v9351: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 271 KiB/s wr, 11 op/s 2023-09-10T04:06:41.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:41 smithi082 ceph-mon[98694]: pgmap v9352: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 401 KiB/s wr, 18 op/s 2023-09-10T04:06:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:41 smithi098 ceph-mon[103124]: pgmap v9352: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 401 KiB/s wr, 18 op/s 2023-09-10T04:06:41.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:41 smithi170 ceph-mon[102832]: pgmap v9352: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 401 KiB/s wr, 18 op/s 2023-09-10T04:06:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17235.317027 secs 2023-09-10T04:06:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:44 smithi170 ceph-mon[102832]: pgmap v9353: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 254 KiB/s wr, 12 op/s 2023-09-10T04:06:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17235.317027 secs 2023-09-10T04:06:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:44 smithi082 ceph-mon[98694]: pgmap v9353: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 254 KiB/s wr, 12 op/s 2023-09-10T04:06:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17235.317027 secs 2023-09-10T04:06:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:44 smithi098 ceph-mon[103124]: pgmap v9353: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 254 KiB/s wr, 12 op/s 2023-09-10T04:06:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17235.973081 secs 2023-09-10T04:06:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17235.973081 secs 2023-09-10T04:06:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17235.973081 secs 2023-09-10T04:06:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:46 smithi170 ceph-mon[102832]: pgmap v9354: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 452 KiB/s wr, 18 op/s 2023-09-10T04:06:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:46 smithi082 ceph-mon[98694]: pgmap v9354: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 452 KiB/s wr, 18 op/s 2023-09-10T04:06:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:46 smithi098 ceph-mon[103124]: pgmap v9354: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 452 KiB/s wr, 18 op/s 2023-09-10T04:06:47.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:47 smithi082 ceph-mon[98694]: pgmap v9355: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 328 KiB/s wr, 12 op/s 2023-09-10T04:06:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:47 smithi098 ceph-mon[103124]: pgmap v9355: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 328 KiB/s wr, 12 op/s 2023-09-10T04:06:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:47 smithi170 ceph-mon[102832]: pgmap v9355: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 328 KiB/s wr, 12 op/s 2023-09-10T04:06:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17239.141729 secs 2023-09-10T04:06:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17239.141729 secs 2023-09-10T04:06:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17239.141729 secs 2023-09-10T04:06:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17240.317087 secs 2023-09-10T04:06:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:49 smithi098 ceph-mon[103124]: pgmap v9356: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 328 KiB/s wr, 12 op/s 2023-09-10T04:06:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17240.317087 secs 2023-09-10T04:06:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:49 smithi170 ceph-mon[102832]: pgmap v9356: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 328 KiB/s wr, 12 op/s 2023-09-10T04:06:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17240.317087 secs 2023-09-10T04:06:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:49 smithi082 ceph-mon[98694]: pgmap v9356: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 328 KiB/s wr, 12 op/s 2023-09-10T04:06:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17240.973150 secs 2023-09-10T04:06:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:06:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:06:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:06:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:06:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17240.973150 secs 2023-09-10T04:06:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:06:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:06:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:06:50.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:06:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17240.973150 secs 2023-09-10T04:06:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:06:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:06:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:06:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:06:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:51 smithi098 ceph-mon[103124]: pgmap v9357: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 446 KiB/s wr, 19 op/s 2023-09-10T04:06:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:51 smithi170 ceph-mon[102832]: pgmap v9357: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 446 KiB/s wr, 19 op/s 2023-09-10T04:06:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:51 smithi082 ceph-mon[98694]: pgmap v9357: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 446 KiB/s wr, 19 op/s 2023-09-10T04:06:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:06:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:06:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:06:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:06:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:06:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:06:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:06:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:06:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:06:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17244.141802 secs 2023-09-10T04:06:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:53 smithi098 ceph-mon[103124]: pgmap v9358: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 316 KiB/s wr, 13 op/s 2023-09-10T04:06:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17244.141802 secs 2023-09-10T04:06:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:53 smithi170 ceph-mon[102832]: pgmap v9358: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 316 KiB/s wr, 13 op/s 2023-09-10T04:06:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17244.141802 secs 2023-09-10T04:06:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:53 smithi082 ceph-mon[98694]: pgmap v9358: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 316 KiB/s wr, 13 op/s 2023-09-10T04:06:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17245.317149 secs 2023-09-10T04:06:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17245.317149 secs 2023-09-10T04:06:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17245.317149 secs 2023-09-10T04:06:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17245.973217 secs 2023-09-10T04:06:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:55 smithi098 ceph-mon[103124]: pgmap v9359: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 372 KiB/s wr, 19 op/s 2023-09-10T04:06:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17245.973217 secs 2023-09-10T04:06:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:55 smithi170 ceph-mon[102832]: pgmap v9359: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 372 KiB/s wr, 19 op/s 2023-09-10T04:06:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17245.973217 secs 2023-09-10T04:06:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:55 smithi082 ceph-mon[98694]: pgmap v9359: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 372 KiB/s wr, 19 op/s 2023-09-10T04:06:57.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:57 smithi082 ceph-mon[98694]: pgmap v9360: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 174 KiB/s wr, 12 op/s 2023-09-10T04:06:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:57 smithi098 ceph-mon[103124]: pgmap v9360: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 174 KiB/s wr, 12 op/s 2023-09-10T04:06:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:57 smithi170 ceph-mon[102832]: pgmap v9360: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 174 KiB/s wr, 12 op/s 2023-09-10T04:06:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17249.141878 secs 2023-09-10T04:06:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17249.141878 secs 2023-09-10T04:06:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17249.141878 secs 2023-09-10T04:06:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17250.317208 secs 2023-09-10T04:06:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:06:59 smithi082 ceph-mon[98694]: pgmap v9361: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 174 KiB/s wr, 12 op/s 2023-09-10T04:06:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17250.317208 secs 2023-09-10T04:06:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:06:59 smithi098 ceph-mon[103124]: pgmap v9361: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 174 KiB/s wr, 12 op/s 2023-09-10T04:06:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17250.317208 secs 2023-09-10T04:06:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:06:59 smithi170 ceph-mon[102832]: pgmap v9361: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 174 KiB/s wr, 12 op/s 2023-09-10T04:07:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17250.973280 secs 2023-09-10T04:07:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17250.973280 secs 2023-09-10T04:07:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17250.973280 secs 2023-09-10T04:07:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:01 smithi098 ceph-mon[103124]: pgmap v9362: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 251 KiB/s wr, 18 op/s 2023-09-10T04:07:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:01 smithi170 ceph-mon[102832]: pgmap v9362: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 251 KiB/s wr, 18 op/s 2023-09-10T04:07:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:01 smithi082 ceph-mon[98694]: pgmap v9362: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 251 KiB/s wr, 18 op/s 2023-09-10T04:07:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17254.141976 secs 2023-09-10T04:07:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17254.141976 secs 2023-09-10T04:07:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17254.141976 secs 2023-09-10T04:07:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17255.317265 secs 2023-09-10T04:07:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:04 smithi170 ceph-mon[102832]: pgmap v9363: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T04:07:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17255.317265 secs 2023-09-10T04:07:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:04 smithi082 ceph-mon[98694]: pgmap v9363: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T04:07:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17255.317265 secs 2023-09-10T04:07:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:04 smithi098 ceph-mon[103124]: pgmap v9363: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T04:07:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17255.973346 secs 2023-09-10T04:07:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17255.973346 secs 2023-09-10T04:07:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17255.973346 secs 2023-09-10T04:07:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:06 smithi170 ceph-mon[102832]: pgmap v9364: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 232 KiB/s wr, 31 op/s 2023-09-10T04:07:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:06 smithi082 ceph-mon[98694]: pgmap v9364: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 232 KiB/s wr, 31 op/s 2023-09-10T04:07:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:06 smithi098 ceph-mon[103124]: pgmap v9364: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 232 KiB/s wr, 31 op/s 2023-09-10T04:07:07.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:07 smithi082 ceph-mon[98694]: pgmap v9365: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 176 KiB/s wr, 25 op/s 2023-09-10T04:07:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:07 smithi098 ceph-mon[103124]: pgmap v9365: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 176 KiB/s wr, 25 op/s 2023-09-10T04:07:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:07 smithi170 ceph-mon[102832]: pgmap v9365: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 176 KiB/s wr, 25 op/s 2023-09-10T04:07:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17259.142042 secs 2023-09-10T04:07:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:08 smithi082 ceph-mon[98694]: osdmap e653: 12 total, 12 up, 12 in 2023-09-10T04:07:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17259.142042 secs 2023-09-10T04:07:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:08 smithi098 ceph-mon[103124]: osdmap e653: 12 total, 12 up, 12 in 2023-09-10T04:07:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17259.142042 secs 2023-09-10T04:07:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:08 smithi170 ceph-mon[102832]: osdmap e653: 12 total, 12 up, 12 in 2023-09-10T04:07:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17260.317348 secs 2023-09-10T04:07:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:09 smithi098 ceph-mon[103124]: pgmap v9367: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 211 KiB/s wr, 30 op/s 2023-09-10T04:07:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17260.317348 secs 2023-09-10T04:07:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:09 smithi170 ceph-mon[102832]: pgmap v9367: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 211 KiB/s wr, 30 op/s 2023-09-10T04:07:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17260.317348 secs 2023-09-10T04:07:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:09 smithi082 ceph-mon[98694]: pgmap v9367: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 211 KiB/s wr, 30 op/s 2023-09-10T04:07:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17260.973413 secs 2023-09-10T04:07:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17260.973413 secs 2023-09-10T04:07:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17260.973413 secs 2023-09-10T04:07:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:11 smithi098 ceph-mon[103124]: pgmap v9368: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 119 KiB/s wr, 29 op/s 2023-09-10T04:07:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:11 smithi170 ceph-mon[102832]: pgmap v9368: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 119 KiB/s wr, 29 op/s 2023-09-10T04:07:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:11 smithi082 ceph-mon[98694]: pgmap v9368: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 119 KiB/s wr, 29 op/s 2023-09-10T04:07:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17264.142121 secs 2023-09-10T04:07:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17264.142121 secs 2023-09-10T04:07:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17264.142121 secs 2023-09-10T04:07:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17265.317406 secs 2023-09-10T04:07:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:14 smithi170 ceph-mon[102832]: osdmap e654: 12 total, 12 up, 12 in 2023-09-10T04:07:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:14 smithi170 ceph-mon[102832]: pgmap v9370: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 6 op/s 2023-09-10T04:07:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17265.317406 secs 2023-09-10T04:07:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:14 smithi082 ceph-mon[98694]: osdmap e654: 12 total, 12 up, 12 in 2023-09-10T04:07:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:14 smithi082 ceph-mon[98694]: pgmap v9370: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 6 op/s 2023-09-10T04:07:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17265.317406 secs 2023-09-10T04:07:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:14 smithi098 ceph-mon[103124]: osdmap e654: 12 total, 12 up, 12 in 2023-09-10T04:07:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:14 smithi098 ceph-mon[103124]: pgmap v9370: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 6 op/s 2023-09-10T04:07:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17265.973481 secs 2023-09-10T04:07:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17265.973481 secs 2023-09-10T04:07:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17265.973481 secs 2023-09-10T04:07:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:16 smithi170 ceph-mon[102832]: pgmap v9371: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 15 op/s 2023-09-10T04:07:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:16 smithi082 ceph-mon[98694]: pgmap v9371: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 15 op/s 2023-09-10T04:07:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:16 smithi098 ceph-mon[103124]: pgmap v9371: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 15 op/s 2023-09-10T04:07:17.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:17 smithi082 ceph-mon[98694]: pgmap v9372: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 13 op/s 2023-09-10T04:07:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:17 smithi098 ceph-mon[103124]: pgmap v9372: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 13 op/s 2023-09-10T04:07:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:17 smithi170 ceph-mon[102832]: pgmap v9372: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 13 op/s 2023-09-10T04:07:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17269.142182 secs 2023-09-10T04:07:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17269.142182 secs 2023-09-10T04:07:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17269.142182 secs 2023-09-10T04:07:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17270.317501 secs 2023-09-10T04:07:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:19 smithi098 ceph-mon[103124]: pgmap v9373: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T04:07:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17270.317501 secs 2023-09-10T04:07:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:19 smithi170 ceph-mon[102832]: pgmap v9373: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T04:07:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17270.317501 secs 2023-09-10T04:07:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:19 smithi082 ceph-mon[98694]: pgmap v9373: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T04:07:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17270.973544 secs 2023-09-10T04:07:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17270.973544 secs 2023-09-10T04:07:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17270.973544 secs 2023-09-10T04:07:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:21 smithi098 ceph-mon[103124]: pgmap v9374: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T04:07:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:21 smithi170 ceph-mon[102832]: pgmap v9374: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T04:07:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:21 smithi082 ceph-mon[98694]: pgmap v9374: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T04:07:22.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17274.142242 secs 2023-09-10T04:07:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17274.142242 secs 2023-09-10T04:07:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17274.142242 secs 2023-09-10T04:07:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17275.317585 secs 2023-09-10T04:07:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:24 smithi170 ceph-mon[102832]: pgmap v9375: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T04:07:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17275.317585 secs 2023-09-10T04:07:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:24 smithi082 ceph-mon[98694]: pgmap v9375: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T04:07:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17275.317585 secs 2023-09-10T04:07:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:24 smithi098 ceph-mon[103124]: pgmap v9375: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T04:07:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17275.973608 secs 2023-09-10T04:07:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17275.973608 secs 2023-09-10T04:07:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17275.973608 secs 2023-09-10T04:07:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:26 smithi170 ceph-mon[102832]: pgmap v9376: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T04:07:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:26 smithi082 ceph-mon[98694]: pgmap v9376: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T04:07:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:26 smithi098 ceph-mon[103124]: pgmap v9376: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T04:07:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:27 smithi082 ceph-mon[98694]: pgmap v9377: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T04:07:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:27 smithi098 ceph-mon[103124]: pgmap v9377: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T04:07:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:27 smithi170 ceph-mon[102832]: pgmap v9377: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T04:07:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17280.317643 secs 2023-09-10T04:07:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17280.317643 secs 2023-09-10T04:07:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17280.317643 secs 2023-09-10T04:07:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17280.973673 secs 2023-09-10T04:07:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:30 smithi170 ceph-mon[102832]: pgmap v9378: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T04:07:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17280.973673 secs 2023-09-10T04:07:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:30 smithi082 ceph-mon[98694]: pgmap v9378: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T04:07:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17280.973673 secs 2023-09-10T04:07:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:30 smithi098 ceph-mon[103124]: pgmap v9378: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T04:07:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:07:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:07:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:07:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:07:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:07:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:07:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:32 smithi170 ceph-mon[102832]: pgmap v9379: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 122 KiB/s wr, 15 op/s 2023-09-10T04:07:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:07:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:07:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:07:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:32 smithi082 ceph-mon[98694]: pgmap v9379: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 122 KiB/s wr, 15 op/s 2023-09-10T04:07:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:07:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:07:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:07:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:32 smithi098 ceph-mon[103124]: pgmap v9379: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 122 KiB/s wr, 15 op/s 2023-09-10T04:07:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17285.317704 secs 2023-09-10T04:07:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:34 smithi170 ceph-mon[102832]: pgmap v9380: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T04:07:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17285.317704 secs 2023-09-10T04:07:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:34 smithi082 ceph-mon[98694]: pgmap v9380: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T04:07:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17285.317704 secs 2023-09-10T04:07:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:34 smithi098 ceph-mon[103124]: pgmap v9380: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T04:07:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17285.973739 secs 2023-09-10T04:07:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17285.973739 secs 2023-09-10T04:07:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17285.973739 secs 2023-09-10T04:07:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:36 smithi170 ceph-mon[102832]: pgmap v9381: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 246 KiB/s wr, 15 op/s 2023-09-10T04:07:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:36 smithi082 ceph-mon[98694]: pgmap v9381: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 246 KiB/s wr, 15 op/s 2023-09-10T04:07:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:36 smithi098 ceph-mon[103124]: pgmap v9381: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 246 KiB/s wr, 15 op/s 2023-09-10T04:07:37.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:37 smithi082 ceph-mon[98694]: pgmap v9382: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 187 KiB/s wr, 10 op/s 2023-09-10T04:07:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:37 smithi098 ceph-mon[103124]: pgmap v9382: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 187 KiB/s wr, 10 op/s 2023-09-10T04:07:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:37 smithi170 ceph-mon[102832]: pgmap v9382: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 187 KiB/s wr, 10 op/s 2023-09-10T04:07:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17290.317757 secs 2023-09-10T04:07:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17290.317757 secs 2023-09-10T04:07:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17290.317757 secs 2023-09-10T04:07:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17290.973803 secs 2023-09-10T04:07:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:40 smithi170 ceph-mon[102832]: pgmap v9383: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 187 KiB/s wr, 10 op/s 2023-09-10T04:07:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17290.973803 secs 2023-09-10T04:07:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:40 smithi082 ceph-mon[98694]: pgmap v9383: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 187 KiB/s wr, 10 op/s 2023-09-10T04:07:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17290.973803 secs 2023-09-10T04:07:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:40 smithi098 ceph-mon[103124]: pgmap v9383: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 187 KiB/s wr, 10 op/s 2023-09-10T04:07:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:41 smithi082 ceph-mon[98694]: pgmap v9384: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 271 KiB/s wr, 16 op/s 2023-09-10T04:07:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:41 smithi098 ceph-mon[103124]: pgmap v9384: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 271 KiB/s wr, 16 op/s 2023-09-10T04:07:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:41 smithi170 ceph-mon[102832]: pgmap v9384: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 271 KiB/s wr, 16 op/s 2023-09-10T04:07:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17295.317815 secs 2023-09-10T04:07:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:44 smithi170 ceph-mon[102832]: pgmap v9385: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 208 KiB/s wr, 11 op/s 2023-09-10T04:07:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17295.317815 secs 2023-09-10T04:07:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:44 smithi082 ceph-mon[98694]: pgmap v9385: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 208 KiB/s wr, 11 op/s 2023-09-10T04:07:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17295.317815 secs 2023-09-10T04:07:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:44 smithi098 ceph-mon[103124]: pgmap v9385: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 208 KiB/s wr, 11 op/s 2023-09-10T04:07:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17295.973880 secs 2023-09-10T04:07:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17295.973880 secs 2023-09-10T04:07:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17295.973880 secs 2023-09-10T04:07:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:46 smithi170 ceph-mon[102832]: pgmap v9386: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 386 KiB/s wr, 18 op/s 2023-09-10T04:07:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:46 smithi082 ceph-mon[98694]: pgmap v9386: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 386 KiB/s wr, 18 op/s 2023-09-10T04:07:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:46 smithi098 ceph-mon[103124]: pgmap v9386: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 386 KiB/s wr, 18 op/s 2023-09-10T04:07:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:47 smithi082 ceph-mon[98694]: pgmap v9387: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T04:07:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:47 smithi098 ceph-mon[103124]: pgmap v9387: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T04:07:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:47 smithi170 ceph-mon[102832]: pgmap v9387: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T04:07:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17300.317873 secs 2023-09-10T04:07:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17300.317873 secs 2023-09-10T04:07:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17300.317873 secs 2023-09-10T04:07:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17300.973947 secs 2023-09-10T04:07:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:50 smithi170 ceph-mon[102832]: pgmap v9388: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T04:07:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:07:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17300.973947 secs 2023-09-10T04:07:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:50 smithi082 ceph-mon[98694]: pgmap v9388: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T04:07:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:07:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17300.973947 secs 2023-09-10T04:07:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:50 smithi098 ceph-mon[103124]: pgmap v9388: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T04:07:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:07:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:07:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:07:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:07:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:07:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:07:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:07:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:07:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:07:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:07:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:52 smithi170 ceph-mon[102832]: pgmap v9389: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 286 KiB/s wr, 17 op/s 2023-09-10T04:07:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:52 smithi082 ceph-mon[98694]: pgmap v9389: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 286 KiB/s wr, 17 op/s 2023-09-10T04:07:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:52 smithi098 ceph-mon[103124]: pgmap v9389: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 286 KiB/s wr, 17 op/s 2023-09-10T04:07:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17305.317932 secs 2023-09-10T04:07:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:54 smithi170 ceph-mon[102832]: pgmap v9390: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 201 KiB/s wr, 11 op/s 2023-09-10T04:07:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17305.317932 secs 2023-09-10T04:07:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:54 smithi082 ceph-mon[98694]: pgmap v9390: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 201 KiB/s wr, 11 op/s 2023-09-10T04:07:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17305.317932 secs 2023-09-10T04:07:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:54 smithi098 ceph-mon[103124]: pgmap v9390: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 201 KiB/s wr, 11 op/s 2023-09-10T04:07:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17305.974012 secs 2023-09-10T04:07:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17305.974012 secs 2023-09-10T04:07:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17305.974012 secs 2023-09-10T04:07:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:56 smithi170 ceph-mon[102832]: pgmap v9391: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 439 KiB/s wr, 18 op/s 2023-09-10T04:07:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:56 smithi082 ceph-mon[98694]: pgmap v9391: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 439 KiB/s wr, 18 op/s 2023-09-10T04:07:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:56 smithi098 ceph-mon[103124]: pgmap v9391: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 439 KiB/s wr, 18 op/s 2023-09-10T04:07:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:57 smithi098 ceph-mon[103124]: pgmap v9392: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 262 KiB/s wr, 11 op/s 2023-09-10T04:07:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:57 smithi170 ceph-mon[102832]: pgmap v9392: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 262 KiB/s wr, 11 op/s 2023-09-10T04:07:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:57 smithi082 ceph-mon[98694]: pgmap v9392: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 262 KiB/s wr, 11 op/s 2023-09-10T04:07:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17309.142729 secs 2023-09-10T04:07:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17309.142729 secs 2023-09-10T04:07:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17309.142729 secs 2023-09-10T04:07:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17310.318015 secs 2023-09-10T04:07:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:07:59 smithi098 ceph-mon[103124]: pgmap v9393: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 262 KiB/s wr, 11 op/s 2023-09-10T04:07:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17310.318015 secs 2023-09-10T04:07:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:07:59 smithi170 ceph-mon[102832]: pgmap v9393: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 262 KiB/s wr, 11 op/s 2023-09-10T04:07:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17310.318015 secs 2023-09-10T04:07:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:07:59 smithi082 ceph-mon[98694]: pgmap v9393: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 262 KiB/s wr, 11 op/s 2023-09-10T04:08:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17310.974077 secs 2023-09-10T04:08:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:08:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:08:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:08:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17310.974077 secs 2023-09-10T04:08:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:08:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:08:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:08:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17310.974077 secs 2023-09-10T04:08:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:08:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:08:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:08:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:01 smithi098 ceph-mon[103124]: pgmap v9394: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 416 KiB/s wr, 31 op/s 2023-09-10T04:08:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:01 smithi170 ceph-mon[102832]: pgmap v9394: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 416 KiB/s wr, 31 op/s 2023-09-10T04:08:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:01 smithi082 ceph-mon[98694]: pgmap v9394: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 416 KiB/s wr, 31 op/s 2023-09-10T04:08:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17314.142798 secs 2023-09-10T04:08:03.768 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:03 smithi098 ceph-mon[103124]: osdmap e655: 12 total, 12 up, 12 in 2023-09-10T04:08:03.768 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:03 smithi098 ceph-mon[103124]: pgmap v9396: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 471 KiB/s wr, 32 op/s 2023-09-10T04:08:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17314.142798 secs 2023-09-10T04:08:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:03 smithi170 ceph-mon[102832]: osdmap e655: 12 total, 12 up, 12 in 2023-09-10T04:08:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:03 smithi170 ceph-mon[102832]: pgmap v9396: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 471 KiB/s wr, 32 op/s 2023-09-10T04:08:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17314.142798 secs 2023-09-10T04:08:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:03 smithi082 ceph-mon[98694]: osdmap e655: 12 total, 12 up, 12 in 2023-09-10T04:08:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:03 smithi082 ceph-mon[98694]: pgmap v9396: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 471 KiB/s wr, 32 op/s 2023-09-10T04:08:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17315.318100 secs 2023-09-10T04:08:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17315.318100 secs 2023-09-10T04:08:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17315.318100 secs 2023-09-10T04:08:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17315.974143 secs 2023-09-10T04:08:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:05 smithi098 ceph-mon[103124]: pgmap v9397: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 337 KiB/s wr, 30 op/s 2023-09-10T04:08:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17315.974143 secs 2023-09-10T04:08:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:05 smithi170 ceph-mon[102832]: pgmap v9397: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 337 KiB/s wr, 30 op/s 2023-09-10T04:08:05.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17315.974143 secs 2023-09-10T04:08:05.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:05 smithi082 ceph-mon[98694]: pgmap v9397: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 337 KiB/s wr, 30 op/s 2023-09-10T04:08:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:07 smithi098 ceph-mon[103124]: pgmap v9398: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 337 KiB/s wr, 30 op/s 2023-09-10T04:08:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:07 smithi170 ceph-mon[102832]: pgmap v9398: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 337 KiB/s wr, 30 op/s 2023-09-10T04:08:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:07 smithi082 ceph-mon[98694]: pgmap v9398: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 337 KiB/s wr, 30 op/s 2023-09-10T04:08:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17319.142868 secs 2023-09-10T04:08:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:08 smithi098 ceph-mon[103124]: osdmap e656: 12 total, 12 up, 12 in 2023-09-10T04:08:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17319.142868 secs 2023-09-10T04:08:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:08 smithi170 ceph-mon[102832]: osdmap e656: 12 total, 12 up, 12 in 2023-09-10T04:08:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17319.142868 secs 2023-09-10T04:08:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:08 smithi082 ceph-mon[98694]: osdmap e656: 12 total, 12 up, 12 in 2023-09-10T04:08:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17320.318160 secs 2023-09-10T04:08:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:09 smithi098 ceph-mon[103124]: pgmap v9400: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 189 KiB/s wr, 7 op/s 2023-09-10T04:08:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17320.318160 secs 2023-09-10T04:08:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:09 smithi170 ceph-mon[102832]: pgmap v9400: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 189 KiB/s wr, 7 op/s 2023-09-10T04:08:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17320.318160 secs 2023-09-10T04:08:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:09 smithi082 ceph-mon[98694]: pgmap v9400: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 189 KiB/s wr, 7 op/s 2023-09-10T04:08:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17320.974182 secs 2023-09-10T04:08:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17320.974182 secs 2023-09-10T04:08:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17320.974182 secs 2023-09-10T04:08:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:11 smithi098 ceph-mon[103124]: pgmap v9401: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 242 KiB/s wr, 13 op/s 2023-09-10T04:08:11.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:11 smithi170 ceph-mon[102832]: pgmap v9401: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 242 KiB/s wr, 13 op/s 2023-09-10T04:08:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:11 smithi082 ceph-mon[98694]: pgmap v9401: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 242 KiB/s wr, 13 op/s 2023-09-10T04:08:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17324.142929 secs 2023-09-10T04:08:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17324.142929 secs 2023-09-10T04:08:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17324.142929 secs 2023-09-10T04:08:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17325.318218 secs 2023-09-10T04:08:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:14 smithi170 ceph-mon[102832]: pgmap v9402: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 216 KiB/s wr, 12 op/s 2023-09-10T04:08:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17325.318218 secs 2023-09-10T04:08:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:14 smithi082 ceph-mon[98694]: pgmap v9402: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 216 KiB/s wr, 12 op/s 2023-09-10T04:08:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17325.318218 secs 2023-09-10T04:08:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:14 smithi098 ceph-mon[103124]: pgmap v9402: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 216 KiB/s wr, 12 op/s 2023-09-10T04:08:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17325.974249 secs 2023-09-10T04:08:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17325.974249 secs 2023-09-10T04:08:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17325.974249 secs 2023-09-10T04:08:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:16 smithi170 ceph-mon[102832]: pgmap v9403: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T04:08:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:16 smithi082 ceph-mon[98694]: pgmap v9403: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T04:08:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:16 smithi098 ceph-mon[103124]: pgmap v9403: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T04:08:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:17 smithi098 ceph-mon[103124]: pgmap v9404: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T04:08:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:17 smithi170 ceph-mon[102832]: pgmap v9404: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T04:08:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:17 smithi082 ceph-mon[98694]: pgmap v9404: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 11 op/s 2023-09-10T04:08:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17329.142990 secs 2023-09-10T04:08:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17329.142990 secs 2023-09-10T04:08:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17329.142990 secs 2023-09-10T04:08:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17330.318279 secs 2023-09-10T04:08:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:19 smithi098 ceph-mon[103124]: pgmap v9405: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T04:08:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17330.318279 secs 2023-09-10T04:08:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:19 smithi170 ceph-mon[102832]: pgmap v9405: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T04:08:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17330.318279 secs 2023-09-10T04:08:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:19 smithi082 ceph-mon[98694]: pgmap v9405: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T04:08:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17330.974314 secs 2023-09-10T04:08:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17330.974314 secs 2023-09-10T04:08:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17330.974314 secs 2023-09-10T04:08:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:21 smithi098 ceph-mon[103124]: pgmap v9406: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 15 op/s 2023-09-10T04:08:21.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:21 smithi170 ceph-mon[102832]: pgmap v9406: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 15 op/s 2023-09-10T04:08:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:21 smithi082 ceph-mon[98694]: pgmap v9406: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 15 op/s 2023-09-10T04:08:22.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17334.143112 secs 2023-09-10T04:08:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17334.143112 secs 2023-09-10T04:08:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17334.143112 secs 2023-09-10T04:08:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17335.318337 secs 2023-09-10T04:08:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:24 smithi170 ceph-mon[102832]: pgmap v9407: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:08:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17335.318337 secs 2023-09-10T04:08:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:24 smithi082 ceph-mon[98694]: pgmap v9407: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:08:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17335.318337 secs 2023-09-10T04:08:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:24 smithi098 ceph-mon[103124]: pgmap v9407: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:08:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17335.974377 secs 2023-09-10T04:08:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17335.974377 secs 2023-09-10T04:08:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17335.974377 secs 2023-09-10T04:08:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:26 smithi170 ceph-mon[102832]: pgmap v9408: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 61 KiB/s wr, 15 op/s 2023-09-10T04:08:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:26 smithi082 ceph-mon[98694]: pgmap v9408: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 61 KiB/s wr, 15 op/s 2023-09-10T04:08:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:26 smithi098 ceph-mon[103124]: pgmap v9408: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 61 KiB/s wr, 15 op/s 2023-09-10T04:08:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:27 smithi098 ceph-mon[103124]: pgmap v9409: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:08:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:27 smithi170 ceph-mon[102832]: pgmap v9409: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:08:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:27 smithi082 ceph-mon[98694]: pgmap v9409: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:08:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17340.318397 secs 2023-09-10T04:08:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17340.318397 secs 2023-09-10T04:08:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17340.318397 secs 2023-09-10T04:08:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17340.974442 secs 2023-09-10T04:08:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:30 smithi170 ceph-mon[102832]: pgmap v9410: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:08:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17340.974442 secs 2023-09-10T04:08:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:30 smithi082 ceph-mon[98694]: pgmap v9410: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:08:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17340.974442 secs 2023-09-10T04:08:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:30 smithi098 ceph-mon[103124]: pgmap v9410: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:08:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:08:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:08:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:08:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:08:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:08:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:08:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:32 smithi170 ceph-mon[102832]: pgmap v9411: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 209 KiB/s wr, 16 op/s 2023-09-10T04:08:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:08:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:08:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:08:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:32 smithi082 ceph-mon[98694]: pgmap v9411: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 209 KiB/s wr, 16 op/s 2023-09-10T04:08:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:08:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:08:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:08:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:32 smithi098 ceph-mon[103124]: pgmap v9411: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 209 KiB/s wr, 16 op/s 2023-09-10T04:08:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17345.318454 secs 2023-09-10T04:08:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:34 smithi170 ceph-mon[102832]: pgmap v9412: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 10 op/s 2023-09-10T04:08:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17345.318454 secs 2023-09-10T04:08:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:34 smithi082 ceph-mon[98694]: pgmap v9412: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 10 op/s 2023-09-10T04:08:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17345.318454 secs 2023-09-10T04:08:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:34 smithi098 ceph-mon[103124]: pgmap v9412: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 10 op/s 2023-09-10T04:08:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17345.974484 secs 2023-09-10T04:08:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17345.974484 secs 2023-09-10T04:08:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17345.974484 secs 2023-09-10T04:08:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:36 smithi170 ceph-mon[102832]: pgmap v9413: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 274 KiB/s wr, 16 op/s 2023-09-10T04:08:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:36 smithi082 ceph-mon[98694]: pgmap v9413: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 274 KiB/s wr, 16 op/s 2023-09-10T04:08:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:36 smithi098 ceph-mon[103124]: pgmap v9413: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 274 KiB/s wr, 16 op/s 2023-09-10T04:08:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:37 smithi098 ceph-mon[103124]: pgmap v9414: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 274 KiB/s wr, 12 op/s 2023-09-10T04:08:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:37 smithi170 ceph-mon[102832]: pgmap v9414: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 274 KiB/s wr, 12 op/s 2023-09-10T04:08:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:37 smithi082 ceph-mon[98694]: pgmap v9414: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 274 KiB/s wr, 12 op/s 2023-09-10T04:08:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17350.318496 secs 2023-09-10T04:08:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17350.318496 secs 2023-09-10T04:08:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17350.318496 secs 2023-09-10T04:08:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17350.974549 secs 2023-09-10T04:08:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:40 smithi170 ceph-mon[102832]: pgmap v9415: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 274 KiB/s wr, 12 op/s 2023-09-10T04:08:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17350.974549 secs 2023-09-10T04:08:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:40 smithi082 ceph-mon[98694]: pgmap v9415: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 274 KiB/s wr, 12 op/s 2023-09-10T04:08:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17350.974549 secs 2023-09-10T04:08:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:40 smithi098 ceph-mon[103124]: pgmap v9415: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 274 KiB/s wr, 12 op/s 2023-09-10T04:08:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:41 smithi098 ceph-mon[103124]: pgmap v9416: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 405 KiB/s wr, 18 op/s 2023-09-10T04:08:41.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:41 smithi170 ceph-mon[102832]: pgmap v9416: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 405 KiB/s wr, 18 op/s 2023-09-10T04:08:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:41 smithi082 ceph-mon[98694]: pgmap v9416: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 405 KiB/s wr, 18 op/s 2023-09-10T04:08:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17355.318554 secs 2023-09-10T04:08:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:44 smithi170 ceph-mon[102832]: pgmap v9417: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 256 KiB/s wr, 12 op/s 2023-09-10T04:08:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17355.318554 secs 2023-09-10T04:08:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:44 smithi082 ceph-mon[98694]: pgmap v9417: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 256 KiB/s wr, 12 op/s 2023-09-10T04:08:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17355.318554 secs 2023-09-10T04:08:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:44 smithi098 ceph-mon[103124]: pgmap v9417: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 256 KiB/s wr, 12 op/s 2023-09-10T04:08:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17355.974614 secs 2023-09-10T04:08:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17355.974614 secs 2023-09-10T04:08:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17355.974614 secs 2023-09-10T04:08:47.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:47 smithi170 ceph-mon[102832]: pgmap v9418: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 452 KiB/s wr, 18 op/s 2023-09-10T04:08:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:47 smithi082 ceph-mon[98694]: pgmap v9418: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 452 KiB/s wr, 18 op/s 2023-09-10T04:08:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:47 smithi098 ceph-mon[103124]: pgmap v9418: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 452 KiB/s wr, 18 op/s 2023-09-10T04:08:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17359.143625 secs 2023-09-10T04:08:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17359.143625 secs 2023-09-10T04:08:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17359.143625 secs 2023-09-10T04:08:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:49 smithi170 ceph-mon[102832]: pgmap v9419: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 328 KiB/s wr, 12 op/s 2023-09-10T04:08:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17360.318619 secs 2023-09-10T04:08:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:49 smithi082 ceph-mon[98694]: pgmap v9419: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 328 KiB/s wr, 12 op/s 2023-09-10T04:08:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17360.318619 secs 2023-09-10T04:08:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:49 smithi098 ceph-mon[103124]: pgmap v9419: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 328 KiB/s wr, 12 op/s 2023-09-10T04:08:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17360.318619 secs 2023-09-10T04:08:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17360.974684 secs 2023-09-10T04:08:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17360.974684 secs 2023-09-10T04:08:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17360.974684 secs 2023-09-10T04:08:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:51 smithi170 ceph-mon[102832]: pgmap v9420: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 328 KiB/s wr, 12 op/s 2023-09-10T04:08:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:08:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:51 smithi082 ceph-mon[98694]: pgmap v9420: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 328 KiB/s wr, 12 op/s 2023-09-10T04:08:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:08:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:51 smithi098 ceph-mon[103124]: pgmap v9420: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 328 KiB/s wr, 12 op/s 2023-09-10T04:08:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:08:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:08:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:08:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:08:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:08:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:08:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:08:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:08:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:08:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:08:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:08:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:08:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:08:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:08:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:08:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:08:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:08:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:08:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:08:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:53 smithi170 ceph-mon[102832]: pgmap v9421: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 450 KiB/s wr, 18 op/s 2023-09-10T04:08:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17364.143712 secs 2023-09-10T04:08:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:53 smithi082 ceph-mon[98694]: pgmap v9421: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 450 KiB/s wr, 18 op/s 2023-09-10T04:08:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17364.143712 secs 2023-09-10T04:08:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:53 smithi098 ceph-mon[103124]: pgmap v9421: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 450 KiB/s wr, 18 op/s 2023-09-10T04:08:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17364.143712 secs 2023-09-10T04:08:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17365.318678 secs 2023-09-10T04:08:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17365.318678 secs 2023-09-10T04:08:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17365.318678 secs 2023-09-10T04:08:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:55 smithi082 ceph-mon[98694]: pgmap v9422: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 319 KiB/s wr, 12 op/s 2023-09-10T04:08:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17365.974749 secs 2023-09-10T04:08:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:55 smithi098 ceph-mon[103124]: pgmap v9422: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 319 KiB/s wr, 12 op/s 2023-09-10T04:08:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17365.974749 secs 2023-09-10T04:08:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:55 smithi170 ceph-mon[102832]: pgmap v9422: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 319 KiB/s wr, 12 op/s 2023-09-10T04:08:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17365.974749 secs 2023-09-10T04:08:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:56 smithi082 ceph-mon[98694]: pgmap v9423: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 375 KiB/s wr, 18 op/s 2023-09-10T04:08:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:56 smithi098 ceph-mon[103124]: pgmap v9423: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 375 KiB/s wr, 18 op/s 2023-09-10T04:08:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:56 smithi170 ceph-mon[102832]: pgmap v9423: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 375 KiB/s wr, 18 op/s 2023-09-10T04:08:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:58 smithi082 ceph-mon[98694]: pgmap v9424: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 179 KiB/s wr, 11 op/s 2023-09-10T04:08:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17369.143775 secs 2023-09-10T04:08:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:58 smithi098 ceph-mon[103124]: pgmap v9424: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 179 KiB/s wr, 11 op/s 2023-09-10T04:08:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17369.143775 secs 2023-09-10T04:08:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:58 smithi170 ceph-mon[102832]: pgmap v9424: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 179 KiB/s wr, 11 op/s 2023-09-10T04:08:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17369.143775 secs 2023-09-10T04:08:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:08:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17370.318763 secs 2023-09-10T04:08:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:08:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17370.318763 secs 2023-09-10T04:08:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:08:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17370.318763 secs 2023-09-10T04:09:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17370.974818 secs 2023-09-10T04:09:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:00 smithi082 ceph-mon[98694]: pgmap v9425: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 179 KiB/s wr, 11 op/s 2023-09-10T04:09:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17370.974818 secs 2023-09-10T04:09:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:00 smithi098 ceph-mon[103124]: pgmap v9425: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 179 KiB/s wr, 11 op/s 2023-09-10T04:09:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17370.974818 secs 2023-09-10T04:09:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:00 smithi170 ceph-mon[102832]: pgmap v9425: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 179 KiB/s wr, 11 op/s 2023-09-10T04:09:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:02 smithi082 ceph-mon[98694]: pgmap v9426: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 271 KiB/s wr, 20 op/s 2023-09-10T04:09:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:02 smithi098 ceph-mon[103124]: pgmap v9426: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 271 KiB/s wr, 20 op/s 2023-09-10T04:09:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:02 smithi170 ceph-mon[102832]: pgmap v9426: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 271 KiB/s wr, 20 op/s 2023-09-10T04:09:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17374.143832 secs 2023-09-10T04:09:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17374.143832 secs 2023-09-10T04:09:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17374.143832 secs 2023-09-10T04:09:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17375.318820 secs 2023-09-10T04:09:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:04 smithi082 ceph-mon[98694]: pgmap v9427: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 149 KiB/s wr, 15 op/s 2023-09-10T04:09:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17375.318820 secs 2023-09-10T04:09:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:04 smithi098 ceph-mon[103124]: pgmap v9427: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 149 KiB/s wr, 15 op/s 2023-09-10T04:09:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17375.318820 secs 2023-09-10T04:09:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:04 smithi170 ceph-mon[102832]: pgmap v9427: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 149 KiB/s wr, 15 op/s 2023-09-10T04:09:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17375.974882 secs 2023-09-10T04:09:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17375.974882 secs 2023-09-10T04:09:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17375.974882 secs 2023-09-10T04:09:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:06 smithi082 ceph-mon[98694]: pgmap v9428: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 218 KiB/s wr, 31 op/s 2023-09-10T04:09:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:06 smithi098 ceph-mon[103124]: pgmap v9428: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 218 KiB/s wr, 31 op/s 2023-09-10T04:09:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:06 smithi170 ceph-mon[102832]: pgmap v9428: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 218 KiB/s wr, 31 op/s 2023-09-10T04:09:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:08 smithi082 ceph-mon[98694]: pgmap v9429: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 162 KiB/s wr, 25 op/s 2023-09-10T04:09:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17379.143893 secs 2023-09-10T04:09:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:08 smithi082 ceph-mon[98694]: osdmap e657: 12 total, 12 up, 12 in 2023-09-10T04:09:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:08 smithi098 ceph-mon[103124]: pgmap v9429: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 162 KiB/s wr, 25 op/s 2023-09-10T04:09:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17379.143893 secs 2023-09-10T04:09:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:08 smithi098 ceph-mon[103124]: osdmap e657: 12 total, 12 up, 12 in 2023-09-10T04:09:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:08 smithi170 ceph-mon[102832]: pgmap v9429: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 162 KiB/s wr, 25 op/s 2023-09-10T04:09:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17379.143893 secs 2023-09-10T04:09:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:08 smithi170 ceph-mon[102832]: osdmap e657: 12 total, 12 up, 12 in 2023-09-10T04:09:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17380.318906 secs 2023-09-10T04:09:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17380.318906 secs 2023-09-10T04:09:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17380.318906 secs 2023-09-10T04:09:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17380.974945 secs 2023-09-10T04:09:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:10 smithi098 ceph-mon[103124]: pgmap v9431: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 194 KiB/s wr, 30 op/s 2023-09-10T04:09:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17380.974945 secs 2023-09-10T04:09:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:10 smithi170 ceph-mon[102832]: pgmap v9431: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 194 KiB/s wr, 30 op/s 2023-09-10T04:09:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17380.974945 secs 2023-09-10T04:09:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:10 smithi082 ceph-mon[98694]: pgmap v9431: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 194 KiB/s wr, 30 op/s 2023-09-10T04:09:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:12 smithi098 ceph-mon[103124]: pgmap v9432: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 83 KiB/s wr, 26 op/s 2023-09-10T04:09:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:12 smithi170 ceph-mon[102832]: pgmap v9432: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 83 KiB/s wr, 26 op/s 2023-09-10T04:09:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:12 smithi082 ceph-mon[98694]: pgmap v9432: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 83 KiB/s wr, 26 op/s 2023-09-10T04:09:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17384.143952 secs 2023-09-10T04:09:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:13 smithi098 ceph-mon[103124]: osdmap e658: 12 total, 12 up, 12 in 2023-09-10T04:09:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17384.143952 secs 2023-09-10T04:09:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:13 smithi170 ceph-mon[102832]: osdmap e658: 12 total, 12 up, 12 in 2023-09-10T04:09:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17384.143952 secs 2023-09-10T04:09:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:13 smithi082 ceph-mon[98694]: osdmap e658: 12 total, 12 up, 12 in 2023-09-10T04:09:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17385.318994 secs 2023-09-10T04:09:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:14 smithi098 ceph-mon[103124]: pgmap v9434: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T04:09:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17385.318994 secs 2023-09-10T04:09:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:14 smithi170 ceph-mon[102832]: pgmap v9434: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T04:09:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17385.318994 secs 2023-09-10T04:09:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:14 smithi082 ceph-mon[98694]: pgmap v9434: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T04:09:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17385.975009 secs 2023-09-10T04:09:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17385.975009 secs 2023-09-10T04:09:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17385.975009 secs 2023-09-10T04:09:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:16 smithi098 ceph-mon[103124]: pgmap v9435: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 90 KiB/s wr, 15 op/s 2023-09-10T04:09:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:16 smithi170 ceph-mon[102832]: pgmap v9435: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 90 KiB/s wr, 15 op/s 2023-09-10T04:09:16.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:16 smithi082 ceph-mon[98694]: pgmap v9435: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 90 KiB/s wr, 15 op/s 2023-09-10T04:09:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:18 smithi098 ceph-mon[103124]: pgmap v9436: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-10T04:09:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17389.144013 secs 2023-09-10T04:09:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:18 smithi170 ceph-mon[102832]: pgmap v9436: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-10T04:09:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17389.144013 secs 2023-09-10T04:09:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:18 smithi082 ceph-mon[98694]: pgmap v9436: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-10T04:09:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17389.144013 secs 2023-09-10T04:09:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17390.319052 secs 2023-09-10T04:09:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17390.319052 secs 2023-09-10T04:09:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17390.319052 secs 2023-09-10T04:09:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17390.975074 secs 2023-09-10T04:09:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:20 smithi098 ceph-mon[103124]: pgmap v9437: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-10T04:09:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17390.975074 secs 2023-09-10T04:09:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:20 smithi170 ceph-mon[102832]: pgmap v9437: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-10T04:09:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17390.975074 secs 2023-09-10T04:09:20.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:20 smithi082 ceph-mon[98694]: pgmap v9437: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 12 op/s 2023-09-10T04:09:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:22 smithi098 ceph-mon[103124]: pgmap v9438: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T04:09:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:22 smithi170 ceph-mon[102832]: pgmap v9438: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T04:09:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:22 smithi082 ceph-mon[98694]: pgmap v9438: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T04:09:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17394.144075 secs 2023-09-10T04:09:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17394.144075 secs 2023-09-10T04:09:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17394.144075 secs 2023-09-10T04:09:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17395.319110 secs 2023-09-10T04:09:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:24 smithi098 ceph-mon[103124]: pgmap v9439: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T04:09:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17395.319110 secs 2023-09-10T04:09:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:24 smithi170 ceph-mon[102832]: pgmap v9439: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T04:09:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17395.319110 secs 2023-09-10T04:09:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:24 smithi082 ceph-mon[98694]: pgmap v9439: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 11 op/s 2023-09-10T04:09:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17395.975141 secs 2023-09-10T04:09:25.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17395.975141 secs 2023-09-10T04:09:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17395.975141 secs 2023-09-10T04:09:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:26 smithi170 ceph-mon[102832]: pgmap v9440: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 60 KiB/s wr, 14 op/s 2023-09-10T04:09:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:26 smithi082 ceph-mon[98694]: pgmap v9440: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 60 KiB/s wr, 14 op/s 2023-09-10T04:09:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:26 smithi098 ceph-mon[103124]: pgmap v9440: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 60 KiB/s wr, 14 op/s 2023-09-10T04:09:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:29 smithi170 ceph-mon[102832]: pgmap v9441: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2023-09-10T04:09:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17400.319167 secs 2023-09-10T04:09:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:29 smithi082 ceph-mon[98694]: pgmap v9441: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2023-09-10T04:09:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17400.319167 secs 2023-09-10T04:09:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:29 smithi098 ceph-mon[103124]: pgmap v9441: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2023-09-10T04:09:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17400.319167 secs 2023-09-10T04:09:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17400.975206 secs 2023-09-10T04:09:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17400.975206 secs 2023-09-10T04:09:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17400.975206 secs 2023-09-10T04:09:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:31 smithi170 ceph-mon[102832]: pgmap v9442: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2023-09-10T04:09:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:09:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:31 smithi082 ceph-mon[98694]: pgmap v9442: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2023-09-10T04:09:31.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:09:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:31 smithi098 ceph-mon[103124]: pgmap v9442: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2023-09-10T04:09:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:09:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:09:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:09:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:09:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:09:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:09:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:09:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:09:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:09:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:09:33.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:33 smithi170 ceph-mon[102832]: pgmap v9443: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 171 KiB/s wr, 15 op/s 2023-09-10T04:09:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:33 smithi082 ceph-mon[98694]: pgmap v9443: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 171 KiB/s wr, 15 op/s 2023-09-10T04:09:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:33 smithi098 ceph-mon[103124]: pgmap v9443: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 171 KiB/s wr, 15 op/s 2023-09-10T04:09:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17405.319227 secs 2023-09-10T04:09:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17405.319227 secs 2023-09-10T04:09:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17405.319227 secs 2023-09-10T04:09:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:35 smithi170 ceph-mon[102832]: pgmap v9444: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 171 KiB/s wr, 10 op/s 2023-09-10T04:09:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17405.975269 secs 2023-09-10T04:09:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:35 smithi082 ceph-mon[98694]: pgmap v9444: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 171 KiB/s wr, 10 op/s 2023-09-10T04:09:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17405.975269 secs 2023-09-10T04:09:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:35 smithi098 ceph-mon[103124]: pgmap v9444: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 171 KiB/s wr, 10 op/s 2023-09-10T04:09:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17405.975269 secs 2023-09-10T04:09:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:37 smithi170 ceph-mon[102832]: pgmap v9445: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 278 KiB/s wr, 17 op/s 2023-09-10T04:09:37.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:37 smithi082 ceph-mon[98694]: pgmap v9445: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 278 KiB/s wr, 17 op/s 2023-09-10T04:09:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:37 smithi098 ceph-mon[103124]: pgmap v9445: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 278 KiB/s wr, 17 op/s 2023-09-10T04:09:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:39 smithi170 ceph-mon[102832]: pgmap v9446: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T04:09:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17410.319310 secs 2023-09-10T04:09:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:39 smithi082 ceph-mon[98694]: pgmap v9446: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T04:09:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17410.319310 secs 2023-09-10T04:09:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:39 smithi098 ceph-mon[103124]: pgmap v9446: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T04:09:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17410.319310 secs 2023-09-10T04:09:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17410.975332 secs 2023-09-10T04:09:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17410.975332 secs 2023-09-10T04:09:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17410.975332 secs 2023-09-10T04:09:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:41 smithi170 ceph-mon[102832]: pgmap v9447: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T04:09:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:41 smithi082 ceph-mon[98694]: pgmap v9447: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T04:09:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:41 smithi098 ceph-mon[103124]: pgmap v9447: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T04:09:42.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:42 smithi082 ceph-mon[98694]: pgmap v9448: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 409 KiB/s wr, 18 op/s 2023-09-10T04:09:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:42 smithi098 ceph-mon[103124]: pgmap v9448: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 409 KiB/s wr, 18 op/s 2023-09-10T04:09:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:42 smithi170 ceph-mon[102832]: pgmap v9448: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 409 KiB/s wr, 18 op/s 2023-09-10T04:09:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17415.319370 secs 2023-09-10T04:09:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17415.319370 secs 2023-09-10T04:09:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17415.319370 secs 2023-09-10T04:09:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:45 smithi170 ceph-mon[102832]: pgmap v9449: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 238 KiB/s wr, 12 op/s 2023-09-10T04:09:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17415.975398 secs 2023-09-10T04:09:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:45 smithi082 ceph-mon[98694]: pgmap v9449: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 238 KiB/s wr, 12 op/s 2023-09-10T04:09:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17415.975398 secs 2023-09-10T04:09:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:45 smithi098 ceph-mon[103124]: pgmap v9449: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 238 KiB/s wr, 12 op/s 2023-09-10T04:09:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17415.975398 secs 2023-09-10T04:09:47.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:47 smithi170 ceph-mon[102832]: pgmap v9450: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 371 KiB/s wr, 18 op/s 2023-09-10T04:09:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:47 smithi082 ceph-mon[98694]: pgmap v9450: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 371 KiB/s wr, 18 op/s 2023-09-10T04:09:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:47 smithi098 ceph-mon[103124]: pgmap v9450: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 371 KiB/s wr, 18 op/s 2023-09-10T04:09:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:49 smithi170 ceph-mon[102832]: pgmap v9451: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T04:09:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17420.319453 secs 2023-09-10T04:09:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:49 smithi082 ceph-mon[98694]: pgmap v9451: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T04:09:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17420.319453 secs 2023-09-10T04:09:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:49 smithi098 ceph-mon[103124]: pgmap v9451: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T04:09:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17420.319453 secs 2023-09-10T04:09:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17420.975461 secs 2023-09-10T04:09:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17420.975461 secs 2023-09-10T04:09:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17420.975461 secs 2023-09-10T04:09:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:51 smithi170 ceph-mon[102832]: pgmap v9452: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T04:09:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:51 smithi082 ceph-mon[98694]: pgmap v9452: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T04:09:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:51 smithi098 ceph-mon[103124]: pgmap v9452: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T04:09:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:09:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:09:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:09:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:09:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:09:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:09:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:09:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:09:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:09:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:09:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:09:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:09:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:53 smithi170 ceph-mon[102832]: pgmap v9453: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 396 KiB/s wr, 18 op/s 2023-09-10T04:09:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17424.144442 secs 2023-09-10T04:09:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:53 smithi082 ceph-mon[98694]: pgmap v9453: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 396 KiB/s wr, 18 op/s 2023-09-10T04:09:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17424.144442 secs 2023-09-10T04:09:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:53 smithi098 ceph-mon[103124]: pgmap v9453: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 396 KiB/s wr, 18 op/s 2023-09-10T04:09:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17424.144442 secs 2023-09-10T04:09:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17425.319492 secs 2023-09-10T04:09:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17425.319492 secs 2023-09-10T04:09:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17425.319492 secs 2023-09-10T04:09:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:55 smithi082 ceph-mon[98694]: pgmap v9454: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T04:09:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17425.975485 secs 2023-09-10T04:09:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:55 smithi098 ceph-mon[103124]: pgmap v9454: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T04:09:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17425.975485 secs 2023-09-10T04:09:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:55 smithi170 ceph-mon[102832]: pgmap v9454: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T04:09:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17425.975485 secs 2023-09-10T04:09:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:09:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:56 smithi082 ceph-mon[98694]: pgmap v9455: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 385 KiB/s wr, 18 op/s 2023-09-10T04:09:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:09:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:09:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:09:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:56 smithi098 ceph-mon[103124]: pgmap v9455: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 385 KiB/s wr, 18 op/s 2023-09-10T04:09:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:09:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:09:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:09:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:56 smithi170 ceph-mon[102832]: pgmap v9455: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 385 KiB/s wr, 18 op/s 2023-09-10T04:09:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:09:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:09:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:58 smithi082 ceph-mon[98694]: pgmap v9456: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 253 KiB/s wr, 12 op/s 2023-09-10T04:09:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17429.144503 secs 2023-09-10T04:09:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:58 smithi098 ceph-mon[103124]: pgmap v9456: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 253 KiB/s wr, 12 op/s 2023-09-10T04:09:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17429.144503 secs 2023-09-10T04:09:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:58 smithi170 ceph-mon[102832]: pgmap v9456: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 253 KiB/s wr, 12 op/s 2023-09-10T04:09:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17429.144503 secs 2023-09-10T04:09:59.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:09:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17430.319549 secs 2023-09-10T04:09:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:09:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17430.319549 secs 2023-09-10T04:09:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:09:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17430.319549 secs 2023-09-10T04:10:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17430.975549 secs 2023-09-10T04:10:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:00 smithi082 ceph-mon[98694]: pgmap v9457: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 253 KiB/s wr, 12 op/s 2023-09-10T04:10:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T04:10:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17430.975549 secs 2023-09-10T04:10:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:00 smithi098 ceph-mon[103124]: pgmap v9457: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 253 KiB/s wr, 12 op/s 2023-09-10T04:10:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T04:10:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17430.975549 secs 2023-09-10T04:10:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:00 smithi170 ceph-mon[102832]: pgmap v9457: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 253 KiB/s wr, 12 op/s 2023-09-10T04:10:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T04:10:02.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:02 smithi082 ceph-mon[98694]: pgmap v9458: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 345 KiB/s wr, 17 op/s 2023-09-10T04:10:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:02 smithi098 ceph-mon[103124]: pgmap v9458: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 345 KiB/s wr, 17 op/s 2023-09-10T04:10:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:02 smithi170 ceph-mon[102832]: pgmap v9458: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 345 KiB/s wr, 17 op/s 2023-09-10T04:10:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17434.144564 secs 2023-09-10T04:10:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17434.144564 secs 2023-09-10T04:10:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17434.144564 secs 2023-09-10T04:10:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17435.319609 secs 2023-09-10T04:10:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:04 smithi082 ceph-mon[98694]: pgmap v9459: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 213 KiB/s wr, 11 op/s 2023-09-10T04:10:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17435.319609 secs 2023-09-10T04:10:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:04 smithi098 ceph-mon[103124]: pgmap v9459: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 213 KiB/s wr, 11 op/s 2023-09-10T04:10:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17435.319609 secs 2023-09-10T04:10:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:04 smithi170 ceph-mon[102832]: pgmap v9459: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 213 KiB/s wr, 11 op/s 2023-09-10T04:10:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17435.975615 secs 2023-09-10T04:10:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17435.975615 secs 2023-09-10T04:10:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17435.975615 secs 2023-09-10T04:10:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:06 smithi082 ceph-mon[98694]: pgmap v9460: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 299 KiB/s wr, 31 op/s 2023-09-10T04:10:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:06 smithi098 ceph-mon[103124]: pgmap v9460: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 299 KiB/s wr, 31 op/s 2023-09-10T04:10:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:06 smithi170 ceph-mon[102832]: pgmap v9460: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 299 KiB/s wr, 31 op/s 2023-09-10T04:10:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:08 smithi098 ceph-mon[103124]: pgmap v9461: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 179 KiB/s wr, 25 op/s 2023-09-10T04:10:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17439.144649 secs 2023-09-10T04:10:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:08 smithi098 ceph-mon[103124]: osdmap e659: 12 total, 12 up, 12 in 2023-09-10T04:10:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:08 smithi170 ceph-mon[102832]: pgmap v9461: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 179 KiB/s wr, 25 op/s 2023-09-10T04:10:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17439.144649 secs 2023-09-10T04:10:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:08 smithi170 ceph-mon[102832]: osdmap e659: 12 total, 12 up, 12 in 2023-09-10T04:10:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:08 smithi082 ceph-mon[98694]: pgmap v9461: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 179 KiB/s wr, 25 op/s 2023-09-10T04:10:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17439.144649 secs 2023-09-10T04:10:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:08 smithi082 ceph-mon[98694]: osdmap e659: 12 total, 12 up, 12 in 2023-09-10T04:10:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17440.319669 secs 2023-09-10T04:10:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17440.319669 secs 2023-09-10T04:10:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17440.319669 secs 2023-09-10T04:10:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17440.975678 secs 2023-09-10T04:10:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:10 smithi098 ceph-mon[103124]: pgmap v9463: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 214 KiB/s wr, 30 op/s 2023-09-10T04:10:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17440.975678 secs 2023-09-10T04:10:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:10 smithi170 ceph-mon[102832]: pgmap v9463: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 214 KiB/s wr, 30 op/s 2023-09-10T04:10:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17440.975678 secs 2023-09-10T04:10:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:10 smithi082 ceph-mon[98694]: pgmap v9463: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 214 KiB/s wr, 30 op/s 2023-09-10T04:10:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:12 smithi098 ceph-mon[103124]: pgmap v9464: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 104 KiB/s wr, 30 op/s 2023-09-10T04:10:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:12 smithi170 ceph-mon[102832]: pgmap v9464: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 104 KiB/s wr, 30 op/s 2023-09-10T04:10:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:12 smithi082 ceph-mon[98694]: pgmap v9464: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 104 KiB/s wr, 30 op/s 2023-09-10T04:10:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17444.144711 secs 2023-09-10T04:10:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:13 smithi098 ceph-mon[103124]: osdmap e660: 12 total, 12 up, 12 in 2023-09-10T04:10:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17444.144711 secs 2023-09-10T04:10:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:13 smithi170 ceph-mon[102832]: osdmap e660: 12 total, 12 up, 12 in 2023-09-10T04:10:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17444.144711 secs 2023-09-10T04:10:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:13 smithi082 ceph-mon[98694]: osdmap e660: 12 total, 12 up, 12 in 2023-09-10T04:10:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17445.319728 secs 2023-09-10T04:10:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:14 smithi098 ceph-mon[103124]: pgmap v9466: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T04:10:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17445.319728 secs 2023-09-10T04:10:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:14 smithi170 ceph-mon[102832]: pgmap v9466: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T04:10:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17445.319728 secs 2023-09-10T04:10:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:14 smithi082 ceph-mon[98694]: pgmap v9466: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T04:10:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17445.975742 secs 2023-09-10T04:10:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17445.975742 secs 2023-09-10T04:10:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17445.975742 secs 2023-09-10T04:10:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:16 smithi098 ceph-mon[103124]: pgmap v9467: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 16 op/s 2023-09-10T04:10:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:16 smithi170 ceph-mon[102832]: pgmap v9467: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 16 op/s 2023-09-10T04:10:16.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:16 smithi082 ceph-mon[98694]: pgmap v9467: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 16 op/s 2023-09-10T04:10:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:18 smithi098 ceph-mon[103124]: pgmap v9468: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 13 op/s 2023-09-10T04:10:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17449.144772 secs 2023-09-10T04:10:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:18 smithi170 ceph-mon[102832]: pgmap v9468: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 13 op/s 2023-09-10T04:10:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17449.144772 secs 2023-09-10T04:10:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:18 smithi082 ceph-mon[98694]: pgmap v9468: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 13 op/s 2023-09-10T04:10:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17449.144772 secs 2023-09-10T04:10:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17450.319809 secs 2023-09-10T04:10:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17450.319809 secs 2023-09-10T04:10:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17450.319809 secs 2023-09-10T04:10:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17450.975805 secs 2023-09-10T04:10:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:20 smithi098 ceph-mon[103124]: pgmap v9469: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 112 KiB/s wr, 13 op/s 2023-09-10T04:10:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17450.975805 secs 2023-09-10T04:10:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:20 smithi170 ceph-mon[102832]: pgmap v9469: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 112 KiB/s wr, 13 op/s 2023-09-10T04:10:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17450.975805 secs 2023-09-10T04:10:20.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:20 smithi082 ceph-mon[98694]: pgmap v9469: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 112 KiB/s wr, 13 op/s 2023-09-10T04:10:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:22 smithi098 ceph-mon[103124]: pgmap v9470: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T04:10:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:22 smithi170 ceph-mon[102832]: pgmap v9470: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T04:10:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:22 smithi082 ceph-mon[98694]: pgmap v9470: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T04:10:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17454.144831 secs 2023-09-10T04:10:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17454.144831 secs 2023-09-10T04:10:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17454.144831 secs 2023-09-10T04:10:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17455.319867 secs 2023-09-10T04:10:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:24 smithi098 ceph-mon[103124]: pgmap v9471: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T04:10:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17455.319867 secs 2023-09-10T04:10:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:24 smithi170 ceph-mon[102832]: pgmap v9471: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T04:10:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17455.319867 secs 2023-09-10T04:10:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:24 smithi082 ceph-mon[98694]: pgmap v9471: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T04:10:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17455.975868 secs 2023-09-10T04:10:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17455.975868 secs 2023-09-10T04:10:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17455.975868 secs 2023-09-10T04:10:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:26 smithi170 ceph-mon[102832]: pgmap v9472: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T04:10:26.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:26 smithi082 ceph-mon[98694]: pgmap v9472: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T04:10:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:26 smithi098 ceph-mon[103124]: pgmap v9472: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T04:10:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:29 smithi170 ceph-mon[102832]: pgmap v9473: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:10:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17460.319924 secs 2023-09-10T04:10:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:29 smithi082 ceph-mon[98694]: pgmap v9473: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:10:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17460.319924 secs 2023-09-10T04:10:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:29 smithi098 ceph-mon[103124]: pgmap v9473: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:10:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17460.319924 secs 2023-09-10T04:10:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17460.975931 secs 2023-09-10T04:10:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17460.975931 secs 2023-09-10T04:10:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17460.975931 secs 2023-09-10T04:10:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:31 smithi170 ceph-mon[102832]: pgmap v9474: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:10:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:10:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:31 smithi082 ceph-mon[98694]: pgmap v9474: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:10:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:10:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:31 smithi098 ceph-mon[103124]: pgmap v9474: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:10:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:10:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:10:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:10:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:10:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:10:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:10:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:10:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:10:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:10:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:10:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:33 smithi082 ceph-mon[98694]: pgmap v9475: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 261 KiB/s wr, 15 op/s 2023-09-10T04:10:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:33 smithi098 ceph-mon[103124]: pgmap v9475: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 261 KiB/s wr, 15 op/s 2023-09-10T04:10:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:33 smithi170 ceph-mon[102832]: pgmap v9475: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 261 KiB/s wr, 15 op/s 2023-09-10T04:10:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17465.320009 secs 2023-09-10T04:10:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17465.320009 secs 2023-09-10T04:10:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17465.320009 secs 2023-09-10T04:10:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:35 smithi082 ceph-mon[98694]: pgmap v9476: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 261 KiB/s wr, 10 op/s 2023-09-10T04:10:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17465.975999 secs 2023-09-10T04:10:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:35 smithi098 ceph-mon[103124]: pgmap v9476: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 261 KiB/s wr, 10 op/s 2023-09-10T04:10:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17465.975999 secs 2023-09-10T04:10:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:35 smithi170 ceph-mon[102832]: pgmap v9476: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 261 KiB/s wr, 10 op/s 2023-09-10T04:10:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17465.975999 secs 2023-09-10T04:10:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:36 smithi082 ceph-mon[98694]: pgmap v9477: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 336 KiB/s wr, 16 op/s 2023-09-10T04:10:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:36 smithi098 ceph-mon[103124]: pgmap v9477: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 336 KiB/s wr, 16 op/s 2023-09-10T04:10:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:36 smithi170 ceph-mon[102832]: pgmap v9477: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 336 KiB/s wr, 16 op/s 2023-09-10T04:10:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:39 smithi170 ceph-mon[102832]: pgmap v9478: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 279 KiB/s wr, 11 op/s 2023-09-10T04:10:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17470.320069 secs 2023-09-10T04:10:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:39 smithi082 ceph-mon[98694]: pgmap v9478: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 279 KiB/s wr, 11 op/s 2023-09-10T04:10:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17470.320069 secs 2023-09-10T04:10:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:39 smithi098 ceph-mon[103124]: pgmap v9478: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 279 KiB/s wr, 11 op/s 2023-09-10T04:10:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17470.320069 secs 2023-09-10T04:10:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17470.976063 secs 2023-09-10T04:10:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17470.976063 secs 2023-09-10T04:10:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17470.976063 secs 2023-09-10T04:10:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:41 smithi170 ceph-mon[102832]: pgmap v9479: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 279 KiB/s wr, 11 op/s 2023-09-10T04:10:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:41 smithi082 ceph-mon[98694]: pgmap v9479: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 279 KiB/s wr, 11 op/s 2023-09-10T04:10:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:41 smithi098 ceph-mon[103124]: pgmap v9479: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 279 KiB/s wr, 11 op/s 2023-09-10T04:10:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:43 smithi170 ceph-mon[102832]: pgmap v9480: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 429 KiB/s wr, 18 op/s 2023-09-10T04:10:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:43 smithi082 ceph-mon[98694]: pgmap v9480: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 429 KiB/s wr, 18 op/s 2023-09-10T04:10:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:43 smithi098 ceph-mon[103124]: pgmap v9480: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 429 KiB/s wr, 18 op/s 2023-09-10T04:10:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17475.320145 secs 2023-09-10T04:10:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17475.320145 secs 2023-09-10T04:10:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17475.320145 secs 2023-09-10T04:10:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:45 smithi082 ceph-mon[98694]: pgmap v9481: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 225 KiB/s wr, 12 op/s 2023-09-10T04:10:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17475.976127 secs 2023-09-10T04:10:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:45 smithi098 ceph-mon[103124]: pgmap v9481: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 225 KiB/s wr, 12 op/s 2023-09-10T04:10:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17475.976127 secs 2023-09-10T04:10:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:45 smithi170 ceph-mon[102832]: pgmap v9481: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 225 KiB/s wr, 12 op/s 2023-09-10T04:10:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17475.976127 secs 2023-09-10T04:10:47.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:47 smithi082 ceph-mon[98694]: pgmap v9482: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 408 KiB/s wr, 19 op/s 2023-09-10T04:10:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:47 smithi098 ceph-mon[103124]: pgmap v9482: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 408 KiB/s wr, 19 op/s 2023-09-10T04:10:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:47 smithi170 ceph-mon[102832]: pgmap v9482: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 408 KiB/s wr, 19 op/s 2023-09-10T04:10:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:49 smithi082 ceph-mon[98694]: pgmap v9483: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T04:10:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17480.320228 secs 2023-09-10T04:10:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:49 smithi098 ceph-mon[103124]: pgmap v9483: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T04:10:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17480.320228 secs 2023-09-10T04:10:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:49 smithi170 ceph-mon[102832]: pgmap v9483: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T04:10:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17480.320228 secs 2023-09-10T04:10:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17480.976273 secs 2023-09-10T04:10:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:50 smithi082 ceph-mon[98694]: pgmap v9484: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T04:10:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17480.976273 secs 2023-09-10T04:10:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:50 smithi098 ceph-mon[103124]: pgmap v9484: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T04:10:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17480.976273 secs 2023-09-10T04:10:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:50 smithi170 ceph-mon[102832]: pgmap v9484: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T04:10:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:52 smithi170 ceph-mon[102832]: pgmap v9485: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 423 KiB/s wr, 19 op/s 2023-09-10T04:10:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:10:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:52 smithi082 ceph-mon[98694]: pgmap v9485: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 423 KiB/s wr, 19 op/s 2023-09-10T04:10:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:10:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:52 smithi098 ceph-mon[103124]: pgmap v9485: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 423 KiB/s wr, 19 op/s 2023-09-10T04:10:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:10:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17484.145285 secs 2023-09-10T04:10:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17484.145285 secs 2023-09-10T04:10:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17484.145285 secs 2023-09-10T04:10:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17485.320288 secs 2023-09-10T04:10:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:54 smithi098 ceph-mon[103124]: pgmap v9486: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 273 KiB/s wr, 13 op/s 2023-09-10T04:10:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17485.320288 secs 2023-09-10T04:10:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:54 smithi170 ceph-mon[102832]: pgmap v9486: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 273 KiB/s wr, 13 op/s 2023-09-10T04:10:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17485.320288 secs 2023-09-10T04:10:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:54 smithi082 ceph-mon[98694]: pgmap v9486: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 273 KiB/s wr, 13 op/s 2023-09-10T04:10:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17485.976336 secs 2023-09-10T04:10:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:10:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:10:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:10:55.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:10:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17485.976336 secs 2023-09-10T04:10:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:10:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:10:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:10:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:10:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17485.976336 secs 2023-09-10T04:10:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:10:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:10:56.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:10:56.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:10:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:56 smithi170 ceph-mon[102832]: pgmap v9487: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 342 KiB/s wr, 18 op/s 2023-09-10T04:10:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:10:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:10:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:10:56.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:56.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:56.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:56.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:56.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:56 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 58316k 2023-09-10T04:10:56.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:56 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 59716366: error parsing value: Value '59716366' is below minimum 939524096 2023-09-10T04:10:56.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:56.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:56.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:56.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:56.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:56 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T04:10:56.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:56 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T04:10:56.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:10:56.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:10:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:56 smithi082 ceph-mon[98694]: pgmap v9487: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 342 KiB/s wr, 18 op/s 2023-09-10T04:10:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:10:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:10:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:10:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:56.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:56.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:56.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:56.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:56 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 58316k 2023-09-10T04:10:56.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:56 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 59716366: error parsing value: Value '59716366' is below minimum 939524096 2023-09-10T04:10:56.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:56.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:56.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:56.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:56.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:56 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T04:10:56.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:56 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T04:10:56.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:10:56.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:10:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:56 smithi098 ceph-mon[103124]: pgmap v9487: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 342 KiB/s wr, 18 op/s 2023-09-10T04:10:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:10:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:10:57.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:10:57.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:57.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:57.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:57.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:57.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:56 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 58316k 2023-09-10T04:10:57.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:56 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 59716366: error parsing value: Value '59716366' is below minimum 939524096 2023-09-10T04:10:57.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:57.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:57.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:57.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:57.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:56 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T04:10:57.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:56 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T04:10:57.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:10:57.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:10:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:57.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:57.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:57 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T04:10:57.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:57 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T04:10:57.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:10:57.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:10:57.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:10:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:57 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T04:10:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:57 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T04:10:57.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:10:57.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:10:57.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:10:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:10:58.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:57 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T04:10:58.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:57 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T04:10:58.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:10:58.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:10:58.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:10:58.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17489.145425 secs 2023-09-10T04:10:58.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:58 smithi170 ceph-mon[102832]: pgmap v9488: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T04:10:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17489.145425 secs 2023-09-10T04:10:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:58 smithi082 ceph-mon[98694]: pgmap v9488: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T04:10:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17489.145425 secs 2023-09-10T04:10:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:58 smithi098 ceph-mon[103124]: pgmap v9488: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T04:10:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:10:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17490.320357 secs 2023-09-10T04:10:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:10:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17490.320357 secs 2023-09-10T04:11:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:10:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17490.320357 secs 2023-09-10T04:11:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17490.976398 secs 2023-09-10T04:11:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:00 smithi170 ceph-mon[102832]: pgmap v9489: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T04:11:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17490.976398 secs 2023-09-10T04:11:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:00 smithi082 ceph-mon[98694]: pgmap v9489: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T04:11:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17490.976398 secs 2023-09-10T04:11:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:00 smithi098 ceph-mon[103124]: pgmap v9489: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T04:11:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:02 smithi170 ceph-mon[102832]: pgmap v9490: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 252 KiB/s wr, 17 op/s 2023-09-10T04:11:02.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:02 smithi082 ceph-mon[98694]: pgmap v9490: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 252 KiB/s wr, 17 op/s 2023-09-10T04:11:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:02 smithi098 ceph-mon[103124]: pgmap v9490: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 252 KiB/s wr, 17 op/s 2023-09-10T04:11:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17494.145487 secs 2023-09-10T04:11:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17494.145487 secs 2023-09-10T04:11:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17494.145487 secs 2023-09-10T04:11:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17495.320416 secs 2023-09-10T04:11:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:04 smithi170 ceph-mon[102832]: pgmap v9491: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T04:11:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17495.320416 secs 2023-09-10T04:11:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:04 smithi082 ceph-mon[98694]: pgmap v9491: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T04:11:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17495.320416 secs 2023-09-10T04:11:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:04 smithi098 ceph-mon[103124]: pgmap v9491: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T04:11:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17495.976464 secs 2023-09-10T04:11:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17495.976464 secs 2023-09-10T04:11:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17495.976464 secs 2023-09-10T04:11:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:06 smithi170 ceph-mon[102832]: pgmap v9492: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 248 KiB/s wr, 30 op/s 2023-09-10T04:11:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:06 smithi082 ceph-mon[98694]: pgmap v9492: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 248 KiB/s wr, 30 op/s 2023-09-10T04:11:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:06 smithi098 ceph-mon[103124]: pgmap v9492: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 248 KiB/s wr, 30 op/s 2023-09-10T04:11:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17499.145546 secs 2023-09-10T04:11:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:08 smithi170 ceph-mon[102832]: pgmap v9493: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 179 KiB/s wr, 25 op/s 2023-09-10T04:11:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:08 smithi170 ceph-mon[102832]: osdmap e661: 12 total, 12 up, 12 in 2023-09-10T04:11:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17499.145546 secs 2023-09-10T04:11:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:08 smithi082 ceph-mon[98694]: pgmap v9493: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 179 KiB/s wr, 25 op/s 2023-09-10T04:11:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:08 smithi082 ceph-mon[98694]: osdmap e661: 12 total, 12 up, 12 in 2023-09-10T04:11:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17499.145546 secs 2023-09-10T04:11:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:08 smithi098 ceph-mon[103124]: pgmap v9493: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 179 KiB/s wr, 25 op/s 2023-09-10T04:11:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:08 smithi098 ceph-mon[103124]: osdmap e661: 12 total, 12 up, 12 in 2023-09-10T04:11:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17500.320476 secs 2023-09-10T04:11:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17500.320476 secs 2023-09-10T04:11:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17500.320476 secs 2023-09-10T04:11:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17500.976530 secs 2023-09-10T04:11:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:10 smithi170 ceph-mon[102832]: pgmap v9495: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 215 KiB/s wr, 30 op/s 2023-09-10T04:11:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17500.976530 secs 2023-09-10T04:11:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:10 smithi082 ceph-mon[98694]: pgmap v9495: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 215 KiB/s wr, 30 op/s 2023-09-10T04:11:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17500.976530 secs 2023-09-10T04:11:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:10 smithi098 ceph-mon[103124]: pgmap v9495: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 215 KiB/s wr, 30 op/s 2023-09-10T04:11:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:12 smithi170 ceph-mon[102832]: pgmap v9496: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 104 KiB/s wr, 29 op/s 2023-09-10T04:11:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:12 smithi082 ceph-mon[98694]: pgmap v9496: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 104 KiB/s wr, 29 op/s 2023-09-10T04:11:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:12 smithi098 ceph-mon[103124]: pgmap v9496: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 104 KiB/s wr, 29 op/s 2023-09-10T04:11:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17504.145605 secs 2023-09-10T04:11:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:13 smithi170 ceph-mon[102832]: osdmap e662: 12 total, 12 up, 12 in 2023-09-10T04:11:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17504.145605 secs 2023-09-10T04:11:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:13 smithi082 ceph-mon[98694]: osdmap e662: 12 total, 12 up, 12 in 2023-09-10T04:11:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17504.145605 secs 2023-09-10T04:11:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:13 smithi098 ceph-mon[103124]: osdmap e662: 12 total, 12 up, 12 in 2023-09-10T04:11:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17505.320511 secs 2023-09-10T04:11:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:14 smithi170 ceph-mon[102832]: pgmap v9498: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:11:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17505.320511 secs 2023-09-10T04:11:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:14 smithi082 ceph-mon[98694]: pgmap v9498: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:11:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17505.320511 secs 2023-09-10T04:11:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:14 smithi098 ceph-mon[103124]: pgmap v9498: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:11:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17505.976597 secs 2023-09-10T04:11:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17505.976597 secs 2023-09-10T04:11:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17505.976597 secs 2023-09-10T04:11:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:16 smithi170 ceph-mon[102832]: pgmap v9499: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 15 op/s 2023-09-10T04:11:16.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:16 smithi082 ceph-mon[98694]: pgmap v9499: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 15 op/s 2023-09-10T04:11:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:16 smithi098 ceph-mon[103124]: pgmap v9499: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 15 op/s 2023-09-10T04:11:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17509.145664 secs 2023-09-10T04:11:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:18 smithi170 ceph-mon[102832]: pgmap v9500: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T04:11:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17509.145664 secs 2023-09-10T04:11:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:18 smithi082 ceph-mon[98694]: pgmap v9500: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T04:11:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17509.145664 secs 2023-09-10T04:11:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:18 smithi098 ceph-mon[103124]: pgmap v9500: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T04:11:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17510.320572 secs 2023-09-10T04:11:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17510.320572 secs 2023-09-10T04:11:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17510.320572 secs 2023-09-10T04:11:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17510.976662 secs 2023-09-10T04:11:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:20 smithi170 ceph-mon[102832]: pgmap v9501: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T04:11:20.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17510.976662 secs 2023-09-10T04:11:20.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:20 smithi082 ceph-mon[98694]: pgmap v9501: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T04:11:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17510.976662 secs 2023-09-10T04:11:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:20 smithi098 ceph-mon[103124]: pgmap v9501: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T04:11:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:22 smithi170 ceph-mon[102832]: pgmap v9502: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T04:11:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:22 smithi082 ceph-mon[98694]: pgmap v9502: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T04:11:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:22 smithi098 ceph-mon[103124]: pgmap v9502: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T04:11:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17514.145725 secs 2023-09-10T04:11:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17514.145725 secs 2023-09-10T04:11:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17514.145725 secs 2023-09-10T04:11:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17515.320630 secs 2023-09-10T04:11:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:24 smithi170 ceph-mon[102832]: pgmap v9503: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T04:11:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17515.320630 secs 2023-09-10T04:11:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:24 smithi082 ceph-mon[98694]: pgmap v9503: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T04:11:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17515.320630 secs 2023-09-10T04:11:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:24 smithi098 ceph-mon[103124]: pgmap v9503: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T04:11:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17515.976726 secs 2023-09-10T04:11:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17515.976726 secs 2023-09-10T04:11:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17515.976726 secs 2023-09-10T04:11:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:26 smithi170 ceph-mon[102832]: pgmap v9504: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 216 KiB/s wr, 15 op/s 2023-09-10T04:11:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:26 smithi082 ceph-mon[98694]: pgmap v9504: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 216 KiB/s wr, 15 op/s 2023-09-10T04:11:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:26 smithi098 ceph-mon[103124]: pgmap v9504: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 216 KiB/s wr, 15 op/s 2023-09-10T04:11:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17519.145772 secs 2023-09-10T04:11:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:28 smithi170 ceph-mon[102832]: pgmap v9505: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T04:11:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17519.145772 secs 2023-09-10T04:11:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:28 smithi082 ceph-mon[98694]: pgmap v9505: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T04:11:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17519.145772 secs 2023-09-10T04:11:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:28 smithi098 ceph-mon[103124]: pgmap v9505: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T04:11:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17520.320688 secs 2023-09-10T04:11:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17520.320688 secs 2023-09-10T04:11:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17520.320688 secs 2023-09-10T04:11:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17520.976790 secs 2023-09-10T04:11:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:30 smithi170 ceph-mon[102832]: pgmap v9506: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T04:11:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17520.976790 secs 2023-09-10T04:11:30.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:30 smithi082 ceph-mon[98694]: pgmap v9506: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T04:11:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17520.976790 secs 2023-09-10T04:11:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:30 smithi098 ceph-mon[103124]: pgmap v9506: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T04:11:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:32 smithi170 ceph-mon[102832]: pgmap v9507: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 212 KiB/s wr, 15 op/s 2023-09-10T04:11:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:32 smithi082 ceph-mon[98694]: pgmap v9507: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 212 KiB/s wr, 15 op/s 2023-09-10T04:11:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:32 smithi098 ceph-mon[103124]: pgmap v9507: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 212 KiB/s wr, 15 op/s 2023-09-10T04:11:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17525.320800 secs 2023-09-10T04:11:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17525.320800 secs 2023-09-10T04:11:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17525.320800 secs 2023-09-10T04:11:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:35 smithi082 ceph-mon[98694]: pgmap v9508: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 211 KiB/s wr, 10 op/s 2023-09-10T04:11:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17525.976854 secs 2023-09-10T04:11:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:11:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:35 smithi098 ceph-mon[103124]: pgmap v9508: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 211 KiB/s wr, 10 op/s 2023-09-10T04:11:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17525.976854 secs 2023-09-10T04:11:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:11:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:35 smithi170 ceph-mon[102832]: pgmap v9508: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 211 KiB/s wr, 10 op/s 2023-09-10T04:11:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17525.976854 secs 2023-09-10T04:11:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:11:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:11:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:11:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:11:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:11:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:11:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:11:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:11:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:11:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:11:37.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:37 smithi082 ceph-mon[98694]: pgmap v9509: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 342 KiB/s wr, 17 op/s 2023-09-10T04:11:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:37 smithi098 ceph-mon[103124]: pgmap v9509: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 342 KiB/s wr, 17 op/s 2023-09-10T04:11:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:37 smithi170 ceph-mon[102832]: pgmap v9509: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 342 KiB/s wr, 17 op/s 2023-09-10T04:11:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:39 smithi082 ceph-mon[98694]: pgmap v9510: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 221 KiB/s wr, 11 op/s 2023-09-10T04:11:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17530.320857 secs 2023-09-10T04:11:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:39 smithi098 ceph-mon[103124]: pgmap v9510: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 221 KiB/s wr, 11 op/s 2023-09-10T04:11:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17530.320857 secs 2023-09-10T04:11:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:39 smithi170 ceph-mon[102832]: pgmap v9510: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 221 KiB/s wr, 11 op/s 2023-09-10T04:11:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17530.320857 secs 2023-09-10T04:11:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17530.976920 secs 2023-09-10T04:11:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17530.976920 secs 2023-09-10T04:11:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17530.976920 secs 2023-09-10T04:11:41.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:41 smithi082 ceph-mon[98694]: pgmap v9511: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 221 KiB/s wr, 11 op/s 2023-09-10T04:11:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:41 smithi098 ceph-mon[103124]: pgmap v9511: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 221 KiB/s wr, 11 op/s 2023-09-10T04:11:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:41 smithi170 ceph-mon[102832]: pgmap v9511: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 221 KiB/s wr, 11 op/s 2023-09-10T04:11:43.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:43 smithi082 ceph-mon[98694]: pgmap v9512: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 366 KiB/s wr, 18 op/s 2023-09-10T04:11:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:43 smithi098 ceph-mon[103124]: pgmap v9512: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 366 KiB/s wr, 18 op/s 2023-09-10T04:11:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:43 smithi170 ceph-mon[102832]: pgmap v9512: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 366 KiB/s wr, 18 op/s 2023-09-10T04:11:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17535.320918 secs 2023-09-10T04:11:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17535.320918 secs 2023-09-10T04:11:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17535.320918 secs 2023-09-10T04:11:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:45 smithi082 ceph-mon[98694]: pgmap v9513: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 276 KiB/s wr, 12 op/s 2023-09-10T04:11:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17535.976982 secs 2023-09-10T04:11:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:45 smithi098 ceph-mon[103124]: pgmap v9513: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 276 KiB/s wr, 12 op/s 2023-09-10T04:11:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17535.976982 secs 2023-09-10T04:11:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:45 smithi170 ceph-mon[102832]: pgmap v9513: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 276 KiB/s wr, 12 op/s 2023-09-10T04:11:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17535.976982 secs 2023-09-10T04:11:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:46 smithi082 ceph-mon[98694]: pgmap v9514: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 402 KiB/s wr, 18 op/s 2023-09-10T04:11:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:46 smithi098 ceph-mon[103124]: pgmap v9514: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 402 KiB/s wr, 18 op/s 2023-09-10T04:11:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:46 smithi170 ceph-mon[102832]: pgmap v9514: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 402 KiB/s wr, 18 op/s 2023-09-10T04:11:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:49 smithi082 ceph-mon[98694]: pgmap v9515: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 271 KiB/s wr, 12 op/s 2023-09-10T04:11:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17540.320978 secs 2023-09-10T04:11:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:49 smithi098 ceph-mon[103124]: pgmap v9515: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 271 KiB/s wr, 12 op/s 2023-09-10T04:11:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17540.320978 secs 2023-09-10T04:11:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:49 smithi170 ceph-mon[102832]: pgmap v9515: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 271 KiB/s wr, 12 op/s 2023-09-10T04:11:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17540.320978 secs 2023-09-10T04:11:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17540.977048 secs 2023-09-10T04:11:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17540.977048 secs 2023-09-10T04:11:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17540.977048 secs 2023-09-10T04:11:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:51 smithi082 ceph-mon[98694]: pgmap v9516: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 271 KiB/s wr, 12 op/s 2023-09-10T04:11:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:51 smithi098 ceph-mon[103124]: pgmap v9516: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 271 KiB/s wr, 12 op/s 2023-09-10T04:11:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:51 smithi170 ceph-mon[102832]: pgmap v9516: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 271 KiB/s wr, 12 op/s 2023-09-10T04:11:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:53 smithi082 ceph-mon[98694]: pgmap v9517: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 411 KiB/s wr, 18 op/s 2023-09-10T04:11:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17544.146143 secs 2023-09-10T04:11:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:53 smithi098 ceph-mon[103124]: pgmap v9517: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 411 KiB/s wr, 18 op/s 2023-09-10T04:11:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17544.146143 secs 2023-09-10T04:11:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:53 smithi170 ceph-mon[102832]: pgmap v9517: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 411 KiB/s wr, 18 op/s 2023-09-10T04:11:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17544.146143 secs 2023-09-10T04:11:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17545.321053 secs 2023-09-10T04:11:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17545.321053 secs 2023-09-10T04:11:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17545.321053 secs 2023-09-10T04:11:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:55 smithi082 ceph-mon[98694]: pgmap v9518: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 266 KiB/s wr, 12 op/s 2023-09-10T04:11:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17545.977116 secs 2023-09-10T04:11:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:55 smithi098 ceph-mon[103124]: pgmap v9518: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 266 KiB/s wr, 12 op/s 2023-09-10T04:11:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17545.977116 secs 2023-09-10T04:11:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:55 smithi170 ceph-mon[102832]: pgmap v9518: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 266 KiB/s wr, 12 op/s 2023-09-10T04:11:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17545.977116 secs 2023-09-10T04:11:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:11:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:11:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:11:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:56 smithi082 ceph-mon[98694]: pgmap v9519: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 336 KiB/s wr, 17 op/s 2023-09-10T04:11:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:11:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:11:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:11:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:56 smithi098 ceph-mon[103124]: pgmap v9519: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 336 KiB/s wr, 17 op/s 2023-09-10T04:11:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:11:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:11:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:11:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:56 smithi170 ceph-mon[102832]: pgmap v9519: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 336 KiB/s wr, 17 op/s 2023-09-10T04:11:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:11:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:11:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:11:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17549.146202 secs 2023-09-10T04:11:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:58 smithi098 ceph-mon[103124]: pgmap v9520: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T04:11:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:11:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:11:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:11:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17549.146202 secs 2023-09-10T04:11:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:58 smithi170 ceph-mon[102832]: pgmap v9520: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T04:11:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:11:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:11:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:11:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17549.146202 secs 2023-09-10T04:11:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:58 smithi082 ceph-mon[98694]: pgmap v9520: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T04:11:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:11:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:11:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:11:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:11:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17550.321111 secs 2023-09-10T04:11:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:11:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17550.321111 secs 2023-09-10T04:11:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:11:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17550.321111 secs 2023-09-10T04:12:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17550.977183 secs 2023-09-10T04:12:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:00 smithi098 ceph-mon[103124]: pgmap v9521: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T04:12:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17550.977183 secs 2023-09-10T04:12:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:00 smithi170 ceph-mon[102832]: pgmap v9521: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T04:12:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17550.977183 secs 2023-09-10T04:12:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:00 smithi082 ceph-mon[98694]: pgmap v9521: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T04:12:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:02 smithi098 ceph-mon[103124]: pgmap v9522: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 303 KiB/s wr, 17 op/s 2023-09-10T04:12:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:02 smithi170 ceph-mon[102832]: pgmap v9522: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 303 KiB/s wr, 17 op/s 2023-09-10T04:12:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:02 smithi082 ceph-mon[98694]: pgmap v9522: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 303 KiB/s wr, 17 op/s 2023-09-10T04:12:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17554.146260 secs 2023-09-10T04:12:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17554.146260 secs 2023-09-10T04:12:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17554.146260 secs 2023-09-10T04:12:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17555.321189 secs 2023-09-10T04:12:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:04 smithi098 ceph-mon[103124]: pgmap v9523: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T04:12:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17555.321189 secs 2023-09-10T04:12:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:04 smithi170 ceph-mon[102832]: pgmap v9523: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T04:12:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17555.321189 secs 2023-09-10T04:12:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:04 smithi082 ceph-mon[98694]: pgmap v9523: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T04:12:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17555.977248 secs 2023-09-10T04:12:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17555.977248 secs 2023-09-10T04:12:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17555.977248 secs 2023-09-10T04:12:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:06 smithi098 ceph-mon[103124]: pgmap v9524: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 249 KiB/s wr, 31 op/s 2023-09-10T04:12:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:06 smithi170 ceph-mon[102832]: pgmap v9524: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 249 KiB/s wr, 31 op/s 2023-09-10T04:12:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:06 smithi082 ceph-mon[98694]: pgmap v9524: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 249 KiB/s wr, 31 op/s 2023-09-10T04:12:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17559.146321 secs 2023-09-10T04:12:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:08 smithi098 ceph-mon[103124]: pgmap v9525: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 180 KiB/s wr, 25 op/s 2023-09-10T04:12:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:08 smithi098 ceph-mon[103124]: osdmap e663: 12 total, 12 up, 12 in 2023-09-10T04:12:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17559.146321 secs 2023-09-10T04:12:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:08 smithi170 ceph-mon[102832]: pgmap v9525: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 180 KiB/s wr, 25 op/s 2023-09-10T04:12:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:08 smithi170 ceph-mon[102832]: osdmap e663: 12 total, 12 up, 12 in 2023-09-10T04:12:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17559.146321 secs 2023-09-10T04:12:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:08 smithi082 ceph-mon[98694]: pgmap v9525: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 180 KiB/s wr, 25 op/s 2023-09-10T04:12:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:08 smithi082 ceph-mon[98694]: osdmap e663: 12 total, 12 up, 12 in 2023-09-10T04:12:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17560.321247 secs 2023-09-10T04:12:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17560.321247 secs 2023-09-10T04:12:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17560.321247 secs 2023-09-10T04:12:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17560.977312 secs 2023-09-10T04:12:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:10 smithi170 ceph-mon[102832]: pgmap v9527: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 216 KiB/s wr, 30 op/s 2023-09-10T04:12:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17560.977312 secs 2023-09-10T04:12:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:10 smithi082 ceph-mon[98694]: pgmap v9527: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 216 KiB/s wr, 30 op/s 2023-09-10T04:12:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17560.977312 secs 2023-09-10T04:12:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:10 smithi098 ceph-mon[103124]: pgmap v9527: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 216 KiB/s wr, 30 op/s 2023-09-10T04:12:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:12 smithi170 ceph-mon[102832]: pgmap v9528: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T04:12:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:12 smithi082 ceph-mon[98694]: pgmap v9528: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T04:12:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:12 smithi098 ceph-mon[103124]: pgmap v9528: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T04:12:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17564.146440 secs 2023-09-10T04:12:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:13 smithi170 ceph-mon[102832]: osdmap e664: 12 total, 12 up, 12 in 2023-09-10T04:12:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17564.146440 secs 2023-09-10T04:12:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:13 smithi082 ceph-mon[98694]: osdmap e664: 12 total, 12 up, 12 in 2023-09-10T04:12:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17564.146440 secs 2023-09-10T04:12:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:13 smithi098 ceph-mon[103124]: osdmap e664: 12 total, 12 up, 12 in 2023-09-10T04:12:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17565.321385 secs 2023-09-10T04:12:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:14 smithi170 ceph-mon[102832]: pgmap v9530: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T04:12:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17565.321385 secs 2023-09-10T04:12:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:14 smithi082 ceph-mon[98694]: pgmap v9530: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T04:12:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17565.321385 secs 2023-09-10T04:12:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:14 smithi098 ceph-mon[103124]: pgmap v9530: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T04:12:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17565.977378 secs 2023-09-10T04:12:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17565.977378 secs 2023-09-10T04:12:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17565.977378 secs 2023-09-10T04:12:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:16 smithi170 ceph-mon[102832]: pgmap v9531: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 142 KiB/s wr, 14 op/s 2023-09-10T04:12:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:16 smithi082 ceph-mon[98694]: pgmap v9531: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 142 KiB/s wr, 14 op/s 2023-09-10T04:12:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:16 smithi098 ceph-mon[103124]: pgmap v9531: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 142 KiB/s wr, 14 op/s 2023-09-10T04:12:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17569.146499 secs 2023-09-10T04:12:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:18 smithi170 ceph-mon[102832]: pgmap v9532: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T04:12:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17569.146499 secs 2023-09-10T04:12:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:18 smithi082 ceph-mon[98694]: pgmap v9532: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T04:12:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17569.146499 secs 2023-09-10T04:12:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:18 smithi098 ceph-mon[103124]: pgmap v9532: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T04:12:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17570.321441 secs 2023-09-10T04:12:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17570.321441 secs 2023-09-10T04:12:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17570.321441 secs 2023-09-10T04:12:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17570.977446 secs 2023-09-10T04:12:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:20 smithi170 ceph-mon[102832]: pgmap v9533: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T04:12:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17570.977446 secs 2023-09-10T04:12:20.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:20 smithi082 ceph-mon[98694]: pgmap v9533: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T04:12:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17570.977446 secs 2023-09-10T04:12:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:20 smithi098 ceph-mon[103124]: pgmap v9533: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T04:12:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:22 smithi170 ceph-mon[102832]: pgmap v9534: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T04:12:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:22 smithi082 ceph-mon[98694]: pgmap v9534: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T04:12:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:22 smithi098 ceph-mon[103124]: pgmap v9534: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T04:12:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17574.146562 secs 2023-09-10T04:12:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17574.146562 secs 2023-09-10T04:12:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17574.146562 secs 2023-09-10T04:12:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17575.321491 secs 2023-09-10T04:12:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:24 smithi170 ceph-mon[102832]: pgmap v9535: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T04:12:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17575.321491 secs 2023-09-10T04:12:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:24 smithi082 ceph-mon[98694]: pgmap v9535: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T04:12:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17575.321491 secs 2023-09-10T04:12:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:24 smithi098 ceph-mon[103124]: pgmap v9535: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T04:12:25.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17575.977509 secs 2023-09-10T04:12:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17575.977509 secs 2023-09-10T04:12:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17575.977509 secs 2023-09-10T04:12:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:26 smithi170 ceph-mon[102832]: pgmap v9536: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 152 KiB/s wr, 15 op/s 2023-09-10T04:12:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:26 smithi082 ceph-mon[98694]: pgmap v9536: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 152 KiB/s wr, 15 op/s 2023-09-10T04:12:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:26 smithi098 ceph-mon[103124]: pgmap v9536: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 152 KiB/s wr, 15 op/s 2023-09-10T04:12:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17579.146620 secs 2023-09-10T04:12:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:28 smithi170 ceph-mon[102832]: pgmap v9537: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:12:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17579.146620 secs 2023-09-10T04:12:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:28 smithi082 ceph-mon[98694]: pgmap v9537: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:12:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17579.146620 secs 2023-09-10T04:12:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:28 smithi098 ceph-mon[103124]: pgmap v9537: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:12:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17580.321548 secs 2023-09-10T04:12:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17580.321548 secs 2023-09-10T04:12:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17580.321548 secs 2023-09-10T04:12:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17580.977576 secs 2023-09-10T04:12:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:30 smithi170 ceph-mon[102832]: pgmap v9538: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:12:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17580.977576 secs 2023-09-10T04:12:30.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:30 smithi082 ceph-mon[98694]: pgmap v9538: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:12:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17580.977576 secs 2023-09-10T04:12:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:30 smithi098 ceph-mon[103124]: pgmap v9538: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:12:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:32 smithi170 ceph-mon[102832]: pgmap v9539: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 213 KiB/s wr, 15 op/s 2023-09-10T04:12:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:32 smithi082 ceph-mon[98694]: pgmap v9539: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 213 KiB/s wr, 15 op/s 2023-09-10T04:12:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:32 smithi098 ceph-mon[103124]: pgmap v9539: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 213 KiB/s wr, 15 op/s 2023-09-10T04:12:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17584.146697 secs 2023-09-10T04:12:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17584.146697 secs 2023-09-10T04:12:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17584.146697 secs 2023-09-10T04:12:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17585.321605 secs 2023-09-10T04:12:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:34 smithi170 ceph-mon[102832]: pgmap v9540: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 213 KiB/s wr, 11 op/s 2023-09-10T04:12:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17585.321605 secs 2023-09-10T04:12:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:34 smithi082 ceph-mon[98694]: pgmap v9540: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 213 KiB/s wr, 11 op/s 2023-09-10T04:12:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17585.321605 secs 2023-09-10T04:12:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:34 smithi098 ceph-mon[103124]: pgmap v9540: 97 pgs: 97 active+clean; 9.7 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 213 KiB/s wr, 11 op/s 2023-09-10T04:12:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17585.977641 secs 2023-09-10T04:12:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17585.977641 secs 2023-09-10T04:12:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17585.977641 secs 2023-09-10T04:12:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:36 smithi170 ceph-mon[102832]: pgmap v9541: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 339 KiB/s wr, 17 op/s 2023-09-10T04:12:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:36 smithi082 ceph-mon[98694]: pgmap v9541: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 339 KiB/s wr, 17 op/s 2023-09-10T04:12:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:36 smithi098 ceph-mon[103124]: pgmap v9541: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 339 KiB/s wr, 17 op/s 2023-09-10T04:12:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:39 smithi082 ceph-mon[98694]: pgmap v9542: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T04:12:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17590.321689 secs 2023-09-10T04:12:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:12:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:39 smithi098 ceph-mon[103124]: pgmap v9542: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T04:12:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17590.321689 secs 2023-09-10T04:12:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:12:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:39 smithi170 ceph-mon[102832]: pgmap v9542: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T04:12:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17590.321689 secs 2023-09-10T04:12:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:12:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17590.977704 secs 2023-09-10T04:12:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:12:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:12:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:12:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17590.977704 secs 2023-09-10T04:12:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:12:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:12:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:12:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17590.977704 secs 2023-09-10T04:12:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:12:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:12:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:12:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:41 smithi082 ceph-mon[98694]: pgmap v9543: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T04:12:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:41 smithi098 ceph-mon[103124]: pgmap v9543: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T04:12:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:41 smithi170 ceph-mon[102832]: pgmap v9543: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T04:12:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:42 smithi082 ceph-mon[98694]: pgmap v9544: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 432 KiB/s wr, 18 op/s 2023-09-10T04:12:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:42 smithi098 ceph-mon[103124]: pgmap v9544: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 432 KiB/s wr, 18 op/s 2023-09-10T04:12:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:42 smithi170 ceph-mon[102832]: pgmap v9544: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 432 KiB/s wr, 18 op/s 2023-09-10T04:12:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17595.321745 secs 2023-09-10T04:12:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17595.321745 secs 2023-09-10T04:12:44.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17595.321745 secs 2023-09-10T04:12:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:45 smithi082 ceph-mon[98694]: pgmap v9545: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T04:12:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17595.977766 secs 2023-09-10T04:12:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:45 smithi098 ceph-mon[103124]: pgmap v9545: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T04:12:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17595.977766 secs 2023-09-10T04:12:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:45 smithi170 ceph-mon[102832]: pgmap v9545: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T04:12:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17595.977766 secs 2023-09-10T04:12:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:46 smithi170 ceph-mon[102832]: pgmap v9546: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 404 KiB/s wr, 18 op/s 2023-09-10T04:12:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:46 smithi082 ceph-mon[98694]: pgmap v9546: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 404 KiB/s wr, 18 op/s 2023-09-10T04:12:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:46 smithi098 ceph-mon[103124]: pgmap v9546: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 404 KiB/s wr, 18 op/s 2023-09-10T04:12:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:49 smithi082 ceph-mon[98694]: pgmap v9547: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T04:12:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17600.321804 secs 2023-09-10T04:12:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:49 smithi098 ceph-mon[103124]: pgmap v9547: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T04:12:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17600.321804 secs 2023-09-10T04:12:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:49 smithi170 ceph-mon[102832]: pgmap v9547: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T04:12:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17600.321804 secs 2023-09-10T04:12:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17600.977829 secs 2023-09-10T04:12:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17600.977829 secs 2023-09-10T04:12:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17600.977829 secs 2023-09-10T04:12:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:51 smithi082 ceph-mon[98694]: pgmap v9548: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T04:12:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:51 smithi098 ceph-mon[103124]: pgmap v9548: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T04:12:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:51 smithi170 ceph-mon[102832]: pgmap v9548: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T04:12:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:52 smithi170 ceph-mon[102832]: pgmap v9549: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 411 KiB/s wr, 19 op/s 2023-09-10T04:12:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:52 smithi082 ceph-mon[98694]: pgmap v9549: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 411 KiB/s wr, 19 op/s 2023-09-10T04:12:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:52 smithi098 ceph-mon[103124]: pgmap v9549: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 411 KiB/s wr, 19 op/s 2023-09-10T04:12:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17604.146959 secs 2023-09-10T04:12:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17604.146959 secs 2023-09-10T04:12:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17604.146959 secs 2023-09-10T04:12:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17605.321888 secs 2023-09-10T04:12:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:54 smithi170 ceph-mon[102832]: pgmap v9550: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T04:12:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17605.321888 secs 2023-09-10T04:12:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:54 smithi082 ceph-mon[98694]: pgmap v9550: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T04:12:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17605.321888 secs 2023-09-10T04:12:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:54 smithi098 ceph-mon[103124]: pgmap v9550: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T04:12:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17605.977896 secs 2023-09-10T04:12:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17605.977896 secs 2023-09-10T04:12:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17605.977896 secs 2023-09-10T04:12:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:56 smithi170 ceph-mon[102832]: pgmap v9551: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 322 KiB/s wr, 18 op/s 2023-09-10T04:12:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:12:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:12:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:12:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:56 smithi082 ceph-mon[98694]: pgmap v9551: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 322 KiB/s wr, 18 op/s 2023-09-10T04:12:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:12:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:12:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:12:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:56 smithi098 ceph-mon[103124]: pgmap v9551: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 322 KiB/s wr, 18 op/s 2023-09-10T04:12:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:12:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:12:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:12:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17609.147065 secs 2023-09-10T04:12:58.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:58 smithi170 ceph-mon[102832]: pgmap v9552: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 196 KiB/s wr, 12 op/s 2023-09-10T04:12:58.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:12:58.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:12:58.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:12:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17609.147065 secs 2023-09-10T04:12:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:58 smithi082 ceph-mon[98694]: pgmap v9552: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 196 KiB/s wr, 12 op/s 2023-09-10T04:12:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:12:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:12:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:12:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17609.147065 secs 2023-09-10T04:12:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:58 smithi098 ceph-mon[103124]: pgmap v9552: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 196 KiB/s wr, 12 op/s 2023-09-10T04:12:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:12:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:12:59.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:12:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17610.321957 secs 2023-09-10T04:12:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:12:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:12:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:12:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:12:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17610.321957 secs 2023-09-10T04:12:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:12:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:12:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:12:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:13:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17610.321957 secs 2023-09-10T04:13:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:13:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:13:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:12:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:13:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17610.977959 secs 2023-09-10T04:13:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:00 smithi170 ceph-mon[102832]: pgmap v9553: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 196 KiB/s wr, 12 op/s 2023-09-10T04:13:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17610.977959 secs 2023-09-10T04:13:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:00 smithi082 ceph-mon[98694]: pgmap v9553: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 196 KiB/s wr, 12 op/s 2023-09-10T04:13:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17610.977959 secs 2023-09-10T04:13:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:00 smithi098 ceph-mon[103124]: pgmap v9553: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 196 KiB/s wr, 12 op/s 2023-09-10T04:13:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:02 smithi170 ceph-mon[102832]: pgmap v9554: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 338 KiB/s wr, 17 op/s 2023-09-10T04:13:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:02 smithi082 ceph-mon[98694]: pgmap v9554: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 338 KiB/s wr, 17 op/s 2023-09-10T04:13:03.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:02 smithi098 ceph-mon[103124]: pgmap v9554: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 338 KiB/s wr, 17 op/s 2023-09-10T04:13:03.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17614.147152 secs 2023-09-10T04:13:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17614.147152 secs 2023-09-10T04:13:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17614.147152 secs 2023-09-10T04:13:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17615.322012 secs 2023-09-10T04:13:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:04 smithi170 ceph-mon[102832]: pgmap v9555: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 204 KiB/s wr, 11 op/s 2023-09-10T04:13:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17615.322012 secs 2023-09-10T04:13:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:04 smithi082 ceph-mon[98694]: pgmap v9555: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 204 KiB/s wr, 11 op/s 2023-09-10T04:13:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17615.322012 secs 2023-09-10T04:13:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:04 smithi098 ceph-mon[103124]: pgmap v9555: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 204 KiB/s wr, 11 op/s 2023-09-10T04:13:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17615.978025 secs 2023-09-10T04:13:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17615.978025 secs 2023-09-10T04:13:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17615.978025 secs 2023-09-10T04:13:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:06 smithi170 ceph-mon[102832]: pgmap v9556: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 291 KiB/s wr, 31 op/s 2023-09-10T04:13:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:06 smithi082 ceph-mon[98694]: pgmap v9556: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 291 KiB/s wr, 31 op/s 2023-09-10T04:13:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:06 smithi098 ceph-mon[103124]: pgmap v9556: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 291 KiB/s wr, 31 op/s 2023-09-10T04:13:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17619.147218 secs 2023-09-10T04:13:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:08 smithi170 ceph-mon[102832]: pgmap v9557: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 229 KiB/s wr, 26 op/s 2023-09-10T04:13:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:08 smithi170 ceph-mon[102832]: osdmap e665: 12 total, 12 up, 12 in 2023-09-10T04:13:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17619.147218 secs 2023-09-10T04:13:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:08 smithi082 ceph-mon[98694]: pgmap v9557: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 229 KiB/s wr, 26 op/s 2023-09-10T04:13:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:08 smithi082 ceph-mon[98694]: osdmap e665: 12 total, 12 up, 12 in 2023-09-10T04:13:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17619.147218 secs 2023-09-10T04:13:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:08 smithi098 ceph-mon[103124]: pgmap v9557: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 229 KiB/s wr, 26 op/s 2023-09-10T04:13:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:08 smithi098 ceph-mon[103124]: osdmap e665: 12 total, 12 up, 12 in 2023-09-10T04:13:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17620.322074 secs 2023-09-10T04:13:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17620.322074 secs 2023-09-10T04:13:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17620.322074 secs 2023-09-10T04:13:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17620.978087 secs 2023-09-10T04:13:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:10 smithi170 ceph-mon[102832]: pgmap v9559: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 275 KiB/s wr, 31 op/s 2023-09-10T04:13:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17620.978087 secs 2023-09-10T04:13:10.969 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:10 smithi082 ceph-mon[98694]: pgmap v9559: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 275 KiB/s wr, 31 op/s 2023-09-10T04:13:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17620.978087 secs 2023-09-10T04:13:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:10 smithi098 ceph-mon[103124]: pgmap v9559: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 275 KiB/s wr, 31 op/s 2023-09-10T04:13:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:12 smithi170 ceph-mon[102832]: pgmap v9560: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 105 KiB/s wr, 30 op/s 2023-09-10T04:13:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:12 smithi082 ceph-mon[98694]: pgmap v9560: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 105 KiB/s wr, 30 op/s 2023-09-10T04:13:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:12 smithi098 ceph-mon[103124]: pgmap v9560: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 105 KiB/s wr, 30 op/s 2023-09-10T04:13:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17624.147276 secs 2023-09-10T04:13:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:13 smithi170 ceph-mon[102832]: osdmap e666: 12 total, 12 up, 12 in 2023-09-10T04:13:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17624.147276 secs 2023-09-10T04:13:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:13 smithi082 ceph-mon[98694]: osdmap e666: 12 total, 12 up, 12 in 2023-09-10T04:13:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17624.147276 secs 2023-09-10T04:13:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:13 smithi098 ceph-mon[103124]: osdmap e666: 12 total, 12 up, 12 in 2023-09-10T04:13:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17625.322134 secs 2023-09-10T04:13:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:14 smithi170 ceph-mon[102832]: pgmap v9562: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T04:13:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17625.322134 secs 2023-09-10T04:13:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:14 smithi082 ceph-mon[98694]: pgmap v9562: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T04:13:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17625.322134 secs 2023-09-10T04:13:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:14 smithi098 ceph-mon[103124]: pgmap v9562: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T04:13:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17625.978151 secs 2023-09-10T04:13:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17625.978151 secs 2023-09-10T04:13:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17625.978151 secs 2023-09-10T04:13:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:16 smithi170 ceph-mon[102832]: pgmap v9563: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 15 op/s 2023-09-10T04:13:16.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:16 smithi082 ceph-mon[98694]: pgmap v9563: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 15 op/s 2023-09-10T04:13:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:16 smithi098 ceph-mon[103124]: pgmap v9563: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 15 op/s 2023-09-10T04:13:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17629.147394 secs 2023-09-10T04:13:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:18 smithi170 ceph-mon[102832]: pgmap v9564: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-09-10T04:13:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17629.147394 secs 2023-09-10T04:13:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:18 smithi082 ceph-mon[98694]: pgmap v9564: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-09-10T04:13:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17629.147394 secs 2023-09-10T04:13:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:18 smithi098 ceph-mon[103124]: pgmap v9564: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 12 op/s 2023-09-10T04:13:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17630.322210 secs 2023-09-10T04:13:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17630.322210 secs 2023-09-10T04:13:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17630.322210 secs 2023-09-10T04:13:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17630.978213 secs 2023-09-10T04:13:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:20 smithi170 ceph-mon[102832]: pgmap v9565: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 12 op/s 2023-09-10T04:13:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17630.978213 secs 2023-09-10T04:13:20.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:20 smithi082 ceph-mon[98694]: pgmap v9565: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 12 op/s 2023-09-10T04:13:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17630.978213 secs 2023-09-10T04:13:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:20 smithi098 ceph-mon[103124]: pgmap v9565: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 12 op/s 2023-09-10T04:13:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:22 smithi170 ceph-mon[102832]: pgmap v9566: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T04:13:22.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:22 smithi082 ceph-mon[98694]: pgmap v9566: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T04:13:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:22 smithi098 ceph-mon[103124]: pgmap v9566: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T04:13:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17634.147512 secs 2023-09-10T04:13:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17634.147512 secs 2023-09-10T04:13:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17634.147512 secs 2023-09-10T04:13:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17635.322269 secs 2023-09-10T04:13:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:24 smithi170 ceph-mon[102832]: pgmap v9567: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T04:13:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17635.322269 secs 2023-09-10T04:13:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:24 smithi082 ceph-mon[98694]: pgmap v9567: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T04:13:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17635.322269 secs 2023-09-10T04:13:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:24 smithi098 ceph-mon[103124]: pgmap v9567: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T04:13:25.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17635.978277 secs 2023-09-10T04:13:25.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17635.978277 secs 2023-09-10T04:13:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17635.978277 secs 2023-09-10T04:13:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:26 smithi170 ceph-mon[102832]: pgmap v9568: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 172 KiB/s wr, 14 op/s 2023-09-10T04:13:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:26 smithi082 ceph-mon[98694]: pgmap v9568: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 172 KiB/s wr, 14 op/s 2023-09-10T04:13:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:26 smithi098 ceph-mon[103124]: pgmap v9568: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 172 KiB/s wr, 14 op/s 2023-09-10T04:13:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17639.147573 secs 2023-09-10T04:13:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:28 smithi170 ceph-mon[102832]: pgmap v9569: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 110 KiB/s wr, 9 op/s 2023-09-10T04:13:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17639.147573 secs 2023-09-10T04:13:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:28 smithi082 ceph-mon[98694]: pgmap v9569: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 110 KiB/s wr, 9 op/s 2023-09-10T04:13:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17639.147573 secs 2023-09-10T04:13:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:28 smithi098 ceph-mon[103124]: pgmap v9569: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 110 KiB/s wr, 9 op/s 2023-09-10T04:13:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17640.322327 secs 2023-09-10T04:13:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17640.322327 secs 2023-09-10T04:13:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17640.322327 secs 2023-09-10T04:13:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17640.978344 secs 2023-09-10T04:13:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:30 smithi170 ceph-mon[102832]: pgmap v9570: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 110 KiB/s wr, 9 op/s 2023-09-10T04:13:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17640.978344 secs 2023-09-10T04:13:30.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:30 smithi082 ceph-mon[98694]: pgmap v9570: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 110 KiB/s wr, 9 op/s 2023-09-10T04:13:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17640.978344 secs 2023-09-10T04:13:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:30 smithi098 ceph-mon[103124]: pgmap v9570: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 110 KiB/s wr, 9 op/s 2023-09-10T04:13:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:32 smithi170 ceph-mon[102832]: pgmap v9571: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 214 KiB/s wr, 15 op/s 2023-09-10T04:13:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:32 smithi082 ceph-mon[98694]: pgmap v9571: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 214 KiB/s wr, 15 op/s 2023-09-10T04:13:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:32 smithi098 ceph-mon[103124]: pgmap v9571: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 214 KiB/s wr, 15 op/s 2023-09-10T04:13:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17644.147663 secs 2023-09-10T04:13:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17644.147663 secs 2023-09-10T04:13:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17644.147663 secs 2023-09-10T04:13:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17645.322412 secs 2023-09-10T04:13:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:34 smithi170 ceph-mon[102832]: pgmap v9572: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 213 KiB/s wr, 11 op/s 2023-09-10T04:13:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17645.322412 secs 2023-09-10T04:13:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:34 smithi082 ceph-mon[98694]: pgmap v9572: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 213 KiB/s wr, 11 op/s 2023-09-10T04:13:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17645.322412 secs 2023-09-10T04:13:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:34 smithi098 ceph-mon[103124]: pgmap v9572: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 213 KiB/s wr, 11 op/s 2023-09-10T04:13:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17645.978408 secs 2023-09-10T04:13:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17645.978408 secs 2023-09-10T04:13:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17645.978408 secs 2023-09-10T04:13:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:36 smithi170 ceph-mon[102832]: pgmap v9573: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 341 KiB/s wr, 17 op/s 2023-09-10T04:13:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:36 smithi082 ceph-mon[98694]: pgmap v9573: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 341 KiB/s wr, 17 op/s 2023-09-10T04:13:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:36 smithi098 ceph-mon[103124]: pgmap v9573: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 341 KiB/s wr, 17 op/s 2023-09-10T04:13:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:38 smithi170 ceph-mon[102832]: pgmap v9574: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 231 KiB/s wr, 12 op/s 2023-09-10T04:13:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:38 smithi082 ceph-mon[98694]: pgmap v9574: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 231 KiB/s wr, 12 op/s 2023-09-10T04:13:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:38 smithi098 ceph-mon[103124]: pgmap v9574: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 231 KiB/s wr, 12 op/s 2023-09-10T04:13:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17650.322472 secs 2023-09-10T04:13:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:13:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17650.322472 secs 2023-09-10T04:13:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:13:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17650.322472 secs 2023-09-10T04:13:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:13:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17650.978477 secs 2023-09-10T04:13:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:40 smithi170 ceph-mon[102832]: pgmap v9575: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 231 KiB/s wr, 12 op/s 2023-09-10T04:13:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:13:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:13:40.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:13:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17650.978477 secs 2023-09-10T04:13:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:40 smithi082 ceph-mon[98694]: pgmap v9575: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 231 KiB/s wr, 12 op/s 2023-09-10T04:13:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:13:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:13:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:13:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17650.978477 secs 2023-09-10T04:13:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:40 smithi098 ceph-mon[103124]: pgmap v9575: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 231 KiB/s wr, 12 op/s 2023-09-10T04:13:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:13:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:13:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:13:42.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:42 smithi170 ceph-mon[102832]: pgmap v9576: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 385 KiB/s wr, 19 op/s 2023-09-10T04:13:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:42 smithi082 ceph-mon[98694]: pgmap v9576: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 385 KiB/s wr, 19 op/s 2023-09-10T04:13:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:42 smithi098 ceph-mon[103124]: pgmap v9576: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 385 KiB/s wr, 19 op/s 2023-09-10T04:13:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17655.322531 secs 2023-09-10T04:13:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:44 smithi098 ceph-mon[103124]: pgmap v9577: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 281 KiB/s wr, 13 op/s 2023-09-10T04:13:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17655.322531 secs 2023-09-10T04:13:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:44 smithi170 ceph-mon[102832]: pgmap v9577: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 281 KiB/s wr, 13 op/s 2023-09-10T04:13:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17655.322531 secs 2023-09-10T04:13:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:44 smithi082 ceph-mon[98694]: pgmap v9577: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 281 KiB/s wr, 13 op/s 2023-09-10T04:13:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17655.978541 secs 2023-09-10T04:13:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17655.978541 secs 2023-09-10T04:13:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17655.978541 secs 2023-09-10T04:13:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:46 smithi098 ceph-mon[103124]: pgmap v9578: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 406 KiB/s wr, 19 op/s 2023-09-10T04:13:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:46 smithi170 ceph-mon[102832]: pgmap v9578: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 406 KiB/s wr, 19 op/s 2023-09-10T04:13:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:46 smithi082 ceph-mon[98694]: pgmap v9578: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 406 KiB/s wr, 19 op/s 2023-09-10T04:13:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:48 smithi170 ceph-mon[102832]: pgmap v9579: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T04:13:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:48 smithi082 ceph-mon[98694]: pgmap v9579: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T04:13:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:48 smithi098 ceph-mon[103124]: pgmap v9579: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T04:13:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17660.322588 secs 2023-09-10T04:13:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17660.322588 secs 2023-09-10T04:13:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17660.322588 secs 2023-09-10T04:13:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17660.978608 secs 2023-09-10T04:13:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:50 smithi170 ceph-mon[102832]: pgmap v9580: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T04:13:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17660.978608 secs 2023-09-10T04:13:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:50 smithi082 ceph-mon[98694]: pgmap v9580: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T04:13:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17660.978608 secs 2023-09-10T04:13:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:50 smithi098 ceph-mon[103124]: pgmap v9580: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T04:13:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:52 smithi170 ceph-mon[102832]: pgmap v9581: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 429 KiB/s wr, 19 op/s 2023-09-10T04:13:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:52 smithi082 ceph-mon[98694]: pgmap v9581: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 429 KiB/s wr, 19 op/s 2023-09-10T04:13:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:52 smithi098 ceph-mon[103124]: pgmap v9581: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 429 KiB/s wr, 19 op/s 2023-09-10T04:13:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17664.147961 secs 2023-09-10T04:13:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17664.147961 secs 2023-09-10T04:13:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17664.147961 secs 2023-09-10T04:13:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17665.322679 secs 2023-09-10T04:13:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:54 smithi170 ceph-mon[102832]: pgmap v9582: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 275 KiB/s wr, 12 op/s 2023-09-10T04:13:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17665.322679 secs 2023-09-10T04:13:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:54 smithi082 ceph-mon[98694]: pgmap v9582: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 275 KiB/s wr, 12 op/s 2023-09-10T04:13:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17665.322679 secs 2023-09-10T04:13:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:54 smithi098 ceph-mon[103124]: pgmap v9582: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 275 KiB/s wr, 12 op/s 2023-09-10T04:13:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17665.978670 secs 2023-09-10T04:13:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17665.978670 secs 2023-09-10T04:13:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17665.978670 secs 2023-09-10T04:13:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:56 smithi170 ceph-mon[102832]: pgmap v9583: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 332 KiB/s wr, 17 op/s 2023-09-10T04:13:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:13:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:13:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:13:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:56 smithi082 ceph-mon[98694]: pgmap v9583: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 332 KiB/s wr, 17 op/s 2023-09-10T04:13:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:13:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:13:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:13:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:56 smithi098 ceph-mon[103124]: pgmap v9583: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 332 KiB/s wr, 17 op/s 2023-09-10T04:13:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:13:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:13:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:13:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17669.148050 secs 2023-09-10T04:13:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:58 smithi170 ceph-mon[102832]: pgmap v9584: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 207 KiB/s wr, 12 op/s 2023-09-10T04:13:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17669.148050 secs 2023-09-10T04:13:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:58 smithi082 ceph-mon[98694]: pgmap v9584: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 207 KiB/s wr, 12 op/s 2023-09-10T04:13:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17669.148050 secs 2023-09-10T04:13:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:58 smithi098 ceph-mon[103124]: pgmap v9584: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 207 KiB/s wr, 12 op/s 2023-09-10T04:13:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17670.322763 secs 2023-09-10T04:13:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:13:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:13:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:13:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:13:59.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:13:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:13:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17670.322763 secs 2023-09-10T04:13:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:13:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:13:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:13:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:13:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:13:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:14:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17670.322763 secs 2023-09-10T04:14:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:14:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:14:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:14:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:14:00.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:13:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:14:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17670.978733 secs 2023-09-10T04:14:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:00 smithi170 ceph-mon[102832]: pgmap v9585: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 207 KiB/s wr, 12 op/s 2023-09-10T04:14:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:14:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:14:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:14:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17670.978733 secs 2023-09-10T04:14:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:00 smithi082 ceph-mon[98694]: pgmap v9585: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 207 KiB/s wr, 12 op/s 2023-09-10T04:14:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:14:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:14:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:14:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17670.978733 secs 2023-09-10T04:14:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:00 smithi098 ceph-mon[103124]: pgmap v9585: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 207 KiB/s wr, 12 op/s 2023-09-10T04:14:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:14:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:14:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:14:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:02 smithi170 ceph-mon[102832]: pgmap v9586: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 301 KiB/s wr, 18 op/s 2023-09-10T04:14:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:02 smithi082 ceph-mon[98694]: pgmap v9586: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 301 KiB/s wr, 18 op/s 2023-09-10T04:14:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:02 smithi098 ceph-mon[103124]: pgmap v9586: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 301 KiB/s wr, 18 op/s 2023-09-10T04:14:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17674.148109 secs 2023-09-10T04:14:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17674.148109 secs 2023-09-10T04:14:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17674.148109 secs 2023-09-10T04:14:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17675.322821 secs 2023-09-10T04:14:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:04 smithi170 ceph-mon[102832]: pgmap v9587: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T04:14:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17675.322821 secs 2023-09-10T04:14:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:04 smithi082 ceph-mon[98694]: pgmap v9587: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T04:14:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17675.322821 secs 2023-09-10T04:14:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:04 smithi098 ceph-mon[103124]: pgmap v9587: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T04:14:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17675.978799 secs 2023-09-10T04:14:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17675.978799 secs 2023-09-10T04:14:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17675.978799 secs 2023-09-10T04:14:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:06 smithi170 ceph-mon[102832]: pgmap v9588: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 238 KiB/s wr, 31 op/s 2023-09-10T04:14:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:06 smithi082 ceph-mon[98694]: pgmap v9588: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 238 KiB/s wr, 31 op/s 2023-09-10T04:14:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:06 smithi098 ceph-mon[103124]: pgmap v9588: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 238 KiB/s wr, 31 op/s 2023-09-10T04:14:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17679.148171 secs 2023-09-10T04:14:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:08 smithi170 ceph-mon[102832]: pgmap v9589: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 181 KiB/s wr, 25 op/s 2023-09-10T04:14:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:08 smithi170 ceph-mon[102832]: osdmap e667: 12 total, 12 up, 12 in 2023-09-10T04:14:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17679.148171 secs 2023-09-10T04:14:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:08 smithi082 ceph-mon[98694]: pgmap v9589: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 181 KiB/s wr, 25 op/s 2023-09-10T04:14:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:08 smithi082 ceph-mon[98694]: osdmap e667: 12 total, 12 up, 12 in 2023-09-10T04:14:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17679.148171 secs 2023-09-10T04:14:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:08 smithi098 ceph-mon[103124]: pgmap v9589: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 181 KiB/s wr, 25 op/s 2023-09-10T04:14:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:08 smithi098 ceph-mon[103124]: osdmap e667: 12 total, 12 up, 12 in 2023-09-10T04:14:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17680.322880 secs 2023-09-10T04:14:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17680.322880 secs 2023-09-10T04:14:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17680.322880 secs 2023-09-10T04:14:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17680.978862 secs 2023-09-10T04:14:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:10 smithi170 ceph-mon[102832]: pgmap v9591: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 217 KiB/s wr, 30 op/s 2023-09-10T04:14:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17680.978862 secs 2023-09-10T04:14:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:10 smithi082 ceph-mon[98694]: pgmap v9591: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 217 KiB/s wr, 30 op/s 2023-09-10T04:14:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17680.978862 secs 2023-09-10T04:14:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:10 smithi098 ceph-mon[103124]: pgmap v9591: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 217 KiB/s wr, 30 op/s 2023-09-10T04:14:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:12 smithi170 ceph-mon[102832]: pgmap v9592: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T04:14:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:12 smithi082 ceph-mon[98694]: pgmap v9592: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T04:14:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:12 smithi098 ceph-mon[103124]: pgmap v9592: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 105 KiB/s wr, 29 op/s 2023-09-10T04:14:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17684.148233 secs 2023-09-10T04:14:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:13 smithi170 ceph-mon[102832]: osdmap e668: 12 total, 12 up, 12 in 2023-09-10T04:14:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17684.148233 secs 2023-09-10T04:14:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:13 smithi082 ceph-mon[98694]: osdmap e668: 12 total, 12 up, 12 in 2023-09-10T04:14:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17684.148233 secs 2023-09-10T04:14:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:13 smithi098 ceph-mon[103124]: osdmap e668: 12 total, 12 up, 12 in 2023-09-10T04:14:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17685.322969 secs 2023-09-10T04:14:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:14 smithi170 ceph-mon[102832]: pgmap v9594: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:14:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17685.322969 secs 2023-09-10T04:14:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:14 smithi082 ceph-mon[98694]: pgmap v9594: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:14:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17685.322969 secs 2023-09-10T04:14:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:14 smithi098 ceph-mon[103124]: pgmap v9594: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:14:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17685.978925 secs 2023-09-10T04:14:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17685.978925 secs 2023-09-10T04:14:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17685.978925 secs 2023-09-10T04:14:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:16 smithi170 ceph-mon[102832]: pgmap v9595: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 143 KiB/s wr, 15 op/s 2023-09-10T04:14:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:16 smithi082 ceph-mon[98694]: pgmap v9595: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 143 KiB/s wr, 15 op/s 2023-09-10T04:14:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:16 smithi098 ceph-mon[103124]: pgmap v9595: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 143 KiB/s wr, 15 op/s 2023-09-10T04:14:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17689.148327 secs 2023-09-10T04:14:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:18 smithi170 ceph-mon[102832]: pgmap v9596: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T04:14:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17689.148327 secs 2023-09-10T04:14:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:18 smithi082 ceph-mon[98694]: pgmap v9596: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T04:14:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17689.148327 secs 2023-09-10T04:14:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:18 smithi098 ceph-mon[103124]: pgmap v9596: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T04:14:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17690.323056 secs 2023-09-10T04:14:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17690.323056 secs 2023-09-10T04:14:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17690.323056 secs 2023-09-10T04:14:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17690.978991 secs 2023-09-10T04:14:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:20 smithi170 ceph-mon[102832]: pgmap v9597: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T04:14:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17690.978991 secs 2023-09-10T04:14:20.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:20 smithi082 ceph-mon[98694]: pgmap v9597: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T04:14:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17690.978991 secs 2023-09-10T04:14:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:20 smithi098 ceph-mon[103124]: pgmap v9597: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T04:14:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:22 smithi170 ceph-mon[102832]: pgmap v9598: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T04:14:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:22 smithi082 ceph-mon[98694]: pgmap v9598: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T04:14:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:22 smithi098 ceph-mon[103124]: pgmap v9598: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T04:14:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17694.148421 secs 2023-09-10T04:14:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17694.148421 secs 2023-09-10T04:14:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17694.148421 secs 2023-09-10T04:14:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17695.323113 secs 2023-09-10T04:14:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:24 smithi170 ceph-mon[102832]: pgmap v9599: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T04:14:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17695.323113 secs 2023-09-10T04:14:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:24 smithi082 ceph-mon[98694]: pgmap v9599: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T04:14:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17695.323113 secs 2023-09-10T04:14:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:24 smithi098 ceph-mon[103124]: pgmap v9599: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T04:14:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17695.979056 secs 2023-09-10T04:14:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17695.979056 secs 2023-09-10T04:14:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17695.979056 secs 2023-09-10T04:14:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:26 smithi170 ceph-mon[102832]: pgmap v9600: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 14 op/s 2023-09-10T04:14:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:26 smithi082 ceph-mon[98694]: pgmap v9600: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 14 op/s 2023-09-10T04:14:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:26 smithi098 ceph-mon[103124]: pgmap v9600: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 14 op/s 2023-09-10T04:14:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17699.148484 secs 2023-09-10T04:14:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:28 smithi170 ceph-mon[102832]: pgmap v9601: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T04:14:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17699.148484 secs 2023-09-10T04:14:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:28 smithi082 ceph-mon[98694]: pgmap v9601: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T04:14:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17699.148484 secs 2023-09-10T04:14:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:28 smithi098 ceph-mon[103124]: pgmap v9601: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T04:14:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17700.323208 secs 2023-09-10T04:14:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17700.979122 secs 2023-09-10T04:14:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17700.323208 secs 2023-09-10T04:14:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17700.979122 secs 2023-09-10T04:14:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17700.323208 secs 2023-09-10T04:14:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17700.979122 secs 2023-09-10T04:14:30.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:30 smithi170 ceph-mon[102832]: pgmap v9602: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T04:14:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:30 smithi082 ceph-mon[98694]: pgmap v9602: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T04:14:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:30 smithi098 ceph-mon[103124]: pgmap v9602: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T04:14:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:32 smithi170 ceph-mon[102832]: pgmap v9603: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 215 KiB/s wr, 15 op/s 2023-09-10T04:14:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:32 smithi082 ceph-mon[98694]: pgmap v9603: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 215 KiB/s wr, 15 op/s 2023-09-10T04:14:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:32 smithi098 ceph-mon[103124]: pgmap v9603: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 215 KiB/s wr, 15 op/s 2023-09-10T04:14:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17704.148544 secs 2023-09-10T04:14:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17704.148544 secs 2023-09-10T04:14:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17704.148544 secs 2023-09-10T04:14:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17705.323267 secs 2023-09-10T04:14:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:34 smithi170 ceph-mon[102832]: pgmap v9604: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 215 KiB/s wr, 11 op/s 2023-09-10T04:14:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17705.979189 secs 2023-09-10T04:14:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17705.323267 secs 2023-09-10T04:14:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:34 smithi082 ceph-mon[98694]: pgmap v9604: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 215 KiB/s wr, 11 op/s 2023-09-10T04:14:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17705.979189 secs 2023-09-10T04:14:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17705.323267 secs 2023-09-10T04:14:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:34 smithi098 ceph-mon[103124]: pgmap v9604: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 215 KiB/s wr, 11 op/s 2023-09-10T04:14:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17705.979189 secs 2023-09-10T04:14:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:36 smithi170 ceph-mon[102832]: pgmap v9605: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 343 KiB/s wr, 17 op/s 2023-09-10T04:14:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:36 smithi082 ceph-mon[98694]: pgmap v9605: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 343 KiB/s wr, 17 op/s 2023-09-10T04:14:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:36 smithi098 ceph-mon[103124]: pgmap v9605: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 343 KiB/s wr, 17 op/s 2023-09-10T04:14:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:38 smithi170 ceph-mon[102832]: pgmap v9606: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-09-10T04:14:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:38 smithi082 ceph-mon[98694]: pgmap v9606: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-09-10T04:14:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:38 smithi098 ceph-mon[103124]: pgmap v9606: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-09-10T04:14:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17710.323324 secs 2023-09-10T04:14:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17710.979254 secs 2023-09-10T04:14:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:14:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17710.323324 secs 2023-09-10T04:14:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17710.979254 secs 2023-09-10T04:14:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:14:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17710.323324 secs 2023-09-10T04:14:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17710.979254 secs 2023-09-10T04:14:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:14:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:40 smithi170 ceph-mon[102832]: pgmap v9607: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-09-10T04:14:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:14:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:14:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:14:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:40 smithi082 ceph-mon[98694]: pgmap v9607: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-09-10T04:14:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:14:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:14:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:14:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:40 smithi098 ceph-mon[103124]: pgmap v9607: 97 pgs: 97 active+clean; 9.8 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-09-10T04:14:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:14:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:14:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:14:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:42 smithi170 ceph-mon[102832]: pgmap v9608: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 489 KiB/s wr, 19 op/s 2023-09-10T04:14:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:42 smithi082 ceph-mon[98694]: pgmap v9608: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 489 KiB/s wr, 19 op/s 2023-09-10T04:14:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:42 smithi098 ceph-mon[103124]: pgmap v9608: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 489 KiB/s wr, 19 op/s 2023-09-10T04:14:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17715.323400 secs 2023-09-10T04:14:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:44 smithi098 ceph-mon[103124]: pgmap v9609: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 333 KiB/s wr, 13 op/s 2023-09-10T04:14:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17715.323400 secs 2023-09-10T04:14:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:44 smithi170 ceph-mon[102832]: pgmap v9609: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 333 KiB/s wr, 13 op/s 2023-09-10T04:14:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17715.323400 secs 2023-09-10T04:14:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:44 smithi082 ceph-mon[98694]: pgmap v9609: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 333 KiB/s wr, 13 op/s 2023-09-10T04:14:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17715.979319 secs 2023-09-10T04:14:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17715.979319 secs 2023-09-10T04:14:45.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17715.979319 secs 2023-09-10T04:14:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:46 smithi098 ceph-mon[103124]: pgmap v9610: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 394 KiB/s wr, 19 op/s 2023-09-10T04:14:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:46 smithi170 ceph-mon[102832]: pgmap v9610: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 394 KiB/s wr, 19 op/s 2023-09-10T04:14:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:46 smithi082 ceph-mon[98694]: pgmap v9610: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 394 KiB/s wr, 19 op/s 2023-09-10T04:14:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:48 smithi170 ceph-mon[102832]: pgmap v9611: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 266 KiB/s wr, 12 op/s 2023-09-10T04:14:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:48 smithi082 ceph-mon[98694]: pgmap v9611: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 266 KiB/s wr, 12 op/s 2023-09-10T04:14:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:48 smithi098 ceph-mon[103124]: pgmap v9611: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 266 KiB/s wr, 12 op/s 2023-09-10T04:14:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17720.323460 secs 2023-09-10T04:14:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17720.323460 secs 2023-09-10T04:14:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17720.323460 secs 2023-09-10T04:14:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17720.979381 secs 2023-09-10T04:14:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:50 smithi170 ceph-mon[102832]: pgmap v9612: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 266 KiB/s wr, 12 op/s 2023-09-10T04:14:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17720.979381 secs 2023-09-10T04:14:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:50 smithi082 ceph-mon[98694]: pgmap v9612: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 266 KiB/s wr, 12 op/s 2023-09-10T04:14:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17720.979381 secs 2023-09-10T04:14:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:50 smithi098 ceph-mon[103124]: pgmap v9612: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 266 KiB/s wr, 12 op/s 2023-09-10T04:14:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:52 smithi170 ceph-mon[102832]: pgmap v9613: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 422 KiB/s wr, 18 op/s 2023-09-10T04:14:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:52 smithi082 ceph-mon[98694]: pgmap v9613: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 422 KiB/s wr, 18 op/s 2023-09-10T04:14:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:52 smithi098 ceph-mon[103124]: pgmap v9613: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 422 KiB/s wr, 18 op/s 2023-09-10T04:14:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17724.148829 secs 2023-09-10T04:14:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17724.148829 secs 2023-09-10T04:14:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17724.148829 secs 2023-09-10T04:14:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17725.323499 secs 2023-09-10T04:14:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:54 smithi170 ceph-mon[102832]: pgmap v9614: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 217 KiB/s wr, 11 op/s 2023-09-10T04:14:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17725.323499 secs 2023-09-10T04:14:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:54 smithi082 ceph-mon[98694]: pgmap v9614: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 217 KiB/s wr, 11 op/s 2023-09-10T04:14:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17725.323499 secs 2023-09-10T04:14:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:54 smithi098 ceph-mon[103124]: pgmap v9614: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 217 KiB/s wr, 11 op/s 2023-09-10T04:14:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17725.979446 secs 2023-09-10T04:14:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17725.979446 secs 2023-09-10T04:14:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17725.979446 secs 2023-09-10T04:14:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:56 smithi170 ceph-mon[102832]: pgmap v9615: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 274 KiB/s wr, 17 op/s 2023-09-10T04:14:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:14:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:14:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:14:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:56 smithi082 ceph-mon[98694]: pgmap v9615: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 274 KiB/s wr, 17 op/s 2023-09-10T04:14:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:14:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:14:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:14:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:56 smithi098 ceph-mon[103124]: pgmap v9615: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 274 KiB/s wr, 17 op/s 2023-09-10T04:14:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:14:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:14:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:14:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17729.148896 secs 2023-09-10T04:14:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:58 smithi170 ceph-mon[102832]: pgmap v9616: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 213 KiB/s wr, 11 op/s 2023-09-10T04:14:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17729.148896 secs 2023-09-10T04:14:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:58 smithi082 ceph-mon[98694]: pgmap v9616: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 213 KiB/s wr, 11 op/s 2023-09-10T04:14:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17729.148896 secs 2023-09-10T04:14:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:58 smithi098 ceph-mon[103124]: pgmap v9616: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 213 KiB/s wr, 11 op/s 2023-09-10T04:14:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:14:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17730.323584 secs 2023-09-10T04:14:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:14:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17730.323584 secs 2023-09-10T04:15:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:14:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17730.323584 secs 2023-09-10T04:15:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17730.979488 secs 2023-09-10T04:15:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:00 smithi170 ceph-mon[102832]: pgmap v9617: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 213 KiB/s wr, 11 op/s 2023-09-10T04:15:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:15:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17730.979488 secs 2023-09-10T04:15:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:00 smithi082 ceph-mon[98694]: pgmap v9617: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 213 KiB/s wr, 11 op/s 2023-09-10T04:15:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:15:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17730.979488 secs 2023-09-10T04:15:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:00 smithi098 ceph-mon[103124]: pgmap v9617: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 213 KiB/s wr, 11 op/s 2023-09-10T04:15:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:15:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:15:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:15:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:15:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:15:01.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:15:01.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:15:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:15:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:15:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:15:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:02 smithi098 ceph-mon[103124]: pgmap v9618: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 346 KiB/s wr, 31 op/s 2023-09-10T04:15:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17734.148996 secs 2023-09-10T04:15:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:02 smithi170 ceph-mon[102832]: pgmap v9618: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 346 KiB/s wr, 31 op/s 2023-09-10T04:15:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17734.148996 secs 2023-09-10T04:15:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:02 smithi082 ceph-mon[98694]: pgmap v9618: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 346 KiB/s wr, 31 op/s 2023-09-10T04:15:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17734.148996 secs 2023-09-10T04:15:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17735.323641 secs 2023-09-10T04:15:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:04 smithi098 ceph-mon[103124]: pgmap v9619: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 191 KiB/s wr, 24 op/s 2023-09-10T04:15:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17735.323641 secs 2023-09-10T04:15:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:04 smithi170 ceph-mon[102832]: pgmap v9619: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 191 KiB/s wr, 24 op/s 2023-09-10T04:15:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17735.323641 secs 2023-09-10T04:15:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:04 smithi082 ceph-mon[98694]: pgmap v9619: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 191 KiB/s wr, 24 op/s 2023-09-10T04:15:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17735.979552 secs 2023-09-10T04:15:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17735.979552 secs 2023-09-10T04:15:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17735.979552 secs 2023-09-10T04:15:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:06 smithi098 ceph-mon[103124]: pgmap v9620: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 252 KiB/s wr, 30 op/s 2023-09-10T04:15:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:06 smithi170 ceph-mon[102832]: pgmap v9620: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 252 KiB/s wr, 30 op/s 2023-09-10T04:15:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:06 smithi082 ceph-mon[98694]: pgmap v9620: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 252 KiB/s wr, 30 op/s 2023-09-10T04:15:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17739.149057 secs 2023-09-10T04:15:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:08 smithi170 ceph-mon[102832]: osdmap e669: 12 total, 12 up, 12 in 2023-09-10T04:15:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:08 smithi170 ceph-mon[102832]: pgmap v9622: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 233 KiB/s wr, 30 op/s 2023-09-10T04:15:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17739.149057 secs 2023-09-10T04:15:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:08 smithi082 ceph-mon[98694]: osdmap e669: 12 total, 12 up, 12 in 2023-09-10T04:15:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:08 smithi082 ceph-mon[98694]: pgmap v9622: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 233 KiB/s wr, 30 op/s 2023-09-10T04:15:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17739.149057 secs 2023-09-10T04:15:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:08 smithi098 ceph-mon[103124]: osdmap e669: 12 total, 12 up, 12 in 2023-09-10T04:15:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:08 smithi098 ceph-mon[103124]: pgmap v9622: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 233 KiB/s wr, 30 op/s 2023-09-10T04:15:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17740.323699 secs 2023-09-10T04:15:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17740.323699 secs 2023-09-10T04:15:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17740.323699 secs 2023-09-10T04:15:10.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17740.979618 secs 2023-09-10T04:15:10.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:10 smithi170 ceph-mon[102832]: pgmap v9623: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 233 KiB/s wr, 30 op/s 2023-09-10T04:15:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17740.979618 secs 2023-09-10T04:15:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:10 smithi082 ceph-mon[98694]: pgmap v9623: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 233 KiB/s wr, 30 op/s 2023-09-10T04:15:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17740.979618 secs 2023-09-10T04:15:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:10 smithi098 ceph-mon[103124]: pgmap v9623: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 233 KiB/s wr, 30 op/s 2023-09-10T04:15:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:12 smithi170 ceph-mon[102832]: pgmap v9624: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 74 KiB/s wr, 13 op/s 2023-09-10T04:15:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:12 smithi082 ceph-mon[98694]: pgmap v9624: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 74 KiB/s wr, 13 op/s 2023-09-10T04:15:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:12 smithi098 ceph-mon[103124]: pgmap v9624: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 74 KiB/s wr, 13 op/s 2023-09-10T04:15:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17744.149147 secs 2023-09-10T04:15:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:13 smithi170 ceph-mon[102832]: osdmap e670: 12 total, 12 up, 12 in 2023-09-10T04:15:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17744.149147 secs 2023-09-10T04:15:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:13 smithi082 ceph-mon[98694]: osdmap e670: 12 total, 12 up, 12 in 2023-09-10T04:15:14.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17744.149147 secs 2023-09-10T04:15:14.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:13 smithi098 ceph-mon[103124]: osdmap e670: 12 total, 12 up, 12 in 2023-09-10T04:15:14.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17745.323756 secs 2023-09-10T04:15:14.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:14 smithi170 ceph-mon[102832]: pgmap v9626: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:15:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17745.323756 secs 2023-09-10T04:15:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:14 smithi082 ceph-mon[98694]: pgmap v9626: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:15:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17745.323756 secs 2023-09-10T04:15:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:14 smithi098 ceph-mon[103124]: pgmap v9626: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:15:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17745.979680 secs 2023-09-10T04:15:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17745.979680 secs 2023-09-10T04:15:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17745.979680 secs 2023-09-10T04:15:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:16 smithi170 ceph-mon[102832]: pgmap v9627: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 143 KiB/s wr, 15 op/s 2023-09-10T04:15:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:16 smithi082 ceph-mon[98694]: pgmap v9627: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 143 KiB/s wr, 15 op/s 2023-09-10T04:15:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:16 smithi098 ceph-mon[103124]: pgmap v9627: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 143 KiB/s wr, 15 op/s 2023-09-10T04:15:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17749.149262 secs 2023-09-10T04:15:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:18 smithi170 ceph-mon[102832]: pgmap v9628: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T04:15:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17749.149262 secs 2023-09-10T04:15:18.990 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:18 smithi082 ceph-mon[98694]: pgmap v9628: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T04:15:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17749.149262 secs 2023-09-10T04:15:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:18 smithi098 ceph-mon[103124]: pgmap v9628: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T04:15:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17750.323814 secs 2023-09-10T04:15:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17750.323814 secs 2023-09-10T04:15:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17750.323814 secs 2023-09-10T04:15:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17750.979744 secs 2023-09-10T04:15:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:20 smithi170 ceph-mon[102832]: pgmap v9629: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T04:15:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17750.979744 secs 2023-09-10T04:15:20.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:20 smithi082 ceph-mon[98694]: pgmap v9629: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T04:15:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17750.979744 secs 2023-09-10T04:15:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:20 smithi098 ceph-mon[103124]: pgmap v9629: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T04:15:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:22 smithi170 ceph-mon[102832]: pgmap v9630: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T04:15:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:22 smithi082 ceph-mon[98694]: pgmap v9630: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T04:15:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:22 smithi098 ceph-mon[103124]: pgmap v9630: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T04:15:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17754.149367 secs 2023-09-10T04:15:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17754.149367 secs 2023-09-10T04:15:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17754.149367 secs 2023-09-10T04:15:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17755.323872 secs 2023-09-10T04:15:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:24 smithi170 ceph-mon[102832]: pgmap v9631: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T04:15:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17755.323872 secs 2023-09-10T04:15:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:24 smithi082 ceph-mon[98694]: pgmap v9631: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T04:15:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17755.323872 secs 2023-09-10T04:15:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:24 smithi098 ceph-mon[103124]: pgmap v9631: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T04:15:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17755.979806 secs 2023-09-10T04:15:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17755.979806 secs 2023-09-10T04:15:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17755.979806 secs 2023-09-10T04:15:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:26 smithi170 ceph-mon[102832]: pgmap v9632: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 154 KiB/s wr, 14 op/s 2023-09-10T04:15:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:26 smithi082 ceph-mon[98694]: pgmap v9632: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 154 KiB/s wr, 14 op/s 2023-09-10T04:15:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:26 smithi098 ceph-mon[103124]: pgmap v9632: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 154 KiB/s wr, 14 op/s 2023-09-10T04:15:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:28 smithi170 ceph-mon[102832]: pgmap v9633: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T04:15:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:28 smithi082 ceph-mon[98694]: pgmap v9633: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T04:15:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:28 smithi098 ceph-mon[103124]: pgmap v9633: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T04:15:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17760.323926 secs 2023-09-10T04:15:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17760.323926 secs 2023-09-10T04:15:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17760.323926 secs 2023-09-10T04:15:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17760.979878 secs 2023-09-10T04:15:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:30 smithi170 ceph-mon[102832]: pgmap v9634: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T04:15:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17760.979878 secs 2023-09-10T04:15:30.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:30 smithi082 ceph-mon[98694]: pgmap v9634: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T04:15:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17760.979878 secs 2023-09-10T04:15:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:30 smithi098 ceph-mon[103124]: pgmap v9634: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T04:15:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:15:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:15:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:15:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:32 smithi170 ceph-mon[102832]: pgmap v9635: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 235 KiB/s wr, 15 op/s 2023-09-10T04:15:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:15:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:15:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:15:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:32 smithi082 ceph-mon[98694]: pgmap v9635: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 235 KiB/s wr, 15 op/s 2023-09-10T04:15:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:15:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:15:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:15:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:32 smithi098 ceph-mon[103124]: pgmap v9635: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 235 KiB/s wr, 15 op/s 2023-09-10T04:15:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:15:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:15:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:15:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17764.149504 secs 2023-09-10T04:15:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17764.149504 secs 2023-09-10T04:15:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17764.149504 secs 2023-09-10T04:15:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17765.323997 secs 2023-09-10T04:15:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:34 smithi170 ceph-mon[102832]: pgmap v9636: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 235 KiB/s wr, 10 op/s 2023-09-10T04:15:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17765.323997 secs 2023-09-10T04:15:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:34 smithi082 ceph-mon[98694]: pgmap v9636: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 235 KiB/s wr, 10 op/s 2023-09-10T04:15:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17765.323997 secs 2023-09-10T04:15:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:34 smithi098 ceph-mon[103124]: pgmap v9636: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 235 KiB/s wr, 10 op/s 2023-09-10T04:15:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17765.979941 secs 2023-09-10T04:15:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17765.979941 secs 2023-09-10T04:15:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17765.979941 secs 2023-09-10T04:15:36.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:36 smithi170 ceph-mon[102832]: pgmap v9637: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 344 KiB/s wr, 17 op/s 2023-09-10T04:15:36.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:15:36.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:15:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:36 smithi082 ceph-mon[98694]: pgmap v9637: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 344 KiB/s wr, 17 op/s 2023-09-10T04:15:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:15:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:15:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:36 smithi098 ceph-mon[103124]: pgmap v9637: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 344 KiB/s wr, 17 op/s 2023-09-10T04:15:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:15:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:15:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:38 smithi170 ceph-mon[102832]: pgmap v9638: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T04:15:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:38 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:15:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:38 smithi082 ceph-mon[98694]: pgmap v9638: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T04:15:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:38 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:15:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:38 smithi098 ceph-mon[103124]: pgmap v9638: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T04:15:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:38 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:15:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17770.324058 secs 2023-09-10T04:15:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:15:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17770.324058 secs 2023-09-10T04:15:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:15:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17770.324058 secs 2023-09-10T04:15:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:15:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17770.980006 secs 2023-09-10T04:15:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:40 smithi170 ceph-mon[102832]: pgmap v9639: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T04:15:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:15:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:15:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17770.980006 secs 2023-09-10T04:15:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:40 smithi082 ceph-mon[98694]: pgmap v9639: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T04:15:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:15:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:15:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17770.980006 secs 2023-09-10T04:15:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:40 smithi098 ceph-mon[103124]: pgmap v9639: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T04:15:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:15:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:15:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:42 smithi170 ceph-mon[102832]: pgmap v9640: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 438 KiB/s wr, 19 op/s 2023-09-10T04:15:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:42 smithi082 ceph-mon[98694]: pgmap v9640: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 438 KiB/s wr, 19 op/s 2023-09-10T04:15:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:42 smithi098 ceph-mon[103124]: pgmap v9640: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 438 KiB/s wr, 19 op/s 2023-09-10T04:15:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:43 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:15:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:43 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:15:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:43 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:15:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17775.324133 secs 2023-09-10T04:15:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:44 smithi170 ceph-mon[102832]: pgmap v9641: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 262 KiB/s wr, 13 op/s 2023-09-10T04:15:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17775.324133 secs 2023-09-10T04:15:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:44 smithi082 ceph-mon[98694]: pgmap v9641: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 262 KiB/s wr, 13 op/s 2023-09-10T04:15:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17775.324133 secs 2023-09-10T04:15:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:44 smithi098 ceph-mon[103124]: pgmap v9641: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 262 KiB/s wr, 13 op/s 2023-09-10T04:15:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17775.980072 secs 2023-09-10T04:15:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17775.980072 secs 2023-09-10T04:15:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17775.980072 secs 2023-09-10T04:15:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:46 smithi170 ceph-mon[102832]: pgmap v9642: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 377 KiB/s wr, 18 op/s 2023-09-10T04:15:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:46 smithi082 ceph-mon[98694]: pgmap v9642: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 377 KiB/s wr, 18 op/s 2023-09-10T04:15:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:46 smithi098 ceph-mon[103124]: pgmap v9642: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 377 KiB/s wr, 18 op/s 2023-09-10T04:15:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:48 smithi170 ceph-mon[102832]: pgmap v9643: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T04:15:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:48 smithi082 ceph-mon[98694]: pgmap v9643: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T04:15:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:48 smithi098 ceph-mon[103124]: pgmap v9643: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T04:15:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17780.324212 secs 2023-09-10T04:15:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17780.980137 secs 2023-09-10T04:15:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17780.324212 secs 2023-09-10T04:15:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17780.980137 secs 2023-09-10T04:15:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17780.324212 secs 2023-09-10T04:15:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17780.980137 secs 2023-09-10T04:15:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:50 smithi170 ceph-mon[102832]: pgmap v9644: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T04:15:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:50 smithi082 ceph-mon[98694]: pgmap v9644: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T04:15:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:50 smithi098 ceph-mon[103124]: pgmap v9644: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T04:15:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:52 smithi170 ceph-mon[102832]: pgmap v9645: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 424 KiB/s wr, 18 op/s 2023-09-10T04:15:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:52 smithi082 ceph-mon[98694]: pgmap v9645: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 424 KiB/s wr, 18 op/s 2023-09-10T04:15:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:52 smithi098 ceph-mon[103124]: pgmap v9645: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 424 KiB/s wr, 18 op/s 2023-09-10T04:15:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17784.149837 secs 2023-09-10T04:15:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17784.149837 secs 2023-09-10T04:15:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17784.149837 secs 2023-09-10T04:15:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17785.324296 secs 2023-09-10T04:15:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:54 smithi170 ceph-mon[102832]: pgmap v9646: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 271 KiB/s wr, 11 op/s 2023-09-10T04:15:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17785.980207 secs 2023-09-10T04:15:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17785.324296 secs 2023-09-10T04:15:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:54 smithi082 ceph-mon[98694]: pgmap v9646: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 271 KiB/s wr, 11 op/s 2023-09-10T04:15:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17785.980207 secs 2023-09-10T04:15:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17785.324296 secs 2023-09-10T04:15:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:54 smithi098 ceph-mon[103124]: pgmap v9646: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 271 KiB/s wr, 11 op/s 2023-09-10T04:15:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17785.980207 secs 2023-09-10T04:15:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:15:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:15:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:15:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:15:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:15:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:15:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:15:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:15:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:15:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:56 smithi098 ceph-mon[103124]: pgmap v9647: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 329 KiB/s wr, 17 op/s 2023-09-10T04:15:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:56 smithi170 ceph-mon[102832]: pgmap v9647: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 329 KiB/s wr, 17 op/s 2023-09-10T04:15:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:56 smithi082 ceph-mon[98694]: pgmap v9647: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 329 KiB/s wr, 17 op/s 2023-09-10T04:15:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17789.149897 secs 2023-09-10T04:15:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:58 smithi098 ceph-mon[103124]: pgmap v9648: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 214 KiB/s wr, 12 op/s 2023-09-10T04:15:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17789.149897 secs 2023-09-10T04:15:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:58 smithi170 ceph-mon[102832]: pgmap v9648: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 214 KiB/s wr, 12 op/s 2023-09-10T04:15:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17789.149897 secs 2023-09-10T04:15:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:58 smithi082 ceph-mon[98694]: pgmap v9648: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 214 KiB/s wr, 12 op/s 2023-09-10T04:16:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17790.324383 secs 2023-09-10T04:16:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:15:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17790.980273 secs 2023-09-10T04:16:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17790.324383 secs 2023-09-10T04:16:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:15:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17790.980273 secs 2023-09-10T04:16:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17790.324383 secs 2023-09-10T04:16:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:15:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17790.980273 secs 2023-09-10T04:16:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:00 smithi098 ceph-mon[103124]: pgmap v9649: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 214 KiB/s wr, 12 op/s 2023-09-10T04:16:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:00 smithi170 ceph-mon[102832]: pgmap v9649: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 214 KiB/s wr, 12 op/s 2023-09-10T04:16:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:00 smithi082 ceph-mon[98694]: pgmap v9649: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 214 KiB/s wr, 12 op/s 2023-09-10T04:16:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:16:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:16:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:16:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:16:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:16:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:16:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:16:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:16:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:16:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:16:02.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:16:02.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:16:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:03 smithi082 ceph-mon[98694]: pgmap v9650: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 330 KiB/s wr, 29 op/s 2023-09-10T04:16:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17794.149956 secs 2023-09-10T04:16:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:03 smithi098 ceph-mon[103124]: pgmap v9650: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 330 KiB/s wr, 29 op/s 2023-09-10T04:16:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17794.149956 secs 2023-09-10T04:16:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:03 smithi170 ceph-mon[102832]: pgmap v9650: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 330 KiB/s wr, 29 op/s 2023-09-10T04:16:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17794.149956 secs 2023-09-10T04:16:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17795.324444 secs 2023-09-10T04:16:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:04 smithi170 ceph-mon[102832]: pgmap v9651: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 174 KiB/s wr, 23 op/s 2023-09-10T04:16:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17795.324444 secs 2023-09-10T04:16:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:04 smithi082 ceph-mon[98694]: pgmap v9651: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 174 KiB/s wr, 23 op/s 2023-09-10T04:16:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17795.324444 secs 2023-09-10T04:16:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:04 smithi098 ceph-mon[103124]: pgmap v9651: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 174 KiB/s wr, 23 op/s 2023-09-10T04:16:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17795.980337 secs 2023-09-10T04:16:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17795.980337 secs 2023-09-10T04:16:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17795.980337 secs 2023-09-10T04:16:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:06 smithi170 ceph-mon[102832]: pgmap v9652: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 240 KiB/s wr, 30 op/s 2023-09-10T04:16:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:06 smithi082 ceph-mon[98694]: pgmap v9652: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 240 KiB/s wr, 30 op/s 2023-09-10T04:16:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:06 smithi098 ceph-mon[103124]: pgmap v9652: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 240 KiB/s wr, 30 op/s 2023-09-10T04:16:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17799.150016 secs 2023-09-10T04:16:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:08 smithi170 ceph-mon[102832]: osdmap e671: 12 total, 12 up, 12 in 2023-09-10T04:16:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:08 smithi170 ceph-mon[102832]: pgmap v9654: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 218 KiB/s wr, 29 op/s 2023-09-10T04:16:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17799.150016 secs 2023-09-10T04:16:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:08 smithi082 ceph-mon[98694]: osdmap e671: 12 total, 12 up, 12 in 2023-09-10T04:16:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:08 smithi082 ceph-mon[98694]: pgmap v9654: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 218 KiB/s wr, 29 op/s 2023-09-10T04:16:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17799.150016 secs 2023-09-10T04:16:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:08 smithi098 ceph-mon[103124]: osdmap e671: 12 total, 12 up, 12 in 2023-09-10T04:16:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:08 smithi098 ceph-mon[103124]: pgmap v9654: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 218 KiB/s wr, 29 op/s 2023-09-10T04:16:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17800.324492 secs 2023-09-10T04:16:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17800.324492 secs 2023-09-10T04:16:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17800.324492 secs 2023-09-10T04:16:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17800.980404 secs 2023-09-10T04:16:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:10 smithi170 ceph-mon[102832]: pgmap v9655: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 218 KiB/s wr, 29 op/s 2023-09-10T04:16:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17800.980404 secs 2023-09-10T04:16:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:10 smithi082 ceph-mon[98694]: pgmap v9655: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 218 KiB/s wr, 29 op/s 2023-09-10T04:16:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17800.980404 secs 2023-09-10T04:16:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:10 smithi098 ceph-mon[103124]: pgmap v9655: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 218 KiB/s wr, 29 op/s 2023-09-10T04:16:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:12 smithi170 ceph-mon[102832]: pgmap v9656: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 79 KiB/s wr, 14 op/s 2023-09-10T04:16:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:12 smithi082 ceph-mon[98694]: pgmap v9656: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 79 KiB/s wr, 14 op/s 2023-09-10T04:16:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:12 smithi098 ceph-mon[103124]: pgmap v9656: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 79 KiB/s wr, 14 op/s 2023-09-10T04:16:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17804.150100 secs 2023-09-10T04:16:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:13 smithi170 ceph-mon[102832]: osdmap e672: 12 total, 12 up, 12 in 2023-09-10T04:16:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17804.150100 secs 2023-09-10T04:16:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:13 smithi082 ceph-mon[98694]: osdmap e672: 12 total, 12 up, 12 in 2023-09-10T04:16:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17804.150100 secs 2023-09-10T04:16:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:13 smithi098 ceph-mon[103124]: osdmap e672: 12 total, 12 up, 12 in 2023-09-10T04:16:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17805.324554 secs 2023-09-10T04:16:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:14 smithi170 ceph-mon[102832]: pgmap v9658: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:16:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17805.324554 secs 2023-09-10T04:16:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:14 smithi082 ceph-mon[98694]: pgmap v9658: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:16:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17805.324554 secs 2023-09-10T04:16:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:14 smithi098 ceph-mon[103124]: pgmap v9658: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:16:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17805.980470 secs 2023-09-10T04:16:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17805.980470 secs 2023-09-10T04:16:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17805.980470 secs 2023-09-10T04:16:16.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:16 smithi170 ceph-mon[102832]: pgmap v9659: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 15 op/s 2023-09-10T04:16:16.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:16 smithi082 ceph-mon[98694]: pgmap v9659: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 15 op/s 2023-09-10T04:16:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:16 smithi098 ceph-mon[103124]: pgmap v9659: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 15 op/s 2023-09-10T04:16:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17809.150158 secs 2023-09-10T04:16:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:18 smithi170 ceph-mon[102832]: pgmap v9660: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T04:16:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17809.150158 secs 2023-09-10T04:16:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:18 smithi082 ceph-mon[98694]: pgmap v9660: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T04:16:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17809.150158 secs 2023-09-10T04:16:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:18 smithi098 ceph-mon[103124]: pgmap v9660: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T04:16:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17810.324609 secs 2023-09-10T04:16:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17810.324609 secs 2023-09-10T04:16:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17810.324609 secs 2023-09-10T04:16:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17810.980485 secs 2023-09-10T04:16:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:20 smithi170 ceph-mon[102832]: pgmap v9661: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T04:16:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17810.980485 secs 2023-09-10T04:16:20.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:20 smithi082 ceph-mon[98694]: pgmap v9661: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T04:16:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17810.980485 secs 2023-09-10T04:16:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:20 smithi098 ceph-mon[103124]: pgmap v9661: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T04:16:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:22 smithi170 ceph-mon[102832]: pgmap v9662: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T04:16:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:22 smithi082 ceph-mon[98694]: pgmap v9662: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T04:16:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:22 smithi098 ceph-mon[103124]: pgmap v9662: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T04:16:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17814.150273 secs 2023-09-10T04:16:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17814.150273 secs 2023-09-10T04:16:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17814.150273 secs 2023-09-10T04:16:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17815.324671 secs 2023-09-10T04:16:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:24 smithi170 ceph-mon[102832]: pgmap v9663: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T04:16:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17815.324671 secs 2023-09-10T04:16:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:24 smithi082 ceph-mon[98694]: pgmap v9663: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T04:16:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17815.324671 secs 2023-09-10T04:16:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:24 smithi098 ceph-mon[103124]: pgmap v9663: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T04:16:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17815.980546 secs 2023-09-10T04:16:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17815.980546 secs 2023-09-10T04:16:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17815.980546 secs 2023-09-10T04:16:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:26 smithi170 ceph-mon[102832]: pgmap v9664: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 155 KiB/s wr, 15 op/s 2023-09-10T04:16:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:26 smithi082 ceph-mon[98694]: pgmap v9664: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 155 KiB/s wr, 15 op/s 2023-09-10T04:16:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:26 smithi098 ceph-mon[103124]: pgmap v9664: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 155 KiB/s wr, 15 op/s 2023-09-10T04:16:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:28 smithi170 ceph-mon[102832]: pgmap v9665: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:16:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:28 smithi082 ceph-mon[98694]: pgmap v9665: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:16:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:28 smithi098 ceph-mon[103124]: pgmap v9665: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:16:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17820.324752 secs 2023-09-10T04:16:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17820.324752 secs 2023-09-10T04:16:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17820.324752 secs 2023-09-10T04:16:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17820.980613 secs 2023-09-10T04:16:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:30 smithi170 ceph-mon[102832]: pgmap v9666: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:16:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17820.980613 secs 2023-09-10T04:16:30.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:30 smithi082 ceph-mon[98694]: pgmap v9666: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:16:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17820.980613 secs 2023-09-10T04:16:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:30 smithi098 ceph-mon[103124]: pgmap v9666: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:16:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:16:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:16:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:16:32.652 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:32 smithi170 ceph-mon[102832]: pgmap v9667: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 217 KiB/s wr, 15 op/s 2023-09-10T04:16:32.652 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:16:32.652 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:16:32.653 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:16:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:32 smithi082 ceph-mon[98694]: pgmap v9667: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 217 KiB/s wr, 15 op/s 2023-09-10T04:16:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:16:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:16:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:16:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:32 smithi098 ceph-mon[103124]: pgmap v9667: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 217 KiB/s wr, 15 op/s 2023-09-10T04:16:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:16:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:16:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:16:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17825.324810 secs 2023-09-10T04:16:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:34 smithi170 ceph-mon[102832]: pgmap v9668: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 217 KiB/s wr, 10 op/s 2023-09-10T04:16:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17825.324810 secs 2023-09-10T04:16:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:34 smithi082 ceph-mon[98694]: pgmap v9668: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 217 KiB/s wr, 10 op/s 2023-09-10T04:16:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17825.324810 secs 2023-09-10T04:16:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:34 smithi098 ceph-mon[103124]: pgmap v9668: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 217 KiB/s wr, 10 op/s 2023-09-10T04:16:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17825.980678 secs 2023-09-10T04:16:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17825.980678 secs 2023-09-10T04:16:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17825.980678 secs 2023-09-10T04:16:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:36 smithi170 ceph-mon[102832]: pgmap v9669: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 346 KiB/s wr, 16 op/s 2023-09-10T04:16:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:36 smithi082 ceph-mon[98694]: pgmap v9669: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 346 KiB/s wr, 16 op/s 2023-09-10T04:16:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:36 smithi098 ceph-mon[103124]: pgmap v9669: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 346 KiB/s wr, 16 op/s 2023-09-10T04:16:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17829.150587 secs 2023-09-10T04:16:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:38 smithi170 ceph-mon[102832]: pgmap v9670: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 286 KiB/s wr, 11 op/s 2023-09-10T04:16:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17829.150587 secs 2023-09-10T04:16:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:38 smithi082 ceph-mon[98694]: pgmap v9670: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 286 KiB/s wr, 11 op/s 2023-09-10T04:16:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17829.150587 secs 2023-09-10T04:16:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:38 smithi098 ceph-mon[103124]: pgmap v9670: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 286 KiB/s wr, 11 op/s 2023-09-10T04:16:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17830.324896 secs 2023-09-10T04:16:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17830.324896 secs 2023-09-10T04:16:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17830.324896 secs 2023-09-10T04:16:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17830.980741 secs 2023-09-10T04:16:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:40 smithi170 ceph-mon[102832]: pgmap v9671: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 286 KiB/s wr, 11 op/s 2023-09-10T04:16:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:16:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:16:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:16:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17830.980741 secs 2023-09-10T04:16:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:40 smithi082 ceph-mon[98694]: pgmap v9671: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 286 KiB/s wr, 11 op/s 2023-09-10T04:16:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:16:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:16:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:16:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17830.980741 secs 2023-09-10T04:16:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:40 smithi098 ceph-mon[103124]: pgmap v9671: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 286 KiB/s wr, 11 op/s 2023-09-10T04:16:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:16:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:16:41.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:16:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:42 smithi170 ceph-mon[102832]: pgmap v9672: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 440 KiB/s wr, 18 op/s 2023-09-10T04:16:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:42 smithi082 ceph-mon[98694]: pgmap v9672: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 440 KiB/s wr, 18 op/s 2023-09-10T04:16:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:42 smithi098 ceph-mon[103124]: pgmap v9672: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 440 KiB/s wr, 18 op/s 2023-09-10T04:16:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17834.150649 secs 2023-09-10T04:16:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17834.150649 secs 2023-09-10T04:16:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17834.150649 secs 2023-09-10T04:16:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17835.324954 secs 2023-09-10T04:16:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:44 smithi170 ceph-mon[102832]: pgmap v9673: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 12 op/s 2023-09-10T04:16:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17835.324954 secs 2023-09-10T04:16:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:44 smithi082 ceph-mon[98694]: pgmap v9673: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 12 op/s 2023-09-10T04:16:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17835.324954 secs 2023-09-10T04:16:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:44 smithi098 ceph-mon[103124]: pgmap v9673: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 12 op/s 2023-09-10T04:16:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17835.980805 secs 2023-09-10T04:16:45.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17835.980805 secs 2023-09-10T04:16:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17835.980805 secs 2023-09-10T04:16:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:46 smithi170 ceph-mon[102832]: pgmap v9674: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 398 KiB/s wr, 17 op/s 2023-09-10T04:16:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:46 smithi082 ceph-mon[98694]: pgmap v9674: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 398 KiB/s wr, 17 op/s 2023-09-10T04:16:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:46 smithi098 ceph-mon[103124]: pgmap v9674: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 398 KiB/s wr, 17 op/s 2023-09-10T04:16:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17839.150711 secs 2023-09-10T04:16:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:48 smithi170 ceph-mon[102832]: pgmap v9675: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 269 KiB/s wr, 11 op/s 2023-09-10T04:16:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17839.150711 secs 2023-09-10T04:16:48.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:48 smithi082 ceph-mon[98694]: pgmap v9675: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 269 KiB/s wr, 11 op/s 2023-09-10T04:16:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17839.150711 secs 2023-09-10T04:16:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:48 smithi098 ceph-mon[103124]: pgmap v9675: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 269 KiB/s wr, 11 op/s 2023-09-10T04:16:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17840.325034 secs 2023-09-10T04:16:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17840.325034 secs 2023-09-10T04:16:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17840.325034 secs 2023-09-10T04:16:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17840.980867 secs 2023-09-10T04:16:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:50 smithi170 ceph-mon[102832]: pgmap v9676: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 269 KiB/s wr, 11 op/s 2023-09-10T04:16:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17840.980867 secs 2023-09-10T04:16:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:50 smithi082 ceph-mon[98694]: pgmap v9676: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 269 KiB/s wr, 11 op/s 2023-09-10T04:16:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17840.980867 secs 2023-09-10T04:16:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:50 smithi098 ceph-mon[103124]: pgmap v9676: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 269 KiB/s wr, 11 op/s 2023-09-10T04:16:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:52 smithi170 ceph-mon[102832]: pgmap v9677: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 426 KiB/s wr, 17 op/s 2023-09-10T04:16:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:52 smithi082 ceph-mon[98694]: pgmap v9677: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 426 KiB/s wr, 17 op/s 2023-09-10T04:16:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:52 smithi098 ceph-mon[103124]: pgmap v9677: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 426 KiB/s wr, 17 op/s 2023-09-10T04:16:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17844.150795 secs 2023-09-10T04:16:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17844.150795 secs 2023-09-10T04:16:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17844.150795 secs 2023-09-10T04:16:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17845.325090 secs 2023-09-10T04:16:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:54 smithi170 ceph-mon[102832]: pgmap v9678: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T04:16:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17845.325090 secs 2023-09-10T04:16:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:54 smithi082 ceph-mon[98694]: pgmap v9678: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T04:16:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17845.325090 secs 2023-09-10T04:16:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:54 smithi098 ceph-mon[103124]: pgmap v9678: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T04:16:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17845.980934 secs 2023-09-10T04:16:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17845.980934 secs 2023-09-10T04:16:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17845.980934 secs 2023-09-10T04:16:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:56 smithi170 ceph-mon[102832]: pgmap v9679: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 380 KiB/s wr, 16 op/s 2023-09-10T04:16:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:56 smithi082 ceph-mon[98694]: pgmap v9679: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 380 KiB/s wr, 16 op/s 2023-09-10T04:16:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:56 smithi098 ceph-mon[103124]: pgmap v9679: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 380 KiB/s wr, 16 op/s 2023-09-10T04:16:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17849.150855 secs 2023-09-10T04:16:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:58 smithi170 ceph-mon[102832]: pgmap v9680: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T04:16:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17849.150855 secs 2023-09-10T04:16:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:58 smithi082 ceph-mon[98694]: pgmap v9680: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T04:16:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17849.150855 secs 2023-09-10T04:16:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:58 smithi098 ceph-mon[103124]: pgmap v9680: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T04:16:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17850.325147 secs 2023-09-10T04:16:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:16:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17850.981000 secs 2023-09-10T04:16:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17850.325147 secs 2023-09-10T04:16:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:16:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17850.981000 secs 2023-09-10T04:17:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17850.325147 secs 2023-09-10T04:17:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:16:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17850.981000 secs 2023-09-10T04:17:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:00 smithi170 ceph-mon[102832]: pgmap v9681: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T04:17:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:00 smithi082 ceph-mon[98694]: pgmap v9681: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T04:17:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:00 smithi098 ceph-mon[103124]: pgmap v9681: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T04:17:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:17:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:17:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:17:02.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:02 smithi082 ceph-mon[98694]: pgmap v9682: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 359 KiB/s wr, 18 op/s 2023-09-10T04:17:02.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:17:02.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:17:02.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:17:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:02 smithi098 ceph-mon[103124]: pgmap v9682: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 359 KiB/s wr, 18 op/s 2023-09-10T04:17:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:17:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:17:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:17:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:02 smithi170 ceph-mon[102832]: pgmap v9682: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 359 KiB/s wr, 18 op/s 2023-09-10T04:17:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:17:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:17:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:17:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17854.150918 secs 2023-09-10T04:17:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17854.150918 secs 2023-09-10T04:17:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17854.150918 secs 2023-09-10T04:17:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17855.325205 secs 2023-09-10T04:17:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:04 smithi082 ceph-mon[98694]: pgmap v9683: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 202 KiB/s wr, 12 op/s 2023-09-10T04:17:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17855.981063 secs 2023-09-10T04:17:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17855.325205 secs 2023-09-10T04:17:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:04 smithi098 ceph-mon[103124]: pgmap v9683: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 202 KiB/s wr, 12 op/s 2023-09-10T04:17:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17855.981063 secs 2023-09-10T04:17:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17855.325205 secs 2023-09-10T04:17:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:04 smithi170 ceph-mon[102832]: pgmap v9683: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 202 KiB/s wr, 12 op/s 2023-09-10T04:17:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17855.981063 secs 2023-09-10T04:17:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:06 smithi082 ceph-mon[98694]: pgmap v9684: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 303 KiB/s wr, 30 op/s 2023-09-10T04:17:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:06 smithi098 ceph-mon[103124]: pgmap v9684: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 303 KiB/s wr, 30 op/s 2023-09-10T04:17:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:06 smithi170 ceph-mon[102832]: pgmap v9684: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 303 KiB/s wr, 30 op/s 2023-09-10T04:17:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17859.150975 secs 2023-09-10T04:17:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:08 smithi082 ceph-mon[98694]: pgmap v9685: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 196 KiB/s wr, 24 op/s 2023-09-10T04:17:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:08 smithi082 ceph-mon[98694]: osdmap e673: 12 total, 12 up, 12 in 2023-09-10T04:17:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17859.150975 secs 2023-09-10T04:17:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:08 smithi098 ceph-mon[103124]: pgmap v9685: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 196 KiB/s wr, 24 op/s 2023-09-10T04:17:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:08 smithi098 ceph-mon[103124]: osdmap e673: 12 total, 12 up, 12 in 2023-09-10T04:17:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17859.150975 secs 2023-09-10T04:17:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:08 smithi170 ceph-mon[102832]: pgmap v9685: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 196 KiB/s wr, 24 op/s 2023-09-10T04:17:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:08 smithi170 ceph-mon[102832]: osdmap e673: 12 total, 12 up, 12 in 2023-09-10T04:17:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17860.325270 secs 2023-09-10T04:17:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17860.981130 secs 2023-09-10T04:17:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17860.325270 secs 2023-09-10T04:17:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17860.981130 secs 2023-09-10T04:17:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17860.325270 secs 2023-09-10T04:17:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17860.981130 secs 2023-09-10T04:17:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:10 smithi082 ceph-mon[98694]: pgmap v9687: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 235 KiB/s wr, 29 op/s 2023-09-10T04:17:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:10 smithi098 ceph-mon[103124]: pgmap v9687: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 235 KiB/s wr, 29 op/s 2023-09-10T04:17:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:10 smithi170 ceph-mon[102832]: pgmap v9687: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 235 KiB/s wr, 29 op/s 2023-09-10T04:17:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:12 smithi082 ceph-mon[98694]: pgmap v9688: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 122 KiB/s wr, 27 op/s 2023-09-10T04:17:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:12 smithi098 ceph-mon[103124]: pgmap v9688: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 122 KiB/s wr, 27 op/s 2023-09-10T04:17:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:12 smithi170 ceph-mon[102832]: pgmap v9688: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 122 KiB/s wr, 27 op/s 2023-09-10T04:17:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17864.151085 secs 2023-09-10T04:17:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:13 smithi082 ceph-mon[98694]: osdmap e674: 12 total, 12 up, 12 in 2023-09-10T04:17:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17864.151085 secs 2023-09-10T04:17:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:13 smithi098 ceph-mon[103124]: osdmap e674: 12 total, 12 up, 12 in 2023-09-10T04:17:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17864.151085 secs 2023-09-10T04:17:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:13 smithi170 ceph-mon[102832]: osdmap e674: 12 total, 12 up, 12 in 2023-09-10T04:17:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17865.325327 secs 2023-09-10T04:17:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:14 smithi098 ceph-mon[103124]: pgmap v9690: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 895 B/s wr, 7 op/s 2023-09-10T04:17:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17865.981193 secs 2023-09-10T04:17:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17865.325327 secs 2023-09-10T04:17:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:14 smithi170 ceph-mon[102832]: pgmap v9690: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 895 B/s wr, 7 op/s 2023-09-10T04:17:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17865.981193 secs 2023-09-10T04:17:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17865.325327 secs 2023-09-10T04:17:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:14 smithi082 ceph-mon[98694]: pgmap v9690: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 895 B/s wr, 7 op/s 2023-09-10T04:17:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17865.981193 secs 2023-09-10T04:17:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:16 smithi098 ceph-mon[103124]: pgmap v9691: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 145 KiB/s wr, 15 op/s 2023-09-10T04:17:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:16 smithi170 ceph-mon[102832]: pgmap v9691: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 145 KiB/s wr, 15 op/s 2023-09-10T04:17:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:16 smithi082 ceph-mon[98694]: pgmap v9691: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 145 KiB/s wr, 15 op/s 2023-09-10T04:17:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17869.151212 secs 2023-09-10T04:17:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:18 smithi098 ceph-mon[103124]: pgmap v9692: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T04:17:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17869.151212 secs 2023-09-10T04:17:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:18 smithi170 ceph-mon[102832]: pgmap v9692: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T04:17:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17869.151212 secs 2023-09-10T04:17:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:18 smithi082 ceph-mon[98694]: pgmap v9692: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T04:17:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17870.325383 secs 2023-09-10T04:17:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17870.981261 secs 2023-09-10T04:17:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17870.325383 secs 2023-09-10T04:17:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17870.981261 secs 2023-09-10T04:17:20.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17870.325383 secs 2023-09-10T04:17:20.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17870.981261 secs 2023-09-10T04:17:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:20 smithi098 ceph-mon[103124]: pgmap v9693: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T04:17:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:20 smithi170 ceph-mon[102832]: pgmap v9693: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T04:17:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:20 smithi082 ceph-mon[98694]: pgmap v9693: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T04:17:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:22 smithi098 ceph-mon[103124]: pgmap v9694: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T04:17:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:22 smithi170 ceph-mon[102832]: pgmap v9694: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T04:17:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:22 smithi082 ceph-mon[98694]: pgmap v9694: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T04:17:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17874.151325 secs 2023-09-10T04:17:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17874.151325 secs 2023-09-10T04:17:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17874.151325 secs 2023-09-10T04:17:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17875.325443 secs 2023-09-10T04:17:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:24 smithi098 ceph-mon[103124]: pgmap v9695: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T04:17:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17875.981323 secs 2023-09-10T04:17:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17875.325443 secs 2023-09-10T04:17:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:24 smithi170 ceph-mon[102832]: pgmap v9695: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T04:17:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17875.981323 secs 2023-09-10T04:17:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17875.325443 secs 2023-09-10T04:17:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:24 smithi082 ceph-mon[98694]: pgmap v9695: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T04:17:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17875.981323 secs 2023-09-10T04:17:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:26 smithi098 ceph-mon[103124]: pgmap v9696: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 156 KiB/s wr, 16 op/s 2023-09-10T04:17:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:26 smithi170 ceph-mon[102832]: pgmap v9696: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 156 KiB/s wr, 16 op/s 2023-09-10T04:17:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:26 smithi082 ceph-mon[98694]: pgmap v9696: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 156 KiB/s wr, 16 op/s 2023-09-10T04:17:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:28 smithi098 ceph-mon[103124]: pgmap v9697: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:17:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:28 smithi170 ceph-mon[102832]: pgmap v9697: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:17:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:28 smithi082 ceph-mon[98694]: pgmap v9697: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:17:30.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17880.325502 secs 2023-09-10T04:17:30.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17880.981391 secs 2023-09-10T04:17:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17880.325502 secs 2023-09-10T04:17:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17880.981391 secs 2023-09-10T04:17:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17880.325502 secs 2023-09-10T04:17:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17880.981391 secs 2023-09-10T04:17:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:30 smithi098 ceph-mon[103124]: pgmap v9698: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:17:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:30 smithi170 ceph-mon[102832]: pgmap v9698: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:17:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:30 smithi082 ceph-mon[98694]: pgmap v9698: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:17:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:17:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:17:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:17:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:32 smithi098 ceph-mon[103124]: pgmap v9699: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 226 KiB/s wr, 16 op/s 2023-09-10T04:17:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:17:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:17:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:17:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:32 smithi170 ceph-mon[102832]: pgmap v9699: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 226 KiB/s wr, 16 op/s 2023-09-10T04:17:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:17:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:17:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:17:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:32 smithi082 ceph-mon[98694]: pgmap v9699: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 226 KiB/s wr, 16 op/s 2023-09-10T04:17:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:17:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:17:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:17:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17885.325571 secs 2023-09-10T04:17:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:34 smithi170 ceph-mon[102832]: pgmap v9700: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 226 KiB/s wr, 11 op/s 2023-09-10T04:17:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17885.325571 secs 2023-09-10T04:17:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:34 smithi082 ceph-mon[98694]: pgmap v9700: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 226 KiB/s wr, 11 op/s 2023-09-10T04:17:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17885.325571 secs 2023-09-10T04:17:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:34 smithi098 ceph-mon[103124]: pgmap v9700: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 226 KiB/s wr, 11 op/s 2023-09-10T04:17:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17885.981453 secs 2023-09-10T04:17:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17885.981453 secs 2023-09-10T04:17:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17885.981453 secs 2023-09-10T04:17:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:36 smithi170 ceph-mon[102832]: pgmap v9701: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 355 KiB/s wr, 16 op/s 2023-09-10T04:17:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:36 smithi082 ceph-mon[98694]: pgmap v9701: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 355 KiB/s wr, 16 op/s 2023-09-10T04:17:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:36 smithi098 ceph-mon[103124]: pgmap v9701: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 355 KiB/s wr, 16 op/s 2023-09-10T04:17:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:38 smithi170 ceph-mon[102832]: pgmap v9702: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T04:17:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:38 smithi082 ceph-mon[98694]: pgmap v9702: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T04:17:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:38 smithi098 ceph-mon[103124]: pgmap v9702: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T04:17:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17890.325620 secs 2023-09-10T04:17:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17890.325620 secs 2023-09-10T04:17:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17890.325620 secs 2023-09-10T04:17:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17890.981487 secs 2023-09-10T04:17:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:40 smithi170 ceph-mon[102832]: pgmap v9703: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T04:17:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17890.981487 secs 2023-09-10T04:17:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:40 smithi082 ceph-mon[98694]: pgmap v9703: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T04:17:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17890.981487 secs 2023-09-10T04:17:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:40 smithi098 ceph-mon[103124]: pgmap v9703: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T04:17:42.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:42 smithi170 ceph-mon[102832]: pgmap v9704: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 450 KiB/s wr, 18 op/s 2023-09-10T04:17:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:42 smithi082 ceph-mon[98694]: pgmap v9704: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 450 KiB/s wr, 18 op/s 2023-09-10T04:17:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:42 smithi098 ceph-mon[103124]: pgmap v9704: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 450 KiB/s wr, 18 op/s 2023-09-10T04:17:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17894.151679 secs 2023-09-10T04:17:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17894.151679 secs 2023-09-10T04:17:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17894.151679 secs 2023-09-10T04:17:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17895.325679 secs 2023-09-10T04:17:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:44 smithi170 ceph-mon[102832]: pgmap v9705: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-09-10T04:17:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:17:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:17:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:17:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17895.325679 secs 2023-09-10T04:17:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:44 smithi082 ceph-mon[98694]: pgmap v9705: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-09-10T04:17:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:17:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:17:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:17:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17895.325679 secs 2023-09-10T04:17:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:44 smithi098 ceph-mon[103124]: pgmap v9705: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-09-10T04:17:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:17:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:17:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:17:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17895.981550 secs 2023-09-10T04:17:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17895.981550 secs 2023-09-10T04:17:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17895.981550 secs 2023-09-10T04:17:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:46 smithi170 ceph-mon[102832]: pgmap v9706: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 400 KiB/s wr, 18 op/s 2023-09-10T04:17:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:46 smithi082 ceph-mon[98694]: pgmap v9706: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 400 KiB/s wr, 18 op/s 2023-09-10T04:17:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:46 smithi098 ceph-mon[103124]: pgmap v9706: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 400 KiB/s wr, 18 op/s 2023-09-10T04:17:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17899.151798 secs 2023-09-10T04:17:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:48 smithi170 ceph-mon[102832]: pgmap v9707: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T04:17:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17899.151798 secs 2023-09-10T04:17:48.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:48 smithi082 ceph-mon[98694]: pgmap v9707: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T04:17:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17899.151798 secs 2023-09-10T04:17:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:48 smithi098 ceph-mon[103124]: pgmap v9707: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T04:17:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17900.325738 secs 2023-09-10T04:17:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17900.325738 secs 2023-09-10T04:17:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17900.325738 secs 2023-09-10T04:17:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17900.981617 secs 2023-09-10T04:17:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:50 smithi170 ceph-mon[102832]: pgmap v9708: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T04:17:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17900.981617 secs 2023-09-10T04:17:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:50 smithi082 ceph-mon[98694]: pgmap v9708: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T04:17:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17900.981617 secs 2023-09-10T04:17:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:50 smithi098 ceph-mon[103124]: pgmap v9708: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T04:17:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:52 smithi170 ceph-mon[102832]: pgmap v9709: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 436 KiB/s wr, 18 op/s 2023-09-10T04:17:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:52 smithi082 ceph-mon[98694]: pgmap v9709: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 436 KiB/s wr, 18 op/s 2023-09-10T04:17:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:52 smithi098 ceph-mon[103124]: pgmap v9709: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 436 KiB/s wr, 18 op/s 2023-09-10T04:17:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17904.151858 secs 2023-09-10T04:17:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17904.151858 secs 2023-09-10T04:17:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17904.151858 secs 2023-09-10T04:17:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17905.325795 secs 2023-09-10T04:17:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:54 smithi170 ceph-mon[102832]: pgmap v9710: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 282 KiB/s wr, 12 op/s 2023-09-10T04:17:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17905.981683 secs 2023-09-10T04:17:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17905.325795 secs 2023-09-10T04:17:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:54 smithi082 ceph-mon[98694]: pgmap v9710: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 282 KiB/s wr, 12 op/s 2023-09-10T04:17:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17905.981683 secs 2023-09-10T04:17:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17905.325795 secs 2023-09-10T04:17:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:54 smithi098 ceph-mon[103124]: pgmap v9710: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 282 KiB/s wr, 12 op/s 2023-09-10T04:17:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17905.981683 secs 2023-09-10T04:17:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:56 smithi170 ceph-mon[102832]: pgmap v9711: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 341 KiB/s wr, 18 op/s 2023-09-10T04:17:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:56 smithi082 ceph-mon[98694]: pgmap v9711: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 341 KiB/s wr, 18 op/s 2023-09-10T04:17:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:56 smithi098 ceph-mon[103124]: pgmap v9711: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 341 KiB/s wr, 18 op/s 2023-09-10T04:17:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17909.151919 secs 2023-09-10T04:17:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:58 smithi170 ceph-mon[102832]: pgmap v9712: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 224 KiB/s wr, 12 op/s 2023-09-10T04:17:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17909.151919 secs 2023-09-10T04:17:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:58 smithi082 ceph-mon[98694]: pgmap v9712: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 224 KiB/s wr, 12 op/s 2023-09-10T04:17:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17909.151919 secs 2023-09-10T04:17:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:58 smithi098 ceph-mon[103124]: pgmap v9712: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 224 KiB/s wr, 12 op/s 2023-09-10T04:17:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17910.325876 secs 2023-09-10T04:17:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:17:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17910.981750 secs 2023-09-10T04:17:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17910.325876 secs 2023-09-10T04:17:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:17:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17910.981750 secs 2023-09-10T04:18:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17910.325876 secs 2023-09-10T04:18:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:17:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17910.981750 secs 2023-09-10T04:18:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:00 smithi170 ceph-mon[102832]: pgmap v9713: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 224 KiB/s wr, 12 op/s 2023-09-10T04:18:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:00 smithi082 ceph-mon[98694]: pgmap v9713: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 224 KiB/s wr, 12 op/s 2023-09-10T04:18:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:00 smithi098 ceph-mon[103124]: pgmap v9713: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 224 KiB/s wr, 12 op/s 2023-09-10T04:18:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:02 smithi170 ceph-mon[102832]: pgmap v9714: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 319 KiB/s wr, 17 op/s 2023-09-10T04:18:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:18:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:02 smithi082 ceph-mon[98694]: pgmap v9714: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 319 KiB/s wr, 17 op/s 2023-09-10T04:18:02.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:18:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:02 smithi098 ceph-mon[103124]: pgmap v9714: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 319 KiB/s wr, 17 op/s 2023-09-10T04:18:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:18:03.870 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17914.151979 secs 2023-09-10T04:18:03.964 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17914.151979 secs 2023-09-10T04:18:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17914.151979 secs 2023-09-10T04:18:04.844 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17915.325938 secs 2023-09-10T04:18:04.845 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:04 smithi098 ceph-mon[103124]: pgmap v9715: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T04:18:04.845 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17915.981812 secs 2023-09-10T04:18:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17915.325938 secs 2023-09-10T04:18:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:04 smithi170 ceph-mon[102832]: pgmap v9715: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T04:18:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17915.981812 secs 2023-09-10T04:18:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17915.325938 secs 2023-09-10T04:18:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:04 smithi082 ceph-mon[98694]: pgmap v9715: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T04:18:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17915.981812 secs 2023-09-10T04:18:07.079 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:06 smithi098 ceph-mon[103124]: pgmap v9716: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 242 KiB/s wr, 31 op/s 2023-09-10T04:18:07.079 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:18:07.080 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:18:07.080 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:18:07.080 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:18:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:06 smithi170 ceph-mon[102832]: pgmap v9716: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 242 KiB/s wr, 31 op/s 2023-09-10T04:18:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:18:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:18:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:18:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:18:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:06 smithi082 ceph-mon[98694]: pgmap v9716: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 242 KiB/s wr, 31 op/s 2023-09-10T04:18:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:18:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:18:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:18:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:18:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17919.152042 secs 2023-09-10T04:18:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:18:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:18:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:18:08.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:18:08.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:18:08.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:08 smithi170 ceph-mon[102832]: pgmap v9717: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 183 KiB/s wr, 25 op/s 2023-09-10T04:18:08.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:08 smithi170 ceph-mon[102832]: osdmap e675: 12 total, 12 up, 12 in 2023-09-10T04:18:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17919.152042 secs 2023-09-10T04:18:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:18:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:18:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:18:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:18:08.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:18:08.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:08 smithi082 ceph-mon[98694]: pgmap v9717: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 183 KiB/s wr, 25 op/s 2023-09-10T04:18:08.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:08 smithi082 ceph-mon[98694]: osdmap e675: 12 total, 12 up, 12 in 2023-09-10T04:18:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17919.152042 secs 2023-09-10T04:18:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:18:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:18:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:18:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:18:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:18:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:08 smithi098 ceph-mon[103124]: pgmap v9717: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 183 KiB/s wr, 25 op/s 2023-09-10T04:18:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:08 smithi098 ceph-mon[103124]: osdmap e675: 12 total, 12 up, 12 in 2023-09-10T04:18:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17920.325997 secs 2023-09-10T04:18:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17920.325997 secs 2023-09-10T04:18:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17920.325997 secs 2023-09-10T04:18:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17920.981875 secs 2023-09-10T04:18:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:10 smithi170 ceph-mon[102832]: pgmap v9719: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 220 KiB/s wr, 30 op/s 2023-09-10T04:18:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17920.981875 secs 2023-09-10T04:18:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:10 smithi082 ceph-mon[98694]: pgmap v9719: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 220 KiB/s wr, 30 op/s 2023-09-10T04:18:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17920.981875 secs 2023-09-10T04:18:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:10 smithi098 ceph-mon[103124]: pgmap v9719: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 220 KiB/s wr, 30 op/s 2023-09-10T04:18:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:12 smithi170 ceph-mon[102832]: pgmap v9720: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 106 KiB/s wr, 29 op/s 2023-09-10T04:18:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:12 smithi082 ceph-mon[98694]: pgmap v9720: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 106 KiB/s wr, 29 op/s 2023-09-10T04:18:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:12 smithi098 ceph-mon[103124]: pgmap v9720: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 106 KiB/s wr, 29 op/s 2023-09-10T04:18:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17924.152101 secs 2023-09-10T04:18:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:13 smithi170 ceph-mon[102832]: osdmap e676: 12 total, 12 up, 12 in 2023-09-10T04:18:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17924.152101 secs 2023-09-10T04:18:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:13 smithi082 ceph-mon[98694]: osdmap e676: 12 total, 12 up, 12 in 2023-09-10T04:18:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17924.152101 secs 2023-09-10T04:18:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:13 smithi098 ceph-mon[103124]: osdmap e676: 12 total, 12 up, 12 in 2023-09-10T04:18:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17925.326078 secs 2023-09-10T04:18:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:14 smithi170 ceph-mon[102832]: pgmap v9722: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:18:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17925.326078 secs 2023-09-10T04:18:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:14 smithi082 ceph-mon[98694]: pgmap v9722: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:18:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17925.326078 secs 2023-09-10T04:18:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:14 smithi098 ceph-mon[103124]: pgmap v9722: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:18:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17925.981936 secs 2023-09-10T04:18:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17925.981936 secs 2023-09-10T04:18:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17925.981936 secs 2023-09-10T04:18:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:16 smithi170 ceph-mon[102832]: pgmap v9723: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 15 op/s 2023-09-10T04:18:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:16 smithi082 ceph-mon[98694]: pgmap v9723: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 15 op/s 2023-09-10T04:18:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:16 smithi098 ceph-mon[103124]: pgmap v9723: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 15 op/s 2023-09-10T04:18:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17929.152189 secs 2023-09-10T04:18:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:18 smithi170 ceph-mon[102832]: pgmap v9724: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T04:18:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17929.152189 secs 2023-09-10T04:18:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:18 smithi082 ceph-mon[98694]: pgmap v9724: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T04:18:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17929.152189 secs 2023-09-10T04:18:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:18 smithi098 ceph-mon[103124]: pgmap v9724: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T04:18:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17930.326134 secs 2023-09-10T04:18:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17930.326134 secs 2023-09-10T04:18:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17930.326134 secs 2023-09-10T04:18:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17930.981999 secs 2023-09-10T04:18:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:20 smithi170 ceph-mon[102832]: pgmap v9725: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T04:18:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17930.981999 secs 2023-09-10T04:18:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:20 smithi082 ceph-mon[98694]: pgmap v9725: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T04:18:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17930.981999 secs 2023-09-10T04:18:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:20 smithi098 ceph-mon[103124]: pgmap v9725: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T04:18:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:22 smithi170 ceph-mon[102832]: pgmap v9726: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T04:18:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:22 smithi082 ceph-mon[98694]: pgmap v9726: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T04:18:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:22 smithi098 ceph-mon[103124]: pgmap v9726: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T04:18:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17934.152249 secs 2023-09-10T04:18:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17934.152249 secs 2023-09-10T04:18:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17934.152249 secs 2023-09-10T04:18:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17935.326190 secs 2023-09-10T04:18:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:24 smithi170 ceph-mon[102832]: pgmap v9727: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T04:18:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17935.982061 secs 2023-09-10T04:18:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17935.326190 secs 2023-09-10T04:18:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:24 smithi082 ceph-mon[98694]: pgmap v9727: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T04:18:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17935.982061 secs 2023-09-10T04:18:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17935.326190 secs 2023-09-10T04:18:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:24 smithi098 ceph-mon[103124]: pgmap v9727: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T04:18:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17935.982061 secs 2023-09-10T04:18:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:26 smithi170 ceph-mon[102832]: pgmap v9728: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 15 op/s 2023-09-10T04:18:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:26 smithi082 ceph-mon[98694]: pgmap v9728: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 15 op/s 2023-09-10T04:18:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:26 smithi098 ceph-mon[103124]: pgmap v9728: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 15 op/s 2023-09-10T04:18:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:28 smithi082 ceph-mon[98694]: pgmap v9729: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 9 op/s 2023-09-10T04:18:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:28 smithi098 ceph-mon[103124]: pgmap v9729: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 9 op/s 2023-09-10T04:18:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:28 smithi170 ceph-mon[102832]: pgmap v9729: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 9 op/s 2023-09-10T04:18:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17940.326258 secs 2023-09-10T04:18:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17940.982129 secs 2023-09-10T04:18:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17940.326258 secs 2023-09-10T04:18:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17940.982129 secs 2023-09-10T04:18:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17940.326258 secs 2023-09-10T04:18:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17940.982129 secs 2023-09-10T04:18:30.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:30 smithi082 ceph-mon[98694]: pgmap v9730: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 9 op/s 2023-09-10T04:18:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:30 smithi098 ceph-mon[103124]: pgmap v9730: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 9 op/s 2023-09-10T04:18:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:30 smithi170 ceph-mon[102832]: pgmap v9730: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 9 op/s 2023-09-10T04:18:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:18:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:18:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:18:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:32 smithi082 ceph-mon[98694]: pgmap v9731: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 227 KiB/s wr, 16 op/s 2023-09-10T04:18:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:18:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:18:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:18:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:32 smithi098 ceph-mon[103124]: pgmap v9731: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 227 KiB/s wr, 16 op/s 2023-09-10T04:18:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:18:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:18:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:18:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:32 smithi170 ceph-mon[102832]: pgmap v9731: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 227 KiB/s wr, 16 op/s 2023-09-10T04:18:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:18:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:18:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:18:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17945.326314 secs 2023-09-10T04:18:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:34 smithi170 ceph-mon[102832]: pgmap v9732: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T04:18:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17945.326314 secs 2023-09-10T04:18:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:34 smithi082 ceph-mon[98694]: pgmap v9732: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T04:18:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17945.326314 secs 2023-09-10T04:18:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:34 smithi098 ceph-mon[103124]: pgmap v9732: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T04:18:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17945.982192 secs 2023-09-10T04:18:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17945.982192 secs 2023-09-10T04:18:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17945.982192 secs 2023-09-10T04:18:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:36 smithi170 ceph-mon[102832]: pgmap v9733: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 357 KiB/s wr, 17 op/s 2023-09-10T04:18:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:36 smithi082 ceph-mon[98694]: pgmap v9733: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 357 KiB/s wr, 17 op/s 2023-09-10T04:18:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:36 smithi098 ceph-mon[103124]: pgmap v9733: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 357 KiB/s wr, 17 op/s 2023-09-10T04:18:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17949.152516 secs 2023-09-10T04:18:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:38 smithi170 ceph-mon[102832]: pgmap v9734: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 297 KiB/s wr, 12 op/s 2023-09-10T04:18:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17949.152516 secs 2023-09-10T04:18:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:38 smithi082 ceph-mon[98694]: pgmap v9734: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 297 KiB/s wr, 12 op/s 2023-09-10T04:18:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17949.152516 secs 2023-09-10T04:18:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:38 smithi098 ceph-mon[103124]: pgmap v9734: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 297 KiB/s wr, 12 op/s 2023-09-10T04:18:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17950.326416 secs 2023-09-10T04:18:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17950.326416 secs 2023-09-10T04:18:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17950.326416 secs 2023-09-10T04:18:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17950.982258 secs 2023-09-10T04:18:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:40 smithi170 ceph-mon[102832]: pgmap v9735: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 297 KiB/s wr, 12 op/s 2023-09-10T04:18:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:18:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:18:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:18:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17950.982258 secs 2023-09-10T04:18:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:40 smithi082 ceph-mon[98694]: pgmap v9735: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 297 KiB/s wr, 12 op/s 2023-09-10T04:18:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:18:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:18:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:18:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17950.982258 secs 2023-09-10T04:18:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:40 smithi098 ceph-mon[103124]: pgmap v9735: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 297 KiB/s wr, 12 op/s 2023-09-10T04:18:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:18:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:18:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:18:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:42 smithi170 ceph-mon[102832]: pgmap v9736: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 453 KiB/s wr, 18 op/s 2023-09-10T04:18:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:42 smithi082 ceph-mon[98694]: pgmap v9736: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 453 KiB/s wr, 18 op/s 2023-09-10T04:18:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:42 smithi098 ceph-mon[103124]: pgmap v9736: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 453 KiB/s wr, 18 op/s 2023-09-10T04:18:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17954.152629 secs 2023-09-10T04:18:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17954.152629 secs 2023-09-10T04:18:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17954.152629 secs 2023-09-10T04:18:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17955.326491 secs 2023-09-10T04:18:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:44 smithi170 ceph-mon[102832]: pgmap v9737: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T04:18:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17955.326491 secs 2023-09-10T04:18:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:44 smithi082 ceph-mon[98694]: pgmap v9737: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T04:18:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17955.326491 secs 2023-09-10T04:18:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:44 smithi098 ceph-mon[103124]: pgmap v9737: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T04:18:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17955.982328 secs 2023-09-10T04:18:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17955.982328 secs 2023-09-10T04:18:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17955.982328 secs 2023-09-10T04:18:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:46 smithi170 ceph-mon[102832]: pgmap v9738: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 402 KiB/s wr, 18 op/s 2023-09-10T04:18:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:46 smithi082 ceph-mon[98694]: pgmap v9738: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 402 KiB/s wr, 18 op/s 2023-09-10T04:18:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:46 smithi098 ceph-mon[103124]: pgmap v9738: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 402 KiB/s wr, 18 op/s 2023-09-10T04:18:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17959.152692 secs 2023-09-10T04:18:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:48 smithi170 ceph-mon[102832]: pgmap v9739: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T04:18:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17959.152692 secs 2023-09-10T04:18:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:48 smithi082 ceph-mon[98694]: pgmap v9739: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T04:18:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17959.152692 secs 2023-09-10T04:18:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:48 smithi098 ceph-mon[103124]: pgmap v9739: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T04:18:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17960.326580 secs 2023-09-10T04:18:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17960.982392 secs 2023-09-10T04:18:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17960.326580 secs 2023-09-10T04:18:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17960.982392 secs 2023-09-10T04:18:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17960.326580 secs 2023-09-10T04:18:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17960.982392 secs 2023-09-10T04:18:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:50 smithi170 ceph-mon[102832]: pgmap v9740: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T04:18:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:50 smithi082 ceph-mon[98694]: pgmap v9740: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T04:18:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:50 smithi098 ceph-mon[103124]: pgmap v9740: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T04:18:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:52 smithi082 ceph-mon[98694]: pgmap v9741: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 430 KiB/s wr, 18 op/s 2023-09-10T04:18:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:52 smithi098 ceph-mon[103124]: pgmap v9741: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 430 KiB/s wr, 18 op/s 2023-09-10T04:18:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:52 smithi170 ceph-mon[102832]: pgmap v9741: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 430 KiB/s wr, 18 op/s 2023-09-10T04:18:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17964.152814 secs 2023-09-10T04:18:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17964.152814 secs 2023-09-10T04:18:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17964.152814 secs 2023-09-10T04:18:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17965.326638 secs 2023-09-10T04:18:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:54 smithi082 ceph-mon[98694]: pgmap v9742: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 275 KiB/s wr, 11 op/s 2023-09-10T04:18:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17965.982457 secs 2023-09-10T04:18:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17965.326638 secs 2023-09-10T04:18:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:54 smithi098 ceph-mon[103124]: pgmap v9742: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 275 KiB/s wr, 11 op/s 2023-09-10T04:18:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17965.982457 secs 2023-09-10T04:18:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17965.326638 secs 2023-09-10T04:18:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:54 smithi170 ceph-mon[102832]: pgmap v9742: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 275 KiB/s wr, 11 op/s 2023-09-10T04:18:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17965.982457 secs 2023-09-10T04:18:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:56 smithi082 ceph-mon[98694]: pgmap v9743: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 334 KiB/s wr, 17 op/s 2023-09-10T04:18:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:56 smithi098 ceph-mon[103124]: pgmap v9743: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 334 KiB/s wr, 17 op/s 2023-09-10T04:18:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:56 smithi170 ceph-mon[102832]: pgmap v9743: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 334 KiB/s wr, 17 op/s 2023-09-10T04:18:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17969.152889 secs 2023-09-10T04:18:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:58 smithi082 ceph-mon[98694]: pgmap v9744: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 217 KiB/s wr, 12 op/s 2023-09-10T04:18:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17969.152889 secs 2023-09-10T04:18:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:58 smithi098 ceph-mon[103124]: pgmap v9744: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 217 KiB/s wr, 12 op/s 2023-09-10T04:18:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17969.152889 secs 2023-09-10T04:18:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:58 smithi170 ceph-mon[102832]: pgmap v9744: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 217 KiB/s wr, 12 op/s 2023-09-10T04:18:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17970.326697 secs 2023-09-10T04:18:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:18:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17970.982491 secs 2023-09-10T04:19:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17970.326697 secs 2023-09-10T04:19:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:18:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17970.982491 secs 2023-09-10T04:19:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17970.326697 secs 2023-09-10T04:19:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:18:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17970.982491 secs 2023-09-10T04:19:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:00 smithi082 ceph-mon[98694]: pgmap v9745: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 217 KiB/s wr, 12 op/s 2023-09-10T04:19:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:00 smithi098 ceph-mon[103124]: pgmap v9745: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 217 KiB/s wr, 12 op/s 2023-09-10T04:19:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:00 smithi170 ceph-mon[102832]: pgmap v9745: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 217 KiB/s wr, 12 op/s 2023-09-10T04:19:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:02 smithi082 ceph-mon[98694]: pgmap v9746: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 312 KiB/s wr, 17 op/s 2023-09-10T04:19:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:02 smithi098 ceph-mon[103124]: pgmap v9746: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 312 KiB/s wr, 17 op/s 2023-09-10T04:19:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:02 smithi170 ceph-mon[102832]: pgmap v9746: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 312 KiB/s wr, 17 op/s 2023-09-10T04:19:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17974.152952 secs 2023-09-10T04:19:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17974.152952 secs 2023-09-10T04:19:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17974.152952 secs 2023-09-10T04:19:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17975.326757 secs 2023-09-10T04:19:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:04 smithi082 ceph-mon[98694]: pgmap v9747: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T04:19:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17975.982563 secs 2023-09-10T04:19:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17975.326757 secs 2023-09-10T04:19:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:04 smithi098 ceph-mon[103124]: pgmap v9747: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T04:19:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17975.982563 secs 2023-09-10T04:19:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17975.326757 secs 2023-09-10T04:19:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:04 smithi170 ceph-mon[102832]: pgmap v9747: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T04:19:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17975.982563 secs 2023-09-10T04:19:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:06 smithi082 ceph-mon[98694]: pgmap v9748: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 242 KiB/s wr, 30 op/s 2023-09-10T04:19:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:06 smithi098 ceph-mon[103124]: pgmap v9748: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 242 KiB/s wr, 30 op/s 2023-09-10T04:19:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:06 smithi170 ceph-mon[102832]: pgmap v9748: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 242 KiB/s wr, 30 op/s 2023-09-10T04:19:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17979.153069 secs 2023-09-10T04:19:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:08 smithi098 ceph-mon[103124]: pgmap v9749: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 183 KiB/s wr, 24 op/s 2023-09-10T04:19:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:08 smithi098 ceph-mon[103124]: osdmap e677: 12 total, 12 up, 12 in 2023-09-10T04:19:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:19:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:19:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:19:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:19:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17979.153069 secs 2023-09-10T04:19:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:08 smithi170 ceph-mon[102832]: pgmap v9749: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 183 KiB/s wr, 24 op/s 2023-09-10T04:19:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:08 smithi170 ceph-mon[102832]: osdmap e677: 12 total, 12 up, 12 in 2023-09-10T04:19:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:19:09.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:19:09.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:19:09.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:19:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17979.153069 secs 2023-09-10T04:19:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:08 smithi082 ceph-mon[98694]: pgmap v9749: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 183 KiB/s wr, 24 op/s 2023-09-10T04:19:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:08 smithi082 ceph-mon[98694]: osdmap e677: 12 total, 12 up, 12 in 2023-09-10T04:19:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:19:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:19:09.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:19:09.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:19:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17980.326844 secs 2023-09-10T04:19:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17980.982630 secs 2023-09-10T04:19:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17980.326844 secs 2023-09-10T04:19:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17980.982630 secs 2023-09-10T04:19:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17980.326844 secs 2023-09-10T04:19:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17980.982630 secs 2023-09-10T04:19:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:10 smithi098 ceph-mon[103124]: pgmap v9751: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 220 KiB/s wr, 29 op/s 2023-09-10T04:19:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:10 smithi170 ceph-mon[102832]: pgmap v9751: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 220 KiB/s wr, 29 op/s 2023-09-10T04:19:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:10 smithi082 ceph-mon[98694]: pgmap v9751: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 220 KiB/s wr, 29 op/s 2023-09-10T04:19:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:12 smithi098 ceph-mon[103124]: pgmap v9752: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 106 KiB/s wr, 29 op/s 2023-09-10T04:19:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:12 smithi170 ceph-mon[102832]: pgmap v9752: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 106 KiB/s wr, 29 op/s 2023-09-10T04:19:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:12 smithi082 ceph-mon[98694]: pgmap v9752: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 106 KiB/s wr, 29 op/s 2023-09-10T04:19:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17984.153160 secs 2023-09-10T04:19:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:13 smithi098 ceph-mon[103124]: osdmap e678: 12 total, 12 up, 12 in 2023-09-10T04:19:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17984.153160 secs 2023-09-10T04:19:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:13 smithi170 ceph-mon[102832]: osdmap e678: 12 total, 12 up, 12 in 2023-09-10T04:19:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17984.153160 secs 2023-09-10T04:19:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:13 smithi082 ceph-mon[98694]: osdmap e678: 12 total, 12 up, 12 in 2023-09-10T04:19:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17985.326902 secs 2023-09-10T04:19:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:14 smithi098 ceph-mon[103124]: pgmap v9754: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:19:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17985.982697 secs 2023-09-10T04:19:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17985.326902 secs 2023-09-10T04:19:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:14 smithi170 ceph-mon[102832]: pgmap v9754: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:19:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17985.982697 secs 2023-09-10T04:19:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17985.326902 secs 2023-09-10T04:19:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:14 smithi082 ceph-mon[98694]: pgmap v9754: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:19:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17985.982697 secs 2023-09-10T04:19:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:16 smithi098 ceph-mon[103124]: pgmap v9755: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 146 KiB/s wr, 15 op/s 2023-09-10T04:19:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:16 smithi170 ceph-mon[102832]: pgmap v9755: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 146 KiB/s wr, 15 op/s 2023-09-10T04:19:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:16 smithi082 ceph-mon[98694]: pgmap v9755: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 146 KiB/s wr, 15 op/s 2023-09-10T04:19:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17989.153219 secs 2023-09-10T04:19:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:18 smithi098 ceph-mon[103124]: pgmap v9756: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T04:19:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17989.153219 secs 2023-09-10T04:19:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:18 smithi170 ceph-mon[102832]: pgmap v9756: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T04:19:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17989.153219 secs 2023-09-10T04:19:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:18 smithi082 ceph-mon[98694]: pgmap v9756: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T04:19:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17990.326961 secs 2023-09-10T04:19:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17990.982760 secs 2023-09-10T04:19:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17990.326961 secs 2023-09-10T04:19:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17990.982760 secs 2023-09-10T04:19:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17990.326961 secs 2023-09-10T04:19:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17990.982760 secs 2023-09-10T04:19:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:20 smithi098 ceph-mon[103124]: pgmap v9757: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T04:19:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:20 smithi170 ceph-mon[102832]: pgmap v9757: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T04:19:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:20 smithi082 ceph-mon[98694]: pgmap v9757: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T04:19:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:22 smithi098 ceph-mon[103124]: pgmap v9758: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T04:19:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:22 smithi170 ceph-mon[102832]: pgmap v9758: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T04:19:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:22 smithi082 ceph-mon[98694]: pgmap v9758: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T04:19:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 17994.153280 secs 2023-09-10T04:19:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 17994.153280 secs 2023-09-10T04:19:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 17994.153280 secs 2023-09-10T04:19:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17995.327049 secs 2023-09-10T04:19:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:24 smithi098 ceph-mon[103124]: pgmap v9759: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T04:19:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 17995.982826 secs 2023-09-10T04:19:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17995.327049 secs 2023-09-10T04:19:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:24 smithi170 ceph-mon[102832]: pgmap v9759: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T04:19:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 17995.982826 secs 2023-09-10T04:19:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17995.327049 secs 2023-09-10T04:19:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:24 smithi082 ceph-mon[98694]: pgmap v9759: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T04:19:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 17995.982826 secs 2023-09-10T04:19:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:26 smithi098 ceph-mon[103124]: pgmap v9760: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 157 KiB/s wr, 15 op/s 2023-09-10T04:19:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:26 smithi170 ceph-mon[102832]: pgmap v9760: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 157 KiB/s wr, 15 op/s 2023-09-10T04:19:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:26 smithi082 ceph-mon[98694]: pgmap v9760: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 157 KiB/s wr, 15 op/s 2023-09-10T04:19:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:28 smithi098 ceph-mon[103124]: pgmap v9761: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:19:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:28 smithi170 ceph-mon[102832]: pgmap v9761: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:19:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:28 smithi082 ceph-mon[98694]: pgmap v9761: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:19:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18000.327106 secs 2023-09-10T04:19:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18000.982891 secs 2023-09-10T04:19:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18000.327106 secs 2023-09-10T04:19:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18000.982891 secs 2023-09-10T04:19:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18000.327106 secs 2023-09-10T04:19:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18000.982891 secs 2023-09-10T04:19:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:30 smithi170 ceph-mon[102832]: pgmap v9762: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:19:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:30 smithi082 ceph-mon[98694]: pgmap v9762: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:19:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:30 smithi098 ceph-mon[103124]: pgmap v9762: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:19:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:19:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:19:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:19:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:32 smithi170 ceph-mon[102832]: pgmap v9763: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 220 KiB/s wr, 15 op/s 2023-09-10T04:19:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:19:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:19:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:19:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:32 smithi082 ceph-mon[98694]: pgmap v9763: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 220 KiB/s wr, 15 op/s 2023-09-10T04:19:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:19:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:19:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:19:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:32 smithi098 ceph-mon[103124]: pgmap v9763: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 220 KiB/s wr, 15 op/s 2023-09-10T04:19:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:19:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:19:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:19:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18005.327163 secs 2023-09-10T04:19:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:34 smithi170 ceph-mon[102832]: pgmap v9764: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 220 KiB/s wr, 10 op/s 2023-09-10T04:19:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18005.327163 secs 2023-09-10T04:19:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:34 smithi082 ceph-mon[98694]: pgmap v9764: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 220 KiB/s wr, 10 op/s 2023-09-10T04:19:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18005.327163 secs 2023-09-10T04:19:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:34 smithi098 ceph-mon[103124]: pgmap v9764: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 220 KiB/s wr, 10 op/s 2023-09-10T04:19:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18005.983022 secs 2023-09-10T04:19:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18005.983022 secs 2023-09-10T04:19:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18005.983022 secs 2023-09-10T04:19:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:36 smithi170 ceph-mon[102832]: pgmap v9765: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 355 KiB/s wr, 16 op/s 2023-09-10T04:19:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:36 smithi082 ceph-mon[98694]: pgmap v9765: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 355 KiB/s wr, 16 op/s 2023-09-10T04:19:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:36 smithi098 ceph-mon[103124]: pgmap v9765: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 355 KiB/s wr, 16 op/s 2023-09-10T04:19:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:38 smithi170 ceph-mon[102832]: pgmap v9766: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 295 KiB/s wr, 11 op/s 2023-09-10T04:19:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:38 smithi082 ceph-mon[98694]: pgmap v9766: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 295 KiB/s wr, 11 op/s 2023-09-10T04:19:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:38 smithi098 ceph-mon[103124]: pgmap v9766: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 295 KiB/s wr, 11 op/s 2023-09-10T04:19:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18010.327231 secs 2023-09-10T04:19:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18010.327231 secs 2023-09-10T04:19:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18010.327231 secs 2023-09-10T04:19:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18010.983077 secs 2023-09-10T04:19:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:40 smithi170 ceph-mon[102832]: pgmap v9767: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 295 KiB/s wr, 11 op/s 2023-09-10T04:19:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18010.983077 secs 2023-09-10T04:19:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:40 smithi082 ceph-mon[98694]: pgmap v9767: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 295 KiB/s wr, 11 op/s 2023-09-10T04:19:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18010.983077 secs 2023-09-10T04:19:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:40 smithi098 ceph-mon[103124]: pgmap v9767: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 295 KiB/s wr, 11 op/s 2023-09-10T04:19:42.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:42 smithi170 ceph-mon[102832]: pgmap v9768: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 446 KiB/s wr, 18 op/s 2023-09-10T04:19:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:42 smithi082 ceph-mon[98694]: pgmap v9768: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 446 KiB/s wr, 18 op/s 2023-09-10T04:19:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:42 smithi098 ceph-mon[103124]: pgmap v9768: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 446 KiB/s wr, 18 op/s 2023-09-10T04:19:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18015.327289 secs 2023-09-10T04:19:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:44 smithi170 ceph-mon[102832]: pgmap v9769: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 286 KiB/s wr, 12 op/s 2023-09-10T04:19:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18015.327289 secs 2023-09-10T04:19:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:44 smithi082 ceph-mon[98694]: pgmap v9769: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 286 KiB/s wr, 12 op/s 2023-09-10T04:19:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18015.327289 secs 2023-09-10T04:19:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:44 smithi098 ceph-mon[103124]: pgmap v9769: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 286 KiB/s wr, 12 op/s 2023-09-10T04:19:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18015.983139 secs 2023-09-10T04:19:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18015.983139 secs 2023-09-10T04:19:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18015.983139 secs 2023-09-10T04:19:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:46 smithi170 ceph-mon[102832]: pgmap v9770: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 404 KiB/s wr, 18 op/s 2023-09-10T04:19:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:46 smithi082 ceph-mon[98694]: pgmap v9770: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 404 KiB/s wr, 18 op/s 2023-09-10T04:19:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:46 smithi098 ceph-mon[103124]: pgmap v9770: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 404 KiB/s wr, 18 op/s 2023-09-10T04:19:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18019.153687 secs 2023-09-10T04:19:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:48 smithi170 ceph-mon[102832]: pgmap v9771: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T04:19:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18019.153687 secs 2023-09-10T04:19:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:48 smithi082 ceph-mon[98694]: pgmap v9771: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T04:19:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18019.153687 secs 2023-09-10T04:19:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:48 smithi098 ceph-mon[103124]: pgmap v9771: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T04:19:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18020.327347 secs 2023-09-10T04:19:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18020.327347 secs 2023-09-10T04:19:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18020.327347 secs 2023-09-10T04:19:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18020.983205 secs 2023-09-10T04:19:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:50 smithi170 ceph-mon[102832]: pgmap v9772: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T04:19:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18020.983205 secs 2023-09-10T04:19:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:50 smithi082 ceph-mon[98694]: pgmap v9772: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T04:19:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18020.983205 secs 2023-09-10T04:19:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:50 smithi098 ceph-mon[103124]: pgmap v9772: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T04:19:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:19:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:19:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:19:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:19:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:19:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:19:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:19:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:19:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:19:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:52 smithi098 ceph-mon[103124]: pgmap v9773: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 427 KiB/s wr, 18 op/s 2023-09-10T04:19:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:52 smithi170 ceph-mon[102832]: pgmap v9773: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 427 KiB/s wr, 18 op/s 2023-09-10T04:19:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:52 smithi082 ceph-mon[98694]: pgmap v9773: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 427 KiB/s wr, 18 op/s 2023-09-10T04:19:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18024.153747 secs 2023-09-10T04:19:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18024.153747 secs 2023-09-10T04:19:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18024.153747 secs 2023-09-10T04:19:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18025.327406 secs 2023-09-10T04:19:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:54 smithi098 ceph-mon[103124]: pgmap v9774: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 276 KiB/s wr, 12 op/s 2023-09-10T04:19:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18025.983266 secs 2023-09-10T04:19:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18025.327406 secs 2023-09-10T04:19:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:54 smithi170 ceph-mon[102832]: pgmap v9774: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 276 KiB/s wr, 12 op/s 2023-09-10T04:19:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18025.983266 secs 2023-09-10T04:19:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18025.327406 secs 2023-09-10T04:19:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:54 smithi082 ceph-mon[98694]: pgmap v9774: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 276 KiB/s wr, 12 op/s 2023-09-10T04:19:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18025.983266 secs 2023-09-10T04:19:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:56 smithi098 ceph-mon[103124]: pgmap v9775: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 335 KiB/s wr, 17 op/s 2023-09-10T04:19:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:56 smithi170 ceph-mon[102832]: pgmap v9775: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 335 KiB/s wr, 17 op/s 2023-09-10T04:19:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:56 smithi082 ceph-mon[98694]: pgmap v9775: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 335 KiB/s wr, 17 op/s 2023-09-10T04:19:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18029.153795 secs 2023-09-10T04:19:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:58 smithi098 ceph-mon[103124]: pgmap v9776: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 218 KiB/s wr, 11 op/s 2023-09-10T04:19:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18029.153795 secs 2023-09-10T04:19:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:58 smithi170 ceph-mon[102832]: pgmap v9776: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 218 KiB/s wr, 11 op/s 2023-09-10T04:19:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18029.153795 secs 2023-09-10T04:19:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:58 smithi082 ceph-mon[98694]: pgmap v9776: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 218 KiB/s wr, 11 op/s 2023-09-10T04:20:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18030.327492 secs 2023-09-10T04:20:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:19:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18030.983331 secs 2023-09-10T04:20:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18030.327492 secs 2023-09-10T04:20:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:19:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18030.983331 secs 2023-09-10T04:20:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18030.327492 secs 2023-09-10T04:20:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:19:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18030.983331 secs 2023-09-10T04:20:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:00 smithi098 ceph-mon[103124]: pgmap v9777: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 218 KiB/s wr, 11 op/s 2023-09-10T04:20:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T04:20:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:00 smithi170 ceph-mon[102832]: pgmap v9777: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 218 KiB/s wr, 11 op/s 2023-09-10T04:20:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T04:20:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:00 smithi082 ceph-mon[98694]: pgmap v9777: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 218 KiB/s wr, 11 op/s 2023-09-10T04:20:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T04:20:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:02 smithi098 ceph-mon[103124]: pgmap v9778: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 314 KiB/s wr, 17 op/s 2023-09-10T04:20:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:02 smithi170 ceph-mon[102832]: pgmap v9778: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 314 KiB/s wr, 17 op/s 2023-09-10T04:20:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:02 smithi082 ceph-mon[98694]: pgmap v9778: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 314 KiB/s wr, 17 op/s 2023-09-10T04:20:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18034.153858 secs 2023-09-10T04:20:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18034.153858 secs 2023-09-10T04:20:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18034.153858 secs 2023-09-10T04:20:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18035.327553 secs 2023-09-10T04:20:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:04 smithi170 ceph-mon[102832]: pgmap v9779: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T04:20:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18035.983393 secs 2023-09-10T04:20:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18035.327553 secs 2023-09-10T04:20:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:04 smithi082 ceph-mon[98694]: pgmap v9779: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T04:20:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18035.983393 secs 2023-09-10T04:20:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18035.327553 secs 2023-09-10T04:20:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:04 smithi098 ceph-mon[103124]: pgmap v9779: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T04:20:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18035.983393 secs 2023-09-10T04:20:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:06 smithi170 ceph-mon[102832]: pgmap v9780: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 256 KiB/s wr, 31 op/s 2023-09-10T04:20:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:06 smithi082 ceph-mon[98694]: pgmap v9780: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 256 KiB/s wr, 31 op/s 2023-09-10T04:20:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:06 smithi098 ceph-mon[103124]: pgmap v9780: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 256 KiB/s wr, 31 op/s 2023-09-10T04:20:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18039.153913 secs 2023-09-10T04:20:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:08 smithi082 ceph-mon[98694]: pgmap v9781: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 198 KiB/s wr, 25 op/s 2023-09-10T04:20:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:08 smithi082 ceph-mon[98694]: osdmap e679: 12 total, 12 up, 12 in 2023-09-10T04:20:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:20:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18039.153913 secs 2023-09-10T04:20:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:08 smithi170 ceph-mon[102832]: pgmap v9781: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 198 KiB/s wr, 25 op/s 2023-09-10T04:20:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:08 smithi170 ceph-mon[102832]: osdmap e679: 12 total, 12 up, 12 in 2023-09-10T04:20:09.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:20:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18039.153913 secs 2023-09-10T04:20:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:08 smithi098 ceph-mon[103124]: pgmap v9781: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 198 KiB/s wr, 25 op/s 2023-09-10T04:20:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:08 smithi098 ceph-mon[103124]: osdmap e679: 12 total, 12 up, 12 in 2023-09-10T04:20:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:20:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18040.327647 secs 2023-09-10T04:20:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18040.983462 secs 2023-09-10T04:20:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:20:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:20:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:20:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18040.327647 secs 2023-09-10T04:20:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18040.983462 secs 2023-09-10T04:20:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:20:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:20:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:20:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18040.327647 secs 2023-09-10T04:20:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18040.983462 secs 2023-09-10T04:20:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:20:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:20:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:20:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:11 smithi170 ceph-mon[102832]: pgmap v9783: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 237 KiB/s wr, 30 op/s 2023-09-10T04:20:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:11 smithi082 ceph-mon[98694]: pgmap v9783: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 237 KiB/s wr, 30 op/s 2023-09-10T04:20:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:11 smithi098 ceph-mon[103124]: pgmap v9783: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 237 KiB/s wr, 30 op/s 2023-09-10T04:20:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:13 smithi170 ceph-mon[102832]: pgmap v9784: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T04:20:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18044.153974 secs 2023-09-10T04:20:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:13 smithi082 ceph-mon[98694]: pgmap v9784: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T04:20:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18044.153974 secs 2023-09-10T04:20:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:13 smithi098 ceph-mon[103124]: pgmap v9784: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T04:20:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18044.153974 secs 2023-09-10T04:20:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18045.327703 secs 2023-09-10T04:20:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:14 smithi170 ceph-mon[102832]: osdmap e680: 12 total, 12 up, 12 in 2023-09-10T04:20:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:14 smithi170 ceph-mon[102832]: pgmap v9786: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:20:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18045.327703 secs 2023-09-10T04:20:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:14 smithi082 ceph-mon[98694]: osdmap e680: 12 total, 12 up, 12 in 2023-09-10T04:20:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:14 smithi082 ceph-mon[98694]: pgmap v9786: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:20:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18045.327703 secs 2023-09-10T04:20:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:14 smithi098 ceph-mon[103124]: osdmap e680: 12 total, 12 up, 12 in 2023-09-10T04:20:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:14 smithi098 ceph-mon[103124]: pgmap v9786: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:20:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18045.983488 secs 2023-09-10T04:20:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18045.983488 secs 2023-09-10T04:20:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18045.983488 secs 2023-09-10T04:20:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:16 smithi170 ceph-mon[102832]: pgmap v9787: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 147 KiB/s wr, 14 op/s 2023-09-10T04:20:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:16 smithi082 ceph-mon[98694]: pgmap v9787: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 147 KiB/s wr, 14 op/s 2023-09-10T04:20:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:16 smithi098 ceph-mon[103124]: pgmap v9787: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 147 KiB/s wr, 14 op/s 2023-09-10T04:20:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18049.154035 secs 2023-09-10T04:20:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:18 smithi170 ceph-mon[102832]: pgmap v9788: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T04:20:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18049.154035 secs 2023-09-10T04:20:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:18 smithi082 ceph-mon[98694]: pgmap v9788: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T04:20:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18049.154035 secs 2023-09-10T04:20:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:18 smithi098 ceph-mon[103124]: pgmap v9788: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T04:20:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18050.327760 secs 2023-09-10T04:20:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18050.983555 secs 2023-09-10T04:20:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18050.327760 secs 2023-09-10T04:20:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18050.983555 secs 2023-09-10T04:20:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18050.327760 secs 2023-09-10T04:20:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18050.983555 secs 2023-09-10T04:20:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:20 smithi082 ceph-mon[98694]: pgmap v9789: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T04:20:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:20 smithi098 ceph-mon[103124]: pgmap v9789: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T04:20:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:20 smithi170 ceph-mon[102832]: pgmap v9789: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T04:20:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:22 smithi082 ceph-mon[98694]: pgmap v9790: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T04:20:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:22 smithi098 ceph-mon[103124]: pgmap v9790: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T04:20:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:22 smithi170 ceph-mon[102832]: pgmap v9790: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T04:20:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18054.154094 secs 2023-09-10T04:20:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18054.154094 secs 2023-09-10T04:20:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18054.154094 secs 2023-09-10T04:20:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18055.327818 secs 2023-09-10T04:20:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:24 smithi082 ceph-mon[98694]: pgmap v9791: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T04:20:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18055.983620 secs 2023-09-10T04:20:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18055.327818 secs 2023-09-10T04:20:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:24 smithi098 ceph-mon[103124]: pgmap v9791: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T04:20:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18055.983620 secs 2023-09-10T04:20:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18055.327818 secs 2023-09-10T04:20:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:24 smithi170 ceph-mon[102832]: pgmap v9791: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T04:20:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18055.983620 secs 2023-09-10T04:20:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:26 smithi082 ceph-mon[98694]: pgmap v9792: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 158 KiB/s wr, 15 op/s 2023-09-10T04:20:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:26 smithi098 ceph-mon[103124]: pgmap v9792: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 158 KiB/s wr, 15 op/s 2023-09-10T04:20:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:26 smithi170 ceph-mon[102832]: pgmap v9792: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 158 KiB/s wr, 15 op/s 2023-09-10T04:20:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:28 smithi082 ceph-mon[98694]: pgmap v9793: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T04:20:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:28 smithi098 ceph-mon[103124]: pgmap v9793: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T04:20:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:28 smithi170 ceph-mon[102832]: pgmap v9793: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T04:20:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18060.327876 secs 2023-09-10T04:20:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18060.983683 secs 2023-09-10T04:20:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18060.327876 secs 2023-09-10T04:20:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18060.983683 secs 2023-09-10T04:20:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18060.327876 secs 2023-09-10T04:20:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18060.983683 secs 2023-09-10T04:20:30.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:30 smithi082 ceph-mon[98694]: pgmap v9794: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T04:20:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:30 smithi098 ceph-mon[103124]: pgmap v9794: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T04:20:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:30 smithi170 ceph-mon[102832]: pgmap v9794: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T04:20:31.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:20:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:20:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:20:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:32 smithi098 ceph-mon[103124]: pgmap v9795: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 221 KiB/s wr, 15 op/s 2023-09-10T04:20:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:20:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:20:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:20:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:32 smithi170 ceph-mon[102832]: pgmap v9795: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 221 KiB/s wr, 15 op/s 2023-09-10T04:20:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:20:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:20:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:20:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:32 smithi082 ceph-mon[98694]: pgmap v9795: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 221 KiB/s wr, 15 op/s 2023-09-10T04:20:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:20:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:20:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:20:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18065.327936 secs 2023-09-10T04:20:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:34 smithi170 ceph-mon[102832]: pgmap v9796: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 221 KiB/s wr, 11 op/s 2023-09-10T04:20:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18065.327936 secs 2023-09-10T04:20:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:34 smithi082 ceph-mon[98694]: pgmap v9796: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 221 KiB/s wr, 11 op/s 2023-09-10T04:20:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18065.327936 secs 2023-09-10T04:20:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:34 smithi098 ceph-mon[103124]: pgmap v9796: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 221 KiB/s wr, 11 op/s 2023-09-10T04:20:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18065.983745 secs 2023-09-10T04:20:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18065.983745 secs 2023-09-10T04:20:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18065.983745 secs 2023-09-10T04:20:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:36 smithi170 ceph-mon[102832]: pgmap v9797: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 352 KiB/s wr, 17 op/s 2023-09-10T04:20:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:36 smithi082 ceph-mon[98694]: pgmap v9797: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 352 KiB/s wr, 17 op/s 2023-09-10T04:20:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:36 smithi098 ceph-mon[103124]: pgmap v9797: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 352 KiB/s wr, 17 op/s 2023-09-10T04:20:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:38 smithi082 ceph-mon[98694]: pgmap v9798: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 292 KiB/s wr, 11 op/s 2023-09-10T04:20:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:38 smithi098 ceph-mon[103124]: pgmap v9798: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 292 KiB/s wr, 11 op/s 2023-09-10T04:20:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:38 smithi170 ceph-mon[102832]: pgmap v9798: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 292 KiB/s wr, 11 op/s 2023-09-10T04:20:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18070.328023 secs 2023-09-10T04:20:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18070.983809 secs 2023-09-10T04:20:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18070.328023 secs 2023-09-10T04:20:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18070.983809 secs 2023-09-10T04:20:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18070.328023 secs 2023-09-10T04:20:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18070.983809 secs 2023-09-10T04:20:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:40 smithi082 ceph-mon[98694]: pgmap v9799: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 292 KiB/s wr, 11 op/s 2023-09-10T04:20:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:40 smithi098 ceph-mon[103124]: pgmap v9799: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 292 KiB/s wr, 11 op/s 2023-09-10T04:20:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:40 smithi170 ceph-mon[102832]: pgmap v9799: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 292 KiB/s wr, 11 op/s 2023-09-10T04:20:42.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:42 smithi082 ceph-mon[98694]: pgmap v9800: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 448 KiB/s wr, 18 op/s 2023-09-10T04:20:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:42 smithi098 ceph-mon[103124]: pgmap v9800: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 448 KiB/s wr, 18 op/s 2023-09-10T04:20:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:42 smithi170 ceph-mon[102832]: pgmap v9800: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 448 KiB/s wr, 18 op/s 2023-09-10T04:20:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18075.328080 secs 2023-09-10T04:20:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:44 smithi170 ceph-mon[102832]: pgmap v9801: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 288 KiB/s wr, 12 op/s 2023-09-10T04:20:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18075.328080 secs 2023-09-10T04:20:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:44 smithi082 ceph-mon[98694]: pgmap v9801: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 288 KiB/s wr, 12 op/s 2023-09-10T04:20:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18075.328080 secs 2023-09-10T04:20:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:44 smithi098 ceph-mon[103124]: pgmap v9801: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 288 KiB/s wr, 12 op/s 2023-09-10T04:20:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18075.983873 secs 2023-09-10T04:20:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18075.983873 secs 2023-09-10T04:20:46.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18075.983873 secs 2023-09-10T04:20:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:46 smithi170 ceph-mon[102832]: pgmap v9802: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 406 KiB/s wr, 18 op/s 2023-09-10T04:20:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:46 smithi082 ceph-mon[98694]: pgmap v9802: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 406 KiB/s wr, 18 op/s 2023-09-10T04:20:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:46 smithi098 ceph-mon[103124]: pgmap v9802: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 406 KiB/s wr, 18 op/s 2023-09-10T04:20:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:48 smithi082 ceph-mon[98694]: pgmap v9803: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 274 KiB/s wr, 12 op/s 2023-09-10T04:20:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:48 smithi098 ceph-mon[103124]: pgmap v9803: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 274 KiB/s wr, 12 op/s 2023-09-10T04:20:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:48 smithi170 ceph-mon[102832]: pgmap v9803: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 274 KiB/s wr, 12 op/s 2023-09-10T04:20:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18080.328161 secs 2023-09-10T04:20:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18080.983937 secs 2023-09-10T04:20:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18080.328161 secs 2023-09-10T04:20:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18080.983937 secs 2023-09-10T04:20:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18080.328161 secs 2023-09-10T04:20:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18080.983937 secs 2023-09-10T04:20:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:50 smithi082 ceph-mon[98694]: pgmap v9804: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 274 KiB/s wr, 12 op/s 2023-09-10T04:20:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:50 smithi098 ceph-mon[103124]: pgmap v9804: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 274 KiB/s wr, 12 op/s 2023-09-10T04:20:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:50 smithi170 ceph-mon[102832]: pgmap v9804: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 274 KiB/s wr, 12 op/s 2023-09-10T04:20:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:52 smithi082 ceph-mon[98694]: pgmap v9805: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 434 KiB/s wr, 19 op/s 2023-09-10T04:20:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:52 smithi098 ceph-mon[103124]: pgmap v9805: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 434 KiB/s wr, 19 op/s 2023-09-10T04:20:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:52 smithi170 ceph-mon[102832]: pgmap v9805: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 434 KiB/s wr, 19 op/s 2023-09-10T04:20:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18084.154608 secs 2023-09-10T04:20:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18084.154608 secs 2023-09-10T04:20:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18084.154608 secs 2023-09-10T04:20:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18085.328245 secs 2023-09-10T04:20:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:54 smithi082 ceph-mon[98694]: pgmap v9806: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T04:20:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18085.984001 secs 2023-09-10T04:20:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18085.328245 secs 2023-09-10T04:20:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:54 smithi098 ceph-mon[103124]: pgmap v9806: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T04:20:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18085.984001 secs 2023-09-10T04:20:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18085.328245 secs 2023-09-10T04:20:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:54 smithi170 ceph-mon[102832]: pgmap v9806: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T04:20:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18085.984001 secs 2023-09-10T04:20:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:20:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:20:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:20:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:20:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:20:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:20:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:20:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:20:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:20:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:56 smithi098 ceph-mon[103124]: pgmap v9807: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 337 KiB/s wr, 18 op/s 2023-09-10T04:20:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:56 smithi170 ceph-mon[102832]: pgmap v9807: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 337 KiB/s wr, 18 op/s 2023-09-10T04:20:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:56 smithi082 ceph-mon[98694]: pgmap v9807: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 337 KiB/s wr, 18 op/s 2023-09-10T04:20:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18089.154702 secs 2023-09-10T04:20:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:58 smithi098 ceph-mon[103124]: pgmap v9808: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 219 KiB/s wr, 12 op/s 2023-09-10T04:20:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18089.154702 secs 2023-09-10T04:20:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:58 smithi170 ceph-mon[102832]: pgmap v9808: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 219 KiB/s wr, 12 op/s 2023-09-10T04:20:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18089.154702 secs 2023-09-10T04:20:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:58 smithi082 ceph-mon[98694]: pgmap v9808: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 219 KiB/s wr, 12 op/s 2023-09-10T04:21:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18090.328303 secs 2023-09-10T04:21:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:20:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18090.984063 secs 2023-09-10T04:21:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18090.328303 secs 2023-09-10T04:21:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:20:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18090.984063 secs 2023-09-10T04:21:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18090.328303 secs 2023-09-10T04:21:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:20:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18090.984063 secs 2023-09-10T04:21:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:00 smithi170 ceph-mon[102832]: pgmap v9809: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 219 KiB/s wr, 12 op/s 2023-09-10T04:21:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:00 smithi082 ceph-mon[98694]: pgmap v9809: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 219 KiB/s wr, 12 op/s 2023-09-10T04:21:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:00 smithi098 ceph-mon[103124]: pgmap v9809: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 219 KiB/s wr, 12 op/s 2023-09-10T04:21:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:02 smithi170 ceph-mon[102832]: pgmap v9810: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 315 KiB/s wr, 17 op/s 2023-09-10T04:21:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:02 smithi082 ceph-mon[98694]: pgmap v9810: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 315 KiB/s wr, 17 op/s 2023-09-10T04:21:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:02 smithi098 ceph-mon[103124]: pgmap v9810: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 315 KiB/s wr, 17 op/s 2023-09-10T04:21:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18094.154765 secs 2023-09-10T04:21:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18094.154765 secs 2023-09-10T04:21:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18094.154765 secs 2023-09-10T04:21:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18095.328361 secs 2023-09-10T04:21:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:04 smithi170 ceph-mon[102832]: pgmap v9811: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 156 KiB/s wr, 11 op/s 2023-09-10T04:21:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18095.984125 secs 2023-09-10T04:21:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18095.328361 secs 2023-09-10T04:21:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:04 smithi082 ceph-mon[98694]: pgmap v9811: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 156 KiB/s wr, 11 op/s 2023-09-10T04:21:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18095.984125 secs 2023-09-10T04:21:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18095.328361 secs 2023-09-10T04:21:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:04 smithi098 ceph-mon[103124]: pgmap v9811: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 156 KiB/s wr, 11 op/s 2023-09-10T04:21:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18095.984125 secs 2023-09-10T04:21:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:06 smithi170 ceph-mon[102832]: pgmap v9812: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 257 KiB/s wr, 31 op/s 2023-09-10T04:21:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:06 smithi082 ceph-mon[98694]: pgmap v9812: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 257 KiB/s wr, 31 op/s 2023-09-10T04:21:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:06 smithi098 ceph-mon[103124]: pgmap v9812: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 257 KiB/s wr, 31 op/s 2023-09-10T04:21:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18099.154825 secs 2023-09-10T04:21:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:08 smithi170 ceph-mon[102832]: pgmap v9813: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 198 KiB/s wr, 24 op/s 2023-09-10T04:21:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:08 smithi170 ceph-mon[102832]: osdmap e681: 12 total, 12 up, 12 in 2023-09-10T04:21:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18099.154825 secs 2023-09-10T04:21:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:08 smithi082 ceph-mon[98694]: pgmap v9813: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 198 KiB/s wr, 24 op/s 2023-09-10T04:21:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:08 smithi082 ceph-mon[98694]: osdmap e681: 12 total, 12 up, 12 in 2023-09-10T04:21:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18099.154825 secs 2023-09-10T04:21:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:08 smithi098 ceph-mon[103124]: pgmap v9813: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 198 KiB/s wr, 24 op/s 2023-09-10T04:21:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:08 smithi098 ceph-mon[103124]: osdmap e681: 12 total, 12 up, 12 in 2023-09-10T04:21:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18100.328419 secs 2023-09-10T04:21:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18100.984191 secs 2023-09-10T04:21:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:21:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18100.328419 secs 2023-09-10T04:21:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18100.984191 secs 2023-09-10T04:21:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:21:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18100.328419 secs 2023-09-10T04:21:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18100.984191 secs 2023-09-10T04:21:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:21:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:10 smithi170 ceph-mon[102832]: pgmap v9815: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 238 KiB/s wr, 29 op/s 2023-09-10T04:21:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:10 smithi082 ceph-mon[98694]: pgmap v9815: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 238 KiB/s wr, 29 op/s 2023-09-10T04:21:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:10 smithi098 ceph-mon[103124]: pgmap v9815: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 238 KiB/s wr, 29 op/s 2023-09-10T04:21:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:12 smithi170 ceph-mon[102832]: pgmap v9816: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T04:21:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:13.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:13.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:13.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:13.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18104.154919 secs 2023-09-10T04:21:13.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:13.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:13.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:13.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:13.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:13.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:12 smithi082 ceph-mon[98694]: pgmap v9816: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T04:21:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:13.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:13.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18104.154919 secs 2023-09-10T04:21:13.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:13.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:13.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:13.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:13.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:13.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:12 smithi098 ceph-mon[103124]: pgmap v9816: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T04:21:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:13.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:13.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:13.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18104.154919 secs 2023-09-10T04:21:13.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:13.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:13.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:13.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:13.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:13.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:13 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 44947k 2023-09-10T04:21:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:13 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 46026158: error parsing value: Value '46026158' is below minimum 939524096 2023-09-10T04:21:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:14.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:14.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:14.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:14.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:14.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:13 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T04:21:14.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:13 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T04:21:14.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:14.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:14.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:14.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:14.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:14.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:14.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:13 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T04:21:14.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:13 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T04:21:14.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:21:14.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:21:14.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:14.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:13 smithi170 ceph-mon[102832]: osdmap e682: 12 total, 12 up, 12 in 2023-09-10T04:21:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:13 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 44947k 2023-09-10T04:21:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:13 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 46026158: error parsing value: Value '46026158' is below minimum 939524096 2023-09-10T04:21:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:14.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:14.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:14.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:14.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:13 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T04:21:14.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:13 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T04:21:14.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:14.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:14.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:14.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:14.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:14.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:14.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:13 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T04:21:14.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:13 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T04:21:14.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:21:14.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:21:14.212 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:14.212 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:13 smithi082 ceph-mon[98694]: osdmap e682: 12 total, 12 up, 12 in 2023-09-10T04:21:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:13 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 44947k 2023-09-10T04:21:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:13 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 46026158: error parsing value: Value '46026158' is below minimum 939524096 2023-09-10T04:21:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:14.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:14.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:14.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:14.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:14.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:13 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T04:21:14.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:13 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T04:21:14.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:14.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:14.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:14.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:14.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:14.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:21:14.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:13 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T04:21:14.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:13 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T04:21:14.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:21:14.258 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:21:14.258 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:21:14.258 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:13 smithi098 ceph-mon[103124]: osdmap e682: 12 total, 12 up, 12 in 2023-09-10T04:21:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18105.328502 secs 2023-09-10T04:21:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:14 smithi170 ceph-mon[102832]: pgmap v9818: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:21:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18105.984257 secs 2023-09-10T04:21:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18105.328502 secs 2023-09-10T04:21:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:14 smithi082 ceph-mon[98694]: pgmap v9818: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:21:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18105.984257 secs 2023-09-10T04:21:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18105.328502 secs 2023-09-10T04:21:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:14 smithi098 ceph-mon[103124]: pgmap v9818: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:21:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18105.984257 secs 2023-09-10T04:21:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:16 smithi170 ceph-mon[102832]: pgmap v9819: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 147 KiB/s wr, 15 op/s 2023-09-10T04:21:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:16 smithi082 ceph-mon[98694]: pgmap v9819: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 147 KiB/s wr, 15 op/s 2023-09-10T04:21:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:16 smithi098 ceph-mon[103124]: pgmap v9819: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 147 KiB/s wr, 15 op/s 2023-09-10T04:21:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18109.154977 secs 2023-09-10T04:21:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18109.154977 secs 2023-09-10T04:21:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18109.154977 secs 2023-09-10T04:21:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:18 smithi170 ceph-mon[102832]: pgmap v9820: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T04:21:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:18 smithi082 ceph-mon[98694]: pgmap v9820: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T04:21:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:18 smithi098 ceph-mon[103124]: pgmap v9820: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T04:21:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18110.328559 secs 2023-09-10T04:21:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18110.984317 secs 2023-09-10T04:21:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18110.328559 secs 2023-09-10T04:21:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18110.984317 secs 2023-09-10T04:21:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18110.328559 secs 2023-09-10T04:21:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18110.984317 secs 2023-09-10T04:21:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:20 smithi170 ceph-mon[102832]: pgmap v9821: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T04:21:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:20 smithi082 ceph-mon[98694]: pgmap v9821: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T04:21:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:20 smithi098 ceph-mon[103124]: pgmap v9821: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T04:21:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:22 smithi170 ceph-mon[102832]: pgmap v9822: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T04:21:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18114.155061 secs 2023-09-10T04:21:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:22 smithi082 ceph-mon[98694]: pgmap v9822: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T04:21:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18114.155061 secs 2023-09-10T04:21:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:22 smithi098 ceph-mon[103124]: pgmap v9822: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T04:21:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18114.155061 secs 2023-09-10T04:21:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18115.328644 secs 2023-09-10T04:21:24.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:24 smithi170 ceph-mon[102832]: pgmap v9823: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T04:21:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18115.328644 secs 2023-09-10T04:21:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:24 smithi082 ceph-mon[98694]: pgmap v9823: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T04:21:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18115.328644 secs 2023-09-10T04:21:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:24 smithi098 ceph-mon[103124]: pgmap v9823: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T04:21:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18115.984382 secs 2023-09-10T04:21:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18115.984382 secs 2023-09-10T04:21:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18115.984382 secs 2023-09-10T04:21:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:26 smithi170 ceph-mon[102832]: pgmap v9824: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 159 KiB/s wr, 15 op/s 2023-09-10T04:21:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:26 smithi082 ceph-mon[98694]: pgmap v9824: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 159 KiB/s wr, 15 op/s 2023-09-10T04:21:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:26 smithi098 ceph-mon[103124]: pgmap v9824: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 159 KiB/s wr, 15 op/s 2023-09-10T04:21:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:28 smithi082 ceph-mon[98694]: pgmap v9825: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T04:21:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:28 smithi098 ceph-mon[103124]: pgmap v9825: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T04:21:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:28 smithi170 ceph-mon[102832]: pgmap v9825: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T04:21:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18120.328701 secs 2023-09-10T04:21:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18120.984446 secs 2023-09-10T04:21:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18120.328701 secs 2023-09-10T04:21:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18120.984446 secs 2023-09-10T04:21:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18120.328701 secs 2023-09-10T04:21:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18120.984446 secs 2023-09-10T04:21:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:30 smithi082 ceph-mon[98694]: pgmap v9826: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T04:21:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:30 smithi098 ceph-mon[103124]: pgmap v9826: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T04:21:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:30 smithi170 ceph-mon[102832]: pgmap v9826: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T04:21:31.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:21:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:21:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:21:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:32 smithi082 ceph-mon[98694]: pgmap v9827: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 222 KiB/s wr, 15 op/s 2023-09-10T04:21:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:21:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:21:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:21:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:32 smithi098 ceph-mon[103124]: pgmap v9827: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 222 KiB/s wr, 15 op/s 2023-09-10T04:21:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:21:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:21:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:21:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:32 smithi170 ceph-mon[102832]: pgmap v9827: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 222 KiB/s wr, 15 op/s 2023-09-10T04:21:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:21:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:21:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:21:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18125.328764 secs 2023-09-10T04:21:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:34 smithi170 ceph-mon[102832]: pgmap v9828: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 222 KiB/s wr, 10 op/s 2023-09-10T04:21:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18125.328764 secs 2023-09-10T04:21:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:34 smithi082 ceph-mon[98694]: pgmap v9828: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 222 KiB/s wr, 10 op/s 2023-09-10T04:21:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18125.328764 secs 2023-09-10T04:21:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:34 smithi098 ceph-mon[103124]: pgmap v9828: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 222 KiB/s wr, 10 op/s 2023-09-10T04:21:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18125.984488 secs 2023-09-10T04:21:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18125.984488 secs 2023-09-10T04:21:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18125.984488 secs 2023-09-10T04:21:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:36 smithi170 ceph-mon[102832]: pgmap v9829: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 354 KiB/s wr, 16 op/s 2023-09-10T04:21:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:36 smithi082 ceph-mon[98694]: pgmap v9829: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 354 KiB/s wr, 16 op/s 2023-09-10T04:21:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:36 smithi098 ceph-mon[103124]: pgmap v9829: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 354 KiB/s wr, 16 op/s 2023-09-10T04:21:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:38 smithi082 ceph-mon[98694]: pgmap v9830: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T04:21:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:38 smithi098 ceph-mon[103124]: pgmap v9830: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T04:21:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:38 smithi170 ceph-mon[102832]: pgmap v9830: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T04:21:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18130.328820 secs 2023-09-10T04:21:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18130.984554 secs 2023-09-10T04:21:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18130.328820 secs 2023-09-10T04:21:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18130.984554 secs 2023-09-10T04:21:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18130.328820 secs 2023-09-10T04:21:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18130.984554 secs 2023-09-10T04:21:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:40 smithi082 ceph-mon[98694]: pgmap v9831: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T04:21:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:40 smithi098 ceph-mon[103124]: pgmap v9831: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T04:21:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:40 smithi170 ceph-mon[102832]: pgmap v9831: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T04:21:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:42 smithi082 ceph-mon[98694]: pgmap v9832: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 450 KiB/s wr, 18 op/s 2023-09-10T04:21:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:42 smithi098 ceph-mon[103124]: pgmap v9832: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 450 KiB/s wr, 18 op/s 2023-09-10T04:21:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:42 smithi170 ceph-mon[102832]: pgmap v9832: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 450 KiB/s wr, 18 op/s 2023-09-10T04:21:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18135.328883 secs 2023-09-10T04:21:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:44 smithi170 ceph-mon[102832]: pgmap v9833: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 289 KiB/s wr, 12 op/s 2023-09-10T04:21:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18135.328883 secs 2023-09-10T04:21:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:44 smithi082 ceph-mon[98694]: pgmap v9833: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 289 KiB/s wr, 12 op/s 2023-09-10T04:21:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18135.328883 secs 2023-09-10T04:21:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:44 smithi098 ceph-mon[103124]: pgmap v9833: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 289 KiB/s wr, 12 op/s 2023-09-10T04:21:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18135.984617 secs 2023-09-10T04:21:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18135.984617 secs 2023-09-10T04:21:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18135.984617 secs 2023-09-10T04:21:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:46 smithi170 ceph-mon[102832]: pgmap v9834: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 408 KiB/s wr, 18 op/s 2023-09-10T04:21:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:46 smithi082 ceph-mon[98694]: pgmap v9834: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 408 KiB/s wr, 18 op/s 2023-09-10T04:21:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:46 smithi098 ceph-mon[103124]: pgmap v9834: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 408 KiB/s wr, 18 op/s 2023-09-10T04:21:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:48 smithi082 ceph-mon[98694]: pgmap v9835: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 276 KiB/s wr, 12 op/s 2023-09-10T04:21:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:48 smithi098 ceph-mon[103124]: pgmap v9835: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 276 KiB/s wr, 12 op/s 2023-09-10T04:21:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:48 smithi170 ceph-mon[102832]: pgmap v9835: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 276 KiB/s wr, 12 op/s 2023-09-10T04:21:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18140.328942 secs 2023-09-10T04:21:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18140.984682 secs 2023-09-10T04:21:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18140.328942 secs 2023-09-10T04:21:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18140.984682 secs 2023-09-10T04:21:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18140.328942 secs 2023-09-10T04:21:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18140.984682 secs 2023-09-10T04:21:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:50 smithi082 ceph-mon[98694]: pgmap v9836: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 276 KiB/s wr, 12 op/s 2023-09-10T04:21:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:50 smithi098 ceph-mon[103124]: pgmap v9836: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 276 KiB/s wr, 12 op/s 2023-09-10T04:21:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:50 smithi170 ceph-mon[102832]: pgmap v9836: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 276 KiB/s wr, 12 op/s 2023-09-10T04:21:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:52 smithi098 ceph-mon[103124]: pgmap v9837: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 436 KiB/s wr, 18 op/s 2023-09-10T04:21:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:52 smithi170 ceph-mon[102832]: pgmap v9837: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 436 KiB/s wr, 18 op/s 2023-09-10T04:21:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:52 smithi082 ceph-mon[98694]: pgmap v9837: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 436 KiB/s wr, 18 op/s 2023-09-10T04:21:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18144.155514 secs 2023-09-10T04:21:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18144.155514 secs 2023-09-10T04:21:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18144.155514 secs 2023-09-10T04:21:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18145.329031 secs 2023-09-10T04:21:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:54 smithi098 ceph-mon[103124]: pgmap v9838: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 11 op/s 2023-09-10T04:21:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18145.984749 secs 2023-09-10T04:21:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18145.329031 secs 2023-09-10T04:21:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:54 smithi170 ceph-mon[102832]: pgmap v9838: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 11 op/s 2023-09-10T04:21:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18145.984749 secs 2023-09-10T04:21:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18145.329031 secs 2023-09-10T04:21:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:54 smithi082 ceph-mon[98694]: pgmap v9838: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 11 op/s 2023-09-10T04:21:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18145.984749 secs 2023-09-10T04:21:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:21:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:21:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:21:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:21:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:21:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:21:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:21:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:21:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:21:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:56 smithi170 ceph-mon[102832]: pgmap v9839: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 339 KiB/s wr, 17 op/s 2023-09-10T04:21:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:56 smithi082 ceph-mon[98694]: pgmap v9839: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 339 KiB/s wr, 17 op/s 2023-09-10T04:21:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:56 smithi098 ceph-mon[103124]: pgmap v9839: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 339 KiB/s wr, 17 op/s 2023-09-10T04:21:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18149.155573 secs 2023-09-10T04:21:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:58 smithi170 ceph-mon[102832]: pgmap v9840: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 221 KiB/s wr, 11 op/s 2023-09-10T04:21:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18149.155573 secs 2023-09-10T04:21:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:58 smithi082 ceph-mon[98694]: pgmap v9840: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 221 KiB/s wr, 11 op/s 2023-09-10T04:21:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18149.155573 secs 2023-09-10T04:21:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:58 smithi098 ceph-mon[103124]: pgmap v9840: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 221 KiB/s wr, 11 op/s 2023-09-10T04:22:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18150.329090 secs 2023-09-10T04:22:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:21:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18150.984809 secs 2023-09-10T04:22:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18150.329090 secs 2023-09-10T04:22:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:21:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18150.984809 secs 2023-09-10T04:22:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18150.329090 secs 2023-09-10T04:22:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:21:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18150.984809 secs 2023-09-10T04:22:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:00 smithi170 ceph-mon[102832]: pgmap v9841: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 221 KiB/s wr, 11 op/s 2023-09-10T04:22:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:00 smithi082 ceph-mon[98694]: pgmap v9841: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 221 KiB/s wr, 11 op/s 2023-09-10T04:22:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:00 smithi098 ceph-mon[103124]: pgmap v9841: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 221 KiB/s wr, 11 op/s 2023-09-10T04:22:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:02 smithi170 ceph-mon[102832]: pgmap v9842: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 362 KiB/s wr, 32 op/s 2023-09-10T04:22:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:02 smithi082 ceph-mon[98694]: pgmap v9842: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 362 KiB/s wr, 32 op/s 2023-09-10T04:22:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:02 smithi098 ceph-mon[103124]: pgmap v9842: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 362 KiB/s wr, 32 op/s 2023-09-10T04:22:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18154.155684 secs 2023-09-10T04:22:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18154.155684 secs 2023-09-10T04:22:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18154.155684 secs 2023-09-10T04:22:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18155.329155 secs 2023-09-10T04:22:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:04 smithi170 ceph-mon[102832]: pgmap v9843: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 201 KiB/s wr, 26 op/s 2023-09-10T04:22:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18155.984872 secs 2023-09-10T04:22:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18155.329155 secs 2023-09-10T04:22:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:04 smithi082 ceph-mon[98694]: pgmap v9843: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 201 KiB/s wr, 26 op/s 2023-09-10T04:22:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18155.984872 secs 2023-09-10T04:22:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18155.329155 secs 2023-09-10T04:22:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:04 smithi098 ceph-mon[103124]: pgmap v9843: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 201 KiB/s wr, 26 op/s 2023-09-10T04:22:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18155.984872 secs 2023-09-10T04:22:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:06 smithi170 ceph-mon[102832]: pgmap v9844: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 259 KiB/s wr, 32 op/s 2023-09-10T04:22:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:06 smithi082 ceph-mon[98694]: pgmap v9844: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 259 KiB/s wr, 32 op/s 2023-09-10T04:22:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:06 smithi098 ceph-mon[103124]: pgmap v9844: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 259 KiB/s wr, 32 op/s 2023-09-10T04:22:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18159.155743 secs 2023-09-10T04:22:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18159.155743 secs 2023-09-10T04:22:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18159.155743 secs 2023-09-10T04:22:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:08 smithi170 ceph-mon[102832]: pgmap v9845: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 198 KiB/s wr, 26 op/s 2023-09-10T04:22:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:08 smithi170 ceph-mon[102832]: osdmap e683: 12 total, 12 up, 12 in 2023-09-10T04:22:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:08 smithi082 ceph-mon[98694]: pgmap v9845: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 198 KiB/s wr, 26 op/s 2023-09-10T04:22:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:08 smithi082 ceph-mon[98694]: osdmap e683: 12 total, 12 up, 12 in 2023-09-10T04:22:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:08 smithi098 ceph-mon[103124]: pgmap v9845: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 198 KiB/s wr, 26 op/s 2023-09-10T04:22:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:08 smithi098 ceph-mon[103124]: osdmap e683: 12 total, 12 up, 12 in 2023-09-10T04:22:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18160.329237 secs 2023-09-10T04:22:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18160.984934 secs 2023-09-10T04:22:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18160.329237 secs 2023-09-10T04:22:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18160.984934 secs 2023-09-10T04:22:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18160.329237 secs 2023-09-10T04:22:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18160.984934 secs 2023-09-10T04:22:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:10 smithi170 ceph-mon[102832]: pgmap v9847: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 238 KiB/s wr, 31 op/s 2023-09-10T04:22:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:10 smithi082 ceph-mon[98694]: pgmap v9847: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 238 KiB/s wr, 31 op/s 2023-09-10T04:22:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:10 smithi098 ceph-mon[103124]: pgmap v9847: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 238 KiB/s wr, 31 op/s 2023-09-10T04:22:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:12 smithi170 ceph-mon[102832]: pgmap v9848: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T04:22:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18164.155798 secs 2023-09-10T04:22:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:12 smithi082 ceph-mon[98694]: pgmap v9848: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T04:22:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18164.155798 secs 2023-09-10T04:22:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:12 smithi098 ceph-mon[103124]: pgmap v9848: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 69 KiB/s wr, 12 op/s 2023-09-10T04:22:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18164.155798 secs 2023-09-10T04:22:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:22:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:13 smithi170 ceph-mon[102832]: osdmap e684: 12 total, 12 up, 12 in 2023-09-10T04:22:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:22:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:13 smithi082 ceph-mon[98694]: osdmap e684: 12 total, 12 up, 12 in 2023-09-10T04:22:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:22:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:13 smithi098 ceph-mon[103124]: osdmap e684: 12 total, 12 up, 12 in 2023-09-10T04:22:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18165.329324 secs 2023-09-10T04:22:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:14 smithi170 ceph-mon[102832]: pgmap v9850: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:22:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18165.985001 secs 2023-09-10T04:22:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:22:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:22:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:22:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18165.329324 secs 2023-09-10T04:22:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:14 smithi082 ceph-mon[98694]: pgmap v9850: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:22:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18165.985001 secs 2023-09-10T04:22:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:22:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:22:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:22:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18165.329324 secs 2023-09-10T04:22:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:14 smithi098 ceph-mon[103124]: pgmap v9850: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:22:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18165.985001 secs 2023-09-10T04:22:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:22:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:22:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:22:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:16 smithi170 ceph-mon[102832]: pgmap v9851: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 148 KiB/s wr, 14 op/s 2023-09-10T04:22:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:16 smithi082 ceph-mon[98694]: pgmap v9851: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 148 KiB/s wr, 14 op/s 2023-09-10T04:22:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:16 smithi098 ceph-mon[103124]: pgmap v9851: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 148 KiB/s wr, 14 op/s 2023-09-10T04:22:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18169.155906 secs 2023-09-10T04:22:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18169.155906 secs 2023-09-10T04:22:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18169.155906 secs 2023-09-10T04:22:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:18 smithi170 ceph-mon[102832]: pgmap v9852: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T04:22:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:18 smithi082 ceph-mon[98694]: pgmap v9852: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T04:22:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:18 smithi098 ceph-mon[103124]: pgmap v9852: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T04:22:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18170.329402 secs 2023-09-10T04:22:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18170.985063 secs 2023-09-10T04:22:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18170.329402 secs 2023-09-10T04:22:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18170.985063 secs 2023-09-10T04:22:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18170.329402 secs 2023-09-10T04:22:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18170.985063 secs 2023-09-10T04:22:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:20 smithi170 ceph-mon[102832]: pgmap v9853: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T04:22:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:20 smithi082 ceph-mon[98694]: pgmap v9853: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T04:22:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:20 smithi098 ceph-mon[103124]: pgmap v9853: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T04:22:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:22 smithi170 ceph-mon[102832]: pgmap v9854: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T04:22:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18174.156000 secs 2023-09-10T04:22:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:22 smithi082 ceph-mon[98694]: pgmap v9854: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T04:22:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18174.156000 secs 2023-09-10T04:22:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:22 smithi098 ceph-mon[103124]: pgmap v9854: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T04:22:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18174.156000 secs 2023-09-10T04:22:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18175.329464 secs 2023-09-10T04:22:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:24 smithi170 ceph-mon[102832]: pgmap v9855: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T04:22:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18175.329464 secs 2023-09-10T04:22:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:24 smithi082 ceph-mon[98694]: pgmap v9855: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T04:22:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18175.329464 secs 2023-09-10T04:22:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:24 smithi098 ceph-mon[103124]: pgmap v9855: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T04:22:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18175.985127 secs 2023-09-10T04:22:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18175.985127 secs 2023-09-10T04:22:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18175.985127 secs 2023-09-10T04:22:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:26 smithi170 ceph-mon[102832]: pgmap v9856: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 169 KiB/s wr, 14 op/s 2023-09-10T04:22:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:26 smithi082 ceph-mon[98694]: pgmap v9856: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 169 KiB/s wr, 14 op/s 2023-09-10T04:22:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:26 smithi098 ceph-mon[103124]: pgmap v9856: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 169 KiB/s wr, 14 op/s 2023-09-10T04:22:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:28 smithi098 ceph-mon[103124]: pgmap v9857: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T04:22:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:28 smithi170 ceph-mon[102832]: pgmap v9857: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T04:22:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:28 smithi082 ceph-mon[98694]: pgmap v9857: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T04:22:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18180.329548 secs 2023-09-10T04:22:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18180.985190 secs 2023-09-10T04:22:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18180.329548 secs 2023-09-10T04:22:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18180.985190 secs 2023-09-10T04:22:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18180.329548 secs 2023-09-10T04:22:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18180.985190 secs 2023-09-10T04:22:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:30 smithi098 ceph-mon[103124]: pgmap v9858: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T04:22:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:30 smithi170 ceph-mon[102832]: pgmap v9858: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T04:22:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:30 smithi082 ceph-mon[98694]: pgmap v9858: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T04:22:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:22:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:22:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:22:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:32 smithi098 ceph-mon[103124]: pgmap v9859: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 223 KiB/s wr, 15 op/s 2023-09-10T04:22:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:22:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:22:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:22:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:32 smithi170 ceph-mon[102832]: pgmap v9859: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 223 KiB/s wr, 15 op/s 2023-09-10T04:22:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:22:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:22:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:22:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:32 smithi082 ceph-mon[98694]: pgmap v9859: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 223 KiB/s wr, 15 op/s 2023-09-10T04:22:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:22:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:22:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:22:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18185.329610 secs 2023-09-10T04:22:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:34 smithi170 ceph-mon[102832]: pgmap v9860: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T04:22:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18185.329610 secs 2023-09-10T04:22:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:34 smithi082 ceph-mon[98694]: pgmap v9860: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T04:22:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18185.329610 secs 2023-09-10T04:22:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:34 smithi098 ceph-mon[103124]: pgmap v9860: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T04:22:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18185.985252 secs 2023-09-10T04:22:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18185.985252 secs 2023-09-10T04:22:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18185.985252 secs 2023-09-10T04:22:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:36 smithi170 ceph-mon[102832]: pgmap v9861: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 355 KiB/s wr, 17 op/s 2023-09-10T04:22:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:36 smithi082 ceph-mon[98694]: pgmap v9861: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 355 KiB/s wr, 17 op/s 2023-09-10T04:22:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:36 smithi098 ceph-mon[103124]: pgmap v9861: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 355 KiB/s wr, 17 op/s 2023-09-10T04:22:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:38 smithi098 ceph-mon[103124]: pgmap v9862: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T04:22:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:38 smithi170 ceph-mon[102832]: pgmap v9862: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T04:22:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:38 smithi082 ceph-mon[98694]: pgmap v9862: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T04:22:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18190.329668 secs 2023-09-10T04:22:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18190.985318 secs 2023-09-10T04:22:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18190.329668 secs 2023-09-10T04:22:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18190.985318 secs 2023-09-10T04:22:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18190.329668 secs 2023-09-10T04:22:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18190.985318 secs 2023-09-10T04:22:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:40 smithi098 ceph-mon[103124]: pgmap v9863: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T04:22:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:40 smithi170 ceph-mon[102832]: pgmap v9863: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T04:22:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:40 smithi082 ceph-mon[98694]: pgmap v9863: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T04:22:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:42 smithi098 ceph-mon[103124]: pgmap v9864: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 443 KiB/s wr, 19 op/s 2023-09-10T04:22:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:42 smithi170 ceph-mon[102832]: pgmap v9864: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 443 KiB/s wr, 19 op/s 2023-09-10T04:22:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:42 smithi082 ceph-mon[98694]: pgmap v9864: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 443 KiB/s wr, 19 op/s 2023-09-10T04:22:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18195.329727 secs 2023-09-10T04:22:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:44 smithi170 ceph-mon[102832]: pgmap v9865: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 291 KiB/s wr, 13 op/s 2023-09-10T04:22:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18195.329727 secs 2023-09-10T04:22:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:44 smithi082 ceph-mon[98694]: pgmap v9865: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 291 KiB/s wr, 13 op/s 2023-09-10T04:22:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18195.329727 secs 2023-09-10T04:22:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:44 smithi098 ceph-mon[103124]: pgmap v9865: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 291 KiB/s wr, 13 op/s 2023-09-10T04:22:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18195.985379 secs 2023-09-10T04:22:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18195.985379 secs 2023-09-10T04:22:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18195.985379 secs 2023-09-10T04:22:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:46 smithi170 ceph-mon[102832]: pgmap v9866: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 409 KiB/s wr, 18 op/s 2023-09-10T04:22:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:46 smithi082 ceph-mon[98694]: pgmap v9866: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 409 KiB/s wr, 18 op/s 2023-09-10T04:22:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:46 smithi098 ceph-mon[103124]: pgmap v9866: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 409 KiB/s wr, 18 op/s 2023-09-10T04:22:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:48 smithi098 ceph-mon[103124]: pgmap v9867: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T04:22:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:48 smithi170 ceph-mon[102832]: pgmap v9867: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T04:22:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:48 smithi082 ceph-mon[98694]: pgmap v9867: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T04:22:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18200.329786 secs 2023-09-10T04:22:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18200.985443 secs 2023-09-10T04:22:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18200.329786 secs 2023-09-10T04:22:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18200.985443 secs 2023-09-10T04:22:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18200.329786 secs 2023-09-10T04:22:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18200.985443 secs 2023-09-10T04:22:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:50 smithi098 ceph-mon[103124]: pgmap v9868: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T04:22:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:50 smithi170 ceph-mon[102832]: pgmap v9868: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T04:22:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:50 smithi082 ceph-mon[98694]: pgmap v9868: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T04:22:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:52 smithi098 ceph-mon[103124]: pgmap v9869: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 438 KiB/s wr, 18 op/s 2023-09-10T04:22:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:52 smithi170 ceph-mon[102832]: pgmap v9869: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 438 KiB/s wr, 18 op/s 2023-09-10T04:22:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:52 smithi082 ceph-mon[98694]: pgmap v9869: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 438 KiB/s wr, 18 op/s 2023-09-10T04:22:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18204.156588 secs 2023-09-10T04:22:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18204.156588 secs 2023-09-10T04:22:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18204.156588 secs 2023-09-10T04:22:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18205.329844 secs 2023-09-10T04:22:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:54 smithi098 ceph-mon[103124]: pgmap v9870: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 280 KiB/s wr, 11 op/s 2023-09-10T04:22:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18205.985493 secs 2023-09-10T04:22:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18205.329844 secs 2023-09-10T04:22:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:54 smithi170 ceph-mon[102832]: pgmap v9870: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 280 KiB/s wr, 11 op/s 2023-09-10T04:22:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18205.985493 secs 2023-09-10T04:22:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18205.329844 secs 2023-09-10T04:22:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:54 smithi082 ceph-mon[98694]: pgmap v9870: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 280 KiB/s wr, 11 op/s 2023-09-10T04:22:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18205.985493 secs 2023-09-10T04:22:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:22:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:22:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:22:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:22:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:22:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:22:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:22:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:22:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:22:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:56 smithi170 ceph-mon[102832]: pgmap v9871: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 340 KiB/s wr, 17 op/s 2023-09-10T04:22:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:56 smithi082 ceph-mon[98694]: pgmap v9871: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 340 KiB/s wr, 17 op/s 2023-09-10T04:22:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:56 smithi098 ceph-mon[103124]: pgmap v9871: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 340 KiB/s wr, 17 op/s 2023-09-10T04:22:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18209.156658 secs 2023-09-10T04:22:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18209.156658 secs 2023-09-10T04:22:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18209.156658 secs 2023-09-10T04:22:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:58 smithi170 ceph-mon[102832]: pgmap v9872: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 221 KiB/s wr, 12 op/s 2023-09-10T04:22:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:58 smithi082 ceph-mon[98694]: pgmap v9872: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 221 KiB/s wr, 12 op/s 2023-09-10T04:22:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:58 smithi098 ceph-mon[103124]: pgmap v9872: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 221 KiB/s wr, 12 op/s 2023-09-10T04:23:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18210.329901 secs 2023-09-10T04:23:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:22:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18210.985553 secs 2023-09-10T04:23:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18210.329901 secs 2023-09-10T04:23:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:22:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18210.985553 secs 2023-09-10T04:23:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18210.329901 secs 2023-09-10T04:23:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:22:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18210.985553 secs 2023-09-10T04:23:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:00 smithi170 ceph-mon[102832]: pgmap v9873: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 221 KiB/s wr, 12 op/s 2023-09-10T04:23:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:00 smithi082 ceph-mon[98694]: pgmap v9873: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 221 KiB/s wr, 12 op/s 2023-09-10T04:23:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:00 smithi098 ceph-mon[103124]: pgmap v9873: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 221 KiB/s wr, 12 op/s 2023-09-10T04:23:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:02 smithi170 ceph-mon[102832]: pgmap v9874: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 351 KiB/s wr, 31 op/s 2023-09-10T04:23:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18214.156736 secs 2023-09-10T04:23:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:02 smithi082 ceph-mon[98694]: pgmap v9874: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 351 KiB/s wr, 31 op/s 2023-09-10T04:23:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18214.156736 secs 2023-09-10T04:23:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:02 smithi098 ceph-mon[103124]: pgmap v9874: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 351 KiB/s wr, 31 op/s 2023-09-10T04:23:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18214.156736 secs 2023-09-10T04:23:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18215.329974 secs 2023-09-10T04:23:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:04 smithi170 ceph-mon[102832]: pgmap v9875: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 190 KiB/s wr, 25 op/s 2023-09-10T04:23:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18215.329974 secs 2023-09-10T04:23:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:04 smithi082 ceph-mon[98694]: pgmap v9875: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 190 KiB/s wr, 25 op/s 2023-09-10T04:23:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18215.329974 secs 2023-09-10T04:23:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:04 smithi098 ceph-mon[103124]: pgmap v9875: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 190 KiB/s wr, 25 op/s 2023-09-10T04:23:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18215.985618 secs 2023-09-10T04:23:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18215.985618 secs 2023-09-10T04:23:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18215.985618 secs 2023-09-10T04:23:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:06 smithi170 ceph-mon[102832]: pgmap v9876: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 246 KiB/s wr, 31 op/s 2023-09-10T04:23:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:06 smithi082 ceph-mon[98694]: pgmap v9876: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 246 KiB/s wr, 31 op/s 2023-09-10T04:23:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:06 smithi098 ceph-mon[103124]: pgmap v9876: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 246 KiB/s wr, 31 op/s 2023-09-10T04:23:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18219.156779 secs 2023-09-10T04:23:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:08 smithi170 ceph-mon[102832]: osdmap e685: 12 total, 12 up, 12 in 2023-09-10T04:23:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:08 smithi170 ceph-mon[102832]: pgmap v9878: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 223 KiB/s wr, 30 op/s 2023-09-10T04:23:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18219.156779 secs 2023-09-10T04:23:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:08 smithi082 ceph-mon[98694]: osdmap e685: 12 total, 12 up, 12 in 2023-09-10T04:23:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:08 smithi082 ceph-mon[98694]: pgmap v9878: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 223 KiB/s wr, 30 op/s 2023-09-10T04:23:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18219.156779 secs 2023-09-10T04:23:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:08 smithi098 ceph-mon[103124]: osdmap e685: 12 total, 12 up, 12 in 2023-09-10T04:23:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:08 smithi098 ceph-mon[103124]: pgmap v9878: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 223 KiB/s wr, 30 op/s 2023-09-10T04:23:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18220.330031 secs 2023-09-10T04:23:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18220.330031 secs 2023-09-10T04:23:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18220.330031 secs 2023-09-10T04:23:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18220.985680 secs 2023-09-10T04:23:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:10 smithi170 ceph-mon[102832]: pgmap v9879: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 223 KiB/s wr, 30 op/s 2023-09-10T04:23:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18220.985680 secs 2023-09-10T04:23:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:10 smithi082 ceph-mon[98694]: pgmap v9879: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 223 KiB/s wr, 30 op/s 2023-09-10T04:23:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18220.985680 secs 2023-09-10T04:23:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:10 smithi098 ceph-mon[103124]: pgmap v9879: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 223 KiB/s wr, 30 op/s 2023-09-10T04:23:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:12 smithi098 ceph-mon[103124]: pgmap v9880: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 67 KiB/s wr, 13 op/s 2023-09-10T04:23:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:12 smithi170 ceph-mon[102832]: pgmap v9880: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 67 KiB/s wr, 13 op/s 2023-09-10T04:23:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:12 smithi082 ceph-mon[98694]: pgmap v9880: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 67 KiB/s wr, 13 op/s 2023-09-10T04:23:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18224.156864 secs 2023-09-10T04:23:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:13 smithi098 ceph-mon[103124]: osdmap e686: 12 total, 12 up, 12 in 2023-09-10T04:23:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18224.156864 secs 2023-09-10T04:23:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:13 smithi170 ceph-mon[102832]: osdmap e686: 12 total, 12 up, 12 in 2023-09-10T04:23:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18224.156864 secs 2023-09-10T04:23:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:13 smithi082 ceph-mon[98694]: osdmap e686: 12 total, 12 up, 12 in 2023-09-10T04:23:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18225.330091 secs 2023-09-10T04:23:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:14 smithi098 ceph-mon[103124]: pgmap v9882: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:23:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18225.985743 secs 2023-09-10T04:23:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:23:15.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:23:15.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:23:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18225.330091 secs 2023-09-10T04:23:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:14 smithi170 ceph-mon[102832]: pgmap v9882: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:23:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18225.985743 secs 2023-09-10T04:23:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:23:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:23:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:23:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18225.330091 secs 2023-09-10T04:23:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:14 smithi082 ceph-mon[98694]: pgmap v9882: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:23:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18225.985743 secs 2023-09-10T04:23:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:23:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:23:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:23:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:23:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:23:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:23:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:23:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:23:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:23:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:23:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:23:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:23:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:17 smithi170 ceph-mon[102832]: pgmap v9883: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 148 KiB/s wr, 15 op/s 2023-09-10T04:23:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:17 smithi082 ceph-mon[98694]: pgmap v9883: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 148 KiB/s wr, 15 op/s 2023-09-10T04:23:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:17 smithi098 ceph-mon[103124]: pgmap v9883: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 148 KiB/s wr, 15 op/s 2023-09-10T04:23:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18229.156976 secs 2023-09-10T04:23:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18229.156976 secs 2023-09-10T04:23:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18229.156976 secs 2023-09-10T04:23:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:19 smithi170 ceph-mon[102832]: pgmap v9884: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T04:23:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18230.330149 secs 2023-09-10T04:23:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:19 smithi082 ceph-mon[98694]: pgmap v9884: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T04:23:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18230.330149 secs 2023-09-10T04:23:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:19 smithi098 ceph-mon[103124]: pgmap v9884: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T04:23:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18230.330149 secs 2023-09-10T04:23:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18230.985807 secs 2023-09-10T04:23:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18230.985807 secs 2023-09-10T04:23:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18230.985807 secs 2023-09-10T04:23:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:21 smithi170 ceph-mon[102832]: pgmap v9885: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T04:23:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:21 smithi082 ceph-mon[98694]: pgmap v9885: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T04:23:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:21 smithi098 ceph-mon[103124]: pgmap v9885: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T04:23:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:23 smithi170 ceph-mon[102832]: pgmap v9886: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T04:23:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18234.157036 secs 2023-09-10T04:23:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:23 smithi082 ceph-mon[98694]: pgmap v9886: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T04:23:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18234.157036 secs 2023-09-10T04:23:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:23 smithi098 ceph-mon[103124]: pgmap v9886: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T04:23:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18234.157036 secs 2023-09-10T04:23:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18235.330208 secs 2023-09-10T04:23:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:24 smithi170 ceph-mon[102832]: pgmap v9887: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T04:23:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18235.330208 secs 2023-09-10T04:23:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:24 smithi082 ceph-mon[98694]: pgmap v9887: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T04:23:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18235.330208 secs 2023-09-10T04:23:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:24 smithi098 ceph-mon[103124]: pgmap v9887: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T04:23:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18235.985867 secs 2023-09-10T04:23:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18235.985867 secs 2023-09-10T04:23:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18235.985867 secs 2023-09-10T04:23:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:26 smithi170 ceph-mon[102832]: pgmap v9888: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 188 KiB/s wr, 14 op/s 2023-09-10T04:23:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:26 smithi082 ceph-mon[98694]: pgmap v9888: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 188 KiB/s wr, 14 op/s 2023-09-10T04:23:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:26 smithi098 ceph-mon[103124]: pgmap v9888: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 188 KiB/s wr, 14 op/s 2023-09-10T04:23:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:28 smithi098 ceph-mon[103124]: pgmap v9889: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 90 KiB/s wr, 9 op/s 2023-09-10T04:23:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:28 smithi170 ceph-mon[102832]: pgmap v9889: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 90 KiB/s wr, 9 op/s 2023-09-10T04:23:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:28 smithi082 ceph-mon[98694]: pgmap v9889: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 90 KiB/s wr, 9 op/s 2023-09-10T04:23:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18240.330290 secs 2023-09-10T04:23:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18240.985929 secs 2023-09-10T04:23:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18240.330290 secs 2023-09-10T04:23:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18240.985929 secs 2023-09-10T04:23:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18240.330290 secs 2023-09-10T04:23:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18240.985929 secs 2023-09-10T04:23:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:30 smithi098 ceph-mon[103124]: pgmap v9890: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 90 KiB/s wr, 9 op/s 2023-09-10T04:23:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:30 smithi170 ceph-mon[102832]: pgmap v9890: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 90 KiB/s wr, 9 op/s 2023-09-10T04:23:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:30 smithi082 ceph-mon[98694]: pgmap v9890: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 90 KiB/s wr, 9 op/s 2023-09-10T04:23:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:23:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:23:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:23:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:32 smithi098 ceph-mon[103124]: pgmap v9891: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 224 KiB/s wr, 15 op/s 2023-09-10T04:23:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:23:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:23:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:23:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:32 smithi170 ceph-mon[102832]: pgmap v9891: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 224 KiB/s wr, 15 op/s 2023-09-10T04:23:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:23:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:23:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:23:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:32 smithi082 ceph-mon[98694]: pgmap v9891: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 224 KiB/s wr, 15 op/s 2023-09-10T04:23:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:23:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:23:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:23:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18245.330369 secs 2023-09-10T04:23:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:34 smithi170 ceph-mon[102832]: pgmap v9892: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 224 KiB/s wr, 11 op/s 2023-09-10T04:23:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18245.330369 secs 2023-09-10T04:23:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:34 smithi082 ceph-mon[98694]: pgmap v9892: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 224 KiB/s wr, 11 op/s 2023-09-10T04:23:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18245.330369 secs 2023-09-10T04:23:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:34 smithi098 ceph-mon[103124]: pgmap v9892: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 224 KiB/s wr, 11 op/s 2023-09-10T04:23:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18245.985989 secs 2023-09-10T04:23:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18245.985989 secs 2023-09-10T04:23:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18245.985989 secs 2023-09-10T04:23:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:36 smithi170 ceph-mon[102832]: pgmap v9893: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 357 KiB/s wr, 17 op/s 2023-09-10T04:23:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:36 smithi082 ceph-mon[98694]: pgmap v9893: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 357 KiB/s wr, 17 op/s 2023-09-10T04:23:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:36 smithi098 ceph-mon[103124]: pgmap v9893: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 357 KiB/s wr, 17 op/s 2023-09-10T04:23:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:38 smithi098 ceph-mon[103124]: pgmap v9894: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T04:23:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:38 smithi170 ceph-mon[102832]: pgmap v9894: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T04:23:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:38 smithi082 ceph-mon[98694]: pgmap v9894: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T04:23:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18250.330453 secs 2023-09-10T04:23:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18250.986051 secs 2023-09-10T04:23:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18250.330453 secs 2023-09-10T04:23:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18250.986051 secs 2023-09-10T04:23:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18250.330453 secs 2023-09-10T04:23:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18250.986051 secs 2023-09-10T04:23:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:40 smithi098 ceph-mon[103124]: pgmap v9895: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T04:23:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:40 smithi170 ceph-mon[102832]: pgmap v9895: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T04:23:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:40 smithi082 ceph-mon[98694]: pgmap v9895: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T04:23:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:42 smithi098 ceph-mon[103124]: pgmap v9896: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 426 KiB/s wr, 20 op/s 2023-09-10T04:23:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:42 smithi170 ceph-mon[102832]: pgmap v9896: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 426 KiB/s wr, 20 op/s 2023-09-10T04:23:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:42 smithi082 ceph-mon[98694]: pgmap v9896: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 426 KiB/s wr, 20 op/s 2023-09-10T04:23:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18255.330536 secs 2023-09-10T04:23:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:44 smithi170 ceph-mon[102832]: pgmap v9897: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 292 KiB/s wr, 13 op/s 2023-09-10T04:23:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18255.330536 secs 2023-09-10T04:23:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:44 smithi082 ceph-mon[98694]: pgmap v9897: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 292 KiB/s wr, 13 op/s 2023-09-10T04:23:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18255.330536 secs 2023-09-10T04:23:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:44 smithi098 ceph-mon[103124]: pgmap v9897: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 292 KiB/s wr, 13 op/s 2023-09-10T04:23:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18255.986117 secs 2023-09-10T04:23:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18255.986117 secs 2023-09-10T04:23:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18255.986117 secs 2023-09-10T04:23:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:46 smithi170 ceph-mon[102832]: pgmap v9898: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 411 KiB/s wr, 19 op/s 2023-09-10T04:23:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:46 smithi082 ceph-mon[98694]: pgmap v9898: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 411 KiB/s wr, 19 op/s 2023-09-10T04:23:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:46 smithi098 ceph-mon[103124]: pgmap v9898: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 411 KiB/s wr, 19 op/s 2023-09-10T04:23:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:48 smithi098 ceph-mon[103124]: pgmap v9899: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T04:23:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:48 smithi170 ceph-mon[102832]: pgmap v9899: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T04:23:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:48 smithi082 ceph-mon[98694]: pgmap v9899: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T04:23:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18260.330594 secs 2023-09-10T04:23:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18260.986180 secs 2023-09-10T04:23:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18260.330594 secs 2023-09-10T04:23:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18260.986180 secs 2023-09-10T04:23:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18260.330594 secs 2023-09-10T04:23:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18260.986180 secs 2023-09-10T04:23:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:50 smithi098 ceph-mon[103124]: pgmap v9900: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T04:23:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:50 smithi170 ceph-mon[102832]: pgmap v9900: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T04:23:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:50 smithi082 ceph-mon[98694]: pgmap v9900: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T04:23:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:52 smithi170 ceph-mon[102832]: pgmap v9901: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 440 KiB/s wr, 18 op/s 2023-09-10T04:23:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:52 smithi082 ceph-mon[98694]: pgmap v9901: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 440 KiB/s wr, 18 op/s 2023-09-10T04:23:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:52 smithi098 ceph-mon[103124]: pgmap v9901: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 440 KiB/s wr, 18 op/s 2023-09-10T04:23:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18264.157482 secs 2023-09-10T04:23:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18264.157482 secs 2023-09-10T04:23:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18264.157482 secs 2023-09-10T04:23:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18265.330655 secs 2023-09-10T04:23:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:54 smithi170 ceph-mon[102832]: pgmap v9902: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 281 KiB/s wr, 11 op/s 2023-09-10T04:23:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18265.986264 secs 2023-09-10T04:23:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18265.330655 secs 2023-09-10T04:23:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:54 smithi082 ceph-mon[98694]: pgmap v9902: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 281 KiB/s wr, 11 op/s 2023-09-10T04:23:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18265.986264 secs 2023-09-10T04:23:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18265.330655 secs 2023-09-10T04:23:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:54 smithi098 ceph-mon[103124]: pgmap v9902: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 281 KiB/s wr, 11 op/s 2023-09-10T04:23:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18265.986264 secs 2023-09-10T04:23:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:23:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:23:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:23:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:23:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:23:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:23:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:23:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:23:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:23:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:56 smithi170 ceph-mon[102832]: pgmap v9903: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 341 KiB/s wr, 17 op/s 2023-09-10T04:23:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:56 smithi082 ceph-mon[98694]: pgmap v9903: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 341 KiB/s wr, 17 op/s 2023-09-10T04:23:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:56 smithi098 ceph-mon[103124]: pgmap v9903: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 341 KiB/s wr, 17 op/s 2023-09-10T04:23:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18269.157542 secs 2023-09-10T04:23:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18269.157542 secs 2023-09-10T04:23:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18269.157542 secs 2023-09-10T04:23:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:58 smithi170 ceph-mon[102832]: pgmap v9904: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 222 KiB/s wr, 11 op/s 2023-09-10T04:23:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:58 smithi082 ceph-mon[98694]: pgmap v9904: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 222 KiB/s wr, 11 op/s 2023-09-10T04:23:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:58 smithi098 ceph-mon[103124]: pgmap v9904: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 222 KiB/s wr, 11 op/s 2023-09-10T04:24:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18270.330715 secs 2023-09-10T04:24:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:23:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18270.986327 secs 2023-09-10T04:24:00.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18270.330715 secs 2023-09-10T04:24:00.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:23:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18270.986327 secs 2023-09-10T04:24:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18270.330715 secs 2023-09-10T04:24:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:23:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18270.986327 secs 2023-09-10T04:24:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:00 smithi170 ceph-mon[102832]: pgmap v9905: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 222 KiB/s wr, 11 op/s 2023-09-10T04:24:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:00 smithi082 ceph-mon[98694]: pgmap v9905: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 222 KiB/s wr, 11 op/s 2023-09-10T04:24:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:00 smithi098 ceph-mon[103124]: pgmap v9905: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 222 KiB/s wr, 11 op/s 2023-09-10T04:24:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:02 smithi170 ceph-mon[102832]: pgmap v9906: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 338 KiB/s wr, 22 op/s 2023-09-10T04:24:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18274.157608 secs 2023-09-10T04:24:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:02 smithi082 ceph-mon[98694]: pgmap v9906: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 338 KiB/s wr, 22 op/s 2023-09-10T04:24:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18274.157608 secs 2023-09-10T04:24:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:02 smithi098 ceph-mon[103124]: pgmap v9906: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 338 KiB/s wr, 22 op/s 2023-09-10T04:24:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18274.157608 secs 2023-09-10T04:24:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18275.330776 secs 2023-09-10T04:24:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:04 smithi170 ceph-mon[102832]: pgmap v9907: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 176 KiB/s wr, 17 op/s 2023-09-10T04:24:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18275.330776 secs 2023-09-10T04:24:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:04 smithi082 ceph-mon[98694]: pgmap v9907: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 176 KiB/s wr, 17 op/s 2023-09-10T04:24:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18275.330776 secs 2023-09-10T04:24:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:04 smithi098 ceph-mon[103124]: pgmap v9907: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 176 KiB/s wr, 17 op/s 2023-09-10T04:24:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18275.986390 secs 2023-09-10T04:24:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18275.986390 secs 2023-09-10T04:24:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18275.986390 secs 2023-09-10T04:24:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:06 smithi170 ceph-mon[102832]: pgmap v9908: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 255 KiB/s wr, 30 op/s 2023-09-10T04:24:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:06 smithi082 ceph-mon[98694]: pgmap v9908: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 255 KiB/s wr, 30 op/s 2023-09-10T04:24:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:06 smithi098 ceph-mon[103124]: pgmap v9908: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 255 KiB/s wr, 30 op/s 2023-09-10T04:24:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18279.157707 secs 2023-09-10T04:24:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:08 smithi082 ceph-mon[98694]: osdmap e687: 12 total, 12 up, 12 in 2023-09-10T04:24:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:08 smithi082 ceph-mon[98694]: pgmap v9910: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 234 KiB/s wr, 30 op/s 2023-09-10T04:24:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18279.157707 secs 2023-09-10T04:24:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:08 smithi098 ceph-mon[103124]: osdmap e687: 12 total, 12 up, 12 in 2023-09-10T04:24:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:08 smithi098 ceph-mon[103124]: pgmap v9910: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 234 KiB/s wr, 30 op/s 2023-09-10T04:24:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18279.157707 secs 2023-09-10T04:24:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:08 smithi170 ceph-mon[102832]: osdmap e687: 12 total, 12 up, 12 in 2023-09-10T04:24:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:08 smithi170 ceph-mon[102832]: pgmap v9910: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 234 KiB/s wr, 30 op/s 2023-09-10T04:24:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18280.330861 secs 2023-09-10T04:24:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18280.330861 secs 2023-09-10T04:24:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18280.330861 secs 2023-09-10T04:24:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18280.986451 secs 2023-09-10T04:24:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:10 smithi082 ceph-mon[98694]: pgmap v9911: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 234 KiB/s wr, 30 op/s 2023-09-10T04:24:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18280.986451 secs 2023-09-10T04:24:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:10 smithi098 ceph-mon[103124]: pgmap v9911: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 234 KiB/s wr, 30 op/s 2023-09-10T04:24:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18280.986451 secs 2023-09-10T04:24:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:10 smithi170 ceph-mon[102832]: pgmap v9911: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 234 KiB/s wr, 30 op/s 2023-09-10T04:24:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:12 smithi098 ceph-mon[103124]: pgmap v9912: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 95 KiB/s wr, 22 op/s 2023-09-10T04:24:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:12 smithi170 ceph-mon[102832]: pgmap v9912: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 95 KiB/s wr, 22 op/s 2023-09-10T04:24:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:12 smithi082 ceph-mon[98694]: pgmap v9912: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 95 KiB/s wr, 22 op/s 2023-09-10T04:24:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18284.157770 secs 2023-09-10T04:24:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:13 smithi098 ceph-mon[103124]: osdmap e688: 12 total, 12 up, 12 in 2023-09-10T04:24:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18284.157770 secs 2023-09-10T04:24:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:13 smithi170 ceph-mon[102832]: osdmap e688: 12 total, 12 up, 12 in 2023-09-10T04:24:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18284.157770 secs 2023-09-10T04:24:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:13 smithi082 ceph-mon[98694]: osdmap e688: 12 total, 12 up, 12 in 2023-09-10T04:24:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18285.330946 secs 2023-09-10T04:24:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:14 smithi098 ceph-mon[103124]: pgmap v9914: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 6 op/s 2023-09-10T04:24:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18285.986485 secs 2023-09-10T04:24:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18285.330946 secs 2023-09-10T04:24:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:14 smithi170 ceph-mon[102832]: pgmap v9914: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 6 op/s 2023-09-10T04:24:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18285.986485 secs 2023-09-10T04:24:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18285.330946 secs 2023-09-10T04:24:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:14 smithi082 ceph-mon[98694]: pgmap v9914: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 6 op/s 2023-09-10T04:24:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18285.986485 secs 2023-09-10T04:24:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:24:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:24:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:24:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:24:16.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:24:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:24:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:24:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:24:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:24:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:24:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:24:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:24:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:24:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:24:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:24:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:16 smithi098 ceph-mon[103124]: pgmap v9915: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 149 KiB/s wr, 14 op/s 2023-09-10T04:24:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:24:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:24:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:24:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:16 smithi170 ceph-mon[102832]: pgmap v9915: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 149 KiB/s wr, 14 op/s 2023-09-10T04:24:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:24:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:24:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:24:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:16 smithi082 ceph-mon[98694]: pgmap v9915: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 149 KiB/s wr, 14 op/s 2023-09-10T04:24:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:24:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:24:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:24:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18289.157831 secs 2023-09-10T04:24:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:18 smithi170 ceph-mon[102832]: pgmap v9916: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T04:24:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18289.157831 secs 2023-09-10T04:24:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:18 smithi082 ceph-mon[98694]: pgmap v9916: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T04:24:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18289.157831 secs 2023-09-10T04:24:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:18 smithi098 ceph-mon[103124]: pgmap v9916: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T04:24:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18290.331003 secs 2023-09-10T04:24:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18290.986547 secs 2023-09-10T04:24:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18290.331003 secs 2023-09-10T04:24:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18290.986547 secs 2023-09-10T04:24:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18290.331003 secs 2023-09-10T04:24:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18290.986547 secs 2023-09-10T04:24:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:20 smithi170 ceph-mon[102832]: pgmap v9917: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T04:24:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:20 smithi082 ceph-mon[98694]: pgmap v9917: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T04:24:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:20 smithi098 ceph-mon[103124]: pgmap v9917: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T04:24:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:22 smithi170 ceph-mon[102832]: pgmap v9918: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T04:24:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:22 smithi082 ceph-mon[98694]: pgmap v9918: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T04:24:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:22 smithi098 ceph-mon[103124]: pgmap v9918: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T04:24:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18294.157930 secs 2023-09-10T04:24:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18294.157930 secs 2023-09-10T04:24:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18294.157930 secs 2023-09-10T04:24:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18295.331101 secs 2023-09-10T04:24:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:24 smithi170 ceph-mon[102832]: pgmap v9919: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T04:24:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18295.986609 secs 2023-09-10T04:24:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18295.331101 secs 2023-09-10T04:24:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:24 smithi082 ceph-mon[98694]: pgmap v9919: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T04:24:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18295.986609 secs 2023-09-10T04:24:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18295.331101 secs 2023-09-10T04:24:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:24 smithi098 ceph-mon[103124]: pgmap v9919: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T04:24:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18295.986609 secs 2023-09-10T04:24:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:26 smithi170 ceph-mon[102832]: pgmap v9920: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 195 KiB/s wr, 14 op/s 2023-09-10T04:24:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:26 smithi082 ceph-mon[98694]: pgmap v9920: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 195 KiB/s wr, 14 op/s 2023-09-10T04:24:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:26 smithi098 ceph-mon[103124]: pgmap v9920: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 195 KiB/s wr, 14 op/s 2023-09-10T04:24:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:28 smithi170 ceph-mon[102832]: pgmap v9921: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 95 KiB/s wr, 9 op/s 2023-09-10T04:24:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:28 smithi082 ceph-mon[98694]: pgmap v9921: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 95 KiB/s wr, 9 op/s 2023-09-10T04:24:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:28 smithi098 ceph-mon[103124]: pgmap v9921: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 95 KiB/s wr, 9 op/s 2023-09-10T04:24:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18300.331159 secs 2023-09-10T04:24:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18300.986675 secs 2023-09-10T04:24:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18300.331159 secs 2023-09-10T04:24:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18300.986675 secs 2023-09-10T04:24:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18300.331159 secs 2023-09-10T04:24:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18300.986675 secs 2023-09-10T04:24:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:30 smithi170 ceph-mon[102832]: pgmap v9922: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 95 KiB/s wr, 9 op/s 2023-09-10T04:24:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:30 smithi082 ceph-mon[98694]: pgmap v9922: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 95 KiB/s wr, 9 op/s 2023-09-10T04:24:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:30 smithi098 ceph-mon[103124]: pgmap v9922: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 95 KiB/s wr, 9 op/s 2023-09-10T04:24:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:24:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:24:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:24:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:32 smithi170 ceph-mon[102832]: pgmap v9923: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 225 KiB/s wr, 15 op/s 2023-09-10T04:24:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:24:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:24:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:24:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:32 smithi082 ceph-mon[98694]: pgmap v9923: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 225 KiB/s wr, 15 op/s 2023-09-10T04:24:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:24:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:24:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:24:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:32 smithi098 ceph-mon[103124]: pgmap v9923: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 225 KiB/s wr, 15 op/s 2023-09-10T04:24:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:24:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:24:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:24:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18305.331242 secs 2023-09-10T04:24:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:34 smithi170 ceph-mon[102832]: pgmap v9924: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 225 KiB/s wr, 11 op/s 2023-09-10T04:24:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18305.331242 secs 2023-09-10T04:24:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:34 smithi082 ceph-mon[98694]: pgmap v9924: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 225 KiB/s wr, 11 op/s 2023-09-10T04:24:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18305.331242 secs 2023-09-10T04:24:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:34 smithi098 ceph-mon[103124]: pgmap v9924: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 225 KiB/s wr, 11 op/s 2023-09-10T04:24:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18305.986734 secs 2023-09-10T04:24:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18305.986734 secs 2023-09-10T04:24:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18305.986734 secs 2023-09-10T04:24:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:36 smithi170 ceph-mon[102832]: pgmap v9925: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 359 KiB/s wr, 17 op/s 2023-09-10T04:24:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:36 smithi082 ceph-mon[98694]: pgmap v9925: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 359 KiB/s wr, 17 op/s 2023-09-10T04:24:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:36 smithi098 ceph-mon[103124]: pgmap v9925: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 359 KiB/s wr, 17 op/s 2023-09-10T04:24:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:38 smithi098 ceph-mon[103124]: pgmap v9926: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 264 KiB/s wr, 11 op/s 2023-09-10T04:24:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:38 smithi170 ceph-mon[102832]: pgmap v9926: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 264 KiB/s wr, 11 op/s 2023-09-10T04:24:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:38 smithi082 ceph-mon[98694]: pgmap v9926: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 264 KiB/s wr, 11 op/s 2023-09-10T04:24:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18310.331301 secs 2023-09-10T04:24:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18310.986795 secs 2023-09-10T04:24:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18310.331301 secs 2023-09-10T04:24:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18310.986795 secs 2023-09-10T04:24:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18310.331301 secs 2023-09-10T04:24:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18310.986795 secs 2023-09-10T04:24:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:40 smithi170 ceph-mon[102832]: pgmap v9927: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 264 KiB/s wr, 11 op/s 2023-09-10T04:24:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:40 smithi082 ceph-mon[98694]: pgmap v9927: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 264 KiB/s wr, 11 op/s 2023-09-10T04:24:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:40 smithi098 ceph-mon[103124]: pgmap v9927: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 264 KiB/s wr, 11 op/s 2023-09-10T04:24:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:42 smithi170 ceph-mon[102832]: pgmap v9928: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 423 KiB/s wr, 18 op/s 2023-09-10T04:24:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:42 smithi082 ceph-mon[98694]: pgmap v9928: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 423 KiB/s wr, 18 op/s 2023-09-10T04:24:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:42 smithi098 ceph-mon[103124]: pgmap v9928: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 423 KiB/s wr, 18 op/s 2023-09-10T04:24:44.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18315.331385 secs 2023-09-10T04:24:44.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:44 smithi170 ceph-mon[102832]: pgmap v9929: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 12 op/s 2023-09-10T04:24:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18315.331385 secs 2023-09-10T04:24:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:44 smithi082 ceph-mon[98694]: pgmap v9929: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 12 op/s 2023-09-10T04:24:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18315.331385 secs 2023-09-10T04:24:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:44 smithi098 ceph-mon[103124]: pgmap v9929: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 12 op/s 2023-09-10T04:24:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18315.986855 secs 2023-09-10T04:24:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18315.986855 secs 2023-09-10T04:24:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18315.986855 secs 2023-09-10T04:24:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:46 smithi170 ceph-mon[102832]: pgmap v9930: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 413 KiB/s wr, 18 op/s 2023-09-10T04:24:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:46 smithi082 ceph-mon[98694]: pgmap v9930: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 413 KiB/s wr, 18 op/s 2023-09-10T04:24:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:46 smithi098 ceph-mon[103124]: pgmap v9930: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 413 KiB/s wr, 18 op/s 2023-09-10T04:24:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:48 smithi170 ceph-mon[102832]: pgmap v9931: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T04:24:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:48 smithi082 ceph-mon[98694]: pgmap v9931: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T04:24:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:48 smithi098 ceph-mon[103124]: pgmap v9931: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T04:24:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18320.331468 secs 2023-09-10T04:24:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18320.986935 secs 2023-09-10T04:24:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18320.331468 secs 2023-09-10T04:24:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18320.986935 secs 2023-09-10T04:24:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18320.331468 secs 2023-09-10T04:24:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18320.986935 secs 2023-09-10T04:24:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:50 smithi170 ceph-mon[102832]: pgmap v9932: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T04:24:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:50 smithi082 ceph-mon[98694]: pgmap v9932: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T04:24:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:50 smithi098 ceph-mon[103124]: pgmap v9932: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T04:24:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:52 smithi170 ceph-mon[102832]: pgmap v9933: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 450 KiB/s wr, 18 op/s 2023-09-10T04:24:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:52 smithi082 ceph-mon[98694]: pgmap v9933: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 450 KiB/s wr, 18 op/s 2023-09-10T04:24:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:52 smithi098 ceph-mon[103124]: pgmap v9933: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 450 KiB/s wr, 18 op/s 2023-09-10T04:24:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18324.158346 secs 2023-09-10T04:24:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18324.158346 secs 2023-09-10T04:24:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18324.158346 secs 2023-09-10T04:24:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18325.331498 secs 2023-09-10T04:24:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:54 smithi170 ceph-mon[102832]: pgmap v9934: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 291 KiB/s wr, 11 op/s 2023-09-10T04:24:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18325.986996 secs 2023-09-10T04:24:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18325.331498 secs 2023-09-10T04:24:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:54 smithi082 ceph-mon[98694]: pgmap v9934: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 291 KiB/s wr, 11 op/s 2023-09-10T04:24:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18325.986996 secs 2023-09-10T04:24:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18325.331498 secs 2023-09-10T04:24:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:54 smithi098 ceph-mon[103124]: pgmap v9934: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 291 KiB/s wr, 11 op/s 2023-09-10T04:24:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18325.986996 secs 2023-09-10T04:24:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:24:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:24:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:24:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:24:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:24:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:24:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:24:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:24:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:24:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:56 smithi170 ceph-mon[102832]: pgmap v9935: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 351 KiB/s wr, 17 op/s 2023-09-10T04:24:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:56 smithi082 ceph-mon[98694]: pgmap v9935: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 351 KiB/s wr, 17 op/s 2023-09-10T04:24:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:56 smithi098 ceph-mon[103124]: pgmap v9935: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 351 KiB/s wr, 17 op/s 2023-09-10T04:24:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18329.158406 secs 2023-09-10T04:24:58.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18329.158406 secs 2023-09-10T04:24:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18329.158406 secs 2023-09-10T04:24:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:58 smithi170 ceph-mon[102832]: pgmap v9936: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T04:24:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:58 smithi082 ceph-mon[98694]: pgmap v9936: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T04:24:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:58 smithi098 ceph-mon[103124]: pgmap v9936: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T04:25:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18330.331558 secs 2023-09-10T04:25:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:24:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18330.987057 secs 2023-09-10T04:25:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18330.331558 secs 2023-09-10T04:25:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:24:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18330.987057 secs 2023-09-10T04:25:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18330.331558 secs 2023-09-10T04:25:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:24:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18330.987057 secs 2023-09-10T04:25:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:00 smithi170 ceph-mon[102832]: pgmap v9937: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T04:25:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:00 smithi082 ceph-mon[98694]: pgmap v9937: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T04:25:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:00 smithi098 ceph-mon[103124]: pgmap v9937: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T04:25:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:02 smithi170 ceph-mon[102832]: pgmap v9938: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 329 KiB/s wr, 17 op/s 2023-09-10T04:25:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18334.158470 secs 2023-09-10T04:25:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:02 smithi082 ceph-mon[98694]: pgmap v9938: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 329 KiB/s wr, 17 op/s 2023-09-10T04:25:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18334.158470 secs 2023-09-10T04:25:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:02 smithi098 ceph-mon[103124]: pgmap v9938: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 329 KiB/s wr, 17 op/s 2023-09-10T04:25:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18334.158470 secs 2023-09-10T04:25:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18335.331615 secs 2023-09-10T04:25:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:04 smithi170 ceph-mon[102832]: pgmap v9939: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T04:25:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18335.331615 secs 2023-09-10T04:25:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:04 smithi082 ceph-mon[98694]: pgmap v9939: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T04:25:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18335.331615 secs 2023-09-10T04:25:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:04 smithi098 ceph-mon[103124]: pgmap v9939: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T04:25:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18335.987120 secs 2023-09-10T04:25:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18335.987120 secs 2023-09-10T04:25:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18335.987120 secs 2023-09-10T04:25:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:06 smithi170 ceph-mon[102832]: pgmap v9940: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 261 KiB/s wr, 31 op/s 2023-09-10T04:25:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:06 smithi082 ceph-mon[98694]: pgmap v9940: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 261 KiB/s wr, 31 op/s 2023-09-10T04:25:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:06 smithi098 ceph-mon[103124]: pgmap v9940: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 261 KiB/s wr, 31 op/s 2023-09-10T04:25:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18339.158528 secs 2023-09-10T04:25:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:08 smithi082 ceph-mon[98694]: osdmap e689: 12 total, 12 up, 12 in 2023-09-10T04:25:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:08 smithi082 ceph-mon[98694]: pgmap v9942: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 241 KiB/s wr, 30 op/s 2023-09-10T04:25:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18339.158528 secs 2023-09-10T04:25:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:08 smithi098 ceph-mon[103124]: osdmap e689: 12 total, 12 up, 12 in 2023-09-10T04:25:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:08 smithi098 ceph-mon[103124]: pgmap v9942: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 241 KiB/s wr, 30 op/s 2023-09-10T04:25:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18339.158528 secs 2023-09-10T04:25:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:08 smithi170 ceph-mon[102832]: osdmap e689: 12 total, 12 up, 12 in 2023-09-10T04:25:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:08 smithi170 ceph-mon[102832]: pgmap v9942: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 241 KiB/s wr, 30 op/s 2023-09-10T04:25:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18340.331677 secs 2023-09-10T04:25:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18340.331677 secs 2023-09-10T04:25:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18340.331677 secs 2023-09-10T04:25:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18340.987179 secs 2023-09-10T04:25:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:10 smithi082 ceph-mon[98694]: pgmap v9943: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 241 KiB/s wr, 30 op/s 2023-09-10T04:25:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18340.987179 secs 2023-09-10T04:25:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:10 smithi098 ceph-mon[103124]: pgmap v9943: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 241 KiB/s wr, 30 op/s 2023-09-10T04:25:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18340.987179 secs 2023-09-10T04:25:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:10 smithi170 ceph-mon[102832]: pgmap v9943: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 241 KiB/s wr, 30 op/s 2023-09-10T04:25:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:12 smithi170 ceph-mon[102832]: pgmap v9944: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 124 KiB/s wr, 29 op/s 2023-09-10T04:25:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:12 smithi082 ceph-mon[98694]: pgmap v9944: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 124 KiB/s wr, 29 op/s 2023-09-10T04:25:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:12 smithi098 ceph-mon[103124]: pgmap v9944: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 124 KiB/s wr, 29 op/s 2023-09-10T04:25:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18344.158591 secs 2023-09-10T04:25:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:13 smithi170 ceph-mon[102832]: osdmap e690: 12 total, 12 up, 12 in 2023-09-10T04:25:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18344.158591 secs 2023-09-10T04:25:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:13 smithi082 ceph-mon[98694]: osdmap e690: 12 total, 12 up, 12 in 2023-09-10T04:25:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18344.158591 secs 2023-09-10T04:25:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:13 smithi098 ceph-mon[103124]: osdmap e690: 12 total, 12 up, 12 in 2023-09-10T04:25:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18345.331736 secs 2023-09-10T04:25:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:14 smithi170 ceph-mon[102832]: pgmap v9946: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:25:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18345.987241 secs 2023-09-10T04:25:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18345.331736 secs 2023-09-10T04:25:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:14 smithi082 ceph-mon[98694]: pgmap v9946: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:25:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18345.987241 secs 2023-09-10T04:25:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18345.331736 secs 2023-09-10T04:25:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:14 smithi098 ceph-mon[103124]: pgmap v9946: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:25:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18345.987241 secs 2023-09-10T04:25:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:16 smithi170 ceph-mon[102832]: pgmap v9947: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 14 op/s 2023-09-10T04:25:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:25:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:16 smithi082 ceph-mon[98694]: pgmap v9947: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 14 op/s 2023-09-10T04:25:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:25:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:16 smithi098 ceph-mon[103124]: pgmap v9947: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 14 op/s 2023-09-10T04:25:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:25:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:25:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:25:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:25:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:25:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:25:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:25:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:25:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:25:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:25:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18349.158688 secs 2023-09-10T04:25:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:18 smithi170 ceph-mon[102832]: pgmap v9948: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T04:25:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18349.158688 secs 2023-09-10T04:25:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:18 smithi082 ceph-mon[98694]: pgmap v9948: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T04:25:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18349.158688 secs 2023-09-10T04:25:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:18 smithi098 ceph-mon[103124]: pgmap v9948: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T04:25:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18350.331834 secs 2023-09-10T04:25:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18350.987303 secs 2023-09-10T04:25:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18350.331834 secs 2023-09-10T04:25:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18350.987303 secs 2023-09-10T04:25:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18350.331834 secs 2023-09-10T04:25:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18350.987303 secs 2023-09-10T04:25:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:20 smithi170 ceph-mon[102832]: pgmap v9949: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T04:25:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:20 smithi082 ceph-mon[98694]: pgmap v9949: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T04:25:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:20 smithi098 ceph-mon[103124]: pgmap v9949: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T04:25:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:22 smithi170 ceph-mon[102832]: pgmap v9950: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T04:25:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:22 smithi082 ceph-mon[98694]: pgmap v9950: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T04:25:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:22 smithi098 ceph-mon[103124]: pgmap v9950: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T04:25:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18354.158748 secs 2023-09-10T04:25:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18354.158748 secs 2023-09-10T04:25:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18354.158748 secs 2023-09-10T04:25:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18355.331941 secs 2023-09-10T04:25:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:24 smithi170 ceph-mon[102832]: pgmap v9951: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T04:25:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18355.987366 secs 2023-09-10T04:25:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18355.331941 secs 2023-09-10T04:25:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:24 smithi082 ceph-mon[98694]: pgmap v9951: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T04:25:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18355.987366 secs 2023-09-10T04:25:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18355.331941 secs 2023-09-10T04:25:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:24 smithi098 ceph-mon[103124]: pgmap v9951: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T04:25:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18355.987366 secs 2023-09-10T04:25:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:26 smithi170 ceph-mon[102832]: pgmap v9952: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 196 KiB/s wr, 14 op/s 2023-09-10T04:25:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:26 smithi082 ceph-mon[98694]: pgmap v9952: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 196 KiB/s wr, 14 op/s 2023-09-10T04:25:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:26 smithi098 ceph-mon[103124]: pgmap v9952: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 196 KiB/s wr, 14 op/s 2023-09-10T04:25:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:28 smithi170 ceph-mon[102832]: pgmap v9953: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 96 KiB/s wr, 9 op/s 2023-09-10T04:25:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:28 smithi082 ceph-mon[98694]: pgmap v9953: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 96 KiB/s wr, 9 op/s 2023-09-10T04:25:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:28 smithi098 ceph-mon[103124]: pgmap v9953: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 96 KiB/s wr, 9 op/s 2023-09-10T04:25:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18360.331993 secs 2023-09-10T04:25:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18360.987426 secs 2023-09-10T04:25:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18360.331993 secs 2023-09-10T04:25:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18360.987426 secs 2023-09-10T04:25:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18360.331993 secs 2023-09-10T04:25:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18360.987426 secs 2023-09-10T04:25:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:30 smithi170 ceph-mon[102832]: pgmap v9954: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 96 KiB/s wr, 9 op/s 2023-09-10T04:25:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:30 smithi082 ceph-mon[98694]: pgmap v9954: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 96 KiB/s wr, 9 op/s 2023-09-10T04:25:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:30 smithi098 ceph-mon[103124]: pgmap v9954: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 96 KiB/s wr, 9 op/s 2023-09-10T04:25:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:25:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:25:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:25:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:32 smithi170 ceph-mon[102832]: pgmap v9955: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 226 KiB/s wr, 16 op/s 2023-09-10T04:25:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:25:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:25:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:25:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:32 smithi082 ceph-mon[98694]: pgmap v9955: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 226 KiB/s wr, 16 op/s 2023-09-10T04:25:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:25:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:25:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:25:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:32 smithi098 ceph-mon[103124]: pgmap v9955: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 226 KiB/s wr, 16 op/s 2023-09-10T04:25:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:25:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:25:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:25:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18365.332082 secs 2023-09-10T04:25:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:34 smithi082 ceph-mon[98694]: pgmap v9956: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 226 KiB/s wr, 11 op/s 2023-09-10T04:25:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18365.332082 secs 2023-09-10T04:25:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:34 smithi098 ceph-mon[103124]: pgmap v9956: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 226 KiB/s wr, 11 op/s 2023-09-10T04:25:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18365.332082 secs 2023-09-10T04:25:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:34 smithi170 ceph-mon[102832]: pgmap v9956: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 226 KiB/s wr, 11 op/s 2023-09-10T04:25:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18365.987484 secs 2023-09-10T04:25:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18365.987484 secs 2023-09-10T04:25:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18365.987484 secs 2023-09-10T04:25:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:36 smithi082 ceph-mon[98694]: pgmap v9957: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 360 KiB/s wr, 17 op/s 2023-09-10T04:25:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:36 smithi098 ceph-mon[103124]: pgmap v9957: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 360 KiB/s wr, 17 op/s 2023-09-10T04:25:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:36 smithi170 ceph-mon[102832]: pgmap v9957: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 360 KiB/s wr, 17 op/s 2023-09-10T04:25:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:38 smithi170 ceph-mon[102832]: pgmap v9958: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T04:25:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:38 smithi082 ceph-mon[98694]: pgmap v9958: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T04:25:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:38 smithi098 ceph-mon[103124]: pgmap v9958: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T04:25:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18370.332139 secs 2023-09-10T04:25:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18370.987544 secs 2023-09-10T04:25:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18370.332139 secs 2023-09-10T04:25:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18370.987544 secs 2023-09-10T04:25:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18370.332139 secs 2023-09-10T04:25:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18370.987544 secs 2023-09-10T04:25:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:40 smithi170 ceph-mon[102832]: pgmap v9959: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T04:25:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:40 smithi082 ceph-mon[98694]: pgmap v9959: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T04:25:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:40 smithi098 ceph-mon[103124]: pgmap v9959: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T04:25:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:42 smithi170 ceph-mon[102832]: pgmap v9960: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 424 KiB/s wr, 19 op/s 2023-09-10T04:25:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:42 smithi082 ceph-mon[98694]: pgmap v9960: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 424 KiB/s wr, 19 op/s 2023-09-10T04:25:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:42 smithi098 ceph-mon[103124]: pgmap v9960: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 424 KiB/s wr, 19 op/s 2023-09-10T04:25:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18375.332223 secs 2023-09-10T04:25:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:44 smithi082 ceph-mon[98694]: pgmap v9961: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T04:25:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18375.332223 secs 2023-09-10T04:25:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:44 smithi098 ceph-mon[103124]: pgmap v9961: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T04:25:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18375.332223 secs 2023-09-10T04:25:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:44 smithi170 ceph-mon[102832]: pgmap v9961: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T04:25:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18375.987603 secs 2023-09-10T04:25:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18375.987603 secs 2023-09-10T04:25:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18375.987603 secs 2023-09-10T04:25:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:46 smithi082 ceph-mon[98694]: pgmap v9962: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 415 KiB/s wr, 18 op/s 2023-09-10T04:25:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:46 smithi098 ceph-mon[103124]: pgmap v9962: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 415 KiB/s wr, 18 op/s 2023-09-10T04:25:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:46 smithi170 ceph-mon[102832]: pgmap v9962: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 415 KiB/s wr, 18 op/s 2023-09-10T04:25:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:48 smithi170 ceph-mon[102832]: pgmap v9963: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T04:25:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:48 smithi082 ceph-mon[98694]: pgmap v9963: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T04:25:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:48 smithi098 ceph-mon[103124]: pgmap v9963: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T04:25:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18380.332305 secs 2023-09-10T04:25:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18380.987666 secs 2023-09-10T04:25:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18380.332305 secs 2023-09-10T04:25:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18380.987666 secs 2023-09-10T04:25:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18380.332305 secs 2023-09-10T04:25:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18380.987666 secs 2023-09-10T04:25:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:50 smithi170 ceph-mon[102832]: pgmap v9964: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T04:25:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:50 smithi082 ceph-mon[98694]: pgmap v9964: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T04:25:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:50 smithi098 ceph-mon[103124]: pgmap v9964: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T04:25:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:52 smithi170 ceph-mon[102832]: pgmap v9965: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 444 KiB/s wr, 19 op/s 2023-09-10T04:25:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:52 smithi082 ceph-mon[98694]: pgmap v9965: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 444 KiB/s wr, 19 op/s 2023-09-10T04:25:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:52 smithi098 ceph-mon[103124]: pgmap v9965: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 444 KiB/s wr, 19 op/s 2023-09-10T04:25:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18384.159097 secs 2023-09-10T04:25:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18384.159097 secs 2023-09-10T04:25:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18384.159097 secs 2023-09-10T04:25:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18385.332390 secs 2023-09-10T04:25:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:54 smithi170 ceph-mon[102832]: pgmap v9966: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-09-10T04:25:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18385.987728 secs 2023-09-10T04:25:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18385.332390 secs 2023-09-10T04:25:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:54 smithi082 ceph-mon[98694]: pgmap v9966: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-09-10T04:25:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18385.987728 secs 2023-09-10T04:25:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18385.332390 secs 2023-09-10T04:25:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:54 smithi098 ceph-mon[103124]: pgmap v9966: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-09-10T04:25:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18385.987728 secs 2023-09-10T04:25:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:25:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:25:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:25:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:25:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:25:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:25:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:25:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:25:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:25:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:56 smithi170 ceph-mon[102832]: pgmap v9967: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 345 KiB/s wr, 18 op/s 2023-09-10T04:25:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:56 smithi082 ceph-mon[98694]: pgmap v9967: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 345 KiB/s wr, 18 op/s 2023-09-10T04:25:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:56 smithi098 ceph-mon[103124]: pgmap v9967: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 345 KiB/s wr, 18 op/s 2023-09-10T04:25:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18389.159178 secs 2023-09-10T04:25:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18389.159178 secs 2023-09-10T04:25:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18389.159178 secs 2023-09-10T04:25:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:58 smithi170 ceph-mon[102832]: pgmap v9968: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 224 KiB/s wr, 12 op/s 2023-09-10T04:25:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:58 smithi082 ceph-mon[98694]: pgmap v9968: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 224 KiB/s wr, 12 op/s 2023-09-10T04:25:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:58 smithi098 ceph-mon[103124]: pgmap v9968: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 224 KiB/s wr, 12 op/s 2023-09-10T04:26:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18390.332448 secs 2023-09-10T04:26:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:25:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18390.987789 secs 2023-09-10T04:26:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18390.332448 secs 2023-09-10T04:26:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:25:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18390.987789 secs 2023-09-10T04:26:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18390.332448 secs 2023-09-10T04:26:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:25:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18390.987789 secs 2023-09-10T04:26:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:00 smithi170 ceph-mon[102832]: pgmap v9969: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 224 KiB/s wr, 12 op/s 2023-09-10T04:26:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:00 smithi082 ceph-mon[98694]: pgmap v9969: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 224 KiB/s wr, 12 op/s 2023-09-10T04:26:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:00 smithi098 ceph-mon[103124]: pgmap v9969: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 224 KiB/s wr, 12 op/s 2023-09-10T04:26:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:02 smithi170 ceph-mon[102832]: pgmap v9970: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 322 KiB/s wr, 18 op/s 2023-09-10T04:26:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18394.159273 secs 2023-09-10T04:26:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:02 smithi082 ceph-mon[98694]: pgmap v9970: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 322 KiB/s wr, 18 op/s 2023-09-10T04:26:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18394.159273 secs 2023-09-10T04:26:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:02 smithi098 ceph-mon[103124]: pgmap v9970: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 322 KiB/s wr, 18 op/s 2023-09-10T04:26:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18394.159273 secs 2023-09-10T04:26:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18395.332540 secs 2023-09-10T04:26:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:04 smithi082 ceph-mon[98694]: pgmap v9971: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T04:26:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18395.332540 secs 2023-09-10T04:26:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:04 smithi098 ceph-mon[103124]: pgmap v9971: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T04:26:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18395.332540 secs 2023-09-10T04:26:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:04 smithi170 ceph-mon[102832]: pgmap v9971: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T04:26:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18395.987890 secs 2023-09-10T04:26:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18395.987890 secs 2023-09-10T04:26:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18395.987890 secs 2023-09-10T04:26:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:06 smithi082 ceph-mon[98694]: pgmap v9972: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 249 KiB/s wr, 31 op/s 2023-09-10T04:26:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:06 smithi098 ceph-mon[103124]: pgmap v9972: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 249 KiB/s wr, 31 op/s 2023-09-10T04:26:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:06 smithi170 ceph-mon[102832]: pgmap v9972: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 249 KiB/s wr, 31 op/s 2023-09-10T04:26:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18399.159331 secs 2023-09-10T04:26:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:08 smithi098 ceph-mon[103124]: osdmap e691: 12 total, 12 up, 12 in 2023-09-10T04:26:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:08 smithi098 ceph-mon[103124]: pgmap v9974: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 225 KiB/s wr, 30 op/s 2023-09-10T04:26:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18399.159331 secs 2023-09-10T04:26:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:08 smithi170 ceph-mon[102832]: osdmap e691: 12 total, 12 up, 12 in 2023-09-10T04:26:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:08 smithi170 ceph-mon[102832]: pgmap v9974: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 225 KiB/s wr, 30 op/s 2023-09-10T04:26:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18399.159331 secs 2023-09-10T04:26:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:08 smithi082 ceph-mon[98694]: osdmap e691: 12 total, 12 up, 12 in 2023-09-10T04:26:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:08 smithi082 ceph-mon[98694]: pgmap v9974: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 225 KiB/s wr, 30 op/s 2023-09-10T04:26:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18400.332598 secs 2023-09-10T04:26:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18400.987956 secs 2023-09-10T04:26:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18400.332598 secs 2023-09-10T04:26:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18400.987956 secs 2023-09-10T04:26:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18400.332598 secs 2023-09-10T04:26:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18400.987956 secs 2023-09-10T04:26:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:10 smithi098 ceph-mon[103124]: pgmap v9975: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 225 KiB/s wr, 30 op/s 2023-09-10T04:26:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:10 smithi170 ceph-mon[102832]: pgmap v9975: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 225 KiB/s wr, 30 op/s 2023-09-10T04:26:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:10 smithi082 ceph-mon[98694]: pgmap v9975: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 225 KiB/s wr, 30 op/s 2023-09-10T04:26:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:12 smithi170 ceph-mon[102832]: pgmap v9976: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 108 KiB/s wr, 30 op/s 2023-09-10T04:26:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:12 smithi082 ceph-mon[98694]: pgmap v9976: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 108 KiB/s wr, 30 op/s 2023-09-10T04:26:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:12 smithi098 ceph-mon[103124]: pgmap v9976: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 108 KiB/s wr, 30 op/s 2023-09-10T04:26:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18404.159414 secs 2023-09-10T04:26:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:13 smithi170 ceph-mon[102832]: osdmap e692: 12 total, 12 up, 12 in 2023-09-10T04:26:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18404.159414 secs 2023-09-10T04:26:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:13 smithi082 ceph-mon[98694]: osdmap e692: 12 total, 12 up, 12 in 2023-09-10T04:26:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18404.159414 secs 2023-09-10T04:26:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:13 smithi098 ceph-mon[103124]: osdmap e692: 12 total, 12 up, 12 in 2023-09-10T04:26:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18405.332655 secs 2023-09-10T04:26:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:14 smithi170 ceph-mon[102832]: pgmap v9978: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:26:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18405.988021 secs 2023-09-10T04:26:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18405.332655 secs 2023-09-10T04:26:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:14 smithi082 ceph-mon[98694]: pgmap v9978: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:26:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18405.988021 secs 2023-09-10T04:26:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18405.332655 secs 2023-09-10T04:26:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:14 smithi098 ceph-mon[103124]: pgmap v9978: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:26:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18405.988021 secs 2023-09-10T04:26:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:16 smithi170 ceph-mon[102832]: pgmap v9979: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T04:26:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:16 smithi082 ceph-mon[98694]: pgmap v9979: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T04:26:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:16 smithi098 ceph-mon[103124]: pgmap v9979: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T04:26:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:26:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:26:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:26:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:26:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:26:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:26:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:26:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:26:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:26:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:26:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:26:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:26:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18409.159472 secs 2023-09-10T04:26:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:18 smithi170 ceph-mon[102832]: pgmap v9980: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T04:26:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18409.159472 secs 2023-09-10T04:26:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:18 smithi082 ceph-mon[98694]: pgmap v9980: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T04:26:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18409.159472 secs 2023-09-10T04:26:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:18 smithi098 ceph-mon[103124]: pgmap v9980: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T04:26:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18410.332714 secs 2023-09-10T04:26:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18410.988081 secs 2023-09-10T04:26:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18410.332714 secs 2023-09-10T04:26:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18410.988081 secs 2023-09-10T04:26:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18410.332714 secs 2023-09-10T04:26:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18410.988081 secs 2023-09-10T04:26:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:20 smithi170 ceph-mon[102832]: pgmap v9981: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T04:26:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:20 smithi082 ceph-mon[98694]: pgmap v9981: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T04:26:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:20 smithi098 ceph-mon[103124]: pgmap v9981: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T04:26:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:22 smithi170 ceph-mon[102832]: pgmap v9982: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T04:26:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:22 smithi082 ceph-mon[98694]: pgmap v9982: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T04:26:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:22 smithi098 ceph-mon[103124]: pgmap v9982: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T04:26:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18414.159548 secs 2023-09-10T04:26:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18414.159548 secs 2023-09-10T04:26:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18414.159548 secs 2023-09-10T04:26:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18415.332788 secs 2023-09-10T04:26:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18415.988143 secs 2023-09-10T04:26:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:24 smithi170 ceph-mon[102832]: pgmap v9983: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T04:26:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18415.332788 secs 2023-09-10T04:26:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18415.988143 secs 2023-09-10T04:26:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:24 smithi082 ceph-mon[98694]: pgmap v9983: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T04:26:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18415.332788 secs 2023-09-10T04:26:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18415.988143 secs 2023-09-10T04:26:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:24 smithi098 ceph-mon[103124]: pgmap v9983: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T04:26:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:26 smithi170 ceph-mon[102832]: pgmap v9984: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 201 KiB/s wr, 14 op/s 2023-09-10T04:26:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:26 smithi082 ceph-mon[98694]: pgmap v9984: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 201 KiB/s wr, 14 op/s 2023-09-10T04:26:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:26 smithi098 ceph-mon[103124]: pgmap v9984: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 201 KiB/s wr, 14 op/s 2023-09-10T04:26:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:28 smithi170 ceph-mon[102832]: pgmap v9985: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T04:26:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:28 smithi082 ceph-mon[98694]: pgmap v9985: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T04:26:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:28 smithi098 ceph-mon[103124]: pgmap v9985: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T04:26:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18420.332874 secs 2023-09-10T04:26:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18420.988203 secs 2023-09-10T04:26:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18420.332874 secs 2023-09-10T04:26:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18420.988203 secs 2023-09-10T04:26:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18420.332874 secs 2023-09-10T04:26:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18420.988203 secs 2023-09-10T04:26:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:30 smithi170 ceph-mon[102832]: pgmap v9986: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T04:26:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:30 smithi082 ceph-mon[98694]: pgmap v9986: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T04:26:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:30 smithi098 ceph-mon[103124]: pgmap v9986: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T04:26:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:26:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:26:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:26:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:32 smithi170 ceph-mon[102832]: pgmap v9987: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 227 KiB/s wr, 15 op/s 2023-09-10T04:26:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:26:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:26:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:26:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:32 smithi082 ceph-mon[98694]: pgmap v9987: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 227 KiB/s wr, 15 op/s 2023-09-10T04:26:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:26:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:26:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:26:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:32 smithi098 ceph-mon[103124]: pgmap v9987: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 227 KiB/s wr, 15 op/s 2023-09-10T04:26:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:26:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:26:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:26:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18425.332955 secs 2023-09-10T04:26:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:34 smithi082 ceph-mon[98694]: pgmap v9988: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T04:26:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18425.332955 secs 2023-09-10T04:26:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:34 smithi098 ceph-mon[103124]: pgmap v9988: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T04:26:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18425.332955 secs 2023-09-10T04:26:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:34 smithi170 ceph-mon[102832]: pgmap v9988: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T04:26:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18425.988266 secs 2023-09-10T04:26:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18425.988266 secs 2023-09-10T04:26:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18425.988266 secs 2023-09-10T04:26:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:36 smithi082 ceph-mon[98694]: pgmap v9989: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 361 KiB/s wr, 17 op/s 2023-09-10T04:26:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:36 smithi098 ceph-mon[103124]: pgmap v9989: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 361 KiB/s wr, 17 op/s 2023-09-10T04:26:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:36 smithi170 ceph-mon[102832]: pgmap v9989: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 361 KiB/s wr, 17 op/s 2023-09-10T04:26:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:38 smithi170 ceph-mon[102832]: pgmap v9990: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T04:26:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:38 smithi082 ceph-mon[98694]: pgmap v9990: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T04:26:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:38 smithi098 ceph-mon[103124]: pgmap v9990: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T04:26:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18430.333013 secs 2023-09-10T04:26:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18430.988326 secs 2023-09-10T04:26:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18430.333013 secs 2023-09-10T04:26:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18430.988326 secs 2023-09-10T04:26:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18430.333013 secs 2023-09-10T04:26:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18430.988326 secs 2023-09-10T04:26:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:40 smithi170 ceph-mon[102832]: pgmap v9991: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T04:26:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:40 smithi082 ceph-mon[98694]: pgmap v9991: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T04:26:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:40 smithi098 ceph-mon[103124]: pgmap v9991: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T04:26:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:42 smithi170 ceph-mon[102832]: pgmap v9992: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 421 KiB/s wr, 19 op/s 2023-09-10T04:26:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:42 smithi082 ceph-mon[98694]: pgmap v9992: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 421 KiB/s wr, 19 op/s 2023-09-10T04:26:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:42 smithi098 ceph-mon[103124]: pgmap v9992: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 421 KiB/s wr, 19 op/s 2023-09-10T04:26:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18435.333070 secs 2023-09-10T04:26:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:44 smithi082 ceph-mon[98694]: pgmap v9993: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 296 KiB/s wr, 13 op/s 2023-09-10T04:26:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18435.333070 secs 2023-09-10T04:26:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:44 smithi098 ceph-mon[103124]: pgmap v9993: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 296 KiB/s wr, 13 op/s 2023-09-10T04:26:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18435.333070 secs 2023-09-10T04:26:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:44 smithi170 ceph-mon[102832]: pgmap v9993: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 296 KiB/s wr, 13 op/s 2023-09-10T04:26:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18435.988388 secs 2023-09-10T04:26:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18435.988388 secs 2023-09-10T04:26:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18435.988388 secs 2023-09-10T04:26:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:46 smithi082 ceph-mon[98694]: pgmap v9994: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 417 KiB/s wr, 19 op/s 2023-09-10T04:26:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:46 smithi098 ceph-mon[103124]: pgmap v9994: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 417 KiB/s wr, 19 op/s 2023-09-10T04:26:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:46 smithi170 ceph-mon[102832]: pgmap v9994: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 417 KiB/s wr, 19 op/s 2023-09-10T04:26:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:48 smithi170 ceph-mon[102832]: pgmap v9995: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 282 KiB/s wr, 13 op/s 2023-09-10T04:26:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:48 smithi082 ceph-mon[98694]: pgmap v9995: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 282 KiB/s wr, 13 op/s 2023-09-10T04:26:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:48 smithi098 ceph-mon[103124]: pgmap v9995: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 282 KiB/s wr, 13 op/s 2023-09-10T04:26:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18440.333128 secs 2023-09-10T04:26:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18440.988451 secs 2023-09-10T04:26:50.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18440.333128 secs 2023-09-10T04:26:50.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18440.988451 secs 2023-09-10T04:26:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18440.333128 secs 2023-09-10T04:26:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18440.988451 secs 2023-09-10T04:26:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:50 smithi170 ceph-mon[102832]: pgmap v9996: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 282 KiB/s wr, 13 op/s 2023-09-10T04:26:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:50 smithi082 ceph-mon[98694]: pgmap v9996: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 282 KiB/s wr, 13 op/s 2023-09-10T04:26:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:50 smithi098 ceph-mon[103124]: pgmap v9996: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 282 KiB/s wr, 13 op/s 2023-09-10T04:26:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:52 smithi170 ceph-mon[102832]: pgmap v9997: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 18 op/s 2023-09-10T04:26:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:52 smithi082 ceph-mon[98694]: pgmap v9997: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 18 op/s 2023-09-10T04:26:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:52 smithi098 ceph-mon[103124]: pgmap v9997: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 18 op/s 2023-09-10T04:26:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18444.159958 secs 2023-09-10T04:26:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18444.159958 secs 2023-09-10T04:26:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18444.159958 secs 2023-09-10T04:26:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18445.333186 secs 2023-09-10T04:26:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18445.988483 secs 2023-09-10T04:26:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:54 smithi170 ceph-mon[102832]: pgmap v9998: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 285 KiB/s wr, 11 op/s 2023-09-10T04:26:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18445.333186 secs 2023-09-10T04:26:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18445.988483 secs 2023-09-10T04:26:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:54 smithi082 ceph-mon[98694]: pgmap v9998: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 285 KiB/s wr, 11 op/s 2023-09-10T04:26:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18445.333186 secs 2023-09-10T04:26:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18445.988483 secs 2023-09-10T04:26:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:54 smithi098 ceph-mon[103124]: pgmap v9998: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 285 KiB/s wr, 11 op/s 2023-09-10T04:26:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:26:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:26:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:26:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:26:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:26:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:26:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:26:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:26:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:26:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:56 smithi170 ceph-mon[102832]: pgmap v9999: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 346 KiB/s wr, 17 op/s 2023-09-10T04:26:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:56 smithi082 ceph-mon[98694]: pgmap v9999: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 346 KiB/s wr, 17 op/s 2023-09-10T04:26:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:56 smithi098 ceph-mon[103124]: pgmap v9999: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 346 KiB/s wr, 17 op/s 2023-09-10T04:26:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18449.160035 secs 2023-09-10T04:26:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18449.160035 secs 2023-09-10T04:26:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18449.160035 secs 2023-09-10T04:26:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:58 smithi170 ceph-mon[102832]: pgmap v10000: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 224 KiB/s wr, 11 op/s 2023-09-10T04:26:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:58 smithi082 ceph-mon[98694]: pgmap v10000: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 224 KiB/s wr, 11 op/s 2023-09-10T04:26:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:58 smithi098 ceph-mon[103124]: pgmap v10000: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 224 KiB/s wr, 11 op/s 2023-09-10T04:27:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18450.333245 secs 2023-09-10T04:27:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:26:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18450.988544 secs 2023-09-10T04:27:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18450.333245 secs 2023-09-10T04:27:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:26:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18450.988544 secs 2023-09-10T04:27:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18450.333245 secs 2023-09-10T04:27:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:26:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18450.988544 secs 2023-09-10T04:27:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:00 smithi170 ceph-mon[102832]: pgmap v10001: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 224 KiB/s wr, 11 op/s 2023-09-10T04:27:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:00 smithi082 ceph-mon[98694]: pgmap v10001: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 224 KiB/s wr, 11 op/s 2023-09-10T04:27:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:00 smithi098 ceph-mon[103124]: pgmap v10001: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 224 KiB/s wr, 11 op/s 2023-09-10T04:27:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:02 smithi170 ceph-mon[102832]: pgmap v10002: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 323 KiB/s wr, 16 op/s 2023-09-10T04:27:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18454.160096 secs 2023-09-10T04:27:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:02 smithi082 ceph-mon[98694]: pgmap v10002: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 323 KiB/s wr, 16 op/s 2023-09-10T04:27:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18454.160096 secs 2023-09-10T04:27:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:02 smithi098 ceph-mon[103124]: pgmap v10002: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 323 KiB/s wr, 16 op/s 2023-09-10T04:27:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18454.160096 secs 2023-09-10T04:27:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18455.333306 secs 2023-09-10T04:27:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:04 smithi098 ceph-mon[103124]: pgmap v10003: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T04:27:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18455.333306 secs 2023-09-10T04:27:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:04 smithi170 ceph-mon[102832]: pgmap v10003: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T04:27:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18455.333306 secs 2023-09-10T04:27:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:04 smithi082 ceph-mon[98694]: pgmap v10003: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T04:27:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18455.988613 secs 2023-09-10T04:27:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18455.988613 secs 2023-09-10T04:27:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18455.988613 secs 2023-09-10T04:27:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:06 smithi098 ceph-mon[103124]: pgmap v10004: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 263 KiB/s wr, 31 op/s 2023-09-10T04:27:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:06 smithi170 ceph-mon[102832]: pgmap v10004: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 263 KiB/s wr, 31 op/s 2023-09-10T04:27:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:06 smithi082 ceph-mon[98694]: pgmap v10004: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 263 KiB/s wr, 31 op/s 2023-09-10T04:27:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18459.160156 secs 2023-09-10T04:27:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:08 smithi098 ceph-mon[103124]: osdmap e693: 12 total, 12 up, 12 in 2023-09-10T04:27:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:08 smithi098 ceph-mon[103124]: pgmap v10006: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 242 KiB/s wr, 30 op/s 2023-09-10T04:27:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18459.160156 secs 2023-09-10T04:27:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:08 smithi170 ceph-mon[102832]: osdmap e693: 12 total, 12 up, 12 in 2023-09-10T04:27:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:08 smithi170 ceph-mon[102832]: pgmap v10006: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 242 KiB/s wr, 30 op/s 2023-09-10T04:27:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18459.160156 secs 2023-09-10T04:27:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:08 smithi082 ceph-mon[98694]: osdmap e693: 12 total, 12 up, 12 in 2023-09-10T04:27:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:08 smithi082 ceph-mon[98694]: pgmap v10006: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 242 KiB/s wr, 30 op/s 2023-09-10T04:27:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18460.333376 secs 2023-09-10T04:27:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18460.988677 secs 2023-09-10T04:27:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18460.333376 secs 2023-09-10T04:27:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18460.988677 secs 2023-09-10T04:27:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18460.333376 secs 2023-09-10T04:27:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18460.988677 secs 2023-09-10T04:27:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:10 smithi098 ceph-mon[103124]: pgmap v10007: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 242 KiB/s wr, 30 op/s 2023-09-10T04:27:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:10 smithi170 ceph-mon[102832]: pgmap v10007: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 242 KiB/s wr, 30 op/s 2023-09-10T04:27:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:10 smithi082 ceph-mon[98694]: pgmap v10007: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 242 KiB/s wr, 30 op/s 2023-09-10T04:27:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:12 smithi170 ceph-mon[102832]: pgmap v10008: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 125 KiB/s wr, 29 op/s 2023-09-10T04:27:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:12 smithi082 ceph-mon[98694]: pgmap v10008: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 125 KiB/s wr, 29 op/s 2023-09-10T04:27:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:12 smithi098 ceph-mon[103124]: pgmap v10008: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 125 KiB/s wr, 29 op/s 2023-09-10T04:27:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18464.160277 secs 2023-09-10T04:27:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:13 smithi170 ceph-mon[102832]: osdmap e694: 12 total, 12 up, 12 in 2023-09-10T04:27:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18464.160277 secs 2023-09-10T04:27:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:13 smithi082 ceph-mon[98694]: osdmap e694: 12 total, 12 up, 12 in 2023-09-10T04:27:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18464.160277 secs 2023-09-10T04:27:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:13 smithi098 ceph-mon[103124]: osdmap e694: 12 total, 12 up, 12 in 2023-09-10T04:27:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18465.333433 secs 2023-09-10T04:27:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18465.988739 secs 2023-09-10T04:27:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:14 smithi170 ceph-mon[102832]: pgmap v10010: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:27:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18465.333433 secs 2023-09-10T04:27:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18465.988739 secs 2023-09-10T04:27:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:14 smithi082 ceph-mon[98694]: pgmap v10010: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:27:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18465.333433 secs 2023-09-10T04:27:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18465.988739 secs 2023-09-10T04:27:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:14 smithi098 ceph-mon[103124]: pgmap v10010: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:27:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:16 smithi170 ceph-mon[102832]: pgmap v10011: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T04:27:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:16 smithi082 ceph-mon[98694]: pgmap v10011: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T04:27:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:16 smithi098 ceph-mon[103124]: pgmap v10011: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T04:27:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18469.160338 secs 2023-09-10T04:27:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:18 smithi170 ceph-mon[102832]: pgmap v10012: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T04:27:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:27:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:27:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:27:19.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:27:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18469.160338 secs 2023-09-10T04:27:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:18 smithi082 ceph-mon[98694]: pgmap v10012: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T04:27:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:27:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:27:19.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:27:19.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:27:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18469.160338 secs 2023-09-10T04:27:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:18 smithi098 ceph-mon[103124]: pgmap v10012: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T04:27:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:27:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:27:19.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:27:19.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:27:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18470.333502 secs 2023-09-10T04:27:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18470.988805 secs 2023-09-10T04:27:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18470.333502 secs 2023-09-10T04:27:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18470.988805 secs 2023-09-10T04:27:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18470.333502 secs 2023-09-10T04:27:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18470.988805 secs 2023-09-10T04:27:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:20 smithi170 ceph-mon[102832]: pgmap v10013: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T04:27:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:20 smithi082 ceph-mon[98694]: pgmap v10013: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T04:27:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:20 smithi098 ceph-mon[103124]: pgmap v10013: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T04:27:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:22 smithi170 ceph-mon[102832]: pgmap v10014: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T04:27:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:22 smithi082 ceph-mon[98694]: pgmap v10014: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T04:27:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:22 smithi098 ceph-mon[103124]: pgmap v10014: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T04:27:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18474.160429 secs 2023-09-10T04:27:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18474.160429 secs 2023-09-10T04:27:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18474.160429 secs 2023-09-10T04:27:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18475.333614 secs 2023-09-10T04:27:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18475.988867 secs 2023-09-10T04:27:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:24 smithi170 ceph-mon[102832]: pgmap v10015: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T04:27:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18475.333614 secs 2023-09-10T04:27:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18475.988867 secs 2023-09-10T04:27:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:24 smithi082 ceph-mon[98694]: pgmap v10015: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T04:27:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18475.333614 secs 2023-09-10T04:27:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18475.988867 secs 2023-09-10T04:27:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:24 smithi098 ceph-mon[103124]: pgmap v10015: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T04:27:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:26 smithi170 ceph-mon[102832]: pgmap v10016: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 202 KiB/s wr, 14 op/s 2023-09-10T04:27:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:26 smithi082 ceph-mon[98694]: pgmap v10016: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 202 KiB/s wr, 14 op/s 2023-09-10T04:27:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:26 smithi098 ceph-mon[103124]: pgmap v10016: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 202 KiB/s wr, 14 op/s 2023-09-10T04:27:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:28 smithi170 ceph-mon[102832]: pgmap v10017: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T04:27:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:28 smithi082 ceph-mon[98694]: pgmap v10017: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T04:27:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:28 smithi098 ceph-mon[103124]: pgmap v10017: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T04:27:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18480.333666 secs 2023-09-10T04:27:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18480.988930 secs 2023-09-10T04:27:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:29 smithi170 ceph-mon[102832]: Health check failed: 1 MDSs behind on trimming (MDS_TRIM) 2023-09-10T04:27:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:29 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:27:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:29 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:27:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18480.333666 secs 2023-09-10T04:27:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18480.988930 secs 2023-09-10T04:27:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:29 smithi082 ceph-mon[98694]: Health check failed: 1 MDSs behind on trimming (MDS_TRIM) 2023-09-10T04:27:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:29 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:27:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:29 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:27:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18480.333666 secs 2023-09-10T04:27:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18480.988930 secs 2023-09-10T04:27:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:29 smithi098 ceph-mon[103124]: Health check failed: 1 MDSs behind on trimming (MDS_TRIM) 2023-09-10T04:27:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:29 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:27:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:29 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:27:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:30 smithi170 ceph-mon[102832]: pgmap v10018: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T04:27:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:30 smithi082 ceph-mon[98694]: pgmap v10018: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T04:27:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:30 smithi098 ceph-mon[103124]: pgmap v10018: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T04:27:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:27:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:27:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:27:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:32 smithi170 ceph-mon[102832]: pgmap v10019: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 228 KiB/s wr, 15 op/s 2023-09-10T04:27:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:27:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:27:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:27:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:32 smithi082 ceph-mon[98694]: pgmap v10019: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 228 KiB/s wr, 15 op/s 2023-09-10T04:27:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:27:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:27:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:27:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:32 smithi098 ceph-mon[103124]: pgmap v10019: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 228 KiB/s wr, 15 op/s 2023-09-10T04:27:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:27:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:27:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:27:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18485.333725 secs 2023-09-10T04:27:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:34 smithi098 ceph-mon[103124]: pgmap v10020: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 228 KiB/s wr, 11 op/s 2023-09-10T04:27:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18485.333725 secs 2023-09-10T04:27:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:34 smithi170 ceph-mon[102832]: pgmap v10020: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 228 KiB/s wr, 11 op/s 2023-09-10T04:27:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18485.333725 secs 2023-09-10T04:27:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:34 smithi082 ceph-mon[98694]: pgmap v10020: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 228 KiB/s wr, 11 op/s 2023-09-10T04:27:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18485.988991 secs 2023-09-10T04:27:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18485.988991 secs 2023-09-10T04:27:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18485.988991 secs 2023-09-10T04:27:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:36 smithi098 ceph-mon[103124]: pgmap v10021: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 363 KiB/s wr, 17 op/s 2023-09-10T04:27:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:36 smithi170 ceph-mon[102832]: pgmap v10021: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 363 KiB/s wr, 17 op/s 2023-09-10T04:27:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:36 smithi082 ceph-mon[98694]: pgmap v10021: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 363 KiB/s wr, 17 op/s 2023-09-10T04:27:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:27:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:27:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:27:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:27:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:27:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:27:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:38 smithi098 ceph-mon[103124]: pgmap v10022: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T04:27:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:38 smithi098 ceph-mon[103124]: Health check update: 2 MDSs behind on trimming (MDS_TRIM) 2023-09-10T04:27:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:38 smithi170 ceph-mon[102832]: pgmap v10022: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T04:27:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:38 smithi170 ceph-mon[102832]: Health check update: 2 MDSs behind on trimming (MDS_TRIM) 2023-09-10T04:27:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:38 smithi082 ceph-mon[98694]: pgmap v10022: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T04:27:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:38 smithi082 ceph-mon[98694]: Health check update: 2 MDSs behind on trimming (MDS_TRIM) 2023-09-10T04:27:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18490.333786 secs 2023-09-10T04:27:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18490.989055 secs 2023-09-10T04:27:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18490.333786 secs 2023-09-10T04:27:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18490.989055 secs 2023-09-10T04:27:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18490.333786 secs 2023-09-10T04:27:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18490.989055 secs 2023-09-10T04:27:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:27:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:27:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:40 smithi098 ceph-mon[103124]: pgmap v10023: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T04:27:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:27:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:27:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:40 smithi170 ceph-mon[102832]: pgmap v10023: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T04:27:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:27:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:27:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:40 smithi082 ceph-mon[98694]: pgmap v10023: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T04:27:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:42 smithi170 ceph-mon[102832]: pgmap v10024: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 423 KiB/s wr, 19 op/s 2023-09-10T04:27:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:42 smithi082 ceph-mon[98694]: pgmap v10024: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 423 KiB/s wr, 19 op/s 2023-09-10T04:27:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:42 smithi098 ceph-mon[103124]: pgmap v10024: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 423 KiB/s wr, 19 op/s 2023-09-10T04:27:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18495.333844 secs 2023-09-10T04:27:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:44 smithi098 ceph-mon[103124]: pgmap v10025: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 297 KiB/s wr, 13 op/s 2023-09-10T04:27:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18495.333844 secs 2023-09-10T04:27:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:44 smithi170 ceph-mon[102832]: pgmap v10025: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 297 KiB/s wr, 13 op/s 2023-09-10T04:27:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18495.333844 secs 2023-09-10T04:27:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:44 smithi082 ceph-mon[98694]: pgmap v10025: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 297 KiB/s wr, 13 op/s 2023-09-10T04:27:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18495.989114 secs 2023-09-10T04:27:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18495.989114 secs 2023-09-10T04:27:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18495.989114 secs 2023-09-10T04:27:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:46 smithi098 ceph-mon[103124]: pgmap v10026: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 419 KiB/s wr, 19 op/s 2023-09-10T04:27:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:46 smithi170 ceph-mon[102832]: pgmap v10026: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 419 KiB/s wr, 19 op/s 2023-09-10T04:27:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:46 smithi082 ceph-mon[98694]: pgmap v10026: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 419 KiB/s wr, 19 op/s 2023-09-10T04:27:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:48 smithi170 ceph-mon[102832]: pgmap v10027: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 283 KiB/s wr, 13 op/s 2023-09-10T04:27:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:48 smithi082 ceph-mon[98694]: pgmap v10027: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 283 KiB/s wr, 13 op/s 2023-09-10T04:27:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:48 smithi098 ceph-mon[103124]: pgmap v10027: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 283 KiB/s wr, 13 op/s 2023-09-10T04:27:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18500.333901 secs 2023-09-10T04:27:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18500.989200 secs 2023-09-10T04:27:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18500.333901 secs 2023-09-10T04:27:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18500.989200 secs 2023-09-10T04:27:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18500.333901 secs 2023-09-10T04:27:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18500.989200 secs 2023-09-10T04:27:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:50 smithi170 ceph-mon[102832]: pgmap v10028: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 283 KiB/s wr, 13 op/s 2023-09-10T04:27:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:50 smithi082 ceph-mon[98694]: pgmap v10028: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 283 KiB/s wr, 13 op/s 2023-09-10T04:27:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:50 smithi098 ceph-mon[103124]: pgmap v10028: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 283 KiB/s wr, 13 op/s 2023-09-10T04:27:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:52 smithi170 ceph-mon[102832]: pgmap v10029: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 447 KiB/s wr, 19 op/s 2023-09-10T04:27:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:52 smithi082 ceph-mon[98694]: pgmap v10029: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 447 KiB/s wr, 19 op/s 2023-09-10T04:27:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:52 smithi098 ceph-mon[103124]: pgmap v10029: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 447 KiB/s wr, 19 op/s 2023-09-10T04:27:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18504.160836 secs 2023-09-10T04:27:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18504.160836 secs 2023-09-10T04:27:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18504.160836 secs 2023-09-10T04:27:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18505.333959 secs 2023-09-10T04:27:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18505.989261 secs 2023-09-10T04:27:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:54 smithi170 ceph-mon[102832]: pgmap v10030: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 286 KiB/s wr, 12 op/s 2023-09-10T04:27:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18505.333959 secs 2023-09-10T04:27:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18505.989261 secs 2023-09-10T04:27:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:54 smithi082 ceph-mon[98694]: pgmap v10030: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 286 KiB/s wr, 12 op/s 2023-09-10T04:27:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18505.333959 secs 2023-09-10T04:27:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18505.989261 secs 2023-09-10T04:27:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:54 smithi098 ceph-mon[103124]: pgmap v10030: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 286 KiB/s wr, 12 op/s 2023-09-10T04:27:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:27:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:27:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:27:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:27:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:27:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:27:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:27:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:27:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:27:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:56 smithi170 ceph-mon[102832]: pgmap v10031: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 347 KiB/s wr, 18 op/s 2023-09-10T04:27:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:56 smithi082 ceph-mon[98694]: pgmap v10031: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 347 KiB/s wr, 18 op/s 2023-09-10T04:27:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:56 smithi098 ceph-mon[103124]: pgmap v10031: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 347 KiB/s wr, 18 op/s 2023-09-10T04:27:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:27:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:27:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18509.160895 secs 2023-09-10T04:27:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:27:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:27:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18509.160895 secs 2023-09-10T04:27:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:27:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:27:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18509.160895 secs 2023-09-10T04:27:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:58 smithi170 ceph-mon[102832]: pgmap v10032: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 226 KiB/s wr, 11 op/s 2023-09-10T04:27:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:58 smithi082 ceph-mon[98694]: pgmap v10032: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 226 KiB/s wr, 11 op/s 2023-09-10T04:27:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:58 smithi098 ceph-mon[103124]: pgmap v10032: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 226 KiB/s wr, 11 op/s 2023-09-10T04:28:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18510.334017 secs 2023-09-10T04:28:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18510.989324 secs 2023-09-10T04:28:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:59 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:28:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:27:59 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:28:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18510.334017 secs 2023-09-10T04:28:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18510.989324 secs 2023-09-10T04:28:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:59 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:28:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:27:59 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:28:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18510.334017 secs 2023-09-10T04:28:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18510.989324 secs 2023-09-10T04:28:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:59 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:28:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:27:59 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:28:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:00 smithi170 ceph-mon[102832]: pgmap v10033: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 226 KiB/s wr, 11 op/s 2023-09-10T04:28:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:00 smithi082 ceph-mon[98694]: pgmap v10033: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 226 KiB/s wr, 11 op/s 2023-09-10T04:28:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:00 smithi098 ceph-mon[103124]: pgmap v10033: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 226 KiB/s wr, 11 op/s 2023-09-10T04:28:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:02 smithi170 ceph-mon[102832]: pgmap v10034: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 325 KiB/s wr, 17 op/s 2023-09-10T04:28:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18514.160987 secs 2023-09-10T04:28:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:02 smithi082 ceph-mon[98694]: pgmap v10034: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 325 KiB/s wr, 17 op/s 2023-09-10T04:28:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18514.160987 secs 2023-09-10T04:28:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:02 smithi098 ceph-mon[103124]: pgmap v10034: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 325 KiB/s wr, 17 op/s 2023-09-10T04:28:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18514.160987 secs 2023-09-10T04:28:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18515.334072 secs 2023-09-10T04:28:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:04 smithi098 ceph-mon[103124]: pgmap v10035: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 11 op/s 2023-09-10T04:28:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18515.334072 secs 2023-09-10T04:28:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:04 smithi170 ceph-mon[102832]: pgmap v10035: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 11 op/s 2023-09-10T04:28:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18515.334072 secs 2023-09-10T04:28:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:04 smithi082 ceph-mon[98694]: pgmap v10035: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 11 op/s 2023-09-10T04:28:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18515.989386 secs 2023-09-10T04:28:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18515.989386 secs 2023-09-10T04:28:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18515.989386 secs 2023-09-10T04:28:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:06 smithi098 ceph-mon[103124]: pgmap v10036: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 264 KiB/s wr, 31 op/s 2023-09-10T04:28:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:06 smithi170 ceph-mon[102832]: pgmap v10036: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 264 KiB/s wr, 31 op/s 2023-09-10T04:28:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:06 smithi082 ceph-mon[98694]: pgmap v10036: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 264 KiB/s wr, 31 op/s 2023-09-10T04:28:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18519.161048 secs 2023-09-10T04:28:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:08 smithi098 ceph-mon[103124]: osdmap e695: 12 total, 12 up, 12 in 2023-09-10T04:28:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:08 smithi098 ceph-mon[103124]: pgmap v10038: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 243 KiB/s wr, 31 op/s 2023-09-10T04:28:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18519.161048 secs 2023-09-10T04:28:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:08 smithi170 ceph-mon[102832]: osdmap e695: 12 total, 12 up, 12 in 2023-09-10T04:28:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:08 smithi170 ceph-mon[102832]: pgmap v10038: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 243 KiB/s wr, 31 op/s 2023-09-10T04:28:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18519.161048 secs 2023-09-10T04:28:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:08 smithi082 ceph-mon[98694]: osdmap e695: 12 total, 12 up, 12 in 2023-09-10T04:28:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:08 smithi082 ceph-mon[98694]: pgmap v10038: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 243 KiB/s wr, 31 op/s 2023-09-10T04:28:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18520.334142 secs 2023-09-10T04:28:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18520.989450 secs 2023-09-10T04:28:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18520.334142 secs 2023-09-10T04:28:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18520.989450 secs 2023-09-10T04:28:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18520.334142 secs 2023-09-10T04:28:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18520.989450 secs 2023-09-10T04:28:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:10 smithi098 ceph-mon[103124]: pgmap v10039: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 243 KiB/s wr, 31 op/s 2023-09-10T04:28:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:10 smithi170 ceph-mon[102832]: pgmap v10039: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 243 KiB/s wr, 31 op/s 2023-09-10T04:28:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:10 smithi082 ceph-mon[98694]: pgmap v10039: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 243 KiB/s wr, 31 op/s 2023-09-10T04:28:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:12 smithi170 ceph-mon[102832]: pgmap v10040: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 125 KiB/s wr, 30 op/s 2023-09-10T04:28:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:12 smithi082 ceph-mon[98694]: pgmap v10040: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 125 KiB/s wr, 30 op/s 2023-09-10T04:28:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:12 smithi098 ceph-mon[103124]: pgmap v10040: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 125 KiB/s wr, 30 op/s 2023-09-10T04:28:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18524.161110 secs 2023-09-10T04:28:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:13 smithi170 ceph-mon[102832]: osdmap e696: 12 total, 12 up, 12 in 2023-09-10T04:28:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18524.161110 secs 2023-09-10T04:28:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:13 smithi082 ceph-mon[98694]: osdmap e696: 12 total, 12 up, 12 in 2023-09-10T04:28:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18524.161110 secs 2023-09-10T04:28:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:13 smithi098 ceph-mon[103124]: osdmap e696: 12 total, 12 up, 12 in 2023-09-10T04:28:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18525.334199 secs 2023-09-10T04:28:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18525.989491 secs 2023-09-10T04:28:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:14 smithi170 ceph-mon[102832]: pgmap v10042: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:28:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18525.334199 secs 2023-09-10T04:28:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18525.989491 secs 2023-09-10T04:28:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:14 smithi082 ceph-mon[98694]: pgmap v10042: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:28:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18525.334199 secs 2023-09-10T04:28:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18525.989491 secs 2023-09-10T04:28:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:14 smithi098 ceph-mon[103124]: pgmap v10042: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:28:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:16 smithi170 ceph-mon[102832]: pgmap v10043: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 15 op/s 2023-09-10T04:28:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:16 smithi082 ceph-mon[98694]: pgmap v10043: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 15 op/s 2023-09-10T04:28:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:16 smithi098 ceph-mon[103124]: pgmap v10043: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 15 op/s 2023-09-10T04:28:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18529.161173 secs 2023-09-10T04:28:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:18 smithi170 ceph-mon[102832]: pgmap v10044: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T04:28:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:28:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18529.161173 secs 2023-09-10T04:28:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:18 smithi082 ceph-mon[98694]: pgmap v10044: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T04:28:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:28:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18529.161173 secs 2023-09-10T04:28:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:18 smithi098 ceph-mon[103124]: pgmap v10044: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T04:28:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:28:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18530.334273 secs 2023-09-10T04:28:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18530.989556 secs 2023-09-10T04:28:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:28:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:28:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:28:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18530.334273 secs 2023-09-10T04:28:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18530.989556 secs 2023-09-10T04:28:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:28:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:28:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:28:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18530.334273 secs 2023-09-10T04:28:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18530.989556 secs 2023-09-10T04:28:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:28:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:28:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:28:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:21 smithi082 ceph-mon[98694]: pgmap v10045: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T04:28:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:21 smithi098 ceph-mon[103124]: pgmap v10045: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T04:28:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:21 smithi170 ceph-mon[102832]: pgmap v10045: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T04:28:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:23 smithi082 ceph-mon[98694]: pgmap v10046: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T04:28:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18534.161235 secs 2023-09-10T04:28:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:23 smithi098 ceph-mon[103124]: pgmap v10046: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T04:28:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18534.161235 secs 2023-09-10T04:28:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:23 smithi170 ceph-mon[102832]: pgmap v10046: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T04:28:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18534.161235 secs 2023-09-10T04:28:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18535.334328 secs 2023-09-10T04:28:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18535.989618 secs 2023-09-10T04:28:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:24 smithi098 ceph-mon[103124]: pgmap v10047: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T04:28:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18535.334328 secs 2023-09-10T04:28:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18535.989618 secs 2023-09-10T04:28:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:24 smithi170 ceph-mon[102832]: pgmap v10047: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T04:28:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18535.334328 secs 2023-09-10T04:28:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18535.989618 secs 2023-09-10T04:28:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:24 smithi082 ceph-mon[98694]: pgmap v10047: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T04:28:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:26 smithi170 ceph-mon[102832]: pgmap v10048: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 198 KiB/s wr, 14 op/s 2023-09-10T04:28:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:26 smithi082 ceph-mon[98694]: pgmap v10048: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 198 KiB/s wr, 14 op/s 2023-09-10T04:28:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:26 smithi098 ceph-mon[103124]: pgmap v10048: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 198 KiB/s wr, 14 op/s 2023-09-10T04:28:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:28 smithi170 ceph-mon[102832]: pgmap v10049: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 97 KiB/s wr, 9 op/s 2023-09-10T04:28:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:28 smithi082 ceph-mon[98694]: pgmap v10049: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 97 KiB/s wr, 9 op/s 2023-09-10T04:28:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:28 smithi098 ceph-mon[103124]: pgmap v10049: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 97 KiB/s wr, 9 op/s 2023-09-10T04:28:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18540.334385 secs 2023-09-10T04:28:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18540.989681 secs 2023-09-10T04:28:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:29 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:28:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:29 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:28:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18540.334385 secs 2023-09-10T04:28:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18540.989681 secs 2023-09-10T04:28:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:29 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:28:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:29 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:28:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18540.334385 secs 2023-09-10T04:28:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18540.989681 secs 2023-09-10T04:28:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:29 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:28:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:29 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:28:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:30 smithi170 ceph-mon[102832]: pgmap v10050: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 97 KiB/s wr, 9 op/s 2023-09-10T04:28:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:30 smithi082 ceph-mon[98694]: pgmap v10050: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 97 KiB/s wr, 9 op/s 2023-09-10T04:28:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:30 smithi098 ceph-mon[103124]: pgmap v10050: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 97 KiB/s wr, 9 op/s 2023-09-10T04:28:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:28:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:28:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:28:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:32 smithi170 ceph-mon[102832]: pgmap v10051: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 229 KiB/s wr, 15 op/s 2023-09-10T04:28:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:28:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:28:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:28:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:32 smithi082 ceph-mon[98694]: pgmap v10051: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 229 KiB/s wr, 15 op/s 2023-09-10T04:28:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:28:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:28:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:28:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:32 smithi098 ceph-mon[103124]: pgmap v10051: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 229 KiB/s wr, 15 op/s 2023-09-10T04:28:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:28:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:28:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:28:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18545.334445 secs 2023-09-10T04:28:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18545.989744 secs 2023-09-10T04:28:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:34 smithi098 ceph-mon[103124]: pgmap v10052: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 229 KiB/s wr, 11 op/s 2023-09-10T04:28:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18545.334445 secs 2023-09-10T04:28:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18545.989744 secs 2023-09-10T04:28:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:34 smithi170 ceph-mon[102832]: pgmap v10052: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 229 KiB/s wr, 11 op/s 2023-09-10T04:28:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18545.334445 secs 2023-09-10T04:28:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18545.989744 secs 2023-09-10T04:28:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:34 smithi082 ceph-mon[98694]: pgmap v10052: 97 pgs: 97 active+clean; 9.9 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 229 KiB/s wr, 11 op/s 2023-09-10T04:28:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:36 smithi170 ceph-mon[102832]: pgmap v10053: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 365 KiB/s wr, 17 op/s 2023-09-10T04:28:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:36 smithi082 ceph-mon[98694]: pgmap v10053: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 365 KiB/s wr, 17 op/s 2023-09-10T04:28:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:36 smithi098 ceph-mon[103124]: pgmap v10053: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 365 KiB/s wr, 17 op/s 2023-09-10T04:28:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:28:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:28:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:28:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:28:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:28:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:28:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:38 smithi170 ceph-mon[102832]: pgmap v10054: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T04:28:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:38 smithi082 ceph-mon[98694]: pgmap v10054: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T04:28:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:38 smithi098 ceph-mon[103124]: pgmap v10054: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T04:28:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18550.334582 secs 2023-09-10T04:28:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18550.989807 secs 2023-09-10T04:28:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18550.334582 secs 2023-09-10T04:28:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18550.989807 secs 2023-09-10T04:28:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18550.334582 secs 2023-09-10T04:28:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18550.989807 secs 2023-09-10T04:28:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:40 smithi170 ceph-mon[102832]: pgmap v10055: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T04:28:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:28:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:28:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:40 smithi082 ceph-mon[98694]: pgmap v10055: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T04:28:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:28:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:28:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:40 smithi098 ceph-mon[103124]: pgmap v10055: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T04:28:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:28:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:28:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:42 smithi170 ceph-mon[102832]: pgmap v10056: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 430 KiB/s wr, 19 op/s 2023-09-10T04:28:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:42 smithi082 ceph-mon[98694]: pgmap v10056: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 430 KiB/s wr, 19 op/s 2023-09-10T04:28:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:42 smithi098 ceph-mon[103124]: pgmap v10056: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 430 KiB/s wr, 19 op/s 2023-09-10T04:28:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18555.334638 secs 2023-09-10T04:28:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18555.989870 secs 2023-09-10T04:28:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:44 smithi098 ceph-mon[103124]: pgmap v10057: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 298 KiB/s wr, 13 op/s 2023-09-10T04:28:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18555.334638 secs 2023-09-10T04:28:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18555.989870 secs 2023-09-10T04:28:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:44 smithi170 ceph-mon[102832]: pgmap v10057: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 298 KiB/s wr, 13 op/s 2023-09-10T04:28:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18555.334638 secs 2023-09-10T04:28:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18555.989870 secs 2023-09-10T04:28:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:44 smithi082 ceph-mon[98694]: pgmap v10057: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 298 KiB/s wr, 13 op/s 2023-09-10T04:28:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:46 smithi170 ceph-mon[102832]: pgmap v10058: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 420 KiB/s wr, 19 op/s 2023-09-10T04:28:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:46 smithi082 ceph-mon[98694]: pgmap v10058: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 420 KiB/s wr, 19 op/s 2023-09-10T04:28:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:46 smithi098 ceph-mon[103124]: pgmap v10058: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 420 KiB/s wr, 19 op/s 2023-09-10T04:28:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:48 smithi170 ceph-mon[102832]: pgmap v10059: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-09-10T04:28:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:48 smithi082 ceph-mon[98694]: pgmap v10059: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-09-10T04:28:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:48 smithi098 ceph-mon[103124]: pgmap v10059: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-09-10T04:28:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18560.334696 secs 2023-09-10T04:28:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18560.989932 secs 2023-09-10T04:28:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18560.334696 secs 2023-09-10T04:28:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18560.989932 secs 2023-09-10T04:28:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18560.334696 secs 2023-09-10T04:28:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18560.989932 secs 2023-09-10T04:28:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:50 smithi170 ceph-mon[102832]: pgmap v10060: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-09-10T04:28:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:50 smithi082 ceph-mon[98694]: pgmap v10060: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-09-10T04:28:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:50 smithi098 ceph-mon[103124]: pgmap v10060: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-09-10T04:28:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:52 smithi170 ceph-mon[102832]: pgmap v10061: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 449 KiB/s wr, 18 op/s 2023-09-10T04:28:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:52 smithi082 ceph-mon[98694]: pgmap v10061: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 449 KiB/s wr, 18 op/s 2023-09-10T04:28:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:52 smithi098 ceph-mon[103124]: pgmap v10061: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 449 KiB/s wr, 18 op/s 2023-09-10T04:28:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18564.161795 secs 2023-09-10T04:28:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18564.161795 secs 2023-09-10T04:28:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18564.161795 secs 2023-09-10T04:28:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18565.334755 secs 2023-09-10T04:28:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18565.989994 secs 2023-09-10T04:28:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:54 smithi170 ceph-mon[102832]: pgmap v10062: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 287 KiB/s wr, 11 op/s 2023-09-10T04:28:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18565.334755 secs 2023-09-10T04:28:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18565.989994 secs 2023-09-10T04:28:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:54 smithi082 ceph-mon[98694]: pgmap v10062: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 287 KiB/s wr, 11 op/s 2023-09-10T04:28:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18565.334755 secs 2023-09-10T04:28:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18565.989994 secs 2023-09-10T04:28:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:54 smithi098 ceph-mon[103124]: pgmap v10062: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 287 KiB/s wr, 11 op/s 2023-09-10T04:28:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:28:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:28:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:28:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:28:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:28:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:28:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:28:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:28:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:28:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:56 smithi170 ceph-mon[102832]: pgmap v10063: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 349 KiB/s wr, 17 op/s 2023-09-10T04:28:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:56 smithi082 ceph-mon[98694]: pgmap v10063: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 349 KiB/s wr, 17 op/s 2023-09-10T04:28:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:56 smithi098 ceph-mon[103124]: pgmap v10063: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 349 KiB/s wr, 17 op/s 2023-09-10T04:28:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:28:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:28:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:28:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:28:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:28:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:28:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18569.161854 secs 2023-09-10T04:28:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:58 smithi170 ceph-mon[102832]: pgmap v10064: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T04:28:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18569.161854 secs 2023-09-10T04:28:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:58 smithi082 ceph-mon[98694]: pgmap v10064: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T04:28:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18569.161854 secs 2023-09-10T04:28:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:58 smithi098 ceph-mon[103124]: pgmap v10064: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T04:29:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18570.334839 secs 2023-09-10T04:29:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18570.990056 secs 2023-09-10T04:29:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:59 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:29:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:28:59 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:29:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18570.334839 secs 2023-09-10T04:29:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18570.990056 secs 2023-09-10T04:29:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:59 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:29:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:28:59 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:29:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18570.334839 secs 2023-09-10T04:29:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18570.990056 secs 2023-09-10T04:29:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:59 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:29:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:28:59 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:29:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:00 smithi170 ceph-mon[102832]: pgmap v10065: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T04:29:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:00 smithi082 ceph-mon[98694]: pgmap v10065: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T04:29:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:00 smithi098 ceph-mon[103124]: pgmap v10065: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T04:29:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:02 smithi170 ceph-mon[102832]: pgmap v10066: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 378 KiB/s wr, 17 op/s 2023-09-10T04:29:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:02 smithi082 ceph-mon[98694]: pgmap v10066: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 378 KiB/s wr, 17 op/s 2023-09-10T04:29:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:02 smithi098 ceph-mon[103124]: pgmap v10066: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 378 KiB/s wr, 17 op/s 2023-09-10T04:29:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18574.161942 secs 2023-09-10T04:29:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18574.161942 secs 2023-09-10T04:29:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18574.161942 secs 2023-09-10T04:29:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18575.334896 secs 2023-09-10T04:29:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18575.990116 secs 2023-09-10T04:29:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:04 smithi170 ceph-mon[102832]: pgmap v10067: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 213 KiB/s wr, 11 op/s 2023-09-10T04:29:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18575.334896 secs 2023-09-10T04:29:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18575.990116 secs 2023-09-10T04:29:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:04 smithi082 ceph-mon[98694]: pgmap v10067: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 213 KiB/s wr, 11 op/s 2023-09-10T04:29:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18575.334896 secs 2023-09-10T04:29:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18575.990116 secs 2023-09-10T04:29:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:04 smithi098 ceph-mon[103124]: pgmap v10067: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 213 KiB/s wr, 11 op/s 2023-09-10T04:29:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:06 smithi170 ceph-mon[102832]: pgmap v10068: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 303 KiB/s wr, 31 op/s 2023-09-10T04:29:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:06 smithi082 ceph-mon[98694]: pgmap v10068: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 303 KiB/s wr, 31 op/s 2023-09-10T04:29:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:06 smithi098 ceph-mon[103124]: pgmap v10068: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 303 KiB/s wr, 31 op/s 2023-09-10T04:29:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18579.162002 secs 2023-09-10T04:29:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18579.162002 secs 2023-09-10T04:29:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18579.162002 secs 2023-09-10T04:29:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:08 smithi170 ceph-mon[102832]: pgmap v10069: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 242 KiB/s wr, 25 op/s 2023-09-10T04:29:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:08 smithi170 ceph-mon[102832]: osdmap e697: 12 total, 12 up, 12 in 2023-09-10T04:29:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:08 smithi082 ceph-mon[98694]: pgmap v10069: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 242 KiB/s wr, 25 op/s 2023-09-10T04:29:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:08 smithi082 ceph-mon[98694]: osdmap e697: 12 total, 12 up, 12 in 2023-09-10T04:29:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:08 smithi098 ceph-mon[103124]: pgmap v10069: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 242 KiB/s wr, 25 op/s 2023-09-10T04:29:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:08 smithi098 ceph-mon[103124]: osdmap e697: 12 total, 12 up, 12 in 2023-09-10T04:29:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18580.334952 secs 2023-09-10T04:29:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18580.990176 secs 2023-09-10T04:29:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18580.334952 secs 2023-09-10T04:29:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18580.990176 secs 2023-09-10T04:29:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18580.334952 secs 2023-09-10T04:29:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18580.990176 secs 2023-09-10T04:29:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:10 smithi082 ceph-mon[98694]: pgmap v10071: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 290 KiB/s wr, 30 op/s 2023-09-10T04:29:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:10 smithi098 ceph-mon[103124]: pgmap v10071: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 290 KiB/s wr, 30 op/s 2023-09-10T04:29:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:10 smithi170 ceph-mon[102832]: pgmap v10071: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 290 KiB/s wr, 30 op/s 2023-09-10T04:29:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:12 smithi082 ceph-mon[98694]: pgmap v10072: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T04:29:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18584.162071 secs 2023-09-10T04:29:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:12 smithi098 ceph-mon[103124]: pgmap v10072: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T04:29:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18584.162071 secs 2023-09-10T04:29:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:12 smithi170 ceph-mon[102832]: pgmap v10072: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 KiB/s wr, 29 op/s 2023-09-10T04:29:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18584.162071 secs 2023-09-10T04:29:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18585.335010 secs 2023-09-10T04:29:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18585.990238 secs 2023-09-10T04:29:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:14 smithi098 ceph-mon[103124]: osdmap e698: 12 total, 12 up, 12 in 2023-09-10T04:29:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:14 smithi098 ceph-mon[103124]: pgmap v10074: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:29:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18585.335010 secs 2023-09-10T04:29:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18585.990238 secs 2023-09-10T04:29:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:14 smithi170 ceph-mon[102832]: osdmap e698: 12 total, 12 up, 12 in 2023-09-10T04:29:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:14 smithi170 ceph-mon[102832]: pgmap v10074: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:29:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18585.335010 secs 2023-09-10T04:29:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18585.990238 secs 2023-09-10T04:29:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:14 smithi082 ceph-mon[98694]: osdmap e698: 12 total, 12 up, 12 in 2023-09-10T04:29:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:14 smithi082 ceph-mon[98694]: pgmap v10074: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:29:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:16 smithi170 ceph-mon[102832]: pgmap v10075: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 14 op/s 2023-09-10T04:29:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:16 smithi082 ceph-mon[98694]: pgmap v10075: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 14 op/s 2023-09-10T04:29:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:16 smithi098 ceph-mon[103124]: pgmap v10075: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 14 op/s 2023-09-10T04:29:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18589.162181 secs 2023-09-10T04:29:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:18 smithi170 ceph-mon[102832]: pgmap v10076: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T04:29:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18589.162181 secs 2023-09-10T04:29:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:18 smithi082 ceph-mon[98694]: pgmap v10076: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T04:29:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18589.162181 secs 2023-09-10T04:29:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:18 smithi098 ceph-mon[103124]: pgmap v10076: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T04:29:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18590.335068 secs 2023-09-10T04:29:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18590.990304 secs 2023-09-10T04:29:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:29:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18590.335068 secs 2023-09-10T04:29:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18590.990304 secs 2023-09-10T04:29:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:29:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18590.335068 secs 2023-09-10T04:29:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18590.990304 secs 2023-09-10T04:29:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:29:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:20 smithi170 ceph-mon[102832]: pgmap v10077: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T04:29:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:29:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:29:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:29:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:20 smithi082 ceph-mon[98694]: pgmap v10077: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T04:29:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:29:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:29:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:29:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:20 smithi098 ceph-mon[103124]: pgmap v10077: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T04:29:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:29:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:29:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:29:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:22 smithi170 ceph-mon[102832]: pgmap v10078: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T04:29:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:22 smithi082 ceph-mon[98694]: pgmap v10078: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T04:29:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:22 smithi098 ceph-mon[103124]: pgmap v10078: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T04:29:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18594.162334 secs 2023-09-10T04:29:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18594.162334 secs 2023-09-10T04:29:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18594.162334 secs 2023-09-10T04:29:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18595.335143 secs 2023-09-10T04:29:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18595.990361 secs 2023-09-10T04:29:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:24 smithi170 ceph-mon[102832]: pgmap v10079: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T04:29:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18595.335143 secs 2023-09-10T04:29:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18595.990361 secs 2023-09-10T04:29:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:24 smithi082 ceph-mon[98694]: pgmap v10079: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T04:29:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18595.335143 secs 2023-09-10T04:29:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18595.990361 secs 2023-09-10T04:29:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:24 smithi098 ceph-mon[103124]: pgmap v10079: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T04:29:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:26 smithi170 ceph-mon[102832]: pgmap v10080: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 165 KiB/s wr, 15 op/s 2023-09-10T04:29:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:26 smithi082 ceph-mon[98694]: pgmap v10080: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 165 KiB/s wr, 15 op/s 2023-09-10T04:29:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:26 smithi098 ceph-mon[103124]: pgmap v10080: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 165 KiB/s wr, 15 op/s 2023-09-10T04:29:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:28 smithi170 ceph-mon[102832]: pgmap v10081: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 9 op/s 2023-09-10T04:29:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:28 smithi082 ceph-mon[98694]: pgmap v10081: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 9 op/s 2023-09-10T04:29:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:28 smithi098 ceph-mon[103124]: pgmap v10081: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 9 op/s 2023-09-10T04:29:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18600.335212 secs 2023-09-10T04:29:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18600.990421 secs 2023-09-10T04:29:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:29 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:29:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:29 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:29:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18600.335212 secs 2023-09-10T04:29:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18600.990421 secs 2023-09-10T04:29:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:29 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:29:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:29 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:29:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18600.335212 secs 2023-09-10T04:29:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18600.990421 secs 2023-09-10T04:29:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:29 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:29:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:29 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:29:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:30 smithi170 ceph-mon[102832]: pgmap v10082: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 9 op/s 2023-09-10T04:29:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:30 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:29:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:30 smithi082 ceph-mon[98694]: pgmap v10082: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 9 op/s 2023-09-10T04:29:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:30 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:29:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:30 smithi098 ceph-mon[103124]: pgmap v10082: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 9 op/s 2023-09-10T04:29:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:30 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:29:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:31 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:29:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:29:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:29:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:31 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:29:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:29:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:29:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:31 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:29:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:29:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:29:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:32 smithi170 ceph-mon[102832]: pgmap v10083: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 230 KiB/s wr, 15 op/s 2023-09-10T04:29:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:32 smithi082 ceph-mon[98694]: pgmap v10083: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 230 KiB/s wr, 15 op/s 2023-09-10T04:29:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:32 smithi098 ceph-mon[103124]: pgmap v10083: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 230 KiB/s wr, 15 op/s 2023-09-10T04:29:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18605.335276 secs 2023-09-10T04:29:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18605.990486 secs 2023-09-10T04:29:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:34 smithi098 ceph-mon[103124]: pgmap v10084: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T04:29:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18605.335276 secs 2023-09-10T04:29:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18605.990486 secs 2023-09-10T04:29:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:34 smithi170 ceph-mon[102832]: pgmap v10084: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T04:29:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18605.335276 secs 2023-09-10T04:29:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18605.990486 secs 2023-09-10T04:29:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:34 smithi082 ceph-mon[98694]: pgmap v10084: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T04:29:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:36 smithi170 ceph-mon[102832]: pgmap v10085: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 367 KiB/s wr, 17 op/s 2023-09-10T04:29:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:36 smithi082 ceph-mon[98694]: pgmap v10085: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 367 KiB/s wr, 17 op/s 2023-09-10T04:29:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:36 smithi098 ceph-mon[103124]: pgmap v10085: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 367 KiB/s wr, 17 op/s 2023-09-10T04:29:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:29:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:29:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:29:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:29:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:29:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:29:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:38 smithi170 ceph-mon[102832]: pgmap v10086: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 304 KiB/s wr, 12 op/s 2023-09-10T04:29:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:38 smithi082 ceph-mon[98694]: pgmap v10086: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 304 KiB/s wr, 12 op/s 2023-09-10T04:29:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:38 smithi098 ceph-mon[103124]: pgmap v10086: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 304 KiB/s wr, 12 op/s 2023-09-10T04:29:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18610.335326 secs 2023-09-10T04:29:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18610.990546 secs 2023-09-10T04:29:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:39 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:29:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:39 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:29:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18610.335326 secs 2023-09-10T04:29:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18610.990546 secs 2023-09-10T04:29:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:39 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:29:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:39 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:29:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18610.335326 secs 2023-09-10T04:29:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18610.990546 secs 2023-09-10T04:29:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:39 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:29:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:39 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:29:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:40 smithi170 ceph-mon[102832]: pgmap v10087: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 304 KiB/s wr, 12 op/s 2023-09-10T04:29:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:40 smithi082 ceph-mon[98694]: pgmap v10087: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 304 KiB/s wr, 12 op/s 2023-09-10T04:29:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:40 smithi098 ceph-mon[103124]: pgmap v10087: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 304 KiB/s wr, 12 op/s 2023-09-10T04:29:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:42 smithi170 ceph-mon[102832]: pgmap v10088: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 467 KiB/s wr, 19 op/s 2023-09-10T04:29:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:42 smithi082 ceph-mon[98694]: pgmap v10088: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 467 KiB/s wr, 19 op/s 2023-09-10T04:29:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:42 smithi098 ceph-mon[103124]: pgmap v10088: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 467 KiB/s wr, 19 op/s 2023-09-10T04:29:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18615.335409 secs 2023-09-10T04:29:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18615.990606 secs 2023-09-10T04:29:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:44 smithi098 ceph-mon[103124]: pgmap v10089: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 300 KiB/s wr, 13 op/s 2023-09-10T04:29:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18615.335409 secs 2023-09-10T04:29:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18615.990606 secs 2023-09-10T04:29:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:44 smithi170 ceph-mon[102832]: pgmap v10089: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 300 KiB/s wr, 13 op/s 2023-09-10T04:29:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18615.335409 secs 2023-09-10T04:29:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18615.990606 secs 2023-09-10T04:29:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:44 smithi082 ceph-mon[98694]: pgmap v10089: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 300 KiB/s wr, 13 op/s 2023-09-10T04:29:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:46 smithi170 ceph-mon[102832]: pgmap v10090: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 423 KiB/s wr, 19 op/s 2023-09-10T04:29:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:46 smithi082 ceph-mon[98694]: pgmap v10090: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 423 KiB/s wr, 19 op/s 2023-09-10T04:29:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:46 smithi098 ceph-mon[103124]: pgmap v10090: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 423 KiB/s wr, 19 op/s 2023-09-10T04:29:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:48 smithi170 ceph-mon[102832]: pgmap v10091: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 286 KiB/s wr, 13 op/s 2023-09-10T04:29:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:48 smithi082 ceph-mon[98694]: pgmap v10091: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 286 KiB/s wr, 13 op/s 2023-09-10T04:29:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:48 smithi098 ceph-mon[103124]: pgmap v10091: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 286 KiB/s wr, 13 op/s 2023-09-10T04:29:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18620.335486 secs 2023-09-10T04:29:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18620.990668 secs 2023-09-10T04:29:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18620.335486 secs 2023-09-10T04:29:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18620.990668 secs 2023-09-10T04:29:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18620.335486 secs 2023-09-10T04:29:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18620.990668 secs 2023-09-10T04:29:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:50 smithi170 ceph-mon[102832]: pgmap v10092: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 286 KiB/s wr, 13 op/s 2023-09-10T04:29:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:50 smithi082 ceph-mon[98694]: pgmap v10092: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 286 KiB/s wr, 13 op/s 2023-09-10T04:29:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:50 smithi098 ceph-mon[103124]: pgmap v10092: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 286 KiB/s wr, 13 op/s 2023-09-10T04:29:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:29:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:29:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:29:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:29:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:29:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:29:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:52 smithi170 ceph-mon[102832]: pgmap v10093: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 460 KiB/s wr, 20 op/s 2023-09-10T04:29:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:52 smithi082 ceph-mon[98694]: pgmap v10093: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 460 KiB/s wr, 20 op/s 2023-09-10T04:29:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:52 smithi098 ceph-mon[103124]: pgmap v10093: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 460 KiB/s wr, 20 op/s 2023-09-10T04:29:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18624.162870 secs 2023-09-10T04:29:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:29:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:29:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18624.162870 secs 2023-09-10T04:29:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:29:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:29:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18624.162870 secs 2023-09-10T04:29:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:29:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:29:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18625.335559 secs 2023-09-10T04:29:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18625.990727 secs 2023-09-10T04:29:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:54 smithi170 ceph-mon[102832]: pgmap v10094: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 297 KiB/s wr, 13 op/s 2023-09-10T04:29:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18625.335559 secs 2023-09-10T04:29:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18625.990727 secs 2023-09-10T04:29:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:54 smithi082 ceph-mon[98694]: pgmap v10094: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 297 KiB/s wr, 13 op/s 2023-09-10T04:29:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18625.335559 secs 2023-09-10T04:29:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18625.990727 secs 2023-09-10T04:29:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:54 smithi098 ceph-mon[103124]: pgmap v10094: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 297 KiB/s wr, 13 op/s 2023-09-10T04:29:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:29:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:29:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:29:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:29:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:29:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:29:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:29:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:29:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:29:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:56 smithi170 ceph-mon[102832]: pgmap v10095: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 359 KiB/s wr, 18 op/s 2023-09-10T04:29:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:56 smithi082 ceph-mon[98694]: pgmap v10095: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 359 KiB/s wr, 18 op/s 2023-09-10T04:29:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:56 smithi098 ceph-mon[103124]: pgmap v10095: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 359 KiB/s wr, 18 op/s 2023-09-10T04:29:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18629.162935 secs 2023-09-10T04:29:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:58 smithi170 ceph-mon[102832]: pgmap v10096: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 237 KiB/s wr, 12 op/s 2023-09-10T04:29:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18629.162935 secs 2023-09-10T04:29:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:58 smithi082 ceph-mon[98694]: pgmap v10096: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 237 KiB/s wr, 12 op/s 2023-09-10T04:29:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18629.162935 secs 2023-09-10T04:29:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:58 smithi098 ceph-mon[103124]: pgmap v10096: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 237 KiB/s wr, 12 op/s 2023-09-10T04:30:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18630.335644 secs 2023-09-10T04:30:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:29:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18630.990817 secs 2023-09-10T04:30:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18630.335644 secs 2023-09-10T04:30:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:29:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18630.990817 secs 2023-09-10T04:30:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18630.335644 secs 2023-09-10T04:30:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:29:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18630.990817 secs 2023-09-10T04:30:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:00 smithi170 ceph-mon[102832]: pgmap v10097: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 237 KiB/s wr, 12 op/s 2023-09-10T04:30:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:00 smithi170 ceph-mon[102832]: Health detail: HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T04:30:01.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:00 smithi170 ceph-mon[102832]: [WRN] MDS_SLOW_REQUEST: 3 MDSs report slow requests 2023-09-10T04:30:01.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:00 smithi170 ceph-mon[102832]: mds.l(mds.4): 3 slow requests are blocked > 180 secs 2023-09-10T04:30:01.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:00 smithi170 ceph-mon[102832]: mds.h(mds.1): 3 slow requests are blocked > 180 secs 2023-09-10T04:30:01.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:00 smithi170 ceph-mon[102832]: mds.b(mds.0): 1 slow requests are blocked > 180 secs 2023-09-10T04:30:01.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:00 smithi170 ceph-mon[102832]: [WRN] MDS_TRIM: 2 MDSs behind on trimming 2023-09-10T04:30:01.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:00 smithi170 ceph-mon[102832]: mds.e(mds.0): Behind on trimming (265/128) max_segments: 128, num_segments: 265 2023-09-10T04:30:01.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:00 smithi170 ceph-mon[102832]: mds.b(mds.0): Behind on trimming (265/128) max_segments: 128, num_segments: 265 2023-09-10T04:30:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:00 smithi082 ceph-mon[98694]: pgmap v10097: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 237 KiB/s wr, 12 op/s 2023-09-10T04:30:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:00 smithi082 ceph-mon[98694]: Health detail: HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T04:30:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:00 smithi082 ceph-mon[98694]: [WRN] MDS_SLOW_REQUEST: 3 MDSs report slow requests 2023-09-10T04:30:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:00 smithi082 ceph-mon[98694]: mds.l(mds.4): 3 slow requests are blocked > 180 secs 2023-09-10T04:30:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:00 smithi082 ceph-mon[98694]: mds.h(mds.1): 3 slow requests are blocked > 180 secs 2023-09-10T04:30:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:00 smithi082 ceph-mon[98694]: mds.b(mds.0): 1 slow requests are blocked > 180 secs 2023-09-10T04:30:01.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:00 smithi082 ceph-mon[98694]: [WRN] MDS_TRIM: 2 MDSs behind on trimming 2023-09-10T04:30:01.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:00 smithi082 ceph-mon[98694]: mds.e(mds.0): Behind on trimming (265/128) max_segments: 128, num_segments: 265 2023-09-10T04:30:01.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:00 smithi082 ceph-mon[98694]: mds.b(mds.0): Behind on trimming (265/128) max_segments: 128, num_segments: 265 2023-09-10T04:30:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:00 smithi098 ceph-mon[103124]: pgmap v10097: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 237 KiB/s wr, 12 op/s 2023-09-10T04:30:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:00 smithi098 ceph-mon[103124]: Health detail: HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T04:30:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:00 smithi098 ceph-mon[103124]: [WRN] MDS_SLOW_REQUEST: 3 MDSs report slow requests 2023-09-10T04:30:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:00 smithi098 ceph-mon[103124]: mds.l(mds.4): 3 slow requests are blocked > 180 secs 2023-09-10T04:30:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:00 smithi098 ceph-mon[103124]: mds.h(mds.1): 3 slow requests are blocked > 180 secs 2023-09-10T04:30:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:00 smithi098 ceph-mon[103124]: mds.b(mds.0): 1 slow requests are blocked > 180 secs 2023-09-10T04:30:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:00 smithi098 ceph-mon[103124]: [WRN] MDS_TRIM: 2 MDSs behind on trimming 2023-09-10T04:30:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:00 smithi098 ceph-mon[103124]: mds.e(mds.0): Behind on trimming (265/128) max_segments: 128, num_segments: 265 2023-09-10T04:30:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:00 smithi098 ceph-mon[103124]: mds.b(mds.0): Behind on trimming (265/128) max_segments: 128, num_segments: 265 2023-09-10T04:30:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:02 smithi170 ceph-mon[102832]: pgmap v10098: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 359 KiB/s wr, 22 op/s 2023-09-10T04:30:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:02 smithi082 ceph-mon[98694]: pgmap v10098: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 359 KiB/s wr, 22 op/s 2023-09-10T04:30:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:02 smithi098 ceph-mon[103124]: pgmap v10098: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 359 KiB/s wr, 22 op/s 2023-09-10T04:30:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18634.163008 secs 2023-09-10T04:30:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18634.163008 secs 2023-09-10T04:30:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18634.163008 secs 2023-09-10T04:30:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18635.335702 secs 2023-09-10T04:30:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18635.990880 secs 2023-09-10T04:30:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:04 smithi170 ceph-mon[102832]: pgmap v10099: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 185 KiB/s wr, 15 op/s 2023-09-10T04:30:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18635.335702 secs 2023-09-10T04:30:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18635.990880 secs 2023-09-10T04:30:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:04 smithi082 ceph-mon[98694]: pgmap v10099: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 185 KiB/s wr, 15 op/s 2023-09-10T04:30:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18635.335702 secs 2023-09-10T04:30:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18635.990880 secs 2023-09-10T04:30:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:04 smithi098 ceph-mon[103124]: pgmap v10099: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 185 KiB/s wr, 15 op/s 2023-09-10T04:30:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:06 smithi170 ceph-mon[102832]: pgmap v10100: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 266 KiB/s wr, 30 op/s 2023-09-10T04:30:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:06 smithi082 ceph-mon[98694]: pgmap v10100: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 266 KiB/s wr, 30 op/s 2023-09-10T04:30:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:06 smithi098 ceph-mon[103124]: pgmap v10100: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 266 KiB/s wr, 30 op/s 2023-09-10T04:30:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18639.163068 secs 2023-09-10T04:30:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:08 smithi170 ceph-mon[102832]: pgmap v10101: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 204 KiB/s wr, 24 op/s 2023-09-10T04:30:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:08 smithi170 ceph-mon[102832]: osdmap e699: 12 total, 12 up, 12 in 2023-09-10T04:30:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18639.163068 secs 2023-09-10T04:30:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:08 smithi082 ceph-mon[98694]: pgmap v10101: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 204 KiB/s wr, 24 op/s 2023-09-10T04:30:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:08 smithi082 ceph-mon[98694]: osdmap e699: 12 total, 12 up, 12 in 2023-09-10T04:30:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18639.163068 secs 2023-09-10T04:30:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:08 smithi098 ceph-mon[103124]: pgmap v10101: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 204 KiB/s wr, 24 op/s 2023-09-10T04:30:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:08 smithi098 ceph-mon[103124]: osdmap e699: 12 total, 12 up, 12 in 2023-09-10T04:30:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18640.335761 secs 2023-09-10T04:30:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18640.990944 secs 2023-09-10T04:30:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18640.335761 secs 2023-09-10T04:30:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18640.990944 secs 2023-09-10T04:30:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18640.335761 secs 2023-09-10T04:30:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18640.990944 secs 2023-09-10T04:30:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:10 smithi170 ceph-mon[102832]: pgmap v10103: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 244 KiB/s wr, 29 op/s 2023-09-10T04:30:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:10 smithi082 ceph-mon[98694]: pgmap v10103: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 244 KiB/s wr, 29 op/s 2023-09-10T04:30:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:10 smithi098 ceph-mon[103124]: pgmap v10103: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 244 KiB/s wr, 29 op/s 2023-09-10T04:30:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:12 smithi170 ceph-mon[102832]: pgmap v10104: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 98 KiB/s wr, 24 op/s 2023-09-10T04:30:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:12 smithi082 ceph-mon[98694]: pgmap v10104: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 98 KiB/s wr, 24 op/s 2023-09-10T04:30:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:12 smithi098 ceph-mon[103124]: pgmap v10104: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 98 KiB/s wr, 24 op/s 2023-09-10T04:30:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18644.163128 secs 2023-09-10T04:30:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:13 smithi170 ceph-mon[102832]: osdmap e700: 12 total, 12 up, 12 in 2023-09-10T04:30:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18644.163128 secs 2023-09-10T04:30:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:13 smithi082 ceph-mon[98694]: osdmap e700: 12 total, 12 up, 12 in 2023-09-10T04:30:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18644.163128 secs 2023-09-10T04:30:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:13 smithi098 ceph-mon[103124]: osdmap e700: 12 total, 12 up, 12 in 2023-09-10T04:30:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18645.335846 secs 2023-09-10T04:30:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18645.991009 secs 2023-09-10T04:30:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:14 smithi170 ceph-mon[102832]: pgmap v10106: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:30:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18645.335846 secs 2023-09-10T04:30:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18645.991009 secs 2023-09-10T04:30:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:14 smithi082 ceph-mon[98694]: pgmap v10106: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:30:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18645.335846 secs 2023-09-10T04:30:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18645.991009 secs 2023-09-10T04:30:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:14 smithi098 ceph-mon[103124]: pgmap v10106: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:30:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:16 smithi170 ceph-mon[102832]: pgmap v10107: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 15 op/s 2023-09-10T04:30:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:16 smithi082 ceph-mon[98694]: pgmap v10107: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 15 op/s 2023-09-10T04:30:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:16 smithi098 ceph-mon[103124]: pgmap v10107: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 15 op/s 2023-09-10T04:30:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18649.163207 secs 2023-09-10T04:30:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:18 smithi170 ceph-mon[102832]: pgmap v10108: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T04:30:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18649.163207 secs 2023-09-10T04:30:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:18 smithi082 ceph-mon[98694]: pgmap v10108: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T04:30:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18649.163207 secs 2023-09-10T04:30:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:18 smithi098 ceph-mon[103124]: pgmap v10108: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T04:30:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18650.335903 secs 2023-09-10T04:30:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18650.991069 secs 2023-09-10T04:30:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18650.335903 secs 2023-09-10T04:30:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18650.991069 secs 2023-09-10T04:30:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18650.335903 secs 2023-09-10T04:30:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18650.991069 secs 2023-09-10T04:30:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:20 smithi082 ceph-mon[98694]: pgmap v10109: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:30:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:30:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:30:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:30:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:30:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:20 smithi098 ceph-mon[103124]: pgmap v10109: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:30:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:30:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:30:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:30:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:30:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:20 smithi170 ceph-mon[102832]: pgmap v10109: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:30:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:30:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:30:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:30:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:30:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:22 smithi082 ceph-mon[98694]: pgmap v10110: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T04:30:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18654.163335 secs 2023-09-10T04:30:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:22 smithi098 ceph-mon[103124]: pgmap v10110: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T04:30:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18654.163335 secs 2023-09-10T04:30:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:22 smithi170 ceph-mon[102832]: pgmap v10110: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T04:30:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18654.163335 secs 2023-09-10T04:30:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18655.335960 secs 2023-09-10T04:30:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18655.991131 secs 2023-09-10T04:30:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:24 smithi170 ceph-mon[102832]: pgmap v10111: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T04:30:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18655.335960 secs 2023-09-10T04:30:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18655.991131 secs 2023-09-10T04:30:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:24 smithi082 ceph-mon[98694]: pgmap v10111: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T04:30:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18655.335960 secs 2023-09-10T04:30:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18655.991131 secs 2023-09-10T04:30:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:24 smithi098 ceph-mon[103124]: pgmap v10111: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T04:30:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:26 smithi170 ceph-mon[102832]: pgmap v10112: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 165 KiB/s wr, 15 op/s 2023-09-10T04:30:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:26 smithi082 ceph-mon[98694]: pgmap v10112: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 165 KiB/s wr, 15 op/s 2023-09-10T04:30:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:26 smithi098 ceph-mon[103124]: pgmap v10112: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 165 KiB/s wr, 15 op/s 2023-09-10T04:30:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:28 smithi170 ceph-mon[102832]: pgmap v10113: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 9 op/s 2023-09-10T04:30:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:28 smithi082 ceph-mon[98694]: pgmap v10113: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 9 op/s 2023-09-10T04:30:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:28 smithi098 ceph-mon[103124]: pgmap v10113: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 9 op/s 2023-09-10T04:30:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18660.336044 secs 2023-09-10T04:30:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18660.991194 secs 2023-09-10T04:30:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:29 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:30:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:29 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:30:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18660.336044 secs 2023-09-10T04:30:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18660.991194 secs 2023-09-10T04:30:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:29 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:30:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:29 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:30:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18660.336044 secs 2023-09-10T04:30:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18660.991194 secs 2023-09-10T04:30:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:29 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:30:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:29 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:30:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:30 smithi170 ceph-mon[102832]: pgmap v10114: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 9 op/s 2023-09-10T04:30:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:30 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:30:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:30 smithi082 ceph-mon[98694]: pgmap v10114: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 9 op/s 2023-09-10T04:30:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:30 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:30:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:30 smithi098 ceph-mon[103124]: pgmap v10114: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 9 op/s 2023-09-10T04:30:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:30 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:30:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:31 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:30:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:30:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:30:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:31 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:30:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:30:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:30:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:31 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:30:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:30:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:30:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:32 smithi170 ceph-mon[102832]: pgmap v10115: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 231 KiB/s wr, 15 op/s 2023-09-10T04:30:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:32 smithi082 ceph-mon[98694]: pgmap v10115: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 231 KiB/s wr, 15 op/s 2023-09-10T04:30:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:32 smithi098 ceph-mon[103124]: pgmap v10115: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 231 KiB/s wr, 15 op/s 2023-09-10T04:30:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18665.336131 secs 2023-09-10T04:30:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18665.991257 secs 2023-09-10T04:30:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:34 smithi170 ceph-mon[102832]: pgmap v10116: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T04:30:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18665.336131 secs 2023-09-10T04:30:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18665.991257 secs 2023-09-10T04:30:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:34 smithi082 ceph-mon[98694]: pgmap v10116: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T04:30:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18665.336131 secs 2023-09-10T04:30:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18665.991257 secs 2023-09-10T04:30:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:34 smithi098 ceph-mon[103124]: pgmap v10116: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T04:30:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:36 smithi170 ceph-mon[102832]: pgmap v10117: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 382 KiB/s wr, 16 op/s 2023-09-10T04:30:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:36 smithi082 ceph-mon[98694]: pgmap v10117: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 382 KiB/s wr, 16 op/s 2023-09-10T04:30:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:36 smithi098 ceph-mon[103124]: pgmap v10117: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 382 KiB/s wr, 16 op/s 2023-09-10T04:30:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:30:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:30:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:30:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:30:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:30:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:30:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:38 smithi170 ceph-mon[102832]: pgmap v10118: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 319 KiB/s wr, 11 op/s 2023-09-10T04:30:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:38 smithi082 ceph-mon[98694]: pgmap v10118: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 319 KiB/s wr, 11 op/s 2023-09-10T04:30:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:38 smithi098 ceph-mon[103124]: pgmap v10118: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 319 KiB/s wr, 11 op/s 2023-09-10T04:30:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18670.336188 secs 2023-09-10T04:30:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18670.991320 secs 2023-09-10T04:30:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:39 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:30:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:39 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:30:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18670.336188 secs 2023-09-10T04:30:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18670.991320 secs 2023-09-10T04:30:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:39 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:30:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:39 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:30:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18670.336188 secs 2023-09-10T04:30:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18670.991320 secs 2023-09-10T04:30:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:39 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:30:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:39 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:30:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:40 smithi170 ceph-mon[102832]: pgmap v10119: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 319 KiB/s wr, 11 op/s 2023-09-10T04:30:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:40 smithi082 ceph-mon[98694]: pgmap v10119: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 319 KiB/s wr, 11 op/s 2023-09-10T04:30:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:40 smithi098 ceph-mon[103124]: pgmap v10119: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 319 KiB/s wr, 11 op/s 2023-09-10T04:30:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:42 smithi170 ceph-mon[102832]: pgmap v10120: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 468 KiB/s wr, 18 op/s 2023-09-10T04:30:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:42 smithi082 ceph-mon[98694]: pgmap v10120: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 468 KiB/s wr, 18 op/s 2023-09-10T04:30:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:42 smithi098 ceph-mon[103124]: pgmap v10120: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 468 KiB/s wr, 18 op/s 2023-09-10T04:30:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18675.336277 secs 2023-09-10T04:30:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18675.991381 secs 2023-09-10T04:30:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:44 smithi170 ceph-mon[102832]: pgmap v10121: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 301 KiB/s wr, 12 op/s 2023-09-10T04:30:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18675.336277 secs 2023-09-10T04:30:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18675.991381 secs 2023-09-10T04:30:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:44 smithi082 ceph-mon[98694]: pgmap v10121: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 301 KiB/s wr, 12 op/s 2023-09-10T04:30:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18675.336277 secs 2023-09-10T04:30:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18675.991381 secs 2023-09-10T04:30:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:44 smithi098 ceph-mon[103124]: pgmap v10121: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 301 KiB/s wr, 12 op/s 2023-09-10T04:30:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:46 smithi170 ceph-mon[102832]: pgmap v10122: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 424 KiB/s wr, 18 op/s 2023-09-10T04:30:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:46 smithi082 ceph-mon[98694]: pgmap v10122: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 424 KiB/s wr, 18 op/s 2023-09-10T04:30:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:46 smithi098 ceph-mon[103124]: pgmap v10122: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 424 KiB/s wr, 18 op/s 2023-09-10T04:30:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:48 smithi170 ceph-mon[102832]: pgmap v10123: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 273 KiB/s wr, 12 op/s 2023-09-10T04:30:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18680.991464 secs 2023-09-10T04:30:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:48 smithi082 ceph-mon[98694]: pgmap v10123: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 273 KiB/s wr, 12 op/s 2023-09-10T04:30:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18680.991464 secs 2023-09-10T04:30:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:48 smithi098 ceph-mon[103124]: pgmap v10123: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 273 KiB/s wr, 12 op/s 2023-09-10T04:30:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18680.991464 secs 2023-09-10T04:30:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18680.336333 secs 2023-09-10T04:30:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:30:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:30:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18680.336333 secs 2023-09-10T04:30:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:30:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:30:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18680.336333 secs 2023-09-10T04:30:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:30:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:30:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:50 smithi170 ceph-mon[102832]: pgmap v10124: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 273 KiB/s wr, 12 op/s 2023-09-10T04:30:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:50 smithi082 ceph-mon[98694]: pgmap v10124: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 273 KiB/s wr, 12 op/s 2023-09-10T04:30:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:50 smithi098 ceph-mon[103124]: pgmap v10124: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 273 KiB/s wr, 12 op/s 2023-09-10T04:30:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:30:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:30:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:30:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:30:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:30:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:30:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:52 smithi170 ceph-mon[102832]: pgmap v10125: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 439 KiB/s wr, 18 op/s 2023-09-10T04:30:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:52 smithi082 ceph-mon[98694]: pgmap v10125: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 439 KiB/s wr, 18 op/s 2023-09-10T04:30:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:52 smithi098 ceph-mon[103124]: pgmap v10125: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 439 KiB/s wr, 18 op/s 2023-09-10T04:30:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18684.163714 secs 2023-09-10T04:30:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18685.991487 secs 2023-09-10T04:30:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18684.163714 secs 2023-09-10T04:30:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18685.991487 secs 2023-09-10T04:30:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18684.163714 secs 2023-09-10T04:30:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18685.991487 secs 2023-09-10T04:30:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18685.336391 secs 2023-09-10T04:30:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:54 smithi170 ceph-mon[102832]: pgmap v10126: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 290 KiB/s wr, 11 op/s 2023-09-10T04:30:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18685.336391 secs 2023-09-10T04:30:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:54 smithi082 ceph-mon[98694]: pgmap v10126: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 290 KiB/s wr, 11 op/s 2023-09-10T04:30:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18685.336391 secs 2023-09-10T04:30:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:54 smithi098 ceph-mon[103124]: pgmap v10126: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 290 KiB/s wr, 11 op/s 2023-09-10T04:30:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:30:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:30:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:30:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:30:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:30:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:30:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:30:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:30:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:30:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:56 smithi170 ceph-mon[102832]: pgmap v10127: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 352 KiB/s wr, 17 op/s 2023-09-10T04:30:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:56 smithi082 ceph-mon[98694]: pgmap v10127: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 352 KiB/s wr, 17 op/s 2023-09-10T04:30:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:56 smithi098 ceph-mon[103124]: pgmap v10127: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 352 KiB/s wr, 17 op/s 2023-09-10T04:30:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18689.163774 secs 2023-09-10T04:30:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:58 smithi170 ceph-mon[102832]: pgmap v10128: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 229 KiB/s wr, 12 op/s 2023-09-10T04:30:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18690.991555 secs 2023-09-10T04:30:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18689.163774 secs 2023-09-10T04:30:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:58 smithi082 ceph-mon[98694]: pgmap v10128: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 229 KiB/s wr, 12 op/s 2023-09-10T04:30:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18690.991555 secs 2023-09-10T04:30:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18689.163774 secs 2023-09-10T04:30:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:58 smithi098 ceph-mon[103124]: pgmap v10128: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 229 KiB/s wr, 12 op/s 2023-09-10T04:30:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18690.991555 secs 2023-09-10T04:31:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:30:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18690.336474 secs 2023-09-10T04:31:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:30:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18690.336474 secs 2023-09-10T04:31:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:30:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18690.336474 secs 2023-09-10T04:31:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:00 smithi170 ceph-mon[102832]: pgmap v10129: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 229 KiB/s wr, 12 op/s 2023-09-10T04:31:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:00 smithi082 ceph-mon[98694]: pgmap v10129: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 229 KiB/s wr, 12 op/s 2023-09-10T04:31:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:00 smithi098 ceph-mon[103124]: pgmap v10129: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 229 KiB/s wr, 12 op/s 2023-09-10T04:31:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:02 smithi170 ceph-mon[102832]: pgmap v10130: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 329 KiB/s wr, 17 op/s 2023-09-10T04:31:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:02 smithi082 ceph-mon[98694]: pgmap v10130: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 329 KiB/s wr, 17 op/s 2023-09-10T04:31:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:02 smithi098 ceph-mon[103124]: pgmap v10130: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 329 KiB/s wr, 17 op/s 2023-09-10T04:31:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18694.163882 secs 2023-09-10T04:31:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18695.991622 secs 2023-09-10T04:31:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18694.163882 secs 2023-09-10T04:31:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18695.991622 secs 2023-09-10T04:31:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18694.163882 secs 2023-09-10T04:31:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18695.991622 secs 2023-09-10T04:31:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18695.336513 secs 2023-09-10T04:31:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:04 smithi170 ceph-mon[102832]: pgmap v10131: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T04:31:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18695.336513 secs 2023-09-10T04:31:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:04 smithi082 ceph-mon[98694]: pgmap v10131: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T04:31:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18695.336513 secs 2023-09-10T04:31:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:04 smithi098 ceph-mon[103124]: pgmap v10131: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T04:31:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:06 smithi170 ceph-mon[102832]: pgmap v10132: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 267 KiB/s wr, 31 op/s 2023-09-10T04:31:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:06 smithi082 ceph-mon[98694]: pgmap v10132: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 267 KiB/s wr, 31 op/s 2023-09-10T04:31:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:06 smithi098 ceph-mon[103124]: pgmap v10132: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 267 KiB/s wr, 31 op/s 2023-09-10T04:31:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18699.164005 secs 2023-09-10T04:31:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:08 smithi170 ceph-mon[102832]: pgmap v10133: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 204 KiB/s wr, 25 op/s 2023-09-10T04:31:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:08 smithi170 ceph-mon[102832]: osdmap e701: 12 total, 12 up, 12 in 2023-09-10T04:31:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18700.991689 secs 2023-09-10T04:31:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18699.164005 secs 2023-09-10T04:31:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:08 smithi082 ceph-mon[98694]: pgmap v10133: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 204 KiB/s wr, 25 op/s 2023-09-10T04:31:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:08 smithi082 ceph-mon[98694]: osdmap e701: 12 total, 12 up, 12 in 2023-09-10T04:31:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18700.991689 secs 2023-09-10T04:31:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18699.164005 secs 2023-09-10T04:31:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:08 smithi098 ceph-mon[103124]: pgmap v10133: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 204 KiB/s wr, 25 op/s 2023-09-10T04:31:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:08 smithi098 ceph-mon[103124]: osdmap e701: 12 total, 12 up, 12 in 2023-09-10T04:31:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18700.991689 secs 2023-09-10T04:31:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18700.336622 secs 2023-09-10T04:31:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18700.336622 secs 2023-09-10T04:31:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18700.336622 secs 2023-09-10T04:31:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:10 smithi170 ceph-mon[102832]: pgmap v10135: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 245 KiB/s wr, 30 op/s 2023-09-10T04:31:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:10 smithi082 ceph-mon[98694]: pgmap v10135: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 245 KiB/s wr, 30 op/s 2023-09-10T04:31:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:10 smithi098 ceph-mon[103124]: pgmap v10135: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 245 KiB/s wr, 30 op/s 2023-09-10T04:31:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:12 smithi170 ceph-mon[102832]: pgmap v10136: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 126 KiB/s wr, 29 op/s 2023-09-10T04:31:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:12 smithi082 ceph-mon[98694]: pgmap v10136: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 126 KiB/s wr, 29 op/s 2023-09-10T04:31:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:12 smithi098 ceph-mon[103124]: pgmap v10136: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 126 KiB/s wr, 29 op/s 2023-09-10T04:31:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18704.164105 secs 2023-09-10T04:31:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18705.991755 secs 2023-09-10T04:31:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:13 smithi170 ceph-mon[102832]: osdmap e702: 12 total, 12 up, 12 in 2023-09-10T04:31:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18704.164105 secs 2023-09-10T04:31:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18705.991755 secs 2023-09-10T04:31:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:13 smithi082 ceph-mon[98694]: osdmap e702: 12 total, 12 up, 12 in 2023-09-10T04:31:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18704.164105 secs 2023-09-10T04:31:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18705.991755 secs 2023-09-10T04:31:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:13 smithi098 ceph-mon[103124]: osdmap e702: 12 total, 12 up, 12 in 2023-09-10T04:31:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18705.336705 secs 2023-09-10T04:31:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:14 smithi170 ceph-mon[102832]: pgmap v10138: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:31:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18705.336705 secs 2023-09-10T04:31:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:14 smithi082 ceph-mon[98694]: pgmap v10138: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:31:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18705.336705 secs 2023-09-10T04:31:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:14 smithi098 ceph-mon[103124]: pgmap v10138: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:31:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:16 smithi082 ceph-mon[98694]: pgmap v10139: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 154 KiB/s wr, 15 op/s 2023-09-10T04:31:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:16 smithi098 ceph-mon[103124]: pgmap v10139: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 154 KiB/s wr, 15 op/s 2023-09-10T04:31:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:16 smithi170 ceph-mon[102832]: pgmap v10139: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 154 KiB/s wr, 15 op/s 2023-09-10T04:31:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18709.164163 secs 2023-09-10T04:31:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18709.164163 secs 2023-09-10T04:31:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18709.164163 secs 2023-09-10T04:31:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:18 smithi082 ceph-mon[98694]: pgmap v10140: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T04:31:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18710.991823 secs 2023-09-10T04:31:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:18 smithi098 ceph-mon[103124]: pgmap v10140: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T04:31:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18710.991823 secs 2023-09-10T04:31:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:18 smithi170 ceph-mon[102832]: pgmap v10140: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T04:31:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18710.991823 secs 2023-09-10T04:31:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18710.336767 secs 2023-09-10T04:31:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18710.336767 secs 2023-09-10T04:31:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18710.336767 secs 2023-09-10T04:31:21.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:20 smithi082 ceph-mon[98694]: pgmap v10141: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:31:21.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:31:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:20 smithi098 ceph-mon[103124]: pgmap v10141: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:31:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:31:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:20 smithi170 ceph-mon[102832]: pgmap v10141: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:31:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:31:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:22 smithi082 ceph-mon[98694]: pgmap v10142: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:31:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18714.164226 secs 2023-09-10T04:31:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:22 smithi098 ceph-mon[103124]: pgmap v10142: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:31:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18714.164226 secs 2023-09-10T04:31:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:22 smithi170 ceph-mon[102832]: pgmap v10142: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:31:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18714.164226 secs 2023-09-10T04:31:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:31:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:31:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:31:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:31:24.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18715.991892 secs 2023-09-10T04:31:24.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:24.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:24.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:24.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:31:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:31:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:31:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:31:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18715.991892 secs 2023-09-10T04:31:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:31:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:31:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:31:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:31:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18715.991892 secs 2023-09-10T04:31:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18715.336833 secs 2023-09-10T04:31:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:25 smithi170 ceph-mon[102832]: pgmap v10143: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T04:31:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:25 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 32102k 2023-09-10T04:31:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:25 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 32872820: error parsing value: Value '32872820' is below minimum 939524096 2023-09-10T04:31:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:31:25.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:31:25.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:25.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:25.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:25.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:25.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:25 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T04:31:25.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:25 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T04:31:25.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:31:25.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:31:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18715.336833 secs 2023-09-10T04:31:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:25 smithi082 ceph-mon[98694]: pgmap v10143: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T04:31:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:25 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 32102k 2023-09-10T04:31:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:25 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 32872820: error parsing value: Value '32872820' is below minimum 939524096 2023-09-10T04:31:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:31:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:31:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:25.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:25.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:25.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:25.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:25 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T04:31:25.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:25 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T04:31:25.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:31:25.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:31:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18715.336833 secs 2023-09-10T04:31:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:25 smithi098 ceph-mon[103124]: pgmap v10143: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T04:31:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:25 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 32102k 2023-09-10T04:31:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:25 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 32872820: error parsing value: Value '32872820' is below minimum 939524096 2023-09-10T04:31:25.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:31:25.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:31:25.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:25.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:25.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:25.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:25.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:25 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T04:31:25.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:25 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T04:31:25.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:31:25.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:31:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:26 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:31:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:31:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:26.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:26.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:26 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T04:31:26.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:26 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T04:31:26.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:31:26.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:31:26.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:31:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:26 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:31:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:31:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:26.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:26 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T04:31:26.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:26 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T04:31:26.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:31:26.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:31:26.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:31:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:26 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:31:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:31:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:31:26.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:26 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T04:31:26.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:26 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T04:31:26.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:31:26.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:31:26.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:31:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:27 smithi170 ceph-mon[102832]: pgmap v10144: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 166 KiB/s wr, 15 op/s 2023-09-10T04:31:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:27 smithi082 ceph-mon[98694]: pgmap v10144: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 166 KiB/s wr, 15 op/s 2023-09-10T04:31:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:27 smithi098 ceph-mon[103124]: pgmap v10144: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 166 KiB/s wr, 15 op/s 2023-09-10T04:31:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:29 smithi170 ceph-mon[102832]: pgmap v10145: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T04:31:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18720.336892 secs 2023-09-10T04:31:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18720.991957 secs 2023-09-10T04:31:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:29 smithi082 ceph-mon[98694]: pgmap v10145: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T04:31:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18720.336892 secs 2023-09-10T04:31:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18720.991957 secs 2023-09-10T04:31:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:29 smithi098 ceph-mon[103124]: pgmap v10145: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T04:31:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18720.336892 secs 2023-09-10T04:31:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18720.991957 secs 2023-09-10T04:31:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:31 smithi082 ceph-mon[98694]: pgmap v10146: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T04:31:31.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:31:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:31 smithi098 ceph-mon[103124]: pgmap v10146: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T04:31:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:31:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:31 smithi170 ceph-mon[102832]: pgmap v10146: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T04:31:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:31:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:31:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:31:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:31:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:31:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:31:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:31:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:31:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:31:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:31:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:33 smithi082 ceph-mon[98694]: pgmap v10147: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 232 KiB/s wr, 15 op/s 2023-09-10T04:31:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:33 smithi098 ceph-mon[103124]: pgmap v10147: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 232 KiB/s wr, 15 op/s 2023-09-10T04:31:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:33 smithi170 ceph-mon[102832]: pgmap v10147: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 232 KiB/s wr, 15 op/s 2023-09-10T04:31:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18725.336949 secs 2023-09-10T04:31:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18725.992023 secs 2023-09-10T04:31:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18725.336949 secs 2023-09-10T04:31:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18725.992023 secs 2023-09-10T04:31:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18725.336949 secs 2023-09-10T04:31:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18725.992023 secs 2023-09-10T04:31:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:35 smithi082 ceph-mon[98694]: pgmap v10148: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 232 KiB/s wr, 10 op/s 2023-09-10T04:31:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:35 smithi098 ceph-mon[103124]: pgmap v10148: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 232 KiB/s wr, 10 op/s 2023-09-10T04:31:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:35 smithi170 ceph-mon[102832]: pgmap v10148: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 232 KiB/s wr, 10 op/s 2023-09-10T04:31:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:37 smithi082 ceph-mon[98694]: pgmap v10149: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 370 KiB/s wr, 17 op/s 2023-09-10T04:31:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:37 smithi098 ceph-mon[103124]: pgmap v10149: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 370 KiB/s wr, 17 op/s 2023-09-10T04:31:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:37 smithi170 ceph-mon[102832]: pgmap v10149: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 370 KiB/s wr, 17 op/s 2023-09-10T04:31:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:31:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:31:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:31:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:31:38.610 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:31:38.611 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:31:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:39 smithi098 ceph-mon[103124]: pgmap v10150: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 306 KiB/s wr, 11 op/s 2023-09-10T04:31:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18730.337007 secs 2023-09-10T04:31:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18730.992086 secs 2023-09-10T04:31:39.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:39 smithi170 ceph-mon[102832]: pgmap v10150: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 306 KiB/s wr, 11 op/s 2023-09-10T04:31:39.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18730.337007 secs 2023-09-10T04:31:39.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18730.992086 secs 2023-09-10T04:31:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:39 smithi082 ceph-mon[98694]: pgmap v10150: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 306 KiB/s wr, 11 op/s 2023-09-10T04:31:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18730.337007 secs 2023-09-10T04:31:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18730.992086 secs 2023-09-10T04:31:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:31:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:31:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:40 smithi170 ceph-mon[102832]: pgmap v10151: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 306 KiB/s wr, 11 op/s 2023-09-10T04:31:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:31:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:31:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:40 smithi082 ceph-mon[98694]: pgmap v10151: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 306 KiB/s wr, 11 op/s 2023-09-10T04:31:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:31:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:31:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:40 smithi098 ceph-mon[103124]: pgmap v10151: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 306 KiB/s wr, 11 op/s 2023-09-10T04:31:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:42 smithi170 ceph-mon[102832]: pgmap v10152: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 470 KiB/s wr, 18 op/s 2023-09-10T04:31:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:42 smithi082 ceph-mon[98694]: pgmap v10152: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 470 KiB/s wr, 18 op/s 2023-09-10T04:31:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:42 smithi098 ceph-mon[103124]: pgmap v10152: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 470 KiB/s wr, 18 op/s 2023-09-10T04:31:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18735.992153 secs 2023-09-10T04:31:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18735.992153 secs 2023-09-10T04:31:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18735.992153 secs 2023-09-10T04:31:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18735.337067 secs 2023-09-10T04:31:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:44 smithi170 ceph-mon[102832]: pgmap v10153: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 302 KiB/s wr, 12 op/s 2023-09-10T04:31:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18735.337067 secs 2023-09-10T04:31:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:44 smithi082 ceph-mon[98694]: pgmap v10153: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 302 KiB/s wr, 12 op/s 2023-09-10T04:31:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18735.337067 secs 2023-09-10T04:31:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:44 smithi098 ceph-mon[103124]: pgmap v10153: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 302 KiB/s wr, 12 op/s 2023-09-10T04:31:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:46 smithi170 ceph-mon[102832]: pgmap v10154: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 426 KiB/s wr, 17 op/s 2023-09-10T04:31:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:46 smithi082 ceph-mon[98694]: pgmap v10154: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 426 KiB/s wr, 17 op/s 2023-09-10T04:31:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:46 smithi098 ceph-mon[103124]: pgmap v10154: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 426 KiB/s wr, 17 op/s 2023-09-10T04:31:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:48 smithi170 ceph-mon[102832]: pgmap v10155: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 288 KiB/s wr, 11 op/s 2023-09-10T04:31:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18740.992218 secs 2023-09-10T04:31:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:48 smithi082 ceph-mon[98694]: pgmap v10155: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 288 KiB/s wr, 11 op/s 2023-09-10T04:31:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18740.992218 secs 2023-09-10T04:31:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:48 smithi098 ceph-mon[103124]: pgmap v10155: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 288 KiB/s wr, 11 op/s 2023-09-10T04:31:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18740.992218 secs 2023-09-10T04:31:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18740.337122 secs 2023-09-10T04:31:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:31:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:31:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18740.337122 secs 2023-09-10T04:31:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:31:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:31:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18740.337122 secs 2023-09-10T04:31:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:31:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:31:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:50 smithi170 ceph-mon[102832]: pgmap v10156: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 288 KiB/s wr, 11 op/s 2023-09-10T04:31:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:50 smithi082 ceph-mon[98694]: pgmap v10156: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 288 KiB/s wr, 11 op/s 2023-09-10T04:31:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:50 smithi098 ceph-mon[103124]: pgmap v10156: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 288 KiB/s wr, 11 op/s 2023-09-10T04:31:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:31:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:31:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:31:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:31:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:31:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:31:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:52 smithi170 ceph-mon[102832]: pgmap v10157: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 455 KiB/s wr, 17 op/s 2023-09-10T04:31:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:52 smithi082 ceph-mon[98694]: pgmap v10157: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 455 KiB/s wr, 17 op/s 2023-09-10T04:31:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:52 smithi098 ceph-mon[103124]: pgmap v10157: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 455 KiB/s wr, 17 op/s 2023-09-10T04:31:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18744.164674 secs 2023-09-10T04:31:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18745.992311 secs 2023-09-10T04:31:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18744.164674 secs 2023-09-10T04:31:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18745.992311 secs 2023-09-10T04:31:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18744.164674 secs 2023-09-10T04:31:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18745.992311 secs 2023-09-10T04:31:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18745.337179 secs 2023-09-10T04:31:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:54 smithi170 ceph-mon[102832]: pgmap v10158: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 291 KiB/s wr, 11 op/s 2023-09-10T04:31:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18745.337179 secs 2023-09-10T04:31:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:54 smithi082 ceph-mon[98694]: pgmap v10158: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 291 KiB/s wr, 11 op/s 2023-09-10T04:31:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18745.337179 secs 2023-09-10T04:31:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:54 smithi098 ceph-mon[103124]: pgmap v10158: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 291 KiB/s wr, 11 op/s 2023-09-10T04:31:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:31:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:31:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:31:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:31:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:31:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:31:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:31:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:31:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:31:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:56 smithi170 ceph-mon[102832]: pgmap v10159: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 354 KiB/s wr, 16 op/s 2023-09-10T04:31:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:56 smithi082 ceph-mon[98694]: pgmap v10159: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 354 KiB/s wr, 16 op/s 2023-09-10T04:31:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:56 smithi098 ceph-mon[103124]: pgmap v10159: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 354 KiB/s wr, 16 op/s 2023-09-10T04:31:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18749.164784 secs 2023-09-10T04:31:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:58 smithi170 ceph-mon[102832]: pgmap v10160: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T04:31:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18750.992383 secs 2023-09-10T04:31:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18749.164784 secs 2023-09-10T04:31:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:58 smithi082 ceph-mon[98694]: pgmap v10160: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T04:31:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18750.992383 secs 2023-09-10T04:31:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18749.164784 secs 2023-09-10T04:31:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:58 smithi098 ceph-mon[103124]: pgmap v10160: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T04:31:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18750.992383 secs 2023-09-10T04:32:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:31:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18750.337261 secs 2023-09-10T04:32:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:31:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18750.337261 secs 2023-09-10T04:32:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:31:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18750.337261 secs 2023-09-10T04:32:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:00 smithi082 ceph-mon[98694]: pgmap v10161: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T04:32:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:00 smithi098 ceph-mon[103124]: pgmap v10161: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T04:32:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:00 smithi170 ceph-mon[102832]: pgmap v10161: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T04:32:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:02 smithi082 ceph-mon[98694]: pgmap v10162: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 330 KiB/s wr, 16 op/s 2023-09-10T04:32:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:02 smithi098 ceph-mon[103124]: pgmap v10162: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 330 KiB/s wr, 16 op/s 2023-09-10T04:32:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:02 smithi170 ceph-mon[102832]: pgmap v10162: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 330 KiB/s wr, 16 op/s 2023-09-10T04:32:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18754.164850 secs 2023-09-10T04:32:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18755.992449 secs 2023-09-10T04:32:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18754.164850 secs 2023-09-10T04:32:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18755.992449 secs 2023-09-10T04:32:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18754.164850 secs 2023-09-10T04:32:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18755.992449 secs 2023-09-10T04:32:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18755.337344 secs 2023-09-10T04:32:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:04 smithi082 ceph-mon[98694]: pgmap v10163: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 163 KiB/s wr, 10 op/s 2023-09-10T04:32:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18755.337344 secs 2023-09-10T04:32:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:04 smithi098 ceph-mon[103124]: pgmap v10163: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 163 KiB/s wr, 10 op/s 2023-09-10T04:32:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18755.337344 secs 2023-09-10T04:32:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:04 smithi170 ceph-mon[102832]: pgmap v10163: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 163 KiB/s wr, 10 op/s 2023-09-10T04:32:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:06 smithi082 ceph-mon[98694]: pgmap v10164: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 268 KiB/s wr, 30 op/s 2023-09-10T04:32:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:06 smithi098 ceph-mon[103124]: pgmap v10164: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 268 KiB/s wr, 30 op/s 2023-09-10T04:32:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:06 smithi170 ceph-mon[102832]: pgmap v10164: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 268 KiB/s wr, 30 op/s 2023-09-10T04:32:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18759.164977 secs 2023-09-10T04:32:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:08 smithi082 ceph-mon[98694]: pgmap v10165: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 205 KiB/s wr, 24 op/s 2023-09-10T04:32:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:08 smithi082 ceph-mon[98694]: osdmap e703: 12 total, 12 up, 12 in 2023-09-10T04:32:09.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18760.992482 secs 2023-09-10T04:32:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18759.164977 secs 2023-09-10T04:32:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:08 smithi098 ceph-mon[103124]: pgmap v10165: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 205 KiB/s wr, 24 op/s 2023-09-10T04:32:09.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:08 smithi098 ceph-mon[103124]: osdmap e703: 12 total, 12 up, 12 in 2023-09-10T04:32:09.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18760.992482 secs 2023-09-10T04:32:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18759.164977 secs 2023-09-10T04:32:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:08 smithi170 ceph-mon[102832]: pgmap v10165: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 205 KiB/s wr, 24 op/s 2023-09-10T04:32:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:08 smithi170 ceph-mon[102832]: osdmap e703: 12 total, 12 up, 12 in 2023-09-10T04:32:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18760.992482 secs 2023-09-10T04:32:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18760.337403 secs 2023-09-10T04:32:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18760.337403 secs 2023-09-10T04:32:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18760.337403 secs 2023-09-10T04:32:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:10 smithi082 ceph-mon[98694]: pgmap v10167: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 246 KiB/s wr, 29 op/s 2023-09-10T04:32:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:10 smithi098 ceph-mon[103124]: pgmap v10167: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 246 KiB/s wr, 29 op/s 2023-09-10T04:32:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:10 smithi170 ceph-mon[102832]: pgmap v10167: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 246 KiB/s wr, 29 op/s 2023-09-10T04:32:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:12 smithi082 ceph-mon[98694]: pgmap v10168: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 126 KiB/s wr, 29 op/s 2023-09-10T04:32:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:12 smithi098 ceph-mon[103124]: pgmap v10168: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 126 KiB/s wr, 29 op/s 2023-09-10T04:32:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:12 smithi170 ceph-mon[102832]: pgmap v10168: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 126 KiB/s wr, 29 op/s 2023-09-10T04:32:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18764.165039 secs 2023-09-10T04:32:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18765.992548 secs 2023-09-10T04:32:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:13 smithi082 ceph-mon[98694]: osdmap e704: 12 total, 12 up, 12 in 2023-09-10T04:32:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18764.165039 secs 2023-09-10T04:32:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18765.992548 secs 2023-09-10T04:32:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:13 smithi098 ceph-mon[103124]: osdmap e704: 12 total, 12 up, 12 in 2023-09-10T04:32:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18764.165039 secs 2023-09-10T04:32:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18765.992548 secs 2023-09-10T04:32:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:13 smithi170 ceph-mon[102832]: osdmap e704: 12 total, 12 up, 12 in 2023-09-10T04:32:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18765.337460 secs 2023-09-10T04:32:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:14 smithi082 ceph-mon[98694]: pgmap v10170: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 6 op/s 2023-09-10T04:32:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18765.337460 secs 2023-09-10T04:32:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:14 smithi098 ceph-mon[103124]: pgmap v10170: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 6 op/s 2023-09-10T04:32:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18765.337460 secs 2023-09-10T04:32:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:14 smithi170 ceph-mon[102832]: pgmap v10170: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 6 op/s 2023-09-10T04:32:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:16 smithi082 ceph-mon[98694]: pgmap v10171: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 15 op/s 2023-09-10T04:32:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:16 smithi098 ceph-mon[103124]: pgmap v10171: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 15 op/s 2023-09-10T04:32:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:16 smithi170 ceph-mon[102832]: pgmap v10171: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 15 op/s 2023-09-10T04:32:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18769.165103 secs 2023-09-10T04:32:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:18 smithi098 ceph-mon[103124]: pgmap v10172: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T04:32:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18770.992614 secs 2023-09-10T04:32:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18769.165103 secs 2023-09-10T04:32:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:18 smithi170 ceph-mon[102832]: pgmap v10172: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T04:32:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18770.992614 secs 2023-09-10T04:32:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18769.165103 secs 2023-09-10T04:32:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:18 smithi082 ceph-mon[98694]: pgmap v10172: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T04:32:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18770.992614 secs 2023-09-10T04:32:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18770.337514 secs 2023-09-10T04:32:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18770.337514 secs 2023-09-10T04:32:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18770.337514 secs 2023-09-10T04:32:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:20 smithi098 ceph-mon[103124]: pgmap v10173: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T04:32:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:20 smithi170 ceph-mon[102832]: pgmap v10173: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T04:32:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:20 smithi082 ceph-mon[98694]: pgmap v10173: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T04:32:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:22 smithi098 ceph-mon[103124]: pgmap v10174: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:32:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18774.165161 secs 2023-09-10T04:32:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:22 smithi170 ceph-mon[102832]: pgmap v10174: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:32:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18774.165161 secs 2023-09-10T04:32:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:22 smithi082 ceph-mon[98694]: pgmap v10174: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:32:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18774.165161 secs 2023-09-10T04:32:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:32:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:32:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18775.992680 secs 2023-09-10T04:32:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:32:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:32:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18775.992680 secs 2023-09-10T04:32:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:32:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:32:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18775.992680 secs 2023-09-10T04:32:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18775.337601 secs 2023-09-10T04:32:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:24 smithi098 ceph-mon[103124]: pgmap v10175: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:32:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18775.337601 secs 2023-09-10T04:32:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:24 smithi170 ceph-mon[102832]: pgmap v10175: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:32:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18775.337601 secs 2023-09-10T04:32:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:24 smithi082 ceph-mon[98694]: pgmap v10175: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:32:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:25 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:32:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:25 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:32:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:32:26.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:32:26.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:32:26.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:32:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:25 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:32:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:25 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:32:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:32:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:32:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:32:26.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:32:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:25 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:32:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:25 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:32:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:32:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:32:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:32:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:32:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:26 smithi098 ceph-mon[103124]: pgmap v10176: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 167 KiB/s wr, 15 op/s 2023-09-10T04:32:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:26 smithi170 ceph-mon[102832]: pgmap v10176: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 167 KiB/s wr, 15 op/s 2023-09-10T04:32:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:26 smithi082 ceph-mon[98694]: pgmap v10176: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 167 KiB/s wr, 15 op/s 2023-09-10T04:32:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18779.165258 secs 2023-09-10T04:32:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18779.165258 secs 2023-09-10T04:32:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18779.165258 secs 2023-09-10T04:32:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:29 smithi098 ceph-mon[103124]: pgmap v10177: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T04:32:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18780.992747 secs 2023-09-10T04:32:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:29 smithi170 ceph-mon[102832]: pgmap v10177: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T04:32:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18780.992747 secs 2023-09-10T04:32:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:29 smithi082 ceph-mon[98694]: pgmap v10177: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T04:32:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18780.992747 secs 2023-09-10T04:32:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18780.337659 secs 2023-09-10T04:32:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18780.337659 secs 2023-09-10T04:32:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18780.337659 secs 2023-09-10T04:32:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:31 smithi170 ceph-mon[102832]: pgmap v10178: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T04:32:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:31 smithi082 ceph-mon[98694]: pgmap v10178: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T04:32:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:31 smithi098 ceph-mon[103124]: pgmap v10178: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T04:32:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:33 smithi170 ceph-mon[102832]: pgmap v10179: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 242 KiB/s wr, 16 op/s 2023-09-10T04:32:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:33 smithi082 ceph-mon[98694]: pgmap v10179: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 242 KiB/s wr, 16 op/s 2023-09-10T04:32:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:33 smithi098 ceph-mon[103124]: pgmap v10179: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 242 KiB/s wr, 16 op/s 2023-09-10T04:32:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18785.992812 secs 2023-09-10T04:32:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18785.992812 secs 2023-09-10T04:32:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18785.992812 secs 2023-09-10T04:32:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18785.337723 secs 2023-09-10T04:32:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:35 smithi170 ceph-mon[102832]: pgmap v10180: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 242 KiB/s wr, 11 op/s 2023-09-10T04:32:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:32:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18785.337723 secs 2023-09-10T04:32:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:35 smithi082 ceph-mon[98694]: pgmap v10180: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 242 KiB/s wr, 11 op/s 2023-09-10T04:32:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:32:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18785.337723 secs 2023-09-10T04:32:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:35 smithi098 ceph-mon[103124]: pgmap v10180: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 242 KiB/s wr, 11 op/s 2023-09-10T04:32:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:32:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:32:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:32:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:32:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:32:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:32:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:32:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:32:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:32:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:32:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:37 smithi170 ceph-mon[102832]: pgmap v10181: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 380 KiB/s wr, 17 op/s 2023-09-10T04:32:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:37 smithi082 ceph-mon[98694]: pgmap v10181: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 380 KiB/s wr, 17 op/s 2023-09-10T04:32:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:37 smithi098 ceph-mon[103124]: pgmap v10181: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 380 KiB/s wr, 17 op/s 2023-09-10T04:32:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:32:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:32:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:32:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:32:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:32:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:32:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:39 smithi170 ceph-mon[102832]: pgmap v10182: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 316 KiB/s wr, 11 op/s 2023-09-10T04:32:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18790.992875 secs 2023-09-10T04:32:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:39 smithi082 ceph-mon[98694]: pgmap v10182: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 316 KiB/s wr, 11 op/s 2023-09-10T04:32:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18790.992875 secs 2023-09-10T04:32:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:39 smithi098 ceph-mon[103124]: pgmap v10182: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 316 KiB/s wr, 11 op/s 2023-09-10T04:32:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18790.992875 secs 2023-09-10T04:32:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18790.337780 secs 2023-09-10T04:32:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:32:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:32:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18790.337780 secs 2023-09-10T04:32:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:32:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:32:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18790.337780 secs 2023-09-10T04:32:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:32:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:32:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:41 smithi170 ceph-mon[102832]: pgmap v10183: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 316 KiB/s wr, 11 op/s 2023-09-10T04:32:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:41 smithi082 ceph-mon[98694]: pgmap v10183: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 316 KiB/s wr, 11 op/s 2023-09-10T04:32:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:41 smithi098 ceph-mon[103124]: pgmap v10183: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 316 KiB/s wr, 11 op/s 2023-09-10T04:32:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:43 smithi170 ceph-mon[102832]: pgmap v10184: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 481 KiB/s wr, 18 op/s 2023-09-10T04:32:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:43 smithi082 ceph-mon[98694]: pgmap v10184: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 481 KiB/s wr, 18 op/s 2023-09-10T04:32:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:43 smithi098 ceph-mon[103124]: pgmap v10184: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 481 KiB/s wr, 18 op/s 2023-09-10T04:32:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18795.992938 secs 2023-09-10T04:32:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18795.992938 secs 2023-09-10T04:32:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18795.992938 secs 2023-09-10T04:32:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18795.337858 secs 2023-09-10T04:32:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:45 smithi170 ceph-mon[102832]: pgmap v10185: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 303 KiB/s wr, 12 op/s 2023-09-10T04:32:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18795.337858 secs 2023-09-10T04:32:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:45 smithi082 ceph-mon[98694]: pgmap v10185: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 303 KiB/s wr, 12 op/s 2023-09-10T04:32:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18795.337858 secs 2023-09-10T04:32:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:45 smithi098 ceph-mon[103124]: pgmap v10185: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 303 KiB/s wr, 12 op/s 2023-09-10T04:32:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:47 smithi170 ceph-mon[102832]: pgmap v10186: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 428 KiB/s wr, 18 op/s 2023-09-10T04:32:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:47 smithi082 ceph-mon[98694]: pgmap v10186: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 428 KiB/s wr, 18 op/s 2023-09-10T04:32:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:47 smithi098 ceph-mon[103124]: pgmap v10186: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 428 KiB/s wr, 18 op/s 2023-09-10T04:32:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:49 smithi170 ceph-mon[102832]: pgmap v10187: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 290 KiB/s wr, 12 op/s 2023-09-10T04:32:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18800.993003 secs 2023-09-10T04:32:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:49 smithi082 ceph-mon[98694]: pgmap v10187: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 290 KiB/s wr, 12 op/s 2023-09-10T04:32:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18800.993003 secs 2023-09-10T04:32:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:49 smithi098 ceph-mon[103124]: pgmap v10187: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 290 KiB/s wr, 12 op/s 2023-09-10T04:32:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18800.993003 secs 2023-09-10T04:32:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18800.337916 secs 2023-09-10T04:32:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:32:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:32:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18800.337916 secs 2023-09-10T04:32:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:32:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:32:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18800.337916 secs 2023-09-10T04:32:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:32:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:32:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:51 smithi170 ceph-mon[102832]: pgmap v10188: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 290 KiB/s wr, 12 op/s 2023-09-10T04:32:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:51 smithi082 ceph-mon[98694]: pgmap v10188: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 290 KiB/s wr, 12 op/s 2023-09-10T04:32:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:51 smithi098 ceph-mon[103124]: pgmap v10188: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 290 KiB/s wr, 12 op/s 2023-09-10T04:32:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:32:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:32:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:32:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:32:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:32:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:32:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:53 smithi170 ceph-mon[102832]: pgmap v10189: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 458 KiB/s wr, 18 op/s 2023-09-10T04:32:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18804.165631 secs 2023-09-10T04:32:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:53 smithi082 ceph-mon[98694]: pgmap v10189: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 458 KiB/s wr, 18 op/s 2023-09-10T04:32:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18804.165631 secs 2023-09-10T04:32:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:53 smithi098 ceph-mon[103124]: pgmap v10189: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 458 KiB/s wr, 18 op/s 2023-09-10T04:32:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18804.165631 secs 2023-09-10T04:32:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18805.337975 secs 2023-09-10T04:32:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18805.993067 secs 2023-09-10T04:32:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18805.337975 secs 2023-09-10T04:32:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18805.993067 secs 2023-09-10T04:32:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18805.337975 secs 2023-09-10T04:32:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18805.993067 secs 2023-09-10T04:32:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:55 smithi170 ceph-mon[102832]: pgmap v10190: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T04:32:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:55 smithi082 ceph-mon[98694]: pgmap v10190: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T04:32:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:55 smithi098 ceph-mon[103124]: pgmap v10190: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T04:32:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:32:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:32:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:32:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:32:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:32:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:32:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:32:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:32:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:32:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:57 smithi170 ceph-mon[102832]: pgmap v10191: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 419 KiB/s wr, 18 op/s 2023-09-10T04:32:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:57 smithi082 ceph-mon[98694]: pgmap v10191: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 419 KiB/s wr, 18 op/s 2023-09-10T04:32:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:57 smithi098 ceph-mon[103124]: pgmap v10191: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 419 KiB/s wr, 18 op/s 2023-09-10T04:32:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18809.165715 secs 2023-09-10T04:32:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18809.165715 secs 2023-09-10T04:32:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18809.165715 secs 2023-09-10T04:32:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:59 smithi170 ceph-mon[102832]: pgmap v10192: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T04:32:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18810.338032 secs 2023-09-10T04:32:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:32:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18810.993133 secs 2023-09-10T04:32:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:59 smithi082 ceph-mon[98694]: pgmap v10192: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T04:32:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18810.338032 secs 2023-09-10T04:32:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:32:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18810.993133 secs 2023-09-10T04:32:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:59 smithi098 ceph-mon[103124]: pgmap v10192: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T04:32:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18810.338032 secs 2023-09-10T04:32:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:32:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18810.993133 secs 2023-09-10T04:33:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:01 smithi170 ceph-mon[102832]: pgmap v10193: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T04:33:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:01 smithi082 ceph-mon[98694]: pgmap v10193: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T04:33:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:01 smithi098 ceph-mon[103124]: pgmap v10193: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T04:33:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:03 smithi170 ceph-mon[102832]: pgmap v10194: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 385 KiB/s wr, 18 op/s 2023-09-10T04:33:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18814.165773 secs 2023-09-10T04:33:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:03 smithi082 ceph-mon[98694]: pgmap v10194: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 385 KiB/s wr, 18 op/s 2023-09-10T04:33:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18814.165773 secs 2023-09-10T04:33:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:03 smithi098 ceph-mon[103124]: pgmap v10194: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 385 KiB/s wr, 18 op/s 2023-09-10T04:33:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18814.165773 secs 2023-09-10T04:33:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18815.338120 secs 2023-09-10T04:33:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18815.993199 secs 2023-09-10T04:33:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18815.338120 secs 2023-09-10T04:33:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18815.993199 secs 2023-09-10T04:33:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18815.338120 secs 2023-09-10T04:33:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18815.993199 secs 2023-09-10T04:33:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:05 smithi170 ceph-mon[102832]: pgmap v10195: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 217 KiB/s wr, 12 op/s 2023-09-10T04:33:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:05 smithi082 ceph-mon[98694]: pgmap v10195: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 217 KiB/s wr, 12 op/s 2023-09-10T04:33:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:05 smithi098 ceph-mon[103124]: pgmap v10195: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 217 KiB/s wr, 12 op/s 2023-09-10T04:33:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:07 smithi170 ceph-mon[102832]: pgmap v10196: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 308 KiB/s wr, 32 op/s 2023-09-10T04:33:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:07 smithi082 ceph-mon[98694]: pgmap v10196: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 308 KiB/s wr, 32 op/s 2023-09-10T04:33:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:07 smithi098 ceph-mon[103124]: pgmap v10196: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 308 KiB/s wr, 32 op/s 2023-09-10T04:33:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18819.165866 secs 2023-09-10T04:33:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18819.165866 secs 2023-09-10T04:33:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18819.165866 secs 2023-09-10T04:33:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:09 smithi170 ceph-mon[102832]: pgmap v10197: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 182 KiB/s wr, 26 op/s 2023-09-10T04:33:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18820.338177 secs 2023-09-10T04:33:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:09 smithi170 ceph-mon[102832]: osdmap e705: 12 total, 12 up, 12 in 2023-09-10T04:33:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18820.993260 secs 2023-09-10T04:33:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:09 smithi082 ceph-mon[98694]: pgmap v10197: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 182 KiB/s wr, 26 op/s 2023-09-10T04:33:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18820.338177 secs 2023-09-10T04:33:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:09 smithi082 ceph-mon[98694]: osdmap e705: 12 total, 12 up, 12 in 2023-09-10T04:33:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18820.993260 secs 2023-09-10T04:33:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:09 smithi098 ceph-mon[103124]: pgmap v10197: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 182 KiB/s wr, 26 op/s 2023-09-10T04:33:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18820.338177 secs 2023-09-10T04:33:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:09 smithi098 ceph-mon[103124]: osdmap e705: 12 total, 12 up, 12 in 2023-09-10T04:33:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18820.993260 secs 2023-09-10T04:33:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:33:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:33:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:33:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:33:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:33:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:33:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:11 smithi170 ceph-mon[102832]: pgmap v10199: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 218 KiB/s wr, 31 op/s 2023-09-10T04:33:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:11 smithi082 ceph-mon[98694]: pgmap v10199: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 218 KiB/s wr, 31 op/s 2023-09-10T04:33:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:11 smithi098 ceph-mon[103124]: pgmap v10199: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 218 KiB/s wr, 31 op/s 2023-09-10T04:33:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:33:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:33:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:33:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:33:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:33:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:33:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:13 smithi170 ceph-mon[102832]: pgmap v10200: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 110 KiB/s wr, 30 op/s 2023-09-10T04:33:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18824.165925 secs 2023-09-10T04:33:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:13 smithi082 ceph-mon[98694]: pgmap v10200: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 110 KiB/s wr, 30 op/s 2023-09-10T04:33:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18824.165925 secs 2023-09-10T04:33:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:13 smithi098 ceph-mon[103124]: pgmap v10200: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 110 KiB/s wr, 30 op/s 2023-09-10T04:33:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18824.165925 secs 2023-09-10T04:33:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18825.338251 secs 2023-09-10T04:33:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18825.993325 secs 2023-09-10T04:33:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:14 smithi170 ceph-mon[102832]: osdmap e706: 12 total, 12 up, 12 in 2023-09-10T04:33:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18825.338251 secs 2023-09-10T04:33:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18825.993325 secs 2023-09-10T04:33:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:14 smithi082 ceph-mon[98694]: osdmap e706: 12 total, 12 up, 12 in 2023-09-10T04:33:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18825.338251 secs 2023-09-10T04:33:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18825.993325 secs 2023-09-10T04:33:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:14 smithi098 ceph-mon[103124]: osdmap e706: 12 total, 12 up, 12 in 2023-09-10T04:33:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:15 smithi170 ceph-mon[102832]: pgmap v10202: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:33:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:15 smithi082 ceph-mon[98694]: pgmap v10202: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:33:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:15 smithi098 ceph-mon[103124]: pgmap v10202: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:33:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:17 smithi170 ceph-mon[102832]: pgmap v10203: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 155 KiB/s wr, 16 op/s 2023-09-10T04:33:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:17 smithi082 ceph-mon[98694]: pgmap v10203: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 155 KiB/s wr, 16 op/s 2023-09-10T04:33:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:17 smithi098 ceph-mon[103124]: pgmap v10203: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 155 KiB/s wr, 16 op/s 2023-09-10T04:33:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18829.165986 secs 2023-09-10T04:33:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18829.165986 secs 2023-09-10T04:33:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18829.165986 secs 2023-09-10T04:33:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:19 smithi170 ceph-mon[102832]: pgmap v10204: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 13 op/s 2023-09-10T04:33:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18830.338334 secs 2023-09-10T04:33:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18830.993389 secs 2023-09-10T04:33:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:19 smithi082 ceph-mon[98694]: pgmap v10204: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 13 op/s 2023-09-10T04:33:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18830.338334 secs 2023-09-10T04:33:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18830.993389 secs 2023-09-10T04:33:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:19 smithi098 ceph-mon[103124]: pgmap v10204: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 13 op/s 2023-09-10T04:33:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18830.338334 secs 2023-09-10T04:33:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18830.993389 secs 2023-09-10T04:33:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:21 smithi170 ceph-mon[102832]: pgmap v10205: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 13 op/s 2023-09-10T04:33:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:21 smithi082 ceph-mon[98694]: pgmap v10205: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 13 op/s 2023-09-10T04:33:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:21 smithi098 ceph-mon[103124]: pgmap v10205: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 13 op/s 2023-09-10T04:33:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:23 smithi170 ceph-mon[102832]: pgmap v10206: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T04:33:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18834.166050 secs 2023-09-10T04:33:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:23 smithi082 ceph-mon[98694]: pgmap v10206: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T04:33:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18834.166050 secs 2023-09-10T04:33:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:23 smithi098 ceph-mon[103124]: pgmap v10206: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T04:33:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18834.166050 secs 2023-09-10T04:33:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18835.338391 secs 2023-09-10T04:33:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18835.993454 secs 2023-09-10T04:33:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18835.338391 secs 2023-09-10T04:33:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18835.993454 secs 2023-09-10T04:33:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18835.338391 secs 2023-09-10T04:33:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18835.993454 secs 2023-09-10T04:33:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:25 smithi170 ceph-mon[102832]: pgmap v10207: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:33:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:25 smithi082 ceph-mon[98694]: pgmap v10207: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:33:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:25 smithi098 ceph-mon[103124]: pgmap v10207: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:33:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:33:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:33:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:33:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:27 smithi098 ceph-mon[103124]: pgmap v10208: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 167 KiB/s wr, 15 op/s 2023-09-10T04:33:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:33:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:33:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:33:27.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:33:27.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:33:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:27 smithi170 ceph-mon[102832]: pgmap v10208: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 167 KiB/s wr, 15 op/s 2023-09-10T04:33:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:33:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:33:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:33:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:33:27.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:33:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:27 smithi082 ceph-mon[98694]: pgmap v10208: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 167 KiB/s wr, 15 op/s 2023-09-10T04:33:27.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:33:27.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:33:27.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:33:27.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:33:27.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:33:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18839.166161 secs 2023-09-10T04:33:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18839.166161 secs 2023-09-10T04:33:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18839.166161 secs 2023-09-10T04:33:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:29 smithi098 ceph-mon[103124]: pgmap v10209: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T04:33:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18840.338447 secs 2023-09-10T04:33:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18840.993486 secs 2023-09-10T04:33:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:29 smithi170 ceph-mon[102832]: pgmap v10209: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T04:33:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18840.338447 secs 2023-09-10T04:33:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18840.993486 secs 2023-09-10T04:33:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:29 smithi082 ceph-mon[98694]: pgmap v10209: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T04:33:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18840.338447 secs 2023-09-10T04:33:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18840.993486 secs 2023-09-10T04:33:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:31 smithi098 ceph-mon[103124]: pgmap v10210: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T04:33:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:31 smithi170 ceph-mon[102832]: pgmap v10210: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T04:33:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:31 smithi082 ceph-mon[98694]: pgmap v10210: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T04:33:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:33 smithi098 ceph-mon[103124]: pgmap v10211: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 236 KiB/s wr, 15 op/s 2023-09-10T04:33:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18844.166224 secs 2023-09-10T04:33:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:33 smithi170 ceph-mon[102832]: pgmap v10211: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 236 KiB/s wr, 15 op/s 2023-09-10T04:33:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18844.166224 secs 2023-09-10T04:33:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:33 smithi082 ceph-mon[98694]: pgmap v10211: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 236 KiB/s wr, 15 op/s 2023-09-10T04:33:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18844.166224 secs 2023-09-10T04:33:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18845.338494 secs 2023-09-10T04:33:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18845.993548 secs 2023-09-10T04:33:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18845.338494 secs 2023-09-10T04:33:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18845.993548 secs 2023-09-10T04:33:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18845.338494 secs 2023-09-10T04:33:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18845.993548 secs 2023-09-10T04:33:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:35 smithi098 ceph-mon[103124]: pgmap v10212: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 236 KiB/s wr, 10 op/s 2023-09-10T04:33:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:35 smithi170 ceph-mon[102832]: pgmap v10212: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 236 KiB/s wr, 10 op/s 2023-09-10T04:33:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:35 smithi082 ceph-mon[98694]: pgmap v10212: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 236 KiB/s wr, 10 op/s 2023-09-10T04:33:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:37 smithi098 ceph-mon[103124]: pgmap v10213: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 373 KiB/s wr, 17 op/s 2023-09-10T04:33:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:37 smithi170 ceph-mon[102832]: pgmap v10213: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 373 KiB/s wr, 17 op/s 2023-09-10T04:33:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:37 smithi082 ceph-mon[98694]: pgmap v10213: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 373 KiB/s wr, 17 op/s 2023-09-10T04:33:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:33:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:33:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:33:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:33:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:33:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:33:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:39 smithi170 ceph-mon[102832]: pgmap v10214: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 309 KiB/s wr, 11 op/s 2023-09-10T04:33:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18850.338581 secs 2023-09-10T04:33:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18850.993612 secs 2023-09-10T04:33:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:33:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:39 smithi082 ceph-mon[98694]: pgmap v10214: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 309 KiB/s wr, 11 op/s 2023-09-10T04:33:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18850.338581 secs 2023-09-10T04:33:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18850.993612 secs 2023-09-10T04:33:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:33:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:39 smithi098 ceph-mon[103124]: pgmap v10214: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 309 KiB/s wr, 11 op/s 2023-09-10T04:33:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18850.338581 secs 2023-09-10T04:33:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18850.993612 secs 2023-09-10T04:33:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:33:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:33:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:33:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:33:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:33:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:33:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:33:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:33:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:33:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:33:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:41 smithi170 ceph-mon[102832]: pgmap v10215: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 309 KiB/s wr, 11 op/s 2023-09-10T04:33:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:41 smithi082 ceph-mon[98694]: pgmap v10215: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 309 KiB/s wr, 11 op/s 2023-09-10T04:33:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:41 smithi098 ceph-mon[103124]: pgmap v10215: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 309 KiB/s wr, 11 op/s 2023-09-10T04:33:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:43 smithi170 ceph-mon[102832]: pgmap v10216: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 474 KiB/s wr, 18 op/s 2023-09-10T04:33:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:43 smithi082 ceph-mon[98694]: pgmap v10216: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 474 KiB/s wr, 18 op/s 2023-09-10T04:33:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:43 smithi098 ceph-mon[103124]: pgmap v10216: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 474 KiB/s wr, 18 op/s 2023-09-10T04:33:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18855.338650 secs 2023-09-10T04:33:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18855.993674 secs 2023-09-10T04:33:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18855.338650 secs 2023-09-10T04:33:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18855.993674 secs 2023-09-10T04:33:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18855.338650 secs 2023-09-10T04:33:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18855.993674 secs 2023-09-10T04:33:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:45 smithi170 ceph-mon[102832]: pgmap v10217: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 303 KiB/s wr, 12 op/s 2023-09-10T04:33:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:45 smithi082 ceph-mon[98694]: pgmap v10217: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 303 KiB/s wr, 12 op/s 2023-09-10T04:33:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:45 smithi098 ceph-mon[103124]: pgmap v10217: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 303 KiB/s wr, 12 op/s 2023-09-10T04:33:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:47 smithi170 ceph-mon[102832]: pgmap v10218: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 428 KiB/s wr, 18 op/s 2023-09-10T04:33:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:47 smithi082 ceph-mon[98694]: pgmap v10218: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 428 KiB/s wr, 18 op/s 2023-09-10T04:33:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:47 smithi098 ceph-mon[103124]: pgmap v10218: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 428 KiB/s wr, 18 op/s 2023-09-10T04:33:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:49 smithi170 ceph-mon[102832]: pgmap v10219: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 291 KiB/s wr, 12 op/s 2023-09-10T04:33:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18860.338707 secs 2023-09-10T04:33:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18860.993740 secs 2023-09-10T04:33:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:49 smithi082 ceph-mon[98694]: pgmap v10219: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 291 KiB/s wr, 12 op/s 2023-09-10T04:33:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18860.338707 secs 2023-09-10T04:33:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18860.993740 secs 2023-09-10T04:33:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:49 smithi098 ceph-mon[103124]: pgmap v10219: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 291 KiB/s wr, 12 op/s 2023-09-10T04:33:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18860.338707 secs 2023-09-10T04:33:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18860.993740 secs 2023-09-10T04:33:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:33:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:33:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:33:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:33:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:33:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:33:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:51 smithi170 ceph-mon[102832]: pgmap v10220: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 291 KiB/s wr, 12 op/s 2023-09-10T04:33:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:51 smithi082 ceph-mon[98694]: pgmap v10220: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 291 KiB/s wr, 12 op/s 2023-09-10T04:33:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:51 smithi098 ceph-mon[103124]: pgmap v10220: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 291 KiB/s wr, 12 op/s 2023-09-10T04:33:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:33:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:33:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:33:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:33:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:33:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:33:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:53 smithi170 ceph-mon[102832]: pgmap v10221: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 468 KiB/s wr, 18 op/s 2023-09-10T04:33:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18864.166611 secs 2023-09-10T04:33:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:53 smithi082 ceph-mon[98694]: pgmap v10221: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 468 KiB/s wr, 18 op/s 2023-09-10T04:33:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18864.166611 secs 2023-09-10T04:33:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:53 smithi098 ceph-mon[103124]: pgmap v10221: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 468 KiB/s wr, 18 op/s 2023-09-10T04:33:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18864.166611 secs 2023-09-10T04:33:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18865.338765 secs 2023-09-10T04:33:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18865.993804 secs 2023-09-10T04:33:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18865.338765 secs 2023-09-10T04:33:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18865.993804 secs 2023-09-10T04:33:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18865.338765 secs 2023-09-10T04:33:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18865.993804 secs 2023-09-10T04:33:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:55 smithi170 ceph-mon[102832]: pgmap v10222: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 302 KiB/s wr, 11 op/s 2023-09-10T04:33:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:55 smithi082 ceph-mon[98694]: pgmap v10222: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 302 KiB/s wr, 11 op/s 2023-09-10T04:33:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:55 smithi098 ceph-mon[103124]: pgmap v10222: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 302 KiB/s wr, 11 op/s 2023-09-10T04:33:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:33:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:33:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:33:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:33:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:33:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:33:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:33:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:33:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:33:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:57 smithi170 ceph-mon[102832]: pgmap v10223: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 366 KiB/s wr, 17 op/s 2023-09-10T04:33:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:57 smithi082 ceph-mon[98694]: pgmap v10223: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 366 KiB/s wr, 17 op/s 2023-09-10T04:33:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:57 smithi098 ceph-mon[103124]: pgmap v10223: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 366 KiB/s wr, 17 op/s 2023-09-10T04:33:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18869.166691 secs 2023-09-10T04:33:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18869.166691 secs 2023-09-10T04:33:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18869.166691 secs 2023-09-10T04:33:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:59 smithi170 ceph-mon[102832]: pgmap v10224: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 240 KiB/s wr, 11 op/s 2023-09-10T04:33:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18870.338822 secs 2023-09-10T04:33:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:33:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18870.993870 secs 2023-09-10T04:33:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:59 smithi082 ceph-mon[98694]: pgmap v10224: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 240 KiB/s wr, 11 op/s 2023-09-10T04:33:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18870.338822 secs 2023-09-10T04:33:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:33:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18870.993870 secs 2023-09-10T04:34:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:59 smithi098 ceph-mon[103124]: pgmap v10224: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 240 KiB/s wr, 11 op/s 2023-09-10T04:34:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18870.338822 secs 2023-09-10T04:34:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:33:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18870.993870 secs 2023-09-10T04:34:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:01 smithi170 ceph-mon[102832]: pgmap v10225: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 240 KiB/s wr, 11 op/s 2023-09-10T04:34:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:01 smithi082 ceph-mon[98694]: pgmap v10225: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 240 KiB/s wr, 11 op/s 2023-09-10T04:34:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:01 smithi098 ceph-mon[103124]: pgmap v10225: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 240 KiB/s wr, 11 op/s 2023-09-10T04:34:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:03 smithi170 ceph-mon[102832]: pgmap v10226: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 342 KiB/s wr, 17 op/s 2023-09-10T04:34:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18874.216134 secs 2023-09-10T04:34:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:03 smithi082 ceph-mon[98694]: pgmap v10226: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 342 KiB/s wr, 17 op/s 2023-09-10T04:34:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18874.216134 secs 2023-09-10T04:34:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:03 smithi098 ceph-mon[103124]: pgmap v10226: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 342 KiB/s wr, 17 op/s 2023-09-10T04:34:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18874.216134 secs 2023-09-10T04:34:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18875.338895 secs 2023-09-10T04:34:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:34:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:34:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18875.993935 secs 2023-09-10T04:34:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18875.338895 secs 2023-09-10T04:34:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:34:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:34:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18875.993935 secs 2023-09-10T04:34:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18875.338895 secs 2023-09-10T04:34:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:34:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:34:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18875.993935 secs 2023-09-10T04:34:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:05 smithi170 ceph-mon[102832]: pgmap v10227: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 11 op/s 2023-09-10T04:34:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:34:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:34:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:05 smithi082 ceph-mon[98694]: pgmap v10227: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 11 op/s 2023-09-10T04:34:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:34:05.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:34:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:05 smithi098 ceph-mon[103124]: pgmap v10227: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 11 op/s 2023-09-10T04:34:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:34:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:34:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:07 smithi170 ceph-mon[102832]: pgmap v10228: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 269 KiB/s wr, 31 op/s 2023-09-10T04:34:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:07 smithi082 ceph-mon[98694]: pgmap v10228: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 269 KiB/s wr, 31 op/s 2023-09-10T04:34:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:07 smithi098 ceph-mon[103124]: pgmap v10228: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 269 KiB/s wr, 31 op/s 2023-09-10T04:34:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18879.216195 secs 2023-09-10T04:34:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:08 smithi170 ceph-mon[102832]: osdmap e707: 12 total, 12 up, 12 in 2023-09-10T04:34:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18879.216195 secs 2023-09-10T04:34:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:08 smithi082 ceph-mon[98694]: osdmap e707: 12 total, 12 up, 12 in 2023-09-10T04:34:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18879.216195 secs 2023-09-10T04:34:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:08 smithi098 ceph-mon[103124]: osdmap e707: 12 total, 12 up, 12 in 2023-09-10T04:34:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:09 smithi170 ceph-mon[102832]: pgmap v10230: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 247 KiB/s wr, 31 op/s 2023-09-10T04:34:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18880.338975 secs 2023-09-10T04:34:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18880.993998 secs 2023-09-10T04:34:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:09 smithi082 ceph-mon[98694]: pgmap v10230: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 247 KiB/s wr, 31 op/s 2023-09-10T04:34:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18880.338975 secs 2023-09-10T04:34:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18880.993998 secs 2023-09-10T04:34:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:09 smithi098 ceph-mon[103124]: pgmap v10230: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 247 KiB/s wr, 31 op/s 2023-09-10T04:34:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18880.338975 secs 2023-09-10T04:34:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18880.993998 secs 2023-09-10T04:34:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:11 smithi170 ceph-mon[102832]: pgmap v10231: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 247 KiB/s wr, 31 op/s 2023-09-10T04:34:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:11 smithi082 ceph-mon[98694]: pgmap v10231: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 247 KiB/s wr, 31 op/s 2023-09-10T04:34:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:11 smithi098 ceph-mon[103124]: pgmap v10231: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 247 KiB/s wr, 31 op/s 2023-09-10T04:34:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:13 smithi170 ceph-mon[102832]: pgmap v10232: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 126 KiB/s wr, 30 op/s 2023-09-10T04:34:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18884.216259 secs 2023-09-10T04:34:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:13 smithi082 ceph-mon[98694]: pgmap v10232: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 126 KiB/s wr, 30 op/s 2023-09-10T04:34:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18884.216259 secs 2023-09-10T04:34:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:13 smithi098 ceph-mon[103124]: pgmap v10232: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 126 KiB/s wr, 30 op/s 2023-09-10T04:34:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18884.216259 secs 2023-09-10T04:34:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18885.339065 secs 2023-09-10T04:34:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18885.994060 secs 2023-09-10T04:34:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:14 smithi170 ceph-mon[102832]: osdmap e708: 12 total, 12 up, 12 in 2023-09-10T04:34:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18885.339065 secs 2023-09-10T04:34:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18885.994060 secs 2023-09-10T04:34:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:14 smithi082 ceph-mon[98694]: osdmap e708: 12 total, 12 up, 12 in 2023-09-10T04:34:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18885.339065 secs 2023-09-10T04:34:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18885.994060 secs 2023-09-10T04:34:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:14 smithi098 ceph-mon[103124]: osdmap e708: 12 total, 12 up, 12 in 2023-09-10T04:34:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:15 smithi170 ceph-mon[102832]: pgmap v10234: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 895 B/s wr, 7 op/s 2023-09-10T04:34:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:15 smithi082 ceph-mon[98694]: pgmap v10234: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 895 B/s wr, 7 op/s 2023-09-10T04:34:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:15 smithi098 ceph-mon[103124]: pgmap v10234: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 895 B/s wr, 7 op/s 2023-09-10T04:34:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:17 smithi170 ceph-mon[102832]: pgmap v10235: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T04:34:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:17 smithi082 ceph-mon[98694]: pgmap v10235: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T04:34:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:17 smithi098 ceph-mon[103124]: pgmap v10235: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T04:34:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18889.216327 secs 2023-09-10T04:34:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18889.216327 secs 2023-09-10T04:34:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18889.216327 secs 2023-09-10T04:34:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:19 smithi170 ceph-mon[102832]: pgmap v10236: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T04:34:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18890.339131 secs 2023-09-10T04:34:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18890.994128 secs 2023-09-10T04:34:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:19 smithi082 ceph-mon[98694]: pgmap v10236: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T04:34:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18890.339131 secs 2023-09-10T04:34:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18890.994128 secs 2023-09-10T04:34:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:19 smithi098 ceph-mon[103124]: pgmap v10236: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T04:34:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18890.339131 secs 2023-09-10T04:34:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18890.994128 secs 2023-09-10T04:34:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:21 smithi170 ceph-mon[102832]: pgmap v10237: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T04:34:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:21 smithi082 ceph-mon[98694]: pgmap v10237: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T04:34:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:21 smithi098 ceph-mon[103124]: pgmap v10237: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T04:34:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:23 smithi170 ceph-mon[102832]: pgmap v10238: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T04:34:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18894.216453 secs 2023-09-10T04:34:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:23 smithi082 ceph-mon[98694]: pgmap v10238: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T04:34:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18894.216453 secs 2023-09-10T04:34:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:23 smithi098 ceph-mon[103124]: pgmap v10238: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T04:34:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18894.216453 secs 2023-09-10T04:34:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18895.339189 secs 2023-09-10T04:34:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18895.994192 secs 2023-09-10T04:34:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18895.339189 secs 2023-09-10T04:34:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18895.994192 secs 2023-09-10T04:34:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18895.339189 secs 2023-09-10T04:34:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18895.994192 secs 2023-09-10T04:34:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:25 smithi170 ceph-mon[102832]: pgmap v10239: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T04:34:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:25 smithi082 ceph-mon[98694]: pgmap v10239: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T04:34:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:25 smithi098 ceph-mon[103124]: pgmap v10239: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T04:34:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:27 smithi170 ceph-mon[102832]: pgmap v10240: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 168 KiB/s wr, 14 op/s 2023-09-10T04:34:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:34:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:34:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:34:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:34:27.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:34:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:27 smithi082 ceph-mon[98694]: pgmap v10240: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 168 KiB/s wr, 14 op/s 2023-09-10T04:34:27.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:34:27.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:34:27.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:34:27.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:34:27.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:34:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:27 smithi098 ceph-mon[103124]: pgmap v10240: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 168 KiB/s wr, 14 op/s 2023-09-10T04:34:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:34:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:34:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:34:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:34:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:34:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18899.216516 secs 2023-09-10T04:34:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:34:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:34:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:34:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18899.216516 secs 2023-09-10T04:34:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:34:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:34:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:34:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18899.216516 secs 2023-09-10T04:34:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:34:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:34:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:34:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:29 smithi170 ceph-mon[102832]: pgmap v10241: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T04:34:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18900.339249 secs 2023-09-10T04:34:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18900.994256 secs 2023-09-10T04:34:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:29 smithi082 ceph-mon[98694]: pgmap v10241: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T04:34:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18900.339249 secs 2023-09-10T04:34:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18900.994256 secs 2023-09-10T04:34:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:29 smithi098 ceph-mon[103124]: pgmap v10241: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T04:34:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18900.339249 secs 2023-09-10T04:34:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18900.994256 secs 2023-09-10T04:34:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:31 smithi170 ceph-mon[102832]: pgmap v10242: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T04:34:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:31 smithi082 ceph-mon[98694]: pgmap v10242: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T04:34:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:31 smithi098 ceph-mon[103124]: pgmap v10242: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T04:34:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:34:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:34:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:34:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:34:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:34:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:34:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:33 smithi170 ceph-mon[102832]: pgmap v10243: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 235 KiB/s wr, 15 op/s 2023-09-10T04:34:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18904.216591 secs 2023-09-10T04:34:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:34:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:34:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:33 smithi082 ceph-mon[98694]: pgmap v10243: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 235 KiB/s wr, 15 op/s 2023-09-10T04:34:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18904.216591 secs 2023-09-10T04:34:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:34:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:34:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:33 smithi098 ceph-mon[103124]: pgmap v10243: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 235 KiB/s wr, 15 op/s 2023-09-10T04:34:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18904.216591 secs 2023-09-10T04:34:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:34:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:34:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18905.339338 secs 2023-09-10T04:34:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18905.994319 secs 2023-09-10T04:34:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18905.339338 secs 2023-09-10T04:34:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18905.994319 secs 2023-09-10T04:34:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18905.339338 secs 2023-09-10T04:34:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18905.994319 secs 2023-09-10T04:34:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:35 smithi170 ceph-mon[102832]: pgmap v10244: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 235 KiB/s wr, 11 op/s 2023-09-10T04:34:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:35 smithi082 ceph-mon[98694]: pgmap v10244: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 235 KiB/s wr, 11 op/s 2023-09-10T04:34:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:35 smithi098 ceph-mon[103124]: pgmap v10244: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 235 KiB/s wr, 11 op/s 2023-09-10T04:34:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:37 smithi170 ceph-mon[102832]: pgmap v10245: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 374 KiB/s wr, 17 op/s 2023-09-10T04:34:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:37 smithi082 ceph-mon[98694]: pgmap v10245: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 374 KiB/s wr, 17 op/s 2023-09-10T04:34:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:37 smithi098 ceph-mon[103124]: pgmap v10245: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 374 KiB/s wr, 17 op/s 2023-09-10T04:34:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:39 smithi170 ceph-mon[102832]: pgmap v10246: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 310 KiB/s wr, 12 op/s 2023-09-10T04:34:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18910.339395 secs 2023-09-10T04:34:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18910.994388 secs 2023-09-10T04:34:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:34:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:39 smithi082 ceph-mon[98694]: pgmap v10246: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 310 KiB/s wr, 12 op/s 2023-09-10T04:34:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18910.339395 secs 2023-09-10T04:34:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18910.994388 secs 2023-09-10T04:34:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:34:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:39 smithi098 ceph-mon[103124]: pgmap v10246: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 310 KiB/s wr, 12 op/s 2023-09-10T04:34:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18910.339395 secs 2023-09-10T04:34:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18910.994388 secs 2023-09-10T04:34:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:34:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:34:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:34:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:34:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:34:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:34:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:34:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:34:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:34:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:34:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:41 smithi170 ceph-mon[102832]: pgmap v10247: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 310 KiB/s wr, 12 op/s 2023-09-10T04:34:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:41 smithi082 ceph-mon[98694]: pgmap v10247: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 310 KiB/s wr, 12 op/s 2023-09-10T04:34:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:41 smithi098 ceph-mon[103124]: pgmap v10247: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 310 KiB/s wr, 12 op/s 2023-09-10T04:34:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:43 smithi170 ceph-mon[102832]: pgmap v10248: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 500 KiB/s wr, 19 op/s 2023-09-10T04:34:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:43 smithi082 ceph-mon[98694]: pgmap v10248: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 500 KiB/s wr, 19 op/s 2023-09-10T04:34:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:43 smithi098 ceph-mon[103124]: pgmap v10248: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 500 KiB/s wr, 19 op/s 2023-09-10T04:34:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18915.339454 secs 2023-09-10T04:34:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18915.994451 secs 2023-09-10T04:34:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18915.339454 secs 2023-09-10T04:34:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18915.994451 secs 2023-09-10T04:34:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18915.339454 secs 2023-09-10T04:34:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18915.994451 secs 2023-09-10T04:34:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:45 smithi082 ceph-mon[98694]: pgmap v10249: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 330 KiB/s wr, 13 op/s 2023-09-10T04:34:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:45 smithi098 ceph-mon[103124]: pgmap v10249: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 330 KiB/s wr, 13 op/s 2023-09-10T04:34:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:45 smithi170 ceph-mon[102832]: pgmap v10249: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 330 KiB/s wr, 13 op/s 2023-09-10T04:34:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:46 smithi098 ceph-mon[103124]: pgmap v10250: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 432 KiB/s wr, 19 op/s 2023-09-10T04:34:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:46 smithi170 ceph-mon[102832]: pgmap v10250: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 432 KiB/s wr, 19 op/s 2023-09-10T04:34:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:46 smithi082 ceph-mon[98694]: pgmap v10250: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 432 KiB/s wr, 19 op/s 2023-09-10T04:34:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:48 smithi082 ceph-mon[98694]: pgmap v10251: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 292 KiB/s wr, 13 op/s 2023-09-10T04:34:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18920.994491 secs 2023-09-10T04:34:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:48 smithi098 ceph-mon[103124]: pgmap v10251: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 292 KiB/s wr, 13 op/s 2023-09-10T04:34:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18920.994491 secs 2023-09-10T04:34:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:48 smithi170 ceph-mon[102832]: pgmap v10251: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 292 KiB/s wr, 13 op/s 2023-09-10T04:34:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18920.994491 secs 2023-09-10T04:34:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18920.339496 secs 2023-09-10T04:34:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:34:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:34:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18920.339496 secs 2023-09-10T04:34:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:34:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:34:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18920.339496 secs 2023-09-10T04:34:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:34:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:34:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:50 smithi082 ceph-mon[98694]: pgmap v10252: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 292 KiB/s wr, 13 op/s 2023-09-10T04:34:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:50 smithi098 ceph-mon[103124]: pgmap v10252: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 292 KiB/s wr, 13 op/s 2023-09-10T04:34:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:50 smithi170 ceph-mon[102832]: pgmap v10252: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 292 KiB/s wr, 13 op/s 2023-09-10T04:34:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:34:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:34:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:34:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:34:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:34:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:34:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:52 smithi098 ceph-mon[103124]: pgmap v10253: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 461 KiB/s wr, 19 op/s 2023-09-10T04:34:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:52 smithi170 ceph-mon[102832]: pgmap v10253: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 461 KiB/s wr, 19 op/s 2023-09-10T04:34:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:52 smithi082 ceph-mon[98694]: pgmap v10253: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 461 KiB/s wr, 19 op/s 2023-09-10T04:34:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18924.216878 secs 2023-09-10T04:34:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18925.339591 secs 2023-09-10T04:34:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18925.994552 secs 2023-09-10T04:34:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18924.216878 secs 2023-09-10T04:34:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18925.339591 secs 2023-09-10T04:34:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18925.994552 secs 2023-09-10T04:34:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18924.216878 secs 2023-09-10T04:34:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18925.339591 secs 2023-09-10T04:34:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18925.994552 secs 2023-09-10T04:34:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:54 smithi098 ceph-mon[103124]: pgmap v10254: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 271 KiB/s wr, 12 op/s 2023-09-10T04:34:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:54 smithi170 ceph-mon[102832]: pgmap v10254: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 271 KiB/s wr, 12 op/s 2023-09-10T04:34:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:54 smithi082 ceph-mon[98694]: pgmap v10254: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 271 KiB/s wr, 12 op/s 2023-09-10T04:34:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:34:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:34:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:34:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:34:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:34:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:34:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:34:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:34:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:34:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:56 smithi098 ceph-mon[103124]: pgmap v10255: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 334 KiB/s wr, 18 op/s 2023-09-10T04:34:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:56 smithi170 ceph-mon[102832]: pgmap v10255: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 334 KiB/s wr, 18 op/s 2023-09-10T04:34:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:56 smithi082 ceph-mon[98694]: pgmap v10255: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 334 KiB/s wr, 18 op/s 2023-09-10T04:34:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:34:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:34:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:34:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:34:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:34:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:34:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18929.216938 secs 2023-09-10T04:34:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:58 smithi098 ceph-mon[103124]: pgmap v10256: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 232 KiB/s wr, 12 op/s 2023-09-10T04:34:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18930.339653 secs 2023-09-10T04:34:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:34:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18930.994615 secs 2023-09-10T04:34:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18929.216938 secs 2023-09-10T04:34:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:58 smithi170 ceph-mon[102832]: pgmap v10256: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 232 KiB/s wr, 12 op/s 2023-09-10T04:34:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18930.339653 secs 2023-09-10T04:34:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:34:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18930.994615 secs 2023-09-10T04:34:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18929.216938 secs 2023-09-10T04:34:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:58 smithi082 ceph-mon[98694]: pgmap v10256: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 232 KiB/s wr, 12 op/s 2023-09-10T04:34:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18930.339653 secs 2023-09-10T04:34:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:34:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18930.994615 secs 2023-09-10T04:35:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:35:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:35:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:00 smithi170 ceph-mon[102832]: pgmap v10257: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 232 KiB/s wr, 12 op/s 2023-09-10T04:35:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:35:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:35:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:00 smithi082 ceph-mon[98694]: pgmap v10257: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 232 KiB/s wr, 12 op/s 2023-09-10T04:35:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:35:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:35:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:00 smithi098 ceph-mon[103124]: pgmap v10257: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 232 KiB/s wr, 12 op/s 2023-09-10T04:35:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:02 smithi082 ceph-mon[98694]: pgmap v10258: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 334 KiB/s wr, 18 op/s 2023-09-10T04:35:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:02 smithi098 ceph-mon[103124]: pgmap v10258: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 334 KiB/s wr, 18 op/s 2023-09-10T04:35:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:02 smithi170 ceph-mon[102832]: pgmap v10258: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 334 KiB/s wr, 18 op/s 2023-09-10T04:35:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18934.216997 secs 2023-09-10T04:35:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18935.339711 secs 2023-09-10T04:35:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18935.994679 secs 2023-09-10T04:35:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18934.216997 secs 2023-09-10T04:35:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18935.339711 secs 2023-09-10T04:35:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18935.994679 secs 2023-09-10T04:35:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18934.216997 secs 2023-09-10T04:35:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18935.339711 secs 2023-09-10T04:35:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18935.994679 secs 2023-09-10T04:35:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:04 smithi082 ceph-mon[98694]: pgmap v10259: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 165 KiB/s wr, 12 op/s 2023-09-10T04:35:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:04 smithi098 ceph-mon[103124]: pgmap v10259: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 165 KiB/s wr, 12 op/s 2023-09-10T04:35:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:04 smithi170 ceph-mon[102832]: pgmap v10259: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 165 KiB/s wr, 12 op/s 2023-09-10T04:35:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:06 smithi082 ceph-mon[98694]: pgmap v10260: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 256 KiB/s wr, 32 op/s 2023-09-10T04:35:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:06 smithi098 ceph-mon[103124]: pgmap v10260: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 256 KiB/s wr, 32 op/s 2023-09-10T04:35:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:06 smithi170 ceph-mon[102832]: pgmap v10260: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 256 KiB/s wr, 32 op/s 2023-09-10T04:35:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18939.217060 secs 2023-09-10T04:35:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:08 smithi082 ceph-mon[98694]: pgmap v10261: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 193 KiB/s wr, 25 op/s 2023-09-10T04:35:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:08 smithi082 ceph-mon[98694]: osdmap e709: 12 total, 12 up, 12 in 2023-09-10T04:35:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18940.339816 secs 2023-09-10T04:35:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18940.994740 secs 2023-09-10T04:35:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18939.217060 secs 2023-09-10T04:35:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:08 smithi098 ceph-mon[103124]: pgmap v10261: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 193 KiB/s wr, 25 op/s 2023-09-10T04:35:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:08 smithi098 ceph-mon[103124]: osdmap e709: 12 total, 12 up, 12 in 2023-09-10T04:35:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18940.339816 secs 2023-09-10T04:35:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18940.994740 secs 2023-09-10T04:35:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18939.217060 secs 2023-09-10T04:35:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:08 smithi170 ceph-mon[102832]: pgmap v10261: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 193 KiB/s wr, 25 op/s 2023-09-10T04:35:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:08 smithi170 ceph-mon[102832]: osdmap e709: 12 total, 12 up, 12 in 2023-09-10T04:35:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18940.339816 secs 2023-09-10T04:35:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18940.994740 secs 2023-09-10T04:35:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:10 smithi082 ceph-mon[98694]: pgmap v10263: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 232 KiB/s wr, 31 op/s 2023-09-10T04:35:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:10 smithi098 ceph-mon[103124]: pgmap v10263: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 232 KiB/s wr, 31 op/s 2023-09-10T04:35:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:10 smithi170 ceph-mon[102832]: pgmap v10263: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 232 KiB/s wr, 31 op/s 2023-09-10T04:35:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:12 smithi082 ceph-mon[98694]: pgmap v10264: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 110 KiB/s wr, 29 op/s 2023-09-10T04:35:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:12 smithi098 ceph-mon[103124]: pgmap v10264: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 110 KiB/s wr, 29 op/s 2023-09-10T04:35:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:12 smithi170 ceph-mon[102832]: pgmap v10264: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 110 KiB/s wr, 29 op/s 2023-09-10T04:35:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18944.217120 secs 2023-09-10T04:35:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18945.339878 secs 2023-09-10T04:35:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18945.994803 secs 2023-09-10T04:35:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:13 smithi082 ceph-mon[98694]: osdmap e710: 12 total, 12 up, 12 in 2023-09-10T04:35:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18944.217120 secs 2023-09-10T04:35:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18945.339878 secs 2023-09-10T04:35:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18945.994803 secs 2023-09-10T04:35:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:13 smithi098 ceph-mon[103124]: osdmap e710: 12 total, 12 up, 12 in 2023-09-10T04:35:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18944.217120 secs 2023-09-10T04:35:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18945.339878 secs 2023-09-10T04:35:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18945.994803 secs 2023-09-10T04:35:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:13 smithi170 ceph-mon[102832]: osdmap e710: 12 total, 12 up, 12 in 2023-09-10T04:35:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:14 smithi082 ceph-mon[98694]: pgmap v10266: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:35:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:14 smithi098 ceph-mon[103124]: pgmap v10266: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:35:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:14 smithi170 ceph-mon[102832]: pgmap v10266: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:35:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:16 smithi098 ceph-mon[103124]: pgmap v10267: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 15 op/s 2023-09-10T04:35:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:16 smithi170 ceph-mon[102832]: pgmap v10267: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 15 op/s 2023-09-10T04:35:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:16 smithi082 ceph-mon[98694]: pgmap v10267: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 15 op/s 2023-09-10T04:35:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18949.217178 secs 2023-09-10T04:35:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:18 smithi098 ceph-mon[103124]: pgmap v10268: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T04:35:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18950.339936 secs 2023-09-10T04:35:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18950.994866 secs 2023-09-10T04:35:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18949.217178 secs 2023-09-10T04:35:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:18 smithi170 ceph-mon[102832]: pgmap v10268: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T04:35:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18950.339936 secs 2023-09-10T04:35:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18950.994866 secs 2023-09-10T04:35:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18949.217178 secs 2023-09-10T04:35:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:18 smithi082 ceph-mon[98694]: pgmap v10268: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T04:35:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18950.339936 secs 2023-09-10T04:35:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18950.994866 secs 2023-09-10T04:35:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:20 smithi098 ceph-mon[103124]: pgmap v10269: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T04:35:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:20 smithi170 ceph-mon[102832]: pgmap v10269: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T04:35:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:20 smithi082 ceph-mon[98694]: pgmap v10269: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 125 KiB/s wr, 12 op/s 2023-09-10T04:35:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:22 smithi098 ceph-mon[103124]: pgmap v10270: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T04:35:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:22 smithi170 ceph-mon[102832]: pgmap v10270: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T04:35:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:22 smithi082 ceph-mon[98694]: pgmap v10270: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T04:35:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18954.217238 secs 2023-09-10T04:35:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18955.340020 secs 2023-09-10T04:35:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18955.994930 secs 2023-09-10T04:35:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18954.217238 secs 2023-09-10T04:35:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18955.340020 secs 2023-09-10T04:35:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18955.994930 secs 2023-09-10T04:35:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18954.217238 secs 2023-09-10T04:35:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18955.340020 secs 2023-09-10T04:35:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18955.994930 secs 2023-09-10T04:35:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:24 smithi098 ceph-mon[103124]: pgmap v10271: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T04:35:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:24 smithi170 ceph-mon[102832]: pgmap v10271: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T04:35:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:24 smithi082 ceph-mon[98694]: pgmap v10271: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T04:35:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:26 smithi098 ceph-mon[103124]: pgmap v10272: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 169 KiB/s wr, 14 op/s 2023-09-10T04:35:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:26 smithi170 ceph-mon[102832]: pgmap v10272: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 169 KiB/s wr, 14 op/s 2023-09-10T04:35:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:26 smithi082 ceph-mon[98694]: pgmap v10272: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 169 KiB/s wr, 14 op/s 2023-09-10T04:35:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18959.217345 secs 2023-09-10T04:35:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:28 smithi098 ceph-mon[103124]: pgmap v10273: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 9 op/s 2023-09-10T04:35:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18960.340080 secs 2023-09-10T04:35:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:35:29.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18960.994997 secs 2023-09-10T04:35:29.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:35:29.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:35:29.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:35:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18959.217345 secs 2023-09-10T04:35:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:28 smithi170 ceph-mon[102832]: pgmap v10273: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 9 op/s 2023-09-10T04:35:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18960.340080 secs 2023-09-10T04:35:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:35:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18960.994997 secs 2023-09-10T04:35:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:35:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:35:29.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:35:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18959.217345 secs 2023-09-10T04:35:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:28 smithi082 ceph-mon[98694]: pgmap v10273: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 9 op/s 2023-09-10T04:35:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18960.340080 secs 2023-09-10T04:35:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:35:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18960.994997 secs 2023-09-10T04:35:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:35:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:35:29.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:35:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:30 smithi098 ceph-mon[103124]: pgmap v10274: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 9 op/s 2023-09-10T04:35:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:30 smithi170 ceph-mon[102832]: pgmap v10274: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 9 op/s 2023-09-10T04:35:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:30 smithi082 ceph-mon[98694]: pgmap v10274: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 9 op/s 2023-09-10T04:35:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:35:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:35:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:35:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:35:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:35:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:35:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:32 smithi098 ceph-mon[103124]: pgmap v10275: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 236 KiB/s wr, 15 op/s 2023-09-10T04:35:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:32 smithi170 ceph-mon[102832]: pgmap v10275: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 236 KiB/s wr, 15 op/s 2023-09-10T04:35:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:32 smithi082 ceph-mon[98694]: pgmap v10275: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 236 KiB/s wr, 15 op/s 2023-09-10T04:35:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18964.217406 secs 2023-09-10T04:35:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:35:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:35:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18965.340140 secs 2023-09-10T04:35:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18965.995060 secs 2023-09-10T04:35:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18964.217406 secs 2023-09-10T04:35:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:35:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:35:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18965.340140 secs 2023-09-10T04:35:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18965.995060 secs 2023-09-10T04:35:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18964.217406 secs 2023-09-10T04:35:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:35:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:35:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18965.340140 secs 2023-09-10T04:35:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18965.995060 secs 2023-09-10T04:35:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:34 smithi098 ceph-mon[103124]: pgmap v10276: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 236 KiB/s wr, 10 op/s 2023-09-10T04:35:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:34 smithi170 ceph-mon[102832]: pgmap v10276: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 236 KiB/s wr, 10 op/s 2023-09-10T04:35:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:34 smithi082 ceph-mon[98694]: pgmap v10276: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 236 KiB/s wr, 10 op/s 2023-09-10T04:35:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:37 smithi098 ceph-mon[103124]: pgmap v10277: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 376 KiB/s wr, 17 op/s 2023-09-10T04:35:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:37 smithi170 ceph-mon[102832]: pgmap v10277: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 376 KiB/s wr, 17 op/s 2023-09-10T04:35:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:37 smithi082 ceph-mon[98694]: pgmap v10277: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 376 KiB/s wr, 17 op/s 2023-09-10T04:35:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:39 smithi098 ceph-mon[103124]: pgmap v10278: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 312 KiB/s wr, 12 op/s 2023-09-10T04:35:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18970.340198 secs 2023-09-10T04:35:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18970.995124 secs 2023-09-10T04:35:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:35:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:39 smithi170 ceph-mon[102832]: pgmap v10278: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 312 KiB/s wr, 12 op/s 2023-09-10T04:35:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18970.340198 secs 2023-09-10T04:35:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18970.995124 secs 2023-09-10T04:35:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:35:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:39 smithi082 ceph-mon[98694]: pgmap v10278: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 312 KiB/s wr, 12 op/s 2023-09-10T04:35:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18970.340198 secs 2023-09-10T04:35:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18970.995124 secs 2023-09-10T04:35:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:35:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:35:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:35:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:35:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:35:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:35:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:35:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:35:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:35:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:35:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:41 smithi170 ceph-mon[102832]: pgmap v10279: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 312 KiB/s wr, 12 op/s 2023-09-10T04:35:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:41 smithi082 ceph-mon[98694]: pgmap v10279: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 312 KiB/s wr, 12 op/s 2023-09-10T04:35:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:41 smithi098 ceph-mon[103124]: pgmap v10279: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 312 KiB/s wr, 12 op/s 2023-09-10T04:35:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:43 smithi170 ceph-mon[102832]: pgmap v10280: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 479 KiB/s wr, 19 op/s 2023-09-10T04:35:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:43 smithi082 ceph-mon[98694]: pgmap v10280: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 479 KiB/s wr, 19 op/s 2023-09-10T04:35:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:43 smithi098 ceph-mon[103124]: pgmap v10280: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 479 KiB/s wr, 19 op/s 2023-09-10T04:35:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:35:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:35:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18975.340286 secs 2023-09-10T04:35:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18975.995187 secs 2023-09-10T04:35:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:35:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:35:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18975.340286 secs 2023-09-10T04:35:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18975.995187 secs 2023-09-10T04:35:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:35:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:35:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18975.340286 secs 2023-09-10T04:35:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18975.995187 secs 2023-09-10T04:35:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:45 smithi170 ceph-mon[102832]: pgmap v10281: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 308 KiB/s wr, 13 op/s 2023-09-10T04:35:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:45 smithi082 ceph-mon[98694]: pgmap v10281: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 308 KiB/s wr, 13 op/s 2023-09-10T04:35:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:45 smithi098 ceph-mon[103124]: pgmap v10281: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 308 KiB/s wr, 13 op/s 2023-09-10T04:35:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:35:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:35:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:35:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:35:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:35:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:35:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:47 smithi170 ceph-mon[102832]: pgmap v10282: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 434 KiB/s wr, 19 op/s 2023-09-10T04:35:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:47 smithi082 ceph-mon[98694]: pgmap v10282: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 434 KiB/s wr, 19 op/s 2023-09-10T04:35:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:47 smithi098 ceph-mon[103124]: pgmap v10282: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 434 KiB/s wr, 19 op/s 2023-09-10T04:35:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:49 smithi170 ceph-mon[102832]: pgmap v10283: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 293 KiB/s wr, 12 op/s 2023-09-10T04:35:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18980.340348 secs 2023-09-10T04:35:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18980.995266 secs 2023-09-10T04:35:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:49 smithi082 ceph-mon[98694]: pgmap v10283: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 293 KiB/s wr, 12 op/s 2023-09-10T04:35:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18980.340348 secs 2023-09-10T04:35:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18980.995266 secs 2023-09-10T04:35:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:49 smithi098 ceph-mon[103124]: pgmap v10283: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 293 KiB/s wr, 12 op/s 2023-09-10T04:35:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18980.340348 secs 2023-09-10T04:35:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18980.995266 secs 2023-09-10T04:35:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:51 smithi170 ceph-mon[102832]: pgmap v10284: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 293 KiB/s wr, 12 op/s 2023-09-10T04:35:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:51 smithi082 ceph-mon[98694]: pgmap v10284: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 293 KiB/s wr, 12 op/s 2023-09-10T04:35:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:51 smithi098 ceph-mon[103124]: pgmap v10284: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 293 KiB/s wr, 12 op/s 2023-09-10T04:35:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:53 smithi170 ceph-mon[102832]: pgmap v10285: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 463 KiB/s wr, 19 op/s 2023-09-10T04:35:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18984.217674 secs 2023-09-10T04:35:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:53 smithi082 ceph-mon[98694]: pgmap v10285: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 463 KiB/s wr, 19 op/s 2023-09-10T04:35:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18984.217674 secs 2023-09-10T04:35:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:53 smithi098 ceph-mon[103124]: pgmap v10285: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 463 KiB/s wr, 19 op/s 2023-09-10T04:35:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18984.217674 secs 2023-09-10T04:35:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18985.340407 secs 2023-09-10T04:35:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18985.995328 secs 2023-09-10T04:35:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18985.340407 secs 2023-09-10T04:35:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18985.995328 secs 2023-09-10T04:35:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18985.340407 secs 2023-09-10T04:35:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18985.995328 secs 2023-09-10T04:35:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:55 smithi170 ceph-mon[102832]: pgmap v10286: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 296 KiB/s wr, 12 op/s 2023-09-10T04:35:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:55 smithi082 ceph-mon[98694]: pgmap v10286: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 296 KiB/s wr, 12 op/s 2023-09-10T04:35:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:55 smithi098 ceph-mon[103124]: pgmap v10286: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 296 KiB/s wr, 12 op/s 2023-09-10T04:35:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:35:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:35:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:35:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:35:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:35:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:35:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:35:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:35:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:35:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:57 smithi170 ceph-mon[102832]: pgmap v10287: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 360 KiB/s wr, 18 op/s 2023-09-10T04:35:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:57 smithi082 ceph-mon[98694]: pgmap v10287: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 360 KiB/s wr, 18 op/s 2023-09-10T04:35:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:57 smithi098 ceph-mon[103124]: pgmap v10287: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 360 KiB/s wr, 18 op/s 2023-09-10T04:35:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:35:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:35:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18989.217737 secs 2023-09-10T04:35:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:35:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:35:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18989.217737 secs 2023-09-10T04:35:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:35:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:35:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18989.217737 secs 2023-09-10T04:35:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:59 smithi170 ceph-mon[102832]: pgmap v10288: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 234 KiB/s wr, 12 op/s 2023-09-10T04:35:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18990.340467 secs 2023-09-10T04:35:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:35:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18990.995389 secs 2023-09-10T04:35:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:59 smithi082 ceph-mon[98694]: pgmap v10288: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 234 KiB/s wr, 12 op/s 2023-09-10T04:35:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18990.340467 secs 2023-09-10T04:35:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:35:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18990.995389 secs 2023-09-10T04:35:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:59 smithi098 ceph-mon[103124]: pgmap v10288: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 234 KiB/s wr, 12 op/s 2023-09-10T04:35:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18990.340467 secs 2023-09-10T04:35:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:35:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18990.995389 secs 2023-09-10T04:36:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:36:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:36:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:36:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:36:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:36:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:36:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:01 smithi170 ceph-mon[102832]: pgmap v10289: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 234 KiB/s wr, 12 op/s 2023-09-10T04:36:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:01 smithi082 ceph-mon[98694]: pgmap v10289: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 234 KiB/s wr, 12 op/s 2023-09-10T04:36:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:01 smithi098 ceph-mon[103124]: pgmap v10289: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 234 KiB/s wr, 12 op/s 2023-09-10T04:36:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:03 smithi170 ceph-mon[102832]: pgmap v10290: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 336 KiB/s wr, 18 op/s 2023-09-10T04:36:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18994.217833 secs 2023-09-10T04:36:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:03 smithi082 ceph-mon[98694]: pgmap v10290: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 336 KiB/s wr, 18 op/s 2023-09-10T04:36:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18994.217833 secs 2023-09-10T04:36:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:03 smithi098 ceph-mon[103124]: pgmap v10290: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 336 KiB/s wr, 18 op/s 2023-09-10T04:36:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18994.217833 secs 2023-09-10T04:36:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18995.340527 secs 2023-09-10T04:36:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 18995.995453 secs 2023-09-10T04:36:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18995.340527 secs 2023-09-10T04:36:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 18995.995453 secs 2023-09-10T04:36:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18995.340527 secs 2023-09-10T04:36:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 18995.995453 secs 2023-09-10T04:36:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:05 smithi170 ceph-mon[102832]: pgmap v10291: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 166 KiB/s wr, 11 op/s 2023-09-10T04:36:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:05 smithi082 ceph-mon[98694]: pgmap v10291: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 166 KiB/s wr, 11 op/s 2023-09-10T04:36:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:05 smithi098 ceph-mon[103124]: pgmap v10291: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 166 KiB/s wr, 11 op/s 2023-09-10T04:36:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:07 smithi170 ceph-mon[102832]: pgmap v10292: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 257 KiB/s wr, 31 op/s 2023-09-10T04:36:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:07 smithi082 ceph-mon[98694]: pgmap v10292: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 257 KiB/s wr, 31 op/s 2023-09-10T04:36:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:07 smithi098 ceph-mon[103124]: pgmap v10292: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 257 KiB/s wr, 31 op/s 2023-09-10T04:36:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 18999.217895 secs 2023-09-10T04:36:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19000.340585 secs 2023-09-10T04:36:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 18999.217895 secs 2023-09-10T04:36:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19000.340585 secs 2023-09-10T04:36:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 18999.217895 secs 2023-09-10T04:36:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19000.340585 secs 2023-09-10T04:36:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:09 smithi170 ceph-mon[102832]: pgmap v10293: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 194 KiB/s wr, 25 op/s 2023-09-10T04:36:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:09 smithi170 ceph-mon[102832]: osdmap e711: 12 total, 12 up, 12 in 2023-09-10T04:36:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19000.995497 secs 2023-09-10T04:36:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:09 smithi082 ceph-mon[98694]: pgmap v10293: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 194 KiB/s wr, 25 op/s 2023-09-10T04:36:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:09 smithi082 ceph-mon[98694]: osdmap e711: 12 total, 12 up, 12 in 2023-09-10T04:36:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19000.995497 secs 2023-09-10T04:36:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:09 smithi098 ceph-mon[103124]: pgmap v10293: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 194 KiB/s wr, 25 op/s 2023-09-10T04:36:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:09 smithi098 ceph-mon[103124]: osdmap e711: 12 total, 12 up, 12 in 2023-09-10T04:36:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19000.995497 secs 2023-09-10T04:36:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:11 smithi170 ceph-mon[102832]: pgmap v10295: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 232 KiB/s wr, 30 op/s 2023-09-10T04:36:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:11 smithi082 ceph-mon[98694]: pgmap v10295: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 232 KiB/s wr, 30 op/s 2023-09-10T04:36:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:11 smithi098 ceph-mon[103124]: pgmap v10295: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 232 KiB/s wr, 30 op/s 2023-09-10T04:36:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:13 smithi170 ceph-mon[102832]: pgmap v10296: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 110 KiB/s wr, 30 op/s 2023-09-10T04:36:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19004.218005 secs 2023-09-10T04:36:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19005.340643 secs 2023-09-10T04:36:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:13 smithi082 ceph-mon[98694]: pgmap v10296: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 110 KiB/s wr, 30 op/s 2023-09-10T04:36:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19004.218005 secs 2023-09-10T04:36:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19005.340643 secs 2023-09-10T04:36:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:13 smithi098 ceph-mon[103124]: pgmap v10296: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 110 KiB/s wr, 30 op/s 2023-09-10T04:36:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19004.218005 secs 2023-09-10T04:36:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19005.340643 secs 2023-09-10T04:36:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19005.995558 secs 2023-09-10T04:36:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:14 smithi170 ceph-mon[102832]: osdmap e712: 12 total, 12 up, 12 in 2023-09-10T04:36:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19005.995558 secs 2023-09-10T04:36:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:14 smithi082 ceph-mon[98694]: osdmap e712: 12 total, 12 up, 12 in 2023-09-10T04:36:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19005.995558 secs 2023-09-10T04:36:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:14 smithi098 ceph-mon[103124]: osdmap e712: 12 total, 12 up, 12 in 2023-09-10T04:36:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:15 smithi170 ceph-mon[102832]: pgmap v10298: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:36:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:15 smithi082 ceph-mon[98694]: pgmap v10298: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:36:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:15 smithi098 ceph-mon[103124]: pgmap v10298: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:36:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:17 smithi170 ceph-mon[102832]: pgmap v10299: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 15 op/s 2023-09-10T04:36:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:17 smithi082 ceph-mon[98694]: pgmap v10299: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 15 op/s 2023-09-10T04:36:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:17 smithi098 ceph-mon[103124]: pgmap v10299: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 15 op/s 2023-09-10T04:36:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19009.218066 secs 2023-09-10T04:36:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19010.340718 secs 2023-09-10T04:36:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19009.218066 secs 2023-09-10T04:36:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19010.340718 secs 2023-09-10T04:36:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19009.218066 secs 2023-09-10T04:36:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19010.340718 secs 2023-09-10T04:36:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:19 smithi170 ceph-mon[102832]: pgmap v10300: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 12 op/s 2023-09-10T04:36:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19010.995620 secs 2023-09-10T04:36:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:19 smithi082 ceph-mon[98694]: pgmap v10300: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 12 op/s 2023-09-10T04:36:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19010.995620 secs 2023-09-10T04:36:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:19 smithi098 ceph-mon[103124]: pgmap v10300: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 12 op/s 2023-09-10T04:36:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19010.995620 secs 2023-09-10T04:36:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:21 smithi170 ceph-mon[102832]: pgmap v10301: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T04:36:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:21 smithi082 ceph-mon[98694]: pgmap v10301: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T04:36:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:21 smithi098 ceph-mon[103124]: pgmap v10301: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T04:36:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:23 smithi170 ceph-mon[102832]: pgmap v10302: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T04:36:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19014.218172 secs 2023-09-10T04:36:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19015.340775 secs 2023-09-10T04:36:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:23 smithi082 ceph-mon[98694]: pgmap v10302: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T04:36:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19014.218172 secs 2023-09-10T04:36:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19015.340775 secs 2023-09-10T04:36:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:23 smithi098 ceph-mon[103124]: pgmap v10302: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T04:36:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19014.218172 secs 2023-09-10T04:36:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19015.340775 secs 2023-09-10T04:36:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19015.995682 secs 2023-09-10T04:36:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19015.995682 secs 2023-09-10T04:36:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19015.995682 secs 2023-09-10T04:36:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:25 smithi170 ceph-mon[102832]: pgmap v10303: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T04:36:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:25 smithi082 ceph-mon[98694]: pgmap v10303: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T04:36:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:25 smithi098 ceph-mon[103124]: pgmap v10303: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T04:36:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:27 smithi170 ceph-mon[102832]: pgmap v10304: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 204 KiB/s wr, 14 op/s 2023-09-10T04:36:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:27 smithi082 ceph-mon[98694]: pgmap v10304: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 204 KiB/s wr, 14 op/s 2023-09-10T04:36:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:27 smithi098 ceph-mon[103124]: pgmap v10304: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 204 KiB/s wr, 14 op/s 2023-09-10T04:36:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19019.218256 secs 2023-09-10T04:36:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19020.340833 secs 2023-09-10T04:36:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19019.218256 secs 2023-09-10T04:36:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19020.340833 secs 2023-09-10T04:36:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19019.218256 secs 2023-09-10T04:36:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19020.340833 secs 2023-09-10T04:36:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:29 smithi170 ceph-mon[102832]: pgmap v10305: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 99 KiB/s wr, 9 op/s 2023-09-10T04:36:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19020.995745 secs 2023-09-10T04:36:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:36:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:29 smithi082 ceph-mon[98694]: pgmap v10305: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 99 KiB/s wr, 9 op/s 2023-09-10T04:36:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19020.995745 secs 2023-09-10T04:36:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:36:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:29 smithi098 ceph-mon[103124]: pgmap v10305: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 99 KiB/s wr, 9 op/s 2023-09-10T04:36:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19020.995745 secs 2023-09-10T04:36:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:36:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:36:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:36:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:36:30.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:36:30.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:36:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:36:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:36:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:36:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:36:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:36:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:36:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:36:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:36:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:36:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:36:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:31 smithi170 ceph-mon[102832]: pgmap v10306: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 99 KiB/s wr, 9 op/s 2023-09-10T04:36:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:31 smithi082 ceph-mon[98694]: pgmap v10306: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 99 KiB/s wr, 9 op/s 2023-09-10T04:36:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:31 smithi098 ceph-mon[103124]: pgmap v10306: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 99 KiB/s wr, 9 op/s 2023-09-10T04:36:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:36:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:36:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:36:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:36:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:36:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:36:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:33 smithi170 ceph-mon[102832]: pgmap v10307: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 237 KiB/s wr, 15 op/s 2023-09-10T04:36:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19024.218341 secs 2023-09-10T04:36:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19025.340892 secs 2023-09-10T04:36:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:33 smithi082 ceph-mon[98694]: pgmap v10307: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 237 KiB/s wr, 15 op/s 2023-09-10T04:36:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19024.218341 secs 2023-09-10T04:36:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19025.340892 secs 2023-09-10T04:36:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:33 smithi098 ceph-mon[103124]: pgmap v10307: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 237 KiB/s wr, 15 op/s 2023-09-10T04:36:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19024.218341 secs 2023-09-10T04:36:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19025.340892 secs 2023-09-10T04:36:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19025.995806 secs 2023-09-10T04:36:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19025.995806 secs 2023-09-10T04:36:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19025.995806 secs 2023-09-10T04:36:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:35 smithi170 ceph-mon[102832]: pgmap v10308: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 237 KiB/s wr, 11 op/s 2023-09-10T04:36:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:35 smithi082 ceph-mon[98694]: pgmap v10308: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 237 KiB/s wr, 11 op/s 2023-09-10T04:36:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:35 smithi098 ceph-mon[103124]: pgmap v10308: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 237 KiB/s wr, 11 op/s 2023-09-10T04:36:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:37 smithi170 ceph-mon[102832]: pgmap v10309: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 378 KiB/s wr, 17 op/s 2023-09-10T04:36:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:36:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:36:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:37 smithi082 ceph-mon[98694]: pgmap v10309: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 378 KiB/s wr, 17 op/s 2023-09-10T04:36:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:36:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:36:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:37 smithi098 ceph-mon[103124]: pgmap v10309: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 378 KiB/s wr, 17 op/s 2023-09-10T04:36:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:36:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:36:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19030.340949 secs 2023-09-10T04:36:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19030.340949 secs 2023-09-10T04:36:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19030.340949 secs 2023-09-10T04:36:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:39 smithi170 ceph-mon[102832]: pgmap v10310: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T04:36:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19030.995885 secs 2023-09-10T04:36:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:36:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:39 smithi082 ceph-mon[98694]: pgmap v10310: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T04:36:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19030.995885 secs 2023-09-10T04:36:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:36:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:39 smithi098 ceph-mon[103124]: pgmap v10310: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T04:36:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19030.995885 secs 2023-09-10T04:36:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:36:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:36:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:36:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:36:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:36:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:36:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:36:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:36:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:36:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:36:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:41 smithi170 ceph-mon[102832]: pgmap v10311: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T04:36:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:41 smithi082 ceph-mon[98694]: pgmap v10311: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T04:36:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:41 smithi098 ceph-mon[103124]: pgmap v10311: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T04:36:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:43 smithi170 ceph-mon[102832]: pgmap v10312: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 447 KiB/s wr, 19 op/s 2023-09-10T04:36:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19035.341031 secs 2023-09-10T04:36:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:43 smithi082 ceph-mon[98694]: pgmap v10312: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 447 KiB/s wr, 19 op/s 2023-09-10T04:36:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19035.341031 secs 2023-09-10T04:36:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:43 smithi098 ceph-mon[103124]: pgmap v10312: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 447 KiB/s wr, 19 op/s 2023-09-10T04:36:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19035.341031 secs 2023-09-10T04:36:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19035.995949 secs 2023-09-10T04:36:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19035.995949 secs 2023-09-10T04:36:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19035.995949 secs 2023-09-10T04:36:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:45 smithi170 ceph-mon[102832]: pgmap v10313: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 309 KiB/s wr, 13 op/s 2023-09-10T04:36:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:45 smithi082 ceph-mon[98694]: pgmap v10313: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 309 KiB/s wr, 13 op/s 2023-09-10T04:36:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:45 smithi098 ceph-mon[103124]: pgmap v10313: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 309 KiB/s wr, 13 op/s 2023-09-10T04:36:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:47 smithi170 ceph-mon[102832]: pgmap v10314: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 436 KiB/s wr, 19 op/s 2023-09-10T04:36:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:47 smithi082 ceph-mon[98694]: pgmap v10314: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 436 KiB/s wr, 19 op/s 2023-09-10T04:36:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:47 smithi098 ceph-mon[103124]: pgmap v10314: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 436 KiB/s wr, 19 op/s 2023-09-10T04:36:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19040.341115 secs 2023-09-10T04:36:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19040.341115 secs 2023-09-10T04:36:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19040.341115 secs 2023-09-10T04:36:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:49 smithi170 ceph-mon[102832]: pgmap v10315: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 295 KiB/s wr, 12 op/s 2023-09-10T04:36:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19040.996013 secs 2023-09-10T04:36:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:49 smithi082 ceph-mon[98694]: pgmap v10315: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 295 KiB/s wr, 12 op/s 2023-09-10T04:36:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19040.996013 secs 2023-09-10T04:36:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:49 smithi098 ceph-mon[103124]: pgmap v10315: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 295 KiB/s wr, 12 op/s 2023-09-10T04:36:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19040.996013 secs 2023-09-10T04:36:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:51 smithi170 ceph-mon[102832]: pgmap v10316: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 295 KiB/s wr, 12 op/s 2023-09-10T04:36:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:51 smithi082 ceph-mon[98694]: pgmap v10316: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 295 KiB/s wr, 12 op/s 2023-09-10T04:36:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:51 smithi098 ceph-mon[103124]: pgmap v10316: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 295 KiB/s wr, 12 op/s 2023-09-10T04:36:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:53 smithi170 ceph-mon[102832]: pgmap v10317: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 466 KiB/s wr, 19 op/s 2023-09-10T04:36:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19044.218638 secs 2023-09-10T04:36:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19045.341170 secs 2023-09-10T04:36:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:53 smithi082 ceph-mon[98694]: pgmap v10317: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 466 KiB/s wr, 19 op/s 2023-09-10T04:36:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19044.218638 secs 2023-09-10T04:36:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19045.341170 secs 2023-09-10T04:36:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:53 smithi098 ceph-mon[103124]: pgmap v10317: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 466 KiB/s wr, 19 op/s 2023-09-10T04:36:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19044.218638 secs 2023-09-10T04:36:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19045.341170 secs 2023-09-10T04:36:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19045.996076 secs 2023-09-10T04:36:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19045.996076 secs 2023-09-10T04:36:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19045.996076 secs 2023-09-10T04:36:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:55 smithi170 ceph-mon[102832]: pgmap v10318: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 298 KiB/s wr, 12 op/s 2023-09-10T04:36:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:55 smithi082 ceph-mon[98694]: pgmap v10318: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 298 KiB/s wr, 12 op/s 2023-09-10T04:36:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:55 smithi098 ceph-mon[103124]: pgmap v10318: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 298 KiB/s wr, 12 op/s 2023-09-10T04:36:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:36:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:36:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:36:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:36:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:36:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:36:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:36:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:36:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:36:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:57 smithi170 ceph-mon[102832]: pgmap v10319: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 362 KiB/s wr, 18 op/s 2023-09-10T04:36:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:36:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:36:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:57 smithi082 ceph-mon[98694]: pgmap v10319: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 362 KiB/s wr, 18 op/s 2023-09-10T04:36:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:36:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:36:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:57 smithi098 ceph-mon[103124]: pgmap v10319: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 362 KiB/s wr, 18 op/s 2023-09-10T04:36:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:36:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:36:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19049.218700 secs 2023-09-10T04:36:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19050.341251 secs 2023-09-10T04:36:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19049.218700 secs 2023-09-10T04:36:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19050.341251 secs 2023-09-10T04:36:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19049.218700 secs 2023-09-10T04:36:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19050.341251 secs 2023-09-10T04:36:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:59 smithi170 ceph-mon[102832]: pgmap v10320: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 235 KiB/s wr, 12 op/s 2023-09-10T04:36:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:36:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19050.996136 secs 2023-09-10T04:36:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:59 smithi082 ceph-mon[98694]: pgmap v10320: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 235 KiB/s wr, 12 op/s 2023-09-10T04:36:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:36:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19050.996136 secs 2023-09-10T04:37:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:59 smithi098 ceph-mon[103124]: pgmap v10320: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 235 KiB/s wr, 12 op/s 2023-09-10T04:37:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:36:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19050.996136 secs 2023-09-10T04:37:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:37:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:37:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:37:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:37:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:37:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:37:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:01 smithi082 ceph-mon[98694]: pgmap v10321: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 235 KiB/s wr, 12 op/s 2023-09-10T04:37:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:01 smithi098 ceph-mon[103124]: pgmap v10321: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 235 KiB/s wr, 12 op/s 2023-09-10T04:37:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:01 smithi170 ceph-mon[102832]: pgmap v10321: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 235 KiB/s wr, 12 op/s 2023-09-10T04:37:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:03 smithi082 ceph-mon[98694]: pgmap v10322: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 360 KiB/s wr, 28 op/s 2023-09-10T04:37:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19054.218827 secs 2023-09-10T04:37:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19055.341337 secs 2023-09-10T04:37:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:03 smithi098 ceph-mon[103124]: pgmap v10322: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 360 KiB/s wr, 28 op/s 2023-09-10T04:37:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19054.218827 secs 2023-09-10T04:37:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19055.341337 secs 2023-09-10T04:37:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:03 smithi170 ceph-mon[102832]: pgmap v10322: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 360 KiB/s wr, 28 op/s 2023-09-10T04:37:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19054.218827 secs 2023-09-10T04:37:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19055.341337 secs 2023-09-10T04:37:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19055.996199 secs 2023-09-10T04:37:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19055.996199 secs 2023-09-10T04:37:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19055.996199 secs 2023-09-10T04:37:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:05 smithi082 ceph-mon[98694]: pgmap v10323: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 189 KiB/s wr, 22 op/s 2023-09-10T04:37:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:05 smithi098 ceph-mon[103124]: pgmap v10323: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 189 KiB/s wr, 22 op/s 2023-09-10T04:37:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:05 smithi170 ceph-mon[102832]: pgmap v10323: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 189 KiB/s wr, 22 op/s 2023-09-10T04:37:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:07 smithi082 ceph-mon[98694]: pgmap v10324: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 258 KiB/s wr, 31 op/s 2023-09-10T04:37:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:07 smithi098 ceph-mon[103124]: pgmap v10324: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 258 KiB/s wr, 31 op/s 2023-09-10T04:37:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:07 smithi170 ceph-mon[102832]: pgmap v10324: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 258 KiB/s wr, 31 op/s 2023-09-10T04:37:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19059.218908 secs 2023-09-10T04:37:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:08 smithi082 ceph-mon[98694]: osdmap e713: 12 total, 12 up, 12 in 2023-09-10T04:37:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19060.341395 secs 2023-09-10T04:37:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19059.218908 secs 2023-09-10T04:37:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:08 smithi098 ceph-mon[103124]: osdmap e713: 12 total, 12 up, 12 in 2023-09-10T04:37:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19060.341395 secs 2023-09-10T04:37:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19059.218908 secs 2023-09-10T04:37:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:08 smithi170 ceph-mon[102832]: osdmap e713: 12 total, 12 up, 12 in 2023-09-10T04:37:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19060.341395 secs 2023-09-10T04:37:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:09 smithi082 ceph-mon[98694]: pgmap v10326: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 233 KiB/s wr, 30 op/s 2023-09-10T04:37:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19060.996260 secs 2023-09-10T04:37:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:09 smithi098 ceph-mon[103124]: pgmap v10326: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 233 KiB/s wr, 30 op/s 2023-09-10T04:37:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19060.996260 secs 2023-09-10T04:37:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:09 smithi170 ceph-mon[102832]: pgmap v10326: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 233 KiB/s wr, 30 op/s 2023-09-10T04:37:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19060.996260 secs 2023-09-10T04:37:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:10 smithi170 ceph-mon[102832]: pgmap v10327: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 233 KiB/s wr, 30 op/s 2023-09-10T04:37:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:10 smithi082 ceph-mon[98694]: pgmap v10327: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 233 KiB/s wr, 30 op/s 2023-09-10T04:37:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:10 smithi098 ceph-mon[103124]: pgmap v10327: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 233 KiB/s wr, 30 op/s 2023-09-10T04:37:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:12 smithi098 ceph-mon[103124]: pgmap v10328: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 84 KiB/s wr, 16 op/s 2023-09-10T04:37:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:12 smithi170 ceph-mon[102832]: pgmap v10328: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 84 KiB/s wr, 16 op/s 2023-09-10T04:37:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:12 smithi082 ceph-mon[98694]: pgmap v10328: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 84 KiB/s wr, 16 op/s 2023-09-10T04:37:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19064.218969 secs 2023-09-10T04:37:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19065.341454 secs 2023-09-10T04:37:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19065.996323 secs 2023-09-10T04:37:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:13 smithi098 ceph-mon[103124]: osdmap e714: 12 total, 12 up, 12 in 2023-09-10T04:37:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19064.218969 secs 2023-09-10T04:37:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19065.341454 secs 2023-09-10T04:37:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19065.996323 secs 2023-09-10T04:37:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:13 smithi170 ceph-mon[102832]: osdmap e714: 12 total, 12 up, 12 in 2023-09-10T04:37:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19064.218969 secs 2023-09-10T04:37:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19065.341454 secs 2023-09-10T04:37:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19065.996323 secs 2023-09-10T04:37:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:13 smithi082 ceph-mon[98694]: osdmap e714: 12 total, 12 up, 12 in 2023-09-10T04:37:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:14 smithi098 ceph-mon[103124]: pgmap v10330: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:37:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:14 smithi170 ceph-mon[102832]: pgmap v10330: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:37:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:14 smithi082 ceph-mon[98694]: pgmap v10330: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:37:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:16 smithi098 ceph-mon[103124]: pgmap v10331: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 14 op/s 2023-09-10T04:37:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:16 smithi170 ceph-mon[102832]: pgmap v10331: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 14 op/s 2023-09-10T04:37:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:16 smithi082 ceph-mon[98694]: pgmap v10331: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 14 op/s 2023-09-10T04:37:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19069.219031 secs 2023-09-10T04:37:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:18 smithi098 ceph-mon[103124]: pgmap v10332: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T04:37:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19070.341536 secs 2023-09-10T04:37:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19070.996385 secs 2023-09-10T04:37:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19069.219031 secs 2023-09-10T04:37:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:18 smithi170 ceph-mon[102832]: pgmap v10332: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T04:37:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19070.341536 secs 2023-09-10T04:37:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19070.996385 secs 2023-09-10T04:37:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19069.219031 secs 2023-09-10T04:37:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:18 smithi082 ceph-mon[98694]: pgmap v10332: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T04:37:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19070.341536 secs 2023-09-10T04:37:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19070.996385 secs 2023-09-10T04:37:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:20 smithi098 ceph-mon[103124]: pgmap v10333: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T04:37:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:20 smithi170 ceph-mon[102832]: pgmap v10333: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T04:37:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:20 smithi082 ceph-mon[98694]: pgmap v10333: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T04:37:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:22 smithi098 ceph-mon[103124]: pgmap v10334: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T04:37:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:22 smithi170 ceph-mon[102832]: pgmap v10334: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T04:37:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:22 smithi082 ceph-mon[98694]: pgmap v10334: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T04:37:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19074.219092 secs 2023-09-10T04:37:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19075.341594 secs 2023-09-10T04:37:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19075.996445 secs 2023-09-10T04:37:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19074.219092 secs 2023-09-10T04:37:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19075.341594 secs 2023-09-10T04:37:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19075.996445 secs 2023-09-10T04:37:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19074.219092 secs 2023-09-10T04:37:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19075.341594 secs 2023-09-10T04:37:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19075.996445 secs 2023-09-10T04:37:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:24 smithi098 ceph-mon[103124]: pgmap v10335: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T04:37:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:24 smithi170 ceph-mon[102832]: pgmap v10335: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T04:37:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:24 smithi082 ceph-mon[98694]: pgmap v10335: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T04:37:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:26 smithi098 ceph-mon[103124]: pgmap v10336: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 170 KiB/s wr, 15 op/s 2023-09-10T04:37:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:26 smithi170 ceph-mon[102832]: pgmap v10336: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 170 KiB/s wr, 15 op/s 2023-09-10T04:37:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:26 smithi082 ceph-mon[98694]: pgmap v10336: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 170 KiB/s wr, 15 op/s 2023-09-10T04:37:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:28 smithi098 ceph-mon[103124]: pgmap v10337: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 9 op/s 2023-09-10T04:37:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19080.341651 secs 2023-09-10T04:37:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19080.996486 secs 2023-09-10T04:37:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:28 smithi170 ceph-mon[102832]: pgmap v10337: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 9 op/s 2023-09-10T04:37:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19080.341651 secs 2023-09-10T04:37:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19080.996486 secs 2023-09-10T04:37:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:28 smithi082 ceph-mon[98694]: pgmap v10337: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 9 op/s 2023-09-10T04:37:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19080.341651 secs 2023-09-10T04:37:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19080.996486 secs 2023-09-10T04:37:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:37:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:37:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:37:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:37:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:37:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:37:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:37:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:37:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:37:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:31 smithi170 ceph-mon[102832]: pgmap v10338: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 9 op/s 2023-09-10T04:37:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:37:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:37:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:37:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:37:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:31 smithi082 ceph-mon[98694]: pgmap v10338: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 9 op/s 2023-09-10T04:37:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:37:31.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:37:31.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:37:31.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:37:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:31 smithi098 ceph-mon[103124]: pgmap v10338: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 9 op/s 2023-09-10T04:37:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:37:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:37:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:37:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:37:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:37:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:37:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:37:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:37:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:37:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:37:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:37:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:37:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:37:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:33 smithi170 ceph-mon[102832]: pgmap v10339: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 238 KiB/s wr, 15 op/s 2023-09-10T04:37:33.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19085.341710 secs 2023-09-10T04:37:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:33 smithi082 ceph-mon[98694]: pgmap v10339: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 238 KiB/s wr, 15 op/s 2023-09-10T04:37:33.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19085.341710 secs 2023-09-10T04:37:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:33 smithi098 ceph-mon[103124]: pgmap v10339: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 238 KiB/s wr, 15 op/s 2023-09-10T04:37:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19085.341710 secs 2023-09-10T04:37:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19085.996548 secs 2023-09-10T04:37:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19085.996548 secs 2023-09-10T04:37:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19085.996548 secs 2023-09-10T04:37:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:35 smithi170 ceph-mon[102832]: pgmap v10340: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 238 KiB/s wr, 11 op/s 2023-09-10T04:37:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:35 smithi082 ceph-mon[98694]: pgmap v10340: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 238 KiB/s wr, 11 op/s 2023-09-10T04:37:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:35 smithi098 ceph-mon[103124]: pgmap v10340: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 238 KiB/s wr, 11 op/s 2023-09-10T04:37:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:37 smithi170 ceph-mon[102832]: pgmap v10341: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 379 KiB/s wr, 17 op/s 2023-09-10T04:37:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:37 smithi082 ceph-mon[98694]: pgmap v10341: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 379 KiB/s wr, 17 op/s 2023-09-10T04:37:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:37 smithi098 ceph-mon[103124]: pgmap v10341: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 379 KiB/s wr, 17 op/s 2023-09-10T04:37:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:37:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:37:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19089.219334 secs 2023-09-10T04:37:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19090.341783 secs 2023-09-10T04:37:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:37:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:37:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19089.219334 secs 2023-09-10T04:37:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19090.341783 secs 2023-09-10T04:37:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:37:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:37:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19089.219334 secs 2023-09-10T04:37:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19090.341783 secs 2023-09-10T04:37:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:39 smithi170 ceph-mon[102832]: pgmap v10342: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 314 KiB/s wr, 11 op/s 2023-09-10T04:37:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19090.996609 secs 2023-09-10T04:37:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:39 smithi082 ceph-mon[98694]: pgmap v10342: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 314 KiB/s wr, 11 op/s 2023-09-10T04:37:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19090.996609 secs 2023-09-10T04:37:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:39 smithi098 ceph-mon[103124]: pgmap v10342: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 314 KiB/s wr, 11 op/s 2023-09-10T04:37:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19090.996609 secs 2023-09-10T04:37:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:37:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:37:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:37:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:37:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:37:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:37:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:37:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:37:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:37:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:41 smithi170 ceph-mon[102832]: pgmap v10343: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 314 KiB/s wr, 11 op/s 2023-09-10T04:37:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:41 smithi082 ceph-mon[98694]: pgmap v10343: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 314 KiB/s wr, 11 op/s 2023-09-10T04:37:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:41 smithi098 ceph-mon[103124]: pgmap v10343: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 314 KiB/s wr, 11 op/s 2023-09-10T04:37:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:43 smithi170 ceph-mon[102832]: pgmap v10344: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 476 KiB/s wr, 19 op/s 2023-09-10T04:37:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19094.219395 secs 2023-09-10T04:37:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19095.341843 secs 2023-09-10T04:37:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:43 smithi082 ceph-mon[98694]: pgmap v10344: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 476 KiB/s wr, 19 op/s 2023-09-10T04:37:43.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19094.219395 secs 2023-09-10T04:37:43.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19095.341843 secs 2023-09-10T04:37:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:43 smithi098 ceph-mon[103124]: pgmap v10344: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 476 KiB/s wr, 19 op/s 2023-09-10T04:37:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19094.219395 secs 2023-09-10T04:37:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19095.341843 secs 2023-09-10T04:37:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19095.996671 secs 2023-09-10T04:37:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19095.996671 secs 2023-09-10T04:37:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19095.996671 secs 2023-09-10T04:37:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:45 smithi170 ceph-mon[102832]: pgmap v10345: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 303 KiB/s wr, 13 op/s 2023-09-10T04:37:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:45 smithi082 ceph-mon[98694]: pgmap v10345: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 303 KiB/s wr, 13 op/s 2023-09-10T04:37:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:45 smithi098 ceph-mon[103124]: pgmap v10345: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 303 KiB/s wr, 13 op/s 2023-09-10T04:37:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:47 smithi170 ceph-mon[102832]: pgmap v10346: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 422 KiB/s wr, 19 op/s 2023-09-10T04:37:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:47 smithi082 ceph-mon[98694]: pgmap v10346: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 422 KiB/s wr, 19 op/s 2023-09-10T04:37:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:47 smithi098 ceph-mon[103124]: pgmap v10346: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 422 KiB/s wr, 19 op/s 2023-09-10T04:37:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19100.341901 secs 2023-09-10T04:37:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19100.341901 secs 2023-09-10T04:37:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19100.341901 secs 2023-09-10T04:37:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:49 smithi170 ceph-mon[102832]: pgmap v10347: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 281 KiB/s wr, 13 op/s 2023-09-10T04:37:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19100.996734 secs 2023-09-10T04:37:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:49 smithi082 ceph-mon[98694]: pgmap v10347: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 281 KiB/s wr, 13 op/s 2023-09-10T04:37:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19100.996734 secs 2023-09-10T04:37:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:49 smithi098 ceph-mon[103124]: pgmap v10347: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 281 KiB/s wr, 13 op/s 2023-09-10T04:37:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19100.996734 secs 2023-09-10T04:37:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:51 smithi170 ceph-mon[102832]: pgmap v10348: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 281 KiB/s wr, 13 op/s 2023-09-10T04:37:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:51 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:37:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:51 smithi082 ceph-mon[98694]: pgmap v10348: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 281 KiB/s wr, 13 op/s 2023-09-10T04:37:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:51 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:37:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:51 smithi098 ceph-mon[103124]: pgmap v10348: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 281 KiB/s wr, 13 op/s 2023-09-10T04:37:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:51 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:37:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:52 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:37:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:37:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:37:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:52 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:37:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:37:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:37:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:52 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:37:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:37:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:37:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:53 smithi170 ceph-mon[102832]: pgmap v10349: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 402 KiB/s wr, 19 op/s 2023-09-10T04:37:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19104.219516 secs 2023-09-10T04:37:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19105.341975 secs 2023-09-10T04:37:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:53 smithi082 ceph-mon[98694]: pgmap v10349: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 402 KiB/s wr, 19 op/s 2023-09-10T04:37:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19104.219516 secs 2023-09-10T04:37:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19105.341975 secs 2023-09-10T04:37:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:53 smithi098 ceph-mon[103124]: pgmap v10349: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 402 KiB/s wr, 19 op/s 2023-09-10T04:37:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19104.219516 secs 2023-09-10T04:37:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19105.341975 secs 2023-09-10T04:37:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19105.996794 secs 2023-09-10T04:37:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19105.996794 secs 2023-09-10T04:37:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19105.996794 secs 2023-09-10T04:37:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:55 smithi170 ceph-mon[102832]: pgmap v10350: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 240 KiB/s wr, 12 op/s 2023-09-10T04:37:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:55 smithi082 ceph-mon[98694]: pgmap v10350: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 240 KiB/s wr, 12 op/s 2023-09-10T04:37:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:55 smithi098 ceph-mon[103124]: pgmap v10350: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 240 KiB/s wr, 12 op/s 2023-09-10T04:37:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:37:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:37:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:37:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:37:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:37:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:37:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:57 smithi170 ceph-mon[102832]: pgmap v10351: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 304 KiB/s wr, 17 op/s 2023-09-10T04:37:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:57 smithi082 ceph-mon[98694]: pgmap v10351: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 304 KiB/s wr, 17 op/s 2023-09-10T04:37:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:57 smithi098 ceph-mon[103124]: pgmap v10351: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 304 KiB/s wr, 17 op/s 2023-09-10T04:37:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:37:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:37:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19109.219574 secs 2023-09-10T04:37:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19110.342058 secs 2023-09-10T04:37:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:37:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:37:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19109.219574 secs 2023-09-10T04:37:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19110.342058 secs 2023-09-10T04:37:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:37:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:37:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19109.219574 secs 2023-09-10T04:37:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19110.342058 secs 2023-09-10T04:37:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:59 smithi170 ceph-mon[102832]: pgmap v10352: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 185 KiB/s wr, 12 op/s 2023-09-10T04:37:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19110.996858 secs 2023-09-10T04:37:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:37:59 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:37:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:59 smithi082 ceph-mon[98694]: pgmap v10352: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 185 KiB/s wr, 12 op/s 2023-09-10T04:37:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19110.996858 secs 2023-09-10T04:37:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:37:59 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:37:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:59 smithi098 ceph-mon[103124]: pgmap v10352: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 185 KiB/s wr, 12 op/s 2023-09-10T04:37:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19110.996858 secs 2023-09-10T04:37:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:37:59 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:38:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:38:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:38:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:38:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:38:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:38:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:38:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:01 smithi082 ceph-mon[98694]: pgmap v10353: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 185 KiB/s wr, 12 op/s 2023-09-10T04:38:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:01 smithi098 ceph-mon[103124]: pgmap v10353: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 185 KiB/s wr, 12 op/s 2023-09-10T04:38:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:01 smithi170 ceph-mon[102832]: pgmap v10353: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 185 KiB/s wr, 12 op/s 2023-09-10T04:38:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:03 smithi082 ceph-mon[98694]: pgmap v10354: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 271 KiB/s wr, 17 op/s 2023-09-10T04:38:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19114.219632 secs 2023-09-10T04:38:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19115.342119 secs 2023-09-10T04:38:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:03 smithi098 ceph-mon[103124]: pgmap v10354: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 271 KiB/s wr, 17 op/s 2023-09-10T04:38:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19114.219632 secs 2023-09-10T04:38:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19115.342119 secs 2023-09-10T04:38:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:03 smithi170 ceph-mon[102832]: pgmap v10354: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 271 KiB/s wr, 17 op/s 2023-09-10T04:38:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19114.219632 secs 2023-09-10T04:38:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19115.342119 secs 2023-09-10T04:38:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19115.996919 secs 2023-09-10T04:38:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19115.996919 secs 2023-09-10T04:38:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19115.996919 secs 2023-09-10T04:38:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:05 smithi082 ceph-mon[98694]: pgmap v10355: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T04:38:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:05 smithi098 ceph-mon[103124]: pgmap v10355: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T04:38:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:05 smithi170 ceph-mon[102832]: pgmap v10355: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T04:38:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:07 smithi082 ceph-mon[98694]: pgmap v10356: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 256 KiB/s wr, 31 op/s 2023-09-10T04:38:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:07 smithi098 ceph-mon[103124]: pgmap v10356: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 256 KiB/s wr, 31 op/s 2023-09-10T04:38:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:07 smithi170 ceph-mon[102832]: pgmap v10356: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 256 KiB/s wr, 31 op/s 2023-09-10T04:38:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19119.219690 secs 2023-09-10T04:38:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19120.342204 secs 2023-09-10T04:38:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19119.219690 secs 2023-09-10T04:38:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19120.342204 secs 2023-09-10T04:38:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19119.219690 secs 2023-09-10T04:38:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19120.342204 secs 2023-09-10T04:38:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:09 smithi082 ceph-mon[98694]: pgmap v10357: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 192 KiB/s wr, 25 op/s 2023-09-10T04:38:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:09 smithi082 ceph-mon[98694]: osdmap e715: 12 total, 12 up, 12 in 2023-09-10T04:38:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19120.996987 secs 2023-09-10T04:38:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:09 smithi098 ceph-mon[103124]: pgmap v10357: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 192 KiB/s wr, 25 op/s 2023-09-10T04:38:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:09 smithi098 ceph-mon[103124]: osdmap e715: 12 total, 12 up, 12 in 2023-09-10T04:38:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19120.996987 secs 2023-09-10T04:38:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:09 smithi170 ceph-mon[102832]: pgmap v10357: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 192 KiB/s wr, 25 op/s 2023-09-10T04:38:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:09 smithi170 ceph-mon[102832]: osdmap e715: 12 total, 12 up, 12 in 2023-09-10T04:38:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19120.996987 secs 2023-09-10T04:38:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:11 smithi082 ceph-mon[98694]: pgmap v10359: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 231 KiB/s wr, 31 op/s 2023-09-10T04:38:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:11 smithi098 ceph-mon[103124]: pgmap v10359: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 231 KiB/s wr, 31 op/s 2023-09-10T04:38:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:11 smithi170 ceph-mon[102832]: pgmap v10359: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 231 KiB/s wr, 31 op/s 2023-09-10T04:38:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:13 smithi082 ceph-mon[98694]: pgmap v10360: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 128 KiB/s wr, 30 op/s 2023-09-10T04:38:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19124.219750 secs 2023-09-10T04:38:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19125.342278 secs 2023-09-10T04:38:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:13 smithi098 ceph-mon[103124]: pgmap v10360: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 128 KiB/s wr, 30 op/s 2023-09-10T04:38:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19124.219750 secs 2023-09-10T04:38:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19125.342278 secs 2023-09-10T04:38:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:13 smithi170 ceph-mon[102832]: pgmap v10360: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 128 KiB/s wr, 30 op/s 2023-09-10T04:38:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19124.219750 secs 2023-09-10T04:38:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19125.342278 secs 2023-09-10T04:38:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19125.997052 secs 2023-09-10T04:38:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:14 smithi082 ceph-mon[98694]: osdmap e716: 12 total, 12 up, 12 in 2023-09-10T04:38:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19125.997052 secs 2023-09-10T04:38:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:14 smithi098 ceph-mon[103124]: osdmap e716: 12 total, 12 up, 12 in 2023-09-10T04:38:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19125.997052 secs 2023-09-10T04:38:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:14 smithi170 ceph-mon[102832]: osdmap e716: 12 total, 12 up, 12 in 2023-09-10T04:38:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:15 smithi082 ceph-mon[98694]: pgmap v10362: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:38:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:15 smithi098 ceph-mon[103124]: pgmap v10362: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:38:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:15 smithi170 ceph-mon[102832]: pgmap v10362: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:38:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:17 smithi082 ceph-mon[98694]: pgmap v10363: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 15 op/s 2023-09-10T04:38:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:17 smithi098 ceph-mon[103124]: pgmap v10363: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 15 op/s 2023-09-10T04:38:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:17 smithi170 ceph-mon[102832]: pgmap v10363: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 15 op/s 2023-09-10T04:38:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19129.219869 secs 2023-09-10T04:38:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19130.342361 secs 2023-09-10T04:38:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19129.219869 secs 2023-09-10T04:38:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19130.342361 secs 2023-09-10T04:38:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19129.219869 secs 2023-09-10T04:38:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19130.342361 secs 2023-09-10T04:38:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:19 smithi098 ceph-mon[103124]: pgmap v10364: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 13 op/s 2023-09-10T04:38:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19130.997119 secs 2023-09-10T04:38:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:19 smithi170 ceph-mon[102832]: pgmap v10364: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 13 op/s 2023-09-10T04:38:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19130.997119 secs 2023-09-10T04:38:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:19 smithi082 ceph-mon[98694]: pgmap v10364: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 13 op/s 2023-09-10T04:38:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19130.997119 secs 2023-09-10T04:38:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:21 smithi098 ceph-mon[103124]: pgmap v10365: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T04:38:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:21 smithi170 ceph-mon[102832]: pgmap v10365: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T04:38:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:21 smithi082 ceph-mon[98694]: pgmap v10365: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T04:38:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:23 smithi098 ceph-mon[103124]: pgmap v10366: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T04:38:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19134.219932 secs 2023-09-10T04:38:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19135.342419 secs 2023-09-10T04:38:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:23 smithi170 ceph-mon[102832]: pgmap v10366: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T04:38:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19134.219932 secs 2023-09-10T04:38:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19135.342419 secs 2023-09-10T04:38:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:23 smithi082 ceph-mon[98694]: pgmap v10366: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T04:38:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19134.219932 secs 2023-09-10T04:38:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19135.342419 secs 2023-09-10T04:38:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19135.997185 secs 2023-09-10T04:38:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19135.997185 secs 2023-09-10T04:38:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19135.997185 secs 2023-09-10T04:38:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:25 smithi098 ceph-mon[103124]: pgmap v10367: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T04:38:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:25 smithi170 ceph-mon[102832]: pgmap v10367: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T04:38:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:25 smithi082 ceph-mon[98694]: pgmap v10367: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T04:38:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:27 smithi098 ceph-mon[103124]: pgmap v10368: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 190 KiB/s wr, 14 op/s 2023-09-10T04:38:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:27 smithi170 ceph-mon[102832]: pgmap v10368: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 190 KiB/s wr, 14 op/s 2023-09-10T04:38:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:27 smithi082 ceph-mon[98694]: pgmap v10368: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 190 KiB/s wr, 14 op/s 2023-09-10T04:38:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19140.342489 secs 2023-09-10T04:38:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19140.342489 secs 2023-09-10T04:38:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19140.342489 secs 2023-09-10T04:38:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:29 smithi098 ceph-mon[103124]: pgmap v10369: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 93 KiB/s wr, 9 op/s 2023-09-10T04:38:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19140.997251 secs 2023-09-10T04:38:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:29 smithi170 ceph-mon[102832]: pgmap v10369: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 93 KiB/s wr, 9 op/s 2023-09-10T04:38:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19140.997251 secs 2023-09-10T04:38:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:29 smithi082 ceph-mon[98694]: pgmap v10369: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 93 KiB/s wr, 9 op/s 2023-09-10T04:38:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19140.997251 secs 2023-09-10T04:38:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:38:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:38:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:38:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:38:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:38:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:38:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:38:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:38:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:38:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:31 smithi098 ceph-mon[103124]: pgmap v10370: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 93 KiB/s wr, 9 op/s 2023-09-10T04:38:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:38:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:38:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:38:31.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:38:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:31 smithi170 ceph-mon[102832]: pgmap v10370: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 93 KiB/s wr, 9 op/s 2023-09-10T04:38:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:38:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:38:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:38:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:38:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:31 smithi082 ceph-mon[98694]: pgmap v10370: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 93 KiB/s wr, 9 op/s 2023-09-10T04:38:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:38:31.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:38:31.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:38:31.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:38:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:38:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:38:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:38:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:38:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:38:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:38:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:38:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:38:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:38:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:33 smithi170 ceph-mon[102832]: pgmap v10371: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 189 KiB/s wr, 15 op/s 2023-09-10T04:38:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19145.342548 secs 2023-09-10T04:38:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:33 smithi082 ceph-mon[98694]: pgmap v10371: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 189 KiB/s wr, 15 op/s 2023-09-10T04:38:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19145.342548 secs 2023-09-10T04:38:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:33 smithi098 ceph-mon[103124]: pgmap v10371: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 189 KiB/s wr, 15 op/s 2023-09-10T04:38:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19145.342548 secs 2023-09-10T04:38:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19145.997314 secs 2023-09-10T04:38:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19145.997314 secs 2023-09-10T04:38:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19145.997314 secs 2023-09-10T04:38:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:35 smithi170 ceph-mon[102832]: pgmap v10372: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 188 KiB/s wr, 11 op/s 2023-09-10T04:38:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:35 smithi082 ceph-mon[98694]: pgmap v10372: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 188 KiB/s wr, 11 op/s 2023-09-10T04:38:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:35 smithi098 ceph-mon[103124]: pgmap v10372: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 188 KiB/s wr, 11 op/s 2023-09-10T04:38:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:37 smithi170 ceph-mon[102832]: pgmap v10373: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 314 KiB/s wr, 17 op/s 2023-09-10T04:38:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:37 smithi082 ceph-mon[98694]: pgmap v10373: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 314 KiB/s wr, 17 op/s 2023-09-10T04:38:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:37 smithi098 ceph-mon[103124]: pgmap v10373: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 314 KiB/s wr, 17 op/s 2023-09-10T04:38:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19149.220168 secs 2023-09-10T04:38:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19150.342617 secs 2023-09-10T04:38:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19149.220168 secs 2023-09-10T04:38:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19150.342617 secs 2023-09-10T04:38:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19149.220168 secs 2023-09-10T04:38:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19150.342617 secs 2023-09-10T04:38:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:39 smithi170 ceph-mon[102832]: pgmap v10374: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 221 KiB/s wr, 12 op/s 2023-09-10T04:38:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19150.997380 secs 2023-09-10T04:38:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:39 smithi082 ceph-mon[98694]: pgmap v10374: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 221 KiB/s wr, 12 op/s 2023-09-10T04:38:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19150.997380 secs 2023-09-10T04:38:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:39 smithi098 ceph-mon[103124]: pgmap v10374: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 221 KiB/s wr, 12 op/s 2023-09-10T04:38:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19150.997380 secs 2023-09-10T04:38:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:38:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:38:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:38:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:38:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:38:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:38:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:38:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:38:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:38:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:41 smithi170 ceph-mon[102832]: pgmap v10375: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 221 KiB/s wr, 12 op/s 2023-09-10T04:38:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:41 smithi082 ceph-mon[98694]: pgmap v10375: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 221 KiB/s wr, 12 op/s 2023-09-10T04:38:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:41 smithi098 ceph-mon[103124]: pgmap v10375: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 221 KiB/s wr, 12 op/s 2023-09-10T04:38:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:43 smithi170 ceph-mon[102832]: pgmap v10376: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 331 KiB/s wr, 19 op/s 2023-09-10T04:38:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:38:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19155.342676 secs 2023-09-10T04:38:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:43 smithi082 ceph-mon[98694]: pgmap v10376: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 331 KiB/s wr, 19 op/s 2023-09-10T04:38:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:38:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19155.342676 secs 2023-09-10T04:38:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:43 smithi098 ceph-mon[103124]: pgmap v10376: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 331 KiB/s wr, 19 op/s 2023-09-10T04:38:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:38:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19155.342676 secs 2023-09-10T04:38:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:38:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:38:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19155.997444 secs 2023-09-10T04:38:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:38:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:38:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19155.997444 secs 2023-09-10T04:38:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:38:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:38:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19155.997444 secs 2023-09-10T04:38:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:45 smithi170 ceph-mon[102832]: pgmap v10377: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 236 KiB/s wr, 13 op/s 2023-09-10T04:38:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:45 smithi082 ceph-mon[98694]: pgmap v10377: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 236 KiB/s wr, 13 op/s 2023-09-10T04:38:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:45 smithi098 ceph-mon[103124]: pgmap v10377: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 236 KiB/s wr, 13 op/s 2023-09-10T04:38:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:47 smithi170 ceph-mon[102832]: pgmap v10378: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 355 KiB/s wr, 18 op/s 2023-09-10T04:38:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:47 smithi082 ceph-mon[98694]: pgmap v10378: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 355 KiB/s wr, 18 op/s 2023-09-10T04:38:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:47 smithi098 ceph-mon[103124]: pgmap v10378: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 355 KiB/s wr, 18 op/s 2023-09-10T04:38:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19160.342734 secs 2023-09-10T04:38:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19160.342734 secs 2023-09-10T04:38:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19160.342734 secs 2023-09-10T04:38:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:49 smithi170 ceph-mon[102832]: pgmap v10379: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 12 op/s 2023-09-10T04:38:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19160.997491 secs 2023-09-10T04:38:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:49 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:38:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:49 smithi082 ceph-mon[98694]: pgmap v10379: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 12 op/s 2023-09-10T04:38:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19160.997491 secs 2023-09-10T04:38:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:49 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:38:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:49 smithi098 ceph-mon[103124]: pgmap v10379: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 12 op/s 2023-09-10T04:38:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19160.997491 secs 2023-09-10T04:38:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:49 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:38:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:38:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:38:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:38:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:38:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:38:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:38:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:51 smithi170 ceph-mon[102832]: pgmap v10380: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 12 op/s 2023-09-10T04:38:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:51 smithi082 ceph-mon[98694]: pgmap v10380: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 12 op/s 2023-09-10T04:38:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:51 smithi098 ceph-mon[103124]: pgmap v10380: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 12 op/s 2023-09-10T04:38:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:38:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:38:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:38:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:38:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:38:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:38:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:53 smithi170 ceph-mon[102832]: pgmap v10381: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 351 KiB/s wr, 18 op/s 2023-09-10T04:38:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19164.220399 secs 2023-09-10T04:38:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19165.342790 secs 2023-09-10T04:38:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:53 smithi082 ceph-mon[98694]: pgmap v10381: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 351 KiB/s wr, 18 op/s 2023-09-10T04:38:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19164.220399 secs 2023-09-10T04:38:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19165.342790 secs 2023-09-10T04:38:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:53 smithi098 ceph-mon[103124]: pgmap v10381: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 351 KiB/s wr, 18 op/s 2023-09-10T04:38:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19164.220399 secs 2023-09-10T04:38:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19165.342790 secs 2023-09-10T04:38:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19165.997558 secs 2023-09-10T04:38:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19165.997558 secs 2023-09-10T04:38:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19165.997558 secs 2023-09-10T04:38:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:55 smithi170 ceph-mon[102832]: pgmap v10382: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 241 KiB/s wr, 11 op/s 2023-09-10T04:38:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:55 smithi082 ceph-mon[98694]: pgmap v10382: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 241 KiB/s wr, 11 op/s 2023-09-10T04:38:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:55 smithi098 ceph-mon[103124]: pgmap v10382: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 241 KiB/s wr, 11 op/s 2023-09-10T04:38:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:38:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:38:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:38:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:38:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:38:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:38:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:38:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:38:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:38:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:57 smithi170 ceph-mon[102832]: pgmap v10383: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 306 KiB/s wr, 18 op/s 2023-09-10T04:38:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:57 smithi082 ceph-mon[98694]: pgmap v10383: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 306 KiB/s wr, 18 op/s 2023-09-10T04:38:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:57 smithi098 ceph-mon[103124]: pgmap v10383: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 306 KiB/s wr, 18 op/s 2023-09-10T04:38:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19169.220458 secs 2023-09-10T04:38:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19170.342872 secs 2023-09-10T04:38:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19169.220458 secs 2023-09-10T04:38:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19170.342872 secs 2023-09-10T04:38:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19169.220458 secs 2023-09-10T04:38:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19170.342872 secs 2023-09-10T04:38:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:59 smithi170 ceph-mon[102832]: pgmap v10384: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 186 KiB/s wr, 12 op/s 2023-09-10T04:38:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:38:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19170.997626 secs 2023-09-10T04:38:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:59 smithi082 ceph-mon[98694]: pgmap v10384: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 186 KiB/s wr, 12 op/s 2023-09-10T04:38:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:38:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19170.997626 secs 2023-09-10T04:39:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:59 smithi098 ceph-mon[103124]: pgmap v10384: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 186 KiB/s wr, 12 op/s 2023-09-10T04:39:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:38:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19170.997626 secs 2023-09-10T04:39:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:01 smithi170 ceph-mon[102832]: pgmap v10385: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 186 KiB/s wr, 12 op/s 2023-09-10T04:39:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:01 smithi082 ceph-mon[98694]: pgmap v10385: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 186 KiB/s wr, 12 op/s 2023-09-10T04:39:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:01 smithi098 ceph-mon[103124]: pgmap v10385: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 186 KiB/s wr, 12 op/s 2023-09-10T04:39:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:03 smithi170 ceph-mon[102832]: pgmap v10386: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 273 KiB/s wr, 17 op/s 2023-09-10T04:39:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19174.220542 secs 2023-09-10T04:39:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19175.342933 secs 2023-09-10T04:39:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:03 smithi082 ceph-mon[98694]: pgmap v10386: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 273 KiB/s wr, 17 op/s 2023-09-10T04:39:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19174.220542 secs 2023-09-10T04:39:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19175.342933 secs 2023-09-10T04:39:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:03 smithi098 ceph-mon[103124]: pgmap v10386: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 273 KiB/s wr, 17 op/s 2023-09-10T04:39:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19174.220542 secs 2023-09-10T04:39:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19175.342933 secs 2023-09-10T04:39:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19175.997690 secs 2023-09-10T04:39:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19175.997690 secs 2023-09-10T04:39:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19175.997690 secs 2023-09-10T04:39:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:05 smithi170 ceph-mon[102832]: pgmap v10387: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T04:39:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:05 smithi082 ceph-mon[98694]: pgmap v10387: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T04:39:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:05 smithi098 ceph-mon[103124]: pgmap v10387: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T04:39:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:07 smithi170 ceph-mon[102832]: pgmap v10388: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 243 KiB/s wr, 30 op/s 2023-09-10T04:39:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:07 smithi082 ceph-mon[98694]: pgmap v10388: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 243 KiB/s wr, 30 op/s 2023-09-10T04:39:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:07 smithi098 ceph-mon[103124]: pgmap v10388: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 243 KiB/s wr, 30 op/s 2023-09-10T04:39:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19179.220601 secs 2023-09-10T04:39:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:08 smithi170 ceph-mon[102832]: osdmap e717: 12 total, 12 up, 12 in 2023-09-10T04:39:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19180.343011 secs 2023-09-10T04:39:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19179.220601 secs 2023-09-10T04:39:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:08 smithi082 ceph-mon[98694]: osdmap e717: 12 total, 12 up, 12 in 2023-09-10T04:39:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19180.343011 secs 2023-09-10T04:39:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19179.220601 secs 2023-09-10T04:39:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:08 smithi098 ceph-mon[103124]: osdmap e717: 12 total, 12 up, 12 in 2023-09-10T04:39:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19180.343011 secs 2023-09-10T04:39:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:09 smithi170 ceph-mon[102832]: pgmap v10390: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 214 KiB/s wr, 29 op/s 2023-09-10T04:39:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19180.997759 secs 2023-09-10T04:39:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:09 smithi082 ceph-mon[98694]: pgmap v10390: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 214 KiB/s wr, 29 op/s 2023-09-10T04:39:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19180.997759 secs 2023-09-10T04:39:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:09 smithi098 ceph-mon[103124]: pgmap v10390: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 214 KiB/s wr, 29 op/s 2023-09-10T04:39:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19180.997759 secs 2023-09-10T04:39:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:11 smithi170 ceph-mon[102832]: pgmap v10391: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 214 KiB/s wr, 29 op/s 2023-09-10T04:39:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:11 smithi082 ceph-mon[98694]: pgmap v10391: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 214 KiB/s wr, 29 op/s 2023-09-10T04:39:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:11 smithi098 ceph-mon[103124]: pgmap v10391: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 214 KiB/s wr, 29 op/s 2023-09-10T04:39:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:13 smithi170 ceph-mon[102832]: pgmap v10392: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 111 KiB/s wr, 29 op/s 2023-09-10T04:39:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19184.220662 secs 2023-09-10T04:39:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19185.343069 secs 2023-09-10T04:39:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:13 smithi082 ceph-mon[98694]: pgmap v10392: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 111 KiB/s wr, 29 op/s 2023-09-10T04:39:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19184.220662 secs 2023-09-10T04:39:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19185.343069 secs 2023-09-10T04:39:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:13 smithi098 ceph-mon[103124]: pgmap v10392: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 111 KiB/s wr, 29 op/s 2023-09-10T04:39:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19184.220662 secs 2023-09-10T04:39:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19185.343069 secs 2023-09-10T04:39:14.624 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19185.997822 secs 2023-09-10T04:39:14.625 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:14 smithi082 ceph-mon[98694]: osdmap e718: 12 total, 12 up, 12 in 2023-09-10T04:39:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19185.997822 secs 2023-09-10T04:39:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:14 smithi170 ceph-mon[102832]: osdmap e718: 12 total, 12 up, 12 in 2023-09-10T04:39:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19185.997822 secs 2023-09-10T04:39:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:14 smithi098 ceph-mon[103124]: osdmap e718: 12 total, 12 up, 12 in 2023-09-10T04:39:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:15 smithi170 ceph-mon[102832]: pgmap v10394: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:39:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:15 smithi082 ceph-mon[98694]: pgmap v10394: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:39:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:15 smithi098 ceph-mon[103124]: pgmap v10394: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:39:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:17 smithi170 ceph-mon[102832]: pgmap v10395: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 14 op/s 2023-09-10T04:39:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:17 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:39:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:17 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:39:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:17 smithi082 ceph-mon[98694]: pgmap v10395: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 14 op/s 2023-09-10T04:39:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:17 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:39:17.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:17 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:39:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:17 smithi098 ceph-mon[103124]: pgmap v10395: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 14 op/s 2023-09-10T04:39:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:17 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:39:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:17 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:39:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19189.220795 secs 2023-09-10T04:39:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19190.343127 secs 2023-09-10T04:39:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19189.220795 secs 2023-09-10T04:39:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19190.343127 secs 2023-09-10T04:39:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19189.220795 secs 2023-09-10T04:39:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19190.343127 secs 2023-09-10T04:39:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:19 smithi170 ceph-mon[102832]: pgmap v10396: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T04:39:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19190.997889 secs 2023-09-10T04:39:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:19 smithi082 ceph-mon[98694]: pgmap v10396: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T04:39:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19190.997889 secs 2023-09-10T04:39:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:19 smithi098 ceph-mon[103124]: pgmap v10396: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T04:39:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19190.997889 secs 2023-09-10T04:39:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:39:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:39:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:39:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:39:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:39:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:39:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:21 smithi170 ceph-mon[102832]: pgmap v10397: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T04:39:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:21 smithi082 ceph-mon[98694]: pgmap v10397: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T04:39:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:21 smithi098 ceph-mon[103124]: pgmap v10397: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T04:39:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:23 smithi170 ceph-mon[102832]: pgmap v10398: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T04:39:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19194.220855 secs 2023-09-10T04:39:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19195.343187 secs 2023-09-10T04:39:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:23 smithi082 ceph-mon[98694]: pgmap v10398: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T04:39:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19194.220855 secs 2023-09-10T04:39:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19195.343187 secs 2023-09-10T04:39:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:23 smithi098 ceph-mon[103124]: pgmap v10398: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T04:39:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19194.220855 secs 2023-09-10T04:39:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19195.343187 secs 2023-09-10T04:39:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19195.997953 secs 2023-09-10T04:39:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19195.997953 secs 2023-09-10T04:39:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19195.997953 secs 2023-09-10T04:39:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:25 smithi170 ceph-mon[102832]: pgmap v10399: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T04:39:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:25 smithi082 ceph-mon[98694]: pgmap v10399: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T04:39:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:25 smithi098 ceph-mon[103124]: pgmap v10399: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T04:39:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:27 smithi170 ceph-mon[102832]: pgmap v10400: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 155 KiB/s wr, 15 op/s 2023-09-10T04:39:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:27 smithi082 ceph-mon[98694]: pgmap v10400: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 155 KiB/s wr, 15 op/s 2023-09-10T04:39:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:27 smithi098 ceph-mon[103124]: pgmap v10400: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 155 KiB/s wr, 15 op/s 2023-09-10T04:39:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19200.343245 secs 2023-09-10T04:39:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19200.343245 secs 2023-09-10T04:39:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19200.343245 secs 2023-09-10T04:39:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:29 smithi170 ceph-mon[102832]: pgmap v10401: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:39:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19200.998018 secs 2023-09-10T04:39:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:29 smithi082 ceph-mon[98694]: pgmap v10401: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:39:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19200.998018 secs 2023-09-10T04:39:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:29 smithi098 ceph-mon[103124]: pgmap v10401: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:39:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19200.998018 secs 2023-09-10T04:39:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:31 smithi082 ceph-mon[98694]: pgmap v10402: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:39:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:39:31.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:39:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:31 smithi098 ceph-mon[103124]: pgmap v10402: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:39:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:39:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:39:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:31 smithi170 ceph-mon[102832]: pgmap v10402: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:39:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:39:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:39:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:39:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:39:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:39:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:39:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:39:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:39:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:39:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:39:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:39:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:39:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:39:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:39:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:39:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:39:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:39:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:39:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:39:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:39:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:33 smithi082 ceph-mon[98694]: pgmap v10403: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 190 KiB/s wr, 15 op/s 2023-09-10T04:39:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19205.343324 secs 2023-09-10T04:39:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:33 smithi098 ceph-mon[103124]: pgmap v10403: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 190 KiB/s wr, 15 op/s 2023-09-10T04:39:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19205.343324 secs 2023-09-10T04:39:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:33 smithi170 ceph-mon[102832]: pgmap v10403: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 190 KiB/s wr, 15 op/s 2023-09-10T04:39:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19205.343324 secs 2023-09-10T04:39:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19205.998081 secs 2023-09-10T04:39:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19205.998081 secs 2023-09-10T04:39:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19205.998081 secs 2023-09-10T04:39:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:35 smithi098 ceph-mon[103124]: pgmap v10404: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 190 KiB/s wr, 10 op/s 2023-09-10T04:39:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:35 smithi170 ceph-mon[102832]: pgmap v10404: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 190 KiB/s wr, 10 op/s 2023-09-10T04:39:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:35 smithi082 ceph-mon[98694]: pgmap v10404: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 190 KiB/s wr, 10 op/s 2023-09-10T04:39:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:37 smithi098 ceph-mon[103124]: pgmap v10405: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 316 KiB/s wr, 16 op/s 2023-09-10T04:39:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:39:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:39:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:37 smithi170 ceph-mon[102832]: pgmap v10405: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 316 KiB/s wr, 16 op/s 2023-09-10T04:39:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:39:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:39:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:37 smithi082 ceph-mon[98694]: pgmap v10405: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 316 KiB/s wr, 16 op/s 2023-09-10T04:39:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:39:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:39:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19209.221088 secs 2023-09-10T04:39:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19210.343403 secs 2023-09-10T04:39:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19209.221088 secs 2023-09-10T04:39:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19210.343403 secs 2023-09-10T04:39:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19209.221088 secs 2023-09-10T04:39:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19210.343403 secs 2023-09-10T04:39:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:39 smithi098 ceph-mon[103124]: pgmap v10406: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 258 KiB/s wr, 11 op/s 2023-09-10T04:39:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19210.998149 secs 2023-09-10T04:39:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:39 smithi170 ceph-mon[102832]: pgmap v10406: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 258 KiB/s wr, 11 op/s 2023-09-10T04:39:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19210.998149 secs 2023-09-10T04:39:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:39 smithi082 ceph-mon[98694]: pgmap v10406: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 258 KiB/s wr, 11 op/s 2023-09-10T04:39:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19210.998149 secs 2023-09-10T04:39:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:39:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:39:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:39:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:39:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:39:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:39:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:39:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:39:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:39:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:41 smithi098 ceph-mon[103124]: pgmap v10407: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 258 KiB/s wr, 11 op/s 2023-09-10T04:39:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:41 smithi170 ceph-mon[102832]: pgmap v10407: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 258 KiB/s wr, 11 op/s 2023-09-10T04:39:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:41 smithi082 ceph-mon[98694]: pgmap v10407: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 258 KiB/s wr, 11 op/s 2023-09-10T04:39:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:42 smithi170 ceph-mon[102832]: pgmap v10408: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 369 KiB/s wr, 18 op/s 2023-09-10T04:39:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:42 smithi082 ceph-mon[98694]: pgmap v10408: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 369 KiB/s wr, 18 op/s 2023-09-10T04:39:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:42 smithi098 ceph-mon[103124]: pgmap v10408: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 369 KiB/s wr, 18 op/s 2023-09-10T04:39:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19214.221149 secs 2023-09-10T04:39:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19215.343465 secs 2023-09-10T04:39:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19215.998215 secs 2023-09-10T04:39:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19214.221149 secs 2023-09-10T04:39:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19215.343465 secs 2023-09-10T04:39:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19215.998215 secs 2023-09-10T04:39:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19214.221149 secs 2023-09-10T04:39:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19215.343465 secs 2023-09-10T04:39:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19215.998215 secs 2023-09-10T04:39:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:44 smithi082 ceph-mon[98694]: pgmap v10409: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 237 KiB/s wr, 12 op/s 2023-09-10T04:39:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:44 smithi098 ceph-mon[103124]: pgmap v10409: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 237 KiB/s wr, 12 op/s 2023-09-10T04:39:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:44 smithi170 ceph-mon[102832]: pgmap v10409: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 237 KiB/s wr, 12 op/s 2023-09-10T04:39:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:47 smithi170 ceph-mon[102832]: pgmap v10410: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 357 KiB/s wr, 17 op/s 2023-09-10T04:39:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:47 smithi082 ceph-mon[98694]: pgmap v10410: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 357 KiB/s wr, 17 op/s 2023-09-10T04:39:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:47 smithi098 ceph-mon[103124]: pgmap v10410: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 357 KiB/s wr, 17 op/s 2023-09-10T04:39:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:49 smithi170 ceph-mon[102832]: pgmap v10411: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T04:39:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19220.343492 secs 2023-09-10T04:39:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19220.998283 secs 2023-09-10T04:39:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:49 smithi082 ceph-mon[98694]: pgmap v10411: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T04:39:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19220.343492 secs 2023-09-10T04:39:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19220.998283 secs 2023-09-10T04:39:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:49 smithi098 ceph-mon[103124]: pgmap v10411: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T04:39:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19220.343492 secs 2023-09-10T04:39:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19220.998283 secs 2023-09-10T04:39:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:51 smithi170 ceph-mon[102832]: pgmap v10412: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T04:39:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:51 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:39:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:51 smithi082 ceph-mon[98694]: pgmap v10412: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T04:39:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:51 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:39:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:51 smithi098 ceph-mon[103124]: pgmap v10412: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T04:39:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:51 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:39:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:52 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:39:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:39:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:39:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:52 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:39:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:39:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:39:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:52 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:39:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:39:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:39:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:53 smithi170 ceph-mon[102832]: pgmap v10413: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 354 KiB/s wr, 17 op/s 2023-09-10T04:39:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:53 smithi082 ceph-mon[98694]: pgmap v10413: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 354 KiB/s wr, 17 op/s 2023-09-10T04:39:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:53 smithi098 ceph-mon[103124]: pgmap v10413: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 354 KiB/s wr, 17 op/s 2023-09-10T04:39:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19224.221323 secs 2023-09-10T04:39:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:39:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:39:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19225.343578 secs 2023-09-10T04:39:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19225.998343 secs 2023-09-10T04:39:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19224.221323 secs 2023-09-10T04:39:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:39:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:39:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19225.343578 secs 2023-09-10T04:39:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19225.998343 secs 2023-09-10T04:39:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19224.221323 secs 2023-09-10T04:39:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:39:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:39:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19225.343578 secs 2023-09-10T04:39:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19225.998343 secs 2023-09-10T04:39:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:55 smithi170 ceph-mon[102832]: pgmap v10414: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 243 KiB/s wr, 11 op/s 2023-09-10T04:39:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:55 smithi082 ceph-mon[98694]: pgmap v10414: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 243 KiB/s wr, 11 op/s 2023-09-10T04:39:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:55 smithi098 ceph-mon[103124]: pgmap v10414: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 243 KiB/s wr, 11 op/s 2023-09-10T04:39:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:39:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:39:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:39:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:39:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:39:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:39:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:57 smithi170 ceph-mon[102832]: pgmap v10415: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 307 KiB/s wr, 16 op/s 2023-09-10T04:39:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:57 smithi082 ceph-mon[98694]: pgmap v10415: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 307 KiB/s wr, 16 op/s 2023-09-10T04:39:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:57 smithi098 ceph-mon[103124]: pgmap v10415: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 307 KiB/s wr, 16 op/s 2023-09-10T04:39:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19229.221387 secs 2023-09-10T04:39:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19229.221387 secs 2023-09-10T04:39:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19229.221387 secs 2023-09-10T04:39:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:59 smithi170 ceph-mon[102832]: pgmap v10416: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 187 KiB/s wr, 11 op/s 2023-09-10T04:39:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19230.343642 secs 2023-09-10T04:39:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19230.998408 secs 2023-09-10T04:39:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:39:59 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:39:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:59 smithi082 ceph-mon[98694]: pgmap v10416: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 187 KiB/s wr, 11 op/s 2023-09-10T04:39:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19230.343642 secs 2023-09-10T04:39:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19230.998408 secs 2023-09-10T04:39:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:39:59 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:39:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:59 smithi098 ceph-mon[103124]: pgmap v10416: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 187 KiB/s wr, 11 op/s 2023-09-10T04:39:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19230.343642 secs 2023-09-10T04:39:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19230.998408 secs 2023-09-10T04:39:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:39:59 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:40:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T04:40:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T04:40:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T04:40:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:01 smithi170 ceph-mon[102832]: pgmap v10417: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 187 KiB/s wr, 11 op/s 2023-09-10T04:40:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:01 smithi082 ceph-mon[98694]: pgmap v10417: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 187 KiB/s wr, 11 op/s 2023-09-10T04:40:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:01 smithi098 ceph-mon[103124]: pgmap v10417: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 187 KiB/s wr, 11 op/s 2023-09-10T04:40:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:03 smithi170 ceph-mon[102832]: pgmap v10418: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 17 op/s 2023-09-10T04:40:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19234.221447 secs 2023-09-10T04:40:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:03 smithi082 ceph-mon[98694]: pgmap v10418: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 17 op/s 2023-09-10T04:40:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19234.221447 secs 2023-09-10T04:40:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:03 smithi098 ceph-mon[103124]: pgmap v10418: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 17 op/s 2023-09-10T04:40:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19234.221447 secs 2023-09-10T04:40:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19235.343701 secs 2023-09-10T04:40:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19235.998475 secs 2023-09-10T04:40:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19235.343701 secs 2023-09-10T04:40:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19235.998475 secs 2023-09-10T04:40:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19235.343701 secs 2023-09-10T04:40:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19235.998475 secs 2023-09-10T04:40:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:05 smithi170 ceph-mon[102832]: pgmap v10419: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 161 KiB/s wr, 11 op/s 2023-09-10T04:40:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:05 smithi082 ceph-mon[98694]: pgmap v10419: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 161 KiB/s wr, 11 op/s 2023-09-10T04:40:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:05 smithi098 ceph-mon[103124]: pgmap v10419: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 161 KiB/s wr, 11 op/s 2023-09-10T04:40:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:07 smithi170 ceph-mon[102832]: pgmap v10420: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 267 KiB/s wr, 30 op/s 2023-09-10T04:40:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:07 smithi082 ceph-mon[98694]: pgmap v10420: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 267 KiB/s wr, 30 op/s 2023-09-10T04:40:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:07 smithi098 ceph-mon[103124]: pgmap v10420: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 267 KiB/s wr, 30 op/s 2023-09-10T04:40:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19239.221546 secs 2023-09-10T04:40:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19239.221546 secs 2023-09-10T04:40:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19239.221546 secs 2023-09-10T04:40:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:09 smithi170 ceph-mon[102832]: pgmap v10421: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 202 KiB/s wr, 25 op/s 2023-09-10T04:40:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19240.343759 secs 2023-09-10T04:40:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:09 smithi170 ceph-mon[102832]: osdmap e719: 12 total, 12 up, 12 in 2023-09-10T04:40:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19240.998542 secs 2023-09-10T04:40:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:09 smithi082 ceph-mon[98694]: pgmap v10421: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 202 KiB/s wr, 25 op/s 2023-09-10T04:40:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19240.343759 secs 2023-09-10T04:40:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:09 smithi082 ceph-mon[98694]: osdmap e719: 12 total, 12 up, 12 in 2023-09-10T04:40:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19240.998542 secs 2023-09-10T04:40:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:09 smithi098 ceph-mon[103124]: pgmap v10421: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 202 KiB/s wr, 25 op/s 2023-09-10T04:40:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19240.343759 secs 2023-09-10T04:40:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:09 smithi098 ceph-mon[103124]: osdmap e719: 12 total, 12 up, 12 in 2023-09-10T04:40:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19240.998542 secs 2023-09-10T04:40:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:11 smithi170 ceph-mon[102832]: pgmap v10423: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 243 KiB/s wr, 30 op/s 2023-09-10T04:40:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:11 smithi082 ceph-mon[98694]: pgmap v10423: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 243 KiB/s wr, 30 op/s 2023-09-10T04:40:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:11 smithi098 ceph-mon[103124]: pgmap v10423: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 243 KiB/s wr, 30 op/s 2023-09-10T04:40:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:13 smithi170 ceph-mon[102832]: pgmap v10424: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T04:40:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19244.221609 secs 2023-09-10T04:40:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:13 smithi082 ceph-mon[98694]: pgmap v10424: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T04:40:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19244.221609 secs 2023-09-10T04:40:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:13 smithi098 ceph-mon[103124]: pgmap v10424: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 128 KiB/s wr, 29 op/s 2023-09-10T04:40:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19244.221609 secs 2023-09-10T04:40:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19245.343817 secs 2023-09-10T04:40:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19245.998604 secs 2023-09-10T04:40:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:14 smithi170 ceph-mon[102832]: osdmap e720: 12 total, 12 up, 12 in 2023-09-10T04:40:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19245.343817 secs 2023-09-10T04:40:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19245.998604 secs 2023-09-10T04:40:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:14 smithi082 ceph-mon[98694]: osdmap e720: 12 total, 12 up, 12 in 2023-09-10T04:40:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19245.343817 secs 2023-09-10T04:40:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19245.998604 secs 2023-09-10T04:40:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:14 smithi098 ceph-mon[103124]: osdmap e720: 12 total, 12 up, 12 in 2023-09-10T04:40:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:15 smithi170 ceph-mon[102832]: pgmap v10426: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:40:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:15 smithi082 ceph-mon[98694]: pgmap v10426: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:40:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:15 smithi098 ceph-mon[103124]: pgmap v10426: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:40:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:17 smithi170 ceph-mon[102832]: pgmap v10427: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 147 KiB/s wr, 15 op/s 2023-09-10T04:40:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:17 smithi082 ceph-mon[98694]: pgmap v10427: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 147 KiB/s wr, 15 op/s 2023-09-10T04:40:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:17 smithi098 ceph-mon[103124]: pgmap v10427: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 147 KiB/s wr, 15 op/s 2023-09-10T04:40:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19249.221686 secs 2023-09-10T04:40:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19249.221686 secs 2023-09-10T04:40:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19249.221686 secs 2023-09-10T04:40:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:19 smithi170 ceph-mon[102832]: pgmap v10428: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T04:40:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19250.343877 secs 2023-09-10T04:40:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19250.998674 secs 2023-09-10T04:40:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:19 smithi082 ceph-mon[98694]: pgmap v10428: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T04:40:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19250.343877 secs 2023-09-10T04:40:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19250.998674 secs 2023-09-10T04:40:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:19 smithi098 ceph-mon[103124]: pgmap v10428: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T04:40:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19250.343877 secs 2023-09-10T04:40:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19250.998674 secs 2023-09-10T04:40:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:21 smithi170 ceph-mon[102832]: pgmap v10429: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T04:40:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:21 smithi082 ceph-mon[98694]: pgmap v10429: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T04:40:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:21 smithi098 ceph-mon[103124]: pgmap v10429: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T04:40:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:23 smithi170 ceph-mon[102832]: pgmap v10430: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T04:40:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19254.221757 secs 2023-09-10T04:40:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19255.343938 secs 2023-09-10T04:40:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:23 smithi082 ceph-mon[98694]: pgmap v10430: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T04:40:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19254.221757 secs 2023-09-10T04:40:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19255.343938 secs 2023-09-10T04:40:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:23 smithi098 ceph-mon[103124]: pgmap v10430: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T04:40:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19254.221757 secs 2023-09-10T04:40:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19255.343938 secs 2023-09-10T04:40:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19255.998738 secs 2023-09-10T04:40:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19255.998738 secs 2023-09-10T04:40:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19255.998738 secs 2023-09-10T04:40:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:25 smithi170 ceph-mon[102832]: pgmap v10431: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T04:40:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:25 smithi082 ceph-mon[98694]: pgmap v10431: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T04:40:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:25 smithi098 ceph-mon[103124]: pgmap v10431: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T04:40:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:27 smithi170 ceph-mon[102832]: pgmap v10432: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 156 KiB/s wr, 15 op/s 2023-09-10T04:40:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:27 smithi082 ceph-mon[98694]: pgmap v10432: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 156 KiB/s wr, 15 op/s 2023-09-10T04:40:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:27 smithi098 ceph-mon[103124]: pgmap v10432: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 156 KiB/s wr, 15 op/s 2023-09-10T04:40:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19260.344000 secs 2023-09-10T04:40:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19260.344000 secs 2023-09-10T04:40:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19260.344000 secs 2023-09-10T04:40:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:29 smithi170 ceph-mon[102832]: pgmap v10433: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:40:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19260.998805 secs 2023-09-10T04:40:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:29 smithi082 ceph-mon[98694]: pgmap v10433: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:40:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19260.998805 secs 2023-09-10T04:40:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:29 smithi098 ceph-mon[103124]: pgmap v10433: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:40:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19260.998805 secs 2023-09-10T04:40:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:40:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:40:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:40:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:40:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:40:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:40:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:31 smithi170 ceph-mon[102832]: pgmap v10434: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:40:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:40:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:31 smithi082 ceph-mon[98694]: pgmap v10434: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:40:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:40:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:31 smithi098 ceph-mon[103124]: pgmap v10434: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:40:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:40:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:40:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:40:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:40:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:40:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:40:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:40:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:40:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:40:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:40:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:40:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:40:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:40:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:33 smithi170 ceph-mon[102832]: pgmap v10435: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 191 KiB/s wr, 15 op/s 2023-09-10T04:40:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:40:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:40:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:40:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19265.344058 secs 2023-09-10T04:40:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:33 smithi082 ceph-mon[98694]: pgmap v10435: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 191 KiB/s wr, 15 op/s 2023-09-10T04:40:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:40:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:40:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:40:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19265.344058 secs 2023-09-10T04:40:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:33 smithi098 ceph-mon[103124]: pgmap v10435: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 191 KiB/s wr, 15 op/s 2023-09-10T04:40:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:40:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:40:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:40:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19265.344058 secs 2023-09-10T04:40:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19265.998895 secs 2023-09-10T04:40:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19265.998895 secs 2023-09-10T04:40:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19265.998895 secs 2023-09-10T04:40:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:35 smithi170 ceph-mon[102832]: pgmap v10436: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T04:40:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:35 smithi082 ceph-mon[98694]: pgmap v10436: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T04:40:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:35 smithi098 ceph-mon[103124]: pgmap v10436: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T04:40:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:37 smithi170 ceph-mon[102832]: pgmap v10437: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 317 KiB/s wr, 17 op/s 2023-09-10T04:40:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:37 smithi082 ceph-mon[98694]: pgmap v10437: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 317 KiB/s wr, 17 op/s 2023-09-10T04:40:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:37 smithi098 ceph-mon[103124]: pgmap v10437: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 317 KiB/s wr, 17 op/s 2023-09-10T04:40:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19269.221955 secs 2023-09-10T04:40:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19270.344126 secs 2023-09-10T04:40:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19269.221955 secs 2023-09-10T04:40:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19270.344126 secs 2023-09-10T04:40:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19269.221955 secs 2023-09-10T04:40:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19270.344126 secs 2023-09-10T04:40:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:39 smithi170 ceph-mon[102832]: pgmap v10438: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 260 KiB/s wr, 11 op/s 2023-09-10T04:40:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19270.998959 secs 2023-09-10T04:40:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:39 smithi082 ceph-mon[98694]: pgmap v10438: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 260 KiB/s wr, 11 op/s 2023-09-10T04:40:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19270.998959 secs 2023-09-10T04:40:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:39 smithi098 ceph-mon[103124]: pgmap v10438: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 260 KiB/s wr, 11 op/s 2023-09-10T04:40:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19270.998959 secs 2023-09-10T04:40:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:40:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:40:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:40:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:40:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:40:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:40:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:40:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:40:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:40:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:41 smithi170 ceph-mon[102832]: pgmap v10439: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 260 KiB/s wr, 11 op/s 2023-09-10T04:40:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:41 smithi082 ceph-mon[98694]: pgmap v10439: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 260 KiB/s wr, 11 op/s 2023-09-10T04:40:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:41 smithi098 ceph-mon[103124]: pgmap v10439: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 260 KiB/s wr, 11 op/s 2023-09-10T04:40:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:43 smithi170 ceph-mon[102832]: pgmap v10440: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 375 KiB/s wr, 18 op/s 2023-09-10T04:40:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19274.222015 secs 2023-09-10T04:40:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19275.344189 secs 2023-09-10T04:40:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:43 smithi082 ceph-mon[98694]: pgmap v10440: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 375 KiB/s wr, 18 op/s 2023-09-10T04:40:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19274.222015 secs 2023-09-10T04:40:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19275.344189 secs 2023-09-10T04:40:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:43 smithi098 ceph-mon[103124]: pgmap v10440: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 375 KiB/s wr, 18 op/s 2023-09-10T04:40:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19274.222015 secs 2023-09-10T04:40:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19275.344189 secs 2023-09-10T04:40:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19275.999026 secs 2023-09-10T04:40:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19275.999026 secs 2023-09-10T04:40:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19275.999026 secs 2023-09-10T04:40:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:45 smithi082 ceph-mon[98694]: pgmap v10441: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 242 KiB/s wr, 12 op/s 2023-09-10T04:40:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:45 smithi098 ceph-mon[103124]: pgmap v10441: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 242 KiB/s wr, 12 op/s 2023-09-10T04:40:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:45 smithi170 ceph-mon[102832]: pgmap v10441: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 242 KiB/s wr, 12 op/s 2023-09-10T04:40:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:47 smithi082 ceph-mon[98694]: pgmap v10442: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 359 KiB/s wr, 18 op/s 2023-09-10T04:40:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:47 smithi098 ceph-mon[103124]: pgmap v10442: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 359 KiB/s wr, 18 op/s 2023-09-10T04:40:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:47 smithi170 ceph-mon[102832]: pgmap v10442: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 359 KiB/s wr, 18 op/s 2023-09-10T04:40:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19280.344275 secs 2023-09-10T04:40:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19280.344275 secs 2023-09-10T04:40:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19280.344275 secs 2023-09-10T04:40:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:49 smithi082 ceph-mon[98694]: pgmap v10443: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 232 KiB/s wr, 13 op/s 2023-09-10T04:40:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19280.999093 secs 2023-09-10T04:40:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:49 smithi098 ceph-mon[103124]: pgmap v10443: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 232 KiB/s wr, 13 op/s 2023-09-10T04:40:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19280.999093 secs 2023-09-10T04:40:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:49 smithi170 ceph-mon[102832]: pgmap v10443: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 232 KiB/s wr, 13 op/s 2023-09-10T04:40:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19280.999093 secs 2023-09-10T04:40:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:40:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:40:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:40:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:40:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:40:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:40:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:51 smithi082 ceph-mon[98694]: pgmap v10444: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 232 KiB/s wr, 13 op/s 2023-09-10T04:40:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:51 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:40:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:51 smithi098 ceph-mon[103124]: pgmap v10444: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 232 KiB/s wr, 13 op/s 2023-09-10T04:40:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:51 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:40:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:51 smithi170 ceph-mon[102832]: pgmap v10444: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 232 KiB/s wr, 13 op/s 2023-09-10T04:40:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:51 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:40:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:52 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:40:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:40:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:40:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:52 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:40:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:40:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:40:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:52 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:40:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:40:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:40:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:53 smithi098 ceph-mon[103124]: pgmap v10445: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 366 KiB/s wr, 19 op/s 2023-09-10T04:40:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19284.222147 secs 2023-09-10T04:40:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19285.344332 secs 2023-09-10T04:40:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:53 smithi170 ceph-mon[102832]: pgmap v10445: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 366 KiB/s wr, 19 op/s 2023-09-10T04:40:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19284.222147 secs 2023-09-10T04:40:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19285.344332 secs 2023-09-10T04:40:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:53 smithi082 ceph-mon[98694]: pgmap v10445: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 366 KiB/s wr, 19 op/s 2023-09-10T04:40:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19284.222147 secs 2023-09-10T04:40:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19285.344332 secs 2023-09-10T04:40:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19285.999156 secs 2023-09-10T04:40:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19285.999156 secs 2023-09-10T04:40:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19285.999156 secs 2023-09-10T04:40:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:55 smithi098 ceph-mon[103124]: pgmap v10446: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 250 KiB/s wr, 13 op/s 2023-09-10T04:40:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:55 smithi170 ceph-mon[102832]: pgmap v10446: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 250 KiB/s wr, 13 op/s 2023-09-10T04:40:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:55 smithi082 ceph-mon[98694]: pgmap v10446: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 250 KiB/s wr, 13 op/s 2023-09-10T04:40:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:40:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:40:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:40:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:40:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:40:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:40:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:57 smithi098 ceph-mon[103124]: pgmap v10447: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 315 KiB/s wr, 19 op/s 2023-09-10T04:40:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:57 smithi170 ceph-mon[102832]: pgmap v10447: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 315 KiB/s wr, 19 op/s 2023-09-10T04:40:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:57 smithi082 ceph-mon[98694]: pgmap v10447: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 315 KiB/s wr, 19 op/s 2023-09-10T04:40:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:40:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:40:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19289.222206 secs 2023-09-10T04:40:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19290.344428 secs 2023-09-10T04:40:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:40:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:40:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19289.222206 secs 2023-09-10T04:40:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19290.344428 secs 2023-09-10T04:40:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:40:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:40:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19289.222206 secs 2023-09-10T04:40:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19290.344428 secs 2023-09-10T04:40:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:59 smithi098 ceph-mon[103124]: pgmap v10448: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 12 op/s 2023-09-10T04:40:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19290.999220 secs 2023-09-10T04:40:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:40:59 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:40:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:59 smithi170 ceph-mon[102832]: pgmap v10448: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 12 op/s 2023-09-10T04:40:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19290.999220 secs 2023-09-10T04:40:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:40:59 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:40:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:59 smithi082 ceph-mon[98694]: pgmap v10448: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 12 op/s 2023-09-10T04:40:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19290.999220 secs 2023-09-10T04:40:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:40:59 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:41:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:41:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:41:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:41:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:41:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:41:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:41:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:01 smithi170 ceph-mon[102832]: pgmap v10449: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 12 op/s 2023-09-10T04:41:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:01 smithi082 ceph-mon[98694]: pgmap v10449: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 12 op/s 2023-09-10T04:41:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:01 smithi098 ceph-mon[103124]: pgmap v10449: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 12 op/s 2023-09-10T04:41:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:03 smithi170 ceph-mon[102832]: pgmap v10450: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 286 KiB/s wr, 18 op/s 2023-09-10T04:41:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19294.222265 secs 2023-09-10T04:41:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19295.344494 secs 2023-09-10T04:41:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:03 smithi082 ceph-mon[98694]: pgmap v10450: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 286 KiB/s wr, 18 op/s 2023-09-10T04:41:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19294.222265 secs 2023-09-10T04:41:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19295.344494 secs 2023-09-10T04:41:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:03 smithi098 ceph-mon[103124]: pgmap v10450: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 286 KiB/s wr, 18 op/s 2023-09-10T04:41:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19294.222265 secs 2023-09-10T04:41:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19295.344494 secs 2023-09-10T04:41:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19295.999287 secs 2023-09-10T04:41:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19295.999287 secs 2023-09-10T04:41:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19295.999287 secs 2023-09-10T04:41:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:05 smithi170 ceph-mon[102832]: pgmap v10451: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 12 op/s 2023-09-10T04:41:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:05 smithi082 ceph-mon[98694]: pgmap v10451: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 12 op/s 2023-09-10T04:41:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:05 smithi098 ceph-mon[103124]: pgmap v10451: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 12 op/s 2023-09-10T04:41:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:07 smithi170 ceph-mon[102832]: pgmap v10452: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 245 KiB/s wr, 31 op/s 2023-09-10T04:41:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:07 smithi082 ceph-mon[98694]: pgmap v10452: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 245 KiB/s wr, 31 op/s 2023-09-10T04:41:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:07 smithi098 ceph-mon[103124]: pgmap v10452: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 245 KiB/s wr, 31 op/s 2023-09-10T04:41:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19299.222326 secs 2023-09-10T04:41:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19300.344579 secs 2023-09-10T04:41:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19299.222326 secs 2023-09-10T04:41:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19300.344579 secs 2023-09-10T04:41:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19299.222326 secs 2023-09-10T04:41:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19300.344579 secs 2023-09-10T04:41:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:09 smithi170 ceph-mon[102832]: pgmap v10453: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 180 KiB/s wr, 25 op/s 2023-09-10T04:41:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:09 smithi170 ceph-mon[102832]: osdmap e721: 12 total, 12 up, 12 in 2023-09-10T04:41:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19300.999352 secs 2023-09-10T04:41:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:09 smithi082 ceph-mon[98694]: pgmap v10453: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 180 KiB/s wr, 25 op/s 2023-09-10T04:41:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:09 smithi082 ceph-mon[98694]: osdmap e721: 12 total, 12 up, 12 in 2023-09-10T04:41:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19300.999352 secs 2023-09-10T04:41:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:09 smithi098 ceph-mon[103124]: pgmap v10453: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 180 KiB/s wr, 25 op/s 2023-09-10T04:41:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:09 smithi098 ceph-mon[103124]: osdmap e721: 12 total, 12 up, 12 in 2023-09-10T04:41:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19300.999352 secs 2023-09-10T04:41:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:11 smithi170 ceph-mon[102832]: pgmap v10455: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 217 KiB/s wr, 31 op/s 2023-09-10T04:41:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:11 smithi082 ceph-mon[98694]: pgmap v10455: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 217 KiB/s wr, 31 op/s 2023-09-10T04:41:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:11 smithi098 ceph-mon[103124]: pgmap v10455: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 217 KiB/s wr, 31 op/s 2023-09-10T04:41:13.286 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:13 smithi170 ceph-mon[102832]: pgmap v10456: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 112 KiB/s wr, 29 op/s 2023-09-10T04:41:13.287 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19304.222386 secs 2023-09-10T04:41:13.287 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19305.344660 secs 2023-09-10T04:41:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:13 smithi082 ceph-mon[98694]: pgmap v10456: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 112 KiB/s wr, 29 op/s 2023-09-10T04:41:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19304.222386 secs 2023-09-10T04:41:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19305.344660 secs 2023-09-10T04:41:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:13 smithi098 ceph-mon[103124]: pgmap v10456: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 112 KiB/s wr, 29 op/s 2023-09-10T04:41:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19304.222386 secs 2023-09-10T04:41:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19305.344660 secs 2023-09-10T04:41:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19305.999417 secs 2023-09-10T04:41:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:14 smithi170 ceph-mon[102832]: osdmap e722: 12 total, 12 up, 12 in 2023-09-10T04:41:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19305.999417 secs 2023-09-10T04:41:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:14 smithi082 ceph-mon[98694]: osdmap e722: 12 total, 12 up, 12 in 2023-09-10T04:41:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19305.999417 secs 2023-09-10T04:41:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:14 smithi098 ceph-mon[103124]: osdmap e722: 12 total, 12 up, 12 in 2023-09-10T04:41:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:15 smithi170 ceph-mon[102832]: pgmap v10458: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:41:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:15 smithi082 ceph-mon[98694]: pgmap v10458: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:41:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:15 smithi098 ceph-mon[103124]: pgmap v10458: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:41:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:17 smithi170 ceph-mon[102832]: pgmap v10459: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 14 op/s 2023-09-10T04:41:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:17 smithi082 ceph-mon[98694]: pgmap v10459: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 14 op/s 2023-09-10T04:41:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:17 smithi098 ceph-mon[103124]: pgmap v10459: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 14 op/s 2023-09-10T04:41:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19309.222521 secs 2023-09-10T04:41:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19310.344724 secs 2023-09-10T04:41:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19309.222521 secs 2023-09-10T04:41:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19310.344724 secs 2023-09-10T04:41:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19309.222521 secs 2023-09-10T04:41:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19310.344724 secs 2023-09-10T04:41:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:19 smithi170 ceph-mon[102832]: pgmap v10460: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T04:41:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19310.999488 secs 2023-09-10T04:41:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:19 smithi082 ceph-mon[98694]: pgmap v10460: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T04:41:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19310.999488 secs 2023-09-10T04:41:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:19 smithi098 ceph-mon[103124]: pgmap v10460: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T04:41:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19310.999488 secs 2023-09-10T04:41:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:21 smithi170 ceph-mon[102832]: pgmap v10461: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T04:41:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:21 smithi082 ceph-mon[98694]: pgmap v10461: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T04:41:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:21 smithi098 ceph-mon[103124]: pgmap v10461: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T04:41:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:23 smithi170 ceph-mon[102832]: pgmap v10462: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T04:41:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19314.222582 secs 2023-09-10T04:41:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19315.344782 secs 2023-09-10T04:41:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:23 smithi082 ceph-mon[98694]: pgmap v10462: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T04:41:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19314.222582 secs 2023-09-10T04:41:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19315.344782 secs 2023-09-10T04:41:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:23 smithi098 ceph-mon[103124]: pgmap v10462: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T04:41:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19314.222582 secs 2023-09-10T04:41:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19315.344782 secs 2023-09-10T04:41:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19315.999552 secs 2023-09-10T04:41:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19315.999552 secs 2023-09-10T04:41:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19315.999552 secs 2023-09-10T04:41:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:25 smithi170 ceph-mon[102832]: pgmap v10463: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T04:41:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:25 smithi082 ceph-mon[98694]: pgmap v10463: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T04:41:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:25 smithi098 ceph-mon[103124]: pgmap v10463: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T04:41:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:27 smithi170 ceph-mon[102832]: pgmap v10464: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 156 KiB/s wr, 14 op/s 2023-09-10T04:41:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:27 smithi082 ceph-mon[98694]: pgmap v10464: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 156 KiB/s wr, 14 op/s 2023-09-10T04:41:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:27 smithi098 ceph-mon[103124]: pgmap v10464: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 156 KiB/s wr, 14 op/s 2023-09-10T04:41:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19320.344846 secs 2023-09-10T04:41:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19320.344846 secs 2023-09-10T04:41:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19320.344846 secs 2023-09-10T04:41:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:29 smithi170 ceph-mon[102832]: pgmap v10465: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:41:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19320.999619 secs 2023-09-10T04:41:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:29 smithi082 ceph-mon[98694]: pgmap v10465: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:41:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19320.999619 secs 2023-09-10T04:41:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:29 smithi098 ceph-mon[103124]: pgmap v10465: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:41:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19320.999619 secs 2023-09-10T04:41:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:31 smithi170 ceph-mon[102832]: pgmap v10466: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:41:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:41:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:31 smithi082 ceph-mon[98694]: pgmap v10466: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:41:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:41:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:31 smithi098 ceph-mon[103124]: pgmap v10466: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 9 op/s 2023-09-10T04:41:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:41:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:41:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:41:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:41:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:41:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:41:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:41:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:41:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:41:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:41:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:33 smithi170 ceph-mon[102832]: pgmap v10467: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 198 KiB/s wr, 15 op/s 2023-09-10T04:41:33.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:41:33.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19325.344906 secs 2023-09-10T04:41:33.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:33 smithi082 ceph-mon[98694]: pgmap v10467: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 198 KiB/s wr, 15 op/s 2023-09-10T04:41:33.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:41:33.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19325.344906 secs 2023-09-10T04:41:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:33 smithi098 ceph-mon[103124]: pgmap v10467: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 198 KiB/s wr, 15 op/s 2023-09-10T04:41:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:41:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19325.344906 secs 2023-09-10T04:41:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19325.999685 secs 2023-09-10T04:41:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19325.999685 secs 2023-09-10T04:41:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19325.999685 secs 2023-09-10T04:41:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:35 smithi170 ceph-mon[102832]: pgmap v10468: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 198 KiB/s wr, 10 op/s 2023-09-10T04:41:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:41:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:41:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:35 smithi082 ceph-mon[98694]: pgmap v10468: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 198 KiB/s wr, 10 op/s 2023-09-10T04:41:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:41:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:41:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:35 smithi098 ceph-mon[103124]: pgmap v10468: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 198 KiB/s wr, 10 op/s 2023-09-10T04:41:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:41:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:41:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:41:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:41:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:41:36.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:41:36.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:36 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 18995k 2023-09-10T04:41:36.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:36 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 19451048: error parsing value: Value '19451048' is below minimum 939524096 2023-09-10T04:41:36.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:41:36.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:41:36.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:41:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:41:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:41:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:41:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:41:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:36 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 18995k 2023-09-10T04:41:36.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:36 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 19451048: error parsing value: Value '19451048' is below minimum 939524096 2023-09-10T04:41:36.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.932 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:41:36.932 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:41:36.932 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:41:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:41:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:41:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:41:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:41:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:36 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 18995k 2023-09-10T04:41:36.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:36 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 19451048: error parsing value: Value '19451048' is below minimum 939524096 2023-09-10T04:41:36.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:41:36.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:41:36.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:41:36.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:41:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:37 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T04:41:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:37 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T04:41:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:37 smithi170 ceph-mon[102832]: pgmap v10469: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 320 KiB/s wr, 17 op/s 2023-09-10T04:41:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:37 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T04:41:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:37 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T04:41:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:37 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T04:41:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:37 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T04:41:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:37 smithi082 ceph-mon[98694]: pgmap v10469: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 320 KiB/s wr, 17 op/s 2023-09-10T04:41:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:37 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T04:41:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:37 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T04:41:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:37 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T04:41:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:37 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T04:41:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:37 smithi098 ceph-mon[103124]: pgmap v10469: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 320 KiB/s wr, 17 op/s 2023-09-10T04:41:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:37 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T04:41:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:37 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T04:41:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19329.222775 secs 2023-09-10T04:41:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:41:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:41:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19330.344962 secs 2023-09-10T04:41:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19329.222775 secs 2023-09-10T04:41:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:41:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:41:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19330.344962 secs 2023-09-10T04:41:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19329.222775 secs 2023-09-10T04:41:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:41:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:41:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19330.344962 secs 2023-09-10T04:41:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:39 smithi170 ceph-mon[102832]: pgmap v10470: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T04:41:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19330.999751 secs 2023-09-10T04:41:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:39 smithi082 ceph-mon[98694]: pgmap v10470: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T04:41:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19330.999751 secs 2023-09-10T04:41:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:39 smithi098 ceph-mon[103124]: pgmap v10470: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T04:41:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19330.999751 secs 2023-09-10T04:41:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:41:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:41:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:41:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:41:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:41:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:41:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:41:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:41:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:41:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:41 smithi170 ceph-mon[102832]: pgmap v10471: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T04:41:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:41 smithi082 ceph-mon[98694]: pgmap v10471: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T04:41:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:41 smithi098 ceph-mon[103124]: pgmap v10471: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T04:41:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:43 smithi170 ceph-mon[102832]: pgmap v10472: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 375 KiB/s wr, 18 op/s 2023-09-10T04:41:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19334.222864 secs 2023-09-10T04:41:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19335.345057 secs 2023-09-10T04:41:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:43 smithi082 ceph-mon[98694]: pgmap v10472: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 375 KiB/s wr, 18 op/s 2023-09-10T04:41:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19334.222864 secs 2023-09-10T04:41:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19335.345057 secs 2023-09-10T04:41:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:43 smithi098 ceph-mon[103124]: pgmap v10472: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 375 KiB/s wr, 18 op/s 2023-09-10T04:41:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19334.222864 secs 2023-09-10T04:41:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19335.345057 secs 2023-09-10T04:41:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19335.999812 secs 2023-09-10T04:41:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19335.999812 secs 2023-09-10T04:41:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19335.999812 secs 2023-09-10T04:41:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:45 smithi170 ceph-mon[102832]: pgmap v10473: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 235 KiB/s wr, 12 op/s 2023-09-10T04:41:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:45 smithi082 ceph-mon[98694]: pgmap v10473: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 235 KiB/s wr, 12 op/s 2023-09-10T04:41:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:45 smithi098 ceph-mon[103124]: pgmap v10473: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 235 KiB/s wr, 12 op/s 2023-09-10T04:41:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:47 smithi170 ceph-mon[102832]: pgmap v10474: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 374 KiB/s wr, 18 op/s 2023-09-10T04:41:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:47 smithi082 ceph-mon[98694]: pgmap v10474: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 374 KiB/s wr, 18 op/s 2023-09-10T04:41:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:47 smithi098 ceph-mon[103124]: pgmap v10474: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 374 KiB/s wr, 18 op/s 2023-09-10T04:41:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19340.345141 secs 2023-09-10T04:41:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19340.345141 secs 2023-09-10T04:41:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19340.345141 secs 2023-09-10T04:41:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:49 smithi170 ceph-mon[102832]: pgmap v10475: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 251 KiB/s wr, 11 op/s 2023-09-10T04:41:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:41:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:41:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19340.999884 secs 2023-09-10T04:41:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:49 smithi082 ceph-mon[98694]: pgmap v10475: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 251 KiB/s wr, 11 op/s 2023-09-10T04:41:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:41:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:41:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19340.999884 secs 2023-09-10T04:41:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:49 smithi098 ceph-mon[103124]: pgmap v10475: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 251 KiB/s wr, 11 op/s 2023-09-10T04:41:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:41:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:41:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19340.999884 secs 2023-09-10T04:41:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:51 smithi170 ceph-mon[102832]: pgmap v10476: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 251 KiB/s wr, 11 op/s 2023-09-10T04:41:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:51 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:41:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:51 smithi082 ceph-mon[98694]: pgmap v10476: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 251 KiB/s wr, 11 op/s 2023-09-10T04:41:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:51 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:41:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:51 smithi098 ceph-mon[103124]: pgmap v10476: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 251 KiB/s wr, 11 op/s 2023-09-10T04:41:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:51 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:41:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:52 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:41:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:41:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:41:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:52 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:41:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:41:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:41:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:52 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:41:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:41:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:41:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:53 smithi170 ceph-mon[102832]: pgmap v10477: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 359 KiB/s wr, 18 op/s 2023-09-10T04:41:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19344.223004 secs 2023-09-10T04:41:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19345.345230 secs 2023-09-10T04:41:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:53 smithi082 ceph-mon[98694]: pgmap v10477: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 359 KiB/s wr, 18 op/s 2023-09-10T04:41:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19344.223004 secs 2023-09-10T04:41:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19345.345230 secs 2023-09-10T04:41:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:53 smithi098 ceph-mon[103124]: pgmap v10477: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 359 KiB/s wr, 18 op/s 2023-09-10T04:41:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19344.223004 secs 2023-09-10T04:41:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19345.345230 secs 2023-09-10T04:41:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19345.999947 secs 2023-09-10T04:41:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19345.999947 secs 2023-09-10T04:41:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19345.999947 secs 2023-09-10T04:41:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:55 smithi170 ceph-mon[102832]: pgmap v10478: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 246 KiB/s wr, 11 op/s 2023-09-10T04:41:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:55 smithi082 ceph-mon[98694]: pgmap v10478: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 246 KiB/s wr, 11 op/s 2023-09-10T04:41:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:55 smithi098 ceph-mon[103124]: pgmap v10478: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 246 KiB/s wr, 11 op/s 2023-09-10T04:41:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:41:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:41:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:41:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:41:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:41:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:41:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:57 smithi170 ceph-mon[102832]: pgmap v10479: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 312 KiB/s wr, 17 op/s 2023-09-10T04:41:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:57 smithi082 ceph-mon[98694]: pgmap v10479: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 312 KiB/s wr, 17 op/s 2023-09-10T04:41:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:57 smithi098 ceph-mon[103124]: pgmap v10479: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 312 KiB/s wr, 17 op/s 2023-09-10T04:41:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19349.223064 secs 2023-09-10T04:41:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19350.345288 secs 2023-09-10T04:41:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19349.223064 secs 2023-09-10T04:41:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19350.345288 secs 2023-09-10T04:41:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19349.223064 secs 2023-09-10T04:41:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19350.345288 secs 2023-09-10T04:41:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:59 smithi170 ceph-mon[102832]: pgmap v10480: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T04:41:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19351.000012 secs 2023-09-10T04:41:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:41:59 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:41:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:59 smithi082 ceph-mon[98694]: pgmap v10480: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T04:41:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19351.000012 secs 2023-09-10T04:41:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:41:59 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:42:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:59 smithi098 ceph-mon[103124]: pgmap v10480: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T04:42:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19351.000012 secs 2023-09-10T04:42:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:41:59 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:42:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:01 smithi170 ceph-mon[102832]: pgmap v10481: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T04:42:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:01 smithi082 ceph-mon[98694]: pgmap v10481: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T04:42:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:01 smithi098 ceph-mon[103124]: pgmap v10481: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T04:42:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:03 smithi170 ceph-mon[102832]: pgmap v10482: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 262 KiB/s wr, 17 op/s 2023-09-10T04:42:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19354.223125 secs 2023-09-10T04:42:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19355.345375 secs 2023-09-10T04:42:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:03 smithi082 ceph-mon[98694]: pgmap v10482: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 262 KiB/s wr, 17 op/s 2023-09-10T04:42:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19354.223125 secs 2023-09-10T04:42:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19355.345375 secs 2023-09-10T04:42:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:03 smithi098 ceph-mon[103124]: pgmap v10482: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 262 KiB/s wr, 17 op/s 2023-09-10T04:42:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19354.223125 secs 2023-09-10T04:42:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19355.345375 secs 2023-09-10T04:42:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19356.000194 secs 2023-09-10T04:42:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19356.000194 secs 2023-09-10T04:42:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19356.000194 secs 2023-09-10T04:42:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:05 smithi170 ceph-mon[102832]: pgmap v10483: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 154 KiB/s wr, 10 op/s 2023-09-10T04:42:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:05 smithi082 ceph-mon[98694]: pgmap v10483: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 154 KiB/s wr, 10 op/s 2023-09-10T04:42:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:05 smithi098 ceph-mon[103124]: pgmap v10483: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 154 KiB/s wr, 10 op/s 2023-09-10T04:42:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:07 smithi170 ceph-mon[102832]: pgmap v10484: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 246 KiB/s wr, 30 op/s 2023-09-10T04:42:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:07 smithi082 ceph-mon[98694]: pgmap v10484: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 246 KiB/s wr, 30 op/s 2023-09-10T04:42:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:07 smithi098 ceph-mon[103124]: pgmap v10484: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 246 KiB/s wr, 30 op/s 2023-09-10T04:42:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19359.223187 secs 2023-09-10T04:42:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:08 smithi170 ceph-mon[102832]: osdmap e723: 12 total, 12 up, 12 in 2023-09-10T04:42:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19360.345448 secs 2023-09-10T04:42:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19359.223187 secs 2023-09-10T04:42:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:08 smithi082 ceph-mon[98694]: osdmap e723: 12 total, 12 up, 12 in 2023-09-10T04:42:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19360.345448 secs 2023-09-10T04:42:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19359.223187 secs 2023-09-10T04:42:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:08 smithi098 ceph-mon[103124]: osdmap e723: 12 total, 12 up, 12 in 2023-09-10T04:42:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19360.345448 secs 2023-09-10T04:42:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:09 smithi170 ceph-mon[102832]: pgmap v10486: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 217 KiB/s wr, 29 op/s 2023-09-10T04:42:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19361.000260 secs 2023-09-10T04:42:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:09 smithi082 ceph-mon[98694]: pgmap v10486: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 217 KiB/s wr, 29 op/s 2023-09-10T04:42:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19361.000260 secs 2023-09-10T04:42:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:09 smithi098 ceph-mon[103124]: pgmap v10486: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 217 KiB/s wr, 29 op/s 2023-09-10T04:42:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19361.000260 secs 2023-09-10T04:42:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:11 smithi170 ceph-mon[102832]: pgmap v10487: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 217 KiB/s wr, 29 op/s 2023-09-10T04:42:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:11 smithi082 ceph-mon[98694]: pgmap v10487: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 217 KiB/s wr, 29 op/s 2023-09-10T04:42:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:11 smithi098 ceph-mon[103124]: pgmap v10487: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 217 KiB/s wr, 29 op/s 2023-09-10T04:42:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:13 smithi170 ceph-mon[102832]: pgmap v10488: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 112 KiB/s wr, 28 op/s 2023-09-10T04:42:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19364.223274 secs 2023-09-10T04:42:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19365.345497 secs 2023-09-10T04:42:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:13 smithi082 ceph-mon[98694]: pgmap v10488: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 112 KiB/s wr, 28 op/s 2023-09-10T04:42:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19364.223274 secs 2023-09-10T04:42:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19365.345497 secs 2023-09-10T04:42:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:13 smithi098 ceph-mon[103124]: pgmap v10488: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 112 KiB/s wr, 28 op/s 2023-09-10T04:42:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19364.223274 secs 2023-09-10T04:42:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19365.345497 secs 2023-09-10T04:42:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19366.000324 secs 2023-09-10T04:42:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:14 smithi170 ceph-mon[102832]: osdmap e724: 12 total, 12 up, 12 in 2023-09-10T04:42:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19366.000324 secs 2023-09-10T04:42:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:14 smithi082 ceph-mon[98694]: osdmap e724: 12 total, 12 up, 12 in 2023-09-10T04:42:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19366.000324 secs 2023-09-10T04:42:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:14 smithi098 ceph-mon[103124]: osdmap e724: 12 total, 12 up, 12 in 2023-09-10T04:42:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:15 smithi170 ceph-mon[102832]: pgmap v10490: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 6 op/s 2023-09-10T04:42:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:15 smithi082 ceph-mon[98694]: pgmap v10490: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 6 op/s 2023-09-10T04:42:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:15 smithi098 ceph-mon[103124]: pgmap v10490: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 6 op/s 2023-09-10T04:42:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:17 smithi170 ceph-mon[102832]: pgmap v10491: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 14 op/s 2023-09-10T04:42:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:17 smithi082 ceph-mon[98694]: pgmap v10491: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 14 op/s 2023-09-10T04:42:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:17 smithi098 ceph-mon[103124]: pgmap v10491: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 14 op/s 2023-09-10T04:42:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19369.223337 secs 2023-09-10T04:42:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19370.345581 secs 2023-09-10T04:42:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19369.223337 secs 2023-09-10T04:42:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19370.345581 secs 2023-09-10T04:42:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19369.223337 secs 2023-09-10T04:42:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19370.345581 secs 2023-09-10T04:42:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:19 smithi082 ceph-mon[98694]: pgmap v10492: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T04:42:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19371.000392 secs 2023-09-10T04:42:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:19 smithi098 ceph-mon[103124]: pgmap v10492: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T04:42:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19371.000392 secs 2023-09-10T04:42:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:19 smithi170 ceph-mon[102832]: pgmap v10492: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T04:42:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19371.000392 secs 2023-09-10T04:42:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:21 smithi082 ceph-mon[98694]: pgmap v10493: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T04:42:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:21 smithi098 ceph-mon[103124]: pgmap v10493: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T04:42:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:21 smithi170 ceph-mon[102832]: pgmap v10493: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T04:42:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:23 smithi082 ceph-mon[98694]: pgmap v10494: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 188 KiB/s wr, 12 op/s 2023-09-10T04:42:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19374.223401 secs 2023-09-10T04:42:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19375.345640 secs 2023-09-10T04:42:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:23 smithi098 ceph-mon[103124]: pgmap v10494: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 188 KiB/s wr, 12 op/s 2023-09-10T04:42:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19374.223401 secs 2023-09-10T04:42:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19375.345640 secs 2023-09-10T04:42:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:23 smithi170 ceph-mon[102832]: pgmap v10494: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 188 KiB/s wr, 12 op/s 2023-09-10T04:42:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19374.223401 secs 2023-09-10T04:42:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19375.345640 secs 2023-09-10T04:42:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:42:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:42:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19376.000458 secs 2023-09-10T04:42:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:42:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:42:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19376.000458 secs 2023-09-10T04:42:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:42:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:42:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19376.000458 secs 2023-09-10T04:42:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:25 smithi082 ceph-mon[98694]: pgmap v10495: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 187 KiB/s wr, 12 op/s 2023-09-10T04:42:25.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:25 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:42:25.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:25 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:42:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:25 smithi098 ceph-mon[103124]: pgmap v10495: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 187 KiB/s wr, 12 op/s 2023-09-10T04:42:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:25 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:42:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:25 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:42:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:25 smithi170 ceph-mon[102832]: pgmap v10495: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 187 KiB/s wr, 12 op/s 2023-09-10T04:42:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:25 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:42:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:25 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:42:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:27 smithi082 ceph-mon[98694]: pgmap v10496: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 215 KiB/s wr, 15 op/s 2023-09-10T04:42:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:27 smithi098 ceph-mon[103124]: pgmap v10496: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 215 KiB/s wr, 15 op/s 2023-09-10T04:42:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:27 smithi170 ceph-mon[102832]: pgmap v10496: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 215 KiB/s wr, 15 op/s 2023-09-10T04:42:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19380.345698 secs 2023-09-10T04:42:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19380.345698 secs 2023-09-10T04:42:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19380.345698 secs 2023-09-10T04:42:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:29 smithi082 ceph-mon[98694]: pgmap v10497: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 116 KiB/s wr, 10 op/s 2023-09-10T04:42:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19381.000487 secs 2023-09-10T04:42:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:29 smithi098 ceph-mon[103124]: pgmap v10497: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 116 KiB/s wr, 10 op/s 2023-09-10T04:42:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19381.000487 secs 2023-09-10T04:42:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:29 smithi170 ceph-mon[102832]: pgmap v10497: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 116 KiB/s wr, 10 op/s 2023-09-10T04:42:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19381.000487 secs 2023-09-10T04:42:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:31 smithi098 ceph-mon[103124]: pgmap v10498: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 116 KiB/s wr, 10 op/s 2023-09-10T04:42:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:42:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:31 smithi170 ceph-mon[102832]: pgmap v10498: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 116 KiB/s wr, 10 op/s 2023-09-10T04:42:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:42:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:31 smithi082 ceph-mon[98694]: pgmap v10498: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 116 KiB/s wr, 10 op/s 2023-09-10T04:42:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:42:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:42:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:42:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:42:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:42:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:42:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:42:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:42:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:42:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:42:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:33 smithi098 ceph-mon[103124]: pgmap v10499: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 193 KiB/s wr, 15 op/s 2023-09-10T04:42:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19385.345819 secs 2023-09-10T04:42:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:33 smithi170 ceph-mon[102832]: pgmap v10499: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 193 KiB/s wr, 15 op/s 2023-09-10T04:42:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19385.345819 secs 2023-09-10T04:42:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:33 smithi082 ceph-mon[98694]: pgmap v10499: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 193 KiB/s wr, 15 op/s 2023-09-10T04:42:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19385.345819 secs 2023-09-10T04:42:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19386.000552 secs 2023-09-10T04:42:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19386.000552 secs 2023-09-10T04:42:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19386.000552 secs 2023-09-10T04:42:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:35 smithi098 ceph-mon[103124]: pgmap v10500: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 135 KiB/s wr, 10 op/s 2023-09-10T04:42:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:35 smithi170 ceph-mon[102832]: pgmap v10500: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 135 KiB/s wr, 10 op/s 2023-09-10T04:42:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:35 smithi082 ceph-mon[98694]: pgmap v10500: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 135 KiB/s wr, 10 op/s 2023-09-10T04:42:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:42:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:42:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:42:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:37 smithi098 ceph-mon[103124]: pgmap v10501: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 295 KiB/s wr, 16 op/s 2023-09-10T04:42:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:42:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:42:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:42:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:42:38.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:42:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:37 smithi170 ceph-mon[102832]: pgmap v10501: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 295 KiB/s wr, 16 op/s 2023-09-10T04:42:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:42:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:42:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:42:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:42:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:42:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:37 smithi082 ceph-mon[98694]: pgmap v10501: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 295 KiB/s wr, 16 op/s 2023-09-10T04:42:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:42:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:42:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:42:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:42:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:42:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19389.223588 secs 2023-09-10T04:42:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19390.345879 secs 2023-09-10T04:42:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19389.223588 secs 2023-09-10T04:42:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19390.345879 secs 2023-09-10T04:42:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19389.223588 secs 2023-09-10T04:42:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19390.345879 secs 2023-09-10T04:42:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:39 smithi098 ceph-mon[103124]: pgmap v10502: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 238 KiB/s wr, 11 op/s 2023-09-10T04:42:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19391.000620 secs 2023-09-10T04:42:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:39 smithi170 ceph-mon[102832]: pgmap v10502: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 238 KiB/s wr, 11 op/s 2023-09-10T04:42:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19391.000620 secs 2023-09-10T04:42:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:39 smithi082 ceph-mon[98694]: pgmap v10502: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 238 KiB/s wr, 11 op/s 2023-09-10T04:42:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19391.000620 secs 2023-09-10T04:42:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:42:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:42:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:42:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:42:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:42:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:42:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:42:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:42:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:42:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:41 smithi098 ceph-mon[103124]: pgmap v10503: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 238 KiB/s wr, 11 op/s 2023-09-10T04:42:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:41 smithi170 ceph-mon[102832]: pgmap v10503: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 238 KiB/s wr, 11 op/s 2023-09-10T04:42:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:41 smithi082 ceph-mon[98694]: pgmap v10503: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 238 KiB/s wr, 11 op/s 2023-09-10T04:42:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:43 smithi098 ceph-mon[103124]: pgmap v10504: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 320 KiB/s wr, 17 op/s 2023-09-10T04:42:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19394.223666 secs 2023-09-10T04:42:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19395.345962 secs 2023-09-10T04:42:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:43 smithi170 ceph-mon[102832]: pgmap v10504: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 320 KiB/s wr, 17 op/s 2023-09-10T04:42:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19394.223666 secs 2023-09-10T04:42:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19395.345962 secs 2023-09-10T04:42:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:43 smithi082 ceph-mon[98694]: pgmap v10504: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 320 KiB/s wr, 17 op/s 2023-09-10T04:42:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19394.223666 secs 2023-09-10T04:42:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19395.345962 secs 2023-09-10T04:42:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19396.000688 secs 2023-09-10T04:42:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19396.000688 secs 2023-09-10T04:42:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19396.000688 secs 2023-09-10T04:42:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:45 smithi098 ceph-mon[103124]: pgmap v10505: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 242 KiB/s wr, 12 op/s 2023-09-10T04:42:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:45 smithi170 ceph-mon[102832]: pgmap v10505: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 242 KiB/s wr, 12 op/s 2023-09-10T04:42:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:45 smithi082 ceph-mon[98694]: pgmap v10505: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 242 KiB/s wr, 12 op/s 2023-09-10T04:42:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:47 smithi098 ceph-mon[103124]: pgmap v10506: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 364 KiB/s wr, 18 op/s 2023-09-10T04:42:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:47 smithi170 ceph-mon[102832]: pgmap v10506: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 364 KiB/s wr, 18 op/s 2023-09-10T04:42:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:47 smithi082 ceph-mon[98694]: pgmap v10506: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 364 KiB/s wr, 18 op/s 2023-09-10T04:42:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19399.223737 secs 2023-09-10T04:42:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19400.346020 secs 2023-09-10T04:42:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19399.223737 secs 2023-09-10T04:42:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19400.346020 secs 2023-09-10T04:42:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19399.223737 secs 2023-09-10T04:42:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19400.346020 secs 2023-09-10T04:42:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:49 smithi098 ceph-mon[103124]: pgmap v10507: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 204 KiB/s wr, 12 op/s 2023-09-10T04:42:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19401.000754 secs 2023-09-10T04:42:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:49 smithi170 ceph-mon[102832]: pgmap v10507: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 204 KiB/s wr, 12 op/s 2023-09-10T04:42:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19401.000754 secs 2023-09-10T04:42:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:49 smithi082 ceph-mon[98694]: pgmap v10507: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 204 KiB/s wr, 12 op/s 2023-09-10T04:42:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19401.000754 secs 2023-09-10T04:42:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:51 smithi170 ceph-mon[102832]: pgmap v10508: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 204 KiB/s wr, 12 op/s 2023-09-10T04:42:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:51 smithi082 ceph-mon[98694]: pgmap v10508: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 204 KiB/s wr, 12 op/s 2023-09-10T04:42:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:51 smithi098 ceph-mon[103124]: pgmap v10508: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 204 KiB/s wr, 12 op/s 2023-09-10T04:42:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:53 smithi170 ceph-mon[102832]: pgmap v10509: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 330 KiB/s wr, 17 op/s 2023-09-10T04:42:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19404.223777 secs 2023-09-10T04:42:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19405.346081 secs 2023-09-10T04:42:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:53 smithi082 ceph-mon[98694]: pgmap v10509: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 330 KiB/s wr, 17 op/s 2023-09-10T04:42:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19404.223777 secs 2023-09-10T04:42:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19405.346081 secs 2023-09-10T04:42:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:53 smithi098 ceph-mon[103124]: pgmap v10509: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 330 KiB/s wr, 17 op/s 2023-09-10T04:42:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19404.223777 secs 2023-09-10T04:42:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19405.346081 secs 2023-09-10T04:42:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19406.000818 secs 2023-09-10T04:42:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19406.000818 secs 2023-09-10T04:42:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19406.000818 secs 2023-09-10T04:42:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:55 smithi170 ceph-mon[102832]: pgmap v10510: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 248 KiB/s wr, 11 op/s 2023-09-10T04:42:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:55 smithi082 ceph-mon[98694]: pgmap v10510: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 248 KiB/s wr, 11 op/s 2023-09-10T04:42:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:55 smithi098 ceph-mon[103124]: pgmap v10510: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 248 KiB/s wr, 11 op/s 2023-09-10T04:42:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:57 smithi170 ceph-mon[102832]: pgmap v10511: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 313 KiB/s wr, 17 op/s 2023-09-10T04:42:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:42:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:42:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:57 smithi082 ceph-mon[98694]: pgmap v10511: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 313 KiB/s wr, 17 op/s 2023-09-10T04:42:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:42:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:42:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:57 smithi098 ceph-mon[103124]: pgmap v10511: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 313 KiB/s wr, 17 op/s 2023-09-10T04:42:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:42:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:42:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19409.223892 secs 2023-09-10T04:42:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19410.346187 secs 2023-09-10T04:42:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19409.223892 secs 2023-09-10T04:42:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19410.346187 secs 2023-09-10T04:42:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19409.223892 secs 2023-09-10T04:42:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19410.346187 secs 2023-09-10T04:43:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:59 smithi170 ceph-mon[102832]: pgmap v10512: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T04:43:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:42:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19411.000882 secs 2023-09-10T04:43:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:59 smithi082 ceph-mon[98694]: pgmap v10512: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T04:43:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:42:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19411.000882 secs 2023-09-10T04:43:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:59 smithi098 ceph-mon[103124]: pgmap v10512: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T04:43:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:42:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19411.000882 secs 2023-09-10T04:43:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:43:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:43:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:00 smithi170 ceph-mon[102832]: pgmap v10513: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T04:43:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:43:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:43:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:00 smithi082 ceph-mon[98694]: pgmap v10513: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T04:43:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:43:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:43:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:00 smithi098 ceph-mon[103124]: pgmap v10513: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T04:43:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:03 smithi170 ceph-mon[102832]: pgmap v10514: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 280 KiB/s wr, 17 op/s 2023-09-10T04:43:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19414.223963 secs 2023-09-10T04:43:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:03 smithi082 ceph-mon[98694]: pgmap v10514: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 280 KiB/s wr, 17 op/s 2023-09-10T04:43:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19414.223963 secs 2023-09-10T04:43:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:03 smithi098 ceph-mon[103124]: pgmap v10514: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 280 KiB/s wr, 17 op/s 2023-09-10T04:43:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19414.223963 secs 2023-09-10T04:43:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19415.346245 secs 2023-09-10T04:43:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19416.000945 secs 2023-09-10T04:43:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19415.346245 secs 2023-09-10T04:43:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19416.000945 secs 2023-09-10T04:43:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19415.346245 secs 2023-09-10T04:43:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19416.000945 secs 2023-09-10T04:43:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:05 smithi170 ceph-mon[102832]: pgmap v10515: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T04:43:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:05 smithi082 ceph-mon[98694]: pgmap v10515: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T04:43:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:05 smithi098 ceph-mon[103124]: pgmap v10515: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T04:43:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:07 smithi170 ceph-mon[102832]: pgmap v10516: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 262 KiB/s wr, 31 op/s 2023-09-10T04:43:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:07 smithi082 ceph-mon[98694]: pgmap v10516: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 262 KiB/s wr, 31 op/s 2023-09-10T04:43:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:07 smithi098 ceph-mon[103124]: pgmap v10516: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 262 KiB/s wr, 31 op/s 2023-09-10T04:43:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19419.224024 secs 2023-09-10T04:43:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19419.224024 secs 2023-09-10T04:43:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19419.224024 secs 2023-09-10T04:43:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:09 smithi170 ceph-mon[102832]: pgmap v10517: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 197 KiB/s wr, 25 op/s 2023-09-10T04:43:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19420.346333 secs 2023-09-10T04:43:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:09 smithi170 ceph-mon[102832]: osdmap e725: 12 total, 12 up, 12 in 2023-09-10T04:43:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19421.001013 secs 2023-09-10T04:43:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:09 smithi082 ceph-mon[98694]: pgmap v10517: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 197 KiB/s wr, 25 op/s 2023-09-10T04:43:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19420.346333 secs 2023-09-10T04:43:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:09 smithi082 ceph-mon[98694]: osdmap e725: 12 total, 12 up, 12 in 2023-09-10T04:43:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19421.001013 secs 2023-09-10T04:43:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:09 smithi098 ceph-mon[103124]: pgmap v10517: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 197 KiB/s wr, 25 op/s 2023-09-10T04:43:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19420.346333 secs 2023-09-10T04:43:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:09 smithi098 ceph-mon[103124]: osdmap e725: 12 total, 12 up, 12 in 2023-09-10T04:43:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19421.001013 secs 2023-09-10T04:43:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:11 smithi170 ceph-mon[102832]: pgmap v10519: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 236 KiB/s wr, 30 op/s 2023-09-10T04:43:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:11 smithi082 ceph-mon[98694]: pgmap v10519: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 236 KiB/s wr, 30 op/s 2023-09-10T04:43:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:11 smithi098 ceph-mon[103124]: pgmap v10519: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 236 KiB/s wr, 30 op/s 2023-09-10T04:43:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:13 smithi170 ceph-mon[102832]: pgmap v10520: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 130 KiB/s wr, 29 op/s 2023-09-10T04:43:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19424.224084 secs 2023-09-10T04:43:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:13 smithi082 ceph-mon[98694]: pgmap v10520: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 130 KiB/s wr, 29 op/s 2023-09-10T04:43:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19424.224084 secs 2023-09-10T04:43:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:13 smithi098 ceph-mon[103124]: pgmap v10520: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 130 KiB/s wr, 29 op/s 2023-09-10T04:43:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19424.224084 secs 2023-09-10T04:43:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19425.346392 secs 2023-09-10T04:43:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19426.001080 secs 2023-09-10T04:43:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:14 smithi170 ceph-mon[102832]: osdmap e726: 12 total, 12 up, 12 in 2023-09-10T04:43:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19425.346392 secs 2023-09-10T04:43:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19426.001080 secs 2023-09-10T04:43:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:14 smithi082 ceph-mon[98694]: osdmap e726: 12 total, 12 up, 12 in 2023-09-10T04:43:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19425.346392 secs 2023-09-10T04:43:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19426.001080 secs 2023-09-10T04:43:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:14 smithi098 ceph-mon[103124]: osdmap e726: 12 total, 12 up, 12 in 2023-09-10T04:43:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:15 smithi170 ceph-mon[102832]: pgmap v10522: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 6 op/s 2023-09-10T04:43:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:15 smithi082 ceph-mon[98694]: pgmap v10522: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 6 op/s 2023-09-10T04:43:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:15 smithi098 ceph-mon[103124]: pgmap v10522: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 6 op/s 2023-09-10T04:43:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:17 smithi170 ceph-mon[102832]: pgmap v10523: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 149 KiB/s wr, 14 op/s 2023-09-10T04:43:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:17 smithi082 ceph-mon[98694]: pgmap v10523: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 149 KiB/s wr, 14 op/s 2023-09-10T04:43:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:17 smithi098 ceph-mon[103124]: pgmap v10523: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 149 KiB/s wr, 14 op/s 2023-09-10T04:43:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19429.224144 secs 2023-09-10T04:43:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19429.224144 secs 2023-09-10T04:43:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19429.224144 secs 2023-09-10T04:43:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:19 smithi170 ceph-mon[102832]: pgmap v10524: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T04:43:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19430.346454 secs 2023-09-10T04:43:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19431.001145 secs 2023-09-10T04:43:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:19 smithi082 ceph-mon[98694]: pgmap v10524: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T04:43:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19430.346454 secs 2023-09-10T04:43:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19431.001145 secs 2023-09-10T04:43:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:19 smithi098 ceph-mon[103124]: pgmap v10524: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T04:43:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19430.346454 secs 2023-09-10T04:43:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19431.001145 secs 2023-09-10T04:43:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:21 smithi170 ceph-mon[102832]: pgmap v10525: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T04:43:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:21 smithi082 ceph-mon[98694]: pgmap v10525: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T04:43:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:21 smithi098 ceph-mon[103124]: pgmap v10525: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T04:43:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:23 smithi170 ceph-mon[102832]: pgmap v10526: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T04:43:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19434.224205 secs 2023-09-10T04:43:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:23 smithi082 ceph-mon[98694]: pgmap v10526: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T04:43:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19434.224205 secs 2023-09-10T04:43:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:23 smithi098 ceph-mon[103124]: pgmap v10526: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T04:43:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19434.224205 secs 2023-09-10T04:43:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19435.346542 secs 2023-09-10T04:43:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19436.001209 secs 2023-09-10T04:43:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19435.346542 secs 2023-09-10T04:43:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19436.001209 secs 2023-09-10T04:43:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19435.346542 secs 2023-09-10T04:43:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19436.001209 secs 2023-09-10T04:43:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:25 smithi170 ceph-mon[102832]: pgmap v10527: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T04:43:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:25 smithi082 ceph-mon[98694]: pgmap v10527: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T04:43:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:25 smithi098 ceph-mon[103124]: pgmap v10527: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T04:43:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:27 smithi170 ceph-mon[102832]: pgmap v10528: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 215 KiB/s wr, 14 op/s 2023-09-10T04:43:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:27 smithi082 ceph-mon[98694]: pgmap v10528: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 215 KiB/s wr, 14 op/s 2023-09-10T04:43:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:27 smithi098 ceph-mon[103124]: pgmap v10528: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 215 KiB/s wr, 14 op/s 2023-09-10T04:43:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:29 smithi170 ceph-mon[102832]: pgmap v10529: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 9 op/s 2023-09-10T04:43:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19440.346606 secs 2023-09-10T04:43:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19441.001274 secs 2023-09-10T04:43:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:29 smithi082 ceph-mon[98694]: pgmap v10529: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 9 op/s 2023-09-10T04:43:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19440.346606 secs 2023-09-10T04:43:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19441.001274 secs 2023-09-10T04:43:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:29 smithi098 ceph-mon[103124]: pgmap v10529: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 9 op/s 2023-09-10T04:43:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19440.346606 secs 2023-09-10T04:43:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19441.001274 secs 2023-09-10T04:43:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:43:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:43:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:43:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:43:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:43:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:43:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:31 smithi170 ceph-mon[102832]: pgmap v10530: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 9 op/s 2023-09-10T04:43:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:43:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:31 smithi082 ceph-mon[98694]: pgmap v10530: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 9 op/s 2023-09-10T04:43:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:43:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:31 smithi098 ceph-mon[103124]: pgmap v10530: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 9 op/s 2023-09-10T04:43:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:43:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:43:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:43:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:43:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:43:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:43:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:43:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:43:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:43:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:43:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:33 smithi170 ceph-mon[102832]: pgmap v10531: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 194 KiB/s wr, 15 op/s 2023-09-10T04:43:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19445.346663 secs 2023-09-10T04:43:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:33 smithi082 ceph-mon[98694]: pgmap v10531: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 194 KiB/s wr, 15 op/s 2023-09-10T04:43:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19445.346663 secs 2023-09-10T04:43:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:33 smithi098 ceph-mon[103124]: pgmap v10531: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 194 KiB/s wr, 15 op/s 2023-09-10T04:43:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19445.346663 secs 2023-09-10T04:43:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19446.001337 secs 2023-09-10T04:43:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19446.001337 secs 2023-09-10T04:43:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19446.001337 secs 2023-09-10T04:43:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:35 smithi170 ceph-mon[102832]: pgmap v10532: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T04:43:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:35 smithi082 ceph-mon[98694]: pgmap v10532: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T04:43:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:35 smithi098 ceph-mon[103124]: pgmap v10532: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T04:43:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:37 smithi170 ceph-mon[102832]: pgmap v10533: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 323 KiB/s wr, 17 op/s 2023-09-10T04:43:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:43:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:37 smithi082 ceph-mon[98694]: pgmap v10533: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 323 KiB/s wr, 17 op/s 2023-09-10T04:43:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:43:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:37 smithi098 ceph-mon[103124]: pgmap v10533: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 323 KiB/s wr, 17 op/s 2023-09-10T04:43:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:43:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:43:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:43:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:43:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:43:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:43:38.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19450.346722 secs 2023-09-10T04:43:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:43:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:43:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:43:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:43:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:43:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19450.346722 secs 2023-09-10T04:43:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:43:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:43:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:43:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:43:38.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:43:38.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19450.346722 secs 2023-09-10T04:43:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:39 smithi170 ceph-mon[102832]: pgmap v10534: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 207 KiB/s wr, 12 op/s 2023-09-10T04:43:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19451.001402 secs 2023-09-10T04:43:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:39 smithi082 ceph-mon[98694]: pgmap v10534: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 207 KiB/s wr, 12 op/s 2023-09-10T04:43:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19451.001402 secs 2023-09-10T04:43:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:39 smithi098 ceph-mon[103124]: pgmap v10534: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 207 KiB/s wr, 12 op/s 2023-09-10T04:43:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19451.001402 secs 2023-09-10T04:43:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:41 smithi170 ceph-mon[102832]: pgmap v10535: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 207 KiB/s wr, 12 op/s 2023-09-10T04:43:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:41 smithi082 ceph-mon[98694]: pgmap v10535: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 207 KiB/s wr, 12 op/s 2023-09-10T04:43:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:41 smithi098 ceph-mon[103124]: pgmap v10535: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 207 KiB/s wr, 12 op/s 2023-09-10T04:43:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:43 smithi170 ceph-mon[102832]: pgmap v10536: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 322 KiB/s wr, 18 op/s 2023-09-10T04:43:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19455.346833 secs 2023-09-10T04:43:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:43 smithi082 ceph-mon[98694]: pgmap v10536: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 322 KiB/s wr, 18 op/s 2023-09-10T04:43:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19455.346833 secs 2023-09-10T04:43:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:43 smithi098 ceph-mon[103124]: pgmap v10536: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 322 KiB/s wr, 18 op/s 2023-09-10T04:43:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19455.346833 secs 2023-09-10T04:43:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19456.001471 secs 2023-09-10T04:43:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19456.001471 secs 2023-09-10T04:43:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19456.001471 secs 2023-09-10T04:43:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:45 smithi170 ceph-mon[102832]: pgmap v10537: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 244 KiB/s wr, 12 op/s 2023-09-10T04:43:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:45 smithi082 ceph-mon[98694]: pgmap v10537: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 244 KiB/s wr, 12 op/s 2023-09-10T04:43:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:45 smithi098 ceph-mon[103124]: pgmap v10537: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 244 KiB/s wr, 12 op/s 2023-09-10T04:43:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:47 smithi170 ceph-mon[102832]: pgmap v10538: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 366 KiB/s wr, 18 op/s 2023-09-10T04:43:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:47 smithi082 ceph-mon[98694]: pgmap v10538: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 366 KiB/s wr, 18 op/s 2023-09-10T04:43:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:47 smithi098 ceph-mon[103124]: pgmap v10538: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 366 KiB/s wr, 18 op/s 2023-09-10T04:43:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19460.346888 secs 2023-09-10T04:43:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19460.346888 secs 2023-09-10T04:43:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19460.346888 secs 2023-09-10T04:43:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:49 smithi170 ceph-mon[102832]: pgmap v10539: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 237 KiB/s wr, 12 op/s 2023-09-10T04:43:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19461.001489 secs 2023-09-10T04:43:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:49 smithi082 ceph-mon[98694]: pgmap v10539: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 237 KiB/s wr, 12 op/s 2023-09-10T04:43:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19461.001489 secs 2023-09-10T04:43:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:49 smithi098 ceph-mon[103124]: pgmap v10539: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 237 KiB/s wr, 12 op/s 2023-09-10T04:43:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19461.001489 secs 2023-09-10T04:43:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:43:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:43:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:43:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:43:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:43:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:43:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:51 smithi170 ceph-mon[102832]: pgmap v10540: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 237 KiB/s wr, 12 op/s 2023-09-10T04:43:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:51 smithi082 ceph-mon[98694]: pgmap v10540: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 237 KiB/s wr, 12 op/s 2023-09-10T04:43:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:51 smithi098 ceph-mon[103124]: pgmap v10540: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 237 KiB/s wr, 12 op/s 2023-09-10T04:43:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:43:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:43:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:43:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:43:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:43:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:43:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:53 smithi082 ceph-mon[98694]: pgmap v10541: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 363 KiB/s wr, 18 op/s 2023-09-10T04:43:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19464.224670 secs 2023-09-10T04:43:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19465.346946 secs 2023-09-10T04:43:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:53 smithi098 ceph-mon[103124]: pgmap v10541: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 363 KiB/s wr, 18 op/s 2023-09-10T04:43:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19464.224670 secs 2023-09-10T04:43:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19465.346946 secs 2023-09-10T04:43:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:53 smithi170 ceph-mon[102832]: pgmap v10541: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 363 KiB/s wr, 18 op/s 2023-09-10T04:43:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19464.224670 secs 2023-09-10T04:43:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19465.346946 secs 2023-09-10T04:43:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19466.001555 secs 2023-09-10T04:43:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19466.001555 secs 2023-09-10T04:43:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19466.001555 secs 2023-09-10T04:43:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:55 smithi082 ceph-mon[98694]: pgmap v10542: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 249 KiB/s wr, 12 op/s 2023-09-10T04:43:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:55 smithi098 ceph-mon[103124]: pgmap v10542: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 249 KiB/s wr, 12 op/s 2023-09-10T04:43:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:55 smithi170 ceph-mon[102832]: pgmap v10542: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 249 KiB/s wr, 12 op/s 2023-09-10T04:43:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:43:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:43:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:43:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:43:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:43:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:43:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:43:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:43:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:43:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:57 smithi082 ceph-mon[98694]: pgmap v10543: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 315 KiB/s wr, 17 op/s 2023-09-10T04:43:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:57 smithi098 ceph-mon[103124]: pgmap v10543: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 315 KiB/s wr, 17 op/s 2023-09-10T04:43:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:57 smithi170 ceph-mon[102832]: pgmap v10543: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 315 KiB/s wr, 17 op/s 2023-09-10T04:43:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19469.224727 secs 2023-09-10T04:43:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19470.347004 secs 2023-09-10T04:43:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19469.224727 secs 2023-09-10T04:43:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19470.347004 secs 2023-09-10T04:43:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19469.224727 secs 2023-09-10T04:43:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19470.347004 secs 2023-09-10T04:43:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:59 smithi082 ceph-mon[98694]: pgmap v10544: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 11 op/s 2023-09-10T04:43:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:43:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19471.001632 secs 2023-09-10T04:43:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:59 smithi098 ceph-mon[103124]: pgmap v10544: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 11 op/s 2023-09-10T04:43:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:43:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19471.001632 secs 2023-09-10T04:43:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:59 smithi170 ceph-mon[102832]: pgmap v10544: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 11 op/s 2023-09-10T04:43:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:43:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19471.001632 secs 2023-09-10T04:44:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:01 smithi098 ceph-mon[103124]: pgmap v10545: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 11 op/s 2023-09-10T04:44:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:01 smithi170 ceph-mon[102832]: pgmap v10545: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 11 op/s 2023-09-10T04:44:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:01 smithi082 ceph-mon[98694]: pgmap v10545: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 11 op/s 2023-09-10T04:44:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:03 smithi098 ceph-mon[103124]: pgmap v10546: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 306 KiB/s wr, 21 op/s 2023-09-10T04:44:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19474.224819 secs 2023-09-10T04:44:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19475.347061 secs 2023-09-10T04:44:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:03 smithi170 ceph-mon[102832]: pgmap v10546: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 306 KiB/s wr, 21 op/s 2023-09-10T04:44:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19474.224819 secs 2023-09-10T04:44:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19475.347061 secs 2023-09-10T04:44:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:03 smithi082 ceph-mon[98694]: pgmap v10546: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 306 KiB/s wr, 21 op/s 2023-09-10T04:44:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19474.224819 secs 2023-09-10T04:44:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19475.347061 secs 2023-09-10T04:44:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19476.001693 secs 2023-09-10T04:44:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19476.001693 secs 2023-09-10T04:44:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19476.001693 secs 2023-09-10T04:44:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:05 smithi098 ceph-mon[103124]: pgmap v10547: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 180 KiB/s wr, 14 op/s 2023-09-10T04:44:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:05 smithi170 ceph-mon[102832]: pgmap v10547: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 180 KiB/s wr, 14 op/s 2023-09-10T04:44:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:05 smithi082 ceph-mon[98694]: pgmap v10547: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 180 KiB/s wr, 14 op/s 2023-09-10T04:44:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:07 smithi098 ceph-mon[103124]: pgmap v10548: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 263 KiB/s wr, 30 op/s 2023-09-10T04:44:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:07 smithi170 ceph-mon[102832]: pgmap v10548: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 263 KiB/s wr, 30 op/s 2023-09-10T04:44:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:07 smithi082 ceph-mon[98694]: pgmap v10548: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 263 KiB/s wr, 30 op/s 2023-09-10T04:44:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19479.224894 secs 2023-09-10T04:44:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19480.347121 secs 2023-09-10T04:44:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19479.224894 secs 2023-09-10T04:44:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19480.347121 secs 2023-09-10T04:44:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19479.224894 secs 2023-09-10T04:44:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19480.347121 secs 2023-09-10T04:44:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:09 smithi098 ceph-mon[103124]: pgmap v10549: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 197 KiB/s wr, 25 op/s 2023-09-10T04:44:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:09 smithi098 ceph-mon[103124]: osdmap e727: 12 total, 12 up, 12 in 2023-09-10T04:44:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19481.001762 secs 2023-09-10T04:44:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:09 smithi170 ceph-mon[102832]: pgmap v10549: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 197 KiB/s wr, 25 op/s 2023-09-10T04:44:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:09 smithi170 ceph-mon[102832]: osdmap e727: 12 total, 12 up, 12 in 2023-09-10T04:44:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19481.001762 secs 2023-09-10T04:44:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:09 smithi082 ceph-mon[98694]: pgmap v10549: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 197 KiB/s wr, 25 op/s 2023-09-10T04:44:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:09 smithi082 ceph-mon[98694]: osdmap e727: 12 total, 12 up, 12 in 2023-09-10T04:44:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19481.001762 secs 2023-09-10T04:44:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:11 smithi098 ceph-mon[103124]: pgmap v10551: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 237 KiB/s wr, 30 op/s 2023-09-10T04:44:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:11 smithi170 ceph-mon[102832]: pgmap v10551: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 237 KiB/s wr, 30 op/s 2023-09-10T04:44:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:11 smithi082 ceph-mon[98694]: pgmap v10551: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 237 KiB/s wr, 30 op/s 2023-09-10T04:44:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:13 smithi098 ceph-mon[103124]: pgmap v10552: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 100 KiB/s wr, 24 op/s 2023-09-10T04:44:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19484.224956 secs 2023-09-10T04:44:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19485.347180 secs 2023-09-10T04:44:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:13 smithi170 ceph-mon[102832]: pgmap v10552: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 100 KiB/s wr, 24 op/s 2023-09-10T04:44:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19484.224956 secs 2023-09-10T04:44:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19485.347180 secs 2023-09-10T04:44:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:13 smithi082 ceph-mon[98694]: pgmap v10552: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 100 KiB/s wr, 24 op/s 2023-09-10T04:44:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19484.224956 secs 2023-09-10T04:44:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19485.347180 secs 2023-09-10T04:44:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19486.001828 secs 2023-09-10T04:44:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:14 smithi098 ceph-mon[103124]: osdmap e728: 12 total, 12 up, 12 in 2023-09-10T04:44:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19486.001828 secs 2023-09-10T04:44:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:14 smithi170 ceph-mon[102832]: osdmap e728: 12 total, 12 up, 12 in 2023-09-10T04:44:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19486.001828 secs 2023-09-10T04:44:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:14 smithi082 ceph-mon[98694]: osdmap e728: 12 total, 12 up, 12 in 2023-09-10T04:44:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:15 smithi098 ceph-mon[103124]: pgmap v10554: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:44:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:15 smithi170 ceph-mon[102832]: pgmap v10554: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:44:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:15 smithi082 ceph-mon[98694]: pgmap v10554: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:44:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:17 smithi098 ceph-mon[103124]: pgmap v10555: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 16 op/s 2023-09-10T04:44:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:17 smithi170 ceph-mon[102832]: pgmap v10555: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 16 op/s 2023-09-10T04:44:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:17 smithi082 ceph-mon[98694]: pgmap v10555: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 16 op/s 2023-09-10T04:44:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:44:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:44:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19489.225015 secs 2023-09-10T04:44:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19490.347238 secs 2023-09-10T04:44:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:44:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:44:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19489.225015 secs 2023-09-10T04:44:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19490.347238 secs 2023-09-10T04:44:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:44:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:44:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19489.225015 secs 2023-09-10T04:44:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19490.347238 secs 2023-09-10T04:44:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:19 smithi098 ceph-mon[103124]: pgmap v10556: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 13 op/s 2023-09-10T04:44:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19491.001895 secs 2023-09-10T04:44:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:19 smithi170 ceph-mon[102832]: pgmap v10556: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 13 op/s 2023-09-10T04:44:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19491.001895 secs 2023-09-10T04:44:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:19 smithi082 ceph-mon[98694]: pgmap v10556: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 13 op/s 2023-09-10T04:44:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19491.001895 secs 2023-09-10T04:44:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:44:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:44:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:44:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:44:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:44:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:44:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:21 smithi170 ceph-mon[102832]: pgmap v10557: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T04:44:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:21 smithi082 ceph-mon[98694]: pgmap v10557: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T04:44:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:21 smithi098 ceph-mon[103124]: pgmap v10557: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T04:44:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:23 smithi170 ceph-mon[102832]: pgmap v10558: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T04:44:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19494.225096 secs 2023-09-10T04:44:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19495.347296 secs 2023-09-10T04:44:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:23 smithi082 ceph-mon[98694]: pgmap v10558: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T04:44:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19494.225096 secs 2023-09-10T04:44:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19495.347296 secs 2023-09-10T04:44:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:23 smithi098 ceph-mon[103124]: pgmap v10558: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T04:44:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19494.225096 secs 2023-09-10T04:44:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19495.347296 secs 2023-09-10T04:44:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19496.001959 secs 2023-09-10T04:44:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19496.001959 secs 2023-09-10T04:44:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19496.001959 secs 2023-09-10T04:44:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:25 smithi170 ceph-mon[102832]: pgmap v10559: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 13 op/s 2023-09-10T04:44:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:25 smithi082 ceph-mon[98694]: pgmap v10559: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 13 op/s 2023-09-10T04:44:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:25 smithi098 ceph-mon[103124]: pgmap v10559: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 13 op/s 2023-09-10T04:44:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:27 smithi170 ceph-mon[102832]: pgmap v10560: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 159 KiB/s wr, 16 op/s 2023-09-10T04:44:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:27 smithi082 ceph-mon[98694]: pgmap v10560: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 159 KiB/s wr, 16 op/s 2023-09-10T04:44:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:27 smithi098 ceph-mon[103124]: pgmap v10560: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 159 KiB/s wr, 16 op/s 2023-09-10T04:44:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19500.347354 secs 2023-09-10T04:44:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19500.347354 secs 2023-09-10T04:44:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19500.347354 secs 2023-09-10T04:44:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:29 smithi170 ceph-mon[102832]: pgmap v10561: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T04:44:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19501.002025 secs 2023-09-10T04:44:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:29 smithi082 ceph-mon[98694]: pgmap v10561: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T04:44:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19501.002025 secs 2023-09-10T04:44:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:29 smithi098 ceph-mon[103124]: pgmap v10561: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T04:44:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19501.002025 secs 2023-09-10T04:44:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:31 smithi170 ceph-mon[102832]: pgmap v10562: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T04:44:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:44:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:31 smithi082 ceph-mon[98694]: pgmap v10562: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T04:44:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:44:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:31 smithi098 ceph-mon[103124]: pgmap v10562: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 10 op/s 2023-09-10T04:44:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:44:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:44:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:44:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:44:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:44:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:44:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:44:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:44:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:44:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:44:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:33 smithi170 ceph-mon[102832]: pgmap v10563: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 195 KiB/s wr, 15 op/s 2023-09-10T04:44:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19505.347417 secs 2023-09-10T04:44:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:33 smithi082 ceph-mon[98694]: pgmap v10563: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 195 KiB/s wr, 15 op/s 2023-09-10T04:44:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19505.347417 secs 2023-09-10T04:44:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:33 smithi098 ceph-mon[103124]: pgmap v10563: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 195 KiB/s wr, 15 op/s 2023-09-10T04:44:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19505.347417 secs 2023-09-10T04:44:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19506.002088 secs 2023-09-10T04:44:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19506.002088 secs 2023-09-10T04:44:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19506.002088 secs 2023-09-10T04:44:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:35 smithi170 ceph-mon[102832]: pgmap v10564: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 10 op/s 2023-09-10T04:44:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:35 smithi082 ceph-mon[98694]: pgmap v10564: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 10 op/s 2023-09-10T04:44:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:35 smithi098 ceph-mon[103124]: pgmap v10564: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 10 op/s 2023-09-10T04:44:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:37 smithi170 ceph-mon[102832]: pgmap v10565: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 324 KiB/s wr, 16 op/s 2023-09-10T04:44:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:37 smithi082 ceph-mon[98694]: pgmap v10565: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 324 KiB/s wr, 16 op/s 2023-09-10T04:44:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:37 smithi098 ceph-mon[103124]: pgmap v10565: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 324 KiB/s wr, 16 op/s 2023-09-10T04:44:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:44:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:44:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:44:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19510.347476 secs 2023-09-10T04:44:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:44:38.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:44:38.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:44:38.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:44:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:44:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:44:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:44:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19510.347476 secs 2023-09-10T04:44:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:44:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:44:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:44:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:44:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:44:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:44:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:44:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19510.347476 secs 2023-09-10T04:44:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:44:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:44:39.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:44:39.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:44:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:39 smithi170 ceph-mon[102832]: pgmap v10566: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 266 KiB/s wr, 11 op/s 2023-09-10T04:44:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19511.002155 secs 2023-09-10T04:44:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:44:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:44:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:44:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:39 smithi082 ceph-mon[98694]: pgmap v10566: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 266 KiB/s wr, 11 op/s 2023-09-10T04:44:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19511.002155 secs 2023-09-10T04:44:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:44:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:44:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:44:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:39 smithi098 ceph-mon[103124]: pgmap v10566: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 266 KiB/s wr, 11 op/s 2023-09-10T04:44:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19511.002155 secs 2023-09-10T04:44:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:44:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:44:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:44:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:44:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:44:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:44:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:44:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:44:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:44:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:41 smithi170 ceph-mon[102832]: pgmap v10567: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 266 KiB/s wr, 11 op/s 2023-09-10T04:44:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:41 smithi082 ceph-mon[98694]: pgmap v10567: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 266 KiB/s wr, 11 op/s 2023-09-10T04:44:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:41 smithi098 ceph-mon[103124]: pgmap v10567: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 266 KiB/s wr, 11 op/s 2023-09-10T04:44:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:43 smithi170 ceph-mon[102832]: pgmap v10568: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 383 KiB/s wr, 18 op/s 2023-09-10T04:44:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19515.347493 secs 2023-09-10T04:44:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:43 smithi082 ceph-mon[98694]: pgmap v10568: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 383 KiB/s wr, 18 op/s 2023-09-10T04:44:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19515.347493 secs 2023-09-10T04:44:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:43 smithi098 ceph-mon[103124]: pgmap v10568: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 383 KiB/s wr, 18 op/s 2023-09-10T04:44:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19515.347493 secs 2023-09-10T04:44:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19516.002221 secs 2023-09-10T04:44:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19516.002221 secs 2023-09-10T04:44:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19516.002221 secs 2023-09-10T04:44:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:45 smithi170 ceph-mon[102832]: pgmap v10569: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 246 KiB/s wr, 12 op/s 2023-09-10T04:44:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:45 smithi082 ceph-mon[98694]: pgmap v10569: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 246 KiB/s wr, 12 op/s 2023-09-10T04:44:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:45 smithi098 ceph-mon[103124]: pgmap v10569: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 246 KiB/s wr, 12 op/s 2023-09-10T04:44:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:47 smithi170 ceph-mon[102832]: pgmap v10570: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 369 KiB/s wr, 18 op/s 2023-09-10T04:44:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:47 smithi082 ceph-mon[98694]: pgmap v10570: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 369 KiB/s wr, 18 op/s 2023-09-10T04:44:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:47 smithi098 ceph-mon[103124]: pgmap v10570: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 369 KiB/s wr, 18 op/s 2023-09-10T04:44:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19520.347609 secs 2023-09-10T04:44:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19520.347609 secs 2023-09-10T04:44:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19520.347609 secs 2023-09-10T04:44:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:49 smithi170 ceph-mon[102832]: pgmap v10571: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 240 KiB/s wr, 12 op/s 2023-09-10T04:44:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19521.002286 secs 2023-09-10T04:44:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:49 smithi082 ceph-mon[98694]: pgmap v10571: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 240 KiB/s wr, 12 op/s 2023-09-10T04:44:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19521.002286 secs 2023-09-10T04:44:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:49 smithi098 ceph-mon[103124]: pgmap v10571: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 240 KiB/s wr, 12 op/s 2023-09-10T04:44:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19521.002286 secs 2023-09-10T04:44:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:44:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:44:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:44:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:44:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:44:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:44:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:51 smithi170 ceph-mon[102832]: pgmap v10572: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 240 KiB/s wr, 12 op/s 2023-09-10T04:44:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:51 smithi082 ceph-mon[98694]: pgmap v10572: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 240 KiB/s wr, 12 op/s 2023-09-10T04:44:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:51 smithi098 ceph-mon[103124]: pgmap v10572: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 240 KiB/s wr, 12 op/s 2023-09-10T04:44:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:44:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:44:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:44:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:44:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:44:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:44:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:53 smithi170 ceph-mon[102832]: pgmap v10573: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 367 KiB/s wr, 18 op/s 2023-09-10T04:44:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19524.225664 secs 2023-09-10T04:44:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19525.347667 secs 2023-09-10T04:44:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:53 smithi082 ceph-mon[98694]: pgmap v10573: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 367 KiB/s wr, 18 op/s 2023-09-10T04:44:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19524.225664 secs 2023-09-10T04:44:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19525.347667 secs 2023-09-10T04:44:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:53 smithi098 ceph-mon[103124]: pgmap v10573: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 367 KiB/s wr, 18 op/s 2023-09-10T04:44:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19524.225664 secs 2023-09-10T04:44:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19525.347667 secs 2023-09-10T04:44:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19526.002350 secs 2023-09-10T04:44:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19526.002350 secs 2023-09-10T04:44:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19526.002350 secs 2023-09-10T04:44:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:55 smithi170 ceph-mon[102832]: pgmap v10574: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 250 KiB/s wr, 12 op/s 2023-09-10T04:44:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:55 smithi082 ceph-mon[98694]: pgmap v10574: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 250 KiB/s wr, 12 op/s 2023-09-10T04:44:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:55 smithi098 ceph-mon[103124]: pgmap v10574: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 250 KiB/s wr, 12 op/s 2023-09-10T04:44:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:44:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:44:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:44:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:44:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:44:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:44:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:44:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:44:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:44:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:57 smithi170 ceph-mon[102832]: pgmap v10575: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 316 KiB/s wr, 18 op/s 2023-09-10T04:44:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:57 smithi082 ceph-mon[98694]: pgmap v10575: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 316 KiB/s wr, 18 op/s 2023-09-10T04:44:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:57 smithi098 ceph-mon[103124]: pgmap v10575: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 316 KiB/s wr, 18 op/s 2023-09-10T04:44:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19529.225786 secs 2023-09-10T04:44:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19530.347726 secs 2023-09-10T04:44:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19529.225786 secs 2023-09-10T04:44:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19530.347726 secs 2023-09-10T04:44:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19529.225786 secs 2023-09-10T04:44:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19530.347726 secs 2023-09-10T04:44:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:59 smithi170 ceph-mon[102832]: pgmap v10576: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 12 op/s 2023-09-10T04:44:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:44:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19531.002415 secs 2023-09-10T04:44:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:59 smithi082 ceph-mon[98694]: pgmap v10576: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 12 op/s 2023-09-10T04:44:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:44:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19531.002415 secs 2023-09-10T04:45:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:59 smithi098 ceph-mon[103124]: pgmap v10576: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 12 op/s 2023-09-10T04:45:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:44:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19531.002415 secs 2023-09-10T04:45:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:01 smithi170 ceph-mon[102832]: pgmap v10577: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 12 op/s 2023-09-10T04:45:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:01 smithi082 ceph-mon[98694]: pgmap v10577: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 12 op/s 2023-09-10T04:45:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:01 smithi098 ceph-mon[103124]: pgmap v10577: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 12 op/s 2023-09-10T04:45:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:03 smithi170 ceph-mon[102832]: pgmap v10578: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 339 KiB/s wr, 18 op/s 2023-09-10T04:45:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19534.225865 secs 2023-09-10T04:45:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19535.347784 secs 2023-09-10T04:45:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:03 smithi082 ceph-mon[98694]: pgmap v10578: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 339 KiB/s wr, 18 op/s 2023-09-10T04:45:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19534.225865 secs 2023-09-10T04:45:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19535.347784 secs 2023-09-10T04:45:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:03 smithi098 ceph-mon[103124]: pgmap v10578: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 339 KiB/s wr, 18 op/s 2023-09-10T04:45:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19534.225865 secs 2023-09-10T04:45:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19535.347784 secs 2023-09-10T04:45:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19536.002482 secs 2023-09-10T04:45:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19536.002482 secs 2023-09-10T04:45:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19536.002482 secs 2023-09-10T04:45:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:05 smithi170 ceph-mon[102832]: pgmap v10579: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 212 KiB/s wr, 11 op/s 2023-09-10T04:45:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:05 smithi082 ceph-mon[98694]: pgmap v10579: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 212 KiB/s wr, 11 op/s 2023-09-10T04:45:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:05 smithi098 ceph-mon[103124]: pgmap v10579: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 212 KiB/s wr, 11 op/s 2023-09-10T04:45:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:07 smithi170 ceph-mon[102832]: pgmap v10580: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 320 KiB/s wr, 31 op/s 2023-09-10T04:45:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:07 smithi082 ceph-mon[98694]: pgmap v10580: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 320 KiB/s wr, 31 op/s 2023-09-10T04:45:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:07 smithi098 ceph-mon[103124]: pgmap v10580: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 320 KiB/s wr, 31 op/s 2023-09-10T04:45:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19539.225974 secs 2023-09-10T04:45:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:08 smithi170 ceph-mon[102832]: osdmap e729: 12 total, 12 up, 12 in 2023-09-10T04:45:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19540.347844 secs 2023-09-10T04:45:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19539.225974 secs 2023-09-10T04:45:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:08 smithi082 ceph-mon[98694]: osdmap e729: 12 total, 12 up, 12 in 2023-09-10T04:45:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19540.347844 secs 2023-09-10T04:45:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19539.225974 secs 2023-09-10T04:45:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:08 smithi098 ceph-mon[103124]: osdmap e729: 12 total, 12 up, 12 in 2023-09-10T04:45:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19540.347844 secs 2023-09-10T04:45:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:09 smithi170 ceph-mon[102832]: pgmap v10582: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 305 KiB/s wr, 30 op/s 2023-09-10T04:45:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19541.002551 secs 2023-09-10T04:45:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:09 smithi082 ceph-mon[98694]: pgmap v10582: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 305 KiB/s wr, 30 op/s 2023-09-10T04:45:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19541.002551 secs 2023-09-10T04:45:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:09 smithi098 ceph-mon[103124]: pgmap v10582: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 305 KiB/s wr, 30 op/s 2023-09-10T04:45:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19541.002551 secs 2023-09-10T04:45:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:11 smithi170 ceph-mon[102832]: pgmap v10583: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 305 KiB/s wr, 30 op/s 2023-09-10T04:45:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:11 smithi082 ceph-mon[98694]: pgmap v10583: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 305 KiB/s wr, 30 op/s 2023-09-10T04:45:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:11 smithi098 ceph-mon[103124]: pgmap v10583: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 305 KiB/s wr, 30 op/s 2023-09-10T04:45:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:13 smithi170 ceph-mon[102832]: pgmap v10584: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 130 KiB/s wr, 29 op/s 2023-09-10T04:45:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19544.226054 secs 2023-09-10T04:45:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19545.347902 secs 2023-09-10T04:45:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:13 smithi082 ceph-mon[98694]: pgmap v10584: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 130 KiB/s wr, 29 op/s 2023-09-10T04:45:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19544.226054 secs 2023-09-10T04:45:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19545.347902 secs 2023-09-10T04:45:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:13 smithi098 ceph-mon[103124]: pgmap v10584: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 130 KiB/s wr, 29 op/s 2023-09-10T04:45:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19544.226054 secs 2023-09-10T04:45:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19545.347902 secs 2023-09-10T04:45:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19546.002615 secs 2023-09-10T04:45:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:14 smithi170 ceph-mon[102832]: osdmap e730: 12 total, 12 up, 12 in 2023-09-10T04:45:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19546.002615 secs 2023-09-10T04:45:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:14 smithi082 ceph-mon[98694]: osdmap e730: 12 total, 12 up, 12 in 2023-09-10T04:45:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19546.002615 secs 2023-09-10T04:45:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:14 smithi098 ceph-mon[103124]: osdmap e730: 12 total, 12 up, 12 in 2023-09-10T04:45:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:15 smithi170 ceph-mon[102832]: pgmap v10586: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:45:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:15 smithi082 ceph-mon[98694]: pgmap v10586: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:45:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:15 smithi098 ceph-mon[103124]: pgmap v10586: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:45:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:17 smithi170 ceph-mon[102832]: pgmap v10587: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 14 op/s 2023-09-10T04:45:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:17 smithi082 ceph-mon[98694]: pgmap v10587: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 14 op/s 2023-09-10T04:45:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:17 smithi098 ceph-mon[103124]: pgmap v10587: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 14 op/s 2023-09-10T04:45:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19549.226149 secs 2023-09-10T04:45:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19550.347963 secs 2023-09-10T04:45:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19549.226149 secs 2023-09-10T04:45:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19550.347963 secs 2023-09-10T04:45:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19549.226149 secs 2023-09-10T04:45:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19550.347963 secs 2023-09-10T04:45:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:19 smithi170 ceph-mon[102832]: pgmap v10588: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T04:45:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19551.002681 secs 2023-09-10T04:45:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:19 smithi082 ceph-mon[98694]: pgmap v10588: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T04:45:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19551.002681 secs 2023-09-10T04:45:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:19 smithi098 ceph-mon[103124]: pgmap v10588: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T04:45:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19551.002681 secs 2023-09-10T04:45:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:21 smithi170 ceph-mon[102832]: pgmap v10589: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T04:45:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:21 smithi082 ceph-mon[98694]: pgmap v10589: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T04:45:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:21 smithi098 ceph-mon[103124]: pgmap v10589: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T04:45:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:23 smithi170 ceph-mon[102832]: pgmap v10590: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T04:45:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19554.226269 secs 2023-09-10T04:45:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19555.348021 secs 2023-09-10T04:45:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:23 smithi082 ceph-mon[98694]: pgmap v10590: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T04:45:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19554.226269 secs 2023-09-10T04:45:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19555.348021 secs 2023-09-10T04:45:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:23 smithi098 ceph-mon[103124]: pgmap v10590: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T04:45:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19554.226269 secs 2023-09-10T04:45:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19555.348021 secs 2023-09-10T04:45:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:45:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:45:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19556.002744 secs 2023-09-10T04:45:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:45:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:45:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19556.002744 secs 2023-09-10T04:45:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:45:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:45:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19556.002744 secs 2023-09-10T04:45:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:25 smithi082 ceph-mon[98694]: pgmap v10591: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T04:45:25.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:25 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:45:25.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:25 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:45:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:25 smithi098 ceph-mon[103124]: pgmap v10591: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T04:45:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:25 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:45:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:25 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:45:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:25 smithi170 ceph-mon[102832]: pgmap v10591: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T04:45:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:25 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:45:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:25 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:45:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:27 smithi082 ceph-mon[98694]: pgmap v10592: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 159 KiB/s wr, 15 op/s 2023-09-10T04:45:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:27 smithi098 ceph-mon[103124]: pgmap v10592: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 159 KiB/s wr, 15 op/s 2023-09-10T04:45:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:27 smithi170 ceph-mon[102832]: pgmap v10592: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 159 KiB/s wr, 15 op/s 2023-09-10T04:45:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19560.348080 secs 2023-09-10T04:45:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19560.348080 secs 2023-09-10T04:45:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19560.348080 secs 2023-09-10T04:45:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:29 smithi082 ceph-mon[98694]: pgmap v10593: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T04:45:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19561.002812 secs 2023-09-10T04:45:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:29 smithi098 ceph-mon[103124]: pgmap v10593: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T04:45:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19561.002812 secs 2023-09-10T04:45:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:29 smithi170 ceph-mon[102832]: pgmap v10593: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T04:45:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19561.002812 secs 2023-09-10T04:45:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:31 smithi082 ceph-mon[98694]: pgmap v10594: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T04:45:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:45:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:31 smithi098 ceph-mon[103124]: pgmap v10594: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T04:45:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:45:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:31 smithi170 ceph-mon[102832]: pgmap v10594: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 59 KiB/s wr, 9 op/s 2023-09-10T04:45:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:45:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:45:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:45:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:45:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:45:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:45:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:45:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:45:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:45:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:45:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:33 smithi082 ceph-mon[98694]: pgmap v10595: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 197 KiB/s wr, 15 op/s 2023-09-10T04:45:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19565.348139 secs 2023-09-10T04:45:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:33 smithi098 ceph-mon[103124]: pgmap v10595: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 197 KiB/s wr, 15 op/s 2023-09-10T04:45:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19565.348139 secs 2023-09-10T04:45:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:33 smithi170 ceph-mon[102832]: pgmap v10595: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 197 KiB/s wr, 15 op/s 2023-09-10T04:45:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19565.348139 secs 2023-09-10T04:45:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19566.002875 secs 2023-09-10T04:45:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19566.002875 secs 2023-09-10T04:45:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19566.002875 secs 2023-09-10T04:45:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:35 smithi098 ceph-mon[103124]: pgmap v10596: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 197 KiB/s wr, 10 op/s 2023-09-10T04:45:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:35 smithi170 ceph-mon[102832]: pgmap v10596: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 197 KiB/s wr, 10 op/s 2023-09-10T04:45:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:35 smithi082 ceph-mon[98694]: pgmap v10596: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 197 KiB/s wr, 10 op/s 2023-09-10T04:45:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:37 smithi098 ceph-mon[103124]: pgmap v10597: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 327 KiB/s wr, 16 op/s 2023-09-10T04:45:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:45:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:45:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:37 smithi170 ceph-mon[102832]: pgmap v10597: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 327 KiB/s wr, 16 op/s 2023-09-10T04:45:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:45:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:45:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:37 smithi082 ceph-mon[98694]: pgmap v10597: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 327 KiB/s wr, 16 op/s 2023-09-10T04:45:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:45:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:45:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19570.348196 secs 2023-09-10T04:45:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19570.348196 secs 2023-09-10T04:45:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19570.348196 secs 2023-09-10T04:45:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:39 smithi098 ceph-mon[103124]: pgmap v10598: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 268 KiB/s wr, 11 op/s 2023-09-10T04:45:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19571.002939 secs 2023-09-10T04:45:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:45:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:39 smithi170 ceph-mon[102832]: pgmap v10598: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 268 KiB/s wr, 11 op/s 2023-09-10T04:45:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19571.002939 secs 2023-09-10T04:45:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:45:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:39 smithi082 ceph-mon[98694]: pgmap v10598: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 268 KiB/s wr, 11 op/s 2023-09-10T04:45:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19571.002939 secs 2023-09-10T04:45:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:45:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:45:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:45:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:45:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:45:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:45:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:45:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:45:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:45:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:45:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:45:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:45:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:45:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:45:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:45:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:45:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:41 smithi098 ceph-mon[103124]: pgmap v10599: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 268 KiB/s wr, 11 op/s 2023-09-10T04:45:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:41 smithi170 ceph-mon[102832]: pgmap v10599: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 268 KiB/s wr, 11 op/s 2023-09-10T04:45:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:41 smithi082 ceph-mon[98694]: pgmap v10599: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 268 KiB/s wr, 11 op/s 2023-09-10T04:45:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:43 smithi098 ceph-mon[103124]: pgmap v10600: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 384 KiB/s wr, 18 op/s 2023-09-10T04:45:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19575.348258 secs 2023-09-10T04:45:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:43 smithi170 ceph-mon[102832]: pgmap v10600: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 384 KiB/s wr, 18 op/s 2023-09-10T04:45:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19575.348258 secs 2023-09-10T04:45:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:43 smithi082 ceph-mon[98694]: pgmap v10600: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 384 KiB/s wr, 18 op/s 2023-09-10T04:45:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19575.348258 secs 2023-09-10T04:45:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19576.003001 secs 2023-09-10T04:45:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19576.003001 secs 2023-09-10T04:45:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19576.003001 secs 2023-09-10T04:45:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:45 smithi098 ceph-mon[103124]: pgmap v10601: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 246 KiB/s wr, 12 op/s 2023-09-10T04:45:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:45 smithi170 ceph-mon[102832]: pgmap v10601: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 246 KiB/s wr, 12 op/s 2023-09-10T04:45:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:45 smithi082 ceph-mon[98694]: pgmap v10601: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 246 KiB/s wr, 12 op/s 2023-09-10T04:45:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:47 smithi098 ceph-mon[103124]: pgmap v10602: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 370 KiB/s wr, 18 op/s 2023-09-10T04:45:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:47 smithi170 ceph-mon[102832]: pgmap v10602: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 370 KiB/s wr, 18 op/s 2023-09-10T04:45:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:47 smithi082 ceph-mon[98694]: pgmap v10602: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 370 KiB/s wr, 18 op/s 2023-09-10T04:45:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19580.348314 secs 2023-09-10T04:45:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19580.348314 secs 2023-09-10T04:45:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19580.348314 secs 2023-09-10T04:45:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:49 smithi098 ceph-mon[103124]: pgmap v10603: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 240 KiB/s wr, 12 op/s 2023-09-10T04:45:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19581.003068 secs 2023-09-10T04:45:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:49 smithi170 ceph-mon[102832]: pgmap v10603: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 240 KiB/s wr, 12 op/s 2023-09-10T04:45:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19581.003068 secs 2023-09-10T04:45:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:49 smithi082 ceph-mon[98694]: pgmap v10603: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 240 KiB/s wr, 12 op/s 2023-09-10T04:45:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19581.003068 secs 2023-09-10T04:45:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:51 smithi098 ceph-mon[103124]: pgmap v10604: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 240 KiB/s wr, 12 op/s 2023-09-10T04:45:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:51 smithi170 ceph-mon[102832]: pgmap v10604: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 240 KiB/s wr, 12 op/s 2023-09-10T04:45:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:51 smithi082 ceph-mon[98694]: pgmap v10604: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 240 KiB/s wr, 12 op/s 2023-09-10T04:45:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:53 smithi098 ceph-mon[103124]: pgmap v10605: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 377 KiB/s wr, 18 op/s 2023-09-10T04:45:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19584.226744 secs 2023-09-10T04:45:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19585.348374 secs 2023-09-10T04:45:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:53 smithi170 ceph-mon[102832]: pgmap v10605: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 377 KiB/s wr, 18 op/s 2023-09-10T04:45:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19584.226744 secs 2023-09-10T04:45:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19585.348374 secs 2023-09-10T04:45:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:53 smithi082 ceph-mon[98694]: pgmap v10605: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 377 KiB/s wr, 18 op/s 2023-09-10T04:45:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19584.226744 secs 2023-09-10T04:45:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19585.348374 secs 2023-09-10T04:45:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19586.003135 secs 2023-09-10T04:45:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19586.003135 secs 2023-09-10T04:45:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19586.003135 secs 2023-09-10T04:45:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:55 smithi098 ceph-mon[103124]: pgmap v10606: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 261 KiB/s wr, 11 op/s 2023-09-10T04:45:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:55 smithi170 ceph-mon[102832]: pgmap v10606: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 261 KiB/s wr, 11 op/s 2023-09-10T04:45:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:55 smithi082 ceph-mon[98694]: pgmap v10606: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 261 KiB/s wr, 11 op/s 2023-09-10T04:45:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:45:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:45:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:45:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:45:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:45:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:45:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:45:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:45:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:45:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:57 smithi170 ceph-mon[102832]: pgmap v10607: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 327 KiB/s wr, 17 op/s 2023-09-10T04:45:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:45:58.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:45:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:57 smithi082 ceph-mon[98694]: pgmap v10607: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 327 KiB/s wr, 17 op/s 2023-09-10T04:45:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:45:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:45:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:57 smithi098 ceph-mon[103124]: pgmap v10607: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 327 KiB/s wr, 17 op/s 2023-09-10T04:45:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:45:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:45:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19589.226818 secs 2023-09-10T04:45:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19590.348432 secs 2023-09-10T04:45:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19589.226818 secs 2023-09-10T04:45:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19590.348432 secs 2023-09-10T04:45:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19589.226818 secs 2023-09-10T04:45:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19590.348432 secs 2023-09-10T04:46:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:59 smithi170 ceph-mon[102832]: pgmap v10608: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 204 KiB/s wr, 12 op/s 2023-09-10T04:46:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:45:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19591.003197 secs 2023-09-10T04:46:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:59 smithi082 ceph-mon[98694]: pgmap v10608: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 204 KiB/s wr, 12 op/s 2023-09-10T04:46:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:45:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19591.003197 secs 2023-09-10T04:46:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:59 smithi098 ceph-mon[103124]: pgmap v10608: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 204 KiB/s wr, 12 op/s 2023-09-10T04:46:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:45:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19591.003197 secs 2023-09-10T04:46:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:46:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:46:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:46:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:46:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:46:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:46:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:01 smithi170 ceph-mon[102832]: pgmap v10609: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 204 KiB/s wr, 12 op/s 2023-09-10T04:46:02.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:01 smithi082 ceph-mon[98694]: pgmap v10609: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 204 KiB/s wr, 12 op/s 2023-09-10T04:46:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:01 smithi098 ceph-mon[103124]: pgmap v10609: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 204 KiB/s wr, 12 op/s 2023-09-10T04:46:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:03 smithi170 ceph-mon[102832]: pgmap v10610: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 307 KiB/s wr, 18 op/s 2023-09-10T04:46:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19594.226878 secs 2023-09-10T04:46:04.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19595.348515 secs 2023-09-10T04:46:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:03 smithi082 ceph-mon[98694]: pgmap v10610: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 307 KiB/s wr, 18 op/s 2023-09-10T04:46:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19594.226878 secs 2023-09-10T04:46:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19595.348515 secs 2023-09-10T04:46:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:03 smithi098 ceph-mon[103124]: pgmap v10610: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 307 KiB/s wr, 18 op/s 2023-09-10T04:46:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19594.226878 secs 2023-09-10T04:46:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19595.348515 secs 2023-09-10T04:46:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19596.003261 secs 2023-09-10T04:46:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19596.003261 secs 2023-09-10T04:46:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19596.003261 secs 2023-09-10T04:46:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:05 smithi170 ceph-mon[102832]: pgmap v10611: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 170 KiB/s wr, 11 op/s 2023-09-10T04:46:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:05 smithi082 ceph-mon[98694]: pgmap v10611: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 170 KiB/s wr, 11 op/s 2023-09-10T04:46:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:05 smithi098 ceph-mon[103124]: pgmap v10611: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 170 KiB/s wr, 11 op/s 2023-09-10T04:46:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:07 smithi170 ceph-mon[102832]: pgmap v10612: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 263 KiB/s wr, 31 op/s 2023-09-10T04:46:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:07 smithi082 ceph-mon[98694]: pgmap v10612: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 263 KiB/s wr, 31 op/s 2023-09-10T04:46:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:07 smithi098 ceph-mon[103124]: pgmap v10612: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 263 KiB/s wr, 31 op/s 2023-09-10T04:46:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19599.226969 secs 2023-09-10T04:46:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:08 smithi170 ceph-mon[102832]: osdmap e731: 12 total, 12 up, 12 in 2023-09-10T04:46:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:08 smithi170 ceph-mon[102832]: pgmap v10614: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 236 KiB/s wr, 30 op/s 2023-09-10T04:46:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19600.348573 secs 2023-09-10T04:46:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19599.226969 secs 2023-09-10T04:46:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:08 smithi082 ceph-mon[98694]: osdmap e731: 12 total, 12 up, 12 in 2023-09-10T04:46:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:08 smithi082 ceph-mon[98694]: pgmap v10614: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 236 KiB/s wr, 30 op/s 2023-09-10T04:46:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19600.348573 secs 2023-09-10T04:46:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19599.226969 secs 2023-09-10T04:46:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:08 smithi098 ceph-mon[103124]: osdmap e731: 12 total, 12 up, 12 in 2023-09-10T04:46:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:08 smithi098 ceph-mon[103124]: pgmap v10614: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 236 KiB/s wr, 30 op/s 2023-09-10T04:46:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19600.348573 secs 2023-09-10T04:46:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19601.003325 secs 2023-09-10T04:46:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19601.003325 secs 2023-09-10T04:46:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19601.003325 secs 2023-09-10T04:46:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:10 smithi170 ceph-mon[102832]: pgmap v10615: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 236 KiB/s wr, 30 op/s 2023-09-10T04:46:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:10 smithi082 ceph-mon[98694]: pgmap v10615: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 236 KiB/s wr, 30 op/s 2023-09-10T04:46:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:10 smithi098 ceph-mon[103124]: pgmap v10615: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 236 KiB/s wr, 30 op/s 2023-09-10T04:46:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:13 smithi170 ceph-mon[102832]: pgmap v10616: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 113 KiB/s wr, 30 op/s 2023-09-10T04:46:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:13 smithi082 ceph-mon[98694]: pgmap v10616: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 113 KiB/s wr, 30 op/s 2023-09-10T04:46:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:13 smithi098 ceph-mon[103124]: pgmap v10616: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 113 KiB/s wr, 30 op/s 2023-09-10T04:46:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19604.227029 secs 2023-09-10T04:46:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19605.348630 secs 2023-09-10T04:46:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19606.003393 secs 2023-09-10T04:46:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:14 smithi170 ceph-mon[102832]: osdmap e732: 12 total, 12 up, 12 in 2023-09-10T04:46:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19604.227029 secs 2023-09-10T04:46:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19605.348630 secs 2023-09-10T04:46:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19606.003393 secs 2023-09-10T04:46:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:14 smithi082 ceph-mon[98694]: osdmap e732: 12 total, 12 up, 12 in 2023-09-10T04:46:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19604.227029 secs 2023-09-10T04:46:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19605.348630 secs 2023-09-10T04:46:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19606.003393 secs 2023-09-10T04:46:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:14 smithi098 ceph-mon[103124]: osdmap e732: 12 total, 12 up, 12 in 2023-09-10T04:46:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:15 smithi170 ceph-mon[102832]: pgmap v10618: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:46:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:15 smithi082 ceph-mon[98694]: pgmap v10618: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:46:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:15 smithi098 ceph-mon[103124]: pgmap v10618: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:46:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:17 smithi170 ceph-mon[102832]: pgmap v10619: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 15 op/s 2023-09-10T04:46:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:17 smithi082 ceph-mon[98694]: pgmap v10619: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 15 op/s 2023-09-10T04:46:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:17 smithi098 ceph-mon[103124]: pgmap v10619: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 15 op/s 2023-09-10T04:46:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19609.227088 secs 2023-09-10T04:46:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19610.348687 secs 2023-09-10T04:46:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:19 smithi170 ceph-mon[102832]: pgmap v10620: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T04:46:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19611.003487 secs 2023-09-10T04:46:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19609.227088 secs 2023-09-10T04:46:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19610.348687 secs 2023-09-10T04:46:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:19 smithi082 ceph-mon[98694]: pgmap v10620: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T04:46:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19611.003487 secs 2023-09-10T04:46:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19609.227088 secs 2023-09-10T04:46:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19610.348687 secs 2023-09-10T04:46:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:19 smithi098 ceph-mon[103124]: pgmap v10620: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T04:46:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19611.003487 secs 2023-09-10T04:46:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:21 smithi170 ceph-mon[102832]: pgmap v10621: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T04:46:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:21 smithi082 ceph-mon[98694]: pgmap v10621: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T04:46:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:21 smithi098 ceph-mon[103124]: pgmap v10621: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T04:46:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:23 smithi170 ceph-mon[102832]: pgmap v10622: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T04:46:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19615.348748 secs 2023-09-10T04:46:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:23 smithi082 ceph-mon[98694]: pgmap v10622: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T04:46:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19615.348748 secs 2023-09-10T04:46:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:23 smithi098 ceph-mon[103124]: pgmap v10622: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T04:46:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19615.348748 secs 2023-09-10T04:46:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19614.227147 secs 2023-09-10T04:46:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19616.003553 secs 2023-09-10T04:46:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19614.227147 secs 2023-09-10T04:46:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19616.003553 secs 2023-09-10T04:46:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19614.227147 secs 2023-09-10T04:46:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19616.003553 secs 2023-09-10T04:46:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:25 smithi170 ceph-mon[102832]: pgmap v10623: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T04:46:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:25 smithi082 ceph-mon[98694]: pgmap v10623: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T04:46:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:25 smithi098 ceph-mon[103124]: pgmap v10623: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T04:46:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:27 smithi170 ceph-mon[102832]: pgmap v10624: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 173 KiB/s wr, 14 op/s 2023-09-10T04:46:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:27 smithi082 ceph-mon[98694]: pgmap v10624: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 173 KiB/s wr, 14 op/s 2023-09-10T04:46:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:27 smithi098 ceph-mon[103124]: pgmap v10624: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 173 KiB/s wr, 14 op/s 2023-09-10T04:46:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19620.348807 secs 2023-09-10T04:46:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19620.348807 secs 2023-09-10T04:46:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19620.348807 secs 2023-09-10T04:46:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:29 smithi170 ceph-mon[102832]: pgmap v10625: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 72 KiB/s wr, 9 op/s 2023-09-10T04:46:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19621.003617 secs 2023-09-10T04:46:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:29 smithi082 ceph-mon[98694]: pgmap v10625: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 72 KiB/s wr, 9 op/s 2023-09-10T04:46:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19621.003617 secs 2023-09-10T04:46:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:29 smithi098 ceph-mon[103124]: pgmap v10625: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 72 KiB/s wr, 9 op/s 2023-09-10T04:46:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19621.003617 secs 2023-09-10T04:46:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:46:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:46:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:46:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:46:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:46:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:46:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:31 smithi082 ceph-mon[98694]: pgmap v10626: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 72 KiB/s wr, 9 op/s 2023-09-10T04:46:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:46:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:31 smithi098 ceph-mon[103124]: pgmap v10626: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 72 KiB/s wr, 9 op/s 2023-09-10T04:46:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:46:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:31 smithi170 ceph-mon[102832]: pgmap v10626: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 72 KiB/s wr, 9 op/s 2023-09-10T04:46:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:46:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:46:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:46:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:46:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:46:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:46:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:46:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:46:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:46:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:46:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:33 smithi082 ceph-mon[98694]: pgmap v10627: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 197 KiB/s wr, 15 op/s 2023-09-10T04:46:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19625.348867 secs 2023-09-10T04:46:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:33 smithi098 ceph-mon[103124]: pgmap v10627: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 197 KiB/s wr, 15 op/s 2023-09-10T04:46:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19625.348867 secs 2023-09-10T04:46:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:33 smithi170 ceph-mon[102832]: pgmap v10627: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 197 KiB/s wr, 15 op/s 2023-09-10T04:46:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19625.348867 secs 2023-09-10T04:46:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19626.003679 secs 2023-09-10T04:46:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19626.003679 secs 2023-09-10T04:46:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19626.003679 secs 2023-09-10T04:46:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:35 smithi082 ceph-mon[98694]: pgmap v10628: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 197 KiB/s wr, 10 op/s 2023-09-10T04:46:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:35 smithi098 ceph-mon[103124]: pgmap v10628: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 197 KiB/s wr, 10 op/s 2023-09-10T04:46:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:35 smithi170 ceph-mon[102832]: pgmap v10628: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 197 KiB/s wr, 10 op/s 2023-09-10T04:46:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:37 smithi082 ceph-mon[98694]: pgmap v10629: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 328 KiB/s wr, 17 op/s 2023-09-10T04:46:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:37 smithi098 ceph-mon[103124]: pgmap v10629: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 328 KiB/s wr, 17 op/s 2023-09-10T04:46:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:37 smithi170 ceph-mon[102832]: pgmap v10629: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 328 KiB/s wr, 17 op/s 2023-09-10T04:46:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19630.348953 secs 2023-09-10T04:46:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19630.348953 secs 2023-09-10T04:46:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19630.348953 secs 2023-09-10T04:46:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:39 smithi098 ceph-mon[103124]: pgmap v10630: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 256 KiB/s wr, 11 op/s 2023-09-10T04:46:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19631.003748 secs 2023-09-10T04:46:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:39 smithi170 ceph-mon[102832]: pgmap v10630: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 256 KiB/s wr, 11 op/s 2023-09-10T04:46:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19631.003748 secs 2023-09-10T04:46:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:39 smithi082 ceph-mon[98694]: pgmap v10630: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 256 KiB/s wr, 11 op/s 2023-09-10T04:46:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19631.003748 secs 2023-09-10T04:46:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:46:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:46:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:46:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:41 smithi098 ceph-mon[103124]: pgmap v10631: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 256 KiB/s wr, 11 op/s 2023-09-10T04:46:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:46:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:46:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:46:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:41 smithi170 ceph-mon[102832]: pgmap v10631: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 256 KiB/s wr, 11 op/s 2023-09-10T04:46:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:46:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:46:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:46:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:41 smithi082 ceph-mon[98694]: pgmap v10631: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 256 KiB/s wr, 11 op/s 2023-09-10T04:46:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:46:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:46:41.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:46:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:43 smithi098 ceph-mon[103124]: pgmap v10632: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 389 KiB/s wr, 18 op/s 2023-09-10T04:46:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19635.349014 secs 2023-09-10T04:46:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:43 smithi170 ceph-mon[102832]: pgmap v10632: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 389 KiB/s wr, 18 op/s 2023-09-10T04:46:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19635.349014 secs 2023-09-10T04:46:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:43 smithi082 ceph-mon[98694]: pgmap v10632: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 389 KiB/s wr, 18 op/s 2023-09-10T04:46:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19635.349014 secs 2023-09-10T04:46:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19636.003810 secs 2023-09-10T04:46:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19636.003810 secs 2023-09-10T04:46:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19636.003810 secs 2023-09-10T04:46:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:45 smithi098 ceph-mon[103124]: pgmap v10633: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 264 KiB/s wr, 13 op/s 2023-09-10T04:46:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:45 smithi170 ceph-mon[102832]: pgmap v10633: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 264 KiB/s wr, 13 op/s 2023-09-10T04:46:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:45 smithi082 ceph-mon[98694]: pgmap v10633: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 264 KiB/s wr, 13 op/s 2023-09-10T04:46:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:47 smithi098 ceph-mon[103124]: pgmap v10634: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 371 KiB/s wr, 19 op/s 2023-09-10T04:46:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:47 smithi170 ceph-mon[102832]: pgmap v10634: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 371 KiB/s wr, 19 op/s 2023-09-10T04:46:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:47 smithi082 ceph-mon[98694]: pgmap v10634: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 371 KiB/s wr, 19 op/s 2023-09-10T04:46:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19640.349108 secs 2023-09-10T04:46:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19640.349108 secs 2023-09-10T04:46:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19640.349108 secs 2023-09-10T04:46:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:49 smithi098 ceph-mon[103124]: pgmap v10635: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 241 KiB/s wr, 12 op/s 2023-09-10T04:46:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19641.003876 secs 2023-09-10T04:46:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:49 smithi170 ceph-mon[102832]: pgmap v10635: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 241 KiB/s wr, 12 op/s 2023-09-10T04:46:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19641.003876 secs 2023-09-10T04:46:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:49 smithi082 ceph-mon[98694]: pgmap v10635: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 241 KiB/s wr, 12 op/s 2023-09-10T04:46:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19641.003876 secs 2023-09-10T04:46:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:46:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:46:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:46:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:46:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:46:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:46:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:51 smithi170 ceph-mon[102832]: pgmap v10636: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 241 KiB/s wr, 12 op/s 2023-09-10T04:46:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:51 smithi082 ceph-mon[98694]: pgmap v10636: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 241 KiB/s wr, 12 op/s 2023-09-10T04:46:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:51 smithi098 ceph-mon[103124]: pgmap v10636: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 241 KiB/s wr, 12 op/s 2023-09-10T04:46:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:46:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:46:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:46:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:46:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:46:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:46:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:53 smithi170 ceph-mon[102832]: pgmap v10637: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 369 KiB/s wr, 19 op/s 2023-09-10T04:46:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19644.227522 secs 2023-09-10T04:46:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19645.349168 secs 2023-09-10T04:46:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:53 smithi082 ceph-mon[98694]: pgmap v10637: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 369 KiB/s wr, 19 op/s 2023-09-10T04:46:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19644.227522 secs 2023-09-10T04:46:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19645.349168 secs 2023-09-10T04:46:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:53 smithi098 ceph-mon[103124]: pgmap v10637: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 369 KiB/s wr, 19 op/s 2023-09-10T04:46:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19644.227522 secs 2023-09-10T04:46:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19645.349168 secs 2023-09-10T04:46:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19646.003940 secs 2023-09-10T04:46:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19646.003940 secs 2023-09-10T04:46:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19646.003940 secs 2023-09-10T04:46:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:55 smithi170 ceph-mon[102832]: pgmap v10638: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 236 KiB/s wr, 12 op/s 2023-09-10T04:46:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:55 smithi082 ceph-mon[98694]: pgmap v10638: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 236 KiB/s wr, 12 op/s 2023-09-10T04:46:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:55 smithi098 ceph-mon[103124]: pgmap v10638: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 236 KiB/s wr, 12 op/s 2023-09-10T04:46:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:46:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:46:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:46:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:46:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:46:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:46:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:46:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:46:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:46:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:57 smithi170 ceph-mon[102832]: pgmap v10639: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 302 KiB/s wr, 17 op/s 2023-09-10T04:46:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:57 smithi082 ceph-mon[98694]: pgmap v10639: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 302 KiB/s wr, 17 op/s 2023-09-10T04:46:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:57 smithi098 ceph-mon[103124]: pgmap v10639: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 302 KiB/s wr, 17 op/s 2023-09-10T04:46:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19649.227642 secs 2023-09-10T04:46:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19650.349251 secs 2023-09-10T04:46:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19649.227642 secs 2023-09-10T04:46:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19650.349251 secs 2023-09-10T04:46:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19649.227642 secs 2023-09-10T04:46:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19650.349251 secs 2023-09-10T04:46:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:59 smithi170 ceph-mon[102832]: pgmap v10640: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 195 KiB/s wr, 11 op/s 2023-09-10T04:46:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:46:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19651.004004 secs 2023-09-10T04:46:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:59 smithi082 ceph-mon[98694]: pgmap v10640: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 195 KiB/s wr, 11 op/s 2023-09-10T04:46:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:46:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19651.004004 secs 2023-09-10T04:46:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:59 smithi098 ceph-mon[103124]: pgmap v10640: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 195 KiB/s wr, 11 op/s 2023-09-10T04:46:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:46:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19651.004004 secs 2023-09-10T04:47:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:01 smithi170 ceph-mon[102832]: pgmap v10641: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 195 KiB/s wr, 11 op/s 2023-09-10T04:47:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:01 smithi082 ceph-mon[98694]: pgmap v10641: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 195 KiB/s wr, 11 op/s 2023-09-10T04:47:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:01 smithi098 ceph-mon[103124]: pgmap v10641: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 195 KiB/s wr, 11 op/s 2023-09-10T04:47:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:03 smithi170 ceph-mon[102832]: pgmap v10642: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 286 KiB/s wr, 17 op/s 2023-09-10T04:47:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19654.227700 secs 2023-09-10T04:47:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19655.349356 secs 2023-09-10T04:47:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:03 smithi082 ceph-mon[98694]: pgmap v10642: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 286 KiB/s wr, 17 op/s 2023-09-10T04:47:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19654.227700 secs 2023-09-10T04:47:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19655.349356 secs 2023-09-10T04:47:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:03 smithi098 ceph-mon[103124]: pgmap v10642: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 286 KiB/s wr, 17 op/s 2023-09-10T04:47:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19654.227700 secs 2023-09-10T04:47:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19655.349356 secs 2023-09-10T04:47:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:47:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:47:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19656.004070 secs 2023-09-10T04:47:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:47:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:47:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19656.004070 secs 2023-09-10T04:47:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:47:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:47:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19656.004070 secs 2023-09-10T04:47:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:05 smithi170 ceph-mon[102832]: pgmap v10643: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 157 KiB/s wr, 11 op/s 2023-09-10T04:47:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:05 smithi082 ceph-mon[98694]: pgmap v10643: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 157 KiB/s wr, 11 op/s 2023-09-10T04:47:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:05 smithi098 ceph-mon[103124]: pgmap v10643: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 157 KiB/s wr, 11 op/s 2023-09-10T04:47:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:47:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:47:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:47:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:47:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:47:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:47:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:07 smithi170 ceph-mon[102832]: pgmap v10644: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 266 KiB/s wr, 31 op/s 2023-09-10T04:47:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:07 smithi082 ceph-mon[98694]: pgmap v10644: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 266 KiB/s wr, 31 op/s 2023-09-10T04:47:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:07 smithi098 ceph-mon[103124]: pgmap v10644: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 266 KiB/s wr, 31 op/s 2023-09-10T04:47:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19659.227764 secs 2023-09-10T04:47:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19660.349413 secs 2023-09-10T04:47:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19659.227764 secs 2023-09-10T04:47:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19660.349413 secs 2023-09-10T04:47:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19659.227764 secs 2023-09-10T04:47:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19660.349413 secs 2023-09-10T04:47:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:09 smithi170 ceph-mon[102832]: pgmap v10645: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 199 KiB/s wr, 26 op/s 2023-09-10T04:47:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:09 smithi170 ceph-mon[102832]: osdmap e733: 12 total, 12 up, 12 in 2023-09-10T04:47:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19661.004133 secs 2023-09-10T04:47:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:09 smithi082 ceph-mon[98694]: pgmap v10645: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 199 KiB/s wr, 26 op/s 2023-09-10T04:47:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:09 smithi082 ceph-mon[98694]: osdmap e733: 12 total, 12 up, 12 in 2023-09-10T04:47:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19661.004133 secs 2023-09-10T04:47:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:09 smithi098 ceph-mon[103124]: pgmap v10645: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 199 KiB/s wr, 26 op/s 2023-09-10T04:47:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:09 smithi098 ceph-mon[103124]: osdmap e733: 12 total, 12 up, 12 in 2023-09-10T04:47:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19661.004133 secs 2023-09-10T04:47:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:11 smithi170 ceph-mon[102832]: pgmap v10647: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 239 KiB/s wr, 31 op/s 2023-09-10T04:47:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:11 smithi082 ceph-mon[98694]: pgmap v10647: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 239 KiB/s wr, 31 op/s 2023-09-10T04:47:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:11 smithi098 ceph-mon[103124]: pgmap v10647: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 239 KiB/s wr, 31 op/s 2023-09-10T04:47:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:13 smithi170 ceph-mon[102832]: pgmap v10648: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 131 KiB/s wr, 30 op/s 2023-09-10T04:47:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19664.227786 secs 2023-09-10T04:47:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19665.349472 secs 2023-09-10T04:47:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:13 smithi082 ceph-mon[98694]: pgmap v10648: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 131 KiB/s wr, 30 op/s 2023-09-10T04:47:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19664.227786 secs 2023-09-10T04:47:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19665.349472 secs 2023-09-10T04:47:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:13 smithi098 ceph-mon[103124]: pgmap v10648: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 131 KiB/s wr, 30 op/s 2023-09-10T04:47:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19664.227786 secs 2023-09-10T04:47:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19665.349472 secs 2023-09-10T04:47:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19666.004199 secs 2023-09-10T04:47:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:14 smithi170 ceph-mon[102832]: osdmap e734: 12 total, 12 up, 12 in 2023-09-10T04:47:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19666.004199 secs 2023-09-10T04:47:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:14 smithi082 ceph-mon[98694]: osdmap e734: 12 total, 12 up, 12 in 2023-09-10T04:47:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19666.004199 secs 2023-09-10T04:47:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:14 smithi098 ceph-mon[103124]: osdmap e734: 12 total, 12 up, 12 in 2023-09-10T04:47:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:15 smithi170 ceph-mon[102832]: pgmap v10650: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:47:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:15 smithi082 ceph-mon[98694]: pgmap v10650: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:47:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:15 smithi098 ceph-mon[103124]: pgmap v10650: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:47:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:17 smithi170 ceph-mon[102832]: pgmap v10651: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 152 KiB/s wr, 15 op/s 2023-09-10T04:47:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:17 smithi082 ceph-mon[98694]: pgmap v10651: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 152 KiB/s wr, 15 op/s 2023-09-10T04:47:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:17 smithi098 ceph-mon[103124]: pgmap v10651: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 152 KiB/s wr, 15 op/s 2023-09-10T04:47:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19669.227873 secs 2023-09-10T04:47:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19670.349490 secs 2023-09-10T04:47:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19669.227873 secs 2023-09-10T04:47:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19670.349490 secs 2023-09-10T04:47:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19669.227873 secs 2023-09-10T04:47:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19670.349490 secs 2023-09-10T04:47:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:19 smithi170 ceph-mon[102832]: pgmap v10652: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T04:47:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19671.004265 secs 2023-09-10T04:47:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:19 smithi082 ceph-mon[98694]: pgmap v10652: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T04:47:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19671.004265 secs 2023-09-10T04:47:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:19 smithi098 ceph-mon[103124]: pgmap v10652: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T04:47:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19671.004265 secs 2023-09-10T04:47:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:21 smithi170 ceph-mon[102832]: pgmap v10653: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T04:47:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:21 smithi082 ceph-mon[98694]: pgmap v10653: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T04:47:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:21 smithi098 ceph-mon[103124]: pgmap v10653: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T04:47:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:23 smithi170 ceph-mon[102832]: pgmap v10654: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T04:47:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19674.227932 secs 2023-09-10T04:47:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19675.349549 secs 2023-09-10T04:47:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:23 smithi082 ceph-mon[98694]: pgmap v10654: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T04:47:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19674.227932 secs 2023-09-10T04:47:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19675.349549 secs 2023-09-10T04:47:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:23 smithi098 ceph-mon[103124]: pgmap v10654: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T04:47:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19674.227932 secs 2023-09-10T04:47:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19675.349549 secs 2023-09-10T04:47:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19676.004326 secs 2023-09-10T04:47:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19676.004326 secs 2023-09-10T04:47:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19676.004326 secs 2023-09-10T04:47:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:25 smithi170 ceph-mon[102832]: pgmap v10655: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T04:47:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:25 smithi082 ceph-mon[98694]: pgmap v10655: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T04:47:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:25 smithi098 ceph-mon[103124]: pgmap v10655: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T04:47:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:27 smithi170 ceph-mon[102832]: pgmap v10656: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 161 KiB/s wr, 14 op/s 2023-09-10T04:47:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:27 smithi082 ceph-mon[98694]: pgmap v10656: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 161 KiB/s wr, 14 op/s 2023-09-10T04:47:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:27 smithi098 ceph-mon[103124]: pgmap v10656: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 161 KiB/s wr, 14 op/s 2023-09-10T04:47:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19680.349626 secs 2023-09-10T04:47:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19680.349626 secs 2023-09-10T04:47:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19680.349626 secs 2023-09-10T04:47:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:29 smithi170 ceph-mon[102832]: pgmap v10657: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 9 op/s 2023-09-10T04:47:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19681.004392 secs 2023-09-10T04:47:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:29 smithi082 ceph-mon[98694]: pgmap v10657: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 9 op/s 2023-09-10T04:47:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19681.004392 secs 2023-09-10T04:47:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:29 smithi098 ceph-mon[103124]: pgmap v10657: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 9 op/s 2023-09-10T04:47:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19681.004392 secs 2023-09-10T04:47:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:31 smithi170 ceph-mon[102832]: pgmap v10658: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 9 op/s 2023-09-10T04:47:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:47:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:31 smithi082 ceph-mon[98694]: pgmap v10658: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 9 op/s 2023-09-10T04:47:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:47:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:31 smithi098 ceph-mon[103124]: pgmap v10658: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 9 op/s 2023-09-10T04:47:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:47:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:47:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:47:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:47:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:47:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:47:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:47:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:47:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:47:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:47:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:33 smithi170 ceph-mon[102832]: pgmap v10659: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 205 KiB/s wr, 15 op/s 2023-09-10T04:47:33.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19685.349715 secs 2023-09-10T04:47:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:33 smithi082 ceph-mon[98694]: pgmap v10659: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 205 KiB/s wr, 15 op/s 2023-09-10T04:47:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19685.349715 secs 2023-09-10T04:47:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:33 smithi098 ceph-mon[103124]: pgmap v10659: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 205 KiB/s wr, 15 op/s 2023-09-10T04:47:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19685.349715 secs 2023-09-10T04:47:34.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19686.004454 secs 2023-09-10T04:47:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19686.004454 secs 2023-09-10T04:47:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19686.004454 secs 2023-09-10T04:47:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:35 smithi170 ceph-mon[102832]: pgmap v10660: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 205 KiB/s wr, 10 op/s 2023-09-10T04:47:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:35 smithi082 ceph-mon[98694]: pgmap v10660: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 205 KiB/s wr, 10 op/s 2023-09-10T04:47:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:35 smithi098 ceph-mon[103124]: pgmap v10660: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 205 KiB/s wr, 10 op/s 2023-09-10T04:47:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:37 smithi170 ceph-mon[102832]: pgmap v10661: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 330 KiB/s wr, 17 op/s 2023-09-10T04:47:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:37 smithi082 ceph-mon[98694]: pgmap v10661: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 330 KiB/s wr, 17 op/s 2023-09-10T04:47:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:37 smithi098 ceph-mon[103124]: pgmap v10661: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 330 KiB/s wr, 17 op/s 2023-09-10T04:47:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:47:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:47:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19690.349770 secs 2023-09-10T04:47:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:47:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:47:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19690.349770 secs 2023-09-10T04:47:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:47:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:47:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19690.349770 secs 2023-09-10T04:47:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:39 smithi170 ceph-mon[102832]: pgmap v10662: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T04:47:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19691.004488 secs 2023-09-10T04:47:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:39 smithi082 ceph-mon[98694]: pgmap v10662: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T04:47:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19691.004488 secs 2023-09-10T04:47:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:39 smithi098 ceph-mon[103124]: pgmap v10662: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T04:47:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19691.004488 secs 2023-09-10T04:47:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:47:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:47:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:47:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:47:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:47:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:47:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:41 smithi170 ceph-mon[102832]: pgmap v10663: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T04:47:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:47:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:47:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:47:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:47:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:41 smithi082 ceph-mon[98694]: pgmap v10663: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T04:47:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:47:41.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:47:41.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:47:41.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:47:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:41 smithi098 ceph-mon[103124]: pgmap v10663: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T04:47:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:47:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:47:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:47:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:47:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:43 smithi170 ceph-mon[102832]: pgmap v10664: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 387 KiB/s wr, 19 op/s 2023-09-10T04:47:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19695.349828 secs 2023-09-10T04:47:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:43 smithi082 ceph-mon[98694]: pgmap v10664: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 387 KiB/s wr, 19 op/s 2023-09-10T04:47:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19695.349828 secs 2023-09-10T04:47:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:43 smithi098 ceph-mon[103124]: pgmap v10664: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 387 KiB/s wr, 19 op/s 2023-09-10T04:47:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19695.349828 secs 2023-09-10T04:47:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19696.004574 secs 2023-09-10T04:47:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19696.004574 secs 2023-09-10T04:47:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19696.004574 secs 2023-09-10T04:47:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:45 smithi170 ceph-mon[102832]: pgmap v10665: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 242 KiB/s wr, 13 op/s 2023-09-10T04:47:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:45 smithi082 ceph-mon[98694]: pgmap v10665: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 242 KiB/s wr, 13 op/s 2023-09-10T04:47:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:45 smithi098 ceph-mon[103124]: pgmap v10665: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 242 KiB/s wr, 13 op/s 2023-09-10T04:47:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:47 smithi170 ceph-mon[102832]: pgmap v10666: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 374 KiB/s wr, 18 op/s 2023-09-10T04:47:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:47 smithi082 ceph-mon[98694]: pgmap v10666: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 374 KiB/s wr, 18 op/s 2023-09-10T04:47:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:47 smithi098 ceph-mon[103124]: pgmap v10666: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 374 KiB/s wr, 18 op/s 2023-09-10T04:47:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19700.349912 secs 2023-09-10T04:47:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19700.349912 secs 2023-09-10T04:47:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19700.349912 secs 2023-09-10T04:47:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:49 smithi170 ceph-mon[102832]: pgmap v10667: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 248 KiB/s wr, 11 op/s 2023-09-10T04:47:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19701.004641 secs 2023-09-10T04:47:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:49 smithi082 ceph-mon[98694]: pgmap v10667: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 248 KiB/s wr, 11 op/s 2023-09-10T04:47:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19701.004641 secs 2023-09-10T04:47:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:49 smithi098 ceph-mon[103124]: pgmap v10667: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 248 KiB/s wr, 11 op/s 2023-09-10T04:47:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19701.004641 secs 2023-09-10T04:47:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:47:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:47:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:47:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:47:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:47:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:47:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:51 smithi170 ceph-mon[102832]: pgmap v10668: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 248 KiB/s wr, 11 op/s 2023-09-10T04:47:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:51 smithi082 ceph-mon[98694]: pgmap v10668: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 248 KiB/s wr, 11 op/s 2023-09-10T04:47:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:51 smithi098 ceph-mon[103124]: pgmap v10668: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 248 KiB/s wr, 11 op/s 2023-09-10T04:47:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:47:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:47:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:47:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:47:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:47:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:47:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:53 smithi082 ceph-mon[98694]: pgmap v10669: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 370 KiB/s wr, 17 op/s 2023-09-10T04:47:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19704.228521 secs 2023-09-10T04:47:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19705.349990 secs 2023-09-10T04:47:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:53 smithi098 ceph-mon[103124]: pgmap v10669: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 370 KiB/s wr, 17 op/s 2023-09-10T04:47:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19704.228521 secs 2023-09-10T04:47:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19705.349990 secs 2023-09-10T04:47:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:53 smithi170 ceph-mon[102832]: pgmap v10669: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 370 KiB/s wr, 17 op/s 2023-09-10T04:47:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19704.228521 secs 2023-09-10T04:47:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19705.349990 secs 2023-09-10T04:47:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19706.004705 secs 2023-09-10T04:47:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19706.004705 secs 2023-09-10T04:47:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19706.004705 secs 2023-09-10T04:47:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:55 smithi082 ceph-mon[98694]: pgmap v10670: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 253 KiB/s wr, 11 op/s 2023-09-10T04:47:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:55 smithi098 ceph-mon[103124]: pgmap v10670: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 253 KiB/s wr, 11 op/s 2023-09-10T04:47:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:55 smithi170 ceph-mon[102832]: pgmap v10670: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 253 KiB/s wr, 11 op/s 2023-09-10T04:47:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:47:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:47:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:47:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:47:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:47:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:47:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:47:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:47:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:47:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:57 smithi082 ceph-mon[98694]: pgmap v10671: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 320 KiB/s wr, 16 op/s 2023-09-10T04:47:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:57 smithi098 ceph-mon[103124]: pgmap v10671: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 320 KiB/s wr, 16 op/s 2023-09-10T04:47:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:57 smithi170 ceph-mon[102832]: pgmap v10671: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 320 KiB/s wr, 16 op/s 2023-09-10T04:47:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19709.228582 secs 2023-09-10T04:47:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19710.350049 secs 2023-09-10T04:47:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19709.228582 secs 2023-09-10T04:47:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19710.350049 secs 2023-09-10T04:47:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19709.228582 secs 2023-09-10T04:47:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19710.350049 secs 2023-09-10T04:47:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:59 smithi082 ceph-mon[98694]: pgmap v10672: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T04:47:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:47:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19711.004773 secs 2023-09-10T04:47:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:59 smithi098 ceph-mon[103124]: pgmap v10672: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T04:47:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:47:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19711.004773 secs 2023-09-10T04:47:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:59 smithi170 ceph-mon[102832]: pgmap v10672: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T04:47:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:47:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19711.004773 secs 2023-09-10T04:48:01.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:01 smithi082 ceph-mon[98694]: pgmap v10673: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T04:48:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:01 smithi098 ceph-mon[103124]: pgmap v10673: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T04:48:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:01 smithi170 ceph-mon[102832]: pgmap v10673: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T04:48:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:03 smithi082 ceph-mon[98694]: pgmap v10674: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 280 KiB/s wr, 16 op/s 2023-09-10T04:48:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19714.228644 secs 2023-09-10T04:48:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19715.350120 secs 2023-09-10T04:48:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:03 smithi098 ceph-mon[103124]: pgmap v10674: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 280 KiB/s wr, 16 op/s 2023-09-10T04:48:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19714.228644 secs 2023-09-10T04:48:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19715.350120 secs 2023-09-10T04:48:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:03 smithi170 ceph-mon[102832]: pgmap v10674: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 280 KiB/s wr, 16 op/s 2023-09-10T04:48:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19714.228644 secs 2023-09-10T04:48:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19715.350120 secs 2023-09-10T04:48:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19716.004834 secs 2023-09-10T04:48:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19716.004834 secs 2023-09-10T04:48:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19716.004834 secs 2023-09-10T04:48:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:05 smithi082 ceph-mon[98694]: pgmap v10675: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 158 KiB/s wr, 10 op/s 2023-09-10T04:48:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:05 smithi098 ceph-mon[103124]: pgmap v10675: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 158 KiB/s wr, 10 op/s 2023-09-10T04:48:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:05 smithi170 ceph-mon[102832]: pgmap v10675: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 158 KiB/s wr, 10 op/s 2023-09-10T04:48:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:07 smithi098 ceph-mon[103124]: pgmap v10676: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 266 KiB/s wr, 30 op/s 2023-09-10T04:48:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:07 smithi170 ceph-mon[102832]: pgmap v10676: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 266 KiB/s wr, 30 op/s 2023-09-10T04:48:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:07 smithi082 ceph-mon[98694]: pgmap v10676: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 266 KiB/s wr, 30 op/s 2023-09-10T04:48:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19719.228716 secs 2023-09-10T04:48:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19720.350176 secs 2023-09-10T04:48:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19719.228716 secs 2023-09-10T04:48:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19720.350176 secs 2023-09-10T04:48:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19719.228716 secs 2023-09-10T04:48:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19720.350176 secs 2023-09-10T04:48:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:09 smithi098 ceph-mon[103124]: pgmap v10677: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 199 KiB/s wr, 24 op/s 2023-09-10T04:48:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:09 smithi098 ceph-mon[103124]: osdmap e735: 12 total, 12 up, 12 in 2023-09-10T04:48:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19721.004901 secs 2023-09-10T04:48:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:09 smithi170 ceph-mon[102832]: pgmap v10677: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 199 KiB/s wr, 24 op/s 2023-09-10T04:48:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:09 smithi170 ceph-mon[102832]: osdmap e735: 12 total, 12 up, 12 in 2023-09-10T04:48:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19721.004901 secs 2023-09-10T04:48:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:09 smithi082 ceph-mon[98694]: pgmap v10677: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 199 KiB/s wr, 24 op/s 2023-09-10T04:48:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:09 smithi082 ceph-mon[98694]: osdmap e735: 12 total, 12 up, 12 in 2023-09-10T04:48:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19721.004901 secs 2023-09-10T04:48:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:11 smithi098 ceph-mon[103124]: pgmap v10679: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 239 KiB/s wr, 29 op/s 2023-09-10T04:48:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:11 smithi170 ceph-mon[102832]: pgmap v10679: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 239 KiB/s wr, 29 op/s 2023-09-10T04:48:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:11 smithi082 ceph-mon[98694]: pgmap v10679: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 239 KiB/s wr, 29 op/s 2023-09-10T04:48:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:13 smithi098 ceph-mon[103124]: pgmap v10680: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 131 KiB/s wr, 29 op/s 2023-09-10T04:48:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19724.228790 secs 2023-09-10T04:48:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19725.350261 secs 2023-09-10T04:48:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:13 smithi170 ceph-mon[102832]: pgmap v10680: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 131 KiB/s wr, 29 op/s 2023-09-10T04:48:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19724.228790 secs 2023-09-10T04:48:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19725.350261 secs 2023-09-10T04:48:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:13 smithi082 ceph-mon[98694]: pgmap v10680: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 131 KiB/s wr, 29 op/s 2023-09-10T04:48:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19724.228790 secs 2023-09-10T04:48:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19725.350261 secs 2023-09-10T04:48:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19726.004964 secs 2023-09-10T04:48:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:14 smithi098 ceph-mon[103124]: osdmap e736: 12 total, 12 up, 12 in 2023-09-10T04:48:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19726.004964 secs 2023-09-10T04:48:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:14 smithi170 ceph-mon[102832]: osdmap e736: 12 total, 12 up, 12 in 2023-09-10T04:48:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19726.004964 secs 2023-09-10T04:48:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:14 smithi082 ceph-mon[98694]: osdmap e736: 12 total, 12 up, 12 in 2023-09-10T04:48:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:15 smithi098 ceph-mon[103124]: pgmap v10682: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:48:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:15 smithi170 ceph-mon[102832]: pgmap v10682: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:48:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:15 smithi082 ceph-mon[98694]: pgmap v10682: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:48:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:17 smithi098 ceph-mon[103124]: pgmap v10683: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 15 op/s 2023-09-10T04:48:17.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:17 smithi170 ceph-mon[102832]: pgmap v10683: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 15 op/s 2023-09-10T04:48:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:17 smithi082 ceph-mon[98694]: pgmap v10683: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 15 op/s 2023-09-10T04:48:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:48:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:48:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19729.228886 secs 2023-09-10T04:48:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19730.350343 secs 2023-09-10T04:48:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:48:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:48:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19729.228886 secs 2023-09-10T04:48:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19730.350343 secs 2023-09-10T04:48:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:48:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:48:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19729.228886 secs 2023-09-10T04:48:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19730.350343 secs 2023-09-10T04:48:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:19 smithi098 ceph-mon[103124]: pgmap v10684: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 13 op/s 2023-09-10T04:48:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19731.005030 secs 2023-09-10T04:48:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:19 smithi170 ceph-mon[102832]: pgmap v10684: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 13 op/s 2023-09-10T04:48:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19731.005030 secs 2023-09-10T04:48:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:19 smithi082 ceph-mon[98694]: pgmap v10684: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 13 op/s 2023-09-10T04:48:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19731.005030 secs 2023-09-10T04:48:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:48:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:48:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:48:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:48:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:48:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:48:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:21 smithi098 ceph-mon[103124]: pgmap v10685: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T04:48:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:21 smithi170 ceph-mon[102832]: pgmap v10685: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T04:48:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:21 smithi082 ceph-mon[98694]: pgmap v10685: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T04:48:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:23 smithi098 ceph-mon[103124]: pgmap v10686: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T04:48:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19734.228946 secs 2023-09-10T04:48:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19735.350402 secs 2023-09-10T04:48:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:23 smithi170 ceph-mon[102832]: pgmap v10686: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T04:48:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19734.228946 secs 2023-09-10T04:48:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19735.350402 secs 2023-09-10T04:48:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:23 smithi082 ceph-mon[98694]: pgmap v10686: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T04:48:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19734.228946 secs 2023-09-10T04:48:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19735.350402 secs 2023-09-10T04:48:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19736.005117 secs 2023-09-10T04:48:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19736.005117 secs 2023-09-10T04:48:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19736.005117 secs 2023-09-10T04:48:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:25 smithi170 ceph-mon[102832]: pgmap v10687: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T04:48:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:25 smithi082 ceph-mon[98694]: pgmap v10687: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T04:48:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:25 smithi098 ceph-mon[103124]: pgmap v10687: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T04:48:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:27 smithi170 ceph-mon[102832]: pgmap v10688: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 161 KiB/s wr, 15 op/s 2023-09-10T04:48:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:27 smithi082 ceph-mon[98694]: pgmap v10688: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 161 KiB/s wr, 15 op/s 2023-09-10T04:48:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:27 smithi098 ceph-mon[103124]: pgmap v10688: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 161 KiB/s wr, 15 op/s 2023-09-10T04:48:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19740.350489 secs 2023-09-10T04:48:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19740.350489 secs 2023-09-10T04:48:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19740.350489 secs 2023-09-10T04:48:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:29 smithi170 ceph-mon[102832]: pgmap v10689: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:48:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19741.005181 secs 2023-09-10T04:48:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:29 smithi082 ceph-mon[98694]: pgmap v10689: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:48:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19741.005181 secs 2023-09-10T04:48:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:29 smithi098 ceph-mon[103124]: pgmap v10689: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:48:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19741.005181 secs 2023-09-10T04:48:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:31 smithi170 ceph-mon[102832]: pgmap v10690: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:48:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:48:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:31 smithi082 ceph-mon[98694]: pgmap v10690: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:48:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:48:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:31 smithi098 ceph-mon[103124]: pgmap v10690: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:48:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:48:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:48:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:48:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:48:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:48:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:48:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:48:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:48:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:48:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:48:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:33 smithi170 ceph-mon[102832]: pgmap v10691: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 199 KiB/s wr, 16 op/s 2023-09-10T04:48:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19745.350573 secs 2023-09-10T04:48:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:33 smithi082 ceph-mon[98694]: pgmap v10691: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 199 KiB/s wr, 16 op/s 2023-09-10T04:48:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19745.350573 secs 2023-09-10T04:48:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:33 smithi098 ceph-mon[103124]: pgmap v10691: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 199 KiB/s wr, 16 op/s 2023-09-10T04:48:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19745.350573 secs 2023-09-10T04:48:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19746.005246 secs 2023-09-10T04:48:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19746.005246 secs 2023-09-10T04:48:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19746.005246 secs 2023-09-10T04:48:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:35 smithi170 ceph-mon[102832]: pgmap v10692: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 199 KiB/s wr, 11 op/s 2023-09-10T04:48:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:35 smithi082 ceph-mon[98694]: pgmap v10692: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 199 KiB/s wr, 11 op/s 2023-09-10T04:48:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:35 smithi098 ceph-mon[103124]: pgmap v10692: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 199 KiB/s wr, 11 op/s 2023-09-10T04:48:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:37 smithi170 ceph-mon[102832]: pgmap v10693: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 331 KiB/s wr, 17 op/s 2023-09-10T04:48:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:37 smithi082 ceph-mon[98694]: pgmap v10693: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 331 KiB/s wr, 17 op/s 2023-09-10T04:48:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:37 smithi098 ceph-mon[103124]: pgmap v10693: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 331 KiB/s wr, 17 op/s 2023-09-10T04:48:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:48:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:48:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19750.350635 secs 2023-09-10T04:48:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:48:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:48:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19750.350635 secs 2023-09-10T04:48:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:48:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:48:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19750.350635 secs 2023-09-10T04:48:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:39 smithi170 ceph-mon[102832]: pgmap v10694: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 271 KiB/s wr, 11 op/s 2023-09-10T04:48:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19751.005316 secs 2023-09-10T04:48:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:39 smithi082 ceph-mon[98694]: pgmap v10694: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 271 KiB/s wr, 11 op/s 2023-09-10T04:48:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19751.005316 secs 2023-09-10T04:48:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:39 smithi098 ceph-mon[103124]: pgmap v10694: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 271 KiB/s wr, 11 op/s 2023-09-10T04:48:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19751.005316 secs 2023-09-10T04:48:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:48:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:48:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:48:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:48:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:48:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:48:41.933 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:41 smithi170 ceph-mon[102832]: pgmap v10695: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T04:48:41.933 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:48:41.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:41 smithi082 ceph-mon[98694]: pgmap v10695: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T04:48:41.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:48:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:41 smithi098 ceph-mon[103124]: pgmap v10695: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T04:48:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:48:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:43 smithi082 ceph-mon[98694]: pgmap v10696: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 389 KiB/s wr, 18 op/s 2023-09-10T04:48:43.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19755.350695 secs 2023-09-10T04:48:43.727 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:43 smithi098 ceph-mon[103124]: pgmap v10696: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 389 KiB/s wr, 18 op/s 2023-09-10T04:48:43.727 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19755.350695 secs 2023-09-10T04:48:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:43 smithi170 ceph-mon[102832]: pgmap v10696: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 389 KiB/s wr, 18 op/s 2023-09-10T04:48:43.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19755.350695 secs 2023-09-10T04:48:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19756.005386 secs 2023-09-10T04:48:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19756.005386 secs 2023-09-10T04:48:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19756.005386 secs 2023-09-10T04:48:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:45 smithi170 ceph-mon[102832]: pgmap v10697: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 250 KiB/s wr, 12 op/s 2023-09-10T04:48:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:48:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:48:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:45 smithi098 ceph-mon[103124]: pgmap v10697: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 250 KiB/s wr, 12 op/s 2023-09-10T04:48:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:48:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:48:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:45 smithi082 ceph-mon[98694]: pgmap v10697: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 250 KiB/s wr, 12 op/s 2023-09-10T04:48:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:48:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:48:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:48:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:48:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:48:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:48:47.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:48:47.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:48:47.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:48:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:48:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:48:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:48:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:48:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:48:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:48:47.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:48:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:48:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:48:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:48:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:48:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:48:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:48:47.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:48:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:47 smithi170 ceph-mon[102832]: pgmap v10698: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 375 KiB/s wr, 18 op/s 2023-09-10T04:48:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:47 smithi082 ceph-mon[98694]: pgmap v10698: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 375 KiB/s wr, 18 op/s 2023-09-10T04:48:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:47 smithi098 ceph-mon[103124]: pgmap v10698: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 375 KiB/s wr, 18 op/s 2023-09-10T04:48:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19760.350755 secs 2023-09-10T04:48:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19760.350755 secs 2023-09-10T04:48:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19760.350755 secs 2023-09-10T04:48:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:49 smithi170 ceph-mon[102832]: pgmap v10699: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 242 KiB/s wr, 12 op/s 2023-09-10T04:48:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19761.005453 secs 2023-09-10T04:48:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:49 smithi082 ceph-mon[98694]: pgmap v10699: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 242 KiB/s wr, 12 op/s 2023-09-10T04:48:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19761.005453 secs 2023-09-10T04:48:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:49 smithi098 ceph-mon[103124]: pgmap v10699: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 242 KiB/s wr, 12 op/s 2023-09-10T04:48:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19761.005453 secs 2023-09-10T04:48:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:51 smithi170 ceph-mon[102832]: pgmap v10700: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 242 KiB/s wr, 12 op/s 2023-09-10T04:48:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:51 smithi082 ceph-mon[98694]: pgmap v10700: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 242 KiB/s wr, 12 op/s 2023-09-10T04:48:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:51 smithi098 ceph-mon[103124]: pgmap v10700: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 242 KiB/s wr, 12 op/s 2023-09-10T04:48:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:53 smithi170 ceph-mon[102832]: pgmap v10701: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 372 KiB/s wr, 17 op/s 2023-09-10T04:48:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19764.229490 secs 2023-09-10T04:48:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19765.350816 secs 2023-09-10T04:48:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:53 smithi082 ceph-mon[98694]: pgmap v10701: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 372 KiB/s wr, 17 op/s 2023-09-10T04:48:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19764.229490 secs 2023-09-10T04:48:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19765.350816 secs 2023-09-10T04:48:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:53 smithi098 ceph-mon[103124]: pgmap v10701: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 372 KiB/s wr, 17 op/s 2023-09-10T04:48:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19764.229490 secs 2023-09-10T04:48:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19765.350816 secs 2023-09-10T04:48:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19766.005515 secs 2023-09-10T04:48:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19766.005515 secs 2023-09-10T04:48:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19766.005515 secs 2023-09-10T04:48:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:55 smithi170 ceph-mon[102832]: pgmap v10702: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 255 KiB/s wr, 10 op/s 2023-09-10T04:48:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:55 smithi082 ceph-mon[98694]: pgmap v10702: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 255 KiB/s wr, 10 op/s 2023-09-10T04:48:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:55 smithi098 ceph-mon[103124]: pgmap v10702: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 255 KiB/s wr, 10 op/s 2023-09-10T04:48:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:48:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:48:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:48:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:48:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:48:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:48:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:48:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:48:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:48:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:57 smithi170 ceph-mon[102832]: pgmap v10703: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 379 KiB/s wr, 16 op/s 2023-09-10T04:48:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:48:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:48:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:57 smithi082 ceph-mon[98694]: pgmap v10703: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 379 KiB/s wr, 16 op/s 2023-09-10T04:48:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:48:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:48:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:57 smithi098 ceph-mon[103124]: pgmap v10703: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 379 KiB/s wr, 16 op/s 2023-09-10T04:48:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:48:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:48:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19769.229584 secs 2023-09-10T04:48:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19770.350905 secs 2023-09-10T04:48:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19769.229584 secs 2023-09-10T04:48:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19770.350905 secs 2023-09-10T04:48:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19769.229584 secs 2023-09-10T04:48:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19770.350905 secs 2023-09-10T04:49:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:59 smithi170 ceph-mon[102832]: pgmap v10704: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 254 KiB/s wr, 11 op/s 2023-09-10T04:49:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:48:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19771.005586 secs 2023-09-10T04:49:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:59 smithi082 ceph-mon[98694]: pgmap v10704: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 254 KiB/s wr, 11 op/s 2023-09-10T04:49:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:48:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19771.005586 secs 2023-09-10T04:49:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:59 smithi098 ceph-mon[103124]: pgmap v10704: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 254 KiB/s wr, 11 op/s 2023-09-10T04:49:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:48:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19771.005586 secs 2023-09-10T04:49:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:49:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:49:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:49:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:49:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:49:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:49:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:01 smithi170 ceph-mon[102832]: pgmap v10705: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 254 KiB/s wr, 11 op/s 2023-09-10T04:49:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:01 smithi082 ceph-mon[98694]: pgmap v10705: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 254 KiB/s wr, 11 op/s 2023-09-10T04:49:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:01 smithi098 ceph-mon[103124]: pgmap v10705: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 254 KiB/s wr, 11 op/s 2023-09-10T04:49:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:03 smithi170 ceph-mon[102832]: pgmap v10706: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 345 KiB/s wr, 17 op/s 2023-09-10T04:49:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19774.229701 secs 2023-09-10T04:49:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19775.350981 secs 2023-09-10T04:49:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:03 smithi082 ceph-mon[98694]: pgmap v10706: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 345 KiB/s wr, 17 op/s 2023-09-10T04:49:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19774.229701 secs 2023-09-10T04:49:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19775.350981 secs 2023-09-10T04:49:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:03 smithi098 ceph-mon[103124]: pgmap v10706: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 345 KiB/s wr, 17 op/s 2023-09-10T04:49:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19774.229701 secs 2023-09-10T04:49:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19775.350981 secs 2023-09-10T04:49:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19776.005653 secs 2023-09-10T04:49:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19776.005653 secs 2023-09-10T04:49:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19776.005653 secs 2023-09-10T04:49:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:05 smithi170 ceph-mon[102832]: pgmap v10707: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 216 KiB/s wr, 11 op/s 2023-09-10T04:49:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:05 smithi082 ceph-mon[98694]: pgmap v10707: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 216 KiB/s wr, 11 op/s 2023-09-10T04:49:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:05 smithi098 ceph-mon[103124]: pgmap v10707: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 216 KiB/s wr, 11 op/s 2023-09-10T04:49:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:07 smithi170 ceph-mon[102832]: pgmap v10708: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 325 KiB/s wr, 31 op/s 2023-09-10T04:49:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:07 smithi082 ceph-mon[98694]: pgmap v10708: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 325 KiB/s wr, 31 op/s 2023-09-10T04:49:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:07 smithi098 ceph-mon[103124]: pgmap v10708: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 325 KiB/s wr, 31 op/s 2023-09-10T04:49:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19779.229764 secs 2023-09-10T04:49:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:08 smithi170 ceph-mon[102832]: osdmap e737: 12 total, 12 up, 12 in 2023-09-10T04:49:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19780.351065 secs 2023-09-10T04:49:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19779.229764 secs 2023-09-10T04:49:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:08 smithi082 ceph-mon[98694]: osdmap e737: 12 total, 12 up, 12 in 2023-09-10T04:49:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19780.351065 secs 2023-09-10T04:49:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19779.229764 secs 2023-09-10T04:49:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:08 smithi098 ceph-mon[103124]: osdmap e737: 12 total, 12 up, 12 in 2023-09-10T04:49:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19780.351065 secs 2023-09-10T04:49:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:09 smithi170 ceph-mon[102832]: pgmap v10710: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 240 KiB/s wr, 30 op/s 2023-09-10T04:49:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19781.005719 secs 2023-09-10T04:49:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:09 smithi082 ceph-mon[98694]: pgmap v10710: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 240 KiB/s wr, 30 op/s 2023-09-10T04:49:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19781.005719 secs 2023-09-10T04:49:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:09 smithi098 ceph-mon[103124]: pgmap v10710: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 240 KiB/s wr, 30 op/s 2023-09-10T04:49:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19781.005719 secs 2023-09-10T04:49:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:10 smithi170 ceph-mon[102832]: pgmap v10711: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 240 KiB/s wr, 30 op/s 2023-09-10T04:49:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:10 smithi082 ceph-mon[98694]: pgmap v10711: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 240 KiB/s wr, 30 op/s 2023-09-10T04:49:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:10 smithi098 ceph-mon[103124]: pgmap v10711: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 240 KiB/s wr, 30 op/s 2023-09-10T04:49:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:13 smithi082 ceph-mon[98694]: pgmap v10712: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 131 KiB/s wr, 29 op/s 2023-09-10T04:49:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19784.229794 secs 2023-09-10T04:49:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19785.351122 secs 2023-09-10T04:49:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:13 smithi098 ceph-mon[103124]: pgmap v10712: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 131 KiB/s wr, 29 op/s 2023-09-10T04:49:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19784.229794 secs 2023-09-10T04:49:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19785.351122 secs 2023-09-10T04:49:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:13 smithi170 ceph-mon[102832]: pgmap v10712: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 131 KiB/s wr, 29 op/s 2023-09-10T04:49:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19784.229794 secs 2023-09-10T04:49:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19785.351122 secs 2023-09-10T04:49:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19786.005784 secs 2023-09-10T04:49:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:14 smithi082 ceph-mon[98694]: osdmap e738: 12 total, 12 up, 12 in 2023-09-10T04:49:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19786.005784 secs 2023-09-10T04:49:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:14 smithi098 ceph-mon[103124]: osdmap e738: 12 total, 12 up, 12 in 2023-09-10T04:49:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19786.005784 secs 2023-09-10T04:49:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:14 smithi170 ceph-mon[102832]: osdmap e738: 12 total, 12 up, 12 in 2023-09-10T04:49:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:15 smithi082 ceph-mon[98694]: pgmap v10714: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:49:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:15 smithi098 ceph-mon[103124]: pgmap v10714: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:49:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:15 smithi170 ceph-mon[102832]: pgmap v10714: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:49:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:17 smithi082 ceph-mon[98694]: pgmap v10715: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 14 op/s 2023-09-10T04:49:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:17 smithi098 ceph-mon[103124]: pgmap v10715: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 14 op/s 2023-09-10T04:49:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:17 smithi170 ceph-mon[102832]: pgmap v10715: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 14 op/s 2023-09-10T04:49:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19789.229855 secs 2023-09-10T04:49:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19790.351178 secs 2023-09-10T04:49:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19789.229855 secs 2023-09-10T04:49:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19790.351178 secs 2023-09-10T04:49:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19789.229855 secs 2023-09-10T04:49:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19790.351178 secs 2023-09-10T04:49:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:19 smithi082 ceph-mon[98694]: pgmap v10716: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T04:49:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19791.005850 secs 2023-09-10T04:49:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:19 smithi098 ceph-mon[103124]: pgmap v10716: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T04:49:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19791.005850 secs 2023-09-10T04:49:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:19 smithi170 ceph-mon[102832]: pgmap v10716: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T04:49:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19791.005850 secs 2023-09-10T04:49:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:21 smithi098 ceph-mon[103124]: pgmap v10717: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T04:49:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:21 smithi170 ceph-mon[102832]: pgmap v10717: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T04:49:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:21 smithi082 ceph-mon[98694]: pgmap v10717: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T04:49:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:23 smithi098 ceph-mon[103124]: pgmap v10718: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T04:49:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19794.229915 secs 2023-09-10T04:49:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19795.351242 secs 2023-09-10T04:49:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:23 smithi170 ceph-mon[102832]: pgmap v10718: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T04:49:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19794.229915 secs 2023-09-10T04:49:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19795.351242 secs 2023-09-10T04:49:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:23 smithi082 ceph-mon[98694]: pgmap v10718: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T04:49:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19794.229915 secs 2023-09-10T04:49:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19795.351242 secs 2023-09-10T04:49:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19796.005910 secs 2023-09-10T04:49:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19796.005910 secs 2023-09-10T04:49:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19796.005910 secs 2023-09-10T04:49:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:25 smithi098 ceph-mon[103124]: pgmap v10719: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T04:49:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:25 smithi170 ceph-mon[102832]: pgmap v10719: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T04:49:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:25 smithi082 ceph-mon[98694]: pgmap v10719: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T04:49:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:27 smithi098 ceph-mon[103124]: pgmap v10720: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 162 KiB/s wr, 15 op/s 2023-09-10T04:49:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:27 smithi170 ceph-mon[102832]: pgmap v10720: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 162 KiB/s wr, 15 op/s 2023-09-10T04:49:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:27 smithi082 ceph-mon[98694]: pgmap v10720: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 162 KiB/s wr, 15 op/s 2023-09-10T04:49:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19800.351303 secs 2023-09-10T04:49:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19800.351303 secs 2023-09-10T04:49:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19800.351303 secs 2023-09-10T04:49:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:29 smithi098 ceph-mon[103124]: pgmap v10721: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:49:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19801.005975 secs 2023-09-10T04:49:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:29 smithi170 ceph-mon[102832]: pgmap v10721: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:49:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19801.005975 secs 2023-09-10T04:49:29.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:29 smithi082 ceph-mon[98694]: pgmap v10721: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:49:29.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19801.005975 secs 2023-09-10T04:49:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:49:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:49:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:49:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:49:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:49:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:49:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:31 smithi098 ceph-mon[103124]: pgmap v10722: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:49:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:49:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:31 smithi170 ceph-mon[102832]: pgmap v10722: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:49:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:49:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:31 smithi082 ceph-mon[98694]: pgmap v10722: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 10 op/s 2023-09-10T04:49:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:49:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:49:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:49:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:49:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:49:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:49:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:49:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:49:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:49:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:49:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:33 smithi098 ceph-mon[103124]: pgmap v10723: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 200 KiB/s wr, 16 op/s 2023-09-10T04:49:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19805.351364 secs 2023-09-10T04:49:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:33 smithi170 ceph-mon[102832]: pgmap v10723: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 200 KiB/s wr, 16 op/s 2023-09-10T04:49:33.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19805.351364 secs 2023-09-10T04:49:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:33 smithi082 ceph-mon[98694]: pgmap v10723: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 200 KiB/s wr, 16 op/s 2023-09-10T04:49:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19805.351364 secs 2023-09-10T04:49:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19806.006038 secs 2023-09-10T04:49:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19806.006038 secs 2023-09-10T04:49:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19806.006038 secs 2023-09-10T04:49:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:35 smithi098 ceph-mon[103124]: pgmap v10724: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 200 KiB/s wr, 11 op/s 2023-09-10T04:49:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:35 smithi170 ceph-mon[102832]: pgmap v10724: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 200 KiB/s wr, 11 op/s 2023-09-10T04:49:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:35 smithi082 ceph-mon[98694]: pgmap v10724: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 200 KiB/s wr, 11 op/s 2023-09-10T04:49:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:37 smithi098 ceph-mon[103124]: pgmap v10725: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 333 KiB/s wr, 17 op/s 2023-09-10T04:49:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:37 smithi170 ceph-mon[102832]: pgmap v10725: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 333 KiB/s wr, 17 op/s 2023-09-10T04:49:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:37 smithi082 ceph-mon[98694]: pgmap v10725: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 333 KiB/s wr, 17 op/s 2023-09-10T04:49:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:49:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:49:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19810.351467 secs 2023-09-10T04:49:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:49:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:49:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19810.351467 secs 2023-09-10T04:49:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:49:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:49:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19810.351467 secs 2023-09-10T04:49:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:39 smithi170 ceph-mon[102832]: pgmap v10726: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 272 KiB/s wr, 12 op/s 2023-09-10T04:49:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19811.006106 secs 2023-09-10T04:49:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:39 smithi082 ceph-mon[98694]: pgmap v10726: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 272 KiB/s wr, 12 op/s 2023-09-10T04:49:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19811.006106 secs 2023-09-10T04:49:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:39 smithi098 ceph-mon[103124]: pgmap v10726: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 272 KiB/s wr, 12 op/s 2023-09-10T04:49:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19811.006106 secs 2023-09-10T04:49:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:49:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:49:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:49:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:49:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:49:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:49:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:41 smithi170 ceph-mon[102832]: pgmap v10727: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 272 KiB/s wr, 12 op/s 2023-09-10T04:49:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:41 smithi082 ceph-mon[98694]: pgmap v10727: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 272 KiB/s wr, 12 op/s 2023-09-10T04:49:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:41 smithi098 ceph-mon[103124]: pgmap v10727: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 272 KiB/s wr, 12 op/s 2023-09-10T04:49:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:43 smithi170 ceph-mon[102832]: pgmap v10728: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 391 KiB/s wr, 19 op/s 2023-09-10T04:49:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19815.351495 secs 2023-09-10T04:49:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:43 smithi082 ceph-mon[98694]: pgmap v10728: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 391 KiB/s wr, 19 op/s 2023-09-10T04:49:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19815.351495 secs 2023-09-10T04:49:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:43 smithi098 ceph-mon[103124]: pgmap v10728: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 391 KiB/s wr, 19 op/s 2023-09-10T04:49:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19815.351495 secs 2023-09-10T04:49:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19816.006167 secs 2023-09-10T04:49:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19816.006167 secs 2023-09-10T04:49:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19816.006167 secs 2023-09-10T04:49:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:45 smithi170 ceph-mon[102832]: pgmap v10729: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 251 KiB/s wr, 13 op/s 2023-09-10T04:49:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:45 smithi082 ceph-mon[98694]: pgmap v10729: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 251 KiB/s wr, 13 op/s 2023-09-10T04:49:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:45 smithi098 ceph-mon[103124]: pgmap v10729: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 251 KiB/s wr, 13 op/s 2023-09-10T04:49:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:49:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:49:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:49:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:47 smithi170 ceph-mon[102832]: pgmap v10730: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 377 KiB/s wr, 19 op/s 2023-09-10T04:49:47.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:49:47.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:49:47.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:49:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:47 smithi082 ceph-mon[98694]: pgmap v10730: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 377 KiB/s wr, 19 op/s 2023-09-10T04:49:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:49:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:49:47.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:49:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:47 smithi098 ceph-mon[103124]: pgmap v10730: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 377 KiB/s wr, 19 op/s 2023-09-10T04:49:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:49:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:49:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:49:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19820.351553 secs 2023-09-10T04:49:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19820.351553 secs 2023-09-10T04:49:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19820.351553 secs 2023-09-10T04:49:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:49 smithi170 ceph-mon[102832]: pgmap v10731: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 245 KiB/s wr, 13 op/s 2023-09-10T04:49:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19821.006235 secs 2023-09-10T04:49:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:49 smithi082 ceph-mon[98694]: pgmap v10731: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 245 KiB/s wr, 13 op/s 2023-09-10T04:49:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19821.006235 secs 2023-09-10T04:49:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:49 smithi098 ceph-mon[103124]: pgmap v10731: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 245 KiB/s wr, 13 op/s 2023-09-10T04:49:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19821.006235 secs 2023-09-10T04:49:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:51 smithi170 ceph-mon[102832]: pgmap v10732: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 245 KiB/s wr, 13 op/s 2023-09-10T04:49:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:51 smithi082 ceph-mon[98694]: pgmap v10732: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 245 KiB/s wr, 13 op/s 2023-09-10T04:49:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:51 smithi098 ceph-mon[103124]: pgmap v10732: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 245 KiB/s wr, 13 op/s 2023-09-10T04:49:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:53 smithi170 ceph-mon[102832]: pgmap v10733: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 385 KiB/s wr, 19 op/s 2023-09-10T04:49:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19824.230373 secs 2023-09-10T04:49:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19825.351642 secs 2023-09-10T04:49:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:53 smithi082 ceph-mon[98694]: pgmap v10733: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 385 KiB/s wr, 19 op/s 2023-09-10T04:49:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19824.230373 secs 2023-09-10T04:49:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19825.351642 secs 2023-09-10T04:49:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:53 smithi098 ceph-mon[103124]: pgmap v10733: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 385 KiB/s wr, 19 op/s 2023-09-10T04:49:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19824.230373 secs 2023-09-10T04:49:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19825.351642 secs 2023-09-10T04:49:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19826.006300 secs 2023-09-10T04:49:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19826.006300 secs 2023-09-10T04:49:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19826.006300 secs 2023-09-10T04:49:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:55 smithi170 ceph-mon[102832]: pgmap v10734: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 266 KiB/s wr, 12 op/s 2023-09-10T04:49:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:55 smithi082 ceph-mon[98694]: pgmap v10734: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 266 KiB/s wr, 12 op/s 2023-09-10T04:49:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:55 smithi098 ceph-mon[103124]: pgmap v10734: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 266 KiB/s wr, 12 op/s 2023-09-10T04:49:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:49:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:49:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:49:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:49:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:49:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:49:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:49:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:49:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:49:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:57 smithi170 ceph-mon[102832]: pgmap v10735: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 334 KiB/s wr, 18 op/s 2023-09-10T04:49:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:57 smithi082 ceph-mon[98694]: pgmap v10735: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 334 KiB/s wr, 18 op/s 2023-09-10T04:49:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:57 smithi098 ceph-mon[103124]: pgmap v10735: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 334 KiB/s wr, 18 op/s 2023-09-10T04:49:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:49:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:49:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19829.230460 secs 2023-09-10T04:49:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19830.351726 secs 2023-09-10T04:49:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:49:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:49:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19829.230460 secs 2023-09-10T04:49:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19830.351726 secs 2023-09-10T04:49:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:49:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:49:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19829.230460 secs 2023-09-10T04:49:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19830.351726 secs 2023-09-10T04:49:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:59 smithi170 ceph-mon[102832]: pgmap v10736: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 208 KiB/s wr, 12 op/s 2023-09-10T04:49:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:49:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19831.006367 secs 2023-09-10T04:49:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:59 smithi082 ceph-mon[98694]: pgmap v10736: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 208 KiB/s wr, 12 op/s 2023-09-10T04:49:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:49:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19831.006367 secs 2023-09-10T04:49:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:59 smithi098 ceph-mon[103124]: pgmap v10736: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 208 KiB/s wr, 12 op/s 2023-09-10T04:49:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:49:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19831.006367 secs 2023-09-10T04:50:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:50:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:50:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T04:50:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:50:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:50:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T04:50:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:50:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:50:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T04:50:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:01 smithi170 ceph-mon[102832]: pgmap v10737: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 208 KiB/s wr, 12 op/s 2023-09-10T04:50:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:01 smithi082 ceph-mon[98694]: pgmap v10737: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 208 KiB/s wr, 12 op/s 2023-09-10T04:50:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:01 smithi098 ceph-mon[103124]: pgmap v10737: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 208 KiB/s wr, 12 op/s 2023-09-10T04:50:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:03 smithi170 ceph-mon[102832]: pgmap v10738: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 299 KiB/s wr, 17 op/s 2023-09-10T04:50:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19834.230521 secs 2023-09-10T04:50:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19835.351811 secs 2023-09-10T04:50:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:03 smithi082 ceph-mon[98694]: pgmap v10738: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 299 KiB/s wr, 17 op/s 2023-09-10T04:50:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19834.230521 secs 2023-09-10T04:50:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19835.351811 secs 2023-09-10T04:50:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:03 smithi098 ceph-mon[103124]: pgmap v10738: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 299 KiB/s wr, 17 op/s 2023-09-10T04:50:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19834.230521 secs 2023-09-10T04:50:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19835.351811 secs 2023-09-10T04:50:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19836.006431 secs 2023-09-10T04:50:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19836.006431 secs 2023-09-10T04:50:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19836.006431 secs 2023-09-10T04:50:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:05 smithi170 ceph-mon[102832]: pgmap v10739: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T04:50:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:05 smithi082 ceph-mon[98694]: pgmap v10739: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T04:50:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:05 smithi098 ceph-mon[103124]: pgmap v10739: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T04:50:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:07 smithi170 ceph-mon[102832]: pgmap v10740: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 254 KiB/s wr, 31 op/s 2023-09-10T04:50:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:07 smithi082 ceph-mon[98694]: pgmap v10740: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 254 KiB/s wr, 31 op/s 2023-09-10T04:50:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:07 smithi098 ceph-mon[103124]: pgmap v10740: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 254 KiB/s wr, 31 op/s 2023-09-10T04:50:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19839.230636 secs 2023-09-10T04:50:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19840.351895 secs 2023-09-10T04:50:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19839.230636 secs 2023-09-10T04:50:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19840.351895 secs 2023-09-10T04:50:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19839.230636 secs 2023-09-10T04:50:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19840.351895 secs 2023-09-10T04:50:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:09 smithi170 ceph-mon[102832]: pgmap v10741: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 186 KiB/s wr, 25 op/s 2023-09-10T04:50:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:09 smithi170 ceph-mon[102832]: osdmap e739: 12 total, 12 up, 12 in 2023-09-10T04:50:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19841.006497 secs 2023-09-10T04:50:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:09 smithi082 ceph-mon[98694]: pgmap v10741: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 186 KiB/s wr, 25 op/s 2023-09-10T04:50:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:09 smithi082 ceph-mon[98694]: osdmap e739: 12 total, 12 up, 12 in 2023-09-10T04:50:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19841.006497 secs 2023-09-10T04:50:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:09 smithi098 ceph-mon[103124]: pgmap v10741: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 186 KiB/s wr, 25 op/s 2023-09-10T04:50:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:09 smithi098 ceph-mon[103124]: osdmap e739: 12 total, 12 up, 12 in 2023-09-10T04:50:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19841.006497 secs 2023-09-10T04:50:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:11 smithi170 ceph-mon[102832]: pgmap v10743: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 223 KiB/s wr, 30 op/s 2023-09-10T04:50:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:11 smithi082 ceph-mon[98694]: pgmap v10743: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 223 KiB/s wr, 30 op/s 2023-09-10T04:50:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:11 smithi098 ceph-mon[103124]: pgmap v10743: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 223 KiB/s wr, 30 op/s 2023-09-10T04:50:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:13 smithi170 ceph-mon[102832]: pgmap v10744: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 114 KiB/s wr, 30 op/s 2023-09-10T04:50:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19844.230694 secs 2023-09-10T04:50:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19845.351965 secs 2023-09-10T04:50:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:13 smithi082 ceph-mon[98694]: pgmap v10744: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 114 KiB/s wr, 30 op/s 2023-09-10T04:50:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19844.230694 secs 2023-09-10T04:50:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19845.351965 secs 2023-09-10T04:50:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:13 smithi098 ceph-mon[103124]: pgmap v10744: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 114 KiB/s wr, 30 op/s 2023-09-10T04:50:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19844.230694 secs 2023-09-10T04:50:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19845.351965 secs 2023-09-10T04:50:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19846.006557 secs 2023-09-10T04:50:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:14 smithi170 ceph-mon[102832]: osdmap e740: 12 total, 12 up, 12 in 2023-09-10T04:50:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19846.006557 secs 2023-09-10T04:50:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:14 smithi082 ceph-mon[98694]: osdmap e740: 12 total, 12 up, 12 in 2023-09-10T04:50:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19846.006557 secs 2023-09-10T04:50:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:14 smithi098 ceph-mon[103124]: osdmap e740: 12 total, 12 up, 12 in 2023-09-10T04:50:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:15 smithi170 ceph-mon[102832]: pgmap v10746: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:50:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:15 smithi082 ceph-mon[98694]: pgmap v10746: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:50:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:15 smithi098 ceph-mon[103124]: pgmap v10746: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:50:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:17 smithi170 ceph-mon[102832]: pgmap v10747: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 15 op/s 2023-09-10T04:50:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:17 smithi082 ceph-mon[98694]: pgmap v10747: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 15 op/s 2023-09-10T04:50:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:17 smithi098 ceph-mon[103124]: pgmap v10747: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 15 op/s 2023-09-10T04:50:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19849.230851 secs 2023-09-10T04:50:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19850.352049 secs 2023-09-10T04:50:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19849.230851 secs 2023-09-10T04:50:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19850.352049 secs 2023-09-10T04:50:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19849.230851 secs 2023-09-10T04:50:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19850.352049 secs 2023-09-10T04:50:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:19 smithi170 ceph-mon[102832]: pgmap v10748: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T04:50:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19851.006623 secs 2023-09-10T04:50:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:19 smithi082 ceph-mon[98694]: pgmap v10748: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T04:50:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19851.006623 secs 2023-09-10T04:50:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:19 smithi098 ceph-mon[103124]: pgmap v10748: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T04:50:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19851.006623 secs 2023-09-10T04:50:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:21 smithi170 ceph-mon[102832]: pgmap v10749: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:50:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:21 smithi082 ceph-mon[98694]: pgmap v10749: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:50:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:21 smithi098 ceph-mon[103124]: pgmap v10749: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:50:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:23 smithi170 ceph-mon[102832]: pgmap v10750: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T04:50:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19854.230911 secs 2023-09-10T04:50:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19855.352144 secs 2023-09-10T04:50:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:23 smithi082 ceph-mon[98694]: pgmap v10750: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T04:50:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19854.230911 secs 2023-09-10T04:50:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19855.352144 secs 2023-09-10T04:50:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:23 smithi098 ceph-mon[103124]: pgmap v10750: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T04:50:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19854.230911 secs 2023-09-10T04:50:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19855.352144 secs 2023-09-10T04:50:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19856.006682 secs 2023-09-10T04:50:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19856.006682 secs 2023-09-10T04:50:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19856.006682 secs 2023-09-10T04:50:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:25 smithi170 ceph-mon[102832]: pgmap v10751: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T04:50:25.711 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:25 smithi082 ceph-mon[98694]: pgmap v10751: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T04:50:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:25 smithi098 ceph-mon[103124]: pgmap v10751: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T04:50:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:27 smithi170 ceph-mon[102832]: pgmap v10752: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 203 KiB/s wr, 14 op/s 2023-09-10T04:50:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:27 smithi082 ceph-mon[98694]: pgmap v10752: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 203 KiB/s wr, 14 op/s 2023-09-10T04:50:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:27 smithi098 ceph-mon[103124]: pgmap v10752: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 203 KiB/s wr, 14 op/s 2023-09-10T04:50:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19860.352203 secs 2023-09-10T04:50:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19860.352203 secs 2023-09-10T04:50:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19860.352203 secs 2023-09-10T04:50:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:29 smithi170 ceph-mon[102832]: pgmap v10753: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T04:50:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19861.006748 secs 2023-09-10T04:50:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:29 smithi082 ceph-mon[98694]: pgmap v10753: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T04:50:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19861.006748 secs 2023-09-10T04:50:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:29 smithi098 ceph-mon[103124]: pgmap v10753: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T04:50:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19861.006748 secs 2023-09-10T04:50:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:50:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:50:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:50:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:50:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:50:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:50:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:31 smithi170 ceph-mon[102832]: pgmap v10754: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T04:50:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:50:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:31 smithi082 ceph-mon[98694]: pgmap v10754: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T04:50:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:50:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:31 smithi098 ceph-mon[103124]: pgmap v10754: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T04:50:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:50:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:50:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:50:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:50:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:50:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:50:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:50:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:50:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:50:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:50:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:33 smithi170 ceph-mon[102832]: pgmap v10755: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 201 KiB/s wr, 16 op/s 2023-09-10T04:50:33.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19865.352290 secs 2023-09-10T04:50:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:33 smithi082 ceph-mon[98694]: pgmap v10755: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 201 KiB/s wr, 16 op/s 2023-09-10T04:50:33.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19865.352290 secs 2023-09-10T04:50:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:33 smithi098 ceph-mon[103124]: pgmap v10755: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 201 KiB/s wr, 16 op/s 2023-09-10T04:50:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19865.352290 secs 2023-09-10T04:50:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19866.006812 secs 2023-09-10T04:50:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19866.006812 secs 2023-09-10T04:50:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19866.006812 secs 2023-09-10T04:50:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:35 smithi170 ceph-mon[102832]: pgmap v10756: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 201 KiB/s wr, 11 op/s 2023-09-10T04:50:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:35 smithi082 ceph-mon[98694]: pgmap v10756: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 201 KiB/s wr, 11 op/s 2023-09-10T04:50:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:35 smithi098 ceph-mon[103124]: pgmap v10756: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 201 KiB/s wr, 11 op/s 2023-09-10T04:50:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:37 smithi170 ceph-mon[102832]: pgmap v10757: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 334 KiB/s wr, 17 op/s 2023-09-10T04:50:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:37 smithi082 ceph-mon[98694]: pgmap v10757: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 334 KiB/s wr, 17 op/s 2023-09-10T04:50:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:37 smithi098 ceph-mon[103124]: pgmap v10757: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 334 KiB/s wr, 17 op/s 2023-09-10T04:50:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19870.352389 secs 2023-09-10T04:50:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19870.352389 secs 2023-09-10T04:50:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19870.352389 secs 2023-09-10T04:50:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:39 smithi082 ceph-mon[98694]: pgmap v10758: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 233 KiB/s wr, 12 op/s 2023-09-10T04:50:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19871.006876 secs 2023-09-10T04:50:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:39 smithi098 ceph-mon[103124]: pgmap v10758: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 233 KiB/s wr, 12 op/s 2023-09-10T04:50:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19871.006876 secs 2023-09-10T04:50:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:39 smithi170 ceph-mon[102832]: pgmap v10758: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 233 KiB/s wr, 12 op/s 2023-09-10T04:50:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19871.006876 secs 2023-09-10T04:50:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:41 smithi082 ceph-mon[98694]: pgmap v10759: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 233 KiB/s wr, 12 op/s 2023-09-10T04:50:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:41 smithi098 ceph-mon[103124]: pgmap v10759: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 233 KiB/s wr, 12 op/s 2023-09-10T04:50:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:41 smithi170 ceph-mon[102832]: pgmap v10759: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 233 KiB/s wr, 12 op/s 2023-09-10T04:50:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:43 smithi082 ceph-mon[98694]: pgmap v10760: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 389 KiB/s wr, 19 op/s 2023-09-10T04:50:43.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19875.352447 secs 2023-09-10T04:50:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:43 smithi098 ceph-mon[103124]: pgmap v10760: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 389 KiB/s wr, 19 op/s 2023-09-10T04:50:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19875.352447 secs 2023-09-10T04:50:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:43 smithi170 ceph-mon[102832]: pgmap v10760: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 389 KiB/s wr, 19 op/s 2023-09-10T04:50:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19875.352447 secs 2023-09-10T04:50:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19876.006941 secs 2023-09-10T04:50:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19876.006941 secs 2023-09-10T04:50:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19876.006941 secs 2023-09-10T04:50:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:45 smithi082 ceph-mon[98694]: pgmap v10761: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 289 KiB/s wr, 12 op/s 2023-09-10T04:50:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:45 smithi098 ceph-mon[103124]: pgmap v10761: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 289 KiB/s wr, 12 op/s 2023-09-10T04:50:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:45 smithi170 ceph-mon[102832]: pgmap v10761: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 289 KiB/s wr, 12 op/s 2023-09-10T04:50:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:47 smithi082 ceph-mon[98694]: pgmap v10762: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 379 KiB/s wr, 19 op/s 2023-09-10T04:50:47.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:50:47.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:50:47.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:50:47.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:50:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:47 smithi098 ceph-mon[103124]: pgmap v10762: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 379 KiB/s wr, 19 op/s 2023-09-10T04:50:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:50:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:50:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:50:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:50:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:47 smithi170 ceph-mon[102832]: pgmap v10762: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 379 KiB/s wr, 19 op/s 2023-09-10T04:50:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:50:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:50:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:50:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:50:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19880.352531 secs 2023-09-10T04:50:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19880.352531 secs 2023-09-10T04:50:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19880.352531 secs 2023-09-10T04:50:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:49 smithi082 ceph-mon[98694]: pgmap v10763: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 246 KiB/s wr, 13 op/s 2023-09-10T04:50:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19881.007005 secs 2023-09-10T04:50:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:49 smithi098 ceph-mon[103124]: pgmap v10763: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 246 KiB/s wr, 13 op/s 2023-09-10T04:50:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19881.007005 secs 2023-09-10T04:50:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:49 smithi170 ceph-mon[102832]: pgmap v10763: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 246 KiB/s wr, 13 op/s 2023-09-10T04:50:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19881.007005 secs 2023-09-10T04:50:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:50:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:50:50.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:50:50.932 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:50:50.962 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:50:50.962 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:50:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:51 smithi098 ceph-mon[103124]: pgmap v10764: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 246 KiB/s wr, 13 op/s 2023-09-10T04:50:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:51 smithi170 ceph-mon[102832]: pgmap v10764: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 246 KiB/s wr, 13 op/s 2023-09-10T04:50:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:51 smithi082 ceph-mon[98694]: pgmap v10764: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 246 KiB/s wr, 13 op/s 2023-09-10T04:50:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:50:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:50:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:50:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:50:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:50:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:50:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:53 smithi098 ceph-mon[103124]: pgmap v10765: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 377 KiB/s wr, 19 op/s 2023-09-10T04:50:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19884.231337 secs 2023-09-10T04:50:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19885.352592 secs 2023-09-10T04:50:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:53 smithi170 ceph-mon[102832]: pgmap v10765: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 377 KiB/s wr, 19 op/s 2023-09-10T04:50:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19884.231337 secs 2023-09-10T04:50:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19885.352592 secs 2023-09-10T04:50:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:53 smithi082 ceph-mon[98694]: pgmap v10765: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 377 KiB/s wr, 19 op/s 2023-09-10T04:50:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19884.231337 secs 2023-09-10T04:50:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19885.352592 secs 2023-09-10T04:50:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19886.007069 secs 2023-09-10T04:50:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19886.007069 secs 2023-09-10T04:50:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19886.007069 secs 2023-09-10T04:50:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:55 smithi098 ceph-mon[103124]: pgmap v10766: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 221 KiB/s wr, 12 op/s 2023-09-10T04:50:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:55 smithi170 ceph-mon[102832]: pgmap v10766: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 221 KiB/s wr, 12 op/s 2023-09-10T04:50:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:55 smithi082 ceph-mon[98694]: pgmap v10766: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 221 KiB/s wr, 12 op/s 2023-09-10T04:50:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:50:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:50:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:50:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:50:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:50:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:50:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:50:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:50:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:50:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:57 smithi098 ceph-mon[103124]: pgmap v10767: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 289 KiB/s wr, 19 op/s 2023-09-10T04:50:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:57 smithi170 ceph-mon[102832]: pgmap v10767: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 289 KiB/s wr, 19 op/s 2023-09-10T04:50:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:57 smithi082 ceph-mon[98694]: pgmap v10767: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 289 KiB/s wr, 19 op/s 2023-09-10T04:50:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19889.231399 secs 2023-09-10T04:50:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19890.352679 secs 2023-09-10T04:50:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19889.231399 secs 2023-09-10T04:50:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19890.352679 secs 2023-09-10T04:50:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19889.231399 secs 2023-09-10T04:50:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19890.352679 secs 2023-09-10T04:50:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:59 smithi098 ceph-mon[103124]: pgmap v10768: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 199 KiB/s wr, 12 op/s 2023-09-10T04:50:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:50:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19891.007135 secs 2023-09-10T04:50:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:59 smithi170 ceph-mon[102832]: pgmap v10768: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 199 KiB/s wr, 12 op/s 2023-09-10T04:50:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:50:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19891.007135 secs 2023-09-10T04:50:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:59 smithi082 ceph-mon[98694]: pgmap v10768: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 199 KiB/s wr, 12 op/s 2023-09-10T04:50:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:50:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19891.007135 secs 2023-09-10T04:51:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:01 smithi098 ceph-mon[103124]: pgmap v10769: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 199 KiB/s wr, 12 op/s 2023-09-10T04:51:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:01 smithi170 ceph-mon[102832]: pgmap v10769: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 199 KiB/s wr, 12 op/s 2023-09-10T04:51:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:01 smithi082 ceph-mon[98694]: pgmap v10769: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 199 KiB/s wr, 12 op/s 2023-09-10T04:51:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:03 smithi098 ceph-mon[103124]: pgmap v10770: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 313 KiB/s wr, 27 op/s 2023-09-10T04:51:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19894.231515 secs 2023-09-10T04:51:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19895.352740 secs 2023-09-10T04:51:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:03 smithi170 ceph-mon[102832]: pgmap v10770: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 313 KiB/s wr, 27 op/s 2023-09-10T04:51:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19894.231515 secs 2023-09-10T04:51:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19895.352740 secs 2023-09-10T04:51:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:03 smithi082 ceph-mon[98694]: pgmap v10770: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 313 KiB/s wr, 27 op/s 2023-09-10T04:51:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19894.231515 secs 2023-09-10T04:51:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19895.352740 secs 2023-09-10T04:51:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19896.007197 secs 2023-09-10T04:51:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19896.007197 secs 2023-09-10T04:51:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19896.007197 secs 2023-09-10T04:51:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:05 smithi170 ceph-mon[102832]: pgmap v10771: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 182 KiB/s wr, 21 op/s 2023-09-10T04:51:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:05 smithi082 ceph-mon[98694]: pgmap v10771: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 182 KiB/s wr, 21 op/s 2023-09-10T04:51:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:05 smithi098 ceph-mon[103124]: pgmap v10771: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 182 KiB/s wr, 21 op/s 2023-09-10T04:51:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:07 smithi170 ceph-mon[102832]: pgmap v10772: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 255 KiB/s wr, 31 op/s 2023-09-10T04:51:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:07 smithi082 ceph-mon[98694]: pgmap v10772: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 255 KiB/s wr, 31 op/s 2023-09-10T04:51:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:07 smithi098 ceph-mon[103124]: pgmap v10772: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 255 KiB/s wr, 31 op/s 2023-09-10T04:51:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19899.231576 secs 2023-09-10T04:51:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19900.352798 secs 2023-09-10T04:51:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19899.231576 secs 2023-09-10T04:51:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19900.352798 secs 2023-09-10T04:51:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19899.231576 secs 2023-09-10T04:51:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19900.352798 secs 2023-09-10T04:51:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:09 smithi170 ceph-mon[102832]: pgmap v10773: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 187 KiB/s wr, 25 op/s 2023-09-10T04:51:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:09 smithi170 ceph-mon[102832]: osdmap e741: 12 total, 12 up, 12 in 2023-09-10T04:51:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19901.007262 secs 2023-09-10T04:51:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:09 smithi082 ceph-mon[98694]: pgmap v10773: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 187 KiB/s wr, 25 op/s 2023-09-10T04:51:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:09 smithi082 ceph-mon[98694]: osdmap e741: 12 total, 12 up, 12 in 2023-09-10T04:51:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19901.007262 secs 2023-09-10T04:51:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:09 smithi098 ceph-mon[103124]: pgmap v10773: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 187 KiB/s wr, 25 op/s 2023-09-10T04:51:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:09 smithi098 ceph-mon[103124]: osdmap e741: 12 total, 12 up, 12 in 2023-09-10T04:51:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19901.007262 secs 2023-09-10T04:51:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:51:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:51:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:51:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:51:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:51:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:51:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:11 smithi170 ceph-mon[102832]: pgmap v10775: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 224 KiB/s wr, 30 op/s 2023-09-10T04:51:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:11 smithi082 ceph-mon[98694]: pgmap v10775: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 224 KiB/s wr, 30 op/s 2023-09-10T04:51:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:11 smithi098 ceph-mon[103124]: pgmap v10775: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 224 KiB/s wr, 30 op/s 2023-09-10T04:51:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:51:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:51:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:51:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:51:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:51:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:51:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:13 smithi170 ceph-mon[102832]: pgmap v10776: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 87 KiB/s wr, 18 op/s 2023-09-10T04:51:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19904.231636 secs 2023-09-10T04:51:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19905.352858 secs 2023-09-10T04:51:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:13 smithi082 ceph-mon[98694]: pgmap v10776: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 87 KiB/s wr, 18 op/s 2023-09-10T04:51:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19904.231636 secs 2023-09-10T04:51:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19905.352858 secs 2023-09-10T04:51:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:13 smithi098 ceph-mon[103124]: pgmap v10776: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 87 KiB/s wr, 18 op/s 2023-09-10T04:51:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19904.231636 secs 2023-09-10T04:51:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19905.352858 secs 2023-09-10T04:51:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19906.007327 secs 2023-09-10T04:51:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:14 smithi170 ceph-mon[102832]: osdmap e742: 12 total, 12 up, 12 in 2023-09-10T04:51:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19906.007327 secs 2023-09-10T04:51:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:14 smithi082 ceph-mon[98694]: osdmap e742: 12 total, 12 up, 12 in 2023-09-10T04:51:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19906.007327 secs 2023-09-10T04:51:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:14 smithi098 ceph-mon[103124]: osdmap e742: 12 total, 12 up, 12 in 2023-09-10T04:51:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:15 smithi170 ceph-mon[102832]: pgmap v10778: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:51:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:15 smithi082 ceph-mon[98694]: pgmap v10778: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:51:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:15 smithi098 ceph-mon[103124]: pgmap v10778: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:51:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:17 smithi170 ceph-mon[102832]: pgmap v10779: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 16 op/s 2023-09-10T04:51:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:17 smithi082 ceph-mon[98694]: pgmap v10779: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 16 op/s 2023-09-10T04:51:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:17 smithi098 ceph-mon[103124]: pgmap v10779: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 16 op/s 2023-09-10T04:51:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19909.231696 secs 2023-09-10T04:51:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19910.352914 secs 2023-09-10T04:51:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19909.231696 secs 2023-09-10T04:51:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19910.352914 secs 2023-09-10T04:51:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19909.231696 secs 2023-09-10T04:51:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19910.352914 secs 2023-09-10T04:51:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:19 smithi170 ceph-mon[102832]: pgmap v10780: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 13 op/s 2023-09-10T04:51:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19911.007417 secs 2023-09-10T04:51:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:19 smithi082 ceph-mon[98694]: pgmap v10780: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 13 op/s 2023-09-10T04:51:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19911.007417 secs 2023-09-10T04:51:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:19 smithi098 ceph-mon[103124]: pgmap v10780: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 13 op/s 2023-09-10T04:51:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19911.007417 secs 2023-09-10T04:51:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:21 smithi170 ceph-mon[102832]: pgmap v10781: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 13 op/s 2023-09-10T04:51:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:21 smithi082 ceph-mon[98694]: pgmap v10781: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 13 op/s 2023-09-10T04:51:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:21 smithi098 ceph-mon[103124]: pgmap v10781: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 13 op/s 2023-09-10T04:51:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:23 smithi170 ceph-mon[102832]: pgmap v10782: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:51:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19914.231818 secs 2023-09-10T04:51:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19915.352972 secs 2023-09-10T04:51:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:23 smithi082 ceph-mon[98694]: pgmap v10782: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:51:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19914.231818 secs 2023-09-10T04:51:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19915.352972 secs 2023-09-10T04:51:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:23 smithi098 ceph-mon[103124]: pgmap v10782: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:51:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19914.231818 secs 2023-09-10T04:51:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19915.352972 secs 2023-09-10T04:51:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19916.007482 secs 2023-09-10T04:51:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19916.007482 secs 2023-09-10T04:51:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19916.007482 secs 2023-09-10T04:51:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:25 smithi170 ceph-mon[102832]: pgmap v10783: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:51:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:25 smithi082 ceph-mon[98694]: pgmap v10783: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:51:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:25 smithi098 ceph-mon[103124]: pgmap v10783: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:51:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:27 smithi170 ceph-mon[102832]: pgmap v10784: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 164 KiB/s wr, 15 op/s 2023-09-10T04:51:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:27 smithi082 ceph-mon[98694]: pgmap v10784: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 164 KiB/s wr, 15 op/s 2023-09-10T04:51:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:27 smithi098 ceph-mon[103124]: pgmap v10784: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 164 KiB/s wr, 15 op/s 2023-09-10T04:51:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19920.353031 secs 2023-09-10T04:51:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19920.353031 secs 2023-09-10T04:51:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19920.353031 secs 2023-09-10T04:51:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:29 smithi170 ceph-mon[102832]: pgmap v10785: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T04:51:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19921.007548 secs 2023-09-10T04:51:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:29 smithi082 ceph-mon[98694]: pgmap v10785: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T04:51:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19921.007548 secs 2023-09-10T04:51:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:29 smithi098 ceph-mon[103124]: pgmap v10785: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T04:51:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19921.007548 secs 2023-09-10T04:51:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:31 smithi170 ceph-mon[102832]: pgmap v10786: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T04:51:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:51:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:31 smithi082 ceph-mon[98694]: pgmap v10786: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T04:51:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:51:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:31 smithi098 ceph-mon[103124]: pgmap v10786: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 9 op/s 2023-09-10T04:51:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:51:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:51:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:51:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:51:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:51:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:51:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:51:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:51:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:51:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:51:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:33 smithi170 ceph-mon[102832]: pgmap v10787: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 215 KiB/s wr, 15 op/s 2023-09-10T04:51:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19925.353121 secs 2023-09-10T04:51:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:33 smithi082 ceph-mon[98694]: pgmap v10787: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 215 KiB/s wr, 15 op/s 2023-09-10T04:51:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19925.353121 secs 2023-09-10T04:51:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:33 smithi098 ceph-mon[103124]: pgmap v10787: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 215 KiB/s wr, 15 op/s 2023-09-10T04:51:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19925.353121 secs 2023-09-10T04:51:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19926.007610 secs 2023-09-10T04:51:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19926.007610 secs 2023-09-10T04:51:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19926.007610 secs 2023-09-10T04:51:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:35 smithi170 ceph-mon[102832]: pgmap v10788: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 215 KiB/s wr, 10 op/s 2023-09-10T04:51:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:35 smithi082 ceph-mon[98694]: pgmap v10788: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 215 KiB/s wr, 10 op/s 2023-09-10T04:51:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:35 smithi098 ceph-mon[103124]: pgmap v10788: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 215 KiB/s wr, 10 op/s 2023-09-10T04:51:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:37 smithi170 ceph-mon[102832]: pgmap v10789: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 336 KiB/s wr, 16 op/s 2023-09-10T04:51:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:37 smithi082 ceph-mon[98694]: pgmap v10789: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 336 KiB/s wr, 16 op/s 2023-09-10T04:51:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:37 smithi098 ceph-mon[103124]: pgmap v10789: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 336 KiB/s wr, 16 op/s 2023-09-10T04:51:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:51:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:51:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19930.353180 secs 2023-09-10T04:51:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:51:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:51:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19930.353180 secs 2023-09-10T04:51:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:51:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:51:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19930.353180 secs 2023-09-10T04:51:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:39 smithi170 ceph-mon[102832]: pgmap v10790: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 275 KiB/s wr, 11 op/s 2023-09-10T04:51:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19931.007676 secs 2023-09-10T04:51:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:39 smithi082 ceph-mon[98694]: pgmap v10790: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 275 KiB/s wr, 11 op/s 2023-09-10T04:51:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19931.007676 secs 2023-09-10T04:51:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:39 smithi098 ceph-mon[103124]: pgmap v10790: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 275 KiB/s wr, 11 op/s 2023-09-10T04:51:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19931.007676 secs 2023-09-10T04:51:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:51:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:51:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:51:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:51:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:51:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:51:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:41 smithi170 ceph-mon[102832]: pgmap v10791: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 275 KiB/s wr, 11 op/s 2023-09-10T04:51:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:41 smithi082 ceph-mon[98694]: pgmap v10791: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 275 KiB/s wr, 11 op/s 2023-09-10T04:51:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:41 smithi098 ceph-mon[103124]: pgmap v10791: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 275 KiB/s wr, 11 op/s 2023-09-10T04:51:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:43 smithi170 ceph-mon[102832]: pgmap v10792: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 395 KiB/s wr, 18 op/s 2023-09-10T04:51:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19935.353238 secs 2023-09-10T04:51:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:43 smithi082 ceph-mon[98694]: pgmap v10792: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 395 KiB/s wr, 18 op/s 2023-09-10T04:51:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19935.353238 secs 2023-09-10T04:51:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:43 smithi098 ceph-mon[103124]: pgmap v10792: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 395 KiB/s wr, 18 op/s 2023-09-10T04:51:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19935.353238 secs 2023-09-10T04:51:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19936.007738 secs 2023-09-10T04:51:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19936.007738 secs 2023-09-10T04:51:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19936.007738 secs 2023-09-10T04:51:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:45 smithi170 ceph-mon[102832]: pgmap v10793: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 241 KiB/s wr, 12 op/s 2023-09-10T04:51:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:45 smithi082 ceph-mon[98694]: pgmap v10793: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 241 KiB/s wr, 12 op/s 2023-09-10T04:51:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:45 smithi098 ceph-mon[103124]: pgmap v10793: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 241 KiB/s wr, 12 op/s 2023-09-10T04:51:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:47 smithi170 ceph-mon[102832]: pgmap v10794: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 374 KiB/s wr, 18 op/s 2023-09-10T04:51:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:51:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:47 smithi082 ceph-mon[98694]: pgmap v10794: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 374 KiB/s wr, 18 op/s 2023-09-10T04:51:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:51:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:47 smithi098 ceph-mon[103124]: pgmap v10794: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 374 KiB/s wr, 18 op/s 2023-09-10T04:51:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:51:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19940.353304 secs 2023-09-10T04:51:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19940.353304 secs 2023-09-10T04:51:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19940.353304 secs 2023-09-10T04:51:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:49 smithi170 ceph-mon[102832]: pgmap v10795: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 253 KiB/s wr, 12 op/s 2023-09-10T04:51:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:51:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:51:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19941.007804 secs 2023-09-10T04:51:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:49 smithi082 ceph-mon[98694]: pgmap v10795: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 253 KiB/s wr, 12 op/s 2023-09-10T04:51:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:51:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:51:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19941.007804 secs 2023-09-10T04:51:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:49 smithi098 ceph-mon[103124]: pgmap v10795: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 253 KiB/s wr, 12 op/s 2023-09-10T04:51:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:51:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:51:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19941.007804 secs 2023-09-10T04:51:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:51 smithi170 ceph-mon[102832]: pgmap v10796: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 253 KiB/s wr, 12 op/s 2023-09-10T04:51:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:51.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:51.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:51.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:51.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:51.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:51.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:51.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:51 smithi082 ceph-mon[98694]: pgmap v10796: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 253 KiB/s wr, 12 op/s 2023-09-10T04:51:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:51.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:51.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:51.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:51.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:51.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:51.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:51.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:51 smithi098 ceph-mon[103124]: pgmap v10796: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 253 KiB/s wr, 12 op/s 2023-09-10T04:51:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:51.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:51.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:51.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:51.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:51.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:51.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:51.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:51.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:52 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi082 to 6150k 2023-09-10T04:51:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:52 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi082 to 6297710: error parsing value: Value '6297710' is below minimum 939524096 2023-09-10T04:51:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:52 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T04:51:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:52 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T04:51:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:52.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:52.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:52.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:52.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:52.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:52.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:52 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T04:51:52.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:52 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T04:51:52.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:51:52.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:51:52.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:52.711 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:51:52.711 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:51:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:52 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi082 to 6150k 2023-09-10T04:51:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:52 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi082 to 6297710: error parsing value: Value '6297710' is below minimum 939524096 2023-09-10T04:51:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:52 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T04:51:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:52 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T04:51:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:52 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T04:51:52.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:52 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T04:51:52.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:51:52.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:51:52.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:52.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:51:52.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:51:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:52 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi082 to 6150k 2023-09-10T04:51:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:52 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi082 to 6297710: error parsing value: Value '6297710' is below minimum 939524096 2023-09-10T04:51:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:52 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T04:51:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:52 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T04:51:52.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:52.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:52.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:52.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:52.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:52.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T04:51:52.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:52 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T04:51:52.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:52 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T04:51:52.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:51:52.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:51:52.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:51:52.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:51:52.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:51:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:53 smithi170 ceph-mon[102832]: pgmap v10797: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 379 KiB/s wr, 18 op/s 2023-09-10T04:51:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19944.232219 secs 2023-09-10T04:51:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19945.353361 secs 2023-09-10T04:51:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:53 smithi082 ceph-mon[98694]: pgmap v10797: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 379 KiB/s wr, 18 op/s 2023-09-10T04:51:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19944.232219 secs 2023-09-10T04:51:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19945.353361 secs 2023-09-10T04:51:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:53 smithi098 ceph-mon[103124]: pgmap v10797: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 379 KiB/s wr, 18 op/s 2023-09-10T04:51:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19944.232219 secs 2023-09-10T04:51:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19945.353361 secs 2023-09-10T04:51:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19946.007864 secs 2023-09-10T04:51:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19946.007864 secs 2023-09-10T04:51:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19946.007864 secs 2023-09-10T04:51:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:55 smithi170 ceph-mon[102832]: pgmap v10798: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 259 KiB/s wr, 11 op/s 2023-09-10T04:51:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:55 smithi082 ceph-mon[98694]: pgmap v10798: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 259 KiB/s wr, 11 op/s 2023-09-10T04:51:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:55 smithi098 ceph-mon[103124]: pgmap v10798: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 259 KiB/s wr, 11 op/s 2023-09-10T04:51:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:51:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:51:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:51:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:51:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:51:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:51:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:51:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:51:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:51:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:57 smithi170 ceph-mon[102832]: pgmap v10799: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 327 KiB/s wr, 17 op/s 2023-09-10T04:51:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:57 smithi082 ceph-mon[98694]: pgmap v10799: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 327 KiB/s wr, 17 op/s 2023-09-10T04:51:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:57 smithi098 ceph-mon[103124]: pgmap v10799: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 327 KiB/s wr, 17 op/s 2023-09-10T04:51:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19949.232278 secs 2023-09-10T04:51:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19950.353418 secs 2023-09-10T04:51:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19949.232278 secs 2023-09-10T04:51:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19950.353418 secs 2023-09-10T04:51:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19949.232278 secs 2023-09-10T04:51:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19950.353418 secs 2023-09-10T04:51:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:59 smithi170 ceph-mon[102832]: pgmap v10800: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T04:51:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:51:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19951.007928 secs 2023-09-10T04:51:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:59 smithi082 ceph-mon[98694]: pgmap v10800: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T04:51:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:51:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19951.007928 secs 2023-09-10T04:52:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:59 smithi098 ceph-mon[103124]: pgmap v10800: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T04:52:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:51:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19951.007928 secs 2023-09-10T04:52:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:01 smithi170 ceph-mon[102832]: pgmap v10801: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T04:52:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:01 smithi082 ceph-mon[98694]: pgmap v10801: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T04:52:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:01 smithi098 ceph-mon[103124]: pgmap v10801: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T04:52:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:03 smithi170 ceph-mon[102832]: pgmap v10802: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 286 KiB/s wr, 17 op/s 2023-09-10T04:52:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19954.232342 secs 2023-09-10T04:52:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19955.353516 secs 2023-09-10T04:52:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:03 smithi082 ceph-mon[98694]: pgmap v10802: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 286 KiB/s wr, 17 op/s 2023-09-10T04:52:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19954.232342 secs 2023-09-10T04:52:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19955.353516 secs 2023-09-10T04:52:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:03 smithi098 ceph-mon[103124]: pgmap v10802: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 286 KiB/s wr, 17 op/s 2023-09-10T04:52:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19954.232342 secs 2023-09-10T04:52:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19955.353516 secs 2023-09-10T04:52:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19956.008015 secs 2023-09-10T04:52:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19956.008015 secs 2023-09-10T04:52:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19956.008015 secs 2023-09-10T04:52:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:05 smithi170 ceph-mon[102832]: pgmap v10803: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 161 KiB/s wr, 10 op/s 2023-09-10T04:52:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:05 smithi082 ceph-mon[98694]: pgmap v10803: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 161 KiB/s wr, 10 op/s 2023-09-10T04:52:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:05 smithi098 ceph-mon[103124]: pgmap v10803: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 161 KiB/s wr, 10 op/s 2023-09-10T04:52:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:07 smithi170 ceph-mon[102832]: pgmap v10804: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 255 KiB/s wr, 30 op/s 2023-09-10T04:52:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:07 smithi082 ceph-mon[98694]: pgmap v10804: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 255 KiB/s wr, 30 op/s 2023-09-10T04:52:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:07 smithi098 ceph-mon[103124]: pgmap v10804: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 255 KiB/s wr, 30 op/s 2023-09-10T04:52:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19959.232400 secs 2023-09-10T04:52:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:08 smithi170 ceph-mon[102832]: osdmap e743: 12 total, 12 up, 12 in 2023-09-10T04:52:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19960.353598 secs 2023-09-10T04:52:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19959.232400 secs 2023-09-10T04:52:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:08 smithi082 ceph-mon[98694]: osdmap e743: 12 total, 12 up, 12 in 2023-09-10T04:52:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19960.353598 secs 2023-09-10T04:52:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19959.232400 secs 2023-09-10T04:52:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:08 smithi098 ceph-mon[103124]: osdmap e743: 12 total, 12 up, 12 in 2023-09-10T04:52:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19960.353598 secs 2023-09-10T04:52:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:09 smithi170 ceph-mon[102832]: pgmap v10806: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 224 KiB/s wr, 29 op/s 2023-09-10T04:52:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19961.008107 secs 2023-09-10T04:52:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:09 smithi082 ceph-mon[98694]: pgmap v10806: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 224 KiB/s wr, 29 op/s 2023-09-10T04:52:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19961.008107 secs 2023-09-10T04:52:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:09 smithi098 ceph-mon[103124]: pgmap v10806: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 224 KiB/s wr, 29 op/s 2023-09-10T04:52:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19961.008107 secs 2023-09-10T04:52:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:11 smithi170 ceph-mon[102832]: pgmap v10807: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 224 KiB/s wr, 29 op/s 2023-09-10T04:52:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:11 smithi082 ceph-mon[98694]: pgmap v10807: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 224 KiB/s wr, 29 op/s 2023-09-10T04:52:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:11 smithi098 ceph-mon[103124]: pgmap v10807: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 224 KiB/s wr, 29 op/s 2023-09-10T04:52:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:13 smithi170 ceph-mon[102832]: pgmap v10808: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T04:52:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19964.232462 secs 2023-09-10T04:52:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19965.353687 secs 2023-09-10T04:52:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:13 smithi082 ceph-mon[98694]: pgmap v10808: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T04:52:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19964.232462 secs 2023-09-10T04:52:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19965.353687 secs 2023-09-10T04:52:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:13 smithi098 ceph-mon[103124]: pgmap v10808: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T04:52:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19964.232462 secs 2023-09-10T04:52:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19965.353687 secs 2023-09-10T04:52:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19966.008173 secs 2023-09-10T04:52:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:14 smithi170 ceph-mon[102832]: osdmap e744: 12 total, 12 up, 12 in 2023-09-10T04:52:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19966.008173 secs 2023-09-10T04:52:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:14 smithi082 ceph-mon[98694]: osdmap e744: 12 total, 12 up, 12 in 2023-09-10T04:52:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19966.008173 secs 2023-09-10T04:52:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:14 smithi098 ceph-mon[103124]: osdmap e744: 12 total, 12 up, 12 in 2023-09-10T04:52:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:15 smithi170 ceph-mon[102832]: pgmap v10810: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:52:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:15 smithi082 ceph-mon[98694]: pgmap v10810: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:52:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:15 smithi098 ceph-mon[103124]: pgmap v10810: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:52:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:17 smithi170 ceph-mon[102832]: pgmap v10811: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 14 op/s 2023-09-10T04:52:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:17 smithi082 ceph-mon[98694]: pgmap v10811: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 14 op/s 2023-09-10T04:52:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:17 smithi098 ceph-mon[103124]: pgmap v10811: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 14 op/s 2023-09-10T04:52:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19969.232604 secs 2023-09-10T04:52:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:52:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:52:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19970.353819 secs 2023-09-10T04:52:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19969.232604 secs 2023-09-10T04:52:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:52:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:52:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19970.353819 secs 2023-09-10T04:52:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19969.232604 secs 2023-09-10T04:52:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:52:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:52:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19970.353819 secs 2023-09-10T04:52:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:19 smithi170 ceph-mon[102832]: pgmap v10812: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T04:52:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19971.008241 secs 2023-09-10T04:52:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:19 smithi082 ceph-mon[98694]: pgmap v10812: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T04:52:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19971.008241 secs 2023-09-10T04:52:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:19 smithi098 ceph-mon[103124]: pgmap v10812: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T04:52:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19971.008241 secs 2023-09-10T04:52:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:52:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:52:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:52:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:52:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:52:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:52:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:21 smithi170 ceph-mon[102832]: pgmap v10813: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T04:52:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:21 smithi082 ceph-mon[98694]: pgmap v10813: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T04:52:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:21 smithi098 ceph-mon[103124]: pgmap v10813: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T04:52:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:23 smithi170 ceph-mon[102832]: pgmap v10814: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T04:52:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 19974.232663 secs 2023-09-10T04:52:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19975.353907 secs 2023-09-10T04:52:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:23 smithi082 ceph-mon[98694]: pgmap v10814: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T04:52:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 19974.232663 secs 2023-09-10T04:52:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19975.353907 secs 2023-09-10T04:52:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:23 smithi098 ceph-mon[103124]: pgmap v10814: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T04:52:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 19974.232663 secs 2023-09-10T04:52:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19975.353907 secs 2023-09-10T04:52:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19976.008303 secs 2023-09-10T04:52:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19976.008303 secs 2023-09-10T04:52:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19976.008303 secs 2023-09-10T04:52:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:25 smithi170 ceph-mon[102832]: pgmap v10815: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:52:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:25 smithi082 ceph-mon[98694]: pgmap v10815: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:52:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:25 smithi098 ceph-mon[103124]: pgmap v10815: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T04:52:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:27 smithi170 ceph-mon[102832]: pgmap v10816: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 164 KiB/s wr, 16 op/s 2023-09-10T04:52:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:27 smithi082 ceph-mon[98694]: pgmap v10816: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 164 KiB/s wr, 16 op/s 2023-09-10T04:52:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:27 smithi098 ceph-mon[103124]: pgmap v10816: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 164 KiB/s wr, 16 op/s 2023-09-10T04:52:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19980.353987 secs 2023-09-10T04:52:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19980.353987 secs 2023-09-10T04:52:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19980.353987 secs 2023-09-10T04:52:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:29 smithi170 ceph-mon[102832]: pgmap v10817: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 61 KiB/s wr, 10 op/s 2023-09-10T04:52:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19981.008366 secs 2023-09-10T04:52:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:29 smithi082 ceph-mon[98694]: pgmap v10817: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 61 KiB/s wr, 10 op/s 2023-09-10T04:52:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19981.008366 secs 2023-09-10T04:52:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:29 smithi098 ceph-mon[103124]: pgmap v10817: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 61 KiB/s wr, 10 op/s 2023-09-10T04:52:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19981.008366 secs 2023-09-10T04:52:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:31 smithi170 ceph-mon[102832]: pgmap v10818: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 61 KiB/s wr, 10 op/s 2023-09-10T04:52:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:52:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:31 smithi082 ceph-mon[98694]: pgmap v10818: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 61 KiB/s wr, 10 op/s 2023-09-10T04:52:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:52:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:31 smithi098 ceph-mon[103124]: pgmap v10818: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 61 KiB/s wr, 10 op/s 2023-09-10T04:52:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:52:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:52:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:52:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:52:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:52:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:52:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:52:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:52:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:52:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:52:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:33 smithi170 ceph-mon[102832]: pgmap v10819: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 204 KiB/s wr, 15 op/s 2023-09-10T04:52:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19985.354044 secs 2023-09-10T04:52:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:33 smithi082 ceph-mon[98694]: pgmap v10819: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 204 KiB/s wr, 15 op/s 2023-09-10T04:52:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19985.354044 secs 2023-09-10T04:52:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:33 smithi098 ceph-mon[103124]: pgmap v10819: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 204 KiB/s wr, 15 op/s 2023-09-10T04:52:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19985.354044 secs 2023-09-10T04:52:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19986.008427 secs 2023-09-10T04:52:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19986.008427 secs 2023-09-10T04:52:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19986.008427 secs 2023-09-10T04:52:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:35 smithi170 ceph-mon[102832]: pgmap v10820: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 204 KiB/s wr, 10 op/s 2023-09-10T04:52:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:35 smithi082 ceph-mon[98694]: pgmap v10820: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 204 KiB/s wr, 10 op/s 2023-09-10T04:52:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:35 smithi098 ceph-mon[103124]: pgmap v10820: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 204 KiB/s wr, 10 op/s 2023-09-10T04:52:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:37 smithi170 ceph-mon[102832]: pgmap v10821: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 338 KiB/s wr, 17 op/s 2023-09-10T04:52:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:52:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:52:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:37 smithi082 ceph-mon[98694]: pgmap v10821: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 338 KiB/s wr, 17 op/s 2023-09-10T04:52:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:52:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:52:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:37 smithi098 ceph-mon[103124]: pgmap v10821: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 338 KiB/s wr, 17 op/s 2023-09-10T04:52:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:52:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:52:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19990.354119 secs 2023-09-10T04:52:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19990.354119 secs 2023-09-10T04:52:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19990.354119 secs 2023-09-10T04:52:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:39 smithi170 ceph-mon[102832]: pgmap v10822: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 11 op/s 2023-09-10T04:52:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19991.008486 secs 2023-09-10T04:52:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:39 smithi082 ceph-mon[98694]: pgmap v10822: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 11 op/s 2023-09-10T04:52:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19991.008486 secs 2023-09-10T04:52:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:39 smithi098 ceph-mon[103124]: pgmap v10822: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 11 op/s 2023-09-10T04:52:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19991.008486 secs 2023-09-10T04:52:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:52:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:52:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:52:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:52:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:52:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:52:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:41 smithi170 ceph-mon[102832]: pgmap v10823: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 11 op/s 2023-09-10T04:52:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:41 smithi082 ceph-mon[98694]: pgmap v10823: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 11 op/s 2023-09-10T04:52:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:41 smithi098 ceph-mon[103124]: pgmap v10823: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 11 op/s 2023-09-10T04:52:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:43 smithi170 ceph-mon[102832]: pgmap v10824: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 398 KiB/s wr, 18 op/s 2023-09-10T04:52:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19995.354173 secs 2023-09-10T04:52:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:43 smithi082 ceph-mon[98694]: pgmap v10824: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 398 KiB/s wr, 18 op/s 2023-09-10T04:52:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19995.354173 secs 2023-09-10T04:52:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:43 smithi098 ceph-mon[103124]: pgmap v10824: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 398 KiB/s wr, 18 op/s 2023-09-10T04:52:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19995.354173 secs 2023-09-10T04:52:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 19996.008550 secs 2023-09-10T04:52:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 19996.008550 secs 2023-09-10T04:52:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 19996.008550 secs 2023-09-10T04:52:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:45 smithi170 ceph-mon[102832]: pgmap v10825: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 255 KiB/s wr, 12 op/s 2023-09-10T04:52:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:45 smithi082 ceph-mon[98694]: pgmap v10825: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 255 KiB/s wr, 12 op/s 2023-09-10T04:52:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:45 smithi098 ceph-mon[103124]: pgmap v10825: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 255 KiB/s wr, 12 op/s 2023-09-10T04:52:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:47 smithi170 ceph-mon[102832]: pgmap v10826: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 383 KiB/s wr, 18 op/s 2023-09-10T04:52:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:47 smithi082 ceph-mon[98694]: pgmap v10826: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 383 KiB/s wr, 18 op/s 2023-09-10T04:52:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:47 smithi098 ceph-mon[103124]: pgmap v10826: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 383 KiB/s wr, 18 op/s 2023-09-10T04:52:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20000.354232 secs 2023-09-10T04:52:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20000.354232 secs 2023-09-10T04:52:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20000.354232 secs 2023-09-10T04:52:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:49 smithi170 ceph-mon[102832]: pgmap v10827: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 248 KiB/s wr, 12 op/s 2023-09-10T04:52:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20001.008617 secs 2023-09-10T04:52:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:49 smithi082 ceph-mon[98694]: pgmap v10827: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 248 KiB/s wr, 12 op/s 2023-09-10T04:52:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20001.008617 secs 2023-09-10T04:52:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:49 smithi098 ceph-mon[103124]: pgmap v10827: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 248 KiB/s wr, 12 op/s 2023-09-10T04:52:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20001.008617 secs 2023-09-10T04:52:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:51 smithi170 ceph-mon[102832]: pgmap v10828: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 248 KiB/s wr, 12 op/s 2023-09-10T04:52:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:52:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:51 smithi082 ceph-mon[98694]: pgmap v10828: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 248 KiB/s wr, 12 op/s 2023-09-10T04:52:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:52:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:51 smithi098 ceph-mon[103124]: pgmap v10828: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 248 KiB/s wr, 12 op/s 2023-09-10T04:52:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:52:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:52:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:52:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:52:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:52:52.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:52:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:52:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:52:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:52:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:52:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:52:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:52:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:52:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:52:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:52:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:52:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:53 smithi170 ceph-mon[102832]: pgmap v10829: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 381 KiB/s wr, 19 op/s 2023-09-10T04:52:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20004.233060 secs 2023-09-10T04:52:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20005.354326 secs 2023-09-10T04:52:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:52:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:52:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:53 smithi082 ceph-mon[98694]: pgmap v10829: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 381 KiB/s wr, 19 op/s 2023-09-10T04:52:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20004.233060 secs 2023-09-10T04:52:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20005.354326 secs 2023-09-10T04:52:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:52:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:52:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:53 smithi098 ceph-mon[103124]: pgmap v10829: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 381 KiB/s wr, 19 op/s 2023-09-10T04:52:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20004.233060 secs 2023-09-10T04:52:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20005.354326 secs 2023-09-10T04:52:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:52:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:52:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20006.008680 secs 2023-09-10T04:52:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20006.008680 secs 2023-09-10T04:52:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20006.008680 secs 2023-09-10T04:52:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:55 smithi170 ceph-mon[102832]: pgmap v10830: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T04:52:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:55 smithi082 ceph-mon[98694]: pgmap v10830: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T04:52:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:55 smithi098 ceph-mon[103124]: pgmap v10830: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T04:52:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:52:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:52:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:52:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:52:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:52:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:52:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:52:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:52:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:52:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:57 smithi082 ceph-mon[98694]: pgmap v10831: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 328 KiB/s wr, 18 op/s 2023-09-10T04:52:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:57 smithi098 ceph-mon[103124]: pgmap v10831: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 328 KiB/s wr, 18 op/s 2023-09-10T04:52:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:57 smithi170 ceph-mon[102832]: pgmap v10831: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 328 KiB/s wr, 18 op/s 2023-09-10T04:52:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20009.233169 secs 2023-09-10T04:52:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20010.354401 secs 2023-09-10T04:52:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20009.233169 secs 2023-09-10T04:52:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20010.354401 secs 2023-09-10T04:52:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20009.233169 secs 2023-09-10T04:52:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20010.354401 secs 2023-09-10T04:52:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:59 smithi082 ceph-mon[98694]: pgmap v10832: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T04:52:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:52:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20011.008747 secs 2023-09-10T04:53:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:59 smithi098 ceph-mon[103124]: pgmap v10832: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T04:53:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:52:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20011.008747 secs 2023-09-10T04:53:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:59 smithi170 ceph-mon[102832]: pgmap v10832: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T04:53:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:52:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20011.008747 secs 2023-09-10T04:53:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:01 smithi082 ceph-mon[98694]: pgmap v10833: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T04:53:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:01 smithi098 ceph-mon[103124]: pgmap v10833: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T04:53:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:01 smithi170 ceph-mon[102832]: pgmap v10833: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T04:53:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:03 smithi082 ceph-mon[98694]: pgmap v10834: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 294 KiB/s wr, 17 op/s 2023-09-10T04:53:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20014.233249 secs 2023-09-10T04:53:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20015.354460 secs 2023-09-10T04:53:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:03 smithi098 ceph-mon[103124]: pgmap v10834: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 294 KiB/s wr, 17 op/s 2023-09-10T04:53:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20014.233249 secs 2023-09-10T04:53:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20015.354460 secs 2023-09-10T04:53:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:03 smithi170 ceph-mon[102832]: pgmap v10834: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 294 KiB/s wr, 17 op/s 2023-09-10T04:53:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20014.233249 secs 2023-09-10T04:53:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20015.354460 secs 2023-09-10T04:53:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20016.008814 secs 2023-09-10T04:53:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20016.008814 secs 2023-09-10T04:53:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20016.008814 secs 2023-09-10T04:53:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:05 smithi082 ceph-mon[98694]: pgmap v10835: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 161 KiB/s wr, 11 op/s 2023-09-10T04:53:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:05 smithi098 ceph-mon[103124]: pgmap v10835: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 161 KiB/s wr, 11 op/s 2023-09-10T04:53:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:05 smithi170 ceph-mon[102832]: pgmap v10835: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 161 KiB/s wr, 11 op/s 2023-09-10T04:53:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:07 smithi082 ceph-mon[98694]: pgmap v10836: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 256 KiB/s wr, 30 op/s 2023-09-10T04:53:08.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:07 smithi098 ceph-mon[103124]: pgmap v10836: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 256 KiB/s wr, 30 op/s 2023-09-10T04:53:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:07 smithi170 ceph-mon[102832]: pgmap v10836: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 256 KiB/s wr, 30 op/s 2023-09-10T04:53:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20019.233347 secs 2023-09-10T04:53:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:08 smithi098 ceph-mon[103124]: osdmap e745: 12 total, 12 up, 12 in 2023-09-10T04:53:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20020.354495 secs 2023-09-10T04:53:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20019.233347 secs 2023-09-10T04:53:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:08 smithi170 ceph-mon[102832]: osdmap e745: 12 total, 12 up, 12 in 2023-09-10T04:53:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20020.354495 secs 2023-09-10T04:53:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20019.233347 secs 2023-09-10T04:53:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:08 smithi082 ceph-mon[98694]: osdmap e745: 12 total, 12 up, 12 in 2023-09-10T04:53:09.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20020.354495 secs 2023-09-10T04:53:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:09 smithi098 ceph-mon[103124]: pgmap v10838: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 225 KiB/s wr, 30 op/s 2023-09-10T04:53:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20021.008878 secs 2023-09-10T04:53:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:09 smithi170 ceph-mon[102832]: pgmap v10838: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 225 KiB/s wr, 30 op/s 2023-09-10T04:53:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20021.008878 secs 2023-09-10T04:53:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:09 smithi082 ceph-mon[98694]: pgmap v10838: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 225 KiB/s wr, 30 op/s 2023-09-10T04:53:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20021.008878 secs 2023-09-10T04:53:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:11 smithi098 ceph-mon[103124]: pgmap v10839: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 225 KiB/s wr, 30 op/s 2023-09-10T04:53:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:11 smithi170 ceph-mon[102832]: pgmap v10839: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 225 KiB/s wr, 30 op/s 2023-09-10T04:53:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:11 smithi082 ceph-mon[98694]: pgmap v10839: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 225 KiB/s wr, 30 op/s 2023-09-10T04:53:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:13 smithi098 ceph-mon[103124]: pgmap v10840: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T04:53:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20024.233406 secs 2023-09-10T04:53:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20025.354555 secs 2023-09-10T04:53:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:13 smithi170 ceph-mon[102832]: pgmap v10840: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T04:53:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20024.233406 secs 2023-09-10T04:53:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20025.354555 secs 2023-09-10T04:53:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:13 smithi082 ceph-mon[98694]: pgmap v10840: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 114 KiB/s wr, 29 op/s 2023-09-10T04:53:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20024.233406 secs 2023-09-10T04:53:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20025.354555 secs 2023-09-10T04:53:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20026.008944 secs 2023-09-10T04:53:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:14 smithi098 ceph-mon[103124]: osdmap e746: 12 total, 12 up, 12 in 2023-09-10T04:53:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20026.008944 secs 2023-09-10T04:53:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:14 smithi170 ceph-mon[102832]: osdmap e746: 12 total, 12 up, 12 in 2023-09-10T04:53:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20026.008944 secs 2023-09-10T04:53:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:14 smithi082 ceph-mon[98694]: osdmap e746: 12 total, 12 up, 12 in 2023-09-10T04:53:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:15 smithi098 ceph-mon[103124]: pgmap v10842: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:53:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:15 smithi170 ceph-mon[102832]: pgmap v10842: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:53:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:15 smithi082 ceph-mon[98694]: pgmap v10842: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:53:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:17 smithi098 ceph-mon[103124]: pgmap v10843: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T04:53:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:17 smithi170 ceph-mon[102832]: pgmap v10843: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T04:53:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:17 smithi082 ceph-mon[98694]: pgmap v10843: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T04:53:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20029.233466 secs 2023-09-10T04:53:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20030.354639 secs 2023-09-10T04:53:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20029.233466 secs 2023-09-10T04:53:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20030.354639 secs 2023-09-10T04:53:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20029.233466 secs 2023-09-10T04:53:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20030.354639 secs 2023-09-10T04:53:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:19 smithi098 ceph-mon[103124]: pgmap v10844: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T04:53:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20031.009008 secs 2023-09-10T04:53:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:19 smithi170 ceph-mon[102832]: pgmap v10844: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T04:53:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20031.009008 secs 2023-09-10T04:53:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:19 smithi082 ceph-mon[98694]: pgmap v10844: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T04:53:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20031.009008 secs 2023-09-10T04:53:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:21 smithi098 ceph-mon[103124]: pgmap v10845: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T04:53:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:21 smithi170 ceph-mon[102832]: pgmap v10845: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T04:53:22.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:21 smithi082 ceph-mon[98694]: pgmap v10845: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T04:53:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:23 smithi098 ceph-mon[103124]: pgmap v10846: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T04:53:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20034.233529 secs 2023-09-10T04:53:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20035.354697 secs 2023-09-10T04:53:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:23 smithi170 ceph-mon[102832]: pgmap v10846: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T04:53:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20034.233529 secs 2023-09-10T04:53:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20035.354697 secs 2023-09-10T04:53:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:23 smithi082 ceph-mon[98694]: pgmap v10846: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T04:53:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20034.233529 secs 2023-09-10T04:53:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20035.354697 secs 2023-09-10T04:53:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20036.009083 secs 2023-09-10T04:53:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20036.009083 secs 2023-09-10T04:53:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20036.009083 secs 2023-09-10T04:53:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:25 smithi098 ceph-mon[103124]: pgmap v10847: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T04:53:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:25 smithi170 ceph-mon[102832]: pgmap v10847: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T04:53:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:25 smithi082 ceph-mon[98694]: pgmap v10847: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T04:53:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:27 smithi098 ceph-mon[103124]: pgmap v10848: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 165 KiB/s wr, 14 op/s 2023-09-10T04:53:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:27 smithi170 ceph-mon[102832]: pgmap v10848: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 165 KiB/s wr, 14 op/s 2023-09-10T04:53:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:27 smithi082 ceph-mon[98694]: pgmap v10848: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 165 KiB/s wr, 14 op/s 2023-09-10T04:53:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20039.233590 secs 2023-09-10T04:53:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20040.354754 secs 2023-09-10T04:53:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20039.233590 secs 2023-09-10T04:53:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20040.354754 secs 2023-09-10T04:53:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20039.233590 secs 2023-09-10T04:53:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20040.354754 secs 2023-09-10T04:53:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:29 smithi098 ceph-mon[103124]: pgmap v10849: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 9 op/s 2023-09-10T04:53:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20041.009151 secs 2023-09-10T04:53:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:29 smithi170 ceph-mon[102832]: pgmap v10849: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 9 op/s 2023-09-10T04:53:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20041.009151 secs 2023-09-10T04:53:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:29 smithi082 ceph-mon[98694]: pgmap v10849: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 9 op/s 2023-09-10T04:53:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20041.009151 secs 2023-09-10T04:53:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:31 smithi098 ceph-mon[103124]: pgmap v10850: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 9 op/s 2023-09-10T04:53:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:31 smithi170 ceph-mon[102832]: pgmap v10850: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 9 op/s 2023-09-10T04:53:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:31 smithi082 ceph-mon[98694]: pgmap v10850: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 62 KiB/s wr, 9 op/s 2023-09-10T04:53:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:53:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:53:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:53:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:53:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:53:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:53:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:33 smithi170 ceph-mon[102832]: pgmap v10851: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 123 KiB/s wr, 14 op/s 2023-09-10T04:53:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:53:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:53:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20045.354814 secs 2023-09-10T04:53:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:33 smithi082 ceph-mon[98694]: pgmap v10851: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 123 KiB/s wr, 14 op/s 2023-09-10T04:53:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:53:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:53:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20045.354814 secs 2023-09-10T04:53:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:33 smithi098 ceph-mon[103124]: pgmap v10851: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 123 KiB/s wr, 14 op/s 2023-09-10T04:53:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:53:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:53:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20045.354814 secs 2023-09-10T04:53:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20046.009215 secs 2023-09-10T04:53:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20046.009215 secs 2023-09-10T04:53:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20046.009215 secs 2023-09-10T04:53:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:35 smithi170 ceph-mon[102832]: pgmap v10852: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T04:53:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:53:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:35 smithi082 ceph-mon[98694]: pgmap v10852: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T04:53:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:53:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:35 smithi098 ceph-mon[103124]: pgmap v10852: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T04:53:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:53:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:53:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:53:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:53:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:53:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:53:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:53:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:53:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:53:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:53:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:37 smithi170 ceph-mon[102832]: pgmap v10853: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 339 KiB/s wr, 16 op/s 2023-09-10T04:53:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:37 smithi082 ceph-mon[98694]: pgmap v10853: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 339 KiB/s wr, 16 op/s 2023-09-10T04:53:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:37 smithi098 ceph-mon[103124]: pgmap v10853: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 339 KiB/s wr, 16 op/s 2023-09-10T04:53:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20050.354874 secs 2023-09-10T04:53:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20050.354874 secs 2023-09-10T04:53:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20050.354874 secs 2023-09-10T04:53:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:39 smithi170 ceph-mon[102832]: pgmap v10854: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 277 KiB/s wr, 11 op/s 2023-09-10T04:53:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20051.009282 secs 2023-09-10T04:53:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:39 smithi082 ceph-mon[98694]: pgmap v10854: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 277 KiB/s wr, 11 op/s 2023-09-10T04:53:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20051.009282 secs 2023-09-10T04:53:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:39 smithi098 ceph-mon[103124]: pgmap v10854: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 277 KiB/s wr, 11 op/s 2023-09-10T04:53:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20051.009282 secs 2023-09-10T04:53:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:53:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:53:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:53:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:53:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:53:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:53:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:41 smithi170 ceph-mon[102832]: pgmap v10855: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 277 KiB/s wr, 11 op/s 2023-09-10T04:53:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:41 smithi082 ceph-mon[98694]: pgmap v10855: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 277 KiB/s wr, 11 op/s 2023-09-10T04:53:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:41 smithi098 ceph-mon[103124]: pgmap v10855: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 277 KiB/s wr, 11 op/s 2023-09-10T04:53:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:43 smithi170 ceph-mon[102832]: pgmap v10856: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 369 KiB/s wr, 18 op/s 2023-09-10T04:53:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20055.354959 secs 2023-09-10T04:53:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:43 smithi082 ceph-mon[98694]: pgmap v10856: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 369 KiB/s wr, 18 op/s 2023-09-10T04:53:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20055.354959 secs 2023-09-10T04:53:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:43 smithi098 ceph-mon[103124]: pgmap v10856: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 369 KiB/s wr, 18 op/s 2023-09-10T04:53:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20055.354959 secs 2023-09-10T04:53:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20056.009345 secs 2023-09-10T04:53:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20056.009345 secs 2023-09-10T04:53:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20056.009345 secs 2023-09-10T04:53:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:45 smithi170 ceph-mon[102832]: pgmap v10857: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 308 KiB/s wr, 13 op/s 2023-09-10T04:53:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:53:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:53:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:45 smithi082 ceph-mon[98694]: pgmap v10857: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 308 KiB/s wr, 13 op/s 2023-09-10T04:53:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:53:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:53:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:45 smithi098 ceph-mon[103124]: pgmap v10857: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 308 KiB/s wr, 13 op/s 2023-09-10T04:53:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:53:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:53:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:47 smithi170 ceph-mon[102832]: pgmap v10858: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 532 KiB/s wr, 19 op/s 2023-09-10T04:53:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:47 smithi082 ceph-mon[98694]: pgmap v10858: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 532 KiB/s wr, 19 op/s 2023-09-10T04:53:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:47 smithi098 ceph-mon[103124]: pgmap v10858: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 532 KiB/s wr, 19 op/s 2023-09-10T04:53:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20060.355017 secs 2023-09-10T04:53:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20060.355017 secs 2023-09-10T04:53:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20060.355017 secs 2023-09-10T04:53:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:49 smithi170 ceph-mon[102832]: pgmap v10859: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 316 KiB/s wr, 13 op/s 2023-09-10T04:53:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20061.009410 secs 2023-09-10T04:53:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:49 smithi082 ceph-mon[98694]: pgmap v10859: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 316 KiB/s wr, 13 op/s 2023-09-10T04:53:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20061.009410 secs 2023-09-10T04:53:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:49 smithi098 ceph-mon[103124]: pgmap v10859: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 316 KiB/s wr, 13 op/s 2023-09-10T04:53:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20061.009410 secs 2023-09-10T04:53:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:51 smithi170 ceph-mon[102832]: pgmap v10860: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 316 KiB/s wr, 13 op/s 2023-09-10T04:53:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:51 smithi082 ceph-mon[98694]: pgmap v10860: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 316 KiB/s wr, 13 op/s 2023-09-10T04:53:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:51 smithi098 ceph-mon[103124]: pgmap v10860: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 316 KiB/s wr, 13 op/s 2023-09-10T04:53:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:53:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:53:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:53:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:53:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:53:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:53:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:53:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:53:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:53:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:53 smithi170 ceph-mon[102832]: pgmap v10861: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 353 KiB/s wr, 19 op/s 2023-09-10T04:53:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20064.234026 secs 2023-09-10T04:53:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20065.355103 secs 2023-09-10T04:53:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:53:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:53:54.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:53:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:53 smithi082 ceph-mon[98694]: pgmap v10861: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 353 KiB/s wr, 19 op/s 2023-09-10T04:53:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20064.234026 secs 2023-09-10T04:53:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20065.355103 secs 2023-09-10T04:53:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:53:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:53:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:53:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:53 smithi098 ceph-mon[103124]: pgmap v10861: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 353 KiB/s wr, 19 op/s 2023-09-10T04:53:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20064.234026 secs 2023-09-10T04:53:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20065.355103 secs 2023-09-10T04:53:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:53:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:53:54.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:53:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20066.009474 secs 2023-09-10T04:53:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20066.009474 secs 2023-09-10T04:53:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20066.009474 secs 2023-09-10T04:53:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:55 smithi170 ceph-mon[102832]: pgmap v10862: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 261 KiB/s wr, 12 op/s 2023-09-10T04:53:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:55 smithi082 ceph-mon[98694]: pgmap v10862: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 261 KiB/s wr, 12 op/s 2023-09-10T04:53:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:55 smithi098 ceph-mon[103124]: pgmap v10862: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 261 KiB/s wr, 12 op/s 2023-09-10T04:53:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:53:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:53:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:53:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:53:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:53:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:53:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:53:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:53:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:53:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:57 smithi170 ceph-mon[102832]: pgmap v10863: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 408 KiB/s wr, 18 op/s 2023-09-10T04:53:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:53:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:53:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:57 smithi082 ceph-mon[98694]: pgmap v10863: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 408 KiB/s wr, 18 op/s 2023-09-10T04:53:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:53:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:53:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:57 smithi098 ceph-mon[103124]: pgmap v10863: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 408 KiB/s wr, 18 op/s 2023-09-10T04:53:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:53:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:53:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20069.234088 secs 2023-09-10T04:53:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20070.355163 secs 2023-09-10T04:53:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20069.234088 secs 2023-09-10T04:53:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20070.355163 secs 2023-09-10T04:53:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20069.234088 secs 2023-09-10T04:53:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20070.355163 secs 2023-09-10T04:54:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:59 smithi170 ceph-mon[102832]: pgmap v10864: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 184 KiB/s wr, 11 op/s 2023-09-10T04:54:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:53:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20071.009538 secs 2023-09-10T04:54:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:59 smithi082 ceph-mon[98694]: pgmap v10864: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 184 KiB/s wr, 11 op/s 2023-09-10T04:54:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:53:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20071.009538 secs 2023-09-10T04:54:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:59 smithi098 ceph-mon[103124]: pgmap v10864: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 184 KiB/s wr, 11 op/s 2023-09-10T04:54:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:53:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20071.009538 secs 2023-09-10T04:54:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:54:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:54:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:54:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:54:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:54:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:54:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:01 smithi170 ceph-mon[102832]: pgmap v10865: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 184 KiB/s wr, 11 op/s 2023-09-10T04:54:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:01 smithi082 ceph-mon[98694]: pgmap v10865: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 184 KiB/s wr, 11 op/s 2023-09-10T04:54:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:01 smithi098 ceph-mon[103124]: pgmap v10865: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 184 KiB/s wr, 11 op/s 2023-09-10T04:54:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:03 smithi170 ceph-mon[102832]: pgmap v10866: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 309 KiB/s wr, 17 op/s 2023-09-10T04:54:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20074.234172 secs 2023-09-10T04:54:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20075.355219 secs 2023-09-10T04:54:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:03 smithi082 ceph-mon[98694]: pgmap v10866: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 309 KiB/s wr, 17 op/s 2023-09-10T04:54:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20074.234172 secs 2023-09-10T04:54:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20075.355219 secs 2023-09-10T04:54:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:03 smithi098 ceph-mon[103124]: pgmap v10866: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 309 KiB/s wr, 17 op/s 2023-09-10T04:54:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20074.234172 secs 2023-09-10T04:54:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20075.355219 secs 2023-09-10T04:54:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20076.009601 secs 2023-09-10T04:54:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20076.009601 secs 2023-09-10T04:54:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20076.009601 secs 2023-09-10T04:54:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:05 smithi170 ceph-mon[102832]: pgmap v10867: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T04:54:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:05 smithi082 ceph-mon[98694]: pgmap v10867: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T04:54:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:05 smithi098 ceph-mon[103124]: pgmap v10867: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T04:54:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:07 smithi170 ceph-mon[102832]: pgmap v10868: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 318 KiB/s wr, 31 op/s 2023-09-10T04:54:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:07 smithi082 ceph-mon[98694]: pgmap v10868: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 318 KiB/s wr, 31 op/s 2023-09-10T04:54:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:07 smithi098 ceph-mon[103124]: pgmap v10868: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 318 KiB/s wr, 31 op/s 2023-09-10T04:54:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20079.234282 secs 2023-09-10T04:54:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:08 smithi170 ceph-mon[102832]: osdmap e747: 12 total, 12 up, 12 in 2023-09-10T04:54:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20080.355278 secs 2023-09-10T04:54:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20079.234282 secs 2023-09-10T04:54:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:08 smithi082 ceph-mon[98694]: osdmap e747: 12 total, 12 up, 12 in 2023-09-10T04:54:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20080.355278 secs 2023-09-10T04:54:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20079.234282 secs 2023-09-10T04:54:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:08 smithi098 ceph-mon[103124]: osdmap e747: 12 total, 12 up, 12 in 2023-09-10T04:54:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20080.355278 secs 2023-09-10T04:54:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:09 smithi170 ceph-mon[102832]: pgmap v10870: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 205 KiB/s wr, 30 op/s 2023-09-10T04:54:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20081.009665 secs 2023-09-10T04:54:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:09 smithi082 ceph-mon[98694]: pgmap v10870: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 205 KiB/s wr, 30 op/s 2023-09-10T04:54:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20081.009665 secs 2023-09-10T04:54:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:09 smithi098 ceph-mon[103124]: pgmap v10870: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 205 KiB/s wr, 30 op/s 2023-09-10T04:54:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20081.009665 secs 2023-09-10T04:54:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:11 smithi170 ceph-mon[102832]: pgmap v10871: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 205 KiB/s wr, 30 op/s 2023-09-10T04:54:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:11 smithi082 ceph-mon[98694]: pgmap v10871: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 205 KiB/s wr, 30 op/s 2023-09-10T04:54:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:11 smithi098 ceph-mon[103124]: pgmap v10871: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 205 KiB/s wr, 30 op/s 2023-09-10T04:54:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:13 smithi170 ceph-mon[102832]: pgmap v10872: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 74 KiB/s wr, 30 op/s 2023-09-10T04:54:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20084.234374 secs 2023-09-10T04:54:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20085.355337 secs 2023-09-10T04:54:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20086.009729 secs 2023-09-10T04:54:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:13 smithi082 ceph-mon[98694]: pgmap v10872: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 74 KiB/s wr, 30 op/s 2023-09-10T04:54:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20084.234374 secs 2023-09-10T04:54:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20085.355337 secs 2023-09-10T04:54:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20086.009729 secs 2023-09-10T04:54:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:13 smithi098 ceph-mon[103124]: pgmap v10872: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 74 KiB/s wr, 30 op/s 2023-09-10T04:54:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20084.234374 secs 2023-09-10T04:54:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20085.355337 secs 2023-09-10T04:54:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20086.009729 secs 2023-09-10T04:54:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:15 smithi170 ceph-mon[102832]: osdmap e748: 12 total, 12 up, 12 in 2023-09-10T04:54:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:15 smithi170 ceph-mon[102832]: pgmap v10874: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 8 op/s 2023-09-10T04:54:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:15 smithi082 ceph-mon[98694]: osdmap e748: 12 total, 12 up, 12 in 2023-09-10T04:54:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:15 smithi082 ceph-mon[98694]: pgmap v10874: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 8 op/s 2023-09-10T04:54:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:15 smithi098 ceph-mon[103124]: osdmap e748: 12 total, 12 up, 12 in 2023-09-10T04:54:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:15 smithi098 ceph-mon[103124]: pgmap v10874: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 8 op/s 2023-09-10T04:54:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:17 smithi170 ceph-mon[102832]: pgmap v10875: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 114 KiB/s wr, 14 op/s 2023-09-10T04:54:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:17 smithi082 ceph-mon[98694]: pgmap v10875: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 114 KiB/s wr, 14 op/s 2023-09-10T04:54:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:17 smithi098 ceph-mon[103124]: pgmap v10875: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 114 KiB/s wr, 14 op/s 2023-09-10T04:54:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20089.234448 secs 2023-09-10T04:54:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20090.355396 secs 2023-09-10T04:54:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20089.234448 secs 2023-09-10T04:54:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20090.355396 secs 2023-09-10T04:54:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20089.234448 secs 2023-09-10T04:54:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20090.355396 secs 2023-09-10T04:54:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:19 smithi170 ceph-mon[102832]: pgmap v10876: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T04:54:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20091.009795 secs 2023-09-10T04:54:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:19 smithi082 ceph-mon[98694]: pgmap v10876: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T04:54:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20091.009795 secs 2023-09-10T04:54:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:19 smithi098 ceph-mon[103124]: pgmap v10876: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T04:54:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20091.009795 secs 2023-09-10T04:54:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:21 smithi170 ceph-mon[102832]: pgmap v10877: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T04:54:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:21 smithi082 ceph-mon[98694]: pgmap v10877: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T04:54:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:21 smithi098 ceph-mon[103124]: pgmap v10877: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T04:54:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:23 smithi170 ceph-mon[102832]: pgmap v10878: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T04:54:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20094.234508 secs 2023-09-10T04:54:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20095.355468 secs 2023-09-10T04:54:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:23 smithi082 ceph-mon[98694]: pgmap v10878: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T04:54:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20094.234508 secs 2023-09-10T04:54:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20095.355468 secs 2023-09-10T04:54:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:23 smithi098 ceph-mon[103124]: pgmap v10878: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T04:54:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20094.234508 secs 2023-09-10T04:54:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20095.355468 secs 2023-09-10T04:54:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20096.009857 secs 2023-09-10T04:54:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20096.009857 secs 2023-09-10T04:54:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20096.009857 secs 2023-09-10T04:54:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:25 smithi170 ceph-mon[102832]: pgmap v10879: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T04:54:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:25 smithi082 ceph-mon[98694]: pgmap v10879: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T04:54:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:25 smithi098 ceph-mon[103124]: pgmap v10879: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T04:54:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:27 smithi170 ceph-mon[102832]: pgmap v10880: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 128 KiB/s wr, 14 op/s 2023-09-10T04:54:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:27 smithi082 ceph-mon[98694]: pgmap v10880: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 128 KiB/s wr, 14 op/s 2023-09-10T04:54:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:27 smithi098 ceph-mon[103124]: pgmap v10880: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 128 KiB/s wr, 14 op/s 2023-09-10T04:54:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20099.234568 secs 2023-09-10T04:54:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20100.355494 secs 2023-09-10T04:54:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20099.234568 secs 2023-09-10T04:54:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20100.355494 secs 2023-09-10T04:54:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20099.234568 secs 2023-09-10T04:54:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20100.355494 secs 2023-09-10T04:54:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:29 smithi170 ceph-mon[102832]: pgmap v10881: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T04:54:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20101.009921 secs 2023-09-10T04:54:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:29 smithi082 ceph-mon[98694]: pgmap v10881: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T04:54:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20101.009921 secs 2023-09-10T04:54:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:29 smithi098 ceph-mon[103124]: pgmap v10881: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T04:54:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20101.009921 secs 2023-09-10T04:54:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:31 smithi170 ceph-mon[102832]: pgmap v10882: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T04:54:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:31 smithi082 ceph-mon[98694]: pgmap v10882: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T04:54:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:31 smithi098 ceph-mon[103124]: pgmap v10882: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T04:54:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:33 smithi170 ceph-mon[102832]: pgmap v10883: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 129 KiB/s wr, 14 op/s 2023-09-10T04:54:33.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20104.234628 secs 2023-09-10T04:54:33.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20105.355553 secs 2023-09-10T04:54:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:33 smithi082 ceph-mon[98694]: pgmap v10883: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 129 KiB/s wr, 14 op/s 2023-09-10T04:54:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20104.234628 secs 2023-09-10T04:54:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20105.355553 secs 2023-09-10T04:54:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:33 smithi098 ceph-mon[103124]: pgmap v10883: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 129 KiB/s wr, 14 op/s 2023-09-10T04:54:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20104.234628 secs 2023-09-10T04:54:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20105.355553 secs 2023-09-10T04:54:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20106.009987 secs 2023-09-10T04:54:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20106.009987 secs 2023-09-10T04:54:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20106.009987 secs 2023-09-10T04:54:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:35 smithi170 ceph-mon[102832]: pgmap v10884: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 129 KiB/s wr, 9 op/s 2023-09-10T04:54:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:35 smithi082 ceph-mon[98694]: pgmap v10884: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 129 KiB/s wr, 9 op/s 2023-09-10T04:54:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:35 smithi098 ceph-mon[103124]: pgmap v10884: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 129 KiB/s wr, 9 op/s 2023-09-10T04:54:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:37 smithi170 ceph-mon[102832]: pgmap v10885: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 261 KiB/s wr, 15 op/s 2023-09-10T04:54:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:37 smithi082 ceph-mon[98694]: pgmap v10885: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 261 KiB/s wr, 15 op/s 2023-09-10T04:54:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:37 smithi098 ceph-mon[103124]: pgmap v10885: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 261 KiB/s wr, 15 op/s 2023-09-10T04:54:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:54:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:54:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20109.234687 secs 2023-09-10T04:54:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20110.355640 secs 2023-09-10T04:54:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:54:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:54:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20109.234687 secs 2023-09-10T04:54:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20110.355640 secs 2023-09-10T04:54:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:54:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:54:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20109.234687 secs 2023-09-10T04:54:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20110.355640 secs 2023-09-10T04:54:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:39 smithi170 ceph-mon[102832]: pgmap v10886: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 197 KiB/s wr, 10 op/s 2023-09-10T04:54:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20111.010053 secs 2023-09-10T04:54:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:39 smithi082 ceph-mon[98694]: pgmap v10886: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 197 KiB/s wr, 10 op/s 2023-09-10T04:54:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20111.010053 secs 2023-09-10T04:54:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:39 smithi098 ceph-mon[103124]: pgmap v10886: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 197 KiB/s wr, 10 op/s 2023-09-10T04:54:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20111.010053 secs 2023-09-10T04:54:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:54:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:54:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:54:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:54:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:54:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:54:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:41 smithi170 ceph-mon[102832]: pgmap v10887: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 197 KiB/s wr, 10 op/s 2023-09-10T04:54:41.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:41 smithi082 ceph-mon[98694]: pgmap v10887: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 197 KiB/s wr, 10 op/s 2023-09-10T04:54:41.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:41 smithi098 ceph-mon[103124]: pgmap v10887: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 197 KiB/s wr, 10 op/s 2023-09-10T04:54:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:43 smithi170 ceph-mon[102832]: pgmap v10888: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 225 KiB/s wr, 17 op/s 2023-09-10T04:54:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:54:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20115.355699 secs 2023-09-10T04:54:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:43 smithi082 ceph-mon[98694]: pgmap v10888: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 225 KiB/s wr, 17 op/s 2023-09-10T04:54:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:54:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20115.355699 secs 2023-09-10T04:54:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:43 smithi098 ceph-mon[103124]: pgmap v10888: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 225 KiB/s wr, 17 op/s 2023-09-10T04:54:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:54:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20115.355699 secs 2023-09-10T04:54:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:44 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:54:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:54:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:54:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20116.010114 secs 2023-09-10T04:54:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:44 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:54:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:54:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:54:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20116.010114 secs 2023-09-10T04:54:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:44 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:54:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:54:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:54:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20116.010114 secs 2023-09-10T04:54:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:45 smithi170 ceph-mon[102832]: pgmap v10889: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T04:54:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:45 smithi082 ceph-mon[98694]: pgmap v10889: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T04:54:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:45 smithi098 ceph-mon[103124]: pgmap v10889: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T04:54:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:47 smithi170 ceph-mon[102832]: pgmap v10890: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 389 KiB/s wr, 19 op/s 2023-09-10T04:54:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:47 smithi082 ceph-mon[98694]: pgmap v10890: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 389 KiB/s wr, 19 op/s 2023-09-10T04:54:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:47 smithi098 ceph-mon[103124]: pgmap v10890: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 389 KiB/s wr, 19 op/s 2023-09-10T04:54:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20120.355757 secs 2023-09-10T04:54:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20120.355757 secs 2023-09-10T04:54:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20120.355757 secs 2023-09-10T04:54:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:49 smithi170 ceph-mon[102832]: pgmap v10891: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 257 KiB/s wr, 13 op/s 2023-09-10T04:54:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20121.010183 secs 2023-09-10T04:54:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:49 smithi082 ceph-mon[98694]: pgmap v10891: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 257 KiB/s wr, 13 op/s 2023-09-10T04:54:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20121.010183 secs 2023-09-10T04:54:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:49 smithi098 ceph-mon[103124]: pgmap v10891: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 257 KiB/s wr, 13 op/s 2023-09-10T04:54:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20121.010183 secs 2023-09-10T04:54:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:51 smithi170 ceph-mon[102832]: pgmap v10892: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 257 KiB/s wr, 13 op/s 2023-09-10T04:54:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:51 smithi082 ceph-mon[98694]: pgmap v10892: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 257 KiB/s wr, 13 op/s 2023-09-10T04:54:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:51 smithi098 ceph-mon[103124]: pgmap v10892: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 257 KiB/s wr, 13 op/s 2023-09-10T04:54:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:53 smithi170 ceph-mon[102832]: pgmap v10893: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 276 KiB/s wr, 19 op/s 2023-09-10T04:54:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20125.355819 secs 2023-09-10T04:54:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:53 smithi082 ceph-mon[98694]: pgmap v10893: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 276 KiB/s wr, 19 op/s 2023-09-10T04:54:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20125.355819 secs 2023-09-10T04:54:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:53 smithi098 ceph-mon[103124]: pgmap v10893: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 276 KiB/s wr, 19 op/s 2023-09-10T04:54:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20125.355819 secs 2023-09-10T04:54:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:54:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:54:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:54:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20126.010246 secs 2023-09-10T04:54:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:54:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:54:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:54:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:54:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:54:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:54:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:54:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20126.010246 secs 2023-09-10T04:54:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:54:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:54:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:54:54.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:54:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:54:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:54:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:54:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20126.010246 secs 2023-09-10T04:54:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:54:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:54:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:54:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:54:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:55 smithi170 ceph-mon[102832]: pgmap v10894: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 248 KiB/s wr, 12 op/s 2023-09-10T04:54:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:54:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:54:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:54:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:55 smithi082 ceph-mon[98694]: pgmap v10894: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 248 KiB/s wr, 12 op/s 2023-09-10T04:54:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:54:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:54:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:54:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:55 smithi098 ceph-mon[103124]: pgmap v10894: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 248 KiB/s wr, 12 op/s 2023-09-10T04:54:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:54:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:54:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:54:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:54:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:54:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:54:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:54:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:54:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:54:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:57 smithi170 ceph-mon[102832]: pgmap v10895: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 616 KiB/s wr, 19 op/s 2023-09-10T04:54:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:57 smithi082 ceph-mon[98694]: pgmap v10895: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 616 KiB/s wr, 19 op/s 2023-09-10T04:54:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:57 smithi098 ceph-mon[103124]: pgmap v10895: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 616 KiB/s wr, 19 op/s 2023-09-10T04:54:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:54:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:54:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20129.235004 secs 2023-09-10T04:54:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20130.355954 secs 2023-09-10T04:54:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:54:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:54:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20129.235004 secs 2023-09-10T04:54:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20130.355954 secs 2023-09-10T04:54:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:54:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:54:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20129.235004 secs 2023-09-10T04:54:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20130.355954 secs 2023-09-10T04:54:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:59 smithi170 ceph-mon[102832]: pgmap v10896: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 388 KiB/s wr, 12 op/s 2023-09-10T04:54:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:54:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20131.010334 secs 2023-09-10T04:54:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:59 smithi082 ceph-mon[98694]: pgmap v10896: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 388 KiB/s wr, 12 op/s 2023-09-10T04:54:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:54:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20131.010334 secs 2023-09-10T04:55:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:59 smithi098 ceph-mon[103124]: pgmap v10896: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 388 KiB/s wr, 12 op/s 2023-09-10T04:55:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:54:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20131.010334 secs 2023-09-10T04:55:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:55:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:55:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:55:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:55:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:55:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:55:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:55:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:55:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:55:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:01 smithi170 ceph-mon[102832]: pgmap v10897: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 388 KiB/s wr, 12 op/s 2023-09-10T04:55:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:01 smithi082 ceph-mon[98694]: pgmap v10897: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 388 KiB/s wr, 12 op/s 2023-09-10T04:55:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:01 smithi098 ceph-mon[103124]: pgmap v10897: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 388 KiB/s wr, 12 op/s 2023-09-10T04:55:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:03 smithi170 ceph-mon[102832]: pgmap v10898: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 524 KiB/s wr, 19 op/s 2023-09-10T04:55:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20134.235067 secs 2023-09-10T04:55:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20135.356011 secs 2023-09-10T04:55:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:03 smithi082 ceph-mon[98694]: pgmap v10898: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 524 KiB/s wr, 19 op/s 2023-09-10T04:55:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20134.235067 secs 2023-09-10T04:55:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20135.356011 secs 2023-09-10T04:55:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:03 smithi098 ceph-mon[103124]: pgmap v10898: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 524 KiB/s wr, 19 op/s 2023-09-10T04:55:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20134.235067 secs 2023-09-10T04:55:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20135.356011 secs 2023-09-10T04:55:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20136.010397 secs 2023-09-10T04:55:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20136.010397 secs 2023-09-10T04:55:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20136.010397 secs 2023-09-10T04:55:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:05 smithi170 ceph-mon[102832]: pgmap v10899: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 505 KiB/s wr, 13 op/s 2023-09-10T04:55:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:05 smithi082 ceph-mon[98694]: pgmap v10899: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 505 KiB/s wr, 13 op/s 2023-09-10T04:55:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:05 smithi098 ceph-mon[103124]: pgmap v10899: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 505 KiB/s wr, 13 op/s 2023-09-10T04:55:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:07 smithi170 ceph-mon[102832]: pgmap v10900: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 688 KiB/s wr, 33 op/s 2023-09-10T04:55:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:07 smithi082 ceph-mon[98694]: pgmap v10900: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 688 KiB/s wr, 33 op/s 2023-09-10T04:55:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:07 smithi098 ceph-mon[103124]: pgmap v10900: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 688 KiB/s wr, 33 op/s 2023-09-10T04:55:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20139.235164 secs 2023-09-10T04:55:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:08 smithi170 ceph-mon[102832]: osdmap e749: 12 total, 12 up, 12 in 2023-09-10T04:55:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20140.356068 secs 2023-09-10T04:55:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20139.235164 secs 2023-09-10T04:55:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:08 smithi082 ceph-mon[98694]: osdmap e749: 12 total, 12 up, 12 in 2023-09-10T04:55:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20140.356068 secs 2023-09-10T04:55:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20139.235164 secs 2023-09-10T04:55:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:08 smithi098 ceph-mon[103124]: osdmap e749: 12 total, 12 up, 12 in 2023-09-10T04:55:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20140.356068 secs 2023-09-10T04:55:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:09 smithi170 ceph-mon[102832]: pgmap v10902: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 383 KiB/s wr, 31 op/s 2023-09-10T04:55:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20141.010463 secs 2023-09-10T04:55:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:09 smithi082 ceph-mon[98694]: pgmap v10902: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 383 KiB/s wr, 31 op/s 2023-09-10T04:55:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20141.010463 secs 2023-09-10T04:55:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:09 smithi098 ceph-mon[103124]: pgmap v10902: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 383 KiB/s wr, 31 op/s 2023-09-10T04:55:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20141.010463 secs 2023-09-10T04:55:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:11 smithi170 ceph-mon[102832]: pgmap v10903: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 383 KiB/s wr, 31 op/s 2023-09-10T04:55:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:11 smithi082 ceph-mon[98694]: pgmap v10903: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 383 KiB/s wr, 31 op/s 2023-09-10T04:55:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:11 smithi098 ceph-mon[103124]: pgmap v10903: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 383 KiB/s wr, 31 op/s 2023-09-10T04:55:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:13 smithi170 ceph-mon[102832]: pgmap v10904: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 298 KiB/s wr, 29 op/s 2023-09-10T04:55:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20144.235241 secs 2023-09-10T04:55:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20145.356161 secs 2023-09-10T04:55:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:13 smithi082 ceph-mon[98694]: pgmap v10904: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 298 KiB/s wr, 29 op/s 2023-09-10T04:55:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20144.235241 secs 2023-09-10T04:55:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20145.356161 secs 2023-09-10T04:55:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:13 smithi098 ceph-mon[103124]: pgmap v10904: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 298 KiB/s wr, 29 op/s 2023-09-10T04:55:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20144.235241 secs 2023-09-10T04:55:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20145.356161 secs 2023-09-10T04:55:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20146.010482 secs 2023-09-10T04:55:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:14 smithi170 ceph-mon[102832]: osdmap e750: 12 total, 12 up, 12 in 2023-09-10T04:55:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20146.010482 secs 2023-09-10T04:55:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:14 smithi082 ceph-mon[98694]: osdmap e750: 12 total, 12 up, 12 in 2023-09-10T04:55:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20146.010482 secs 2023-09-10T04:55:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:14 smithi098 ceph-mon[103124]: osdmap e750: 12 total, 12 up, 12 in 2023-09-10T04:55:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:15 smithi170 ceph-mon[102832]: pgmap v10906: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 98 KiB/s wr, 7 op/s 2023-09-10T04:55:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:15 smithi082 ceph-mon[98694]: pgmap v10906: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 98 KiB/s wr, 7 op/s 2023-09-10T04:55:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:15 smithi098 ceph-mon[103124]: pgmap v10906: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 98 KiB/s wr, 7 op/s 2023-09-10T04:55:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:17 smithi170 ceph-mon[102832]: pgmap v10907: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 13 op/s 2023-09-10T04:55:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:17 smithi082 ceph-mon[98694]: pgmap v10907: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 13 op/s 2023-09-10T04:55:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:17 smithi098 ceph-mon[103124]: pgmap v10907: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 13 op/s 2023-09-10T04:55:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20149.235304 secs 2023-09-10T04:55:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20150.356219 secs 2023-09-10T04:55:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20149.235304 secs 2023-09-10T04:55:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20150.356219 secs 2023-09-10T04:55:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20149.235304 secs 2023-09-10T04:55:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20150.356219 secs 2023-09-10T04:55:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:19 smithi170 ceph-mon[102832]: pgmap v10908: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T04:55:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20151.010550 secs 2023-09-10T04:55:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:19 smithi082 ceph-mon[98694]: pgmap v10908: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T04:55:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20151.010550 secs 2023-09-10T04:55:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:19 smithi098 ceph-mon[103124]: pgmap v10908: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T04:55:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20151.010550 secs 2023-09-10T04:55:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:21 smithi170 ceph-mon[102832]: pgmap v10909: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T04:55:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:21 smithi082 ceph-mon[98694]: pgmap v10909: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T04:55:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:21 smithi098 ceph-mon[103124]: pgmap v10909: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T04:55:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:23 smithi170 ceph-mon[102832]: pgmap v10910: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T04:55:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20154.235424 secs 2023-09-10T04:55:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20155.356272 secs 2023-09-10T04:55:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:23 smithi082 ceph-mon[98694]: pgmap v10910: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T04:55:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20154.235424 secs 2023-09-10T04:55:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20155.356272 secs 2023-09-10T04:55:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:23 smithi098 ceph-mon[103124]: pgmap v10910: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T04:55:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20154.235424 secs 2023-09-10T04:55:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20155.356272 secs 2023-09-10T04:55:24.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20156.010610 secs 2023-09-10T04:55:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20156.010610 secs 2023-09-10T04:55:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20156.010610 secs 2023-09-10T04:55:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:25 smithi170 ceph-mon[102832]: pgmap v10911: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T04:55:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:25 smithi082 ceph-mon[98694]: pgmap v10911: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T04:55:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:25 smithi098 ceph-mon[103124]: pgmap v10911: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T04:55:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:27 smithi170 ceph-mon[102832]: pgmap v10912: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 15 op/s 2023-09-10T04:55:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:27 smithi082 ceph-mon[98694]: pgmap v10912: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 15 op/s 2023-09-10T04:55:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:27 smithi098 ceph-mon[103124]: pgmap v10912: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 15 op/s 2023-09-10T04:55:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20159.235487 secs 2023-09-10T04:55:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20160.356333 secs 2023-09-10T04:55:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20159.235487 secs 2023-09-10T04:55:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20160.356333 secs 2023-09-10T04:55:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20159.235487 secs 2023-09-10T04:55:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20160.356333 secs 2023-09-10T04:55:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:29 smithi170 ceph-mon[102832]: pgmap v10913: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T04:55:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20161.010677 secs 2023-09-10T04:55:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:29 smithi082 ceph-mon[98694]: pgmap v10913: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T04:55:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20161.010677 secs 2023-09-10T04:55:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:29 smithi098 ceph-mon[103124]: pgmap v10913: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T04:55:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20161.010677 secs 2023-09-10T04:55:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:55:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:55:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:55:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:55:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:55:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:55:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:31 smithi170 ceph-mon[102832]: pgmap v10914: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T04:55:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:31 smithi082 ceph-mon[98694]: pgmap v10914: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T04:55:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:31 smithi098 ceph-mon[103124]: pgmap v10914: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T04:55:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:55:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:55:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:55:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:55:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:55:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:55:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:33 smithi170 ceph-mon[102832]: pgmap v10915: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 207 KiB/s wr, 16 op/s 2023-09-10T04:55:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20165.356392 secs 2023-09-10T04:55:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:33 smithi082 ceph-mon[98694]: pgmap v10915: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 207 KiB/s wr, 16 op/s 2023-09-10T04:55:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20165.356392 secs 2023-09-10T04:55:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:33 smithi098 ceph-mon[103124]: pgmap v10915: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 207 KiB/s wr, 16 op/s 2023-09-10T04:55:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20165.356392 secs 2023-09-10T04:55:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20166.010741 secs 2023-09-10T04:55:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20166.010741 secs 2023-09-10T04:55:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20166.010741 secs 2023-09-10T04:55:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:35 smithi170 ceph-mon[102832]: pgmap v10916: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 10 op/s 2023-09-10T04:55:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:55:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:35 smithi082 ceph-mon[98694]: pgmap v10916: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 10 op/s 2023-09-10T04:55:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:55:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:35 smithi098 ceph-mon[103124]: pgmap v10916: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 10 op/s 2023-09-10T04:55:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:55:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:55:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:55:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:55:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:55:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:55:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:55:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:55:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:55:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:55:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:37 smithi170 ceph-mon[102832]: pgmap v10917: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 345 KiB/s wr, 37 op/s 2023-09-10T04:55:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:55:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:55:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:37 smithi082 ceph-mon[98694]: pgmap v10917: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 345 KiB/s wr, 37 op/s 2023-09-10T04:55:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:55:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:55:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:37 smithi098 ceph-mon[103124]: pgmap v10917: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 345 KiB/s wr, 37 op/s 2023-09-10T04:55:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:55:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:55:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20170.356480 secs 2023-09-10T04:55:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20170.356480 secs 2023-09-10T04:55:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20170.356480 secs 2023-09-10T04:55:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:39 smithi170 ceph-mon[102832]: pgmap v10918: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 345 KiB/s wr, 32 op/s 2023-09-10T04:55:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20171.010806 secs 2023-09-10T04:55:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:39 smithi082 ceph-mon[98694]: pgmap v10918: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 345 KiB/s wr, 32 op/s 2023-09-10T04:55:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20171.010806 secs 2023-09-10T04:55:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:39 smithi098 ceph-mon[103124]: pgmap v10918: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 345 KiB/s wr, 32 op/s 2023-09-10T04:55:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20171.010806 secs 2023-09-10T04:55:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:55:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:55:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:55:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:55:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:55:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:55:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:41 smithi170 ceph-mon[102832]: pgmap v10919: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 345 KiB/s wr, 32 op/s 2023-09-10T04:55:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:41 smithi082 ceph-mon[98694]: pgmap v10919: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 345 KiB/s wr, 32 op/s 2023-09-10T04:55:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:41 smithi098 ceph-mon[103124]: pgmap v10919: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 345 KiB/s wr, 32 op/s 2023-09-10T04:55:43.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:43 smithi170 ceph-mon[102832]: pgmap v10920: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 438 KiB/s wr, 38 op/s 2023-09-10T04:55:43.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20175.356536 secs 2023-09-10T04:55:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:43 smithi082 ceph-mon[98694]: pgmap v10920: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 438 KiB/s wr, 38 op/s 2023-09-10T04:55:43.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20175.356536 secs 2023-09-10T04:55:44.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:43 smithi098 ceph-mon[103124]: pgmap v10920: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 438 KiB/s wr, 38 op/s 2023-09-10T04:55:44.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20175.356536 secs 2023-09-10T04:55:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20176.010867 secs 2023-09-10T04:55:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20176.010867 secs 2023-09-10T04:55:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20176.010867 secs 2023-09-10T04:55:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:45 smithi170 ceph-mon[102832]: pgmap v10921: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 297 KiB/s wr, 32 op/s 2023-09-10T04:55:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:45 smithi082 ceph-mon[98694]: pgmap v10921: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 297 KiB/s wr, 32 op/s 2023-09-10T04:55:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:45 smithi098 ceph-mon[103124]: pgmap v10921: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 297 KiB/s wr, 32 op/s 2023-09-10T04:55:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:47 smithi170 ceph-mon[102832]: pgmap v10922: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 635 KiB/s wr, 39 op/s 2023-09-10T04:55:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:47 smithi082 ceph-mon[98694]: pgmap v10922: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 635 KiB/s wr, 39 op/s 2023-09-10T04:55:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:47 smithi098 ceph-mon[103124]: pgmap v10922: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 635 KiB/s wr, 39 op/s 2023-09-10T04:55:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20179.235773 secs 2023-09-10T04:55:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20180.356597 secs 2023-09-10T04:55:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20179.235773 secs 2023-09-10T04:55:48.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20180.356597 secs 2023-09-10T04:55:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20179.235773 secs 2023-09-10T04:55:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20180.356597 secs 2023-09-10T04:55:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:49 smithi170 ceph-mon[102832]: pgmap v10923: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 432 KiB/s wr, 13 op/s 2023-09-10T04:55:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:55:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:55:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20181.010930 secs 2023-09-10T04:55:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:49 smithi082 ceph-mon[98694]: pgmap v10923: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 432 KiB/s wr, 13 op/s 2023-09-10T04:55:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:55:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:55:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20181.010930 secs 2023-09-10T04:55:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:49 smithi098 ceph-mon[103124]: pgmap v10923: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 432 KiB/s wr, 13 op/s 2023-09-10T04:55:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:55:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:55:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20181.010930 secs 2023-09-10T04:55:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:51 smithi170 ceph-mon[102832]: pgmap v10924: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 432 KiB/s wr, 13 op/s 2023-09-10T04:55:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:51 smithi082 ceph-mon[98694]: pgmap v10924: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 432 KiB/s wr, 13 op/s 2023-09-10T04:55:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:51 smithi098 ceph-mon[103124]: pgmap v10924: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 432 KiB/s wr, 13 op/s 2023-09-10T04:55:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:55:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:55:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:55:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:55:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:55:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:55:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:55:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:55:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:55:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:53 smithi082 ceph-mon[98694]: pgmap v10925: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 432 KiB/s wr, 19 op/s 2023-09-10T04:55:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20184.235842 secs 2023-09-10T04:55:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20185.356655 secs 2023-09-10T04:55:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:53 smithi098 ceph-mon[103124]: pgmap v10925: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 432 KiB/s wr, 19 op/s 2023-09-10T04:55:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20184.235842 secs 2023-09-10T04:55:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20185.356655 secs 2023-09-10T04:55:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:53 smithi170 ceph-mon[102832]: pgmap v10925: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 432 KiB/s wr, 19 op/s 2023-09-10T04:55:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20184.235842 secs 2023-09-10T04:55:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20185.356655 secs 2023-09-10T04:55:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20186.010993 secs 2023-09-10T04:55:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:55:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20186.010993 secs 2023-09-10T04:55:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:55:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20186.010993 secs 2023-09-10T04:55:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:55:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:55 smithi082 ceph-mon[98694]: pgmap v10926: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 339 KiB/s wr, 12 op/s 2023-09-10T04:55:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:55:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:55:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:55:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:55 smithi098 ceph-mon[103124]: pgmap v10926: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 339 KiB/s wr, 12 op/s 2023-09-10T04:55:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:55:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:55:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:55:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:55 smithi170 ceph-mon[102832]: pgmap v10926: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 339 KiB/s wr, 12 op/s 2023-09-10T04:55:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:55:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:55:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:55:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:57 smithi082 ceph-mon[98694]: pgmap v10927: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 477 KiB/s wr, 18 op/s 2023-09-10T04:55:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:57 smithi098 ceph-mon[103124]: pgmap v10927: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 477 KiB/s wr, 18 op/s 2023-09-10T04:55:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:57 smithi170 ceph-mon[102832]: pgmap v10927: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 477 KiB/s wr, 18 op/s 2023-09-10T04:55:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20189.235901 secs 2023-09-10T04:55:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20190.356719 secs 2023-09-10T04:55:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20189.235901 secs 2023-09-10T04:55:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20190.356719 secs 2023-09-10T04:55:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20189.235901 secs 2023-09-10T04:55:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20190.356719 secs 2023-09-10T04:55:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:59 smithi082 ceph-mon[98694]: pgmap v10928: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 10 op/s 2023-09-10T04:55:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:55:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20191.011061 secs 2023-09-10T04:56:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:59 smithi098 ceph-mon[103124]: pgmap v10928: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 10 op/s 2023-09-10T04:56:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:55:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20191.011061 secs 2023-09-10T04:56:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:59 smithi170 ceph-mon[102832]: pgmap v10928: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 10 op/s 2023-09-10T04:56:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:55:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20191.011061 secs 2023-09-10T04:56:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:01 smithi082 ceph-mon[98694]: pgmap v10929: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T04:56:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:01 smithi098 ceph-mon[103124]: pgmap v10929: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T04:56:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:01 smithi170 ceph-mon[102832]: pgmap v10929: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T04:56:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:03 smithi082 ceph-mon[98694]: pgmap v10930: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 210 KiB/s wr, 15 op/s 2023-09-10T04:56:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20194.235963 secs 2023-09-10T04:56:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20195.356778 secs 2023-09-10T04:56:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:03 smithi098 ceph-mon[103124]: pgmap v10930: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 210 KiB/s wr, 15 op/s 2023-09-10T04:56:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20194.235963 secs 2023-09-10T04:56:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20195.356778 secs 2023-09-10T04:56:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:03 smithi170 ceph-mon[102832]: pgmap v10930: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 210 KiB/s wr, 15 op/s 2023-09-10T04:56:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20194.235963 secs 2023-09-10T04:56:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20195.356778 secs 2023-09-10T04:56:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20196.011128 secs 2023-09-10T04:56:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20196.011128 secs 2023-09-10T04:56:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20196.011128 secs 2023-09-10T04:56:05.962 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:05 smithi082 ceph-mon[98694]: pgmap v10931: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 210 KiB/s wr, 10 op/s 2023-09-10T04:56:06.008 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:05 smithi098 ceph-mon[103124]: pgmap v10931: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 210 KiB/s wr, 10 op/s 2023-09-10T04:56:06.185 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:05 smithi170 ceph-mon[102832]: pgmap v10931: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 210 KiB/s wr, 10 op/s 2023-09-10T04:56:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:07 smithi098 ceph-mon[103124]: pgmap v10932: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 320 KiB/s wr, 29 op/s 2023-09-10T04:56:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:07 smithi170 ceph-mon[102832]: pgmap v10932: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 320 KiB/s wr, 29 op/s 2023-09-10T04:56:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:07 smithi082 ceph-mon[98694]: pgmap v10932: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 320 KiB/s wr, 29 op/s 2023-09-10T04:56:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20199.236040 secs 2023-09-10T04:56:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:08 smithi098 ceph-mon[103124]: osdmap e751: 12 total, 12 up, 12 in 2023-09-10T04:56:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20200.356837 secs 2023-09-10T04:56:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20199.236040 secs 2023-09-10T04:56:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:08 smithi170 ceph-mon[102832]: osdmap e751: 12 total, 12 up, 12 in 2023-09-10T04:56:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20200.356837 secs 2023-09-10T04:56:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20199.236040 secs 2023-09-10T04:56:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:08 smithi082 ceph-mon[98694]: osdmap e751: 12 total, 12 up, 12 in 2023-09-10T04:56:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20200.356837 secs 2023-09-10T04:56:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:09 smithi098 ceph-mon[103124]: pgmap v10934: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 218 KiB/s wr, 29 op/s 2023-09-10T04:56:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20201.011192 secs 2023-09-10T04:56:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:09 smithi170 ceph-mon[102832]: pgmap v10934: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 218 KiB/s wr, 29 op/s 2023-09-10T04:56:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20201.011192 secs 2023-09-10T04:56:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:09 smithi082 ceph-mon[98694]: pgmap v10934: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 218 KiB/s wr, 29 op/s 2023-09-10T04:56:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20201.011192 secs 2023-09-10T04:56:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:11 smithi098 ceph-mon[103124]: pgmap v10935: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 218 KiB/s wr, 29 op/s 2023-09-10T04:56:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:11 smithi170 ceph-mon[102832]: pgmap v10935: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 218 KiB/s wr, 29 op/s 2023-09-10T04:56:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:11 smithi082 ceph-mon[98694]: pgmap v10935: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 218 KiB/s wr, 29 op/s 2023-09-10T04:56:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:13 smithi098 ceph-mon[103124]: pgmap v10936: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 133 KiB/s wr, 29 op/s 2023-09-10T04:56:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20204.236097 secs 2023-09-10T04:56:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20205.356894 secs 2023-09-10T04:56:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:13 smithi170 ceph-mon[102832]: pgmap v10936: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 133 KiB/s wr, 29 op/s 2023-09-10T04:56:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20204.236097 secs 2023-09-10T04:56:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20205.356894 secs 2023-09-10T04:56:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:13 smithi082 ceph-mon[98694]: pgmap v10936: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 133 KiB/s wr, 29 op/s 2023-09-10T04:56:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20204.236097 secs 2023-09-10T04:56:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20205.356894 secs 2023-09-10T04:56:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20206.011256 secs 2023-09-10T04:56:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:14 smithi098 ceph-mon[103124]: osdmap e752: 12 total, 12 up, 12 in 2023-09-10T04:56:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20206.011256 secs 2023-09-10T04:56:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:14 smithi170 ceph-mon[102832]: osdmap e752: 12 total, 12 up, 12 in 2023-09-10T04:56:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20206.011256 secs 2023-09-10T04:56:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:14 smithi082 ceph-mon[98694]: osdmap e752: 12 total, 12 up, 12 in 2023-09-10T04:56:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:15 smithi098 ceph-mon[103124]: pgmap v10938: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:56:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:15 smithi170 ceph-mon[102832]: pgmap v10938: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:56:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:15 smithi082 ceph-mon[98694]: pgmap v10938: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:56:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:17 smithi098 ceph-mon[103124]: pgmap v10939: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 91 KiB/s wr, 14 op/s 2023-09-10T04:56:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:17 smithi170 ceph-mon[102832]: pgmap v10939: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 91 KiB/s wr, 14 op/s 2023-09-10T04:56:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:17 smithi082 ceph-mon[98694]: pgmap v10939: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 91 KiB/s wr, 14 op/s 2023-09-10T04:56:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20209.236157 secs 2023-09-10T04:56:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20210.356979 secs 2023-09-10T04:56:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20209.236157 secs 2023-09-10T04:56:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20210.356979 secs 2023-09-10T04:56:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20209.236157 secs 2023-09-10T04:56:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20210.356979 secs 2023-09-10T04:56:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:19 smithi098 ceph-mon[103124]: pgmap v10940: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T04:56:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20211.011325 secs 2023-09-10T04:56:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:19 smithi170 ceph-mon[102832]: pgmap v10940: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T04:56:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20211.011325 secs 2023-09-10T04:56:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:19 smithi082 ceph-mon[98694]: pgmap v10940: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T04:56:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20211.011325 secs 2023-09-10T04:56:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:21 smithi098 ceph-mon[103124]: pgmap v10941: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T04:56:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:21 smithi170 ceph-mon[102832]: pgmap v10941: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T04:56:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:21 smithi082 ceph-mon[98694]: pgmap v10941: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T04:56:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:23 smithi098 ceph-mon[103124]: pgmap v10942: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T04:56:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20214.236219 secs 2023-09-10T04:56:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20215.357061 secs 2023-09-10T04:56:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:23 smithi170 ceph-mon[102832]: pgmap v10942: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T04:56:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20214.236219 secs 2023-09-10T04:56:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20215.357061 secs 2023-09-10T04:56:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:23 smithi082 ceph-mon[98694]: pgmap v10942: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T04:56:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20214.236219 secs 2023-09-10T04:56:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20215.357061 secs 2023-09-10T04:56:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20216.011392 secs 2023-09-10T04:56:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:56:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:56:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20216.011392 secs 2023-09-10T04:56:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:56:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:56:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20216.011392 secs 2023-09-10T04:56:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:56:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:56:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:25 smithi170 ceph-mon[102832]: pgmap v10943: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T04:56:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:25 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:56:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:25 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:56:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:25 smithi082 ceph-mon[98694]: pgmap v10943: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T04:56:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:25 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:56:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:25 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:56:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:25 smithi098 ceph-mon[103124]: pgmap v10943: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T04:56:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:25 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:56:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:25 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:56:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:27 smithi170 ceph-mon[102832]: pgmap v10944: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 129 KiB/s wr, 15 op/s 2023-09-10T04:56:28.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:27 smithi082 ceph-mon[98694]: pgmap v10944: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 129 KiB/s wr, 15 op/s 2023-09-10T04:56:28.265 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:27 smithi098 ceph-mon[103124]: pgmap v10944: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 129 KiB/s wr, 15 op/s 2023-09-10T04:56:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20220.357120 secs 2023-09-10T04:56:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20220.357120 secs 2023-09-10T04:56:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20220.357120 secs 2023-09-10T04:56:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:29 smithi170 ceph-mon[102832]: pgmap v10945: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T04:56:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20221.011457 secs 2023-09-10T04:56:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:29 smithi082 ceph-mon[98694]: pgmap v10945: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T04:56:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20221.011457 secs 2023-09-10T04:56:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:29 smithi098 ceph-mon[103124]: pgmap v10945: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T04:56:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20221.011457 secs 2023-09-10T04:56:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:31 smithi170 ceph-mon[102832]: pgmap v10946: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 12 op/s 2023-09-10T04:56:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:56:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:31 smithi082 ceph-mon[98694]: pgmap v10946: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 12 op/s 2023-09-10T04:56:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:56:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:31 smithi098 ceph-mon[103124]: pgmap v10946: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 12 op/s 2023-09-10T04:56:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:56:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:56:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:56:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:56:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:56:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:56:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:56:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:56:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:56:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:56:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:33 smithi170 ceph-mon[102832]: pgmap v10947: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 128 KiB/s wr, 13 op/s 2023-09-10T04:56:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20225.357176 secs 2023-09-10T04:56:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:33 smithi082 ceph-mon[98694]: pgmap v10947: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 128 KiB/s wr, 13 op/s 2023-09-10T04:56:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20225.357176 secs 2023-09-10T04:56:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:33 smithi098 ceph-mon[103124]: pgmap v10947: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 128 KiB/s wr, 13 op/s 2023-09-10T04:56:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20225.357176 secs 2023-09-10T04:56:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20226.011490 secs 2023-09-10T04:56:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20226.011490 secs 2023-09-10T04:56:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20226.011490 secs 2023-09-10T04:56:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:35 smithi170 ceph-mon[102832]: pgmap v10948: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 10 op/s 2023-09-10T04:56:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:35 smithi082 ceph-mon[98694]: pgmap v10948: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 10 op/s 2023-09-10T04:56:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:35 smithi098 ceph-mon[103124]: pgmap v10948: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 10 op/s 2023-09-10T04:56:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:37 smithi170 ceph-mon[102832]: pgmap v10949: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 264 KiB/s wr, 15 op/s 2023-09-10T04:56:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:37 smithi082 ceph-mon[98694]: pgmap v10949: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 264 KiB/s wr, 15 op/s 2023-09-10T04:56:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:37 smithi098 ceph-mon[103124]: pgmap v10949: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 264 KiB/s wr, 15 op/s 2023-09-10T04:56:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20230.357235 secs 2023-09-10T04:56:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20230.357235 secs 2023-09-10T04:56:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20230.357235 secs 2023-09-10T04:56:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:39 smithi170 ceph-mon[102832]: pgmap v10950: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 200 KiB/s wr, 10 op/s 2023-09-10T04:56:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20231.011555 secs 2023-09-10T04:56:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:39 smithi082 ceph-mon[98694]: pgmap v10950: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 200 KiB/s wr, 10 op/s 2023-09-10T04:56:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20231.011555 secs 2023-09-10T04:56:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:39 smithi098 ceph-mon[103124]: pgmap v10950: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 200 KiB/s wr, 10 op/s 2023-09-10T04:56:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20231.011555 secs 2023-09-10T04:56:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:41 smithi170 ceph-mon[102832]: pgmap v10951: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 251 KiB/s wr, 13 op/s 2023-09-10T04:56:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:41 smithi082 ceph-mon[98694]: pgmap v10951: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 251 KiB/s wr, 13 op/s 2023-09-10T04:56:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:41 smithi098 ceph-mon[103124]: pgmap v10951: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 251 KiB/s wr, 13 op/s 2023-09-10T04:56:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:43 smithi170 ceph-mon[102832]: pgmap v10952: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 224 KiB/s wr, 14 op/s 2023-09-10T04:56:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20235.357293 secs 2023-09-10T04:56:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:43 smithi082 ceph-mon[98694]: pgmap v10952: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 224 KiB/s wr, 14 op/s 2023-09-10T04:56:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20235.357293 secs 2023-09-10T04:56:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:43 smithi098 ceph-mon[103124]: pgmap v10952: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 224 KiB/s wr, 14 op/s 2023-09-10T04:56:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20235.357293 secs 2023-09-10T04:56:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20236.011619 secs 2023-09-10T04:56:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:56:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:56:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20236.011619 secs 2023-09-10T04:56:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:56:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:56:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20236.011619 secs 2023-09-10T04:56:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:56:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:56:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:45 smithi170 ceph-mon[102832]: pgmap v10953: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 224 KiB/s wr, 11 op/s 2023-09-10T04:56:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:56:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:56:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:45 smithi082 ceph-mon[98694]: pgmap v10953: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 224 KiB/s wr, 11 op/s 2023-09-10T04:56:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:56:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:56:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:45 smithi098 ceph-mon[103124]: pgmap v10953: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 224 KiB/s wr, 11 op/s 2023-09-10T04:56:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:56:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:56:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:47 smithi170 ceph-mon[102832]: pgmap v10954: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 394 KiB/s wr, 18 op/s 2023-09-10T04:56:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:47 smithi082 ceph-mon[98694]: pgmap v10954: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 394 KiB/s wr, 18 op/s 2023-09-10T04:56:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:47 smithi098 ceph-mon[103124]: pgmap v10954: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 394 KiB/s wr, 18 op/s 2023-09-10T04:56:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20240.357349 secs 2023-09-10T04:56:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20240.357349 secs 2023-09-10T04:56:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20240.357349 secs 2023-09-10T04:56:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:49 smithi170 ceph-mon[102832]: pgmap v10955: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 257 KiB/s wr, 12 op/s 2023-09-10T04:56:50.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20241.011684 secs 2023-09-10T04:56:50.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:49 smithi082 ceph-mon[98694]: pgmap v10955: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 257 KiB/s wr, 12 op/s 2023-09-10T04:56:50.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20241.011684 secs 2023-09-10T04:56:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:49 smithi098 ceph-mon[103124]: pgmap v10955: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 257 KiB/s wr, 12 op/s 2023-09-10T04:56:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20241.011684 secs 2023-09-10T04:56:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:51 smithi170 ceph-mon[102832]: pgmap v10956: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 257 KiB/s wr, 14 op/s 2023-09-10T04:56:52.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:51 smithi082 ceph-mon[98694]: pgmap v10956: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 257 KiB/s wr, 14 op/s 2023-09-10T04:56:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:51 smithi098 ceph-mon[103124]: pgmap v10956: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 257 KiB/s wr, 14 op/s 2023-09-10T04:56:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:53 smithi170 ceph-mon[102832]: pgmap v10957: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 228 KiB/s wr, 15 op/s 2023-09-10T04:56:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20245.357435 secs 2023-09-10T04:56:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:53 smithi082 ceph-mon[98694]: pgmap v10957: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 228 KiB/s wr, 15 op/s 2023-09-10T04:56:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20245.357435 secs 2023-09-10T04:56:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:53 smithi098 ceph-mon[103124]: pgmap v10957: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 228 KiB/s wr, 15 op/s 2023-09-10T04:56:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20245.357435 secs 2023-09-10T04:56:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20246.011745 secs 2023-09-10T04:56:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20246.011745 secs 2023-09-10T04:56:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20246.011745 secs 2023-09-10T04:56:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:55 smithi170 ceph-mon[102832]: pgmap v10958: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T04:56:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:56:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:55 smithi082 ceph-mon[98694]: pgmap v10958: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T04:56:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:56:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:55 smithi098 ceph-mon[103124]: pgmap v10958: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T04:56:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:56:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:56:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:56:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:56:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:56:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:56:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:56:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:56:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:56:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:56:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:57 smithi170 ceph-mon[102832]: pgmap v10959: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 435 KiB/s wr, 18 op/s 2023-09-10T04:56:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:56:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:56:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:57 smithi082 ceph-mon[98694]: pgmap v10959: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 435 KiB/s wr, 18 op/s 2023-09-10T04:56:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:56:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:56:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:57 smithi098 ceph-mon[103124]: pgmap v10959: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 435 KiB/s wr, 18 op/s 2023-09-10T04:56:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:56:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:56:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20249.236691 secs 2023-09-10T04:56:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20250.357497 secs 2023-09-10T04:56:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20249.236691 secs 2023-09-10T04:56:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20250.357497 secs 2023-09-10T04:56:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20249.236691 secs 2023-09-10T04:56:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20250.357497 secs 2023-09-10T04:57:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:59 smithi170 ceph-mon[102832]: pgmap v10960: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T04:57:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:56:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20251.011812 secs 2023-09-10T04:57:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:59 smithi082 ceph-mon[98694]: pgmap v10960: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T04:57:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:56:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20251.011812 secs 2023-09-10T04:57:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:59 smithi098 ceph-mon[103124]: pgmap v10960: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T04:57:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:56:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20251.011812 secs 2023-09-10T04:57:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:57:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:57:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:57:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:57:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:57:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:57:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:57:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:57:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:57:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:01 smithi170 ceph-mon[102832]: pgmap v10961: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 377 KiB/s wr, 14 op/s 2023-09-10T04:57:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:01 smithi082 ceph-mon[98694]: pgmap v10961: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 377 KiB/s wr, 14 op/s 2023-09-10T04:57:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:01 smithi098 ceph-mon[103124]: pgmap v10961: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 377 KiB/s wr, 14 op/s 2023-09-10T04:57:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:03 smithi170 ceph-mon[102832]: pgmap v10962: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 392 KiB/s wr, 16 op/s 2023-09-10T04:57:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20254.236750 secs 2023-09-10T04:57:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20255.357566 secs 2023-09-10T04:57:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:03 smithi082 ceph-mon[98694]: pgmap v10962: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 392 KiB/s wr, 16 op/s 2023-09-10T04:57:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20254.236750 secs 2023-09-10T04:57:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20255.357566 secs 2023-09-10T04:57:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:03 smithi098 ceph-mon[103124]: pgmap v10962: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 392 KiB/s wr, 16 op/s 2023-09-10T04:57:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20254.236750 secs 2023-09-10T04:57:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20255.357566 secs 2023-09-10T04:57:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20256.011871 secs 2023-09-10T04:57:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20256.011871 secs 2023-09-10T04:57:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20256.011871 secs 2023-09-10T04:57:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:05 smithi170 ceph-mon[102832]: pgmap v10963: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 370 KiB/s wr, 12 op/s 2023-09-10T04:57:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:05 smithi082 ceph-mon[98694]: pgmap v10963: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 370 KiB/s wr, 12 op/s 2023-09-10T04:57:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:05 smithi098 ceph-mon[103124]: pgmap v10963: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 370 KiB/s wr, 12 op/s 2023-09-10T04:57:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:07 smithi170 ceph-mon[102832]: pgmap v10964: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 554 KiB/s wr, 32 op/s 2023-09-10T04:57:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:07 smithi082 ceph-mon[98694]: pgmap v10964: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 554 KiB/s wr, 32 op/s 2023-09-10T04:57:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:07 smithi098 ceph-mon[103124]: pgmap v10964: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 554 KiB/s wr, 32 op/s 2023-09-10T04:57:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20259.236804 secs 2023-09-10T04:57:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:08 smithi170 ceph-mon[102832]: osdmap e753: 12 total, 12 up, 12 in 2023-09-10T04:57:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20260.357625 secs 2023-09-10T04:57:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20259.236804 secs 2023-09-10T04:57:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:08 smithi082 ceph-mon[98694]: osdmap e753: 12 total, 12 up, 12 in 2023-09-10T04:57:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20260.357625 secs 2023-09-10T04:57:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20259.236804 secs 2023-09-10T04:57:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:08 smithi098 ceph-mon[103124]: osdmap e753: 12 total, 12 up, 12 in 2023-09-10T04:57:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20260.357625 secs 2023-09-10T04:57:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:09 smithi170 ceph-mon[102832]: pgmap v10966: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 373 KiB/s wr, 30 op/s 2023-09-10T04:57:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20261.011937 secs 2023-09-10T04:57:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:09 smithi082 ceph-mon[98694]: pgmap v10966: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 373 KiB/s wr, 30 op/s 2023-09-10T04:57:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20261.011937 secs 2023-09-10T04:57:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:09 smithi098 ceph-mon[103124]: pgmap v10966: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 373 KiB/s wr, 30 op/s 2023-09-10T04:57:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20261.011937 secs 2023-09-10T04:57:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:11 smithi170 ceph-mon[102832]: pgmap v10967: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 268 KiB/s wr, 30 op/s 2023-09-10T04:57:12.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:11 smithi082 ceph-mon[98694]: pgmap v10967: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 268 KiB/s wr, 30 op/s 2023-09-10T04:57:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:11 smithi098 ceph-mon[103124]: pgmap v10967: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 268 KiB/s wr, 30 op/s 2023-09-10T04:57:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:13 smithi170 ceph-mon[102832]: pgmap v10968: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 281 KiB/s wr, 29 op/s 2023-09-10T04:57:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20264.236870 secs 2023-09-10T04:57:14.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20265.357682 secs 2023-09-10T04:57:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:13 smithi082 ceph-mon[98694]: pgmap v10968: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 281 KiB/s wr, 29 op/s 2023-09-10T04:57:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20264.236870 secs 2023-09-10T04:57:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20265.357682 secs 2023-09-10T04:57:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:13 smithi098 ceph-mon[103124]: pgmap v10968: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 281 KiB/s wr, 29 op/s 2023-09-10T04:57:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20264.236870 secs 2023-09-10T04:57:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20265.357682 secs 2023-09-10T04:57:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20266.012000 secs 2023-09-10T04:57:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:14 smithi170 ceph-mon[102832]: osdmap e754: 12 total, 12 up, 12 in 2023-09-10T04:57:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20266.012000 secs 2023-09-10T04:57:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:14 smithi082 ceph-mon[98694]: osdmap e754: 12 total, 12 up, 12 in 2023-09-10T04:57:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20266.012000 secs 2023-09-10T04:57:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:14 smithi098 ceph-mon[103124]: osdmap e754: 12 total, 12 up, 12 in 2023-09-10T04:57:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:15 smithi170 ceph-mon[102832]: pgmap v10970: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 76 KiB/s wr, 7 op/s 2023-09-10T04:57:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:15 smithi082 ceph-mon[98694]: pgmap v10970: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 76 KiB/s wr, 7 op/s 2023-09-10T04:57:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:15 smithi098 ceph-mon[103124]: pgmap v10970: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 76 KiB/s wr, 7 op/s 2023-09-10T04:57:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:57:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:57:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:57:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:57:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:57:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:57:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:17 smithi170 ceph-mon[102832]: pgmap v10971: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 14 op/s 2023-09-10T04:57:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:17 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:57:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:17 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:57:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:17 smithi082 ceph-mon[98694]: pgmap v10971: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 14 op/s 2023-09-10T04:57:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:17 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:57:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:17 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:57:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:17 smithi098 ceph-mon[103124]: pgmap v10971: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 14 op/s 2023-09-10T04:57:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:17 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:57:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:17 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:57:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20269.236930 secs 2023-09-10T04:57:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20270.357744 secs 2023-09-10T04:57:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20269.236930 secs 2023-09-10T04:57:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20270.357744 secs 2023-09-10T04:57:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20269.236930 secs 2023-09-10T04:57:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20270.357744 secs 2023-09-10T04:57:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:19 smithi170 ceph-mon[102832]: pgmap v10972: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 12 op/s 2023-09-10T04:57:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20271.012066 secs 2023-09-10T04:57:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:19 smithi082 ceph-mon[98694]: pgmap v10972: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 12 op/s 2023-09-10T04:57:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20271.012066 secs 2023-09-10T04:57:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:19 smithi098 ceph-mon[103124]: pgmap v10972: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 12 op/s 2023-09-10T04:57:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20271.012066 secs 2023-09-10T04:57:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:21 smithi170 ceph-mon[102832]: pgmap v10973: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T04:57:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:21 smithi082 ceph-mon[98694]: pgmap v10973: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T04:57:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:21 smithi098 ceph-mon[103124]: pgmap v10973: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T04:57:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:23 smithi170 ceph-mon[102832]: pgmap v10974: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 12 op/s 2023-09-10T04:57:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20274.236990 secs 2023-09-10T04:57:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20275.357800 secs 2023-09-10T04:57:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20276.012130 secs 2023-09-10T04:57:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:23 smithi082 ceph-mon[98694]: pgmap v10974: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 12 op/s 2023-09-10T04:57:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20274.236990 secs 2023-09-10T04:57:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20275.357800 secs 2023-09-10T04:57:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20276.012130 secs 2023-09-10T04:57:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:23 smithi098 ceph-mon[103124]: pgmap v10974: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 12 op/s 2023-09-10T04:57:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20274.236990 secs 2023-09-10T04:57:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20275.357800 secs 2023-09-10T04:57:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20276.012130 secs 2023-09-10T04:57:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:25 smithi098 ceph-mon[103124]: pgmap v10975: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T04:57:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:25 smithi170 ceph-mon[102832]: pgmap v10975: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T04:57:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:25 smithi082 ceph-mon[98694]: pgmap v10975: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 12 op/s 2023-09-10T04:57:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:27 smithi170 ceph-mon[102832]: pgmap v10976: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 128 KiB/s wr, 14 op/s 2023-09-10T04:57:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:27 smithi082 ceph-mon[98694]: pgmap v10976: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 128 KiB/s wr, 14 op/s 2023-09-10T04:57:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:27 smithi098 ceph-mon[103124]: pgmap v10976: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 128 KiB/s wr, 14 op/s 2023-09-10T04:57:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20280.357861 secs 2023-09-10T04:57:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20280.357861 secs 2023-09-10T04:57:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20280.357861 secs 2023-09-10T04:57:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:29 smithi170 ceph-mon[102832]: pgmap v10977: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 128 KiB/s wr, 9 op/s 2023-09-10T04:57:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20281.012197 secs 2023-09-10T04:57:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:29 smithi082 ceph-mon[98694]: pgmap v10977: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 128 KiB/s wr, 9 op/s 2023-09-10T04:57:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20281.012197 secs 2023-09-10T04:57:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:29 smithi098 ceph-mon[103124]: pgmap v10977: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 128 KiB/s wr, 9 op/s 2023-09-10T04:57:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20281.012197 secs 2023-09-10T04:57:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:31 smithi170 ceph-mon[102832]: pgmap v10978: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T04:57:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:57:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:31 smithi082 ceph-mon[98694]: pgmap v10978: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T04:57:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:57:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:31 smithi098 ceph-mon[103124]: pgmap v10978: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T04:57:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:57:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:57:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:57:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:57:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:57:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:57:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:57:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:57:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:57:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:57:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:33 smithi170 ceph-mon[102832]: pgmap v10979: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 209 KiB/s wr, 13 op/s 2023-09-10T04:57:33.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20285.357919 secs 2023-09-10T04:57:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:33 smithi082 ceph-mon[98694]: pgmap v10979: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 209 KiB/s wr, 13 op/s 2023-09-10T04:57:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20285.357919 secs 2023-09-10T04:57:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:33 smithi098 ceph-mon[103124]: pgmap v10979: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 209 KiB/s wr, 13 op/s 2023-09-10T04:57:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20285.357919 secs 2023-09-10T04:57:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20286.012259 secs 2023-09-10T04:57:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20286.012259 secs 2023-09-10T04:57:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20286.012259 secs 2023-09-10T04:57:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:35 smithi170 ceph-mon[102832]: pgmap v10980: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 10 op/s 2023-09-10T04:57:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:35 smithi082 ceph-mon[98694]: pgmap v10980: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 10 op/s 2023-09-10T04:57:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:35 smithi098 ceph-mon[103124]: pgmap v10980: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 10 op/s 2023-09-10T04:57:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:37 smithi170 ceph-mon[102832]: pgmap v10981: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 283 KiB/s wr, 16 op/s 2023-09-10T04:57:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:37 smithi082 ceph-mon[98694]: pgmap v10981: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 283 KiB/s wr, 16 op/s 2023-09-10T04:57:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:37 smithi098 ceph-mon[103124]: pgmap v10981: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 283 KiB/s wr, 16 op/s 2023-09-10T04:57:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:57:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:57:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20290.357978 secs 2023-09-10T04:57:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:57:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:57:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20290.357978 secs 2023-09-10T04:57:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:57:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:57:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20290.357978 secs 2023-09-10T04:57:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:39 smithi170 ceph-mon[102832]: pgmap v10982: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 221 KiB/s wr, 11 op/s 2023-09-10T04:57:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20291.012329 secs 2023-09-10T04:57:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:39 smithi082 ceph-mon[98694]: pgmap v10982: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 221 KiB/s wr, 11 op/s 2023-09-10T04:57:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20291.012329 secs 2023-09-10T04:57:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:39 smithi098 ceph-mon[103124]: pgmap v10982: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 221 KiB/s wr, 11 op/s 2023-09-10T04:57:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20291.012329 secs 2023-09-10T04:57:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:57:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:57:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:57:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:57:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:57:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:57:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:41 smithi170 ceph-mon[102832]: pgmap v10983: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 314 KiB/s wr, 14 op/s 2023-09-10T04:57:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:41 smithi082 ceph-mon[98694]: pgmap v10983: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 314 KiB/s wr, 14 op/s 2023-09-10T04:57:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:41 smithi098 ceph-mon[103124]: pgmap v10983: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 314 KiB/s wr, 14 op/s 2023-09-10T04:57:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:43 smithi170 ceph-mon[102832]: pgmap v10984: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 329 KiB/s wr, 16 op/s 2023-09-10T04:57:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20295.358038 secs 2023-09-10T04:57:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:43 smithi082 ceph-mon[98694]: pgmap v10984: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 329 KiB/s wr, 16 op/s 2023-09-10T04:57:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20295.358038 secs 2023-09-10T04:57:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:43 smithi098 ceph-mon[103124]: pgmap v10984: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 329 KiB/s wr, 16 op/s 2023-09-10T04:57:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20295.358038 secs 2023-09-10T04:57:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20296.012390 secs 2023-09-10T04:57:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20296.012390 secs 2023-09-10T04:57:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20296.012390 secs 2023-09-10T04:57:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:45 smithi170 ceph-mon[102832]: pgmap v10985: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 248 KiB/s wr, 12 op/s 2023-09-10T04:57:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:45 smithi082 ceph-mon[98694]: pgmap v10985: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 248 KiB/s wr, 12 op/s 2023-09-10T04:57:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:45 smithi098 ceph-mon[103124]: pgmap v10985: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 248 KiB/s wr, 12 op/s 2023-09-10T04:57:47.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:47 smithi170 ceph-mon[102832]: pgmap v10986: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 459 KiB/s wr, 18 op/s 2023-09-10T04:57:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:47 smithi082 ceph-mon[98694]: pgmap v10986: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 459 KiB/s wr, 18 op/s 2023-09-10T04:57:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:47 smithi098 ceph-mon[103124]: pgmap v10986: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 459 KiB/s wr, 18 op/s 2023-09-10T04:57:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20299.237387 secs 2023-09-10T04:57:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20300.358099 secs 2023-09-10T04:57:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20299.237387 secs 2023-09-10T04:57:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20300.358099 secs 2023-09-10T04:57:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20299.237387 secs 2023-09-10T04:57:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20300.358099 secs 2023-09-10T04:57:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:49 smithi170 ceph-mon[102832]: pgmap v10987: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 319 KiB/s wr, 12 op/s 2023-09-10T04:57:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20301.012454 secs 2023-09-10T04:57:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:49 smithi082 ceph-mon[98694]: pgmap v10987: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 319 KiB/s wr, 12 op/s 2023-09-10T04:57:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20301.012454 secs 2023-09-10T04:57:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:49 smithi098 ceph-mon[103124]: pgmap v10987: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 319 KiB/s wr, 12 op/s 2023-09-10T04:57:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20301.012454 secs 2023-09-10T04:57:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:57:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:57:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:57:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:57:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:57:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:57:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:51 smithi170 ceph-mon[102832]: pgmap v10988: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 373 KiB/s wr, 14 op/s 2023-09-10T04:57:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:51 smithi082 ceph-mon[98694]: pgmap v10988: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 373 KiB/s wr, 14 op/s 2023-09-10T04:57:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:51 smithi098 ceph-mon[103124]: pgmap v10988: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 373 KiB/s wr, 14 op/s 2023-09-10T04:57:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:57:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:57:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:57:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:57:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:57:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:57:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:57:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:57:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:57:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:53 smithi170 ceph-mon[102832]: pgmap v10989: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 344 KiB/s wr, 17 op/s 2023-09-10T04:57:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20304.237450 secs 2023-09-10T04:57:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20305.358160 secs 2023-09-10T04:57:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:53 smithi082 ceph-mon[98694]: pgmap v10989: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 344 KiB/s wr, 17 op/s 2023-09-10T04:57:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20304.237450 secs 2023-09-10T04:57:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20305.358160 secs 2023-09-10T04:57:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:53 smithi098 ceph-mon[103124]: pgmap v10989: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 344 KiB/s wr, 17 op/s 2023-09-10T04:57:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20304.237450 secs 2023-09-10T04:57:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20305.358160 secs 2023-09-10T04:57:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20306.012480 secs 2023-09-10T04:57:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20306.012480 secs 2023-09-10T04:57:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20306.012480 secs 2023-09-10T04:57:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:55 smithi082 ceph-mon[98694]: pgmap v10990: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 328 KiB/s wr, 13 op/s 2023-09-10T04:57:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:55 smithi098 ceph-mon[103124]: pgmap v10990: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 328 KiB/s wr, 13 op/s 2023-09-10T04:57:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:55 smithi170 ceph-mon[102832]: pgmap v10990: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 328 KiB/s wr, 13 op/s 2023-09-10T04:57:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:57:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:57:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:57:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:57 smithi082 ceph-mon[98694]: pgmap v10991: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 457 KiB/s wr, 19 op/s 2023-09-10T04:57:57.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:57:57.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:57:57.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:57:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:57 smithi098 ceph-mon[103124]: pgmap v10991: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 457 KiB/s wr, 19 op/s 2023-09-10T04:57:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:57:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:57:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:57:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:57 smithi170 ceph-mon[102832]: pgmap v10991: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 457 KiB/s wr, 19 op/s 2023-09-10T04:57:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:57:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:57:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:57:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20309.237512 secs 2023-09-10T04:57:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20310.358220 secs 2023-09-10T04:57:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20309.237512 secs 2023-09-10T04:57:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20310.358220 secs 2023-09-10T04:57:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20309.237512 secs 2023-09-10T04:57:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20310.358220 secs 2023-09-10T04:57:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:59 smithi082 ceph-mon[98694]: pgmap v10992: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 247 KiB/s wr, 12 op/s 2023-09-10T04:57:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:57:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20311.012574 secs 2023-09-10T04:57:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:59 smithi098 ceph-mon[103124]: pgmap v10992: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 247 KiB/s wr, 12 op/s 2023-09-10T04:57:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:57:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20311.012574 secs 2023-09-10T04:57:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:59 smithi170 ceph-mon[102832]: pgmap v10992: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 247 KiB/s wr, 12 op/s 2023-09-10T04:57:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:57:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20311.012574 secs 2023-09-10T04:58:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:01 smithi082 ceph-mon[98694]: pgmap v10993: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 258 KiB/s wr, 15 op/s 2023-09-10T04:58:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:01 smithi098 ceph-mon[103124]: pgmap v10993: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 258 KiB/s wr, 15 op/s 2023-09-10T04:58:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:01 smithi170 ceph-mon[102832]: pgmap v10993: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 258 KiB/s wr, 15 op/s 2023-09-10T04:58:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:03 smithi098 ceph-mon[103124]: pgmap v10994: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 226 KiB/s wr, 29 op/s 2023-09-10T04:58:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20314.237584 secs 2023-09-10T04:58:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20315.358305 secs 2023-09-10T04:58:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:03 smithi170 ceph-mon[102832]: pgmap v10994: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 226 KiB/s wr, 29 op/s 2023-09-10T04:58:03.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20314.237584 secs 2023-09-10T04:58:03.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20315.358305 secs 2023-09-10T04:58:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:03 smithi082 ceph-mon[98694]: pgmap v10994: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 226 KiB/s wr, 29 op/s 2023-09-10T04:58:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20314.237584 secs 2023-09-10T04:58:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20315.358305 secs 2023-09-10T04:58:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:04 smithi098 ceph-mon[103124]: osdmap e755: 12 total, 12 up, 12 in 2023-09-10T04:58:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20316.012640 secs 2023-09-10T04:58:04.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:04 smithi170 ceph-mon[102832]: osdmap e755: 12 total, 12 up, 12 in 2023-09-10T04:58:04.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20316.012640 secs 2023-09-10T04:58:04.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:04 smithi082 ceph-mon[98694]: osdmap e755: 12 total, 12 up, 12 in 2023-09-10T04:58:04.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20316.012640 secs 2023-09-10T04:58:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:05 smithi098 ceph-mon[103124]: pgmap v10996: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 196 KiB/s wr, 29 op/s 2023-09-10T04:58:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:05 smithi170 ceph-mon[102832]: pgmap v10996: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 196 KiB/s wr, 29 op/s 2023-09-10T04:58:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:05 smithi082 ceph-mon[98694]: pgmap v10996: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 196 KiB/s wr, 29 op/s 2023-09-10T04:58:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:07 smithi098 ceph-mon[103124]: pgmap v10997: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 196 KiB/s wr, 30 op/s 2023-09-10T04:58:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:07 smithi170 ceph-mon[102832]: pgmap v10997: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 196 KiB/s wr, 30 op/s 2023-09-10T04:58:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:07 smithi082 ceph-mon[98694]: pgmap v10997: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 196 KiB/s wr, 30 op/s 2023-09-10T04:58:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20319.237660 secs 2023-09-10T04:58:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20320.358388 secs 2023-09-10T04:58:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20319.237660 secs 2023-09-10T04:58:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20320.358388 secs 2023-09-10T04:58:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20319.237660 secs 2023-09-10T04:58:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20320.358388 secs 2023-09-10T04:58:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:09 smithi098 ceph-mon[103124]: pgmap v10998: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 196 KiB/s wr, 30 op/s 2023-09-10T04:58:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20321.012705 secs 2023-09-10T04:58:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:09 smithi098 ceph-mon[103124]: osdmap e756: 12 total, 12 up, 12 in 2023-09-10T04:58:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:09 smithi170 ceph-mon[102832]: pgmap v10998: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 196 KiB/s wr, 30 op/s 2023-09-10T04:58:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20321.012705 secs 2023-09-10T04:58:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:09 smithi170 ceph-mon[102832]: osdmap e756: 12 total, 12 up, 12 in 2023-09-10T04:58:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:09 smithi082 ceph-mon[98694]: pgmap v10998: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 196 KiB/s wr, 30 op/s 2023-09-10T04:58:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20321.012705 secs 2023-09-10T04:58:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:09 smithi082 ceph-mon[98694]: osdmap e756: 12 total, 12 up, 12 in 2023-09-10T04:58:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:11 smithi098 ceph-mon[103124]: pgmap v11000: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 194 KiB/s wr, 12 op/s 2023-09-10T04:58:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:11 smithi170 ceph-mon[102832]: pgmap v11000: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 194 KiB/s wr, 12 op/s 2023-09-10T04:58:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:11 smithi082 ceph-mon[98694]: pgmap v11000: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 194 KiB/s wr, 12 op/s 2023-09-10T04:58:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:13 smithi098 ceph-mon[103124]: pgmap v11001: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 197 KiB/s wr, 15 op/s 2023-09-10T04:58:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20324.237757 secs 2023-09-10T04:58:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20325.358446 secs 2023-09-10T04:58:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:13 smithi170 ceph-mon[102832]: pgmap v11001: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 197 KiB/s wr, 15 op/s 2023-09-10T04:58:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20324.237757 secs 2023-09-10T04:58:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20325.358446 secs 2023-09-10T04:58:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:13 smithi082 ceph-mon[98694]: pgmap v11001: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 197 KiB/s wr, 15 op/s 2023-09-10T04:58:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20324.237757 secs 2023-09-10T04:58:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20325.358446 secs 2023-09-10T04:58:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20326.012768 secs 2023-09-10T04:58:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20326.012768 secs 2023-09-10T04:58:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20326.012768 secs 2023-09-10T04:58:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:15 smithi098 ceph-mon[103124]: pgmap v11002: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 174 KiB/s wr, 13 op/s 2023-09-10T04:58:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:15 smithi170 ceph-mon[102832]: pgmap v11002: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 174 KiB/s wr, 13 op/s 2023-09-10T04:58:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:15 smithi082 ceph-mon[98694]: pgmap v11002: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 174 KiB/s wr, 13 op/s 2023-09-10T04:58:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:17 smithi098 ceph-mon[103124]: pgmap v11003: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 11 op/s 2023-09-10T04:58:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:17 smithi170 ceph-mon[102832]: pgmap v11003: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 11 op/s 2023-09-10T04:58:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:17 smithi082 ceph-mon[98694]: pgmap v11003: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 11 op/s 2023-09-10T04:58:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20329.237817 secs 2023-09-10T04:58:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20330.358528 secs 2023-09-10T04:58:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20329.237817 secs 2023-09-10T04:58:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20330.358528 secs 2023-09-10T04:58:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20329.237817 secs 2023-09-10T04:58:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20330.358528 secs 2023-09-10T04:58:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:19 smithi098 ceph-mon[103124]: pgmap v11004: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 11 op/s 2023-09-10T04:58:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20331.012832 secs 2023-09-10T04:58:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:19 smithi170 ceph-mon[102832]: pgmap v11004: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 11 op/s 2023-09-10T04:58:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20331.012832 secs 2023-09-10T04:58:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:19 smithi082 ceph-mon[98694]: pgmap v11004: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 11 op/s 2023-09-10T04:58:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20331.012832 secs 2023-09-10T04:58:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:21 smithi098 ceph-mon[103124]: pgmap v11005: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 13 op/s 2023-09-10T04:58:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:21 smithi170 ceph-mon[102832]: pgmap v11005: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 13 op/s 2023-09-10T04:58:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:21 smithi082 ceph-mon[98694]: pgmap v11005: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 13 op/s 2023-09-10T04:58:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:23 smithi098 ceph-mon[103124]: pgmap v11006: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 81 KiB/s wr, 13 op/s 2023-09-10T04:58:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20334.237886 secs 2023-09-10T04:58:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20335.358591 secs 2023-09-10T04:58:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:23 smithi170 ceph-mon[102832]: pgmap v11006: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 81 KiB/s wr, 13 op/s 2023-09-10T04:58:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20334.237886 secs 2023-09-10T04:58:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20335.358591 secs 2023-09-10T04:58:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:23 smithi082 ceph-mon[98694]: pgmap v11006: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 81 KiB/s wr, 13 op/s 2023-09-10T04:58:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20334.237886 secs 2023-09-10T04:58:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20335.358591 secs 2023-09-10T04:58:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20336.012896 secs 2023-09-10T04:58:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20336.012896 secs 2023-09-10T04:58:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20336.012896 secs 2023-09-10T04:58:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:25 smithi098 ceph-mon[103124]: pgmap v11007: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 10 op/s 2023-09-10T04:58:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:25 smithi170 ceph-mon[102832]: pgmap v11007: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 10 op/s 2023-09-10T04:58:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:25 smithi082 ceph-mon[98694]: pgmap v11007: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 10 op/s 2023-09-10T04:58:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:27 smithi098 ceph-mon[103124]: pgmap v11008: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 66 KiB/s wr, 15 op/s 2023-09-10T04:58:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:27 smithi170 ceph-mon[102832]: pgmap v11008: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 66 KiB/s wr, 15 op/s 2023-09-10T04:58:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:27 smithi082 ceph-mon[98694]: pgmap v11008: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 66 KiB/s wr, 15 op/s 2023-09-10T04:58:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20340.358658 secs 2023-09-10T04:58:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20340.358658 secs 2023-09-10T04:58:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20340.358658 secs 2023-09-10T04:58:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:29 smithi170 ceph-mon[102832]: pgmap v11009: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T04:58:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20341.012965 secs 2023-09-10T04:58:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:29 smithi082 ceph-mon[98694]: pgmap v11009: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T04:58:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20341.012965 secs 2023-09-10T04:58:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:29 smithi098 ceph-mon[103124]: pgmap v11009: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 10 op/s 2023-09-10T04:58:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20341.012965 secs 2023-09-10T04:58:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:58:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:58:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:58:30.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:58:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:58:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:58:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:31 smithi170 ceph-mon[102832]: pgmap v11010: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 12 op/s 2023-09-10T04:58:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:58:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:31 smithi082 ceph-mon[98694]: pgmap v11010: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 12 op/s 2023-09-10T04:58:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:58:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:31 smithi098 ceph-mon[103124]: pgmap v11010: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 12 op/s 2023-09-10T04:58:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:58:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:58:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:58:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:58:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:58:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:58:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:58:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:58:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:58:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:58:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:33 smithi170 ceph-mon[102832]: pgmap v11011: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 208 KiB/s wr, 13 op/s 2023-09-10T04:58:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20345.358719 secs 2023-09-10T04:58:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:33 smithi082 ceph-mon[98694]: pgmap v11011: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 208 KiB/s wr, 13 op/s 2023-09-10T04:58:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20345.358719 secs 2023-09-10T04:58:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:33 smithi098 ceph-mon[103124]: pgmap v11011: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 208 KiB/s wr, 13 op/s 2023-09-10T04:58:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20345.358719 secs 2023-09-10T04:58:34.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20346.013028 secs 2023-09-10T04:58:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20346.013028 secs 2023-09-10T04:58:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20346.013028 secs 2023-09-10T04:58:35.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:35 smithi170 ceph-mon[102832]: pgmap v11012: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 10 op/s 2023-09-10T04:58:35.963 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:35 smithi082 ceph-mon[98694]: pgmap v11012: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 10 op/s 2023-09-10T04:58:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:35 smithi098 ceph-mon[103124]: pgmap v11012: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 10 op/s 2023-09-10T04:58:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:37 smithi170 ceph-mon[102832]: pgmap v11013: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 284 KiB/s wr, 16 op/s 2023-09-10T04:58:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:37 smithi082 ceph-mon[98694]: pgmap v11013: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 284 KiB/s wr, 16 op/s 2023-09-10T04:58:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:37 smithi098 ceph-mon[103124]: pgmap v11013: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 284 KiB/s wr, 16 op/s 2023-09-10T04:58:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20350.358801 secs 2023-09-10T04:58:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20350.358801 secs 2023-09-10T04:58:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20350.358801 secs 2023-09-10T04:58:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:39 smithi170 ceph-mon[102832]: pgmap v11014: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 284 KiB/s wr, 11 op/s 2023-09-10T04:58:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20351.013095 secs 2023-09-10T04:58:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:39 smithi082 ceph-mon[98694]: pgmap v11014: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 284 KiB/s wr, 11 op/s 2023-09-10T04:58:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20351.013095 secs 2023-09-10T04:58:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:39 smithi098 ceph-mon[103124]: pgmap v11014: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 284 KiB/s wr, 11 op/s 2023-09-10T04:58:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20351.013095 secs 2023-09-10T04:58:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:58:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:58:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:58:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:58:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:58:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:58:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:41 smithi170 ceph-mon[102832]: pgmap v11015: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 380 KiB/s wr, 15 op/s 2023-09-10T04:58:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:41 smithi082 ceph-mon[98694]: pgmap v11015: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 380 KiB/s wr, 15 op/s 2023-09-10T04:58:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:41 smithi098 ceph-mon[103124]: pgmap v11015: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 380 KiB/s wr, 15 op/s 2023-09-10T04:58:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:43 smithi170 ceph-mon[102832]: pgmap v11016: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 265 KiB/s wr, 15 op/s 2023-09-10T04:58:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20355.358859 secs 2023-09-10T04:58:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:43 smithi082 ceph-mon[98694]: pgmap v11016: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 265 KiB/s wr, 15 op/s 2023-09-10T04:58:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20355.358859 secs 2023-09-10T04:58:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:43 smithi098 ceph-mon[103124]: pgmap v11016: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 265 KiB/s wr, 15 op/s 2023-09-10T04:58:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20355.358859 secs 2023-09-10T04:58:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20356.013156 secs 2023-09-10T04:58:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20356.013156 secs 2023-09-10T04:58:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20356.013156 secs 2023-09-10T04:58:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:45 smithi170 ceph-mon[102832]: pgmap v11017: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 250 KiB/s wr, 12 op/s 2023-09-10T04:58:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:45 smithi082 ceph-mon[98694]: pgmap v11017: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 250 KiB/s wr, 12 op/s 2023-09-10T04:58:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:45 smithi098 ceph-mon[103124]: pgmap v11017: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 250 KiB/s wr, 12 op/s 2023-09-10T04:58:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:58:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:58:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:58:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:58:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:58:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:58:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:47 smithi170 ceph-mon[102832]: pgmap v11018: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 381 KiB/s wr, 18 op/s 2023-09-10T04:58:47.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:47 smithi082 ceph-mon[98694]: pgmap v11018: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 381 KiB/s wr, 18 op/s 2023-09-10T04:58:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:47 smithi098 ceph-mon[103124]: pgmap v11018: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 381 KiB/s wr, 18 op/s 2023-09-10T04:58:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20360.358918 secs 2023-09-10T04:58:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20360.358918 secs 2023-09-10T04:58:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20360.358918 secs 2023-09-10T04:58:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:49 smithi170 ceph-mon[102832]: pgmap v11019: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 243 KiB/s wr, 12 op/s 2023-09-10T04:58:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20361.013236 secs 2023-09-10T04:58:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:49 smithi082 ceph-mon[98694]: pgmap v11019: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 243 KiB/s wr, 12 op/s 2023-09-10T04:58:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20361.013236 secs 2023-09-10T04:58:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:49 smithi098 ceph-mon[103124]: pgmap v11019: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 243 KiB/s wr, 12 op/s 2023-09-10T04:58:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20361.013236 secs 2023-09-10T04:58:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:51 smithi170 ceph-mon[102832]: pgmap v11020: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 359 KiB/s wr, 16 op/s 2023-09-10T04:58:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:51 smithi082 ceph-mon[98694]: pgmap v11020: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 359 KiB/s wr, 16 op/s 2023-09-10T04:58:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:51 smithi098 ceph-mon[103124]: pgmap v11020: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 359 KiB/s wr, 16 op/s 2023-09-10T04:58:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:53 smithi170 ceph-mon[102832]: pgmap v11021: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 14 op/s 2023-09-10T04:58:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20364.271309 secs 2023-09-10T04:58:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20365.358983 secs 2023-09-10T04:58:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:53 smithi082 ceph-mon[98694]: pgmap v11021: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 14 op/s 2023-09-10T04:58:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20364.271309 secs 2023-09-10T04:58:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20365.358983 secs 2023-09-10T04:58:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:53 smithi098 ceph-mon[103124]: pgmap v11021: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 14 op/s 2023-09-10T04:58:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20364.271309 secs 2023-09-10T04:58:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20365.358983 secs 2023-09-10T04:58:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20366.013297 secs 2023-09-10T04:58:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20366.013297 secs 2023-09-10T04:58:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20366.013297 secs 2023-09-10T04:58:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:55 smithi170 ceph-mon[102832]: pgmap v11022: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 11 op/s 2023-09-10T04:58:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:55 smithi082 ceph-mon[98694]: pgmap v11022: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 11 op/s 2023-09-10T04:58:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:55 smithi098 ceph-mon[103124]: pgmap v11022: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 11 op/s 2023-09-10T04:58:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:58:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:58:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:58:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:58:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:58:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:58:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:58:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:58:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:58:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:57 smithi170 ceph-mon[102832]: pgmap v11023: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 410 KiB/s wr, 17 op/s 2023-09-10T04:58:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:58:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:58:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:58:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:58:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:57 smithi082 ceph-mon[98694]: pgmap v11023: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 410 KiB/s wr, 17 op/s 2023-09-10T04:58:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:58:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:58:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:58:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:58:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:57 smithi098 ceph-mon[103124]: pgmap v11023: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 410 KiB/s wr, 17 op/s 2023-09-10T04:58:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:58:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:58:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:58:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:58:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20369.271427 secs 2023-09-10T04:58:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:58:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:58:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20370.359042 secs 2023-09-10T04:58:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20369.271427 secs 2023-09-10T04:58:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:58:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:58:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20370.359042 secs 2023-09-10T04:58:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20369.271427 secs 2023-09-10T04:58:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:58:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:58:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20370.359042 secs 2023-09-10T04:58:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:59 smithi170 ceph-mon[102832]: pgmap v11024: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 11 op/s 2023-09-10T04:58:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:58:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20371.013364 secs 2023-09-10T04:58:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:59 smithi082 ceph-mon[98694]: pgmap v11024: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 11 op/s 2023-09-10T04:58:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:58:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20371.013364 secs 2023-09-10T04:59:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:59 smithi098 ceph-mon[103124]: pgmap v11024: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 11 op/s 2023-09-10T04:59:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:58:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20371.013364 secs 2023-09-10T04:59:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:59:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:59:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:59:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:59:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:59:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:59:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:01 smithi170 ceph-mon[102832]: pgmap v11025: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 373 KiB/s wr, 17 op/s 2023-09-10T04:59:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:01 smithi082 ceph-mon[98694]: pgmap v11025: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 373 KiB/s wr, 17 op/s 2023-09-10T04:59:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:01 smithi098 ceph-mon[103124]: pgmap v11025: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 373 KiB/s wr, 17 op/s 2023-09-10T04:59:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:03 smithi170 ceph-mon[102832]: pgmap v11026: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 258 KiB/s wr, 13 op/s 2023-09-10T04:59:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20374.271495 secs 2023-09-10T04:59:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20375.359127 secs 2023-09-10T04:59:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:03 smithi082 ceph-mon[98694]: pgmap v11026: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 258 KiB/s wr, 13 op/s 2023-09-10T04:59:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20374.271495 secs 2023-09-10T04:59:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20375.359127 secs 2023-09-10T04:59:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:03 smithi098 ceph-mon[103124]: pgmap v11026: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 258 KiB/s wr, 13 op/s 2023-09-10T04:59:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20374.271495 secs 2023-09-10T04:59:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20375.359127 secs 2023-09-10T04:59:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20376.013423 secs 2023-09-10T04:59:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20376.013423 secs 2023-09-10T04:59:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20376.013423 secs 2023-09-10T04:59:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:05 smithi170 ceph-mon[102832]: pgmap v11027: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T04:59:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:05 smithi082 ceph-mon[98694]: pgmap v11027: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T04:59:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:05 smithi098 ceph-mon[103124]: pgmap v11027: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T04:59:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:07 smithi170 ceph-mon[102832]: pgmap v11028: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 338 KiB/s wr, 30 op/s 2023-09-10T04:59:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:07 smithi082 ceph-mon[98694]: pgmap v11028: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 338 KiB/s wr, 30 op/s 2023-09-10T04:59:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:07 smithi098 ceph-mon[103124]: pgmap v11028: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 338 KiB/s wr, 30 op/s 2023-09-10T04:59:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20379.271613 secs 2023-09-10T04:59:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:08 smithi170 ceph-mon[102832]: osdmap e757: 12 total, 12 up, 12 in 2023-09-10T04:59:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20380.359210 secs 2023-09-10T04:59:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20379.271613 secs 2023-09-10T04:59:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:08 smithi082 ceph-mon[98694]: osdmap e757: 12 total, 12 up, 12 in 2023-09-10T04:59:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20380.359210 secs 2023-09-10T04:59:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20379.271613 secs 2023-09-10T04:59:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:08 smithi098 ceph-mon[103124]: osdmap e757: 12 total, 12 up, 12 in 2023-09-10T04:59:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20380.359210 secs 2023-09-10T04:59:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:09 smithi170 ceph-mon[102832]: pgmap v11030: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 247 KiB/s wr, 30 op/s 2023-09-10T04:59:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20381.013488 secs 2023-09-10T04:59:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:09 smithi082 ceph-mon[98694]: pgmap v11030: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 247 KiB/s wr, 30 op/s 2023-09-10T04:59:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20381.013488 secs 2023-09-10T04:59:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:09 smithi098 ceph-mon[103124]: pgmap v11030: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 247 KiB/s wr, 30 op/s 2023-09-10T04:59:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20381.013488 secs 2023-09-10T04:59:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:11 smithi170 ceph-mon[102832]: pgmap v11031: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 29 op/s 2023-09-10T04:59:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:11 smithi082 ceph-mon[98694]: pgmap v11031: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 29 op/s 2023-09-10T04:59:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:11 smithi098 ceph-mon[103124]: pgmap v11031: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 29 op/s 2023-09-10T04:59:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:13 smithi170 ceph-mon[102832]: pgmap v11032: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 29 op/s 2023-09-10T04:59:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20384.271684 secs 2023-09-10T04:59:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20385.359299 secs 2023-09-10T04:59:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:13 smithi082 ceph-mon[98694]: pgmap v11032: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 29 op/s 2023-09-10T04:59:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20384.271684 secs 2023-09-10T04:59:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20385.359299 secs 2023-09-10T04:59:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:13 smithi098 ceph-mon[103124]: pgmap v11032: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 29 op/s 2023-09-10T04:59:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20384.271684 secs 2023-09-10T04:59:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20385.359299 secs 2023-09-10T04:59:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20386.013551 secs 2023-09-10T04:59:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:14 smithi170 ceph-mon[102832]: osdmap e758: 12 total, 12 up, 12 in 2023-09-10T04:59:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20386.013551 secs 2023-09-10T04:59:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:14 smithi082 ceph-mon[98694]: osdmap e758: 12 total, 12 up, 12 in 2023-09-10T04:59:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20386.013551 secs 2023-09-10T04:59:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:14 smithi098 ceph-mon[103124]: osdmap e758: 12 total, 12 up, 12 in 2023-09-10T04:59:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:15 smithi170 ceph-mon[102832]: pgmap v11034: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:59:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:15 smithi082 ceph-mon[98694]: pgmap v11034: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:59:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:15 smithi098 ceph-mon[103124]: pgmap v11034: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T04:59:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:17 smithi170 ceph-mon[102832]: pgmap v11035: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 147 KiB/s wr, 13 op/s 2023-09-10T04:59:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:17 smithi082 ceph-mon[98694]: pgmap v11035: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 147 KiB/s wr, 13 op/s 2023-09-10T04:59:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:17 smithi098 ceph-mon[103124]: pgmap v11035: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 147 KiB/s wr, 13 op/s 2023-09-10T04:59:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20389.271770 secs 2023-09-10T04:59:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20390.359354 secs 2023-09-10T04:59:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20389.271770 secs 2023-09-10T04:59:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20390.359354 secs 2023-09-10T04:59:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20389.271770 secs 2023-09-10T04:59:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20390.359354 secs 2023-09-10T04:59:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:19 smithi082 ceph-mon[98694]: pgmap v11036: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T04:59:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20391.013616 secs 2023-09-10T04:59:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:19 smithi098 ceph-mon[103124]: pgmap v11036: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T04:59:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20391.013616 secs 2023-09-10T04:59:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:19 smithi170 ceph-mon[102832]: pgmap v11036: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T04:59:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20391.013616 secs 2023-09-10T04:59:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:21 smithi082 ceph-mon[98694]: pgmap v11037: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T04:59:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:21 smithi098 ceph-mon[103124]: pgmap v11037: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T04:59:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:21 smithi170 ceph-mon[102832]: pgmap v11037: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T04:59:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:23 smithi082 ceph-mon[98694]: pgmap v11038: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T04:59:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20394.271889 secs 2023-09-10T04:59:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20395.359411 secs 2023-09-10T04:59:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:23 smithi098 ceph-mon[103124]: pgmap v11038: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T04:59:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20394.271889 secs 2023-09-10T04:59:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20395.359411 secs 2023-09-10T04:59:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:23 smithi170 ceph-mon[102832]: pgmap v11038: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T04:59:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20394.271889 secs 2023-09-10T04:59:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20395.359411 secs 2023-09-10T04:59:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20396.013675 secs 2023-09-10T04:59:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20396.013675 secs 2023-09-10T04:59:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20396.013675 secs 2023-09-10T04:59:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:25 smithi082 ceph-mon[98694]: pgmap v11039: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T04:59:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:25 smithi098 ceph-mon[103124]: pgmap v11039: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T04:59:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:25 smithi170 ceph-mon[102832]: pgmap v11039: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T04:59:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:27 smithi082 ceph-mon[98694]: pgmap v11040: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 169 KiB/s wr, 15 op/s 2023-09-10T04:59:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:27 smithi098 ceph-mon[103124]: pgmap v11040: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 169 KiB/s wr, 15 op/s 2023-09-10T04:59:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:27 smithi170 ceph-mon[102832]: pgmap v11040: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 169 KiB/s wr, 15 op/s 2023-09-10T04:59:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20400.359469 secs 2023-09-10T04:59:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20400.359469 secs 2023-09-10T04:59:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20400.359469 secs 2023-09-10T04:59:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:29 smithi082 ceph-mon[98694]: pgmap v11041: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T04:59:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:29 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:59:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:29 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:59:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20401.013738 secs 2023-09-10T04:59:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:29 smithi098 ceph-mon[103124]: pgmap v11041: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T04:59:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:29 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:59:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:29 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:59:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20401.013738 secs 2023-09-10T04:59:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:29 smithi170 ceph-mon[102832]: pgmap v11041: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-09-10T04:59:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:29 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:59:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:29 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:59:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20401.013738 secs 2023-09-10T04:59:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:31 smithi082 ceph-mon[98694]: pgmap v11042: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 211 KiB/s wr, 16 op/s 2023-09-10T04:59:31.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:59:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:31 smithi098 ceph-mon[103124]: pgmap v11042: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 211 KiB/s wr, 16 op/s 2023-09-10T04:59:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:59:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:31 smithi170 ceph-mon[102832]: pgmap v11042: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 211 KiB/s wr, 16 op/s 2023-09-10T04:59:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T04:59:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:59:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:59:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:59:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:59:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:59:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:59:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:59:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:59:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:59:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:33 smithi082 ceph-mon[98694]: pgmap v11043: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T04:59:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20405.359496 secs 2023-09-10T04:59:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:33 smithi098 ceph-mon[103124]: pgmap v11043: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T04:59:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20405.359496 secs 2023-09-10T04:59:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:33 smithi170 ceph-mon[102832]: pgmap v11043: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T04:59:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20405.359496 secs 2023-09-10T04:59:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20406.013801 secs 2023-09-10T04:59:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20406.013801 secs 2023-09-10T04:59:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20406.013801 secs 2023-09-10T04:59:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:35 smithi098 ceph-mon[103124]: pgmap v11044: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T04:59:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:35 smithi170 ceph-mon[102832]: pgmap v11044: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T04:59:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:35 smithi082 ceph-mon[98694]: pgmap v11044: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 210 KiB/s wr, 11 op/s 2023-09-10T04:59:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:37 smithi098 ceph-mon[103124]: pgmap v11045: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 348 KiB/s wr, 17 op/s 2023-09-10T04:59:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:59:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:59:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:37 smithi170 ceph-mon[102832]: pgmap v11045: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 348 KiB/s wr, 17 op/s 2023-09-10T04:59:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:59:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:59:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:37 smithi082 ceph-mon[98694]: pgmap v11045: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 348 KiB/s wr, 17 op/s 2023-09-10T04:59:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:59:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:59:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20410.359585 secs 2023-09-10T04:59:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20410.359585 secs 2023-09-10T04:59:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20410.359585 secs 2023-09-10T04:59:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:39 smithi098 ceph-mon[103124]: pgmap v11046: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 285 KiB/s wr, 11 op/s 2023-09-10T04:59:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20411.013865 secs 2023-09-10T04:59:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:39 smithi170 ceph-mon[102832]: pgmap v11046: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 285 KiB/s wr, 11 op/s 2023-09-10T04:59:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20411.013865 secs 2023-09-10T04:59:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:39 smithi082 ceph-mon[98694]: pgmap v11046: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 285 KiB/s wr, 11 op/s 2023-09-10T04:59:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20411.013865 secs 2023-09-10T04:59:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:59:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:59:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:59:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:59:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:59:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:59:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:41 smithi098 ceph-mon[103124]: pgmap v11047: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 410 KiB/s wr, 18 op/s 2023-09-10T04:59:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:41 smithi170 ceph-mon[102832]: pgmap v11047: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 410 KiB/s wr, 18 op/s 2023-09-10T04:59:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:41 smithi082 ceph-mon[98694]: pgmap v11047: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 410 KiB/s wr, 18 op/s 2023-09-10T04:59:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:43 smithi098 ceph-mon[103124]: pgmap v11048: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T04:59:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20415.359643 secs 2023-09-10T04:59:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:43 smithi170 ceph-mon[102832]: pgmap v11048: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T04:59:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20415.359643 secs 2023-09-10T04:59:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:43 smithi082 ceph-mon[98694]: pgmap v11048: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T04:59:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20415.359643 secs 2023-09-10T04:59:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20416.013928 secs 2023-09-10T04:59:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20416.013928 secs 2023-09-10T04:59:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20416.013928 secs 2023-09-10T04:59:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:45 smithi098 ceph-mon[103124]: pgmap v11049: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T04:59:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:45 smithi170 ceph-mon[102832]: pgmap v11049: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T04:59:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:45 smithi082 ceph-mon[98694]: pgmap v11049: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T04:59:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:47 smithi098 ceph-mon[103124]: pgmap v11050: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 395 KiB/s wr, 18 op/s 2023-09-10T04:59:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:47 smithi170 ceph-mon[102832]: pgmap v11050: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 395 KiB/s wr, 18 op/s 2023-09-10T04:59:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:47 smithi082 ceph-mon[98694]: pgmap v11050: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 395 KiB/s wr, 18 op/s 2023-09-10T04:59:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20420.359702 secs 2023-09-10T04:59:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20420.359702 secs 2023-09-10T04:59:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20420.359702 secs 2023-09-10T04:59:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:49 smithi098 ceph-mon[103124]: pgmap v11051: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 257 KiB/s wr, 12 op/s 2023-09-10T04:59:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20421.013995 secs 2023-09-10T04:59:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:49 smithi170 ceph-mon[102832]: pgmap v11051: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 257 KiB/s wr, 12 op/s 2023-09-10T04:59:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20421.013995 secs 2023-09-10T04:59:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:49 smithi082 ceph-mon[98694]: pgmap v11051: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 257 KiB/s wr, 12 op/s 2023-09-10T04:59:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20421.013995 secs 2023-09-10T04:59:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:51 smithi098 ceph-mon[103124]: pgmap v11052: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 384 KiB/s wr, 18 op/s 2023-09-10T04:59:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:51 smithi170 ceph-mon[102832]: pgmap v11052: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 384 KiB/s wr, 18 op/s 2023-09-10T04:59:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:51 smithi082 ceph-mon[98694]: pgmap v11052: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 384 KiB/s wr, 18 op/s 2023-09-10T04:59:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:53 smithi170 ceph-mon[102832]: pgmap v11053: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 258 KiB/s wr, 12 op/s 2023-09-10T04:59:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20424.272345 secs 2023-09-10T04:59:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20425.359760 secs 2023-09-10T04:59:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:53 smithi082 ceph-mon[98694]: pgmap v11053: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 258 KiB/s wr, 12 op/s 2023-09-10T04:59:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20424.272345 secs 2023-09-10T04:59:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20425.359760 secs 2023-09-10T04:59:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:53 smithi098 ceph-mon[103124]: pgmap v11053: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 258 KiB/s wr, 12 op/s 2023-09-10T04:59:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20424.272345 secs 2023-09-10T04:59:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20425.359760 secs 2023-09-10T04:59:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20426.014060 secs 2023-09-10T04:59:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20426.014060 secs 2023-09-10T04:59:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20426.014060 secs 2023-09-10T04:59:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:55 smithi170 ceph-mon[102832]: pgmap v11054: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 258 KiB/s wr, 12 op/s 2023-09-10T04:59:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:55 smithi082 ceph-mon[98694]: pgmap v11054: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 258 KiB/s wr, 12 op/s 2023-09-10T04:59:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:55 smithi098 ceph-mon[103124]: pgmap v11054: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 258 KiB/s wr, 12 op/s 2023-09-10T04:59:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:59:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:59:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:59:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:59:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:59:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:59:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T04:59:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T04:59:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:59:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:57 smithi170 ceph-mon[102832]: pgmap v11055: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 319 KiB/s wr, 17 op/s 2023-09-10T04:59:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:59:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:59:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:59:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:57 smithi082 ceph-mon[98694]: pgmap v11055: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 319 KiB/s wr, 17 op/s 2023-09-10T04:59:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:59:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:59:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:59:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:57 smithi098 ceph-mon[103124]: pgmap v11055: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 319 KiB/s wr, 17 op/s 2023-09-10T04:59:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T04:59:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T04:59:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T04:59:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20429.272457 secs 2023-09-10T04:59:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:59:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:59:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:59:59.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20430.359849 secs 2023-09-10T04:59:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20429.272457 secs 2023-09-10T04:59:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:59:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:59:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:59:59.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20430.359849 secs 2023-09-10T04:59:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20429.272457 secs 2023-09-10T04:59:59.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T04:59:59.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T04:59:59.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T04:59:59.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20430.359849 secs 2023-09-10T05:00:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:59 smithi170 ceph-mon[102832]: pgmap v11056: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 11 op/s 2023-09-10T05:00:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 04:59:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20431.014103 secs 2023-09-10T05:00:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:59 smithi082 ceph-mon[98694]: pgmap v11056: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 11 op/s 2023-09-10T05:00:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 04:59:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20431.014103 secs 2023-09-10T05:00:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:59 smithi098 ceph-mon[103124]: pgmap v11056: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 11 op/s 2023-09-10T05:00:00.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 04:59:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20431.014103 secs 2023-09-10T05:00:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:00:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:00:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T05:00:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:00:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:00:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T05:00:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:00:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:00:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T05:00:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:01 smithi170 ceph-mon[102832]: pgmap v11057: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 17 op/s 2023-09-10T05:00:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:01 smithi082 ceph-mon[98694]: pgmap v11057: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 17 op/s 2023-09-10T05:00:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:01 smithi098 ceph-mon[103124]: pgmap v11057: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 17 op/s 2023-09-10T05:00:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:03 smithi170 ceph-mon[102832]: pgmap v11058: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 156 KiB/s wr, 11 op/s 2023-09-10T05:00:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20434.272521 secs 2023-09-10T05:00:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20435.359909 secs 2023-09-10T05:00:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:03 smithi082 ceph-mon[98694]: pgmap v11058: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 156 KiB/s wr, 11 op/s 2023-09-10T05:00:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20434.272521 secs 2023-09-10T05:00:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20435.359909 secs 2023-09-10T05:00:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:03 smithi098 ceph-mon[103124]: pgmap v11058: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 156 KiB/s wr, 11 op/s 2023-09-10T05:00:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20434.272521 secs 2023-09-10T05:00:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20435.359909 secs 2023-09-10T05:00:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20436.014168 secs 2023-09-10T05:00:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20436.014168 secs 2023-09-10T05:00:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20436.014168 secs 2023-09-10T05:00:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:05 smithi170 ceph-mon[102832]: pgmap v11059: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 156 KiB/s wr, 11 op/s 2023-09-10T05:00:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:05 smithi082 ceph-mon[98694]: pgmap v11059: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 156 KiB/s wr, 11 op/s 2023-09-10T05:00:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:05 smithi098 ceph-mon[103124]: pgmap v11059: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 156 KiB/s wr, 11 op/s 2023-09-10T05:00:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:07 smithi170 ceph-mon[102832]: pgmap v11060: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 267 KiB/s wr, 31 op/s 2023-09-10T05:00:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:07 smithi082 ceph-mon[98694]: pgmap v11060: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 267 KiB/s wr, 31 op/s 2023-09-10T05:00:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:07 smithi098 ceph-mon[103124]: pgmap v11060: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 267 KiB/s wr, 31 op/s 2023-09-10T05:00:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20439.272580 secs 2023-09-10T05:00:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:08 smithi170 ceph-mon[102832]: osdmap e759: 12 total, 12 up, 12 in 2023-09-10T05:00:09.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20440.359984 secs 2023-09-10T05:00:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20439.272580 secs 2023-09-10T05:00:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:08 smithi082 ceph-mon[98694]: osdmap e759: 12 total, 12 up, 12 in 2023-09-10T05:00:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20440.359984 secs 2023-09-10T05:00:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20439.272580 secs 2023-09-10T05:00:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:08 smithi098 ceph-mon[103124]: osdmap e759: 12 total, 12 up, 12 in 2023-09-10T05:00:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20440.359984 secs 2023-09-10T05:00:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:09 smithi170 ceph-mon[102832]: pgmap v11062: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 248 KiB/s wr, 31 op/s 2023-09-10T05:00:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20441.014231 secs 2023-09-10T05:00:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:09 smithi082 ceph-mon[98694]: pgmap v11062: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 248 KiB/s wr, 31 op/s 2023-09-10T05:00:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20441.014231 secs 2023-09-10T05:00:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:09 smithi098 ceph-mon[103124]: pgmap v11062: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 248 KiB/s wr, 31 op/s 2023-09-10T05:00:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20441.014231 secs 2023-09-10T05:00:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:11 smithi170 ceph-mon[102832]: pgmap v11063: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 30 op/s 2023-09-10T05:00:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:11 smithi082 ceph-mon[98694]: pgmap v11063: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 30 op/s 2023-09-10T05:00:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:11 smithi098 ceph-mon[103124]: pgmap v11063: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 30 op/s 2023-09-10T05:00:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:13 smithi170 ceph-mon[102832]: pgmap v11064: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 30 op/s 2023-09-10T05:00:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20444.272640 secs 2023-09-10T05:00:14.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20445.360042 secs 2023-09-10T05:00:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:13 smithi082 ceph-mon[98694]: pgmap v11064: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 30 op/s 2023-09-10T05:00:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20444.272640 secs 2023-09-10T05:00:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20445.360042 secs 2023-09-10T05:00:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:13 smithi098 ceph-mon[103124]: pgmap v11064: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 30 op/s 2023-09-10T05:00:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20444.272640 secs 2023-09-10T05:00:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20445.360042 secs 2023-09-10T05:00:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20446.014294 secs 2023-09-10T05:00:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:14 smithi170 ceph-mon[102832]: osdmap e760: 12 total, 12 up, 12 in 2023-09-10T05:00:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20446.014294 secs 2023-09-10T05:00:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:14 smithi082 ceph-mon[98694]: osdmap e760: 12 total, 12 up, 12 in 2023-09-10T05:00:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20446.014294 secs 2023-09-10T05:00:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:14 smithi098 ceph-mon[103124]: osdmap e760: 12 total, 12 up, 12 in 2023-09-10T05:00:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:15 smithi170 ceph-mon[102832]: pgmap v11066: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T05:00:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:15 smithi082 ceph-mon[98694]: pgmap v11066: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T05:00:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:15 smithi098 ceph-mon[103124]: pgmap v11066: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T05:00:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:17 smithi170 ceph-mon[102832]: pgmap v11067: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 14 op/s 2023-09-10T05:00:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:17 smithi082 ceph-mon[98694]: pgmap v11067: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 14 op/s 2023-09-10T05:00:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:17 smithi098 ceph-mon[103124]: pgmap v11067: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 14 op/s 2023-09-10T05:00:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20449.272698 secs 2023-09-10T05:00:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20450.360112 secs 2023-09-10T05:00:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20449.272698 secs 2023-09-10T05:00:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20450.360112 secs 2023-09-10T05:00:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20449.272698 secs 2023-09-10T05:00:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20450.360112 secs 2023-09-10T05:00:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:19 smithi170 ceph-mon[102832]: pgmap v11068: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T05:00:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20451.014359 secs 2023-09-10T05:00:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:19 smithi082 ceph-mon[98694]: pgmap v11068: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T05:00:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20451.014359 secs 2023-09-10T05:00:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:19 smithi098 ceph-mon[103124]: pgmap v11068: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T05:00:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20451.014359 secs 2023-09-10T05:00:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:21 smithi170 ceph-mon[102832]: pgmap v11069: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:00:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:21 smithi082 ceph-mon[98694]: pgmap v11069: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:00:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:21 smithi098 ceph-mon[103124]: pgmap v11069: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:00:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:23 smithi170 ceph-mon[102832]: pgmap v11070: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:00:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20454.272786 secs 2023-09-10T05:00:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20455.360168 secs 2023-09-10T05:00:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:23 smithi082 ceph-mon[98694]: pgmap v11070: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:00:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20454.272786 secs 2023-09-10T05:00:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20455.360168 secs 2023-09-10T05:00:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:23 smithi098 ceph-mon[103124]: pgmap v11070: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:00:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20454.272786 secs 2023-09-10T05:00:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20455.360168 secs 2023-09-10T05:00:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20456.014419 secs 2023-09-10T05:00:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20456.014419 secs 2023-09-10T05:00:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20456.014419 secs 2023-09-10T05:00:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:25 smithi170 ceph-mon[102832]: pgmap v11071: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T05:00:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:25 smithi082 ceph-mon[98694]: pgmap v11071: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T05:00:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:25 smithi098 ceph-mon[103124]: pgmap v11071: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T05:00:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:27 smithi170 ceph-mon[102832]: pgmap v11072: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 130 KiB/s wr, 14 op/s 2023-09-10T05:00:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:27 smithi082 ceph-mon[98694]: pgmap v11072: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 130 KiB/s wr, 14 op/s 2023-09-10T05:00:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:27 smithi098 ceph-mon[103124]: pgmap v11072: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 130 KiB/s wr, 14 op/s 2023-09-10T05:00:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20460.360227 secs 2023-09-10T05:00:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20460.360227 secs 2023-09-10T05:00:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20460.360227 secs 2023-09-10T05:00:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:29 smithi170 ceph-mon[102832]: pgmap v11073: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 63 KiB/s wr, 9 op/s 2023-09-10T05:00:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20461.014487 secs 2023-09-10T05:00:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:29 smithi082 ceph-mon[98694]: pgmap v11073: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 63 KiB/s wr, 9 op/s 2023-09-10T05:00:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20461.014487 secs 2023-09-10T05:00:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:29 smithi098 ceph-mon[103124]: pgmap v11073: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 63 KiB/s wr, 9 op/s 2023-09-10T05:00:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20461.014487 secs 2023-09-10T05:00:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:31 smithi170 ceph-mon[102832]: pgmap v11074: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 211 KiB/s wr, 15 op/s 2023-09-10T05:00:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:00:32.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:31 smithi082 ceph-mon[98694]: pgmap v11074: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 211 KiB/s wr, 15 op/s 2023-09-10T05:00:32.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:00:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:31 smithi098 ceph-mon[103124]: pgmap v11074: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 211 KiB/s wr, 15 op/s 2023-09-10T05:00:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:00:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:00:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:00:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:00:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:00:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:00:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:00:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:00:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:00:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:00:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:33 smithi170 ceph-mon[102832]: pgmap v11075: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 211 KiB/s wr, 10 op/s 2023-09-10T05:00:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:00:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:00:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20465.360315 secs 2023-09-10T05:00:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:33 smithi082 ceph-mon[98694]: pgmap v11075: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 211 KiB/s wr, 10 op/s 2023-09-10T05:00:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:00:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:00:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20465.360315 secs 2023-09-10T05:00:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:33 smithi098 ceph-mon[103124]: pgmap v11075: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 211 KiB/s wr, 10 op/s 2023-09-10T05:00:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:00:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:00:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20465.360315 secs 2023-09-10T05:00:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20466.014551 secs 2023-09-10T05:00:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20466.014551 secs 2023-09-10T05:00:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20466.014551 secs 2023-09-10T05:00:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:35 smithi170 ceph-mon[102832]: pgmap v11076: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 211 KiB/s wr, 10 op/s 2023-09-10T05:00:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:35 smithi082 ceph-mon[98694]: pgmap v11076: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 211 KiB/s wr, 10 op/s 2023-09-10T05:00:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:35 smithi098 ceph-mon[103124]: pgmap v11076: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 211 KiB/s wr, 10 op/s 2023-09-10T05:00:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:37 smithi170 ceph-mon[102832]: pgmap v11077: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 350 KiB/s wr, 17 op/s 2023-09-10T05:00:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:37 smithi082 ceph-mon[98694]: pgmap v11077: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 350 KiB/s wr, 17 op/s 2023-09-10T05:00:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:37 smithi098 ceph-mon[103124]: pgmap v11077: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 350 KiB/s wr, 17 op/s 2023-09-10T05:00:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20470.360375 secs 2023-09-10T05:00:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20470.360375 secs 2023-09-10T05:00:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20470.360375 secs 2023-09-10T05:00:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:39 smithi170 ceph-mon[102832]: pgmap v11078: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 287 KiB/s wr, 12 op/s 2023-09-10T05:00:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20471.014613 secs 2023-09-10T05:00:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:39 smithi082 ceph-mon[98694]: pgmap v11078: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 287 KiB/s wr, 12 op/s 2023-09-10T05:00:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20471.014613 secs 2023-09-10T05:00:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:39 smithi098 ceph-mon[103124]: pgmap v11078: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 287 KiB/s wr, 12 op/s 2023-09-10T05:00:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20471.014613 secs 2023-09-10T05:00:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:41 smithi170 ceph-mon[102832]: pgmap v11079: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 413 KiB/s wr, 18 op/s 2023-09-10T05:00:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:41 smithi082 ceph-mon[98694]: pgmap v11079: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 413 KiB/s wr, 18 op/s 2023-09-10T05:00:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:41 smithi098 ceph-mon[103124]: pgmap v11079: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 413 KiB/s wr, 18 op/s 2023-09-10T05:00:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:43 smithi170 ceph-mon[102832]: pgmap v11080: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 265 KiB/s wr, 12 op/s 2023-09-10T05:00:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20475.360455 secs 2023-09-10T05:00:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:43 smithi082 ceph-mon[98694]: pgmap v11080: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 265 KiB/s wr, 12 op/s 2023-09-10T05:00:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20475.360455 secs 2023-09-10T05:00:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:43 smithi098 ceph-mon[103124]: pgmap v11080: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 265 KiB/s wr, 12 op/s 2023-09-10T05:00:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20475.360455 secs 2023-09-10T05:00:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20476.014674 secs 2023-09-10T05:00:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20476.014674 secs 2023-09-10T05:00:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20476.014674 secs 2023-09-10T05:00:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:45 smithi170 ceph-mon[102832]: pgmap v11081: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 265 KiB/s wr, 12 op/s 2023-09-10T05:00:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:45 smithi082 ceph-mon[98694]: pgmap v11081: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 265 KiB/s wr, 12 op/s 2023-09-10T05:00:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:45 smithi098 ceph-mon[103124]: pgmap v11081: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 265 KiB/s wr, 12 op/s 2023-09-10T05:00:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:47 smithi170 ceph-mon[102832]: pgmap v11082: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 397 KiB/s wr, 19 op/s 2023-09-10T05:00:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:47 smithi082 ceph-mon[98694]: pgmap v11082: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 397 KiB/s wr, 19 op/s 2023-09-10T05:00:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:47 smithi098 ceph-mon[103124]: pgmap v11082: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 397 KiB/s wr, 19 op/s 2023-09-10T05:00:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20480.360525 secs 2023-09-10T05:00:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20480.360525 secs 2023-09-10T05:00:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20480.360525 secs 2023-09-10T05:00:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:49 smithi170 ceph-mon[102832]: pgmap v11083: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 258 KiB/s wr, 12 op/s 2023-09-10T05:00:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20481.014741 secs 2023-09-10T05:00:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:00:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:00:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:49 smithi082 ceph-mon[98694]: pgmap v11083: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 258 KiB/s wr, 12 op/s 2023-09-10T05:00:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20481.014741 secs 2023-09-10T05:00:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:00:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:00:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:49 smithi098 ceph-mon[103124]: pgmap v11083: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 258 KiB/s wr, 12 op/s 2023-09-10T05:00:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20481.014741 secs 2023-09-10T05:00:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:00:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:00:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:51 smithi170 ceph-mon[102832]: pgmap v11084: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 396 KiB/s wr, 19 op/s 2023-09-10T05:00:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:00:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:00:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:51 smithi082 ceph-mon[98694]: pgmap v11084: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 396 KiB/s wr, 19 op/s 2023-09-10T05:00:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:00:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:00:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:51 smithi098 ceph-mon[103124]: pgmap v11084: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 396 KiB/s wr, 19 op/s 2023-09-10T05:00:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:00:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:00:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:53 smithi170 ceph-mon[102832]: pgmap v11085: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T05:00:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20484.273240 secs 2023-09-10T05:00:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20485.360583 secs 2023-09-10T05:00:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:53 smithi082 ceph-mon[98694]: pgmap v11085: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T05:00:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20484.273240 secs 2023-09-10T05:00:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20485.360583 secs 2023-09-10T05:00:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:53 smithi098 ceph-mon[103124]: pgmap v11085: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T05:00:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20484.273240 secs 2023-09-10T05:00:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20485.360583 secs 2023-09-10T05:00:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20486.014803 secs 2023-09-10T05:00:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20486.014803 secs 2023-09-10T05:00:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20486.014803 secs 2023-09-10T05:00:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:55 smithi170 ceph-mon[102832]: pgmap v11086: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T05:00:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:00:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:00:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:00:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:55 smithi082 ceph-mon[98694]: pgmap v11086: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T05:00:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:00:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:00:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:00:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:55 smithi098 ceph-mon[103124]: pgmap v11086: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T05:00:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:00:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:00:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:00:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:57 smithi170 ceph-mon[102832]: pgmap v11087: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 340 KiB/s wr, 17 op/s 2023-09-10T05:00:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:57 smithi082 ceph-mon[98694]: pgmap v11087: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 340 KiB/s wr, 17 op/s 2023-09-10T05:00:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:57 smithi098 ceph-mon[103124]: pgmap v11087: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 340 KiB/s wr, 17 op/s 2023-09-10T05:00:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20489.273370 secs 2023-09-10T05:00:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20490.360642 secs 2023-09-10T05:00:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:00:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:00:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:00:59.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:00:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20489.273370 secs 2023-09-10T05:00:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20490.360642 secs 2023-09-10T05:00:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:00:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:00:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:00:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:00:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20489.273370 secs 2023-09-10T05:00:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20490.360642 secs 2023-09-10T05:00:59.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:00:59.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:00:59.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:00:59.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:01:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:59 smithi170 ceph-mon[102832]: pgmap v11088: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 208 KiB/s wr, 11 op/s 2023-09-10T05:01:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:00:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20491.014873 secs 2023-09-10T05:01:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:59 smithi082 ceph-mon[98694]: pgmap v11088: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 208 KiB/s wr, 11 op/s 2023-09-10T05:01:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:00:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20491.014873 secs 2023-09-10T05:01:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:59 smithi098 ceph-mon[103124]: pgmap v11088: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 208 KiB/s wr, 11 op/s 2023-09-10T05:01:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:00:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20491.014873 secs 2023-09-10T05:01:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:01 smithi082 ceph-mon[98694]: pgmap v11089: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 337 KiB/s wr, 17 op/s 2023-09-10T05:01:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:01 smithi170 ceph-mon[102832]: pgmap v11089: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 337 KiB/s wr, 17 op/s 2023-09-10T05:01:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:01 smithi098 ceph-mon[103124]: pgmap v11089: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 337 KiB/s wr, 17 op/s 2023-09-10T05:01:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:03 smithi170 ceph-mon[102832]: pgmap v11090: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 199 KiB/s wr, 11 op/s 2023-09-10T05:01:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20494.273433 secs 2023-09-10T05:01:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20495.360710 secs 2023-09-10T05:01:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20496.014935 secs 2023-09-10T05:01:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:03 smithi082 ceph-mon[98694]: pgmap v11090: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 199 KiB/s wr, 11 op/s 2023-09-10T05:01:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20494.273433 secs 2023-09-10T05:01:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20495.360710 secs 2023-09-10T05:01:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20496.014935 secs 2023-09-10T05:01:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:03 smithi098 ceph-mon[103124]: pgmap v11090: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 199 KiB/s wr, 11 op/s 2023-09-10T05:01:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20494.273433 secs 2023-09-10T05:01:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20495.360710 secs 2023-09-10T05:01:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20496.014935 secs 2023-09-10T05:01:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:05 smithi170 ceph-mon[102832]: pgmap v11091: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 199 KiB/s wr, 11 op/s 2023-09-10T05:01:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:05 smithi082 ceph-mon[98694]: pgmap v11091: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 199 KiB/s wr, 11 op/s 2023-09-10T05:01:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:05 smithi098 ceph-mon[103124]: pgmap v11091: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 199 KiB/s wr, 11 op/s 2023-09-10T05:01:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:07 smithi170 ceph-mon[102832]: pgmap v11092: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 324 KiB/s wr, 32 op/s 2023-09-10T05:01:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:07 smithi082 ceph-mon[98694]: pgmap v11092: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 324 KiB/s wr, 32 op/s 2023-09-10T05:01:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:07 smithi098 ceph-mon[103124]: pgmap v11092: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 324 KiB/s wr, 32 op/s 2023-09-10T05:01:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20499.273530 secs 2023-09-10T05:01:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20500.360768 secs 2023-09-10T05:01:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20499.273530 secs 2023-09-10T05:01:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20500.360768 secs 2023-09-10T05:01:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20499.273530 secs 2023-09-10T05:01:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20500.360768 secs 2023-09-10T05:01:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:09 smithi170 ceph-mon[102832]: pgmap v11093: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 254 KiB/s wr, 26 op/s 2023-09-10T05:01:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:09 smithi170 ceph-mon[102832]: osdmap e761: 12 total, 12 up, 12 in 2023-09-10T05:01:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20501.015001 secs 2023-09-10T05:01:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:09 smithi082 ceph-mon[98694]: pgmap v11093: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 254 KiB/s wr, 26 op/s 2023-09-10T05:01:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:09 smithi082 ceph-mon[98694]: osdmap e761: 12 total, 12 up, 12 in 2023-09-10T05:01:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20501.015001 secs 2023-09-10T05:01:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:09 smithi098 ceph-mon[103124]: pgmap v11093: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 254 KiB/s wr, 26 op/s 2023-09-10T05:01:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:09 smithi098 ceph-mon[103124]: osdmap e761: 12 total, 12 up, 12 in 2023-09-10T05:01:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20501.015001 secs 2023-09-10T05:01:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:01:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:01:10.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:01:10.762 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:01:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:01:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:01:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:11 smithi082 ceph-mon[98694]: pgmap v11095: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 151 KiB/s wr, 31 op/s 2023-09-10T05:01:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:11 smithi098 ceph-mon[103124]: pgmap v11095: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 151 KiB/s wr, 31 op/s 2023-09-10T05:01:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:11 smithi170 ceph-mon[102832]: pgmap v11095: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 151 KiB/s wr, 31 op/s 2023-09-10T05:01:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:01:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:01:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:01:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:01:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:01:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:01:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:13 smithi082 ceph-mon[98694]: pgmap v11096: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 151 KiB/s wr, 31 op/s 2023-09-10T05:01:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20504.273637 secs 2023-09-10T05:01:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20505.360826 secs 2023-09-10T05:01:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:13 smithi098 ceph-mon[103124]: pgmap v11096: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 151 KiB/s wr, 31 op/s 2023-09-10T05:01:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20504.273637 secs 2023-09-10T05:01:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20505.360826 secs 2023-09-10T05:01:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:13 smithi170 ceph-mon[102832]: pgmap v11096: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 151 KiB/s wr, 31 op/s 2023-09-10T05:01:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20504.273637 secs 2023-09-10T05:01:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20505.360826 secs 2023-09-10T05:01:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20506.015063 secs 2023-09-10T05:01:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:14 smithi082 ceph-mon[98694]: osdmap e762: 12 total, 12 up, 12 in 2023-09-10T05:01:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20506.015063 secs 2023-09-10T05:01:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:14 smithi098 ceph-mon[103124]: osdmap e762: 12 total, 12 up, 12 in 2023-09-10T05:01:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20506.015063 secs 2023-09-10T05:01:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:14 smithi170 ceph-mon[102832]: osdmap e762: 12 total, 12 up, 12 in 2023-09-10T05:01:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:15 smithi082 ceph-mon[98694]: pgmap v11098: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T05:01:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:15 smithi098 ceph-mon[103124]: pgmap v11098: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T05:01:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:15 smithi170 ceph-mon[102832]: pgmap v11098: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2023-09-10T05:01:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:17 smithi082 ceph-mon[98694]: pgmap v11099: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 16 op/s 2023-09-10T05:01:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:17 smithi098 ceph-mon[103124]: pgmap v11099: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 16 op/s 2023-09-10T05:01:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:17 smithi170 ceph-mon[102832]: pgmap v11099: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 16 op/s 2023-09-10T05:01:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20509.273698 secs 2023-09-10T05:01:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20510.360897 secs 2023-09-10T05:01:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20509.273698 secs 2023-09-10T05:01:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20510.360897 secs 2023-09-10T05:01:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20509.273698 secs 2023-09-10T05:01:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20510.360897 secs 2023-09-10T05:01:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:19 smithi082 ceph-mon[98694]: pgmap v11100: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 13 op/s 2023-09-10T05:01:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20511.015128 secs 2023-09-10T05:01:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:19 smithi098 ceph-mon[103124]: pgmap v11100: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 13 op/s 2023-09-10T05:01:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20511.015128 secs 2023-09-10T05:01:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:19 smithi170 ceph-mon[102832]: pgmap v11100: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 13 op/s 2023-09-10T05:01:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20511.015128 secs 2023-09-10T05:01:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:21 smithi098 ceph-mon[103124]: pgmap v11101: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T05:01:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:21 smithi170 ceph-mon[102832]: pgmap v11101: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T05:01:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:21 smithi082 ceph-mon[98694]: pgmap v11101: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T05:01:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:23 smithi098 ceph-mon[103124]: pgmap v11102: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T05:01:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20514.273755 secs 2023-09-10T05:01:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20515.360955 secs 2023-09-10T05:01:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:23 smithi170 ceph-mon[102832]: pgmap v11102: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T05:01:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20514.273755 secs 2023-09-10T05:01:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20515.360955 secs 2023-09-10T05:01:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:23 smithi082 ceph-mon[98694]: pgmap v11102: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 12 op/s 2023-09-10T05:01:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20514.273755 secs 2023-09-10T05:01:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20515.360955 secs 2023-09-10T05:01:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20516.015192 secs 2023-09-10T05:01:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20516.015192 secs 2023-09-10T05:01:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20516.015192 secs 2023-09-10T05:01:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:25 smithi098 ceph-mon[103124]: pgmap v11103: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T05:01:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:25 smithi170 ceph-mon[102832]: pgmap v11103: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T05:01:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:25 smithi082 ceph-mon[98694]: pgmap v11103: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T05:01:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:27 smithi098 ceph-mon[103124]: pgmap v11104: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 162 KiB/s wr, 15 op/s 2023-09-10T05:01:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:27 smithi170 ceph-mon[102832]: pgmap v11104: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 162 KiB/s wr, 15 op/s 2023-09-10T05:01:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:27 smithi082 ceph-mon[98694]: pgmap v11104: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 162 KiB/s wr, 15 op/s 2023-09-10T05:01:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20520.361008 secs 2023-09-10T05:01:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20520.361008 secs 2023-09-10T05:01:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20520.361008 secs 2023-09-10T05:01:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:29 smithi098 ceph-mon[103124]: pgmap v11105: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T05:01:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20521.015258 secs 2023-09-10T05:01:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:29 smithi170 ceph-mon[102832]: pgmap v11105: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T05:01:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20521.015258 secs 2023-09-10T05:01:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:29 smithi082 ceph-mon[98694]: pgmap v11105: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T05:01:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20521.015258 secs 2023-09-10T05:01:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:31 smithi098 ceph-mon[103124]: pgmap v11106: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 212 KiB/s wr, 15 op/s 2023-09-10T05:01:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:01:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:31 smithi170 ceph-mon[102832]: pgmap v11106: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 212 KiB/s wr, 15 op/s 2023-09-10T05:01:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:01:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:31 smithi082 ceph-mon[98694]: pgmap v11106: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 212 KiB/s wr, 15 op/s 2023-09-10T05:01:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:01:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:01:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:01:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:01:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:01:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:01:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:01:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:01:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:01:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:01:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:33 smithi098 ceph-mon[103124]: pgmap v11107: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 212 KiB/s wr, 10 op/s 2023-09-10T05:01:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20525.361075 secs 2023-09-10T05:01:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:33 smithi170 ceph-mon[102832]: pgmap v11107: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 212 KiB/s wr, 10 op/s 2023-09-10T05:01:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20525.361075 secs 2023-09-10T05:01:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:33 smithi082 ceph-mon[98694]: pgmap v11107: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 212 KiB/s wr, 10 op/s 2023-09-10T05:01:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20525.361075 secs 2023-09-10T05:01:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:01:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:01:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20526.015318 secs 2023-09-10T05:01:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:01:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:01:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20526.015318 secs 2023-09-10T05:01:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:01:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:01:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20526.015318 secs 2023-09-10T05:01:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:35 smithi098 ceph-mon[103124]: pgmap v11108: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 212 KiB/s wr, 10 op/s 2023-09-10T05:01:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:35 smithi170 ceph-mon[102832]: pgmap v11108: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 212 KiB/s wr, 10 op/s 2023-09-10T05:01:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:35 smithi082 ceph-mon[98694]: pgmap v11108: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 212 KiB/s wr, 10 op/s 2023-09-10T05:01:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:37 smithi098 ceph-mon[103124]: pgmap v11109: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 351 KiB/s wr, 17 op/s 2023-09-10T05:01:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:37 smithi170 ceph-mon[102832]: pgmap v11109: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 351 KiB/s wr, 17 op/s 2023-09-10T05:01:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:37 smithi082 ceph-mon[98694]: pgmap v11109: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 351 KiB/s wr, 17 op/s 2023-09-10T05:01:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20530.361133 secs 2023-09-10T05:01:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20530.361133 secs 2023-09-10T05:01:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20530.361133 secs 2023-09-10T05:01:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:39 smithi170 ceph-mon[102832]: pgmap v11110: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 288 KiB/s wr, 12 op/s 2023-09-10T05:01:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20531.015385 secs 2023-09-10T05:01:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:39 smithi082 ceph-mon[98694]: pgmap v11110: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 288 KiB/s wr, 12 op/s 2023-09-10T05:01:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20531.015385 secs 2023-09-10T05:01:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:39 smithi098 ceph-mon[103124]: pgmap v11110: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 288 KiB/s wr, 12 op/s 2023-09-10T05:01:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20531.015385 secs 2023-09-10T05:01:41.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:41 smithi170 ceph-mon[102832]: pgmap v11111: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 415 KiB/s wr, 18 op/s 2023-09-10T05:01:41.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:41 smithi082 ceph-mon[98694]: pgmap v11111: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 415 KiB/s wr, 18 op/s 2023-09-10T05:01:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:41 smithi098 ceph-mon[103124]: pgmap v11111: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 415 KiB/s wr, 18 op/s 2023-09-10T05:01:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:43 smithi170 ceph-mon[102832]: pgmap v11112: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 266 KiB/s wr, 12 op/s 2023-09-10T05:01:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20535.361190 secs 2023-09-10T05:01:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:43 smithi082 ceph-mon[98694]: pgmap v11112: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 266 KiB/s wr, 12 op/s 2023-09-10T05:01:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20535.361190 secs 2023-09-10T05:01:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:43 smithi098 ceph-mon[103124]: pgmap v11112: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 266 KiB/s wr, 12 op/s 2023-09-10T05:01:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20535.361190 secs 2023-09-10T05:01:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20536.015446 secs 2023-09-10T05:01:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20536.015446 secs 2023-09-10T05:01:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20536.015446 secs 2023-09-10T05:01:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:45 smithi170 ceph-mon[102832]: pgmap v11113: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 266 KiB/s wr, 12 op/s 2023-09-10T05:01:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:45 smithi082 ceph-mon[98694]: pgmap v11113: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 266 KiB/s wr, 12 op/s 2023-09-10T05:01:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:45 smithi098 ceph-mon[103124]: pgmap v11113: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 266 KiB/s wr, 12 op/s 2023-09-10T05:01:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:47 smithi170 ceph-mon[102832]: pgmap v11114: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 399 KiB/s wr, 18 op/s 2023-09-10T05:01:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:47 smithi082 ceph-mon[98694]: pgmap v11114: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 399 KiB/s wr, 18 op/s 2023-09-10T05:01:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:47 smithi098 ceph-mon[103124]: pgmap v11114: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 399 KiB/s wr, 18 op/s 2023-09-10T05:01:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20540.361249 secs 2023-09-10T05:01:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20540.361249 secs 2023-09-10T05:01:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20540.361249 secs 2023-09-10T05:01:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:49 smithi170 ceph-mon[102832]: pgmap v11115: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 259 KiB/s wr, 12 op/s 2023-09-10T05:01:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20541.015513 secs 2023-09-10T05:01:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:49 smithi082 ceph-mon[98694]: pgmap v11115: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 259 KiB/s wr, 12 op/s 2023-09-10T05:01:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20541.015513 secs 2023-09-10T05:01:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:49 smithi098 ceph-mon[103124]: pgmap v11115: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 259 KiB/s wr, 12 op/s 2023-09-10T05:01:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20541.015513 secs 2023-09-10T05:01:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:01:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:01:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:01:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:01:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:01:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:01:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:51 smithi170 ceph-mon[102832]: pgmap v11116: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 408 KiB/s wr, 18 op/s 2023-09-10T05:01:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:51 smithi082 ceph-mon[98694]: pgmap v11116: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 408 KiB/s wr, 18 op/s 2023-09-10T05:01:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:51 smithi098 ceph-mon[103124]: pgmap v11116: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 408 KiB/s wr, 18 op/s 2023-09-10T05:01:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:01:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:01:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:01:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:01:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:01:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:01:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:53 smithi170 ceph-mon[102832]: pgmap v11117: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T05:01:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20544.274176 secs 2023-09-10T05:01:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20545.361322 secs 2023-09-10T05:01:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:53 smithi082 ceph-mon[98694]: pgmap v11117: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T05:01:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20544.274176 secs 2023-09-10T05:01:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20545.361322 secs 2023-09-10T05:01:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:53 smithi098 ceph-mon[103124]: pgmap v11117: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T05:01:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20544.274176 secs 2023-09-10T05:01:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20545.361322 secs 2023-09-10T05:01:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20546.015576 secs 2023-09-10T05:01:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20546.015576 secs 2023-09-10T05:01:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20546.015576 secs 2023-09-10T05:01:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:55 smithi170 ceph-mon[102832]: pgmap v11118: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T05:01:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:55 smithi082 ceph-mon[98694]: pgmap v11118: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T05:01:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:55 smithi098 ceph-mon[103124]: pgmap v11118: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T05:01:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:01:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:01:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:01:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:01:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:01:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:01:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:01:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:01:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:01:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:57 smithi170 ceph-mon[102832]: pgmap v11119: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 352 KiB/s wr, 18 op/s 2023-09-10T05:01:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:57 smithi082 ceph-mon[98694]: pgmap v11119: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 352 KiB/s wr, 18 op/s 2023-09-10T05:01:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:57 smithi098 ceph-mon[103124]: pgmap v11119: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 352 KiB/s wr, 18 op/s 2023-09-10T05:01:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20549.274308 secs 2023-09-10T05:01:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20550.361380 secs 2023-09-10T05:01:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20549.274308 secs 2023-09-10T05:01:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20550.361380 secs 2023-09-10T05:01:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20549.274308 secs 2023-09-10T05:01:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20550.361380 secs 2023-09-10T05:01:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:59 smithi170 ceph-mon[102832]: pgmap v11120: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 219 KiB/s wr, 12 op/s 2023-09-10T05:01:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20551.015641 secs 2023-09-10T05:01:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:01:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:01:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:59 smithi082 ceph-mon[98694]: pgmap v11120: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 219 KiB/s wr, 12 op/s 2023-09-10T05:01:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20551.015641 secs 2023-09-10T05:01:59.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:01:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:02:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:59 smithi098 ceph-mon[103124]: pgmap v11120: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 219 KiB/s wr, 12 op/s 2023-09-10T05:02:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20551.015641 secs 2023-09-10T05:02:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:01:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:02:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:01 smithi170 ceph-mon[102832]: pgmap v11121: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 315 KiB/s wr, 17 op/s 2023-09-10T05:02:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:02:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:02:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:01 smithi082 ceph-mon[98694]: pgmap v11121: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 315 KiB/s wr, 17 op/s 2023-09-10T05:02:01.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:02:01.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:02:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:01 smithi098 ceph-mon[103124]: pgmap v11121: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 315 KiB/s wr, 17 op/s 2023-09-10T05:02:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:02:02.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:02:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:02:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:02:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:02:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]': finished 2023-09-10T05:02:03.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:02:03.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:02:03.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:02:03.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:02:03.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:02:03.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:02:03.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:02:03.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:02:03.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:02:03.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:02:03.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:02:03.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:02:03.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:02:03.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:02:03.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:02:03.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:02:03.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]': finished 2023-09-10T05:02:03.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:02:03.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:02:03.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:02:03.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:02:03.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:02:03.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:02:03.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:02:03.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:02:03.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:02:03.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:02:03.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:02:03.212 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:02:03.212 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:02:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:02:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:02:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:02:03.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]': finished 2023-09-10T05:02:03.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:02:03.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:02:03.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:02:03.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:02:03.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:02:03.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:02:03.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:02:03.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:02:03.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:02:03.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:02:03.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:02:03.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:02:03.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:02:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:03 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T05:02:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:03 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T05:02:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:03 smithi170 ceph-mon[102832]: pgmap v11122: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 10 op/s 2023-09-10T05:02:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20554.274371 secs 2023-09-10T05:02:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:03 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T05:02:04.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:03 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T05:02:04.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20555.361440 secs 2023-09-10T05:02:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:03 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T05:02:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:03 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T05:02:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:03 smithi082 ceph-mon[98694]: pgmap v11122: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 10 op/s 2023-09-10T05:02:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20554.274371 secs 2023-09-10T05:02:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:03 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T05:02:04.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:03 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T05:02:04.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20555.361440 secs 2023-09-10T05:02:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:03 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T05:02:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:03 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T05:02:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:03 smithi098 ceph-mon[103124]: pgmap v11122: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 10 op/s 2023-09-10T05:02:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20554.274371 secs 2023-09-10T05:02:04.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:03 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T05:02:04.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:03 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T05:02:04.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20555.361440 secs 2023-09-10T05:02:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20556.015704 secs 2023-09-10T05:02:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20556.015704 secs 2023-09-10T05:02:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20556.015704 secs 2023-09-10T05:02:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:05 smithi170 ceph-mon[102832]: pgmap v11123: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 10 op/s 2023-09-10T05:02:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:05 smithi082 ceph-mon[98694]: pgmap v11123: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 10 op/s 2023-09-10T05:02:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:05 smithi098 ceph-mon[103124]: pgmap v11123: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 10 op/s 2023-09-10T05:02:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:07 smithi170 ceph-mon[102832]: pgmap v11124: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 263 KiB/s wr, 30 op/s 2023-09-10T05:02:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:07 smithi082 ceph-mon[98694]: pgmap v11124: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 263 KiB/s wr, 30 op/s 2023-09-10T05:02:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:07 smithi098 ceph-mon[103124]: pgmap v11124: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 263 KiB/s wr, 30 op/s 2023-09-10T05:02:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20559.274434 secs 2023-09-10T05:02:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:08 smithi170 ceph-mon[102832]: osdmap e763: 12 total, 12 up, 12 in 2023-09-10T05:02:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20560.361516 secs 2023-09-10T05:02:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20559.274434 secs 2023-09-10T05:02:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:08 smithi082 ceph-mon[98694]: osdmap e763: 12 total, 12 up, 12 in 2023-09-10T05:02:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20560.361516 secs 2023-09-10T05:02:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20559.274434 secs 2023-09-10T05:02:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:08 smithi098 ceph-mon[103124]: osdmap e763: 12 total, 12 up, 12 in 2023-09-10T05:02:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20560.361516 secs 2023-09-10T05:02:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:09 smithi170 ceph-mon[102832]: pgmap v11126: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 230 KiB/s wr, 29 op/s 2023-09-10T05:02:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20561.015770 secs 2023-09-10T05:02:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:09 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:02:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:09 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:02:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:09 smithi082 ceph-mon[98694]: pgmap v11126: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 230 KiB/s wr, 29 op/s 2023-09-10T05:02:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20561.015770 secs 2023-09-10T05:02:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:09 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:02:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:09 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:02:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:09 smithi098 ceph-mon[103124]: pgmap v11126: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 230 KiB/s wr, 29 op/s 2023-09-10T05:02:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20561.015770 secs 2023-09-10T05:02:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:09 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:02:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:09 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:02:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:11 smithi170 ceph-mon[102832]: pgmap v11127: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 116 KiB/s wr, 30 op/s 2023-09-10T05:02:12.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:11 smithi082 ceph-mon[98694]: pgmap v11127: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 116 KiB/s wr, 30 op/s 2023-09-10T05:02:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:11 smithi098 ceph-mon[103124]: pgmap v11127: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 116 KiB/s wr, 30 op/s 2023-09-10T05:02:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:02:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:02:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:02:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:02:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:02:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:02:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:13 smithi170 ceph-mon[102832]: pgmap v11128: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 116 KiB/s wr, 30 op/s 2023-09-10T05:02:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20564.274552 secs 2023-09-10T05:02:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20565.361576 secs 2023-09-10T05:02:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:13 smithi082 ceph-mon[98694]: pgmap v11128: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 116 KiB/s wr, 30 op/s 2023-09-10T05:02:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20564.274552 secs 2023-09-10T05:02:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20565.361576 secs 2023-09-10T05:02:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:13 smithi098 ceph-mon[103124]: pgmap v11128: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 116 KiB/s wr, 30 op/s 2023-09-10T05:02:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20564.274552 secs 2023-09-10T05:02:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20565.361576 secs 2023-09-10T05:02:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20566.015830 secs 2023-09-10T05:02:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:14 smithi170 ceph-mon[102832]: osdmap e764: 12 total, 12 up, 12 in 2023-09-10T05:02:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20566.015830 secs 2023-09-10T05:02:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:14 smithi082 ceph-mon[98694]: osdmap e764: 12 total, 12 up, 12 in 2023-09-10T05:02:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20566.015830 secs 2023-09-10T05:02:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:14 smithi098 ceph-mon[103124]: osdmap e764: 12 total, 12 up, 12 in 2023-09-10T05:02:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:15 smithi170 ceph-mon[102832]: pgmap v11130: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2023-09-10T05:02:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:15 smithi082 ceph-mon[98694]: pgmap v11130: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2023-09-10T05:02:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:15 smithi098 ceph-mon[103124]: pgmap v11130: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2023-09-10T05:02:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:17 smithi170 ceph-mon[102832]: pgmap v11131: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 14 op/s 2023-09-10T05:02:18.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:17 smithi082 ceph-mon[98694]: pgmap v11131: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 14 op/s 2023-09-10T05:02:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:17 smithi098 ceph-mon[103124]: pgmap v11131: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 14 op/s 2023-09-10T05:02:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20569.274653 secs 2023-09-10T05:02:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20570.361635 secs 2023-09-10T05:02:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20569.274653 secs 2023-09-10T05:02:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20570.361635 secs 2023-09-10T05:02:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20569.274653 secs 2023-09-10T05:02:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20570.361635 secs 2023-09-10T05:02:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:19 smithi170 ceph-mon[102832]: pgmap v11132: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 130 KiB/s wr, 12 op/s 2023-09-10T05:02:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20571.015896 secs 2023-09-10T05:02:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:19 smithi082 ceph-mon[98694]: pgmap v11132: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 130 KiB/s wr, 12 op/s 2023-09-10T05:02:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20571.015896 secs 2023-09-10T05:02:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:19 smithi098 ceph-mon[103124]: pgmap v11132: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 130 KiB/s wr, 12 op/s 2023-09-10T05:02:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20571.015896 secs 2023-09-10T05:02:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:21 smithi170 ceph-mon[102832]: pgmap v11133: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T05:02:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:21 smithi082 ceph-mon[98694]: pgmap v11133: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T05:02:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:21 smithi098 ceph-mon[103124]: pgmap v11133: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T05:02:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:23 smithi170 ceph-mon[102832]: pgmap v11134: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T05:02:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20574.274717 secs 2023-09-10T05:02:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20575.361697 secs 2023-09-10T05:02:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:23 smithi082 ceph-mon[98694]: pgmap v11134: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T05:02:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20574.274717 secs 2023-09-10T05:02:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20575.361697 secs 2023-09-10T05:02:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:23 smithi098 ceph-mon[103124]: pgmap v11134: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T05:02:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20574.274717 secs 2023-09-10T05:02:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20575.361697 secs 2023-09-10T05:02:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20576.015958 secs 2023-09-10T05:02:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20576.015958 secs 2023-09-10T05:02:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20576.015958 secs 2023-09-10T05:02:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:25 smithi170 ceph-mon[102832]: pgmap v11135: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T05:02:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:25 smithi082 ceph-mon[98694]: pgmap v11135: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T05:02:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:25 smithi098 ceph-mon[103124]: pgmap v11135: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T05:02:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:27 smithi170 ceph-mon[102832]: pgmap v11136: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 108 KiB/s wr, 14 op/s 2023-09-10T05:02:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:27 smithi082 ceph-mon[98694]: pgmap v11136: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 108 KiB/s wr, 14 op/s 2023-09-10T05:02:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:27 smithi098 ceph-mon[103124]: pgmap v11136: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 108 KiB/s wr, 14 op/s 2023-09-10T05:02:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20580.361756 secs 2023-09-10T05:02:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20580.361756 secs 2023-09-10T05:02:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20580.361756 secs 2023-09-10T05:02:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:29 smithi170 ceph-mon[102832]: pgmap v11137: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1023 B/s wr, 9 op/s 2023-09-10T05:02:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20581.016024 secs 2023-09-10T05:02:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:29 smithi082 ceph-mon[98694]: pgmap v11137: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1023 B/s wr, 9 op/s 2023-09-10T05:02:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20581.016024 secs 2023-09-10T05:02:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:29 smithi098 ceph-mon[103124]: pgmap v11137: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1023 B/s wr, 9 op/s 2023-09-10T05:02:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20581.016024 secs 2023-09-10T05:02:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:31 smithi170 ceph-mon[102832]: pgmap v11138: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 163 KiB/s wr, 14 op/s 2023-09-10T05:02:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:02:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:31 smithi082 ceph-mon[98694]: pgmap v11138: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 163 KiB/s wr, 14 op/s 2023-09-10T05:02:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:02:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:31 smithi098 ceph-mon[103124]: pgmap v11138: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 163 KiB/s wr, 14 op/s 2023-09-10T05:02:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:02:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:02:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:02:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:02:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:02:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:02:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:02:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:02:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:02:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:02:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:33 smithi170 ceph-mon[102832]: pgmap v11139: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T05:02:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20585.361817 secs 2023-09-10T05:02:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:33 smithi082 ceph-mon[98694]: pgmap v11139: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T05:02:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20585.361817 secs 2023-09-10T05:02:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:33 smithi098 ceph-mon[103124]: pgmap v11139: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T05:02:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20585.361817 secs 2023-09-10T05:02:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20586.016086 secs 2023-09-10T05:02:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20586.016086 secs 2023-09-10T05:02:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20586.016086 secs 2023-09-10T05:02:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:35 smithi170 ceph-mon[102832]: pgmap v11140: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T05:02:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:35 smithi082 ceph-mon[98694]: pgmap v11140: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T05:02:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:35 smithi098 ceph-mon[103124]: pgmap v11140: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T05:02:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:37 smithi170 ceph-mon[102832]: pgmap v11141: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 312 KiB/s wr, 16 op/s 2023-09-10T05:02:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:02:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:02:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:37 smithi082 ceph-mon[98694]: pgmap v11141: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 312 KiB/s wr, 16 op/s 2023-09-10T05:02:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:02:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:02:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:37 smithi098 ceph-mon[103124]: pgmap v11141: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 312 KiB/s wr, 16 op/s 2023-09-10T05:02:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:02:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:02:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20590.361876 secs 2023-09-10T05:02:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20590.361876 secs 2023-09-10T05:02:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20590.361876 secs 2023-09-10T05:02:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:39 smithi170 ceph-mon[102832]: pgmap v11142: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 312 KiB/s wr, 11 op/s 2023-09-10T05:02:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20591.016150 secs 2023-09-10T05:02:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:39 smithi082 ceph-mon[98694]: pgmap v11142: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 312 KiB/s wr, 11 op/s 2023-09-10T05:02:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20591.016150 secs 2023-09-10T05:02:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:39 smithi098 ceph-mon[103124]: pgmap v11142: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 312 KiB/s wr, 11 op/s 2023-09-10T05:02:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20591.016150 secs 2023-09-10T05:02:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:02:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:02:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:02:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:02:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:02:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:02:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:41 smithi170 ceph-mon[102832]: pgmap v11143: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 436 KiB/s wr, 18 op/s 2023-09-10T05:02:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:41 smithi082 ceph-mon[98694]: pgmap v11143: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 436 KiB/s wr, 18 op/s 2023-09-10T05:02:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:41 smithi098 ceph-mon[103124]: pgmap v11143: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 436 KiB/s wr, 18 op/s 2023-09-10T05:02:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:43 smithi170 ceph-mon[102832]: pgmap v11144: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 275 KiB/s wr, 13 op/s 2023-09-10T05:02:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20595.361962 secs 2023-09-10T05:02:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:43 smithi082 ceph-mon[98694]: pgmap v11144: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 275 KiB/s wr, 13 op/s 2023-09-10T05:02:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20595.361962 secs 2023-09-10T05:02:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:43 smithi098 ceph-mon[103124]: pgmap v11144: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 275 KiB/s wr, 13 op/s 2023-09-10T05:02:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20595.361962 secs 2023-09-10T05:02:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20596.016210 secs 2023-09-10T05:02:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20596.016210 secs 2023-09-10T05:02:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20596.016210 secs 2023-09-10T05:02:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:45 smithi170 ceph-mon[102832]: pgmap v11145: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 275 KiB/s wr, 13 op/s 2023-09-10T05:02:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:45 smithi082 ceph-mon[98694]: pgmap v11145: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 275 KiB/s wr, 13 op/s 2023-09-10T05:02:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:45 smithi098 ceph-mon[103124]: pgmap v11145: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 275 KiB/s wr, 13 op/s 2023-09-10T05:02:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:47 smithi170 ceph-mon[102832]: pgmap v11146: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 490 KiB/s wr, 19 op/s 2023-09-10T05:02:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:47 smithi082 ceph-mon[98694]: pgmap v11146: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 490 KiB/s wr, 19 op/s 2023-09-10T05:02:48.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:47 smithi098 ceph-mon[103124]: pgmap v11146: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 490 KiB/s wr, 19 op/s 2023-09-10T05:02:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20599.275104 secs 2023-09-10T05:02:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20600.362021 secs 2023-09-10T05:02:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20599.275104 secs 2023-09-10T05:02:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20600.362021 secs 2023-09-10T05:02:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20599.275104 secs 2023-09-10T05:02:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20600.362021 secs 2023-09-10T05:02:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:49 smithi170 ceph-mon[102832]: pgmap v11147: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 340 KiB/s wr, 13 op/s 2023-09-10T05:02:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20601.016276 secs 2023-09-10T05:02:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:02:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:02:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:49 smithi082 ceph-mon[98694]: pgmap v11147: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 340 KiB/s wr, 13 op/s 2023-09-10T05:02:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20601.016276 secs 2023-09-10T05:02:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:02:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:02:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:49 smithi098 ceph-mon[103124]: pgmap v11147: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 340 KiB/s wr, 13 op/s 2023-09-10T05:02:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20601.016276 secs 2023-09-10T05:02:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:02:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:02:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:50 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:02:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:02:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:02:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:50 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:02:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:02:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:02:51.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:50 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:02:51.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:02:51.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:02:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:51 smithi082 ceph-mon[98694]: pgmap v11148: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 469 KiB/s wr, 19 op/s 2023-09-10T05:02:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:51 smithi098 ceph-mon[103124]: pgmap v11148: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 469 KiB/s wr, 19 op/s 2023-09-10T05:02:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:51 smithi170 ceph-mon[102832]: pgmap v11148: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 469 KiB/s wr, 19 op/s 2023-09-10T05:02:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:53 smithi082 ceph-mon[98694]: pgmap v11149: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 345 KiB/s wr, 12 op/s 2023-09-10T05:02:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20604.275194 secs 2023-09-10T05:02:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20605.362079 secs 2023-09-10T05:02:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20606.016336 secs 2023-09-10T05:02:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:53 smithi098 ceph-mon[103124]: pgmap v11149: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 345 KiB/s wr, 12 op/s 2023-09-10T05:02:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20604.275194 secs 2023-09-10T05:02:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20605.362079 secs 2023-09-10T05:02:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20606.016336 secs 2023-09-10T05:02:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:53 smithi170 ceph-mon[102832]: pgmap v11149: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 345 KiB/s wr, 12 op/s 2023-09-10T05:02:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20604.275194 secs 2023-09-10T05:02:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20605.362079 secs 2023-09-10T05:02:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20606.016336 secs 2023-09-10T05:02:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:55 smithi170 ceph-mon[102832]: pgmap v11150: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 345 KiB/s wr, 12 op/s 2023-09-10T05:02:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:55 smithi082 ceph-mon[98694]: pgmap v11150: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 345 KiB/s wr, 12 op/s 2023-09-10T05:02:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:55 smithi098 ceph-mon[103124]: pgmap v11150: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 345 KiB/s wr, 12 op/s 2023-09-10T05:02:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:57 smithi082 ceph-mon[98694]: pgmap v11151: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 488 KiB/s wr, 17 op/s 2023-09-10T05:02:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:57 smithi098 ceph-mon[103124]: pgmap v11151: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 488 KiB/s wr, 17 op/s 2023-09-10T05:02:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:57 smithi170 ceph-mon[102832]: pgmap v11151: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 488 KiB/s wr, 17 op/s 2023-09-10T05:02:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20609.275253 secs 2023-09-10T05:02:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20610.362137 secs 2023-09-10T05:02:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20609.275253 secs 2023-09-10T05:02:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20610.362137 secs 2023-09-10T05:02:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20609.275253 secs 2023-09-10T05:02:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20610.362137 secs 2023-09-10T05:02:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:59 smithi082 ceph-mon[98694]: pgmap v11152: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 273 KiB/s wr, 11 op/s 2023-09-10T05:02:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:02:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20611.016403 secs 2023-09-10T05:02:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:59 smithi098 ceph-mon[103124]: pgmap v11152: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 273 KiB/s wr, 11 op/s 2023-09-10T05:02:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:02:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20611.016403 secs 2023-09-10T05:02:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:59 smithi170 ceph-mon[102832]: pgmap v11152: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 273 KiB/s wr, 11 op/s 2023-09-10T05:02:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:02:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20611.016403 secs 2023-09-10T05:03:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:01 smithi098 ceph-mon[103124]: pgmap v11153: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 288 KiB/s wr, 16 op/s 2023-09-10T05:03:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:01 smithi170 ceph-mon[102832]: pgmap v11153: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 288 KiB/s wr, 16 op/s 2023-09-10T05:03:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:01 smithi082 ceph-mon[98694]: pgmap v11153: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 288 KiB/s wr, 16 op/s 2023-09-10T05:03:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:03 smithi098 ceph-mon[103124]: pgmap v11154: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 10 op/s 2023-09-10T05:03:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20614.275346 secs 2023-09-10T05:03:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:03:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20615.362226 secs 2023-09-10T05:03:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:03:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:03:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:03:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:03 smithi170 ceph-mon[102832]: pgmap v11154: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 10 op/s 2023-09-10T05:03:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20614.275346 secs 2023-09-10T05:03:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:03:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20615.362226 secs 2023-09-10T05:03:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:03:03.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:03:03.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:03:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:03 smithi082 ceph-mon[98694]: pgmap v11154: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 10 op/s 2023-09-10T05:03:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20614.275346 secs 2023-09-10T05:03:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:03:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20615.362226 secs 2023-09-10T05:03:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:03:03.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:03:03.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:03:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20616.016463 secs 2023-09-10T05:03:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20616.016463 secs 2023-09-10T05:03:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20616.016463 secs 2023-09-10T05:03:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:05 smithi098 ceph-mon[103124]: pgmap v11155: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 10 op/s 2023-09-10T05:03:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:05 smithi170 ceph-mon[102832]: pgmap v11155: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 10 op/s 2023-09-10T05:03:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:05 smithi082 ceph-mon[98694]: pgmap v11155: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 10 op/s 2023-09-10T05:03:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:07 smithi098 ceph-mon[103124]: pgmap v11156: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 271 KiB/s wr, 30 op/s 2023-09-10T05:03:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:07 smithi170 ceph-mon[102832]: pgmap v11156: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 271 KiB/s wr, 30 op/s 2023-09-10T05:03:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:07 smithi082 ceph-mon[98694]: pgmap v11156: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 271 KiB/s wr, 30 op/s 2023-09-10T05:03:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20619.275406 secs 2023-09-10T05:03:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20620.362308 secs 2023-09-10T05:03:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20619.275406 secs 2023-09-10T05:03:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20620.362308 secs 2023-09-10T05:03:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20619.275406 secs 2023-09-10T05:03:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20620.362308 secs 2023-09-10T05:03:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:09 smithi098 ceph-mon[103124]: pgmap v11157: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 127 KiB/s wr, 25 op/s 2023-09-10T05:03:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:09 smithi098 ceph-mon[103124]: osdmap e765: 12 total, 12 up, 12 in 2023-09-10T05:03:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20621.016487 secs 2023-09-10T05:03:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:09 smithi170 ceph-mon[102832]: pgmap v11157: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 127 KiB/s wr, 25 op/s 2023-09-10T05:03:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:09 smithi170 ceph-mon[102832]: osdmap e765: 12 total, 12 up, 12 in 2023-09-10T05:03:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20621.016487 secs 2023-09-10T05:03:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:09 smithi082 ceph-mon[98694]: pgmap v11157: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 127 KiB/s wr, 25 op/s 2023-09-10T05:03:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:09 smithi082 ceph-mon[98694]: osdmap e765: 12 total, 12 up, 12 in 2023-09-10T05:03:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20621.016487 secs 2023-09-10T05:03:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:11 smithi098 ceph-mon[103124]: pgmap v11159: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 29 op/s 2023-09-10T05:03:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:11 smithi170 ceph-mon[102832]: pgmap v11159: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 29 op/s 2023-09-10T05:03:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:11 smithi082 ceph-mon[98694]: pgmap v11159: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 29 op/s 2023-09-10T05:03:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:13 smithi098 ceph-mon[103124]: pgmap v11160: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 29 op/s 2023-09-10T05:03:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20624.275505 secs 2023-09-10T05:03:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20625.362368 secs 2023-09-10T05:03:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:13 smithi170 ceph-mon[102832]: pgmap v11160: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 29 op/s 2023-09-10T05:03:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20624.275505 secs 2023-09-10T05:03:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20625.362368 secs 2023-09-10T05:03:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:13 smithi082 ceph-mon[98694]: pgmap v11160: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 29 op/s 2023-09-10T05:03:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20624.275505 secs 2023-09-10T05:03:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20625.362368 secs 2023-09-10T05:03:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20626.016553 secs 2023-09-10T05:03:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:14 smithi098 ceph-mon[103124]: osdmap e766: 12 total, 12 up, 12 in 2023-09-10T05:03:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20626.016553 secs 2023-09-10T05:03:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:14 smithi170 ceph-mon[102832]: osdmap e766: 12 total, 12 up, 12 in 2023-09-10T05:03:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20626.016553 secs 2023-09-10T05:03:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:14 smithi082 ceph-mon[98694]: osdmap e766: 12 total, 12 up, 12 in 2023-09-10T05:03:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:15 smithi098 ceph-mon[103124]: pgmap v11162: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2023-09-10T05:03:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:15 smithi170 ceph-mon[102832]: pgmap v11162: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2023-09-10T05:03:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:15 smithi082 ceph-mon[98694]: pgmap v11162: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2023-09-10T05:03:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:17 smithi098 ceph-mon[103124]: pgmap v11163: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 100 KiB/s wr, 15 op/s 2023-09-10T05:03:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:17 smithi170 ceph-mon[102832]: pgmap v11163: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 100 KiB/s wr, 15 op/s 2023-09-10T05:03:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:17 smithi082 ceph-mon[98694]: pgmap v11163: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 100 KiB/s wr, 15 op/s 2023-09-10T05:03:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20629.275624 secs 2023-09-10T05:03:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20630.362452 secs 2023-09-10T05:03:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20629.275624 secs 2023-09-10T05:03:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20630.362452 secs 2023-09-10T05:03:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20629.275624 secs 2023-09-10T05:03:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20630.362452 secs 2023-09-10T05:03:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:19 smithi098 ceph-mon[103124]: pgmap v11164: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 12 op/s 2023-09-10T05:03:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20631.016620 secs 2023-09-10T05:03:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:19 smithi170 ceph-mon[102832]: pgmap v11164: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 12 op/s 2023-09-10T05:03:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20631.016620 secs 2023-09-10T05:03:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:19 smithi082 ceph-mon[98694]: pgmap v11164: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 12 op/s 2023-09-10T05:03:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20631.016620 secs 2023-09-10T05:03:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:21 smithi098 ceph-mon[103124]: pgmap v11165: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T05:03:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:21 smithi170 ceph-mon[102832]: pgmap v11165: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T05:03:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:21 smithi082 ceph-mon[98694]: pgmap v11165: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T05:03:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:23 smithi098 ceph-mon[103124]: pgmap v11166: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T05:03:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20634.275683 secs 2023-09-10T05:03:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20635.362542 secs 2023-09-10T05:03:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:23 smithi170 ceph-mon[102832]: pgmap v11166: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T05:03:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20634.275683 secs 2023-09-10T05:03:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20635.362542 secs 2023-09-10T05:03:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:23 smithi082 ceph-mon[98694]: pgmap v11166: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T05:03:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20634.275683 secs 2023-09-10T05:03:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20635.362542 secs 2023-09-10T05:03:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:03:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:03:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20636.016682 secs 2023-09-10T05:03:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:03:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:03:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20636.016682 secs 2023-09-10T05:03:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:03:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:03:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20636.016682 secs 2023-09-10T05:03:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:25 smithi170 ceph-mon[102832]: pgmap v11167: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T05:03:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:25 smithi082 ceph-mon[98694]: pgmap v11167: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T05:03:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:25 smithi098 ceph-mon[103124]: pgmap v11167: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T05:03:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:26 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:03:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:03:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:26 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:03:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:03:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:26 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:03:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:03:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:27 smithi170 ceph-mon[102832]: pgmap v11168: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 134 KiB/s wr, 15 op/s 2023-09-10T05:03:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:27 smithi082 ceph-mon[98694]: pgmap v11168: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 134 KiB/s wr, 15 op/s 2023-09-10T05:03:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:27 smithi098 ceph-mon[103124]: pgmap v11168: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 134 KiB/s wr, 15 op/s 2023-09-10T05:03:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20640.362602 secs 2023-09-10T05:03:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20640.362602 secs 2023-09-10T05:03:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20640.362602 secs 2023-09-10T05:03:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:29 smithi170 ceph-mon[102832]: pgmap v11169: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 9 op/s 2023-09-10T05:03:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20641.016750 secs 2023-09-10T05:03:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:29 smithi082 ceph-mon[98694]: pgmap v11169: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 9 op/s 2023-09-10T05:03:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20641.016750 secs 2023-09-10T05:03:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:29 smithi098 ceph-mon[103124]: pgmap v11169: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 9 op/s 2023-09-10T05:03:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20641.016750 secs 2023-09-10T05:03:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:31 smithi170 ceph-mon[102832]: pgmap v11170: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 131 KiB/s wr, 14 op/s 2023-09-10T05:03:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:03:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:31 smithi082 ceph-mon[98694]: pgmap v11170: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 131 KiB/s wr, 14 op/s 2023-09-10T05:03:31.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:03:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:31 smithi098 ceph-mon[103124]: pgmap v11170: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 131 KiB/s wr, 14 op/s 2023-09-10T05:03:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:03:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:03:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:03:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:03:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:03:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:03:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:03:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:03:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:03:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:03:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:33 smithi170 ceph-mon[102832]: pgmap v11171: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-09-10T05:03:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20645.362679 secs 2023-09-10T05:03:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:33 smithi082 ceph-mon[98694]: pgmap v11171: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-09-10T05:03:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20645.362679 secs 2023-09-10T05:03:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:33 smithi098 ceph-mon[103124]: pgmap v11171: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-09-10T05:03:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20645.362679 secs 2023-09-10T05:03:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20646.016812 secs 2023-09-10T05:03:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20646.016812 secs 2023-09-10T05:03:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20646.016812 secs 2023-09-10T05:03:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:35 smithi170 ceph-mon[102832]: pgmap v11172: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-09-10T05:03:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:35 smithi082 ceph-mon[98694]: pgmap v11172: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-09-10T05:03:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:35 smithi098 ceph-mon[103124]: pgmap v11172: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-09-10T05:03:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:37 smithi170 ceph-mon[102832]: pgmap v11173: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 272 KiB/s wr, 14 op/s 2023-09-10T05:03:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:37 smithi082 ceph-mon[98694]: pgmap v11173: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 272 KiB/s wr, 14 op/s 2023-09-10T05:03:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:37 smithi098 ceph-mon[103124]: pgmap v11173: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 272 KiB/s wr, 14 op/s 2023-09-10T05:03:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20650.362760 secs 2023-09-10T05:03:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20650.362760 secs 2023-09-10T05:03:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20650.362760 secs 2023-09-10T05:03:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:39 smithi170 ceph-mon[102832]: pgmap v11174: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 205 KiB/s wr, 10 op/s 2023-09-10T05:03:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20651.016879 secs 2023-09-10T05:03:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:39 smithi082 ceph-mon[98694]: pgmap v11174: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 205 KiB/s wr, 10 op/s 2023-09-10T05:03:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20651.016879 secs 2023-09-10T05:03:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:39 smithi098 ceph-mon[103124]: pgmap v11174: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 205 KiB/s wr, 10 op/s 2023-09-10T05:03:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20651.016879 secs 2023-09-10T05:03:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:41 smithi170 ceph-mon[102832]: pgmap v11175: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 250 KiB/s wr, 16 op/s 2023-09-10T05:03:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:41 smithi082 ceph-mon[98694]: pgmap v11175: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 250 KiB/s wr, 16 op/s 2023-09-10T05:03:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:41 smithi098 ceph-mon[103124]: pgmap v11175: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 250 KiB/s wr, 16 op/s 2023-09-10T05:03:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:43 smithi170 ceph-mon[102832]: pgmap v11176: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T05:03:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20655.362819 secs 2023-09-10T05:03:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:43 smithi082 ceph-mon[98694]: pgmap v11176: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T05:03:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20655.362819 secs 2023-09-10T05:03:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:43 smithi098 ceph-mon[103124]: pgmap v11176: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T05:03:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20655.362819 secs 2023-09-10T05:03:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:03:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:03:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20656.016939 secs 2023-09-10T05:03:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:03:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:03:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20656.016939 secs 2023-09-10T05:03:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:03:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:03:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20656.016939 secs 2023-09-10T05:03:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:45 smithi170 ceph-mon[102832]: pgmap v11177: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T05:03:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:45 smithi082 ceph-mon[98694]: pgmap v11177: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T05:03:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:45 smithi098 ceph-mon[103124]: pgmap v11177: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T05:03:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:03:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:03:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:03:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:03:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:03:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:03:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:47 smithi170 ceph-mon[102832]: pgmap v11178: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 406 KiB/s wr, 18 op/s 2023-09-10T05:03:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:47 smithi082 ceph-mon[98694]: pgmap v11178: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 406 KiB/s wr, 18 op/s 2023-09-10T05:03:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:47 smithi098 ceph-mon[103124]: pgmap v11178: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 406 KiB/s wr, 18 op/s 2023-09-10T05:03:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20660.362898 secs 2023-09-10T05:03:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20660.362898 secs 2023-09-10T05:03:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20660.362898 secs 2023-09-10T05:03:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:49 smithi170 ceph-mon[102832]: pgmap v11179: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 266 KiB/s wr, 13 op/s 2023-09-10T05:03:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20661.017005 secs 2023-09-10T05:03:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:49 smithi082 ceph-mon[98694]: pgmap v11179: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 266 KiB/s wr, 13 op/s 2023-09-10T05:03:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20661.017005 secs 2023-09-10T05:03:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:49 smithi098 ceph-mon[103124]: pgmap v11179: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 266 KiB/s wr, 13 op/s 2023-09-10T05:03:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20661.017005 secs 2023-09-10T05:03:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:51 smithi170 ceph-mon[102832]: pgmap v11180: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 403 KiB/s wr, 18 op/s 2023-09-10T05:03:51.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:51 smithi082 ceph-mon[98694]: pgmap v11180: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 403 KiB/s wr, 18 op/s 2023-09-10T05:03:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:51 smithi098 ceph-mon[103124]: pgmap v11180: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 403 KiB/s wr, 18 op/s 2023-09-10T05:03:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:53 smithi170 ceph-mon[102832]: pgmap v11181: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 358 KiB/s wr, 12 op/s 2023-09-10T05:03:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20664.276015 secs 2023-09-10T05:03:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20665.362961 secs 2023-09-10T05:03:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:53 smithi082 ceph-mon[98694]: pgmap v11181: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 358 KiB/s wr, 12 op/s 2023-09-10T05:03:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20664.276015 secs 2023-09-10T05:03:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20665.362961 secs 2023-09-10T05:03:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:53 smithi098 ceph-mon[103124]: pgmap v11181: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 358 KiB/s wr, 12 op/s 2023-09-10T05:03:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20664.276015 secs 2023-09-10T05:03:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20665.362961 secs 2023-09-10T05:03:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:03:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:03:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20666.017066 secs 2023-09-10T05:03:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:03:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:03:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20666.017066 secs 2023-09-10T05:03:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:03:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:03:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20666.017066 secs 2023-09-10T05:03:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:55 smithi170 ceph-mon[102832]: pgmap v11182: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 358 KiB/s wr, 12 op/s 2023-09-10T05:03:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:55 smithi082 ceph-mon[98694]: pgmap v11182: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 358 KiB/s wr, 12 op/s 2023-09-10T05:03:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:55 smithi098 ceph-mon[103124]: pgmap v11182: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 358 KiB/s wr, 12 op/s 2023-09-10T05:03:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:03:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:03:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:03:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:03:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:03:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:03:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:03:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:03:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:03:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:57 smithi170 ceph-mon[102832]: pgmap v11183: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 612 KiB/s wr, 18 op/s 2023-09-10T05:03:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:57 smithi082 ceph-mon[98694]: pgmap v11183: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 612 KiB/s wr, 18 op/s 2023-09-10T05:03:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:57 smithi098 ceph-mon[103124]: pgmap v11183: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 612 KiB/s wr, 18 op/s 2023-09-10T05:03:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20669.276073 secs 2023-09-10T05:03:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20670.363020 secs 2023-09-10T05:03:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20669.276073 secs 2023-09-10T05:03:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20670.363020 secs 2023-09-10T05:03:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20669.276073 secs 2023-09-10T05:03:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20670.363020 secs 2023-09-10T05:03:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:59 smithi170 ceph-mon[102832]: pgmap v11184: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 392 KiB/s wr, 12 op/s 2023-09-10T05:03:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:03:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20671.017131 secs 2023-09-10T05:03:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:59 smithi082 ceph-mon[98694]: pgmap v11184: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 392 KiB/s wr, 12 op/s 2023-09-10T05:03:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:03:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20671.017131 secs 2023-09-10T05:04:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:59 smithi098 ceph-mon[103124]: pgmap v11184: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 392 KiB/s wr, 12 op/s 2023-09-10T05:04:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:03:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20671.017131 secs 2023-09-10T05:04:01.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:01 smithi170 ceph-mon[102832]: pgmap v11185: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 532 KiB/s wr, 17 op/s 2023-09-10T05:04:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:01 smithi082 ceph-mon[98694]: pgmap v11185: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 532 KiB/s wr, 17 op/s 2023-09-10T05:04:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:01 smithi098 ceph-mon[103124]: pgmap v11185: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 532 KiB/s wr, 17 op/s 2023-09-10T05:04:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:03 smithi170 ceph-mon[102832]: pgmap v11186: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 395 KiB/s wr, 12 op/s 2023-09-10T05:04:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20674.276165 secs 2023-09-10T05:04:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20675.363078 secs 2023-09-10T05:04:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:04:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:03 smithi082 ceph-mon[98694]: pgmap v11186: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 395 KiB/s wr, 12 op/s 2023-09-10T05:04:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20674.276165 secs 2023-09-10T05:04:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20675.363078 secs 2023-09-10T05:04:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:04:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:03 smithi098 ceph-mon[103124]: pgmap v11186: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 395 KiB/s wr, 12 op/s 2023-09-10T05:04:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20674.276165 secs 2023-09-10T05:04:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20675.363078 secs 2023-09-10T05:04:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:04:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:04:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:04:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20676.017186 secs 2023-09-10T05:04:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:04:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:04:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:04:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:04:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:04:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:04:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:04:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20676.017186 secs 2023-09-10T05:04:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:04:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:04:04.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:04:04.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:04:04.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:04:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:04:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:04:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20676.017186 secs 2023-09-10T05:04:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:04:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:04:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:04:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:04:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:04:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:05 smithi170 ceph-mon[102832]: pgmap v11187: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 395 KiB/s wr, 12 op/s 2023-09-10T05:04:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:05 smithi082 ceph-mon[98694]: pgmap v11187: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 395 KiB/s wr, 12 op/s 2023-09-10T05:04:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:05 smithi098 ceph-mon[103124]: pgmap v11187: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 395 KiB/s wr, 12 op/s 2023-09-10T05:04:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:07 smithi170 ceph-mon[102832]: pgmap v11188: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 454 KiB/s wr, 32 op/s 2023-09-10T05:04:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:07 smithi082 ceph-mon[98694]: pgmap v11188: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 454 KiB/s wr, 32 op/s 2023-09-10T05:04:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:07 smithi098 ceph-mon[103124]: pgmap v11188: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 454 KiB/s wr, 32 op/s 2023-09-10T05:04:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20679.276227 secs 2023-09-10T05:04:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:08 smithi170 ceph-mon[102832]: osdmap e767: 12 total, 12 up, 12 in 2023-09-10T05:04:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20680.363136 secs 2023-09-10T05:04:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20679.276227 secs 2023-09-10T05:04:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:08 smithi082 ceph-mon[98694]: osdmap e767: 12 total, 12 up, 12 in 2023-09-10T05:04:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20680.363136 secs 2023-09-10T05:04:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20679.276227 secs 2023-09-10T05:04:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:08 smithi098 ceph-mon[103124]: osdmap e767: 12 total, 12 up, 12 in 2023-09-10T05:04:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20680.363136 secs 2023-09-10T05:04:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:09 smithi170 ceph-mon[102832]: pgmap v11190: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 240 KiB/s wr, 30 op/s 2023-09-10T05:04:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:09 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:04:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:09 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:04:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20681.017251 secs 2023-09-10T05:04:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:09 smithi082 ceph-mon[98694]: pgmap v11190: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 240 KiB/s wr, 30 op/s 2023-09-10T05:04:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:09 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:04:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:09 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:04:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20681.017251 secs 2023-09-10T05:04:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:09 smithi098 ceph-mon[103124]: pgmap v11190: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 240 KiB/s wr, 30 op/s 2023-09-10T05:04:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:09 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:04:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:09 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:04:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20681.017251 secs 2023-09-10T05:04:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:11 smithi170 ceph-mon[102832]: pgmap v11191: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 90 KiB/s wr, 30 op/s 2023-09-10T05:04:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:11 smithi082 ceph-mon[98694]: pgmap v11191: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 90 KiB/s wr, 30 op/s 2023-09-10T05:04:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:11 smithi098 ceph-mon[103124]: pgmap v11191: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 90 KiB/s wr, 30 op/s 2023-09-10T05:04:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:13 smithi170 ceph-mon[102832]: pgmap v11192: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 90 KiB/s wr, 30 op/s 2023-09-10T05:04:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20684.276316 secs 2023-09-10T05:04:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20685.363195 secs 2023-09-10T05:04:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:13 smithi082 ceph-mon[98694]: pgmap v11192: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 90 KiB/s wr, 30 op/s 2023-09-10T05:04:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20684.276316 secs 2023-09-10T05:04:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20685.363195 secs 2023-09-10T05:04:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:13 smithi098 ceph-mon[103124]: pgmap v11192: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 90 KiB/s wr, 30 op/s 2023-09-10T05:04:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20684.276316 secs 2023-09-10T05:04:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20685.363195 secs 2023-09-10T05:04:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20686.017313 secs 2023-09-10T05:04:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:14 smithi170 ceph-mon[102832]: osdmap e768: 12 total, 12 up, 12 in 2023-09-10T05:04:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20686.017313 secs 2023-09-10T05:04:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:14 smithi082 ceph-mon[98694]: osdmap e768: 12 total, 12 up, 12 in 2023-09-10T05:04:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20686.017313 secs 2023-09-10T05:04:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:14 smithi098 ceph-mon[103124]: osdmap e768: 12 total, 12 up, 12 in 2023-09-10T05:04:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:15 smithi170 ceph-mon[102832]: pgmap v11194: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 8 op/s 2023-09-10T05:04:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:15 smithi082 ceph-mon[98694]: pgmap v11194: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 8 op/s 2023-09-10T05:04:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:15 smithi098 ceph-mon[103124]: pgmap v11194: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 8 op/s 2023-09-10T05:04:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:17 smithi170 ceph-mon[102832]: pgmap v11195: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 14 op/s 2023-09-10T05:04:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:17 smithi082 ceph-mon[98694]: pgmap v11195: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 14 op/s 2023-09-10T05:04:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:17 smithi098 ceph-mon[103124]: pgmap v11195: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 14 op/s 2023-09-10T05:04:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20689.276376 secs 2023-09-10T05:04:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20690.363253 secs 2023-09-10T05:04:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20689.276376 secs 2023-09-10T05:04:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20690.363253 secs 2023-09-10T05:04:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20689.276376 secs 2023-09-10T05:04:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20690.363253 secs 2023-09-10T05:04:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:19 smithi170 ceph-mon[102832]: pgmap v11196: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-09-10T05:04:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20691.017378 secs 2023-09-10T05:04:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:19 smithi082 ceph-mon[98694]: pgmap v11196: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-09-10T05:04:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20691.017378 secs 2023-09-10T05:04:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:19 smithi098 ceph-mon[103124]: pgmap v11196: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-09-10T05:04:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20691.017378 secs 2023-09-10T05:04:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:21 smithi170 ceph-mon[102832]: pgmap v11197: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T05:04:21.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:21 smithi082 ceph-mon[98694]: pgmap v11197: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T05:04:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:21 smithi098 ceph-mon[103124]: pgmap v11197: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T05:04:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:23 smithi170 ceph-mon[102832]: pgmap v11198: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T05:04:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20694.276435 secs 2023-09-10T05:04:23.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20695.363343 secs 2023-09-10T05:04:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:23 smithi082 ceph-mon[98694]: pgmap v11198: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T05:04:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20694.276435 secs 2023-09-10T05:04:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20695.363343 secs 2023-09-10T05:04:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:23 smithi098 ceph-mon[103124]: pgmap v11198: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T05:04:24.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20694.276435 secs 2023-09-10T05:04:24.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20695.363343 secs 2023-09-10T05:04:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20696.017441 secs 2023-09-10T05:04:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20696.017441 secs 2023-09-10T05:04:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20696.017441 secs 2023-09-10T05:04:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:25 smithi170 ceph-mon[102832]: pgmap v11199: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:04:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:25 smithi082 ceph-mon[98694]: pgmap v11199: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:04:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:25 smithi098 ceph-mon[103124]: pgmap v11199: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:04:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:27 smithi170 ceph-mon[102832]: pgmap v11200: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 68 KiB/s wr, 14 op/s 2023-09-10T05:04:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:27 smithi082 ceph-mon[98694]: pgmap v11200: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 68 KiB/s wr, 14 op/s 2023-09-10T05:04:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:27 smithi098 ceph-mon[103124]: pgmap v11200: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 68 KiB/s wr, 14 op/s 2023-09-10T05:04:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20700.363401 secs 2023-09-10T05:04:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20700.363401 secs 2023-09-10T05:04:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20700.363401 secs 2023-09-10T05:04:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:29 smithi170 ceph-mon[102832]: pgmap v11201: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 68 KiB/s wr, 9 op/s 2023-09-10T05:04:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20701.017489 secs 2023-09-10T05:04:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:29 smithi082 ceph-mon[98694]: pgmap v11201: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 68 KiB/s wr, 9 op/s 2023-09-10T05:04:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20701.017489 secs 2023-09-10T05:04:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:29 smithi098 ceph-mon[103124]: pgmap v11201: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 68 KiB/s wr, 9 op/s 2023-09-10T05:04:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20701.017489 secs 2023-09-10T05:04:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:31 smithi170 ceph-mon[102832]: pgmap v11202: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 232 KiB/s wr, 15 op/s 2023-09-10T05:04:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:04:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:31 smithi082 ceph-mon[98694]: pgmap v11202: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 232 KiB/s wr, 15 op/s 2023-09-10T05:04:31.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:04:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:31 smithi098 ceph-mon[103124]: pgmap v11202: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 232 KiB/s wr, 15 op/s 2023-09-10T05:04:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:04:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:04:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:04:32.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:04:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:04:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:04:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:04:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:04:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:04:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:04:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:33 smithi082 ceph-mon[98694]: pgmap v11203: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 165 KiB/s wr, 10 op/s 2023-09-10T05:04:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20705.363463 secs 2023-09-10T05:04:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:33 smithi098 ceph-mon[103124]: pgmap v11203: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 165 KiB/s wr, 10 op/s 2023-09-10T05:04:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20705.363463 secs 2023-09-10T05:04:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:33 smithi170 ceph-mon[102832]: pgmap v11203: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 165 KiB/s wr, 10 op/s 2023-09-10T05:04:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20705.363463 secs 2023-09-10T05:04:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20706.017554 secs 2023-09-10T05:04:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20706.017554 secs 2023-09-10T05:04:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20706.017554 secs 2023-09-10T05:04:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:35 smithi082 ceph-mon[98694]: pgmap v11204: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 165 KiB/s wr, 10 op/s 2023-09-10T05:04:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:35 smithi098 ceph-mon[103124]: pgmap v11204: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 165 KiB/s wr, 10 op/s 2023-09-10T05:04:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:35 smithi170 ceph-mon[102832]: pgmap v11204: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 165 KiB/s wr, 10 op/s 2023-09-10T05:04:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:37 smithi082 ceph-mon[98694]: pgmap v11205: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 317 KiB/s wr, 17 op/s 2023-09-10T05:04:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:04:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:04:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:37 smithi098 ceph-mon[103124]: pgmap v11205: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 317 KiB/s wr, 17 op/s 2023-09-10T05:04:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:04:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:04:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:37 smithi170 ceph-mon[102832]: pgmap v11205: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 317 KiB/s wr, 17 op/s 2023-09-10T05:04:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:04:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:04:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20710.363551 secs 2023-09-10T05:04:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20710.363551 secs 2023-09-10T05:04:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20710.363551 secs 2023-09-10T05:04:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:39 smithi082 ceph-mon[98694]: pgmap v11206: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 316 KiB/s wr, 12 op/s 2023-09-10T05:04:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20711.017621 secs 2023-09-10T05:04:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:39 smithi098 ceph-mon[103124]: pgmap v11206: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 316 KiB/s wr, 12 op/s 2023-09-10T05:04:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20711.017621 secs 2023-09-10T05:04:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:39 smithi170 ceph-mon[102832]: pgmap v11206: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 316 KiB/s wr, 12 op/s 2023-09-10T05:04:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20711.017621 secs 2023-09-10T05:04:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:04:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:04:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:04:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:04:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:04:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:04:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:41 smithi098 ceph-mon[103124]: pgmap v11207: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 441 KiB/s wr, 19 op/s 2023-09-10T05:04:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:41 smithi170 ceph-mon[102832]: pgmap v11207: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 441 KiB/s wr, 19 op/s 2023-09-10T05:04:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:41 smithi082 ceph-mon[98694]: pgmap v11207: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 441 KiB/s wr, 19 op/s 2023-09-10T05:04:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:43 smithi098 ceph-mon[103124]: pgmap v11208: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 13 op/s 2023-09-10T05:04:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20715.363634 secs 2023-09-10T05:04:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:43 smithi170 ceph-mon[102832]: pgmap v11208: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 13 op/s 2023-09-10T05:04:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20715.363634 secs 2023-09-10T05:04:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:43 smithi082 ceph-mon[98694]: pgmap v11208: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 13 op/s 2023-09-10T05:04:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20715.363634 secs 2023-09-10T05:04:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20716.017684 secs 2023-09-10T05:04:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20716.017684 secs 2023-09-10T05:04:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20716.017684 secs 2023-09-10T05:04:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:45 smithi098 ceph-mon[103124]: pgmap v11209: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 13 op/s 2023-09-10T05:04:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:45 smithi170 ceph-mon[102832]: pgmap v11209: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 13 op/s 2023-09-10T05:04:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:45 smithi082 ceph-mon[98694]: pgmap v11209: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 13 op/s 2023-09-10T05:04:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:47 smithi098 ceph-mon[103124]: pgmap v11210: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 496 KiB/s wr, 20 op/s 2023-09-10T05:04:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:47 smithi170 ceph-mon[102832]: pgmap v11210: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 496 KiB/s wr, 20 op/s 2023-09-10T05:04:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:47 smithi082 ceph-mon[98694]: pgmap v11210: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 496 KiB/s wr, 20 op/s 2023-09-10T05:04:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20719.276738 secs 2023-09-10T05:04:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20720.363693 secs 2023-09-10T05:04:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20719.276738 secs 2023-09-10T05:04:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20720.363693 secs 2023-09-10T05:04:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20719.276738 secs 2023-09-10T05:04:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20720.363693 secs 2023-09-10T05:04:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:49 smithi098 ceph-mon[103124]: pgmap v11211: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 345 KiB/s wr, 13 op/s 2023-09-10T05:04:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:04:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:04:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20721.017749 secs 2023-09-10T05:04:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:49 smithi170 ceph-mon[102832]: pgmap v11211: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 345 KiB/s wr, 13 op/s 2023-09-10T05:04:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:04:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:04:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20721.017749 secs 2023-09-10T05:04:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:49 smithi082 ceph-mon[98694]: pgmap v11211: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 345 KiB/s wr, 13 op/s 2023-09-10T05:04:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:04:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:04:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20721.017749 secs 2023-09-10T05:04:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:51 smithi098 ceph-mon[103124]: pgmap v11212: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 474 KiB/s wr, 19 op/s 2023-09-10T05:04:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:51 smithi170 ceph-mon[102832]: pgmap v11212: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 474 KiB/s wr, 19 op/s 2023-09-10T05:04:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:51 smithi082 ceph-mon[98694]: pgmap v11212: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 474 KiB/s wr, 19 op/s 2023-09-10T05:04:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:04:53.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:04:53.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:04:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:04:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:04:53.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:04:53.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:04:53.212 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:04:53.212 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:04:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:53 smithi098 ceph-mon[103124]: pgmap v11213: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 348 KiB/s wr, 12 op/s 2023-09-10T05:04:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20724.276774 secs 2023-09-10T05:04:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20725.363753 secs 2023-09-10T05:04:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:53 smithi170 ceph-mon[102832]: pgmap v11213: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 348 KiB/s wr, 12 op/s 2023-09-10T05:04:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20724.276774 secs 2023-09-10T05:04:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20725.363753 secs 2023-09-10T05:04:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:53 smithi082 ceph-mon[98694]: pgmap v11213: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 348 KiB/s wr, 12 op/s 2023-09-10T05:04:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20724.276774 secs 2023-09-10T05:04:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20725.363753 secs 2023-09-10T05:04:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20726.017813 secs 2023-09-10T05:04:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20726.017813 secs 2023-09-10T05:04:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20726.017813 secs 2023-09-10T05:04:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:55 smithi098 ceph-mon[103124]: pgmap v11214: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 348 KiB/s wr, 12 op/s 2023-09-10T05:04:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:55 smithi170 ceph-mon[102832]: pgmap v11214: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 348 KiB/s wr, 12 op/s 2023-09-10T05:04:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:55 smithi082 ceph-mon[98694]: pgmap v11214: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 348 KiB/s wr, 12 op/s 2023-09-10T05:04:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:57 smithi098 ceph-mon[103124]: pgmap v11215: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 490 KiB/s wr, 18 op/s 2023-09-10T05:04:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:57 smithi170 ceph-mon[102832]: pgmap v11215: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 490 KiB/s wr, 18 op/s 2023-09-10T05:04:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:57 smithi082 ceph-mon[98694]: pgmap v11215: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 490 KiB/s wr, 18 op/s 2023-09-10T05:04:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20729.276834 secs 2023-09-10T05:04:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20730.363837 secs 2023-09-10T05:04:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20729.276834 secs 2023-09-10T05:04:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20730.363837 secs 2023-09-10T05:04:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20729.276834 secs 2023-09-10T05:04:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20730.363837 secs 2023-09-10T05:05:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:59 smithi098 ceph-mon[103124]: pgmap v11216: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 271 KiB/s wr, 11 op/s 2023-09-10T05:05:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:04:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20731.017879 secs 2023-09-10T05:05:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:59 smithi170 ceph-mon[102832]: pgmap v11216: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 271 KiB/s wr, 11 op/s 2023-09-10T05:05:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:04:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20731.017879 secs 2023-09-10T05:05:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:59 smithi082 ceph-mon[98694]: pgmap v11216: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 271 KiB/s wr, 11 op/s 2023-09-10T05:05:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:04:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20731.017879 secs 2023-09-10T05:05:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:01 smithi098 ceph-mon[103124]: pgmap v11217: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 315 KiB/s wr, 26 op/s 2023-09-10T05:05:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:01 smithi170 ceph-mon[102832]: pgmap v11217: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 315 KiB/s wr, 26 op/s 2023-09-10T05:05:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:01 smithi082 ceph-mon[98694]: pgmap v11217: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 315 KiB/s wr, 26 op/s 2023-09-10T05:05:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:03 smithi098 ceph-mon[103124]: pgmap v11218: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 186 KiB/s wr, 20 op/s 2023-09-10T05:05:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20734.276921 secs 2023-09-10T05:05:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:03 smithi098 ceph-mon[103124]: osdmap e769: 12 total, 12 up, 12 in 2023-09-10T05:05:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20735.363897 secs 2023-09-10T05:05:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:03 smithi170 ceph-mon[102832]: pgmap v11218: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 186 KiB/s wr, 20 op/s 2023-09-10T05:05:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20734.276921 secs 2023-09-10T05:05:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:03 smithi170 ceph-mon[102832]: osdmap e769: 12 total, 12 up, 12 in 2023-09-10T05:05:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20735.363897 secs 2023-09-10T05:05:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:03 smithi082 ceph-mon[98694]: pgmap v11218: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 186 KiB/s wr, 20 op/s 2023-09-10T05:05:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20734.276921 secs 2023-09-10T05:05:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:03 smithi082 ceph-mon[98694]: osdmap e769: 12 total, 12 up, 12 in 2023-09-10T05:05:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20735.363897 secs 2023-09-10T05:05:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20736.017942 secs 2023-09-10T05:05:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:05:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20736.017942 secs 2023-09-10T05:05:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:05:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20736.017942 secs 2023-09-10T05:05:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:05:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:05 smithi098 ceph-mon[103124]: pgmap v11220: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 223 KiB/s wr, 24 op/s 2023-09-10T05:05:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:05:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:05:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:05:06.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:05:06.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:05:06.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:05:06.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:05:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:05 smithi170 ceph-mon[102832]: pgmap v11220: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 223 KiB/s wr, 24 op/s 2023-09-10T05:05:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:05:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:05:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:05:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:05:06.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:05:06.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:05:06.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:05:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:05 smithi082 ceph-mon[98694]: pgmap v11220: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 223 KiB/s wr, 24 op/s 2023-09-10T05:05:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:05:06.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:05:06.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:05:06.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:05:06.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:05:06.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:05:06.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:05:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:07 smithi170 ceph-mon[102832]: pgmap v11221: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 200 KiB/s wr, 30 op/s 2023-09-10T05:05:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:07 smithi082 ceph-mon[98694]: pgmap v11221: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 200 KiB/s wr, 30 op/s 2023-09-10T05:05:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:07 smithi098 ceph-mon[103124]: pgmap v11221: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 200 KiB/s wr, 30 op/s 2023-09-10T05:05:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20739.276980 secs 2023-09-10T05:05:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20740.363954 secs 2023-09-10T05:05:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20739.276980 secs 2023-09-10T05:05:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20740.363954 secs 2023-09-10T05:05:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20739.276980 secs 2023-09-10T05:05:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20740.363954 secs 2023-09-10T05:05:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:09 smithi170 ceph-mon[102832]: pgmap v11222: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 200 KiB/s wr, 30 op/s 2023-09-10T05:05:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20741.018008 secs 2023-09-10T05:05:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:09 smithi170 ceph-mon[102832]: osdmap e770: 12 total, 12 up, 12 in 2023-09-10T05:05:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:09 smithi082 ceph-mon[98694]: pgmap v11222: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 200 KiB/s wr, 30 op/s 2023-09-10T05:05:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20741.018008 secs 2023-09-10T05:05:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:09 smithi082 ceph-mon[98694]: osdmap e770: 12 total, 12 up, 12 in 2023-09-10T05:05:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:09 smithi098 ceph-mon[103124]: pgmap v11222: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 200 KiB/s wr, 30 op/s 2023-09-10T05:05:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20741.018008 secs 2023-09-10T05:05:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:09 smithi098 ceph-mon[103124]: osdmap e770: 12 total, 12 up, 12 in 2023-09-10T05:05:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:11 smithi170 ceph-mon[102832]: pgmap v11224: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 185 KiB/s wr, 24 op/s 2023-09-10T05:05:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:11 smithi082 ceph-mon[98694]: pgmap v11224: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 185 KiB/s wr, 24 op/s 2023-09-10T05:05:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:11 smithi098 ceph-mon[103124]: pgmap v11224: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 185 KiB/s wr, 24 op/s 2023-09-10T05:05:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:05:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:05:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:05:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:05:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:05:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:05:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:13 smithi170 ceph-mon[102832]: pgmap v11225: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 153 KiB/s wr, 20 op/s 2023-09-10T05:05:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20744.277038 secs 2023-09-10T05:05:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:05:14.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:05:14.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20745.364011 secs 2023-09-10T05:05:14.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:13 smithi082 ceph-mon[98694]: pgmap v11225: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 153 KiB/s wr, 20 op/s 2023-09-10T05:05:14.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20744.277038 secs 2023-09-10T05:05:14.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:05:14.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:05:14.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20745.364011 secs 2023-09-10T05:05:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:13 smithi098 ceph-mon[103124]: pgmap v11225: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 153 KiB/s wr, 20 op/s 2023-09-10T05:05:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20744.277038 secs 2023-09-10T05:05:14.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:05:14.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:05:14.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20745.364011 secs 2023-09-10T05:05:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20746.018073 secs 2023-09-10T05:05:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20746.018073 secs 2023-09-10T05:05:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20746.018073 secs 2023-09-10T05:05:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:15 smithi170 ceph-mon[102832]: pgmap v11226: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 148 KiB/s wr, 19 op/s 2023-09-10T05:05:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:15 smithi082 ceph-mon[98694]: pgmap v11226: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 148 KiB/s wr, 19 op/s 2023-09-10T05:05:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:15 smithi098 ceph-mon[103124]: pgmap v11226: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 148 KiB/s wr, 19 op/s 2023-09-10T05:05:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:17 smithi170 ceph-mon[102832]: pgmap v11227: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 12 op/s 2023-09-10T05:05:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:17 smithi082 ceph-mon[98694]: pgmap v11227: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 12 op/s 2023-09-10T05:05:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:17 smithi098 ceph-mon[103124]: pgmap v11227: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 12 op/s 2023-09-10T05:05:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20749.277097 secs 2023-09-10T05:05:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20750.364089 secs 2023-09-10T05:05:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20749.277097 secs 2023-09-10T05:05:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20750.364089 secs 2023-09-10T05:05:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20749.277097 secs 2023-09-10T05:05:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20750.364089 secs 2023-09-10T05:05:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:19 smithi170 ceph-mon[102832]: pgmap v11228: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 12 op/s 2023-09-10T05:05:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20751.018138 secs 2023-09-10T05:05:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:19 smithi082 ceph-mon[98694]: pgmap v11228: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 12 op/s 2023-09-10T05:05:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20751.018138 secs 2023-09-10T05:05:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:19 smithi098 ceph-mon[103124]: pgmap v11228: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 12 op/s 2023-09-10T05:05:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20751.018138 secs 2023-09-10T05:05:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:21 smithi170 ceph-mon[102832]: pgmap v11229: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 74 KiB/s wr, 16 op/s 2023-09-10T05:05:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:21 smithi082 ceph-mon[98694]: pgmap v11229: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 74 KiB/s wr, 16 op/s 2023-09-10T05:05:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:21 smithi098 ceph-mon[103124]: pgmap v11229: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 74 KiB/s wr, 16 op/s 2023-09-10T05:05:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:23 smithi170 ceph-mon[102832]: pgmap v11230: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T05:05:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20754.277158 secs 2023-09-10T05:05:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20755.364147 secs 2023-09-10T05:05:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:23 smithi082 ceph-mon[98694]: pgmap v11230: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T05:05:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20754.277158 secs 2023-09-10T05:05:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20755.364147 secs 2023-09-10T05:05:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:23 smithi098 ceph-mon[103124]: pgmap v11230: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T05:05:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20754.277158 secs 2023-09-10T05:05:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20755.364147 secs 2023-09-10T05:05:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20756.018200 secs 2023-09-10T05:05:25.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20756.018200 secs 2023-09-10T05:05:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20756.018200 secs 2023-09-10T05:05:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:25 smithi170 ceph-mon[102832]: pgmap v11231: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T05:05:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:25 smithi082 ceph-mon[98694]: pgmap v11231: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T05:05:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:25 smithi098 ceph-mon[103124]: pgmap v11231: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T05:05:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:27 smithi170 ceph-mon[102832]: pgmap v11232: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 68 KiB/s wr, 15 op/s 2023-09-10T05:05:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:27 smithi082 ceph-mon[98694]: pgmap v11232: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 68 KiB/s wr, 15 op/s 2023-09-10T05:05:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:27 smithi098 ceph-mon[103124]: pgmap v11232: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 68 KiB/s wr, 15 op/s 2023-09-10T05:05:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20760.364250 secs 2023-09-10T05:05:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20760.364250 secs 2023-09-10T05:05:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20760.364250 secs 2023-09-10T05:05:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:29 smithi170 ceph-mon[102832]: pgmap v11233: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2023-09-10T05:05:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20761.018268 secs 2023-09-10T05:05:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:29 smithi082 ceph-mon[98694]: pgmap v11233: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2023-09-10T05:05:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20761.018268 secs 2023-09-10T05:05:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:29 smithi098 ceph-mon[103124]: pgmap v11233: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2023-09-10T05:05:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20761.018268 secs 2023-09-10T05:05:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:31 smithi170 ceph-mon[102832]: pgmap v11234: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 152 KiB/s wr, 16 op/s 2023-09-10T05:05:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:05:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:31 smithi082 ceph-mon[98694]: pgmap v11234: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 152 KiB/s wr, 16 op/s 2023-09-10T05:05:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:05:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:31 smithi098 ceph-mon[103124]: pgmap v11234: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 152 KiB/s wr, 16 op/s 2023-09-10T05:05:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:05:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:05:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:05:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:05:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:05:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:05:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:05:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:05:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:05:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:05:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:33 smithi170 ceph-mon[102832]: pgmap v11235: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T05:05:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20765.364311 secs 2023-09-10T05:05:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:33 smithi082 ceph-mon[98694]: pgmap v11235: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T05:05:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20765.364311 secs 2023-09-10T05:05:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:33 smithi098 ceph-mon[103124]: pgmap v11235: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T05:05:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20765.364311 secs 2023-09-10T05:05:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20766.018333 secs 2023-09-10T05:05:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20766.018333 secs 2023-09-10T05:05:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20766.018333 secs 2023-09-10T05:05:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:35 smithi170 ceph-mon[102832]: pgmap v11236: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T05:05:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:35 smithi082 ceph-mon[98694]: pgmap v11236: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T05:05:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:35 smithi098 ceph-mon[103124]: pgmap v11236: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T05:05:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:37 smithi170 ceph-mon[102832]: pgmap v11237: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 304 KiB/s wr, 17 op/s 2023-09-10T05:05:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:05:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:05:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:37 smithi082 ceph-mon[98694]: pgmap v11237: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 304 KiB/s wr, 17 op/s 2023-09-10T05:05:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:05:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:05:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:37 smithi098 ceph-mon[103124]: pgmap v11237: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 304 KiB/s wr, 17 op/s 2023-09-10T05:05:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:05:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:05:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20770.364368 secs 2023-09-10T05:05:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20770.364368 secs 2023-09-10T05:05:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20770.364368 secs 2023-09-10T05:05:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:39 smithi098 ceph-mon[103124]: pgmap v11238: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 303 KiB/s wr, 11 op/s 2023-09-10T05:05:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20771.018400 secs 2023-09-10T05:05:40.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:39 smithi170 ceph-mon[102832]: pgmap v11238: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 303 KiB/s wr, 11 op/s 2023-09-10T05:05:40.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20771.018400 secs 2023-09-10T05:05:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:39 smithi082 ceph-mon[98694]: pgmap v11238: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 303 KiB/s wr, 11 op/s 2023-09-10T05:05:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20771.018400 secs 2023-09-10T05:05:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:05:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:05:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:05:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:05:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:05:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:05:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:41 smithi170 ceph-mon[102832]: pgmap v11239: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 409 KiB/s wr, 18 op/s 2023-09-10T05:05:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:41 smithi082 ceph-mon[98694]: pgmap v11239: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 409 KiB/s wr, 18 op/s 2023-09-10T05:05:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:41 smithi098 ceph-mon[103124]: pgmap v11239: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 409 KiB/s wr, 18 op/s 2023-09-10T05:05:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:43 smithi170 ceph-mon[102832]: pgmap v11240: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 258 KiB/s wr, 12 op/s 2023-09-10T05:05:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20775.364473 secs 2023-09-10T05:05:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:43 smithi082 ceph-mon[98694]: pgmap v11240: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 258 KiB/s wr, 12 op/s 2023-09-10T05:05:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20775.364473 secs 2023-09-10T05:05:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:43 smithi098 ceph-mon[103124]: pgmap v11240: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 258 KiB/s wr, 12 op/s 2023-09-10T05:05:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20775.364473 secs 2023-09-10T05:05:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20776.018461 secs 2023-09-10T05:05:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20776.018461 secs 2023-09-10T05:05:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20776.018461 secs 2023-09-10T05:05:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:45 smithi170 ceph-mon[102832]: pgmap v11241: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 258 KiB/s wr, 12 op/s 2023-09-10T05:05:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:45 smithi082 ceph-mon[98694]: pgmap v11241: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 258 KiB/s wr, 12 op/s 2023-09-10T05:05:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:45 smithi098 ceph-mon[103124]: pgmap v11241: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 258 KiB/s wr, 12 op/s 2023-09-10T05:05:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:47 smithi170 ceph-mon[102832]: pgmap v11242: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 393 KiB/s wr, 18 op/s 2023-09-10T05:05:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:47 smithi082 ceph-mon[98694]: pgmap v11242: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 393 KiB/s wr, 18 op/s 2023-09-10T05:05:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:47 smithi098 ceph-mon[103124]: pgmap v11242: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 393 KiB/s wr, 18 op/s 2023-09-10T05:05:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20780.364552 secs 2023-09-10T05:05:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20780.364552 secs 2023-09-10T05:05:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20780.364552 secs 2023-09-10T05:05:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:49 smithi170 ceph-mon[102832]: pgmap v11243: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 241 KiB/s wr, 12 op/s 2023-09-10T05:05:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20781.018490 secs 2023-09-10T05:05:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:49 smithi082 ceph-mon[98694]: pgmap v11243: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 241 KiB/s wr, 12 op/s 2023-09-10T05:05:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20781.018490 secs 2023-09-10T05:05:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:49 smithi098 ceph-mon[103124]: pgmap v11243: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 241 KiB/s wr, 12 op/s 2023-09-10T05:05:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20781.018490 secs 2023-09-10T05:05:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:51 smithi170 ceph-mon[102832]: pgmap v11244: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 382 KiB/s wr, 18 op/s 2023-09-10T05:05:52.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:51 smithi082 ceph-mon[98694]: pgmap v11244: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 382 KiB/s wr, 18 op/s 2023-09-10T05:05:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:51 smithi098 ceph-mon[103124]: pgmap v11244: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 382 KiB/s wr, 18 op/s 2023-09-10T05:05:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:05:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:05:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:05:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:05:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:05:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:05:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:53 smithi170 ceph-mon[102832]: pgmap v11245: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 276 KiB/s wr, 12 op/s 2023-09-10T05:05:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20784.277563 secs 2023-09-10T05:05:54.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:05:54.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:05:54.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20785.364604 secs 2023-09-10T05:05:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:53 smithi082 ceph-mon[98694]: pgmap v11245: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 276 KiB/s wr, 12 op/s 2023-09-10T05:05:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20784.277563 secs 2023-09-10T05:05:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:05:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:05:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20785.364604 secs 2023-09-10T05:05:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:53 smithi098 ceph-mon[103124]: pgmap v11245: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 276 KiB/s wr, 12 op/s 2023-09-10T05:05:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20784.277563 secs 2023-09-10T05:05:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:05:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:05:54.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20785.364604 secs 2023-09-10T05:05:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20786.018552 secs 2023-09-10T05:05:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20786.018552 secs 2023-09-10T05:05:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20786.018552 secs 2023-09-10T05:05:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:55 smithi170 ceph-mon[102832]: pgmap v11246: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 276 KiB/s wr, 12 op/s 2023-09-10T05:05:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:05:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:05:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:05:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:55 smithi082 ceph-mon[98694]: pgmap v11246: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 276 KiB/s wr, 12 op/s 2023-09-10T05:05:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:05:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:05:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:05:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:55 smithi098 ceph-mon[103124]: pgmap v11246: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 276 KiB/s wr, 12 op/s 2023-09-10T05:05:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:05:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:05:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:05:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:57 smithi170 ceph-mon[102832]: pgmap v11247: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 413 KiB/s wr, 17 op/s 2023-09-10T05:05:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:57 smithi082 ceph-mon[98694]: pgmap v11247: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 413 KiB/s wr, 17 op/s 2023-09-10T05:05:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:57 smithi098 ceph-mon[103124]: pgmap v11247: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 413 KiB/s wr, 17 op/s 2023-09-10T05:05:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20789.277622 secs 2023-09-10T05:05:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20790.364692 secs 2023-09-10T05:05:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20789.277622 secs 2023-09-10T05:05:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20790.364692 secs 2023-09-10T05:05:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20789.277622 secs 2023-09-10T05:05:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20790.364692 secs 2023-09-10T05:06:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:59 smithi170 ceph-mon[102832]: pgmap v11248: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T05:06:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:05:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20791.018617 secs 2023-09-10T05:06:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:59 smithi082 ceph-mon[98694]: pgmap v11248: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T05:06:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:05:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20791.018617 secs 2023-09-10T05:06:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:59 smithi098 ceph-mon[103124]: pgmap v11248: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T05:06:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:05:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20791.018617 secs 2023-09-10T05:06:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:01 smithi170 ceph-mon[102832]: pgmap v11249: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 375 KiB/s wr, 17 op/s 2023-09-10T05:06:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:01 smithi082 ceph-mon[98694]: pgmap v11249: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 375 KiB/s wr, 17 op/s 2023-09-10T05:06:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:01 smithi098 ceph-mon[103124]: pgmap v11249: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 375 KiB/s wr, 17 op/s 2023-09-10T05:06:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:03 smithi170 ceph-mon[102832]: pgmap v11250: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 234 KiB/s wr, 11 op/s 2023-09-10T05:06:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20794.277695 secs 2023-09-10T05:06:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20795.364752 secs 2023-09-10T05:06:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:03 smithi082 ceph-mon[98694]: pgmap v11250: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 234 KiB/s wr, 11 op/s 2023-09-10T05:06:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20794.277695 secs 2023-09-10T05:06:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20795.364752 secs 2023-09-10T05:06:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:03 smithi098 ceph-mon[103124]: pgmap v11250: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 234 KiB/s wr, 11 op/s 2023-09-10T05:06:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20794.277695 secs 2023-09-10T05:06:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20795.364752 secs 2023-09-10T05:06:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20796.018678 secs 2023-09-10T05:06:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20796.018678 secs 2023-09-10T05:06:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20796.018678 secs 2023-09-10T05:06:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:05 smithi170 ceph-mon[102832]: pgmap v11251: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 234 KiB/s wr, 11 op/s 2023-09-10T05:06:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:06:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:05 smithi082 ceph-mon[98694]: pgmap v11251: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 234 KiB/s wr, 11 op/s 2023-09-10T05:06:06.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:06:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:05 smithi098 ceph-mon[103124]: pgmap v11251: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 234 KiB/s wr, 11 op/s 2023-09-10T05:06:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:06:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:06:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:06:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:06:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:06:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:06:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:06:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:06:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:06:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:06:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:07 smithi170 ceph-mon[102832]: pgmap v11252: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 346 KiB/s wr, 31 op/s 2023-09-10T05:06:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:07 smithi082 ceph-mon[98694]: pgmap v11252: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 346 KiB/s wr, 31 op/s 2023-09-10T05:06:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:07 smithi098 ceph-mon[103124]: pgmap v11252: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 346 KiB/s wr, 31 op/s 2023-09-10T05:06:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20799.277753 secs 2023-09-10T05:06:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20800.364836 secs 2023-09-10T05:06:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:08 smithi170 ceph-mon[102832]: osdmap e771: 12 total, 12 up, 12 in 2023-09-10T05:06:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20799.277753 secs 2023-09-10T05:06:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20800.364836 secs 2023-09-10T05:06:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:08 smithi082 ceph-mon[98694]: osdmap e771: 12 total, 12 up, 12 in 2023-09-10T05:06:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20799.277753 secs 2023-09-10T05:06:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20800.364836 secs 2023-09-10T05:06:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:08 smithi098 ceph-mon[103124]: osdmap e771: 12 total, 12 up, 12 in 2023-09-10T05:06:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:09 smithi170 ceph-mon[102832]: pgmap v11254: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 252 KiB/s wr, 30 op/s 2023-09-10T05:06:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20801.018744 secs 2023-09-10T05:06:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:09 smithi082 ceph-mon[98694]: pgmap v11254: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 252 KiB/s wr, 30 op/s 2023-09-10T05:06:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20801.018744 secs 2023-09-10T05:06:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:09 smithi098 ceph-mon[103124]: pgmap v11254: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 252 KiB/s wr, 30 op/s 2023-09-10T05:06:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20801.018744 secs 2023-09-10T05:06:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:11 smithi170 ceph-mon[102832]: pgmap v11255: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 136 KiB/s wr, 29 op/s 2023-09-10T05:06:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:11 smithi082 ceph-mon[98694]: pgmap v11255: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 136 KiB/s wr, 29 op/s 2023-09-10T05:06:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:11 smithi098 ceph-mon[103124]: pgmap v11255: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 136 KiB/s wr, 29 op/s 2023-09-10T05:06:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:13 smithi170 ceph-mon[102832]: pgmap v11256: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 136 KiB/s wr, 29 op/s 2023-09-10T05:06:14.318 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20804.277793 secs 2023-09-10T05:06:14.318 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20805.364895 secs 2023-09-10T05:06:14.320 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:13 smithi098 ceph-mon[103124]: pgmap v11256: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 136 KiB/s wr, 29 op/s 2023-09-10T05:06:14.320 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20804.277793 secs 2023-09-10T05:06:14.321 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20805.364895 secs 2023-09-10T05:06:14.321 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:13 smithi082 ceph-mon[98694]: pgmap v11256: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 136 KiB/s wr, 29 op/s 2023-09-10T05:06:14.321 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20804.277793 secs 2023-09-10T05:06:14.321 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20805.364895 secs 2023-09-10T05:06:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20806.018807 secs 2023-09-10T05:06:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:14 smithi170 ceph-mon[102832]: osdmap e772: 12 total, 12 up, 12 in 2023-09-10T05:06:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20806.018807 secs 2023-09-10T05:06:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:14 smithi082 ceph-mon[98694]: osdmap e772: 12 total, 12 up, 12 in 2023-09-10T05:06:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20806.018807 secs 2023-09-10T05:06:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:14 smithi098 ceph-mon[103124]: osdmap e772: 12 total, 12 up, 12 in 2023-09-10T05:06:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:15 smithi170 ceph-mon[102832]: pgmap v11258: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2023-09-10T05:06:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:15 smithi082 ceph-mon[98694]: pgmap v11258: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2023-09-10T05:06:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:15 smithi098 ceph-mon[103124]: pgmap v11258: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2023-09-10T05:06:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:17 smithi170 ceph-mon[102832]: pgmap v11259: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 164 KiB/s wr, 14 op/s 2023-09-10T05:06:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:17 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:06:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:17 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:06:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:17 smithi082 ceph-mon[98694]: pgmap v11259: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 164 KiB/s wr, 14 op/s 2023-09-10T05:06:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:17 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:06:18.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:17 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:06:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:17 smithi098 ceph-mon[103124]: pgmap v11259: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 164 KiB/s wr, 14 op/s 2023-09-10T05:06:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:17 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:06:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:17 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:06:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20809.277854 secs 2023-09-10T05:06:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20810.364954 secs 2023-09-10T05:06:19.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20809.277854 secs 2023-09-10T05:06:19.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20810.364954 secs 2023-09-10T05:06:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20809.277854 secs 2023-09-10T05:06:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20810.364954 secs 2023-09-10T05:06:20.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:19 smithi082 ceph-mon[98694]: pgmap v11260: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T05:06:20.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20811.018873 secs 2023-09-10T05:06:20.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:19 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:06:20.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:19 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:06:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:19 smithi098 ceph-mon[103124]: pgmap v11260: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T05:06:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20811.018873 secs 2023-09-10T05:06:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:19 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:06:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:19 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:06:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:19 smithi170 ceph-mon[102832]: pgmap v11260: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T05:06:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20811.018873 secs 2023-09-10T05:06:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:19 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:06:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:19 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:06:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:21 smithi082 ceph-mon[98694]: pgmap v11261: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 12 op/s 2023-09-10T05:06:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:21 smithi098 ceph-mon[103124]: pgmap v11261: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 12 op/s 2023-09-10T05:06:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:21 smithi170 ceph-mon[102832]: pgmap v11261: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 12 op/s 2023-09-10T05:06:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:23 smithi082 ceph-mon[98694]: pgmap v11262: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 12 op/s 2023-09-10T05:06:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20814.277914 secs 2023-09-10T05:06:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20815.365014 secs 2023-09-10T05:06:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:23 smithi098 ceph-mon[103124]: pgmap v11262: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 12 op/s 2023-09-10T05:06:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20814.277914 secs 2023-09-10T05:06:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20815.365014 secs 2023-09-10T05:06:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:23 smithi170 ceph-mon[102832]: pgmap v11262: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 12 op/s 2023-09-10T05:06:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20814.277914 secs 2023-09-10T05:06:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20815.365014 secs 2023-09-10T05:06:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20816.018935 secs 2023-09-10T05:06:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20816.018935 secs 2023-09-10T05:06:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20816.018935 secs 2023-09-10T05:06:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:25 smithi082 ceph-mon[98694]: pgmap v11263: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T05:06:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:25 smithi098 ceph-mon[103124]: pgmap v11263: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T05:06:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:25 smithi170 ceph-mon[102832]: pgmap v11263: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T05:06:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:27 smithi082 ceph-mon[98694]: pgmap v11264: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 177 KiB/s wr, 15 op/s 2023-09-10T05:06:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:27 smithi098 ceph-mon[103124]: pgmap v11264: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 177 KiB/s wr, 15 op/s 2023-09-10T05:06:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:27 smithi170 ceph-mon[102832]: pgmap v11264: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 177 KiB/s wr, 15 op/s 2023-09-10T05:06:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20820.365071 secs 2023-09-10T05:06:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20821.019003 secs 2023-09-10T05:06:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20820.365071 secs 2023-09-10T05:06:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20821.019003 secs 2023-09-10T05:06:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20820.365071 secs 2023-09-10T05:06:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20821.019003 secs 2023-09-10T05:06:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:29 smithi082 ceph-mon[98694]: pgmap v11265: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 10 op/s 2023-09-10T05:06:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:29 smithi098 ceph-mon[103124]: pgmap v11265: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 10 op/s 2023-09-10T05:06:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:29 smithi170 ceph-mon[102832]: pgmap v11265: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 10 op/s 2023-09-10T05:06:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:30 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:06:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:30 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:06:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:30 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:06:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:31 smithi082 ceph-mon[98694]: pgmap v11266: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 218 KiB/s wr, 16 op/s 2023-09-10T05:06:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:31 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:06:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:06:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:06:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:31 smithi098 ceph-mon[103124]: pgmap v11266: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 218 KiB/s wr, 16 op/s 2023-09-10T05:06:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:31 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:06:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:06:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:06:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:31 smithi170 ceph-mon[102832]: pgmap v11266: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 218 KiB/s wr, 16 op/s 2023-09-10T05:06:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:31 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:06:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:06:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:06:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:33 smithi082 ceph-mon[98694]: pgmap v11267: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 217 KiB/s wr, 10 op/s 2023-09-10T05:06:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20825.365129 secs 2023-09-10T05:06:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20826.019065 secs 2023-09-10T05:06:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:33 smithi098 ceph-mon[103124]: pgmap v11267: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 217 KiB/s wr, 10 op/s 2023-09-10T05:06:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20825.365129 secs 2023-09-10T05:06:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20826.019065 secs 2023-09-10T05:06:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:33 smithi170 ceph-mon[102832]: pgmap v11267: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 217 KiB/s wr, 10 op/s 2023-09-10T05:06:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20825.365129 secs 2023-09-10T05:06:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20826.019065 secs 2023-09-10T05:06:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:35 smithi170 ceph-mon[102832]: pgmap v11268: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 217 KiB/s wr, 10 op/s 2023-09-10T05:06:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:35 smithi082 ceph-mon[98694]: pgmap v11268: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 217 KiB/s wr, 10 op/s 2023-09-10T05:06:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:35 smithi098 ceph-mon[103124]: pgmap v11268: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 217 KiB/s wr, 10 op/s 2023-09-10T05:06:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:37 smithi170 ceph-mon[102832]: pgmap v11269: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 360 KiB/s wr, 16 op/s 2023-09-10T05:06:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:37 smithi082 ceph-mon[98694]: pgmap v11269: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 360 KiB/s wr, 16 op/s 2023-09-10T05:06:38.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:37 smithi098 ceph-mon[103124]: pgmap v11269: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 360 KiB/s wr, 16 op/s 2023-09-10T05:06:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:06:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:06:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20830.365229 secs 2023-09-10T05:06:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:06:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:06:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20830.365229 secs 2023-09-10T05:06:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:06:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:06:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20830.365229 secs 2023-09-10T05:06:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:39 smithi170 ceph-mon[102832]: pgmap v11270: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 292 KiB/s wr, 11 op/s 2023-09-10T05:06:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20831.019133 secs 2023-09-10T05:06:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:39 smithi082 ceph-mon[98694]: pgmap v11270: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 292 KiB/s wr, 11 op/s 2023-09-10T05:06:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20831.019133 secs 2023-09-10T05:06:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:39 smithi098 ceph-mon[103124]: pgmap v11270: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 292 KiB/s wr, 11 op/s 2023-09-10T05:06:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20831.019133 secs 2023-09-10T05:06:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:06:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:06:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:06:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:06:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:06:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:06:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:41 smithi170 ceph-mon[102832]: pgmap v11271: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 409 KiB/s wr, 17 op/s 2023-09-10T05:06:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:41 smithi082 ceph-mon[98694]: pgmap v11271: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 409 KiB/s wr, 17 op/s 2023-09-10T05:06:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:41 smithi098 ceph-mon[103124]: pgmap v11271: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 409 KiB/s wr, 17 op/s 2023-09-10T05:06:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:43 smithi170 ceph-mon[102832]: pgmap v11272: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 259 KiB/s wr, 11 op/s 2023-09-10T05:06:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20835.365312 secs 2023-09-10T05:06:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:43 smithi082 ceph-mon[98694]: pgmap v11272: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 259 KiB/s wr, 11 op/s 2023-09-10T05:06:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20835.365312 secs 2023-09-10T05:06:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:43 smithi098 ceph-mon[103124]: pgmap v11272: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 259 KiB/s wr, 11 op/s 2023-09-10T05:06:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20835.365312 secs 2023-09-10T05:06:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20836.019193 secs 2023-09-10T05:06:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20836.019193 secs 2023-09-10T05:06:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20836.019193 secs 2023-09-10T05:06:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:45 smithi170 ceph-mon[102832]: pgmap v11273: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 259 KiB/s wr, 11 op/s 2023-09-10T05:06:45.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:45 smithi082 ceph-mon[98694]: pgmap v11273: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 259 KiB/s wr, 11 op/s 2023-09-10T05:06:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:45 smithi098 ceph-mon[103124]: pgmap v11273: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 259 KiB/s wr, 11 op/s 2023-09-10T05:06:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:47 smithi170 ceph-mon[102832]: pgmap v11274: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 485 KiB/s wr, 18 op/s 2023-09-10T05:06:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:47 smithi082 ceph-mon[98694]: pgmap v11274: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 485 KiB/s wr, 18 op/s 2023-09-10T05:06:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:47 smithi098 ceph-mon[103124]: pgmap v11274: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 485 KiB/s wr, 18 op/s 2023-09-10T05:06:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20839.278282 secs 2023-09-10T05:06:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20840.365369 secs 2023-09-10T05:06:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20839.278282 secs 2023-09-10T05:06:48.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20840.365369 secs 2023-09-10T05:06:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20839.278282 secs 2023-09-10T05:06:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20840.365369 secs 2023-09-10T05:06:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:49 smithi170 ceph-mon[102832]: pgmap v11275: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 342 KiB/s wr, 12 op/s 2023-09-10T05:06:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20841.019257 secs 2023-09-10T05:06:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:49 smithi082 ceph-mon[98694]: pgmap v11275: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 342 KiB/s wr, 12 op/s 2023-09-10T05:06:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20841.019257 secs 2023-09-10T05:06:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:49 smithi098 ceph-mon[103124]: pgmap v11275: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 342 KiB/s wr, 12 op/s 2023-09-10T05:06:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20841.019257 secs 2023-09-10T05:06:50.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:06:50.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:06:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:06:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:06:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:06:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:06:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:51 smithi170 ceph-mon[102832]: pgmap v11276: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 470 KiB/s wr, 18 op/s 2023-09-10T05:06:51.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:51 smithi082 ceph-mon[98694]: pgmap v11276: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 470 KiB/s wr, 18 op/s 2023-09-10T05:06:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:51 smithi098 ceph-mon[103124]: pgmap v11276: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 470 KiB/s wr, 18 op/s 2023-09-10T05:06:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:06:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:06:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:06:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:06:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:06:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:06:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:06:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:06:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:06:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:53 smithi170 ceph-mon[102832]: pgmap v11277: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 353 KiB/s wr, 12 op/s 2023-09-10T05:06:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20844.278374 secs 2023-09-10T05:06:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20845.365460 secs 2023-09-10T05:06:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:53 smithi082 ceph-mon[98694]: pgmap v11277: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 353 KiB/s wr, 12 op/s 2023-09-10T05:06:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20844.278374 secs 2023-09-10T05:06:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20845.365460 secs 2023-09-10T05:06:54.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:53 smithi098 ceph-mon[103124]: pgmap v11277: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 353 KiB/s wr, 12 op/s 2023-09-10T05:06:54.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20844.278374 secs 2023-09-10T05:06:54.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20845.365460 secs 2023-09-10T05:06:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20846.019317 secs 2023-09-10T05:06:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20846.019317 secs 2023-09-10T05:06:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20846.019317 secs 2023-09-10T05:06:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:55 smithi170 ceph-mon[102832]: pgmap v11278: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 353 KiB/s wr, 12 op/s 2023-09-10T05:06:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:55 smithi082 ceph-mon[98694]: pgmap v11278: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 353 KiB/s wr, 12 op/s 2023-09-10T05:06:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:55 smithi098 ceph-mon[103124]: pgmap v11278: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 353 KiB/s wr, 12 op/s 2023-09-10T05:06:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:57 smithi170 ceph-mon[102832]: pgmap v11279: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 425 KiB/s wr, 18 op/s 2023-09-10T05:06:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:57 smithi082 ceph-mon[98694]: pgmap v11279: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 425 KiB/s wr, 18 op/s 2023-09-10T05:06:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:57 smithi098 ceph-mon[103124]: pgmap v11279: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 425 KiB/s wr, 18 op/s 2023-09-10T05:06:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20849.278436 secs 2023-09-10T05:06:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20850.365540 secs 2023-09-10T05:06:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20849.278436 secs 2023-09-10T05:06:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20850.365540 secs 2023-09-10T05:06:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20849.278436 secs 2023-09-10T05:06:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20850.365540 secs 2023-09-10T05:06:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:59 smithi170 ceph-mon[102832]: pgmap v11280: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 200 KiB/s wr, 11 op/s 2023-09-10T05:06:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:06:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20851.019385 secs 2023-09-10T05:06:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:59 smithi082 ceph-mon[98694]: pgmap v11280: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 200 KiB/s wr, 11 op/s 2023-09-10T05:06:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:06:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20851.019385 secs 2023-09-10T05:07:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:59 smithi098 ceph-mon[103124]: pgmap v11280: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 200 KiB/s wr, 11 op/s 2023-09-10T05:07:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:06:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20851.019385 secs 2023-09-10T05:07:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:01 smithi170 ceph-mon[102832]: pgmap v11281: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 280 KiB/s wr, 16 op/s 2023-09-10T05:07:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:01 smithi082 ceph-mon[98694]: pgmap v11281: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 280 KiB/s wr, 16 op/s 2023-09-10T05:07:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:01 smithi098 ceph-mon[103124]: pgmap v11281: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 280 KiB/s wr, 16 op/s 2023-09-10T05:07:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:03 smithi170 ceph-mon[102832]: pgmap v11282: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 153 KiB/s wr, 10 op/s 2023-09-10T05:07:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20854.278534 secs 2023-09-10T05:07:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20855.365624 secs 2023-09-10T05:07:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:03 smithi082 ceph-mon[98694]: pgmap v11282: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 153 KiB/s wr, 10 op/s 2023-09-10T05:07:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20854.278534 secs 2023-09-10T05:07:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20855.365624 secs 2023-09-10T05:07:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:03 smithi098 ceph-mon[103124]: pgmap v11282: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 153 KiB/s wr, 10 op/s 2023-09-10T05:07:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20854.278534 secs 2023-09-10T05:07:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20855.365624 secs 2023-09-10T05:07:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20856.019450 secs 2023-09-10T05:07:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20856.019450 secs 2023-09-10T05:07:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20856.019450 secs 2023-09-10T05:07:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:05 smithi170 ceph-mon[102832]: pgmap v11283: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 153 KiB/s wr, 10 op/s 2023-09-10T05:07:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:05 smithi082 ceph-mon[98694]: pgmap v11283: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 153 KiB/s wr, 10 op/s 2023-09-10T05:07:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:05 smithi098 ceph-mon[103124]: pgmap v11283: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 153 KiB/s wr, 10 op/s 2023-09-10T05:07:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:07:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:07:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:07:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:07 smithi170 ceph-mon[102832]: pgmap v11284: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 240 KiB/s wr, 29 op/s 2023-09-10T05:07:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:07:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:07:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:07:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:07 smithi082 ceph-mon[98694]: pgmap v11284: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 240 KiB/s wr, 29 op/s 2023-09-10T05:07:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:07:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:07:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:07:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:07 smithi098 ceph-mon[103124]: pgmap v11284: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 240 KiB/s wr, 29 op/s 2023-09-10T05:07:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:07:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:07:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:07:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20859.278594 secs 2023-09-10T05:07:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20860.365679 secs 2023-09-10T05:07:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:09 smithi170 ceph-mon[102832]: osdmap e773: 12 total, 12 up, 12 in 2023-09-10T05:07:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:09 smithi170 ceph-mon[102832]: pgmap v11286: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 201 KiB/s wr, 29 op/s 2023-09-10T05:07:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20861.019493 secs 2023-09-10T05:07:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20859.278594 secs 2023-09-10T05:07:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20860.365679 secs 2023-09-10T05:07:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:09 smithi082 ceph-mon[98694]: osdmap e773: 12 total, 12 up, 12 in 2023-09-10T05:07:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:09 smithi082 ceph-mon[98694]: pgmap v11286: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 201 KiB/s wr, 29 op/s 2023-09-10T05:07:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20861.019493 secs 2023-09-10T05:07:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20859.278594 secs 2023-09-10T05:07:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20860.365679 secs 2023-09-10T05:07:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:09 smithi098 ceph-mon[103124]: osdmap e773: 12 total, 12 up, 12 in 2023-09-10T05:07:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:09 smithi098 ceph-mon[103124]: pgmap v11286: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 201 KiB/s wr, 29 op/s 2023-09-10T05:07:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20861.019493 secs 2023-09-10T05:07:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:11 smithi170 ceph-mon[102832]: pgmap v11287: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 105 KiB/s wr, 28 op/s 2023-09-10T05:07:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:11 smithi082 ceph-mon[98694]: pgmap v11287: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 105 KiB/s wr, 28 op/s 2023-09-10T05:07:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:11 smithi098 ceph-mon[103124]: pgmap v11287: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 105 KiB/s wr, 28 op/s 2023-09-10T05:07:13.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:13 smithi170 ceph-mon[102832]: pgmap v11288: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 105 KiB/s wr, 28 op/s 2023-09-10T05:07:13.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20864.278681 secs 2023-09-10T05:07:13.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20865.365738 secs 2023-09-10T05:07:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:13 smithi082 ceph-mon[98694]: pgmap v11288: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 105 KiB/s wr, 28 op/s 2023-09-10T05:07:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20864.278681 secs 2023-09-10T05:07:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20865.365738 secs 2023-09-10T05:07:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:13 smithi098 ceph-mon[103124]: pgmap v11288: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 105 KiB/s wr, 28 op/s 2023-09-10T05:07:14.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20864.278681 secs 2023-09-10T05:07:14.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20865.365738 secs 2023-09-10T05:07:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20866.019556 secs 2023-09-10T05:07:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:14 smithi170 ceph-mon[102832]: osdmap e774: 12 total, 12 up, 12 in 2023-09-10T05:07:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20866.019556 secs 2023-09-10T05:07:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:14 smithi082 ceph-mon[98694]: osdmap e774: 12 total, 12 up, 12 in 2023-09-10T05:07:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20866.019556 secs 2023-09-10T05:07:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:14 smithi098 ceph-mon[103124]: osdmap e774: 12 total, 12 up, 12 in 2023-09-10T05:07:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:15 smithi170 ceph-mon[102832]: pgmap v11290: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 383 B/s wr, 6 op/s 2023-09-10T05:07:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:15 smithi082 ceph-mon[98694]: pgmap v11290: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 383 B/s wr, 6 op/s 2023-09-10T05:07:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:15 smithi098 ceph-mon[103124]: pgmap v11290: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 383 B/s wr, 6 op/s 2023-09-10T05:07:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:17 smithi170 ceph-mon[102832]: pgmap v11291: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 98 KiB/s wr, 14 op/s 2023-09-10T05:07:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:17 smithi082 ceph-mon[98694]: pgmap v11291: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 98 KiB/s wr, 14 op/s 2023-09-10T05:07:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:17 smithi098 ceph-mon[103124]: pgmap v11291: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 98 KiB/s wr, 14 op/s 2023-09-10T05:07:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20869.278739 secs 2023-09-10T05:07:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20870.365798 secs 2023-09-10T05:07:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20869.278739 secs 2023-09-10T05:07:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20870.365798 secs 2023-09-10T05:07:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20869.278739 secs 2023-09-10T05:07:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20870.365798 secs 2023-09-10T05:07:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:19 smithi170 ceph-mon[102832]: pgmap v11292: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-10T05:07:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20871.019622 secs 2023-09-10T05:07:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:19 smithi082 ceph-mon[98694]: pgmap v11292: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-10T05:07:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20871.019622 secs 2023-09-10T05:07:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:19 smithi098 ceph-mon[103124]: pgmap v11292: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-10T05:07:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20871.019622 secs 2023-09-10T05:07:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:21 smithi170 ceph-mon[102832]: pgmap v11293: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-10T05:07:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:21 smithi082 ceph-mon[98694]: pgmap v11293: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-10T05:07:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:21 smithi098 ceph-mon[103124]: pgmap v11293: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-10T05:07:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:23 smithi170 ceph-mon[102832]: pgmap v11294: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-10T05:07:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20874.278773 secs 2023-09-10T05:07:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20875.365850 secs 2023-09-10T05:07:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:23 smithi082 ceph-mon[98694]: pgmap v11294: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-10T05:07:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20874.278773 secs 2023-09-10T05:07:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20875.365850 secs 2023-09-10T05:07:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:23 smithi098 ceph-mon[103124]: pgmap v11294: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 12 op/s 2023-09-10T05:07:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20874.278773 secs 2023-09-10T05:07:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20875.365850 secs 2023-09-10T05:07:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20876.019682 secs 2023-09-10T05:07:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20876.019682 secs 2023-09-10T05:07:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20876.019682 secs 2023-09-10T05:07:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:25 smithi170 ceph-mon[102832]: pgmap v11295: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 12 op/s 2023-09-10T05:07:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:25 smithi082 ceph-mon[98694]: pgmap v11295: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 12 op/s 2023-09-10T05:07:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:25 smithi098 ceph-mon[103124]: pgmap v11295: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 12 op/s 2023-09-10T05:07:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:27 smithi170 ceph-mon[102832]: pgmap v11296: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 69 KiB/s wr, 15 op/s 2023-09-10T05:07:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:27 smithi082 ceph-mon[98694]: pgmap v11296: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 69 KiB/s wr, 15 op/s 2023-09-10T05:07:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:27 smithi098 ceph-mon[103124]: pgmap v11296: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 69 KiB/s wr, 15 op/s 2023-09-10T05:07:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20879.278841 secs 2023-09-10T05:07:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20880.365913 secs 2023-09-10T05:07:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20879.278841 secs 2023-09-10T05:07:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20880.365913 secs 2023-09-10T05:07:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20879.278841 secs 2023-09-10T05:07:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20880.365913 secs 2023-09-10T05:07:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:29 smithi170 ceph-mon[102832]: pgmap v11297: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2023-09-10T05:07:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20881.019747 secs 2023-09-10T05:07:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:29 smithi082 ceph-mon[98694]: pgmap v11297: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2023-09-10T05:07:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20881.019747 secs 2023-09-10T05:07:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:29 smithi098 ceph-mon[103124]: pgmap v11297: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2023-09-10T05:07:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20881.019747 secs 2023-09-10T05:07:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:07:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:07:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:07:30.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:07:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:07:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:07:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:31 smithi170 ceph-mon[102832]: pgmap v11298: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 153 KiB/s wr, 15 op/s 2023-09-10T05:07:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:31 smithi082 ceph-mon[98694]: pgmap v11298: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 153 KiB/s wr, 15 op/s 2023-09-10T05:07:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:31 smithi098 ceph-mon[103124]: pgmap v11298: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 153 KiB/s wr, 15 op/s 2023-09-10T05:07:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:07:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:07:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:07:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:07:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:07:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:07:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:33 smithi170 ceph-mon[102832]: pgmap v11299: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 10 op/s 2023-09-10T05:07:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20885.365972 secs 2023-09-10T05:07:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:33 smithi082 ceph-mon[98694]: pgmap v11299: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 10 op/s 2023-09-10T05:07:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20885.365972 secs 2023-09-10T05:07:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:33 smithi098 ceph-mon[103124]: pgmap v11299: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 10 op/s 2023-09-10T05:07:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20885.365972 secs 2023-09-10T05:07:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20886.019807 secs 2023-09-10T05:07:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20886.019807 secs 2023-09-10T05:07:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20886.019807 secs 2023-09-10T05:07:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:35 smithi170 ceph-mon[102832]: pgmap v11300: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 10 op/s 2023-09-10T05:07:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:07:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:35 smithi082 ceph-mon[98694]: pgmap v11300: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 10 op/s 2023-09-10T05:07:35.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:07:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:35 smithi098 ceph-mon[103124]: pgmap v11300: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 10 op/s 2023-09-10T05:07:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:07:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:07:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:07:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:07:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:07:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:07:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:07:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:07:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:07:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:07:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:37 smithi170 ceph-mon[102832]: pgmap v11301: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 307 KiB/s wr, 16 op/s 2023-09-10T05:07:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:37 smithi082 ceph-mon[98694]: pgmap v11301: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 307 KiB/s wr, 16 op/s 2023-09-10T05:07:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:37 smithi098 ceph-mon[103124]: pgmap v11301: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 307 KiB/s wr, 16 op/s 2023-09-10T05:07:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:07:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:07:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20890.366030 secs 2023-09-10T05:07:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:07:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:07:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20890.366030 secs 2023-09-10T05:07:39.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:07:39.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:07:39.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20890.366030 secs 2023-09-10T05:07:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:39 smithi170 ceph-mon[102832]: pgmap v11302: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 306 KiB/s wr, 11 op/s 2023-09-10T05:07:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20891.019897 secs 2023-09-10T05:07:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:39 smithi082 ceph-mon[98694]: pgmap v11302: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 306 KiB/s wr, 11 op/s 2023-09-10T05:07:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20891.019897 secs 2023-09-10T05:07:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:39 smithi098 ceph-mon[103124]: pgmap v11302: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 306 KiB/s wr, 11 op/s 2023-09-10T05:07:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20891.019897 secs 2023-09-10T05:07:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:07:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:07:40.962 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:07:40.963 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:07:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:07:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:07:41.935 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:41 smithi170 ceph-mon[102832]: pgmap v11303: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 413 KiB/s wr, 18 op/s 2023-09-10T05:07:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:41 smithi082 ceph-mon[98694]: pgmap v11303: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 413 KiB/s wr, 18 op/s 2023-09-10T05:07:42.008 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:41 smithi098 ceph-mon[103124]: pgmap v11303: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 413 KiB/s wr, 18 op/s 2023-09-10T05:07:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:43 smithi170 ceph-mon[102832]: pgmap v11304: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 261 KiB/s wr, 12 op/s 2023-09-10T05:07:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20895.366089 secs 2023-09-10T05:07:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:43 smithi082 ceph-mon[98694]: pgmap v11304: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 261 KiB/s wr, 12 op/s 2023-09-10T05:07:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20895.366089 secs 2023-09-10T05:07:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:43 smithi098 ceph-mon[103124]: pgmap v11304: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 261 KiB/s wr, 12 op/s 2023-09-10T05:07:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20895.366089 secs 2023-09-10T05:07:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20896.020044 secs 2023-09-10T05:07:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20896.020044 secs 2023-09-10T05:07:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20896.020044 secs 2023-09-10T05:07:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:45 smithi170 ceph-mon[102832]: pgmap v11305: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 261 KiB/s wr, 12 op/s 2023-09-10T05:07:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:45 smithi082 ceph-mon[98694]: pgmap v11305: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 261 KiB/s wr, 12 op/s 2023-09-10T05:07:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:45 smithi098 ceph-mon[103124]: pgmap v11305: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 261 KiB/s wr, 12 op/s 2023-09-10T05:07:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:47 smithi170 ceph-mon[102832]: pgmap v11306: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 397 KiB/s wr, 18 op/s 2023-09-10T05:07:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:47 smithi082 ceph-mon[98694]: pgmap v11306: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 397 KiB/s wr, 18 op/s 2023-09-10T05:07:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:47 smithi098 ceph-mon[103124]: pgmap v11306: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 397 KiB/s wr, 18 op/s 2023-09-10T05:07:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20900.366149 secs 2023-09-10T05:07:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20900.366149 secs 2023-09-10T05:07:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20900.366149 secs 2023-09-10T05:07:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:49 smithi170 ceph-mon[102832]: pgmap v11307: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 244 KiB/s wr, 12 op/s 2023-09-10T05:07:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20901.020108 secs 2023-09-10T05:07:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:49 smithi082 ceph-mon[98694]: pgmap v11307: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 244 KiB/s wr, 12 op/s 2023-09-10T05:07:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20901.020108 secs 2023-09-10T05:07:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:49 smithi098 ceph-mon[103124]: pgmap v11307: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 244 KiB/s wr, 12 op/s 2023-09-10T05:07:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20901.020108 secs 2023-09-10T05:07:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:51 smithi170 ceph-mon[102832]: pgmap v11308: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 386 KiB/s wr, 19 op/s 2023-09-10T05:07:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:51 smithi082 ceph-mon[98694]: pgmap v11308: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 386 KiB/s wr, 19 op/s 2023-09-10T05:07:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:51 smithi098 ceph-mon[103124]: pgmap v11308: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 386 KiB/s wr, 19 op/s 2023-09-10T05:07:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:53 smithi170 ceph-mon[102832]: pgmap v11309: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T05:07:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20904.279198 secs 2023-09-10T05:07:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20905.366207 secs 2023-09-10T05:07:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:53 smithi082 ceph-mon[98694]: pgmap v11309: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T05:07:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20904.279198 secs 2023-09-10T05:07:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20905.366207 secs 2023-09-10T05:07:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:53 smithi098 ceph-mon[103124]: pgmap v11309: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T05:07:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20904.279198 secs 2023-09-10T05:07:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20905.366207 secs 2023-09-10T05:07:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20906.020172 secs 2023-09-10T05:07:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20906.020172 secs 2023-09-10T05:07:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20906.020172 secs 2023-09-10T05:07:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:55 smithi170 ceph-mon[102832]: pgmap v11310: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T05:07:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:55 smithi082 ceph-mon[98694]: pgmap v11310: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T05:07:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:55 smithi098 ceph-mon[103124]: pgmap v11310: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T05:07:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:07:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:07:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:07:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:07:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:07:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:07:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:07:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:07:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:07:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:57 smithi170 ceph-mon[102832]: pgmap v11311: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 416 KiB/s wr, 18 op/s 2023-09-10T05:07:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:07:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:07:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:57 smithi082 ceph-mon[98694]: pgmap v11311: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 416 KiB/s wr, 18 op/s 2023-09-10T05:07:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:07:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:07:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:57 smithi098 ceph-mon[103124]: pgmap v11311: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 416 KiB/s wr, 18 op/s 2023-09-10T05:07:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:07:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:07:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20909.279258 secs 2023-09-10T05:07:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20910.366290 secs 2023-09-10T05:07:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20909.279258 secs 2023-09-10T05:07:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20910.366290 secs 2023-09-10T05:07:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20909.279258 secs 2023-09-10T05:07:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20910.366290 secs 2023-09-10T05:07:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:59 smithi082 ceph-mon[98694]: pgmap v11312: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 280 KiB/s wr, 12 op/s 2023-09-10T05:07:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:07:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20911.020237 secs 2023-09-10T05:08:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:59 smithi098 ceph-mon[103124]: pgmap v11312: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 280 KiB/s wr, 12 op/s 2023-09-10T05:08:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:07:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20911.020237 secs 2023-09-10T05:08:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:59 smithi170 ceph-mon[102832]: pgmap v11312: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 280 KiB/s wr, 12 op/s 2023-09-10T05:08:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:07:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20911.020237 secs 2023-09-10T05:08:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:08:00.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:08:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:08:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:08:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:08:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:08:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:01 smithi082 ceph-mon[98694]: pgmap v11313: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 379 KiB/s wr, 18 op/s 2023-09-10T05:08:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:01 smithi098 ceph-mon[103124]: pgmap v11313: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 379 KiB/s wr, 18 op/s 2023-09-10T05:08:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:01 smithi170 ceph-mon[102832]: pgmap v11313: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 379 KiB/s wr, 18 op/s 2023-09-10T05:08:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:03 smithi082 ceph-mon[98694]: pgmap v11314: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 236 KiB/s wr, 12 op/s 2023-09-10T05:08:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20914.279317 secs 2023-09-10T05:08:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20915.366392 secs 2023-09-10T05:08:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:03 smithi098 ceph-mon[103124]: pgmap v11314: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 236 KiB/s wr, 12 op/s 2023-09-10T05:08:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20914.279317 secs 2023-09-10T05:08:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20915.366392 secs 2023-09-10T05:08:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:03 smithi170 ceph-mon[102832]: pgmap v11314: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 236 KiB/s wr, 12 op/s 2023-09-10T05:08:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20914.279317 secs 2023-09-10T05:08:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20915.366392 secs 2023-09-10T05:08:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20916.020298 secs 2023-09-10T05:08:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20916.020298 secs 2023-09-10T05:08:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20916.020298 secs 2023-09-10T05:08:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:05 smithi082 ceph-mon[98694]: pgmap v11315: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 236 KiB/s wr, 12 op/s 2023-09-10T05:08:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:05 smithi098 ceph-mon[103124]: pgmap v11315: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 236 KiB/s wr, 12 op/s 2023-09-10T05:08:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:05 smithi170 ceph-mon[102832]: pgmap v11315: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 236 KiB/s wr, 12 op/s 2023-09-10T05:08:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:07 smithi082 ceph-mon[98694]: pgmap v11316: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 349 KiB/s wr, 32 op/s 2023-09-10T05:08:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:08:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:07 smithi098 ceph-mon[103124]: pgmap v11316: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 349 KiB/s wr, 32 op/s 2023-09-10T05:08:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:08:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:07 smithi170 ceph-mon[102832]: pgmap v11316: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 349 KiB/s wr, 32 op/s 2023-09-10T05:08:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:08:09.008 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20919.279377 secs 2023-09-10T05:08:09.008 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:08 smithi098 ceph-mon[103124]: osdmap e775: 12 total, 12 up, 12 in 2023-09-10T05:08:09.009 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:08:09.009 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:08:09.009 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:08:09.009 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20920.366449 secs 2023-09-10T05:08:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20919.279377 secs 2023-09-10T05:08:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:08 smithi170 ceph-mon[102832]: osdmap e775: 12 total, 12 up, 12 in 2023-09-10T05:08:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:08:09.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:08:09.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:08:09.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20920.366449 secs 2023-09-10T05:08:09.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20919.279377 secs 2023-09-10T05:08:09.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:08 smithi082 ceph-mon[98694]: osdmap e775: 12 total, 12 up, 12 in 2023-09-10T05:08:09.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:08:09.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:08:09.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:08:09.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20920.366449 secs 2023-09-10T05:08:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:09 smithi170 ceph-mon[102832]: pgmap v11318: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 254 KiB/s wr, 31 op/s 2023-09-10T05:08:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20921.020363 secs 2023-09-10T05:08:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:09 smithi082 ceph-mon[98694]: pgmap v11318: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 254 KiB/s wr, 31 op/s 2023-09-10T05:08:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20921.020363 secs 2023-09-10T05:08:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:09 smithi098 ceph-mon[103124]: pgmap v11318: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 254 KiB/s wr, 31 op/s 2023-09-10T05:08:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20921.020363 secs 2023-09-10T05:08:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:11 smithi170 ceph-mon[102832]: pgmap v11319: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 136 KiB/s wr, 29 op/s 2023-09-10T05:08:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:11 smithi082 ceph-mon[98694]: pgmap v11319: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 136 KiB/s wr, 29 op/s 2023-09-10T05:08:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:11 smithi098 ceph-mon[103124]: pgmap v11319: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 136 KiB/s wr, 29 op/s 2023-09-10T05:08:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:13 smithi170 ceph-mon[102832]: pgmap v11320: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 136 KiB/s wr, 29 op/s 2023-09-10T05:08:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20924.279460 secs 2023-09-10T05:08:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20925.366531 secs 2023-09-10T05:08:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:13 smithi082 ceph-mon[98694]: pgmap v11320: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 136 KiB/s wr, 29 op/s 2023-09-10T05:08:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20924.279460 secs 2023-09-10T05:08:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20925.366531 secs 2023-09-10T05:08:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:13 smithi098 ceph-mon[103124]: pgmap v11320: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 136 KiB/s wr, 29 op/s 2023-09-10T05:08:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20924.279460 secs 2023-09-10T05:08:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20925.366531 secs 2023-09-10T05:08:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20926.020425 secs 2023-09-10T05:08:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:14 smithi170 ceph-mon[102832]: osdmap e776: 12 total, 12 up, 12 in 2023-09-10T05:08:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20926.020425 secs 2023-09-10T05:08:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:14 smithi082 ceph-mon[98694]: osdmap e776: 12 total, 12 up, 12 in 2023-09-10T05:08:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20926.020425 secs 2023-09-10T05:08:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:14 smithi098 ceph-mon[103124]: osdmap e776: 12 total, 12 up, 12 in 2023-09-10T05:08:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:15 smithi170 ceph-mon[102832]: pgmap v11322: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T05:08:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:15 smithi082 ceph-mon[98694]: pgmap v11322: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T05:08:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:15 smithi098 ceph-mon[103124]: pgmap v11322: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T05:08:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:17 smithi170 ceph-mon[102832]: pgmap v11323: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 150 KiB/s wr, 13 op/s 2023-09-10T05:08:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:17 smithi082 ceph-mon[98694]: pgmap v11323: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 150 KiB/s wr, 13 op/s 2023-09-10T05:08:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:17 smithi098 ceph-mon[103124]: pgmap v11323: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 150 KiB/s wr, 13 op/s 2023-09-10T05:08:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20929.279540 secs 2023-09-10T05:08:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20930.366593 secs 2023-09-10T05:08:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20929.279540 secs 2023-09-10T05:08:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20930.366593 secs 2023-09-10T05:08:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20929.279540 secs 2023-09-10T05:08:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20930.366593 secs 2023-09-10T05:08:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:19 smithi170 ceph-mon[102832]: pgmap v11324: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T05:08:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20931.020488 secs 2023-09-10T05:08:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:19 smithi082 ceph-mon[98694]: pgmap v11324: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T05:08:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20931.020488 secs 2023-09-10T05:08:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:19 smithi098 ceph-mon[103124]: pgmap v11324: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T05:08:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20931.020488 secs 2023-09-10T05:08:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:21 smithi170 ceph-mon[102832]: pgmap v11325: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T05:08:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:21 smithi082 ceph-mon[98694]: pgmap v11325: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T05:08:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:21 smithi098 ceph-mon[103124]: pgmap v11325: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T05:08:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:23 smithi170 ceph-mon[102832]: pgmap v11326: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T05:08:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20934.279625 secs 2023-09-10T05:08:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20935.366652 secs 2023-09-10T05:08:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:23 smithi082 ceph-mon[98694]: pgmap v11326: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T05:08:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20934.279625 secs 2023-09-10T05:08:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20935.366652 secs 2023-09-10T05:08:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:23 smithi098 ceph-mon[103124]: pgmap v11326: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T05:08:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20934.279625 secs 2023-09-10T05:08:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20935.366652 secs 2023-09-10T05:08:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20936.020551 secs 2023-09-10T05:08:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20936.020551 secs 2023-09-10T05:08:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20936.020551 secs 2023-09-10T05:08:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:25 smithi170 ceph-mon[102832]: pgmap v11327: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 131 KiB/s wr, 11 op/s 2023-09-10T05:08:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:25 smithi082 ceph-mon[98694]: pgmap v11327: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 131 KiB/s wr, 11 op/s 2023-09-10T05:08:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:25 smithi098 ceph-mon[103124]: pgmap v11327: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 131 KiB/s wr, 11 op/s 2023-09-10T05:08:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:27 smithi170 ceph-mon[102832]: pgmap v11328: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 125 KiB/s wr, 14 op/s 2023-09-10T05:08:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:27 smithi082 ceph-mon[98694]: pgmap v11328: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 125 KiB/s wr, 14 op/s 2023-09-10T05:08:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:27 smithi098 ceph-mon[103124]: pgmap v11328: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 125 KiB/s wr, 14 op/s 2023-09-10T05:08:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20939.279686 secs 2023-09-10T05:08:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20940.366709 secs 2023-09-10T05:08:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20939.279686 secs 2023-09-10T05:08:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20940.366709 secs 2023-09-10T05:08:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20939.279686 secs 2023-09-10T05:08:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20940.366709 secs 2023-09-10T05:08:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:29 smithi170 ceph-mon[102832]: pgmap v11329: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 9 op/s 2023-09-10T05:08:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20941.020615 secs 2023-09-10T05:08:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:29 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:08:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:29 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:08:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:29 smithi082 ceph-mon[98694]: pgmap v11329: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 9 op/s 2023-09-10T05:08:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20941.020615 secs 2023-09-10T05:08:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:29 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:08:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:29 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:08:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:29 smithi098 ceph-mon[103124]: pgmap v11329: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 9 op/s 2023-09-10T05:08:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20941.020615 secs 2023-09-10T05:08:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:29 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:08:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:29 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:08:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:31 smithi170 ceph-mon[102832]: pgmap v11330: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 152 KiB/s wr, 15 op/s 2023-09-10T05:08:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:08:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:08:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:31 smithi082 ceph-mon[98694]: pgmap v11330: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 152 KiB/s wr, 15 op/s 2023-09-10T05:08:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:08:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:08:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:31 smithi098 ceph-mon[103124]: pgmap v11330: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 152 KiB/s wr, 15 op/s 2023-09-10T05:08:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:08:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:08:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:33 smithi170 ceph-mon[102832]: pgmap v11331: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T05:08:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20945.366768 secs 2023-09-10T05:08:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:33 smithi082 ceph-mon[98694]: pgmap v11331: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T05:08:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20945.366768 secs 2023-09-10T05:08:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:33 smithi098 ceph-mon[103124]: pgmap v11331: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T05:08:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20945.366768 secs 2023-09-10T05:08:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20946.020677 secs 2023-09-10T05:08:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:34 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:08:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20946.020677 secs 2023-09-10T05:08:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:34 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:08:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20946.020677 secs 2023-09-10T05:08:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:34 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:08:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:35 smithi170 ceph-mon[102832]: pgmap v11332: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T05:08:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:35 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:08:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:35 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:08:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:35 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:08:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:35 smithi082 ceph-mon[98694]: pgmap v11332: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T05:08:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:35 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:08:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:35 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:08:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:35 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:08:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:35 smithi098 ceph-mon[103124]: pgmap v11332: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T05:08:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:35 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:08:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:35 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:08:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:35 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:08:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:37 smithi170 ceph-mon[102832]: pgmap v11333: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 367 KiB/s wr, 38 op/s 2023-09-10T05:08:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:37 smithi082 ceph-mon[98694]: pgmap v11333: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 367 KiB/s wr, 38 op/s 2023-09-10T05:08:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:37 smithi098 ceph-mon[103124]: pgmap v11333: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 367 KiB/s wr, 38 op/s 2023-09-10T05:08:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20950.366853 secs 2023-09-10T05:08:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20950.366853 secs 2023-09-10T05:08:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20950.366853 secs 2023-09-10T05:08:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:39 smithi170 ceph-mon[102832]: pgmap v11334: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 353 KiB/s wr, 33 op/s 2023-09-10T05:08:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20951.020742 secs 2023-09-10T05:08:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:39 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:08:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:39 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:08:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:39 smithi082 ceph-mon[98694]: pgmap v11334: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 353 KiB/s wr, 33 op/s 2023-09-10T05:08:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20951.020742 secs 2023-09-10T05:08:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:39 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:08:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:39 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:08:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:39 smithi098 ceph-mon[103124]: pgmap v11334: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 353 KiB/s wr, 33 op/s 2023-09-10T05:08:40.380 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20951.020742 secs 2023-09-10T05:08:40.380 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:39 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:08:40.381 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:39 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:08:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:41 smithi170 ceph-mon[102832]: pgmap v11335: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 452 KiB/s wr, 40 op/s 2023-09-10T05:08:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:41 smithi082 ceph-mon[98694]: pgmap v11335: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 452 KiB/s wr, 40 op/s 2023-09-10T05:08:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:41 smithi098 ceph-mon[103124]: pgmap v11335: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 452 KiB/s wr, 40 op/s 2023-09-10T05:08:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:43 smithi170 ceph-mon[102832]: pgmap v11336: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 316 KiB/s wr, 34 op/s 2023-09-10T05:08:44.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20955.366910 secs 2023-09-10T05:08:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:43 smithi082 ceph-mon[98694]: pgmap v11336: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 316 KiB/s wr, 34 op/s 2023-09-10T05:08:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20955.366910 secs 2023-09-10T05:08:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:43 smithi098 ceph-mon[103124]: pgmap v11336: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 316 KiB/s wr, 34 op/s 2023-09-10T05:08:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20955.366910 secs 2023-09-10T05:08:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20956.020804 secs 2023-09-10T05:08:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20956.020804 secs 2023-09-10T05:08:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20956.020804 secs 2023-09-10T05:08:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:45 smithi170 ceph-mon[102832]: pgmap v11337: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 316 KiB/s wr, 34 op/s 2023-09-10T05:08:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:08:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:08:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:45 smithi082 ceph-mon[98694]: pgmap v11337: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 316 KiB/s wr, 34 op/s 2023-09-10T05:08:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:08:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:08:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:45 smithi098 ceph-mon[103124]: pgmap v11337: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 316 KiB/s wr, 34 op/s 2023-09-10T05:08:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:08:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:08:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:47 smithi170 ceph-mon[102832]: pgmap v11338: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 674 KiB/s wr, 41 op/s 2023-09-10T05:08:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:47 smithi082 ceph-mon[98694]: pgmap v11338: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 674 KiB/s wr, 41 op/s 2023-09-10T05:08:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:47 smithi098 ceph-mon[103124]: pgmap v11338: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 674 KiB/s wr, 41 op/s 2023-09-10T05:08:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20959.280008 secs 2023-09-10T05:08:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20960.366993 secs 2023-09-10T05:08:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20959.280008 secs 2023-09-10T05:08:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20960.366993 secs 2023-09-10T05:08:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20959.280008 secs 2023-09-10T05:08:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20960.366993 secs 2023-09-10T05:08:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:49 smithi170 ceph-mon[102832]: pgmap v11339: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 458 KiB/s wr, 13 op/s 2023-09-10T05:08:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20961.020868 secs 2023-09-10T05:08:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:08:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:08:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:49 smithi082 ceph-mon[98694]: pgmap v11339: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 458 KiB/s wr, 13 op/s 2023-09-10T05:08:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20961.020868 secs 2023-09-10T05:08:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:08:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:08:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:49 smithi098 ceph-mon[103124]: pgmap v11339: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 458 KiB/s wr, 13 op/s 2023-09-10T05:08:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20961.020868 secs 2023-09-10T05:08:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:08:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:08:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:50 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:08:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:08:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:08:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:50 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:08:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:08:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:08:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:50 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:08:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:08:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:08:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:51 smithi098 ceph-mon[103124]: pgmap v11340: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 458 KiB/s wr, 18 op/s 2023-09-10T05:08:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:51 smithi170 ceph-mon[102832]: pgmap v11340: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 458 KiB/s wr, 18 op/s 2023-09-10T05:08:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:51 smithi082 ceph-mon[98694]: pgmap v11340: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 458 KiB/s wr, 18 op/s 2023-09-10T05:08:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:53 smithi098 ceph-mon[103124]: pgmap v11341: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 359 KiB/s wr, 11 op/s 2023-09-10T05:08:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20964.280110 secs 2023-09-10T05:08:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20965.367051 secs 2023-09-10T05:08:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20966.020927 secs 2023-09-10T05:08:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:53 smithi170 ceph-mon[102832]: pgmap v11341: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 359 KiB/s wr, 11 op/s 2023-09-10T05:08:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20964.280110 secs 2023-09-10T05:08:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20965.367051 secs 2023-09-10T05:08:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20966.020927 secs 2023-09-10T05:08:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:53 smithi082 ceph-mon[98694]: pgmap v11341: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 359 KiB/s wr, 11 op/s 2023-09-10T05:08:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20964.280110 secs 2023-09-10T05:08:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20965.367051 secs 2023-09-10T05:08:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20966.020927 secs 2023-09-10T05:08:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:55 smithi082 ceph-mon[98694]: pgmap v11342: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 359 KiB/s wr, 11 op/s 2023-09-10T05:08:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:55 smithi098 ceph-mon[103124]: pgmap v11342: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 359 KiB/s wr, 11 op/s 2023-09-10T05:08:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:55 smithi170 ceph-mon[102832]: pgmap v11342: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 359 KiB/s wr, 11 op/s 2023-09-10T05:08:57.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:57 smithi170 ceph-mon[102832]: pgmap v11343: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 567 KiB/s wr, 17 op/s 2023-09-10T05:08:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:57 smithi082 ceph-mon[98694]: pgmap v11343: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 567 KiB/s wr, 17 op/s 2023-09-10T05:08:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:57 smithi098 ceph-mon[103124]: pgmap v11343: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 567 KiB/s wr, 17 op/s 2023-09-10T05:08:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20969.280197 secs 2023-09-10T05:08:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20970.367108 secs 2023-09-10T05:08:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20969.280197 secs 2023-09-10T05:08:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20970.367108 secs 2023-09-10T05:08:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20969.280197 secs 2023-09-10T05:08:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20970.367108 secs 2023-09-10T05:08:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:59 smithi170 ceph-mon[102832]: pgmap v11344: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 209 KiB/s wr, 11 op/s 2023-09-10T05:08:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:08:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20971.020991 secs 2023-09-10T05:08:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:59 smithi082 ceph-mon[98694]: pgmap v11344: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 209 KiB/s wr, 11 op/s 2023-09-10T05:08:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:08:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20971.020991 secs 2023-09-10T05:09:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:59 smithi098 ceph-mon[103124]: pgmap v11344: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 209 KiB/s wr, 11 op/s 2023-09-10T05:09:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:08:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20971.020991 secs 2023-09-10T05:09:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:01 smithi170 ceph-mon[102832]: pgmap v11345: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 290 KiB/s wr, 16 op/s 2023-09-10T05:09:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:01 smithi082 ceph-mon[98694]: pgmap v11345: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 290 KiB/s wr, 16 op/s 2023-09-10T05:09:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:01 smithi098 ceph-mon[103124]: pgmap v11345: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 290 KiB/s wr, 16 op/s 2023-09-10T05:09:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:03 smithi170 ceph-mon[102832]: pgmap v11346: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 289 KiB/s wr, 10 op/s 2023-09-10T05:09:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20974.280295 secs 2023-09-10T05:09:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20975.367166 secs 2023-09-10T05:09:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:03 smithi082 ceph-mon[98694]: pgmap v11346: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 289 KiB/s wr, 10 op/s 2023-09-10T05:09:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20974.280295 secs 2023-09-10T05:09:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20975.367166 secs 2023-09-10T05:09:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:03 smithi098 ceph-mon[103124]: pgmap v11346: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 289 KiB/s wr, 10 op/s 2023-09-10T05:09:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20974.280295 secs 2023-09-10T05:09:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20975.367166 secs 2023-09-10T05:09:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20976.021051 secs 2023-09-10T05:09:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20976.021051 secs 2023-09-10T05:09:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20976.021051 secs 2023-09-10T05:09:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:05 smithi170 ceph-mon[102832]: pgmap v11347: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 289 KiB/s wr, 10 op/s 2023-09-10T05:09:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:05 smithi082 ceph-mon[98694]: pgmap v11347: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 289 KiB/s wr, 10 op/s 2023-09-10T05:09:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:05 smithi098 ceph-mon[103124]: pgmap v11347: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 289 KiB/s wr, 10 op/s 2023-09-10T05:09:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:07 smithi170 ceph-mon[102832]: pgmap v11348: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 387 KiB/s wr, 30 op/s 2023-09-10T05:09:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:07 smithi082 ceph-mon[98694]: pgmap v11348: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 387 KiB/s wr, 30 op/s 2023-09-10T05:09:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:07 smithi098 ceph-mon[103124]: pgmap v11348: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 387 KiB/s wr, 30 op/s 2023-09-10T05:09:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20979.280357 secs 2023-09-10T05:09:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:09:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20980.367226 secs 2023-09-10T05:09:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:09:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:09:08.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:09:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20979.280357 secs 2023-09-10T05:09:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:09:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20980.367226 secs 2023-09-10T05:09:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:09:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:09:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:09:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20979.280357 secs 2023-09-10T05:09:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:09:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20980.367226 secs 2023-09-10T05:09:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:09:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:09:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:09:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:09 smithi170 ceph-mon[102832]: pgmap v11349: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 179 KiB/s wr, 24 op/s 2023-09-10T05:09:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:09 smithi170 ceph-mon[102832]: osdmap e777: 12 total, 12 up, 12 in 2023-09-10T05:09:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20981.021115 secs 2023-09-10T05:09:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:09 smithi082 ceph-mon[98694]: pgmap v11349: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 179 KiB/s wr, 24 op/s 2023-09-10T05:09:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:09 smithi082 ceph-mon[98694]: osdmap e777: 12 total, 12 up, 12 in 2023-09-10T05:09:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20981.021115 secs 2023-09-10T05:09:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:09 smithi098 ceph-mon[103124]: pgmap v11349: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 179 KiB/s wr, 24 op/s 2023-09-10T05:09:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:09 smithi098 ceph-mon[103124]: osdmap e777: 12 total, 12 up, 12 in 2023-09-10T05:09:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20981.021115 secs 2023-09-10T05:09:11.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:11 smithi170 ceph-mon[102832]: pgmap v11351: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 118 KiB/s wr, 30 op/s 2023-09-10T05:09:11.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:11 smithi082 ceph-mon[98694]: pgmap v11351: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 118 KiB/s wr, 30 op/s 2023-09-10T05:09:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:11 smithi098 ceph-mon[103124]: pgmap v11351: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 118 KiB/s wr, 30 op/s 2023-09-10T05:09:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:13 smithi170 ceph-mon[102832]: pgmap v11352: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 118 KiB/s wr, 30 op/s 2023-09-10T05:09:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20984.280431 secs 2023-09-10T05:09:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20985.367284 secs 2023-09-10T05:09:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:13 smithi082 ceph-mon[98694]: pgmap v11352: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 118 KiB/s wr, 30 op/s 2023-09-10T05:09:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20984.280431 secs 2023-09-10T05:09:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20985.367284 secs 2023-09-10T05:09:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:13 smithi098 ceph-mon[103124]: pgmap v11352: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 118 KiB/s wr, 30 op/s 2023-09-10T05:09:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20984.280431 secs 2023-09-10T05:09:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20985.367284 secs 2023-09-10T05:09:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:14 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:09:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:14 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:09:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20986.021177 secs 2023-09-10T05:09:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:14 smithi170 ceph-mon[102832]: osdmap e778: 12 total, 12 up, 12 in 2023-09-10T05:09:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:14 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:09:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:14 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:09:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20986.021177 secs 2023-09-10T05:09:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:14 smithi082 ceph-mon[98694]: osdmap e778: 12 total, 12 up, 12 in 2023-09-10T05:09:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:14 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:09:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:14 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:09:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20986.021177 secs 2023-09-10T05:09:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:14 smithi098 ceph-mon[103124]: osdmap e778: 12 total, 12 up, 12 in 2023-09-10T05:09:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:15 smithi170 ceph-mon[102832]: pgmap v11354: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T05:09:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:15 smithi082 ceph-mon[98694]: pgmap v11354: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T05:09:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:15 smithi098 ceph-mon[103124]: pgmap v11354: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 639 B/s wr, 7 op/s 2023-09-10T05:09:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:09:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:09:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:09:16.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:09:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:09:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:09:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:17 smithi170 ceph-mon[102832]: pgmap v11355: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 103 KiB/s wr, 15 op/s 2023-09-10T05:09:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:17 smithi082 ceph-mon[98694]: pgmap v11355: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 103 KiB/s wr, 15 op/s 2023-09-10T05:09:18.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:17 smithi098 ceph-mon[103124]: pgmap v11355: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 103 KiB/s wr, 15 op/s 2023-09-10T05:09:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20989.326481 secs 2023-09-10T05:09:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20990.367343 secs 2023-09-10T05:09:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20989.326481 secs 2023-09-10T05:09:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20990.367343 secs 2023-09-10T05:09:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20989.326481 secs 2023-09-10T05:09:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20990.367343 secs 2023-09-10T05:09:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:19 smithi170 ceph-mon[102832]: pgmap v11356: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 83 KiB/s wr, 12 op/s 2023-09-10T05:09:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20991.021240 secs 2023-09-10T05:09:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:19 smithi082 ceph-mon[98694]: pgmap v11356: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 83 KiB/s wr, 12 op/s 2023-09-10T05:09:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20991.021240 secs 2023-09-10T05:09:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:19 smithi098 ceph-mon[103124]: pgmap v11356: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 83 KiB/s wr, 12 op/s 2023-09-10T05:09:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20991.021240 secs 2023-09-10T05:09:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:21 smithi170 ceph-mon[102832]: pgmap v11357: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:09:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:21 smithi082 ceph-mon[98694]: pgmap v11357: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:09:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:21 smithi098 ceph-mon[103124]: pgmap v11357: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:09:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:23 smithi170 ceph-mon[102832]: pgmap v11358: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:09:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 20994.326590 secs 2023-09-10T05:09:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20995.367400 secs 2023-09-10T05:09:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:23 smithi082 ceph-mon[98694]: pgmap v11358: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:09:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 20994.326590 secs 2023-09-10T05:09:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20995.367400 secs 2023-09-10T05:09:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:23 smithi098 ceph-mon[103124]: pgmap v11358: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:09:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 20994.326590 secs 2023-09-10T05:09:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20995.367400 secs 2023-09-10T05:09:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 20996.021302 secs 2023-09-10T05:09:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 20996.021302 secs 2023-09-10T05:09:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 20996.021302 secs 2023-09-10T05:09:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:25 smithi170 ceph-mon[102832]: pgmap v11359: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T05:09:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:25 smithi082 ceph-mon[98694]: pgmap v11359: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T05:09:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:25 smithi098 ceph-mon[103124]: pgmap v11359: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T05:09:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:27 smithi170 ceph-mon[102832]: pgmap v11360: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 137 KiB/s wr, 14 op/s 2023-09-10T05:09:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:27 smithi082 ceph-mon[98694]: pgmap v11360: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 137 KiB/s wr, 14 op/s 2023-09-10T05:09:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:27 smithi098 ceph-mon[103124]: pgmap v11360: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 137 KiB/s wr, 14 op/s 2023-09-10T05:09:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21000.367461 secs 2023-09-10T05:09:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21000.367461 secs 2023-09-10T05:09:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21000.367461 secs 2023-09-10T05:09:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:29 smithi170 ceph-mon[102832]: pgmap v11361: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 9 op/s 2023-09-10T05:09:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21001.021390 secs 2023-09-10T05:09:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:29 smithi082 ceph-mon[98694]: pgmap v11361: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 9 op/s 2023-09-10T05:09:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21001.021390 secs 2023-09-10T05:09:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:29 smithi098 ceph-mon[103124]: pgmap v11361: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 9 op/s 2023-09-10T05:09:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21001.021390 secs 2023-09-10T05:09:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:31 smithi170 ceph-mon[102832]: pgmap v11362: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 135 KiB/s wr, 14 op/s 2023-09-10T05:09:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:09:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:31 smithi082 ceph-mon[98694]: pgmap v11362: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 135 KiB/s wr, 14 op/s 2023-09-10T05:09:31.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:09:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:31 smithi098 ceph-mon[103124]: pgmap v11362: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 135 KiB/s wr, 14 op/s 2023-09-10T05:09:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:09:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:09:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:09:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:09:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:09:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:09:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:09:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:09:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:09:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:09:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:33 smithi170 ceph-mon[102832]: pgmap v11363: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 9 op/s 2023-09-10T05:09:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21005.367563 secs 2023-09-10T05:09:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:33 smithi082 ceph-mon[98694]: pgmap v11363: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 9 op/s 2023-09-10T05:09:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21005.367563 secs 2023-09-10T05:09:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:33 smithi098 ceph-mon[103124]: pgmap v11363: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 9 op/s 2023-09-10T05:09:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21005.367563 secs 2023-09-10T05:09:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21006.021450 secs 2023-09-10T05:09:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21006.021450 secs 2023-09-10T05:09:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21006.021450 secs 2023-09-10T05:09:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:35 smithi170 ceph-mon[102832]: pgmap v11364: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 9 op/s 2023-09-10T05:09:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:35 smithi082 ceph-mon[98694]: pgmap v11364: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 9 op/s 2023-09-10T05:09:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:35 smithi098 ceph-mon[103124]: pgmap v11364: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 9 op/s 2023-09-10T05:09:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:37 smithi170 ceph-mon[102832]: pgmap v11365: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 278 KiB/s wr, 14 op/s 2023-09-10T05:09:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:09:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:09:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:37 smithi082 ceph-mon[98694]: pgmap v11365: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 278 KiB/s wr, 14 op/s 2023-09-10T05:09:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:09:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:09:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:37 smithi098 ceph-mon[103124]: pgmap v11365: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 278 KiB/s wr, 14 op/s 2023-09-10T05:09:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:09:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:09:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21010.367620 secs 2023-09-10T05:09:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21010.367620 secs 2023-09-10T05:09:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21010.367620 secs 2023-09-10T05:09:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:39 smithi170 ceph-mon[102832]: pgmap v11366: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 211 KiB/s wr, 10 op/s 2023-09-10T05:09:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21011.021518 secs 2023-09-10T05:09:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:39 smithi082 ceph-mon[98694]: pgmap v11366: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 211 KiB/s wr, 10 op/s 2023-09-10T05:09:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21011.021518 secs 2023-09-10T05:09:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:39 smithi098 ceph-mon[103124]: pgmap v11366: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 211 KiB/s wr, 10 op/s 2023-09-10T05:09:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21011.021518 secs 2023-09-10T05:09:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:09:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:09:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:09:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:09:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:09:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:09:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:41 smithi170 ceph-mon[102832]: pgmap v11367: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 241 KiB/s wr, 16 op/s 2023-09-10T05:09:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:41 smithi082 ceph-mon[98694]: pgmap v11367: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 241 KiB/s wr, 16 op/s 2023-09-10T05:09:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:41 smithi098 ceph-mon[103124]: pgmap v11367: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 241 KiB/s wr, 16 op/s 2023-09-10T05:09:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:43 smithi170 ceph-mon[102832]: pgmap v11368: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T05:09:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21015.367678 secs 2023-09-10T05:09:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:43 smithi082 ceph-mon[98694]: pgmap v11368: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T05:09:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21015.367678 secs 2023-09-10T05:09:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:43 smithi098 ceph-mon[103124]: pgmap v11368: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T05:09:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21015.367678 secs 2023-09-10T05:09:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21016.021579 secs 2023-09-10T05:09:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21016.021579 secs 2023-09-10T05:09:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21016.021579 secs 2023-09-10T05:09:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:45 smithi170 ceph-mon[102832]: pgmap v11369: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T05:09:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:45 smithi082 ceph-mon[98694]: pgmap v11369: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T05:09:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:45 smithi098 ceph-mon[103124]: pgmap v11369: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 174 KiB/s wr, 11 op/s 2023-09-10T05:09:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:47 smithi082 ceph-mon[98694]: pgmap v11370: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 417 KiB/s wr, 18 op/s 2023-09-10T05:09:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:47 smithi098 ceph-mon[103124]: pgmap v11370: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 417 KiB/s wr, 18 op/s 2023-09-10T05:09:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:47 smithi170 ceph-mon[102832]: pgmap v11370: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 417 KiB/s wr, 18 op/s 2023-09-10T05:09:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21020.367737 secs 2023-09-10T05:09:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21020.367737 secs 2023-09-10T05:09:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21020.367737 secs 2023-09-10T05:09:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:49 smithi082 ceph-mon[98694]: pgmap v11371: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 273 KiB/s wr, 13 op/s 2023-09-10T05:09:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21021.021646 secs 2023-09-10T05:09:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:49 smithi098 ceph-mon[103124]: pgmap v11371: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 273 KiB/s wr, 13 op/s 2023-09-10T05:09:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21021.021646 secs 2023-09-10T05:09:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:49 smithi170 ceph-mon[102832]: pgmap v11371: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 273 KiB/s wr, 13 op/s 2023-09-10T05:09:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21021.021646 secs 2023-09-10T05:09:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:51 smithi082 ceph-mon[98694]: pgmap v11372: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 355 KiB/s wr, 18 op/s 2023-09-10T05:09:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:51 smithi098 ceph-mon[103124]: pgmap v11372: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 355 KiB/s wr, 18 op/s 2023-09-10T05:09:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:51 smithi170 ceph-mon[102832]: pgmap v11372: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 355 KiB/s wr, 18 op/s 2023-09-10T05:09:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:53 smithi082 ceph-mon[98694]: pgmap v11373: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 324 KiB/s wr, 12 op/s 2023-09-10T05:09:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:09:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:09:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21025.367819 secs 2023-09-10T05:09:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:53 smithi098 ceph-mon[103124]: pgmap v11373: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 324 KiB/s wr, 12 op/s 2023-09-10T05:09:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:09:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:09:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21025.367819 secs 2023-09-10T05:09:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:53 smithi170 ceph-mon[102832]: pgmap v11373: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 324 KiB/s wr, 12 op/s 2023-09-10T05:09:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:09:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:09:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21025.367819 secs 2023-09-10T05:09:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21026.021708 secs 2023-09-10T05:09:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21026.021708 secs 2023-09-10T05:09:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21026.021708 secs 2023-09-10T05:09:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:55 smithi098 ceph-mon[103124]: pgmap v11374: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 324 KiB/s wr, 12 op/s 2023-09-10T05:09:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:55 smithi170 ceph-mon[102832]: pgmap v11374: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 324 KiB/s wr, 12 op/s 2023-09-10T05:09:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:55 smithi082 ceph-mon[98694]: pgmap v11374: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 324 KiB/s wr, 12 op/s 2023-09-10T05:09:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:09:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:09:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:09:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:09:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:09:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:09:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:57 smithi098 ceph-mon[103124]: pgmap v11375: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 535 KiB/s wr, 19 op/s 2023-09-10T05:09:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:57 smithi170 ceph-mon[102832]: pgmap v11375: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 535 KiB/s wr, 19 op/s 2023-09-10T05:09:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:57 smithi082 ceph-mon[98694]: pgmap v11375: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 535 KiB/s wr, 19 op/s 2023-09-10T05:09:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21029.327080 secs 2023-09-10T05:09:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21030.367904 secs 2023-09-10T05:09:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21029.327080 secs 2023-09-10T05:09:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21030.367904 secs 2023-09-10T05:09:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21029.327080 secs 2023-09-10T05:09:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21030.367904 secs 2023-09-10T05:10:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:59 smithi098 ceph-mon[103124]: pgmap v11376: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 292 KiB/s wr, 12 op/s 2023-09-10T05:10:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:09:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21031.021777 secs 2023-09-10T05:10:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:59 smithi170 ceph-mon[102832]: pgmap v11376: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 292 KiB/s wr, 12 op/s 2023-09-10T05:10:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:09:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21031.021777 secs 2023-09-10T05:10:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:59 smithi082 ceph-mon[98694]: pgmap v11376: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 292 KiB/s wr, 12 op/s 2023-09-10T05:10:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:09:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21031.021777 secs 2023-09-10T05:10:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:10:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:10:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T05:10:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:10:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:10:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T05:10:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:10:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:10:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T05:10:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:01 smithi098 ceph-mon[103124]: pgmap v11377: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 436 KiB/s wr, 18 op/s 2023-09-10T05:10:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:01 smithi170 ceph-mon[102832]: pgmap v11377: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 436 KiB/s wr, 18 op/s 2023-09-10T05:10:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:01 smithi082 ceph-mon[98694]: pgmap v11377: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 436 KiB/s wr, 18 op/s 2023-09-10T05:10:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:03 smithi098 ceph-mon[103124]: pgmap v11378: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 355 KiB/s wr, 12 op/s 2023-09-10T05:10:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21034.327181 secs 2023-09-10T05:10:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21035.367988 secs 2023-09-10T05:10:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:03 smithi170 ceph-mon[102832]: pgmap v11378: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 355 KiB/s wr, 12 op/s 2023-09-10T05:10:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21034.327181 secs 2023-09-10T05:10:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21035.367988 secs 2023-09-10T05:10:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:03 smithi082 ceph-mon[98694]: pgmap v11378: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 355 KiB/s wr, 12 op/s 2023-09-10T05:10:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21034.327181 secs 2023-09-10T05:10:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21035.367988 secs 2023-09-10T05:10:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:10:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21036.021839 secs 2023-09-10T05:10:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:10:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21036.021839 secs 2023-09-10T05:10:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:10:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21036.021839 secs 2023-09-10T05:10:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:05 smithi098 ceph-mon[103124]: pgmap v11379: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 355 KiB/s wr, 12 op/s 2023-09-10T05:10:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:10:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:05 smithi170 ceph-mon[102832]: pgmap v11379: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 355 KiB/s wr, 12 op/s 2023-09-10T05:10:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:10:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:05 smithi082 ceph-mon[98694]: pgmap v11379: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 355 KiB/s wr, 12 op/s 2023-09-10T05:10:06.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:10:06.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:07 smithi098 ceph-mon[103124]: pgmap v11380: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 547 KiB/s wr, 33 op/s 2023-09-10T05:10:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:07 smithi170 ceph-mon[102832]: pgmap v11380: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 547 KiB/s wr, 33 op/s 2023-09-10T05:10:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:07 smithi082 ceph-mon[98694]: pgmap v11380: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 547 KiB/s wr, 33 op/s 2023-09-10T05:10:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21039.327241 secs 2023-09-10T05:10:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:08 smithi170 ceph-mon[102832]: osdmap e779: 12 total, 12 up, 12 in 2023-09-10T05:10:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21040.368046 secs 2023-09-10T05:10:09.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:10:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21039.327241 secs 2023-09-10T05:10:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:08 smithi082 ceph-mon[98694]: osdmap e779: 12 total, 12 up, 12 in 2023-09-10T05:10:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21040.368046 secs 2023-09-10T05:10:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:10:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21039.327241 secs 2023-09-10T05:10:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:08 smithi098 ceph-mon[103124]: osdmap e779: 12 total, 12 up, 12 in 2023-09-10T05:10:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21040.368046 secs 2023-09-10T05:10:09.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:10:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:09 smithi170 ceph-mon[102832]: pgmap v11382: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 403 KiB/s wr, 31 op/s 2023-09-10T05:10:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21041.021908 secs 2023-09-10T05:10:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:10:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:10:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:10:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:09 smithi082 ceph-mon[98694]: pgmap v11382: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 403 KiB/s wr, 31 op/s 2023-09-10T05:10:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21041.021908 secs 2023-09-10T05:10:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:10:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:10:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:10:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:09 smithi098 ceph-mon[103124]: pgmap v11382: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 403 KiB/s wr, 31 op/s 2023-09-10T05:10:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21041.021908 secs 2023-09-10T05:10:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:10:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:10:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:10:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:11 smithi170 ceph-mon[102832]: pgmap v11383: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 294 KiB/s wr, 30 op/s 2023-09-10T05:10:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:11 smithi082 ceph-mon[98694]: pgmap v11383: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 294 KiB/s wr, 30 op/s 2023-09-10T05:10:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:11 smithi098 ceph-mon[103124]: pgmap v11383: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 294 KiB/s wr, 30 op/s 2023-09-10T05:10:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:13 smithi170 ceph-mon[102832]: pgmap v11384: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 294 KiB/s wr, 30 op/s 2023-09-10T05:10:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21044.327324 secs 2023-09-10T05:10:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21045.368097 secs 2023-09-10T05:10:14.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:13 smithi082 ceph-mon[98694]: pgmap v11384: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 294 KiB/s wr, 30 op/s 2023-09-10T05:10:14.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21044.327324 secs 2023-09-10T05:10:14.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21045.368097 secs 2023-09-10T05:10:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:13 smithi098 ceph-mon[103124]: pgmap v11384: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 294 KiB/s wr, 30 op/s 2023-09-10T05:10:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21044.327324 secs 2023-09-10T05:10:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21045.368097 secs 2023-09-10T05:10:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21046.021970 secs 2023-09-10T05:10:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:14 smithi170 ceph-mon[102832]: osdmap e780: 12 total, 12 up, 12 in 2023-09-10T05:10:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21046.021970 secs 2023-09-10T05:10:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:14 smithi082 ceph-mon[98694]: osdmap e780: 12 total, 12 up, 12 in 2023-09-10T05:10:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21046.021970 secs 2023-09-10T05:10:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:14 smithi098 ceph-mon[103124]: osdmap e780: 12 total, 12 up, 12 in 2023-09-10T05:10:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:15 smithi170 ceph-mon[102832]: pgmap v11386: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 79 KiB/s wr, 7 op/s 2023-09-10T05:10:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:15 smithi082 ceph-mon[98694]: pgmap v11386: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 79 KiB/s wr, 7 op/s 2023-09-10T05:10:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:15 smithi098 ceph-mon[103124]: pgmap v11386: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 79 KiB/s wr, 7 op/s 2023-09-10T05:10:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:17 smithi170 ceph-mon[102832]: pgmap v11387: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 13 op/s 2023-09-10T05:10:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:17 smithi082 ceph-mon[98694]: pgmap v11387: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 13 op/s 2023-09-10T05:10:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:17 smithi098 ceph-mon[103124]: pgmap v11387: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 73 KiB/s wr, 13 op/s 2023-09-10T05:10:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21049.327386 secs 2023-09-10T05:10:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21050.368157 secs 2023-09-10T05:10:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21049.327386 secs 2023-09-10T05:10:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21050.368157 secs 2023-09-10T05:10:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21049.327386 secs 2023-09-10T05:10:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21050.368157 secs 2023-09-10T05:10:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:19 smithi170 ceph-mon[102832]: pgmap v11388: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T05:10:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21051.022039 secs 2023-09-10T05:10:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:19 smithi082 ceph-mon[98694]: pgmap v11388: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T05:10:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21051.022039 secs 2023-09-10T05:10:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:19 smithi098 ceph-mon[103124]: pgmap v11388: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T05:10:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21051.022039 secs 2023-09-10T05:10:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:21 smithi170 ceph-mon[102832]: pgmap v11389: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-09-10T05:10:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:21 smithi082 ceph-mon[98694]: pgmap v11389: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-09-10T05:10:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:21 smithi098 ceph-mon[103124]: pgmap v11389: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-09-10T05:10:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:23 smithi170 ceph-mon[102832]: pgmap v11390: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-09-10T05:10:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21054.327478 secs 2023-09-10T05:10:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21055.368217 secs 2023-09-10T05:10:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:23 smithi082 ceph-mon[98694]: pgmap v11390: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-09-10T05:10:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21054.327478 secs 2023-09-10T05:10:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21055.368217 secs 2023-09-10T05:10:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:23 smithi098 ceph-mon[103124]: pgmap v11390: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-09-10T05:10:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21054.327478 secs 2023-09-10T05:10:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21055.368217 secs 2023-09-10T05:10:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21056.022100 secs 2023-09-10T05:10:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21056.022100 secs 2023-09-10T05:10:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21056.022100 secs 2023-09-10T05:10:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:25 smithi170 ceph-mon[102832]: pgmap v11391: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-09-10T05:10:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:25 smithi082 ceph-mon[98694]: pgmap v11391: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-09-10T05:10:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:25 smithi098 ceph-mon[103124]: pgmap v11391: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-09-10T05:10:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:27 smithi170 ceph-mon[102832]: pgmap v11392: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 70 KiB/s wr, 15 op/s 2023-09-10T05:10:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:27 smithi082 ceph-mon[98694]: pgmap v11392: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 70 KiB/s wr, 15 op/s 2023-09-10T05:10:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:27 smithi098 ceph-mon[103124]: pgmap v11392: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 70 KiB/s wr, 15 op/s 2023-09-10T05:10:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21060.368278 secs 2023-09-10T05:10:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21060.368278 secs 2023-09-10T05:10:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21060.368278 secs 2023-09-10T05:10:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:29 smithi170 ceph-mon[102832]: pgmap v11393: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T05:10:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21061.022171 secs 2023-09-10T05:10:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:29 smithi082 ceph-mon[98694]: pgmap v11393: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T05:10:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21061.022171 secs 2023-09-10T05:10:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:29 smithi098 ceph-mon[103124]: pgmap v11393: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T05:10:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21061.022171 secs 2023-09-10T05:10:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:31 smithi170 ceph-mon[102832]: pgmap v11394: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 178 KiB/s wr, 15 op/s 2023-09-10T05:10:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:10:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:31 smithi082 ceph-mon[98694]: pgmap v11394: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 178 KiB/s wr, 15 op/s 2023-09-10T05:10:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:10:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:31 smithi098 ceph-mon[103124]: pgmap v11394: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 178 KiB/s wr, 15 op/s 2023-09-10T05:10:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:10:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:10:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:10:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:10:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:10:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:10:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:10:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:33 smithi170 ceph-mon[102832]: pgmap v11395: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 178 KiB/s wr, 10 op/s 2023-09-10T05:10:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:10:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21065.368337 secs 2023-09-10T05:10:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:33 smithi082 ceph-mon[98694]: pgmap v11395: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 178 KiB/s wr, 10 op/s 2023-09-10T05:10:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:10:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21065.368337 secs 2023-09-10T05:10:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:33 smithi098 ceph-mon[103124]: pgmap v11395: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 178 KiB/s wr, 10 op/s 2023-09-10T05:10:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:10:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21065.368337 secs 2023-09-10T05:10:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21066.022229 secs 2023-09-10T05:10:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21066.022229 secs 2023-09-10T05:10:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21066.022229 secs 2023-09-10T05:10:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:35 smithi170 ceph-mon[102832]: pgmap v11396: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 178 KiB/s wr, 10 op/s 2023-09-10T05:10:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:35 smithi082 ceph-mon[98694]: pgmap v11396: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 178 KiB/s wr, 10 op/s 2023-09-10T05:10:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:35 smithi098 ceph-mon[103124]: pgmap v11396: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 178 KiB/s wr, 10 op/s 2023-09-10T05:10:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:10:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:10:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:10:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:37 smithi170 ceph-mon[102832]: pgmap v11397: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 497 KiB/s wr, 17 op/s 2023-09-10T05:10:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:37 smithi082 ceph-mon[98694]: pgmap v11397: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 497 KiB/s wr, 17 op/s 2023-09-10T05:10:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:37 smithi098 ceph-mon[103124]: pgmap v11397: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 497 KiB/s wr, 17 op/s 2023-09-10T05:10:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21070.368395 secs 2023-09-10T05:10:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21070.368395 secs 2023-09-10T05:10:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21070.368395 secs 2023-09-10T05:10:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:39 smithi170 ceph-mon[102832]: pgmap v11398: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 428 KiB/s wr, 11 op/s 2023-09-10T05:10:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21071.022295 secs 2023-09-10T05:10:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:39 smithi082 ceph-mon[98694]: pgmap v11398: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 428 KiB/s wr, 11 op/s 2023-09-10T05:10:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21071.022295 secs 2023-09-10T05:10:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:39 smithi098 ceph-mon[103124]: pgmap v11398: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 428 KiB/s wr, 11 op/s 2023-09-10T05:10:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21071.022295 secs 2023-09-10T05:10:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:41 smithi170 ceph-mon[102832]: pgmap v11399: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 482 KiB/s wr, 17 op/s 2023-09-10T05:10:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:41 smithi082 ceph-mon[98694]: pgmap v11399: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 482 KiB/s wr, 17 op/s 2023-09-10T05:10:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:41 smithi098 ceph-mon[103124]: pgmap v11399: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 482 KiB/s wr, 17 op/s 2023-09-10T05:10:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:43 smithi170 ceph-mon[102832]: pgmap v11400: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 374 KiB/s wr, 12 op/s 2023-09-10T05:10:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21075.368452 secs 2023-09-10T05:10:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:43 smithi082 ceph-mon[98694]: pgmap v11400: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 374 KiB/s wr, 12 op/s 2023-09-10T05:10:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21075.368452 secs 2023-09-10T05:10:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:43 smithi098 ceph-mon[103124]: pgmap v11400: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 374 KiB/s wr, 12 op/s 2023-09-10T05:10:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21075.368452 secs 2023-09-10T05:10:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21076.022356 secs 2023-09-10T05:10:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:10:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21076.022356 secs 2023-09-10T05:10:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:10:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21076.022356 secs 2023-09-10T05:10:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:10:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:45 smithi170 ceph-mon[102832]: pgmap v11401: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 374 KiB/s wr, 12 op/s 2023-09-10T05:10:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:10:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:45 smithi082 ceph-mon[98694]: pgmap v11401: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 374 KiB/s wr, 12 op/s 2023-09-10T05:10:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:10:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:45 smithi098 ceph-mon[103124]: pgmap v11401: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 374 KiB/s wr, 12 op/s 2023-09-10T05:10:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:10:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:47 smithi170 ceph-mon[102832]: pgmap v11402: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 577 KiB/s wr, 19 op/s 2023-09-10T05:10:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:47 smithi082 ceph-mon[98694]: pgmap v11402: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 577 KiB/s wr, 19 op/s 2023-09-10T05:10:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:47 smithi098 ceph-mon[103124]: pgmap v11402: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 577 KiB/s wr, 19 op/s 2023-09-10T05:10:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21080.368522 secs 2023-09-10T05:10:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21080.368522 secs 2023-09-10T05:10:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21080.368522 secs 2023-09-10T05:10:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:49 smithi170 ceph-mon[102832]: pgmap v11403: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 257 KiB/s wr, 12 op/s 2023-09-10T05:10:50.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21081.022422 secs 2023-09-10T05:10:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:49 smithi082 ceph-mon[98694]: pgmap v11403: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 257 KiB/s wr, 12 op/s 2023-09-10T05:10:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21081.022422 secs 2023-09-10T05:10:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:49 smithi098 ceph-mon[103124]: pgmap v11403: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 257 KiB/s wr, 12 op/s 2023-09-10T05:10:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21081.022422 secs 2023-09-10T05:10:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:51 smithi170 ceph-mon[102832]: pgmap v11404: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 508 KiB/s wr, 18 op/s 2023-09-10T05:10:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:51 smithi082 ceph-mon[98694]: pgmap v11404: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 508 KiB/s wr, 18 op/s 2023-09-10T05:10:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:51 smithi098 ceph-mon[103124]: pgmap v11404: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 508 KiB/s wr, 18 op/s 2023-09-10T05:10:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:53 smithi170 ceph-mon[102832]: pgmap v11405: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 454 KiB/s wr, 13 op/s 2023-09-10T05:10:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21084.327915 secs 2023-09-10T05:10:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:10:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:54.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21085.368581 secs 2023-09-10T05:10:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:53 smithi082 ceph-mon[98694]: pgmap v11405: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 454 KiB/s wr, 13 op/s 2023-09-10T05:10:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21084.327915 secs 2023-09-10T05:10:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:10:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:54.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21085.368581 secs 2023-09-10T05:10:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:53 smithi098 ceph-mon[103124]: pgmap v11405: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 454 KiB/s wr, 13 op/s 2023-09-10T05:10:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21084.327915 secs 2023-09-10T05:10:54.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:10:54.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:54.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21085.368581 secs 2023-09-10T05:10:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21086.022487 secs 2023-09-10T05:10:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21086.022487 secs 2023-09-10T05:10:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21086.022487 secs 2023-09-10T05:10:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:55 smithi170 ceph-mon[102832]: pgmap v11406: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 454 KiB/s wr, 13 op/s 2023-09-10T05:10:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:55 smithi082 ceph-mon[98694]: pgmap v11406: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 454 KiB/s wr, 13 op/s 2023-09-10T05:10:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:55 smithi098 ceph-mon[103124]: pgmap v11406: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 454 KiB/s wr, 13 op/s 2023-09-10T05:10:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:10:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:10:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:10:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:10:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:10:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:10:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:10:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:57 smithi170 ceph-mon[102832]: pgmap v11407: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 496 KiB/s wr, 18 op/s 2023-09-10T05:10:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:57 smithi082 ceph-mon[98694]: pgmap v11407: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 496 KiB/s wr, 18 op/s 2023-09-10T05:10:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:57 smithi098 ceph-mon[103124]: pgmap v11407: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 496 KiB/s wr, 18 op/s 2023-09-10T05:10:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21089.327975 secs 2023-09-10T05:10:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21090.368668 secs 2023-09-10T05:10:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21089.327975 secs 2023-09-10T05:10:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21090.368668 secs 2023-09-10T05:10:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21089.327975 secs 2023-09-10T05:10:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21090.368668 secs 2023-09-10T05:11:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:59 smithi170 ceph-mon[102832]: pgmap v11408: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 293 KiB/s wr, 12 op/s 2023-09-10T05:11:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:10:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21091.022551 secs 2023-09-10T05:11:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:59 smithi082 ceph-mon[98694]: pgmap v11408: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 293 KiB/s wr, 12 op/s 2023-09-10T05:11:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:10:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21091.022551 secs 2023-09-10T05:11:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:59 smithi098 ceph-mon[103124]: pgmap v11408: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 293 KiB/s wr, 12 op/s 2023-09-10T05:11:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:10:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21091.022551 secs 2023-09-10T05:11:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:01 smithi170 ceph-mon[102832]: pgmap v11409: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 511 KiB/s wr, 32 op/s 2023-09-10T05:11:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:01 smithi082 ceph-mon[98694]: pgmap v11409: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 511 KiB/s wr, 32 op/s 2023-09-10T05:11:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:01 smithi098 ceph-mon[103124]: pgmap v11409: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 511 KiB/s wr, 32 op/s 2023-09-10T05:11:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:03 smithi170 ceph-mon[102832]: pgmap v11410: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 260 KiB/s wr, 25 op/s 2023-09-10T05:11:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21094.328036 secs 2023-09-10T05:11:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21095.368726 secs 2023-09-10T05:11:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:03 smithi082 ceph-mon[98694]: pgmap v11410: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 260 KiB/s wr, 25 op/s 2023-09-10T05:11:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21094.328036 secs 2023-09-10T05:11:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21095.368726 secs 2023-09-10T05:11:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:03 smithi098 ceph-mon[103124]: pgmap v11410: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 260 KiB/s wr, 25 op/s 2023-09-10T05:11:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21094.328036 secs 2023-09-10T05:11:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21095.368726 secs 2023-09-10T05:11:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21096.022611 secs 2023-09-10T05:11:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21096.022611 secs 2023-09-10T05:11:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21096.022611 secs 2023-09-10T05:11:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:05 smithi170 ceph-mon[102832]: pgmap v11411: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 260 KiB/s wr, 25 op/s 2023-09-10T05:11:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:05 smithi082 ceph-mon[98694]: pgmap v11411: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 260 KiB/s wr, 25 op/s 2023-09-10T05:11:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:05 smithi098 ceph-mon[103124]: pgmap v11411: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 260 KiB/s wr, 25 op/s 2023-09-10T05:11:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:07 smithi170 ceph-mon[102832]: pgmap v11412: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 313 KiB/s wr, 30 op/s 2023-09-10T05:11:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:07 smithi082 ceph-mon[98694]: pgmap v11412: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 313 KiB/s wr, 30 op/s 2023-09-10T05:11:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:07 smithi098 ceph-mon[103124]: pgmap v11412: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 313 KiB/s wr, 30 op/s 2023-09-10T05:11:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21099.328097 secs 2023-09-10T05:11:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:08 smithi170 ceph-mon[102832]: osdmap e781: 12 total, 12 up, 12 in 2023-09-10T05:11:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21100.368809 secs 2023-09-10T05:11:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21099.328097 secs 2023-09-10T05:11:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:08 smithi082 ceph-mon[98694]: osdmap e781: 12 total, 12 up, 12 in 2023-09-10T05:11:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21100.368809 secs 2023-09-10T05:11:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21099.328097 secs 2023-09-10T05:11:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:08 smithi098 ceph-mon[103124]: osdmap e781: 12 total, 12 up, 12 in 2023-09-10T05:11:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21100.368809 secs 2023-09-10T05:11:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:09 smithi170 ceph-mon[102832]: pgmap v11414: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 324 KiB/s wr, 29 op/s 2023-09-10T05:11:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21101.022676 secs 2023-09-10T05:11:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:11:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:09 smithi082 ceph-mon[98694]: pgmap v11414: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 324 KiB/s wr, 29 op/s 2023-09-10T05:11:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21101.022676 secs 2023-09-10T05:11:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:11:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:09 smithi098 ceph-mon[103124]: pgmap v11414: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 324 KiB/s wr, 29 op/s 2023-09-10T05:11:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21101.022676 secs 2023-09-10T05:11:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:11:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:11:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:11:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:11:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:11:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:11:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:11:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:11:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:11:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:11:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:12 smithi170 ceph-mon[102832]: pgmap v11415: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:11:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:12 smithi082 ceph-mon[98694]: pgmap v11415: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:11:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:12 smithi098 ceph-mon[103124]: pgmap v11415: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:11:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:14 smithi170 ceph-mon[102832]: pgmap v11416: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:11:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21104.328184 secs 2023-09-10T05:11:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21105.368879 secs 2023-09-10T05:11:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21106.022738 secs 2023-09-10T05:11:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:14 smithi082 ceph-mon[98694]: pgmap v11416: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:11:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21104.328184 secs 2023-09-10T05:11:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21105.368879 secs 2023-09-10T05:11:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21106.022738 secs 2023-09-10T05:11:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:14 smithi098 ceph-mon[103124]: pgmap v11416: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:11:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21104.328184 secs 2023-09-10T05:11:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21105.368879 secs 2023-09-10T05:11:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21106.022738 secs 2023-09-10T05:11:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:15 smithi098 ceph-mon[103124]: osdmap e782: 12 total, 12 up, 12 in 2023-09-10T05:11:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:15 smithi098 ceph-mon[103124]: pgmap v11418: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 7 op/s 2023-09-10T05:11:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:15 smithi170 ceph-mon[102832]: osdmap e782: 12 total, 12 up, 12 in 2023-09-10T05:11:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:15 smithi170 ceph-mon[102832]: pgmap v11418: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 7 op/s 2023-09-10T05:11:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:15 smithi082 ceph-mon[98694]: osdmap e782: 12 total, 12 up, 12 in 2023-09-10T05:11:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:15 smithi082 ceph-mon[98694]: pgmap v11418: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 7 op/s 2023-09-10T05:11:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:17 smithi170 ceph-mon[102832]: pgmap v11419: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 15 op/s 2023-09-10T05:11:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:17 smithi082 ceph-mon[98694]: pgmap v11419: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 15 op/s 2023-09-10T05:11:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:17 smithi098 ceph-mon[103124]: pgmap v11419: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 15 op/s 2023-09-10T05:11:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21109.328244 secs 2023-09-10T05:11:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21110.368939 secs 2023-09-10T05:11:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21109.328244 secs 2023-09-10T05:11:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21110.368939 secs 2023-09-10T05:11:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21109.328244 secs 2023-09-10T05:11:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21110.368939 secs 2023-09-10T05:11:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:19 smithi170 ceph-mon[102832]: pgmap v11420: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T05:11:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21111.022801 secs 2023-09-10T05:11:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:19 smithi082 ceph-mon[98694]: pgmap v11420: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T05:11:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21111.022801 secs 2023-09-10T05:11:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:19 smithi098 ceph-mon[103124]: pgmap v11420: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T05:11:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21111.022801 secs 2023-09-10T05:11:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:21 smithi170 ceph-mon[102832]: pgmap v11421: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T05:11:21.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:21 smithi082 ceph-mon[98694]: pgmap v11421: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T05:11:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:21 smithi098 ceph-mon[103124]: pgmap v11421: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T05:11:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:23 smithi170 ceph-mon[102832]: pgmap v11422: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T05:11:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21114.328309 secs 2023-09-10T05:11:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21115.369021 secs 2023-09-10T05:11:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:23 smithi082 ceph-mon[98694]: pgmap v11422: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T05:11:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21114.328309 secs 2023-09-10T05:11:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21115.369021 secs 2023-09-10T05:11:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:23 smithi098 ceph-mon[103124]: pgmap v11422: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T05:11:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21114.328309 secs 2023-09-10T05:11:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21115.369021 secs 2023-09-10T05:11:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:11:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:11:24.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21116.022861 secs 2023-09-10T05:11:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:11:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:11:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21116.022861 secs 2023-09-10T05:11:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:11:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:11:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21116.022861 secs 2023-09-10T05:11:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:25 smithi170 ceph-mon[102832]: pgmap v11423: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 12 op/s 2023-09-10T05:11:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:25 smithi082 ceph-mon[98694]: pgmap v11423: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 12 op/s 2023-09-10T05:11:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:25 smithi098 ceph-mon[103124]: pgmap v11423: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 12 op/s 2023-09-10T05:11:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:26 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:11:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:11:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:26 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:11:26.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:11:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:26 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:11:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:11:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:27 smithi170 ceph-mon[102832]: pgmap v11424: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 15 op/s 2023-09-10T05:11:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:27 smithi082 ceph-mon[98694]: pgmap v11424: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 15 op/s 2023-09-10T05:11:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:27 smithi098 ceph-mon[103124]: pgmap v11424: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 15 op/s 2023-09-10T05:11:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21120.369076 secs 2023-09-10T05:11:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21120.369076 secs 2023-09-10T05:11:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21120.369076 secs 2023-09-10T05:11:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:29 smithi170 ceph-mon[102832]: pgmap v11425: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T05:11:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21121.022928 secs 2023-09-10T05:11:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:29 smithi082 ceph-mon[98694]: pgmap v11425: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T05:11:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21121.022928 secs 2023-09-10T05:11:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:29 smithi098 ceph-mon[103124]: pgmap v11425: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T05:11:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21121.022928 secs 2023-09-10T05:11:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:31 smithi170 ceph-mon[102832]: pgmap v11426: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 223 KiB/s wr, 16 op/s 2023-09-10T05:11:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:11:31.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:31 smithi082 ceph-mon[98694]: pgmap v11426: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 223 KiB/s wr, 16 op/s 2023-09-10T05:11:31.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:11:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:31 smithi098 ceph-mon[103124]: pgmap v11426: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 223 KiB/s wr, 16 op/s 2023-09-10T05:11:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:11:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:11:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:11:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:11:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:11:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:11:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:11:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:11:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:11:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:11:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:33 smithi170 ceph-mon[102832]: pgmap v11427: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 222 KiB/s wr, 11 op/s 2023-09-10T05:11:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21125.369134 secs 2023-09-10T05:11:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:33 smithi082 ceph-mon[98694]: pgmap v11427: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 222 KiB/s wr, 11 op/s 2023-09-10T05:11:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21125.369134 secs 2023-09-10T05:11:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:33 smithi098 ceph-mon[103124]: pgmap v11427: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 222 KiB/s wr, 11 op/s 2023-09-10T05:11:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21125.369134 secs 2023-09-10T05:11:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21126.022992 secs 2023-09-10T05:11:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21126.022992 secs 2023-09-10T05:11:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21126.022992 secs 2023-09-10T05:11:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:35 smithi170 ceph-mon[102832]: pgmap v11428: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 222 KiB/s wr, 11 op/s 2023-09-10T05:11:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:35 smithi082 ceph-mon[98694]: pgmap v11428: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 222 KiB/s wr, 11 op/s 2023-09-10T05:11:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:35 smithi098 ceph-mon[103124]: pgmap v11428: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 222 KiB/s wr, 11 op/s 2023-09-10T05:11:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:37 smithi170 ceph-mon[102832]: pgmap v11429: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 378 KiB/s wr, 18 op/s 2023-09-10T05:11:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:11:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:11:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:37 smithi082 ceph-mon[98694]: pgmap v11429: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 378 KiB/s wr, 18 op/s 2023-09-10T05:11:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:11:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:11:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:37 smithi098 ceph-mon[103124]: pgmap v11429: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 378 KiB/s wr, 18 op/s 2023-09-10T05:11:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:11:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:11:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21130.369217 secs 2023-09-10T05:11:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21130.369217 secs 2023-09-10T05:11:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21130.369217 secs 2023-09-10T05:11:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:39 smithi170 ceph-mon[102832]: pgmap v11430: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 310 KiB/s wr, 12 op/s 2023-09-10T05:11:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21131.023056 secs 2023-09-10T05:11:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:39 smithi082 ceph-mon[98694]: pgmap v11430: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 310 KiB/s wr, 12 op/s 2023-09-10T05:11:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21131.023056 secs 2023-09-10T05:11:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:39 smithi098 ceph-mon[103124]: pgmap v11430: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 310 KiB/s wr, 12 op/s 2023-09-10T05:11:40.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21131.023056 secs 2023-09-10T05:11:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:11:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:11:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:11:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:11:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:11:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:11:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:41 smithi082 ceph-mon[98694]: pgmap v11431: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 419 KiB/s wr, 19 op/s 2023-09-10T05:11:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:41 smithi098 ceph-mon[103124]: pgmap v11431: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 419 KiB/s wr, 19 op/s 2023-09-10T05:11:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:41 smithi170 ceph-mon[102832]: pgmap v11431: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 419 KiB/s wr, 19 op/s 2023-09-10T05:11:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:43 smithi082 ceph-mon[98694]: pgmap v11432: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 266 KiB/s wr, 14 op/s 2023-09-10T05:11:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21135.369275 secs 2023-09-10T05:11:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:43 smithi098 ceph-mon[103124]: pgmap v11432: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 266 KiB/s wr, 14 op/s 2023-09-10T05:11:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21135.369275 secs 2023-09-10T05:11:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:43 smithi170 ceph-mon[102832]: pgmap v11432: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 266 KiB/s wr, 14 op/s 2023-09-10T05:11:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21135.369275 secs 2023-09-10T05:11:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21136.023116 secs 2023-09-10T05:11:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21136.023116 secs 2023-09-10T05:11:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21136.023116 secs 2023-09-10T05:11:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:45 smithi082 ceph-mon[98694]: pgmap v11433: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 266 KiB/s wr, 14 op/s 2023-09-10T05:11:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:45 smithi098 ceph-mon[103124]: pgmap v11433: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 266 KiB/s wr, 14 op/s 2023-09-10T05:11:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:45 smithi170 ceph-mon[102832]: pgmap v11433: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 266 KiB/s wr, 14 op/s 2023-09-10T05:11:47.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:47 smithi082 ceph-mon[98694]: pgmap v11434: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 490 KiB/s wr, 20 op/s 2023-09-10T05:11:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:47 smithi098 ceph-mon[103124]: pgmap v11434: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 490 KiB/s wr, 20 op/s 2023-09-10T05:11:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:47 smithi170 ceph-mon[102832]: pgmap v11434: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 490 KiB/s wr, 20 op/s 2023-09-10T05:11:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21139.328689 secs 2023-09-10T05:11:48.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21140.369348 secs 2023-09-10T05:11:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21139.328689 secs 2023-09-10T05:11:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21140.369348 secs 2023-09-10T05:11:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21139.328689 secs 2023-09-10T05:11:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21140.369348 secs 2023-09-10T05:11:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:49 smithi082 ceph-mon[98694]: pgmap v11435: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 334 KiB/s wr, 13 op/s 2023-09-10T05:11:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:11:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:11:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21141.023180 secs 2023-09-10T05:11:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:49 smithi098 ceph-mon[103124]: pgmap v11435: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 334 KiB/s wr, 13 op/s 2023-09-10T05:11:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:11:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:11:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21141.023180 secs 2023-09-10T05:11:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:49 smithi170 ceph-mon[102832]: pgmap v11435: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 334 KiB/s wr, 13 op/s 2023-09-10T05:11:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:11:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:11:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21141.023180 secs 2023-09-10T05:11:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:51 smithi082 ceph-mon[98694]: pgmap v11436: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 460 KiB/s wr, 19 op/s 2023-09-10T05:11:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:51 smithi098 ceph-mon[103124]: pgmap v11436: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 460 KiB/s wr, 19 op/s 2023-09-10T05:11:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:51 smithi170 ceph-mon[102832]: pgmap v11436: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 460 KiB/s wr, 19 op/s 2023-09-10T05:11:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:11:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:11:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:11:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:11:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:11:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:11:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:11:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:11:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:11:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:53 smithi098 ceph-mon[103124]: pgmap v11437: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 350 KiB/s wr, 12 op/s 2023-09-10T05:11:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21144.328823 secs 2023-09-10T05:11:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21145.369406 secs 2023-09-10T05:11:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:53 smithi170 ceph-mon[102832]: pgmap v11437: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 350 KiB/s wr, 12 op/s 2023-09-10T05:11:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21144.328823 secs 2023-09-10T05:11:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21145.369406 secs 2023-09-10T05:11:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:53 smithi082 ceph-mon[98694]: pgmap v11437: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 350 KiB/s wr, 12 op/s 2023-09-10T05:11:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21144.328823 secs 2023-09-10T05:11:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21145.369406 secs 2023-09-10T05:11:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21146.023242 secs 2023-09-10T05:11:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21146.023242 secs 2023-09-10T05:11:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21146.023242 secs 2023-09-10T05:11:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:55 smithi098 ceph-mon[103124]: pgmap v11438: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 350 KiB/s wr, 12 op/s 2023-09-10T05:11:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:55 smithi170 ceph-mon[102832]: pgmap v11438: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 350 KiB/s wr, 12 op/s 2023-09-10T05:11:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:55 smithi082 ceph-mon[98694]: pgmap v11438: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 350 KiB/s wr, 12 op/s 2023-09-10T05:11:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:57 smithi098 ceph-mon[103124]: pgmap v11439: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 490 KiB/s wr, 17 op/s 2023-09-10T05:11:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:57 smithi170 ceph-mon[102832]: pgmap v11439: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 490 KiB/s wr, 17 op/s 2023-09-10T05:11:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:57 smithi082 ceph-mon[98694]: pgmap v11439: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 490 KiB/s wr, 17 op/s 2023-09-10T05:11:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21149.328884 secs 2023-09-10T05:11:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21150.369486 secs 2023-09-10T05:11:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21149.328884 secs 2023-09-10T05:11:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21150.369486 secs 2023-09-10T05:11:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21149.328884 secs 2023-09-10T05:11:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21150.369486 secs 2023-09-10T05:12:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:59 smithi098 ceph-mon[103124]: pgmap v11440: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 266 KiB/s wr, 11 op/s 2023-09-10T05:12:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:11:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21151.023304 secs 2023-09-10T05:12:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:59 smithi170 ceph-mon[102832]: pgmap v11440: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 266 KiB/s wr, 11 op/s 2023-09-10T05:12:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:11:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21151.023304 secs 2023-09-10T05:12:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:59 smithi082 ceph-mon[98694]: pgmap v11440: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 266 KiB/s wr, 11 op/s 2023-09-10T05:12:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:11:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21151.023304 secs 2023-09-10T05:12:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:01 smithi098 ceph-mon[103124]: pgmap v11441: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 315 KiB/s wr, 29 op/s 2023-09-10T05:12:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:01 smithi170 ceph-mon[102832]: pgmap v11441: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 315 KiB/s wr, 29 op/s 2023-09-10T05:12:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:01 smithi082 ceph-mon[98694]: pgmap v11441: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 315 KiB/s wr, 29 op/s 2023-09-10T05:12:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:03 smithi098 ceph-mon[103124]: pgmap v11442: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 189 KiB/s wr, 23 op/s 2023-09-10T05:12:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21154.328943 secs 2023-09-10T05:12:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21155.369567 secs 2023-09-10T05:12:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:03 smithi170 ceph-mon[102832]: pgmap v11442: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 189 KiB/s wr, 23 op/s 2023-09-10T05:12:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21154.328943 secs 2023-09-10T05:12:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21155.369567 secs 2023-09-10T05:12:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:03 smithi082 ceph-mon[98694]: pgmap v11442: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 189 KiB/s wr, 23 op/s 2023-09-10T05:12:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21154.328943 secs 2023-09-10T05:12:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21155.369567 secs 2023-09-10T05:12:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21156.023372 secs 2023-09-10T05:12:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21156.023372 secs 2023-09-10T05:12:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21156.023372 secs 2023-09-10T05:12:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:05 smithi098 ceph-mon[103124]: pgmap v11443: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 189 KiB/s wr, 23 op/s 2023-09-10T05:12:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:05 smithi170 ceph-mon[102832]: pgmap v11443: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 189 KiB/s wr, 23 op/s 2023-09-10T05:12:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:05 smithi082 ceph-mon[98694]: pgmap v11443: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 189 KiB/s wr, 23 op/s 2023-09-10T05:12:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:07 smithi098 ceph-mon[103124]: pgmap v11444: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 270 KiB/s wr, 30 op/s 2023-09-10T05:12:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:07 smithi170 ceph-mon[102832]: pgmap v11444: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 270 KiB/s wr, 30 op/s 2023-09-10T05:12:08.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:07 smithi082 ceph-mon[98694]: pgmap v11444: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 270 KiB/s wr, 30 op/s 2023-09-10T05:12:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21159.329003 secs 2023-09-10T05:12:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:08 smithi098 ceph-mon[103124]: osdmap e783: 12 total, 12 up, 12 in 2023-09-10T05:12:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21160.369626 secs 2023-09-10T05:12:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21159.329003 secs 2023-09-10T05:12:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:08 smithi170 ceph-mon[102832]: osdmap e783: 12 total, 12 up, 12 in 2023-09-10T05:12:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21160.369626 secs 2023-09-10T05:12:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21159.329003 secs 2023-09-10T05:12:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:08 smithi082 ceph-mon[98694]: osdmap e783: 12 total, 12 up, 12 in 2023-09-10T05:12:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21160.369626 secs 2023-09-10T05:12:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:09 smithi098 ceph-mon[103124]: pgmap v11446: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 155 KiB/s wr, 30 op/s 2023-09-10T05:12:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21161.023435 secs 2023-09-10T05:12:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:09 smithi170 ceph-mon[102832]: pgmap v11446: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 155 KiB/s wr, 30 op/s 2023-09-10T05:12:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21161.023435 secs 2023-09-10T05:12:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:09 smithi082 ceph-mon[98694]: pgmap v11446: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 155 KiB/s wr, 30 op/s 2023-09-10T05:12:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21161.023435 secs 2023-09-10T05:12:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:12:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:12:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:12:12.076 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:11 smithi098 ceph-mon[103124]: pgmap v11447: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 96 KiB/s wr, 13 op/s 2023-09-10T05:12:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:11 smithi170 ceph-mon[102832]: pgmap v11447: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 96 KiB/s wr, 13 op/s 2023-09-10T05:12:12.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:11 smithi082 ceph-mon[98694]: pgmap v11447: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 96 KiB/s wr, 13 op/s 2023-09-10T05:12:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:13 smithi170 ceph-mon[102832]: pgmap v11448: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 96 KiB/s wr, 13 op/s 2023-09-10T05:12:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21164.329062 secs 2023-09-10T05:12:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:12:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:12:14.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:12:14.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:12:14.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21165.369686 secs 2023-09-10T05:12:14.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:12:14.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:12:14.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:12:14.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:12:14.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:12:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:13 smithi082 ceph-mon[98694]: pgmap v11448: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 96 KiB/s wr, 13 op/s 2023-09-10T05:12:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21164.329062 secs 2023-09-10T05:12:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:12:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:12:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:12:14.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:12:14.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21165.369686 secs 2023-09-10T05:12:14.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:12:14.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:12:14.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:12:14.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:12:14.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:12:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:13 smithi098 ceph-mon[103124]: pgmap v11448: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 96 KiB/s wr, 13 op/s 2023-09-10T05:12:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21164.329062 secs 2023-09-10T05:12:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:12:14.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:12:14.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:12:14.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:12:14.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21165.369686 secs 2023-09-10T05:12:14.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:12:14.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:12:14.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:12:14.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:12:14.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:12:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:14 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T05:12:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:14 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T05:12:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21166.023481 secs 2023-09-10T05:12:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:12:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:12:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:14 smithi170 ceph-mon[102832]: osdmap e784: 12 total, 12 up, 12 in 2023-09-10T05:12:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:12:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:12:15.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:12:15.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:12:15.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:12:15.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:12:15.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:12:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:14 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T05:12:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:14 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T05:12:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21166.023481 secs 2023-09-10T05:12:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:12:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:12:15.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:14 smithi082 ceph-mon[98694]: osdmap e784: 12 total, 12 up, 12 in 2023-09-10T05:12:15.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:12:15.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:12:15.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:12:15.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:12:15.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:12:15.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:12:15.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:12:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:14 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T05:12:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:14 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T05:12:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21166.023481 secs 2023-09-10T05:12:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:12:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:12:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:14 smithi098 ceph-mon[103124]: osdmap e784: 12 total, 12 up, 12 in 2023-09-10T05:12:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:12:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:12:15.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:12:15.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:12:15.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:12:15.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:12:15.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:12:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:15 smithi170 ceph-mon[102832]: pgmap v11450: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 383 B/s wr, 6 op/s 2023-09-10T05:12:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:15 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T05:12:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:15 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T05:12:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:15 smithi082 ceph-mon[98694]: pgmap v11450: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 383 B/s wr, 6 op/s 2023-09-10T05:12:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:15 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T05:12:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:15 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T05:12:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:15 smithi098 ceph-mon[103124]: pgmap v11450: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 383 B/s wr, 6 op/s 2023-09-10T05:12:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:15 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T05:12:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:15 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T05:12:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:17 smithi170 ceph-mon[102832]: pgmap v11451: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 94 KiB/s wr, 13 op/s 2023-09-10T05:12:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:17 smithi082 ceph-mon[98694]: pgmap v11451: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 94 KiB/s wr, 13 op/s 2023-09-10T05:12:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:17 smithi098 ceph-mon[103124]: pgmap v11451: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 94 KiB/s wr, 13 op/s 2023-09-10T05:12:19.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21169.329124 secs 2023-09-10T05:12:19.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21170.369745 secs 2023-09-10T05:12:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21169.329124 secs 2023-09-10T05:12:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21170.369745 secs 2023-09-10T05:12:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21169.329124 secs 2023-09-10T05:12:19.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21170.369745 secs 2023-09-10T05:12:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:19 smithi170 ceph-mon[102832]: pgmap v11452: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:12:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21171.023548 secs 2023-09-10T05:12:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:19 smithi082 ceph-mon[98694]: pgmap v11452: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:12:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21171.023548 secs 2023-09-10T05:12:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:19 smithi098 ceph-mon[103124]: pgmap v11452: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:12:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21171.023548 secs 2023-09-10T05:12:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:21 smithi170 ceph-mon[102832]: pgmap v11453: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T05:12:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:21 smithi082 ceph-mon[98694]: pgmap v11453: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T05:12:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:21 smithi098 ceph-mon[103124]: pgmap v11453: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T05:12:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:23 smithi170 ceph-mon[102832]: pgmap v11454: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T05:12:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21174.329206 secs 2023-09-10T05:12:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21175.369803 secs 2023-09-10T05:12:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:23 smithi082 ceph-mon[98694]: pgmap v11454: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T05:12:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21174.329206 secs 2023-09-10T05:12:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21175.369803 secs 2023-09-10T05:12:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:23 smithi098 ceph-mon[103124]: pgmap v11454: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T05:12:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21174.329206 secs 2023-09-10T05:12:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21175.369803 secs 2023-09-10T05:12:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21176.023609 secs 2023-09-10T05:12:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:12:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:12:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21176.023609 secs 2023-09-10T05:12:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:12:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:12:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21176.023609 secs 2023-09-10T05:12:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:12:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:12:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:25 smithi170 ceph-mon[102832]: pgmap v11455: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 12 op/s 2023-09-10T05:12:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:25 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:12:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:25 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:12:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:25 smithi082 ceph-mon[98694]: pgmap v11455: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 12 op/s 2023-09-10T05:12:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:25 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:12:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:25 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:12:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:25 smithi098 ceph-mon[103124]: pgmap v11455: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 12 op/s 2023-09-10T05:12:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:25 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:12:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:25 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:12:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:27 smithi170 ceph-mon[102832]: pgmap v11456: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 15 op/s 2023-09-10T05:12:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:27 smithi082 ceph-mon[98694]: pgmap v11456: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 15 op/s 2023-09-10T05:12:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:27 smithi098 ceph-mon[103124]: pgmap v11456: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 15 op/s 2023-09-10T05:12:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21180.369859 secs 2023-09-10T05:12:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21180.369859 secs 2023-09-10T05:12:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21180.369859 secs 2023-09-10T05:12:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:29 smithi170 ceph-mon[102832]: pgmap v11457: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T05:12:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21181.023672 secs 2023-09-10T05:12:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:29 smithi082 ceph-mon[98694]: pgmap v11457: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T05:12:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21181.023672 secs 2023-09-10T05:12:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:29 smithi098 ceph-mon[103124]: pgmap v11457: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T05:12:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21181.023672 secs 2023-09-10T05:12:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:31 smithi170 ceph-mon[102832]: pgmap v11458: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 137 KiB/s wr, 14 op/s 2023-09-10T05:12:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:12:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:31 smithi082 ceph-mon[98694]: pgmap v11458: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 137 KiB/s wr, 14 op/s 2023-09-10T05:12:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:12:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:31 smithi098 ceph-mon[103124]: pgmap v11458: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 137 KiB/s wr, 14 op/s 2023-09-10T05:12:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:12:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:12:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:12:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:12:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:12:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:12:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:12:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:12:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:12:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:12:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:33 smithi170 ceph-mon[102832]: pgmap v11459: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 9 op/s 2023-09-10T05:12:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21185.369918 secs 2023-09-10T05:12:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:33 smithi082 ceph-mon[98694]: pgmap v11459: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 9 op/s 2023-09-10T05:12:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21185.369918 secs 2023-09-10T05:12:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:33 smithi098 ceph-mon[103124]: pgmap v11459: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 9 op/s 2023-09-10T05:12:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21185.369918 secs 2023-09-10T05:12:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21186.023732 secs 2023-09-10T05:12:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21186.023732 secs 2023-09-10T05:12:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21186.023732 secs 2023-09-10T05:12:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:35 smithi170 ceph-mon[102832]: pgmap v11460: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 9 op/s 2023-09-10T05:12:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:35 smithi082 ceph-mon[98694]: pgmap v11460: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 9 op/s 2023-09-10T05:12:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:35 smithi098 ceph-mon[103124]: pgmap v11460: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 9 op/s 2023-09-10T05:12:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:37 smithi170 ceph-mon[102832]: pgmap v11461: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 15 op/s 2023-09-10T05:12:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:37 smithi082 ceph-mon[98694]: pgmap v11461: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 15 op/s 2023-09-10T05:12:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:37 smithi098 ceph-mon[103124]: pgmap v11461: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 15 op/s 2023-09-10T05:12:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21190.369977 secs 2023-09-10T05:12:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21190.369977 secs 2023-09-10T05:12:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21190.369977 secs 2023-09-10T05:12:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:39 smithi170 ceph-mon[102832]: pgmap v11462: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 214 KiB/s wr, 10 op/s 2023-09-10T05:12:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21191.023799 secs 2023-09-10T05:12:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:39 smithi082 ceph-mon[98694]: pgmap v11462: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 214 KiB/s wr, 10 op/s 2023-09-10T05:12:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21191.023799 secs 2023-09-10T05:12:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:39 smithi098 ceph-mon[103124]: pgmap v11462: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 214 KiB/s wr, 10 op/s 2023-09-10T05:12:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21191.023799 secs 2023-09-10T05:12:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:41 smithi170 ceph-mon[102832]: pgmap v11463: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 329 KiB/s wr, 16 op/s 2023-09-10T05:12:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:41 smithi082 ceph-mon[98694]: pgmap v11463: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 329 KiB/s wr, 16 op/s 2023-09-10T05:12:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:41 smithi098 ceph-mon[103124]: pgmap v11463: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 329 KiB/s wr, 16 op/s 2023-09-10T05:12:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:43 smithi170 ceph-mon[102832]: pgmap v11464: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 261 KiB/s wr, 11 op/s 2023-09-10T05:12:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21195.370083 secs 2023-09-10T05:12:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:43 smithi082 ceph-mon[98694]: pgmap v11464: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 261 KiB/s wr, 11 op/s 2023-09-10T05:12:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21195.370083 secs 2023-09-10T05:12:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:43 smithi098 ceph-mon[103124]: pgmap v11464: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 261 KiB/s wr, 11 op/s 2023-09-10T05:12:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21195.370083 secs 2023-09-10T05:12:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21196.023862 secs 2023-09-10T05:12:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:12:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:12:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21196.023862 secs 2023-09-10T05:12:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:12:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:12:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21196.023862 secs 2023-09-10T05:12:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:12:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:12:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:45 smithi170 ceph-mon[102832]: pgmap v11465: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 261 KiB/s wr, 11 op/s 2023-09-10T05:12:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:12:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:12:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:45 smithi082 ceph-mon[98694]: pgmap v11465: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 261 KiB/s wr, 11 op/s 2023-09-10T05:12:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:12:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:12:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:45 smithi098 ceph-mon[103124]: pgmap v11465: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 261 KiB/s wr, 11 op/s 2023-09-10T05:12:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:12:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:12:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:47 smithi170 ceph-mon[102832]: pgmap v11466: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 445 KiB/s wr, 18 op/s 2023-09-10T05:12:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:47 smithi082 ceph-mon[98694]: pgmap v11466: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 445 KiB/s wr, 18 op/s 2023-09-10T05:12:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:47 smithi098 ceph-mon[103124]: pgmap v11466: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 445 KiB/s wr, 18 op/s 2023-09-10T05:12:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21200.370140 secs 2023-09-10T05:12:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21200.370140 secs 2023-09-10T05:12:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21200.370140 secs 2023-09-10T05:12:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:49 smithi170 ceph-mon[102832]: pgmap v11467: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 298 KiB/s wr, 12 op/s 2023-09-10T05:12:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21201.023928 secs 2023-09-10T05:12:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:49 smithi082 ceph-mon[98694]: pgmap v11467: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 298 KiB/s wr, 12 op/s 2023-09-10T05:12:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21201.023928 secs 2023-09-10T05:12:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:49 smithi098 ceph-mon[103124]: pgmap v11467: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 298 KiB/s wr, 12 op/s 2023-09-10T05:12:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21201.023928 secs 2023-09-10T05:12:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:51 smithi170 ceph-mon[102832]: pgmap v11468: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 322 KiB/s wr, 18 op/s 2023-09-10T05:12:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:51 smithi082 ceph-mon[98694]: pgmap v11468: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 322 KiB/s wr, 18 op/s 2023-09-10T05:12:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:51 smithi098 ceph-mon[103124]: pgmap v11468: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 322 KiB/s wr, 18 op/s 2023-09-10T05:12:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:53 smithi170 ceph-mon[102832]: pgmap v11469: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 207 KiB/s wr, 12 op/s 2023-09-10T05:12:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:12:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:12:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21205.370222 secs 2023-09-10T05:12:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:53 smithi082 ceph-mon[98694]: pgmap v11469: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 207 KiB/s wr, 12 op/s 2023-09-10T05:12:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:12:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:12:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21205.370222 secs 2023-09-10T05:12:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:53 smithi098 ceph-mon[103124]: pgmap v11469: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 207 KiB/s wr, 12 op/s 2023-09-10T05:12:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:12:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:12:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21205.370222 secs 2023-09-10T05:12:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21206.023990 secs 2023-09-10T05:12:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21206.023990 secs 2023-09-10T05:12:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21206.023990 secs 2023-09-10T05:12:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:55 smithi170 ceph-mon[102832]: pgmap v11470: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 207 KiB/s wr, 12 op/s 2023-09-10T05:12:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:55 smithi082 ceph-mon[98694]: pgmap v11470: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 207 KiB/s wr, 12 op/s 2023-09-10T05:12:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:55 smithi098 ceph-mon[103124]: pgmap v11470: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 207 KiB/s wr, 12 op/s 2023-09-10T05:12:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:12:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:12:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:12:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:12:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:12:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:12:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:57 smithi170 ceph-mon[102832]: pgmap v11471: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 469 KiB/s wr, 18 op/s 2023-09-10T05:12:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:57 smithi082 ceph-mon[98694]: pgmap v11471: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 469 KiB/s wr, 18 op/s 2023-09-10T05:12:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:57 smithi098 ceph-mon[103124]: pgmap v11471: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 469 KiB/s wr, 18 op/s 2023-09-10T05:12:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21209.329802 secs 2023-09-10T05:12:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21210.370306 secs 2023-09-10T05:12:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21209.329802 secs 2023-09-10T05:12:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21210.370306 secs 2023-09-10T05:12:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21209.329802 secs 2023-09-10T05:12:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21210.370306 secs 2023-09-10T05:13:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:59 smithi170 ceph-mon[102832]: pgmap v11472: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 286 KiB/s wr, 12 op/s 2023-09-10T05:13:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21211.024055 secs 2023-09-10T05:13:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:59 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:13:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:59 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:13:00.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:12:59 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:59 smithi082 ceph-mon[98694]: pgmap v11472: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 286 KiB/s wr, 12 op/s 2023-09-10T05:13:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21211.024055 secs 2023-09-10T05:13:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:59 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:13:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:59 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:13:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:12:59 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:59 smithi098 ceph-mon[103124]: pgmap v11472: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 286 KiB/s wr, 12 op/s 2023-09-10T05:13:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21211.024055 secs 2023-09-10T05:13:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:59 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:13:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:59 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:13:00.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:12:59 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:01 smithi170 ceph-mon[102832]: pgmap v11473: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 432 KiB/s wr, 19 op/s 2023-09-10T05:13:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:01 smithi082 ceph-mon[98694]: pgmap v11473: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 432 KiB/s wr, 19 op/s 2023-09-10T05:13:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:01 smithi098 ceph-mon[103124]: pgmap v11473: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 432 KiB/s wr, 19 op/s 2023-09-10T05:13:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:03 smithi170 ceph-mon[102832]: pgmap v11474: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 409 KiB/s wr, 14 op/s 2023-09-10T05:13:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21214.329865 secs 2023-09-10T05:13:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:03 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:13:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:03 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:04.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21215.370365 secs 2023-09-10T05:13:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:03 smithi082 ceph-mon[98694]: pgmap v11474: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 409 KiB/s wr, 14 op/s 2023-09-10T05:13:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21214.329865 secs 2023-09-10T05:13:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:03 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:13:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:03 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21215.370365 secs 2023-09-10T05:13:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:03 smithi098 ceph-mon[103124]: pgmap v11474: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 409 KiB/s wr, 14 op/s 2023-09-10T05:13:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21214.329865 secs 2023-09-10T05:13:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:03 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:13:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:03 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:04.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21215.370365 secs 2023-09-10T05:13:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21216.024114 secs 2023-09-10T05:13:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21216.024114 secs 2023-09-10T05:13:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21216.024114 secs 2023-09-10T05:13:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:05 smithi170 ceph-mon[102832]: pgmap v11475: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 409 KiB/s wr, 14 op/s 2023-09-10T05:13:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:13:06.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:05 smithi082 ceph-mon[98694]: pgmap v11475: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 409 KiB/s wr, 14 op/s 2023-09-10T05:13:06.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:13:06.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:05 smithi098 ceph-mon[103124]: pgmap v11475: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 409 KiB/s wr, 14 op/s 2023-09-10T05:13:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:13:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:07 smithi170 ceph-mon[102832]: pgmap v11476: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 591 KiB/s wr, 32 op/s 2023-09-10T05:13:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:07 smithi082 ceph-mon[98694]: pgmap v11476: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 591 KiB/s wr, 32 op/s 2023-09-10T05:13:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:07 smithi098 ceph-mon[103124]: pgmap v11476: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 591 KiB/s wr, 32 op/s 2023-09-10T05:13:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21219.329926 secs 2023-09-10T05:13:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:08 smithi082 ceph-mon[98694]: osdmap e785: 12 total, 12 up, 12 in 2023-09-10T05:13:09.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21220.370471 secs 2023-09-10T05:13:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21219.329926 secs 2023-09-10T05:13:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:08 smithi098 ceph-mon[103124]: osdmap e785: 12 total, 12 up, 12 in 2023-09-10T05:13:09.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21220.370471 secs 2023-09-10T05:13:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21219.329926 secs 2023-09-10T05:13:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:08 smithi170 ceph-mon[102832]: osdmap e785: 12 total, 12 up, 12 in 2023-09-10T05:13:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21220.370471 secs 2023-09-10T05:13:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:09 smithi082 ceph-mon[98694]: pgmap v11478: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 395 KiB/s wr, 30 op/s 2023-09-10T05:13:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21221.024179 secs 2023-09-10T05:13:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:09 smithi098 ceph-mon[103124]: pgmap v11478: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 395 KiB/s wr, 30 op/s 2023-09-10T05:13:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21221.024179 secs 2023-09-10T05:13:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:09 smithi170 ceph-mon[102832]: pgmap v11478: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 395 KiB/s wr, 30 op/s 2023-09-10T05:13:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21221.024179 secs 2023-09-10T05:13:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:11 smithi082 ceph-mon[98694]: pgmap v11479: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 302 KiB/s wr, 28 op/s 2023-09-10T05:13:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:11 smithi098 ceph-mon[103124]: pgmap v11479: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 302 KiB/s wr, 28 op/s 2023-09-10T05:13:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:11 smithi170 ceph-mon[102832]: pgmap v11479: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 302 KiB/s wr, 28 op/s 2023-09-10T05:13:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:13 smithi082 ceph-mon[98694]: pgmap v11480: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 302 KiB/s wr, 28 op/s 2023-09-10T05:13:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21224.329985 secs 2023-09-10T05:13:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21225.370492 secs 2023-09-10T05:13:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:13 smithi098 ceph-mon[103124]: pgmap v11480: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 302 KiB/s wr, 28 op/s 2023-09-10T05:13:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21224.329985 secs 2023-09-10T05:13:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21225.370492 secs 2023-09-10T05:13:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:13 smithi170 ceph-mon[102832]: pgmap v11480: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 302 KiB/s wr, 28 op/s 2023-09-10T05:13:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21224.329985 secs 2023-09-10T05:13:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21225.370492 secs 2023-09-10T05:13:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21226.024240 secs 2023-09-10T05:13:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:14 smithi082 ceph-mon[98694]: osdmap e786: 12 total, 12 up, 12 in 2023-09-10T05:13:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:13:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21226.024240 secs 2023-09-10T05:13:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:14 smithi098 ceph-mon[103124]: osdmap e786: 12 total, 12 up, 12 in 2023-09-10T05:13:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:13:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21226.024240 secs 2023-09-10T05:13:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:14 smithi170 ceph-mon[102832]: osdmap e786: 12 total, 12 up, 12 in 2023-09-10T05:13:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:13:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:15 smithi082 ceph-mon[98694]: pgmap v11482: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 104 KiB/s wr, 8 op/s 2023-09-10T05:13:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:13:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:13:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:13:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:15 smithi098 ceph-mon[103124]: pgmap v11482: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 104 KiB/s wr, 8 op/s 2023-09-10T05:13:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:13:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:13:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:13:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:15 smithi170 ceph-mon[102832]: pgmap v11482: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 104 KiB/s wr, 8 op/s 2023-09-10T05:13:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:13:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:13:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:13:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:17 smithi082 ceph-mon[98694]: pgmap v11483: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 97 KiB/s wr, 13 op/s 2023-09-10T05:13:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:17 smithi098 ceph-mon[103124]: pgmap v11483: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 97 KiB/s wr, 13 op/s 2023-09-10T05:13:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:17 smithi170 ceph-mon[102832]: pgmap v11483: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 97 KiB/s wr, 13 op/s 2023-09-10T05:13:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21229.330044 secs 2023-09-10T05:13:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21230.370550 secs 2023-09-10T05:13:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21229.330044 secs 2023-09-10T05:13:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21230.370550 secs 2023-09-10T05:13:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21229.330044 secs 2023-09-10T05:13:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21230.370550 secs 2023-09-10T05:13:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:19 smithi082 ceph-mon[98694]: pgmap v11484: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T05:13:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21231.024306 secs 2023-09-10T05:13:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:19 smithi098 ceph-mon[103124]: pgmap v11484: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T05:13:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21231.024306 secs 2023-09-10T05:13:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:19 smithi170 ceph-mon[102832]: pgmap v11484: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T05:13:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21231.024306 secs 2023-09-10T05:13:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:21 smithi082 ceph-mon[98694]: pgmap v11485: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T05:13:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:21 smithi098 ceph-mon[103124]: pgmap v11485: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T05:13:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:21 smithi170 ceph-mon[102832]: pgmap v11485: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T05:13:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:23 smithi082 ceph-mon[98694]: pgmap v11486: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T05:13:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21234.330103 secs 2023-09-10T05:13:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21235.370637 secs 2023-09-10T05:13:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:23 smithi098 ceph-mon[103124]: pgmap v11486: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T05:13:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21234.330103 secs 2023-09-10T05:13:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21235.370637 secs 2023-09-10T05:13:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:23 smithi170 ceph-mon[102832]: pgmap v11486: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T05:13:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21234.330103 secs 2023-09-10T05:13:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21235.370637 secs 2023-09-10T05:13:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21236.024367 secs 2023-09-10T05:13:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21236.024367 secs 2023-09-10T05:13:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21236.024367 secs 2023-09-10T05:13:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:25 smithi082 ceph-mon[98694]: pgmap v11487: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T05:13:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:25 smithi098 ceph-mon[103124]: pgmap v11487: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T05:13:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:25 smithi170 ceph-mon[102832]: pgmap v11487: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T05:13:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:27 smithi098 ceph-mon[103124]: pgmap v11488: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T05:13:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:27 smithi170 ceph-mon[102832]: pgmap v11488: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T05:13:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:27 smithi082 ceph-mon[98694]: pgmap v11488: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-09-10T05:13:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21240.370697 secs 2023-09-10T05:13:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21240.370697 secs 2023-09-10T05:13:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21240.370697 secs 2023-09-10T05:13:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:29 smithi098 ceph-mon[103124]: pgmap v11489: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T05:13:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21241.024431 secs 2023-09-10T05:13:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:29 smithi170 ceph-mon[102832]: pgmap v11489: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T05:13:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21241.024431 secs 2023-09-10T05:13:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:29 smithi082 ceph-mon[98694]: pgmap v11489: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T05:13:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21241.024431 secs 2023-09-10T05:13:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:30 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:13:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:30 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:13:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:30 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:13:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:31 smithi098 ceph-mon[103124]: pgmap v11490: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 241 KiB/s wr, 15 op/s 2023-09-10T05:13:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:31 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:13:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:13:32.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:31 smithi170 ceph-mon[102832]: pgmap v11490: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 241 KiB/s wr, 15 op/s 2023-09-10T05:13:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:31 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:13:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:13:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:31 smithi082 ceph-mon[98694]: pgmap v11490: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 241 KiB/s wr, 15 op/s 2023-09-10T05:13:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:31 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:13:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:13:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:33 smithi098 ceph-mon[103124]: pgmap v11491: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 171 KiB/s wr, 10 op/s 2023-09-10T05:13:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:13:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21245.370787 secs 2023-09-10T05:13:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:33 smithi170 ceph-mon[102832]: pgmap v11491: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 171 KiB/s wr, 10 op/s 2023-09-10T05:13:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:13:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21245.370787 secs 2023-09-10T05:13:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:33 smithi082 ceph-mon[98694]: pgmap v11491: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 171 KiB/s wr, 10 op/s 2023-09-10T05:13:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:13:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21245.370787 secs 2023-09-10T05:13:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21246.024491 secs 2023-09-10T05:13:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21246.024491 secs 2023-09-10T05:13:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21246.024491 secs 2023-09-10T05:13:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:35 smithi098 ceph-mon[103124]: pgmap v11492: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 171 KiB/s wr, 10 op/s 2023-09-10T05:13:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:35 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:13:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:35 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:35 smithi170 ceph-mon[102832]: pgmap v11492: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 171 KiB/s wr, 10 op/s 2023-09-10T05:13:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:35 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:13:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:35 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:35 smithi082 ceph-mon[98694]: pgmap v11492: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 171 KiB/s wr, 10 op/s 2023-09-10T05:13:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:35 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:13:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:35 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:37 smithi098 ceph-mon[103124]: pgmap v11493: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 505 KiB/s wr, 17 op/s 2023-09-10T05:13:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:13:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:37 smithi170 ceph-mon[102832]: pgmap v11493: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 505 KiB/s wr, 17 op/s 2023-09-10T05:13:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:13:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:37 smithi082 ceph-mon[98694]: pgmap v11493: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 505 KiB/s wr, 17 op/s 2023-09-10T05:13:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:13:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21250.370845 secs 2023-09-10T05:13:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21251.024558 secs 2023-09-10T05:13:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21250.370845 secs 2023-09-10T05:13:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21251.024558 secs 2023-09-10T05:13:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21250.370845 secs 2023-09-10T05:13:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21251.024558 secs 2023-09-10T05:13:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:39 smithi098 ceph-mon[103124]: pgmap v11494: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 504 KiB/s wr, 12 op/s 2023-09-10T05:13:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:39 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:13:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:39 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:39 smithi170 ceph-mon[102832]: pgmap v11494: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 504 KiB/s wr, 12 op/s 2023-09-10T05:13:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:39 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:13:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:39 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:39 smithi082 ceph-mon[98694]: pgmap v11494: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 504 KiB/s wr, 12 op/s 2023-09-10T05:13:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:39 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:13:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:39 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:41 smithi098 ceph-mon[103124]: pgmap v11495: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 624 KiB/s wr, 19 op/s 2023-09-10T05:13:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:41 smithi170 ceph-mon[102832]: pgmap v11495: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 624 KiB/s wr, 19 op/s 2023-09-10T05:13:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:41 smithi082 ceph-mon[98694]: pgmap v11495: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 624 KiB/s wr, 19 op/s 2023-09-10T05:13:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:43 smithi098 ceph-mon[103124]: pgmap v11496: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 454 KiB/s wr, 13 op/s 2023-09-10T05:13:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21255.370916 secs 2023-09-10T05:13:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21256.024621 secs 2023-09-10T05:13:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:43 smithi170 ceph-mon[102832]: pgmap v11496: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 454 KiB/s wr, 13 op/s 2023-09-10T05:13:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21255.370916 secs 2023-09-10T05:13:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21256.024621 secs 2023-09-10T05:13:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:43 smithi082 ceph-mon[98694]: pgmap v11496: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 454 KiB/s wr, 13 op/s 2023-09-10T05:13:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21255.370916 secs 2023-09-10T05:13:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21256.024621 secs 2023-09-10T05:13:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:45 smithi170 ceph-mon[102832]: pgmap v11497: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 454 KiB/s wr, 13 op/s 2023-09-10T05:13:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:45 smithi082 ceph-mon[98694]: pgmap v11497: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 454 KiB/s wr, 13 op/s 2023-09-10T05:13:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:45 smithi098 ceph-mon[103124]: pgmap v11497: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 454 KiB/s wr, 13 op/s 2023-09-10T05:13:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:47 smithi170 ceph-mon[102832]: pgmap v11498: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 602 KiB/s wr, 19 op/s 2023-09-10T05:13:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:47 smithi082 ceph-mon[98694]: pgmap v11498: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 602 KiB/s wr, 19 op/s 2023-09-10T05:13:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:47 smithi098 ceph-mon[103124]: pgmap v11498: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 602 KiB/s wr, 19 op/s 2023-09-10T05:13:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21259.330502 secs 2023-09-10T05:13:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21260.370973 secs 2023-09-10T05:13:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21259.330502 secs 2023-09-10T05:13:48.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21260.370973 secs 2023-09-10T05:13:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21259.330502 secs 2023-09-10T05:13:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21260.370973 secs 2023-09-10T05:13:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:49 smithi170 ceph-mon[102832]: pgmap v11499: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 268 KiB/s wr, 13 op/s 2023-09-10T05:13:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:13:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21261.024687 secs 2023-09-10T05:13:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:49 smithi082 ceph-mon[98694]: pgmap v11499: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 268 KiB/s wr, 13 op/s 2023-09-10T05:13:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:13:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21261.024687 secs 2023-09-10T05:13:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:49 smithi098 ceph-mon[103124]: pgmap v11499: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 268 KiB/s wr, 13 op/s 2023-09-10T05:13:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:13:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:50.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21261.024687 secs 2023-09-10T05:13:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:51 smithi170 ceph-mon[102832]: pgmap v11500: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 440 KiB/s wr, 18 op/s 2023-09-10T05:13:51.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:51 smithi082 ceph-mon[98694]: pgmap v11500: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 440 KiB/s wr, 18 op/s 2023-09-10T05:13:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:51 smithi098 ceph-mon[103124]: pgmap v11500: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 440 KiB/s wr, 18 op/s 2023-09-10T05:13:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:13:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:13:52.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:13:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:13:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:13:53.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:13:53.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:13:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:53 smithi170 ceph-mon[102832]: pgmap v11501: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 320 KiB/s wr, 12 op/s 2023-09-10T05:13:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21264.330564 secs 2023-09-10T05:13:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21265.371056 secs 2023-09-10T05:13:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:53 smithi082 ceph-mon[98694]: pgmap v11501: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 320 KiB/s wr, 12 op/s 2023-09-10T05:13:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21264.330564 secs 2023-09-10T05:13:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21265.371056 secs 2023-09-10T05:13:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:53 smithi098 ceph-mon[103124]: pgmap v11501: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 320 KiB/s wr, 12 op/s 2023-09-10T05:13:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21264.330564 secs 2023-09-10T05:13:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21265.371056 secs 2023-09-10T05:13:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21266.024749 secs 2023-09-10T05:13:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21266.024749 secs 2023-09-10T05:13:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21266.024749 secs 2023-09-10T05:13:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:55 smithi170 ceph-mon[102832]: pgmap v11502: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 320 KiB/s wr, 12 op/s 2023-09-10T05:13:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:55 smithi082 ceph-mon[98694]: pgmap v11502: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 320 KiB/s wr, 12 op/s 2023-09-10T05:13:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:55 smithi098 ceph-mon[103124]: pgmap v11502: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 320 KiB/s wr, 12 op/s 2023-09-10T05:13:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:57 smithi170 ceph-mon[102832]: pgmap v11503: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 480 KiB/s wr, 17 op/s 2023-09-10T05:13:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:57 smithi082 ceph-mon[98694]: pgmap v11503: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 480 KiB/s wr, 17 op/s 2023-09-10T05:13:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:57 smithi098 ceph-mon[103124]: pgmap v11503: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 480 KiB/s wr, 17 op/s 2023-09-10T05:13:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21269.330677 secs 2023-09-10T05:13:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21270.371114 secs 2023-09-10T05:13:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21269.330677 secs 2023-09-10T05:13:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21270.371114 secs 2023-09-10T05:13:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21269.330677 secs 2023-09-10T05:13:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21270.371114 secs 2023-09-10T05:13:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:59 smithi170 ceph-mon[102832]: pgmap v11504: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 332 KiB/s wr, 11 op/s 2023-09-10T05:13:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:13:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21271.024814 secs 2023-09-10T05:13:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:59 smithi082 ceph-mon[98694]: pgmap v11504: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 332 KiB/s wr, 11 op/s 2023-09-10T05:13:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:13:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21271.024814 secs 2023-09-10T05:14:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:59 smithi098 ceph-mon[103124]: pgmap v11504: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 332 KiB/s wr, 11 op/s 2023-09-10T05:14:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:13:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21271.024814 secs 2023-09-10T05:14:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:01 smithi170 ceph-mon[102832]: pgmap v11505: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 348 KiB/s wr, 16 op/s 2023-09-10T05:14:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:01 smithi082 ceph-mon[98694]: pgmap v11505: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 348 KiB/s wr, 16 op/s 2023-09-10T05:14:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:01 smithi098 ceph-mon[103124]: pgmap v11505: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 348 KiB/s wr, 16 op/s 2023-09-10T05:14:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:03 smithi170 ceph-mon[102832]: pgmap v11506: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 10 op/s 2023-09-10T05:14:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21274.330737 secs 2023-09-10T05:14:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21275.371199 secs 2023-09-10T05:14:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:03 smithi082 ceph-mon[98694]: pgmap v11506: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 10 op/s 2023-09-10T05:14:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21274.330737 secs 2023-09-10T05:14:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21275.371199 secs 2023-09-10T05:14:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:03 smithi098 ceph-mon[103124]: pgmap v11506: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 10 op/s 2023-09-10T05:14:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21274.330737 secs 2023-09-10T05:14:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21275.371199 secs 2023-09-10T05:14:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21276.024877 secs 2023-09-10T05:14:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21276.024877 secs 2023-09-10T05:14:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21276.024877 secs 2023-09-10T05:14:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:05 smithi082 ceph-mon[98694]: pgmap v11507: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 10 op/s 2023-09-10T05:14:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:05 smithi098 ceph-mon[103124]: pgmap v11507: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 10 op/s 2023-09-10T05:14:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:05 smithi170 ceph-mon[102832]: pgmap v11507: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 10 op/s 2023-09-10T05:14:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:07 smithi082 ceph-mon[98694]: pgmap v11508: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 286 KiB/s wr, 30 op/s 2023-09-10T05:14:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:07 smithi098 ceph-mon[103124]: pgmap v11508: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 286 KiB/s wr, 30 op/s 2023-09-10T05:14:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:07 smithi170 ceph-mon[102832]: pgmap v11508: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 286 KiB/s wr, 30 op/s 2023-09-10T05:14:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21279.330822 secs 2023-09-10T05:14:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:08 smithi082 ceph-mon[98694]: osdmap e787: 12 total, 12 up, 12 in 2023-09-10T05:14:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21280.371287 secs 2023-09-10T05:14:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21279.330822 secs 2023-09-10T05:14:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:08 smithi098 ceph-mon[103124]: osdmap e787: 12 total, 12 up, 12 in 2023-09-10T05:14:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21280.371287 secs 2023-09-10T05:14:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21279.330822 secs 2023-09-10T05:14:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:08 smithi170 ceph-mon[102832]: osdmap e787: 12 total, 12 up, 12 in 2023-09-10T05:14:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21280.371287 secs 2023-09-10T05:14:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:09 smithi082 ceph-mon[98694]: pgmap v11510: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 151 KiB/s wr, 29 op/s 2023-09-10T05:14:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21281.024943 secs 2023-09-10T05:14:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:09 smithi098 ceph-mon[103124]: pgmap v11510: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 151 KiB/s wr, 29 op/s 2023-09-10T05:14:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21281.024943 secs 2023-09-10T05:14:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:09 smithi170 ceph-mon[102832]: pgmap v11510: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 151 KiB/s wr, 29 op/s 2023-09-10T05:14:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21281.024943 secs 2023-09-10T05:14:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:11 smithi082 ceph-mon[98694]: pgmap v11511: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 151 KiB/s wr, 29 op/s 2023-09-10T05:14:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:11 smithi098 ceph-mon[103124]: pgmap v11511: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 151 KiB/s wr, 29 op/s 2023-09-10T05:14:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:11 smithi170 ceph-mon[102832]: pgmap v11511: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 151 KiB/s wr, 29 op/s 2023-09-10T05:14:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:13 smithi082 ceph-mon[98694]: pgmap v11512: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 151 KiB/s wr, 29 op/s 2023-09-10T05:14:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21284.330883 secs 2023-09-10T05:14:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21285.371346 secs 2023-09-10T05:14:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:13 smithi098 ceph-mon[103124]: pgmap v11512: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 151 KiB/s wr, 29 op/s 2023-09-10T05:14:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21284.330883 secs 2023-09-10T05:14:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21285.371346 secs 2023-09-10T05:14:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:13 smithi170 ceph-mon[102832]: pgmap v11512: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 151 KiB/s wr, 29 op/s 2023-09-10T05:14:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21284.330883 secs 2023-09-10T05:14:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21285.371346 secs 2023-09-10T05:14:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21286.025003 secs 2023-09-10T05:14:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:14 smithi082 ceph-mon[98694]: osdmap e788: 12 total, 12 up, 12 in 2023-09-10T05:14:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21286.025003 secs 2023-09-10T05:14:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:14 smithi098 ceph-mon[103124]: osdmap e788: 12 total, 12 up, 12 in 2023-09-10T05:14:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21286.025003 secs 2023-09-10T05:14:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:14 smithi170 ceph-mon[102832]: osdmap e788: 12 total, 12 up, 12 in 2023-09-10T05:14:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:15 smithi082 ceph-mon[98694]: pgmap v11514: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 7 op/s 2023-09-10T05:14:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:14:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:15 smithi098 ceph-mon[103124]: pgmap v11514: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 7 op/s 2023-09-10T05:14:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:14:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:15 smithi170 ceph-mon[102832]: pgmap v11514: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 7 op/s 2023-09-10T05:14:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:14:17.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:14:17.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:14:17.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:14:17.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:14:17.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:14:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:14:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:14:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:14:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:14:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:14:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:14:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:14:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:14:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:14:17.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:14:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:18 smithi170 ceph-mon[102832]: pgmap v11515: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 13 op/s 2023-09-10T05:14:18.601 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:18 smithi098 ceph-mon[103124]: pgmap v11515: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 13 op/s 2023-09-10T05:14:18.602 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:18 smithi082 ceph-mon[98694]: pgmap v11515: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 117 KiB/s wr, 13 op/s 2023-09-10T05:14:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21289.330943 secs 2023-09-10T05:14:19.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21290.371405 secs 2023-09-10T05:14:19.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21291.025069 secs 2023-09-10T05:14:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21289.330943 secs 2023-09-10T05:14:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21290.371405 secs 2023-09-10T05:14:19.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21291.025069 secs 2023-09-10T05:14:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21289.330943 secs 2023-09-10T05:14:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21290.371405 secs 2023-09-10T05:14:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21291.025069 secs 2023-09-10T05:14:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:20 smithi170 ceph-mon[102832]: pgmap v11516: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T05:14:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:20 smithi082 ceph-mon[98694]: pgmap v11516: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T05:14:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:20 smithi098 ceph-mon[103124]: pgmap v11516: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T05:14:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:22 smithi170 ceph-mon[102832]: pgmap v11517: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T05:14:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:22 smithi082 ceph-mon[98694]: pgmap v11517: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T05:14:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:22 smithi098 ceph-mon[103124]: pgmap v11517: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T05:14:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:24 smithi170 ceph-mon[102832]: pgmap v11518: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T05:14:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21294.331003 secs 2023-09-10T05:14:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21295.371463 secs 2023-09-10T05:14:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21296.025131 secs 2023-09-10T05:14:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:24 smithi082 ceph-mon[98694]: pgmap v11518: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T05:14:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21294.331003 secs 2023-09-10T05:14:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21295.371463 secs 2023-09-10T05:14:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21296.025131 secs 2023-09-10T05:14:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:24 smithi098 ceph-mon[103124]: pgmap v11518: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T05:14:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21294.331003 secs 2023-09-10T05:14:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21295.371463 secs 2023-09-10T05:14:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21296.025131 secs 2023-09-10T05:14:25.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:25 smithi170 ceph-mon[102832]: pgmap v11519: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T05:14:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:25 smithi082 ceph-mon[98694]: pgmap v11519: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T05:14:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:25 smithi098 ceph-mon[103124]: pgmap v11519: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 84 KiB/s wr, 11 op/s 2023-09-10T05:14:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:27 smithi170 ceph-mon[102832]: pgmap v11520: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 138 KiB/s wr, 14 op/s 2023-09-10T05:14:27.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:27 smithi082 ceph-mon[98694]: pgmap v11520: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 138 KiB/s wr, 14 op/s 2023-09-10T05:14:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:27 smithi098 ceph-mon[103124]: pgmap v11520: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 138 KiB/s wr, 14 op/s 2023-09-10T05:14:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21300.371518 secs 2023-09-10T05:14:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21300.371518 secs 2023-09-10T05:14:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21300.371518 secs 2023-09-10T05:14:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:29 smithi170 ceph-mon[102832]: pgmap v11521: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 9 op/s 2023-09-10T05:14:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:29 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:14:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:29 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:14:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21301.025198 secs 2023-09-10T05:14:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:29 smithi082 ceph-mon[98694]: pgmap v11521: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 9 op/s 2023-09-10T05:14:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:29 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:14:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:29 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:14:29.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21301.025198 secs 2023-09-10T05:14:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:29 smithi098 ceph-mon[103124]: pgmap v11521: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 KiB/s wr, 9 op/s 2023-09-10T05:14:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:29 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:14:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:29 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:14:30.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21301.025198 secs 2023-09-10T05:14:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:31 smithi170 ceph-mon[102832]: pgmap v11522: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 227 KiB/s wr, 16 op/s 2023-09-10T05:14:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:14:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:31 smithi082 ceph-mon[98694]: pgmap v11522: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 227 KiB/s wr, 16 op/s 2023-09-10T05:14:31.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:14:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:31 smithi098 ceph-mon[103124]: pgmap v11522: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 227 KiB/s wr, 16 op/s 2023-09-10T05:14:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:14:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:14:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:14:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:14:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:14:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:14:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:14:33.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:14:33.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:14:33.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:14:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:33 smithi170 ceph-mon[102832]: pgmap v11523: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T05:14:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21305.371576 secs 2023-09-10T05:14:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:33 smithi082 ceph-mon[98694]: pgmap v11523: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T05:14:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21305.371576 secs 2023-09-10T05:14:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:33 smithi098 ceph-mon[103124]: pgmap v11523: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T05:14:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21305.371576 secs 2023-09-10T05:14:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21306.025261 secs 2023-09-10T05:14:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21306.025261 secs 2023-09-10T05:14:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21306.025261 secs 2023-09-10T05:14:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:35 smithi170 ceph-mon[102832]: pgmap v11524: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T05:14:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:35 smithi082 ceph-mon[98694]: pgmap v11524: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T05:14:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:35 smithi098 ceph-mon[103124]: pgmap v11524: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T05:14:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:37 smithi082 ceph-mon[98694]: pgmap v11525: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 375 KiB/s wr, 18 op/s 2023-09-10T05:14:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:14:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:14:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:37 smithi098 ceph-mon[103124]: pgmap v11525: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 375 KiB/s wr, 18 op/s 2023-09-10T05:14:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:14:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:14:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:37 smithi170 ceph-mon[102832]: pgmap v11525: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 375 KiB/s wr, 18 op/s 2023-09-10T05:14:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:14:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:14:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21310.371635 secs 2023-09-10T05:14:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21310.371635 secs 2023-09-10T05:14:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21310.371635 secs 2023-09-10T05:14:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:39 smithi082 ceph-mon[98694]: pgmap v11526: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 307 KiB/s wr, 12 op/s 2023-09-10T05:14:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21311.025328 secs 2023-09-10T05:14:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:39 smithi098 ceph-mon[103124]: pgmap v11526: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 307 KiB/s wr, 12 op/s 2023-09-10T05:14:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21311.025328 secs 2023-09-10T05:14:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:39 smithi170 ceph-mon[102832]: pgmap v11526: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 307 KiB/s wr, 12 op/s 2023-09-10T05:14:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21311.025328 secs 2023-09-10T05:14:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:14:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:14:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:14:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:14:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:14:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:14:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:41 smithi082 ceph-mon[98694]: pgmap v11527: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 445 KiB/s wr, 19 op/s 2023-09-10T05:14:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:41 smithi098 ceph-mon[103124]: pgmap v11527: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 445 KiB/s wr, 19 op/s 2023-09-10T05:14:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:41 smithi170 ceph-mon[102832]: pgmap v11527: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 445 KiB/s wr, 19 op/s 2023-09-10T05:14:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:43 smithi082 ceph-mon[98694]: pgmap v11528: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 287 KiB/s wr, 13 op/s 2023-09-10T05:14:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21315.371705 secs 2023-09-10T05:14:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:43 smithi098 ceph-mon[103124]: pgmap v11528: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 287 KiB/s wr, 13 op/s 2023-09-10T05:14:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21315.371705 secs 2023-09-10T05:14:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:43 smithi170 ceph-mon[102832]: pgmap v11528: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 287 KiB/s wr, 13 op/s 2023-09-10T05:14:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21315.371705 secs 2023-09-10T05:14:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21316.025388 secs 2023-09-10T05:14:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21316.025388 secs 2023-09-10T05:14:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21316.025388 secs 2023-09-10T05:14:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:45 smithi082 ceph-mon[98694]: pgmap v11529: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 287 KiB/s wr, 13 op/s 2023-09-10T05:14:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:45 smithi098 ceph-mon[103124]: pgmap v11529: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 287 KiB/s wr, 13 op/s 2023-09-10T05:14:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:45 smithi170 ceph-mon[102832]: pgmap v11529: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 287 KiB/s wr, 13 op/s 2023-09-10T05:14:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:47 smithi082 ceph-mon[98694]: pgmap v11530: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 427 KiB/s wr, 19 op/s 2023-09-10T05:14:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:47 smithi098 ceph-mon[103124]: pgmap v11530: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 427 KiB/s wr, 19 op/s 2023-09-10T05:14:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:47 smithi170 ceph-mon[102832]: pgmap v11530: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 427 KiB/s wr, 19 op/s 2023-09-10T05:14:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21320.371761 secs 2023-09-10T05:14:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21320.371761 secs 2023-09-10T05:14:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21320.371761 secs 2023-09-10T05:14:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:49 smithi098 ceph-mon[103124]: pgmap v11531: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 13 op/s 2023-09-10T05:14:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21321.025456 secs 2023-09-10T05:14:50.183 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:49 smithi170 ceph-mon[102832]: pgmap v11531: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 13 op/s 2023-09-10T05:14:50.184 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21321.025456 secs 2023-09-10T05:14:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:49 smithi082 ceph-mon[98694]: pgmap v11531: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 13 op/s 2023-09-10T05:14:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21321.025456 secs 2023-09-10T05:14:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:51 smithi098 ceph-mon[103124]: pgmap v11532: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 417 KiB/s wr, 19 op/s 2023-09-10T05:14:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:51 smithi170 ceph-mon[102832]: pgmap v11532: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 417 KiB/s wr, 19 op/s 2023-09-10T05:14:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:51 smithi082 ceph-mon[98694]: pgmap v11532: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 417 KiB/s wr, 19 op/s 2023-09-10T05:14:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:53 smithi098 ceph-mon[103124]: pgmap v11533: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T05:14:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21324.331500 secs 2023-09-10T05:14:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:14:54.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:14:54.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21325.371824 secs 2023-09-10T05:14:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:53 smithi170 ceph-mon[102832]: pgmap v11533: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T05:14:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21324.331500 secs 2023-09-10T05:14:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:14:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:14:54.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21325.371824 secs 2023-09-10T05:14:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:53 smithi082 ceph-mon[98694]: pgmap v11533: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T05:14:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21324.331500 secs 2023-09-10T05:14:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:14:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:14:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21325.371824 secs 2023-09-10T05:14:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21326.025480 secs 2023-09-10T05:14:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21326.025480 secs 2023-09-10T05:14:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21326.025480 secs 2023-09-10T05:14:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:55 smithi098 ceph-mon[103124]: pgmap v11534: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T05:14:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:55 smithi170 ceph-mon[102832]: pgmap v11534: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T05:14:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:55 smithi082 ceph-mon[98694]: pgmap v11534: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T05:14:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:14:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:14:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:14:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:14:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:14:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:14:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:14:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:14:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:14:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:57 smithi098 ceph-mon[103124]: pgmap v11535: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 344 KiB/s wr, 18 op/s 2023-09-10T05:14:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:57 smithi170 ceph-mon[102832]: pgmap v11535: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 344 KiB/s wr, 18 op/s 2023-09-10T05:14:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:57 smithi082 ceph-mon[98694]: pgmap v11535: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 344 KiB/s wr, 18 op/s 2023-09-10T05:14:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21329.331558 secs 2023-09-10T05:14:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21330.371906 secs 2023-09-10T05:14:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21329.331558 secs 2023-09-10T05:14:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21330.371906 secs 2023-09-10T05:14:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21329.331558 secs 2023-09-10T05:14:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21330.371906 secs 2023-09-10T05:15:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:59 smithi098 ceph-mon[103124]: pgmap v11536: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 203 KiB/s wr, 12 op/s 2023-09-10T05:15:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:14:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21331.025545 secs 2023-09-10T05:15:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:59 smithi170 ceph-mon[102832]: pgmap v11536: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 203 KiB/s wr, 12 op/s 2023-09-10T05:15:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:14:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21331.025545 secs 2023-09-10T05:15:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:59 smithi082 ceph-mon[98694]: pgmap v11536: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 203 KiB/s wr, 12 op/s 2023-09-10T05:15:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:14:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21331.025545 secs 2023-09-10T05:15:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:01 smithi098 ceph-mon[103124]: pgmap v11537: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 306 KiB/s wr, 17 op/s 2023-09-10T05:15:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:01 smithi170 ceph-mon[102832]: pgmap v11537: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 306 KiB/s wr, 17 op/s 2023-09-10T05:15:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:01 smithi082 ceph-mon[98694]: pgmap v11537: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 306 KiB/s wr, 17 op/s 2023-09-10T05:15:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:03 smithi098 ceph-mon[103124]: pgmap v11538: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T05:15:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21334.331619 secs 2023-09-10T05:15:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21335.371965 secs 2023-09-10T05:15:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:03 smithi170 ceph-mon[102832]: pgmap v11538: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T05:15:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21334.331619 secs 2023-09-10T05:15:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21335.371965 secs 2023-09-10T05:15:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:03 smithi082 ceph-mon[98694]: pgmap v11538: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T05:15:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21334.331619 secs 2023-09-10T05:15:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21335.371965 secs 2023-09-10T05:15:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21336.025611 secs 2023-09-10T05:15:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21336.025611 secs 2023-09-10T05:15:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21336.025611 secs 2023-09-10T05:15:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:05 smithi170 ceph-mon[102832]: pgmap v11539: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T05:15:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:05 smithi082 ceph-mon[98694]: pgmap v11539: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T05:15:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:05 smithi098 ceph-mon[103124]: pgmap v11539: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T05:15:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:07 smithi170 ceph-mon[102832]: pgmap v11540: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 266 KiB/s wr, 30 op/s 2023-09-10T05:15:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:07 smithi082 ceph-mon[98694]: pgmap v11540: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 266 KiB/s wr, 30 op/s 2023-09-10T05:15:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:07 smithi098 ceph-mon[103124]: pgmap v11540: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 266 KiB/s wr, 30 op/s 2023-09-10T05:15:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21339.331680 secs 2023-09-10T05:15:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:08 smithi170 ceph-mon[102832]: osdmap e789: 12 total, 12 up, 12 in 2023-09-10T05:15:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21340.372024 secs 2023-09-10T05:15:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21339.331680 secs 2023-09-10T05:15:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:08 smithi082 ceph-mon[98694]: osdmap e789: 12 total, 12 up, 12 in 2023-09-10T05:15:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21340.372024 secs 2023-09-10T05:15:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21339.331680 secs 2023-09-10T05:15:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:08 smithi098 ceph-mon[103124]: osdmap e789: 12 total, 12 up, 12 in 2023-09-10T05:15:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21340.372024 secs 2023-09-10T05:15:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:09 smithi170 ceph-mon[102832]: pgmap v11542: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 241 KiB/s wr, 29 op/s 2023-09-10T05:15:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21341.025680 secs 2023-09-10T05:15:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:09 smithi082 ceph-mon[98694]: pgmap v11542: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 241 KiB/s wr, 29 op/s 2023-09-10T05:15:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21341.025680 secs 2023-09-10T05:15:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:09 smithi098 ceph-mon[103124]: pgmap v11542: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 241 KiB/s wr, 29 op/s 2023-09-10T05:15:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21341.025680 secs 2023-09-10T05:15:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:11 smithi170 ceph-mon[102832]: pgmap v11543: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 119 KiB/s wr, 28 op/s 2023-09-10T05:15:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:11 smithi082 ceph-mon[98694]: pgmap v11543: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 119 KiB/s wr, 28 op/s 2023-09-10T05:15:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:11 smithi098 ceph-mon[103124]: pgmap v11543: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 119 KiB/s wr, 28 op/s 2023-09-10T05:15:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:13 smithi170 ceph-mon[102832]: pgmap v11544: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 119 KiB/s wr, 28 op/s 2023-09-10T05:15:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21344.331767 secs 2023-09-10T05:15:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21345.372081 secs 2023-09-10T05:15:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:13 smithi082 ceph-mon[98694]: pgmap v11544: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 119 KiB/s wr, 28 op/s 2023-09-10T05:15:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21344.331767 secs 2023-09-10T05:15:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21345.372081 secs 2023-09-10T05:15:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:13 smithi098 ceph-mon[103124]: pgmap v11544: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 119 KiB/s wr, 28 op/s 2023-09-10T05:15:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21344.331767 secs 2023-09-10T05:15:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21345.372081 secs 2023-09-10T05:15:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21346.025739 secs 2023-09-10T05:15:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:14 smithi170 ceph-mon[102832]: osdmap e790: 12 total, 12 up, 12 in 2023-09-10T05:15:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21346.025739 secs 2023-09-10T05:15:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:14 smithi082 ceph-mon[98694]: osdmap e790: 12 total, 12 up, 12 in 2023-09-10T05:15:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21346.025739 secs 2023-09-10T05:15:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:14 smithi098 ceph-mon[103124]: osdmap e790: 12 total, 12 up, 12 in 2023-09-10T05:15:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:15 smithi170 ceph-mon[102832]: pgmap v11546: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-09-10T05:15:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:15 smithi082 ceph-mon[98694]: pgmap v11546: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-09-10T05:15:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:15 smithi098 ceph-mon[103124]: pgmap v11546: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-09-10T05:15:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:15:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:15:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:15:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:17 smithi170 ceph-mon[102832]: pgmap v11547: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-09-10T05:15:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:15:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:15:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:15:18.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:15:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:17 smithi082 ceph-mon[98694]: pgmap v11547: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-09-10T05:15:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:15:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:15:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:15:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:15:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:17 smithi098 ceph-mon[103124]: pgmap v11547: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-09-10T05:15:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:15:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:15:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:15:18.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:15:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21349.331825 secs 2023-09-10T05:15:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:15:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:15:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:15:19.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21350.372138 secs 2023-09-10T05:15:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21349.331825 secs 2023-09-10T05:15:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:15:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:15:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:15:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21350.372138 secs 2023-09-10T05:15:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21349.331825 secs 2023-09-10T05:15:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:15:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:15:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:15:19.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21350.372138 secs 2023-09-10T05:15:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:19 smithi170 ceph-mon[102832]: pgmap v11548: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-09-10T05:15:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21351.025806 secs 2023-09-10T05:15:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:19 smithi082 ceph-mon[98694]: pgmap v11548: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-09-10T05:15:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21351.025806 secs 2023-09-10T05:15:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:19 smithi098 ceph-mon[103124]: pgmap v11548: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-09-10T05:15:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21351.025806 secs 2023-09-10T05:15:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:21 smithi170 ceph-mon[102832]: pgmap v11549: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T05:15:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:21 smithi082 ceph-mon[98694]: pgmap v11549: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T05:15:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:21 smithi098 ceph-mon[103124]: pgmap v11549: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T05:15:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:23 smithi170 ceph-mon[102832]: pgmap v11550: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T05:15:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21354.331884 secs 2023-09-10T05:15:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21355.372196 secs 2023-09-10T05:15:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:23 smithi082 ceph-mon[98694]: pgmap v11550: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T05:15:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21354.331884 secs 2023-09-10T05:15:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21355.372196 secs 2023-09-10T05:15:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:23 smithi098 ceph-mon[103124]: pgmap v11550: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 11 op/s 2023-09-10T05:15:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21354.331884 secs 2023-09-10T05:15:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21355.372196 secs 2023-09-10T05:15:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21356.025868 secs 2023-09-10T05:15:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21356.025868 secs 2023-09-10T05:15:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21356.025868 secs 2023-09-10T05:15:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:25 smithi170 ceph-mon[102832]: pgmap v11551: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T05:15:26.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:25 smithi082 ceph-mon[98694]: pgmap v11551: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T05:15:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:25 smithi098 ceph-mon[103124]: pgmap v11551: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T05:15:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:27 smithi170 ceph-mon[102832]: pgmap v11552: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 14 op/s 2023-09-10T05:15:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:27 smithi082 ceph-mon[98694]: pgmap v11552: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 14 op/s 2023-09-10T05:15:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:27 smithi098 ceph-mon[103124]: pgmap v11552: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 14 op/s 2023-09-10T05:15:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21360.372257 secs 2023-09-10T05:15:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21360.372257 secs 2023-09-10T05:15:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21360.372257 secs 2023-09-10T05:15:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:29 smithi170 ceph-mon[102832]: pgmap v11553: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 9 op/s 2023-09-10T05:15:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21361.025931 secs 2023-09-10T05:15:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:29 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:15:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:29 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:15:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:29 smithi082 ceph-mon[98694]: pgmap v11553: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 9 op/s 2023-09-10T05:15:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21361.025931 secs 2023-09-10T05:15:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:29 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:15:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:29 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:15:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:29 smithi098 ceph-mon[103124]: pgmap v11553: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 9 op/s 2023-09-10T05:15:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21361.025931 secs 2023-09-10T05:15:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:29 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:15:30.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:29 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:15:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:31 smithi170 ceph-mon[102832]: pgmap v11554: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 245 KiB/s wr, 15 op/s 2023-09-10T05:15:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:15:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:31 smithi082 ceph-mon[98694]: pgmap v11554: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 245 KiB/s wr, 15 op/s 2023-09-10T05:15:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:15:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:31 smithi098 ceph-mon[103124]: pgmap v11554: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 245 KiB/s wr, 15 op/s 2023-09-10T05:15:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:15:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:15:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:15:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:15:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:15:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:15:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:15:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:15:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:15:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:15:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:33 smithi170 ceph-mon[102832]: pgmap v11555: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T05:15:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21365.372333 secs 2023-09-10T05:15:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:33 smithi082 ceph-mon[98694]: pgmap v11555: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T05:15:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21365.372333 secs 2023-09-10T05:15:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:33 smithi098 ceph-mon[103124]: pgmap v11555: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T05:15:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21365.372333 secs 2023-09-10T05:15:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21366.025992 secs 2023-09-10T05:15:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21366.025992 secs 2023-09-10T05:15:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21366.025992 secs 2023-09-10T05:15:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:35 smithi170 ceph-mon[102832]: pgmap v11556: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T05:15:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:35 smithi082 ceph-mon[98694]: pgmap v11556: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T05:15:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:35 smithi098 ceph-mon[103124]: pgmap v11556: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T05:15:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:37 smithi170 ceph-mon[102832]: pgmap v11557: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 334 KiB/s wr, 16 op/s 2023-09-10T05:15:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:37 smithi082 ceph-mon[98694]: pgmap v11557: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 334 KiB/s wr, 16 op/s 2023-09-10T05:15:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:37 smithi098 ceph-mon[103124]: pgmap v11557: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 334 KiB/s wr, 16 op/s 2023-09-10T05:15:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21370.372390 secs 2023-09-10T05:15:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21370.372390 secs 2023-09-10T05:15:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21370.372390 secs 2023-09-10T05:15:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:39 smithi170 ceph-mon[102832]: pgmap v11558: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 268 KiB/s wr, 11 op/s 2023-09-10T05:15:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21371.026060 secs 2023-09-10T05:15:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:39 smithi082 ceph-mon[98694]: pgmap v11558: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 268 KiB/s wr, 11 op/s 2023-09-10T05:15:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21371.026060 secs 2023-09-10T05:15:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:39 smithi098 ceph-mon[103124]: pgmap v11558: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 268 KiB/s wr, 11 op/s 2023-09-10T05:15:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21371.026060 secs 2023-09-10T05:15:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:15:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:15:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:15:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:15:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:15:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:15:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:41 smithi170 ceph-mon[102832]: pgmap v11559: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 403 KiB/s wr, 18 op/s 2023-09-10T05:15:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:41 smithi082 ceph-mon[98694]: pgmap v11559: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 403 KiB/s wr, 18 op/s 2023-09-10T05:15:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:41 smithi098 ceph-mon[103124]: pgmap v11559: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 403 KiB/s wr, 18 op/s 2023-09-10T05:15:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:43 smithi170 ceph-mon[102832]: pgmap v11560: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T05:15:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21375.372475 secs 2023-09-10T05:15:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:43 smithi082 ceph-mon[98694]: pgmap v11560: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T05:15:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21375.372475 secs 2023-09-10T05:15:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:43 smithi098 ceph-mon[103124]: pgmap v11560: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T05:15:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21375.372475 secs 2023-09-10T05:15:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21376.026119 secs 2023-09-10T05:15:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21376.026119 secs 2023-09-10T05:15:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21376.026119 secs 2023-09-10T05:15:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:45 smithi170 ceph-mon[102832]: pgmap v11561: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T05:15:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:15:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:15:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:45 smithi082 ceph-mon[98694]: pgmap v11561: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T05:15:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:15:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:15:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:45 smithi098 ceph-mon[103124]: pgmap v11561: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T05:15:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:15:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:15:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:47 smithi170 ceph-mon[102832]: pgmap v11562: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 524 KiB/s wr, 19 op/s 2023-09-10T05:15:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21379.332333 secs 2023-09-10T05:15:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:47 smithi082 ceph-mon[98694]: pgmap v11562: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 524 KiB/s wr, 19 op/s 2023-09-10T05:15:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21379.332333 secs 2023-09-10T05:15:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:47 smithi098 ceph-mon[103124]: pgmap v11562: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 524 KiB/s wr, 19 op/s 2023-09-10T05:15:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21379.332333 secs 2023-09-10T05:15:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21380.372530 secs 2023-09-10T05:15:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21380.372530 secs 2023-09-10T05:15:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21380.372530 secs 2023-09-10T05:15:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:49 smithi170 ceph-mon[102832]: pgmap v11563: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 364 KiB/s wr, 13 op/s 2023-09-10T05:15:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21381.026184 secs 2023-09-10T05:15:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:49 smithi082 ceph-mon[98694]: pgmap v11563: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 364 KiB/s wr, 13 op/s 2023-09-10T05:15:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21381.026184 secs 2023-09-10T05:15:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:49 smithi098 ceph-mon[103124]: pgmap v11563: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 364 KiB/s wr, 13 op/s 2023-09-10T05:15:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21381.026184 secs 2023-09-10T05:15:52.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:51 smithi170 ceph-mon[102832]: pgmap v11564: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 503 KiB/s wr, 18 op/s 2023-09-10T05:15:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:51 smithi082 ceph-mon[98694]: pgmap v11564: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 503 KiB/s wr, 18 op/s 2023-09-10T05:15:52.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:51 smithi098 ceph-mon[103124]: pgmap v11564: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 503 KiB/s wr, 18 op/s 2023-09-10T05:15:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:15:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:15:53.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:15:53.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21384.332454 secs 2023-09-10T05:15:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:15:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:15:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:15:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21384.332454 secs 2023-09-10T05:15:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:15:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:15:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:15:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21384.332454 secs 2023-09-10T05:15:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:53 smithi170 ceph-mon[102832]: pgmap v11565: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 368 KiB/s wr, 12 op/s 2023-09-10T05:15:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21385.372616 secs 2023-09-10T05:15:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:53 smithi082 ceph-mon[98694]: pgmap v11565: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 368 KiB/s wr, 12 op/s 2023-09-10T05:15:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21385.372616 secs 2023-09-10T05:15:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:53 smithi098 ceph-mon[103124]: pgmap v11565: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 368 KiB/s wr, 12 op/s 2023-09-10T05:15:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21385.372616 secs 2023-09-10T05:15:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21386.026244 secs 2023-09-10T05:15:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21386.026244 secs 2023-09-10T05:15:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21386.026244 secs 2023-09-10T05:15:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:55 smithi170 ceph-mon[102832]: pgmap v11566: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 368 KiB/s wr, 12 op/s 2023-09-10T05:15:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:55 smithi082 ceph-mon[98694]: pgmap v11566: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 368 KiB/s wr, 12 op/s 2023-09-10T05:15:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:55 smithi098 ceph-mon[103124]: pgmap v11566: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 368 KiB/s wr, 12 op/s 2023-09-10T05:15:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:57 smithi170 ceph-mon[102832]: pgmap v11567: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 453 KiB/s wr, 18 op/s 2023-09-10T05:15:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:15:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:15:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21389.332528 secs 2023-09-10T05:15:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:57 smithi082 ceph-mon[98694]: pgmap v11567: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 453 KiB/s wr, 18 op/s 2023-09-10T05:15:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:15:58.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:15:58.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21389.332528 secs 2023-09-10T05:15:58.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:57 smithi098 ceph-mon[103124]: pgmap v11567: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 453 KiB/s wr, 18 op/s 2023-09-10T05:15:58.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:15:58.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:15:58.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21389.332528 secs 2023-09-10T05:15:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21390.372673 secs 2023-09-10T05:15:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21390.372673 secs 2023-09-10T05:15:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21390.372673 secs 2023-09-10T05:16:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:59 smithi170 ceph-mon[102832]: pgmap v11568: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T05:16:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21391.026311 secs 2023-09-10T05:16:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:59 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:16:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:15:59 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:16:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:59 smithi082 ceph-mon[98694]: pgmap v11568: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T05:16:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21391.026311 secs 2023-09-10T05:16:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:59 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:16:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:15:59 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:16:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:59 smithi098 ceph-mon[103124]: pgmap v11568: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T05:16:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21391.026311 secs 2023-09-10T05:16:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:59 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:16:00.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:15:59 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:16:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:01 smithi170 ceph-mon[102832]: pgmap v11569: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 307 KiB/s wr, 16 op/s 2023-09-10T05:16:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:01 smithi082 ceph-mon[98694]: pgmap v11569: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 307 KiB/s wr, 16 op/s 2023-09-10T05:16:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:01 smithi098 ceph-mon[103124]: pgmap v11569: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 307 KiB/s wr, 16 op/s 2023-09-10T05:16:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21394.332591 secs 2023-09-10T05:16:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21394.332591 secs 2023-09-10T05:16:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21394.332591 secs 2023-09-10T05:16:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:03 smithi170 ceph-mon[102832]: pgmap v11570: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 169 KiB/s wr, 11 op/s 2023-09-10T05:16:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21395.372730 secs 2023-09-10T05:16:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:03 smithi082 ceph-mon[98694]: pgmap v11570: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 169 KiB/s wr, 11 op/s 2023-09-10T05:16:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21395.372730 secs 2023-09-10T05:16:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:03 smithi098 ceph-mon[103124]: pgmap v11570: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 169 KiB/s wr, 11 op/s 2023-09-10T05:16:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21395.372730 secs 2023-09-10T05:16:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21396.026374 secs 2023-09-10T05:16:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21396.026374 secs 2023-09-10T05:16:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21396.026374 secs 2023-09-10T05:16:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:05 smithi170 ceph-mon[102832]: pgmap v11571: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 169 KiB/s wr, 11 op/s 2023-09-10T05:16:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:05 smithi082 ceph-mon[98694]: pgmap v11571: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 169 KiB/s wr, 11 op/s 2023-09-10T05:16:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:05 smithi098 ceph-mon[103124]: pgmap v11571: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 169 KiB/s wr, 11 op/s 2023-09-10T05:16:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:07 smithi170 ceph-mon[102832]: pgmap v11572: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 273 KiB/s wr, 31 op/s 2023-09-10T05:16:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21399.332700 secs 2023-09-10T05:16:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:07 smithi082 ceph-mon[98694]: pgmap v11572: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 273 KiB/s wr, 31 op/s 2023-09-10T05:16:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21399.332700 secs 2023-09-10T05:16:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:07 smithi098 ceph-mon[103124]: pgmap v11572: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 273 KiB/s wr, 31 op/s 2023-09-10T05:16:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21399.332700 secs 2023-09-10T05:16:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:08 smithi082 ceph-mon[98694]: osdmap e791: 12 total, 12 up, 12 in 2023-09-10T05:16:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21400.372791 secs 2023-09-10T05:16:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:08 smithi098 ceph-mon[103124]: osdmap e791: 12 total, 12 up, 12 in 2023-09-10T05:16:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21400.372791 secs 2023-09-10T05:16:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:08 smithi170 ceph-mon[102832]: osdmap e791: 12 total, 12 up, 12 in 2023-09-10T05:16:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21400.372791 secs 2023-09-10T05:16:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:09 smithi082 ceph-mon[98694]: pgmap v11574: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 226 KiB/s wr, 30 op/s 2023-09-10T05:16:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21401.026439 secs 2023-09-10T05:16:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:09 smithi098 ceph-mon[103124]: pgmap v11574: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 226 KiB/s wr, 30 op/s 2023-09-10T05:16:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21401.026439 secs 2023-09-10T05:16:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:09 smithi170 ceph-mon[102832]: pgmap v11574: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 226 KiB/s wr, 30 op/s 2023-09-10T05:16:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21401.026439 secs 2023-09-10T05:16:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:11 smithi082 ceph-mon[98694]: pgmap v11575: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 126 KiB/s wr, 29 op/s 2023-09-10T05:16:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:11 smithi098 ceph-mon[103124]: pgmap v11575: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 126 KiB/s wr, 29 op/s 2023-09-10T05:16:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:11 smithi170 ceph-mon[102832]: pgmap v11575: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 126 KiB/s wr, 29 op/s 2023-09-10T05:16:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21404.332767 secs 2023-09-10T05:16:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21404.332767 secs 2023-09-10T05:16:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21404.332767 secs 2023-09-10T05:16:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:13 smithi082 ceph-mon[98694]: pgmap v11576: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 126 KiB/s wr, 29 op/s 2023-09-10T05:16:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21405.372880 secs 2023-09-10T05:16:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:13 smithi098 ceph-mon[103124]: pgmap v11576: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 126 KiB/s wr, 29 op/s 2023-09-10T05:16:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21405.372880 secs 2023-09-10T05:16:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:13 smithi170 ceph-mon[102832]: pgmap v11576: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 126 KiB/s wr, 29 op/s 2023-09-10T05:16:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21405.372880 secs 2023-09-10T05:16:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21406.026482 secs 2023-09-10T05:16:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:14 smithi082 ceph-mon[98694]: osdmap e792: 12 total, 12 up, 12 in 2023-09-10T05:16:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21406.026482 secs 2023-09-10T05:16:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:14 smithi098 ceph-mon[103124]: osdmap e792: 12 total, 12 up, 12 in 2023-09-10T05:16:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21406.026482 secs 2023-09-10T05:16:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:14 smithi170 ceph-mon[102832]: osdmap e792: 12 total, 12 up, 12 in 2023-09-10T05:16:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:15 smithi082 ceph-mon[98694]: pgmap v11578: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-09-10T05:16:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:15 smithi098 ceph-mon[103124]: pgmap v11578: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-09-10T05:16:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:15 smithi170 ceph-mon[102832]: pgmap v11578: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-09-10T05:16:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:17 smithi082 ceph-mon[98694]: pgmap v11579: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2023-09-10T05:16:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21409.332826 secs 2023-09-10T05:16:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:16:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:17 smithi098 ceph-mon[103124]: pgmap v11579: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2023-09-10T05:16:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21409.332826 secs 2023-09-10T05:16:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:16:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:17 smithi170 ceph-mon[102832]: pgmap v11579: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2023-09-10T05:16:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21409.332826 secs 2023-09-10T05:16:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:16:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21410.372943 secs 2023-09-10T05:16:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:16:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:16:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:16:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21410.372943 secs 2023-09-10T05:16:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:16:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:16:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:16:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21410.372943 secs 2023-09-10T05:16:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:16:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:16:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:16:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:19 smithi098 ceph-mon[103124]: pgmap v11580: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-09-10T05:16:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21411.026553 secs 2023-09-10T05:16:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:19 smithi170 ceph-mon[102832]: pgmap v11580: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-09-10T05:16:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21411.026553 secs 2023-09-10T05:16:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:19 smithi082 ceph-mon[98694]: pgmap v11580: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-09-10T05:16:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21411.026553 secs 2023-09-10T05:16:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:21 smithi098 ceph-mon[103124]: pgmap v11581: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T05:16:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:21 smithi170 ceph-mon[102832]: pgmap v11581: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T05:16:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:21 smithi082 ceph-mon[98694]: pgmap v11581: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T05:16:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21414.332887 secs 2023-09-10T05:16:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21414.332887 secs 2023-09-10T05:16:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21414.332887 secs 2023-09-10T05:16:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:23 smithi098 ceph-mon[103124]: pgmap v11582: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T05:16:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21415.373002 secs 2023-09-10T05:16:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:23 smithi170 ceph-mon[102832]: pgmap v11582: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T05:16:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21415.373002 secs 2023-09-10T05:16:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:23 smithi082 ceph-mon[98694]: pgmap v11582: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T05:16:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21415.373002 secs 2023-09-10T05:16:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21416.026614 secs 2023-09-10T05:16:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21416.026614 secs 2023-09-10T05:16:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21416.026614 secs 2023-09-10T05:16:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:25 smithi098 ceph-mon[103124]: pgmap v11583: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T05:16:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:25 smithi170 ceph-mon[102832]: pgmap v11583: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T05:16:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:25 smithi082 ceph-mon[98694]: pgmap v11583: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T05:16:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:27 smithi098 ceph-mon[103124]: pgmap v11584: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 71 KiB/s wr, 14 op/s 2023-09-10T05:16:28.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21419.332967 secs 2023-09-10T05:16:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:27 smithi170 ceph-mon[102832]: pgmap v11584: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 71 KiB/s wr, 14 op/s 2023-09-10T05:16:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21419.332967 secs 2023-09-10T05:16:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:27 smithi082 ceph-mon[98694]: pgmap v11584: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 71 KiB/s wr, 14 op/s 2023-09-10T05:16:28.607 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21419.332967 secs 2023-09-10T05:16:29.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21420.373086 secs 2023-09-10T05:16:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21420.373086 secs 2023-09-10T05:16:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21420.373086 secs 2023-09-10T05:16:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:29 smithi098 ceph-mon[103124]: pgmap v11585: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T05:16:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21421.026679 secs 2023-09-10T05:16:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:29 smithi170 ceph-mon[102832]: pgmap v11585: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T05:16:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21421.026679 secs 2023-09-10T05:16:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:29 smithi082 ceph-mon[98694]: pgmap v11585: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T05:16:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21421.026679 secs 2023-09-10T05:16:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:31 smithi098 ceph-mon[103124]: pgmap v11586: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 232 KiB/s wr, 15 op/s 2023-09-10T05:16:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:16:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:16:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:31 smithi170 ceph-mon[102832]: pgmap v11586: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 232 KiB/s wr, 15 op/s 2023-09-10T05:16:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:16:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:16:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:31 smithi082 ceph-mon[98694]: pgmap v11586: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 232 KiB/s wr, 15 op/s 2023-09-10T05:16:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:16:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:16:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21424.333080 secs 2023-09-10T05:16:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21424.333080 secs 2023-09-10T05:16:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21424.333080 secs 2023-09-10T05:16:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:33 smithi098 ceph-mon[103124]: pgmap v11587: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 161 KiB/s wr, 10 op/s 2023-09-10T05:16:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:16:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:16:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21425.373171 secs 2023-09-10T05:16:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21426.026739 secs 2023-09-10T05:16:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:33 smithi170 ceph-mon[102832]: pgmap v11587: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 161 KiB/s wr, 10 op/s 2023-09-10T05:16:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:16:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:16:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21425.373171 secs 2023-09-10T05:16:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21426.026739 secs 2023-09-10T05:16:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:33 smithi082 ceph-mon[98694]: pgmap v11587: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 161 KiB/s wr, 10 op/s 2023-09-10T05:16:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:16:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:16:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21425.373171 secs 2023-09-10T05:16:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21426.026739 secs 2023-09-10T05:16:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:35 smithi170 ceph-mon[102832]: pgmap v11588: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 161 KiB/s wr, 10 op/s 2023-09-10T05:16:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:35 smithi082 ceph-mon[98694]: pgmap v11588: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 161 KiB/s wr, 10 op/s 2023-09-10T05:16:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:35 smithi098 ceph-mon[103124]: pgmap v11588: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 161 KiB/s wr, 10 op/s 2023-09-10T05:16:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:37 smithi082 ceph-mon[98694]: pgmap v11589: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 311 KiB/s wr, 16 op/s 2023-09-10T05:16:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:37 smithi098 ceph-mon[103124]: pgmap v11589: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 311 KiB/s wr, 16 op/s 2023-09-10T05:16:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:37 smithi170 ceph-mon[102832]: pgmap v11589: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 311 KiB/s wr, 16 op/s 2023-09-10T05:16:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21430.373260 secs 2023-09-10T05:16:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21430.373260 secs 2023-09-10T05:16:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21430.373260 secs 2023-09-10T05:16:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:39 smithi082 ceph-mon[98694]: pgmap v11590: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 311 KiB/s wr, 12 op/s 2023-09-10T05:16:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21431.026807 secs 2023-09-10T05:16:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:16:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:39 smithi098 ceph-mon[103124]: pgmap v11590: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 311 KiB/s wr, 12 op/s 2023-09-10T05:16:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21431.026807 secs 2023-09-10T05:16:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:16:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:39 smithi170 ceph-mon[102832]: pgmap v11590: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 311 KiB/s wr, 12 op/s 2023-09-10T05:16:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21431.026807 secs 2023-09-10T05:16:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:16:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:16:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:16:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:16:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:16:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:16:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:16:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:16:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:16:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:16:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:41 smithi098 ceph-mon[103124]: pgmap v11591: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 466 KiB/s wr, 18 op/s 2023-09-10T05:16:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:41 smithi170 ceph-mon[102832]: pgmap v11591: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 466 KiB/s wr, 18 op/s 2023-09-10T05:16:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:41 smithi082 ceph-mon[98694]: pgmap v11591: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 466 KiB/s wr, 18 op/s 2023-09-10T05:16:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:43 smithi098 ceph-mon[103124]: pgmap v11592: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 306 KiB/s wr, 13 op/s 2023-09-10T05:16:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21435.373318 secs 2023-09-10T05:16:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:43 smithi170 ceph-mon[102832]: pgmap v11592: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 306 KiB/s wr, 13 op/s 2023-09-10T05:16:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21435.373318 secs 2023-09-10T05:16:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:43 smithi082 ceph-mon[98694]: pgmap v11592: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 306 KiB/s wr, 13 op/s 2023-09-10T05:16:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21435.373318 secs 2023-09-10T05:16:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21436.026868 secs 2023-09-10T05:16:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21436.026868 secs 2023-09-10T05:16:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21436.026868 secs 2023-09-10T05:16:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:45 smithi098 ceph-mon[103124]: pgmap v11593: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 306 KiB/s wr, 13 op/s 2023-09-10T05:16:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:45 smithi170 ceph-mon[102832]: pgmap v11593: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 306 KiB/s wr, 13 op/s 2023-09-10T05:16:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:45 smithi082 ceph-mon[98694]: pgmap v11593: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 306 KiB/s wr, 13 op/s 2023-09-10T05:16:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:47 smithi098 ceph-mon[103124]: pgmap v11594: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 416 KiB/s wr, 19 op/s 2023-09-10T05:16:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:47 smithi170 ceph-mon[102832]: pgmap v11594: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 416 KiB/s wr, 19 op/s 2023-09-10T05:16:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:47 smithi082 ceph-mon[98694]: pgmap v11594: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 416 KiB/s wr, 19 op/s 2023-09-10T05:16:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21440.373389 secs 2023-09-10T05:16:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21440.373389 secs 2023-09-10T05:16:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21440.373389 secs 2023-09-10T05:16:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:49 smithi098 ceph-mon[103124]: pgmap v11595: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 266 KiB/s wr, 13 op/s 2023-09-10T05:16:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:16:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:16:50.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21441.026931 secs 2023-09-10T05:16:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:49 smithi170 ceph-mon[102832]: pgmap v11595: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 266 KiB/s wr, 13 op/s 2023-09-10T05:16:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:16:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:16:50.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21441.026931 secs 2023-09-10T05:16:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:49 smithi082 ceph-mon[98694]: pgmap v11595: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 266 KiB/s wr, 13 op/s 2023-09-10T05:16:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:16:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:16:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21441.026931 secs 2023-09-10T05:16:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:51 smithi098 ceph-mon[103124]: pgmap v11596: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 417 KiB/s wr, 19 op/s 2023-09-10T05:16:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:51 smithi170 ceph-mon[102832]: pgmap v11596: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 417 KiB/s wr, 19 op/s 2023-09-10T05:16:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:51 smithi082 ceph-mon[98694]: pgmap v11596: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 417 KiB/s wr, 19 op/s 2023-09-10T05:16:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:16:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:16:53.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21444.333344 secs 2023-09-10T05:16:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:16:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:16:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21444.333344 secs 2023-09-10T05:16:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:16:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:16:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21444.333344 secs 2023-09-10T05:16:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:53 smithi098 ceph-mon[103124]: pgmap v11597: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T05:16:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21445.373447 secs 2023-09-10T05:16:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:53 smithi170 ceph-mon[102832]: pgmap v11597: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T05:16:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21445.373447 secs 2023-09-10T05:16:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:53 smithi082 ceph-mon[98694]: pgmap v11597: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T05:16:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21445.373447 secs 2023-09-10T05:16:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21446.026999 secs 2023-09-10T05:16:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21446.026999 secs 2023-09-10T05:16:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21446.026999 secs 2023-09-10T05:16:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:55 smithi170 ceph-mon[102832]: pgmap v11598: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T05:16:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:55 smithi082 ceph-mon[98694]: pgmap v11598: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T05:16:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:55 smithi098 ceph-mon[103124]: pgmap v11598: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 262 KiB/s wr, 12 op/s 2023-09-10T05:16:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:16:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:16:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:16:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:16:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:16:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:16:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:16:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:16:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:16:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:57 smithi170 ceph-mon[102832]: pgmap v11599: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 405 KiB/s wr, 18 op/s 2023-09-10T05:16:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:16:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:16:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21449.333404 secs 2023-09-10T05:16:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:57 smithi082 ceph-mon[98694]: pgmap v11599: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 405 KiB/s wr, 18 op/s 2023-09-10T05:16:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:16:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:16:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21449.333404 secs 2023-09-10T05:16:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:57 smithi098 ceph-mon[103124]: pgmap v11599: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 405 KiB/s wr, 18 op/s 2023-09-10T05:16:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:16:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:16:58.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21449.333404 secs 2023-09-10T05:16:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21450.373493 secs 2023-09-10T05:16:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21450.373493 secs 2023-09-10T05:16:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21450.373493 secs 2023-09-10T05:17:00.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:59 smithi170 ceph-mon[102832]: pgmap v11600: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T05:17:00.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:16:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21451.027067 secs 2023-09-10T05:17:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:59 smithi082 ceph-mon[98694]: pgmap v11600: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T05:17:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:16:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21451.027067 secs 2023-09-10T05:17:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:59 smithi098 ceph-mon[103124]: pgmap v11600: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T05:17:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:16:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21451.027067 secs 2023-09-10T05:17:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:17:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:17:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:17:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:17:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:17:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:17:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:01 smithi170 ceph-mon[102832]: pgmap v11601: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 429 KiB/s wr, 18 op/s 2023-09-10T05:17:02.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:01 smithi082 ceph-mon[98694]: pgmap v11601: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 429 KiB/s wr, 18 op/s 2023-09-10T05:17:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:01 smithi098 ceph-mon[103124]: pgmap v11601: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 429 KiB/s wr, 18 op/s 2023-09-10T05:17:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21454.333479 secs 2023-09-10T05:17:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21454.333479 secs 2023-09-10T05:17:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21454.333479 secs 2023-09-10T05:17:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:03 smithi170 ceph-mon[102832]: pgmap v11602: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T05:17:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21455.373578 secs 2023-09-10T05:17:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:03 smithi082 ceph-mon[98694]: pgmap v11602: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T05:17:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21455.373578 secs 2023-09-10T05:17:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:03 smithi098 ceph-mon[103124]: pgmap v11602: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T05:17:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21455.373578 secs 2023-09-10T05:17:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21456.027133 secs 2023-09-10T05:17:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21456.027133 secs 2023-09-10T05:17:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21456.027133 secs 2023-09-10T05:17:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:05 smithi170 ceph-mon[102832]: pgmap v11603: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T05:17:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:05 smithi082 ceph-mon[98694]: pgmap v11603: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T05:17:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:05 smithi098 ceph-mon[103124]: pgmap v11603: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T05:17:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:07 smithi170 ceph-mon[102832]: pgmap v11604: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 412 KiB/s wr, 32 op/s 2023-09-10T05:17:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21459.333595 secs 2023-09-10T05:17:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:07 smithi082 ceph-mon[98694]: pgmap v11604: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 412 KiB/s wr, 32 op/s 2023-09-10T05:17:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21459.333595 secs 2023-09-10T05:17:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:07 smithi098 ceph-mon[103124]: pgmap v11604: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 412 KiB/s wr, 32 op/s 2023-09-10T05:17:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21459.333595 secs 2023-09-10T05:17:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:08 smithi170 ceph-mon[102832]: osdmap e793: 12 total, 12 up, 12 in 2023-09-10T05:17:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21460.373636 secs 2023-09-10T05:17:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:08 smithi082 ceph-mon[98694]: osdmap e793: 12 total, 12 up, 12 in 2023-09-10T05:17:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21460.373636 secs 2023-09-10T05:17:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:08 smithi098 ceph-mon[103124]: osdmap e793: 12 total, 12 up, 12 in 2023-09-10T05:17:09.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21460.373636 secs 2023-09-10T05:17:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:09 smithi170 ceph-mon[102832]: pgmap v11606: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 323 KiB/s wr, 31 op/s 2023-09-10T05:17:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21461.027197 secs 2023-09-10T05:17:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:09 smithi082 ceph-mon[98694]: pgmap v11606: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 323 KiB/s wr, 31 op/s 2023-09-10T05:17:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21461.027197 secs 2023-09-10T05:17:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:09 smithi098 ceph-mon[103124]: pgmap v11606: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 323 KiB/s wr, 31 op/s 2023-09-10T05:17:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21461.027197 secs 2023-09-10T05:17:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:11 smithi170 ceph-mon[102832]: pgmap v11607: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 163 KiB/s wr, 29 op/s 2023-09-10T05:17:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:11 smithi082 ceph-mon[98694]: pgmap v11607: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 163 KiB/s wr, 29 op/s 2023-09-10T05:17:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:11 smithi098 ceph-mon[103124]: pgmap v11607: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 163 KiB/s wr, 29 op/s 2023-09-10T05:17:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21464.333689 secs 2023-09-10T05:17:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21464.333689 secs 2023-09-10T05:17:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21464.333689 secs 2023-09-10T05:17:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:13 smithi170 ceph-mon[102832]: pgmap v11608: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 163 KiB/s wr, 29 op/s 2023-09-10T05:17:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21465.373694 secs 2023-09-10T05:17:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:13 smithi082 ceph-mon[98694]: pgmap v11608: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 163 KiB/s wr, 29 op/s 2023-09-10T05:17:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21465.373694 secs 2023-09-10T05:17:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:13 smithi098 ceph-mon[103124]: pgmap v11608: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 163 KiB/s wr, 29 op/s 2023-09-10T05:17:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21465.373694 secs 2023-09-10T05:17:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21466.027262 secs 2023-09-10T05:17:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:14 smithi170 ceph-mon[102832]: osdmap e794: 12 total, 12 up, 12 in 2023-09-10T05:17:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21466.027262 secs 2023-09-10T05:17:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:14 smithi082 ceph-mon[98694]: osdmap e794: 12 total, 12 up, 12 in 2023-09-10T05:17:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21466.027262 secs 2023-09-10T05:17:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:14 smithi098 ceph-mon[103124]: osdmap e794: 12 total, 12 up, 12 in 2023-09-10T05:17:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:15 smithi170 ceph-mon[102832]: pgmap v11610: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2023-09-10T05:17:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:15 smithi082 ceph-mon[98694]: pgmap v11610: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2023-09-10T05:17:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:15 smithi098 ceph-mon[103124]: pgmap v11610: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2023-09-10T05:17:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:17 smithi170 ceph-mon[102832]: pgmap v11611: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 12 op/s 2023-09-10T05:17:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21469.333750 secs 2023-09-10T05:17:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:17 smithi082 ceph-mon[98694]: pgmap v11611: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 12 op/s 2023-09-10T05:17:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21469.333750 secs 2023-09-10T05:17:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:17 smithi098 ceph-mon[103124]: pgmap v11611: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 12 op/s 2023-09-10T05:17:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21469.333750 secs 2023-09-10T05:17:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21470.373754 secs 2023-09-10T05:17:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:17:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21470.373754 secs 2023-09-10T05:17:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:17:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21470.373754 secs 2023-09-10T05:17:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:17:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:19 smithi170 ceph-mon[102832]: pgmap v11612: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T05:17:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21471.027331 secs 2023-09-10T05:17:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:17:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:17:20.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:17:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:19 smithi082 ceph-mon[98694]: pgmap v11612: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T05:17:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21471.027331 secs 2023-09-10T05:17:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:17:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:17:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:17:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:19 smithi098 ceph-mon[103124]: pgmap v11612: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T05:17:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21471.027331 secs 2023-09-10T05:17:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:17:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:17:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:17:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:21 smithi170 ceph-mon[102832]: pgmap v11613: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T05:17:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:21 smithi082 ceph-mon[98694]: pgmap v11613: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T05:17:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:21 smithi098 ceph-mon[103124]: pgmap v11613: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T05:17:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21474.333845 secs 2023-09-10T05:17:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21474.333845 secs 2023-09-10T05:17:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21474.333845 secs 2023-09-10T05:17:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:23 smithi170 ceph-mon[102832]: pgmap v11614: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T05:17:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21475.373837 secs 2023-09-10T05:17:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:23 smithi082 ceph-mon[98694]: pgmap v11614: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T05:17:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21475.373837 secs 2023-09-10T05:17:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:23 smithi098 ceph-mon[103124]: pgmap v11614: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T05:17:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21475.373837 secs 2023-09-10T05:17:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21476.027394 secs 2023-09-10T05:17:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21476.027394 secs 2023-09-10T05:17:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21476.027394 secs 2023-09-10T05:17:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:25 smithi170 ceph-mon[102832]: pgmap v11615: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T05:17:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:25 smithi082 ceph-mon[98694]: pgmap v11615: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T05:17:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:25 smithi098 ceph-mon[103124]: pgmap v11615: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T05:17:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:27 smithi170 ceph-mon[102832]: pgmap v11616: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 176 KiB/s wr, 14 op/s 2023-09-10T05:17:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21479.334002 secs 2023-09-10T05:17:28.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:27 smithi082 ceph-mon[98694]: pgmap v11616: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 176 KiB/s wr, 14 op/s 2023-09-10T05:17:28.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21479.334002 secs 2023-09-10T05:17:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:27 smithi098 ceph-mon[103124]: pgmap v11616: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 176 KiB/s wr, 14 op/s 2023-09-10T05:17:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21479.334002 secs 2023-09-10T05:17:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21480.373919 secs 2023-09-10T05:17:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21480.373919 secs 2023-09-10T05:17:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21480.373919 secs 2023-09-10T05:17:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:29 smithi170 ceph-mon[102832]: pgmap v11617: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 72 KiB/s wr, 10 op/s 2023-09-10T05:17:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21481.027520 secs 2023-09-10T05:17:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:29 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:17:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:29 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:17:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:29 smithi082 ceph-mon[98694]: pgmap v11617: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 72 KiB/s wr, 10 op/s 2023-09-10T05:17:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21481.027520 secs 2023-09-10T05:17:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:29 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:17:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:29 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:17:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:29 smithi098 ceph-mon[103124]: pgmap v11617: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 72 KiB/s wr, 10 op/s 2023-09-10T05:17:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21481.027520 secs 2023-09-10T05:17:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:29 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:17:30.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:29 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:17:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:31 smithi170 ceph-mon[102832]: pgmap v11618: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 231 KiB/s wr, 15 op/s 2023-09-10T05:17:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:31 smithi082 ceph-mon[98694]: pgmap v11618: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 231 KiB/s wr, 15 op/s 2023-09-10T05:17:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:31 smithi098 ceph-mon[103124]: pgmap v11618: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 231 KiB/s wr, 15 op/s 2023-09-10T05:17:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:17:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:17:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21484.334096 secs 2023-09-10T05:17:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:17:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:17:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21484.334096 secs 2023-09-10T05:17:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:17:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:17:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21484.334096 secs 2023-09-10T05:17:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:33 smithi170 ceph-mon[102832]: pgmap v11619: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T05:17:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21485.373976 secs 2023-09-10T05:17:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:33 smithi082 ceph-mon[98694]: pgmap v11619: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T05:17:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21485.373976 secs 2023-09-10T05:17:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:33 smithi098 ceph-mon[103124]: pgmap v11619: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T05:17:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21485.373976 secs 2023-09-10T05:17:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21486.027578 secs 2023-09-10T05:17:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21486.027578 secs 2023-09-10T05:17:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21486.027578 secs 2023-09-10T05:17:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:35 smithi170 ceph-mon[102832]: pgmap v11620: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T05:17:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:35 smithi082 ceph-mon[98694]: pgmap v11620: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T05:17:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:35 smithi098 ceph-mon[103124]: pgmap v11620: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T05:17:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:37 smithi170 ceph-mon[102832]: pgmap v11621: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 394 KiB/s wr, 18 op/s 2023-09-10T05:17:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:17:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:17:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:37 smithi082 ceph-mon[98694]: pgmap v11621: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 394 KiB/s wr, 18 op/s 2023-09-10T05:17:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:17:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:17:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:37 smithi098 ceph-mon[103124]: pgmap v11621: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 394 KiB/s wr, 18 op/s 2023-09-10T05:17:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:17:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:17:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21490.374051 secs 2023-09-10T05:17:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21490.374051 secs 2023-09-10T05:17:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21490.374051 secs 2023-09-10T05:17:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:39 smithi170 ceph-mon[102832]: pgmap v11622: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 323 KiB/s wr, 12 op/s 2023-09-10T05:17:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21491.027645 secs 2023-09-10T05:17:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:17:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:39 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:17:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:39 smithi082 ceph-mon[98694]: pgmap v11622: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 323 KiB/s wr, 12 op/s 2023-09-10T05:17:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21491.027645 secs 2023-09-10T05:17:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:17:40.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:39 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:17:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:39 smithi098 ceph-mon[103124]: pgmap v11622: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 323 KiB/s wr, 12 op/s 2023-09-10T05:17:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21491.027645 secs 2023-09-10T05:17:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:17:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:39 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:17:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:17:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:17:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:17:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:17:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:17:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:17:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:41 smithi170 ceph-mon[102832]: pgmap v11623: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 435 KiB/s wr, 19 op/s 2023-09-10T05:17:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:41 smithi082 ceph-mon[98694]: pgmap v11623: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 435 KiB/s wr, 19 op/s 2023-09-10T05:17:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:41 smithi098 ceph-mon[103124]: pgmap v11623: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 435 KiB/s wr, 19 op/s 2023-09-10T05:17:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:43 smithi170 ceph-mon[102832]: pgmap v11624: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 276 KiB/s wr, 13 op/s 2023-09-10T05:17:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21495.374112 secs 2023-09-10T05:17:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:43 smithi082 ceph-mon[98694]: pgmap v11624: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 276 KiB/s wr, 13 op/s 2023-09-10T05:17:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21495.374112 secs 2023-09-10T05:17:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:43 smithi098 ceph-mon[103124]: pgmap v11624: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 276 KiB/s wr, 13 op/s 2023-09-10T05:17:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21495.374112 secs 2023-09-10T05:17:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21496.027712 secs 2023-09-10T05:17:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21496.027712 secs 2023-09-10T05:17:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21496.027712 secs 2023-09-10T05:17:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:45 smithi170 ceph-mon[102832]: pgmap v11625: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 276 KiB/s wr, 13 op/s 2023-09-10T05:17:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:45 smithi082 ceph-mon[98694]: pgmap v11625: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 276 KiB/s wr, 13 op/s 2023-09-10T05:17:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:45 smithi098 ceph-mon[103124]: pgmap v11625: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 276 KiB/s wr, 13 op/s 2023-09-10T05:17:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:47 smithi170 ceph-mon[102832]: pgmap v11626: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 507 KiB/s wr, 19 op/s 2023-09-10T05:17:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21499.334281 secs 2023-09-10T05:17:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:47 smithi082 ceph-mon[98694]: pgmap v11626: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 507 KiB/s wr, 19 op/s 2023-09-10T05:17:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21499.334281 secs 2023-09-10T05:17:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:47 smithi098 ceph-mon[103124]: pgmap v11626: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 507 KiB/s wr, 19 op/s 2023-09-10T05:17:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21499.334281 secs 2023-09-10T05:17:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21500.374171 secs 2023-09-10T05:17:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21500.374171 secs 2023-09-10T05:17:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21500.374171 secs 2023-09-10T05:17:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:49 smithi170 ceph-mon[102832]: pgmap v11627: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 343 KiB/s wr, 12 op/s 2023-09-10T05:17:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21501.027828 secs 2023-09-10T05:17:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:17:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:17:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:49 smithi082 ceph-mon[98694]: pgmap v11627: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 343 KiB/s wr, 12 op/s 2023-09-10T05:17:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21501.027828 secs 2023-09-10T05:17:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:17:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:17:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:49 smithi098 ceph-mon[103124]: pgmap v11627: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 343 KiB/s wr, 12 op/s 2023-09-10T05:17:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21501.027828 secs 2023-09-10T05:17:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:17:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:17:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:50 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:17:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:17:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:17:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:50 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:17:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:17:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:17:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:50 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:17:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:17:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:17:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:51 smithi098 ceph-mon[103124]: pgmap v11628: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 485 KiB/s wr, 19 op/s 2023-09-10T05:17:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:51 smithi170 ceph-mon[102832]: pgmap v11628: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 485 KiB/s wr, 19 op/s 2023-09-10T05:17:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:51 smithi082 ceph-mon[98694]: pgmap v11628: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 485 KiB/s wr, 19 op/s 2023-09-10T05:17:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21504.334391 secs 2023-09-10T05:17:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21504.334391 secs 2023-09-10T05:17:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21504.334391 secs 2023-09-10T05:17:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:53 smithi098 ceph-mon[103124]: pgmap v11629: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 373 KiB/s wr, 12 op/s 2023-09-10T05:17:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21505.374230 secs 2023-09-10T05:17:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:53 smithi170 ceph-mon[102832]: pgmap v11629: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 373 KiB/s wr, 12 op/s 2023-09-10T05:17:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21505.374230 secs 2023-09-10T05:17:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:53 smithi082 ceph-mon[98694]: pgmap v11629: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 373 KiB/s wr, 12 op/s 2023-09-10T05:17:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21505.374230 secs 2023-09-10T05:17:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21506.027891 secs 2023-09-10T05:17:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21506.027891 secs 2023-09-10T05:17:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21506.027891 secs 2023-09-10T05:17:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:55 smithi098 ceph-mon[103124]: pgmap v11630: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 373 KiB/s wr, 12 op/s 2023-09-10T05:17:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:55 smithi170 ceph-mon[102832]: pgmap v11630: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 373 KiB/s wr, 12 op/s 2023-09-10T05:17:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:55 smithi082 ceph-mon[98694]: pgmap v11630: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 373 KiB/s wr, 12 op/s 2023-09-10T05:17:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:57 smithi098 ceph-mon[103124]: pgmap v11631: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 517 KiB/s wr, 18 op/s 2023-09-10T05:17:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21509.334456 secs 2023-09-10T05:17:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:57 smithi170 ceph-mon[102832]: pgmap v11631: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 517 KiB/s wr, 18 op/s 2023-09-10T05:17:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21509.334456 secs 2023-09-10T05:17:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:57 smithi082 ceph-mon[98694]: pgmap v11631: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 517 KiB/s wr, 18 op/s 2023-09-10T05:17:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21509.334456 secs 2023-09-10T05:17:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21510.374285 secs 2023-09-10T05:17:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21510.374285 secs 2023-09-10T05:17:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21510.374285 secs 2023-09-10T05:18:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:59 smithi098 ceph-mon[103124]: pgmap v11632: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T05:18:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:17:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21511.027956 secs 2023-09-10T05:18:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:59 smithi170 ceph-mon[102832]: pgmap v11632: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T05:18:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:17:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21511.027956 secs 2023-09-10T05:18:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:59 smithi082 ceph-mon[98694]: pgmap v11632: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T05:18:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:17:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21511.027956 secs 2023-09-10T05:18:02.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:01 smithi098 ceph-mon[103124]: pgmap v11633: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 330 KiB/s wr, 31 op/s 2023-09-10T05:18:02.827 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:01 smithi082 ceph-mon[98694]: pgmap v11633: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 330 KiB/s wr, 31 op/s 2023-09-10T05:18:02.828 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:01 smithi170 ceph-mon[102832]: pgmap v11633: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 330 KiB/s wr, 31 op/s 2023-09-10T05:18:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21514.334517 secs 2023-09-10T05:18:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21514.334517 secs 2023-09-10T05:18:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21514.334517 secs 2023-09-10T05:18:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:03 smithi098 ceph-mon[103124]: pgmap v11634: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 189 KiB/s wr, 24 op/s 2023-09-10T05:18:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21515.374344 secs 2023-09-10T05:18:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:03 smithi170 ceph-mon[102832]: pgmap v11634: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 189 KiB/s wr, 24 op/s 2023-09-10T05:18:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21515.374344 secs 2023-09-10T05:18:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:03 smithi082 ceph-mon[98694]: pgmap v11634: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 189 KiB/s wr, 24 op/s 2023-09-10T05:18:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21515.374344 secs 2023-09-10T05:18:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21516.028022 secs 2023-09-10T05:18:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21516.028022 secs 2023-09-10T05:18:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21516.028022 secs 2023-09-10T05:18:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:06 smithi098 ceph-mon[103124]: pgmap v11635: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 189 KiB/s wr, 24 op/s 2023-09-10T05:18:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:06 smithi170 ceph-mon[102832]: pgmap v11635: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 189 KiB/s wr, 24 op/s 2023-09-10T05:18:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:06 smithi082 ceph-mon[98694]: pgmap v11635: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 189 KiB/s wr, 24 op/s 2023-09-10T05:18:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:08 smithi098 ceph-mon[103124]: pgmap v11636: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 259 KiB/s wr, 29 op/s 2023-09-10T05:18:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21519.334578 secs 2023-09-10T05:18:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:08 smithi170 ceph-mon[102832]: pgmap v11636: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 259 KiB/s wr, 29 op/s 2023-09-10T05:18:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21519.334578 secs 2023-09-10T05:18:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:08 smithi082 ceph-mon[98694]: pgmap v11636: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 259 KiB/s wr, 29 op/s 2023-09-10T05:18:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21519.334578 secs 2023-09-10T05:18:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:09 smithi170 ceph-mon[102832]: osdmap e795: 12 total, 12 up, 12 in 2023-09-10T05:18:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21520.374405 secs 2023-09-10T05:18:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21521.028092 secs 2023-09-10T05:18:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:09 smithi082 ceph-mon[98694]: osdmap e795: 12 total, 12 up, 12 in 2023-09-10T05:18:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21520.374405 secs 2023-09-10T05:18:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21521.028092 secs 2023-09-10T05:18:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:09 smithi098 ceph-mon[103124]: osdmap e795: 12 total, 12 up, 12 in 2023-09-10T05:18:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21520.374405 secs 2023-09-10T05:18:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21521.028092 secs 2023-09-10T05:18:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:10 smithi170 ceph-mon[102832]: pgmap v11638: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 139 KiB/s wr, 29 op/s 2023-09-10T05:18:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:10 smithi082 ceph-mon[98694]: pgmap v11638: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 139 KiB/s wr, 29 op/s 2023-09-10T05:18:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:10 smithi098 ceph-mon[103124]: pgmap v11638: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 139 KiB/s wr, 29 op/s 2023-09-10T05:18:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:12 smithi170 ceph-mon[102832]: pgmap v11639: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T05:18:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:12 smithi082 ceph-mon[98694]: pgmap v11639: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T05:18:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:12 smithi098 ceph-mon[103124]: pgmap v11639: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T05:18:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21524.334641 secs 2023-09-10T05:18:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21524.334641 secs 2023-09-10T05:18:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21524.334641 secs 2023-09-10T05:18:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:14 smithi170 ceph-mon[102832]: pgmap v11640: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T05:18:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21525.374481 secs 2023-09-10T05:18:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21526.028157 secs 2023-09-10T05:18:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:14 smithi082 ceph-mon[98694]: pgmap v11640: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T05:18:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21525.374481 secs 2023-09-10T05:18:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21526.028157 secs 2023-09-10T05:18:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:14 smithi098 ceph-mon[103124]: pgmap v11640: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 11 op/s 2023-09-10T05:18:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21525.374481 secs 2023-09-10T05:18:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21526.028157 secs 2023-09-10T05:18:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:15 smithi170 ceph-mon[102832]: osdmap e796: 12 total, 12 up, 12 in 2023-09-10T05:18:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:15 smithi170 ceph-mon[102832]: pgmap v11642: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-09-10T05:18:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:15 smithi082 ceph-mon[98694]: osdmap e796: 12 total, 12 up, 12 in 2023-09-10T05:18:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:15 smithi082 ceph-mon[98694]: pgmap v11642: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-09-10T05:18:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:15 smithi098 ceph-mon[103124]: osdmap e796: 12 total, 12 up, 12 in 2023-09-10T05:18:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:15 smithi098 ceph-mon[103124]: pgmap v11642: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-09-10T05:18:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:17 smithi098 ceph-mon[103124]: pgmap v11643: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2023-09-10T05:18:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21529.334754 secs 2023-09-10T05:18:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:17 smithi170 ceph-mon[102832]: pgmap v11643: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2023-09-10T05:18:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21529.334754 secs 2023-09-10T05:18:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:17 smithi082 ceph-mon[98694]: pgmap v11643: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2023-09-10T05:18:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21529.334754 secs 2023-09-10T05:18:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21530.374544 secs 2023-09-10T05:18:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21530.374544 secs 2023-09-10T05:18:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21530.374544 secs 2023-09-10T05:18:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:19 smithi098 ceph-mon[103124]: pgmap v11644: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2023-09-10T05:18:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21531.028222 secs 2023-09-10T05:18:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:18:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:19 smithi170 ceph-mon[102832]: pgmap v11644: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2023-09-10T05:18:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21531.028222 secs 2023-09-10T05:18:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:18:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:19 smithi082 ceph-mon[98694]: pgmap v11644: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2023-09-10T05:18:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21531.028222 secs 2023-09-10T05:18:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:18:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:18:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:18:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:18:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:18:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:18:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:18:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:18:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:18:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:18:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:21 smithi082 ceph-mon[98694]: pgmap v11645: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 12 op/s 2023-09-10T05:18:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:21 smithi098 ceph-mon[103124]: pgmap v11645: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 12 op/s 2023-09-10T05:18:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:21 smithi170 ceph-mon[102832]: pgmap v11645: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 12 op/s 2023-09-10T05:18:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21534.334775 secs 2023-09-10T05:18:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21534.334775 secs 2023-09-10T05:18:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21534.334775 secs 2023-09-10T05:18:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:23 smithi082 ceph-mon[98694]: pgmap v11646: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 12 op/s 2023-09-10T05:18:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21535.374600 secs 2023-09-10T05:18:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:23 smithi098 ceph-mon[103124]: pgmap v11646: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 12 op/s 2023-09-10T05:18:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21535.374600 secs 2023-09-10T05:18:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:23 smithi170 ceph-mon[102832]: pgmap v11646: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 12 op/s 2023-09-10T05:18:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21535.374600 secs 2023-09-10T05:18:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21536.028288 secs 2023-09-10T05:18:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21536.028288 secs 2023-09-10T05:18:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21536.028288 secs 2023-09-10T05:18:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:25 smithi082 ceph-mon[98694]: pgmap v11647: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 12 op/s 2023-09-10T05:18:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:25 smithi098 ceph-mon[103124]: pgmap v11647: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 12 op/s 2023-09-10T05:18:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:25 smithi170 ceph-mon[102832]: pgmap v11647: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 85 KiB/s wr, 12 op/s 2023-09-10T05:18:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:27 smithi098 ceph-mon[103124]: pgmap v11648: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 72 KiB/s wr, 15 op/s 2023-09-10T05:18:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:27 smithi170 ceph-mon[102832]: pgmap v11648: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 72 KiB/s wr, 15 op/s 2023-09-10T05:18:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:27 smithi082 ceph-mon[98694]: pgmap v11648: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 72 KiB/s wr, 15 op/s 2023-09-10T05:18:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21540.374658 secs 2023-09-10T05:18:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21540.374658 secs 2023-09-10T05:18:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21540.374658 secs 2023-09-10T05:18:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:29 smithi098 ceph-mon[103124]: pgmap v11649: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 10 op/s 2023-09-10T05:18:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21541.028356 secs 2023-09-10T05:18:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:29 smithi170 ceph-mon[102832]: pgmap v11649: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 10 op/s 2023-09-10T05:18:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21541.028356 secs 2023-09-10T05:18:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:29 smithi082 ceph-mon[98694]: pgmap v11649: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 10 op/s 2023-09-10T05:18:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21541.028356 secs 2023-09-10T05:18:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:30 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:18:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:30 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:18:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:30 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:18:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:31 smithi098 ceph-mon[103124]: pgmap v11650: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 141 KiB/s wr, 15 op/s 2023-09-10T05:18:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:31 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:18:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:18:32.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:18:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:31 smithi170 ceph-mon[102832]: pgmap v11650: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 141 KiB/s wr, 15 op/s 2023-09-10T05:18:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:31 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:18:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:18:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:18:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:31 smithi082 ceph-mon[98694]: pgmap v11650: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 141 KiB/s wr, 15 op/s 2023-09-10T05:18:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:31 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:18:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:18:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:18:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21544.334903 secs 2023-09-10T05:18:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21544.334903 secs 2023-09-10T05:18:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21544.334903 secs 2023-09-10T05:18:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:33 smithi098 ceph-mon[103124]: pgmap v11651: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T05:18:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:18:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:18:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21545.374742 secs 2023-09-10T05:18:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:33 smithi170 ceph-mon[102832]: pgmap v11651: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T05:18:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:18:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:18:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21545.374742 secs 2023-09-10T05:18:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:33 smithi082 ceph-mon[98694]: pgmap v11651: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T05:18:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:18:34.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:18:34.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21545.374742 secs 2023-09-10T05:18:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21546.028423 secs 2023-09-10T05:18:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21546.028423 secs 2023-09-10T05:18:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21546.028423 secs 2023-09-10T05:18:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:35 smithi098 ceph-mon[103124]: pgmap v11652: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T05:18:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:35 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:18:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:35 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:18:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:35 smithi170 ceph-mon[102832]: pgmap v11652: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T05:18:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:35 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:18:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:35 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:18:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:35 smithi082 ceph-mon[98694]: pgmap v11652: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T05:18:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:35 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:18:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:35 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:18:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:37 smithi098 ceph-mon[103124]: pgmap v11653: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 313 KiB/s wr, 16 op/s 2023-09-10T05:18:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21549.334966 secs 2023-09-10T05:18:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:37 smithi170 ceph-mon[102832]: pgmap v11653: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 313 KiB/s wr, 16 op/s 2023-09-10T05:18:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21549.334966 secs 2023-09-10T05:18:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:37 smithi082 ceph-mon[98694]: pgmap v11653: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 313 KiB/s wr, 16 op/s 2023-09-10T05:18:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21549.334966 secs 2023-09-10T05:18:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21550.374800 secs 2023-09-10T05:18:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21550.374800 secs 2023-09-10T05:18:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21550.374800 secs 2023-09-10T05:18:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:39 smithi098 ceph-mon[103124]: pgmap v11654: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 313 KiB/s wr, 11 op/s 2023-09-10T05:18:40.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21551.028489 secs 2023-09-10T05:18:40.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:39 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:18:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:39 smithi170 ceph-mon[102832]: pgmap v11654: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 313 KiB/s wr, 11 op/s 2023-09-10T05:18:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21551.028489 secs 2023-09-10T05:18:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:39 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:18:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:39 smithi082 ceph-mon[98694]: pgmap v11654: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 313 KiB/s wr, 11 op/s 2023-09-10T05:18:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21551.028489 secs 2023-09-10T05:18:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:39 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:18:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:41 smithi098 ceph-mon[103124]: pgmap v11655: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 357 KiB/s wr, 16 op/s 2023-09-10T05:18:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:41 smithi170 ceph-mon[102832]: pgmap v11655: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 357 KiB/s wr, 16 op/s 2023-09-10T05:18:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:41 smithi082 ceph-mon[98694]: pgmap v11655: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 357 KiB/s wr, 16 op/s 2023-09-10T05:18:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:18:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:18:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:42 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:18:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:44 smithi098 ceph-mon[103124]: pgmap v11656: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 287 KiB/s wr, 11 op/s 2023-09-10T05:18:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:18:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:18:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21555.374861 secs 2023-09-10T05:18:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:44 smithi170 ceph-mon[102832]: pgmap v11656: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 287 KiB/s wr, 11 op/s 2023-09-10T05:18:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:18:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:18:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21555.374861 secs 2023-09-10T05:18:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:44 smithi082 ceph-mon[98694]: pgmap v11656: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 287 KiB/s wr, 11 op/s 2023-09-10T05:18:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:18:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:18:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21555.374861 secs 2023-09-10T05:18:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21556.028557 secs 2023-09-10T05:18:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:45 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:18:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21556.028557 secs 2023-09-10T05:18:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:45 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:18:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21556.028557 secs 2023-09-10T05:18:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:45 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:18:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:46 smithi170 ceph-mon[102832]: pgmap v11657: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 287 KiB/s wr, 11 op/s 2023-09-10T05:18:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:18:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:18:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:46 smithi082 ceph-mon[98694]: pgmap v11657: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 287 KiB/s wr, 11 op/s 2023-09-10T05:18:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:18:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:18:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:46 smithi098 ceph-mon[103124]: pgmap v11657: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 287 KiB/s wr, 11 op/s 2023-09-10T05:18:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:18:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:18:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:48 smithi170 ceph-mon[102832]: pgmap v11658: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 512 KiB/s wr, 18 op/s 2023-09-10T05:18:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:48 smithi082 ceph-mon[98694]: pgmap v11658: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 512 KiB/s wr, 18 op/s 2023-09-10T05:18:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:48 smithi098 ceph-mon[103124]: pgmap v11658: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 512 KiB/s wr, 18 op/s 2023-09-10T05:18:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21560.374922 secs 2023-09-10T05:18:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21561.028623 secs 2023-09-10T05:18:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21560.374922 secs 2023-09-10T05:18:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21561.028623 secs 2023-09-10T05:18:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21560.374922 secs 2023-09-10T05:18:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21561.028623 secs 2023-09-10T05:18:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:50 smithi170 ceph-mon[102832]: pgmap v11659: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 269 KiB/s wr, 11 op/s 2023-09-10T05:18:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:50 smithi082 ceph-mon[98694]: pgmap v11659: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 269 KiB/s wr, 11 op/s 2023-09-10T05:18:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:50 smithi098 ceph-mon[103124]: pgmap v11659: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 269 KiB/s wr, 11 op/s 2023-09-10T05:18:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:52 smithi170 ceph-mon[102832]: pgmap v11660: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 312 KiB/s wr, 17 op/s 2023-09-10T05:18:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:52 smithi082 ceph-mon[98694]: pgmap v11660: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 312 KiB/s wr, 17 op/s 2023-09-10T05:18:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:52 smithi098 ceph-mon[103124]: pgmap v11660: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 312 KiB/s wr, 17 op/s 2023-09-10T05:18:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21564.335213 secs 2023-09-10T05:18:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21564.335213 secs 2023-09-10T05:18:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21564.335213 secs 2023-09-10T05:18:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:54 smithi170 ceph-mon[102832]: pgmap v11661: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 268 KiB/s wr, 11 op/s 2023-09-10T05:18:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:18:54.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:18:54.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21565.374981 secs 2023-09-10T05:18:54.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21566.028689 secs 2023-09-10T05:18:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:54 smithi082 ceph-mon[98694]: pgmap v11661: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 268 KiB/s wr, 11 op/s 2023-09-10T05:18:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:18:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:18:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21565.374981 secs 2023-09-10T05:18:54.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21566.028689 secs 2023-09-10T05:18:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:54 smithi098 ceph-mon[103124]: pgmap v11661: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 268 KiB/s wr, 11 op/s 2023-09-10T05:18:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:18:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:18:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21565.374981 secs 2023-09-10T05:18:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21566.028689 secs 2023-09-10T05:18:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:55 smithi170 ceph-mon[102832]: pgmap v11662: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 268 KiB/s wr, 11 op/s 2023-09-10T05:18:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:55 smithi082 ceph-mon[98694]: pgmap v11662: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 268 KiB/s wr, 11 op/s 2023-09-10T05:18:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:55 smithi098 ceph-mon[103124]: pgmap v11662: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 268 KiB/s wr, 11 op/s 2023-09-10T05:18:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:18:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:18:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:18:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:18:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:18:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:18:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:18:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:18:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:18:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:57 smithi170 ceph-mon[102832]: pgmap v11663: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 530 KiB/s wr, 17 op/s 2023-09-10T05:18:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:57 smithi082 ceph-mon[98694]: pgmap v11663: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 530 KiB/s wr, 17 op/s 2023-09-10T05:18:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:57 smithi098 ceph-mon[103124]: pgmap v11663: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 530 KiB/s wr, 17 op/s 2023-09-10T05:18:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21569.335275 secs 2023-09-10T05:18:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21570.375064 secs 2023-09-10T05:18:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21569.335275 secs 2023-09-10T05:18:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21570.375064 secs 2023-09-10T05:18:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21569.335275 secs 2023-09-10T05:18:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21570.375064 secs 2023-09-10T05:18:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:59 smithi170 ceph-mon[102832]: pgmap v11664: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 306 KiB/s wr, 11 op/s 2023-09-10T05:18:59.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:18:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21571.028757 secs 2023-09-10T05:18:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:59 smithi082 ceph-mon[98694]: pgmap v11664: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 306 KiB/s wr, 11 op/s 2023-09-10T05:18:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:18:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21571.028757 secs 2023-09-10T05:19:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:59 smithi098 ceph-mon[103124]: pgmap v11664: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 306 KiB/s wr, 11 op/s 2023-09-10T05:19:00.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:18:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21571.028757 secs 2023-09-10T05:19:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:01 smithi098 ceph-mon[103124]: pgmap v11665: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 497 KiB/s wr, 21 op/s 2023-09-10T05:19:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:01 smithi170 ceph-mon[102832]: pgmap v11665: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 497 KiB/s wr, 21 op/s 2023-09-10T05:19:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:01 smithi082 ceph-mon[98694]: pgmap v11665: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 497 KiB/s wr, 21 op/s 2023-09-10T05:19:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21574.335355 secs 2023-09-10T05:19:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21574.335355 secs 2023-09-10T05:19:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21574.335355 secs 2023-09-10T05:19:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:03 smithi098 ceph-mon[103124]: pgmap v11666: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 454 KiB/s wr, 15 op/s 2023-09-10T05:19:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21575.375123 secs 2023-09-10T05:19:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:03 smithi170 ceph-mon[102832]: pgmap v11666: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 454 KiB/s wr, 15 op/s 2023-09-10T05:19:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21575.375123 secs 2023-09-10T05:19:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:03 smithi082 ceph-mon[98694]: pgmap v11666: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 454 KiB/s wr, 15 op/s 2023-09-10T05:19:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21575.375123 secs 2023-09-10T05:19:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21576.028823 secs 2023-09-10T05:19:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21576.028823 secs 2023-09-10T05:19:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21576.028823 secs 2023-09-10T05:19:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:05 smithi170 ceph-mon[102832]: pgmap v11667: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 454 KiB/s wr, 15 op/s 2023-09-10T05:19:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:05 smithi082 ceph-mon[98694]: pgmap v11667: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 454 KiB/s wr, 15 op/s 2023-09-10T05:19:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:05 smithi098 ceph-mon[103124]: pgmap v11667: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 454 KiB/s wr, 15 op/s 2023-09-10T05:19:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:07 smithi170 ceph-mon[102832]: pgmap v11668: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 549 KiB/s wr, 32 op/s 2023-09-10T05:19:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21579.335414 secs 2023-09-10T05:19:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:07 smithi082 ceph-mon[98694]: pgmap v11668: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 549 KiB/s wr, 32 op/s 2023-09-10T05:19:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21579.335414 secs 2023-09-10T05:19:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:07 smithi098 ceph-mon[103124]: pgmap v11668: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 549 KiB/s wr, 32 op/s 2023-09-10T05:19:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21579.335414 secs 2023-09-10T05:19:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:08 smithi170 ceph-mon[102832]: osdmap e797: 12 total, 12 up, 12 in 2023-09-10T05:19:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21580.375183 secs 2023-09-10T05:19:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:08 smithi082 ceph-mon[98694]: osdmap e797: 12 total, 12 up, 12 in 2023-09-10T05:19:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21580.375183 secs 2023-09-10T05:19:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:08 smithi098 ceph-mon[103124]: osdmap e797: 12 total, 12 up, 12 in 2023-09-10T05:19:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21580.375183 secs 2023-09-10T05:19:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:09 smithi170 ceph-mon[102832]: pgmap v11670: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 344 KiB/s wr, 31 op/s 2023-09-10T05:19:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21581.028888 secs 2023-09-10T05:19:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:09 smithi082 ceph-mon[98694]: pgmap v11670: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 344 KiB/s wr, 31 op/s 2023-09-10T05:19:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21581.028888 secs 2023-09-10T05:19:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:09 smithi098 ceph-mon[103124]: pgmap v11670: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 344 KiB/s wr, 31 op/s 2023-09-10T05:19:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21581.028888 secs 2023-09-10T05:19:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:11 smithi170 ceph-mon[102832]: pgmap v11671: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 115 KiB/s wr, 25 op/s 2023-09-10T05:19:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:11 smithi082 ceph-mon[98694]: pgmap v11671: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 115 KiB/s wr, 25 op/s 2023-09-10T05:19:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:11 smithi098 ceph-mon[103124]: pgmap v11671: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 115 KiB/s wr, 25 op/s 2023-09-10T05:19:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21584.335481 secs 2023-09-10T05:19:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21584.335481 secs 2023-09-10T05:19:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21584.335481 secs 2023-09-10T05:19:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:13 smithi170 ceph-mon[102832]: pgmap v11672: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 115 KiB/s wr, 25 op/s 2023-09-10T05:19:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:19:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:19:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21585.375241 secs 2023-09-10T05:19:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:13 smithi082 ceph-mon[98694]: pgmap v11672: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 115 KiB/s wr, 25 op/s 2023-09-10T05:19:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:19:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:19:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21585.375241 secs 2023-09-10T05:19:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:13 smithi098 ceph-mon[103124]: pgmap v11672: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 115 KiB/s wr, 25 op/s 2023-09-10T05:19:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:19:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:19:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21585.375241 secs 2023-09-10T05:19:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21586.028955 secs 2023-09-10T05:19:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:14 smithi170 ceph-mon[102832]: osdmap e798: 12 total, 12 up, 12 in 2023-09-10T05:19:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21586.028955 secs 2023-09-10T05:19:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:14 smithi082 ceph-mon[98694]: osdmap e798: 12 total, 12 up, 12 in 2023-09-10T05:19:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21586.028955 secs 2023-09-10T05:19:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:14 smithi098 ceph-mon[103124]: osdmap e798: 12 total, 12 up, 12 in 2023-09-10T05:19:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:15 smithi170 ceph-mon[102832]: pgmap v11674: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 383 B/s wr, 7 op/s 2023-09-10T05:19:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:15 smithi082 ceph-mon[98694]: pgmap v11674: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 383 B/s wr, 7 op/s 2023-09-10T05:19:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:15 smithi098 ceph-mon[103124]: pgmap v11674: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 383 B/s wr, 7 op/s 2023-09-10T05:19:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:19:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:19:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:19:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:19:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:19:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:19:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:17 smithi170 ceph-mon[102832]: pgmap v11675: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 116 KiB/s wr, 13 op/s 2023-09-10T05:19:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21589.335580 secs 2023-09-10T05:19:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:17 smithi082 ceph-mon[98694]: pgmap v11675: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 116 KiB/s wr, 13 op/s 2023-09-10T05:19:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21589.335580 secs 2023-09-10T05:19:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:17 smithi098 ceph-mon[103124]: pgmap v11675: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 116 KiB/s wr, 13 op/s 2023-09-10T05:19:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21589.335580 secs 2023-09-10T05:19:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21590.375312 secs 2023-09-10T05:19:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21590.375312 secs 2023-09-10T05:19:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21590.375312 secs 2023-09-10T05:19:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:19 smithi170 ceph-mon[102832]: pgmap v11676: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T05:19:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21591.029022 secs 2023-09-10T05:19:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:19 smithi082 ceph-mon[98694]: pgmap v11676: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T05:19:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21591.029022 secs 2023-09-10T05:19:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:19 smithi098 ceph-mon[103124]: pgmap v11676: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T05:19:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21591.029022 secs 2023-09-10T05:19:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:19:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:19:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:19:22.039 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:21 smithi170 ceph-mon[102832]: pgmap v11677: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 168 KiB/s wr, 11 op/s 2023-09-10T05:19:22.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:21 smithi082 ceph-mon[98694]: pgmap v11677: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 168 KiB/s wr, 11 op/s 2023-09-10T05:19:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:21 smithi098 ceph-mon[103124]: pgmap v11677: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 168 KiB/s wr, 11 op/s 2023-09-10T05:19:23.093 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21594.335663 secs 2023-09-10T05:19:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21594.335663 secs 2023-09-10T05:19:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21594.335663 secs 2023-09-10T05:19:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:23 smithi098 ceph-mon[103124]: pgmap v11678: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 168 KiB/s wr, 11 op/s 2023-09-10T05:19:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21595.375374 secs 2023-09-10T05:19:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:19:24.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:19:24.123 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:23 smithi170 ceph-mon[102832]: pgmap v11678: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 168 KiB/s wr, 11 op/s 2023-09-10T05:19:24.123 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21595.375374 secs 2023-09-10T05:19:24.124 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:19:24.124 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:19:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:23 smithi082 ceph-mon[98694]: pgmap v11678: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 168 KiB/s wr, 11 op/s 2023-09-10T05:19:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21595.375374 secs 2023-09-10T05:19:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:19:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:19:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21596.029091 secs 2023-09-10T05:19:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:19:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:19:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21596.029091 secs 2023-09-10T05:19:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:19:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:19:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21596.029091 secs 2023-09-10T05:19:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:19:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:19:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:25 smithi170 ceph-mon[102832]: pgmap v11679: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T05:19:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:25 smithi082 ceph-mon[98694]: pgmap v11679: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T05:19:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:25 smithi098 ceph-mon[103124]: pgmap v11679: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T05:19:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:27 smithi098 ceph-mon[103124]: pgmap v11680: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 14 op/s 2023-09-10T05:19:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:19:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:19:28.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:19:28.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:19:28.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:19:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:27 smithi170 ceph-mon[102832]: pgmap v11680: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 14 op/s 2023-09-10T05:19:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:19:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:19:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:19:28.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:19:28.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:19:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:27 smithi082 ceph-mon[98694]: pgmap v11680: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 14 op/s 2023-09-10T05:19:28.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:19:28.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:19:28.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:19:28.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:19:28.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:19:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21600.375433 secs 2023-09-10T05:19:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21600.375433 secs 2023-09-10T05:19:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21600.375433 secs 2023-09-10T05:19:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:29 smithi098 ceph-mon[103124]: pgmap v11681: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 9 op/s 2023-09-10T05:19:30.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21601.029158 secs 2023-09-10T05:19:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:29 smithi170 ceph-mon[102832]: pgmap v11681: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 9 op/s 2023-09-10T05:19:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21601.029158 secs 2023-09-10T05:19:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:29 smithi082 ceph-mon[98694]: pgmap v11681: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 9 op/s 2023-09-10T05:19:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21601.029158 secs 2023-09-10T05:19:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:31 smithi098 ceph-mon[103124]: pgmap v11682: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 161 KiB/s wr, 15 op/s 2023-09-10T05:19:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:19:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:31 smithi170 ceph-mon[102832]: pgmap v11682: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 161 KiB/s wr, 15 op/s 2023-09-10T05:19:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:19:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:31 smithi082 ceph-mon[98694]: pgmap v11682: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 161 KiB/s wr, 15 op/s 2023-09-10T05:19:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:19:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:19:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:19:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:19:33.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:32 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21604.335776 secs 2023-09-10T05:19:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:19:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:19:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:19:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:32 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21604.335776 secs 2023-09-10T05:19:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:19:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:19:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:19:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:32 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21604.335776 secs 2023-09-10T05:19:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:33 smithi098 ceph-mon[103124]: pgmap v11683: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 107 KiB/s wr, 10 op/s 2023-09-10T05:19:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21605.375497 secs 2023-09-10T05:19:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:33 smithi170 ceph-mon[102832]: pgmap v11683: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 107 KiB/s wr, 10 op/s 2023-09-10T05:19:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21605.375497 secs 2023-09-10T05:19:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:33 smithi082 ceph-mon[98694]: pgmap v11683: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 107 KiB/s wr, 10 op/s 2023-09-10T05:19:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21605.375497 secs 2023-09-10T05:19:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21606.029221 secs 2023-09-10T05:19:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21606.029221 secs 2023-09-10T05:19:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21606.029221 secs 2023-09-10T05:19:36.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:35 smithi098 ceph-mon[103124]: pgmap v11684: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 107 KiB/s wr, 10 op/s 2023-09-10T05:19:36.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:35 smithi170 ceph-mon[102832]: pgmap v11684: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 107 KiB/s wr, 10 op/s 2023-09-10T05:19:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:35 smithi082 ceph-mon[98694]: pgmap v11684: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 107 KiB/s wr, 10 op/s 2023-09-10T05:19:37.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:19:37.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:19:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:19:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:19:37.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:19:37.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:19:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:37 smithi170 ceph-mon[102832]: pgmap v11685: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 270 KiB/s wr, 16 op/s 2023-09-10T05:19:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:19:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:19:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:37 smithi082 ceph-mon[98694]: pgmap v11685: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 270 KiB/s wr, 16 op/s 2023-09-10T05:19:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:19:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:19:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:37 smithi098 ceph-mon[103124]: pgmap v11685: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 270 KiB/s wr, 16 op/s 2023-09-10T05:19:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:19:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:19:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21610.375555 secs 2023-09-10T05:19:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21610.375555 secs 2023-09-10T05:19:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21610.375555 secs 2023-09-10T05:19:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:39 smithi170 ceph-mon[102832]: pgmap v11686: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T05:19:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21611.029287 secs 2023-09-10T05:19:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:19:40.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:39 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:19:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:39 smithi082 ceph-mon[98694]: pgmap v11686: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T05:19:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21611.029287 secs 2023-09-10T05:19:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:19:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:39 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:19:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:39 smithi098 ceph-mon[103124]: pgmap v11686: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T05:19:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21611.029287 secs 2023-09-10T05:19:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:19:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:39 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:19:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:19:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:19:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:19:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:19:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:19:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:19:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:41 smithi170 ceph-mon[102832]: pgmap v11687: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 482 KiB/s wr, 19 op/s 2023-09-10T05:19:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:41 smithi082 ceph-mon[98694]: pgmap v11687: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 482 KiB/s wr, 19 op/s 2023-09-10T05:19:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:41 smithi098 ceph-mon[103124]: pgmap v11687: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 482 KiB/s wr, 19 op/s 2023-09-10T05:19:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:43 smithi170 ceph-mon[102832]: pgmap v11688: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 376 KiB/s wr, 13 op/s 2023-09-10T05:19:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21615.375613 secs 2023-09-10T05:19:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:43 smithi082 ceph-mon[98694]: pgmap v11688: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 376 KiB/s wr, 13 op/s 2023-09-10T05:19:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21615.375613 secs 2023-09-10T05:19:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:43 smithi098 ceph-mon[103124]: pgmap v11688: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 376 KiB/s wr, 13 op/s 2023-09-10T05:19:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21615.375613 secs 2023-09-10T05:19:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21616.029351 secs 2023-09-10T05:19:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:44 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:19:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21616.029351 secs 2023-09-10T05:19:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:44 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:19:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21616.029351 secs 2023-09-10T05:19:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:44 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:19:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:45 smithi170 ceph-mon[102832]: pgmap v11689: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 376 KiB/s wr, 13 op/s 2023-09-10T05:19:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:45 smithi082 ceph-mon[98694]: pgmap v11689: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 376 KiB/s wr, 13 op/s 2023-09-10T05:19:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:45 smithi098 ceph-mon[103124]: pgmap v11689: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 376 KiB/s wr, 13 op/s 2023-09-10T05:19:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:47 smithi170 ceph-mon[102832]: pgmap v11690: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 473 KiB/s wr, 20 op/s 2023-09-10T05:19:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21619.336010 secs 2023-09-10T05:19:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:47 smithi082 ceph-mon[98694]: pgmap v11690: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 473 KiB/s wr, 20 op/s 2023-09-10T05:19:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21619.336010 secs 2023-09-10T05:19:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:47 smithi098 ceph-mon[103124]: pgmap v11690: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 473 KiB/s wr, 20 op/s 2023-09-10T05:19:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21619.336010 secs 2023-09-10T05:19:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21620.375676 secs 2023-09-10T05:19:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21620.375676 secs 2023-09-10T05:19:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21620.375676 secs 2023-09-10T05:19:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:49 smithi170 ceph-mon[102832]: pgmap v11691: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 310 KiB/s wr, 13 op/s 2023-09-10T05:19:50.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21621.029416 secs 2023-09-10T05:19:50.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:19:50.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:19:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:49 smithi082 ceph-mon[98694]: pgmap v11691: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 310 KiB/s wr, 13 op/s 2023-09-10T05:19:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21621.029416 secs 2023-09-10T05:19:50.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:19:50.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:19:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:49 smithi098 ceph-mon[103124]: pgmap v11691: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 310 KiB/s wr, 13 op/s 2023-09-10T05:19:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21621.029416 secs 2023-09-10T05:19:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:19:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:19:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:51 smithi170 ceph-mon[102832]: pgmap v11692: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 413 KiB/s wr, 19 op/s 2023-09-10T05:19:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:51 smithi082 ceph-mon[98694]: pgmap v11692: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 413 KiB/s wr, 19 op/s 2023-09-10T05:19:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:51 smithi098 ceph-mon[103124]: pgmap v11692: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 413 KiB/s wr, 19 op/s 2023-09-10T05:19:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:19:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:19:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:19:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21624.336084 secs 2023-09-10T05:19:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:19:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:19:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:19:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21624.336084 secs 2023-09-10T05:19:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:19:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:19:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:19:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21624.336084 secs 2023-09-10T05:19:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:53 smithi170 ceph-mon[102832]: pgmap v11693: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 200 KiB/s wr, 12 op/s 2023-09-10T05:19:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21625.375733 secs 2023-09-10T05:19:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:53 smithi082 ceph-mon[98694]: pgmap v11693: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 200 KiB/s wr, 12 op/s 2023-09-10T05:19:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21625.375733 secs 2023-09-10T05:19:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:53 smithi098 ceph-mon[103124]: pgmap v11693: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 200 KiB/s wr, 12 op/s 2023-09-10T05:19:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21625.375733 secs 2023-09-10T05:19:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21626.029487 secs 2023-09-10T05:19:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21626.029487 secs 2023-09-10T05:19:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21626.029487 secs 2023-09-10T05:19:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:55 smithi170 ceph-mon[102832]: pgmap v11694: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 200 KiB/s wr, 12 op/s 2023-09-10T05:19:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:55 smithi082 ceph-mon[98694]: pgmap v11694: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 200 KiB/s wr, 12 op/s 2023-09-10T05:19:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:55 smithi098 ceph-mon[103124]: pgmap v11694: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 200 KiB/s wr, 12 op/s 2023-09-10T05:19:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:57 smithi170 ceph-mon[102832]: pgmap v11695: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 17 op/s 2023-09-10T05:19:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21629.336143 secs 2023-09-10T05:19:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:57 smithi082 ceph-mon[98694]: pgmap v11695: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 17 op/s 2023-09-10T05:19:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21629.336143 secs 2023-09-10T05:19:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:57 smithi098 ceph-mon[103124]: pgmap v11695: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 17 op/s 2023-09-10T05:19:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21629.336143 secs 2023-09-10T05:19:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21630.375791 secs 2023-09-10T05:19:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21630.375791 secs 2023-09-10T05:19:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21630.375791 secs 2023-09-10T05:20:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:59 smithi170 ceph-mon[102832]: pgmap v11696: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 11 op/s 2023-09-10T05:20:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:19:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21631.029552 secs 2023-09-10T05:20:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:59 smithi082 ceph-mon[98694]: pgmap v11696: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 11 op/s 2023-09-10T05:20:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:19:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21631.029552 secs 2023-09-10T05:20:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:59 smithi098 ceph-mon[103124]: pgmap v11696: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 11 op/s 2023-09-10T05:20:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:19:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21631.029552 secs 2023-09-10T05:20:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T05:20:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T05:20:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T05:20:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:01 smithi170 ceph-mon[102832]: pgmap v11697: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 179 KiB/s wr, 15 op/s 2023-09-10T05:20:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:01 smithi082 ceph-mon[98694]: pgmap v11697: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 179 KiB/s wr, 15 op/s 2023-09-10T05:20:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:01 smithi098 ceph-mon[103124]: pgmap v11697: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 179 KiB/s wr, 15 op/s 2023-09-10T05:20:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21634.336201 secs 2023-09-10T05:20:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21634.336201 secs 2023-09-10T05:20:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21634.336201 secs 2023-09-10T05:20:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:03 smithi170 ceph-mon[102832]: pgmap v11698: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T05:20:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21635.375857 secs 2023-09-10T05:20:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:03 smithi082 ceph-mon[98694]: pgmap v11698: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T05:20:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21635.375857 secs 2023-09-10T05:20:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:03 smithi098 ceph-mon[103124]: pgmap v11698: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T05:20:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21635.375857 secs 2023-09-10T05:20:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21636.029617 secs 2023-09-10T05:20:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21636.029617 secs 2023-09-10T05:20:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21636.029617 secs 2023-09-10T05:20:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:05 smithi170 ceph-mon[102832]: pgmap v11699: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T05:20:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:05 smithi082 ceph-mon[98694]: pgmap v11699: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T05:20:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:05 smithi098 ceph-mon[103124]: pgmap v11699: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T05:20:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:07 smithi170 ceph-mon[102832]: pgmap v11700: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 175 KiB/s wr, 28 op/s 2023-09-10T05:20:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21639.336311 secs 2023-09-10T05:20:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:07 smithi082 ceph-mon[98694]: pgmap v11700: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 175 KiB/s wr, 28 op/s 2023-09-10T05:20:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21639.336311 secs 2023-09-10T05:20:08.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:07 smithi098 ceph-mon[103124]: pgmap v11700: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 175 KiB/s wr, 28 op/s 2023-09-10T05:20:08.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21639.336311 secs 2023-09-10T05:20:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:08 smithi170 ceph-mon[102832]: osdmap e799: 12 total, 12 up, 12 in 2023-09-10T05:20:09.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21640.375915 secs 2023-09-10T05:20:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:08 smithi082 ceph-mon[98694]: osdmap e799: 12 total, 12 up, 12 in 2023-09-10T05:20:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21640.375915 secs 2023-09-10T05:20:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:08 smithi098 ceph-mon[103124]: osdmap e799: 12 total, 12 up, 12 in 2023-09-10T05:20:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21640.375915 secs 2023-09-10T05:20:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:09 smithi170 ceph-mon[102832]: pgmap v11702: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 119 KiB/s wr, 27 op/s 2023-09-10T05:20:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21641.029682 secs 2023-09-10T05:20:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:09 smithi082 ceph-mon[98694]: pgmap v11702: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 119 KiB/s wr, 27 op/s 2023-09-10T05:20:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21641.029682 secs 2023-09-10T05:20:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:09 smithi098 ceph-mon[103124]: pgmap v11702: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 119 KiB/s wr, 27 op/s 2023-09-10T05:20:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21641.029682 secs 2023-09-10T05:20:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:11 smithi170 ceph-mon[102832]: pgmap v11703: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 119 KiB/s wr, 28 op/s 2023-09-10T05:20:12.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:11 smithi082 ceph-mon[98694]: pgmap v11703: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 119 KiB/s wr, 28 op/s 2023-09-10T05:20:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:11 smithi098 ceph-mon[103124]: pgmap v11703: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 119 KiB/s wr, 28 op/s 2023-09-10T05:20:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21644.336371 secs 2023-09-10T05:20:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21644.336371 secs 2023-09-10T05:20:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21644.336371 secs 2023-09-10T05:20:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:13 smithi170 ceph-mon[102832]: pgmap v11704: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 119 KiB/s wr, 28 op/s 2023-09-10T05:20:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21645.375980 secs 2023-09-10T05:20:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:13 smithi082 ceph-mon[98694]: pgmap v11704: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 119 KiB/s wr, 28 op/s 2023-09-10T05:20:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21645.375980 secs 2023-09-10T05:20:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:13 smithi098 ceph-mon[103124]: pgmap v11704: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 119 KiB/s wr, 28 op/s 2023-09-10T05:20:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21645.375980 secs 2023-09-10T05:20:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21646.029750 secs 2023-09-10T05:20:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:14 smithi170 ceph-mon[102832]: osdmap e800: 12 total, 12 up, 12 in 2023-09-10T05:20:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21646.029750 secs 2023-09-10T05:20:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:14 smithi082 ceph-mon[98694]: osdmap e800: 12 total, 12 up, 12 in 2023-09-10T05:20:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21646.029750 secs 2023-09-10T05:20:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:14 smithi098 ceph-mon[103124]: osdmap e800: 12 total, 12 up, 12 in 2023-09-10T05:20:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:15 smithi170 ceph-mon[102832]: pgmap v11706: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:20:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:15 smithi082 ceph-mon[98694]: pgmap v11706: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:20:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:15 smithi098 ceph-mon[103124]: pgmap v11706: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:20:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:17 smithi170 ceph-mon[102832]: pgmap v11707: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 347 B/s wr, 11 op/s 2023-09-10T05:20:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21649.336432 secs 2023-09-10T05:20:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:17 smithi082 ceph-mon[98694]: pgmap v11707: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 347 B/s wr, 11 op/s 2023-09-10T05:20:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21649.336432 secs 2023-09-10T05:20:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:17 smithi098 ceph-mon[103124]: pgmap v11707: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 347 B/s wr, 11 op/s 2023-09-10T05:20:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21649.336432 secs 2023-09-10T05:20:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21650.376038 secs 2023-09-10T05:20:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21650.376038 secs 2023-09-10T05:20:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21650.376038 secs 2023-09-10T05:20:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:19 smithi170 ceph-mon[102832]: pgmap v11708: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T05:20:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21651.029818 secs 2023-09-10T05:20:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:19 smithi082 ceph-mon[98694]: pgmap v11708: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T05:20:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21651.029818 secs 2023-09-10T05:20:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:19 smithi098 ceph-mon[103124]: pgmap v11708: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T05:20:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21651.029818 secs 2023-09-10T05:20:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:21 smithi170 ceph-mon[102832]: pgmap v11709: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T05:20:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:21 smithi082 ceph-mon[98694]: pgmap v11709: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T05:20:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:21 smithi098 ceph-mon[103124]: pgmap v11709: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T05:20:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21654.336549 secs 2023-09-10T05:20:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21654.336549 secs 2023-09-10T05:20:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21654.336549 secs 2023-09-10T05:20:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:23 smithi170 ceph-mon[102832]: pgmap v11710: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T05:20:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21655.376096 secs 2023-09-10T05:20:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:23 smithi082 ceph-mon[98694]: pgmap v11710: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T05:20:24.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21655.376096 secs 2023-09-10T05:20:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:23 smithi098 ceph-mon[103124]: pgmap v11710: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T05:20:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21655.376096 secs 2023-09-10T05:20:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21656.029884 secs 2023-09-10T05:20:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21656.029884 secs 2023-09-10T05:20:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21656.029884 secs 2023-09-10T05:20:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:25 smithi170 ceph-mon[102832]: pgmap v11711: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T05:20:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:25 smithi082 ceph-mon[98694]: pgmap v11711: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T05:20:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:25 smithi098 ceph-mon[103124]: pgmap v11711: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 11 op/s 2023-09-10T05:20:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:27 smithi170 ceph-mon[102832]: pgmap v11712: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 66 KiB/s wr, 14 op/s 2023-09-10T05:20:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:20:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:20:28.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:20:28.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:20:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:27 smithi082 ceph-mon[98694]: pgmap v11712: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 66 KiB/s wr, 14 op/s 2023-09-10T05:20:28.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:20:28.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:20:28.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:20:28.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:20:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:27 smithi098 ceph-mon[103124]: pgmap v11712: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 66 KiB/s wr, 14 op/s 2023-09-10T05:20:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:20:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:20:28.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:20:28.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:20:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21660.376154 secs 2023-09-10T05:20:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21660.376154 secs 2023-09-10T05:20:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21660.376154 secs 2023-09-10T05:20:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:29 smithi170 ceph-mon[102832]: pgmap v11713: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 9 op/s 2023-09-10T05:20:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21661.029947 secs 2023-09-10T05:20:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:29 smithi082 ceph-mon[98694]: pgmap v11713: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 9 op/s 2023-09-10T05:20:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21661.029947 secs 2023-09-10T05:20:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:29 smithi098 ceph-mon[103124]: pgmap v11713: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 9 op/s 2023-09-10T05:20:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21661.029947 secs 2023-09-10T05:20:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:30 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:20:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:30 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:20:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:30 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:20:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:31 smithi170 ceph-mon[102832]: pgmap v11714: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 191 KiB/s wr, 16 op/s 2023-09-10T05:20:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:31 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:20:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:20:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:20:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:31 smithi082 ceph-mon[98694]: pgmap v11714: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 191 KiB/s wr, 16 op/s 2023-09-10T05:20:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:31 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:20:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:20:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:20:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:31 smithi098 ceph-mon[103124]: pgmap v11714: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 191 KiB/s wr, 16 op/s 2023-09-10T05:20:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:31 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:20:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:20:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:20:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:33 smithi170 ceph-mon[102832]: pgmap v11715: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T05:20:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:20:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:20:34.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21665.376212 secs 2023-09-10T05:20:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:33 smithi082 ceph-mon[98694]: pgmap v11715: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T05:20:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:20:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:20:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21665.376212 secs 2023-09-10T05:20:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:33 smithi098 ceph-mon[103124]: pgmap v11715: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T05:20:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:20:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:20:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21665.376212 secs 2023-09-10T05:20:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21666.030015 secs 2023-09-10T05:20:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21666.030015 secs 2023-09-10T05:20:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21666.030015 secs 2023-09-10T05:20:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:35 smithi170 ceph-mon[102832]: pgmap v11716: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T05:20:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:35 smithi082 ceph-mon[98694]: pgmap v11716: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T05:20:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:35 smithi098 ceph-mon[103124]: pgmap v11716: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T05:20:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:37 smithi170 ceph-mon[102832]: pgmap v11717: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 310 KiB/s wr, 17 op/s 2023-09-10T05:20:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:37 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21669.336734 secs 2023-09-10T05:20:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:37 smithi082 ceph-mon[98694]: pgmap v11717: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 310 KiB/s wr, 17 op/s 2023-09-10T05:20:39.193 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:37 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21669.336734 secs 2023-09-10T05:20:39.194 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:37 smithi098 ceph-mon[103124]: pgmap v11717: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 310 KiB/s wr, 17 op/s 2023-09-10T05:20:39.194 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:37 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21669.336734 secs 2023-09-10T05:20:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21670.376300 secs 2023-09-10T05:20:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21670.376300 secs 2023-09-10T05:20:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21670.376300 secs 2023-09-10T05:20:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:39 smithi082 ceph-mon[98694]: pgmap v11718: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 310 KiB/s wr, 12 op/s 2023-09-10T05:20:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21671.030082 secs 2023-09-10T05:20:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:39 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:20:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:39 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:20:40.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:39 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:20:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:39 smithi098 ceph-mon[103124]: pgmap v11718: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 310 KiB/s wr, 12 op/s 2023-09-10T05:20:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21671.030082 secs 2023-09-10T05:20:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:39 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:20:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:39 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:20:40.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:39 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:20:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:39 smithi170 ceph-mon[102832]: pgmap v11718: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 310 KiB/s wr, 12 op/s 2023-09-10T05:20:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21671.030082 secs 2023-09-10T05:20:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:39 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:20:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:39 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:20:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:39 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:20:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:41 smithi082 ceph-mon[98694]: pgmap v11719: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 502 KiB/s wr, 19 op/s 2023-09-10T05:20:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:41 smithi098 ceph-mon[103124]: pgmap v11719: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 502 KiB/s wr, 19 op/s 2023-09-10T05:20:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:41 smithi170 ceph-mon[102832]: pgmap v11719: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 502 KiB/s wr, 19 op/s 2023-09-10T05:20:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:42 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:20:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:42 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:20:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:42 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:20:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:43 smithi082 ceph-mon[98694]: pgmap v11720: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 377 KiB/s wr, 13 op/s 2023-09-10T05:20:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:20:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:20:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21675.376359 secs 2023-09-10T05:20:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:43 smithi098 ceph-mon[103124]: pgmap v11720: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 377 KiB/s wr, 13 op/s 2023-09-10T05:20:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:20:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:20:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21675.376359 secs 2023-09-10T05:20:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:43 smithi170 ceph-mon[102832]: pgmap v11720: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 377 KiB/s wr, 13 op/s 2023-09-10T05:20:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:20:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:20:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21675.376359 secs 2023-09-10T05:20:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21676.030146 secs 2023-09-10T05:20:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:44 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:20:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21676.030146 secs 2023-09-10T05:20:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:44 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:20:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21676.030146 secs 2023-09-10T05:20:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:44 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:20:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:45 smithi082 ceph-mon[98694]: pgmap v11721: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 377 KiB/s wr, 13 op/s 2023-09-10T05:20:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:20:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:20:46.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:45 smithi098 ceph-mon[103124]: pgmap v11721: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 377 KiB/s wr, 13 op/s 2023-09-10T05:20:46.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:20:46.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:20:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:45 smithi170 ceph-mon[102832]: pgmap v11721: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 377 KiB/s wr, 13 op/s 2023-09-10T05:20:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:20:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:20:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:47 smithi082 ceph-mon[98694]: pgmap v11722: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 475 KiB/s wr, 19 op/s 2023-09-10T05:20:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21679.336887 secs 2023-09-10T05:20:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:47 smithi098 ceph-mon[103124]: pgmap v11722: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 475 KiB/s wr, 19 op/s 2023-09-10T05:20:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21679.336887 secs 2023-09-10T05:20:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:47 smithi170 ceph-mon[102832]: pgmap v11722: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 475 KiB/s wr, 19 op/s 2023-09-10T05:20:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21679.336887 secs 2023-09-10T05:20:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21680.376419 secs 2023-09-10T05:20:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21680.376419 secs 2023-09-10T05:20:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21680.376419 secs 2023-09-10T05:20:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:49 smithi082 ceph-mon[98694]: pgmap v11723: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 290 KiB/s wr, 13 op/s 2023-09-10T05:20:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21681.030212 secs 2023-09-10T05:20:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:49 smithi098 ceph-mon[103124]: pgmap v11723: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 290 KiB/s wr, 13 op/s 2023-09-10T05:20:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21681.030212 secs 2023-09-10T05:20:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:49 smithi170 ceph-mon[102832]: pgmap v11723: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 290 KiB/s wr, 13 op/s 2023-09-10T05:20:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21681.030212 secs 2023-09-10T05:20:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:20:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:20:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:20:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:20:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:20:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:20:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:20:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:20:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:20:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:52 smithi170 ceph-mon[102832]: pgmap v11724: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 394 KiB/s wr, 19 op/s 2023-09-10T05:20:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:52 smithi082 ceph-mon[98694]: pgmap v11724: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 394 KiB/s wr, 19 op/s 2023-09-10T05:20:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:52 smithi098 ceph-mon[103124]: pgmap v11724: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 394 KiB/s wr, 19 op/s 2023-09-10T05:20:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21684.336997 secs 2023-09-10T05:20:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21684.336997 secs 2023-09-10T05:20:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21684.336997 secs 2023-09-10T05:20:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:54 smithi170 ceph-mon[102832]: pgmap v11725: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T05:20:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21685.376480 secs 2023-09-10T05:20:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21686.030279 secs 2023-09-10T05:20:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:54 smithi082 ceph-mon[98694]: pgmap v11725: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T05:20:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21685.376480 secs 2023-09-10T05:20:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21686.030279 secs 2023-09-10T05:20:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:54 smithi098 ceph-mon[103124]: pgmap v11725: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T05:20:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21685.376480 secs 2023-09-10T05:20:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21686.030279 secs 2023-09-10T05:20:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:55 smithi170 ceph-mon[102832]: pgmap v11726: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T05:20:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:55 smithi082 ceph-mon[98694]: pgmap v11726: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T05:20:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:55 smithi098 ceph-mon[103124]: pgmap v11726: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 201 KiB/s wr, 12 op/s 2023-09-10T05:20:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:57 smithi170 ceph-mon[102832]: pgmap v11727: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 278 KiB/s wr, 16 op/s 2023-09-10T05:20:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21689.337056 secs 2023-09-10T05:20:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:57 smithi082 ceph-mon[98694]: pgmap v11727: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 278 KiB/s wr, 16 op/s 2023-09-10T05:20:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21689.337056 secs 2023-09-10T05:20:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:57 smithi098 ceph-mon[103124]: pgmap v11727: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 278 KiB/s wr, 16 op/s 2023-09-10T05:20:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21689.337056 secs 2023-09-10T05:20:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21690.376540 secs 2023-09-10T05:20:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21690.376540 secs 2023-09-10T05:20:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21690.376540 secs 2023-09-10T05:21:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:59 smithi170 ceph-mon[102832]: pgmap v11728: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 10 op/s 2023-09-10T05:21:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:20:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21691.030346 secs 2023-09-10T05:21:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:59 smithi082 ceph-mon[98694]: pgmap v11728: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 10 op/s 2023-09-10T05:21:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:20:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21691.030346 secs 2023-09-10T05:21:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:59 smithi098 ceph-mon[103124]: pgmap v11728: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 10 op/s 2023-09-10T05:21:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:20:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21691.030346 secs 2023-09-10T05:21:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:01 smithi170 ceph-mon[102832]: pgmap v11729: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 14 op/s 2023-09-10T05:21:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:01 smithi082 ceph-mon[98694]: pgmap v11729: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 14 op/s 2023-09-10T05:21:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:01 smithi098 ceph-mon[103124]: pgmap v11729: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 14 op/s 2023-09-10T05:21:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21694.337118 secs 2023-09-10T05:21:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21694.337118 secs 2023-09-10T05:21:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21694.337118 secs 2023-09-10T05:21:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:03 smithi170 ceph-mon[102832]: pgmap v11730: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T05:21:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21695.376597 secs 2023-09-10T05:21:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:03 smithi082 ceph-mon[98694]: pgmap v11730: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T05:21:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21695.376597 secs 2023-09-10T05:21:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:03 smithi098 ceph-mon[103124]: pgmap v11730: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T05:21:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21695.376597 secs 2023-09-10T05:21:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21696.030410 secs 2023-09-10T05:21:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21696.030410 secs 2023-09-10T05:21:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21696.030410 secs 2023-09-10T05:21:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:05 smithi170 ceph-mon[102832]: pgmap v11731: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T05:21:06.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:05 smithi082 ceph-mon[98694]: pgmap v11731: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T05:21:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:05 smithi098 ceph-mon[103124]: pgmap v11731: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T05:21:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:07 smithi170 ceph-mon[102832]: pgmap v11732: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 176 KiB/s wr, 28 op/s 2023-09-10T05:21:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21699.337232 secs 2023-09-10T05:21:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:07 smithi082 ceph-mon[98694]: pgmap v11732: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 176 KiB/s wr, 28 op/s 2023-09-10T05:21:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21699.337232 secs 2023-09-10T05:21:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:07 smithi098 ceph-mon[103124]: pgmap v11732: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 176 KiB/s wr, 28 op/s 2023-09-10T05:21:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21699.337232 secs 2023-09-10T05:21:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:08 smithi170 ceph-mon[102832]: osdmap e801: 12 total, 12 up, 12 in 2023-09-10T05:21:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21700.376678 secs 2023-09-10T05:21:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:08 smithi082 ceph-mon[98694]: osdmap e801: 12 total, 12 up, 12 in 2023-09-10T05:21:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21700.376678 secs 2023-09-10T05:21:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:08 smithi098 ceph-mon[103124]: osdmap e801: 12 total, 12 up, 12 in 2023-09-10T05:21:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21700.376678 secs 2023-09-10T05:21:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:09 smithi170 ceph-mon[102832]: pgmap v11734: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 119 KiB/s wr, 27 op/s 2023-09-10T05:21:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21701.030480 secs 2023-09-10T05:21:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:09 smithi082 ceph-mon[98694]: pgmap v11734: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 119 KiB/s wr, 27 op/s 2023-09-10T05:21:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21701.030480 secs 2023-09-10T05:21:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:09 smithi098 ceph-mon[103124]: pgmap v11734: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 119 KiB/s wr, 27 op/s 2023-09-10T05:21:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21701.030480 secs 2023-09-10T05:21:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:11 smithi170 ceph-mon[102832]: pgmap v11735: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 119 KiB/s wr, 28 op/s 2023-09-10T05:21:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:11 smithi082 ceph-mon[98694]: pgmap v11735: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 119 KiB/s wr, 28 op/s 2023-09-10T05:21:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:11 smithi098 ceph-mon[103124]: pgmap v11735: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 119 KiB/s wr, 28 op/s 2023-09-10T05:21:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21704.337292 secs 2023-09-10T05:21:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21704.337292 secs 2023-09-10T05:21:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21704.337292 secs 2023-09-10T05:21:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:13 smithi170 ceph-mon[102832]: pgmap v11736: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 119 KiB/s wr, 28 op/s 2023-09-10T05:21:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21705.376763 secs 2023-09-10T05:21:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:13 smithi082 ceph-mon[98694]: pgmap v11736: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 119 KiB/s wr, 28 op/s 2023-09-10T05:21:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21705.376763 secs 2023-09-10T05:21:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:13 smithi098 ceph-mon[103124]: pgmap v11736: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 119 KiB/s wr, 28 op/s 2023-09-10T05:21:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21705.376763 secs 2023-09-10T05:21:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21706.030544 secs 2023-09-10T05:21:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:14 smithi170 ceph-mon[102832]: osdmap e802: 12 total, 12 up, 12 in 2023-09-10T05:21:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21706.030544 secs 2023-09-10T05:21:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:14 smithi082 ceph-mon[98694]: osdmap e802: 12 total, 12 up, 12 in 2023-09-10T05:21:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21706.030544 secs 2023-09-10T05:21:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:14 smithi098 ceph-mon[103124]: osdmap e802: 12 total, 12 up, 12 in 2023-09-10T05:21:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:15 smithi170 ceph-mon[102832]: pgmap v11738: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:21:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:15 smithi082 ceph-mon[98694]: pgmap v11738: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:21:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:15 smithi098 ceph-mon[103124]: pgmap v11738: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:21:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:17 smithi170 ceph-mon[102832]: pgmap v11739: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2023-09-10T05:21:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21709.337351 secs 2023-09-10T05:21:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:17 smithi082 ceph-mon[98694]: pgmap v11739: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2023-09-10T05:21:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21709.337351 secs 2023-09-10T05:21:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:17 smithi098 ceph-mon[103124]: pgmap v11739: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2023-09-10T05:21:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21709.337351 secs 2023-09-10T05:21:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21710.376824 secs 2023-09-10T05:21:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21710.376824 secs 2023-09-10T05:21:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21710.376824 secs 2023-09-10T05:21:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:19 smithi170 ceph-mon[102832]: pgmap v11740: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:21:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21711.030607 secs 2023-09-10T05:21:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:19 smithi082 ceph-mon[98694]: pgmap v11740: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:21:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21711.030607 secs 2023-09-10T05:21:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:19 smithi098 ceph-mon[103124]: pgmap v11740: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:21:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21711.030607 secs 2023-09-10T05:21:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:21 smithi170 ceph-mon[102832]: pgmap v11741: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:21:22.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:21 smithi082 ceph-mon[98694]: pgmap v11741: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:21:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:21 smithi098 ceph-mon[103124]: pgmap v11741: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:21:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21714.337436 secs 2023-09-10T05:21:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21714.337436 secs 2023-09-10T05:21:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21714.337436 secs 2023-09-10T05:21:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:23 smithi170 ceph-mon[102832]: pgmap v11742: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:21:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21715.376882 secs 2023-09-10T05:21:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:23 smithi082 ceph-mon[98694]: pgmap v11742: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:21:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21715.376882 secs 2023-09-10T05:21:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:23 smithi098 ceph-mon[103124]: pgmap v11742: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:21:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21715.376882 secs 2023-09-10T05:21:25.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21716.030671 secs 2023-09-10T05:21:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21716.030671 secs 2023-09-10T05:21:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21716.030671 secs 2023-09-10T05:21:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:25 smithi170 ceph-mon[102832]: pgmap v11743: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 203 B/s wr, 10 op/s 2023-09-10T05:21:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:25 smithi082 ceph-mon[98694]: pgmap v11743: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 203 B/s wr, 10 op/s 2023-09-10T05:21:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:25 smithi098 ceph-mon[103124]: pgmap v11743: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 203 B/s wr, 10 op/s 2023-09-10T05:21:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:27 smithi170 ceph-mon[102832]: pgmap v11744: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 28 KiB/s wr, 13 op/s 2023-09-10T05:21:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:21:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:27 smithi082 ceph-mon[98694]: pgmap v11744: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 28 KiB/s wr, 13 op/s 2023-09-10T05:21:28.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:21:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:27 smithi098 ceph-mon[103124]: pgmap v11744: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 28 KiB/s wr, 13 op/s 2023-09-10T05:21:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:21:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21720.376967 secs 2023-09-10T05:21:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:21:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:21:29.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:21:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21720.376967 secs 2023-09-10T05:21:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:21:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:21:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:21:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21720.376967 secs 2023-09-10T05:21:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:21:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:21:29.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:21:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:29 smithi170 ceph-mon[102832]: pgmap v11745: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 9 op/s 2023-09-10T05:21:30.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21721.030735 secs 2023-09-10T05:21:30.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:29 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:21:30.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:29 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:21:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:29 smithi082 ceph-mon[98694]: pgmap v11745: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 9 op/s 2023-09-10T05:21:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21721.030735 secs 2023-09-10T05:21:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:29 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:21:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:29 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:21:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:29 smithi098 ceph-mon[103124]: pgmap v11745: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 9 op/s 2023-09-10T05:21:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21721.030735 secs 2023-09-10T05:21:30.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:29 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:21:30.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:29 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:21:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:31 smithi170 ceph-mon[102832]: pgmap v11746: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 111 KiB/s wr, 15 op/s 2023-09-10T05:21:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:21:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:31 smithi082 ceph-mon[98694]: pgmap v11746: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 111 KiB/s wr, 15 op/s 2023-09-10T05:21:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:21:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:31 smithi098 ceph-mon[103124]: pgmap v11746: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 111 KiB/s wr, 15 op/s 2023-09-10T05:21:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:21:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:21:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:21:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:21:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:21:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:21:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:21:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:21:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:21:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:21:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:33 smithi170 ceph-mon[102832]: pgmap v11747: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T05:21:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21725.377044 secs 2023-09-10T05:21:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:33 smithi082 ceph-mon[98694]: pgmap v11747: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T05:21:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21725.377044 secs 2023-09-10T05:21:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:33 smithi098 ceph-mon[103124]: pgmap v11747: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T05:21:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21725.377044 secs 2023-09-10T05:21:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21726.030801 secs 2023-09-10T05:21:35.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21726.030801 secs 2023-09-10T05:21:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21726.030801 secs 2023-09-10T05:21:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:35 smithi170 ceph-mon[102832]: pgmap v11748: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T05:21:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:35 smithi082 ceph-mon[98694]: pgmap v11748: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T05:21:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:35 smithi098 ceph-mon[103124]: pgmap v11748: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T05:21:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:37 smithi170 ceph-mon[102832]: pgmap v11749: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 274 KiB/s wr, 17 op/s 2023-09-10T05:21:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:37 smithi082 ceph-mon[98694]: pgmap v11749: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 274 KiB/s wr, 17 op/s 2023-09-10T05:21:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:37 smithi098 ceph-mon[103124]: pgmap v11749: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 274 KiB/s wr, 17 op/s 2023-09-10T05:21:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21730.377116 secs 2023-09-10T05:21:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21730.377116 secs 2023-09-10T05:21:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21730.377116 secs 2023-09-10T05:21:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:39 smithi170 ceph-mon[102832]: pgmap v11750: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 247 KiB/s wr, 12 op/s 2023-09-10T05:21:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21731.030866 secs 2023-09-10T05:21:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:39 smithi082 ceph-mon[98694]: pgmap v11750: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 247 KiB/s wr, 12 op/s 2023-09-10T05:21:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21731.030866 secs 2023-09-10T05:21:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:39 smithi098 ceph-mon[103124]: pgmap v11750: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 247 KiB/s wr, 12 op/s 2023-09-10T05:21:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21731.030866 secs 2023-09-10T05:21:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:41 smithi170 ceph-mon[102832]: pgmap v11751: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 461 KiB/s wr, 19 op/s 2023-09-10T05:21:42.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:41 smithi082 ceph-mon[98694]: pgmap v11751: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 461 KiB/s wr, 19 op/s 2023-09-10T05:21:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:41 smithi098 ceph-mon[103124]: pgmap v11751: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 461 KiB/s wr, 19 op/s 2023-09-10T05:21:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21734.337719 secs 2023-09-10T05:21:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21734.337719 secs 2023-09-10T05:21:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21734.337719 secs 2023-09-10T05:21:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:43 smithi170 ceph-mon[102832]: pgmap v11752: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 378 KiB/s wr, 13 op/s 2023-09-10T05:21:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21735.377173 secs 2023-09-10T05:21:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:43 smithi082 ceph-mon[98694]: pgmap v11752: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 378 KiB/s wr, 13 op/s 2023-09-10T05:21:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21735.377173 secs 2023-09-10T05:21:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:43 smithi098 ceph-mon[103124]: pgmap v11752: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 378 KiB/s wr, 13 op/s 2023-09-10T05:21:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21735.377173 secs 2023-09-10T05:21:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21736.030932 secs 2023-09-10T05:21:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:21:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:21:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:21:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21736.030932 secs 2023-09-10T05:21:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:21:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:21:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:21:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21736.030932 secs 2023-09-10T05:21:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:21:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:21:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:21:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:45 smithi170 ceph-mon[102832]: pgmap v11753: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 378 KiB/s wr, 13 op/s 2023-09-10T05:21:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:21:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:21:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:45 smithi082 ceph-mon[98694]: pgmap v11753: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 378 KiB/s wr, 13 op/s 2023-09-10T05:21:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:21:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:21:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:45 smithi098 ceph-mon[103124]: pgmap v11753: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 378 KiB/s wr, 13 op/s 2023-09-10T05:21:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:21:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:21:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:47 smithi170 ceph-mon[102832]: pgmap v11754: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 476 KiB/s wr, 19 op/s 2023-09-10T05:21:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21739.337773 secs 2023-09-10T05:21:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:47 smithi082 ceph-mon[98694]: pgmap v11754: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 476 KiB/s wr, 19 op/s 2023-09-10T05:21:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21739.337773 secs 2023-09-10T05:21:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:47 smithi098 ceph-mon[103124]: pgmap v11754: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 476 KiB/s wr, 19 op/s 2023-09-10T05:21:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21739.337773 secs 2023-09-10T05:21:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21740.377231 secs 2023-09-10T05:21:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21740.377231 secs 2023-09-10T05:21:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21740.377231 secs 2023-09-10T05:21:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:49 smithi170 ceph-mon[102832]: pgmap v11755: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 313 KiB/s wr, 13 op/s 2023-09-10T05:21:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21741.030998 secs 2023-09-10T05:21:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:49 smithi082 ceph-mon[98694]: pgmap v11755: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 313 KiB/s wr, 13 op/s 2023-09-10T05:21:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21741.030998 secs 2023-09-10T05:21:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:49 smithi098 ceph-mon[103124]: pgmap v11755: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 313 KiB/s wr, 13 op/s 2023-09-10T05:21:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21741.030998 secs 2023-09-10T05:21:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:51 smithi170 ceph-mon[102832]: pgmap v11756: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 416 KiB/s wr, 19 op/s 2023-09-10T05:21:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:51 smithi082 ceph-mon[98694]: pgmap v11756: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 416 KiB/s wr, 19 op/s 2023-09-10T05:21:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:51 smithi098 ceph-mon[103124]: pgmap v11756: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 416 KiB/s wr, 19 op/s 2023-09-10T05:21:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21744.337833 secs 2023-09-10T05:21:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21744.337833 secs 2023-09-10T05:21:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21744.337833 secs 2023-09-10T05:21:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:53 smithi170 ceph-mon[102832]: pgmap v11757: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 202 KiB/s wr, 11 op/s 2023-09-10T05:21:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21745.377307 secs 2023-09-10T05:21:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:53 smithi082 ceph-mon[98694]: pgmap v11757: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 202 KiB/s wr, 11 op/s 2023-09-10T05:21:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21745.377307 secs 2023-09-10T05:21:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:53 smithi098 ceph-mon[103124]: pgmap v11757: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 202 KiB/s wr, 11 op/s 2023-09-10T05:21:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21745.377307 secs 2023-09-10T05:21:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21746.031062 secs 2023-09-10T05:21:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21746.031062 secs 2023-09-10T05:21:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21746.031062 secs 2023-09-10T05:21:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:55 smithi170 ceph-mon[102832]: pgmap v11758: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 202 KiB/s wr, 11 op/s 2023-09-10T05:21:56.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:55 smithi082 ceph-mon[98694]: pgmap v11758: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 202 KiB/s wr, 11 op/s 2023-09-10T05:21:56.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:55 smithi098 ceph-mon[103124]: pgmap v11758: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 202 KiB/s wr, 11 op/s 2023-09-10T05:21:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:57 smithi170 ceph-mon[102832]: pgmap v11759: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 279 KiB/s wr, 16 op/s 2023-09-10T05:21:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21749.337953 secs 2023-09-10T05:21:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:57 smithi082 ceph-mon[98694]: pgmap v11759: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 279 KiB/s wr, 16 op/s 2023-09-10T05:21:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21749.337953 secs 2023-09-10T05:21:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:57 smithi098 ceph-mon[103124]: pgmap v11759: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 279 KiB/s wr, 16 op/s 2023-09-10T05:21:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21749.337953 secs 2023-09-10T05:21:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21750.377396 secs 2023-09-10T05:21:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21750.377396 secs 2023-09-10T05:21:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21750.377396 secs 2023-09-10T05:22:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:59 smithi170 ceph-mon[102832]: pgmap v11760: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 180 KiB/s wr, 10 op/s 2023-09-10T05:22:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:21:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21751.031127 secs 2023-09-10T05:22:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:59 smithi082 ceph-mon[98694]: pgmap v11760: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 180 KiB/s wr, 10 op/s 2023-09-10T05:22:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:21:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21751.031127 secs 2023-09-10T05:22:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:59 smithi098 ceph-mon[103124]: pgmap v11760: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 180 KiB/s wr, 10 op/s 2023-09-10T05:22:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:21:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21751.031127 secs 2023-09-10T05:22:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:01 smithi082 ceph-mon[98694]: pgmap v11761: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 180 KiB/s wr, 14 op/s 2023-09-10T05:22:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:01 smithi098 ceph-mon[103124]: pgmap v11761: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 180 KiB/s wr, 14 op/s 2023-09-10T05:22:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:01 smithi170 ceph-mon[102832]: pgmap v11761: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 180 KiB/s wr, 14 op/s 2023-09-10T05:22:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21754.338013 secs 2023-09-10T05:22:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21754.338013 secs 2023-09-10T05:22:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21754.338013 secs 2023-09-10T05:22:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:03 smithi082 ceph-mon[98694]: pgmap v11762: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 9 op/s 2023-09-10T05:22:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21755.377453 secs 2023-09-10T05:22:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:03 smithi098 ceph-mon[103124]: pgmap v11762: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 9 op/s 2023-09-10T05:22:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21755.377453 secs 2023-09-10T05:22:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:03 smithi170 ceph-mon[102832]: pgmap v11762: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 9 op/s 2023-09-10T05:22:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21755.377453 secs 2023-09-10T05:22:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21756.031215 secs 2023-09-10T05:22:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21756.031215 secs 2023-09-10T05:22:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21756.031215 secs 2023-09-10T05:22:06.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:05 smithi082 ceph-mon[98694]: pgmap v11763: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 9 op/s 2023-09-10T05:22:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:05 smithi098 ceph-mon[103124]: pgmap v11763: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 9 op/s 2023-09-10T05:22:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:05 smithi170 ceph-mon[102832]: pgmap v11763: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 9 op/s 2023-09-10T05:22:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:07 smithi082 ceph-mon[98694]: pgmap v11764: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 176 KiB/s wr, 28 op/s 2023-09-10T05:22:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21759.338076 secs 2023-09-10T05:22:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:07 smithi098 ceph-mon[103124]: pgmap v11764: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 176 KiB/s wr, 28 op/s 2023-09-10T05:22:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21759.338076 secs 2023-09-10T05:22:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:07 smithi170 ceph-mon[102832]: pgmap v11764: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 176 KiB/s wr, 28 op/s 2023-09-10T05:22:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21759.338076 secs 2023-09-10T05:22:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:08 smithi098 ceph-mon[103124]: osdmap e803: 12 total, 12 up, 12 in 2023-09-10T05:22:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21760.377495 secs 2023-09-10T05:22:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:08 smithi170 ceph-mon[102832]: osdmap e803: 12 total, 12 up, 12 in 2023-09-10T05:22:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21760.377495 secs 2023-09-10T05:22:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:08 smithi082 ceph-mon[98694]: osdmap e803: 12 total, 12 up, 12 in 2023-09-10T05:22:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21760.377495 secs 2023-09-10T05:22:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:09 smithi098 ceph-mon[103124]: pgmap v11766: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 120 KiB/s wr, 28 op/s 2023-09-10T05:22:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21761.031281 secs 2023-09-10T05:22:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:09 smithi170 ceph-mon[102832]: pgmap v11766: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 120 KiB/s wr, 28 op/s 2023-09-10T05:22:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21761.031281 secs 2023-09-10T05:22:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:09 smithi082 ceph-mon[98694]: pgmap v11766: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 120 KiB/s wr, 28 op/s 2023-09-10T05:22:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21761.031281 secs 2023-09-10T05:22:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:11 smithi098 ceph-mon[103124]: pgmap v11767: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 120 KiB/s wr, 29 op/s 2023-09-10T05:22:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:11 smithi170 ceph-mon[102832]: pgmap v11767: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 120 KiB/s wr, 29 op/s 2023-09-10T05:22:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:11 smithi082 ceph-mon[98694]: pgmap v11767: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 120 KiB/s wr, 29 op/s 2023-09-10T05:22:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21764.338135 secs 2023-09-10T05:22:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21764.338135 secs 2023-09-10T05:22:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21764.338135 secs 2023-09-10T05:22:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:13 smithi098 ceph-mon[103124]: pgmap v11768: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 120 KiB/s wr, 29 op/s 2023-09-10T05:22:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21765.377552 secs 2023-09-10T05:22:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:13 smithi170 ceph-mon[102832]: pgmap v11768: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 120 KiB/s wr, 29 op/s 2023-09-10T05:22:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21765.377552 secs 2023-09-10T05:22:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:13 smithi082 ceph-mon[98694]: pgmap v11768: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 120 KiB/s wr, 29 op/s 2023-09-10T05:22:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21765.377552 secs 2023-09-10T05:22:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21766.031344 secs 2023-09-10T05:22:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:14 smithi098 ceph-mon[103124]: osdmap e804: 12 total, 12 up, 12 in 2023-09-10T05:22:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21766.031344 secs 2023-09-10T05:22:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:14 smithi170 ceph-mon[102832]: osdmap e804: 12 total, 12 up, 12 in 2023-09-10T05:22:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21766.031344 secs 2023-09-10T05:22:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:14 smithi082 ceph-mon[98694]: osdmap e804: 12 total, 12 up, 12 in 2023-09-10T05:22:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:15 smithi098 ceph-mon[103124]: pgmap v11770: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T05:22:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:15 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:22:16.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:15 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:22:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:15 smithi170 ceph-mon[102832]: pgmap v11770: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T05:22:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:15 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:22:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:15 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:22:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:15 smithi082 ceph-mon[98694]: pgmap v11770: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T05:22:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:15 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:22:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:15 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:22:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:17 smithi098 ceph-mon[103124]: pgmap v11771: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 232 B/s wr, 13 op/s 2023-09-10T05:22:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:17 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:22:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:17 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:22:18.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21769.338195 secs 2023-09-10T05:22:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:17 smithi170 ceph-mon[102832]: pgmap v11771: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 232 B/s wr, 13 op/s 2023-09-10T05:22:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:17 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:22:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:17 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:22:18.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21769.338195 secs 2023-09-10T05:22:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:17 smithi082 ceph-mon[98694]: pgmap v11771: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 232 B/s wr, 13 op/s 2023-09-10T05:22:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:17 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:22:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:17 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:22:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21769.338195 secs 2023-09-10T05:22:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21770.377623 secs 2023-09-10T05:22:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21770.377623 secs 2023-09-10T05:22:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21770.377623 secs 2023-09-10T05:22:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:19 smithi098 ceph-mon[103124]: pgmap v11772: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T05:22:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21771.031410 secs 2023-09-10T05:22:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:19 smithi170 ceph-mon[102832]: pgmap v11772: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T05:22:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21771.031410 secs 2023-09-10T05:22:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:19 smithi082 ceph-mon[98694]: pgmap v11772: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T05:22:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21771.031410 secs 2023-09-10T05:22:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:21 smithi098 ceph-mon[103124]: pgmap v11773: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T05:22:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:21 smithi170 ceph-mon[102832]: pgmap v11773: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T05:22:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:21 smithi082 ceph-mon[98694]: pgmap v11773: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T05:22:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21774.338274 secs 2023-09-10T05:22:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21774.338274 secs 2023-09-10T05:22:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21774.338274 secs 2023-09-10T05:22:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:23 smithi098 ceph-mon[103124]: pgmap v11774: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T05:22:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21775.377682 secs 2023-09-10T05:22:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:23 smithi170 ceph-mon[102832]: pgmap v11774: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T05:22:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21775.377682 secs 2023-09-10T05:22:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:23 smithi082 ceph-mon[98694]: pgmap v11774: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T05:22:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21775.377682 secs 2023-09-10T05:22:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21776.031481 secs 2023-09-10T05:22:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21776.031481 secs 2023-09-10T05:22:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21776.031481 secs 2023-09-10T05:22:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:25 smithi098 ceph-mon[103124]: pgmap v11775: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 203 B/s wr, 11 op/s 2023-09-10T05:22:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:25 smithi170 ceph-mon[102832]: pgmap v11775: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 203 B/s wr, 11 op/s 2023-09-10T05:22:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:25 smithi082 ceph-mon[98694]: pgmap v11775: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 203 B/s wr, 11 op/s 2023-09-10T05:22:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:27 smithi098 ceph-mon[103124]: pgmap v11776: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 14 op/s 2023-09-10T05:22:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:27 smithi170 ceph-mon[102832]: pgmap v11776: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 14 op/s 2023-09-10T05:22:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:27 smithi082 ceph-mon[98694]: pgmap v11776: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 14 op/s 2023-09-10T05:22:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21780.377739 secs 2023-09-10T05:22:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:22:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21780.377739 secs 2023-09-10T05:22:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:22:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21780.377739 secs 2023-09-10T05:22:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:22:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:30 smithi170 ceph-mon[102832]: pgmap v11777: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-10T05:22:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21781.031544 secs 2023-09-10T05:22:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:30 smithi082 ceph-mon[98694]: pgmap v11777: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-10T05:22:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21781.031544 secs 2023-09-10T05:22:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:30 smithi098 ceph-mon[103124]: pgmap v11777: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-10T05:22:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21781.031544 secs 2023-09-10T05:22:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:22:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:22:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:22:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:32 smithi170 ceph-mon[102832]: pgmap v11778: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 113 KiB/s wr, 15 op/s 2023-09-10T05:22:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:22:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:22:32.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:22:32.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:32.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:32.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:32.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:32.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:32.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:32.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:32.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:32.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:22:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:32 smithi082 ceph-mon[98694]: pgmap v11778: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 113 KiB/s wr, 15 op/s 2023-09-10T05:22:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:22:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:22:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:22:32.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:32.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:32.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:32.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:32.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:32.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:32.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:32.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:32.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:22:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:32 smithi098 ceph-mon[103124]: pgmap v11778: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 113 KiB/s wr, 15 op/s 2023-09-10T05:22:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:22:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:22:32.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:22:32.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:32.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:32.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:32.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:32.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:32.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:32.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:32.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:32.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:22:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:22:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:22:33.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:22:33.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:22:33.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:33 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T05:22:33.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:33 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T05:22:33.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:33.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:33.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:22:33.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:22:33.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:22:33.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:22:33.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:33 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T05:22:33.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:33 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T05:22:33.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:22:33.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:22:33.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:33.462 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:33 smithi082 ceph-mon[98694]: pgmap v11779: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T05:22:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:33.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:22:33.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:22:33.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:22:33.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:22:33.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:33 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T05:22:33.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:33 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T05:22:33.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:33.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:33.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:22:33.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:22:33.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:22:33.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:22:33.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:33 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T05:22:33.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:33 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T05:22:33.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:22:33.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:22:33.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:33.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:33 smithi098 ceph-mon[103124]: pgmap v11779: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T05:22:33.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:33.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:33.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:22:33.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:22:33.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:22:33.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:22:33.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:33 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T05:22:33.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:33 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T05:22:33.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:33.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:33.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:22:33.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:22:33.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:22:33.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:22:33.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:33 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T05:22:33.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:33 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T05:22:33.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:22:33.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:22:33.682 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:22:33.682 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:33 smithi170 ceph-mon[102832]: pgmap v11779: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T05:22:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21785.377797 secs 2023-09-10T05:22:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21786.031609 secs 2023-09-10T05:22:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21785.377797 secs 2023-09-10T05:22:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21786.031609 secs 2023-09-10T05:22:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21785.377797 secs 2023-09-10T05:22:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21786.031609 secs 2023-09-10T05:22:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:35 smithi170 ceph-mon[102832]: pgmap v11780: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T05:22:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:35 smithi082 ceph-mon[98694]: pgmap v11780: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T05:22:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:35 smithi098 ceph-mon[103124]: pgmap v11780: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T05:22:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:22:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:22:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:37 smithi082 ceph-mon[98694]: pgmap v11781: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 278 KiB/s wr, 16 op/s 2023-09-10T05:22:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:22:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:22:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:37 smithi098 ceph-mon[103124]: pgmap v11781: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 278 KiB/s wr, 16 op/s 2023-09-10T05:22:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:22:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:22:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:37 smithi170 ceph-mon[102832]: pgmap v11781: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 278 KiB/s wr, 16 op/s 2023-09-10T05:22:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21790.377881 secs 2023-09-10T05:22:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21790.377881 secs 2023-09-10T05:22:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21790.377881 secs 2023-09-10T05:22:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:39 smithi082 ceph-mon[98694]: pgmap v11782: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 11 op/s 2023-09-10T05:22:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21791.031679 secs 2023-09-10T05:22:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:39 smithi098 ceph-mon[103124]: pgmap v11782: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 11 op/s 2023-09-10T05:22:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21791.031679 secs 2023-09-10T05:22:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:39 smithi170 ceph-mon[102832]: pgmap v11782: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 11 op/s 2023-09-10T05:22:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21791.031679 secs 2023-09-10T05:22:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:22:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:22:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:22:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:22:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:22:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:22:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:41 smithi082 ceph-mon[98694]: pgmap v11783: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 493 KiB/s wr, 18 op/s 2023-09-10T05:22:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:41 smithi098 ceph-mon[103124]: pgmap v11783: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 493 KiB/s wr, 18 op/s 2023-09-10T05:22:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:41 smithi170 ceph-mon[102832]: pgmap v11783: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 493 KiB/s wr, 18 op/s 2023-09-10T05:22:42.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21794.338628 secs 2023-09-10T05:22:43.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21794.338628 secs 2023-09-10T05:22:43.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21794.338628 secs 2023-09-10T05:22:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:43 smithi082 ceph-mon[98694]: pgmap v11784: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 380 KiB/s wr, 13 op/s 2023-09-10T05:22:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21795.377939 secs 2023-09-10T05:22:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:43 smithi098 ceph-mon[103124]: pgmap v11784: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 380 KiB/s wr, 13 op/s 2023-09-10T05:22:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21795.377939 secs 2023-09-10T05:22:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:43 smithi170 ceph-mon[102832]: pgmap v11784: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 380 KiB/s wr, 13 op/s 2023-09-10T05:22:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21795.377939 secs 2023-09-10T05:22:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:22:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:22:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:22:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21796.031747 secs 2023-09-10T05:22:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:22:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:22:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:22:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21796.031747 secs 2023-09-10T05:22:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:22:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:22:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:22:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21796.031747 secs 2023-09-10T05:22:45.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:45 smithi082 ceph-mon[98694]: pgmap v11785: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 380 KiB/s wr, 13 op/s 2023-09-10T05:22:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:45 smithi098 ceph-mon[103124]: pgmap v11785: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 380 KiB/s wr, 13 op/s 2023-09-10T05:22:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:45 smithi170 ceph-mon[102832]: pgmap v11785: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 380 KiB/s wr, 13 op/s 2023-09-10T05:22:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:47 smithi170 ceph-mon[102832]: pgmap v11786: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 479 KiB/s wr, 19 op/s 2023-09-10T05:22:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21799.338689 secs 2023-09-10T05:22:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:47 smithi082 ceph-mon[98694]: pgmap v11786: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 479 KiB/s wr, 19 op/s 2023-09-10T05:22:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21799.338689 secs 2023-09-10T05:22:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:47 smithi098 ceph-mon[103124]: pgmap v11786: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 479 KiB/s wr, 19 op/s 2023-09-10T05:22:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21799.338689 secs 2023-09-10T05:22:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21800.378000 secs 2023-09-10T05:22:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21800.378000 secs 2023-09-10T05:22:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21800.378000 secs 2023-09-10T05:22:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:49 smithi170 ceph-mon[102832]: pgmap v11787: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 315 KiB/s wr, 13 op/s 2023-09-10T05:22:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21801.031813 secs 2023-09-10T05:22:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:49 smithi082 ceph-mon[98694]: pgmap v11787: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 315 KiB/s wr, 13 op/s 2023-09-10T05:22:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21801.031813 secs 2023-09-10T05:22:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:49 smithi098 ceph-mon[103124]: pgmap v11787: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 315 KiB/s wr, 13 op/s 2023-09-10T05:22:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21801.031813 secs 2023-09-10T05:22:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:51 smithi170 ceph-mon[102832]: pgmap v11788: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 419 KiB/s wr, 19 op/s 2023-09-10T05:22:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:51 smithi082 ceph-mon[98694]: pgmap v11788: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 419 KiB/s wr, 19 op/s 2023-09-10T05:22:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:51 smithi098 ceph-mon[103124]: pgmap v11788: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 419 KiB/s wr, 19 op/s 2023-09-10T05:22:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21804.338820 secs 2023-09-10T05:22:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21804.338820 secs 2023-09-10T05:22:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21804.338820 secs 2023-09-10T05:22:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:53 smithi170 ceph-mon[102832]: pgmap v11789: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 203 KiB/s wr, 11 op/s 2023-09-10T05:22:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21805.378083 secs 2023-09-10T05:22:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:53 smithi082 ceph-mon[98694]: pgmap v11789: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 203 KiB/s wr, 11 op/s 2023-09-10T05:22:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21805.378083 secs 2023-09-10T05:22:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:53 smithi098 ceph-mon[103124]: pgmap v11789: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 203 KiB/s wr, 11 op/s 2023-09-10T05:22:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21805.378083 secs 2023-09-10T05:22:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21806.031878 secs 2023-09-10T05:22:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21806.031878 secs 2023-09-10T05:22:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21806.031878 secs 2023-09-10T05:22:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:55 smithi170 ceph-mon[102832]: pgmap v11790: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 203 KiB/s wr, 11 op/s 2023-09-10T05:22:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:55 smithi082 ceph-mon[98694]: pgmap v11790: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 203 KiB/s wr, 11 op/s 2023-09-10T05:22:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:55 smithi098 ceph-mon[103124]: pgmap v11790: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 203 KiB/s wr, 11 op/s 2023-09-10T05:22:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:57 smithi170 ceph-mon[102832]: pgmap v11791: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 280 KiB/s wr, 17 op/s 2023-09-10T05:22:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:22:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:22:58.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21809.338880 secs 2023-09-10T05:22:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:57 smithi082 ceph-mon[98694]: pgmap v11791: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 280 KiB/s wr, 17 op/s 2023-09-10T05:22:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:22:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:22:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21809.338880 secs 2023-09-10T05:22:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:57 smithi098 ceph-mon[103124]: pgmap v11791: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 280 KiB/s wr, 17 op/s 2023-09-10T05:22:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:22:58.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:22:58.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21809.338880 secs 2023-09-10T05:22:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21810.378142 secs 2023-09-10T05:22:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21810.378142 secs 2023-09-10T05:22:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21810.378142 secs 2023-09-10T05:23:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:59 smithi170 ceph-mon[102832]: pgmap v11792: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 181 KiB/s wr, 10 op/s 2023-09-10T05:23:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:22:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21811.031943 secs 2023-09-10T05:23:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:59 smithi082 ceph-mon[98694]: pgmap v11792: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 181 KiB/s wr, 10 op/s 2023-09-10T05:23:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:22:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21811.031943 secs 2023-09-10T05:23:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:59 smithi098 ceph-mon[103124]: pgmap v11792: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 181 KiB/s wr, 10 op/s 2023-09-10T05:23:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:22:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21811.031943 secs 2023-09-10T05:23:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:23:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:23:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:23:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:23:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:23:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:23:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:01 smithi170 ceph-mon[102832]: pgmap v11793: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 181 KiB/s wr, 15 op/s 2023-09-10T05:23:02.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:01 smithi082 ceph-mon[98694]: pgmap v11793: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 181 KiB/s wr, 15 op/s 2023-09-10T05:23:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:01 smithi098 ceph-mon[103124]: pgmap v11793: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 181 KiB/s wr, 15 op/s 2023-09-10T05:23:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21814.339023 secs 2023-09-10T05:23:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21814.339023 secs 2023-09-10T05:23:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21814.339023 secs 2023-09-10T05:23:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:03 smithi170 ceph-mon[102832]: pgmap v11794: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 9 op/s 2023-09-10T05:23:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21815.378201 secs 2023-09-10T05:23:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:03 smithi082 ceph-mon[98694]: pgmap v11794: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 9 op/s 2023-09-10T05:23:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21815.378201 secs 2023-09-10T05:23:04.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:03 smithi098 ceph-mon[103124]: pgmap v11794: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 9 op/s 2023-09-10T05:23:04.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21815.378201 secs 2023-09-10T05:23:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21816.032007 secs 2023-09-10T05:23:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21816.032007 secs 2023-09-10T05:23:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21816.032007 secs 2023-09-10T05:23:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:05 smithi170 ceph-mon[102832]: pgmap v11795: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 9 op/s 2023-09-10T05:23:06.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:05 smithi082 ceph-mon[98694]: pgmap v11795: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 9 op/s 2023-09-10T05:23:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:05 smithi098 ceph-mon[103124]: pgmap v11795: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 9 op/s 2023-09-10T05:23:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:07 smithi170 ceph-mon[102832]: pgmap v11796: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 177 KiB/s wr, 29 op/s 2023-09-10T05:23:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21819.339081 secs 2023-09-10T05:23:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:07 smithi082 ceph-mon[98694]: pgmap v11796: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 177 KiB/s wr, 29 op/s 2023-09-10T05:23:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21819.339081 secs 2023-09-10T05:23:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:07 smithi098 ceph-mon[103124]: pgmap v11796: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 177 KiB/s wr, 29 op/s 2023-09-10T05:23:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21819.339081 secs 2023-09-10T05:23:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:08 smithi170 ceph-mon[102832]: osdmap e805: 12 total, 12 up, 12 in 2023-09-10T05:23:09.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21820.378294 secs 2023-09-10T05:23:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:08 smithi082 ceph-mon[98694]: osdmap e805: 12 total, 12 up, 12 in 2023-09-10T05:23:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21820.378294 secs 2023-09-10T05:23:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:08 smithi098 ceph-mon[103124]: osdmap e805: 12 total, 12 up, 12 in 2023-09-10T05:23:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21820.378294 secs 2023-09-10T05:23:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:09 smithi170 ceph-mon[102832]: pgmap v11798: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 120 KiB/s wr, 28 op/s 2023-09-10T05:23:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21821.032088 secs 2023-09-10T05:23:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:09 smithi082 ceph-mon[98694]: pgmap v11798: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 120 KiB/s wr, 28 op/s 2023-09-10T05:23:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21821.032088 secs 2023-09-10T05:23:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:09 smithi098 ceph-mon[103124]: pgmap v11798: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 120 KiB/s wr, 28 op/s 2023-09-10T05:23:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21821.032088 secs 2023-09-10T05:23:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:11 smithi170 ceph-mon[102832]: pgmap v11799: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 120 KiB/s wr, 28 op/s 2023-09-10T05:23:12.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:11 smithi082 ceph-mon[98694]: pgmap v11799: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 120 KiB/s wr, 28 op/s 2023-09-10T05:23:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:11 smithi098 ceph-mon[103124]: pgmap v11799: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 120 KiB/s wr, 28 op/s 2023-09-10T05:23:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21824.339142 secs 2023-09-10T05:23:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21824.339142 secs 2023-09-10T05:23:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21824.339142 secs 2023-09-10T05:23:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:13 smithi170 ceph-mon[102832]: pgmap v11800: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 120 KiB/s wr, 28 op/s 2023-09-10T05:23:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21825.378350 secs 2023-09-10T05:23:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:13 smithi082 ceph-mon[98694]: pgmap v11800: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 120 KiB/s wr, 28 op/s 2023-09-10T05:23:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21825.378350 secs 2023-09-10T05:23:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:13 smithi098 ceph-mon[103124]: pgmap v11800: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 120 KiB/s wr, 28 op/s 2023-09-10T05:23:14.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21825.378350 secs 2023-09-10T05:23:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21826.032152 secs 2023-09-10T05:23:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:14 smithi170 ceph-mon[102832]: osdmap e806: 12 total, 12 up, 12 in 2023-09-10T05:23:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21826.032152 secs 2023-09-10T05:23:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:14 smithi082 ceph-mon[98694]: osdmap e806: 12 total, 12 up, 12 in 2023-09-10T05:23:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21826.032152 secs 2023-09-10T05:23:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:14 smithi098 ceph-mon[103124]: osdmap e806: 12 total, 12 up, 12 in 2023-09-10T05:23:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:15 smithi170 ceph-mon[102832]: pgmap v11802: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:23:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:15 smithi082 ceph-mon[98694]: pgmap v11802: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:23:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:15 smithi098 ceph-mon[103124]: pgmap v11802: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:23:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:17 smithi170 ceph-mon[102832]: pgmap v11803: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 345 B/s wr, 12 op/s 2023-09-10T05:23:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21829.339236 secs 2023-09-10T05:23:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:17 smithi082 ceph-mon[98694]: pgmap v11803: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 345 B/s wr, 12 op/s 2023-09-10T05:23:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21829.339236 secs 2023-09-10T05:23:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:17 smithi098 ceph-mon[103124]: pgmap v11803: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 345 B/s wr, 12 op/s 2023-09-10T05:23:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21829.339236 secs 2023-09-10T05:23:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21830.378409 secs 2023-09-10T05:23:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21830.378409 secs 2023-09-10T05:23:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21830.378409 secs 2023-09-10T05:23:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:19 smithi170 ceph-mon[102832]: pgmap v11804: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T05:23:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21831.032217 secs 2023-09-10T05:23:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:19 smithi082 ceph-mon[98694]: pgmap v11804: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T05:23:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21831.032217 secs 2023-09-10T05:23:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:19 smithi098 ceph-mon[103124]: pgmap v11804: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T05:23:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21831.032217 secs 2023-09-10T05:23:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:21 smithi170 ceph-mon[102832]: pgmap v11805: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:23:22.222 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:21 smithi082 ceph-mon[98694]: pgmap v11805: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:23:22.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:21 smithi098 ceph-mon[103124]: pgmap v11805: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:23:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21834.339297 secs 2023-09-10T05:23:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21834.339297 secs 2023-09-10T05:23:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21834.339297 secs 2023-09-10T05:23:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:23 smithi170 ceph-mon[102832]: pgmap v11806: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:23:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21835.378467 secs 2023-09-10T05:23:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:23 smithi082 ceph-mon[98694]: pgmap v11806: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:23:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21835.378467 secs 2023-09-10T05:23:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:23 smithi098 ceph-mon[103124]: pgmap v11806: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:23:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21835.378467 secs 2023-09-10T05:23:25.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21836.032285 secs 2023-09-10T05:23:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21836.032285 secs 2023-09-10T05:23:25.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21836.032285 secs 2023-09-10T05:23:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:25 smithi170 ceph-mon[102832]: pgmap v11807: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T05:23:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:25 smithi082 ceph-mon[98694]: pgmap v11807: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T05:23:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:25 smithi098 ceph-mon[103124]: pgmap v11807: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T05:23:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:27 smithi170 ceph-mon[102832]: pgmap v11808: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 139 KiB/s wr, 14 op/s 2023-09-10T05:23:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:27 smithi082 ceph-mon[98694]: pgmap v11808: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 139 KiB/s wr, 14 op/s 2023-09-10T05:23:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:27 smithi098 ceph-mon[103124]: pgmap v11808: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 139 KiB/s wr, 14 op/s 2023-09-10T05:23:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21840.378494 secs 2023-09-10T05:23:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21840.378494 secs 2023-09-10T05:23:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21840.378494 secs 2023-09-10T05:23:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:29 smithi170 ceph-mon[102832]: pgmap v11809: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 9 op/s 2023-09-10T05:23:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21841.032348 secs 2023-09-10T05:23:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:29 smithi082 ceph-mon[98694]: pgmap v11809: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 9 op/s 2023-09-10T05:23:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21841.032348 secs 2023-09-10T05:23:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:29 smithi098 ceph-mon[103124]: pgmap v11809: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 9 op/s 2023-09-10T05:23:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21841.032348 secs 2023-09-10T05:23:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:30 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:23:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:30 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:23:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:30 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:23:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:31 smithi170 ceph-mon[102832]: pgmap v11810: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 260 KiB/s wr, 14 op/s 2023-09-10T05:23:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:31 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:23:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:23:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:23:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:31 smithi082 ceph-mon[98694]: pgmap v11810: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 260 KiB/s wr, 14 op/s 2023-09-10T05:23:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:31 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:23:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:23:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:23:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:31 smithi098 ceph-mon[103124]: pgmap v11810: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 260 KiB/s wr, 14 op/s 2023-09-10T05:23:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:31 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:23:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:23:32.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:23:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:23:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:23:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:23:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:33 smithi082 ceph-mon[98694]: pgmap v11811: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 193 KiB/s wr, 9 op/s 2023-09-10T05:23:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21845.378558 secs 2023-09-10T05:23:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:23:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:23:34.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:23:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:33 smithi098 ceph-mon[103124]: pgmap v11811: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 193 KiB/s wr, 9 op/s 2023-09-10T05:23:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21845.378558 secs 2023-09-10T05:23:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:23:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:23:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:23:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:33 smithi170 ceph-mon[102832]: pgmap v11811: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 193 KiB/s wr, 9 op/s 2023-09-10T05:23:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21845.378558 secs 2023-09-10T05:23:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:23:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:23:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:23:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21846.032413 secs 2023-09-10T05:23:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21846.032413 secs 2023-09-10T05:23:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21846.032413 secs 2023-09-10T05:23:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:35 smithi082 ceph-mon[98694]: pgmap v11812: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 193 KiB/s wr, 9 op/s 2023-09-10T05:23:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:35 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:23:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:35 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:23:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:35 smithi098 ceph-mon[103124]: pgmap v11812: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 193 KiB/s wr, 9 op/s 2023-09-10T05:23:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:35 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:23:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:35 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:23:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:35 smithi170 ceph-mon[102832]: pgmap v11812: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 193 KiB/s wr, 9 op/s 2023-09-10T05:23:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:35 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:23:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:35 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:23:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:37 smithi082 ceph-mon[98694]: pgmap v11813: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 366 KiB/s wr, 15 op/s 2023-09-10T05:23:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:23:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:23:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:37 smithi098 ceph-mon[103124]: pgmap v11813: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 366 KiB/s wr, 15 op/s 2023-09-10T05:23:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:23:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:23:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:37 smithi170 ceph-mon[102832]: pgmap v11813: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 366 KiB/s wr, 15 op/s 2023-09-10T05:23:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:23:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:23:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21850.378615 secs 2023-09-10T05:23:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21850.378615 secs 2023-09-10T05:23:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21850.378615 secs 2023-09-10T05:23:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:39 smithi082 ceph-mon[98694]: pgmap v11814: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 294 KiB/s wr, 10 op/s 2023-09-10T05:23:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21851.032480 secs 2023-09-10T05:23:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:39 smithi098 ceph-mon[103124]: pgmap v11814: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 294 KiB/s wr, 10 op/s 2023-09-10T05:23:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21851.032480 secs 2023-09-10T05:23:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:39 smithi170 ceph-mon[102832]: pgmap v11814: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 294 KiB/s wr, 10 op/s 2023-09-10T05:23:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21851.032480 secs 2023-09-10T05:23:42.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:41 smithi082 ceph-mon[98694]: pgmap v11815: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 407 KiB/s wr, 17 op/s 2023-09-10T05:23:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:41 smithi098 ceph-mon[103124]: pgmap v11815: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 407 KiB/s wr, 17 op/s 2023-09-10T05:23:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:41 smithi170 ceph-mon[102832]: pgmap v11815: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 407 KiB/s wr, 17 op/s 2023-09-10T05:23:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21854.339572 secs 2023-09-10T05:23:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21854.339572 secs 2023-09-10T05:23:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21854.339572 secs 2023-09-10T05:23:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:43 smithi098 ceph-mon[103124]: pgmap v11816: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T05:23:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:23:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:23:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:23:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21855.378675 secs 2023-09-10T05:23:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:43 smithi170 ceph-mon[102832]: pgmap v11816: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T05:23:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:23:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:23:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:23:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21855.378675 secs 2023-09-10T05:23:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:43 smithi082 ceph-mon[98694]: pgmap v11816: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T05:23:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:23:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:23:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:23:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21855.378675 secs 2023-09-10T05:23:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21856.032545 secs 2023-09-10T05:23:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21856.032545 secs 2023-09-10T05:23:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21856.032545 secs 2023-09-10T05:23:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:45 smithi098 ceph-mon[103124]: pgmap v11817: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T05:23:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:45 smithi170 ceph-mon[102832]: pgmap v11817: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T05:23:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:45 smithi082 ceph-mon[98694]: pgmap v11817: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T05:23:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:47 smithi098 ceph-mon[103124]: pgmap v11818: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 478 KiB/s wr, 20 op/s 2023-09-10T05:23:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21859.339632 secs 2023-09-10T05:23:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:47 smithi170 ceph-mon[102832]: pgmap v11818: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 478 KiB/s wr, 20 op/s 2023-09-10T05:23:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21859.339632 secs 2023-09-10T05:23:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:47 smithi082 ceph-mon[98694]: pgmap v11818: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 478 KiB/s wr, 20 op/s 2023-09-10T05:23:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21859.339632 secs 2023-09-10T05:23:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21860.378734 secs 2023-09-10T05:23:49.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21860.378734 secs 2023-09-10T05:23:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21860.378734 secs 2023-09-10T05:23:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:49 smithi098 ceph-mon[103124]: pgmap v11819: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 305 KiB/s wr, 14 op/s 2023-09-10T05:23:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21861.032609 secs 2023-09-10T05:23:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:23:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:23:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:49 smithi170 ceph-mon[102832]: pgmap v11819: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 305 KiB/s wr, 14 op/s 2023-09-10T05:23:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21861.032609 secs 2023-09-10T05:23:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:23:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:23:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:49 smithi082 ceph-mon[98694]: pgmap v11819: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 305 KiB/s wr, 14 op/s 2023-09-10T05:23:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21861.032609 secs 2023-09-10T05:23:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:23:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:23:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:51 smithi098 ceph-mon[103124]: pgmap v11820: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 364 KiB/s wr, 20 op/s 2023-09-10T05:23:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:23:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:23:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:51 smithi170 ceph-mon[102832]: pgmap v11820: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 364 KiB/s wr, 20 op/s 2023-09-10T05:23:52.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:23:52.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:23:52.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:51 smithi082 ceph-mon[98694]: pgmap v11820: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 364 KiB/s wr, 20 op/s 2023-09-10T05:23:52.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:23:52.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:23:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21864.339718 secs 2023-09-10T05:23:53.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21864.339718 secs 2023-09-10T05:23:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21864.339718 secs 2023-09-10T05:23:54.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:53 smithi098 ceph-mon[103124]: pgmap v11821: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 251 KiB/s wr, 13 op/s 2023-09-10T05:23:54.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21865.378793 secs 2023-09-10T05:23:54.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:53 smithi170 ceph-mon[102832]: pgmap v11821: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 251 KiB/s wr, 13 op/s 2023-09-10T05:23:54.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21865.378793 secs 2023-09-10T05:23:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:53 smithi082 ceph-mon[98694]: pgmap v11821: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 251 KiB/s wr, 13 op/s 2023-09-10T05:23:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21865.378793 secs 2023-09-10T05:23:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21866.032677 secs 2023-09-10T05:23:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21866.032677 secs 2023-09-10T05:23:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21866.032677 secs 2023-09-10T05:23:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:55 smithi098 ceph-mon[103124]: pgmap v11822: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 251 KiB/s wr, 13 op/s 2023-09-10T05:23:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:55 smithi170 ceph-mon[102832]: pgmap v11822: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 251 KiB/s wr, 13 op/s 2023-09-10T05:23:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:55 smithi082 ceph-mon[98694]: pgmap v11822: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 251 KiB/s wr, 13 op/s 2023-09-10T05:23:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:57 smithi098 ceph-mon[103124]: pgmap v11823: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 418 KiB/s wr, 19 op/s 2023-09-10T05:23:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21869.339802 secs 2023-09-10T05:23:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:57 smithi170 ceph-mon[102832]: pgmap v11823: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 418 KiB/s wr, 19 op/s 2023-09-10T05:23:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21869.339802 secs 2023-09-10T05:23:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:57 smithi082 ceph-mon[98694]: pgmap v11823: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 418 KiB/s wr, 19 op/s 2023-09-10T05:23:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21869.339802 secs 2023-09-10T05:23:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:23:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21870.378854 secs 2023-09-10T05:23:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:23:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21870.378854 secs 2023-09-10T05:23:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:23:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21870.378854 secs 2023-09-10T05:24:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21871.032741 secs 2023-09-10T05:24:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:00 smithi170 ceph-mon[102832]: pgmap v11824: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 226 KiB/s wr, 11 op/s 2023-09-10T05:24:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21871.032741 secs 2023-09-10T05:24:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:00 smithi082 ceph-mon[98694]: pgmap v11824: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 226 KiB/s wr, 11 op/s 2023-09-10T05:24:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21871.032741 secs 2023-09-10T05:24:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:00 smithi098 ceph-mon[103124]: pgmap v11824: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 226 KiB/s wr, 11 op/s 2023-09-10T05:24:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:02 smithi170 ceph-mon[102832]: pgmap v11825: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 281 KiB/s wr, 16 op/s 2023-09-10T05:24:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:02 smithi082 ceph-mon[98694]: pgmap v11825: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 281 KiB/s wr, 16 op/s 2023-09-10T05:24:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:02 smithi098 ceph-mon[103124]: pgmap v11825: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 281 KiB/s wr, 16 op/s 2023-09-10T05:24:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21874.339924 secs 2023-09-10T05:24:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21874.339924 secs 2023-09-10T05:24:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21874.339924 secs 2023-09-10T05:24:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:04 smithi170 ceph-mon[102832]: pgmap v11826: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 222 KiB/s wr, 10 op/s 2023-09-10T05:24:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21875.378938 secs 2023-09-10T05:24:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:04 smithi082 ceph-mon[98694]: pgmap v11826: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 222 KiB/s wr, 10 op/s 2023-09-10T05:24:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21875.378938 secs 2023-09-10T05:24:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:04 smithi098 ceph-mon[103124]: pgmap v11826: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 222 KiB/s wr, 10 op/s 2023-09-10T05:24:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21875.378938 secs 2023-09-10T05:24:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21876.032808 secs 2023-09-10T05:24:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21876.032808 secs 2023-09-10T05:24:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21876.032808 secs 2023-09-10T05:24:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:06 smithi170 ceph-mon[102832]: pgmap v11827: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 222 KiB/s wr, 10 op/s 2023-09-10T05:24:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:06 smithi082 ceph-mon[98694]: pgmap v11827: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 222 KiB/s wr, 10 op/s 2023-09-10T05:24:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:06 smithi098 ceph-mon[103124]: pgmap v11827: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 222 KiB/s wr, 10 op/s 2023-09-10T05:24:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:08 smithi170 ceph-mon[102832]: pgmap v11828: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 267 KiB/s wr, 29 op/s 2023-09-10T05:24:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21879.339982 secs 2023-09-10T05:24:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:08 smithi082 ceph-mon[98694]: pgmap v11828: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 267 KiB/s wr, 29 op/s 2023-09-10T05:24:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21879.339982 secs 2023-09-10T05:24:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:08 smithi098 ceph-mon[103124]: pgmap v11828: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 267 KiB/s wr, 29 op/s 2023-09-10T05:24:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21879.339982 secs 2023-09-10T05:24:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:09 smithi170 ceph-mon[102832]: osdmap e807: 12 total, 12 up, 12 in 2023-09-10T05:24:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21880.379004 secs 2023-09-10T05:24:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:09 smithi082 ceph-mon[98694]: osdmap e807: 12 total, 12 up, 12 in 2023-09-10T05:24:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21880.379004 secs 2023-09-10T05:24:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:09 smithi098 ceph-mon[103124]: osdmap e807: 12 total, 12 up, 12 in 2023-09-10T05:24:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21880.379004 secs 2023-09-10T05:24:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21881.032875 secs 2023-09-10T05:24:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:10 smithi170 ceph-mon[102832]: pgmap v11830: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 120 KiB/s wr, 28 op/s 2023-09-10T05:24:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21881.032875 secs 2023-09-10T05:24:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:10 smithi082 ceph-mon[98694]: pgmap v11830: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 120 KiB/s wr, 28 op/s 2023-09-10T05:24:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21881.032875 secs 2023-09-10T05:24:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:10 smithi098 ceph-mon[103124]: pgmap v11830: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 120 KiB/s wr, 28 op/s 2023-09-10T05:24:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:12 smithi170 ceph-mon[102832]: pgmap v11831: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T05:24:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:12 smithi082 ceph-mon[98694]: pgmap v11831: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T05:24:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:12 smithi098 ceph-mon[103124]: pgmap v11831: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T05:24:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21884.340043 secs 2023-09-10T05:24:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21884.340043 secs 2023-09-10T05:24:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21884.340043 secs 2023-09-10T05:24:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:14 smithi170 ceph-mon[102832]: pgmap v11832: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T05:24:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21885.379063 secs 2023-09-10T05:24:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21886.032941 secs 2023-09-10T05:24:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:14 smithi082 ceph-mon[98694]: pgmap v11832: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T05:24:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21885.379063 secs 2023-09-10T05:24:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21886.032941 secs 2023-09-10T05:24:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:14 smithi098 ceph-mon[103124]: pgmap v11832: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T05:24:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21885.379063 secs 2023-09-10T05:24:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21886.032941 secs 2023-09-10T05:24:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:15 smithi082 ceph-mon[98694]: osdmap e808: 12 total, 12 up, 12 in 2023-09-10T05:24:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:15 smithi082 ceph-mon[98694]: pgmap v11834: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:24:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:15 smithi098 ceph-mon[103124]: osdmap e808: 12 total, 12 up, 12 in 2023-09-10T05:24:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:15 smithi098 ceph-mon[103124]: pgmap v11834: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:24:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:15 smithi170 ceph-mon[102832]: osdmap e808: 12 total, 12 up, 12 in 2023-09-10T05:24:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:15 smithi170 ceph-mon[102832]: pgmap v11834: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:24:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:17 smithi170 ceph-mon[102832]: pgmap v11835: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 234 B/s wr, 11 op/s 2023-09-10T05:24:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21889.340142 secs 2023-09-10T05:24:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:17 smithi082 ceph-mon[98694]: pgmap v11835: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 234 B/s wr, 11 op/s 2023-09-10T05:24:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21889.340142 secs 2023-09-10T05:24:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:17 smithi098 ceph-mon[103124]: pgmap v11835: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 234 B/s wr, 11 op/s 2023-09-10T05:24:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21889.340142 secs 2023-09-10T05:24:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21890.379145 secs 2023-09-10T05:24:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21890.379145 secs 2023-09-10T05:24:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21890.379145 secs 2023-09-10T05:24:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21891.033005 secs 2023-09-10T05:24:20.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:19 smithi170 ceph-mon[102832]: pgmap v11836: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:24:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21891.033005 secs 2023-09-10T05:24:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:19 smithi082 ceph-mon[98694]: pgmap v11836: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:24:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21891.033005 secs 2023-09-10T05:24:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:19 smithi098 ceph-mon[103124]: pgmap v11836: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:24:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:21 smithi170 ceph-mon[102832]: pgmap v11837: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 614 B/s wr, 10 op/s 2023-09-10T05:24:22.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:21 smithi082 ceph-mon[98694]: pgmap v11837: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 614 B/s wr, 10 op/s 2023-09-10T05:24:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:21 smithi098 ceph-mon[103124]: pgmap v11837: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 614 B/s wr, 10 op/s 2023-09-10T05:24:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21894.340203 secs 2023-09-10T05:24:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21894.340203 secs 2023-09-10T05:24:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21894.340203 secs 2023-09-10T05:24:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:23 smithi170 ceph-mon[102832]: pgmap v11838: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 614 B/s wr, 10 op/s 2023-09-10T05:24:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21895.379202 secs 2023-09-10T05:24:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:23 smithi082 ceph-mon[98694]: pgmap v11838: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 614 B/s wr, 10 op/s 2023-09-10T05:24:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21895.379202 secs 2023-09-10T05:24:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:23 smithi098 ceph-mon[103124]: pgmap v11838: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 614 B/s wr, 10 op/s 2023-09-10T05:24:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21895.379202 secs 2023-09-10T05:24:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21896.033068 secs 2023-09-10T05:24:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21896.033068 secs 2023-09-10T05:24:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21896.033068 secs 2023-09-10T05:24:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:25 smithi170 ceph-mon[102832]: pgmap v11839: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 609 B/s wr, 10 op/s 2023-09-10T05:24:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:25 smithi082 ceph-mon[98694]: pgmap v11839: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 609 B/s wr, 10 op/s 2023-09-10T05:24:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:25 smithi098 ceph-mon[103124]: pgmap v11839: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 609 B/s wr, 10 op/s 2023-09-10T05:24:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:27 smithi170 ceph-mon[102832]: pgmap v11840: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 73 KiB/s wr, 14 op/s 2023-09-10T05:24:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:27 smithi082 ceph-mon[98694]: pgmap v11840: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 73 KiB/s wr, 14 op/s 2023-09-10T05:24:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:27 smithi098 ceph-mon[103124]: pgmap v11840: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 73 KiB/s wr, 14 op/s 2023-09-10T05:24:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21900.379275 secs 2023-09-10T05:24:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21900.379275 secs 2023-09-10T05:24:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21900.379275 secs 2023-09-10T05:24:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21901.033132 secs 2023-09-10T05:24:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:29 smithi170 ceph-mon[102832]: pgmap v11841: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 9 op/s 2023-09-10T05:24:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21901.033132 secs 2023-09-10T05:24:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:29 smithi082 ceph-mon[98694]: pgmap v11841: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 9 op/s 2023-09-10T05:24:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21901.033132 secs 2023-09-10T05:24:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:29 smithi098 ceph-mon[103124]: pgmap v11841: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 9 op/s 2023-09-10T05:24:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:31 smithi170 ceph-mon[102832]: pgmap v11842: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 145 KiB/s wr, 15 op/s 2023-09-10T05:24:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:24:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:31 smithi082 ceph-mon[98694]: pgmap v11842: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 145 KiB/s wr, 15 op/s 2023-09-10T05:24:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:24:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:31 smithi098 ceph-mon[103124]: pgmap v11842: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 145 KiB/s wr, 15 op/s 2023-09-10T05:24:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:24:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:24:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:24:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:24:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:24:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:24:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:24:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:24:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:24:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:24:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:33 smithi170 ceph-mon[102832]: pgmap v11843: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 145 KiB/s wr, 10 op/s 2023-09-10T05:24:34.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:24:34.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:24:34.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21905.379332 secs 2023-09-10T05:24:34.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:24:34.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:24:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:33 smithi082 ceph-mon[98694]: pgmap v11843: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 145 KiB/s wr, 10 op/s 2023-09-10T05:24:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:24:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:24:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21905.379332 secs 2023-09-10T05:24:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:24:34.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:24:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:33 smithi098 ceph-mon[103124]: pgmap v11843: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 145 KiB/s wr, 10 op/s 2023-09-10T05:24:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:24:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:24:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21905.379332 secs 2023-09-10T05:24:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:24:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:24:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21906.033198 secs 2023-09-10T05:24:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:24:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:24:35.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:24:35.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:24:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21906.033198 secs 2023-09-10T05:24:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:24:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:24:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:24:35.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:24:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21906.033198 secs 2023-09-10T05:24:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:24:35.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:24:35.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:24:35.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:24:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:35 smithi170 ceph-mon[102832]: pgmap v11844: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 145 KiB/s wr, 10 op/s 2023-09-10T05:24:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:35 smithi082 ceph-mon[98694]: pgmap v11844: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 145 KiB/s wr, 10 op/s 2023-09-10T05:24:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:35 smithi098 ceph-mon[103124]: pgmap v11844: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 145 KiB/s wr, 10 op/s 2023-09-10T05:24:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:24:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:24:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:24:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:24:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:24:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:24:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:37 smithi170 ceph-mon[102832]: pgmap v11845: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 529 KiB/s wr, 17 op/s 2023-09-10T05:24:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:24:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:24:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:37 smithi082 ceph-mon[98694]: pgmap v11845: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 529 KiB/s wr, 17 op/s 2023-09-10T05:24:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:24:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:24:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:37 smithi098 ceph-mon[103124]: pgmap v11845: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 529 KiB/s wr, 17 op/s 2023-09-10T05:24:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:24:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:24:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21910.379390 secs 2023-09-10T05:24:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21910.379390 secs 2023-09-10T05:24:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21910.379390 secs 2023-09-10T05:24:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21911.033264 secs 2023-09-10T05:24:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:39 smithi170 ceph-mon[102832]: pgmap v11846: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 456 KiB/s wr, 12 op/s 2023-09-10T05:24:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21911.033264 secs 2023-09-10T05:24:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:39 smithi082 ceph-mon[98694]: pgmap v11846: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 456 KiB/s wr, 12 op/s 2023-09-10T05:24:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21911.033264 secs 2023-09-10T05:24:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:39 smithi098 ceph-mon[103124]: pgmap v11846: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 456 KiB/s wr, 12 op/s 2023-09-10T05:24:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:41 smithi170 ceph-mon[102832]: pgmap v11847: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 564 KiB/s wr, 18 op/s 2023-09-10T05:24:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:41 smithi082 ceph-mon[98694]: pgmap v11847: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 564 KiB/s wr, 18 op/s 2023-09-10T05:24:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:41 smithi098 ceph-mon[103124]: pgmap v11847: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 564 KiB/s wr, 18 op/s 2023-09-10T05:24:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:43 smithi170 ceph-mon[102832]: pgmap v11848: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 492 KiB/s wr, 13 op/s 2023-09-10T05:24:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:24:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:24:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21915.379447 secs 2023-09-10T05:24:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:43 smithi082 ceph-mon[98694]: pgmap v11848: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 492 KiB/s wr, 13 op/s 2023-09-10T05:24:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:24:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:24:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21915.379447 secs 2023-09-10T05:24:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:43 smithi098 ceph-mon[103124]: pgmap v11848: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 492 KiB/s wr, 13 op/s 2023-09-10T05:24:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:24:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:24:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21915.379447 secs 2023-09-10T05:24:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21916.033329 secs 2023-09-10T05:24:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21916.033329 secs 2023-09-10T05:24:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21916.033329 secs 2023-09-10T05:24:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:45 smithi170 ceph-mon[102832]: pgmap v11849: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 492 KiB/s wr, 13 op/s 2023-09-10T05:24:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:24:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:24:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:45 smithi082 ceph-mon[98694]: pgmap v11849: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 492 KiB/s wr, 13 op/s 2023-09-10T05:24:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:24:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:24:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:45 smithi098 ceph-mon[103124]: pgmap v11849: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 492 KiB/s wr, 13 op/s 2023-09-10T05:24:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:24:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:24:48.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:47 smithi170 ceph-mon[102832]: pgmap v11850: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 771 KiB/s wr, 21 op/s 2023-09-10T05:24:48.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21919.340570 secs 2023-09-10T05:24:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:47 smithi082 ceph-mon[98694]: pgmap v11850: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 771 KiB/s wr, 21 op/s 2023-09-10T05:24:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21919.340570 secs 2023-09-10T05:24:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:47 smithi098 ceph-mon[103124]: pgmap v11850: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 771 KiB/s wr, 21 op/s 2023-09-10T05:24:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21919.340570 secs 2023-09-10T05:24:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21920.379490 secs 2023-09-10T05:24:49.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21920.379490 secs 2023-09-10T05:24:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21920.379490 secs 2023-09-10T05:24:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21921.033392 secs 2023-09-10T05:24:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:49 smithi082 ceph-mon[98694]: pgmap v11851: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 387 KiB/s wr, 13 op/s 2023-09-10T05:24:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21921.033392 secs 2023-09-10T05:24:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:49 smithi098 ceph-mon[103124]: pgmap v11851: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 387 KiB/s wr, 13 op/s 2023-09-10T05:24:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21921.033392 secs 2023-09-10T05:24:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:49 smithi170 ceph-mon[102832]: pgmap v11851: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 387 KiB/s wr, 13 op/s 2023-09-10T05:24:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:50 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:24:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:24:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:24:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:50 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:24:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:24:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:24:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:50 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:24:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:24:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:24:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:51 smithi098 ceph-mon[103124]: pgmap v11852: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 387 KiB/s wr, 19 op/s 2023-09-10T05:24:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:51 smithi170 ceph-mon[102832]: pgmap v11852: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 387 KiB/s wr, 19 op/s 2023-09-10T05:24:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:51 smithi082 ceph-mon[98694]: pgmap v11852: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 387 KiB/s wr, 19 op/s 2023-09-10T05:24:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21924.340634 secs 2023-09-10T05:24:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21924.340634 secs 2023-09-10T05:24:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21924.340634 secs 2023-09-10T05:24:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:53 smithi098 ceph-mon[103124]: pgmap v11853: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 13 op/s 2023-09-10T05:24:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21925.379553 secs 2023-09-10T05:24:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:53 smithi170 ceph-mon[102832]: pgmap v11853: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 13 op/s 2023-09-10T05:24:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21925.379553 secs 2023-09-10T05:24:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:53 smithi082 ceph-mon[98694]: pgmap v11853: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 13 op/s 2023-09-10T05:24:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21925.379553 secs 2023-09-10T05:24:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21926.033457 secs 2023-09-10T05:24:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21926.033457 secs 2023-09-10T05:24:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21926.033457 secs 2023-09-10T05:24:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:55 smithi098 ceph-mon[103124]: pgmap v11854: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 13 op/s 2023-09-10T05:24:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:55 smithi170 ceph-mon[102832]: pgmap v11854: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 13 op/s 2023-09-10T05:24:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:55 smithi082 ceph-mon[98694]: pgmap v11854: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 13 op/s 2023-09-10T05:24:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:58 smithi098 ceph-mon[103124]: pgmap v11855: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 435 KiB/s wr, 18 op/s 2023-09-10T05:24:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21929.340703 secs 2023-09-10T05:24:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:58 smithi170 ceph-mon[102832]: pgmap v11855: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 435 KiB/s wr, 18 op/s 2023-09-10T05:24:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21929.340703 secs 2023-09-10T05:24:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:58 smithi082 ceph-mon[98694]: pgmap v11855: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 435 KiB/s wr, 18 op/s 2023-09-10T05:24:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21929.340703 secs 2023-09-10T05:24:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:24:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21930.379626 secs 2023-09-10T05:24:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:24:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21930.379626 secs 2023-09-10T05:24:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:24:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21930.379626 secs 2023-09-10T05:25:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21931.033483 secs 2023-09-10T05:25:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:00 smithi170 ceph-mon[102832]: pgmap v11856: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 157 KiB/s wr, 11 op/s 2023-09-10T05:25:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21931.033483 secs 2023-09-10T05:25:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:00 smithi082 ceph-mon[98694]: pgmap v11856: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 157 KiB/s wr, 11 op/s 2023-09-10T05:25:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21931.033483 secs 2023-09-10T05:25:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:00 smithi098 ceph-mon[103124]: pgmap v11856: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 157 KiB/s wr, 11 op/s 2023-09-10T05:25:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:02 smithi170 ceph-mon[102832]: pgmap v11857: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 257 KiB/s wr, 30 op/s 2023-09-10T05:25:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:02 smithi082 ceph-mon[98694]: pgmap v11857: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 257 KiB/s wr, 30 op/s 2023-09-10T05:25:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:02 smithi098 ceph-mon[103124]: pgmap v11857: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 257 KiB/s wr, 30 op/s 2023-09-10T05:25:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21934.340769 secs 2023-09-10T05:25:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21934.340769 secs 2023-09-10T05:25:03.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21934.340769 secs 2023-09-10T05:25:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:04 smithi170 ceph-mon[102832]: pgmap v11858: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 256 KiB/s wr, 24 op/s 2023-09-10T05:25:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:25:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:25:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21935.379683 secs 2023-09-10T05:25:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:04 smithi082 ceph-mon[98694]: pgmap v11858: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 256 KiB/s wr, 24 op/s 2023-09-10T05:25:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:25:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:25:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21935.379683 secs 2023-09-10T05:25:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:04 smithi098 ceph-mon[103124]: pgmap v11858: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 256 KiB/s wr, 24 op/s 2023-09-10T05:25:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:25:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:25:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21935.379683 secs 2023-09-10T05:25:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21936.033549 secs 2023-09-10T05:25:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21936.033549 secs 2023-09-10T05:25:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21936.033549 secs 2023-09-10T05:25:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:06 smithi170 ceph-mon[102832]: pgmap v11859: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 257 KiB/s wr, 24 op/s 2023-09-10T05:25:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:25:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:25:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:06 smithi082 ceph-mon[98694]: pgmap v11859: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 257 KiB/s wr, 24 op/s 2023-09-10T05:25:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:25:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:25:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:06 smithi098 ceph-mon[103124]: pgmap v11859: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 257 KiB/s wr, 24 op/s 2023-09-10T05:25:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:25:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:25:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:08 smithi170 ceph-mon[102832]: pgmap v11860: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 257 KiB/s wr, 29 op/s 2023-09-10T05:25:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21939.340821 secs 2023-09-10T05:25:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:08 smithi082 ceph-mon[98694]: pgmap v11860: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 257 KiB/s wr, 29 op/s 2023-09-10T05:25:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21939.340821 secs 2023-09-10T05:25:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:08 smithi098 ceph-mon[103124]: pgmap v11860: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 257 KiB/s wr, 29 op/s 2023-09-10T05:25:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21939.340821 secs 2023-09-10T05:25:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:09 smithi170 ceph-mon[102832]: osdmap e809: 12 total, 12 up, 12 in 2023-09-10T05:25:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21940.379758 secs 2023-09-10T05:25:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21941.033615 secs 2023-09-10T05:25:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:09 smithi082 ceph-mon[98694]: osdmap e809: 12 total, 12 up, 12 in 2023-09-10T05:25:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21940.379758 secs 2023-09-10T05:25:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21941.033615 secs 2023-09-10T05:25:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:09 smithi098 ceph-mon[103124]: osdmap e809: 12 total, 12 up, 12 in 2023-09-10T05:25:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21940.379758 secs 2023-09-10T05:25:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21941.033615 secs 2023-09-10T05:25:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:10 smithi170 ceph-mon[102832]: pgmap v11862: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 120 KiB/s wr, 29 op/s 2023-09-10T05:25:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:10 smithi082 ceph-mon[98694]: pgmap v11862: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 120 KiB/s wr, 29 op/s 2023-09-10T05:25:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:10 smithi098 ceph-mon[103124]: pgmap v11862: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 120 KiB/s wr, 29 op/s 2023-09-10T05:25:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:12 smithi170 ceph-mon[102832]: pgmap v11863: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T05:25:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:12 smithi082 ceph-mon[98694]: pgmap v11863: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T05:25:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:12 smithi098 ceph-mon[103124]: pgmap v11863: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T05:25:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21944.340882 secs 2023-09-10T05:25:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21944.340882 secs 2023-09-10T05:25:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21944.340882 secs 2023-09-10T05:25:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:14 smithi170 ceph-mon[102832]: pgmap v11864: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T05:25:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21945.379816 secs 2023-09-10T05:25:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21946.033678 secs 2023-09-10T05:25:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:14 smithi082 ceph-mon[98694]: pgmap v11864: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T05:25:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21945.379816 secs 2023-09-10T05:25:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21946.033678 secs 2023-09-10T05:25:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:14 smithi098 ceph-mon[103124]: pgmap v11864: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T05:25:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21945.379816 secs 2023-09-10T05:25:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21946.033678 secs 2023-09-10T05:25:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:15 smithi098 ceph-mon[103124]: osdmap e810: 12 total, 12 up, 12 in 2023-09-10T05:25:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:15 smithi098 ceph-mon[103124]: pgmap v11866: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T05:25:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:15 smithi170 ceph-mon[102832]: osdmap e810: 12 total, 12 up, 12 in 2023-09-10T05:25:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:15 smithi170 ceph-mon[102832]: pgmap v11866: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T05:25:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:15 smithi082 ceph-mon[98694]: osdmap e810: 12 total, 12 up, 12 in 2023-09-10T05:25:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:15 smithi082 ceph-mon[98694]: pgmap v11866: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T05:25:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:17 smithi170 ceph-mon[102832]: pgmap v11867: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 92 KiB/s wr, 13 op/s 2023-09-10T05:25:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21949.340943 secs 2023-09-10T05:25:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:17 smithi082 ceph-mon[98694]: pgmap v11867: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 92 KiB/s wr, 13 op/s 2023-09-10T05:25:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21949.340943 secs 2023-09-10T05:25:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:17 smithi098 ceph-mon[103124]: pgmap v11867: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 92 KiB/s wr, 13 op/s 2023-09-10T05:25:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21949.340943 secs 2023-09-10T05:25:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21950.379874 secs 2023-09-10T05:25:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21950.379874 secs 2023-09-10T05:25:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21950.379874 secs 2023-09-10T05:25:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21951.033742 secs 2023-09-10T05:25:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:19 smithi170 ceph-mon[102832]: pgmap v11868: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T05:25:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21951.033742 secs 2023-09-10T05:25:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:19 smithi082 ceph-mon[98694]: pgmap v11868: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T05:25:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21951.033742 secs 2023-09-10T05:25:20.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:19 smithi098 ceph-mon[103124]: pgmap v11868: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 KiB/s wr, 12 op/s 2023-09-10T05:25:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:21 smithi170 ceph-mon[102832]: pgmap v11869: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 168 KiB/s wr, 11 op/s 2023-09-10T05:25:22.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:21 smithi082 ceph-mon[98694]: pgmap v11869: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 168 KiB/s wr, 11 op/s 2023-09-10T05:25:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:21 smithi098 ceph-mon[103124]: pgmap v11869: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 168 KiB/s wr, 11 op/s 2023-09-10T05:25:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21954.341031 secs 2023-09-10T05:25:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21954.341031 secs 2023-09-10T05:25:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21954.341031 secs 2023-09-10T05:25:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:23 smithi170 ceph-mon[102832]: pgmap v11870: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 168 KiB/s wr, 11 op/s 2023-09-10T05:25:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21955.379942 secs 2023-09-10T05:25:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:23 smithi082 ceph-mon[98694]: pgmap v11870: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 168 KiB/s wr, 11 op/s 2023-09-10T05:25:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21955.379942 secs 2023-09-10T05:25:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:23 smithi098 ceph-mon[103124]: pgmap v11870: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 168 KiB/s wr, 11 op/s 2023-09-10T05:25:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21955.379942 secs 2023-09-10T05:25:25.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21956.033806 secs 2023-09-10T05:25:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21956.033806 secs 2023-09-10T05:25:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21956.033806 secs 2023-09-10T05:25:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:25 smithi170 ceph-mon[102832]: pgmap v11871: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 166 KiB/s wr, 11 op/s 2023-09-10T05:25:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:25 smithi082 ceph-mon[98694]: pgmap v11871: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 166 KiB/s wr, 11 op/s 2023-09-10T05:25:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:25 smithi098 ceph-mon[103124]: pgmap v11871: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 166 KiB/s wr, 11 op/s 2023-09-10T05:25:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:27 smithi170 ceph-mon[102832]: pgmap v11872: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 140 KiB/s wr, 14 op/s 2023-09-10T05:25:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:27 smithi082 ceph-mon[98694]: pgmap v11872: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 140 KiB/s wr, 14 op/s 2023-09-10T05:25:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:27 smithi098 ceph-mon[103124]: pgmap v11872: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 140 KiB/s wr, 14 op/s 2023-09-10T05:25:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21960.380001 secs 2023-09-10T05:25:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21960.380001 secs 2023-09-10T05:25:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21960.380001 secs 2023-09-10T05:25:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21961.033874 secs 2023-09-10T05:25:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:29 smithi170 ceph-mon[102832]: pgmap v11873: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 9 op/s 2023-09-10T05:25:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21961.033874 secs 2023-09-10T05:25:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:29 smithi082 ceph-mon[98694]: pgmap v11873: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 9 op/s 2023-09-10T05:25:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21961.033874 secs 2023-09-10T05:25:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:29 smithi098 ceph-mon[103124]: pgmap v11873: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 73 KiB/s wr, 9 op/s 2023-09-10T05:25:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:31 smithi170 ceph-mon[102832]: pgmap v11874: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 271 KiB/s wr, 14 op/s 2023-09-10T05:25:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:25:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:31 smithi082 ceph-mon[98694]: pgmap v11874: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 271 KiB/s wr, 14 op/s 2023-09-10T05:25:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:25:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:31 smithi098 ceph-mon[103124]: pgmap v11874: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 271 KiB/s wr, 14 op/s 2023-09-10T05:25:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:25:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:25:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:25:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:25:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:25:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:25:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:25:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:25:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:25:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:25:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:33 smithi170 ceph-mon[102832]: pgmap v11875: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 9 op/s 2023-09-10T05:25:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:25:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:25:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21965.380061 secs 2023-09-10T05:25:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:33 smithi082 ceph-mon[98694]: pgmap v11875: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 9 op/s 2023-09-10T05:25:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:25:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:25:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21965.380061 secs 2023-09-10T05:25:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:33 smithi098 ceph-mon[103124]: pgmap v11875: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 9 op/s 2023-09-10T05:25:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:25:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:25:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21965.380061 secs 2023-09-10T05:25:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21966.033938 secs 2023-09-10T05:25:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:25:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21966.033938 secs 2023-09-10T05:25:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:25:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21966.033938 secs 2023-09-10T05:25:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:25:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:36 smithi170 ceph-mon[102832]: pgmap v11876: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 9 op/s 2023-09-10T05:25:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:25:36.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:25:36.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:25:36.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:25:36.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:25:36.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:25:36.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:25:36.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:25:36.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:25:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:36 smithi082 ceph-mon[98694]: pgmap v11876: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 9 op/s 2023-09-10T05:25:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:25:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:25:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:25:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:25:36.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:25:36.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:25:36.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:25:36.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:25:36.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:25:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:36 smithi098 ceph-mon[103124]: pgmap v11876: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 9 op/s 2023-09-10T05:25:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:25:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:25:36.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:25:36.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:25:36.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:25:36.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:25:36.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:25:36.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:25:36.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:25:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:38 smithi170 ceph-mon[102832]: pgmap v11877: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 457 KiB/s wr, 16 op/s 2023-09-10T05:25:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:38 smithi082 ceph-mon[98694]: pgmap v11877: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 457 KiB/s wr, 16 op/s 2023-09-10T05:25:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:38 smithi098 ceph-mon[103124]: pgmap v11877: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 457 KiB/s wr, 16 op/s 2023-09-10T05:25:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21970.380144 secs 2023-09-10T05:25:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21970.380144 secs 2023-09-10T05:25:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21970.380144 secs 2023-09-10T05:25:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21971.034001 secs 2023-09-10T05:25:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:40 smithi170 ceph-mon[102832]: pgmap v11878: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 457 KiB/s wr, 11 op/s 2023-09-10T05:25:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21971.034001 secs 2023-09-10T05:25:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:40 smithi082 ceph-mon[98694]: pgmap v11878: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 457 KiB/s wr, 11 op/s 2023-09-10T05:25:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21971.034001 secs 2023-09-10T05:25:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:40 smithi098 ceph-mon[103124]: pgmap v11878: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 457 KiB/s wr, 11 op/s 2023-09-10T05:25:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:42 smithi170 ceph-mon[102832]: pgmap v11879: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 591 KiB/s wr, 18 op/s 2023-09-10T05:25:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:42 smithi082 ceph-mon[98694]: pgmap v11879: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 591 KiB/s wr, 18 op/s 2023-09-10T05:25:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:42 smithi098 ceph-mon[103124]: pgmap v11879: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 591 KiB/s wr, 18 op/s 2023-09-10T05:25:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21974.341347 secs 2023-09-10T05:25:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21974.341347 secs 2023-09-10T05:25:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21974.341347 secs 2023-09-10T05:25:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:44 smithi170 ceph-mon[102832]: pgmap v11880: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T05:25:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:25:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:25:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:25:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21975.380203 secs 2023-09-10T05:25:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21976.034064 secs 2023-09-10T05:25:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:44 smithi082 ceph-mon[98694]: pgmap v11880: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T05:25:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:25:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:25:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:25:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21975.380203 secs 2023-09-10T05:25:44.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21976.034064 secs 2023-09-10T05:25:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:44 smithi098 ceph-mon[103124]: pgmap v11880: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T05:25:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:25:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:25:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:25:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21975.380203 secs 2023-09-10T05:25:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21976.034064 secs 2023-09-10T05:25:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:46 smithi170 ceph-mon[102832]: pgmap v11881: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T05:25:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:25:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:25:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:46 smithi082 ceph-mon[98694]: pgmap v11881: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T05:25:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:25:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:25:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:46 smithi098 ceph-mon[103124]: pgmap v11881: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T05:25:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:25:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:25:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:48 smithi170 ceph-mon[102832]: pgmap v11882: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 670 KiB/s wr, 20 op/s 2023-09-10T05:25:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21979.341407 secs 2023-09-10T05:25:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:48 smithi082 ceph-mon[98694]: pgmap v11882: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 670 KiB/s wr, 20 op/s 2023-09-10T05:25:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21979.341407 secs 2023-09-10T05:25:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:48 smithi098 ceph-mon[103124]: pgmap v11882: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 670 KiB/s wr, 20 op/s 2023-09-10T05:25:48.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21979.341407 secs 2023-09-10T05:25:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21980.380261 secs 2023-09-10T05:25:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21981.034129 secs 2023-09-10T05:25:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21980.380261 secs 2023-09-10T05:25:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21981.034129 secs 2023-09-10T05:25:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21980.380261 secs 2023-09-10T05:25:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21981.034129 secs 2023-09-10T05:25:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:50 smithi170 ceph-mon[102832]: pgmap v11883: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 410 KiB/s wr, 14 op/s 2023-09-10T05:25:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:50 smithi082 ceph-mon[98694]: pgmap v11883: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 410 KiB/s wr, 14 op/s 2023-09-10T05:25:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:50 smithi098 ceph-mon[103124]: pgmap v11883: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 410 KiB/s wr, 14 op/s 2023-09-10T05:25:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:52 smithi170 ceph-mon[102832]: pgmap v11884: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 411 KiB/s wr, 19 op/s 2023-09-10T05:25:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:52 smithi082 ceph-mon[98694]: pgmap v11884: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 411 KiB/s wr, 19 op/s 2023-09-10T05:25:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:52 smithi098 ceph-mon[103124]: pgmap v11884: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 411 KiB/s wr, 19 op/s 2023-09-10T05:25:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21984.341498 secs 2023-09-10T05:25:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21984.341498 secs 2023-09-10T05:25:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21984.341498 secs 2023-09-10T05:25:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:54 smithi170 ceph-mon[102832]: pgmap v11885: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T05:25:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21985.380320 secs 2023-09-10T05:25:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21986.034196 secs 2023-09-10T05:25:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:54 smithi082 ceph-mon[98694]: pgmap v11885: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T05:25:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21985.380320 secs 2023-09-10T05:25:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21986.034196 secs 2023-09-10T05:25:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:54 smithi098 ceph-mon[103124]: pgmap v11885: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T05:25:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21985.380320 secs 2023-09-10T05:25:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21986.034196 secs 2023-09-10T05:25:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:55 smithi098 ceph-mon[103124]: pgmap v11886: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T05:25:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:55 smithi170 ceph-mon[102832]: pgmap v11886: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T05:25:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:55 smithi082 ceph-mon[98694]: pgmap v11886: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T05:25:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:25:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:25:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:57 smithi098 ceph-mon[103124]: pgmap v11887: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 445 KiB/s wr, 17 op/s 2023-09-10T05:25:58.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21989.341605 secs 2023-09-10T05:25:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:25:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:25:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:57 smithi170 ceph-mon[102832]: pgmap v11887: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 445 KiB/s wr, 17 op/s 2023-09-10T05:25:58.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21989.341605 secs 2023-09-10T05:25:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:25:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:25:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:57 smithi082 ceph-mon[98694]: pgmap v11887: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 445 KiB/s wr, 17 op/s 2023-09-10T05:25:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21989.341605 secs 2023-09-10T05:25:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21990.380381 secs 2023-09-10T05:25:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21990.380381 secs 2023-09-10T05:25:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21990.380381 secs 2023-09-10T05:26:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21991.034265 secs 2023-09-10T05:26:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:25:59 smithi170 ceph-mon[102832]: pgmap v11888: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 168 KiB/s wr, 10 op/s 2023-09-10T05:26:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21991.034265 secs 2023-09-10T05:26:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:25:59 smithi082 ceph-mon[98694]: pgmap v11888: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 168 KiB/s wr, 10 op/s 2023-09-10T05:26:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21991.034265 secs 2023-09-10T05:26:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:25:59 smithi098 ceph-mon[103124]: pgmap v11888: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 168 KiB/s wr, 10 op/s 2023-09-10T05:26:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:26:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:26:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:26:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:26:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:26:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:26:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:01 smithi170 ceph-mon[102832]: pgmap v11889: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 246 KiB/s wr, 23 op/s 2023-09-10T05:26:02.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:01 smithi082 ceph-mon[98694]: pgmap v11889: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 246 KiB/s wr, 23 op/s 2023-09-10T05:26:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:01 smithi098 ceph-mon[103124]: pgmap v11889: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 246 KiB/s wr, 23 op/s 2023-09-10T05:26:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21994.341667 secs 2023-09-10T05:26:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21994.341667 secs 2023-09-10T05:26:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21994.341667 secs 2023-09-10T05:26:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:03 smithi170 ceph-mon[102832]: pgmap v11890: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 246 KiB/s wr, 18 op/s 2023-09-10T05:26:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21995.380458 secs 2023-09-10T05:26:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:03 smithi082 ceph-mon[98694]: pgmap v11890: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 246 KiB/s wr, 18 op/s 2023-09-10T05:26:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21995.380458 secs 2023-09-10T05:26:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:03 smithi098 ceph-mon[103124]: pgmap v11890: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 246 KiB/s wr, 18 op/s 2023-09-10T05:26:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21995.380458 secs 2023-09-10T05:26:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 21996.034332 secs 2023-09-10T05:26:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 21996.034332 secs 2023-09-10T05:26:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 21996.034332 secs 2023-09-10T05:26:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:05 smithi170 ceph-mon[102832]: pgmap v11891: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 246 KiB/s wr, 18 op/s 2023-09-10T05:26:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:05 smithi082 ceph-mon[98694]: pgmap v11891: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 246 KiB/s wr, 18 op/s 2023-09-10T05:26:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:05 smithi098 ceph-mon[103124]: pgmap v11891: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 246 KiB/s wr, 18 op/s 2023-09-10T05:26:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:07 smithi170 ceph-mon[102832]: pgmap v11892: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 268 KiB/s wr, 29 op/s 2023-09-10T05:26:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 21999.341759 secs 2023-09-10T05:26:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:07 smithi082 ceph-mon[98694]: pgmap v11892: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 268 KiB/s wr, 29 op/s 2023-09-10T05:26:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 21999.341759 secs 2023-09-10T05:26:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:07 smithi098 ceph-mon[103124]: pgmap v11892: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 268 KiB/s wr, 29 op/s 2023-09-10T05:26:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 21999.341759 secs 2023-09-10T05:26:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:08 smithi170 ceph-mon[102832]: osdmap e811: 12 total, 12 up, 12 in 2023-09-10T05:26:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22000.380497 secs 2023-09-10T05:26:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:08 smithi082 ceph-mon[98694]: osdmap e811: 12 total, 12 up, 12 in 2023-09-10T05:26:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22000.380497 secs 2023-09-10T05:26:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:08 smithi098 ceph-mon[103124]: osdmap e811: 12 total, 12 up, 12 in 2023-09-10T05:26:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22000.380497 secs 2023-09-10T05:26:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22001.034394 secs 2023-09-10T05:26:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:09 smithi170 ceph-mon[102832]: pgmap v11894: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 28 op/s 2023-09-10T05:26:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22001.034394 secs 2023-09-10T05:26:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:09 smithi082 ceph-mon[98694]: pgmap v11894: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 28 op/s 2023-09-10T05:26:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22001.034394 secs 2023-09-10T05:26:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:09 smithi098 ceph-mon[103124]: pgmap v11894: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 28 op/s 2023-09-10T05:26:12.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:11 smithi170 ceph-mon[102832]: pgmap v11895: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 18 op/s 2023-09-10T05:26:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:11 smithi082 ceph-mon[98694]: pgmap v11895: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 18 op/s 2023-09-10T05:26:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:11 smithi098 ceph-mon[103124]: pgmap v11895: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 18 op/s 2023-09-10T05:26:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22004.341777 secs 2023-09-10T05:26:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22004.341777 secs 2023-09-10T05:26:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22004.341777 secs 2023-09-10T05:26:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:13 smithi170 ceph-mon[102832]: pgmap v11896: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 18 op/s 2023-09-10T05:26:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22005.380581 secs 2023-09-10T05:26:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:13 smithi082 ceph-mon[98694]: pgmap v11896: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 18 op/s 2023-09-10T05:26:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22005.380581 secs 2023-09-10T05:26:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:13 smithi098 ceph-mon[103124]: pgmap v11896: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 18 op/s 2023-09-10T05:26:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22005.380581 secs 2023-09-10T05:26:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22006.034461 secs 2023-09-10T05:26:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:14 smithi170 ceph-mon[102832]: osdmap e812: 12 total, 12 up, 12 in 2023-09-10T05:26:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22006.034461 secs 2023-09-10T05:26:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:14 smithi082 ceph-mon[98694]: osdmap e812: 12 total, 12 up, 12 in 2023-09-10T05:26:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22006.034461 secs 2023-09-10T05:26:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:14 smithi098 ceph-mon[103124]: osdmap e812: 12 total, 12 up, 12 in 2023-09-10T05:26:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:15 smithi170 ceph-mon[102832]: pgmap v11898: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:26:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:15 smithi082 ceph-mon[98694]: pgmap v11898: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:26:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:15 smithi098 ceph-mon[103124]: pgmap v11898: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:26:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:17 smithi170 ceph-mon[102832]: pgmap v11899: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 228 B/s wr, 11 op/s 2023-09-10T05:26:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22009.341840 secs 2023-09-10T05:26:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:17 smithi082 ceph-mon[98694]: pgmap v11899: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 228 B/s wr, 11 op/s 2023-09-10T05:26:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22009.341840 secs 2023-09-10T05:26:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:17 smithi098 ceph-mon[103124]: pgmap v11899: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 228 B/s wr, 11 op/s 2023-09-10T05:26:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22009.341840 secs 2023-09-10T05:26:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22010.380639 secs 2023-09-10T05:26:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22010.380639 secs 2023-09-10T05:26:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22010.380639 secs 2023-09-10T05:26:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22011.034486 secs 2023-09-10T05:26:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:19 smithi170 ceph-mon[102832]: pgmap v11900: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:26:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22011.034486 secs 2023-09-10T05:26:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:19 smithi082 ceph-mon[98694]: pgmap v11900: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:26:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22011.034486 secs 2023-09-10T05:26:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:19 smithi098 ceph-mon[103124]: pgmap v11900: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:26:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:21 smithi170 ceph-mon[102832]: pgmap v11901: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 614 B/s wr, 10 op/s 2023-09-10T05:26:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:21 smithi082 ceph-mon[98694]: pgmap v11901: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 614 B/s wr, 10 op/s 2023-09-10T05:26:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:21 smithi098 ceph-mon[103124]: pgmap v11901: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 614 B/s wr, 10 op/s 2023-09-10T05:26:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22014.341902 secs 2023-09-10T05:26:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22014.341902 secs 2023-09-10T05:26:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22014.341902 secs 2023-09-10T05:26:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:23 smithi170 ceph-mon[102832]: pgmap v11902: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 614 B/s wr, 10 op/s 2023-09-10T05:26:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22015.380707 secs 2023-09-10T05:26:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:23 smithi082 ceph-mon[98694]: pgmap v11902: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 614 B/s wr, 10 op/s 2023-09-10T05:26:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22015.380707 secs 2023-09-10T05:26:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:23 smithi098 ceph-mon[103124]: pgmap v11902: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 614 B/s wr, 10 op/s 2023-09-10T05:26:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22015.380707 secs 2023-09-10T05:26:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22016.034552 secs 2023-09-10T05:26:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22016.034552 secs 2023-09-10T05:26:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22016.034552 secs 2023-09-10T05:26:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:25 smithi170 ceph-mon[102832]: pgmap v11903: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 609 B/s wr, 10 op/s 2023-09-10T05:26:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:25 smithi082 ceph-mon[98694]: pgmap v11903: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 609 B/s wr, 10 op/s 2023-09-10T05:26:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:25 smithi098 ceph-mon[103124]: pgmap v11903: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 609 B/s wr, 10 op/s 2023-09-10T05:26:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:27 smithi170 ceph-mon[102832]: pgmap v11904: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 74 KiB/s wr, 13 op/s 2023-09-10T05:26:28.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:27 smithi082 ceph-mon[98694]: pgmap v11904: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 74 KiB/s wr, 13 op/s 2023-09-10T05:26:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:27 smithi098 ceph-mon[103124]: pgmap v11904: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 74 KiB/s wr, 13 op/s 2023-09-10T05:26:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22020.380764 secs 2023-09-10T05:26:29.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22020.380764 secs 2023-09-10T05:26:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22020.380764 secs 2023-09-10T05:26:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22021.034616 secs 2023-09-10T05:26:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:29 smithi082 ceph-mon[98694]: pgmap v11905: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 9 op/s 2023-09-10T05:26:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22021.034616 secs 2023-09-10T05:26:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:29 smithi098 ceph-mon[103124]: pgmap v11905: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 9 op/s 2023-09-10T05:26:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22021.034616 secs 2023-09-10T05:26:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:29 smithi170 ceph-mon[102832]: pgmap v11905: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 9 op/s 2023-09-10T05:26:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:30 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:26:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:30 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:26:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:30 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:26:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:31 smithi082 ceph-mon[98694]: pgmap v11906: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 218 KiB/s wr, 14 op/s 2023-09-10T05:26:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:31 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:26:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:26:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:26:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:31 smithi098 ceph-mon[103124]: pgmap v11906: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 218 KiB/s wr, 14 op/s 2023-09-10T05:26:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:31 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:26:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:26:32.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:26:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:31 smithi170 ceph-mon[102832]: pgmap v11906: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 218 KiB/s wr, 14 op/s 2023-09-10T05:26:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:31 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:26:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:26:32.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:26:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:33 smithi082 ceph-mon[98694]: pgmap v11907: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 218 KiB/s wr, 9 op/s 2023-09-10T05:26:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22025.380829 secs 2023-09-10T05:26:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:33 smithi098 ceph-mon[103124]: pgmap v11907: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 218 KiB/s wr, 9 op/s 2023-09-10T05:26:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22025.380829 secs 2023-09-10T05:26:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:33 smithi170 ceph-mon[102832]: pgmap v11907: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 218 KiB/s wr, 9 op/s 2023-09-10T05:26:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22025.380829 secs 2023-09-10T05:26:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22026.034678 secs 2023-09-10T05:26:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22026.034678 secs 2023-09-10T05:26:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22026.034678 secs 2023-09-10T05:26:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:35 smithi082 ceph-mon[98694]: pgmap v11908: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 218 KiB/s wr, 9 op/s 2023-09-10T05:26:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:26:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:35 smithi098 ceph-mon[103124]: pgmap v11908: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 218 KiB/s wr, 9 op/s 2023-09-10T05:26:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:26:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:35 smithi170 ceph-mon[102832]: pgmap v11908: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 218 KiB/s wr, 9 op/s 2023-09-10T05:26:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:26:37.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:26:37.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:26:37.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:26:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:26:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:26:37.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:26:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:26:37.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:26:37.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:26:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:37 smithi082 ceph-mon[98694]: pgmap v11909: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 371 KiB/s wr, 15 op/s 2023-09-10T05:26:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:26:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:26:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:37 smithi098 ceph-mon[103124]: pgmap v11909: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 371 KiB/s wr, 15 op/s 2023-09-10T05:26:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:26:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:26:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:37 smithi170 ceph-mon[102832]: pgmap v11909: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 371 KiB/s wr, 15 op/s 2023-09-10T05:26:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:26:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:26:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22030.380887 secs 2023-09-10T05:26:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22030.380887 secs 2023-09-10T05:26:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22030.380887 secs 2023-09-10T05:26:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22031.034740 secs 2023-09-10T05:26:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:39 smithi082 ceph-mon[98694]: pgmap v11910: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 297 KiB/s wr, 10 op/s 2023-09-10T05:26:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:39 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:26:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:39 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:26:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22031.034740 secs 2023-09-10T05:26:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:39 smithi098 ceph-mon[103124]: pgmap v11910: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 297 KiB/s wr, 10 op/s 2023-09-10T05:26:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:39 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:26:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:39 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:26:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22031.034740 secs 2023-09-10T05:26:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:39 smithi170 ceph-mon[102832]: pgmap v11910: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 297 KiB/s wr, 10 op/s 2023-09-10T05:26:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:39 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:26:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:39 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:26:42.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:41 smithi082 ceph-mon[98694]: pgmap v11911: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 411 KiB/s wr, 17 op/s 2023-09-10T05:26:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:41 smithi098 ceph-mon[103124]: pgmap v11911: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 411 KiB/s wr, 17 op/s 2023-09-10T05:26:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:41 smithi170 ceph-mon[102832]: pgmap v11911: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 411 KiB/s wr, 17 op/s 2023-09-10T05:26:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:43 smithi098 ceph-mon[103124]: pgmap v11912: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 267 KiB/s wr, 12 op/s 2023-09-10T05:26:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22035.380945 secs 2023-09-10T05:26:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:43 smithi170 ceph-mon[102832]: pgmap v11912: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 267 KiB/s wr, 12 op/s 2023-09-10T05:26:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22035.380945 secs 2023-09-10T05:26:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:43 smithi082 ceph-mon[98694]: pgmap v11912: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 267 KiB/s wr, 12 op/s 2023-09-10T05:26:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22035.380945 secs 2023-09-10T05:26:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22036.034801 secs 2023-09-10T05:26:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22036.034801 secs 2023-09-10T05:26:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22036.034801 secs 2023-09-10T05:26:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:45 smithi098 ceph-mon[103124]: pgmap v11913: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 267 KiB/s wr, 12 op/s 2023-09-10T05:26:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:45 smithi170 ceph-mon[102832]: pgmap v11913: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 267 KiB/s wr, 12 op/s 2023-09-10T05:26:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:45 smithi082 ceph-mon[98694]: pgmap v11913: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 267 KiB/s wr, 12 op/s 2023-09-10T05:26:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:47 smithi098 ceph-mon[103124]: pgmap v11914: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 462 KiB/s wr, 19 op/s 2023-09-10T05:26:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22039.342359 secs 2023-09-10T05:26:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:47 smithi170 ceph-mon[102832]: pgmap v11914: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 462 KiB/s wr, 19 op/s 2023-09-10T05:26:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22039.342359 secs 2023-09-10T05:26:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:47 smithi082 ceph-mon[98694]: pgmap v11914: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 462 KiB/s wr, 19 op/s 2023-09-10T05:26:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22039.342359 secs 2023-09-10T05:26:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22040.381029 secs 2023-09-10T05:26:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22040.381029 secs 2023-09-10T05:26:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22040.381029 secs 2023-09-10T05:26:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22041.034868 secs 2023-09-10T05:26:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:49 smithi098 ceph-mon[103124]: pgmap v11915: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 309 KiB/s wr, 13 op/s 2023-09-10T05:26:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:26:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:26:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22041.034868 secs 2023-09-10T05:26:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:49 smithi170 ceph-mon[102832]: pgmap v11915: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 309 KiB/s wr, 13 op/s 2023-09-10T05:26:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:26:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:26:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22041.034868 secs 2023-09-10T05:26:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:49 smithi082 ceph-mon[98694]: pgmap v11915: 97 pgs: 97 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 309 KiB/s wr, 13 op/s 2023-09-10T05:26:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:26:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:26:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:26:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:26:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:26:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:26:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:26:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:26:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:26:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:26:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:26:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:52 smithi170 ceph-mon[102832]: pgmap v11916: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 368 KiB/s wr, 19 op/s 2023-09-10T05:26:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:52 smithi082 ceph-mon[98694]: pgmap v11916: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 368 KiB/s wr, 19 op/s 2023-09-10T05:26:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:52 smithi098 ceph-mon[103124]: pgmap v11916: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 368 KiB/s wr, 19 op/s 2023-09-10T05:26:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22044.342418 secs 2023-09-10T05:26:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22044.342418 secs 2023-09-10T05:26:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22044.342418 secs 2023-09-10T05:26:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:54 smithi170 ceph-mon[102832]: pgmap v11917: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 254 KiB/s wr, 12 op/s 2023-09-10T05:26:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22045.381112 secs 2023-09-10T05:26:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:54 smithi082 ceph-mon[98694]: pgmap v11917: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 254 KiB/s wr, 12 op/s 2023-09-10T05:26:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22045.381112 secs 2023-09-10T05:26:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:54 smithi098 ceph-mon[103124]: pgmap v11917: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 254 KiB/s wr, 12 op/s 2023-09-10T05:26:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22045.381112 secs 2023-09-10T05:26:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22046.034929 secs 2023-09-10T05:26:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22046.034929 secs 2023-09-10T05:26:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22046.034929 secs 2023-09-10T05:26:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:56 smithi170 ceph-mon[102832]: pgmap v11918: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 254 KiB/s wr, 12 op/s 2023-09-10T05:26:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:56 smithi082 ceph-mon[98694]: pgmap v11918: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 254 KiB/s wr, 12 op/s 2023-09-10T05:26:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:56 smithi098 ceph-mon[103124]: pgmap v11918: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 254 KiB/s wr, 12 op/s 2023-09-10T05:26:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:58 smithi170 ceph-mon[102832]: pgmap v11919: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 422 KiB/s wr, 18 op/s 2023-09-10T05:26:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22049.342484 secs 2023-09-10T05:26:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:58 smithi082 ceph-mon[98694]: pgmap v11919: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 422 KiB/s wr, 18 op/s 2023-09-10T05:26:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22049.342484 secs 2023-09-10T05:26:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:58 smithi098 ceph-mon[103124]: pgmap v11919: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 422 KiB/s wr, 18 op/s 2023-09-10T05:26:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22049.342484 secs 2023-09-10T05:26:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22050.381211 secs 2023-09-10T05:26:59.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:26:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22051.034997 secs 2023-09-10T05:26:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22050.381211 secs 2023-09-10T05:26:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:26:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22051.034997 secs 2023-09-10T05:26:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22050.381211 secs 2023-09-10T05:26:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:26:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22051.034997 secs 2023-09-10T05:27:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:00 smithi170 ceph-mon[102832]: pgmap v11920: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 228 KiB/s wr, 11 op/s 2023-09-10T05:27:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:00 smithi082 ceph-mon[98694]: pgmap v11920: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 228 KiB/s wr, 11 op/s 2023-09-10T05:27:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:00 smithi098 ceph-mon[103124]: pgmap v11920: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 228 KiB/s wr, 11 op/s 2023-09-10T05:27:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:02 smithi170 ceph-mon[102832]: pgmap v11921: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 284 KiB/s wr, 16 op/s 2023-09-10T05:27:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:02 smithi082 ceph-mon[98694]: pgmap v11921: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 284 KiB/s wr, 16 op/s 2023-09-10T05:27:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:02 smithi098 ceph-mon[103124]: pgmap v11921: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 284 KiB/s wr, 16 op/s 2023-09-10T05:27:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22054.342557 secs 2023-09-10T05:27:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22054.342557 secs 2023-09-10T05:27:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22054.342557 secs 2023-09-10T05:27:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:04 smithi170 ceph-mon[102832]: pgmap v11922: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 225 KiB/s wr, 10 op/s 2023-09-10T05:27:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22055.381271 secs 2023-09-10T05:27:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22056.035059 secs 2023-09-10T05:27:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:04 smithi082 ceph-mon[98694]: pgmap v11922: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 225 KiB/s wr, 10 op/s 2023-09-10T05:27:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22055.381271 secs 2023-09-10T05:27:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22056.035059 secs 2023-09-10T05:27:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:04 smithi098 ceph-mon[103124]: pgmap v11922: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 225 KiB/s wr, 10 op/s 2023-09-10T05:27:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22055.381271 secs 2023-09-10T05:27:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22056.035059 secs 2023-09-10T05:27:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:05 smithi098 ceph-mon[103124]: pgmap v11923: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 225 KiB/s wr, 10 op/s 2023-09-10T05:27:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:05 smithi170 ceph-mon[102832]: pgmap v11923: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 225 KiB/s wr, 10 op/s 2023-09-10T05:27:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:05 smithi082 ceph-mon[98694]: pgmap v11923: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 225 KiB/s wr, 10 op/s 2023-09-10T05:27:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:07 smithi170 ceph-mon[102832]: pgmap v11924: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 270 KiB/s wr, 29 op/s 2023-09-10T05:27:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22059.342681 secs 2023-09-10T05:27:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:07 smithi082 ceph-mon[98694]: pgmap v11924: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 270 KiB/s wr, 29 op/s 2023-09-10T05:27:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22059.342681 secs 2023-09-10T05:27:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:07 smithi098 ceph-mon[103124]: pgmap v11924: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 270 KiB/s wr, 29 op/s 2023-09-10T05:27:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22059.342681 secs 2023-09-10T05:27:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:08 smithi170 ceph-mon[102832]: osdmap e813: 12 total, 12 up, 12 in 2023-09-10T05:27:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22060.381334 secs 2023-09-10T05:27:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:08 smithi082 ceph-mon[98694]: osdmap e813: 12 total, 12 up, 12 in 2023-09-10T05:27:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22060.381334 secs 2023-09-10T05:27:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:08 smithi098 ceph-mon[103124]: osdmap e813: 12 total, 12 up, 12 in 2023-09-10T05:27:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22060.381334 secs 2023-09-10T05:27:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22061.035120 secs 2023-09-10T05:27:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:09 smithi170 ceph-mon[102832]: pgmap v11926: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 28 op/s 2023-09-10T05:27:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22061.035120 secs 2023-09-10T05:27:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:09 smithi082 ceph-mon[98694]: pgmap v11926: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 28 op/s 2023-09-10T05:27:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22061.035120 secs 2023-09-10T05:27:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:09 smithi098 ceph-mon[103124]: pgmap v11926: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 28 op/s 2023-09-10T05:27:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:11 smithi170 ceph-mon[102832]: pgmap v11927: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2023-09-10T05:27:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:11 smithi082 ceph-mon[98694]: pgmap v11927: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2023-09-10T05:27:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:11 smithi098 ceph-mon[103124]: pgmap v11927: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2023-09-10T05:27:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22064.342773 secs 2023-09-10T05:27:13.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22064.342773 secs 2023-09-10T05:27:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22064.342773 secs 2023-09-10T05:27:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:13 smithi170 ceph-mon[102832]: pgmap v11928: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2023-09-10T05:27:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22065.381393 secs 2023-09-10T05:27:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:13 smithi082 ceph-mon[98694]: pgmap v11928: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2023-09-10T05:27:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22065.381393 secs 2023-09-10T05:27:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:13 smithi098 ceph-mon[103124]: pgmap v11928: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2023-09-10T05:27:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22065.381393 secs 2023-09-10T05:27:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22066.035187 secs 2023-09-10T05:27:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:14 smithi170 ceph-mon[102832]: osdmap e814: 12 total, 12 up, 12 in 2023-09-10T05:27:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22066.035187 secs 2023-09-10T05:27:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:14 smithi082 ceph-mon[98694]: osdmap e814: 12 total, 12 up, 12 in 2023-09-10T05:27:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22066.035187 secs 2023-09-10T05:27:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:14 smithi098 ceph-mon[103124]: osdmap e814: 12 total, 12 up, 12 in 2023-09-10T05:27:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:15 smithi170 ceph-mon[102832]: pgmap v11930: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:27:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:15 smithi082 ceph-mon[98694]: pgmap v11930: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:27:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:15 smithi098 ceph-mon[103124]: pgmap v11930: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:27:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:17 smithi170 ceph-mon[102832]: pgmap v11931: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 343 B/s wr, 11 op/s 2023-09-10T05:27:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22069.342840 secs 2023-09-10T05:27:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:17 smithi082 ceph-mon[98694]: pgmap v11931: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 343 B/s wr, 11 op/s 2023-09-10T05:27:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22069.342840 secs 2023-09-10T05:27:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:17 smithi098 ceph-mon[103124]: pgmap v11931: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 343 B/s wr, 11 op/s 2023-09-10T05:27:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22069.342840 secs 2023-09-10T05:27:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22070.381453 secs 2023-09-10T05:27:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22070.381453 secs 2023-09-10T05:27:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22070.381453 secs 2023-09-10T05:27:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22071.035252 secs 2023-09-10T05:27:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:19 smithi170 ceph-mon[102832]: pgmap v11932: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T05:27:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22071.035252 secs 2023-09-10T05:27:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:19 smithi082 ceph-mon[98694]: pgmap v11932: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T05:27:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22071.035252 secs 2023-09-10T05:27:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:19 smithi098 ceph-mon[103124]: pgmap v11932: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T05:27:22.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:21 smithi082 ceph-mon[98694]: pgmap v11933: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:27:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:21 smithi098 ceph-mon[103124]: pgmap v11933: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:27:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:21 smithi170 ceph-mon[102832]: pgmap v11933: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:27:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22074.342901 secs 2023-09-10T05:27:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22074.342901 secs 2023-09-10T05:27:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22074.342901 secs 2023-09-10T05:27:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:23 smithi082 ceph-mon[98694]: pgmap v11934: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:27:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22075.381493 secs 2023-09-10T05:27:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:23 smithi098 ceph-mon[103124]: pgmap v11934: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:27:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22075.381493 secs 2023-09-10T05:27:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:23 smithi170 ceph-mon[102832]: pgmap v11934: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:27:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22075.381493 secs 2023-09-10T05:27:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22076.035313 secs 2023-09-10T05:27:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22076.035313 secs 2023-09-10T05:27:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22076.035313 secs 2023-09-10T05:27:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:25 smithi082 ceph-mon[98694]: pgmap v11935: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:27:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:25 smithi098 ceph-mon[103124]: pgmap v11935: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:27:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:25 smithi170 ceph-mon[102832]: pgmap v11935: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:27:28.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:27 smithi082 ceph-mon[98694]: pgmap v11936: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 68 KiB/s wr, 14 op/s 2023-09-10T05:27:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:27 smithi098 ceph-mon[103124]: pgmap v11936: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 68 KiB/s wr, 14 op/s 2023-09-10T05:27:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:27 smithi170 ceph-mon[102832]: pgmap v11936: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 68 KiB/s wr, 14 op/s 2023-09-10T05:27:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22080.381551 secs 2023-09-10T05:27:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22080.381551 secs 2023-09-10T05:27:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22080.381551 secs 2023-09-10T05:27:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22081.035375 secs 2023-09-10T05:27:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:29 smithi082 ceph-mon[98694]: pgmap v11937: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T05:27:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:29 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:27:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:29 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:27:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22081.035375 secs 2023-09-10T05:27:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:29 smithi098 ceph-mon[103124]: pgmap v11937: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T05:27:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:29 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:27:30.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:29 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:27:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22081.035375 secs 2023-09-10T05:27:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:29 smithi170 ceph-mon[102832]: pgmap v11937: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 67 KiB/s wr, 10 op/s 2023-09-10T05:27:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:29 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:27:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:29 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:27:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:30 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:27:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:30 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:27:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:30 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:27:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:31 smithi082 ceph-mon[98694]: pgmap v11938: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 181 KiB/s wr, 16 op/s 2023-09-10T05:27:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:31 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:27:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:27:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:27:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:31 smithi098 ceph-mon[103124]: pgmap v11938: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 181 KiB/s wr, 16 op/s 2023-09-10T05:27:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:31 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:27:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:27:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:27:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:31 smithi170 ceph-mon[102832]: pgmap v11938: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 181 KiB/s wr, 16 op/s 2023-09-10T05:27:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:31 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:27:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:27:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:27:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:33 smithi082 ceph-mon[98694]: pgmap v11939: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T05:27:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22085.381610 secs 2023-09-10T05:27:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:33 smithi098 ceph-mon[103124]: pgmap v11939: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T05:27:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22085.381610 secs 2023-09-10T05:27:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:33 smithi170 ceph-mon[102832]: pgmap v11939: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T05:27:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22085.381610 secs 2023-09-10T05:27:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22086.035439 secs 2023-09-10T05:27:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22086.035439 secs 2023-09-10T05:27:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22086.035439 secs 2023-09-10T05:27:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:35 smithi098 ceph-mon[103124]: pgmap v11940: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T05:27:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:35 smithi170 ceph-mon[102832]: pgmap v11940: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T05:27:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:35 smithi082 ceph-mon[98694]: pgmap v11940: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T05:27:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:27:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:27:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:27:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:37 smithi098 ceph-mon[103124]: pgmap v11941: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 282 KiB/s wr, 17 op/s 2023-09-10T05:27:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:27:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:27:38.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:27:38.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:27:38.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:27:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:37 smithi170 ceph-mon[102832]: pgmap v11941: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 282 KiB/s wr, 17 op/s 2023-09-10T05:27:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:27:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:27:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:27:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:27:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:27:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:37 smithi082 ceph-mon[98694]: pgmap v11941: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 282 KiB/s wr, 17 op/s 2023-09-10T05:27:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:27:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:27:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:27:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:27:38.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:27:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22090.381685 secs 2023-09-10T05:27:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22090.381685 secs 2023-09-10T05:27:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22090.381685 secs 2023-09-10T05:27:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22091.035487 secs 2023-09-10T05:27:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:39 smithi098 ceph-mon[103124]: pgmap v11942: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T05:27:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:39 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:27:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:39 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:27:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22091.035487 secs 2023-09-10T05:27:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:39 smithi170 ceph-mon[102832]: pgmap v11942: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T05:27:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:39 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:27:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:39 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:27:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22091.035487 secs 2023-09-10T05:27:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:39 smithi082 ceph-mon[98694]: pgmap v11942: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T05:27:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:39 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:27:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:39 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:27:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:41 smithi098 ceph-mon[103124]: pgmap v11943: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 501 KiB/s wr, 19 op/s 2023-09-10T05:27:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:41 smithi170 ceph-mon[102832]: pgmap v11943: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 501 KiB/s wr, 19 op/s 2023-09-10T05:27:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:41 smithi082 ceph-mon[98694]: pgmap v11943: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 501 KiB/s wr, 19 op/s 2023-09-10T05:27:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22094.343216 secs 2023-09-10T05:27:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22094.343216 secs 2023-09-10T05:27:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22094.343216 secs 2023-09-10T05:27:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:43 smithi098 ceph-mon[103124]: pgmap v11944: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 388 KiB/s wr, 13 op/s 2023-09-10T05:27:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:27:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:27:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:27:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22095.381768 secs 2023-09-10T05:27:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:43 smithi170 ceph-mon[102832]: pgmap v11944: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 388 KiB/s wr, 13 op/s 2023-09-10T05:27:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:27:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:27:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:27:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22095.381768 secs 2023-09-10T05:27:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:43 smithi082 ceph-mon[98694]: pgmap v11944: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 388 KiB/s wr, 13 op/s 2023-09-10T05:27:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:27:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:27:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:27:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22095.381768 secs 2023-09-10T05:27:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22096.035551 secs 2023-09-10T05:27:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22096.035551 secs 2023-09-10T05:27:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22096.035551 secs 2023-09-10T05:27:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:45 smithi098 ceph-mon[103124]: pgmap v11945: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 388 KiB/s wr, 13 op/s 2023-09-10T05:27:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:45 smithi170 ceph-mon[102832]: pgmap v11945: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 388 KiB/s wr, 13 op/s 2023-09-10T05:27:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:45 smithi082 ceph-mon[98694]: pgmap v11945: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 388 KiB/s wr, 13 op/s 2023-09-10T05:27:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:47 smithi098 ceph-mon[103124]: pgmap v11946: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 489 KiB/s wr, 20 op/s 2023-09-10T05:27:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22099.343276 secs 2023-09-10T05:27:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:47 smithi170 ceph-mon[102832]: pgmap v11946: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 489 KiB/s wr, 20 op/s 2023-09-10T05:27:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22099.343276 secs 2023-09-10T05:27:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:47 smithi082 ceph-mon[98694]: pgmap v11946: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 489 KiB/s wr, 20 op/s 2023-09-10T05:27:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22099.343276 secs 2023-09-10T05:27:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22100.381856 secs 2023-09-10T05:27:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22100.381856 secs 2023-09-10T05:27:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22100.381856 secs 2023-09-10T05:27:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22101.035614 secs 2023-09-10T05:27:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:49 smithi098 ceph-mon[103124]: pgmap v11947: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 321 KiB/s wr, 13 op/s 2023-09-10T05:27:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22101.035614 secs 2023-09-10T05:27:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:49 smithi170 ceph-mon[102832]: pgmap v11947: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 321 KiB/s wr, 13 op/s 2023-09-10T05:27:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22101.035614 secs 2023-09-10T05:27:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:49 smithi082 ceph-mon[98694]: pgmap v11947: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 321 KiB/s wr, 13 op/s 2023-09-10T05:27:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:51 smithi098 ceph-mon[103124]: pgmap v11948: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 427 KiB/s wr, 18 op/s 2023-09-10T05:27:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:51 smithi170 ceph-mon[102832]: pgmap v11948: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 427 KiB/s wr, 18 op/s 2023-09-10T05:27:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:51 smithi082 ceph-mon[98694]: pgmap v11948: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 427 KiB/s wr, 18 op/s 2023-09-10T05:27:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22104.343336 secs 2023-09-10T05:27:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22104.343336 secs 2023-09-10T05:27:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22104.343336 secs 2023-09-10T05:27:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:54 smithi098 ceph-mon[103124]: pgmap v11949: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 207 KiB/s wr, 11 op/s 2023-09-10T05:27:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22105.381914 secs 2023-09-10T05:27:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:54 smithi170 ceph-mon[102832]: pgmap v11949: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 207 KiB/s wr, 11 op/s 2023-09-10T05:27:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22105.381914 secs 2023-09-10T05:27:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:54 smithi082 ceph-mon[98694]: pgmap v11949: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 207 KiB/s wr, 11 op/s 2023-09-10T05:27:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22105.381914 secs 2023-09-10T05:27:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22106.035703 secs 2023-09-10T05:27:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22106.035703 secs 2023-09-10T05:27:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22106.035703 secs 2023-09-10T05:27:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:56 smithi170 ceph-mon[102832]: pgmap v11950: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 207 KiB/s wr, 11 op/s 2023-09-10T05:27:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:56 smithi082 ceph-mon[98694]: pgmap v11950: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 207 KiB/s wr, 11 op/s 2023-09-10T05:27:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:56 smithi098 ceph-mon[103124]: pgmap v11950: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 207 KiB/s wr, 11 op/s 2023-09-10T05:27:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:58 smithi170 ceph-mon[102832]: pgmap v11951: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 285 KiB/s wr, 16 op/s 2023-09-10T05:27:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:27:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:27:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22109.343397 secs 2023-09-10T05:27:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:58 smithi082 ceph-mon[98694]: pgmap v11951: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 285 KiB/s wr, 16 op/s 2023-09-10T05:27:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:27:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:27:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22109.343397 secs 2023-09-10T05:27:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:58 smithi098 ceph-mon[103124]: pgmap v11951: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 285 KiB/s wr, 16 op/s 2023-09-10T05:27:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:27:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:27:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22109.343397 secs 2023-09-10T05:27:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:27:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22110.381975 secs 2023-09-10T05:27:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:27:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22110.381975 secs 2023-09-10T05:27:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:27:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22110.381975 secs 2023-09-10T05:28:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22111.035765 secs 2023-09-10T05:28:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:00 smithi170 ceph-mon[102832]: pgmap v11952: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 184 KiB/s wr, 10 op/s 2023-09-10T05:28:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:28:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:28:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22111.035765 secs 2023-09-10T05:28:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:00 smithi082 ceph-mon[98694]: pgmap v11952: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 184 KiB/s wr, 10 op/s 2023-09-10T05:28:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:28:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:28:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22111.035765 secs 2023-09-10T05:28:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:00 smithi098 ceph-mon[103124]: pgmap v11952: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 184 KiB/s wr, 10 op/s 2023-09-10T05:28:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:28:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:28:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:02 smithi170 ceph-mon[102832]: pgmap v11953: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 184 KiB/s wr, 14 op/s 2023-09-10T05:28:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:02 smithi082 ceph-mon[98694]: pgmap v11953: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 184 KiB/s wr, 14 op/s 2023-09-10T05:28:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:02 smithi098 ceph-mon[103124]: pgmap v11953: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 184 KiB/s wr, 14 op/s 2023-09-10T05:28:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22114.343474 secs 2023-09-10T05:28:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22114.343474 secs 2023-09-10T05:28:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22114.343474 secs 2023-09-10T05:28:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:04 smithi170 ceph-mon[102832]: pgmap v11954: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 9 op/s 2023-09-10T05:28:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22115.382033 secs 2023-09-10T05:28:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:04 smithi082 ceph-mon[98694]: pgmap v11954: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 9 op/s 2023-09-10T05:28:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22115.382033 secs 2023-09-10T05:28:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:04 smithi098 ceph-mon[103124]: pgmap v11954: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 9 op/s 2023-09-10T05:28:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22115.382033 secs 2023-09-10T05:28:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22116.035828 secs 2023-09-10T05:28:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22116.035828 secs 2023-09-10T05:28:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22116.035828 secs 2023-09-10T05:28:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:06 smithi170 ceph-mon[102832]: pgmap v11955: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 9 op/s 2023-09-10T05:28:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:06 smithi082 ceph-mon[98694]: pgmap v11955: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 9 op/s 2023-09-10T05:28:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:06 smithi098 ceph-mon[103124]: pgmap v11955: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 9 op/s 2023-09-10T05:28:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:08 smithi170 ceph-mon[102832]: pgmap v11956: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 179 KiB/s wr, 28 op/s 2023-09-10T05:28:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22119.343585 secs 2023-09-10T05:28:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:08 smithi082 ceph-mon[98694]: pgmap v11956: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 179 KiB/s wr, 28 op/s 2023-09-10T05:28:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22119.343585 secs 2023-09-10T05:28:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:08 smithi098 ceph-mon[103124]: pgmap v11956: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 179 KiB/s wr, 28 op/s 2023-09-10T05:28:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22119.343585 secs 2023-09-10T05:28:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:09 smithi170 ceph-mon[102832]: osdmap e815: 12 total, 12 up, 12 in 2023-09-10T05:28:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22120.382093 secs 2023-09-10T05:28:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:09 smithi082 ceph-mon[98694]: osdmap e815: 12 total, 12 up, 12 in 2023-09-10T05:28:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22120.382093 secs 2023-09-10T05:28:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:09 smithi098 ceph-mon[103124]: osdmap e815: 12 total, 12 up, 12 in 2023-09-10T05:28:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22120.382093 secs 2023-09-10T05:28:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22121.035892 secs 2023-09-10T05:28:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:10 smithi170 ceph-mon[102832]: pgmap v11958: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 28 op/s 2023-09-10T05:28:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22121.035892 secs 2023-09-10T05:28:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:10 smithi082 ceph-mon[98694]: pgmap v11958: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 28 op/s 2023-09-10T05:28:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22121.035892 secs 2023-09-10T05:28:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:10 smithi098 ceph-mon[103124]: pgmap v11958: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 28 op/s 2023-09-10T05:28:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:12 smithi170 ceph-mon[102832]: pgmap v11959: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 28 op/s 2023-09-10T05:28:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:12 smithi082 ceph-mon[98694]: pgmap v11959: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 28 op/s 2023-09-10T05:28:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:12 smithi098 ceph-mon[103124]: pgmap v11959: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 28 op/s 2023-09-10T05:28:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22124.343645 secs 2023-09-10T05:28:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22124.343645 secs 2023-09-10T05:28:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22124.343645 secs 2023-09-10T05:28:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:14 smithi170 ceph-mon[102832]: pgmap v11960: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 28 op/s 2023-09-10T05:28:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22125.382153 secs 2023-09-10T05:28:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:14 smithi082 ceph-mon[98694]: pgmap v11960: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 28 op/s 2023-09-10T05:28:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22125.382153 secs 2023-09-10T05:28:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:14 smithi098 ceph-mon[103124]: pgmap v11960: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 28 op/s 2023-09-10T05:28:14.550 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22125.382153 secs 2023-09-10T05:28:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22126.035955 secs 2023-09-10T05:28:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:15 smithi170 ceph-mon[102832]: osdmap e816: 12 total, 12 up, 12 in 2023-09-10T05:28:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22126.035955 secs 2023-09-10T05:28:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:15 smithi082 ceph-mon[98694]: osdmap e816: 12 total, 12 up, 12 in 2023-09-10T05:28:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22126.035955 secs 2023-09-10T05:28:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:15 smithi098 ceph-mon[103124]: osdmap e816: 12 total, 12 up, 12 in 2023-09-10T05:28:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:16 smithi170 ceph-mon[102832]: pgmap v11962: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:28:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:16 smithi082 ceph-mon[98694]: pgmap v11962: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:28:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:16 smithi098 ceph-mon[103124]: pgmap v11962: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:28:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:18 smithi170 ceph-mon[102832]: pgmap v11963: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 232 B/s wr, 11 op/s 2023-09-10T05:28:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22129.343706 secs 2023-09-10T05:28:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:18 smithi082 ceph-mon[98694]: pgmap v11963: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 232 B/s wr, 11 op/s 2023-09-10T05:28:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22129.343706 secs 2023-09-10T05:28:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:18 smithi098 ceph-mon[103124]: pgmap v11963: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 232 B/s wr, 11 op/s 2023-09-10T05:28:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22129.343706 secs 2023-09-10T05:28:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22130.382222 secs 2023-09-10T05:28:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22130.382222 secs 2023-09-10T05:28:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22130.382222 secs 2023-09-10T05:28:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22131.036017 secs 2023-09-10T05:28:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:20 smithi170 ceph-mon[102832]: pgmap v11964: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:28:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22131.036017 secs 2023-09-10T05:28:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:20 smithi082 ceph-mon[98694]: pgmap v11964: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:28:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22131.036017 secs 2023-09-10T05:28:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:20 smithi098 ceph-mon[103124]: pgmap v11964: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:28:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:22 smithi170 ceph-mon[102832]: pgmap v11965: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:28:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:22 smithi082 ceph-mon[98694]: pgmap v11965: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:28:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:22 smithi098 ceph-mon[103124]: pgmap v11965: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:28:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22134.343771 secs 2023-09-10T05:28:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22134.343771 secs 2023-09-10T05:28:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22134.343771 secs 2023-09-10T05:28:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:24 smithi170 ceph-mon[102832]: pgmap v11966: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:28:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22135.382279 secs 2023-09-10T05:28:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22136.036079 secs 2023-09-10T05:28:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:24 smithi082 ceph-mon[98694]: pgmap v11966: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:28:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22135.382279 secs 2023-09-10T05:28:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22136.036079 secs 2023-09-10T05:28:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:24 smithi098 ceph-mon[103124]: pgmap v11966: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:28:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22135.382279 secs 2023-09-10T05:28:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22136.036079 secs 2023-09-10T05:28:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:25 smithi170 ceph-mon[102832]: pgmap v11967: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 202 B/s wr, 10 op/s 2023-09-10T05:28:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:25 smithi082 ceph-mon[98694]: pgmap v11967: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 202 B/s wr, 10 op/s 2023-09-10T05:28:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:25 smithi098 ceph-mon[103124]: pgmap v11967: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 202 B/s wr, 10 op/s 2023-09-10T05:28:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:27 smithi170 ceph-mon[102832]: pgmap v11968: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 13 op/s 2023-09-10T05:28:28.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:27 smithi082 ceph-mon[98694]: pgmap v11968: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 13 op/s 2023-09-10T05:28:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:27 smithi098 ceph-mon[103124]: pgmap v11968: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 13 op/s 2023-09-10T05:28:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22140.382368 secs 2023-09-10T05:28:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22140.382368 secs 2023-09-10T05:28:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22140.382368 secs 2023-09-10T05:28:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22141.036143 secs 2023-09-10T05:28:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:29 smithi170 ceph-mon[102832]: pgmap v11969: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-10T05:28:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22141.036143 secs 2023-09-10T05:28:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:29 smithi082 ceph-mon[98694]: pgmap v11969: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-10T05:28:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22141.036143 secs 2023-09-10T05:28:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:29 smithi098 ceph-mon[103124]: pgmap v11969: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-10T05:28:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:30 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:28:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:30 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:28:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:30 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:28:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:31 smithi170 ceph-mon[102832]: pgmap v11970: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 114 KiB/s wr, 15 op/s 2023-09-10T05:28:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:31 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:28:32.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:28:32.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:28:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:31 smithi082 ceph-mon[98694]: pgmap v11970: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 114 KiB/s wr, 15 op/s 2023-09-10T05:28:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:31 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:28:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:28:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:28:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:31 smithi098 ceph-mon[103124]: pgmap v11970: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 114 KiB/s wr, 15 op/s 2023-09-10T05:28:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:31 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:28:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:28:32.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:28:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:33 smithi170 ceph-mon[102832]: pgmap v11971: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 10 op/s 2023-09-10T05:28:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22145.382441 secs 2023-09-10T05:28:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:33 smithi082 ceph-mon[98694]: pgmap v11971: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 10 op/s 2023-09-10T05:28:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22145.382441 secs 2023-09-10T05:28:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:33 smithi098 ceph-mon[103124]: pgmap v11971: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 10 op/s 2023-09-10T05:28:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22145.382441 secs 2023-09-10T05:28:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22146.036203 secs 2023-09-10T05:28:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22146.036203 secs 2023-09-10T05:28:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22146.036203 secs 2023-09-10T05:28:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:35 smithi082 ceph-mon[98694]: pgmap v11972: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 10 op/s 2023-09-10T05:28:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:35 smithi098 ceph-mon[103124]: pgmap v11972: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 10 op/s 2023-09-10T05:28:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:35 smithi170 ceph-mon[102832]: pgmap v11972: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 10 op/s 2023-09-10T05:28:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:37 smithi082 ceph-mon[98694]: pgmap v11973: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 305 KiB/s wr, 17 op/s 2023-09-10T05:28:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:28:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:28:38.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:28:38.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:28:38.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:28:38.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:28:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:37 smithi098 ceph-mon[103124]: pgmap v11973: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 305 KiB/s wr, 17 op/s 2023-09-10T05:28:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:28:38.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:28:38.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:28:38.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:28:38.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:28:38.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:28:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:37 smithi170 ceph-mon[102832]: pgmap v11973: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 305 KiB/s wr, 17 op/s 2023-09-10T05:28:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:28:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:28:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:28:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:28:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:28:38.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:28:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22150.382502 secs 2023-09-10T05:28:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22150.382502 secs 2023-09-10T05:28:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22150.382502 secs 2023-09-10T05:28:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22151.036266 secs 2023-09-10T05:28:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:39 smithi082 ceph-mon[98694]: pgmap v11974: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 304 KiB/s wr, 12 op/s 2023-09-10T05:28:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:39 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:28:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:39 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:28:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22151.036266 secs 2023-09-10T05:28:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:39 smithi098 ceph-mon[103124]: pgmap v11974: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 304 KiB/s wr, 12 op/s 2023-09-10T05:28:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:39 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:28:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:39 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:28:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22151.036266 secs 2023-09-10T05:28:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:39 smithi170 ceph-mon[102832]: pgmap v11974: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 304 KiB/s wr, 12 op/s 2023-09-10T05:28:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:39 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:28:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:39 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:28:42.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:41 smithi082 ceph-mon[98694]: pgmap v11975: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 504 KiB/s wr, 19 op/s 2023-09-10T05:28:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:41 smithi098 ceph-mon[103124]: pgmap v11975: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 504 KiB/s wr, 19 op/s 2023-09-10T05:28:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:41 smithi170 ceph-mon[102832]: pgmap v11975: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 504 KiB/s wr, 19 op/s 2023-09-10T05:28:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:42 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22154.344035 secs 2023-09-10T05:28:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:42 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22154.344035 secs 2023-09-10T05:28:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:42 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22154.344035 secs 2023-09-10T05:28:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:43 smithi082 ceph-mon[98694]: pgmap v11976: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 390 KiB/s wr, 13 op/s 2023-09-10T05:28:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:28:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:28:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:28:44.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22155.382560 secs 2023-09-10T05:28:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:43 smithi098 ceph-mon[103124]: pgmap v11976: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 390 KiB/s wr, 13 op/s 2023-09-10T05:28:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:28:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:28:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:28:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22155.382560 secs 2023-09-10T05:28:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:43 smithi170 ceph-mon[102832]: pgmap v11976: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 390 KiB/s wr, 13 op/s 2023-09-10T05:28:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:28:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:28:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:28:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22155.382560 secs 2023-09-10T05:28:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22156.036352 secs 2023-09-10T05:28:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22156.036352 secs 2023-09-10T05:28:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22156.036352 secs 2023-09-10T05:28:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:45 smithi082 ceph-mon[98694]: pgmap v11977: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 390 KiB/s wr, 13 op/s 2023-09-10T05:28:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:45 smithi098 ceph-mon[103124]: pgmap v11977: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 390 KiB/s wr, 13 op/s 2023-09-10T05:28:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:45 smithi170 ceph-mon[102832]: pgmap v11977: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 390 KiB/s wr, 13 op/s 2023-09-10T05:28:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:47 smithi082 ceph-mon[98694]: pgmap v11978: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 491 KiB/s wr, 20 op/s 2023-09-10T05:28:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22159.344122 secs 2023-09-10T05:28:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:47 smithi098 ceph-mon[103124]: pgmap v11978: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 491 KiB/s wr, 20 op/s 2023-09-10T05:28:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22159.344122 secs 2023-09-10T05:28:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:47 smithi170 ceph-mon[102832]: pgmap v11978: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 491 KiB/s wr, 20 op/s 2023-09-10T05:28:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22159.344122 secs 2023-09-10T05:28:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22160.382650 secs 2023-09-10T05:28:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22160.382650 secs 2023-09-10T05:28:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22160.382650 secs 2023-09-10T05:28:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22161.036414 secs 2023-09-10T05:28:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:49 smithi098 ceph-mon[103124]: pgmap v11979: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 301 KiB/s wr, 13 op/s 2023-09-10T05:28:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:28:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:28:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22161.036414 secs 2023-09-10T05:28:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:49 smithi170 ceph-mon[102832]: pgmap v11979: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 301 KiB/s wr, 13 op/s 2023-09-10T05:28:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:28:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:28:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22161.036414 secs 2023-09-10T05:28:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:49 smithi082 ceph-mon[98694]: pgmap v11979: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 301 KiB/s wr, 13 op/s 2023-09-10T05:28:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:28:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:28:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:51 smithi098 ceph-mon[103124]: pgmap v11980: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 409 KiB/s wr, 20 op/s 2023-09-10T05:28:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:28:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:28:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:51 smithi170 ceph-mon[102832]: pgmap v11980: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 409 KiB/s wr, 20 op/s 2023-09-10T05:28:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:28:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:28:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:51 smithi082 ceph-mon[98694]: pgmap v11980: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 409 KiB/s wr, 20 op/s 2023-09-10T05:28:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:28:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:28:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:52 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22164.344207 secs 2023-09-10T05:28:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:52 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22164.344207 secs 2023-09-10T05:28:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:52 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22164.344207 secs 2023-09-10T05:28:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:53 smithi098 ceph-mon[103124]: pgmap v11981: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 210 KiB/s wr, 13 op/s 2023-09-10T05:28:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22165.382708 secs 2023-09-10T05:28:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:53 smithi170 ceph-mon[102832]: pgmap v11981: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 210 KiB/s wr, 13 op/s 2023-09-10T05:28:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22165.382708 secs 2023-09-10T05:28:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:53 smithi082 ceph-mon[98694]: pgmap v11981: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 210 KiB/s wr, 13 op/s 2023-09-10T05:28:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22165.382708 secs 2023-09-10T05:28:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22166.036480 secs 2023-09-10T05:28:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22166.036480 secs 2023-09-10T05:28:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22166.036480 secs 2023-09-10T05:28:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:55 smithi098 ceph-mon[103124]: pgmap v11982: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 210 KiB/s wr, 13 op/s 2023-09-10T05:28:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:55 smithi170 ceph-mon[102832]: pgmap v11982: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 210 KiB/s wr, 13 op/s 2023-09-10T05:28:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:55 smithi082 ceph-mon[98694]: pgmap v11982: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 210 KiB/s wr, 13 op/s 2023-09-10T05:28:58.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:57 smithi098 ceph-mon[103124]: pgmap v11983: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 289 KiB/s wr, 18 op/s 2023-09-10T05:28:58.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22169.344266 secs 2023-09-10T05:28:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:57 smithi170 ceph-mon[102832]: pgmap v11983: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 289 KiB/s wr, 18 op/s 2023-09-10T05:28:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22169.344266 secs 2023-09-10T05:28:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:57 smithi082 ceph-mon[98694]: pgmap v11983: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 289 KiB/s wr, 18 op/s 2023-09-10T05:28:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22169.344266 secs 2023-09-10T05:28:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22170.382792 secs 2023-09-10T05:28:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22170.382792 secs 2023-09-10T05:28:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22170.382792 secs 2023-09-10T05:29:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22171.036542 secs 2023-09-10T05:29:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:28:59 smithi098 ceph-mon[103124]: pgmap v11984: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 187 KiB/s wr, 11 op/s 2023-09-10T05:29:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22171.036542 secs 2023-09-10T05:29:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:28:59 smithi170 ceph-mon[102832]: pgmap v11984: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 187 KiB/s wr, 11 op/s 2023-09-10T05:29:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22171.036542 secs 2023-09-10T05:29:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:28:59 smithi082 ceph-mon[98694]: pgmap v11984: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 187 KiB/s wr, 11 op/s 2023-09-10T05:29:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:01 smithi098 ceph-mon[103124]: pgmap v11985: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 16 op/s 2023-09-10T05:29:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:01 smithi170 ceph-mon[102832]: pgmap v11985: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 16 op/s 2023-09-10T05:29:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:01 smithi082 ceph-mon[98694]: pgmap v11985: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 16 op/s 2023-09-10T05:29:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22174.344325 secs 2023-09-10T05:29:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22174.344325 secs 2023-09-10T05:29:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22174.344325 secs 2023-09-10T05:29:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:04 smithi098 ceph-mon[103124]: pgmap v11986: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 9 op/s 2023-09-10T05:29:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22175.382852 secs 2023-09-10T05:29:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:04 smithi170 ceph-mon[102832]: pgmap v11986: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 9 op/s 2023-09-10T05:29:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22175.382852 secs 2023-09-10T05:29:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:04 smithi082 ceph-mon[98694]: pgmap v11986: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 9 op/s 2023-09-10T05:29:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22175.382852 secs 2023-09-10T05:29:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22176.036604 secs 2023-09-10T05:29:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22176.036604 secs 2023-09-10T05:29:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22176.036604 secs 2023-09-10T05:29:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:06 smithi098 ceph-mon[103124]: pgmap v11987: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 9 op/s 2023-09-10T05:29:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:06 smithi170 ceph-mon[102832]: pgmap v11987: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 9 op/s 2023-09-10T05:29:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:06 smithi082 ceph-mon[98694]: pgmap v11987: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 9 op/s 2023-09-10T05:29:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:08 smithi170 ceph-mon[102832]: pgmap v11988: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 180 KiB/s wr, 28 op/s 2023-09-10T05:29:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22179.344399 secs 2023-09-10T05:29:08.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:08 smithi082 ceph-mon[98694]: pgmap v11988: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 180 KiB/s wr, 28 op/s 2023-09-10T05:29:08.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22179.344399 secs 2023-09-10T05:29:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:08 smithi098 ceph-mon[103124]: pgmap v11988: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 180 KiB/s wr, 28 op/s 2023-09-10T05:29:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22179.344399 secs 2023-09-10T05:29:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:09 smithi170 ceph-mon[102832]: osdmap e817: 12 total, 12 up, 12 in 2023-09-10T05:29:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22180.382913 secs 2023-09-10T05:29:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:09 smithi082 ceph-mon[98694]: osdmap e817: 12 total, 12 up, 12 in 2023-09-10T05:29:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22180.382913 secs 2023-09-10T05:29:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:09 smithi098 ceph-mon[103124]: osdmap e817: 12 total, 12 up, 12 in 2023-09-10T05:29:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22180.382913 secs 2023-09-10T05:29:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22181.036666 secs 2023-09-10T05:29:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:10 smithi170 ceph-mon[102832]: pgmap v11990: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 28 op/s 2023-09-10T05:29:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22181.036666 secs 2023-09-10T05:29:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:10 smithi082 ceph-mon[98694]: pgmap v11990: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 28 op/s 2023-09-10T05:29:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22181.036666 secs 2023-09-10T05:29:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:10 smithi098 ceph-mon[103124]: pgmap v11990: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 28 op/s 2023-09-10T05:29:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:12 smithi170 ceph-mon[102832]: pgmap v11991: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 28 op/s 2023-09-10T05:29:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:12 smithi082 ceph-mon[98694]: pgmap v11991: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 28 op/s 2023-09-10T05:29:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:12 smithi098 ceph-mon[103124]: pgmap v11991: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 28 op/s 2023-09-10T05:29:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22184.344523 secs 2023-09-10T05:29:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22184.344523 secs 2023-09-10T05:29:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22184.344523 secs 2023-09-10T05:29:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:14 smithi170 ceph-mon[102832]: pgmap v11992: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 28 op/s 2023-09-10T05:29:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22185.383002 secs 2023-09-10T05:29:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:14 smithi082 ceph-mon[98694]: pgmap v11992: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 28 op/s 2023-09-10T05:29:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22185.383002 secs 2023-09-10T05:29:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:14 smithi098 ceph-mon[103124]: pgmap v11992: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 121 KiB/s wr, 28 op/s 2023-09-10T05:29:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22185.383002 secs 2023-09-10T05:29:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22186.036727 secs 2023-09-10T05:29:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:15 smithi170 ceph-mon[102832]: osdmap e818: 12 total, 12 up, 12 in 2023-09-10T05:29:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22186.036727 secs 2023-09-10T05:29:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:15 smithi082 ceph-mon[98694]: osdmap e818: 12 total, 12 up, 12 in 2023-09-10T05:29:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22186.036727 secs 2023-09-10T05:29:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:15 smithi098 ceph-mon[103124]: osdmap e818: 12 total, 12 up, 12 in 2023-09-10T05:29:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:16 smithi170 ceph-mon[102832]: pgmap v11994: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:29:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:16 smithi082 ceph-mon[98694]: pgmap v11994: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:29:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:16 smithi098 ceph-mon[103124]: pgmap v11994: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:29:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:18 smithi170 ceph-mon[102832]: pgmap v11995: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 231 B/s wr, 11 op/s 2023-09-10T05:29:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22189.344604 secs 2023-09-10T05:29:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:18 smithi082 ceph-mon[98694]: pgmap v11995: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 231 B/s wr, 11 op/s 2023-09-10T05:29:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22189.344604 secs 2023-09-10T05:29:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:18 smithi098 ceph-mon[103124]: pgmap v11995: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 231 B/s wr, 11 op/s 2023-09-10T05:29:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22189.344604 secs 2023-09-10T05:29:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22190.383097 secs 2023-09-10T05:29:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22190.383097 secs 2023-09-10T05:29:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22190.383097 secs 2023-09-10T05:29:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22191.036790 secs 2023-09-10T05:29:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:20 smithi170 ceph-mon[102832]: pgmap v11996: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:29:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22191.036790 secs 2023-09-10T05:29:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:20 smithi082 ceph-mon[98694]: pgmap v11996: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:29:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22191.036790 secs 2023-09-10T05:29:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:20 smithi098 ceph-mon[103124]: pgmap v11996: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:29:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:22 smithi170 ceph-mon[102832]: pgmap v11997: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:29:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:22 smithi082 ceph-mon[98694]: pgmap v11997: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:29:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:22 smithi098 ceph-mon[103124]: pgmap v11997: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:29:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22194.344692 secs 2023-09-10T05:29:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22194.344692 secs 2023-09-10T05:29:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22194.344692 secs 2023-09-10T05:29:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:24 smithi170 ceph-mon[102832]: pgmap v11998: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:29:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22195.383155 secs 2023-09-10T05:29:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:24 smithi082 ceph-mon[98694]: pgmap v11998: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:29:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22195.383155 secs 2023-09-10T05:29:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:24 smithi098 ceph-mon[103124]: pgmap v11998: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:29:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22195.383155 secs 2023-09-10T05:29:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22196.036851 secs 2023-09-10T05:29:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22196.036851 secs 2023-09-10T05:29:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22196.036851 secs 2023-09-10T05:29:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:26 smithi170 ceph-mon[102832]: pgmap v11999: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 202 B/s wr, 10 op/s 2023-09-10T05:29:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:26 smithi082 ceph-mon[98694]: pgmap v11999: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 202 B/s wr, 10 op/s 2023-09-10T05:29:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:26 smithi098 ceph-mon[103124]: pgmap v11999: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 202 B/s wr, 10 op/s 2023-09-10T05:29:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:28 smithi170 ceph-mon[102832]: pgmap v12000: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 13 op/s 2023-09-10T05:29:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:28 smithi082 ceph-mon[98694]: pgmap v12000: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 13 op/s 2023-09-10T05:29:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:28 smithi098 ceph-mon[103124]: pgmap v12000: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 13 op/s 2023-09-10T05:29:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22200.383212 secs 2023-09-10T05:29:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22200.383212 secs 2023-09-10T05:29:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22200.383212 secs 2023-09-10T05:29:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22201.036910 secs 2023-09-10T05:29:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:30 smithi170 ceph-mon[102832]: pgmap v12001: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-10T05:29:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22201.036910 secs 2023-09-10T05:29:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:30 smithi082 ceph-mon[98694]: pgmap v12001: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-10T05:29:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22201.036910 secs 2023-09-10T05:29:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:30 smithi098 ceph-mon[103124]: pgmap v12001: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-10T05:29:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:29:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:29:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:29:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:32 smithi170 ceph-mon[102832]: pgmap v12002: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 169 KiB/s wr, 15 op/s 2023-09-10T05:29:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:29:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:29:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:29:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:32 smithi082 ceph-mon[98694]: pgmap v12002: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 169 KiB/s wr, 15 op/s 2023-09-10T05:29:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:29:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:29:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:29:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:32 smithi098 ceph-mon[103124]: pgmap v12002: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 169 KiB/s wr, 15 op/s 2023-09-10T05:29:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:29:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:29:32.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:29:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:34 smithi170 ceph-mon[102832]: pgmap v12003: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 168 KiB/s wr, 10 op/s 2023-09-10T05:29:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22205.383272 secs 2023-09-10T05:29:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22206.036970 secs 2023-09-10T05:29:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:34 smithi082 ceph-mon[98694]: pgmap v12003: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 168 KiB/s wr, 10 op/s 2023-09-10T05:29:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22205.383272 secs 2023-09-10T05:29:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22206.036970 secs 2023-09-10T05:29:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:34 smithi098 ceph-mon[103124]: pgmap v12003: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 168 KiB/s wr, 10 op/s 2023-09-10T05:29:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22205.383272 secs 2023-09-10T05:29:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22206.036970 secs 2023-09-10T05:29:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:35 smithi170 ceph-mon[102832]: pgmap v12004: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 168 KiB/s wr, 10 op/s 2023-09-10T05:29:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:35 smithi082 ceph-mon[98694]: pgmap v12004: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 168 KiB/s wr, 10 op/s 2023-09-10T05:29:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:35 smithi098 ceph-mon[103124]: pgmap v12004: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 168 KiB/s wr, 10 op/s 2023-09-10T05:29:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:29:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:29:38.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:37 smithi170 ceph-mon[102832]: pgmap v12005: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 286 KiB/s wr, 16 op/s 2023-09-10T05:29:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:29:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:29:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:37 smithi082 ceph-mon[98694]: pgmap v12005: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 286 KiB/s wr, 16 op/s 2023-09-10T05:29:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:29:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:29:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:37 smithi098 ceph-mon[103124]: pgmap v12005: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 286 KiB/s wr, 16 op/s 2023-09-10T05:29:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:29:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22210.383348 secs 2023-09-10T05:29:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:29:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:29:39.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:29:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:29:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22210.383348 secs 2023-09-10T05:29:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:29:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:29:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:29:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:29:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22210.383348 secs 2023-09-10T05:29:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:29:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:29:39.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:29:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22211.037031 secs 2023-09-10T05:29:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:39 smithi170 ceph-mon[102832]: pgmap v12006: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T05:29:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22211.037031 secs 2023-09-10T05:29:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:39 smithi082 ceph-mon[98694]: pgmap v12006: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T05:29:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22211.037031 secs 2023-09-10T05:29:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:39 smithi098 ceph-mon[103124]: pgmap v12006: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T05:29:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:29:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:29:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:29:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:29:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:29:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:29:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:41 smithi170 ceph-mon[102832]: pgmap v12007: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 508 KiB/s wr, 19 op/s 2023-09-10T05:29:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:41 smithi082 ceph-mon[98694]: pgmap v12007: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 508 KiB/s wr, 19 op/s 2023-09-10T05:29:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:41 smithi098 ceph-mon[103124]: pgmap v12007: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 508 KiB/s wr, 19 op/s 2023-09-10T05:29:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:43 smithi082 ceph-mon[98694]: pgmap v12008: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 340 KiB/s wr, 13 op/s 2023-09-10T05:29:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22215.383407 secs 2023-09-10T05:29:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:43 smithi098 ceph-mon[103124]: pgmap v12008: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 340 KiB/s wr, 13 op/s 2023-09-10T05:29:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22215.383407 secs 2023-09-10T05:29:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:43 smithi170 ceph-mon[102832]: pgmap v12008: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 340 KiB/s wr, 13 op/s 2023-09-10T05:29:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22215.383407 secs 2023-09-10T05:29:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22216.037094 secs 2023-09-10T05:29:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22216.037094 secs 2023-09-10T05:29:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22216.037094 secs 2023-09-10T05:29:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:45 smithi082 ceph-mon[98694]: pgmap v12009: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 340 KiB/s wr, 13 op/s 2023-09-10T05:29:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:45 smithi098 ceph-mon[103124]: pgmap v12009: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 340 KiB/s wr, 13 op/s 2023-09-10T05:29:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:45 smithi170 ceph-mon[102832]: pgmap v12009: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 340 KiB/s wr, 13 op/s 2023-09-10T05:29:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:47 smithi082 ceph-mon[98694]: pgmap v12010: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 441 KiB/s wr, 20 op/s 2023-09-10T05:29:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:47 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22219.345040 secs 2023-09-10T05:29:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:47 smithi098 ceph-mon[103124]: pgmap v12010: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 441 KiB/s wr, 20 op/s 2023-09-10T05:29:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:47 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22219.345040 secs 2023-09-10T05:29:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:47 smithi170 ceph-mon[102832]: pgmap v12010: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 441 KiB/s wr, 20 op/s 2023-09-10T05:29:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:47 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22219.345040 secs 2023-09-10T05:29:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22220.383466 secs 2023-09-10T05:29:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22220.383466 secs 2023-09-10T05:29:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22220.383466 secs 2023-09-10T05:29:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22221.037160 secs 2023-09-10T05:29:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:49 smithi082 ceph-mon[98694]: pgmap v12011: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 324 KiB/s wr, 13 op/s 2023-09-10T05:29:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:29:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:29:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22221.037160 secs 2023-09-10T05:29:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:49 smithi098 ceph-mon[103124]: pgmap v12011: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 324 KiB/s wr, 13 op/s 2023-09-10T05:29:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:29:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:29:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22221.037160 secs 2023-09-10T05:29:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:49 smithi170 ceph-mon[102832]: pgmap v12011: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 324 KiB/s wr, 13 op/s 2023-09-10T05:29:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:29:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:29:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:29:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:29:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:29:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:29:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:29:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:29:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:29:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:29:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:29:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:52 smithi170 ceph-mon[102832]: pgmap v12012: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 431 KiB/s wr, 19 op/s 2023-09-10T05:29:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:52 smithi082 ceph-mon[98694]: pgmap v12012: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 431 KiB/s wr, 19 op/s 2023-09-10T05:29:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:52 smithi098 ceph-mon[103124]: pgmap v12012: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 431 KiB/s wr, 19 op/s 2023-09-10T05:29:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22224.345142 secs 2023-09-10T05:29:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22224.345142 secs 2023-09-10T05:29:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22224.345142 secs 2023-09-10T05:29:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:54 smithi170 ceph-mon[102832]: pgmap v12013: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 209 KiB/s wr, 12 op/s 2023-09-10T05:29:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22225.383491 secs 2023-09-10T05:29:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:54 smithi082 ceph-mon[98694]: pgmap v12013: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 209 KiB/s wr, 12 op/s 2023-09-10T05:29:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22225.383491 secs 2023-09-10T05:29:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:54 smithi098 ceph-mon[103124]: pgmap v12013: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 209 KiB/s wr, 12 op/s 2023-09-10T05:29:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22225.383491 secs 2023-09-10T05:29:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22226.037226 secs 2023-09-10T05:29:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22226.037226 secs 2023-09-10T05:29:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22226.037226 secs 2023-09-10T05:29:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:56 smithi170 ceph-mon[102832]: pgmap v12014: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 209 KiB/s wr, 12 op/s 2023-09-10T05:29:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:56 smithi082 ceph-mon[98694]: pgmap v12014: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 209 KiB/s wr, 12 op/s 2023-09-10T05:29:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:56 smithi098 ceph-mon[103124]: pgmap v12014: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 209 KiB/s wr, 12 op/s 2023-09-10T05:29:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:58 smithi170 ceph-mon[102832]: pgmap v12015: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 288 KiB/s wr, 17 op/s 2023-09-10T05:29:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22229.345201 secs 2023-09-10T05:29:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:58 smithi082 ceph-mon[98694]: pgmap v12015: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 288 KiB/s wr, 17 op/s 2023-09-10T05:29:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22229.345201 secs 2023-09-10T05:29:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:58 smithi098 ceph-mon[103124]: pgmap v12015: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 288 KiB/s wr, 17 op/s 2023-09-10T05:29:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22229.345201 secs 2023-09-10T05:29:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:29:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22230.383551 secs 2023-09-10T05:29:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:29:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22230.383551 secs 2023-09-10T05:29:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:29:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22230.383551 secs 2023-09-10T05:30:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22231.037294 secs 2023-09-10T05:30:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:00 smithi170 ceph-mon[102832]: pgmap v12016: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T05:30:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T05:30:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22231.037294 secs 2023-09-10T05:30:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:00 smithi082 ceph-mon[98694]: pgmap v12016: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T05:30:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T05:30:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22231.037294 secs 2023-09-10T05:30:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:00 smithi098 ceph-mon[103124]: pgmap v12016: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T05:30:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T05:30:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:02 smithi170 ceph-mon[102832]: pgmap v12017: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 186 KiB/s wr, 15 op/s 2023-09-10T05:30:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:02 smithi082 ceph-mon[98694]: pgmap v12017: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 186 KiB/s wr, 15 op/s 2023-09-10T05:30:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:02 smithi098 ceph-mon[103124]: pgmap v12017: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 186 KiB/s wr, 15 op/s 2023-09-10T05:30:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22234.345323 secs 2023-09-10T05:30:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22234.345323 secs 2023-09-10T05:30:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22234.345323 secs 2023-09-10T05:30:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:04 smithi170 ceph-mon[102832]: pgmap v12018: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 10 op/s 2023-09-10T05:30:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22235.383608 secs 2023-09-10T05:30:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:04 smithi082 ceph-mon[98694]: pgmap v12018: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 10 op/s 2023-09-10T05:30:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22235.383608 secs 2023-09-10T05:30:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:04 smithi098 ceph-mon[103124]: pgmap v12018: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 10 op/s 2023-09-10T05:30:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22235.383608 secs 2023-09-10T05:30:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22236.037360 secs 2023-09-10T05:30:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22236.037360 secs 2023-09-10T05:30:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22236.037360 secs 2023-09-10T05:30:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:06 smithi170 ceph-mon[102832]: pgmap v12019: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 10 op/s 2023-09-10T05:30:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:06 smithi082 ceph-mon[98694]: pgmap v12019: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 10 op/s 2023-09-10T05:30:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:06 smithi098 ceph-mon[103124]: pgmap v12019: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 10 op/s 2023-09-10T05:30:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:08 smithi170 ceph-mon[102832]: pgmap v12020: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 180 KiB/s wr, 29 op/s 2023-09-10T05:30:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22239.345433 secs 2023-09-10T05:30:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:08 smithi082 ceph-mon[98694]: pgmap v12020: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 180 KiB/s wr, 29 op/s 2023-09-10T05:30:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22239.345433 secs 2023-09-10T05:30:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:08 smithi098 ceph-mon[103124]: pgmap v12020: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 180 KiB/s wr, 29 op/s 2023-09-10T05:30:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22239.345433 secs 2023-09-10T05:30:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:09 smithi170 ceph-mon[102832]: osdmap e819: 12 total, 12 up, 12 in 2023-09-10T05:30:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22240.383667 secs 2023-09-10T05:30:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:09 smithi082 ceph-mon[98694]: osdmap e819: 12 total, 12 up, 12 in 2023-09-10T05:30:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22240.383667 secs 2023-09-10T05:30:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:09 smithi098 ceph-mon[103124]: osdmap e819: 12 total, 12 up, 12 in 2023-09-10T05:30:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22240.383667 secs 2023-09-10T05:30:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22241.037429 secs 2023-09-10T05:30:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:10 smithi170 ceph-mon[102832]: pgmap v12022: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 121 KiB/s wr, 28 op/s 2023-09-10T05:30:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22241.037429 secs 2023-09-10T05:30:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:10 smithi082 ceph-mon[98694]: pgmap v12022: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 121 KiB/s wr, 28 op/s 2023-09-10T05:30:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22241.037429 secs 2023-09-10T05:30:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:10 smithi098 ceph-mon[103124]: pgmap v12022: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 121 KiB/s wr, 28 op/s 2023-09-10T05:30:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:12 smithi170 ceph-mon[102832]: pgmap v12023: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 122 KiB/s wr, 28 op/s 2023-09-10T05:30:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:12 smithi082 ceph-mon[98694]: pgmap v12023: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 122 KiB/s wr, 28 op/s 2023-09-10T05:30:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:12 smithi098 ceph-mon[103124]: pgmap v12023: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 122 KiB/s wr, 28 op/s 2023-09-10T05:30:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22244.345493 secs 2023-09-10T05:30:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22244.345493 secs 2023-09-10T05:30:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22244.345493 secs 2023-09-10T05:30:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:14 smithi170 ceph-mon[102832]: pgmap v12024: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 122 KiB/s wr, 28 op/s 2023-09-10T05:30:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22245.383726 secs 2023-09-10T05:30:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22246.037485 secs 2023-09-10T05:30:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:14 smithi082 ceph-mon[98694]: pgmap v12024: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 122 KiB/s wr, 28 op/s 2023-09-10T05:30:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22245.383726 secs 2023-09-10T05:30:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22246.037485 secs 2023-09-10T05:30:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:14 smithi098 ceph-mon[103124]: pgmap v12024: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 122 KiB/s wr, 28 op/s 2023-09-10T05:30:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22245.383726 secs 2023-09-10T05:30:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22246.037485 secs 2023-09-10T05:30:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:15 smithi170 ceph-mon[102832]: osdmap e820: 12 total, 12 up, 12 in 2023-09-10T05:30:16.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:15 smithi170 ceph-mon[102832]: pgmap v12026: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:30:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:15 smithi082 ceph-mon[98694]: osdmap e820: 12 total, 12 up, 12 in 2023-09-10T05:30:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:15 smithi082 ceph-mon[98694]: pgmap v12026: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:30:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:15 smithi098 ceph-mon[103124]: osdmap e820: 12 total, 12 up, 12 in 2023-09-10T05:30:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:15 smithi098 ceph-mon[103124]: pgmap v12026: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:30:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:17 smithi082 ceph-mon[98694]: pgmap v12027: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 231 B/s wr, 11 op/s 2023-09-10T05:30:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22249.345553 secs 2023-09-10T05:30:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:17 smithi098 ceph-mon[103124]: pgmap v12027: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 231 B/s wr, 11 op/s 2023-09-10T05:30:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22249.345553 secs 2023-09-10T05:30:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:17 smithi170 ceph-mon[102832]: pgmap v12027: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 231 B/s wr, 11 op/s 2023-09-10T05:30:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22249.345553 secs 2023-09-10T05:30:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22250.383784 secs 2023-09-10T05:30:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22250.383784 secs 2023-09-10T05:30:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22250.383784 secs 2023-09-10T05:30:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22251.037550 secs 2023-09-10T05:30:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:19 smithi082 ceph-mon[98694]: pgmap v12028: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:30:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22251.037550 secs 2023-09-10T05:30:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:19 smithi098 ceph-mon[103124]: pgmap v12028: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:30:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22251.037550 secs 2023-09-10T05:30:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:19 smithi170 ceph-mon[102832]: pgmap v12028: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:30:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:21 smithi082 ceph-mon[98694]: pgmap v12029: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 818 B/s wr, 10 op/s 2023-09-10T05:30:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:21 smithi098 ceph-mon[103124]: pgmap v12029: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 818 B/s wr, 10 op/s 2023-09-10T05:30:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:21 smithi170 ceph-mon[102832]: pgmap v12029: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 818 B/s wr, 10 op/s 2023-09-10T05:30:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22254.345611 secs 2023-09-10T05:30:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22254.345611 secs 2023-09-10T05:30:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22254.345611 secs 2023-09-10T05:30:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:23 smithi098 ceph-mon[103124]: pgmap v12030: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 818 B/s wr, 10 op/s 2023-09-10T05:30:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22255.383862 secs 2023-09-10T05:30:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:23 smithi170 ceph-mon[102832]: pgmap v12030: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 818 B/s wr, 10 op/s 2023-09-10T05:30:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22255.383862 secs 2023-09-10T05:30:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:23 smithi082 ceph-mon[98694]: pgmap v12030: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 818 B/s wr, 10 op/s 2023-09-10T05:30:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22255.383862 secs 2023-09-10T05:30:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22256.037613 secs 2023-09-10T05:30:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22256.037613 secs 2023-09-10T05:30:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22256.037613 secs 2023-09-10T05:30:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:25 smithi098 ceph-mon[103124]: pgmap v12031: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 811 B/s wr, 10 op/s 2023-09-10T05:30:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:25 smithi170 ceph-mon[102832]: pgmap v12031: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 811 B/s wr, 10 op/s 2023-09-10T05:30:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:25 smithi082 ceph-mon[98694]: pgmap v12031: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 811 B/s wr, 10 op/s 2023-09-10T05:30:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:27 smithi098 ceph-mon[103124]: pgmap v12032: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 75 KiB/s wr, 14 op/s 2023-09-10T05:30:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:27 smithi170 ceph-mon[102832]: pgmap v12032: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 75 KiB/s wr, 14 op/s 2023-09-10T05:30:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:27 smithi082 ceph-mon[98694]: pgmap v12032: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 75 KiB/s wr, 14 op/s 2023-09-10T05:30:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22260.383916 secs 2023-09-10T05:30:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22260.383916 secs 2023-09-10T05:30:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22260.383916 secs 2023-09-10T05:30:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22261.037676 secs 2023-09-10T05:30:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:29 smithi098 ceph-mon[103124]: pgmap v12033: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 9 op/s 2023-09-10T05:30:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22261.037676 secs 2023-09-10T05:30:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:29 smithi170 ceph-mon[102832]: pgmap v12033: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 9 op/s 2023-09-10T05:30:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22261.037676 secs 2023-09-10T05:30:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:29 smithi082 ceph-mon[98694]: pgmap v12033: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 9 op/s 2023-09-10T05:30:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:30 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:30:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:30 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:30:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:30 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:30:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:31 smithi098 ceph-mon[103124]: pgmap v12034: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 149 KiB/s wr, 15 op/s 2023-09-10T05:30:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:31 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:30:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:30:32.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:30:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:31 smithi170 ceph-mon[102832]: pgmap v12034: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 149 KiB/s wr, 15 op/s 2023-09-10T05:30:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:31 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:30:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:30:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:30:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:31 smithi082 ceph-mon[98694]: pgmap v12034: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 149 KiB/s wr, 15 op/s 2023-09-10T05:30:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:31 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:30:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:30:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:30:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:33 smithi098 ceph-mon[103124]: pgmap v12035: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T05:30:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:30:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:30:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22265.383976 secs 2023-09-10T05:30:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:33 smithi170 ceph-mon[102832]: pgmap v12035: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T05:30:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:30:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:30:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22265.383976 secs 2023-09-10T05:30:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:33 smithi082 ceph-mon[98694]: pgmap v12035: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T05:30:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:30:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:30:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22265.383976 secs 2023-09-10T05:30:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22266.037742 secs 2023-09-10T05:30:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22266.037742 secs 2023-09-10T05:30:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22266.037742 secs 2023-09-10T05:30:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:35 smithi098 ceph-mon[103124]: pgmap v12036: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T05:30:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:35 smithi170 ceph-mon[102832]: pgmap v12036: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T05:30:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:35 smithi082 ceph-mon[98694]: pgmap v12036: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T05:30:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:37 smithi098 ceph-mon[103124]: pgmap v12037: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 376 KiB/s wr, 15 op/s 2023-09-10T05:30:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:30:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:30:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:37 smithi170 ceph-mon[102832]: pgmap v12037: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 376 KiB/s wr, 15 op/s 2023-09-10T05:30:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:30:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:30:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:37 smithi082 ceph-mon[98694]: pgmap v12037: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 376 KiB/s wr, 15 op/s 2023-09-10T05:30:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:30:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:30:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22270.384062 secs 2023-09-10T05:30:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:30:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22270.384062 secs 2023-09-10T05:30:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:30:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22270.384062 secs 2023-09-10T05:30:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:30:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22271.037808 secs 2023-09-10T05:30:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:40 smithi098 ceph-mon[103124]: pgmap v12038: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 302 KiB/s wr, 10 op/s 2023-09-10T05:30:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:30:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:30:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:30:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:30:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:30:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22271.037808 secs 2023-09-10T05:30:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:40 smithi170 ceph-mon[102832]: pgmap v12038: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 302 KiB/s wr, 10 op/s 2023-09-10T05:30:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:30:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:30:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:30:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:30:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:30:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22271.037808 secs 2023-09-10T05:30:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:40 smithi082 ceph-mon[98694]: pgmap v12038: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 302 KiB/s wr, 10 op/s 2023-09-10T05:30:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:30:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:30:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:30:40.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:30:40.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:30:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:42 smithi170 ceph-mon[102832]: pgmap v12039: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 546 KiB/s wr, 17 op/s 2023-09-10T05:30:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:42 smithi082 ceph-mon[98694]: pgmap v12039: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 546 KiB/s wr, 17 op/s 2023-09-10T05:30:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:42 smithi098 ceph-mon[103124]: pgmap v12039: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 546 KiB/s wr, 17 op/s 2023-09-10T05:30:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:44 smithi170 ceph-mon[102832]: pgmap v12040: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 472 KiB/s wr, 12 op/s 2023-09-10T05:30:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22275.384146 secs 2023-09-10T05:30:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:44 smithi082 ceph-mon[98694]: pgmap v12040: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 472 KiB/s wr, 12 op/s 2023-09-10T05:30:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22275.384146 secs 2023-09-10T05:30:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:44 smithi098 ceph-mon[103124]: pgmap v12040: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 472 KiB/s wr, 12 op/s 2023-09-10T05:30:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22275.384146 secs 2023-09-10T05:30:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22276.037874 secs 2023-09-10T05:30:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22276.037874 secs 2023-09-10T05:30:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22276.037874 secs 2023-09-10T05:30:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:46 smithi170 ceph-mon[102832]: pgmap v12041: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 472 KiB/s wr, 12 op/s 2023-09-10T05:30:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:46 smithi082 ceph-mon[98694]: pgmap v12041: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 472 KiB/s wr, 12 op/s 2023-09-10T05:30:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:46 smithi098 ceph-mon[103124]: pgmap v12041: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 472 KiB/s wr, 12 op/s 2023-09-10T05:30:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:48 smithi170 ceph-mon[102832]: pgmap v12042: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 689 KiB/s wr, 19 op/s 2023-09-10T05:30:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:48 smithi082 ceph-mon[98694]: pgmap v12042: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 689 KiB/s wr, 19 op/s 2023-09-10T05:30:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:48 smithi098 ceph-mon[103124]: pgmap v12042: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 689 KiB/s wr, 19 op/s 2023-09-10T05:30:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22280.384240 secs 2023-09-10T05:30:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22280.384240 secs 2023-09-10T05:30:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22280.384240 secs 2023-09-10T05:30:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22281.037938 secs 2023-09-10T05:30:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:50 smithi170 ceph-mon[102832]: pgmap v12043: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 460 KiB/s wr, 13 op/s 2023-09-10T05:30:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:30:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:30:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22281.037938 secs 2023-09-10T05:30:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:50 smithi082 ceph-mon[98694]: pgmap v12043: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 460 KiB/s wr, 13 op/s 2023-09-10T05:30:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:30:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:30:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22281.037938 secs 2023-09-10T05:30:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:50 smithi098 ceph-mon[103124]: pgmap v12043: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 460 KiB/s wr, 13 op/s 2023-09-10T05:30:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:30:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:30:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:52 smithi170 ceph-mon[102832]: pgmap v12044: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 465 KiB/s wr, 19 op/s 2023-09-10T05:30:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:30:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:30:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:52 smithi082 ceph-mon[98694]: pgmap v12044: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 465 KiB/s wr, 19 op/s 2023-09-10T05:30:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:30:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:30:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:52 smithi098 ceph-mon[103124]: pgmap v12044: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 465 KiB/s wr, 19 op/s 2023-09-10T05:30:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:30:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:30:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:54 smithi170 ceph-mon[102832]: pgmap v12045: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 222 KiB/s wr, 12 op/s 2023-09-10T05:30:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22285.384299 secs 2023-09-10T05:30:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:54 smithi082 ceph-mon[98694]: pgmap v12045: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 222 KiB/s wr, 12 op/s 2023-09-10T05:30:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22285.384299 secs 2023-09-10T05:30:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:54 smithi098 ceph-mon[103124]: pgmap v12045: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 222 KiB/s wr, 12 op/s 2023-09-10T05:30:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22285.384299 secs 2023-09-10T05:30:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22286.038030 secs 2023-09-10T05:30:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22286.038030 secs 2023-09-10T05:30:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22286.038030 secs 2023-09-10T05:30:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:56 smithi170 ceph-mon[102832]: pgmap v12046: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 222 KiB/s wr, 12 op/s 2023-09-10T05:30:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:56 smithi082 ceph-mon[98694]: pgmap v12046: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 222 KiB/s wr, 12 op/s 2023-09-10T05:30:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:56 smithi098 ceph-mon[103124]: pgmap v12046: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 222 KiB/s wr, 12 op/s 2023-09-10T05:30:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:58 smithi170 ceph-mon[102832]: pgmap v12047: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 394 KiB/s wr, 18 op/s 2023-09-10T05:30:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:30:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:30:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22289.346115 secs 2023-09-10T05:30:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:58 smithi082 ceph-mon[98694]: pgmap v12047: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 394 KiB/s wr, 18 op/s 2023-09-10T05:30:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:30:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:30:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22289.346115 secs 2023-09-10T05:30:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:58 smithi098 ceph-mon[103124]: pgmap v12047: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 394 KiB/s wr, 18 op/s 2023-09-10T05:30:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:30:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:30:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22289.346115 secs 2023-09-10T05:30:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:30:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22290.384358 secs 2023-09-10T05:30:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:30:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22290.384358 secs 2023-09-10T05:30:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:30:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22290.384358 secs 2023-09-10T05:31:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22291.038098 secs 2023-09-10T05:31:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:00 smithi170 ceph-mon[102832]: pgmap v12048: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T05:31:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:31:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:31:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:31:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22291.038098 secs 2023-09-10T05:31:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:00 smithi082 ceph-mon[98694]: pgmap v12048: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T05:31:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:31:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:31:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:31:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22291.038098 secs 2023-09-10T05:31:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:00 smithi098 ceph-mon[103124]: pgmap v12048: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T05:31:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:31:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:31:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:31:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:02 smithi170 ceph-mon[102832]: pgmap v12049: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 439 KiB/s wr, 18 op/s 2023-09-10T05:31:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:02 smithi082 ceph-mon[98694]: pgmap v12049: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 439 KiB/s wr, 18 op/s 2023-09-10T05:31:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:02 smithi098 ceph-mon[103124]: pgmap v12049: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 439 KiB/s wr, 18 op/s 2023-09-10T05:31:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22294.346209 secs 2023-09-10T05:31:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22294.346209 secs 2023-09-10T05:31:03.546 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22294.346209 secs 2023-09-10T05:31:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:04 smithi170 ceph-mon[102832]: pgmap v12050: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 434 KiB/s wr, 12 op/s 2023-09-10T05:31:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22295.384417 secs 2023-09-10T05:31:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:04 smithi082 ceph-mon[98694]: pgmap v12050: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 434 KiB/s wr, 12 op/s 2023-09-10T05:31:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22295.384417 secs 2023-09-10T05:31:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:04 smithi098 ceph-mon[103124]: pgmap v12050: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 434 KiB/s wr, 12 op/s 2023-09-10T05:31:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22295.384417 secs 2023-09-10T05:31:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22296.038162 secs 2023-09-10T05:31:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22296.038162 secs 2023-09-10T05:31:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22296.038162 secs 2023-09-10T05:31:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:06 smithi170 ceph-mon[102832]: pgmap v12051: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 434 KiB/s wr, 12 op/s 2023-09-10T05:31:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:06 smithi082 ceph-mon[98694]: pgmap v12051: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 434 KiB/s wr, 12 op/s 2023-09-10T05:31:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:06 smithi098 ceph-mon[103124]: pgmap v12051: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 434 KiB/s wr, 12 op/s 2023-09-10T05:31:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:08 smithi170 ceph-mon[102832]: pgmap v12052: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 650 KiB/s wr, 32 op/s 2023-09-10T05:31:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22299.346272 secs 2023-09-10T05:31:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:08 smithi082 ceph-mon[98694]: pgmap v12052: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 650 KiB/s wr, 32 op/s 2023-09-10T05:31:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22299.346272 secs 2023-09-10T05:31:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:08 smithi098 ceph-mon[103124]: pgmap v12052: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 650 KiB/s wr, 32 op/s 2023-09-10T05:31:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22299.346272 secs 2023-09-10T05:31:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:09 smithi170 ceph-mon[102832]: osdmap e821: 12 total, 12 up, 12 in 2023-09-10T05:31:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22300.384504 secs 2023-09-10T05:31:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:09 smithi082 ceph-mon[98694]: osdmap e821: 12 total, 12 up, 12 in 2023-09-10T05:31:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22300.384504 secs 2023-09-10T05:31:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:09 smithi098 ceph-mon[103124]: osdmap e821: 12 total, 12 up, 12 in 2023-09-10T05:31:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22300.384504 secs 2023-09-10T05:31:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22301.038231 secs 2023-09-10T05:31:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:10 smithi170 ceph-mon[102832]: pgmap v12054: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 573 KiB/s wr, 32 op/s 2023-09-10T05:31:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22301.038231 secs 2023-09-10T05:31:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:10 smithi082 ceph-mon[98694]: pgmap v12054: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 573 KiB/s wr, 32 op/s 2023-09-10T05:31:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22301.038231 secs 2023-09-10T05:31:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:10 smithi098 ceph-mon[103124]: pgmap v12054: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 573 KiB/s wr, 32 op/s 2023-09-10T05:31:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:12 smithi170 ceph-mon[102832]: pgmap v12055: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 328 KiB/s wr, 30 op/s 2023-09-10T05:31:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:12 smithi082 ceph-mon[98694]: pgmap v12055: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 328 KiB/s wr, 30 op/s 2023-09-10T05:31:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:12 smithi098 ceph-mon[103124]: pgmap v12055: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 328 KiB/s wr, 30 op/s 2023-09-10T05:31:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22304.346335 secs 2023-09-10T05:31:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22304.346335 secs 2023-09-10T05:31:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22304.346335 secs 2023-09-10T05:31:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:14 smithi170 ceph-mon[102832]: pgmap v12056: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 328 KiB/s wr, 30 op/s 2023-09-10T05:31:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22305.384563 secs 2023-09-10T05:31:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22306.038293 secs 2023-09-10T05:31:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:14 smithi082 ceph-mon[98694]: pgmap v12056: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 328 KiB/s wr, 30 op/s 2023-09-10T05:31:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22305.384563 secs 2023-09-10T05:31:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22306.038293 secs 2023-09-10T05:31:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:14 smithi098 ceph-mon[103124]: pgmap v12056: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 328 KiB/s wr, 30 op/s 2023-09-10T05:31:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22305.384563 secs 2023-09-10T05:31:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22306.038293 secs 2023-09-10T05:31:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:15 smithi170 ceph-mon[102832]: osdmap e822: 12 total, 12 up, 12 in 2023-09-10T05:31:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:15 smithi170 ceph-mon[102832]: pgmap v12058: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 7 op/s 2023-09-10T05:31:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:15 smithi082 ceph-mon[98694]: osdmap e822: 12 total, 12 up, 12 in 2023-09-10T05:31:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:15 smithi082 ceph-mon[98694]: pgmap v12058: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 7 op/s 2023-09-10T05:31:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:15 smithi098 ceph-mon[103124]: osdmap e822: 12 total, 12 up, 12 in 2023-09-10T05:31:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:15 smithi098 ceph-mon[103124]: pgmap v12058: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 7 op/s 2023-09-10T05:31:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:17 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:31:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:17 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:31:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:17 smithi170 ceph-mon[102832]: pgmap v12059: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 237 KiB/s wr, 13 op/s 2023-09-10T05:31:18.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22309.346424 secs 2023-09-10T05:31:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:17 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:31:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:17 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:31:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:17 smithi082 ceph-mon[98694]: pgmap v12059: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 237 KiB/s wr, 13 op/s 2023-09-10T05:31:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22309.346424 secs 2023-09-10T05:31:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:17 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:31:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:17 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:31:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:17 smithi098 ceph-mon[103124]: pgmap v12059: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 237 KiB/s wr, 13 op/s 2023-09-10T05:31:18.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22309.346424 secs 2023-09-10T05:31:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22310.384623 secs 2023-09-10T05:31:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22310.384623 secs 2023-09-10T05:31:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22310.384623 secs 2023-09-10T05:31:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22311.038363 secs 2023-09-10T05:31:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:19 smithi170 ceph-mon[102832]: pgmap v12060: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 210 KiB/s wr, 12 op/s 2023-09-10T05:31:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22311.038363 secs 2023-09-10T05:31:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:19 smithi082 ceph-mon[98694]: pgmap v12060: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 210 KiB/s wr, 12 op/s 2023-09-10T05:31:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22311.038363 secs 2023-09-10T05:31:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:19 smithi098 ceph-mon[103124]: pgmap v12060: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 210 KiB/s wr, 12 op/s 2023-09-10T05:31:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:31:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:31:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:31:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:31:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:31:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:31:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:21 smithi170 ceph-mon[102832]: pgmap v12061: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 12 op/s 2023-09-10T05:31:22.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:21 smithi082 ceph-mon[98694]: pgmap v12061: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 12 op/s 2023-09-10T05:31:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:21 smithi098 ceph-mon[103124]: pgmap v12061: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 12 op/s 2023-09-10T05:31:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:22 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22314.346488 secs 2023-09-10T05:31:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:22 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22314.346488 secs 2023-09-10T05:31:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:22 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22314.346488 secs 2023-09-10T05:31:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:23 smithi170 ceph-mon[102832]: pgmap v12062: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 12 op/s 2023-09-10T05:31:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22315.384692 secs 2023-09-10T05:31:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:23 smithi082 ceph-mon[98694]: pgmap v12062: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 12 op/s 2023-09-10T05:31:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22315.384692 secs 2023-09-10T05:31:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:23 smithi098 ceph-mon[103124]: pgmap v12062: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 12 op/s 2023-09-10T05:31:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22315.384692 secs 2023-09-10T05:31:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22316.038428 secs 2023-09-10T05:31:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22316.038428 secs 2023-09-10T05:31:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22316.038428 secs 2023-09-10T05:31:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:25 smithi170 ceph-mon[102832]: pgmap v12063: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 12 op/s 2023-09-10T05:31:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:25 smithi082 ceph-mon[98694]: pgmap v12063: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 12 op/s 2023-09-10T05:31:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:25 smithi098 ceph-mon[103124]: pgmap v12063: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 12 op/s 2023-09-10T05:31:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:27 smithi098 ceph-mon[103124]: pgmap v12064: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 117 KiB/s wr, 15 op/s 2023-09-10T05:31:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:27 smithi170 ceph-mon[102832]: pgmap v12064: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 117 KiB/s wr, 15 op/s 2023-09-10T05:31:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:27 smithi082 ceph-mon[98694]: pgmap v12064: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 117 KiB/s wr, 15 op/s 2023-09-10T05:31:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22320.384764 secs 2023-09-10T05:31:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22320.384764 secs 2023-09-10T05:31:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22320.384764 secs 2023-09-10T05:31:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22321.038492 secs 2023-09-10T05:31:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:29 smithi098 ceph-mon[103124]: pgmap v12065: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-10T05:31:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22321.038492 secs 2023-09-10T05:31:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:29 smithi170 ceph-mon[102832]: pgmap v12065: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-10T05:31:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22321.038492 secs 2023-09-10T05:31:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:29 smithi082 ceph-mon[98694]: pgmap v12065: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-10T05:31:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:30 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:31:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:30 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:31:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:30 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:31:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:31 smithi098 ceph-mon[103124]: pgmap v12066: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 74 KiB/s wr, 14 op/s 2023-09-10T05:31:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:31 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:31:32.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:31:32.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:31:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:31 smithi170 ceph-mon[102832]: pgmap v12066: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 74 KiB/s wr, 14 op/s 2023-09-10T05:31:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:31 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:31:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:31:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:31:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:31 smithi082 ceph-mon[98694]: pgmap v12066: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 74 KiB/s wr, 14 op/s 2023-09-10T05:31:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:31 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:31:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:31:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:31:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:33 smithi098 ceph-mon[103124]: pgmap v12067: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 9 op/s 2023-09-10T05:31:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22325.384824 secs 2023-09-10T05:31:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:33 smithi170 ceph-mon[102832]: pgmap v12067: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 9 op/s 2023-09-10T05:31:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22325.384824 secs 2023-09-10T05:31:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:33 smithi082 ceph-mon[98694]: pgmap v12067: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 9 op/s 2023-09-10T05:31:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22325.384824 secs 2023-09-10T05:31:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22326.038556 secs 2023-09-10T05:31:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22326.038556 secs 2023-09-10T05:31:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22326.038556 secs 2023-09-10T05:31:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:35 smithi098 ceph-mon[103124]: pgmap v12068: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 9 op/s 2023-09-10T05:31:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:35 smithi170 ceph-mon[102832]: pgmap v12068: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 9 op/s 2023-09-10T05:31:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:35 smithi082 ceph-mon[98694]: pgmap v12068: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 9 op/s 2023-09-10T05:31:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:37 smithi098 ceph-mon[103124]: pgmap v12069: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 303 KiB/s wr, 15 op/s 2023-09-10T05:31:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:31:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:31:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:37 smithi170 ceph-mon[102832]: pgmap v12069: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 303 KiB/s wr, 15 op/s 2023-09-10T05:31:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:31:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:31:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:37 smithi082 ceph-mon[98694]: pgmap v12069: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 303 KiB/s wr, 15 op/s 2023-09-10T05:31:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:31:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:31:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22330.384898 secs 2023-09-10T05:31:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22330.384898 secs 2023-09-10T05:31:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22330.384898 secs 2023-09-10T05:31:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22331.038620 secs 2023-09-10T05:31:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:39 smithi098 ceph-mon[103124]: pgmap v12070: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 303 KiB/s wr, 10 op/s 2023-09-10T05:31:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:39 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:31:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:39 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:31:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:31:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22331.038620 secs 2023-09-10T05:31:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:39 smithi170 ceph-mon[102832]: pgmap v12070: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 303 KiB/s wr, 10 op/s 2023-09-10T05:31:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:39 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:31:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:39 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:31:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:31:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22331.038620 secs 2023-09-10T05:31:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:39 smithi082 ceph-mon[98694]: pgmap v12070: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 303 KiB/s wr, 10 op/s 2023-09-10T05:31:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:39 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:31:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:39 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:31:40.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:31:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:31:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:31:41.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:31:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:31:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:31:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:31:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:31:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:31:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:31:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:42 smithi170 ceph-mon[102832]: pgmap v12071: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 420 KiB/s wr, 16 op/s 2023-09-10T05:31:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:42 smithi082 ceph-mon[98694]: pgmap v12071: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 420 KiB/s wr, 16 op/s 2023-09-10T05:31:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:42 smithi098 ceph-mon[103124]: pgmap v12071: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 420 KiB/s wr, 16 op/s 2023-09-10T05:31:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:44 smithi170 ceph-mon[102832]: pgmap v12072: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 346 KiB/s wr, 11 op/s 2023-09-10T05:31:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22335.384958 secs 2023-09-10T05:31:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:44 smithi082 ceph-mon[98694]: pgmap v12072: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 346 KiB/s wr, 11 op/s 2023-09-10T05:31:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22335.384958 secs 2023-09-10T05:31:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:44 smithi098 ceph-mon[103124]: pgmap v12072: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 346 KiB/s wr, 11 op/s 2023-09-10T05:31:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22335.384958 secs 2023-09-10T05:31:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22336.038685 secs 2023-09-10T05:31:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22336.038685 secs 2023-09-10T05:31:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22336.038685 secs 2023-09-10T05:31:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:46 smithi170 ceph-mon[102832]: pgmap v12073: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 346 KiB/s wr, 11 op/s 2023-09-10T05:31:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:46 smithi082 ceph-mon[98694]: pgmap v12073: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 346 KiB/s wr, 11 op/s 2023-09-10T05:31:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:46 smithi098 ceph-mon[103124]: pgmap v12073: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 346 KiB/s wr, 11 op/s 2023-09-10T05:31:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:48 smithi170 ceph-mon[102832]: pgmap v12074: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 568 KiB/s wr, 19 op/s 2023-09-10T05:31:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:48 smithi082 ceph-mon[98694]: pgmap v12074: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 568 KiB/s wr, 19 op/s 2023-09-10T05:31:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:48 smithi098 ceph-mon[103124]: pgmap v12074: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 568 KiB/s wr, 19 op/s 2023-09-10T05:31:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22340.385042 secs 2023-09-10T05:31:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22341.038750 secs 2023-09-10T05:31:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22340.385042 secs 2023-09-10T05:31:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22341.038750 secs 2023-09-10T05:31:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22340.385042 secs 2023-09-10T05:31:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22341.038750 secs 2023-09-10T05:31:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:50 smithi170 ceph-mon[102832]: pgmap v12075: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 339 KiB/s wr, 13 op/s 2023-09-10T05:31:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:50 smithi082 ceph-mon[98694]: pgmap v12075: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 339 KiB/s wr, 13 op/s 2023-09-10T05:31:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:50 smithi098 ceph-mon[103124]: pgmap v12075: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 339 KiB/s wr, 13 op/s 2023-09-10T05:31:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:52 smithi170 ceph-mon[102832]: pgmap v12076: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 376 KiB/s wr, 19 op/s 2023-09-10T05:31:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:52 smithi082 ceph-mon[98694]: pgmap v12076: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 376 KiB/s wr, 19 op/s 2023-09-10T05:31:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:52 smithi098 ceph-mon[103124]: pgmap v12076: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 376 KiB/s wr, 19 op/s 2023-09-10T05:31:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22344.346897 secs 2023-09-10T05:31:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22344.346897 secs 2023-09-10T05:31:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22344.346897 secs 2023-09-10T05:31:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:54 smithi170 ceph-mon[102832]: pgmap v12077: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 259 KiB/s wr, 13 op/s 2023-09-10T05:31:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:31:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:31:54.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22345.385101 secs 2023-09-10T05:31:54.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22346.038814 secs 2023-09-10T05:31:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:54 smithi082 ceph-mon[98694]: pgmap v12077: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 259 KiB/s wr, 13 op/s 2023-09-10T05:31:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:31:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:31:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22345.385101 secs 2023-09-10T05:31:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22346.038814 secs 2023-09-10T05:31:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:54 smithi098 ceph-mon[103124]: pgmap v12077: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 259 KiB/s wr, 13 op/s 2023-09-10T05:31:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:31:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:31:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22345.385101 secs 2023-09-10T05:31:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22346.038814 secs 2023-09-10T05:31:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:55 smithi170 ceph-mon[102832]: pgmap v12078: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 259 KiB/s wr, 13 op/s 2023-09-10T05:31:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:55 smithi082 ceph-mon[98694]: pgmap v12078: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 259 KiB/s wr, 13 op/s 2023-09-10T05:31:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:55 smithi098 ceph-mon[103124]: pgmap v12078: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 259 KiB/s wr, 13 op/s 2023-09-10T05:31:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:31:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:31:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:31:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:31:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:31:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:31:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:31:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:31:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:31:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:57 smithi170 ceph-mon[102832]: pgmap v12079: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 431 KiB/s wr, 19 op/s 2023-09-10T05:31:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22349.346961 secs 2023-09-10T05:31:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:57 smithi082 ceph-mon[98694]: pgmap v12079: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 431 KiB/s wr, 19 op/s 2023-09-10T05:31:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22349.346961 secs 2023-09-10T05:31:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:57 smithi098 ceph-mon[103124]: pgmap v12079: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 431 KiB/s wr, 19 op/s 2023-09-10T05:31:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22349.346961 secs 2023-09-10T05:31:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22350.385186 secs 2023-09-10T05:31:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22350.385186 secs 2023-09-10T05:31:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22350.385186 secs 2023-09-10T05:32:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22351.038881 secs 2023-09-10T05:32:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:31:59 smithi170 ceph-mon[102832]: pgmap v12080: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 209 KiB/s wr, 12 op/s 2023-09-10T05:32:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22351.038881 secs 2023-09-10T05:32:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:31:59 smithi082 ceph-mon[98694]: pgmap v12080: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 209 KiB/s wr, 12 op/s 2023-09-10T05:32:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22351.038881 secs 2023-09-10T05:32:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:31:59 smithi098 ceph-mon[103124]: pgmap v12080: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 209 KiB/s wr, 12 op/s 2023-09-10T05:32:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:01 smithi098 ceph-mon[103124]: pgmap v12081: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 311 KiB/s wr, 31 op/s 2023-09-10T05:32:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:01 smithi170 ceph-mon[102832]: pgmap v12081: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 311 KiB/s wr, 31 op/s 2023-09-10T05:32:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:01 smithi082 ceph-mon[98694]: pgmap v12081: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 311 KiB/s wr, 31 op/s 2023-09-10T05:32:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:02 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22354.347026 secs 2023-09-10T05:32:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:02 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22354.347026 secs 2023-09-10T05:32:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:02 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22354.347026 secs 2023-09-10T05:32:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:03 smithi098 ceph-mon[103124]: pgmap v12082: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 274 KiB/s wr, 25 op/s 2023-09-10T05:32:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22355.385271 secs 2023-09-10T05:32:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:03 smithi170 ceph-mon[102832]: pgmap v12082: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 274 KiB/s wr, 25 op/s 2023-09-10T05:32:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22355.385271 secs 2023-09-10T05:32:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:03 smithi082 ceph-mon[98694]: pgmap v12082: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 274 KiB/s wr, 25 op/s 2023-09-10T05:32:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22355.385271 secs 2023-09-10T05:32:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22356.038946 secs 2023-09-10T05:32:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22356.038946 secs 2023-09-10T05:32:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22356.038946 secs 2023-09-10T05:32:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:05 smithi098 ceph-mon[103124]: pgmap v12083: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 274 KiB/s wr, 25 op/s 2023-09-10T05:32:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:05 smithi170 ceph-mon[102832]: pgmap v12083: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 274 KiB/s wr, 25 op/s 2023-09-10T05:32:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:05 smithi082 ceph-mon[98694]: pgmap v12083: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 274 KiB/s wr, 25 op/s 2023-09-10T05:32:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:07 smithi098 ceph-mon[103124]: pgmap v12084: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 274 KiB/s wr, 30 op/s 2023-09-10T05:32:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22359.347084 secs 2023-09-10T05:32:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:07 smithi170 ceph-mon[102832]: pgmap v12084: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 274 KiB/s wr, 30 op/s 2023-09-10T05:32:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22359.347084 secs 2023-09-10T05:32:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:07 smithi082 ceph-mon[98694]: pgmap v12084: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 274 KiB/s wr, 30 op/s 2023-09-10T05:32:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22359.347084 secs 2023-09-10T05:32:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:09 smithi098 ceph-mon[103124]: osdmap e823: 12 total, 12 up, 12 in 2023-09-10T05:32:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22360.385356 secs 2023-09-10T05:32:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:08 smithi170 ceph-mon[102832]: osdmap e823: 12 total, 12 up, 12 in 2023-09-10T05:32:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22360.385356 secs 2023-09-10T05:32:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:08 smithi082 ceph-mon[98694]: osdmap e823: 12 total, 12 up, 12 in 2023-09-10T05:32:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22360.385356 secs 2023-09-10T05:32:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22361.039013 secs 2023-09-10T05:32:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:10 smithi098 ceph-mon[103124]: pgmap v12086: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 122 KiB/s wr, 28 op/s 2023-09-10T05:32:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22361.039013 secs 2023-09-10T05:32:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:09 smithi170 ceph-mon[102832]: pgmap v12086: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 122 KiB/s wr, 28 op/s 2023-09-10T05:32:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22361.039013 secs 2023-09-10T05:32:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:09 smithi082 ceph-mon[98694]: pgmap v12086: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 122 KiB/s wr, 28 op/s 2023-09-10T05:32:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:12 smithi098 ceph-mon[103124]: pgmap v12087: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:32:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:12 smithi170 ceph-mon[102832]: pgmap v12087: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:32:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:12 smithi082 ceph-mon[98694]: pgmap v12087: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:32:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22364.347144 secs 2023-09-10T05:32:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22364.347144 secs 2023-09-10T05:32:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22364.347144 secs 2023-09-10T05:32:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:14 smithi170 ceph-mon[102832]: pgmap v12088: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:32:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22365.385416 secs 2023-09-10T05:32:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:14 smithi082 ceph-mon[98694]: pgmap v12088: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:32:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22365.385416 secs 2023-09-10T05:32:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:14 smithi098 ceph-mon[103124]: pgmap v12088: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:32:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22365.385416 secs 2023-09-10T05:32:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22366.039076 secs 2023-09-10T05:32:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:15 smithi170 ceph-mon[102832]: osdmap e824: 12 total, 12 up, 12 in 2023-09-10T05:32:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22366.039076 secs 2023-09-10T05:32:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:15 smithi082 ceph-mon[98694]: osdmap e824: 12 total, 12 up, 12 in 2023-09-10T05:32:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22366.039076 secs 2023-09-10T05:32:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:15 smithi098 ceph-mon[103124]: osdmap e824: 12 total, 12 up, 12 in 2023-09-10T05:32:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:16 smithi170 ceph-mon[102832]: pgmap v12090: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:32:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:16 smithi082 ceph-mon[98694]: pgmap v12090: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:32:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:16 smithi098 ceph-mon[103124]: pgmap v12090: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:32:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:18 smithi170 ceph-mon[102832]: pgmap v12091: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 229 B/s wr, 11 op/s 2023-09-10T05:32:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22369.347205 secs 2023-09-10T05:32:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:18 smithi082 ceph-mon[98694]: pgmap v12091: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 229 B/s wr, 11 op/s 2023-09-10T05:32:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22369.347205 secs 2023-09-10T05:32:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:18 smithi098 ceph-mon[103124]: pgmap v12091: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 229 B/s wr, 11 op/s 2023-09-10T05:32:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22369.347205 secs 2023-09-10T05:32:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22370.385479 secs 2023-09-10T05:32:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22370.385479 secs 2023-09-10T05:32:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22370.385479 secs 2023-09-10T05:32:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22371.039142 secs 2023-09-10T05:32:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:20 smithi170 ceph-mon[102832]: pgmap v12092: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:32:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22371.039142 secs 2023-09-10T05:32:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:20 smithi082 ceph-mon[98694]: pgmap v12092: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:32:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22371.039142 secs 2023-09-10T05:32:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:20 smithi098 ceph-mon[103124]: pgmap v12092: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:32:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:22 smithi170 ceph-mon[102832]: pgmap v12093: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:32:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:22 smithi082 ceph-mon[98694]: pgmap v12093: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:32:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:22 smithi098 ceph-mon[103124]: pgmap v12093: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:32:23.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22374.347267 secs 2023-09-10T05:32:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22374.347267 secs 2023-09-10T05:32:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22374.347267 secs 2023-09-10T05:32:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:24 smithi170 ceph-mon[102832]: pgmap v12094: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:32:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:32:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:32:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22375.385535 secs 2023-09-10T05:32:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:24 smithi082 ceph-mon[98694]: pgmap v12094: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:32:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:32:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:32:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22375.385535 secs 2023-09-10T05:32:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:24 smithi098 ceph-mon[103124]: pgmap v12094: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:32:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:32:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:32:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22375.385535 secs 2023-09-10T05:32:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22376.039205 secs 2023-09-10T05:32:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22376.039205 secs 2023-09-10T05:32:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22376.039205 secs 2023-09-10T05:32:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:26 smithi170 ceph-mon[102832]: pgmap v12095: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T05:32:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:26 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:32:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:32:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:26 smithi082 ceph-mon[98694]: pgmap v12095: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T05:32:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:26 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:32:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:32:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:26 smithi098 ceph-mon[103124]: pgmap v12095: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T05:32:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:26 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:32:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:32:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:28 smithi170 ceph-mon[102832]: pgmap v12096: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 159 KiB/s wr, 14 op/s 2023-09-10T05:32:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:28 smithi082 ceph-mon[98694]: pgmap v12096: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 159 KiB/s wr, 14 op/s 2023-09-10T05:32:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:28 smithi098 ceph-mon[103124]: pgmap v12096: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 159 KiB/s wr, 14 op/s 2023-09-10T05:32:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22380.385638 secs 2023-09-10T05:32:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22380.385638 secs 2023-09-10T05:32:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22380.385638 secs 2023-09-10T05:32:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22381.039270 secs 2023-09-10T05:32:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:30 smithi170 ceph-mon[102832]: pgmap v12097: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 10 op/s 2023-09-10T05:32:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22381.039270 secs 2023-09-10T05:32:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:30 smithi082 ceph-mon[98694]: pgmap v12097: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 10 op/s 2023-09-10T05:32:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22381.039270 secs 2023-09-10T05:32:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:30 smithi098 ceph-mon[103124]: pgmap v12097: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 10 op/s 2023-09-10T05:32:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:32:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:32:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:32:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:32 smithi170 ceph-mon[102832]: pgmap v12098: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 293 KiB/s wr, 15 op/s 2023-09-10T05:32:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:32:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:32:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:32:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:32 smithi082 ceph-mon[98694]: pgmap v12098: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 293 KiB/s wr, 15 op/s 2023-09-10T05:32:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:32:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:32:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:32:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:32 smithi098 ceph-mon[103124]: pgmap v12098: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 293 KiB/s wr, 15 op/s 2023-09-10T05:32:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:32:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:32:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:32:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:34 smithi170 ceph-mon[102832]: pgmap v12099: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 224 KiB/s wr, 10 op/s 2023-09-10T05:32:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22385.385725 secs 2023-09-10T05:32:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:34 smithi082 ceph-mon[98694]: pgmap v12099: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 224 KiB/s wr, 10 op/s 2023-09-10T05:32:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22385.385725 secs 2023-09-10T05:32:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:34 smithi098 ceph-mon[103124]: pgmap v12099: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 224 KiB/s wr, 10 op/s 2023-09-10T05:32:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22385.385725 secs 2023-09-10T05:32:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22386.039333 secs 2023-09-10T05:32:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22386.039333 secs 2023-09-10T05:32:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22386.039333 secs 2023-09-10T05:32:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:36 smithi170 ceph-mon[102832]: pgmap v12100: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 224 KiB/s wr, 10 op/s 2023-09-10T05:32:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:36 smithi082 ceph-mon[98694]: pgmap v12100: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 224 KiB/s wr, 10 op/s 2023-09-10T05:32:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:36 smithi098 ceph-mon[103124]: pgmap v12100: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 224 KiB/s wr, 10 op/s 2023-09-10T05:32:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:38 smithi170 ceph-mon[102832]: pgmap v12101: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 409 KiB/s wr, 16 op/s 2023-09-10T05:32:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:32:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:32:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:38 smithi082 ceph-mon[98694]: pgmap v12101: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 409 KiB/s wr, 16 op/s 2023-09-10T05:32:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:32:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:32:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:38 smithi098 ceph-mon[103124]: pgmap v12101: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 409 KiB/s wr, 16 op/s 2023-09-10T05:32:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:32:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:32:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22390.385815 secs 2023-09-10T05:32:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22390.385815 secs 2023-09-10T05:32:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22390.385815 secs 2023-09-10T05:32:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22391.039400 secs 2023-09-10T05:32:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:40 smithi170 ceph-mon[102832]: pgmap v12102: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 319 KiB/s wr, 10 op/s 2023-09-10T05:32:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:32:40.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:32:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22391.039400 secs 2023-09-10T05:32:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:40 smithi082 ceph-mon[98694]: pgmap v12102: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 319 KiB/s wr, 10 op/s 2023-09-10T05:32:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:32:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:32:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22391.039400 secs 2023-09-10T05:32:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:40 smithi098 ceph-mon[103124]: pgmap v12102: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 319 KiB/s wr, 10 op/s 2023-09-10T05:32:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:32:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:32:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:32:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:32:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:32:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:42 smithi170 ceph-mon[102832]: pgmap v12103: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 432 KiB/s wr, 17 op/s 2023-09-10T05:32:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:42 smithi082 ceph-mon[98694]: pgmap v12103: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 432 KiB/s wr, 17 op/s 2023-09-10T05:32:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:42 smithi098 ceph-mon[103124]: pgmap v12103: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 432 KiB/s wr, 17 op/s 2023-09-10T05:32:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:32:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:32:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:43 smithi170 ceph-mon[102832]: pgmap v12104: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 298 KiB/s wr, 11 op/s 2023-09-10T05:32:44.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:32:44.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:32:44.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22395.385875 secs 2023-09-10T05:32:44.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:32:44.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:32:44.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:32:44.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:32:44.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:32:44.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:32:44.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:32:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:32:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:32:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:43 smithi082 ceph-mon[98694]: pgmap v12104: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 298 KiB/s wr, 11 op/s 2023-09-10T05:32:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:32:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:32:44.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22395.385875 secs 2023-09-10T05:32:44.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:32:44.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:32:44.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:32:44.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:32:44.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:32:44.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:32:44.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:32:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:32:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:32:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:43 smithi098 ceph-mon[103124]: pgmap v12104: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 298 KiB/s wr, 11 op/s 2023-09-10T05:32:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:32:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:32:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22395.385875 secs 2023-09-10T05:32:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:32:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:32:44.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:32:44.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:32:44.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:32:44.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:32:44.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:32:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:44 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T05:32:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:44 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T05:32:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22396.039465 secs 2023-09-10T05:32:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:32:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:32:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:32:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:32:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:44 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T05:32:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:44 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T05:32:45.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:32:45.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:32:45.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:32:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:44 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T05:32:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:44 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T05:32:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22396.039465 secs 2023-09-10T05:32:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:32:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:32:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:32:45.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:32:45.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:44 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T05:32:45.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:44 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T05:32:45.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:32:45.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:32:45.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:32:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:44 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T05:32:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:44 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T05:32:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22396.039465 secs 2023-09-10T05:32:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:32:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:32:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:32:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:32:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:44 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T05:32:45.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:44 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T05:32:45.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:32:45.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:32:45.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:32:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:45 smithi170 ceph-mon[102832]: pgmap v12105: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 298 KiB/s wr, 11 op/s 2023-09-10T05:32:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:45 smithi082 ceph-mon[98694]: pgmap v12105: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 298 KiB/s wr, 11 op/s 2023-09-10T05:32:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:45 smithi098 ceph-mon[103124]: pgmap v12105: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 298 KiB/s wr, 11 op/s 2023-09-10T05:32:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:47 smithi098 ceph-mon[103124]: pgmap v12106: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 535 KiB/s wr, 19 op/s 2023-09-10T05:32:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:47 smithi170 ceph-mon[102832]: pgmap v12106: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 535 KiB/s wr, 19 op/s 2023-09-10T05:32:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:47 smithi082 ceph-mon[98694]: pgmap v12106: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 535 KiB/s wr, 19 op/s 2023-09-10T05:32:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22400.385933 secs 2023-09-10T05:32:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22400.385933 secs 2023-09-10T05:32:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22400.385933 secs 2023-09-10T05:32:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22401.039487 secs 2023-09-10T05:32:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:49 smithi098 ceph-mon[103124]: pgmap v12107: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 350 KiB/s wr, 13 op/s 2023-09-10T05:32:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22401.039487 secs 2023-09-10T05:32:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:49 smithi170 ceph-mon[102832]: pgmap v12107: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 350 KiB/s wr, 13 op/s 2023-09-10T05:32:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22401.039487 secs 2023-09-10T05:32:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:49 smithi082 ceph-mon[98694]: pgmap v12107: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 350 KiB/s wr, 13 op/s 2023-09-10T05:32:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:51 smithi098 ceph-mon[103124]: pgmap v12108: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 409 KiB/s wr, 20 op/s 2023-09-10T05:32:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:51 smithi170 ceph-mon[102832]: pgmap v12108: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 409 KiB/s wr, 20 op/s 2023-09-10T05:32:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:51 smithi082 ceph-mon[98694]: pgmap v12108: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 409 KiB/s wr, 20 op/s 2023-09-10T05:32:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22404.347724 secs 2023-09-10T05:32:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22404.347724 secs 2023-09-10T05:32:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22404.347724 secs 2023-09-10T05:32:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:54 smithi170 ceph-mon[102832]: pgmap v12109: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 296 KiB/s wr, 13 op/s 2023-09-10T05:32:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:32:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:32:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22405.385992 secs 2023-09-10T05:32:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:54 smithi082 ceph-mon[98694]: pgmap v12109: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 296 KiB/s wr, 13 op/s 2023-09-10T05:32:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:32:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:32:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22405.385992 secs 2023-09-10T05:32:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:54 smithi098 ceph-mon[103124]: pgmap v12109: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 296 KiB/s wr, 13 op/s 2023-09-10T05:32:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:32:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:32:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22405.385992 secs 2023-09-10T05:32:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22406.039555 secs 2023-09-10T05:32:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22406.039555 secs 2023-09-10T05:32:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22406.039555 secs 2023-09-10T05:32:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:56 smithi170 ceph-mon[102832]: pgmap v12110: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 296 KiB/s wr, 13 op/s 2023-09-10T05:32:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:32:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:32:56.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:32:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:56 smithi082 ceph-mon[98694]: pgmap v12110: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 296 KiB/s wr, 13 op/s 2023-09-10T05:32:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:32:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:32:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:32:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:56 smithi098 ceph-mon[103124]: pgmap v12110: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 296 KiB/s wr, 13 op/s 2023-09-10T05:32:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:32:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:32:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:32:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:58 smithi170 ceph-mon[102832]: pgmap v12111: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 484 KiB/s wr, 20 op/s 2023-09-10T05:32:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22409.347774 secs 2023-09-10T05:32:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:58 smithi082 ceph-mon[98694]: pgmap v12111: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 484 KiB/s wr, 20 op/s 2023-09-10T05:32:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22409.347774 secs 2023-09-10T05:32:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:58 smithi098 ceph-mon[103124]: pgmap v12111: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 484 KiB/s wr, 20 op/s 2023-09-10T05:32:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22409.347774 secs 2023-09-10T05:32:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:32:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22410.386056 secs 2023-09-10T05:32:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:32:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22410.386056 secs 2023-09-10T05:32:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:32:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22410.386056 secs 2023-09-10T05:33:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22411.039622 secs 2023-09-10T05:33:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:00 smithi170 ceph-mon[102832]: pgmap v12112: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 247 KiB/s wr, 13 op/s 2023-09-10T05:33:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22411.039622 secs 2023-09-10T05:33:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:00 smithi082 ceph-mon[98694]: pgmap v12112: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 247 KiB/s wr, 13 op/s 2023-09-10T05:33:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22411.039622 secs 2023-09-10T05:33:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:00 smithi098 ceph-mon[103124]: pgmap v12112: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 247 KiB/s wr, 13 op/s 2023-09-10T05:33:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:02 smithi170 ceph-mon[102832]: pgmap v12113: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 326 KiB/s wr, 31 op/s 2023-09-10T05:33:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:02 smithi082 ceph-mon[98694]: pgmap v12113: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 326 KiB/s wr, 31 op/s 2023-09-10T05:33:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:02 smithi098 ceph-mon[103124]: pgmap v12113: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 326 KiB/s wr, 31 op/s 2023-09-10T05:33:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22414.347841 secs 2023-09-10T05:33:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22414.347841 secs 2023-09-10T05:33:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22414.347841 secs 2023-09-10T05:33:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:04 smithi170 ceph-mon[102832]: pgmap v12114: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 267 KiB/s wr, 24 op/s 2023-09-10T05:33:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22415.386145 secs 2023-09-10T05:33:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:04 smithi082 ceph-mon[98694]: pgmap v12114: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 267 KiB/s wr, 24 op/s 2023-09-10T05:33:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22415.386145 secs 2023-09-10T05:33:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:04 smithi098 ceph-mon[103124]: pgmap v12114: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 267 KiB/s wr, 24 op/s 2023-09-10T05:33:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22415.386145 secs 2023-09-10T05:33:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22416.039689 secs 2023-09-10T05:33:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22416.039689 secs 2023-09-10T05:33:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22416.039689 secs 2023-09-10T05:33:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:06 smithi170 ceph-mon[102832]: pgmap v12115: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 267 KiB/s wr, 24 op/s 2023-09-10T05:33:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:06 smithi082 ceph-mon[98694]: pgmap v12115: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 267 KiB/s wr, 24 op/s 2023-09-10T05:33:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:06 smithi098 ceph-mon[103124]: pgmap v12115: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 267 KiB/s wr, 24 op/s 2023-09-10T05:33:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:08 smithi170 ceph-mon[102832]: pgmap v12116: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 290 KiB/s wr, 32 op/s 2023-09-10T05:33:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22419.347904 secs 2023-09-10T05:33:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:08 smithi082 ceph-mon[98694]: pgmap v12116: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 290 KiB/s wr, 32 op/s 2023-09-10T05:33:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22419.347904 secs 2023-09-10T05:33:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:08 smithi098 ceph-mon[103124]: pgmap v12116: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 290 KiB/s wr, 32 op/s 2023-09-10T05:33:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22419.347904 secs 2023-09-10T05:33:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:09 smithi170 ceph-mon[102832]: osdmap e825: 12 total, 12 up, 12 in 2023-09-10T05:33:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22420.386204 secs 2023-09-10T05:33:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:09 smithi082 ceph-mon[98694]: osdmap e825: 12 total, 12 up, 12 in 2023-09-10T05:33:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22420.386204 secs 2023-09-10T05:33:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:09 smithi098 ceph-mon[103124]: osdmap e825: 12 total, 12 up, 12 in 2023-09-10T05:33:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22420.386204 secs 2023-09-10T05:33:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22421.039753 secs 2023-09-10T05:33:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:10 smithi170 ceph-mon[102832]: pgmap v12118: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 122 KiB/s wr, 30 op/s 2023-09-10T05:33:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22421.039753 secs 2023-09-10T05:33:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:10 smithi082 ceph-mon[98694]: pgmap v12118: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 122 KiB/s wr, 30 op/s 2023-09-10T05:33:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22421.039753 secs 2023-09-10T05:33:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:10 smithi098 ceph-mon[103124]: pgmap v12118: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 122 KiB/s wr, 30 op/s 2023-09-10T05:33:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:12 smithi170 ceph-mon[102832]: pgmap v12119: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 27 KiB/s wr, 15 op/s 2023-09-10T05:33:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:12 smithi082 ceph-mon[98694]: pgmap v12119: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 27 KiB/s wr, 15 op/s 2023-09-10T05:33:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:12 smithi098 ceph-mon[103124]: pgmap v12119: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 27 KiB/s wr, 15 op/s 2023-09-10T05:33:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22424.347974 secs 2023-09-10T05:33:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22424.347974 secs 2023-09-10T05:33:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22424.347974 secs 2023-09-10T05:33:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:14 smithi170 ceph-mon[102832]: pgmap v12120: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 27 KiB/s wr, 15 op/s 2023-09-10T05:33:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22425.386263 secs 2023-09-10T05:33:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:14 smithi082 ceph-mon[98694]: pgmap v12120: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 27 KiB/s wr, 15 op/s 2023-09-10T05:33:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22425.386263 secs 2023-09-10T05:33:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:14 smithi098 ceph-mon[103124]: pgmap v12120: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 27 KiB/s wr, 15 op/s 2023-09-10T05:33:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22425.386263 secs 2023-09-10T05:33:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22426.039817 secs 2023-09-10T05:33:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:15 smithi170 ceph-mon[102832]: osdmap e826: 12 total, 12 up, 12 in 2023-09-10T05:33:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22426.039817 secs 2023-09-10T05:33:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:15 smithi082 ceph-mon[98694]: osdmap e826: 12 total, 12 up, 12 in 2023-09-10T05:33:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22426.039817 secs 2023-09-10T05:33:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:15 smithi098 ceph-mon[103124]: osdmap e826: 12 total, 12 up, 12 in 2023-09-10T05:33:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:16 smithi170 ceph-mon[102832]: pgmap v12122: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 7 op/s 2023-09-10T05:33:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:16 smithi082 ceph-mon[98694]: pgmap v12122: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 7 op/s 2023-09-10T05:33:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:16 smithi098 ceph-mon[103124]: pgmap v12122: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 7 op/s 2023-09-10T05:33:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:18 smithi170 ceph-mon[102832]: pgmap v12123: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 345 B/s wr, 13 op/s 2023-09-10T05:33:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22429.348036 secs 2023-09-10T05:33:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:18 smithi082 ceph-mon[98694]: pgmap v12123: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 345 B/s wr, 13 op/s 2023-09-10T05:33:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22429.348036 secs 2023-09-10T05:33:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:18 smithi098 ceph-mon[103124]: pgmap v12123: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 345 B/s wr, 13 op/s 2023-09-10T05:33:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22429.348036 secs 2023-09-10T05:33:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22430.386321 secs 2023-09-10T05:33:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22430.386321 secs 2023-09-10T05:33:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22430.386321 secs 2023-09-10T05:33:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22431.039884 secs 2023-09-10T05:33:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:20 smithi170 ceph-mon[102832]: pgmap v12124: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T05:33:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22431.039884 secs 2023-09-10T05:33:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:20 smithi082 ceph-mon[98694]: pgmap v12124: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T05:33:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22431.039884 secs 2023-09-10T05:33:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:20 smithi098 ceph-mon[103124]: pgmap v12124: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T05:33:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:22 smithi170 ceph-mon[102832]: pgmap v12125: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T05:33:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:22 smithi082 ceph-mon[98694]: pgmap v12125: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T05:33:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:22 smithi098 ceph-mon[103124]: pgmap v12125: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T05:33:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22434.348098 secs 2023-09-10T05:33:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22434.348098 secs 2023-09-10T05:33:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22434.348098 secs 2023-09-10T05:33:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:24 smithi170 ceph-mon[102832]: pgmap v12126: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T05:33:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22435.386380 secs 2023-09-10T05:33:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22436.039949 secs 2023-09-10T05:33:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:24 smithi082 ceph-mon[98694]: pgmap v12126: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T05:33:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22435.386380 secs 2023-09-10T05:33:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22436.039949 secs 2023-09-10T05:33:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:24 smithi098 ceph-mon[103124]: pgmap v12126: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T05:33:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22435.386380 secs 2023-09-10T05:33:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22436.039949 secs 2023-09-10T05:33:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:25 smithi170 ceph-mon[102832]: pgmap v12127: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 101 B/s wr, 11 op/s 2023-09-10T05:33:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:25 smithi082 ceph-mon[98694]: pgmap v12127: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 101 B/s wr, 11 op/s 2023-09-10T05:33:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:25 smithi098 ceph-mon[103124]: pgmap v12127: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 101 B/s wr, 11 op/s 2023-09-10T05:33:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:28 smithi098 ceph-mon[103124]: pgmap v12128: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 14 op/s 2023-09-10T05:33:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:28 smithi170 ceph-mon[102832]: pgmap v12128: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 14 op/s 2023-09-10T05:33:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:28 smithi082 ceph-mon[98694]: pgmap v12128: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 14 op/s 2023-09-10T05:33:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22440.386465 secs 2023-09-10T05:33:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22440.386465 secs 2023-09-10T05:33:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22440.386465 secs 2023-09-10T05:33:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22441.040019 secs 2023-09-10T05:33:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:30 smithi170 ceph-mon[102832]: pgmap v12129: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2023-09-10T05:33:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22441.040019 secs 2023-09-10T05:33:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:30 smithi082 ceph-mon[98694]: pgmap v12129: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2023-09-10T05:33:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22441.040019 secs 2023-09-10T05:33:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:30 smithi098 ceph-mon[103124]: pgmap v12129: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2023-09-10T05:33:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:33:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:33:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:33:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:32 smithi170 ceph-mon[102832]: pgmap v12130: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 91 KiB/s wr, 14 op/s 2023-09-10T05:33:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:33:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:33:32.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:33:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:32 smithi082 ceph-mon[98694]: pgmap v12130: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 91 KiB/s wr, 14 op/s 2023-09-10T05:33:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:33:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:33:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:33:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:32 smithi098 ceph-mon[103124]: pgmap v12130: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 91 KiB/s wr, 14 op/s 2023-09-10T05:33:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:33:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:33:32.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:33:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:34 smithi170 ceph-mon[102832]: pgmap v12131: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T05:33:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:33:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:33:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22445.386512 secs 2023-09-10T05:33:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:34 smithi082 ceph-mon[98694]: pgmap v12131: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T05:33:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:33:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:33:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22445.386512 secs 2023-09-10T05:33:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:34 smithi098 ceph-mon[103124]: pgmap v12131: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T05:33:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:33:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:33:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22445.386512 secs 2023-09-10T05:33:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22446.040081 secs 2023-09-10T05:33:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22446.040081 secs 2023-09-10T05:33:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22446.040081 secs 2023-09-10T05:33:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:36 smithi170 ceph-mon[102832]: pgmap v12132: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T05:33:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:33:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:33:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:36 smithi082 ceph-mon[98694]: pgmap v12132: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T05:33:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:33:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:33:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:36 smithi098 ceph-mon[103124]: pgmap v12132: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T05:33:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:33:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:33:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:38 smithi170 ceph-mon[102832]: pgmap v12133: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 472 KiB/s wr, 17 op/s 2023-09-10T05:33:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:33:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:33:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:38 smithi082 ceph-mon[98694]: pgmap v12133: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 472 KiB/s wr, 17 op/s 2023-09-10T05:33:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:33:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:33:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:38 smithi098 ceph-mon[103124]: pgmap v12133: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 472 KiB/s wr, 17 op/s 2023-09-10T05:33:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:33:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:33:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22450.386569 secs 2023-09-10T05:33:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22450.386569 secs 2023-09-10T05:33:39.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22450.386569 secs 2023-09-10T05:33:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22451.040145 secs 2023-09-10T05:33:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:40 smithi170 ceph-mon[102832]: pgmap v12134: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 471 KiB/s wr, 12 op/s 2023-09-10T05:33:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:33:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:33:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22451.040145 secs 2023-09-10T05:33:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:40 smithi082 ceph-mon[98694]: pgmap v12134: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 471 KiB/s wr, 12 op/s 2023-09-10T05:33:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:33:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:33:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22451.040145 secs 2023-09-10T05:33:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:40 smithi098 ceph-mon[103124]: pgmap v12134: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 471 KiB/s wr, 12 op/s 2023-09-10T05:33:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:33:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:33:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:42 smithi170 ceph-mon[102832]: pgmap v12135: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 584 KiB/s wr, 18 op/s 2023-09-10T05:33:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:42 smithi082 ceph-mon[98694]: pgmap v12135: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 584 KiB/s wr, 18 op/s 2023-09-10T05:33:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:42 smithi098 ceph-mon[103124]: pgmap v12135: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 584 KiB/s wr, 18 op/s 2023-09-10T05:33:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:44 smithi170 ceph-mon[102832]: pgmap v12136: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 494 KiB/s wr, 13 op/s 2023-09-10T05:33:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22455.386629 secs 2023-09-10T05:33:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:44 smithi082 ceph-mon[98694]: pgmap v12136: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 494 KiB/s wr, 13 op/s 2023-09-10T05:33:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22455.386629 secs 2023-09-10T05:33:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:44 smithi098 ceph-mon[103124]: pgmap v12136: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 494 KiB/s wr, 13 op/s 2023-09-10T05:33:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22455.386629 secs 2023-09-10T05:33:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22456.040208 secs 2023-09-10T05:33:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:33:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:33:45.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:33:45.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:33:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22456.040208 secs 2023-09-10T05:33:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:33:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:33:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:33:45.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:33:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22456.040208 secs 2023-09-10T05:33:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:33:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:33:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:33:45.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:33:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:46 smithi170 ceph-mon[102832]: pgmap v12137: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 494 KiB/s wr, 13 op/s 2023-09-10T05:33:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:46 smithi082 ceph-mon[98694]: pgmap v12137: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 494 KiB/s wr, 13 op/s 2023-09-10T05:33:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:46 smithi098 ceph-mon[103124]: pgmap v12137: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 494 KiB/s wr, 13 op/s 2023-09-10T05:33:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:48 smithi170 ceph-mon[102832]: pgmap v12138: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 783 KiB/s wr, 21 op/s 2023-09-10T05:33:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22459.348454 secs 2023-09-10T05:33:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:48 smithi082 ceph-mon[98694]: pgmap v12138: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 783 KiB/s wr, 21 op/s 2023-09-10T05:33:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22459.348454 secs 2023-09-10T05:33:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:48 smithi098 ceph-mon[103124]: pgmap v12138: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 783 KiB/s wr, 21 op/s 2023-09-10T05:33:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22459.348454 secs 2023-09-10T05:33:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22460.386711 secs 2023-09-10T05:33:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22460.386711 secs 2023-09-10T05:33:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22460.386711 secs 2023-09-10T05:33:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22461.040272 secs 2023-09-10T05:33:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:50 smithi170 ceph-mon[102832]: pgmap v12139: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 401 KiB/s wr, 14 op/s 2023-09-10T05:33:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:33:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:33:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22461.040272 secs 2023-09-10T05:33:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:50 smithi082 ceph-mon[98694]: pgmap v12139: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 401 KiB/s wr, 14 op/s 2023-09-10T05:33:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:33:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:33:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22461.040272 secs 2023-09-10T05:33:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:50 smithi098 ceph-mon[103124]: pgmap v12139: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 401 KiB/s wr, 14 op/s 2023-09-10T05:33:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:33:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:33:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:33:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:33:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:33:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:33:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:33:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:33:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:33:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:33:51.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:33:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:52 smithi170 ceph-mon[102832]: pgmap v12140: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 401 KiB/s wr, 20 op/s 2023-09-10T05:33:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:52 smithi082 ceph-mon[98694]: pgmap v12140: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 401 KiB/s wr, 20 op/s 2023-09-10T05:33:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:52 smithi098 ceph-mon[103124]: pgmap v12140: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 401 KiB/s wr, 20 op/s 2023-09-10T05:33:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22464.348567 secs 2023-09-10T05:33:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22464.348567 secs 2023-09-10T05:33:53.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22464.348567 secs 2023-09-10T05:33:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:54 smithi170 ceph-mon[102832]: pgmap v12141: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 289 KiB/s wr, 13 op/s 2023-09-10T05:33:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22465.386796 secs 2023-09-10T05:33:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22466.040338 secs 2023-09-10T05:33:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:54 smithi082 ceph-mon[98694]: pgmap v12141: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 289 KiB/s wr, 13 op/s 2023-09-10T05:33:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22465.386796 secs 2023-09-10T05:33:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22466.040338 secs 2023-09-10T05:33:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:54 smithi098 ceph-mon[103124]: pgmap v12141: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 289 KiB/s wr, 13 op/s 2023-09-10T05:33:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22465.386796 secs 2023-09-10T05:33:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22466.040338 secs 2023-09-10T05:33:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:55 smithi170 ceph-mon[102832]: pgmap v12142: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 289 KiB/s wr, 13 op/s 2023-09-10T05:33:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:55 smithi082 ceph-mon[98694]: pgmap v12142: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 289 KiB/s wr, 13 op/s 2023-09-10T05:33:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:55 smithi098 ceph-mon[103124]: pgmap v12142: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 289 KiB/s wr, 13 op/s 2023-09-10T05:33:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:58 smithi170 ceph-mon[102832]: pgmap v12143: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 450 KiB/s wr, 18 op/s 2023-09-10T05:33:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22469.348627 secs 2023-09-10T05:33:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:58 smithi082 ceph-mon[98694]: pgmap v12143: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 450 KiB/s wr, 18 op/s 2023-09-10T05:33:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22469.348627 secs 2023-09-10T05:33:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:58 smithi098 ceph-mon[103124]: pgmap v12143: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 450 KiB/s wr, 18 op/s 2023-09-10T05:33:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22469.348627 secs 2023-09-10T05:33:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:33:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22470.386891 secs 2023-09-10T05:33:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:33:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22470.386891 secs 2023-09-10T05:33:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:33:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22470.386891 secs 2023-09-10T05:34:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22471.040402 secs 2023-09-10T05:34:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:00 smithi170 ceph-mon[102832]: pgmap v12144: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T05:34:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22471.040402 secs 2023-09-10T05:34:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:00 smithi082 ceph-mon[98694]: pgmap v12144: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T05:34:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22471.040402 secs 2023-09-10T05:34:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:00 smithi098 ceph-mon[103124]: pgmap v12144: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T05:34:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:02 smithi170 ceph-mon[102832]: pgmap v12145: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 219 KiB/s wr, 16 op/s 2023-09-10T05:34:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:02 smithi082 ceph-mon[98694]: pgmap v12145: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 219 KiB/s wr, 16 op/s 2023-09-10T05:34:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:02 smithi098 ceph-mon[103124]: pgmap v12145: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 219 KiB/s wr, 16 op/s 2023-09-10T05:34:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22474.348698 secs 2023-09-10T05:34:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22474.348698 secs 2023-09-10T05:34:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22474.348698 secs 2023-09-10T05:34:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:04 smithi170 ceph-mon[102832]: pgmap v12146: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 219 KiB/s wr, 11 op/s 2023-09-10T05:34:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22475.386949 secs 2023-09-10T05:34:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:04 smithi082 ceph-mon[98694]: pgmap v12146: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 219 KiB/s wr, 11 op/s 2023-09-10T05:34:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22475.386949 secs 2023-09-10T05:34:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:04 smithi098 ceph-mon[103124]: pgmap v12146: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 219 KiB/s wr, 11 op/s 2023-09-10T05:34:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22475.386949 secs 2023-09-10T05:34:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22476.040465 secs 2023-09-10T05:34:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22476.040465 secs 2023-09-10T05:34:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22476.040465 secs 2023-09-10T05:34:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:06 smithi170 ceph-mon[102832]: pgmap v12147: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 219 KiB/s wr, 11 op/s 2023-09-10T05:34:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:06 smithi082 ceph-mon[98694]: pgmap v12147: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 219 KiB/s wr, 11 op/s 2023-09-10T05:34:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:06 smithi098 ceph-mon[103124]: pgmap v12147: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 219 KiB/s wr, 11 op/s 2023-09-10T05:34:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:08 smithi170 ceph-mon[102832]: pgmap v12148: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 264 KiB/s wr, 28 op/s 2023-09-10T05:34:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22479.348824 secs 2023-09-10T05:34:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:08 smithi082 ceph-mon[98694]: pgmap v12148: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 264 KiB/s wr, 28 op/s 2023-09-10T05:34:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22479.348824 secs 2023-09-10T05:34:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:08 smithi098 ceph-mon[103124]: pgmap v12148: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 264 KiB/s wr, 28 op/s 2023-09-10T05:34:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22479.348824 secs 2023-09-10T05:34:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:09 smithi170 ceph-mon[102832]: osdmap e827: 12 total, 12 up, 12 in 2023-09-10T05:34:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22480.387007 secs 2023-09-10T05:34:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:09 smithi082 ceph-mon[98694]: osdmap e827: 12 total, 12 up, 12 in 2023-09-10T05:34:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22480.387007 secs 2023-09-10T05:34:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:09 smithi098 ceph-mon[103124]: osdmap e827: 12 total, 12 up, 12 in 2023-09-10T05:34:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22480.387007 secs 2023-09-10T05:34:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22481.040489 secs 2023-09-10T05:34:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:10 smithi170 ceph-mon[102832]: pgmap v12150: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 28 op/s 2023-09-10T05:34:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22481.040489 secs 2023-09-10T05:34:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:10 smithi082 ceph-mon[98694]: pgmap v12150: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 28 op/s 2023-09-10T05:34:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22481.040489 secs 2023-09-10T05:34:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:10 smithi098 ceph-mon[103124]: pgmap v12150: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 28 op/s 2023-09-10T05:34:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:12 smithi170 ceph-mon[102832]: pgmap v12151: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 54 KiB/s wr, 26 op/s 2023-09-10T05:34:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:12 smithi082 ceph-mon[98694]: pgmap v12151: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 54 KiB/s wr, 26 op/s 2023-09-10T05:34:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:12 smithi098 ceph-mon[103124]: pgmap v12151: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 54 KiB/s wr, 26 op/s 2023-09-10T05:34:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22484.348887 secs 2023-09-10T05:34:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22484.348887 secs 2023-09-10T05:34:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22484.348887 secs 2023-09-10T05:34:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:14 smithi170 ceph-mon[102832]: pgmap v12152: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 54 KiB/s wr, 26 op/s 2023-09-10T05:34:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22485.387064 secs 2023-09-10T05:34:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:14 smithi082 ceph-mon[98694]: pgmap v12152: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 54 KiB/s wr, 26 op/s 2023-09-10T05:34:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22485.387064 secs 2023-09-10T05:34:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:14 smithi098 ceph-mon[103124]: pgmap v12152: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 54 KiB/s wr, 26 op/s 2023-09-10T05:34:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22485.387064 secs 2023-09-10T05:34:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22486.040558 secs 2023-09-10T05:34:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:15 smithi170 ceph-mon[102832]: osdmap e828: 12 total, 12 up, 12 in 2023-09-10T05:34:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22486.040558 secs 2023-09-10T05:34:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:15 smithi082 ceph-mon[98694]: osdmap e828: 12 total, 12 up, 12 in 2023-09-10T05:34:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22486.040558 secs 2023-09-10T05:34:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:15 smithi098 ceph-mon[103124]: osdmap e828: 12 total, 12 up, 12 in 2023-09-10T05:34:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:16 smithi170 ceph-mon[102832]: pgmap v12154: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:34:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:16 smithi082 ceph-mon[98694]: pgmap v12154: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:34:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:16 smithi098 ceph-mon[103124]: pgmap v12154: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:34:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:18 smithi170 ceph-mon[102832]: pgmap v12155: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T05:34:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22489.348996 secs 2023-09-10T05:34:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:18 smithi082 ceph-mon[98694]: pgmap v12155: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T05:34:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22489.348996 secs 2023-09-10T05:34:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:18 smithi098 ceph-mon[103124]: pgmap v12155: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T05:34:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22489.348996 secs 2023-09-10T05:34:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22490.387157 secs 2023-09-10T05:34:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22490.387157 secs 2023-09-10T05:34:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22490.387157 secs 2023-09-10T05:34:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22491.040625 secs 2023-09-10T05:34:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:20 smithi170 ceph-mon[102832]: pgmap v12156: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:34:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22491.040625 secs 2023-09-10T05:34:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:20 smithi082 ceph-mon[98694]: pgmap v12156: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:34:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22491.040625 secs 2023-09-10T05:34:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:20 smithi098 ceph-mon[103124]: pgmap v12156: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:34:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:22 smithi170 ceph-mon[102832]: pgmap v12157: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 173 KiB/s wr, 12 op/s 2023-09-10T05:34:22.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:22 smithi082 ceph-mon[98694]: pgmap v12157: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 173 KiB/s wr, 12 op/s 2023-09-10T05:34:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:22 smithi098 ceph-mon[103124]: pgmap v12157: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 173 KiB/s wr, 12 op/s 2023-09-10T05:34:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22494.349111 secs 2023-09-10T05:34:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22494.349111 secs 2023-09-10T05:34:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22494.349111 secs 2023-09-10T05:34:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:24 smithi170 ceph-mon[102832]: pgmap v12158: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 173 KiB/s wr, 12 op/s 2023-09-10T05:34:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22495.387216 secs 2023-09-10T05:34:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:24 smithi082 ceph-mon[98694]: pgmap v12158: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 173 KiB/s wr, 12 op/s 2023-09-10T05:34:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22495.387216 secs 2023-09-10T05:34:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:24 smithi098 ceph-mon[103124]: pgmap v12158: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 173 KiB/s wr, 12 op/s 2023-09-10T05:34:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22495.387216 secs 2023-09-10T05:34:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22496.040691 secs 2023-09-10T05:34:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22496.040691 secs 2023-09-10T05:34:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22496.040691 secs 2023-09-10T05:34:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:26 smithi170 ceph-mon[102832]: pgmap v12159: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 172 KiB/s wr, 12 op/s 2023-09-10T05:34:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:26 smithi082 ceph-mon[98694]: pgmap v12159: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 172 KiB/s wr, 12 op/s 2023-09-10T05:34:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:26 smithi098 ceph-mon[103124]: pgmap v12159: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 172 KiB/s wr, 12 op/s 2023-09-10T05:34:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:28 smithi170 ceph-mon[102832]: pgmap v12160: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 145 KiB/s wr, 15 op/s 2023-09-10T05:34:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:28 smithi082 ceph-mon[98694]: pgmap v12160: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 145 KiB/s wr, 15 op/s 2023-09-10T05:34:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:28 smithi098 ceph-mon[103124]: pgmap v12160: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 145 KiB/s wr, 15 op/s 2023-09-10T05:34:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22500.387278 secs 2023-09-10T05:34:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22500.387278 secs 2023-09-10T05:34:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22500.387278 secs 2023-09-10T05:34:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22501.040754 secs 2023-09-10T05:34:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:30 smithi170 ceph-mon[102832]: pgmap v12161: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 10 op/s 2023-09-10T05:34:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22501.040754 secs 2023-09-10T05:34:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:30 smithi082 ceph-mon[98694]: pgmap v12161: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 10 op/s 2023-09-10T05:34:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22501.040754 secs 2023-09-10T05:34:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:30 smithi098 ceph-mon[103124]: pgmap v12161: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 10 op/s 2023-09-10T05:34:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:34:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:34:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:34:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:32 smithi170 ceph-mon[102832]: pgmap v12162: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T05:34:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:34:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:34:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:34:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:32 smithi082 ceph-mon[98694]: pgmap v12162: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T05:34:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:34:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:34:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:34:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:32 smithi098 ceph-mon[103124]: pgmap v12162: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T05:34:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:34:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:34:32.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:34:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:34 smithi170 ceph-mon[102832]: pgmap v12163: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T05:34:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:34:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:34:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22505.387338 secs 2023-09-10T05:34:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22506.040821 secs 2023-09-10T05:34:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:34 smithi082 ceph-mon[98694]: pgmap v12163: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T05:34:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:34:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:34:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22505.387338 secs 2023-09-10T05:34:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22506.040821 secs 2023-09-10T05:34:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:34 smithi098 ceph-mon[103124]: pgmap v12163: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T05:34:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:34:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:34:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22505.387338 secs 2023-09-10T05:34:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22506.040821 secs 2023-09-10T05:34:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:35 smithi170 ceph-mon[102832]: pgmap v12164: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T05:34:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:35 smithi082 ceph-mon[98694]: pgmap v12164: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T05:34:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:35 smithi098 ceph-mon[103124]: pgmap v12164: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T05:34:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:34:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:34:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:34:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:34:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:34:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:34:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:34:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:34:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:37 smithi170 ceph-mon[102832]: pgmap v12165: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 474 KiB/s wr, 17 op/s 2023-09-10T05:34:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:34:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:34:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:37 smithi082 ceph-mon[98694]: pgmap v12165: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 474 KiB/s wr, 17 op/s 2023-09-10T05:34:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:34:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:34:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:37 smithi098 ceph-mon[103124]: pgmap v12165: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 474 KiB/s wr, 17 op/s 2023-09-10T05:34:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22510.387397 secs 2023-09-10T05:34:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22510.387397 secs 2023-09-10T05:34:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22510.387397 secs 2023-09-10T05:34:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22511.040886 secs 2023-09-10T05:34:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:39 smithi170 ceph-mon[102832]: pgmap v12166: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 474 KiB/s wr, 12 op/s 2023-09-10T05:34:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22511.040886 secs 2023-09-10T05:34:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:39 smithi082 ceph-mon[98694]: pgmap v12166: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 474 KiB/s wr, 12 op/s 2023-09-10T05:34:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22511.040886 secs 2023-09-10T05:34:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:39 smithi098 ceph-mon[103124]: pgmap v12166: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 474 KiB/s wr, 12 op/s 2023-09-10T05:34:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:42 smithi170 ceph-mon[102832]: pgmap v12167: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 587 KiB/s wr, 18 op/s 2023-09-10T05:34:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:42 smithi082 ceph-mon[98694]: pgmap v12167: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 587 KiB/s wr, 18 op/s 2023-09-10T05:34:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:42 smithi098 ceph-mon[103124]: pgmap v12167: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 587 KiB/s wr, 18 op/s 2023-09-10T05:34:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:44 smithi170 ceph-mon[102832]: pgmap v12168: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 512 KiB/s wr, 13 op/s 2023-09-10T05:34:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:34:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:34:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22515.387456 secs 2023-09-10T05:34:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:44 smithi082 ceph-mon[98694]: pgmap v12168: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 512 KiB/s wr, 13 op/s 2023-09-10T05:34:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:34:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:34:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22515.387456 secs 2023-09-10T05:34:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:44 smithi098 ceph-mon[103124]: pgmap v12168: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 512 KiB/s wr, 13 op/s 2023-09-10T05:34:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:34:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:34:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22515.387456 secs 2023-09-10T05:34:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22516.040955 secs 2023-09-10T05:34:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:34:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22516.040955 secs 2023-09-10T05:34:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:34:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22516.040955 secs 2023-09-10T05:34:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:34:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:46 smithi170 ceph-mon[102832]: pgmap v12169: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 512 KiB/s wr, 13 op/s 2023-09-10T05:34:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:34:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:34:46.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:34:46.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:34:46.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:34:46.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:34:46.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:34:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:46 smithi082 ceph-mon[98694]: pgmap v12169: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 512 KiB/s wr, 13 op/s 2023-09-10T05:34:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:34:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:34:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:34:46.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:34:46.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:34:46.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:34:46.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:34:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:46 smithi098 ceph-mon[103124]: pgmap v12169: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 512 KiB/s wr, 13 op/s 2023-09-10T05:34:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:34:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:34:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:34:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:34:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:34:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:34:46.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:34:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:48 smithi170 ceph-mon[102832]: pgmap v12170: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 825 KiB/s wr, 21 op/s 2023-09-10T05:34:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22519.349480 secs 2023-09-10T05:34:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:48 smithi082 ceph-mon[98694]: pgmap v12170: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 825 KiB/s wr, 21 op/s 2023-09-10T05:34:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22519.349480 secs 2023-09-10T05:34:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:48 smithi098 ceph-mon[103124]: pgmap v12170: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 825 KiB/s wr, 21 op/s 2023-09-10T05:34:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22519.349480 secs 2023-09-10T05:34:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22520.387491 secs 2023-09-10T05:34:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22520.387491 secs 2023-09-10T05:34:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22520.387491 secs 2023-09-10T05:34:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22521.041026 secs 2023-09-10T05:34:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:50 smithi170 ceph-mon[102832]: pgmap v12171: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 426 KiB/s wr, 14 op/s 2023-09-10T05:34:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22521.041026 secs 2023-09-10T05:34:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:50 smithi082 ceph-mon[98694]: pgmap v12171: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 426 KiB/s wr, 14 op/s 2023-09-10T05:34:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22521.041026 secs 2023-09-10T05:34:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:50 smithi098 ceph-mon[103124]: pgmap v12171: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 426 KiB/s wr, 14 op/s 2023-09-10T05:34:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:34:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:34:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:34:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:34:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:34:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:34:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:34:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:34:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:34:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:52 smithi170 ceph-mon[102832]: pgmap v12172: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 426 KiB/s wr, 20 op/s 2023-09-10T05:34:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:52 smithi082 ceph-mon[98694]: pgmap v12172: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 426 KiB/s wr, 20 op/s 2023-09-10T05:34:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:52 smithi098 ceph-mon[103124]: pgmap v12172: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 426 KiB/s wr, 20 op/s 2023-09-10T05:34:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22524.349541 secs 2023-09-10T05:34:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22524.349541 secs 2023-09-10T05:34:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22524.349541 secs 2023-09-10T05:34:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:54 smithi170 ceph-mon[102832]: pgmap v12173: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 313 KiB/s wr, 13 op/s 2023-09-10T05:34:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22525.387575 secs 2023-09-10T05:34:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22526.041089 secs 2023-09-10T05:34:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:54 smithi082 ceph-mon[98694]: pgmap v12173: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 313 KiB/s wr, 13 op/s 2023-09-10T05:34:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22525.387575 secs 2023-09-10T05:34:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22526.041089 secs 2023-09-10T05:34:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:54 smithi098 ceph-mon[103124]: pgmap v12173: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 313 KiB/s wr, 13 op/s 2023-09-10T05:34:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22525.387575 secs 2023-09-10T05:34:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22526.041089 secs 2023-09-10T05:34:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:55 smithi170 ceph-mon[102832]: pgmap v12174: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 313 KiB/s wr, 13 op/s 2023-09-10T05:34:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:55 smithi082 ceph-mon[98694]: pgmap v12174: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 313 KiB/s wr, 13 op/s 2023-09-10T05:34:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:55 smithi098 ceph-mon[103124]: pgmap v12174: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 313 KiB/s wr, 13 op/s 2023-09-10T05:34:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:58 smithi170 ceph-mon[102832]: pgmap v12175: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 486 KiB/s wr, 18 op/s 2023-09-10T05:34:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22529.349604 secs 2023-09-10T05:34:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:58 smithi082 ceph-mon[98694]: pgmap v12175: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 486 KiB/s wr, 18 op/s 2023-09-10T05:34:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22529.349604 secs 2023-09-10T05:34:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:58 smithi098 ceph-mon[103124]: pgmap v12175: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 486 KiB/s wr, 18 op/s 2023-09-10T05:34:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22529.349604 secs 2023-09-10T05:34:59.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:34:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22530.387634 secs 2023-09-10T05:34:59.462 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:34:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22530.387634 secs 2023-09-10T05:34:59.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:34:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22530.387634 secs 2023-09-10T05:35:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22531.041154 secs 2023-09-10T05:35:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:00 smithi170 ceph-mon[102832]: pgmap v12176: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T05:35:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22531.041154 secs 2023-09-10T05:35:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:00 smithi082 ceph-mon[98694]: pgmap v12176: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T05:35:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22531.041154 secs 2023-09-10T05:35:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:00 smithi098 ceph-mon[103124]: pgmap v12176: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T05:35:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:02 smithi170 ceph-mon[102832]: pgmap v12177: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 232 KiB/s wr, 15 op/s 2023-09-10T05:35:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:02 smithi082 ceph-mon[98694]: pgmap v12177: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 232 KiB/s wr, 15 op/s 2023-09-10T05:35:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:02 smithi098 ceph-mon[103124]: pgmap v12177: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 232 KiB/s wr, 15 op/s 2023-09-10T05:35:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22534.349664 secs 2023-09-10T05:35:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22534.349664 secs 2023-09-10T05:35:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22534.349664 secs 2023-09-10T05:35:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:04 smithi170 ceph-mon[102832]: pgmap v12178: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 231 KiB/s wr, 9 op/s 2023-09-10T05:35:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:35:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:35:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22535.387695 secs 2023-09-10T05:35:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:04 smithi082 ceph-mon[98694]: pgmap v12178: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 231 KiB/s wr, 9 op/s 2023-09-10T05:35:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:35:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:35:04.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22535.387695 secs 2023-09-10T05:35:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:04 smithi098 ceph-mon[103124]: pgmap v12178: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 231 KiB/s wr, 9 op/s 2023-09-10T05:35:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:35:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:35:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22535.387695 secs 2023-09-10T05:35:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22536.041218 secs 2023-09-10T05:35:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22536.041218 secs 2023-09-10T05:35:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22536.041218 secs 2023-09-10T05:35:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:06 smithi170 ceph-mon[102832]: pgmap v12179: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 231 KiB/s wr, 9 op/s 2023-09-10T05:35:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:35:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:35:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:06 smithi082 ceph-mon[98694]: pgmap v12179: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 231 KiB/s wr, 9 op/s 2023-09-10T05:35:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:35:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:35:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:06 smithi098 ceph-mon[103124]: pgmap v12179: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 231 KiB/s wr, 9 op/s 2023-09-10T05:35:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:35:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:35:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:08 smithi170 ceph-mon[102832]: pgmap v12180: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 276 KiB/s wr, 28 op/s 2023-09-10T05:35:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22539.349797 secs 2023-09-10T05:35:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:08 smithi082 ceph-mon[98694]: pgmap v12180: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 276 KiB/s wr, 28 op/s 2023-09-10T05:35:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22539.349797 secs 2023-09-10T05:35:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:08 smithi098 ceph-mon[103124]: pgmap v12180: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 276 KiB/s wr, 28 op/s 2023-09-10T05:35:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22539.349797 secs 2023-09-10T05:35:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:09 smithi170 ceph-mon[102832]: osdmap e829: 12 total, 12 up, 12 in 2023-09-10T05:35:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22540.387755 secs 2023-09-10T05:35:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:09 smithi082 ceph-mon[98694]: osdmap e829: 12 total, 12 up, 12 in 2023-09-10T05:35:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22540.387755 secs 2023-09-10T05:35:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:09 smithi098 ceph-mon[103124]: osdmap e829: 12 total, 12 up, 12 in 2023-09-10T05:35:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22540.387755 secs 2023-09-10T05:35:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22541.041283 secs 2023-09-10T05:35:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:10 smithi170 ceph-mon[102832]: pgmap v12182: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T05:35:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22541.041283 secs 2023-09-10T05:35:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:10 smithi082 ceph-mon[98694]: pgmap v12182: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T05:35:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22541.041283 secs 2023-09-10T05:35:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:10 smithi098 ceph-mon[103124]: pgmap v12182: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T05:35:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:12 smithi170 ceph-mon[102832]: pgmap v12183: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T05:35:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:12 smithi082 ceph-mon[98694]: pgmap v12183: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T05:35:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:12 smithi098 ceph-mon[103124]: pgmap v12183: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T05:35:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22544.349857 secs 2023-09-10T05:35:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22544.349857 secs 2023-09-10T05:35:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22544.349857 secs 2023-09-10T05:35:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:14 smithi170 ceph-mon[102832]: pgmap v12184: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T05:35:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22545.387840 secs 2023-09-10T05:35:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:14 smithi082 ceph-mon[98694]: pgmap v12184: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T05:35:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22545.387840 secs 2023-09-10T05:35:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:14 smithi098 ceph-mon[103124]: pgmap v12184: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T05:35:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22545.387840 secs 2023-09-10T05:35:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22546.041347 secs 2023-09-10T05:35:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:15 smithi170 ceph-mon[102832]: osdmap e830: 12 total, 12 up, 12 in 2023-09-10T05:35:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22546.041347 secs 2023-09-10T05:35:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:15 smithi082 ceph-mon[98694]: osdmap e830: 12 total, 12 up, 12 in 2023-09-10T05:35:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22546.041347 secs 2023-09-10T05:35:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:15 smithi098 ceph-mon[103124]: osdmap e830: 12 total, 12 up, 12 in 2023-09-10T05:35:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:16 smithi170 ceph-mon[102832]: pgmap v12186: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-10T05:35:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:16 smithi082 ceph-mon[98694]: pgmap v12186: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-10T05:35:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:16 smithi098 ceph-mon[103124]: pgmap v12186: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-10T05:35:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:18 smithi170 ceph-mon[102832]: pgmap v12187: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 344 B/s wr, 12 op/s 2023-09-10T05:35:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22549.349968 secs 2023-09-10T05:35:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:18 smithi082 ceph-mon[98694]: pgmap v12187: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 344 B/s wr, 12 op/s 2023-09-10T05:35:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22549.349968 secs 2023-09-10T05:35:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:18 smithi098 ceph-mon[103124]: pgmap v12187: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 344 B/s wr, 12 op/s 2023-09-10T05:35:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22549.349968 secs 2023-09-10T05:35:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22550.387897 secs 2023-09-10T05:35:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22550.387897 secs 2023-09-10T05:35:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22550.387897 secs 2023-09-10T05:35:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22551.041416 secs 2023-09-10T05:35:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:20 smithi170 ceph-mon[102832]: pgmap v12188: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T05:35:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22551.041416 secs 2023-09-10T05:35:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:20 smithi082 ceph-mon[98694]: pgmap v12188: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T05:35:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22551.041416 secs 2023-09-10T05:35:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:20 smithi098 ceph-mon[103124]: pgmap v12188: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T05:35:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:22 smithi170 ceph-mon[102832]: pgmap v12189: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 614 B/s wr, 10 op/s 2023-09-10T05:35:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:22 smithi082 ceph-mon[98694]: pgmap v12189: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 614 B/s wr, 10 op/s 2023-09-10T05:35:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:22 smithi098 ceph-mon[103124]: pgmap v12189: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 614 B/s wr, 10 op/s 2023-09-10T05:35:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22554.350080 secs 2023-09-10T05:35:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22554.350080 secs 2023-09-10T05:35:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22554.350080 secs 2023-09-10T05:35:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:24 smithi170 ceph-mon[102832]: pgmap v12190: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 614 B/s wr, 10 op/s 2023-09-10T05:35:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22555.387956 secs 2023-09-10T05:35:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22556.041480 secs 2023-09-10T05:35:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:24 smithi082 ceph-mon[98694]: pgmap v12190: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 614 B/s wr, 10 op/s 2023-09-10T05:35:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22555.387956 secs 2023-09-10T05:35:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22556.041480 secs 2023-09-10T05:35:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:24 smithi098 ceph-mon[103124]: pgmap v12190: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 614 B/s wr, 10 op/s 2023-09-10T05:35:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22555.387956 secs 2023-09-10T05:35:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22556.041480 secs 2023-09-10T05:35:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:25 smithi170 ceph-mon[102832]: pgmap v12191: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 608 B/s wr, 10 op/s 2023-09-10T05:35:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:25 smithi082 ceph-mon[98694]: pgmap v12191: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 608 B/s wr, 10 op/s 2023-09-10T05:35:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:25 smithi098 ceph-mon[103124]: pgmap v12191: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 608 B/s wr, 10 op/s 2023-09-10T05:35:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:28 smithi170 ceph-mon[102832]: pgmap v12192: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 76 KiB/s wr, 13 op/s 2023-09-10T05:35:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:28 smithi082 ceph-mon[98694]: pgmap v12192: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 76 KiB/s wr, 13 op/s 2023-09-10T05:35:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:28 smithi098 ceph-mon[103124]: pgmap v12192: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 76 KiB/s wr, 13 op/s 2023-09-10T05:35:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22560.388041 secs 2023-09-10T05:35:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22560.388041 secs 2023-09-10T05:35:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22560.388041 secs 2023-09-10T05:35:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22561.041545 secs 2023-09-10T05:35:30.915 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:30 smithi170 ceph-mon[102832]: pgmap v12193: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T05:35:30.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22561.041545 secs 2023-09-10T05:35:30.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:30 smithi098 ceph-mon[103124]: pgmap v12193: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T05:35:30.917 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22561.041545 secs 2023-09-10T05:35:30.917 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:30 smithi082 ceph-mon[98694]: pgmap v12193: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T05:35:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:35:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:35:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:35:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:32 smithi170 ceph-mon[102832]: pgmap v12194: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 152 KiB/s wr, 14 op/s 2023-09-10T05:35:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:35:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:35:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:35:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:32 smithi082 ceph-mon[98694]: pgmap v12194: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 152 KiB/s wr, 14 op/s 2023-09-10T05:35:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:35:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:35:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:35:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:32 smithi098 ceph-mon[103124]: pgmap v12194: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 152 KiB/s wr, 14 op/s 2023-09-10T05:35:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:35:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:35:32.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:35:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:34 smithi170 ceph-mon[102832]: pgmap v12195: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 9 op/s 2023-09-10T05:35:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22565.388100 secs 2023-09-10T05:35:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:34 smithi082 ceph-mon[98694]: pgmap v12195: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 9 op/s 2023-09-10T05:35:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22565.388100 secs 2023-09-10T05:35:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:34 smithi098 ceph-mon[103124]: pgmap v12195: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 9 op/s 2023-09-10T05:35:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22565.388100 secs 2023-09-10T05:35:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22566.041609 secs 2023-09-10T05:35:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22566.041609 secs 2023-09-10T05:35:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22566.041609 secs 2023-09-10T05:35:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:36 smithi170 ceph-mon[102832]: pgmap v12196: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 9 op/s 2023-09-10T05:35:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:36 smithi082 ceph-mon[98694]: pgmap v12196: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 9 op/s 2023-09-10T05:35:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:36 smithi098 ceph-mon[103124]: pgmap v12196: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 9 op/s 2023-09-10T05:35:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:38 smithi170 ceph-mon[102832]: pgmap v12197: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 384 KiB/s wr, 15 op/s 2023-09-10T05:35:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:35:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:35:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:38 smithi082 ceph-mon[98694]: pgmap v12197: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 384 KiB/s wr, 15 op/s 2023-09-10T05:35:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:35:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:35:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:38 smithi098 ceph-mon[103124]: pgmap v12197: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 384 KiB/s wr, 15 op/s 2023-09-10T05:35:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:35:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:35:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22570.388160 secs 2023-09-10T05:35:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22570.388160 secs 2023-09-10T05:35:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22570.388160 secs 2023-09-10T05:35:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22571.041674 secs 2023-09-10T05:35:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:40 smithi170 ceph-mon[102832]: pgmap v12198: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 308 KiB/s wr, 10 op/s 2023-09-10T05:35:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:35:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:35:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22571.041674 secs 2023-09-10T05:35:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:40 smithi082 ceph-mon[98694]: pgmap v12198: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 308 KiB/s wr, 10 op/s 2023-09-10T05:35:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:35:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:35:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22571.041674 secs 2023-09-10T05:35:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:40 smithi098 ceph-mon[103124]: pgmap v12198: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 308 KiB/s wr, 10 op/s 2023-09-10T05:35:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:35:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:35:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:42 smithi170 ceph-mon[102832]: pgmap v12199: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 427 KiB/s wr, 18 op/s 2023-09-10T05:35:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:42 smithi082 ceph-mon[98694]: pgmap v12199: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 427 KiB/s wr, 18 op/s 2023-09-10T05:35:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:42 smithi098 ceph-mon[103124]: pgmap v12199: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 427 KiB/s wr, 18 op/s 2023-09-10T05:35:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:44 smithi170 ceph-mon[102832]: pgmap v12200: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 352 KiB/s wr, 13 op/s 2023-09-10T05:35:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22575.388218 secs 2023-09-10T05:35:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:44 smithi082 ceph-mon[98694]: pgmap v12200: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 352 KiB/s wr, 13 op/s 2023-09-10T05:35:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22575.388218 secs 2023-09-10T05:35:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:44 smithi098 ceph-mon[103124]: pgmap v12200: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 352 KiB/s wr, 13 op/s 2023-09-10T05:35:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22575.388218 secs 2023-09-10T05:35:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22576.041737 secs 2023-09-10T05:35:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22576.041737 secs 2023-09-10T05:35:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22576.041737 secs 2023-09-10T05:35:46.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:46 smithi170 ceph-mon[102832]: pgmap v12201: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 352 KiB/s wr, 13 op/s 2023-09-10T05:35:46.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:35:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:46 smithi082 ceph-mon[98694]: pgmap v12201: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 352 KiB/s wr, 13 op/s 2023-09-10T05:35:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:35:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:46 smithi098 ceph-mon[103124]: pgmap v12201: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 352 KiB/s wr, 13 op/s 2023-09-10T05:35:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:35:47.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:35:47.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:35:47.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:35:47.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:35:47.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:47 smithi170 ceph-mon[102832]: pgmap v12202: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 553 KiB/s wr, 20 op/s 2023-09-10T05:35:47.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:35:47.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:35:47.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:35:47.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:35:47.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:35:47.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:35:47.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:35:47.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:47 smithi082 ceph-mon[98694]: pgmap v12202: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 553 KiB/s wr, 20 op/s 2023-09-10T05:35:47.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:35:47.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:35:47.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:35:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:35:47.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:35:47.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:35:47.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:35:47.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:47 smithi098 ceph-mon[103124]: pgmap v12202: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 553 KiB/s wr, 20 op/s 2023-09-10T05:35:47.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:35:47.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:35:47.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:35:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22580.388276 secs 2023-09-10T05:35:48.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22580.388276 secs 2023-09-10T05:35:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22580.388276 secs 2023-09-10T05:35:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22581.041801 secs 2023-09-10T05:35:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:49 smithi170 ceph-mon[102832]: pgmap v12203: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 321 KiB/s wr, 14 op/s 2023-09-10T05:35:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22581.041801 secs 2023-09-10T05:35:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:49 smithi082 ceph-mon[98694]: pgmap v12203: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 321 KiB/s wr, 14 op/s 2023-09-10T05:35:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22581.041801 secs 2023-09-10T05:35:50.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:49 smithi098 ceph-mon[103124]: pgmap v12203: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 321 KiB/s wr, 14 op/s 2023-09-10T05:35:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:35:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:35:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:35:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:35:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:35:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:35:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:51 smithi170 ceph-mon[102832]: pgmap v12204: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 382 KiB/s wr, 20 op/s 2023-09-10T05:35:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:51 smithi082 ceph-mon[98694]: pgmap v12204: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 382 KiB/s wr, 20 op/s 2023-09-10T05:35:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:51 smithi098 ceph-mon[103124]: pgmap v12204: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 382 KiB/s wr, 20 op/s 2023-09-10T05:35:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:35:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:35:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:35:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:35:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:35:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:35:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22584.350490 secs 2023-09-10T05:35:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:53 smithi170 ceph-mon[102832]: pgmap v12205: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 263 KiB/s wr, 12 op/s 2023-09-10T05:35:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22585.388333 secs 2023-09-10T05:35:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22584.350490 secs 2023-09-10T05:35:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:53 smithi082 ceph-mon[98694]: pgmap v12205: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 263 KiB/s wr, 12 op/s 2023-09-10T05:35:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22585.388333 secs 2023-09-10T05:35:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22584.350490 secs 2023-09-10T05:35:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:53 smithi098 ceph-mon[103124]: pgmap v12205: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 263 KiB/s wr, 12 op/s 2023-09-10T05:35:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22585.388333 secs 2023-09-10T05:35:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22586.041867 secs 2023-09-10T05:35:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22586.041867 secs 2023-09-10T05:35:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22586.041867 secs 2023-09-10T05:35:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:55 smithi170 ceph-mon[102832]: pgmap v12206: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 263 KiB/s wr, 12 op/s 2023-09-10T05:35:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:55 smithi082 ceph-mon[98694]: pgmap v12206: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 263 KiB/s wr, 12 op/s 2023-09-10T05:35:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:55 smithi098 ceph-mon[103124]: pgmap v12206: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 263 KiB/s wr, 12 op/s 2023-09-10T05:35:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:35:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:35:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:35:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:35:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:35:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:35:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:35:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:35:57.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:35:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:57 smithi170 ceph-mon[102832]: pgmap v12207: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 438 KiB/s wr, 19 op/s 2023-09-10T05:35:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22589.350603 secs 2023-09-10T05:35:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:57 smithi082 ceph-mon[98694]: pgmap v12207: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 438 KiB/s wr, 19 op/s 2023-09-10T05:35:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22589.350603 secs 2023-09-10T05:35:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:57 smithi098 ceph-mon[103124]: pgmap v12207: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 438 KiB/s wr, 19 op/s 2023-09-10T05:35:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22589.350603 secs 2023-09-10T05:35:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22590.388393 secs 2023-09-10T05:35:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22590.388393 secs 2023-09-10T05:35:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22590.388393 secs 2023-09-10T05:36:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22591.041932 secs 2023-09-10T05:36:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:35:59 smithi170 ceph-mon[102832]: pgmap v12208: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 236 KiB/s wr, 12 op/s 2023-09-10T05:36:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22591.041932 secs 2023-09-10T05:36:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:35:59 smithi082 ceph-mon[98694]: pgmap v12208: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 236 KiB/s wr, 12 op/s 2023-09-10T05:36:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22591.041932 secs 2023-09-10T05:36:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:35:59 smithi098 ceph-mon[103124]: pgmap v12208: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 236 KiB/s wr, 12 op/s 2023-09-10T05:36:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:02 smithi170 ceph-mon[102832]: pgmap v12209: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 294 KiB/s wr, 16 op/s 2023-09-10T05:36:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:02 smithi082 ceph-mon[98694]: pgmap v12209: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 294 KiB/s wr, 16 op/s 2023-09-10T05:36:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:02 smithi098 ceph-mon[103124]: pgmap v12209: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 294 KiB/s wr, 16 op/s 2023-09-10T05:36:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22594.350686 secs 2023-09-10T05:36:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22594.350686 secs 2023-09-10T05:36:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22594.350686 secs 2023-09-10T05:36:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:04 smithi170 ceph-mon[102832]: pgmap v12210: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 232 KiB/s wr, 10 op/s 2023-09-10T05:36:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22595.388451 secs 2023-09-10T05:36:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:04 smithi082 ceph-mon[98694]: pgmap v12210: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 232 KiB/s wr, 10 op/s 2023-09-10T05:36:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22595.388451 secs 2023-09-10T05:36:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:04 smithi098 ceph-mon[103124]: pgmap v12210: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 232 KiB/s wr, 10 op/s 2023-09-10T05:36:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22595.388451 secs 2023-09-10T05:36:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22596.041996 secs 2023-09-10T05:36:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22596.041996 secs 2023-09-10T05:36:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22596.041996 secs 2023-09-10T05:36:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:06 smithi170 ceph-mon[102832]: pgmap v12211: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 232 KiB/s wr, 10 op/s 2023-09-10T05:36:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:06 smithi082 ceph-mon[98694]: pgmap v12211: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 232 KiB/s wr, 10 op/s 2023-09-10T05:36:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:06 smithi098 ceph-mon[103124]: pgmap v12211: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 232 KiB/s wr, 10 op/s 2023-09-10T05:36:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:08 smithi170 ceph-mon[102832]: pgmap v12212: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 277 KiB/s wr, 29 op/s 2023-09-10T05:36:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22599.350788 secs 2023-09-10T05:36:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:08 smithi082 ceph-mon[98694]: pgmap v12212: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 277 KiB/s wr, 29 op/s 2023-09-10T05:36:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22599.350788 secs 2023-09-10T05:36:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:08 smithi098 ceph-mon[103124]: pgmap v12212: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 277 KiB/s wr, 29 op/s 2023-09-10T05:36:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22599.350788 secs 2023-09-10T05:36:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:09 smithi170 ceph-mon[102832]: osdmap e831: 12 total, 12 up, 12 in 2023-09-10T05:36:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22600.388535 secs 2023-09-10T05:36:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:09 smithi082 ceph-mon[98694]: osdmap e831: 12 total, 12 up, 12 in 2023-09-10T05:36:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22600.388535 secs 2023-09-10T05:36:09.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:09 smithi098 ceph-mon[103124]: osdmap e831: 12 total, 12 up, 12 in 2023-09-10T05:36:09.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22600.388535 secs 2023-09-10T05:36:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22601.042061 secs 2023-09-10T05:36:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:10 smithi170 ceph-mon[102832]: pgmap v12214: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 28 op/s 2023-09-10T05:36:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22601.042061 secs 2023-09-10T05:36:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:10 smithi082 ceph-mon[98694]: pgmap v12214: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 28 op/s 2023-09-10T05:36:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22601.042061 secs 2023-09-10T05:36:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:10 smithi098 ceph-mon[103124]: pgmap v12214: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 28 op/s 2023-09-10T05:36:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:12 smithi170 ceph-mon[102832]: pgmap v12215: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T05:36:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:12 smithi082 ceph-mon[98694]: pgmap v12215: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T05:36:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:12 smithi098 ceph-mon[103124]: pgmap v12215: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T05:36:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22604.350877 secs 2023-09-10T05:36:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22604.350877 secs 2023-09-10T05:36:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22604.350877 secs 2023-09-10T05:36:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:14 smithi170 ceph-mon[102832]: pgmap v12216: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T05:36:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22605.388593 secs 2023-09-10T05:36:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:14 smithi082 ceph-mon[98694]: pgmap v12216: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T05:36:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22605.388593 secs 2023-09-10T05:36:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:14 smithi098 ceph-mon[103124]: pgmap v12216: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T05:36:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22605.388593 secs 2023-09-10T05:36:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22606.042126 secs 2023-09-10T05:36:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:15 smithi170 ceph-mon[102832]: osdmap e832: 12 total, 12 up, 12 in 2023-09-10T05:36:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22606.042126 secs 2023-09-10T05:36:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:15 smithi082 ceph-mon[98694]: osdmap e832: 12 total, 12 up, 12 in 2023-09-10T05:36:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22606.042126 secs 2023-09-10T05:36:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:15 smithi098 ceph-mon[103124]: osdmap e832: 12 total, 12 up, 12 in 2023-09-10T05:36:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:16 smithi170 ceph-mon[102832]: pgmap v12218: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:36:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:16 smithi082 ceph-mon[98694]: pgmap v12218: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:36:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:16 smithi098 ceph-mon[103124]: pgmap v12218: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:36:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:18 smithi170 ceph-mon[102832]: pgmap v12219: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2023-09-10T05:36:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22609.350937 secs 2023-09-10T05:36:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:18 smithi082 ceph-mon[98694]: pgmap v12219: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2023-09-10T05:36:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22609.350937 secs 2023-09-10T05:36:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:18 smithi098 ceph-mon[103124]: pgmap v12219: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2023-09-10T05:36:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22609.350937 secs 2023-09-10T05:36:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22610.388675 secs 2023-09-10T05:36:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22610.388675 secs 2023-09-10T05:36:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22610.388675 secs 2023-09-10T05:36:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22611.042192 secs 2023-09-10T05:36:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:20 smithi170 ceph-mon[102832]: pgmap v12220: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:36:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22611.042192 secs 2023-09-10T05:36:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:20 smithi082 ceph-mon[98694]: pgmap v12220: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:36:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22611.042192 secs 2023-09-10T05:36:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:20 smithi098 ceph-mon[103124]: pgmap v12220: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:36:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:22 smithi170 ceph-mon[102832]: pgmap v12221: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:36:22.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:22 smithi082 ceph-mon[98694]: pgmap v12221: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:36:22.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:22 smithi098 ceph-mon[103124]: pgmap v12221: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:36:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22614.351000 secs 2023-09-10T05:36:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22614.351000 secs 2023-09-10T05:36:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22614.351000 secs 2023-09-10T05:36:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:24 smithi170 ceph-mon[102832]: pgmap v12222: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:36:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22615.388739 secs 2023-09-10T05:36:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22616.042256 secs 2023-09-10T05:36:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:24 smithi082 ceph-mon[98694]: pgmap v12222: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:36:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22615.388739 secs 2023-09-10T05:36:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22616.042256 secs 2023-09-10T05:36:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:24 smithi098 ceph-mon[103124]: pgmap v12222: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 83 KiB/s wr, 11 op/s 2023-09-10T05:36:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22615.388739 secs 2023-09-10T05:36:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22616.042256 secs 2023-09-10T05:36:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:25 smithi170 ceph-mon[102832]: pgmap v12223: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T05:36:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:25 smithi082 ceph-mon[98694]: pgmap v12223: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T05:36:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:25 smithi098 ceph-mon[103124]: pgmap v12223: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T05:36:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:28 smithi170 ceph-mon[102832]: pgmap v12224: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 70 KiB/s wr, 14 op/s 2023-09-10T05:36:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:28 smithi082 ceph-mon[98694]: pgmap v12224: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 70 KiB/s wr, 14 op/s 2023-09-10T05:36:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:28 smithi098 ceph-mon[103124]: pgmap v12224: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 70 KiB/s wr, 14 op/s 2023-09-10T05:36:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22620.388796 secs 2023-09-10T05:36:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22620.388796 secs 2023-09-10T05:36:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22620.388796 secs 2023-09-10T05:36:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22621.042320 secs 2023-09-10T05:36:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:30 smithi170 ceph-mon[102832]: pgmap v12225: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T05:36:30.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:36:30.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:36:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22621.042320 secs 2023-09-10T05:36:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:30 smithi082 ceph-mon[98694]: pgmap v12225: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T05:36:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:36:30.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:36:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22621.042320 secs 2023-09-10T05:36:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:30 smithi098 ceph-mon[103124]: pgmap v12225: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T05:36:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:36:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:36:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:36:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:36:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:36:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:32 smithi170 ceph-mon[102832]: pgmap v12226: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 188 KiB/s wr, 16 op/s 2023-09-10T05:36:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:36:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:36:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:36:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:32 smithi082 ceph-mon[98694]: pgmap v12226: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 188 KiB/s wr, 16 op/s 2023-09-10T05:36:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:36:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:36:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:36:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:32 smithi098 ceph-mon[103124]: pgmap v12226: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 188 KiB/s wr, 16 op/s 2023-09-10T05:36:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:36:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:36:32.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:36:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:34 smithi170 ceph-mon[102832]: pgmap v12227: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T05:36:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22625.388854 secs 2023-09-10T05:36:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:34 smithi082 ceph-mon[98694]: pgmap v12227: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T05:36:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22625.388854 secs 2023-09-10T05:36:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:34 smithi098 ceph-mon[103124]: pgmap v12227: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T05:36:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22625.388854 secs 2023-09-10T05:36:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22626.042387 secs 2023-09-10T05:36:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22626.042387 secs 2023-09-10T05:36:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22626.042387 secs 2023-09-10T05:36:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:36 smithi170 ceph-mon[102832]: pgmap v12228: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T05:36:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:36 smithi082 ceph-mon[98694]: pgmap v12228: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T05:36:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:36 smithi098 ceph-mon[103124]: pgmap v12228: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T05:36:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:38 smithi170 ceph-mon[102832]: pgmap v12229: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 292 KiB/s wr, 17 op/s 2023-09-10T05:36:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:36:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:36:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:38 smithi082 ceph-mon[98694]: pgmap v12229: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 292 KiB/s wr, 17 op/s 2023-09-10T05:36:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:36:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:36:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:38 smithi098 ceph-mon[103124]: pgmap v12229: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 292 KiB/s wr, 17 op/s 2023-09-10T05:36:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:36:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:36:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22630.388937 secs 2023-09-10T05:36:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22630.388937 secs 2023-09-10T05:36:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22630.388937 secs 2023-09-10T05:36:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22631.042450 secs 2023-09-10T05:36:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:40 smithi170 ceph-mon[102832]: pgmap v12230: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 291 KiB/s wr, 12 op/s 2023-09-10T05:36:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:36:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:36:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22631.042450 secs 2023-09-10T05:36:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:40 smithi082 ceph-mon[98694]: pgmap v12230: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 291 KiB/s wr, 12 op/s 2023-09-10T05:36:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:36:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:36:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22631.042450 secs 2023-09-10T05:36:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:40 smithi098 ceph-mon[103124]: pgmap v12230: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 291 KiB/s wr, 12 op/s 2023-09-10T05:36:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:36:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:36:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:42 smithi170 ceph-mon[102832]: pgmap v12231: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 521 KiB/s wr, 20 op/s 2023-09-10T05:36:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:42 smithi082 ceph-mon[98694]: pgmap v12231: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 521 KiB/s wr, 20 op/s 2023-09-10T05:36:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:42 smithi098 ceph-mon[103124]: pgmap v12231: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 521 KiB/s wr, 20 op/s 2023-09-10T05:36:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:44 smithi170 ceph-mon[102832]: pgmap v12232: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 403 KiB/s wr, 13 op/s 2023-09-10T05:36:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22635.389000 secs 2023-09-10T05:36:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:44 smithi082 ceph-mon[98694]: pgmap v12232: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 403 KiB/s wr, 13 op/s 2023-09-10T05:36:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22635.389000 secs 2023-09-10T05:36:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:44 smithi098 ceph-mon[103124]: pgmap v12232: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 403 KiB/s wr, 13 op/s 2023-09-10T05:36:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22635.389000 secs 2023-09-10T05:36:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22636.042484 secs 2023-09-10T05:36:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22636.042484 secs 2023-09-10T05:36:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22636.042484 secs 2023-09-10T05:36:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:46 smithi170 ceph-mon[102832]: pgmap v12233: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 403 KiB/s wr, 13 op/s 2023-09-10T05:36:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:46 smithi082 ceph-mon[98694]: pgmap v12233: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 403 KiB/s wr, 13 op/s 2023-09-10T05:36:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:46 smithi098 ceph-mon[103124]: pgmap v12233: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 403 KiB/s wr, 13 op/s 2023-09-10T05:36:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:48 smithi170 ceph-mon[102832]: pgmap v12234: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 507 KiB/s wr, 19 op/s 2023-09-10T05:36:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:36:48.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22639.351369 secs 2023-09-10T05:36:48.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:36:48.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:36:48.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:36:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:48 smithi082 ceph-mon[98694]: pgmap v12234: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 507 KiB/s wr, 19 op/s 2023-09-10T05:36:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:36:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22639.351369 secs 2023-09-10T05:36:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:36:48.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:36:48.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:36:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:48 smithi098 ceph-mon[103124]: pgmap v12234: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 507 KiB/s wr, 19 op/s 2023-09-10T05:36:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:36:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22639.351369 secs 2023-09-10T05:36:48.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:36:48.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:36:48.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:36:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22640.389059 secs 2023-09-10T05:36:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22640.389059 secs 2023-09-10T05:36:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22640.389059 secs 2023-09-10T05:36:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22641.042548 secs 2023-09-10T05:36:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:50 smithi170 ceph-mon[102832]: pgmap v12235: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 334 KiB/s wr, 13 op/s 2023-09-10T05:36:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22641.042548 secs 2023-09-10T05:36:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:50 smithi082 ceph-mon[98694]: pgmap v12235: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 334 KiB/s wr, 13 op/s 2023-09-10T05:36:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22641.042548 secs 2023-09-10T05:36:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:50 smithi098 ceph-mon[103124]: pgmap v12235: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 334 KiB/s wr, 13 op/s 2023-09-10T05:36:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:36:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:36:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:36:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:36:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:36:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:36:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:36:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:36:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:36:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:52 smithi170 ceph-mon[102832]: pgmap v12236: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 444 KiB/s wr, 20 op/s 2023-09-10T05:36:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:52 smithi082 ceph-mon[98694]: pgmap v12236: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 444 KiB/s wr, 20 op/s 2023-09-10T05:36:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:52 smithi098 ceph-mon[103124]: pgmap v12236: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 444 KiB/s wr, 20 op/s 2023-09-10T05:36:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22644.351484 secs 2023-09-10T05:36:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22644.351484 secs 2023-09-10T05:36:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22644.351484 secs 2023-09-10T05:36:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:54 smithi082 ceph-mon[98694]: pgmap v12237: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 214 KiB/s wr, 12 op/s 2023-09-10T05:36:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22645.389120 secs 2023-09-10T05:36:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22646.042614 secs 2023-09-10T05:36:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:54 smithi098 ceph-mon[103124]: pgmap v12237: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 214 KiB/s wr, 12 op/s 2023-09-10T05:36:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22645.389120 secs 2023-09-10T05:36:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22646.042614 secs 2023-09-10T05:36:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:54 smithi170 ceph-mon[102832]: pgmap v12237: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 214 KiB/s wr, 12 op/s 2023-09-10T05:36:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22645.389120 secs 2023-09-10T05:36:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22646.042614 secs 2023-09-10T05:36:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:55 smithi170 ceph-mon[102832]: pgmap v12238: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 214 KiB/s wr, 12 op/s 2023-09-10T05:36:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:55 smithi082 ceph-mon[98694]: pgmap v12238: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 214 KiB/s wr, 12 op/s 2023-09-10T05:36:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:55 smithi098 ceph-mon[103124]: pgmap v12238: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 214 KiB/s wr, 12 op/s 2023-09-10T05:36:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:36:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:36:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:57 smithi082 ceph-mon[98694]: pgmap v12239: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 295 KiB/s wr, 17 op/s 2023-09-10T05:36:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22649.351607 secs 2023-09-10T05:36:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:36:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:36:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:57 smithi098 ceph-mon[103124]: pgmap v12239: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 295 KiB/s wr, 17 op/s 2023-09-10T05:36:58.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22649.351607 secs 2023-09-10T05:36:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:36:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:36:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:57 smithi170 ceph-mon[102832]: pgmap v12239: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 295 KiB/s wr, 17 op/s 2023-09-10T05:36:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22649.351607 secs 2023-09-10T05:36:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22650.389203 secs 2023-09-10T05:36:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22650.389203 secs 2023-09-10T05:36:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22650.389203 secs 2023-09-10T05:37:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22651.042682 secs 2023-09-10T05:37:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:59 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:37:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:59 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:37:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:36:59 smithi082 ceph-mon[98694]: pgmap v12240: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T05:37:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22651.042682 secs 2023-09-10T05:37:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:59 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:37:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:59 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:37:00.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:36:59 smithi098 ceph-mon[103124]: pgmap v12240: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T05:37:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22651.042682 secs 2023-09-10T05:37:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:59 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:37:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:59 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:37:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:36:59 smithi170 ceph-mon[102832]: pgmap v12240: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T05:37:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:02 smithi170 ceph-mon[102832]: pgmap v12241: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 191 KiB/s wr, 16 op/s 2023-09-10T05:37:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:02 smithi082 ceph-mon[98694]: pgmap v12241: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 191 KiB/s wr, 16 op/s 2023-09-10T05:37:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:02 smithi098 ceph-mon[103124]: pgmap v12241: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 191 KiB/s wr, 16 op/s 2023-09-10T05:37:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22654.351669 secs 2023-09-10T05:37:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22654.351669 secs 2023-09-10T05:37:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22654.351669 secs 2023-09-10T05:37:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:04 smithi170 ceph-mon[102832]: pgmap v12242: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T05:37:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22655.389261 secs 2023-09-10T05:37:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:04 smithi082 ceph-mon[98694]: pgmap v12242: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T05:37:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22655.389261 secs 2023-09-10T05:37:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:04 smithi098 ceph-mon[103124]: pgmap v12242: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T05:37:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22655.389261 secs 2023-09-10T05:37:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22656.042746 secs 2023-09-10T05:37:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22656.042746 secs 2023-09-10T05:37:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22656.042746 secs 2023-09-10T05:37:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:06 smithi170 ceph-mon[102832]: pgmap v12243: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T05:37:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:06 smithi082 ceph-mon[98694]: pgmap v12243: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T05:37:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:06 smithi098 ceph-mon[103124]: pgmap v12243: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T05:37:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:08 smithi170 ceph-mon[102832]: pgmap v12244: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 184 KiB/s wr, 29 op/s 2023-09-10T05:37:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22659.351766 secs 2023-09-10T05:37:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:08 smithi082 ceph-mon[98694]: pgmap v12244: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 184 KiB/s wr, 29 op/s 2023-09-10T05:37:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22659.351766 secs 2023-09-10T05:37:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:08 smithi098 ceph-mon[103124]: pgmap v12244: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 184 KiB/s wr, 29 op/s 2023-09-10T05:37:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22659.351766 secs 2023-09-10T05:37:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:09 smithi170 ceph-mon[102832]: osdmap e833: 12 total, 12 up, 12 in 2023-09-10T05:37:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22660.389346 secs 2023-09-10T05:37:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:09 smithi082 ceph-mon[98694]: osdmap e833: 12 total, 12 up, 12 in 2023-09-10T05:37:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22660.389346 secs 2023-09-10T05:37:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:09 smithi098 ceph-mon[103124]: osdmap e833: 12 total, 12 up, 12 in 2023-09-10T05:37:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22660.389346 secs 2023-09-10T05:37:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22661.042811 secs 2023-09-10T05:37:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:10 smithi170 ceph-mon[102832]: pgmap v12246: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T05:37:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22661.042811 secs 2023-09-10T05:37:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:10 smithi082 ceph-mon[98694]: pgmap v12246: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T05:37:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22661.042811 secs 2023-09-10T05:37:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:10 smithi098 ceph-mon[103124]: pgmap v12246: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T05:37:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:12 smithi170 ceph-mon[102832]: pgmap v12247: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T05:37:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:12 smithi082 ceph-mon[98694]: pgmap v12247: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T05:37:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:12 smithi098 ceph-mon[103124]: pgmap v12247: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T05:37:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22664.351771 secs 2023-09-10T05:37:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22664.351771 secs 2023-09-10T05:37:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22664.351771 secs 2023-09-10T05:37:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:14 smithi170 ceph-mon[102832]: pgmap v12248: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T05:37:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22665.389405 secs 2023-09-10T05:37:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22666.042879 secs 2023-09-10T05:37:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:14 smithi082 ceph-mon[98694]: pgmap v12248: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T05:37:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22665.389405 secs 2023-09-10T05:37:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22666.042879 secs 2023-09-10T05:37:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:14 smithi098 ceph-mon[103124]: pgmap v12248: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 29 op/s 2023-09-10T05:37:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22665.389405 secs 2023-09-10T05:37:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22666.042879 secs 2023-09-10T05:37:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:15 smithi170 ceph-mon[102832]: osdmap e834: 12 total, 12 up, 12 in 2023-09-10T05:37:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:15 smithi170 ceph-mon[102832]: pgmap v12250: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T05:37:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:15 smithi082 ceph-mon[98694]: osdmap e834: 12 total, 12 up, 12 in 2023-09-10T05:37:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:15 smithi082 ceph-mon[98694]: pgmap v12250: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T05:37:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:15 smithi098 ceph-mon[103124]: osdmap e834: 12 total, 12 up, 12 in 2023-09-10T05:37:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:15 smithi098 ceph-mon[103124]: pgmap v12250: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T05:37:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:18 smithi170 ceph-mon[102832]: pgmap v12251: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 344 B/s wr, 12 op/s 2023-09-10T05:37:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22669.351893 secs 2023-09-10T05:37:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:18 smithi082 ceph-mon[98694]: pgmap v12251: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 344 B/s wr, 12 op/s 2023-09-10T05:37:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22669.351893 secs 2023-09-10T05:37:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:18 smithi098 ceph-mon[103124]: pgmap v12251: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 344 B/s wr, 12 op/s 2023-09-10T05:37:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22669.351893 secs 2023-09-10T05:37:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22670.389462 secs 2023-09-10T05:37:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22670.389462 secs 2023-09-10T05:37:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22670.389462 secs 2023-09-10T05:37:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22671.042946 secs 2023-09-10T05:37:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:20 smithi170 ceph-mon[102832]: pgmap v12252: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T05:37:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22671.042946 secs 2023-09-10T05:37:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:20 smithi082 ceph-mon[98694]: pgmap v12252: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T05:37:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22671.042946 secs 2023-09-10T05:37:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:20 smithi098 ceph-mon[103124]: pgmap v12252: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T05:37:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:22 smithi170 ceph-mon[102832]: pgmap v12253: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T05:37:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:22 smithi082 ceph-mon[98694]: pgmap v12253: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T05:37:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:22 smithi098 ceph-mon[103124]: pgmap v12253: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T05:37:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22674.351950 secs 2023-09-10T05:37:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22674.351950 secs 2023-09-10T05:37:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22674.351950 secs 2023-09-10T05:37:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:24 smithi170 ceph-mon[102832]: pgmap v12254: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T05:37:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22675.389492 secs 2023-09-10T05:37:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:24 smithi082 ceph-mon[98694]: pgmap v12254: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T05:37:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22675.389492 secs 2023-09-10T05:37:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:24 smithi098 ceph-mon[103124]: pgmap v12254: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T05:37:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22675.389492 secs 2023-09-10T05:37:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22676.043012 secs 2023-09-10T05:37:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22676.043012 secs 2023-09-10T05:37:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22676.043012 secs 2023-09-10T05:37:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:26 smithi170 ceph-mon[102832]: pgmap v12255: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 304 B/s wr, 10 op/s 2023-09-10T05:37:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:26 smithi082 ceph-mon[98694]: pgmap v12255: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 304 B/s wr, 10 op/s 2023-09-10T05:37:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:26 smithi098 ceph-mon[103124]: pgmap v12255: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 304 B/s wr, 10 op/s 2023-09-10T05:37:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:28 smithi170 ceph-mon[102832]: pgmap v12256: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 853 B/s wr, 13 op/s 2023-09-10T05:37:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:28 smithi082 ceph-mon[98694]: pgmap v12256: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 853 B/s wr, 13 op/s 2023-09-10T05:37:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:28 smithi098 ceph-mon[103124]: pgmap v12256: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 853 B/s wr, 13 op/s 2023-09-10T05:37:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22680.389553 secs 2023-09-10T05:37:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22680.389553 secs 2023-09-10T05:37:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22680.389553 secs 2023-09-10T05:37:30.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22681.043079 secs 2023-09-10T05:37:30.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:30 smithi170 ceph-mon[102832]: pgmap v12257: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-10T05:37:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22681.043079 secs 2023-09-10T05:37:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:30 smithi082 ceph-mon[98694]: pgmap v12257: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-10T05:37:30.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22681.043079 secs 2023-09-10T05:37:30.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:30 smithi098 ceph-mon[103124]: pgmap v12257: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-10T05:37:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:37:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:37:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:37:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:32 smithi170 ceph-mon[102832]: pgmap v12258: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 122 KiB/s wr, 14 op/s 2023-09-10T05:37:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:37:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:37:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:37:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:32 smithi082 ceph-mon[98694]: pgmap v12258: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 122 KiB/s wr, 14 op/s 2023-09-10T05:37:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:37:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:37:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:37:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:32 smithi098 ceph-mon[103124]: pgmap v12258: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 122 KiB/s wr, 14 op/s 2023-09-10T05:37:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:37:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:37:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:37:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:34 smithi170 ceph-mon[102832]: pgmap v12259: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T05:37:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22685.389614 secs 2023-09-10T05:37:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:34 smithi082 ceph-mon[98694]: pgmap v12259: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T05:37:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22685.389614 secs 2023-09-10T05:37:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:34 smithi098 ceph-mon[103124]: pgmap v12259: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T05:37:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22685.389614 secs 2023-09-10T05:37:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22686.043144 secs 2023-09-10T05:37:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22686.043144 secs 2023-09-10T05:37:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22686.043144 secs 2023-09-10T05:37:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:36 smithi170 ceph-mon[102832]: pgmap v12260: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T05:37:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:36 smithi082 ceph-mon[98694]: pgmap v12260: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T05:37:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:36 smithi098 ceph-mon[103124]: pgmap v12260: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T05:37:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:38 smithi170 ceph-mon[102832]: pgmap v12261: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 319 KiB/s wr, 16 op/s 2023-09-10T05:37:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:37:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:37:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:38 smithi082 ceph-mon[98694]: pgmap v12261: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 319 KiB/s wr, 16 op/s 2023-09-10T05:37:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:37:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:37:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:38 smithi098 ceph-mon[103124]: pgmap v12261: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 319 KiB/s wr, 16 op/s 2023-09-10T05:37:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:37:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:37:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22690.389675 secs 2023-09-10T05:37:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22691.043208 secs 2023-09-10T05:37:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22690.389675 secs 2023-09-10T05:37:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22691.043208 secs 2023-09-10T05:37:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22690.389675 secs 2023-09-10T05:37:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22691.043208 secs 2023-09-10T05:37:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:40 smithi170 ceph-mon[102832]: pgmap v12262: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 318 KiB/s wr, 11 op/s 2023-09-10T05:37:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:37:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:37:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:40 smithi082 ceph-mon[98694]: pgmap v12262: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 318 KiB/s wr, 11 op/s 2023-09-10T05:37:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:37:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:37:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:40 smithi098 ceph-mon[103124]: pgmap v12262: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 318 KiB/s wr, 11 op/s 2023-09-10T05:37:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:37:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:37:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:42 smithi170 ceph-mon[102832]: pgmap v12263: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 526 KiB/s wr, 19 op/s 2023-09-10T05:37:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:42 smithi082 ceph-mon[98694]: pgmap v12263: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 526 KiB/s wr, 19 op/s 2023-09-10T05:37:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:42 smithi098 ceph-mon[103124]: pgmap v12263: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 526 KiB/s wr, 19 op/s 2023-09-10T05:37:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:44 smithi170 ceph-mon[102832]: pgmap v12264: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 405 KiB/s wr, 13 op/s 2023-09-10T05:37:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22695.389732 secs 2023-09-10T05:37:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22696.043272 secs 2023-09-10T05:37:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:44 smithi082 ceph-mon[98694]: pgmap v12264: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 405 KiB/s wr, 13 op/s 2023-09-10T05:37:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22695.389732 secs 2023-09-10T05:37:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22696.043272 secs 2023-09-10T05:37:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:44 smithi098 ceph-mon[103124]: pgmap v12264: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 405 KiB/s wr, 13 op/s 2023-09-10T05:37:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22695.389732 secs 2023-09-10T05:37:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22696.043272 secs 2023-09-10T05:37:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:45 smithi082 ceph-mon[98694]: pgmap v12265: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 405 KiB/s wr, 13 op/s 2023-09-10T05:37:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:45 smithi098 ceph-mon[103124]: pgmap v12265: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 405 KiB/s wr, 13 op/s 2023-09-10T05:37:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:45 smithi170 ceph-mon[102832]: pgmap v12265: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 405 KiB/s wr, 13 op/s 2023-09-10T05:37:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:48 smithi170 ceph-mon[102832]: pgmap v12266: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 510 KiB/s wr, 19 op/s 2023-09-10T05:37:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22699.352338 secs 2023-09-10T05:37:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:37:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:48 smithi082 ceph-mon[98694]: pgmap v12266: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 510 KiB/s wr, 19 op/s 2023-09-10T05:37:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22699.352338 secs 2023-09-10T05:37:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:37:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:48 smithi098 ceph-mon[103124]: pgmap v12266: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 510 KiB/s wr, 19 op/s 2023-09-10T05:37:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22699.352338 secs 2023-09-10T05:37:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:37:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22700.389798 secs 2023-09-10T05:37:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:37:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:37:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:37:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22700.389798 secs 2023-09-10T05:37:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:37:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:37:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:37:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22700.389798 secs 2023-09-10T05:37:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:37:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:37:49.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:37:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22701.043338 secs 2023-09-10T05:37:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:50 smithi170 ceph-mon[102832]: pgmap v12267: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 312 KiB/s wr, 13 op/s 2023-09-10T05:37:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:37:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:37:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22701.043338 secs 2023-09-10T05:37:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:50 smithi082 ceph-mon[98694]: pgmap v12267: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 312 KiB/s wr, 13 op/s 2023-09-10T05:37:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:37:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:37:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22701.043338 secs 2023-09-10T05:37:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:50 smithi098 ceph-mon[103124]: pgmap v12267: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 312 KiB/s wr, 13 op/s 2023-09-10T05:37:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:37:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:37:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:37:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:37:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:37:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:37:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:37:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:37:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:37:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:37:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:37:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:52 smithi170 ceph-mon[102832]: pgmap v12268: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 422 KiB/s wr, 19 op/s 2023-09-10T05:37:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:52 smithi082 ceph-mon[98694]: pgmap v12268: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 422 KiB/s wr, 19 op/s 2023-09-10T05:37:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:52 smithi098 ceph-mon[103124]: pgmap v12268: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 422 KiB/s wr, 19 op/s 2023-09-10T05:37:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22704.352398 secs 2023-09-10T05:37:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22704.352398 secs 2023-09-10T05:37:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22704.352398 secs 2023-09-10T05:37:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:54 smithi170 ceph-mon[102832]: pgmap v12269: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 215 KiB/s wr, 12 op/s 2023-09-10T05:37:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22705.389856 secs 2023-09-10T05:37:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22706.043404 secs 2023-09-10T05:37:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:54 smithi082 ceph-mon[98694]: pgmap v12269: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 215 KiB/s wr, 12 op/s 2023-09-10T05:37:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22705.389856 secs 2023-09-10T05:37:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22706.043404 secs 2023-09-10T05:37:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:54 smithi098 ceph-mon[103124]: pgmap v12269: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 215 KiB/s wr, 12 op/s 2023-09-10T05:37:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22705.389856 secs 2023-09-10T05:37:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22706.043404 secs 2023-09-10T05:37:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:55 smithi082 ceph-mon[98694]: pgmap v12270: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 215 KiB/s wr, 12 op/s 2023-09-10T05:37:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:55 smithi098 ceph-mon[103124]: pgmap v12270: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 215 KiB/s wr, 12 op/s 2023-09-10T05:37:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:55 smithi170 ceph-mon[102832]: pgmap v12270: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 215 KiB/s wr, 12 op/s 2023-09-10T05:37:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:58 smithi170 ceph-mon[102832]: pgmap v12271: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 296 KiB/s wr, 17 op/s 2023-09-10T05:37:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22709.352492 secs 2023-09-10T05:37:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:58 smithi082 ceph-mon[98694]: pgmap v12271: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 296 KiB/s wr, 17 op/s 2023-09-10T05:37:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22709.352492 secs 2023-09-10T05:37:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:58 smithi098 ceph-mon[103124]: pgmap v12271: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 296 KiB/s wr, 17 op/s 2023-09-10T05:37:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22709.352492 secs 2023-09-10T05:37:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:37:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22710.389914 secs 2023-09-10T05:37:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:37:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22710.389914 secs 2023-09-10T05:37:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:37:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22710.389914 secs 2023-09-10T05:38:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22711.043472 secs 2023-09-10T05:38:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:00 smithi170 ceph-mon[102832]: pgmap v12272: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 11 op/s 2023-09-10T05:38:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22711.043472 secs 2023-09-10T05:38:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:00 smithi082 ceph-mon[98694]: pgmap v12272: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 11 op/s 2023-09-10T05:38:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22711.043472 secs 2023-09-10T05:38:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:00 smithi098 ceph-mon[103124]: pgmap v12272: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 11 op/s 2023-09-10T05:38:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:02 smithi170 ceph-mon[102832]: pgmap v12273: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 192 KiB/s wr, 15 op/s 2023-09-10T05:38:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:02 smithi082 ceph-mon[98694]: pgmap v12273: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 192 KiB/s wr, 15 op/s 2023-09-10T05:38:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:02 smithi098 ceph-mon[103124]: pgmap v12273: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 192 KiB/s wr, 15 op/s 2023-09-10T05:38:03.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22714.352554 secs 2023-09-10T05:38:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22714.352554 secs 2023-09-10T05:38:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22714.352554 secs 2023-09-10T05:38:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:04 smithi170 ceph-mon[102832]: pgmap v12274: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 9 op/s 2023-09-10T05:38:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22715.389987 secs 2023-09-10T05:38:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:04 smithi082 ceph-mon[98694]: pgmap v12274: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 9 op/s 2023-09-10T05:38:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22715.389987 secs 2023-09-10T05:38:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:04 smithi098 ceph-mon[103124]: pgmap v12274: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 9 op/s 2023-09-10T05:38:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22715.389987 secs 2023-09-10T05:38:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22716.043486 secs 2023-09-10T05:38:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22716.043486 secs 2023-09-10T05:38:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22716.043486 secs 2023-09-10T05:38:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:06 smithi170 ceph-mon[102832]: pgmap v12275: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 9 op/s 2023-09-10T05:38:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:06 smithi082 ceph-mon[98694]: pgmap v12275: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 9 op/s 2023-09-10T05:38:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:06 smithi098 ceph-mon[103124]: pgmap v12275: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 9 op/s 2023-09-10T05:38:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:08 smithi170 ceph-mon[102832]: pgmap v12276: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 184 KiB/s wr, 29 op/s 2023-09-10T05:38:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22719.352669 secs 2023-09-10T05:38:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:08 smithi082 ceph-mon[98694]: pgmap v12276: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 184 KiB/s wr, 29 op/s 2023-09-10T05:38:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22719.352669 secs 2023-09-10T05:38:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:08 smithi098 ceph-mon[103124]: pgmap v12276: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 184 KiB/s wr, 29 op/s 2023-09-10T05:38:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22719.352669 secs 2023-09-10T05:38:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:09 smithi170 ceph-mon[102832]: osdmap e835: 12 total, 12 up, 12 in 2023-09-10T05:38:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22720.390055 secs 2023-09-10T05:38:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:09 smithi082 ceph-mon[98694]: osdmap e835: 12 total, 12 up, 12 in 2023-09-10T05:38:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22720.390055 secs 2023-09-10T05:38:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:09 smithi098 ceph-mon[103124]: osdmap e835: 12 total, 12 up, 12 in 2023-09-10T05:38:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22720.390055 secs 2023-09-10T05:38:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22721.043553 secs 2023-09-10T05:38:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:10 smithi170 ceph-mon[102832]: pgmap v12278: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 123 KiB/s wr, 28 op/s 2023-09-10T05:38:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22721.043553 secs 2023-09-10T05:38:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:10 smithi082 ceph-mon[98694]: pgmap v12278: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 123 KiB/s wr, 28 op/s 2023-09-10T05:38:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22721.043553 secs 2023-09-10T05:38:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:10 smithi098 ceph-mon[103124]: pgmap v12278: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 123 KiB/s wr, 28 op/s 2023-09-10T05:38:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:12 smithi170 ceph-mon[102832]: pgmap v12279: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 123 KiB/s wr, 28 op/s 2023-09-10T05:38:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:12 smithi082 ceph-mon[98694]: pgmap v12279: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 123 KiB/s wr, 28 op/s 2023-09-10T05:38:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:12 smithi098 ceph-mon[103124]: pgmap v12279: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 123 KiB/s wr, 28 op/s 2023-09-10T05:38:13.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22724.352730 secs 2023-09-10T05:38:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22724.352730 secs 2023-09-10T05:38:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22724.352730 secs 2023-09-10T05:38:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:14 smithi170 ceph-mon[102832]: pgmap v12280: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 123 KiB/s wr, 28 op/s 2023-09-10T05:38:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22725.390142 secs 2023-09-10T05:38:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22726.043616 secs 2023-09-10T05:38:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:14 smithi082 ceph-mon[98694]: pgmap v12280: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 123 KiB/s wr, 28 op/s 2023-09-10T05:38:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22725.390142 secs 2023-09-10T05:38:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22726.043616 secs 2023-09-10T05:38:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:14 smithi098 ceph-mon[103124]: pgmap v12280: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 123 KiB/s wr, 28 op/s 2023-09-10T05:38:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22725.390142 secs 2023-09-10T05:38:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22726.043616 secs 2023-09-10T05:38:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:15 smithi082 ceph-mon[98694]: osdmap e836: 12 total, 12 up, 12 in 2023-09-10T05:38:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:15 smithi082 ceph-mon[98694]: pgmap v12282: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:38:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:15 smithi098 ceph-mon[103124]: osdmap e836: 12 total, 12 up, 12 in 2023-09-10T05:38:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:15 smithi098 ceph-mon[103124]: pgmap v12282: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:38:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:15 smithi170 ceph-mon[102832]: osdmap e836: 12 total, 12 up, 12 in 2023-09-10T05:38:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:15 smithi170 ceph-mon[102832]: pgmap v12282: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T05:38:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:18 smithi170 ceph-mon[102832]: pgmap v12283: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2023-09-10T05:38:18.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22729.352773 secs 2023-09-10T05:38:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:18 smithi082 ceph-mon[98694]: pgmap v12283: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2023-09-10T05:38:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22729.352773 secs 2023-09-10T05:38:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:18 smithi098 ceph-mon[103124]: pgmap v12283: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2023-09-10T05:38:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22729.352773 secs 2023-09-10T05:38:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22730.390226 secs 2023-09-10T05:38:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22730.390226 secs 2023-09-10T05:38:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22730.390226 secs 2023-09-10T05:38:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22731.043687 secs 2023-09-10T05:38:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:20 smithi170 ceph-mon[102832]: pgmap v12284: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:38:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22731.043687 secs 2023-09-10T05:38:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:20 smithi082 ceph-mon[98694]: pgmap v12284: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:38:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22731.043687 secs 2023-09-10T05:38:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:20 smithi098 ceph-mon[103124]: pgmap v12284: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:38:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:22 smithi170 ceph-mon[102832]: pgmap v12285: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:38:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:22 smithi082 ceph-mon[98694]: pgmap v12285: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:38:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:22 smithi098 ceph-mon[103124]: pgmap v12285: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:38:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22734.352846 secs 2023-09-10T05:38:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22734.352846 secs 2023-09-10T05:38:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22734.352846 secs 2023-09-10T05:38:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:24 smithi170 ceph-mon[102832]: pgmap v12286: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:38:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22735.390284 secs 2023-09-10T05:38:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:24 smithi082 ceph-mon[98694]: pgmap v12286: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:38:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22735.390284 secs 2023-09-10T05:38:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:24 smithi098 ceph-mon[103124]: pgmap v12286: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T05:38:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22735.390284 secs 2023-09-10T05:38:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22736.043746 secs 2023-09-10T05:38:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22736.043746 secs 2023-09-10T05:38:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22736.043746 secs 2023-09-10T05:38:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:26 smithi170 ceph-mon[102832]: pgmap v12287: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 202 B/s wr, 10 op/s 2023-09-10T05:38:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:26 smithi082 ceph-mon[98694]: pgmap v12287: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 202 B/s wr, 10 op/s 2023-09-10T05:38:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:26 smithi098 ceph-mon[103124]: pgmap v12287: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 202 B/s wr, 10 op/s 2023-09-10T05:38:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:28 smithi170 ceph-mon[102832]: pgmap v12288: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 13 op/s 2023-09-10T05:38:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:28 smithi082 ceph-mon[98694]: pgmap v12288: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 13 op/s 2023-09-10T05:38:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:28 smithi098 ceph-mon[103124]: pgmap v12288: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 13 op/s 2023-09-10T05:38:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22740.390344 secs 2023-09-10T05:38:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22740.390344 secs 2023-09-10T05:38:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22740.390344 secs 2023-09-10T05:38:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22741.043835 secs 2023-09-10T05:38:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:30 smithi170 ceph-mon[102832]: pgmap v12289: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-10T05:38:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22741.043835 secs 2023-09-10T05:38:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:30 smithi082 ceph-mon[98694]: pgmap v12289: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-10T05:38:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22741.043835 secs 2023-09-10T05:38:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:30 smithi098 ceph-mon[103124]: pgmap v12289: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-10T05:38:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:38:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:38:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:38:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:32 smithi170 ceph-mon[102832]: pgmap v12290: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 139 KiB/s wr, 15 op/s 2023-09-10T05:38:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:38:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:38:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:38:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:32 smithi082 ceph-mon[98694]: pgmap v12290: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 139 KiB/s wr, 15 op/s 2023-09-10T05:38:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:38:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:38:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:38:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:32 smithi098 ceph-mon[103124]: pgmap v12290: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 139 KiB/s wr, 15 op/s 2023-09-10T05:38:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:38:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:38:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:38:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:34 smithi170 ceph-mon[102832]: pgmap v12291: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T05:38:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22745.390432 secs 2023-09-10T05:38:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22746.043901 secs 2023-09-10T05:38:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:34 smithi082 ceph-mon[98694]: pgmap v12291: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T05:38:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22745.390432 secs 2023-09-10T05:38:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22746.043901 secs 2023-09-10T05:38:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:34 smithi098 ceph-mon[103124]: pgmap v12291: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T05:38:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22745.390432 secs 2023-09-10T05:38:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22746.043901 secs 2023-09-10T05:38:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:35 smithi082 ceph-mon[98694]: pgmap v12292: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T05:38:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:35 smithi098 ceph-mon[103124]: pgmap v12292: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T05:38:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:35 smithi170 ceph-mon[102832]: pgmap v12292: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T05:38:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:38:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:38:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:37 smithi082 ceph-mon[98694]: pgmap v12293: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 294 KiB/s wr, 16 op/s 2023-09-10T05:38:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:38:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:38:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:37 smithi098 ceph-mon[103124]: pgmap v12293: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 294 KiB/s wr, 16 op/s 2023-09-10T05:38:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:38:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:38:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:37 smithi170 ceph-mon[102832]: pgmap v12293: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 294 KiB/s wr, 16 op/s 2023-09-10T05:38:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22750.390492 secs 2023-09-10T05:38:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22750.390492 secs 2023-09-10T05:38:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22750.390492 secs 2023-09-10T05:38:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22751.043966 secs 2023-09-10T05:38:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:39 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:38:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:39 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:38:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:39 smithi098 ceph-mon[103124]: pgmap v12294: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 293 KiB/s wr, 12 op/s 2023-09-10T05:38:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22751.043966 secs 2023-09-10T05:38:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:39 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:38:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:39 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:38:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:39 smithi170 ceph-mon[102832]: pgmap v12294: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 293 KiB/s wr, 12 op/s 2023-09-10T05:38:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22751.043966 secs 2023-09-10T05:38:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:39 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:38:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:39 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:38:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:39 smithi082 ceph-mon[98694]: pgmap v12294: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 293 KiB/s wr, 12 op/s 2023-09-10T05:38:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:42 smithi170 ceph-mon[102832]: pgmap v12295: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 524 KiB/s wr, 19 op/s 2023-09-10T05:38:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:42 smithi082 ceph-mon[98694]: pgmap v12295: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 524 KiB/s wr, 19 op/s 2023-09-10T05:38:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:42 smithi098 ceph-mon[103124]: pgmap v12295: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 524 KiB/s wr, 19 op/s 2023-09-10T05:38:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:44 smithi170 ceph-mon[102832]: pgmap v12296: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 386 KiB/s wr, 13 op/s 2023-09-10T05:38:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:38:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:38:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22755.390550 secs 2023-09-10T05:38:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:44 smithi082 ceph-mon[98694]: pgmap v12296: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 386 KiB/s wr, 13 op/s 2023-09-10T05:38:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:38:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:38:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22755.390550 secs 2023-09-10T05:38:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:44 smithi098 ceph-mon[103124]: pgmap v12296: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 386 KiB/s wr, 13 op/s 2023-09-10T05:38:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:38:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:38:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22755.390550 secs 2023-09-10T05:38:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22756.044032 secs 2023-09-10T05:38:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22756.044032 secs 2023-09-10T05:38:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22756.044032 secs 2023-09-10T05:38:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:46 smithi170 ceph-mon[102832]: pgmap v12297: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 386 KiB/s wr, 13 op/s 2023-09-10T05:38:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:38:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:38:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:46 smithi082 ceph-mon[98694]: pgmap v12297: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 386 KiB/s wr, 13 op/s 2023-09-10T05:38:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:38:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:38:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:46 smithi098 ceph-mon[103124]: pgmap v12297: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 386 KiB/s wr, 13 op/s 2023-09-10T05:38:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:38:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:38:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:48 smithi170 ceph-mon[102832]: pgmap v12298: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 491 KiB/s wr, 20 op/s 2023-09-10T05:38:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22759.353183 secs 2023-09-10T05:38:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:48 smithi082 ceph-mon[98694]: pgmap v12298: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 491 KiB/s wr, 20 op/s 2023-09-10T05:38:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22759.353183 secs 2023-09-10T05:38:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:48 smithi098 ceph-mon[103124]: pgmap v12298: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 491 KiB/s wr, 20 op/s 2023-09-10T05:38:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22759.353183 secs 2023-09-10T05:38:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22760.390607 secs 2023-09-10T05:38:49.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:38:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22760.390607 secs 2023-09-10T05:38:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:38:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22760.390607 secs 2023-09-10T05:38:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:38:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22761.044101 secs 2023-09-10T05:38:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:50 smithi170 ceph-mon[102832]: pgmap v12299: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 337 KiB/s wr, 14 op/s 2023-09-10T05:38:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:38:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:38:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:38:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22761.044101 secs 2023-09-10T05:38:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:50 smithi082 ceph-mon[98694]: pgmap v12299: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 337 KiB/s wr, 14 op/s 2023-09-10T05:38:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:38:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:38:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:38:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22761.044101 secs 2023-09-10T05:38:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:50 smithi098 ceph-mon[103124]: pgmap v12299: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 337 KiB/s wr, 14 op/s 2023-09-10T05:38:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:38:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:38:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:38:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:38:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:38:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:38:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:38:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:38:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:38:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:38:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:38:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:38:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:52 smithi170 ceph-mon[102832]: pgmap v12300: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 447 KiB/s wr, 19 op/s 2023-09-10T05:38:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:52 smithi082 ceph-mon[98694]: pgmap v12300: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 447 KiB/s wr, 19 op/s 2023-09-10T05:38:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:52 smithi098 ceph-mon[103124]: pgmap v12300: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 447 KiB/s wr, 19 op/s 2023-09-10T05:38:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22764.353267 secs 2023-09-10T05:38:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22764.353267 secs 2023-09-10T05:38:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22764.353267 secs 2023-09-10T05:38:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:54 smithi082 ceph-mon[98694]: pgmap v12301: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 216 KiB/s wr, 12 op/s 2023-09-10T05:38:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22765.390666 secs 2023-09-10T05:38:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22766.044165 secs 2023-09-10T05:38:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:54 smithi098 ceph-mon[103124]: pgmap v12301: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 216 KiB/s wr, 12 op/s 2023-09-10T05:38:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22765.390666 secs 2023-09-10T05:38:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22766.044165 secs 2023-09-10T05:38:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:54 smithi170 ceph-mon[102832]: pgmap v12301: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 216 KiB/s wr, 12 op/s 2023-09-10T05:38:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22765.390666 secs 2023-09-10T05:38:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22766.044165 secs 2023-09-10T05:38:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:55 smithi098 ceph-mon[103124]: pgmap v12302: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 216 KiB/s wr, 12 op/s 2023-09-10T05:38:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:55 smithi170 ceph-mon[102832]: pgmap v12302: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 216 KiB/s wr, 12 op/s 2023-09-10T05:38:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:55 smithi082 ceph-mon[98694]: pgmap v12302: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 216 KiB/s wr, 12 op/s 2023-09-10T05:38:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:58 smithi170 ceph-mon[102832]: pgmap v12303: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 297 KiB/s wr, 16 op/s 2023-09-10T05:38:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22769.353326 secs 2023-09-10T05:38:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:58 smithi082 ceph-mon[98694]: pgmap v12303: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 297 KiB/s wr, 16 op/s 2023-09-10T05:38:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22769.353326 secs 2023-09-10T05:38:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:58 smithi098 ceph-mon[103124]: pgmap v12303: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 297 KiB/s wr, 16 op/s 2023-09-10T05:38:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22769.353326 secs 2023-09-10T05:38:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:38:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22770.390752 secs 2023-09-10T05:38:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:38:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22770.390752 secs 2023-09-10T05:38:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:38:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22770.390752 secs 2023-09-10T05:39:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22771.044230 secs 2023-09-10T05:39:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:00 smithi170 ceph-mon[102832]: pgmap v12304: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 192 KiB/s wr, 10 op/s 2023-09-10T05:39:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22771.044230 secs 2023-09-10T05:39:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:00 smithi082 ceph-mon[98694]: pgmap v12304: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 192 KiB/s wr, 10 op/s 2023-09-10T05:39:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22771.044230 secs 2023-09-10T05:39:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:00 smithi098 ceph-mon[103124]: pgmap v12304: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 192 KiB/s wr, 10 op/s 2023-09-10T05:39:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:02 smithi170 ceph-mon[102832]: pgmap v12305: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 237 KiB/s wr, 29 op/s 2023-09-10T05:39:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:02 smithi082 ceph-mon[98694]: pgmap v12305: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 237 KiB/s wr, 29 op/s 2023-09-10T05:39:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:02 smithi098 ceph-mon[103124]: pgmap v12305: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 237 KiB/s wr, 29 op/s 2023-09-10T05:39:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22774.353445 secs 2023-09-10T05:39:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22774.353445 secs 2023-09-10T05:39:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22774.353445 secs 2023-09-10T05:39:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:04 smithi170 ceph-mon[102832]: pgmap v12306: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 126 KiB/s wr, 23 op/s 2023-09-10T05:39:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22775.390812 secs 2023-09-10T05:39:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:04 smithi082 ceph-mon[98694]: pgmap v12306: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 126 KiB/s wr, 23 op/s 2023-09-10T05:39:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22775.390812 secs 2023-09-10T05:39:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:04 smithi098 ceph-mon[103124]: pgmap v12306: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 126 KiB/s wr, 23 op/s 2023-09-10T05:39:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22775.390812 secs 2023-09-10T05:39:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22776.044293 secs 2023-09-10T05:39:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22776.044293 secs 2023-09-10T05:39:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22776.044293 secs 2023-09-10T05:39:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:06 smithi170 ceph-mon[102832]: pgmap v12307: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 126 KiB/s wr, 23 op/s 2023-09-10T05:39:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:06 smithi082 ceph-mon[98694]: pgmap v12307: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 126 KiB/s wr, 23 op/s 2023-09-10T05:39:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:06 smithi098 ceph-mon[103124]: pgmap v12307: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 126 KiB/s wr, 23 op/s 2023-09-10T05:39:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:08 smithi170 ceph-mon[102832]: pgmap v12308: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 185 KiB/s wr, 28 op/s 2023-09-10T05:39:08.434 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22779.353547 secs 2023-09-10T05:39:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:08 smithi082 ceph-mon[98694]: pgmap v12308: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 185 KiB/s wr, 28 op/s 2023-09-10T05:39:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22779.353547 secs 2023-09-10T05:39:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:08 smithi098 ceph-mon[103124]: pgmap v12308: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 185 KiB/s wr, 28 op/s 2023-09-10T05:39:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22779.353547 secs 2023-09-10T05:39:09.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22780.390874 secs 2023-09-10T05:39:09.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:09 smithi170 ceph-mon[102832]: osdmap e837: 12 total, 12 up, 12 in 2023-09-10T05:39:09.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22781.044358 secs 2023-09-10T05:39:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22780.390874 secs 2023-09-10T05:39:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:09 smithi082 ceph-mon[98694]: osdmap e837: 12 total, 12 up, 12 in 2023-09-10T05:39:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22781.044358 secs 2023-09-10T05:39:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22780.390874 secs 2023-09-10T05:39:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:09 smithi098 ceph-mon[103124]: osdmap e837: 12 total, 12 up, 12 in 2023-09-10T05:39:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22781.044358 secs 2023-09-10T05:39:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:10 smithi170 ceph-mon[102832]: pgmap v12310: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 124 KiB/s wr, 27 op/s 2023-09-10T05:39:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:10 smithi082 ceph-mon[98694]: pgmap v12310: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 124 KiB/s wr, 27 op/s 2023-09-10T05:39:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:10 smithi098 ceph-mon[103124]: pgmap v12310: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 124 KiB/s wr, 27 op/s 2023-09-10T05:39:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:12 smithi170 ceph-mon[102832]: pgmap v12311: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T05:39:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:12 smithi082 ceph-mon[98694]: pgmap v12311: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T05:39:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:12 smithi098 ceph-mon[103124]: pgmap v12311: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T05:39:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22784.353624 secs 2023-09-10T05:39:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22784.353624 secs 2023-09-10T05:39:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22784.353624 secs 2023-09-10T05:39:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:14 smithi170 ceph-mon[102832]: pgmap v12312: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T05:39:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22785.390934 secs 2023-09-10T05:39:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22786.044426 secs 2023-09-10T05:39:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:14 smithi082 ceph-mon[98694]: pgmap v12312: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T05:39:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22785.390934 secs 2023-09-10T05:39:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22786.044426 secs 2023-09-10T05:39:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:14 smithi098 ceph-mon[103124]: pgmap v12312: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 70 KiB/s wr, 10 op/s 2023-09-10T05:39:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22785.390934 secs 2023-09-10T05:39:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22786.044426 secs 2023-09-10T05:39:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:15 smithi098 ceph-mon[103124]: osdmap e838: 12 total, 12 up, 12 in 2023-09-10T05:39:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:15 smithi098 ceph-mon[103124]: pgmap v12314: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-10T05:39:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:15 smithi170 ceph-mon[102832]: osdmap e838: 12 total, 12 up, 12 in 2023-09-10T05:39:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:15 smithi170 ceph-mon[102832]: pgmap v12314: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-10T05:39:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:15 smithi082 ceph-mon[98694]: osdmap e838: 12 total, 12 up, 12 in 2023-09-10T05:39:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:15 smithi082 ceph-mon[98694]: pgmap v12314: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-10T05:39:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:18 smithi170 ceph-mon[102832]: pgmap v12315: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 803 B/s wr, 12 op/s 2023-09-10T05:39:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22789.353682 secs 2023-09-10T05:39:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:18 smithi082 ceph-mon[98694]: pgmap v12315: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 803 B/s wr, 12 op/s 2023-09-10T05:39:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22789.353682 secs 2023-09-10T05:39:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:18 smithi098 ceph-mon[103124]: pgmap v12315: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 803 B/s wr, 12 op/s 2023-09-10T05:39:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22789.353682 secs 2023-09-10T05:39:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22790.390995 secs 2023-09-10T05:39:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22790.390995 secs 2023-09-10T05:39:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22790.390995 secs 2023-09-10T05:39:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22791.044486 secs 2023-09-10T05:39:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:20 smithi170 ceph-mon[102832]: pgmap v12316: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 716 B/s wr, 11 op/s 2023-09-10T05:39:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22791.044486 secs 2023-09-10T05:39:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:20 smithi082 ceph-mon[98694]: pgmap v12316: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 716 B/s wr, 11 op/s 2023-09-10T05:39:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22791.044486 secs 2023-09-10T05:39:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:20 smithi098 ceph-mon[103124]: pgmap v12316: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 716 B/s wr, 11 op/s 2023-09-10T05:39:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:22 smithi170 ceph-mon[102832]: pgmap v12317: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T05:39:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:22 smithi082 ceph-mon[98694]: pgmap v12317: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T05:39:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:22 smithi098 ceph-mon[103124]: pgmap v12317: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T05:39:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22794.353743 secs 2023-09-10T05:39:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22794.353743 secs 2023-09-10T05:39:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22794.353743 secs 2023-09-10T05:39:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:24 smithi170 ceph-mon[102832]: pgmap v12318: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T05:39:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22795.391073 secs 2023-09-10T05:39:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22796.044550 secs 2023-09-10T05:39:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:24 smithi082 ceph-mon[98694]: pgmap v12318: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T05:39:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22795.391073 secs 2023-09-10T05:39:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22796.044550 secs 2023-09-10T05:39:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:24 smithi098 ceph-mon[103124]: pgmap v12318: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T05:39:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22795.391073 secs 2023-09-10T05:39:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22796.044550 secs 2023-09-10T05:39:26.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:25 smithi098 ceph-mon[103124]: pgmap v12319: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T05:39:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:25 smithi170 ceph-mon[102832]: pgmap v12319: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T05:39:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:25 smithi082 ceph-mon[98694]: pgmap v12319: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T05:39:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:28 smithi170 ceph-mon[102832]: pgmap v12320: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 78 KiB/s wr, 15 op/s 2023-09-10T05:39:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:28 smithi082 ceph-mon[98694]: pgmap v12320: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 78 KiB/s wr, 15 op/s 2023-09-10T05:39:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:28 smithi098 ceph-mon[103124]: pgmap v12320: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 78 KiB/s wr, 15 op/s 2023-09-10T05:39:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22800.391132 secs 2023-09-10T05:39:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22800.391132 secs 2023-09-10T05:39:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22800.391132 secs 2023-09-10T05:39:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22801.044618 secs 2023-09-10T05:39:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:30 smithi170 ceph-mon[102832]: pgmap v12321: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 10 op/s 2023-09-10T05:39:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22801.044618 secs 2023-09-10T05:39:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:30 smithi082 ceph-mon[98694]: pgmap v12321: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 10 op/s 2023-09-10T05:39:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22801.044618 secs 2023-09-10T05:39:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:30 smithi098 ceph-mon[103124]: pgmap v12321: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 10 op/s 2023-09-10T05:39:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:39:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:39:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:39:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:32 smithi170 ceph-mon[102832]: pgmap v12322: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 15 op/s 2023-09-10T05:39:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:39:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:39:32.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:39:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:32 smithi082 ceph-mon[98694]: pgmap v12322: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 15 op/s 2023-09-10T05:39:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:39:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:39:32.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:39:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:32 smithi098 ceph-mon[103124]: pgmap v12322: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 15 op/s 2023-09-10T05:39:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:39:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:39:32.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:39:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22804.353832 secs 2023-09-10T05:39:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22804.353832 secs 2023-09-10T05:39:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22804.353832 secs 2023-09-10T05:39:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:34 smithi170 ceph-mon[102832]: pgmap v12323: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 10 op/s 2023-09-10T05:39:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:39:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:39:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22805.391193 secs 2023-09-10T05:39:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22806.044683 secs 2023-09-10T05:39:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:34 smithi082 ceph-mon[98694]: pgmap v12323: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 10 op/s 2023-09-10T05:39:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:39:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:39:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22805.391193 secs 2023-09-10T05:39:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22806.044683 secs 2023-09-10T05:39:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:34 smithi098 ceph-mon[103124]: pgmap v12323: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 10 op/s 2023-09-10T05:39:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:39:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:39:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22805.391193 secs 2023-09-10T05:39:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22806.044683 secs 2023-09-10T05:39:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:36 smithi170 ceph-mon[102832]: pgmap v12324: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 10 op/s 2023-09-10T05:39:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:39:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:39:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:36 smithi082 ceph-mon[98694]: pgmap v12324: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 10 op/s 2023-09-10T05:39:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:39:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:39:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:36 smithi098 ceph-mon[103124]: pgmap v12324: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 10 op/s 2023-09-10T05:39:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:39:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:39:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:38 smithi170 ceph-mon[102832]: pgmap v12325: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 236 KiB/s wr, 15 op/s 2023-09-10T05:39:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:38 smithi082 ceph-mon[98694]: pgmap v12325: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 236 KiB/s wr, 15 op/s 2023-09-10T05:39:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:38 smithi098 ceph-mon[103124]: pgmap v12325: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 236 KiB/s wr, 15 op/s 2023-09-10T05:39:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22810.391250 secs 2023-09-10T05:39:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22811.044748 secs 2023-09-10T05:39:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:39:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22810.391250 secs 2023-09-10T05:39:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22811.044748 secs 2023-09-10T05:39:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:39:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22810.391250 secs 2023-09-10T05:39:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22811.044748 secs 2023-09-10T05:39:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:39:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:40 smithi170 ceph-mon[102832]: pgmap v12326: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 236 KiB/s wr, 10 op/s 2023-09-10T05:39:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:39:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:39:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:39:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:40 smithi082 ceph-mon[98694]: pgmap v12326: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 236 KiB/s wr, 10 op/s 2023-09-10T05:39:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:39:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:39:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:39:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:40 smithi098 ceph-mon[103124]: pgmap v12326: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 236 KiB/s wr, 10 op/s 2023-09-10T05:39:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:39:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:39:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:39:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:42 smithi170 ceph-mon[102832]: pgmap v12327: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 476 KiB/s wr, 17 op/s 2023-09-10T05:39:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:42 smithi082 ceph-mon[98694]: pgmap v12327: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 476 KiB/s wr, 17 op/s 2023-09-10T05:39:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:42 smithi098 ceph-mon[103124]: pgmap v12327: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 476 KiB/s wr, 17 op/s 2023-09-10T05:39:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:44 smithi170 ceph-mon[102832]: pgmap v12328: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 398 KiB/s wr, 11 op/s 2023-09-10T05:39:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22815.391311 secs 2023-09-10T05:39:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22816.044818 secs 2023-09-10T05:39:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:44 smithi082 ceph-mon[98694]: pgmap v12328: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 398 KiB/s wr, 11 op/s 2023-09-10T05:39:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22815.391311 secs 2023-09-10T05:39:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22816.044818 secs 2023-09-10T05:39:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:44 smithi098 ceph-mon[103124]: pgmap v12328: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 398 KiB/s wr, 11 op/s 2023-09-10T05:39:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22815.391311 secs 2023-09-10T05:39:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22816.044818 secs 2023-09-10T05:39:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:46 smithi170 ceph-mon[102832]: pgmap v12329: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 398 KiB/s wr, 11 op/s 2023-09-10T05:39:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:39:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:39:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:46 smithi082 ceph-mon[98694]: pgmap v12329: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 398 KiB/s wr, 11 op/s 2023-09-10T05:39:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:39:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:39:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:46 smithi098 ceph-mon[103124]: pgmap v12329: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 398 KiB/s wr, 11 op/s 2023-09-10T05:39:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:39:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:39:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:48 smithi170 ceph-mon[102832]: pgmap v12330: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 659 KiB/s wr, 18 op/s 2023-09-10T05:39:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:48 smithi082 ceph-mon[98694]: pgmap v12330: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 659 KiB/s wr, 18 op/s 2023-09-10T05:39:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:48 smithi098 ceph-mon[103124]: pgmap v12330: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 659 KiB/s wr, 18 op/s 2023-09-10T05:39:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22820.391395 secs 2023-09-10T05:39:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22821.044881 secs 2023-09-10T05:39:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22820.391395 secs 2023-09-10T05:39:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22821.044881 secs 2023-09-10T05:39:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22820.391395 secs 2023-09-10T05:39:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22821.044881 secs 2023-09-10T05:39:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:50 smithi170 ceph-mon[102832]: pgmap v12331: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 501 KiB/s wr, 13 op/s 2023-09-10T05:39:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:39:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:39:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:39:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:39:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:39:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:39:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:50 smithi082 ceph-mon[98694]: pgmap v12331: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 501 KiB/s wr, 13 op/s 2023-09-10T05:39:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:39:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:39:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:39:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:39:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:39:50.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:39:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:50 smithi098 ceph-mon[103124]: pgmap v12331: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 501 KiB/s wr, 13 op/s 2023-09-10T05:39:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:39:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:39:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:39:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:39:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:39:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:39:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:52 smithi082 ceph-mon[98694]: pgmap v12332: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 771 KiB/s wr, 20 op/s 2023-09-10T05:39:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:39:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:39:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:52 smithi098 ceph-mon[103124]: pgmap v12332: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 771 KiB/s wr, 20 op/s 2023-09-10T05:39:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:39:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:39:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:52 smithi170 ceph-mon[102832]: pgmap v12332: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 771 KiB/s wr, 20 op/s 2023-09-10T05:39:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:39:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:39:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22824.354118 secs 2023-09-10T05:39:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22824.354118 secs 2023-09-10T05:39:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22824.354118 secs 2023-09-10T05:39:54.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:54 smithi082 ceph-mon[98694]: pgmap v12333: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 531 KiB/s wr, 13 op/s 2023-09-10T05:39:54.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22825.391453 secs 2023-09-10T05:39:54.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:39:54.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:39:54.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22826.044948 secs 2023-09-10T05:39:54.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:54 smithi098 ceph-mon[103124]: pgmap v12333: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 531 KiB/s wr, 13 op/s 2023-09-10T05:39:54.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22825.391453 secs 2023-09-10T05:39:54.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:39:54.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:39:54.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22826.044948 secs 2023-09-10T05:39:54.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:54 smithi170 ceph-mon[102832]: pgmap v12333: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 531 KiB/s wr, 13 op/s 2023-09-10T05:39:54.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22825.391453 secs 2023-09-10T05:39:54.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:39:54.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:39:54.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22826.044948 secs 2023-09-10T05:39:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:55 smithi098 ceph-mon[103124]: pgmap v12334: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 531 KiB/s wr, 13 op/s 2023-09-10T05:39:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:39:56.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:39:56.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:39:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:55 smithi170 ceph-mon[102832]: pgmap v12334: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 531 KiB/s wr, 13 op/s 2023-09-10T05:39:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:39:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:39:56.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:39:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:55 smithi082 ceph-mon[98694]: pgmap v12334: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 531 KiB/s wr, 13 op/s 2023-09-10T05:39:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:39:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:39:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:39:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:58 smithi170 ceph-mon[102832]: pgmap v12335: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 667 KiB/s wr, 19 op/s 2023-09-10T05:39:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22829.354177 secs 2023-09-10T05:39:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:58 smithi082 ceph-mon[98694]: pgmap v12335: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 667 KiB/s wr, 19 op/s 2023-09-10T05:39:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22829.354177 secs 2023-09-10T05:39:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:58 smithi098 ceph-mon[103124]: pgmap v12335: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 667 KiB/s wr, 19 op/s 2023-09-10T05:39:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22829.354177 secs 2023-09-10T05:39:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:39:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22830.391541 secs 2023-09-10T05:39:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:39:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22830.391541 secs 2023-09-10T05:39:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:39:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22830.391541 secs 2023-09-10T05:40:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22831.045013 secs 2023-09-10T05:40:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:00 smithi170 ceph-mon[102832]: pgmap v12336: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 407 KiB/s wr, 12 op/s 2023-09-10T05:40:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T05:40:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22831.045013 secs 2023-09-10T05:40:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:00 smithi082 ceph-mon[98694]: pgmap v12336: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 407 KiB/s wr, 12 op/s 2023-09-10T05:40:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T05:40:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22831.045013 secs 2023-09-10T05:40:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:00 smithi098 ceph-mon[103124]: pgmap v12336: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 407 KiB/s wr, 12 op/s 2023-09-10T05:40:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T05:40:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:02 smithi170 ceph-mon[102832]: pgmap v12337: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 581 KiB/s wr, 30 op/s 2023-09-10T05:40:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:02 smithi082 ceph-mon[98694]: pgmap v12337: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 581 KiB/s wr, 30 op/s 2023-09-10T05:40:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:02 smithi098 ceph-mon[103124]: pgmap v12337: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 581 KiB/s wr, 30 op/s 2023-09-10T05:40:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22834.354289 secs 2023-09-10T05:40:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22834.354289 secs 2023-09-10T05:40:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22834.354289 secs 2023-09-10T05:40:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:04 smithi170 ceph-mon[102832]: pgmap v12338: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 311 KiB/s wr, 24 op/s 2023-09-10T05:40:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22835.391599 secs 2023-09-10T05:40:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22836.045077 secs 2023-09-10T05:40:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:04 smithi082 ceph-mon[98694]: pgmap v12338: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 311 KiB/s wr, 24 op/s 2023-09-10T05:40:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22835.391599 secs 2023-09-10T05:40:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22836.045077 secs 2023-09-10T05:40:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:04 smithi098 ceph-mon[103124]: pgmap v12338: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 311 KiB/s wr, 24 op/s 2023-09-10T05:40:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22835.391599 secs 2023-09-10T05:40:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22836.045077 secs 2023-09-10T05:40:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:05 smithi098 ceph-mon[103124]: pgmap v12339: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 311 KiB/s wr, 24 op/s 2023-09-10T05:40:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:05 smithi170 ceph-mon[102832]: pgmap v12339: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 311 KiB/s wr, 24 op/s 2023-09-10T05:40:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:05 smithi082 ceph-mon[98694]: pgmap v12339: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 311 KiB/s wr, 24 op/s 2023-09-10T05:40:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:08 smithi170 ceph-mon[102832]: pgmap v12340: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 351 KiB/s wr, 31 op/s 2023-09-10T05:40:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22839.354348 secs 2023-09-10T05:40:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:08 smithi082 ceph-mon[98694]: pgmap v12340: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 351 KiB/s wr, 31 op/s 2023-09-10T05:40:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22839.354348 secs 2023-09-10T05:40:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:08 smithi098 ceph-mon[103124]: pgmap v12340: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 351 KiB/s wr, 31 op/s 2023-09-10T05:40:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22839.354348 secs 2023-09-10T05:40:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:09 smithi170 ceph-mon[102832]: osdmap e839: 12 total, 12 up, 12 in 2023-09-10T05:40:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22840.391660 secs 2023-09-10T05:40:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:09 smithi082 ceph-mon[98694]: osdmap e839: 12 total, 12 up, 12 in 2023-09-10T05:40:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22840.391660 secs 2023-09-10T05:40:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:09 smithi098 ceph-mon[103124]: osdmap e839: 12 total, 12 up, 12 in 2023-09-10T05:40:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22840.391660 secs 2023-09-10T05:40:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22841.045143 secs 2023-09-10T05:40:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:10 smithi170 ceph-mon[102832]: pgmap v12342: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 257 KiB/s wr, 30 op/s 2023-09-10T05:40:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22841.045143 secs 2023-09-10T05:40:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:10 smithi082 ceph-mon[98694]: pgmap v12342: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 257 KiB/s wr, 30 op/s 2023-09-10T05:40:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22841.045143 secs 2023-09-10T05:40:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:10 smithi098 ceph-mon[103124]: pgmap v12342: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 257 KiB/s wr, 30 op/s 2023-09-10T05:40:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:12 smithi170 ceph-mon[102832]: pgmap v12343: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 48 KiB/s wr, 13 op/s 2023-09-10T05:40:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:12 smithi082 ceph-mon[98694]: pgmap v12343: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 48 KiB/s wr, 13 op/s 2023-09-10T05:40:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:12 smithi098 ceph-mon[103124]: pgmap v12343: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 48 KiB/s wr, 13 op/s 2023-09-10T05:40:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22844.354421 secs 2023-09-10T05:40:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22844.354421 secs 2023-09-10T05:40:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22844.354421 secs 2023-09-10T05:40:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:14 smithi170 ceph-mon[102832]: pgmap v12344: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 48 KiB/s wr, 13 op/s 2023-09-10T05:40:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22845.391719 secs 2023-09-10T05:40:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22846.045206 secs 2023-09-10T05:40:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:14 smithi082 ceph-mon[98694]: pgmap v12344: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 48 KiB/s wr, 13 op/s 2023-09-10T05:40:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22845.391719 secs 2023-09-10T05:40:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22846.045206 secs 2023-09-10T05:40:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:14 smithi098 ceph-mon[103124]: pgmap v12344: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 48 KiB/s wr, 13 op/s 2023-09-10T05:40:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22845.391719 secs 2023-09-10T05:40:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22846.045206 secs 2023-09-10T05:40:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:15 smithi098 ceph-mon[103124]: osdmap e840: 12 total, 12 up, 12 in 2023-09-10T05:40:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:15 smithi098 ceph-mon[103124]: pgmap v12346: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 383 B/s wr, 6 op/s 2023-09-10T05:40:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:15 smithi170 ceph-mon[102832]: osdmap e840: 12 total, 12 up, 12 in 2023-09-10T05:40:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:15 smithi170 ceph-mon[102832]: pgmap v12346: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 383 B/s wr, 6 op/s 2023-09-10T05:40:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:15 smithi082 ceph-mon[98694]: osdmap e840: 12 total, 12 up, 12 in 2023-09-10T05:40:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:15 smithi082 ceph-mon[98694]: pgmap v12346: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 383 B/s wr, 6 op/s 2023-09-10T05:40:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:18 smithi170 ceph-mon[102832]: pgmap v12347: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 13 op/s 2023-09-10T05:40:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22849.354479 secs 2023-09-10T05:40:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:18 smithi082 ceph-mon[98694]: pgmap v12347: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 13 op/s 2023-09-10T05:40:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22849.354479 secs 2023-09-10T05:40:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:18 smithi098 ceph-mon[103124]: pgmap v12347: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 13 op/s 2023-09-10T05:40:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22849.354479 secs 2023-09-10T05:40:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22850.391778 secs 2023-09-10T05:40:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22850.391778 secs 2023-09-10T05:40:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22850.391778 secs 2023-09-10T05:40:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22851.045278 secs 2023-09-10T05:40:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:20 smithi170 ceph-mon[102832]: pgmap v12348: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:40:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22851.045278 secs 2023-09-10T05:40:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:20 smithi082 ceph-mon[98694]: pgmap v12348: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:40:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22851.045278 secs 2023-09-10T05:40:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:20 smithi098 ceph-mon[103124]: pgmap v12348: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:40:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:22 smithi170 ceph-mon[102832]: pgmap v12349: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:40:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:22 smithi082 ceph-mon[98694]: pgmap v12349: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:40:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:22 smithi098 ceph-mon[103124]: pgmap v12349: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:40:23.434 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22854.354540 secs 2023-09-10T05:40:23.467 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22854.354540 secs 2023-09-10T05:40:23.509 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22854.354540 secs 2023-09-10T05:40:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:24 smithi170 ceph-mon[102832]: pgmap v12350: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:40:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22855.391838 secs 2023-09-10T05:40:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22856.045342 secs 2023-09-10T05:40:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:24 smithi082 ceph-mon[98694]: pgmap v12350: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:40:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22855.391838 secs 2023-09-10T05:40:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22856.045342 secs 2023-09-10T05:40:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:24 smithi098 ceph-mon[103124]: pgmap v12350: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:40:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22855.391838 secs 2023-09-10T05:40:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22856.045342 secs 2023-09-10T05:40:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:25 smithi098 ceph-mon[103124]: pgmap v12351: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:40:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:25 smithi170 ceph-mon[102832]: pgmap v12351: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:40:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:25 smithi082 ceph-mon[98694]: pgmap v12351: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:40:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:28 smithi170 ceph-mon[102832]: pgmap v12352: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 114 KiB/s wr, 14 op/s 2023-09-10T05:40:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:28 smithi082 ceph-mon[98694]: pgmap v12352: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 114 KiB/s wr, 14 op/s 2023-09-10T05:40:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:28 smithi098 ceph-mon[103124]: pgmap v12352: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 114 KiB/s wr, 14 op/s 2023-09-10T05:40:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22860.391900 secs 2023-09-10T05:40:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22860.391900 secs 2023-09-10T05:40:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22860.391900 secs 2023-09-10T05:40:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22861.045408 secs 2023-09-10T05:40:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:30 smithi170 ceph-mon[102832]: pgmap v12353: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 96 KiB/s wr, 9 op/s 2023-09-10T05:40:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22861.045408 secs 2023-09-10T05:40:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:30 smithi082 ceph-mon[98694]: pgmap v12353: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 96 KiB/s wr, 9 op/s 2023-09-10T05:40:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22861.045408 secs 2023-09-10T05:40:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:30 smithi098 ceph-mon[103124]: pgmap v12353: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 96 KiB/s wr, 9 op/s 2023-09-10T05:40:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:40:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:40:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:40:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:32 smithi170 ceph-mon[102832]: pgmap v12354: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 146 KiB/s wr, 14 op/s 2023-09-10T05:40:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:40:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:40:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:40:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:32 smithi082 ceph-mon[98694]: pgmap v12354: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 146 KiB/s wr, 14 op/s 2023-09-10T05:40:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:40:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:40:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:40:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:32 smithi098 ceph-mon[103124]: pgmap v12354: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 146 KiB/s wr, 14 op/s 2023-09-10T05:40:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:40:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:40:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:40:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22864.354661 secs 2023-09-10T05:40:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22864.354661 secs 2023-09-10T05:40:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22864.354661 secs 2023-09-10T05:40:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:34 smithi170 ceph-mon[102832]: pgmap v12355: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 146 KiB/s wr, 10 op/s 2023-09-10T05:40:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22865.391981 secs 2023-09-10T05:40:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22866.045480 secs 2023-09-10T05:40:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:34 smithi082 ceph-mon[98694]: pgmap v12355: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 146 KiB/s wr, 10 op/s 2023-09-10T05:40:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22865.391981 secs 2023-09-10T05:40:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22866.045480 secs 2023-09-10T05:40:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:34 smithi098 ceph-mon[103124]: pgmap v12355: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 146 KiB/s wr, 10 op/s 2023-09-10T05:40:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22865.391981 secs 2023-09-10T05:40:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22866.045480 secs 2023-09-10T05:40:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:36 smithi170 ceph-mon[102832]: pgmap v12356: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 146 KiB/s wr, 10 op/s 2023-09-10T05:40:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:40:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:40:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:36 smithi082 ceph-mon[98694]: pgmap v12356: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 146 KiB/s wr, 10 op/s 2023-09-10T05:40:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:40:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:40:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:36 smithi098 ceph-mon[103124]: pgmap v12356: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 146 KiB/s wr, 10 op/s 2023-09-10T05:40:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:40:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:40:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:38 smithi170 ceph-mon[102832]: pgmap v12357: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 304 KiB/s wr, 15 op/s 2023-09-10T05:40:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:40:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:40:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22869.354723 secs 2023-09-10T05:40:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:38 smithi082 ceph-mon[98694]: pgmap v12357: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 304 KiB/s wr, 15 op/s 2023-09-10T05:40:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:40:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:40:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22869.354723 secs 2023-09-10T05:40:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:38 smithi098 ceph-mon[103124]: pgmap v12357: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 304 KiB/s wr, 15 op/s 2023-09-10T05:40:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:40:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:40:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22869.354723 secs 2023-09-10T05:40:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22870.392040 secs 2023-09-10T05:40:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22871.045545 secs 2023-09-10T05:40:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22870.392040 secs 2023-09-10T05:40:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22871.045545 secs 2023-09-10T05:40:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22870.392040 secs 2023-09-10T05:40:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22871.045545 secs 2023-09-10T05:40:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:40 smithi170 ceph-mon[102832]: pgmap v12358: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 209 KiB/s wr, 10 op/s 2023-09-10T05:40:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:40:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:40 smithi082 ceph-mon[98694]: pgmap v12358: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 209 KiB/s wr, 10 op/s 2023-09-10T05:40:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:40:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:40 smithi098 ceph-mon[103124]: pgmap v12358: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 209 KiB/s wr, 10 op/s 2023-09-10T05:40:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:40:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:42 smithi170 ceph-mon[102832]: pgmap v12359: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 239 KiB/s wr, 16 op/s 2023-09-10T05:40:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:42 smithi082 ceph-mon[98694]: pgmap v12359: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 239 KiB/s wr, 16 op/s 2023-09-10T05:40:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:42 smithi098 ceph-mon[103124]: pgmap v12359: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 239 KiB/s wr, 16 op/s 2023-09-10T05:40:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:40:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:40:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:40:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:44 smithi170 ceph-mon[102832]: pgmap v12360: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T05:40:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22875.392097 secs 2023-09-10T05:40:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:40:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:40:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22876.045608 secs 2023-09-10T05:40:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:44 smithi082 ceph-mon[98694]: pgmap v12360: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T05:40:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22875.392097 secs 2023-09-10T05:40:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:40:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:40:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22876.045608 secs 2023-09-10T05:40:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:44 smithi098 ceph-mon[103124]: pgmap v12360: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T05:40:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22875.392097 secs 2023-09-10T05:40:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:40:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:40:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22876.045608 secs 2023-09-10T05:40:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:45 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:40:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:45 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:40:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:45 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:40:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:46 smithi082 ceph-mon[98694]: pgmap v12361: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T05:40:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:40:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:40:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:46 smithi098 ceph-mon[103124]: pgmap v12361: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T05:40:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:40:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:40:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:46 smithi170 ceph-mon[102832]: pgmap v12361: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T05:40:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:40:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:40:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:48 smithi082 ceph-mon[98694]: pgmap v12362: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 451 KiB/s wr, 18 op/s 2023-09-10T05:40:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:48 smithi098 ceph-mon[103124]: pgmap v12362: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 451 KiB/s wr, 18 op/s 2023-09-10T05:40:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:48 smithi170 ceph-mon[102832]: pgmap v12362: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 451 KiB/s wr, 18 op/s 2023-09-10T05:40:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22880.392153 secs 2023-09-10T05:40:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22881.045676 secs 2023-09-10T05:40:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22880.392153 secs 2023-09-10T05:40:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22881.045676 secs 2023-09-10T05:40:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22880.392153 secs 2023-09-10T05:40:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22881.045676 secs 2023-09-10T05:40:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:50 smithi082 ceph-mon[98694]: pgmap v12363: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 13 op/s 2023-09-10T05:40:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:40:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:50 smithi098 ceph-mon[103124]: pgmap v12363: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 13 op/s 2023-09-10T05:40:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:40:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:50 smithi170 ceph-mon[102832]: pgmap v12363: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 13 op/s 2023-09-10T05:40:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:40:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:40:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:40:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:40:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:40:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:40:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:40:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:40:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:40:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:40:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:52 smithi082 ceph-mon[98694]: pgmap v12364: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 317 KiB/s wr, 19 op/s 2023-09-10T05:40:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:52 smithi098 ceph-mon[103124]: pgmap v12364: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 317 KiB/s wr, 19 op/s 2023-09-10T05:40:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:52 smithi170 ceph-mon[102832]: pgmap v12364: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 317 KiB/s wr, 19 op/s 2023-09-10T05:40:54.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:54 smithi082 ceph-mon[98694]: pgmap v12365: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 287 KiB/s wr, 13 op/s 2023-09-10T05:40:54.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22885.392214 secs 2023-09-10T05:40:54.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22886.045740 secs 2023-09-10T05:40:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:54 smithi098 ceph-mon[103124]: pgmap v12365: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 287 KiB/s wr, 13 op/s 2023-09-10T05:40:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22885.392214 secs 2023-09-10T05:40:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22886.045740 secs 2023-09-10T05:40:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:54 smithi170 ceph-mon[102832]: pgmap v12365: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 287 KiB/s wr, 13 op/s 2023-09-10T05:40:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22885.392214 secs 2023-09-10T05:40:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22886.045740 secs 2023-09-10T05:40:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:55 smithi098 ceph-mon[103124]: pgmap v12366: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 287 KiB/s wr, 13 op/s 2023-09-10T05:40:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:55 smithi170 ceph-mon[102832]: pgmap v12366: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 287 KiB/s wr, 13 op/s 2023-09-10T05:40:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:55 smithi082 ceph-mon[98694]: pgmap v12366: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 287 KiB/s wr, 13 op/s 2023-09-10T05:40:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:40:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:40:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:57 smithi098 ceph-mon[103124]: pgmap v12367: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 566 KiB/s wr, 20 op/s 2023-09-10T05:40:58.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22889.355021 secs 2023-09-10T05:40:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:40:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:40:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:57 smithi170 ceph-mon[102832]: pgmap v12367: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 566 KiB/s wr, 20 op/s 2023-09-10T05:40:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22889.355021 secs 2023-09-10T05:40:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:40:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:40:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:57 smithi082 ceph-mon[98694]: pgmap v12367: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 566 KiB/s wr, 20 op/s 2023-09-10T05:40:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22889.355021 secs 2023-09-10T05:40:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22890.392271 secs 2023-09-10T05:40:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22890.392271 secs 2023-09-10T05:40:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22890.392271 secs 2023-09-10T05:41:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22891.045806 secs 2023-09-10T05:41:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:59 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:41:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:59 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:41:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:59 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:00.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:40:59 smithi098 ceph-mon[103124]: pgmap v12368: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 304 KiB/s wr, 13 op/s 2023-09-10T05:41:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22891.045806 secs 2023-09-10T05:41:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:59 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:41:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:59 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:41:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:59 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:40:59 smithi170 ceph-mon[102832]: pgmap v12368: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 304 KiB/s wr, 13 op/s 2023-09-10T05:41:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22891.045806 secs 2023-09-10T05:41:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:59 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:41:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:59 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:41:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:59 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:40:59 smithi082 ceph-mon[98694]: pgmap v12368: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 304 KiB/s wr, 13 op/s 2023-09-10T05:41:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:02 smithi170 ceph-mon[102832]: pgmap v12369: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 446 KiB/s wr, 21 op/s 2023-09-10T05:41:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:02 smithi082 ceph-mon[98694]: pgmap v12369: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 446 KiB/s wr, 21 op/s 2023-09-10T05:41:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:02 smithi098 ceph-mon[103124]: pgmap v12369: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 446 KiB/s wr, 21 op/s 2023-09-10T05:41:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22894.355106 secs 2023-09-10T05:41:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22894.355106 secs 2023-09-10T05:41:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22894.355106 secs 2023-09-10T05:41:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:04 smithi170 ceph-mon[102832]: pgmap v12370: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 421 KiB/s wr, 14 op/s 2023-09-10T05:41:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22895.392329 secs 2023-09-10T05:41:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22896.045871 secs 2023-09-10T05:41:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:04 smithi082 ceph-mon[98694]: pgmap v12370: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 421 KiB/s wr, 14 op/s 2023-09-10T05:41:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22895.392329 secs 2023-09-10T05:41:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22896.045871 secs 2023-09-10T05:41:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:04 smithi098 ceph-mon[103124]: pgmap v12370: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 421 KiB/s wr, 14 op/s 2023-09-10T05:41:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22895.392329 secs 2023-09-10T05:41:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22896.045871 secs 2023-09-10T05:41:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:05 smithi098 ceph-mon[103124]: pgmap v12371: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 421 KiB/s wr, 14 op/s 2023-09-10T05:41:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:05 smithi170 ceph-mon[102832]: pgmap v12371: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 421 KiB/s wr, 14 op/s 2023-09-10T05:41:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:05 smithi082 ceph-mon[98694]: pgmap v12371: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 421 KiB/s wr, 14 op/s 2023-09-10T05:41:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:08 smithi170 ceph-mon[102832]: pgmap v12372: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 645 KiB/s wr, 33 op/s 2023-09-10T05:41:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22899.355167 secs 2023-09-10T05:41:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:08 smithi082 ceph-mon[98694]: pgmap v12372: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 645 KiB/s wr, 33 op/s 2023-09-10T05:41:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22899.355167 secs 2023-09-10T05:41:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:08 smithi098 ceph-mon[103124]: pgmap v12372: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 645 KiB/s wr, 33 op/s 2023-09-10T05:41:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22899.355167 secs 2023-09-10T05:41:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22900.392385 secs 2023-09-10T05:41:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:09 smithi170 ceph-mon[102832]: osdmap e841: 12 total, 12 up, 12 in 2023-09-10T05:41:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22901.045938 secs 2023-09-10T05:41:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22900.392385 secs 2023-09-10T05:41:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:09 smithi082 ceph-mon[98694]: osdmap e841: 12 total, 12 up, 12 in 2023-09-10T05:41:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22901.045938 secs 2023-09-10T05:41:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22900.392385 secs 2023-09-10T05:41:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:09 smithi098 ceph-mon[103124]: osdmap e841: 12 total, 12 up, 12 in 2023-09-10T05:41:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22901.045938 secs 2023-09-10T05:41:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:10 smithi170 ceph-mon[102832]: pgmap v12374: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 439 KiB/s wr, 31 op/s 2023-09-10T05:41:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:10 smithi082 ceph-mon[98694]: pgmap v12374: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 439 KiB/s wr, 31 op/s 2023-09-10T05:41:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:10 smithi098 ceph-mon[103124]: pgmap v12374: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 439 KiB/s wr, 31 op/s 2023-09-10T05:41:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:12 smithi170 ceph-mon[102832]: pgmap v12375: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 339 KiB/s wr, 28 op/s 2023-09-10T05:41:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:12 smithi082 ceph-mon[98694]: pgmap v12375: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 339 KiB/s wr, 28 op/s 2023-09-10T05:41:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:12 smithi098 ceph-mon[103124]: pgmap v12375: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 339 KiB/s wr, 28 op/s 2023-09-10T05:41:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22904.355255 secs 2023-09-10T05:41:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22904.355255 secs 2023-09-10T05:41:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22904.355255 secs 2023-09-10T05:41:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:14 smithi170 ceph-mon[102832]: pgmap v12376: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 339 KiB/s wr, 28 op/s 2023-09-10T05:41:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22905.392444 secs 2023-09-10T05:41:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22906.046000 secs 2023-09-10T05:41:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:14 smithi082 ceph-mon[98694]: pgmap v12376: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 339 KiB/s wr, 28 op/s 2023-09-10T05:41:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22905.392444 secs 2023-09-10T05:41:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22906.046000 secs 2023-09-10T05:41:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:14 smithi098 ceph-mon[103124]: pgmap v12376: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 339 KiB/s wr, 28 op/s 2023-09-10T05:41:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22905.392444 secs 2023-09-10T05:41:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22906.046000 secs 2023-09-10T05:41:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:15 smithi098 ceph-mon[103124]: osdmap e842: 12 total, 12 up, 12 in 2023-09-10T05:41:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:15 smithi098 ceph-mon[103124]: pgmap v12378: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 89 KiB/s wr, 7 op/s 2023-09-10T05:41:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:15 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:41:16.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:15 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:15 smithi170 ceph-mon[102832]: osdmap e842: 12 total, 12 up, 12 in 2023-09-10T05:41:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:15 smithi170 ceph-mon[102832]: pgmap v12378: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 89 KiB/s wr, 7 op/s 2023-09-10T05:41:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:15 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:41:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:15 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:15 smithi082 ceph-mon[98694]: osdmap e842: 12 total, 12 up, 12 in 2023-09-10T05:41:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:15 smithi082 ceph-mon[98694]: pgmap v12378: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 89 KiB/s wr, 7 op/s 2023-09-10T05:41:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:15 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:41:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:15 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:41:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:18 smithi170 ceph-mon[102832]: pgmap v12379: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 103 KiB/s wr, 14 op/s 2023-09-10T05:41:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22909.355316 secs 2023-09-10T05:41:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:41:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:18 smithi082 ceph-mon[98694]: pgmap v12379: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 103 KiB/s wr, 14 op/s 2023-09-10T05:41:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22909.355316 secs 2023-09-10T05:41:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:41:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:18 smithi098 ceph-mon[103124]: pgmap v12379: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 103 KiB/s wr, 14 op/s 2023-09-10T05:41:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22909.355316 secs 2023-09-10T05:41:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22910.392522 secs 2023-09-10T05:41:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22910.392522 secs 2023-09-10T05:41:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22910.392522 secs 2023-09-10T05:41:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22911.046067 secs 2023-09-10T05:41:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:20 smithi170 ceph-mon[102832]: pgmap v12380: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 13 op/s 2023-09-10T05:41:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22911.046067 secs 2023-09-10T05:41:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:20 smithi082 ceph-mon[98694]: pgmap v12380: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 13 op/s 2023-09-10T05:41:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22911.046067 secs 2023-09-10T05:41:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:20 smithi098 ceph-mon[103124]: pgmap v12380: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 13 op/s 2023-09-10T05:41:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:22 smithi170 ceph-mon[102832]: pgmap v12381: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-09-10T05:41:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:22 smithi082 ceph-mon[98694]: pgmap v12381: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-09-10T05:41:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:22 smithi098 ceph-mon[103124]: pgmap v12381: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-09-10T05:41:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22914.355377 secs 2023-09-10T05:41:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22914.355377 secs 2023-09-10T05:41:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22914.355377 secs 2023-09-10T05:41:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:24 smithi170 ceph-mon[102832]: pgmap v12382: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-09-10T05:41:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22915.392579 secs 2023-09-10T05:41:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22916.046132 secs 2023-09-10T05:41:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:24 smithi082 ceph-mon[98694]: pgmap v12382: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-09-10T05:41:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22915.392579 secs 2023-09-10T05:41:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22916.046132 secs 2023-09-10T05:41:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:24 smithi098 ceph-mon[103124]: pgmap v12382: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-09-10T05:41:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22915.392579 secs 2023-09-10T05:41:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22916.046132 secs 2023-09-10T05:41:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:26 smithi170 ceph-mon[102832]: pgmap v12383: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-09-10T05:41:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:26 smithi082 ceph-mon[98694]: pgmap v12383: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-09-10T05:41:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:26 smithi098 ceph-mon[103124]: pgmap v12383: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-09-10T05:41:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:28 smithi170 ceph-mon[102832]: pgmap v12384: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 94 KiB/s wr, 16 op/s 2023-09-10T05:41:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:28 smithi082 ceph-mon[98694]: pgmap v12384: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 94 KiB/s wr, 16 op/s 2023-09-10T05:41:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:28 smithi098 ceph-mon[103124]: pgmap v12384: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 94 KiB/s wr, 16 op/s 2023-09-10T05:41:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22920.392637 secs 2023-09-10T05:41:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22921.046222 secs 2023-09-10T05:41:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22920.392637 secs 2023-09-10T05:41:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22921.046222 secs 2023-09-10T05:41:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22920.392637 secs 2023-09-10T05:41:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22921.046222 secs 2023-09-10T05:41:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:30 smithi170 ceph-mon[102832]: pgmap v12385: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 10 op/s 2023-09-10T05:41:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:30 smithi082 ceph-mon[98694]: pgmap v12385: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 10 op/s 2023-09-10T05:41:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:30 smithi098 ceph-mon[103124]: pgmap v12385: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 10 op/s 2023-09-10T05:41:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:41:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:41:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:41:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:32 smithi170 ceph-mon[102832]: pgmap v12386: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 149 KiB/s wr, 15 op/s 2023-09-10T05:41:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:41:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:41:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:32 smithi082 ceph-mon[98694]: pgmap v12386: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 149 KiB/s wr, 15 op/s 2023-09-10T05:41:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:41:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:41:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:32 smithi098 ceph-mon[103124]: pgmap v12386: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 149 KiB/s wr, 15 op/s 2023-09-10T05:41:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:41:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:41:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:34 smithi170 ceph-mon[102832]: pgmap v12387: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T05:41:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22925.392720 secs 2023-09-10T05:41:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:41:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22926.046286 secs 2023-09-10T05:41:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:34 smithi082 ceph-mon[98694]: pgmap v12387: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T05:41:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22925.392720 secs 2023-09-10T05:41:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:41:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22926.046286 secs 2023-09-10T05:41:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:34 smithi098 ceph-mon[103124]: pgmap v12387: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T05:41:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22925.392720 secs 2023-09-10T05:41:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:41:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22926.046286 secs 2023-09-10T05:41:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:36 smithi170 ceph-mon[102832]: pgmap v12388: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T05:41:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:41:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:36 smithi082 ceph-mon[98694]: pgmap v12388: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T05:41:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:41:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:36 smithi098 ceph-mon[103124]: pgmap v12388: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T05:41:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:41:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:38 smithi170 ceph-mon[102832]: pgmap v12389: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 544 KiB/s wr, 17 op/s 2023-09-10T05:41:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22929.355558 secs 2023-09-10T05:41:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:38 smithi082 ceph-mon[98694]: pgmap v12389: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 544 KiB/s wr, 17 op/s 2023-09-10T05:41:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22929.355558 secs 2023-09-10T05:41:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:38 smithi098 ceph-mon[103124]: pgmap v12389: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 544 KiB/s wr, 17 op/s 2023-09-10T05:41:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22929.355558 secs 2023-09-10T05:41:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22930.392789 secs 2023-09-10T05:41:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22931.046350 secs 2023-09-10T05:41:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22930.392789 secs 2023-09-10T05:41:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22931.046350 secs 2023-09-10T05:41:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22930.392789 secs 2023-09-10T05:41:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22931.046350 secs 2023-09-10T05:41:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:40 smithi170 ceph-mon[102832]: pgmap v12390: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 468 KiB/s wr, 11 op/s 2023-09-10T05:41:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:41:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:41:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:40 smithi082 ceph-mon[98694]: pgmap v12390: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 468 KiB/s wr, 11 op/s 2023-09-10T05:41:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:41:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:41:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:40 smithi098 ceph-mon[103124]: pgmap v12390: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 468 KiB/s wr, 11 op/s 2023-09-10T05:41:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:41:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:41:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:42 smithi082 ceph-mon[98694]: pgmap v12391: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 482 KiB/s wr, 17 op/s 2023-09-10T05:41:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:42 smithi098 ceph-mon[103124]: pgmap v12391: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 482 KiB/s wr, 17 op/s 2023-09-10T05:41:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:42 smithi170 ceph-mon[102832]: pgmap v12391: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 482 KiB/s wr, 17 op/s 2023-09-10T05:41:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:41:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:41:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:41:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:44 smithi082 ceph-mon[98694]: pgmap v12392: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 410 KiB/s wr, 12 op/s 2023-09-10T05:41:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22935.392848 secs 2023-09-10T05:41:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:41:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22936.046415 secs 2023-09-10T05:41:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:44 smithi098 ceph-mon[103124]: pgmap v12392: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 410 KiB/s wr, 12 op/s 2023-09-10T05:41:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22935.392848 secs 2023-09-10T05:41:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:41:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22936.046415 secs 2023-09-10T05:41:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:44 smithi170 ceph-mon[102832]: pgmap v12392: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 410 KiB/s wr, 12 op/s 2023-09-10T05:41:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22935.392848 secs 2023-09-10T05:41:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:41:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22936.046415 secs 2023-09-10T05:41:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:45 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:41:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:45 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:41:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:45 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:41:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:46 smithi082 ceph-mon[98694]: pgmap v12393: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 410 KiB/s wr, 12 op/s 2023-09-10T05:41:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:41:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:46 smithi098 ceph-mon[103124]: pgmap v12393: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 410 KiB/s wr, 12 op/s 2023-09-10T05:41:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:41:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:46 smithi170 ceph-mon[102832]: pgmap v12393: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 410 KiB/s wr, 12 op/s 2023-09-10T05:41:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:41:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:48 smithi082 ceph-mon[98694]: pgmap v12394: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 674 KiB/s wr, 19 op/s 2023-09-10T05:41:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:48 smithi098 ceph-mon[103124]: pgmap v12394: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 674 KiB/s wr, 19 op/s 2023-09-10T05:41:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:48 smithi170 ceph-mon[102832]: pgmap v12394: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 674 KiB/s wr, 19 op/s 2023-09-10T05:41:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22940.392931 secs 2023-09-10T05:41:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22941.046487 secs 2023-09-10T05:41:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22940.392931 secs 2023-09-10T05:41:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22941.046487 secs 2023-09-10T05:41:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22940.392931 secs 2023-09-10T05:41:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22941.046487 secs 2023-09-10T05:41:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:50 smithi082 ceph-mon[98694]: pgmap v12395: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T05:41:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:50 smithi098 ceph-mon[103124]: pgmap v12395: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T05:41:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:50 smithi170 ceph-mon[102832]: pgmap v12395: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T05:41:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:41:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:41:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:41:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:52 smithi098 ceph-mon[103124]: pgmap v12396: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 552 KiB/s wr, 18 op/s 2023-09-10T05:41:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:41:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:41:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:41:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:52 smithi170 ceph-mon[102832]: pgmap v12396: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 552 KiB/s wr, 18 op/s 2023-09-10T05:41:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:41:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:41:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:41:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:52 smithi082 ceph-mon[98694]: pgmap v12396: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 552 KiB/s wr, 18 op/s 2023-09-10T05:41:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:41:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:41:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:41:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22944.355737 secs 2023-09-10T05:41:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22944.355737 secs 2023-09-10T05:41:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22944.355737 secs 2023-09-10T05:41:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:54 smithi098 ceph-mon[103124]: pgmap v12397: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 538 KiB/s wr, 13 op/s 2023-09-10T05:41:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22945.392990 secs 2023-09-10T05:41:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:41:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22946.046552 secs 2023-09-10T05:41:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:54 smithi170 ceph-mon[102832]: pgmap v12397: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 538 KiB/s wr, 13 op/s 2023-09-10T05:41:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22945.392990 secs 2023-09-10T05:41:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:41:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22946.046552 secs 2023-09-10T05:41:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:54 smithi082 ceph-mon[98694]: pgmap v12397: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 538 KiB/s wr, 13 op/s 2023-09-10T05:41:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22945.392990 secs 2023-09-10T05:41:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:41:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22946.046552 secs 2023-09-10T05:41:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:56 smithi170 ceph-mon[102832]: pgmap v12398: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 538 KiB/s wr, 13 op/s 2023-09-10T05:41:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:41:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:41:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:56 smithi082 ceph-mon[98694]: pgmap v12398: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 538 KiB/s wr, 13 op/s 2023-09-10T05:41:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:41:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:41:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:56 smithi098 ceph-mon[103124]: pgmap v12398: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 538 KiB/s wr, 13 op/s 2023-09-10T05:41:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:41:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:41:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:41:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:58 smithi170 ceph-mon[102832]: pgmap v12399: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 585 KiB/s wr, 18 op/s 2023-09-10T05:41:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22949.355806 secs 2023-09-10T05:41:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:58 smithi082 ceph-mon[98694]: pgmap v12399: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 585 KiB/s wr, 18 op/s 2023-09-10T05:41:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22949.355806 secs 2023-09-10T05:41:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:58 smithi098 ceph-mon[103124]: pgmap v12399: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 585 KiB/s wr, 18 op/s 2023-09-10T05:41:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22949.355806 secs 2023-09-10T05:41:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22950.393070 secs 2023-09-10T05:41:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:41:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22951.046620 secs 2023-09-10T05:41:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22950.393070 secs 2023-09-10T05:41:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:41:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22951.046620 secs 2023-09-10T05:41:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22950.393070 secs 2023-09-10T05:41:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:41:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22951.046620 secs 2023-09-10T05:42:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:00 smithi170 ceph-mon[102832]: pgmap v12400: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 321 KiB/s wr, 11 op/s 2023-09-10T05:42:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:00 smithi082 ceph-mon[98694]: pgmap v12400: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 321 KiB/s wr, 11 op/s 2023-09-10T05:42:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:00 smithi098 ceph-mon[103124]: pgmap v12400: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 321 KiB/s wr, 11 op/s 2023-09-10T05:42:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:02 smithi170 ceph-mon[102832]: pgmap v12401: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 483 KiB/s wr, 16 op/s 2023-09-10T05:42:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:02 smithi082 ceph-mon[98694]: pgmap v12401: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 483 KiB/s wr, 16 op/s 2023-09-10T05:42:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:02 smithi098 ceph-mon[103124]: pgmap v12401: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 483 KiB/s wr, 16 op/s 2023-09-10T05:42:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22954.355898 secs 2023-09-10T05:42:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22954.355898 secs 2023-09-10T05:42:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22954.355898 secs 2023-09-10T05:42:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:04 smithi170 ceph-mon[102832]: pgmap v12402: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 209 KiB/s wr, 10 op/s 2023-09-10T05:42:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22955.393128 secs 2023-09-10T05:42:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22956.046683 secs 2023-09-10T05:42:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:04 smithi082 ceph-mon[98694]: pgmap v12402: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 209 KiB/s wr, 10 op/s 2023-09-10T05:42:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22955.393128 secs 2023-09-10T05:42:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22956.046683 secs 2023-09-10T05:42:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:04 smithi098 ceph-mon[103124]: pgmap v12402: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 209 KiB/s wr, 10 op/s 2023-09-10T05:42:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22955.393128 secs 2023-09-10T05:42:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22956.046683 secs 2023-09-10T05:42:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:06 smithi170 ceph-mon[102832]: pgmap v12403: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 209 KiB/s wr, 10 op/s 2023-09-10T05:42:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:06 smithi082 ceph-mon[98694]: pgmap v12403: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 209 KiB/s wr, 10 op/s 2023-09-10T05:42:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:06 smithi098 ceph-mon[103124]: pgmap v12403: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 209 KiB/s wr, 10 op/s 2023-09-10T05:42:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:08 smithi170 ceph-mon[102832]: pgmap v12404: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 330 KiB/s wr, 30 op/s 2023-09-10T05:42:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22959.355978 secs 2023-09-10T05:42:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:08 smithi082 ceph-mon[98694]: pgmap v12404: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 330 KiB/s wr, 30 op/s 2023-09-10T05:42:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22959.355978 secs 2023-09-10T05:42:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:08 smithi098 ceph-mon[103124]: pgmap v12404: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 330 KiB/s wr, 30 op/s 2023-09-10T05:42:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22959.355978 secs 2023-09-10T05:42:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22960.393212 secs 2023-09-10T05:42:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:09 smithi170 ceph-mon[102832]: osdmap e843: 12 total, 12 up, 12 in 2023-09-10T05:42:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22961.046750 secs 2023-09-10T05:42:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22960.393212 secs 2023-09-10T05:42:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:09 smithi082 ceph-mon[98694]: osdmap e843: 12 total, 12 up, 12 in 2023-09-10T05:42:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22961.046750 secs 2023-09-10T05:42:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22960.393212 secs 2023-09-10T05:42:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:09 smithi098 ceph-mon[103124]: osdmap e843: 12 total, 12 up, 12 in 2023-09-10T05:42:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22961.046750 secs 2023-09-10T05:42:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:10 smithi170 ceph-mon[102832]: pgmap v12406: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 340 KiB/s wr, 29 op/s 2023-09-10T05:42:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:10 smithi082 ceph-mon[98694]: pgmap v12406: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 340 KiB/s wr, 29 op/s 2023-09-10T05:42:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:10 smithi098 ceph-mon[103124]: pgmap v12406: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 340 KiB/s wr, 29 op/s 2023-09-10T05:42:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:12 smithi170 ceph-mon[102832]: pgmap v12407: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 167 KiB/s wr, 29 op/s 2023-09-10T05:42:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:12 smithi082 ceph-mon[98694]: pgmap v12407: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 167 KiB/s wr, 29 op/s 2023-09-10T05:42:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:12 smithi098 ceph-mon[103124]: pgmap v12407: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 167 KiB/s wr, 29 op/s 2023-09-10T05:42:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22964.356041 secs 2023-09-10T05:42:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22964.356041 secs 2023-09-10T05:42:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22964.356041 secs 2023-09-10T05:42:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:14 smithi082 ceph-mon[98694]: pgmap v12408: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 167 KiB/s wr, 29 op/s 2023-09-10T05:42:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22965.393271 secs 2023-09-10T05:42:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22966.046817 secs 2023-09-10T05:42:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:14 smithi098 ceph-mon[103124]: pgmap v12408: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 167 KiB/s wr, 29 op/s 2023-09-10T05:42:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22965.393271 secs 2023-09-10T05:42:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22966.046817 secs 2023-09-10T05:42:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:14 smithi170 ceph-mon[102832]: pgmap v12408: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 167 KiB/s wr, 29 op/s 2023-09-10T05:42:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22965.393271 secs 2023-09-10T05:42:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22966.046817 secs 2023-09-10T05:42:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:16 smithi170 ceph-mon[102832]: osdmap e844: 12 total, 12 up, 12 in 2023-09-10T05:42:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:16 smithi170 ceph-mon[102832]: pgmap v12410: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 7 op/s 2023-09-10T05:42:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:16 smithi082 ceph-mon[98694]: osdmap e844: 12 total, 12 up, 12 in 2023-09-10T05:42:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:16 smithi082 ceph-mon[98694]: pgmap v12410: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 7 op/s 2023-09-10T05:42:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:16 smithi098 ceph-mon[103124]: osdmap e844: 12 total, 12 up, 12 in 2023-09-10T05:42:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:16 smithi098 ceph-mon[103124]: pgmap v12410: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 7 op/s 2023-09-10T05:42:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:18 smithi170 ceph-mon[102832]: pgmap v12411: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 13 op/s 2023-09-10T05:42:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22969.356128 secs 2023-09-10T05:42:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:18 smithi082 ceph-mon[98694]: pgmap v12411: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 13 op/s 2023-09-10T05:42:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22969.356128 secs 2023-09-10T05:42:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:18 smithi098 ceph-mon[103124]: pgmap v12411: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 13 op/s 2023-09-10T05:42:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22969.356128 secs 2023-09-10T05:42:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22970.393330 secs 2023-09-10T05:42:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22971.046884 secs 2023-09-10T05:42:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22970.393330 secs 2023-09-10T05:42:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22971.046884 secs 2023-09-10T05:42:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22970.393330 secs 2023-09-10T05:42:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22971.046884 secs 2023-09-10T05:42:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:20 smithi170 ceph-mon[102832]: pgmap v12412: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:42:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:20 smithi082 ceph-mon[98694]: pgmap v12412: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:42:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:20 smithi098 ceph-mon[103124]: pgmap v12412: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:42:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:22 smithi170 ceph-mon[102832]: pgmap v12413: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T05:42:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:22 smithi082 ceph-mon[98694]: pgmap v12413: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T05:42:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:22 smithi098 ceph-mon[103124]: pgmap v12413: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T05:42:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22974.356213 secs 2023-09-10T05:42:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22974.356213 secs 2023-09-10T05:42:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22974.356213 secs 2023-09-10T05:42:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:24 smithi170 ceph-mon[102832]: pgmap v12414: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T05:42:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22975.393388 secs 2023-09-10T05:42:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22976.046949 secs 2023-09-10T05:42:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:24 smithi082 ceph-mon[98694]: pgmap v12414: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T05:42:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22975.393388 secs 2023-09-10T05:42:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22976.046949 secs 2023-09-10T05:42:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:24 smithi098 ceph-mon[103124]: pgmap v12414: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T05:42:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22975.393388 secs 2023-09-10T05:42:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22976.046949 secs 2023-09-10T05:42:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:26 smithi170 ceph-mon[102832]: pgmap v12415: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T05:42:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:26 smithi082 ceph-mon[98694]: pgmap v12415: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T05:42:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:26 smithi098 ceph-mon[103124]: pgmap v12415: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T05:42:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:28 smithi170 ceph-mon[102832]: pgmap v12416: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 15 op/s 2023-09-10T05:42:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:28 smithi082 ceph-mon[98694]: pgmap v12416: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 15 op/s 2023-09-10T05:42:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:28 smithi098 ceph-mon[103124]: pgmap v12416: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 15 op/s 2023-09-10T05:42:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22980.393448 secs 2023-09-10T05:42:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22981.047017 secs 2023-09-10T05:42:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22980.393448 secs 2023-09-10T05:42:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22981.047017 secs 2023-09-10T05:42:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22980.393448 secs 2023-09-10T05:42:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22981.047017 secs 2023-09-10T05:42:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:30 smithi170 ceph-mon[102832]: pgmap v12417: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 10 op/s 2023-09-10T05:42:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:42:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:42:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:30 smithi082 ceph-mon[98694]: pgmap v12417: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 10 op/s 2023-09-10T05:42:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:42:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:42:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:30 smithi098 ceph-mon[103124]: pgmap v12417: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 10 op/s 2023-09-10T05:42:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:42:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:42:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:42:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:42:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:42:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:32 smithi082 ceph-mon[98694]: pgmap v12418: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 198 KiB/s wr, 16 op/s 2023-09-10T05:42:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:42:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:42:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:42:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:32 smithi098 ceph-mon[103124]: pgmap v12418: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 198 KiB/s wr, 16 op/s 2023-09-10T05:42:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:42:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:42:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:42:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:32 smithi170 ceph-mon[102832]: pgmap v12418: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 198 KiB/s wr, 16 op/s 2023-09-10T05:42:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:42:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:42:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:42:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:34 smithi082 ceph-mon[98694]: pgmap v12419: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T05:42:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22985.393532 secs 2023-09-10T05:42:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22986.047083 secs 2023-09-10T05:42:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:34 smithi098 ceph-mon[103124]: pgmap v12419: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T05:42:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22985.393532 secs 2023-09-10T05:42:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22986.047083 secs 2023-09-10T05:42:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:34 smithi170 ceph-mon[102832]: pgmap v12419: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T05:42:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22985.393532 secs 2023-09-10T05:42:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22986.047083 secs 2023-09-10T05:42:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:36 smithi170 ceph-mon[102832]: pgmap v12420: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T05:42:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:36 smithi082 ceph-mon[98694]: pgmap v12420: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T05:42:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:36 smithi098 ceph-mon[103124]: pgmap v12420: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T05:42:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:42:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:42:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:38 smithi170 ceph-mon[102832]: pgmap v12421: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 294 KiB/s wr, 17 op/s 2023-09-10T05:42:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22989.356407 secs 2023-09-10T05:42:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:42:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:42:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:38 smithi082 ceph-mon[98694]: pgmap v12421: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 294 KiB/s wr, 17 op/s 2023-09-10T05:42:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22989.356407 secs 2023-09-10T05:42:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:42:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:42:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:38 smithi098 ceph-mon[103124]: pgmap v12421: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 294 KiB/s wr, 17 op/s 2023-09-10T05:42:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22989.356407 secs 2023-09-10T05:42:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22990.393599 secs 2023-09-10T05:42:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22990.393599 secs 2023-09-10T05:42:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22990.393599 secs 2023-09-10T05:42:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22991.047149 secs 2023-09-10T05:42:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:42:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:42:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:42:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:40 smithi170 ceph-mon[102832]: pgmap v12422: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T05:42:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22991.047149 secs 2023-09-10T05:42:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:42:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:42:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:42:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:40 smithi082 ceph-mon[98694]: pgmap v12422: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T05:42:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22991.047149 secs 2023-09-10T05:42:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:42:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:42:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:42:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:40 smithi098 ceph-mon[103124]: pgmap v12422: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T05:42:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:42 smithi170 ceph-mon[102832]: pgmap v12423: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 430 KiB/s wr, 19 op/s 2023-09-10T05:42:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:42 smithi082 ceph-mon[98694]: pgmap v12423: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 430 KiB/s wr, 19 op/s 2023-09-10T05:42:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:42 smithi098 ceph-mon[103124]: pgmap v12423: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 430 KiB/s wr, 19 op/s 2023-09-10T05:42:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22994.356460 secs 2023-09-10T05:42:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22994.356460 secs 2023-09-10T05:42:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22994.356460 secs 2023-09-10T05:42:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:44 smithi170 ceph-mon[102832]: pgmap v12424: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 309 KiB/s wr, 13 op/s 2023-09-10T05:42:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22995.393658 secs 2023-09-10T05:42:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 22996.047212 secs 2023-09-10T05:42:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:44 smithi082 ceph-mon[98694]: pgmap v12424: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 309 KiB/s wr, 13 op/s 2023-09-10T05:42:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22995.393658 secs 2023-09-10T05:42:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 22996.047212 secs 2023-09-10T05:42:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:44 smithi098 ceph-mon[103124]: pgmap v12424: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 309 KiB/s wr, 13 op/s 2023-09-10T05:42:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22995.393658 secs 2023-09-10T05:42:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 22996.047212 secs 2023-09-10T05:42:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:46 smithi170 ceph-mon[102832]: pgmap v12425: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 309 KiB/s wr, 13 op/s 2023-09-10T05:42:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:46 smithi082 ceph-mon[98694]: pgmap v12425: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 309 KiB/s wr, 13 op/s 2023-09-10T05:42:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:46 smithi098 ceph-mon[103124]: pgmap v12425: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 309 KiB/s wr, 13 op/s 2023-09-10T05:42:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:48 smithi170 ceph-mon[102832]: pgmap v12426: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 555 KiB/s wr, 19 op/s 2023-09-10T05:42:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 22999.356542 secs 2023-09-10T05:42:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:48 smithi082 ceph-mon[98694]: pgmap v12426: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 555 KiB/s wr, 19 op/s 2023-09-10T05:42:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 22999.356542 secs 2023-09-10T05:42:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:48 smithi098 ceph-mon[103124]: pgmap v12426: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 555 KiB/s wr, 19 op/s 2023-09-10T05:42:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 22999.356542 secs 2023-09-10T05:42:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23000.393743 secs 2023-09-10T05:42:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23001.047274 secs 2023-09-10T05:42:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23000.393743 secs 2023-09-10T05:42:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23001.047274 secs 2023-09-10T05:42:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23000.393743 secs 2023-09-10T05:42:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23001.047274 secs 2023-09-10T05:42:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:50 smithi170 ceph-mon[102832]: pgmap v12427: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 383 KiB/s wr, 13 op/s 2023-09-10T05:42:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:42:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:42:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:50 smithi082 ceph-mon[98694]: pgmap v12427: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 383 KiB/s wr, 13 op/s 2023-09-10T05:42:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:42:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:42:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:50 smithi098 ceph-mon[103124]: pgmap v12427: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 383 KiB/s wr, 13 op/s 2023-09-10T05:42:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:42:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:42:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:52 smithi082 ceph-mon[98694]: pgmap v12428: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 527 KiB/s wr, 19 op/s 2023-09-10T05:42:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:42:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:42:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:42:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:52 smithi098 ceph-mon[103124]: pgmap v12428: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 527 KiB/s wr, 19 op/s 2023-09-10T05:42:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:42:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:42:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:42:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:52 smithi170 ceph-mon[102832]: pgmap v12428: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 527 KiB/s wr, 19 op/s 2023-09-10T05:42:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:42:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:42:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:42:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23004.356628 secs 2023-09-10T05:42:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23004.356628 secs 2023-09-10T05:42:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23004.356628 secs 2023-09-10T05:42:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:54 smithi082 ceph-mon[98694]: pgmap v12429: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 391 KiB/s wr, 12 op/s 2023-09-10T05:42:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23005.393803 secs 2023-09-10T05:42:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23006.047341 secs 2023-09-10T05:42:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:54 smithi098 ceph-mon[103124]: pgmap v12429: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 391 KiB/s wr, 12 op/s 2023-09-10T05:42:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23005.393803 secs 2023-09-10T05:42:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23006.047341 secs 2023-09-10T05:42:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:54 smithi170 ceph-mon[102832]: pgmap v12429: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 391 KiB/s wr, 12 op/s 2023-09-10T05:42:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23005.393803 secs 2023-09-10T05:42:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23006.047341 secs 2023-09-10T05:42:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:42:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:42:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:42:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:42:55.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:42:55.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:42:55.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:42:55.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:42:55.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:42:55.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:42:55.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:42:55.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:42:55.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:42:55.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:42:55.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:42:55.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:42:55.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:42:55.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:42:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:42:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:42:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:42:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:42:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:42:55.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:42:55.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:42:55.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:42:55.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:42:55.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:42:55.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:42:55.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:42:55.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:42:55.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:42:55.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:42:55.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:42:55.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:42:55.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:42:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:42:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:42:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:42:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:42:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:42:55.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:42:55.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:42:55.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:42:55.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:42:55.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:42:55.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:42:55.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:42:55.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:42:55.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:42:55.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:42:55.711 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:42:55.711 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:42:55.711 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:42:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:56 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T05:42:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:56 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T05:42:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:56 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T05:42:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:56 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T05:42:56.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:56 smithi098 ceph-mon[103124]: pgmap v12430: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 391 KiB/s wr, 12 op/s 2023-09-10T05:42:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:56 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T05:42:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:56 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T05:42:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:56 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T05:42:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:56 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T05:42:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:56 smithi170 ceph-mon[102832]: pgmap v12430: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 391 KiB/s wr, 12 op/s 2023-09-10T05:42:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:56 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T05:42:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:56 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T05:42:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:56 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T05:42:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:56 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T05:42:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:56 smithi082 ceph-mon[98694]: pgmap v12430: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 391 KiB/s wr, 12 op/s 2023-09-10T05:42:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:58 smithi098 ceph-mon[103124]: pgmap v12431: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 567 KiB/s wr, 17 op/s 2023-09-10T05:42:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23009.356686 secs 2023-09-10T05:42:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:58 smithi170 ceph-mon[102832]: pgmap v12431: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 567 KiB/s wr, 17 op/s 2023-09-10T05:42:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23009.356686 secs 2023-09-10T05:42:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:58 smithi082 ceph-mon[98694]: pgmap v12431: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 567 KiB/s wr, 17 op/s 2023-09-10T05:42:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23009.356686 secs 2023-09-10T05:42:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23010.393858 secs 2023-09-10T05:42:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:42:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23011.047406 secs 2023-09-10T05:42:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23010.393858 secs 2023-09-10T05:42:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:42:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23011.047406 secs 2023-09-10T05:42:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23010.393858 secs 2023-09-10T05:42:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:42:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23011.047406 secs 2023-09-10T05:43:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:00 smithi098 ceph-mon[103124]: pgmap v12432: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 321 KiB/s wr, 11 op/s 2023-09-10T05:43:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:00 smithi170 ceph-mon[102832]: pgmap v12432: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 321 KiB/s wr, 11 op/s 2023-09-10T05:43:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:00 smithi082 ceph-mon[98694]: pgmap v12432: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 321 KiB/s wr, 11 op/s 2023-09-10T05:43:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:02 smithi098 ceph-mon[103124]: pgmap v12433: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 321 KiB/s wr, 15 op/s 2023-09-10T05:43:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:02 smithi170 ceph-mon[102832]: pgmap v12433: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 321 KiB/s wr, 15 op/s 2023-09-10T05:43:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:02 smithi082 ceph-mon[98694]: pgmap v12433: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 321 KiB/s wr, 15 op/s 2023-09-10T05:43:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23014.356745 secs 2023-09-10T05:43:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23014.356745 secs 2023-09-10T05:43:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23014.356745 secs 2023-09-10T05:43:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:04 smithi098 ceph-mon[103124]: pgmap v12434: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 9 op/s 2023-09-10T05:43:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23015.393917 secs 2023-09-10T05:43:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23016.047477 secs 2023-09-10T05:43:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:04 smithi170 ceph-mon[102832]: pgmap v12434: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 9 op/s 2023-09-10T05:43:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23015.393917 secs 2023-09-10T05:43:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23016.047477 secs 2023-09-10T05:43:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:04 smithi082 ceph-mon[98694]: pgmap v12434: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 9 op/s 2023-09-10T05:43:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23015.393917 secs 2023-09-10T05:43:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23016.047477 secs 2023-09-10T05:43:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:06 smithi170 ceph-mon[102832]: pgmap v12435: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 9 op/s 2023-09-10T05:43:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:06 smithi082 ceph-mon[98694]: pgmap v12435: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 9 op/s 2023-09-10T05:43:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:06 smithi098 ceph-mon[103124]: pgmap v12435: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 9 op/s 2023-09-10T05:43:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:08 smithi170 ceph-mon[102832]: pgmap v12436: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 286 KiB/s wr, 29 op/s 2023-09-10T05:43:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23019.422123 secs 2023-09-10T05:43:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:08 smithi082 ceph-mon[98694]: pgmap v12436: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 286 KiB/s wr, 29 op/s 2023-09-10T05:43:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23019.422123 secs 2023-09-10T05:43:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:08 smithi098 ceph-mon[103124]: pgmap v12436: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 286 KiB/s wr, 29 op/s 2023-09-10T05:43:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23019.422123 secs 2023-09-10T05:43:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23020.394010 secs 2023-09-10T05:43:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:09 smithi082 ceph-mon[98694]: osdmap e845: 12 total, 12 up, 12 in 2023-09-10T05:43:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23021.047539 secs 2023-09-10T05:43:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23020.394010 secs 2023-09-10T05:43:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:09 smithi098 ceph-mon[103124]: osdmap e845: 12 total, 12 up, 12 in 2023-09-10T05:43:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23021.047539 secs 2023-09-10T05:43:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23020.394010 secs 2023-09-10T05:43:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:09 smithi170 ceph-mon[102832]: osdmap e845: 12 total, 12 up, 12 in 2023-09-10T05:43:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23021.047539 secs 2023-09-10T05:43:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:10 smithi082 ceph-mon[98694]: pgmap v12438: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 131 KiB/s wr, 29 op/s 2023-09-10T05:43:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:10 smithi098 ceph-mon[103124]: pgmap v12438: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 131 KiB/s wr, 29 op/s 2023-09-10T05:43:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:10 smithi170 ceph-mon[102832]: pgmap v12438: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 131 KiB/s wr, 29 op/s 2023-09-10T05:43:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:12 smithi082 ceph-mon[98694]: pgmap v12439: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 153 KiB/s wr, 29 op/s 2023-09-10T05:43:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:12 smithi098 ceph-mon[103124]: pgmap v12439: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 153 KiB/s wr, 29 op/s 2023-09-10T05:43:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:12 smithi170 ceph-mon[102832]: pgmap v12439: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 153 KiB/s wr, 29 op/s 2023-09-10T05:43:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23024.422244 secs 2023-09-10T05:43:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23024.422244 secs 2023-09-10T05:43:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23024.422244 secs 2023-09-10T05:43:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:14 smithi082 ceph-mon[98694]: pgmap v12440: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 153 KiB/s wr, 29 op/s 2023-09-10T05:43:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23025.394068 secs 2023-09-10T05:43:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23026.047606 secs 2023-09-10T05:43:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:14 smithi098 ceph-mon[103124]: pgmap v12440: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 153 KiB/s wr, 29 op/s 2023-09-10T05:43:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23025.394068 secs 2023-09-10T05:43:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23026.047606 secs 2023-09-10T05:43:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:14 smithi170 ceph-mon[102832]: pgmap v12440: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 153 KiB/s wr, 29 op/s 2023-09-10T05:43:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23025.394068 secs 2023-09-10T05:43:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23026.047606 secs 2023-09-10T05:43:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:16 smithi170 ceph-mon[102832]: osdmap e846: 12 total, 12 up, 12 in 2023-09-10T05:43:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:16 smithi170 ceph-mon[102832]: pgmap v12442: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 7 op/s 2023-09-10T05:43:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:16 smithi082 ceph-mon[98694]: osdmap e846: 12 total, 12 up, 12 in 2023-09-10T05:43:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:16 smithi082 ceph-mon[98694]: pgmap v12442: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 7 op/s 2023-09-10T05:43:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:16 smithi098 ceph-mon[103124]: osdmap e846: 12 total, 12 up, 12 in 2023-09-10T05:43:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:16 smithi098 ceph-mon[103124]: pgmap v12442: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 7 op/s 2023-09-10T05:43:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:18 smithi082 ceph-mon[98694]: pgmap v12443: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 13 op/s 2023-09-10T05:43:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23029.422305 secs 2023-09-10T05:43:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:18 smithi098 ceph-mon[103124]: pgmap v12443: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 13 op/s 2023-09-10T05:43:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23029.422305 secs 2023-09-10T05:43:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:18 smithi170 ceph-mon[102832]: pgmap v12443: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 13 op/s 2023-09-10T05:43:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23029.422305 secs 2023-09-10T05:43:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23030.394144 secs 2023-09-10T05:43:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23031.047673 secs 2023-09-10T05:43:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23030.394144 secs 2023-09-10T05:43:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23031.047673 secs 2023-09-10T05:43:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23030.394144 secs 2023-09-10T05:43:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23031.047673 secs 2023-09-10T05:43:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:20 smithi082 ceph-mon[98694]: pgmap v12444: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:43:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:20 smithi098 ceph-mon[103124]: pgmap v12444: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:43:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:20 smithi170 ceph-mon[102832]: pgmap v12444: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:43:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:22 smithi082 ceph-mon[98694]: pgmap v12445: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T05:43:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:22 smithi098 ceph-mon[103124]: pgmap v12445: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T05:43:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:22 smithi170 ceph-mon[102832]: pgmap v12445: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T05:43:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23034.422423 secs 2023-09-10T05:43:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23034.422423 secs 2023-09-10T05:43:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23034.422423 secs 2023-09-10T05:43:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:24 smithi082 ceph-mon[98694]: pgmap v12446: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T05:43:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23035.394202 secs 2023-09-10T05:43:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23036.047761 secs 2023-09-10T05:43:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:24 smithi098 ceph-mon[103124]: pgmap v12446: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T05:43:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23035.394202 secs 2023-09-10T05:43:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23036.047761 secs 2023-09-10T05:43:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:24 smithi170 ceph-mon[102832]: pgmap v12446: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 12 op/s 2023-09-10T05:43:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23035.394202 secs 2023-09-10T05:43:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23036.047761 secs 2023-09-10T05:43:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:26 smithi170 ceph-mon[102832]: pgmap v12447: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T05:43:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:26 smithi082 ceph-mon[98694]: pgmap v12447: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T05:43:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:26 smithi098 ceph-mon[103124]: pgmap v12447: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T05:43:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:28 smithi082 ceph-mon[98694]: pgmap v12448: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 15 op/s 2023-09-10T05:43:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:28 smithi098 ceph-mon[103124]: pgmap v12448: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 15 op/s 2023-09-10T05:43:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:28 smithi170 ceph-mon[102832]: pgmap v12448: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 KiB/s wr, 15 op/s 2023-09-10T05:43:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:29 smithi082 ceph-mon[98694]: 3 slow requests, 3 included below; oldest blocked for > 23040.394259 secs 2023-09-10T05:43:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:29 smithi082 ceph-mon[98694]: slow request 23040.394259 seconds old, received at 2023-09-09T23:19:27.740042+0000: client_request(mds.4:47 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently acquired locks 2023-09-10T05:43:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:29 smithi082 ceph-mon[98694]: slow request 23040.393879 seconds old, received at 2023-09-09T23:19:27.740422+0000: client_request(mds.4:48 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently requesting remote authpins 2023-09-10T05:43:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:29 smithi082 ceph-mon[98694]: slow request 23040.391657 seconds old, received at 2023-09-09T23:19:27.742643+0000: client_request(client.24913:3869 unlink #0x10000000007/00000001000000010000000B 2023-09-09T23:19:27.741863+0000 caller_uid=26, caller_gid=26{26,}) currently requesting remote authpins 2023-09-10T05:43:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:29 smithi082 ceph-mon[98694]: 3 slow requests, 3 included below; oldest blocked for > 23041.047825 secs 2023-09-10T05:43:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:29 smithi082 ceph-mon[98694]: slow request 23041.047824 seconds old, received at 2023-09-09T23:19:27.740800+0000: peer_request:mds.4:47 currently acquired locks 2023-09-10T05:43:29.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:29 smithi082 ceph-mon[98694]: slow request 23041.047590 seconds old, received at 2023-09-09T23:19:27.741034+0000: peer_request(mds.4:48.0 authpin) currently dispatched 2023-09-10T05:43:29.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:29 smithi082 ceph-mon[98694]: slow request 23041.045555 seconds old, received at 2023-09-09T23:19:27.743070+0000: peer_request(client.24913:3869.0 authpin) currently dispatched 2023-09-10T05:43:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:29 smithi098 ceph-mon[103124]: 3 slow requests, 3 included below; oldest blocked for > 23040.394259 secs 2023-09-10T05:43:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:29 smithi098 ceph-mon[103124]: slow request 23040.394259 seconds old, received at 2023-09-09T23:19:27.740042+0000: client_request(mds.4:47 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently acquired locks 2023-09-10T05:43:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:29 smithi098 ceph-mon[103124]: slow request 23040.393879 seconds old, received at 2023-09-09T23:19:27.740422+0000: client_request(mds.4:48 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently requesting remote authpins 2023-09-10T05:43:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:29 smithi098 ceph-mon[103124]: slow request 23040.391657 seconds old, received at 2023-09-09T23:19:27.742643+0000: client_request(client.24913:3869 unlink #0x10000000007/00000001000000010000000B 2023-09-09T23:19:27.741863+0000 caller_uid=26, caller_gid=26{26,}) currently requesting remote authpins 2023-09-10T05:43:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:29 smithi098 ceph-mon[103124]: 3 slow requests, 3 included below; oldest blocked for > 23041.047825 secs 2023-09-10T05:43:29.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:29 smithi098 ceph-mon[103124]: slow request 23041.047824 seconds old, received at 2023-09-09T23:19:27.740800+0000: peer_request:mds.4:47 currently acquired locks 2023-09-10T05:43:29.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:29 smithi098 ceph-mon[103124]: slow request 23041.047590 seconds old, received at 2023-09-09T23:19:27.741034+0000: peer_request(mds.4:48.0 authpin) currently dispatched 2023-09-10T05:43:29.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:29 smithi098 ceph-mon[103124]: slow request 23041.045555 seconds old, received at 2023-09-09T23:19:27.743070+0000: peer_request(client.24913:3869.0 authpin) currently dispatched 2023-09-10T05:43:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:29 smithi170 ceph-mon[102832]: 3 slow requests, 3 included below; oldest blocked for > 23040.394259 secs 2023-09-10T05:43:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:29 smithi170 ceph-mon[102832]: slow request 23040.394259 seconds old, received at 2023-09-09T23:19:27.740042+0000: client_request(mds.4:47 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently acquired locks 2023-09-10T05:43:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:29 smithi170 ceph-mon[102832]: slow request 23040.393879 seconds old, received at 2023-09-09T23:19:27.740422+0000: client_request(mds.4:48 rename #0x10000000007/00000001000000010000000B #0x60d/20000000203 caller_uid=0, caller_gid=0{}) currently requesting remote authpins 2023-09-10T05:43:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:29 smithi170 ceph-mon[102832]: slow request 23040.391657 seconds old, received at 2023-09-09T23:19:27.742643+0000: client_request(client.24913:3869 unlink #0x10000000007/00000001000000010000000B 2023-09-09T23:19:27.741863+0000 caller_uid=26, caller_gid=26{26,}) currently requesting remote authpins 2023-09-10T05:43:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:29 smithi170 ceph-mon[102832]: 3 slow requests, 3 included below; oldest blocked for > 23041.047825 secs 2023-09-10T05:43:29.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:29 smithi170 ceph-mon[102832]: slow request 23041.047824 seconds old, received at 2023-09-09T23:19:27.740800+0000: peer_request:mds.4:47 currently acquired locks 2023-09-10T05:43:29.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:29 smithi170 ceph-mon[102832]: slow request 23041.047590 seconds old, received at 2023-09-09T23:19:27.741034+0000: peer_request(mds.4:48.0 authpin) currently dispatched 2023-09-10T05:43:29.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:29 smithi170 ceph-mon[102832]: slow request 23041.045555 seconds old, received at 2023-09-09T23:19:27.743070+0000: peer_request(client.24913:3869.0 authpin) currently dispatched 2023-09-10T05:43:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:30 smithi082 ceph-mon[98694]: pgmap v12449: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 10 op/s 2023-09-10T05:43:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:43:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:43:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:30 smithi098 ceph-mon[103124]: pgmap v12449: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 10 op/s 2023-09-10T05:43:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:43:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:43:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:30 smithi170 ceph-mon[102832]: pgmap v12449: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 10 op/s 2023-09-10T05:43:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:43:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:43:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:43:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:43:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:43:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:32 smithi082 ceph-mon[98694]: pgmap v12450: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 184 KiB/s wr, 15 op/s 2023-09-10T05:43:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:43:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:43:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:43:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:32 smithi098 ceph-mon[103124]: pgmap v12450: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 184 KiB/s wr, 15 op/s 2023-09-10T05:43:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:43:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:43:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:43:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:32 smithi170 ceph-mon[102832]: pgmap v12450: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 184 KiB/s wr, 15 op/s 2023-09-10T05:43:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:43:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:43:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:43:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:34 smithi082 ceph-mon[98694]: pgmap v12451: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 10 op/s 2023-09-10T05:43:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23045.394349 secs 2023-09-10T05:43:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23046.047890 secs 2023-09-10T05:43:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:34 smithi098 ceph-mon[103124]: pgmap v12451: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 10 op/s 2023-09-10T05:43:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23045.394349 secs 2023-09-10T05:43:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23046.047890 secs 2023-09-10T05:43:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:34 smithi170 ceph-mon[102832]: pgmap v12451: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 10 op/s 2023-09-10T05:43:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23045.394349 secs 2023-09-10T05:43:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23046.047890 secs 2023-09-10T05:43:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:36 smithi170 ceph-mon[102832]: pgmap v12452: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 10 op/s 2023-09-10T05:43:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:36 smithi082 ceph-mon[98694]: pgmap v12452: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 10 op/s 2023-09-10T05:43:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:36 smithi098 ceph-mon[103124]: pgmap v12452: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 10 op/s 2023-09-10T05:43:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:43:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:43:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:38 smithi170 ceph-mon[102832]: pgmap v12453: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 281 KiB/s wr, 16 op/s 2023-09-10T05:43:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:43:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:43:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:38 smithi082 ceph-mon[98694]: pgmap v12453: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 281 KiB/s wr, 16 op/s 2023-09-10T05:43:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:43:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:43:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:38 smithi098 ceph-mon[103124]: pgmap v12453: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 281 KiB/s wr, 16 op/s 2023-09-10T05:43:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23050.394409 secs 2023-09-10T05:43:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23050.394409 secs 2023-09-10T05:43:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23050.394409 secs 2023-09-10T05:43:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23051.047959 secs 2023-09-10T05:43:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:43:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:43:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:40 smithi170 ceph-mon[102832]: pgmap v12454: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 281 KiB/s wr, 11 op/s 2023-09-10T05:43:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23051.047959 secs 2023-09-10T05:43:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:43:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:43:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:40 smithi082 ceph-mon[98694]: pgmap v12454: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 281 KiB/s wr, 11 op/s 2023-09-10T05:43:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23051.047959 secs 2023-09-10T05:43:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:43:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:43:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:40 smithi098 ceph-mon[103124]: pgmap v12454: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 281 KiB/s wr, 11 op/s 2023-09-10T05:43:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:42 smithi082 ceph-mon[98694]: pgmap v12455: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 418 KiB/s wr, 18 op/s 2023-09-10T05:43:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:42 smithi098 ceph-mon[103124]: pgmap v12455: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 418 KiB/s wr, 18 op/s 2023-09-10T05:43:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:42 smithi170 ceph-mon[102832]: pgmap v12455: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 418 KiB/s wr, 18 op/s 2023-09-10T05:43:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:43 smithi082 ceph-mon[98694]: 1 slow requests, 1 included below; oldest blocked for > 23054.422775 secs 2023-09-10T05:43:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:43 smithi082 ceph-mon[98694]: slow request 23054.422775 seconds old, received at 2023-09-09T23:19:28.303507+0000: peer_request:mds.4:47 currently dispatched 2023-09-10T05:43:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:43 smithi098 ceph-mon[103124]: 1 slow requests, 1 included below; oldest blocked for > 23054.422775 secs 2023-09-10T05:43:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:43 smithi098 ceph-mon[103124]: slow request 23054.422775 seconds old, received at 2023-09-09T23:19:28.303507+0000: peer_request:mds.4:47 currently dispatched 2023-09-10T05:43:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:43 smithi170 ceph-mon[102832]: 1 slow requests, 1 included below; oldest blocked for > 23054.422775 secs 2023-09-10T05:43:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:43 smithi170 ceph-mon[102832]: slow request 23054.422775 seconds old, received at 2023-09-09T23:19:28.303507+0000: peer_request:mds.4:47 currently dispatched 2023-09-10T05:43:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:44 smithi082 ceph-mon[98694]: pgmap v12456: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 310 KiB/s wr, 13 op/s 2023-09-10T05:43:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23055.394465 secs 2023-09-10T05:43:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:43:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:43:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:43:44.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23056.048027 secs 2023-09-10T05:43:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:44 smithi098 ceph-mon[103124]: pgmap v12456: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 310 KiB/s wr, 13 op/s 2023-09-10T05:43:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23055.394465 secs 2023-09-10T05:43:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:43:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:43:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:43:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23056.048027 secs 2023-09-10T05:43:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:44 smithi170 ceph-mon[102832]: pgmap v12456: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 310 KiB/s wr, 13 op/s 2023-09-10T05:43:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23055.394465 secs 2023-09-10T05:43:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:43:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:43:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:43:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23056.048027 secs 2023-09-10T05:43:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:46 smithi170 ceph-mon[102832]: pgmap v12457: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 310 KiB/s wr, 13 op/s 2023-09-10T05:43:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:46 smithi082 ceph-mon[98694]: pgmap v12457: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 310 KiB/s wr, 13 op/s 2023-09-10T05:43:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:46 smithi098 ceph-mon[103124]: pgmap v12457: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 310 KiB/s wr, 13 op/s 2023-09-10T05:43:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:48 smithi082 ceph-mon[98694]: pgmap v12458: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 465 KiB/s wr, 19 op/s 2023-09-10T05:43:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23059.422836 secs 2023-09-10T05:43:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:48 smithi098 ceph-mon[103124]: pgmap v12458: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 465 KiB/s wr, 19 op/s 2023-09-10T05:43:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23059.422836 secs 2023-09-10T05:43:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:48 smithi170 ceph-mon[102832]: pgmap v12458: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 465 KiB/s wr, 19 op/s 2023-09-10T05:43:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23059.422836 secs 2023-09-10T05:43:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23060.394549 secs 2023-09-10T05:43:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23061.048093 secs 2023-09-10T05:43:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23060.394549 secs 2023-09-10T05:43:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23061.048093 secs 2023-09-10T05:43:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23060.394549 secs 2023-09-10T05:43:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23061.048093 secs 2023-09-10T05:43:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:50 smithi082 ceph-mon[98694]: pgmap v12459: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 292 KiB/s wr, 13 op/s 2023-09-10T05:43:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:50 smithi098 ceph-mon[103124]: pgmap v12459: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 292 KiB/s wr, 13 op/s 2023-09-10T05:43:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:50 smithi170 ceph-mon[102832]: pgmap v12459: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 292 KiB/s wr, 13 op/s 2023-09-10T05:43:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:52 smithi082 ceph-mon[98694]: pgmap v12460: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 455 KiB/s wr, 19 op/s 2023-09-10T05:43:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:43:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:43:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:52 smithi098 ceph-mon[103124]: pgmap v12460: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 455 KiB/s wr, 19 op/s 2023-09-10T05:43:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:43:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:43:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:52 smithi170 ceph-mon[102832]: pgmap v12460: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 455 KiB/s wr, 19 op/s 2023-09-10T05:43:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:43:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:43:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23064.422897 secs 2023-09-10T05:43:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23064.422897 secs 2023-09-10T05:43:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23064.422897 secs 2023-09-10T05:43:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23065.394621 secs 2023-09-10T05:43:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:54 smithi082 ceph-mon[98694]: pgmap v12461: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 319 KiB/s wr, 12 op/s 2023-09-10T05:43:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:43:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:43:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23066.048159 secs 2023-09-10T05:43:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23065.394621 secs 2023-09-10T05:43:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:54 smithi098 ceph-mon[103124]: pgmap v12461: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 319 KiB/s wr, 12 op/s 2023-09-10T05:43:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:43:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:43:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23066.048159 secs 2023-09-10T05:43:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23065.394621 secs 2023-09-10T05:43:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:54 smithi170 ceph-mon[102832]: pgmap v12461: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 319 KiB/s wr, 12 op/s 2023-09-10T05:43:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:43:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:43:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23066.048159 secs 2023-09-10T05:43:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:43:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:56 smithi170 ceph-mon[102832]: pgmap v12462: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 319 KiB/s wr, 12 op/s 2023-09-10T05:43:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:43:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:43:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:43:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:43:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:56 smithi082 ceph-mon[98694]: pgmap v12462: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 319 KiB/s wr, 12 op/s 2023-09-10T05:43:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:43:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:43:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:43:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:43:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:56 smithi098 ceph-mon[103124]: pgmap v12462: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 319 KiB/s wr, 12 op/s 2023-09-10T05:43:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:43:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:43:56.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:43:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:58 smithi082 ceph-mon[98694]: pgmap v12463: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 496 KiB/s wr, 18 op/s 2023-09-10T05:43:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23069.423013 secs 2023-09-10T05:43:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:58 smithi098 ceph-mon[103124]: pgmap v12463: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 496 KiB/s wr, 18 op/s 2023-09-10T05:43:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23069.423013 secs 2023-09-10T05:43:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:58 smithi170 ceph-mon[102832]: pgmap v12463: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 496 KiB/s wr, 18 op/s 2023-09-10T05:43:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23069.423013 secs 2023-09-10T05:43:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23070.394676 secs 2023-09-10T05:43:59.463 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:43:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23071.048227 secs 2023-09-10T05:43:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23070.394676 secs 2023-09-10T05:43:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:43:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23071.048227 secs 2023-09-10T05:43:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23070.394676 secs 2023-09-10T05:43:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:43:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23071.048227 secs 2023-09-10T05:44:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:00 smithi082 ceph-mon[98694]: pgmap v12464: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 341 KiB/s wr, 12 op/s 2023-09-10T05:44:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:00 smithi098 ceph-mon[103124]: pgmap v12464: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 341 KiB/s wr, 12 op/s 2023-09-10T05:44:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:00 smithi170 ceph-mon[102832]: pgmap v12464: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 341 KiB/s wr, 12 op/s 2023-09-10T05:44:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:02 smithi082 ceph-mon[98694]: pgmap v12465: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 17 op/s 2023-09-10T05:44:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:02 smithi098 ceph-mon[103124]: pgmap v12465: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 17 op/s 2023-09-10T05:44:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:02 smithi170 ceph-mon[102832]: pgmap v12465: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 17 op/s 2023-09-10T05:44:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23074.423073 secs 2023-09-10T05:44:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23074.423073 secs 2023-09-10T05:44:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23074.423073 secs 2023-09-10T05:44:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23075.394737 secs 2023-09-10T05:44:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:04 smithi082 ceph-mon[98694]: pgmap v12466: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 281 KiB/s wr, 10 op/s 2023-09-10T05:44:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23076.048294 secs 2023-09-10T05:44:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23075.394737 secs 2023-09-10T05:44:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:04 smithi098 ceph-mon[103124]: pgmap v12466: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 281 KiB/s wr, 10 op/s 2023-09-10T05:44:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23076.048294 secs 2023-09-10T05:44:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23075.394737 secs 2023-09-10T05:44:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:04 smithi170 ceph-mon[102832]: pgmap v12466: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 281 KiB/s wr, 10 op/s 2023-09-10T05:44:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23076.048294 secs 2023-09-10T05:44:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:06 smithi170 ceph-mon[102832]: pgmap v12467: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 281 KiB/s wr, 10 op/s 2023-09-10T05:44:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:06 smithi082 ceph-mon[98694]: pgmap v12467: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 281 KiB/s wr, 10 op/s 2023-09-10T05:44:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:06 smithi098 ceph-mon[103124]: pgmap v12467: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 281 KiB/s wr, 10 op/s 2023-09-10T05:44:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:08 smithi082 ceph-mon[98694]: pgmap v12468: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 403 KiB/s wr, 30 op/s 2023-09-10T05:44:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23079.423167 secs 2023-09-10T05:44:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:08 smithi098 ceph-mon[103124]: pgmap v12468: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 403 KiB/s wr, 30 op/s 2023-09-10T05:44:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23079.423167 secs 2023-09-10T05:44:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:08 smithi170 ceph-mon[102832]: pgmap v12468: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 403 KiB/s wr, 30 op/s 2023-09-10T05:44:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23079.423167 secs 2023-09-10T05:44:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23080.394822 secs 2023-09-10T05:44:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:09 smithi082 ceph-mon[98694]: osdmap e847: 12 total, 12 up, 12 in 2023-09-10T05:44:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23081.048360 secs 2023-09-10T05:44:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23080.394822 secs 2023-09-10T05:44:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:09 smithi098 ceph-mon[103124]: osdmap e847: 12 total, 12 up, 12 in 2023-09-10T05:44:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23081.048360 secs 2023-09-10T05:44:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23080.394822 secs 2023-09-10T05:44:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:09 smithi170 ceph-mon[102832]: osdmap e847: 12 total, 12 up, 12 in 2023-09-10T05:44:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23081.048360 secs 2023-09-10T05:44:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:10 smithi082 ceph-mon[98694]: pgmap v12470: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 271 KiB/s wr, 30 op/s 2023-09-10T05:44:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:10 smithi098 ceph-mon[103124]: pgmap v12470: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 271 KiB/s wr, 30 op/s 2023-09-10T05:44:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:10 smithi170 ceph-mon[102832]: pgmap v12470: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 271 KiB/s wr, 30 op/s 2023-09-10T05:44:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:12 smithi098 ceph-mon[103124]: pgmap v12471: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 167 KiB/s wr, 30 op/s 2023-09-10T05:44:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:12 smithi170 ceph-mon[102832]: pgmap v12471: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 167 KiB/s wr, 30 op/s 2023-09-10T05:44:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:12 smithi082 ceph-mon[98694]: pgmap v12471: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 167 KiB/s wr, 30 op/s 2023-09-10T05:44:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23084.423228 secs 2023-09-10T05:44:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23084.423228 secs 2023-09-10T05:44:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23084.423228 secs 2023-09-10T05:44:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23085.394900 secs 2023-09-10T05:44:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:14 smithi098 ceph-mon[103124]: pgmap v12472: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 167 KiB/s wr, 30 op/s 2023-09-10T05:44:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23086.048443 secs 2023-09-10T05:44:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23085.394900 secs 2023-09-10T05:44:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:14 smithi170 ceph-mon[102832]: pgmap v12472: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 167 KiB/s wr, 30 op/s 2023-09-10T05:44:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23086.048443 secs 2023-09-10T05:44:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23085.394900 secs 2023-09-10T05:44:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:14 smithi082 ceph-mon[98694]: pgmap v12472: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 167 KiB/s wr, 30 op/s 2023-09-10T05:44:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23086.048443 secs 2023-09-10T05:44:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:16 smithi170 ceph-mon[102832]: osdmap e848: 12 total, 12 up, 12 in 2023-09-10T05:44:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:16 smithi170 ceph-mon[102832]: pgmap v12474: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 8 op/s 2023-09-10T05:44:16.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:44:16.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:44:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:16 smithi082 ceph-mon[98694]: osdmap e848: 12 total, 12 up, 12 in 2023-09-10T05:44:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:16 smithi082 ceph-mon[98694]: pgmap v12474: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 8 op/s 2023-09-10T05:44:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:44:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:44:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:16 smithi098 ceph-mon[103124]: osdmap e848: 12 total, 12 up, 12 in 2023-09-10T05:44:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:16 smithi098 ceph-mon[103124]: pgmap v12474: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 8 op/s 2023-09-10T05:44:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:44:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:44:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:44:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:44:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:18 smithi170 ceph-mon[102832]: pgmap v12475: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 14 op/s 2023-09-10T05:44:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23089.423316 secs 2023-09-10T05:44:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:44:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:44:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:18 smithi082 ceph-mon[98694]: pgmap v12475: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 14 op/s 2023-09-10T05:44:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23089.423316 secs 2023-09-10T05:44:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:44:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:44:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:18 smithi098 ceph-mon[103124]: pgmap v12475: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 14 op/s 2023-09-10T05:44:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23089.423316 secs 2023-09-10T05:44:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23090.394958 secs 2023-09-10T05:44:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23090.394958 secs 2023-09-10T05:44:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23090.394958 secs 2023-09-10T05:44:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23091.048487 secs 2023-09-10T05:44:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:20 smithi170 ceph-mon[102832]: pgmap v12476: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 79 KiB/s wr, 12 op/s 2023-09-10T05:44:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23091.048487 secs 2023-09-10T05:44:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:20 smithi082 ceph-mon[98694]: pgmap v12476: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 79 KiB/s wr, 12 op/s 2023-09-10T05:44:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23091.048487 secs 2023-09-10T05:44:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:20 smithi098 ceph-mon[103124]: pgmap v12476: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 79 KiB/s wr, 12 op/s 2023-09-10T05:44:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:22 smithi082 ceph-mon[98694]: pgmap v12477: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T05:44:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:22 smithi098 ceph-mon[103124]: pgmap v12477: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T05:44:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:22 smithi170 ceph-mon[102832]: pgmap v12477: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T05:44:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23094.423387 secs 2023-09-10T05:44:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23094.423387 secs 2023-09-10T05:44:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23094.423387 secs 2023-09-10T05:44:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23095.395015 secs 2023-09-10T05:44:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:24 smithi098 ceph-mon[103124]: pgmap v12478: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T05:44:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23096.048551 secs 2023-09-10T05:44:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23095.395015 secs 2023-09-10T05:44:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:24 smithi170 ceph-mon[102832]: pgmap v12478: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T05:44:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23096.048551 secs 2023-09-10T05:44:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23095.395015 secs 2023-09-10T05:44:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:24 smithi082 ceph-mon[98694]: pgmap v12478: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T05:44:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23096.048551 secs 2023-09-10T05:44:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:26 smithi170 ceph-mon[102832]: pgmap v12479: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 12 op/s 2023-09-10T05:44:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:26 smithi082 ceph-mon[98694]: pgmap v12479: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 12 op/s 2023-09-10T05:44:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:26 smithi098 ceph-mon[103124]: pgmap v12479: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 12 op/s 2023-09-10T05:44:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:28 smithi082 ceph-mon[98694]: pgmap v12480: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 125 KiB/s wr, 14 op/s 2023-09-10T05:44:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:28 smithi098 ceph-mon[103124]: pgmap v12480: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 125 KiB/s wr, 14 op/s 2023-09-10T05:44:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:28 smithi170 ceph-mon[102832]: pgmap v12480: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 125 KiB/s wr, 14 op/s 2023-09-10T05:44:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23100.395094 secs 2023-09-10T05:44:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23101.048616 secs 2023-09-10T05:44:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23100.395094 secs 2023-09-10T05:44:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23101.048616 secs 2023-09-10T05:44:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23100.395094 secs 2023-09-10T05:44:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23101.048616 secs 2023-09-10T05:44:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:30 smithi082 ceph-mon[98694]: pgmap v12481: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 9 op/s 2023-09-10T05:44:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:30 smithi098 ceph-mon[103124]: pgmap v12481: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 9 op/s 2023-09-10T05:44:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:30 smithi170 ceph-mon[102832]: pgmap v12481: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 9 op/s 2023-09-10T05:44:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:44:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:44:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:44:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:32 smithi098 ceph-mon[103124]: pgmap v12482: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 200 KiB/s wr, 15 op/s 2023-09-10T05:44:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:44:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:44:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:44:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:32 smithi170 ceph-mon[102832]: pgmap v12482: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 200 KiB/s wr, 15 op/s 2023-09-10T05:44:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:44:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:44:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:44:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:32 smithi082 ceph-mon[98694]: pgmap v12482: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 200 KiB/s wr, 15 op/s 2023-09-10T05:44:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:44:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:44:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:44:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23105.395153 secs 2023-09-10T05:44:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:34 smithi098 ceph-mon[103124]: pgmap v12483: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T05:44:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23106.048679 secs 2023-09-10T05:44:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23105.395153 secs 2023-09-10T05:44:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:34 smithi170 ceph-mon[102832]: pgmap v12483: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T05:44:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23106.048679 secs 2023-09-10T05:44:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23105.395153 secs 2023-09-10T05:44:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:34 smithi082 ceph-mon[98694]: pgmap v12483: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T05:44:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23106.048679 secs 2023-09-10T05:44:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:36 smithi170 ceph-mon[102832]: pgmap v12484: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T05:44:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:36 smithi082 ceph-mon[98694]: pgmap v12484: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T05:44:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:36 smithi098 ceph-mon[103124]: pgmap v12484: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T05:44:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:44:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:44:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:38 smithi170 ceph-mon[102832]: pgmap v12485: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 296 KiB/s wr, 16 op/s 2023-09-10T05:44:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:44:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:44:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:38 smithi082 ceph-mon[98694]: pgmap v12485: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 296 KiB/s wr, 16 op/s 2023-09-10T05:44:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:44:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:44:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:38 smithi098 ceph-mon[103124]: pgmap v12485: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 296 KiB/s wr, 16 op/s 2023-09-10T05:44:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23110.395241 secs 2023-09-10T05:44:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23110.395241 secs 2023-09-10T05:44:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23110.395241 secs 2023-09-10T05:44:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23111.048744 secs 2023-09-10T05:44:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:44:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:44:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:40 smithi170 ceph-mon[102832]: pgmap v12486: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 296 KiB/s wr, 12 op/s 2023-09-10T05:44:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23111.048744 secs 2023-09-10T05:44:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:44:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:44:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:40 smithi082 ceph-mon[98694]: pgmap v12486: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 296 KiB/s wr, 12 op/s 2023-09-10T05:44:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23111.048744 secs 2023-09-10T05:44:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:44:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:44:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:40 smithi098 ceph-mon[103124]: pgmap v12486: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 296 KiB/s wr, 12 op/s 2023-09-10T05:44:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:42 smithi082 ceph-mon[98694]: pgmap v12487: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 433 KiB/s wr, 18 op/s 2023-09-10T05:44:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:42 smithi098 ceph-mon[103124]: pgmap v12487: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 433 KiB/s wr, 18 op/s 2023-09-10T05:44:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:42 smithi170 ceph-mon[102832]: pgmap v12487: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 433 KiB/s wr, 18 op/s 2023-09-10T05:44:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23115.395329 secs 2023-09-10T05:44:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:44 smithi082 ceph-mon[98694]: pgmap v12488: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 311 KiB/s wr, 12 op/s 2023-09-10T05:44:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23116.048809 secs 2023-09-10T05:44:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23115.395329 secs 2023-09-10T05:44:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:44 smithi098 ceph-mon[103124]: pgmap v12488: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 311 KiB/s wr, 12 op/s 2023-09-10T05:44:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23116.048809 secs 2023-09-10T05:44:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23115.395329 secs 2023-09-10T05:44:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:44 smithi170 ceph-mon[102832]: pgmap v12488: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 311 KiB/s wr, 12 op/s 2023-09-10T05:44:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23116.048809 secs 2023-09-10T05:44:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:46 smithi170 ceph-mon[102832]: pgmap v12489: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 311 KiB/s wr, 12 op/s 2023-09-10T05:44:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:46 smithi082 ceph-mon[98694]: pgmap v12489: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 311 KiB/s wr, 12 op/s 2023-09-10T05:44:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:46 smithi098 ceph-mon[103124]: pgmap v12489: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 311 KiB/s wr, 12 op/s 2023-09-10T05:44:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:48 smithi082 ceph-mon[98694]: pgmap v12490: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 558 KiB/s wr, 19 op/s 2023-09-10T05:44:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23119.423774 secs 2023-09-10T05:44:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:48 smithi098 ceph-mon[103124]: pgmap v12490: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 558 KiB/s wr, 19 op/s 2023-09-10T05:44:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23119.423774 secs 2023-09-10T05:44:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:48 smithi170 ceph-mon[102832]: pgmap v12490: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 558 KiB/s wr, 19 op/s 2023-09-10T05:44:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23119.423774 secs 2023-09-10T05:44:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23120.395389 secs 2023-09-10T05:44:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23121.048891 secs 2023-09-10T05:44:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23120.395389 secs 2023-09-10T05:44:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23121.048891 secs 2023-09-10T05:44:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23120.395389 secs 2023-09-10T05:44:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23121.048891 secs 2023-09-10T05:44:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:50 smithi098 ceph-mon[103124]: pgmap v12491: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 385 KiB/s wr, 12 op/s 2023-09-10T05:44:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:44:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:44:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:50 smithi170 ceph-mon[102832]: pgmap v12491: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 385 KiB/s wr, 12 op/s 2023-09-10T05:44:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:44:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:44:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:50 smithi082 ceph-mon[98694]: pgmap v12491: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 385 KiB/s wr, 12 op/s 2023-09-10T05:44:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:44:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:44:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:44:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:44:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:44:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:44:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:44:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:44:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:44:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:44:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:44:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:52 smithi170 ceph-mon[102832]: pgmap v12492: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 543 KiB/s wr, 19 op/s 2023-09-10T05:44:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:52 smithi082 ceph-mon[98694]: pgmap v12492: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 543 KiB/s wr, 19 op/s 2023-09-10T05:44:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:52 smithi098 ceph-mon[103124]: pgmap v12492: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 543 KiB/s wr, 19 op/s 2023-09-10T05:44:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23124.423863 secs 2023-09-10T05:44:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23124.423863 secs 2023-09-10T05:44:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23124.423863 secs 2023-09-10T05:44:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23125.395447 secs 2023-09-10T05:44:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:54 smithi170 ceph-mon[102832]: pgmap v12493: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 405 KiB/s wr, 12 op/s 2023-09-10T05:44:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23126.048954 secs 2023-09-10T05:44:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23125.395447 secs 2023-09-10T05:44:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:54 smithi082 ceph-mon[98694]: pgmap v12493: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 405 KiB/s wr, 12 op/s 2023-09-10T05:44:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23126.048954 secs 2023-09-10T05:44:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23125.395447 secs 2023-09-10T05:44:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:54 smithi098 ceph-mon[103124]: pgmap v12493: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 405 KiB/s wr, 12 op/s 2023-09-10T05:44:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23126.048954 secs 2023-09-10T05:44:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:56 smithi170 ceph-mon[102832]: pgmap v12494: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 405 KiB/s wr, 12 op/s 2023-09-10T05:44:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:44:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:56 smithi082 ceph-mon[98694]: pgmap v12494: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 405 KiB/s wr, 12 op/s 2023-09-10T05:44:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:44:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:56 smithi098 ceph-mon[103124]: pgmap v12494: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 405 KiB/s wr, 12 op/s 2023-09-10T05:44:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:44:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:44:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:44:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:44:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:44:57.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:44:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:44:57.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:44:57.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:44:57.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:44:57.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:44:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:44:57.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:44:57.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:44:57.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:44:57.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:44:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:58 smithi170 ceph-mon[102832]: pgmap v12495: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 583 KiB/s wr, 18 op/s 2023-09-10T05:44:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:44:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:44:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23129.423925 secs 2023-09-10T05:44:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:58 smithi082 ceph-mon[98694]: pgmap v12495: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 583 KiB/s wr, 18 op/s 2023-09-10T05:44:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:44:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:44:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23129.423925 secs 2023-09-10T05:44:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:58 smithi098 ceph-mon[103124]: pgmap v12495: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 583 KiB/s wr, 18 op/s 2023-09-10T05:44:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:44:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:44:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23129.423925 secs 2023-09-10T05:44:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23130.395494 secs 2023-09-10T05:44:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:44:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23131.049041 secs 2023-09-10T05:44:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23130.395494 secs 2023-09-10T05:44:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:44:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23131.049041 secs 2023-09-10T05:44:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23130.395494 secs 2023-09-10T05:44:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:44:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23131.049041 secs 2023-09-10T05:45:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:00 smithi170 ceph-mon[102832]: pgmap v12496: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 336 KiB/s wr, 11 op/s 2023-09-10T05:45:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:45:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:45:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:00 smithi082 ceph-mon[98694]: pgmap v12496: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 336 KiB/s wr, 11 op/s 2023-09-10T05:45:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:45:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:45:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:00 smithi098 ceph-mon[103124]: pgmap v12496: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 336 KiB/s wr, 11 op/s 2023-09-10T05:45:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:45:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:45:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:02 smithi170 ceph-mon[102832]: pgmap v12497: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 336 KiB/s wr, 17 op/s 2023-09-10T05:45:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:02 smithi082 ceph-mon[98694]: pgmap v12497: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 336 KiB/s wr, 17 op/s 2023-09-10T05:45:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:02 smithi098 ceph-mon[103124]: pgmap v12497: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 336 KiB/s wr, 17 op/s 2023-09-10T05:45:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23134.423987 secs 2023-09-10T05:45:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23134.423987 secs 2023-09-10T05:45:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23134.423987 secs 2023-09-10T05:45:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23135.395552 secs 2023-09-10T05:45:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:04 smithi170 ceph-mon[102832]: pgmap v12498: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T05:45:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23136.049107 secs 2023-09-10T05:45:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23135.395552 secs 2023-09-10T05:45:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:04 smithi082 ceph-mon[98694]: pgmap v12498: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T05:45:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23136.049107 secs 2023-09-10T05:45:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23135.395552 secs 2023-09-10T05:45:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:04 smithi098 ceph-mon[103124]: pgmap v12498: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T05:45:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23136.049107 secs 2023-09-10T05:45:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:06 smithi170 ceph-mon[102832]: pgmap v12499: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T05:45:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:06 smithi082 ceph-mon[98694]: pgmap v12499: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T05:45:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:06 smithi098 ceph-mon[103124]: pgmap v12499: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T05:45:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:08 smithi098 ceph-mon[103124]: pgmap v12500: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 288 KiB/s wr, 31 op/s 2023-09-10T05:45:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23139.424048 secs 2023-09-10T05:45:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:08 smithi170 ceph-mon[102832]: pgmap v12500: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 288 KiB/s wr, 31 op/s 2023-09-10T05:45:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23139.424048 secs 2023-09-10T05:45:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:08 smithi082 ceph-mon[98694]: pgmap v12500: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 288 KiB/s wr, 31 op/s 2023-09-10T05:45:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23139.424048 secs 2023-09-10T05:45:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23140.395610 secs 2023-09-10T05:45:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:09 smithi098 ceph-mon[103124]: osdmap e849: 12 total, 12 up, 12 in 2023-09-10T05:45:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23141.049173 secs 2023-09-10T05:45:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23140.395610 secs 2023-09-10T05:45:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:09 smithi170 ceph-mon[102832]: osdmap e849: 12 total, 12 up, 12 in 2023-09-10T05:45:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23141.049173 secs 2023-09-10T05:45:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23140.395610 secs 2023-09-10T05:45:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:09 smithi082 ceph-mon[98694]: osdmap e849: 12 total, 12 up, 12 in 2023-09-10T05:45:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23141.049173 secs 2023-09-10T05:45:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:10 smithi098 ceph-mon[103124]: pgmap v12502: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 132 KiB/s wr, 30 op/s 2023-09-10T05:45:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:10 smithi170 ceph-mon[102832]: pgmap v12502: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 132 KiB/s wr, 30 op/s 2023-09-10T05:45:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:10 smithi082 ceph-mon[98694]: pgmap v12502: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 132 KiB/s wr, 30 op/s 2023-09-10T05:45:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:12 smithi098 ceph-mon[103124]: pgmap v12503: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 153 KiB/s wr, 29 op/s 2023-09-10T05:45:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:12 smithi170 ceph-mon[102832]: pgmap v12503: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 153 KiB/s wr, 29 op/s 2023-09-10T05:45:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:12 smithi082 ceph-mon[98694]: pgmap v12503: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 153 KiB/s wr, 29 op/s 2023-09-10T05:45:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23144.424117 secs 2023-09-10T05:45:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23144.424117 secs 2023-09-10T05:45:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23144.424117 secs 2023-09-10T05:45:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23145.395669 secs 2023-09-10T05:45:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:14 smithi098 ceph-mon[103124]: pgmap v12504: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 153 KiB/s wr, 29 op/s 2023-09-10T05:45:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23146.049240 secs 2023-09-10T05:45:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23145.395669 secs 2023-09-10T05:45:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:14 smithi170 ceph-mon[102832]: pgmap v12504: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 153 KiB/s wr, 29 op/s 2023-09-10T05:45:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23146.049240 secs 2023-09-10T05:45:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23145.395669 secs 2023-09-10T05:45:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:14 smithi082 ceph-mon[98694]: pgmap v12504: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 153 KiB/s wr, 29 op/s 2023-09-10T05:45:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23146.049240 secs 2023-09-10T05:45:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:16 smithi170 ceph-mon[102832]: osdmap e850: 12 total, 12 up, 12 in 2023-09-10T05:45:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:16 smithi170 ceph-mon[102832]: pgmap v12506: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 7 op/s 2023-09-10T05:45:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:16 smithi082 ceph-mon[98694]: osdmap e850: 12 total, 12 up, 12 in 2023-09-10T05:45:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:16 smithi082 ceph-mon[98694]: pgmap v12506: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 7 op/s 2023-09-10T05:45:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:16 smithi098 ceph-mon[103124]: osdmap e850: 12 total, 12 up, 12 in 2023-09-10T05:45:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:16 smithi098 ceph-mon[103124]: pgmap v12506: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 7 op/s 2023-09-10T05:45:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:18 smithi098 ceph-mon[103124]: pgmap v12507: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 13 op/s 2023-09-10T05:45:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23149.424177 secs 2023-09-10T05:45:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:18 smithi170 ceph-mon[102832]: pgmap v12507: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 13 op/s 2023-09-10T05:45:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23149.424177 secs 2023-09-10T05:45:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:18 smithi082 ceph-mon[98694]: pgmap v12507: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 13 op/s 2023-09-10T05:45:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23149.424177 secs 2023-09-10T05:45:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23150.395726 secs 2023-09-10T05:45:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23151.049307 secs 2023-09-10T05:45:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23150.395726 secs 2023-09-10T05:45:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23151.049307 secs 2023-09-10T05:45:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23150.395726 secs 2023-09-10T05:45:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23151.049307 secs 2023-09-10T05:45:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:20 smithi098 ceph-mon[103124]: pgmap v12508: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:45:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:20 smithi170 ceph-mon[102832]: pgmap v12508: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:45:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:20 smithi082 ceph-mon[98694]: pgmap v12508: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:45:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:22 smithi098 ceph-mon[103124]: pgmap v12509: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T05:45:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:22 smithi170 ceph-mon[102832]: pgmap v12509: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T05:45:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:22 smithi082 ceph-mon[98694]: pgmap v12509: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T05:45:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23154.424236 secs 2023-09-10T05:45:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23154.424236 secs 2023-09-10T05:45:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23154.424236 secs 2023-09-10T05:45:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23155.395776 secs 2023-09-10T05:45:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:24 smithi098 ceph-mon[103124]: pgmap v12510: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T05:45:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23156.049372 secs 2023-09-10T05:45:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23155.395776 secs 2023-09-10T05:45:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:24 smithi170 ceph-mon[102832]: pgmap v12510: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T05:45:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23156.049372 secs 2023-09-10T05:45:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23155.395776 secs 2023-09-10T05:45:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:24 smithi082 ceph-mon[98694]: pgmap v12510: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T05:45:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23156.049372 secs 2023-09-10T05:45:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:26 smithi170 ceph-mon[102832]: pgmap v12511: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T05:45:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:26 smithi082 ceph-mon[98694]: pgmap v12511: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T05:45:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:26 smithi098 ceph-mon[103124]: pgmap v12511: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 91 KiB/s wr, 11 op/s 2023-09-10T05:45:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:28 smithi098 ceph-mon[103124]: pgmap v12512: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 77 KiB/s wr, 14 op/s 2023-09-10T05:45:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:28 smithi170 ceph-mon[102832]: pgmap v12512: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 77 KiB/s wr, 14 op/s 2023-09-10T05:45:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:28 smithi082 ceph-mon[98694]: pgmap v12512: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 77 KiB/s wr, 14 op/s 2023-09-10T05:45:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23160.395834 secs 2023-09-10T05:45:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23161.049440 secs 2023-09-10T05:45:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23160.395834 secs 2023-09-10T05:45:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23161.049440 secs 2023-09-10T05:45:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23160.395834 secs 2023-09-10T05:45:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23161.049440 secs 2023-09-10T05:45:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:30 smithi098 ceph-mon[103124]: pgmap v12513: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 9 op/s 2023-09-10T05:45:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:30 smithi170 ceph-mon[102832]: pgmap v12513: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 9 op/s 2023-09-10T05:45:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:30 smithi082 ceph-mon[98694]: pgmap v12513: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 9 op/s 2023-09-10T05:45:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:45:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:45:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:45:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:32 smithi098 ceph-mon[103124]: pgmap v12514: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 186 KiB/s wr, 15 op/s 2023-09-10T05:45:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:45:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:45:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:45:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:32 smithi170 ceph-mon[102832]: pgmap v12514: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 186 KiB/s wr, 15 op/s 2023-09-10T05:45:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:45:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:45:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:45:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:32 smithi082 ceph-mon[98694]: pgmap v12514: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 186 KiB/s wr, 15 op/s 2023-09-10T05:45:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:45:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:45:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:45:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23165.395894 secs 2023-09-10T05:45:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:34 smithi098 ceph-mon[103124]: pgmap v12515: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T05:45:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23166.049572 secs 2023-09-10T05:45:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23165.395894 secs 2023-09-10T05:45:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:34 smithi170 ceph-mon[102832]: pgmap v12515: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T05:45:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23166.049572 secs 2023-09-10T05:45:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23165.395894 secs 2023-09-10T05:45:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:34 smithi082 ceph-mon[98694]: pgmap v12515: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T05:45:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23166.049572 secs 2023-09-10T05:45:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:36 smithi170 ceph-mon[102832]: pgmap v12516: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T05:45:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:36 smithi082 ceph-mon[98694]: pgmap v12516: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T05:45:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:36 smithi098 ceph-mon[103124]: pgmap v12516: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T05:45:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:45:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:45:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:38 smithi170 ceph-mon[102832]: pgmap v12517: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 284 KiB/s wr, 16 op/s 2023-09-10T05:45:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:45:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:45:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:38 smithi082 ceph-mon[98694]: pgmap v12517: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 284 KiB/s wr, 16 op/s 2023-09-10T05:45:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:45:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:45:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:38 smithi098 ceph-mon[103124]: pgmap v12517: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 284 KiB/s wr, 16 op/s 2023-09-10T05:45:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23170.395954 secs 2023-09-10T05:45:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23170.395954 secs 2023-09-10T05:45:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23170.395954 secs 2023-09-10T05:45:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23171.049642 secs 2023-09-10T05:45:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:45:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:45:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:40 smithi170 ceph-mon[102832]: pgmap v12518: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 12 op/s 2023-09-10T05:45:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23171.049642 secs 2023-09-10T05:45:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:45:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:45:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:40 smithi082 ceph-mon[98694]: pgmap v12518: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 12 op/s 2023-09-10T05:45:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23171.049642 secs 2023-09-10T05:45:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:45:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:45:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:40 smithi098 ceph-mon[103124]: pgmap v12518: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 12 op/s 2023-09-10T05:45:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:42 smithi098 ceph-mon[103124]: pgmap v12519: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 421 KiB/s wr, 18 op/s 2023-09-10T05:45:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:42 smithi170 ceph-mon[102832]: pgmap v12519: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 421 KiB/s wr, 18 op/s 2023-09-10T05:45:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:42 smithi082 ceph-mon[98694]: pgmap v12519: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 421 KiB/s wr, 18 op/s 2023-09-10T05:45:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23175.396032 secs 2023-09-10T05:45:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:44 smithi098 ceph-mon[103124]: pgmap v12520: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 312 KiB/s wr, 12 op/s 2023-09-10T05:45:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23176.049706 secs 2023-09-10T05:45:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23175.396032 secs 2023-09-10T05:45:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:44 smithi170 ceph-mon[102832]: pgmap v12520: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 312 KiB/s wr, 12 op/s 2023-09-10T05:45:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23176.049706 secs 2023-09-10T05:45:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23175.396032 secs 2023-09-10T05:45:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:44 smithi082 ceph-mon[98694]: pgmap v12520: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 312 KiB/s wr, 12 op/s 2023-09-10T05:45:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23176.049706 secs 2023-09-10T05:45:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:46 smithi170 ceph-mon[102832]: pgmap v12521: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 312 KiB/s wr, 12 op/s 2023-09-10T05:45:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:46 smithi082 ceph-mon[98694]: pgmap v12521: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 312 KiB/s wr, 12 op/s 2023-09-10T05:45:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:46 smithi098 ceph-mon[103124]: pgmap v12521: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 312 KiB/s wr, 12 op/s 2023-09-10T05:45:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:48 smithi098 ceph-mon[103124]: pgmap v12522: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 468 KiB/s wr, 18 op/s 2023-09-10T05:45:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:48 smithi170 ceph-mon[102832]: pgmap v12522: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 468 KiB/s wr, 18 op/s 2023-09-10T05:45:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:48 smithi082 ceph-mon[98694]: pgmap v12522: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 468 KiB/s wr, 18 op/s 2023-09-10T05:45:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23180.396104 secs 2023-09-10T05:45:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23181.049770 secs 2023-09-10T05:45:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23180.396104 secs 2023-09-10T05:45:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23181.049770 secs 2023-09-10T05:45:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23180.396104 secs 2023-09-10T05:45:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23181.049770 secs 2023-09-10T05:45:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:50 smithi098 ceph-mon[103124]: pgmap v12523: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T05:45:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:45:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:45:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:50 smithi170 ceph-mon[102832]: pgmap v12523: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T05:45:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:45:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:45:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:50 smithi082 ceph-mon[98694]: pgmap v12523: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 294 KiB/s wr, 12 op/s 2023-09-10T05:45:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:45:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:45:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:52 smithi098 ceph-mon[103124]: pgmap v12524: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 459 KiB/s wr, 18 op/s 2023-09-10T05:45:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:45:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:45:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:52 smithi170 ceph-mon[102832]: pgmap v12524: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 459 KiB/s wr, 18 op/s 2023-09-10T05:45:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:45:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:45:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:52 smithi082 ceph-mon[98694]: pgmap v12524: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 459 KiB/s wr, 18 op/s 2023-09-10T05:45:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:45:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:45:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23184.424672 secs 2023-09-10T05:45:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23184.424672 secs 2023-09-10T05:45:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23184.424672 secs 2023-09-10T05:45:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23185.396164 secs 2023-09-10T05:45:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:54 smithi098 ceph-mon[103124]: pgmap v12525: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 322 KiB/s wr, 12 op/s 2023-09-10T05:45:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23186.049838 secs 2023-09-10T05:45:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23185.396164 secs 2023-09-10T05:45:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:54 smithi170 ceph-mon[102832]: pgmap v12525: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 322 KiB/s wr, 12 op/s 2023-09-10T05:45:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23186.049838 secs 2023-09-10T05:45:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23185.396164 secs 2023-09-10T05:45:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:54 smithi082 ceph-mon[98694]: pgmap v12525: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 322 KiB/s wr, 12 op/s 2023-09-10T05:45:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23186.049838 secs 2023-09-10T05:45:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:56 smithi170 ceph-mon[102832]: pgmap v12526: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 322 KiB/s wr, 12 op/s 2023-09-10T05:45:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:45:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:45:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:45:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:56 smithi082 ceph-mon[98694]: pgmap v12526: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 322 KiB/s wr, 12 op/s 2023-09-10T05:45:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:45:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:45:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:45:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:56 smithi098 ceph-mon[103124]: pgmap v12526: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 322 KiB/s wr, 12 op/s 2023-09-10T05:45:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:45:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:45:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:45:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:45:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:45:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:45:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:45:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:45:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:58 smithi170 ceph-mon[102832]: pgmap v12527: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 500 KiB/s wr, 18 op/s 2023-09-10T05:45:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:45:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:45:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:45:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:45:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23189.424791 secs 2023-09-10T05:45:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:45:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:45:58.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:45:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:45:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:45:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:58 smithi082 ceph-mon[98694]: pgmap v12527: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 500 KiB/s wr, 18 op/s 2023-09-10T05:45:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:45:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:45:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:45:58.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:45:58.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23189.424791 secs 2023-09-10T05:45:58.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:45:58.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:45:58.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:45:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:45:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:45:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:58 smithi098 ceph-mon[103124]: pgmap v12527: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 500 KiB/s wr, 18 op/s 2023-09-10T05:45:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:45:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:45:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:45:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:45:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23189.424791 secs 2023-09-10T05:45:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:45:58.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:45:58.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:45:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:45:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23190.396223 secs 2023-09-10T05:45:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:45:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23190.396223 secs 2023-09-10T05:45:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:45:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23190.396223 secs 2023-09-10T05:46:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23191.049902 secs 2023-09-10T05:46:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:46:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:46:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:00 smithi170 ceph-mon[102832]: pgmap v12528: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 343 KiB/s wr, 11 op/s 2023-09-10T05:46:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23191.049902 secs 2023-09-10T05:46:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:46:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:46:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:00 smithi082 ceph-mon[98694]: pgmap v12528: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 343 KiB/s wr, 11 op/s 2023-09-10T05:46:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23191.049902 secs 2023-09-10T05:46:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:46:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:46:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:00 smithi098 ceph-mon[103124]: pgmap v12528: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 343 KiB/s wr, 11 op/s 2023-09-10T05:46:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:02 smithi098 ceph-mon[103124]: pgmap v12529: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 459 KiB/s wr, 28 op/s 2023-09-10T05:46:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:02 smithi170 ceph-mon[102832]: pgmap v12529: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 459 KiB/s wr, 28 op/s 2023-09-10T05:46:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:02 smithi082 ceph-mon[98694]: pgmap v12529: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 459 KiB/s wr, 28 op/s 2023-09-10T05:46:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23194.424849 secs 2023-09-10T05:46:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23194.424849 secs 2023-09-10T05:46:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23194.424849 secs 2023-09-10T05:46:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23195.396283 secs 2023-09-10T05:46:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:04 smithi098 ceph-mon[103124]: pgmap v12530: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 294 KiB/s wr, 22 op/s 2023-09-10T05:46:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23196.049970 secs 2023-09-10T05:46:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23195.396283 secs 2023-09-10T05:46:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:04 smithi170 ceph-mon[102832]: pgmap v12530: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 294 KiB/s wr, 22 op/s 2023-09-10T05:46:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23196.049970 secs 2023-09-10T05:46:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23195.396283 secs 2023-09-10T05:46:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:04 smithi082 ceph-mon[98694]: pgmap v12530: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 294 KiB/s wr, 22 op/s 2023-09-10T05:46:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23196.049970 secs 2023-09-10T05:46:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:06 smithi170 ceph-mon[102832]: pgmap v12531: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 294 KiB/s wr, 22 op/s 2023-09-10T05:46:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:06 smithi082 ceph-mon[98694]: pgmap v12531: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 294 KiB/s wr, 22 op/s 2023-09-10T05:46:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:06 smithi098 ceph-mon[103124]: pgmap v12531: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 294 KiB/s wr, 22 op/s 2023-09-10T05:46:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:08 smithi098 ceph-mon[103124]: pgmap v12532: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 393 KiB/s wr, 31 op/s 2023-09-10T05:46:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23199.424921 secs 2023-09-10T05:46:08.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:08 smithi170 ceph-mon[102832]: pgmap v12532: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 393 KiB/s wr, 31 op/s 2023-09-10T05:46:08.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23199.424921 secs 2023-09-10T05:46:08.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:08 smithi082 ceph-mon[98694]: pgmap v12532: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 393 KiB/s wr, 31 op/s 2023-09-10T05:46:08.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23199.424921 secs 2023-09-10T05:46:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23200.396341 secs 2023-09-10T05:46:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:09 smithi098 ceph-mon[103124]: osdmap e851: 12 total, 12 up, 12 in 2023-09-10T05:46:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23201.050033 secs 2023-09-10T05:46:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23200.396341 secs 2023-09-10T05:46:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:09 smithi170 ceph-mon[102832]: osdmap e851: 12 total, 12 up, 12 in 2023-09-10T05:46:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23201.050033 secs 2023-09-10T05:46:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23200.396341 secs 2023-09-10T05:46:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:09 smithi082 ceph-mon[98694]: osdmap e851: 12 total, 12 up, 12 in 2023-09-10T05:46:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23201.050033 secs 2023-09-10T05:46:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:10 smithi098 ceph-mon[103124]: pgmap v12534: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 258 KiB/s wr, 30 op/s 2023-09-10T05:46:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:10 smithi170 ceph-mon[102832]: pgmap v12534: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 258 KiB/s wr, 30 op/s 2023-09-10T05:46:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:10 smithi082 ceph-mon[98694]: pgmap v12534: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 258 KiB/s wr, 30 op/s 2023-09-10T05:46:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:12 smithi098 ceph-mon[103124]: pgmap v12535: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 141 KiB/s wr, 16 op/s 2023-09-10T05:46:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:12 smithi170 ceph-mon[102832]: pgmap v12535: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 141 KiB/s wr, 16 op/s 2023-09-10T05:46:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:12 smithi082 ceph-mon[98694]: pgmap v12535: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 141 KiB/s wr, 16 op/s 2023-09-10T05:46:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23204.424990 secs 2023-09-10T05:46:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23204.424990 secs 2023-09-10T05:46:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23204.424990 secs 2023-09-10T05:46:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23205.396416 secs 2023-09-10T05:46:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:14 smithi170 ceph-mon[102832]: pgmap v12536: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 141 KiB/s wr, 16 op/s 2023-09-10T05:46:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23206.050097 secs 2023-09-10T05:46:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23205.396416 secs 2023-09-10T05:46:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:14 smithi082 ceph-mon[98694]: pgmap v12536: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 141 KiB/s wr, 16 op/s 2023-09-10T05:46:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23206.050097 secs 2023-09-10T05:46:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23205.396416 secs 2023-09-10T05:46:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:14 smithi098 ceph-mon[103124]: pgmap v12536: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 141 KiB/s wr, 16 op/s 2023-09-10T05:46:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23206.050097 secs 2023-09-10T05:46:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:16 smithi170 ceph-mon[102832]: osdmap e852: 12 total, 12 up, 12 in 2023-09-10T05:46:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:16 smithi170 ceph-mon[102832]: pgmap v12538: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 7 op/s 2023-09-10T05:46:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:16 smithi082 ceph-mon[98694]: osdmap e852: 12 total, 12 up, 12 in 2023-09-10T05:46:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:16 smithi082 ceph-mon[98694]: pgmap v12538: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 7 op/s 2023-09-10T05:46:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:16 smithi098 ceph-mon[103124]: osdmap e852: 12 total, 12 up, 12 in 2023-09-10T05:46:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:16 smithi098 ceph-mon[103124]: pgmap v12538: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 7 op/s 2023-09-10T05:46:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:18 smithi098 ceph-mon[103124]: pgmap v12539: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 13 op/s 2023-09-10T05:46:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23209.425053 secs 2023-09-10T05:46:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:18 smithi170 ceph-mon[102832]: pgmap v12539: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 13 op/s 2023-09-10T05:46:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23209.425053 secs 2023-09-10T05:46:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:18 smithi082 ceph-mon[98694]: pgmap v12539: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 13 op/s 2023-09-10T05:46:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23209.425053 secs 2023-09-10T05:46:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23210.396486 secs 2023-09-10T05:46:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23211.050162 secs 2023-09-10T05:46:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23210.396486 secs 2023-09-10T05:46:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23211.050162 secs 2023-09-10T05:46:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23210.396486 secs 2023-09-10T05:46:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23211.050162 secs 2023-09-10T05:46:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:20 smithi098 ceph-mon[103124]: pgmap v12540: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T05:46:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:20 smithi170 ceph-mon[102832]: pgmap v12540: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T05:46:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:20 smithi082 ceph-mon[98694]: pgmap v12540: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T05:46:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:22 smithi098 ceph-mon[103124]: pgmap v12541: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T05:46:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:22 smithi170 ceph-mon[102832]: pgmap v12541: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T05:46:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:22 smithi082 ceph-mon[98694]: pgmap v12541: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T05:46:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23214.425112 secs 2023-09-10T05:46:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23214.425112 secs 2023-09-10T05:46:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23214.425112 secs 2023-09-10T05:46:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23215.396493 secs 2023-09-10T05:46:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:24 smithi098 ceph-mon[103124]: pgmap v12542: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T05:46:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23216.050226 secs 2023-09-10T05:46:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23215.396493 secs 2023-09-10T05:46:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:24 smithi170 ceph-mon[102832]: pgmap v12542: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T05:46:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23216.050226 secs 2023-09-10T05:46:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23215.396493 secs 2023-09-10T05:46:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:24 smithi082 ceph-mon[98694]: pgmap v12542: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T05:46:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23216.050226 secs 2023-09-10T05:46:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:26 smithi170 ceph-mon[102832]: pgmap v12543: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T05:46:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:26 smithi082 ceph-mon[98694]: pgmap v12543: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T05:46:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:26 smithi098 ceph-mon[103124]: pgmap v12543: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T05:46:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:28 smithi098 ceph-mon[103124]: pgmap v12544: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 126 KiB/s wr, 14 op/s 2023-09-10T05:46:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:28 smithi170 ceph-mon[102832]: pgmap v12544: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 126 KiB/s wr, 14 op/s 2023-09-10T05:46:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:28 smithi082 ceph-mon[98694]: pgmap v12544: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 126 KiB/s wr, 14 op/s 2023-09-10T05:46:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23220.396552 secs 2023-09-10T05:46:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23221.050289 secs 2023-09-10T05:46:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23220.396552 secs 2023-09-10T05:46:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23221.050289 secs 2023-09-10T05:46:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23220.396552 secs 2023-09-10T05:46:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23221.050289 secs 2023-09-10T05:46:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:30 smithi098 ceph-mon[103124]: pgmap v12545: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 9 op/s 2023-09-10T05:46:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:30 smithi170 ceph-mon[102832]: pgmap v12545: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 9 op/s 2023-09-10T05:46:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:30 smithi082 ceph-mon[98694]: pgmap v12545: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 9 op/s 2023-09-10T05:46:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:46:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:46:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:46:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:32 smithi098 ceph-mon[103124]: pgmap v12546: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 202 KiB/s wr, 15 op/s 2023-09-10T05:46:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:46:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:46:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:46:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:32 smithi170 ceph-mon[102832]: pgmap v12546: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 202 KiB/s wr, 15 op/s 2023-09-10T05:46:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:46:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:46:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:46:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:32 smithi082 ceph-mon[98694]: pgmap v12546: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 202 KiB/s wr, 15 op/s 2023-09-10T05:46:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:46:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:46:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:46:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23225.396610 secs 2023-09-10T05:46:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:34 smithi170 ceph-mon[102832]: pgmap v12547: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T05:46:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:46:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:46:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23226.050355 secs 2023-09-10T05:46:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23225.396610 secs 2023-09-10T05:46:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:34 smithi082 ceph-mon[98694]: pgmap v12547: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T05:46:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:46:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:46:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23226.050355 secs 2023-09-10T05:46:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23225.396610 secs 2023-09-10T05:46:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:34 smithi098 ceph-mon[103124]: pgmap v12547: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T05:46:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:46:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:46:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23226.050355 secs 2023-09-10T05:46:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:36 smithi170 ceph-mon[102832]: pgmap v12548: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T05:46:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:36 smithi082 ceph-mon[98694]: pgmap v12548: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T05:46:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:36 smithi098 ceph-mon[103124]: pgmap v12548: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T05:46:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:38 smithi098 ceph-mon[103124]: pgmap v12549: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 299 KiB/s wr, 17 op/s 2023-09-10T05:46:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:38 smithi170 ceph-mon[102832]: pgmap v12549: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 299 KiB/s wr, 17 op/s 2023-09-10T05:46:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:38 smithi082 ceph-mon[98694]: pgmap v12549: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 299 KiB/s wr, 17 op/s 2023-09-10T05:46:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23230.396671 secs 2023-09-10T05:46:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23231.050419 secs 2023-09-10T05:46:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23230.396671 secs 2023-09-10T05:46:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23231.050419 secs 2023-09-10T05:46:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23230.396671 secs 2023-09-10T05:46:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23231.050419 secs 2023-09-10T05:46:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:40 smithi098 ceph-mon[103124]: pgmap v12550: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 299 KiB/s wr, 13 op/s 2023-09-10T05:46:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:40 smithi170 ceph-mon[102832]: pgmap v12550: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 299 KiB/s wr, 13 op/s 2023-09-10T05:46:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:40 smithi082 ceph-mon[98694]: pgmap v12550: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 299 KiB/s wr, 13 op/s 2023-09-10T05:46:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:42 smithi098 ceph-mon[103124]: pgmap v12551: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 490 KiB/s wr, 19 op/s 2023-09-10T05:46:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:42 smithi170 ceph-mon[102832]: pgmap v12551: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 490 KiB/s wr, 19 op/s 2023-09-10T05:46:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:42 smithi082 ceph-mon[98694]: pgmap v12551: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 490 KiB/s wr, 19 op/s 2023-09-10T05:46:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23235.396733 secs 2023-09-10T05:46:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:44 smithi170 ceph-mon[102832]: pgmap v12552: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 366 KiB/s wr, 13 op/s 2023-09-10T05:46:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:46:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:46:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23236.050489 secs 2023-09-10T05:46:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23235.396733 secs 2023-09-10T05:46:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:44 smithi082 ceph-mon[98694]: pgmap v12552: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 366 KiB/s wr, 13 op/s 2023-09-10T05:46:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:46:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:46:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23236.050489 secs 2023-09-10T05:46:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23235.396733 secs 2023-09-10T05:46:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:44 smithi098 ceph-mon[103124]: pgmap v12552: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 366 KiB/s wr, 13 op/s 2023-09-10T05:46:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:46:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:46:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23236.050489 secs 2023-09-10T05:46:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:46 smithi170 ceph-mon[102832]: pgmap v12553: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 366 KiB/s wr, 13 op/s 2023-09-10T05:46:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:46:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:46:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:46 smithi082 ceph-mon[98694]: pgmap v12553: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 366 KiB/s wr, 13 op/s 2023-09-10T05:46:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:46:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:46:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:46 smithi098 ceph-mon[103124]: pgmap v12553: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 366 KiB/s wr, 13 op/s 2023-09-10T05:46:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:46:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:46:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:48 smithi170 ceph-mon[102832]: pgmap v12554: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 564 KiB/s wr, 20 op/s 2023-09-10T05:46:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23239.425462 secs 2023-09-10T05:46:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:48 smithi082 ceph-mon[98694]: pgmap v12554: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 564 KiB/s wr, 20 op/s 2023-09-10T05:46:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23239.425462 secs 2023-09-10T05:46:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:48 smithi098 ceph-mon[103124]: pgmap v12554: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 564 KiB/s wr, 20 op/s 2023-09-10T05:46:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23239.425462 secs 2023-09-10T05:46:49.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23240.396850 secs 2023-09-10T05:46:49.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23241.050555 secs 2023-09-10T05:46:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23240.396850 secs 2023-09-10T05:46:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23241.050555 secs 2023-09-10T05:46:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23240.396850 secs 2023-09-10T05:46:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23241.050555 secs 2023-09-10T05:46:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:50 smithi170 ceph-mon[102832]: pgmap v12555: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 389 KiB/s wr, 13 op/s 2023-09-10T05:46:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:50 smithi082 ceph-mon[98694]: pgmap v12555: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 389 KiB/s wr, 13 op/s 2023-09-10T05:46:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:50 smithi098 ceph-mon[103124]: pgmap v12555: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 389 KiB/s wr, 13 op/s 2023-09-10T05:46:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:46:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:46:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:46:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:46:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:46:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:46:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:46:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:46:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:46:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:52 smithi170 ceph-mon[102832]: pgmap v12556: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 536 KiB/s wr, 19 op/s 2023-09-10T05:46:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:52 smithi082 ceph-mon[98694]: pgmap v12556: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 536 KiB/s wr, 19 op/s 2023-09-10T05:46:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:52 smithi098 ceph-mon[103124]: pgmap v12556: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 536 KiB/s wr, 19 op/s 2023-09-10T05:46:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23244.425524 secs 2023-09-10T05:46:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23244.425524 secs 2023-09-10T05:46:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23244.425524 secs 2023-09-10T05:46:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23245.396914 secs 2023-09-10T05:46:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:54 smithi170 ceph-mon[102832]: pgmap v12557: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 344 KiB/s wr, 12 op/s 2023-09-10T05:46:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23246.050619 secs 2023-09-10T05:46:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23245.396914 secs 2023-09-10T05:46:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:54 smithi082 ceph-mon[98694]: pgmap v12557: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 344 KiB/s wr, 12 op/s 2023-09-10T05:46:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23246.050619 secs 2023-09-10T05:46:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23245.396914 secs 2023-09-10T05:46:54.805 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:54 smithi098 ceph-mon[103124]: pgmap v12557: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 344 KiB/s wr, 12 op/s 2023-09-10T05:46:54.805 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23246.050619 secs 2023-09-10T05:46:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:56 smithi170 ceph-mon[102832]: pgmap v12558: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 344 KiB/s wr, 12 op/s 2023-09-10T05:46:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:56 smithi082 ceph-mon[98694]: pgmap v12558: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 344 KiB/s wr, 12 op/s 2023-09-10T05:46:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:56 smithi098 ceph-mon[103124]: pgmap v12558: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 344 KiB/s wr, 12 op/s 2023-09-10T05:46:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:46:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:46:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:58 smithi170 ceph-mon[102832]: pgmap v12559: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 450 KiB/s wr, 18 op/s 2023-09-10T05:46:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23249.425585 secs 2023-09-10T05:46:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:46:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:46:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:58 smithi082 ceph-mon[98694]: pgmap v12559: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 450 KiB/s wr, 18 op/s 2023-09-10T05:46:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23249.425585 secs 2023-09-10T05:46:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:46:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:46:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:58 smithi098 ceph-mon[103124]: pgmap v12559: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 450 KiB/s wr, 18 op/s 2023-09-10T05:46:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23249.425585 secs 2023-09-10T05:46:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:46:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23250.396975 secs 2023-09-10T05:46:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:46:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:46:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:46:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:46:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:46:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23250.396975 secs 2023-09-10T05:46:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:46:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:46:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:46:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:46:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:46:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23250.396975 secs 2023-09-10T05:46:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:46:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:46:59.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:46:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:47:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23251.050684 secs 2023-09-10T05:47:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:00 smithi170 ceph-mon[102832]: pgmap v12560: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 253 KiB/s wr, 11 op/s 2023-09-10T05:47:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:47:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:47:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23251.050684 secs 2023-09-10T05:47:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:00 smithi082 ceph-mon[98694]: pgmap v12560: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 253 KiB/s wr, 11 op/s 2023-09-10T05:47:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:47:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:47:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23251.050684 secs 2023-09-10T05:47:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:00 smithi098 ceph-mon[103124]: pgmap v12560: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 253 KiB/s wr, 11 op/s 2023-09-10T05:47:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:47:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:47:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:02 smithi098 ceph-mon[103124]: pgmap v12561: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 327 KiB/s wr, 16 op/s 2023-09-10T05:47:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:02 smithi170 ceph-mon[102832]: pgmap v12561: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 327 KiB/s wr, 16 op/s 2023-09-10T05:47:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:02 smithi082 ceph-mon[98694]: pgmap v12561: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 327 KiB/s wr, 16 op/s 2023-09-10T05:47:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23254.425645 secs 2023-09-10T05:47:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23254.425645 secs 2023-09-10T05:47:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23254.425645 secs 2023-09-10T05:47:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23255.397033 secs 2023-09-10T05:47:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:04 smithi098 ceph-mon[103124]: pgmap v12562: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 10 op/s 2023-09-10T05:47:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23256.050754 secs 2023-09-10T05:47:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23255.397033 secs 2023-09-10T05:47:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:04 smithi170 ceph-mon[102832]: pgmap v12562: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 10 op/s 2023-09-10T05:47:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23256.050754 secs 2023-09-10T05:47:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23255.397033 secs 2023-09-10T05:47:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:04 smithi082 ceph-mon[98694]: pgmap v12562: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 10 op/s 2023-09-10T05:47:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23256.050754 secs 2023-09-10T05:47:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:06 smithi170 ceph-mon[102832]: pgmap v12563: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 10 op/s 2023-09-10T05:47:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:06 smithi082 ceph-mon[98694]: pgmap v12563: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 10 op/s 2023-09-10T05:47:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:06 smithi098 ceph-mon[103124]: pgmap v12563: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 10 op/s 2023-09-10T05:47:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:08 smithi098 ceph-mon[103124]: pgmap v12564: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 279 KiB/s wr, 30 op/s 2023-09-10T05:47:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23259.425706 secs 2023-09-10T05:47:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:08 smithi170 ceph-mon[102832]: pgmap v12564: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 279 KiB/s wr, 30 op/s 2023-09-10T05:47:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23259.425706 secs 2023-09-10T05:47:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:08 smithi082 ceph-mon[98694]: pgmap v12564: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 279 KiB/s wr, 30 op/s 2023-09-10T05:47:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23259.425706 secs 2023-09-10T05:47:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23260.397120 secs 2023-09-10T05:47:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:09 smithi170 ceph-mon[102832]: osdmap e853: 12 total, 12 up, 12 in 2023-09-10T05:47:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23261.050902 secs 2023-09-10T05:47:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23260.397120 secs 2023-09-10T05:47:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:09 smithi082 ceph-mon[98694]: osdmap e853: 12 total, 12 up, 12 in 2023-09-10T05:47:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23261.050902 secs 2023-09-10T05:47:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23260.397120 secs 2023-09-10T05:47:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:09 smithi098 ceph-mon[103124]: osdmap e853: 12 total, 12 up, 12 in 2023-09-10T05:47:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23261.050902 secs 2023-09-10T05:47:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:10 smithi170 ceph-mon[102832]: pgmap v12566: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 208 KiB/s wr, 30 op/s 2023-09-10T05:47:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:10 smithi082 ceph-mon[98694]: pgmap v12566: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 208 KiB/s wr, 30 op/s 2023-09-10T05:47:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:10 smithi098 ceph-mon[103124]: pgmap v12566: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 208 KiB/s wr, 30 op/s 2023-09-10T05:47:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:12 smithi170 ceph-mon[102832]: pgmap v12567: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 120 KiB/s wr, 30 op/s 2023-09-10T05:47:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:12 smithi082 ceph-mon[98694]: pgmap v12567: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 120 KiB/s wr, 30 op/s 2023-09-10T05:47:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:12 smithi098 ceph-mon[103124]: pgmap v12567: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 120 KiB/s wr, 30 op/s 2023-09-10T05:47:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23264.425808 secs 2023-09-10T05:47:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23264.425808 secs 2023-09-10T05:47:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23264.425808 secs 2023-09-10T05:47:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23265.397181 secs 2023-09-10T05:47:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:14 smithi170 ceph-mon[102832]: pgmap v12568: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 120 KiB/s wr, 30 op/s 2023-09-10T05:47:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23266.050965 secs 2023-09-10T05:47:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23265.397181 secs 2023-09-10T05:47:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:14 smithi082 ceph-mon[98694]: pgmap v12568: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 120 KiB/s wr, 30 op/s 2023-09-10T05:47:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23266.050965 secs 2023-09-10T05:47:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23265.397181 secs 2023-09-10T05:47:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:14 smithi098 ceph-mon[103124]: pgmap v12568: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 120 KiB/s wr, 30 op/s 2023-09-10T05:47:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23266.050965 secs 2023-09-10T05:47:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:16 smithi170 ceph-mon[102832]: osdmap e854: 12 total, 12 up, 12 in 2023-09-10T05:47:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:16 smithi170 ceph-mon[102832]: pgmap v12570: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-09-10T05:47:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:16 smithi082 ceph-mon[98694]: osdmap e854: 12 total, 12 up, 12 in 2023-09-10T05:47:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:16 smithi082 ceph-mon[98694]: pgmap v12570: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-09-10T05:47:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:16 smithi098 ceph-mon[103124]: osdmap e854: 12 total, 12 up, 12 in 2023-09-10T05:47:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:16 smithi098 ceph-mon[103124]: pgmap v12570: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-09-10T05:47:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:18 smithi098 ceph-mon[103124]: pgmap v12571: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2023-09-10T05:47:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23269.425892 secs 2023-09-10T05:47:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:18 smithi170 ceph-mon[102832]: pgmap v12571: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2023-09-10T05:47:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23269.425892 secs 2023-09-10T05:47:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:18 smithi082 ceph-mon[98694]: pgmap v12571: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2023-09-10T05:47:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23269.425892 secs 2023-09-10T05:47:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23270.397239 secs 2023-09-10T05:47:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23271.051030 secs 2023-09-10T05:47:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23270.397239 secs 2023-09-10T05:47:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23271.051030 secs 2023-09-10T05:47:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23270.397239 secs 2023-09-10T05:47:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23271.051030 secs 2023-09-10T05:47:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:20 smithi170 ceph-mon[102832]: pgmap v12572: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-09-10T05:47:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:20 smithi082 ceph-mon[98694]: pgmap v12572: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-09-10T05:47:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:20 smithi098 ceph-mon[103124]: pgmap v12572: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-09-10T05:47:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:22 smithi170 ceph-mon[102832]: pgmap v12573: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T05:47:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:22 smithi082 ceph-mon[98694]: pgmap v12573: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T05:47:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:22 smithi098 ceph-mon[103124]: pgmap v12573: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T05:47:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23274.425952 secs 2023-09-10T05:47:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23274.425952 secs 2023-09-10T05:47:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23274.425952 secs 2023-09-10T05:47:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23275.397298 secs 2023-09-10T05:47:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:24 smithi170 ceph-mon[102832]: pgmap v12574: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T05:47:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23276.051096 secs 2023-09-10T05:47:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23275.397298 secs 2023-09-10T05:47:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:24 smithi082 ceph-mon[98694]: pgmap v12574: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T05:47:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23276.051096 secs 2023-09-10T05:47:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23275.397298 secs 2023-09-10T05:47:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:24 smithi098 ceph-mon[103124]: pgmap v12574: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 11 op/s 2023-09-10T05:47:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23276.051096 secs 2023-09-10T05:47:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:26 smithi170 ceph-mon[102832]: pgmap v12575: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T05:47:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:26 smithi082 ceph-mon[98694]: pgmap v12575: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T05:47:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:26 smithi098 ceph-mon[103124]: pgmap v12575: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 11 op/s 2023-09-10T05:47:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:28 smithi170 ceph-mon[102832]: pgmap v12576: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 78 KiB/s wr, 14 op/s 2023-09-10T05:47:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:28 smithi082 ceph-mon[98694]: pgmap v12576: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 78 KiB/s wr, 14 op/s 2023-09-10T05:47:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:28 smithi098 ceph-mon[103124]: pgmap v12576: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 78 KiB/s wr, 14 op/s 2023-09-10T05:47:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23280.397358 secs 2023-09-10T05:47:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23281.051164 secs 2023-09-10T05:47:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23280.397358 secs 2023-09-10T05:47:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23281.051164 secs 2023-09-10T05:47:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23280.397358 secs 2023-09-10T05:47:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23281.051164 secs 2023-09-10T05:47:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:30 smithi170 ceph-mon[102832]: pgmap v12577: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 9 op/s 2023-09-10T05:47:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:30 smithi082 ceph-mon[98694]: pgmap v12577: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 9 op/s 2023-09-10T05:47:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:30 smithi098 ceph-mon[103124]: pgmap v12577: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 9 op/s 2023-09-10T05:47:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:47:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:47:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:47:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:32 smithi170 ceph-mon[102832]: pgmap v12578: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 188 KiB/s wr, 15 op/s 2023-09-10T05:47:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:47:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:47:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:47:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:32 smithi082 ceph-mon[98694]: pgmap v12578: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 188 KiB/s wr, 15 op/s 2023-09-10T05:47:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:47:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:47:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:47:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:32 smithi098 ceph-mon[103124]: pgmap v12578: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 188 KiB/s wr, 15 op/s 2023-09-10T05:47:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:47:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:47:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:47:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23285.397416 secs 2023-09-10T05:47:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:34 smithi170 ceph-mon[102832]: pgmap v12579: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 111 KiB/s wr, 10 op/s 2023-09-10T05:47:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:47:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:47:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23286.051230 secs 2023-09-10T05:47:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23285.397416 secs 2023-09-10T05:47:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:34 smithi082 ceph-mon[98694]: pgmap v12579: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 111 KiB/s wr, 10 op/s 2023-09-10T05:47:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:47:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:47:34.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23286.051230 secs 2023-09-10T05:47:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23285.397416 secs 2023-09-10T05:47:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:34 smithi098 ceph-mon[103124]: pgmap v12579: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 111 KiB/s wr, 10 op/s 2023-09-10T05:47:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:47:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:47:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23286.051230 secs 2023-09-10T05:47:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:36 smithi170 ceph-mon[102832]: pgmap v12580: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 111 KiB/s wr, 10 op/s 2023-09-10T05:47:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:36 smithi082 ceph-mon[98694]: pgmap v12580: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 111 KiB/s wr, 10 op/s 2023-09-10T05:47:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:36 smithi098 ceph-mon[103124]: pgmap v12580: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 111 KiB/s wr, 10 op/s 2023-09-10T05:47:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:38 smithi170 ceph-mon[102832]: pgmap v12581: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 286 KiB/s wr, 16 op/s 2023-09-10T05:47:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:38 smithi082 ceph-mon[98694]: pgmap v12581: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 286 KiB/s wr, 16 op/s 2023-09-10T05:47:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:38 smithi098 ceph-mon[103124]: pgmap v12581: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 286 KiB/s wr, 16 op/s 2023-09-10T05:47:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23290.397473 secs 2023-09-10T05:47:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23291.051294 secs 2023-09-10T05:47:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23290.397473 secs 2023-09-10T05:47:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23291.051294 secs 2023-09-10T05:47:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23290.397473 secs 2023-09-10T05:47:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23291.051294 secs 2023-09-10T05:47:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:40 smithi170 ceph-mon[102832]: pgmap v12582: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 286 KiB/s wr, 11 op/s 2023-09-10T05:47:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:40 smithi082 ceph-mon[98694]: pgmap v12582: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 286 KiB/s wr, 11 op/s 2023-09-10T05:47:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:40 smithi098 ceph-mon[103124]: pgmap v12582: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 286 KiB/s wr, 11 op/s 2023-09-10T05:47:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:42 smithi170 ceph-mon[102832]: pgmap v12583: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 476 KiB/s wr, 18 op/s 2023-09-10T05:47:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:42 smithi082 ceph-mon[98694]: pgmap v12583: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 476 KiB/s wr, 18 op/s 2023-09-10T05:47:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:42 smithi098 ceph-mon[103124]: pgmap v12583: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 476 KiB/s wr, 18 op/s 2023-09-10T05:47:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23295.397533 secs 2023-09-10T05:47:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:44 smithi170 ceph-mon[102832]: pgmap v12584: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 366 KiB/s wr, 12 op/s 2023-09-10T05:47:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23296.051359 secs 2023-09-10T05:47:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23295.397533 secs 2023-09-10T05:47:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:44 smithi082 ceph-mon[98694]: pgmap v12584: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 366 KiB/s wr, 12 op/s 2023-09-10T05:47:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23296.051359 secs 2023-09-10T05:47:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23295.397533 secs 2023-09-10T05:47:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:44 smithi098 ceph-mon[103124]: pgmap v12584: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 366 KiB/s wr, 12 op/s 2023-09-10T05:47:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23296.051359 secs 2023-09-10T05:47:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:46 smithi170 ceph-mon[102832]: pgmap v12585: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 366 KiB/s wr, 12 op/s 2023-09-10T05:47:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:46 smithi082 ceph-mon[98694]: pgmap v12585: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 366 KiB/s wr, 12 op/s 2023-09-10T05:47:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:46 smithi098 ceph-mon[103124]: pgmap v12585: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 366 KiB/s wr, 12 op/s 2023-09-10T05:47:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:48 smithi170 ceph-mon[102832]: pgmap v12586: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 472 KiB/s wr, 18 op/s 2023-09-10T05:47:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:48 smithi082 ceph-mon[98694]: pgmap v12586: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 472 KiB/s wr, 18 op/s 2023-09-10T05:47:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:48 smithi098 ceph-mon[103124]: pgmap v12586: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 472 KiB/s wr, 18 op/s 2023-09-10T05:47:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23300.397592 secs 2023-09-10T05:47:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23301.051425 secs 2023-09-10T05:47:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23300.397592 secs 2023-09-10T05:47:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23301.051425 secs 2023-09-10T05:47:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23300.397592 secs 2023-09-10T05:47:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23301.051425 secs 2023-09-10T05:47:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:50 smithi170 ceph-mon[102832]: pgmap v12587: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 297 KiB/s wr, 12 op/s 2023-09-10T05:47:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:47:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:47:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:50 smithi082 ceph-mon[98694]: pgmap v12587: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 297 KiB/s wr, 12 op/s 2023-09-10T05:47:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:47:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:47:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:50 smithi098 ceph-mon[103124]: pgmap v12587: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 297 KiB/s wr, 12 op/s 2023-09-10T05:47:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:47:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:47:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:52 smithi170 ceph-mon[102832]: pgmap v12588: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 441 KiB/s wr, 18 op/s 2023-09-10T05:47:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:47:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:47:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:52 smithi082 ceph-mon[98694]: pgmap v12588: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 441 KiB/s wr, 18 op/s 2023-09-10T05:47:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:47:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:47:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:52 smithi098 ceph-mon[103124]: pgmap v12588: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 441 KiB/s wr, 18 op/s 2023-09-10T05:47:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:47:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:47:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23304.426676 secs 2023-09-10T05:47:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23304.426676 secs 2023-09-10T05:47:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23304.426676 secs 2023-09-10T05:47:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23305.397649 secs 2023-09-10T05:47:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:54 smithi170 ceph-mon[102832]: pgmap v12589: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 251 KiB/s wr, 12 op/s 2023-09-10T05:47:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23306.051492 secs 2023-09-10T05:47:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23305.397649 secs 2023-09-10T05:47:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:54 smithi082 ceph-mon[98694]: pgmap v12589: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 251 KiB/s wr, 12 op/s 2023-09-10T05:47:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23306.051492 secs 2023-09-10T05:47:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23305.397649 secs 2023-09-10T05:47:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:54 smithi098 ceph-mon[103124]: pgmap v12589: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 251 KiB/s wr, 12 op/s 2023-09-10T05:47:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23306.051492 secs 2023-09-10T05:47:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:56 smithi170 ceph-mon[102832]: pgmap v12590: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 251 KiB/s wr, 12 op/s 2023-09-10T05:47:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:47:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:47:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:47:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:56 smithi082 ceph-mon[98694]: pgmap v12590: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 251 KiB/s wr, 12 op/s 2023-09-10T05:47:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:47:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:47:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:47:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:56 smithi098 ceph-mon[103124]: pgmap v12590: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 251 KiB/s wr, 12 op/s 2023-09-10T05:47:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:47:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:47:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:47:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:47:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:47:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:58 smithi170 ceph-mon[102832]: pgmap v12591: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 397 KiB/s wr, 18 op/s 2023-09-10T05:47:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23309.426737 secs 2023-09-10T05:47:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:47:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:47:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:58 smithi082 ceph-mon[98694]: pgmap v12591: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 397 KiB/s wr, 18 op/s 2023-09-10T05:47:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23309.426737 secs 2023-09-10T05:47:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:47:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:47:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:58 smithi098 ceph-mon[103124]: pgmap v12591: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 397 KiB/s wr, 18 op/s 2023-09-10T05:47:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23309.426737 secs 2023-09-10T05:47:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23310.397707 secs 2023-09-10T05:47:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:47:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:47:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23310.397707 secs 2023-09-10T05:47:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:47:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:47:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23310.397707 secs 2023-09-10T05:47:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:47:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:48:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23311.051558 secs 2023-09-10T05:48:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:48:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:48:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:00 smithi170 ceph-mon[102832]: pgmap v12592: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 291 KiB/s wr, 12 op/s 2023-09-10T05:48:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:48:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:48:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:48:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23311.051558 secs 2023-09-10T05:48:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:48:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:48:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:00 smithi082 ceph-mon[98694]: pgmap v12592: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 291 KiB/s wr, 12 op/s 2023-09-10T05:48:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:48:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:48:00.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:48:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23311.051558 secs 2023-09-10T05:48:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:48:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:48:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:00 smithi098 ceph-mon[103124]: pgmap v12592: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 291 KiB/s wr, 12 op/s 2023-09-10T05:48:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:48:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:48:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:48:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:02 smithi170 ceph-mon[102832]: pgmap v12593: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 384 KiB/s wr, 17 op/s 2023-09-10T05:48:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:02 smithi082 ceph-mon[98694]: pgmap v12593: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 384 KiB/s wr, 17 op/s 2023-09-10T05:48:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:02 smithi098 ceph-mon[103124]: pgmap v12593: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 384 KiB/s wr, 17 op/s 2023-09-10T05:48:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23314.426797 secs 2023-09-10T05:48:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23314.426797 secs 2023-09-10T05:48:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23314.426797 secs 2023-09-10T05:48:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23315.397799 secs 2023-09-10T05:48:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:04 smithi170 ceph-mon[102832]: pgmap v12594: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 240 KiB/s wr, 11 op/s 2023-09-10T05:48:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23316.051622 secs 2023-09-10T05:48:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23315.397799 secs 2023-09-10T05:48:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:04 smithi082 ceph-mon[98694]: pgmap v12594: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 240 KiB/s wr, 11 op/s 2023-09-10T05:48:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23316.051622 secs 2023-09-10T05:48:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23315.397799 secs 2023-09-10T05:48:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:04 smithi098 ceph-mon[103124]: pgmap v12594: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 240 KiB/s wr, 11 op/s 2023-09-10T05:48:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23316.051622 secs 2023-09-10T05:48:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:06 smithi170 ceph-mon[102832]: pgmap v12595: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 240 KiB/s wr, 11 op/s 2023-09-10T05:48:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:06 smithi082 ceph-mon[98694]: pgmap v12595: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 240 KiB/s wr, 11 op/s 2023-09-10T05:48:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:06 smithi098 ceph-mon[103124]: pgmap v12595: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 240 KiB/s wr, 11 op/s 2023-09-10T05:48:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:08 smithi170 ceph-mon[102832]: pgmap v12596: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 363 KiB/s wr, 31 op/s 2023-09-10T05:48:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23319.426869 secs 2023-09-10T05:48:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:08 smithi082 ceph-mon[98694]: pgmap v12596: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 363 KiB/s wr, 31 op/s 2023-09-10T05:48:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23319.426869 secs 2023-09-10T05:48:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:08 smithi098 ceph-mon[103124]: pgmap v12596: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 363 KiB/s wr, 31 op/s 2023-09-10T05:48:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23319.426869 secs 2023-09-10T05:48:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23320.397857 secs 2023-09-10T05:48:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:09 smithi170 ceph-mon[102832]: osdmap e855: 12 total, 12 up, 12 in 2023-09-10T05:48:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23321.051686 secs 2023-09-10T05:48:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23320.397857 secs 2023-09-10T05:48:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:09 smithi082 ceph-mon[98694]: osdmap e855: 12 total, 12 up, 12 in 2023-09-10T05:48:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23321.051686 secs 2023-09-10T05:48:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23320.397857 secs 2023-09-10T05:48:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:09 smithi098 ceph-mon[103124]: osdmap e855: 12 total, 12 up, 12 in 2023-09-10T05:48:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23321.051686 secs 2023-09-10T05:48:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:10 smithi170 ceph-mon[102832]: pgmap v12598: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 260 KiB/s wr, 30 op/s 2023-09-10T05:48:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:10 smithi082 ceph-mon[98694]: pgmap v12598: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 260 KiB/s wr, 30 op/s 2023-09-10T05:48:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:10 smithi098 ceph-mon[103124]: pgmap v12598: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 260 KiB/s wr, 30 op/s 2023-09-10T05:48:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:12 smithi170 ceph-mon[102832]: pgmap v12599: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 169 KiB/s wr, 30 op/s 2023-09-10T05:48:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:12 smithi082 ceph-mon[98694]: pgmap v12599: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 169 KiB/s wr, 30 op/s 2023-09-10T05:48:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:12 smithi098 ceph-mon[103124]: pgmap v12599: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 169 KiB/s wr, 30 op/s 2023-09-10T05:48:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23324.426927 secs 2023-09-10T05:48:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23324.426927 secs 2023-09-10T05:48:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23324.426927 secs 2023-09-10T05:48:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23325.397914 secs 2023-09-10T05:48:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:14 smithi170 ceph-mon[102832]: pgmap v12600: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 169 KiB/s wr, 30 op/s 2023-09-10T05:48:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23326.051751 secs 2023-09-10T05:48:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23325.397914 secs 2023-09-10T05:48:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:14 smithi082 ceph-mon[98694]: pgmap v12600: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 169 KiB/s wr, 30 op/s 2023-09-10T05:48:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23326.051751 secs 2023-09-10T05:48:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23325.397914 secs 2023-09-10T05:48:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:14 smithi098 ceph-mon[103124]: pgmap v12600: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 169 KiB/s wr, 30 op/s 2023-09-10T05:48:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23326.051751 secs 2023-09-10T05:48:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:16 smithi170 ceph-mon[102832]: osdmap e856: 12 total, 12 up, 12 in 2023-09-10T05:48:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:16 smithi170 ceph-mon[102832]: pgmap v12602: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 7 op/s 2023-09-10T05:48:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:16 smithi082 ceph-mon[98694]: osdmap e856: 12 total, 12 up, 12 in 2023-09-10T05:48:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:16 smithi082 ceph-mon[98694]: pgmap v12602: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 7 op/s 2023-09-10T05:48:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:16 smithi098 ceph-mon[103124]: osdmap e856: 12 total, 12 up, 12 in 2023-09-10T05:48:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:16 smithi098 ceph-mon[103124]: pgmap v12602: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 7 op/s 2023-09-10T05:48:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:18 smithi170 ceph-mon[102832]: pgmap v12603: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 13 op/s 2023-09-10T05:48:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23329.426988 secs 2023-09-10T05:48:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:18 smithi082 ceph-mon[98694]: pgmap v12603: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 13 op/s 2023-09-10T05:48:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23329.426988 secs 2023-09-10T05:48:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:18 smithi098 ceph-mon[103124]: pgmap v12603: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 13 op/s 2023-09-10T05:48:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23329.426988 secs 2023-09-10T05:48:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23330.397970 secs 2023-09-10T05:48:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23331.051818 secs 2023-09-10T05:48:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23330.397970 secs 2023-09-10T05:48:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23331.051818 secs 2023-09-10T05:48:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23330.397970 secs 2023-09-10T05:48:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23331.051818 secs 2023-09-10T05:48:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:20 smithi170 ceph-mon[102832]: pgmap v12604: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 12 op/s 2023-09-10T05:48:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:20 smithi082 ceph-mon[98694]: pgmap v12604: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 12 op/s 2023-09-10T05:48:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:20 smithi098 ceph-mon[103124]: pgmap v12604: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 12 op/s 2023-09-10T05:48:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:22 smithi170 ceph-mon[102832]: pgmap v12605: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 166 KiB/s wr, 12 op/s 2023-09-10T05:48:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:22 smithi082 ceph-mon[98694]: pgmap v12605: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 166 KiB/s wr, 12 op/s 2023-09-10T05:48:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:22 smithi098 ceph-mon[103124]: pgmap v12605: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 166 KiB/s wr, 12 op/s 2023-09-10T05:48:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23334.427069 secs 2023-09-10T05:48:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23334.427069 secs 2023-09-10T05:48:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23334.427069 secs 2023-09-10T05:48:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23335.398028 secs 2023-09-10T05:48:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:24 smithi170 ceph-mon[102832]: pgmap v12606: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 166 KiB/s wr, 12 op/s 2023-09-10T05:48:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23336.051882 secs 2023-09-10T05:48:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23335.398028 secs 2023-09-10T05:48:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:24 smithi082 ceph-mon[98694]: pgmap v12606: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 166 KiB/s wr, 12 op/s 2023-09-10T05:48:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23336.051882 secs 2023-09-10T05:48:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23335.398028 secs 2023-09-10T05:48:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:24 smithi098 ceph-mon[103124]: pgmap v12606: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 166 KiB/s wr, 12 op/s 2023-09-10T05:48:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23336.051882 secs 2023-09-10T05:48:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:26 smithi170 ceph-mon[102832]: pgmap v12607: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T05:48:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:26 smithi082 ceph-mon[98694]: pgmap v12607: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T05:48:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:26 smithi098 ceph-mon[103124]: pgmap v12607: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T05:48:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:28 smithi170 ceph-mon[102832]: pgmap v12608: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 14 op/s 2023-09-10T05:48:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:28 smithi082 ceph-mon[98694]: pgmap v12608: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 14 op/s 2023-09-10T05:48:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:28 smithi098 ceph-mon[103124]: pgmap v12608: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 14 op/s 2023-09-10T05:48:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23340.398088 secs 2023-09-10T05:48:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23341.051950 secs 2023-09-10T05:48:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23340.398088 secs 2023-09-10T05:48:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23341.051950 secs 2023-09-10T05:48:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23340.398088 secs 2023-09-10T05:48:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23341.051950 secs 2023-09-10T05:48:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:30 smithi170 ceph-mon[102832]: pgmap v12609: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 9 op/s 2023-09-10T05:48:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:48:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:48:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:30 smithi082 ceph-mon[98694]: pgmap v12609: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 9 op/s 2023-09-10T05:48:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:48:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:48:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:30 smithi098 ceph-mon[103124]: pgmap v12609: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 9 op/s 2023-09-10T05:48:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:48:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:48:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:48:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:48:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:48:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:32 smithi170 ceph-mon[102832]: pgmap v12610: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 172 KiB/s wr, 15 op/s 2023-09-10T05:48:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:48:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:48:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:48:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:32 smithi082 ceph-mon[98694]: pgmap v12610: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 172 KiB/s wr, 15 op/s 2023-09-10T05:48:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:48:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:48:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:48:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:32 smithi098 ceph-mon[103124]: pgmap v12610: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 172 KiB/s wr, 15 op/s 2023-09-10T05:48:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:48:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:48:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:48:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23345.398149 secs 2023-09-10T05:48:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:34 smithi170 ceph-mon[102832]: pgmap v12611: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 10 op/s 2023-09-10T05:48:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23346.052016 secs 2023-09-10T05:48:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23345.398149 secs 2023-09-10T05:48:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:34 smithi082 ceph-mon[98694]: pgmap v12611: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 10 op/s 2023-09-10T05:48:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23346.052016 secs 2023-09-10T05:48:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23345.398149 secs 2023-09-10T05:48:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:34 smithi098 ceph-mon[103124]: pgmap v12611: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 10 op/s 2023-09-10T05:48:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23346.052016 secs 2023-09-10T05:48:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:36 smithi170 ceph-mon[102832]: pgmap v12612: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 10 op/s 2023-09-10T05:48:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:36 smithi082 ceph-mon[98694]: pgmap v12612: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 10 op/s 2023-09-10T05:48:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:36 smithi098 ceph-mon[103124]: pgmap v12612: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 10 op/s 2023-09-10T05:48:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:38 smithi170 ceph-mon[102832]: pgmap v12613: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 258 KiB/s wr, 15 op/s 2023-09-10T05:48:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:38 smithi082 ceph-mon[98694]: pgmap v12613: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 258 KiB/s wr, 15 op/s 2023-09-10T05:48:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:38 smithi098 ceph-mon[103124]: pgmap v12613: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 258 KiB/s wr, 15 op/s 2023-09-10T05:48:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23350.398212 secs 2023-09-10T05:48:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23351.052081 secs 2023-09-10T05:48:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23350.398212 secs 2023-09-10T05:48:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23351.052081 secs 2023-09-10T05:48:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23350.398212 secs 2023-09-10T05:48:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23351.052081 secs 2023-09-10T05:48:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:40 smithi170 ceph-mon[102832]: pgmap v12614: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 258 KiB/s wr, 11 op/s 2023-09-10T05:48:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:40 smithi082 ceph-mon[98694]: pgmap v12614: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 258 KiB/s wr, 11 op/s 2023-09-10T05:48:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:40 smithi098 ceph-mon[103124]: pgmap v12614: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 258 KiB/s wr, 11 op/s 2023-09-10T05:48:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:42 smithi170 ceph-mon[102832]: pgmap v12615: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 302 KiB/s wr, 17 op/s 2023-09-10T05:48:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:42 smithi082 ceph-mon[98694]: pgmap v12615: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 302 KiB/s wr, 17 op/s 2023-09-10T05:48:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:42 smithi098 ceph-mon[103124]: pgmap v12615: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 302 KiB/s wr, 17 op/s 2023-09-10T05:48:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23355.398306 secs 2023-09-10T05:48:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:44 smithi170 ceph-mon[102832]: pgmap v12616: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 209 KiB/s wr, 11 op/s 2023-09-10T05:48:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:48:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:48:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23356.052145 secs 2023-09-10T05:48:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23355.398306 secs 2023-09-10T05:48:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:44 smithi082 ceph-mon[98694]: pgmap v12616: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 209 KiB/s wr, 11 op/s 2023-09-10T05:48:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:48:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:48:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23356.052145 secs 2023-09-10T05:48:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23355.398306 secs 2023-09-10T05:48:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:44 smithi098 ceph-mon[103124]: pgmap v12616: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 209 KiB/s wr, 11 op/s 2023-09-10T05:48:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:48:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:48:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23356.052145 secs 2023-09-10T05:48:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:46 smithi170 ceph-mon[102832]: pgmap v12617: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 209 KiB/s wr, 11 op/s 2023-09-10T05:48:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:48:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:48:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:46 smithi082 ceph-mon[98694]: pgmap v12617: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 209 KiB/s wr, 11 op/s 2023-09-10T05:48:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:48:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:48:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:46 smithi098 ceph-mon[103124]: pgmap v12617: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 209 KiB/s wr, 11 op/s 2023-09-10T05:48:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:48:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:48:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:48 smithi170 ceph-mon[102832]: pgmap v12618: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 495 KiB/s wr, 18 op/s 2023-09-10T05:48:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:48 smithi082 ceph-mon[98694]: pgmap v12618: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 495 KiB/s wr, 18 op/s 2023-09-10T05:48:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:48 smithi098 ceph-mon[103124]: pgmap v12618: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 495 KiB/s wr, 18 op/s 2023-09-10T05:48:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23360.398366 secs 2023-09-10T05:48:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23361.052212 secs 2023-09-10T05:48:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23360.398366 secs 2023-09-10T05:48:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23361.052212 secs 2023-09-10T05:48:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23360.398366 secs 2023-09-10T05:48:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23361.052212 secs 2023-09-10T05:48:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:50 smithi170 ceph-mon[102832]: pgmap v12619: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 331 KiB/s wr, 12 op/s 2023-09-10T05:48:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:50 smithi082 ceph-mon[98694]: pgmap v12619: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 331 KiB/s wr, 12 op/s 2023-09-10T05:48:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:50 smithi098 ceph-mon[103124]: pgmap v12619: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 331 KiB/s wr, 12 op/s 2023-09-10T05:48:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:52 smithi170 ceph-mon[102832]: pgmap v12620: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 367 KiB/s wr, 18 op/s 2023-09-10T05:48:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:52 smithi082 ceph-mon[98694]: pgmap v12620: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 367 KiB/s wr, 18 op/s 2023-09-10T05:48:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:52 smithi098 ceph-mon[103124]: pgmap v12620: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 367 KiB/s wr, 18 op/s 2023-09-10T05:48:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23365.398429 secs 2023-09-10T05:48:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:54 smithi170 ceph-mon[102832]: pgmap v12621: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 323 KiB/s wr, 12 op/s 2023-09-10T05:48:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:48:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:48:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23366.052276 secs 2023-09-10T05:48:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23365.398429 secs 2023-09-10T05:48:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:54 smithi082 ceph-mon[98694]: pgmap v12621: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 323 KiB/s wr, 12 op/s 2023-09-10T05:48:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:48:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:48:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23366.052276 secs 2023-09-10T05:48:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23365.398429 secs 2023-09-10T05:48:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:54 smithi098 ceph-mon[103124]: pgmap v12621: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 323 KiB/s wr, 12 op/s 2023-09-10T05:48:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:48:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:48:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23366.052276 secs 2023-09-10T05:48:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:56 smithi170 ceph-mon[102832]: pgmap v12622: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 433 KiB/s wr, 14 op/s 2023-09-10T05:48:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:48:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:48:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:56 smithi082 ceph-mon[98694]: pgmap v12622: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 433 KiB/s wr, 14 op/s 2023-09-10T05:48:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:48:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:48:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:56 smithi098 ceph-mon[103124]: pgmap v12622: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 433 KiB/s wr, 14 op/s 2023-09-10T05:48:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:48:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:48:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:58 smithi170 ceph-mon[102832]: pgmap v12623: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 612 KiB/s wr, 19 op/s 2023-09-10T05:48:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23369.427589 secs 2023-09-10T05:48:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:58 smithi082 ceph-mon[98694]: pgmap v12623: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 612 KiB/s wr, 19 op/s 2023-09-10T05:48:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23369.427589 secs 2023-09-10T05:48:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:58 smithi098 ceph-mon[103124]: pgmap v12623: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 612 KiB/s wr, 19 op/s 2023-09-10T05:48:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23369.427589 secs 2023-09-10T05:48:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23370.398493 secs 2023-09-10T05:48:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:48:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23371.052340 secs 2023-09-10T05:48:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23370.398493 secs 2023-09-10T05:48:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:48:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23371.052340 secs 2023-09-10T05:48:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23370.398493 secs 2023-09-10T05:48:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:48:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23371.052340 secs 2023-09-10T05:49:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:00 smithi170 ceph-mon[102832]: pgmap v12624: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 326 KiB/s wr, 12 op/s 2023-09-10T05:49:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:49:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:49:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:49:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:49:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:49:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:49:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:49:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:00 smithi082 ceph-mon[98694]: pgmap v12624: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 326 KiB/s wr, 12 op/s 2023-09-10T05:49:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:49:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:49:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:49:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:49:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:49:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:49:00.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:49:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:00 smithi098 ceph-mon[103124]: pgmap v12624: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 326 KiB/s wr, 12 op/s 2023-09-10T05:49:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:49:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:49:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:49:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:49:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:49:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:49:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:49:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:02 smithi170 ceph-mon[102832]: pgmap v12625: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 400 KiB/s wr, 18 op/s 2023-09-10T05:49:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:02 smithi082 ceph-mon[98694]: pgmap v12625: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 400 KiB/s wr, 18 op/s 2023-09-10T05:49:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:02 smithi098 ceph-mon[103124]: pgmap v12625: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 400 KiB/s wr, 18 op/s 2023-09-10T05:49:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23374.427649 secs 2023-09-10T05:49:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23374.427649 secs 2023-09-10T05:49:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23374.427649 secs 2023-09-10T05:49:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23375.398551 secs 2023-09-10T05:49:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:04 smithi170 ceph-mon[102832]: pgmap v12626: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 364 KiB/s wr, 12 op/s 2023-09-10T05:49:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23376.052404 secs 2023-09-10T05:49:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23375.398551 secs 2023-09-10T05:49:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:04 smithi082 ceph-mon[98694]: pgmap v12626: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 364 KiB/s wr, 12 op/s 2023-09-10T05:49:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23376.052404 secs 2023-09-10T05:49:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23375.398551 secs 2023-09-10T05:49:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:04 smithi098 ceph-mon[103124]: pgmap v12626: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 364 KiB/s wr, 12 op/s 2023-09-10T05:49:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23376.052404 secs 2023-09-10T05:49:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:06 smithi170 ceph-mon[102832]: pgmap v12627: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 541 KiB/s wr, 16 op/s 2023-09-10T05:49:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:06 smithi082 ceph-mon[98694]: pgmap v12627: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 541 KiB/s wr, 16 op/s 2023-09-10T05:49:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:06 smithi098 ceph-mon[103124]: pgmap v12627: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 541 KiB/s wr, 16 op/s 2023-09-10T05:49:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:08 smithi170 ceph-mon[102832]: pgmap v12628: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 470 KiB/s wr, 30 op/s 2023-09-10T05:49:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23379.427748 secs 2023-09-10T05:49:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:08 smithi082 ceph-mon[98694]: pgmap v12628: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 470 KiB/s wr, 30 op/s 2023-09-10T05:49:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23379.427748 secs 2023-09-10T05:49:08.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:08 smithi098 ceph-mon[103124]: pgmap v12628: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 470 KiB/s wr, 30 op/s 2023-09-10T05:49:08.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23379.427748 secs 2023-09-10T05:49:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23380.398611 secs 2023-09-10T05:49:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:09 smithi170 ceph-mon[102832]: osdmap e857: 12 total, 12 up, 12 in 2023-09-10T05:49:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23381.052473 secs 2023-09-10T05:49:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23380.398611 secs 2023-09-10T05:49:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:09 smithi082 ceph-mon[98694]: osdmap e857: 12 total, 12 up, 12 in 2023-09-10T05:49:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23381.052473 secs 2023-09-10T05:49:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23380.398611 secs 2023-09-10T05:49:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:09 smithi098 ceph-mon[103124]: osdmap e857: 12 total, 12 up, 12 in 2023-09-10T05:49:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23381.052473 secs 2023-09-10T05:49:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:10 smithi170 ceph-mon[102832]: pgmap v12630: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 349 KiB/s wr, 30 op/s 2023-09-10T05:49:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:10 smithi082 ceph-mon[98694]: pgmap v12630: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 349 KiB/s wr, 30 op/s 2023-09-10T05:49:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:10 smithi098 ceph-mon[103124]: pgmap v12630: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 349 KiB/s wr, 30 op/s 2023-09-10T05:49:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:12 smithi170 ceph-mon[102832]: pgmap v12631: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 260 KiB/s wr, 29 op/s 2023-09-10T05:49:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:12 smithi082 ceph-mon[98694]: pgmap v12631: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 260 KiB/s wr, 29 op/s 2023-09-10T05:49:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:12 smithi098 ceph-mon[103124]: pgmap v12631: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 260 KiB/s wr, 29 op/s 2023-09-10T05:49:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23384.427811 secs 2023-09-10T05:49:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23384.427811 secs 2023-09-10T05:49:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23384.427811 secs 2023-09-10T05:49:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23385.398670 secs 2023-09-10T05:49:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:14 smithi170 ceph-mon[102832]: pgmap v12632: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 260 KiB/s wr, 29 op/s 2023-09-10T05:49:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23386.052533 secs 2023-09-10T05:49:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23385.398670 secs 2023-09-10T05:49:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:14 smithi082 ceph-mon[98694]: pgmap v12632: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 260 KiB/s wr, 29 op/s 2023-09-10T05:49:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23386.052533 secs 2023-09-10T05:49:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23385.398670 secs 2023-09-10T05:49:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:14 smithi098 ceph-mon[103124]: pgmap v12632: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 260 KiB/s wr, 29 op/s 2023-09-10T05:49:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23386.052533 secs 2023-09-10T05:49:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:16 smithi170 ceph-mon[102832]: osdmap e858: 12 total, 12 up, 12 in 2023-09-10T05:49:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:16 smithi170 ceph-mon[102832]: pgmap v12634: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 895 B/s wr, 8 op/s 2023-09-10T05:49:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:16 smithi082 ceph-mon[98694]: osdmap e858: 12 total, 12 up, 12 in 2023-09-10T05:49:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:16 smithi082 ceph-mon[98694]: pgmap v12634: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 895 B/s wr, 8 op/s 2023-09-10T05:49:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:16 smithi098 ceph-mon[103124]: osdmap e858: 12 total, 12 up, 12 in 2023-09-10T05:49:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:16 smithi098 ceph-mon[103124]: pgmap v12634: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 895 B/s wr, 8 op/s 2023-09-10T05:49:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:18 smithi170 ceph-mon[102832]: pgmap v12635: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 13 op/s 2023-09-10T05:49:18.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23389.427888 secs 2023-09-10T05:49:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:18 smithi082 ceph-mon[98694]: pgmap v12635: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 13 op/s 2023-09-10T05:49:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23389.427888 secs 2023-09-10T05:49:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:18 smithi098 ceph-mon[103124]: pgmap v12635: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 13 op/s 2023-09-10T05:49:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23389.427888 secs 2023-09-10T05:49:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23390.398731 secs 2023-09-10T05:49:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23391.052598 secs 2023-09-10T05:49:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23390.398731 secs 2023-09-10T05:49:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23391.052598 secs 2023-09-10T05:49:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23390.398731 secs 2023-09-10T05:49:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23391.052598 secs 2023-09-10T05:49:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:20 smithi170 ceph-mon[102832]: pgmap v12636: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:49:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:20 smithi082 ceph-mon[98694]: pgmap v12636: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:49:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:20 smithi098 ceph-mon[103124]: pgmap v12636: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:49:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:22 smithi170 ceph-mon[102832]: pgmap v12637: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:49:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:22 smithi082 ceph-mon[98694]: pgmap v12637: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:49:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:22 smithi098 ceph-mon[103124]: pgmap v12637: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:49:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23394.427949 secs 2023-09-10T05:49:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23394.427949 secs 2023-09-10T05:49:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23394.427949 secs 2023-09-10T05:49:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23395.398790 secs 2023-09-10T05:49:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:24 smithi170 ceph-mon[102832]: pgmap v12638: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:49:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23396.052661 secs 2023-09-10T05:49:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23395.398790 secs 2023-09-10T05:49:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:24 smithi082 ceph-mon[98694]: pgmap v12638: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:49:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23396.052661 secs 2023-09-10T05:49:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23395.398790 secs 2023-09-10T05:49:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:24 smithi098 ceph-mon[103124]: pgmap v12638: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-09-10T05:49:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23396.052661 secs 2023-09-10T05:49:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:26 smithi170 ceph-mon[102832]: pgmap v12639: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T05:49:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:26 smithi082 ceph-mon[98694]: pgmap v12639: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T05:49:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:26 smithi098 ceph-mon[103124]: pgmap v12639: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T05:49:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:28 smithi170 ceph-mon[102832]: pgmap v12640: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 97 KiB/s wr, 14 op/s 2023-09-10T05:49:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:28 smithi082 ceph-mon[98694]: pgmap v12640: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 97 KiB/s wr, 14 op/s 2023-09-10T05:49:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:28 smithi098 ceph-mon[103124]: pgmap v12640: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 97 KiB/s wr, 14 op/s 2023-09-10T05:49:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23400.398848 secs 2023-09-10T05:49:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23401.052725 secs 2023-09-10T05:49:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23400.398848 secs 2023-09-10T05:49:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23401.052725 secs 2023-09-10T05:49:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23400.398848 secs 2023-09-10T05:49:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23401.052725 secs 2023-09-10T05:49:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:30 smithi170 ceph-mon[102832]: pgmap v12641: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 79 KiB/s wr, 10 op/s 2023-09-10T05:49:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:30 smithi082 ceph-mon[98694]: pgmap v12641: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 79 KiB/s wr, 10 op/s 2023-09-10T05:49:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:30 smithi098 ceph-mon[103124]: pgmap v12641: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 79 KiB/s wr, 10 op/s 2023-09-10T05:49:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:49:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:49:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:49:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:32 smithi170 ceph-mon[102832]: pgmap v12642: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 153 KiB/s wr, 15 op/s 2023-09-10T05:49:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:49:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:49:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:49:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:32 smithi082 ceph-mon[98694]: pgmap v12642: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 153 KiB/s wr, 15 op/s 2023-09-10T05:49:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:49:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:49:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:49:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:32 smithi098 ceph-mon[103124]: pgmap v12642: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 153 KiB/s wr, 15 op/s 2023-09-10T05:49:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:49:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:49:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:49:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23405.398927 secs 2023-09-10T05:49:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:34 smithi170 ceph-mon[102832]: pgmap v12643: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 153 KiB/s wr, 10 op/s 2023-09-10T05:49:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:49:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:49:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23406.052788 secs 2023-09-10T05:49:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23405.398927 secs 2023-09-10T05:49:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:34 smithi082 ceph-mon[98694]: pgmap v12643: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 153 KiB/s wr, 10 op/s 2023-09-10T05:49:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:49:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:49:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23406.052788 secs 2023-09-10T05:49:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23405.398927 secs 2023-09-10T05:49:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:34 smithi098 ceph-mon[103124]: pgmap v12643: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 153 KiB/s wr, 10 op/s 2023-09-10T05:49:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:49:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:49:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23406.052788 secs 2023-09-10T05:49:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:36 smithi170 ceph-mon[102832]: pgmap v12644: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 13 op/s 2023-09-10T05:49:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:49:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:49:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:36 smithi082 ceph-mon[98694]: pgmap v12644: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 13 op/s 2023-09-10T05:49:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:49:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:49:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:36 smithi098 ceph-mon[103124]: pgmap v12644: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 13 op/s 2023-09-10T05:49:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:49:36.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:49:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:38 smithi170 ceph-mon[102832]: pgmap v12645: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 558 KiB/s wr, 17 op/s 2023-09-10T05:49:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:38 smithi082 ceph-mon[98694]: pgmap v12645: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 558 KiB/s wr, 17 op/s 2023-09-10T05:49:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:38 smithi098 ceph-mon[103124]: pgmap v12645: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 558 KiB/s wr, 17 op/s 2023-09-10T05:49:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23410.399006 secs 2023-09-10T05:49:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23411.052853 secs 2023-09-10T05:49:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23410.399006 secs 2023-09-10T05:49:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23411.052853 secs 2023-09-10T05:49:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23410.399006 secs 2023-09-10T05:49:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23411.052853 secs 2023-09-10T05:49:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:40 smithi170 ceph-mon[102832]: pgmap v12646: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 483 KiB/s wr, 13 op/s 2023-09-10T05:49:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:40 smithi082 ceph-mon[98694]: pgmap v12646: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 483 KiB/s wr, 13 op/s 2023-09-10T05:49:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:40 smithi098 ceph-mon[103124]: pgmap v12646: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 483 KiB/s wr, 13 op/s 2023-09-10T05:49:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:42 smithi170 ceph-mon[102832]: pgmap v12647: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 498 KiB/s wr, 19 op/s 2023-09-10T05:49:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:42 smithi082 ceph-mon[98694]: pgmap v12647: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 498 KiB/s wr, 19 op/s 2023-09-10T05:49:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:42 smithi098 ceph-mon[103124]: pgmap v12647: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 498 KiB/s wr, 19 op/s 2023-09-10T05:49:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23415.399070 secs 2023-09-10T05:49:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:44 smithi170 ceph-mon[102832]: pgmap v12648: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 424 KiB/s wr, 14 op/s 2023-09-10T05:49:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:49:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:49:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23416.052917 secs 2023-09-10T05:49:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23415.399070 secs 2023-09-10T05:49:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:44 smithi082 ceph-mon[98694]: pgmap v12648: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 424 KiB/s wr, 14 op/s 2023-09-10T05:49:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:49:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:49:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23416.052917 secs 2023-09-10T05:49:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23415.399070 secs 2023-09-10T05:49:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:44 smithi098 ceph-mon[103124]: pgmap v12648: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 424 KiB/s wr, 14 op/s 2023-09-10T05:49:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:49:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:49:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23416.052917 secs 2023-09-10T05:49:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:46 smithi170 ceph-mon[102832]: pgmap v12649: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 424 KiB/s wr, 16 op/s 2023-09-10T05:49:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:49:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:49:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:46 smithi082 ceph-mon[98694]: pgmap v12649: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 424 KiB/s wr, 16 op/s 2023-09-10T05:49:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:49:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:49:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:46 smithi098 ceph-mon[103124]: pgmap v12649: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 424 KiB/s wr, 16 op/s 2023-09-10T05:49:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:49:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:49:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:48 smithi170 ceph-mon[102832]: pgmap v12650: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 696 KiB/s wr, 18 op/s 2023-09-10T05:49:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:48 smithi082 ceph-mon[98694]: pgmap v12650: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 696 KiB/s wr, 18 op/s 2023-09-10T05:49:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:48 smithi098 ceph-mon[103124]: pgmap v12650: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 696 KiB/s wr, 18 op/s 2023-09-10T05:49:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23420.399155 secs 2023-09-10T05:49:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23421.052983 secs 2023-09-10T05:49:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23420.399155 secs 2023-09-10T05:49:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23421.052983 secs 2023-09-10T05:49:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23420.399155 secs 2023-09-10T05:49:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23421.052983 secs 2023-09-10T05:49:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:50 smithi170 ceph-mon[102832]: pgmap v12651: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 288 KiB/s wr, 13 op/s 2023-09-10T05:49:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:50 smithi082 ceph-mon[98694]: pgmap v12651: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 288 KiB/s wr, 13 op/s 2023-09-10T05:49:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:50 smithi098 ceph-mon[103124]: pgmap v12651: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 288 KiB/s wr, 13 op/s 2023-09-10T05:49:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:52 smithi170 ceph-mon[102832]: pgmap v12652: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 571 KiB/s wr, 20 op/s 2023-09-10T05:49:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:52 smithi082 ceph-mon[98694]: pgmap v12652: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 571 KiB/s wr, 20 op/s 2023-09-10T05:49:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:52 smithi098 ceph-mon[103124]: pgmap v12652: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 571 KiB/s wr, 20 op/s 2023-09-10T05:49:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23424.428373 secs 2023-09-10T05:49:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23424.428373 secs 2023-09-10T05:49:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23424.428373 secs 2023-09-10T05:49:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23425.399212 secs 2023-09-10T05:49:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:54 smithi170 ceph-mon[102832]: pgmap v12653: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 556 KiB/s wr, 13 op/s 2023-09-10T05:49:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:49:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:49:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23426.053048 secs 2023-09-10T05:49:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23425.399212 secs 2023-09-10T05:49:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:54 smithi082 ceph-mon[98694]: pgmap v12653: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 556 KiB/s wr, 13 op/s 2023-09-10T05:49:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:49:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:49:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23426.053048 secs 2023-09-10T05:49:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23425.399212 secs 2023-09-10T05:49:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:54 smithi098 ceph-mon[103124]: pgmap v12653: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 556 KiB/s wr, 13 op/s 2023-09-10T05:49:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:49:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:49:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23426.053048 secs 2023-09-10T05:49:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:56 smithi170 ceph-mon[102832]: pgmap v12654: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 557 KiB/s wr, 16 op/s 2023-09-10T05:49:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:49:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:49:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:49:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:56 smithi082 ceph-mon[98694]: pgmap v12654: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 557 KiB/s wr, 16 op/s 2023-09-10T05:49:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:49:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:49:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:49:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:56 smithi098 ceph-mon[103124]: pgmap v12654: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 557 KiB/s wr, 16 op/s 2023-09-10T05:49:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:49:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:49:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:49:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:58 smithi170 ceph-mon[102832]: pgmap v12655: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 605 KiB/s wr, 16 op/s 2023-09-10T05:49:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23429.428436 secs 2023-09-10T05:49:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:58 smithi082 ceph-mon[98694]: pgmap v12655: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 605 KiB/s wr, 16 op/s 2023-09-10T05:49:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23429.428436 secs 2023-09-10T05:49:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:58 smithi098 ceph-mon[103124]: pgmap v12655: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 605 KiB/s wr, 16 op/s 2023-09-10T05:49:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23429.428436 secs 2023-09-10T05:49:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23430.399270 secs 2023-09-10T05:49:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:49:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23431.053117 secs 2023-09-10T05:49:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23430.399270 secs 2023-09-10T05:49:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:49:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23431.053117 secs 2023-09-10T05:49:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23430.399270 secs 2023-09-10T05:49:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:49:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23431.053117 secs 2023-09-10T05:50:00.558 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:00 smithi098 ceph-mon[103124]: pgmap v12656: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T05:50:00.558 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T05:50:00.559 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:50:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:00 smithi170 ceph-mon[102832]: pgmap v12656: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T05:50:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T05:50:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:50:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:00 smithi082 ceph-mon[98694]: pgmap v12656: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T05:50:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T05:50:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:50:02.628 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:02 smithi170 ceph-mon[102832]: pgmap v12657: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 499 KiB/s wr, 16 op/s 2023-09-10T05:50:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:02 smithi082 ceph-mon[98694]: pgmap v12657: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 499 KiB/s wr, 16 op/s 2023-09-10T05:50:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:02 smithi098 ceph-mon[103124]: pgmap v12657: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 499 KiB/s wr, 16 op/s 2023-09-10T05:50:03.578 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23434.428527 secs 2023-09-10T05:50:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23434.428527 secs 2023-09-10T05:50:03.695 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23434.428527 secs 2023-09-10T05:50:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23435.399330 secs 2023-09-10T05:50:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:04 smithi098 ceph-mon[103124]: pgmap v12658: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 215 KiB/s wr, 9 op/s 2023-09-10T05:50:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:50:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:50:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23436.053187 secs 2023-09-10T05:50:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23435.399330 secs 2023-09-10T05:50:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:04 smithi170 ceph-mon[102832]: pgmap v12658: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 215 KiB/s wr, 9 op/s 2023-09-10T05:50:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:50:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:50:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23436.053187 secs 2023-09-10T05:50:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23435.399330 secs 2023-09-10T05:50:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:04 smithi082 ceph-mon[98694]: pgmap v12658: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 215 KiB/s wr, 9 op/s 2023-09-10T05:50:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:50:04.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:50:04.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23436.053187 secs 2023-09-10T05:50:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:50:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:50:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:50:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:50:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:50:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:50:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:06 smithi170 ceph-mon[102832]: pgmap v12659: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 257 KiB/s wr, 15 op/s 2023-09-10T05:50:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:06 smithi082 ceph-mon[98694]: pgmap v12659: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 257 KiB/s wr, 15 op/s 2023-09-10T05:50:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:06 smithi098 ceph-mon[103124]: pgmap v12659: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 257 KiB/s wr, 15 op/s 2023-09-10T05:50:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:08 smithi170 ceph-mon[102832]: pgmap v12660: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 411 KiB/s wr, 27 op/s 2023-09-10T05:50:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23439.428587 secs 2023-09-10T05:50:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:08 smithi082 ceph-mon[98694]: pgmap v12660: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 411 KiB/s wr, 27 op/s 2023-09-10T05:50:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23439.428587 secs 2023-09-10T05:50:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:08 smithi098 ceph-mon[103124]: pgmap v12660: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 411 KiB/s wr, 27 op/s 2023-09-10T05:50:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23439.428587 secs 2023-09-10T05:50:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23440.399415 secs 2023-09-10T05:50:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:09 smithi170 ceph-mon[102832]: osdmap e859: 12 total, 12 up, 12 in 2023-09-10T05:50:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23441.053253 secs 2023-09-10T05:50:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23440.399415 secs 2023-09-10T05:50:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:09 smithi082 ceph-mon[98694]: osdmap e859: 12 total, 12 up, 12 in 2023-09-10T05:50:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23441.053253 secs 2023-09-10T05:50:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23440.399415 secs 2023-09-10T05:50:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:09 smithi098 ceph-mon[103124]: osdmap e859: 12 total, 12 up, 12 in 2023-09-10T05:50:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23441.053253 secs 2023-09-10T05:50:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:10 smithi170 ceph-mon[102832]: pgmap v12662: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 435 KiB/s wr, 29 op/s 2023-09-10T05:50:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:10 smithi082 ceph-mon[98694]: pgmap v12662: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 435 KiB/s wr, 29 op/s 2023-09-10T05:50:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:10 smithi098 ceph-mon[103124]: pgmap v12662: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 435 KiB/s wr, 29 op/s 2023-09-10T05:50:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:50:11.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:50:11.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:50:11.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:50:11.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:50:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:50:11.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:50:11.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:50:11.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:50:11.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:50:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:50:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:50:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:50:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:50:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:50:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:12 smithi170 ceph-mon[102832]: pgmap v12663: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 258 KiB/s wr, 31 op/s 2023-09-10T05:50:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:12 smithi082 ceph-mon[98694]: pgmap v12663: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 258 KiB/s wr, 31 op/s 2023-09-10T05:50:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:12 smithi098 ceph-mon[103124]: pgmap v12663: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 258 KiB/s wr, 31 op/s 2023-09-10T05:50:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23444.428645 secs 2023-09-10T05:50:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23444.428645 secs 2023-09-10T05:50:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23444.428645 secs 2023-09-10T05:50:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23445.399474 secs 2023-09-10T05:50:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:14 smithi170 ceph-mon[102832]: pgmap v12664: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 258 KiB/s wr, 31 op/s 2023-09-10T05:50:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23446.053322 secs 2023-09-10T05:50:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23445.399474 secs 2023-09-10T05:50:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:14 smithi082 ceph-mon[98694]: pgmap v12664: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 258 KiB/s wr, 31 op/s 2023-09-10T05:50:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23446.053322 secs 2023-09-10T05:50:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23445.399474 secs 2023-09-10T05:50:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:14 smithi098 ceph-mon[103124]: pgmap v12664: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 258 KiB/s wr, 31 op/s 2023-09-10T05:50:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23446.053322 secs 2023-09-10T05:50:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:16 smithi170 ceph-mon[102832]: osdmap e860: 12 total, 12 up, 12 in 2023-09-10T05:50:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:16 smithi170 ceph-mon[102832]: pgmap v12666: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 12 op/s 2023-09-10T05:50:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:50:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:50:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:16 smithi082 ceph-mon[98694]: osdmap e860: 12 total, 12 up, 12 in 2023-09-10T05:50:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:16 smithi082 ceph-mon[98694]: pgmap v12666: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 12 op/s 2023-09-10T05:50:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:50:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:50:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:16 smithi098 ceph-mon[103124]: osdmap e860: 12 total, 12 up, 12 in 2023-09-10T05:50:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:16 smithi098 ceph-mon[103124]: pgmap v12666: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 12 op/s 2023-09-10T05:50:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:50:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:50:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:50:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:50:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:18 smithi170 ceph-mon[102832]: pgmap v12667: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 14 op/s 2023-09-10T05:50:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23449.428707 secs 2023-09-10T05:50:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:50:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:50:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:18 smithi082 ceph-mon[98694]: pgmap v12667: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 14 op/s 2023-09-10T05:50:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23449.428707 secs 2023-09-10T05:50:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:50:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:50:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:18 smithi098 ceph-mon[103124]: pgmap v12667: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 14 op/s 2023-09-10T05:50:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23449.428707 secs 2023-09-10T05:50:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23450.399533 secs 2023-09-10T05:50:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23450.399533 secs 2023-09-10T05:50:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23450.399533 secs 2023-09-10T05:50:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23451.053387 secs 2023-09-10T05:50:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:20 smithi170 ceph-mon[102832]: pgmap v12668: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T05:50:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23451.053387 secs 2023-09-10T05:50:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:20 smithi082 ceph-mon[98694]: pgmap v12668: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T05:50:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23451.053387 secs 2023-09-10T05:50:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:20 smithi098 ceph-mon[103124]: pgmap v12668: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T05:50:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:22 smithi170 ceph-mon[102832]: pgmap v12669: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T05:50:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:22 smithi082 ceph-mon[98694]: pgmap v12669: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T05:50:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:22 smithi098 ceph-mon[103124]: pgmap v12669: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T05:50:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23454.428771 secs 2023-09-10T05:50:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23454.428771 secs 2023-09-10T05:50:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23454.428771 secs 2023-09-10T05:50:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23455.399591 secs 2023-09-10T05:50:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:24 smithi170 ceph-mon[102832]: pgmap v12670: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T05:50:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23456.053449 secs 2023-09-10T05:50:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23455.399591 secs 2023-09-10T05:50:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:24 smithi082 ceph-mon[98694]: pgmap v12670: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T05:50:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23456.053449 secs 2023-09-10T05:50:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23455.399591 secs 2023-09-10T05:50:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:24 smithi098 ceph-mon[103124]: pgmap v12670: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T05:50:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23456.053449 secs 2023-09-10T05:50:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:26 smithi170 ceph-mon[102832]: pgmap v12671: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 94 KiB/s wr, 15 op/s 2023-09-10T05:50:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:26 smithi082 ceph-mon[98694]: pgmap v12671: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 94 KiB/s wr, 15 op/s 2023-09-10T05:50:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:26 smithi098 ceph-mon[103124]: pgmap v12671: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 94 KiB/s wr, 15 op/s 2023-09-10T05:50:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:28 smithi170 ceph-mon[102832]: pgmap v12672: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T05:50:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:28 smithi082 ceph-mon[98694]: pgmap v12672: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T05:50:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:28 smithi098 ceph-mon[103124]: pgmap v12672: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T05:50:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23460.399664 secs 2023-09-10T05:50:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23461.053489 secs 2023-09-10T05:50:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23460.399664 secs 2023-09-10T05:50:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23461.053489 secs 2023-09-10T05:50:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23460.399664 secs 2023-09-10T05:50:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23461.053489 secs 2023-09-10T05:50:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:30 smithi170 ceph-mon[102832]: pgmap v12673: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 9 op/s 2023-09-10T05:50:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:30 smithi082 ceph-mon[98694]: pgmap v12673: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 9 op/s 2023-09-10T05:50:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:30 smithi098 ceph-mon[103124]: pgmap v12673: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 9 op/s 2023-09-10T05:50:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:50:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:50:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:50:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:32 smithi170 ceph-mon[102832]: pgmap v12674: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 173 KiB/s wr, 14 op/s 2023-09-10T05:50:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:50:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:50:32.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:50:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:32 smithi082 ceph-mon[98694]: pgmap v12674: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 173 KiB/s wr, 14 op/s 2023-09-10T05:50:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:50:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:50:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:50:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:32 smithi098 ceph-mon[103124]: pgmap v12674: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 173 KiB/s wr, 14 op/s 2023-09-10T05:50:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:50:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:50:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:50:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23465.399741 secs 2023-09-10T05:50:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:34 smithi170 ceph-mon[102832]: pgmap v12675: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 9 op/s 2023-09-10T05:50:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23466.053554 secs 2023-09-10T05:50:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23465.399741 secs 2023-09-10T05:50:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:34 smithi082 ceph-mon[98694]: pgmap v12675: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 9 op/s 2023-09-10T05:50:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23466.053554 secs 2023-09-10T05:50:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23465.399741 secs 2023-09-10T05:50:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:34 smithi098 ceph-mon[103124]: pgmap v12675: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 9 op/s 2023-09-10T05:50:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23466.053554 secs 2023-09-10T05:50:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:36 smithi170 ceph-mon[102832]: pgmap v12676: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 245 KiB/s wr, 13 op/s 2023-09-10T05:50:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:36 smithi082 ceph-mon[98694]: pgmap v12676: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 245 KiB/s wr, 13 op/s 2023-09-10T05:50:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:36 smithi098 ceph-mon[103124]: pgmap v12676: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 245 KiB/s wr, 13 op/s 2023-09-10T05:50:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:38 smithi170 ceph-mon[102832]: pgmap v12677: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 263 KiB/s wr, 11 op/s 2023-09-10T05:50:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:38 smithi082 ceph-mon[98694]: pgmap v12677: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 263 KiB/s wr, 11 op/s 2023-09-10T05:50:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:38 smithi098 ceph-mon[103124]: pgmap v12677: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 263 KiB/s wr, 11 op/s 2023-09-10T05:50:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23470.399797 secs 2023-09-10T05:50:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23471.053621 secs 2023-09-10T05:50:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23470.399797 secs 2023-09-10T05:50:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23471.053621 secs 2023-09-10T05:50:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23470.399797 secs 2023-09-10T05:50:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23471.053621 secs 2023-09-10T05:50:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:40 smithi170 ceph-mon[102832]: pgmap v12678: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 263 KiB/s wr, 10 op/s 2023-09-10T05:50:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:40 smithi082 ceph-mon[98694]: pgmap v12678: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 263 KiB/s wr, 10 op/s 2023-09-10T05:50:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:40 smithi098 ceph-mon[103124]: pgmap v12678: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 263 KiB/s wr, 10 op/s 2023-09-10T05:50:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:42 smithi170 ceph-mon[102832]: pgmap v12679: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 379 KiB/s wr, 17 op/s 2023-09-10T05:50:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:42 smithi082 ceph-mon[98694]: pgmap v12679: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 379 KiB/s wr, 17 op/s 2023-09-10T05:50:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:42 smithi098 ceph-mon[103124]: pgmap v12679: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 379 KiB/s wr, 17 op/s 2023-09-10T05:50:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23475.399883 secs 2023-09-10T05:50:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:44 smithi170 ceph-mon[102832]: pgmap v12680: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-09-10T05:50:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:50:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:50:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23476.053688 secs 2023-09-10T05:50:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23475.399883 secs 2023-09-10T05:50:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:44 smithi082 ceph-mon[98694]: pgmap v12680: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-09-10T05:50:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:50:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:50:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23476.053688 secs 2023-09-10T05:50:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23475.399883 secs 2023-09-10T05:50:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:44 smithi098 ceph-mon[103124]: pgmap v12680: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-09-10T05:50:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:50:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:50:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23476.053688 secs 2023-09-10T05:50:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:46 smithi170 ceph-mon[102832]: pgmap v12681: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 483 KiB/s wr, 17 op/s 2023-09-10T05:50:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:50:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:50:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:46 smithi082 ceph-mon[98694]: pgmap v12681: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 483 KiB/s wr, 17 op/s 2023-09-10T05:50:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:50:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:50:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:46 smithi098 ceph-mon[103124]: pgmap v12681: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 483 KiB/s wr, 17 op/s 2023-09-10T05:50:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:50:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:50:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:48 smithi170 ceph-mon[102832]: pgmap v12682: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 351 KiB/s wr, 15 op/s 2023-09-10T05:50:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:48 smithi082 ceph-mon[98694]: pgmap v12682: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 351 KiB/s wr, 15 op/s 2023-09-10T05:50:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:48 smithi098 ceph-mon[103124]: pgmap v12682: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 351 KiB/s wr, 15 op/s 2023-09-10T05:50:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23480.399941 secs 2023-09-10T05:50:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23481.053754 secs 2023-09-10T05:50:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23480.399941 secs 2023-09-10T05:50:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23481.053754 secs 2023-09-10T05:50:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23480.399941 secs 2023-09-10T05:50:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23481.053754 secs 2023-09-10T05:50:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:50 smithi170 ceph-mon[102832]: pgmap v12683: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 333 KiB/s wr, 13 op/s 2023-09-10T05:50:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:50 smithi082 ceph-mon[98694]: pgmap v12683: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 333 KiB/s wr, 13 op/s 2023-09-10T05:50:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:50 smithi098 ceph-mon[103124]: pgmap v12683: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 333 KiB/s wr, 13 op/s 2023-09-10T05:50:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:52 smithi170 ceph-mon[102832]: pgmap v12684: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 380 KiB/s wr, 19 op/s 2023-09-10T05:50:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:52 smithi082 ceph-mon[98694]: pgmap v12684: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 380 KiB/s wr, 19 op/s 2023-09-10T05:50:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:52 smithi098 ceph-mon[103124]: pgmap v12684: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 380 KiB/s wr, 19 op/s 2023-09-10T05:50:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23485.400002 secs 2023-09-10T05:50:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:54 smithi170 ceph-mon[102832]: pgmap v12685: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 265 KiB/s wr, 12 op/s 2023-09-10T05:50:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:50:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:50:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23486.053818 secs 2023-09-10T05:50:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23485.400002 secs 2023-09-10T05:50:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:54 smithi082 ceph-mon[98694]: pgmap v12685: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 265 KiB/s wr, 12 op/s 2023-09-10T05:50:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:50:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:50:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23486.053818 secs 2023-09-10T05:50:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23485.400002 secs 2023-09-10T05:50:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:54 smithi098 ceph-mon[103124]: pgmap v12685: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 265 KiB/s wr, 12 op/s 2023-09-10T05:50:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:50:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:50:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23486.053818 secs 2023-09-10T05:50:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:56 smithi170 ceph-mon[102832]: pgmap v12686: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 657 KiB/s wr, 18 op/s 2023-09-10T05:50:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:50:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:50:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:56 smithi082 ceph-mon[98694]: pgmap v12686: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 657 KiB/s wr, 18 op/s 2023-09-10T05:50:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:50:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:50:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:56 smithi098 ceph-mon[103124]: pgmap v12686: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 657 KiB/s wr, 18 op/s 2023-09-10T05:50:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:50:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:50:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:58 smithi170 ceph-mon[102832]: pgmap v12687: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 494 KiB/s wr, 14 op/s 2023-09-10T05:50:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:50:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:50:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23489.429416 secs 2023-09-10T05:50:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:58 smithi082 ceph-mon[98694]: pgmap v12687: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 494 KiB/s wr, 14 op/s 2023-09-10T05:50:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:50:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:50:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23489.429416 secs 2023-09-10T05:50:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:58 smithi098 ceph-mon[103124]: pgmap v12687: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 494 KiB/s wr, 14 op/s 2023-09-10T05:50:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:50:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:50:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23489.429416 secs 2023-09-10T05:50:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23490.400059 secs 2023-09-10T05:50:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:50:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23491.053887 secs 2023-09-10T05:50:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23490.400059 secs 2023-09-10T05:50:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:50:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23491.053887 secs 2023-09-10T05:50:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23490.400059 secs 2023-09-10T05:50:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:50:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23491.053887 secs 2023-09-10T05:51:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:00 smithi170 ceph-mon[102832]: pgmap v12688: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 476 KiB/s wr, 12 op/s 2023-09-10T05:51:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:51:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:51:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:51:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:00 smithi082 ceph-mon[98694]: pgmap v12688: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 476 KiB/s wr, 12 op/s 2023-09-10T05:51:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:51:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:51:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:51:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:00 smithi098 ceph-mon[103124]: pgmap v12688: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 476 KiB/s wr, 12 op/s 2023-09-10T05:51:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:51:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:51:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:51:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:02 smithi170 ceph-mon[102832]: pgmap v12689: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 476 KiB/s wr, 18 op/s 2023-09-10T05:51:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:02 smithi082 ceph-mon[98694]: pgmap v12689: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 476 KiB/s wr, 18 op/s 2023-09-10T05:51:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:02 smithi098 ceph-mon[103124]: pgmap v12689: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 476 KiB/s wr, 18 op/s 2023-09-10T05:51:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23494.429528 secs 2023-09-10T05:51:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23494.429528 secs 2023-09-10T05:51:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23494.429528 secs 2023-09-10T05:51:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23495.400136 secs 2023-09-10T05:51:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:04 smithi170 ceph-mon[102832]: pgmap v12690: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 428 KiB/s wr, 13 op/s 2023-09-10T05:51:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23496.053952 secs 2023-09-10T05:51:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23495.400136 secs 2023-09-10T05:51:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:04 smithi082 ceph-mon[98694]: pgmap v12690: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 428 KiB/s wr, 13 op/s 2023-09-10T05:51:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23496.053952 secs 2023-09-10T05:51:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23495.400136 secs 2023-09-10T05:51:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:04 smithi098 ceph-mon[103124]: pgmap v12690: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 428 KiB/s wr, 13 op/s 2023-09-10T05:51:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23496.053952 secs 2023-09-10T05:51:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:06 smithi170 ceph-mon[102832]: pgmap v12691: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 641 KiB/s wr, 32 op/s 2023-09-10T05:51:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:06 smithi082 ceph-mon[98694]: pgmap v12691: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 641 KiB/s wr, 32 op/s 2023-09-10T05:51:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:06 smithi098 ceph-mon[103124]: pgmap v12691: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 641 KiB/s wr, 32 op/s 2023-09-10T05:51:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:08 smithi170 ceph-mon[102832]: pgmap v12692: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 267 KiB/s wr, 27 op/s 2023-09-10T05:51:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23499.429587 secs 2023-09-10T05:51:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:08 smithi082 ceph-mon[98694]: pgmap v12692: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 267 KiB/s wr, 27 op/s 2023-09-10T05:51:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23499.429587 secs 2023-09-10T05:51:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:08 smithi098 ceph-mon[103124]: pgmap v12692: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 267 KiB/s wr, 27 op/s 2023-09-10T05:51:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23499.429587 secs 2023-09-10T05:51:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23500.400219 secs 2023-09-10T05:51:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:09 smithi170 ceph-mon[102832]: osdmap e861: 12 total, 12 up, 12 in 2023-09-10T05:51:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23501.054019 secs 2023-09-10T05:51:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23500.400219 secs 2023-09-10T05:51:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:09 smithi082 ceph-mon[98694]: osdmap e861: 12 total, 12 up, 12 in 2023-09-10T05:51:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23501.054019 secs 2023-09-10T05:51:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23500.400219 secs 2023-09-10T05:51:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:09 smithi098 ceph-mon[103124]: osdmap e861: 12 total, 12 up, 12 in 2023-09-10T05:51:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23501.054019 secs 2023-09-10T05:51:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:10 smithi170 ceph-mon[102832]: pgmap v12694: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 277 KiB/s wr, 30 op/s 2023-09-10T05:51:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:10 smithi082 ceph-mon[98694]: pgmap v12694: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 277 KiB/s wr, 30 op/s 2023-09-10T05:51:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:10 smithi098 ceph-mon[103124]: pgmap v12694: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 277 KiB/s wr, 30 op/s 2023-09-10T05:51:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:51:11.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:51:11.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:51:11.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:51:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:51:11.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:51:11.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:51:11.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:51:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:51:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:51:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:51:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:51:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:12 smithi170 ceph-mon[102832]: pgmap v12695: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 277 KiB/s wr, 29 op/s 2023-09-10T05:51:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:12 smithi082 ceph-mon[98694]: pgmap v12695: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 277 KiB/s wr, 29 op/s 2023-09-10T05:51:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:12 smithi098 ceph-mon[103124]: pgmap v12695: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 277 KiB/s wr, 29 op/s 2023-09-10T05:51:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23504.429646 secs 2023-09-10T05:51:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23504.429646 secs 2023-09-10T05:51:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23504.429646 secs 2023-09-10T05:51:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23505.400296 secs 2023-09-10T05:51:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:14 smithi170 ceph-mon[102832]: pgmap v12696: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 277 KiB/s wr, 29 op/s 2023-09-10T05:51:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23506.054086 secs 2023-09-10T05:51:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23505.400296 secs 2023-09-10T05:51:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:14 smithi082 ceph-mon[98694]: pgmap v12696: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 277 KiB/s wr, 29 op/s 2023-09-10T05:51:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23506.054086 secs 2023-09-10T05:51:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23505.400296 secs 2023-09-10T05:51:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:14 smithi098 ceph-mon[103124]: pgmap v12696: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 277 KiB/s wr, 29 op/s 2023-09-10T05:51:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23506.054086 secs 2023-09-10T05:51:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:16 smithi170 ceph-mon[102832]: osdmap e862: 12 total, 12 up, 12 in 2023-09-10T05:51:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:16 smithi170 ceph-mon[102832]: pgmap v12698: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T05:51:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:16 smithi082 ceph-mon[98694]: osdmap e862: 12 total, 12 up, 12 in 2023-09-10T05:51:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:16 smithi082 ceph-mon[98694]: pgmap v12698: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T05:51:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:16 smithi098 ceph-mon[103124]: osdmap e862: 12 total, 12 up, 12 in 2023-09-10T05:51:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:16 smithi098 ceph-mon[103124]: pgmap v12698: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T05:51:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:18 smithi170 ceph-mon[102832]: pgmap v12699: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T05:51:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23509.429767 secs 2023-09-10T05:51:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:18 smithi082 ceph-mon[98694]: pgmap v12699: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T05:51:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23509.429767 secs 2023-09-10T05:51:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:18 smithi098 ceph-mon[103124]: pgmap v12699: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T05:51:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23509.429767 secs 2023-09-10T05:51:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23510.400381 secs 2023-09-10T05:51:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23511.054150 secs 2023-09-10T05:51:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23510.400381 secs 2023-09-10T05:51:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23511.054150 secs 2023-09-10T05:51:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23510.400381 secs 2023-09-10T05:51:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23511.054150 secs 2023-09-10T05:51:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:20 smithi170 ceph-mon[102832]: pgmap v12700: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T05:51:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:20 smithi082 ceph-mon[98694]: pgmap v12700: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T05:51:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:20 smithi098 ceph-mon[103124]: pgmap v12700: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T05:51:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:22 smithi170 ceph-mon[102832]: pgmap v12701: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T05:51:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:22 smithi082 ceph-mon[98694]: pgmap v12701: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T05:51:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:22 smithi098 ceph-mon[103124]: pgmap v12701: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T05:51:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23514.429834 secs 2023-09-10T05:51:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23514.429834 secs 2023-09-10T05:51:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23514.429834 secs 2023-09-10T05:51:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23515.400469 secs 2023-09-10T05:51:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:24 smithi170 ceph-mon[102832]: pgmap v12702: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T05:51:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23516.054218 secs 2023-09-10T05:51:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23515.400469 secs 2023-09-10T05:51:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:24 smithi082 ceph-mon[98694]: pgmap v12702: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T05:51:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23516.054218 secs 2023-09-10T05:51:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23515.400469 secs 2023-09-10T05:51:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:24 smithi098 ceph-mon[103124]: pgmap v12702: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T05:51:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23516.054218 secs 2023-09-10T05:51:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:26 smithi170 ceph-mon[102832]: pgmap v12703: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 181 KiB/s wr, 16 op/s 2023-09-10T05:51:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:26 smithi082 ceph-mon[98694]: pgmap v12703: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 181 KiB/s wr, 16 op/s 2023-09-10T05:51:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:26 smithi098 ceph-mon[103124]: pgmap v12703: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 181 KiB/s wr, 16 op/s 2023-09-10T05:51:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:28 smithi170 ceph-mon[102832]: pgmap v12704: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:51:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:28 smithi082 ceph-mon[98694]: pgmap v12704: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:51:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:28 smithi098 ceph-mon[103124]: pgmap v12704: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:51:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23520.400523 secs 2023-09-10T05:51:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23521.054285 secs 2023-09-10T05:51:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23520.400523 secs 2023-09-10T05:51:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23521.054285 secs 2023-09-10T05:51:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23520.400523 secs 2023-09-10T05:51:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23521.054285 secs 2023-09-10T05:51:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:30 smithi170 ceph-mon[102832]: pgmap v12705: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 10 op/s 2023-09-10T05:51:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:30 smithi082 ceph-mon[98694]: pgmap v12705: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 10 op/s 2023-09-10T05:51:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:30 smithi098 ceph-mon[103124]: pgmap v12705: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 10 op/s 2023-09-10T05:51:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:51:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:51:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:51:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:32 smithi170 ceph-mon[102832]: pgmap v12706: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 15 op/s 2023-09-10T05:51:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:51:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:51:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:51:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:32 smithi082 ceph-mon[98694]: pgmap v12706: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 15 op/s 2023-09-10T05:51:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:51:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:51:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:51:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:32 smithi098 ceph-mon[103124]: pgmap v12706: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 15 op/s 2023-09-10T05:51:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:51:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:51:32.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:51:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23525.400585 secs 2023-09-10T05:51:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:34 smithi170 ceph-mon[102832]: pgmap v12707: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 154 KiB/s wr, 10 op/s 2023-09-10T05:51:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:51:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:51:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23526.054349 secs 2023-09-10T05:51:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23525.400585 secs 2023-09-10T05:51:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:34 smithi082 ceph-mon[98694]: pgmap v12707: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 154 KiB/s wr, 10 op/s 2023-09-10T05:51:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:51:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:51:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23526.054349 secs 2023-09-10T05:51:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23525.400585 secs 2023-09-10T05:51:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:34 smithi098 ceph-mon[103124]: pgmap v12707: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 154 KiB/s wr, 10 op/s 2023-09-10T05:51:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:51:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:51:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23526.054349 secs 2023-09-10T05:51:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:36 smithi170 ceph-mon[102832]: pgmap v12708: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 531 KiB/s wr, 17 op/s 2023-09-10T05:51:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:51:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:51:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:36 smithi082 ceph-mon[98694]: pgmap v12708: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 531 KiB/s wr, 17 op/s 2023-09-10T05:51:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:51:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:51:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:36 smithi098 ceph-mon[103124]: pgmap v12708: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 531 KiB/s wr, 17 op/s 2023-09-10T05:51:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:51:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:51:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:38 smithi170 ceph-mon[102832]: pgmap v12709: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 489 KiB/s wr, 14 op/s 2023-09-10T05:51:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:51:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:51:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:38 smithi082 ceph-mon[98694]: pgmap v12709: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 489 KiB/s wr, 14 op/s 2023-09-10T05:51:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:51:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:51:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:38 smithi098 ceph-mon[103124]: pgmap v12709: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 489 KiB/s wr, 14 op/s 2023-09-10T05:51:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:51:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:51:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23530.400643 secs 2023-09-10T05:51:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23531.054414 secs 2023-09-10T05:51:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23530.400643 secs 2023-09-10T05:51:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23531.054414 secs 2023-09-10T05:51:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23530.400643 secs 2023-09-10T05:51:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23531.054414 secs 2023-09-10T05:51:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:40 smithi170 ceph-mon[102832]: pgmap v12710: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 488 KiB/s wr, 13 op/s 2023-09-10T05:51:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:40 smithi082 ceph-mon[98694]: pgmap v12710: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 488 KiB/s wr, 13 op/s 2023-09-10T05:51:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:40 smithi098 ceph-mon[103124]: pgmap v12710: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 488 KiB/s wr, 13 op/s 2023-09-10T05:51:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:42 smithi170 ceph-mon[102832]: pgmap v12711: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 503 KiB/s wr, 18 op/s 2023-09-10T05:51:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:42 smithi082 ceph-mon[98694]: pgmap v12711: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 503 KiB/s wr, 18 op/s 2023-09-10T05:51:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:42 smithi098 ceph-mon[103124]: pgmap v12711: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 503 KiB/s wr, 18 op/s 2023-09-10T05:51:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23535.400703 secs 2023-09-10T05:51:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:44 smithi170 ceph-mon[102832]: pgmap v12712: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 428 KiB/s wr, 13 op/s 2023-09-10T05:51:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23536.054483 secs 2023-09-10T05:51:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23535.400703 secs 2023-09-10T05:51:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:44 smithi082 ceph-mon[98694]: pgmap v12712: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 428 KiB/s wr, 13 op/s 2023-09-10T05:51:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23536.054483 secs 2023-09-10T05:51:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23535.400703 secs 2023-09-10T05:51:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:44 smithi098 ceph-mon[103124]: pgmap v12712: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 428 KiB/s wr, 13 op/s 2023-09-10T05:51:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23536.054483 secs 2023-09-10T05:51:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:46 smithi082 ceph-mon[98694]: pgmap v12713: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 684 KiB/s wr, 18 op/s 2023-09-10T05:51:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:46 smithi098 ceph-mon[103124]: pgmap v12713: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 684 KiB/s wr, 18 op/s 2023-09-10T05:51:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:46 smithi170 ceph-mon[102832]: pgmap v12713: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 684 KiB/s wr, 18 op/s 2023-09-10T05:51:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:48 smithi170 ceph-mon[102832]: pgmap v12714: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 326 KiB/s wr, 13 op/s 2023-09-10T05:51:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:48 smithi082 ceph-mon[98694]: pgmap v12714: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 326 KiB/s wr, 13 op/s 2023-09-10T05:51:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:48 smithi098 ceph-mon[103124]: pgmap v12714: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 326 KiB/s wr, 13 op/s 2023-09-10T05:51:49.342 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23540.400792 secs 2023-09-10T05:51:49.342 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23541.054549 secs 2023-09-10T05:51:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23540.400792 secs 2023-09-10T05:51:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23541.054549 secs 2023-09-10T05:51:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23540.400792 secs 2023-09-10T05:51:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23541.054549 secs 2023-09-10T05:51:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:50 smithi170 ceph-mon[102832]: pgmap v12715: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 290 KiB/s wr, 11 op/s 2023-09-10T05:51:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:51:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:51:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:50 smithi082 ceph-mon[98694]: pgmap v12715: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 290 KiB/s wr, 11 op/s 2023-09-10T05:51:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:51:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:51:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:50 smithi098 ceph-mon[103124]: pgmap v12715: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 290 KiB/s wr, 11 op/s 2023-09-10T05:51:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:51:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:51:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:52 smithi170 ceph-mon[102832]: pgmap v12716: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 588 KiB/s wr, 18 op/s 2023-09-10T05:51:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:51:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:51:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:52 smithi082 ceph-mon[98694]: pgmap v12716: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 588 KiB/s wr, 18 op/s 2023-09-10T05:51:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:51:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:51:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:52 smithi098 ceph-mon[103124]: pgmap v12716: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 588 KiB/s wr, 18 op/s 2023-09-10T05:51:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:51:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:51:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23544.430337 secs 2023-09-10T05:51:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23544.430337 secs 2023-09-10T05:51:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23544.430337 secs 2023-09-10T05:51:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23545.400850 secs 2023-09-10T05:51:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:54 smithi170 ceph-mon[102832]: pgmap v12717: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 573 KiB/s wr, 12 op/s 2023-09-10T05:51:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23546.054614 secs 2023-09-10T05:51:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23545.400850 secs 2023-09-10T05:51:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:54 smithi082 ceph-mon[98694]: pgmap v12717: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 573 KiB/s wr, 12 op/s 2023-09-10T05:51:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23546.054614 secs 2023-09-10T05:51:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23545.400850 secs 2023-09-10T05:51:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:54 smithi098 ceph-mon[103124]: pgmap v12717: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 573 KiB/s wr, 12 op/s 2023-09-10T05:51:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23546.054614 secs 2023-09-10T05:51:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:56 smithi082 ceph-mon[98694]: pgmap v12718: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 646 KiB/s wr, 17 op/s 2023-09-10T05:51:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:51:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:51:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:51:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:56 smithi098 ceph-mon[103124]: pgmap v12718: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 646 KiB/s wr, 17 op/s 2023-09-10T05:51:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:51:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:51:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:51:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:56 smithi170 ceph-mon[102832]: pgmap v12718: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 646 KiB/s wr, 17 op/s 2023-09-10T05:51:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:51:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:51:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:51:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:51:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:51:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:58 smithi082 ceph-mon[98694]: pgmap v12719: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 390 KiB/s wr, 13 op/s 2023-09-10T05:51:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23549.430398 secs 2023-09-10T05:51:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:51:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:51:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:58 smithi098 ceph-mon[103124]: pgmap v12719: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 390 KiB/s wr, 13 op/s 2023-09-10T05:51:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23549.430398 secs 2023-09-10T05:51:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:51:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:51:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:58 smithi170 ceph-mon[102832]: pgmap v12719: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 390 KiB/s wr, 13 op/s 2023-09-10T05:51:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23549.430398 secs 2023-09-10T05:51:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:51:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23550.401052 secs 2023-09-10T05:51:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:51:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23550.401052 secs 2023-09-10T05:51:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:51:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23550.401052 secs 2023-09-10T05:52:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23551.054677 secs 2023-09-10T05:52:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:52:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:52:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:00 smithi082 ceph-mon[98694]: pgmap v12720: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 372 KiB/s wr, 11 op/s 2023-09-10T05:52:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23551.054677 secs 2023-09-10T05:52:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:52:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:52:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:00 smithi098 ceph-mon[103124]: pgmap v12720: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 372 KiB/s wr, 11 op/s 2023-09-10T05:52:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23551.054677 secs 2023-09-10T05:52:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:52:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:52:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:00 smithi170 ceph-mon[102832]: pgmap v12720: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 372 KiB/s wr, 11 op/s 2023-09-10T05:52:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:02 smithi170 ceph-mon[102832]: pgmap v12721: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 467 KiB/s wr, 16 op/s 2023-09-10T05:52:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:02 smithi082 ceph-mon[98694]: pgmap v12721: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 467 KiB/s wr, 16 op/s 2023-09-10T05:52:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:02 smithi098 ceph-mon[103124]: pgmap v12721: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 467 KiB/s wr, 16 op/s 2023-09-10T05:52:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23554.430483 secs 2023-09-10T05:52:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23554.430483 secs 2023-09-10T05:52:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23554.430483 secs 2023-09-10T05:52:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23555.401135 secs 2023-09-10T05:52:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:04 smithi170 ceph-mon[102832]: pgmap v12722: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 168 KiB/s wr, 10 op/s 2023-09-10T05:52:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23556.054745 secs 2023-09-10T05:52:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23555.401135 secs 2023-09-10T05:52:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:04 smithi082 ceph-mon[98694]: pgmap v12722: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 168 KiB/s wr, 10 op/s 2023-09-10T05:52:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23556.054745 secs 2023-09-10T05:52:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23555.401135 secs 2023-09-10T05:52:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:04 smithi098 ceph-mon[103124]: pgmap v12722: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 168 KiB/s wr, 10 op/s 2023-09-10T05:52:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23556.054745 secs 2023-09-10T05:52:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:06 smithi082 ceph-mon[98694]: pgmap v12723: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 274 KiB/s wr, 29 op/s 2023-09-10T05:52:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:06 smithi098 ceph-mon[103124]: pgmap v12723: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 274 KiB/s wr, 29 op/s 2023-09-10T05:52:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:06 smithi170 ceph-mon[102832]: pgmap v12723: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 274 KiB/s wr, 29 op/s 2023-09-10T05:52:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:08 smithi170 ceph-mon[102832]: pgmap v12724: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 222 KiB/s wr, 26 op/s 2023-09-10T05:52:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23559.430606 secs 2023-09-10T05:52:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:08 smithi082 ceph-mon[98694]: pgmap v12724: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 222 KiB/s wr, 26 op/s 2023-09-10T05:52:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23559.430606 secs 2023-09-10T05:52:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:08 smithi098 ceph-mon[103124]: pgmap v12724: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 222 KiB/s wr, 26 op/s 2023-09-10T05:52:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23559.430606 secs 2023-09-10T05:52:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23560.401195 secs 2023-09-10T05:52:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:09 smithi170 ceph-mon[102832]: osdmap e863: 12 total, 12 up, 12 in 2023-09-10T05:52:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23561.054812 secs 2023-09-10T05:52:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23560.401195 secs 2023-09-10T05:52:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:09 smithi082 ceph-mon[98694]: osdmap e863: 12 total, 12 up, 12 in 2023-09-10T05:52:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23561.054812 secs 2023-09-10T05:52:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23560.401195 secs 2023-09-10T05:52:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:09 smithi098 ceph-mon[103124]: osdmap e863: 12 total, 12 up, 12 in 2023-09-10T05:52:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23561.054812 secs 2023-09-10T05:52:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:10 smithi170 ceph-mon[102832]: pgmap v12726: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 266 KiB/s wr, 30 op/s 2023-09-10T05:52:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:10 smithi082 ceph-mon[98694]: pgmap v12726: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 266 KiB/s wr, 30 op/s 2023-09-10T05:52:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:10 smithi098 ceph-mon[103124]: pgmap v12726: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 266 KiB/s wr, 30 op/s 2023-09-10T05:52:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:52:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:52:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:52:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:12 smithi170 ceph-mon[102832]: pgmap v12727: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 172 KiB/s wr, 31 op/s 2023-09-10T05:52:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:52:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:52:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:52:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:12 smithi082 ceph-mon[98694]: pgmap v12727: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 172 KiB/s wr, 31 op/s 2023-09-10T05:52:12.752 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:52:12.753 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:52:12.753 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:52:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:12 smithi098 ceph-mon[103124]: pgmap v12727: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 172 KiB/s wr, 31 op/s 2023-09-10T05:52:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:52:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:52:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:52:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23564.430704 secs 2023-09-10T05:52:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23564.430704 secs 2023-09-10T05:52:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23564.430704 secs 2023-09-10T05:52:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23565.401253 secs 2023-09-10T05:52:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:14 smithi170 ceph-mon[102832]: pgmap v12728: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 172 KiB/s wr, 31 op/s 2023-09-10T05:52:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23566.054879 secs 2023-09-10T05:52:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23565.401253 secs 2023-09-10T05:52:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:14 smithi082 ceph-mon[98694]: pgmap v12728: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 172 KiB/s wr, 31 op/s 2023-09-10T05:52:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23566.054879 secs 2023-09-10T05:52:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23565.401253 secs 2023-09-10T05:52:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:14 smithi098 ceph-mon[103124]: pgmap v12728: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 172 KiB/s wr, 31 op/s 2023-09-10T05:52:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23566.054879 secs 2023-09-10T05:52:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:16 smithi082 ceph-mon[98694]: osdmap e864: 12 total, 12 up, 12 in 2023-09-10T05:52:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:16 smithi082 ceph-mon[98694]: pgmap v12730: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 116 KiB/s wr, 15 op/s 2023-09-10T05:52:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:16 smithi098 ceph-mon[103124]: osdmap e864: 12 total, 12 up, 12 in 2023-09-10T05:52:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:16 smithi098 ceph-mon[103124]: pgmap v12730: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 116 KiB/s wr, 15 op/s 2023-09-10T05:52:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:16 smithi170 ceph-mon[102832]: osdmap e864: 12 total, 12 up, 12 in 2023-09-10T05:52:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:16 smithi170 ceph-mon[102832]: pgmap v12730: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 116 KiB/s wr, 15 op/s 2023-09-10T05:52:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:18 smithi170 ceph-mon[102832]: pgmap v12731: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 104 KiB/s wr, 15 op/s 2023-09-10T05:52:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23569.430779 secs 2023-09-10T05:52:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:18 smithi082 ceph-mon[98694]: pgmap v12731: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 104 KiB/s wr, 15 op/s 2023-09-10T05:52:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23569.430779 secs 2023-09-10T05:52:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:18 smithi098 ceph-mon[103124]: pgmap v12731: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 104 KiB/s wr, 15 op/s 2023-09-10T05:52:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23569.430779 secs 2023-09-10T05:52:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23570.401336 secs 2023-09-10T05:52:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23571.054945 secs 2023-09-10T05:52:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23570.401336 secs 2023-09-10T05:52:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23571.054945 secs 2023-09-10T05:52:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23570.401336 secs 2023-09-10T05:52:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23571.054945 secs 2023-09-10T05:52:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:20 smithi170 ceph-mon[102832]: pgmap v12732: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 93 KiB/s wr, 13 op/s 2023-09-10T05:52:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:20 smithi082 ceph-mon[98694]: pgmap v12732: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 93 KiB/s wr, 13 op/s 2023-09-10T05:52:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:20 smithi098 ceph-mon[103124]: pgmap v12732: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 93 KiB/s wr, 13 op/s 2023-09-10T05:52:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:22 smithi170 ceph-mon[102832]: pgmap v12733: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 13 op/s 2023-09-10T05:52:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:22 smithi082 ceph-mon[98694]: pgmap v12733: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 13 op/s 2023-09-10T05:52:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:22 smithi098 ceph-mon[103124]: pgmap v12733: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 13 op/s 2023-09-10T05:52:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23574.430866 secs 2023-09-10T05:52:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23574.430866 secs 2023-09-10T05:52:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23574.430866 secs 2023-09-10T05:52:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23575.401409 secs 2023-09-10T05:52:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:24 smithi170 ceph-mon[102832]: pgmap v12734: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 13 op/s 2023-09-10T05:52:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23576.055009 secs 2023-09-10T05:52:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23575.401409 secs 2023-09-10T05:52:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:24 smithi082 ceph-mon[98694]: pgmap v12734: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 13 op/s 2023-09-10T05:52:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23576.055009 secs 2023-09-10T05:52:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23575.401409 secs 2023-09-10T05:52:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:24 smithi098 ceph-mon[103124]: pgmap v12734: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 13 op/s 2023-09-10T05:52:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23576.055009 secs 2023-09-10T05:52:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:26 smithi082 ceph-mon[98694]: pgmap v12735: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 167 KiB/s wr, 18 op/s 2023-09-10T05:52:26.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:26 smithi098 ceph-mon[103124]: pgmap v12735: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 167 KiB/s wr, 18 op/s 2023-09-10T05:52:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:26 smithi170 ceph-mon[102832]: pgmap v12735: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 167 KiB/s wr, 18 op/s 2023-09-10T05:52:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:28 smithi170 ceph-mon[102832]: pgmap v12736: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 12 op/s 2023-09-10T05:52:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:28 smithi082 ceph-mon[98694]: pgmap v12736: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 12 op/s 2023-09-10T05:52:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:28 smithi098 ceph-mon[103124]: pgmap v12736: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 12 op/s 2023-09-10T05:52:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23580.401466 secs 2023-09-10T05:52:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23581.055074 secs 2023-09-10T05:52:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23580.401466 secs 2023-09-10T05:52:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23581.055074 secs 2023-09-10T05:52:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23580.401466 secs 2023-09-10T05:52:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23581.055074 secs 2023-09-10T05:52:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:30 smithi170 ceph-mon[102832]: pgmap v12737: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T05:52:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:52:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:52:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:30 smithi082 ceph-mon[98694]: pgmap v12737: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T05:52:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:52:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:52:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:30 smithi098 ceph-mon[103124]: pgmap v12737: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 11 op/s 2023-09-10T05:52:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:52:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:52:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:52:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:52:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:52:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:32 smithi170 ceph-mon[102832]: pgmap v12738: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 175 KiB/s wr, 16 op/s 2023-09-10T05:52:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:52:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:52:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:52:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:32 smithi082 ceph-mon[98694]: pgmap v12738: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 175 KiB/s wr, 16 op/s 2023-09-10T05:52:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:52:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:52:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:52:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:32 smithi098 ceph-mon[103124]: pgmap v12738: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 175 KiB/s wr, 16 op/s 2023-09-10T05:52:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:52:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:52:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:52:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23585.401493 secs 2023-09-10T05:52:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:34 smithi170 ceph-mon[102832]: pgmap v12739: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 10 op/s 2023-09-10T05:52:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23586.055140 secs 2023-09-10T05:52:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23585.401493 secs 2023-09-10T05:52:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:34 smithi082 ceph-mon[98694]: pgmap v12739: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 10 op/s 2023-09-10T05:52:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23586.055140 secs 2023-09-10T05:52:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23585.401493 secs 2023-09-10T05:52:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:34 smithi098 ceph-mon[103124]: pgmap v12739: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 10 op/s 2023-09-10T05:52:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23586.055140 secs 2023-09-10T05:52:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:36 smithi082 ceph-mon[98694]: pgmap v12740: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 16 op/s 2023-09-10T05:52:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:36 smithi098 ceph-mon[103124]: pgmap v12740: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 16 op/s 2023-09-10T05:52:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:36 smithi170 ceph-mon[102832]: pgmap v12740: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 16 op/s 2023-09-10T05:52:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:38 smithi170 ceph-mon[102832]: pgmap v12741: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T05:52:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:38 smithi082 ceph-mon[98694]: pgmap v12741: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T05:52:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:38 smithi098 ceph-mon[103124]: pgmap v12741: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T05:52:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23590.401551 secs 2023-09-10T05:52:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23591.055208 secs 2023-09-10T05:52:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23590.401551 secs 2023-09-10T05:52:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23591.055208 secs 2023-09-10T05:52:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23590.401551 secs 2023-09-10T05:52:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23591.055208 secs 2023-09-10T05:52:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:40 smithi170 ceph-mon[102832]: pgmap v12742: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 264 KiB/s wr, 11 op/s 2023-09-10T05:52:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:40 smithi082 ceph-mon[98694]: pgmap v12742: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 264 KiB/s wr, 11 op/s 2023-09-10T05:52:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:40 smithi098 ceph-mon[103124]: pgmap v12742: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 264 KiB/s wr, 11 op/s 2023-09-10T05:52:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:42 smithi170 ceph-mon[102832]: pgmap v12743: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 366 KiB/s wr, 17 op/s 2023-09-10T05:52:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:42 smithi082 ceph-mon[98694]: pgmap v12743: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 366 KiB/s wr, 17 op/s 2023-09-10T05:52:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:42 smithi098 ceph-mon[103124]: pgmap v12743: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 366 KiB/s wr, 17 op/s 2023-09-10T05:52:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23595.401609 secs 2023-09-10T05:52:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:44 smithi170 ceph-mon[102832]: pgmap v12744: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 271 KiB/s wr, 12 op/s 2023-09-10T05:52:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:52:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:52:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23596.055273 secs 2023-09-10T05:52:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23595.401609 secs 2023-09-10T05:52:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:44 smithi082 ceph-mon[98694]: pgmap v12744: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 271 KiB/s wr, 12 op/s 2023-09-10T05:52:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:52:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:52:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23596.055273 secs 2023-09-10T05:52:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23595.401609 secs 2023-09-10T05:52:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:44 smithi098 ceph-mon[103124]: pgmap v12744: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 271 KiB/s wr, 12 op/s 2023-09-10T05:52:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:52:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:52:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23596.055273 secs 2023-09-10T05:52:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:46 smithi170 ceph-mon[102832]: pgmap v12745: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 290 KiB/s wr, 16 op/s 2023-09-10T05:52:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:52:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:52:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:46 smithi082 ceph-mon[98694]: pgmap v12745: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 290 KiB/s wr, 16 op/s 2023-09-10T05:52:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:52:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:52:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:46 smithi098 ceph-mon[103124]: pgmap v12745: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 290 KiB/s wr, 16 op/s 2023-09-10T05:52:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:52:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:52:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:48 smithi170 ceph-mon[102832]: pgmap v12746: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 325 KiB/s wr, 13 op/s 2023-09-10T05:52:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:48 smithi082 ceph-mon[98694]: pgmap v12746: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 325 KiB/s wr, 13 op/s 2023-09-10T05:52:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:48 smithi098 ceph-mon[103124]: pgmap v12746: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 325 KiB/s wr, 13 op/s 2023-09-10T05:52:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23600.401670 secs 2023-09-10T05:52:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23601.055338 secs 2023-09-10T05:52:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23600.401670 secs 2023-09-10T05:52:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23601.055338 secs 2023-09-10T05:52:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23600.401670 secs 2023-09-10T05:52:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23601.055338 secs 2023-09-10T05:52:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:50 smithi170 ceph-mon[102832]: pgmap v12747: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 325 KiB/s wr, 12 op/s 2023-09-10T05:52:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:50 smithi082 ceph-mon[98694]: pgmap v12747: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 325 KiB/s wr, 12 op/s 2023-09-10T05:52:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:50 smithi098 ceph-mon[103124]: pgmap v12747: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 325 KiB/s wr, 12 op/s 2023-09-10T05:52:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:52 smithi170 ceph-mon[102832]: pgmap v12748: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 436 KiB/s wr, 18 op/s 2023-09-10T05:52:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:52 smithi082 ceph-mon[98694]: pgmap v12748: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 436 KiB/s wr, 18 op/s 2023-09-10T05:52:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:52 smithi098 ceph-mon[103124]: pgmap v12748: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 436 KiB/s wr, 18 op/s 2023-09-10T05:52:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23605.401746 secs 2023-09-10T05:52:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:54 smithi170 ceph-mon[102832]: pgmap v12749: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 333 KiB/s wr, 11 op/s 2023-09-10T05:52:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:52:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:52:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23606.055406 secs 2023-09-10T05:52:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23605.401746 secs 2023-09-10T05:52:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:54 smithi082 ceph-mon[98694]: pgmap v12749: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 333 KiB/s wr, 11 op/s 2023-09-10T05:52:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:52:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:52:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23606.055406 secs 2023-09-10T05:52:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23605.401746 secs 2023-09-10T05:52:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:54 smithi098 ceph-mon[103124]: pgmap v12749: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 333 KiB/s wr, 11 op/s 2023-09-10T05:52:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:52:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:52:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23606.055406 secs 2023-09-10T05:52:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:56 smithi170 ceph-mon[102832]: pgmap v12750: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 369 KiB/s wr, 16 op/s 2023-09-10T05:52:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:52:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:52:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:56 smithi082 ceph-mon[98694]: pgmap v12750: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 369 KiB/s wr, 16 op/s 2023-09-10T05:52:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:52:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:52:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:56 smithi098 ceph-mon[103124]: pgmap v12750: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 369 KiB/s wr, 16 op/s 2023-09-10T05:52:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:52:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:52:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:58 smithi170 ceph-mon[102832]: pgmap v12751: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 501 KiB/s wr, 13 op/s 2023-09-10T05:52:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23609.431422 secs 2023-09-10T05:52:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:58 smithi082 ceph-mon[98694]: pgmap v12751: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 501 KiB/s wr, 13 op/s 2023-09-10T05:52:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23609.431422 secs 2023-09-10T05:52:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:58 smithi098 ceph-mon[103124]: pgmap v12751: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 501 KiB/s wr, 13 op/s 2023-09-10T05:52:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23609.431422 secs 2023-09-10T05:52:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23610.401804 secs 2023-09-10T05:52:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:52:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23611.055475 secs 2023-09-10T05:52:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23610.401804 secs 2023-09-10T05:52:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:52:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23611.055475 secs 2023-09-10T05:52:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23610.401804 secs 2023-09-10T05:52:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:52:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23611.055475 secs 2023-09-10T05:53:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:00 smithi170 ceph-mon[102832]: pgmap v12752: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 297 KiB/s wr, 12 op/s 2023-09-10T05:53:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:53:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:53:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:00 smithi082 ceph-mon[98694]: pgmap v12752: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 297 KiB/s wr, 12 op/s 2023-09-10T05:53:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:53:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:53:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:00 smithi098 ceph-mon[103124]: pgmap v12752: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 297 KiB/s wr, 12 op/s 2023-09-10T05:53:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:53:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:53:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:02 smithi170 ceph-mon[102832]: pgmap v12753: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 470 KiB/s wr, 27 op/s 2023-09-10T05:53:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:02 smithi082 ceph-mon[98694]: pgmap v12753: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 470 KiB/s wr, 27 op/s 2023-09-10T05:53:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:02 smithi098 ceph-mon[103124]: pgmap v12753: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 470 KiB/s wr, 27 op/s 2023-09-10T05:53:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23614.431482 secs 2023-09-10T05:53:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23614.431482 secs 2023-09-10T05:53:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23614.431482 secs 2023-09-10T05:53:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23615.401860 secs 2023-09-10T05:53:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:04 smithi170 ceph-mon[102832]: pgmap v12754: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 359 KiB/s wr, 22 op/s 2023-09-10T05:53:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:53:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:04 smithi170 ceph-mon[102832]: osdmap e865: 12 total, 12 up, 12 in 2023-09-10T05:53:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23616.055543 secs 2023-09-10T05:53:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23615.401860 secs 2023-09-10T05:53:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:04 smithi082 ceph-mon[98694]: pgmap v12754: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 359 KiB/s wr, 22 op/s 2023-09-10T05:53:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:53:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:04 smithi082 ceph-mon[98694]: osdmap e865: 12 total, 12 up, 12 in 2023-09-10T05:53:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23616.055543 secs 2023-09-10T05:53:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23615.401860 secs 2023-09-10T05:53:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:04 smithi098 ceph-mon[103124]: pgmap v12754: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 359 KiB/s wr, 22 op/s 2023-09-10T05:53:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:53:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:04 smithi098 ceph-mon[103124]: osdmap e865: 12 total, 12 up, 12 in 2023-09-10T05:53:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23616.055543 secs 2023-09-10T05:53:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:06 smithi170 ceph-mon[102832]: pgmap v12756: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 467 KiB/s wr, 33 op/s 2023-09-10T05:53:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:53:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:06 smithi082 ceph-mon[98694]: pgmap v12756: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 467 KiB/s wr, 33 op/s 2023-09-10T05:53:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:53:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:06 smithi098 ceph-mon[103124]: pgmap v12756: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 467 KiB/s wr, 33 op/s 2023-09-10T05:53:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:53:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:08 smithi170 ceph-mon[102832]: pgmap v12757: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 371 KiB/s wr, 32 op/s 2023-09-10T05:53:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23619.431542 secs 2023-09-10T05:53:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:08 smithi082 ceph-mon[98694]: pgmap v12757: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 371 KiB/s wr, 32 op/s 2023-09-10T05:53:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23619.431542 secs 2023-09-10T05:53:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:08 smithi098 ceph-mon[103124]: pgmap v12757: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 371 KiB/s wr, 32 op/s 2023-09-10T05:53:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23619.431542 secs 2023-09-10T05:53:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23620.401919 secs 2023-09-10T05:53:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23621.055607 secs 2023-09-10T05:53:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23620.401919 secs 2023-09-10T05:53:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23621.055607 secs 2023-09-10T05:53:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23620.401919 secs 2023-09-10T05:53:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23621.055607 secs 2023-09-10T05:53:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:10 smithi082 ceph-mon[98694]: pgmap v12758: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 371 KiB/s wr, 32 op/s 2023-09-10T05:53:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:10 smithi082 ceph-mon[98694]: osdmap e866: 12 total, 12 up, 12 in 2023-09-10T05:53:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:10 smithi098 ceph-mon[103124]: pgmap v12758: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 371 KiB/s wr, 32 op/s 2023-09-10T05:53:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:10 smithi098 ceph-mon[103124]: osdmap e866: 12 total, 12 up, 12 in 2023-09-10T05:53:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:10 smithi170 ceph-mon[102832]: pgmap v12758: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 371 KiB/s wr, 32 op/s 2023-09-10T05:53:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:10 smithi170 ceph-mon[102832]: osdmap e866: 12 total, 12 up, 12 in 2023-09-10T05:53:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:12 smithi082 ceph-mon[98694]: pgmap v12760: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 296 KiB/s wr, 24 op/s 2023-09-10T05:53:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:53:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:12 smithi098 ceph-mon[103124]: pgmap v12760: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 296 KiB/s wr, 24 op/s 2023-09-10T05:53:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:53:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:12 smithi170 ceph-mon[102832]: pgmap v12760: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 296 KiB/s wr, 24 op/s 2023-09-10T05:53:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:53:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23624.431652 secs 2023-09-10T05:53:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23624.431652 secs 2023-09-10T05:53:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23624.431652 secs 2023-09-10T05:53:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23625.401999 secs 2023-09-10T05:53:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:14 smithi082 ceph-mon[98694]: pgmap v12761: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 240 KiB/s wr, 20 op/s 2023-09-10T05:53:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23626.055675 secs 2023-09-10T05:53:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23625.401999 secs 2023-09-10T05:53:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:14 smithi098 ceph-mon[103124]: pgmap v12761: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 240 KiB/s wr, 20 op/s 2023-09-10T05:53:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23626.055675 secs 2023-09-10T05:53:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23625.401999 secs 2023-09-10T05:53:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:14 smithi170 ceph-mon[102832]: pgmap v12761: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 240 KiB/s wr, 20 op/s 2023-09-10T05:53:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23626.055675 secs 2023-09-10T05:53:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:16.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:16.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:16.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:16.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:15 smithi098 ceph-mon[103124]: pgmap v12762: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 159 KiB/s wr, 12 op/s 2023-09-10T05:53:16.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:16.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:16.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:53:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:16.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:15 smithi170 ceph-mon[102832]: pgmap v12762: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 159 KiB/s wr, 12 op/s 2023-09-10T05:53:16.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:16.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:16.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:53:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:15 smithi082 ceph-mon[98694]: pgmap v12762: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 159 KiB/s wr, 12 op/s 2023-09-10T05:53:16.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:16.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:16.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:53:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:53:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:53:17.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:53:17.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:53:17.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:16 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T05:53:17.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:16 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T05:53:17.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:17.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:17.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:53:17.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:53:17.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:53:17.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:53:17.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:16 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T05:53:17.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:16 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T05:53:17.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:53:17.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:53:17.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:53:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:53:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:53:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:53:17.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:16 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T05:53:17.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:16 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T05:53:17.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:17.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:17.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:53:17.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:53:17.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:53:17.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:53:17.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:16 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T05:53:17.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:16 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T05:53:17.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:53:17.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:53:17.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:53:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:53:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:53:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:53:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:16 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T05:53:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:16 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T05:53:17.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:17.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:17.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:53:17.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:53:17.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:53:17.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T05:53:17.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:16 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T05:53:17.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:16 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T05:53:17.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:53:17.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:53:17.258 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:53:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:17 smithi170 ceph-mon[102832]: pgmap v12763: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T05:53:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:17 smithi082 ceph-mon[98694]: pgmap v12763: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T05:53:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:17 smithi098 ceph-mon[103124]: pgmap v12763: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T05:53:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23629.431714 secs 2023-09-10T05:53:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23630.402057 secs 2023-09-10T05:53:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23629.431714 secs 2023-09-10T05:53:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23630.402057 secs 2023-09-10T05:53:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23629.431714 secs 2023-09-10T05:53:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23630.402057 secs 2023-09-10T05:53:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23631.055740 secs 2023-09-10T05:53:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:19 smithi170 ceph-mon[102832]: pgmap v12764: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T05:53:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23631.055740 secs 2023-09-10T05:53:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:19 smithi082 ceph-mon[98694]: pgmap v12764: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T05:53:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23631.055740 secs 2023-09-10T05:53:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:19 smithi098 ceph-mon[103124]: pgmap v12764: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 74 KiB/s wr, 11 op/s 2023-09-10T05:53:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:22 smithi170 ceph-mon[102832]: pgmap v12765: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 16 op/s 2023-09-10T05:53:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:22 smithi082 ceph-mon[98694]: pgmap v12765: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 16 op/s 2023-09-10T05:53:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:22 smithi098 ceph-mon[103124]: pgmap v12765: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 16 op/s 2023-09-10T05:53:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23634.431774 secs 2023-09-10T05:53:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23634.431774 secs 2023-09-10T05:53:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23634.431774 secs 2023-09-10T05:53:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23635.402115 secs 2023-09-10T05:53:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:24 smithi170 ceph-mon[102832]: pgmap v12766: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2023-09-10T05:53:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23636.055808 secs 2023-09-10T05:53:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23635.402115 secs 2023-09-10T05:53:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:24 smithi082 ceph-mon[98694]: pgmap v12766: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2023-09-10T05:53:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23636.055808 secs 2023-09-10T05:53:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23635.402115 secs 2023-09-10T05:53:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:24 smithi098 ceph-mon[103124]: pgmap v12766: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2023-09-10T05:53:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23636.055808 secs 2023-09-10T05:53:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:26 smithi098 ceph-mon[103124]: pgmap v12767: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 14 op/s 2023-09-10T05:53:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:26 smithi170 ceph-mon[102832]: pgmap v12767: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 14 op/s 2023-09-10T05:53:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:26 smithi082 ceph-mon[98694]: pgmap v12767: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 14 op/s 2023-09-10T05:53:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:28 smithi170 ceph-mon[102832]: pgmap v12768: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:53:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:28 smithi082 ceph-mon[98694]: pgmap v12768: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:53:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:28 smithi098 ceph-mon[103124]: pgmap v12768: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 80 KiB/s wr, 11 op/s 2023-09-10T05:53:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23640.402174 secs 2023-09-10T05:53:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23641.055874 secs 2023-09-10T05:53:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23640.402174 secs 2023-09-10T05:53:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23641.055874 secs 2023-09-10T05:53:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23640.402174 secs 2023-09-10T05:53:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23641.055874 secs 2023-09-10T05:53:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:30 smithi170 ceph-mon[102832]: pgmap v12769: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T05:53:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:30 smithi082 ceph-mon[98694]: pgmap v12769: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T05:53:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:30 smithi098 ceph-mon[103124]: pgmap v12769: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T05:53:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:53:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:53:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:53:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:32 smithi170 ceph-mon[102832]: pgmap v12770: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 156 KiB/s wr, 15 op/s 2023-09-10T05:53:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:53:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:53:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:32 smithi082 ceph-mon[98694]: pgmap v12770: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 156 KiB/s wr, 15 op/s 2023-09-10T05:53:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:53:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:53:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:32 smithi098 ceph-mon[103124]: pgmap v12770: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 156 KiB/s wr, 15 op/s 2023-09-10T05:53:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:53:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:53:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23645.402233 secs 2023-09-10T05:53:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:34 smithi170 ceph-mon[102832]: pgmap v12771: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 9 op/s 2023-09-10T05:53:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:53:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23646.055936 secs 2023-09-10T05:53:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23645.402233 secs 2023-09-10T05:53:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:34 smithi082 ceph-mon[98694]: pgmap v12771: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 9 op/s 2023-09-10T05:53:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:53:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23646.055936 secs 2023-09-10T05:53:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23645.402233 secs 2023-09-10T05:53:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:34 smithi098 ceph-mon[103124]: pgmap v12771: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 9 op/s 2023-09-10T05:53:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:53:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23646.055936 secs 2023-09-10T05:53:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:36 smithi170 ceph-mon[102832]: pgmap v12772: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 292 KiB/s wr, 14 op/s 2023-09-10T05:53:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:53:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:36 smithi082 ceph-mon[98694]: pgmap v12772: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 292 KiB/s wr, 14 op/s 2023-09-10T05:53:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:53:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:36 smithi098 ceph-mon[103124]: pgmap v12772: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 292 KiB/s wr, 14 op/s 2023-09-10T05:53:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:53:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:38 smithi170 ceph-mon[102832]: pgmap v12773: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 491 KiB/s wr, 11 op/s 2023-09-10T05:53:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:38 smithi082 ceph-mon[98694]: pgmap v12773: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 491 KiB/s wr, 11 op/s 2023-09-10T05:53:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:38 smithi098 ceph-mon[103124]: pgmap v12773: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 491 KiB/s wr, 11 op/s 2023-09-10T05:53:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23650.402290 secs 2023-09-10T05:53:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23651.056002 secs 2023-09-10T05:53:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23650.402290 secs 2023-09-10T05:53:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23651.056002 secs 2023-09-10T05:53:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23650.402290 secs 2023-09-10T05:53:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23651.056002 secs 2023-09-10T05:53:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:40 smithi170 ceph-mon[102832]: pgmap v12774: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 491 KiB/s wr, 11 op/s 2023-09-10T05:53:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:40 smithi082 ceph-mon[98694]: pgmap v12774: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 491 KiB/s wr, 11 op/s 2023-09-10T05:53:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:40 smithi098 ceph-mon[103124]: pgmap v12774: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 491 KiB/s wr, 11 op/s 2023-09-10T05:53:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:42 smithi170 ceph-mon[102832]: pgmap v12775: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 555 KiB/s wr, 17 op/s 2023-09-10T05:53:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:42 smithi082 ceph-mon[98694]: pgmap v12775: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 555 KiB/s wr, 17 op/s 2023-09-10T05:53:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:42 smithi098 ceph-mon[103124]: pgmap v12775: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 555 KiB/s wr, 17 op/s 2023-09-10T05:53:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23655.402353 secs 2023-09-10T05:53:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:44 smithi170 ceph-mon[102832]: pgmap v12776: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 479 KiB/s wr, 12 op/s 2023-09-10T05:53:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:53:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23656.056069 secs 2023-09-10T05:53:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23655.402353 secs 2023-09-10T05:53:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:44 smithi082 ceph-mon[98694]: pgmap v12776: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 479 KiB/s wr, 12 op/s 2023-09-10T05:53:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:53:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23656.056069 secs 2023-09-10T05:53:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23655.402353 secs 2023-09-10T05:53:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:44 smithi098 ceph-mon[103124]: pgmap v12776: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 479 KiB/s wr, 12 op/s 2023-09-10T05:53:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:53:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23656.056069 secs 2023-09-10T05:53:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:46 smithi170 ceph-mon[102832]: pgmap v12777: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 498 KiB/s wr, 16 op/s 2023-09-10T05:53:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:53:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:46 smithi082 ceph-mon[98694]: pgmap v12777: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 498 KiB/s wr, 16 op/s 2023-09-10T05:53:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:53:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:46 smithi098 ceph-mon[103124]: pgmap v12777: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 498 KiB/s wr, 16 op/s 2023-09-10T05:53:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:53:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:48 smithi170 ceph-mon[102832]: pgmap v12778: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 570 KiB/s wr, 14 op/s 2023-09-10T05:53:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:48 smithi082 ceph-mon[98694]: pgmap v12778: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 570 KiB/s wr, 14 op/s 2023-09-10T05:53:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:48 smithi098 ceph-mon[103124]: pgmap v12778: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 570 KiB/s wr, 14 op/s 2023-09-10T05:53:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23660.402448 secs 2023-09-10T05:53:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23661.056136 secs 2023-09-10T05:53:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23660.402448 secs 2023-09-10T05:53:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23661.056136 secs 2023-09-10T05:53:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23660.402448 secs 2023-09-10T05:53:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23661.056136 secs 2023-09-10T05:53:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:50 smithi170 ceph-mon[102832]: pgmap v12779: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 292 KiB/s wr, 12 op/s 2023-09-10T05:53:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:53:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:50 smithi082 ceph-mon[98694]: pgmap v12779: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 292 KiB/s wr, 12 op/s 2023-09-10T05:53:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:53:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:50 smithi098 ceph-mon[103124]: pgmap v12779: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 292 KiB/s wr, 12 op/s 2023-09-10T05:53:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:53:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:52 smithi170 ceph-mon[102832]: pgmap v12780: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 581 KiB/s wr, 19 op/s 2023-09-10T05:53:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:53:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:52 smithi082 ceph-mon[98694]: pgmap v12780: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 581 KiB/s wr, 19 op/s 2023-09-10T05:53:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:53:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:52 smithi098 ceph-mon[103124]: pgmap v12780: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 581 KiB/s wr, 19 op/s 2023-09-10T05:53:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:53:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23664.432185 secs 2023-09-10T05:53:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23664.432185 secs 2023-09-10T05:53:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23664.432185 secs 2023-09-10T05:53:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23665.402495 secs 2023-09-10T05:53:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:54 smithi170 ceph-mon[102832]: pgmap v12781: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 516 KiB/s wr, 13 op/s 2023-09-10T05:53:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23666.056199 secs 2023-09-10T05:53:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23665.402495 secs 2023-09-10T05:53:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:54 smithi082 ceph-mon[98694]: pgmap v12781: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 516 KiB/s wr, 13 op/s 2023-09-10T05:53:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23666.056199 secs 2023-09-10T05:53:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23665.402495 secs 2023-09-10T05:53:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:54 smithi098 ceph-mon[103124]: pgmap v12781: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 516 KiB/s wr, 13 op/s 2023-09-10T05:53:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23666.056199 secs 2023-09-10T05:53:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:56 smithi098 ceph-mon[103124]: pgmap v12782: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 639 KiB/s wr, 17 op/s 2023-09-10T05:53:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:53:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:53:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:56 smithi170 ceph-mon[102832]: pgmap v12782: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 639 KiB/s wr, 17 op/s 2023-09-10T05:53:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:53:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:53:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:56 smithi082 ceph-mon[98694]: pgmap v12782: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 639 KiB/s wr, 17 op/s 2023-09-10T05:53:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:53:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:53:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:53:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:58 smithi170 ceph-mon[102832]: pgmap v12783: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 620 KiB/s wr, 14 op/s 2023-09-10T05:53:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23669.432245 secs 2023-09-10T05:53:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:58 smithi082 ceph-mon[98694]: pgmap v12783: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 620 KiB/s wr, 14 op/s 2023-09-10T05:53:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23669.432245 secs 2023-09-10T05:53:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:58 smithi098 ceph-mon[103124]: pgmap v12783: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 620 KiB/s wr, 14 op/s 2023-09-10T05:53:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23669.432245 secs 2023-09-10T05:53:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23670.402518 secs 2023-09-10T05:53:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:53:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23671.056264 secs 2023-09-10T05:53:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23670.402518 secs 2023-09-10T05:53:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:53:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23671.056264 secs 2023-09-10T05:53:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23670.402518 secs 2023-09-10T05:53:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:53:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23671.056264 secs 2023-09-10T05:54:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:00 smithi170 ceph-mon[102832]: pgmap v12784: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 411 KiB/s wr, 12 op/s 2023-09-10T05:54:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:00 smithi082 ceph-mon[98694]: pgmap v12784: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 411 KiB/s wr, 12 op/s 2023-09-10T05:54:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:00 smithi098 ceph-mon[103124]: pgmap v12784: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 411 KiB/s wr, 12 op/s 2023-09-10T05:54:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:02 smithi170 ceph-mon[102832]: pgmap v12785: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 580 KiB/s wr, 17 op/s 2023-09-10T05:54:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:02 smithi082 ceph-mon[98694]: pgmap v12785: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 580 KiB/s wr, 17 op/s 2023-09-10T05:54:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:02 smithi098 ceph-mon[103124]: pgmap v12785: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 580 KiB/s wr, 17 op/s 2023-09-10T05:54:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23674.432304 secs 2023-09-10T05:54:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23674.432304 secs 2023-09-10T05:54:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23674.432304 secs 2023-09-10T05:54:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23675.402580 secs 2023-09-10T05:54:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:04 smithi170 ceph-mon[102832]: pgmap v12786: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 292 KiB/s wr, 10 op/s 2023-09-10T05:54:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:54:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:54:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23676.056332 secs 2023-09-10T05:54:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23675.402580 secs 2023-09-10T05:54:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:04 smithi082 ceph-mon[98694]: pgmap v12786: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 292 KiB/s wr, 10 op/s 2023-09-10T05:54:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:54:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:54:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23676.056332 secs 2023-09-10T05:54:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23675.402580 secs 2023-09-10T05:54:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:04 smithi098 ceph-mon[103124]: pgmap v12786: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 292 KiB/s wr, 10 op/s 2023-09-10T05:54:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:54:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:54:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23676.056332 secs 2023-09-10T05:54:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:06 smithi170 ceph-mon[102832]: pgmap v12787: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 354 KiB/s wr, 29 op/s 2023-09-10T05:54:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:54:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:54:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:06 smithi082 ceph-mon[98694]: pgmap v12787: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 354 KiB/s wr, 29 op/s 2023-09-10T05:54:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:54:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:54:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:06 smithi098 ceph-mon[103124]: pgmap v12787: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 354 KiB/s wr, 29 op/s 2023-09-10T05:54:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:54:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:54:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:08 smithi170 ceph-mon[102832]: pgmap v12788: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 293 KiB/s wr, 26 op/s 2023-09-10T05:54:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23679.432366 secs 2023-09-10T05:54:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:08 smithi082 ceph-mon[98694]: pgmap v12788: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 293 KiB/s wr, 26 op/s 2023-09-10T05:54:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23679.432366 secs 2023-09-10T05:54:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:08 smithi098 ceph-mon[103124]: pgmap v12788: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 293 KiB/s wr, 26 op/s 2023-09-10T05:54:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23679.432366 secs 2023-09-10T05:54:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23680.402648 secs 2023-09-10T05:54:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:09 smithi170 ceph-mon[102832]: osdmap e867: 12 total, 12 up, 12 in 2023-09-10T05:54:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23681.056396 secs 2023-09-10T05:54:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23680.402648 secs 2023-09-10T05:54:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:09 smithi082 ceph-mon[98694]: osdmap e867: 12 total, 12 up, 12 in 2023-09-10T05:54:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23681.056396 secs 2023-09-10T05:54:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23680.402648 secs 2023-09-10T05:54:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:09 smithi098 ceph-mon[103124]: osdmap e867: 12 total, 12 up, 12 in 2023-09-10T05:54:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23681.056396 secs 2023-09-10T05:54:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:10 smithi170 ceph-mon[102832]: pgmap v12790: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 352 KiB/s wr, 30 op/s 2023-09-10T05:54:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:10 smithi082 ceph-mon[98694]: pgmap v12790: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 352 KiB/s wr, 30 op/s 2023-09-10T05:54:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:10 smithi098 ceph-mon[103124]: pgmap v12790: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 352 KiB/s wr, 30 op/s 2023-09-10T05:54:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:12 smithi170 ceph-mon[102832]: pgmap v12791: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 171 KiB/s wr, 30 op/s 2023-09-10T05:54:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:12 smithi082 ceph-mon[98694]: pgmap v12791: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 171 KiB/s wr, 30 op/s 2023-09-10T05:54:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:12 smithi098 ceph-mon[103124]: pgmap v12791: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 171 KiB/s wr, 30 op/s 2023-09-10T05:54:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23684.432428 secs 2023-09-10T05:54:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23684.432428 secs 2023-09-10T05:54:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23684.432428 secs 2023-09-10T05:54:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23685.402708 secs 2023-09-10T05:54:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:14 smithi170 ceph-mon[102832]: pgmap v12792: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 171 KiB/s wr, 30 op/s 2023-09-10T05:54:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23686.056463 secs 2023-09-10T05:54:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23685.402708 secs 2023-09-10T05:54:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:14 smithi082 ceph-mon[98694]: pgmap v12792: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 171 KiB/s wr, 30 op/s 2023-09-10T05:54:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23686.056463 secs 2023-09-10T05:54:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23685.402708 secs 2023-09-10T05:54:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:14 smithi098 ceph-mon[103124]: pgmap v12792: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 171 KiB/s wr, 30 op/s 2023-09-10T05:54:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23686.056463 secs 2023-09-10T05:54:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:16 smithi098 ceph-mon[103124]: osdmap e868: 12 total, 12 up, 12 in 2023-09-10T05:54:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:16 smithi098 ceph-mon[103124]: pgmap v12794: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 13 op/s 2023-09-10T05:54:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:16 smithi170 ceph-mon[102832]: osdmap e868: 12 total, 12 up, 12 in 2023-09-10T05:54:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:16 smithi170 ceph-mon[102832]: pgmap v12794: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 13 op/s 2023-09-10T05:54:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:16 smithi082 ceph-mon[98694]: osdmap e868: 12 total, 12 up, 12 in 2023-09-10T05:54:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:16 smithi082 ceph-mon[98694]: pgmap v12794: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 13 op/s 2023-09-10T05:54:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:54:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:54:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:54:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:54:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:54:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:54:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:54:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:54:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:54:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:54:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:54:17.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:54:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:18 smithi098 ceph-mon[103124]: pgmap v12795: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T05:54:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23689.432489 secs 2023-09-10T05:54:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:18 smithi170 ceph-mon[102832]: pgmap v12795: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T05:54:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23689.432489 secs 2023-09-10T05:54:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:18 smithi082 ceph-mon[98694]: pgmap v12795: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T05:54:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23689.432489 secs 2023-09-10T05:54:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23690.402767 secs 2023-09-10T05:54:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23691.056493 secs 2023-09-10T05:54:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23690.402767 secs 2023-09-10T05:54:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23691.056493 secs 2023-09-10T05:54:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23690.402767 secs 2023-09-10T05:54:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23691.056493 secs 2023-09-10T05:54:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:20 smithi098 ceph-mon[103124]: pgmap v12796: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T05:54:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:20 smithi170 ceph-mon[102832]: pgmap v12796: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T05:54:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:20 smithi082 ceph-mon[98694]: pgmap v12796: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T05:54:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:22 smithi170 ceph-mon[102832]: pgmap v12797: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T05:54:22.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:22 smithi082 ceph-mon[98694]: pgmap v12797: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T05:54:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:22 smithi098 ceph-mon[103124]: pgmap v12797: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T05:54:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23694.432548 secs 2023-09-10T05:54:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23694.432548 secs 2023-09-10T05:54:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23694.432548 secs 2023-09-10T05:54:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23695.402854 secs 2023-09-10T05:54:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:24 smithi170 ceph-mon[102832]: pgmap v12798: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T05:54:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23696.056558 secs 2023-09-10T05:54:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23695.402854 secs 2023-09-10T05:54:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:24 smithi082 ceph-mon[98694]: pgmap v12798: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T05:54:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23696.056558 secs 2023-09-10T05:54:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23695.402854 secs 2023-09-10T05:54:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:24 smithi098 ceph-mon[103124]: pgmap v12798: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T05:54:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23696.056558 secs 2023-09-10T05:54:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:26 smithi098 ceph-mon[103124]: pgmap v12799: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 95 KiB/s wr, 15 op/s 2023-09-10T05:54:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:26 smithi170 ceph-mon[102832]: pgmap v12799: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 95 KiB/s wr, 15 op/s 2023-09-10T05:54:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:26 smithi082 ceph-mon[98694]: pgmap v12799: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 95 KiB/s wr, 15 op/s 2023-09-10T05:54:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:28 smithi170 ceph-mon[102832]: pgmap v12800: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T05:54:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:28 smithi082 ceph-mon[98694]: pgmap v12800: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T05:54:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:28 smithi098 ceph-mon[103124]: pgmap v12800: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T05:54:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23700.402939 secs 2023-09-10T05:54:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23701.056623 secs 2023-09-10T05:54:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23700.402939 secs 2023-09-10T05:54:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23701.056623 secs 2023-09-10T05:54:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23700.402939 secs 2023-09-10T05:54:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23701.056623 secs 2023-09-10T05:54:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:30 smithi170 ceph-mon[102832]: pgmap v12801: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 80 KiB/s wr, 9 op/s 2023-09-10T05:54:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:30 smithi082 ceph-mon[98694]: pgmap v12801: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 80 KiB/s wr, 9 op/s 2023-09-10T05:54:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:30 smithi098 ceph-mon[103124]: pgmap v12801: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 80 KiB/s wr, 9 op/s 2023-09-10T05:54:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:54:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:54:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:54:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:32 smithi170 ceph-mon[102832]: pgmap v12802: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 212 KiB/s wr, 14 op/s 2023-09-10T05:54:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:54:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:54:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:54:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:32 smithi082 ceph-mon[98694]: pgmap v12802: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 212 KiB/s wr, 14 op/s 2023-09-10T05:54:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:54:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:54:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:54:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:32 smithi098 ceph-mon[103124]: pgmap v12802: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 212 KiB/s wr, 14 op/s 2023-09-10T05:54:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:54:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:54:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:54:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23705.402998 secs 2023-09-10T05:54:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:34 smithi170 ceph-mon[102832]: pgmap v12803: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 132 KiB/s wr, 9 op/s 2023-09-10T05:54:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23706.056690 secs 2023-09-10T05:54:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23705.402998 secs 2023-09-10T05:54:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:34 smithi082 ceph-mon[98694]: pgmap v12803: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 132 KiB/s wr, 9 op/s 2023-09-10T05:54:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23706.056690 secs 2023-09-10T05:54:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23705.402998 secs 2023-09-10T05:54:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:34 smithi098 ceph-mon[103124]: pgmap v12803: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 132 KiB/s wr, 9 op/s 2023-09-10T05:54:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23706.056690 secs 2023-09-10T05:54:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:36 smithi098 ceph-mon[103124]: pgmap v12804: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 266 KiB/s wr, 14 op/s 2023-09-10T05:54:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:36 smithi170 ceph-mon[102832]: pgmap v12804: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 266 KiB/s wr, 14 op/s 2023-09-10T05:54:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:36 smithi082 ceph-mon[98694]: pgmap v12804: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 266 KiB/s wr, 14 op/s 2023-09-10T05:54:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:54:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:54:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:38 smithi098 ceph-mon[103124]: pgmap v12805: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 266 KiB/s wr, 11 op/s 2023-09-10T05:54:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:54:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:54:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:38 smithi170 ceph-mon[102832]: pgmap v12805: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 266 KiB/s wr, 11 op/s 2023-09-10T05:54:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:54:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:54:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:38 smithi082 ceph-mon[98694]: pgmap v12805: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 266 KiB/s wr, 11 op/s 2023-09-10T05:54:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23710.403079 secs 2023-09-10T05:54:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23711.056757 secs 2023-09-10T05:54:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23710.403079 secs 2023-09-10T05:54:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23711.056757 secs 2023-09-10T05:54:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23710.403079 secs 2023-09-10T05:54:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23711.056757 secs 2023-09-10T05:54:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:54:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:54:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:40 smithi098 ceph-mon[103124]: pgmap v12806: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 266 KiB/s wr, 11 op/s 2023-09-10T05:54:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:54:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:54:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:40 smithi170 ceph-mon[102832]: pgmap v12806: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 266 KiB/s wr, 11 op/s 2023-09-10T05:54:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:54:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:54:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:40 smithi082 ceph-mon[98694]: pgmap v12806: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 266 KiB/s wr, 11 op/s 2023-09-10T05:54:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:42 smithi170 ceph-mon[102832]: pgmap v12807: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 385 KiB/s wr, 17 op/s 2023-09-10T05:54:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:42 smithi082 ceph-mon[98694]: pgmap v12807: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 385 KiB/s wr, 17 op/s 2023-09-10T05:54:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:42 smithi098 ceph-mon[103124]: pgmap v12807: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 385 KiB/s wr, 17 op/s 2023-09-10T05:54:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23715.403131 secs 2023-09-10T05:54:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:44 smithi170 ceph-mon[102832]: pgmap v12808: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 253 KiB/s wr, 12 op/s 2023-09-10T05:54:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23716.056821 secs 2023-09-10T05:54:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23715.403131 secs 2023-09-10T05:54:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:44 smithi082 ceph-mon[98694]: pgmap v12808: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 253 KiB/s wr, 12 op/s 2023-09-10T05:54:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23716.056821 secs 2023-09-10T05:54:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23715.403131 secs 2023-09-10T05:54:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:44 smithi098 ceph-mon[103124]: pgmap v12808: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 253 KiB/s wr, 12 op/s 2023-09-10T05:54:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23716.056821 secs 2023-09-10T05:54:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:46 smithi098 ceph-mon[103124]: pgmap v12809: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 462 KiB/s wr, 18 op/s 2023-09-10T05:54:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:46 smithi170 ceph-mon[102832]: pgmap v12809: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 462 KiB/s wr, 18 op/s 2023-09-10T05:54:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:46 smithi082 ceph-mon[98694]: pgmap v12809: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 462 KiB/s wr, 18 op/s 2023-09-10T05:54:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:48 smithi170 ceph-mon[102832]: pgmap v12810: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 328 KiB/s wr, 14 op/s 2023-09-10T05:54:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:48 smithi082 ceph-mon[98694]: pgmap v12810: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 328 KiB/s wr, 14 op/s 2023-09-10T05:54:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:48 smithi098 ceph-mon[103124]: pgmap v12810: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 328 KiB/s wr, 14 op/s 2023-09-10T05:54:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23720.403190 secs 2023-09-10T05:54:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23721.056913 secs 2023-09-10T05:54:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23720.403190 secs 2023-09-10T05:54:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23721.056913 secs 2023-09-10T05:54:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23720.403190 secs 2023-09-10T05:54:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23721.056913 secs 2023-09-10T05:54:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:50 smithi170 ceph-mon[102832]: pgmap v12811: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 328 KiB/s wr, 12 op/s 2023-09-10T05:54:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:54:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:54:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:50 smithi082 ceph-mon[98694]: pgmap v12811: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 328 KiB/s wr, 12 op/s 2023-09-10T05:54:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:54:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:54:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:50 smithi098 ceph-mon[103124]: pgmap v12811: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 328 KiB/s wr, 12 op/s 2023-09-10T05:54:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:54:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:54:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:52 smithi170 ceph-mon[102832]: pgmap v12812: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 440 KiB/s wr, 19 op/s 2023-09-10T05:54:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:54:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:54:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:52 smithi082 ceph-mon[98694]: pgmap v12812: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 440 KiB/s wr, 19 op/s 2023-09-10T05:54:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:54:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:54:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:52 smithi098 ceph-mon[103124]: pgmap v12812: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 440 KiB/s wr, 19 op/s 2023-09-10T05:54:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:54:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:54:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23725.403259 secs 2023-09-10T05:54:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:54 smithi170 ceph-mon[102832]: pgmap v12813: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 320 KiB/s wr, 12 op/s 2023-09-10T05:54:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23726.056977 secs 2023-09-10T05:54:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23725.403259 secs 2023-09-10T05:54:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:54 smithi082 ceph-mon[98694]: pgmap v12813: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 320 KiB/s wr, 12 op/s 2023-09-10T05:54:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23726.056977 secs 2023-09-10T05:54:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23725.403259 secs 2023-09-10T05:54:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:54 smithi098 ceph-mon[103124]: pgmap v12813: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 320 KiB/s wr, 12 op/s 2023-09-10T05:54:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23726.056977 secs 2023-09-10T05:54:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:56 smithi098 ceph-mon[103124]: pgmap v12814: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 508 KiB/s wr, 19 op/s 2023-09-10T05:54:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:56 smithi170 ceph-mon[102832]: pgmap v12814: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 508 KiB/s wr, 19 op/s 2023-09-10T05:54:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:56 smithi082 ceph-mon[98694]: pgmap v12814: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 508 KiB/s wr, 19 op/s 2023-09-10T05:54:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:58 smithi170 ceph-mon[102832]: pgmap v12815: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 299 KiB/s wr, 12 op/s 2023-09-10T05:54:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23729.433094 secs 2023-09-10T05:54:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:58 smithi082 ceph-mon[98694]: pgmap v12815: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 299 KiB/s wr, 12 op/s 2023-09-10T05:54:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23729.433094 secs 2023-09-10T05:54:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:58 smithi098 ceph-mon[103124]: pgmap v12815: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 299 KiB/s wr, 12 op/s 2023-09-10T05:54:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23729.433094 secs 2023-09-10T05:54:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23730.403319 secs 2023-09-10T05:54:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:54:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23731.057042 secs 2023-09-10T05:54:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23730.403319 secs 2023-09-10T05:54:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:54:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23731.057042 secs 2023-09-10T05:54:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23730.403319 secs 2023-09-10T05:54:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:54:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23731.057042 secs 2023-09-10T05:55:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:00 smithi170 ceph-mon[102832]: pgmap v12816: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 299 KiB/s wr, 12 op/s 2023-09-10T05:55:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:55:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:55:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:00 smithi082 ceph-mon[98694]: pgmap v12816: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 299 KiB/s wr, 12 op/s 2023-09-10T05:55:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:55:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:55:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:00 smithi098 ceph-mon[103124]: pgmap v12816: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 299 KiB/s wr, 12 op/s 2023-09-10T05:55:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:55:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:55:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:02 smithi170 ceph-mon[102832]: pgmap v12817: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 462 KiB/s wr, 19 op/s 2023-09-10T05:55:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:02 smithi082 ceph-mon[98694]: pgmap v12817: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 462 KiB/s wr, 19 op/s 2023-09-10T05:55:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:02 smithi098 ceph-mon[103124]: pgmap v12817: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 462 KiB/s wr, 19 op/s 2023-09-10T05:55:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23734.433175 secs 2023-09-10T05:55:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23734.433175 secs 2023-09-10T05:55:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23734.433175 secs 2023-09-10T05:55:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23735.403379 secs 2023-09-10T05:55:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:04 smithi170 ceph-mon[102832]: pgmap v12818: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 351 KiB/s wr, 13 op/s 2023-09-10T05:55:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:55:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23736.057108 secs 2023-09-10T05:55:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23735.403379 secs 2023-09-10T05:55:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:04 smithi082 ceph-mon[98694]: pgmap v12818: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 351 KiB/s wr, 13 op/s 2023-09-10T05:55:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:55:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23736.057108 secs 2023-09-10T05:55:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23735.403379 secs 2023-09-10T05:55:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:04 smithi098 ceph-mon[103124]: pgmap v12818: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 351 KiB/s wr, 13 op/s 2023-09-10T05:55:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:55:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23736.057108 secs 2023-09-10T05:55:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:06 smithi170 ceph-mon[102832]: pgmap v12819: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 510 KiB/s wr, 33 op/s 2023-09-10T05:55:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:55:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:06 smithi082 ceph-mon[98694]: pgmap v12819: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 510 KiB/s wr, 33 op/s 2023-09-10T05:55:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:55:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:06 smithi098 ceph-mon[103124]: pgmap v12819: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 510 KiB/s wr, 33 op/s 2023-09-10T05:55:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:55:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:08 smithi170 ceph-mon[102832]: pgmap v12820: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 322 KiB/s wr, 26 op/s 2023-09-10T05:55:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23739.433235 secs 2023-09-10T05:55:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:08 smithi082 ceph-mon[98694]: pgmap v12820: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 322 KiB/s wr, 26 op/s 2023-09-10T05:55:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23739.433235 secs 2023-09-10T05:55:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:08 smithi098 ceph-mon[103124]: pgmap v12820: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 322 KiB/s wr, 26 op/s 2023-09-10T05:55:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23739.433235 secs 2023-09-10T05:55:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23740.403438 secs 2023-09-10T05:55:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:09 smithi170 ceph-mon[102832]: osdmap e869: 12 total, 12 up, 12 in 2023-09-10T05:55:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23741.057172 secs 2023-09-10T05:55:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23740.403438 secs 2023-09-10T05:55:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:09 smithi082 ceph-mon[98694]: osdmap e869: 12 total, 12 up, 12 in 2023-09-10T05:55:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23741.057172 secs 2023-09-10T05:55:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23740.403438 secs 2023-09-10T05:55:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:09 smithi098 ceph-mon[103124]: osdmap e869: 12 total, 12 up, 12 in 2023-09-10T05:55:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23741.057172 secs 2023-09-10T05:55:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:10 smithi170 ceph-mon[102832]: pgmap v12822: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 386 KiB/s wr, 32 op/s 2023-09-10T05:55:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:10 smithi082 ceph-mon[98694]: pgmap v12822: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 386 KiB/s wr, 32 op/s 2023-09-10T05:55:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:10 smithi098 ceph-mon[103124]: pgmap v12822: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 386 KiB/s wr, 32 op/s 2023-09-10T05:55:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:12 smithi170 ceph-mon[102832]: pgmap v12823: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 265 KiB/s wr, 29 op/s 2023-09-10T05:55:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:12 smithi082 ceph-mon[98694]: pgmap v12823: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 265 KiB/s wr, 29 op/s 2023-09-10T05:55:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:12 smithi098 ceph-mon[103124]: pgmap v12823: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 265 KiB/s wr, 29 op/s 2023-09-10T05:55:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23744.433302 secs 2023-09-10T05:55:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23744.433302 secs 2023-09-10T05:55:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23744.433302 secs 2023-09-10T05:55:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23745.403499 secs 2023-09-10T05:55:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:14 smithi082 ceph-mon[98694]: pgmap v12824: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 265 KiB/s wr, 29 op/s 2023-09-10T05:55:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23746.057239 secs 2023-09-10T05:55:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23745.403499 secs 2023-09-10T05:55:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:14 smithi098 ceph-mon[103124]: pgmap v12824: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 265 KiB/s wr, 29 op/s 2023-09-10T05:55:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23746.057239 secs 2023-09-10T05:55:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23745.403499 secs 2023-09-10T05:55:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:14 smithi170 ceph-mon[102832]: pgmap v12824: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 265 KiB/s wr, 29 op/s 2023-09-10T05:55:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23746.057239 secs 2023-09-10T05:55:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:16 smithi098 ceph-mon[103124]: osdmap e870: 12 total, 12 up, 12 in 2023-09-10T05:55:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:16 smithi098 ceph-mon[103124]: pgmap v12826: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 93 KiB/s wr, 13 op/s 2023-09-10T05:55:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:16 smithi170 ceph-mon[102832]: osdmap e870: 12 total, 12 up, 12 in 2023-09-10T05:55:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:16 smithi170 ceph-mon[102832]: pgmap v12826: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 93 KiB/s wr, 13 op/s 2023-09-10T05:55:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:16 smithi082 ceph-mon[98694]: osdmap e870: 12 total, 12 up, 12 in 2023-09-10T05:55:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:16 smithi082 ceph-mon[98694]: pgmap v12826: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 93 KiB/s wr, 13 op/s 2023-09-10T05:55:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:55:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:55:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:55:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:18 smithi170 ceph-mon[102832]: pgmap v12827: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T05:55:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:55:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:55:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23749.433362 secs 2023-09-10T05:55:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:55:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:55:18.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:55:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:18 smithi082 ceph-mon[98694]: pgmap v12827: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T05:55:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:55:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:55:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23749.433362 secs 2023-09-10T05:55:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:55:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:55:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:55:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:18 smithi098 ceph-mon[103124]: pgmap v12827: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 84 KiB/s wr, 12 op/s 2023-09-10T05:55:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:55:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:55:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23749.433362 secs 2023-09-10T05:55:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:55:18.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:55:18.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:55:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23750.403557 secs 2023-09-10T05:55:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23751.057302 secs 2023-09-10T05:55:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23750.403557 secs 2023-09-10T05:55:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23751.057302 secs 2023-09-10T05:55:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23750.403557 secs 2023-09-10T05:55:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23751.057302 secs 2023-09-10T05:55:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:20 smithi170 ceph-mon[102832]: pgmap v12828: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T05:55:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:20 smithi082 ceph-mon[98694]: pgmap v12828: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T05:55:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:20 smithi098 ceph-mon[103124]: pgmap v12828: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 10 op/s 2023-09-10T05:55:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:22 smithi170 ceph-mon[102832]: pgmap v12829: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-09-10T05:55:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:22 smithi082 ceph-mon[98694]: pgmap v12829: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-09-10T05:55:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:22 smithi098 ceph-mon[103124]: pgmap v12829: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-09-10T05:55:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23754.433453 secs 2023-09-10T05:55:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23754.433453 secs 2023-09-10T05:55:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23754.433453 secs 2023-09-10T05:55:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23755.403615 secs 2023-09-10T05:55:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:24 smithi170 ceph-mon[102832]: pgmap v12830: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-09-10T05:55:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23756.057369 secs 2023-09-10T05:55:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23755.403615 secs 2023-09-10T05:55:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:24 smithi082 ceph-mon[98694]: pgmap v12830: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-09-10T05:55:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23756.057369 secs 2023-09-10T05:55:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23755.403615 secs 2023-09-10T05:55:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:24 smithi098 ceph-mon[103124]: pgmap v12830: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-09-10T05:55:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23756.057369 secs 2023-09-10T05:55:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:26 smithi098 ceph-mon[103124]: pgmap v12831: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 96 KiB/s wr, 17 op/s 2023-09-10T05:55:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:26 smithi170 ceph-mon[102832]: pgmap v12831: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 96 KiB/s wr, 17 op/s 2023-09-10T05:55:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:26 smithi082 ceph-mon[98694]: pgmap v12831: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 96 KiB/s wr, 17 op/s 2023-09-10T05:55:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:28 smithi170 ceph-mon[102832]: pgmap v12832: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T05:55:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:28 smithi082 ceph-mon[98694]: pgmap v12832: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T05:55:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:28 smithi098 ceph-mon[103124]: pgmap v12832: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T05:55:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23760.403682 secs 2023-09-10T05:55:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23761.057435 secs 2023-09-10T05:55:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23760.403682 secs 2023-09-10T05:55:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23761.057435 secs 2023-09-10T05:55:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23760.403682 secs 2023-09-10T05:55:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23761.057435 secs 2023-09-10T05:55:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:30 smithi170 ceph-mon[102832]: pgmap v12833: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T05:55:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:30 smithi082 ceph-mon[98694]: pgmap v12833: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T05:55:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:30 smithi098 ceph-mon[103124]: pgmap v12833: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T05:55:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:55:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:55:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:55:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:32 smithi170 ceph-mon[102832]: pgmap v12834: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 15 op/s 2023-09-10T05:55:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:55:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:55:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:32 smithi082 ceph-mon[98694]: pgmap v12834: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 15 op/s 2023-09-10T05:55:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:55:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:55:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:32 smithi098 ceph-mon[103124]: pgmap v12834: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 15 op/s 2023-09-10T05:55:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:55:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:55:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23765.403744 secs 2023-09-10T05:55:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:34 smithi170 ceph-mon[102832]: pgmap v12835: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 156 KiB/s wr, 10 op/s 2023-09-10T05:55:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:55:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23766.057485 secs 2023-09-10T05:55:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23765.403744 secs 2023-09-10T05:55:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:34 smithi082 ceph-mon[98694]: pgmap v12835: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 156 KiB/s wr, 10 op/s 2023-09-10T05:55:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:55:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23766.057485 secs 2023-09-10T05:55:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23765.403744 secs 2023-09-10T05:55:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:34 smithi098 ceph-mon[103124]: pgmap v12835: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 156 KiB/s wr, 10 op/s 2023-09-10T05:55:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:55:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23766.057485 secs 2023-09-10T05:55:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:36 smithi170 ceph-mon[102832]: pgmap v12836: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 574 KiB/s wr, 17 op/s 2023-09-10T05:55:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:55:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:36 smithi082 ceph-mon[98694]: pgmap v12836: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 574 KiB/s wr, 17 op/s 2023-09-10T05:55:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:55:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:36 smithi098 ceph-mon[103124]: pgmap v12836: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 574 KiB/s wr, 17 op/s 2023-09-10T05:55:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:55:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:38 smithi170 ceph-mon[102832]: pgmap v12837: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 494 KiB/s wr, 12 op/s 2023-09-10T05:55:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:38 smithi082 ceph-mon[98694]: pgmap v12837: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 494 KiB/s wr, 12 op/s 2023-09-10T05:55:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:38 smithi098 ceph-mon[103124]: pgmap v12837: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 494 KiB/s wr, 12 op/s 2023-09-10T05:55:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23770.403830 secs 2023-09-10T05:55:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23771.057550 secs 2023-09-10T05:55:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23770.403830 secs 2023-09-10T05:55:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23771.057550 secs 2023-09-10T05:55:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23770.403830 secs 2023-09-10T05:55:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23771.057550 secs 2023-09-10T05:55:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:40 smithi082 ceph-mon[98694]: pgmap v12838: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 494 KiB/s wr, 12 op/s 2023-09-10T05:55:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:40 smithi098 ceph-mon[103124]: pgmap v12838: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 494 KiB/s wr, 12 op/s 2023-09-10T05:55:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:40 smithi170 ceph-mon[102832]: pgmap v12838: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 494 KiB/s wr, 12 op/s 2023-09-10T05:55:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:42 smithi082 ceph-mon[98694]: pgmap v12839: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 510 KiB/s wr, 18 op/s 2023-09-10T05:55:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:42 smithi098 ceph-mon[103124]: pgmap v12839: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 510 KiB/s wr, 18 op/s 2023-09-10T05:55:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:42 smithi170 ceph-mon[102832]: pgmap v12839: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 510 KiB/s wr, 18 op/s 2023-09-10T05:55:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23775.403888 secs 2023-09-10T05:55:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:44 smithi082 ceph-mon[98694]: pgmap v12840: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 434 KiB/s wr, 13 op/s 2023-09-10T05:55:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:55:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23776.057614 secs 2023-09-10T05:55:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23775.403888 secs 2023-09-10T05:55:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:44 smithi098 ceph-mon[103124]: pgmap v12840: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 434 KiB/s wr, 13 op/s 2023-09-10T05:55:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:55:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23776.057614 secs 2023-09-10T05:55:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23775.403888 secs 2023-09-10T05:55:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:44 smithi170 ceph-mon[102832]: pgmap v12840: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 434 KiB/s wr, 13 op/s 2023-09-10T05:55:44.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:55:44.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:44.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23776.057614 secs 2023-09-10T05:55:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:46 smithi098 ceph-mon[103124]: pgmap v12841: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 713 KiB/s wr, 19 op/s 2023-09-10T05:55:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:46 smithi170 ceph-mon[102832]: pgmap v12841: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 713 KiB/s wr, 19 op/s 2023-09-10T05:55:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:46 smithi082 ceph-mon[98694]: pgmap v12841: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 713 KiB/s wr, 19 op/s 2023-09-10T05:55:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:48 smithi170 ceph-mon[102832]: pgmap v12842: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 295 KiB/s wr, 12 op/s 2023-09-10T05:55:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:48 smithi082 ceph-mon[98694]: pgmap v12842: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 295 KiB/s wr, 12 op/s 2023-09-10T05:55:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:48 smithi098 ceph-mon[103124]: pgmap v12842: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 295 KiB/s wr, 12 op/s 2023-09-10T05:55:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23780.403954 secs 2023-09-10T05:55:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23781.057678 secs 2023-09-10T05:55:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23780.403954 secs 2023-09-10T05:55:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23781.057678 secs 2023-09-10T05:55:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23780.403954 secs 2023-09-10T05:55:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23781.057678 secs 2023-09-10T05:55:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:50 smithi170 ceph-mon[102832]: pgmap v12843: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 295 KiB/s wr, 12 op/s 2023-09-10T05:55:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:55:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:50 smithi082 ceph-mon[98694]: pgmap v12843: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 295 KiB/s wr, 12 op/s 2023-09-10T05:55:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:55:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:50 smithi098 ceph-mon[103124]: pgmap v12843: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 295 KiB/s wr, 12 op/s 2023-09-10T05:55:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:55:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:52 smithi170 ceph-mon[102832]: pgmap v12844: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 585 KiB/s wr, 19 op/s 2023-09-10T05:55:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:52 smithi082 ceph-mon[98694]: pgmap v12844: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 585 KiB/s wr, 19 op/s 2023-09-10T05:55:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:52 smithi098 ceph-mon[103124]: pgmap v12844: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 585 KiB/s wr, 19 op/s 2023-09-10T05:55:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23784.433774 secs 2023-09-10T05:55:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23784.433774 secs 2023-09-10T05:55:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23784.433774 secs 2023-09-10T05:55:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23785.404014 secs 2023-09-10T05:55:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:54 smithi170 ceph-mon[102832]: pgmap v12845: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 569 KiB/s wr, 13 op/s 2023-09-10T05:55:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:55:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23786.057745 secs 2023-09-10T05:55:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23785.404014 secs 2023-09-10T05:55:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:54 smithi082 ceph-mon[98694]: pgmap v12845: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 569 KiB/s wr, 13 op/s 2023-09-10T05:55:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:55:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23786.057745 secs 2023-09-10T05:55:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23785.404014 secs 2023-09-10T05:55:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:54 smithi098 ceph-mon[103124]: pgmap v12845: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 569 KiB/s wr, 13 op/s 2023-09-10T05:55:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:55:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23786.057745 secs 2023-09-10T05:55:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:56 smithi098 ceph-mon[103124]: pgmap v12846: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 618 KiB/s wr, 18 op/s 2023-09-10T05:55:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:55:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:55:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:56 smithi170 ceph-mon[102832]: pgmap v12846: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 618 KiB/s wr, 18 op/s 2023-09-10T05:55:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:55:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:55:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:56 smithi082 ceph-mon[98694]: pgmap v12846: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 618 KiB/s wr, 18 op/s 2023-09-10T05:55:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:55:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:55:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:55:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:58 smithi170 ceph-mon[102832]: pgmap v12847: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 339 KiB/s wr, 12 op/s 2023-09-10T05:55:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23789.433834 secs 2023-09-10T05:55:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:58 smithi082 ceph-mon[98694]: pgmap v12847: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 339 KiB/s wr, 12 op/s 2023-09-10T05:55:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23789.433834 secs 2023-09-10T05:55:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:58 smithi098 ceph-mon[103124]: pgmap v12847: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 339 KiB/s wr, 12 op/s 2023-09-10T05:55:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23789.433834 secs 2023-09-10T05:55:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23790.404075 secs 2023-09-10T05:55:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:55:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23791.057810 secs 2023-09-10T05:55:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23790.404075 secs 2023-09-10T05:55:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:55:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23791.057810 secs 2023-09-10T05:55:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23790.404075 secs 2023-09-10T05:55:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:55:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23791.057810 secs 2023-09-10T05:56:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:00 smithi170 ceph-mon[102832]: pgmap v12848: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 339 KiB/s wr, 12 op/s 2023-09-10T05:56:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:00 smithi082 ceph-mon[98694]: pgmap v12848: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 339 KiB/s wr, 12 op/s 2023-09-10T05:56:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:00 smithi098 ceph-mon[103124]: pgmap v12848: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 339 KiB/s wr, 12 op/s 2023-09-10T05:56:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:02 smithi170 ceph-mon[102832]: pgmap v12849: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 510 KiB/s wr, 16 op/s 2023-09-10T05:56:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:02 smithi082 ceph-mon[98694]: pgmap v12849: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 510 KiB/s wr, 16 op/s 2023-09-10T05:56:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:02 smithi098 ceph-mon[103124]: pgmap v12849: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 510 KiB/s wr, 16 op/s 2023-09-10T05:56:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23794.433892 secs 2023-09-10T05:56:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23794.433892 secs 2023-09-10T05:56:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23794.433892 secs 2023-09-10T05:56:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23795.404133 secs 2023-09-10T05:56:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:04 smithi170 ceph-mon[102832]: pgmap v12850: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 220 KiB/s wr, 9 op/s 2023-09-10T05:56:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23796.057873 secs 2023-09-10T05:56:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23795.404133 secs 2023-09-10T05:56:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:04 smithi082 ceph-mon[98694]: pgmap v12850: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 220 KiB/s wr, 9 op/s 2023-09-10T05:56:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23796.057873 secs 2023-09-10T05:56:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23795.404133 secs 2023-09-10T05:56:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:04 smithi098 ceph-mon[103124]: pgmap v12850: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 220 KiB/s wr, 9 op/s 2023-09-10T05:56:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23796.057873 secs 2023-09-10T05:56:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:06 smithi170 ceph-mon[102832]: pgmap v12851: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 344 KiB/s wr, 29 op/s 2023-09-10T05:56:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:06 smithi082 ceph-mon[98694]: pgmap v12851: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 344 KiB/s wr, 29 op/s 2023-09-10T05:56:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:06 smithi098 ceph-mon[103124]: pgmap v12851: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 344 KiB/s wr, 29 op/s 2023-09-10T05:56:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:08 smithi170 ceph-mon[102832]: pgmap v12852: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 295 KiB/s wr, 24 op/s 2023-09-10T05:56:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23799.433953 secs 2023-09-10T05:56:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:08 smithi082 ceph-mon[98694]: pgmap v12852: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 295 KiB/s wr, 24 op/s 2023-09-10T05:56:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23799.433953 secs 2023-09-10T05:56:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:08 smithi098 ceph-mon[103124]: pgmap v12852: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 295 KiB/s wr, 24 op/s 2023-09-10T05:56:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23799.433953 secs 2023-09-10T05:56:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23800.404192 secs 2023-09-10T05:56:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:09 smithi170 ceph-mon[102832]: osdmap e871: 12 total, 12 up, 12 in 2023-09-10T05:56:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23801.057942 secs 2023-09-10T05:56:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23800.404192 secs 2023-09-10T05:56:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:09 smithi082 ceph-mon[98694]: osdmap e871: 12 total, 12 up, 12 in 2023-09-10T05:56:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23801.057942 secs 2023-09-10T05:56:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23800.404192 secs 2023-09-10T05:56:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:09 smithi098 ceph-mon[103124]: osdmap e871: 12 total, 12 up, 12 in 2023-09-10T05:56:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23801.057942 secs 2023-09-10T05:56:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:10 smithi170 ceph-mon[102832]: pgmap v12854: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 354 KiB/s wr, 29 op/s 2023-09-10T05:56:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:10 smithi082 ceph-mon[98694]: pgmap v12854: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 354 KiB/s wr, 29 op/s 2023-09-10T05:56:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:10 smithi098 ceph-mon[103124]: pgmap v12854: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 354 KiB/s wr, 29 op/s 2023-09-10T05:56:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:12 smithi170 ceph-mon[102832]: pgmap v12855: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 172 KiB/s wr, 29 op/s 2023-09-10T05:56:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:12 smithi082 ceph-mon[98694]: pgmap v12855: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 172 KiB/s wr, 29 op/s 2023-09-10T05:56:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:12 smithi098 ceph-mon[103124]: pgmap v12855: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 172 KiB/s wr, 29 op/s 2023-09-10T05:56:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23804.434074 secs 2023-09-10T05:56:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23804.434074 secs 2023-09-10T05:56:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23804.434074 secs 2023-09-10T05:56:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23805.404252 secs 2023-09-10T05:56:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:14 smithi082 ceph-mon[98694]: pgmap v12856: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 172 KiB/s wr, 29 op/s 2023-09-10T05:56:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23806.058031 secs 2023-09-10T05:56:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23805.404252 secs 2023-09-10T05:56:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:14 smithi098 ceph-mon[103124]: pgmap v12856: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 172 KiB/s wr, 29 op/s 2023-09-10T05:56:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23806.058031 secs 2023-09-10T05:56:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23805.404252 secs 2023-09-10T05:56:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:14 smithi170 ceph-mon[102832]: pgmap v12856: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 172 KiB/s wr, 29 op/s 2023-09-10T05:56:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23806.058031 secs 2023-09-10T05:56:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:16 smithi098 ceph-mon[103124]: osdmap e872: 12 total, 12 up, 12 in 2023-09-10T05:56:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:16 smithi098 ceph-mon[103124]: pgmap v12858: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 15 op/s 2023-09-10T05:56:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:16 smithi170 ceph-mon[102832]: osdmap e872: 12 total, 12 up, 12 in 2023-09-10T05:56:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:16 smithi170 ceph-mon[102832]: pgmap v12858: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 15 op/s 2023-09-10T05:56:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:16 smithi082 ceph-mon[98694]: osdmap e872: 12 total, 12 up, 12 in 2023-09-10T05:56:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:16 smithi082 ceph-mon[98694]: pgmap v12858: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 15 op/s 2023-09-10T05:56:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:18 smithi170 ceph-mon[102832]: pgmap v12859: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T05:56:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23809.434199 secs 2023-09-10T05:56:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:56:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:18 smithi082 ceph-mon[98694]: pgmap v12859: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T05:56:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23809.434199 secs 2023-09-10T05:56:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:56:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:18 smithi098 ceph-mon[103124]: pgmap v12859: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T05:56:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23809.434199 secs 2023-09-10T05:56:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:56:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23810.404350 secs 2023-09-10T05:56:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:56:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:56:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:56:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:56:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23811.058100 secs 2023-09-10T05:56:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:56:19.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:56:19.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:56:19.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:19 smithi170 ceph-mon[102832]: pgmap v12860: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T05:56:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23810.404350 secs 2023-09-10T05:56:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:56:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:56:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:56:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:56:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23811.058100 secs 2023-09-10T05:56:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:56:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:56:19.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:56:19.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:19 smithi082 ceph-mon[98694]: pgmap v12860: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T05:56:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23810.404350 secs 2023-09-10T05:56:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:56:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:56:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:56:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:56:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23811.058100 secs 2023-09-10T05:56:20.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:56:20.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:56:20.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:56:20.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:19 smithi098 ceph-mon[103124]: pgmap v12860: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T05:56:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:22 smithi170 ceph-mon[102832]: pgmap v12861: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T05:56:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:22 smithi082 ceph-mon[98694]: pgmap v12861: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T05:56:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:22 smithi098 ceph-mon[103124]: pgmap v12861: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T05:56:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23814.434259 secs 2023-09-10T05:56:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23814.434259 secs 2023-09-10T05:56:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23814.434259 secs 2023-09-10T05:56:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23815.404407 secs 2023-09-10T05:56:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:24 smithi170 ceph-mon[102832]: pgmap v12862: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T05:56:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23816.058164 secs 2023-09-10T05:56:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23815.404407 secs 2023-09-10T05:56:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:24 smithi082 ceph-mon[98694]: pgmap v12862: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T05:56:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23816.058164 secs 2023-09-10T05:56:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23815.404407 secs 2023-09-10T05:56:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:24 smithi098 ceph-mon[103124]: pgmap v12862: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T05:56:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23816.058164 secs 2023-09-10T05:56:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:26 smithi170 ceph-mon[102832]: pgmap v12863: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 96 KiB/s wr, 18 op/s 2023-09-10T05:56:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:26 smithi082 ceph-mon[98694]: pgmap v12863: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 96 KiB/s wr, 18 op/s 2023-09-10T05:56:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:26 smithi098 ceph-mon[103124]: pgmap v12863: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 96 KiB/s wr, 18 op/s 2023-09-10T05:56:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:28 smithi170 ceph-mon[102832]: pgmap v12864: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T05:56:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:28 smithi082 ceph-mon[98694]: pgmap v12864: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T05:56:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:28 smithi098 ceph-mon[103124]: pgmap v12864: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T05:56:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23820.404463 secs 2023-09-10T05:56:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23821.058240 secs 2023-09-10T05:56:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23820.404463 secs 2023-09-10T05:56:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23821.058240 secs 2023-09-10T05:56:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23820.404463 secs 2023-09-10T05:56:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23821.058240 secs 2023-09-10T05:56:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:30 smithi170 ceph-mon[102832]: pgmap v12865: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T05:56:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:56:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:56:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:30 smithi082 ceph-mon[98694]: pgmap v12865: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T05:56:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:56:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:56:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:30 smithi098 ceph-mon[103124]: pgmap v12865: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 80 KiB/s wr, 10 op/s 2023-09-10T05:56:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:56:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:56:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:56:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:56:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:56:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:32 smithi170 ceph-mon[102832]: pgmap v12866: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 177 KiB/s wr, 15 op/s 2023-09-10T05:56:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:56:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:56:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:56:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:32 smithi082 ceph-mon[98694]: pgmap v12866: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 177 KiB/s wr, 15 op/s 2023-09-10T05:56:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:56:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:56:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:56:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:32 smithi098 ceph-mon[103124]: pgmap v12866: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 177 KiB/s wr, 15 op/s 2023-09-10T05:56:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:56:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:56:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:56:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23825.404493 secs 2023-09-10T05:56:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:34 smithi082 ceph-mon[98694]: pgmap v12867: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 9 op/s 2023-09-10T05:56:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23826.058297 secs 2023-09-10T05:56:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23825.404493 secs 2023-09-10T05:56:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:34 smithi098 ceph-mon[103124]: pgmap v12867: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 9 op/s 2023-09-10T05:56:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23826.058297 secs 2023-09-10T05:56:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23825.404493 secs 2023-09-10T05:56:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:34 smithi170 ceph-mon[102832]: pgmap v12867: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 9 op/s 2023-09-10T05:56:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23826.058297 secs 2023-09-10T05:56:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:36 smithi170 ceph-mon[102832]: pgmap v12868: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 268 KiB/s wr, 15 op/s 2023-09-10T05:56:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:36 smithi082 ceph-mon[98694]: pgmap v12868: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 268 KiB/s wr, 15 op/s 2023-09-10T05:56:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:36 smithi098 ceph-mon[103124]: pgmap v12868: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 268 KiB/s wr, 15 op/s 2023-09-10T05:56:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:38 smithi170 ceph-mon[102832]: pgmap v12869: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 268 KiB/s wr, 10 op/s 2023-09-10T05:56:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:38 smithi082 ceph-mon[98694]: pgmap v12869: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 268 KiB/s wr, 10 op/s 2023-09-10T05:56:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:38 smithi098 ceph-mon[103124]: pgmap v12869: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 268 KiB/s wr, 10 op/s 2023-09-10T05:56:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23830.404552 secs 2023-09-10T05:56:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23831.058364 secs 2023-09-10T05:56:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23830.404552 secs 2023-09-10T05:56:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23831.058364 secs 2023-09-10T05:56:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23830.404552 secs 2023-09-10T05:56:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23831.058364 secs 2023-09-10T05:56:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:40 smithi170 ceph-mon[102832]: pgmap v12870: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 268 KiB/s wr, 10 op/s 2023-09-10T05:56:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:40 smithi082 ceph-mon[98694]: pgmap v12870: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 268 KiB/s wr, 10 op/s 2023-09-10T05:56:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:40 smithi098 ceph-mon[103124]: pgmap v12870: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 268 KiB/s wr, 10 op/s 2023-09-10T05:56:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:42 smithi082 ceph-mon[98694]: pgmap v12871: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 355 KiB/s wr, 16 op/s 2023-09-10T05:56:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:42 smithi098 ceph-mon[103124]: pgmap v12871: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 355 KiB/s wr, 16 op/s 2023-09-10T05:56:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:42 smithi170 ceph-mon[102832]: pgmap v12871: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 355 KiB/s wr, 16 op/s 2023-09-10T05:56:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23835.404611 secs 2023-09-10T05:56:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:44 smithi082 ceph-mon[98694]: pgmap v12872: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 258 KiB/s wr, 11 op/s 2023-09-10T05:56:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:56:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:56:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23836.058429 secs 2023-09-10T05:56:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23835.404611 secs 2023-09-10T05:56:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:44 smithi098 ceph-mon[103124]: pgmap v12872: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 258 KiB/s wr, 11 op/s 2023-09-10T05:56:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:56:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:56:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23836.058429 secs 2023-09-10T05:56:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23835.404611 secs 2023-09-10T05:56:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:44 smithi170 ceph-mon[102832]: pgmap v12872: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 258 KiB/s wr, 11 op/s 2023-09-10T05:56:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:56:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:56:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23836.058429 secs 2023-09-10T05:56:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:46 smithi082 ceph-mon[98694]: pgmap v12873: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 501 KiB/s wr, 17 op/s 2023-09-10T05:56:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:56:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:56:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:46 smithi098 ceph-mon[103124]: pgmap v12873: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 501 KiB/s wr, 17 op/s 2023-09-10T05:56:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:56:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:56:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:46 smithi170 ceph-mon[102832]: pgmap v12873: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 501 KiB/s wr, 17 op/s 2023-09-10T05:56:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:56:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:56:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:48 smithi082 ceph-mon[98694]: pgmap v12874: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 330 KiB/s wr, 12 op/s 2023-09-10T05:56:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:48 smithi098 ceph-mon[103124]: pgmap v12874: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 330 KiB/s wr, 12 op/s 2023-09-10T05:56:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:48 smithi170 ceph-mon[102832]: pgmap v12874: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 330 KiB/s wr, 12 op/s 2023-09-10T05:56:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23840.404671 secs 2023-09-10T05:56:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23841.058487 secs 2023-09-10T05:56:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23840.404671 secs 2023-09-10T05:56:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23841.058487 secs 2023-09-10T05:56:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23840.404671 secs 2023-09-10T05:56:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23841.058487 secs 2023-09-10T05:56:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:50 smithi082 ceph-mon[98694]: pgmap v12875: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 330 KiB/s wr, 12 op/s 2023-09-10T05:56:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:50 smithi098 ceph-mon[103124]: pgmap v12875: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 330 KiB/s wr, 12 op/s 2023-09-10T05:56:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:50 smithi170 ceph-mon[102832]: pgmap v12875: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 330 KiB/s wr, 12 op/s 2023-09-10T05:56:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:52 smithi082 ceph-mon[98694]: pgmap v12876: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 500 KiB/s wr, 18 op/s 2023-09-10T05:56:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:52 smithi098 ceph-mon[103124]: pgmap v12876: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 500 KiB/s wr, 18 op/s 2023-09-10T05:56:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:52 smithi170 ceph-mon[102832]: pgmap v12876: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 500 KiB/s wr, 18 op/s 2023-09-10T05:56:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23844.434772 secs 2023-09-10T05:56:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23844.434772 secs 2023-09-10T05:56:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23844.434772 secs 2023-09-10T05:56:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23845.404732 secs 2023-09-10T05:56:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:54 smithi098 ceph-mon[103124]: pgmap v12877: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 413 KiB/s wr, 12 op/s 2023-09-10T05:56:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:56:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:56:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23846.058552 secs 2023-09-10T05:56:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23845.404732 secs 2023-09-10T05:56:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:54 smithi170 ceph-mon[102832]: pgmap v12877: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 413 KiB/s wr, 12 op/s 2023-09-10T05:56:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:56:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:56:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23846.058552 secs 2023-09-10T05:56:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23845.404732 secs 2023-09-10T05:56:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:54 smithi082 ceph-mon[98694]: pgmap v12877: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 413 KiB/s wr, 12 op/s 2023-09-10T05:56:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:56:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:56:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23846.058552 secs 2023-09-10T05:56:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:56 smithi170 ceph-mon[102832]: pgmap v12878: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 568 KiB/s wr, 18 op/s 2023-09-10T05:56:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:56:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:56:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:56:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:56 smithi082 ceph-mon[98694]: pgmap v12878: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 568 KiB/s wr, 18 op/s 2023-09-10T05:56:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:56:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:56:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:56:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:56 smithi098 ceph-mon[103124]: pgmap v12878: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 568 KiB/s wr, 18 op/s 2023-09-10T05:56:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:56:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:56:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:56:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:58 smithi170 ceph-mon[102832]: pgmap v12879: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 326 KiB/s wr, 12 op/s 2023-09-10T05:56:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23849.434893 secs 2023-09-10T05:56:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:58 smithi082 ceph-mon[98694]: pgmap v12879: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 326 KiB/s wr, 12 op/s 2023-09-10T05:56:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23849.434893 secs 2023-09-10T05:56:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:58 smithi098 ceph-mon[103124]: pgmap v12879: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 326 KiB/s wr, 12 op/s 2023-09-10T05:56:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23849.434893 secs 2023-09-10T05:56:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23850.404791 secs 2023-09-10T05:56:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:56:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23851.058621 secs 2023-09-10T05:56:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23850.404791 secs 2023-09-10T05:56:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:56:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23851.058621 secs 2023-09-10T05:56:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23850.404791 secs 2023-09-10T05:56:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:56:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23851.058621 secs 2023-09-10T05:57:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:00 smithi082 ceph-mon[98694]: pgmap v12880: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 326 KiB/s wr, 12 op/s 2023-09-10T05:57:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:00 smithi098 ceph-mon[103124]: pgmap v12880: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 326 KiB/s wr, 12 op/s 2023-09-10T05:57:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:00 smithi170 ceph-mon[102832]: pgmap v12880: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 326 KiB/s wr, 12 op/s 2023-09-10T05:57:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:02 smithi082 ceph-mon[98694]: pgmap v12881: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 413 KiB/s wr, 17 op/s 2023-09-10T05:57:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:02 smithi098 ceph-mon[103124]: pgmap v12881: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 413 KiB/s wr, 17 op/s 2023-09-10T05:57:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:02 smithi170 ceph-mon[102832]: pgmap v12881: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 413 KiB/s wr, 17 op/s 2023-09-10T05:57:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23854.435003 secs 2023-09-10T05:57:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23854.435003 secs 2023-09-10T05:57:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23854.435003 secs 2023-09-10T05:57:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23855.404849 secs 2023-09-10T05:57:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:04 smithi082 ceph-mon[98694]: pgmap v12882: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 243 KiB/s wr, 11 op/s 2023-09-10T05:57:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23856.058684 secs 2023-09-10T05:57:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23855.404849 secs 2023-09-10T05:57:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:04 smithi098 ceph-mon[103124]: pgmap v12882: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 243 KiB/s wr, 11 op/s 2023-09-10T05:57:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23856.058684 secs 2023-09-10T05:57:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23855.404849 secs 2023-09-10T05:57:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:04 smithi170 ceph-mon[102832]: pgmap v12882: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 243 KiB/s wr, 11 op/s 2023-09-10T05:57:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23856.058684 secs 2023-09-10T05:57:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:06 smithi170 ceph-mon[102832]: pgmap v12883: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 306 KiB/s wr, 30 op/s 2023-09-10T05:57:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:06 smithi082 ceph-mon[98694]: pgmap v12883: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 306 KiB/s wr, 30 op/s 2023-09-10T05:57:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:06 smithi098 ceph-mon[103124]: pgmap v12883: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 306 KiB/s wr, 30 op/s 2023-09-10T05:57:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:08 smithi082 ceph-mon[98694]: pgmap v12884: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 150 KiB/s wr, 24 op/s 2023-09-10T05:57:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23859.435065 secs 2023-09-10T05:57:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:08 smithi098 ceph-mon[103124]: pgmap v12884: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 150 KiB/s wr, 24 op/s 2023-09-10T05:57:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23859.435065 secs 2023-09-10T05:57:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:08 smithi170 ceph-mon[102832]: pgmap v12884: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 150 KiB/s wr, 24 op/s 2023-09-10T05:57:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23859.435065 secs 2023-09-10T05:57:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23860.404936 secs 2023-09-10T05:57:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:09 smithi082 ceph-mon[98694]: osdmap e873: 12 total, 12 up, 12 in 2023-09-10T05:57:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23861.058750 secs 2023-09-10T05:57:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23860.404936 secs 2023-09-10T05:57:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:09 smithi098 ceph-mon[103124]: osdmap e873: 12 total, 12 up, 12 in 2023-09-10T05:57:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23861.058750 secs 2023-09-10T05:57:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23860.404936 secs 2023-09-10T05:57:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:09 smithi170 ceph-mon[102832]: osdmap e873: 12 total, 12 up, 12 in 2023-09-10T05:57:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23861.058750 secs 2023-09-10T05:57:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:10 smithi082 ceph-mon[98694]: pgmap v12886: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 180 KiB/s wr, 29 op/s 2023-09-10T05:57:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:10 smithi098 ceph-mon[103124]: pgmap v12886: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 180 KiB/s wr, 29 op/s 2023-09-10T05:57:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:10 smithi170 ceph-mon[102832]: pgmap v12886: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 180 KiB/s wr, 29 op/s 2023-09-10T05:57:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:12 smithi082 ceph-mon[98694]: pgmap v12887: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T05:57:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:12 smithi098 ceph-mon[103124]: pgmap v12887: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T05:57:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:12 smithi170 ceph-mon[102832]: pgmap v12887: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T05:57:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23864.435157 secs 2023-09-10T05:57:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23864.435157 secs 2023-09-10T05:57:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23864.435157 secs 2023-09-10T05:57:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23865.405016 secs 2023-09-10T05:57:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:14 smithi082 ceph-mon[98694]: pgmap v12888: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T05:57:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23866.058813 secs 2023-09-10T05:57:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23865.405016 secs 2023-09-10T05:57:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:14 smithi098 ceph-mon[103124]: pgmap v12888: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T05:57:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23866.058813 secs 2023-09-10T05:57:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23865.405016 secs 2023-09-10T05:57:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:14 smithi170 ceph-mon[102832]: pgmap v12888: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 29 op/s 2023-09-10T05:57:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23866.058813 secs 2023-09-10T05:57:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:16 smithi170 ceph-mon[102832]: osdmap e874: 12 total, 12 up, 12 in 2023-09-10T05:57:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:16 smithi170 ceph-mon[102832]: pgmap v12890: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 KiB/s wr, 14 op/s 2023-09-10T05:57:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:16 smithi082 ceph-mon[98694]: osdmap e874: 12 total, 12 up, 12 in 2023-09-10T05:57:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:16 smithi082 ceph-mon[98694]: pgmap v12890: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 KiB/s wr, 14 op/s 2023-09-10T05:57:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:16 smithi098 ceph-mon[103124]: osdmap e874: 12 total, 12 up, 12 in 2023-09-10T05:57:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:16 smithi098 ceph-mon[103124]: pgmap v12890: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 KiB/s wr, 14 op/s 2023-09-10T05:57:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:18 smithi082 ceph-mon[98694]: pgmap v12891: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 12 op/s 2023-09-10T05:57:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23869.435217 secs 2023-09-10T05:57:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:18 smithi098 ceph-mon[103124]: pgmap v12891: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 12 op/s 2023-09-10T05:57:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23869.435217 secs 2023-09-10T05:57:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:18 smithi170 ceph-mon[102832]: pgmap v12891: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 12 op/s 2023-09-10T05:57:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23869.435217 secs 2023-09-10T05:57:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23870.405102 secs 2023-09-10T05:57:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23871.058879 secs 2023-09-10T05:57:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:57:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23870.405102 secs 2023-09-10T05:57:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23871.058879 secs 2023-09-10T05:57:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:57:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23870.405102 secs 2023-09-10T05:57:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23871.058879 secs 2023-09-10T05:57:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:57:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:20 smithi082 ceph-mon[98694]: pgmap v12892: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T05:57:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:57:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:57:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:57:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:20 smithi098 ceph-mon[103124]: pgmap v12892: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T05:57:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:57:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:57:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:57:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:20 smithi170 ceph-mon[102832]: pgmap v12892: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T05:57:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:57:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:57:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:57:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:22 smithi082 ceph-mon[98694]: pgmap v12893: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T05:57:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:22 smithi098 ceph-mon[103124]: pgmap v12893: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T05:57:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:22 smithi170 ceph-mon[102832]: pgmap v12893: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T05:57:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23874.435276 secs 2023-09-10T05:57:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23874.435276 secs 2023-09-10T05:57:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23874.435276 secs 2023-09-10T05:57:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23875.405199 secs 2023-09-10T05:57:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:24 smithi082 ceph-mon[98694]: pgmap v12894: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T05:57:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23876.058946 secs 2023-09-10T05:57:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23875.405199 secs 2023-09-10T05:57:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:24 smithi098 ceph-mon[103124]: pgmap v12894: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T05:57:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23876.058946 secs 2023-09-10T05:57:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23875.405199 secs 2023-09-10T05:57:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:24 smithi170 ceph-mon[102832]: pgmap v12894: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T05:57:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23876.058946 secs 2023-09-10T05:57:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:26 smithi170 ceph-mon[102832]: pgmap v12895: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 118 KiB/s wr, 17 op/s 2023-09-10T05:57:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:26 smithi082 ceph-mon[98694]: pgmap v12895: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 118 KiB/s wr, 17 op/s 2023-09-10T05:57:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:26 smithi098 ceph-mon[103124]: pgmap v12895: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 118 KiB/s wr, 17 op/s 2023-09-10T05:57:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:28 smithi082 ceph-mon[98694]: pgmap v12896: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T05:57:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:28 smithi098 ceph-mon[103124]: pgmap v12896: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T05:57:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:28 smithi170 ceph-mon[102832]: pgmap v12896: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T05:57:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23880.405259 secs 2023-09-10T05:57:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23881.059011 secs 2023-09-10T05:57:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23880.405259 secs 2023-09-10T05:57:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23881.059011 secs 2023-09-10T05:57:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23880.405259 secs 2023-09-10T05:57:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23881.059011 secs 2023-09-10T05:57:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:30 smithi082 ceph-mon[98694]: pgmap v12897: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 81 KiB/s wr, 9 op/s 2023-09-10T05:57:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:30 smithi098 ceph-mon[103124]: pgmap v12897: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 81 KiB/s wr, 9 op/s 2023-09-10T05:57:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:30 smithi170 ceph-mon[102832]: pgmap v12897: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 81 KiB/s wr, 9 op/s 2023-09-10T05:57:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:57:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:57:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:57:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:32 smithi082 ceph-mon[98694]: pgmap v12898: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 158 KiB/s wr, 15 op/s 2023-09-10T05:57:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:57:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:57:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:57:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:32 smithi098 ceph-mon[103124]: pgmap v12898: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 158 KiB/s wr, 15 op/s 2023-09-10T05:57:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:57:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:57:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:57:32.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:32 smithi170 ceph-mon[102832]: pgmap v12898: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 158 KiB/s wr, 15 op/s 2023-09-10T05:57:32.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:57:32.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:57:32.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:57:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23885.405344 secs 2023-09-10T05:57:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:34 smithi082 ceph-mon[98694]: pgmap v12899: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 158 KiB/s wr, 10 op/s 2023-09-10T05:57:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:57:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:57:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23886.059075 secs 2023-09-10T05:57:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23885.405344 secs 2023-09-10T05:57:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:34 smithi098 ceph-mon[103124]: pgmap v12899: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 158 KiB/s wr, 10 op/s 2023-09-10T05:57:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:57:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:57:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23886.059075 secs 2023-09-10T05:57:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23885.405344 secs 2023-09-10T05:57:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:34 smithi170 ceph-mon[102832]: pgmap v12899: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 158 KiB/s wr, 10 op/s 2023-09-10T05:57:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:57:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:57:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23886.059075 secs 2023-09-10T05:57:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:36 smithi082 ceph-mon[98694]: pgmap v12900: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 579 KiB/s wr, 17 op/s 2023-09-10T05:57:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:57:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:57:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:36 smithi098 ceph-mon[103124]: pgmap v12900: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 579 KiB/s wr, 17 op/s 2023-09-10T05:57:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:57:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:57:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:36 smithi170 ceph-mon[102832]: pgmap v12900: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 579 KiB/s wr, 17 op/s 2023-09-10T05:57:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:57:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:57:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:38 smithi098 ceph-mon[103124]: pgmap v12901: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 499 KiB/s wr, 12 op/s 2023-09-10T05:57:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:57:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:57:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:38 smithi170 ceph-mon[102832]: pgmap v12901: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 499 KiB/s wr, 12 op/s 2023-09-10T05:57:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:57:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:57:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:38 smithi082 ceph-mon[98694]: pgmap v12901: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 499 KiB/s wr, 12 op/s 2023-09-10T05:57:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:57:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:57:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23890.405402 secs 2023-09-10T05:57:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23891.059142 secs 2023-09-10T05:57:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23890.405402 secs 2023-09-10T05:57:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23891.059142 secs 2023-09-10T05:57:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23890.405402 secs 2023-09-10T05:57:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23891.059142 secs 2023-09-10T05:57:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:40 smithi098 ceph-mon[103124]: pgmap v12902: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 499 KiB/s wr, 12 op/s 2023-09-10T05:57:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:40 smithi170 ceph-mon[102832]: pgmap v12902: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 499 KiB/s wr, 12 op/s 2023-09-10T05:57:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:40 smithi082 ceph-mon[98694]: pgmap v12902: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 499 KiB/s wr, 12 op/s 2023-09-10T05:57:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:42 smithi098 ceph-mon[103124]: pgmap v12903: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 514 KiB/s wr, 18 op/s 2023-09-10T05:57:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:42 smithi170 ceph-mon[102832]: pgmap v12903: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 514 KiB/s wr, 18 op/s 2023-09-10T05:57:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:42 smithi082 ceph-mon[98694]: pgmap v12903: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 514 KiB/s wr, 18 op/s 2023-09-10T05:57:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23895.405460 secs 2023-09-10T05:57:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:44 smithi098 ceph-mon[103124]: pgmap v12904: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 437 KiB/s wr, 13 op/s 2023-09-10T05:57:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23896.059206 secs 2023-09-10T05:57:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23895.405460 secs 2023-09-10T05:57:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:44 smithi170 ceph-mon[102832]: pgmap v12904: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 437 KiB/s wr, 13 op/s 2023-09-10T05:57:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23896.059206 secs 2023-09-10T05:57:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23895.405460 secs 2023-09-10T05:57:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:44 smithi082 ceph-mon[98694]: pgmap v12904: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 437 KiB/s wr, 13 op/s 2023-09-10T05:57:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23896.059206 secs 2023-09-10T05:57:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:46 smithi170 ceph-mon[102832]: pgmap v12905: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 719 KiB/s wr, 19 op/s 2023-09-10T05:57:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:46 smithi082 ceph-mon[98694]: pgmap v12905: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 719 KiB/s wr, 19 op/s 2023-09-10T05:57:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:46 smithi098 ceph-mon[103124]: pgmap v12905: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 719 KiB/s wr, 19 op/s 2023-09-10T05:57:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:48 smithi082 ceph-mon[98694]: pgmap v12906: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 297 KiB/s wr, 12 op/s 2023-09-10T05:57:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:48 smithi098 ceph-mon[103124]: pgmap v12906: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 297 KiB/s wr, 12 op/s 2023-09-10T05:57:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:48 smithi170 ceph-mon[102832]: pgmap v12906: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 297 KiB/s wr, 12 op/s 2023-09-10T05:57:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23900.405517 secs 2023-09-10T05:57:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23901.059271 secs 2023-09-10T05:57:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23900.405517 secs 2023-09-10T05:57:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23901.059271 secs 2023-09-10T05:57:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23900.405517 secs 2023-09-10T05:57:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23901.059271 secs 2023-09-10T05:57:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:50 smithi082 ceph-mon[98694]: pgmap v12907: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 297 KiB/s wr, 12 op/s 2023-09-10T05:57:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:57:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:57:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:50 smithi098 ceph-mon[103124]: pgmap v12907: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 297 KiB/s wr, 12 op/s 2023-09-10T05:57:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:57:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:57:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:50 smithi170 ceph-mon[102832]: pgmap v12907: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 297 KiB/s wr, 12 op/s 2023-09-10T05:57:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:57:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:57:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:52 smithi082 ceph-mon[98694]: pgmap v12908: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 622 KiB/s wr, 19 op/s 2023-09-10T05:57:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:57:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:57:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:52 smithi098 ceph-mon[103124]: pgmap v12908: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 622 KiB/s wr, 19 op/s 2023-09-10T05:57:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:57:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:57:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:52 smithi170 ceph-mon[102832]: pgmap v12908: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 622 KiB/s wr, 19 op/s 2023-09-10T05:57:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:57:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:57:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23904.435755 secs 2023-09-10T05:57:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23904.435755 secs 2023-09-10T05:57:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23904.435755 secs 2023-09-10T05:57:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23905.405577 secs 2023-09-10T05:57:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:54 smithi098 ceph-mon[103124]: pgmap v12909: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 607 KiB/s wr, 13 op/s 2023-09-10T05:57:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23906.059338 secs 2023-09-10T05:57:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23905.405577 secs 2023-09-10T05:57:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:54 smithi170 ceph-mon[102832]: pgmap v12909: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 607 KiB/s wr, 13 op/s 2023-09-10T05:57:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23906.059338 secs 2023-09-10T05:57:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23905.405577 secs 2023-09-10T05:57:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:54 smithi082 ceph-mon[98694]: pgmap v12909: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 607 KiB/s wr, 13 op/s 2023-09-10T05:57:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23906.059338 secs 2023-09-10T05:57:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:56 smithi170 ceph-mon[102832]: pgmap v12910: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 697 KiB/s wr, 19 op/s 2023-09-10T05:57:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:57:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:57:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:57:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:56 smithi082 ceph-mon[98694]: pgmap v12910: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 697 KiB/s wr, 19 op/s 2023-09-10T05:57:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:57:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:57:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:57:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:56 smithi098 ceph-mon[103124]: pgmap v12910: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 697 KiB/s wr, 19 op/s 2023-09-10T05:57:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:57:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:57:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:57:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:57:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:57:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:58 smithi170 ceph-mon[102832]: pgmap v12911: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 416 KiB/s wr, 13 op/s 2023-09-10T05:57:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23909.435792 secs 2023-09-10T05:57:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:57:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:57:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:58 smithi082 ceph-mon[98694]: pgmap v12911: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 416 KiB/s wr, 13 op/s 2023-09-10T05:57:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23909.435792 secs 2023-09-10T05:57:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:57:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:57:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:58 smithi098 ceph-mon[103124]: pgmap v12911: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 416 KiB/s wr, 13 op/s 2023-09-10T05:57:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23909.435792 secs 2023-09-10T05:57:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23910.405635 secs 2023-09-10T05:57:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:57:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23911.059422 secs 2023-09-10T05:57:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23910.405635 secs 2023-09-10T05:57:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:57:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23911.059422 secs 2023-09-10T05:57:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23910.405635 secs 2023-09-10T05:57:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:57:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23911.059422 secs 2023-09-10T05:58:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:58:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:58:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:00 smithi170 ceph-mon[102832]: pgmap v12912: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 416 KiB/s wr, 13 op/s 2023-09-10T05:58:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:58:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:58:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:00 smithi082 ceph-mon[98694]: pgmap v12912: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 416 KiB/s wr, 13 op/s 2023-09-10T05:58:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:58:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:58:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:00 smithi098 ceph-mon[103124]: pgmap v12912: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 416 KiB/s wr, 13 op/s 2023-09-10T05:58:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:02 smithi082 ceph-mon[98694]: pgmap v12913: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 588 KiB/s wr, 18 op/s 2023-09-10T05:58:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:02 smithi098 ceph-mon[103124]: pgmap v12913: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 588 KiB/s wr, 18 op/s 2023-09-10T05:58:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:02 smithi170 ceph-mon[102832]: pgmap v12913: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 588 KiB/s wr, 18 op/s 2023-09-10T05:58:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23914.435852 secs 2023-09-10T05:58:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23914.435852 secs 2023-09-10T05:58:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23914.435852 secs 2023-09-10T05:58:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23915.405745 secs 2023-09-10T05:58:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:04 smithi082 ceph-mon[98694]: pgmap v12914: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 262 KiB/s wr, 11 op/s 2023-09-10T05:58:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23916.059487 secs 2023-09-10T05:58:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23915.405745 secs 2023-09-10T05:58:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:04 smithi098 ceph-mon[103124]: pgmap v12914: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 262 KiB/s wr, 11 op/s 2023-09-10T05:58:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23916.059487 secs 2023-09-10T05:58:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23915.405745 secs 2023-09-10T05:58:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:04 smithi170 ceph-mon[102832]: pgmap v12914: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 262 KiB/s wr, 11 op/s 2023-09-10T05:58:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23916.059487 secs 2023-09-10T05:58:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:06 smithi170 ceph-mon[102832]: pgmap v12915: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 388 KiB/s wr, 32 op/s 2023-09-10T05:58:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:06 smithi082 ceph-mon[98694]: pgmap v12915: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 388 KiB/s wr, 32 op/s 2023-09-10T05:58:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:06 smithi098 ceph-mon[103124]: pgmap v12915: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 388 KiB/s wr, 32 op/s 2023-09-10T05:58:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:08 smithi082 ceph-mon[98694]: pgmap v12916: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 297 KiB/s wr, 26 op/s 2023-09-10T05:58:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23919.435913 secs 2023-09-10T05:58:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:08 smithi098 ceph-mon[103124]: pgmap v12916: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 297 KiB/s wr, 26 op/s 2023-09-10T05:58:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23919.435913 secs 2023-09-10T05:58:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:08 smithi170 ceph-mon[102832]: pgmap v12916: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 297 KiB/s wr, 26 op/s 2023-09-10T05:58:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23919.435913 secs 2023-09-10T05:58:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23920.405821 secs 2023-09-10T05:58:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:09 smithi098 ceph-mon[103124]: osdmap e875: 12 total, 12 up, 12 in 2023-09-10T05:58:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23921.059555 secs 2023-09-10T05:58:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23920.405821 secs 2023-09-10T05:58:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:09 smithi170 ceph-mon[102832]: osdmap e875: 12 total, 12 up, 12 in 2023-09-10T05:58:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23921.059555 secs 2023-09-10T05:58:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23920.405821 secs 2023-09-10T05:58:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:09 smithi082 ceph-mon[98694]: osdmap e875: 12 total, 12 up, 12 in 2023-09-10T05:58:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23921.059555 secs 2023-09-10T05:58:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:10 smithi098 ceph-mon[103124]: pgmap v12918: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 356 KiB/s wr, 31 op/s 2023-09-10T05:58:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:10 smithi170 ceph-mon[102832]: pgmap v12918: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 356 KiB/s wr, 31 op/s 2023-09-10T05:58:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:10 smithi082 ceph-mon[98694]: pgmap v12918: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 356 KiB/s wr, 31 op/s 2023-09-10T05:58:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:12 smithi098 ceph-mon[103124]: pgmap v12919: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 173 KiB/s wr, 31 op/s 2023-09-10T05:58:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:12 smithi170 ceph-mon[102832]: pgmap v12919: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 173 KiB/s wr, 31 op/s 2023-09-10T05:58:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:12 smithi082 ceph-mon[98694]: pgmap v12919: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 173 KiB/s wr, 31 op/s 2023-09-10T05:58:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23924.435971 secs 2023-09-10T05:58:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23924.435971 secs 2023-09-10T05:58:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23924.435971 secs 2023-09-10T05:58:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23925.405878 secs 2023-09-10T05:58:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:14 smithi098 ceph-mon[103124]: pgmap v12920: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 173 KiB/s wr, 31 op/s 2023-09-10T05:58:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23926.059619 secs 2023-09-10T05:58:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23925.405878 secs 2023-09-10T05:58:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:14 smithi170 ceph-mon[102832]: pgmap v12920: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 173 KiB/s wr, 31 op/s 2023-09-10T05:58:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23926.059619 secs 2023-09-10T05:58:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23925.405878 secs 2023-09-10T05:58:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:14 smithi082 ceph-mon[98694]: pgmap v12920: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 173 KiB/s wr, 31 op/s 2023-09-10T05:58:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23926.059619 secs 2023-09-10T05:58:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:16 smithi170 ceph-mon[102832]: osdmap e876: 12 total, 12 up, 12 in 2023-09-10T05:58:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:16 smithi170 ceph-mon[102832]: pgmap v12922: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 15 op/s 2023-09-10T05:58:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:16 smithi082 ceph-mon[98694]: osdmap e876: 12 total, 12 up, 12 in 2023-09-10T05:58:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:16 smithi082 ceph-mon[98694]: pgmap v12922: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 15 op/s 2023-09-10T05:58:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:16 smithi098 ceph-mon[103124]: osdmap e876: 12 total, 12 up, 12 in 2023-09-10T05:58:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:16 smithi098 ceph-mon[103124]: pgmap v12922: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 15 op/s 2023-09-10T05:58:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:18 smithi082 ceph-mon[98694]: pgmap v12923: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 14 op/s 2023-09-10T05:58:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23929.436029 secs 2023-09-10T05:58:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:18 smithi098 ceph-mon[103124]: pgmap v12923: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 14 op/s 2023-09-10T05:58:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23929.436029 secs 2023-09-10T05:58:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:18 smithi170 ceph-mon[102832]: pgmap v12923: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 14 op/s 2023-09-10T05:58:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23929.436029 secs 2023-09-10T05:58:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23930.405935 secs 2023-09-10T05:58:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23931.059685 secs 2023-09-10T05:58:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23930.405935 secs 2023-09-10T05:58:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23931.059685 secs 2023-09-10T05:58:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23930.405935 secs 2023-09-10T05:58:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23931.059685 secs 2023-09-10T05:58:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:20 smithi098 ceph-mon[103124]: pgmap v12924: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T05:58:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:58:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:20 smithi170 ceph-mon[102832]: pgmap v12924: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T05:58:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:58:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:20 smithi082 ceph-mon[98694]: pgmap v12924: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T05:58:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:58:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:58:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:58:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:58:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:58:21.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:58:21.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:58:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:58:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:58:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:58:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:22 smithi098 ceph-mon[103124]: pgmap v12925: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T05:58:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:22 smithi170 ceph-mon[102832]: pgmap v12925: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T05:58:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:22 smithi082 ceph-mon[98694]: pgmap v12925: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T05:58:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23934.436113 secs 2023-09-10T05:58:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23934.436113 secs 2023-09-10T05:58:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23934.436113 secs 2023-09-10T05:58:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23935.406024 secs 2023-09-10T05:58:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:24 smithi098 ceph-mon[103124]: pgmap v12926: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T05:58:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23936.059749 secs 2023-09-10T05:58:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23935.406024 secs 2023-09-10T05:58:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:24 smithi170 ceph-mon[102832]: pgmap v12926: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T05:58:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23936.059749 secs 2023-09-10T05:58:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23935.406024 secs 2023-09-10T05:58:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:24 smithi082 ceph-mon[98694]: pgmap v12926: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T05:58:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23936.059749 secs 2023-09-10T05:58:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:26 smithi170 ceph-mon[102832]: pgmap v12927: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 96 KiB/s wr, 14 op/s 2023-09-10T05:58:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:26 smithi082 ceph-mon[98694]: pgmap v12927: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 96 KiB/s wr, 14 op/s 2023-09-10T05:58:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:26 smithi098 ceph-mon[103124]: pgmap v12927: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 96 KiB/s wr, 14 op/s 2023-09-10T05:58:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:28 smithi082 ceph-mon[98694]: pgmap v12928: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T05:58:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:28 smithi098 ceph-mon[103124]: pgmap v12928: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T05:58:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:28 smithi170 ceph-mon[102832]: pgmap v12928: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T05:58:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23940.406100 secs 2023-09-10T05:58:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23941.059816 secs 2023-09-10T05:58:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23940.406100 secs 2023-09-10T05:58:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23941.059816 secs 2023-09-10T05:58:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23940.406100 secs 2023-09-10T05:58:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23941.059816 secs 2023-09-10T05:58:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:30 smithi098 ceph-mon[103124]: pgmap v12929: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T05:58:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:58:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:58:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:30 smithi170 ceph-mon[102832]: pgmap v12929: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T05:58:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:58:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:58:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:30 smithi082 ceph-mon[98694]: pgmap v12929: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T05:58:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:58:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:58:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:58:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:58:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:58:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:32 smithi098 ceph-mon[103124]: pgmap v12930: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 179 KiB/s wr, 15 op/s 2023-09-10T05:58:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:58:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:58:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:58:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:32 smithi170 ceph-mon[102832]: pgmap v12930: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 179 KiB/s wr, 15 op/s 2023-09-10T05:58:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:58:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:58:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:58:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:32 smithi082 ceph-mon[98694]: pgmap v12930: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 179 KiB/s wr, 15 op/s 2023-09-10T05:58:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:58:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:58:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:58:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23945.406156 secs 2023-09-10T05:58:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:34 smithi098 ceph-mon[103124]: pgmap v12931: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 10 op/s 2023-09-10T05:58:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23946.059882 secs 2023-09-10T05:58:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23945.406156 secs 2023-09-10T05:58:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:34 smithi170 ceph-mon[102832]: pgmap v12931: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 10 op/s 2023-09-10T05:58:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23946.059882 secs 2023-09-10T05:58:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23945.406156 secs 2023-09-10T05:58:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:34 smithi082 ceph-mon[98694]: pgmap v12931: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 10 op/s 2023-09-10T05:58:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23946.059882 secs 2023-09-10T05:58:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:36 smithi170 ceph-mon[102832]: pgmap v12932: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 270 KiB/s wr, 15 op/s 2023-09-10T05:58:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:36 smithi082 ceph-mon[98694]: pgmap v12932: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 270 KiB/s wr, 15 op/s 2023-09-10T05:58:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:36 smithi098 ceph-mon[103124]: pgmap v12932: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 270 KiB/s wr, 15 op/s 2023-09-10T05:58:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:38 smithi082 ceph-mon[98694]: pgmap v12933: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 270 KiB/s wr, 11 op/s 2023-09-10T05:58:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:38 smithi098 ceph-mon[103124]: pgmap v12933: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 270 KiB/s wr, 11 op/s 2023-09-10T05:58:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:38 smithi170 ceph-mon[102832]: pgmap v12933: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 270 KiB/s wr, 11 op/s 2023-09-10T05:58:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23950.406225 secs 2023-09-10T05:58:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23951.059944 secs 2023-09-10T05:58:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23950.406225 secs 2023-09-10T05:58:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23951.059944 secs 2023-09-10T05:58:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23950.406225 secs 2023-09-10T05:58:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23951.059944 secs 2023-09-10T05:58:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:40 smithi098 ceph-mon[103124]: pgmap v12934: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 270 KiB/s wr, 11 op/s 2023-09-10T05:58:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:40 smithi170 ceph-mon[102832]: pgmap v12934: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 270 KiB/s wr, 11 op/s 2023-09-10T05:58:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:40 smithi082 ceph-mon[98694]: pgmap v12934: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 270 KiB/s wr, 11 op/s 2023-09-10T05:58:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:42 smithi098 ceph-mon[103124]: pgmap v12935: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 391 KiB/s wr, 17 op/s 2023-09-10T05:58:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:42 smithi170 ceph-mon[102832]: pgmap v12935: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 391 KiB/s wr, 17 op/s 2023-09-10T05:58:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:42 smithi082 ceph-mon[98694]: pgmap v12935: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 391 KiB/s wr, 17 op/s 2023-09-10T05:58:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23955.406312 secs 2023-09-10T05:58:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:44 smithi098 ceph-mon[103124]: pgmap v12936: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 12 op/s 2023-09-10T05:58:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:58:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:58:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23956.060010 secs 2023-09-10T05:58:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23955.406312 secs 2023-09-10T05:58:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:44 smithi170 ceph-mon[102832]: pgmap v12936: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 12 op/s 2023-09-10T05:58:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:58:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:58:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23956.060010 secs 2023-09-10T05:58:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23955.406312 secs 2023-09-10T05:58:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:44 smithi082 ceph-mon[98694]: pgmap v12936: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 12 op/s 2023-09-10T05:58:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:58:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:58:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23956.060010 secs 2023-09-10T05:58:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:46 smithi098 ceph-mon[103124]: pgmap v12937: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 484 KiB/s wr, 17 op/s 2023-09-10T05:58:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:58:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:58:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:46 smithi170 ceph-mon[102832]: pgmap v12937: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 484 KiB/s wr, 17 op/s 2023-09-10T05:58:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:58:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:58:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:46 smithi082 ceph-mon[98694]: pgmap v12937: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 484 KiB/s wr, 17 op/s 2023-09-10T05:58:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:58:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:58:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:48 smithi098 ceph-mon[103124]: pgmap v12938: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 312 KiB/s wr, 12 op/s 2023-09-10T05:58:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:48 smithi170 ceph-mon[102832]: pgmap v12938: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 312 KiB/s wr, 12 op/s 2023-09-10T05:58:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:48 smithi082 ceph-mon[98694]: pgmap v12938: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 312 KiB/s wr, 12 op/s 2023-09-10T05:58:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23960.406370 secs 2023-09-10T05:58:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23961.060074 secs 2023-09-10T05:58:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23960.406370 secs 2023-09-10T05:58:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23961.060074 secs 2023-09-10T05:58:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23960.406370 secs 2023-09-10T05:58:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23961.060074 secs 2023-09-10T05:58:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:50 smithi098 ceph-mon[103124]: pgmap v12939: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 312 KiB/s wr, 12 op/s 2023-09-10T05:58:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:50 smithi170 ceph-mon[102832]: pgmap v12939: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 312 KiB/s wr, 12 op/s 2023-09-10T05:58:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:50 smithi082 ceph-mon[98694]: pgmap v12939: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 312 KiB/s wr, 12 op/s 2023-09-10T05:58:52.741 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:52 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mon-b[103100]: problem writing to /var/log/ceph/ceph-mon.b.log: (28) No space left on device 2023-09-10T05:58:52.742 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:52 smithi098 ceph-mon[103124]: pgmap v12940: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 444 KiB/s wr, 18 op/s 2023-09-10T05:58:52.742 INFO:journalctl@ceph.osd.4.smithi098.stdout:Sep 10 05:58:52 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-4[116158]: problem writing to /var/log/ceph/ceph-osd.4.log: (28) No space left on device 2023-09-10T05:58:52.742 INFO:journalctl@ceph.osd.1.smithi098.stdout:Sep 10 05:58:52 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-1[108236]: problem writing to /var/log/ceph/ceph-osd.1.log: (28) No space left on device 2023-09-10T05:58:52.743 INFO:journalctl@ceph.osd.7.smithi098.stdout:Sep 10 05:58:52 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-7[124140]: problem writing to /var/log/ceph/ceph-osd.7.log: (28) No space left on device 2023-09-10T05:58:52.743 INFO:journalctl@ceph.osd.10.smithi098.stdout:Sep 10 05:58:52 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-osd-10[132273]: problem writing to /var/log/ceph/ceph-osd.10.log: (28) No space left on device 2023-09-10T05:58:52.744 INFO:journalctl@ceph.mds.b.smithi098.stdout:Sep 10 05:58:52 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mds-b[139333]: problem writing to /var/log/ceph/ceph-mds.b.log: (28) No space left on device 2023-09-10T05:58:52.744 INFO:journalctl@ceph.mds.e.smithi098.stdout:Sep 10 05:58:52 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mds-e[138647]: problem writing to /var/log/ceph/ceph-mds.e.log: (28) No space left on device 2023-09-10T05:58:52.744 INFO:journalctl@ceph.mds.k.smithi098.stdout:Sep 10 05:58:52 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mds-k[138306]: problem writing to /var/log/ceph/ceph-mds.k.log: (28) No space left on device 2023-09-10T05:58:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:52 smithi170 ceph-mon[102832]: pgmap v12940: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 444 KiB/s wr, 18 op/s 2023-09-10T05:58:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:52 smithi082 ceph-mon[98694]: pgmap v12940: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 444 KiB/s wr, 18 op/s 2023-09-10T05:58:53.003 INFO:journalctl@ceph.mds.h.smithi098.stdout:Sep 10 05:58:52 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mds-h[138988]: problem writing to /var/log/ceph/ceph-mds.h.log: (28) No space left on device 2023-09-10T05:58:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23965.406430 secs 2023-09-10T05:58:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:54 smithi098 ceph-mon[103124]: pgmap v12941: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 323 KiB/s wr, 11 op/s 2023-09-10T05:58:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:58:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:58:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23966.060145 secs 2023-09-10T05:58:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23965.406430 secs 2023-09-10T05:58:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:54 smithi170 ceph-mon[102832]: pgmap v12941: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 323 KiB/s wr, 11 op/s 2023-09-10T05:58:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:58:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:58:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23966.060145 secs 2023-09-10T05:58:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23965.406430 secs 2023-09-10T05:58:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:54 smithi082 ceph-mon[98694]: pgmap v12941: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 323 KiB/s wr, 11 op/s 2023-09-10T05:58:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:58:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:58:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23966.060145 secs 2023-09-10T05:58:55.753 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Sep 10 05:58:55 smithi098 ceph-3a1ab9b6-4f65-11ee-9ab7-7b867c8bd7da-mgr-y[104603]: problem writing to /var/log/ceph/ceph-mgr.y.log: (28) No space left on device 2023-09-10T05:58:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:56 smithi098 ceph-mon[103124]: pgmap v12942: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 474 KiB/s wr, 17 op/s 2023-09-10T05:58:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:58:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:58:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:56 smithi170 ceph-mon[102832]: pgmap v12942: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 474 KiB/s wr, 17 op/s 2023-09-10T05:58:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:58:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:58:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:56 smithi082 ceph-mon[98694]: pgmap v12942: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 474 KiB/s wr, 17 op/s 2023-09-10T05:58:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:58:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:58:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:58 smithi170 ceph-mon[102832]: pgmap v12943: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 283 KiB/s wr, 12 op/s 2023-09-10T05:58:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:58:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:58:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23969.436676 secs 2023-09-10T05:58:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:58 smithi082 ceph-mon[98694]: pgmap v12943: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 283 KiB/s wr, 12 op/s 2023-09-10T05:58:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:58:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:58:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23969.436676 secs 2023-09-10T05:58:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:58 smithi098 ceph-mon[103124]: pgmap v12943: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 283 KiB/s wr, 12 op/s 2023-09-10T05:58:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:58:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:58:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23969.436676 secs 2023-09-10T05:58:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23970.406508 secs 2023-09-10T05:58:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:58:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23971.060210 secs 2023-09-10T05:58:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23970.406508 secs 2023-09-10T05:58:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:58:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23971.060210 secs 2023-09-10T05:59:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23970.406508 secs 2023-09-10T05:59:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:58:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23971.060210 secs 2023-09-10T05:59:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:00 smithi170 ceph-mon[102832]: pgmap v12944: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-09-10T05:59:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:59:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:59:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:59:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:00 smithi082 ceph-mon[98694]: pgmap v12944: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-09-10T05:59:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:59:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:59:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:59:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:00 smithi098 ceph-mon[103124]: pgmap v12944: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-09-10T05:59:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:59:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:59:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:59:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:02 smithi170 ceph-mon[102832]: pgmap v12945: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 731 KiB/s wr, 32 op/s 2023-09-10T05:59:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:02 smithi082 ceph-mon[98694]: pgmap v12945: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 731 KiB/s wr, 32 op/s 2023-09-10T05:59:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:02 smithi098 ceph-mon[103124]: pgmap v12945: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 731 KiB/s wr, 32 op/s 2023-09-10T05:59:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23974.436749 secs 2023-09-10T05:59:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23974.436749 secs 2023-09-10T05:59:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23974.436749 secs 2023-09-10T05:59:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23975.406619 secs 2023-09-10T05:59:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:04 smithi170 ceph-mon[102832]: pgmap v12946: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 598 KiB/s wr, 26 op/s 2023-09-10T05:59:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:04 smithi170 ceph-mon[102832]: osdmap e877: 12 total, 12 up, 12 in 2023-09-10T05:59:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23976.060278 secs 2023-09-10T05:59:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23975.406619 secs 2023-09-10T05:59:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:04 smithi082 ceph-mon[98694]: pgmap v12946: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 598 KiB/s wr, 26 op/s 2023-09-10T05:59:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:04 smithi082 ceph-mon[98694]: osdmap e877: 12 total, 12 up, 12 in 2023-09-10T05:59:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23976.060278 secs 2023-09-10T05:59:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23975.406619 secs 2023-09-10T05:59:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:04 smithi098 ceph-mon[103124]: pgmap v12946: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 598 KiB/s wr, 26 op/s 2023-09-10T05:59:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:04 smithi098 ceph-mon[103124]: osdmap e877: 12 total, 12 up, 12 in 2023-09-10T05:59:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23976.060278 secs 2023-09-10T05:59:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:06 smithi170 ceph-mon[102832]: pgmap v12948: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 685 KiB/s wr, 32 op/s 2023-09-10T05:59:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:06 smithi082 ceph-mon[98694]: pgmap v12948: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 685 KiB/s wr, 32 op/s 2023-09-10T05:59:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:06 smithi098 ceph-mon[103124]: pgmap v12948: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 685 KiB/s wr, 32 op/s 2023-09-10T05:59:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:08 smithi098 ceph-mon[103124]: pgmap v12949: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 685 KiB/s wr, 32 op/s 2023-09-10T05:59:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23979.436774 secs 2023-09-10T05:59:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:08 smithi170 ceph-mon[102832]: pgmap v12949: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 685 KiB/s wr, 32 op/s 2023-09-10T05:59:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23979.436774 secs 2023-09-10T05:59:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:08 smithi082 ceph-mon[98694]: pgmap v12949: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 685 KiB/s wr, 32 op/s 2023-09-10T05:59:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23979.436774 secs 2023-09-10T05:59:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23980.406702 secs 2023-09-10T05:59:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23981.060342 secs 2023-09-10T05:59:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23980.406702 secs 2023-09-10T05:59:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23981.060342 secs 2023-09-10T05:59:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23980.406702 secs 2023-09-10T05:59:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23981.060342 secs 2023-09-10T05:59:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:10 smithi098 ceph-mon[103124]: pgmap v12950: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 685 KiB/s wr, 32 op/s 2023-09-10T05:59:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:10 smithi098 ceph-mon[103124]: osdmap e878: 12 total, 12 up, 12 in 2023-09-10T05:59:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:10 smithi170 ceph-mon[102832]: pgmap v12950: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 685 KiB/s wr, 32 op/s 2023-09-10T05:59:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:10 smithi170 ceph-mon[102832]: osdmap e878: 12 total, 12 up, 12 in 2023-09-10T05:59:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:10 smithi082 ceph-mon[98694]: pgmap v12950: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 685 KiB/s wr, 32 op/s 2023-09-10T05:59:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:10 smithi082 ceph-mon[98694]: osdmap e878: 12 total, 12 up, 12 in 2023-09-10T05:59:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:12 smithi098 ceph-mon[103124]: pgmap v12952: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 186 KiB/s wr, 15 op/s 2023-09-10T05:59:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:12 smithi170 ceph-mon[102832]: pgmap v12952: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 186 KiB/s wr, 15 op/s 2023-09-10T05:59:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:12 smithi082 ceph-mon[98694]: pgmap v12952: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 186 KiB/s wr, 15 op/s 2023-09-10T05:59:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23984.436835 secs 2023-09-10T05:59:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23984.436835 secs 2023-09-10T05:59:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23984.436835 secs 2023-09-10T05:59:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23985.406761 secs 2023-09-10T05:59:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:14 smithi098 ceph-mon[103124]: pgmap v12953: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T05:59:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23986.060411 secs 2023-09-10T05:59:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23985.406761 secs 2023-09-10T05:59:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:14 smithi170 ceph-mon[102832]: pgmap v12953: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T05:59:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23986.060411 secs 2023-09-10T05:59:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23985.406761 secs 2023-09-10T05:59:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:14 smithi082 ceph-mon[98694]: pgmap v12953: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 150 KiB/s wr, 12 op/s 2023-09-10T05:59:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23986.060411 secs 2023-09-10T05:59:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:16 smithi170 ceph-mon[102832]: pgmap v12954: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-10T05:59:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:16 smithi082 ceph-mon[98694]: pgmap v12954: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-10T05:59:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:16 smithi098 ceph-mon[103124]: pgmap v12954: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-10T05:59:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:18 smithi098 ceph-mon[103124]: pgmap v12955: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-10T05:59:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23989.436922 secs 2023-09-10T05:59:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:18 smithi170 ceph-mon[102832]: pgmap v12955: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-10T05:59:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23989.436922 secs 2023-09-10T05:59:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:18 smithi082 ceph-mon[98694]: pgmap v12955: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-10T05:59:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23989.436922 secs 2023-09-10T05:59:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23990.406820 secs 2023-09-10T05:59:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23991.060481 secs 2023-09-10T05:59:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23990.406820 secs 2023-09-10T05:59:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23991.060481 secs 2023-09-10T05:59:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23990.406820 secs 2023-09-10T05:59:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23991.060481 secs 2023-09-10T05:59:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:20 smithi098 ceph-mon[103124]: pgmap v12956: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-10T05:59:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:20 smithi170 ceph-mon[102832]: pgmap v12956: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-10T05:59:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:20 smithi082 ceph-mon[98694]: pgmap v12956: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-10T05:59:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:59:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:59:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:59:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:59:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:59:21.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:59:21.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:59:21.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:59:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T05:59:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T05:59:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T05:59:21.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T05:59:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:22 smithi098 ceph-mon[103124]: pgmap v12957: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-09-10T05:59:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:22 smithi170 ceph-mon[102832]: pgmap v12957: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-09-10T05:59:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:22 smithi082 ceph-mon[98694]: pgmap v12957: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-09-10T05:59:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 23994.437051 secs 2023-09-10T05:59:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 23994.437051 secs 2023-09-10T05:59:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 23994.437051 secs 2023-09-10T05:59:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23995.406880 secs 2023-09-10T05:59:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:24 smithi098 ceph-mon[103124]: pgmap v12958: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2023-09-10T05:59:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 23996.060546 secs 2023-09-10T05:59:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23995.406880 secs 2023-09-10T05:59:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:24 smithi170 ceph-mon[102832]: pgmap v12958: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2023-09-10T05:59:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 23996.060546 secs 2023-09-10T05:59:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23995.406880 secs 2023-09-10T05:59:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:24 smithi082 ceph-mon[98694]: pgmap v12958: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2023-09-10T05:59:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 23996.060546 secs 2023-09-10T05:59:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:26 smithi170 ceph-mon[102832]: pgmap v12959: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 82 KiB/s wr, 14 op/s 2023-09-10T05:59:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:26 smithi082 ceph-mon[98694]: pgmap v12959: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 82 KiB/s wr, 14 op/s 2023-09-10T05:59:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:26 smithi098 ceph-mon[103124]: pgmap v12959: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 82 KiB/s wr, 14 op/s 2023-09-10T05:59:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:28 smithi098 ceph-mon[103124]: pgmap v12960: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T05:59:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:28 smithi170 ceph-mon[102832]: pgmap v12960: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T05:59:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:28 smithi082 ceph-mon[98694]: pgmap v12960: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T05:59:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24000.406937 secs 2023-09-10T05:59:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24001.060611 secs 2023-09-10T05:59:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24000.406937 secs 2023-09-10T05:59:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24001.060611 secs 2023-09-10T05:59:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24000.406937 secs 2023-09-10T05:59:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24001.060611 secs 2023-09-10T05:59:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:30 smithi098 ceph-mon[103124]: pgmap v12961: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T05:59:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:30 smithi170 ceph-mon[102832]: pgmap v12961: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T05:59:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:30 smithi082 ceph-mon[98694]: pgmap v12961: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 81 KiB/s wr, 10 op/s 2023-09-10T05:59:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:59:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:59:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T05:59:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:32 smithi098 ceph-mon[103124]: pgmap v12962: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 159 KiB/s wr, 15 op/s 2023-09-10T05:59:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:59:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:59:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:59:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:32 smithi170 ceph-mon[102832]: pgmap v12962: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 159 KiB/s wr, 15 op/s 2023-09-10T05:59:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:59:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:59:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:59:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:32 smithi082 ceph-mon[98694]: pgmap v12962: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 159 KiB/s wr, 15 op/s 2023-09-10T05:59:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:59:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:59:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:59:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24005.406995 secs 2023-09-10T05:59:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:34 smithi098 ceph-mon[103124]: pgmap v12963: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 159 KiB/s wr, 10 op/s 2023-09-10T05:59:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:59:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:59:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24006.060675 secs 2023-09-10T05:59:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24005.406995 secs 2023-09-10T05:59:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:34 smithi170 ceph-mon[102832]: pgmap v12963: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 159 KiB/s wr, 10 op/s 2023-09-10T05:59:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:59:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:59:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24006.060675 secs 2023-09-10T05:59:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24005.406995 secs 2023-09-10T05:59:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:34 smithi082 ceph-mon[98694]: pgmap v12963: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 159 KiB/s wr, 10 op/s 2023-09-10T05:59:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:59:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:59:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24006.060675 secs 2023-09-10T05:59:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:36 smithi098 ceph-mon[103124]: pgmap v12964: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 583 KiB/s wr, 18 op/s 2023-09-10T05:59:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:59:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:59:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:36 smithi170 ceph-mon[102832]: pgmap v12964: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 583 KiB/s wr, 18 op/s 2023-09-10T05:59:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:59:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:59:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:36 smithi082 ceph-mon[98694]: pgmap v12964: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 583 KiB/s wr, 18 op/s 2023-09-10T05:59:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:59:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:59:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:38 smithi098 ceph-mon[103124]: pgmap v12965: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 502 KiB/s wr, 12 op/s 2023-09-10T05:59:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:38 smithi170 ceph-mon[102832]: pgmap v12965: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 502 KiB/s wr, 12 op/s 2023-09-10T05:59:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:38 smithi082 ceph-mon[98694]: pgmap v12965: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 502 KiB/s wr, 12 op/s 2023-09-10T05:59:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24010.407058 secs 2023-09-10T05:59:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24011.060743 secs 2023-09-10T05:59:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24010.407058 secs 2023-09-10T05:59:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24011.060743 secs 2023-09-10T05:59:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24010.407058 secs 2023-09-10T05:59:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24011.060743 secs 2023-09-10T05:59:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:40 smithi098 ceph-mon[103124]: pgmap v12966: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 502 KiB/s wr, 12 op/s 2023-09-10T05:59:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:40 smithi170 ceph-mon[102832]: pgmap v12966: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 502 KiB/s wr, 12 op/s 2023-09-10T05:59:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:40 smithi082 ceph-mon[98694]: pgmap v12966: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 502 KiB/s wr, 12 op/s 2023-09-10T05:59:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:42 smithi098 ceph-mon[103124]: pgmap v12967: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 519 KiB/s wr, 18 op/s 2023-09-10T05:59:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:42 smithi170 ceph-mon[102832]: pgmap v12967: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 519 KiB/s wr, 18 op/s 2023-09-10T05:59:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:42 smithi082 ceph-mon[98694]: pgmap v12967: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 519 KiB/s wr, 18 op/s 2023-09-10T05:59:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24015.407142 secs 2023-09-10T05:59:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:44 smithi170 ceph-mon[102832]: pgmap v12968: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 441 KiB/s wr, 13 op/s 2023-09-10T05:59:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:59:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:59:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24016.060808 secs 2023-09-10T05:59:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24015.407142 secs 2023-09-10T05:59:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:44 smithi082 ceph-mon[98694]: pgmap v12968: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 441 KiB/s wr, 13 op/s 2023-09-10T05:59:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:59:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:59:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24016.060808 secs 2023-09-10T05:59:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24015.407142 secs 2023-09-10T05:59:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:44 smithi098 ceph-mon[103124]: pgmap v12968: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 441 KiB/s wr, 13 op/s 2023-09-10T05:59:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:59:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:59:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24016.060808 secs 2023-09-10T05:59:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:46 smithi170 ceph-mon[102832]: pgmap v12969: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 724 KiB/s wr, 19 op/s 2023-09-10T05:59:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:59:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:59:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:46 smithi082 ceph-mon[98694]: pgmap v12969: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 724 KiB/s wr, 19 op/s 2023-09-10T05:59:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:59:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:59:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:46 smithi098 ceph-mon[103124]: pgmap v12969: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 724 KiB/s wr, 19 op/s 2023-09-10T05:59:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:59:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:59:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:48 smithi170 ceph-mon[102832]: pgmap v12970: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 299 KiB/s wr, 12 op/s 2023-09-10T05:59:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:48 smithi082 ceph-mon[98694]: pgmap v12970: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 299 KiB/s wr, 12 op/s 2023-09-10T05:59:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:48 smithi098 ceph-mon[103124]: pgmap v12970: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 299 KiB/s wr, 12 op/s 2023-09-10T05:59:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24020.407221 secs 2023-09-10T05:59:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24021.060896 secs 2023-09-10T05:59:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24020.407221 secs 2023-09-10T05:59:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24021.060896 secs 2023-09-10T05:59:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24020.407221 secs 2023-09-10T05:59:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24021.060896 secs 2023-09-10T05:59:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:50 smithi170 ceph-mon[102832]: pgmap v12971: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 299 KiB/s wr, 12 op/s 2023-09-10T05:59:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:59:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:59:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:50 smithi082 ceph-mon[98694]: pgmap v12971: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 299 KiB/s wr, 12 op/s 2023-09-10T05:59:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:59:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:59:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:50 smithi098 ceph-mon[103124]: pgmap v12971: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 299 KiB/s wr, 12 op/s 2023-09-10T05:59:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T05:59:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:59:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:52 smithi170 ceph-mon[102832]: pgmap v12972: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 656 KiB/s wr, 19 op/s 2023-09-10T05:59:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:59:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:59:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:52 smithi082 ceph-mon[98694]: pgmap v12972: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 656 KiB/s wr, 19 op/s 2023-09-10T05:59:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:59:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:59:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:52 smithi098 ceph-mon[103124]: pgmap v12972: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 656 KiB/s wr, 19 op/s 2023-09-10T05:59:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:59:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:59:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24024.437523 secs 2023-09-10T05:59:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24024.437523 secs 2023-09-10T05:59:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24024.437523 secs 2023-09-10T05:59:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24025.407284 secs 2023-09-10T05:59:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:54 smithi170 ceph-mon[102832]: pgmap v12973: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 640 KiB/s wr, 13 op/s 2023-09-10T05:59:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24026.060962 secs 2023-09-10T05:59:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24025.407284 secs 2023-09-10T05:59:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:54 smithi082 ceph-mon[98694]: pgmap v12973: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 640 KiB/s wr, 13 op/s 2023-09-10T05:59:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24026.060962 secs 2023-09-10T05:59:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24025.407284 secs 2023-09-10T05:59:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:54 smithi098 ceph-mon[103124]: pgmap v12973: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 640 KiB/s wr, 13 op/s 2023-09-10T05:59:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24026.060962 secs 2023-09-10T05:59:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:56 smithi170 ceph-mon[102832]: pgmap v12974: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 715 KiB/s wr, 18 op/s 2023-09-10T05:59:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:59:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:59:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:59:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:56 smithi082 ceph-mon[98694]: pgmap v12974: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 715 KiB/s wr, 18 op/s 2023-09-10T05:59:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:59:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:59:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:59:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:56 smithi098 ceph-mon[103124]: pgmap v12974: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 715 KiB/s wr, 18 op/s 2023-09-10T05:59:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T05:59:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T05:59:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T05:59:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:58 smithi098 ceph-mon[103124]: pgmap v12975: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 432 KiB/s wr, 12 op/s 2023-09-10T05:59:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24029.437586 secs 2023-09-10T05:59:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:58 smithi170 ceph-mon[102832]: pgmap v12975: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 432 KiB/s wr, 12 op/s 2023-09-10T05:59:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24029.437586 secs 2023-09-10T05:59:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:58 smithi082 ceph-mon[98694]: pgmap v12975: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 432 KiB/s wr, 12 op/s 2023-09-10T05:59:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24029.437586 secs 2023-09-10T05:59:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24030.407360 secs 2023-09-10T05:59:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 05:59:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24031.061028 secs 2023-09-10T05:59:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24030.407360 secs 2023-09-10T05:59:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 05:59:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24031.061028 secs 2023-09-10T05:59:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24030.407360 secs 2023-09-10T05:59:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 05:59:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24031.061028 secs 2023-09-10T06:00:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:00 smithi098 ceph-mon[103124]: pgmap v12976: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 432 KiB/s wr, 12 op/s 2023-09-10T06:00:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T06:00:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:00 smithi170 ceph-mon[102832]: pgmap v12976: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 432 KiB/s wr, 12 op/s 2023-09-10T06:00:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T06:00:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:00 smithi082 ceph-mon[98694]: pgmap v12976: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 432 KiB/s wr, 12 op/s 2023-09-10T06:00:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T06:00:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:02 smithi098 ceph-mon[103124]: pgmap v12977: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 552 KiB/s wr, 22 op/s 2023-09-10T06:00:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:02 smithi170 ceph-mon[102832]: pgmap v12977: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 552 KiB/s wr, 22 op/s 2023-09-10T06:00:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:02 smithi082 ceph-mon[98694]: pgmap v12977: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 552 KiB/s wr, 22 op/s 2023-09-10T06:00:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24034.437645 secs 2023-09-10T06:00:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24034.437645 secs 2023-09-10T06:00:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24034.437645 secs 2023-09-10T06:00:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24035.407418 secs 2023-09-10T06:00:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:04 smithi098 ceph-mon[103124]: pgmap v12978: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 196 KiB/s wr, 15 op/s 2023-09-10T06:00:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24036.061092 secs 2023-09-10T06:00:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24035.407418 secs 2023-09-10T06:00:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:04 smithi170 ceph-mon[102832]: pgmap v12978: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 196 KiB/s wr, 15 op/s 2023-09-10T06:00:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24036.061092 secs 2023-09-10T06:00:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24035.407418 secs 2023-09-10T06:00:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:04 smithi082 ceph-mon[98694]: pgmap v12978: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 196 KiB/s wr, 15 op/s 2023-09-10T06:00:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24036.061092 secs 2023-09-10T06:00:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:06 smithi170 ceph-mon[102832]: pgmap v12979: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 299 KiB/s wr, 29 op/s 2023-09-10T06:00:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:06 smithi082 ceph-mon[98694]: pgmap v12979: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 299 KiB/s wr, 29 op/s 2023-09-10T06:00:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:06 smithi098 ceph-mon[103124]: pgmap v12979: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 299 KiB/s wr, 29 op/s 2023-09-10T06:00:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:08 smithi098 ceph-mon[103124]: pgmap v12980: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 224 KiB/s wr, 24 op/s 2023-09-10T06:00:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24039.437707 secs 2023-09-10T06:00:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:08 smithi170 ceph-mon[102832]: pgmap v12980: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 224 KiB/s wr, 24 op/s 2023-09-10T06:00:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24039.437707 secs 2023-09-10T06:00:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:08 smithi082 ceph-mon[98694]: pgmap v12980: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 224 KiB/s wr, 24 op/s 2023-09-10T06:00:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24039.437707 secs 2023-09-10T06:00:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24040.407476 secs 2023-09-10T06:00:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:09 smithi098 ceph-mon[103124]: osdmap e879: 12 total, 12 up, 12 in 2023-09-10T06:00:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24041.061160 secs 2023-09-10T06:00:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24040.407476 secs 2023-09-10T06:00:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:09 smithi170 ceph-mon[102832]: osdmap e879: 12 total, 12 up, 12 in 2023-09-10T06:00:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24041.061160 secs 2023-09-10T06:00:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24040.407476 secs 2023-09-10T06:00:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:09 smithi082 ceph-mon[98694]: osdmap e879: 12 total, 12 up, 12 in 2023-09-10T06:00:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24041.061160 secs 2023-09-10T06:00:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:10 smithi098 ceph-mon[103124]: pgmap v12982: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 269 KiB/s wr, 29 op/s 2023-09-10T06:00:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:10 smithi170 ceph-mon[102832]: pgmap v12982: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 269 KiB/s wr, 29 op/s 2023-09-10T06:00:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:10 smithi082 ceph-mon[98694]: pgmap v12982: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 269 KiB/s wr, 29 op/s 2023-09-10T06:00:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:12 smithi098 ceph-mon[103124]: pgmap v12983: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 147 KiB/s wr, 23 op/s 2023-09-10T06:00:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:12 smithi170 ceph-mon[102832]: pgmap v12983: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 147 KiB/s wr, 23 op/s 2023-09-10T06:00:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:12 smithi082 ceph-mon[98694]: pgmap v12983: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 147 KiB/s wr, 23 op/s 2023-09-10T06:00:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24044.437773 secs 2023-09-10T06:00:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24044.437773 secs 2023-09-10T06:00:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24044.437773 secs 2023-09-10T06:00:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24045.407497 secs 2023-09-10T06:00:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:14 smithi170 ceph-mon[102832]: pgmap v12984: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 147 KiB/s wr, 23 op/s 2023-09-10T06:00:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24046.061222 secs 2023-09-10T06:00:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24045.407497 secs 2023-09-10T06:00:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:14 smithi082 ceph-mon[98694]: pgmap v12984: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 147 KiB/s wr, 23 op/s 2023-09-10T06:00:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24046.061222 secs 2023-09-10T06:00:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24045.407497 secs 2023-09-10T06:00:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:14 smithi098 ceph-mon[103124]: pgmap v12984: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 147 KiB/s wr, 23 op/s 2023-09-10T06:00:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24046.061222 secs 2023-09-10T06:00:16.579 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:16 smithi170 ceph-mon[102832]: osdmap e880: 12 total, 12 up, 12 in 2023-09-10T06:00:16.579 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:16 smithi170 ceph-mon[102832]: pgmap v12986: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 149 KiB/s wr, 16 op/s 2023-09-10T06:00:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:16 smithi082 ceph-mon[98694]: osdmap e880: 12 total, 12 up, 12 in 2023-09-10T06:00:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:16 smithi082 ceph-mon[98694]: pgmap v12986: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 149 KiB/s wr, 16 op/s 2023-09-10T06:00:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:16 smithi098 ceph-mon[103124]: osdmap e880: 12 total, 12 up, 12 in 2023-09-10T06:00:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:16 smithi098 ceph-mon[103124]: pgmap v12986: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 149 KiB/s wr, 16 op/s 2023-09-10T06:00:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:18 smithi098 ceph-mon[103124]: pgmap v12987: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 14 op/s 2023-09-10T06:00:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24049.437839 secs 2023-09-10T06:00:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:18 smithi170 ceph-mon[102832]: pgmap v12987: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 14 op/s 2023-09-10T06:00:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24049.437839 secs 2023-09-10T06:00:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:18 smithi082 ceph-mon[98694]: pgmap v12987: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 14 op/s 2023-09-10T06:00:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24049.437839 secs 2023-09-10T06:00:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24050.407552 secs 2023-09-10T06:00:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24051.061289 secs 2023-09-10T06:00:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24050.407552 secs 2023-09-10T06:00:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24051.061289 secs 2023-09-10T06:00:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24050.407552 secs 2023-09-10T06:00:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24051.061289 secs 2023-09-10T06:00:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:20 smithi098 ceph-mon[103124]: pgmap v12988: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T06:00:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:20 smithi170 ceph-mon[102832]: pgmap v12988: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T06:00:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:20 smithi082 ceph-mon[98694]: pgmap v12988: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T06:00:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:00:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:00:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:00:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:22 smithi098 ceph-mon[103124]: pgmap v12989: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T06:00:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:00:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:00:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:00:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:22 smithi170 ceph-mon[102832]: pgmap v12989: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T06:00:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:00:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:00:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:00:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:22 smithi082 ceph-mon[98694]: pgmap v12989: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T06:00:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:00:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:00:22.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:00:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24054.437911 secs 2023-09-10T06:00:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24054.437911 secs 2023-09-10T06:00:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24054.437911 secs 2023-09-10T06:00:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24055.407611 secs 2023-09-10T06:00:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:24 smithi170 ceph-mon[102832]: pgmap v12990: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T06:00:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:00:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:00:24.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24056.061353 secs 2023-09-10T06:00:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24055.407611 secs 2023-09-10T06:00:24.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:24 smithi082 ceph-mon[98694]: pgmap v12990: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T06:00:24.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:00:24.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:00:24.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24056.061353 secs 2023-09-10T06:00:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24055.407611 secs 2023-09-10T06:00:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:24 smithi098 ceph-mon[103124]: pgmap v12990: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T06:00:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:00:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:00:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24056.061353 secs 2023-09-10T06:00:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:26 smithi170 ceph-mon[102832]: pgmap v12991: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 13 op/s 2023-09-10T06:00:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:26 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:00:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:00:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:26 smithi082 ceph-mon[98694]: pgmap v12991: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 13 op/s 2023-09-10T06:00:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:26 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:00:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:00:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:26 smithi098 ceph-mon[103124]: pgmap v12991: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 13 op/s 2023-09-10T06:00:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:26 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:00:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:00:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:28 smithi170 ceph-mon[102832]: pgmap v12992: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T06:00:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:28 smithi082 ceph-mon[98694]: pgmap v12992: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T06:00:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:28 smithi098 ceph-mon[103124]: pgmap v12992: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T06:00:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24060.407694 secs 2023-09-10T06:00:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24061.061419 secs 2023-09-10T06:00:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24060.407694 secs 2023-09-10T06:00:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24061.061419 secs 2023-09-10T06:00:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24060.407694 secs 2023-09-10T06:00:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24061.061419 secs 2023-09-10T06:00:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:30 smithi170 ceph-mon[102832]: pgmap v12993: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T06:00:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:30 smithi082 ceph-mon[98694]: pgmap v12993: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T06:00:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:30 smithi098 ceph-mon[103124]: pgmap v12993: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T06:00:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:00:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:00:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:00:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:32 smithi170 ceph-mon[102832]: pgmap v12994: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 237 KiB/s wr, 15 op/s 2023-09-10T06:00:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:00:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:00:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:00:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:32 smithi082 ceph-mon[98694]: pgmap v12994: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 237 KiB/s wr, 15 op/s 2023-09-10T06:00:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:00:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:00:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:00:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:32 smithi098 ceph-mon[103124]: pgmap v12994: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 237 KiB/s wr, 15 op/s 2023-09-10T06:00:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:00:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:00:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:00:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24065.407752 secs 2023-09-10T06:00:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:34 smithi170 ceph-mon[102832]: pgmap v12995: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 237 KiB/s wr, 10 op/s 2023-09-10T06:00:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24066.061489 secs 2023-09-10T06:00:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24065.407752 secs 2023-09-10T06:00:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:34 smithi082 ceph-mon[98694]: pgmap v12995: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 237 KiB/s wr, 10 op/s 2023-09-10T06:00:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24066.061489 secs 2023-09-10T06:00:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24065.407752 secs 2023-09-10T06:00:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:34 smithi098 ceph-mon[103124]: pgmap v12995: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 237 KiB/s wr, 10 op/s 2023-09-10T06:00:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24066.061489 secs 2023-09-10T06:00:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:36 smithi170 ceph-mon[102832]: pgmap v12996: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 359 KiB/s wr, 16 op/s 2023-09-10T06:00:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:36 smithi082 ceph-mon[98694]: pgmap v12996: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 359 KiB/s wr, 16 op/s 2023-09-10T06:00:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:36 smithi098 ceph-mon[103124]: pgmap v12996: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 359 KiB/s wr, 16 op/s 2023-09-10T06:00:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:00:38.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:00:38.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:38 smithi170 ceph-mon[102832]: pgmap v12997: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 263 KiB/s wr, 11 op/s 2023-09-10T06:00:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:00:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:00:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:38 smithi082 ceph-mon[98694]: pgmap v12997: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 263 KiB/s wr, 11 op/s 2023-09-10T06:00:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:00:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:00:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:38 smithi098 ceph-mon[103124]: pgmap v12997: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 263 KiB/s wr, 11 op/s 2023-09-10T06:00:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24070.407824 secs 2023-09-10T06:00:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24071.061556 secs 2023-09-10T06:00:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24070.407824 secs 2023-09-10T06:00:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24071.061556 secs 2023-09-10T06:00:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24070.407824 secs 2023-09-10T06:00:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24071.061556 secs 2023-09-10T06:00:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:00:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:00:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:40 smithi170 ceph-mon[102832]: pgmap v12998: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 263 KiB/s wr, 11 op/s 2023-09-10T06:00:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:00:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:00:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:40 smithi082 ceph-mon[98694]: pgmap v12998: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 263 KiB/s wr, 11 op/s 2023-09-10T06:00:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:00:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:00:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:40 smithi098 ceph-mon[103124]: pgmap v12998: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 263 KiB/s wr, 11 op/s 2023-09-10T06:00:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:42 smithi098 ceph-mon[103124]: pgmap v12999: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 578 KiB/s wr, 18 op/s 2023-09-10T06:00:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:42 smithi170 ceph-mon[102832]: pgmap v12999: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 578 KiB/s wr, 18 op/s 2023-09-10T06:00:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:42 smithi082 ceph-mon[98694]: pgmap v12999: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 578 KiB/s wr, 18 op/s 2023-09-10T06:00:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24075.407884 secs 2023-09-10T06:00:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:44 smithi098 ceph-mon[103124]: pgmap v13000: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 437 KiB/s wr, 12 op/s 2023-09-10T06:00:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:00:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:00:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24076.061619 secs 2023-09-10T06:00:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24075.407884 secs 2023-09-10T06:00:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:44 smithi170 ceph-mon[102832]: pgmap v13000: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 437 KiB/s wr, 12 op/s 2023-09-10T06:00:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:00:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:00:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24076.061619 secs 2023-09-10T06:00:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24075.407884 secs 2023-09-10T06:00:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:44 smithi082 ceph-mon[98694]: pgmap v13000: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 437 KiB/s wr, 12 op/s 2023-09-10T06:00:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:00:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:00:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24076.061619 secs 2023-09-10T06:00:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:46 smithi098 ceph-mon[103124]: pgmap v13001: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 682 KiB/s wr, 19 op/s 2023-09-10T06:00:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:00:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:00:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:46 smithi170 ceph-mon[102832]: pgmap v13001: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 682 KiB/s wr, 19 op/s 2023-09-10T06:00:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:00:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:00:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:46 smithi082 ceph-mon[98694]: pgmap v13001: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 682 KiB/s wr, 19 op/s 2023-09-10T06:00:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:00:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:00:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:48 smithi170 ceph-mon[102832]: pgmap v13002: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 560 KiB/s wr, 13 op/s 2023-09-10T06:00:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:48 smithi082 ceph-mon[98694]: pgmap v13002: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 560 KiB/s wr, 13 op/s 2023-09-10T06:00:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:48 smithi098 ceph-mon[103124]: pgmap v13002: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 560 KiB/s wr, 13 op/s 2023-09-10T06:00:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24080.407968 secs 2023-09-10T06:00:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24081.061683 secs 2023-09-10T06:00:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24080.407968 secs 2023-09-10T06:00:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24081.061683 secs 2023-09-10T06:00:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24080.407968 secs 2023-09-10T06:00:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24081.061683 secs 2023-09-10T06:00:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:50 smithi170 ceph-mon[102832]: pgmap v13003: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 560 KiB/s wr, 13 op/s 2023-09-10T06:00:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:50 smithi082 ceph-mon[98694]: pgmap v13003: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 560 KiB/s wr, 13 op/s 2023-09-10T06:00:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:50 smithi098 ceph-mon[103124]: pgmap v13003: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 560 KiB/s wr, 13 op/s 2023-09-10T06:00:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:52 smithi170 ceph-mon[102832]: pgmap v13004: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 670 KiB/s wr, 20 op/s 2023-09-10T06:00:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:52 smithi082 ceph-mon[98694]: pgmap v13004: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 670 KiB/s wr, 20 op/s 2023-09-10T06:00:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:52 smithi098 ceph-mon[103124]: pgmap v13004: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 670 KiB/s wr, 20 op/s 2023-09-10T06:00:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24084.438388 secs 2023-09-10T06:00:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24084.438388 secs 2023-09-10T06:00:54.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24084.438388 secs 2023-09-10T06:00:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24085.408025 secs 2023-09-10T06:00:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:54 smithi170 ceph-mon[102832]: pgmap v13005: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 355 KiB/s wr, 13 op/s 2023-09-10T06:00:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:00:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:00:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24086.061749 secs 2023-09-10T06:00:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24085.408025 secs 2023-09-10T06:00:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:54 smithi082 ceph-mon[98694]: pgmap v13005: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 355 KiB/s wr, 13 op/s 2023-09-10T06:00:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:00:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:00:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24086.061749 secs 2023-09-10T06:00:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24085.408025 secs 2023-09-10T06:00:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:54 smithi098 ceph-mon[103124]: pgmap v13005: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 355 KiB/s wr, 13 op/s 2023-09-10T06:00:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:00:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:00:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24086.061749 secs 2023-09-10T06:00:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:56 smithi170 ceph-mon[102832]: pgmap v13006: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 602 KiB/s wr, 19 op/s 2023-09-10T06:00:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:00:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:00:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:00:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:56 smithi082 ceph-mon[98694]: pgmap v13006: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 602 KiB/s wr, 19 op/s 2023-09-10T06:00:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:00:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:00:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:00:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:56 smithi098 ceph-mon[103124]: pgmap v13006: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 602 KiB/s wr, 19 op/s 2023-09-10T06:00:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:00:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:00:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:00:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:58 smithi098 ceph-mon[103124]: pgmap v13007: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 357 KiB/s wr, 12 op/s 2023-09-10T06:00:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24089.438448 secs 2023-09-10T06:00:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:58 smithi170 ceph-mon[102832]: pgmap v13007: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 357 KiB/s wr, 12 op/s 2023-09-10T06:00:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24089.438448 secs 2023-09-10T06:00:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:58 smithi082 ceph-mon[98694]: pgmap v13007: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 357 KiB/s wr, 12 op/s 2023-09-10T06:00:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24089.438448 secs 2023-09-10T06:00:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24090.408085 secs 2023-09-10T06:00:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:00:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24091.061814 secs 2023-09-10T06:00:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24090.408085 secs 2023-09-10T06:00:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:00:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24091.061814 secs 2023-09-10T06:01:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24090.408085 secs 2023-09-10T06:01:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:00:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24091.061814 secs 2023-09-10T06:01:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:00 smithi170 ceph-mon[102832]: pgmap v13008: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 357 KiB/s wr, 12 op/s 2023-09-10T06:01:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:00 smithi082 ceph-mon[98694]: pgmap v13008: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 357 KiB/s wr, 12 op/s 2023-09-10T06:01:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:00 smithi098 ceph-mon[103124]: pgmap v13008: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 357 KiB/s wr, 12 op/s 2023-09-10T06:01:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:02 smithi170 ceph-mon[102832]: pgmap v13009: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 542 KiB/s wr, 18 op/s 2023-09-10T06:01:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:02 smithi082 ceph-mon[98694]: pgmap v13009: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 542 KiB/s wr, 18 op/s 2023-09-10T06:01:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:02 smithi098 ceph-mon[103124]: pgmap v13009: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 542 KiB/s wr, 18 op/s 2023-09-10T06:01:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24094.438585 secs 2023-09-10T06:01:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24094.438585 secs 2023-09-10T06:01:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24094.438585 secs 2023-09-10T06:01:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24095.408168 secs 2023-09-10T06:01:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:04 smithi170 ceph-mon[102832]: pgmap v13010: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 432 KiB/s wr, 11 op/s 2023-09-10T06:01:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:01:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:01:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24096.061878 secs 2023-09-10T06:01:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24095.408168 secs 2023-09-10T06:01:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:04 smithi082 ceph-mon[98694]: pgmap v13010: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 432 KiB/s wr, 11 op/s 2023-09-10T06:01:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:01:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:01:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24096.061878 secs 2023-09-10T06:01:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24095.408168 secs 2023-09-10T06:01:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:04 smithi098 ceph-mon[103124]: pgmap v13010: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 432 KiB/s wr, 11 op/s 2023-09-10T06:01:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:01:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:01:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24096.061878 secs 2023-09-10T06:01:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:06 smithi170 ceph-mon[102832]: pgmap v13011: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 558 KiB/s wr, 31 op/s 2023-09-10T06:01:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:01:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:01:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:06 smithi082 ceph-mon[98694]: pgmap v13011: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 558 KiB/s wr, 31 op/s 2023-09-10T06:01:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:01:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:01:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:06 smithi098 ceph-mon[103124]: pgmap v13011: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 558 KiB/s wr, 31 op/s 2023-09-10T06:01:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:01:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:01:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:08 smithi170 ceph-mon[102832]: pgmap v13012: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 310 KiB/s wr, 25 op/s 2023-09-10T06:01:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24099.438644 secs 2023-09-10T06:01:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:08 smithi082 ceph-mon[98694]: pgmap v13012: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 310 KiB/s wr, 25 op/s 2023-09-10T06:01:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24099.438644 secs 2023-09-10T06:01:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:08 smithi098 ceph-mon[103124]: pgmap v13012: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 310 KiB/s wr, 25 op/s 2023-09-10T06:01:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24099.438644 secs 2023-09-10T06:01:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24100.408225 secs 2023-09-10T06:01:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:09 smithi170 ceph-mon[102832]: osdmap e881: 12 total, 12 up, 12 in 2023-09-10T06:01:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24101.061944 secs 2023-09-10T06:01:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24100.408225 secs 2023-09-10T06:01:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:09 smithi082 ceph-mon[98694]: osdmap e881: 12 total, 12 up, 12 in 2023-09-10T06:01:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24101.061944 secs 2023-09-10T06:01:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24100.408225 secs 2023-09-10T06:01:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:09 smithi098 ceph-mon[103124]: osdmap e881: 12 total, 12 up, 12 in 2023-09-10T06:01:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24101.061944 secs 2023-09-10T06:01:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:10 smithi170 ceph-mon[102832]: pgmap v13014: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 373 KiB/s wr, 31 op/s 2023-09-10T06:01:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:10 smithi082 ceph-mon[98694]: pgmap v13014: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 373 KiB/s wr, 31 op/s 2023-09-10T06:01:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:10 smithi098 ceph-mon[103124]: pgmap v13014: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 373 KiB/s wr, 31 op/s 2023-09-10T06:01:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:12 smithi170 ceph-mon[102832]: pgmap v13015: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 152 KiB/s wr, 30 op/s 2023-09-10T06:01:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:12 smithi082 ceph-mon[98694]: pgmap v13015: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 152 KiB/s wr, 30 op/s 2023-09-10T06:01:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:12 smithi098 ceph-mon[103124]: pgmap v13015: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 152 KiB/s wr, 30 op/s 2023-09-10T06:01:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24104.438704 secs 2023-09-10T06:01:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24104.438704 secs 2023-09-10T06:01:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24104.438704 secs 2023-09-10T06:01:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24105.408285 secs 2023-09-10T06:01:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:14 smithi170 ceph-mon[102832]: pgmap v13016: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 152 KiB/s wr, 30 op/s 2023-09-10T06:01:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24106.062010 secs 2023-09-10T06:01:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24105.408285 secs 2023-09-10T06:01:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:14 smithi082 ceph-mon[98694]: pgmap v13016: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 152 KiB/s wr, 30 op/s 2023-09-10T06:01:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24106.062010 secs 2023-09-10T06:01:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24105.408285 secs 2023-09-10T06:01:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:14 smithi098 ceph-mon[103124]: pgmap v13016: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 152 KiB/s wr, 30 op/s 2023-09-10T06:01:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24106.062010 secs 2023-09-10T06:01:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:16 smithi170 ceph-mon[102832]: osdmap e882: 12 total, 12 up, 12 in 2023-09-10T06:01:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:16 smithi170 ceph-mon[102832]: pgmap v13018: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 15 op/s 2023-09-10T06:01:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:16 smithi082 ceph-mon[98694]: osdmap e882: 12 total, 12 up, 12 in 2023-09-10T06:01:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:16 smithi082 ceph-mon[98694]: pgmap v13018: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 15 op/s 2023-09-10T06:01:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:16 smithi098 ceph-mon[103124]: osdmap e882: 12 total, 12 up, 12 in 2023-09-10T06:01:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:16 smithi098 ceph-mon[103124]: pgmap v13018: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 15 op/s 2023-09-10T06:01:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:18 smithi170 ceph-mon[102832]: pgmap v13019: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 14 op/s 2023-09-10T06:01:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24109.438770 secs 2023-09-10T06:01:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:18 smithi082 ceph-mon[98694]: pgmap v13019: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 14 op/s 2023-09-10T06:01:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24109.438770 secs 2023-09-10T06:01:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:18 smithi098 ceph-mon[103124]: pgmap v13019: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 14 op/s 2023-09-10T06:01:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24109.438770 secs 2023-09-10T06:01:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24110.408343 secs 2023-09-10T06:01:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24111.062076 secs 2023-09-10T06:01:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24110.408343 secs 2023-09-10T06:01:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24111.062076 secs 2023-09-10T06:01:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24110.408343 secs 2023-09-10T06:01:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24111.062076 secs 2023-09-10T06:01:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:20 smithi170 ceph-mon[102832]: pgmap v13020: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T06:01:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:20 smithi082 ceph-mon[98694]: pgmap v13020: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T06:01:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:20 smithi098 ceph-mon[103124]: pgmap v13020: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T06:01:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:22 smithi170 ceph-mon[102832]: pgmap v13021: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T06:01:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:01:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:22 smithi082 ceph-mon[98694]: pgmap v13021: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T06:01:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:01:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:22 smithi098 ceph-mon[103124]: pgmap v13021: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T06:01:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:01:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:01:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:01:23.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:01:23.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24114.438871 secs 2023-09-10T06:01:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:01:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:01:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:01:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24114.438871 secs 2023-09-10T06:01:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:01:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:01:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:01:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24114.438871 secs 2023-09-10T06:01:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24115.408404 secs 2023-09-10T06:01:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:24 smithi170 ceph-mon[102832]: pgmap v13022: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T06:01:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24116.062141 secs 2023-09-10T06:01:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24115.408404 secs 2023-09-10T06:01:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:24 smithi082 ceph-mon[98694]: pgmap v13022: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T06:01:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24116.062141 secs 2023-09-10T06:01:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24115.408404 secs 2023-09-10T06:01:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:24 smithi098 ceph-mon[103124]: pgmap v13022: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T06:01:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24116.062141 secs 2023-09-10T06:01:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:26 smithi170 ceph-mon[102832]: pgmap v13023: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 11 op/s 2023-09-10T06:01:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:26 smithi082 ceph-mon[98694]: pgmap v13023: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 11 op/s 2023-09-10T06:01:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:26 smithi098 ceph-mon[103124]: pgmap v13023: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 11 op/s 2023-09-10T06:01:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:28 smithi170 ceph-mon[102832]: pgmap v13024: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T06:01:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:28 smithi082 ceph-mon[98694]: pgmap v13024: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T06:01:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:28 smithi098 ceph-mon[103124]: pgmap v13024: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T06:01:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24120.408463 secs 2023-09-10T06:01:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24121.062203 secs 2023-09-10T06:01:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24120.408463 secs 2023-09-10T06:01:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24121.062203 secs 2023-09-10T06:01:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24120.408463 secs 2023-09-10T06:01:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24121.062203 secs 2023-09-10T06:01:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:30 smithi170 ceph-mon[102832]: pgmap v13025: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T06:01:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:30 smithi082 ceph-mon[98694]: pgmap v13025: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T06:01:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:30 smithi098 ceph-mon[103124]: pgmap v13025: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T06:01:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:01:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:01:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:01:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:32 smithi170 ceph-mon[102832]: pgmap v13026: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 207 KiB/s wr, 14 op/s 2023-09-10T06:01:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:01:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:01:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:01:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:32 smithi082 ceph-mon[98694]: pgmap v13026: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 207 KiB/s wr, 14 op/s 2023-09-10T06:01:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:01:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:01:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:01:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:32 smithi098 ceph-mon[103124]: pgmap v13026: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 207 KiB/s wr, 14 op/s 2023-09-10T06:01:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:01:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:01:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:01:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24125.408529 secs 2023-09-10T06:01:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:34 smithi170 ceph-mon[102832]: pgmap v13027: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 126 KiB/s wr, 9 op/s 2023-09-10T06:01:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:01:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:01:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24126.062269 secs 2023-09-10T06:01:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24125.408529 secs 2023-09-10T06:01:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:34 smithi082 ceph-mon[98694]: pgmap v13027: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 126 KiB/s wr, 9 op/s 2023-09-10T06:01:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:01:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:01:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24126.062269 secs 2023-09-10T06:01:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24125.408529 secs 2023-09-10T06:01:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:34 smithi098 ceph-mon[103124]: pgmap v13027: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 126 KiB/s wr, 9 op/s 2023-09-10T06:01:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:01:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:01:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24126.062269 secs 2023-09-10T06:01:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:36 smithi170 ceph-mon[102832]: pgmap v13028: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 356 KiB/s wr, 16 op/s 2023-09-10T06:01:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:01:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:01:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:36 smithi082 ceph-mon[98694]: pgmap v13028: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 356 KiB/s wr, 16 op/s 2023-09-10T06:01:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:01:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:01:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:36 smithi098 ceph-mon[103124]: pgmap v13028: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 356 KiB/s wr, 16 op/s 2023-09-10T06:01:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:01:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:01:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:38 smithi170 ceph-mon[102832]: pgmap v13029: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 356 KiB/s wr, 12 op/s 2023-09-10T06:01:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:38 smithi082 ceph-mon[98694]: pgmap v13029: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 356 KiB/s wr, 12 op/s 2023-09-10T06:01:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:38 smithi098 ceph-mon[103124]: pgmap v13029: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 356 KiB/s wr, 12 op/s 2023-09-10T06:01:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24130.408590 secs 2023-09-10T06:01:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24131.062335 secs 2023-09-10T06:01:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24130.408590 secs 2023-09-10T06:01:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24131.062335 secs 2023-09-10T06:01:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24130.408590 secs 2023-09-10T06:01:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24131.062335 secs 2023-09-10T06:01:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:40 smithi170 ceph-mon[102832]: pgmap v13030: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 356 KiB/s wr, 12 op/s 2023-09-10T06:01:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:40 smithi082 ceph-mon[98694]: pgmap v13030: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 356 KiB/s wr, 12 op/s 2023-09-10T06:01:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:40 smithi098 ceph-mon[103124]: pgmap v13030: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 356 KiB/s wr, 12 op/s 2023-09-10T06:01:42.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:42 smithi170 ceph-mon[102832]: pgmap v13031: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 372 KiB/s wr, 17 op/s 2023-09-10T06:01:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:42 smithi082 ceph-mon[98694]: pgmap v13031: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 372 KiB/s wr, 17 op/s 2023-09-10T06:01:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:42 smithi098 ceph-mon[103124]: pgmap v13031: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 372 KiB/s wr, 17 op/s 2023-09-10T06:01:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24135.408642 secs 2023-09-10T06:01:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:44 smithi170 ceph-mon[102832]: pgmap v13032: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 247 KiB/s wr, 12 op/s 2023-09-10T06:01:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:01:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:01:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24136.062399 secs 2023-09-10T06:01:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24135.408642 secs 2023-09-10T06:01:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:44 smithi082 ceph-mon[98694]: pgmap v13032: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 247 KiB/s wr, 12 op/s 2023-09-10T06:01:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:01:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:01:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24136.062399 secs 2023-09-10T06:01:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24135.408642 secs 2023-09-10T06:01:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:44 smithi098 ceph-mon[103124]: pgmap v13032: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 247 KiB/s wr, 12 op/s 2023-09-10T06:01:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:01:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:01:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24136.062399 secs 2023-09-10T06:01:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:46 smithi170 ceph-mon[102832]: pgmap v13033: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 685 KiB/s wr, 18 op/s 2023-09-10T06:01:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:01:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:01:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:46 smithi082 ceph-mon[98694]: pgmap v13033: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 685 KiB/s wr, 18 op/s 2023-09-10T06:01:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:01:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:01:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:46 smithi098 ceph-mon[103124]: pgmap v13033: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 685 KiB/s wr, 18 op/s 2023-09-10T06:01:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:01:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:01:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:48 smithi170 ceph-mon[102832]: pgmap v13034: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 455 KiB/s wr, 11 op/s 2023-09-10T06:01:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:48 smithi082 ceph-mon[98694]: pgmap v13034: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 455 KiB/s wr, 11 op/s 2023-09-10T06:01:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:48 smithi098 ceph-mon[103124]: pgmap v13034: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 455 KiB/s wr, 11 op/s 2023-09-10T06:01:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24140.408705 secs 2023-09-10T06:01:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24141.062462 secs 2023-09-10T06:01:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24140.408705 secs 2023-09-10T06:01:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24141.062462 secs 2023-09-10T06:01:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24140.408705 secs 2023-09-10T06:01:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24141.062462 secs 2023-09-10T06:01:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:50 smithi170 ceph-mon[102832]: pgmap v13035: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 455 KiB/s wr, 11 op/s 2023-09-10T06:01:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:50 smithi082 ceph-mon[98694]: pgmap v13035: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 455 KiB/s wr, 11 op/s 2023-09-10T06:01:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:50 smithi098 ceph-mon[103124]: pgmap v13035: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 455 KiB/s wr, 11 op/s 2023-09-10T06:01:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:52 smithi170 ceph-mon[102832]: pgmap v13036: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 528 KiB/s wr, 17 op/s 2023-09-10T06:01:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:52 smithi082 ceph-mon[98694]: pgmap v13036: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 528 KiB/s wr, 17 op/s 2023-09-10T06:01:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:52 smithi098 ceph-mon[103124]: pgmap v13036: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 528 KiB/s wr, 17 op/s 2023-09-10T06:01:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24145.408761 secs 2023-09-10T06:01:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:54 smithi170 ceph-mon[102832]: pgmap v13037: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 511 KiB/s wr, 12 op/s 2023-09-10T06:01:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24146.062491 secs 2023-09-10T06:01:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24145.408761 secs 2023-09-10T06:01:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:54 smithi082 ceph-mon[98694]: pgmap v13037: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 511 KiB/s wr, 12 op/s 2023-09-10T06:01:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24146.062491 secs 2023-09-10T06:01:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24145.408761 secs 2023-09-10T06:01:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:54 smithi098 ceph-mon[103124]: pgmap v13037: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 511 KiB/s wr, 12 op/s 2023-09-10T06:01:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24146.062491 secs 2023-09-10T06:01:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:56 smithi170 ceph-mon[102832]: pgmap v13038: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 632 KiB/s wr, 17 op/s 2023-09-10T06:01:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:56 smithi082 ceph-mon[98694]: pgmap v13038: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 632 KiB/s wr, 17 op/s 2023-09-10T06:01:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:56 smithi098 ceph-mon[103124]: pgmap v13038: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 632 KiB/s wr, 17 op/s 2023-09-10T06:01:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:01:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:01:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:58 smithi170 ceph-mon[102832]: pgmap v13039: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T06:01:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24149.439425 secs 2023-09-10T06:01:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:01:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:01:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:58 smithi082 ceph-mon[98694]: pgmap v13039: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T06:01:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24149.439425 secs 2023-09-10T06:01:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:01:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:01:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:58 smithi098 ceph-mon[103124]: pgmap v13039: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T06:01:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24149.439425 secs 2023-09-10T06:01:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24150.408817 secs 2023-09-10T06:01:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:01:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24151.062557 secs 2023-09-10T06:01:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24150.408817 secs 2023-09-10T06:01:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:01:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24151.062557 secs 2023-09-10T06:01:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24150.408817 secs 2023-09-10T06:01:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:01:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24151.062557 secs 2023-09-10T06:02:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:02:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:02:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:02:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:00 smithi170 ceph-mon[102832]: pgmap v13040: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T06:02:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:02:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:02:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:02:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:00 smithi082 ceph-mon[98694]: pgmap v13040: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T06:02:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:02:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:02:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:02:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:00 smithi098 ceph-mon[103124]: pgmap v13040: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T06:02:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:02 smithi170 ceph-mon[102832]: pgmap v13041: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 367 KiB/s wr, 16 op/s 2023-09-10T06:02:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:02 smithi082 ceph-mon[98694]: pgmap v13041: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 367 KiB/s wr, 16 op/s 2023-09-10T06:02:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:02 smithi098 ceph-mon[103124]: pgmap v13041: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 367 KiB/s wr, 16 op/s 2023-09-10T06:02:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24154.439492 secs 2023-09-10T06:02:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24154.439492 secs 2023-09-10T06:02:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24154.439492 secs 2023-09-10T06:02:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24155.408877 secs 2023-09-10T06:02:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:04 smithi170 ceph-mon[102832]: pgmap v13042: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 294 KiB/s wr, 11 op/s 2023-09-10T06:02:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24156.062621 secs 2023-09-10T06:02:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24155.408877 secs 2023-09-10T06:02:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:04 smithi082 ceph-mon[98694]: pgmap v13042: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 294 KiB/s wr, 11 op/s 2023-09-10T06:02:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24156.062621 secs 2023-09-10T06:02:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24155.408877 secs 2023-09-10T06:02:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:04 smithi098 ceph-mon[103124]: pgmap v13042: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 294 KiB/s wr, 11 op/s 2023-09-10T06:02:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24156.062621 secs 2023-09-10T06:02:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:06 smithi170 ceph-mon[102832]: pgmap v13043: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 421 KiB/s wr, 31 op/s 2023-09-10T06:02:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:06 smithi082 ceph-mon[98694]: pgmap v13043: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 421 KiB/s wr, 31 op/s 2023-09-10T06:02:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:06 smithi098 ceph-mon[103124]: pgmap v13043: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 421 KiB/s wr, 31 op/s 2023-09-10T06:02:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:08 smithi170 ceph-mon[102832]: pgmap v13044: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 300 KiB/s wr, 25 op/s 2023-09-10T06:02:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24159.439551 secs 2023-09-10T06:02:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:08 smithi082 ceph-mon[98694]: pgmap v13044: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 300 KiB/s wr, 25 op/s 2023-09-10T06:02:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24159.439551 secs 2023-09-10T06:02:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:08 smithi098 ceph-mon[103124]: pgmap v13044: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 300 KiB/s wr, 25 op/s 2023-09-10T06:02:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24159.439551 secs 2023-09-10T06:02:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24160.408934 secs 2023-09-10T06:02:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:09 smithi170 ceph-mon[102832]: osdmap e883: 12 total, 12 up, 12 in 2023-09-10T06:02:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24161.062688 secs 2023-09-10T06:02:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24160.408934 secs 2023-09-10T06:02:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:09 smithi082 ceph-mon[98694]: osdmap e883: 12 total, 12 up, 12 in 2023-09-10T06:02:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24161.062688 secs 2023-09-10T06:02:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24160.408934 secs 2023-09-10T06:02:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:09 smithi098 ceph-mon[103124]: osdmap e883: 12 total, 12 up, 12 in 2023-09-10T06:02:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24161.062688 secs 2023-09-10T06:02:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:10 smithi170 ceph-mon[102832]: pgmap v13046: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 360 KiB/s wr, 30 op/s 2023-09-10T06:02:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:10 smithi082 ceph-mon[98694]: pgmap v13046: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 360 KiB/s wr, 30 op/s 2023-09-10T06:02:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:10 smithi098 ceph-mon[103124]: pgmap v13046: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 360 KiB/s wr, 30 op/s 2023-09-10T06:02:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:12 smithi170 ceph-mon[102832]: pgmap v13047: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 174 KiB/s wr, 30 op/s 2023-09-10T06:02:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:12 smithi082 ceph-mon[98694]: pgmap v13047: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 174 KiB/s wr, 30 op/s 2023-09-10T06:02:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:12 smithi098 ceph-mon[103124]: pgmap v13047: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 174 KiB/s wr, 30 op/s 2023-09-10T06:02:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24164.439615 secs 2023-09-10T06:02:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24164.439615 secs 2023-09-10T06:02:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24164.439615 secs 2023-09-10T06:02:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24165.408992 secs 2023-09-10T06:02:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:14 smithi170 ceph-mon[102832]: pgmap v13048: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 174 KiB/s wr, 30 op/s 2023-09-10T06:02:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24166.062752 secs 2023-09-10T06:02:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24165.408992 secs 2023-09-10T06:02:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:14 smithi082 ceph-mon[98694]: pgmap v13048: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 174 KiB/s wr, 30 op/s 2023-09-10T06:02:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24166.062752 secs 2023-09-10T06:02:15.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24165.408992 secs 2023-09-10T06:02:15.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:14 smithi098 ceph-mon[103124]: pgmap v13048: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 174 KiB/s wr, 30 op/s 2023-09-10T06:02:15.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24166.062752 secs 2023-09-10T06:02:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:16 smithi170 ceph-mon[102832]: osdmap e884: 12 total, 12 up, 12 in 2023-09-10T06:02:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:16 smithi170 ceph-mon[102832]: pgmap v13050: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 14 op/s 2023-09-10T06:02:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:16 smithi082 ceph-mon[98694]: osdmap e884: 12 total, 12 up, 12 in 2023-09-10T06:02:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:16 smithi082 ceph-mon[98694]: pgmap v13050: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 14 op/s 2023-09-10T06:02:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:16 smithi098 ceph-mon[103124]: osdmap e884: 12 total, 12 up, 12 in 2023-09-10T06:02:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:16 smithi098 ceph-mon[103124]: pgmap v13050: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 14 op/s 2023-09-10T06:02:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:18 smithi170 ceph-mon[102832]: pgmap v13051: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T06:02:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24169.439680 secs 2023-09-10T06:02:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:18 smithi082 ceph-mon[98694]: pgmap v13051: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T06:02:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24169.439680 secs 2023-09-10T06:02:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:18 smithi098 ceph-mon[103124]: pgmap v13051: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T06:02:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24169.439680 secs 2023-09-10T06:02:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24170.409048 secs 2023-09-10T06:02:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24171.062819 secs 2023-09-10T06:02:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24170.409048 secs 2023-09-10T06:02:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24171.062819 secs 2023-09-10T06:02:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24170.409048 secs 2023-09-10T06:02:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24171.062819 secs 2023-09-10T06:02:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:20 smithi170 ceph-mon[102832]: pgmap v13052: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T06:02:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:20 smithi082 ceph-mon[98694]: pgmap v13052: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T06:02:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:20 smithi098 ceph-mon[103124]: pgmap v13052: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T06:02:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:22 smithi170 ceph-mon[102832]: pgmap v13053: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T06:02:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:22 smithi082 ceph-mon[98694]: pgmap v13053: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T06:02:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:22 smithi098 ceph-mon[103124]: pgmap v13053: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T06:02:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24174.439752 secs 2023-09-10T06:02:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:02:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:02:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:02:23.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:02:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24174.439752 secs 2023-09-10T06:02:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:02:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:02:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:02:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:02:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24174.439752 secs 2023-09-10T06:02:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:02:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:02:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:02:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:02:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24175.409108 secs 2023-09-10T06:02:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:24 smithi170 ceph-mon[102832]: pgmap v13054: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T06:02:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24176.062887 secs 2023-09-10T06:02:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24175.409108 secs 2023-09-10T06:02:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:24 smithi082 ceph-mon[98694]: pgmap v13054: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T06:02:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24176.062887 secs 2023-09-10T06:02:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24175.409108 secs 2023-09-10T06:02:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:24 smithi098 ceph-mon[103124]: pgmap v13054: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T06:02:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24176.062887 secs 2023-09-10T06:02:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:26 smithi170 ceph-mon[102832]: pgmap v13055: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T06:02:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:26 smithi082 ceph-mon[98694]: pgmap v13055: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T06:02:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:26 smithi098 ceph-mon[103124]: pgmap v13055: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T06:02:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:28 smithi170 ceph-mon[102832]: pgmap v13056: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 10 op/s 2023-09-10T06:02:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:28 smithi082 ceph-mon[98694]: pgmap v13056: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 10 op/s 2023-09-10T06:02:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:28 smithi098 ceph-mon[103124]: pgmap v13056: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 10 op/s 2023-09-10T06:02:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24180.409167 secs 2023-09-10T06:02:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24181.062950 secs 2023-09-10T06:02:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24180.409167 secs 2023-09-10T06:02:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24181.062950 secs 2023-09-10T06:02:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24180.409167 secs 2023-09-10T06:02:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24181.062950 secs 2023-09-10T06:02:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:30 smithi170 ceph-mon[102832]: pgmap v13057: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 10 op/s 2023-09-10T06:02:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:02:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:02:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:30 smithi082 ceph-mon[98694]: pgmap v13057: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 10 op/s 2023-09-10T06:02:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:02:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:02:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:30 smithi098 ceph-mon[103124]: pgmap v13057: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 10 op/s 2023-09-10T06:02:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:02:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:02:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:02:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:02:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:02:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:32 smithi170 ceph-mon[102832]: pgmap v13058: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 164 KiB/s wr, 15 op/s 2023-09-10T06:02:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:02:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:02:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:02:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:32 smithi082 ceph-mon[98694]: pgmap v13058: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 164 KiB/s wr, 15 op/s 2023-09-10T06:02:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:02:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:02:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:02:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:32 smithi098 ceph-mon[103124]: pgmap v13058: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 164 KiB/s wr, 15 op/s 2023-09-10T06:02:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:02:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:02:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:02:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24185.409225 secs 2023-09-10T06:02:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:34 smithi170 ceph-mon[102832]: pgmap v13059: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 10 op/s 2023-09-10T06:02:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24186.063017 secs 2023-09-10T06:02:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24185.409225 secs 2023-09-10T06:02:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:34 smithi082 ceph-mon[98694]: pgmap v13059: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 10 op/s 2023-09-10T06:02:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24186.063017 secs 2023-09-10T06:02:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24185.409225 secs 2023-09-10T06:02:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:34 smithi098 ceph-mon[103124]: pgmap v13059: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 10 op/s 2023-09-10T06:02:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24186.063017 secs 2023-09-10T06:02:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:36 smithi170 ceph-mon[102832]: pgmap v13060: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 357 KiB/s wr, 16 op/s 2023-09-10T06:02:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:36 smithi082 ceph-mon[98694]: pgmap v13060: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 357 KiB/s wr, 16 op/s 2023-09-10T06:02:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:36 smithi098 ceph-mon[103124]: pgmap v13060: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 357 KiB/s wr, 16 op/s 2023-09-10T06:02:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:02:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:02:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:38 smithi170 ceph-mon[102832]: pgmap v13061: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 357 KiB/s wr, 11 op/s 2023-09-10T06:02:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:02:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:02:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:38 smithi082 ceph-mon[98694]: pgmap v13061: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 357 KiB/s wr, 11 op/s 2023-09-10T06:02:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:02:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:02:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:38 smithi098 ceph-mon[103124]: pgmap v13061: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 357 KiB/s wr, 11 op/s 2023-09-10T06:02:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24190.409310 secs 2023-09-10T06:02:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24191.063081 secs 2023-09-10T06:02:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24190.409310 secs 2023-09-10T06:02:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24191.063081 secs 2023-09-10T06:02:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24190.409310 secs 2023-09-10T06:02:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24191.063081 secs 2023-09-10T06:02:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:02:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:02:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:40 smithi170 ceph-mon[102832]: pgmap v13062: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 357 KiB/s wr, 11 op/s 2023-09-10T06:02:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:02:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:02:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:40 smithi082 ceph-mon[98694]: pgmap v13062: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 357 KiB/s wr, 11 op/s 2023-09-10T06:02:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:02:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:02:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:40 smithi098 ceph-mon[103124]: pgmap v13062: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 357 KiB/s wr, 11 op/s 2023-09-10T06:02:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:42 smithi170 ceph-mon[102832]: pgmap v13063: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 374 KiB/s wr, 17 op/s 2023-09-10T06:02:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:42 smithi082 ceph-mon[98694]: pgmap v13063: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 374 KiB/s wr, 17 op/s 2023-09-10T06:02:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:42 smithi098 ceph-mon[103124]: pgmap v13063: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 374 KiB/s wr, 17 op/s 2023-09-10T06:02:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24195.409398 secs 2023-09-10T06:02:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:44 smithi170 ceph-mon[102832]: pgmap v13064: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 292 KiB/s wr, 12 op/s 2023-09-10T06:02:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:02:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:02:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24196.063148 secs 2023-09-10T06:02:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24195.409398 secs 2023-09-10T06:02:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:44 smithi082 ceph-mon[98694]: pgmap v13064: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 292 KiB/s wr, 12 op/s 2023-09-10T06:02:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:02:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:02:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24196.063148 secs 2023-09-10T06:02:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24195.409398 secs 2023-09-10T06:02:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:44 smithi098 ceph-mon[103124]: pgmap v13064: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 292 KiB/s wr, 12 op/s 2023-09-10T06:02:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:02:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:02:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24196.063148 secs 2023-09-10T06:02:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:46 smithi170 ceph-mon[102832]: pgmap v13065: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 732 KiB/s wr, 19 op/s 2023-09-10T06:02:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:02:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:02:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:46 smithi082 ceph-mon[98694]: pgmap v13065: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 732 KiB/s wr, 19 op/s 2023-09-10T06:02:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:02:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:02:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:46 smithi098 ceph-mon[103124]: pgmap v13065: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 732 KiB/s wr, 19 op/s 2023-09-10T06:02:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:02:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:02:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:48 smithi170 ceph-mon[102832]: pgmap v13066: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 456 KiB/s wr, 12 op/s 2023-09-10T06:02:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:48 smithi082 ceph-mon[98694]: pgmap v13066: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 456 KiB/s wr, 12 op/s 2023-09-10T06:02:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:48 smithi098 ceph-mon[103124]: pgmap v13066: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 456 KiB/s wr, 12 op/s 2023-09-10T06:02:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24200.409457 secs 2023-09-10T06:02:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24201.063209 secs 2023-09-10T06:02:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24200.409457 secs 2023-09-10T06:02:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24201.063209 secs 2023-09-10T06:02:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24200.409457 secs 2023-09-10T06:02:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24201.063209 secs 2023-09-10T06:02:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:50 smithi170 ceph-mon[102832]: pgmap v13067: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 456 KiB/s wr, 12 op/s 2023-09-10T06:02:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:50 smithi082 ceph-mon[98694]: pgmap v13067: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 456 KiB/s wr, 12 op/s 2023-09-10T06:02:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:50 smithi098 ceph-mon[103124]: pgmap v13067: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 456 KiB/s wr, 12 op/s 2023-09-10T06:02:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:52 smithi170 ceph-mon[102832]: pgmap v13068: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 504 KiB/s wr, 18 op/s 2023-09-10T06:02:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:52 smithi082 ceph-mon[98694]: pgmap v13068: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 504 KiB/s wr, 18 op/s 2023-09-10T06:02:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:52 smithi098 ceph-mon[103124]: pgmap v13068: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 504 KiB/s wr, 18 op/s 2023-09-10T06:02:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24205.409496 secs 2023-09-10T06:02:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:54 smithi170 ceph-mon[102832]: pgmap v13069: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 487 KiB/s wr, 12 op/s 2023-09-10T06:02:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24206.063273 secs 2023-09-10T06:02:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24205.409496 secs 2023-09-10T06:02:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:54 smithi082 ceph-mon[98694]: pgmap v13069: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 487 KiB/s wr, 12 op/s 2023-09-10T06:02:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24206.063273 secs 2023-09-10T06:02:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24205.409496 secs 2023-09-10T06:02:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:54 smithi098 ceph-mon[103124]: pgmap v13069: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 487 KiB/s wr, 12 op/s 2023-09-10T06:02:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24206.063273 secs 2023-09-10T06:02:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:56 smithi170 ceph-mon[102832]: pgmap v13070: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 634 KiB/s wr, 18 op/s 2023-09-10T06:02:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:56 smithi082 ceph-mon[98694]: pgmap v13070: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 634 KiB/s wr, 18 op/s 2023-09-10T06:02:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:56 smithi098 ceph-mon[103124]: pgmap v13070: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 634 KiB/s wr, 18 op/s 2023-09-10T06:02:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:02:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:02:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:58 smithi170 ceph-mon[102832]: pgmap v13071: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T06:02:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24209.440221 secs 2023-09-10T06:02:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:02:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:02:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:58 smithi082 ceph-mon[98694]: pgmap v13071: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T06:02:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24209.440221 secs 2023-09-10T06:02:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:02:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:02:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:58 smithi098 ceph-mon[103124]: pgmap v13071: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T06:02:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24209.440221 secs 2023-09-10T06:02:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24210.409552 secs 2023-09-10T06:02:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:02:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24211.063340 secs 2023-09-10T06:02:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24210.409552 secs 2023-09-10T06:02:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:02:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24211.063340 secs 2023-09-10T06:02:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24210.409552 secs 2023-09-10T06:02:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:02:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24211.063340 secs 2023-09-10T06:03:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:03:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:03:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:03:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:00 smithi170 ceph-mon[102832]: pgmap v13072: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T06:03:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:03:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:03:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:03:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:00 smithi082 ceph-mon[98694]: pgmap v13072: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T06:03:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:03:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:03:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:03:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:00 smithi098 ceph-mon[103124]: pgmap v13072: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T06:03:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:02 smithi170 ceph-mon[102832]: pgmap v13073: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 369 KiB/s wr, 17 op/s 2023-09-10T06:03:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:02 smithi082 ceph-mon[98694]: pgmap v13073: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 369 KiB/s wr, 17 op/s 2023-09-10T06:03:03.010 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:02 smithi098 ceph-mon[103124]: pgmap v13073: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 369 KiB/s wr, 17 op/s 2023-09-10T06:03:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24214.440311 secs 2023-09-10T06:03:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24214.440311 secs 2023-09-10T06:03:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24214.440311 secs 2023-09-10T06:03:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24215.409610 secs 2023-09-10T06:03:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:04 smithi170 ceph-mon[102832]: pgmap v13074: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 322 KiB/s wr, 11 op/s 2023-09-10T06:03:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24216.063402 secs 2023-09-10T06:03:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24215.409610 secs 2023-09-10T06:03:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:04 smithi082 ceph-mon[98694]: pgmap v13074: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 322 KiB/s wr, 11 op/s 2023-09-10T06:03:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24216.063402 secs 2023-09-10T06:03:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24215.409610 secs 2023-09-10T06:03:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:04 smithi098 ceph-mon[103124]: pgmap v13074: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 322 KiB/s wr, 11 op/s 2023-09-10T06:03:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24216.063402 secs 2023-09-10T06:03:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:06 smithi170 ceph-mon[102832]: pgmap v13075: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 448 KiB/s wr, 31 op/s 2023-09-10T06:03:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:06 smithi082 ceph-mon[98694]: pgmap v13075: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 448 KiB/s wr, 31 op/s 2023-09-10T06:03:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:06 smithi098 ceph-mon[103124]: pgmap v13075: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 448 KiB/s wr, 31 op/s 2023-09-10T06:03:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:08 smithi170 ceph-mon[102832]: pgmap v13076: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 301 KiB/s wr, 26 op/s 2023-09-10T06:03:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24219.440373 secs 2023-09-10T06:03:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:08 smithi082 ceph-mon[98694]: pgmap v13076: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 301 KiB/s wr, 26 op/s 2023-09-10T06:03:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24219.440373 secs 2023-09-10T06:03:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:08 smithi098 ceph-mon[103124]: pgmap v13076: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 301 KiB/s wr, 26 op/s 2023-09-10T06:03:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24219.440373 secs 2023-09-10T06:03:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24220.409671 secs 2023-09-10T06:03:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:09 smithi170 ceph-mon[102832]: osdmap e885: 12 total, 12 up, 12 in 2023-09-10T06:03:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24221.063477 secs 2023-09-10T06:03:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24220.409671 secs 2023-09-10T06:03:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:09 smithi082 ceph-mon[98694]: osdmap e885: 12 total, 12 up, 12 in 2023-09-10T06:03:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24221.063477 secs 2023-09-10T06:03:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24220.409671 secs 2023-09-10T06:03:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:09 smithi098 ceph-mon[103124]: osdmap e885: 12 total, 12 up, 12 in 2023-09-10T06:03:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24221.063477 secs 2023-09-10T06:03:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:10 smithi170 ceph-mon[102832]: pgmap v13078: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 361 KiB/s wr, 31 op/s 2023-09-10T06:03:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:10 smithi082 ceph-mon[98694]: pgmap v13078: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 361 KiB/s wr, 31 op/s 2023-09-10T06:03:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:10 smithi098 ceph-mon[103124]: pgmap v13078: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 361 KiB/s wr, 31 op/s 2023-09-10T06:03:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:12 smithi170 ceph-mon[102832]: pgmap v13079: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 174 KiB/s wr, 30 op/s 2023-09-10T06:03:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:12 smithi082 ceph-mon[98694]: pgmap v13079: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 174 KiB/s wr, 30 op/s 2023-09-10T06:03:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:12 smithi098 ceph-mon[103124]: pgmap v13079: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 174 KiB/s wr, 30 op/s 2023-09-10T06:03:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24224.440440 secs 2023-09-10T06:03:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24224.440440 secs 2023-09-10T06:03:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24224.440440 secs 2023-09-10T06:03:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24225.409732 secs 2023-09-10T06:03:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:14 smithi170 ceph-mon[102832]: pgmap v13080: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 174 KiB/s wr, 30 op/s 2023-09-10T06:03:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24226.063487 secs 2023-09-10T06:03:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24225.409732 secs 2023-09-10T06:03:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:14 smithi082 ceph-mon[98694]: pgmap v13080: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 174 KiB/s wr, 30 op/s 2023-09-10T06:03:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24226.063487 secs 2023-09-10T06:03:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24225.409732 secs 2023-09-10T06:03:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:14 smithi098 ceph-mon[103124]: pgmap v13080: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 174 KiB/s wr, 30 op/s 2023-09-10T06:03:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24226.063487 secs 2023-09-10T06:03:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:16 smithi170 ceph-mon[102832]: osdmap e886: 12 total, 12 up, 12 in 2023-09-10T06:03:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:16 smithi170 ceph-mon[102832]: pgmap v13082: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 15 op/s 2023-09-10T06:03:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:16 smithi082 ceph-mon[98694]: osdmap e886: 12 total, 12 up, 12 in 2023-09-10T06:03:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:16 smithi082 ceph-mon[98694]: pgmap v13082: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 15 op/s 2023-09-10T06:03:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:16 smithi098 ceph-mon[103124]: osdmap e886: 12 total, 12 up, 12 in 2023-09-10T06:03:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:16 smithi098 ceph-mon[103124]: pgmap v13082: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 15 op/s 2023-09-10T06:03:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:18 smithi170 ceph-mon[102832]: pgmap v13083: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 14 op/s 2023-09-10T06:03:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24229.440501 secs 2023-09-10T06:03:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:18 smithi082 ceph-mon[98694]: pgmap v13083: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 14 op/s 2023-09-10T06:03:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24229.440501 secs 2023-09-10T06:03:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:18 smithi098 ceph-mon[103124]: pgmap v13083: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 14 op/s 2023-09-10T06:03:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24229.440501 secs 2023-09-10T06:03:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24230.409791 secs 2023-09-10T06:03:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24231.063551 secs 2023-09-10T06:03:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24230.409791 secs 2023-09-10T06:03:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24231.063551 secs 2023-09-10T06:03:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24230.409791 secs 2023-09-10T06:03:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24231.063551 secs 2023-09-10T06:03:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:20 smithi170 ceph-mon[102832]: pgmap v13084: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T06:03:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:20 smithi082 ceph-mon[98694]: pgmap v13084: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T06:03:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:20 smithi098 ceph-mon[103124]: pgmap v13084: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T06:03:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:22 smithi170 ceph-mon[102832]: pgmap v13085: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T06:03:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:22 smithi082 ceph-mon[98694]: pgmap v13085: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T06:03:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:22 smithi098 ceph-mon[103124]: pgmap v13085: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T06:03:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24234.440563 secs 2023-09-10T06:03:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24234.440563 secs 2023-09-10T06:03:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24234.440563 secs 2023-09-10T06:03:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24235.409869 secs 2023-09-10T06:03:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:24 smithi170 ceph-mon[102832]: pgmap v13086: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T06:03:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:03:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24236.063618 secs 2023-09-10T06:03:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24235.409869 secs 2023-09-10T06:03:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:24 smithi082 ceph-mon[98694]: pgmap v13086: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T06:03:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:03:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24236.063618 secs 2023-09-10T06:03:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24235.409869 secs 2023-09-10T06:03:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:24 smithi098 ceph-mon[103124]: pgmap v13086: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T06:03:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:03:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24236.063618 secs 2023-09-10T06:03:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:26 smithi170 ceph-mon[102832]: pgmap v13087: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 97 KiB/s wr, 11 op/s 2023-09-10T06:03:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:03:26.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:03:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:26 smithi082 ceph-mon[98694]: pgmap v13087: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 97 KiB/s wr, 11 op/s 2023-09-10T06:03:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:03:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:03:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:26 smithi098 ceph-mon[103124]: pgmap v13087: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 97 KiB/s wr, 11 op/s 2023-09-10T06:03:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:03:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:03:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:03:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:03:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:03:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:03:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:03:27.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:03:27.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:03:27.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:03:27.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:03:27.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:03:27.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:03:27.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:03:27.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:03:27.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:03:27.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:03:27.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:03:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:03:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:03:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:03:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:03:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:03:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:03:27.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:03:27.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:03:27.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:03:27.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:03:27.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:03:27.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:03:27.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:03:27.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:03:27.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:03:27.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:03:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:03:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:03:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:03:27.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:03:27.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:03:27.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:03:27.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:03:27.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:03:27.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:03:27.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:03:27.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:03:27.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:03:27.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:03:27.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:03:27.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:03:27.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:03:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:28 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T06:03:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:28 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T06:03:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:28 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T06:03:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:28 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T06:03:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:28 smithi098 ceph-mon[103124]: pgmap v13088: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T06:03:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:28 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T06:03:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:28 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T06:03:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:28 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T06:03:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:28 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T06:03:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:28 smithi170 ceph-mon[102832]: pgmap v13088: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T06:03:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:28 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T06:03:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:28 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T06:03:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:28 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T06:03:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:28 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T06:03:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:28 smithi082 ceph-mon[98694]: pgmap v13088: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T06:03:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24240.409930 secs 2023-09-10T06:03:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24241.063682 secs 2023-09-10T06:03:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24240.409930 secs 2023-09-10T06:03:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24241.063682 secs 2023-09-10T06:03:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24240.409930 secs 2023-09-10T06:03:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24241.063682 secs 2023-09-10T06:03:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:30 smithi098 ceph-mon[103124]: pgmap v13089: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T06:03:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:30 smithi170 ceph-mon[102832]: pgmap v13089: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T06:03:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:30 smithi082 ceph-mon[98694]: pgmap v13089: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T06:03:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:03:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:03:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:03:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:03:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:03:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:03:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:32 smithi098 ceph-mon[103124]: pgmap v13090: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 164 KiB/s wr, 14 op/s 2023-09-10T06:03:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:03:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:03:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:03:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:32 smithi170 ceph-mon[102832]: pgmap v13090: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 164 KiB/s wr, 14 op/s 2023-09-10T06:03:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:03:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:03:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:03:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:32 smithi082 ceph-mon[98694]: pgmap v13090: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 164 KiB/s wr, 14 op/s 2023-09-10T06:03:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24245.409988 secs 2023-09-10T06:03:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:03:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:03:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:34 smithi098 ceph-mon[103124]: pgmap v13091: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T06:03:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24246.063745 secs 2023-09-10T06:03:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24245.409988 secs 2023-09-10T06:03:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:03:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:03:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:34 smithi170 ceph-mon[102832]: pgmap v13091: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T06:03:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24246.063745 secs 2023-09-10T06:03:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24245.409988 secs 2023-09-10T06:03:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:03:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:03:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:34 smithi082 ceph-mon[98694]: pgmap v13091: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T06:03:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24246.063745 secs 2023-09-10T06:03:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:36 smithi170 ceph-mon[102832]: pgmap v13092: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 359 KiB/s wr, 16 op/s 2023-09-10T06:03:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:36 smithi082 ceph-mon[98694]: pgmap v13092: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 359 KiB/s wr, 16 op/s 2023-09-10T06:03:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:36 smithi098 ceph-mon[103124]: pgmap v13092: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 359 KiB/s wr, 16 op/s 2023-09-10T06:03:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:38 smithi170 ceph-mon[102832]: pgmap v13093: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 358 KiB/s wr, 12 op/s 2023-09-10T06:03:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:38 smithi082 ceph-mon[98694]: pgmap v13093: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 358 KiB/s wr, 12 op/s 2023-09-10T06:03:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:38 smithi098 ceph-mon[103124]: pgmap v13093: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 358 KiB/s wr, 12 op/s 2023-09-10T06:03:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24250.410047 secs 2023-09-10T06:03:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24251.063810 secs 2023-09-10T06:03:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24250.410047 secs 2023-09-10T06:03:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24251.063810 secs 2023-09-10T06:03:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24250.410047 secs 2023-09-10T06:03:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24251.063810 secs 2023-09-10T06:03:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:40 smithi170 ceph-mon[102832]: pgmap v13094: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 358 KiB/s wr, 12 op/s 2023-09-10T06:03:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:40 smithi082 ceph-mon[98694]: pgmap v13094: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 358 KiB/s wr, 12 op/s 2023-09-10T06:03:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:40 smithi098 ceph-mon[103124]: pgmap v13094: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 358 KiB/s wr, 12 op/s 2023-09-10T06:03:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:42 smithi170 ceph-mon[102832]: pgmap v13095: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 433 KiB/s wr, 18 op/s 2023-09-10T06:03:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:42 smithi082 ceph-mon[98694]: pgmap v13095: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 433 KiB/s wr, 18 op/s 2023-09-10T06:03:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:42 smithi098 ceph-mon[103124]: pgmap v13095: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 433 KiB/s wr, 18 op/s 2023-09-10T06:03:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24255.410106 secs 2023-09-10T06:03:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:44 smithi170 ceph-mon[102832]: pgmap v13096: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 350 KiB/s wr, 12 op/s 2023-09-10T06:03:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:03:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:03:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24256.063874 secs 2023-09-10T06:03:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24255.410106 secs 2023-09-10T06:03:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:44 smithi082 ceph-mon[98694]: pgmap v13096: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 350 KiB/s wr, 12 op/s 2023-09-10T06:03:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:03:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:03:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24256.063874 secs 2023-09-10T06:03:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24255.410106 secs 2023-09-10T06:03:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:44 smithi098 ceph-mon[103124]: pgmap v13096: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 350 KiB/s wr, 12 op/s 2023-09-10T06:03:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:03:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:03:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24256.063874 secs 2023-09-10T06:03:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:46 smithi170 ceph-mon[102832]: pgmap v13097: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 734 KiB/s wr, 20 op/s 2023-09-10T06:03:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:03:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:03:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:46 smithi082 ceph-mon[98694]: pgmap v13097: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 734 KiB/s wr, 20 op/s 2023-09-10T06:03:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:03:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:03:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:46 smithi098 ceph-mon[103124]: pgmap v13097: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 734 KiB/s wr, 20 op/s 2023-09-10T06:03:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:03:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:03:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:48 smithi170 ceph-mon[102832]: pgmap v13098: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 458 KiB/s wr, 13 op/s 2023-09-10T06:03:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:48 smithi082 ceph-mon[98694]: pgmap v13098: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 458 KiB/s wr, 13 op/s 2023-09-10T06:03:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:48 smithi098 ceph-mon[103124]: pgmap v13098: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 458 KiB/s wr, 13 op/s 2023-09-10T06:03:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24260.410169 secs 2023-09-10T06:03:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24261.063939 secs 2023-09-10T06:03:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24260.410169 secs 2023-09-10T06:03:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24261.063939 secs 2023-09-10T06:03:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24260.410169 secs 2023-09-10T06:03:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24261.063939 secs 2023-09-10T06:03:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:50 smithi170 ceph-mon[102832]: pgmap v13099: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 458 KiB/s wr, 13 op/s 2023-09-10T06:03:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:50 smithi082 ceph-mon[98694]: pgmap v13099: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 458 KiB/s wr, 13 op/s 2023-09-10T06:03:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:50 smithi098 ceph-mon[103124]: pgmap v13099: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 458 KiB/s wr, 13 op/s 2023-09-10T06:03:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:52 smithi170 ceph-mon[102832]: pgmap v13100: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 505 KiB/s wr, 18 op/s 2023-09-10T06:03:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:03:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:03:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:52 smithi082 ceph-mon[98694]: pgmap v13100: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 505 KiB/s wr, 18 op/s 2023-09-10T06:03:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:03:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:03:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:52 smithi098 ceph-mon[103124]: pgmap v13100: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 505 KiB/s wr, 18 op/s 2023-09-10T06:03:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:03:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:03:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:53 smithi170 ceph-mon[102832]: pgmap v13101: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 431 KiB/s wr, 12 op/s 2023-09-10T06:03:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:03:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:03:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:53 smithi082 ceph-mon[98694]: pgmap v13101: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 431 KiB/s wr, 12 op/s 2023-09-10T06:03:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:03:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:03:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:53 smithi098 ceph-mon[103124]: pgmap v13101: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 431 KiB/s wr, 12 op/s 2023-09-10T06:03:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:03:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:03:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24265.410228 secs 2023-09-10T06:03:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24266.064004 secs 2023-09-10T06:03:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24265.410228 secs 2023-09-10T06:03:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24266.064004 secs 2023-09-10T06:03:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24265.410228 secs 2023-09-10T06:03:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24266.064004 secs 2023-09-10T06:03:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:56 smithi170 ceph-mon[102832]: pgmap v13102: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 578 KiB/s wr, 18 op/s 2023-09-10T06:03:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:56 smithi082 ceph-mon[98694]: pgmap v13102: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 578 KiB/s wr, 18 op/s 2023-09-10T06:03:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:56 smithi098 ceph-mon[103124]: pgmap v13102: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 578 KiB/s wr, 18 op/s 2023-09-10T06:03:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:58 smithi170 ceph-mon[102832]: pgmap v13103: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T06:03:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24269.440974 secs 2023-09-10T06:03:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:58 smithi082 ceph-mon[98694]: pgmap v13103: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T06:03:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24269.440974 secs 2023-09-10T06:03:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:58 smithi098 ceph-mon[103124]: pgmap v13103: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T06:03:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24269.440974 secs 2023-09-10T06:03:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24270.410313 secs 2023-09-10T06:03:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:03:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24271.064068 secs 2023-09-10T06:03:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24270.410313 secs 2023-09-10T06:03:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:03:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24271.064068 secs 2023-09-10T06:04:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24270.410313 secs 2023-09-10T06:04:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:03:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24271.064068 secs 2023-09-10T06:04:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:00 smithi170 ceph-mon[102832]: pgmap v13104: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T06:04:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:04:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:04:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:00 smithi082 ceph-mon[98694]: pgmap v13104: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T06:04:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:04:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:04:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:00 smithi098 ceph-mon[103124]: pgmap v13104: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T06:04:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:04:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:04:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:02 smithi170 ceph-mon[102832]: pgmap v13105: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 407 KiB/s wr, 17 op/s 2023-09-10T06:04:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:02 smithi082 ceph-mon[98694]: pgmap v13105: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 407 KiB/s wr, 17 op/s 2023-09-10T06:04:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:02 smithi098 ceph-mon[103124]: pgmap v13105: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 407 KiB/s wr, 17 op/s 2023-09-10T06:04:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24274.441033 secs 2023-09-10T06:04:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24274.441033 secs 2023-09-10T06:04:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24274.441033 secs 2023-09-10T06:04:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24275.410399 secs 2023-09-10T06:04:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:04 smithi170 ceph-mon[102832]: pgmap v13106: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 360 KiB/s wr, 11 op/s 2023-09-10T06:04:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:04:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24276.064135 secs 2023-09-10T06:04:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24275.410399 secs 2023-09-10T06:04:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:04 smithi082 ceph-mon[98694]: pgmap v13106: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 360 KiB/s wr, 11 op/s 2023-09-10T06:04:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:04:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24276.064135 secs 2023-09-10T06:04:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24275.410399 secs 2023-09-10T06:04:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:04 smithi098 ceph-mon[103124]: pgmap v13106: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 360 KiB/s wr, 11 op/s 2023-09-10T06:04:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:04:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24276.064135 secs 2023-09-10T06:04:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:06 smithi170 ceph-mon[102832]: pgmap v13107: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 486 KiB/s wr, 32 op/s 2023-09-10T06:04:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:04:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:06 smithi082 ceph-mon[98694]: pgmap v13107: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 486 KiB/s wr, 32 op/s 2023-09-10T06:04:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:04:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:06 smithi098 ceph-mon[103124]: pgmap v13107: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 486 KiB/s wr, 32 op/s 2023-09-10T06:04:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:04:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:08 smithi170 ceph-mon[102832]: pgmap v13108: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 339 KiB/s wr, 26 op/s 2023-09-10T06:04:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24279.441094 secs 2023-09-10T06:04:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:08 smithi082 ceph-mon[98694]: pgmap v13108: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 339 KiB/s wr, 26 op/s 2023-09-10T06:04:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24279.441094 secs 2023-09-10T06:04:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:08 smithi098 ceph-mon[103124]: pgmap v13108: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 339 KiB/s wr, 26 op/s 2023-09-10T06:04:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24279.441094 secs 2023-09-10T06:04:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24280.410457 secs 2023-09-10T06:04:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:09 smithi170 ceph-mon[102832]: osdmap e887: 12 total, 12 up, 12 in 2023-09-10T06:04:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24281.064201 secs 2023-09-10T06:04:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24280.410457 secs 2023-09-10T06:04:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:09 smithi082 ceph-mon[98694]: osdmap e887: 12 total, 12 up, 12 in 2023-09-10T06:04:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24281.064201 secs 2023-09-10T06:04:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24280.410457 secs 2023-09-10T06:04:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:09 smithi098 ceph-mon[103124]: osdmap e887: 12 total, 12 up, 12 in 2023-09-10T06:04:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24281.064201 secs 2023-09-10T06:04:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:10 smithi170 ceph-mon[102832]: pgmap v13110: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 407 KiB/s wr, 31 op/s 2023-09-10T06:04:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:10 smithi082 ceph-mon[98694]: pgmap v13110: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 407 KiB/s wr, 31 op/s 2023-09-10T06:04:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:10 smithi098 ceph-mon[103124]: pgmap v13110: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 407 KiB/s wr, 31 op/s 2023-09-10T06:04:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:12 smithi170 ceph-mon[102832]: pgmap v13111: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 174 KiB/s wr, 30 op/s 2023-09-10T06:04:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:12 smithi082 ceph-mon[98694]: pgmap v13111: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 174 KiB/s wr, 30 op/s 2023-09-10T06:04:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:12 smithi098 ceph-mon[103124]: pgmap v13111: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 174 KiB/s wr, 30 op/s 2023-09-10T06:04:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24284.441154 secs 2023-09-10T06:04:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24284.441154 secs 2023-09-10T06:04:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24284.441154 secs 2023-09-10T06:04:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24285.410496 secs 2023-09-10T06:04:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:14 smithi170 ceph-mon[102832]: pgmap v13112: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 174 KiB/s wr, 30 op/s 2023-09-10T06:04:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24286.064265 secs 2023-09-10T06:04:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24285.410496 secs 2023-09-10T06:04:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:14 smithi082 ceph-mon[98694]: pgmap v13112: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 174 KiB/s wr, 30 op/s 2023-09-10T06:04:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24286.064265 secs 2023-09-10T06:04:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24285.410496 secs 2023-09-10T06:04:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:14 smithi098 ceph-mon[103124]: pgmap v13112: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 174 KiB/s wr, 30 op/s 2023-09-10T06:04:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24286.064265 secs 2023-09-10T06:04:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:16 smithi170 ceph-mon[102832]: osdmap e888: 12 total, 12 up, 12 in 2023-09-10T06:04:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:16 smithi170 ceph-mon[102832]: pgmap v13114: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 16 op/s 2023-09-10T06:04:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:16 smithi082 ceph-mon[98694]: osdmap e888: 12 total, 12 up, 12 in 2023-09-10T06:04:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:16 smithi082 ceph-mon[98694]: pgmap v13114: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 16 op/s 2023-09-10T06:04:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:16 smithi098 ceph-mon[103124]: osdmap e888: 12 total, 12 up, 12 in 2023-09-10T06:04:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:16 smithi098 ceph-mon[103124]: pgmap v13114: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 16 op/s 2023-09-10T06:04:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:18 smithi170 ceph-mon[102832]: pgmap v13115: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 14 op/s 2023-09-10T06:04:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24289.441220 secs 2023-09-10T06:04:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:18 smithi082 ceph-mon[98694]: pgmap v13115: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 14 op/s 2023-09-10T06:04:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24289.441220 secs 2023-09-10T06:04:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:18 smithi098 ceph-mon[103124]: pgmap v13115: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 14 op/s 2023-09-10T06:04:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24289.441220 secs 2023-09-10T06:04:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24290.410554 secs 2023-09-10T06:04:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24291.064332 secs 2023-09-10T06:04:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24290.410554 secs 2023-09-10T06:04:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24291.064332 secs 2023-09-10T06:04:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24290.410554 secs 2023-09-10T06:04:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24291.064332 secs 2023-09-10T06:04:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:20 smithi170 ceph-mon[102832]: pgmap v13116: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 12 op/s 2023-09-10T06:04:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:20 smithi082 ceph-mon[98694]: pgmap v13116: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 12 op/s 2023-09-10T06:04:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:20 smithi098 ceph-mon[103124]: pgmap v13116: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 12 op/s 2023-09-10T06:04:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:22 smithi170 ceph-mon[102832]: pgmap v13117: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T06:04:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:22 smithi082 ceph-mon[98694]: pgmap v13117: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T06:04:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:22 smithi098 ceph-mon[103124]: pgmap v13117: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T06:04:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24294.441311 secs 2023-09-10T06:04:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24294.441311 secs 2023-09-10T06:04:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24294.441311 secs 2023-09-10T06:04:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24295.410613 secs 2023-09-10T06:04:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:24 smithi170 ceph-mon[102832]: pgmap v13118: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T06:04:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24296.064397 secs 2023-09-10T06:04:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24295.410613 secs 2023-09-10T06:04:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:24 smithi082 ceph-mon[98694]: pgmap v13118: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T06:04:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24296.064397 secs 2023-09-10T06:04:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24295.410613 secs 2023-09-10T06:04:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:24 smithi098 ceph-mon[103124]: pgmap v13118: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 12 op/s 2023-09-10T06:04:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24296.064397 secs 2023-09-10T06:04:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:26 smithi170 ceph-mon[102832]: pgmap v13119: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 11 op/s 2023-09-10T06:04:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:26 smithi082 ceph-mon[98694]: pgmap v13119: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 11 op/s 2023-09-10T06:04:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:26 smithi098 ceph-mon[103124]: pgmap v13119: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 11 op/s 2023-09-10T06:04:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:04:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:04:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:04:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:28 smithi098 ceph-mon[103124]: pgmap v13120: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T06:04:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:04:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:04:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:04:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:28 smithi170 ceph-mon[102832]: pgmap v13120: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T06:04:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:04:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:04:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:04:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:28 smithi082 ceph-mon[98694]: pgmap v13120: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T06:04:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:04:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:04:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:04:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24300.410673 secs 2023-09-10T06:04:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24301.064465 secs 2023-09-10T06:04:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24300.410673 secs 2023-09-10T06:04:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24301.064465 secs 2023-09-10T06:04:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24300.410673 secs 2023-09-10T06:04:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24301.064465 secs 2023-09-10T06:04:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:30 smithi098 ceph-mon[103124]: pgmap v13121: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T06:04:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:30 smithi170 ceph-mon[102832]: pgmap v13121: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T06:04:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:30 smithi082 ceph-mon[98694]: pgmap v13121: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T06:04:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:04:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:04:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:04:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:04:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:04:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:32 smithi098 ceph-mon[103124]: pgmap v13122: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 176 KiB/s wr, 14 op/s 2023-09-10T06:04:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:04:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:04:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:32 smithi170 ceph-mon[102832]: pgmap v13122: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 176 KiB/s wr, 14 op/s 2023-09-10T06:04:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:04:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:04:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:32 smithi082 ceph-mon[98694]: pgmap v13122: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 176 KiB/s wr, 14 op/s 2023-09-10T06:04:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24305.410759 secs 2023-09-10T06:04:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:34 smithi170 ceph-mon[102832]: pgmap v13123: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 94 KiB/s wr, 9 op/s 2023-09-10T06:04:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:04:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24306.064486 secs 2023-09-10T06:04:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24305.410759 secs 2023-09-10T06:04:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:34 smithi082 ceph-mon[98694]: pgmap v13123: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 94 KiB/s wr, 9 op/s 2023-09-10T06:04:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:04:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24306.064486 secs 2023-09-10T06:04:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24305.410759 secs 2023-09-10T06:04:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:34 smithi098 ceph-mon[103124]: pgmap v13123: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 94 KiB/s wr, 9 op/s 2023-09-10T06:04:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:04:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24306.064486 secs 2023-09-10T06:04:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:36 smithi170 ceph-mon[102832]: pgmap v13124: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 360 KiB/s wr, 16 op/s 2023-09-10T06:04:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:04:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:36 smithi082 ceph-mon[98694]: pgmap v13124: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 360 KiB/s wr, 16 op/s 2023-09-10T06:04:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:04:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:36 smithi098 ceph-mon[103124]: pgmap v13124: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 360 KiB/s wr, 16 op/s 2023-09-10T06:04:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:04:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:38 smithi170 ceph-mon[102832]: pgmap v13125: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 360 KiB/s wr, 11 op/s 2023-09-10T06:04:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:38 smithi082 ceph-mon[98694]: pgmap v13125: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 360 KiB/s wr, 11 op/s 2023-09-10T06:04:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:38 smithi098 ceph-mon[103124]: pgmap v13125: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 360 KiB/s wr, 11 op/s 2023-09-10T06:04:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24310.410839 secs 2023-09-10T06:04:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24311.064549 secs 2023-09-10T06:04:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24310.410839 secs 2023-09-10T06:04:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24311.064549 secs 2023-09-10T06:04:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24310.410839 secs 2023-09-10T06:04:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24311.064549 secs 2023-09-10T06:04:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:40 smithi170 ceph-mon[102832]: pgmap v13126: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 360 KiB/s wr, 11 op/s 2023-09-10T06:04:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:40 smithi082 ceph-mon[98694]: pgmap v13126: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 360 KiB/s wr, 11 op/s 2023-09-10T06:04:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:40 smithi098 ceph-mon[103124]: pgmap v13126: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 360 KiB/s wr, 11 op/s 2023-09-10T06:04:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:42 smithi170 ceph-mon[102832]: pgmap v13127: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 376 KiB/s wr, 17 op/s 2023-09-10T06:04:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:42 smithi082 ceph-mon[98694]: pgmap v13127: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 376 KiB/s wr, 17 op/s 2023-09-10T06:04:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:42 smithi098 ceph-mon[103124]: pgmap v13127: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 376 KiB/s wr, 17 op/s 2023-09-10T06:04:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24315.410898 secs 2023-09-10T06:04:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:44 smithi170 ceph-mon[102832]: pgmap v13128: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 282 KiB/s wr, 11 op/s 2023-09-10T06:04:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:04:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24316.064613 secs 2023-09-10T06:04:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24315.410898 secs 2023-09-10T06:04:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:44 smithi082 ceph-mon[98694]: pgmap v13128: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 282 KiB/s wr, 11 op/s 2023-09-10T06:04:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:04:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24316.064613 secs 2023-09-10T06:04:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24315.410898 secs 2023-09-10T06:04:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:44 smithi098 ceph-mon[103124]: pgmap v13128: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 282 KiB/s wr, 11 op/s 2023-09-10T06:04:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:04:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24316.064613 secs 2023-09-10T06:04:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:46 smithi170 ceph-mon[102832]: pgmap v13129: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 739 KiB/s wr, 18 op/s 2023-09-10T06:04:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:04:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:46 smithi082 ceph-mon[98694]: pgmap v13129: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 739 KiB/s wr, 18 op/s 2023-09-10T06:04:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:04:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:46 smithi098 ceph-mon[103124]: pgmap v13129: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 739 KiB/s wr, 18 op/s 2023-09-10T06:04:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:04:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:48 smithi170 ceph-mon[102832]: pgmap v13130: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 474 KiB/s wr, 12 op/s 2023-09-10T06:04:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:48 smithi082 ceph-mon[98694]: pgmap v13130: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 474 KiB/s wr, 12 op/s 2023-09-10T06:04:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:48 smithi098 ceph-mon[103124]: pgmap v13130: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 474 KiB/s wr, 12 op/s 2023-09-10T06:04:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24320.410957 secs 2023-09-10T06:04:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24321.064678 secs 2023-09-10T06:04:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24320.410957 secs 2023-09-10T06:04:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24321.064678 secs 2023-09-10T06:04:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24320.410957 secs 2023-09-10T06:04:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24321.064678 secs 2023-09-10T06:04:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:50 smithi170 ceph-mon[102832]: pgmap v13131: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 474 KiB/s wr, 12 op/s 2023-09-10T06:04:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:50 smithi082 ceph-mon[98694]: pgmap v13131: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 474 KiB/s wr, 12 op/s 2023-09-10T06:04:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:50 smithi098 ceph-mon[103124]: pgmap v13131: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 474 KiB/s wr, 12 op/s 2023-09-10T06:04:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:52 smithi170 ceph-mon[102832]: pgmap v13132: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 570 KiB/s wr, 18 op/s 2023-09-10T06:04:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:52 smithi082 ceph-mon[98694]: pgmap v13132: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 570 KiB/s wr, 18 op/s 2023-09-10T06:04:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:52 smithi098 ceph-mon[103124]: pgmap v13132: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 570 KiB/s wr, 18 op/s 2023-09-10T06:04:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24325.411023 secs 2023-09-10T06:04:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:54 smithi170 ceph-mon[102832]: pgmap v13133: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 554 KiB/s wr, 13 op/s 2023-09-10T06:04:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:04:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24326.064747 secs 2023-09-10T06:04:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24325.411023 secs 2023-09-10T06:04:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:54 smithi082 ceph-mon[98694]: pgmap v13133: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 554 KiB/s wr, 13 op/s 2023-09-10T06:04:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:04:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24326.064747 secs 2023-09-10T06:04:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24325.411023 secs 2023-09-10T06:04:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:54 smithi098 ceph-mon[103124]: pgmap v13133: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 554 KiB/s wr, 13 op/s 2023-09-10T06:04:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:04:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24326.064747 secs 2023-09-10T06:04:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:56 smithi170 ceph-mon[102832]: pgmap v13134: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 638 KiB/s wr, 19 op/s 2023-09-10T06:04:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:04:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:56 smithi082 ceph-mon[98694]: pgmap v13134: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 638 KiB/s wr, 19 op/s 2023-09-10T06:04:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:04:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:56 smithi098 ceph-mon[103124]: pgmap v13134: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 638 KiB/s wr, 19 op/s 2023-09-10T06:04:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:04:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:04:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:58 smithi170 ceph-mon[102832]: pgmap v13135: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 181 KiB/s wr, 11 op/s 2023-09-10T06:04:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24329.441793 secs 2023-09-10T06:04:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:58 smithi082 ceph-mon[98694]: pgmap v13135: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 181 KiB/s wr, 11 op/s 2023-09-10T06:04:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24329.441793 secs 2023-09-10T06:04:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:58 smithi098 ceph-mon[103124]: pgmap v13135: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 181 KiB/s wr, 11 op/s 2023-09-10T06:04:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24329.441793 secs 2023-09-10T06:04:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24330.411079 secs 2023-09-10T06:04:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:04:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24331.064810 secs 2023-09-10T06:04:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24330.411079 secs 2023-09-10T06:04:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:04:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24331.064810 secs 2023-09-10T06:05:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24330.411079 secs 2023-09-10T06:05:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:04:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24331.064810 secs 2023-09-10T06:05:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:00 smithi170 ceph-mon[102832]: pgmap v13136: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 181 KiB/s wr, 11 op/s 2023-09-10T06:05:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:05:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:05:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:05:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:00 smithi082 ceph-mon[98694]: pgmap v13136: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 181 KiB/s wr, 11 op/s 2023-09-10T06:05:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:05:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:05:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:05:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:00 smithi098 ceph-mon[103124]: pgmap v13136: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 181 KiB/s wr, 11 op/s 2023-09-10T06:05:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:05:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:05:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:05:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:02 smithi170 ceph-mon[102832]: pgmap v13137: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 358 KiB/s wr, 17 op/s 2023-09-10T06:05:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:02 smithi082 ceph-mon[98694]: pgmap v13137: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 358 KiB/s wr, 17 op/s 2023-09-10T06:05:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:02 smithi098 ceph-mon[103124]: pgmap v13137: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 358 KiB/s wr, 17 op/s 2023-09-10T06:05:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24334.441881 secs 2023-09-10T06:05:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24334.441881 secs 2023-09-10T06:05:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24334.441881 secs 2023-09-10T06:05:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24335.411140 secs 2023-09-10T06:05:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:04 smithi170 ceph-mon[102832]: pgmap v13138: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 261 KiB/s wr, 11 op/s 2023-09-10T06:05:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24336.064874 secs 2023-09-10T06:05:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24335.411140 secs 2023-09-10T06:05:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:04 smithi082 ceph-mon[98694]: pgmap v13138: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 261 KiB/s wr, 11 op/s 2023-09-10T06:05:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24336.064874 secs 2023-09-10T06:05:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24335.411140 secs 2023-09-10T06:05:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:04 smithi098 ceph-mon[103124]: pgmap v13138: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 261 KiB/s wr, 11 op/s 2023-09-10T06:05:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24336.064874 secs 2023-09-10T06:05:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:06 smithi170 ceph-mon[102832]: pgmap v13139: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 388 KiB/s wr, 31 op/s 2023-09-10T06:05:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:06 smithi082 ceph-mon[98694]: pgmap v13139: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 388 KiB/s wr, 31 op/s 2023-09-10T06:05:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:06 smithi098 ceph-mon[103124]: pgmap v13139: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 388 KiB/s wr, 31 op/s 2023-09-10T06:05:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:08 smithi170 ceph-mon[102832]: pgmap v13140: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 303 KiB/s wr, 26 op/s 2023-09-10T06:05:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24339.441941 secs 2023-09-10T06:05:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:08 smithi082 ceph-mon[98694]: pgmap v13140: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 303 KiB/s wr, 26 op/s 2023-09-10T06:05:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24339.441941 secs 2023-09-10T06:05:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:08 smithi098 ceph-mon[103124]: pgmap v13140: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 303 KiB/s wr, 26 op/s 2023-09-10T06:05:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24339.441941 secs 2023-09-10T06:05:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24340.411198 secs 2023-09-10T06:05:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:09 smithi170 ceph-mon[102832]: osdmap e889: 12 total, 12 up, 12 in 2023-09-10T06:05:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24341.064940 secs 2023-09-10T06:05:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24340.411198 secs 2023-09-10T06:05:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:09 smithi082 ceph-mon[98694]: osdmap e889: 12 total, 12 up, 12 in 2023-09-10T06:05:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24341.064940 secs 2023-09-10T06:05:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24340.411198 secs 2023-09-10T06:05:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:09 smithi098 ceph-mon[103124]: osdmap e889: 12 total, 12 up, 12 in 2023-09-10T06:05:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24341.064940 secs 2023-09-10T06:05:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:10 smithi170 ceph-mon[102832]: pgmap v13142: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 363 KiB/s wr, 31 op/s 2023-09-10T06:05:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:10 smithi082 ceph-mon[98694]: pgmap v13142: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 363 KiB/s wr, 31 op/s 2023-09-10T06:05:11.020 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:10 smithi098 ceph-mon[103124]: pgmap v13142: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 363 KiB/s wr, 31 op/s 2023-09-10T06:05:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:12 smithi170 ceph-mon[102832]: pgmap v13143: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 174 KiB/s wr, 31 op/s 2023-09-10T06:05:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:12 smithi082 ceph-mon[98694]: pgmap v13143: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 174 KiB/s wr, 31 op/s 2023-09-10T06:05:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:12 smithi098 ceph-mon[103124]: pgmap v13143: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 174 KiB/s wr, 31 op/s 2023-09-10T06:05:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24344.442033 secs 2023-09-10T06:05:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24344.442033 secs 2023-09-10T06:05:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24344.442033 secs 2023-09-10T06:05:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24345.411256 secs 2023-09-10T06:05:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:14 smithi170 ceph-mon[102832]: pgmap v13144: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 174 KiB/s wr, 31 op/s 2023-09-10T06:05:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24346.065005 secs 2023-09-10T06:05:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24345.411256 secs 2023-09-10T06:05:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:14 smithi082 ceph-mon[98694]: pgmap v13144: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 174 KiB/s wr, 31 op/s 2023-09-10T06:05:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24346.065005 secs 2023-09-10T06:05:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24345.411256 secs 2023-09-10T06:05:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:14 smithi098 ceph-mon[103124]: pgmap v13144: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 174 KiB/s wr, 31 op/s 2023-09-10T06:05:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24346.065005 secs 2023-09-10T06:05:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:16 smithi170 ceph-mon[102832]: osdmap e890: 12 total, 12 up, 12 in 2023-09-10T06:05:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:16 smithi170 ceph-mon[102832]: pgmap v13146: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 16 op/s 2023-09-10T06:05:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:05:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:05:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:16 smithi082 ceph-mon[98694]: osdmap e890: 12 total, 12 up, 12 in 2023-09-10T06:05:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:16 smithi082 ceph-mon[98694]: pgmap v13146: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 16 op/s 2023-09-10T06:05:16.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:05:16.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:05:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:16 smithi098 ceph-mon[103124]: osdmap e890: 12 total, 12 up, 12 in 2023-09-10T06:05:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:16 smithi098 ceph-mon[103124]: pgmap v13146: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 16 op/s 2023-09-10T06:05:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:05:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:05:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:05:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:05:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:18 smithi170 ceph-mon[102832]: pgmap v13147: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 14 op/s 2023-09-10T06:05:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24349.442124 secs 2023-09-10T06:05:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:05:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:05:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:18 smithi082 ceph-mon[98694]: pgmap v13147: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 14 op/s 2023-09-10T06:05:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24349.442124 secs 2023-09-10T06:05:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:05:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:05:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:18 smithi098 ceph-mon[103124]: pgmap v13147: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 14 op/s 2023-09-10T06:05:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24349.442124 secs 2023-09-10T06:05:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24350.411313 secs 2023-09-10T06:05:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24351.065069 secs 2023-09-10T06:05:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24350.411313 secs 2023-09-10T06:05:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24351.065069 secs 2023-09-10T06:05:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24350.411313 secs 2023-09-10T06:05:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24351.065069 secs 2023-09-10T06:05:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:20 smithi082 ceph-mon[98694]: pgmap v13148: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 13 op/s 2023-09-10T06:05:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:20 smithi098 ceph-mon[103124]: pgmap v13148: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 13 op/s 2023-09-10T06:05:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:20 smithi170 ceph-mon[102832]: pgmap v13148: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 13 op/s 2023-09-10T06:05:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:22 smithi170 ceph-mon[102832]: pgmap v13149: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 12 op/s 2023-09-10T06:05:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:22 smithi082 ceph-mon[98694]: pgmap v13149: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 12 op/s 2023-09-10T06:05:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:22 smithi098 ceph-mon[103124]: pgmap v13149: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 12 op/s 2023-09-10T06:05:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24354.442182 secs 2023-09-10T06:05:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24354.442182 secs 2023-09-10T06:05:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24354.442182 secs 2023-09-10T06:05:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24355.411370 secs 2023-09-10T06:05:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:24 smithi170 ceph-mon[102832]: pgmap v13150: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 12 op/s 2023-09-10T06:05:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24356.065133 secs 2023-09-10T06:05:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24355.411370 secs 2023-09-10T06:05:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:24 smithi082 ceph-mon[98694]: pgmap v13150: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 12 op/s 2023-09-10T06:05:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24356.065133 secs 2023-09-10T06:05:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24355.411370 secs 2023-09-10T06:05:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:24 smithi098 ceph-mon[103124]: pgmap v13150: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 12 op/s 2023-09-10T06:05:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24356.065133 secs 2023-09-10T06:05:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:26 smithi170 ceph-mon[102832]: pgmap v13151: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 11 op/s 2023-09-10T06:05:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:26 smithi082 ceph-mon[98694]: pgmap v13151: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 11 op/s 2023-09-10T06:05:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:26 smithi098 ceph-mon[103124]: pgmap v13151: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 11 op/s 2023-09-10T06:05:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:28 smithi170 ceph-mon[102832]: pgmap v13152: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T06:05:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:05:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:28 smithi082 ceph-mon[98694]: pgmap v13152: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T06:05:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:05:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:28 smithi098 ceph-mon[103124]: pgmap v13152: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T06:05:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:05:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24360.411430 secs 2023-09-10T06:05:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:05:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:05:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24361.065199 secs 2023-09-10T06:05:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:05:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:05:29.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:05:29.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:29 smithi170 ceph-mon[102832]: pgmap v13153: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T06:05:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24360.411430 secs 2023-09-10T06:05:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:05:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:05:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24361.065199 secs 2023-09-10T06:05:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:05:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:05:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:05:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:29 smithi082 ceph-mon[98694]: pgmap v13153: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T06:05:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24360.411430 secs 2023-09-10T06:05:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:05:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:05:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24361.065199 secs 2023-09-10T06:05:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:05:30.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:05:30.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:05:30.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:29 smithi098 ceph-mon[103124]: pgmap v13153: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 9 op/s 2023-09-10T06:05:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:05:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:05:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:05:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:05:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:05:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:05:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:32 smithi170 ceph-mon[102832]: pgmap v13154: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 166 KiB/s wr, 15 op/s 2023-09-10T06:05:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:05:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:05:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:05:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:32 smithi082 ceph-mon[98694]: pgmap v13154: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 166 KiB/s wr, 15 op/s 2023-09-10T06:05:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:05:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:05:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:05:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:32 smithi098 ceph-mon[103124]: pgmap v13154: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 166 KiB/s wr, 15 op/s 2023-09-10T06:05:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24365.411510 secs 2023-09-10T06:05:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:34 smithi170 ceph-mon[102832]: pgmap v13155: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-09-10T06:05:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24366.065263 secs 2023-09-10T06:05:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24365.411510 secs 2023-09-10T06:05:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:34 smithi082 ceph-mon[98694]: pgmap v13155: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-09-10T06:05:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24366.065263 secs 2023-09-10T06:05:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24365.411510 secs 2023-09-10T06:05:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:34 smithi098 ceph-mon[103124]: pgmap v13155: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-09-10T06:05:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24366.065263 secs 2023-09-10T06:05:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:36 smithi170 ceph-mon[102832]: pgmap v13156: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 361 KiB/s wr, 16 op/s 2023-09-10T06:05:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:05:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:05:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:36 smithi082 ceph-mon[98694]: pgmap v13156: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 361 KiB/s wr, 16 op/s 2023-09-10T06:05:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:05:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:05:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:36 smithi098 ceph-mon[103124]: pgmap v13156: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 361 KiB/s wr, 16 op/s 2023-09-10T06:05:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:05:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:05:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:38 smithi170 ceph-mon[102832]: pgmap v13157: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 361 KiB/s wr, 12 op/s 2023-09-10T06:05:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:05:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:05:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:38 smithi082 ceph-mon[98694]: pgmap v13157: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 361 KiB/s wr, 12 op/s 2023-09-10T06:05:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:05:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:05:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:38 smithi098 ceph-mon[103124]: pgmap v13157: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 361 KiB/s wr, 12 op/s 2023-09-10T06:05:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:05:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:05:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24370.411570 secs 2023-09-10T06:05:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24371.065332 secs 2023-09-10T06:05:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24370.411570 secs 2023-09-10T06:05:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24371.065332 secs 2023-09-10T06:05:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24370.411570 secs 2023-09-10T06:05:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24371.065332 secs 2023-09-10T06:05:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:40 smithi170 ceph-mon[102832]: pgmap v13158: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 361 KiB/s wr, 12 op/s 2023-09-10T06:05:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:40 smithi082 ceph-mon[98694]: pgmap v13158: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 361 KiB/s wr, 12 op/s 2023-09-10T06:05:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:40 smithi098 ceph-mon[103124]: pgmap v13158: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 361 KiB/s wr, 12 op/s 2023-09-10T06:05:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:42 smithi170 ceph-mon[102832]: pgmap v13159: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 378 KiB/s wr, 18 op/s 2023-09-10T06:05:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:42 smithi082 ceph-mon[98694]: pgmap v13159: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 378 KiB/s wr, 18 op/s 2023-09-10T06:05:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:42 smithi098 ceph-mon[103124]: pgmap v13159: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 378 KiB/s wr, 18 op/s 2023-09-10T06:05:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24375.411632 secs 2023-09-10T06:05:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:44 smithi170 ceph-mon[102832]: pgmap v13160: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 295 KiB/s wr, 12 op/s 2023-09-10T06:05:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:05:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:05:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24376.065396 secs 2023-09-10T06:05:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24375.411632 secs 2023-09-10T06:05:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:44 smithi082 ceph-mon[98694]: pgmap v13160: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 295 KiB/s wr, 12 op/s 2023-09-10T06:05:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:05:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:05:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24376.065396 secs 2023-09-10T06:05:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24375.411632 secs 2023-09-10T06:05:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:44 smithi098 ceph-mon[103124]: pgmap v13160: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 295 KiB/s wr, 12 op/s 2023-09-10T06:05:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:05:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:05:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24376.065396 secs 2023-09-10T06:05:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:46 smithi170 ceph-mon[102832]: pgmap v13161: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 740 KiB/s wr, 19 op/s 2023-09-10T06:05:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:05:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:05:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:46 smithi082 ceph-mon[98694]: pgmap v13161: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 740 KiB/s wr, 19 op/s 2023-09-10T06:05:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:05:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:05:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:46 smithi098 ceph-mon[103124]: pgmap v13161: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 740 KiB/s wr, 19 op/s 2023-09-10T06:05:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:05:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:05:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:47 smithi170 ceph-mon[102832]: pgmap v13162: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 462 KiB/s wr, 12 op/s 2023-09-10T06:05:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:47 smithi082 ceph-mon[98694]: pgmap v13162: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 462 KiB/s wr, 12 op/s 2023-09-10T06:05:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:47 smithi098 ceph-mon[103124]: pgmap v13162: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 462 KiB/s wr, 12 op/s 2023-09-10T06:05:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24380.411711 secs 2023-09-10T06:05:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24380.411711 secs 2023-09-10T06:05:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24380.411711 secs 2023-09-10T06:05:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24381.065462 secs 2023-09-10T06:05:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:49 smithi170 ceph-mon[102832]: pgmap v13163: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 462 KiB/s wr, 12 op/s 2023-09-10T06:05:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24381.065462 secs 2023-09-10T06:05:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:49 smithi082 ceph-mon[98694]: pgmap v13163: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 462 KiB/s wr, 12 op/s 2023-09-10T06:05:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24381.065462 secs 2023-09-10T06:05:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:49 smithi098 ceph-mon[103124]: pgmap v13163: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 462 KiB/s wr, 12 op/s 2023-09-10T06:05:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:52 smithi170 ceph-mon[102832]: pgmap v13164: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 496 KiB/s wr, 18 op/s 2023-09-10T06:05:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:52 smithi082 ceph-mon[98694]: pgmap v13164: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 496 KiB/s wr, 18 op/s 2023-09-10T06:05:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:52 smithi098 ceph-mon[103124]: pgmap v13164: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 496 KiB/s wr, 18 op/s 2023-09-10T06:05:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24385.411769 secs 2023-09-10T06:05:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:54 smithi170 ceph-mon[102832]: pgmap v13165: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 479 KiB/s wr, 12 op/s 2023-09-10T06:05:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24386.065485 secs 2023-09-10T06:05:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24385.411769 secs 2023-09-10T06:05:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:54 smithi082 ceph-mon[98694]: pgmap v13165: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 479 KiB/s wr, 12 op/s 2023-09-10T06:05:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24386.065485 secs 2023-09-10T06:05:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24385.411769 secs 2023-09-10T06:05:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:54 smithi098 ceph-mon[103124]: pgmap v13165: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 479 KiB/s wr, 12 op/s 2023-09-10T06:05:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24386.065485 secs 2023-09-10T06:05:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:56 smithi170 ceph-mon[102832]: pgmap v13166: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 641 KiB/s wr, 18 op/s 2023-09-10T06:05:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:56 smithi082 ceph-mon[98694]: pgmap v13166: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 641 KiB/s wr, 18 op/s 2023-09-10T06:05:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:56 smithi098 ceph-mon[103124]: pgmap v13166: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 641 KiB/s wr, 18 op/s 2023-09-10T06:05:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:05:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:05:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:58 smithi082 ceph-mon[98694]: pgmap v13167: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 11 op/s 2023-09-10T06:05:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24389.442698 secs 2023-09-10T06:05:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:05:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:05:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:58 smithi098 ceph-mon[103124]: pgmap v13167: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 11 op/s 2023-09-10T06:05:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24389.442698 secs 2023-09-10T06:05:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:05:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:05:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:58 smithi170 ceph-mon[102832]: pgmap v13167: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 11 op/s 2023-09-10T06:05:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24389.442698 secs 2023-09-10T06:05:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24390.411854 secs 2023-09-10T06:05:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:05:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24391.065549 secs 2023-09-10T06:05:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24390.411854 secs 2023-09-10T06:05:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:05:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24391.065549 secs 2023-09-10T06:05:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24390.411854 secs 2023-09-10T06:05:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:05:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24391.065549 secs 2023-09-10T06:06:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:06:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:06:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:06:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:00 smithi082 ceph-mon[98694]: pgmap v13168: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 11 op/s 2023-09-10T06:06:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:06:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:06:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:06:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:00 smithi098 ceph-mon[103124]: pgmap v13168: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 11 op/s 2023-09-10T06:06:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:06:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:06:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:06:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:00 smithi170 ceph-mon[102832]: pgmap v13168: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 11 op/s 2023-09-10T06:06:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:02 smithi170 ceph-mon[102832]: pgmap v13169: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 421 KiB/s wr, 31 op/s 2023-09-10T06:06:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:02 smithi082 ceph-mon[98694]: pgmap v13169: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 421 KiB/s wr, 31 op/s 2023-09-10T06:06:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:02 smithi098 ceph-mon[103124]: pgmap v13169: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 421 KiB/s wr, 31 op/s 2023-09-10T06:06:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24394.442833 secs 2023-09-10T06:06:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24394.442833 secs 2023-09-10T06:06:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24394.442833 secs 2023-09-10T06:06:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24395.411946 secs 2023-09-10T06:06:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:04 smithi170 ceph-mon[102832]: pgmap v13170: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 386 KiB/s wr, 26 op/s 2023-09-10T06:06:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24396.065614 secs 2023-09-10T06:06:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24395.411946 secs 2023-09-10T06:06:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:04 smithi082 ceph-mon[98694]: pgmap v13170: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 386 KiB/s wr, 26 op/s 2023-09-10T06:06:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24396.065614 secs 2023-09-10T06:06:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24395.411946 secs 2023-09-10T06:06:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:04 smithi098 ceph-mon[103124]: pgmap v13170: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 386 KiB/s wr, 26 op/s 2023-09-10T06:06:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24396.065614 secs 2023-09-10T06:06:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:06 smithi170 ceph-mon[102832]: pgmap v13171: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 465 KiB/s wr, 32 op/s 2023-09-10T06:06:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:06 smithi082 ceph-mon[98694]: pgmap v13171: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 465 KiB/s wr, 32 op/s 2023-09-10T06:06:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:06 smithi098 ceph-mon[103124]: pgmap v13171: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 465 KiB/s wr, 32 op/s 2023-09-10T06:06:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:08 smithi170 ceph-mon[102832]: pgmap v13172: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 304 KiB/s wr, 26 op/s 2023-09-10T06:06:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24399.442901 secs 2023-09-10T06:06:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:08 smithi082 ceph-mon[98694]: pgmap v13172: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 304 KiB/s wr, 26 op/s 2023-09-10T06:06:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24399.442901 secs 2023-09-10T06:06:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:08 smithi098 ceph-mon[103124]: pgmap v13172: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 304 KiB/s wr, 26 op/s 2023-09-10T06:06:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24399.442901 secs 2023-09-10T06:06:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24400.412012 secs 2023-09-10T06:06:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:09 smithi170 ceph-mon[102832]: osdmap e891: 12 total, 12 up, 12 in 2023-09-10T06:06:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24401.065680 secs 2023-09-10T06:06:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24400.412012 secs 2023-09-10T06:06:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:09 smithi082 ceph-mon[98694]: osdmap e891: 12 total, 12 up, 12 in 2023-09-10T06:06:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24401.065680 secs 2023-09-10T06:06:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24400.412012 secs 2023-09-10T06:06:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:09 smithi098 ceph-mon[103124]: osdmap e891: 12 total, 12 up, 12 in 2023-09-10T06:06:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24401.065680 secs 2023-09-10T06:06:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:10 smithi170 ceph-mon[102832]: pgmap v13174: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 364 KiB/s wr, 31 op/s 2023-09-10T06:06:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:10 smithi082 ceph-mon[98694]: pgmap v13174: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 364 KiB/s wr, 31 op/s 2023-09-10T06:06:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:10 smithi098 ceph-mon[103124]: pgmap v13174: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 364 KiB/s wr, 31 op/s 2023-09-10T06:06:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:12 smithi170 ceph-mon[102832]: pgmap v13175: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 117 KiB/s wr, 13 op/s 2023-09-10T06:06:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:12 smithi082 ceph-mon[98694]: pgmap v13175: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 117 KiB/s wr, 13 op/s 2023-09-10T06:06:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:12 smithi098 ceph-mon[103124]: pgmap v13175: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 117 KiB/s wr, 13 op/s 2023-09-10T06:06:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24404.443022 secs 2023-09-10T06:06:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24404.443022 secs 2023-09-10T06:06:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24404.443022 secs 2023-09-10T06:06:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24405.412098 secs 2023-09-10T06:06:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:14 smithi170 ceph-mon[102832]: pgmap v13176: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 117 KiB/s wr, 13 op/s 2023-09-10T06:06:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24406.065737 secs 2023-09-10T06:06:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24405.412098 secs 2023-09-10T06:06:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:14 smithi082 ceph-mon[98694]: pgmap v13176: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 117 KiB/s wr, 13 op/s 2023-09-10T06:06:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24406.065737 secs 2023-09-10T06:06:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24405.412098 secs 2023-09-10T06:06:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:14 smithi098 ceph-mon[103124]: pgmap v13176: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 117 KiB/s wr, 13 op/s 2023-09-10T06:06:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24406.065737 secs 2023-09-10T06:06:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:16 smithi170 ceph-mon[102832]: osdmap e892: 12 total, 12 up, 12 in 2023-09-10T06:06:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:16 smithi170 ceph-mon[102832]: pgmap v13178: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 15 op/s 2023-09-10T06:06:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:16 smithi082 ceph-mon[98694]: osdmap e892: 12 total, 12 up, 12 in 2023-09-10T06:06:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:16 smithi082 ceph-mon[98694]: pgmap v13178: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 15 op/s 2023-09-10T06:06:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:16 smithi098 ceph-mon[103124]: osdmap e892: 12 total, 12 up, 12 in 2023-09-10T06:06:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:16 smithi098 ceph-mon[103124]: pgmap v13178: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 15 op/s 2023-09-10T06:06:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:18 smithi170 ceph-mon[102832]: pgmap v13179: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 14 op/s 2023-09-10T06:06:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24409.443084 secs 2023-09-10T06:06:18.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:18 smithi082 ceph-mon[98694]: pgmap v13179: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 14 op/s 2023-09-10T06:06:18.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24409.443084 secs 2023-09-10T06:06:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:18 smithi098 ceph-mon[103124]: pgmap v13179: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 14 op/s 2023-09-10T06:06:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24409.443084 secs 2023-09-10T06:06:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24410.412182 secs 2023-09-10T06:06:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24411.065804 secs 2023-09-10T06:06:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24410.412182 secs 2023-09-10T06:06:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24411.065804 secs 2023-09-10T06:06:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24410.412182 secs 2023-09-10T06:06:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24411.065804 secs 2023-09-10T06:06:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:20 smithi170 ceph-mon[102832]: pgmap v13180: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T06:06:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:20 smithi082 ceph-mon[98694]: pgmap v13180: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T06:06:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:20 smithi098 ceph-mon[103124]: pgmap v13180: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T06:06:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:22 smithi170 ceph-mon[102832]: pgmap v13181: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 12 op/s 2023-09-10T06:06:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:22 smithi082 ceph-mon[98694]: pgmap v13181: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 12 op/s 2023-09-10T06:06:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:22 smithi098 ceph-mon[103124]: pgmap v13181: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 12 op/s 2023-09-10T06:06:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24414.443146 secs 2023-09-10T06:06:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24414.443146 secs 2023-09-10T06:06:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24414.443146 secs 2023-09-10T06:06:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24415.412239 secs 2023-09-10T06:06:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:24 smithi170 ceph-mon[102832]: pgmap v13182: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 12 op/s 2023-09-10T06:06:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24416.065870 secs 2023-09-10T06:06:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24415.412239 secs 2023-09-10T06:06:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:24 smithi082 ceph-mon[98694]: pgmap v13182: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 12 op/s 2023-09-10T06:06:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24416.065870 secs 2023-09-10T06:06:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24415.412239 secs 2023-09-10T06:06:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:24 smithi098 ceph-mon[103124]: pgmap v13182: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 12 op/s 2023-09-10T06:06:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24416.065870 secs 2023-09-10T06:06:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:26 smithi082 ceph-mon[98694]: pgmap v13183: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T06:06:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:26 smithi098 ceph-mon[103124]: pgmap v13183: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T06:06:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:26 smithi170 ceph-mon[102832]: pgmap v13183: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T06:06:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:28 smithi170 ceph-mon[102832]: pgmap v13184: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-09-10T06:06:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24419.443260 secs 2023-09-10T06:06:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:28 smithi082 ceph-mon[98694]: pgmap v13184: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-09-10T06:06:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24419.443260 secs 2023-09-10T06:06:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:28 smithi098 ceph-mon[103124]: pgmap v13184: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-09-10T06:06:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24419.443260 secs 2023-09-10T06:06:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24420.412293 secs 2023-09-10T06:06:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24421.065939 secs 2023-09-10T06:06:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:06:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24420.412293 secs 2023-09-10T06:06:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24421.065939 secs 2023-09-10T06:06:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:06:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24420.412293 secs 2023-09-10T06:06:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24421.065939 secs 2023-09-10T06:06:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:06:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:30 smithi170 ceph-mon[102832]: pgmap v13185: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-09-10T06:06:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:06:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:06:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:06:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:06:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:06:30.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:06:30.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:06:30.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:06:30.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:06:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:30 smithi082 ceph-mon[98694]: pgmap v13185: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-09-10T06:06:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:06:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:06:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:06:30.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:06:30.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:06:30.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:06:30.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:06:30.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:06:30.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:06:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:30 smithi098 ceph-mon[103124]: pgmap v13185: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 10 op/s 2023-09-10T06:06:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:06:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:06:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:06:31.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:06:31.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:06:31.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:06:31.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:06:31.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:06:31.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:06:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:32 smithi170 ceph-mon[102832]: pgmap v13186: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 166 KiB/s wr, 15 op/s 2023-09-10T06:06:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:06:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:06:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:32 smithi082 ceph-mon[98694]: pgmap v13186: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 166 KiB/s wr, 15 op/s 2023-09-10T06:06:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:06:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:06:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:32 smithi098 ceph-mon[103124]: pgmap v13186: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 166 KiB/s wr, 15 op/s 2023-09-10T06:06:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:06:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:06:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24425.412352 secs 2023-09-10T06:06:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:34 smithi170 ceph-mon[102832]: pgmap v13187: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T06:06:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24426.066026 secs 2023-09-10T06:06:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24425.412352 secs 2023-09-10T06:06:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:34 smithi082 ceph-mon[98694]: pgmap v13187: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T06:06:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24426.066026 secs 2023-09-10T06:06:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24425.412352 secs 2023-09-10T06:06:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:34 smithi098 ceph-mon[103124]: pgmap v13187: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T06:06:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24426.066026 secs 2023-09-10T06:06:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:06:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:06:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:06:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:36 smithi170 ceph-mon[102832]: pgmap v13188: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 372 KiB/s wr, 16 op/s 2023-09-10T06:06:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:06:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:06:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:06:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:36 smithi082 ceph-mon[98694]: pgmap v13188: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 372 KiB/s wr, 16 op/s 2023-09-10T06:06:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:06:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:06:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:06:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:36 smithi098 ceph-mon[103124]: pgmap v13188: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 372 KiB/s wr, 16 op/s 2023-09-10T06:06:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:06:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:06:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:06:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:37 smithi170 ceph-mon[102832]: pgmap v13189: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 372 KiB/s wr, 12 op/s 2023-09-10T06:06:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:37 smithi082 ceph-mon[98694]: pgmap v13189: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 372 KiB/s wr, 12 op/s 2023-09-10T06:06:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:37 smithi098 ceph-mon[103124]: pgmap v13189: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 372 KiB/s wr, 12 op/s 2023-09-10T06:06:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24430.412410 secs 2023-09-10T06:06:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24430.412410 secs 2023-09-10T06:06:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24430.412410 secs 2023-09-10T06:06:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24431.066095 secs 2023-09-10T06:06:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:39 smithi170 ceph-mon[102832]: pgmap v13190: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 372 KiB/s wr, 12 op/s 2023-09-10T06:06:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24431.066095 secs 2023-09-10T06:06:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:39 smithi082 ceph-mon[98694]: pgmap v13190: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 372 KiB/s wr, 12 op/s 2023-09-10T06:06:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24431.066095 secs 2023-09-10T06:06:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:39 smithi098 ceph-mon[103124]: pgmap v13190: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 372 KiB/s wr, 12 op/s 2023-09-10T06:06:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:06:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:06:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:06:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:06:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:06:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:06:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:41 smithi170 ceph-mon[102832]: pgmap v13191: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 382 KiB/s wr, 18 op/s 2023-09-10T06:06:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:41 smithi082 ceph-mon[98694]: pgmap v13191: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 382 KiB/s wr, 18 op/s 2023-09-10T06:06:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:41 smithi098 ceph-mon[103124]: pgmap v13191: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 382 KiB/s wr, 18 op/s 2023-09-10T06:06:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:06:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:06:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24435.412464 secs 2023-09-10T06:06:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:43 smithi098 ceph-mon[103124]: pgmap v13192: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 298 KiB/s wr, 12 op/s 2023-09-10T06:06:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:06:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:06:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24435.412464 secs 2023-09-10T06:06:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:43 smithi170 ceph-mon[102832]: pgmap v13192: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 298 KiB/s wr, 12 op/s 2023-09-10T06:06:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:06:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:06:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24435.412464 secs 2023-09-10T06:06:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:43 smithi082 ceph-mon[98694]: pgmap v13192: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 298 KiB/s wr, 12 op/s 2023-09-10T06:06:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24436.066160 secs 2023-09-10T06:06:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24436.066160 secs 2023-09-10T06:06:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24436.066160 secs 2023-09-10T06:06:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:06:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:06:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:45 smithi098 ceph-mon[103124]: pgmap v13193: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 744 KiB/s wr, 19 op/s 2023-09-10T06:06:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:06:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:06:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:45 smithi170 ceph-mon[102832]: pgmap v13193: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 744 KiB/s wr, 19 op/s 2023-09-10T06:06:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:06:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:06:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:45 smithi082 ceph-mon[98694]: pgmap v13193: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 744 KiB/s wr, 19 op/s 2023-09-10T06:06:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:48 smithi170 ceph-mon[102832]: pgmap v13194: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 456 KiB/s wr, 13 op/s 2023-09-10T06:06:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:48 smithi082 ceph-mon[98694]: pgmap v13194: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 456 KiB/s wr, 13 op/s 2023-09-10T06:06:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:48 smithi098 ceph-mon[103124]: pgmap v13194: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 456 KiB/s wr, 13 op/s 2023-09-10T06:06:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24440.412493 secs 2023-09-10T06:06:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24441.066224 secs 2023-09-10T06:06:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24440.412493 secs 2023-09-10T06:06:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24441.066224 secs 2023-09-10T06:06:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24440.412493 secs 2023-09-10T06:06:50.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24441.066224 secs 2023-09-10T06:06:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:50 smithi170 ceph-mon[102832]: pgmap v13195: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 456 KiB/s wr, 13 op/s 2023-09-10T06:06:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:06:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:06:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:50 smithi082 ceph-mon[98694]: pgmap v13195: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 456 KiB/s wr, 13 op/s 2023-09-10T06:06:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:06:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:06:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:50 smithi098 ceph-mon[103124]: pgmap v13195: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 456 KiB/s wr, 13 op/s 2023-09-10T06:06:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:06:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:06:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:52 smithi170 ceph-mon[102832]: pgmap v13196: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 504 KiB/s wr, 18 op/s 2023-09-10T06:06:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:52 smithi082 ceph-mon[98694]: pgmap v13196: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 504 KiB/s wr, 18 op/s 2023-09-10T06:06:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:52 smithi098 ceph-mon[103124]: pgmap v13196: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 504 KiB/s wr, 18 op/s 2023-09-10T06:06:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24445.412555 secs 2023-09-10T06:06:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:54 smithi170 ceph-mon[102832]: pgmap v13197: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 494 KiB/s wr, 12 op/s 2023-09-10T06:06:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24446.066288 secs 2023-09-10T06:06:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24445.412555 secs 2023-09-10T06:06:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:54 smithi082 ceph-mon[98694]: pgmap v13197: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 494 KiB/s wr, 12 op/s 2023-09-10T06:06:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24446.066288 secs 2023-09-10T06:06:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24445.412555 secs 2023-09-10T06:06:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:54 smithi098 ceph-mon[103124]: pgmap v13197: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 494 KiB/s wr, 12 op/s 2023-09-10T06:06:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24446.066288 secs 2023-09-10T06:06:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:56 smithi082 ceph-mon[98694]: pgmap v13198: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 643 KiB/s wr, 17 op/s 2023-09-10T06:06:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:56 smithi098 ceph-mon[103124]: pgmap v13198: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 643 KiB/s wr, 17 op/s 2023-09-10T06:06:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:56 smithi170 ceph-mon[102832]: pgmap v13198: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 643 KiB/s wr, 17 op/s 2023-09-10T06:06:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:58 smithi170 ceph-mon[102832]: pgmap v13199: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 197 KiB/s wr, 10 op/s 2023-09-10T06:06:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24449.443720 secs 2023-09-10T06:06:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:58 smithi082 ceph-mon[98694]: pgmap v13199: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 197 KiB/s wr, 10 op/s 2023-09-10T06:06:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24449.443720 secs 2023-09-10T06:06:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:58 smithi098 ceph-mon[103124]: pgmap v13199: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 197 KiB/s wr, 10 op/s 2023-09-10T06:06:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24449.443720 secs 2023-09-10T06:06:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24450.412641 secs 2023-09-10T06:06:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:06:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24451.066352 secs 2023-09-10T06:06:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24450.412641 secs 2023-09-10T06:06:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:06:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24451.066352 secs 2023-09-10T06:07:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24450.412641 secs 2023-09-10T06:07:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:06:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24451.066352 secs 2023-09-10T06:07:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:00 smithi170 ceph-mon[102832]: pgmap v13200: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 197 KiB/s wr, 10 op/s 2023-09-10T06:07:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:07:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:07:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:07:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:00 smithi082 ceph-mon[98694]: pgmap v13200: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 197 KiB/s wr, 10 op/s 2023-09-10T06:07:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:07:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:07:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:07:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:00 smithi098 ceph-mon[103124]: pgmap v13200: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 197 KiB/s wr, 10 op/s 2023-09-10T06:07:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:07:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:07:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:07:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:02 smithi170 ceph-mon[102832]: pgmap v13201: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 396 KiB/s wr, 24 op/s 2023-09-10T06:07:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:02 smithi082 ceph-mon[98694]: pgmap v13201: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 396 KiB/s wr, 24 op/s 2023-09-10T06:07:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:02 smithi098 ceph-mon[103124]: pgmap v13201: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 396 KiB/s wr, 24 op/s 2023-09-10T06:07:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24454.443777 secs 2023-09-10T06:07:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24454.443777 secs 2023-09-10T06:07:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24454.443777 secs 2023-09-10T06:07:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24455.412700 secs 2023-09-10T06:07:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:04 smithi170 ceph-mon[102832]: pgmap v13202: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 349 KiB/s wr, 18 op/s 2023-09-10T06:07:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:07:04.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:07:04.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24456.066418 secs 2023-09-10T06:07:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24455.412700 secs 2023-09-10T06:07:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:04 smithi082 ceph-mon[98694]: pgmap v13202: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 349 KiB/s wr, 18 op/s 2023-09-10T06:07:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:07:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:07:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24456.066418 secs 2023-09-10T06:07:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24455.412700 secs 2023-09-10T06:07:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:04 smithi098 ceph-mon[103124]: pgmap v13202: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 349 KiB/s wr, 18 op/s 2023-09-10T06:07:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:07:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:07:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24456.066418 secs 2023-09-10T06:07:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:06 smithi082 ceph-mon[98694]: pgmap v13203: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 454 KiB/s wr, 31 op/s 2023-09-10T06:07:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:07:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:07:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:06 smithi098 ceph-mon[103124]: pgmap v13203: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 454 KiB/s wr, 31 op/s 2023-09-10T06:07:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:07:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:07:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:06 smithi170 ceph-mon[102832]: pgmap v13203: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 454 KiB/s wr, 31 op/s 2023-09-10T06:07:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:07:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:07:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:08 smithi170 ceph-mon[102832]: pgmap v13204: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 305 KiB/s wr, 25 op/s 2023-09-10T06:07:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24459.443838 secs 2023-09-10T06:07:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:08 smithi082 ceph-mon[98694]: pgmap v13204: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 305 KiB/s wr, 25 op/s 2023-09-10T06:07:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24459.443838 secs 2023-09-10T06:07:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:08 smithi098 ceph-mon[103124]: pgmap v13204: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 305 KiB/s wr, 25 op/s 2023-09-10T06:07:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24459.443838 secs 2023-09-10T06:07:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24460.412782 secs 2023-09-10T06:07:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:09 smithi170 ceph-mon[102832]: osdmap e893: 12 total, 12 up, 12 in 2023-09-10T06:07:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24461.066485 secs 2023-09-10T06:07:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24460.412782 secs 2023-09-10T06:07:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:09 smithi082 ceph-mon[98694]: osdmap e893: 12 total, 12 up, 12 in 2023-09-10T06:07:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24461.066485 secs 2023-09-10T06:07:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24460.412782 secs 2023-09-10T06:07:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:09 smithi098 ceph-mon[103124]: osdmap e893: 12 total, 12 up, 12 in 2023-09-10T06:07:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24461.066485 secs 2023-09-10T06:07:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:10 smithi170 ceph-mon[102832]: pgmap v13206: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 365 KiB/s wr, 30 op/s 2023-09-10T06:07:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:10 smithi082 ceph-mon[98694]: pgmap v13206: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 365 KiB/s wr, 30 op/s 2023-09-10T06:07:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:10 smithi098 ceph-mon[103124]: pgmap v13206: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 365 KiB/s wr, 30 op/s 2023-09-10T06:07:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:11 smithi170 ceph-mon[102832]: pgmap v13207: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 148 KiB/s wr, 20 op/s 2023-09-10T06:07:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:11 smithi082 ceph-mon[98694]: pgmap v13207: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 148 KiB/s wr, 20 op/s 2023-09-10T06:07:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:11 smithi098 ceph-mon[103124]: pgmap v13207: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 148 KiB/s wr, 20 op/s 2023-09-10T06:07:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24464.443898 secs 2023-09-10T06:07:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24464.443898 secs 2023-09-10T06:07:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24464.443898 secs 2023-09-10T06:07:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24465.412862 secs 2023-09-10T06:07:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:14 smithi082 ceph-mon[98694]: pgmap v13208: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 148 KiB/s wr, 20 op/s 2023-09-10T06:07:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24465.412862 secs 2023-09-10T06:07:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:14 smithi098 ceph-mon[103124]: pgmap v13208: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 148 KiB/s wr, 20 op/s 2023-09-10T06:07:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24465.412862 secs 2023-09-10T06:07:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:14 smithi170 ceph-mon[102832]: pgmap v13208: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 148 KiB/s wr, 20 op/s 2023-09-10T06:07:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24466.066552 secs 2023-09-10T06:07:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24466.066552 secs 2023-09-10T06:07:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24466.066552 secs 2023-09-10T06:07:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:16 smithi082 ceph-mon[98694]: osdmap e894: 12 total, 12 up, 12 in 2023-09-10T06:07:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:16 smithi082 ceph-mon[98694]: pgmap v13210: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 14 op/s 2023-09-10T06:07:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:16 smithi098 ceph-mon[103124]: osdmap e894: 12 total, 12 up, 12 in 2023-09-10T06:07:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:16 smithi098 ceph-mon[103124]: pgmap v13210: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 14 op/s 2023-09-10T06:07:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:16 smithi170 ceph-mon[102832]: osdmap e894: 12 total, 12 up, 12 in 2023-09-10T06:07:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:16 smithi170 ceph-mon[102832]: pgmap v13210: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 14 op/s 2023-09-10T06:07:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:18 smithi170 ceph-mon[102832]: pgmap v13211: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T06:07:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24469.443958 secs 2023-09-10T06:07:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:18 smithi082 ceph-mon[98694]: pgmap v13211: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T06:07:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24469.443958 secs 2023-09-10T06:07:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:18 smithi098 ceph-mon[103124]: pgmap v13211: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T06:07:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24469.443958 secs 2023-09-10T06:07:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24470.412920 secs 2023-09-10T06:07:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24471.066615 secs 2023-09-10T06:07:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24470.412920 secs 2023-09-10T06:07:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24471.066615 secs 2023-09-10T06:07:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24470.412920 secs 2023-09-10T06:07:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24471.066615 secs 2023-09-10T06:07:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:20 smithi170 ceph-mon[102832]: pgmap v13212: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T06:07:20.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:20 smithi082 ceph-mon[98694]: pgmap v13212: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T06:07:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:20 smithi098 ceph-mon[103124]: pgmap v13212: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T06:07:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:22 smithi170 ceph-mon[102832]: pgmap v13213: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T06:07:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:22 smithi082 ceph-mon[98694]: pgmap v13213: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T06:07:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:22 smithi098 ceph-mon[103124]: pgmap v13213: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T06:07:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24474.444018 secs 2023-09-10T06:07:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24474.444018 secs 2023-09-10T06:07:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24474.444018 secs 2023-09-10T06:07:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24475.412979 secs 2023-09-10T06:07:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:24 smithi170 ceph-mon[102832]: pgmap v13214: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T06:07:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24476.066682 secs 2023-09-10T06:07:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24475.412979 secs 2023-09-10T06:07:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:24 smithi082 ceph-mon[98694]: pgmap v13214: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T06:07:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24476.066682 secs 2023-09-10T06:07:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24475.412979 secs 2023-09-10T06:07:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:24 smithi098 ceph-mon[103124]: pgmap v13214: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T06:07:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24476.066682 secs 2023-09-10T06:07:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:26 smithi082 ceph-mon[98694]: pgmap v13215: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T06:07:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:26 smithi098 ceph-mon[103124]: pgmap v13215: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T06:07:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:26 smithi170 ceph-mon[102832]: pgmap v13215: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T06:07:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:28 smithi170 ceph-mon[102832]: pgmap v13216: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 9 op/s 2023-09-10T06:07:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24479.444077 secs 2023-09-10T06:07:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:28 smithi082 ceph-mon[98694]: pgmap v13216: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 9 op/s 2023-09-10T06:07:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24479.444077 secs 2023-09-10T06:07:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:28 smithi098 ceph-mon[103124]: pgmap v13216: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 9 op/s 2023-09-10T06:07:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24479.444077 secs 2023-09-10T06:07:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24480.413064 secs 2023-09-10T06:07:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24481.066747 secs 2023-09-10T06:07:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24480.413064 secs 2023-09-10T06:07:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24481.066747 secs 2023-09-10T06:07:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24480.413064 secs 2023-09-10T06:07:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24481.066747 secs 2023-09-10T06:07:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:30 smithi170 ceph-mon[102832]: pgmap v13217: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 9 op/s 2023-09-10T06:07:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:07:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:30 smithi082 ceph-mon[98694]: pgmap v13217: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 9 op/s 2023-09-10T06:07:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:07:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:30 smithi098 ceph-mon[103124]: pgmap v13217: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 9 op/s 2023-09-10T06:07:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:07:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:07:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:07:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:07:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:07:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:07:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:07:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:07:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:07:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:07:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:32 smithi170 ceph-mon[102832]: pgmap v13218: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 190 KiB/s wr, 14 op/s 2023-09-10T06:07:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:32 smithi082 ceph-mon[98694]: pgmap v13218: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 190 KiB/s wr, 14 op/s 2023-09-10T06:07:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:32 smithi098 ceph-mon[103124]: pgmap v13218: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 190 KiB/s wr, 14 op/s 2023-09-10T06:07:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:33 smithi170 ceph-mon[102832]: pgmap v13219: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 107 KiB/s wr, 9 op/s 2023-09-10T06:07:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:33 smithi082 ceph-mon[98694]: pgmap v13219: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 107 KiB/s wr, 9 op/s 2023-09-10T06:07:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:33 smithi098 ceph-mon[103124]: pgmap v13219: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 107 KiB/s wr, 9 op/s 2023-09-10T06:07:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24485.413149 secs 2023-09-10T06:07:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:07:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:07:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24486.066813 secs 2023-09-10T06:07:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24485.413149 secs 2023-09-10T06:07:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:07:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:07:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24486.066813 secs 2023-09-10T06:07:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24485.413149 secs 2023-09-10T06:07:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:07:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:07:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24486.066813 secs 2023-09-10T06:07:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:07:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:35 smithi170 ceph-mon[102832]: pgmap v13220: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 364 KiB/s wr, 16 op/s 2023-09-10T06:07:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:07:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:35 smithi082 ceph-mon[98694]: pgmap v13220: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 364 KiB/s wr, 16 op/s 2023-09-10T06:07:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:07:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:35 smithi098 ceph-mon[103124]: pgmap v13220: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 364 KiB/s wr, 16 op/s 2023-09-10T06:07:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:07:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:07:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:07:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:07:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:07:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:07:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:07:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:07:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:07:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:37 smithi170 ceph-mon[102832]: pgmap v13221: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 364 KiB/s wr, 12 op/s 2023-09-10T06:07:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:37 smithi082 ceph-mon[98694]: pgmap v13221: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 364 KiB/s wr, 12 op/s 2023-09-10T06:07:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:37 smithi098 ceph-mon[103124]: pgmap v13221: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 364 KiB/s wr, 12 op/s 2023-09-10T06:07:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24490.413207 secs 2023-09-10T06:07:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24490.413207 secs 2023-09-10T06:07:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24490.413207 secs 2023-09-10T06:07:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24491.066877 secs 2023-09-10T06:07:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:39 smithi170 ceph-mon[102832]: pgmap v13222: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 364 KiB/s wr, 12 op/s 2023-09-10T06:07:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24491.066877 secs 2023-09-10T06:07:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:39 smithi082 ceph-mon[98694]: pgmap v13222: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 364 KiB/s wr, 12 op/s 2023-09-10T06:07:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24491.066877 secs 2023-09-10T06:07:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:39 smithi098 ceph-mon[103124]: pgmap v13222: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 364 KiB/s wr, 12 op/s 2023-09-10T06:07:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:42 smithi170 ceph-mon[102832]: pgmap v13223: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 381 KiB/s wr, 17 op/s 2023-09-10T06:07:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:42 smithi082 ceph-mon[98694]: pgmap v13223: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 381 KiB/s wr, 17 op/s 2023-09-10T06:07:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:42 smithi098 ceph-mon[103124]: pgmap v13223: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 381 KiB/s wr, 17 op/s 2023-09-10T06:07:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24495.413266 secs 2023-09-10T06:07:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:44 smithi170 ceph-mon[102832]: pgmap v13224: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 274 KiB/s wr, 11 op/s 2023-09-10T06:07:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:07:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:07:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24496.066940 secs 2023-09-10T06:07:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24495.413266 secs 2023-09-10T06:07:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:44 smithi082 ceph-mon[98694]: pgmap v13224: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 274 KiB/s wr, 11 op/s 2023-09-10T06:07:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:07:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:07:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24496.066940 secs 2023-09-10T06:07:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24495.413266 secs 2023-09-10T06:07:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:44 smithi098 ceph-mon[103124]: pgmap v13224: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 274 KiB/s wr, 11 op/s 2023-09-10T06:07:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:07:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:07:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24496.066940 secs 2023-09-10T06:07:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:46 smithi082 ceph-mon[98694]: pgmap v13225: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 722 KiB/s wr, 18 op/s 2023-09-10T06:07:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:07:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:07:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:46 smithi098 ceph-mon[103124]: pgmap v13225: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 722 KiB/s wr, 18 op/s 2023-09-10T06:07:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:07:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:07:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:46 smithi170 ceph-mon[102832]: pgmap v13225: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 722 KiB/s wr, 18 op/s 2023-09-10T06:07:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:07:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:07:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:48 smithi170 ceph-mon[102832]: pgmap v13226: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 464 KiB/s wr, 11 op/s 2023-09-10T06:07:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:48 smithi082 ceph-mon[98694]: pgmap v13226: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 464 KiB/s wr, 11 op/s 2023-09-10T06:07:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:48 smithi098 ceph-mon[103124]: pgmap v13226: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 464 KiB/s wr, 11 op/s 2023-09-10T06:07:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24500.413326 secs 2023-09-10T06:07:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24501.067006 secs 2023-09-10T06:07:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24500.413326 secs 2023-09-10T06:07:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24501.067006 secs 2023-09-10T06:07:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24500.413326 secs 2023-09-10T06:07:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24501.067006 secs 2023-09-10T06:07:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:50 smithi170 ceph-mon[102832]: pgmap v13227: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 464 KiB/s wr, 11 op/s 2023-09-10T06:07:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:50 smithi082 ceph-mon[98694]: pgmap v13227: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 464 KiB/s wr, 11 op/s 2023-09-10T06:07:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:50 smithi098 ceph-mon[103124]: pgmap v13227: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 464 KiB/s wr, 11 op/s 2023-09-10T06:07:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:51 smithi170 ceph-mon[102832]: pgmap v13228: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 501 KiB/s wr, 17 op/s 2023-09-10T06:07:51.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:51 smithi082 ceph-mon[98694]: pgmap v13228: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 501 KiB/s wr, 17 op/s 2023-09-10T06:07:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:51 smithi098 ceph-mon[103124]: pgmap v13228: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 501 KiB/s wr, 17 op/s 2023-09-10T06:07:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24505.413389 secs 2023-09-10T06:07:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:54 smithi170 ceph-mon[102832]: pgmap v13229: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 484 KiB/s wr, 12 op/s 2023-09-10T06:07:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24506.067072 secs 2023-09-10T06:07:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24505.413389 secs 2023-09-10T06:07:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:54 smithi082 ceph-mon[98694]: pgmap v13229: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 484 KiB/s wr, 12 op/s 2023-09-10T06:07:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24506.067072 secs 2023-09-10T06:07:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24505.413389 secs 2023-09-10T06:07:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:54 smithi098 ceph-mon[103124]: pgmap v13229: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 484 KiB/s wr, 12 op/s 2023-09-10T06:07:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24506.067072 secs 2023-09-10T06:07:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:56 smithi098 ceph-mon[103124]: pgmap v13230: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 645 KiB/s wr, 18 op/s 2023-09-10T06:07:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:56 smithi170 ceph-mon[102832]: pgmap v13230: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 645 KiB/s wr, 18 op/s 2023-09-10T06:07:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:56 smithi082 ceph-mon[98694]: pgmap v13230: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 645 KiB/s wr, 18 op/s 2023-09-10T06:07:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:07:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:07:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:58 smithi098 ceph-mon[103124]: pgmap v13231: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 198 KiB/s wr, 11 op/s 2023-09-10T06:07:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24509.444510 secs 2023-09-10T06:07:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:07:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:07:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:58 smithi170 ceph-mon[102832]: pgmap v13231: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 198 KiB/s wr, 11 op/s 2023-09-10T06:07:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24509.444510 secs 2023-09-10T06:07:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:07:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:07:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:58 smithi082 ceph-mon[98694]: pgmap v13231: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 198 KiB/s wr, 11 op/s 2023-09-10T06:07:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24509.444510 secs 2023-09-10T06:07:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24510.413474 secs 2023-09-10T06:07:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:07:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24511.067138 secs 2023-09-10T06:07:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24510.413474 secs 2023-09-10T06:07:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:07:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24511.067138 secs 2023-09-10T06:07:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24510.413474 secs 2023-09-10T06:07:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:07:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24511.067138 secs 2023-09-10T06:08:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:08:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:08:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:08:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:00 smithi098 ceph-mon[103124]: pgmap v13232: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 198 KiB/s wr, 11 op/s 2023-09-10T06:08:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:08:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:08:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:08:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:00 smithi170 ceph-mon[102832]: pgmap v13232: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 198 KiB/s wr, 11 op/s 2023-09-10T06:08:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:08:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:08:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:08:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:00 smithi082 ceph-mon[98694]: pgmap v13232: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 198 KiB/s wr, 11 op/s 2023-09-10T06:08:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:02 smithi170 ceph-mon[102832]: pgmap v13233: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 375 KiB/s wr, 17 op/s 2023-09-10T06:08:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:02 smithi082 ceph-mon[98694]: pgmap v13233: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 375 KiB/s wr, 17 op/s 2023-09-10T06:08:03.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:02 smithi098 ceph-mon[103124]: pgmap v13233: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 375 KiB/s wr, 17 op/s 2023-09-10T06:08:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24514.444577 secs 2023-09-10T06:08:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24514.444577 secs 2023-09-10T06:08:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24514.444577 secs 2023-09-10T06:08:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24515.413493 secs 2023-09-10T06:08:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:04 smithi170 ceph-mon[102832]: pgmap v13234: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 339 KiB/s wr, 11 op/s 2023-09-10T06:08:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24516.067205 secs 2023-09-10T06:08:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24515.413493 secs 2023-09-10T06:08:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:04 smithi082 ceph-mon[98694]: pgmap v13234: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 339 KiB/s wr, 11 op/s 2023-09-10T06:08:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24516.067205 secs 2023-09-10T06:08:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24515.413493 secs 2023-09-10T06:08:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:04 smithi098 ceph-mon[103124]: pgmap v13234: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 339 KiB/s wr, 11 op/s 2023-09-10T06:08:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24516.067205 secs 2023-09-10T06:08:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:06 smithi098 ceph-mon[103124]: pgmap v13235: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 467 KiB/s wr, 31 op/s 2023-09-10T06:08:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:06 smithi170 ceph-mon[102832]: pgmap v13235: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 467 KiB/s wr, 31 op/s 2023-09-10T06:08:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:06 smithi082 ceph-mon[98694]: pgmap v13235: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 467 KiB/s wr, 31 op/s 2023-09-10T06:08:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:08 smithi170 ceph-mon[102832]: pgmap v13236: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 305 KiB/s wr, 26 op/s 2023-09-10T06:08:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24519.444668 secs 2023-09-10T06:08:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:08 smithi082 ceph-mon[98694]: pgmap v13236: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 305 KiB/s wr, 26 op/s 2023-09-10T06:08:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24519.444668 secs 2023-09-10T06:08:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:08 smithi098 ceph-mon[103124]: pgmap v13236: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 305 KiB/s wr, 26 op/s 2023-09-10T06:08:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24519.444668 secs 2023-09-10T06:08:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24520.413553 secs 2023-09-10T06:08:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:09 smithi170 ceph-mon[102832]: osdmap e895: 12 total, 12 up, 12 in 2023-09-10T06:08:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24521.067270 secs 2023-09-10T06:08:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:09 smithi170 ceph-mon[102832]: pgmap v13238: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 366 KiB/s wr, 31 op/s 2023-09-10T06:08:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24520.413553 secs 2023-09-10T06:08:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:09 smithi082 ceph-mon[98694]: osdmap e895: 12 total, 12 up, 12 in 2023-09-10T06:08:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24521.067270 secs 2023-09-10T06:08:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:09 smithi082 ceph-mon[98694]: pgmap v13238: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 366 KiB/s wr, 31 op/s 2023-09-10T06:08:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24520.413553 secs 2023-09-10T06:08:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:09 smithi098 ceph-mon[103124]: osdmap e895: 12 total, 12 up, 12 in 2023-09-10T06:08:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24521.067270 secs 2023-09-10T06:08:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:09 smithi098 ceph-mon[103124]: pgmap v13238: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 366 KiB/s wr, 31 op/s 2023-09-10T06:08:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:12 smithi170 ceph-mon[102832]: pgmap v13239: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 176 KiB/s wr, 31 op/s 2023-09-10T06:08:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:12 smithi082 ceph-mon[98694]: pgmap v13239: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 176 KiB/s wr, 31 op/s 2023-09-10T06:08:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:12 smithi098 ceph-mon[103124]: pgmap v13239: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 176 KiB/s wr, 31 op/s 2023-09-10T06:08:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24524.444758 secs 2023-09-10T06:08:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24524.444758 secs 2023-09-10T06:08:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24524.444758 secs 2023-09-10T06:08:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24525.413610 secs 2023-09-10T06:08:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:14 smithi170 ceph-mon[102832]: pgmap v13240: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 176 KiB/s wr, 31 op/s 2023-09-10T06:08:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24526.067337 secs 2023-09-10T06:08:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24525.413610 secs 2023-09-10T06:08:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:14 smithi082 ceph-mon[98694]: pgmap v13240: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 176 KiB/s wr, 31 op/s 2023-09-10T06:08:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24526.067337 secs 2023-09-10T06:08:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24525.413610 secs 2023-09-10T06:08:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:14 smithi098 ceph-mon[103124]: pgmap v13240: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 176 KiB/s wr, 31 op/s 2023-09-10T06:08:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24526.067337 secs 2023-09-10T06:08:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:16 smithi098 ceph-mon[103124]: osdmap e896: 12 total, 12 up, 12 in 2023-09-10T06:08:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:16 smithi098 ceph-mon[103124]: pgmap v13242: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 16 op/s 2023-09-10T06:08:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:08:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:08:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:16 smithi170 ceph-mon[102832]: osdmap e896: 12 total, 12 up, 12 in 2023-09-10T06:08:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:16 smithi170 ceph-mon[102832]: pgmap v13242: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 16 op/s 2023-09-10T06:08:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:08:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:08:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:16 smithi082 ceph-mon[98694]: osdmap e896: 12 total, 12 up, 12 in 2023-09-10T06:08:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:16 smithi082 ceph-mon[98694]: pgmap v13242: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 16 op/s 2023-09-10T06:08:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:08:16.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:08:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:08:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:08:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:18 smithi098 ceph-mon[103124]: pgmap v13243: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 14 op/s 2023-09-10T06:08:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24529.444778 secs 2023-09-10T06:08:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:08:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:08:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:18 smithi170 ceph-mon[102832]: pgmap v13243: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 14 op/s 2023-09-10T06:08:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24529.444778 secs 2023-09-10T06:08:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:08:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:08:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:18 smithi082 ceph-mon[98694]: pgmap v13243: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 14 op/s 2023-09-10T06:08:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24529.444778 secs 2023-09-10T06:08:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24530.413668 secs 2023-09-10T06:08:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24531.067401 secs 2023-09-10T06:08:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24530.413668 secs 2023-09-10T06:08:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24531.067401 secs 2023-09-10T06:08:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24530.413668 secs 2023-09-10T06:08:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24531.067401 secs 2023-09-10T06:08:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:20 smithi098 ceph-mon[103124]: pgmap v13244: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T06:08:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:20 smithi170 ceph-mon[102832]: pgmap v13244: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T06:08:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:20 smithi082 ceph-mon[98694]: pgmap v13244: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-09-10T06:08:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:22 smithi170 ceph-mon[102832]: pgmap v13245: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T06:08:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:22 smithi082 ceph-mon[98694]: pgmap v13245: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T06:08:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:22 smithi098 ceph-mon[103124]: pgmap v13245: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T06:08:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24534.444835 secs 2023-09-10T06:08:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24534.444835 secs 2023-09-10T06:08:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24534.444835 secs 2023-09-10T06:08:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24535.413729 secs 2023-09-10T06:08:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:24 smithi170 ceph-mon[102832]: pgmap v13246: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T06:08:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24536.067465 secs 2023-09-10T06:08:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24535.413729 secs 2023-09-10T06:08:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:24 smithi082 ceph-mon[98694]: pgmap v13246: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T06:08:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24536.067465 secs 2023-09-10T06:08:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24535.413729 secs 2023-09-10T06:08:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:24 smithi098 ceph-mon[103124]: pgmap v13246: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T06:08:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24536.067465 secs 2023-09-10T06:08:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:26 smithi098 ceph-mon[103124]: pgmap v13247: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T06:08:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:26 smithi170 ceph-mon[102832]: pgmap v13247: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T06:08:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:26 smithi082 ceph-mon[98694]: pgmap v13247: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T06:08:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:28 smithi170 ceph-mon[102832]: pgmap v13248: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T06:08:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24539.444925 secs 2023-09-10T06:08:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:28 smithi082 ceph-mon[98694]: pgmap v13248: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T06:08:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24539.444925 secs 2023-09-10T06:08:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:28 smithi098 ceph-mon[103124]: pgmap v13248: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T06:08:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24539.444925 secs 2023-09-10T06:08:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24540.413790 secs 2023-09-10T06:08:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24541.067490 secs 2023-09-10T06:08:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:29 smithi170 ceph-mon[102832]: pgmap v13249: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T06:08:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24540.413790 secs 2023-09-10T06:08:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24541.067490 secs 2023-09-10T06:08:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:29 smithi082 ceph-mon[98694]: pgmap v13249: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T06:08:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24540.413790 secs 2023-09-10T06:08:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24541.067490 secs 2023-09-10T06:08:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:29 smithi098 ceph-mon[103124]: pgmap v13249: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T06:08:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:08:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:08:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:08:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:32 smithi098 ceph-mon[103124]: pgmap v13250: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 168 KiB/s wr, 15 op/s 2023-09-10T06:08:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:08:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:08:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:08:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:32 smithi170 ceph-mon[102832]: pgmap v13250: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 168 KiB/s wr, 15 op/s 2023-09-10T06:08:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:08:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:08:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:08:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:32 smithi082 ceph-mon[98694]: pgmap v13250: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 168 KiB/s wr, 15 op/s 2023-09-10T06:08:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:08:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:08:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:08:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:33 smithi082 ceph-mon[98694]: pgmap v13251: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 9 op/s 2023-09-10T06:08:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:33 smithi098 ceph-mon[103124]: pgmap v13251: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 9 op/s 2023-09-10T06:08:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:33 smithi170 ceph-mon[102832]: pgmap v13251: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 9 op/s 2023-09-10T06:08:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24545.413851 secs 2023-09-10T06:08:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24546.067557 secs 2023-09-10T06:08:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24545.413851 secs 2023-09-10T06:08:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24546.067557 secs 2023-09-10T06:08:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24545.413851 secs 2023-09-10T06:08:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24546.067557 secs 2023-09-10T06:08:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:08:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:35 smithi082 ceph-mon[98694]: pgmap v13252: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 366 KiB/s wr, 17 op/s 2023-09-10T06:08:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:08:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:35 smithi098 ceph-mon[103124]: pgmap v13252: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 366 KiB/s wr, 17 op/s 2023-09-10T06:08:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:08:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:35 smithi170 ceph-mon[102832]: pgmap v13252: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 366 KiB/s wr, 17 op/s 2023-09-10T06:08:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:08:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:08:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:08:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:08:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:08:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:08:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:08:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:08:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:08:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:08:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:08:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:38 smithi098 ceph-mon[103124]: pgmap v13253: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 365 KiB/s wr, 12 op/s 2023-09-10T06:08:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:08:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:08:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:38 smithi170 ceph-mon[102832]: pgmap v13253: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 365 KiB/s wr, 12 op/s 2023-09-10T06:08:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:08:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:08:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:38 smithi082 ceph-mon[98694]: pgmap v13253: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 365 KiB/s wr, 12 op/s 2023-09-10T06:08:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24550.413908 secs 2023-09-10T06:08:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24551.067621 secs 2023-09-10T06:08:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24550.413908 secs 2023-09-10T06:08:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24551.067621 secs 2023-09-10T06:08:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24550.413908 secs 2023-09-10T06:08:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24551.067621 secs 2023-09-10T06:08:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:40 smithi098 ceph-mon[103124]: pgmap v13254: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 365 KiB/s wr, 12 op/s 2023-09-10T06:08:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:40 smithi170 ceph-mon[102832]: pgmap v13254: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 365 KiB/s wr, 12 op/s 2023-09-10T06:08:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:40 smithi082 ceph-mon[98694]: pgmap v13254: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 365 KiB/s wr, 12 op/s 2023-09-10T06:08:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:41 smithi082 ceph-mon[98694]: pgmap v13255: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 382 KiB/s wr, 17 op/s 2023-09-10T06:08:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:41 smithi098 ceph-mon[103124]: pgmap v13255: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 382 KiB/s wr, 17 op/s 2023-09-10T06:08:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:41 smithi170 ceph-mon[102832]: pgmap v13255: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 382 KiB/s wr, 17 op/s 2023-09-10T06:08:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:08:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:08:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24555.413977 secs 2023-09-10T06:08:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:43 smithi098 ceph-mon[103124]: pgmap v13256: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 298 KiB/s wr, 12 op/s 2023-09-10T06:08:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:08:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:08:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24555.413977 secs 2023-09-10T06:08:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:43 smithi170 ceph-mon[102832]: pgmap v13256: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 298 KiB/s wr, 12 op/s 2023-09-10T06:08:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:08:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:08:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24555.413977 secs 2023-09-10T06:08:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:43 smithi082 ceph-mon[98694]: pgmap v13256: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 298 KiB/s wr, 12 op/s 2023-09-10T06:08:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24556.067686 secs 2023-09-10T06:08:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24556.067686 secs 2023-09-10T06:08:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24556.067686 secs 2023-09-10T06:08:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:08:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:08:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:46 smithi098 ceph-mon[103124]: pgmap v13257: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 747 KiB/s wr, 19 op/s 2023-09-10T06:08:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:08:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:08:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:46 smithi170 ceph-mon[102832]: pgmap v13257: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 747 KiB/s wr, 19 op/s 2023-09-10T06:08:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:08:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:08:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:45 smithi082 ceph-mon[98694]: pgmap v13257: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 747 KiB/s wr, 19 op/s 2023-09-10T06:08:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:47 smithi082 ceph-mon[98694]: pgmap v13258: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 466 KiB/s wr, 12 op/s 2023-09-10T06:08:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:47 smithi098 ceph-mon[103124]: pgmap v13258: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 466 KiB/s wr, 12 op/s 2023-09-10T06:08:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:47 smithi170 ceph-mon[102832]: pgmap v13258: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 466 KiB/s wr, 12 op/s 2023-09-10T06:08:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24560.414064 secs 2023-09-10T06:08:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24560.414064 secs 2023-09-10T06:08:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24560.414064 secs 2023-09-10T06:08:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24561.067750 secs 2023-09-10T06:08:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:49 smithi170 ceph-mon[102832]: pgmap v13259: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 466 KiB/s wr, 12 op/s 2023-09-10T06:08:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24561.067750 secs 2023-09-10T06:08:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:49 smithi082 ceph-mon[98694]: pgmap v13259: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 466 KiB/s wr, 12 op/s 2023-09-10T06:08:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24561.067750 secs 2023-09-10T06:08:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:49 smithi098 ceph-mon[103124]: pgmap v13259: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 466 KiB/s wr, 12 op/s 2023-09-10T06:08:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:51 smithi082 ceph-mon[98694]: pgmap v13260: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 501 KiB/s wr, 18 op/s 2023-09-10T06:08:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:51 smithi098 ceph-mon[103124]: pgmap v13260: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 501 KiB/s wr, 18 op/s 2023-09-10T06:08:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:51 smithi170 ceph-mon[102832]: pgmap v13260: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 501 KiB/s wr, 18 op/s 2023-09-10T06:08:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:53 smithi082 ceph-mon[98694]: pgmap v13261: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 484 KiB/s wr, 12 op/s 2023-09-10T06:08:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:53 smithi098 ceph-mon[103124]: pgmap v13261: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 484 KiB/s wr, 12 op/s 2023-09-10T06:08:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:53 smithi170 ceph-mon[102832]: pgmap v13261: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 484 KiB/s wr, 12 op/s 2023-09-10T06:08:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24565.414124 secs 2023-09-10T06:08:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24566.067813 secs 2023-09-10T06:08:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24565.414124 secs 2023-09-10T06:08:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24566.067813 secs 2023-09-10T06:08:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24565.414124 secs 2023-09-10T06:08:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24566.067813 secs 2023-09-10T06:08:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:56 smithi098 ceph-mon[103124]: pgmap v13262: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 647 KiB/s wr, 19 op/s 2023-09-10T06:08:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:56 smithi170 ceph-mon[102832]: pgmap v13262: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 647 KiB/s wr, 19 op/s 2023-09-10T06:08:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:56 smithi082 ceph-mon[98694]: pgmap v13262: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 647 KiB/s wr, 19 op/s 2023-09-10T06:08:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:08:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:08:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:58 smithi098 ceph-mon[103124]: pgmap v13263: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 12 op/s 2023-09-10T06:08:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24569.445379 secs 2023-09-10T06:08:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:08:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:08:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:58 smithi170 ceph-mon[102832]: pgmap v13263: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 12 op/s 2023-09-10T06:08:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24569.445379 secs 2023-09-10T06:08:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:08:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:08:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:58 smithi082 ceph-mon[98694]: pgmap v13263: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 12 op/s 2023-09-10T06:08:58.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24569.445379 secs 2023-09-10T06:08:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24570.414192 secs 2023-09-10T06:08:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:08:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24571.067881 secs 2023-09-10T06:08:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24570.414192 secs 2023-09-10T06:08:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:08:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24571.067881 secs 2023-09-10T06:08:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24570.414192 secs 2023-09-10T06:08:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:08:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24571.067881 secs 2023-09-10T06:09:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:09:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:09:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:09:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:00 smithi098 ceph-mon[103124]: pgmap v13264: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 12 op/s 2023-09-10T06:09:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:09:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:09:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:09:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:00 smithi170 ceph-mon[102832]: pgmap v13264: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 12 op/s 2023-09-10T06:09:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:09:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:09:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:09:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:00 smithi082 ceph-mon[98694]: pgmap v13264: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 12 op/s 2023-09-10T06:09:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:01 smithi082 ceph-mon[98694]: pgmap v13265: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 377 KiB/s wr, 17 op/s 2023-09-10T06:09:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:01 smithi098 ceph-mon[103124]: pgmap v13265: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 377 KiB/s wr, 17 op/s 2023-09-10T06:09:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:01 smithi170 ceph-mon[102832]: pgmap v13265: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 377 KiB/s wr, 17 op/s 2023-09-10T06:09:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24574.445439 secs 2023-09-10T06:09:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24574.445439 secs 2023-09-10T06:09:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24574.445439 secs 2023-09-10T06:09:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24575.414245 secs 2023-09-10T06:09:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:04 smithi098 ceph-mon[103124]: pgmap v13266: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 341 KiB/s wr, 11 op/s 2023-09-10T06:09:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24575.414245 secs 2023-09-10T06:09:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:04 smithi170 ceph-mon[102832]: pgmap v13266: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 341 KiB/s wr, 11 op/s 2023-09-10T06:09:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24575.414245 secs 2023-09-10T06:09:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:04 smithi082 ceph-mon[98694]: pgmap v13266: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 341 KiB/s wr, 11 op/s 2023-09-10T06:09:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24576.067945 secs 2023-09-10T06:09:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24576.067945 secs 2023-09-10T06:09:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24576.067945 secs 2023-09-10T06:09:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:06 smithi098 ceph-mon[103124]: pgmap v13267: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 469 KiB/s wr, 31 op/s 2023-09-10T06:09:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:06 smithi170 ceph-mon[102832]: pgmap v13267: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 469 KiB/s wr, 31 op/s 2023-09-10T06:09:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:06 smithi082 ceph-mon[98694]: pgmap v13267: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 469 KiB/s wr, 31 op/s 2023-09-10T06:09:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:07 smithi082 ceph-mon[98694]: pgmap v13268: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 306 KiB/s wr, 25 op/s 2023-09-10T06:09:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:07 smithi098 ceph-mon[103124]: pgmap v13268: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 306 KiB/s wr, 25 op/s 2023-09-10T06:09:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:07 smithi170 ceph-mon[102832]: pgmap v13268: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 306 KiB/s wr, 25 op/s 2023-09-10T06:09:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24579.445500 secs 2023-09-10T06:09:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24579.445500 secs 2023-09-10T06:09:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24579.445500 secs 2023-09-10T06:09:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24580.414304 secs 2023-09-10T06:09:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:09 smithi082 ceph-mon[98694]: osdmap e897: 12 total, 12 up, 12 in 2023-09-10T06:09:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24581.068012 secs 2023-09-10T06:09:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:09 smithi082 ceph-mon[98694]: pgmap v13270: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 367 KiB/s wr, 30 op/s 2023-09-10T06:09:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24580.414304 secs 2023-09-10T06:09:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:09 smithi098 ceph-mon[103124]: osdmap e897: 12 total, 12 up, 12 in 2023-09-10T06:09:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24581.068012 secs 2023-09-10T06:09:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:09 smithi098 ceph-mon[103124]: pgmap v13270: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 367 KiB/s wr, 30 op/s 2023-09-10T06:09:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24580.414304 secs 2023-09-10T06:09:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:09 smithi170 ceph-mon[102832]: osdmap e897: 12 total, 12 up, 12 in 2023-09-10T06:09:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24581.068012 secs 2023-09-10T06:09:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:09 smithi170 ceph-mon[102832]: pgmap v13270: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 367 KiB/s wr, 30 op/s 2023-09-10T06:09:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:11 smithi082 ceph-mon[98694]: pgmap v13271: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 176 KiB/s wr, 29 op/s 2023-09-10T06:09:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:11 smithi098 ceph-mon[103124]: pgmap v13271: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 176 KiB/s wr, 29 op/s 2023-09-10T06:09:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:11 smithi170 ceph-mon[102832]: pgmap v13271: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 176 KiB/s wr, 29 op/s 2023-09-10T06:09:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24584.445563 secs 2023-09-10T06:09:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24584.445563 secs 2023-09-10T06:09:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24584.445563 secs 2023-09-10T06:09:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24585.414363 secs 2023-09-10T06:09:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:14 smithi098 ceph-mon[103124]: pgmap v13272: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 176 KiB/s wr, 29 op/s 2023-09-10T06:09:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24585.414363 secs 2023-09-10T06:09:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:14 smithi170 ceph-mon[102832]: pgmap v13272: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 176 KiB/s wr, 29 op/s 2023-09-10T06:09:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24585.414363 secs 2023-09-10T06:09:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:14 smithi082 ceph-mon[98694]: pgmap v13272: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 176 KiB/s wr, 29 op/s 2023-09-10T06:09:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24586.068076 secs 2023-09-10T06:09:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24586.068076 secs 2023-09-10T06:09:15.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24586.068076 secs 2023-09-10T06:09:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:16 smithi098 ceph-mon[103124]: osdmap e898: 12 total, 12 up, 12 in 2023-09-10T06:09:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:16 smithi098 ceph-mon[103124]: pgmap v13274: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 14 op/s 2023-09-10T06:09:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:16 smithi170 ceph-mon[102832]: osdmap e898: 12 total, 12 up, 12 in 2023-09-10T06:09:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:16 smithi170 ceph-mon[102832]: pgmap v13274: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 14 op/s 2023-09-10T06:09:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:16 smithi082 ceph-mon[98694]: osdmap e898: 12 total, 12 up, 12 in 2023-09-10T06:09:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:16 smithi082 ceph-mon[98694]: pgmap v13274: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 14 op/s 2023-09-10T06:09:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:17 smithi082 ceph-mon[98694]: pgmap v13275: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T06:09:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:17 smithi098 ceph-mon[103124]: pgmap v13275: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T06:09:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:17 smithi170 ceph-mon[102832]: pgmap v13275: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T06:09:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24589.445642 secs 2023-09-10T06:09:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24589.445642 secs 2023-09-10T06:09:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24589.445642 secs 2023-09-10T06:09:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24590.414422 secs 2023-09-10T06:09:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24591.068140 secs 2023-09-10T06:09:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:19 smithi082 ceph-mon[98694]: pgmap v13276: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T06:09:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24590.414422 secs 2023-09-10T06:09:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24591.068140 secs 2023-09-10T06:09:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:19 smithi098 ceph-mon[103124]: pgmap v13276: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T06:09:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24590.414422 secs 2023-09-10T06:09:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24591.068140 secs 2023-09-10T06:09:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:19 smithi170 ceph-mon[102832]: pgmap v13276: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T06:09:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:21 smithi082 ceph-mon[98694]: pgmap v13277: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T06:09:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:21 smithi098 ceph-mon[103124]: pgmap v13277: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T06:09:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:21 smithi170 ceph-mon[102832]: pgmap v13277: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T06:09:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24594.445701 secs 2023-09-10T06:09:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24594.445701 secs 2023-09-10T06:09:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24594.445701 secs 2023-09-10T06:09:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24595.414505 secs 2023-09-10T06:09:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:24 smithi098 ceph-mon[103124]: pgmap v13278: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T06:09:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24595.414505 secs 2023-09-10T06:09:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:24 smithi170 ceph-mon[102832]: pgmap v13278: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T06:09:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24595.414505 secs 2023-09-10T06:09:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:24 smithi082 ceph-mon[98694]: pgmap v13278: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T06:09:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24596.068205 secs 2023-09-10T06:09:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24596.068205 secs 2023-09-10T06:09:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24596.068205 secs 2023-09-10T06:09:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:26 smithi098 ceph-mon[103124]: pgmap v13279: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T06:09:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:26 smithi170 ceph-mon[102832]: pgmap v13279: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T06:09:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:26 smithi082 ceph-mon[98694]: pgmap v13279: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T06:09:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:27 smithi082 ceph-mon[98694]: pgmap v13280: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T06:09:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:27 smithi098 ceph-mon[103124]: pgmap v13280: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T06:09:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:27 smithi170 ceph-mon[102832]: pgmap v13280: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T06:09:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24600.414563 secs 2023-09-10T06:09:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:29 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:09:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:29 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:09:29.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24601.068271 secs 2023-09-10T06:09:29.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:29 smithi082 ceph-mon[98694]: pgmap v13281: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T06:09:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24600.414563 secs 2023-09-10T06:09:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:29 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:09:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:29 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:09:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24601.068271 secs 2023-09-10T06:09:30.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:29 smithi098 ceph-mon[103124]: pgmap v13281: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T06:09:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24600.414563 secs 2023-09-10T06:09:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:29 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:09:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:29 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:09:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24601.068271 secs 2023-09-10T06:09:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:29 smithi170 ceph-mon[102832]: pgmap v13281: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 84 KiB/s wr, 10 op/s 2023-09-10T06:09:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:09:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:09:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:09:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:09:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:09:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:09:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:32 smithi098 ceph-mon[103124]: pgmap v13282: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 168 KiB/s wr, 15 op/s 2023-09-10T06:09:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:09:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:09:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:09:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:09:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:32 smithi170 ceph-mon[102832]: pgmap v13282: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 168 KiB/s wr, 15 op/s 2023-09-10T06:09:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:09:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:09:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:09:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:09:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:32 smithi082 ceph-mon[98694]: pgmap v13282: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 168 KiB/s wr, 15 op/s 2023-09-10T06:09:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:09:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:09:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:09:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:09:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24604.445777 secs 2023-09-10T06:09:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:09:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:09:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:09:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24604.445777 secs 2023-09-10T06:09:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:09:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:09:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:09:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24604.445777 secs 2023-09-10T06:09:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24605.414624 secs 2023-09-10T06:09:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:34 smithi098 ceph-mon[103124]: pgmap v13283: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T06:09:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24606.068335 secs 2023-09-10T06:09:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24605.414624 secs 2023-09-10T06:09:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:34 smithi170 ceph-mon[102832]: pgmap v13283: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T06:09:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24606.068335 secs 2023-09-10T06:09:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24605.414624 secs 2023-09-10T06:09:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:34 smithi082 ceph-mon[98694]: pgmap v13283: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T06:09:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24606.068335 secs 2023-09-10T06:09:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:09:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:09:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:36 smithi170 ceph-mon[102832]: pgmap v13284: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 367 KiB/s wr, 17 op/s 2023-09-10T06:09:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:09:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:09:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:36 smithi082 ceph-mon[98694]: pgmap v13284: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 367 KiB/s wr, 17 op/s 2023-09-10T06:09:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:09:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:09:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:36 smithi098 ceph-mon[103124]: pgmap v13284: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 367 KiB/s wr, 17 op/s 2023-09-10T06:09:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:09:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:09:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:38 smithi170 ceph-mon[102832]: pgmap v13285: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 367 KiB/s wr, 12 op/s 2023-09-10T06:09:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24609.445866 secs 2023-09-10T06:09:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:09:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:09:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:38 smithi082 ceph-mon[98694]: pgmap v13285: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 367 KiB/s wr, 12 op/s 2023-09-10T06:09:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24609.445866 secs 2023-09-10T06:09:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:09:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:09:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:38 smithi098 ceph-mon[103124]: pgmap v13285: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 367 KiB/s wr, 12 op/s 2023-09-10T06:09:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24609.445866 secs 2023-09-10T06:09:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24610.414708 secs 2023-09-10T06:09:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24611.068402 secs 2023-09-10T06:09:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24610.414708 secs 2023-09-10T06:09:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24611.068402 secs 2023-09-10T06:09:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24610.414708 secs 2023-09-10T06:09:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24611.068402 secs 2023-09-10T06:09:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:09:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:09:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:40 smithi170 ceph-mon[102832]: pgmap v13286: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 367 KiB/s wr, 12 op/s 2023-09-10T06:09:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:09:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:09:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:09:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:40 smithi082 ceph-mon[98694]: pgmap v13286: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 367 KiB/s wr, 12 op/s 2023-09-10T06:09:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:09:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:09:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:09:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:40 smithi098 ceph-mon[103124]: pgmap v13286: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 367 KiB/s wr, 12 op/s 2023-09-10T06:09:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:09:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:41 smithi082 ceph-mon[98694]: pgmap v13287: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 384 KiB/s wr, 18 op/s 2023-09-10T06:09:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:41 smithi098 ceph-mon[103124]: pgmap v13287: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 384 KiB/s wr, 18 op/s 2023-09-10T06:09:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:41 smithi170 ceph-mon[102832]: pgmap v13287: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 384 KiB/s wr, 18 op/s 2023-09-10T06:09:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:42 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:09:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:42 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:09:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:42 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:09:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:09:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:09:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24615.414769 secs 2023-09-10T06:09:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:43 smithi098 ceph-mon[103124]: pgmap v13288: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 300 KiB/s wr, 12 op/s 2023-09-10T06:09:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:09:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:09:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24615.414769 secs 2023-09-10T06:09:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:43 smithi170 ceph-mon[102832]: pgmap v13288: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 300 KiB/s wr, 12 op/s 2023-09-10T06:09:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:09:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:09:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24615.414769 secs 2023-09-10T06:09:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:43 smithi082 ceph-mon[98694]: pgmap v13288: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 300 KiB/s wr, 12 op/s 2023-09-10T06:09:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24616.068464 secs 2023-09-10T06:09:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24616.068464 secs 2023-09-10T06:09:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24616.068464 secs 2023-09-10T06:09:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:09:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:09:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:46 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:09:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:46 smithi170 ceph-mon[102832]: pgmap v13289: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 751 KiB/s wr, 19 op/s 2023-09-10T06:09:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:09:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:09:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:46 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:09:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:46 smithi082 ceph-mon[98694]: pgmap v13289: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 751 KiB/s wr, 19 op/s 2023-09-10T06:09:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:09:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:09:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:46 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:09:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:46 smithi098 ceph-mon[103124]: pgmap v13289: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 751 KiB/s wr, 19 op/s 2023-09-10T06:09:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:47 smithi082 ceph-mon[98694]: pgmap v13290: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 469 KiB/s wr, 12 op/s 2023-09-10T06:09:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:47 smithi098 ceph-mon[103124]: pgmap v13290: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 469 KiB/s wr, 12 op/s 2023-09-10T06:09:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:47 smithi170 ceph-mon[102832]: pgmap v13290: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 469 KiB/s wr, 12 op/s 2023-09-10T06:09:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24620.414826 secs 2023-09-10T06:09:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24620.414826 secs 2023-09-10T06:09:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24620.414826 secs 2023-09-10T06:09:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24621.068488 secs 2023-09-10T06:09:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:49 smithi170 ceph-mon[102832]: pgmap v13291: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 469 KiB/s wr, 12 op/s 2023-09-10T06:09:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24621.068488 secs 2023-09-10T06:09:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:49 smithi082 ceph-mon[98694]: pgmap v13291: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 469 KiB/s wr, 12 op/s 2023-09-10T06:09:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24621.068488 secs 2023-09-10T06:09:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:49 smithi098 ceph-mon[103124]: pgmap v13291: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 469 KiB/s wr, 12 op/s 2023-09-10T06:09:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:51 smithi082 ceph-mon[98694]: pgmap v13292: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 504 KiB/s wr, 17 op/s 2023-09-10T06:09:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:51 smithi098 ceph-mon[103124]: pgmap v13292: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 504 KiB/s wr, 17 op/s 2023-09-10T06:09:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:51 smithi170 ceph-mon[102832]: pgmap v13292: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 504 KiB/s wr, 17 op/s 2023-09-10T06:09:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:53 smithi082 ceph-mon[98694]: pgmap v13293: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 486 KiB/s wr, 11 op/s 2023-09-10T06:09:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:53 smithi098 ceph-mon[103124]: pgmap v13293: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 486 KiB/s wr, 11 op/s 2023-09-10T06:09:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:53 smithi170 ceph-mon[102832]: pgmap v13293: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 486 KiB/s wr, 11 op/s 2023-09-10T06:09:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24625.414888 secs 2023-09-10T06:09:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24626.068564 secs 2023-09-10T06:09:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24625.414888 secs 2023-09-10T06:09:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24626.068564 secs 2023-09-10T06:09:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24625.414888 secs 2023-09-10T06:09:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24626.068564 secs 2023-09-10T06:09:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:56 smithi098 ceph-mon[103124]: pgmap v13294: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 650 KiB/s wr, 16 op/s 2023-09-10T06:09:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:56 smithi170 ceph-mon[102832]: pgmap v13294: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 650 KiB/s wr, 16 op/s 2023-09-10T06:09:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:56 smithi082 ceph-mon[98694]: pgmap v13294: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 650 KiB/s wr, 16 op/s 2023-09-10T06:09:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:09:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:09:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:58 smithi098 ceph-mon[103124]: pgmap v13295: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 199 KiB/s wr, 10 op/s 2023-09-10T06:09:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24629.446153 secs 2023-09-10T06:09:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:09:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:09:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:58 smithi170 ceph-mon[102832]: pgmap v13295: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 199 KiB/s wr, 10 op/s 2023-09-10T06:09:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24629.446153 secs 2023-09-10T06:09:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:09:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:09:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:58 smithi082 ceph-mon[98694]: pgmap v13295: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 199 KiB/s wr, 10 op/s 2023-09-10T06:09:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24629.446153 secs 2023-09-10T06:09:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24630.414947 secs 2023-09-10T06:09:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:09:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24631.068618 secs 2023-09-10T06:09:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24630.414947 secs 2023-09-10T06:09:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:09:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24631.068618 secs 2023-09-10T06:09:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24630.414947 secs 2023-09-10T06:09:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:09:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24631.068618 secs 2023-09-10T06:10:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:10:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:10:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:10:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:00 smithi098 ceph-mon[103124]: pgmap v13296: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 199 KiB/s wr, 10 op/s 2023-09-10T06:10:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T06:10:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:10:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:10:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:10:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:00 smithi170 ceph-mon[102832]: pgmap v13296: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 199 KiB/s wr, 10 op/s 2023-09-10T06:10:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T06:10:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:10:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:10:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:10:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:00 smithi082 ceph-mon[98694]: pgmap v13296: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 199 KiB/s wr, 10 op/s 2023-09-10T06:10:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T06:10:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:01 smithi082 ceph-mon[98694]: pgmap v13297: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 378 KiB/s wr, 16 op/s 2023-09-10T06:10:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:01 smithi098 ceph-mon[103124]: pgmap v13297: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 378 KiB/s wr, 16 op/s 2023-09-10T06:10:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:01 smithi170 ceph-mon[102832]: pgmap v13297: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 378 KiB/s wr, 16 op/s 2023-09-10T06:10:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24634.446210 secs 2023-09-10T06:10:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24634.446210 secs 2023-09-10T06:10:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24634.446210 secs 2023-09-10T06:10:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24635.415007 secs 2023-09-10T06:10:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:04 smithi098 ceph-mon[103124]: pgmap v13298: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 343 KiB/s wr, 10 op/s 2023-09-10T06:10:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24635.415007 secs 2023-09-10T06:10:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:04 smithi170 ceph-mon[102832]: pgmap v13298: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 343 KiB/s wr, 10 op/s 2023-09-10T06:10:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24635.415007 secs 2023-09-10T06:10:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:04 smithi082 ceph-mon[98694]: pgmap v13298: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 343 KiB/s wr, 10 op/s 2023-09-10T06:10:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24636.068680 secs 2023-09-10T06:10:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24636.068680 secs 2023-09-10T06:10:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24636.068680 secs 2023-09-10T06:10:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:06 smithi098 ceph-mon[103124]: pgmap v13299: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 471 KiB/s wr, 30 op/s 2023-09-10T06:10:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:06 smithi170 ceph-mon[102832]: pgmap v13299: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 471 KiB/s wr, 30 op/s 2023-09-10T06:10:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:06 smithi082 ceph-mon[98694]: pgmap v13299: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 471 KiB/s wr, 30 op/s 2023-09-10T06:10:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:07 smithi082 ceph-mon[98694]: pgmap v13300: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 307 KiB/s wr, 25 op/s 2023-09-10T06:10:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:07 smithi098 ceph-mon[103124]: pgmap v13300: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 307 KiB/s wr, 25 op/s 2023-09-10T06:10:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:07 smithi170 ceph-mon[102832]: pgmap v13300: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 307 KiB/s wr, 25 op/s 2023-09-10T06:10:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24639.446287 secs 2023-09-10T06:10:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24639.446287 secs 2023-09-10T06:10:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24639.446287 secs 2023-09-10T06:10:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24640.415070 secs 2023-09-10T06:10:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:09 smithi098 ceph-mon[103124]: osdmap e899: 12 total, 12 up, 12 in 2023-09-10T06:10:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24641.068745 secs 2023-09-10T06:10:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:09 smithi098 ceph-mon[103124]: pgmap v13302: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 368 KiB/s wr, 30 op/s 2023-09-10T06:10:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24640.415070 secs 2023-09-10T06:10:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:09 smithi170 ceph-mon[102832]: osdmap e899: 12 total, 12 up, 12 in 2023-09-10T06:10:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24641.068745 secs 2023-09-10T06:10:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:09 smithi170 ceph-mon[102832]: pgmap v13302: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 368 KiB/s wr, 30 op/s 2023-09-10T06:10:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24640.415070 secs 2023-09-10T06:10:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:09 smithi082 ceph-mon[98694]: osdmap e899: 12 total, 12 up, 12 in 2023-09-10T06:10:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24641.068745 secs 2023-09-10T06:10:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:09 smithi082 ceph-mon[98694]: pgmap v13302: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 368 KiB/s wr, 30 op/s 2023-09-10T06:10:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:11 smithi082 ceph-mon[98694]: pgmap v13303: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 176 KiB/s wr, 29 op/s 2023-09-10T06:10:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:11 smithi098 ceph-mon[103124]: pgmap v13303: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 176 KiB/s wr, 29 op/s 2023-09-10T06:10:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:11 smithi170 ceph-mon[102832]: pgmap v13303: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 176 KiB/s wr, 29 op/s 2023-09-10T06:10:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24644.446368 secs 2023-09-10T06:10:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24644.446368 secs 2023-09-10T06:10:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24644.446368 secs 2023-09-10T06:10:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24645.415129 secs 2023-09-10T06:10:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:14 smithi098 ceph-mon[103124]: pgmap v13304: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 176 KiB/s wr, 29 op/s 2023-09-10T06:10:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24645.415129 secs 2023-09-10T06:10:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:14 smithi170 ceph-mon[102832]: pgmap v13304: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 176 KiB/s wr, 29 op/s 2023-09-10T06:10:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24645.415129 secs 2023-09-10T06:10:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:14 smithi082 ceph-mon[98694]: pgmap v13304: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 176 KiB/s wr, 29 op/s 2023-09-10T06:10:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24646.068810 secs 2023-09-10T06:10:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24646.068810 secs 2023-09-10T06:10:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24646.068810 secs 2023-09-10T06:10:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:16 smithi098 ceph-mon[103124]: osdmap e900: 12 total, 12 up, 12 in 2023-09-10T06:10:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:16 smithi098 ceph-mon[103124]: pgmap v13306: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 14 op/s 2023-09-10T06:10:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:16 smithi170 ceph-mon[102832]: osdmap e900: 12 total, 12 up, 12 in 2023-09-10T06:10:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:16 smithi170 ceph-mon[102832]: pgmap v13306: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 14 op/s 2023-09-10T06:10:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:16 smithi082 ceph-mon[98694]: osdmap e900: 12 total, 12 up, 12 in 2023-09-10T06:10:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:16 smithi082 ceph-mon[98694]: pgmap v13306: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 14 op/s 2023-09-10T06:10:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:17 smithi082 ceph-mon[98694]: pgmap v13307: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T06:10:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:17 smithi098 ceph-mon[103124]: pgmap v13307: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T06:10:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:17 smithi170 ceph-mon[102832]: pgmap v13307: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T06:10:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24649.446427 secs 2023-09-10T06:10:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24649.446427 secs 2023-09-10T06:10:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24649.446427 secs 2023-09-10T06:10:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24650.415188 secs 2023-09-10T06:10:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24651.068875 secs 2023-09-10T06:10:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:19 smithi098 ceph-mon[103124]: pgmap v13308: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T06:10:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24650.415188 secs 2023-09-10T06:10:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24651.068875 secs 2023-09-10T06:10:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:19 smithi170 ceph-mon[102832]: pgmap v13308: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T06:10:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24650.415188 secs 2023-09-10T06:10:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24651.068875 secs 2023-09-10T06:10:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:19 smithi082 ceph-mon[98694]: pgmap v13308: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2023-09-10T06:10:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:21 smithi082 ceph-mon[98694]: pgmap v13309: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 101 KiB/s wr, 11 op/s 2023-09-10T06:10:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:21 smithi098 ceph-mon[103124]: pgmap v13309: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 101 KiB/s wr, 11 op/s 2023-09-10T06:10:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:21 smithi170 ceph-mon[102832]: pgmap v13309: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 101 KiB/s wr, 11 op/s 2023-09-10T06:10:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24654.446519 secs 2023-09-10T06:10:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24654.446519 secs 2023-09-10T06:10:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24654.446519 secs 2023-09-10T06:10:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24655.415248 secs 2023-09-10T06:10:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:24 smithi098 ceph-mon[103124]: pgmap v13310: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 101 KiB/s wr, 11 op/s 2023-09-10T06:10:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24655.415248 secs 2023-09-10T06:10:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:24 smithi170 ceph-mon[102832]: pgmap v13310: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 101 KiB/s wr, 11 op/s 2023-09-10T06:10:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24655.415248 secs 2023-09-10T06:10:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:24 smithi082 ceph-mon[98694]: pgmap v13310: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 101 KiB/s wr, 11 op/s 2023-09-10T06:10:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24656.068937 secs 2023-09-10T06:10:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24656.068937 secs 2023-09-10T06:10:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24656.068937 secs 2023-09-10T06:10:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:26 smithi098 ceph-mon[103124]: pgmap v13311: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T06:10:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:26 smithi170 ceph-mon[102832]: pgmap v13311: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T06:10:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:26 smithi082 ceph-mon[98694]: pgmap v13311: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T06:10:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:27 smithi082 ceph-mon[98694]: pgmap v13312: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 9 op/s 2023-09-10T06:10:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:27 smithi098 ceph-mon[103124]: pgmap v13312: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 9 op/s 2023-09-10T06:10:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:27 smithi170 ceph-mon[102832]: pgmap v13312: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 9 op/s 2023-09-10T06:10:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24660.415305 secs 2023-09-10T06:10:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24660.415305 secs 2023-09-10T06:10:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24660.415305 secs 2023-09-10T06:10:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24661.069001 secs 2023-09-10T06:10:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:29 smithi170 ceph-mon[102832]: pgmap v13313: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 9 op/s 2023-09-10T06:10:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24661.069001 secs 2023-09-10T06:10:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:29 smithi082 ceph-mon[98694]: pgmap v13313: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 9 op/s 2023-09-10T06:10:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24661.069001 secs 2023-09-10T06:10:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:29 smithi098 ceph-mon[103124]: pgmap v13313: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 9 op/s 2023-09-10T06:10:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:10:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:10:31.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:10:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:10:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:10:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:10:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:32 smithi098 ceph-mon[103124]: pgmap v13314: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 169 KiB/s wr, 15 op/s 2023-09-10T06:10:32.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:10:32.912 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:10:32.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:10:32.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:32 smithi170 ceph-mon[102832]: pgmap v13314: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 169 KiB/s wr, 15 op/s 2023-09-10T06:10:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:10:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:10:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:10:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:32 smithi082 ceph-mon[98694]: pgmap v13314: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 169 KiB/s wr, 15 op/s 2023-09-10T06:10:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:10:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:10:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:10:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:10:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:10:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:10:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:10:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:10:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:10:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:10:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:10:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:10:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24665.415372 secs 2023-09-10T06:10:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:10:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:10:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:34 smithi170 ceph-mon[102832]: pgmap v13315: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T06:10:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24666.069066 secs 2023-09-10T06:10:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24665.415372 secs 2023-09-10T06:10:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:10:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:10:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:34 smithi082 ceph-mon[98694]: pgmap v13315: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T06:10:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24666.069066 secs 2023-09-10T06:10:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24665.415372 secs 2023-09-10T06:10:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:10:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:10:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:34 smithi098 ceph-mon[103124]: pgmap v13315: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T06:10:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24666.069066 secs 2023-09-10T06:10:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:36 smithi170 ceph-mon[102832]: pgmap v13316: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 368 KiB/s wr, 16 op/s 2023-09-10T06:10:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:36 smithi082 ceph-mon[98694]: pgmap v13316: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 368 KiB/s wr, 16 op/s 2023-09-10T06:10:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:36 smithi098 ceph-mon[103124]: pgmap v13316: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 368 KiB/s wr, 16 op/s 2023-09-10T06:10:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:37 smithi082 ceph-mon[98694]: pgmap v13317: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 368 KiB/s wr, 11 op/s 2023-09-10T06:10:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:37 smithi098 ceph-mon[103124]: pgmap v13317: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 368 KiB/s wr, 11 op/s 2023-09-10T06:10:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:37 smithi170 ceph-mon[102832]: pgmap v13317: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 368 KiB/s wr, 11 op/s 2023-09-10T06:10:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24669.446742 secs 2023-09-10T06:10:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24669.446742 secs 2023-09-10T06:10:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24669.446742 secs 2023-09-10T06:10:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24670.415431 secs 2023-09-10T06:10:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24671.069129 secs 2023-09-10T06:10:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:39 smithi098 ceph-mon[103124]: pgmap v13318: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 368 KiB/s wr, 11 op/s 2023-09-10T06:10:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24670.415431 secs 2023-09-10T06:10:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24671.069129 secs 2023-09-10T06:10:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:39 smithi170 ceph-mon[102832]: pgmap v13318: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 368 KiB/s wr, 11 op/s 2023-09-10T06:10:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24670.415431 secs 2023-09-10T06:10:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24671.069129 secs 2023-09-10T06:10:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:39 smithi082 ceph-mon[98694]: pgmap v13318: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 368 KiB/s wr, 11 op/s 2023-09-10T06:10:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:10:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:10:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:10:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:10:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:10:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:10:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:10:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:10:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:10:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:41 smithi098 ceph-mon[103124]: pgmap v13319: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 456 KiB/s wr, 17 op/s 2023-09-10T06:10:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:41 smithi170 ceph-mon[102832]: pgmap v13319: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 456 KiB/s wr, 17 op/s 2023-09-10T06:10:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:41 smithi082 ceph-mon[98694]: pgmap v13319: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 456 KiB/s wr, 17 op/s 2023-09-10T06:10:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:42 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:10:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:42 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:10:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:42 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:10:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:10:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:10:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24675.415491 secs 2023-09-10T06:10:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:43 smithi098 ceph-mon[103124]: pgmap v13320: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 371 KiB/s wr, 12 op/s 2023-09-10T06:10:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:10:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:10:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24675.415491 secs 2023-09-10T06:10:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:43 smithi170 ceph-mon[102832]: pgmap v13320: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 371 KiB/s wr, 12 op/s 2023-09-10T06:10:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:10:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:10:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24675.415491 secs 2023-09-10T06:10:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:43 smithi082 ceph-mon[98694]: pgmap v13320: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 371 KiB/s wr, 12 op/s 2023-09-10T06:10:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24676.069195 secs 2023-09-10T06:10:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24676.069195 secs 2023-09-10T06:10:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24676.069195 secs 2023-09-10T06:10:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:10:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:10:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:46 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:10:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:46 smithi098 ceph-mon[103124]: pgmap v13321: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 753 KiB/s wr, 19 op/s 2023-09-10T06:10:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:10:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:10:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:46 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:10:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:46 smithi170 ceph-mon[102832]: pgmap v13321: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 753 KiB/s wr, 19 op/s 2023-09-10T06:10:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:10:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:10:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:46 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:10:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:46 smithi082 ceph-mon[98694]: pgmap v13321: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 753 KiB/s wr, 19 op/s 2023-09-10T06:10:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:47 smithi082 ceph-mon[98694]: pgmap v13322: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 470 KiB/s wr, 13 op/s 2023-09-10T06:10:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:47 smithi098 ceph-mon[103124]: pgmap v13322: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 470 KiB/s wr, 13 op/s 2023-09-10T06:10:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:47 smithi170 ceph-mon[102832]: pgmap v13322: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 470 KiB/s wr, 13 op/s 2023-09-10T06:10:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24680.415550 secs 2023-09-10T06:10:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24680.415550 secs 2023-09-10T06:10:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24680.415550 secs 2023-09-10T06:10:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24681.069258 secs 2023-09-10T06:10:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:49 smithi170 ceph-mon[102832]: pgmap v13323: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 470 KiB/s wr, 13 op/s 2023-09-10T06:10:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24681.069258 secs 2023-09-10T06:10:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:49 smithi082 ceph-mon[98694]: pgmap v13323: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 470 KiB/s wr, 13 op/s 2023-09-10T06:10:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24681.069258 secs 2023-09-10T06:10:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:49 smithi098 ceph-mon[103124]: pgmap v13323: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 470 KiB/s wr, 13 op/s 2023-09-10T06:10:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:10:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:10:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:52 smithi170 ceph-mon[102832]: pgmap v13324: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 506 KiB/s wr, 18 op/s 2023-09-10T06:10:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:10:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:10:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:52 smithi082 ceph-mon[98694]: pgmap v13324: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 506 KiB/s wr, 18 op/s 2023-09-10T06:10:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:10:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:10:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:52 smithi098 ceph-mon[103124]: pgmap v13324: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 506 KiB/s wr, 18 op/s 2023-09-10T06:10:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24685.415639 secs 2023-09-10T06:10:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:10:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:10:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:54 smithi170 ceph-mon[102832]: pgmap v13325: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 417 KiB/s wr, 12 op/s 2023-09-10T06:10:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24686.069322 secs 2023-09-10T06:10:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24685.415639 secs 2023-09-10T06:10:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:10:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:10:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:54 smithi082 ceph-mon[98694]: pgmap v13325: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 417 KiB/s wr, 12 op/s 2023-09-10T06:10:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24686.069322 secs 2023-09-10T06:10:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24685.415639 secs 2023-09-10T06:10:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:10:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:10:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:54 smithi098 ceph-mon[103124]: pgmap v13325: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 417 KiB/s wr, 12 op/s 2023-09-10T06:10:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24686.069322 secs 2023-09-10T06:10:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:56 smithi170 ceph-mon[102832]: pgmap v13326: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 582 KiB/s wr, 18 op/s 2023-09-10T06:10:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:56 smithi082 ceph-mon[98694]: pgmap v13326: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 582 KiB/s wr, 18 op/s 2023-09-10T06:10:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:56 smithi098 ceph-mon[103124]: pgmap v13326: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 582 KiB/s wr, 18 op/s 2023-09-10T06:10:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:57 smithi098 ceph-mon[103124]: pgmap v13327: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 200 KiB/s wr, 10 op/s 2023-09-10T06:10:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:57 smithi170 ceph-mon[102832]: pgmap v13327: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 200 KiB/s wr, 10 op/s 2023-09-10T06:10:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:57 smithi082 ceph-mon[98694]: pgmap v13327: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 200 KiB/s wr, 10 op/s 2023-09-10T06:10:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24689.447007 secs 2023-09-10T06:10:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24689.447007 secs 2023-09-10T06:10:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24689.447007 secs 2023-09-10T06:11:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24690.415696 secs 2023-09-10T06:11:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24691.069390 secs 2023-09-10T06:11:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:10:59 smithi098 ceph-mon[103124]: pgmap v13328: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 200 KiB/s wr, 10 op/s 2023-09-10T06:11:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24690.415696 secs 2023-09-10T06:11:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24691.069390 secs 2023-09-10T06:11:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:10:59 smithi170 ceph-mon[102832]: pgmap v13328: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 200 KiB/s wr, 10 op/s 2023-09-10T06:11:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24690.415696 secs 2023-09-10T06:11:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24691.069390 secs 2023-09-10T06:11:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:10:59 smithi082 ceph-mon[98694]: pgmap v13328: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 200 KiB/s wr, 10 op/s 2023-09-10T06:11:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:11:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:11:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:11:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:11:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:11:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:11:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:01 smithi098 ceph-mon[103124]: pgmap v13329: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 406 KiB/s wr, 16 op/s 2023-09-10T06:11:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:01 smithi170 ceph-mon[102832]: pgmap v13329: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 406 KiB/s wr, 16 op/s 2023-09-10T06:11:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:01 smithi082 ceph-mon[98694]: pgmap v13329: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 406 KiB/s wr, 16 op/s 2023-09-10T06:11:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24694.447101 secs 2023-09-10T06:11:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:03 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:11:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:03 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24695.415750 secs 2023-09-10T06:11:04.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:03 smithi098 ceph-mon[103124]: pgmap v13330: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 370 KiB/s wr, 11 op/s 2023-09-10T06:11:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24694.447101 secs 2023-09-10T06:11:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:03 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:11:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:03 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:04.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24695.415750 secs 2023-09-10T06:11:04.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:03 smithi170 ceph-mon[102832]: pgmap v13330: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 370 KiB/s wr, 11 op/s 2023-09-10T06:11:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24694.447101 secs 2023-09-10T06:11:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:03 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:11:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:03 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24695.415750 secs 2023-09-10T06:11:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:03 smithi082 ceph-mon[98694]: pgmap v13330: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 370 KiB/s wr, 11 op/s 2023-09-10T06:11:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24696.069453 secs 2023-09-10T06:11:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24696.069453 secs 2023-09-10T06:11:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24696.069453 secs 2023-09-10T06:11:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:11:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:06 smithi170 ceph-mon[102832]: pgmap v13331: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 498 KiB/s wr, 31 op/s 2023-09-10T06:11:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:11:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:06 smithi082 ceph-mon[98694]: pgmap v13331: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 498 KiB/s wr, 31 op/s 2023-09-10T06:11:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:11:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:06 smithi098 ceph-mon[103124]: pgmap v13331: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 498 KiB/s wr, 31 op/s 2023-09-10T06:11:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:07 smithi098 ceph-mon[103124]: pgmap v13332: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 334 KiB/s wr, 25 op/s 2023-09-10T06:11:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:07 smithi170 ceph-mon[102832]: pgmap v13332: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 334 KiB/s wr, 25 op/s 2023-09-10T06:11:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:07 smithi082 ceph-mon[98694]: pgmap v13332: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 334 KiB/s wr, 25 op/s 2023-09-10T06:11:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24699.447190 secs 2023-09-10T06:11:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24699.447190 secs 2023-09-10T06:11:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24699.447190 secs 2023-09-10T06:11:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24700.415805 secs 2023-09-10T06:11:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:09 smithi098 ceph-mon[103124]: osdmap e901: 12 total, 12 up, 12 in 2023-09-10T06:11:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24701.069486 secs 2023-09-10T06:11:10.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:09 smithi098 ceph-mon[103124]: pgmap v13334: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 401 KiB/s wr, 30 op/s 2023-09-10T06:11:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24700.415805 secs 2023-09-10T06:11:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:09 smithi170 ceph-mon[102832]: osdmap e901: 12 total, 12 up, 12 in 2023-09-10T06:11:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24701.069486 secs 2023-09-10T06:11:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:09 smithi170 ceph-mon[102832]: pgmap v13334: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 401 KiB/s wr, 30 op/s 2023-09-10T06:11:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24700.415805 secs 2023-09-10T06:11:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:09 smithi082 ceph-mon[98694]: osdmap e901: 12 total, 12 up, 12 in 2023-09-10T06:11:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24701.069486 secs 2023-09-10T06:11:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:09 smithi082 ceph-mon[98694]: pgmap v13334: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 401 KiB/s wr, 30 op/s 2023-09-10T06:11:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:11 smithi098 ceph-mon[103124]: pgmap v13335: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 239 KiB/s wr, 30 op/s 2023-09-10T06:11:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:11 smithi170 ceph-mon[102832]: pgmap v13335: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 239 KiB/s wr, 30 op/s 2023-09-10T06:11:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:11 smithi082 ceph-mon[98694]: pgmap v13335: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 239 KiB/s wr, 30 op/s 2023-09-10T06:11:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24704.447270 secs 2023-09-10T06:11:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24704.447270 secs 2023-09-10T06:11:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24704.447270 secs 2023-09-10T06:11:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24705.415857 secs 2023-09-10T06:11:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:14 smithi098 ceph-mon[103124]: pgmap v13336: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 239 KiB/s wr, 30 op/s 2023-09-10T06:11:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24705.415857 secs 2023-09-10T06:11:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:14 smithi170 ceph-mon[102832]: pgmap v13336: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 239 KiB/s wr, 30 op/s 2023-09-10T06:11:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24705.415857 secs 2023-09-10T06:11:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:14 smithi082 ceph-mon[98694]: pgmap v13336: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 239 KiB/s wr, 30 op/s 2023-09-10T06:11:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24706.069552 secs 2023-09-10T06:11:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24706.069552 secs 2023-09-10T06:11:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24706.069552 secs 2023-09-10T06:11:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:16 smithi170 ceph-mon[102832]: osdmap e902: 12 total, 12 up, 12 in 2023-09-10T06:11:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:16 smithi170 ceph-mon[102832]: pgmap v13338: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 107 KiB/s wr, 15 op/s 2023-09-10T06:11:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:16 smithi082 ceph-mon[98694]: osdmap e902: 12 total, 12 up, 12 in 2023-09-10T06:11:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:16 smithi082 ceph-mon[98694]: pgmap v13338: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 107 KiB/s wr, 15 op/s 2023-09-10T06:11:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:16 smithi098 ceph-mon[103124]: osdmap e902: 12 total, 12 up, 12 in 2023-09-10T06:11:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:16 smithi098 ceph-mon[103124]: pgmap v13338: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 107 KiB/s wr, 15 op/s 2023-09-10T06:11:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:17 smithi098 ceph-mon[103124]: pgmap v13339: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 13 op/s 2023-09-10T06:11:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:17 smithi170 ceph-mon[102832]: pgmap v13339: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 13 op/s 2023-09-10T06:11:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:17 smithi082 ceph-mon[98694]: pgmap v13339: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 13 op/s 2023-09-10T06:11:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24709.447365 secs 2023-09-10T06:11:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24709.447365 secs 2023-09-10T06:11:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24709.447365 secs 2023-09-10T06:11:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24710.415946 secs 2023-09-10T06:11:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24711.069619 secs 2023-09-10T06:11:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:19 smithi098 ceph-mon[103124]: pgmap v13340: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 12 op/s 2023-09-10T06:11:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24710.415946 secs 2023-09-10T06:11:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24711.069619 secs 2023-09-10T06:11:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:19 smithi170 ceph-mon[102832]: pgmap v13340: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 12 op/s 2023-09-10T06:11:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24710.415946 secs 2023-09-10T06:11:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24711.069619 secs 2023-09-10T06:11:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:19 smithi082 ceph-mon[98694]: pgmap v13340: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 12 op/s 2023-09-10T06:11:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:21 smithi098 ceph-mon[103124]: pgmap v13341: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T06:11:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:21 smithi170 ceph-mon[102832]: pgmap v13341: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T06:11:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:21 smithi082 ceph-mon[98694]: pgmap v13341: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T06:11:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24714.447423 secs 2023-09-10T06:11:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24714.447423 secs 2023-09-10T06:11:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24714.447423 secs 2023-09-10T06:11:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24715.416008 secs 2023-09-10T06:11:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:24 smithi098 ceph-mon[103124]: pgmap v13342: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T06:11:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24715.416008 secs 2023-09-10T06:11:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:24 smithi170 ceph-mon[102832]: pgmap v13342: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T06:11:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24715.416008 secs 2023-09-10T06:11:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:24 smithi082 ceph-mon[98694]: pgmap v13342: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T06:11:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24716.069683 secs 2023-09-10T06:11:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24716.069683 secs 2023-09-10T06:11:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24716.069683 secs 2023-09-10T06:11:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:26 smithi170 ceph-mon[102832]: pgmap v13343: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T06:11:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:26 smithi082 ceph-mon[98694]: pgmap v13343: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T06:11:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:26 smithi098 ceph-mon[103124]: pgmap v13343: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T06:11:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:27 smithi098 ceph-mon[103124]: pgmap v13344: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 9 op/s 2023-09-10T06:11:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:27 smithi170 ceph-mon[102832]: pgmap v13344: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 9 op/s 2023-09-10T06:11:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:27 smithi082 ceph-mon[98694]: pgmap v13344: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 9 op/s 2023-09-10T06:11:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24720.416067 secs 2023-09-10T06:11:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24720.416067 secs 2023-09-10T06:11:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24720.416067 secs 2023-09-10T06:11:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24721.069748 secs 2023-09-10T06:11:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:29 smithi082 ceph-mon[98694]: pgmap v13345: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 9 op/s 2023-09-10T06:11:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24721.069748 secs 2023-09-10T06:11:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:29 smithi098 ceph-mon[103124]: pgmap v13345: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 9 op/s 2023-09-10T06:11:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24721.069748 secs 2023-09-10T06:11:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:29 smithi170 ceph-mon[102832]: pgmap v13345: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 84 KiB/s wr, 9 op/s 2023-09-10T06:11:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:11:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:11:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:11:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:11:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:11:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:32 smithi170 ceph-mon[102832]: pgmap v13346: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 178 KiB/s wr, 14 op/s 2023-09-10T06:11:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:11:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:11:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:32 smithi082 ceph-mon[98694]: pgmap v13346: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 178 KiB/s wr, 14 op/s 2023-09-10T06:11:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:11:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:11:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:32 smithi098 ceph-mon[103124]: pgmap v13346: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 178 KiB/s wr, 14 op/s 2023-09-10T06:11:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:11:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:11:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:11:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24725.416136 secs 2023-09-10T06:11:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:11:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:34 smithi170 ceph-mon[102832]: pgmap v13347: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 94 KiB/s wr, 9 op/s 2023-09-10T06:11:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24726.069814 secs 2023-09-10T06:11:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:11:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:11:34.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:11:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24725.416136 secs 2023-09-10T06:11:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:11:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:34 smithi082 ceph-mon[98694]: pgmap v13347: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 94 KiB/s wr, 9 op/s 2023-09-10T06:11:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24726.069814 secs 2023-09-10T06:11:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:11:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:11:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:11:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24725.416136 secs 2023-09-10T06:11:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:11:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:34 smithi098 ceph-mon[103124]: pgmap v13347: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 94 KiB/s wr, 9 op/s 2023-09-10T06:11:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24726.069814 secs 2023-09-10T06:11:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:11:35.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:11:35.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:11:36.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:11:36.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:36.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:36 smithi170 ceph-mon[102832]: pgmap v13348: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 370 KiB/s wr, 16 op/s 2023-09-10T06:11:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:11:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:36 smithi082 ceph-mon[98694]: pgmap v13348: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 370 KiB/s wr, 16 op/s 2023-09-10T06:11:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:11:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:36 smithi098 ceph-mon[103124]: pgmap v13348: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 370 KiB/s wr, 16 op/s 2023-09-10T06:11:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:37 smithi098 ceph-mon[103124]: pgmap v13349: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 369 KiB/s wr, 12 op/s 2023-09-10T06:11:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:37 smithi170 ceph-mon[102832]: pgmap v13349: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 369 KiB/s wr, 12 op/s 2023-09-10T06:11:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:37 smithi082 ceph-mon[98694]: pgmap v13349: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 369 KiB/s wr, 12 op/s 2023-09-10T06:11:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24729.447630 secs 2023-09-10T06:11:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24729.447630 secs 2023-09-10T06:11:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24729.447630 secs 2023-09-10T06:11:40.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24730.416193 secs 2023-09-10T06:11:40.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24731.069877 secs 2023-09-10T06:11:40.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:39 smithi098 ceph-mon[103124]: pgmap v13350: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 369 KiB/s wr, 12 op/s 2023-09-10T06:11:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24730.416193 secs 2023-09-10T06:11:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24731.069877 secs 2023-09-10T06:11:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:39 smithi170 ceph-mon[102832]: pgmap v13350: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 369 KiB/s wr, 12 op/s 2023-09-10T06:11:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24730.416193 secs 2023-09-10T06:11:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24731.069877 secs 2023-09-10T06:11:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:39 smithi082 ceph-mon[98694]: pgmap v13350: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 369 KiB/s wr, 12 op/s 2023-09-10T06:11:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:11:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:11:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:11:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:11:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:11:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:11:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:41 smithi098 ceph-mon[103124]: pgmap v13351: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 387 KiB/s wr, 19 op/s 2023-09-10T06:11:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:41 smithi170 ceph-mon[102832]: pgmap v13351: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 387 KiB/s wr, 19 op/s 2023-09-10T06:11:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:41 smithi082 ceph-mon[98694]: pgmap v13351: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 387 KiB/s wr, 19 op/s 2023-09-10T06:11:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24734.447690 secs 2023-09-10T06:11:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:11:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24735.416250 secs 2023-09-10T06:11:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:43 smithi098 ceph-mon[103124]: pgmap v13352: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 293 KiB/s wr, 13 op/s 2023-09-10T06:11:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24734.447690 secs 2023-09-10T06:11:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:11:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24735.416250 secs 2023-09-10T06:11:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:43 smithi170 ceph-mon[102832]: pgmap v13352: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 293 KiB/s wr, 13 op/s 2023-09-10T06:11:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24734.447690 secs 2023-09-10T06:11:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:11:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24735.416250 secs 2023-09-10T06:11:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:43 smithi082 ceph-mon[98694]: pgmap v13352: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 293 KiB/s wr, 13 op/s 2023-09-10T06:11:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24736.069940 secs 2023-09-10T06:11:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24736.069940 secs 2023-09-10T06:11:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24736.069940 secs 2023-09-10T06:11:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:11:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:45 smithi098 ceph-mon[103124]: pgmap v13353: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 748 KiB/s wr, 21 op/s 2023-09-10T06:11:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:11:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:45 smithi170 ceph-mon[102832]: pgmap v13353: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 748 KiB/s wr, 21 op/s 2023-09-10T06:11:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:11:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:45 smithi082 ceph-mon[98694]: pgmap v13353: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 748 KiB/s wr, 21 op/s 2023-09-10T06:11:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:47 smithi098 ceph-mon[103124]: pgmap v13354: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 472 KiB/s wr, 13 op/s 2023-09-10T06:11:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:47 smithi170 ceph-mon[102832]: pgmap v13354: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 472 KiB/s wr, 13 op/s 2023-09-10T06:11:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:47 smithi082 ceph-mon[98694]: pgmap v13354: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 472 KiB/s wr, 13 op/s 2023-09-10T06:11:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24740.416309 secs 2023-09-10T06:11:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24740.416309 secs 2023-09-10T06:11:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24740.416309 secs 2023-09-10T06:11:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24741.070007 secs 2023-09-10T06:11:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:49 smithi082 ceph-mon[98694]: pgmap v13355: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 472 KiB/s wr, 13 op/s 2023-09-10T06:11:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24741.070007 secs 2023-09-10T06:11:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:49 smithi098 ceph-mon[103124]: pgmap v13355: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 472 KiB/s wr, 13 op/s 2023-09-10T06:11:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24741.070007 secs 2023-09-10T06:11:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:49 smithi170 ceph-mon[102832]: pgmap v13355: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 472 KiB/s wr, 13 op/s 2023-09-10T06:11:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:51 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:11:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:51 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:11:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:51 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:11:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:52 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:11:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:11:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:52 smithi170 ceph-mon[102832]: pgmap v13356: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 598 KiB/s wr, 19 op/s 2023-09-10T06:11:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:52 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:11:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:11:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:52 smithi082 ceph-mon[98694]: pgmap v13356: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 598 KiB/s wr, 19 op/s 2023-09-10T06:11:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:52 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:11:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:11:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:52 smithi098 ceph-mon[103124]: pgmap v13356: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 598 KiB/s wr, 19 op/s 2023-09-10T06:11:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24745.416371 secs 2023-09-10T06:11:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:11:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:54 smithi170 ceph-mon[102832]: pgmap v13357: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 581 KiB/s wr, 13 op/s 2023-09-10T06:11:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24746.070074 secs 2023-09-10T06:11:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24745.416371 secs 2023-09-10T06:11:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:11:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:54 smithi082 ceph-mon[98694]: pgmap v13357: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 581 KiB/s wr, 13 op/s 2023-09-10T06:11:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24746.070074 secs 2023-09-10T06:11:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24745.416371 secs 2023-09-10T06:11:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:11:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:11:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:54 smithi098 ceph-mon[103124]: pgmap v13357: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 581 KiB/s wr, 13 op/s 2023-09-10T06:11:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24746.070074 secs 2023-09-10T06:11:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:56 smithi170 ceph-mon[102832]: pgmap v13358: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 655 KiB/s wr, 19 op/s 2023-09-10T06:11:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:56 smithi082 ceph-mon[98694]: pgmap v13358: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 655 KiB/s wr, 19 op/s 2023-09-10T06:11:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:56 smithi098 ceph-mon[103124]: pgmap v13358: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 655 KiB/s wr, 19 op/s 2023-09-10T06:11:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:57 smithi098 ceph-mon[103124]: pgmap v13359: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 200 KiB/s wr, 12 op/s 2023-09-10T06:11:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:57 smithi170 ceph-mon[102832]: pgmap v13359: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 200 KiB/s wr, 12 op/s 2023-09-10T06:11:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:57 smithi082 ceph-mon[98694]: pgmap v13359: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 200 KiB/s wr, 12 op/s 2023-09-10T06:11:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24749.447837 secs 2023-09-10T06:11:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24749.447837 secs 2023-09-10T06:11:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24749.447837 secs 2023-09-10T06:12:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24750.416424 secs 2023-09-10T06:12:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24751.070141 secs 2023-09-10T06:12:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:11:59 smithi098 ceph-mon[103124]: pgmap v13360: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 200 KiB/s wr, 12 op/s 2023-09-10T06:12:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24750.416424 secs 2023-09-10T06:12:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24751.070141 secs 2023-09-10T06:12:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:11:59 smithi170 ceph-mon[102832]: pgmap v13360: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 200 KiB/s wr, 12 op/s 2023-09-10T06:12:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24750.416424 secs 2023-09-10T06:12:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24751.070141 secs 2023-09-10T06:12:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:11:59 smithi082 ceph-mon[98694]: pgmap v13360: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 200 KiB/s wr, 12 op/s 2023-09-10T06:12:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:12:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:12:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:12:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:12:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:01.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:12:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:12:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:01 smithi098 ceph-mon[103124]: pgmap v13361: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 426 KiB/s wr, 32 op/s 2023-09-10T06:12:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:01 smithi170 ceph-mon[102832]: pgmap v13361: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 426 KiB/s wr, 32 op/s 2023-09-10T06:12:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:01 smithi082 ceph-mon[98694]: pgmap v13361: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 426 KiB/s wr, 32 op/s 2023-09-10T06:12:04.260 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24754.447899 secs 2023-09-10T06:12:04.261 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:03 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:12:04.261 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:03 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:04.261 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24755.416479 secs 2023-09-10T06:12:04.261 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:03 smithi098 ceph-mon[103124]: pgmap v13362: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 299 KiB/s wr, 26 op/s 2023-09-10T06:12:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24754.447899 secs 2023-09-10T06:12:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:03 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:12:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:03 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24755.416479 secs 2023-09-10T06:12:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:03 smithi170 ceph-mon[102832]: pgmap v13362: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 299 KiB/s wr, 26 op/s 2023-09-10T06:12:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24754.447899 secs 2023-09-10T06:12:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:03 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:12:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:03 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24755.416479 secs 2023-09-10T06:12:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:03 smithi082 ceph-mon[98694]: pgmap v13362: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 299 KiB/s wr, 26 op/s 2023-09-10T06:12:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24756.070203 secs 2023-09-10T06:12:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24756.070203 secs 2023-09-10T06:12:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24756.070203 secs 2023-09-10T06:12:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:12:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:06 smithi098 ceph-mon[103124]: pgmap v13363: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 383 KiB/s wr, 32 op/s 2023-09-10T06:12:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:12:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:06 smithi170 ceph-mon[102832]: pgmap v13363: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 383 KiB/s wr, 32 op/s 2023-09-10T06:12:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:12:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:05 smithi082 ceph-mon[98694]: pgmap v13363: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 383 KiB/s wr, 32 op/s 2023-09-10T06:12:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:07 smithi098 ceph-mon[103124]: pgmap v13364: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 309 KiB/s wr, 26 op/s 2023-09-10T06:12:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:07 smithi170 ceph-mon[102832]: pgmap v13364: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 309 KiB/s wr, 26 op/s 2023-09-10T06:12:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:07 smithi082 ceph-mon[98694]: pgmap v13364: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 309 KiB/s wr, 26 op/s 2023-09-10T06:12:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24759.448015 secs 2023-09-10T06:12:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24759.448015 secs 2023-09-10T06:12:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24759.448015 secs 2023-09-10T06:12:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24760.416581 secs 2023-09-10T06:12:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:09 smithi098 ceph-mon[103124]: osdmap e903: 12 total, 12 up, 12 in 2023-09-10T06:12:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24761.070272 secs 2023-09-10T06:12:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:09 smithi098 ceph-mon[103124]: pgmap v13366: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 370 KiB/s wr, 31 op/s 2023-09-10T06:12:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24760.416581 secs 2023-09-10T06:12:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:09 smithi170 ceph-mon[102832]: osdmap e903: 12 total, 12 up, 12 in 2023-09-10T06:12:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24761.070272 secs 2023-09-10T06:12:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:09 smithi170 ceph-mon[102832]: pgmap v13366: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 370 KiB/s wr, 31 op/s 2023-09-10T06:12:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24760.416581 secs 2023-09-10T06:12:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:09 smithi082 ceph-mon[98694]: osdmap e903: 12 total, 12 up, 12 in 2023-09-10T06:12:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24761.070272 secs 2023-09-10T06:12:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:09 smithi082 ceph-mon[98694]: pgmap v13366: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 370 KiB/s wr, 31 op/s 2023-09-10T06:12:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:11 smithi098 ceph-mon[103124]: pgmap v13367: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 13 op/s 2023-09-10T06:12:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:11 smithi170 ceph-mon[102832]: pgmap v13367: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 13 op/s 2023-09-10T06:12:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:11 smithi082 ceph-mon[98694]: pgmap v13367: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 13 op/s 2023-09-10T06:12:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24764.448103 secs 2023-09-10T06:12:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24764.448103 secs 2023-09-10T06:12:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24764.448103 secs 2023-09-10T06:12:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24765.416662 secs 2023-09-10T06:12:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:14 smithi098 ceph-mon[103124]: pgmap v13368: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 13 op/s 2023-09-10T06:12:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24765.416662 secs 2023-09-10T06:12:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:14 smithi170 ceph-mon[102832]: pgmap v13368: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 13 op/s 2023-09-10T06:12:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24765.416662 secs 2023-09-10T06:12:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:14 smithi082 ceph-mon[98694]: pgmap v13368: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 13 op/s 2023-09-10T06:12:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24766.070334 secs 2023-09-10T06:12:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24766.070334 secs 2023-09-10T06:12:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24766.070334 secs 2023-09-10T06:12:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:16 smithi170 ceph-mon[102832]: osdmap e904: 12 total, 12 up, 12 in 2023-09-10T06:12:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:16 smithi170 ceph-mon[102832]: pgmap v13370: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 16 op/s 2023-09-10T06:12:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:16 smithi082 ceph-mon[98694]: osdmap e904: 12 total, 12 up, 12 in 2023-09-10T06:12:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:16 smithi082 ceph-mon[98694]: pgmap v13370: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 16 op/s 2023-09-10T06:12:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:16 smithi098 ceph-mon[103124]: osdmap e904: 12 total, 12 up, 12 in 2023-09-10T06:12:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:16 smithi098 ceph-mon[103124]: pgmap v13370: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 16 op/s 2023-09-10T06:12:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:17 smithi098 ceph-mon[103124]: pgmap v13371: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 14 op/s 2023-09-10T06:12:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:17 smithi170 ceph-mon[102832]: pgmap v13371: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 14 op/s 2023-09-10T06:12:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:17 smithi082 ceph-mon[98694]: pgmap v13371: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 14 op/s 2023-09-10T06:12:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24769.448167 secs 2023-09-10T06:12:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24769.448167 secs 2023-09-10T06:12:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24769.448167 secs 2023-09-10T06:12:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24770.416745 secs 2023-09-10T06:12:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24771.070399 secs 2023-09-10T06:12:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:19 smithi098 ceph-mon[103124]: pgmap v13372: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T06:12:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24770.416745 secs 2023-09-10T06:12:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24771.070399 secs 2023-09-10T06:12:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:19 smithi170 ceph-mon[102832]: pgmap v13372: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T06:12:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24770.416745 secs 2023-09-10T06:12:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24771.070399 secs 2023-09-10T06:12:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:19 smithi082 ceph-mon[98694]: pgmap v13372: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T06:12:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:21 smithi098 ceph-mon[103124]: pgmap v13373: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 12 op/s 2023-09-10T06:12:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:21 smithi170 ceph-mon[102832]: pgmap v13373: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 12 op/s 2023-09-10T06:12:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:21 smithi082 ceph-mon[98694]: pgmap v13373: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 12 op/s 2023-09-10T06:12:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24774.448277 secs 2023-09-10T06:12:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24774.448277 secs 2023-09-10T06:12:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24774.448277 secs 2023-09-10T06:12:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24775.416805 secs 2023-09-10T06:12:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:24 smithi098 ceph-mon[103124]: pgmap v13374: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 12 op/s 2023-09-10T06:12:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24775.416805 secs 2023-09-10T06:12:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:24 smithi170 ceph-mon[102832]: pgmap v13374: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 12 op/s 2023-09-10T06:12:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24775.416805 secs 2023-09-10T06:12:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:24 smithi082 ceph-mon[98694]: pgmap v13374: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 12 op/s 2023-09-10T06:12:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24776.070462 secs 2023-09-10T06:12:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24776.070462 secs 2023-09-10T06:12:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24776.070462 secs 2023-09-10T06:12:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:26 smithi170 ceph-mon[102832]: pgmap v13375: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 100 KiB/s wr, 10 op/s 2023-09-10T06:12:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:26 smithi082 ceph-mon[98694]: pgmap v13375: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 100 KiB/s wr, 10 op/s 2023-09-10T06:12:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:26 smithi098 ceph-mon[103124]: pgmap v13375: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 100 KiB/s wr, 10 op/s 2023-09-10T06:12:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:27 smithi098 ceph-mon[103124]: pgmap v13376: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 85 KiB/s wr, 9 op/s 2023-09-10T06:12:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:27 smithi170 ceph-mon[102832]: pgmap v13376: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 85 KiB/s wr, 9 op/s 2023-09-10T06:12:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:27 smithi082 ceph-mon[98694]: pgmap v13376: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 85 KiB/s wr, 9 op/s 2023-09-10T06:12:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24780.416868 secs 2023-09-10T06:12:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24780.416868 secs 2023-09-10T06:12:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24780.416868 secs 2023-09-10T06:12:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24781.070486 secs 2023-09-10T06:12:30.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:29 smithi170 ceph-mon[102832]: pgmap v13377: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 85 KiB/s wr, 9 op/s 2023-09-10T06:12:30.649 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24781.070486 secs 2023-09-10T06:12:30.650 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:29 smithi098 ceph-mon[103124]: pgmap v13377: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 85 KiB/s wr, 9 op/s 2023-09-10T06:12:30.650 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24781.070486 secs 2023-09-10T06:12:30.650 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:29 smithi082 ceph-mon[98694]: pgmap v13377: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 85 KiB/s wr, 9 op/s 2023-09-10T06:12:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:12:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:12:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:12:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:12:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:12:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:32 smithi170 ceph-mon[102832]: pgmap v13378: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 170 KiB/s wr, 14 op/s 2023-09-10T06:12:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:12:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:12:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:32 smithi082 ceph-mon[98694]: pgmap v13378: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 170 KiB/s wr, 14 op/s 2023-09-10T06:12:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:12:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:12:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:32 smithi098 ceph-mon[103124]: pgmap v13378: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 170 KiB/s wr, 14 op/s 2023-09-10T06:12:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:33 smithi098 ceph-mon[103124]: pgmap v13379: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 9 op/s 2023-09-10T06:12:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:33 smithi170 ceph-mon[102832]: pgmap v13379: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 9 op/s 2023-09-10T06:12:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:33 smithi082 ceph-mon[98694]: pgmap v13379: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 9 op/s 2023-09-10T06:12:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24785.416927 secs 2023-09-10T06:12:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24786.070549 secs 2023-09-10T06:12:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:12:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24785.416927 secs 2023-09-10T06:12:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24786.070549 secs 2023-09-10T06:12:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:12:35.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24785.416927 secs 2023-09-10T06:12:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24786.070549 secs 2023-09-10T06:12:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:12:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:12:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:12:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:12:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:35 smithi098 ceph-mon[103124]: pgmap v13380: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 371 KiB/s wr, 16 op/s 2023-09-10T06:12:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:12:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:12:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:12:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:35 smithi170 ceph-mon[102832]: pgmap v13380: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 371 KiB/s wr, 16 op/s 2023-09-10T06:12:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:12:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:12:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:12:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:35 smithi082 ceph-mon[98694]: pgmap v13380: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 371 KiB/s wr, 16 op/s 2023-09-10T06:12:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:12:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:12:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:12:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:12:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:38 smithi170 ceph-mon[102832]: pgmap v13381: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 371 KiB/s wr, 12 op/s 2023-09-10T06:12:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24789.448477 secs 2023-09-10T06:12:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:12:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:38 smithi082 ceph-mon[98694]: pgmap v13381: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 371 KiB/s wr, 12 op/s 2023-09-10T06:12:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24789.448477 secs 2023-09-10T06:12:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:12:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:38 smithi098 ceph-mon[103124]: pgmap v13381: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 371 KiB/s wr, 12 op/s 2023-09-10T06:12:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24789.448477 secs 2023-09-10T06:12:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24790.417006 secs 2023-09-10T06:12:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24791.070613 secs 2023-09-10T06:12:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24790.417006 secs 2023-09-10T06:12:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24791.070613 secs 2023-09-10T06:12:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24790.417006 secs 2023-09-10T06:12:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24791.070613 secs 2023-09-10T06:12:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:12:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:12:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:40 smithi170 ceph-mon[102832]: pgmap v13382: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 371 KiB/s wr, 12 op/s 2023-09-10T06:12:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:12:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:12:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:40 smithi082 ceph-mon[98694]: pgmap v13382: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 371 KiB/s wr, 12 op/s 2023-09-10T06:12:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:12:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:12:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:40 smithi098 ceph-mon[103124]: pgmap v13382: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 371 KiB/s wr, 12 op/s 2023-09-10T06:12:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:41 smithi098 ceph-mon[103124]: pgmap v13383: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 388 KiB/s wr, 17 op/s 2023-09-10T06:12:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:41 smithi170 ceph-mon[102832]: pgmap v13383: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 388 KiB/s wr, 17 op/s 2023-09-10T06:12:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:41 smithi082 ceph-mon[98694]: pgmap v13383: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 388 KiB/s wr, 17 op/s 2023-09-10T06:12:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:42 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:12:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:42 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:12:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:42 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:12:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:12:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24795.417065 secs 2023-09-10T06:12:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:43 smithi098 ceph-mon[103124]: pgmap v13384: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 303 KiB/s wr, 12 op/s 2023-09-10T06:12:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:12:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24795.417065 secs 2023-09-10T06:12:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:43 smithi170 ceph-mon[102832]: pgmap v13384: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 303 KiB/s wr, 12 op/s 2023-09-10T06:12:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:12:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24795.417065 secs 2023-09-10T06:12:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:43 smithi082 ceph-mon[98694]: pgmap v13384: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 303 KiB/s wr, 12 op/s 2023-09-10T06:12:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24796.070675 secs 2023-09-10T06:12:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24796.070675 secs 2023-09-10T06:12:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24796.070675 secs 2023-09-10T06:12:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:12:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:46 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:12:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:46 smithi170 ceph-mon[102832]: pgmap v13385: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 758 KiB/s wr, 19 op/s 2023-09-10T06:12:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:12:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:46 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:12:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:46 smithi082 ceph-mon[98694]: pgmap v13385: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 758 KiB/s wr, 19 op/s 2023-09-10T06:12:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:12:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:46 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:12:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:46 smithi098 ceph-mon[103124]: pgmap v13385: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 758 KiB/s wr, 19 op/s 2023-09-10T06:12:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:47 smithi098 ceph-mon[103124]: pgmap v13386: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 473 KiB/s wr, 12 op/s 2023-09-10T06:12:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:47 smithi170 ceph-mon[102832]: pgmap v13386: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 473 KiB/s wr, 12 op/s 2023-09-10T06:12:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:47 smithi082 ceph-mon[98694]: pgmap v13386: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 473 KiB/s wr, 12 op/s 2023-09-10T06:12:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24800.417124 secs 2023-09-10T06:12:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24800.417124 secs 2023-09-10T06:12:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24800.417124 secs 2023-09-10T06:12:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24801.070738 secs 2023-09-10T06:12:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:49 smithi082 ceph-mon[98694]: pgmap v13387: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 473 KiB/s wr, 12 op/s 2023-09-10T06:12:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24801.070738 secs 2023-09-10T06:12:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:49 smithi098 ceph-mon[103124]: pgmap v13387: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 473 KiB/s wr, 12 op/s 2023-09-10T06:12:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24801.070738 secs 2023-09-10T06:12:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:49 smithi170 ceph-mon[102832]: pgmap v13387: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 473 KiB/s wr, 12 op/s 2023-09-10T06:12:52.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:51 smithi098 ceph-mon[103124]: pgmap v13388: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 571 KiB/s wr, 18 op/s 2023-09-10T06:12:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:51 smithi170 ceph-mon[102832]: pgmap v13388: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 571 KiB/s wr, 18 op/s 2023-09-10T06:12:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:51 smithi082 ceph-mon[98694]: pgmap v13388: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 571 KiB/s wr, 18 op/s 2023-09-10T06:12:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:53 smithi098 ceph-mon[103124]: pgmap v13389: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 553 KiB/s wr, 12 op/s 2023-09-10T06:12:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:53 smithi170 ceph-mon[102832]: pgmap v13389: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 553 KiB/s wr, 12 op/s 2023-09-10T06:12:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:53 smithi082 ceph-mon[98694]: pgmap v13389: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 553 KiB/s wr, 12 op/s 2023-09-10T06:12:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24805.417183 secs 2023-09-10T06:12:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24806.070802 secs 2023-09-10T06:12:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24805.417183 secs 2023-09-10T06:12:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24806.070802 secs 2023-09-10T06:12:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24805.417183 secs 2023-09-10T06:12:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24806.070802 secs 2023-09-10T06:12:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:56 smithi170 ceph-mon[102832]: pgmap v13390: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 657 KiB/s wr, 18 op/s 2023-09-10T06:12:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:56 smithi082 ceph-mon[98694]: pgmap v13390: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 657 KiB/s wr, 18 op/s 2023-09-10T06:12:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:56 smithi098 ceph-mon[103124]: pgmap v13390: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 657 KiB/s wr, 18 op/s 2023-09-10T06:12:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:12:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:58 smithi170 ceph-mon[102832]: pgmap v13391: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 201 KiB/s wr, 11 op/s 2023-09-10T06:12:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24809.448715 secs 2023-09-10T06:12:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:12:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:58 smithi082 ceph-mon[98694]: pgmap v13391: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 201 KiB/s wr, 11 op/s 2023-09-10T06:12:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24809.448715 secs 2023-09-10T06:12:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:12:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:12:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:58 smithi098 ceph-mon[103124]: pgmap v13391: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 201 KiB/s wr, 11 op/s 2023-09-10T06:12:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24809.448715 secs 2023-09-10T06:12:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24810.417250 secs 2023-09-10T06:12:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:12:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24811.070867 secs 2023-09-10T06:12:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24810.417250 secs 2023-09-10T06:12:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:12:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24811.070867 secs 2023-09-10T06:13:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24810.417250 secs 2023-09-10T06:13:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:12:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24811.070867 secs 2023-09-10T06:13:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:13:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:13:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:13:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:00 smithi170 ceph-mon[102832]: pgmap v13392: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 201 KiB/s wr, 11 op/s 2023-09-10T06:13:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:13:01.009 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:13:01.009 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:13:01.009 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:00 smithi082 ceph-mon[98694]: pgmap v13392: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 201 KiB/s wr, 11 op/s 2023-09-10T06:13:01.010 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:13:01.010 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:13:01.010 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:13:01.011 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:00 smithi098 ceph-mon[103124]: pgmap v13392: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 201 KiB/s wr, 11 op/s 2023-09-10T06:13:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:01 smithi098 ceph-mon[103124]: pgmap v13393: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 427 KiB/s wr, 31 op/s 2023-09-10T06:13:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:01 smithi170 ceph-mon[102832]: pgmap v13393: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 427 KiB/s wr, 31 op/s 2023-09-10T06:13:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:01 smithi082 ceph-mon[98694]: pgmap v13393: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 427 KiB/s wr, 31 op/s 2023-09-10T06:13:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24814.448774 secs 2023-09-10T06:13:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24814.448774 secs 2023-09-10T06:13:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24814.448774 secs 2023-09-10T06:13:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24815.417309 secs 2023-09-10T06:13:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:04 smithi098 ceph-mon[103124]: pgmap v13394: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 329 KiB/s wr, 26 op/s 2023-09-10T06:13:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24815.417309 secs 2023-09-10T06:13:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:04 smithi170 ceph-mon[102832]: pgmap v13394: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 329 KiB/s wr, 26 op/s 2023-09-10T06:13:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24815.417309 secs 2023-09-10T06:13:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:04 smithi082 ceph-mon[98694]: pgmap v13394: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 329 KiB/s wr, 26 op/s 2023-09-10T06:13:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24816.070930 secs 2023-09-10T06:13:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24816.070930 secs 2023-09-10T06:13:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24816.070930 secs 2023-09-10T06:13:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:06 smithi170 ceph-mon[102832]: pgmap v13395: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 413 KiB/s wr, 31 op/s 2023-09-10T06:13:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:06 smithi082 ceph-mon[98694]: pgmap v13395: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 413 KiB/s wr, 31 op/s 2023-09-10T06:13:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:06 smithi098 ceph-mon[103124]: pgmap v13395: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 413 KiB/s wr, 31 op/s 2023-09-10T06:13:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:07 smithi098 ceph-mon[103124]: pgmap v13396: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 309 KiB/s wr, 25 op/s 2023-09-10T06:13:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:07 smithi170 ceph-mon[102832]: pgmap v13396: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 309 KiB/s wr, 25 op/s 2023-09-10T06:13:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:07 smithi082 ceph-mon[98694]: pgmap v13396: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 309 KiB/s wr, 25 op/s 2023-09-10T06:13:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24819.448838 secs 2023-09-10T06:13:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24819.448838 secs 2023-09-10T06:13:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24819.448838 secs 2023-09-10T06:13:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24820.417368 secs 2023-09-10T06:13:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:09 smithi098 ceph-mon[103124]: osdmap e905: 12 total, 12 up, 12 in 2023-09-10T06:13:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24821.070995 secs 2023-09-10T06:13:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:09 smithi098 ceph-mon[103124]: pgmap v13398: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 371 KiB/s wr, 30 op/s 2023-09-10T06:13:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24820.417368 secs 2023-09-10T06:13:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:09 smithi170 ceph-mon[102832]: osdmap e905: 12 total, 12 up, 12 in 2023-09-10T06:13:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24821.070995 secs 2023-09-10T06:13:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:09 smithi170 ceph-mon[102832]: pgmap v13398: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 371 KiB/s wr, 30 op/s 2023-09-10T06:13:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24820.417368 secs 2023-09-10T06:13:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:09 smithi082 ceph-mon[98694]: osdmap e905: 12 total, 12 up, 12 in 2023-09-10T06:13:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24821.070995 secs 2023-09-10T06:13:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:09 smithi082 ceph-mon[98694]: pgmap v13398: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 371 KiB/s wr, 30 op/s 2023-09-10T06:13:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:11 smithi098 ceph-mon[103124]: pgmap v13399: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T06:13:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:11 smithi170 ceph-mon[102832]: pgmap v13399: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T06:13:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:11 smithi082 ceph-mon[98694]: pgmap v13399: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T06:13:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24824.448942 secs 2023-09-10T06:13:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24824.448942 secs 2023-09-10T06:13:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24824.448942 secs 2023-09-10T06:13:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24825.417426 secs 2023-09-10T06:13:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:14 smithi098 ceph-mon[103124]: pgmap v13400: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T06:13:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24825.417426 secs 2023-09-10T06:13:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:14 smithi170 ceph-mon[102832]: pgmap v13400: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T06:13:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24825.417426 secs 2023-09-10T06:13:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:14 smithi082 ceph-mon[98694]: pgmap v13400: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T06:13:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24826.071061 secs 2023-09-10T06:13:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24826.071061 secs 2023-09-10T06:13:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24826.071061 secs 2023-09-10T06:13:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:16 smithi170 ceph-mon[102832]: osdmap e906: 12 total, 12 up, 12 in 2023-09-10T06:13:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:16 smithi170 ceph-mon[102832]: pgmap v13402: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 14 op/s 2023-09-10T06:13:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:16 smithi082 ceph-mon[98694]: osdmap e906: 12 total, 12 up, 12 in 2023-09-10T06:13:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:16 smithi082 ceph-mon[98694]: pgmap v13402: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 14 op/s 2023-09-10T06:13:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:16 smithi098 ceph-mon[103124]: osdmap e906: 12 total, 12 up, 12 in 2023-09-10T06:13:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:16 smithi098 ceph-mon[103124]: pgmap v13402: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 14 op/s 2023-09-10T06:13:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:17 smithi098 ceph-mon[103124]: pgmap v13403: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T06:13:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:17 smithi170 ceph-mon[102832]: pgmap v13403: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T06:13:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:17 smithi082 ceph-mon[98694]: pgmap v13403: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T06:13:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24829.449001 secs 2023-09-10T06:13:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24829.449001 secs 2023-09-10T06:13:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24829.449001 secs 2023-09-10T06:13:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24830.417488 secs 2023-09-10T06:13:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24831.071134 secs 2023-09-10T06:13:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:19 smithi098 ceph-mon[103124]: pgmap v13404: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-10T06:13:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24830.417488 secs 2023-09-10T06:13:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24831.071134 secs 2023-09-10T06:13:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:19 smithi170 ceph-mon[102832]: pgmap v13404: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-10T06:13:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24830.417488 secs 2023-09-10T06:13:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24831.071134 secs 2023-09-10T06:13:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:19 smithi082 ceph-mon[98694]: pgmap v13404: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-09-10T06:13:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:21 smithi098 ceph-mon[103124]: pgmap v13405: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T06:13:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:21 smithi170 ceph-mon[102832]: pgmap v13405: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T06:13:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:21 smithi082 ceph-mon[98694]: pgmap v13405: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T06:13:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24834.449063 secs 2023-09-10T06:13:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24834.449063 secs 2023-09-10T06:13:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24834.449063 secs 2023-09-10T06:13:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24835.417527 secs 2023-09-10T06:13:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:24 smithi098 ceph-mon[103124]: pgmap v13406: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T06:13:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24835.417527 secs 2023-09-10T06:13:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:24 smithi170 ceph-mon[102832]: pgmap v13406: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T06:13:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24835.417527 secs 2023-09-10T06:13:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:24 smithi082 ceph-mon[98694]: pgmap v13406: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T06:13:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24836.071194 secs 2023-09-10T06:13:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24836.071194 secs 2023-09-10T06:13:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24836.071194 secs 2023-09-10T06:13:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:26 smithi170 ceph-mon[102832]: pgmap v13407: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T06:13:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:26 smithi082 ceph-mon[98694]: pgmap v13407: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T06:13:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:26 smithi098 ceph-mon[103124]: pgmap v13407: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T06:13:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:27 smithi098 ceph-mon[103124]: pgmap v13408: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T06:13:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:27 smithi170 ceph-mon[102832]: pgmap v13408: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T06:13:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:27 smithi082 ceph-mon[98694]: pgmap v13408: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T06:13:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24840.417610 secs 2023-09-10T06:13:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:29 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:13:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:29 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:13:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24841.071255 secs 2023-09-10T06:13:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:29 smithi098 ceph-mon[103124]: pgmap v13409: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T06:13:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24840.417610 secs 2023-09-10T06:13:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:29 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:13:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:29 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:13:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24841.071255 secs 2023-09-10T06:13:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:29 smithi170 ceph-mon[102832]: pgmap v13409: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T06:13:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24840.417610 secs 2023-09-10T06:13:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:29 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:13:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:29 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:13:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24841.071255 secs 2023-09-10T06:13:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:29 smithi082 ceph-mon[98694]: pgmap v13409: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 10 op/s 2023-09-10T06:13:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:13:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:13:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:13:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:13:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:13:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:13:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:32 smithi170 ceph-mon[102832]: pgmap v13410: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 171 KiB/s wr, 15 op/s 2023-09-10T06:13:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:13:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:13:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:13:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:32 smithi082 ceph-mon[98694]: pgmap v13410: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 171 KiB/s wr, 15 op/s 2023-09-10T06:13:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:13:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:13:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:13:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:32 smithi098 ceph-mon[103124]: pgmap v13410: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 171 KiB/s wr, 15 op/s 2023-09-10T06:13:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:33 smithi098 ceph-mon[103124]: pgmap v13411: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 10 op/s 2023-09-10T06:13:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:33 smithi170 ceph-mon[102832]: pgmap v13411: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 10 op/s 2023-09-10T06:13:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:33 smithi082 ceph-mon[98694]: pgmap v13411: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 10 op/s 2023-09-10T06:13:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24845.417670 secs 2023-09-10T06:13:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24846.071326 secs 2023-09-10T06:13:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24845.417670 secs 2023-09-10T06:13:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24846.071326 secs 2023-09-10T06:13:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24845.417670 secs 2023-09-10T06:13:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24846.071326 secs 2023-09-10T06:13:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:13:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:35 smithi098 ceph-mon[103124]: pgmap v13412: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 372 KiB/s wr, 17 op/s 2023-09-10T06:13:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:13:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:35 smithi170 ceph-mon[102832]: pgmap v13412: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 372 KiB/s wr, 17 op/s 2023-09-10T06:13:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:13:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:35 smithi082 ceph-mon[98694]: pgmap v13412: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 372 KiB/s wr, 17 op/s 2023-09-10T06:13:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:13:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:13:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:13:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:13:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:38 smithi170 ceph-mon[102832]: pgmap v13413: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 372 KiB/s wr, 12 op/s 2023-09-10T06:13:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:13:38.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:13:38.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:13:38.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:13:38.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:13:38.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:13:38.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:13:38.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:13:38.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:13:38.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:13:38.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:13:38.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:13:38.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:13:38.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:13:38.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:13:38.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:13:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:13:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:13:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:13:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:13:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:38 smithi082 ceph-mon[98694]: pgmap v13413: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 372 KiB/s wr, 12 op/s 2023-09-10T06:13:38.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:13:38.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:13:38.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:13:38.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:13:38.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:13:38.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:13:38.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:13:38.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:13:38.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:13:38.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:13:38.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:13:38.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:13:38.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:13:38.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:13:38.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:13:38.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:13:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:13:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:13:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:13:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:13:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:38 smithi098 ceph-mon[103124]: pgmap v13413: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 372 KiB/s wr, 12 op/s 2023-09-10T06:13:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:13:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:13:39.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:13:39.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:13:39.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:13:39.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:13:39.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:13:39.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:13:39.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:13:39.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:13:39.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:13:39.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:13:39.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:13:39.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:13:39.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:13:39.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:13:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24850.417729 secs 2023-09-10T06:13:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:39 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T06:13:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:39 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T06:13:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:39 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T06:13:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:39 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T06:13:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24851.071390 secs 2023-09-10T06:13:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24850.417729 secs 2023-09-10T06:13:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:39 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T06:13:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:39 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T06:13:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:39 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T06:13:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:39 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T06:13:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24851.071390 secs 2023-09-10T06:13:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24850.417729 secs 2023-09-10T06:13:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:39 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T06:13:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:39 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T06:13:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:39 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T06:13:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:39 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T06:13:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24851.071390 secs 2023-09-10T06:13:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:13:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:13:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:40 smithi170 ceph-mon[102832]: pgmap v13414: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 372 KiB/s wr, 12 op/s 2023-09-10T06:13:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:13:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:13:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:40 smithi082 ceph-mon[98694]: pgmap v13414: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 372 KiB/s wr, 12 op/s 2023-09-10T06:13:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:13:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:13:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:40 smithi098 ceph-mon[103124]: pgmap v13414: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 372 KiB/s wr, 12 op/s 2023-09-10T06:13:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:41 smithi098 ceph-mon[103124]: pgmap v13415: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 390 KiB/s wr, 18 op/s 2023-09-10T06:13:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:41 smithi170 ceph-mon[102832]: pgmap v13415: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 390 KiB/s wr, 18 op/s 2023-09-10T06:13:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:41 smithi082 ceph-mon[98694]: pgmap v13415: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 390 KiB/s wr, 18 op/s 2023-09-10T06:13:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:13:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:13:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:13:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:13:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:13:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:13:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:13:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:13:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:13:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:44 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24854.449398 secs 2023-09-10T06:13:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24855.417814 secs 2023-09-10T06:13:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:44 smithi170 ceph-mon[102832]: pgmap v13416: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 304 KiB/s wr, 12 op/s 2023-09-10T06:13:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:44 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24854.449398 secs 2023-09-10T06:13:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24855.417814 secs 2023-09-10T06:13:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:44 smithi082 ceph-mon[98694]: pgmap v13416: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 304 KiB/s wr, 12 op/s 2023-09-10T06:13:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:44 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24854.449398 secs 2023-09-10T06:13:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24855.417814 secs 2023-09-10T06:13:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:44 smithi098 ceph-mon[103124]: pgmap v13416: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 304 KiB/s wr, 12 op/s 2023-09-10T06:13:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24856.071453 secs 2023-09-10T06:13:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24856.071453 secs 2023-09-10T06:13:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24856.071453 secs 2023-09-10T06:13:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:13:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:13:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:46 smithi170 ceph-mon[102832]: pgmap v13417: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 761 KiB/s wr, 19 op/s 2023-09-10T06:13:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:13:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:13:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:46 smithi082 ceph-mon[98694]: pgmap v13417: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 761 KiB/s wr, 19 op/s 2023-09-10T06:13:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:13:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:13:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:46 smithi098 ceph-mon[103124]: pgmap v13417: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 761 KiB/s wr, 19 op/s 2023-09-10T06:13:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:47 smithi098 ceph-mon[103124]: pgmap v13418: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 475 KiB/s wr, 13 op/s 2023-09-10T06:13:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:47 smithi170 ceph-mon[102832]: pgmap v13418: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 475 KiB/s wr, 13 op/s 2023-09-10T06:13:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:47 smithi082 ceph-mon[98694]: pgmap v13418: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 475 KiB/s wr, 13 op/s 2023-09-10T06:13:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24859.449494 secs 2023-09-10T06:13:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24859.449494 secs 2023-09-10T06:13:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24859.449494 secs 2023-09-10T06:13:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24860.417874 secs 2023-09-10T06:13:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24861.071489 secs 2023-09-10T06:13:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:49 smithi098 ceph-mon[103124]: pgmap v13419: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 475 KiB/s wr, 13 op/s 2023-09-10T06:13:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24860.417874 secs 2023-09-10T06:13:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24861.071489 secs 2023-09-10T06:13:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:49 smithi170 ceph-mon[102832]: pgmap v13419: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 475 KiB/s wr, 13 op/s 2023-09-10T06:13:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24860.417874 secs 2023-09-10T06:13:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24861.071489 secs 2023-09-10T06:13:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:49 smithi082 ceph-mon[98694]: pgmap v13419: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 475 KiB/s wr, 13 op/s 2023-09-10T06:13:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:51 smithi098 ceph-mon[103124]: pgmap v13420: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 524 KiB/s wr, 18 op/s 2023-09-10T06:13:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:51 smithi170 ceph-mon[102832]: pgmap v13420: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 524 KiB/s wr, 18 op/s 2023-09-10T06:13:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:51 smithi082 ceph-mon[98694]: pgmap v13420: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 524 KiB/s wr, 18 op/s 2023-09-10T06:13:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24864.449624 secs 2023-09-10T06:13:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24864.449624 secs 2023-09-10T06:13:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24864.449624 secs 2023-09-10T06:13:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24865.417935 secs 2023-09-10T06:13:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:54 smithi098 ceph-mon[103124]: pgmap v13421: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 506 KiB/s wr, 13 op/s 2023-09-10T06:13:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24865.417935 secs 2023-09-10T06:13:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:54 smithi170 ceph-mon[102832]: pgmap v13421: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 506 KiB/s wr, 13 op/s 2023-09-10T06:13:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24865.417935 secs 2023-09-10T06:13:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:54 smithi082 ceph-mon[98694]: pgmap v13421: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 506 KiB/s wr, 13 op/s 2023-09-10T06:13:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24866.071552 secs 2023-09-10T06:13:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24866.071552 secs 2023-09-10T06:13:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24866.071552 secs 2023-09-10T06:13:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:56 smithi170 ceph-mon[102832]: pgmap v13422: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 659 KiB/s wr, 18 op/s 2023-09-10T06:13:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:56 smithi082 ceph-mon[98694]: pgmap v13422: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 659 KiB/s wr, 18 op/s 2023-09-10T06:13:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:56 smithi098 ceph-mon[103124]: pgmap v13422: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 659 KiB/s wr, 18 op/s 2023-09-10T06:13:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:13:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:13:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:58 smithi170 ceph-mon[102832]: pgmap v13423: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 202 KiB/s wr, 11 op/s 2023-09-10T06:13:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24869.449684 secs 2023-09-10T06:13:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:13:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:13:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:58 smithi082 ceph-mon[98694]: pgmap v13423: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 202 KiB/s wr, 11 op/s 2023-09-10T06:13:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24869.449684 secs 2023-09-10T06:13:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:13:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:13:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:58 smithi098 ceph-mon[103124]: pgmap v13423: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 202 KiB/s wr, 11 op/s 2023-09-10T06:13:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24869.449684 secs 2023-09-10T06:13:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24870.417994 secs 2023-09-10T06:13:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:13:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24871.071616 secs 2023-09-10T06:13:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24870.417994 secs 2023-09-10T06:13:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:13:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24871.071616 secs 2023-09-10T06:14:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24870.417994 secs 2023-09-10T06:14:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:13:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24871.071616 secs 2023-09-10T06:14:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:14:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:14:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:00 smithi170 ceph-mon[102832]: pgmap v13424: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 202 KiB/s wr, 11 op/s 2023-09-10T06:14:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:14:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:14:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:00 smithi082 ceph-mon[98694]: pgmap v13424: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 202 KiB/s wr, 11 op/s 2023-09-10T06:14:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:14:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:14:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:00 smithi098 ceph-mon[103124]: pgmap v13424: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 202 KiB/s wr, 11 op/s 2023-09-10T06:14:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:01 smithi098 ceph-mon[103124]: pgmap v13425: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 387 KiB/s wr, 17 op/s 2023-09-10T06:14:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:01 smithi170 ceph-mon[102832]: pgmap v13425: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 387 KiB/s wr, 17 op/s 2023-09-10T06:14:02.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:01 smithi082 ceph-mon[98694]: pgmap v13425: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 387 KiB/s wr, 17 op/s 2023-09-10T06:14:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24874.449809 secs 2023-09-10T06:14:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24874.449809 secs 2023-09-10T06:14:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24874.449809 secs 2023-09-10T06:14:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24875.418081 secs 2023-09-10T06:14:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:04 smithi098 ceph-mon[103124]: pgmap v13426: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 338 KiB/s wr, 11 op/s 2023-09-10T06:14:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24875.418081 secs 2023-09-10T06:14:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:04 smithi170 ceph-mon[102832]: pgmap v13426: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 338 KiB/s wr, 11 op/s 2023-09-10T06:14:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24875.418081 secs 2023-09-10T06:14:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:04 smithi082 ceph-mon[98694]: pgmap v13426: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 338 KiB/s wr, 11 op/s 2023-09-10T06:14:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24876.071679 secs 2023-09-10T06:14:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24876.071679 secs 2023-09-10T06:14:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24876.071679 secs 2023-09-10T06:14:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:06 smithi170 ceph-mon[102832]: pgmap v13427: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 463 KiB/s wr, 32 op/s 2023-09-10T06:14:06.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:06 smithi082 ceph-mon[98694]: pgmap v13427: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 463 KiB/s wr, 32 op/s 2023-09-10T06:14:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:06 smithi098 ceph-mon[103124]: pgmap v13427: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 463 KiB/s wr, 32 op/s 2023-09-10T06:14:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:07 smithi098 ceph-mon[103124]: pgmap v13428: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 310 KiB/s wr, 26 op/s 2023-09-10T06:14:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:07 smithi170 ceph-mon[102832]: pgmap v13428: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 310 KiB/s wr, 26 op/s 2023-09-10T06:14:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:07 smithi082 ceph-mon[98694]: pgmap v13428: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 310 KiB/s wr, 26 op/s 2023-09-10T06:14:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24879.449867 secs 2023-09-10T06:14:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24879.449867 secs 2023-09-10T06:14:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24879.449867 secs 2023-09-10T06:14:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24880.418166 secs 2023-09-10T06:14:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:09 smithi098 ceph-mon[103124]: osdmap e907: 12 total, 12 up, 12 in 2023-09-10T06:14:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24881.071744 secs 2023-09-10T06:14:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:09 smithi098 ceph-mon[103124]: pgmap v13430: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 372 KiB/s wr, 31 op/s 2023-09-10T06:14:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24880.418166 secs 2023-09-10T06:14:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:09 smithi170 ceph-mon[102832]: osdmap e907: 12 total, 12 up, 12 in 2023-09-10T06:14:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24881.071744 secs 2023-09-10T06:14:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:09 smithi170 ceph-mon[102832]: pgmap v13430: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 372 KiB/s wr, 31 op/s 2023-09-10T06:14:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24880.418166 secs 2023-09-10T06:14:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:09 smithi082 ceph-mon[98694]: osdmap e907: 12 total, 12 up, 12 in 2023-09-10T06:14:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24881.071744 secs 2023-09-10T06:14:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:09 smithi082 ceph-mon[98694]: pgmap v13430: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 372 KiB/s wr, 31 op/s 2023-09-10T06:14:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:11 smithi098 ceph-mon[103124]: pgmap v13431: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 173 KiB/s wr, 31 op/s 2023-09-10T06:14:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:11 smithi170 ceph-mon[102832]: pgmap v13431: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 173 KiB/s wr, 31 op/s 2023-09-10T06:14:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:11 smithi082 ceph-mon[98694]: pgmap v13431: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 173 KiB/s wr, 31 op/s 2023-09-10T06:14:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24884.450000 secs 2023-09-10T06:14:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24884.450000 secs 2023-09-10T06:14:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24884.450000 secs 2023-09-10T06:14:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24885.458575 secs 2023-09-10T06:14:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:14 smithi098 ceph-mon[103124]: pgmap v13432: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 173 KiB/s wr, 31 op/s 2023-09-10T06:14:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24885.458575 secs 2023-09-10T06:14:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:14 smithi170 ceph-mon[102832]: pgmap v13432: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 173 KiB/s wr, 31 op/s 2023-09-10T06:14:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24885.458575 secs 2023-09-10T06:14:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:14 smithi082 ceph-mon[98694]: pgmap v13432: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 173 KiB/s wr, 31 op/s 2023-09-10T06:14:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24886.071806 secs 2023-09-10T06:14:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24886.071806 secs 2023-09-10T06:14:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24886.071806 secs 2023-09-10T06:14:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:16 smithi170 ceph-mon[102832]: osdmap e908: 12 total, 12 up, 12 in 2023-09-10T06:14:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:16 smithi170 ceph-mon[102832]: pgmap v13434: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 15 op/s 2023-09-10T06:14:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:16 smithi082 ceph-mon[98694]: osdmap e908: 12 total, 12 up, 12 in 2023-09-10T06:14:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:16 smithi082 ceph-mon[98694]: pgmap v13434: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 15 op/s 2023-09-10T06:14:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:16 smithi098 ceph-mon[103124]: osdmap e908: 12 total, 12 up, 12 in 2023-09-10T06:14:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:16 smithi098 ceph-mon[103124]: pgmap v13434: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 15 op/s 2023-09-10T06:14:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:17 smithi098 ceph-mon[103124]: pgmap v13435: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T06:14:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:17 smithi170 ceph-mon[102832]: pgmap v13435: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T06:14:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:17 smithi082 ceph-mon[98694]: pgmap v13435: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T06:14:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24889.450060 secs 2023-09-10T06:14:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24889.450060 secs 2023-09-10T06:14:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24889.450060 secs 2023-09-10T06:14:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24890.458663 secs 2023-09-10T06:14:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24891.071875 secs 2023-09-10T06:14:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:19 smithi098 ceph-mon[103124]: pgmap v13436: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T06:14:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24890.458663 secs 2023-09-10T06:14:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24891.071875 secs 2023-09-10T06:14:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:19 smithi170 ceph-mon[102832]: pgmap v13436: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T06:14:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24890.458663 secs 2023-09-10T06:14:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24891.071875 secs 2023-09-10T06:14:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:19 smithi082 ceph-mon[98694]: pgmap v13436: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T06:14:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:21 smithi098 ceph-mon[103124]: pgmap v13437: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T06:14:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:21 smithi170 ceph-mon[102832]: pgmap v13437: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T06:14:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:21 smithi082 ceph-mon[98694]: pgmap v13437: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T06:14:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24894.450161 secs 2023-09-10T06:14:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24894.450161 secs 2023-09-10T06:14:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24894.450161 secs 2023-09-10T06:14:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24895.458745 secs 2023-09-10T06:14:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:24 smithi098 ceph-mon[103124]: pgmap v13438: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T06:14:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24895.458745 secs 2023-09-10T06:14:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:24 smithi170 ceph-mon[102832]: pgmap v13438: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T06:14:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24895.458745 secs 2023-09-10T06:14:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:24 smithi082 ceph-mon[98694]: pgmap v13438: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T06:14:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24896.071940 secs 2023-09-10T06:14:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24896.071940 secs 2023-09-10T06:14:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24896.071940 secs 2023-09-10T06:14:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:26 smithi170 ceph-mon[102832]: pgmap v13439: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 101 KiB/s wr, 11 op/s 2023-09-10T06:14:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:26 smithi082 ceph-mon[98694]: pgmap v13439: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 101 KiB/s wr, 11 op/s 2023-09-10T06:14:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:26 smithi098 ceph-mon[103124]: pgmap v13439: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 101 KiB/s wr, 11 op/s 2023-09-10T06:14:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:27 smithi098 ceph-mon[103124]: pgmap v13440: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 9 op/s 2023-09-10T06:14:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:27 smithi170 ceph-mon[102832]: pgmap v13440: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 9 op/s 2023-09-10T06:14:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:27 smithi082 ceph-mon[98694]: pgmap v13440: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 9 op/s 2023-09-10T06:14:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24900.458803 secs 2023-09-10T06:14:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24900.458803 secs 2023-09-10T06:14:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24900.458803 secs 2023-09-10T06:14:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24901.072005 secs 2023-09-10T06:14:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:29 smithi082 ceph-mon[98694]: pgmap v13441: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 9 op/s 2023-09-10T06:14:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24901.072005 secs 2023-09-10T06:14:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:29 smithi098 ceph-mon[103124]: pgmap v13441: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 9 op/s 2023-09-10T06:14:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24901.072005 secs 2023-09-10T06:14:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:29 smithi170 ceph-mon[102832]: pgmap v13441: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 85 KiB/s wr, 9 op/s 2023-09-10T06:14:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:14:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:14:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:14:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:14:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:14:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:14:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:32 smithi170 ceph-mon[102832]: pgmap v13442: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 172 KiB/s wr, 15 op/s 2023-09-10T06:14:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:14:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:14:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:14:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:32 smithi082 ceph-mon[98694]: pgmap v13442: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 172 KiB/s wr, 15 op/s 2023-09-10T06:14:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:14:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:14:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:14:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:32 smithi098 ceph-mon[103124]: pgmap v13442: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 172 KiB/s wr, 15 op/s 2023-09-10T06:14:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24905.458859 secs 2023-09-10T06:14:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:14:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:14:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:34 smithi170 ceph-mon[102832]: pgmap v13443: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 10 op/s 2023-09-10T06:14:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24906.072072 secs 2023-09-10T06:14:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24905.458859 secs 2023-09-10T06:14:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:14:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:14:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:34 smithi082 ceph-mon[98694]: pgmap v13443: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 10 op/s 2023-09-10T06:14:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24906.072072 secs 2023-09-10T06:14:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24905.458859 secs 2023-09-10T06:14:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:14:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:14:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:34 smithi098 ceph-mon[103124]: pgmap v13443: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 10 op/s 2023-09-10T06:14:35.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24906.072072 secs 2023-09-10T06:14:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:36 smithi170 ceph-mon[102832]: pgmap v13444: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 386 KiB/s wr, 17 op/s 2023-09-10T06:14:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:36 smithi082 ceph-mon[98694]: pgmap v13444: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 386 KiB/s wr, 17 op/s 2023-09-10T06:14:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:36 smithi098 ceph-mon[103124]: pgmap v13444: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 386 KiB/s wr, 17 op/s 2023-09-10T06:14:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:37 smithi098 ceph-mon[103124]: pgmap v13445: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 386 KiB/s wr, 13 op/s 2023-09-10T06:14:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:37 smithi170 ceph-mon[102832]: pgmap v13445: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 386 KiB/s wr, 13 op/s 2023-09-10T06:14:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:37 smithi082 ceph-mon[98694]: pgmap v13445: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 386 KiB/s wr, 13 op/s 2023-09-10T06:14:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:14:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:14:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:14:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24910.458945 secs 2023-09-10T06:14:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24911.072137 secs 2023-09-10T06:14:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:14:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:14:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:14:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:39 smithi098 ceph-mon[103124]: pgmap v13446: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 386 KiB/s wr, 13 op/s 2023-09-10T06:14:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24910.458945 secs 2023-09-10T06:14:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24911.072137 secs 2023-09-10T06:14:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:14:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:14:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:14:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:39 smithi170 ceph-mon[102832]: pgmap v13446: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 386 KiB/s wr, 13 op/s 2023-09-10T06:14:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24910.458945 secs 2023-09-10T06:14:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24911.072137 secs 2023-09-10T06:14:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:14:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:14:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:14:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:39 smithi082 ceph-mon[98694]: pgmap v13446: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 386 KiB/s wr, 13 op/s 2023-09-10T06:14:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:14:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:14:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:14:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:14:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:14:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:14:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:41 smithi170 ceph-mon[102832]: pgmap v13447: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 392 KiB/s wr, 19 op/s 2023-09-10T06:14:42.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:41 smithi082 ceph-mon[98694]: pgmap v13447: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 392 KiB/s wr, 19 op/s 2023-09-10T06:14:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:41 smithi098 ceph-mon[103124]: pgmap v13447: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 392 KiB/s wr, 19 op/s 2023-09-10T06:14:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:14:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:14:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24915.459004 secs 2023-09-10T06:14:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:43 smithi098 ceph-mon[103124]: pgmap v13448: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 305 KiB/s wr, 13 op/s 2023-09-10T06:14:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:14:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:14:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24915.459004 secs 2023-09-10T06:14:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:43 smithi170 ceph-mon[102832]: pgmap v13448: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 305 KiB/s wr, 13 op/s 2023-09-10T06:14:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:14:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:14:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24915.459004 secs 2023-09-10T06:14:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:43 smithi082 ceph-mon[98694]: pgmap v13448: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 305 KiB/s wr, 13 op/s 2023-09-10T06:14:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24916.072203 secs 2023-09-10T06:14:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24916.072203 secs 2023-09-10T06:14:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24916.072203 secs 2023-09-10T06:14:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:14:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:14:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:46 smithi170 ceph-mon[102832]: pgmap v13449: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 764 KiB/s wr, 19 op/s 2023-09-10T06:14:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:14:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:14:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:46 smithi082 ceph-mon[98694]: pgmap v13449: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 764 KiB/s wr, 19 op/s 2023-09-10T06:14:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:14:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:14:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:46 smithi098 ceph-mon[103124]: pgmap v13449: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 764 KiB/s wr, 19 op/s 2023-09-10T06:14:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:47 smithi098 ceph-mon[103124]: pgmap v13450: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 465 KiB/s wr, 12 op/s 2023-09-10T06:14:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:47 smithi170 ceph-mon[102832]: pgmap v13450: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 465 KiB/s wr, 12 op/s 2023-09-10T06:14:48.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:47 smithi082 ceph-mon[98694]: pgmap v13450: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 465 KiB/s wr, 12 op/s 2023-09-10T06:14:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24920.459064 secs 2023-09-10T06:14:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:14:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:14:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24921.072266 secs 2023-09-10T06:14:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:49 smithi170 ceph-mon[102832]: pgmap v13451: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 465 KiB/s wr, 12 op/s 2023-09-10T06:14:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24920.459064 secs 2023-09-10T06:14:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:14:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:14:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24921.072266 secs 2023-09-10T06:14:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:49 smithi082 ceph-mon[98694]: pgmap v13451: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 465 KiB/s wr, 12 op/s 2023-09-10T06:14:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24920.459064 secs 2023-09-10T06:14:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:14:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:14:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24921.072266 secs 2023-09-10T06:14:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:49 smithi098 ceph-mon[103124]: pgmap v13451: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 465 KiB/s wr, 12 op/s 2023-09-10T06:14:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:52 smithi098 ceph-mon[103124]: pgmap v13452: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 514 KiB/s wr, 18 op/s 2023-09-10T06:14:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:52 smithi170 ceph-mon[102832]: pgmap v13452: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 514 KiB/s wr, 18 op/s 2023-09-10T06:14:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:52 smithi082 ceph-mon[98694]: pgmap v13452: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 514 KiB/s wr, 18 op/s 2023-09-10T06:14:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24925.459124 secs 2023-09-10T06:14:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:54 smithi098 ceph-mon[103124]: pgmap v13453: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 508 KiB/s wr, 12 op/s 2023-09-10T06:14:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24926.072331 secs 2023-09-10T06:14:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24925.459124 secs 2023-09-10T06:14:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:54 smithi170 ceph-mon[102832]: pgmap v13453: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 508 KiB/s wr, 12 op/s 2023-09-10T06:14:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24926.072331 secs 2023-09-10T06:14:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24925.459124 secs 2023-09-10T06:14:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:54 smithi082 ceph-mon[98694]: pgmap v13453: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 508 KiB/s wr, 12 op/s 2023-09-10T06:14:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24926.072331 secs 2023-09-10T06:14:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:57 smithi170 ceph-mon[102832]: pgmap v13454: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 662 KiB/s wr, 18 op/s 2023-09-10T06:14:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:57 smithi082 ceph-mon[98694]: pgmap v13454: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 662 KiB/s wr, 18 op/s 2023-09-10T06:14:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:57 smithi098 ceph-mon[103124]: pgmap v13454: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 662 KiB/s wr, 18 op/s 2023-09-10T06:14:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:14:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:14:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24929.450668 secs 2023-09-10T06:14:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:14:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:14:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24929.450668 secs 2023-09-10T06:14:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:14:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:14:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24929.450668 secs 2023-09-10T06:14:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:59 smithi170 ceph-mon[102832]: pgmap v13455: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 203 KiB/s wr, 11 op/s 2023-09-10T06:14:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24930.459203 secs 2023-09-10T06:14:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:14:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24931.072395 secs 2023-09-10T06:14:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:59 smithi082 ceph-mon[98694]: pgmap v13455: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 203 KiB/s wr, 11 op/s 2023-09-10T06:14:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24930.459203 secs 2023-09-10T06:14:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:14:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24931.072395 secs 2023-09-10T06:15:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:59 smithi098 ceph-mon[103124]: pgmap v13455: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 203 KiB/s wr, 11 op/s 2023-09-10T06:15:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24930.459203 secs 2023-09-10T06:15:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:14:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24931.072395 secs 2023-09-10T06:15:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:15:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:15:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:15:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:15:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:15:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:15:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:15:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:15:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:15:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:01 smithi170 ceph-mon[102832]: pgmap v13456: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 203 KiB/s wr, 11 op/s 2023-09-10T06:15:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:01 smithi082 ceph-mon[98694]: pgmap v13456: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 203 KiB/s wr, 11 op/s 2023-09-10T06:15:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:01 smithi098 ceph-mon[103124]: pgmap v13456: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 203 KiB/s wr, 11 op/s 2023-09-10T06:15:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:03 smithi170 ceph-mon[102832]: pgmap v13457: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 385 KiB/s wr, 17 op/s 2023-09-10T06:15:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24934.450753 secs 2023-09-10T06:15:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:03 smithi082 ceph-mon[98694]: pgmap v13457: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 385 KiB/s wr, 17 op/s 2023-09-10T06:15:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24934.450753 secs 2023-09-10T06:15:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:03 smithi098 ceph-mon[103124]: pgmap v13457: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 385 KiB/s wr, 17 op/s 2023-09-10T06:15:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24934.450753 secs 2023-09-10T06:15:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24935.459293 secs 2023-09-10T06:15:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24936.072460 secs 2023-09-10T06:15:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24935.459293 secs 2023-09-10T06:15:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24936.072460 secs 2023-09-10T06:15:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24935.459293 secs 2023-09-10T06:15:06.148 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24936.072460 secs 2023-09-10T06:15:06.149 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:05 smithi082 ceph-mon[98694]: pgmap v13458: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 336 KiB/s wr, 11 op/s 2023-09-10T06:15:06.149 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:05 smithi170 ceph-mon[102832]: pgmap v13458: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 336 KiB/s wr, 11 op/s 2023-09-10T06:15:06.150 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:05 smithi098 ceph-mon[103124]: pgmap v13458: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 336 KiB/s wr, 11 op/s 2023-09-10T06:15:07.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:07 smithi170 ceph-mon[102832]: pgmap v13459: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 465 KiB/s wr, 31 op/s 2023-09-10T06:15:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:07 smithi082 ceph-mon[98694]: pgmap v13459: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 465 KiB/s wr, 31 op/s 2023-09-10T06:15:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:07 smithi098 ceph-mon[103124]: pgmap v13459: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 465 KiB/s wr, 31 op/s 2023-09-10T06:15:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24939.450785 secs 2023-09-10T06:15:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24939.450785 secs 2023-09-10T06:15:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24939.450785 secs 2023-09-10T06:15:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:09 smithi170 ceph-mon[102832]: pgmap v13460: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 311 KiB/s wr, 26 op/s 2023-09-10T06:15:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24940.459350 secs 2023-09-10T06:15:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:09 smithi170 ceph-mon[102832]: osdmap e909: 12 total, 12 up, 12 in 2023-09-10T06:15:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24941.072486 secs 2023-09-10T06:15:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:09 smithi082 ceph-mon[98694]: pgmap v13460: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 311 KiB/s wr, 26 op/s 2023-09-10T06:15:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24940.459350 secs 2023-09-10T06:15:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:09 smithi082 ceph-mon[98694]: osdmap e909: 12 total, 12 up, 12 in 2023-09-10T06:15:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24941.072486 secs 2023-09-10T06:15:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:09 smithi098 ceph-mon[103124]: pgmap v13460: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 311 KiB/s wr, 26 op/s 2023-09-10T06:15:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24940.459350 secs 2023-09-10T06:15:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:09 smithi098 ceph-mon[103124]: osdmap e909: 12 total, 12 up, 12 in 2023-09-10T06:15:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24941.072486 secs 2023-09-10T06:15:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:11 smithi170 ceph-mon[102832]: pgmap v13462: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 374 KiB/s wr, 31 op/s 2023-09-10T06:15:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:11 smithi082 ceph-mon[98694]: pgmap v13462: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 374 KiB/s wr, 31 op/s 2023-09-10T06:15:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:11 smithi098 ceph-mon[103124]: pgmap v13462: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 374 KiB/s wr, 31 op/s 2023-09-10T06:15:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:13 smithi170 ceph-mon[102832]: pgmap v13463: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 178 KiB/s wr, 30 op/s 2023-09-10T06:15:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24944.450844 secs 2023-09-10T06:15:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:13 smithi082 ceph-mon[98694]: pgmap v13463: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 178 KiB/s wr, 30 op/s 2023-09-10T06:15:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24944.450844 secs 2023-09-10T06:15:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:13 smithi098 ceph-mon[103124]: pgmap v13463: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 178 KiB/s wr, 30 op/s 2023-09-10T06:15:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24944.450844 secs 2023-09-10T06:15:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24945.459435 secs 2023-09-10T06:15:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24946.072551 secs 2023-09-10T06:15:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24945.459435 secs 2023-09-10T06:15:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24946.072551 secs 2023-09-10T06:15:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24945.459435 secs 2023-09-10T06:15:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24946.072551 secs 2023-09-10T06:15:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:15 smithi170 ceph-mon[102832]: pgmap v13464: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 178 KiB/s wr, 30 op/s 2023-09-10T06:15:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:15 smithi170 ceph-mon[102832]: osdmap e910: 12 total, 12 up, 12 in 2023-09-10T06:15:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:15 smithi082 ceph-mon[98694]: pgmap v13464: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 178 KiB/s wr, 30 op/s 2023-09-10T06:15:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:15 smithi082 ceph-mon[98694]: osdmap e910: 12 total, 12 up, 12 in 2023-09-10T06:15:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:15 smithi098 ceph-mon[103124]: pgmap v13464: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 178 KiB/s wr, 30 op/s 2023-09-10T06:15:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:15 smithi098 ceph-mon[103124]: osdmap e910: 12 total, 12 up, 12 in 2023-09-10T06:15:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:17 smithi170 ceph-mon[102832]: pgmap v13466: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 15 op/s 2023-09-10T06:15:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:17 smithi082 ceph-mon[98694]: pgmap v13466: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 15 op/s 2023-09-10T06:15:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:17 smithi098 ceph-mon[103124]: pgmap v13466: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 15 op/s 2023-09-10T06:15:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24949.450962 secs 2023-09-10T06:15:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24949.450962 secs 2023-09-10T06:15:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24949.450962 secs 2023-09-10T06:15:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:19 smithi170 ceph-mon[102832]: pgmap v13467: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T06:15:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24950.459522 secs 2023-09-10T06:15:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24951.072616 secs 2023-09-10T06:15:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:19 smithi082 ceph-mon[98694]: pgmap v13467: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T06:15:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24950.459522 secs 2023-09-10T06:15:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24951.072616 secs 2023-09-10T06:15:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:19 smithi098 ceph-mon[103124]: pgmap v13467: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T06:15:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24950.459522 secs 2023-09-10T06:15:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24951.072616 secs 2023-09-10T06:15:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:21 smithi170 ceph-mon[102832]: pgmap v13468: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T06:15:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:21 smithi082 ceph-mon[98694]: pgmap v13468: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T06:15:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:21 smithi098 ceph-mon[103124]: pgmap v13468: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T06:15:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:23 smithi170 ceph-mon[102832]: pgmap v13469: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T06:15:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24954.451065 secs 2023-09-10T06:15:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:23 smithi082 ceph-mon[98694]: pgmap v13469: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T06:15:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24954.451065 secs 2023-09-10T06:15:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:23 smithi098 ceph-mon[103124]: pgmap v13469: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T06:15:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24954.451065 secs 2023-09-10T06:15:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24955.459580 secs 2023-09-10T06:15:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24956.072682 secs 2023-09-10T06:15:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24955.459580 secs 2023-09-10T06:15:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24956.072682 secs 2023-09-10T06:15:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24955.459580 secs 2023-09-10T06:15:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24956.072682 secs 2023-09-10T06:15:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:25 smithi170 ceph-mon[102832]: pgmap v13470: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T06:15:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:25 smithi082 ceph-mon[98694]: pgmap v13470: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T06:15:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:25 smithi098 ceph-mon[103124]: pgmap v13470: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T06:15:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:27 smithi170 ceph-mon[102832]: pgmap v13471: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T06:15:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:27 smithi082 ceph-mon[98694]: pgmap v13471: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T06:15:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:27 smithi098 ceph-mon[103124]: pgmap v13471: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T06:15:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:29 smithi170 ceph-mon[102832]: pgmap v13472: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 9 op/s 2023-09-10T06:15:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24960.459661 secs 2023-09-10T06:15:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24961.072743 secs 2023-09-10T06:15:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:29 smithi082 ceph-mon[98694]: pgmap v13472: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 9 op/s 2023-09-10T06:15:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24960.459661 secs 2023-09-10T06:15:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24961.072743 secs 2023-09-10T06:15:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:29 smithi098 ceph-mon[103124]: pgmap v13472: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 9 op/s 2023-09-10T06:15:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24960.459661 secs 2023-09-10T06:15:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24961.072743 secs 2023-09-10T06:15:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:31 smithi082 ceph-mon[98694]: pgmap v13473: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 9 op/s 2023-09-10T06:15:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:15:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:31 smithi098 ceph-mon[103124]: pgmap v13473: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 9 op/s 2023-09-10T06:15:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:15:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:31 smithi170 ceph-mon[102832]: pgmap v13473: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 86 KiB/s wr, 9 op/s 2023-09-10T06:15:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:15:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:15:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:15:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:15:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:15:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:15:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:15:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:15:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:15:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:15:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:33 smithi082 ceph-mon[98694]: pgmap v13474: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 172 KiB/s wr, 14 op/s 2023-09-10T06:15:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:15:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:15:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:33 smithi098 ceph-mon[103124]: pgmap v13474: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 172 KiB/s wr, 14 op/s 2023-09-10T06:15:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:15:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:15:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:33 smithi170 ceph-mon[102832]: pgmap v13474: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 172 KiB/s wr, 14 op/s 2023-09-10T06:15:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:15:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:15:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24965.459721 secs 2023-09-10T06:15:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24966.072804 secs 2023-09-10T06:15:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24965.459721 secs 2023-09-10T06:15:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24966.072804 secs 2023-09-10T06:15:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24965.459721 secs 2023-09-10T06:15:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24966.072804 secs 2023-09-10T06:15:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:35 smithi082 ceph-mon[98694]: pgmap v13475: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 9 op/s 2023-09-10T06:15:36.299 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:35 smithi170 ceph-mon[102832]: pgmap v13475: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 9 op/s 2023-09-10T06:15:36.300 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:35 smithi098 ceph-mon[103124]: pgmap v13475: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 9 op/s 2023-09-10T06:15:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:37 smithi082 ceph-mon[98694]: pgmap v13476: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 377 KiB/s wr, 16 op/s 2023-09-10T06:15:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:37 smithi098 ceph-mon[103124]: pgmap v13476: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 377 KiB/s wr, 16 op/s 2023-09-10T06:15:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:37 smithi170 ceph-mon[102832]: pgmap v13476: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 377 KiB/s wr, 16 op/s 2023-09-10T06:15:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:38 smithi170 ceph-mon[102832]: pgmap v13477: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 377 KiB/s wr, 12 op/s 2023-09-10T06:15:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:38 smithi082 ceph-mon[98694]: pgmap v13477: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 377 KiB/s wr, 12 op/s 2023-09-10T06:15:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:38 smithi098 ceph-mon[103124]: pgmap v13477: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 377 KiB/s wr, 12 op/s 2023-09-10T06:15:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24970.459781 secs 2023-09-10T06:15:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24971.072875 secs 2023-09-10T06:15:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:15:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:15:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:15:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24970.459781 secs 2023-09-10T06:15:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24971.072875 secs 2023-09-10T06:15:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:15:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:15:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:15:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24970.459781 secs 2023-09-10T06:15:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24971.072875 secs 2023-09-10T06:15:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:15:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:15:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:15:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:40 smithi170 ceph-mon[102832]: pgmap v13478: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 377 KiB/s wr, 12 op/s 2023-09-10T06:15:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:15:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:15:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:15:41.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:40 smithi082 ceph-mon[98694]: pgmap v13478: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 377 KiB/s wr, 12 op/s 2023-09-10T06:15:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:15:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:15:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:15:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:40 smithi098 ceph-mon[103124]: pgmap v13478: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 377 KiB/s wr, 12 op/s 2023-09-10T06:15:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:15:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:15:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:15:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:42 smithi170 ceph-mon[102832]: pgmap v13479: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 462 KiB/s wr, 17 op/s 2023-09-10T06:15:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:42 smithi082 ceph-mon[98694]: pgmap v13479: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 462 KiB/s wr, 17 op/s 2023-09-10T06:15:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:42 smithi098 ceph-mon[103124]: pgmap v13479: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 462 KiB/s wr, 17 op/s 2023-09-10T06:15:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24975.459840 secs 2023-09-10T06:15:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:44 smithi170 ceph-mon[102832]: pgmap v13480: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 375 KiB/s wr, 12 op/s 2023-09-10T06:15:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:15:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:15:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24976.072936 secs 2023-09-10T06:15:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24975.459840 secs 2023-09-10T06:15:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:44 smithi082 ceph-mon[98694]: pgmap v13480: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 375 KiB/s wr, 12 op/s 2023-09-10T06:15:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:15:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:15:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24976.072936 secs 2023-09-10T06:15:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24975.459840 secs 2023-09-10T06:15:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:44 smithi098 ceph-mon[103124]: pgmap v13480: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 375 KiB/s wr, 12 op/s 2023-09-10T06:15:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:15:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:15:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24976.072936 secs 2023-09-10T06:15:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:15:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:15:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:15:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:15:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:15:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:15:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:47 smithi170 ceph-mon[102832]: pgmap v13481: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 756 KiB/s wr, 18 op/s 2023-09-10T06:15:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:47 smithi082 ceph-mon[98694]: pgmap v13481: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 756 KiB/s wr, 18 op/s 2023-09-10T06:15:48.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:47 smithi098 ceph-mon[103124]: pgmap v13481: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 756 KiB/s wr, 18 op/s 2023-09-10T06:15:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:49 smithi170 ceph-mon[102832]: pgmap v13482: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 465 KiB/s wr, 12 op/s 2023-09-10T06:15:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24980.459923 secs 2023-09-10T06:15:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24981.072999 secs 2023-09-10T06:15:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:49 smithi082 ceph-mon[98694]: pgmap v13482: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 465 KiB/s wr, 12 op/s 2023-09-10T06:15:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24980.459923 secs 2023-09-10T06:15:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24981.072999 secs 2023-09-10T06:15:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:49 smithi098 ceph-mon[103124]: pgmap v13482: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 465 KiB/s wr, 12 op/s 2023-09-10T06:15:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24980.459923 secs 2023-09-10T06:15:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24981.072999 secs 2023-09-10T06:15:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:15:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:15:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:15:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:15:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:15:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:15:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:51 smithi170 ceph-mon[102832]: pgmap v13483: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 465 KiB/s wr, 12 op/s 2023-09-10T06:15:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:51 smithi082 ceph-mon[98694]: pgmap v13483: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 465 KiB/s wr, 12 op/s 2023-09-10T06:15:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:51 smithi098 ceph-mon[103124]: pgmap v13483: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 465 KiB/s wr, 12 op/s 2023-09-10T06:15:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:15:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:15:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:15:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:15:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:15:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:15:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:53 smithi170 ceph-mon[102832]: pgmap v13484: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 515 KiB/s wr, 17 op/s 2023-09-10T06:15:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:53 smithi082 ceph-mon[98694]: pgmap v13484: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 515 KiB/s wr, 17 op/s 2023-09-10T06:15:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:53 smithi098 ceph-mon[103124]: pgmap v13484: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 515 KiB/s wr, 17 op/s 2023-09-10T06:15:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24985.460008 secs 2023-09-10T06:15:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24986.073063 secs 2023-09-10T06:15:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24985.460008 secs 2023-09-10T06:15:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24986.073063 secs 2023-09-10T06:15:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24985.460008 secs 2023-09-10T06:15:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24986.073063 secs 2023-09-10T06:15:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:55 smithi170 ceph-mon[102832]: pgmap v13485: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 431 KiB/s wr, 12 op/s 2023-09-10T06:15:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:55 smithi082 ceph-mon[98694]: pgmap v13485: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 431 KiB/s wr, 12 op/s 2023-09-10T06:15:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:55 smithi098 ceph-mon[103124]: pgmap v13485: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 431 KiB/s wr, 12 op/s 2023-09-10T06:15:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:57 smithi170 ceph-mon[102832]: pgmap v13486: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 611 KiB/s wr, 18 op/s 2023-09-10T06:15:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:57 smithi082 ceph-mon[98694]: pgmap v13486: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 611 KiB/s wr, 18 op/s 2023-09-10T06:15:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:57 smithi098 ceph-mon[103124]: pgmap v13486: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 611 KiB/s wr, 18 op/s 2023-09-10T06:15:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24989.451549 secs 2023-09-10T06:15:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24989.451549 secs 2023-09-10T06:15:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24989.451549 secs 2023-09-10T06:15:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:59 smithi170 ceph-mon[102832]: pgmap v13487: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T06:15:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24990.460092 secs 2023-09-10T06:15:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:15:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24991.073128 secs 2023-09-10T06:15:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:59 smithi082 ceph-mon[98694]: pgmap v13487: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T06:15:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24990.460092 secs 2023-09-10T06:15:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:15:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24991.073128 secs 2023-09-10T06:16:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:59 smithi098 ceph-mon[103124]: pgmap v13487: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T06:16:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24990.460092 secs 2023-09-10T06:16:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:15:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24991.073128 secs 2023-09-10T06:16:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:16:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:16:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:16:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:16:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:16:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:16:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:01 smithi170 ceph-mon[102832]: pgmap v13488: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T06:16:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:01 smithi082 ceph-mon[98694]: pgmap v13488: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T06:16:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:01 smithi098 ceph-mon[103124]: pgmap v13488: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T06:16:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:03 smithi170 ceph-mon[102832]: pgmap v13489: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 426 KiB/s wr, 17 op/s 2023-09-10T06:16:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24994.451669 secs 2023-09-10T06:16:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:03 smithi082 ceph-mon[98694]: pgmap v13489: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 426 KiB/s wr, 17 op/s 2023-09-10T06:16:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24994.451669 secs 2023-09-10T06:16:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:03 smithi098 ceph-mon[103124]: pgmap v13489: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 426 KiB/s wr, 17 op/s 2023-09-10T06:16:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24994.451669 secs 2023-09-10T06:16:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24995.460151 secs 2023-09-10T06:16:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:16:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 24996.073193 secs 2023-09-10T06:16:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24995.460151 secs 2023-09-10T06:16:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:16:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 24996.073193 secs 2023-09-10T06:16:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24995.460151 secs 2023-09-10T06:16:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:16:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 24996.073193 secs 2023-09-10T06:16:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:05 smithi170 ceph-mon[102832]: pgmap v13490: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 377 KiB/s wr, 11 op/s 2023-09-10T06:16:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:05 smithi082 ceph-mon[98694]: pgmap v13490: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 377 KiB/s wr, 11 op/s 2023-09-10T06:16:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:05 smithi098 ceph-mon[103124]: pgmap v13490: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 377 KiB/s wr, 11 op/s 2023-09-10T06:16:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:16:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:06.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:16:06.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:16:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:07 smithi170 ceph-mon[102832]: pgmap v13491: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 506 KiB/s wr, 32 op/s 2023-09-10T06:16:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:07 smithi082 ceph-mon[98694]: pgmap v13491: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 506 KiB/s wr, 32 op/s 2023-09-10T06:16:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:07 smithi098 ceph-mon[103124]: pgmap v13491: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 506 KiB/s wr, 32 op/s 2023-09-10T06:16:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 24999.451770 secs 2023-09-10T06:16:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 24999.451770 secs 2023-09-10T06:16:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 24999.451770 secs 2023-09-10T06:16:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:09 smithi082 ceph-mon[98694]: pgmap v13492: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 326 KiB/s wr, 26 op/s 2023-09-10T06:16:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25000.460236 secs 2023-09-10T06:16:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:09 smithi082 ceph-mon[98694]: osdmap e911: 12 total, 12 up, 12 in 2023-09-10T06:16:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25001.073260 secs 2023-09-10T06:16:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:09 smithi098 ceph-mon[103124]: pgmap v13492: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 326 KiB/s wr, 26 op/s 2023-09-10T06:16:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25000.460236 secs 2023-09-10T06:16:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:09 smithi098 ceph-mon[103124]: osdmap e911: 12 total, 12 up, 12 in 2023-09-10T06:16:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25001.073260 secs 2023-09-10T06:16:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:09 smithi170 ceph-mon[102832]: pgmap v13492: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 326 KiB/s wr, 26 op/s 2023-09-10T06:16:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25000.460236 secs 2023-09-10T06:16:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:09 smithi170 ceph-mon[102832]: osdmap e911: 12 total, 12 up, 12 in 2023-09-10T06:16:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25001.073260 secs 2023-09-10T06:16:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:11 smithi082 ceph-mon[98694]: pgmap v13494: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 391 KiB/s wr, 31 op/s 2023-09-10T06:16:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:11 smithi098 ceph-mon[103124]: pgmap v13494: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 391 KiB/s wr, 31 op/s 2023-09-10T06:16:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:11 smithi170 ceph-mon[102832]: pgmap v13494: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 391 KiB/s wr, 31 op/s 2023-09-10T06:16:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:13 smithi082 ceph-mon[98694]: pgmap v13495: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 178 KiB/s wr, 30 op/s 2023-09-10T06:16:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25004.451830 secs 2023-09-10T06:16:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:13 smithi098 ceph-mon[103124]: pgmap v13495: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 178 KiB/s wr, 30 op/s 2023-09-10T06:16:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25004.451830 secs 2023-09-10T06:16:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:13 smithi170 ceph-mon[102832]: pgmap v13495: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 178 KiB/s wr, 30 op/s 2023-09-10T06:16:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25004.451830 secs 2023-09-10T06:16:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25005.460307 secs 2023-09-10T06:16:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25006.073326 secs 2023-09-10T06:16:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25005.460307 secs 2023-09-10T06:16:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25006.073326 secs 2023-09-10T06:16:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25005.460307 secs 2023-09-10T06:16:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25006.073326 secs 2023-09-10T06:16:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:15 smithi082 ceph-mon[98694]: pgmap v13496: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 178 KiB/s wr, 30 op/s 2023-09-10T06:16:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:15 smithi082 ceph-mon[98694]: osdmap e912: 12 total, 12 up, 12 in 2023-09-10T06:16:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:15 smithi098 ceph-mon[103124]: pgmap v13496: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 178 KiB/s wr, 30 op/s 2023-09-10T06:16:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:15 smithi098 ceph-mon[103124]: osdmap e912: 12 total, 12 up, 12 in 2023-09-10T06:16:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:15 smithi170 ceph-mon[102832]: pgmap v13496: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 178 KiB/s wr, 30 op/s 2023-09-10T06:16:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:15 smithi170 ceph-mon[102832]: osdmap e912: 12 total, 12 up, 12 in 2023-09-10T06:16:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:17 smithi082 ceph-mon[98694]: pgmap v13498: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 15 op/s 2023-09-10T06:16:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:17 smithi098 ceph-mon[103124]: pgmap v13498: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 15 op/s 2023-09-10T06:16:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:17 smithi170 ceph-mon[102832]: pgmap v13498: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 15 op/s 2023-09-10T06:16:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25009.451890 secs 2023-09-10T06:16:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25009.451890 secs 2023-09-10T06:16:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25009.451890 secs 2023-09-10T06:16:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:19 smithi082 ceph-mon[98694]: pgmap v13499: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T06:16:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25010.460365 secs 2023-09-10T06:16:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25011.073390 secs 2023-09-10T06:16:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:19 smithi098 ceph-mon[103124]: pgmap v13499: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T06:16:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25010.460365 secs 2023-09-10T06:16:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25011.073390 secs 2023-09-10T06:16:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:19 smithi170 ceph-mon[102832]: pgmap v13499: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-09-10T06:16:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25010.460365 secs 2023-09-10T06:16:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25011.073390 secs 2023-09-10T06:16:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:21 smithi098 ceph-mon[103124]: pgmap v13500: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T06:16:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:21 smithi170 ceph-mon[102832]: pgmap v13500: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T06:16:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:21 smithi082 ceph-mon[98694]: pgmap v13500: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T06:16:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:22 smithi098 ceph-mon[103124]: pgmap v13501: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:16:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:22 smithi170 ceph-mon[102832]: pgmap v13501: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:16:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:22 smithi082 ceph-mon[98694]: pgmap v13501: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:16:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25014.451951 secs 2023-09-10T06:16:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25014.451951 secs 2023-09-10T06:16:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25014.451951 secs 2023-09-10T06:16:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25015.460427 secs 2023-09-10T06:16:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:24 smithi098 ceph-mon[103124]: pgmap v13502: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:16:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25016.073455 secs 2023-09-10T06:16:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25015.460427 secs 2023-09-10T06:16:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:24 smithi170 ceph-mon[102832]: pgmap v13502: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:16:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25016.073455 secs 2023-09-10T06:16:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25015.460427 secs 2023-09-10T06:16:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:24 smithi082 ceph-mon[98694]: pgmap v13502: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:16:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25016.073455 secs 2023-09-10T06:16:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:27 smithi170 ceph-mon[102832]: pgmap v13503: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T06:16:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:27 smithi082 ceph-mon[98694]: pgmap v13503: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T06:16:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:27 smithi098 ceph-mon[103124]: pgmap v13503: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T06:16:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:29 smithi170 ceph-mon[102832]: pgmap v13504: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 9 op/s 2023-09-10T06:16:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25020.460489 secs 2023-09-10T06:16:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25021.073483 secs 2023-09-10T06:16:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:29 smithi082 ceph-mon[98694]: pgmap v13504: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 9 op/s 2023-09-10T06:16:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25020.460489 secs 2023-09-10T06:16:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25021.073483 secs 2023-09-10T06:16:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:29 smithi098 ceph-mon[103124]: pgmap v13504: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 9 op/s 2023-09-10T06:16:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25020.460489 secs 2023-09-10T06:16:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25021.073483 secs 2023-09-10T06:16:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:31 smithi170 ceph-mon[102832]: pgmap v13505: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 9 op/s 2023-09-10T06:16:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:16:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:31 smithi082 ceph-mon[98694]: pgmap v13505: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 9 op/s 2023-09-10T06:16:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:16:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:31 smithi098 ceph-mon[103124]: pgmap v13505: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 9 op/s 2023-09-10T06:16:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:16:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:16:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:16:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:16:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:16:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:16:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:16:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:33 smithi170 ceph-mon[102832]: pgmap v13506: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 210 KiB/s wr, 14 op/s 2023-09-10T06:16:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:33 smithi082 ceph-mon[98694]: pgmap v13506: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 210 KiB/s wr, 14 op/s 2023-09-10T06:16:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:33 smithi098 ceph-mon[103124]: pgmap v13506: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 210 KiB/s wr, 14 op/s 2023-09-10T06:16:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25025.460493 secs 2023-09-10T06:16:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:16:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25026.073546 secs 2023-09-10T06:16:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25025.460493 secs 2023-09-10T06:16:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:16:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25026.073546 secs 2023-09-10T06:16:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25025.460493 secs 2023-09-10T06:16:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:16:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25026.073546 secs 2023-09-10T06:16:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:35 smithi170 ceph-mon[102832]: pgmap v13507: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 124 KiB/s wr, 9 op/s 2023-09-10T06:16:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:35 smithi082 ceph-mon[98694]: pgmap v13507: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 124 KiB/s wr, 9 op/s 2023-09-10T06:16:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:35 smithi098 ceph-mon[103124]: pgmap v13507: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 124 KiB/s wr, 9 op/s 2023-09-10T06:16:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:16:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:16:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:16:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:37 smithi082 ceph-mon[98694]: pgmap v13508: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 377 KiB/s wr, 17 op/s 2023-09-10T06:16:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:37 smithi098 ceph-mon[103124]: pgmap v13508: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 377 KiB/s wr, 17 op/s 2023-09-10T06:16:38.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:37 smithi170 ceph-mon[102832]: pgmap v13508: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 377 KiB/s wr, 17 op/s 2023-09-10T06:16:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:39 smithi082 ceph-mon[98694]: pgmap v13509: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 377 KiB/s wr, 12 op/s 2023-09-10T06:16:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25030.460576 secs 2023-09-10T06:16:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25031.073608 secs 2023-09-10T06:16:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:39 smithi098 ceph-mon[103124]: pgmap v13509: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 377 KiB/s wr, 12 op/s 2023-09-10T06:16:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25030.460576 secs 2023-09-10T06:16:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25031.073608 secs 2023-09-10T06:16:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:39 smithi170 ceph-mon[102832]: pgmap v13509: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 377 KiB/s wr, 12 op/s 2023-09-10T06:16:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25030.460576 secs 2023-09-10T06:16:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25031.073608 secs 2023-09-10T06:16:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:16:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:16:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:16:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:41 smithi082 ceph-mon[98694]: pgmap v13510: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 377 KiB/s wr, 12 op/s 2023-09-10T06:16:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:16:41.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:16:41.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:16:41.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:16:41.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:16:41.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:16:41.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:16:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:41 smithi098 ceph-mon[103124]: pgmap v13510: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 377 KiB/s wr, 12 op/s 2023-09-10T06:16:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:16:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:16:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:16:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:16:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:16:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:16:42.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:16:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:41 smithi170 ceph-mon[102832]: pgmap v13510: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 377 KiB/s wr, 12 op/s 2023-09-10T06:16:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:16:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:16:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:16:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:16:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:16:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:16:42.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:16:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:43 smithi082 ceph-mon[98694]: pgmap v13511: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 395 KiB/s wr, 18 op/s 2023-09-10T06:16:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:43 smithi098 ceph-mon[103124]: pgmap v13511: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 395 KiB/s wr, 18 op/s 2023-09-10T06:16:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:43 smithi170 ceph-mon[102832]: pgmap v13511: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 395 KiB/s wr, 18 op/s 2023-09-10T06:16:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25035.460635 secs 2023-09-10T06:16:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:16:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25036.073672 secs 2023-09-10T06:16:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25035.460635 secs 2023-09-10T06:16:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:16:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25036.073672 secs 2023-09-10T06:16:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25035.460635 secs 2023-09-10T06:16:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:16:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25036.073672 secs 2023-09-10T06:16:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:45 smithi098 ceph-mon[103124]: pgmap v13512: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 271 KiB/s wr, 12 op/s 2023-09-10T06:16:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:16:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:45 smithi170 ceph-mon[102832]: pgmap v13512: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 271 KiB/s wr, 12 op/s 2023-09-10T06:16:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:16:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:45 smithi082 ceph-mon[98694]: pgmap v13512: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 271 KiB/s wr, 12 op/s 2023-09-10T06:16:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:16:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:47 smithi098 ceph-mon[103124]: pgmap v13513: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 741 KiB/s wr, 19 op/s 2023-09-10T06:16:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:47 smithi170 ceph-mon[102832]: pgmap v13513: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 741 KiB/s wr, 19 op/s 2023-09-10T06:16:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:47 smithi082 ceph-mon[98694]: pgmap v13513: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 741 KiB/s wr, 19 op/s 2023-09-10T06:16:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:49 smithi098 ceph-mon[103124]: pgmap v13514: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 488 KiB/s wr, 12 op/s 2023-09-10T06:16:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25040.460693 secs 2023-09-10T06:16:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25041.073735 secs 2023-09-10T06:16:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:49 smithi170 ceph-mon[102832]: pgmap v13514: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 488 KiB/s wr, 12 op/s 2023-09-10T06:16:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25040.460693 secs 2023-09-10T06:16:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25041.073735 secs 2023-09-10T06:16:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:49 smithi082 ceph-mon[98694]: pgmap v13514: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 488 KiB/s wr, 12 op/s 2023-09-10T06:16:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25040.460693 secs 2023-09-10T06:16:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25041.073735 secs 2023-09-10T06:16:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:51 smithi098 ceph-mon[103124]: pgmap v13515: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 488 KiB/s wr, 12 op/s 2023-09-10T06:16:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:51 smithi170 ceph-mon[102832]: pgmap v13515: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 488 KiB/s wr, 12 op/s 2023-09-10T06:16:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:51 smithi082 ceph-mon[98694]: pgmap v13515: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 488 KiB/s wr, 12 op/s 2023-09-10T06:16:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:53 smithi098 ceph-mon[103124]: pgmap v13516: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 597 KiB/s wr, 18 op/s 2023-09-10T06:16:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:16:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:53 smithi170 ceph-mon[102832]: pgmap v13516: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 597 KiB/s wr, 18 op/s 2023-09-10T06:16:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:16:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:53 smithi082 ceph-mon[98694]: pgmap v13516: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 597 KiB/s wr, 18 op/s 2023-09-10T06:16:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:16:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25045.460753 secs 2023-09-10T06:16:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:54 smithi098 ceph-mon[103124]: pgmap v13517: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 578 KiB/s wr, 13 op/s 2023-09-10T06:16:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25046.073798 secs 2023-09-10T06:16:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25045.460753 secs 2023-09-10T06:16:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:54 smithi170 ceph-mon[102832]: pgmap v13517: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 578 KiB/s wr, 13 op/s 2023-09-10T06:16:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25046.073798 secs 2023-09-10T06:16:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25045.460753 secs 2023-09-10T06:16:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:54 smithi082 ceph-mon[98694]: pgmap v13517: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 578 KiB/s wr, 13 op/s 2023-09-10T06:16:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25046.073798 secs 2023-09-10T06:16:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:16:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:56 smithi098 ceph-mon[103124]: pgmap v13518: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 668 KiB/s wr, 19 op/s 2023-09-10T06:16:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:16:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:56 smithi170 ceph-mon[102832]: pgmap v13518: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 668 KiB/s wr, 19 op/s 2023-09-10T06:16:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:16:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:16:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:56 smithi082 ceph-mon[98694]: pgmap v13518: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 668 KiB/s wr, 19 op/s 2023-09-10T06:16:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:58 smithi170 ceph-mon[102832]: pgmap v13519: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 198 KiB/s wr, 12 op/s 2023-09-10T06:16:59.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25049.452461 secs 2023-09-10T06:16:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:58 smithi082 ceph-mon[98694]: pgmap v13519: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 198 KiB/s wr, 12 op/s 2023-09-10T06:16:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25049.452461 secs 2023-09-10T06:16:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:58 smithi098 ceph-mon[103124]: pgmap v13519: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 198 KiB/s wr, 12 op/s 2023-09-10T06:16:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25049.452461 secs 2023-09-10T06:17:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25050.460804 secs 2023-09-10T06:17:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:16:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25051.073863 secs 2023-09-10T06:17:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25050.460804 secs 2023-09-10T06:17:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:16:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25051.073863 secs 2023-09-10T06:17:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25050.460804 secs 2023-09-10T06:17:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:16:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25051.073863 secs 2023-09-10T06:17:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:00 smithi170 ceph-mon[102832]: pgmap v13520: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 198 KiB/s wr, 12 op/s 2023-09-10T06:17:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:17:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:17:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:17:01.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:00 smithi082 ceph-mon[98694]: pgmap v13520: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 198 KiB/s wr, 12 op/s 2023-09-10T06:17:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:17:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:17:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:17:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:00 smithi098 ceph-mon[103124]: pgmap v13520: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 198 KiB/s wr, 12 op/s 2023-09-10T06:17:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:17:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:17:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:17:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:02 smithi170 ceph-mon[102832]: pgmap v13521: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 381 KiB/s wr, 18 op/s 2023-09-10T06:17:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:02 smithi082 ceph-mon[98694]: pgmap v13521: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 381 KiB/s wr, 18 op/s 2023-09-10T06:17:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:02 smithi098 ceph-mon[103124]: pgmap v13521: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 381 KiB/s wr, 18 op/s 2023-09-10T06:17:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25054.452560 secs 2023-09-10T06:17:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25054.452560 secs 2023-09-10T06:17:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25054.452560 secs 2023-09-10T06:17:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25055.460863 secs 2023-09-10T06:17:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:04 smithi170 ceph-mon[102832]: pgmap v13522: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 273 KiB/s wr, 11 op/s 2023-09-10T06:17:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:17:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:17:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25056.073927 secs 2023-09-10T06:17:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25055.460863 secs 2023-09-10T06:17:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:04 smithi082 ceph-mon[98694]: pgmap v13522: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 273 KiB/s wr, 11 op/s 2023-09-10T06:17:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:17:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:17:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25056.073927 secs 2023-09-10T06:17:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25055.460863 secs 2023-09-10T06:17:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:04 smithi098 ceph-mon[103124]: pgmap v13522: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 273 KiB/s wr, 11 op/s 2023-09-10T06:17:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:17:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:17:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25056.073927 secs 2023-09-10T06:17:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:17:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:17:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:17:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:17:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:17:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:17:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:07 smithi170 ceph-mon[102832]: pgmap v13523: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 403 KiB/s wr, 31 op/s 2023-09-10T06:17:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:07 smithi082 ceph-mon[98694]: pgmap v13523: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 403 KiB/s wr, 31 op/s 2023-09-10T06:17:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:07 smithi098 ceph-mon[103124]: pgmap v13523: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 403 KiB/s wr, 31 op/s 2023-09-10T06:17:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25059.452619 secs 2023-09-10T06:17:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25059.452619 secs 2023-09-10T06:17:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25059.452619 secs 2023-09-10T06:17:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:09 smithi170 ceph-mon[102832]: pgmap v13524: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 313 KiB/s wr, 26 op/s 2023-09-10T06:17:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25060.460923 secs 2023-09-10T06:17:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:09 smithi170 ceph-mon[102832]: osdmap e913: 12 total, 12 up, 12 in 2023-09-10T06:17:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25061.073994 secs 2023-09-10T06:17:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:09 smithi082 ceph-mon[98694]: pgmap v13524: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 313 KiB/s wr, 26 op/s 2023-09-10T06:17:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25060.460923 secs 2023-09-10T06:17:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:09 smithi082 ceph-mon[98694]: osdmap e913: 12 total, 12 up, 12 in 2023-09-10T06:17:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25061.073994 secs 2023-09-10T06:17:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:09 smithi098 ceph-mon[103124]: pgmap v13524: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 313 KiB/s wr, 26 op/s 2023-09-10T06:17:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25060.460923 secs 2023-09-10T06:17:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:09 smithi098 ceph-mon[103124]: osdmap e913: 12 total, 12 up, 12 in 2023-09-10T06:17:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25061.073994 secs 2023-09-10T06:17:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:11 smithi170 ceph-mon[102832]: pgmap v13526: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 376 KiB/s wr, 31 op/s 2023-09-10T06:17:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:11 smithi082 ceph-mon[98694]: pgmap v13526: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 376 KiB/s wr, 31 op/s 2023-09-10T06:17:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:11 smithi098 ceph-mon[103124]: pgmap v13526: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 376 KiB/s wr, 31 op/s 2023-09-10T06:17:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:13 smithi170 ceph-mon[102832]: pgmap v13527: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 180 KiB/s wr, 30 op/s 2023-09-10T06:17:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25064.452683 secs 2023-09-10T06:17:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:13 smithi082 ceph-mon[98694]: pgmap v13527: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 180 KiB/s wr, 30 op/s 2023-09-10T06:17:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25064.452683 secs 2023-09-10T06:17:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:13 smithi098 ceph-mon[103124]: pgmap v13527: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 180 KiB/s wr, 30 op/s 2023-09-10T06:17:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25064.452683 secs 2023-09-10T06:17:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25065.460981 secs 2023-09-10T06:17:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25066.074058 secs 2023-09-10T06:17:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25065.460981 secs 2023-09-10T06:17:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25066.074058 secs 2023-09-10T06:17:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25065.460981 secs 2023-09-10T06:17:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25066.074058 secs 2023-09-10T06:17:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:15 smithi170 ceph-mon[102832]: pgmap v13528: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 180 KiB/s wr, 30 op/s 2023-09-10T06:17:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:15 smithi170 ceph-mon[102832]: osdmap e914: 12 total, 12 up, 12 in 2023-09-10T06:17:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:15 smithi082 ceph-mon[98694]: pgmap v13528: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 180 KiB/s wr, 30 op/s 2023-09-10T06:17:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:15 smithi082 ceph-mon[98694]: osdmap e914: 12 total, 12 up, 12 in 2023-09-10T06:17:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:15 smithi098 ceph-mon[103124]: pgmap v13528: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 180 KiB/s wr, 30 op/s 2023-09-10T06:17:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:15 smithi098 ceph-mon[103124]: osdmap e914: 12 total, 12 up, 12 in 2023-09-10T06:17:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:17 smithi170 ceph-mon[102832]: pgmap v13530: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 15 op/s 2023-09-10T06:17:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:17 smithi082 ceph-mon[98694]: pgmap v13530: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 15 op/s 2023-09-10T06:17:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:17 smithi098 ceph-mon[103124]: pgmap v13530: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 15 op/s 2023-09-10T06:17:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25069.452772 secs 2023-09-10T06:17:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25069.452772 secs 2023-09-10T06:17:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25069.452772 secs 2023-09-10T06:17:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:19 smithi170 ceph-mon[102832]: pgmap v13531: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 14 op/s 2023-09-10T06:17:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25070.461041 secs 2023-09-10T06:17:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25071.074118 secs 2023-09-10T06:17:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:19 smithi082 ceph-mon[98694]: pgmap v13531: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 14 op/s 2023-09-10T06:17:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25070.461041 secs 2023-09-10T06:17:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25071.074118 secs 2023-09-10T06:17:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:19 smithi098 ceph-mon[103124]: pgmap v13531: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 14 op/s 2023-09-10T06:17:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25070.461041 secs 2023-09-10T06:17:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25071.074118 secs 2023-09-10T06:17:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:21 smithi170 ceph-mon[102832]: pgmap v13532: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T06:17:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:21 smithi082 ceph-mon[98694]: pgmap v13532: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T06:17:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:21 smithi098 ceph-mon[103124]: pgmap v13532: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T06:17:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:23 smithi170 ceph-mon[102832]: pgmap v13533: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:17:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25074.452834 secs 2023-09-10T06:17:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:23 smithi082 ceph-mon[98694]: pgmap v13533: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:17:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25074.452834 secs 2023-09-10T06:17:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:23 smithi098 ceph-mon[103124]: pgmap v13533: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:17:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25074.452834 secs 2023-09-10T06:17:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25075.461098 secs 2023-09-10T06:17:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25076.074182 secs 2023-09-10T06:17:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25075.461098 secs 2023-09-10T06:17:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25076.074182 secs 2023-09-10T06:17:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25075.461098 secs 2023-09-10T06:17:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25076.074182 secs 2023-09-10T06:17:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:25 smithi170 ceph-mon[102832]: pgmap v13534: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:17:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:25 smithi082 ceph-mon[98694]: pgmap v13534: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:17:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:25 smithi098 ceph-mon[103124]: pgmap v13534: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:17:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:27 smithi170 ceph-mon[102832]: pgmap v13535: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T06:17:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:27 smithi082 ceph-mon[98694]: pgmap v13535: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T06:17:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:27 smithi098 ceph-mon[103124]: pgmap v13535: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 11 op/s 2023-09-10T06:17:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:29 smithi170 ceph-mon[102832]: pgmap v13536: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 9 op/s 2023-09-10T06:17:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25080.461157 secs 2023-09-10T06:17:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25081.074266 secs 2023-09-10T06:17:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:29 smithi082 ceph-mon[98694]: pgmap v13536: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 9 op/s 2023-09-10T06:17:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25080.461157 secs 2023-09-10T06:17:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25081.074266 secs 2023-09-10T06:17:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:29 smithi098 ceph-mon[103124]: pgmap v13536: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 9 op/s 2023-09-10T06:17:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25080.461157 secs 2023-09-10T06:17:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25081.074266 secs 2023-09-10T06:17:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:31 smithi082 ceph-mon[98694]: pgmap v13537: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 9 op/s 2023-09-10T06:17:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:17:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:31 smithi098 ceph-mon[103124]: pgmap v13537: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 9 op/s 2023-09-10T06:17:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:17:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:31 smithi170 ceph-mon[102832]: pgmap v13537: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 9 op/s 2023-09-10T06:17:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:17:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:17:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:17:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:17:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:17:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:17:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:17:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:17:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:17:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:17:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:33 smithi082 ceph-mon[98694]: pgmap v13538: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 174 KiB/s wr, 14 op/s 2023-09-10T06:17:34.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:33 smithi098 ceph-mon[103124]: pgmap v13538: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 174 KiB/s wr, 14 op/s 2023-09-10T06:17:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:33 smithi170 ceph-mon[102832]: pgmap v13538: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 174 KiB/s wr, 14 op/s 2023-09-10T06:17:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25085.461216 secs 2023-09-10T06:17:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25086.074326 secs 2023-09-10T06:17:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25085.461216 secs 2023-09-10T06:17:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25086.074326 secs 2023-09-10T06:17:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25085.461216 secs 2023-09-10T06:17:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25086.074326 secs 2023-09-10T06:17:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:35 smithi082 ceph-mon[98694]: pgmap v13539: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 88 KiB/s wr, 9 op/s 2023-09-10T06:17:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:35 smithi098 ceph-mon[103124]: pgmap v13539: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 88 KiB/s wr, 9 op/s 2023-09-10T06:17:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:35 smithi170 ceph-mon[102832]: pgmap v13539: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 88 KiB/s wr, 9 op/s 2023-09-10T06:17:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:17:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:17:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:17:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:17:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:17:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:17:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:37 smithi082 ceph-mon[98694]: pgmap v13540: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 379 KiB/s wr, 16 op/s 2023-09-10T06:17:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:17:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:17:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:37 smithi098 ceph-mon[103124]: pgmap v13540: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 379 KiB/s wr, 16 op/s 2023-09-10T06:17:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:17:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:17:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:37 smithi170 ceph-mon[102832]: pgmap v13540: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 379 KiB/s wr, 16 op/s 2023-09-10T06:17:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:17:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:17:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:39 smithi098 ceph-mon[103124]: pgmap v13541: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 379 KiB/s wr, 11 op/s 2023-09-10T06:17:40.016 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25090.461304 secs 2023-09-10T06:17:40.016 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25091.074386 secs 2023-09-10T06:17:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:39 smithi170 ceph-mon[102832]: pgmap v13541: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 379 KiB/s wr, 11 op/s 2023-09-10T06:17:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25090.461304 secs 2023-09-10T06:17:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25091.074386 secs 2023-09-10T06:17:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:39 smithi082 ceph-mon[98694]: pgmap v13541: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 379 KiB/s wr, 11 op/s 2023-09-10T06:17:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25090.461304 secs 2023-09-10T06:17:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25091.074386 secs 2023-09-10T06:17:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:41 smithi098 ceph-mon[103124]: pgmap v13542: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 379 KiB/s wr, 11 op/s 2023-09-10T06:17:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:17:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:41 smithi170 ceph-mon[102832]: pgmap v13542: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 379 KiB/s wr, 11 op/s 2023-09-10T06:17:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:17:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:41 smithi082 ceph-mon[98694]: pgmap v13542: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 379 KiB/s wr, 11 op/s 2023-09-10T06:17:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:17:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:17:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:17:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:17:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:17:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:17:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:17:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:17:43.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:17:43.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:17:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:43 smithi098 ceph-mon[103124]: pgmap v13543: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 398 KiB/s wr, 18 op/s 2023-09-10T06:17:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:43 smithi170 ceph-mon[102832]: pgmap v13543: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 398 KiB/s wr, 18 op/s 2023-09-10T06:17:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:43 smithi082 ceph-mon[98694]: pgmap v13543: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 398 KiB/s wr, 18 op/s 2023-09-10T06:17:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25095.461362 secs 2023-09-10T06:17:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:17:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:17:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25096.074448 secs 2023-09-10T06:17:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25095.461362 secs 2023-09-10T06:17:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:17:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:17:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25096.074448 secs 2023-09-10T06:17:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25095.461362 secs 2023-09-10T06:17:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:17:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:17:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25096.074448 secs 2023-09-10T06:17:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:45 smithi098 ceph-mon[103124]: pgmap v13544: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 310 KiB/s wr, 12 op/s 2023-09-10T06:17:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:17:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:17:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:45 smithi170 ceph-mon[102832]: pgmap v13544: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 310 KiB/s wr, 12 op/s 2023-09-10T06:17:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:17:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:17:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:45 smithi082 ceph-mon[98694]: pgmap v13544: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 310 KiB/s wr, 12 op/s 2023-09-10T06:17:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:17:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:17:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:46 smithi098 ceph-mon[103124]: pgmap v13545: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 775 KiB/s wr, 19 op/s 2023-09-10T06:17:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:46 smithi170 ceph-mon[102832]: pgmap v13545: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 775 KiB/s wr, 19 op/s 2023-09-10T06:17:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:46 smithi082 ceph-mon[98694]: pgmap v13545: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 775 KiB/s wr, 19 op/s 2023-09-10T06:17:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:48 smithi170 ceph-mon[102832]: pgmap v13546: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 483 KiB/s wr, 13 op/s 2023-09-10T06:17:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:48 smithi082 ceph-mon[98694]: pgmap v13546: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 483 KiB/s wr, 13 op/s 2023-09-10T06:17:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:48 smithi098 ceph-mon[103124]: pgmap v13546: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 483 KiB/s wr, 13 op/s 2023-09-10T06:17:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25100.461415 secs 2023-09-10T06:17:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25101.074490 secs 2023-09-10T06:17:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25100.461415 secs 2023-09-10T06:17:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25101.074490 secs 2023-09-10T06:17:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25100.461415 secs 2023-09-10T06:17:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25101.074490 secs 2023-09-10T06:17:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:51 smithi170 ceph-mon[102832]: pgmap v13547: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 483 KiB/s wr, 13 op/s 2023-09-10T06:17:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:51 smithi082 ceph-mon[98694]: pgmap v13547: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 483 KiB/s wr, 13 op/s 2023-09-10T06:17:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:51 smithi098 ceph-mon[103124]: pgmap v13547: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 483 KiB/s wr, 13 op/s 2023-09-10T06:17:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:53 smithi170 ceph-mon[102832]: pgmap v13548: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 573 KiB/s wr, 19 op/s 2023-09-10T06:17:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:53 smithi082 ceph-mon[98694]: pgmap v13548: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 573 KiB/s wr, 19 op/s 2023-09-10T06:17:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:53 smithi098 ceph-mon[103124]: pgmap v13548: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 573 KiB/s wr, 19 op/s 2023-09-10T06:17:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25105.461505 secs 2023-09-10T06:17:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25106.074550 secs 2023-09-10T06:17:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25105.461505 secs 2023-09-10T06:17:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25106.074550 secs 2023-09-10T06:17:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25105.461505 secs 2023-09-10T06:17:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25106.074550 secs 2023-09-10T06:17:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:55 smithi170 ceph-mon[102832]: pgmap v13549: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 555 KiB/s wr, 12 op/s 2023-09-10T06:17:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:55 smithi082 ceph-mon[98694]: pgmap v13549: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 555 KiB/s wr, 12 op/s 2023-09-10T06:17:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:55 smithi098 ceph-mon[103124]: pgmap v13549: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 555 KiB/s wr, 12 op/s 2023-09-10T06:17:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:57 smithi170 ceph-mon[102832]: pgmap v13550: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 670 KiB/s wr, 18 op/s 2023-09-10T06:17:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:57 smithi082 ceph-mon[98694]: pgmap v13550: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 670 KiB/s wr, 18 op/s 2023-09-10T06:17:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:57 smithi098 ceph-mon[103124]: pgmap v13550: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 670 KiB/s wr, 18 op/s 2023-09-10T06:17:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:17:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:17:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25109.453333 secs 2023-09-10T06:17:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:17:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:17:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25109.453333 secs 2023-09-10T06:17:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:17:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:17:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25109.453333 secs 2023-09-10T06:17:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:59 smithi170 ceph-mon[102832]: pgmap v13551: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 206 KiB/s wr, 11 op/s 2023-09-10T06:17:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25110.461565 secs 2023-09-10T06:17:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:17:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25111.074614 secs 2023-09-10T06:17:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:59 smithi082 ceph-mon[98694]: pgmap v13551: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 206 KiB/s wr, 11 op/s 2023-09-10T06:17:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25110.461565 secs 2023-09-10T06:17:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:17:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25111.074614 secs 2023-09-10T06:18:00.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:59 smithi098 ceph-mon[103124]: pgmap v13551: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 206 KiB/s wr, 11 op/s 2023-09-10T06:18:00.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25110.461565 secs 2023-09-10T06:18:00.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:17:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25111.074614 secs 2023-09-10T06:18:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:18:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:18:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:18:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:18:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:18:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:18:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:18:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:18:01.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:18:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:01 smithi170 ceph-mon[102832]: pgmap v13552: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 206 KiB/s wr, 11 op/s 2023-09-10T06:18:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:01 smithi082 ceph-mon[98694]: pgmap v13552: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 206 KiB/s wr, 11 op/s 2023-09-10T06:18:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:01 smithi098 ceph-mon[103124]: pgmap v13552: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 206 KiB/s wr, 11 op/s 2023-09-10T06:18:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:03 smithi170 ceph-mon[102832]: pgmap v13553: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 390 KiB/s wr, 17 op/s 2023-09-10T06:18:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25114.453393 secs 2023-09-10T06:18:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:03 smithi082 ceph-mon[98694]: pgmap v13553: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 390 KiB/s wr, 17 op/s 2023-09-10T06:18:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25114.453393 secs 2023-09-10T06:18:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:03 smithi098 ceph-mon[103124]: pgmap v13553: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 390 KiB/s wr, 17 op/s 2023-09-10T06:18:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25114.453393 secs 2023-09-10T06:18:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25115.461627 secs 2023-09-10T06:18:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25116.074677 secs 2023-09-10T06:18:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25115.461627 secs 2023-09-10T06:18:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25116.074677 secs 2023-09-10T06:18:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25115.461627 secs 2023-09-10T06:18:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25116.074677 secs 2023-09-10T06:18:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:05 smithi082 ceph-mon[98694]: pgmap v13554: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 300 KiB/s wr, 11 op/s 2023-09-10T06:18:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:05 smithi098 ceph-mon[103124]: pgmap v13554: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 300 KiB/s wr, 11 op/s 2023-09-10T06:18:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:05 smithi170 ceph-mon[102832]: pgmap v13554: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 300 KiB/s wr, 11 op/s 2023-09-10T06:18:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:07 smithi082 ceph-mon[98694]: pgmap v13555: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 430 KiB/s wr, 31 op/s 2023-09-10T06:18:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:07 smithi098 ceph-mon[103124]: pgmap v13555: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 430 KiB/s wr, 31 op/s 2023-09-10T06:18:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:07 smithi170 ceph-mon[102832]: pgmap v13555: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 430 KiB/s wr, 31 op/s 2023-09-10T06:18:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25119.453453 secs 2023-09-10T06:18:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25119.453453 secs 2023-09-10T06:18:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25119.453453 secs 2023-09-10T06:18:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:09 smithi082 ceph-mon[98694]: pgmap v13556: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 314 KiB/s wr, 26 op/s 2023-09-10T06:18:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25120.461686 secs 2023-09-10T06:18:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:09 smithi082 ceph-mon[98694]: osdmap e915: 12 total, 12 up, 12 in 2023-09-10T06:18:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25121.074738 secs 2023-09-10T06:18:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:09 smithi098 ceph-mon[103124]: pgmap v13556: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 314 KiB/s wr, 26 op/s 2023-09-10T06:18:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25120.461686 secs 2023-09-10T06:18:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:09 smithi098 ceph-mon[103124]: osdmap e915: 12 total, 12 up, 12 in 2023-09-10T06:18:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25121.074738 secs 2023-09-10T06:18:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:09 smithi170 ceph-mon[102832]: pgmap v13556: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 314 KiB/s wr, 26 op/s 2023-09-10T06:18:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25120.461686 secs 2023-09-10T06:18:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:09 smithi170 ceph-mon[102832]: osdmap e915: 12 total, 12 up, 12 in 2023-09-10T06:18:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25121.074738 secs 2023-09-10T06:18:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:11 smithi082 ceph-mon[98694]: pgmap v13558: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 377 KiB/s wr, 31 op/s 2023-09-10T06:18:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:11 smithi098 ceph-mon[103124]: pgmap v13558: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 377 KiB/s wr, 31 op/s 2023-09-10T06:18:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:11 smithi170 ceph-mon[102832]: pgmap v13558: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 377 KiB/s wr, 31 op/s 2023-09-10T06:18:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:13 smithi082 ceph-mon[98694]: pgmap v13559: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 181 KiB/s wr, 31 op/s 2023-09-10T06:18:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25124.453513 secs 2023-09-10T06:18:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:13 smithi098 ceph-mon[103124]: pgmap v13559: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 181 KiB/s wr, 31 op/s 2023-09-10T06:18:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25124.453513 secs 2023-09-10T06:18:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:13 smithi170 ceph-mon[102832]: pgmap v13559: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 181 KiB/s wr, 31 op/s 2023-09-10T06:18:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25124.453513 secs 2023-09-10T06:18:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25125.461742 secs 2023-09-10T06:18:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25126.074804 secs 2023-09-10T06:18:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25125.461742 secs 2023-09-10T06:18:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25126.074804 secs 2023-09-10T06:18:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25125.461742 secs 2023-09-10T06:18:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25126.074804 secs 2023-09-10T06:18:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:15 smithi082 ceph-mon[98694]: pgmap v13560: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 181 KiB/s wr, 31 op/s 2023-09-10T06:18:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:15 smithi082 ceph-mon[98694]: osdmap e916: 12 total, 12 up, 12 in 2023-09-10T06:18:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:15 smithi098 ceph-mon[103124]: pgmap v13560: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 181 KiB/s wr, 31 op/s 2023-09-10T06:18:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:15 smithi098 ceph-mon[103124]: osdmap e916: 12 total, 12 up, 12 in 2023-09-10T06:18:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:15 smithi170 ceph-mon[102832]: pgmap v13560: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 181 KiB/s wr, 31 op/s 2023-09-10T06:18:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:15 smithi170 ceph-mon[102832]: osdmap e916: 12 total, 12 up, 12 in 2023-09-10T06:18:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:18:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:18:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:18:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:18:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:18:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:18:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:17 smithi098 ceph-mon[103124]: pgmap v13562: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 16 op/s 2023-09-10T06:18:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:17 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:18:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:17 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:18:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:17 smithi170 ceph-mon[102832]: pgmap v13562: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 16 op/s 2023-09-10T06:18:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:17 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:18:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:17 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:18:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:17 smithi082 ceph-mon[98694]: pgmap v13562: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 16 op/s 2023-09-10T06:18:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:17 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:18:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:17 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:18:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25129.453575 secs 2023-09-10T06:18:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25129.453575 secs 2023-09-10T06:18:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25129.453575 secs 2023-09-10T06:18:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:19 smithi098 ceph-mon[103124]: pgmap v13563: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 14 op/s 2023-09-10T06:18:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25130.461825 secs 2023-09-10T06:18:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25131.074867 secs 2023-09-10T06:18:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:19 smithi170 ceph-mon[102832]: pgmap v13563: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 14 op/s 2023-09-10T06:18:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25130.461825 secs 2023-09-10T06:18:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25131.074867 secs 2023-09-10T06:18:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:19 smithi082 ceph-mon[98694]: pgmap v13563: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 14 op/s 2023-09-10T06:18:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25130.461825 secs 2023-09-10T06:18:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25131.074867 secs 2023-09-10T06:18:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:21 smithi098 ceph-mon[103124]: pgmap v13564: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T06:18:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:21 smithi170 ceph-mon[102832]: pgmap v13564: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T06:18:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:21 smithi082 ceph-mon[98694]: pgmap v13564: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T06:18:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:23 smithi098 ceph-mon[103124]: pgmap v13565: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:18:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25134.453638 secs 2023-09-10T06:18:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:23 smithi170 ceph-mon[102832]: pgmap v13565: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:18:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25134.453638 secs 2023-09-10T06:18:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:23 smithi082 ceph-mon[98694]: pgmap v13565: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:18:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25134.453638 secs 2023-09-10T06:18:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25135.461884 secs 2023-09-10T06:18:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25136.074929 secs 2023-09-10T06:18:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25135.461884 secs 2023-09-10T06:18:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25136.074929 secs 2023-09-10T06:18:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25135.461884 secs 2023-09-10T06:18:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25136.074929 secs 2023-09-10T06:18:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:25 smithi098 ceph-mon[103124]: pgmap v13566: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:18:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:25 smithi170 ceph-mon[102832]: pgmap v13566: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:18:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:25 smithi082 ceph-mon[98694]: pgmap v13566: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:18:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:26 smithi098 ceph-mon[103124]: pgmap v13567: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T06:18:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:26 smithi170 ceph-mon[102832]: pgmap v13567: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T06:18:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:26 smithi082 ceph-mon[98694]: pgmap v13567: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 12 op/s 2023-09-10T06:18:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:28 smithi170 ceph-mon[102832]: pgmap v13568: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T06:18:29.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:28 smithi082 ceph-mon[98694]: pgmap v13568: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T06:18:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:28 smithi098 ceph-mon[103124]: pgmap v13568: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T06:18:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25140.461942 secs 2023-09-10T06:18:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25141.074994 secs 2023-09-10T06:18:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25140.461942 secs 2023-09-10T06:18:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25141.074994 secs 2023-09-10T06:18:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25140.461942 secs 2023-09-10T06:18:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25141.074994 secs 2023-09-10T06:18:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:31 smithi170 ceph-mon[102832]: pgmap v13569: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T06:18:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:18:31.963 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:31 smithi082 ceph-mon[98694]: pgmap v13569: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T06:18:31.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:18:32.009 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:31 smithi098 ceph-mon[103124]: pgmap v13569: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T06:18:32.010 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:18:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:18:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:18:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:18:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:18:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:18:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:18:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:18:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:18:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:18:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:33 smithi170 ceph-mon[102832]: pgmap v13570: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 174 KiB/s wr, 15 op/s 2023-09-10T06:18:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:33 smithi082 ceph-mon[98694]: pgmap v13570: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 174 KiB/s wr, 15 op/s 2023-09-10T06:18:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:33 smithi098 ceph-mon[103124]: pgmap v13570: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 174 KiB/s wr, 15 op/s 2023-09-10T06:18:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25145.462000 secs 2023-09-10T06:18:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25146.075053 secs 2023-09-10T06:18:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25145.462000 secs 2023-09-10T06:18:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25146.075053 secs 2023-09-10T06:18:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25145.462000 secs 2023-09-10T06:18:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25146.075053 secs 2023-09-10T06:18:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:35 smithi170 ceph-mon[102832]: pgmap v13571: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T06:18:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:35 smithi082 ceph-mon[98694]: pgmap v13571: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T06:18:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:35 smithi098 ceph-mon[103124]: pgmap v13571: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T06:18:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:18:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:18:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:18:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:18:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:18:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:18:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:37 smithi170 ceph-mon[102832]: pgmap v13572: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 381 KiB/s wr, 16 op/s 2023-09-10T06:18:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:37 smithi082 ceph-mon[98694]: pgmap v13572: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 381 KiB/s wr, 16 op/s 2023-09-10T06:18:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:37 smithi098 ceph-mon[103124]: pgmap v13572: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 381 KiB/s wr, 16 op/s 2023-09-10T06:18:38.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:18:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:18:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:18:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:18:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:18:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:18:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:39 smithi082 ceph-mon[98694]: pgmap v13573: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 380 KiB/s wr, 12 op/s 2023-09-10T06:18:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25150.462057 secs 2023-09-10T06:18:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25151.075118 secs 2023-09-10T06:18:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:39 smithi098 ceph-mon[103124]: pgmap v13573: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 380 KiB/s wr, 12 op/s 2023-09-10T06:18:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25150.462057 secs 2023-09-10T06:18:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25151.075118 secs 2023-09-10T06:18:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:39 smithi170 ceph-mon[102832]: pgmap v13573: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 380 KiB/s wr, 12 op/s 2023-09-10T06:18:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25150.462057 secs 2023-09-10T06:18:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25151.075118 secs 2023-09-10T06:18:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:41 smithi082 ceph-mon[98694]: pgmap v13574: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 380 KiB/s wr, 12 op/s 2023-09-10T06:18:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:41 smithi098 ceph-mon[103124]: pgmap v13574: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 380 KiB/s wr, 12 op/s 2023-09-10T06:18:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:41 smithi170 ceph-mon[102832]: pgmap v13574: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 380 KiB/s wr, 12 op/s 2023-09-10T06:18:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:18:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:18:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:18:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:43 smithi082 ceph-mon[98694]: pgmap v13575: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 399 KiB/s wr, 18 op/s 2023-09-10T06:18:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:18:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:18:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:18:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:43 smithi098 ceph-mon[103124]: pgmap v13575: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 399 KiB/s wr, 18 op/s 2023-09-10T06:18:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:18:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:18:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:18:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:43 smithi170 ceph-mon[102832]: pgmap v13575: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 399 KiB/s wr, 18 op/s 2023-09-10T06:18:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:18:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:18:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:18:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25155.462117 secs 2023-09-10T06:18:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:44 smithi170 ceph-mon[102832]: pgmap v13576: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 311 KiB/s wr, 12 op/s 2023-09-10T06:18:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25156.075180 secs 2023-09-10T06:18:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:18:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:18:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25155.462117 secs 2023-09-10T06:18:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:44 smithi082 ceph-mon[98694]: pgmap v13576: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 311 KiB/s wr, 12 op/s 2023-09-10T06:18:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25156.075180 secs 2023-09-10T06:18:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:18:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:18:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25155.462117 secs 2023-09-10T06:18:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:44 smithi098 ceph-mon[103124]: pgmap v13576: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 311 KiB/s wr, 12 op/s 2023-09-10T06:18:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25156.075180 secs 2023-09-10T06:18:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:18:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:18:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:18:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:18:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:18:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:18:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:18:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:18:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:47 smithi170 ceph-mon[102832]: pgmap v13577: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 777 KiB/s wr, 19 op/s 2023-09-10T06:18:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:47 smithi082 ceph-mon[98694]: pgmap v13577: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 777 KiB/s wr, 19 op/s 2023-09-10T06:18:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:47 smithi098 ceph-mon[103124]: pgmap v13577: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 777 KiB/s wr, 19 op/s 2023-09-10T06:18:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:49 smithi170 ceph-mon[102832]: pgmap v13578: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 485 KiB/s wr, 12 op/s 2023-09-10T06:18:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25160.462176 secs 2023-09-10T06:18:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25161.075244 secs 2023-09-10T06:18:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:49 smithi082 ceph-mon[98694]: pgmap v13578: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 485 KiB/s wr, 12 op/s 2023-09-10T06:18:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25160.462176 secs 2023-09-10T06:18:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25161.075244 secs 2023-09-10T06:18:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:49 smithi098 ceph-mon[103124]: pgmap v13578: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 485 KiB/s wr, 12 op/s 2023-09-10T06:18:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25160.462176 secs 2023-09-10T06:18:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25161.075244 secs 2023-09-10T06:18:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:51 smithi170 ceph-mon[102832]: pgmap v13579: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 485 KiB/s wr, 12 op/s 2023-09-10T06:18:51.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:51 smithi082 ceph-mon[98694]: pgmap v13579: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 485 KiB/s wr, 12 op/s 2023-09-10T06:18:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:51 smithi098 ceph-mon[103124]: pgmap v13579: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 485 KiB/s wr, 12 op/s 2023-09-10T06:18:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:53 smithi170 ceph-mon[102832]: pgmap v13580: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 536 KiB/s wr, 17 op/s 2023-09-10T06:18:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:53 smithi082 ceph-mon[98694]: pgmap v13580: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 536 KiB/s wr, 17 op/s 2023-09-10T06:18:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:53 smithi098 ceph-mon[103124]: pgmap v13580: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 536 KiB/s wr, 17 op/s 2023-09-10T06:18:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25165.462234 secs 2023-09-10T06:18:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25166.075308 secs 2023-09-10T06:18:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25165.462234 secs 2023-09-10T06:18:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25166.075308 secs 2023-09-10T06:18:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25165.462234 secs 2023-09-10T06:18:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25166.075308 secs 2023-09-10T06:18:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:55 smithi170 ceph-mon[102832]: pgmap v13581: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 517 KiB/s wr, 11 op/s 2023-09-10T06:18:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:55 smithi082 ceph-mon[98694]: pgmap v13581: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 517 KiB/s wr, 11 op/s 2023-09-10T06:18:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:55 smithi098 ceph-mon[103124]: pgmap v13581: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 517 KiB/s wr, 11 op/s 2023-09-10T06:18:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:57 smithi170 ceph-mon[102832]: pgmap v13582: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 673 KiB/s wr, 17 op/s 2023-09-10T06:18:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:57 smithi082 ceph-mon[98694]: pgmap v13582: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 673 KiB/s wr, 17 op/s 2023-09-10T06:18:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:57 smithi098 ceph-mon[103124]: pgmap v13582: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 673 KiB/s wr, 17 op/s 2023-09-10T06:18:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:18:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:18:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25169.454310 secs 2023-09-10T06:18:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:18:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:18:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25169.454310 secs 2023-09-10T06:18:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:18:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:18:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25169.454310 secs 2023-09-10T06:18:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:59 smithi170 ceph-mon[102832]: pgmap v13583: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 207 KiB/s wr, 11 op/s 2023-09-10T06:18:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25170.462312 secs 2023-09-10T06:18:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:18:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25171.075371 secs 2023-09-10T06:18:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:59 smithi082 ceph-mon[98694]: pgmap v13583: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 207 KiB/s wr, 11 op/s 2023-09-10T06:18:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25170.462312 secs 2023-09-10T06:18:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:18:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25171.075371 secs 2023-09-10T06:19:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:59 smithi098 ceph-mon[103124]: pgmap v13583: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 207 KiB/s wr, 11 op/s 2023-09-10T06:19:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25170.462312 secs 2023-09-10T06:19:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:18:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25171.075371 secs 2023-09-10T06:19:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:19:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:19:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:19:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:19:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:19:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:19:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:19:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:19:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:19:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:01 smithi082 ceph-mon[98694]: pgmap v13584: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 207 KiB/s wr, 11 op/s 2023-09-10T06:19:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:01 smithi098 ceph-mon[103124]: pgmap v13584: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 207 KiB/s wr, 11 op/s 2023-09-10T06:19:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:01 smithi170 ceph-mon[102832]: pgmap v13584: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 207 KiB/s wr, 11 op/s 2023-09-10T06:19:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:03 smithi082 ceph-mon[98694]: pgmap v13585: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 436 KiB/s wr, 31 op/s 2023-09-10T06:19:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25174.454371 secs 2023-09-10T06:19:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:03 smithi098 ceph-mon[103124]: pgmap v13585: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 436 KiB/s wr, 31 op/s 2023-09-10T06:19:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25174.454371 secs 2023-09-10T06:19:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:03 smithi170 ceph-mon[102832]: pgmap v13585: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 436 KiB/s wr, 31 op/s 2023-09-10T06:19:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25174.454371 secs 2023-09-10T06:19:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25175.462430 secs 2023-09-10T06:19:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25176.075430 secs 2023-09-10T06:19:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25175.462430 secs 2023-09-10T06:19:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25176.075430 secs 2023-09-10T06:19:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25175.462430 secs 2023-09-10T06:19:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25176.075430 secs 2023-09-10T06:19:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:05 smithi082 ceph-mon[98694]: pgmap v13586: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 385 KiB/s wr, 25 op/s 2023-09-10T06:19:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:05 smithi098 ceph-mon[103124]: pgmap v13586: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 385 KiB/s wr, 25 op/s 2023-09-10T06:19:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:05 smithi170 ceph-mon[102832]: pgmap v13586: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 385 KiB/s wr, 25 op/s 2023-09-10T06:19:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:07 smithi082 ceph-mon[98694]: pgmap v13587: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 477 KiB/s wr, 31 op/s 2023-09-10T06:19:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:07 smithi098 ceph-mon[103124]: pgmap v13587: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 477 KiB/s wr, 31 op/s 2023-09-10T06:19:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:07 smithi170 ceph-mon[102832]: pgmap v13587: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 477 KiB/s wr, 31 op/s 2023-09-10T06:19:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25179.454457 secs 2023-09-10T06:19:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25179.454457 secs 2023-09-10T06:19:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25179.454457 secs 2023-09-10T06:19:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:09 smithi098 ceph-mon[103124]: pgmap v13588: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 321 KiB/s wr, 25 op/s 2023-09-10T06:19:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25180.462494 secs 2023-09-10T06:19:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:09 smithi098 ceph-mon[103124]: osdmap e917: 12 total, 12 up, 12 in 2023-09-10T06:19:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25181.075487 secs 2023-09-10T06:19:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:09 smithi170 ceph-mon[102832]: pgmap v13588: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 321 KiB/s wr, 25 op/s 2023-09-10T06:19:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25180.462494 secs 2023-09-10T06:19:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:09 smithi170 ceph-mon[102832]: osdmap e917: 12 total, 12 up, 12 in 2023-09-10T06:19:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25181.075487 secs 2023-09-10T06:19:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:09 smithi082 ceph-mon[98694]: pgmap v13588: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 321 KiB/s wr, 25 op/s 2023-09-10T06:19:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25180.462494 secs 2023-09-10T06:19:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:09 smithi082 ceph-mon[98694]: osdmap e917: 12 total, 12 up, 12 in 2023-09-10T06:19:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25181.075487 secs 2023-09-10T06:19:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:11 smithi098 ceph-mon[103124]: pgmap v13590: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 385 KiB/s wr, 30 op/s 2023-09-10T06:19:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:11 smithi170 ceph-mon[102832]: pgmap v13590: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 385 KiB/s wr, 30 op/s 2023-09-10T06:19:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:11 smithi082 ceph-mon[98694]: pgmap v13590: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 385 KiB/s wr, 30 op/s 2023-09-10T06:19:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:13 smithi098 ceph-mon[103124]: pgmap v13591: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 127 KiB/s wr, 13 op/s 2023-09-10T06:19:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25184.454526 secs 2023-09-10T06:19:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:13 smithi170 ceph-mon[102832]: pgmap v13591: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 127 KiB/s wr, 13 op/s 2023-09-10T06:19:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25184.454526 secs 2023-09-10T06:19:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:13 smithi082 ceph-mon[98694]: pgmap v13591: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 127 KiB/s wr, 13 op/s 2023-09-10T06:19:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25184.454526 secs 2023-09-10T06:19:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25185.462575 secs 2023-09-10T06:19:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25186.075549 secs 2023-09-10T06:19:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25185.462575 secs 2023-09-10T06:19:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25186.075549 secs 2023-09-10T06:19:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25185.462575 secs 2023-09-10T06:19:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25186.075549 secs 2023-09-10T06:19:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:15 smithi098 ceph-mon[103124]: pgmap v13592: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 127 KiB/s wr, 13 op/s 2023-09-10T06:19:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:15 smithi098 ceph-mon[103124]: osdmap e918: 12 total, 12 up, 12 in 2023-09-10T06:19:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:15 smithi170 ceph-mon[102832]: pgmap v13592: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 127 KiB/s wr, 13 op/s 2023-09-10T06:19:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:15 smithi170 ceph-mon[102832]: osdmap e918: 12 total, 12 up, 12 in 2023-09-10T06:19:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:15 smithi082 ceph-mon[98694]: pgmap v13592: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 127 KiB/s wr, 13 op/s 2023-09-10T06:19:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:15 smithi082 ceph-mon[98694]: osdmap e918: 12 total, 12 up, 12 in 2023-09-10T06:19:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:17 smithi098 ceph-mon[103124]: pgmap v13594: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 22 KiB/s wr, 16 op/s 2023-09-10T06:19:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:17 smithi170 ceph-mon[102832]: pgmap v13594: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 22 KiB/s wr, 16 op/s 2023-09-10T06:19:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:17 smithi082 ceph-mon[98694]: pgmap v13594: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 22 KiB/s wr, 16 op/s 2023-09-10T06:19:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25189.454588 secs 2023-09-10T06:19:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25189.454588 secs 2023-09-10T06:19:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25189.454588 secs 2023-09-10T06:19:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:19 smithi098 ceph-mon[103124]: pgmap v13595: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 19 KiB/s wr, 14 op/s 2023-09-10T06:19:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25190.462635 secs 2023-09-10T06:19:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25191.075612 secs 2023-09-10T06:19:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:19 smithi170 ceph-mon[102832]: pgmap v13595: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 19 KiB/s wr, 14 op/s 2023-09-10T06:19:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25190.462635 secs 2023-09-10T06:19:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25191.075612 secs 2023-09-10T06:19:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:19 smithi082 ceph-mon[98694]: pgmap v13595: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 19 KiB/s wr, 14 op/s 2023-09-10T06:19:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25190.462635 secs 2023-09-10T06:19:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25191.075612 secs 2023-09-10T06:19:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:21 smithi098 ceph-mon[103124]: pgmap v13596: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 18 KiB/s wr, 12 op/s 2023-09-10T06:19:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:21 smithi170 ceph-mon[102832]: pgmap v13596: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 18 KiB/s wr, 12 op/s 2023-09-10T06:19:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:21 smithi082 ceph-mon[98694]: pgmap v13596: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 18 KiB/s wr, 12 op/s 2023-09-10T06:19:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:23 smithi098 ceph-mon[103124]: pgmap v13597: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 13 op/s 2023-09-10T06:19:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25194.454682 secs 2023-09-10T06:19:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:23 smithi170 ceph-mon[102832]: pgmap v13597: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 13 op/s 2023-09-10T06:19:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25194.454682 secs 2023-09-10T06:19:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:23 smithi082 ceph-mon[98694]: pgmap v13597: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 13 op/s 2023-09-10T06:19:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25194.454682 secs 2023-09-10T06:19:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25195.462692 secs 2023-09-10T06:19:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25196.075674 secs 2023-09-10T06:19:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25195.462692 secs 2023-09-10T06:19:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25196.075674 secs 2023-09-10T06:19:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25195.462692 secs 2023-09-10T06:19:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25196.075674 secs 2023-09-10T06:19:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:25 smithi098 ceph-mon[103124]: pgmap v13598: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 13 op/s 2023-09-10T06:19:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:25 smithi170 ceph-mon[102832]: pgmap v13598: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 13 op/s 2023-09-10T06:19:26.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:25 smithi082 ceph-mon[98694]: pgmap v13598: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 13 op/s 2023-09-10T06:19:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:26 smithi170 ceph-mon[102832]: pgmap v13599: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 13 op/s 2023-09-10T06:19:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:26 smithi082 ceph-mon[98694]: pgmap v13599: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 13 op/s 2023-09-10T06:19:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:26 smithi098 ceph-mon[103124]: pgmap v13599: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 13 op/s 2023-09-10T06:19:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:28 smithi170 ceph-mon[102832]: pgmap v13600: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T06:19:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:28 smithi082 ceph-mon[98694]: pgmap v13600: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T06:19:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:28 smithi098 ceph-mon[103124]: pgmap v13600: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T06:19:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25200.462751 secs 2023-09-10T06:19:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:29 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:19:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:29 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:19:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25201.075736 secs 2023-09-10T06:19:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25200.462751 secs 2023-09-10T06:19:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:29 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:19:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:29 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:19:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25201.075736 secs 2023-09-10T06:19:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25200.462751 secs 2023-09-10T06:19:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:29 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:19:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:29 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:19:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25201.075736 secs 2023-09-10T06:19:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:31 smithi170 ceph-mon[102832]: pgmap v13601: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T06:19:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:19:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:31 smithi082 ceph-mon[98694]: pgmap v13601: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T06:19:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:19:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:31 smithi098 ceph-mon[103124]: pgmap v13601: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 11 op/s 2023-09-10T06:19:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:19:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:19:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:19:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:19:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:19:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:19:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:19:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:19:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:19:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:19:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:33 smithi170 ceph-mon[102832]: pgmap v13602: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 194 KiB/s wr, 17 op/s 2023-09-10T06:19:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:33 smithi082 ceph-mon[98694]: pgmap v13602: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 194 KiB/s wr, 17 op/s 2023-09-10T06:19:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:33 smithi098 ceph-mon[103124]: pgmap v13602: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 194 KiB/s wr, 17 op/s 2023-09-10T06:19:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25205.462805 secs 2023-09-10T06:19:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25206.075795 secs 2023-09-10T06:19:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25205.462805 secs 2023-09-10T06:19:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25206.075795 secs 2023-09-10T06:19:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25205.462805 secs 2023-09-10T06:19:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25206.075795 secs 2023-09-10T06:19:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:35 smithi082 ceph-mon[98694]: pgmap v13603: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 107 KiB/s wr, 10 op/s 2023-09-10T06:19:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:35 smithi098 ceph-mon[103124]: pgmap v13603: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 107 KiB/s wr, 10 op/s 2023-09-10T06:19:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:35 smithi170 ceph-mon[102832]: pgmap v13603: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 107 KiB/s wr, 10 op/s 2023-09-10T06:19:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:37 smithi082 ceph-mon[98694]: pgmap v13604: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 293 KiB/s wr, 16 op/s 2023-09-10T06:19:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:37 smithi098 ceph-mon[103124]: pgmap v13604: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 293 KiB/s wr, 16 op/s 2023-09-10T06:19:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:37 smithi170 ceph-mon[102832]: pgmap v13604: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 293 KiB/s wr, 16 op/s 2023-09-10T06:19:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:39 smithi082 ceph-mon[98694]: pgmap v13605: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T06:19:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25210.462867 secs 2023-09-10T06:19:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25211.075859 secs 2023-09-10T06:19:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:39 smithi098 ceph-mon[103124]: pgmap v13605: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T06:19:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25210.462867 secs 2023-09-10T06:19:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25211.075859 secs 2023-09-10T06:19:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:39 smithi170 ceph-mon[102832]: pgmap v13605: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T06:19:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25210.462867 secs 2023-09-10T06:19:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25211.075859 secs 2023-09-10T06:19:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:41 smithi082 ceph-mon[98694]: pgmap v13606: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T06:19:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:41 smithi098 ceph-mon[103124]: pgmap v13606: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T06:19:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:41 smithi170 ceph-mon[102832]: pgmap v13606: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T06:19:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:43 smithi082 ceph-mon[98694]: pgmap v13607: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 346 KiB/s wr, 17 op/s 2023-09-10T06:19:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:19:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:19:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:19:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:19:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:43 smithi098 ceph-mon[103124]: pgmap v13607: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 346 KiB/s wr, 17 op/s 2023-09-10T06:19:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:19:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:19:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:19:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:19:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:43 smithi170 ceph-mon[102832]: pgmap v13607: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 346 KiB/s wr, 17 op/s 2023-09-10T06:19:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:19:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:19:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:19:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:19:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25215.462929 secs 2023-09-10T06:19:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:19:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:19:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25216.075938 secs 2023-09-10T06:19:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25215.462929 secs 2023-09-10T06:19:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:19:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:19:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25216.075938 secs 2023-09-10T06:19:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25215.462929 secs 2023-09-10T06:19:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:19:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:19:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25216.075938 secs 2023-09-10T06:19:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:45 smithi082 ceph-mon[98694]: pgmap v13608: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 239 KiB/s wr, 11 op/s 2023-09-10T06:19:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:19:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:19:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:45 smithi098 ceph-mon[103124]: pgmap v13608: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 239 KiB/s wr, 11 op/s 2023-09-10T06:19:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:19:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:19:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:45 smithi170 ceph-mon[102832]: pgmap v13608: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 239 KiB/s wr, 11 op/s 2023-09-10T06:19:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:19:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:19:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:47 smithi082 ceph-mon[98694]: pgmap v13609: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 548 KiB/s wr, 18 op/s 2023-09-10T06:19:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:47 smithi098 ceph-mon[103124]: pgmap v13609: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 548 KiB/s wr, 18 op/s 2023-09-10T06:19:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:47 smithi170 ceph-mon[102832]: pgmap v13609: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 548 KiB/s wr, 18 op/s 2023-09-10T06:19:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:49 smithi082 ceph-mon[98694]: pgmap v13610: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 362 KiB/s wr, 12 op/s 2023-09-10T06:19:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25220.462987 secs 2023-09-10T06:19:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25221.075999 secs 2023-09-10T06:19:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:49 smithi098 ceph-mon[103124]: pgmap v13610: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 362 KiB/s wr, 12 op/s 2023-09-10T06:19:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25220.462987 secs 2023-09-10T06:19:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25221.075999 secs 2023-09-10T06:19:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:49 smithi170 ceph-mon[102832]: pgmap v13610: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 362 KiB/s wr, 12 op/s 2023-09-10T06:19:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25220.462987 secs 2023-09-10T06:19:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25221.075999 secs 2023-09-10T06:19:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:51 smithi098 ceph-mon[103124]: pgmap v13611: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 362 KiB/s wr, 12 op/s 2023-09-10T06:19:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:51 smithi170 ceph-mon[102832]: pgmap v13611: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 362 KiB/s wr, 12 op/s 2023-09-10T06:19:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:51 smithi082 ceph-mon[98694]: pgmap v13611: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 362 KiB/s wr, 12 op/s 2023-09-10T06:19:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:53 smithi098 ceph-mon[103124]: pgmap v13612: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 484 KiB/s wr, 18 op/s 2023-09-10T06:19:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:19:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:19:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:53 smithi170 ceph-mon[102832]: pgmap v13612: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 484 KiB/s wr, 18 op/s 2023-09-10T06:19:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:19:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:19:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:53 smithi082 ceph-mon[98694]: pgmap v13612: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 484 KiB/s wr, 18 op/s 2023-09-10T06:19:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:19:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:19:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25225.463070 secs 2023-09-10T06:19:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25226.076063 secs 2023-09-10T06:19:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25225.463070 secs 2023-09-10T06:19:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25226.076063 secs 2023-09-10T06:19:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25225.463070 secs 2023-09-10T06:19:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25226.076063 secs 2023-09-10T06:19:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:55 smithi098 ceph-mon[103124]: pgmap v13613: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 431 KiB/s wr, 13 op/s 2023-09-10T06:19:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:55 smithi170 ceph-mon[102832]: pgmap v13613: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 431 KiB/s wr, 13 op/s 2023-09-10T06:19:56.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:55 smithi082 ceph-mon[98694]: pgmap v13613: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 431 KiB/s wr, 13 op/s 2023-09-10T06:19:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:19:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:19:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:19:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:19:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:19:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:19:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:57 smithi098 ceph-mon[103124]: pgmap v13614: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 638 KiB/s wr, 19 op/s 2023-09-10T06:19:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:57 smithi170 ceph-mon[102832]: pgmap v13614: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 638 KiB/s wr, 19 op/s 2023-09-10T06:19:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:57 smithi082 ceph-mon[98694]: pgmap v13614: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 638 KiB/s wr, 19 op/s 2023-09-10T06:19:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25229.455162 secs 2023-09-10T06:19:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25229.455162 secs 2023-09-10T06:19:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25229.455162 secs 2023-09-10T06:20:00.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:59 smithi098 ceph-mon[103124]: pgmap v13615: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 329 KiB/s wr, 12 op/s 2023-09-10T06:20:00.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25230.463157 secs 2023-09-10T06:20:00.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:19:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25231.076122 secs 2023-09-10T06:20:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:59 smithi170 ceph-mon[102832]: pgmap v13615: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 329 KiB/s wr, 12 op/s 2023-09-10T06:20:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25230.463157 secs 2023-09-10T06:20:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:19:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25231.076122 secs 2023-09-10T06:20:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:59 smithi082 ceph-mon[98694]: pgmap v13615: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 329 KiB/s wr, 12 op/s 2023-09-10T06:20:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25230.463157 secs 2023-09-10T06:20:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:19:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25231.076122 secs 2023-09-10T06:20:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:20:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:20:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T06:20:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:20:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:20:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T06:20:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:20:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:20:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T06:20:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:01 smithi098 ceph-mon[103124]: pgmap v13616: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 329 KiB/s wr, 12 op/s 2023-09-10T06:20:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:01 smithi170 ceph-mon[102832]: pgmap v13616: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 329 KiB/s wr, 12 op/s 2023-09-10T06:20:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:01 smithi082 ceph-mon[98694]: pgmap v13616: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 329 KiB/s wr, 12 op/s 2023-09-10T06:20:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:03 smithi170 ceph-mon[102832]: pgmap v13617: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 522 KiB/s wr, 21 op/s 2023-09-10T06:20:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25234.455220 secs 2023-09-10T06:20:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:03 smithi082 ceph-mon[98694]: pgmap v13617: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 522 KiB/s wr, 21 op/s 2023-09-10T06:20:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25234.455220 secs 2023-09-10T06:20:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:03 smithi098 ceph-mon[103124]: pgmap v13617: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 522 KiB/s wr, 21 op/s 2023-09-10T06:20:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25234.455220 secs 2023-09-10T06:20:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25235.463215 secs 2023-09-10T06:20:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:20:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:04 smithi170 ceph-mon[102832]: osdmap e919: 12 total, 12 up, 12 in 2023-09-10T06:20:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:04 smithi170 ceph-mon[102832]: pgmap v13619: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 481 KiB/s wr, 18 op/s 2023-09-10T06:20:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25236.076189 secs 2023-09-10T06:20:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25235.463215 secs 2023-09-10T06:20:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:20:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:04 smithi082 ceph-mon[98694]: osdmap e919: 12 total, 12 up, 12 in 2023-09-10T06:20:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:04 smithi082 ceph-mon[98694]: pgmap v13619: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 481 KiB/s wr, 18 op/s 2023-09-10T06:20:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25236.076189 secs 2023-09-10T06:20:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25235.463215 secs 2023-09-10T06:20:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:20:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:04 smithi098 ceph-mon[103124]: osdmap e919: 12 total, 12 up, 12 in 2023-09-10T06:20:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:04 smithi098 ceph-mon[103124]: pgmap v13619: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 481 KiB/s wr, 18 op/s 2023-09-10T06:20:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25236.076189 secs 2023-09-10T06:20:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:20:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:20:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:20:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:06 smithi170 ceph-mon[102832]: pgmap v13620: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 415 KiB/s wr, 31 op/s 2023-09-10T06:20:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:06 smithi082 ceph-mon[98694]: pgmap v13620: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 415 KiB/s wr, 31 op/s 2023-09-10T06:20:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:06 smithi098 ceph-mon[103124]: pgmap v13620: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 415 KiB/s wr, 31 op/s 2023-09-10T06:20:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25239.455280 secs 2023-09-10T06:20:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:08 smithi170 ceph-mon[102832]: pgmap v13621: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 415 KiB/s wr, 31 op/s 2023-09-10T06:20:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25239.455280 secs 2023-09-10T06:20:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:08 smithi082 ceph-mon[98694]: pgmap v13621: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 415 KiB/s wr, 31 op/s 2023-09-10T06:20:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25239.455280 secs 2023-09-10T06:20:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:08 smithi098 ceph-mon[103124]: pgmap v13621: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 415 KiB/s wr, 31 op/s 2023-09-10T06:20:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25240.463268 secs 2023-09-10T06:20:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25241.076251 secs 2023-09-10T06:20:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25240.463268 secs 2023-09-10T06:20:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25241.076251 secs 2023-09-10T06:20:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25240.463268 secs 2023-09-10T06:20:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25241.076251 secs 2023-09-10T06:20:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:11 smithi170 ceph-mon[102832]: pgmap v13622: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 415 KiB/s wr, 31 op/s 2023-09-10T06:20:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:11 smithi170 ceph-mon[102832]: osdmap e920: 12 total, 12 up, 12 in 2023-09-10T06:20:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:11 smithi082 ceph-mon[98694]: pgmap v13622: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 415 KiB/s wr, 31 op/s 2023-09-10T06:20:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:11 smithi082 ceph-mon[98694]: osdmap e920: 12 total, 12 up, 12 in 2023-09-10T06:20:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:11 smithi098 ceph-mon[103124]: pgmap v13622: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 415 KiB/s wr, 31 op/s 2023-09-10T06:20:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:11 smithi098 ceph-mon[103124]: osdmap e920: 12 total, 12 up, 12 in 2023-09-10T06:20:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:13 smithi170 ceph-mon[102832]: pgmap v13624: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 328 KiB/s wr, 33 op/s 2023-09-10T06:20:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25244.455340 secs 2023-09-10T06:20:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:13 smithi082 ceph-mon[98694]: pgmap v13624: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 328 KiB/s wr, 33 op/s 2023-09-10T06:20:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25244.455340 secs 2023-09-10T06:20:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:13 smithi098 ceph-mon[103124]: pgmap v13624: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 328 KiB/s wr, 33 op/s 2023-09-10T06:20:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25244.455340 secs 2023-09-10T06:20:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25245.463327 secs 2023-09-10T06:20:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25246.076322 secs 2023-09-10T06:20:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25245.463327 secs 2023-09-10T06:20:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25246.076322 secs 2023-09-10T06:20:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25245.463327 secs 2023-09-10T06:20:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25246.076322 secs 2023-09-10T06:20:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:15 smithi082 ceph-mon[98694]: pgmap v13625: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 264 KiB/s wr, 26 op/s 2023-09-10T06:20:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:15 smithi098 ceph-mon[103124]: pgmap v13625: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 264 KiB/s wr, 26 op/s 2023-09-10T06:20:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:15 smithi170 ceph-mon[102832]: pgmap v13625: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 264 KiB/s wr, 26 op/s 2023-09-10T06:20:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:17 smithi082 ceph-mon[98694]: pgmap v13626: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T06:20:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:17 smithi098 ceph-mon[103124]: pgmap v13626: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T06:20:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:17 smithi170 ceph-mon[102832]: pgmap v13626: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T06:20:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25249.455402 secs 2023-09-10T06:20:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25249.455402 secs 2023-09-10T06:20:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25249.455402 secs 2023-09-10T06:20:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:19 smithi082 ceph-mon[98694]: pgmap v13627: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T06:20:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25250.463411 secs 2023-09-10T06:20:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25251.076382 secs 2023-09-10T06:20:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:19 smithi098 ceph-mon[103124]: pgmap v13627: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T06:20:20.054 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25250.463411 secs 2023-09-10T06:20:20.055 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25251.076382 secs 2023-09-10T06:20:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:19 smithi170 ceph-mon[102832]: pgmap v13627: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T06:20:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25250.463411 secs 2023-09-10T06:20:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25251.076382 secs 2023-09-10T06:20:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:21 smithi082 ceph-mon[98694]: pgmap v13628: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T06:20:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:21 smithi098 ceph-mon[103124]: pgmap v13628: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T06:20:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:21 smithi170 ceph-mon[102832]: pgmap v13628: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T06:20:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:23 smithi082 ceph-mon[98694]: pgmap v13629: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 10 op/s 2023-09-10T06:20:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25254.455462 secs 2023-09-10T06:20:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:23 smithi098 ceph-mon[103124]: pgmap v13629: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 10 op/s 2023-09-10T06:20:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25254.455462 secs 2023-09-10T06:20:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:23 smithi170 ceph-mon[102832]: pgmap v13629: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 10 op/s 2023-09-10T06:20:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25254.455462 secs 2023-09-10T06:20:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25255.463487 secs 2023-09-10T06:20:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25256.076441 secs 2023-09-10T06:20:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25255.463487 secs 2023-09-10T06:20:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25256.076441 secs 2023-09-10T06:20:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25255.463487 secs 2023-09-10T06:20:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25256.076441 secs 2023-09-10T06:20:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:25 smithi098 ceph-mon[103124]: pgmap v13630: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 9 op/s 2023-09-10T06:20:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:25 smithi170 ceph-mon[102832]: pgmap v13630: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 9 op/s 2023-09-10T06:20:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:25 smithi082 ceph-mon[98694]: pgmap v13630: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 9 op/s 2023-09-10T06:20:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:27 smithi098 ceph-mon[103124]: pgmap v13631: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 107 KiB/s wr, 14 op/s 2023-09-10T06:20:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:27 smithi170 ceph-mon[102832]: pgmap v13631: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 107 KiB/s wr, 14 op/s 2023-09-10T06:20:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:27 smithi082 ceph-mon[98694]: pgmap v13631: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 107 KiB/s wr, 14 op/s 2023-09-10T06:20:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25259.455532 secs 2023-09-10T06:20:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25259.455532 secs 2023-09-10T06:20:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25259.455532 secs 2023-09-10T06:20:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:29 smithi098 ceph-mon[103124]: pgmap v13632: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 88 KiB/s wr, 9 op/s 2023-09-10T06:20:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25260.463545 secs 2023-09-10T06:20:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25261.076487 secs 2023-09-10T06:20:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:29 smithi170 ceph-mon[102832]: pgmap v13632: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 88 KiB/s wr, 9 op/s 2023-09-10T06:20:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25260.463545 secs 2023-09-10T06:20:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25261.076487 secs 2023-09-10T06:20:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:29 smithi082 ceph-mon[98694]: pgmap v13632: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 88 KiB/s wr, 9 op/s 2023-09-10T06:20:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25260.463545 secs 2023-09-10T06:20:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25261.076487 secs 2023-09-10T06:20:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:31 smithi098 ceph-mon[103124]: pgmap v13633: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 88 KiB/s wr, 9 op/s 2023-09-10T06:20:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:31 smithi170 ceph-mon[102832]: pgmap v13633: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 88 KiB/s wr, 9 op/s 2023-09-10T06:20:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:31 smithi082 ceph-mon[98694]: pgmap v13633: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 88 KiB/s wr, 9 op/s 2023-09-10T06:20:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:33 smithi098 ceph-mon[103124]: pgmap v13634: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 172 KiB/s wr, 15 op/s 2023-09-10T06:20:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:20:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:33 smithi170 ceph-mon[102832]: pgmap v13634: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 172 KiB/s wr, 15 op/s 2023-09-10T06:20:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:20:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:33 smithi082 ceph-mon[98694]: pgmap v13634: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 172 KiB/s wr, 15 op/s 2023-09-10T06:20:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:20:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25265.463619 secs 2023-09-10T06:20:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25266.076552 secs 2023-09-10T06:20:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25265.463619 secs 2023-09-10T06:20:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25266.076552 secs 2023-09-10T06:20:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25265.463619 secs 2023-09-10T06:20:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25266.076552 secs 2023-09-10T06:20:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:35 smithi170 ceph-mon[102832]: pgmap v13635: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 172 KiB/s wr, 10 op/s 2023-09-10T06:20:36.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:20:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:35 smithi082 ceph-mon[98694]: pgmap v13635: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 172 KiB/s wr, 10 op/s 2023-09-10T06:20:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:20:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:35 smithi098 ceph-mon[103124]: pgmap v13635: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 172 KiB/s wr, 10 op/s 2023-09-10T06:20:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:20:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:20:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:20:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:37.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:20:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:20:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:20:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:20:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:37 smithi170 ceph-mon[102832]: pgmap v13636: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 629 KiB/s wr, 17 op/s 2023-09-10T06:20:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:20:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:38.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:37 smithi082 ceph-mon[98694]: pgmap v13636: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 629 KiB/s wr, 17 op/s 2023-09-10T06:20:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:20:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:37 smithi098 ceph-mon[103124]: pgmap v13636: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 629 KiB/s wr, 17 op/s 2023-09-10T06:20:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:20:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:38 smithi170 ceph-mon[102832]: pgmap v13637: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 542 KiB/s wr, 12 op/s 2023-09-10T06:20:39.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:38 smithi082 ceph-mon[98694]: pgmap v13637: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 542 KiB/s wr, 12 op/s 2023-09-10T06:20:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:38 smithi098 ceph-mon[103124]: pgmap v13637: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 542 KiB/s wr, 12 op/s 2023-09-10T06:20:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25270.463678 secs 2023-09-10T06:20:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25271.076621 secs 2023-09-10T06:20:40.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25270.463678 secs 2023-09-10T06:20:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25271.076621 secs 2023-09-10T06:20:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25270.463678 secs 2023-09-10T06:20:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25271.076621 secs 2023-09-10T06:20:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:41 smithi082 ceph-mon[98694]: pgmap v13638: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 542 KiB/s wr, 12 op/s 2023-09-10T06:20:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:41 smithi098 ceph-mon[103124]: pgmap v13638: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 542 KiB/s wr, 12 op/s 2023-09-10T06:20:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:41 smithi170 ceph-mon[102832]: pgmap v13638: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 542 KiB/s wr, 12 op/s 2023-09-10T06:20:43.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:43 smithi170 ceph-mon[102832]: pgmap v13639: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 561 KiB/s wr, 18 op/s 2023-09-10T06:20:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:43 smithi082 ceph-mon[98694]: pgmap v13639: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 561 KiB/s wr, 18 op/s 2023-09-10T06:20:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:43 smithi098 ceph-mon[103124]: pgmap v13639: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 561 KiB/s wr, 18 op/s 2023-09-10T06:20:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25275.463737 secs 2023-09-10T06:20:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:20:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:20:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25276.076692 secs 2023-09-10T06:20:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25275.463737 secs 2023-09-10T06:20:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:20:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:20:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25276.076692 secs 2023-09-10T06:20:45.026 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25275.463737 secs 2023-09-10T06:20:45.027 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:20:45.027 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:45.027 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:20:45.027 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25276.076692 secs 2023-09-10T06:20:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:45 smithi098 ceph-mon[103124]: pgmap v13640: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 476 KiB/s wr, 13 op/s 2023-09-10T06:20:45.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:45 smithi170 ceph-mon[102832]: pgmap v13640: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 476 KiB/s wr, 13 op/s 2023-09-10T06:20:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:45 smithi082 ceph-mon[98694]: pgmap v13640: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 476 KiB/s wr, 13 op/s 2023-09-10T06:20:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:47 smithi170 ceph-mon[102832]: pgmap v13641: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 782 KiB/s wr, 19 op/s 2023-09-10T06:20:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:20:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:20:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:20:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:20:47.948 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:47 smithi098 ceph-mon[103124]: pgmap v13641: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 782 KiB/s wr, 19 op/s 2023-09-10T06:20:47.948 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:20:47.949 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:20:47.949 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:20:47.949 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:20:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:47 smithi082 ceph-mon[98694]: pgmap v13641: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 782 KiB/s wr, 19 op/s 2023-09-10T06:20:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:20:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:20:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:20:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:20:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:48 smithi082 ceph-mon[98694]: pgmap v13642: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 325 KiB/s wr, 12 op/s 2023-09-10T06:20:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:20:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:20:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:20:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:20:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:20:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25280.463796 secs 2023-09-10T06:20:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:48 smithi098 ceph-mon[103124]: pgmap v13642: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 325 KiB/s wr, 12 op/s 2023-09-10T06:20:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:20:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:20:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:20:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:20:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:20:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25280.463796 secs 2023-09-10T06:20:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:48 smithi170 ceph-mon[102832]: pgmap v13642: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 325 KiB/s wr, 12 op/s 2023-09-10T06:20:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:20:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:20:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:20:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:20:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:20:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25280.463796 secs 2023-09-10T06:20:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25281.076759 secs 2023-09-10T06:20:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:20:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25281.076759 secs 2023-09-10T06:20:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:20:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25281.076759 secs 2023-09-10T06:20:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:20:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:51 smithi082 ceph-mon[98694]: pgmap v13643: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 325 KiB/s wr, 12 op/s 2023-09-10T06:20:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:51 smithi098 ceph-mon[103124]: pgmap v13643: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 325 KiB/s wr, 12 op/s 2023-09-10T06:20:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:51 smithi170 ceph-mon[102832]: pgmap v13643: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 325 KiB/s wr, 12 op/s 2023-09-10T06:20:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:53 smithi082 ceph-mon[98694]: pgmap v13644: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 629 KiB/s wr, 18 op/s 2023-09-10T06:20:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:20:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25284.455915 secs 2023-09-10T06:20:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:53 smithi098 ceph-mon[103124]: pgmap v13644: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 629 KiB/s wr, 18 op/s 2023-09-10T06:20:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:20:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25284.455915 secs 2023-09-10T06:20:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:53 smithi170 ceph-mon[102832]: pgmap v13644: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 629 KiB/s wr, 18 op/s 2023-09-10T06:20:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:20:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25284.455915 secs 2023-09-10T06:20:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25285.463856 secs 2023-09-10T06:20:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25286.076826 secs 2023-09-10T06:20:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25285.463856 secs 2023-09-10T06:20:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25286.076826 secs 2023-09-10T06:20:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25285.463856 secs 2023-09-10T06:20:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25286.076826 secs 2023-09-10T06:20:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:55 smithi082 ceph-mon[98694]: pgmap v13645: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 610 KiB/s wr, 13 op/s 2023-09-10T06:20:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:55 smithi098 ceph-mon[103124]: pgmap v13645: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 610 KiB/s wr, 13 op/s 2023-09-10T06:20:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:55 smithi170 ceph-mon[102832]: pgmap v13645: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 610 KiB/s wr, 13 op/s 2023-09-10T06:20:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:20:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:20:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:20:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:20:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:20:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:20:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:20:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:57 smithi098 ceph-mon[103124]: pgmap v13646: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 677 KiB/s wr, 18 op/s 2023-09-10T06:20:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:57 smithi170 ceph-mon[102832]: pgmap v13646: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 677 KiB/s wr, 18 op/s 2023-09-10T06:20:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:57 smithi082 ceph-mon[98694]: pgmap v13646: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 677 KiB/s wr, 18 op/s 2023-09-10T06:20:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25289.455989 secs 2023-09-10T06:20:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25289.455989 secs 2023-09-10T06:20:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25289.455989 secs 2023-09-10T06:21:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:59 smithi098 ceph-mon[103124]: pgmap v13647: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 371 KiB/s wr, 12 op/s 2023-09-10T06:21:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25290.463935 secs 2023-09-10T06:21:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:20:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25291.076890 secs 2023-09-10T06:21:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:59 smithi170 ceph-mon[102832]: pgmap v13647: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 371 KiB/s wr, 12 op/s 2023-09-10T06:21:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25290.463935 secs 2023-09-10T06:21:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:20:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25291.076890 secs 2023-09-10T06:21:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:59 smithi082 ceph-mon[98694]: pgmap v13647: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 371 KiB/s wr, 12 op/s 2023-09-10T06:21:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25290.463935 secs 2023-09-10T06:21:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:20:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25291.076890 secs 2023-09-10T06:21:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:01 smithi098 ceph-mon[103124]: pgmap v13648: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 371 KiB/s wr, 12 op/s 2023-09-10T06:21:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:01 smithi170 ceph-mon[102832]: pgmap v13648: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 371 KiB/s wr, 12 op/s 2023-09-10T06:21:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:01 smithi082 ceph-mon[98694]: pgmap v13648: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 371 KiB/s wr, 12 op/s 2023-09-10T06:21:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:03 smithi098 ceph-mon[103124]: pgmap v13649: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 584 KiB/s wr, 16 op/s 2023-09-10T06:21:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25294.456050 secs 2023-09-10T06:21:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:03 smithi170 ceph-mon[102832]: pgmap v13649: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 584 KiB/s wr, 16 op/s 2023-09-10T06:21:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25294.456050 secs 2023-09-10T06:21:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:03 smithi082 ceph-mon[98694]: pgmap v13649: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 584 KiB/s wr, 16 op/s 2023-09-10T06:21:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25294.456050 secs 2023-09-10T06:21:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25295.464022 secs 2023-09-10T06:21:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25296.076954 secs 2023-09-10T06:21:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25295.464022 secs 2023-09-10T06:21:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25296.076954 secs 2023-09-10T06:21:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25295.464022 secs 2023-09-10T06:21:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25296.076954 secs 2023-09-10T06:21:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:05 smithi098 ceph-mon[103124]: pgmap v13650: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 279 KiB/s wr, 10 op/s 2023-09-10T06:21:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:05 smithi170 ceph-mon[102832]: pgmap v13650: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 279 KiB/s wr, 10 op/s 2023-09-10T06:21:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:05 smithi082 ceph-mon[98694]: pgmap v13650: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 279 KiB/s wr, 10 op/s 2023-09-10T06:21:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:07 smithi098 ceph-mon[103124]: pgmap v13651: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 410 KiB/s wr, 30 op/s 2023-09-10T06:21:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:07 smithi170 ceph-mon[102832]: pgmap v13651: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 410 KiB/s wr, 30 op/s 2023-09-10T06:21:08.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:07 smithi082 ceph-mon[98694]: pgmap v13651: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 410 KiB/s wr, 30 op/s 2023-09-10T06:21:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25299.456111 secs 2023-09-10T06:21:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25299.456111 secs 2023-09-10T06:21:09.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25299.456111 secs 2023-09-10T06:21:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:09 smithi098 ceph-mon[103124]: pgmap v13652: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 344 KiB/s wr, 25 op/s 2023-09-10T06:21:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25300.464106 secs 2023-09-10T06:21:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:09 smithi098 ceph-mon[103124]: osdmap e921: 12 total, 12 up, 12 in 2023-09-10T06:21:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:09 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:21:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:09 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25301.077019 secs 2023-09-10T06:21:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:09 smithi170 ceph-mon[102832]: pgmap v13652: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 344 KiB/s wr, 25 op/s 2023-09-10T06:21:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25300.464106 secs 2023-09-10T06:21:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:09 smithi170 ceph-mon[102832]: osdmap e921: 12 total, 12 up, 12 in 2023-09-10T06:21:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:09 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:21:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:09 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25301.077019 secs 2023-09-10T06:21:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:09 smithi082 ceph-mon[98694]: pgmap v13652: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 344 KiB/s wr, 25 op/s 2023-09-10T06:21:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25300.464106 secs 2023-09-10T06:21:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:09 smithi082 ceph-mon[98694]: osdmap e921: 12 total, 12 up, 12 in 2023-09-10T06:21:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:09 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:21:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:09 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25301.077019 secs 2023-09-10T06:21:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:11 smithi098 ceph-mon[103124]: pgmap v13654: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 412 KiB/s wr, 30 op/s 2023-09-10T06:21:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:11 smithi170 ceph-mon[102832]: pgmap v13654: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 412 KiB/s wr, 30 op/s 2023-09-10T06:21:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:11 smithi082 ceph-mon[98694]: pgmap v13654: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 412 KiB/s wr, 30 op/s 2023-09-10T06:21:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:21:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:21:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:21:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:13 smithi098 ceph-mon[103124]: pgmap v13655: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 181 KiB/s wr, 30 op/s 2023-09-10T06:21:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25304.456228 secs 2023-09-10T06:21:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:13 smithi170 ceph-mon[102832]: pgmap v13655: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 181 KiB/s wr, 30 op/s 2023-09-10T06:21:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25304.456228 secs 2023-09-10T06:21:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:13 smithi082 ceph-mon[98694]: pgmap v13655: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 181 KiB/s wr, 30 op/s 2023-09-10T06:21:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25304.456228 secs 2023-09-10T06:21:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25305.464164 secs 2023-09-10T06:21:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25306.077082 secs 2023-09-10T06:21:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25305.464164 secs 2023-09-10T06:21:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25306.077082 secs 2023-09-10T06:21:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25305.464164 secs 2023-09-10T06:21:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25306.077082 secs 2023-09-10T06:21:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:15 smithi170 ceph-mon[102832]: pgmap v13656: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 181 KiB/s wr, 30 op/s 2023-09-10T06:21:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:15 smithi170 ceph-mon[102832]: osdmap e922: 12 total, 12 up, 12 in 2023-09-10T06:21:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:15 smithi082 ceph-mon[98694]: pgmap v13656: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 181 KiB/s wr, 30 op/s 2023-09-10T06:21:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:15 smithi082 ceph-mon[98694]: osdmap e922: 12 total, 12 up, 12 in 2023-09-10T06:21:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:15 smithi098 ceph-mon[103124]: pgmap v13656: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 181 KiB/s wr, 30 op/s 2023-09-10T06:21:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:15 smithi098 ceph-mon[103124]: osdmap e922: 12 total, 12 up, 12 in 2023-09-10T06:21:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:16 smithi170 ceph-mon[102832]: pgmap v13658: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 130 KiB/s wr, 15 op/s 2023-09-10T06:21:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:16 smithi082 ceph-mon[98694]: pgmap v13658: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 130 KiB/s wr, 15 op/s 2023-09-10T06:21:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:16 smithi098 ceph-mon[103124]: pgmap v13658: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 130 KiB/s wr, 15 op/s 2023-09-10T06:21:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25309.456287 secs 2023-09-10T06:21:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:18 smithi170 ceph-mon[102832]: pgmap v13659: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 14 op/s 2023-09-10T06:21:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25309.456287 secs 2023-09-10T06:21:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:18 smithi082 ceph-mon[98694]: pgmap v13659: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 14 op/s 2023-09-10T06:21:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25309.456287 secs 2023-09-10T06:21:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:18 smithi098 ceph-mon[103124]: pgmap v13659: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 14 op/s 2023-09-10T06:21:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25310.464241 secs 2023-09-10T06:21:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25311.077147 secs 2023-09-10T06:21:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25310.464241 secs 2023-09-10T06:21:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25311.077147 secs 2023-09-10T06:21:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25310.464241 secs 2023-09-10T06:21:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25311.077147 secs 2023-09-10T06:21:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:21 smithi082 ceph-mon[98694]: pgmap v13660: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:21:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:21 smithi098 ceph-mon[103124]: pgmap v13660: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:21:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:21 smithi170 ceph-mon[102832]: pgmap v13660: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:21:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:23 smithi082 ceph-mon[98694]: pgmap v13661: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 12 op/s 2023-09-10T06:21:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25314.456398 secs 2023-09-10T06:21:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:23 smithi098 ceph-mon[103124]: pgmap v13661: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 12 op/s 2023-09-10T06:21:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25314.456398 secs 2023-09-10T06:21:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:23 smithi170 ceph-mon[102832]: pgmap v13661: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 12 op/s 2023-09-10T06:21:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25314.456398 secs 2023-09-10T06:21:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25315.464298 secs 2023-09-10T06:21:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25316.077211 secs 2023-09-10T06:21:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25315.464298 secs 2023-09-10T06:21:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25316.077211 secs 2023-09-10T06:21:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25315.464298 secs 2023-09-10T06:21:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25316.077211 secs 2023-09-10T06:21:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:25 smithi082 ceph-mon[98694]: pgmap v13662: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 12 op/s 2023-09-10T06:21:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:25 smithi098 ceph-mon[103124]: pgmap v13662: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 12 op/s 2023-09-10T06:21:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:25 smithi170 ceph-mon[102832]: pgmap v13662: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 12 op/s 2023-09-10T06:21:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:27 smithi082 ceph-mon[98694]: pgmap v13663: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T06:21:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:27 smithi098 ceph-mon[103124]: pgmap v13663: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T06:21:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:27 smithi170 ceph-mon[102832]: pgmap v13663: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T06:21:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:29 smithi082 ceph-mon[98694]: pgmap v13664: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 9 op/s 2023-09-10T06:21:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25320.464355 secs 2023-09-10T06:21:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25321.077276 secs 2023-09-10T06:21:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:29 smithi098 ceph-mon[103124]: pgmap v13664: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 9 op/s 2023-09-10T06:21:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25320.464355 secs 2023-09-10T06:21:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25321.077276 secs 2023-09-10T06:21:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:29 smithi170 ceph-mon[102832]: pgmap v13664: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 9 op/s 2023-09-10T06:21:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25320.464355 secs 2023-09-10T06:21:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25321.077276 secs 2023-09-10T06:21:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:31 smithi098 ceph-mon[103124]: pgmap v13665: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 9 op/s 2023-09-10T06:21:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:21:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:31 smithi170 ceph-mon[102832]: pgmap v13665: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 9 op/s 2023-09-10T06:21:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:21:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:31 smithi082 ceph-mon[98694]: pgmap v13665: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 9 op/s 2023-09-10T06:21:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:21:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:21:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:21:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:21:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:21:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:21:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:21:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:33 smithi098 ceph-mon[103124]: pgmap v13666: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 234 KiB/s wr, 14 op/s 2023-09-10T06:21:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25324.456520 secs 2023-09-10T06:21:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:33 smithi170 ceph-mon[102832]: pgmap v13666: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 234 KiB/s wr, 14 op/s 2023-09-10T06:21:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25324.456520 secs 2023-09-10T06:21:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:33 smithi082 ceph-mon[98694]: pgmap v13666: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 234 KiB/s wr, 14 op/s 2023-09-10T06:21:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25324.456520 secs 2023-09-10T06:21:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25325.464413 secs 2023-09-10T06:21:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25326.077342 secs 2023-09-10T06:21:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25325.464413 secs 2023-09-10T06:21:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25326.077342 secs 2023-09-10T06:21:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25325.464413 secs 2023-09-10T06:21:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25326.077342 secs 2023-09-10T06:21:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:35 smithi098 ceph-mon[103124]: pgmap v13667: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 147 KiB/s wr, 9 op/s 2023-09-10T06:21:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:35 smithi170 ceph-mon[102832]: pgmap v13667: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 147 KiB/s wr, 9 op/s 2023-09-10T06:21:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:35 smithi082 ceph-mon[98694]: pgmap v13667: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 147 KiB/s wr, 9 op/s 2023-09-10T06:21:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:21:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:21:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:21:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:37 smithi098 ceph-mon[103124]: pgmap v13668: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 296 KiB/s wr, 15 op/s 2023-09-10T06:21:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:21:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:37 smithi170 ceph-mon[102832]: pgmap v13668: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 296 KiB/s wr, 15 op/s 2023-09-10T06:21:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:21:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:37 smithi082 ceph-mon[98694]: pgmap v13668: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 296 KiB/s wr, 15 op/s 2023-09-10T06:21:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:21:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:39 smithi098 ceph-mon[103124]: pgmap v13669: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T06:21:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25330.464498 secs 2023-09-10T06:21:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25331.077406 secs 2023-09-10T06:21:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:21:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:39 smithi170 ceph-mon[102832]: pgmap v13669: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T06:21:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25330.464498 secs 2023-09-10T06:21:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25331.077406 secs 2023-09-10T06:21:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:21:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:39 smithi082 ceph-mon[98694]: pgmap v13669: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T06:21:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25330.464498 secs 2023-09-10T06:21:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25331.077406 secs 2023-09-10T06:21:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:21:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:21:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:21:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:21:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:21:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:21:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:21:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:41 smithi170 ceph-mon[102832]: pgmap v13670: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T06:21:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:41 smithi082 ceph-mon[98694]: pgmap v13670: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T06:21:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:41 smithi098 ceph-mon[103124]: pgmap v13670: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T06:21:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:43 smithi170 ceph-mon[102832]: pgmap v13671: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 428 KiB/s wr, 17 op/s 2023-09-10T06:21:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:43 smithi082 ceph-mon[98694]: pgmap v13671: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 428 KiB/s wr, 17 op/s 2023-09-10T06:21:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:43 smithi098 ceph-mon[103124]: pgmap v13671: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 428 KiB/s wr, 17 op/s 2023-09-10T06:21:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25335.464576 secs 2023-09-10T06:21:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:21:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25336.077474 secs 2023-09-10T06:21:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25335.464576 secs 2023-09-10T06:21:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:21:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25336.077474 secs 2023-09-10T06:21:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25335.464576 secs 2023-09-10T06:21:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:21:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25336.077474 secs 2023-09-10T06:21:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:45 smithi170 ceph-mon[102832]: pgmap v13672: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T06:21:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:21:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:46.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:45 smithi082 ceph-mon[98694]: pgmap v13672: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T06:21:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:21:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:45 smithi098 ceph-mon[103124]: pgmap v13672: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 281 KiB/s wr, 12 op/s 2023-09-10T06:21:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:21:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:46 smithi170 ceph-mon[102832]: pgmap v13673: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 514 KiB/s wr, 19 op/s 2023-09-10T06:21:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:46 smithi082 ceph-mon[98694]: pgmap v13673: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 514 KiB/s wr, 19 op/s 2023-09-10T06:21:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:46 smithi098 ceph-mon[103124]: pgmap v13673: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 514 KiB/s wr, 19 op/s 2023-09-10T06:21:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:48 smithi170 ceph-mon[102832]: pgmap v13674: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 365 KiB/s wr, 13 op/s 2023-09-10T06:21:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:21:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:21:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:21:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:21:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:48 smithi082 ceph-mon[98694]: pgmap v13674: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 365 KiB/s wr, 13 op/s 2023-09-10T06:21:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:21:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:21:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:21:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:21:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:48 smithi098 ceph-mon[103124]: pgmap v13674: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 365 KiB/s wr, 13 op/s 2023-09-10T06:21:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:21:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:21:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:21:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:21:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25340.464660 secs 2023-09-10T06:21:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25341.077544 secs 2023-09-10T06:21:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25340.464660 secs 2023-09-10T06:21:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25341.077544 secs 2023-09-10T06:21:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25340.464660 secs 2023-09-10T06:21:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25341.077544 secs 2023-09-10T06:21:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:51 smithi082 ceph-mon[98694]: pgmap v13675: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 365 KiB/s wr, 13 op/s 2023-09-10T06:21:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:51 smithi098 ceph-mon[103124]: pgmap v13675: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 365 KiB/s wr, 13 op/s 2023-09-10T06:21:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:51 smithi170 ceph-mon[102832]: pgmap v13675: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 365 KiB/s wr, 13 op/s 2023-09-10T06:21:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:53 smithi082 ceph-mon[98694]: pgmap v13676: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 488 KiB/s wr, 19 op/s 2023-09-10T06:21:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:21:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:53 smithi098 ceph-mon[103124]: pgmap v13676: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 488 KiB/s wr, 19 op/s 2023-09-10T06:21:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:21:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:53 smithi170 ceph-mon[102832]: pgmap v13676: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 488 KiB/s wr, 19 op/s 2023-09-10T06:21:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:21:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25345.464724 secs 2023-09-10T06:21:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25346.077609 secs 2023-09-10T06:21:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25345.464724 secs 2023-09-10T06:21:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25346.077609 secs 2023-09-10T06:21:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25345.464724 secs 2023-09-10T06:21:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25346.077609 secs 2023-09-10T06:21:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:55 smithi098 ceph-mon[103124]: pgmap v13677: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 356 KiB/s wr, 12 op/s 2023-09-10T06:21:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:55 smithi170 ceph-mon[102832]: pgmap v13677: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 356 KiB/s wr, 12 op/s 2023-09-10T06:21:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:55 smithi082 ceph-mon[98694]: pgmap v13677: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 356 KiB/s wr, 12 op/s 2023-09-10T06:21:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:21:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:21:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:57.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:21:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:21:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:57 smithi098 ceph-mon[103124]: pgmap v13678: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 728 KiB/s wr, 19 op/s 2023-09-10T06:21:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:57 smithi170 ceph-mon[102832]: pgmap v13678: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 728 KiB/s wr, 19 op/s 2023-09-10T06:21:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:57 smithi082 ceph-mon[98694]: pgmap v13678: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 728 KiB/s wr, 19 op/s 2023-09-10T06:21:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25349.456874 secs 2023-09-10T06:21:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25349.456874 secs 2023-09-10T06:21:59.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25349.456874 secs 2023-09-10T06:22:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:59 smithi098 ceph-mon[103124]: pgmap v13679: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 495 KiB/s wr, 12 op/s 2023-09-10T06:22:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25350.464784 secs 2023-09-10T06:22:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:21:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25351.077676 secs 2023-09-10T06:22:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:59 smithi170 ceph-mon[102832]: pgmap v13679: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 495 KiB/s wr, 12 op/s 2023-09-10T06:22:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25350.464784 secs 2023-09-10T06:22:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:21:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25351.077676 secs 2023-09-10T06:22:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:59 smithi082 ceph-mon[98694]: pgmap v13679: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 495 KiB/s wr, 12 op/s 2023-09-10T06:22:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25350.464784 secs 2023-09-10T06:22:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:21:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25351.077676 secs 2023-09-10T06:22:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:22:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:22:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:22:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:22:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:22:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:22:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:01 smithi098 ceph-mon[103124]: pgmap v13680: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 495 KiB/s wr, 12 op/s 2023-09-10T06:22:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:01 smithi170 ceph-mon[102832]: pgmap v13680: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 495 KiB/s wr, 12 op/s 2023-09-10T06:22:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:01 smithi082 ceph-mon[98694]: pgmap v13680: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 495 KiB/s wr, 12 op/s 2023-09-10T06:22:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:03 smithi170 ceph-mon[102832]: pgmap v13681: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 576 KiB/s wr, 17 op/s 2023-09-10T06:22:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25354.456934 secs 2023-09-10T06:22:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:03 smithi082 ceph-mon[98694]: pgmap v13681: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 576 KiB/s wr, 17 op/s 2023-09-10T06:22:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25354.456934 secs 2023-09-10T06:22:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:03 smithi098 ceph-mon[103124]: pgmap v13681: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 576 KiB/s wr, 17 op/s 2023-09-10T06:22:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25354.456934 secs 2023-09-10T06:22:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25355.464844 secs 2023-09-10T06:22:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:22:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25356.077740 secs 2023-09-10T06:22:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25355.464844 secs 2023-09-10T06:22:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:22:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25356.077740 secs 2023-09-10T06:22:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25355.464844 secs 2023-09-10T06:22:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:22:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25356.077740 secs 2023-09-10T06:22:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:05 smithi170 ceph-mon[102832]: pgmap v13682: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 453 KiB/s wr, 12 op/s 2023-09-10T06:22:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:22:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:05 smithi082 ceph-mon[98694]: pgmap v13682: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 453 KiB/s wr, 12 op/s 2023-09-10T06:22:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:22:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:05 smithi098 ceph-mon[103124]: pgmap v13682: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 453 KiB/s wr, 12 op/s 2023-09-10T06:22:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:22:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:07 smithi170 ceph-mon[102832]: pgmap v13683: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 623 KiB/s wr, 32 op/s 2023-09-10T06:22:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:07 smithi082 ceph-mon[98694]: pgmap v13683: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 623 KiB/s wr, 32 op/s 2023-09-10T06:22:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:07 smithi098 ceph-mon[103124]: pgmap v13683: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 623 KiB/s wr, 32 op/s 2023-09-10T06:22:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25359.456993 secs 2023-09-10T06:22:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:08 smithi170 ceph-mon[102832]: osdmap e923: 12 total, 12 up, 12 in 2023-09-10T06:22:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25359.456993 secs 2023-09-10T06:22:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:08 smithi082 ceph-mon[98694]: osdmap e923: 12 total, 12 up, 12 in 2023-09-10T06:22:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25359.456993 secs 2023-09-10T06:22:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:08 smithi098 ceph-mon[103124]: osdmap e923: 12 total, 12 up, 12 in 2023-09-10T06:22:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:09 smithi170 ceph-mon[102832]: pgmap v13685: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 301 KiB/s wr, 31 op/s 2023-09-10T06:22:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25360.464904 secs 2023-09-10T06:22:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25361.077808 secs 2023-09-10T06:22:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:09 smithi082 ceph-mon[98694]: pgmap v13685: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 301 KiB/s wr, 31 op/s 2023-09-10T06:22:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25360.464904 secs 2023-09-10T06:22:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25361.077808 secs 2023-09-10T06:22:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:09 smithi098 ceph-mon[103124]: pgmap v13685: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 301 KiB/s wr, 31 op/s 2023-09-10T06:22:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25360.464904 secs 2023-09-10T06:22:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25361.077808 secs 2023-09-10T06:22:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:11 smithi082 ceph-mon[98694]: pgmap v13686: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 301 KiB/s wr, 31 op/s 2023-09-10T06:22:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:11 smithi098 ceph-mon[103124]: pgmap v13686: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 301 KiB/s wr, 31 op/s 2023-09-10T06:22:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:11 smithi170 ceph-mon[102832]: pgmap v13686: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 301 KiB/s wr, 31 op/s 2023-09-10T06:22:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:13 smithi082 ceph-mon[98694]: pgmap v13687: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 284 KiB/s wr, 30 op/s 2023-09-10T06:22:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25364.457055 secs 2023-09-10T06:22:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:13 smithi098 ceph-mon[103124]: pgmap v13687: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 284 KiB/s wr, 30 op/s 2023-09-10T06:22:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25364.457055 secs 2023-09-10T06:22:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:13 smithi170 ceph-mon[102832]: pgmap v13687: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 284 KiB/s wr, 30 op/s 2023-09-10T06:22:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25364.457055 secs 2023-09-10T06:22:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25365.464961 secs 2023-09-10T06:22:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25366.077874 secs 2023-09-10T06:22:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25365.464961 secs 2023-09-10T06:22:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25366.077874 secs 2023-09-10T06:22:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25365.464961 secs 2023-09-10T06:22:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25366.077874 secs 2023-09-10T06:22:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:15 smithi098 ceph-mon[103124]: pgmap v13688: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 284 KiB/s wr, 30 op/s 2023-09-10T06:22:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:15 smithi098 ceph-mon[103124]: osdmap e924: 12 total, 12 up, 12 in 2023-09-10T06:22:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:15 smithi170 ceph-mon[102832]: pgmap v13688: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 284 KiB/s wr, 30 op/s 2023-09-10T06:22:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:15 smithi170 ceph-mon[102832]: osdmap e924: 12 total, 12 up, 12 in 2023-09-10T06:22:16.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:15 smithi082 ceph-mon[98694]: pgmap v13688: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 284 KiB/s wr, 30 op/s 2023-09-10T06:22:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:15 smithi082 ceph-mon[98694]: osdmap e924: 12 total, 12 up, 12 in 2023-09-10T06:22:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:17 smithi098 ceph-mon[103124]: pgmap v13690: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 101 KiB/s wr, 14 op/s 2023-09-10T06:22:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:17 smithi170 ceph-mon[102832]: pgmap v13690: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 101 KiB/s wr, 14 op/s 2023-09-10T06:22:18.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:17 smithi082 ceph-mon[98694]: pgmap v13690: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 101 KiB/s wr, 14 op/s 2023-09-10T06:22:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25369.457118 secs 2023-09-10T06:22:19.183 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25369.457118 secs 2023-09-10T06:22:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25369.457118 secs 2023-09-10T06:22:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:19 smithi098 ceph-mon[103124]: pgmap v13691: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T06:22:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25370.465021 secs 2023-09-10T06:22:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25371.077943 secs 2023-09-10T06:22:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:19 smithi170 ceph-mon[102832]: pgmap v13691: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T06:22:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25370.465021 secs 2023-09-10T06:22:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25371.077943 secs 2023-09-10T06:22:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:19 smithi082 ceph-mon[98694]: pgmap v13691: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T06:22:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25370.465021 secs 2023-09-10T06:22:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25371.077943 secs 2023-09-10T06:22:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:21 smithi098 ceph-mon[103124]: pgmap v13692: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T06:22:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:21 smithi170 ceph-mon[102832]: pgmap v13692: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T06:22:22.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:21 smithi082 ceph-mon[98694]: pgmap v13692: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 81 KiB/s wr, 11 op/s 2023-09-10T06:22:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:23 smithi098 ceph-mon[103124]: pgmap v13693: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2023-09-10T06:22:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25374.457179 secs 2023-09-10T06:22:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:23 smithi170 ceph-mon[102832]: pgmap v13693: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2023-09-10T06:22:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25374.457179 secs 2023-09-10T06:22:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:23 smithi082 ceph-mon[98694]: pgmap v13693: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2023-09-10T06:22:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25374.457179 secs 2023-09-10T06:22:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25375.465098 secs 2023-09-10T06:22:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25376.078009 secs 2023-09-10T06:22:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25375.465098 secs 2023-09-10T06:22:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25376.078009 secs 2023-09-10T06:22:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25375.465098 secs 2023-09-10T06:22:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25376.078009 secs 2023-09-10T06:22:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:25 smithi098 ceph-mon[103124]: pgmap v13694: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2023-09-10T06:22:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:25 smithi170 ceph-mon[102832]: pgmap v13694: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2023-09-10T06:22:26.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:25 smithi082 ceph-mon[98694]: pgmap v13694: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2023-09-10T06:22:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:27 smithi098 ceph-mon[103124]: pgmap v13695: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:22:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:27 smithi170 ceph-mon[102832]: pgmap v13695: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:22:28.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:27 smithi082 ceph-mon[98694]: pgmap v13695: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:22:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25379.457239 secs 2023-09-10T06:22:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25379.457239 secs 2023-09-10T06:22:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25379.457239 secs 2023-09-10T06:22:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:29 smithi098 ceph-mon[103124]: pgmap v13696: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T06:22:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25380.465156 secs 2023-09-10T06:22:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25381.078077 secs 2023-09-10T06:22:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:29 smithi170 ceph-mon[102832]: pgmap v13696: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T06:22:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25380.465156 secs 2023-09-10T06:22:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25381.078077 secs 2023-09-10T06:22:30.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:29 smithi082 ceph-mon[98694]: pgmap v13696: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T06:22:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25380.465156 secs 2023-09-10T06:22:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25381.078077 secs 2023-09-10T06:22:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:31 smithi098 ceph-mon[103124]: pgmap v13697: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T06:22:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:31 smithi170 ceph-mon[102832]: pgmap v13697: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T06:22:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:31 smithi082 ceph-mon[98694]: pgmap v13697: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T06:22:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:33 smithi170 ceph-mon[102832]: pgmap v13698: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 204 KiB/s wr, 15 op/s 2023-09-10T06:22:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:22:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:33 smithi082 ceph-mon[98694]: pgmap v13698: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 204 KiB/s wr, 15 op/s 2023-09-10T06:22:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:22:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:33 smithi098 ceph-mon[103124]: pgmap v13698: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 204 KiB/s wr, 15 op/s 2023-09-10T06:22:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:22:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25385.465214 secs 2023-09-10T06:22:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25386.078143 secs 2023-09-10T06:22:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25385.465214 secs 2023-09-10T06:22:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25386.078143 secs 2023-09-10T06:22:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25385.465214 secs 2023-09-10T06:22:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25386.078143 secs 2023-09-10T06:22:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:35 smithi170 ceph-mon[102832]: pgmap v13699: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 204 KiB/s wr, 10 op/s 2023-09-10T06:22:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:22:36.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:35 smithi082 ceph-mon[98694]: pgmap v13699: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 204 KiB/s wr, 10 op/s 2023-09-10T06:22:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:22:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:35 smithi098 ceph-mon[103124]: pgmap v13699: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 204 KiB/s wr, 10 op/s 2023-09-10T06:22:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:22:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:22:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:22:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:36 smithi170 ceph-mon[102832]: pgmap v13700: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 633 KiB/s wr, 17 op/s 2023-09-10T06:22:37.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:22:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:22:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:36 smithi082 ceph-mon[98694]: pgmap v13700: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 633 KiB/s wr, 17 op/s 2023-09-10T06:22:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:22:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:22:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:36 smithi098 ceph-mon[103124]: pgmap v13700: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 633 KiB/s wr, 17 op/s 2023-09-10T06:22:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:22:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:22:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:22:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:38 smithi170 ceph-mon[102832]: pgmap v13701: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 545 KiB/s wr, 12 op/s 2023-09-10T06:22:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:38 smithi082 ceph-mon[98694]: pgmap v13701: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 545 KiB/s wr, 12 op/s 2023-09-10T06:22:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:38 smithi098 ceph-mon[103124]: pgmap v13701: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 545 KiB/s wr, 12 op/s 2023-09-10T06:22:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25390.465299 secs 2023-09-10T06:22:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25391.078210 secs 2023-09-10T06:22:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25390.465299 secs 2023-09-10T06:22:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25391.078210 secs 2023-09-10T06:22:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25390.465299 secs 2023-09-10T06:22:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25391.078210 secs 2023-09-10T06:22:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:22:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:40 smithi170 ceph-mon[102832]: pgmap v13702: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 545 KiB/s wr, 12 op/s 2023-09-10T06:22:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:22:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:40 smithi082 ceph-mon[98694]: pgmap v13702: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 545 KiB/s wr, 12 op/s 2023-09-10T06:22:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:22:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:40 smithi098 ceph-mon[103124]: pgmap v13702: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 545 KiB/s wr, 12 op/s 2023-09-10T06:22:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:42 smithi170 ceph-mon[102832]: pgmap v13703: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 565 KiB/s wr, 18 op/s 2023-09-10T06:22:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:42 smithi082 ceph-mon[98694]: pgmap v13703: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 565 KiB/s wr, 18 op/s 2023-09-10T06:22:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:42 smithi098 ceph-mon[103124]: pgmap v13703: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 565 KiB/s wr, 18 op/s 2023-09-10T06:22:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25395.465358 secs 2023-09-10T06:22:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25396.078274 secs 2023-09-10T06:22:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:44 smithi170 ceph-mon[102832]: pgmap v13704: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 449 KiB/s wr, 13 op/s 2023-09-10T06:22:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25395.465358 secs 2023-09-10T06:22:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25396.078274 secs 2023-09-10T06:22:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:44 smithi082 ceph-mon[98694]: pgmap v13704: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 449 KiB/s wr, 13 op/s 2023-09-10T06:22:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25395.465358 secs 2023-09-10T06:22:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25396.078274 secs 2023-09-10T06:22:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:44 smithi098 ceph-mon[103124]: pgmap v13704: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 449 KiB/s wr, 13 op/s 2023-09-10T06:22:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:47 smithi098 ceph-mon[103124]: pgmap v13705: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 756 KiB/s wr, 19 op/s 2023-09-10T06:22:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:47 smithi170 ceph-mon[102832]: pgmap v13705: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 756 KiB/s wr, 19 op/s 2023-09-10T06:22:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:47 smithi082 ceph-mon[98694]: pgmap v13705: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 756 KiB/s wr, 19 op/s 2023-09-10T06:22:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:49 smithi098 ceph-mon[103124]: pgmap v13706: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 327 KiB/s wr, 12 op/s 2023-09-10T06:22:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25400.465418 secs 2023-09-10T06:22:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:22:50.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:50.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25401.078336 secs 2023-09-10T06:22:50.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:22:50.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:22:50.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:22:50.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:22:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:49 smithi170 ceph-mon[102832]: pgmap v13706: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 327 KiB/s wr, 12 op/s 2023-09-10T06:22:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25400.465418 secs 2023-09-10T06:22:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:22:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:50.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25401.078336 secs 2023-09-10T06:22:50.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:22:50.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:22:50.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:22:50.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:22:50.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:49 smithi082 ceph-mon[98694]: pgmap v13706: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 327 KiB/s wr, 12 op/s 2023-09-10T06:22:50.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25400.465418 secs 2023-09-10T06:22:50.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:22:50.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:50.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25401.078336 secs 2023-09-10T06:22:50.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:22:50.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:22:50.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:22:50.212 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:22:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:51 smithi098 ceph-mon[103124]: pgmap v13707: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 327 KiB/s wr, 12 op/s 2023-09-10T06:22:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:51 smithi170 ceph-mon[102832]: pgmap v13707: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 327 KiB/s wr, 12 op/s 2023-09-10T06:22:52.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:51 smithi082 ceph-mon[98694]: pgmap v13707: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 327 KiB/s wr, 12 op/s 2023-09-10T06:22:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:22:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:22:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:22:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:53 smithi098 ceph-mon[103124]: pgmap v13708: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 714 KiB/s wr, 18 op/s 2023-09-10T06:22:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:22:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25404.457665 secs 2023-09-10T06:22:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:53 smithi170 ceph-mon[102832]: pgmap v13708: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 714 KiB/s wr, 18 op/s 2023-09-10T06:22:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:22:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25404.457665 secs 2023-09-10T06:22:54.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:53 smithi082 ceph-mon[98694]: pgmap v13708: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 714 KiB/s wr, 18 op/s 2023-09-10T06:22:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:22:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25404.457665 secs 2023-09-10T06:22:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25405.465475 secs 2023-09-10T06:22:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25406.078402 secs 2023-09-10T06:22:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25405.465475 secs 2023-09-10T06:22:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25406.078402 secs 2023-09-10T06:22:55.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25405.465475 secs 2023-09-10T06:22:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25406.078402 secs 2023-09-10T06:22:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:55 smithi170 ceph-mon[102832]: pgmap v13709: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 695 KiB/s wr, 13 op/s 2023-09-10T06:22:56.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:55 smithi082 ceph-mon[98694]: pgmap v13709: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 695 KiB/s wr, 13 op/s 2023-09-10T06:22:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:55 smithi098 ceph-mon[103124]: pgmap v13709: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 695 KiB/s wr, 13 op/s 2023-09-10T06:22:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:22:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:22:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:22:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:22:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:22:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:22:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:22:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:57 smithi170 ceph-mon[102832]: pgmap v13710: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 776 KiB/s wr, 18 op/s 2023-09-10T06:22:58.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:57 smithi082 ceph-mon[98694]: pgmap v13710: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 776 KiB/s wr, 18 op/s 2023-09-10T06:22:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:57 smithi098 ceph-mon[103124]: pgmap v13710: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 776 KiB/s wr, 18 op/s 2023-09-10T06:22:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25409.457728 secs 2023-09-10T06:22:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25409.457728 secs 2023-09-10T06:22:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25409.457728 secs 2023-09-10T06:23:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:59 smithi170 ceph-mon[102832]: pgmap v13711: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 468 KiB/s wr, 12 op/s 2023-09-10T06:23:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25410.465494 secs 2023-09-10T06:23:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:22:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25411.078466 secs 2023-09-10T06:23:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:59 smithi082 ceph-mon[98694]: pgmap v13711: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 468 KiB/s wr, 12 op/s 2023-09-10T06:23:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25410.465494 secs 2023-09-10T06:23:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:22:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25411.078466 secs 2023-09-10T06:23:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:59 smithi098 ceph-mon[103124]: pgmap v13711: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 468 KiB/s wr, 12 op/s 2023-09-10T06:23:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25410.465494 secs 2023-09-10T06:23:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:22:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25411.078466 secs 2023-09-10T06:23:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:01 smithi170 ceph-mon[102832]: pgmap v13712: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 468 KiB/s wr, 12 op/s 2023-09-10T06:23:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:01 smithi082 ceph-mon[98694]: pgmap v13712: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 468 KiB/s wr, 12 op/s 2023-09-10T06:23:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:01 smithi098 ceph-mon[103124]: pgmap v13712: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 468 KiB/s wr, 12 op/s 2023-09-10T06:23:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:03 smithi170 ceph-mon[102832]: pgmap v13713: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 574 KiB/s wr, 16 op/s 2023-09-10T06:23:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25414.457775 secs 2023-09-10T06:23:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:03 smithi082 ceph-mon[98694]: pgmap v13713: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 574 KiB/s wr, 16 op/s 2023-09-10T06:23:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25414.457775 secs 2023-09-10T06:23:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:03 smithi098 ceph-mon[103124]: pgmap v13713: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 574 KiB/s wr, 16 op/s 2023-09-10T06:23:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25414.457775 secs 2023-09-10T06:23:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25415.465576 secs 2023-09-10T06:23:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25416.078530 secs 2023-09-10T06:23:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25415.465576 secs 2023-09-10T06:23:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25416.078530 secs 2023-09-10T06:23:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25415.465576 secs 2023-09-10T06:23:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25416.078530 secs 2023-09-10T06:23:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:05 smithi170 ceph-mon[102832]: pgmap v13714: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 10 op/s 2023-09-10T06:23:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:05 smithi082 ceph-mon[98694]: pgmap v13714: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 10 op/s 2023-09-10T06:23:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:05 smithi098 ceph-mon[103124]: pgmap v13714: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 10 op/s 2023-09-10T06:23:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:07 smithi170 ceph-mon[102832]: pgmap v13715: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 318 KiB/s wr, 29 op/s 2023-09-10T06:23:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:07 smithi082 ceph-mon[98694]: pgmap v13715: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 318 KiB/s wr, 29 op/s 2023-09-10T06:23:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:07 smithi098 ceph-mon[103124]: pgmap v13715: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 318 KiB/s wr, 29 op/s 2023-09-10T06:23:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25419.457837 secs 2023-09-10T06:23:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:08 smithi170 ceph-mon[102832]: osdmap e925: 12 total, 12 up, 12 in 2023-09-10T06:23:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:08 smithi170 ceph-mon[102832]: pgmap v13717: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 285 KiB/s wr, 29 op/s 2023-09-10T06:23:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25419.457837 secs 2023-09-10T06:23:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:08 smithi082 ceph-mon[98694]: osdmap e925: 12 total, 12 up, 12 in 2023-09-10T06:23:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:08 smithi082 ceph-mon[98694]: pgmap v13717: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 285 KiB/s wr, 29 op/s 2023-09-10T06:23:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25419.457837 secs 2023-09-10T06:23:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:08 smithi098 ceph-mon[103124]: osdmap e925: 12 total, 12 up, 12 in 2023-09-10T06:23:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:08 smithi098 ceph-mon[103124]: pgmap v13717: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 285 KiB/s wr, 29 op/s 2023-09-10T06:23:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25420.465651 secs 2023-09-10T06:23:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25421.078596 secs 2023-09-10T06:23:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25420.465651 secs 2023-09-10T06:23:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25421.078596 secs 2023-09-10T06:23:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25420.465651 secs 2023-09-10T06:23:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25421.078596 secs 2023-09-10T06:23:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:11 smithi098 ceph-mon[103124]: pgmap v13718: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 285 KiB/s wr, 29 op/s 2023-09-10T06:23:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:11 smithi170 ceph-mon[102832]: pgmap v13718: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 285 KiB/s wr, 29 op/s 2023-09-10T06:23:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:11 smithi082 ceph-mon[98694]: pgmap v13718: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 285 KiB/s wr, 29 op/s 2023-09-10T06:23:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:13 smithi098 ceph-mon[103124]: pgmap v13719: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 182 KiB/s wr, 29 op/s 2023-09-10T06:23:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25424.457966 secs 2023-09-10T06:23:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:13 smithi170 ceph-mon[102832]: pgmap v13719: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 182 KiB/s wr, 29 op/s 2023-09-10T06:23:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25424.457966 secs 2023-09-10T06:23:14.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:13 smithi082 ceph-mon[98694]: pgmap v13719: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 182 KiB/s wr, 29 op/s 2023-09-10T06:23:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25424.457966 secs 2023-09-10T06:23:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25425.465713 secs 2023-09-10T06:23:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25426.078663 secs 2023-09-10T06:23:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25425.465713 secs 2023-09-10T06:23:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25426.078663 secs 2023-09-10T06:23:15.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25425.465713 secs 2023-09-10T06:23:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25426.078663 secs 2023-09-10T06:23:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:15 smithi098 ceph-mon[103124]: pgmap v13720: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 182 KiB/s wr, 29 op/s 2023-09-10T06:23:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:15 smithi098 ceph-mon[103124]: osdmap e926: 12 total, 12 up, 12 in 2023-09-10T06:23:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:15 smithi170 ceph-mon[102832]: pgmap v13720: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 182 KiB/s wr, 29 op/s 2023-09-10T06:23:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:15 smithi170 ceph-mon[102832]: osdmap e926: 12 total, 12 up, 12 in 2023-09-10T06:23:16.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:15 smithi082 ceph-mon[98694]: pgmap v13720: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 182 KiB/s wr, 29 op/s 2023-09-10T06:23:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:15 smithi082 ceph-mon[98694]: osdmap e926: 12 total, 12 up, 12 in 2023-09-10T06:23:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:17 smithi098 ceph-mon[103124]: pgmap v13722: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 14 op/s 2023-09-10T06:23:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:17 smithi170 ceph-mon[102832]: pgmap v13722: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 14 op/s 2023-09-10T06:23:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:17 smithi082 ceph-mon[98694]: pgmap v13722: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 14 op/s 2023-09-10T06:23:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25429.458047 secs 2023-09-10T06:23:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25429.458047 secs 2023-09-10T06:23:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25429.458047 secs 2023-09-10T06:23:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:19 smithi098 ceph-mon[103124]: pgmap v13723: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T06:23:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25430.465794 secs 2023-09-10T06:23:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25431.078726 secs 2023-09-10T06:23:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:19 smithi170 ceph-mon[102832]: pgmap v13723: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T06:23:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25430.465794 secs 2023-09-10T06:23:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25431.078726 secs 2023-09-10T06:23:20.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:19 smithi082 ceph-mon[98694]: pgmap v13723: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T06:23:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25430.465794 secs 2023-09-10T06:23:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25431.078726 secs 2023-09-10T06:23:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:21 smithi098 ceph-mon[103124]: pgmap v13724: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T06:23:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:21 smithi170 ceph-mon[102832]: pgmap v13724: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T06:23:22.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:21 smithi082 ceph-mon[98694]: pgmap v13724: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T06:23:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:23 smithi098 ceph-mon[103124]: pgmap v13725: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T06:23:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25434.458109 secs 2023-09-10T06:23:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:23 smithi170 ceph-mon[102832]: pgmap v13725: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T06:23:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25434.458109 secs 2023-09-10T06:23:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:23 smithi082 ceph-mon[98694]: pgmap v13725: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T06:23:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25434.458109 secs 2023-09-10T06:23:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25435.465876 secs 2023-09-10T06:23:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25436.078793 secs 2023-09-10T06:23:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25435.465876 secs 2023-09-10T06:23:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25436.078793 secs 2023-09-10T06:23:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25435.465876 secs 2023-09-10T06:23:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25436.078793 secs 2023-09-10T06:23:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:25 smithi170 ceph-mon[102832]: pgmap v13726: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T06:23:26.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:25 smithi082 ceph-mon[98694]: pgmap v13726: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T06:23:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:25 smithi098 ceph-mon[103124]: pgmap v13726: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T06:23:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:27 smithi170 ceph-mon[102832]: pgmap v13727: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:23:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:27 smithi082 ceph-mon[98694]: pgmap v13727: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:23:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:27 smithi098 ceph-mon[103124]: pgmap v13727: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:23:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25439.458168 secs 2023-09-10T06:23:29.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25439.458168 secs 2023-09-10T06:23:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25439.458168 secs 2023-09-10T06:23:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:29 smithi170 ceph-mon[102832]: pgmap v13728: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T06:23:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25440.465935 secs 2023-09-10T06:23:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:29 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:23:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:29 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:23:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25441.078858 secs 2023-09-10T06:23:30.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:29 smithi082 ceph-mon[98694]: pgmap v13728: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T06:23:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25440.465935 secs 2023-09-10T06:23:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:29 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:23:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:29 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:23:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25441.078858 secs 2023-09-10T06:23:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:29 smithi098 ceph-mon[103124]: pgmap v13728: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T06:23:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25440.465935 secs 2023-09-10T06:23:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:29 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:23:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:29 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:23:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25441.078858 secs 2023-09-10T06:23:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:31 smithi170 ceph-mon[102832]: pgmap v13729: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T06:23:32.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:31 smithi082 ceph-mon[98694]: pgmap v13729: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T06:23:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:31 smithi098 ceph-mon[103124]: pgmap v13729: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T06:23:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:23:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:23:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:23:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:23:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:23:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:23:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:33 smithi170 ceph-mon[102832]: pgmap v13730: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 198 KiB/s wr, 15 op/s 2023-09-10T06:23:34.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:33 smithi082 ceph-mon[98694]: pgmap v13730: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 198 KiB/s wr, 15 op/s 2023-09-10T06:23:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:33 smithi098 ceph-mon[103124]: pgmap v13730: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 198 KiB/s wr, 15 op/s 2023-09-10T06:23:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25445.465995 secs 2023-09-10T06:23:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25446.078923 secs 2023-09-10T06:23:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25445.465995 secs 2023-09-10T06:23:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25446.078923 secs 2023-09-10T06:23:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25445.465995 secs 2023-09-10T06:23:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25446.078923 secs 2023-09-10T06:23:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:35 smithi170 ceph-mon[102832]: pgmap v13731: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 110 KiB/s wr, 10 op/s 2023-09-10T06:23:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:23:36.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:35 smithi082 ceph-mon[98694]: pgmap v13731: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 110 KiB/s wr, 10 op/s 2023-09-10T06:23:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:23:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:35 smithi098 ceph-mon[103124]: pgmap v13731: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 110 KiB/s wr, 10 op/s 2023-09-10T06:23:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:23:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:23:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:23:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:23:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:36 smithi170 ceph-mon[102832]: pgmap v13732: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 298 KiB/s wr, 15 op/s 2023-09-10T06:23:37.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:23:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:23:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:23:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:36 smithi082 ceph-mon[98694]: pgmap v13732: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 298 KiB/s wr, 15 op/s 2023-09-10T06:23:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:23:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:23:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:23:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:36 smithi098 ceph-mon[103124]: pgmap v13732: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 298 KiB/s wr, 15 op/s 2023-09-10T06:23:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:38 smithi170 ceph-mon[102832]: pgmap v13733: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 298 KiB/s wr, 11 op/s 2023-09-10T06:23:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:38 smithi082 ceph-mon[98694]: pgmap v13733: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 298 KiB/s wr, 11 op/s 2023-09-10T06:23:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:38 smithi098 ceph-mon[103124]: pgmap v13733: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 298 KiB/s wr, 11 op/s 2023-09-10T06:23:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25450.466053 secs 2023-09-10T06:23:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25451.078986 secs 2023-09-10T06:23:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25450.466053 secs 2023-09-10T06:23:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25451.078986 secs 2023-09-10T06:23:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25450.466053 secs 2023-09-10T06:23:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25451.078986 secs 2023-09-10T06:23:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:41 smithi098 ceph-mon[103124]: pgmap v13734: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 298 KiB/s wr, 11 op/s 2023-09-10T06:23:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:41 smithi170 ceph-mon[102832]: pgmap v13734: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 298 KiB/s wr, 11 op/s 2023-09-10T06:23:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:41 smithi082 ceph-mon[98694]: pgmap v13734: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 298 KiB/s wr, 11 op/s 2023-09-10T06:23:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:43 smithi098 ceph-mon[103124]: pgmap v13735: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 361 KiB/s wr, 17 op/s 2023-09-10T06:23:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:43 smithi170 ceph-mon[102832]: pgmap v13735: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 361 KiB/s wr, 17 op/s 2023-09-10T06:23:44.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:43 smithi082 ceph-mon[98694]: pgmap v13735: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 361 KiB/s wr, 17 op/s 2023-09-10T06:23:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25455.466112 secs 2023-09-10T06:23:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:23:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:23:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25456.079052 secs 2023-09-10T06:23:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25455.466112 secs 2023-09-10T06:23:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:23:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:23:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25456.079052 secs 2023-09-10T06:23:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25455.466112 secs 2023-09-10T06:23:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:23:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:23:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25456.079052 secs 2023-09-10T06:23:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:45 smithi098 ceph-mon[103124]: pgmap v13736: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 251 KiB/s wr, 12 op/s 2023-09-10T06:23:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:23:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:23:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:45 smithi170 ceph-mon[102832]: pgmap v13736: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 251 KiB/s wr, 12 op/s 2023-09-10T06:23:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:23:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:23:46.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:45 smithi082 ceph-mon[98694]: pgmap v13736: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 251 KiB/s wr, 12 op/s 2023-09-10T06:23:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:23:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:23:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:47 smithi098 ceph-mon[103124]: pgmap v13737: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 556 KiB/s wr, 18 op/s 2023-09-10T06:23:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:47 smithi170 ceph-mon[102832]: pgmap v13737: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 556 KiB/s wr, 18 op/s 2023-09-10T06:23:48.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:47 smithi082 ceph-mon[98694]: pgmap v13737: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 556 KiB/s wr, 18 op/s 2023-09-10T06:23:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:49 smithi098 ceph-mon[103124]: pgmap v13738: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 367 KiB/s wr, 12 op/s 2023-09-10T06:23:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25460.466170 secs 2023-09-10T06:23:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25461.079120 secs 2023-09-10T06:23:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:23:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:49 smithi170 ceph-mon[102832]: pgmap v13738: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 367 KiB/s wr, 12 op/s 2023-09-10T06:23:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25460.466170 secs 2023-09-10T06:23:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25461.079120 secs 2023-09-10T06:23:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:23:50.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:49 smithi082 ceph-mon[98694]: pgmap v13738: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 367 KiB/s wr, 12 op/s 2023-09-10T06:23:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25460.466170 secs 2023-09-10T06:23:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25461.079120 secs 2023-09-10T06:23:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:23:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:51 smithi098 ceph-mon[103124]: pgmap v13739: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 367 KiB/s wr, 12 op/s 2023-09-10T06:23:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:51 smithi170 ceph-mon[102832]: pgmap v13739: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 367 KiB/s wr, 12 op/s 2023-09-10T06:23:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:51 smithi082 ceph-mon[98694]: pgmap v13739: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 367 KiB/s wr, 12 op/s 2023-09-10T06:23:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:53 smithi170 ceph-mon[102832]: pgmap v13740: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 490 KiB/s wr, 18 op/s 2023-09-10T06:23:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:23:53.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:23:53.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:23:53.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:23:53.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:23:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:53 smithi082 ceph-mon[98694]: pgmap v13740: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 490 KiB/s wr, 18 op/s 2023-09-10T06:23:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:23:53.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:23:53.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:23:53.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:23:53.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:23:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:53 smithi098 ceph-mon[103124]: pgmap v13740: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 490 KiB/s wr, 18 op/s 2023-09-10T06:23:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:23:53.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:53.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:23:53.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:23:53.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:23:53.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:23:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25465.466230 secs 2023-09-10T06:23:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:54 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T06:23:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:54 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T06:23:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:23:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:23:54.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:23:54.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:23:54.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:54 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T06:23:54.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:54 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T06:23:54.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:23:54.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:23:54.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:23:54.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:23:54.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:54.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25466.079187 secs 2023-09-10T06:23:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25465.466230 secs 2023-09-10T06:23:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:54 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T06:23:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:54 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T06:23:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:23:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:23:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:23:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:23:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:54 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T06:23:54.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:54 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T06:23:54.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:23:54.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:23:54.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:23:54.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:23:54.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:54.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25466.079187 secs 2023-09-10T06:23:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25465.466230 secs 2023-09-10T06:23:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:54 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T06:23:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:54 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T06:23:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:23:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:23:55.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:23:55.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:23:55.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:54 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T06:23:55.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:54 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T06:23:55.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:23:55.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:23:55.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:23:55.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:23:55.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:23:55.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25466.079187 secs 2023-09-10T06:23:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:55 smithi170 ceph-mon[102832]: pgmap v13741: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 428 KiB/s wr, 12 op/s 2023-09-10T06:23:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:55 smithi082 ceph-mon[98694]: pgmap v13741: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 428 KiB/s wr, 12 op/s 2023-09-10T06:23:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:55 smithi098 ceph-mon[103124]: pgmap v13741: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 428 KiB/s wr, 12 op/s 2023-09-10T06:23:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:23:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:23:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:23:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:23:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:23:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:23:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:57 smithi098 ceph-mon[103124]: pgmap v13742: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 817 KiB/s wr, 19 op/s 2023-09-10T06:23:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:57 smithi170 ceph-mon[102832]: pgmap v13742: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 817 KiB/s wr, 19 op/s 2023-09-10T06:23:58.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:57 smithi082 ceph-mon[98694]: pgmap v13742: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 817 KiB/s wr, 19 op/s 2023-09-10T06:23:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25469.458573 secs 2023-09-10T06:23:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25469.458573 secs 2023-09-10T06:23:59.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25469.458573 secs 2023-09-10T06:24:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:59 smithi098 ceph-mon[103124]: pgmap v13743: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 512 KiB/s wr, 12 op/s 2023-09-10T06:24:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25470.466288 secs 2023-09-10T06:24:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:23:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25471.079255 secs 2023-09-10T06:24:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:59 smithi170 ceph-mon[102832]: pgmap v13743: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 512 KiB/s wr, 12 op/s 2023-09-10T06:24:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25470.466288 secs 2023-09-10T06:24:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:23:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25471.079255 secs 2023-09-10T06:24:00.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:59 smithi082 ceph-mon[98694]: pgmap v13743: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 512 KiB/s wr, 12 op/s 2023-09-10T06:24:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25470.466288 secs 2023-09-10T06:24:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:23:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25471.079255 secs 2023-09-10T06:24:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:24:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:24:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:01.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:24:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:24:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:24:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:24:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:01 smithi170 ceph-mon[102832]: pgmap v13744: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 512 KiB/s wr, 12 op/s 2023-09-10T06:24:02.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:01 smithi082 ceph-mon[98694]: pgmap v13744: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 512 KiB/s wr, 12 op/s 2023-09-10T06:24:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:01 smithi098 ceph-mon[103124]: pgmap v13744: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 512 KiB/s wr, 12 op/s 2023-09-10T06:24:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:03 smithi170 ceph-mon[102832]: pgmap v13745: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 607 KiB/s wr, 18 op/s 2023-09-10T06:24:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25474.458669 secs 2023-09-10T06:24:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:03 smithi082 ceph-mon[98694]: pgmap v13745: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 607 KiB/s wr, 18 op/s 2023-09-10T06:24:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25474.458669 secs 2023-09-10T06:24:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:03 smithi098 ceph-mon[103124]: pgmap v13745: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 607 KiB/s wr, 18 op/s 2023-09-10T06:24:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25474.458669 secs 2023-09-10T06:24:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25475.466348 secs 2023-09-10T06:24:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25476.079319 secs 2023-09-10T06:24:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25475.466348 secs 2023-09-10T06:24:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25476.079319 secs 2023-09-10T06:24:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25475.466348 secs 2023-09-10T06:24:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25476.079319 secs 2023-09-10T06:24:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:05 smithi170 ceph-mon[102832]: pgmap v13746: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 484 KiB/s wr, 12 op/s 2023-09-10T06:24:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:05 smithi082 ceph-mon[98694]: pgmap v13746: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 484 KiB/s wr, 12 op/s 2023-09-10T06:24:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:05 smithi098 ceph-mon[103124]: pgmap v13746: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 484 KiB/s wr, 12 op/s 2023-09-10T06:24:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:07 smithi170 ceph-mon[102832]: pgmap v13747: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 655 KiB/s wr, 32 op/s 2023-09-10T06:24:08.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:07 smithi082 ceph-mon[98694]: pgmap v13747: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 655 KiB/s wr, 32 op/s 2023-09-10T06:24:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:07 smithi098 ceph-mon[103124]: pgmap v13747: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 655 KiB/s wr, 32 op/s 2023-09-10T06:24:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25479.458732 secs 2023-09-10T06:24:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:08 smithi170 ceph-mon[102832]: osdmap e927: 12 total, 12 up, 12 in 2023-09-10T06:24:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25479.458732 secs 2023-09-10T06:24:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:08 smithi082 ceph-mon[98694]: osdmap e927: 12 total, 12 up, 12 in 2023-09-10T06:24:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25479.458732 secs 2023-09-10T06:24:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:08 smithi098 ceph-mon[103124]: osdmap e927: 12 total, 12 up, 12 in 2023-09-10T06:24:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:09 smithi170 ceph-mon[102832]: pgmap v13749: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 319 KiB/s wr, 31 op/s 2023-09-10T06:24:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25480.466407 secs 2023-09-10T06:24:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25481.079383 secs 2023-09-10T06:24:10.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:09 smithi082 ceph-mon[98694]: pgmap v13749: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 319 KiB/s wr, 31 op/s 2023-09-10T06:24:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25480.466407 secs 2023-09-10T06:24:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25481.079383 secs 2023-09-10T06:24:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:09 smithi098 ceph-mon[103124]: pgmap v13749: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 319 KiB/s wr, 31 op/s 2023-09-10T06:24:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25480.466407 secs 2023-09-10T06:24:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25481.079383 secs 2023-09-10T06:24:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:11 smithi098 ceph-mon[103124]: pgmap v13750: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 319 KiB/s wr, 31 op/s 2023-09-10T06:24:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:11 smithi170 ceph-mon[102832]: pgmap v13750: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 319 KiB/s wr, 31 op/s 2023-09-10T06:24:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:11 smithi082 ceph-mon[98694]: pgmap v13750: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 319 KiB/s wr, 31 op/s 2023-09-10T06:24:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:13 smithi098 ceph-mon[103124]: pgmap v13751: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 286 KiB/s wr, 29 op/s 2023-09-10T06:24:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25484.458778 secs 2023-09-10T06:24:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:13 smithi170 ceph-mon[102832]: pgmap v13751: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 286 KiB/s wr, 29 op/s 2023-09-10T06:24:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25484.458778 secs 2023-09-10T06:24:14.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:13 smithi082 ceph-mon[98694]: pgmap v13751: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 286 KiB/s wr, 29 op/s 2023-09-10T06:24:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25484.458778 secs 2023-09-10T06:24:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25485.466467 secs 2023-09-10T06:24:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25486.079446 secs 2023-09-10T06:24:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25485.466467 secs 2023-09-10T06:24:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25486.079446 secs 2023-09-10T06:24:15.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25485.466467 secs 2023-09-10T06:24:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25486.079446 secs 2023-09-10T06:24:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:15 smithi170 ceph-mon[102832]: pgmap v13752: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 286 KiB/s wr, 29 op/s 2023-09-10T06:24:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:15 smithi170 ceph-mon[102832]: osdmap e928: 12 total, 12 up, 12 in 2023-09-10T06:24:16.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:15 smithi082 ceph-mon[98694]: pgmap v13752: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 286 KiB/s wr, 29 op/s 2023-09-10T06:24:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:15 smithi082 ceph-mon[98694]: osdmap e928: 12 total, 12 up, 12 in 2023-09-10T06:24:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:15 smithi098 ceph-mon[103124]: pgmap v13752: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 286 KiB/s wr, 29 op/s 2023-09-10T06:24:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:15 smithi098 ceph-mon[103124]: osdmap e928: 12 total, 12 up, 12 in 2023-09-10T06:24:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:24:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:17.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:24:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:24:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:17 smithi170 ceph-mon[102832]: pgmap v13754: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 102 KiB/s wr, 14 op/s 2023-09-10T06:24:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:17 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:24:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:17 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:17 smithi082 ceph-mon[98694]: pgmap v13754: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 102 KiB/s wr, 14 op/s 2023-09-10T06:24:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:17 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:24:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:17 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:17 smithi098 ceph-mon[103124]: pgmap v13754: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 102 KiB/s wr, 14 op/s 2023-09-10T06:24:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:17 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:24:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:17 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25489.458838 secs 2023-09-10T06:24:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25489.458838 secs 2023-09-10T06:24:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25489.458838 secs 2023-09-10T06:24:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:19 smithi170 ceph-mon[102832]: pgmap v13755: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T06:24:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25490.466495 secs 2023-09-10T06:24:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25491.079487 secs 2023-09-10T06:24:20.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:19 smithi082 ceph-mon[98694]: pgmap v13755: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T06:24:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25490.466495 secs 2023-09-10T06:24:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25491.079487 secs 2023-09-10T06:24:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:19 smithi098 ceph-mon[103124]: pgmap v13755: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T06:24:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25490.466495 secs 2023-09-10T06:24:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25491.079487 secs 2023-09-10T06:24:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:21 smithi170 ceph-mon[102832]: pgmap v13756: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T06:24:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:21 smithi082 ceph-mon[98694]: pgmap v13756: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T06:24:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:21 smithi098 ceph-mon[103124]: pgmap v13756: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 11 op/s 2023-09-10T06:24:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:23 smithi170 ceph-mon[102832]: pgmap v13757: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-09-10T06:24:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25494.458900 secs 2023-09-10T06:24:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:23 smithi082 ceph-mon[98694]: pgmap v13757: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-09-10T06:24:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25494.458900 secs 2023-09-10T06:24:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:23 smithi098 ceph-mon[103124]: pgmap v13757: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-09-10T06:24:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25494.458900 secs 2023-09-10T06:24:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25495.466566 secs 2023-09-10T06:24:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25496.079552 secs 2023-09-10T06:24:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25495.466566 secs 2023-09-10T06:24:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25496.079552 secs 2023-09-10T06:24:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25495.466566 secs 2023-09-10T06:24:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25496.079552 secs 2023-09-10T06:24:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:25 smithi170 ceph-mon[102832]: pgmap v13758: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-09-10T06:24:26.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:25 smithi082 ceph-mon[98694]: pgmap v13758: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-09-10T06:24:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:25 smithi098 ceph-mon[103124]: pgmap v13758: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-09-10T06:24:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:27 smithi170 ceph-mon[102832]: pgmap v13759: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T06:24:28.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:27 smithi082 ceph-mon[98694]: pgmap v13759: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T06:24:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:27 smithi098 ceph-mon[103124]: pgmap v13759: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T06:24:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:29 smithi170 ceph-mon[102832]: pgmap v13760: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T06:24:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25500.466626 secs 2023-09-10T06:24:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25501.079619 secs 2023-09-10T06:24:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:29 smithi082 ceph-mon[98694]: pgmap v13760: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T06:24:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25500.466626 secs 2023-09-10T06:24:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25501.079619 secs 2023-09-10T06:24:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:29 smithi098 ceph-mon[103124]: pgmap v13760: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T06:24:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25500.466626 secs 2023-09-10T06:24:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25501.079619 secs 2023-09-10T06:24:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:31 smithi098 ceph-mon[103124]: pgmap v13761: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T06:24:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:24:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:31 smithi170 ceph-mon[102832]: pgmap v13761: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T06:24:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:24:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:31 smithi082 ceph-mon[98694]: pgmap v13761: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T06:24:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:24:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:24:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:24:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:24:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:24:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:24:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:24:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:33 smithi098 ceph-mon[103124]: pgmap v13762: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 175 KiB/s wr, 15 op/s 2023-09-10T06:24:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:24:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:33 smithi170 ceph-mon[102832]: pgmap v13762: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 175 KiB/s wr, 15 op/s 2023-09-10T06:24:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:24:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:33 smithi082 ceph-mon[98694]: pgmap v13762: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 175 KiB/s wr, 15 op/s 2023-09-10T06:24:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:24:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25505.466684 secs 2023-09-10T06:24:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25506.079686 secs 2023-09-10T06:24:35.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25505.466684 secs 2023-09-10T06:24:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25506.079686 secs 2023-09-10T06:24:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25505.466684 secs 2023-09-10T06:24:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25506.079686 secs 2023-09-10T06:24:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:35 smithi170 ceph-mon[102832]: pgmap v13763: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T06:24:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:35 smithi082 ceph-mon[98694]: pgmap v13763: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T06:24:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:35 smithi098 ceph-mon[103124]: pgmap v13763: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T06:24:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:24:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:37.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:24:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:24:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:37 smithi170 ceph-mon[102832]: pgmap v13764: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 638 KiB/s wr, 17 op/s 2023-09-10T06:24:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:37 smithi082 ceph-mon[98694]: pgmap v13764: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 638 KiB/s wr, 17 op/s 2023-09-10T06:24:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:37 smithi098 ceph-mon[103124]: pgmap v13764: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 638 KiB/s wr, 17 op/s 2023-09-10T06:24:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:39 smithi170 ceph-mon[102832]: pgmap v13765: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 549 KiB/s wr, 12 op/s 2023-09-10T06:24:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25510.466745 secs 2023-09-10T06:24:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25511.079752 secs 2023-09-10T06:24:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:39 smithi082 ceph-mon[98694]: pgmap v13765: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 549 KiB/s wr, 12 op/s 2023-09-10T06:24:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25510.466745 secs 2023-09-10T06:24:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25511.079752 secs 2023-09-10T06:24:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:39 smithi098 ceph-mon[103124]: pgmap v13765: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 549 KiB/s wr, 12 op/s 2023-09-10T06:24:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25510.466745 secs 2023-09-10T06:24:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25511.079752 secs 2023-09-10T06:24:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:41 smithi170 ceph-mon[102832]: pgmap v13766: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 550 KiB/s wr, 12 op/s 2023-09-10T06:24:42.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:41 smithi082 ceph-mon[98694]: pgmap v13766: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 550 KiB/s wr, 12 op/s 2023-09-10T06:24:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:41 smithi098 ceph-mon[103124]: pgmap v13766: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 550 KiB/s wr, 12 op/s 2023-09-10T06:24:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:43 smithi170 ceph-mon[102832]: pgmap v13767: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 634 KiB/s wr, 18 op/s 2023-09-10T06:24:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:43 smithi082 ceph-mon[98694]: pgmap v13767: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 634 KiB/s wr, 18 op/s 2023-09-10T06:24:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:43 smithi098 ceph-mon[103124]: pgmap v13767: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 634 KiB/s wr, 18 op/s 2023-09-10T06:24:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25515.466807 secs 2023-09-10T06:24:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:24:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25516.079816 secs 2023-09-10T06:24:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25515.466807 secs 2023-09-10T06:24:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:24:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25516.079816 secs 2023-09-10T06:24:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25515.466807 secs 2023-09-10T06:24:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:24:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25516.079816 secs 2023-09-10T06:24:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:45 smithi170 ceph-mon[102832]: pgmap v13768: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 548 KiB/s wr, 13 op/s 2023-09-10T06:24:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:24:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:45 smithi082 ceph-mon[98694]: pgmap v13768: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 548 KiB/s wr, 13 op/s 2023-09-10T06:24:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:24:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:45 smithi098 ceph-mon[103124]: pgmap v13768: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 548 KiB/s wr, 13 op/s 2023-09-10T06:24:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:24:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:46 smithi170 ceph-mon[102832]: pgmap v13769: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 792 KiB/s wr, 20 op/s 2023-09-10T06:24:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:46 smithi082 ceph-mon[98694]: pgmap v13769: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 792 KiB/s wr, 20 op/s 2023-09-10T06:24:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:46 smithi098 ceph-mon[103124]: pgmap v13769: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 792 KiB/s wr, 20 op/s 2023-09-10T06:24:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:48 smithi170 ceph-mon[102832]: pgmap v13770: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 329 KiB/s wr, 12 op/s 2023-09-10T06:24:49.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:48 smithi082 ceph-mon[98694]: pgmap v13770: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 329 KiB/s wr, 12 op/s 2023-09-10T06:24:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:48 smithi098 ceph-mon[103124]: pgmap v13770: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 329 KiB/s wr, 12 op/s 2023-09-10T06:24:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25520.466865 secs 2023-09-10T06:24:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25521.079884 secs 2023-09-10T06:24:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:24:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25520.466865 secs 2023-09-10T06:24:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25521.079884 secs 2023-09-10T06:24:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:24:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25520.466865 secs 2023-09-10T06:24:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25521.079884 secs 2023-09-10T06:24:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:24:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:51 smithi098 ceph-mon[103124]: pgmap v13771: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 329 KiB/s wr, 12 op/s 2023-09-10T06:24:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:51 smithi170 ceph-mon[102832]: pgmap v13771: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 329 KiB/s wr, 12 op/s 2023-09-10T06:24:52.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:51 smithi082 ceph-mon[98694]: pgmap v13771: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 329 KiB/s wr, 12 op/s 2023-09-10T06:24:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:24:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:24:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:24:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:53 smithi170 ceph-mon[102832]: pgmap v13772: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 638 KiB/s wr, 19 op/s 2023-09-10T06:24:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25524.459449 secs 2023-09-10T06:24:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:24:54.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:53 smithi082 ceph-mon[98694]: pgmap v13772: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 638 KiB/s wr, 19 op/s 2023-09-10T06:24:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25524.459449 secs 2023-09-10T06:24:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:24:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:53 smithi098 ceph-mon[103124]: pgmap v13772: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 638 KiB/s wr, 19 op/s 2023-09-10T06:24:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25524.459449 secs 2023-09-10T06:24:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:24:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25525.466924 secs 2023-09-10T06:24:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25526.079952 secs 2023-09-10T06:24:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:24:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:24:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:24:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25525.466924 secs 2023-09-10T06:24:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25526.079952 secs 2023-09-10T06:24:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:24:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:24:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:24:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25525.466924 secs 2023-09-10T06:24:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25526.079952 secs 2023-09-10T06:24:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:24:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:24:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:24:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:55 smithi170 ceph-mon[102832]: pgmap v13773: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 553 KiB/s wr, 13 op/s 2023-09-10T06:24:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:55 smithi082 ceph-mon[98694]: pgmap v13773: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 553 KiB/s wr, 13 op/s 2023-09-10T06:24:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:55 smithi098 ceph-mon[103124]: pgmap v13773: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 553 KiB/s wr, 13 op/s 2023-09-10T06:24:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:24:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:24:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:24:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:24:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:24:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:24:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:57 smithi170 ceph-mon[102832]: pgmap v13774: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 701 KiB/s wr, 19 op/s 2023-09-10T06:24:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:24:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:58.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:57 smithi082 ceph-mon[98694]: pgmap v13774: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 701 KiB/s wr, 19 op/s 2023-09-10T06:24:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:24:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:57 smithi098 ceph-mon[103124]: pgmap v13774: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 701 KiB/s wr, 19 op/s 2023-09-10T06:24:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:24:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:24:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25529.459511 secs 2023-09-10T06:24:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25529.459511 secs 2023-09-10T06:24:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25529.459511 secs 2023-09-10T06:25:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:59 smithi170 ceph-mon[102832]: pgmap v13775: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 457 KiB/s wr, 12 op/s 2023-09-10T06:25:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25530.466985 secs 2023-09-10T06:25:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:24:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25531.080016 secs 2023-09-10T06:25:00.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:59 smithi082 ceph-mon[98694]: pgmap v13775: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 457 KiB/s wr, 12 op/s 2023-09-10T06:25:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25530.466985 secs 2023-09-10T06:25:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:24:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25531.080016 secs 2023-09-10T06:25:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:59 smithi098 ceph-mon[103124]: pgmap v13775: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 457 KiB/s wr, 12 op/s 2023-09-10T06:25:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25530.466985 secs 2023-09-10T06:25:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:24:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25531.080016 secs 2023-09-10T06:25:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:25:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:25:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:00 smithi170 ceph-mon[102832]: pgmap v13776: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 457 KiB/s wr, 12 op/s 2023-09-10T06:25:01.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:25:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:25:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:00 smithi082 ceph-mon[98694]: pgmap v13776: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 457 KiB/s wr, 12 op/s 2023-09-10T06:25:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:25:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:25:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:00 smithi098 ceph-mon[103124]: pgmap v13776: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 457 KiB/s wr, 12 op/s 2023-09-10T06:25:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:02 smithi082 ceph-mon[98694]: pgmap v13777: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 645 KiB/s wr, 18 op/s 2023-09-10T06:25:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:02 smithi098 ceph-mon[103124]: pgmap v13777: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 645 KiB/s wr, 18 op/s 2023-09-10T06:25:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:02 smithi170 ceph-mon[102832]: pgmap v13777: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 645 KiB/s wr, 18 op/s 2023-09-10T06:25:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25534.459607 secs 2023-09-10T06:25:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25534.459607 secs 2023-09-10T06:25:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25534.459607 secs 2023-09-10T06:25:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25535.467044 secs 2023-09-10T06:25:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25536.080080 secs 2023-09-10T06:25:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:04 smithi082 ceph-mon[98694]: pgmap v13778: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 336 KiB/s wr, 11 op/s 2023-09-10T06:25:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25535.467044 secs 2023-09-10T06:25:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25536.080080 secs 2023-09-10T06:25:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:04 smithi098 ceph-mon[103124]: pgmap v13778: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 336 KiB/s wr, 11 op/s 2023-09-10T06:25:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25535.467044 secs 2023-09-10T06:25:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25536.080080 secs 2023-09-10T06:25:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:04 smithi170 ceph-mon[102832]: pgmap v13778: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 336 KiB/s wr, 11 op/s 2023-09-10T06:25:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:07 smithi170 ceph-mon[102832]: pgmap v13779: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 468 KiB/s wr, 31 op/s 2023-09-10T06:25:08.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:07 smithi082 ceph-mon[98694]: pgmap v13779: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 468 KiB/s wr, 31 op/s 2023-09-10T06:25:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:07 smithi098 ceph-mon[103124]: pgmap v13779: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 468 KiB/s wr, 31 op/s 2023-09-10T06:25:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25539.459667 secs 2023-09-10T06:25:09.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25539.459667 secs 2023-09-10T06:25:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25539.459667 secs 2023-09-10T06:25:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:09 smithi170 ceph-mon[102832]: pgmap v13780: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 320 KiB/s wr, 25 op/s 2023-09-10T06:25:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25540.467105 secs 2023-09-10T06:25:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:09 smithi170 ceph-mon[102832]: osdmap e929: 12 total, 12 up, 12 in 2023-09-10T06:25:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25541.080145 secs 2023-09-10T06:25:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:09 smithi082 ceph-mon[98694]: pgmap v13780: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 320 KiB/s wr, 25 op/s 2023-09-10T06:25:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25540.467105 secs 2023-09-10T06:25:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:09 smithi082 ceph-mon[98694]: osdmap e929: 12 total, 12 up, 12 in 2023-09-10T06:25:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25541.080145 secs 2023-09-10T06:25:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:09 smithi098 ceph-mon[103124]: pgmap v13780: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 320 KiB/s wr, 25 op/s 2023-09-10T06:25:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25540.467105 secs 2023-09-10T06:25:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:09 smithi098 ceph-mon[103124]: osdmap e929: 12 total, 12 up, 12 in 2023-09-10T06:25:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25541.080145 secs 2023-09-10T06:25:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:11 smithi170 ceph-mon[102832]: pgmap v13782: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 384 KiB/s wr, 30 op/s 2023-09-10T06:25:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:11 smithi082 ceph-mon[98694]: pgmap v13782: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 384 KiB/s wr, 30 op/s 2023-09-10T06:25:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:11 smithi098 ceph-mon[103124]: pgmap v13782: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 384 KiB/s wr, 30 op/s 2023-09-10T06:25:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:13 smithi170 ceph-mon[102832]: pgmap v13783: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 182 KiB/s wr, 30 op/s 2023-09-10T06:25:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25544.459724 secs 2023-09-10T06:25:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:13 smithi082 ceph-mon[98694]: pgmap v13783: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 182 KiB/s wr, 30 op/s 2023-09-10T06:25:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25544.459724 secs 2023-09-10T06:25:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:13 smithi098 ceph-mon[103124]: pgmap v13783: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 182 KiB/s wr, 30 op/s 2023-09-10T06:25:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25544.459724 secs 2023-09-10T06:25:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25545.467163 secs 2023-09-10T06:25:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25546.080211 secs 2023-09-10T06:25:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25545.467163 secs 2023-09-10T06:25:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25546.080211 secs 2023-09-10T06:25:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25545.467163 secs 2023-09-10T06:25:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25546.080211 secs 2023-09-10T06:25:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:15 smithi170 ceph-mon[102832]: pgmap v13784: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 182 KiB/s wr, 30 op/s 2023-09-10T06:25:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:15 smithi170 ceph-mon[102832]: osdmap e930: 12 total, 12 up, 12 in 2023-09-10T06:25:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:15 smithi082 ceph-mon[98694]: pgmap v13784: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 182 KiB/s wr, 30 op/s 2023-09-10T06:25:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:15 smithi082 ceph-mon[98694]: osdmap e930: 12 total, 12 up, 12 in 2023-09-10T06:25:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:15 smithi098 ceph-mon[103124]: pgmap v13784: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 182 KiB/s wr, 30 op/s 2023-09-10T06:25:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:15 smithi098 ceph-mon[103124]: osdmap e930: 12 total, 12 up, 12 in 2023-09-10T06:25:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:17 smithi170 ceph-mon[102832]: pgmap v13786: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 15 op/s 2023-09-10T06:25:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:17 smithi082 ceph-mon[98694]: pgmap v13786: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 15 op/s 2023-09-10T06:25:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:17 smithi098 ceph-mon[103124]: pgmap v13786: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 15 op/s 2023-09-10T06:25:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25549.459784 secs 2023-09-10T06:25:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25549.459784 secs 2023-09-10T06:25:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25549.459784 secs 2023-09-10T06:25:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:19 smithi170 ceph-mon[102832]: pgmap v13787: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 13 op/s 2023-09-10T06:25:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25550.467222 secs 2023-09-10T06:25:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25551.080277 secs 2023-09-10T06:25:20.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:19 smithi082 ceph-mon[98694]: pgmap v13787: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 13 op/s 2023-09-10T06:25:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25550.467222 secs 2023-09-10T06:25:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25551.080277 secs 2023-09-10T06:25:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:19 smithi098 ceph-mon[103124]: pgmap v13787: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 13 op/s 2023-09-10T06:25:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25550.467222 secs 2023-09-10T06:25:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25551.080277 secs 2023-09-10T06:25:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:21 smithi170 ceph-mon[102832]: pgmap v13788: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 12 op/s 2023-09-10T06:25:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:21 smithi082 ceph-mon[98694]: pgmap v13788: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 12 op/s 2023-09-10T06:25:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:21 smithi098 ceph-mon[103124]: pgmap v13788: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 12 op/s 2023-09-10T06:25:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:23 smithi170 ceph-mon[102832]: pgmap v13789: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T06:25:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25554.459892 secs 2023-09-10T06:25:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:23 smithi082 ceph-mon[98694]: pgmap v13789: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T06:25:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25554.459892 secs 2023-09-10T06:25:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:23 smithi098 ceph-mon[103124]: pgmap v13789: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T06:25:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25554.459892 secs 2023-09-10T06:25:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25555.467299 secs 2023-09-10T06:25:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25556.080343 secs 2023-09-10T06:25:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25555.467299 secs 2023-09-10T06:25:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25556.080343 secs 2023-09-10T06:25:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25555.467299 secs 2023-09-10T06:25:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25556.080343 secs 2023-09-10T06:25:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:25 smithi170 ceph-mon[102832]: pgmap v13790: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T06:25:26.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:25 smithi082 ceph-mon[98694]: pgmap v13790: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T06:25:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:25 smithi098 ceph-mon[103124]: pgmap v13790: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T06:25:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:27 smithi170 ceph-mon[102832]: pgmap v13791: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T06:25:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:27 smithi082 ceph-mon[98694]: pgmap v13791: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T06:25:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:27 smithi098 ceph-mon[103124]: pgmap v13791: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T06:25:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:29 smithi170 ceph-mon[102832]: pgmap v13792: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 9 op/s 2023-09-10T06:25:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25560.467358 secs 2023-09-10T06:25:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:29 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:25:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:29 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:25:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25561.080405 secs 2023-09-10T06:25:30.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:29 smithi082 ceph-mon[98694]: pgmap v13792: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 9 op/s 2023-09-10T06:25:30.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25560.467358 secs 2023-09-10T06:25:30.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:29 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:25:30.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:29 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:25:30.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25561.080405 secs 2023-09-10T06:25:30.259 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:29 smithi098 ceph-mon[103124]: pgmap v13792: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 9 op/s 2023-09-10T06:25:30.260 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25560.467358 secs 2023-09-10T06:25:30.260 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:29 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:25:30.260 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:29 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:25:30.260 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25561.080405 secs 2023-09-10T06:25:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:31 smithi170 ceph-mon[102832]: pgmap v13793: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 9 op/s 2023-09-10T06:25:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:25:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:31 smithi082 ceph-mon[98694]: pgmap v13793: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 9 op/s 2023-09-10T06:25:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:25:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:31 smithi098 ceph-mon[103124]: pgmap v13793: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 9 op/s 2023-09-10T06:25:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:25:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:25:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:25:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:25:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:25:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:25:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:25:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:25:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:25:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:25:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:33 smithi170 ceph-mon[102832]: pgmap v13794: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 199 KiB/s wr, 15 op/s 2023-09-10T06:25:34.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:33 smithi082 ceph-mon[98694]: pgmap v13794: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 199 KiB/s wr, 15 op/s 2023-09-10T06:25:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:33 smithi098 ceph-mon[103124]: pgmap v13794: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 199 KiB/s wr, 15 op/s 2023-09-10T06:25:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25565.467419 secs 2023-09-10T06:25:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25566.080474 secs 2023-09-10T06:25:35.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25565.467419 secs 2023-09-10T06:25:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25566.080474 secs 2023-09-10T06:25:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25565.467419 secs 2023-09-10T06:25:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25566.080474 secs 2023-09-10T06:25:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:35 smithi170 ceph-mon[102832]: pgmap v13795: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 10 op/s 2023-09-10T06:25:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:35 smithi082 ceph-mon[98694]: pgmap v13795: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 10 op/s 2023-09-10T06:25:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:35 smithi098 ceph-mon[103124]: pgmap v13795: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 10 op/s 2023-09-10T06:25:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:37 smithi170 ceph-mon[102832]: pgmap v13796: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 297 KiB/s wr, 15 op/s 2023-09-10T06:25:38.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:37 smithi082 ceph-mon[98694]: pgmap v13796: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 297 KiB/s wr, 15 op/s 2023-09-10T06:25:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:37 smithi098 ceph-mon[103124]: pgmap v13796: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 297 KiB/s wr, 15 op/s 2023-09-10T06:25:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:39 smithi170 ceph-mon[102832]: pgmap v13797: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 297 KiB/s wr, 11 op/s 2023-09-10T06:25:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25570.467503 secs 2023-09-10T06:25:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25571.080538 secs 2023-09-10T06:25:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:39 smithi082 ceph-mon[98694]: pgmap v13797: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 297 KiB/s wr, 11 op/s 2023-09-10T06:25:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25570.467503 secs 2023-09-10T06:25:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25571.080538 secs 2023-09-10T06:25:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:39 smithi098 ceph-mon[103124]: pgmap v13797: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 297 KiB/s wr, 11 op/s 2023-09-10T06:25:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25570.467503 secs 2023-09-10T06:25:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25571.080538 secs 2023-09-10T06:25:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:41 smithi170 ceph-mon[102832]: pgmap v13798: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 297 KiB/s wr, 11 op/s 2023-09-10T06:25:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:41 smithi082 ceph-mon[98694]: pgmap v13798: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 297 KiB/s wr, 11 op/s 2023-09-10T06:25:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:41 smithi098 ceph-mon[103124]: pgmap v13798: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 297 KiB/s wr, 11 op/s 2023-09-10T06:25:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:43 smithi170 ceph-mon[102832]: pgmap v13799: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 431 KiB/s wr, 18 op/s 2023-09-10T06:25:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:43 smithi082 ceph-mon[98694]: pgmap v13799: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 431 KiB/s wr, 18 op/s 2023-09-10T06:25:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:43 smithi098 ceph-mon[103124]: pgmap v13799: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 431 KiB/s wr, 18 op/s 2023-09-10T06:25:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25575.467559 secs 2023-09-10T06:25:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:25:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:25:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25576.080600 secs 2023-09-10T06:25:45.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25575.467559 secs 2023-09-10T06:25:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:25:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:25:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25576.080600 secs 2023-09-10T06:25:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25575.467559 secs 2023-09-10T06:25:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:25:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:25:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25576.080600 secs 2023-09-10T06:25:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:45 smithi170 ceph-mon[102832]: pgmap v13800: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 321 KiB/s wr, 12 op/s 2023-09-10T06:25:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:25:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:25:46.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:45 smithi082 ceph-mon[98694]: pgmap v13800: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 321 KiB/s wr, 12 op/s 2023-09-10T06:25:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:25:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:25:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:45 smithi098 ceph-mon[103124]: pgmap v13800: 97 pgs: 97 active+clean; 12 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 321 KiB/s wr, 12 op/s 2023-09-10T06:25:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:25:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:25:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:47 smithi170 ceph-mon[102832]: pgmap v13801: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 571 KiB/s wr, 18 op/s 2023-09-10T06:25:48.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:47 smithi082 ceph-mon[98694]: pgmap v13801: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 571 KiB/s wr, 18 op/s 2023-09-10T06:25:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:47 smithi098 ceph-mon[103124]: pgmap v13801: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 571 KiB/s wr, 18 op/s 2023-09-10T06:25:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:49 smithi170 ceph-mon[102832]: pgmap v13802: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 384 KiB/s wr, 13 op/s 2023-09-10T06:25:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25580.467616 secs 2023-09-10T06:25:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25581.080664 secs 2023-09-10T06:25:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:49 smithi082 ceph-mon[98694]: pgmap v13802: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 384 KiB/s wr, 13 op/s 2023-09-10T06:25:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25580.467616 secs 2023-09-10T06:25:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25581.080664 secs 2023-09-10T06:25:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:49 smithi098 ceph-mon[103124]: pgmap v13802: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 384 KiB/s wr, 13 op/s 2023-09-10T06:25:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25580.467616 secs 2023-09-10T06:25:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25581.080664 secs 2023-09-10T06:25:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:51 smithi170 ceph-mon[102832]: pgmap v13803: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 384 KiB/s wr, 13 op/s 2023-09-10T06:25:52.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:51 smithi082 ceph-mon[98694]: pgmap v13803: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 384 KiB/s wr, 13 op/s 2023-09-10T06:25:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:51 smithi098 ceph-mon[103124]: pgmap v13803: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 384 KiB/s wr, 13 op/s 2023-09-10T06:25:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:53 smithi170 ceph-mon[102832]: pgmap v13804: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 485 KiB/s wr, 18 op/s 2023-09-10T06:25:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:25:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:25:54.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:53 smithi082 ceph-mon[98694]: pgmap v13804: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 485 KiB/s wr, 18 op/s 2023-09-10T06:25:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:25:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:25:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:53 smithi098 ceph-mon[103124]: pgmap v13804: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 485 KiB/s wr, 18 op/s 2023-09-10T06:25:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:25:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:25:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25585.467675 secs 2023-09-10T06:25:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25586.080734 secs 2023-09-10T06:25:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:25:55.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25585.467675 secs 2023-09-10T06:25:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25586.080734 secs 2023-09-10T06:25:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:25:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25585.467675 secs 2023-09-10T06:25:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25586.080734 secs 2023-09-10T06:25:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:25:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:55 smithi170 ceph-mon[102832]: pgmap v13805: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 352 KiB/s wr, 12 op/s 2023-09-10T06:25:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:25:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:25:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:25:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:25:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:25:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:55 smithi082 ceph-mon[98694]: pgmap v13805: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 352 KiB/s wr, 12 op/s 2023-09-10T06:25:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:25:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:25:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:25:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:25:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:25:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:55 smithi098 ceph-mon[103124]: pgmap v13805: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 352 KiB/s wr, 12 op/s 2023-09-10T06:25:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:25:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:25:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:25:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:25:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:25:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:25:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:25:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:56 smithi170 ceph-mon[102832]: pgmap v13806: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 790 KiB/s wr, 19 op/s 2023-09-10T06:25:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:25:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:25:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:56 smithi082 ceph-mon[98694]: pgmap v13806: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 790 KiB/s wr, 19 op/s 2023-09-10T06:25:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:25:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:25:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:56 smithi098 ceph-mon[103124]: pgmap v13806: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 790 KiB/s wr, 19 op/s 2023-09-10T06:25:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:25:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:25:58.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:25:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:25:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:25:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:25:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25589.460313 secs 2023-09-10T06:25:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:58 smithi170 ceph-mon[102832]: pgmap v13807: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 539 KiB/s wr, 13 op/s 2023-09-10T06:25:59.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25589.460313 secs 2023-09-10T06:25:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:58 smithi082 ceph-mon[98694]: pgmap v13807: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 539 KiB/s wr, 13 op/s 2023-09-10T06:25:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25589.460313 secs 2023-09-10T06:25:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:58 smithi098 ceph-mon[103124]: pgmap v13807: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 539 KiB/s wr, 13 op/s 2023-09-10T06:26:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25590.467733 secs 2023-09-10T06:26:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:25:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25591.080793 secs 2023-09-10T06:26:00.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25590.467733 secs 2023-09-10T06:26:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:25:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25591.080793 secs 2023-09-10T06:26:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25590.467733 secs 2023-09-10T06:26:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:25:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25591.080793 secs 2023-09-10T06:26:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:26:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:00 smithi170 ceph-mon[102832]: pgmap v13808: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 539 KiB/s wr, 13 op/s 2023-09-10T06:26:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:26:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:26:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:26:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:00 smithi082 ceph-mon[98694]: pgmap v13808: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 539 KiB/s wr, 13 op/s 2023-09-10T06:26:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:26:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:26:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:26:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:00 smithi098 ceph-mon[103124]: pgmap v13808: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 539 KiB/s wr, 13 op/s 2023-09-10T06:26:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:26:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:26:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:02 smithi170 ceph-mon[102832]: pgmap v13809: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 573 KiB/s wr, 32 op/s 2023-09-10T06:26:03.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:02 smithi082 ceph-mon[98694]: pgmap v13809: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 573 KiB/s wr, 32 op/s 2023-09-10T06:26:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:02 smithi098 ceph-mon[103124]: pgmap v13809: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 573 KiB/s wr, 32 op/s 2023-09-10T06:26:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25594.460374 secs 2023-09-10T06:26:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:03 smithi170 ceph-mon[102832]: osdmap e931: 12 total, 12 up, 12 in 2023-09-10T06:26:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25594.460374 secs 2023-09-10T06:26:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:03 smithi082 ceph-mon[98694]: osdmap e931: 12 total, 12 up, 12 in 2023-09-10T06:26:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25594.460374 secs 2023-09-10T06:26:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:03 smithi098 ceph-mon[103124]: osdmap e931: 12 total, 12 up, 12 in 2023-09-10T06:26:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25595.467792 secs 2023-09-10T06:26:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25596.080855 secs 2023-09-10T06:26:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:04 smithi170 ceph-mon[102832]: pgmap v13811: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 567 KiB/s wr, 32 op/s 2023-09-10T06:26:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25595.467792 secs 2023-09-10T06:26:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25596.080855 secs 2023-09-10T06:26:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:04 smithi082 ceph-mon[98694]: pgmap v13811: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 567 KiB/s wr, 32 op/s 2023-09-10T06:26:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25595.467792 secs 2023-09-10T06:26:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25596.080855 secs 2023-09-10T06:26:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:04 smithi098 ceph-mon[103124]: pgmap v13811: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 567 KiB/s wr, 32 op/s 2023-09-10T06:26:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:07 smithi170 ceph-mon[102832]: pgmap v13812: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 300 KiB/s wr, 30 op/s 2023-09-10T06:26:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:07 smithi082 ceph-mon[98694]: pgmap v13812: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 300 KiB/s wr, 30 op/s 2023-09-10T06:26:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:07 smithi098 ceph-mon[103124]: pgmap v13812: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 300 KiB/s wr, 30 op/s 2023-09-10T06:26:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25599.460432 secs 2023-09-10T06:26:09.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25599.460432 secs 2023-09-10T06:26:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25599.460432 secs 2023-09-10T06:26:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:09 smithi170 ceph-mon[102832]: pgmap v13813: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 300 KiB/s wr, 30 op/s 2023-09-10T06:26:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25600.467891 secs 2023-09-10T06:26:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25601.080920 secs 2023-09-10T06:26:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:09 smithi082 ceph-mon[98694]: pgmap v13813: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 300 KiB/s wr, 30 op/s 2023-09-10T06:26:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25600.467891 secs 2023-09-10T06:26:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25601.080920 secs 2023-09-10T06:26:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:09 smithi098 ceph-mon[103124]: pgmap v13813: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 300 KiB/s wr, 30 op/s 2023-09-10T06:26:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25600.467891 secs 2023-09-10T06:26:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25601.080920 secs 2023-09-10T06:26:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:11 smithi170 ceph-mon[102832]: pgmap v13814: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 300 KiB/s wr, 30 op/s 2023-09-10T06:26:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:11 smithi170 ceph-mon[102832]: osdmap e932: 12 total, 12 up, 12 in 2023-09-10T06:26:12.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:11 smithi082 ceph-mon[98694]: pgmap v13814: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 300 KiB/s wr, 30 op/s 2023-09-10T06:26:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:11 smithi082 ceph-mon[98694]: osdmap e932: 12 total, 12 up, 12 in 2023-09-10T06:26:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:11 smithi098 ceph-mon[103124]: pgmap v13814: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 300 KiB/s wr, 30 op/s 2023-09-10T06:26:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:11 smithi098 ceph-mon[103124]: osdmap e932: 12 total, 12 up, 12 in 2023-09-10T06:26:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:13 smithi170 ceph-mon[102832]: pgmap v13816: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 288 KiB/s wr, 14 op/s 2023-09-10T06:26:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25604.460493 secs 2023-09-10T06:26:14.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:13 smithi082 ceph-mon[98694]: pgmap v13816: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 288 KiB/s wr, 14 op/s 2023-09-10T06:26:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25604.460493 secs 2023-09-10T06:26:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:13 smithi098 ceph-mon[103124]: pgmap v13816: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 288 KiB/s wr, 14 op/s 2023-09-10T06:26:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25604.460493 secs 2023-09-10T06:26:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25605.467954 secs 2023-09-10T06:26:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25606.080985 secs 2023-09-10T06:26:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25605.467954 secs 2023-09-10T06:26:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25606.080985 secs 2023-09-10T06:26:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25605.467954 secs 2023-09-10T06:26:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25606.080985 secs 2023-09-10T06:26:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:15 smithi170 ceph-mon[102832]: pgmap v13817: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 259 KiB/s wr, 13 op/s 2023-09-10T06:26:16.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:15 smithi082 ceph-mon[98694]: pgmap v13817: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 259 KiB/s wr, 13 op/s 2023-09-10T06:26:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:15 smithi098 ceph-mon[103124]: pgmap v13817: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 259 KiB/s wr, 13 op/s 2023-09-10T06:26:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:17 smithi170 ceph-mon[102832]: pgmap v13818: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T06:26:18.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:17 smithi082 ceph-mon[98694]: pgmap v13818: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T06:26:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:17 smithi098 ceph-mon[103124]: pgmap v13818: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T06:26:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25609.460554 secs 2023-09-10T06:26:19.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25609.460554 secs 2023-09-10T06:26:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25609.460554 secs 2023-09-10T06:26:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:19 smithi170 ceph-mon[102832]: pgmap v13819: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T06:26:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25610.468051 secs 2023-09-10T06:26:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25611.081052 secs 2023-09-10T06:26:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:19 smithi082 ceph-mon[98694]: pgmap v13819: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T06:26:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25610.468051 secs 2023-09-10T06:26:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25611.081052 secs 2023-09-10T06:26:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:19 smithi098 ceph-mon[103124]: pgmap v13819: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T06:26:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25610.468051 secs 2023-09-10T06:26:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25611.081052 secs 2023-09-10T06:26:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:21 smithi170 ceph-mon[102832]: pgmap v13820: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T06:26:22.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:21 smithi082 ceph-mon[98694]: pgmap v13820: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T06:26:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:21 smithi098 ceph-mon[103124]: pgmap v13820: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T06:26:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:23 smithi170 ceph-mon[102832]: pgmap v13821: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T06:26:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25614.460614 secs 2023-09-10T06:26:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:23 smithi082 ceph-mon[98694]: pgmap v13821: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T06:26:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25614.460614 secs 2023-09-10T06:26:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:23 smithi098 ceph-mon[103124]: pgmap v13821: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T06:26:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25614.460614 secs 2023-09-10T06:26:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25615.468108 secs 2023-09-10T06:26:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25616.081118 secs 2023-09-10T06:26:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25615.468108 secs 2023-09-10T06:26:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25616.081118 secs 2023-09-10T06:26:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25615.468108 secs 2023-09-10T06:26:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25616.081118 secs 2023-09-10T06:26:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:25 smithi170 ceph-mon[102832]: pgmap v13822: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T06:26:26.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:25 smithi082 ceph-mon[98694]: pgmap v13822: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T06:26:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:25 smithi098 ceph-mon[103124]: pgmap v13822: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T06:26:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:26 smithi170 ceph-mon[102832]: pgmap v13823: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 188 KiB/s wr, 15 op/s 2023-09-10T06:26:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:26 smithi082 ceph-mon[98694]: pgmap v13823: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 188 KiB/s wr, 15 op/s 2023-09-10T06:26:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:26 smithi098 ceph-mon[103124]: pgmap v13823: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 188 KiB/s wr, 15 op/s 2023-09-10T06:26:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:28 smithi082 ceph-mon[98694]: pgmap v13824: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T06:26:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25620.468166 secs 2023-09-10T06:26:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:28 smithi098 ceph-mon[103124]: pgmap v13824: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T06:26:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25620.468166 secs 2023-09-10T06:26:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:28 smithi170 ceph-mon[102832]: pgmap v13824: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T06:26:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25620.468166 secs 2023-09-10T06:26:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25621.081185 secs 2023-09-10T06:26:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25621.081185 secs 2023-09-10T06:26:30.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25621.081185 secs 2023-09-10T06:26:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:31 smithi170 ceph-mon[102832]: pgmap v13825: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T06:26:32.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:26:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:31 smithi082 ceph-mon[98694]: pgmap v13825: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T06:26:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:26:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:31 smithi098 ceph-mon[103124]: pgmap v13825: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 10 op/s 2023-09-10T06:26:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:26:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:26:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:26:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:26:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:26:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:26:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:26:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:26:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:26:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:26:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:33 smithi170 ceph-mon[102832]: pgmap v13826: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 170 KiB/s wr, 15 op/s 2023-09-10T06:26:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:26:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:26:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:33 smithi082 ceph-mon[98694]: pgmap v13826: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 170 KiB/s wr, 15 op/s 2023-09-10T06:26:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:26:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:26:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:33 smithi098 ceph-mon[103124]: pgmap v13826: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 170 KiB/s wr, 15 op/s 2023-09-10T06:26:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:26:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:26:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25625.468249 secs 2023-09-10T06:26:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25626.081252 secs 2023-09-10T06:26:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25625.468249 secs 2023-09-10T06:26:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25626.081252 secs 2023-09-10T06:26:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25625.468249 secs 2023-09-10T06:26:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25626.081252 secs 2023-09-10T06:26:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:35 smithi170 ceph-mon[102832]: pgmap v13827: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 11 op/s 2023-09-10T06:26:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:35 smithi082 ceph-mon[98694]: pgmap v13827: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 11 op/s 2023-09-10T06:26:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:35 smithi098 ceph-mon[103124]: pgmap v13827: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 11 op/s 2023-09-10T06:26:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:26:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:26:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:26:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:26:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:26:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:26:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:37 smithi170 ceph-mon[102832]: pgmap v13828: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 623 KiB/s wr, 18 op/s 2023-09-10T06:26:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:37 smithi082 ceph-mon[98694]: pgmap v13828: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 623 KiB/s wr, 18 op/s 2023-09-10T06:26:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:37 smithi098 ceph-mon[103124]: pgmap v13828: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 623 KiB/s wr, 18 op/s 2023-09-10T06:26:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:38 smithi170 ceph-mon[102832]: pgmap v13829: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 537 KiB/s wr, 12 op/s 2023-09-10T06:26:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:38 smithi082 ceph-mon[98694]: pgmap v13829: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 537 KiB/s wr, 12 op/s 2023-09-10T06:26:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:38 smithi098 ceph-mon[103124]: pgmap v13829: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 537 KiB/s wr, 12 op/s 2023-09-10T06:26:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25630.468305 secs 2023-09-10T06:26:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25631.081320 secs 2023-09-10T06:26:40.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25630.468305 secs 2023-09-10T06:26:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25631.081320 secs 2023-09-10T06:26:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25630.468305 secs 2023-09-10T06:26:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25631.081320 secs 2023-09-10T06:26:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:41 smithi170 ceph-mon[102832]: pgmap v13830: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 537 KiB/s wr, 12 op/s 2023-09-10T06:26:42.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:41 smithi082 ceph-mon[98694]: pgmap v13830: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 537 KiB/s wr, 12 op/s 2023-09-10T06:26:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:41 smithi098 ceph-mon[103124]: pgmap v13830: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 537 KiB/s wr, 12 op/s 2023-09-10T06:26:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:43 smithi170 ceph-mon[102832]: pgmap v13831: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 552 KiB/s wr, 18 op/s 2023-09-10T06:26:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:43 smithi082 ceph-mon[98694]: pgmap v13831: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 552 KiB/s wr, 18 op/s 2023-09-10T06:26:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:43 smithi098 ceph-mon[103124]: pgmap v13831: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 552 KiB/s wr, 18 op/s 2023-09-10T06:26:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25635.468389 secs 2023-09-10T06:26:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:26:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:26:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25636.081380 secs 2023-09-10T06:26:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25635.468389 secs 2023-09-10T06:26:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:26:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:26:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25636.081380 secs 2023-09-10T06:26:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25635.468389 secs 2023-09-10T06:26:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:26:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:26:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25636.081380 secs 2023-09-10T06:26:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:45 smithi170 ceph-mon[102832]: pgmap v13832: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 470 KiB/s wr, 13 op/s 2023-09-10T06:26:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:26:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:26:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:45 smithi082 ceph-mon[98694]: pgmap v13832: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 470 KiB/s wr, 13 op/s 2023-09-10T06:26:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:26:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:26:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:45 smithi098 ceph-mon[103124]: pgmap v13832: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 470 KiB/s wr, 13 op/s 2023-09-10T06:26:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:26:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:26:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:47 smithi170 ceph-mon[102832]: pgmap v13833: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 772 KiB/s wr, 19 op/s 2023-09-10T06:26:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:47 smithi082 ceph-mon[98694]: pgmap v13833: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 772 KiB/s wr, 19 op/s 2023-09-10T06:26:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:47 smithi098 ceph-mon[103124]: pgmap v13833: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 772 KiB/s wr, 19 op/s 2023-09-10T06:26:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:49 smithi170 ceph-mon[102832]: pgmap v13834: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 318 KiB/s wr, 12 op/s 2023-09-10T06:26:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25640.468448 secs 2023-09-10T06:26:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:26:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:26:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25641.081439 secs 2023-09-10T06:26:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:49 smithi082 ceph-mon[98694]: pgmap v13834: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 318 KiB/s wr, 12 op/s 2023-09-10T06:26:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25640.468448 secs 2023-09-10T06:26:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:26:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:26:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25641.081439 secs 2023-09-10T06:26:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:49 smithi098 ceph-mon[103124]: pgmap v13834: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 318 KiB/s wr, 12 op/s 2023-09-10T06:26:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25640.468448 secs 2023-09-10T06:26:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:26:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:26:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25641.081439 secs 2023-09-10T06:26:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:51 smithi170 ceph-mon[102832]: pgmap v13835: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 318 KiB/s wr, 12 op/s 2023-09-10T06:26:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:51 smithi082 ceph-mon[98694]: pgmap v13835: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 318 KiB/s wr, 12 op/s 2023-09-10T06:26:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:51 smithi098 ceph-mon[103124]: pgmap v13835: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 318 KiB/s wr, 12 op/s 2023-09-10T06:26:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:26:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:26:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:26:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:26:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:26:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:26:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:53 smithi170 ceph-mon[102832]: pgmap v13836: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 632 KiB/s wr, 19 op/s 2023-09-10T06:26:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25644.461107 secs 2023-09-10T06:26:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:53 smithi082 ceph-mon[98694]: pgmap v13836: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 632 KiB/s wr, 19 op/s 2023-09-10T06:26:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25644.461107 secs 2023-09-10T06:26:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:53 smithi098 ceph-mon[103124]: pgmap v13836: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 632 KiB/s wr, 19 op/s 2023-09-10T06:26:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25644.461107 secs 2023-09-10T06:26:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25645.468506 secs 2023-09-10T06:26:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25646.081492 secs 2023-09-10T06:26:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:54 smithi170 ceph-mon[102832]: pgmap v13837: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 616 KiB/s wr, 13 op/s 2023-09-10T06:26:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25645.468506 secs 2023-09-10T06:26:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25646.081492 secs 2023-09-10T06:26:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:54 smithi082 ceph-mon[98694]: pgmap v13837: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 616 KiB/s wr, 13 op/s 2023-09-10T06:26:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25645.468506 secs 2023-09-10T06:26:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25646.081492 secs 2023-09-10T06:26:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:54 smithi098 ceph-mon[103124]: pgmap v13837: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 616 KiB/s wr, 13 op/s 2023-09-10T06:26:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:26:56.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:26:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:26:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:26:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:26:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:26:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:56 smithi170 ceph-mon[102832]: pgmap v13838: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 671 KiB/s wr, 19 op/s 2023-09-10T06:26:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:26:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:26:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:26:57.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:26:57.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:26:57.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:26:57.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:26:57.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:26:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:26:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:26:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:56 smithi082 ceph-mon[98694]: pgmap v13838: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 671 KiB/s wr, 19 op/s 2023-09-10T06:26:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:26:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:26:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:26:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:26:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:26:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:26:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:26:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:26:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:26:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:26:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:56 smithi098 ceph-mon[103124]: pgmap v13838: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 671 KiB/s wr, 19 op/s 2023-09-10T06:26:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:26:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:26:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:26:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:26:57.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:26:57.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:26:57.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:26:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25649.461170 secs 2023-09-10T06:26:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:58 smithi098 ceph-mon[103124]: pgmap v13839: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 368 KiB/s wr, 12 op/s 2023-09-10T06:26:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25650.468564 secs 2023-09-10T06:26:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25649.461170 secs 2023-09-10T06:26:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:58 smithi170 ceph-mon[102832]: pgmap v13839: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 368 KiB/s wr, 12 op/s 2023-09-10T06:26:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25650.468564 secs 2023-09-10T06:26:59.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25649.461170 secs 2023-09-10T06:26:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:58 smithi082 ceph-mon[98694]: pgmap v13839: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 368 KiB/s wr, 12 op/s 2023-09-10T06:26:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25650.468564 secs 2023-09-10T06:27:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:26:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25651.081556 secs 2023-09-10T06:27:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:26:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25651.081556 secs 2023-09-10T06:27:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:26:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25651.081556 secs 2023-09-10T06:27:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:01 smithi170 ceph-mon[102832]: pgmap v13840: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 368 KiB/s wr, 12 op/s 2023-09-10T06:27:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:01 smithi082 ceph-mon[98694]: pgmap v13840: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 368 KiB/s wr, 12 op/s 2023-09-10T06:27:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:01 smithi098 ceph-mon[103124]: pgmap v13840: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 368 KiB/s wr, 12 op/s 2023-09-10T06:27:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:03 smithi170 ceph-mon[102832]: pgmap v13841: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 572 KiB/s wr, 20 op/s 2023-09-10T06:27:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25654.461245 secs 2023-09-10T06:27:04.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:03 smithi082 ceph-mon[98694]: pgmap v13841: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 572 KiB/s wr, 20 op/s 2023-09-10T06:27:04.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25654.461245 secs 2023-09-10T06:27:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:03 smithi098 ceph-mon[103124]: pgmap v13841: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 572 KiB/s wr, 20 op/s 2023-09-10T06:27:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25654.461245 secs 2023-09-10T06:27:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25655.468628 secs 2023-09-10T06:27:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:27:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25656.081619 secs 2023-09-10T06:27:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25655.468628 secs 2023-09-10T06:27:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:27:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25656.081619 secs 2023-09-10T06:27:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25655.468628 secs 2023-09-10T06:27:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:27:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25656.081619 secs 2023-09-10T06:27:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:05 smithi170 ceph-mon[102832]: pgmap v13842: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 257 KiB/s wr, 13 op/s 2023-09-10T06:27:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:27:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:05 smithi082 ceph-mon[98694]: pgmap v13842: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 257 KiB/s wr, 13 op/s 2023-09-10T06:27:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:27:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:05 smithi098 ceph-mon[103124]: pgmap v13842: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 257 KiB/s wr, 13 op/s 2023-09-10T06:27:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:27:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:07 smithi170 ceph-mon[102832]: pgmap v13843: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 372 KiB/s wr, 31 op/s 2023-09-10T06:27:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:07 smithi082 ceph-mon[98694]: pgmap v13843: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 372 KiB/s wr, 31 op/s 2023-09-10T06:27:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:07 smithi098 ceph-mon[103124]: pgmap v13843: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 372 KiB/s wr, 31 op/s 2023-09-10T06:27:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25659.461304 secs 2023-09-10T06:27:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:08 smithi170 ceph-mon[102832]: osdmap e933: 12 total, 12 up, 12 in 2023-09-10T06:27:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25659.461304 secs 2023-09-10T06:27:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:08 smithi082 ceph-mon[98694]: osdmap e933: 12 total, 12 up, 12 in 2023-09-10T06:27:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25659.461304 secs 2023-09-10T06:27:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:08 smithi098 ceph-mon[103124]: osdmap e933: 12 total, 12 up, 12 in 2023-09-10T06:27:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:09 smithi170 ceph-mon[102832]: pgmap v13845: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 381 KiB/s wr, 31 op/s 2023-09-10T06:27:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25660.468702 secs 2023-09-10T06:27:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25661.081687 secs 2023-09-10T06:27:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:09 smithi082 ceph-mon[98694]: pgmap v13845: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 381 KiB/s wr, 31 op/s 2023-09-10T06:27:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25660.468702 secs 2023-09-10T06:27:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25661.081687 secs 2023-09-10T06:27:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:09 smithi098 ceph-mon[103124]: pgmap v13845: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 381 KiB/s wr, 31 op/s 2023-09-10T06:27:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25660.468702 secs 2023-09-10T06:27:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25661.081687 secs 2023-09-10T06:27:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:11 smithi170 ceph-mon[102832]: pgmap v13846: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 381 KiB/s wr, 31 op/s 2023-09-10T06:27:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:11 smithi082 ceph-mon[98694]: pgmap v13846: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 381 KiB/s wr, 31 op/s 2023-09-10T06:27:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:11 smithi098 ceph-mon[103124]: pgmap v13846: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 381 KiB/s wr, 31 op/s 2023-09-10T06:27:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:13 smithi170 ceph-mon[102832]: pgmap v13847: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 161 KiB/s wr, 28 op/s 2023-09-10T06:27:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25664.461388 secs 2023-09-10T06:27:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:13 smithi082 ceph-mon[98694]: pgmap v13847: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 161 KiB/s wr, 28 op/s 2023-09-10T06:27:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25664.461388 secs 2023-09-10T06:27:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:13 smithi098 ceph-mon[103124]: pgmap v13847: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 161 KiB/s wr, 28 op/s 2023-09-10T06:27:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25664.461388 secs 2023-09-10T06:27:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25665.468762 secs 2023-09-10T06:27:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25666.081751 secs 2023-09-10T06:27:15.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25665.468762 secs 2023-09-10T06:27:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25666.081751 secs 2023-09-10T06:27:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25665.468762 secs 2023-09-10T06:27:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25666.081751 secs 2023-09-10T06:27:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:15 smithi170 ceph-mon[102832]: pgmap v13848: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 161 KiB/s wr, 28 op/s 2023-09-10T06:27:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:15 smithi170 ceph-mon[102832]: osdmap e934: 12 total, 12 up, 12 in 2023-09-10T06:27:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:15 smithi082 ceph-mon[98694]: pgmap v13848: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 161 KiB/s wr, 28 op/s 2023-09-10T06:27:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:15 smithi082 ceph-mon[98694]: osdmap e934: 12 total, 12 up, 12 in 2023-09-10T06:27:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:15 smithi098 ceph-mon[103124]: pgmap v13848: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 161 KiB/s wr, 28 op/s 2023-09-10T06:27:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:15 smithi098 ceph-mon[103124]: osdmap e934: 12 total, 12 up, 12 in 2023-09-10T06:27:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:17 smithi170 ceph-mon[102832]: pgmap v13850: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 16 op/s 2023-09-10T06:27:18.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:17 smithi082 ceph-mon[98694]: pgmap v13850: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 16 op/s 2023-09-10T06:27:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:17 smithi098 ceph-mon[103124]: pgmap v13850: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 16 op/s 2023-09-10T06:27:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25669.461466 secs 2023-09-10T06:27:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25669.461466 secs 2023-09-10T06:27:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25669.461466 secs 2023-09-10T06:27:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:19 smithi170 ceph-mon[102832]: pgmap v13851: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 13 op/s 2023-09-10T06:27:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25670.468824 secs 2023-09-10T06:27:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25671.081818 secs 2023-09-10T06:27:20.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:19 smithi082 ceph-mon[98694]: pgmap v13851: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 13 op/s 2023-09-10T06:27:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25670.468824 secs 2023-09-10T06:27:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25671.081818 secs 2023-09-10T06:27:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:19 smithi098 ceph-mon[103124]: pgmap v13851: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 13 op/s 2023-09-10T06:27:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25670.468824 secs 2023-09-10T06:27:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25671.081818 secs 2023-09-10T06:27:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:21 smithi170 ceph-mon[102832]: pgmap v13852: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 13 op/s 2023-09-10T06:27:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:21 smithi082 ceph-mon[98694]: pgmap v13852: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 13 op/s 2023-09-10T06:27:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:21 smithi098 ceph-mon[103124]: pgmap v13852: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 13 op/s 2023-09-10T06:27:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:23 smithi170 ceph-mon[102832]: pgmap v13853: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T06:27:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25674.461528 secs 2023-09-10T06:27:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:23 smithi082 ceph-mon[98694]: pgmap v13853: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T06:27:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25674.461528 secs 2023-09-10T06:27:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:23 smithi098 ceph-mon[103124]: pgmap v13853: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T06:27:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25674.461528 secs 2023-09-10T06:27:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25675.468883 secs 2023-09-10T06:27:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25676.081881 secs 2023-09-10T06:27:25.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25675.468883 secs 2023-09-10T06:27:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25676.081881 secs 2023-09-10T06:27:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25675.468883 secs 2023-09-10T06:27:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25676.081881 secs 2023-09-10T06:27:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:25 smithi170 ceph-mon[102832]: pgmap v13854: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T06:27:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:25 smithi082 ceph-mon[98694]: pgmap v13854: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T06:27:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:25 smithi098 ceph-mon[103124]: pgmap v13854: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T06:27:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:27 smithi170 ceph-mon[102832]: pgmap v13855: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 201 KiB/s wr, 11 op/s 2023-09-10T06:27:28.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:27 smithi082 ceph-mon[98694]: pgmap v13855: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 201 KiB/s wr, 11 op/s 2023-09-10T06:27:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:27 smithi098 ceph-mon[103124]: pgmap v13855: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 201 KiB/s wr, 11 op/s 2023-09-10T06:27:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:29 smithi170 ceph-mon[102832]: pgmap v13856: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 170 KiB/s wr, 9 op/s 2023-09-10T06:27:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25680.468945 secs 2023-09-10T06:27:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25681.081952 secs 2023-09-10T06:27:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:29 smithi082 ceph-mon[98694]: pgmap v13856: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 170 KiB/s wr, 9 op/s 2023-09-10T06:27:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25680.468945 secs 2023-09-10T06:27:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25681.081952 secs 2023-09-10T06:27:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:29 smithi098 ceph-mon[103124]: pgmap v13856: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 170 KiB/s wr, 9 op/s 2023-09-10T06:27:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25680.468945 secs 2023-09-10T06:27:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25681.081952 secs 2023-09-10T06:27:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:31 smithi170 ceph-mon[102832]: pgmap v13857: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 170 KiB/s wr, 9 op/s 2023-09-10T06:27:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:27:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:31 smithi082 ceph-mon[98694]: pgmap v13857: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 170 KiB/s wr, 9 op/s 2023-09-10T06:27:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:27:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:31 smithi098 ceph-mon[103124]: pgmap v13857: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 170 KiB/s wr, 9 op/s 2023-09-10T06:27:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:27:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:27:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:27:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:27:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:27:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:27:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:27:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:33 smithi170 ceph-mon[102832]: pgmap v13858: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 339 KiB/s wr, 15 op/s 2023-09-10T06:27:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:33 smithi082 ceph-mon[98694]: pgmap v13858: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 339 KiB/s wr, 15 op/s 2023-09-10T06:27:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:33 smithi098 ceph-mon[103124]: pgmap v13858: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 339 KiB/s wr, 15 op/s 2023-09-10T06:27:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25685.469003 secs 2023-09-10T06:27:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25686.082018 secs 2023-09-10T06:27:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25685.469003 secs 2023-09-10T06:27:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25686.082018 secs 2023-09-10T06:27:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25685.469003 secs 2023-09-10T06:27:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25686.082018 secs 2023-09-10T06:27:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:35 smithi170 ceph-mon[102832]: pgmap v13859: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 253 KiB/s wr, 10 op/s 2023-09-10T06:27:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:35 smithi082 ceph-mon[98694]: pgmap v13859: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 253 KiB/s wr, 10 op/s 2023-09-10T06:27:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:35 smithi098 ceph-mon[103124]: pgmap v13859: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 253 KiB/s wr, 10 op/s 2023-09-10T06:27:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:27:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:36 smithi170 ceph-mon[102832]: pgmap v13860: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 437 KiB/s wr, 16 op/s 2023-09-10T06:27:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:27:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:36 smithi082 ceph-mon[98694]: pgmap v13860: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 437 KiB/s wr, 16 op/s 2023-09-10T06:27:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:27:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:36 smithi098 ceph-mon[103124]: pgmap v13860: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 437 KiB/s wr, 16 op/s 2023-09-10T06:27:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:27:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:27:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:27:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:38 smithi170 ceph-mon[102832]: pgmap v13861: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 353 KiB/s wr, 11 op/s 2023-09-10T06:27:39.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:38 smithi082 ceph-mon[98694]: pgmap v13861: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 353 KiB/s wr, 11 op/s 2023-09-10T06:27:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:38 smithi098 ceph-mon[103124]: pgmap v13861: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 353 KiB/s wr, 11 op/s 2023-09-10T06:27:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25690.469062 secs 2023-09-10T06:27:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25691.082086 secs 2023-09-10T06:27:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25690.469062 secs 2023-09-10T06:27:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25691.082086 secs 2023-09-10T06:27:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25690.469062 secs 2023-09-10T06:27:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25691.082086 secs 2023-09-10T06:27:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:27:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:40 smithi170 ceph-mon[102832]: pgmap v13862: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 353 KiB/s wr, 11 op/s 2023-09-10T06:27:41.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:27:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:40 smithi082 ceph-mon[98694]: pgmap v13862: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 353 KiB/s wr, 11 op/s 2023-09-10T06:27:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:27:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:40 smithi098 ceph-mon[103124]: pgmap v13862: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 353 KiB/s wr, 11 op/s 2023-09-10T06:27:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:42 smithi098 ceph-mon[103124]: pgmap v13863: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 637 KiB/s wr, 37 op/s 2023-09-10T06:27:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:42 smithi170 ceph-mon[102832]: pgmap v13863: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 637 KiB/s wr, 37 op/s 2023-09-10T06:27:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:42 smithi082 ceph-mon[98694]: pgmap v13863: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 637 KiB/s wr, 37 op/s 2023-09-10T06:27:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:27:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25695.469121 secs 2023-09-10T06:27:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:27:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25695.469121 secs 2023-09-10T06:27:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:27:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25695.469121 secs 2023-09-10T06:27:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25696.082151 secs 2023-09-10T06:27:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:44 smithi098 ceph-mon[103124]: pgmap v13864: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 468 KiB/s wr, 31 op/s 2023-09-10T06:27:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25696.082151 secs 2023-09-10T06:27:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:44 smithi170 ceph-mon[102832]: pgmap v13864: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 468 KiB/s wr, 31 op/s 2023-09-10T06:27:45.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25696.082151 secs 2023-09-10T06:27:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:44 smithi082 ceph-mon[98694]: pgmap v13864: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 468 KiB/s wr, 31 op/s 2023-09-10T06:27:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:47 smithi170 ceph-mon[102832]: pgmap v13865: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 770 KiB/s wr, 38 op/s 2023-09-10T06:27:48.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:47 smithi082 ceph-mon[98694]: pgmap v13865: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 770 KiB/s wr, 38 op/s 2023-09-10T06:27:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:47 smithi098 ceph-mon[103124]: pgmap v13865: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 770 KiB/s wr, 38 op/s 2023-09-10T06:27:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:49 smithi170 ceph-mon[102832]: pgmap v13866: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 586 KiB/s wr, 32 op/s 2023-09-10T06:27:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25700.469180 secs 2023-09-10T06:27:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:27:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25701.082210 secs 2023-09-10T06:27:50.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:49 smithi082 ceph-mon[98694]: pgmap v13866: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 586 KiB/s wr, 32 op/s 2023-09-10T06:27:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25700.469180 secs 2023-09-10T06:27:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:27:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25701.082210 secs 2023-09-10T06:27:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:49 smithi098 ceph-mon[103124]: pgmap v13866: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 586 KiB/s wr, 32 op/s 2023-09-10T06:27:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25700.469180 secs 2023-09-10T06:27:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:27:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25701.082210 secs 2023-09-10T06:27:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:51 smithi170 ceph-mon[102832]: pgmap v13867: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 586 KiB/s wr, 32 op/s 2023-09-10T06:27:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:51 smithi082 ceph-mon[98694]: pgmap v13867: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 586 KiB/s wr, 32 op/s 2023-09-10T06:27:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:51 smithi098 ceph-mon[103124]: pgmap v13867: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 586 KiB/s wr, 32 op/s 2023-09-10T06:27:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:53 smithi170 ceph-mon[102832]: pgmap v13868: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 866 KiB/s wr, 38 op/s 2023-09-10T06:27:54.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:53 smithi082 ceph-mon[98694]: pgmap v13868: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 866 KiB/s wr, 38 op/s 2023-09-10T06:27:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:53 smithi098 ceph-mon[103124]: pgmap v13868: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 866 KiB/s wr, 38 op/s 2023-09-10T06:27:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25705.469240 secs 2023-09-10T06:27:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25706.082277 secs 2023-09-10T06:27:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25705.469240 secs 2023-09-10T06:27:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25706.082277 secs 2023-09-10T06:27:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25705.469240 secs 2023-09-10T06:27:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25706.082277 secs 2023-09-10T06:27:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:55 smithi170 ceph-mon[102832]: pgmap v13869: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 582 KiB/s wr, 12 op/s 2023-09-10T06:27:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:55 smithi082 ceph-mon[98694]: pgmap v13869: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 582 KiB/s wr, 12 op/s 2023-09-10T06:27:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:55 smithi098 ceph-mon[103124]: pgmap v13869: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 582 KiB/s wr, 12 op/s 2023-09-10T06:27:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:27:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:56 smithi170 ceph-mon[102832]: pgmap v13870: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 671 KiB/s wr, 18 op/s 2023-09-10T06:27:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:27:57.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:27:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:56 smithi082 ceph-mon[98694]: pgmap v13870: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 671 KiB/s wr, 18 op/s 2023-09-10T06:27:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:27:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:27:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:56 smithi098 ceph-mon[103124]: pgmap v13870: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 671 KiB/s wr, 18 op/s 2023-09-10T06:27:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:27:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:27:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:27:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:27:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:27:58.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:27:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:27:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:27:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:27:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:27:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:27:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:27:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:27:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:27:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25709.462089 secs 2023-09-10T06:27:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:58 smithi170 ceph-mon[102832]: pgmap v13871: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 368 KiB/s wr, 11 op/s 2023-09-10T06:27:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25709.462089 secs 2023-09-10T06:27:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:58 smithi082 ceph-mon[98694]: pgmap v13871: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 368 KiB/s wr, 11 op/s 2023-09-10T06:27:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25709.462089 secs 2023-09-10T06:27:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:58 smithi098 ceph-mon[103124]: pgmap v13871: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 368 KiB/s wr, 11 op/s 2023-09-10T06:28:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25710.469326 secs 2023-09-10T06:28:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:27:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25711.082342 secs 2023-09-10T06:28:00.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25710.469326 secs 2023-09-10T06:28:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:27:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25711.082342 secs 2023-09-10T06:28:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25710.469326 secs 2023-09-10T06:28:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:27:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25711.082342 secs 2023-09-10T06:28:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:28:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:28:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:28:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:00 smithi170 ceph-mon[102832]: pgmap v13872: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 368 KiB/s wr, 11 op/s 2023-09-10T06:28:01.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:28:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:28:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:28:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:00 smithi082 ceph-mon[98694]: pgmap v13872: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 368 KiB/s wr, 11 op/s 2023-09-10T06:28:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:28:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:28:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:28:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:00 smithi098 ceph-mon[103124]: pgmap v13872: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 368 KiB/s wr, 11 op/s 2023-09-10T06:28:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:02 smithi098 ceph-mon[103124]: pgmap v13873: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 554 KiB/s wr, 17 op/s 2023-09-10T06:28:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:02 smithi170 ceph-mon[102832]: pgmap v13873: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 554 KiB/s wr, 17 op/s 2023-09-10T06:28:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:02 smithi082 ceph-mon[98694]: pgmap v13873: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 554 KiB/s wr, 17 op/s 2023-09-10T06:28:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25714.462180 secs 2023-09-10T06:28:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25715.469416 secs 2023-09-10T06:28:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25714.462180 secs 2023-09-10T06:28:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25715.469416 secs 2023-09-10T06:28:04.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25714.462180 secs 2023-09-10T06:28:04.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25715.469416 secs 2023-09-10T06:28:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25716.082409 secs 2023-09-10T06:28:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:04 smithi098 ceph-mon[103124]: pgmap v13874: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 275 KiB/s wr, 11 op/s 2023-09-10T06:28:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25716.082409 secs 2023-09-10T06:28:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:04 smithi170 ceph-mon[102832]: pgmap v13874: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 275 KiB/s wr, 11 op/s 2023-09-10T06:28:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25716.082409 secs 2023-09-10T06:28:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:04 smithi082 ceph-mon[98694]: pgmap v13874: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 275 KiB/s wr, 11 op/s 2023-09-10T06:28:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:07 smithi170 ceph-mon[102832]: pgmap v13875: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 407 KiB/s wr, 31 op/s 2023-09-10T06:28:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:07 smithi082 ceph-mon[98694]: pgmap v13875: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 407 KiB/s wr, 31 op/s 2023-09-10T06:28:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:07 smithi098 ceph-mon[103124]: pgmap v13875: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 407 KiB/s wr, 31 op/s 2023-09-10T06:28:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25719.462264 secs 2023-09-10T06:28:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:08 smithi170 ceph-mon[102832]: osdmap e935: 12 total, 12 up, 12 in 2023-09-10T06:28:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25719.462264 secs 2023-09-10T06:28:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:08 smithi082 ceph-mon[98694]: osdmap e935: 12 total, 12 up, 12 in 2023-09-10T06:28:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25719.462264 secs 2023-09-10T06:28:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:08 smithi098 ceph-mon[103124]: osdmap e935: 12 total, 12 up, 12 in 2023-09-10T06:28:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:09 smithi170 ceph-mon[102832]: pgmap v13877: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 382 KiB/s wr, 31 op/s 2023-09-10T06:28:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25720.469499 secs 2023-09-10T06:28:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25721.082485 secs 2023-09-10T06:28:10.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:09 smithi082 ceph-mon[98694]: pgmap v13877: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 382 KiB/s wr, 31 op/s 2023-09-10T06:28:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25720.469499 secs 2023-09-10T06:28:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25721.082485 secs 2023-09-10T06:28:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:09 smithi098 ceph-mon[103124]: pgmap v13877: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 382 KiB/s wr, 31 op/s 2023-09-10T06:28:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25720.469499 secs 2023-09-10T06:28:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25721.082485 secs 2023-09-10T06:28:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:11 smithi170 ceph-mon[102832]: pgmap v13878: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 382 KiB/s wr, 31 op/s 2023-09-10T06:28:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:11 smithi082 ceph-mon[98694]: pgmap v13878: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 382 KiB/s wr, 31 op/s 2023-09-10T06:28:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:11 smithi098 ceph-mon[103124]: pgmap v13878: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 382 KiB/s wr, 31 op/s 2023-09-10T06:28:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:13 smithi170 ceph-mon[102832]: pgmap v13879: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 160 KiB/s wr, 30 op/s 2023-09-10T06:28:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25724.462324 secs 2023-09-10T06:28:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:13 smithi082 ceph-mon[98694]: pgmap v13879: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 160 KiB/s wr, 30 op/s 2023-09-10T06:28:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25724.462324 secs 2023-09-10T06:28:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:13 smithi098 ceph-mon[103124]: pgmap v13879: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 160 KiB/s wr, 30 op/s 2023-09-10T06:28:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25724.462324 secs 2023-09-10T06:28:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25725.469559 secs 2023-09-10T06:28:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25726.082547 secs 2023-09-10T06:28:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25725.469559 secs 2023-09-10T06:28:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25726.082547 secs 2023-09-10T06:28:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25725.469559 secs 2023-09-10T06:28:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25726.082547 secs 2023-09-10T06:28:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:15 smithi170 ceph-mon[102832]: pgmap v13880: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 160 KiB/s wr, 30 op/s 2023-09-10T06:28:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:15 smithi170 ceph-mon[102832]: osdmap e936: 12 total, 12 up, 12 in 2023-09-10T06:28:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:15 smithi082 ceph-mon[98694]: pgmap v13880: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 160 KiB/s wr, 30 op/s 2023-09-10T06:28:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:15 smithi082 ceph-mon[98694]: osdmap e936: 12 total, 12 up, 12 in 2023-09-10T06:28:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:15 smithi098 ceph-mon[103124]: pgmap v13880: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 160 KiB/s wr, 30 op/s 2023-09-10T06:28:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:15 smithi098 ceph-mon[103124]: osdmap e936: 12 total, 12 up, 12 in 2023-09-10T06:28:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:28:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:28:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:16 smithi170 ceph-mon[102832]: pgmap v13882: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.0 KiB/s wr, 15 op/s 2023-09-10T06:28:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:28:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:28:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:16 smithi082 ceph-mon[98694]: pgmap v13882: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.0 KiB/s wr, 15 op/s 2023-09-10T06:28:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:28:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:28:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:16 smithi098 ceph-mon[103124]: pgmap v13882: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.0 KiB/s wr, 15 op/s 2023-09-10T06:28:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:17 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:28:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:17 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:28:18.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:17 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:28:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:17 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:28:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:17 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:28:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:17 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:28:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25729.462385 secs 2023-09-10T06:28:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:18 smithi170 ceph-mon[102832]: pgmap v13883: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-09-10T06:28:19.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25729.462385 secs 2023-09-10T06:28:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:18 smithi082 ceph-mon[98694]: pgmap v13883: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-09-10T06:28:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25729.462385 secs 2023-09-10T06:28:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:18 smithi098 ceph-mon[103124]: pgmap v13883: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-09-10T06:28:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25730.469619 secs 2023-09-10T06:28:20.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25731.082610 secs 2023-09-10T06:28:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25730.469619 secs 2023-09-10T06:28:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25731.082610 secs 2023-09-10T06:28:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25730.469619 secs 2023-09-10T06:28:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25731.082610 secs 2023-09-10T06:28:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:21 smithi170 ceph-mon[102832]: pgmap v13884: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-09-10T06:28:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:21 smithi082 ceph-mon[98694]: pgmap v13884: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-09-10T06:28:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:21 smithi098 ceph-mon[103124]: pgmap v13884: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-09-10T06:28:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:23 smithi170 ceph-mon[102832]: pgmap v13885: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T06:28:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25734.462442 secs 2023-09-10T06:28:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:23 smithi082 ceph-mon[98694]: pgmap v13885: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T06:28:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25734.462442 secs 2023-09-10T06:28:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:23 smithi098 ceph-mon[103124]: pgmap v13885: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T06:28:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25734.462442 secs 2023-09-10T06:28:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25735.469725 secs 2023-09-10T06:28:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25736.082676 secs 2023-09-10T06:28:25.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25735.469725 secs 2023-09-10T06:28:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25736.082676 secs 2023-09-10T06:28:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25735.469725 secs 2023-09-10T06:28:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25736.082676 secs 2023-09-10T06:28:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:25 smithi170 ceph-mon[102832]: pgmap v13886: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T06:28:26.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:25 smithi082 ceph-mon[98694]: pgmap v13886: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T06:28:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:25 smithi098 ceph-mon[103124]: pgmap v13886: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T06:28:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:27 smithi170 ceph-mon[102832]: pgmap v13887: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T06:28:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:27 smithi082 ceph-mon[98694]: pgmap v13887: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T06:28:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:27 smithi098 ceph-mon[103124]: pgmap v13887: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 103 KiB/s wr, 11 op/s 2023-09-10T06:28:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25739.462524 secs 2023-09-10T06:28:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25739.462524 secs 2023-09-10T06:28:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25739.462524 secs 2023-09-10T06:28:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:29 smithi170 ceph-mon[102832]: pgmap v13888: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 9 op/s 2023-09-10T06:28:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25740.469780 secs 2023-09-10T06:28:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25741.082745 secs 2023-09-10T06:28:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:29 smithi082 ceph-mon[98694]: pgmap v13888: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 9 op/s 2023-09-10T06:28:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25740.469780 secs 2023-09-10T06:28:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25741.082745 secs 2023-09-10T06:28:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:29 smithi098 ceph-mon[103124]: pgmap v13888: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 9 op/s 2023-09-10T06:28:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25740.469780 secs 2023-09-10T06:28:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25741.082745 secs 2023-09-10T06:28:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:31 smithi170 ceph-mon[102832]: pgmap v13889: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 9 op/s 2023-09-10T06:28:32.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:31 smithi082 ceph-mon[98694]: pgmap v13889: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 9 op/s 2023-09-10T06:28:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:31 smithi098 ceph-mon[103124]: pgmap v13889: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 9 op/s 2023-09-10T06:28:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:33 smithi170 ceph-mon[102832]: pgmap v13890: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 339 KiB/s wr, 14 op/s 2023-09-10T06:28:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25744.462584 secs 2023-09-10T06:28:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:33 smithi082 ceph-mon[98694]: pgmap v13890: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 339 KiB/s wr, 14 op/s 2023-09-10T06:28:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25744.462584 secs 2023-09-10T06:28:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:33 smithi098 ceph-mon[103124]: pgmap v13890: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 339 KiB/s wr, 14 op/s 2023-09-10T06:28:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25744.462584 secs 2023-09-10T06:28:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25745.469852 secs 2023-09-10T06:28:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25746.082810 secs 2023-09-10T06:28:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25745.469852 secs 2023-09-10T06:28:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25746.082810 secs 2023-09-10T06:28:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25745.469852 secs 2023-09-10T06:28:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25746.082810 secs 2023-09-10T06:28:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:35 smithi170 ceph-mon[102832]: pgmap v13891: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 252 KiB/s wr, 9 op/s 2023-09-10T06:28:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:35 smithi082 ceph-mon[98694]: pgmap v13891: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 252 KiB/s wr, 9 op/s 2023-09-10T06:28:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:35 smithi098 ceph-mon[103124]: pgmap v13891: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 252 KiB/s wr, 9 op/s 2023-09-10T06:28:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:36 smithi170 ceph-mon[102832]: pgmap v13892: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 370 KiB/s wr, 15 op/s 2023-09-10T06:28:37.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:36 smithi082 ceph-mon[98694]: pgmap v13892: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 370 KiB/s wr, 15 op/s 2023-09-10T06:28:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:36 smithi098 ceph-mon[103124]: pgmap v13892: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 370 KiB/s wr, 15 op/s 2023-09-10T06:28:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:28:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:28:38.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:28:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:28:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:28:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:28:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:38 smithi170 ceph-mon[102832]: pgmap v13893: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 370 KiB/s wr, 11 op/s 2023-09-10T06:28:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:38 smithi082 ceph-mon[98694]: pgmap v13893: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 370 KiB/s wr, 11 op/s 2023-09-10T06:28:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:38 smithi098 ceph-mon[103124]: pgmap v13893: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 370 KiB/s wr, 11 op/s 2023-09-10T06:28:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25750.469955 secs 2023-09-10T06:28:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25751.082874 secs 2023-09-10T06:28:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:28:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25750.469955 secs 2023-09-10T06:28:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25751.082874 secs 2023-09-10T06:28:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:28:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25750.469955 secs 2023-09-10T06:28:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25751.082874 secs 2023-09-10T06:28:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:28:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:28:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:28:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:28:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:40 smithi170 ceph-mon[102832]: pgmap v13894: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 370 KiB/s wr, 11 op/s 2023-09-10T06:28:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:28:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:28:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:28:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:40 smithi082 ceph-mon[98694]: pgmap v13894: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 370 KiB/s wr, 11 op/s 2023-09-10T06:28:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:28:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:28:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:28:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:40 smithi098 ceph-mon[103124]: pgmap v13894: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 370 KiB/s wr, 11 op/s 2023-09-10T06:28:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:42 smithi098 ceph-mon[103124]: pgmap v13895: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 419 KiB/s wr, 17 op/s 2023-09-10T06:28:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:42 smithi170 ceph-mon[102832]: pgmap v13895: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 419 KiB/s wr, 17 op/s 2023-09-10T06:28:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:42 smithi082 ceph-mon[98694]: pgmap v13895: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 419 KiB/s wr, 17 op/s 2023-09-10T06:28:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25755.470013 secs 2023-09-10T06:28:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25755.470013 secs 2023-09-10T06:28:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25755.470013 secs 2023-09-10T06:28:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25756.082939 secs 2023-09-10T06:28:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:45 smithi098 ceph-mon[103124]: pgmap v13896: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 167 KiB/s wr, 12 op/s 2023-09-10T06:28:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25756.082939 secs 2023-09-10T06:28:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:45 smithi170 ceph-mon[102832]: pgmap v13896: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 167 KiB/s wr, 12 op/s 2023-09-10T06:28:45.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25756.082939 secs 2023-09-10T06:28:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:45 smithi082 ceph-mon[98694]: pgmap v13896: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 167 KiB/s wr, 12 op/s 2023-09-10T06:28:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:47 smithi170 ceph-mon[102832]: pgmap v13897: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 488 KiB/s wr, 19 op/s 2023-09-10T06:28:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:47 smithi082 ceph-mon[98694]: pgmap v13897: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 488 KiB/s wr, 19 op/s 2023-09-10T06:28:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:47 smithi098 ceph-mon[103124]: pgmap v13897: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 488 KiB/s wr, 19 op/s 2023-09-10T06:28:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:49 smithi170 ceph-mon[102832]: pgmap v13898: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 370 KiB/s wr, 13 op/s 2023-09-10T06:28:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25760.470091 secs 2023-09-10T06:28:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25761.083005 secs 2023-09-10T06:28:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:28:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:28:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:49 smithi082 ceph-mon[98694]: pgmap v13898: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 370 KiB/s wr, 13 op/s 2023-09-10T06:28:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25760.470091 secs 2023-09-10T06:28:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25761.083005 secs 2023-09-10T06:28:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:28:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:28:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:49 smithi098 ceph-mon[103124]: pgmap v13898: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 370 KiB/s wr, 13 op/s 2023-09-10T06:28:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25760.470091 secs 2023-09-10T06:28:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25761.083005 secs 2023-09-10T06:28:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:28:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:28:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:51 smithi170 ceph-mon[102832]: pgmap v13899: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 370 KiB/s wr, 13 op/s 2023-09-10T06:28:52.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:51 smithi082 ceph-mon[98694]: pgmap v13899: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 370 KiB/s wr, 13 op/s 2023-09-10T06:28:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:51 smithi098 ceph-mon[103124]: pgmap v13899: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 370 KiB/s wr, 13 op/s 2023-09-10T06:28:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:28:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:28:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:52 smithi170 ceph-mon[102832]: pgmap v13900: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 411 KiB/s wr, 18 op/s 2023-09-10T06:28:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:28:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:28:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:52 smithi082 ceph-mon[98694]: pgmap v13900: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 411 KiB/s wr, 18 op/s 2023-09-10T06:28:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:28:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:28:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:52 smithi098 ceph-mon[103124]: pgmap v13900: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 411 KiB/s wr, 18 op/s 2023-09-10T06:28:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:28:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:28:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:28:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:28:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:28:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:28:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25765.470148 secs 2023-09-10T06:28:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25766.083068 secs 2023-09-10T06:28:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:54 smithi170 ceph-mon[102832]: pgmap v13901: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 362 KiB/s wr, 12 op/s 2023-09-10T06:28:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25765.470148 secs 2023-09-10T06:28:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25766.083068 secs 2023-09-10T06:28:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:54 smithi082 ceph-mon[98694]: pgmap v13901: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 362 KiB/s wr, 12 op/s 2023-09-10T06:28:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25765.470148 secs 2023-09-10T06:28:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25766.083068 secs 2023-09-10T06:28:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:54 smithi098 ceph-mon[103124]: pgmap v13901: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 362 KiB/s wr, 12 op/s 2023-09-10T06:28:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:28:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:28:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:56 smithi170 ceph-mon[102832]: pgmap v13902: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 726 KiB/s wr, 19 op/s 2023-09-10T06:28:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:28:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:28:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:56 smithi082 ceph-mon[98694]: pgmap v13902: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 726 KiB/s wr, 19 op/s 2023-09-10T06:28:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:28:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:28:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:56 smithi098 ceph-mon[103124]: pgmap v13902: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 726 KiB/s wr, 19 op/s 2023-09-10T06:28:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:28:58.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:28:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:28:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25769.462923 secs 2023-09-10T06:28:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:58 smithi170 ceph-mon[102832]: pgmap v13903: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 405 KiB/s wr, 12 op/s 2023-09-10T06:28:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:28:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:28:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:28:59.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25769.462923 secs 2023-09-10T06:28:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:58 smithi082 ceph-mon[98694]: pgmap v13903: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 405 KiB/s wr, 12 op/s 2023-09-10T06:28:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:28:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:28:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:28:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25769.462923 secs 2023-09-10T06:28:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:58 smithi098 ceph-mon[103124]: pgmap v13903: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 405 KiB/s wr, 12 op/s 2023-09-10T06:28:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:28:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:28:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:29:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25770.470206 secs 2023-09-10T06:29:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:28:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25771.083132 secs 2023-09-10T06:29:00.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25770.470206 secs 2023-09-10T06:29:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:28:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25771.083132 secs 2023-09-10T06:29:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25770.470206 secs 2023-09-10T06:29:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:28:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25771.083132 secs 2023-09-10T06:29:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:29:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:29:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:00 smithi170 ceph-mon[102832]: pgmap v13904: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 405 KiB/s wr, 12 op/s 2023-09-10T06:29:01.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:29:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:29:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:00 smithi082 ceph-mon[98694]: pgmap v13904: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 405 KiB/s wr, 12 op/s 2023-09-10T06:29:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:29:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:29:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:00 smithi098 ceph-mon[103124]: pgmap v13904: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 405 KiB/s wr, 12 op/s 2023-09-10T06:29:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:03 smithi098 ceph-mon[103124]: pgmap v13905: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 487 KiB/s wr, 18 op/s 2023-09-10T06:29:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:03 smithi170 ceph-mon[102832]: pgmap v13905: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 487 KiB/s wr, 18 op/s 2023-09-10T06:29:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:03 smithi082 ceph-mon[98694]: pgmap v13905: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 487 KiB/s wr, 18 op/s 2023-09-10T06:29:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25774.463015 secs 2023-09-10T06:29:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25775.470288 secs 2023-09-10T06:29:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25774.463015 secs 2023-09-10T06:29:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25775.470288 secs 2023-09-10T06:29:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25774.463015 secs 2023-09-10T06:29:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25775.470288 secs 2023-09-10T06:29:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25776.083194 secs 2023-09-10T06:29:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:05 smithi170 ceph-mon[102832]: pgmap v13906: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 446 KiB/s wr, 12 op/s 2023-09-10T06:29:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25776.083194 secs 2023-09-10T06:29:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:05 smithi082 ceph-mon[98694]: pgmap v13906: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 446 KiB/s wr, 12 op/s 2023-09-10T06:29:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25776.083194 secs 2023-09-10T06:29:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:05 smithi098 ceph-mon[103124]: pgmap v13906: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 446 KiB/s wr, 12 op/s 2023-09-10T06:29:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:07 smithi170 ceph-mon[102832]: pgmap v13907: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 697 KiB/s wr, 32 op/s 2023-09-10T06:29:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:07 smithi082 ceph-mon[98694]: pgmap v13907: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 697 KiB/s wr, 32 op/s 2023-09-10T06:29:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:07 smithi098 ceph-mon[103124]: pgmap v13907: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 697 KiB/s wr, 32 op/s 2023-09-10T06:29:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25779.463077 secs 2023-09-10T06:29:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:08 smithi170 ceph-mon[102832]: osdmap e937: 12 total, 12 up, 12 in 2023-09-10T06:29:09.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25779.463077 secs 2023-09-10T06:29:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:08 smithi082 ceph-mon[98694]: osdmap e937: 12 total, 12 up, 12 in 2023-09-10T06:29:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25779.463077 secs 2023-09-10T06:29:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:08 smithi098 ceph-mon[103124]: osdmap e937: 12 total, 12 up, 12 in 2023-09-10T06:29:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:09 smithi170 ceph-mon[102832]: pgmap v13909: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 400 KiB/s wr, 31 op/s 2023-09-10T06:29:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25780.470368 secs 2023-09-10T06:29:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25781.083258 secs 2023-09-10T06:29:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:09 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:29:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:09 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:09 smithi082 ceph-mon[98694]: pgmap v13909: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 400 KiB/s wr, 31 op/s 2023-09-10T06:29:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25780.470368 secs 2023-09-10T06:29:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25781.083258 secs 2023-09-10T06:29:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:09 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:29:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:09 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:09 smithi098 ceph-mon[103124]: pgmap v13909: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 400 KiB/s wr, 31 op/s 2023-09-10T06:29:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25780.470368 secs 2023-09-10T06:29:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25781.083258 secs 2023-09-10T06:29:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:09 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:29:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:09 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:11 smithi170 ceph-mon[102832]: pgmap v13910: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 400 KiB/s wr, 31 op/s 2023-09-10T06:29:12.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:11 smithi082 ceph-mon[98694]: pgmap v13910: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 400 KiB/s wr, 31 op/s 2023-09-10T06:29:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:11 smithi098 ceph-mon[103124]: pgmap v13910: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 400 KiB/s wr, 31 op/s 2023-09-10T06:29:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:29:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:12 smithi170 ceph-mon[102832]: pgmap v13911: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 302 KiB/s wr, 30 op/s 2023-09-10T06:29:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:29:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:12 smithi082 ceph-mon[98694]: pgmap v13911: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 302 KiB/s wr, 30 op/s 2023-09-10T06:29:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:29:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:12 smithi098 ceph-mon[103124]: pgmap v13911: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 302 KiB/s wr, 30 op/s 2023-09-10T06:29:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25784.463229 secs 2023-09-10T06:29:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25784.463229 secs 2023-09-10T06:29:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25784.463229 secs 2023-09-10T06:29:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25785.470425 secs 2023-09-10T06:29:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25786.083324 secs 2023-09-10T06:29:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:14 smithi170 ceph-mon[102832]: pgmap v13912: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 302 KiB/s wr, 30 op/s 2023-09-10T06:29:15.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25785.470425 secs 2023-09-10T06:29:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25786.083324 secs 2023-09-10T06:29:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:14 smithi082 ceph-mon[98694]: pgmap v13912: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 302 KiB/s wr, 30 op/s 2023-09-10T06:29:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25785.470425 secs 2023-09-10T06:29:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25786.083324 secs 2023-09-10T06:29:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:14 smithi098 ceph-mon[103124]: pgmap v13912: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 302 KiB/s wr, 30 op/s 2023-09-10T06:29:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:16 smithi170 ceph-mon[102832]: osdmap e938: 12 total, 12 up, 12 in 2023-09-10T06:29:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:16 smithi082 ceph-mon[98694]: osdmap e938: 12 total, 12 up, 12 in 2023-09-10T06:29:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:16 smithi098 ceph-mon[103124]: osdmap e938: 12 total, 12 up, 12 in 2023-09-10T06:29:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:17 smithi170 ceph-mon[102832]: pgmap v13914: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 15 op/s 2023-09-10T06:29:18.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:17 smithi082 ceph-mon[98694]: pgmap v13914: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 15 op/s 2023-09-10T06:29:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:17 smithi098 ceph-mon[103124]: pgmap v13914: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 15 op/s 2023-09-10T06:29:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25789.463290 secs 2023-09-10T06:29:19.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25789.463290 secs 2023-09-10T06:29:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25789.463290 secs 2023-09-10T06:29:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:19 smithi170 ceph-mon[102832]: pgmap v13915: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T06:29:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25790.470510 secs 2023-09-10T06:29:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25791.083387 secs 2023-09-10T06:29:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:19 smithi082 ceph-mon[98694]: pgmap v13915: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T06:29:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25790.470510 secs 2023-09-10T06:29:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25791.083387 secs 2023-09-10T06:29:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:19 smithi098 ceph-mon[103124]: pgmap v13915: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T06:29:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25790.470510 secs 2023-09-10T06:29:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25791.083387 secs 2023-09-10T06:29:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:21 smithi170 ceph-mon[102832]: pgmap v13916: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T06:29:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:21 smithi082 ceph-mon[98694]: pgmap v13916: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T06:29:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:21 smithi098 ceph-mon[103124]: pgmap v13916: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T06:29:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:22 smithi170 ceph-mon[102832]: pgmap v13917: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T06:29:23.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:22 smithi082 ceph-mon[98694]: pgmap v13917: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T06:29:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:22 smithi098 ceph-mon[103124]: pgmap v13917: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T06:29:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25794.463370 secs 2023-09-10T06:29:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25794.463370 secs 2023-09-10T06:29:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25794.463370 secs 2023-09-10T06:29:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25795.470570 secs 2023-09-10T06:29:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25796.083455 secs 2023-09-10T06:29:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:24 smithi170 ceph-mon[102832]: pgmap v13918: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T06:29:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25795.470570 secs 2023-09-10T06:29:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25796.083455 secs 2023-09-10T06:29:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:24 smithi082 ceph-mon[98694]: pgmap v13918: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T06:29:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25795.470570 secs 2023-09-10T06:29:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25796.083455 secs 2023-09-10T06:29:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:24 smithi098 ceph-mon[103124]: pgmap v13918: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T06:29:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:27 smithi170 ceph-mon[102832]: pgmap v13919: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:29:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:27 smithi082 ceph-mon[98694]: pgmap v13919: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:29:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:27 smithi098 ceph-mon[103124]: pgmap v13919: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:29:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25799.463453 secs 2023-09-10T06:29:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25799.463453 secs 2023-09-10T06:29:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25799.463453 secs 2023-09-10T06:29:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:29 smithi170 ceph-mon[102832]: pgmap v13920: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T06:29:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25800.470628 secs 2023-09-10T06:29:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25801.083489 secs 2023-09-10T06:29:30.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:29 smithi082 ceph-mon[98694]: pgmap v13920: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T06:29:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25800.470628 secs 2023-09-10T06:29:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25801.083489 secs 2023-09-10T06:29:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:29 smithi098 ceph-mon[103124]: pgmap v13920: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T06:29:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25800.470628 secs 2023-09-10T06:29:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25801.083489 secs 2023-09-10T06:29:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:31 smithi170 ceph-mon[102832]: pgmap v13921: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T06:29:32.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:31 smithi082 ceph-mon[98694]: pgmap v13921: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T06:29:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:31 smithi098 ceph-mon[103124]: pgmap v13921: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T06:29:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:33 smithi170 ceph-mon[102832]: pgmap v13922: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 223 KiB/s wr, 15 op/s 2023-09-10T06:29:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:29:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:33 smithi082 ceph-mon[98694]: pgmap v13922: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 223 KiB/s wr, 15 op/s 2023-09-10T06:29:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:29:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:33 smithi098 ceph-mon[103124]: pgmap v13922: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 223 KiB/s wr, 15 op/s 2023-09-10T06:29:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:29:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25805.470690 secs 2023-09-10T06:29:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25806.083554 secs 2023-09-10T06:29:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:34 smithi170 ceph-mon[102832]: pgmap v13923: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 222 KiB/s wr, 10 op/s 2023-09-10T06:29:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25805.470690 secs 2023-09-10T06:29:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25806.083554 secs 2023-09-10T06:29:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:34 smithi082 ceph-mon[98694]: pgmap v13923: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 222 KiB/s wr, 10 op/s 2023-09-10T06:29:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25805.470690 secs 2023-09-10T06:29:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25806.083554 secs 2023-09-10T06:29:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:34 smithi098 ceph-mon[103124]: pgmap v13923: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 222 KiB/s wr, 10 op/s 2023-09-10T06:29:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:29:36.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:29:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:29:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:29:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:29:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:36 smithi170 ceph-mon[102832]: pgmap v13924: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 630 KiB/s wr, 17 op/s 2023-09-10T06:29:37.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:29:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:29:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:36 smithi082 ceph-mon[98694]: pgmap v13924: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 630 KiB/s wr, 17 op/s 2023-09-10T06:29:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:29:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:29:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:36 smithi098 ceph-mon[103124]: pgmap v13924: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 630 KiB/s wr, 17 op/s 2023-09-10T06:29:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:29:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:38.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:29:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:29:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:38 smithi170 ceph-mon[102832]: pgmap v13925: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 542 KiB/s wr, 12 op/s 2023-09-10T06:29:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:38 smithi082 ceph-mon[98694]: pgmap v13925: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 542 KiB/s wr, 12 op/s 2023-09-10T06:29:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:38 smithi098 ceph-mon[103124]: pgmap v13925: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 542 KiB/s wr, 12 op/s 2023-09-10T06:29:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25810.470750 secs 2023-09-10T06:29:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25811.083618 secs 2023-09-10T06:29:40.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25810.470750 secs 2023-09-10T06:29:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25811.083618 secs 2023-09-10T06:29:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25810.470750 secs 2023-09-10T06:29:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25811.083618 secs 2023-09-10T06:29:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:29:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:40 smithi170 ceph-mon[102832]: pgmap v13926: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 542 KiB/s wr, 12 op/s 2023-09-10T06:29:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:29:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:40 smithi082 ceph-mon[98694]: pgmap v13926: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 542 KiB/s wr, 12 op/s 2023-09-10T06:29:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:29:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:40 smithi098 ceph-mon[103124]: pgmap v13926: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 542 KiB/s wr, 12 op/s 2023-09-10T06:29:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:43 smithi170 ceph-mon[102832]: pgmap v13927: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 558 KiB/s wr, 18 op/s 2023-09-10T06:29:43.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:43 smithi082 ceph-mon[98694]: pgmap v13927: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 558 KiB/s wr, 18 op/s 2023-09-10T06:29:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:43 smithi098 ceph-mon[103124]: pgmap v13927: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 558 KiB/s wr, 18 op/s 2023-09-10T06:29:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25815.470806 secs 2023-09-10T06:29:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25815.470806 secs 2023-09-10T06:29:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25815.470806 secs 2023-09-10T06:29:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25816.083679 secs 2023-09-10T06:29:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:45 smithi170 ceph-mon[102832]: pgmap v13928: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 423 KiB/s wr, 13 op/s 2023-09-10T06:29:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25816.083679 secs 2023-09-10T06:29:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:45 smithi082 ceph-mon[98694]: pgmap v13928: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 423 KiB/s wr, 13 op/s 2023-09-10T06:29:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25816.083679 secs 2023-09-10T06:29:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:45 smithi098 ceph-mon[103124]: pgmap v13928: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 423 KiB/s wr, 13 op/s 2023-09-10T06:29:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:47 smithi170 ceph-mon[102832]: pgmap v13929: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 743 KiB/s wr, 20 op/s 2023-09-10T06:29:48.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:47 smithi082 ceph-mon[98694]: pgmap v13929: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 743 KiB/s wr, 20 op/s 2023-09-10T06:29:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:47 smithi098 ceph-mon[103124]: pgmap v13929: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 743 KiB/s wr, 20 op/s 2023-09-10T06:29:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:49 smithi170 ceph-mon[102832]: pgmap v13930: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 336 KiB/s wr, 12 op/s 2023-09-10T06:29:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25820.470885 secs 2023-09-10T06:29:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25821.083743 secs 2023-09-10T06:29:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:29:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:49 smithi082 ceph-mon[98694]: pgmap v13930: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 336 KiB/s wr, 12 op/s 2023-09-10T06:29:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25820.470885 secs 2023-09-10T06:29:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25821.083743 secs 2023-09-10T06:29:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:29:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:49 smithi098 ceph-mon[103124]: pgmap v13930: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 336 KiB/s wr, 12 op/s 2023-09-10T06:29:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25820.470885 secs 2023-09-10T06:29:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25821.083743 secs 2023-09-10T06:29:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:29:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:51 smithi170 ceph-mon[102832]: pgmap v13931: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 336 KiB/s wr, 12 op/s 2023-09-10T06:29:52.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:51 smithi082 ceph-mon[98694]: pgmap v13931: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 336 KiB/s wr, 12 op/s 2023-09-10T06:29:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:51 smithi098 ceph-mon[103124]: pgmap v13931: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 336 KiB/s wr, 12 op/s 2023-09-10T06:29:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:29:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:52 smithi170 ceph-mon[102832]: pgmap v13932: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 624 KiB/s wr, 19 op/s 2023-09-10T06:29:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:29:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:52 smithi082 ceph-mon[98694]: pgmap v13932: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 624 KiB/s wr, 19 op/s 2023-09-10T06:29:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:29:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:52 smithi098 ceph-mon[103124]: pgmap v13932: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 624 KiB/s wr, 19 op/s 2023-09-10T06:29:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25824.463793 secs 2023-09-10T06:29:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25824.463793 secs 2023-09-10T06:29:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25824.463793 secs 2023-09-10T06:29:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25825.470947 secs 2023-09-10T06:29:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25826.083808 secs 2023-09-10T06:29:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:54 smithi170 ceph-mon[102832]: pgmap v13933: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 608 KiB/s wr, 13 op/s 2023-09-10T06:29:55.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25825.470947 secs 2023-09-10T06:29:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25826.083808 secs 2023-09-10T06:29:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:54 smithi082 ceph-mon[98694]: pgmap v13933: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 608 KiB/s wr, 13 op/s 2023-09-10T06:29:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25825.470947 secs 2023-09-10T06:29:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25826.083808 secs 2023-09-10T06:29:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:54 smithi098 ceph-mon[103124]: pgmap v13933: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 608 KiB/s wr, 13 op/s 2023-09-10T06:29:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:29:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:29:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:29:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:29:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:29:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:29:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:57 smithi170 ceph-mon[102832]: pgmap v13934: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 700 KiB/s wr, 18 op/s 2023-09-10T06:29:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:29:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:58.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:57 smithi082 ceph-mon[98694]: pgmap v13934: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 700 KiB/s wr, 18 op/s 2023-09-10T06:29:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:29:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:57 smithi098 ceph-mon[103124]: pgmap v13934: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 700 KiB/s wr, 18 op/s 2023-09-10T06:29:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:29:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:29:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25829.463854 secs 2023-09-10T06:29:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:29:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:29:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:29:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:29:59.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25829.463854 secs 2023-09-10T06:29:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:29:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:29:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:29:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:29:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25829.463854 secs 2023-09-10T06:29:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:29:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:29:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:29:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:30:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:59 smithi170 ceph-mon[102832]: pgmap v13935: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 380 KiB/s wr, 11 op/s 2023-09-10T06:30:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25830.471006 secs 2023-09-10T06:30:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:29:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25831.083875 secs 2023-09-10T06:30:00.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:59 smithi082 ceph-mon[98694]: pgmap v13935: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 380 KiB/s wr, 11 op/s 2023-09-10T06:30:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25830.471006 secs 2023-09-10T06:30:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:29:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25831.083875 secs 2023-09-10T06:30:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:59 smithi098 ceph-mon[103124]: pgmap v13935: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 380 KiB/s wr, 11 op/s 2023-09-10T06:30:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25830.471006 secs 2023-09-10T06:30:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:29:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25831.083875 secs 2023-09-10T06:30:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:30:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:00 smithi170 ceph-mon[102832]: pgmap v13936: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 380 KiB/s wr, 11 op/s 2023-09-10T06:30:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T06:30:01.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:30:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:00 smithi082 ceph-mon[98694]: pgmap v13936: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 380 KiB/s wr, 11 op/s 2023-09-10T06:30:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T06:30:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:30:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:00 smithi098 ceph-mon[103124]: pgmap v13936: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 380 KiB/s wr, 11 op/s 2023-09-10T06:30:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T06:30:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:03 smithi170 ceph-mon[102832]: pgmap v13937: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 626 KiB/s wr, 17 op/s 2023-09-10T06:30:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:03 smithi082 ceph-mon[98694]: pgmap v13937: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 626 KiB/s wr, 17 op/s 2023-09-10T06:30:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:03 smithi098 ceph-mon[103124]: pgmap v13937: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 626 KiB/s wr, 17 op/s 2023-09-10T06:30:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25834.463915 secs 2023-09-10T06:30:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25835.471055 secs 2023-09-10T06:30:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25834.463915 secs 2023-09-10T06:30:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25835.471055 secs 2023-09-10T06:30:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25834.463915 secs 2023-09-10T06:30:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25835.471055 secs 2023-09-10T06:30:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25836.083940 secs 2023-09-10T06:30:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:05 smithi170 ceph-mon[102832]: pgmap v13938: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 337 KiB/s wr, 10 op/s 2023-09-10T06:30:05.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25836.083940 secs 2023-09-10T06:30:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:05 smithi082 ceph-mon[98694]: pgmap v13938: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 337 KiB/s wr, 10 op/s 2023-09-10T06:30:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25836.083940 secs 2023-09-10T06:30:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:05 smithi098 ceph-mon[103124]: pgmap v13938: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 337 KiB/s wr, 10 op/s 2023-09-10T06:30:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:07 smithi170 ceph-mon[102832]: pgmap v13939: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 470 KiB/s wr, 31 op/s 2023-09-10T06:30:08.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:07 smithi082 ceph-mon[98694]: pgmap v13939: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 470 KiB/s wr, 31 op/s 2023-09-10T06:30:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:07 smithi098 ceph-mon[103124]: pgmap v13939: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 470 KiB/s wr, 31 op/s 2023-09-10T06:30:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25839.463978 secs 2023-09-10T06:30:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:08 smithi170 ceph-mon[102832]: osdmap e939: 12 total, 12 up, 12 in 2023-09-10T06:30:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25839.463978 secs 2023-09-10T06:30:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:08 smithi082 ceph-mon[98694]: osdmap e939: 12 total, 12 up, 12 in 2023-09-10T06:30:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25839.463978 secs 2023-09-10T06:30:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:08 smithi098 ceph-mon[103124]: osdmap e939: 12 total, 12 up, 12 in 2023-09-10T06:30:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:09 smithi170 ceph-mon[102832]: pgmap v13941: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 454 KiB/s wr, 30 op/s 2023-09-10T06:30:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25840.471114 secs 2023-09-10T06:30:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25841.084004 secs 2023-09-10T06:30:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:09 smithi082 ceph-mon[98694]: pgmap v13941: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 454 KiB/s wr, 30 op/s 2023-09-10T06:30:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25840.471114 secs 2023-09-10T06:30:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25841.084004 secs 2023-09-10T06:30:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:09 smithi098 ceph-mon[103124]: pgmap v13941: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 454 KiB/s wr, 30 op/s 2023-09-10T06:30:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25840.471114 secs 2023-09-10T06:30:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25841.084004 secs 2023-09-10T06:30:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:11 smithi170 ceph-mon[102832]: pgmap v13942: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 454 KiB/s wr, 30 op/s 2023-09-10T06:30:12.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:11 smithi082 ceph-mon[98694]: pgmap v13942: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 454 KiB/s wr, 30 op/s 2023-09-10T06:30:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:11 smithi098 ceph-mon[103124]: pgmap v13942: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 454 KiB/s wr, 30 op/s 2023-09-10T06:30:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:13 smithi170 ceph-mon[102832]: pgmap v13943: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 183 KiB/s wr, 30 op/s 2023-09-10T06:30:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25844.464037 secs 2023-09-10T06:30:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:13 smithi082 ceph-mon[98694]: pgmap v13943: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 183 KiB/s wr, 30 op/s 2023-09-10T06:30:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25844.464037 secs 2023-09-10T06:30:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:13 smithi098 ceph-mon[103124]: pgmap v13943: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 183 KiB/s wr, 30 op/s 2023-09-10T06:30:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25844.464037 secs 2023-09-10T06:30:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25845.471178 secs 2023-09-10T06:30:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25846.084068 secs 2023-09-10T06:30:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25845.471178 secs 2023-09-10T06:30:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25846.084068 secs 2023-09-10T06:30:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25845.471178 secs 2023-09-10T06:30:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25846.084068 secs 2023-09-10T06:30:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:15 smithi170 ceph-mon[102832]: pgmap v13944: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 183 KiB/s wr, 30 op/s 2023-09-10T06:30:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:15 smithi170 ceph-mon[102832]: osdmap e940: 12 total, 12 up, 12 in 2023-09-10T06:30:16.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:15 smithi082 ceph-mon[98694]: pgmap v13944: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 183 KiB/s wr, 30 op/s 2023-09-10T06:30:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:15 smithi082 ceph-mon[98694]: osdmap e940: 12 total, 12 up, 12 in 2023-09-10T06:30:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:15 smithi098 ceph-mon[103124]: pgmap v13944: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 183 KiB/s wr, 30 op/s 2023-09-10T06:30:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:15 smithi098 ceph-mon[103124]: osdmap e940: 12 total, 12 up, 12 in 2023-09-10T06:30:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:16 smithi170 ceph-mon[102832]: pgmap v13946: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 15 op/s 2023-09-10T06:30:17.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:16 smithi082 ceph-mon[98694]: pgmap v13946: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 15 op/s 2023-09-10T06:30:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:16 smithi098 ceph-mon[103124]: pgmap v13946: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 15 op/s 2023-09-10T06:30:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25849.464099 secs 2023-09-10T06:30:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:19 smithi170 ceph-mon[102832]: pgmap v13947: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T06:30:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25850.471238 secs 2023-09-10T06:30:19.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25849.464099 secs 2023-09-10T06:30:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:19 smithi082 ceph-mon[98694]: pgmap v13947: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T06:30:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25850.471238 secs 2023-09-10T06:30:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25849.464099 secs 2023-09-10T06:30:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:19 smithi098 ceph-mon[103124]: pgmap v13947: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T06:30:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25850.471238 secs 2023-09-10T06:30:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25851.084130 secs 2023-09-10T06:30:20.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25851.084130 secs 2023-09-10T06:30:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25851.084130 secs 2023-09-10T06:30:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:21 smithi170 ceph-mon[102832]: pgmap v13948: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T06:30:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:21 smithi082 ceph-mon[98694]: pgmap v13948: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T06:30:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:21 smithi098 ceph-mon[103124]: pgmap v13948: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-09-10T06:30:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:23 smithi170 ceph-mon[102832]: pgmap v13949: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T06:30:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25854.464202 secs 2023-09-10T06:30:24.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:23 smithi082 ceph-mon[98694]: pgmap v13949: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T06:30:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25854.464202 secs 2023-09-10T06:30:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:23 smithi098 ceph-mon[103124]: pgmap v13949: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T06:30:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25854.464202 secs 2023-09-10T06:30:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25855.471320 secs 2023-09-10T06:30:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25856.084193 secs 2023-09-10T06:30:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25855.471320 secs 2023-09-10T06:30:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25856.084193 secs 2023-09-10T06:30:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25855.471320 secs 2023-09-10T06:30:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25856.084193 secs 2023-09-10T06:30:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:25 smithi170 ceph-mon[102832]: pgmap v13950: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T06:30:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:25 smithi082 ceph-mon[98694]: pgmap v13950: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T06:30:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:25 smithi098 ceph-mon[103124]: pgmap v13950: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T06:30:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:26 smithi170 ceph-mon[102832]: pgmap v13951: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:30:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:26 smithi082 ceph-mon[98694]: pgmap v13951: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:30:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:26 smithi098 ceph-mon[103124]: pgmap v13951: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T06:30:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:29 smithi170 ceph-mon[102832]: pgmap v13952: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T06:30:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25860.471408 secs 2023-09-10T06:30:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:29 smithi082 ceph-mon[98694]: pgmap v13952: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T06:30:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25860.471408 secs 2023-09-10T06:30:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:29 smithi098 ceph-mon[103124]: pgmap v13952: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T06:30:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25860.471408 secs 2023-09-10T06:30:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25861.084260 secs 2023-09-10T06:30:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:30:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25861.084260 secs 2023-09-10T06:30:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:30:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25861.084260 secs 2023-09-10T06:30:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:30:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:31 smithi170 ceph-mon[102832]: pgmap v13953: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T06:30:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:30:32.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:31 smithi082 ceph-mon[98694]: pgmap v13953: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T06:30:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:30:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:31 smithi098 ceph-mon[103124]: pgmap v13953: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 10 op/s 2023-09-10T06:30:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:30:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:30:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:30:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:32 smithi170 ceph-mon[102832]: pgmap v13954: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 275 KiB/s wr, 15 op/s 2023-09-10T06:30:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:30:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:30:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:32 smithi082 ceph-mon[98694]: pgmap v13954: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 275 KiB/s wr, 15 op/s 2023-09-10T06:30:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:30:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:30:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:32 smithi098 ceph-mon[103124]: pgmap v13954: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 275 KiB/s wr, 15 op/s 2023-09-10T06:30:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25864.464343 secs 2023-09-10T06:30:34.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25864.464343 secs 2023-09-10T06:30:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25864.464343 secs 2023-09-10T06:30:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25865.471467 secs 2023-09-10T06:30:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25866.084326 secs 2023-09-10T06:30:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:34 smithi170 ceph-mon[102832]: pgmap v13955: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 10 op/s 2023-09-10T06:30:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25865.471467 secs 2023-09-10T06:30:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25866.084326 secs 2023-09-10T06:30:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:34 smithi082 ceph-mon[98694]: pgmap v13955: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 10 op/s 2023-09-10T06:30:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25865.471467 secs 2023-09-10T06:30:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25866.084326 secs 2023-09-10T06:30:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:34 smithi098 ceph-mon[103124]: pgmap v13955: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 10 op/s 2023-09-10T06:30:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:30:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:36 smithi170 ceph-mon[102832]: pgmap v13956: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 373 KiB/s wr, 16 op/s 2023-09-10T06:30:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:30:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:36 smithi082 ceph-mon[98694]: pgmap v13956: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 373 KiB/s wr, 16 op/s 2023-09-10T06:30:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:30:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:36 smithi098 ceph-mon[103124]: pgmap v13956: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 373 KiB/s wr, 16 op/s 2023-09-10T06:30:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:30:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:38.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:30:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:30:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:38 smithi170 ceph-mon[102832]: pgmap v13957: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 373 KiB/s wr, 11 op/s 2023-09-10T06:30:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:38 smithi082 ceph-mon[98694]: pgmap v13957: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 373 KiB/s wr, 11 op/s 2023-09-10T06:30:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:38 smithi098 ceph-mon[103124]: pgmap v13957: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 373 KiB/s wr, 11 op/s 2023-09-10T06:30:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25870.471493 secs 2023-09-10T06:30:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25871.084392 secs 2023-09-10T06:30:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:30:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25870.471493 secs 2023-09-10T06:30:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25871.084392 secs 2023-09-10T06:30:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:30:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25870.471493 secs 2023-09-10T06:30:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25871.084392 secs 2023-09-10T06:30:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:30:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:30:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:30:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:40 smithi170 ceph-mon[102832]: pgmap v13958: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 373 KiB/s wr, 11 op/s 2023-09-10T06:30:41.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:30:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:30:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:40 smithi082 ceph-mon[98694]: pgmap v13958: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 373 KiB/s wr, 11 op/s 2023-09-10T06:30:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:30:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:30:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:40 smithi098 ceph-mon[103124]: pgmap v13958: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 373 KiB/s wr, 11 op/s 2023-09-10T06:30:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:43 smithi170 ceph-mon[102832]: pgmap v13959: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 505 KiB/s wr, 18 op/s 2023-09-10T06:30:43.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:43 smithi082 ceph-mon[98694]: pgmap v13959: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 505 KiB/s wr, 18 op/s 2023-09-10T06:30:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:43 smithi098 ceph-mon[103124]: pgmap v13959: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 505 KiB/s wr, 18 op/s 2023-09-10T06:30:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25875.471553 secs 2023-09-10T06:30:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25875.471553 secs 2023-09-10T06:30:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25875.471553 secs 2023-09-10T06:30:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25876.084456 secs 2023-09-10T06:30:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:45 smithi170 ceph-mon[102832]: pgmap v13960: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 318 KiB/s wr, 12 op/s 2023-09-10T06:30:45.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25876.084456 secs 2023-09-10T06:30:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:45 smithi082 ceph-mon[98694]: pgmap v13960: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 318 KiB/s wr, 12 op/s 2023-09-10T06:30:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25876.084456 secs 2023-09-10T06:30:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:45 smithi098 ceph-mon[103124]: pgmap v13960: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 318 KiB/s wr, 12 op/s 2023-09-10T06:30:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:47 smithi170 ceph-mon[102832]: pgmap v13961: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 559 KiB/s wr, 19 op/s 2023-09-10T06:30:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:47 smithi082 ceph-mon[98694]: pgmap v13961: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 559 KiB/s wr, 19 op/s 2023-09-10T06:30:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:47 smithi098 ceph-mon[103124]: pgmap v13961: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 559 KiB/s wr, 19 op/s 2023-09-10T06:30:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:49 smithi170 ceph-mon[102832]: pgmap v13962: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 373 KiB/s wr, 13 op/s 2023-09-10T06:30:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25880.471613 secs 2023-09-10T06:30:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25881.084488 secs 2023-09-10T06:30:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:30:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:49 smithi082 ceph-mon[98694]: pgmap v13962: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 373 KiB/s wr, 13 op/s 2023-09-10T06:30:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25880.471613 secs 2023-09-10T06:30:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25881.084488 secs 2023-09-10T06:30:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:30:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:49 smithi098 ceph-mon[103124]: pgmap v13962: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 373 KiB/s wr, 13 op/s 2023-09-10T06:30:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25880.471613 secs 2023-09-10T06:30:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25881.084488 secs 2023-09-10T06:30:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:30:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:51 smithi170 ceph-mon[102832]: pgmap v13963: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 373 KiB/s wr, 13 op/s 2023-09-10T06:30:52.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:51 smithi082 ceph-mon[98694]: pgmap v13963: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 373 KiB/s wr, 13 op/s 2023-09-10T06:30:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:51 smithi098 ceph-mon[103124]: pgmap v13963: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 373 KiB/s wr, 13 op/s 2023-09-10T06:30:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:30:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:52 smithi170 ceph-mon[102832]: pgmap v13964: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 414 KiB/s wr, 19 op/s 2023-09-10T06:30:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:30:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:52 smithi082 ceph-mon[98694]: pgmap v13964: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 414 KiB/s wr, 19 op/s 2023-09-10T06:30:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:30:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:52 smithi098 ceph-mon[103124]: pgmap v13964: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 414 KiB/s wr, 19 op/s 2023-09-10T06:30:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:30:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:30:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:30:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25885.471693 secs 2023-09-10T06:30:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25886.084551 secs 2023-09-10T06:30:55.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:54 smithi170 ceph-mon[102832]: pgmap v13965: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 283 KiB/s wr, 12 op/s 2023-09-10T06:30:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25885.471693 secs 2023-09-10T06:30:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25886.084551 secs 2023-09-10T06:30:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:54 smithi082 ceph-mon[98694]: pgmap v13965: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 283 KiB/s wr, 12 op/s 2023-09-10T06:30:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25885.471693 secs 2023-09-10T06:30:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25886.084551 secs 2023-09-10T06:30:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:54 smithi098 ceph-mon[103124]: pgmap v13965: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 283 KiB/s wr, 12 op/s 2023-09-10T06:30:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:30:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:56 smithi170 ceph-mon[102832]: pgmap v13966: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 774 KiB/s wr, 19 op/s 2023-09-10T06:30:57.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:30:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:56 smithi082 ceph-mon[98694]: pgmap v13966: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 774 KiB/s wr, 19 op/s 2023-09-10T06:30:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:30:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:30:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:56 smithi098 ceph-mon[103124]: pgmap v13966: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 774 KiB/s wr, 19 op/s 2023-09-10T06:30:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25889.464781 secs 2023-09-10T06:30:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:59 smithi170 ceph-mon[102832]: pgmap v13967: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 533 KiB/s wr, 12 op/s 2023-09-10T06:30:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25890.471781 secs 2023-09-10T06:30:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:30:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:30:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25889.464781 secs 2023-09-10T06:30:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:59 smithi082 ceph-mon[98694]: pgmap v13967: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 533 KiB/s wr, 12 op/s 2023-09-10T06:30:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25890.471781 secs 2023-09-10T06:30:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:30:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:30:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25889.464781 secs 2023-09-10T06:30:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:59 smithi098 ceph-mon[103124]: pgmap v13967: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 533 KiB/s wr, 12 op/s 2023-09-10T06:30:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25890.471781 secs 2023-09-10T06:30:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:30:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:31:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25891.084619 secs 2023-09-10T06:31:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:31:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:31:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:31:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:31:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:31:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25891.084619 secs 2023-09-10T06:31:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:31:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:31:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:31:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:31:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:31:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25891.084619 secs 2023-09-10T06:31:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:31:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:31:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:31:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:31:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:31:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:01 smithi170 ceph-mon[102832]: pgmap v13968: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 533 KiB/s wr, 12 op/s 2023-09-10T06:31:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:01 smithi082 ceph-mon[98694]: pgmap v13968: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 533 KiB/s wr, 12 op/s 2023-09-10T06:31:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:01 smithi098 ceph-mon[103124]: pgmap v13968: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 533 KiB/s wr, 12 op/s 2023-09-10T06:31:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:03 smithi170 ceph-mon[102832]: pgmap v13969: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 534 KiB/s wr, 19 op/s 2023-09-10T06:31:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25894.464843 secs 2023-09-10T06:31:04.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:03 smithi082 ceph-mon[98694]: pgmap v13969: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 534 KiB/s wr, 19 op/s 2023-09-10T06:31:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25894.464843 secs 2023-09-10T06:31:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:03 smithi098 ceph-mon[103124]: pgmap v13969: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 534 KiB/s wr, 19 op/s 2023-09-10T06:31:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25894.464843 secs 2023-09-10T06:31:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25895.471868 secs 2023-09-10T06:31:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25896.084682 secs 2023-09-10T06:31:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:04 smithi170 ceph-mon[102832]: pgmap v13970: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 492 KiB/s wr, 13 op/s 2023-09-10T06:31:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25895.471868 secs 2023-09-10T06:31:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25896.084682 secs 2023-09-10T06:31:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:04 smithi082 ceph-mon[98694]: pgmap v13970: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 492 KiB/s wr, 13 op/s 2023-09-10T06:31:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25895.471868 secs 2023-09-10T06:31:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25896.084682 secs 2023-09-10T06:31:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:04 smithi098 ceph-mon[103124]: pgmap v13970: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 492 KiB/s wr, 13 op/s 2023-09-10T06:31:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:07 smithi170 ceph-mon[102832]: pgmap v13971: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 730 KiB/s wr, 33 op/s 2023-09-10T06:31:08.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:07 smithi082 ceph-mon[98694]: pgmap v13971: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 730 KiB/s wr, 33 op/s 2023-09-10T06:31:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:07 smithi098 ceph-mon[103124]: pgmap v13971: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 730 KiB/s wr, 33 op/s 2023-09-10T06:31:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25899.464932 secs 2023-09-10T06:31:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:08 smithi170 ceph-mon[102832]: osdmap e941: 12 total, 12 up, 12 in 2023-09-10T06:31:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:08 smithi170 ceph-mon[102832]: pgmap v13973: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 286 KiB/s wr, 31 op/s 2023-09-10T06:31:09.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25899.464932 secs 2023-09-10T06:31:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:08 smithi082 ceph-mon[98694]: osdmap e941: 12 total, 12 up, 12 in 2023-09-10T06:31:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:08 smithi082 ceph-mon[98694]: pgmap v13973: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 286 KiB/s wr, 31 op/s 2023-09-10T06:31:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25899.464932 secs 2023-09-10T06:31:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:08 smithi098 ceph-mon[103124]: osdmap e941: 12 total, 12 up, 12 in 2023-09-10T06:31:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:08 smithi098 ceph-mon[103124]: pgmap v13973: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 286 KiB/s wr, 31 op/s 2023-09-10T06:31:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25900.471930 secs 2023-09-10T06:31:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25901.084753 secs 2023-09-10T06:31:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25900.471930 secs 2023-09-10T06:31:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25901.084753 secs 2023-09-10T06:31:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25900.471930 secs 2023-09-10T06:31:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25901.084753 secs 2023-09-10T06:31:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:11 smithi170 ceph-mon[102832]: pgmap v13974: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 286 KiB/s wr, 31 op/s 2023-09-10T06:31:12.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:11 smithi082 ceph-mon[98694]: pgmap v13974: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 286 KiB/s wr, 31 op/s 2023-09-10T06:31:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:11 smithi098 ceph-mon[103124]: pgmap v13974: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 286 KiB/s wr, 31 op/s 2023-09-10T06:31:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:12 smithi170 ceph-mon[102832]: pgmap v13975: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 287 KiB/s wr, 30 op/s 2023-09-10T06:31:13.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:12 smithi082 ceph-mon[98694]: pgmap v13975: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 287 KiB/s wr, 30 op/s 2023-09-10T06:31:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:12 smithi098 ceph-mon[103124]: pgmap v13975: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 287 KiB/s wr, 30 op/s 2023-09-10T06:31:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25904.464991 secs 2023-09-10T06:31:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25904.464991 secs 2023-09-10T06:31:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25904.464991 secs 2023-09-10T06:31:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25905.471987 secs 2023-09-10T06:31:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25906.084816 secs 2023-09-10T06:31:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:14 smithi170 ceph-mon[102832]: pgmap v13976: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 287 KiB/s wr, 30 op/s 2023-09-10T06:31:15.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25905.471987 secs 2023-09-10T06:31:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25906.084816 secs 2023-09-10T06:31:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:14 smithi082 ceph-mon[98694]: pgmap v13976: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 287 KiB/s wr, 30 op/s 2023-09-10T06:31:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25905.471987 secs 2023-09-10T06:31:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25906.084816 secs 2023-09-10T06:31:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:14 smithi098 ceph-mon[103124]: pgmap v13976: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 287 KiB/s wr, 30 op/s 2023-09-10T06:31:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:16 smithi170 ceph-mon[102832]: osdmap e942: 12 total, 12 up, 12 in 2023-09-10T06:31:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:31:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:17.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:16 smithi082 ceph-mon[98694]: osdmap e942: 12 total, 12 up, 12 in 2023-09-10T06:31:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:31:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:16 smithi098 ceph-mon[103124]: osdmap e942: 12 total, 12 up, 12 in 2023-09-10T06:31:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:31:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:17 smithi170 ceph-mon[102832]: pgmap v13978: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2023-09-10T06:31:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:17 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:31:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:17 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:18.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:17 smithi082 ceph-mon[98694]: pgmap v13978: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2023-09-10T06:31:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:17 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:31:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:17 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:17 smithi098 ceph-mon[103124]: pgmap v13978: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.2 KiB/s wr, 15 op/s 2023-09-10T06:31:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:17 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:31:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:17 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25909.465074 secs 2023-09-10T06:31:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:18 smithi170 ceph-mon[102832]: pgmap v13979: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-09-10T06:31:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25909.465074 secs 2023-09-10T06:31:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:18 smithi082 ceph-mon[98694]: pgmap v13979: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-09-10T06:31:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25909.465074 secs 2023-09-10T06:31:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:18 smithi098 ceph-mon[103124]: pgmap v13979: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-09-10T06:31:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25910.472048 secs 2023-09-10T06:31:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25911.084882 secs 2023-09-10T06:31:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25910.472048 secs 2023-09-10T06:31:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25911.084882 secs 2023-09-10T06:31:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25910.472048 secs 2023-09-10T06:31:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25911.084882 secs 2023-09-10T06:31:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:20 smithi170 ceph-mon[102832]: pgmap v13980: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-09-10T06:31:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:20 smithi082 ceph-mon[98694]: pgmap v13980: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-09-10T06:31:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:20 smithi098 ceph-mon[103124]: pgmap v13980: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-09-10T06:31:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:23 smithi170 ceph-mon[102832]: pgmap v13981: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T06:31:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:23 smithi082 ceph-mon[98694]: pgmap v13981: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T06:31:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:23 smithi098 ceph-mon[103124]: pgmap v13981: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T06:31:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25914.465212 secs 2023-09-10T06:31:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25915.472105 secs 2023-09-10T06:31:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25914.465212 secs 2023-09-10T06:31:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25915.472105 secs 2023-09-10T06:31:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25914.465212 secs 2023-09-10T06:31:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25915.472105 secs 2023-09-10T06:31:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25916.084944 secs 2023-09-10T06:31:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:25 smithi170 ceph-mon[102832]: pgmap v13982: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T06:31:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25916.084944 secs 2023-09-10T06:31:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:25 smithi082 ceph-mon[98694]: pgmap v13982: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T06:31:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25916.084944 secs 2023-09-10T06:31:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:25 smithi098 ceph-mon[103124]: pgmap v13982: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T06:31:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:26 smithi170 ceph-mon[102832]: pgmap v13983: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T06:31:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:26 smithi082 ceph-mon[98694]: pgmap v13983: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T06:31:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:26 smithi098 ceph-mon[103124]: pgmap v13983: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T06:31:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25919.465275 secs 2023-09-10T06:31:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:29 smithi170 ceph-mon[102832]: pgmap v13984: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 9 op/s 2023-09-10T06:31:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25920.472184 secs 2023-09-10T06:31:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25919.465275 secs 2023-09-10T06:31:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:29 smithi082 ceph-mon[98694]: pgmap v13984: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 9 op/s 2023-09-10T06:31:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25920.472184 secs 2023-09-10T06:31:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25919.465275 secs 2023-09-10T06:31:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:29 smithi098 ceph-mon[103124]: pgmap v13984: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 9 op/s 2023-09-10T06:31:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25920.472184 secs 2023-09-10T06:31:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25921.085008 secs 2023-09-10T06:31:30.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25921.085008 secs 2023-09-10T06:31:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25921.085008 secs 2023-09-10T06:31:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:31 smithi170 ceph-mon[102832]: pgmap v13985: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 9 op/s 2023-09-10T06:31:31.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:31 smithi082 ceph-mon[98694]: pgmap v13985: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 9 op/s 2023-09-10T06:31:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:31 smithi098 ceph-mon[103124]: pgmap v13985: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 9 op/s 2023-09-10T06:31:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:33 smithi170 ceph-mon[102832]: pgmap v13986: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 295 KiB/s wr, 14 op/s 2023-09-10T06:31:33.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:33 smithi082 ceph-mon[98694]: pgmap v13986: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 295 KiB/s wr, 14 op/s 2023-09-10T06:31:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:33 smithi098 ceph-mon[103124]: pgmap v13986: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 295 KiB/s wr, 14 op/s 2023-09-10T06:31:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:31:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25925.472244 secs 2023-09-10T06:31:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:31:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25925.472244 secs 2023-09-10T06:31:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:31:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25925.472244 secs 2023-09-10T06:31:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25926.085073 secs 2023-09-10T06:31:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:35 smithi170 ceph-mon[102832]: pgmap v13987: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 207 KiB/s wr, 9 op/s 2023-09-10T06:31:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:31:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25926.085073 secs 2023-09-10T06:31:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:35 smithi082 ceph-mon[98694]: pgmap v13987: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 207 KiB/s wr, 9 op/s 2023-09-10T06:31:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:31:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25926.085073 secs 2023-09-10T06:31:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:35 smithi098 ceph-mon[103124]: pgmap v13987: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 207 KiB/s wr, 9 op/s 2023-09-10T06:31:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:31:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:31:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:31:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:31:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:31:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:31:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:31:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:37 smithi170 ceph-mon[102832]: pgmap v13988: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 464 KiB/s wr, 16 op/s 2023-09-10T06:31:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:37 smithi082 ceph-mon[98694]: pgmap v13988: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 464 KiB/s wr, 16 op/s 2023-09-10T06:31:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:37 smithi098 ceph-mon[103124]: pgmap v13988: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 464 KiB/s wr, 16 op/s 2023-09-10T06:31:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:39 smithi170 ceph-mon[102832]: pgmap v13989: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 464 KiB/s wr, 12 op/s 2023-09-10T06:31:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25930.472303 secs 2023-09-10T06:31:39.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:39 smithi082 ceph-mon[98694]: pgmap v13989: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 464 KiB/s wr, 12 op/s 2023-09-10T06:31:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25930.472303 secs 2023-09-10T06:31:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:39 smithi098 ceph-mon[103124]: pgmap v13989: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 464 KiB/s wr, 12 op/s 2023-09-10T06:31:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25930.472303 secs 2023-09-10T06:31:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25931.085162 secs 2023-09-10T06:31:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25931.085162 secs 2023-09-10T06:31:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25931.085162 secs 2023-09-10T06:31:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:41 smithi170 ceph-mon[102832]: pgmap v13990: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 464 KiB/s wr, 12 op/s 2023-09-10T06:31:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:41 smithi082 ceph-mon[98694]: pgmap v13990: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 464 KiB/s wr, 12 op/s 2023-09-10T06:31:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:41 smithi098 ceph-mon[103124]: pgmap v13990: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 464 KiB/s wr, 12 op/s 2023-09-10T06:31:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:43 smithi170 ceph-mon[102832]: pgmap v13991: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 480 KiB/s wr, 17 op/s 2023-09-10T06:31:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:43 smithi082 ceph-mon[98694]: pgmap v13991: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 480 KiB/s wr, 17 op/s 2023-09-10T06:31:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:43 smithi098 ceph-mon[103124]: pgmap v13991: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 480 KiB/s wr, 17 op/s 2023-09-10T06:31:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:31:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25935.472366 secs 2023-09-10T06:31:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:31:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25935.472366 secs 2023-09-10T06:31:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:31:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25935.472366 secs 2023-09-10T06:31:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25936.085228 secs 2023-09-10T06:31:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:45 smithi170 ceph-mon[102832]: pgmap v13992: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 273 KiB/s wr, 12 op/s 2023-09-10T06:31:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25936.085228 secs 2023-09-10T06:31:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:45 smithi082 ceph-mon[98694]: pgmap v13992: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 273 KiB/s wr, 12 op/s 2023-09-10T06:31:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25936.085228 secs 2023-09-10T06:31:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:45 smithi098 ceph-mon[103124]: pgmap v13992: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 273 KiB/s wr, 12 op/s 2023-09-10T06:31:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:31:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:31:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:31:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:47 smithi170 ceph-mon[102832]: pgmap v13993: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 747 KiB/s wr, 19 op/s 2023-09-10T06:31:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:47 smithi082 ceph-mon[98694]: pgmap v13993: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 747 KiB/s wr, 19 op/s 2023-09-10T06:31:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:47 smithi098 ceph-mon[103124]: pgmap v13993: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 747 KiB/s wr, 19 op/s 2023-09-10T06:31:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:49 smithi170 ceph-mon[102832]: pgmap v13994: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 490 KiB/s wr, 12 op/s 2023-09-10T06:31:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25940.472426 secs 2023-09-10T06:31:49.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:49 smithi082 ceph-mon[98694]: pgmap v13994: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 490 KiB/s wr, 12 op/s 2023-09-10T06:31:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25940.472426 secs 2023-09-10T06:31:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:49 smithi098 ceph-mon[103124]: pgmap v13994: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 490 KiB/s wr, 12 op/s 2023-09-10T06:31:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25940.472426 secs 2023-09-10T06:31:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25941.085292 secs 2023-09-10T06:31:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25941.085292 secs 2023-09-10T06:31:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25941.085292 secs 2023-09-10T06:31:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:51 smithi170 ceph-mon[102832]: pgmap v13995: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 490 KiB/s wr, 12 op/s 2023-09-10T06:31:51.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:51 smithi082 ceph-mon[98694]: pgmap v13995: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 490 KiB/s wr, 12 op/s 2023-09-10T06:31:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:51 smithi098 ceph-mon[103124]: pgmap v13995: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 490 KiB/s wr, 12 op/s 2023-09-10T06:31:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:31:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:52 smithi170 ceph-mon[102832]: pgmap v13996: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 773 KiB/s wr, 18 op/s 2023-09-10T06:31:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:31:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:52 smithi082 ceph-mon[98694]: pgmap v13996: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 773 KiB/s wr, 18 op/s 2023-09-10T06:31:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:31:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:52 smithi098 ceph-mon[103124]: pgmap v13996: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 773 KiB/s wr, 18 op/s 2023-09-10T06:31:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:31:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:31:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:31:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25945.472490 secs 2023-09-10T06:31:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25946.085356 secs 2023-09-10T06:31:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:54 smithi170 ceph-mon[102832]: pgmap v13997: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 757 KiB/s wr, 12 op/s 2023-09-10T06:31:55.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25945.472490 secs 2023-09-10T06:31:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25946.085356 secs 2023-09-10T06:31:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:54 smithi082 ceph-mon[98694]: pgmap v13997: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 757 KiB/s wr, 12 op/s 2023-09-10T06:31:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25945.472490 secs 2023-09-10T06:31:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25946.085356 secs 2023-09-10T06:31:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:54 smithi098 ceph-mon[103124]: pgmap v13997: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 757 KiB/s wr, 12 op/s 2023-09-10T06:31:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:56 smithi170 ceph-mon[102832]: pgmap v13998: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 847 KiB/s wr, 18 op/s 2023-09-10T06:31:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:56 smithi082 ceph-mon[98694]: pgmap v13998: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 847 KiB/s wr, 18 op/s 2023-09-10T06:31:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:56 smithi098 ceph-mon[103124]: pgmap v13998: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 847 KiB/s wr, 18 op/s 2023-09-10T06:31:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:31:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:58.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:31:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:31:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:31:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25949.465702 secs 2023-09-10T06:31:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:58 smithi170 ceph-mon[102832]: pgmap v13999: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 372 KiB/s wr, 11 op/s 2023-09-10T06:31:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25949.465702 secs 2023-09-10T06:31:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:58 smithi082 ceph-mon[98694]: pgmap v13999: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 372 KiB/s wr, 11 op/s 2023-09-10T06:31:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25949.465702 secs 2023-09-10T06:31:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:58 smithi098 ceph-mon[103124]: pgmap v13999: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 372 KiB/s wr, 11 op/s 2023-09-10T06:32:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25950.472548 secs 2023-09-10T06:32:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25951.085421 secs 2023-09-10T06:32:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:31:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:32:00.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25950.472548 secs 2023-09-10T06:32:00.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25951.085421 secs 2023-09-10T06:32:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:31:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:32:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25950.472548 secs 2023-09-10T06:32:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25951.085421 secs 2023-09-10T06:32:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:31:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:32:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:32:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:32:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:00 smithi170 ceph-mon[102832]: pgmap v14000: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 372 KiB/s wr, 11 op/s 2023-09-10T06:32:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:32:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:32:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:32:01.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:32:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:32:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:00 smithi082 ceph-mon[98694]: pgmap v14000: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 372 KiB/s wr, 11 op/s 2023-09-10T06:32:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:32:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:32:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:32:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:32:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:32:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:00 smithi098 ceph-mon[103124]: pgmap v14000: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 372 KiB/s wr, 11 op/s 2023-09-10T06:32:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:32:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:32:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:32:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:03 smithi170 ceph-mon[102832]: pgmap v14001: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 561 KiB/s wr, 17 op/s 2023-09-10T06:32:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:03 smithi082 ceph-mon[98694]: pgmap v14001: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 561 KiB/s wr, 17 op/s 2023-09-10T06:32:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:03 smithi098 ceph-mon[103124]: pgmap v14001: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 561 KiB/s wr, 17 op/s 2023-09-10T06:32:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25954.465763 secs 2023-09-10T06:32:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25955.472618 secs 2023-09-10T06:32:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25954.465763 secs 2023-09-10T06:32:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25955.472618 secs 2023-09-10T06:32:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25954.465763 secs 2023-09-10T06:32:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25955.472618 secs 2023-09-10T06:32:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25956.085487 secs 2023-09-10T06:32:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:05 smithi170 ceph-mon[102832]: pgmap v14002: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 278 KiB/s wr, 11 op/s 2023-09-10T06:32:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25956.085487 secs 2023-09-10T06:32:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:05 smithi082 ceph-mon[98694]: pgmap v14002: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 278 KiB/s wr, 11 op/s 2023-09-10T06:32:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25956.085487 secs 2023-09-10T06:32:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:05 smithi098 ceph-mon[103124]: pgmap v14002: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 278 KiB/s wr, 11 op/s 2023-09-10T06:32:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:06 smithi170 ceph-mon[102832]: pgmap v14003: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 411 KiB/s wr, 31 op/s 2023-09-10T06:32:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:06 smithi082 ceph-mon[98694]: pgmap v14003: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 411 KiB/s wr, 31 op/s 2023-09-10T06:32:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:06 smithi098 ceph-mon[103124]: pgmap v14003: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 411 KiB/s wr, 31 op/s 2023-09-10T06:32:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25959.465784 secs 2023-09-10T06:32:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:08 smithi170 ceph-mon[102832]: osdmap e943: 12 total, 12 up, 12 in 2023-09-10T06:32:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:08 smithi170 ceph-mon[102832]: pgmap v14005: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 386 KiB/s wr, 31 op/s 2023-09-10T06:32:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25959.465784 secs 2023-09-10T06:32:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:08 smithi082 ceph-mon[98694]: osdmap e943: 12 total, 12 up, 12 in 2023-09-10T06:32:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:08 smithi082 ceph-mon[98694]: pgmap v14005: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 386 KiB/s wr, 31 op/s 2023-09-10T06:32:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25959.465784 secs 2023-09-10T06:32:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:08 smithi098 ceph-mon[103124]: osdmap e943: 12 total, 12 up, 12 in 2023-09-10T06:32:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:08 smithi098 ceph-mon[103124]: pgmap v14005: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 386 KiB/s wr, 31 op/s 2023-09-10T06:32:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25960.472676 secs 2023-09-10T06:32:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25961.085553 secs 2023-09-10T06:32:10.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25960.472676 secs 2023-09-10T06:32:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25961.085553 secs 2023-09-10T06:32:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25960.472676 secs 2023-09-10T06:32:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25961.085553 secs 2023-09-10T06:32:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:10 smithi170 ceph-mon[102832]: pgmap v14006: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 386 KiB/s wr, 31 op/s 2023-09-10T06:32:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:10 smithi082 ceph-mon[98694]: pgmap v14006: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 386 KiB/s wr, 31 op/s 2023-09-10T06:32:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:10 smithi098 ceph-mon[103124]: pgmap v14006: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 386 KiB/s wr, 31 op/s 2023-09-10T06:32:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:13 smithi170 ceph-mon[102832]: pgmap v14007: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 184 KiB/s wr, 29 op/s 2023-09-10T06:32:13.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:13 smithi082 ceph-mon[98694]: pgmap v14007: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 184 KiB/s wr, 29 op/s 2023-09-10T06:32:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:13 smithi098 ceph-mon[103124]: pgmap v14007: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 184 KiB/s wr, 29 op/s 2023-09-10T06:32:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25964.465849 secs 2023-09-10T06:32:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25965.472734 secs 2023-09-10T06:32:14.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25964.465849 secs 2023-09-10T06:32:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25965.472734 secs 2023-09-10T06:32:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25964.465849 secs 2023-09-10T06:32:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25965.472734 secs 2023-09-10T06:32:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25966.085615 secs 2023-09-10T06:32:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:15 smithi170 ceph-mon[102832]: pgmap v14008: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 184 KiB/s wr, 29 op/s 2023-09-10T06:32:15.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25966.085615 secs 2023-09-10T06:32:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:15 smithi082 ceph-mon[98694]: pgmap v14008: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 184 KiB/s wr, 29 op/s 2023-09-10T06:32:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25966.085615 secs 2023-09-10T06:32:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:15 smithi098 ceph-mon[103124]: pgmap v14008: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 184 KiB/s wr, 29 op/s 2023-09-10T06:32:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:16 smithi170 ceph-mon[102832]: osdmap e944: 12 total, 12 up, 12 in 2023-09-10T06:32:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:16 smithi170 ceph-mon[102832]: pgmap v14010: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 14 op/s 2023-09-10T06:32:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:16 smithi082 ceph-mon[98694]: osdmap e944: 12 total, 12 up, 12 in 2023-09-10T06:32:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:16 smithi082 ceph-mon[98694]: pgmap v14010: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 14 op/s 2023-09-10T06:32:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:16 smithi098 ceph-mon[103124]: osdmap e944: 12 total, 12 up, 12 in 2023-09-10T06:32:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:16 smithi098 ceph-mon[103124]: pgmap v14010: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 14 op/s 2023-09-10T06:32:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25969.465910 secs 2023-09-10T06:32:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:19 smithi170 ceph-mon[102832]: pgmap v14011: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T06:32:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25970.472790 secs 2023-09-10T06:32:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25969.465910 secs 2023-09-10T06:32:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:19 smithi082 ceph-mon[98694]: pgmap v14011: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T06:32:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25970.472790 secs 2023-09-10T06:32:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25969.465910 secs 2023-09-10T06:32:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:19 smithi098 ceph-mon[103124]: pgmap v14011: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T06:32:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25970.472790 secs 2023-09-10T06:32:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25971.085681 secs 2023-09-10T06:32:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25971.085681 secs 2023-09-10T06:32:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25971.085681 secs 2023-09-10T06:32:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:21 smithi170 ceph-mon[102832]: pgmap v14012: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T06:32:21.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:21 smithi082 ceph-mon[98694]: pgmap v14012: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T06:32:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:21 smithi098 ceph-mon[103124]: pgmap v14012: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T06:32:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:23 smithi170 ceph-mon[102832]: pgmap v14013: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T06:32:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:23 smithi082 ceph-mon[98694]: pgmap v14013: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T06:32:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:23 smithi098 ceph-mon[103124]: pgmap v14013: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T06:32:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25974.466070 secs 2023-09-10T06:32:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25975.472850 secs 2023-09-10T06:32:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25974.466070 secs 2023-09-10T06:32:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25975.472850 secs 2023-09-10T06:32:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25974.466070 secs 2023-09-10T06:32:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25975.472850 secs 2023-09-10T06:32:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25976.085742 secs 2023-09-10T06:32:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:25 smithi170 ceph-mon[102832]: pgmap v14014: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T06:32:25.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25976.085742 secs 2023-09-10T06:32:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:25 smithi082 ceph-mon[98694]: pgmap v14014: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T06:32:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25976.085742 secs 2023-09-10T06:32:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:25 smithi098 ceph-mon[103124]: pgmap v14014: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T06:32:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:26 smithi170 ceph-mon[102832]: pgmap v14015: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 205 KiB/s wr, 12 op/s 2023-09-10T06:32:27.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:26 smithi082 ceph-mon[98694]: pgmap v14015: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 205 KiB/s wr, 12 op/s 2023-09-10T06:32:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:26 smithi098 ceph-mon[103124]: pgmap v14015: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 205 KiB/s wr, 12 op/s 2023-09-10T06:32:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:29 smithi170 ceph-mon[102832]: pgmap v14016: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 173 KiB/s wr, 10 op/s 2023-09-10T06:32:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25980.472908 secs 2023-09-10T06:32:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:29 smithi082 ceph-mon[98694]: pgmap v14016: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 173 KiB/s wr, 10 op/s 2023-09-10T06:32:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25980.472908 secs 2023-09-10T06:32:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:29 smithi098 ceph-mon[103124]: pgmap v14016: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 173 KiB/s wr, 10 op/s 2023-09-10T06:32:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25980.472908 secs 2023-09-10T06:32:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25981.085810 secs 2023-09-10T06:32:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:32:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25981.085810 secs 2023-09-10T06:32:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:32:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25981.085810 secs 2023-09-10T06:32:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:32:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:31 smithi170 ceph-mon[102832]: pgmap v14017: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 173 KiB/s wr, 10 op/s 2023-09-10T06:32:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:32:31.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:31 smithi082 ceph-mon[98694]: pgmap v14017: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 173 KiB/s wr, 10 op/s 2023-09-10T06:32:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:32:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:31 smithi098 ceph-mon[103124]: pgmap v14017: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 173 KiB/s wr, 10 op/s 2023-09-10T06:32:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:32:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:32:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:32:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:32.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:32:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:32:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:32:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:32:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:33 smithi170 ceph-mon[102832]: pgmap v14018: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 345 KiB/s wr, 15 op/s 2023-09-10T06:32:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:33 smithi082 ceph-mon[98694]: pgmap v14018: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 345 KiB/s wr, 15 op/s 2023-09-10T06:32:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:33 smithi098 ceph-mon[103124]: pgmap v14018: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 345 KiB/s wr, 15 op/s 2023-09-10T06:32:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25984.466232 secs 2023-09-10T06:32:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25985.472978 secs 2023-09-10T06:32:34.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25984.466232 secs 2023-09-10T06:32:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25985.472978 secs 2023-09-10T06:32:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25984.466232 secs 2023-09-10T06:32:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25985.472978 secs 2023-09-10T06:32:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25986.085883 secs 2023-09-10T06:32:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:35 smithi170 ceph-mon[102832]: pgmap v14019: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 257 KiB/s wr, 10 op/s 2023-09-10T06:32:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25986.085883 secs 2023-09-10T06:32:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:35 smithi082 ceph-mon[98694]: pgmap v14019: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 257 KiB/s wr, 10 op/s 2023-09-10T06:32:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25986.085883 secs 2023-09-10T06:32:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:35 smithi098 ceph-mon[103124]: pgmap v14019: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 257 KiB/s wr, 10 op/s 2023-09-10T06:32:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:32:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:36.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:32:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:32:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:37 smithi170 ceph-mon[102832]: pgmap v14020: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 17 op/s 2023-09-10T06:32:37.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:37 smithi082 ceph-mon[98694]: pgmap v14020: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 17 op/s 2023-09-10T06:32:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:37 smithi098 ceph-mon[103124]: pgmap v14020: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 17 op/s 2023-09-10T06:32:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:32:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:32:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:32:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:39 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 25989.466343 secs 2023-09-10T06:32:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:39 smithi170 ceph-mon[102832]: pgmap v14021: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 359 KiB/s wr, 12 op/s 2023-09-10T06:32:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25990.473036 secs 2023-09-10T06:32:39.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:39 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 25989.466343 secs 2023-09-10T06:32:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:39 smithi082 ceph-mon[98694]: pgmap v14021: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 359 KiB/s wr, 12 op/s 2023-09-10T06:32:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25990.473036 secs 2023-09-10T06:32:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:39 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 25989.466343 secs 2023-09-10T06:32:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:39 smithi098 ceph-mon[103124]: pgmap v14021: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 359 KiB/s wr, 12 op/s 2023-09-10T06:32:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25990.473036 secs 2023-09-10T06:32:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25991.085970 secs 2023-09-10T06:32:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:32:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25991.085970 secs 2023-09-10T06:32:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:32:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25991.085970 secs 2023-09-10T06:32:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:32:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:41 smithi170 ceph-mon[102832]: pgmap v14022: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 359 KiB/s wr, 12 op/s 2023-09-10T06:32:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:41 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:32:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:41 smithi082 ceph-mon[98694]: pgmap v14022: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 359 KiB/s wr, 12 op/s 2023-09-10T06:32:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:41 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:32:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:41 smithi098 ceph-mon[103124]: pgmap v14022: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 359 KiB/s wr, 12 op/s 2023-09-10T06:32:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:41 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:32:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:43 smithi170 ceph-mon[102832]: pgmap v14023: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 647 KiB/s wr, 18 op/s 2023-09-10T06:32:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:32:43.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:43 smithi082 ceph-mon[98694]: pgmap v14023: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 647 KiB/s wr, 18 op/s 2023-09-10T06:32:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:32:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:43 smithi098 ceph-mon[103124]: pgmap v14023: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 647 KiB/s wr, 18 op/s 2023-09-10T06:32:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:32:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:32:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25995.473121 secs 2023-09-10T06:32:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:32:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25995.473121 secs 2023-09-10T06:32:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:32:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25995.473121 secs 2023-09-10T06:32:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 25996.086032 secs 2023-09-10T06:32:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:45 smithi170 ceph-mon[102832]: pgmap v14024: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 475 KiB/s wr, 13 op/s 2023-09-10T06:32:45.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 25996.086032 secs 2023-09-10T06:32:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:45 smithi082 ceph-mon[98694]: pgmap v14024: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 475 KiB/s wr, 13 op/s 2023-09-10T06:32:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 25996.086032 secs 2023-09-10T06:32:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:45 smithi098 ceph-mon[103124]: pgmap v14024: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 475 KiB/s wr, 13 op/s 2023-09-10T06:32:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:32:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:46 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:32:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:32:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:46 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:32:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:32:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:46 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:32:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:47 smithi170 ceph-mon[102832]: pgmap v14025: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 783 KiB/s wr, 19 op/s 2023-09-10T06:32:47.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:47 smithi082 ceph-mon[98694]: pgmap v14025: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 783 KiB/s wr, 19 op/s 2023-09-10T06:32:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:47 smithi098 ceph-mon[103124]: pgmap v14025: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 783 KiB/s wr, 19 op/s 2023-09-10T06:32:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:49 smithi170 ceph-mon[102832]: pgmap v14026: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 596 KiB/s wr, 12 op/s 2023-09-10T06:32:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26000.473260 secs 2023-09-10T06:32:49.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:49 smithi082 ceph-mon[98694]: pgmap v14026: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 596 KiB/s wr, 12 op/s 2023-09-10T06:32:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26000.473260 secs 2023-09-10T06:32:49.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:49 smithi098 ceph-mon[103124]: pgmap v14026: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 596 KiB/s wr, 12 op/s 2023-09-10T06:32:49.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26000.473260 secs 2023-09-10T06:32:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26001.086098 secs 2023-09-10T06:32:50.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26001.086098 secs 2023-09-10T06:32:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26001.086098 secs 2023-09-10T06:32:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:51 smithi170 ceph-mon[102832]: pgmap v14027: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 596 KiB/s wr, 12 op/s 2023-09-10T06:32:51.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:51 smithi082 ceph-mon[98694]: pgmap v14027: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 596 KiB/s wr, 12 op/s 2023-09-10T06:32:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:51 smithi098 ceph-mon[103124]: pgmap v14027: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 596 KiB/s wr, 12 op/s 2023-09-10T06:32:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:53 smithi170 ceph-mon[102832]: pgmap v14028: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 660 KiB/s wr, 19 op/s 2023-09-10T06:32:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:53 smithi082 ceph-mon[98694]: pgmap v14028: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 660 KiB/s wr, 19 op/s 2023-09-10T06:32:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:53 smithi098 ceph-mon[103124]: pgmap v14028: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 660 KiB/s wr, 19 op/s 2023-09-10T06:32:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26004.466578 secs 2023-09-10T06:32:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:32:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26005.473316 secs 2023-09-10T06:32:54.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26004.466578 secs 2023-09-10T06:32:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:32:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26005.473316 secs 2023-09-10T06:32:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26004.466578 secs 2023-09-10T06:32:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:32:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26005.473316 secs 2023-09-10T06:32:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26006.086185 secs 2023-09-10T06:32:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:55 smithi170 ceph-mon[102832]: pgmap v14029: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 373 KiB/s wr, 12 op/s 2023-09-10T06:32:55.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26006.086185 secs 2023-09-10T06:32:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:55 smithi082 ceph-mon[98694]: pgmap v14029: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 373 KiB/s wr, 12 op/s 2023-09-10T06:32:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26006.086185 secs 2023-09-10T06:32:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:55 smithi098 ceph-mon[103124]: pgmap v14029: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 373 KiB/s wr, 12 op/s 2023-09-10T06:32:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:32:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:32:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:56.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:32:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:32:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:32:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:32:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:57.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:57 smithi082 ceph-mon[98694]: pgmap v14030: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 684 KiB/s wr, 18 op/s 2023-09-10T06:32:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:57 smithi098 ceph-mon[103124]: pgmap v14030: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 684 KiB/s wr, 18 op/s 2023-09-10T06:32:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:57 smithi170 ceph-mon[102832]: pgmap v14030: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 684 KiB/s wr, 18 op/s 2023-09-10T06:32:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:32:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:32:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:32:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:32:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26009.466666 secs 2023-09-10T06:32:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:59 smithi082 ceph-mon[98694]: pgmap v14031: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 376 KiB/s wr, 12 op/s 2023-09-10T06:32:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:32:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26010.473398 secs 2023-09-10T06:32:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26009.466666 secs 2023-09-10T06:32:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:59 smithi098 ceph-mon[103124]: pgmap v14031: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 376 KiB/s wr, 12 op/s 2023-09-10T06:32:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:32:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26010.473398 secs 2023-09-10T06:32:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26009.466666 secs 2023-09-10T06:32:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:59 smithi170 ceph-mon[102832]: pgmap v14031: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 376 KiB/s wr, 12 op/s 2023-09-10T06:32:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:32:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26010.473398 secs 2023-09-10T06:33:00.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26011.086252 secs 2023-09-10T06:33:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:33:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:33:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26011.086252 secs 2023-09-10T06:33:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:33:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:33:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26011.086252 secs 2023-09-10T06:33:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:33:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:33:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:01 smithi082 ceph-mon[98694]: pgmap v14032: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 376 KiB/s wr, 12 op/s 2023-09-10T06:33:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:33:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:33:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:33:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:33:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:01 smithi098 ceph-mon[103124]: pgmap v14032: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 376 KiB/s wr, 12 op/s 2023-09-10T06:33:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:33:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:33:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:33:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:33:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:01 smithi170 ceph-mon[102832]: pgmap v14032: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 376 KiB/s wr, 12 op/s 2023-09-10T06:33:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:33:01.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:33:01.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:33:01.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:33:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:03 smithi082 ceph-mon[98694]: pgmap v14033: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 610 KiB/s wr, 32 op/s 2023-09-10T06:33:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:03 smithi098 ceph-mon[103124]: pgmap v14033: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 610 KiB/s wr, 32 op/s 2023-09-10T06:33:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:03 smithi170 ceph-mon[102832]: pgmap v14033: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 610 KiB/s wr, 32 op/s 2023-09-10T06:33:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26014.466751 secs 2023-09-10T06:33:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26015.473459 secs 2023-09-10T06:33:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26014.466751 secs 2023-09-10T06:33:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26015.473459 secs 2023-09-10T06:33:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26014.466751 secs 2023-09-10T06:33:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26015.473459 secs 2023-09-10T06:33:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26016.086315 secs 2023-09-10T06:33:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:05 smithi098 ceph-mon[103124]: pgmap v14034: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 545 KiB/s wr, 25 op/s 2023-09-10T06:33:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26016.086315 secs 2023-09-10T06:33:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:05 smithi170 ceph-mon[102832]: pgmap v14034: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 545 KiB/s wr, 25 op/s 2023-09-10T06:33:05.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26016.086315 secs 2023-09-10T06:33:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:05 smithi082 ceph-mon[98694]: pgmap v14034: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 545 KiB/s wr, 25 op/s 2023-09-10T06:33:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:06 smithi170 ceph-mon[102832]: pgmap v14035: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 634 KiB/s wr, 31 op/s 2023-09-10T06:33:07.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:06 smithi082 ceph-mon[98694]: pgmap v14035: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 634 KiB/s wr, 31 op/s 2023-09-10T06:33:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:06 smithi098 ceph-mon[103124]: pgmap v14035: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 634 KiB/s wr, 31 op/s 2023-09-10T06:33:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26019.466784 secs 2023-09-10T06:33:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:08 smithi170 ceph-mon[102832]: osdmap e945: 12 total, 12 up, 12 in 2023-09-10T06:33:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:08 smithi170 ceph-mon[102832]: pgmap v14037: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 387 KiB/s wr, 30 op/s 2023-09-10T06:33:09.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26019.466784 secs 2023-09-10T06:33:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:08 smithi082 ceph-mon[98694]: osdmap e945: 12 total, 12 up, 12 in 2023-09-10T06:33:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:08 smithi082 ceph-mon[98694]: pgmap v14037: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 387 KiB/s wr, 30 op/s 2023-09-10T06:33:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26019.466784 secs 2023-09-10T06:33:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:08 smithi098 ceph-mon[103124]: osdmap e945: 12 total, 12 up, 12 in 2023-09-10T06:33:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:08 smithi098 ceph-mon[103124]: pgmap v14037: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 387 KiB/s wr, 30 op/s 2023-09-10T06:33:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26020.473518 secs 2023-09-10T06:33:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26021.086383 secs 2023-09-10T06:33:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26020.473518 secs 2023-09-10T06:33:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26021.086383 secs 2023-09-10T06:33:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26020.473518 secs 2023-09-10T06:33:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26021.086383 secs 2023-09-10T06:33:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:10 smithi170 ceph-mon[102832]: pgmap v14038: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 387 KiB/s wr, 30 op/s 2023-09-10T06:33:11.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:10 smithi082 ceph-mon[98694]: pgmap v14038: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 387 KiB/s wr, 30 op/s 2023-09-10T06:33:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:10 smithi098 ceph-mon[103124]: pgmap v14038: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 387 KiB/s wr, 30 op/s 2023-09-10T06:33:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:13 smithi170 ceph-mon[102832]: pgmap v14039: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T06:33:13.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:13 smithi082 ceph-mon[98694]: pgmap v14039: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T06:33:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:13 smithi098 ceph-mon[103124]: pgmap v14039: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T06:33:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26024.466847 secs 2023-09-10T06:33:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26025.473577 secs 2023-09-10T06:33:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26024.466847 secs 2023-09-10T06:33:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26025.473577 secs 2023-09-10T06:33:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26024.466847 secs 2023-09-10T06:33:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26025.473577 secs 2023-09-10T06:33:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26026.086447 secs 2023-09-10T06:33:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:15 smithi170 ceph-mon[102832]: pgmap v14040: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T06:33:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26026.086447 secs 2023-09-10T06:33:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:15 smithi082 ceph-mon[98694]: pgmap v14040: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T06:33:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26026.086447 secs 2023-09-10T06:33:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:15 smithi098 ceph-mon[103124]: pgmap v14040: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T06:33:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:16 smithi170 ceph-mon[102832]: osdmap e946: 12 total, 12 up, 12 in 2023-09-10T06:33:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:16 smithi170 ceph-mon[102832]: pgmap v14042: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 15 op/s 2023-09-10T06:33:17.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:16 smithi082 ceph-mon[98694]: osdmap e946: 12 total, 12 up, 12 in 2023-09-10T06:33:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:16 smithi082 ceph-mon[98694]: pgmap v14042: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 15 op/s 2023-09-10T06:33:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:16 smithi098 ceph-mon[103124]: osdmap e946: 12 total, 12 up, 12 in 2023-09-10T06:33:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:16 smithi098 ceph-mon[103124]: pgmap v14042: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 15 op/s 2023-09-10T06:33:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26029.466904 secs 2023-09-10T06:33:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:19 smithi170 ceph-mon[102832]: pgmap v14043: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 76 KiB/s wr, 12 op/s 2023-09-10T06:33:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26030.473660 secs 2023-09-10T06:33:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26029.466904 secs 2023-09-10T06:33:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:19 smithi082 ceph-mon[98694]: pgmap v14043: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 76 KiB/s wr, 12 op/s 2023-09-10T06:33:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26030.473660 secs 2023-09-10T06:33:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26029.466904 secs 2023-09-10T06:33:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:19 smithi098 ceph-mon[103124]: pgmap v14043: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 76 KiB/s wr, 12 op/s 2023-09-10T06:33:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26030.473660 secs 2023-09-10T06:33:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26031.086492 secs 2023-09-10T06:33:20.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26031.086492 secs 2023-09-10T06:33:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26031.086492 secs 2023-09-10T06:33:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:21 smithi170 ceph-mon[102832]: pgmap v14044: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 76 KiB/s wr, 12 op/s 2023-09-10T06:33:21.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:21 smithi082 ceph-mon[98694]: pgmap v14044: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 76 KiB/s wr, 12 op/s 2023-09-10T06:33:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:21 smithi098 ceph-mon[103124]: pgmap v14044: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 76 KiB/s wr, 12 op/s 2023-09-10T06:33:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:23 smithi170 ceph-mon[102832]: pgmap v14045: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 12 op/s 2023-09-10T06:33:23.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:23 smithi082 ceph-mon[98694]: pgmap v14045: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 12 op/s 2023-09-10T06:33:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:23 smithi098 ceph-mon[103124]: pgmap v14045: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 12 op/s 2023-09-10T06:33:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26034.466963 secs 2023-09-10T06:33:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26035.473719 secs 2023-09-10T06:33:24.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26034.466963 secs 2023-09-10T06:33:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26035.473719 secs 2023-09-10T06:33:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26034.466963 secs 2023-09-10T06:33:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26035.473719 secs 2023-09-10T06:33:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26036.086554 secs 2023-09-10T06:33:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:25 smithi170 ceph-mon[102832]: pgmap v14046: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 12 op/s 2023-09-10T06:33:25.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26036.086554 secs 2023-09-10T06:33:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:25 smithi082 ceph-mon[98694]: pgmap v14046: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 12 op/s 2023-09-10T06:33:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26036.086554 secs 2023-09-10T06:33:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:25 smithi098 ceph-mon[103124]: pgmap v14046: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 12 op/s 2023-09-10T06:33:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:26 smithi170 ceph-mon[102832]: pgmap v14047: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T06:33:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:26 smithi082 ceph-mon[98694]: pgmap v14047: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T06:33:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:26 smithi098 ceph-mon[103124]: pgmap v14047: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T06:33:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:29 smithi170 ceph-mon[102832]: pgmap v14048: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 89 KiB/s wr, 9 op/s 2023-09-10T06:33:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26040.473775 secs 2023-09-10T06:33:29.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:29 smithi082 ceph-mon[98694]: pgmap v14048: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 89 KiB/s wr, 9 op/s 2023-09-10T06:33:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26040.473775 secs 2023-09-10T06:33:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:29 smithi098 ceph-mon[103124]: pgmap v14048: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 89 KiB/s wr, 9 op/s 2023-09-10T06:33:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26040.473775 secs 2023-09-10T06:33:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26041.086627 secs 2023-09-10T06:33:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:33:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:33:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26041.086627 secs 2023-09-10T06:33:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:33:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:33:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26041.086627 secs 2023-09-10T06:33:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:33:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:33:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:31 smithi170 ceph-mon[102832]: pgmap v14049: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 89 KiB/s wr, 9 op/s 2023-09-10T06:33:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:33:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:31 smithi082 ceph-mon[98694]: pgmap v14049: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 89 KiB/s wr, 9 op/s 2023-09-10T06:33:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:33:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:31 smithi098 ceph-mon[103124]: pgmap v14049: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 89 KiB/s wr, 9 op/s 2023-09-10T06:33:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:33:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:33:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:33:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:33:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:33:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:33:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:33:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:33:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:33:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:33:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:33 smithi170 ceph-mon[102832]: pgmap v14050: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 278 KiB/s wr, 15 op/s 2023-09-10T06:33:33.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:33 smithi082 ceph-mon[98694]: pgmap v14050: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 278 KiB/s wr, 15 op/s 2023-09-10T06:33:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:33 smithi098 ceph-mon[103124]: pgmap v14050: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 278 KiB/s wr, 15 op/s 2023-09-10T06:33:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26044.467149 secs 2023-09-10T06:33:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26045.473833 secs 2023-09-10T06:33:34.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26044.467149 secs 2023-09-10T06:33:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26045.473833 secs 2023-09-10T06:33:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26044.467149 secs 2023-09-10T06:33:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26045.473833 secs 2023-09-10T06:33:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26046.086714 secs 2023-09-10T06:33:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:35 smithi170 ceph-mon[102832]: pgmap v14051: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 189 KiB/s wr, 10 op/s 2023-09-10T06:33:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26046.086714 secs 2023-09-10T06:33:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:35 smithi082 ceph-mon[98694]: pgmap v14051: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 189 KiB/s wr, 10 op/s 2023-09-10T06:33:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26046.086714 secs 2023-09-10T06:33:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:35 smithi098 ceph-mon[103124]: pgmap v14051: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 189 KiB/s wr, 10 op/s 2023-09-10T06:33:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:33:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:33:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:33:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:33:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:33:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:33:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:37 smithi170 ceph-mon[102832]: pgmap v14052: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 377 KiB/s wr, 16 op/s 2023-09-10T06:33:37.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:37 smithi082 ceph-mon[98694]: pgmap v14052: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 377 KiB/s wr, 16 op/s 2023-09-10T06:33:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:37 smithi098 ceph-mon[103124]: pgmap v14052: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 377 KiB/s wr, 16 op/s 2023-09-10T06:33:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:33:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:33:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:33:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:33:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:33:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:33:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:39 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26049.467244 secs 2023-09-10T06:33:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:39 smithi170 ceph-mon[102832]: pgmap v14053: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 377 KiB/s wr, 12 op/s 2023-09-10T06:33:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26050.473893 secs 2023-09-10T06:33:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:39 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26049.467244 secs 2023-09-10T06:33:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:39 smithi082 ceph-mon[98694]: pgmap v14053: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 377 KiB/s wr, 12 op/s 2023-09-10T06:33:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26050.473893 secs 2023-09-10T06:33:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:39 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26049.467244 secs 2023-09-10T06:33:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:39 smithi098 ceph-mon[103124]: pgmap v14053: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 377 KiB/s wr, 12 op/s 2023-09-10T06:33:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26050.473893 secs 2023-09-10T06:33:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26051.086780 secs 2023-09-10T06:33:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:33:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:33:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26051.086780 secs 2023-09-10T06:33:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:33:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:33:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26051.086780 secs 2023-09-10T06:33:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:33:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:33:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:41 smithi170 ceph-mon[102832]: pgmap v14054: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 377 KiB/s wr, 12 op/s 2023-09-10T06:33:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:41 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:33:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:41 smithi082 ceph-mon[98694]: pgmap v14054: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 377 KiB/s wr, 12 op/s 2023-09-10T06:33:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:41 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:33:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:41 smithi098 ceph-mon[103124]: pgmap v14054: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 377 KiB/s wr, 12 op/s 2023-09-10T06:33:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:41 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:33:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:43 smithi170 ceph-mon[102832]: pgmap v14055: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 426 KiB/s wr, 18 op/s 2023-09-10T06:33:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:33:43.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:43 smithi082 ceph-mon[98694]: pgmap v14055: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 426 KiB/s wr, 18 op/s 2023-09-10T06:33:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:33:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:43 smithi098 ceph-mon[103124]: pgmap v14055: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 426 KiB/s wr, 18 op/s 2023-09-10T06:33:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:33:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:33:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:33:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26055.473973 secs 2023-09-10T06:33:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:33:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:33:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26055.473973 secs 2023-09-10T06:33:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:33:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:33:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26055.473973 secs 2023-09-10T06:33:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26056.086844 secs 2023-09-10T06:33:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:45 smithi170 ceph-mon[102832]: pgmap v14056: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 237 KiB/s wr, 12 op/s 2023-09-10T06:33:45.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26056.086844 secs 2023-09-10T06:33:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:45 smithi082 ceph-mon[98694]: pgmap v14056: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 237 KiB/s wr, 12 op/s 2023-09-10T06:33:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26056.086844 secs 2023-09-10T06:33:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:45 smithi098 ceph-mon[103124]: pgmap v14056: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 237 KiB/s wr, 12 op/s 2023-09-10T06:33:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:46 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:33:46.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:46 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:33:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:46 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:33:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:47 smithi170 ceph-mon[102832]: pgmap v14057: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 564 KiB/s wr, 19 op/s 2023-09-10T06:33:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:47 smithi082 ceph-mon[98694]: pgmap v14057: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 564 KiB/s wr, 19 op/s 2023-09-10T06:33:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:47 smithi098 ceph-mon[103124]: pgmap v14057: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 564 KiB/s wr, 19 op/s 2023-09-10T06:33:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:49 smithi170 ceph-mon[102832]: pgmap v14058: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 376 KiB/s wr, 13 op/s 2023-09-10T06:33:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26060.474045 secs 2023-09-10T06:33:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:49 smithi082 ceph-mon[98694]: pgmap v14058: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 376 KiB/s wr, 13 op/s 2023-09-10T06:33:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26060.474045 secs 2023-09-10T06:33:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:49 smithi098 ceph-mon[103124]: pgmap v14058: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 376 KiB/s wr, 13 op/s 2023-09-10T06:33:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26060.474045 secs 2023-09-10T06:33:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26061.086917 secs 2023-09-10T06:33:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:33:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:33:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26061.086917 secs 2023-09-10T06:33:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:33:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:33:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26061.086917 secs 2023-09-10T06:33:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:33:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:33:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:51 smithi082 ceph-mon[98694]: pgmap v14059: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 376 KiB/s wr, 13 op/s 2023-09-10T06:33:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:51 smithi098 ceph-mon[103124]: pgmap v14059: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 376 KiB/s wr, 13 op/s 2023-09-10T06:33:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:51 smithi170 ceph-mon[102832]: pgmap v14059: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 376 KiB/s wr, 13 op/s 2023-09-10T06:33:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:33:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:33:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:52 smithi170 ceph-mon[102832]: pgmap v14060: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 418 KiB/s wr, 19 op/s 2023-09-10T06:33:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:33:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:33:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:52 smithi082 ceph-mon[98694]: pgmap v14060: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 418 KiB/s wr, 19 op/s 2023-09-10T06:33:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:33:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:33:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:52 smithi098 ceph-mon[103124]: pgmap v14060: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 418 KiB/s wr, 19 op/s 2023-09-10T06:33:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26065.474106 secs 2023-09-10T06:33:54.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26065.474106 secs 2023-09-10T06:33:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26065.474106 secs 2023-09-10T06:33:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26066.086981 secs 2023-09-10T06:33:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:55 smithi170 ceph-mon[102832]: pgmap v14061: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 369 KiB/s wr, 12 op/s 2023-09-10T06:33:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26066.086981 secs 2023-09-10T06:33:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:55 smithi082 ceph-mon[98694]: pgmap v14061: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 369 KiB/s wr, 12 op/s 2023-09-10T06:33:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26066.086981 secs 2023-09-10T06:33:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:55 smithi098 ceph-mon[103124]: pgmap v14061: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 369 KiB/s wr, 12 op/s 2023-09-10T06:33:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:56 smithi170 ceph-mon[102832]: pgmap v14062: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 696 KiB/s wr, 19 op/s 2023-09-10T06:33:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:56 smithi082 ceph-mon[98694]: pgmap v14062: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 696 KiB/s wr, 19 op/s 2023-09-10T06:33:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:56 smithi098 ceph-mon[103124]: pgmap v14062: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 696 KiB/s wr, 19 op/s 2023-09-10T06:33:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26069.467494 secs 2023-09-10T06:33:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:59 smithi170 ceph-mon[102832]: pgmap v14063: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 369 KiB/s wr, 12 op/s 2023-09-10T06:33:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:33:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26070.474185 secs 2023-09-10T06:33:59.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26069.467494 secs 2023-09-10T06:33:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:59 smithi082 ceph-mon[98694]: pgmap v14063: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 369 KiB/s wr, 12 op/s 2023-09-10T06:33:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:33:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26070.474185 secs 2023-09-10T06:33:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26069.467494 secs 2023-09-10T06:33:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:59 smithi098 ceph-mon[103124]: pgmap v14063: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 369 KiB/s wr, 12 op/s 2023-09-10T06:33:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:33:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26070.474185 secs 2023-09-10T06:34:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26071.087046 secs 2023-09-10T06:34:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:34:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:34:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:00.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26071.087046 secs 2023-09-10T06:34:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:34:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:34:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26071.087046 secs 2023-09-10T06:34:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:34:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:34:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:01 smithi170 ceph-mon[102832]: pgmap v14064: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 369 KiB/s wr, 12 op/s 2023-09-10T06:34:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:34:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:01 smithi082 ceph-mon[98694]: pgmap v14064: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 369 KiB/s wr, 12 op/s 2023-09-10T06:34:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:34:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:01 smithi098 ceph-mon[103124]: pgmap v14064: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 369 KiB/s wr, 12 op/s 2023-09-10T06:34:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:34:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:03 smithi170 ceph-mon[102832]: pgmap v14065: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 478 KiB/s wr, 22 op/s 2023-09-10T06:34:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:03 smithi082 ceph-mon[98694]: pgmap v14065: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 478 KiB/s wr, 22 op/s 2023-09-10T06:34:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:03 smithi098 ceph-mon[103124]: pgmap v14065: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 478 KiB/s wr, 22 op/s 2023-09-10T06:34:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26074.467554 secs 2023-09-10T06:34:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:04 smithi170 ceph-mon[102832]: osdmap e947: 12 total, 12 up, 12 in 2023-09-10T06:34:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:34:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26075.474243 secs 2023-09-10T06:34:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:34:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:34:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:34:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:34:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:34:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:34:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26074.467554 secs 2023-09-10T06:34:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:04 smithi082 ceph-mon[98694]: osdmap e947: 12 total, 12 up, 12 in 2023-09-10T06:34:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:34:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26075.474243 secs 2023-09-10T06:34:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:34:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:34:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:34:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:34:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:34:04.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:34:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26074.467554 secs 2023-09-10T06:34:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:04 smithi098 ceph-mon[103124]: osdmap e947: 12 total, 12 up, 12 in 2023-09-10T06:34:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:34:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26075.474243 secs 2023-09-10T06:34:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:34:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:34:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:34:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:34:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:34:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:34:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26076.087112 secs 2023-09-10T06:34:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:05 smithi170 ceph-mon[102832]: pgmap v14067: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 523 KiB/s wr, 20 op/s 2023-09-10T06:34:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:34:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:34:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:34:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:34:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:34:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:05 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T06:34:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:05 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T06:34:05.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:34:05.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:34:05.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:34:05.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:34:05.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:05 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T06:34:05.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:05 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T06:34:05.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:34:05.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:34:05.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:34:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26076.087112 secs 2023-09-10T06:34:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:05 smithi082 ceph-mon[98694]: pgmap v14067: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 523 KiB/s wr, 20 op/s 2023-09-10T06:34:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:34:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:34:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:34:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:34:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:34:05.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:05 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T06:34:05.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:05 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T06:34:05.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:34:05.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:34:05.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:34:05.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:34:05.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:05 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T06:34:05.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:05 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T06:34:05.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:34:05.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:34:05.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:34:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26076.087112 secs 2023-09-10T06:34:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:05 smithi098 ceph-mon[103124]: pgmap v14067: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 523 KiB/s wr, 20 op/s 2023-09-10T06:34:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:34:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:34:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:34:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:34:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:34:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:05 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T06:34:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:05 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T06:34:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:34:05.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:34:05.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:34:05.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:34:05.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:05 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T06:34:05.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:05 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T06:34:05.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:34:05.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:34:05.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:34:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:34:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:34:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:34:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:07 smithi170 ceph-mon[102832]: pgmap v14068: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 408 KiB/s wr, 31 op/s 2023-09-10T06:34:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:07 smithi082 ceph-mon[98694]: pgmap v14068: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 408 KiB/s wr, 31 op/s 2023-09-10T06:34:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:07 smithi098 ceph-mon[103124]: pgmap v14068: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 408 KiB/s wr, 31 op/s 2023-09-10T06:34:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26079.467624 secs 2023-09-10T06:34:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26079.467624 secs 2023-09-10T06:34:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26079.467624 secs 2023-09-10T06:34:09.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:09 smithi170 ceph-mon[102832]: pgmap v14069: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 408 KiB/s wr, 31 op/s 2023-09-10T06:34:09.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26080.474302 secs 2023-09-10T06:34:09.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26081.087180 secs 2023-09-10T06:34:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:09 smithi082 ceph-mon[98694]: pgmap v14069: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 408 KiB/s wr, 31 op/s 2023-09-10T06:34:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26080.474302 secs 2023-09-10T06:34:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26081.087180 secs 2023-09-10T06:34:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:09 smithi098 ceph-mon[103124]: pgmap v14069: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 408 KiB/s wr, 31 op/s 2023-09-10T06:34:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26080.474302 secs 2023-09-10T06:34:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26081.087180 secs 2023-09-10T06:34:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:11 smithi170 ceph-mon[102832]: pgmap v14070: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 408 KiB/s wr, 31 op/s 2023-09-10T06:34:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:11 smithi170 ceph-mon[102832]: osdmap e948: 12 total, 12 up, 12 in 2023-09-10T06:34:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:11 smithi082 ceph-mon[98694]: pgmap v14070: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 408 KiB/s wr, 31 op/s 2023-09-10T06:34:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:11 smithi082 ceph-mon[98694]: osdmap e948: 12 total, 12 up, 12 in 2023-09-10T06:34:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:11 smithi098 ceph-mon[103124]: pgmap v14070: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 408 KiB/s wr, 31 op/s 2023-09-10T06:34:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:11 smithi098 ceph-mon[103124]: osdmap e948: 12 total, 12 up, 12 in 2023-09-10T06:34:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:13 smithi170 ceph-mon[102832]: pgmap v14072: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 312 KiB/s wr, 27 op/s 2023-09-10T06:34:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26084.467683 secs 2023-09-10T06:34:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:13 smithi082 ceph-mon[98694]: pgmap v14072: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 312 KiB/s wr, 27 op/s 2023-09-10T06:34:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26084.467683 secs 2023-09-10T06:34:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:13 smithi098 ceph-mon[103124]: pgmap v14072: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 312 KiB/s wr, 27 op/s 2023-09-10T06:34:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26084.467683 secs 2023-09-10T06:34:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26085.474363 secs 2023-09-10T06:34:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26086.087243 secs 2023-09-10T06:34:14.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26085.474363 secs 2023-09-10T06:34:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26086.087243 secs 2023-09-10T06:34:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26085.474363 secs 2023-09-10T06:34:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26086.087243 secs 2023-09-10T06:34:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:15 smithi170 ceph-mon[102832]: pgmap v14073: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 279 KiB/s wr, 24 op/s 2023-09-10T06:34:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:15 smithi082 ceph-mon[98694]: pgmap v14073: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 279 KiB/s wr, 24 op/s 2023-09-10T06:34:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:15 smithi098 ceph-mon[103124]: pgmap v14073: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 279 KiB/s wr, 24 op/s 2023-09-10T06:34:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:16 smithi170 ceph-mon[102832]: pgmap v14074: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T06:34:17.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:16 smithi082 ceph-mon[98694]: pgmap v14074: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T06:34:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:16 smithi098 ceph-mon[103124]: pgmap v14074: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T06:34:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26089.467746 secs 2023-09-10T06:34:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:19 smithi170 ceph-mon[102832]: pgmap v14075: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T06:34:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26090.474421 secs 2023-09-10T06:34:19.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26089.467746 secs 2023-09-10T06:34:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:19 smithi082 ceph-mon[98694]: pgmap v14075: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T06:34:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26090.474421 secs 2023-09-10T06:34:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26089.467746 secs 2023-09-10T06:34:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:19 smithi098 ceph-mon[103124]: pgmap v14075: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T06:34:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26090.474421 secs 2023-09-10T06:34:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26091.087347 secs 2023-09-10T06:34:20.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26091.087347 secs 2023-09-10T06:34:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26091.087347 secs 2023-09-10T06:34:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:21 smithi170 ceph-mon[102832]: pgmap v14076: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T06:34:21.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:21 smithi082 ceph-mon[98694]: pgmap v14076: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T06:34:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:21 smithi098 ceph-mon[103124]: pgmap v14076: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 12 op/s 2023-09-10T06:34:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:23 smithi170 ceph-mon[102832]: pgmap v14077: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T06:34:23.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:23 smithi082 ceph-mon[98694]: pgmap v14077: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T06:34:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:23 smithi098 ceph-mon[103124]: pgmap v14077: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T06:34:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26094.467774 secs 2023-09-10T06:34:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:34:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26095.474503 secs 2023-09-10T06:34:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26094.467774 secs 2023-09-10T06:34:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:34:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26095.474503 secs 2023-09-10T06:34:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26094.467774 secs 2023-09-10T06:34:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:34:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26095.474503 secs 2023-09-10T06:34:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26096.087413 secs 2023-09-10T06:34:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:25 smithi170 ceph-mon[102832]: pgmap v14078: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 104 KiB/s wr, 10 op/s 2023-09-10T06:34:25.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26096.087413 secs 2023-09-10T06:34:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:25 smithi082 ceph-mon[98694]: pgmap v14078: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 104 KiB/s wr, 10 op/s 2023-09-10T06:34:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26096.087413 secs 2023-09-10T06:34:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:25 smithi098 ceph-mon[103124]: pgmap v14078: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 104 KiB/s wr, 10 op/s 2023-09-10T06:34:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:26 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:34:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:26.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:26 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:34:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:26 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:34:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:27 smithi170 ceph-mon[102832]: pgmap v14079: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 15 op/s 2023-09-10T06:34:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:27 smithi082 ceph-mon[98694]: pgmap v14079: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 15 op/s 2023-09-10T06:34:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:27 smithi098 ceph-mon[103124]: pgmap v14079: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 15 op/s 2023-09-10T06:34:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:29 smithi170 ceph-mon[102832]: pgmap v14080: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T06:34:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26100.474561 secs 2023-09-10T06:34:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:29 smithi082 ceph-mon[98694]: pgmap v14080: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T06:34:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26100.474561 secs 2023-09-10T06:34:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:29 smithi098 ceph-mon[103124]: pgmap v14080: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T06:34:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26100.474561 secs 2023-09-10T06:34:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26101.087484 secs 2023-09-10T06:34:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26101.087484 secs 2023-09-10T06:34:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26101.087484 secs 2023-09-10T06:34:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:31 smithi170 ceph-mon[102832]: pgmap v14081: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T06:34:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:34:31.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:31 smithi082 ceph-mon[98694]: pgmap v14081: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T06:34:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:34:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:31 smithi098 ceph-mon[103124]: pgmap v14081: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 10 op/s 2023-09-10T06:34:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:34:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:34:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:34:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:32.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:34:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:34:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:34:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:34:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:33 smithi170 ceph-mon[102832]: pgmap v14082: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 175 KiB/s wr, 15 op/s 2023-09-10T06:34:33.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:33 smithi082 ceph-mon[98694]: pgmap v14082: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 175 KiB/s wr, 15 op/s 2023-09-10T06:34:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:33 smithi098 ceph-mon[103124]: pgmap v14082: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 175 KiB/s wr, 15 op/s 2023-09-10T06:34:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26104.467901 secs 2023-09-10T06:34:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:34:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26105.474643 secs 2023-09-10T06:34:34.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26104.467901 secs 2023-09-10T06:34:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:34:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26105.474643 secs 2023-09-10T06:34:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26104.467901 secs 2023-09-10T06:34:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:34:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26105.474643 secs 2023-09-10T06:34:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26106.087547 secs 2023-09-10T06:34:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:35 smithi170 ceph-mon[102832]: pgmap v14083: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 175 KiB/s wr, 9 op/s 2023-09-10T06:34:35.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26106.087547 secs 2023-09-10T06:34:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:35 smithi082 ceph-mon[98694]: pgmap v14083: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 175 KiB/s wr, 9 op/s 2023-09-10T06:34:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26106.087547 secs 2023-09-10T06:34:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:35 smithi098 ceph-mon[103124]: pgmap v14083: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 175 KiB/s wr, 9 op/s 2023-09-10T06:34:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:34:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:36.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:34:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:34:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:37 smithi170 ceph-mon[102832]: pgmap v14084: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 641 KiB/s wr, 16 op/s 2023-09-10T06:34:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:37 smithi082 ceph-mon[98694]: pgmap v14084: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 641 KiB/s wr, 16 op/s 2023-09-10T06:34:37.585 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:37 smithi098 ceph-mon[103124]: pgmap v14084: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 641 KiB/s wr, 16 op/s 2023-09-10T06:34:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:39 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26109.467961 secs 2023-09-10T06:34:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:39 smithi170 ceph-mon[102832]: pgmap v14085: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 552 KiB/s wr, 11 op/s 2023-09-10T06:34:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26110.474704 secs 2023-09-10T06:34:39.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:39 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26109.467961 secs 2023-09-10T06:34:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:39 smithi082 ceph-mon[98694]: pgmap v14085: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 552 KiB/s wr, 11 op/s 2023-09-10T06:34:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26110.474704 secs 2023-09-10T06:34:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:39 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26109.467961 secs 2023-09-10T06:34:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:39 smithi098 ceph-mon[103124]: pgmap v14085: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 552 KiB/s wr, 11 op/s 2023-09-10T06:34:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26110.474704 secs 2023-09-10T06:34:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26111.087612 secs 2023-09-10T06:34:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26111.087612 secs 2023-09-10T06:34:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26111.087612 secs 2023-09-10T06:34:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:41 smithi082 ceph-mon[98694]: pgmap v14086: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 552 KiB/s wr, 11 op/s 2023-09-10T06:34:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:41 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:34:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:41 smithi098 ceph-mon[103124]: pgmap v14086: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 552 KiB/s wr, 11 op/s 2023-09-10T06:34:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:41 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:34:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:41 smithi170 ceph-mon[102832]: pgmap v14086: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 552 KiB/s wr, 11 op/s 2023-09-10T06:34:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:41 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:34:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:43 smithi082 ceph-mon[98694]: pgmap v14087: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 621 KiB/s wr, 17 op/s 2023-09-10T06:34:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:34:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:43 smithi098 ceph-mon[103124]: pgmap v14087: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 621 KiB/s wr, 17 op/s 2023-09-10T06:34:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:34:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:43 smithi170 ceph-mon[102832]: pgmap v14087: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 621 KiB/s wr, 17 op/s 2023-09-10T06:34:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:34:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:34:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26115.474762 secs 2023-09-10T06:34:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:34:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26115.474762 secs 2023-09-10T06:34:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:34:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26115.474762 secs 2023-09-10T06:34:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26116.087673 secs 2023-09-10T06:34:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:45 smithi082 ceph-mon[98694]: pgmap v14088: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 535 KiB/s wr, 12 op/s 2023-09-10T06:34:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26116.087673 secs 2023-09-10T06:34:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:45 smithi098 ceph-mon[103124]: pgmap v14088: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 535 KiB/s wr, 12 op/s 2023-09-10T06:34:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26116.087673 secs 2023-09-10T06:34:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:45 smithi170 ceph-mon[102832]: pgmap v14088: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 535 KiB/s wr, 12 op/s 2023-09-10T06:34:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:34:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:46 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:34:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:34:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:46 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:34:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:34:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:46 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:34:47.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:47 smithi098 ceph-mon[103124]: pgmap v14089: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 793 KiB/s wr, 18 op/s 2023-09-10T06:34:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:47 smithi170 ceph-mon[102832]: pgmap v14089: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 793 KiB/s wr, 18 op/s 2023-09-10T06:34:47.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:47 smithi082 ceph-mon[98694]: pgmap v14089: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 793 KiB/s wr, 18 op/s 2023-09-10T06:34:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:49 smithi098 ceph-mon[103124]: pgmap v14090: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 327 KiB/s wr, 12 op/s 2023-09-10T06:34:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26120.474825 secs 2023-09-10T06:34:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:49 smithi170 ceph-mon[102832]: pgmap v14090: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 327 KiB/s wr, 12 op/s 2023-09-10T06:34:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26120.474825 secs 2023-09-10T06:34:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:49 smithi082 ceph-mon[98694]: pgmap v14090: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 327 KiB/s wr, 12 op/s 2023-09-10T06:34:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26120.474825 secs 2023-09-10T06:34:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26121.087739 secs 2023-09-10T06:34:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:34:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26121.087739 secs 2023-09-10T06:34:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:34:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:50.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26121.087739 secs 2023-09-10T06:34:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:34:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:51 smithi098 ceph-mon[103124]: pgmap v14091: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 327 KiB/s wr, 12 op/s 2023-09-10T06:34:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:51 smithi170 ceph-mon[102832]: pgmap v14091: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 327 KiB/s wr, 12 op/s 2023-09-10T06:34:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:51 smithi082 ceph-mon[98694]: pgmap v14091: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 327 KiB/s wr, 12 op/s 2023-09-10T06:34:53.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:34:53.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:53.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:52 smithi082 ceph-mon[98694]: pgmap v14092: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 636 KiB/s wr, 19 op/s 2023-09-10T06:34:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:34:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:52 smithi098 ceph-mon[103124]: pgmap v14092: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 636 KiB/s wr, 19 op/s 2023-09-10T06:34:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:34:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:52 smithi170 ceph-mon[102832]: pgmap v14092: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 636 KiB/s wr, 19 op/s 2023-09-10T06:34:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26124.468140 secs 2023-09-10T06:34:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26124.468140 secs 2023-09-10T06:34:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26124.468140 secs 2023-09-10T06:34:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26125.474887 secs 2023-09-10T06:34:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26126.087802 secs 2023-09-10T06:34:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:54 smithi082 ceph-mon[98694]: pgmap v14093: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 566 KiB/s wr, 13 op/s 2023-09-10T06:34:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26125.474887 secs 2023-09-10T06:34:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26126.087802 secs 2023-09-10T06:34:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:54 smithi098 ceph-mon[103124]: pgmap v14093: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 566 KiB/s wr, 13 op/s 2023-09-10T06:34:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26125.474887 secs 2023-09-10T06:34:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26126.087802 secs 2023-09-10T06:34:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:54 smithi170 ceph-mon[102832]: pgmap v14093: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 566 KiB/s wr, 13 op/s 2023-09-10T06:34:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:34:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:34:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:56 smithi170 ceph-mon[102832]: pgmap v14094: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 676 KiB/s wr, 19 op/s 2023-09-10T06:34:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:34:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:34:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:56 smithi082 ceph-mon[98694]: pgmap v14094: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 676 KiB/s wr, 19 op/s 2023-09-10T06:34:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:34:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:34:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:56 smithi098 ceph-mon[103124]: pgmap v14094: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 676 KiB/s wr, 19 op/s 2023-09-10T06:34:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:34:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:34:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:34:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:34:59.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26129.468201 secs 2023-09-10T06:34:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:58 smithi082 ceph-mon[98694]: pgmap v14095: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 419 KiB/s wr, 13 op/s 2023-09-10T06:34:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26129.468201 secs 2023-09-10T06:34:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:58 smithi098 ceph-mon[103124]: pgmap v14095: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 419 KiB/s wr, 13 op/s 2023-09-10T06:34:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26129.468201 secs 2023-09-10T06:34:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:58 smithi170 ceph-mon[102832]: pgmap v14095: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 419 KiB/s wr, 13 op/s 2023-09-10T06:35:00.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26130.474945 secs 2023-09-10T06:35:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:34:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26131.087870 secs 2023-09-10T06:35:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26130.474945 secs 2023-09-10T06:35:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:34:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26131.087870 secs 2023-09-10T06:35:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26130.474945 secs 2023-09-10T06:35:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:34:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26131.087870 secs 2023-09-10T06:35:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:35:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:35:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:00 smithi082 ceph-mon[98694]: pgmap v14096: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 419 KiB/s wr, 13 op/s 2023-09-10T06:35:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:35:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:35:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:00 smithi098 ceph-mon[103124]: pgmap v14096: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 419 KiB/s wr, 13 op/s 2023-09-10T06:35:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:35:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:35:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:00 smithi170 ceph-mon[102832]: pgmap v14096: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 419 KiB/s wr, 13 op/s 2023-09-10T06:35:03.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:03 smithi170 ceph-mon[102832]: pgmap v14097: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 609 KiB/s wr, 19 op/s 2023-09-10T06:35:03.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:03 smithi082 ceph-mon[98694]: pgmap v14097: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 609 KiB/s wr, 19 op/s 2023-09-10T06:35:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:03 smithi098 ceph-mon[103124]: pgmap v14097: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 609 KiB/s wr, 19 op/s 2023-09-10T06:35:04.433 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26134.468262 secs 2023-09-10T06:35:04.433 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26135.475031 secs 2023-09-10T06:35:04.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26134.468262 secs 2023-09-10T06:35:04.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26135.475031 secs 2023-09-10T06:35:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26134.468262 secs 2023-09-10T06:35:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26135.475031 secs 2023-09-10T06:35:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26136.087934 secs 2023-09-10T06:35:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:05 smithi170 ceph-mon[102832]: pgmap v14098: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 300 KiB/s wr, 11 op/s 2023-09-10T06:35:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:35:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26136.087934 secs 2023-09-10T06:35:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:05 smithi082 ceph-mon[98694]: pgmap v14098: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 300 KiB/s wr, 11 op/s 2023-09-10T06:35:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:35:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26136.087934 secs 2023-09-10T06:35:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:05 smithi098 ceph-mon[103124]: pgmap v14098: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 300 KiB/s wr, 11 op/s 2023-09-10T06:35:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:35:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:35:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:35:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:35:06.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:35:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:35:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:35:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:35:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:35:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:35:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:07 smithi170 ceph-mon[102832]: pgmap v14099: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 435 KiB/s wr, 32 op/s 2023-09-10T06:35:07.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:07 smithi082 ceph-mon[98694]: pgmap v14099: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 435 KiB/s wr, 32 op/s 2023-09-10T06:35:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:07 smithi098 ceph-mon[103124]: pgmap v14099: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 435 KiB/s wr, 32 op/s 2023-09-10T06:35:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26139.468345 secs 2023-09-10T06:35:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:09 smithi170 ceph-mon[102832]: pgmap v14100: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 325 KiB/s wr, 26 op/s 2023-09-10T06:35:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:09 smithi170 ceph-mon[102832]: osdmap e949: 12 total, 12 up, 12 in 2023-09-10T06:35:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26140.475090 secs 2023-09-10T06:35:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26139.468345 secs 2023-09-10T06:35:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:09 smithi082 ceph-mon[98694]: pgmap v14100: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 325 KiB/s wr, 26 op/s 2023-09-10T06:35:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:09 smithi082 ceph-mon[98694]: osdmap e949: 12 total, 12 up, 12 in 2023-09-10T06:35:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26140.475090 secs 2023-09-10T06:35:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26139.468345 secs 2023-09-10T06:35:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:09 smithi098 ceph-mon[103124]: pgmap v14100: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 325 KiB/s wr, 26 op/s 2023-09-10T06:35:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:09 smithi098 ceph-mon[103124]: osdmap e949: 12 total, 12 up, 12 in 2023-09-10T06:35:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26140.475090 secs 2023-09-10T06:35:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26141.087996 secs 2023-09-10T06:35:10.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26141.087996 secs 2023-09-10T06:35:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26141.087996 secs 2023-09-10T06:35:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:11 smithi170 ceph-mon[102832]: pgmap v14102: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 389 KiB/s wr, 31 op/s 2023-09-10T06:35:11.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:11 smithi082 ceph-mon[98694]: pgmap v14102: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 389 KiB/s wr, 31 op/s 2023-09-10T06:35:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:11 smithi098 ceph-mon[103124]: pgmap v14102: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 389 KiB/s wr, 31 op/s 2023-09-10T06:35:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:13 smithi170 ceph-mon[102832]: pgmap v14103: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 185 KiB/s wr, 31 op/s 2023-09-10T06:35:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:13 smithi082 ceph-mon[98694]: pgmap v14103: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 185 KiB/s wr, 31 op/s 2023-09-10T06:35:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:13 smithi098 ceph-mon[103124]: pgmap v14103: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 185 KiB/s wr, 31 op/s 2023-09-10T06:35:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26144.468405 secs 2023-09-10T06:35:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26145.475150 secs 2023-09-10T06:35:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26144.468405 secs 2023-09-10T06:35:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26145.475150 secs 2023-09-10T06:35:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26144.468405 secs 2023-09-10T06:35:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26145.475150 secs 2023-09-10T06:35:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26146.088062 secs 2023-09-10T06:35:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:15 smithi170 ceph-mon[102832]: pgmap v14104: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 185 KiB/s wr, 31 op/s 2023-09-10T06:35:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26146.088062 secs 2023-09-10T06:35:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:15 smithi082 ceph-mon[98694]: pgmap v14104: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 185 KiB/s wr, 31 op/s 2023-09-10T06:35:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26146.088062 secs 2023-09-10T06:35:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:15 smithi098 ceph-mon[103124]: pgmap v14104: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 185 KiB/s wr, 31 op/s 2023-09-10T06:35:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:16 smithi082 ceph-mon[98694]: osdmap e950: 12 total, 12 up, 12 in 2023-09-10T06:35:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:16 smithi082 ceph-mon[98694]: pgmap v14106: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 227 KiB/s wr, 16 op/s 2023-09-10T06:35:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:16 smithi098 ceph-mon[103124]: osdmap e950: 12 total, 12 up, 12 in 2023-09-10T06:35:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:16 smithi098 ceph-mon[103124]: pgmap v14106: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 227 KiB/s wr, 16 op/s 2023-09-10T06:35:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:16 smithi170 ceph-mon[102832]: osdmap e950: 12 total, 12 up, 12 in 2023-09-10T06:35:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:16 smithi170 ceph-mon[102832]: pgmap v14106: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 227 KiB/s wr, 16 op/s 2023-09-10T06:35:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26149.468463 secs 2023-09-10T06:35:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:19 smithi170 ceph-mon[102832]: pgmap v14107: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 13 op/s 2023-09-10T06:35:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26150.475244 secs 2023-09-10T06:35:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26149.468463 secs 2023-09-10T06:35:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:19 smithi082 ceph-mon[98694]: pgmap v14107: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 13 op/s 2023-09-10T06:35:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26150.475244 secs 2023-09-10T06:35:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26149.468463 secs 2023-09-10T06:35:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:19 smithi098 ceph-mon[103124]: pgmap v14107: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 13 op/s 2023-09-10T06:35:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26150.475244 secs 2023-09-10T06:35:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26151.088128 secs 2023-09-10T06:35:20.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26151.088128 secs 2023-09-10T06:35:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26151.088128 secs 2023-09-10T06:35:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:21 smithi170 ceph-mon[102832]: pgmap v14108: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 182 KiB/s wr, 13 op/s 2023-09-10T06:35:21.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:21 smithi082 ceph-mon[98694]: pgmap v14108: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 182 KiB/s wr, 13 op/s 2023-09-10T06:35:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:21 smithi098 ceph-mon[103124]: pgmap v14108: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 182 KiB/s wr, 13 op/s 2023-09-10T06:35:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:23 smithi170 ceph-mon[102832]: pgmap v14109: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T06:35:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:23 smithi082 ceph-mon[98694]: pgmap v14109: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T06:35:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:23 smithi098 ceph-mon[103124]: pgmap v14109: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T06:35:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26154.468523 secs 2023-09-10T06:35:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:35:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:35:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26155.475301 secs 2023-09-10T06:35:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26154.468523 secs 2023-09-10T06:35:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:35:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:35:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26155.475301 secs 2023-09-10T06:35:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26154.468523 secs 2023-09-10T06:35:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:35:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:35:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26155.475301 secs 2023-09-10T06:35:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26156.088191 secs 2023-09-10T06:35:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:25 smithi170 ceph-mon[102832]: pgmap v14110: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T06:35:25.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26156.088191 secs 2023-09-10T06:35:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:25 smithi082 ceph-mon[98694]: pgmap v14110: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T06:35:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26156.088191 secs 2023-09-10T06:35:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:25 smithi098 ceph-mon[103124]: pgmap v14110: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T06:35:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:26 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:35:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:35:26.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:26 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:35:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:35:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:26 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:35:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:35:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:27 smithi170 ceph-mon[102832]: pgmap v14111: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T06:35:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:27 smithi082 ceph-mon[98694]: pgmap v14111: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T06:35:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:27 smithi098 ceph-mon[103124]: pgmap v14111: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T06:35:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:29 smithi170 ceph-mon[102832]: pgmap v14112: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T06:35:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26160.475362 secs 2023-09-10T06:35:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:29 smithi082 ceph-mon[98694]: pgmap v14112: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T06:35:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26160.475362 secs 2023-09-10T06:35:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:29 smithi098 ceph-mon[103124]: pgmap v14112: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T06:35:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26160.475362 secs 2023-09-10T06:35:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26161.088255 secs 2023-09-10T06:35:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26161.088255 secs 2023-09-10T06:35:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26161.088255 secs 2023-09-10T06:35:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:31 smithi170 ceph-mon[102832]: pgmap v14113: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T06:35:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:35:31.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:31 smithi082 ceph-mon[98694]: pgmap v14113: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T06:35:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:35:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:31 smithi098 ceph-mon[103124]: pgmap v14113: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T06:35:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:35:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:35:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:35:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:35:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:35:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:35:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:35:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:35:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:35:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:35:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:33 smithi170 ceph-mon[102832]: pgmap v14114: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 219 KiB/s wr, 15 op/s 2023-09-10T06:35:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:33 smithi082 ceph-mon[98694]: pgmap v14114: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 219 KiB/s wr, 15 op/s 2023-09-10T06:35:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:33 smithi098 ceph-mon[103124]: pgmap v14114: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 219 KiB/s wr, 15 op/s 2023-09-10T06:35:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26165.475421 secs 2023-09-10T06:35:34.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26165.475421 secs 2023-09-10T06:35:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26165.475421 secs 2023-09-10T06:35:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26166.088318 secs 2023-09-10T06:35:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:35 smithi170 ceph-mon[102832]: pgmap v14115: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 219 KiB/s wr, 10 op/s 2023-09-10T06:35:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26166.088318 secs 2023-09-10T06:35:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:35 smithi082 ceph-mon[98694]: pgmap v14115: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 219 KiB/s wr, 10 op/s 2023-09-10T06:35:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26166.088318 secs 2023-09-10T06:35:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:35 smithi098 ceph-mon[103124]: pgmap v14115: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 219 KiB/s wr, 10 op/s 2023-09-10T06:35:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:36 smithi082 ceph-mon[98694]: pgmap v14116: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 324 KiB/s wr, 15 op/s 2023-09-10T06:35:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:36 smithi098 ceph-mon[103124]: pgmap v14116: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 324 KiB/s wr, 15 op/s 2023-09-10T06:35:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:36 smithi170 ceph-mon[102832]: pgmap v14116: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 324 KiB/s wr, 15 op/s 2023-09-10T06:35:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:35:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:35:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:35:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:35:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:35:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:35:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26169.468766 secs 2023-09-10T06:35:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:38 smithi082 ceph-mon[98694]: pgmap v14117: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 219 KiB/s wr, 10 op/s 2023-09-10T06:35:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26169.468766 secs 2023-09-10T06:35:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:38 smithi098 ceph-mon[103124]: pgmap v14117: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 219 KiB/s wr, 10 op/s 2023-09-10T06:35:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26169.468766 secs 2023-09-10T06:35:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:38 smithi170 ceph-mon[102832]: pgmap v14117: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 219 KiB/s wr, 10 op/s 2023-09-10T06:35:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26170.475480 secs 2023-09-10T06:35:40.307 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26171.088383 secs 2023-09-10T06:35:40.308 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26170.475480 secs 2023-09-10T06:35:40.308 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26171.088383 secs 2023-09-10T06:35:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26170.475480 secs 2023-09-10T06:35:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26171.088383 secs 2023-09-10T06:35:41.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:35:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:35:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:35:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:40 smithi082 ceph-mon[98694]: pgmap v14118: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 219 KiB/s wr, 10 op/s 2023-09-10T06:35:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:35:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:35:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:35:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:40 smithi098 ceph-mon[103124]: pgmap v14118: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 219 KiB/s wr, 10 op/s 2023-09-10T06:35:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:35:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:35:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:35:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:40 smithi170 ceph-mon[102832]: pgmap v14118: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 219 KiB/s wr, 10 op/s 2023-09-10T06:35:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:43 smithi170 ceph-mon[102832]: pgmap v14119: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 553 KiB/s wr, 38 op/s 2023-09-10T06:35:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:43 smithi082 ceph-mon[98694]: pgmap v14119: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 553 KiB/s wr, 38 op/s 2023-09-10T06:35:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:43 smithi098 ceph-mon[103124]: pgmap v14119: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 553 KiB/s wr, 38 op/s 2023-09-10T06:35:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:44 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26174.468817 secs 2023-09-10T06:35:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:35:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:35:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26175.475540 secs 2023-09-10T06:35:44.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:44 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26174.468817 secs 2023-09-10T06:35:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:35:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:35:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26175.475540 secs 2023-09-10T06:35:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:44 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26174.468817 secs 2023-09-10T06:35:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:35:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:35:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26175.475540 secs 2023-09-10T06:35:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26176.088448 secs 2023-09-10T06:35:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:45 smithi170 ceph-mon[102832]: pgmap v14120: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 440 KiB/s wr, 32 op/s 2023-09-10T06:35:45.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26176.088448 secs 2023-09-10T06:35:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:45 smithi082 ceph-mon[98694]: pgmap v14120: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 440 KiB/s wr, 32 op/s 2023-09-10T06:35:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26176.088448 secs 2023-09-10T06:35:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:45 smithi098 ceph-mon[103124]: pgmap v14120: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 440 KiB/s wr, 32 op/s 2023-09-10T06:35:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:35:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:35:46.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:35:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:35:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:35:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:35:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:47 smithi170 ceph-mon[102832]: pgmap v14121: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 720 KiB/s wr, 39 op/s 2023-09-10T06:35:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:47 smithi082 ceph-mon[98694]: pgmap v14121: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 720 KiB/s wr, 39 op/s 2023-09-10T06:35:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:47 smithi098 ceph-mon[103124]: pgmap v14121: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 720 KiB/s wr, 39 op/s 2023-09-10T06:35:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:49 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26179.468902 secs 2023-09-10T06:35:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:49 smithi170 ceph-mon[102832]: pgmap v14122: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 615 KiB/s wr, 34 op/s 2023-09-10T06:35:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26180.475598 secs 2023-09-10T06:35:49.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:49 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26179.468902 secs 2023-09-10T06:35:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:49 smithi082 ceph-mon[98694]: pgmap v14122: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 615 KiB/s wr, 34 op/s 2023-09-10T06:35:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26180.475598 secs 2023-09-10T06:35:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:49 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26179.468902 secs 2023-09-10T06:35:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:49 smithi098 ceph-mon[103124]: pgmap v14122: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 615 KiB/s wr, 34 op/s 2023-09-10T06:35:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26180.475598 secs 2023-09-10T06:35:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26181.088490 secs 2023-09-10T06:35:50.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26181.088490 secs 2023-09-10T06:35:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26181.088490 secs 2023-09-10T06:35:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:51 smithi170 ceph-mon[102832]: pgmap v14123: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 615 KiB/s wr, 34 op/s 2023-09-10T06:35:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:51 smithi082 ceph-mon[98694]: pgmap v14123: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 615 KiB/s wr, 34 op/s 2023-09-10T06:35:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:51 smithi098 ceph-mon[103124]: pgmap v14123: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 615 KiB/s wr, 34 op/s 2023-09-10T06:35:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:53 smithi170 ceph-mon[102832]: pgmap v14124: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 682 KiB/s wr, 39 op/s 2023-09-10T06:35:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:53 smithi082 ceph-mon[98694]: pgmap v14124: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 682 KiB/s wr, 39 op/s 2023-09-10T06:35:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:53 smithi098 ceph-mon[103124]: pgmap v14124: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 682 KiB/s wr, 39 op/s 2023-09-10T06:35:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26184.468965 secs 2023-09-10T06:35:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26185.475684 secs 2023-09-10T06:35:54.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26184.468965 secs 2023-09-10T06:35:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26185.475684 secs 2023-09-10T06:35:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26184.468965 secs 2023-09-10T06:35:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26185.475684 secs 2023-09-10T06:35:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26186.088555 secs 2023-09-10T06:35:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:55 smithi170 ceph-mon[102832]: pgmap v14125: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 347 KiB/s wr, 12 op/s 2023-09-10T06:35:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26186.088555 secs 2023-09-10T06:35:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:55 smithi082 ceph-mon[98694]: pgmap v14125: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 347 KiB/s wr, 12 op/s 2023-09-10T06:35:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26186.088555 secs 2023-09-10T06:35:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:55 smithi098 ceph-mon[103124]: pgmap v14125: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 347 KiB/s wr, 12 op/s 2023-09-10T06:35:57.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:56 smithi082 ceph-mon[98694]: pgmap v14126: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 504 KiB/s wr, 18 op/s 2023-09-10T06:35:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:56 smithi098 ceph-mon[103124]: pgmap v14126: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 504 KiB/s wr, 18 op/s 2023-09-10T06:35:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:56 smithi170 ceph-mon[102832]: pgmap v14126: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 504 KiB/s wr, 18 op/s 2023-09-10T06:35:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:35:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:35:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:35:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:35:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:35:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:35:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26189.469025 secs 2023-09-10T06:35:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:58 smithi082 ceph-mon[98694]: pgmap v14127: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T06:35:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26189.469025 secs 2023-09-10T06:35:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:58 smithi098 ceph-mon[103124]: pgmap v14127: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T06:35:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26189.469025 secs 2023-09-10T06:35:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:58 smithi170 ceph-mon[102832]: pgmap v14127: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T06:36:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26190.475745 secs 2023-09-10T06:36:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:35:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26191.088621 secs 2023-09-10T06:36:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26190.475745 secs 2023-09-10T06:36:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:35:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26191.088621 secs 2023-09-10T06:36:00.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26190.475745 secs 2023-09-10T06:36:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:35:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26191.088621 secs 2023-09-10T06:36:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:36:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:36:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:00 smithi098 ceph-mon[103124]: pgmap v14128: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T06:36:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:36:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:36:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:00 smithi170 ceph-mon[102832]: pgmap v14128: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T06:36:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:36:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:36:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:00 smithi082 ceph-mon[98694]: pgmap v14128: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T06:36:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:03 smithi170 ceph-mon[102832]: pgmap v14129: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 442 KiB/s wr, 17 op/s 2023-09-10T06:36:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:03 smithi082 ceph-mon[98694]: pgmap v14129: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 442 KiB/s wr, 17 op/s 2023-09-10T06:36:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:03 smithi098 ceph-mon[103124]: pgmap v14129: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 442 KiB/s wr, 17 op/s 2023-09-10T06:36:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26194.469085 secs 2023-09-10T06:36:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26195.475807 secs 2023-09-10T06:36:04.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26194.469085 secs 2023-09-10T06:36:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26195.475807 secs 2023-09-10T06:36:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26194.469085 secs 2023-09-10T06:36:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26195.475807 secs 2023-09-10T06:36:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26196.088685 secs 2023-09-10T06:36:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:05 smithi170 ceph-mon[102832]: pgmap v14130: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 375 KiB/s wr, 11 op/s 2023-09-10T06:36:05.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26196.088685 secs 2023-09-10T06:36:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:05 smithi082 ceph-mon[98694]: pgmap v14130: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 375 KiB/s wr, 11 op/s 2023-09-10T06:36:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26196.088685 secs 2023-09-10T06:36:05.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:05 smithi098 ceph-mon[103124]: pgmap v14130: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 375 KiB/s wr, 11 op/s 2023-09-10T06:36:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:36:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:36:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:36:06.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:36:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:36:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:36:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:36:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:36:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:36:07.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:07 smithi170 ceph-mon[102832]: pgmap v14131: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 510 KiB/s wr, 31 op/s 2023-09-10T06:36:07.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:36:07.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:36:07.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:36:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:07 smithi082 ceph-mon[98694]: pgmap v14131: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 510 KiB/s wr, 31 op/s 2023-09-10T06:36:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:36:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:36:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:36:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:07 smithi098 ceph-mon[103124]: pgmap v14131: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 510 KiB/s wr, 31 op/s 2023-09-10T06:36:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:36:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:36:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:36:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26199.469195 secs 2023-09-10T06:36:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:09 smithi170 ceph-mon[102832]: pgmap v14132: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 353 KiB/s wr, 25 op/s 2023-09-10T06:36:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:09 smithi170 ceph-mon[102832]: osdmap e951: 12 total, 12 up, 12 in 2023-09-10T06:36:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26200.475870 secs 2023-09-10T06:36:09.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26199.469195 secs 2023-09-10T06:36:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:09 smithi082 ceph-mon[98694]: pgmap v14132: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 353 KiB/s wr, 25 op/s 2023-09-10T06:36:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:09 smithi082 ceph-mon[98694]: osdmap e951: 12 total, 12 up, 12 in 2023-09-10T06:36:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26200.475870 secs 2023-09-10T06:36:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26199.469195 secs 2023-09-10T06:36:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:09 smithi098 ceph-mon[103124]: pgmap v14132: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 353 KiB/s wr, 25 op/s 2023-09-10T06:36:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:09 smithi098 ceph-mon[103124]: osdmap e951: 12 total, 12 up, 12 in 2023-09-10T06:36:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26200.475870 secs 2023-09-10T06:36:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26201.088750 secs 2023-09-10T06:36:10.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26201.088750 secs 2023-09-10T06:36:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26201.088750 secs 2023-09-10T06:36:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:11 smithi170 ceph-mon[102832]: pgmap v14134: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 423 KiB/s wr, 31 op/s 2023-09-10T06:36:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:11 smithi082 ceph-mon[98694]: pgmap v14134: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 423 KiB/s wr, 31 op/s 2023-09-10T06:36:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:11 smithi098 ceph-mon[103124]: pgmap v14134: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 423 KiB/s wr, 31 op/s 2023-09-10T06:36:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:13 smithi170 ceph-mon[102832]: pgmap v14135: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 162 KiB/s wr, 29 op/s 2023-09-10T06:36:13.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:13 smithi082 ceph-mon[98694]: pgmap v14135: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 162 KiB/s wr, 29 op/s 2023-09-10T06:36:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:13 smithi098 ceph-mon[103124]: pgmap v14135: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 162 KiB/s wr, 29 op/s 2023-09-10T06:36:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26204.469254 secs 2023-09-10T06:36:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26205.475953 secs 2023-09-10T06:36:14.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26204.469254 secs 2023-09-10T06:36:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26205.475953 secs 2023-09-10T06:36:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26204.469254 secs 2023-09-10T06:36:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26205.475953 secs 2023-09-10T06:36:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26206.088814 secs 2023-09-10T06:36:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:15 smithi170 ceph-mon[102832]: pgmap v14136: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 162 KiB/s wr, 29 op/s 2023-09-10T06:36:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26206.088814 secs 2023-09-10T06:36:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:15 smithi082 ceph-mon[98694]: pgmap v14136: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 162 KiB/s wr, 29 op/s 2023-09-10T06:36:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26206.088814 secs 2023-09-10T06:36:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:15 smithi098 ceph-mon[103124]: pgmap v14136: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 162 KiB/s wr, 29 op/s 2023-09-10T06:36:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:16 smithi082 ceph-mon[98694]: osdmap e952: 12 total, 12 up, 12 in 2023-09-10T06:36:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:16 smithi082 ceph-mon[98694]: pgmap v14138: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2023-09-10T06:36:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:16 smithi098 ceph-mon[103124]: osdmap e952: 12 total, 12 up, 12 in 2023-09-10T06:36:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:16 smithi098 ceph-mon[103124]: pgmap v14138: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2023-09-10T06:36:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:16 smithi170 ceph-mon[102832]: osdmap e952: 12 total, 12 up, 12 in 2023-09-10T06:36:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:16 smithi170 ceph-mon[102832]: pgmap v14138: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2023-09-10T06:36:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26209.469313 secs 2023-09-10T06:36:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:19 smithi170 ceph-mon[102832]: pgmap v14139: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-09-10T06:36:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26210.476011 secs 2023-09-10T06:36:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26209.469313 secs 2023-09-10T06:36:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:19 smithi082 ceph-mon[98694]: pgmap v14139: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-09-10T06:36:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26210.476011 secs 2023-09-10T06:36:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26209.469313 secs 2023-09-10T06:36:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:19 smithi098 ceph-mon[103124]: pgmap v14139: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-09-10T06:36:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26210.476011 secs 2023-09-10T06:36:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26211.088878 secs 2023-09-10T06:36:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26211.088878 secs 2023-09-10T06:36:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26211.088878 secs 2023-09-10T06:36:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:21 smithi170 ceph-mon[102832]: pgmap v14140: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-09-10T06:36:21.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:21 smithi082 ceph-mon[98694]: pgmap v14140: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-09-10T06:36:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:21 smithi098 ceph-mon[103124]: pgmap v14140: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-09-10T06:36:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:23 smithi170 ceph-mon[102832]: pgmap v14141: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T06:36:23.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:23 smithi082 ceph-mon[98694]: pgmap v14141: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T06:36:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:23 smithi098 ceph-mon[103124]: pgmap v14141: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T06:36:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26214.469371 secs 2023-09-10T06:36:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26215.476095 secs 2023-09-10T06:36:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26214.469371 secs 2023-09-10T06:36:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26215.476095 secs 2023-09-10T06:36:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26214.469371 secs 2023-09-10T06:36:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26215.476095 secs 2023-09-10T06:36:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26216.088941 secs 2023-09-10T06:36:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:25 smithi170 ceph-mon[102832]: pgmap v14142: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T06:36:25.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26216.088941 secs 2023-09-10T06:36:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:25 smithi082 ceph-mon[98694]: pgmap v14142: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T06:36:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26216.088941 secs 2023-09-10T06:36:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:25 smithi098 ceph-mon[103124]: pgmap v14142: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T06:36:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:26 smithi098 ceph-mon[103124]: pgmap v14143: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T06:36:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:26 smithi170 ceph-mon[102832]: pgmap v14143: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T06:36:27.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:26 smithi082 ceph-mon[98694]: pgmap v14143: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 12 op/s 2023-09-10T06:36:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:29 smithi170 ceph-mon[102832]: pgmap v14144: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T06:36:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26220.476155 secs 2023-09-10T06:36:29.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:29 smithi082 ceph-mon[98694]: pgmap v14144: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T06:36:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26220.476155 secs 2023-09-10T06:36:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:29 smithi098 ceph-mon[103124]: pgmap v14144: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T06:36:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26220.476155 secs 2023-09-10T06:36:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26221.089006 secs 2023-09-10T06:36:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:36:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:36:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26221.089006 secs 2023-09-10T06:36:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:36:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:36:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26221.089006 secs 2023-09-10T06:36:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:36:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:36:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:31 smithi170 ceph-mon[102832]: pgmap v14145: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T06:36:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:36:31.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:31 smithi082 ceph-mon[98694]: pgmap v14145: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T06:36:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:36:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:31 smithi098 ceph-mon[103124]: pgmap v14145: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T06:36:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:36:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:36:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:36:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:36:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:36:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:36:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:36:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:36:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:36:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:36:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:33 smithi170 ceph-mon[102832]: pgmap v14146: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 280 KiB/s wr, 15 op/s 2023-09-10T06:36:33.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:33 smithi082 ceph-mon[98694]: pgmap v14146: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 280 KiB/s wr, 15 op/s 2023-09-10T06:36:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:33 smithi098 ceph-mon[103124]: pgmap v14146: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 280 KiB/s wr, 15 op/s 2023-09-10T06:36:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26225.476240 secs 2023-09-10T06:36:34.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26225.476240 secs 2023-09-10T06:36:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26225.476240 secs 2023-09-10T06:36:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26226.089069 secs 2023-09-10T06:36:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:35 smithi170 ceph-mon[102832]: pgmap v14147: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 191 KiB/s wr, 10 op/s 2023-09-10T06:36:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26226.089069 secs 2023-09-10T06:36:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:35 smithi082 ceph-mon[98694]: pgmap v14147: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 191 KiB/s wr, 10 op/s 2023-09-10T06:36:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26226.089069 secs 2023-09-10T06:36:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:35 smithi098 ceph-mon[103124]: pgmap v14147: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 191 KiB/s wr, 10 op/s 2023-09-10T06:36:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:36 smithi098 ceph-mon[103124]: pgmap v14148: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 306 KiB/s wr, 16 op/s 2023-09-10T06:36:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:36 smithi170 ceph-mon[102832]: pgmap v14148: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 306 KiB/s wr, 16 op/s 2023-09-10T06:36:37.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:36 smithi082 ceph-mon[98694]: pgmap v14148: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 306 KiB/s wr, 16 op/s 2023-09-10T06:36:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:39 smithi170 ceph-mon[102832]: pgmap v14149: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 306 KiB/s wr, 11 op/s 2023-09-10T06:36:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26230.476300 secs 2023-09-10T06:36:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:39 smithi082 ceph-mon[98694]: pgmap v14149: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 306 KiB/s wr, 11 op/s 2023-09-10T06:36:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26230.476300 secs 2023-09-10T06:36:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:39 smithi098 ceph-mon[103124]: pgmap v14149: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 306 KiB/s wr, 11 op/s 2023-09-10T06:36:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26230.476300 secs 2023-09-10T06:36:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26231.089131 secs 2023-09-10T06:36:40.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26231.089131 secs 2023-09-10T06:36:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26231.089131 secs 2023-09-10T06:36:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:41 smithi170 ceph-mon[102832]: pgmap v14150: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 306 KiB/s wr, 11 op/s 2023-09-10T06:36:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:41 smithi082 ceph-mon[98694]: pgmap v14150: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 306 KiB/s wr, 11 op/s 2023-09-10T06:36:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:41 smithi098 ceph-mon[103124]: pgmap v14150: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 306 KiB/s wr, 11 op/s 2023-09-10T06:36:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:43 smithi170 ceph-mon[102832]: pgmap v14151: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 347 KiB/s wr, 17 op/s 2023-09-10T06:36:43.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:43 smithi082 ceph-mon[98694]: pgmap v14151: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 347 KiB/s wr, 17 op/s 2023-09-10T06:36:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:43 smithi098 ceph-mon[103124]: pgmap v14151: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 347 KiB/s wr, 17 op/s 2023-09-10T06:36:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:36:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:36:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26235.476359 secs 2023-09-10T06:36:44.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:36:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:36:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26235.476359 secs 2023-09-10T06:36:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:36:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:36:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26235.476359 secs 2023-09-10T06:36:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26236.089192 secs 2023-09-10T06:36:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:45 smithi170 ceph-mon[102832]: pgmap v14152: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T06:36:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26236.089192 secs 2023-09-10T06:36:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:45 smithi082 ceph-mon[98694]: pgmap v14152: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T06:36:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26236.089192 secs 2023-09-10T06:36:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:45 smithi098 ceph-mon[103124]: pgmap v14152: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T06:36:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:36:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:36:46.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:36:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:36:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:36:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:36:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:47 smithi170 ceph-mon[102832]: pgmap v14153: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 497 KiB/s wr, 18 op/s 2023-09-10T06:36:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:47 smithi082 ceph-mon[98694]: pgmap v14153: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 497 KiB/s wr, 18 op/s 2023-09-10T06:36:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:47 smithi098 ceph-mon[103124]: pgmap v14153: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 497 KiB/s wr, 18 op/s 2023-09-10T06:36:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:49 smithi170 ceph-mon[102832]: pgmap v14154: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 382 KiB/s wr, 13 op/s 2023-09-10T06:36:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26240.476417 secs 2023-09-10T06:36:49.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:49 smithi082 ceph-mon[98694]: pgmap v14154: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 382 KiB/s wr, 13 op/s 2023-09-10T06:36:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26240.476417 secs 2023-09-10T06:36:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:49 smithi098 ceph-mon[103124]: pgmap v14154: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 382 KiB/s wr, 13 op/s 2023-09-10T06:36:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26240.476417 secs 2023-09-10T06:36:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26241.089257 secs 2023-09-10T06:36:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26241.089257 secs 2023-09-10T06:36:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26241.089257 secs 2023-09-10T06:36:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:51 smithi082 ceph-mon[98694]: pgmap v14155: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 382 KiB/s wr, 13 op/s 2023-09-10T06:36:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:51 smithi098 ceph-mon[103124]: pgmap v14155: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 382 KiB/s wr, 13 op/s 2023-09-10T06:36:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:51 smithi170 ceph-mon[102832]: pgmap v14155: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 382 KiB/s wr, 13 op/s 2023-09-10T06:36:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:36:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:36:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:52 smithi098 ceph-mon[103124]: pgmap v14156: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 498 KiB/s wr, 19 op/s 2023-09-10T06:36:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:36:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:36:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:52 smithi170 ceph-mon[102832]: pgmap v14156: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 498 KiB/s wr, 19 op/s 2023-09-10T06:36:53.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:36:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:36:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:52 smithi082 ceph-mon[98694]: pgmap v14156: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 498 KiB/s wr, 19 op/s 2023-09-10T06:36:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26244.469767 secs 2023-09-10T06:36:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:36:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:36:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26244.469767 secs 2023-09-10T06:36:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:36:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:36:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26244.469767 secs 2023-09-10T06:36:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:36:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:36:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26245.476505 secs 2023-09-10T06:36:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26246.089322 secs 2023-09-10T06:36:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:54 smithi098 ceph-mon[103124]: pgmap v14157: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 457 KiB/s wr, 13 op/s 2023-09-10T06:36:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26245.476505 secs 2023-09-10T06:36:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26246.089322 secs 2023-09-10T06:36:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:54 smithi170 ceph-mon[102832]: pgmap v14157: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 457 KiB/s wr, 13 op/s 2023-09-10T06:36:55.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26245.476505 secs 2023-09-10T06:36:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26246.089322 secs 2023-09-10T06:36:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:54 smithi082 ceph-mon[98694]: pgmap v14157: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 457 KiB/s wr, 13 op/s 2023-09-10T06:36:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:36:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:36:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:36:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:36:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:36:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:36:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:36:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:36:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:36:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:56 smithi098 ceph-mon[103124]: pgmap v14158: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 704 KiB/s wr, 19 op/s 2023-09-10T06:36:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:56 smithi170 ceph-mon[102832]: pgmap v14158: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 704 KiB/s wr, 19 op/s 2023-09-10T06:36:57.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:56 smithi082 ceph-mon[98694]: pgmap v14158: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 704 KiB/s wr, 19 op/s 2023-09-10T06:36:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26249.469827 secs 2023-09-10T06:36:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:59 smithi170 ceph-mon[102832]: pgmap v14159: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 363 KiB/s wr, 13 op/s 2023-09-10T06:36:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:36:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26250.476562 secs 2023-09-10T06:36:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26249.469827 secs 2023-09-10T06:36:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:59 smithi082 ceph-mon[98694]: pgmap v14159: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 363 KiB/s wr, 13 op/s 2023-09-10T06:36:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:36:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26250.476562 secs 2023-09-10T06:36:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26249.469827 secs 2023-09-10T06:36:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:59 smithi098 ceph-mon[103124]: pgmap v14159: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 363 KiB/s wr, 13 op/s 2023-09-10T06:36:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:36:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26250.476562 secs 2023-09-10T06:37:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26251.089389 secs 2023-09-10T06:37:00.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26251.089389 secs 2023-09-10T06:37:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26251.089389 secs 2023-09-10T06:37:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:01 smithi170 ceph-mon[102832]: pgmap v14160: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 363 KiB/s wr, 13 op/s 2023-09-10T06:37:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:01 smithi082 ceph-mon[98694]: pgmap v14160: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 363 KiB/s wr, 13 op/s 2023-09-10T06:37:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:01 smithi098 ceph-mon[103124]: pgmap v14160: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 363 KiB/s wr, 13 op/s 2023-09-10T06:37:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:03 smithi170 ceph-mon[102832]: pgmap v14161: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 450 KiB/s wr, 19 op/s 2023-09-10T06:37:03.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:03 smithi082 ceph-mon[98694]: pgmap v14161: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 450 KiB/s wr, 19 op/s 2023-09-10T06:37:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:03 smithi098 ceph-mon[103124]: pgmap v14161: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 450 KiB/s wr, 19 op/s 2023-09-10T06:37:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26254.469888 secs 2023-09-10T06:37:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:37:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26255.476619 secs 2023-09-10T06:37:04.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26254.469888 secs 2023-09-10T06:37:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:37:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26255.476619 secs 2023-09-10T06:37:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26254.469888 secs 2023-09-10T06:37:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:37:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26255.476619 secs 2023-09-10T06:37:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26256.089453 secs 2023-09-10T06:37:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:05 smithi170 ceph-mon[102832]: pgmap v14162: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 334 KiB/s wr, 12 op/s 2023-09-10T06:37:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26256.089453 secs 2023-09-10T06:37:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:05 smithi082 ceph-mon[98694]: pgmap v14162: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 334 KiB/s wr, 12 op/s 2023-09-10T06:37:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26256.089453 secs 2023-09-10T06:37:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:05 smithi098 ceph-mon[103124]: pgmap v14162: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 334 KiB/s wr, 12 op/s 2023-09-10T06:37:06.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:37:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:37:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:37:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:07.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:07 smithi082 ceph-mon[98694]: pgmap v14163: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 590 KiB/s wr, 33 op/s 2023-09-10T06:37:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:37:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:37:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:37:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:37:07.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:37:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:07 smithi098 ceph-mon[103124]: pgmap v14163: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 590 KiB/s wr, 33 op/s 2023-09-10T06:37:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:37:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:37:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:37:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:37:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:37:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:07 smithi170 ceph-mon[102832]: pgmap v14163: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 590 KiB/s wr, 33 op/s 2023-09-10T06:37:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:37:07.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:37:07.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:37:07.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:37:07.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:37:08.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:37:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:37:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:37:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:37:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:37:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:37:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:37:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:37:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:37:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26259.469947 secs 2023-09-10T06:37:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:09 smithi082 ceph-mon[98694]: pgmap v14164: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 342 KiB/s wr, 26 op/s 2023-09-10T06:37:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26260.476680 secs 2023-09-10T06:37:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:09 smithi082 ceph-mon[98694]: osdmap e953: 12 total, 12 up, 12 in 2023-09-10T06:37:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26259.469947 secs 2023-09-10T06:37:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:09 smithi098 ceph-mon[103124]: pgmap v14164: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 342 KiB/s wr, 26 op/s 2023-09-10T06:37:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26260.476680 secs 2023-09-10T06:37:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:09 smithi098 ceph-mon[103124]: osdmap e953: 12 total, 12 up, 12 in 2023-09-10T06:37:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26259.469947 secs 2023-09-10T06:37:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:09 smithi170 ceph-mon[102832]: pgmap v14164: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 342 KiB/s wr, 26 op/s 2023-09-10T06:37:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26260.476680 secs 2023-09-10T06:37:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:09 smithi170 ceph-mon[102832]: osdmap e953: 12 total, 12 up, 12 in 2023-09-10T06:37:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26261.089521 secs 2023-09-10T06:37:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26261.089521 secs 2023-09-10T06:37:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26261.089521 secs 2023-09-10T06:37:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:11 smithi082 ceph-mon[98694]: pgmap v14166: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 411 KiB/s wr, 32 op/s 2023-09-10T06:37:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:11 smithi098 ceph-mon[103124]: pgmap v14166: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 411 KiB/s wr, 32 op/s 2023-09-10T06:37:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:11 smithi170 ceph-mon[102832]: pgmap v14166: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 411 KiB/s wr, 32 op/s 2023-09-10T06:37:13.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:13 smithi082 ceph-mon[98694]: pgmap v14167: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 309 KiB/s wr, 30 op/s 2023-09-10T06:37:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:13 smithi098 ceph-mon[103124]: pgmap v14167: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 309 KiB/s wr, 30 op/s 2023-09-10T06:37:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:13 smithi170 ceph-mon[102832]: pgmap v14167: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 309 KiB/s wr, 30 op/s 2023-09-10T06:37:14.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26264.470040 secs 2023-09-10T06:37:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26265.476766 secs 2023-09-10T06:37:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26264.470040 secs 2023-09-10T06:37:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26265.476766 secs 2023-09-10T06:37:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26264.470040 secs 2023-09-10T06:37:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26265.476766 secs 2023-09-10T06:37:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26266.089586 secs 2023-09-10T06:37:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:15 smithi082 ceph-mon[98694]: pgmap v14168: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 309 KiB/s wr, 30 op/s 2023-09-10T06:37:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26266.089586 secs 2023-09-10T06:37:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:15 smithi098 ceph-mon[103124]: pgmap v14168: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 309 KiB/s wr, 30 op/s 2023-09-10T06:37:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26266.089586 secs 2023-09-10T06:37:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:15 smithi170 ceph-mon[102832]: pgmap v14168: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 309 KiB/s wr, 30 op/s 2023-09-10T06:37:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:16 smithi098 ceph-mon[103124]: osdmap e954: 12 total, 12 up, 12 in 2023-09-10T06:37:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:16 smithi098 ceph-mon[103124]: pgmap v14170: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 15 op/s 2023-09-10T06:37:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:16 smithi170 ceph-mon[102832]: osdmap e954: 12 total, 12 up, 12 in 2023-09-10T06:37:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:16 smithi170 ceph-mon[102832]: pgmap v14170: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 15 op/s 2023-09-10T06:37:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:16 smithi082 ceph-mon[98694]: osdmap e954: 12 total, 12 up, 12 in 2023-09-10T06:37:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:16 smithi082 ceph-mon[98694]: pgmap v14170: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 15 op/s 2023-09-10T06:37:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:17 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:37:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:17 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:17 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:37:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:17 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:17 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:37:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:17 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26269.470115 secs 2023-09-10T06:37:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:18 smithi098 ceph-mon[103124]: pgmap v14171: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 12 op/s 2023-09-10T06:37:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26269.470115 secs 2023-09-10T06:37:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:18 smithi170 ceph-mon[102832]: pgmap v14171: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 12 op/s 2023-09-10T06:37:19.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26269.470115 secs 2023-09-10T06:37:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:18 smithi082 ceph-mon[98694]: pgmap v14171: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 12 op/s 2023-09-10T06:37:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26270.476823 secs 2023-09-10T06:37:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26271.089655 secs 2023-09-10T06:37:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:19 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:37:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:19 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26270.476823 secs 2023-09-10T06:37:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26271.089655 secs 2023-09-10T06:37:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:19 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:37:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:19 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:20.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26270.476823 secs 2023-09-10T06:37:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26271.089655 secs 2023-09-10T06:37:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:19 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:37:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:19 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:20 smithi098 ceph-mon[103124]: pgmap v14172: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T06:37:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:20 smithi170 ceph-mon[102832]: pgmap v14172: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T06:37:21.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:20 smithi082 ceph-mon[98694]: pgmap v14172: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 126 KiB/s wr, 12 op/s 2023-09-10T06:37:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:23 smithi170 ceph-mon[102832]: pgmap v14173: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 235 KiB/s wr, 13 op/s 2023-09-10T06:37:23.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:23 smithi082 ceph-mon[98694]: pgmap v14173: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 235 KiB/s wr, 13 op/s 2023-09-10T06:37:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:23 smithi098 ceph-mon[103124]: pgmap v14173: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 235 KiB/s wr, 13 op/s 2023-09-10T06:37:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26274.470187 secs 2023-09-10T06:37:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26275.476908 secs 2023-09-10T06:37:24.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26274.470187 secs 2023-09-10T06:37:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26275.476908 secs 2023-09-10T06:37:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26274.470187 secs 2023-09-10T06:37:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26275.476908 secs 2023-09-10T06:37:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26276.089717 secs 2023-09-10T06:37:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:25 smithi170 ceph-mon[102832]: pgmap v14174: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 235 KiB/s wr, 13 op/s 2023-09-10T06:37:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26276.089717 secs 2023-09-10T06:37:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:25 smithi082 ceph-mon[98694]: pgmap v14174: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 235 KiB/s wr, 13 op/s 2023-09-10T06:37:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26276.089717 secs 2023-09-10T06:37:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:25 smithi098 ceph-mon[103124]: pgmap v14174: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 235 KiB/s wr, 13 op/s 2023-09-10T06:37:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:26 smithi098 ceph-mon[103124]: pgmap v14175: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T06:37:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:26 smithi170 ceph-mon[102832]: pgmap v14175: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T06:37:27.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:26 smithi082 ceph-mon[98694]: pgmap v14175: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T06:37:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:29 smithi170 ceph-mon[102832]: pgmap v14176: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T06:37:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26280.477000 secs 2023-09-10T06:37:29.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:29 smithi082 ceph-mon[98694]: pgmap v14176: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T06:37:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26280.477000 secs 2023-09-10T06:37:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:29 smithi098 ceph-mon[103124]: pgmap v14176: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T06:37:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26280.477000 secs 2023-09-10T06:37:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26281.089781 secs 2023-09-10T06:37:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26281.089781 secs 2023-09-10T06:37:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26281.089781 secs 2023-09-10T06:37:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:31 smithi170 ceph-mon[102832]: pgmap v14177: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T06:37:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:37:31.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:31 smithi082 ceph-mon[98694]: pgmap v14177: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T06:37:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:37:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:31 smithi098 ceph-mon[103124]: pgmap v14177: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T06:37:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:37:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:37:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:37:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:32.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:37:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:37:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:37:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:37:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:33 smithi082 ceph-mon[98694]: pgmap v14178: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 265 KiB/s wr, 15 op/s 2023-09-10T06:37:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:33 smithi098 ceph-mon[103124]: pgmap v14178: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 265 KiB/s wr, 15 op/s 2023-09-10T06:37:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:33 smithi170 ceph-mon[102832]: pgmap v14178: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 265 KiB/s wr, 15 op/s 2023-09-10T06:37:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26285.477087 secs 2023-09-10T06:37:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26285.477087 secs 2023-09-10T06:37:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26285.477087 secs 2023-09-10T06:37:35.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26286.089842 secs 2023-09-10T06:37:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:35 smithi082 ceph-mon[98694]: pgmap v14179: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 173 KiB/s wr, 10 op/s 2023-09-10T06:37:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26286.089842 secs 2023-09-10T06:37:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:35 smithi098 ceph-mon[103124]: pgmap v14179: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 173 KiB/s wr, 10 op/s 2023-09-10T06:37:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26286.089842 secs 2023-09-10T06:37:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:35 smithi170 ceph-mon[102832]: pgmap v14179: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 173 KiB/s wr, 10 op/s 2023-09-10T06:37:36.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:37:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:37:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:37:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:37 smithi082 ceph-mon[98694]: pgmap v14180: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 390 KiB/s wr, 17 op/s 2023-09-10T06:37:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:37 smithi098 ceph-mon[103124]: pgmap v14180: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 390 KiB/s wr, 17 op/s 2023-09-10T06:37:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:37 smithi170 ceph-mon[102832]: pgmap v14180: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 390 KiB/s wr, 17 op/s 2023-09-10T06:37:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:37:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:37:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:37:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:39.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:39 smithi082 ceph-mon[98694]: pgmap v14181: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 390 KiB/s wr, 12 op/s 2023-09-10T06:37:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26290.477140 secs 2023-09-10T06:37:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:39 smithi098 ceph-mon[103124]: pgmap v14181: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 390 KiB/s wr, 12 op/s 2023-09-10T06:37:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26290.477140 secs 2023-09-10T06:37:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:39 smithi170 ceph-mon[102832]: pgmap v14181: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 390 KiB/s wr, 12 op/s 2023-09-10T06:37:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26290.477140 secs 2023-09-10T06:37:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26291.089909 secs 2023-09-10T06:37:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26291.089909 secs 2023-09-10T06:37:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26291.089909 secs 2023-09-10T06:37:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:41 smithi098 ceph-mon[103124]: pgmap v14182: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 390 KiB/s wr, 12 op/s 2023-09-10T06:37:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:41 smithi170 ceph-mon[102832]: pgmap v14182: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 390 KiB/s wr, 12 op/s 2023-09-10T06:37:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:41 smithi082 ceph-mon[98694]: pgmap v14182: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 390 KiB/s wr, 12 op/s 2023-09-10T06:37:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:43 smithi098 ceph-mon[103124]: pgmap v14183: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 407 KiB/s wr, 18 op/s 2023-09-10T06:37:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:43 smithi170 ceph-mon[102832]: pgmap v14183: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 407 KiB/s wr, 18 op/s 2023-09-10T06:37:43.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:43 smithi082 ceph-mon[98694]: pgmap v14183: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 407 KiB/s wr, 18 op/s 2023-09-10T06:37:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26295.477198 secs 2023-09-10T06:37:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:37:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26295.477198 secs 2023-09-10T06:37:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:37:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26295.477198 secs 2023-09-10T06:37:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:37:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26296.089973 secs 2023-09-10T06:37:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:45 smithi098 ceph-mon[103124]: pgmap v14184: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 234 KiB/s wr, 13 op/s 2023-09-10T06:37:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26296.089973 secs 2023-09-10T06:37:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:45 smithi170 ceph-mon[102832]: pgmap v14184: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 234 KiB/s wr, 13 op/s 2023-09-10T06:37:45.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26296.089973 secs 2023-09-10T06:37:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:45 smithi082 ceph-mon[98694]: pgmap v14184: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 234 KiB/s wr, 13 op/s 2023-09-10T06:37:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:37:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:37:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:46.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:37:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:47 smithi098 ceph-mon[103124]: pgmap v14185: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 718 KiB/s wr, 20 op/s 2023-09-10T06:37:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:47 smithi170 ceph-mon[102832]: pgmap v14185: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 718 KiB/s wr, 20 op/s 2023-09-10T06:37:47.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:47 smithi082 ceph-mon[98694]: pgmap v14185: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 718 KiB/s wr, 20 op/s 2023-09-10T06:37:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:49 smithi098 ceph-mon[103124]: pgmap v14186: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 501 KiB/s wr, 13 op/s 2023-09-10T06:37:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26300.477253 secs 2023-09-10T06:37:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:49 smithi170 ceph-mon[102832]: pgmap v14186: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 501 KiB/s wr, 13 op/s 2023-09-10T06:37:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26300.477253 secs 2023-09-10T06:37:49.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:49 smithi082 ceph-mon[98694]: pgmap v14186: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 501 KiB/s wr, 13 op/s 2023-09-10T06:37:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26300.477253 secs 2023-09-10T06:37:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26301.090036 secs 2023-09-10T06:37:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26301.090036 secs 2023-09-10T06:37:50.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26301.090036 secs 2023-09-10T06:37:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:51 smithi170 ceph-mon[102832]: pgmap v14187: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 501 KiB/s wr, 13 op/s 2023-09-10T06:37:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:51 smithi082 ceph-mon[98694]: pgmap v14187: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 501 KiB/s wr, 13 op/s 2023-09-10T06:37:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:51 smithi098 ceph-mon[103124]: pgmap v14187: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 501 KiB/s wr, 13 op/s 2023-09-10T06:37:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:53 smithi170 ceph-mon[102832]: pgmap v14188: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 621 KiB/s wr, 19 op/s 2023-09-10T06:37:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:53 smithi082 ceph-mon[98694]: pgmap v14188: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 621 KiB/s wr, 19 op/s 2023-09-10T06:37:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:53 smithi098 ceph-mon[103124]: pgmap v14188: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 621 KiB/s wr, 19 op/s 2023-09-10T06:37:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26305.477312 secs 2023-09-10T06:37:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26305.477312 secs 2023-09-10T06:37:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26305.477312 secs 2023-09-10T06:37:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26306.090101 secs 2023-09-10T06:37:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:55 smithi170 ceph-mon[102832]: pgmap v14189: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 604 KiB/s wr, 13 op/s 2023-09-10T06:37:55.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26306.090101 secs 2023-09-10T06:37:55.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:55 smithi082 ceph-mon[98694]: pgmap v14189: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 604 KiB/s wr, 13 op/s 2023-09-10T06:37:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26306.090101 secs 2023-09-10T06:37:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:55 smithi098 ceph-mon[103124]: pgmap v14189: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 604 KiB/s wr, 13 op/s 2023-09-10T06:37:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:56 smithi098 ceph-mon[103124]: pgmap v14190: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 695 KiB/s wr, 19 op/s 2023-09-10T06:37:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:56 smithi170 ceph-mon[102832]: pgmap v14190: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 695 KiB/s wr, 19 op/s 2023-09-10T06:37:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:56 smithi082 ceph-mon[98694]: pgmap v14190: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 695 KiB/s wr, 19 op/s 2023-09-10T06:37:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:37:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:37:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:37:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:37:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26309.470719 secs 2023-09-10T06:37:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:58 smithi098 ceph-mon[103124]: pgmap v14191: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 212 KiB/s wr, 11 op/s 2023-09-10T06:37:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26309.470719 secs 2023-09-10T06:37:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:58 smithi170 ceph-mon[102832]: pgmap v14191: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 212 KiB/s wr, 11 op/s 2023-09-10T06:37:59.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26309.470719 secs 2023-09-10T06:37:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:58 smithi082 ceph-mon[98694]: pgmap v14191: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 212 KiB/s wr, 11 op/s 2023-09-10T06:38:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26310.477370 secs 2023-09-10T06:38:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26311.090170 secs 2023-09-10T06:38:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:59 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:38:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:59 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:38:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:37:59 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:38:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26310.477370 secs 2023-09-10T06:38:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26311.090170 secs 2023-09-10T06:38:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:59 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:38:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:59 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:38:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:37:59 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:38:00.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26310.477370 secs 2023-09-10T06:38:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26311.090170 secs 2023-09-10T06:38:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:59 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:38:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:59 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:38:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:37:59 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:38:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:00 smithi098 ceph-mon[103124]: pgmap v14192: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 212 KiB/s wr, 11 op/s 2023-09-10T06:38:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:00 smithi170 ceph-mon[102832]: pgmap v14192: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 212 KiB/s wr, 11 op/s 2023-09-10T06:38:01.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:00 smithi082 ceph-mon[98694]: pgmap v14192: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 212 KiB/s wr, 11 op/s 2023-09-10T06:38:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:03 smithi170 ceph-mon[102832]: pgmap v14193: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 404 KiB/s wr, 17 op/s 2023-09-10T06:38:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:03 smithi082 ceph-mon[98694]: pgmap v14193: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 404 KiB/s wr, 17 op/s 2023-09-10T06:38:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:03 smithi098 ceph-mon[103124]: pgmap v14193: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 404 KiB/s wr, 17 op/s 2023-09-10T06:38:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26314.470775 secs 2023-09-10T06:38:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26315.477430 secs 2023-09-10T06:38:04.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26314.470775 secs 2023-09-10T06:38:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26315.477430 secs 2023-09-10T06:38:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26314.470775 secs 2023-09-10T06:38:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26315.477430 secs 2023-09-10T06:38:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26316.090233 secs 2023-09-10T06:38:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:05 smithi170 ceph-mon[102832]: pgmap v14194: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 284 KiB/s wr, 11 op/s 2023-09-10T06:38:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26316.090233 secs 2023-09-10T06:38:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:05 smithi082 ceph-mon[98694]: pgmap v14194: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 284 KiB/s wr, 11 op/s 2023-09-10T06:38:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26316.090233 secs 2023-09-10T06:38:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:05 smithi098 ceph-mon[103124]: pgmap v14194: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 284 KiB/s wr, 11 op/s 2023-09-10T06:38:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:06 smithi098 ceph-mon[103124]: pgmap v14195: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 418 KiB/s wr, 31 op/s 2023-09-10T06:38:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:06 smithi170 ceph-mon[102832]: pgmap v14195: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 418 KiB/s wr, 31 op/s 2023-09-10T06:38:07.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:06 smithi082 ceph-mon[98694]: pgmap v14195: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 418 KiB/s wr, 31 op/s 2023-09-10T06:38:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:38:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:38:08.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:38:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26319.471098 secs 2023-09-10T06:38:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:08 smithi098 ceph-mon[103124]: osdmap e955: 12 total, 12 up, 12 in 2023-09-10T06:38:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:08 smithi098 ceph-mon[103124]: pgmap v14197: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 392 KiB/s wr, 30 op/s 2023-09-10T06:38:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:38:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:38:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:38:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26319.471098 secs 2023-09-10T06:38:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:08 smithi170 ceph-mon[102832]: osdmap e955: 12 total, 12 up, 12 in 2023-09-10T06:38:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:08 smithi170 ceph-mon[102832]: pgmap v14197: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 392 KiB/s wr, 30 op/s 2023-09-10T06:38:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:38:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:38:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:38:09.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26319.471098 secs 2023-09-10T06:38:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:08 smithi082 ceph-mon[98694]: osdmap e955: 12 total, 12 up, 12 in 2023-09-10T06:38:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:08 smithi082 ceph-mon[98694]: pgmap v14197: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 392 KiB/s wr, 30 op/s 2023-09-10T06:38:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:38:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:38:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:38:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26320.477514 secs 2023-09-10T06:38:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26321.090299 secs 2023-09-10T06:38:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26320.477514 secs 2023-09-10T06:38:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26321.090299 secs 2023-09-10T06:38:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26320.477514 secs 2023-09-10T06:38:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26321.090299 secs 2023-09-10T06:38:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:11 smithi098 ceph-mon[103124]: pgmap v14198: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 392 KiB/s wr, 30 op/s 2023-09-10T06:38:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:11 smithi170 ceph-mon[102832]: pgmap v14198: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 392 KiB/s wr, 30 op/s 2023-09-10T06:38:11.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:11 smithi082 ceph-mon[98694]: pgmap v14198: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 392 KiB/s wr, 30 op/s 2023-09-10T06:38:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:13 smithi170 ceph-mon[102832]: pgmap v14199: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 185 KiB/s wr, 29 op/s 2023-09-10T06:38:13.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:13 smithi082 ceph-mon[98694]: pgmap v14199: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 185 KiB/s wr, 29 op/s 2023-09-10T06:38:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:13 smithi098 ceph-mon[103124]: pgmap v14199: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 185 KiB/s wr, 29 op/s 2023-09-10T06:38:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26324.471207 secs 2023-09-10T06:38:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26325.477597 secs 2023-09-10T06:38:14.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26324.471207 secs 2023-09-10T06:38:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26325.477597 secs 2023-09-10T06:38:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26324.471207 secs 2023-09-10T06:38:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26325.477597 secs 2023-09-10T06:38:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26326.090362 secs 2023-09-10T06:38:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:15 smithi170 ceph-mon[102832]: pgmap v14200: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 185 KiB/s wr, 29 op/s 2023-09-10T06:38:15.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26326.090362 secs 2023-09-10T06:38:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:15 smithi082 ceph-mon[98694]: pgmap v14200: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 185 KiB/s wr, 29 op/s 2023-09-10T06:38:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26326.090362 secs 2023-09-10T06:38:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:15 smithi098 ceph-mon[103124]: pgmap v14200: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 185 KiB/s wr, 29 op/s 2023-09-10T06:38:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:16 smithi098 ceph-mon[103124]: osdmap e956: 12 total, 12 up, 12 in 2023-09-10T06:38:17.270 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:16 smithi098 ceph-mon[103124]: pgmap v14202: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 14 op/s 2023-09-10T06:38:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:16 smithi170 ceph-mon[102832]: osdmap e956: 12 total, 12 up, 12 in 2023-09-10T06:38:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:16 smithi170 ceph-mon[102832]: pgmap v14202: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 14 op/s 2023-09-10T06:38:17.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:16 smithi082 ceph-mon[98694]: osdmap e956: 12 total, 12 up, 12 in 2023-09-10T06:38:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:16 smithi082 ceph-mon[98694]: pgmap v14202: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 14 op/s 2023-09-10T06:38:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26329.471306 secs 2023-09-10T06:38:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:19 smithi170 ceph-mon[102832]: pgmap v14203: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T06:38:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26330.477657 secs 2023-09-10T06:38:19.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26329.471306 secs 2023-09-10T06:38:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:19 smithi082 ceph-mon[98694]: pgmap v14203: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T06:38:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26330.477657 secs 2023-09-10T06:38:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26329.471306 secs 2023-09-10T06:38:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:19 smithi098 ceph-mon[103124]: pgmap v14203: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T06:38:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26330.477657 secs 2023-09-10T06:38:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26331.090424 secs 2023-09-10T06:38:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26331.090424 secs 2023-09-10T06:38:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26331.090424 secs 2023-09-10T06:38:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:21 smithi170 ceph-mon[102832]: pgmap v14204: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T06:38:21.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:21 smithi082 ceph-mon[98694]: pgmap v14204: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T06:38:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:21 smithi098 ceph-mon[103124]: pgmap v14204: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T06:38:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:23 smithi082 ceph-mon[98694]: pgmap v14205: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T06:38:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:23 smithi098 ceph-mon[103124]: pgmap v14205: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T06:38:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:23 smithi170 ceph-mon[102832]: pgmap v14205: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T06:38:24.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26334.471395 secs 2023-09-10T06:38:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26335.477763 secs 2023-09-10T06:38:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26334.471395 secs 2023-09-10T06:38:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26335.477763 secs 2023-09-10T06:38:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26334.471395 secs 2023-09-10T06:38:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26335.477763 secs 2023-09-10T06:38:25.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26336.090486 secs 2023-09-10T06:38:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:25 smithi082 ceph-mon[98694]: pgmap v14206: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T06:38:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26336.090486 secs 2023-09-10T06:38:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:25 smithi098 ceph-mon[103124]: pgmap v14206: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T06:38:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26336.090486 secs 2023-09-10T06:38:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:25 smithi170 ceph-mon[102832]: pgmap v14206: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T06:38:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:26 smithi098 ceph-mon[103124]: pgmap v14207: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 107 KiB/s wr, 12 op/s 2023-09-10T06:38:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:26 smithi170 ceph-mon[102832]: pgmap v14207: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 107 KiB/s wr, 12 op/s 2023-09-10T06:38:27.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:26 smithi082 ceph-mon[98694]: pgmap v14207: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 107 KiB/s wr, 12 op/s 2023-09-10T06:38:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:29 smithi170 ceph-mon[102832]: pgmap v14208: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T06:38:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26340.477822 secs 2023-09-10T06:38:29.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:29 smithi082 ceph-mon[98694]: pgmap v14208: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T06:38:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26340.477822 secs 2023-09-10T06:38:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:29 smithi098 ceph-mon[103124]: pgmap v14208: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T06:38:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26340.477822 secs 2023-09-10T06:38:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26341.090562 secs 2023-09-10T06:38:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:38:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:38:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26341.090562 secs 2023-09-10T06:38:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:38:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:38:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26341.090562 secs 2023-09-10T06:38:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:38:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:38:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:31 smithi082 ceph-mon[98694]: pgmap v14209: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T06:38:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:38:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:31 smithi098 ceph-mon[103124]: pgmap v14209: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T06:38:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:38:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:31 smithi170 ceph-mon[102832]: pgmap v14209: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 10 op/s 2023-09-10T06:38:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:38:32.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:38:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:38:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:38:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:38:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:38:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:38:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:38:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:38:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:38:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:33 smithi082 ceph-mon[98694]: pgmap v14210: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 15 op/s 2023-09-10T06:38:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:33 smithi098 ceph-mon[103124]: pgmap v14210: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 15 op/s 2023-09-10T06:38:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:33 smithi170 ceph-mon[102832]: pgmap v14210: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 15 op/s 2023-09-10T06:38:34.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26345.477881 secs 2023-09-10T06:38:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26345.477881 secs 2023-09-10T06:38:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26345.477881 secs 2023-09-10T06:38:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26346.090626 secs 2023-09-10T06:38:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:35 smithi082 ceph-mon[98694]: pgmap v14211: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 192 KiB/s wr, 9 op/s 2023-09-10T06:38:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26346.090626 secs 2023-09-10T06:38:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:35 smithi098 ceph-mon[103124]: pgmap v14211: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 192 KiB/s wr, 9 op/s 2023-09-10T06:38:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26346.090626 secs 2023-09-10T06:38:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:35 smithi170 ceph-mon[102832]: pgmap v14211: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 192 KiB/s wr, 9 op/s 2023-09-10T06:38:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:36 smithi098 ceph-mon[103124]: pgmap v14212: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 299 KiB/s wr, 15 op/s 2023-09-10T06:38:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:36 smithi170 ceph-mon[102832]: pgmap v14212: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 299 KiB/s wr, 15 op/s 2023-09-10T06:38:37.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:36 smithi082 ceph-mon[98694]: pgmap v14212: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 299 KiB/s wr, 15 op/s 2023-09-10T06:38:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:38:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:38:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:38:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:38:38.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:38:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:38:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:39 smithi098 ceph-mon[103124]: pgmap v14213: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 299 KiB/s wr, 11 op/s 2023-09-10T06:38:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:38 smithi170 ceph-mon[102832]: pgmap v14213: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 299 KiB/s wr, 11 op/s 2023-09-10T06:38:39.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:38 smithi082 ceph-mon[98694]: pgmap v14213: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 299 KiB/s wr, 11 op/s 2023-09-10T06:38:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26350.477965 secs 2023-09-10T06:38:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26351.090692 secs 2023-09-10T06:38:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:38:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:38:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26350.477965 secs 2023-09-10T06:38:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26351.090692 secs 2023-09-10T06:38:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:38:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:38:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26350.477965 secs 2023-09-10T06:38:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26351.090692 secs 2023-09-10T06:38:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:38:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:38:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:41 smithi170 ceph-mon[102832]: pgmap v14214: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 299 KiB/s wr, 11 op/s 2023-09-10T06:38:41.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:41 smithi082 ceph-mon[98694]: pgmap v14214: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 299 KiB/s wr, 11 op/s 2023-09-10T06:38:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:41 smithi098 ceph-mon[103124]: pgmap v14214: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 299 KiB/s wr, 11 op/s 2023-09-10T06:38:43.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:43 smithi082 ceph-mon[98694]: pgmap v14215: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 350 KiB/s wr, 18 op/s 2023-09-10T06:38:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:43 smithi098 ceph-mon[103124]: pgmap v14215: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 350 KiB/s wr, 18 op/s 2023-09-10T06:38:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:43 smithi170 ceph-mon[102832]: pgmap v14215: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 350 KiB/s wr, 18 op/s 2023-09-10T06:38:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26355.478047 secs 2023-09-10T06:38:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26355.478047 secs 2023-09-10T06:38:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26355.478047 secs 2023-09-10T06:38:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26356.090781 secs 2023-09-10T06:38:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:45 smithi082 ceph-mon[98694]: pgmap v14216: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T06:38:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26356.090781 secs 2023-09-10T06:38:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:45 smithi098 ceph-mon[103124]: pgmap v14216: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T06:38:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26356.090781 secs 2023-09-10T06:38:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:45 smithi170 ceph-mon[102832]: pgmap v14216: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T06:38:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:46 smithi098 ceph-mon[103124]: pgmap v14217: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 491 KiB/s wr, 19 op/s 2023-09-10T06:38:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:46 smithi170 ceph-mon[102832]: pgmap v14217: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 491 KiB/s wr, 19 op/s 2023-09-10T06:38:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:46 smithi082 ceph-mon[98694]: pgmap v14217: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 491 KiB/s wr, 19 op/s 2023-09-10T06:38:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:49 smithi082 ceph-mon[98694]: pgmap v14218: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 385 KiB/s wr, 13 op/s 2023-09-10T06:38:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26360.478113 secs 2023-09-10T06:38:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:49 smithi098 ceph-mon[103124]: pgmap v14218: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 385 KiB/s wr, 13 op/s 2023-09-10T06:38:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26360.478113 secs 2023-09-10T06:38:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:49 smithi170 ceph-mon[102832]: pgmap v14218: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 385 KiB/s wr, 13 op/s 2023-09-10T06:38:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26360.478113 secs 2023-09-10T06:38:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26361.090846 secs 2023-09-10T06:38:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:38:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:38:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26361.090846 secs 2023-09-10T06:38:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:38:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:38:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26361.090846 secs 2023-09-10T06:38:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:38:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:38:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:51 smithi082 ceph-mon[98694]: pgmap v14219: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 385 KiB/s wr, 13 op/s 2023-09-10T06:38:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:51 smithi098 ceph-mon[103124]: pgmap v14219: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 385 KiB/s wr, 13 op/s 2023-09-10T06:38:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:51 smithi170 ceph-mon[102832]: pgmap v14219: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 385 KiB/s wr, 13 op/s 2023-09-10T06:38:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:38:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:38:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:38:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:38:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:38:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:38:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:53 smithi098 ceph-mon[103124]: pgmap v14220: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 428 KiB/s wr, 19 op/s 2023-09-10T06:38:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:53 smithi170 ceph-mon[102832]: pgmap v14220: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 428 KiB/s wr, 19 op/s 2023-09-10T06:38:53.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:53 smithi082 ceph-mon[98694]: pgmap v14220: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 428 KiB/s wr, 19 op/s 2023-09-10T06:38:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:38:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:38:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26365.478173 secs 2023-09-10T06:38:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:38:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:38:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26365.478173 secs 2023-09-10T06:38:54.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:38:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:38:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26365.478173 secs 2023-09-10T06:38:55.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26366.090909 secs 2023-09-10T06:38:55.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:55 smithi098 ceph-mon[103124]: pgmap v14221: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 376 KiB/s wr, 12 op/s 2023-09-10T06:38:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26366.090909 secs 2023-09-10T06:38:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:55 smithi170 ceph-mon[102832]: pgmap v14221: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 376 KiB/s wr, 12 op/s 2023-09-10T06:38:55.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26366.090909 secs 2023-09-10T06:38:55.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:55 smithi082 ceph-mon[98694]: pgmap v14221: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 376 KiB/s wr, 12 op/s 2023-09-10T06:38:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:38:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:38:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:38:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:38:56.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:38:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:38:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:57 smithi098 ceph-mon[103124]: pgmap v14222: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 965 KiB/s wr, 20 op/s 2023-09-10T06:38:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:57 smithi170 ceph-mon[102832]: pgmap v14222: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 965 KiB/s wr, 20 op/s 2023-09-10T06:38:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:57 smithi082 ceph-mon[98694]: pgmap v14222: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 965 KiB/s wr, 20 op/s 2023-09-10T06:38:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26369.471885 secs 2023-09-10T06:38:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:59 smithi098 ceph-mon[103124]: pgmap v14223: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 631 KiB/s wr, 13 op/s 2023-09-10T06:38:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:38:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26370.478232 secs 2023-09-10T06:38:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26369.471885 secs 2023-09-10T06:38:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:59 smithi170 ceph-mon[102832]: pgmap v14223: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 631 KiB/s wr, 13 op/s 2023-09-10T06:38:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:38:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26370.478232 secs 2023-09-10T06:38:59.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26369.471885 secs 2023-09-10T06:38:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:59 smithi082 ceph-mon[98694]: pgmap v14223: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 631 KiB/s wr, 13 op/s 2023-09-10T06:38:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:38:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26370.478232 secs 2023-09-10T06:39:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26371.090978 secs 2023-09-10T06:39:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:39:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:39:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26371.090978 secs 2023-09-10T06:39:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:39:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:39:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:00.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26371.090978 secs 2023-09-10T06:39:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:39:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:39:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:01 smithi098 ceph-mon[103124]: pgmap v14224: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 631 KiB/s wr, 13 op/s 2023-09-10T06:39:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:01 smithi170 ceph-mon[102832]: pgmap v14224: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 631 KiB/s wr, 13 op/s 2023-09-10T06:39:01.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:01 smithi082 ceph-mon[98694]: pgmap v14224: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 631 KiB/s wr, 13 op/s 2023-09-10T06:39:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:03 smithi098 ceph-mon[103124]: pgmap v14225: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 761 KiB/s wr, 33 op/s 2023-09-10T06:39:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:03 smithi170 ceph-mon[102832]: pgmap v14225: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 761 KiB/s wr, 33 op/s 2023-09-10T06:39:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:03 smithi082 ceph-mon[98694]: pgmap v14225: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 761 KiB/s wr, 33 op/s 2023-09-10T06:39:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26374.471944 secs 2023-09-10T06:39:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26375.478291 secs 2023-09-10T06:39:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:39:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:04 smithi098 ceph-mon[103124]: osdmap e957: 12 total, 12 up, 12 in 2023-09-10T06:39:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26374.471944 secs 2023-09-10T06:39:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26375.478291 secs 2023-09-10T06:39:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:39:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:04 smithi170 ceph-mon[102832]: osdmap e957: 12 total, 12 up, 12 in 2023-09-10T06:39:04.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26374.471944 secs 2023-09-10T06:39:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26375.478291 secs 2023-09-10T06:39:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:39:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:04 smithi082 ceph-mon[98694]: osdmap e957: 12 total, 12 up, 12 in 2023-09-10T06:39:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26376.091042 secs 2023-09-10T06:39:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:05 smithi170 ceph-mon[102832]: pgmap v14227: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 861 KiB/s wr, 33 op/s 2023-09-10T06:39:05.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26376.091042 secs 2023-09-10T06:39:05.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:05 smithi082 ceph-mon[98694]: pgmap v14227: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 861 KiB/s wr, 33 op/s 2023-09-10T06:39:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26376.091042 secs 2023-09-10T06:39:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:05 smithi098 ceph-mon[103124]: pgmap v14227: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 861 KiB/s wr, 33 op/s 2023-09-10T06:39:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:39:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:06.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:39:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:39:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:07 smithi170 ceph-mon[102832]: pgmap v14228: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 309 KiB/s wr, 31 op/s 2023-09-10T06:39:07.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:07 smithi082 ceph-mon[98694]: pgmap v14228: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 309 KiB/s wr, 31 op/s 2023-09-10T06:39:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:07 smithi098 ceph-mon[103124]: pgmap v14228: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 309 KiB/s wr, 31 op/s 2023-09-10T06:39:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26379.472006 secs 2023-09-10T06:39:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:09 smithi170 ceph-mon[102832]: pgmap v14229: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 309 KiB/s wr, 31 op/s 2023-09-10T06:39:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26380.478350 secs 2023-09-10T06:39:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:39:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:39:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:39:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:39:09.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26379.472006 secs 2023-09-10T06:39:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:09 smithi082 ceph-mon[98694]: pgmap v14229: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 309 KiB/s wr, 31 op/s 2023-09-10T06:39:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26380.478350 secs 2023-09-10T06:39:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:39:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:39:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:39:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:39:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26379.472006 secs 2023-09-10T06:39:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:09 smithi098 ceph-mon[103124]: pgmap v14229: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 309 KiB/s wr, 31 op/s 2023-09-10T06:39:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26380.478350 secs 2023-09-10T06:39:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:39:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:39:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:39:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:39:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26381.091106 secs 2023-09-10T06:39:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26381.091106 secs 2023-09-10T06:39:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26381.091106 secs 2023-09-10T06:39:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:11 smithi170 ceph-mon[102832]: pgmap v14230: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 309 KiB/s wr, 31 op/s 2023-09-10T06:39:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:11 smithi170 ceph-mon[102832]: osdmap e958: 12 total, 12 up, 12 in 2023-09-10T06:39:11.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:11 smithi082 ceph-mon[98694]: pgmap v14230: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 309 KiB/s wr, 31 op/s 2023-09-10T06:39:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:11 smithi082 ceph-mon[98694]: osdmap e958: 12 total, 12 up, 12 in 2023-09-10T06:39:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:11 smithi098 ceph-mon[103124]: pgmap v14230: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 309 KiB/s wr, 31 op/s 2023-09-10T06:39:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:11 smithi098 ceph-mon[103124]: osdmap e958: 12 total, 12 up, 12 in 2023-09-10T06:39:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:13 smithi170 ceph-mon[102832]: pgmap v14232: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 269 KiB/s wr, 14 op/s 2023-09-10T06:39:13.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:13 smithi082 ceph-mon[98694]: pgmap v14232: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 269 KiB/s wr, 14 op/s 2023-09-10T06:39:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:13 smithi098 ceph-mon[103124]: pgmap v14232: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 269 KiB/s wr, 14 op/s 2023-09-10T06:39:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26384.472089 secs 2023-09-10T06:39:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26385.478410 secs 2023-09-10T06:39:14.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26384.472089 secs 2023-09-10T06:39:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26385.478410 secs 2023-09-10T06:39:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26384.472089 secs 2023-09-10T06:39:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26385.478410 secs 2023-09-10T06:39:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26386.091171 secs 2023-09-10T06:39:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:15 smithi170 ceph-mon[102832]: pgmap v14233: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 239 KiB/s wr, 13 op/s 2023-09-10T06:39:15.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26386.091171 secs 2023-09-10T06:39:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:15 smithi082 ceph-mon[98694]: pgmap v14233: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 239 KiB/s wr, 13 op/s 2023-09-10T06:39:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26386.091171 secs 2023-09-10T06:39:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:15 smithi098 ceph-mon[103124]: pgmap v14233: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 239 KiB/s wr, 13 op/s 2023-09-10T06:39:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:17 smithi170 ceph-mon[102832]: pgmap v14234: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 12 op/s 2023-09-10T06:39:17.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:17 smithi082 ceph-mon[98694]: pgmap v14234: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 12 op/s 2023-09-10T06:39:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:17 smithi098 ceph-mon[103124]: pgmap v14234: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 12 op/s 2023-09-10T06:39:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26389.472149 secs 2023-09-10T06:39:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:19 smithi082 ceph-mon[98694]: pgmap v14235: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 12 op/s 2023-09-10T06:39:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26390.478468 secs 2023-09-10T06:39:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26389.472149 secs 2023-09-10T06:39:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:19 smithi098 ceph-mon[103124]: pgmap v14235: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 12 op/s 2023-09-10T06:39:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26390.478468 secs 2023-09-10T06:39:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26389.472149 secs 2023-09-10T06:39:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:19 smithi170 ceph-mon[102832]: pgmap v14235: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 12 op/s 2023-09-10T06:39:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26390.478468 secs 2023-09-10T06:39:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26391.091237 secs 2023-09-10T06:39:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26391.091237 secs 2023-09-10T06:39:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26391.091237 secs 2023-09-10T06:39:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:21 smithi082 ceph-mon[98694]: pgmap v14236: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 12 op/s 2023-09-10T06:39:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:21 smithi098 ceph-mon[103124]: pgmap v14236: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 12 op/s 2023-09-10T06:39:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:21 smithi170 ceph-mon[102832]: pgmap v14236: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 12 op/s 2023-09-10T06:39:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:23 smithi082 ceph-mon[98694]: pgmap v14237: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 10 op/s 2023-09-10T06:39:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:23 smithi098 ceph-mon[103124]: pgmap v14237: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 10 op/s 2023-09-10T06:39:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:23 smithi170 ceph-mon[102832]: pgmap v14237: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 10 op/s 2023-09-10T06:39:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26394.472259 secs 2023-09-10T06:39:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26395.478555 secs 2023-09-10T06:39:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26394.472259 secs 2023-09-10T06:39:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26395.478555 secs 2023-09-10T06:39:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26394.472259 secs 2023-09-10T06:39:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26395.478555 secs 2023-09-10T06:39:25.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26396.091301 secs 2023-09-10T06:39:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:25 smithi082 ceph-mon[98694]: pgmap v14238: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T06:39:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26396.091301 secs 2023-09-10T06:39:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:25 smithi098 ceph-mon[103124]: pgmap v14238: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T06:39:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26396.091301 secs 2023-09-10T06:39:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:25 smithi170 ceph-mon[102832]: pgmap v14238: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T06:39:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:27 smithi170 ceph-mon[102832]: pgmap v14239: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 91 KiB/s wr, 14 op/s 2023-09-10T06:39:27.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:27 smithi082 ceph-mon[98694]: pgmap v14239: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 91 KiB/s wr, 14 op/s 2023-09-10T06:39:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:27 smithi098 ceph-mon[103124]: pgmap v14239: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 91 KiB/s wr, 14 op/s 2023-09-10T06:39:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:29 smithi082 ceph-mon[98694]: pgmap v14240: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 9 op/s 2023-09-10T06:39:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26400.478646 secs 2023-09-10T06:39:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:29 smithi098 ceph-mon[103124]: pgmap v14240: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 9 op/s 2023-09-10T06:39:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26400.478646 secs 2023-09-10T06:39:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:29 smithi170 ceph-mon[102832]: pgmap v14240: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 9 op/s 2023-09-10T06:39:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26400.478646 secs 2023-09-10T06:39:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26401.091364 secs 2023-09-10T06:39:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26401.091364 secs 2023-09-10T06:39:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26401.091364 secs 2023-09-10T06:39:31.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:31 smithi082 ceph-mon[98694]: pgmap v14241: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 9 op/s 2023-09-10T06:39:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:39:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:31 smithi098 ceph-mon[103124]: pgmap v14241: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 9 op/s 2023-09-10T06:39:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:39:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:31 smithi170 ceph-mon[102832]: pgmap v14241: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 9 op/s 2023-09-10T06:39:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:39:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:39:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:39:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:39:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:39:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:32.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:39:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:39:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:33 smithi098 ceph-mon[103124]: pgmap v14242: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 265 KiB/s wr, 15 op/s 2023-09-10T06:39:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:33 smithi170 ceph-mon[102832]: pgmap v14242: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 265 KiB/s wr, 15 op/s 2023-09-10T06:39:33.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:33 smithi082 ceph-mon[98694]: pgmap v14242: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 265 KiB/s wr, 15 op/s 2023-09-10T06:39:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:39:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26405.478705 secs 2023-09-10T06:39:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:39:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26405.478705 secs 2023-09-10T06:39:34.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:39:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26405.478705 secs 2023-09-10T06:39:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26406.091431 secs 2023-09-10T06:39:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:35 smithi098 ceph-mon[103124]: pgmap v14243: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T06:39:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26406.091431 secs 2023-09-10T06:39:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:35 smithi170 ceph-mon[102832]: pgmap v14243: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T06:39:35.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26406.091431 secs 2023-09-10T06:39:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:35 smithi082 ceph-mon[98694]: pgmap v14243: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 10 op/s 2023-09-10T06:39:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:37 smithi170 ceph-mon[102832]: pgmap v14244: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 393 KiB/s wr, 17 op/s 2023-09-10T06:39:37.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:37 smithi082 ceph-mon[98694]: pgmap v14244: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 393 KiB/s wr, 17 op/s 2023-09-10T06:39:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:37 smithi098 ceph-mon[103124]: pgmap v14244: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 393 KiB/s wr, 17 op/s 2023-09-10T06:39:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:39 smithi082 ceph-mon[98694]: pgmap v14245: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 393 KiB/s wr, 12 op/s 2023-09-10T06:39:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26410.478764 secs 2023-09-10T06:39:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:39 smithi098 ceph-mon[103124]: pgmap v14245: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 393 KiB/s wr, 12 op/s 2023-09-10T06:39:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26410.478764 secs 2023-09-10T06:39:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:39 smithi170 ceph-mon[102832]: pgmap v14245: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 393 KiB/s wr, 12 op/s 2023-09-10T06:39:39.724 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26410.478764 secs 2023-09-10T06:39:40.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26411.091490 secs 2023-09-10T06:39:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26411.091490 secs 2023-09-10T06:39:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26411.091490 secs 2023-09-10T06:39:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:41 smithi082 ceph-mon[98694]: pgmap v14246: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 393 KiB/s wr, 12 op/s 2023-09-10T06:39:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:41 smithi098 ceph-mon[103124]: pgmap v14246: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 393 KiB/s wr, 12 op/s 2023-09-10T06:39:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:41 smithi170 ceph-mon[102832]: pgmap v14246: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 393 KiB/s wr, 12 op/s 2023-09-10T06:39:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:43 smithi098 ceph-mon[103124]: pgmap v14247: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 444 KiB/s wr, 17 op/s 2023-09-10T06:39:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:43 smithi170 ceph-mon[102832]: pgmap v14247: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 444 KiB/s wr, 17 op/s 2023-09-10T06:39:43.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:43 smithi082 ceph-mon[98694]: pgmap v14247: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 444 KiB/s wr, 17 op/s 2023-09-10T06:39:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:39:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26415.478824 secs 2023-09-10T06:39:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:39:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26415.478824 secs 2023-09-10T06:39:44.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:39:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26415.478824 secs 2023-09-10T06:39:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26416.091552 secs 2023-09-10T06:39:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:45 smithi098 ceph-mon[103124]: pgmap v14248: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T06:39:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26416.091552 secs 2023-09-10T06:39:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:45 smithi170 ceph-mon[102832]: pgmap v14248: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T06:39:45.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26416.091552 secs 2023-09-10T06:39:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:45 smithi082 ceph-mon[98694]: pgmap v14248: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T06:39:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:39:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:39:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:46.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:39:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:47 smithi098 ceph-mon[103124]: pgmap v14249: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 723 KiB/s wr, 19 op/s 2023-09-10T06:39:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:47 smithi170 ceph-mon[102832]: pgmap v14249: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 723 KiB/s wr, 19 op/s 2023-09-10T06:39:47.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:47 smithi082 ceph-mon[98694]: pgmap v14249: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 723 KiB/s wr, 19 op/s 2023-09-10T06:39:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:49 smithi098 ceph-mon[103124]: pgmap v14250: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 504 KiB/s wr, 12 op/s 2023-09-10T06:39:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26420.478881 secs 2023-09-10T06:39:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:49 smithi170 ceph-mon[102832]: pgmap v14250: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 504 KiB/s wr, 12 op/s 2023-09-10T06:39:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26420.478881 secs 2023-09-10T06:39:49.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:49 smithi082 ceph-mon[98694]: pgmap v14250: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 504 KiB/s wr, 12 op/s 2023-09-10T06:39:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26420.478881 secs 2023-09-10T06:39:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26421.091618 secs 2023-09-10T06:39:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:39:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26421.091618 secs 2023-09-10T06:39:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:39:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:50.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26421.091618 secs 2023-09-10T06:39:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:39:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:51 smithi098 ceph-mon[103124]: pgmap v14251: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 504 KiB/s wr, 12 op/s 2023-09-10T06:39:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:51 smithi170 ceph-mon[102832]: pgmap v14251: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 504 KiB/s wr, 12 op/s 2023-09-10T06:39:51.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:51 smithi082 ceph-mon[98694]: pgmap v14251: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 504 KiB/s wr, 12 op/s 2023-09-10T06:39:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:39:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:39:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:52.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:39:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:53 smithi170 ceph-mon[102832]: pgmap v14252: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 541 KiB/s wr, 18 op/s 2023-09-10T06:39:53.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:53 smithi082 ceph-mon[98694]: pgmap v14252: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 541 KiB/s wr, 18 op/s 2023-09-10T06:39:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:53 smithi098 ceph-mon[103124]: pgmap v14252: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 541 KiB/s wr, 18 op/s 2023-09-10T06:39:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26425.478943 secs 2023-09-10T06:39:54.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26425.478943 secs 2023-09-10T06:39:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26425.478943 secs 2023-09-10T06:39:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26426.091679 secs 2023-09-10T06:39:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:55 smithi170 ceph-mon[102832]: pgmap v14253: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 489 KiB/s wr, 12 op/s 2023-09-10T06:39:55.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26426.091679 secs 2023-09-10T06:39:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:55 smithi082 ceph-mon[98694]: pgmap v14253: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 489 KiB/s wr, 12 op/s 2023-09-10T06:39:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26426.091679 secs 2023-09-10T06:39:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:55 smithi098 ceph-mon[103124]: pgmap v14253: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 489 KiB/s wr, 12 op/s 2023-09-10T06:39:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:57 smithi170 ceph-mon[102832]: pgmap v14254: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 666 KiB/s wr, 17 op/s 2023-09-10T06:39:57.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:57 smithi082 ceph-mon[98694]: pgmap v14254: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 666 KiB/s wr, 17 op/s 2023-09-10T06:39:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:57 smithi098 ceph-mon[103124]: pgmap v14254: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 666 KiB/s wr, 17 op/s 2023-09-10T06:39:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:39:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:58.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:39:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:39:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:39:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26429.472689 secs 2023-09-10T06:39:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:39:59 smithi170 ceph-mon[102832]: pgmap v14255: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 213 KiB/s wr, 10 op/s 2023-09-10T06:39:59.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26429.472689 secs 2023-09-10T06:39:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:39:59 smithi082 ceph-mon[98694]: pgmap v14255: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 213 KiB/s wr, 10 op/s 2023-09-10T06:39:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26429.472689 secs 2023-09-10T06:39:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:39:59 smithi098 ceph-mon[103124]: pgmap v14255: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 213 KiB/s wr, 10 op/s 2023-09-10T06:40:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26430.479002 secs 2023-09-10T06:40:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26431.091743 secs 2023-09-10T06:40:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:40:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:40:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:40:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T06:40:00.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26430.479002 secs 2023-09-10T06:40:00.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26431.091743 secs 2023-09-10T06:40:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:40:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:40:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:40:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T06:40:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26430.479002 secs 2023-09-10T06:40:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26431.091743 secs 2023-09-10T06:40:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:40:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:40:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:40:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T06:40:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:01 smithi170 ceph-mon[102832]: pgmap v14256: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 213 KiB/s wr, 10 op/s 2023-09-10T06:40:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:01 smithi082 ceph-mon[98694]: pgmap v14256: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 213 KiB/s wr, 10 op/s 2023-09-10T06:40:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:01 smithi098 ceph-mon[103124]: pgmap v14256: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 213 KiB/s wr, 10 op/s 2023-09-10T06:40:03.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:03 smithi082 ceph-mon[98694]: pgmap v14257: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 460 KiB/s wr, 31 op/s 2023-09-10T06:40:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:03 smithi098 ceph-mon[103124]: pgmap v14257: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 460 KiB/s wr, 31 op/s 2023-09-10T06:40:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:03 smithi170 ceph-mon[102832]: pgmap v14257: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 460 KiB/s wr, 31 op/s 2023-09-10T06:40:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26434.472747 secs 2023-09-10T06:40:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26435.479060 secs 2023-09-10T06:40:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26434.472747 secs 2023-09-10T06:40:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26435.479060 secs 2023-09-10T06:40:04.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26434.472747 secs 2023-09-10T06:40:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26435.479060 secs 2023-09-10T06:40:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26436.091802 secs 2023-09-10T06:40:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:05 smithi098 ceph-mon[103124]: pgmap v14258: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 423 KiB/s wr, 25 op/s 2023-09-10T06:40:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26436.091802 secs 2023-09-10T06:40:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:05 smithi170 ceph-mon[102832]: pgmap v14258: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 423 KiB/s wr, 25 op/s 2023-09-10T06:40:05.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26436.091802 secs 2023-09-10T06:40:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:05 smithi082 ceph-mon[98694]: pgmap v14258: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 423 KiB/s wr, 25 op/s 2023-09-10T06:40:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:07 smithi170 ceph-mon[102832]: pgmap v14259: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 524 KiB/s wr, 31 op/s 2023-09-10T06:40:07.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:07 smithi082 ceph-mon[98694]: pgmap v14259: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 524 KiB/s wr, 31 op/s 2023-09-10T06:40:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:07 smithi098 ceph-mon[103124]: pgmap v14259: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 524 KiB/s wr, 31 op/s 2023-09-10T06:40:09.331 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26439.472775 secs 2023-09-10T06:40:09.332 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:09 smithi082 ceph-mon[98694]: osdmap e959: 12 total, 12 up, 12 in 2023-09-10T06:40:09.332 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:09 smithi082 ceph-mon[98694]: pgmap v14261: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 417 KiB/s wr, 31 op/s 2023-09-10T06:40:09.334 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26439.472775 secs 2023-09-10T06:40:09.335 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:09 smithi170 ceph-mon[102832]: osdmap e959: 12 total, 12 up, 12 in 2023-09-10T06:40:09.335 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:09 smithi170 ceph-mon[102832]: pgmap v14261: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 417 KiB/s wr, 31 op/s 2023-09-10T06:40:09.368 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26439.472775 secs 2023-09-10T06:40:09.368 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:09 smithi098 ceph-mon[103124]: osdmap e959: 12 total, 12 up, 12 in 2023-09-10T06:40:09.368 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:09 smithi098 ceph-mon[103124]: pgmap v14261: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 417 KiB/s wr, 31 op/s 2023-09-10T06:40:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26440.479150 secs 2023-09-10T06:40:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26441.091870 secs 2023-09-10T06:40:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:40:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:40:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:40:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:40:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26440.479150 secs 2023-09-10T06:40:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26441.091870 secs 2023-09-10T06:40:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:40:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:40:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:40:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:40:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26440.479150 secs 2023-09-10T06:40:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26441.091870 secs 2023-09-10T06:40:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:40:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:40:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:40:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:40:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:11 smithi170 ceph-mon[102832]: pgmap v14262: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 417 KiB/s wr, 31 op/s 2023-09-10T06:40:11.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:11 smithi082 ceph-mon[98694]: pgmap v14262: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 417 KiB/s wr, 31 op/s 2023-09-10T06:40:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:11 smithi098 ceph-mon[103124]: pgmap v14262: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 417 KiB/s wr, 31 op/s 2023-09-10T06:40:13.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:13 smithi082 ceph-mon[98694]: pgmap v14263: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 140 KiB/s wr, 14 op/s 2023-09-10T06:40:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:13 smithi098 ceph-mon[103124]: pgmap v14263: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 140 KiB/s wr, 14 op/s 2023-09-10T06:40:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:13 smithi170 ceph-mon[102832]: pgmap v14263: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 140 KiB/s wr, 14 op/s 2023-09-10T06:40:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26444.472836 secs 2023-09-10T06:40:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26445.479211 secs 2023-09-10T06:40:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26444.472836 secs 2023-09-10T06:40:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26445.479211 secs 2023-09-10T06:40:14.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26444.472836 secs 2023-09-10T06:40:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26445.479211 secs 2023-09-10T06:40:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26446.091931 secs 2023-09-10T06:40:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:15 smithi098 ceph-mon[103124]: pgmap v14264: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 140 KiB/s wr, 14 op/s 2023-09-10T06:40:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26446.091931 secs 2023-09-10T06:40:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:15 smithi170 ceph-mon[102832]: pgmap v14264: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 140 KiB/s wr, 14 op/s 2023-09-10T06:40:15.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26446.091931 secs 2023-09-10T06:40:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:15 smithi082 ceph-mon[98694]: pgmap v14264: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 140 KiB/s wr, 14 op/s 2023-09-10T06:40:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:17 smithi170 ceph-mon[102832]: osdmap e960: 12 total, 12 up, 12 in 2023-09-10T06:40:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:17 smithi170 ceph-mon[102832]: pgmap v14266: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 157 KiB/s wr, 16 op/s 2023-09-10T06:40:17.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:17 smithi082 ceph-mon[98694]: osdmap e960: 12 total, 12 up, 12 in 2023-09-10T06:40:17.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:17 smithi082 ceph-mon[98694]: pgmap v14266: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 157 KiB/s wr, 16 op/s 2023-09-10T06:40:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:17 smithi098 ceph-mon[103124]: osdmap e960: 12 total, 12 up, 12 in 2023-09-10T06:40:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:17 smithi098 ceph-mon[103124]: pgmap v14266: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 157 KiB/s wr, 16 op/s 2023-09-10T06:40:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26449.472898 secs 2023-09-10T06:40:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:19 smithi082 ceph-mon[98694]: pgmap v14267: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 128 KiB/s wr, 13 op/s 2023-09-10T06:40:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26450.479269 secs 2023-09-10T06:40:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26449.472898 secs 2023-09-10T06:40:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:19 smithi098 ceph-mon[103124]: pgmap v14267: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 128 KiB/s wr, 13 op/s 2023-09-10T06:40:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26450.479269 secs 2023-09-10T06:40:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26449.472898 secs 2023-09-10T06:40:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:19 smithi170 ceph-mon[102832]: pgmap v14267: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 128 KiB/s wr, 13 op/s 2023-09-10T06:40:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26450.479269 secs 2023-09-10T06:40:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26451.091995 secs 2023-09-10T06:40:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26451.091995 secs 2023-09-10T06:40:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26451.091995 secs 2023-09-10T06:40:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:21 smithi098 ceph-mon[103124]: pgmap v14268: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 128 KiB/s wr, 13 op/s 2023-09-10T06:40:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:21 smithi170 ceph-mon[102832]: pgmap v14268: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 128 KiB/s wr, 13 op/s 2023-09-10T06:40:21.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:21 smithi082 ceph-mon[98694]: pgmap v14268: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 128 KiB/s wr, 13 op/s 2023-09-10T06:40:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:23 smithi098 ceph-mon[103124]: pgmap v14269: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T06:40:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:23 smithi170 ceph-mon[102832]: pgmap v14269: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T06:40:23.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:23 smithi082 ceph-mon[98694]: pgmap v14269: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T06:40:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26454.472955 secs 2023-09-10T06:40:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26455.479331 secs 2023-09-10T06:40:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:40:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:40:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26454.472955 secs 2023-09-10T06:40:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26455.479331 secs 2023-09-10T06:40:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:40:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:40:24.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26454.472955 secs 2023-09-10T06:40:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26455.479331 secs 2023-09-10T06:40:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:40:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:40:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26456.092058 secs 2023-09-10T06:40:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:25 smithi098 ceph-mon[103124]: pgmap v14270: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T06:40:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26456.092058 secs 2023-09-10T06:40:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:25 smithi170 ceph-mon[102832]: pgmap v14270: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T06:40:25.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26456.092058 secs 2023-09-10T06:40:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:25 smithi082 ceph-mon[98694]: pgmap v14270: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T06:40:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:26 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:40:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:40:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:26 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:40:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:40:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:26 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:40:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:40:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:27 smithi098 ceph-mon[103124]: pgmap v14271: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T06:40:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:27 smithi170 ceph-mon[102832]: pgmap v14271: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T06:40:27.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:27 smithi082 ceph-mon[98694]: pgmap v14271: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T06:40:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:29 smithi098 ceph-mon[103124]: pgmap v14272: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 107 KiB/s wr, 10 op/s 2023-09-10T06:40:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26460.479385 secs 2023-09-10T06:40:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:29 smithi170 ceph-mon[102832]: pgmap v14272: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 107 KiB/s wr, 10 op/s 2023-09-10T06:40:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26460.479385 secs 2023-09-10T06:40:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:29 smithi082 ceph-mon[98694]: pgmap v14272: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 107 KiB/s wr, 10 op/s 2023-09-10T06:40:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26460.479385 secs 2023-09-10T06:40:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26461.092126 secs 2023-09-10T06:40:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26461.092126 secs 2023-09-10T06:40:30.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26461.092126 secs 2023-09-10T06:40:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:31 smithi170 ceph-mon[102832]: pgmap v14273: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 107 KiB/s wr, 10 op/s 2023-09-10T06:40:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:40:31.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:31 smithi082 ceph-mon[98694]: pgmap v14273: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 107 KiB/s wr, 10 op/s 2023-09-10T06:40:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:40:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:31 smithi098 ceph-mon[103124]: pgmap v14273: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 107 KiB/s wr, 10 op/s 2023-09-10T06:40:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:40:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:40:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:40:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:40:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:40:32.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:40:32.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:40:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:40:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:40:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:40:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:33 smithi170 ceph-mon[102832]: pgmap v14274: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 222 KiB/s wr, 15 op/s 2023-09-10T06:40:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:33 smithi082 ceph-mon[98694]: pgmap v14274: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 222 KiB/s wr, 15 op/s 2023-09-10T06:40:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:33 smithi098 ceph-mon[103124]: pgmap v14274: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 222 KiB/s wr, 15 op/s 2023-09-10T06:40:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26465.479444 secs 2023-09-10T06:40:34.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26465.479444 secs 2023-09-10T06:40:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26465.479444 secs 2023-09-10T06:40:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26466.092191 secs 2023-09-10T06:40:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:35 smithi170 ceph-mon[102832]: pgmap v14275: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 221 KiB/s wr, 10 op/s 2023-09-10T06:40:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26466.092191 secs 2023-09-10T06:40:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:35 smithi082 ceph-mon[98694]: pgmap v14275: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 221 KiB/s wr, 10 op/s 2023-09-10T06:40:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26466.092191 secs 2023-09-10T06:40:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:35 smithi098 ceph-mon[103124]: pgmap v14275: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 221 KiB/s wr, 10 op/s 2023-09-10T06:40:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:37 smithi170 ceph-mon[102832]: pgmap v14276: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 329 KiB/s wr, 16 op/s 2023-09-10T06:40:37.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:37 smithi082 ceph-mon[98694]: pgmap v14276: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 329 KiB/s wr, 16 op/s 2023-09-10T06:40:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:37 smithi098 ceph-mon[103124]: pgmap v14276: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 329 KiB/s wr, 16 op/s 2023-09-10T06:40:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:40:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:40:38.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:40:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:40:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:40:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:40:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:39 smithi170 ceph-mon[102832]: pgmap v14277: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 222 KiB/s wr, 11 op/s 2023-09-10T06:40:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26470.479492 secs 2023-09-10T06:40:39.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:39 smithi082 ceph-mon[98694]: pgmap v14277: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 222 KiB/s wr, 11 op/s 2023-09-10T06:40:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26470.479492 secs 2023-09-10T06:40:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:39 smithi098 ceph-mon[103124]: pgmap v14277: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 222 KiB/s wr, 11 op/s 2023-09-10T06:40:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26470.479492 secs 2023-09-10T06:40:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26471.092259 secs 2023-09-10T06:40:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:40:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:40:40.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26471.092259 secs 2023-09-10T06:40:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:40:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:40:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26471.092259 secs 2023-09-10T06:40:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:40:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:40:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:41 smithi170 ceph-mon[102832]: pgmap v14278: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 222 KiB/s wr, 11 op/s 2023-09-10T06:40:41.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:41 smithi082 ceph-mon[98694]: pgmap v14278: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 222 KiB/s wr, 11 op/s 2023-09-10T06:40:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:41 smithi098 ceph-mon[103124]: pgmap v14278: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 222 KiB/s wr, 11 op/s 2023-09-10T06:40:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:43 smithi098 ceph-mon[103124]: pgmap v14279: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 561 KiB/s wr, 17 op/s 2023-09-10T06:40:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:43 smithi170 ceph-mon[102832]: pgmap v14279: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 561 KiB/s wr, 17 op/s 2023-09-10T06:40:43.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:43 smithi082 ceph-mon[98694]: pgmap v14279: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 561 KiB/s wr, 17 op/s 2023-09-10T06:40:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:40:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:40:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26475.479557 secs 2023-09-10T06:40:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:40:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:40:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26475.479557 secs 2023-09-10T06:40:44.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:40:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:40:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26475.479557 secs 2023-09-10T06:40:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26476.092321 secs 2023-09-10T06:40:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:45 smithi098 ceph-mon[103124]: pgmap v14280: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 447 KiB/s wr, 12 op/s 2023-09-10T06:40:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26476.092321 secs 2023-09-10T06:40:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:45 smithi170 ceph-mon[102832]: pgmap v14280: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 447 KiB/s wr, 12 op/s 2023-09-10T06:40:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26476.092321 secs 2023-09-10T06:40:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:45 smithi082 ceph-mon[98694]: pgmap v14280: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 447 KiB/s wr, 12 op/s 2023-09-10T06:40:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:40:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:40:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:40:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:40:46.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:40:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:40:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:47 smithi098 ceph-mon[103124]: pgmap v14281: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 733 KiB/s wr, 18 op/s 2023-09-10T06:40:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:47 smithi170 ceph-mon[102832]: pgmap v14281: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 733 KiB/s wr, 18 op/s 2023-09-10T06:40:47.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:47 smithi082 ceph-mon[98694]: pgmap v14281: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 733 KiB/s wr, 18 op/s 2023-09-10T06:40:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:49 smithi098 ceph-mon[103124]: pgmap v14282: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 625 KiB/s wr, 12 op/s 2023-09-10T06:40:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26480.479615 secs 2023-09-10T06:40:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:49 smithi170 ceph-mon[102832]: pgmap v14282: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 625 KiB/s wr, 12 op/s 2023-09-10T06:40:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26480.479615 secs 2023-09-10T06:40:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:49 smithi082 ceph-mon[98694]: pgmap v14282: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 625 KiB/s wr, 12 op/s 2023-09-10T06:40:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26480.479615 secs 2023-09-10T06:40:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26481.092385 secs 2023-09-10T06:40:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26481.092385 secs 2023-09-10T06:40:50.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26481.092385 secs 2023-09-10T06:40:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:51 smithi098 ceph-mon[103124]: pgmap v14283: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 625 KiB/s wr, 12 op/s 2023-09-10T06:40:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:51 smithi170 ceph-mon[102832]: pgmap v14283: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 625 KiB/s wr, 12 op/s 2023-09-10T06:40:51.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:51 smithi082 ceph-mon[98694]: pgmap v14283: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 625 KiB/s wr, 12 op/s 2023-09-10T06:40:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:53 smithi098 ceph-mon[103124]: pgmap v14284: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 708 KiB/s wr, 18 op/s 2023-09-10T06:40:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:53 smithi170 ceph-mon[102832]: pgmap v14284: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 708 KiB/s wr, 18 op/s 2023-09-10T06:40:53.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:53 smithi082 ceph-mon[98694]: pgmap v14284: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 708 KiB/s wr, 18 op/s 2023-09-10T06:40:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26485.479685 secs 2023-09-10T06:40:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26485.479685 secs 2023-09-10T06:40:54.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26485.479685 secs 2023-09-10T06:40:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26486.092446 secs 2023-09-10T06:40:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:55 smithi170 ceph-mon[102832]: pgmap v14285: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 368 KiB/s wr, 12 op/s 2023-09-10T06:40:55.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26486.092446 secs 2023-09-10T06:40:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:55 smithi082 ceph-mon[98694]: pgmap v14285: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 368 KiB/s wr, 12 op/s 2023-09-10T06:40:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26486.092446 secs 2023-09-10T06:40:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:55 smithi098 ceph-mon[103124]: pgmap v14285: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 368 KiB/s wr, 12 op/s 2023-09-10T06:40:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:57 smithi170 ceph-mon[102832]: pgmap v14286: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 513 KiB/s wr, 18 op/s 2023-09-10T06:40:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:57 smithi082 ceph-mon[98694]: pgmap v14286: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 513 KiB/s wr, 18 op/s 2023-09-10T06:40:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:57 smithi098 ceph-mon[103124]: pgmap v14286: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 513 KiB/s wr, 18 op/s 2023-09-10T06:40:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:40:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:40:58.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:40:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:40:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:40:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:40:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26489.473399 secs 2023-09-10T06:40:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:59 smithi170 ceph-mon[102832]: pgmap v14287: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 228 KiB/s wr, 12 op/s 2023-09-10T06:40:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:40:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26490.479768 secs 2023-09-10T06:40:59.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26489.473399 secs 2023-09-10T06:40:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:59 smithi082 ceph-mon[98694]: pgmap v14287: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 228 KiB/s wr, 12 op/s 2023-09-10T06:40:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:40:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26490.479768 secs 2023-09-10T06:40:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26489.473399 secs 2023-09-10T06:40:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:59 smithi098 ceph-mon[103124]: pgmap v14287: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 228 KiB/s wr, 12 op/s 2023-09-10T06:40:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:40:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26490.479768 secs 2023-09-10T06:41:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26491.092492 secs 2023-09-10T06:41:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:41:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:41:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:41:00.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26491.092492 secs 2023-09-10T06:41:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:41:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:41:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:41:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26491.092492 secs 2023-09-10T06:41:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:41:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:41:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:41:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:01 smithi170 ceph-mon[102832]: pgmap v14288: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 228 KiB/s wr, 12 op/s 2023-09-10T06:41:01.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:01 smithi082 ceph-mon[98694]: pgmap v14288: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 228 KiB/s wr, 12 op/s 2023-09-10T06:41:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:01 smithi098 ceph-mon[103124]: pgmap v14288: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 228 KiB/s wr, 12 op/s 2023-09-10T06:41:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:03 smithi098 ceph-mon[103124]: pgmap v14289: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 462 KiB/s wr, 21 op/s 2023-09-10T06:41:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:03 smithi170 ceph-mon[102832]: pgmap v14289: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 462 KiB/s wr, 21 op/s 2023-09-10T06:41:03.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:03 smithi082 ceph-mon[98694]: pgmap v14289: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 462 KiB/s wr, 21 op/s 2023-09-10T06:41:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26494.473508 secs 2023-09-10T06:41:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26495.479845 secs 2023-09-10T06:41:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26494.473508 secs 2023-09-10T06:41:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26495.479845 secs 2023-09-10T06:41:04.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26494.473508 secs 2023-09-10T06:41:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26495.479845 secs 2023-09-10T06:41:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26496.092556 secs 2023-09-10T06:41:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:05 smithi098 ceph-mon[103124]: pgmap v14290: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 380 KiB/s wr, 15 op/s 2023-09-10T06:41:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26496.092556 secs 2023-09-10T06:41:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:05 smithi170 ceph-mon[102832]: pgmap v14290: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 380 KiB/s wr, 15 op/s 2023-09-10T06:41:05.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26496.092556 secs 2023-09-10T06:41:05.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:05 smithi082 ceph-mon[98694]: pgmap v14290: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 380 KiB/s wr, 15 op/s 2023-09-10T06:41:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:07 smithi170 ceph-mon[102832]: pgmap v14291: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 504 KiB/s wr, 32 op/s 2023-09-10T06:41:07.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:07 smithi082 ceph-mon[98694]: pgmap v14291: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 504 KiB/s wr, 32 op/s 2023-09-10T06:41:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:07 smithi098 ceph-mon[103124]: pgmap v14291: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 504 KiB/s wr, 32 op/s 2023-09-10T06:41:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26499.473570 secs 2023-09-10T06:41:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:09 smithi170 ceph-mon[102832]: osdmap e961: 12 total, 12 up, 12 in 2023-09-10T06:41:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:09 smithi170 ceph-mon[102832]: pgmap v14293: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 431 KiB/s wr, 31 op/s 2023-09-10T06:41:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26500.479929 secs 2023-09-10T06:41:09.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26499.473570 secs 2023-09-10T06:41:09.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:09 smithi082 ceph-mon[98694]: osdmap e961: 12 total, 12 up, 12 in 2023-09-10T06:41:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:09 smithi082 ceph-mon[98694]: pgmap v14293: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 431 KiB/s wr, 31 op/s 2023-09-10T06:41:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26500.479929 secs 2023-09-10T06:41:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26499.473570 secs 2023-09-10T06:41:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:09 smithi098 ceph-mon[103124]: osdmap e961: 12 total, 12 up, 12 in 2023-09-10T06:41:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:09 smithi098 ceph-mon[103124]: pgmap v14293: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 431 KiB/s wr, 31 op/s 2023-09-10T06:41:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26500.479929 secs 2023-09-10T06:41:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26501.092623 secs 2023-09-10T06:41:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:41:10.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26501.092623 secs 2023-09-10T06:41:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:41:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26501.092623 secs 2023-09-10T06:41:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:41:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:11 smithi170 ceph-mon[102832]: pgmap v14294: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 431 KiB/s wr, 31 op/s 2023-09-10T06:41:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:41:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:41:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:41:11.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:11 smithi082 ceph-mon[98694]: pgmap v14294: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 431 KiB/s wr, 31 op/s 2023-09-10T06:41:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:41:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:41:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:41:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:11 smithi098 ceph-mon[103124]: pgmap v14294: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 431 KiB/s wr, 31 op/s 2023-09-10T06:41:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:41:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:41:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:41:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:13 smithi098 ceph-mon[103124]: pgmap v14295: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 174 KiB/s wr, 26 op/s 2023-09-10T06:41:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:13 smithi170 ceph-mon[102832]: pgmap v14295: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 174 KiB/s wr, 26 op/s 2023-09-10T06:41:13.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:13 smithi082 ceph-mon[98694]: pgmap v14295: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 174 KiB/s wr, 26 op/s 2023-09-10T06:41:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26504.473631 secs 2023-09-10T06:41:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26505.480007 secs 2023-09-10T06:41:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26504.473631 secs 2023-09-10T06:41:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26505.480007 secs 2023-09-10T06:41:14.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26504.473631 secs 2023-09-10T06:41:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26505.480007 secs 2023-09-10T06:41:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26506.092688 secs 2023-09-10T06:41:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:15 smithi098 ceph-mon[103124]: pgmap v14296: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 174 KiB/s wr, 26 op/s 2023-09-10T06:41:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26506.092688 secs 2023-09-10T06:41:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:15 smithi170 ceph-mon[102832]: pgmap v14296: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 174 KiB/s wr, 26 op/s 2023-09-10T06:41:15.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26506.092688 secs 2023-09-10T06:41:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:15 smithi082 ceph-mon[98694]: pgmap v14296: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 174 KiB/s wr, 26 op/s 2023-09-10T06:41:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:17 smithi170 ceph-mon[102832]: osdmap e962: 12 total, 12 up, 12 in 2023-09-10T06:41:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:17 smithi170 ceph-mon[102832]: pgmap v14298: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 15 op/s 2023-09-10T06:41:17.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:17 smithi082 ceph-mon[98694]: osdmap e962: 12 total, 12 up, 12 in 2023-09-10T06:41:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:17 smithi082 ceph-mon[98694]: pgmap v14298: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 15 op/s 2023-09-10T06:41:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:17 smithi098 ceph-mon[103124]: osdmap e962: 12 total, 12 up, 12 in 2023-09-10T06:41:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:17 smithi098 ceph-mon[103124]: pgmap v14298: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 15 op/s 2023-09-10T06:41:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26509.473691 secs 2023-09-10T06:41:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:19 smithi098 ceph-mon[103124]: pgmap v14299: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 12 op/s 2023-09-10T06:41:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26510.480065 secs 2023-09-10T06:41:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26509.473691 secs 2023-09-10T06:41:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:19 smithi170 ceph-mon[102832]: pgmap v14299: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 12 op/s 2023-09-10T06:41:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26510.480065 secs 2023-09-10T06:41:19.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26509.473691 secs 2023-09-10T06:41:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:19 smithi082 ceph-mon[98694]: pgmap v14299: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 12 op/s 2023-09-10T06:41:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26510.480065 secs 2023-09-10T06:41:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26511.092751 secs 2023-09-10T06:41:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26511.092751 secs 2023-09-10T06:41:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26511.092751 secs 2023-09-10T06:41:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:21 smithi098 ceph-mon[103124]: pgmap v14300: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 12 op/s 2023-09-10T06:41:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:21 smithi170 ceph-mon[102832]: pgmap v14300: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 12 op/s 2023-09-10T06:41:21.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:21 smithi082 ceph-mon[98694]: pgmap v14300: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 12 op/s 2023-09-10T06:41:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:23 smithi098 ceph-mon[103124]: pgmap v14301: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T06:41:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:23 smithi170 ceph-mon[102832]: pgmap v14301: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T06:41:23.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:23 smithi082 ceph-mon[98694]: pgmap v14301: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T06:41:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26514.473753 secs 2023-09-10T06:41:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26515.480150 secs 2023-09-10T06:41:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26514.473753 secs 2023-09-10T06:41:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26515.480150 secs 2023-09-10T06:41:24.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26514.473753 secs 2023-09-10T06:41:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26515.480150 secs 2023-09-10T06:41:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26516.092812 secs 2023-09-10T06:41:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:25 smithi098 ceph-mon[103124]: pgmap v14302: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T06:41:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26516.092812 secs 2023-09-10T06:41:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:25 smithi170 ceph-mon[102832]: pgmap v14302: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T06:41:25.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26516.092812 secs 2023-09-10T06:41:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:25 smithi082 ceph-mon[98694]: pgmap v14302: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T06:41:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:27 smithi170 ceph-mon[102832]: pgmap v14303: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T06:41:27.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:27 smithi082 ceph-mon[98694]: pgmap v14303: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T06:41:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:27 smithi098 ceph-mon[103124]: pgmap v14303: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T06:41:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:29 smithi098 ceph-mon[103124]: pgmap v14304: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T06:41:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26520.480223 secs 2023-09-10T06:41:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:29 smithi170 ceph-mon[102832]: pgmap v14304: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T06:41:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26520.480223 secs 2023-09-10T06:41:29.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:29 smithi082 ceph-mon[98694]: pgmap v14304: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T06:41:29.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26520.480223 secs 2023-09-10T06:41:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26521.092876 secs 2023-09-10T06:41:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:41:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:41:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26521.092876 secs 2023-09-10T06:41:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:41:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:41:30.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26521.092876 secs 2023-09-10T06:41:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:41:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:41:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:31 smithi098 ceph-mon[103124]: pgmap v14305: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T06:41:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:41:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:31 smithi170 ceph-mon[102832]: pgmap v14305: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T06:41:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:41:31.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:31 smithi082 ceph-mon[98694]: pgmap v14305: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T06:41:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:41:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:41:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:41:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:41:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:41:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:41:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:41:32.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:41:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:41:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:41:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:33 smithi098 ceph-mon[103124]: pgmap v14306: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 286 KiB/s wr, 15 op/s 2023-09-10T06:41:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:33 smithi170 ceph-mon[102832]: pgmap v14306: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 286 KiB/s wr, 15 op/s 2023-09-10T06:41:33.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:33 smithi082 ceph-mon[98694]: pgmap v14306: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 286 KiB/s wr, 15 op/s 2023-09-10T06:41:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26525.480306 secs 2023-09-10T06:41:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26525.480306 secs 2023-09-10T06:41:34.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26525.480306 secs 2023-09-10T06:41:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26526.092941 secs 2023-09-10T06:41:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:35 smithi098 ceph-mon[103124]: pgmap v14307: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 9 op/s 2023-09-10T06:41:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26526.092941 secs 2023-09-10T06:41:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:35 smithi170 ceph-mon[102832]: pgmap v14307: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 9 op/s 2023-09-10T06:41:35.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26526.092941 secs 2023-09-10T06:41:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:35 smithi082 ceph-mon[98694]: pgmap v14307: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 9 op/s 2023-09-10T06:41:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:37 smithi170 ceph-mon[102832]: pgmap v14308: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 314 KiB/s wr, 15 op/s 2023-09-10T06:41:37.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:37 smithi082 ceph-mon[98694]: pgmap v14308: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 314 KiB/s wr, 15 op/s 2023-09-10T06:41:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:37 smithi098 ceph-mon[103124]: pgmap v14308: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 314 KiB/s wr, 15 op/s 2023-09-10T06:41:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:39 smithi098 ceph-mon[103124]: pgmap v14309: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 314 KiB/s wr, 10 op/s 2023-09-10T06:41:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26530.480365 secs 2023-09-10T06:41:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:39 smithi170 ceph-mon[102832]: pgmap v14309: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 314 KiB/s wr, 10 op/s 2023-09-10T06:41:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26530.480365 secs 2023-09-10T06:41:39.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:39 smithi082 ceph-mon[98694]: pgmap v14309: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 314 KiB/s wr, 10 op/s 2023-09-10T06:41:39.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26530.480365 secs 2023-09-10T06:41:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26531.093008 secs 2023-09-10T06:41:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26531.093008 secs 2023-09-10T06:41:40.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26531.093008 secs 2023-09-10T06:41:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:41 smithi098 ceph-mon[103124]: pgmap v14310: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 314 KiB/s wr, 10 op/s 2023-09-10T06:41:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:41 smithi170 ceph-mon[102832]: pgmap v14310: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 314 KiB/s wr, 10 op/s 2023-09-10T06:41:41.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:41 smithi082 ceph-mon[98694]: pgmap v14310: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 314 KiB/s wr, 10 op/s 2023-09-10T06:41:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:43 smithi098 ceph-mon[103124]: pgmap v14311: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 354 KiB/s wr, 16 op/s 2023-09-10T06:41:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:43 smithi170 ceph-mon[102832]: pgmap v14311: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 354 KiB/s wr, 16 op/s 2023-09-10T06:41:43.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:43 smithi082 ceph-mon[98694]: pgmap v14311: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 354 KiB/s wr, 16 op/s 2023-09-10T06:41:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26535.480423 secs 2023-09-10T06:41:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:41:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:41:44.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26535.480423 secs 2023-09-10T06:41:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:41:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:41:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26535.480423 secs 2023-09-10T06:41:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:41:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:41:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26536.093073 secs 2023-09-10T06:41:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:45 smithi170 ceph-mon[102832]: pgmap v14312: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T06:41:45.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26536.093073 secs 2023-09-10T06:41:45.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:45 smithi082 ceph-mon[98694]: pgmap v14312: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T06:41:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26536.093073 secs 2023-09-10T06:41:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:45 smithi098 ceph-mon[103124]: pgmap v14312: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T06:41:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:41:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:41:46.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:41:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:41:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:41:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:41:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:47 smithi170 ceph-mon[102832]: pgmap v14313: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 502 KiB/s wr, 18 op/s 2023-09-10T06:41:47.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:47 smithi082 ceph-mon[98694]: pgmap v14313: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 502 KiB/s wr, 18 op/s 2023-09-10T06:41:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:47 smithi098 ceph-mon[103124]: pgmap v14313: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 502 KiB/s wr, 18 op/s 2023-09-10T06:41:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:49 smithi170 ceph-mon[102832]: pgmap v14314: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 383 KiB/s wr, 13 op/s 2023-09-10T06:41:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26540.480501 secs 2023-09-10T06:41:49.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:49 smithi082 ceph-mon[98694]: pgmap v14314: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 383 KiB/s wr, 13 op/s 2023-09-10T06:41:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26540.480501 secs 2023-09-10T06:41:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:49 smithi098 ceph-mon[103124]: pgmap v14314: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 383 KiB/s wr, 13 op/s 2023-09-10T06:41:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26540.480501 secs 2023-09-10T06:41:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26541.093131 secs 2023-09-10T06:41:50.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26541.093131 secs 2023-09-10T06:41:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26541.093131 secs 2023-09-10T06:41:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:51 smithi170 ceph-mon[102832]: pgmap v14315: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 383 KiB/s wr, 13 op/s 2023-09-10T06:41:51.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:51 smithi082 ceph-mon[98694]: pgmap v14315: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 383 KiB/s wr, 13 op/s 2023-09-10T06:41:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:51 smithi098 ceph-mon[103124]: pgmap v14315: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 383 KiB/s wr, 13 op/s 2023-09-10T06:41:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:53 smithi170 ceph-mon[102832]: pgmap v14316: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 420 KiB/s wr, 19 op/s 2023-09-10T06:41:53.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:53 smithi082 ceph-mon[98694]: pgmap v14316: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 420 KiB/s wr, 19 op/s 2023-09-10T06:41:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:53 smithi098 ceph-mon[103124]: pgmap v14316: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 420 KiB/s wr, 19 op/s 2023-09-10T06:41:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26545.480558 secs 2023-09-10T06:41:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:41:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:41:54.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26545.480558 secs 2023-09-10T06:41:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:41:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:41:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26545.480558 secs 2023-09-10T06:41:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:41:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:41:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26546.093194 secs 2023-09-10T06:41:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:55 smithi170 ceph-mon[102832]: pgmap v14317: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 380 KiB/s wr, 12 op/s 2023-09-10T06:41:55.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26546.093194 secs 2023-09-10T06:41:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:55 smithi082 ceph-mon[98694]: pgmap v14317: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 380 KiB/s wr, 12 op/s 2023-09-10T06:41:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26546.093194 secs 2023-09-10T06:41:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:55 smithi098 ceph-mon[103124]: pgmap v14317: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 380 KiB/s wr, 12 op/s 2023-09-10T06:41:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:41:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:41:56.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:41:56.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:41:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:41:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:41:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:57 smithi170 ceph-mon[102832]: pgmap v14318: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 843 KiB/s wr, 19 op/s 2023-09-10T06:41:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:57 smithi082 ceph-mon[98694]: pgmap v14318: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 843 KiB/s wr, 19 op/s 2023-09-10T06:41:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:57 smithi098 ceph-mon[103124]: pgmap v14318: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 843 KiB/s wr, 19 op/s 2023-09-10T06:41:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:41:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:41:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26549.474272 secs 2023-09-10T06:41:58.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:41:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:41:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26549.474272 secs 2023-09-10T06:41:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:41:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:41:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26549.474272 secs 2023-09-10T06:41:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:59 smithi170 ceph-mon[102832]: pgmap v14319: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 500 KiB/s wr, 12 op/s 2023-09-10T06:41:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:41:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26550.480619 secs 2023-09-10T06:41:59.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:59 smithi082 ceph-mon[98694]: pgmap v14319: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 500 KiB/s wr, 12 op/s 2023-09-10T06:41:59.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:41:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26550.480619 secs 2023-09-10T06:41:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:59 smithi098 ceph-mon[103124]: pgmap v14319: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 500 KiB/s wr, 12 op/s 2023-09-10T06:41:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:41:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26550.480619 secs 2023-09-10T06:42:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26551.093258 secs 2023-09-10T06:42:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:42:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:42:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:42:00.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26551.093258 secs 2023-09-10T06:42:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:42:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:42:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:42:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26551.093258 secs 2023-09-10T06:42:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:42:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:42:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:42:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:01 smithi170 ceph-mon[102832]: pgmap v14320: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 500 KiB/s wr, 12 op/s 2023-09-10T06:42:01.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:01 smithi082 ceph-mon[98694]: pgmap v14320: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 500 KiB/s wr, 12 op/s 2023-09-10T06:42:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:01 smithi098 ceph-mon[103124]: pgmap v14320: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 500 KiB/s wr, 12 op/s 2023-09-10T06:42:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:03 smithi170 ceph-mon[102832]: pgmap v14321: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 501 KiB/s wr, 18 op/s 2023-09-10T06:42:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26554.474334 secs 2023-09-10T06:42:03.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:03 smithi082 ceph-mon[98694]: pgmap v14321: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 501 KiB/s wr, 18 op/s 2023-09-10T06:42:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26554.474334 secs 2023-09-10T06:42:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:03 smithi098 ceph-mon[103124]: pgmap v14321: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 501 KiB/s wr, 18 op/s 2023-09-10T06:42:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26554.474334 secs 2023-09-10T06:42:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26555.480678 secs 2023-09-10T06:42:04.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26555.480678 secs 2023-09-10T06:42:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26555.480678 secs 2023-09-10T06:42:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26556.093323 secs 2023-09-10T06:42:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:05 smithi170 ceph-mon[102832]: pgmap v14322: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 463 KiB/s wr, 12 op/s 2023-09-10T06:42:05.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26556.093323 secs 2023-09-10T06:42:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:05 smithi082 ceph-mon[98694]: pgmap v14322: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 463 KiB/s wr, 12 op/s 2023-09-10T06:42:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26556.093323 secs 2023-09-10T06:42:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:05 smithi098 ceph-mon[103124]: pgmap v14322: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 463 KiB/s wr, 12 op/s 2023-09-10T06:42:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:07 smithi170 ceph-mon[102832]: pgmap v14323: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 721 KiB/s wr, 32 op/s 2023-09-10T06:42:07.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:07 smithi082 ceph-mon[98694]: pgmap v14323: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 721 KiB/s wr, 32 op/s 2023-09-10T06:42:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:07 smithi098 ceph-mon[103124]: pgmap v14323: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 721 KiB/s wr, 32 op/s 2023-09-10T06:42:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26559.474437 secs 2023-09-10T06:42:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:09 smithi170 ceph-mon[102832]: osdmap e963: 12 total, 12 up, 12 in 2023-09-10T06:42:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:09 smithi170 ceph-mon[102832]: pgmap v14325: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 309 KiB/s wr, 31 op/s 2023-09-10T06:42:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26560.480737 secs 2023-09-10T06:42:09.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26559.474437 secs 2023-09-10T06:42:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:09 smithi082 ceph-mon[98694]: osdmap e963: 12 total, 12 up, 12 in 2023-09-10T06:42:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:09 smithi082 ceph-mon[98694]: pgmap v14325: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 309 KiB/s wr, 31 op/s 2023-09-10T06:42:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26560.480737 secs 2023-09-10T06:42:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26559.474437 secs 2023-09-10T06:42:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:09 smithi098 ceph-mon[103124]: osdmap e963: 12 total, 12 up, 12 in 2023-09-10T06:42:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:09 smithi098 ceph-mon[103124]: pgmap v14325: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 309 KiB/s wr, 31 op/s 2023-09-10T06:42:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26560.480737 secs 2023-09-10T06:42:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26561.093389 secs 2023-09-10T06:42:10.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26561.093389 secs 2023-09-10T06:42:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26561.093389 secs 2023-09-10T06:42:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:11 smithi170 ceph-mon[102832]: pgmap v14326: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 309 KiB/s wr, 31 op/s 2023-09-10T06:42:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:42:11.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:11 smithi082 ceph-mon[98694]: pgmap v14326: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 309 KiB/s wr, 31 op/s 2023-09-10T06:42:11.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:42:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:11 smithi098 ceph-mon[103124]: pgmap v14326: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 309 KiB/s wr, 31 op/s 2023-09-10T06:42:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:42:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:42:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:42:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:42:12.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:42:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:42:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:42:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:42:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:42:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:42:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:13 smithi170 ceph-mon[102832]: pgmap v14327: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 334 KiB/s wr, 30 op/s 2023-09-10T06:42:13.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:13 smithi082 ceph-mon[98694]: pgmap v14327: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 334 KiB/s wr, 30 op/s 2023-09-10T06:42:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:13 smithi098 ceph-mon[103124]: pgmap v14327: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 334 KiB/s wr, 30 op/s 2023-09-10T06:42:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26564.474546 secs 2023-09-10T06:42:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26565.480795 secs 2023-09-10T06:42:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26564.474546 secs 2023-09-10T06:42:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26565.480795 secs 2023-09-10T06:42:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26564.474546 secs 2023-09-10T06:42:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26565.480795 secs 2023-09-10T06:42:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26566.093452 secs 2023-09-10T06:42:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:15 smithi170 ceph-mon[102832]: pgmap v14328: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 334 KiB/s wr, 30 op/s 2023-09-10T06:42:15.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26566.093452 secs 2023-09-10T06:42:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:15 smithi082 ceph-mon[98694]: pgmap v14328: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 334 KiB/s wr, 30 op/s 2023-09-10T06:42:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26566.093452 secs 2023-09-10T06:42:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:15 smithi098 ceph-mon[103124]: pgmap v14328: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 334 KiB/s wr, 30 op/s 2023-09-10T06:42:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:17 smithi170 ceph-mon[102832]: osdmap e964: 12 total, 12 up, 12 in 2023-09-10T06:42:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:17 smithi170 ceph-mon[102832]: pgmap v14330: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 14 op/s 2023-09-10T06:42:17.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:17 smithi082 ceph-mon[98694]: osdmap e964: 12 total, 12 up, 12 in 2023-09-10T06:42:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:17 smithi082 ceph-mon[98694]: pgmap v14330: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 14 op/s 2023-09-10T06:42:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:17 smithi098 ceph-mon[103124]: osdmap e964: 12 total, 12 up, 12 in 2023-09-10T06:42:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:17 smithi098 ceph-mon[103124]: pgmap v14330: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 14 op/s 2023-09-10T06:42:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26569.474605 secs 2023-09-10T06:42:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:19 smithi098 ceph-mon[103124]: pgmap v14331: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T06:42:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26570.480853 secs 2023-09-10T06:42:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26569.474605 secs 2023-09-10T06:42:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:19 smithi170 ceph-mon[102832]: pgmap v14331: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T06:42:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26570.480853 secs 2023-09-10T06:42:19.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26569.474605 secs 2023-09-10T06:42:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:19 smithi082 ceph-mon[98694]: pgmap v14331: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T06:42:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26570.480853 secs 2023-09-10T06:42:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26571.093487 secs 2023-09-10T06:42:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26571.093487 secs 2023-09-10T06:42:20.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26571.093487 secs 2023-09-10T06:42:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:21 smithi098 ceph-mon[103124]: pgmap v14332: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T06:42:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:21 smithi170 ceph-mon[102832]: pgmap v14332: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T06:42:21.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:21 smithi082 ceph-mon[98694]: pgmap v14332: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-09-10T06:42:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:23 smithi098 ceph-mon[103124]: pgmap v14333: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T06:42:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:23 smithi170 ceph-mon[102832]: pgmap v14333: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T06:42:23.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:23 smithi082 ceph-mon[98694]: pgmap v14333: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T06:42:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26574.474666 secs 2023-09-10T06:42:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26575.480910 secs 2023-09-10T06:42:24.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26574.474666 secs 2023-09-10T06:42:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26575.480910 secs 2023-09-10T06:42:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26574.474666 secs 2023-09-10T06:42:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26575.480910 secs 2023-09-10T06:42:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26576.093550 secs 2023-09-10T06:42:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:25 smithi170 ceph-mon[102832]: pgmap v14334: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T06:42:25.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26576.093550 secs 2023-09-10T06:42:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:25 smithi082 ceph-mon[98694]: pgmap v14334: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T06:42:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26576.093550 secs 2023-09-10T06:42:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:25 smithi098 ceph-mon[103124]: pgmap v14334: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T06:42:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:27 smithi170 ceph-mon[102832]: pgmap v14335: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T06:42:27.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:27 smithi082 ceph-mon[98694]: pgmap v14335: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T06:42:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:27 smithi098 ceph-mon[103124]: pgmap v14335: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T06:42:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:29 smithi098 ceph-mon[103124]: pgmap v14336: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T06:42:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26580.480968 secs 2023-09-10T06:42:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:29 smithi170 ceph-mon[102832]: pgmap v14336: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T06:42:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26580.480968 secs 2023-09-10T06:42:29.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:29 smithi082 ceph-mon[98694]: pgmap v14336: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T06:42:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26580.480968 secs 2023-09-10T06:42:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26581.093616 secs 2023-09-10T06:42:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:42:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:42:30.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26581.093616 secs 2023-09-10T06:42:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:42:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:42:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26581.093616 secs 2023-09-10T06:42:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:42:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:42:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:31 smithi170 ceph-mon[102832]: pgmap v14337: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T06:42:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:42:31.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:31 smithi082 ceph-mon[98694]: pgmap v14337: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T06:42:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:42:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:31 smithi098 ceph-mon[103124]: pgmap v14337: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T06:42:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:42:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:42:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:42:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:42:32.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:42:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:42:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:42:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:42:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:42:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:42:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:33 smithi170 ceph-mon[102832]: pgmap v14338: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 267 KiB/s wr, 16 op/s 2023-09-10T06:42:33.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:33 smithi082 ceph-mon[98694]: pgmap v14338: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 267 KiB/s wr, 16 op/s 2023-09-10T06:42:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:33 smithi098 ceph-mon[103124]: pgmap v14338: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 267 KiB/s wr, 16 op/s 2023-09-10T06:42:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26585.481027 secs 2023-09-10T06:42:34.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26585.481027 secs 2023-09-10T06:42:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26585.481027 secs 2023-09-10T06:42:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26586.093681 secs 2023-09-10T06:42:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:35 smithi170 ceph-mon[102832]: pgmap v14339: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 10 op/s 2023-09-10T06:42:35.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26586.093681 secs 2023-09-10T06:42:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:35 smithi082 ceph-mon[98694]: pgmap v14339: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 10 op/s 2023-09-10T06:42:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26586.093681 secs 2023-09-10T06:42:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:35 smithi098 ceph-mon[103124]: pgmap v14339: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 10 op/s 2023-09-10T06:42:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:37 smithi170 ceph-mon[102832]: pgmap v14340: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 397 KiB/s wr, 17 op/s 2023-09-10T06:42:37.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:37 smithi082 ceph-mon[98694]: pgmap v14340: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 397 KiB/s wr, 17 op/s 2023-09-10T06:42:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:37 smithi098 ceph-mon[103124]: pgmap v14340: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 397 KiB/s wr, 17 op/s 2023-09-10T06:42:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:42:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:42:38.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:42:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:42:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:42:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:42:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:39 smithi170 ceph-mon[102832]: pgmap v14341: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 397 KiB/s wr, 12 op/s 2023-09-10T06:42:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26590.481113 secs 2023-09-10T06:42:39.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:39 smithi082 ceph-mon[98694]: pgmap v14341: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 397 KiB/s wr, 12 op/s 2023-09-10T06:42:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26590.481113 secs 2023-09-10T06:42:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:39 smithi098 ceph-mon[103124]: pgmap v14341: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 397 KiB/s wr, 12 op/s 2023-09-10T06:42:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26590.481113 secs 2023-09-10T06:42:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26591.093750 secs 2023-09-10T06:42:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:42:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:42:40.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26591.093750 secs 2023-09-10T06:42:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:42:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:42:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26591.093750 secs 2023-09-10T06:42:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:42:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:42:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:41 smithi170 ceph-mon[102832]: pgmap v14342: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 397 KiB/s wr, 12 op/s 2023-09-10T06:42:41.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:41 smithi082 ceph-mon[98694]: pgmap v14342: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 397 KiB/s wr, 12 op/s 2023-09-10T06:42:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:41 smithi098 ceph-mon[103124]: pgmap v14342: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 397 KiB/s wr, 12 op/s 2023-09-10T06:42:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:43 smithi170 ceph-mon[102832]: pgmap v14343: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 415 KiB/s wr, 18 op/s 2023-09-10T06:42:43.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:43 smithi082 ceph-mon[98694]: pgmap v14343: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 415 KiB/s wr, 18 op/s 2023-09-10T06:42:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:43 smithi098 ceph-mon[103124]: pgmap v14343: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 415 KiB/s wr, 18 op/s 2023-09-10T06:42:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26595.481171 secs 2023-09-10T06:42:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:42:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:42:44.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26595.481171 secs 2023-09-10T06:42:44.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:42:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:42:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26595.481171 secs 2023-09-10T06:42:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:42:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:42:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26596.093815 secs 2023-09-10T06:42:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:45 smithi170 ceph-mon[102832]: pgmap v14344: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 239 KiB/s wr, 12 op/s 2023-09-10T06:42:45.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26596.093815 secs 2023-09-10T06:42:45.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:45 smithi082 ceph-mon[98694]: pgmap v14344: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 239 KiB/s wr, 12 op/s 2023-09-10T06:42:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26596.093815 secs 2023-09-10T06:42:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:45 smithi098 ceph-mon[103124]: pgmap v14344: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 239 KiB/s wr, 12 op/s 2023-09-10T06:42:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:42:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:42:46.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:42:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:42:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:42:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:42:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:47 smithi170 ceph-mon[102832]: pgmap v14345: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 731 KiB/s wr, 20 op/s 2023-09-10T06:42:47.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:47 smithi082 ceph-mon[98694]: pgmap v14345: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 731 KiB/s wr, 20 op/s 2023-09-10T06:42:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:47 smithi098 ceph-mon[103124]: pgmap v14345: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 731 KiB/s wr, 20 op/s 2023-09-10T06:42:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:49 smithi170 ceph-mon[102832]: pgmap v14346: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 510 KiB/s wr, 13 op/s 2023-09-10T06:42:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26600.481228 secs 2023-09-10T06:42:49.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:49 smithi082 ceph-mon[98694]: pgmap v14346: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 510 KiB/s wr, 13 op/s 2023-09-10T06:42:49.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26600.481228 secs 2023-09-10T06:42:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:49 smithi098 ceph-mon[103124]: pgmap v14346: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 510 KiB/s wr, 13 op/s 2023-09-10T06:42:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26600.481228 secs 2023-09-10T06:42:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26601.093882 secs 2023-09-10T06:42:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26601.093882 secs 2023-09-10T06:42:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26601.093882 secs 2023-09-10T06:42:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:51 smithi170 ceph-mon[102832]: pgmap v14347: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 510 KiB/s wr, 13 op/s 2023-09-10T06:42:51.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:51 smithi082 ceph-mon[98694]: pgmap v14347: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 510 KiB/s wr, 13 op/s 2023-09-10T06:42:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:51 smithi098 ceph-mon[103124]: pgmap v14347: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 510 KiB/s wr, 13 op/s 2023-09-10T06:42:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:53 smithi170 ceph-mon[102832]: pgmap v14348: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 632 KiB/s wr, 19 op/s 2023-09-10T06:42:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:53 smithi082 ceph-mon[98694]: pgmap v14348: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 632 KiB/s wr, 19 op/s 2023-09-10T06:42:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:53 smithi098 ceph-mon[103124]: pgmap v14348: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 632 KiB/s wr, 19 op/s 2023-09-10T06:42:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26605.481313 secs 2023-09-10T06:42:54.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26605.481313 secs 2023-09-10T06:42:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26605.481313 secs 2023-09-10T06:42:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26606.093948 secs 2023-09-10T06:42:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:55 smithi170 ceph-mon[102832]: pgmap v14349: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 615 KiB/s wr, 13 op/s 2023-09-10T06:42:55.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26606.093948 secs 2023-09-10T06:42:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:55 smithi082 ceph-mon[98694]: pgmap v14349: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 615 KiB/s wr, 13 op/s 2023-09-10T06:42:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26606.093948 secs 2023-09-10T06:42:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:55 smithi098 ceph-mon[103124]: pgmap v14349: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 615 KiB/s wr, 13 op/s 2023-09-10T06:42:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:57 smithi170 ceph-mon[102832]: pgmap v14350: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 709 KiB/s wr, 19 op/s 2023-09-10T06:42:57.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:57 smithi082 ceph-mon[98694]: pgmap v14350: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 709 KiB/s wr, 19 op/s 2023-09-10T06:42:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:57 smithi098 ceph-mon[103124]: pgmap v14350: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 709 KiB/s wr, 19 op/s 2023-09-10T06:42:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:42:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:42:58.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:42:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:42:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:42:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:42:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26609.475274 secs 2023-09-10T06:42:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:59 smithi170 ceph-mon[102832]: pgmap v14351: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 217 KiB/s wr, 11 op/s 2023-09-10T06:42:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:42:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26610.481395 secs 2023-09-10T06:42:59.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26609.475274 secs 2023-09-10T06:42:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:59 smithi082 ceph-mon[98694]: pgmap v14351: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 217 KiB/s wr, 11 op/s 2023-09-10T06:42:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:42:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26610.481395 secs 2023-09-10T06:42:59.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26609.475274 secs 2023-09-10T06:42:59.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:59 smithi098 ceph-mon[103124]: pgmap v14351: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 217 KiB/s wr, 11 op/s 2023-09-10T06:42:59.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:42:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26610.481395 secs 2023-09-10T06:43:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26611.094017 secs 2023-09-10T06:43:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:43:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:43:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:00.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26611.094017 secs 2023-09-10T06:43:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:43:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:43:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26611.094017 secs 2023-09-10T06:43:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:43:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:43:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:01 smithi170 ceph-mon[102832]: pgmap v14352: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 217 KiB/s wr, 11 op/s 2023-09-10T06:43:01.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:01 smithi082 ceph-mon[98694]: pgmap v14352: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 217 KiB/s wr, 11 op/s 2023-09-10T06:43:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:01 smithi098 ceph-mon[103124]: pgmap v14352: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 217 KiB/s wr, 11 op/s 2023-09-10T06:43:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:03 smithi170 ceph-mon[102832]: pgmap v14353: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 412 KiB/s wr, 17 op/s 2023-09-10T06:43:03.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:03 smithi082 ceph-mon[98694]: pgmap v14353: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 412 KiB/s wr, 17 op/s 2023-09-10T06:43:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:03 smithi098 ceph-mon[103124]: pgmap v14353: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 412 KiB/s wr, 17 op/s 2023-09-10T06:43:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26614.475335 secs 2023-09-10T06:43:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26615.481479 secs 2023-09-10T06:43:04.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26614.475335 secs 2023-09-10T06:43:04.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26615.481479 secs 2023-09-10T06:43:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26614.475335 secs 2023-09-10T06:43:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26615.481479 secs 2023-09-10T06:43:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26616.094079 secs 2023-09-10T06:43:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:05 smithi170 ceph-mon[102832]: pgmap v14354: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 289 KiB/s wr, 11 op/s 2023-09-10T06:43:05.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26616.094079 secs 2023-09-10T06:43:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:05 smithi082 ceph-mon[98694]: pgmap v14354: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 289 KiB/s wr, 11 op/s 2023-09-10T06:43:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26616.094079 secs 2023-09-10T06:43:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:05 smithi098 ceph-mon[103124]: pgmap v14354: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 289 KiB/s wr, 11 op/s 2023-09-10T06:43:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:07 smithi170 ceph-mon[102832]: pgmap v14355: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 425 KiB/s wr, 31 op/s 2023-09-10T06:43:07.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:07 smithi082 ceph-mon[98694]: pgmap v14355: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 425 KiB/s wr, 31 op/s 2023-09-10T06:43:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:07 smithi098 ceph-mon[103124]: pgmap v14355: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 425 KiB/s wr, 31 op/s 2023-09-10T06:43:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26619.475393 secs 2023-09-10T06:43:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:09 smithi170 ceph-mon[102832]: osdmap e965: 12 total, 12 up, 12 in 2023-09-10T06:43:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:09 smithi170 ceph-mon[102832]: pgmap v14357: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 398 KiB/s wr, 30 op/s 2023-09-10T06:43:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26620.481533 secs 2023-09-10T06:43:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26619.475393 secs 2023-09-10T06:43:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:09 smithi082 ceph-mon[98694]: osdmap e965: 12 total, 12 up, 12 in 2023-09-10T06:43:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:09 smithi082 ceph-mon[98694]: pgmap v14357: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 398 KiB/s wr, 30 op/s 2023-09-10T06:43:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26620.481533 secs 2023-09-10T06:43:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26619.475393 secs 2023-09-10T06:43:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:09 smithi098 ceph-mon[103124]: osdmap e965: 12 total, 12 up, 12 in 2023-09-10T06:43:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:09 smithi098 ceph-mon[103124]: pgmap v14357: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 398 KiB/s wr, 30 op/s 2023-09-10T06:43:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26620.481533 secs 2023-09-10T06:43:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26621.094146 secs 2023-09-10T06:43:10.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26621.094146 secs 2023-09-10T06:43:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26621.094146 secs 2023-09-10T06:43:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:11 smithi170 ceph-mon[102832]: pgmap v14358: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 398 KiB/s wr, 30 op/s 2023-09-10T06:43:11.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:11 smithi082 ceph-mon[98694]: pgmap v14358: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 398 KiB/s wr, 30 op/s 2023-09-10T06:43:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:11 smithi098 ceph-mon[103124]: pgmap v14358: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 398 KiB/s wr, 30 op/s 2023-09-10T06:43:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:43:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:43:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:43:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:43:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:43:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:43:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:43:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:43:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:43:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:43:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:43:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:43:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:13 smithi170 ceph-mon[102832]: pgmap v14359: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 189 KiB/s wr, 30 op/s 2023-09-10T06:43:13.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:13 smithi082 ceph-mon[98694]: pgmap v14359: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 189 KiB/s wr, 30 op/s 2023-09-10T06:43:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:13 smithi098 ceph-mon[103124]: pgmap v14359: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 189 KiB/s wr, 30 op/s 2023-09-10T06:43:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26624.475457 secs 2023-09-10T06:43:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26625.481591 secs 2023-09-10T06:43:14.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26624.475457 secs 2023-09-10T06:43:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26625.481591 secs 2023-09-10T06:43:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26624.475457 secs 2023-09-10T06:43:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26625.481591 secs 2023-09-10T06:43:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26626.094211 secs 2023-09-10T06:43:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:15 smithi170 ceph-mon[102832]: pgmap v14360: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 189 KiB/s wr, 30 op/s 2023-09-10T06:43:15.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26626.094211 secs 2023-09-10T06:43:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:15 smithi082 ceph-mon[98694]: pgmap v14360: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 189 KiB/s wr, 30 op/s 2023-09-10T06:43:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26626.094211 secs 2023-09-10T06:43:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:15 smithi098 ceph-mon[103124]: pgmap v14360: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 189 KiB/s wr, 30 op/s 2023-09-10T06:43:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:17 smithi170 ceph-mon[102832]: osdmap e966: 12 total, 12 up, 12 in 2023-09-10T06:43:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:17 smithi170 ceph-mon[102832]: pgmap v14362: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-09-10T06:43:17.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:17 smithi082 ceph-mon[98694]: osdmap e966: 12 total, 12 up, 12 in 2023-09-10T06:43:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:17 smithi082 ceph-mon[98694]: pgmap v14362: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-09-10T06:43:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:17 smithi098 ceph-mon[103124]: osdmap e966: 12 total, 12 up, 12 in 2023-09-10T06:43:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:17 smithi098 ceph-mon[103124]: pgmap v14362: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-09-10T06:43:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26629.475518 secs 2023-09-10T06:43:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:19 smithi170 ceph-mon[102832]: pgmap v14363: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-09-10T06:43:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26630.481649 secs 2023-09-10T06:43:19.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26629.475518 secs 2023-09-10T06:43:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:19 smithi082 ceph-mon[98694]: pgmap v14363: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-09-10T06:43:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26630.481649 secs 2023-09-10T06:43:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26629.475518 secs 2023-09-10T06:43:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:19 smithi098 ceph-mon[103124]: pgmap v14363: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-09-10T06:43:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26630.481649 secs 2023-09-10T06:43:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26631.094278 secs 2023-09-10T06:43:20.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26631.094278 secs 2023-09-10T06:43:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26631.094278 secs 2023-09-10T06:43:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:21 smithi170 ceph-mon[102832]: pgmap v14364: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-09-10T06:43:21.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:21 smithi082 ceph-mon[98694]: pgmap v14364: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-09-10T06:43:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:21 smithi098 ceph-mon[103124]: pgmap v14364: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-09-10T06:43:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:23 smithi170 ceph-mon[102832]: pgmap v14365: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T06:43:23.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:23 smithi082 ceph-mon[98694]: pgmap v14365: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T06:43:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:23 smithi098 ceph-mon[103124]: pgmap v14365: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T06:43:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26634.475622 secs 2023-09-10T06:43:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26635.481703 secs 2023-09-10T06:43:24.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26634.475622 secs 2023-09-10T06:43:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26635.481703 secs 2023-09-10T06:43:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26634.475622 secs 2023-09-10T06:43:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26635.481703 secs 2023-09-10T06:43:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26636.094344 secs 2023-09-10T06:43:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:25 smithi170 ceph-mon[102832]: pgmap v14366: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T06:43:25.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26636.094344 secs 2023-09-10T06:43:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:25 smithi082 ceph-mon[98694]: pgmap v14366: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T06:43:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26636.094344 secs 2023-09-10T06:43:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:25 smithi098 ceph-mon[103124]: pgmap v14366: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-09-10T06:43:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:27 smithi170 ceph-mon[102832]: pgmap v14367: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T06:43:27.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:27 smithi082 ceph-mon[98694]: pgmap v14367: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T06:43:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:27 smithi098 ceph-mon[103124]: pgmap v14367: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 108 KiB/s wr, 12 op/s 2023-09-10T06:43:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:29 smithi170 ceph-mon[102832]: pgmap v14368: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T06:43:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26640.481762 secs 2023-09-10T06:43:29.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:29 smithi082 ceph-mon[98694]: pgmap v14368: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T06:43:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26640.481762 secs 2023-09-10T06:43:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:29 smithi098 ceph-mon[103124]: pgmap v14368: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T06:43:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26640.481762 secs 2023-09-10T06:43:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26641.094407 secs 2023-09-10T06:43:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:43:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:30.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26641.094407 secs 2023-09-10T06:43:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:43:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26641.094407 secs 2023-09-10T06:43:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:43:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:31 smithi170 ceph-mon[102832]: pgmap v14369: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T06:43:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:43:31.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:31 smithi082 ceph-mon[98694]: pgmap v14369: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T06:43:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:43:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:31 smithi098 ceph-mon[103124]: pgmap v14369: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 10 op/s 2023-09-10T06:43:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:43:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:43:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:43:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:32.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:43:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:43:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:43:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:43:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:33 smithi170 ceph-mon[102832]: pgmap v14370: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 287 KiB/s wr, 15 op/s 2023-09-10T06:43:33.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:33 smithi082 ceph-mon[98694]: pgmap v14370: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 287 KiB/s wr, 15 op/s 2023-09-10T06:43:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:33 smithi098 ceph-mon[103124]: pgmap v14370: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 287 KiB/s wr, 15 op/s 2023-09-10T06:43:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26645.481820 secs 2023-09-10T06:43:34.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26645.481820 secs 2023-09-10T06:43:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26645.481820 secs 2023-09-10T06:43:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26646.094475 secs 2023-09-10T06:43:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:35 smithi170 ceph-mon[102832]: pgmap v14371: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 9 op/s 2023-09-10T06:43:35.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26646.094475 secs 2023-09-10T06:43:35.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:35 smithi082 ceph-mon[98694]: pgmap v14371: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 9 op/s 2023-09-10T06:43:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26646.094475 secs 2023-09-10T06:43:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:35 smithi098 ceph-mon[103124]: pgmap v14371: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 9 op/s 2023-09-10T06:43:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:37 smithi170 ceph-mon[102832]: pgmap v14372: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 305 KiB/s wr, 15 op/s 2023-09-10T06:43:37.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:37 smithi082 ceph-mon[98694]: pgmap v14372: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 305 KiB/s wr, 15 op/s 2023-09-10T06:43:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:37 smithi098 ceph-mon[103124]: pgmap v14372: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 305 KiB/s wr, 15 op/s 2023-09-10T06:43:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:43:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:38.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:43:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:43:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:39 smithi170 ceph-mon[102832]: pgmap v14373: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 305 KiB/s wr, 11 op/s 2023-09-10T06:43:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26650.481891 secs 2023-09-10T06:43:39.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:39 smithi082 ceph-mon[98694]: pgmap v14373: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 305 KiB/s wr, 11 op/s 2023-09-10T06:43:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26650.481891 secs 2023-09-10T06:43:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:39 smithi098 ceph-mon[103124]: pgmap v14373: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 305 KiB/s wr, 11 op/s 2023-09-10T06:43:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26650.481891 secs 2023-09-10T06:43:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26651.094539 secs 2023-09-10T06:43:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:43:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:40.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26651.094539 secs 2023-09-10T06:43:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:43:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26651.094539 secs 2023-09-10T06:43:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:43:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:41 smithi170 ceph-mon[102832]: pgmap v14374: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 305 KiB/s wr, 11 op/s 2023-09-10T06:43:41.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:41 smithi082 ceph-mon[98694]: pgmap v14374: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 305 KiB/s wr, 11 op/s 2023-09-10T06:43:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:41 smithi098 ceph-mon[103124]: pgmap v14374: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 305 KiB/s wr, 11 op/s 2023-09-10T06:43:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:43 smithi170 ceph-mon[102832]: pgmap v14375: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 456 KiB/s wr, 17 op/s 2023-09-10T06:43:43.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:43 smithi082 ceph-mon[98694]: pgmap v14375: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 456 KiB/s wr, 17 op/s 2023-09-10T06:43:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:43 smithi098 ceph-mon[103124]: pgmap v14375: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 456 KiB/s wr, 17 op/s 2023-09-10T06:43:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26655.481952 secs 2023-09-10T06:43:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:43:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:44.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26655.481952 secs 2023-09-10T06:43:44.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:43:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26655.481952 secs 2023-09-10T06:43:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:43:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26656.094604 secs 2023-09-10T06:43:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:45 smithi170 ceph-mon[102832]: pgmap v14376: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T06:43:45.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26656.094604 secs 2023-09-10T06:43:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:45 smithi082 ceph-mon[98694]: pgmap v14376: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T06:43:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26656.094604 secs 2023-09-10T06:43:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:45 smithi098 ceph-mon[103124]: pgmap v14376: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T06:43:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:43:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:46.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:43:46.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:43:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:47 smithi170 ceph-mon[102832]: pgmap v14377: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 601 KiB/s wr, 19 op/s 2023-09-10T06:43:47.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:47 smithi082 ceph-mon[98694]: pgmap v14377: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 601 KiB/s wr, 19 op/s 2023-09-10T06:43:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:47 smithi098 ceph-mon[103124]: pgmap v14377: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 601 KiB/s wr, 19 op/s 2023-09-10T06:43:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:49 smithi170 ceph-mon[102832]: pgmap v14378: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 491 KiB/s wr, 13 op/s 2023-09-10T06:43:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26660.482035 secs 2023-09-10T06:43:49.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:49 smithi082 ceph-mon[98694]: pgmap v14378: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 491 KiB/s wr, 13 op/s 2023-09-10T06:43:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26660.482035 secs 2023-09-10T06:43:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:49 smithi098 ceph-mon[103124]: pgmap v14378: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 491 KiB/s wr, 13 op/s 2023-09-10T06:43:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26660.482035 secs 2023-09-10T06:43:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26661.094667 secs 2023-09-10T06:43:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:43:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:50.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26661.094667 secs 2023-09-10T06:43:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:43:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26661.094667 secs 2023-09-10T06:43:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:43:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:51 smithi170 ceph-mon[102832]: pgmap v14379: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 492 KiB/s wr, 13 op/s 2023-09-10T06:43:51.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:51 smithi082 ceph-mon[98694]: pgmap v14379: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 492 KiB/s wr, 13 op/s 2023-09-10T06:43:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:51 smithi098 ceph-mon[103124]: pgmap v14379: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 492 KiB/s wr, 13 op/s 2023-09-10T06:43:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:53 smithi170 ceph-mon[102832]: pgmap v14380: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 535 KiB/s wr, 19 op/s 2023-09-10T06:43:53.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:53 smithi082 ceph-mon[98694]: pgmap v14380: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 535 KiB/s wr, 19 op/s 2023-09-10T06:43:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:53 smithi098 ceph-mon[103124]: pgmap v14380: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 535 KiB/s wr, 19 op/s 2023-09-10T06:43:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26665.482116 secs 2023-09-10T06:43:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:43:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:54.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26665.482116 secs 2023-09-10T06:43:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:43:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26665.482116 secs 2023-09-10T06:43:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:43:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26666.094732 secs 2023-09-10T06:43:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:55 smithi170 ceph-mon[102832]: pgmap v14381: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 384 KiB/s wr, 12 op/s 2023-09-10T06:43:55.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26666.094732 secs 2023-09-10T06:43:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:55 smithi082 ceph-mon[98694]: pgmap v14381: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 384 KiB/s wr, 12 op/s 2023-09-10T06:43:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26666.094732 secs 2023-09-10T06:43:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:55 smithi098 ceph-mon[103124]: pgmap v14381: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 384 KiB/s wr, 12 op/s 2023-09-10T06:43:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:43:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:56.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:43:56.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:43:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:43:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:57 smithi170 ceph-mon[102832]: pgmap v14382: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 710 KiB/s wr, 19 op/s 2023-09-10T06:43:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:57 smithi082 ceph-mon[98694]: pgmap v14382: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 710 KiB/s wr, 19 op/s 2023-09-10T06:43:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:57 smithi098 ceph-mon[103124]: pgmap v14382: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 710 KiB/s wr, 19 op/s 2023-09-10T06:43:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26669.476145 secs 2023-09-10T06:43:58.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26669.476145 secs 2023-09-10T06:43:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26669.476145 secs 2023-09-10T06:43:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:59 smithi170 ceph-mon[102832]: pgmap v14383: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 369 KiB/s wr, 12 op/s 2023-09-10T06:43:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:43:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26670.482177 secs 2023-09-10T06:43:59.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:59 smithi082 ceph-mon[98694]: pgmap v14383: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 369 KiB/s wr, 12 op/s 2023-09-10T06:43:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:43:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26670.482177 secs 2023-09-10T06:43:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:59 smithi098 ceph-mon[103124]: pgmap v14383: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 369 KiB/s wr, 12 op/s 2023-09-10T06:43:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:43:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26670.482177 secs 2023-09-10T06:44:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26671.094820 secs 2023-09-10T06:44:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:44:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:44:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:00.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26671.094820 secs 2023-09-10T06:44:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:44:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:44:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26671.094820 secs 2023-09-10T06:44:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:44:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:44:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:01 smithi170 ceph-mon[102832]: pgmap v14384: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 543 KiB/s wr, 17 op/s 2023-09-10T06:44:01.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:01 smithi082 ceph-mon[98694]: pgmap v14384: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 543 KiB/s wr, 17 op/s 2023-09-10T06:44:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:01 smithi098 ceph-mon[103124]: pgmap v14384: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 543 KiB/s wr, 17 op/s 2023-09-10T06:44:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:03 smithi170 ceph-mon[102832]: pgmap v14385: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 543 KiB/s wr, 18 op/s 2023-09-10T06:44:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26674.476202 secs 2023-09-10T06:44:03.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:03 smithi082 ceph-mon[98694]: pgmap v14385: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 543 KiB/s wr, 18 op/s 2023-09-10T06:44:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26674.476202 secs 2023-09-10T06:44:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:03 smithi098 ceph-mon[103124]: pgmap v14385: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 543 KiB/s wr, 18 op/s 2023-09-10T06:44:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26674.476202 secs 2023-09-10T06:44:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26675.482248 secs 2023-09-10T06:44:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:44:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:04.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26675.482248 secs 2023-09-10T06:44:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:44:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26675.482248 secs 2023-09-10T06:44:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:44:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26676.094884 secs 2023-09-10T06:44:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:05 smithi170 ceph-mon[102832]: pgmap v14386: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 499 KiB/s wr, 12 op/s 2023-09-10T06:44:05.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26676.094884 secs 2023-09-10T06:44:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:05 smithi082 ceph-mon[98694]: pgmap v14386: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 499 KiB/s wr, 12 op/s 2023-09-10T06:44:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26676.094884 secs 2023-09-10T06:44:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:05 smithi098 ceph-mon[103124]: pgmap v14386: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 499 KiB/s wr, 12 op/s 2023-09-10T06:44:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:44:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:06.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:44:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:44:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:07 smithi170 ceph-mon[102832]: pgmap v14387: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 675 KiB/s wr, 33 op/s 2023-09-10T06:44:07.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:07 smithi082 ceph-mon[98694]: pgmap v14387: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 675 KiB/s wr, 33 op/s 2023-09-10T06:44:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:07 smithi098 ceph-mon[103124]: pgmap v14387: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 675 KiB/s wr, 33 op/s 2023-09-10T06:44:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26679.476274 secs 2023-09-10T06:44:08.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26679.476274 secs 2023-09-10T06:44:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26679.476274 secs 2023-09-10T06:44:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26680.482327 secs 2023-09-10T06:44:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:09 smithi170 ceph-mon[102832]: pgmap v14388: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 349 KiB/s wr, 26 op/s 2023-09-10T06:44:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:09 smithi170 ceph-mon[102832]: osdmap e967: 12 total, 12 up, 12 in 2023-09-10T06:44:09.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26680.482327 secs 2023-09-10T06:44:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:09 smithi082 ceph-mon[98694]: pgmap v14388: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 349 KiB/s wr, 26 op/s 2023-09-10T06:44:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:09 smithi082 ceph-mon[98694]: osdmap e967: 12 total, 12 up, 12 in 2023-09-10T06:44:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26680.482327 secs 2023-09-10T06:44:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:09 smithi098 ceph-mon[103124]: pgmap v14388: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 349 KiB/s wr, 26 op/s 2023-09-10T06:44:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:09 smithi098 ceph-mon[103124]: osdmap e967: 12 total, 12 up, 12 in 2023-09-10T06:44:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26681.094952 secs 2023-09-10T06:44:10.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26681.094952 secs 2023-09-10T06:44:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26681.094952 secs 2023-09-10T06:44:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:11 smithi170 ceph-mon[102832]: pgmap v14390: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 212 KiB/s wr, 30 op/s 2023-09-10T06:44:11.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:11 smithi082 ceph-mon[98694]: pgmap v14390: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 212 KiB/s wr, 30 op/s 2023-09-10T06:44:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:11 smithi098 ceph-mon[103124]: pgmap v14390: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 212 KiB/s wr, 30 op/s 2023-09-10T06:44:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:44:12.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:44:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:44:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:13 smithi170 ceph-mon[102832]: pgmap v14391: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 297 KiB/s wr, 30 op/s 2023-09-10T06:44:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26684.476338 secs 2023-09-10T06:44:13.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:13 smithi082 ceph-mon[98694]: pgmap v14391: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 297 KiB/s wr, 30 op/s 2023-09-10T06:44:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26684.476338 secs 2023-09-10T06:44:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:13 smithi098 ceph-mon[103124]: pgmap v14391: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 297 KiB/s wr, 30 op/s 2023-09-10T06:44:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26684.476338 secs 2023-09-10T06:44:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26685.482387 secs 2023-09-10T06:44:14.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26685.482387 secs 2023-09-10T06:44:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26685.482387 secs 2023-09-10T06:44:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26686.095018 secs 2023-09-10T06:44:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:15 smithi170 ceph-mon[102832]: pgmap v14392: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 297 KiB/s wr, 30 op/s 2023-09-10T06:44:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:44:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:44:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:44:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:44:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:44:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:44:15.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:44:15.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:44:15.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:44:15.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:44:15.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:44:15.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26686.095018 secs 2023-09-10T06:44:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:15 smithi082 ceph-mon[98694]: pgmap v14392: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 297 KiB/s wr, 30 op/s 2023-09-10T06:44:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:44:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:44:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:44:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:44:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:44:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:44:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:44:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:44:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:44:15.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:44:15.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:44:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26686.095018 secs 2023-09-10T06:44:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:15 smithi098 ceph-mon[103124]: pgmap v14392: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 297 KiB/s wr, 30 op/s 2023-09-10T06:44:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:44:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:44:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:44:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:44:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:44:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:44:15.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:44:15.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:44:15.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:44:15.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:44:15.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:44:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:16 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T06:44:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:16 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T06:44:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:44:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:44:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:44:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:44:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:16 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T06:44:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:16 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T06:44:16.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:44:16.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:44:16.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:44:16.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:16 smithi170 ceph-mon[102832]: osdmap e968: 12 total, 12 up, 12 in 2023-09-10T06:44:16.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:16 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T06:44:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:16 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T06:44:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:44:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:44:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:44:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:44:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:16 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T06:44:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:16 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T06:44:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:44:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:44:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:44:16.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:16 smithi082 ceph-mon[98694]: osdmap e968: 12 total, 12 up, 12 in 2023-09-10T06:44:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:16 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T06:44:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:16 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T06:44:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:44:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:44:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:44:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:44:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:16 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T06:44:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:16 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T06:44:16.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:44:16.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:44:16.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:44:16.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:16 smithi098 ceph-mon[103124]: osdmap e968: 12 total, 12 up, 12 in 2023-09-10T06:44:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:17 smithi170 ceph-mon[102832]: pgmap v14394: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 109 KiB/s wr, 15 op/s 2023-09-10T06:44:17.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:17 smithi082 ceph-mon[98694]: pgmap v14394: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 109 KiB/s wr, 15 op/s 2023-09-10T06:44:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:17 smithi098 ceph-mon[103124]: pgmap v14394: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 109 KiB/s wr, 15 op/s 2023-09-10T06:44:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26689.476397 secs 2023-09-10T06:44:18.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26689.476397 secs 2023-09-10T06:44:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26689.476397 secs 2023-09-10T06:44:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26690.482460 secs 2023-09-10T06:44:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:19 smithi170 ceph-mon[102832]: pgmap v14395: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T06:44:19.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26690.482460 secs 2023-09-10T06:44:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:19 smithi082 ceph-mon[98694]: pgmap v14395: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T06:44:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26690.482460 secs 2023-09-10T06:44:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:19 smithi098 ceph-mon[103124]: pgmap v14395: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 12 op/s 2023-09-10T06:44:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26691.095082 secs 2023-09-10T06:44:20.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26691.095082 secs 2023-09-10T06:44:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26691.095082 secs 2023-09-10T06:44:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:21 smithi170 ceph-mon[102832]: pgmap v14396: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 12 op/s 2023-09-10T06:44:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:21 smithi082 ceph-mon[98694]: pgmap v14396: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 12 op/s 2023-09-10T06:44:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:21 smithi098 ceph-mon[103124]: pgmap v14396: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 12 op/s 2023-09-10T06:44:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:23 smithi170 ceph-mon[102832]: pgmap v14397: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T06:44:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26694.476501 secs 2023-09-10T06:44:23.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:23 smithi082 ceph-mon[98694]: pgmap v14397: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T06:44:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26694.476501 secs 2023-09-10T06:44:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:23 smithi098 ceph-mon[103124]: pgmap v14397: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T06:44:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26694.476501 secs 2023-09-10T06:44:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26695.482497 secs 2023-09-10T06:44:24.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26695.482497 secs 2023-09-10T06:44:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26695.482497 secs 2023-09-10T06:44:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26696.095143 secs 2023-09-10T06:44:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:25 smithi170 ceph-mon[102832]: pgmap v14398: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T06:44:25.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26696.095143 secs 2023-09-10T06:44:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:25 smithi082 ceph-mon[98694]: pgmap v14398: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T06:44:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26696.095143 secs 2023-09-10T06:44:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:25 smithi098 ceph-mon[103124]: pgmap v14398: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T06:44:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:27 smithi170 ceph-mon[102832]: pgmap v14399: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 11 op/s 2023-09-10T06:44:27.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:27 smithi082 ceph-mon[98694]: pgmap v14399: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 11 op/s 2023-09-10T06:44:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:27 smithi098 ceph-mon[103124]: pgmap v14399: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 11 op/s 2023-09-10T06:44:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:29 smithi170 ceph-mon[102832]: pgmap v14400: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 92 KiB/s wr, 9 op/s 2023-09-10T06:44:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26700.616648 secs 2023-09-10T06:44:29.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:29 smithi082 ceph-mon[98694]: pgmap v14400: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 92 KiB/s wr, 9 op/s 2023-09-10T06:44:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26700.616648 secs 2023-09-10T06:44:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:29 smithi098 ceph-mon[103124]: pgmap v14400: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 92 KiB/s wr, 9 op/s 2023-09-10T06:44:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26700.616648 secs 2023-09-10T06:44:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26701.095210 secs 2023-09-10T06:44:30.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26701.095210 secs 2023-09-10T06:44:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26701.095210 secs 2023-09-10T06:44:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:31 smithi170 ceph-mon[102832]: pgmap v14401: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 13 op/s 2023-09-10T06:44:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:44:31.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:31 smithi082 ceph-mon[98694]: pgmap v14401: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 13 op/s 2023-09-10T06:44:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:44:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:31 smithi098 ceph-mon[103124]: pgmap v14401: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 13 op/s 2023-09-10T06:44:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:44:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:44:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:44:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:32.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:44:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:44:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:44:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:44:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:33 smithi170 ceph-mon[102832]: pgmap v14402: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 268 KiB/s wr, 10 op/s 2023-09-10T06:44:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:33 smithi082 ceph-mon[98694]: pgmap v14402: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 268 KiB/s wr, 10 op/s 2023-09-10T06:44:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:33 smithi098 ceph-mon[103124]: pgmap v14402: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 268 KiB/s wr, 10 op/s 2023-09-10T06:44:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:44:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26705.616706 secs 2023-09-10T06:44:34.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:44:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26705.616706 secs 2023-09-10T06:44:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:44:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26705.616706 secs 2023-09-10T06:44:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26706.095272 secs 2023-09-10T06:44:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:35 smithi170 ceph-mon[102832]: pgmap v14403: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 9 op/s 2023-09-10T06:44:35.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26706.095272 secs 2023-09-10T06:44:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:35 smithi082 ceph-mon[98694]: pgmap v14403: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 9 op/s 2023-09-10T06:44:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26706.095272 secs 2023-09-10T06:44:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:35 smithi098 ceph-mon[103124]: pgmap v14403: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 9 op/s 2023-09-10T06:44:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:44:36.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:36.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:44:36.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:44:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:37 smithi170 ceph-mon[102832]: pgmap v14404: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 401 KiB/s wr, 16 op/s 2023-09-10T06:44:37.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:37 smithi082 ceph-mon[98694]: pgmap v14404: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 401 KiB/s wr, 16 op/s 2023-09-10T06:44:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:37 smithi098 ceph-mon[103124]: pgmap v14404: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 401 KiB/s wr, 16 op/s 2023-09-10T06:44:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:39 smithi170 ceph-mon[102832]: pgmap v14405: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 401 KiB/s wr, 11 op/s 2023-09-10T06:44:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26710.616795 secs 2023-09-10T06:44:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:39 smithi082 ceph-mon[98694]: pgmap v14405: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 401 KiB/s wr, 11 op/s 2023-09-10T06:44:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26710.616795 secs 2023-09-10T06:44:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:39 smithi098 ceph-mon[103124]: pgmap v14405: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 401 KiB/s wr, 11 op/s 2023-09-10T06:44:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26710.616795 secs 2023-09-10T06:44:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26711.095337 secs 2023-09-10T06:44:40.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26711.095337 secs 2023-09-10T06:44:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26711.095337 secs 2023-09-10T06:44:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:41 smithi170 ceph-mon[102832]: pgmap v14406: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 419 KiB/s wr, 16 op/s 2023-09-10T06:44:41.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:41 smithi082 ceph-mon[98694]: pgmap v14406: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 419 KiB/s wr, 16 op/s 2023-09-10T06:44:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:41 smithi098 ceph-mon[103124]: pgmap v14406: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 419 KiB/s wr, 16 op/s 2023-09-10T06:44:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:43 smithi170 ceph-mon[102832]: pgmap v14407: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 370 KiB/s wr, 12 op/s 2023-09-10T06:44:43.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:43 smithi082 ceph-mon[98694]: pgmap v14407: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 370 KiB/s wr, 12 op/s 2023-09-10T06:44:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:43 smithi098 ceph-mon[103124]: pgmap v14407: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 370 KiB/s wr, 12 op/s 2023-09-10T06:44:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:44:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26715.616855 secs 2023-09-10T06:44:44.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:44:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26715.616855 secs 2023-09-10T06:44:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:44:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26715.616855 secs 2023-09-10T06:44:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26716.095399 secs 2023-09-10T06:44:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:45 smithi170 ceph-mon[102832]: pgmap v14408: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 242 KiB/s wr, 11 op/s 2023-09-10T06:44:45.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26716.095399 secs 2023-09-10T06:44:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:45 smithi082 ceph-mon[98694]: pgmap v14408: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 242 KiB/s wr, 11 op/s 2023-09-10T06:44:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26716.095399 secs 2023-09-10T06:44:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:45 smithi098 ceph-mon[103124]: pgmap v14408: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 242 KiB/s wr, 11 op/s 2023-09-10T06:44:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:44:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:46.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:44:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:44:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:47 smithi170 ceph-mon[102832]: pgmap v14409: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 737 KiB/s wr, 18 op/s 2023-09-10T06:44:47.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:47 smithi082 ceph-mon[98694]: pgmap v14409: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 737 KiB/s wr, 18 op/s 2023-09-10T06:44:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:47 smithi098 ceph-mon[103124]: pgmap v14409: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 737 KiB/s wr, 18 op/s 2023-09-10T06:44:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:49 smithi170 ceph-mon[102832]: pgmap v14410: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 513 KiB/s wr, 12 op/s 2023-09-10T06:44:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26720.616914 secs 2023-09-10T06:44:49.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:49 smithi082 ceph-mon[98694]: pgmap v14410: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 513 KiB/s wr, 12 op/s 2023-09-10T06:44:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26720.616914 secs 2023-09-10T06:44:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:49 smithi098 ceph-mon[103124]: pgmap v14410: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 513 KiB/s wr, 12 op/s 2023-09-10T06:44:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26720.616914 secs 2023-09-10T06:44:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26721.095462 secs 2023-09-10T06:44:50.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26721.095462 secs 2023-09-10T06:44:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26721.095462 secs 2023-09-10T06:44:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:51 smithi170 ceph-mon[102832]: pgmap v14411: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 551 KiB/s wr, 17 op/s 2023-09-10T06:44:51.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:51 smithi082 ceph-mon[98694]: pgmap v14411: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 551 KiB/s wr, 17 op/s 2023-09-10T06:44:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:51 smithi098 ceph-mon[103124]: pgmap v14411: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 551 KiB/s wr, 17 op/s 2023-09-10T06:44:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:44:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:52.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:44:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:44:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:53 smithi170 ceph-mon[102832]: pgmap v14412: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 534 KiB/s wr, 13 op/s 2023-09-10T06:44:53.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:53 smithi082 ceph-mon[98694]: pgmap v14412: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 534 KiB/s wr, 13 op/s 2023-09-10T06:44:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:53 smithi098 ceph-mon[103124]: pgmap v14412: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 534 KiB/s wr, 13 op/s 2023-09-10T06:44:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26725.616999 secs 2023-09-10T06:44:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:44:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:54.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26725.616999 secs 2023-09-10T06:44:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:44:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26725.616999 secs 2023-09-10T06:44:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:44:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:44:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26726.095525 secs 2023-09-10T06:44:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:55 smithi170 ceph-mon[102832]: pgmap v14413: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 534 KiB/s wr, 12 op/s 2023-09-10T06:44:55.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26726.095525 secs 2023-09-10T06:44:55.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:55 smithi082 ceph-mon[98694]: pgmap v14413: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 534 KiB/s wr, 12 op/s 2023-09-10T06:44:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26726.095525 secs 2023-09-10T06:44:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:55 smithi098 ceph-mon[103124]: pgmap v14413: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 534 KiB/s wr, 12 op/s 2023-09-10T06:44:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:57 smithi170 ceph-mon[102832]: pgmap v14414: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 714 KiB/s wr, 18 op/s 2023-09-10T06:44:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:57 smithi082 ceph-mon[98694]: pgmap v14414: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 714 KiB/s wr, 18 op/s 2023-09-10T06:44:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:57 smithi098 ceph-mon[103124]: pgmap v14414: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 714 KiB/s wr, 18 op/s 2023-09-10T06:44:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26729.476995 secs 2023-09-10T06:44:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:59 smithi170 ceph-mon[102832]: pgmap v14415: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 218 KiB/s wr, 11 op/s 2023-09-10T06:44:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:44:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26730.617059 secs 2023-09-10T06:44:59.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26729.476995 secs 2023-09-10T06:44:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:59 smithi082 ceph-mon[98694]: pgmap v14415: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 218 KiB/s wr, 11 op/s 2023-09-10T06:44:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:44:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26730.617059 secs 2023-09-10T06:44:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26729.476995 secs 2023-09-10T06:44:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:59 smithi098 ceph-mon[103124]: pgmap v14415: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 218 KiB/s wr, 11 op/s 2023-09-10T06:44:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:44:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26730.617059 secs 2023-09-10T06:45:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26731.095588 secs 2023-09-10T06:45:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:45:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:45:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:00.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26731.095588 secs 2023-09-10T06:45:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:45:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:45:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26731.095588 secs 2023-09-10T06:45:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:45:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:45:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:01 smithi170 ceph-mon[102832]: pgmap v14416: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 344 KiB/s wr, 15 op/s 2023-09-10T06:45:01.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:01 smithi082 ceph-mon[98694]: pgmap v14416: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 344 KiB/s wr, 15 op/s 2023-09-10T06:45:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:01 smithi098 ceph-mon[103124]: pgmap v14416: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 344 KiB/s wr, 15 op/s 2023-09-10T06:45:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:03 smithi170 ceph-mon[102832]: pgmap v14417: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 406 KiB/s wr, 11 op/s 2023-09-10T06:45:03.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:03 smithi082 ceph-mon[98694]: pgmap v14417: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 406 KiB/s wr, 11 op/s 2023-09-10T06:45:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:03 smithi098 ceph-mon[103124]: pgmap v14417: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 406 KiB/s wr, 11 op/s 2023-09-10T06:45:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26734.477095 secs 2023-09-10T06:45:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:45:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26735.617116 secs 2023-09-10T06:45:04.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26734.477095 secs 2023-09-10T06:45:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:45:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26735.617116 secs 2023-09-10T06:45:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26734.477095 secs 2023-09-10T06:45:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:45:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26735.617116 secs 2023-09-10T06:45:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26736.095650 secs 2023-09-10T06:45:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:05 smithi170 ceph-mon[102832]: pgmap v14418: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 406 KiB/s wr, 11 op/s 2023-09-10T06:45:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26736.095650 secs 2023-09-10T06:45:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:05 smithi082 ceph-mon[98694]: pgmap v14418: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 406 KiB/s wr, 11 op/s 2023-09-10T06:45:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26736.095650 secs 2023-09-10T06:45:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:05 smithi098 ceph-mon[103124]: pgmap v14418: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 406 KiB/s wr, 11 op/s 2023-09-10T06:45:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:45:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:06.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:45:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:45:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:07 smithi170 ceph-mon[102832]: pgmap v14419: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 547 KiB/s wr, 32 op/s 2023-09-10T06:45:07.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:07 smithi082 ceph-mon[98694]: pgmap v14419: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 547 KiB/s wr, 32 op/s 2023-09-10T06:45:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:07 smithi098 ceph-mon[103124]: pgmap v14419: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 547 KiB/s wr, 32 op/s 2023-09-10T06:45:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26739.477156 secs 2023-09-10T06:45:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:08 smithi170 ceph-mon[102832]: osdmap e969: 12 total, 12 up, 12 in 2023-09-10T06:45:08.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26739.477156 secs 2023-09-10T06:45:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:08 smithi082 ceph-mon[98694]: osdmap e969: 12 total, 12 up, 12 in 2023-09-10T06:45:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26739.477156 secs 2023-09-10T06:45:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:08 smithi098 ceph-mon[103124]: osdmap e969: 12 total, 12 up, 12 in 2023-09-10T06:45:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:09 smithi170 ceph-mon[102832]: pgmap v14420: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 366 KiB/s wr, 26 op/s 2023-09-10T06:45:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26740.617204 secs 2023-09-10T06:45:09.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:09 smithi082 ceph-mon[98694]: pgmap v14420: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 366 KiB/s wr, 26 op/s 2023-09-10T06:45:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26740.617204 secs 2023-09-10T06:45:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:09 smithi098 ceph-mon[103124]: pgmap v14420: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 366 KiB/s wr, 26 op/s 2023-09-10T06:45:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26740.617204 secs 2023-09-10T06:45:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26741.095714 secs 2023-09-10T06:45:10.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26741.095714 secs 2023-09-10T06:45:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26741.095714 secs 2023-09-10T06:45:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:11 smithi170 ceph-mon[102832]: pgmap v14422: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 310 KiB/s wr, 31 op/s 2023-09-10T06:45:11.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:11 smithi082 ceph-mon[98694]: pgmap v14422: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 310 KiB/s wr, 31 op/s 2023-09-10T06:45:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:11 smithi098 ceph-mon[103124]: pgmap v14422: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 310 KiB/s wr, 31 op/s 2023-09-10T06:45:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:13 smithi170 ceph-mon[102832]: pgmap v14423: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 190 KiB/s wr, 31 op/s 2023-09-10T06:45:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26744.477215 secs 2023-09-10T06:45:13.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:13 smithi082 ceph-mon[98694]: pgmap v14423: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 190 KiB/s wr, 31 op/s 2023-09-10T06:45:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26744.477215 secs 2023-09-10T06:45:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:13 smithi098 ceph-mon[103124]: pgmap v14423: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 190 KiB/s wr, 31 op/s 2023-09-10T06:45:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26744.477215 secs 2023-09-10T06:45:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26745.617261 secs 2023-09-10T06:45:14.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26745.617261 secs 2023-09-10T06:45:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26745.617261 secs 2023-09-10T06:45:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26746.095776 secs 2023-09-10T06:45:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:15 smithi170 ceph-mon[102832]: pgmap v14424: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 190 KiB/s wr, 31 op/s 2023-09-10T06:45:15.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26746.095776 secs 2023-09-10T06:45:15.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:15 smithi082 ceph-mon[98694]: pgmap v14424: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 190 KiB/s wr, 31 op/s 2023-09-10T06:45:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26746.095776 secs 2023-09-10T06:45:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:15 smithi098 ceph-mon[103124]: pgmap v14424: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 190 KiB/s wr, 31 op/s 2023-09-10T06:45:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:45:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:16 smithi170 ceph-mon[102832]: osdmap e970: 12 total, 12 up, 12 in 2023-09-10T06:45:16.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:45:16.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:16 smithi082 ceph-mon[98694]: osdmap e970: 12 total, 12 up, 12 in 2023-09-10T06:45:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:45:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:16 smithi098 ceph-mon[103124]: osdmap e970: 12 total, 12 up, 12 in 2023-09-10T06:45:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:17 smithi170 ceph-mon[102832]: pgmap v14426: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 250 KiB/s wr, 16 op/s 2023-09-10T06:45:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:45:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:45:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:45:17.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:17 smithi082 ceph-mon[98694]: pgmap v14426: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 250 KiB/s wr, 16 op/s 2023-09-10T06:45:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:45:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:45:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:45:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:17 smithi098 ceph-mon[103124]: pgmap v14426: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 250 KiB/s wr, 16 op/s 2023-09-10T06:45:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:45:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:45:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:45:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26749.477278 secs 2023-09-10T06:45:18.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26749.477278 secs 2023-09-10T06:45:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26749.477278 secs 2023-09-10T06:45:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:19 smithi170 ceph-mon[102832]: pgmap v14427: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 202 KiB/s wr, 13 op/s 2023-09-10T06:45:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26750.617319 secs 2023-09-10T06:45:19.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:19 smithi082 ceph-mon[98694]: pgmap v14427: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 202 KiB/s wr, 13 op/s 2023-09-10T06:45:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26750.617319 secs 2023-09-10T06:45:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:19 smithi098 ceph-mon[103124]: pgmap v14427: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 202 KiB/s wr, 13 op/s 2023-09-10T06:45:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26750.617319 secs 2023-09-10T06:45:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26751.095840 secs 2023-09-10T06:45:20.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26751.095840 secs 2023-09-10T06:45:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26751.095840 secs 2023-09-10T06:45:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:21 smithi170 ceph-mon[102832]: pgmap v14428: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 12 op/s 2023-09-10T06:45:21.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:21 smithi082 ceph-mon[98694]: pgmap v14428: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 12 op/s 2023-09-10T06:45:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:21 smithi098 ceph-mon[103124]: pgmap v14428: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 12 op/s 2023-09-10T06:45:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:23 smithi170 ceph-mon[102832]: pgmap v14429: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 12 op/s 2023-09-10T06:45:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26754.477338 secs 2023-09-10T06:45:23.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:23 smithi082 ceph-mon[98694]: pgmap v14429: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 12 op/s 2023-09-10T06:45:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26754.477338 secs 2023-09-10T06:45:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:23 smithi098 ceph-mon[103124]: pgmap v14429: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 12 op/s 2023-09-10T06:45:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26754.477338 secs 2023-09-10T06:45:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26755.617389 secs 2023-09-10T06:45:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:45:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:24.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26755.617389 secs 2023-09-10T06:45:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:45:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26755.617389 secs 2023-09-10T06:45:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:45:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26756.095901 secs 2023-09-10T06:45:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:25 smithi170 ceph-mon[102832]: pgmap v14430: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 12 op/s 2023-09-10T06:45:25.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26756.095901 secs 2023-09-10T06:45:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:25 smithi082 ceph-mon[98694]: pgmap v14430: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 12 op/s 2023-09-10T06:45:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26756.095901 secs 2023-09-10T06:45:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:25 smithi098 ceph-mon[103124]: pgmap v14430: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 12 op/s 2023-09-10T06:45:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:26 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:45:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:26 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:45:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:26 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:45:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:27 smithi170 ceph-mon[102832]: pgmap v14431: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 12 op/s 2023-09-10T06:45:27.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:27 smithi082 ceph-mon[98694]: pgmap v14431: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 12 op/s 2023-09-10T06:45:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:27 smithi098 ceph-mon[103124]: pgmap v14431: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 12 op/s 2023-09-10T06:45:29.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:29 smithi082 ceph-mon[98694]: pgmap v14432: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T06:45:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26760.617446 secs 2023-09-10T06:45:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:29 smithi098 ceph-mon[103124]: pgmap v14432: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T06:45:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26760.617446 secs 2023-09-10T06:45:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:29 smithi170 ceph-mon[102832]: pgmap v14432: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T06:45:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26760.617446 secs 2023-09-10T06:45:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26761.095968 secs 2023-09-10T06:45:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26761.095968 secs 2023-09-10T06:45:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26761.095968 secs 2023-09-10T06:45:31.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:31 smithi082 ceph-mon[98694]: pgmap v14433: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 227 KiB/s wr, 14 op/s 2023-09-10T06:45:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:45:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:31 smithi098 ceph-mon[103124]: pgmap v14433: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 227 KiB/s wr, 14 op/s 2023-09-10T06:45:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:45:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:31 smithi170 ceph-mon[102832]: pgmap v14433: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 227 KiB/s wr, 14 op/s 2023-09-10T06:45:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:45:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:45:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:45:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:45:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:45:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:45:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:45:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:33 smithi082 ceph-mon[98694]: pgmap v14434: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T06:45:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:33 smithi098 ceph-mon[103124]: pgmap v14434: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T06:45:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:33 smithi170 ceph-mon[102832]: pgmap v14434: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T06:45:34.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26765.617516 secs 2023-09-10T06:45:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26765.617516 secs 2023-09-10T06:45:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26765.617516 secs 2023-09-10T06:45:35.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26766.096029 secs 2023-09-10T06:45:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:35 smithi082 ceph-mon[98694]: pgmap v14435: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 10 op/s 2023-09-10T06:45:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26766.096029 secs 2023-09-10T06:45:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:35 smithi098 ceph-mon[103124]: pgmap v14435: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 10 op/s 2023-09-10T06:45:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26766.096029 secs 2023-09-10T06:45:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:35 smithi170 ceph-mon[102832]: pgmap v14435: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 10 op/s 2023-09-10T06:45:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:37 smithi170 ceph-mon[102832]: pgmap v14436: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 336 KiB/s wr, 16 op/s 2023-09-10T06:45:37.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:37 smithi082 ceph-mon[98694]: pgmap v14436: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 336 KiB/s wr, 16 op/s 2023-09-10T06:45:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:37 smithi098 ceph-mon[103124]: pgmap v14436: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 336 KiB/s wr, 16 op/s 2023-09-10T06:45:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:45:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:45:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:45:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:39 smithi170 ceph-mon[102832]: pgmap v14437: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 10 op/s 2023-09-10T06:45:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26770.617576 secs 2023-09-10T06:45:39.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:39 smithi082 ceph-mon[98694]: pgmap v14437: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 10 op/s 2023-09-10T06:45:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26770.617576 secs 2023-09-10T06:45:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:39 smithi098 ceph-mon[103124]: pgmap v14437: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 10 op/s 2023-09-10T06:45:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26770.617576 secs 2023-09-10T06:45:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26771.096093 secs 2023-09-10T06:45:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:45:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:40.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26771.096093 secs 2023-09-10T06:45:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:45:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26771.096093 secs 2023-09-10T06:45:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:45:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:41 smithi170 ceph-mon[102832]: pgmap v14438: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 573 KiB/s wr, 37 op/s 2023-09-10T06:45:41.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:41 smithi082 ceph-mon[98694]: pgmap v14438: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 573 KiB/s wr, 37 op/s 2023-09-10T06:45:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:41 smithi098 ceph-mon[103124]: pgmap v14438: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 573 KiB/s wr, 37 op/s 2023-09-10T06:45:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:43 smithi170 ceph-mon[102832]: pgmap v14439: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 455 KiB/s wr, 33 op/s 2023-09-10T06:45:43.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:43 smithi082 ceph-mon[98694]: pgmap v14439: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 455 KiB/s wr, 33 op/s 2023-09-10T06:45:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:43 smithi098 ceph-mon[103124]: pgmap v14439: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 455 KiB/s wr, 33 op/s 2023-09-10T06:45:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:45:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26775.617636 secs 2023-09-10T06:45:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:45:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26775.617636 secs 2023-09-10T06:45:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:45:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26775.617636 secs 2023-09-10T06:45:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26776.096153 secs 2023-09-10T06:45:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:45 smithi170 ceph-mon[102832]: pgmap v14440: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 455 KiB/s wr, 33 op/s 2023-09-10T06:45:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26776.096153 secs 2023-09-10T06:45:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:45 smithi082 ceph-mon[98694]: pgmap v14440: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 455 KiB/s wr, 33 op/s 2023-09-10T06:45:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26776.096153 secs 2023-09-10T06:45:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:45 smithi098 ceph-mon[103124]: pgmap v14440: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 455 KiB/s wr, 33 op/s 2023-09-10T06:45:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:45:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:46.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:45:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:45:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:47 smithi170 ceph-mon[102832]: pgmap v14441: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 747 KiB/s wr, 40 op/s 2023-09-10T06:45:47.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:47 smithi082 ceph-mon[98694]: pgmap v14441: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 747 KiB/s wr, 40 op/s 2023-09-10T06:45:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:47 smithi098 ceph-mon[103124]: pgmap v14441: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 747 KiB/s wr, 40 op/s 2023-09-10T06:45:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:49 smithi170 ceph-mon[102832]: pgmap v14442: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 637 KiB/s wr, 34 op/s 2023-09-10T06:45:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26780.617706 secs 2023-09-10T06:45:49.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:49 smithi082 ceph-mon[98694]: pgmap v14442: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 637 KiB/s wr, 34 op/s 2023-09-10T06:45:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26780.617706 secs 2023-09-10T06:45:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:49 smithi098 ceph-mon[103124]: pgmap v14442: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 637 KiB/s wr, 34 op/s 2023-09-10T06:45:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26780.617706 secs 2023-09-10T06:45:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26781.096219 secs 2023-09-10T06:45:50.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26781.096219 secs 2023-09-10T06:45:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26781.096219 secs 2023-09-10T06:45:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:51 smithi170 ceph-mon[102832]: pgmap v14443: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 707 KiB/s wr, 40 op/s 2023-09-10T06:45:51.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:51 smithi082 ceph-mon[98694]: pgmap v14443: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 707 KiB/s wr, 40 op/s 2023-09-10T06:45:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:51 smithi098 ceph-mon[103124]: pgmap v14443: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 707 KiB/s wr, 40 op/s 2023-09-10T06:45:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:53 smithi170 ceph-mon[102832]: pgmap v14444: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 360 KiB/s wr, 13 op/s 2023-09-10T06:45:53.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:53 smithi082 ceph-mon[98694]: pgmap v14444: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 360 KiB/s wr, 13 op/s 2023-09-10T06:45:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:53 smithi098 ceph-mon[103124]: pgmap v14444: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 360 KiB/s wr, 13 op/s 2023-09-10T06:45:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26785.617767 secs 2023-09-10T06:45:54.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26785.617767 secs 2023-09-10T06:45:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26785.617767 secs 2023-09-10T06:45:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26786.096285 secs 2023-09-10T06:45:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:55 smithi170 ceph-mon[102832]: pgmap v14445: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 360 KiB/s wr, 13 op/s 2023-09-10T06:45:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26786.096285 secs 2023-09-10T06:45:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:55 smithi082 ceph-mon[98694]: pgmap v14445: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 360 KiB/s wr, 13 op/s 2023-09-10T06:45:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26786.096285 secs 2023-09-10T06:45:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:55 smithi098 ceph-mon[103124]: pgmap v14445: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 360 KiB/s wr, 13 op/s 2023-09-10T06:45:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:57 smithi170 ceph-mon[102832]: pgmap v14446: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 523 KiB/s wr, 19 op/s 2023-09-10T06:45:57.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:57 smithi082 ceph-mon[98694]: pgmap v14446: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 523 KiB/s wr, 19 op/s 2023-09-10T06:45:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:57 smithi098 ceph-mon[103124]: pgmap v14446: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 523 KiB/s wr, 19 op/s 2023-09-10T06:45:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:45:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:58.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:45:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:45:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:45:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26789.477800 secs 2023-09-10T06:45:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:59 smithi170 ceph-mon[102832]: pgmap v14447: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 232 KiB/s wr, 12 op/s 2023-09-10T06:45:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:45:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26790.617851 secs 2023-09-10T06:45:59.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26789.477800 secs 2023-09-10T06:45:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:59 smithi082 ceph-mon[98694]: pgmap v14447: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 232 KiB/s wr, 12 op/s 2023-09-10T06:45:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:45:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26790.617851 secs 2023-09-10T06:45:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26789.477800 secs 2023-09-10T06:45:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:59 smithi098 ceph-mon[103124]: pgmap v14447: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 232 KiB/s wr, 12 op/s 2023-09-10T06:45:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:45:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26790.617851 secs 2023-09-10T06:46:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26791.096353 secs 2023-09-10T06:46:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:46:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:46:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:46:00.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26791.096353 secs 2023-09-10T06:46:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:46:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:46:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:46:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26791.096353 secs 2023-09-10T06:46:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:46:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:46:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:46:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:01 smithi170 ceph-mon[102832]: pgmap v14448: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 464 KiB/s wr, 24 op/s 2023-09-10T06:46:01.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:01 smithi082 ceph-mon[98694]: pgmap v14448: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 464 KiB/s wr, 24 op/s 2023-09-10T06:46:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:01 smithi098 ceph-mon[103124]: pgmap v14448: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 464 KiB/s wr, 24 op/s 2023-09-10T06:46:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:03 smithi170 ceph-mon[102832]: pgmap v14449: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 395 KiB/s wr, 19 op/s 2023-09-10T06:46:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:03 smithi082 ceph-mon[98694]: pgmap v14449: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 395 KiB/s wr, 19 op/s 2023-09-10T06:46:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:03 smithi098 ceph-mon[103124]: pgmap v14449: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 395 KiB/s wr, 19 op/s 2023-09-10T06:46:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26794.477862 secs 2023-09-10T06:46:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26795.617908 secs 2023-09-10T06:46:04.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26794.477862 secs 2023-09-10T06:46:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26795.617908 secs 2023-09-10T06:46:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26794.477862 secs 2023-09-10T06:46:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26795.617908 secs 2023-09-10T06:46:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26796.096416 secs 2023-09-10T06:46:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:05 smithi170 ceph-mon[102832]: pgmap v14450: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 395 KiB/s wr, 19 op/s 2023-09-10T06:46:05.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26796.096416 secs 2023-09-10T06:46:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:05 smithi082 ceph-mon[98694]: pgmap v14450: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 395 KiB/s wr, 19 op/s 2023-09-10T06:46:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26796.096416 secs 2023-09-10T06:46:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:05 smithi098 ceph-mon[103124]: pgmap v14450: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 395 KiB/s wr, 19 op/s 2023-09-10T06:46:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:07 smithi170 ceph-mon[102832]: pgmap v14451: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 509 KiB/s wr, 32 op/s 2023-09-10T06:46:07.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:07 smithi082 ceph-mon[98694]: pgmap v14451: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 509 KiB/s wr, 32 op/s 2023-09-10T06:46:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:07 smithi098 ceph-mon[103124]: pgmap v14451: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 509 KiB/s wr, 32 op/s 2023-09-10T06:46:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26799.477923 secs 2023-09-10T06:46:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:09 smithi170 ceph-mon[102832]: osdmap e971: 12 total, 12 up, 12 in 2023-09-10T06:46:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:09 smithi170 ceph-mon[102832]: pgmap v14453: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 416 KiB/s wr, 31 op/s 2023-09-10T06:46:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26800.617987 secs 2023-09-10T06:46:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26799.477923 secs 2023-09-10T06:46:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:09 smithi082 ceph-mon[98694]: osdmap e971: 12 total, 12 up, 12 in 2023-09-10T06:46:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:09 smithi082 ceph-mon[98694]: pgmap v14453: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 416 KiB/s wr, 31 op/s 2023-09-10T06:46:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26800.617987 secs 2023-09-10T06:46:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26799.477923 secs 2023-09-10T06:46:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:09 smithi098 ceph-mon[103124]: osdmap e971: 12 total, 12 up, 12 in 2023-09-10T06:46:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:09 smithi098 ceph-mon[103124]: pgmap v14453: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 416 KiB/s wr, 31 op/s 2023-09-10T06:46:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26800.617987 secs 2023-09-10T06:46:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26801.096485 secs 2023-09-10T06:46:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26801.096485 secs 2023-09-10T06:46:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26801.096485 secs 2023-09-10T06:46:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:11 smithi170 ceph-mon[102832]: pgmap v14454: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 137 KiB/s wr, 21 op/s 2023-09-10T06:46:11.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:11 smithi082 ceph-mon[98694]: pgmap v14454: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 137 KiB/s wr, 21 op/s 2023-09-10T06:46:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:11 smithi098 ceph-mon[103124]: pgmap v14454: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 137 KiB/s wr, 21 op/s 2023-09-10T06:46:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:13 smithi170 ceph-mon[102832]: pgmap v14455: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 138 KiB/s wr, 21 op/s 2023-09-10T06:46:13.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:13 smithi082 ceph-mon[98694]: pgmap v14455: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 138 KiB/s wr, 21 op/s 2023-09-10T06:46:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:13 smithi098 ceph-mon[103124]: pgmap v14455: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 138 KiB/s wr, 21 op/s 2023-09-10T06:46:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26804.477981 secs 2023-09-10T06:46:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26805.618046 secs 2023-09-10T06:46:14.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26804.477981 secs 2023-09-10T06:46:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26805.618046 secs 2023-09-10T06:46:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26804.477981 secs 2023-09-10T06:46:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26805.618046 secs 2023-09-10T06:46:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26806.096546 secs 2023-09-10T06:46:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:15 smithi170 ceph-mon[102832]: pgmap v14456: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 138 KiB/s wr, 21 op/s 2023-09-10T06:46:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26806.096546 secs 2023-09-10T06:46:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:15 smithi082 ceph-mon[98694]: pgmap v14456: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 138 KiB/s wr, 21 op/s 2023-09-10T06:46:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26806.096546 secs 2023-09-10T06:46:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:15 smithi098 ceph-mon[103124]: pgmap v14456: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 138 KiB/s wr, 21 op/s 2023-09-10T06:46:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:17 smithi170 ceph-mon[102832]: osdmap e972: 12 total, 12 up, 12 in 2023-09-10T06:46:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:17 smithi170 ceph-mon[102832]: pgmap v14458: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.1 KiB/s wr, 15 op/s 2023-09-10T06:46:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:46:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:46:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:46:17.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:17 smithi082 ceph-mon[98694]: osdmap e972: 12 total, 12 up, 12 in 2023-09-10T06:46:17.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:17 smithi082 ceph-mon[98694]: pgmap v14458: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.1 KiB/s wr, 15 op/s 2023-09-10T06:46:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:46:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:46:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:46:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:17 smithi098 ceph-mon[103124]: osdmap e972: 12 total, 12 up, 12 in 2023-09-10T06:46:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:17 smithi098 ceph-mon[103124]: pgmap v14458: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.1 KiB/s wr, 15 op/s 2023-09-10T06:46:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:46:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:46:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:46:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:46:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:46:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:46:18.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:46:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:46:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:46:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:46:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:46:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:46:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26809.478048 secs 2023-09-10T06:46:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:19 smithi170 ceph-mon[102832]: pgmap v14459: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2023-09-10T06:46:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26810.618105 secs 2023-09-10T06:46:19.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26809.478048 secs 2023-09-10T06:46:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:19 smithi082 ceph-mon[98694]: pgmap v14459: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2023-09-10T06:46:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26810.618105 secs 2023-09-10T06:46:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26809.478048 secs 2023-09-10T06:46:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:19 smithi098 ceph-mon[103124]: pgmap v14459: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2023-09-10T06:46:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26810.618105 secs 2023-09-10T06:46:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26811.096611 secs 2023-09-10T06:46:20.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26811.096611 secs 2023-09-10T06:46:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26811.096611 secs 2023-09-10T06:46:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:21 smithi170 ceph-mon[102832]: pgmap v14460: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 KiB/s wr, 12 op/s 2023-09-10T06:46:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:21 smithi082 ceph-mon[98694]: pgmap v14460: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 KiB/s wr, 12 op/s 2023-09-10T06:46:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:21 smithi098 ceph-mon[103124]: pgmap v14460: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 KiB/s wr, 12 op/s 2023-09-10T06:46:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:23 smithi170 ceph-mon[102832]: pgmap v14461: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T06:46:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:23 smithi082 ceph-mon[98694]: pgmap v14461: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T06:46:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:23 smithi098 ceph-mon[103124]: pgmap v14461: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T06:46:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26814.478107 secs 2023-09-10T06:46:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26815.618164 secs 2023-09-10T06:46:24.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26814.478107 secs 2023-09-10T06:46:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26815.618164 secs 2023-09-10T06:46:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26814.478107 secs 2023-09-10T06:46:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26815.618164 secs 2023-09-10T06:46:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26816.096673 secs 2023-09-10T06:46:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:25 smithi170 ceph-mon[102832]: pgmap v14462: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T06:46:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26816.096673 secs 2023-09-10T06:46:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:25 smithi082 ceph-mon[98694]: pgmap v14462: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T06:46:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26816.096673 secs 2023-09-10T06:46:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:25 smithi098 ceph-mon[103124]: pgmap v14462: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T06:46:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:27 smithi170 ceph-mon[102832]: pgmap v14463: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T06:46:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:27 smithi082 ceph-mon[98694]: pgmap v14463: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T06:46:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:27 smithi098 ceph-mon[103124]: pgmap v14463: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T06:46:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26819.478167 secs 2023-09-10T06:46:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:29 smithi170 ceph-mon[102832]: pgmap v14464: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T06:46:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26820.618224 secs 2023-09-10T06:46:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26819.478167 secs 2023-09-10T06:46:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:29 smithi082 ceph-mon[98694]: pgmap v14464: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T06:46:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26820.618224 secs 2023-09-10T06:46:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26819.478167 secs 2023-09-10T06:46:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:29 smithi098 ceph-mon[103124]: pgmap v14464: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T06:46:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26820.618224 secs 2023-09-10T06:46:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26821.096741 secs 2023-09-10T06:46:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:46:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:46:30.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26821.096741 secs 2023-09-10T06:46:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:46:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:46:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26821.096741 secs 2023-09-10T06:46:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:46:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:46:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:31 smithi170 ceph-mon[102832]: pgmap v14465: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 14 op/s 2023-09-10T06:46:31.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:31 smithi082 ceph-mon[98694]: pgmap v14465: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 14 op/s 2023-09-10T06:46:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:31 smithi098 ceph-mon[103124]: pgmap v14465: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 93 KiB/s wr, 14 op/s 2023-09-10T06:46:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:46:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:46:32.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:46:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:46:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:46:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:46:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:33 smithi170 ceph-mon[102832]: pgmap v14466: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 284 KiB/s wr, 10 op/s 2023-09-10T06:46:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26824.478226 secs 2023-09-10T06:46:33.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:33 smithi082 ceph-mon[98694]: pgmap v14466: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 284 KiB/s wr, 10 op/s 2023-09-10T06:46:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26824.478226 secs 2023-09-10T06:46:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:33 smithi098 ceph-mon[103124]: pgmap v14466: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 284 KiB/s wr, 10 op/s 2023-09-10T06:46:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26824.478226 secs 2023-09-10T06:46:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26825.618324 secs 2023-09-10T06:46:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26825.618324 secs 2023-09-10T06:46:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26825.618324 secs 2023-09-10T06:46:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26826.096805 secs 2023-09-10T06:46:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:35 smithi170 ceph-mon[102832]: pgmap v14467: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 9 op/s 2023-09-10T06:46:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26826.096805 secs 2023-09-10T06:46:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:35 smithi082 ceph-mon[98694]: pgmap v14467: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 9 op/s 2023-09-10T06:46:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26826.096805 secs 2023-09-10T06:46:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:35 smithi098 ceph-mon[103124]: pgmap v14467: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 9 op/s 2023-09-10T06:46:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:37 smithi170 ceph-mon[102832]: pgmap v14468: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 308 KiB/s wr, 16 op/s 2023-09-10T06:46:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:37 smithi082 ceph-mon[98694]: pgmap v14468: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 308 KiB/s wr, 16 op/s 2023-09-10T06:46:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:37 smithi098 ceph-mon[103124]: pgmap v14468: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 308 KiB/s wr, 16 op/s 2023-09-10T06:46:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:46:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:46:38.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:46:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:46:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:46:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:46:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:39 smithi170 ceph-mon[102832]: pgmap v14469: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 308 KiB/s wr, 11 op/s 2023-09-10T06:46:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26830.618409 secs 2023-09-10T06:46:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:46:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:39 smithi082 ceph-mon[98694]: pgmap v14469: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 308 KiB/s wr, 11 op/s 2023-09-10T06:46:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26830.618409 secs 2023-09-10T06:46:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:46:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:39 smithi098 ceph-mon[103124]: pgmap v14469: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 308 KiB/s wr, 11 op/s 2023-09-10T06:46:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26830.618409 secs 2023-09-10T06:46:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:46:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26831.096870 secs 2023-09-10T06:46:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:46:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:46:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:46:40.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26831.096870 secs 2023-09-10T06:46:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:46:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:46:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:46:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26831.096870 secs 2023-09-10T06:46:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:46:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:46:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:46:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:41 smithi170 ceph-mon[102832]: pgmap v14470: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 361 KiB/s wr, 16 op/s 2023-09-10T06:46:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:41 smithi082 ceph-mon[98694]: pgmap v14470: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 361 KiB/s wr, 16 op/s 2023-09-10T06:46:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:41 smithi098 ceph-mon[103124]: pgmap v14470: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 361 KiB/s wr, 16 op/s 2023-09-10T06:46:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:43 smithi170 ceph-mon[102832]: pgmap v14471: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 361 KiB/s wr, 13 op/s 2023-09-10T06:46:43.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:43 smithi082 ceph-mon[98694]: pgmap v14471: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 361 KiB/s wr, 13 op/s 2023-09-10T06:46:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:43 smithi098 ceph-mon[103124]: pgmap v14471: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 361 KiB/s wr, 13 op/s 2023-09-10T06:46:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26835.618466 secs 2023-09-10T06:46:44.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26835.618466 secs 2023-09-10T06:46:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26835.618466 secs 2023-09-10T06:46:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26836.096933 secs 2023-09-10T06:46:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:45 smithi170 ceph-mon[102832]: pgmap v14472: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T06:46:45.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26836.096933 secs 2023-09-10T06:46:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:45 smithi082 ceph-mon[98694]: pgmap v14472: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T06:46:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26836.096933 secs 2023-09-10T06:46:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:45 smithi098 ceph-mon[103124]: pgmap v14472: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T06:46:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:47 smithi170 ceph-mon[102832]: pgmap v14473: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 506 KiB/s wr, 19 op/s 2023-09-10T06:46:47.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:47 smithi082 ceph-mon[98694]: pgmap v14473: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 506 KiB/s wr, 19 op/s 2023-09-10T06:46:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:47 smithi098 ceph-mon[103124]: pgmap v14473: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 506 KiB/s wr, 19 op/s 2023-09-10T06:46:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:49 smithi170 ceph-mon[102832]: pgmap v14474: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 396 KiB/s wr, 13 op/s 2023-09-10T06:46:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26840.618526 secs 2023-09-10T06:46:49.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:49 smithi082 ceph-mon[98694]: pgmap v14474: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 396 KiB/s wr, 13 op/s 2023-09-10T06:46:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26840.618526 secs 2023-09-10T06:46:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:49 smithi098 ceph-mon[103124]: pgmap v14474: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 396 KiB/s wr, 13 op/s 2023-09-10T06:46:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26840.618526 secs 2023-09-10T06:46:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26841.096998 secs 2023-09-10T06:46:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:46:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:46:50.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26841.096998 secs 2023-09-10T06:46:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:46:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:46:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26841.096998 secs 2023-09-10T06:46:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:46:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:46:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:51 smithi170 ceph-mon[102832]: pgmap v14475: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 396 KiB/s wr, 17 op/s 2023-09-10T06:46:51.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:51 smithi082 ceph-mon[98694]: pgmap v14475: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 396 KiB/s wr, 17 op/s 2023-09-10T06:46:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:51 smithi098 ceph-mon[103124]: pgmap v14475: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 396 KiB/s wr, 17 op/s 2023-09-10T06:46:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:46:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:46:52.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:46:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:46:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:46:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:46:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:53 smithi170 ceph-mon[102832]: pgmap v14476: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 386 KiB/s wr, 13 op/s 2023-09-10T06:46:53.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:53 smithi082 ceph-mon[98694]: pgmap v14476: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 386 KiB/s wr, 13 op/s 2023-09-10T06:46:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:53 smithi098 ceph-mon[103124]: pgmap v14476: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 386 KiB/s wr, 13 op/s 2023-09-10T06:46:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26845.618584 secs 2023-09-10T06:46:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:46:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:46:54.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26845.618584 secs 2023-09-10T06:46:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:46:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:46:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26845.618584 secs 2023-09-10T06:46:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:46:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:46:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26846.097059 secs 2023-09-10T06:46:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:55 smithi170 ceph-mon[102832]: pgmap v14477: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 386 KiB/s wr, 12 op/s 2023-09-10T06:46:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26846.097059 secs 2023-09-10T06:46:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:55 smithi082 ceph-mon[98694]: pgmap v14477: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 386 KiB/s wr, 12 op/s 2023-09-10T06:46:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26846.097059 secs 2023-09-10T06:46:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:55 smithi098 ceph-mon[103124]: pgmap v14477: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 386 KiB/s wr, 12 op/s 2023-09-10T06:46:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:46:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:46:56.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:46:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:46:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:46:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:46:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:57 smithi170 ceph-mon[102832]: pgmap v14478: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 902 KiB/s wr, 19 op/s 2023-09-10T06:46:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:57 smithi082 ceph-mon[98694]: pgmap v14478: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 902 KiB/s wr, 19 op/s 2023-09-10T06:46:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:57 smithi098 ceph-mon[103124]: pgmap v14478: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 902 KiB/s wr, 19 op/s 2023-09-10T06:46:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26849.478537 secs 2023-09-10T06:46:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26849.478537 secs 2023-09-10T06:46:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26849.478537 secs 2023-09-10T06:46:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:59 smithi170 ceph-mon[102832]: pgmap v14479: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 560 KiB/s wr, 12 op/s 2023-09-10T06:46:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:46:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26850.618642 secs 2023-09-10T06:46:59.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:59 smithi082 ceph-mon[98694]: pgmap v14479: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 560 KiB/s wr, 12 op/s 2023-09-10T06:46:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:46:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26850.618642 secs 2023-09-10T06:46:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:59 smithi098 ceph-mon[103124]: pgmap v14479: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 560 KiB/s wr, 12 op/s 2023-09-10T06:46:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:46:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26850.618642 secs 2023-09-10T06:47:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26851.097133 secs 2023-09-10T06:47:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:47:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:47:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:00.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26851.097133 secs 2023-09-10T06:47:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:47:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:47:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26851.097133 secs 2023-09-10T06:47:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:47:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:47:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:01 smithi170 ceph-mon[102832]: pgmap v14480: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 647 KiB/s wr, 18 op/s 2023-09-10T06:47:01.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:01 smithi082 ceph-mon[98694]: pgmap v14480: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 647 KiB/s wr, 18 op/s 2023-09-10T06:47:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:01 smithi098 ceph-mon[103124]: pgmap v14480: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 647 KiB/s wr, 18 op/s 2023-09-10T06:47:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:03 smithi170 ceph-mon[102832]: pgmap v14481: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 647 KiB/s wr, 14 op/s 2023-09-10T06:47:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26854.478633 secs 2023-09-10T06:47:03.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:03 smithi082 ceph-mon[98694]: pgmap v14481: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 647 KiB/s wr, 14 op/s 2023-09-10T06:47:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26854.478633 secs 2023-09-10T06:47:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:03 smithi098 ceph-mon[103124]: pgmap v14481: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 647 KiB/s wr, 14 op/s 2023-09-10T06:47:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26854.478633 secs 2023-09-10T06:47:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26855.618700 secs 2023-09-10T06:47:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:04 smithi170 ceph-mon[102832]: osdmap e973: 12 total, 12 up, 12 in 2023-09-10T06:47:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:47:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:04.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26855.618700 secs 2023-09-10T06:47:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:04 smithi082 ceph-mon[98694]: osdmap e973: 12 total, 12 up, 12 in 2023-09-10T06:47:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:47:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26855.618700 secs 2023-09-10T06:47:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:04 smithi098 ceph-mon[103124]: osdmap e973: 12 total, 12 up, 12 in 2023-09-10T06:47:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:47:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26856.097199 secs 2023-09-10T06:47:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:05 smithi170 ceph-mon[102832]: pgmap v14483: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 724 KiB/s wr, 16 op/s 2023-09-10T06:47:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26856.097199 secs 2023-09-10T06:47:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:05 smithi082 ceph-mon[98694]: pgmap v14483: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 724 KiB/s wr, 16 op/s 2023-09-10T06:47:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26856.097199 secs 2023-09-10T06:47:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:05 smithi098 ceph-mon[103124]: pgmap v14483: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 724 KiB/s wr, 16 op/s 2023-09-10T06:47:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:47:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:06.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:47:06.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:47:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:07 smithi170 ceph-mon[102832]: pgmap v14484: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 422 KiB/s wr, 32 op/s 2023-09-10T06:47:07.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:07 smithi082 ceph-mon[98694]: pgmap v14484: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 422 KiB/s wr, 32 op/s 2023-09-10T06:47:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:07 smithi098 ceph-mon[103124]: pgmap v14484: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 422 KiB/s wr, 32 op/s 2023-09-10T06:47:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26859.478715 secs 2023-09-10T06:47:08.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26859.478715 secs 2023-09-10T06:47:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26859.478715 secs 2023-09-10T06:47:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:09 smithi170 ceph-mon[102832]: pgmap v14485: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 422 KiB/s wr, 32 op/s 2023-09-10T06:47:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26860.618759 secs 2023-09-10T06:47:09.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:09 smithi082 ceph-mon[98694]: pgmap v14485: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 422 KiB/s wr, 32 op/s 2023-09-10T06:47:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26860.618759 secs 2023-09-10T06:47:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:09 smithi098 ceph-mon[103124]: pgmap v14485: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 422 KiB/s wr, 32 op/s 2023-09-10T06:47:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26860.618759 secs 2023-09-10T06:47:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26861.097264 secs 2023-09-10T06:47:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26861.097264 secs 2023-09-10T06:47:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26861.097264 secs 2023-09-10T06:47:11.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:11 smithi082 ceph-mon[98694]: pgmap v14486: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 342 KiB/s wr, 32 op/s 2023-09-10T06:47:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:11 smithi082 ceph-mon[98694]: osdmap e974: 12 total, 12 up, 12 in 2023-09-10T06:47:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:11 smithi098 ceph-mon[103124]: pgmap v14486: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 342 KiB/s wr, 32 op/s 2023-09-10T06:47:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:11 smithi098 ceph-mon[103124]: osdmap e974: 12 total, 12 up, 12 in 2023-09-10T06:47:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:11 smithi170 ceph-mon[102832]: pgmap v14486: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 342 KiB/s wr, 32 op/s 2023-09-10T06:47:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:11 smithi170 ceph-mon[102832]: osdmap e974: 12 total, 12 up, 12 in 2023-09-10T06:47:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:13 smithi082 ceph-mon[98694]: pgmap v14488: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 386 KiB/s wr, 35 op/s 2023-09-10T06:47:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26864.478781 secs 2023-09-10T06:47:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:13 smithi098 ceph-mon[103124]: pgmap v14488: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 386 KiB/s wr, 35 op/s 2023-09-10T06:47:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26864.478781 secs 2023-09-10T06:47:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:13 smithi170 ceph-mon[102832]: pgmap v14488: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 386 KiB/s wr, 35 op/s 2023-09-10T06:47:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26864.478781 secs 2023-09-10T06:47:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26865.618838 secs 2023-09-10T06:47:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26865.618838 secs 2023-09-10T06:47:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26865.618838 secs 2023-09-10T06:47:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26866.097329 secs 2023-09-10T06:47:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:15 smithi082 ceph-mon[98694]: pgmap v14489: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 342 KiB/s wr, 31 op/s 2023-09-10T06:47:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26866.097329 secs 2023-09-10T06:47:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:15 smithi098 ceph-mon[103124]: pgmap v14489: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 342 KiB/s wr, 31 op/s 2023-09-10T06:47:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26866.097329 secs 2023-09-10T06:47:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:15 smithi170 ceph-mon[102832]: pgmap v14489: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 342 KiB/s wr, 31 op/s 2023-09-10T06:47:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:17 smithi170 ceph-mon[102832]: pgmap v14490: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 130 KiB/s wr, 12 op/s 2023-09-10T06:47:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:17 smithi082 ceph-mon[98694]: pgmap v14490: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 130 KiB/s wr, 12 op/s 2023-09-10T06:47:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:17 smithi098 ceph-mon[103124]: pgmap v14490: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 130 KiB/s wr, 12 op/s 2023-09-10T06:47:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:47:18.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:47:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:47:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26869.478880 secs 2023-09-10T06:47:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:47:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:47:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:47:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:47:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:19 smithi098 ceph-mon[103124]: pgmap v14491: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 130 KiB/s wr, 12 op/s 2023-09-10T06:47:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26870.618919 secs 2023-09-10T06:47:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:47:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:47:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:47:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26869.478880 secs 2023-09-10T06:47:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:47:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:47:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:47:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:47:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:19 smithi170 ceph-mon[102832]: pgmap v14491: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 130 KiB/s wr, 12 op/s 2023-09-10T06:47:19.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26870.618919 secs 2023-09-10T06:47:19.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:47:19.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:47:19.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:47:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26869.478880 secs 2023-09-10T06:47:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:47:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:47:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:47:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:47:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:19 smithi082 ceph-mon[98694]: pgmap v14491: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 130 KiB/s wr, 12 op/s 2023-09-10T06:47:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26870.618919 secs 2023-09-10T06:47:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:47:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:47:19.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:47:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26871.097393 secs 2023-09-10T06:47:20.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26871.097393 secs 2023-09-10T06:47:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26871.097393 secs 2023-09-10T06:47:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:21 smithi170 ceph-mon[102832]: pgmap v14492: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 215 KiB/s wr, 12 op/s 2023-09-10T06:47:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:21 smithi082 ceph-mon[98694]: pgmap v14492: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 215 KiB/s wr, 12 op/s 2023-09-10T06:47:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:21 smithi098 ceph-mon[103124]: pgmap v14492: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 215 KiB/s wr, 12 op/s 2023-09-10T06:47:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:23 smithi170 ceph-mon[102832]: pgmap v14493: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T06:47:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26874.478941 secs 2023-09-10T06:47:23.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:23 smithi082 ceph-mon[98694]: pgmap v14493: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T06:47:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26874.478941 secs 2023-09-10T06:47:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:23 smithi098 ceph-mon[103124]: pgmap v14493: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T06:47:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26874.478941 secs 2023-09-10T06:47:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26875.618978 secs 2023-09-10T06:47:24.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26875.618978 secs 2023-09-10T06:47:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26875.618978 secs 2023-09-10T06:47:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26876.097457 secs 2023-09-10T06:47:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:25 smithi170 ceph-mon[102832]: pgmap v14494: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 10 op/s 2023-09-10T06:47:25.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26876.097457 secs 2023-09-10T06:47:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:25 smithi082 ceph-mon[98694]: pgmap v14494: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 10 op/s 2023-09-10T06:47:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26876.097457 secs 2023-09-10T06:47:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:25 smithi098 ceph-mon[103124]: pgmap v14494: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 10 op/s 2023-09-10T06:47:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:27 smithi170 ceph-mon[102832]: pgmap v14495: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 179 KiB/s wr, 15 op/s 2023-09-10T06:47:27.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:27 smithi082 ceph-mon[98694]: pgmap v14495: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 179 KiB/s wr, 15 op/s 2023-09-10T06:47:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:27 smithi098 ceph-mon[103124]: pgmap v14495: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 179 KiB/s wr, 15 op/s 2023-09-10T06:47:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26879.479045 secs 2023-09-10T06:47:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26879.479045 secs 2023-09-10T06:47:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26879.479045 secs 2023-09-10T06:47:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:29 smithi170 ceph-mon[102832]: pgmap v14496: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 93 KiB/s wr, 10 op/s 2023-09-10T06:47:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26880.619039 secs 2023-09-10T06:47:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:29 smithi082 ceph-mon[98694]: pgmap v14496: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 93 KiB/s wr, 10 op/s 2023-09-10T06:47:29.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26880.619039 secs 2023-09-10T06:47:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:29 smithi098 ceph-mon[103124]: pgmap v14496: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 93 KiB/s wr, 10 op/s 2023-09-10T06:47:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26880.619039 secs 2023-09-10T06:47:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26881.097491 secs 2023-09-10T06:47:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:47:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:30.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26881.097491 secs 2023-09-10T06:47:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:47:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26881.097491 secs 2023-09-10T06:47:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:47:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:31 smithi170 ceph-mon[102832]: pgmap v14497: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 272 KiB/s wr, 15 op/s 2023-09-10T06:47:31.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:31 smithi082 ceph-mon[98694]: pgmap v14497: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 272 KiB/s wr, 15 op/s 2023-09-10T06:47:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:31 smithi098 ceph-mon[103124]: pgmap v14497: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 272 KiB/s wr, 15 op/s 2023-09-10T06:47:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:47:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:32.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:47:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:47:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:33 smithi170 ceph-mon[102832]: pgmap v14498: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 10 op/s 2023-09-10T06:47:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26884.479107 secs 2023-09-10T06:47:33.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:33 smithi082 ceph-mon[98694]: pgmap v14498: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 10 op/s 2023-09-10T06:47:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26884.479107 secs 2023-09-10T06:47:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:33 smithi098 ceph-mon[103124]: pgmap v14498: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 10 op/s 2023-09-10T06:47:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26884.479107 secs 2023-09-10T06:47:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26885.619097 secs 2023-09-10T06:47:34.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26885.619097 secs 2023-09-10T06:47:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26885.619097 secs 2023-09-10T06:47:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26886.097556 secs 2023-09-10T06:47:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:35 smithi170 ceph-mon[102832]: pgmap v14499: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 10 op/s 2023-09-10T06:47:35.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26886.097556 secs 2023-09-10T06:47:35.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:35 smithi082 ceph-mon[98694]: pgmap v14499: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 10 op/s 2023-09-10T06:47:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26886.097556 secs 2023-09-10T06:47:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:35 smithi098 ceph-mon[103124]: pgmap v14499: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 10 op/s 2023-09-10T06:47:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:37 smithi170 ceph-mon[102832]: pgmap v14500: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 405 KiB/s wr, 17 op/s 2023-09-10T06:47:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:37 smithi082 ceph-mon[98694]: pgmap v14500: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 405 KiB/s wr, 17 op/s 2023-09-10T06:47:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:37 smithi098 ceph-mon[103124]: pgmap v14500: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 405 KiB/s wr, 17 op/s 2023-09-10T06:47:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:47:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:47:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:47:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:39 smithi170 ceph-mon[102832]: pgmap v14501: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 405 KiB/s wr, 12 op/s 2023-09-10T06:47:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26890.619155 secs 2023-09-10T06:47:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:47:39.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:39 smithi082 ceph-mon[98694]: pgmap v14501: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 405 KiB/s wr, 12 op/s 2023-09-10T06:47:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26890.619155 secs 2023-09-10T06:47:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:47:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:39 smithi098 ceph-mon[103124]: pgmap v14501: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 405 KiB/s wr, 12 op/s 2023-09-10T06:47:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26890.619155 secs 2023-09-10T06:47:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:47:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26891.097621 secs 2023-09-10T06:47:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:47:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:47:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:40.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26891.097621 secs 2023-09-10T06:47:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:47:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:47:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26891.097621 secs 2023-09-10T06:47:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:47:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:47:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:41 smithi170 ceph-mon[102832]: pgmap v14502: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 423 KiB/s wr, 17 op/s 2023-09-10T06:47:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:41 smithi082 ceph-mon[98694]: pgmap v14502: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 423 KiB/s wr, 17 op/s 2023-09-10T06:47:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:41 smithi098 ceph-mon[103124]: pgmap v14502: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 423 KiB/s wr, 17 op/s 2023-09-10T06:47:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:43 smithi170 ceph-mon[102832]: pgmap v14503: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 244 KiB/s wr, 13 op/s 2023-09-10T06:47:43.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:43 smithi082 ceph-mon[98694]: pgmap v14503: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 244 KiB/s wr, 13 op/s 2023-09-10T06:47:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:43 smithi098 ceph-mon[103124]: pgmap v14503: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 244 KiB/s wr, 13 op/s 2023-09-10T06:47:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:47:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26895.619216 secs 2023-09-10T06:47:44.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:47:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26895.619216 secs 2023-09-10T06:47:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:47:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26895.619216 secs 2023-09-10T06:47:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26896.097709 secs 2023-09-10T06:47:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:45 smithi170 ceph-mon[102832]: pgmap v14504: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 244 KiB/s wr, 12 op/s 2023-09-10T06:47:45.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26896.097709 secs 2023-09-10T06:47:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:45 smithi082 ceph-mon[98694]: pgmap v14504: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 244 KiB/s wr, 12 op/s 2023-09-10T06:47:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26896.097709 secs 2023-09-10T06:47:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:45 smithi098 ceph-mon[103124]: pgmap v14504: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 244 KiB/s wr, 12 op/s 2023-09-10T06:47:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:47:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:47:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:47:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:47 smithi170 ceph-mon[102832]: pgmap v14505: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 744 KiB/s wr, 20 op/s 2023-09-10T06:47:47.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:47 smithi082 ceph-mon[98694]: pgmap v14505: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 744 KiB/s wr, 20 op/s 2023-09-10T06:47:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:47 smithi098 ceph-mon[103124]: pgmap v14505: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 744 KiB/s wr, 20 op/s 2023-09-10T06:47:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:49 smithi170 ceph-mon[102832]: pgmap v14506: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 518 KiB/s wr, 13 op/s 2023-09-10T06:47:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26900.619274 secs 2023-09-10T06:47:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:49 smithi082 ceph-mon[98694]: pgmap v14506: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 518 KiB/s wr, 13 op/s 2023-09-10T06:47:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26900.619274 secs 2023-09-10T06:47:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:49 smithi098 ceph-mon[103124]: pgmap v14506: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 518 KiB/s wr, 13 op/s 2023-09-10T06:47:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26900.619274 secs 2023-09-10T06:47:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26901.097773 secs 2023-09-10T06:47:50.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26901.097773 secs 2023-09-10T06:47:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26901.097773 secs 2023-09-10T06:47:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:51 smithi170 ceph-mon[102832]: pgmap v14507: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 643 KiB/s wr, 18 op/s 2023-09-10T06:47:51.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:51 smithi082 ceph-mon[98694]: pgmap v14507: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 643 KiB/s wr, 18 op/s 2023-09-10T06:47:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:51 smithi098 ceph-mon[103124]: pgmap v14507: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 643 KiB/s wr, 18 op/s 2023-09-10T06:47:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:53 smithi170 ceph-mon[102832]: pgmap v14508: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 625 KiB/s wr, 14 op/s 2023-09-10T06:47:53.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:53 smithi082 ceph-mon[98694]: pgmap v14508: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 625 KiB/s wr, 14 op/s 2023-09-10T06:47:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:53 smithi098 ceph-mon[103124]: pgmap v14508: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 625 KiB/s wr, 14 op/s 2023-09-10T06:47:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26905.619360 secs 2023-09-10T06:47:54.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26905.619360 secs 2023-09-10T06:47:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26905.619360 secs 2023-09-10T06:47:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26906.097835 secs 2023-09-10T06:47:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:55 smithi170 ceph-mon[102832]: pgmap v14509: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 625 KiB/s wr, 13 op/s 2023-09-10T06:47:55.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26906.097835 secs 2023-09-10T06:47:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:55 smithi082 ceph-mon[98694]: pgmap v14509: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 625 KiB/s wr, 13 op/s 2023-09-10T06:47:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26906.097835 secs 2023-09-10T06:47:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:55 smithi098 ceph-mon[103124]: pgmap v14509: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 625 KiB/s wr, 13 op/s 2023-09-10T06:47:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:57 smithi170 ceph-mon[102832]: pgmap v14510: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 720 KiB/s wr, 19 op/s 2023-09-10T06:47:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:57 smithi082 ceph-mon[98694]: pgmap v14510: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 720 KiB/s wr, 19 op/s 2023-09-10T06:47:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:57 smithi098 ceph-mon[103124]: pgmap v14510: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 720 KiB/s wr, 19 op/s 2023-09-10T06:47:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:47:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:47:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:47:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:47:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26909.479458 secs 2023-09-10T06:47:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:59 smithi170 ceph-mon[102832]: pgmap v14511: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 220 KiB/s wr, 11 op/s 2023-09-10T06:47:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:47:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26910.619419 secs 2023-09-10T06:47:59.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26909.479458 secs 2023-09-10T06:47:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:59 smithi082 ceph-mon[98694]: pgmap v14511: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 220 KiB/s wr, 11 op/s 2023-09-10T06:47:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:47:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26910.619419 secs 2023-09-10T06:47:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26909.479458 secs 2023-09-10T06:47:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:59 smithi098 ceph-mon[103124]: pgmap v14511: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 220 KiB/s wr, 11 op/s 2023-09-10T06:47:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:47:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26910.619419 secs 2023-09-10T06:48:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26911.097899 secs 2023-09-10T06:48:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:48:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:48:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:00.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26911.097899 secs 2023-09-10T06:48:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:48:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:48:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26911.097899 secs 2023-09-10T06:48:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:48:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:48:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:01 smithi170 ceph-mon[102832]: pgmap v14512: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 418 KiB/s wr, 16 op/s 2023-09-10T06:48:01.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:01 smithi082 ceph-mon[98694]: pgmap v14512: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 418 KiB/s wr, 16 op/s 2023-09-10T06:48:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:01 smithi098 ceph-mon[103124]: pgmap v14512: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 418 KiB/s wr, 16 op/s 2023-09-10T06:48:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:03 smithi170 ceph-mon[102832]: pgmap v14513: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T06:48:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26914.479516 secs 2023-09-10T06:48:03.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:03 smithi082 ceph-mon[98694]: pgmap v14513: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T06:48:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26914.479516 secs 2023-09-10T06:48:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:03 smithi098 ceph-mon[103124]: pgmap v14513: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T06:48:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26914.479516 secs 2023-09-10T06:48:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26915.619517 secs 2023-09-10T06:48:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26915.619517 secs 2023-09-10T06:48:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26915.619517 secs 2023-09-10T06:48:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26916.097961 secs 2023-09-10T06:48:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:05 smithi170 ceph-mon[102832]: pgmap v14514: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T06:48:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26916.097961 secs 2023-09-10T06:48:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:05 smithi082 ceph-mon[98694]: pgmap v14514: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T06:48:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26916.097961 secs 2023-09-10T06:48:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:05 smithi098 ceph-mon[103124]: pgmap v14514: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T06:48:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:07 smithi170 ceph-mon[102832]: pgmap v14515: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 431 KiB/s wr, 31 op/s 2023-09-10T06:48:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:07 smithi082 ceph-mon[98694]: pgmap v14515: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 431 KiB/s wr, 31 op/s 2023-09-10T06:48:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:07 smithi098 ceph-mon[103124]: pgmap v14515: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 431 KiB/s wr, 31 op/s 2023-09-10T06:48:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26919.479579 secs 2023-09-10T06:48:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:09 smithi170 ceph-mon[102832]: osdmap e975: 12 total, 12 up, 12 in 2023-09-10T06:48:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:09 smithi170 ceph-mon[102832]: pgmap v14517: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 403 KiB/s wr, 31 op/s 2023-09-10T06:48:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26920.619577 secs 2023-09-10T06:48:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26919.479579 secs 2023-09-10T06:48:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:09 smithi082 ceph-mon[98694]: osdmap e975: 12 total, 12 up, 12 in 2023-09-10T06:48:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:09 smithi082 ceph-mon[98694]: pgmap v14517: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 403 KiB/s wr, 31 op/s 2023-09-10T06:48:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26920.619577 secs 2023-09-10T06:48:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26919.479579 secs 2023-09-10T06:48:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:09 smithi098 ceph-mon[103124]: osdmap e975: 12 total, 12 up, 12 in 2023-09-10T06:48:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:09 smithi098 ceph-mon[103124]: pgmap v14517: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 403 KiB/s wr, 31 op/s 2023-09-10T06:48:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26920.619577 secs 2023-09-10T06:48:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26921.098028 secs 2023-09-10T06:48:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26921.098028 secs 2023-09-10T06:48:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26921.098028 secs 2023-09-10T06:48:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:11 smithi170 ceph-mon[102832]: pgmap v14518: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 190 KiB/s wr, 30 op/s 2023-09-10T06:48:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:11 smithi082 ceph-mon[98694]: pgmap v14518: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 190 KiB/s wr, 30 op/s 2023-09-10T06:48:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:11 smithi098 ceph-mon[103124]: pgmap v14518: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 190 KiB/s wr, 30 op/s 2023-09-10T06:48:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:13 smithi170 ceph-mon[102832]: pgmap v14519: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 190 KiB/s wr, 30 op/s 2023-09-10T06:48:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26924.479639 secs 2023-09-10T06:48:13.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:13 smithi082 ceph-mon[98694]: pgmap v14519: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 190 KiB/s wr, 30 op/s 2023-09-10T06:48:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26924.479639 secs 2023-09-10T06:48:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:13 smithi098 ceph-mon[103124]: pgmap v14519: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 190 KiB/s wr, 30 op/s 2023-09-10T06:48:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26924.479639 secs 2023-09-10T06:48:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26925.619636 secs 2023-09-10T06:48:14.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26925.619636 secs 2023-09-10T06:48:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26925.619636 secs 2023-09-10T06:48:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26926.098092 secs 2023-09-10T06:48:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:15 smithi170 ceph-mon[102832]: pgmap v14520: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 190 KiB/s wr, 30 op/s 2023-09-10T06:48:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26926.098092 secs 2023-09-10T06:48:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:15 smithi082 ceph-mon[98694]: pgmap v14520: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 190 KiB/s wr, 30 op/s 2023-09-10T06:48:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26926.098092 secs 2023-09-10T06:48:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:15 smithi098 ceph-mon[103124]: pgmap v14520: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 190 KiB/s wr, 30 op/s 2023-09-10T06:48:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:17 smithi170 ceph-mon[102832]: osdmap e976: 12 total, 12 up, 12 in 2023-09-10T06:48:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:17 smithi170 ceph-mon[102832]: pgmap v14522: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 167 KiB/s wr, 15 op/s 2023-09-10T06:48:17.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:17 smithi082 ceph-mon[98694]: osdmap e976: 12 total, 12 up, 12 in 2023-09-10T06:48:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:17 smithi082 ceph-mon[98694]: pgmap v14522: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 167 KiB/s wr, 15 op/s 2023-09-10T06:48:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:17 smithi098 ceph-mon[103124]: osdmap e976: 12 total, 12 up, 12 in 2023-09-10T06:48:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:17 smithi098 ceph-mon[103124]: pgmap v14522: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 167 KiB/s wr, 15 op/s 2023-09-10T06:48:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26929.479698 secs 2023-09-10T06:48:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26929.479698 secs 2023-09-10T06:48:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26929.479698 secs 2023-09-10T06:48:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:19 smithi170 ceph-mon[102832]: pgmap v14523: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 12 op/s 2023-09-10T06:48:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26930.619724 secs 2023-09-10T06:48:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:48:19.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:19 smithi082 ceph-mon[98694]: pgmap v14523: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 12 op/s 2023-09-10T06:48:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26930.619724 secs 2023-09-10T06:48:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:48:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:19 smithi098 ceph-mon[103124]: pgmap v14523: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 12 op/s 2023-09-10T06:48:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26930.619724 secs 2023-09-10T06:48:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:48:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26931.098159 secs 2023-09-10T06:48:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:48:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:48:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:48:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26931.098159 secs 2023-09-10T06:48:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:48:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:48:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:48:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26931.098159 secs 2023-09-10T06:48:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:48:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:48:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:48:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:21 smithi170 ceph-mon[102832]: pgmap v14524: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T06:48:21.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:21 smithi082 ceph-mon[98694]: pgmap v14524: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T06:48:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:21 smithi098 ceph-mon[103124]: pgmap v14524: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T06:48:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:23 smithi170 ceph-mon[102832]: pgmap v14525: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T06:48:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26934.479758 secs 2023-09-10T06:48:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:23 smithi082 ceph-mon[98694]: pgmap v14525: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T06:48:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26934.479758 secs 2023-09-10T06:48:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:23 smithi098 ceph-mon[103124]: pgmap v14525: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T06:48:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26934.479758 secs 2023-09-10T06:48:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26935.619782 secs 2023-09-10T06:48:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:48:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26936.098224 secs 2023-09-10T06:48:24.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26935.619782 secs 2023-09-10T06:48:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:48:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26936.098224 secs 2023-09-10T06:48:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26935.619782 secs 2023-09-10T06:48:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:48:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26936.098224 secs 2023-09-10T06:48:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:25 smithi170 ceph-mon[102832]: pgmap v14526: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T06:48:25.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:25 smithi082 ceph-mon[98694]: pgmap v14526: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T06:48:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:25 smithi098 ceph-mon[103124]: pgmap v14526: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T06:48:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:26 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:48:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:26 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:48:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:26 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:48:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:27 smithi170 ceph-mon[102832]: pgmap v14527: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 130 KiB/s wr, 11 op/s 2023-09-10T06:48:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:27 smithi082 ceph-mon[98694]: pgmap v14527: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 130 KiB/s wr, 11 op/s 2023-09-10T06:48:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:27 smithi098 ceph-mon[103124]: pgmap v14527: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 130 KiB/s wr, 11 op/s 2023-09-10T06:48:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:29 smithi170 ceph-mon[102832]: pgmap v14528: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 10 op/s 2023-09-10T06:48:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26940.619897 secs 2023-09-10T06:48:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26941.098289 secs 2023-09-10T06:48:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:29 smithi082 ceph-mon[98694]: pgmap v14528: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 10 op/s 2023-09-10T06:48:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26940.619897 secs 2023-09-10T06:48:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26941.098289 secs 2023-09-10T06:48:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:29 smithi098 ceph-mon[103124]: pgmap v14528: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 10 op/s 2023-09-10T06:48:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26940.619897 secs 2023-09-10T06:48:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26941.098289 secs 2023-09-10T06:48:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:31 smithi170 ceph-mon[102832]: pgmap v14529: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 228 KiB/s wr, 14 op/s 2023-09-10T06:48:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:48:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:31 smithi082 ceph-mon[98694]: pgmap v14529: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 228 KiB/s wr, 14 op/s 2023-09-10T06:48:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:48:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:31 smithi098 ceph-mon[103124]: pgmap v14529: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 228 KiB/s wr, 14 op/s 2023-09-10T06:48:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:48:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:48:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:48:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:48:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:48:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:48:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:48:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:33 smithi170 ceph-mon[102832]: pgmap v14530: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 228 KiB/s wr, 10 op/s 2023-09-10T06:48:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26944.479854 secs 2023-09-10T06:48:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:33 smithi082 ceph-mon[98694]: pgmap v14530: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 228 KiB/s wr, 10 op/s 2023-09-10T06:48:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26944.479854 secs 2023-09-10T06:48:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:33 smithi098 ceph-mon[103124]: pgmap v14530: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 228 KiB/s wr, 10 op/s 2023-09-10T06:48:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26944.479854 secs 2023-09-10T06:48:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26945.619955 secs 2023-09-10T06:48:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26946.098353 secs 2023-09-10T06:48:34.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26945.619955 secs 2023-09-10T06:48:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26946.098353 secs 2023-09-10T06:48:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26945.619955 secs 2023-09-10T06:48:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26946.098353 secs 2023-09-10T06:48:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:35 smithi170 ceph-mon[102832]: pgmap v14531: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 228 KiB/s wr, 9 op/s 2023-09-10T06:48:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:35 smithi082 ceph-mon[98694]: pgmap v14531: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 228 KiB/s wr, 9 op/s 2023-09-10T06:48:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:35 smithi098 ceph-mon[103124]: pgmap v14531: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 228 KiB/s wr, 9 op/s 2023-09-10T06:48:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:48:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:48:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:48:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:37 smithi170 ceph-mon[102832]: pgmap v14532: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 338 KiB/s wr, 15 op/s 2023-09-10T06:48:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:37 smithi082 ceph-mon[98694]: pgmap v14532: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 338 KiB/s wr, 15 op/s 2023-09-10T06:48:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:37 smithi098 ceph-mon[103124]: pgmap v14532: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 338 KiB/s wr, 15 op/s 2023-09-10T06:48:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:48:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:48:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:48:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:39 smithi170 ceph-mon[102832]: pgmap v14533: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 228 KiB/s wr, 10 op/s 2023-09-10T06:48:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26950.620014 secs 2023-09-10T06:48:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26951.098416 secs 2023-09-10T06:48:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:48:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:39 smithi082 ceph-mon[98694]: pgmap v14533: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 228 KiB/s wr, 10 op/s 2023-09-10T06:48:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26950.620014 secs 2023-09-10T06:48:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26951.098416 secs 2023-09-10T06:48:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:48:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:39 smithi098 ceph-mon[103124]: pgmap v14533: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 228 KiB/s wr, 10 op/s 2023-09-10T06:48:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26950.620014 secs 2023-09-10T06:48:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26951.098416 secs 2023-09-10T06:48:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:48:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:48:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:48:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:40.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:48:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:48:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:48:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:48:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:41 smithi170 ceph-mon[102832]: pgmap v14534: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 580 KiB/s wr, 16 op/s 2023-09-10T06:48:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:41 smithi082 ceph-mon[98694]: pgmap v14534: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 580 KiB/s wr, 16 op/s 2023-09-10T06:48:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:41 smithi098 ceph-mon[103124]: pgmap v14534: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 580 KiB/s wr, 16 op/s 2023-09-10T06:48:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:43 smithi170 ceph-mon[102832]: pgmap v14535: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 462 KiB/s wr, 12 op/s 2023-09-10T06:48:43.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:43 smithi082 ceph-mon[98694]: pgmap v14535: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 462 KiB/s wr, 12 op/s 2023-09-10T06:48:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:43 smithi098 ceph-mon[103124]: pgmap v14535: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 462 KiB/s wr, 12 op/s 2023-09-10T06:48:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26955.620072 secs 2023-09-10T06:48:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:48:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26956.098484 secs 2023-09-10T06:48:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26955.620072 secs 2023-09-10T06:48:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:48:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26956.098484 secs 2023-09-10T06:48:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26955.620072 secs 2023-09-10T06:48:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:48:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26956.098484 secs 2023-09-10T06:48:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:45 smithi170 ceph-mon[102832]: pgmap v14536: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 462 KiB/s wr, 12 op/s 2023-09-10T06:48:45.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:45 smithi082 ceph-mon[98694]: pgmap v14536: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 462 KiB/s wr, 12 op/s 2023-09-10T06:48:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:45 smithi098 ceph-mon[103124]: pgmap v14536: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 462 KiB/s wr, 12 op/s 2023-09-10T06:48:46.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:48:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:48:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:48:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:47 smithi082 ceph-mon[98694]: pgmap v14537: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 753 KiB/s wr, 18 op/s 2023-09-10T06:48:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:47 smithi098 ceph-mon[103124]: pgmap v14537: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 753 KiB/s wr, 18 op/s 2023-09-10T06:48:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:47 smithi170 ceph-mon[102832]: pgmap v14537: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 753 KiB/s wr, 18 op/s 2023-09-10T06:48:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:49 smithi082 ceph-mon[98694]: pgmap v14538: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 643 KiB/s wr, 12 op/s 2023-09-10T06:48:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26960.620156 secs 2023-09-10T06:48:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26961.098551 secs 2023-09-10T06:48:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:49 smithi098 ceph-mon[103124]: pgmap v14538: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 643 KiB/s wr, 12 op/s 2023-09-10T06:48:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26960.620156 secs 2023-09-10T06:48:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26961.098551 secs 2023-09-10T06:48:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:49 smithi170 ceph-mon[102832]: pgmap v14538: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 643 KiB/s wr, 12 op/s 2023-09-10T06:48:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26960.620156 secs 2023-09-10T06:48:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26961.098551 secs 2023-09-10T06:48:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:51 smithi082 ceph-mon[98694]: pgmap v14539: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 735 KiB/s wr, 19 op/s 2023-09-10T06:48:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:51 smithi098 ceph-mon[103124]: pgmap v14539: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 735 KiB/s wr, 19 op/s 2023-09-10T06:48:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:51 smithi170 ceph-mon[102832]: pgmap v14539: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 735 KiB/s wr, 19 op/s 2023-09-10T06:48:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:53 smithi082 ceph-mon[98694]: pgmap v14540: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 383 KiB/s wr, 13 op/s 2023-09-10T06:48:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26964.480117 secs 2023-09-10T06:48:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:53 smithi098 ceph-mon[103124]: pgmap v14540: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 383 KiB/s wr, 13 op/s 2023-09-10T06:48:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26964.480117 secs 2023-09-10T06:48:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:53 smithi170 ceph-mon[102832]: pgmap v14540: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 383 KiB/s wr, 13 op/s 2023-09-10T06:48:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26964.480117 secs 2023-09-10T06:48:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26965.620237 secs 2023-09-10T06:48:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:48:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26966.098616 secs 2023-09-10T06:48:55.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26965.620237 secs 2023-09-10T06:48:55.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:48:55.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:55.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26966.098616 secs 2023-09-10T06:48:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26965.620237 secs 2023-09-10T06:48:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:48:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26966.098616 secs 2023-09-10T06:48:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:55 smithi082 ceph-mon[98694]: pgmap v14541: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 383 KiB/s wr, 12 op/s 2023-09-10T06:48:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:55 smithi098 ceph-mon[103124]: pgmap v14541: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 383 KiB/s wr, 12 op/s 2023-09-10T06:48:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:55 smithi170 ceph-mon[102832]: pgmap v14541: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 383 KiB/s wr, 12 op/s 2023-09-10T06:48:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:48:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:48:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:48:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:48:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:48:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:48:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:57 smithi098 ceph-mon[103124]: pgmap v14542: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 702 KiB/s wr, 18 op/s 2023-09-10T06:48:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:57 smithi170 ceph-mon[102832]: pgmap v14542: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 702 KiB/s wr, 18 op/s 2023-09-10T06:48:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:57 smithi082 ceph-mon[98694]: pgmap v14542: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 702 KiB/s wr, 18 op/s 2023-09-10T06:48:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:48:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26969.480179 secs 2023-09-10T06:48:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:48:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26969.480179 secs 2023-09-10T06:48:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:48:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:48:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26969.480179 secs 2023-09-10T06:49:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:59 smithi098 ceph-mon[103124]: pgmap v14543: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 410 KiB/s wr, 12 op/s 2023-09-10T06:49:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26970.620320 secs 2023-09-10T06:49:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:48:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26971.098685 secs 2023-09-10T06:49:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:59 smithi170 ceph-mon[102832]: pgmap v14543: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 410 KiB/s wr, 12 op/s 2023-09-10T06:49:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26970.620320 secs 2023-09-10T06:49:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:48:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26971.098685 secs 2023-09-10T06:49:00.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:59 smithi082 ceph-mon[98694]: pgmap v14543: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 410 KiB/s wr, 12 op/s 2023-09-10T06:49:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26970.620320 secs 2023-09-10T06:49:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:48:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26971.098685 secs 2023-09-10T06:49:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:49:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:49:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:49:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:49:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:49:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:49:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:01 smithi098 ceph-mon[103124]: pgmap v14544: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 622 KiB/s wr, 17 op/s 2023-09-10T06:49:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:01 smithi170 ceph-mon[102832]: pgmap v14544: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 622 KiB/s wr, 17 op/s 2023-09-10T06:49:02.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:01 smithi082 ceph-mon[98694]: pgmap v14544: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 622 KiB/s wr, 17 op/s 2023-09-10T06:49:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:03 smithi098 ceph-mon[103124]: pgmap v14545: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 530 KiB/s wr, 11 op/s 2023-09-10T06:49:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26974.480251 secs 2023-09-10T06:49:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:03 smithi170 ceph-mon[102832]: pgmap v14545: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 530 KiB/s wr, 11 op/s 2023-09-10T06:49:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26974.480251 secs 2023-09-10T06:49:04.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:03 smithi082 ceph-mon[98694]: pgmap v14545: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 530 KiB/s wr, 11 op/s 2023-09-10T06:49:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26974.480251 secs 2023-09-10T06:49:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26975.620377 secs 2023-09-10T06:49:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26976.098748 secs 2023-09-10T06:49:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26975.620377 secs 2023-09-10T06:49:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26976.098748 secs 2023-09-10T06:49:05.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26975.620377 secs 2023-09-10T06:49:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26976.098748 secs 2023-09-10T06:49:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:05 smithi098 ceph-mon[103124]: pgmap v14546: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 530 KiB/s wr, 11 op/s 2023-09-10T06:49:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:05 smithi170 ceph-mon[102832]: pgmap v14546: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 530 KiB/s wr, 11 op/s 2023-09-10T06:49:06.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:05 smithi082 ceph-mon[98694]: pgmap v14546: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 530 KiB/s wr, 11 op/s 2023-09-10T06:49:07.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:07 smithi170 ceph-mon[102832]: pgmap v14547: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 668 KiB/s wr, 31 op/s 2023-09-10T06:49:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:07 smithi082 ceph-mon[98694]: pgmap v14547: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 668 KiB/s wr, 31 op/s 2023-09-10T06:49:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:07 smithi098 ceph-mon[103124]: pgmap v14547: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 668 KiB/s wr, 31 op/s 2023-09-10T06:49:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26979.480313 secs 2023-09-10T06:49:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:09 smithi082 ceph-mon[98694]: osdmap e977: 12 total, 12 up, 12 in 2023-09-10T06:49:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:09 smithi082 ceph-mon[98694]: pgmap v14549: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 419 KiB/s wr, 31 op/s 2023-09-10T06:49:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26980.620460 secs 2023-09-10T06:49:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26979.480313 secs 2023-09-10T06:49:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:09 smithi098 ceph-mon[103124]: osdmap e977: 12 total, 12 up, 12 in 2023-09-10T06:49:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:09 smithi098 ceph-mon[103124]: pgmap v14549: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 419 KiB/s wr, 31 op/s 2023-09-10T06:49:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26980.620460 secs 2023-09-10T06:49:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26979.480313 secs 2023-09-10T06:49:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:09 smithi170 ceph-mon[102832]: osdmap e977: 12 total, 12 up, 12 in 2023-09-10T06:49:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:09 smithi170 ceph-mon[102832]: pgmap v14549: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 419 KiB/s wr, 31 op/s 2023-09-10T06:49:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26980.620460 secs 2023-09-10T06:49:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26981.098813 secs 2023-09-10T06:49:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26981.098813 secs 2023-09-10T06:49:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26981.098813 secs 2023-09-10T06:49:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:11 smithi082 ceph-mon[98694]: pgmap v14550: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 190 KiB/s wr, 30 op/s 2023-09-10T06:49:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:11 smithi098 ceph-mon[103124]: pgmap v14550: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 190 KiB/s wr, 30 op/s 2023-09-10T06:49:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:11 smithi170 ceph-mon[102832]: pgmap v14550: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 190 KiB/s wr, 30 op/s 2023-09-10T06:49:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:13 smithi170 ceph-mon[102832]: pgmap v14551: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 190 KiB/s wr, 30 op/s 2023-09-10T06:49:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26984.480374 secs 2023-09-10T06:49:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:13 smithi082 ceph-mon[98694]: pgmap v14551: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 190 KiB/s wr, 30 op/s 2023-09-10T06:49:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26984.480374 secs 2023-09-10T06:49:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:13 smithi098 ceph-mon[103124]: pgmap v14551: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 190 KiB/s wr, 30 op/s 2023-09-10T06:49:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26984.480374 secs 2023-09-10T06:49:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26985.620546 secs 2023-09-10T06:49:14.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26985.620546 secs 2023-09-10T06:49:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26985.620546 secs 2023-09-10T06:49:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26986.098877 secs 2023-09-10T06:49:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:15 smithi170 ceph-mon[102832]: pgmap v14552: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 190 KiB/s wr, 30 op/s 2023-09-10T06:49:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26986.098877 secs 2023-09-10T06:49:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:15 smithi082 ceph-mon[98694]: pgmap v14552: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 190 KiB/s wr, 30 op/s 2023-09-10T06:49:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26986.098877 secs 2023-09-10T06:49:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:15 smithi098 ceph-mon[103124]: pgmap v14552: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 190 KiB/s wr, 30 op/s 2023-09-10T06:49:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:17 smithi170 ceph-mon[102832]: osdmap e978: 12 total, 12 up, 12 in 2023-09-10T06:49:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:17 smithi170 ceph-mon[102832]: pgmap v14554: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-09-10T06:49:17.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:17 smithi082 ceph-mon[98694]: osdmap e978: 12 total, 12 up, 12 in 2023-09-10T06:49:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:17 smithi082 ceph-mon[98694]: pgmap v14554: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-09-10T06:49:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:17 smithi098 ceph-mon[103124]: osdmap e978: 12 total, 12 up, 12 in 2023-09-10T06:49:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:17 smithi098 ceph-mon[103124]: pgmap v14554: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-09-10T06:49:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26989.480434 secs 2023-09-10T06:49:18.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26989.480434 secs 2023-09-10T06:49:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26989.480434 secs 2023-09-10T06:49:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:19 smithi170 ceph-mon[102832]: pgmap v14555: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-10T06:49:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26990.620607 secs 2023-09-10T06:49:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:19 smithi082 ceph-mon[98694]: pgmap v14555: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-10T06:49:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26990.620607 secs 2023-09-10T06:49:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:19 smithi098 ceph-mon[103124]: pgmap v14555: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-10T06:49:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26990.620607 secs 2023-09-10T06:49:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26991.098950 secs 2023-09-10T06:49:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:49:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:49:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:49:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:49:20.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26991.098950 secs 2023-09-10T06:49:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:49:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:49:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:49:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:49:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26991.098950 secs 2023-09-10T06:49:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:49:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:49:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:49:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:49:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:21 smithi170 ceph-mon[102832]: pgmap v14556: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T06:49:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:21 smithi082 ceph-mon[98694]: pgmap v14556: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T06:49:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:21 smithi098 ceph-mon[103124]: pgmap v14556: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T06:49:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:23 smithi170 ceph-mon[102832]: pgmap v14557: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T06:49:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 26994.480499 secs 2023-09-10T06:49:23.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:23 smithi082 ceph-mon[98694]: pgmap v14557: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T06:49:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 26994.480499 secs 2023-09-10T06:49:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:23 smithi098 ceph-mon[103124]: pgmap v14557: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T06:49:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 26994.480499 secs 2023-09-10T06:49:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26995.620666 secs 2023-09-10T06:49:24.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26995.620666 secs 2023-09-10T06:49:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26995.620666 secs 2023-09-10T06:49:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 26996.099007 secs 2023-09-10T06:49:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:25 smithi170 ceph-mon[102832]: pgmap v14558: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T06:49:25.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 26996.099007 secs 2023-09-10T06:49:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:25 smithi082 ceph-mon[98694]: pgmap v14558: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T06:49:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 26996.099007 secs 2023-09-10T06:49:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:25 smithi098 ceph-mon[103124]: pgmap v14558: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T06:49:27.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:27 smithi082 ceph-mon[98694]: pgmap v14559: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T06:49:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:27 smithi098 ceph-mon[103124]: pgmap v14559: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T06:49:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:27 smithi170 ceph-mon[102832]: pgmap v14559: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T06:49:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:29 smithi170 ceph-mon[102832]: pgmap v14560: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 9 op/s 2023-09-10T06:49:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27000.620723 secs 2023-09-10T06:49:29.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:29 smithi082 ceph-mon[98694]: pgmap v14560: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 9 op/s 2023-09-10T06:49:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27000.620723 secs 2023-09-10T06:49:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:29 smithi098 ceph-mon[103124]: pgmap v14560: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 9 op/s 2023-09-10T06:49:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27000.620723 secs 2023-09-10T06:49:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27001.099071 secs 2023-09-10T06:49:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:49:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:49:30.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27001.099071 secs 2023-09-10T06:49:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:49:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:49:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27001.099071 secs 2023-09-10T06:49:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:49:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:49:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:31 smithi170 ceph-mon[102832]: pgmap v14561: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 294 KiB/s wr, 15 op/s 2023-09-10T06:49:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:49:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:31 smithi082 ceph-mon[98694]: pgmap v14561: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 294 KiB/s wr, 15 op/s 2023-09-10T06:49:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:49:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:31 smithi098 ceph-mon[103124]: pgmap v14561: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 294 KiB/s wr, 15 op/s 2023-09-10T06:49:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:49:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:49:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:49:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:49:32.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:49:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:49:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:49:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:49:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:49:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:49:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:33 smithi170 ceph-mon[102832]: pgmap v14562: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 200 KiB/s wr, 10 op/s 2023-09-10T06:49:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:33 smithi082 ceph-mon[98694]: pgmap v14562: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 200 KiB/s wr, 10 op/s 2023-09-10T06:49:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:33 smithi098 ceph-mon[103124]: pgmap v14562: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 200 KiB/s wr, 10 op/s 2023-09-10T06:49:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27005.620783 secs 2023-09-10T06:49:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27005.620783 secs 2023-09-10T06:49:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27005.620783 secs 2023-09-10T06:49:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27006.099133 secs 2023-09-10T06:49:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:35 smithi170 ceph-mon[102832]: pgmap v14563: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 200 KiB/s wr, 10 op/s 2023-09-10T06:49:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27006.099133 secs 2023-09-10T06:49:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:35 smithi082 ceph-mon[98694]: pgmap v14563: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 200 KiB/s wr, 10 op/s 2023-09-10T06:49:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27006.099133 secs 2023-09-10T06:49:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:35 smithi098 ceph-mon[103124]: pgmap v14563: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 200 KiB/s wr, 10 op/s 2023-09-10T06:49:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:37 smithi082 ceph-mon[98694]: pgmap v14564: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 312 KiB/s wr, 16 op/s 2023-09-10T06:49:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:37 smithi098 ceph-mon[103124]: pgmap v14564: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 312 KiB/s wr, 16 op/s 2023-09-10T06:49:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:37 smithi170 ceph-mon[102832]: pgmap v14564: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 312 KiB/s wr, 16 op/s 2023-09-10T06:49:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27009.480699 secs 2023-09-10T06:49:38.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27009.480699 secs 2023-09-10T06:49:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27009.480699 secs 2023-09-10T06:49:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:39 smithi170 ceph-mon[102832]: pgmap v14565: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 311 KiB/s wr, 11 op/s 2023-09-10T06:49:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27010.620840 secs 2023-09-10T06:49:39.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:39 smithi082 ceph-mon[98694]: pgmap v14565: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 311 KiB/s wr, 11 op/s 2023-09-10T06:49:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27010.620840 secs 2023-09-10T06:49:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:39 smithi098 ceph-mon[103124]: pgmap v14565: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 311 KiB/s wr, 11 op/s 2023-09-10T06:49:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27010.620840 secs 2023-09-10T06:49:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27011.099200 secs 2023-09-10T06:49:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:49:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:49:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:49:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27011.099200 secs 2023-09-10T06:49:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:49:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:49:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:49:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27011.099200 secs 2023-09-10T06:49:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:49:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:49:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:49:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:41 smithi170 ceph-mon[102832]: pgmap v14566: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 392 KiB/s wr, 17 op/s 2023-09-10T06:49:41.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:41 smithi082 ceph-mon[98694]: pgmap v14566: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 392 KiB/s wr, 17 op/s 2023-09-10T06:49:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:41 smithi098 ceph-mon[103124]: pgmap v14566: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 392 KiB/s wr, 17 op/s 2023-09-10T06:49:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:43 smithi170 ceph-mon[102832]: pgmap v14567: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 192 KiB/s wr, 12 op/s 2023-09-10T06:49:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:49:43.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:43 smithi082 ceph-mon[98694]: pgmap v14567: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 192 KiB/s wr, 12 op/s 2023-09-10T06:49:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:49:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:43 smithi098 ceph-mon[103124]: pgmap v14567: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 192 KiB/s wr, 12 op/s 2023-09-10T06:49:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:49:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27015.620924 secs 2023-09-10T06:49:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:49:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:49:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27015.620924 secs 2023-09-10T06:49:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:49:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:49:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27015.620924 secs 2023-09-10T06:49:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:49:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:49:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27016.099265 secs 2023-09-10T06:49:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:45 smithi170 ceph-mon[102832]: pgmap v14568: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 192 KiB/s wr, 12 op/s 2023-09-10T06:49:45.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27016.099265 secs 2023-09-10T06:49:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:45 smithi082 ceph-mon[98694]: pgmap v14568: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 192 KiB/s wr, 12 op/s 2023-09-10T06:49:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27016.099265 secs 2023-09-10T06:49:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:45 smithi098 ceph-mon[103124]: pgmap v14568: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 192 KiB/s wr, 12 op/s 2023-09-10T06:49:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:49:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:49:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:46 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:49:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:49:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:49:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:46 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:49:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:49:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:49:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:46 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:49:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:47 smithi170 ceph-mon[102832]: pgmap v14569: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 555 KiB/s wr, 19 op/s 2023-09-10T06:49:47.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:47 smithi082 ceph-mon[98694]: pgmap v14569: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 555 KiB/s wr, 19 op/s 2023-09-10T06:49:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:47 smithi098 ceph-mon[103124]: pgmap v14569: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 555 KiB/s wr, 19 op/s 2023-09-10T06:49:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:49 smithi170 ceph-mon[102832]: pgmap v14570: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 444 KiB/s wr, 12 op/s 2023-09-10T06:49:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27020.621009 secs 2023-09-10T06:49:49.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:49 smithi082 ceph-mon[98694]: pgmap v14570: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 444 KiB/s wr, 12 op/s 2023-09-10T06:49:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27020.621009 secs 2023-09-10T06:49:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:49 smithi098 ceph-mon[103124]: pgmap v14570: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 444 KiB/s wr, 12 op/s 2023-09-10T06:49:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27020.621009 secs 2023-09-10T06:49:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27021.099330 secs 2023-09-10T06:49:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27021.099330 secs 2023-09-10T06:49:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27021.099330 secs 2023-09-10T06:49:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:51 smithi082 ceph-mon[98694]: pgmap v14571: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 445 KiB/s wr, 18 op/s 2023-09-10T06:49:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:51 smithi098 ceph-mon[103124]: pgmap v14571: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 445 KiB/s wr, 18 op/s 2023-09-10T06:49:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:51 smithi170 ceph-mon[102832]: pgmap v14571: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 445 KiB/s wr, 18 op/s 2023-09-10T06:49:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:53 smithi082 ceph-mon[98694]: pgmap v14572: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 364 KiB/s wr, 12 op/s 2023-09-10T06:49:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:53 smithi098 ceph-mon[103124]: pgmap v14572: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 364 KiB/s wr, 12 op/s 2023-09-10T06:49:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:53 smithi170 ceph-mon[102832]: pgmap v14572: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 364 KiB/s wr, 12 op/s 2023-09-10T06:49:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27025.621070 secs 2023-09-10T06:49:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:49:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:49:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27025.621070 secs 2023-09-10T06:49:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:49:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:49:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27025.621070 secs 2023-09-10T06:49:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:49:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:49:55.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27026.099393 secs 2023-09-10T06:49:55.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:55 smithi082 ceph-mon[98694]: pgmap v14573: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 364 KiB/s wr, 12 op/s 2023-09-10T06:49:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27026.099393 secs 2023-09-10T06:49:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:55 smithi098 ceph-mon[103124]: pgmap v14573: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 364 KiB/s wr, 12 op/s 2023-09-10T06:49:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27026.099393 secs 2023-09-10T06:49:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:55 smithi170 ceph-mon[102832]: pgmap v14573: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 364 KiB/s wr, 12 op/s 2023-09-10T06:49:56.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:49:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:49:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:49:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:49:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:49:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:49:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:57 smithi082 ceph-mon[98694]: pgmap v14574: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 712 KiB/s wr, 19 op/s 2023-09-10T06:49:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:57 smithi098 ceph-mon[103124]: pgmap v14574: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 712 KiB/s wr, 19 op/s 2023-09-10T06:49:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:57 smithi170 ceph-mon[102832]: pgmap v14574: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 712 KiB/s wr, 19 op/s 2023-09-10T06:49:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27029.481011 secs 2023-09-10T06:49:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27029.481011 secs 2023-09-10T06:49:58.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27029.481011 secs 2023-09-10T06:49:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:59 smithi098 ceph-mon[103124]: pgmap v14575: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 349 KiB/s wr, 12 op/s 2023-09-10T06:49:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:49:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27030.621130 secs 2023-09-10T06:49:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:59 smithi170 ceph-mon[102832]: pgmap v14575: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 349 KiB/s wr, 12 op/s 2023-09-10T06:49:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:49:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27030.621130 secs 2023-09-10T06:49:59.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:59 smithi082 ceph-mon[98694]: pgmap v14575: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 349 KiB/s wr, 12 op/s 2023-09-10T06:49:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:49:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27030.621130 secs 2023-09-10T06:50:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27031.099460 secs 2023-09-10T06:50:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:50:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:50:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T06:50:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27031.099460 secs 2023-09-10T06:50:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:50:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:50:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T06:50:00.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27031.099460 secs 2023-09-10T06:50:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:50:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:50:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T06:50:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:01 smithi098 ceph-mon[103124]: pgmap v14576: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 451 KiB/s wr, 18 op/s 2023-09-10T06:50:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:01 smithi170 ceph-mon[102832]: pgmap v14576: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 451 KiB/s wr, 18 op/s 2023-09-10T06:50:01.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:01 smithi082 ceph-mon[98694]: pgmap v14576: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 451 KiB/s wr, 18 op/s 2023-09-10T06:50:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:03 smithi098 ceph-mon[103124]: pgmap v14577: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 450 KiB/s wr, 12 op/s 2023-09-10T06:50:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27034.481070 secs 2023-09-10T06:50:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:03 smithi170 ceph-mon[102832]: pgmap v14577: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 450 KiB/s wr, 12 op/s 2023-09-10T06:50:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27034.481070 secs 2023-09-10T06:50:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:03 smithi082 ceph-mon[98694]: pgmap v14577: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 450 KiB/s wr, 12 op/s 2023-09-10T06:50:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27034.481070 secs 2023-09-10T06:50:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27035.621187 secs 2023-09-10T06:50:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:50:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27035.621187 secs 2023-09-10T06:50:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:50:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:04.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27035.621187 secs 2023-09-10T06:50:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:50:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27036.099486 secs 2023-09-10T06:50:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:05 smithi098 ceph-mon[103124]: pgmap v14578: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 450 KiB/s wr, 12 op/s 2023-09-10T06:50:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27036.099486 secs 2023-09-10T06:50:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:05 smithi170 ceph-mon[102832]: pgmap v14578: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 450 KiB/s wr, 12 op/s 2023-09-10T06:50:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27036.099486 secs 2023-09-10T06:50:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:05 smithi082 ceph-mon[98694]: pgmap v14578: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 450 KiB/s wr, 12 op/s 2023-09-10T06:50:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:50:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:50:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:50:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:07 smithi098 ceph-mon[103124]: pgmap v14579: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 688 KiB/s wr, 33 op/s 2023-09-10T06:50:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:07 smithi170 ceph-mon[102832]: pgmap v14579: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 688 KiB/s wr, 33 op/s 2023-09-10T06:50:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:07 smithi082 ceph-mon[98694]: pgmap v14579: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 688 KiB/s wr, 33 op/s 2023-09-10T06:50:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27039.481131 secs 2023-09-10T06:50:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27039.481131 secs 2023-09-10T06:50:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27039.481131 secs 2023-09-10T06:50:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:09 smithi098 ceph-mon[103124]: pgmap v14580: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 340 KiB/s wr, 26 op/s 2023-09-10T06:50:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27040.621248 secs 2023-09-10T06:50:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:09 smithi098 ceph-mon[103124]: osdmap e979: 12 total, 12 up, 12 in 2023-09-10T06:50:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:09 smithi170 ceph-mon[102832]: pgmap v14580: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 340 KiB/s wr, 26 op/s 2023-09-10T06:50:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27040.621248 secs 2023-09-10T06:50:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:09 smithi170 ceph-mon[102832]: osdmap e979: 12 total, 12 up, 12 in 2023-09-10T06:50:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:09 smithi082 ceph-mon[98694]: pgmap v14580: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 340 KiB/s wr, 26 op/s 2023-09-10T06:50:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27040.621248 secs 2023-09-10T06:50:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:09 smithi082 ceph-mon[98694]: osdmap e979: 12 total, 12 up, 12 in 2023-09-10T06:50:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27041.099552 secs 2023-09-10T06:50:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27041.099552 secs 2023-09-10T06:50:10.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27041.099552 secs 2023-09-10T06:50:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:11 smithi170 ceph-mon[102832]: pgmap v14582: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 312 KiB/s wr, 31 op/s 2023-09-10T06:50:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:11 smithi082 ceph-mon[98694]: pgmap v14582: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 312 KiB/s wr, 31 op/s 2023-09-10T06:50:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:11 smithi098 ceph-mon[103124]: pgmap v14582: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 312 KiB/s wr, 31 op/s 2023-09-10T06:50:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:13 smithi170 ceph-mon[102832]: pgmap v14583: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 312 KiB/s wr, 31 op/s 2023-09-10T06:50:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27044.481217 secs 2023-09-10T06:50:13.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:13 smithi082 ceph-mon[98694]: pgmap v14583: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 312 KiB/s wr, 31 op/s 2023-09-10T06:50:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27044.481217 secs 2023-09-10T06:50:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:13 smithi098 ceph-mon[103124]: pgmap v14583: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 312 KiB/s wr, 31 op/s 2023-09-10T06:50:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27044.481217 secs 2023-09-10T06:50:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27045.621324 secs 2023-09-10T06:50:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27045.621324 secs 2023-09-10T06:50:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27045.621324 secs 2023-09-10T06:50:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27046.099614 secs 2023-09-10T06:50:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:15 smithi170 ceph-mon[102832]: pgmap v14584: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 312 KiB/s wr, 31 op/s 2023-09-10T06:50:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27046.099614 secs 2023-09-10T06:50:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:15 smithi082 ceph-mon[98694]: pgmap v14584: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 312 KiB/s wr, 31 op/s 2023-09-10T06:50:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27046.099614 secs 2023-09-10T06:50:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:15 smithi098 ceph-mon[103124]: pgmap v14584: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 312 KiB/s wr, 31 op/s 2023-09-10T06:50:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:17 smithi082 ceph-mon[98694]: osdmap e980: 12 total, 12 up, 12 in 2023-09-10T06:50:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:17 smithi082 ceph-mon[98694]: pgmap v14586: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 15 op/s 2023-09-10T06:50:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:17 smithi098 ceph-mon[103124]: osdmap e980: 12 total, 12 up, 12 in 2023-09-10T06:50:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:17 smithi098 ceph-mon[103124]: pgmap v14586: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 15 op/s 2023-09-10T06:50:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:17 smithi170 ceph-mon[102832]: osdmap e980: 12 total, 12 up, 12 in 2023-09-10T06:50:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:17 smithi170 ceph-mon[102832]: pgmap v14586: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 15 op/s 2023-09-10T06:50:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27049.481279 secs 2023-09-10T06:50:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27049.481279 secs 2023-09-10T06:50:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27049.481279 secs 2023-09-10T06:50:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:19 smithi170 ceph-mon[102832]: pgmap v14587: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 13 op/s 2023-09-10T06:50:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27050.621382 secs 2023-09-10T06:50:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:19 smithi082 ceph-mon[98694]: pgmap v14587: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 13 op/s 2023-09-10T06:50:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27050.621382 secs 2023-09-10T06:50:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:19 smithi098 ceph-mon[103124]: pgmap v14587: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 13 op/s 2023-09-10T06:50:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27050.621382 secs 2023-09-10T06:50:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27051.099681 secs 2023-09-10T06:50:20.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27051.099681 secs 2023-09-10T06:50:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27051.099681 secs 2023-09-10T06:50:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:21 smithi170 ceph-mon[102832]: pgmap v14588: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T06:50:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:50:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:50:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:50:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:50:21.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:21 smithi082 ceph-mon[98694]: pgmap v14588: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T06:50:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:50:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:50:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:50:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:50:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:21 smithi098 ceph-mon[103124]: pgmap v14588: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T06:50:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:50:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:50:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:50:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:50:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:23 smithi170 ceph-mon[102832]: pgmap v14589: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T06:50:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27054.481357 secs 2023-09-10T06:50:23.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:23 smithi082 ceph-mon[98694]: pgmap v14589: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T06:50:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27054.481357 secs 2023-09-10T06:50:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:23 smithi098 ceph-mon[103124]: pgmap v14589: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T06:50:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27054.481357 secs 2023-09-10T06:50:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27055.621502 secs 2023-09-10T06:50:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27055.621502 secs 2023-09-10T06:50:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27055.621502 secs 2023-09-10T06:50:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27056.099742 secs 2023-09-10T06:50:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:25 smithi170 ceph-mon[102832]: pgmap v14590: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T06:50:25.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27056.099742 secs 2023-09-10T06:50:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:25 smithi082 ceph-mon[98694]: pgmap v14590: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T06:50:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27056.099742 secs 2023-09-10T06:50:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:25 smithi098 ceph-mon[103124]: pgmap v14590: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T06:50:27.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:27 smithi082 ceph-mon[98694]: pgmap v14591: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T06:50:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:27 smithi098 ceph-mon[103124]: pgmap v14591: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T06:50:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:27 smithi170 ceph-mon[102832]: pgmap v14591: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 11 op/s 2023-09-10T06:50:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:29 smithi170 ceph-mon[102832]: pgmap v14592: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 94 KiB/s wr, 9 op/s 2023-09-10T06:50:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27060.621565 secs 2023-09-10T06:50:29.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:29 smithi082 ceph-mon[98694]: pgmap v14592: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 94 KiB/s wr, 9 op/s 2023-09-10T06:50:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27060.621565 secs 2023-09-10T06:50:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:29 smithi098 ceph-mon[103124]: pgmap v14592: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 94 KiB/s wr, 9 op/s 2023-09-10T06:50:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27060.621565 secs 2023-09-10T06:50:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27061.099805 secs 2023-09-10T06:50:30.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27061.099805 secs 2023-09-10T06:50:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27061.099805 secs 2023-09-10T06:50:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:31 smithi170 ceph-mon[102832]: pgmap v14593: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 274 KiB/s wr, 15 op/s 2023-09-10T06:50:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:50:31.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:31 smithi082 ceph-mon[98694]: pgmap v14593: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 274 KiB/s wr, 15 op/s 2023-09-10T06:50:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:50:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:31 smithi098 ceph-mon[103124]: pgmap v14593: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 274 KiB/s wr, 15 op/s 2023-09-10T06:50:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:50:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:50:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:50:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:32.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:50:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:50:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:50:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:50:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:33 smithi170 ceph-mon[102832]: pgmap v14594: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 181 KiB/s wr, 10 op/s 2023-09-10T06:50:33.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:33 smithi082 ceph-mon[98694]: pgmap v14594: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 181 KiB/s wr, 10 op/s 2023-09-10T06:50:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:33 smithi098 ceph-mon[103124]: pgmap v14594: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 181 KiB/s wr, 10 op/s 2023-09-10T06:50:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27065.621623 secs 2023-09-10T06:50:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:50:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27065.621623 secs 2023-09-10T06:50:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:50:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27065.621623 secs 2023-09-10T06:50:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:50:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27066.099870 secs 2023-09-10T06:50:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:35 smithi170 ceph-mon[102832]: pgmap v14595: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 181 KiB/s wr, 10 op/s 2023-09-10T06:50:35.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27066.099870 secs 2023-09-10T06:50:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:35 smithi082 ceph-mon[98694]: pgmap v14595: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 181 KiB/s wr, 10 op/s 2023-09-10T06:50:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27066.099870 secs 2023-09-10T06:50:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:35 smithi098 ceph-mon[103124]: pgmap v14595: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 181 KiB/s wr, 10 op/s 2023-09-10T06:50:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:37 smithi082 ceph-mon[98694]: pgmap v14596: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 408 KiB/s wr, 16 op/s 2023-09-10T06:50:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:37 smithi098 ceph-mon[103124]: pgmap v14596: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 408 KiB/s wr, 16 op/s 2023-09-10T06:50:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:37 smithi170 ceph-mon[102832]: pgmap v14596: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 408 KiB/s wr, 16 op/s 2023-09-10T06:50:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:50:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:50:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:50:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:39 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27069.481540 secs 2023-09-10T06:50:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:39 smithi098 ceph-mon[103124]: pgmap v14597: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 408 KiB/s wr, 12 op/s 2023-09-10T06:50:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27070.621683 secs 2023-09-10T06:50:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:39 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27069.481540 secs 2023-09-10T06:50:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:39 smithi170 ceph-mon[102832]: pgmap v14597: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 408 KiB/s wr, 12 op/s 2023-09-10T06:50:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27070.621683 secs 2023-09-10T06:50:39.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:39 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27069.481540 secs 2023-09-10T06:50:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:39 smithi082 ceph-mon[98694]: pgmap v14597: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 408 KiB/s wr, 12 op/s 2023-09-10T06:50:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27070.621683 secs 2023-09-10T06:50:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27071.099935 secs 2023-09-10T06:50:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:40 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:50:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:50:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27071.099935 secs 2023-09-10T06:50:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:40 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:50:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:50:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:40.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27071.099935 secs 2023-09-10T06:50:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:40 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:50:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:50:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:41 smithi098 ceph-mon[103124]: pgmap v14598: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 427 KiB/s wr, 18 op/s 2023-09-10T06:50:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:41 smithi170 ceph-mon[102832]: pgmap v14598: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 427 KiB/s wr, 18 op/s 2023-09-10T06:50:41.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:41 smithi082 ceph-mon[98694]: pgmap v14598: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 427 KiB/s wr, 18 op/s 2023-09-10T06:50:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:43 smithi170 ceph-mon[102832]: pgmap v14599: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 247 KiB/s wr, 12 op/s 2023-09-10T06:50:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27074.481601 secs 2023-09-10T06:50:43.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:43 smithi082 ceph-mon[98694]: pgmap v14599: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 247 KiB/s wr, 12 op/s 2023-09-10T06:50:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27074.481601 secs 2023-09-10T06:50:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:43 smithi098 ceph-mon[103124]: pgmap v14599: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 247 KiB/s wr, 12 op/s 2023-09-10T06:50:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27074.481601 secs 2023-09-10T06:50:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27075.621746 secs 2023-09-10T06:50:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:50:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:44.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27075.621746 secs 2023-09-10T06:50:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:50:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27075.621746 secs 2023-09-10T06:50:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:50:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27076.100015 secs 2023-09-10T06:50:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:45 smithi170 ceph-mon[102832]: pgmap v14600: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 247 KiB/s wr, 12 op/s 2023-09-10T06:50:45.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27076.100015 secs 2023-09-10T06:50:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:45 smithi082 ceph-mon[98694]: pgmap v14600: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 247 KiB/s wr, 12 op/s 2023-09-10T06:50:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27076.100015 secs 2023-09-10T06:50:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:45 smithi098 ceph-mon[103124]: pgmap v14600: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 247 KiB/s wr, 12 op/s 2023-09-10T06:50:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:50:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:50:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:50:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:47 smithi170 ceph-mon[102832]: pgmap v14601: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 751 KiB/s wr, 19 op/s 2023-09-10T06:50:47.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:47 smithi082 ceph-mon[98694]: pgmap v14601: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 751 KiB/s wr, 19 op/s 2023-09-10T06:50:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:47 smithi098 ceph-mon[103124]: pgmap v14601: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 751 KiB/s wr, 19 op/s 2023-09-10T06:50:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:49 smithi170 ceph-mon[102832]: pgmap v14602: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 523 KiB/s wr, 12 op/s 2023-09-10T06:50:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27080.621807 secs 2023-09-10T06:50:49.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:49 smithi082 ceph-mon[98694]: pgmap v14602: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 523 KiB/s wr, 12 op/s 2023-09-10T06:50:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27080.621807 secs 2023-09-10T06:50:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:49 smithi098 ceph-mon[103124]: pgmap v14602: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 523 KiB/s wr, 12 op/s 2023-09-10T06:50:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27080.621807 secs 2023-09-10T06:50:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27081.100082 secs 2023-09-10T06:50:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27081.100082 secs 2023-09-10T06:50:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27081.100082 secs 2023-09-10T06:50:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:51 smithi170 ceph-mon[102832]: pgmap v14603: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 562 KiB/s wr, 18 op/s 2023-09-10T06:50:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:51 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:50:51.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:51 smithi082 ceph-mon[98694]: pgmap v14603: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 562 KiB/s wr, 18 op/s 2023-09-10T06:50:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:51 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:50:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:51 smithi098 ceph-mon[103124]: pgmap v14603: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 562 KiB/s wr, 18 op/s 2023-09-10T06:50:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:51 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:50:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:52 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:50:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:50:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:52 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:50:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:50:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:52 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:50:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:50:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:53.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:53 smithi082 ceph-mon[98694]: pgmap v14604: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 543 KiB/s wr, 12 op/s 2023-09-10T06:50:53.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27084.481778 secs 2023-09-10T06:50:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:53 smithi098 ceph-mon[103124]: pgmap v14604: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 543 KiB/s wr, 12 op/s 2023-09-10T06:50:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27084.481778 secs 2023-09-10T06:50:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:53 smithi170 ceph-mon[102832]: pgmap v14604: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 543 KiB/s wr, 12 op/s 2023-09-10T06:50:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27084.481778 secs 2023-09-10T06:50:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27085.621866 secs 2023-09-10T06:50:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27085.621866 secs 2023-09-10T06:50:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27085.621866 secs 2023-09-10T06:50:55.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27086.100144 secs 2023-09-10T06:50:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:55 smithi082 ceph-mon[98694]: pgmap v14605: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 543 KiB/s wr, 12 op/s 2023-09-10T06:50:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27086.100144 secs 2023-09-10T06:50:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:55 smithi098 ceph-mon[103124]: pgmap v14605: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 543 KiB/s wr, 12 op/s 2023-09-10T06:50:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27086.100144 secs 2023-09-10T06:50:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:55 smithi170 ceph-mon[102832]: pgmap v14605: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 543 KiB/s wr, 12 op/s 2023-09-10T06:50:56.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:50:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:50:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:50:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:57 smithi082 ceph-mon[98694]: pgmap v14606: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 900 KiB/s wr, 18 op/s 2023-09-10T06:50:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:57 smithi098 ceph-mon[103124]: pgmap v14606: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 900 KiB/s wr, 18 op/s 2023-09-10T06:50:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:57 smithi170 ceph-mon[102832]: pgmap v14606: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 900 KiB/s wr, 18 op/s 2023-09-10T06:50:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:50:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27089.481839 secs 2023-09-10T06:50:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:50:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27089.481839 secs 2023-09-10T06:50:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:50:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:50:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27089.481839 secs 2023-09-10T06:50:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:59 smithi098 ceph-mon[103124]: pgmap v14607: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 396 KiB/s wr, 11 op/s 2023-09-10T06:50:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:50:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27090.621924 secs 2023-09-10T06:50:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:59 smithi170 ceph-mon[102832]: pgmap v14607: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 396 KiB/s wr, 11 op/s 2023-09-10T06:50:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:50:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27090.621924 secs 2023-09-10T06:50:59.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:59 smithi082 ceph-mon[98694]: pgmap v14607: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 396 KiB/s wr, 11 op/s 2023-09-10T06:50:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:50:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27090.621924 secs 2023-09-10T06:51:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27091.100210 secs 2023-09-10T06:51:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27091.100210 secs 2023-09-10T06:51:00.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27091.100210 secs 2023-09-10T06:51:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:01 smithi098 ceph-mon[103124]: pgmap v14608: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 596 KiB/s wr, 17 op/s 2023-09-10T06:51:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:01 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:51:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:01 smithi170 ceph-mon[102832]: pgmap v14608: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 596 KiB/s wr, 17 op/s 2023-09-10T06:51:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:01 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:51:01.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:01 smithi082 ceph-mon[98694]: pgmap v14608: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 596 KiB/s wr, 17 op/s 2023-09-10T06:51:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:01 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:51:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:03 smithi098 ceph-mon[103124]: pgmap v14609: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 557 KiB/s wr, 11 op/s 2023-09-10T06:51:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27094.481900 secs 2023-09-10T06:51:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:03 smithi170 ceph-mon[102832]: pgmap v14609: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 557 KiB/s wr, 11 op/s 2023-09-10T06:51:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27094.481900 secs 2023-09-10T06:51:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:03 smithi082 ceph-mon[98694]: pgmap v14609: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 557 KiB/s wr, 11 op/s 2023-09-10T06:51:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27094.481900 secs 2023-09-10T06:51:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27095.621986 secs 2023-09-10T06:51:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:51:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27095.621986 secs 2023-09-10T06:51:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:51:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:04.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27095.621986 secs 2023-09-10T06:51:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:51:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27096.100275 secs 2023-09-10T06:51:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:05 smithi098 ceph-mon[103124]: pgmap v14610: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 557 KiB/s wr, 11 op/s 2023-09-10T06:51:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27096.100275 secs 2023-09-10T06:51:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:05 smithi170 ceph-mon[102832]: pgmap v14610: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 557 KiB/s wr, 11 op/s 2023-09-10T06:51:05.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27096.100275 secs 2023-09-10T06:51:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:05 smithi082 ceph-mon[98694]: pgmap v14610: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 557 KiB/s wr, 11 op/s 2023-09-10T06:51:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:51:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:51:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:51:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:07 smithi098 ceph-mon[103124]: pgmap v14611: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 696 KiB/s wr, 32 op/s 2023-09-10T06:51:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:07 smithi170 ceph-mon[102832]: pgmap v14611: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 696 KiB/s wr, 32 op/s 2023-09-10T06:51:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:07 smithi082 ceph-mon[98694]: pgmap v14611: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 696 KiB/s wr, 32 op/s 2023-09-10T06:51:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27099.481961 secs 2023-09-10T06:51:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27099.481961 secs 2023-09-10T06:51:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27099.481961 secs 2023-09-10T06:51:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:09 smithi170 ceph-mon[102832]: pgmap v14612: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 339 KiB/s wr, 26 op/s 2023-09-10T06:51:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27100.622069 secs 2023-09-10T06:51:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:09 smithi170 ceph-mon[102832]: osdmap e981: 12 total, 12 up, 12 in 2023-09-10T06:51:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:09 smithi082 ceph-mon[98694]: pgmap v14612: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 339 KiB/s wr, 26 op/s 2023-09-10T06:51:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27100.622069 secs 2023-09-10T06:51:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:09 smithi082 ceph-mon[98694]: osdmap e981: 12 total, 12 up, 12 in 2023-09-10T06:51:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:09 smithi098 ceph-mon[103124]: pgmap v14612: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 339 KiB/s wr, 26 op/s 2023-09-10T06:51:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27100.622069 secs 2023-09-10T06:51:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:09 smithi098 ceph-mon[103124]: osdmap e981: 12 total, 12 up, 12 in 2023-09-10T06:51:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27101.100343 secs 2023-09-10T06:51:10.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27101.100343 secs 2023-09-10T06:51:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27101.100343 secs 2023-09-10T06:51:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:11 smithi170 ceph-mon[102832]: pgmap v14614: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 191 KiB/s wr, 30 op/s 2023-09-10T06:51:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:11 smithi082 ceph-mon[98694]: pgmap v14614: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 191 KiB/s wr, 30 op/s 2023-09-10T06:51:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:11 smithi098 ceph-mon[103124]: pgmap v14614: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 191 KiB/s wr, 30 op/s 2023-09-10T06:51:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:13 smithi170 ceph-mon[102832]: pgmap v14615: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 191 KiB/s wr, 30 op/s 2023-09-10T06:51:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27104.482022 secs 2023-09-10T06:51:13.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:13 smithi082 ceph-mon[98694]: pgmap v14615: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 191 KiB/s wr, 30 op/s 2023-09-10T06:51:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27104.482022 secs 2023-09-10T06:51:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:13 smithi098 ceph-mon[103124]: pgmap v14615: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 191 KiB/s wr, 30 op/s 2023-09-10T06:51:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27104.482022 secs 2023-09-10T06:51:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27105.622127 secs 2023-09-10T06:51:14.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27105.622127 secs 2023-09-10T06:51:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27105.622127 secs 2023-09-10T06:51:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27106.100407 secs 2023-09-10T06:51:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:15 smithi170 ceph-mon[102832]: pgmap v14616: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 191 KiB/s wr, 30 op/s 2023-09-10T06:51:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27106.100407 secs 2023-09-10T06:51:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:15 smithi082 ceph-mon[98694]: pgmap v14616: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 191 KiB/s wr, 30 op/s 2023-09-10T06:51:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27106.100407 secs 2023-09-10T06:51:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:15 smithi098 ceph-mon[103124]: pgmap v14616: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 191 KiB/s wr, 30 op/s 2023-09-10T06:51:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:17 smithi082 ceph-mon[98694]: osdmap e982: 12 total, 12 up, 12 in 2023-09-10T06:51:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:17 smithi082 ceph-mon[98694]: pgmap v14618: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 172 KiB/s wr, 15 op/s 2023-09-10T06:51:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:17 smithi098 ceph-mon[103124]: osdmap e982: 12 total, 12 up, 12 in 2023-09-10T06:51:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:17 smithi098 ceph-mon[103124]: pgmap v14618: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 172 KiB/s wr, 15 op/s 2023-09-10T06:51:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:17 smithi170 ceph-mon[102832]: osdmap e982: 12 total, 12 up, 12 in 2023-09-10T06:51:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:17 smithi170 ceph-mon[102832]: pgmap v14618: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 172 KiB/s wr, 15 op/s 2023-09-10T06:51:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27109.482080 secs 2023-09-10T06:51:18.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27109.482080 secs 2023-09-10T06:51:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27109.482080 secs 2023-09-10T06:51:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:19 smithi170 ceph-mon[102832]: pgmap v14619: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 12 op/s 2023-09-10T06:51:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27110.622216 secs 2023-09-10T06:51:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:19 smithi082 ceph-mon[98694]: pgmap v14619: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 12 op/s 2023-09-10T06:51:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27110.622216 secs 2023-09-10T06:51:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:19 smithi098 ceph-mon[103124]: pgmap v14619: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 12 op/s 2023-09-10T06:51:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27110.622216 secs 2023-09-10T06:51:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27111.100476 secs 2023-09-10T06:51:20.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27111.100476 secs 2023-09-10T06:51:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27111.100476 secs 2023-09-10T06:51:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:21 smithi170 ceph-mon[102832]: pgmap v14620: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T06:51:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:51:21.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:21 smithi082 ceph-mon[98694]: pgmap v14620: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T06:51:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:51:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:21 smithi098 ceph-mon[103124]: pgmap v14620: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T06:51:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:51:23.589 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:23 smithi170 ceph-mon[102832]: pgmap v14621: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T06:51:23.589 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27114.482145 secs 2023-09-10T06:51:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:23 smithi082 ceph-mon[98694]: pgmap v14621: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T06:51:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27114.482145 secs 2023-09-10T06:51:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:23 smithi098 ceph-mon[103124]: pgmap v14621: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T06:51:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27114.482145 secs 2023-09-10T06:51:24.596 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27115.622289 secs 2023-09-10T06:51:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27115.622289 secs 2023-09-10T06:51:24.870 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27115.622289 secs 2023-09-10T06:51:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27116.100542 secs 2023-09-10T06:51:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:25 smithi170 ceph-mon[102832]: pgmap v14622: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T06:51:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:51:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:51:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:51:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:51:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:51:25.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:51:25.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:51:25.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:51:25.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:51:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27116.100542 secs 2023-09-10T06:51:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:25 smithi082 ceph-mon[98694]: pgmap v14622: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T06:51:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:51:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:51:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:51:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:51:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:51:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:51:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:51:25.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:51:25.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:51:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27116.100542 secs 2023-09-10T06:51:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:25 smithi098 ceph-mon[103124]: pgmap v14622: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T06:51:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:51:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:51:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:51:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:51:25.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:51:25.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:51:25.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:51:25.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:51:25.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:51:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:27 smithi098 ceph-mon[103124]: pgmap v14623: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T06:51:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:27 smithi170 ceph-mon[102832]: pgmap v14623: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T06:51:27.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:27 smithi082 ceph-mon[98694]: pgmap v14623: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 11 op/s 2023-09-10T06:51:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:29 smithi170 ceph-mon[102832]: pgmap v14624: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 9 op/s 2023-09-10T06:51:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27120.622377 secs 2023-09-10T06:51:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:29 smithi082 ceph-mon[98694]: pgmap v14624: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 9 op/s 2023-09-10T06:51:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27120.622377 secs 2023-09-10T06:51:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:29 smithi098 ceph-mon[103124]: pgmap v14624: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 9 op/s 2023-09-10T06:51:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27120.622377 secs 2023-09-10T06:51:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27121.100607 secs 2023-09-10T06:51:30.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27121.100607 secs 2023-09-10T06:51:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27121.100607 secs 2023-09-10T06:51:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:31 smithi170 ceph-mon[102832]: pgmap v14625: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 230 KiB/s wr, 14 op/s 2023-09-10T06:51:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:51:31.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:31 smithi082 ceph-mon[98694]: pgmap v14625: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 230 KiB/s wr, 14 op/s 2023-09-10T06:51:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:51:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:31 smithi098 ceph-mon[103124]: pgmap v14625: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 230 KiB/s wr, 14 op/s 2023-09-10T06:51:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:51:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:51:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:51:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:32.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:51:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:51:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:51:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:51:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:33.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:33 smithi170 ceph-mon[102832]: pgmap v14626: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 230 KiB/s wr, 10 op/s 2023-09-10T06:51:33.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:33 smithi082 ceph-mon[98694]: pgmap v14626: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 230 KiB/s wr, 10 op/s 2023-09-10T06:51:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:33 smithi098 ceph-mon[103124]: pgmap v14626: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 230 KiB/s wr, 10 op/s 2023-09-10T06:51:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27125.622437 secs 2023-09-10T06:51:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:51:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27125.622437 secs 2023-09-10T06:51:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:51:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27125.622437 secs 2023-09-10T06:51:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:51:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27126.100669 secs 2023-09-10T06:51:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:35 smithi082 ceph-mon[98694]: pgmap v14627: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 230 KiB/s wr, 10 op/s 2023-09-10T06:51:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27126.100669 secs 2023-09-10T06:51:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:35 smithi098 ceph-mon[103124]: pgmap v14627: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 230 KiB/s wr, 10 op/s 2023-09-10T06:51:35.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27126.100669 secs 2023-09-10T06:51:35.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:35 smithi170 ceph-mon[102832]: pgmap v14627: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 230 KiB/s wr, 10 op/s 2023-09-10T06:51:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:51:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:51:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:51:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:37.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:37 smithi082 ceph-mon[98694]: pgmap v14628: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 343 KiB/s wr, 15 op/s 2023-09-10T06:51:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:37 smithi098 ceph-mon[103124]: pgmap v14628: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 343 KiB/s wr, 15 op/s 2023-09-10T06:51:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:37 smithi170 ceph-mon[102832]: pgmap v14628: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 343 KiB/s wr, 15 op/s 2023-09-10T06:51:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:51:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:51:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:51:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:39 smithi082 ceph-mon[98694]: pgmap v14629: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 232 KiB/s wr, 11 op/s 2023-09-10T06:51:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27130.622516 secs 2023-09-10T06:51:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:39 smithi098 ceph-mon[103124]: pgmap v14629: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 232 KiB/s wr, 11 op/s 2023-09-10T06:51:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27130.622516 secs 2023-09-10T06:51:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:39 smithi170 ceph-mon[102832]: pgmap v14629: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 232 KiB/s wr, 11 op/s 2023-09-10T06:51:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27130.622516 secs 2023-09-10T06:51:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27131.100735 secs 2023-09-10T06:51:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:51:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27131.100735 secs 2023-09-10T06:51:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:51:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:40.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27131.100735 secs 2023-09-10T06:51:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:51:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:41 smithi098 ceph-mon[103124]: pgmap v14630: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 585 KiB/s wr, 18 op/s 2023-09-10T06:51:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:41 smithi170 ceph-mon[102832]: pgmap v14630: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 585 KiB/s wr, 18 op/s 2023-09-10T06:51:41.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:41 smithi082 ceph-mon[98694]: pgmap v14630: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 585 KiB/s wr, 18 op/s 2023-09-10T06:51:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:43 smithi098 ceph-mon[103124]: pgmap v14631: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 466 KiB/s wr, 12 op/s 2023-09-10T06:51:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27134.482420 secs 2023-09-10T06:51:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:43 smithi170 ceph-mon[102832]: pgmap v14631: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 466 KiB/s wr, 12 op/s 2023-09-10T06:51:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27134.482420 secs 2023-09-10T06:51:43.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:43 smithi082 ceph-mon[98694]: pgmap v14631: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 466 KiB/s wr, 12 op/s 2023-09-10T06:51:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27134.482420 secs 2023-09-10T06:51:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27135.622593 secs 2023-09-10T06:51:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:51:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:51:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27135.622593 secs 2023-09-10T06:51:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:51:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:51:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:44.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27135.622593 secs 2023-09-10T06:51:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:51:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:51:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27136.100798 secs 2023-09-10T06:51:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:45 smithi098 ceph-mon[103124]: pgmap v14632: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 466 KiB/s wr, 12 op/s 2023-09-10T06:51:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27136.100798 secs 2023-09-10T06:51:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:45 smithi170 ceph-mon[102832]: pgmap v14632: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 466 KiB/s wr, 12 op/s 2023-09-10T06:51:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27136.100798 secs 2023-09-10T06:51:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:45 smithi082 ceph-mon[98694]: pgmap v14632: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 466 KiB/s wr, 12 op/s 2023-09-10T06:51:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:47 smithi098 ceph-mon[103124]: pgmap v14633: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 748 KiB/s wr, 19 op/s 2023-09-10T06:51:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:47 smithi170 ceph-mon[102832]: pgmap v14633: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 748 KiB/s wr, 19 op/s 2023-09-10T06:51:47.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:47 smithi082 ceph-mon[98694]: pgmap v14633: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 748 KiB/s wr, 19 op/s 2023-09-10T06:51:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27139.482500 secs 2023-09-10T06:51:48.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27139.482500 secs 2023-09-10T06:51:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27139.482500 secs 2023-09-10T06:51:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:49 smithi170 ceph-mon[102832]: pgmap v14634: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 635 KiB/s wr, 14 op/s 2023-09-10T06:51:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27140.622652 secs 2023-09-10T06:51:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:49 smithi082 ceph-mon[98694]: pgmap v14634: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 635 KiB/s wr, 14 op/s 2023-09-10T06:51:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27140.622652 secs 2023-09-10T06:51:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:49 smithi098 ceph-mon[103124]: pgmap v14634: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 635 KiB/s wr, 14 op/s 2023-09-10T06:51:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27140.622652 secs 2023-09-10T06:51:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27141.100861 secs 2023-09-10T06:51:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:51:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:50.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27141.100861 secs 2023-09-10T06:51:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:51:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27141.100861 secs 2023-09-10T06:51:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:51:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:51 smithi170 ceph-mon[102832]: pgmap v14635: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 720 KiB/s wr, 20 op/s 2023-09-10T06:51:51.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:51 smithi082 ceph-mon[98694]: pgmap v14635: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 720 KiB/s wr, 20 op/s 2023-09-10T06:51:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:51 smithi098 ceph-mon[103124]: pgmap v14635: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 720 KiB/s wr, 20 op/s 2023-09-10T06:51:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:53 smithi170 ceph-mon[102832]: pgmap v14636: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 368 KiB/s wr, 13 op/s 2023-09-10T06:51:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27144.482560 secs 2023-09-10T06:51:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:53 smithi082 ceph-mon[98694]: pgmap v14636: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 368 KiB/s wr, 13 op/s 2023-09-10T06:51:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27144.482560 secs 2023-09-10T06:51:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:53 smithi098 ceph-mon[103124]: pgmap v14636: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 368 KiB/s wr, 13 op/s 2023-09-10T06:51:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27144.482560 secs 2023-09-10T06:51:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27145.622710 secs 2023-09-10T06:51:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27145.622710 secs 2023-09-10T06:51:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27145.622710 secs 2023-09-10T06:51:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27146.100926 secs 2023-09-10T06:51:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:55 smithi082 ceph-mon[98694]: pgmap v14637: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 368 KiB/s wr, 13 op/s 2023-09-10T06:51:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27146.100926 secs 2023-09-10T06:51:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:55 smithi098 ceph-mon[103124]: pgmap v14637: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 368 KiB/s wr, 13 op/s 2023-09-10T06:51:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27146.100926 secs 2023-09-10T06:51:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:55 smithi170 ceph-mon[102832]: pgmap v14637: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 368 KiB/s wr, 13 op/s 2023-09-10T06:51:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:57 smithi098 ceph-mon[103124]: pgmap v14638: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 548 KiB/s wr, 19 op/s 2023-09-10T06:51:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:57 smithi170 ceph-mon[102832]: pgmap v14638: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 548 KiB/s wr, 19 op/s 2023-09-10T06:51:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:57 smithi082 ceph-mon[98694]: pgmap v14638: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 548 KiB/s wr, 19 op/s 2023-09-10T06:51:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:51:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:51:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:58.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:51:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:51:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27149.482654 secs 2023-09-10T06:51:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:59 smithi098 ceph-mon[103124]: pgmap v14639: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 266 KiB/s wr, 12 op/s 2023-09-10T06:51:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:51:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27150.622770 secs 2023-09-10T06:51:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27149.482654 secs 2023-09-10T06:51:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:59 smithi170 ceph-mon[102832]: pgmap v14639: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 266 KiB/s wr, 12 op/s 2023-09-10T06:51:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:51:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27150.622770 secs 2023-09-10T06:51:59.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27149.482654 secs 2023-09-10T06:51:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:59 smithi082 ceph-mon[98694]: pgmap v14639: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 266 KiB/s wr, 12 op/s 2023-09-10T06:51:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:51:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27150.622770 secs 2023-09-10T06:52:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27151.100990 secs 2023-09-10T06:52:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:52:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:52:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27151.100990 secs 2023-09-10T06:52:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:52:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:52:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27151.100990 secs 2023-09-10T06:52:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:52:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:52:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:01 smithi098 ceph-mon[103124]: pgmap v14640: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 525 KiB/s wr, 32 op/s 2023-09-10T06:52:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:01 smithi170 ceph-mon[102832]: pgmap v14640: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 525 KiB/s wr, 32 op/s 2023-09-10T06:52:01.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:01 smithi082 ceph-mon[98694]: pgmap v14640: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 525 KiB/s wr, 32 op/s 2023-09-10T06:52:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:03 smithi170 ceph-mon[102832]: pgmap v14641: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 439 KiB/s wr, 25 op/s 2023-09-10T06:52:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27154.482746 secs 2023-09-10T06:52:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:03 smithi082 ceph-mon[98694]: pgmap v14641: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 439 KiB/s wr, 25 op/s 2023-09-10T06:52:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27154.482746 secs 2023-09-10T06:52:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:03 smithi098 ceph-mon[103124]: pgmap v14641: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 439 KiB/s wr, 25 op/s 2023-09-10T06:52:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27154.482746 secs 2023-09-10T06:52:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27155.622840 secs 2023-09-10T06:52:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27155.622840 secs 2023-09-10T06:52:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27155.622840 secs 2023-09-10T06:52:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27156.101054 secs 2023-09-10T06:52:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:05 smithi170 ceph-mon[102832]: pgmap v14642: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 439 KiB/s wr, 25 op/s 2023-09-10T06:52:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27156.101054 secs 2023-09-10T06:52:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:05 smithi082 ceph-mon[98694]: pgmap v14642: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 439 KiB/s wr, 25 op/s 2023-09-10T06:52:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27156.101054 secs 2023-09-10T06:52:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:05 smithi098 ceph-mon[103124]: pgmap v14642: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 439 KiB/s wr, 25 op/s 2023-09-10T06:52:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:07 smithi098 ceph-mon[103124]: pgmap v14643: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 533 KiB/s wr, 32 op/s 2023-09-10T06:52:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:07 smithi170 ceph-mon[102832]: pgmap v14643: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 533 KiB/s wr, 32 op/s 2023-09-10T06:52:07.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:07 smithi082 ceph-mon[98694]: pgmap v14643: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 533 KiB/s wr, 32 op/s 2023-09-10T06:52:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27159.482777 secs 2023-09-10T06:52:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:09 smithi098 ceph-mon[103124]: osdmap e983: 12 total, 12 up, 12 in 2023-09-10T06:52:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27160.622901 secs 2023-09-10T06:52:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:09 smithi098 ceph-mon[103124]: pgmap v14645: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 422 KiB/s wr, 31 op/s 2023-09-10T06:52:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27159.482777 secs 2023-09-10T06:52:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:09 smithi170 ceph-mon[102832]: osdmap e983: 12 total, 12 up, 12 in 2023-09-10T06:52:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27160.622901 secs 2023-09-10T06:52:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:09 smithi170 ceph-mon[102832]: pgmap v14645: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 422 KiB/s wr, 31 op/s 2023-09-10T06:52:09.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27159.482777 secs 2023-09-10T06:52:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:09 smithi082 ceph-mon[98694]: osdmap e983: 12 total, 12 up, 12 in 2023-09-10T06:52:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27160.622901 secs 2023-09-10T06:52:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:09 smithi082 ceph-mon[98694]: pgmap v14645: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 422 KiB/s wr, 31 op/s 2023-09-10T06:52:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27161.101116 secs 2023-09-10T06:52:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27161.101116 secs 2023-09-10T06:52:10.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27161.101116 secs 2023-09-10T06:52:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:11 smithi098 ceph-mon[103124]: pgmap v14646: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 137 KiB/s wr, 13 op/s 2023-09-10T06:52:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:11 smithi170 ceph-mon[102832]: pgmap v14646: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 137 KiB/s wr, 13 op/s 2023-09-10T06:52:11.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:11 smithi082 ceph-mon[98694]: pgmap v14646: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 137 KiB/s wr, 13 op/s 2023-09-10T06:52:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:13 smithi170 ceph-mon[102832]: pgmap v14647: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 137 KiB/s wr, 13 op/s 2023-09-10T06:52:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27164.482839 secs 2023-09-10T06:52:13.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:13 smithi082 ceph-mon[98694]: pgmap v14647: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 137 KiB/s wr, 13 op/s 2023-09-10T06:52:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27164.482839 secs 2023-09-10T06:52:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:13 smithi098 ceph-mon[103124]: pgmap v14647: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 137 KiB/s wr, 13 op/s 2023-09-10T06:52:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27164.482839 secs 2023-09-10T06:52:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27165.622967 secs 2023-09-10T06:52:14.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27165.622967 secs 2023-09-10T06:52:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27165.622967 secs 2023-09-10T06:52:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27166.101178 secs 2023-09-10T06:52:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:15 smithi170 ceph-mon[102832]: pgmap v14648: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 137 KiB/s wr, 13 op/s 2023-09-10T06:52:15.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27166.101178 secs 2023-09-10T06:52:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:15 smithi082 ceph-mon[98694]: pgmap v14648: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 137 KiB/s wr, 13 op/s 2023-09-10T06:52:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27166.101178 secs 2023-09-10T06:52:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:15 smithi098 ceph-mon[103124]: pgmap v14648: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 137 KiB/s wr, 13 op/s 2023-09-10T06:52:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:17 smithi098 ceph-mon[103124]: osdmap e984: 12 total, 12 up, 12 in 2023-09-10T06:52:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:17 smithi098 ceph-mon[103124]: pgmap v14650: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 15 op/s 2023-09-10T06:52:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:17 smithi170 ceph-mon[102832]: osdmap e984: 12 total, 12 up, 12 in 2023-09-10T06:52:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:17 smithi170 ceph-mon[102832]: pgmap v14650: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 15 op/s 2023-09-10T06:52:17.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:17 smithi082 ceph-mon[98694]: osdmap e984: 12 total, 12 up, 12 in 2023-09-10T06:52:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:17 smithi082 ceph-mon[98694]: pgmap v14650: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 15 op/s 2023-09-10T06:52:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27169.482944 secs 2023-09-10T06:52:18.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27169.482944 secs 2023-09-10T06:52:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27169.482944 secs 2023-09-10T06:52:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27170.623030 secs 2023-09-10T06:52:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:19 smithi170 ceph-mon[102832]: pgmap v14651: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-10T06:52:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27170.623030 secs 2023-09-10T06:52:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:19 smithi082 ceph-mon[98694]: pgmap v14651: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-10T06:52:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27170.623030 secs 2023-09-10T06:52:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:19 smithi098 ceph-mon[103124]: pgmap v14651: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-09-10T06:52:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27171.101242 secs 2023-09-10T06:52:20.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27171.101242 secs 2023-09-10T06:52:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27171.101242 secs 2023-09-10T06:52:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:21 smithi082 ceph-mon[98694]: pgmap v14652: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T06:52:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:21 smithi098 ceph-mon[103124]: pgmap v14652: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T06:52:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:21 smithi170 ceph-mon[102832]: pgmap v14652: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T06:52:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:23 smithi082 ceph-mon[98694]: pgmap v14653: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T06:52:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27174.483006 secs 2023-09-10T06:52:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:23 smithi098 ceph-mon[103124]: pgmap v14653: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T06:52:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27174.483006 secs 2023-09-10T06:52:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:23 smithi170 ceph-mon[102832]: pgmap v14653: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T06:52:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27174.483006 secs 2023-09-10T06:52:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27175.623088 secs 2023-09-10T06:52:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27175.623088 secs 2023-09-10T06:52:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27175.623088 secs 2023-09-10T06:52:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27176.101303 secs 2023-09-10T06:52:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:25 smithi082 ceph-mon[98694]: pgmap v14654: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T06:52:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:52:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27176.101303 secs 2023-09-10T06:52:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:25 smithi098 ceph-mon[103124]: pgmap v14654: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T06:52:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:52:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27176.101303 secs 2023-09-10T06:52:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:25 smithi170 ceph-mon[102832]: pgmap v14654: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T06:52:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:52:26.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:52:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:52:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:52:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:52:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:52:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:52:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:52:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:52:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:52:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:27 smithi082 ceph-mon[98694]: pgmap v14655: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T06:52:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:27 smithi098 ceph-mon[103124]: pgmap v14655: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T06:52:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:27 smithi170 ceph-mon[102832]: pgmap v14655: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 12 op/s 2023-09-10T06:52:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27180.623147 secs 2023-09-10T06:52:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:29 smithi082 ceph-mon[98694]: pgmap v14656: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 10 op/s 2023-09-10T06:52:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27180.623147 secs 2023-09-10T06:52:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:29 smithi098 ceph-mon[103124]: pgmap v14656: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 10 op/s 2023-09-10T06:52:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27180.623147 secs 2023-09-10T06:52:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:29 smithi170 ceph-mon[102832]: pgmap v14656: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 10 op/s 2023-09-10T06:52:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27181.101369 secs 2023-09-10T06:52:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:52:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:52:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27181.101369 secs 2023-09-10T06:52:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:52:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:52:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27181.101369 secs 2023-09-10T06:52:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:52:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:52:31.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:31 smithi082 ceph-mon[98694]: pgmap v14657: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 296 KiB/s wr, 15 op/s 2023-09-10T06:52:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:52:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:31 smithi098 ceph-mon[103124]: pgmap v14657: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 296 KiB/s wr, 15 op/s 2023-09-10T06:52:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:52:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:31 smithi170 ceph-mon[102832]: pgmap v14657: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 296 KiB/s wr, 15 op/s 2023-09-10T06:52:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:52:32.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:52:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:52:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:52:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:52:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:52:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:52:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:52:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:52:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:52:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:33 smithi098 ceph-mon[103124]: pgmap v14658: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 202 KiB/s wr, 10 op/s 2023-09-10T06:52:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:33 smithi170 ceph-mon[102832]: pgmap v14658: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 202 KiB/s wr, 10 op/s 2023-09-10T06:52:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:33 smithi082 ceph-mon[98694]: pgmap v14658: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 202 KiB/s wr, 10 op/s 2023-09-10T06:52:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27185.623208 secs 2023-09-10T06:52:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27185.623208 secs 2023-09-10T06:52:34.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27185.623208 secs 2023-09-10T06:52:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27186.101432 secs 2023-09-10T06:52:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:35 smithi098 ceph-mon[103124]: pgmap v14659: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 202 KiB/s wr, 10 op/s 2023-09-10T06:52:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27186.101432 secs 2023-09-10T06:52:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:35 smithi170 ceph-mon[102832]: pgmap v14659: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 202 KiB/s wr, 10 op/s 2023-09-10T06:52:35.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27186.101432 secs 2023-09-10T06:52:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:35 smithi082 ceph-mon[98694]: pgmap v14659: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 202 KiB/s wr, 10 op/s 2023-09-10T06:52:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:37 smithi098 ceph-mon[103124]: pgmap v14660: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 337 KiB/s wr, 15 op/s 2023-09-10T06:52:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:37 smithi170 ceph-mon[102832]: pgmap v14660: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 337 KiB/s wr, 15 op/s 2023-09-10T06:52:37.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:37 smithi082 ceph-mon[98694]: pgmap v14660: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 337 KiB/s wr, 15 op/s 2023-09-10T06:52:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27190.623267 secs 2023-09-10T06:52:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:39 smithi170 ceph-mon[102832]: pgmap v14661: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 337 KiB/s wr, 10 op/s 2023-09-10T06:52:39.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27190.623267 secs 2023-09-10T06:52:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:39 smithi082 ceph-mon[98694]: pgmap v14661: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 337 KiB/s wr, 10 op/s 2023-09-10T06:52:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27190.623267 secs 2023-09-10T06:52:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:39 smithi098 ceph-mon[103124]: pgmap v14661: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 337 KiB/s wr, 10 op/s 2023-09-10T06:52:40.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27191.101486 secs 2023-09-10T06:52:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27191.101486 secs 2023-09-10T06:52:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27191.101486 secs 2023-09-10T06:52:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:41 smithi082 ceph-mon[98694]: pgmap v14662: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 369 KiB/s wr, 17 op/s 2023-09-10T06:52:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:41 smithi098 ceph-mon[103124]: pgmap v14662: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 369 KiB/s wr, 17 op/s 2023-09-10T06:52:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:41 smithi170 ceph-mon[102832]: pgmap v14662: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 369 KiB/s wr, 17 op/s 2023-09-10T06:52:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:43 smithi082 ceph-mon[98694]: pgmap v14663: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 166 KiB/s wr, 12 op/s 2023-09-10T06:52:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:43 smithi098 ceph-mon[103124]: pgmap v14663: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 166 KiB/s wr, 12 op/s 2023-09-10T06:52:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:43 smithi170 ceph-mon[102832]: pgmap v14663: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 166 KiB/s wr, 12 op/s 2023-09-10T06:52:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27195.623346 secs 2023-09-10T06:52:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:52:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:52:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27195.623346 secs 2023-09-10T06:52:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:52:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:52:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27195.623346 secs 2023-09-10T06:52:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:52:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:52:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27196.101548 secs 2023-09-10T06:52:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:45 smithi098 ceph-mon[103124]: pgmap v14664: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 166 KiB/s wr, 12 op/s 2023-09-10T06:52:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27196.101548 secs 2023-09-10T06:52:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:45 smithi170 ceph-mon[102832]: pgmap v14664: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 166 KiB/s wr, 12 op/s 2023-09-10T06:52:45.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27196.101548 secs 2023-09-10T06:52:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:45 smithi082 ceph-mon[98694]: pgmap v14664: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 166 KiB/s wr, 12 op/s 2023-09-10T06:52:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:52:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:52:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:52:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:52:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:52:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:52:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:47 smithi098 ceph-mon[103124]: pgmap v14665: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 518 KiB/s wr, 19 op/s 2023-09-10T06:52:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:47 smithi170 ceph-mon[102832]: pgmap v14665: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 518 KiB/s wr, 19 op/s 2023-09-10T06:52:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:47 smithi082 ceph-mon[98694]: pgmap v14665: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 518 KiB/s wr, 19 op/s 2023-09-10T06:52:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27200.623405 secs 2023-09-10T06:52:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:49 smithi098 ceph-mon[103124]: pgmap v14666: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 384 KiB/s wr, 13 op/s 2023-09-10T06:52:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27200.623405 secs 2023-09-10T06:52:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:49 smithi170 ceph-mon[102832]: pgmap v14666: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 384 KiB/s wr, 13 op/s 2023-09-10T06:52:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27200.623405 secs 2023-09-10T06:52:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:49 smithi082 ceph-mon[98694]: pgmap v14666: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 384 KiB/s wr, 13 op/s 2023-09-10T06:52:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27201.101616 secs 2023-09-10T06:52:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27201.101616 secs 2023-09-10T06:52:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27201.101616 secs 2023-09-10T06:52:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:51 smithi098 ceph-mon[103124]: pgmap v14667: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 515 KiB/s wr, 19 op/s 2023-09-10T06:52:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:51 smithi170 ceph-mon[102832]: pgmap v14667: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 515 KiB/s wr, 19 op/s 2023-09-10T06:52:51.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:51 smithi082 ceph-mon[98694]: pgmap v14667: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 515 KiB/s wr, 19 op/s 2023-09-10T06:52:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:52:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:52:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:52:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:52:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:52:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:52:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:53 smithi098 ceph-mon[103124]: pgmap v14668: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 483 KiB/s wr, 13 op/s 2023-09-10T06:52:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27204.483434 secs 2023-09-10T06:52:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:53 smithi170 ceph-mon[102832]: pgmap v14668: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 483 KiB/s wr, 13 op/s 2023-09-10T06:52:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27204.483434 secs 2023-09-10T06:52:53.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:53 smithi082 ceph-mon[98694]: pgmap v14668: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 483 KiB/s wr, 13 op/s 2023-09-10T06:52:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27204.483434 secs 2023-09-10T06:52:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27205.623463 secs 2023-09-10T06:52:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:52:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:52:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27205.623463 secs 2023-09-10T06:52:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:52:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:52:54.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27205.623463 secs 2023-09-10T06:52:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:52:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:52:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27206.101681 secs 2023-09-10T06:52:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:55 smithi170 ceph-mon[102832]: pgmap v14669: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 483 KiB/s wr, 13 op/s 2023-09-10T06:52:55.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27206.101681 secs 2023-09-10T06:52:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:55 smithi082 ceph-mon[98694]: pgmap v14669: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 483 KiB/s wr, 13 op/s 2023-09-10T06:52:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27206.101681 secs 2023-09-10T06:52:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:55 smithi098 ceph-mon[103124]: pgmap v14669: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 483 KiB/s wr, 13 op/s 2023-09-10T06:52:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:52:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:52:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:52:56.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:52:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:52:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:52:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:52:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:52:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:52:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:57 smithi170 ceph-mon[102832]: pgmap v14670: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 745 KiB/s wr, 20 op/s 2023-09-10T06:52:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:57 smithi082 ceph-mon[98694]: pgmap v14670: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 745 KiB/s wr, 20 op/s 2023-09-10T06:52:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:57 smithi098 ceph-mon[103124]: pgmap v14670: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 745 KiB/s wr, 20 op/s 2023-09-10T06:52:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:52:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:52:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27209.483494 secs 2023-09-10T06:52:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:52:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:52:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27209.483494 secs 2023-09-10T06:52:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:52:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:52:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27209.483494 secs 2023-09-10T06:52:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27210.623520 secs 2023-09-10T06:52:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:52:59 smithi170 ceph-mon[102832]: pgmap v14671: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T06:52:59.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27210.623520 secs 2023-09-10T06:52:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:52:59 smithi082 ceph-mon[98694]: pgmap v14671: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T06:53:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27210.623520 secs 2023-09-10T06:53:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:52:59 smithi098 ceph-mon[103124]: pgmap v14671: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T06:53:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27211.101748 secs 2023-09-10T06:53:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:53:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:00.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27211.101748 secs 2023-09-10T06:53:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:53:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27211.101748 secs 2023-09-10T06:53:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:53:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:01 smithi170 ceph-mon[102832]: pgmap v14672: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 501 KiB/s wr, 21 op/s 2023-09-10T06:53:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:01 smithi082 ceph-mon[98694]: pgmap v14672: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 501 KiB/s wr, 21 op/s 2023-09-10T06:53:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:01 smithi098 ceph-mon[103124]: pgmap v14672: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 501 KiB/s wr, 21 op/s 2023-09-10T06:53:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:03 smithi170 ceph-mon[102832]: pgmap v14673: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 370 KiB/s wr, 15 op/s 2023-09-10T06:53:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27214.483626 secs 2023-09-10T06:53:03.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:03 smithi082 ceph-mon[98694]: pgmap v14673: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 370 KiB/s wr, 15 op/s 2023-09-10T06:53:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27214.483626 secs 2023-09-10T06:53:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:03 smithi098 ceph-mon[103124]: pgmap v14673: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 370 KiB/s wr, 15 op/s 2023-09-10T06:53:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27214.483626 secs 2023-09-10T06:53:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27215.623578 secs 2023-09-10T06:53:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:04 smithi170 ceph-mon[102832]: osdmap e985: 12 total, 12 up, 12 in 2023-09-10T06:53:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27216.101813 secs 2023-09-10T06:53:04.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27215.623578 secs 2023-09-10T06:53:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:04 smithi082 ceph-mon[98694]: osdmap e985: 12 total, 12 up, 12 in 2023-09-10T06:53:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27216.101813 secs 2023-09-10T06:53:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27215.623578 secs 2023-09-10T06:53:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:04 smithi098 ceph-mon[103124]: osdmap e985: 12 total, 12 up, 12 in 2023-09-10T06:53:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27216.101813 secs 2023-09-10T06:53:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:05 smithi170 ceph-mon[102832]: pgmap v14675: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 444 KiB/s wr, 19 op/s 2023-09-10T06:53:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:05 smithi082 ceph-mon[98694]: pgmap v14675: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 444 KiB/s wr, 19 op/s 2023-09-10T06:53:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:05 smithi098 ceph-mon[103124]: pgmap v14675: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 444 KiB/s wr, 19 op/s 2023-09-10T06:53:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:07 smithi098 ceph-mon[103124]: pgmap v14676: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 430 KiB/s wr, 31 op/s 2023-09-10T06:53:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:07 smithi170 ceph-mon[102832]: pgmap v14676: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 430 KiB/s wr, 31 op/s 2023-09-10T06:53:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:07 smithi082 ceph-mon[98694]: pgmap v14676: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 430 KiB/s wr, 31 op/s 2023-09-10T06:53:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27219.483684 secs 2023-09-10T06:53:08.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27219.483684 secs 2023-09-10T06:53:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27219.483684 secs 2023-09-10T06:53:09.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27220.623636 secs 2023-09-10T06:53:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:09 smithi082 ceph-mon[98694]: pgmap v14677: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 430 KiB/s wr, 31 op/s 2023-09-10T06:53:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27220.623636 secs 2023-09-10T06:53:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:09 smithi098 ceph-mon[103124]: pgmap v14677: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 430 KiB/s wr, 31 op/s 2023-09-10T06:53:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27220.623636 secs 2023-09-10T06:53:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:09 smithi170 ceph-mon[102832]: pgmap v14677: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 430 KiB/s wr, 31 op/s 2023-09-10T06:53:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27221.101880 secs 2023-09-10T06:53:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27221.101880 secs 2023-09-10T06:53:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27221.101880 secs 2023-09-10T06:53:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:11 smithi082 ceph-mon[98694]: pgmap v14678: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 326 KiB/s wr, 26 op/s 2023-09-10T06:53:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:11 smithi082 ceph-mon[98694]: osdmap e986: 12 total, 12 up, 12 in 2023-09-10T06:53:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:11 smithi098 ceph-mon[103124]: pgmap v14678: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 326 KiB/s wr, 26 op/s 2023-09-10T06:53:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:11 smithi098 ceph-mon[103124]: osdmap e986: 12 total, 12 up, 12 in 2023-09-10T06:53:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:11 smithi170 ceph-mon[102832]: pgmap v14678: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 326 KiB/s wr, 26 op/s 2023-09-10T06:53:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:11 smithi170 ceph-mon[102832]: osdmap e986: 12 total, 12 up, 12 in 2023-09-10T06:53:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:13 smithi082 ceph-mon[98694]: pgmap v14680: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 368 KiB/s wr, 30 op/s 2023-09-10T06:53:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27224.483748 secs 2023-09-10T06:53:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:13 smithi098 ceph-mon[103124]: pgmap v14680: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 368 KiB/s wr, 30 op/s 2023-09-10T06:53:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27224.483748 secs 2023-09-10T06:53:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:13 smithi170 ceph-mon[102832]: pgmap v14680: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 368 KiB/s wr, 30 op/s 2023-09-10T06:53:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27224.483748 secs 2023-09-10T06:53:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27225.623698 secs 2023-09-10T06:53:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27225.623698 secs 2023-09-10T06:53:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27225.623698 secs 2023-09-10T06:53:15.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27226.101944 secs 2023-09-10T06:53:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:15 smithi082 ceph-mon[98694]: pgmap v14681: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 326 KiB/s wr, 26 op/s 2023-09-10T06:53:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27226.101944 secs 2023-09-10T06:53:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:15 smithi098 ceph-mon[103124]: pgmap v14681: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 326 KiB/s wr, 26 op/s 2023-09-10T06:53:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27226.101944 secs 2023-09-10T06:53:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:15 smithi170 ceph-mon[102832]: pgmap v14681: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 326 KiB/s wr, 26 op/s 2023-09-10T06:53:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:17 smithi098 ceph-mon[103124]: pgmap v14682: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T06:53:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:17 smithi170 ceph-mon[102832]: pgmap v14682: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T06:53:17.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:17 smithi082 ceph-mon[98694]: pgmap v14682: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T06:53:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:53:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:53:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:53:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27229.483814 secs 2023-09-10T06:53:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27230.623757 secs 2023-09-10T06:53:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:19 smithi098 ceph-mon[103124]: pgmap v14683: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T06:53:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27229.483814 secs 2023-09-10T06:53:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27230.623757 secs 2023-09-10T06:53:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:19 smithi170 ceph-mon[102832]: pgmap v14683: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T06:53:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27229.483814 secs 2023-09-10T06:53:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27230.623757 secs 2023-09-10T06:53:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:19 smithi082 ceph-mon[98694]: pgmap v14683: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 12 op/s 2023-09-10T06:53:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27231.102008 secs 2023-09-10T06:53:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:53:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27231.102008 secs 2023-09-10T06:53:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:53:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:20.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27231.102008 secs 2023-09-10T06:53:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:53:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:21 smithi098 ceph-mon[103124]: pgmap v14684: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 219 KiB/s wr, 12 op/s 2023-09-10T06:53:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:21 smithi170 ceph-mon[102832]: pgmap v14684: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 219 KiB/s wr, 12 op/s 2023-09-10T06:53:21.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:21 smithi082 ceph-mon[98694]: pgmap v14684: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 219 KiB/s wr, 12 op/s 2023-09-10T06:53:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:23 smithi082 ceph-mon[98694]: pgmap v14685: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 11 op/s 2023-09-10T06:53:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27234.483982 secs 2023-09-10T06:53:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:23 smithi098 ceph-mon[103124]: pgmap v14685: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 11 op/s 2023-09-10T06:53:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27234.483982 secs 2023-09-10T06:53:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:23 smithi170 ceph-mon[102832]: pgmap v14685: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 11 op/s 2023-09-10T06:53:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27234.483982 secs 2023-09-10T06:53:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27235.623816 secs 2023-09-10T06:53:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27235.623816 secs 2023-09-10T06:53:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27235.623816 secs 2023-09-10T06:53:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27236.102072 secs 2023-09-10T06:53:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:25 smithi082 ceph-mon[98694]: pgmap v14686: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 182 KiB/s wr, 10 op/s 2023-09-10T06:53:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27236.102072 secs 2023-09-10T06:53:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:25 smithi098 ceph-mon[103124]: pgmap v14686: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 182 KiB/s wr, 10 op/s 2023-09-10T06:53:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27236.102072 secs 2023-09-10T06:53:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:25 smithi170 ceph-mon[102832]: pgmap v14686: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 182 KiB/s wr, 10 op/s 2023-09-10T06:53:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:53:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:53:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:53:27.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:27 smithi082 ceph-mon[98694]: pgmap v14687: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T06:53:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:53:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:53:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:53:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:27 smithi098 ceph-mon[103124]: pgmap v14687: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T06:53:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:53:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:53:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:53:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:27 smithi170 ceph-mon[102832]: pgmap v14687: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T06:53:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:53:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:53:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:53:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27240.623901 secs 2023-09-10T06:53:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:29 smithi082 ceph-mon[98694]: pgmap v14688: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 9 op/s 2023-09-10T06:53:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27240.623901 secs 2023-09-10T06:53:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:29 smithi098 ceph-mon[103124]: pgmap v14688: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 9 op/s 2023-09-10T06:53:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27240.623901 secs 2023-09-10T06:53:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:29 smithi170 ceph-mon[102832]: pgmap v14688: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 9 op/s 2023-09-10T06:53:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27241.102136 secs 2023-09-10T06:53:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27241.102136 secs 2023-09-10T06:53:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27241.102136 secs 2023-09-10T06:53:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:31 smithi098 ceph-mon[103124]: pgmap v14689: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 279 KiB/s wr, 14 op/s 2023-09-10T06:53:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:53:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:31 smithi170 ceph-mon[102832]: pgmap v14689: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 279 KiB/s wr, 14 op/s 2023-09-10T06:53:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:53:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:31 smithi082 ceph-mon[98694]: pgmap v14689: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 279 KiB/s wr, 14 op/s 2023-09-10T06:53:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:53:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:53:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:53:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:53:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:53:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:53:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:53:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:33 smithi098 ceph-mon[103124]: pgmap v14690: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 184 KiB/s wr, 9 op/s 2023-09-10T06:53:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:33 smithi170 ceph-mon[102832]: pgmap v14690: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 184 KiB/s wr, 9 op/s 2023-09-10T06:53:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:33 smithi082 ceph-mon[98694]: pgmap v14690: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 184 KiB/s wr, 9 op/s 2023-09-10T06:53:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27245.624003 secs 2023-09-10T06:53:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27245.624003 secs 2023-09-10T06:53:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27245.624003 secs 2023-09-10T06:53:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27246.102201 secs 2023-09-10T06:53:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:35 smithi098 ceph-mon[103124]: pgmap v14691: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 184 KiB/s wr, 9 op/s 2023-09-10T06:53:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27246.102201 secs 2023-09-10T06:53:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:35 smithi170 ceph-mon[102832]: pgmap v14691: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 184 KiB/s wr, 9 op/s 2023-09-10T06:53:35.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27246.102201 secs 2023-09-10T06:53:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:35 smithi082 ceph-mon[98694]: pgmap v14691: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 184 KiB/s wr, 9 op/s 2023-09-10T06:53:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:53:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:53:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:36.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:53:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:37 smithi098 ceph-mon[103124]: pgmap v14692: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 414 KiB/s wr, 16 op/s 2023-09-10T06:53:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:37 smithi170 ceph-mon[102832]: pgmap v14692: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 414 KiB/s wr, 16 op/s 2023-09-10T06:53:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:37 smithi082 ceph-mon[98694]: pgmap v14692: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 414 KiB/s wr, 16 op/s 2023-09-10T06:53:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:53:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:53:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:53:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27250.624062 secs 2023-09-10T06:53:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:39 smithi098 ceph-mon[103124]: pgmap v14693: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 414 KiB/s wr, 12 op/s 2023-09-10T06:53:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27250.624062 secs 2023-09-10T06:53:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:39 smithi170 ceph-mon[102832]: pgmap v14693: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 414 KiB/s wr, 12 op/s 2023-09-10T06:53:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27250.624062 secs 2023-09-10T06:53:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:39 smithi082 ceph-mon[98694]: pgmap v14693: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 414 KiB/s wr, 12 op/s 2023-09-10T06:53:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27251.102264 secs 2023-09-10T06:53:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27251.102264 secs 2023-09-10T06:53:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27251.102264 secs 2023-09-10T06:53:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:41 smithi170 ceph-mon[102832]: pgmap v14694: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 433 KiB/s wr, 18 op/s 2023-09-10T06:53:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:41 smithi082 ceph-mon[98694]: pgmap v14694: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 433 KiB/s wr, 18 op/s 2023-09-10T06:53:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:41 smithi098 ceph-mon[103124]: pgmap v14694: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 433 KiB/s wr, 18 op/s 2023-09-10T06:53:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:43 smithi170 ceph-mon[102832]: pgmap v14695: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 249 KiB/s wr, 12 op/s 2023-09-10T06:53:43.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:43 smithi082 ceph-mon[98694]: pgmap v14695: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 249 KiB/s wr, 12 op/s 2023-09-10T06:53:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:43 smithi098 ceph-mon[103124]: pgmap v14695: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 249 KiB/s wr, 12 op/s 2023-09-10T06:53:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27255.624145 secs 2023-09-10T06:53:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:53:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:44.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27255.624145 secs 2023-09-10T06:53:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:53:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27255.624145 secs 2023-09-10T06:53:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:53:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27256.102333 secs 2023-09-10T06:53:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:45 smithi170 ceph-mon[102832]: pgmap v14696: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 249 KiB/s wr, 12 op/s 2023-09-10T06:53:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27256.102333 secs 2023-09-10T06:53:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:45 smithi082 ceph-mon[98694]: pgmap v14696: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 249 KiB/s wr, 12 op/s 2023-09-10T06:53:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27256.102333 secs 2023-09-10T06:53:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:45 smithi098 ceph-mon[103124]: pgmap v14696: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 249 KiB/s wr, 12 op/s 2023-09-10T06:53:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:53:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:53:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:53:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:47 smithi170 ceph-mon[102832]: pgmap v14697: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 761 KiB/s wr, 20 op/s 2023-09-10T06:53:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:47 smithi082 ceph-mon[98694]: pgmap v14697: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 761 KiB/s wr, 20 op/s 2023-09-10T06:53:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:47 smithi098 ceph-mon[103124]: pgmap v14697: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 761 KiB/s wr, 20 op/s 2023-09-10T06:53:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27260.624236 secs 2023-09-10T06:53:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:49 smithi170 ceph-mon[102832]: pgmap v14698: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 531 KiB/s wr, 13 op/s 2023-09-10T06:53:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27260.624236 secs 2023-09-10T06:53:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:49 smithi082 ceph-mon[98694]: pgmap v14698: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 531 KiB/s wr, 13 op/s 2023-09-10T06:53:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27260.624236 secs 2023-09-10T06:53:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:49 smithi098 ceph-mon[103124]: pgmap v14698: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 531 KiB/s wr, 13 op/s 2023-09-10T06:53:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27261.102400 secs 2023-09-10T06:53:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27261.102400 secs 2023-09-10T06:53:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27261.102400 secs 2023-09-10T06:53:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:51 smithi170 ceph-mon[102832]: pgmap v14699: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 671 KiB/s wr, 19 op/s 2023-09-10T06:53:51.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:51 smithi082 ceph-mon[98694]: pgmap v14699: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 671 KiB/s wr, 19 op/s 2023-09-10T06:53:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:51 smithi098 ceph-mon[103124]: pgmap v14699: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 671 KiB/s wr, 19 op/s 2023-09-10T06:53:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:53 smithi170 ceph-mon[102832]: pgmap v14700: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 651 KiB/s wr, 13 op/s 2023-09-10T06:53:53.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:53 smithi082 ceph-mon[98694]: pgmap v14700: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 651 KiB/s wr, 13 op/s 2023-09-10T06:53:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:53 smithi098 ceph-mon[103124]: pgmap v14700: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 651 KiB/s wr, 13 op/s 2023-09-10T06:53:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27265.624279 secs 2023-09-10T06:53:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:53:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27266.102466 secs 2023-09-10T06:53:54.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27265.624279 secs 2023-09-10T06:53:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:53:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27266.102466 secs 2023-09-10T06:53:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27265.624279 secs 2023-09-10T06:53:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:53:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27266.102466 secs 2023-09-10T06:53:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:55 smithi170 ceph-mon[102832]: pgmap v14701: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 651 KiB/s wr, 13 op/s 2023-09-10T06:53:55.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:55 smithi082 ceph-mon[98694]: pgmap v14701: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 651 KiB/s wr, 13 op/s 2023-09-10T06:53:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:55 smithi098 ceph-mon[103124]: pgmap v14701: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 651 KiB/s wr, 13 op/s 2023-09-10T06:53:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:53:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:56.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:53:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:53:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:53:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:57 smithi170 ceph-mon[102832]: pgmap v14702: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 734 KiB/s wr, 19 op/s 2023-09-10T06:53:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:57 smithi082 ceph-mon[98694]: pgmap v14702: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 734 KiB/s wr, 19 op/s 2023-09-10T06:53:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:57 smithi098 ceph-mon[103124]: pgmap v14702: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 734 KiB/s wr, 19 op/s 2023-09-10T06:53:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27269.484489 secs 2023-09-10T06:53:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27269.484489 secs 2023-09-10T06:53:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27269.484489 secs 2023-09-10T06:53:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27270.624357 secs 2023-09-10T06:53:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:59 smithi170 ceph-mon[102832]: pgmap v14703: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 222 KiB/s wr, 12 op/s 2023-09-10T06:53:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:53:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27271.102488 secs 2023-09-10T06:53:59.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27270.624357 secs 2023-09-10T06:53:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:59 smithi082 ceph-mon[98694]: pgmap v14703: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 222 KiB/s wr, 12 op/s 2023-09-10T06:53:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:53:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27271.102488 secs 2023-09-10T06:54:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27270.624357 secs 2023-09-10T06:54:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:59 smithi098 ceph-mon[103124]: pgmap v14703: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 222 KiB/s wr, 12 op/s 2023-09-10T06:54:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:53:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27271.102488 secs 2023-09-10T06:54:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:54:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:54:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:00.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:54:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:54:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:54:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:54:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:01 smithi170 ceph-mon[102832]: pgmap v14704: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 423 KiB/s wr, 18 op/s 2023-09-10T06:54:01.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:01 smithi082 ceph-mon[98694]: pgmap v14704: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 423 KiB/s wr, 18 op/s 2023-09-10T06:54:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:01 smithi098 ceph-mon[103124]: pgmap v14704: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 423 KiB/s wr, 18 op/s 2023-09-10T06:54:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:03 smithi170 ceph-mon[102832]: pgmap v14705: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 284 KiB/s wr, 11 op/s 2023-09-10T06:54:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27274.484547 secs 2023-09-10T06:54:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:03 smithi082 ceph-mon[98694]: pgmap v14705: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 284 KiB/s wr, 11 op/s 2023-09-10T06:54:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27274.484547 secs 2023-09-10T06:54:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:03 smithi098 ceph-mon[103124]: pgmap v14705: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 284 KiB/s wr, 11 op/s 2023-09-10T06:54:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27274.484547 secs 2023-09-10T06:54:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27275.624428 secs 2023-09-10T06:54:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:54:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27276.102553 secs 2023-09-10T06:54:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27275.624428 secs 2023-09-10T06:54:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:54:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27276.102553 secs 2023-09-10T06:54:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27275.624428 secs 2023-09-10T06:54:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:54:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27276.102553 secs 2023-09-10T06:54:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:05 smithi170 ceph-mon[102832]: pgmap v14706: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 284 KiB/s wr, 11 op/s 2023-09-10T06:54:05.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:05 smithi082 ceph-mon[98694]: pgmap v14706: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 284 KiB/s wr, 11 op/s 2023-09-10T06:54:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:05 smithi098 ceph-mon[103124]: pgmap v14706: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 284 KiB/s wr, 11 op/s 2023-09-10T06:54:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:54:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:06.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:54:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:54:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:07 smithi170 ceph-mon[102832]: pgmap v14707: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 424 KiB/s wr, 32 op/s 2023-09-10T06:54:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:07 smithi082 ceph-mon[98694]: pgmap v14707: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 424 KiB/s wr, 32 op/s 2023-09-10T06:54:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:07 smithi098 ceph-mon[103124]: pgmap v14707: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 424 KiB/s wr, 32 op/s 2023-09-10T06:54:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27279.484635 secs 2023-09-10T06:54:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27279.484635 secs 2023-09-10T06:54:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27279.484635 secs 2023-09-10T06:54:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27280.624488 secs 2023-09-10T06:54:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:09 smithi170 ceph-mon[102832]: pgmap v14708: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 341 KiB/s wr, 26 op/s 2023-09-10T06:54:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:09 smithi170 ceph-mon[102832]: osdmap e987: 12 total, 12 up, 12 in 2023-09-10T06:54:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27281.102622 secs 2023-09-10T06:54:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27280.624488 secs 2023-09-10T06:54:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:09 smithi082 ceph-mon[98694]: pgmap v14708: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 341 KiB/s wr, 26 op/s 2023-09-10T06:54:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:09 smithi082 ceph-mon[98694]: osdmap e987: 12 total, 12 up, 12 in 2023-09-10T06:54:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27281.102622 secs 2023-09-10T06:54:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27280.624488 secs 2023-09-10T06:54:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:09 smithi098 ceph-mon[103124]: pgmap v14708: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 341 KiB/s wr, 26 op/s 2023-09-10T06:54:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:09 smithi098 ceph-mon[103124]: osdmap e987: 12 total, 12 up, 12 in 2023-09-10T06:54:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27281.102622 secs 2023-09-10T06:54:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:11 smithi170 ceph-mon[102832]: pgmap v14710: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 192 KiB/s wr, 30 op/s 2023-09-10T06:54:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:11 smithi082 ceph-mon[98694]: pgmap v14710: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 192 KiB/s wr, 30 op/s 2023-09-10T06:54:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:11 smithi098 ceph-mon[103124]: pgmap v14710: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 192 KiB/s wr, 30 op/s 2023-09-10T06:54:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:13 smithi170 ceph-mon[102832]: pgmap v14711: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 192 KiB/s wr, 30 op/s 2023-09-10T06:54:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27284.484694 secs 2023-09-10T06:54:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:13 smithi082 ceph-mon[98694]: pgmap v14711: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 192 KiB/s wr, 30 op/s 2023-09-10T06:54:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27284.484694 secs 2023-09-10T06:54:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:13 smithi098 ceph-mon[103124]: pgmap v14711: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 192 KiB/s wr, 30 op/s 2023-09-10T06:54:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27284.484694 secs 2023-09-10T06:54:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27285.624557 secs 2023-09-10T06:54:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27286.102687 secs 2023-09-10T06:54:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27285.624557 secs 2023-09-10T06:54:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27286.102687 secs 2023-09-10T06:54:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27285.624557 secs 2023-09-10T06:54:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27286.102687 secs 2023-09-10T06:54:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:15 smithi170 ceph-mon[102832]: pgmap v14712: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 192 KiB/s wr, 30 op/s 2023-09-10T06:54:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:15 smithi082 ceph-mon[98694]: pgmap v14712: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 192 KiB/s wr, 30 op/s 2023-09-10T06:54:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:15 smithi098 ceph-mon[103124]: pgmap v14712: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 192 KiB/s wr, 30 op/s 2023-09-10T06:54:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:17 smithi170 ceph-mon[102832]: osdmap e988: 12 total, 12 up, 12 in 2023-09-10T06:54:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:17 smithi170 ceph-mon[102832]: pgmap v14714: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 173 KiB/s wr, 15 op/s 2023-09-10T06:54:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:17 smithi082 ceph-mon[98694]: osdmap e988: 12 total, 12 up, 12 in 2023-09-10T06:54:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:17 smithi082 ceph-mon[98694]: pgmap v14714: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 173 KiB/s wr, 15 op/s 2023-09-10T06:54:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:17 smithi098 ceph-mon[103124]: osdmap e988: 12 total, 12 up, 12 in 2023-09-10T06:54:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:17 smithi098 ceph-mon[103124]: pgmap v14714: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 173 KiB/s wr, 15 op/s 2023-09-10T06:54:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27289.484797 secs 2023-09-10T06:54:18.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27289.484797 secs 2023-09-10T06:54:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27289.484797 secs 2023-09-10T06:54:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27290.624642 secs 2023-09-10T06:54:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:19 smithi098 ceph-mon[103124]: pgmap v14715: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 12 op/s 2023-09-10T06:54:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27290.624642 secs 2023-09-10T06:54:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:19 smithi170 ceph-mon[102832]: pgmap v14715: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 12 op/s 2023-09-10T06:54:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27290.624642 secs 2023-09-10T06:54:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:19 smithi082 ceph-mon[98694]: pgmap v14715: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 12 op/s 2023-09-10T06:54:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27291.102753 secs 2023-09-10T06:54:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27291.102753 secs 2023-09-10T06:54:20.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27291.102753 secs 2023-09-10T06:54:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:21 smithi098 ceph-mon[103124]: pgmap v14716: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T06:54:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:21 smithi170 ceph-mon[102832]: pgmap v14716: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T06:54:21.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:21 smithi082 ceph-mon[98694]: pgmap v14716: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T06:54:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:23 smithi098 ceph-mon[103124]: pgmap v14717: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T06:54:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27294.484858 secs 2023-09-10T06:54:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:23 smithi170 ceph-mon[102832]: pgmap v14717: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T06:54:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27294.484858 secs 2023-09-10T06:54:23.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:23 smithi082 ceph-mon[98694]: pgmap v14717: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T06:54:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27294.484858 secs 2023-09-10T06:54:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27295.624705 secs 2023-09-10T06:54:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27295.624705 secs 2023-09-10T06:54:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27295.624705 secs 2023-09-10T06:54:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27296.102814 secs 2023-09-10T06:54:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:25 smithi098 ceph-mon[103124]: pgmap v14718: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T06:54:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27296.102814 secs 2023-09-10T06:54:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:25 smithi170 ceph-mon[102832]: pgmap v14718: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T06:54:25.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27296.102814 secs 2023-09-10T06:54:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:25 smithi082 ceph-mon[98694]: pgmap v14718: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T06:54:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:27 smithi170 ceph-mon[102832]: pgmap v14719: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T06:54:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:54:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:27 smithi082 ceph-mon[98694]: pgmap v14719: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T06:54:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:54:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:27 smithi098 ceph-mon[103124]: pgmap v14719: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T06:54:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:54:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27300.624769 secs 2023-09-10T06:54:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:29 smithi098 ceph-mon[103124]: pgmap v14720: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 9 op/s 2023-09-10T06:54:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27300.624769 secs 2023-09-10T06:54:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:29 smithi170 ceph-mon[102832]: pgmap v14720: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 9 op/s 2023-09-10T06:54:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27300.624769 secs 2023-09-10T06:54:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:29 smithi082 ceph-mon[98694]: pgmap v14720: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 9 op/s 2023-09-10T06:54:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27301.102877 secs 2023-09-10T06:54:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:30.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:30.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:54:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27301.102877 secs 2023-09-10T06:54:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:30.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:30.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:54:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27301.102877 secs 2023-09-10T06:54:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:30.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:30.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:30.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:30.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:30.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:54:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:31 smithi170 ceph-mon[102832]: pgmap v14721: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 233 KiB/s wr, 14 op/s 2023-09-10T06:54:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:54:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:54:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:54:31.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:54:31.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:31 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T06:54:31.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:31 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T06:54:31.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:31.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:31.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:54:31.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:54:31.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:54:31.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:54:31.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:31 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T06:54:31.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:31 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T06:54:31.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:54:31.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:54:31.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:31.932 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:54:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:31 smithi082 ceph-mon[98694]: pgmap v14721: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 233 KiB/s wr, 14 op/s 2023-09-10T06:54:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:54:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:54:31.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:54:31.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:54:31.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:31 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T06:54:31.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:31 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T06:54:31.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:31.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:31.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:54:31.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:54:31.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:54:31.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:54:31.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:31 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T06:54:31.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:31 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T06:54:31.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:54:31.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:54:31.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:31.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:54:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:31 smithi098 ceph-mon[103124]: pgmap v14721: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 233 KiB/s wr, 14 op/s 2023-09-10T06:54:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:54:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:54:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:54:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:54:32.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:31 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T06:54:32.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:31 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T06:54:32.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:32.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:32.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:54:32.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:54:32.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:54:32.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T06:54:32.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:31 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T06:54:32.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:31 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T06:54:32.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:54:32.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:54:32.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:54:32.008 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:54:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:54:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:54:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:54:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:54:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:54:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:54:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:33 smithi170 ceph-mon[102832]: pgmap v14722: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 232 KiB/s wr, 10 op/s 2023-09-10T06:54:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:54:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:33 smithi082 ceph-mon[98694]: pgmap v14722: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 232 KiB/s wr, 10 op/s 2023-09-10T06:54:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:54:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:33 smithi098 ceph-mon[103124]: pgmap v14722: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 232 KiB/s wr, 10 op/s 2023-09-10T06:54:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:54:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27305.624826 secs 2023-09-10T06:54:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27306.102944 secs 2023-09-10T06:54:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27305.624826 secs 2023-09-10T06:54:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27306.102944 secs 2023-09-10T06:54:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27305.624826 secs 2023-09-10T06:54:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27306.102944 secs 2023-09-10T06:54:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:35 smithi170 ceph-mon[102832]: pgmap v14723: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 232 KiB/s wr, 10 op/s 2023-09-10T06:54:36.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:35 smithi082 ceph-mon[98694]: pgmap v14723: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 232 KiB/s wr, 10 op/s 2023-09-10T06:54:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:35 smithi098 ceph-mon[103124]: pgmap v14723: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 232 KiB/s wr, 10 op/s 2023-09-10T06:54:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:54:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:54:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:54:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:37 smithi170 ceph-mon[102832]: pgmap v14724: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 347 KiB/s wr, 15 op/s 2023-09-10T06:54:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:54:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:38.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:37 smithi082 ceph-mon[98694]: pgmap v14724: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 347 KiB/s wr, 15 op/s 2023-09-10T06:54:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:54:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:37 smithi098 ceph-mon[103124]: pgmap v14724: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 347 KiB/s wr, 15 op/s 2023-09-10T06:54:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:54:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27310.624886 secs 2023-09-10T06:54:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:39 smithi170 ceph-mon[102832]: pgmap v14725: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 235 KiB/s wr, 11 op/s 2023-09-10T06:54:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27311.103009 secs 2023-09-10T06:54:40.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27310.624886 secs 2023-09-10T06:54:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:39 smithi082 ceph-mon[98694]: pgmap v14725: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 235 KiB/s wr, 11 op/s 2023-09-10T06:54:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27311.103009 secs 2023-09-10T06:54:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27310.624886 secs 2023-09-10T06:54:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:39 smithi098 ceph-mon[103124]: pgmap v14725: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 235 KiB/s wr, 11 op/s 2023-09-10T06:54:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27311.103009 secs 2023-09-10T06:54:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:54:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:41.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:54:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:54:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:41 smithi170 ceph-mon[102832]: pgmap v14726: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 687 KiB/s wr, 19 op/s 2023-09-10T06:54:42.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:41 smithi082 ceph-mon[98694]: pgmap v14726: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 687 KiB/s wr, 19 op/s 2023-09-10T06:54:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:41 smithi098 ceph-mon[103124]: pgmap v14726: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 687 KiB/s wr, 19 op/s 2023-09-10T06:54:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:43 smithi170 ceph-mon[102832]: pgmap v14727: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 567 KiB/s wr, 13 op/s 2023-09-10T06:54:44.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:43 smithi082 ceph-mon[98694]: pgmap v14727: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 567 KiB/s wr, 13 op/s 2023-09-10T06:54:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:43 smithi098 ceph-mon[103124]: pgmap v14727: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 567 KiB/s wr, 13 op/s 2023-09-10T06:54:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27315.624946 secs 2023-09-10T06:54:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:54:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27316.103071 secs 2023-09-10T06:54:45.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27315.624946 secs 2023-09-10T06:54:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:54:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27316.103071 secs 2023-09-10T06:54:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27315.624946 secs 2023-09-10T06:54:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:54:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27316.103071 secs 2023-09-10T06:54:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:45 smithi170 ceph-mon[102832]: pgmap v14728: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 567 KiB/s wr, 13 op/s 2023-09-10T06:54:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:54:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:46.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:45 smithi082 ceph-mon[98694]: pgmap v14728: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 567 KiB/s wr, 13 op/s 2023-09-10T06:54:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:54:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:45 smithi098 ceph-mon[103124]: pgmap v14728: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 567 KiB/s wr, 13 op/s 2023-09-10T06:54:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:54:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:47 smithi170 ceph-mon[102832]: pgmap v14729: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 820 KiB/s wr, 22 op/s 2023-09-10T06:54:47.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:47 smithi082 ceph-mon[98694]: pgmap v14729: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 820 KiB/s wr, 22 op/s 2023-09-10T06:54:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:47 smithi098 ceph-mon[103124]: pgmap v14729: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 820 KiB/s wr, 22 op/s 2023-09-10T06:54:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27319.485278 secs 2023-09-10T06:54:48.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27319.485278 secs 2023-09-10T06:54:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27319.485278 secs 2023-09-10T06:54:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27320.625042 secs 2023-09-10T06:54:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:49 smithi098 ceph-mon[103124]: pgmap v14730: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 706 KiB/s wr, 16 op/s 2023-09-10T06:54:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27320.625042 secs 2023-09-10T06:54:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:49 smithi170 ceph-mon[102832]: pgmap v14730: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 706 KiB/s wr, 16 op/s 2023-09-10T06:54:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27320.625042 secs 2023-09-10T06:54:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:49 smithi082 ceph-mon[98694]: pgmap v14730: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 706 KiB/s wr, 16 op/s 2023-09-10T06:54:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27321.103137 secs 2023-09-10T06:54:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27321.103137 secs 2023-09-10T06:54:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27321.103137 secs 2023-09-10T06:54:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:51 smithi098 ceph-mon[103124]: pgmap v14731: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 725 KiB/s wr, 22 op/s 2023-09-10T06:54:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:51 smithi170 ceph-mon[102832]: pgmap v14731: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 725 KiB/s wr, 22 op/s 2023-09-10T06:54:51.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:51 smithi082 ceph-mon[98694]: pgmap v14731: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 725 KiB/s wr, 22 op/s 2023-09-10T06:54:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:54:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:54:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:54:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:54:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:52.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:54:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:54:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:54:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:53 smithi098 ceph-mon[103124]: pgmap v14732: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 272 KiB/s wr, 14 op/s 2023-09-10T06:54:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27324.485341 secs 2023-09-10T06:54:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:53 smithi170 ceph-mon[102832]: pgmap v14732: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 272 KiB/s wr, 14 op/s 2023-09-10T06:54:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27324.485341 secs 2023-09-10T06:54:53.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:53 smithi082 ceph-mon[98694]: pgmap v14732: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 272 KiB/s wr, 14 op/s 2023-09-10T06:54:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27324.485341 secs 2023-09-10T06:54:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27325.625101 secs 2023-09-10T06:54:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27325.625101 secs 2023-09-10T06:54:54.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27325.625101 secs 2023-09-10T06:54:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27326.103199 secs 2023-09-10T06:54:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:55 smithi098 ceph-mon[103124]: pgmap v14733: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 272 KiB/s wr, 14 op/s 2023-09-10T06:54:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27326.103199 secs 2023-09-10T06:54:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:55 smithi170 ceph-mon[102832]: pgmap v14733: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 272 KiB/s wr, 14 op/s 2023-09-10T06:54:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27326.103199 secs 2023-09-10T06:54:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:55 smithi082 ceph-mon[98694]: pgmap v14733: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 272 KiB/s wr, 14 op/s 2023-09-10T06:54:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:57 smithi170 ceph-mon[102832]: pgmap v14734: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 460 KiB/s wr, 19 op/s 2023-09-10T06:54:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:57 smithi082 ceph-mon[98694]: pgmap v14734: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 460 KiB/s wr, 19 op/s 2023-09-10T06:54:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:57 smithi098 ceph-mon[103124]: pgmap v14734: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 460 KiB/s wr, 19 op/s 2023-09-10T06:54:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27329.485418 secs 2023-09-10T06:54:58.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27329.485418 secs 2023-09-10T06:54:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27329.485418 secs 2023-09-10T06:54:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27330.625163 secs 2023-09-10T06:54:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:54:59 smithi098 ceph-mon[103124]: pgmap v14735: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 206 KiB/s wr, 11 op/s 2023-09-10T06:54:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27330.625163 secs 2023-09-10T06:54:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:54:59 smithi170 ceph-mon[102832]: pgmap v14735: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 206 KiB/s wr, 11 op/s 2023-09-10T06:54:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27330.625163 secs 2023-09-10T06:54:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:54:59 smithi082 ceph-mon[98694]: pgmap v14735: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 206 KiB/s wr, 11 op/s 2023-09-10T06:55:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27331.103265 secs 2023-09-10T06:55:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27331.103265 secs 2023-09-10T06:55:00.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27331.103265 secs 2023-09-10T06:55:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:01 smithi098 ceph-mon[103124]: pgmap v14736: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 206 KiB/s wr, 15 op/s 2023-09-10T06:55:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:01 smithi170 ceph-mon[102832]: pgmap v14736: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 206 KiB/s wr, 15 op/s 2023-09-10T06:55:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:01 smithi082 ceph-mon[98694]: pgmap v14736: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 206 KiB/s wr, 15 op/s 2023-09-10T06:55:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:03 smithi098 ceph-mon[103124]: pgmap v14737: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 188 KiB/s wr, 9 op/s 2023-09-10T06:55:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27334.485497 secs 2023-09-10T06:55:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:03 smithi170 ceph-mon[102832]: pgmap v14737: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 188 KiB/s wr, 9 op/s 2023-09-10T06:55:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27334.485497 secs 2023-09-10T06:55:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:03 smithi082 ceph-mon[98694]: pgmap v14737: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 188 KiB/s wr, 9 op/s 2023-09-10T06:55:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27334.485497 secs 2023-09-10T06:55:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27335.625218 secs 2023-09-10T06:55:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27335.625218 secs 2023-09-10T06:55:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27335.625218 secs 2023-09-10T06:55:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27336.103324 secs 2023-09-10T06:55:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:05 smithi098 ceph-mon[103124]: pgmap v14738: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 188 KiB/s wr, 9 op/s 2023-09-10T06:55:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27336.103324 secs 2023-09-10T06:55:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:05 smithi170 ceph-mon[102832]: pgmap v14738: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 188 KiB/s wr, 9 op/s 2023-09-10T06:55:05.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27336.103324 secs 2023-09-10T06:55:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:05 smithi082 ceph-mon[98694]: pgmap v14738: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 188 KiB/s wr, 9 op/s 2023-09-10T06:55:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:07 smithi170 ceph-mon[102832]: pgmap v14739: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 233 KiB/s wr, 28 op/s 2023-09-10T06:55:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:07 smithi082 ceph-mon[98694]: pgmap v14739: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 233 KiB/s wr, 28 op/s 2023-09-10T06:55:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:07 smithi098 ceph-mon[103124]: pgmap v14739: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 233 KiB/s wr, 28 op/s 2023-09-10T06:55:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27339.485558 secs 2023-09-10T06:55:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:09 smithi170 ceph-mon[102832]: osdmap e989: 12 total, 12 up, 12 in 2023-09-10T06:55:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27340.625277 secs 2023-09-10T06:55:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:09 smithi170 ceph-mon[102832]: pgmap v14741: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T06:55:09.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27339.485558 secs 2023-09-10T06:55:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:09 smithi082 ceph-mon[98694]: osdmap e989: 12 total, 12 up, 12 in 2023-09-10T06:55:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27340.625277 secs 2023-09-10T06:55:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:09 smithi082 ceph-mon[98694]: pgmap v14741: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T06:55:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27339.485558 secs 2023-09-10T06:55:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:09 smithi098 ceph-mon[103124]: osdmap e989: 12 total, 12 up, 12 in 2023-09-10T06:55:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27340.625277 secs 2023-09-10T06:55:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:09 smithi098 ceph-mon[103124]: pgmap v14741: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T06:55:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27341.103388 secs 2023-09-10T06:55:10.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27341.103388 secs 2023-09-10T06:55:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27341.103388 secs 2023-09-10T06:55:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:11 smithi170 ceph-mon[102832]: pgmap v14742: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T06:55:11.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:11 smithi082 ceph-mon[98694]: pgmap v14742: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T06:55:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:11 smithi098 ceph-mon[103124]: pgmap v14742: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T06:55:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:13 smithi098 ceph-mon[103124]: pgmap v14743: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T06:55:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27344.485656 secs 2023-09-10T06:55:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:13 smithi170 ceph-mon[102832]: pgmap v14743: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T06:55:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27344.485656 secs 2023-09-10T06:55:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:13 smithi082 ceph-mon[98694]: pgmap v14743: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T06:55:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27344.485656 secs 2023-09-10T06:55:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27345.625356 secs 2023-09-10T06:55:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27345.625356 secs 2023-09-10T06:55:14.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27345.625356 secs 2023-09-10T06:55:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27346.103456 secs 2023-09-10T06:55:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:15 smithi098 ceph-mon[103124]: pgmap v14744: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T06:55:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27346.103456 secs 2023-09-10T06:55:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:15 smithi170 ceph-mon[102832]: pgmap v14744: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T06:55:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27346.103456 secs 2023-09-10T06:55:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:15 smithi082 ceph-mon[98694]: pgmap v14744: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T06:55:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:17 smithi170 ceph-mon[102832]: osdmap e990: 12 total, 12 up, 12 in 2023-09-10T06:55:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:17 smithi170 ceph-mon[102832]: pgmap v14746: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 130 KiB/s wr, 13 op/s 2023-09-10T06:55:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:17 smithi082 ceph-mon[98694]: osdmap e990: 12 total, 12 up, 12 in 2023-09-10T06:55:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:17 smithi082 ceph-mon[98694]: pgmap v14746: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 130 KiB/s wr, 13 op/s 2023-09-10T06:55:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:17 smithi098 ceph-mon[103124]: osdmap e990: 12 total, 12 up, 12 in 2023-09-10T06:55:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:17 smithi098 ceph-mon[103124]: pgmap v14746: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 130 KiB/s wr, 13 op/s 2023-09-10T06:55:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27349.485714 secs 2023-09-10T06:55:18.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27349.485714 secs 2023-09-10T06:55:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27349.485714 secs 2023-09-10T06:55:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27350.625420 secs 2023-09-10T06:55:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:19 smithi170 ceph-mon[102832]: pgmap v14747: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T06:55:19.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27350.625420 secs 2023-09-10T06:55:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:19 smithi082 ceph-mon[98694]: pgmap v14747: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T06:55:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27350.625420 secs 2023-09-10T06:55:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:19 smithi098 ceph-mon[103124]: pgmap v14747: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T06:55:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27351.103491 secs 2023-09-10T06:55:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27351.103491 secs 2023-09-10T06:55:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27351.103491 secs 2023-09-10T06:55:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:21 smithi082 ceph-mon[98694]: pgmap v14748: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 107 KiB/s wr, 12 op/s 2023-09-10T06:55:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:21 smithi098 ceph-mon[103124]: pgmap v14748: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 107 KiB/s wr, 12 op/s 2023-09-10T06:55:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:21 smithi170 ceph-mon[102832]: pgmap v14748: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 107 KiB/s wr, 12 op/s 2023-09-10T06:55:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:23 smithi082 ceph-mon[98694]: pgmap v14749: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 107 KiB/s wr, 12 op/s 2023-09-10T06:55:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27354.485775 secs 2023-09-10T06:55:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:23 smithi098 ceph-mon[103124]: pgmap v14749: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 107 KiB/s wr, 12 op/s 2023-09-10T06:55:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27354.485775 secs 2023-09-10T06:55:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:23 smithi170 ceph-mon[102832]: pgmap v14749: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 107 KiB/s wr, 12 op/s 2023-09-10T06:55:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27354.485775 secs 2023-09-10T06:55:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27355.625478 secs 2023-09-10T06:55:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:55:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:55:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27355.625478 secs 2023-09-10T06:55:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:55:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:55:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27355.625478 secs 2023-09-10T06:55:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:55:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:55:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27356.103553 secs 2023-09-10T06:55:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:25 smithi082 ceph-mon[98694]: pgmap v14750: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 107 KiB/s wr, 12 op/s 2023-09-10T06:55:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27356.103553 secs 2023-09-10T06:55:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:25 smithi098 ceph-mon[103124]: pgmap v14750: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 107 KiB/s wr, 12 op/s 2023-09-10T06:55:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27356.103553 secs 2023-09-10T06:55:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:25 smithi170 ceph-mon[102832]: pgmap v14750: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 107 KiB/s wr, 12 op/s 2023-09-10T06:55:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:26 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:55:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:55:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:26 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:55:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:55:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:26 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:55:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:55:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:27 smithi098 ceph-mon[103124]: pgmap v14751: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 12 op/s 2023-09-10T06:55:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:27 smithi170 ceph-mon[102832]: pgmap v14751: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 12 op/s 2023-09-10T06:55:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:27 smithi082 ceph-mon[98694]: pgmap v14751: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 12 op/s 2023-09-10T06:55:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27360.625497 secs 2023-09-10T06:55:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:29 smithi098 ceph-mon[103124]: pgmap v14752: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T06:55:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27360.625497 secs 2023-09-10T06:55:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:29 smithi170 ceph-mon[102832]: pgmap v14752: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T06:55:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27360.625497 secs 2023-09-10T06:55:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:29 smithi082 ceph-mon[98694]: pgmap v14752: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T06:55:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27361.103621 secs 2023-09-10T06:55:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27361.103621 secs 2023-09-10T06:55:30.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27361.103621 secs 2023-09-10T06:55:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:31 smithi098 ceph-mon[103124]: pgmap v14753: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 234 KiB/s wr, 15 op/s 2023-09-10T06:55:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:55:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:55:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:31 smithi170 ceph-mon[102832]: pgmap v14753: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 234 KiB/s wr, 15 op/s 2023-09-10T06:55:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:55:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:55:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:31 smithi082 ceph-mon[98694]: pgmap v14753: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 234 KiB/s wr, 15 op/s 2023-09-10T06:55:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:55:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:55:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:55:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:55:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:55:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:55:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:55:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:55:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:55:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:55:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:55:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:55:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:55:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:55:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:55:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:55:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:55:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:55:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:55:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:55:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:33 smithi170 ceph-mon[102832]: pgmap v14754: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 233 KiB/s wr, 10 op/s 2023-09-10T06:55:33.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:33 smithi082 ceph-mon[98694]: pgmap v14754: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 233 KiB/s wr, 10 op/s 2023-09-10T06:55:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:33 smithi098 ceph-mon[103124]: pgmap v14754: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 233 KiB/s wr, 10 op/s 2023-09-10T06:55:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27365.625556 secs 2023-09-10T06:55:34.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27365.625556 secs 2023-09-10T06:55:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27365.625556 secs 2023-09-10T06:55:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27366.103681 secs 2023-09-10T06:55:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:35 smithi170 ceph-mon[102832]: pgmap v14755: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 233 KiB/s wr, 10 op/s 2023-09-10T06:55:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27366.103681 secs 2023-09-10T06:55:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:35 smithi082 ceph-mon[98694]: pgmap v14755: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 233 KiB/s wr, 10 op/s 2023-09-10T06:55:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27366.103681 secs 2023-09-10T06:55:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:35 smithi098 ceph-mon[103124]: pgmap v14755: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 233 KiB/s wr, 10 op/s 2023-09-10T06:55:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:37 smithi170 ceph-mon[102832]: pgmap v14756: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 346 KiB/s wr, 16 op/s 2023-09-10T06:55:37.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:37 smithi082 ceph-mon[98694]: pgmap v14756: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 346 KiB/s wr, 16 op/s 2023-09-10T06:55:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:37 smithi098 ceph-mon[103124]: pgmap v14756: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 346 KiB/s wr, 16 op/s 2023-09-10T06:55:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:55:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:55:38.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:55:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:55:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:55:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:55:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27370.625612 secs 2023-09-10T06:55:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:39 smithi170 ceph-mon[102832]: pgmap v14757: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 234 KiB/s wr, 11 op/s 2023-09-10T06:55:39.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27370.625612 secs 2023-09-10T06:55:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:39 smithi082 ceph-mon[98694]: pgmap v14757: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 234 KiB/s wr, 11 op/s 2023-09-10T06:55:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27370.625612 secs 2023-09-10T06:55:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:39 smithi098 ceph-mon[103124]: pgmap v14757: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 234 KiB/s wr, 11 op/s 2023-09-10T06:55:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27371.103746 secs 2023-09-10T06:55:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:55:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:55:40.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27371.103746 secs 2023-09-10T06:55:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:55:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:55:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27371.103746 secs 2023-09-10T06:55:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:55:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:55:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:41 smithi170 ceph-mon[102832]: pgmap v14758: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 386 KiB/s wr, 18 op/s 2023-09-10T06:55:41.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:41 smithi082 ceph-mon[98694]: pgmap v14758: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 386 KiB/s wr, 18 op/s 2023-09-10T06:55:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:41 smithi098 ceph-mon[103124]: pgmap v14758: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 386 KiB/s wr, 18 op/s 2023-09-10T06:55:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:43 smithi098 ceph-mon[103124]: pgmap v14759: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 13 op/s 2023-09-10T06:55:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:43 smithi170 ceph-mon[102832]: pgmap v14759: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 13 op/s 2023-09-10T06:55:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:43 smithi082 ceph-mon[98694]: pgmap v14759: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 13 op/s 2023-09-10T06:55:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27375.625671 secs 2023-09-10T06:55:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27375.625671 secs 2023-09-10T06:55:44.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27375.625671 secs 2023-09-10T06:55:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27376.103810 secs 2023-09-10T06:55:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:45 smithi098 ceph-mon[103124]: pgmap v14760: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 13 op/s 2023-09-10T06:55:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27376.103810 secs 2023-09-10T06:55:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:45 smithi170 ceph-mon[102832]: pgmap v14760: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 13 op/s 2023-09-10T06:55:45.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27376.103810 secs 2023-09-10T06:55:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:45 smithi082 ceph-mon[98694]: pgmap v14760: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 13 op/s 2023-09-10T06:55:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:47 smithi170 ceph-mon[102832]: pgmap v14761: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 510 KiB/s wr, 20 op/s 2023-09-10T06:55:47.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:47 smithi082 ceph-mon[98694]: pgmap v14761: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 510 KiB/s wr, 20 op/s 2023-09-10T06:55:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:47 smithi098 ceph-mon[103124]: pgmap v14761: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 510 KiB/s wr, 20 op/s 2023-09-10T06:55:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27380.625740 secs 2023-09-10T06:55:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:49 smithi098 ceph-mon[103124]: pgmap v14762: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 397 KiB/s wr, 14 op/s 2023-09-10T06:55:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27380.625740 secs 2023-09-10T06:55:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:49 smithi170 ceph-mon[102832]: pgmap v14762: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 397 KiB/s wr, 14 op/s 2023-09-10T06:55:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27380.625740 secs 2023-09-10T06:55:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:49 smithi082 ceph-mon[98694]: pgmap v14762: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 397 KiB/s wr, 14 op/s 2023-09-10T06:55:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27381.103879 secs 2023-09-10T06:55:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:55:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:55:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27381.103879 secs 2023-09-10T06:55:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:55:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:55:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27381.103879 secs 2023-09-10T06:55:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:55:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:55:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:51 smithi170 ceph-mon[102832]: pgmap v14763: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 501 KiB/s wr, 20 op/s 2023-09-10T06:55:51.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:51 smithi082 ceph-mon[98694]: pgmap v14763: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 501 KiB/s wr, 20 op/s 2023-09-10T06:55:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:51 smithi098 ceph-mon[103124]: pgmap v14763: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 501 KiB/s wr, 20 op/s 2023-09-10T06:55:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:55:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:55:52.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:55:52.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:55:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:55:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:55:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:53 smithi170 ceph-mon[102832]: pgmap v14764: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 349 KiB/s wr, 13 op/s 2023-09-10T06:55:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27384.486162 secs 2023-09-10T06:55:53.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:53 smithi082 ceph-mon[98694]: pgmap v14764: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 349 KiB/s wr, 13 op/s 2023-09-10T06:55:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27384.486162 secs 2023-09-10T06:55:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:53 smithi098 ceph-mon[103124]: pgmap v14764: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 349 KiB/s wr, 13 op/s 2023-09-10T06:55:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27384.486162 secs 2023-09-10T06:55:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27385.625799 secs 2023-09-10T06:55:54.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27385.625799 secs 2023-09-10T06:55:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27385.625799 secs 2023-09-10T06:55:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27386.103942 secs 2023-09-10T06:55:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:55 smithi170 ceph-mon[102832]: pgmap v14765: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 349 KiB/s wr, 13 op/s 2023-09-10T06:55:55.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27386.103942 secs 2023-09-10T06:55:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:55 smithi082 ceph-mon[98694]: pgmap v14765: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 349 KiB/s wr, 13 op/s 2023-09-10T06:55:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27386.103942 secs 2023-09-10T06:55:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:55 smithi098 ceph-mon[103124]: pgmap v14765: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 349 KiB/s wr, 13 op/s 2023-09-10T06:55:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:55:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:55:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:55:56.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:55:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:55:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:55:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:55:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:55:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:55:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:57 smithi170 ceph-mon[102832]: pgmap v14766: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 567 KiB/s wr, 19 op/s 2023-09-10T06:55:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:57 smithi082 ceph-mon[98694]: pgmap v14766: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 567 KiB/s wr, 19 op/s 2023-09-10T06:55:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:57 smithi098 ceph-mon[103124]: pgmap v14766: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 567 KiB/s wr, 19 op/s 2023-09-10T06:55:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27389.486271 secs 2023-09-10T06:55:58.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27389.486271 secs 2023-09-10T06:55:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27389.486271 secs 2023-09-10T06:55:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27390.625857 secs 2023-09-10T06:55:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:55:59 smithi170 ceph-mon[102832]: pgmap v14767: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 322 KiB/s wr, 12 op/s 2023-09-10T06:55:59.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27390.625857 secs 2023-09-10T06:55:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:55:59 smithi082 ceph-mon[98694]: pgmap v14767: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 322 KiB/s wr, 12 op/s 2023-09-10T06:56:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27390.625857 secs 2023-09-10T06:56:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:55:59 smithi098 ceph-mon[103124]: pgmap v14767: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 322 KiB/s wr, 12 op/s 2023-09-10T06:56:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27391.104006 secs 2023-09-10T06:56:00.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27391.104006 secs 2023-09-10T06:56:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27391.104006 secs 2023-09-10T06:56:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:01 smithi170 ceph-mon[102832]: pgmap v14768: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 396 KiB/s wr, 17 op/s 2023-09-10T06:56:01.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:01 smithi082 ceph-mon[98694]: pgmap v14768: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 396 KiB/s wr, 17 op/s 2023-09-10T06:56:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:01 smithi098 ceph-mon[103124]: pgmap v14768: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 396 KiB/s wr, 17 op/s 2023-09-10T06:56:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:03 smithi170 ceph-mon[102832]: pgmap v14769: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 292 KiB/s wr, 11 op/s 2023-09-10T06:56:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27394.486332 secs 2023-09-10T06:56:03.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:03 smithi082 ceph-mon[98694]: pgmap v14769: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 292 KiB/s wr, 11 op/s 2023-09-10T06:56:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27394.486332 secs 2023-09-10T06:56:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:03 smithi098 ceph-mon[103124]: pgmap v14769: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 292 KiB/s wr, 11 op/s 2023-09-10T06:56:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27394.486332 secs 2023-09-10T06:56:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27395.625950 secs 2023-09-10T06:56:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:56:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:56:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27395.625950 secs 2023-09-10T06:56:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:56:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:56:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27395.625950 secs 2023-09-10T06:56:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:56:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:56:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27396.104076 secs 2023-09-10T06:56:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:05 smithi170 ceph-mon[102832]: pgmap v14770: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 292 KiB/s wr, 11 op/s 2023-09-10T06:56:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27396.104076 secs 2023-09-10T06:56:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:05 smithi082 ceph-mon[98694]: pgmap v14770: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 292 KiB/s wr, 11 op/s 2023-09-10T06:56:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27396.104076 secs 2023-09-10T06:56:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:05 smithi098 ceph-mon[103124]: pgmap v14770: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 292 KiB/s wr, 11 op/s 2023-09-10T06:56:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:56:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:56:06.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:56:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:56:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:56:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:56:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:07 smithi170 ceph-mon[102832]: pgmap v14771: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 337 KiB/s wr, 30 op/s 2023-09-10T06:56:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:07 smithi082 ceph-mon[98694]: pgmap v14771: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 337 KiB/s wr, 30 op/s 2023-09-10T06:56:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:07 smithi098 ceph-mon[103124]: pgmap v14771: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 337 KiB/s wr, 30 op/s 2023-09-10T06:56:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27399.486390 secs 2023-09-10T06:56:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27399.486390 secs 2023-09-10T06:56:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27399.486390 secs 2023-09-10T06:56:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27400.626038 secs 2023-09-10T06:56:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:09 smithi170 ceph-mon[102832]: pgmap v14772: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 119 KiB/s wr, 24 op/s 2023-09-10T06:56:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:09 smithi170 ceph-mon[102832]: osdmap e991: 12 total, 12 up, 12 in 2023-09-10T06:56:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27401.104136 secs 2023-09-10T06:56:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27400.626038 secs 2023-09-10T06:56:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:09 smithi082 ceph-mon[98694]: pgmap v14772: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 119 KiB/s wr, 24 op/s 2023-09-10T06:56:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:09 smithi082 ceph-mon[98694]: osdmap e991: 12 total, 12 up, 12 in 2023-09-10T06:56:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27401.104136 secs 2023-09-10T06:56:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27400.626038 secs 2023-09-10T06:56:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:09 smithi098 ceph-mon[103124]: pgmap v14772: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 119 KiB/s wr, 24 op/s 2023-09-10T06:56:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:09 smithi098 ceph-mon[103124]: osdmap e991: 12 total, 12 up, 12 in 2023-09-10T06:56:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27401.104136 secs 2023-09-10T06:56:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:11 smithi170 ceph-mon[102832]: pgmap v14774: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T06:56:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:11 smithi170 ceph-mon[102832]: osdmap e992: 12 total, 12 up, 12 in 2023-09-10T06:56:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:11 smithi082 ceph-mon[98694]: pgmap v14774: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T06:56:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:11 smithi082 ceph-mon[98694]: osdmap e992: 12 total, 12 up, 12 in 2023-09-10T06:56:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:11 smithi098 ceph-mon[103124]: pgmap v14774: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T06:56:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:11 smithi098 ceph-mon[103124]: osdmap e992: 12 total, 12 up, 12 in 2023-09-10T06:56:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:13 smithi170 ceph-mon[102832]: pgmap v14776: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 67 KiB/s wr, 35 op/s 2023-09-10T06:56:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27404.486452 secs 2023-09-10T06:56:13.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:13 smithi082 ceph-mon[98694]: pgmap v14776: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 67 KiB/s wr, 35 op/s 2023-09-10T06:56:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27404.486452 secs 2023-09-10T06:56:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:13 smithi098 ceph-mon[103124]: pgmap v14776: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 67 KiB/s wr, 35 op/s 2023-09-10T06:56:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27404.486452 secs 2023-09-10T06:56:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27405.626097 secs 2023-09-10T06:56:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27406.104200 secs 2023-09-10T06:56:14.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27405.626097 secs 2023-09-10T06:56:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27406.104200 secs 2023-09-10T06:56:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27405.626097 secs 2023-09-10T06:56:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27406.104200 secs 2023-09-10T06:56:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:15 smithi170 ceph-mon[102832]: pgmap v14777: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T06:56:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:15 smithi082 ceph-mon[98694]: pgmap v14777: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T06:56:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:15 smithi098 ceph-mon[103124]: pgmap v14777: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T06:56:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:17 smithi170 ceph-mon[102832]: pgmap v14778: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T06:56:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:17 smithi082 ceph-mon[98694]: pgmap v14778: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T06:56:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:17 smithi098 ceph-mon[103124]: pgmap v14778: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T06:56:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27409.486513 secs 2023-09-10T06:56:18.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27409.486513 secs 2023-09-10T06:56:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27409.486513 secs 2023-09-10T06:56:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27410.626169 secs 2023-09-10T06:56:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:19 smithi082 ceph-mon[98694]: pgmap v14779: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 207 B/s wr, 10 op/s 2023-09-10T06:56:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27410.626169 secs 2023-09-10T06:56:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:19 smithi098 ceph-mon[103124]: pgmap v14779: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 207 B/s wr, 10 op/s 2023-09-10T06:56:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27410.626169 secs 2023-09-10T06:56:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:19 smithi170 ceph-mon[102832]: pgmap v14779: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 207 B/s wr, 10 op/s 2023-09-10T06:56:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27411.104266 secs 2023-09-10T06:56:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27411.104266 secs 2023-09-10T06:56:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27411.104266 secs 2023-09-10T06:56:21.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:21 smithi082 ceph-mon[98694]: pgmap v14780: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T06:56:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:21 smithi098 ceph-mon[103124]: pgmap v14780: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T06:56:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:21 smithi170 ceph-mon[102832]: pgmap v14780: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T06:56:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:23 smithi098 ceph-mon[103124]: pgmap v14781: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T06:56:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27414.486636 secs 2023-09-10T06:56:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:23 smithi170 ceph-mon[102832]: pgmap v14781: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T06:56:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27414.486636 secs 2023-09-10T06:56:23.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:23 smithi082 ceph-mon[98694]: pgmap v14781: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T06:56:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27414.486636 secs 2023-09-10T06:56:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27415.626229 secs 2023-09-10T06:56:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27415.626229 secs 2023-09-10T06:56:24.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27415.626229 secs 2023-09-10T06:56:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27416.104330 secs 2023-09-10T06:56:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:25 smithi098 ceph-mon[103124]: pgmap v14782: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T06:56:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27416.104330 secs 2023-09-10T06:56:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:25 smithi170 ceph-mon[102832]: pgmap v14782: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T06:56:25.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27416.104330 secs 2023-09-10T06:56:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:25 smithi082 ceph-mon[98694]: pgmap v14782: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T06:56:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:27 smithi170 ceph-mon[102832]: pgmap v14783: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 511 B/s wr, 13 op/s 2023-09-10T06:56:27.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:27 smithi082 ceph-mon[98694]: pgmap v14783: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 511 B/s wr, 13 op/s 2023-09-10T06:56:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:27 smithi098 ceph-mon[103124]: pgmap v14783: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 511 B/s wr, 13 op/s 2023-09-10T06:56:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27420.626310 secs 2023-09-10T06:56:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:29 smithi098 ceph-mon[103124]: pgmap v14784: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T06:56:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27420.626310 secs 2023-09-10T06:56:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:29 smithi170 ceph-mon[102832]: pgmap v14784: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T06:56:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27420.626310 secs 2023-09-10T06:56:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:29 smithi082 ceph-mon[98694]: pgmap v14784: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T06:56:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27421.104397 secs 2023-09-10T06:56:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27421.104397 secs 2023-09-10T06:56:30.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27421.104397 secs 2023-09-10T06:56:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:31 smithi098 ceph-mon[103124]: pgmap v14785: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 187 KiB/s wr, 14 op/s 2023-09-10T06:56:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:56:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:31 smithi170 ceph-mon[102832]: pgmap v14785: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 187 KiB/s wr, 14 op/s 2023-09-10T06:56:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:56:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:31 smithi082 ceph-mon[98694]: pgmap v14785: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 187 KiB/s wr, 14 op/s 2023-09-10T06:56:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:56:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:56:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:56:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:56:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:56:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:56:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:56:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:56:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:56:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:56:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:56:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:56:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:56:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:56:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:56:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:56:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:56:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:56:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:56:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:33 smithi170 ceph-mon[102832]: pgmap v14786: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 9 op/s 2023-09-10T06:56:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:56:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:56:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:56:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:33 smithi082 ceph-mon[98694]: pgmap v14786: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 9 op/s 2023-09-10T06:56:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:56:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:56:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:56:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:33 smithi098 ceph-mon[103124]: pgmap v14786: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 9 op/s 2023-09-10T06:56:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:56:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:56:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:56:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27425.626367 secs 2023-09-10T06:56:34.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27425.626367 secs 2023-09-10T06:56:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27425.626367 secs 2023-09-10T06:56:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27426.104459 secs 2023-09-10T06:56:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:35 smithi170 ceph-mon[102832]: pgmap v14787: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 9 op/s 2023-09-10T06:56:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27426.104459 secs 2023-09-10T06:56:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:35 smithi082 ceph-mon[98694]: pgmap v14787: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 9 op/s 2023-09-10T06:56:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27426.104459 secs 2023-09-10T06:56:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:35 smithi098 ceph-mon[103124]: pgmap v14787: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 9 op/s 2023-09-10T06:56:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:37 smithi170 ceph-mon[102832]: pgmap v14788: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 333 KiB/s wr, 16 op/s 2023-09-10T06:56:37.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:37 smithi082 ceph-mon[98694]: pgmap v14788: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 333 KiB/s wr, 16 op/s 2023-09-10T06:56:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:37 smithi098 ceph-mon[103124]: pgmap v14788: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 333 KiB/s wr, 16 op/s 2023-09-10T06:56:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:56:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:56:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:56:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:56:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:56:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:56:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27430.626457 secs 2023-09-10T06:56:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:39 smithi170 ceph-mon[102832]: pgmap v14789: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 332 KiB/s wr, 11 op/s 2023-09-10T06:56:39.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27430.626457 secs 2023-09-10T06:56:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:39 smithi082 ceph-mon[98694]: pgmap v14789: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 332 KiB/s wr, 11 op/s 2023-09-10T06:56:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27430.626457 secs 2023-09-10T06:56:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:39 smithi098 ceph-mon[103124]: pgmap v14789: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 332 KiB/s wr, 11 op/s 2023-09-10T06:56:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27431.104486 secs 2023-09-10T06:56:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:56:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:56:40.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27431.104486 secs 2023-09-10T06:56:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:56:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:56:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27431.104486 secs 2023-09-10T06:56:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:56:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:56:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:41 smithi170 ceph-mon[102832]: pgmap v14790: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 357 KiB/s wr, 17 op/s 2023-09-10T06:56:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:41 smithi082 ceph-mon[98694]: pgmap v14790: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 357 KiB/s wr, 17 op/s 2023-09-10T06:56:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:41 smithi098 ceph-mon[103124]: pgmap v14790: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 357 KiB/s wr, 17 op/s 2023-09-10T06:56:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:43 smithi098 ceph-mon[103124]: pgmap v14791: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 171 KiB/s wr, 12 op/s 2023-09-10T06:56:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:43 smithi170 ceph-mon[102832]: pgmap v14791: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 171 KiB/s wr, 12 op/s 2023-09-10T06:56:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:43 smithi082 ceph-mon[98694]: pgmap v14791: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 171 KiB/s wr, 12 op/s 2023-09-10T06:56:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27435.626537 secs 2023-09-10T06:56:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27435.626537 secs 2023-09-10T06:56:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27435.626537 secs 2023-09-10T06:56:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27436.104550 secs 2023-09-10T06:56:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:45 smithi170 ceph-mon[102832]: pgmap v14792: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 171 KiB/s wr, 12 op/s 2023-09-10T06:56:45.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27436.104550 secs 2023-09-10T06:56:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:45 smithi082 ceph-mon[98694]: pgmap v14792: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 171 KiB/s wr, 12 op/s 2023-09-10T06:56:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27436.104550 secs 2023-09-10T06:56:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:45 smithi098 ceph-mon[103124]: pgmap v14792: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 171 KiB/s wr, 12 op/s 2023-09-10T06:56:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:47 smithi170 ceph-mon[102832]: pgmap v14793: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 516 KiB/s wr, 19 op/s 2023-09-10T06:56:47.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:47 smithi082 ceph-mon[98694]: pgmap v14793: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 516 KiB/s wr, 19 op/s 2023-09-10T06:56:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:47 smithi098 ceph-mon[103124]: pgmap v14793: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 516 KiB/s wr, 19 op/s 2023-09-10T06:56:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27440.626595 secs 2023-09-10T06:56:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:49 smithi170 ceph-mon[102832]: pgmap v14794: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 370 KiB/s wr, 13 op/s 2023-09-10T06:56:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27440.626595 secs 2023-09-10T06:56:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:49 smithi082 ceph-mon[98694]: pgmap v14794: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 370 KiB/s wr, 13 op/s 2023-09-10T06:56:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27440.626595 secs 2023-09-10T06:56:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:49 smithi098 ceph-mon[103124]: pgmap v14794: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 370 KiB/s wr, 13 op/s 2023-09-10T06:56:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27441.104612 secs 2023-09-10T06:56:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:56:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:56:50.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27441.104612 secs 2023-09-10T06:56:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:56:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:56:51.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27441.104612 secs 2023-09-10T06:56:51.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:56:51.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:56:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:51 smithi170 ceph-mon[102832]: pgmap v14795: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 469 KiB/s wr, 19 op/s 2023-09-10T06:56:51.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:51 smithi082 ceph-mon[98694]: pgmap v14795: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 469 KiB/s wr, 19 op/s 2023-09-10T06:56:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:51 smithi098 ceph-mon[103124]: pgmap v14795: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 469 KiB/s wr, 19 op/s 2023-09-10T06:56:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:56:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:56:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:56:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:56:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:56:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:56:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:53 smithi170 ceph-mon[102832]: pgmap v14796: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 444 KiB/s wr, 13 op/s 2023-09-10T06:56:53.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27444.487064 secs 2023-09-10T06:56:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:53 smithi082 ceph-mon[98694]: pgmap v14796: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 444 KiB/s wr, 13 op/s 2023-09-10T06:56:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27444.487064 secs 2023-09-10T06:56:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:53 smithi098 ceph-mon[103124]: pgmap v14796: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 444 KiB/s wr, 13 op/s 2023-09-10T06:56:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27444.487064 secs 2023-09-10T06:56:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27445.626654 secs 2023-09-10T06:56:54.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27445.626654 secs 2023-09-10T06:56:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27445.626654 secs 2023-09-10T06:56:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27446.104675 secs 2023-09-10T06:56:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:55 smithi170 ceph-mon[102832]: pgmap v14797: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 444 KiB/s wr, 13 op/s 2023-09-10T06:56:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27446.104675 secs 2023-09-10T06:56:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:55 smithi082 ceph-mon[98694]: pgmap v14797: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 444 KiB/s wr, 13 op/s 2023-09-10T06:56:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27446.104675 secs 2023-09-10T06:56:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:55 smithi098 ceph-mon[103124]: pgmap v14797: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 444 KiB/s wr, 13 op/s 2023-09-10T06:56:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:56:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:56:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:56:56.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:56:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:56:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:56:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:56:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:56:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:56:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:57 smithi170 ceph-mon[102832]: pgmap v14798: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 665 KiB/s wr, 19 op/s 2023-09-10T06:56:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:57 smithi082 ceph-mon[98694]: pgmap v14798: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 665 KiB/s wr, 19 op/s 2023-09-10T06:56:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:57 smithi098 ceph-mon[103124]: pgmap v14798: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 665 KiB/s wr, 19 op/s 2023-09-10T06:56:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27449.487123 secs 2023-09-10T06:56:58.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27449.487123 secs 2023-09-10T06:56:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27449.487123 secs 2023-09-10T06:56:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27450.626712 secs 2023-09-10T06:56:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:56:59 smithi170 ceph-mon[102832]: pgmap v14799: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 320 KiB/s wr, 12 op/s 2023-09-10T06:56:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27450.626712 secs 2023-09-10T06:56:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:56:59 smithi082 ceph-mon[98694]: pgmap v14799: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 320 KiB/s wr, 12 op/s 2023-09-10T06:57:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27450.626712 secs 2023-09-10T06:57:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:56:59 smithi098 ceph-mon[103124]: pgmap v14799: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 320 KiB/s wr, 12 op/s 2023-09-10T06:57:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27451.104738 secs 2023-09-10T06:57:00.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27451.104738 secs 2023-09-10T06:57:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27451.104738 secs 2023-09-10T06:57:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:01 smithi170 ceph-mon[102832]: pgmap v14800: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 394 KiB/s wr, 17 op/s 2023-09-10T06:57:01.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:01 smithi082 ceph-mon[98694]: pgmap v14800: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 394 KiB/s wr, 17 op/s 2023-09-10T06:57:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:01 smithi098 ceph-mon[103124]: pgmap v14800: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 394 KiB/s wr, 17 op/s 2023-09-10T06:57:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:03 smithi170 ceph-mon[102832]: pgmap v14801: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T06:57:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27454.487224 secs 2023-09-10T06:57:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:03 smithi082 ceph-mon[98694]: pgmap v14801: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T06:57:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27454.487224 secs 2023-09-10T06:57:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:03 smithi098 ceph-mon[103124]: pgmap v14801: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T06:57:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27454.487224 secs 2023-09-10T06:57:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27455.626770 secs 2023-09-10T06:57:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:57:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:57:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27455.626770 secs 2023-09-10T06:57:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:57:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:57:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27455.626770 secs 2023-09-10T06:57:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:57:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:57:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27456.104808 secs 2023-09-10T06:57:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:05 smithi170 ceph-mon[102832]: pgmap v14802: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T06:57:05.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27456.104808 secs 2023-09-10T06:57:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:05 smithi082 ceph-mon[98694]: pgmap v14802: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T06:57:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27456.104808 secs 2023-09-10T06:57:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:05 smithi098 ceph-mon[103124]: pgmap v14802: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T06:57:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:57:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:57:06.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:57:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:57:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:57:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:57:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:07 smithi170 ceph-mon[102832]: pgmap v14803: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 399 KiB/s wr, 30 op/s 2023-09-10T06:57:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:07 smithi082 ceph-mon[98694]: pgmap v14803: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 399 KiB/s wr, 30 op/s 2023-09-10T06:57:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:07 smithi098 ceph-mon[103124]: pgmap v14803: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 399 KiB/s wr, 30 op/s 2023-09-10T06:57:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27459.487287 secs 2023-09-10T06:57:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27459.487287 secs 2023-09-10T06:57:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27459.487287 secs 2023-09-10T06:57:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27460.626850 secs 2023-09-10T06:57:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:09 smithi170 ceph-mon[102832]: pgmap v14804: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 177 KiB/s wr, 24 op/s 2023-09-10T06:57:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:09 smithi170 ceph-mon[102832]: osdmap e993: 12 total, 12 up, 12 in 2023-09-10T06:57:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27461.104926 secs 2023-09-10T06:57:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27460.626850 secs 2023-09-10T06:57:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:09 smithi082 ceph-mon[98694]: pgmap v14804: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 177 KiB/s wr, 24 op/s 2023-09-10T06:57:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:09 smithi082 ceph-mon[98694]: osdmap e993: 12 total, 12 up, 12 in 2023-09-10T06:57:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27461.104926 secs 2023-09-10T06:57:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27460.626850 secs 2023-09-10T06:57:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:09 smithi098 ceph-mon[103124]: pgmap v14804: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 177 KiB/s wr, 24 op/s 2023-09-10T06:57:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:09 smithi098 ceph-mon[103124]: osdmap e993: 12 total, 12 up, 12 in 2023-09-10T06:57:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27461.104926 secs 2023-09-10T06:57:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:11 smithi170 ceph-mon[102832]: pgmap v14806: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 124 KiB/s wr, 29 op/s 2023-09-10T06:57:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:11 smithi170 ceph-mon[102832]: osdmap e994: 12 total, 12 up, 12 in 2023-09-10T06:57:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:11 smithi082 ceph-mon[98694]: pgmap v14806: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 124 KiB/s wr, 29 op/s 2023-09-10T06:57:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:11 smithi082 ceph-mon[98694]: osdmap e994: 12 total, 12 up, 12 in 2023-09-10T06:57:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:11 smithi098 ceph-mon[103124]: pgmap v14806: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 124 KiB/s wr, 29 op/s 2023-09-10T06:57:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:11 smithi098 ceph-mon[103124]: osdmap e994: 12 total, 12 up, 12 in 2023-09-10T06:57:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:13 smithi170 ceph-mon[102832]: pgmap v14808: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 155 KiB/s wr, 36 op/s 2023-09-10T06:57:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27464.487348 secs 2023-09-10T06:57:13.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:13 smithi082 ceph-mon[98694]: pgmap v14808: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 155 KiB/s wr, 36 op/s 2023-09-10T06:57:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27464.487348 secs 2023-09-10T06:57:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:13 smithi098 ceph-mon[103124]: pgmap v14808: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 155 KiB/s wr, 36 op/s 2023-09-10T06:57:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27464.487348 secs 2023-09-10T06:57:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27465.626909 secs 2023-09-10T06:57:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27466.104990 secs 2023-09-10T06:57:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27465.626909 secs 2023-09-10T06:57:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27466.104990 secs 2023-09-10T06:57:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27465.626909 secs 2023-09-10T06:57:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27466.104990 secs 2023-09-10T06:57:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:15 smithi170 ceph-mon[102832]: pgmap v14809: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T06:57:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:15 smithi082 ceph-mon[98694]: pgmap v14809: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T06:57:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:15 smithi098 ceph-mon[103124]: pgmap v14809: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T06:57:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:17 smithi170 ceph-mon[102832]: pgmap v14810: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 14 op/s 2023-09-10T06:57:17.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:17 smithi082 ceph-mon[98694]: pgmap v14810: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 14 op/s 2023-09-10T06:57:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:17 smithi098 ceph-mon[103124]: pgmap v14810: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 14 op/s 2023-09-10T06:57:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27469.487410 secs 2023-09-10T06:57:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27469.487410 secs 2023-09-10T06:57:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27469.487410 secs 2023-09-10T06:57:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27470.627013 secs 2023-09-10T06:57:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:19 smithi082 ceph-mon[98694]: pgmap v14811: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 207 B/s wr, 11 op/s 2023-09-10T06:57:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27470.627013 secs 2023-09-10T06:57:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:19 smithi098 ceph-mon[103124]: pgmap v14811: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 207 B/s wr, 11 op/s 2023-09-10T06:57:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27470.627013 secs 2023-09-10T06:57:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:19 smithi170 ceph-mon[102832]: pgmap v14811: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 207 B/s wr, 11 op/s 2023-09-10T06:57:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27471.105056 secs 2023-09-10T06:57:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27471.105056 secs 2023-09-10T06:57:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27471.105056 secs 2023-09-10T06:57:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:21 smithi082 ceph-mon[98694]: pgmap v14812: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T06:57:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:21 smithi098 ceph-mon[103124]: pgmap v14812: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T06:57:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:21 smithi170 ceph-mon[102832]: pgmap v14812: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T06:57:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:23 smithi170 ceph-mon[102832]: pgmap v14813: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T06:57:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27474.487476 secs 2023-09-10T06:57:23.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:23 smithi082 ceph-mon[98694]: pgmap v14813: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T06:57:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27474.487476 secs 2023-09-10T06:57:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:23 smithi098 ceph-mon[103124]: pgmap v14813: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T06:57:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27474.487476 secs 2023-09-10T06:57:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27475.627071 secs 2023-09-10T06:57:24.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27475.627071 secs 2023-09-10T06:57:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27475.627071 secs 2023-09-10T06:57:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27476.105118 secs 2023-09-10T06:57:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:25 smithi170 ceph-mon[102832]: pgmap v14814: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-10T06:57:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27476.105118 secs 2023-09-10T06:57:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:25 smithi082 ceph-mon[98694]: pgmap v14814: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-10T06:57:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27476.105118 secs 2023-09-10T06:57:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:25 smithi098 ceph-mon[103124]: pgmap v14814: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-10T06:57:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:27 smithi170 ceph-mon[102832]: pgmap v14815: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 14 op/s 2023-09-10T06:57:27.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:27 smithi082 ceph-mon[98694]: pgmap v14815: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 14 op/s 2023-09-10T06:57:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:27 smithi098 ceph-mon[103124]: pgmap v14815: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 14 op/s 2023-09-10T06:57:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27480.627129 secs 2023-09-10T06:57:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:29 smithi170 ceph-mon[102832]: pgmap v14816: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2023-09-10T06:57:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27480.627129 secs 2023-09-10T06:57:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:29 smithi082 ceph-mon[98694]: pgmap v14816: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2023-09-10T06:57:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27480.627129 secs 2023-09-10T06:57:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:29 smithi098 ceph-mon[103124]: pgmap v14816: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2023-09-10T06:57:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27481.105184 secs 2023-09-10T06:57:30.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27481.105184 secs 2023-09-10T06:57:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27481.105184 secs 2023-09-10T06:57:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:31 smithi170 ceph-mon[102832]: pgmap v14817: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 188 KiB/s wr, 14 op/s 2023-09-10T06:57:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:57:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:31 smithi082 ceph-mon[98694]: pgmap v14817: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 188 KiB/s wr, 14 op/s 2023-09-10T06:57:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:57:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:31 smithi098 ceph-mon[103124]: pgmap v14817: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 188 KiB/s wr, 14 op/s 2023-09-10T06:57:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:57:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:57:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:57:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:57:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:57:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:57:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:57:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:57:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:57:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:57:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:33 smithi170 ceph-mon[102832]: pgmap v14818: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 188 KiB/s wr, 9 op/s 2023-09-10T06:57:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:57:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:57:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:57:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:57:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:57:33.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:33 smithi082 ceph-mon[98694]: pgmap v14818: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 188 KiB/s wr, 9 op/s 2023-09-10T06:57:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:57:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:57:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:57:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:57:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:57:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:33 smithi098 ceph-mon[103124]: pgmap v14818: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 188 KiB/s wr, 9 op/s 2023-09-10T06:57:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:57:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:57:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:57:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:57:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:57:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27485.627182 secs 2023-09-10T06:57:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:57:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:57:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:57:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27485.627182 secs 2023-09-10T06:57:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:57:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:57:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:57:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27485.627182 secs 2023-09-10T06:57:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:57:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:57:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:57:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27486.105250 secs 2023-09-10T06:57:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:35 smithi170 ceph-mon[102832]: pgmap v14819: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 188 KiB/s wr, 9 op/s 2023-09-10T06:57:35.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27486.105250 secs 2023-09-10T06:57:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:35 smithi082 ceph-mon[98694]: pgmap v14819: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 188 KiB/s wr, 9 op/s 2023-09-10T06:57:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27486.105250 secs 2023-09-10T06:57:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:35 smithi098 ceph-mon[103124]: pgmap v14819: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 188 KiB/s wr, 9 op/s 2023-09-10T06:57:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:37 smithi170 ceph-mon[102832]: pgmap v14820: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 333 KiB/s wr, 15 op/s 2023-09-10T06:57:37.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:37 smithi082 ceph-mon[98694]: pgmap v14820: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 333 KiB/s wr, 15 op/s 2023-09-10T06:57:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:37 smithi098 ceph-mon[103124]: pgmap v14820: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 333 KiB/s wr, 15 op/s 2023-09-10T06:57:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:57:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:57:38.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:57:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:57:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:57:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:57:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27490.627262 secs 2023-09-10T06:57:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:39 smithi170 ceph-mon[102832]: pgmap v14821: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 333 KiB/s wr, 10 op/s 2023-09-10T06:57:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27490.627262 secs 2023-09-10T06:57:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:39 smithi082 ceph-mon[98694]: pgmap v14821: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 333 KiB/s wr, 10 op/s 2023-09-10T06:57:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27490.627262 secs 2023-09-10T06:57:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:39 smithi098 ceph-mon[103124]: pgmap v14821: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 333 KiB/s wr, 10 op/s 2023-09-10T06:57:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27491.105313 secs 2023-09-10T06:57:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:57:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:57:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27491.105313 secs 2023-09-10T06:57:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:57:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:57:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27491.105313 secs 2023-09-10T06:57:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:57:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:57:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:41 smithi170 ceph-mon[102832]: pgmap v14822: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 458 KiB/s wr, 17 op/s 2023-09-10T06:57:41.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:41 smithi082 ceph-mon[98694]: pgmap v14822: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 458 KiB/s wr, 17 op/s 2023-09-10T06:57:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:41 smithi098 ceph-mon[103124]: pgmap v14822: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 458 KiB/s wr, 17 op/s 2023-09-10T06:57:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:43 smithi170 ceph-mon[102832]: pgmap v14823: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T06:57:43.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:43 smithi082 ceph-mon[98694]: pgmap v14823: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T06:57:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:43 smithi098 ceph-mon[103124]: pgmap v14823: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T06:57:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27495.627320 secs 2023-09-10T06:57:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27495.627320 secs 2023-09-10T06:57:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27495.627320 secs 2023-09-10T06:57:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27496.105374 secs 2023-09-10T06:57:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:45 smithi170 ceph-mon[102832]: pgmap v14824: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T06:57:45.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27496.105374 secs 2023-09-10T06:57:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:45 smithi082 ceph-mon[98694]: pgmap v14824: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T06:57:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27496.105374 secs 2023-09-10T06:57:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:45 smithi098 ceph-mon[103124]: pgmap v14824: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T06:57:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:47 smithi170 ceph-mon[102832]: pgmap v14825: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 502 KiB/s wr, 19 op/s 2023-09-10T06:57:47.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:47 smithi082 ceph-mon[98694]: pgmap v14825: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 502 KiB/s wr, 19 op/s 2023-09-10T06:57:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:47 smithi098 ceph-mon[103124]: pgmap v14825: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 502 KiB/s wr, 19 op/s 2023-09-10T06:57:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27500.627380 secs 2023-09-10T06:57:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:49 smithi170 ceph-mon[102832]: pgmap v14826: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 356 KiB/s wr, 13 op/s 2023-09-10T06:57:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27500.627380 secs 2023-09-10T06:57:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:49 smithi082 ceph-mon[98694]: pgmap v14826: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 356 KiB/s wr, 13 op/s 2023-09-10T06:57:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27500.627380 secs 2023-09-10T06:57:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:49 smithi098 ceph-mon[103124]: pgmap v14826: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 356 KiB/s wr, 13 op/s 2023-09-10T06:57:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27501.105439 secs 2023-09-10T06:57:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:57:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:57:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27501.105439 secs 2023-09-10T06:57:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:57:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:57:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27501.105439 secs 2023-09-10T06:57:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:57:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:57:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:51 smithi170 ceph-mon[102832]: pgmap v14827: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 488 KiB/s wr, 19 op/s 2023-09-10T06:57:51.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:51 smithi082 ceph-mon[98694]: pgmap v14827: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 488 KiB/s wr, 19 op/s 2023-09-10T06:57:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:51 smithi098 ceph-mon[103124]: pgmap v14827: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 488 KiB/s wr, 19 op/s 2023-09-10T06:57:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:57:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:57:52.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:57:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:57:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:57:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:57:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:53 smithi170 ceph-mon[102832]: pgmap v14828: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 363 KiB/s wr, 13 op/s 2023-09-10T06:57:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27504.487899 secs 2023-09-10T06:57:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:53 smithi082 ceph-mon[98694]: pgmap v14828: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 363 KiB/s wr, 13 op/s 2023-09-10T06:57:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27504.487899 secs 2023-09-10T06:57:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:53 smithi098 ceph-mon[103124]: pgmap v14828: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 363 KiB/s wr, 13 op/s 2023-09-10T06:57:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27504.487899 secs 2023-09-10T06:57:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27505.627437 secs 2023-09-10T06:57:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27505.627437 secs 2023-09-10T06:57:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27505.627437 secs 2023-09-10T06:57:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27506.105484 secs 2023-09-10T06:57:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:55 smithi170 ceph-mon[102832]: pgmap v14829: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 363 KiB/s wr, 13 op/s 2023-09-10T06:57:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27506.105484 secs 2023-09-10T06:57:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:55 smithi082 ceph-mon[98694]: pgmap v14829: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 363 KiB/s wr, 13 op/s 2023-09-10T06:57:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27506.105484 secs 2023-09-10T06:57:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:55 smithi098 ceph-mon[103124]: pgmap v14829: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 363 KiB/s wr, 13 op/s 2023-09-10T06:57:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:57:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:57:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:57:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:57:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:57:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:57:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:57:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:57:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:57:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:57 smithi170 ceph-mon[102832]: pgmap v14830: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 568 KiB/s wr, 19 op/s 2023-09-10T06:57:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:57 smithi082 ceph-mon[98694]: pgmap v14830: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 568 KiB/s wr, 19 op/s 2023-09-10T06:57:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:57 smithi098 ceph-mon[103124]: pgmap v14830: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 568 KiB/s wr, 19 op/s 2023-09-10T06:57:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27509.487956 secs 2023-09-10T06:57:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27509.487956 secs 2023-09-10T06:57:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27509.487956 secs 2023-09-10T06:57:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27510.627519 secs 2023-09-10T06:57:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:59 smithi170 ceph-mon[102832]: pgmap v14831: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 336 KiB/s wr, 12 op/s 2023-09-10T06:57:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:57:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27511.105552 secs 2023-09-10T06:57:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27510.627519 secs 2023-09-10T06:57:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:59 smithi082 ceph-mon[98694]: pgmap v14831: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 336 KiB/s wr, 12 op/s 2023-09-10T06:57:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:57:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27511.105552 secs 2023-09-10T06:58:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27510.627519 secs 2023-09-10T06:58:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:59 smithi098 ceph-mon[103124]: pgmap v14831: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 336 KiB/s wr, 12 op/s 2023-09-10T06:58:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:57:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27511.105552 secs 2023-09-10T06:58:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:01 smithi170 ceph-mon[102832]: pgmap v14832: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 410 KiB/s wr, 17 op/s 2023-09-10T06:58:01.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:01 smithi082 ceph-mon[98694]: pgmap v14832: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 410 KiB/s wr, 17 op/s 2023-09-10T06:58:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:01 smithi098 ceph-mon[103124]: pgmap v14832: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 410 KiB/s wr, 17 op/s 2023-09-10T06:58:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:03 smithi170 ceph-mon[102832]: pgmap v14833: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 11 op/s 2023-09-10T06:58:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27514.488049 secs 2023-09-10T06:58:03.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:03 smithi082 ceph-mon[98694]: pgmap v14833: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 11 op/s 2023-09-10T06:58:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27514.488049 secs 2023-09-10T06:58:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:03 smithi098 ceph-mon[103124]: pgmap v14833: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 11 op/s 2023-09-10T06:58:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27514.488049 secs 2023-09-10T06:58:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27515.627574 secs 2023-09-10T06:58:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:58:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:58:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27516.105612 secs 2023-09-10T06:58:04.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27515.627574 secs 2023-09-10T06:58:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:58:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:58:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27516.105612 secs 2023-09-10T06:58:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27515.627574 secs 2023-09-10T06:58:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:58:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:58:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27516.105612 secs 2023-09-10T06:58:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:05 smithi170 ceph-mon[102832]: pgmap v14834: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 11 op/s 2023-09-10T06:58:05.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:05 smithi082 ceph-mon[98694]: pgmap v14834: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 11 op/s 2023-09-10T06:58:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:05 smithi098 ceph-mon[103124]: pgmap v14834: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 11 op/s 2023-09-10T06:58:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:58:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:58:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:58:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:58:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:58:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:58:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:07 smithi170 ceph-mon[102832]: pgmap v14835: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 382 KiB/s wr, 30 op/s 2023-09-10T06:58:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:07 smithi082 ceph-mon[98694]: pgmap v14835: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 382 KiB/s wr, 30 op/s 2023-09-10T06:58:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:07 smithi098 ceph-mon[103124]: pgmap v14835: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 382 KiB/s wr, 30 op/s 2023-09-10T06:58:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27519.488108 secs 2023-09-10T06:58:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27519.488108 secs 2023-09-10T06:58:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27519.488108 secs 2023-09-10T06:58:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27520.627633 secs 2023-09-10T06:58:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:09 smithi170 ceph-mon[102832]: pgmap v14836: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 178 KiB/s wr, 24 op/s 2023-09-10T06:58:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:09 smithi170 ceph-mon[102832]: osdmap e995: 12 total, 12 up, 12 in 2023-09-10T06:58:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27521.105680 secs 2023-09-10T06:58:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27520.627633 secs 2023-09-10T06:58:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:09 smithi082 ceph-mon[98694]: pgmap v14836: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 178 KiB/s wr, 24 op/s 2023-09-10T06:58:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:09 smithi082 ceph-mon[98694]: osdmap e995: 12 total, 12 up, 12 in 2023-09-10T06:58:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27521.105680 secs 2023-09-10T06:58:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27520.627633 secs 2023-09-10T06:58:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:09 smithi098 ceph-mon[103124]: pgmap v14836: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 178 KiB/s wr, 24 op/s 2023-09-10T06:58:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:09 smithi098 ceph-mon[103124]: osdmap e995: 12 total, 12 up, 12 in 2023-09-10T06:58:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27521.105680 secs 2023-09-10T06:58:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:11 smithi170 ceph-mon[102832]: pgmap v14838: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 125 KiB/s wr, 28 op/s 2023-09-10T06:58:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:11 smithi170 ceph-mon[102832]: osdmap e996: 12 total, 12 up, 12 in 2023-09-10T06:58:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:11 smithi082 ceph-mon[98694]: pgmap v14838: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 125 KiB/s wr, 28 op/s 2023-09-10T06:58:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:11 smithi082 ceph-mon[98694]: osdmap e996: 12 total, 12 up, 12 in 2023-09-10T06:58:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:11 smithi098 ceph-mon[103124]: pgmap v14838: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 125 KiB/s wr, 28 op/s 2023-09-10T06:58:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:11 smithi098 ceph-mon[103124]: osdmap e996: 12 total, 12 up, 12 in 2023-09-10T06:58:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:13 smithi170 ceph-mon[102832]: pgmap v14840: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 156 KiB/s wr, 35 op/s 2023-09-10T06:58:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27524.488220 secs 2023-09-10T06:58:13.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:13 smithi082 ceph-mon[98694]: pgmap v14840: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 156 KiB/s wr, 35 op/s 2023-09-10T06:58:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27524.488220 secs 2023-09-10T06:58:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:13 smithi098 ceph-mon[103124]: pgmap v14840: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 156 KiB/s wr, 35 op/s 2023-09-10T06:58:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27524.488220 secs 2023-09-10T06:58:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27525.627693 secs 2023-09-10T06:58:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27526.105741 secs 2023-09-10T06:58:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27525.627693 secs 2023-09-10T06:58:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27526.105741 secs 2023-09-10T06:58:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27525.627693 secs 2023-09-10T06:58:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27526.105741 secs 2023-09-10T06:58:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:15 smithi170 ceph-mon[102832]: pgmap v14841: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T06:58:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:15 smithi082 ceph-mon[98694]: pgmap v14841: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T06:58:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:15 smithi098 ceph-mon[103124]: pgmap v14841: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T06:58:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:17 smithi170 ceph-mon[102832]: pgmap v14842: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T06:58:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:17 smithi082 ceph-mon[98694]: pgmap v14842: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T06:58:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:17 smithi098 ceph-mon[103124]: pgmap v14842: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T06:58:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27529.488318 secs 2023-09-10T06:58:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27529.488318 secs 2023-09-10T06:58:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27529.488318 secs 2023-09-10T06:58:19.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27530.627755 secs 2023-09-10T06:58:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:19 smithi082 ceph-mon[98694]: pgmap v14843: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 207 B/s wr, 10 op/s 2023-09-10T06:58:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27530.627755 secs 2023-09-10T06:58:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:19 smithi098 ceph-mon[103124]: pgmap v14843: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 207 B/s wr, 10 op/s 2023-09-10T06:58:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27530.627755 secs 2023-09-10T06:58:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:19 smithi170 ceph-mon[102832]: pgmap v14843: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 207 B/s wr, 10 op/s 2023-09-10T06:58:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27531.105805 secs 2023-09-10T06:58:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27531.105805 secs 2023-09-10T06:58:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27531.105805 secs 2023-09-10T06:58:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:21 smithi082 ceph-mon[98694]: pgmap v14844: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T06:58:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:21 smithi098 ceph-mon[103124]: pgmap v14844: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T06:58:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:21 smithi170 ceph-mon[102832]: pgmap v14844: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T06:58:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:23 smithi170 ceph-mon[102832]: pgmap v14845: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T06:58:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27534.488433 secs 2023-09-10T06:58:23.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:23 smithi082 ceph-mon[98694]: pgmap v14845: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T06:58:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27534.488433 secs 2023-09-10T06:58:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:23 smithi098 ceph-mon[103124]: pgmap v14845: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T06:58:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27534.488433 secs 2023-09-10T06:58:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27535.627813 secs 2023-09-10T06:58:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27535.627813 secs 2023-09-10T06:58:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27535.627813 secs 2023-09-10T06:58:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27536.105868 secs 2023-09-10T06:58:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:25 smithi170 ceph-mon[102832]: pgmap v14846: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T06:58:25.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27536.105868 secs 2023-09-10T06:58:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:25 smithi082 ceph-mon[98694]: pgmap v14846: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T06:58:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27536.105868 secs 2023-09-10T06:58:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:25 smithi098 ceph-mon[103124]: pgmap v14846: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T06:58:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:27 smithi170 ceph-mon[102832]: pgmap v14847: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 90 KiB/s wr, 13 op/s 2023-09-10T06:58:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:27 smithi082 ceph-mon[98694]: pgmap v14847: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 90 KiB/s wr, 13 op/s 2023-09-10T06:58:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:27 smithi098 ceph-mon[103124]: pgmap v14847: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 90 KiB/s wr, 13 op/s 2023-09-10T06:58:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27540.627871 secs 2023-09-10T06:58:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:29 smithi170 ceph-mon[102832]: pgmap v14848: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 9 op/s 2023-09-10T06:58:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27540.627871 secs 2023-09-10T06:58:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:29 smithi082 ceph-mon[98694]: pgmap v14848: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 9 op/s 2023-09-10T06:58:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27540.627871 secs 2023-09-10T06:58:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:29 smithi098 ceph-mon[103124]: pgmap v14848: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 9 op/s 2023-09-10T06:58:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27541.105930 secs 2023-09-10T06:58:30.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27541.105930 secs 2023-09-10T06:58:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27541.105930 secs 2023-09-10T06:58:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:31 smithi170 ceph-mon[102832]: pgmap v14849: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 277 KiB/s wr, 14 op/s 2023-09-10T06:58:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:58:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:31 smithi082 ceph-mon[98694]: pgmap v14849: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 277 KiB/s wr, 14 op/s 2023-09-10T06:58:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:58:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:31 smithi098 ceph-mon[103124]: pgmap v14849: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 277 KiB/s wr, 14 op/s 2023-09-10T06:58:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:58:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:58:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:58:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:58:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:58:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:58:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:58:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:58:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:58:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:58:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:33 smithi170 ceph-mon[102832]: pgmap v14850: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 10 op/s 2023-09-10T06:58:33.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:33 smithi082 ceph-mon[98694]: pgmap v14850: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 10 op/s 2023-09-10T06:58:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:33 smithi098 ceph-mon[103124]: pgmap v14850: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 10 op/s 2023-09-10T06:58:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27545.627985 secs 2023-09-10T06:58:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:58:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:58:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:58:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:58:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27545.627985 secs 2023-09-10T06:58:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:58:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:58:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:58:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:58:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27545.627985 secs 2023-09-10T06:58:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:58:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:58:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:58:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:58:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27546.105996 secs 2023-09-10T06:58:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:35 smithi170 ceph-mon[102832]: pgmap v14851: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 10 op/s 2023-09-10T06:58:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27546.105996 secs 2023-09-10T06:58:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:35 smithi082 ceph-mon[98694]: pgmap v14851: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 10 op/s 2023-09-10T06:58:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27546.105996 secs 2023-09-10T06:58:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:35 smithi098 ceph-mon[103124]: pgmap v14851: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 10 op/s 2023-09-10T06:58:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:58:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:58:36.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:58:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:58:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:58:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:58:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:37 smithi170 ceph-mon[102832]: pgmap v14852: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 424 KiB/s wr, 16 op/s 2023-09-10T06:58:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:37 smithi082 ceph-mon[98694]: pgmap v14852: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 424 KiB/s wr, 16 op/s 2023-09-10T06:58:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:37 smithi098 ceph-mon[103124]: pgmap v14852: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 424 KiB/s wr, 16 op/s 2023-09-10T06:58:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:58:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:58:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:58:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:58:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:58:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:58:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27550.628066 secs 2023-09-10T06:58:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:39 smithi170 ceph-mon[102832]: pgmap v14853: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 334 KiB/s wr, 11 op/s 2023-09-10T06:58:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27551.106059 secs 2023-09-10T06:58:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27550.628066 secs 2023-09-10T06:58:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:39 smithi082 ceph-mon[98694]: pgmap v14853: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 334 KiB/s wr, 11 op/s 2023-09-10T06:58:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27551.106059 secs 2023-09-10T06:58:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27550.628066 secs 2023-09-10T06:58:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:39 smithi098 ceph-mon[103124]: pgmap v14853: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 334 KiB/s wr, 11 op/s 2023-09-10T06:58:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27551.106059 secs 2023-09-10T06:58:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:41 smithi170 ceph-mon[102832]: pgmap v14854: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 459 KiB/s wr, 17 op/s 2023-09-10T06:58:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:41 smithi082 ceph-mon[98694]: pgmap v14854: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 459 KiB/s wr, 17 op/s 2023-09-10T06:58:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:41 smithi098 ceph-mon[103124]: pgmap v14854: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 459 KiB/s wr, 17 op/s 2023-09-10T06:58:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:43 smithi170 ceph-mon[102832]: pgmap v14855: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 272 KiB/s wr, 12 op/s 2023-09-10T06:58:43.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:43 smithi082 ceph-mon[98694]: pgmap v14855: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 272 KiB/s wr, 12 op/s 2023-09-10T06:58:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:43 smithi098 ceph-mon[103124]: pgmap v14855: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 272 KiB/s wr, 12 op/s 2023-09-10T06:58:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27555.628127 secs 2023-09-10T06:58:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:58:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:58:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27556.106119 secs 2023-09-10T06:58:44.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27555.628127 secs 2023-09-10T06:58:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:58:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:58:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27556.106119 secs 2023-09-10T06:58:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27555.628127 secs 2023-09-10T06:58:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:58:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:58:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27556.106119 secs 2023-09-10T06:58:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:45 smithi170 ceph-mon[102832]: pgmap v14856: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 272 KiB/s wr, 12 op/s 2023-09-10T06:58:45.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:45 smithi082 ceph-mon[98694]: pgmap v14856: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 272 KiB/s wr, 12 op/s 2023-09-10T06:58:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:45 smithi098 ceph-mon[103124]: pgmap v14856: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 272 KiB/s wr, 12 op/s 2023-09-10T06:58:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:58:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:58:46.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:58:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:58:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:58:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:58:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:47 smithi170 ceph-mon[102832]: pgmap v14857: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 504 KiB/s wr, 20 op/s 2023-09-10T06:58:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:47 smithi082 ceph-mon[98694]: pgmap v14857: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 504 KiB/s wr, 20 op/s 2023-09-10T06:58:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:47 smithi098 ceph-mon[103124]: pgmap v14857: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 504 KiB/s wr, 20 op/s 2023-09-10T06:58:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27560.628213 secs 2023-09-10T06:58:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:49 smithi170 ceph-mon[102832]: pgmap v14858: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 358 KiB/s wr, 13 op/s 2023-09-10T06:58:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27561.106184 secs 2023-09-10T06:58:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27560.628213 secs 2023-09-10T06:58:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:49 smithi082 ceph-mon[98694]: pgmap v14858: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 358 KiB/s wr, 13 op/s 2023-09-10T06:58:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27561.106184 secs 2023-09-10T06:58:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27560.628213 secs 2023-09-10T06:58:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:49 smithi098 ceph-mon[103124]: pgmap v14858: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 358 KiB/s wr, 13 op/s 2023-09-10T06:58:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27561.106184 secs 2023-09-10T06:58:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:51 smithi170 ceph-mon[102832]: pgmap v14859: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 490 KiB/s wr, 20 op/s 2023-09-10T06:58:51.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:51 smithi082 ceph-mon[98694]: pgmap v14859: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 490 KiB/s wr, 20 op/s 2023-09-10T06:58:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:51 smithi098 ceph-mon[103124]: pgmap v14859: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 490 KiB/s wr, 20 op/s 2023-09-10T06:58:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:53 smithi170 ceph-mon[102832]: pgmap v14860: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 364 KiB/s wr, 13 op/s 2023-09-10T06:58:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27564.488890 secs 2023-09-10T06:58:53.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:53 smithi082 ceph-mon[98694]: pgmap v14860: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 364 KiB/s wr, 13 op/s 2023-09-10T06:58:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27564.488890 secs 2023-09-10T06:58:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:53 smithi098 ceph-mon[103124]: pgmap v14860: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 364 KiB/s wr, 13 op/s 2023-09-10T06:58:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27564.488890 secs 2023-09-10T06:58:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27565.628272 secs 2023-09-10T06:58:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:58:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:58:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27566.106244 secs 2023-09-10T06:58:54.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27565.628272 secs 2023-09-10T06:58:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:58:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:58:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27566.106244 secs 2023-09-10T06:58:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27565.628272 secs 2023-09-10T06:58:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:58:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:58:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27566.106244 secs 2023-09-10T06:58:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:55 smithi170 ceph-mon[102832]: pgmap v14861: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 364 KiB/s wr, 13 op/s 2023-09-10T06:58:55.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:55 smithi082 ceph-mon[98694]: pgmap v14861: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 364 KiB/s wr, 13 op/s 2023-09-10T06:58:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:55 smithi098 ceph-mon[103124]: pgmap v14861: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 364 KiB/s wr, 13 op/s 2023-09-10T06:58:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:58:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:58:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:58:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:58:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:58:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:58:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:58:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:58:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:58:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:57 smithi170 ceph-mon[102832]: pgmap v14862: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 570 KiB/s wr, 19 op/s 2023-09-10T06:58:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:57 smithi082 ceph-mon[98694]: pgmap v14862: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 570 KiB/s wr, 19 op/s 2023-09-10T06:58:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:57 smithi098 ceph-mon[103124]: pgmap v14862: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 570 KiB/s wr, 19 op/s 2023-09-10T06:58:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27569.488949 secs 2023-09-10T06:58:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27569.488949 secs 2023-09-10T06:58:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27569.488949 secs 2023-09-10T06:58:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27570.628357 secs 2023-09-10T06:58:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:59 smithi170 ceph-mon[102832]: pgmap v14863: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 337 KiB/s wr, 12 op/s 2023-09-10T06:58:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:58:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27571.106310 secs 2023-09-10T06:58:59.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27570.628357 secs 2023-09-10T06:58:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:59 smithi082 ceph-mon[98694]: pgmap v14863: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 337 KiB/s wr, 12 op/s 2023-09-10T06:58:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:58:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27571.106310 secs 2023-09-10T06:59:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27570.628357 secs 2023-09-10T06:59:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:59 smithi098 ceph-mon[103124]: pgmap v14863: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 337 KiB/s wr, 12 op/s 2023-09-10T06:59:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:58:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27571.106310 secs 2023-09-10T06:59:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:01 smithi170 ceph-mon[102832]: pgmap v14864: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 451 KiB/s wr, 31 op/s 2023-09-10T06:59:01.970 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:01 smithi082 ceph-mon[98694]: pgmap v14864: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 451 KiB/s wr, 31 op/s 2023-09-10T06:59:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:01 smithi098 ceph-mon[103124]: pgmap v14864: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 451 KiB/s wr, 31 op/s 2023-09-10T06:59:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:03 smithi170 ceph-mon[102832]: pgmap v14865: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 319 KiB/s wr, 25 op/s 2023-09-10T06:59:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27574.489046 secs 2023-09-10T06:59:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:03 smithi082 ceph-mon[98694]: pgmap v14865: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 319 KiB/s wr, 25 op/s 2023-09-10T06:59:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27574.489046 secs 2023-09-10T06:59:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:03 smithi098 ceph-mon[103124]: pgmap v14865: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 319 KiB/s wr, 25 op/s 2023-09-10T06:59:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27574.489046 secs 2023-09-10T06:59:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27575.628444 secs 2023-09-10T06:59:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27576.106371 secs 2023-09-10T06:59:04.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27575.628444 secs 2023-09-10T06:59:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27576.106371 secs 2023-09-10T06:59:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27575.628444 secs 2023-09-10T06:59:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27576.106371 secs 2023-09-10T06:59:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:05 smithi170 ceph-mon[102832]: pgmap v14866: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 319 KiB/s wr, 25 op/s 2023-09-10T06:59:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:05 smithi082 ceph-mon[98694]: pgmap v14866: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 319 KiB/s wr, 25 op/s 2023-09-10T06:59:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:05 smithi098 ceph-mon[103124]: pgmap v14866: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 319 KiB/s wr, 25 op/s 2023-09-10T06:59:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:07 smithi170 ceph-mon[102832]: pgmap v14867: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 383 KiB/s wr, 29 op/s 2023-09-10T06:59:07.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:07 smithi082 ceph-mon[98694]: pgmap v14867: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 383 KiB/s wr, 29 op/s 2023-09-10T06:59:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:07 smithi098 ceph-mon[103124]: pgmap v14867: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 383 KiB/s wr, 29 op/s 2023-09-10T06:59:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27579.489130 secs 2023-09-10T06:59:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:09 smithi170 ceph-mon[102832]: osdmap e997: 12 total, 12 up, 12 in 2023-09-10T06:59:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27580.628517 secs 2023-09-10T06:59:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:09 smithi170 ceph-mon[102832]: pgmap v14869: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 214 KiB/s wr, 28 op/s 2023-09-10T06:59:09.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27579.489130 secs 2023-09-10T06:59:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:09 smithi082 ceph-mon[98694]: osdmap e997: 12 total, 12 up, 12 in 2023-09-10T06:59:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27580.628517 secs 2023-09-10T06:59:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:09 smithi082 ceph-mon[98694]: pgmap v14869: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 214 KiB/s wr, 28 op/s 2023-09-10T06:59:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27579.489130 secs 2023-09-10T06:59:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:09 smithi098 ceph-mon[103124]: osdmap e997: 12 total, 12 up, 12 in 2023-09-10T06:59:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27580.628517 secs 2023-09-10T06:59:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:09 smithi098 ceph-mon[103124]: pgmap v14869: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 214 KiB/s wr, 28 op/s 2023-09-10T06:59:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27581.106437 secs 2023-09-10T06:59:10.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27581.106437 secs 2023-09-10T06:59:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27581.106437 secs 2023-09-10T06:59:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:11 smithi170 ceph-mon[102832]: pgmap v14870: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 78 KiB/s wr, 10 op/s 2023-09-10T06:59:11.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:11 smithi082 ceph-mon[98694]: pgmap v14870: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 78 KiB/s wr, 10 op/s 2023-09-10T06:59:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:11 smithi098 ceph-mon[103124]: pgmap v14870: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 78 KiB/s wr, 10 op/s 2023-09-10T06:59:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:13 smithi170 ceph-mon[102832]: pgmap v14871: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 78 KiB/s wr, 10 op/s 2023-09-10T06:59:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27584.489193 secs 2023-09-10T06:59:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:13 smithi082 ceph-mon[98694]: pgmap v14871: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 78 KiB/s wr, 10 op/s 2023-09-10T06:59:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27584.489193 secs 2023-09-10T06:59:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:13 smithi098 ceph-mon[103124]: pgmap v14871: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 78 KiB/s wr, 10 op/s 2023-09-10T06:59:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27584.489193 secs 2023-09-10T06:59:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27585.628601 secs 2023-09-10T06:59:14.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27585.628601 secs 2023-09-10T06:59:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27585.628601 secs 2023-09-10T06:59:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27586.106480 secs 2023-09-10T06:59:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:15 smithi170 ceph-mon[102832]: pgmap v14872: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 78 KiB/s wr, 10 op/s 2023-09-10T06:59:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27586.106480 secs 2023-09-10T06:59:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:15 smithi082 ceph-mon[98694]: pgmap v14872: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 78 KiB/s wr, 10 op/s 2023-09-10T06:59:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27586.106480 secs 2023-09-10T06:59:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:15 smithi098 ceph-mon[103124]: pgmap v14872: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 78 KiB/s wr, 10 op/s 2023-09-10T06:59:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:17 smithi170 ceph-mon[102832]: osdmap e998: 12 total, 12 up, 12 in 2023-09-10T06:59:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:17 smithi170 ceph-mon[102832]: pgmap v14874: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 376 B/s wr, 12 op/s 2023-09-10T06:59:17.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:17 smithi082 ceph-mon[98694]: osdmap e998: 12 total, 12 up, 12 in 2023-09-10T06:59:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:17 smithi082 ceph-mon[98694]: pgmap v14874: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 376 B/s wr, 12 op/s 2023-09-10T06:59:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:17 smithi098 ceph-mon[103124]: osdmap e998: 12 total, 12 up, 12 in 2023-09-10T06:59:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:17 smithi098 ceph-mon[103124]: pgmap v14874: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 376 B/s wr, 12 op/s 2023-09-10T06:59:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27589.489254 secs 2023-09-10T06:59:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27589.489254 secs 2023-09-10T06:59:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27589.489254 secs 2023-09-10T06:59:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27590.628658 secs 2023-09-10T06:59:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:19 smithi082 ceph-mon[98694]: pgmap v14875: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T06:59:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27590.628658 secs 2023-09-10T06:59:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:19 smithi098 ceph-mon[103124]: pgmap v14875: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T06:59:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27590.628658 secs 2023-09-10T06:59:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:19 smithi170 ceph-mon[102832]: pgmap v14875: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T06:59:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27591.106546 secs 2023-09-10T06:59:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27591.106546 secs 2023-09-10T06:59:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27591.106546 secs 2023-09-10T06:59:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:21 smithi082 ceph-mon[98694]: pgmap v14876: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 10 op/s 2023-09-10T06:59:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:21 smithi098 ceph-mon[103124]: pgmap v14876: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 10 op/s 2023-09-10T06:59:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:21 smithi170 ceph-mon[102832]: pgmap v14876: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 10 op/s 2023-09-10T06:59:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:23 smithi170 ceph-mon[102832]: pgmap v14877: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 10 op/s 2023-09-10T06:59:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27594.489313 secs 2023-09-10T06:59:23.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:23 smithi082 ceph-mon[98694]: pgmap v14877: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 10 op/s 2023-09-10T06:59:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27594.489313 secs 2023-09-10T06:59:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:23 smithi098 ceph-mon[103124]: pgmap v14877: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 10 op/s 2023-09-10T06:59:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27594.489313 secs 2023-09-10T06:59:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27595.628715 secs 2023-09-10T06:59:24.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27595.628715 secs 2023-09-10T06:59:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27595.628715 secs 2023-09-10T06:59:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27596.106606 secs 2023-09-10T06:59:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:25 smithi170 ceph-mon[102832]: pgmap v14878: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 10 op/s 2023-09-10T06:59:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27596.106606 secs 2023-09-10T06:59:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:25 smithi082 ceph-mon[98694]: pgmap v14878: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 10 op/s 2023-09-10T06:59:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27596.106606 secs 2023-09-10T06:59:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:25 smithi098 ceph-mon[103124]: pgmap v14878: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 10 op/s 2023-09-10T06:59:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:27 smithi170 ceph-mon[102832]: pgmap v14879: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T06:59:27.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:27 smithi082 ceph-mon[98694]: pgmap v14879: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T06:59:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:27 smithi098 ceph-mon[103124]: pgmap v14879: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T06:59:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27600.628774 secs 2023-09-10T06:59:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:29 smithi170 ceph-mon[102832]: pgmap v14880: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 9 op/s 2023-09-10T06:59:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27600.628774 secs 2023-09-10T06:59:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:29 smithi082 ceph-mon[98694]: pgmap v14880: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 9 op/s 2023-09-10T06:59:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27600.628774 secs 2023-09-10T06:59:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:29 smithi098 ceph-mon[103124]: pgmap v14880: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 9 op/s 2023-09-10T06:59:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27601.106672 secs 2023-09-10T06:59:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27601.106672 secs 2023-09-10T06:59:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27601.106672 secs 2023-09-10T06:59:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:31 smithi170 ceph-mon[102832]: pgmap v14881: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 189 KiB/s wr, 14 op/s 2023-09-10T06:59:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:59:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:31 smithi082 ceph-mon[98694]: pgmap v14881: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 189 KiB/s wr, 14 op/s 2023-09-10T06:59:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:59:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:31 smithi098 ceph-mon[103124]: pgmap v14881: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 189 KiB/s wr, 14 op/s 2023-09-10T06:59:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T06:59:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:59:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:59:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:59:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:59:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:59:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:59:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:59:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:59:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:59:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:33 smithi170 ceph-mon[102832]: pgmap v14882: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 189 KiB/s wr, 10 op/s 2023-09-10T06:59:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:33 smithi082 ceph-mon[98694]: pgmap v14882: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 189 KiB/s wr, 10 op/s 2023-09-10T06:59:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:33 smithi098 ceph-mon[103124]: pgmap v14882: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 189 KiB/s wr, 10 op/s 2023-09-10T06:59:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27605.628834 secs 2023-09-10T06:59:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:59:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:59:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27606.106732 secs 2023-09-10T06:59:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27605.628834 secs 2023-09-10T06:59:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:59:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:59:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27606.106732 secs 2023-09-10T06:59:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27605.628834 secs 2023-09-10T06:59:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:59:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:59:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27606.106732 secs 2023-09-10T06:59:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:35 smithi170 ceph-mon[102832]: pgmap v14883: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 189 KiB/s wr, 10 op/s 2023-09-10T06:59:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:59:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:59:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:59:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:59:35.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:35 smithi082 ceph-mon[98694]: pgmap v14883: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 189 KiB/s wr, 10 op/s 2023-09-10T06:59:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:59:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:59:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:59:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:59:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:35 smithi098 ceph-mon[103124]: pgmap v14883: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 189 KiB/s wr, 10 op/s 2023-09-10T06:59:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T06:59:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T06:59:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T06:59:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T06:59:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:37 smithi170 ceph-mon[102832]: pgmap v14884: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 303 KiB/s wr, 15 op/s 2023-09-10T06:59:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:37 smithi082 ceph-mon[98694]: pgmap v14884: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 303 KiB/s wr, 15 op/s 2023-09-10T06:59:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:37 smithi098 ceph-mon[103124]: pgmap v14884: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 303 KiB/s wr, 15 op/s 2023-09-10T06:59:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27610.628895 secs 2023-09-10T06:59:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:39 smithi170 ceph-mon[102832]: pgmap v14885: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 208 KiB/s wr, 10 op/s 2023-09-10T06:59:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27610.628895 secs 2023-09-10T06:59:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:39 smithi082 ceph-mon[98694]: pgmap v14885: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 208 KiB/s wr, 10 op/s 2023-09-10T06:59:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27610.628895 secs 2023-09-10T06:59:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:39 smithi098 ceph-mon[103124]: pgmap v14885: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 208 KiB/s wr, 10 op/s 2023-09-10T06:59:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27611.106797 secs 2023-09-10T06:59:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27611.106797 secs 2023-09-10T06:59:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27611.106797 secs 2023-09-10T06:59:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:41 smithi170 ceph-mon[102832]: pgmap v14886: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 248 KiB/s wr, 16 op/s 2023-09-10T06:59:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:41 smithi082 ceph-mon[98694]: pgmap v14886: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 248 KiB/s wr, 16 op/s 2023-09-10T06:59:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:41 smithi098 ceph-mon[103124]: pgmap v14886: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 248 KiB/s wr, 16 op/s 2023-09-10T06:59:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:43 smithi170 ceph-mon[102832]: pgmap v14887: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T06:59:43.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:43 smithi082 ceph-mon[98694]: pgmap v14887: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T06:59:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:43 smithi098 ceph-mon[103124]: pgmap v14887: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T06:59:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27615.628952 secs 2023-09-10T06:59:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:59:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:59:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27615.628952 secs 2023-09-10T06:59:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:59:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:59:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27615.628952 secs 2023-09-10T06:59:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:59:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:59:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27616.106864 secs 2023-09-10T06:59:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:45 smithi170 ceph-mon[102832]: pgmap v14888: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T06:59:45.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27616.106864 secs 2023-09-10T06:59:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:45 smithi082 ceph-mon[98694]: pgmap v14888: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T06:59:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27616.106864 secs 2023-09-10T06:59:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:45 smithi098 ceph-mon[103124]: pgmap v14888: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T06:59:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:59:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:59:46.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:59:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:59:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:59:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:59:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:47 smithi170 ceph-mon[102832]: pgmap v14889: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 533 KiB/s wr, 18 op/s 2023-09-10T06:59:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:47 smithi082 ceph-mon[98694]: pgmap v14889: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 533 KiB/s wr, 18 op/s 2023-09-10T06:59:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:47 smithi098 ceph-mon[103124]: pgmap v14889: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 533 KiB/s wr, 18 op/s 2023-09-10T06:59:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27620.629010 secs 2023-09-10T06:59:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:49 smithi170 ceph-mon[102832]: pgmap v14890: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 419 KiB/s wr, 12 op/s 2023-09-10T06:59:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27621.106926 secs 2023-09-10T06:59:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27620.629010 secs 2023-09-10T06:59:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:49 smithi082 ceph-mon[98694]: pgmap v14890: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 419 KiB/s wr, 12 op/s 2023-09-10T06:59:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27621.106926 secs 2023-09-10T06:59:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27620.629010 secs 2023-09-10T06:59:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:49 smithi098 ceph-mon[103124]: pgmap v14890: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 419 KiB/s wr, 12 op/s 2023-09-10T06:59:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27621.106926 secs 2023-09-10T06:59:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:51 smithi170 ceph-mon[102832]: pgmap v14891: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 444 KiB/s wr, 18 op/s 2023-09-10T06:59:51.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:51 smithi082 ceph-mon[98694]: pgmap v14891: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 444 KiB/s wr, 18 op/s 2023-09-10T06:59:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:51 smithi098 ceph-mon[103124]: pgmap v14891: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 444 KiB/s wr, 18 op/s 2023-09-10T06:59:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:53 smithi170 ceph-mon[102832]: pgmap v14892: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 404 KiB/s wr, 13 op/s 2023-09-10T06:59:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27624.489727 secs 2023-09-10T06:59:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:53 smithi082 ceph-mon[98694]: pgmap v14892: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 404 KiB/s wr, 13 op/s 2023-09-10T06:59:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27624.489727 secs 2023-09-10T06:59:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:53 smithi098 ceph-mon[103124]: pgmap v14892: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 404 KiB/s wr, 13 op/s 2023-09-10T06:59:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27624.489727 secs 2023-09-10T06:59:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27625.629069 secs 2023-09-10T06:59:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27626.106988 secs 2023-09-10T06:59:54.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27625.629069 secs 2023-09-10T06:59:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27626.106988 secs 2023-09-10T06:59:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27625.629069 secs 2023-09-10T06:59:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27626.106988 secs 2023-09-10T06:59:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:55 smithi170 ceph-mon[102832]: pgmap v14893: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 404 KiB/s wr, 13 op/s 2023-09-10T06:59:55.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:55 smithi082 ceph-mon[98694]: pgmap v14893: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 404 KiB/s wr, 13 op/s 2023-09-10T06:59:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:55 smithi098 ceph-mon[103124]: pgmap v14893: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 404 KiB/s wr, 13 op/s 2023-09-10T06:59:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:59:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:59:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:59:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:59:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:59:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:59:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T06:59:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T06:59:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:59:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:57 smithi170 ceph-mon[102832]: pgmap v14894: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 643 KiB/s wr, 19 op/s 2023-09-10T06:59:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:57 smithi082 ceph-mon[98694]: pgmap v14894: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 643 KiB/s wr, 19 op/s 2023-09-10T06:59:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:57 smithi098 ceph-mon[103124]: pgmap v14894: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 643 KiB/s wr, 19 op/s 2023-09-10T06:59:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:59:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:59:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27629.489813 secs 2023-09-10T06:59:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:59:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:59:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27629.489813 secs 2023-09-10T06:59:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T06:59:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T06:59:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27629.489813 secs 2023-09-10T06:59:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27630.629125 secs 2023-09-10T06:59:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:59 smithi170 ceph-mon[102832]: pgmap v14895: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T06:59:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 06:59:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27631.107053 secs 2023-09-10T06:59:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27630.629125 secs 2023-09-10T06:59:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:59 smithi082 ceph-mon[98694]: pgmap v14895: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T06:59:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 06:59:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27631.107053 secs 2023-09-10T07:00:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27630.629125 secs 2023-09-10T07:00:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:59 smithi098 ceph-mon[103124]: pgmap v14895: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 264 KiB/s wr, 12 op/s 2023-09-10T07:00:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 06:59:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27631.107053 secs 2023-09-10T07:00:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:00:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:00:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T07:00:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:00:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:00:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T07:00:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:00:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:00:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T07:00:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:01 smithi170 ceph-mon[102832]: pgmap v14896: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 350 KiB/s wr, 19 op/s 2023-09-10T07:00:01.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:01 smithi082 ceph-mon[98694]: pgmap v14896: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 350 KiB/s wr, 19 op/s 2023-09-10T07:00:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:01 smithi098 ceph-mon[103124]: pgmap v14896: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 350 KiB/s wr, 19 op/s 2023-09-10T07:00:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:03 smithi170 ceph-mon[102832]: pgmap v14897: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 325 KiB/s wr, 13 op/s 2023-09-10T07:00:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27634.489871 secs 2023-09-10T07:00:03.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:03 smithi082 ceph-mon[98694]: pgmap v14897: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 325 KiB/s wr, 13 op/s 2023-09-10T07:00:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27634.489871 secs 2023-09-10T07:00:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:03 smithi098 ceph-mon[103124]: pgmap v14897: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 325 KiB/s wr, 13 op/s 2023-09-10T07:00:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27634.489871 secs 2023-09-10T07:00:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27635.629182 secs 2023-09-10T07:00:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27636.107115 secs 2023-09-10T07:00:04.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27635.629182 secs 2023-09-10T07:00:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27636.107115 secs 2023-09-10T07:00:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27635.629182 secs 2023-09-10T07:00:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27636.107115 secs 2023-09-10T07:00:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:05 smithi170 ceph-mon[102832]: pgmap v14898: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 325 KiB/s wr, 13 op/s 2023-09-10T07:00:05.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:05 smithi082 ceph-mon[98694]: pgmap v14898: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 325 KiB/s wr, 13 op/s 2023-09-10T07:00:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:05 smithi098 ceph-mon[103124]: pgmap v14898: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 325 KiB/s wr, 13 op/s 2023-09-10T07:00:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:07 smithi170 ceph-mon[102832]: pgmap v14899: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 358 KiB/s wr, 30 op/s 2023-09-10T07:00:07.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:07 smithi082 ceph-mon[98694]: pgmap v14899: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 358 KiB/s wr, 30 op/s 2023-09-10T07:00:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:07 smithi098 ceph-mon[103124]: pgmap v14899: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 358 KiB/s wr, 30 op/s 2023-09-10T07:00:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27639.489932 secs 2023-09-10T07:00:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:09 smithi170 ceph-mon[102832]: osdmap e999: 12 total, 12 up, 12 in 2023-09-10T07:00:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27640.629241 secs 2023-09-10T07:00:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:09 smithi170 ceph-mon[102832]: pgmap v14901: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 143 KiB/s wr, 28 op/s 2023-09-10T07:00:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27639.489932 secs 2023-09-10T07:00:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:09 smithi082 ceph-mon[98694]: osdmap e999: 12 total, 12 up, 12 in 2023-09-10T07:00:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27640.629241 secs 2023-09-10T07:00:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:09 smithi082 ceph-mon[98694]: pgmap v14901: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 143 KiB/s wr, 28 op/s 2023-09-10T07:00:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27639.489932 secs 2023-09-10T07:00:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:09 smithi098 ceph-mon[103124]: osdmap e999: 12 total, 12 up, 12 in 2023-09-10T07:00:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27640.629241 secs 2023-09-10T07:00:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:09 smithi098 ceph-mon[103124]: pgmap v14901: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 143 KiB/s wr, 28 op/s 2023-09-10T07:00:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27641.107189 secs 2023-09-10T07:00:10.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27641.107189 secs 2023-09-10T07:00:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27641.107189 secs 2023-09-10T07:00:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:11 smithi170 ceph-mon[102832]: pgmap v14902: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 40 KiB/s wr, 25 op/s 2023-09-10T07:00:11.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:11 smithi082 ceph-mon[98694]: pgmap v14902: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 40 KiB/s wr, 25 op/s 2023-09-10T07:00:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:11 smithi098 ceph-mon[103124]: pgmap v14902: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 40 KiB/s wr, 25 op/s 2023-09-10T07:00:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:13 smithi170 ceph-mon[102832]: pgmap v14903: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 40 KiB/s wr, 25 op/s 2023-09-10T07:00:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27644.489992 secs 2023-09-10T07:00:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:13 smithi082 ceph-mon[98694]: pgmap v14903: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 40 KiB/s wr, 25 op/s 2023-09-10T07:00:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27644.489992 secs 2023-09-10T07:00:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:13 smithi098 ceph-mon[103124]: pgmap v14903: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 40 KiB/s wr, 25 op/s 2023-09-10T07:00:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27644.489992 secs 2023-09-10T07:00:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27645.629299 secs 2023-09-10T07:00:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27645.629299 secs 2023-09-10T07:00:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27645.629299 secs 2023-09-10T07:00:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27646.107251 secs 2023-09-10T07:00:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:15 smithi170 ceph-mon[102832]: pgmap v14904: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 40 KiB/s wr, 25 op/s 2023-09-10T07:00:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27646.107251 secs 2023-09-10T07:00:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:15 smithi082 ceph-mon[98694]: pgmap v14904: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 40 KiB/s wr, 25 op/s 2023-09-10T07:00:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27646.107251 secs 2023-09-10T07:00:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:15 smithi098 ceph-mon[103124]: pgmap v14904: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 40 KiB/s wr, 25 op/s 2023-09-10T07:00:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:17 smithi170 ceph-mon[102832]: osdmap e1000: 12 total, 12 up, 12 in 2023-09-10T07:00:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:17 smithi170 ceph-mon[102832]: pgmap v14906: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 251 B/s wr, 12 op/s 2023-09-10T07:00:17.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:17 smithi082 ceph-mon[98694]: osdmap e1000: 12 total, 12 up, 12 in 2023-09-10T07:00:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:17 smithi082 ceph-mon[98694]: pgmap v14906: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 251 B/s wr, 12 op/s 2023-09-10T07:00:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:17 smithi098 ceph-mon[103124]: osdmap e1000: 12 total, 12 up, 12 in 2023-09-10T07:00:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:17 smithi098 ceph-mon[103124]: pgmap v14906: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 251 B/s wr, 12 op/s 2023-09-10T07:00:18.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27649.490050 secs 2023-09-10T07:00:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27649.490050 secs 2023-09-10T07:00:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27649.490050 secs 2023-09-10T07:00:19.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27650.629387 secs 2023-09-10T07:00:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:19 smithi082 ceph-mon[98694]: pgmap v14907: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:00:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27650.629387 secs 2023-09-10T07:00:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:19 smithi098 ceph-mon[103124]: pgmap v14907: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:00:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27650.629387 secs 2023-09-10T07:00:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:19 smithi170 ceph-mon[102832]: pgmap v14907: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:00:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27651.107315 secs 2023-09-10T07:00:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27651.107315 secs 2023-09-10T07:00:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27651.107315 secs 2023-09-10T07:00:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:21 smithi082 ceph-mon[98694]: pgmap v14908: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:00:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:21 smithi098 ceph-mon[103124]: pgmap v14908: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:00:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:21 smithi170 ceph-mon[102832]: pgmap v14908: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:00:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:23 smithi170 ceph-mon[102832]: pgmap v14909: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:00:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27654.490114 secs 2023-09-10T07:00:23.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:23 smithi082 ceph-mon[98694]: pgmap v14909: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:00:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27654.490114 secs 2023-09-10T07:00:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:23 smithi098 ceph-mon[103124]: pgmap v14909: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:00:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27654.490114 secs 2023-09-10T07:00:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27655.629475 secs 2023-09-10T07:00:24.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27655.629475 secs 2023-09-10T07:00:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27655.629475 secs 2023-09-10T07:00:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27656.107375 secs 2023-09-10T07:00:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:25 smithi170 ceph-mon[102832]: pgmap v14910: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:00:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27656.107375 secs 2023-09-10T07:00:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:25 smithi082 ceph-mon[98694]: pgmap v14910: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:00:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27656.107375 secs 2023-09-10T07:00:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:25 smithi098 ceph-mon[103124]: pgmap v14910: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:00:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:27 smithi170 ceph-mon[102832]: pgmap v14911: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 100 B/s wr, 10 op/s 2023-09-10T07:00:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:27 smithi082 ceph-mon[98694]: pgmap v14911: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 100 B/s wr, 10 op/s 2023-09-10T07:00:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:27 smithi098 ceph-mon[103124]: pgmap v14911: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 100 B/s wr, 10 op/s 2023-09-10T07:00:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27660.629497 secs 2023-09-10T07:00:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:29 smithi170 ceph-mon[102832]: pgmap v14912: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:00:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27660.629497 secs 2023-09-10T07:00:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:29 smithi082 ceph-mon[98694]: pgmap v14912: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:00:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27660.629497 secs 2023-09-10T07:00:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:29 smithi098 ceph-mon[103124]: pgmap v14912: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:00:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27661.107440 secs 2023-09-10T07:00:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27661.107440 secs 2023-09-10T07:00:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27661.107440 secs 2023-09-10T07:00:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:31 smithi170 ceph-mon[102832]: pgmap v14913: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 94 KiB/s wr, 13 op/s 2023-09-10T07:00:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:00:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:31 smithi082 ceph-mon[98694]: pgmap v14913: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 94 KiB/s wr, 13 op/s 2023-09-10T07:00:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:00:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:31 smithi098 ceph-mon[103124]: pgmap v14913: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 94 KiB/s wr, 13 op/s 2023-09-10T07:00:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:00:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:00:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:00:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:00:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:00:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:00:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:00:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:00:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:00:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:00:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:33 smithi170 ceph-mon[102832]: pgmap v14914: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 9 op/s 2023-09-10T07:00:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:33 smithi082 ceph-mon[98694]: pgmap v14914: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 9 op/s 2023-09-10T07:00:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:33 smithi098 ceph-mon[103124]: pgmap v14914: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 9 op/s 2023-09-10T07:00:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27665.629575 secs 2023-09-10T07:00:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27666.107487 secs 2023-09-10T07:00:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27665.629575 secs 2023-09-10T07:00:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27666.107487 secs 2023-09-10T07:00:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27665.629575 secs 2023-09-10T07:00:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27666.107487 secs 2023-09-10T07:00:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:35 smithi170 ceph-mon[102832]: pgmap v14915: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 9 op/s 2023-09-10T07:00:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:00:35.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:35 smithi082 ceph-mon[98694]: pgmap v14915: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 9 op/s 2023-09-10T07:00:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:00:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:35 smithi098 ceph-mon[103124]: pgmap v14915: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 9 op/s 2023-09-10T07:00:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:00:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:00:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:00:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:00:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:00:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:00:36.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:00:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:00:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:00:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:00:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:00:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:00:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:00:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:00:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:00:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:00:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:37 smithi170 ceph-mon[102832]: pgmap v14916: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 329 KiB/s wr, 16 op/s 2023-09-10T07:00:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:37 smithi082 ceph-mon[98694]: pgmap v14916: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 329 KiB/s wr, 16 op/s 2023-09-10T07:00:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:37 smithi098 ceph-mon[103124]: pgmap v14916: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 329 KiB/s wr, 16 op/s 2023-09-10T07:00:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:00:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:00:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:00:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:00:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:00:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:00:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27670.629695 secs 2023-09-10T07:00:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:39 smithi170 ceph-mon[102832]: pgmap v14917: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 329 KiB/s wr, 12 op/s 2023-09-10T07:00:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27671.107549 secs 2023-09-10T07:00:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27670.629695 secs 2023-09-10T07:00:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:39 smithi082 ceph-mon[98694]: pgmap v14917: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 329 KiB/s wr, 12 op/s 2023-09-10T07:00:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27671.107549 secs 2023-09-10T07:00:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27670.629695 secs 2023-09-10T07:00:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:39 smithi098 ceph-mon[103124]: pgmap v14917: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 329 KiB/s wr, 12 op/s 2023-09-10T07:00:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27671.107549 secs 2023-09-10T07:00:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:41 smithi170 ceph-mon[102832]: pgmap v14918: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 350 KiB/s wr, 17 op/s 2023-09-10T07:00:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:41 smithi082 ceph-mon[98694]: pgmap v14918: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 350 KiB/s wr, 17 op/s 2023-09-10T07:00:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:41 smithi098 ceph-mon[103124]: pgmap v14918: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 350 KiB/s wr, 17 op/s 2023-09-10T07:00:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:43 smithi170 ceph-mon[102832]: pgmap v14919: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 256 KiB/s wr, 12 op/s 2023-09-10T07:00:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:43 smithi082 ceph-mon[98694]: pgmap v14919: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 256 KiB/s wr, 12 op/s 2023-09-10T07:00:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:43 smithi098 ceph-mon[103124]: pgmap v14919: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 256 KiB/s wr, 12 op/s 2023-09-10T07:00:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27675.629769 secs 2023-09-10T07:00:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:00:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:00:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27676.107612 secs 2023-09-10T07:00:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27675.629769 secs 2023-09-10T07:00:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:00:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:00:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27676.107612 secs 2023-09-10T07:00:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27675.629769 secs 2023-09-10T07:00:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:00:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:00:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27676.107612 secs 2023-09-10T07:00:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:45 smithi170 ceph-mon[102832]: pgmap v14920: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 256 KiB/s wr, 12 op/s 2023-09-10T07:00:45.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:45 smithi082 ceph-mon[98694]: pgmap v14920: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 256 KiB/s wr, 12 op/s 2023-09-10T07:00:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:45 smithi098 ceph-mon[103124]: pgmap v14920: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 256 KiB/s wr, 12 op/s 2023-09-10T07:00:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:00:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:00:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:00:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:00:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:00:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:00:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:47 smithi170 ceph-mon[102832]: pgmap v14921: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 640 KiB/s wr, 19 op/s 2023-09-10T07:00:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:47 smithi082 ceph-mon[98694]: pgmap v14921: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 640 KiB/s wr, 19 op/s 2023-09-10T07:00:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:47 smithi098 ceph-mon[103124]: pgmap v14921: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 640 KiB/s wr, 19 op/s 2023-09-10T07:00:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27680.629828 secs 2023-09-10T07:00:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:49 smithi170 ceph-mon[102832]: pgmap v14922: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 405 KiB/s wr, 12 op/s 2023-09-10T07:00:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27681.107676 secs 2023-09-10T07:00:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27680.629828 secs 2023-09-10T07:00:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:49 smithi082 ceph-mon[98694]: pgmap v14922: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 405 KiB/s wr, 12 op/s 2023-09-10T07:00:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27681.107676 secs 2023-09-10T07:00:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27680.629828 secs 2023-09-10T07:00:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:49 smithi098 ceph-mon[103124]: pgmap v14922: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 405 KiB/s wr, 12 op/s 2023-09-10T07:00:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27681.107676 secs 2023-09-10T07:00:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:51 smithi170 ceph-mon[102832]: pgmap v14923: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 701 KiB/s wr, 19 op/s 2023-09-10T07:00:51.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:51 smithi082 ceph-mon[98694]: pgmap v14923: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 701 KiB/s wr, 19 op/s 2023-09-10T07:00:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:51 smithi098 ceph-mon[103124]: pgmap v14923: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 701 KiB/s wr, 19 op/s 2023-09-10T07:00:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:53 smithi170 ceph-mon[102832]: pgmap v14924: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 681 KiB/s wr, 13 op/s 2023-09-10T07:00:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27684.490547 secs 2023-09-10T07:00:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:53 smithi082 ceph-mon[98694]: pgmap v14924: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 681 KiB/s wr, 13 op/s 2023-09-10T07:00:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27684.490547 secs 2023-09-10T07:00:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:53 smithi098 ceph-mon[103124]: pgmap v14924: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 681 KiB/s wr, 13 op/s 2023-09-10T07:00:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27684.490547 secs 2023-09-10T07:00:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27685.629887 secs 2023-09-10T07:00:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:00:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:00:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27686.107742 secs 2023-09-10T07:00:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27685.629887 secs 2023-09-10T07:00:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:00:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:00:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27686.107742 secs 2023-09-10T07:00:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27685.629887 secs 2023-09-10T07:00:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:00:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:00:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27686.107742 secs 2023-09-10T07:00:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:55 smithi170 ceph-mon[102832]: pgmap v14925: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 681 KiB/s wr, 13 op/s 2023-09-10T07:00:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:55 smithi082 ceph-mon[98694]: pgmap v14925: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 681 KiB/s wr, 13 op/s 2023-09-10T07:00:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:55 smithi098 ceph-mon[103124]: pgmap v14925: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 681 KiB/s wr, 13 op/s 2023-09-10T07:00:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:00:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:00:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:00:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:00:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:00:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:00:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:00:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:00:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:00:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:57 smithi082 ceph-mon[98694]: pgmap v14926: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 977 KiB/s wr, 20 op/s 2023-09-10T07:00:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:57 smithi098 ceph-mon[103124]: pgmap v14926: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 977 KiB/s wr, 20 op/s 2023-09-10T07:00:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:57 smithi170 ceph-mon[102832]: pgmap v14926: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 977 KiB/s wr, 20 op/s 2023-09-10T07:00:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:00:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:00:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27689.490616 secs 2023-09-10T07:00:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:00:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:00:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27689.490616 secs 2023-09-10T07:00:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:00:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:00:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27689.490616 secs 2023-09-10T07:00:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27690.629946 secs 2023-09-10T07:00:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:59 smithi082 ceph-mon[98694]: pgmap v14927: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 593 KiB/s wr, 13 op/s 2023-09-10T07:00:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:00:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27691.107804 secs 2023-09-10T07:01:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27690.629946 secs 2023-09-10T07:01:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:59 smithi098 ceph-mon[103124]: pgmap v14927: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 593 KiB/s wr, 13 op/s 2023-09-10T07:01:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:00:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27691.107804 secs 2023-09-10T07:01:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27690.629946 secs 2023-09-10T07:01:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:59 smithi170 ceph-mon[102832]: pgmap v14927: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 593 KiB/s wr, 13 op/s 2023-09-10T07:01:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:00:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27691.107804 secs 2023-09-10T07:01:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:01:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:01:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:01:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:01:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:01:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:01:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:01 smithi082 ceph-mon[98694]: pgmap v14928: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 710 KiB/s wr, 18 op/s 2023-09-10T07:01:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:01 smithi098 ceph-mon[103124]: pgmap v14928: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 710 KiB/s wr, 18 op/s 2023-09-10T07:01:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:01 smithi170 ceph-mon[102832]: pgmap v14928: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 710 KiB/s wr, 18 op/s 2023-09-10T07:01:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:03 smithi082 ceph-mon[98694]: pgmap v14929: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 414 KiB/s wr, 12 op/s 2023-09-10T07:01:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27694.490673 secs 2023-09-10T07:01:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:03 smithi098 ceph-mon[103124]: pgmap v14929: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 414 KiB/s wr, 12 op/s 2023-09-10T07:01:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27694.490673 secs 2023-09-10T07:01:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:03 smithi170 ceph-mon[102832]: pgmap v14929: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 414 KiB/s wr, 12 op/s 2023-09-10T07:01:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27694.490673 secs 2023-09-10T07:01:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27695.630021 secs 2023-09-10T07:01:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27696.107867 secs 2023-09-10T07:01:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27695.630021 secs 2023-09-10T07:01:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27696.107867 secs 2023-09-10T07:01:05.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27695.630021 secs 2023-09-10T07:01:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27696.107867 secs 2023-09-10T07:01:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:05 smithi098 ceph-mon[103124]: pgmap v14930: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 414 KiB/s wr, 12 op/s 2023-09-10T07:01:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:05 smithi170 ceph-mon[102832]: pgmap v14930: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 414 KiB/s wr, 12 op/s 2023-09-10T07:01:06.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:05 smithi082 ceph-mon[98694]: pgmap v14930: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 414 KiB/s wr, 12 op/s 2023-09-10T07:01:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:07 smithi170 ceph-mon[102832]: pgmap v14931: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 533 KiB/s wr, 32 op/s 2023-09-10T07:01:07.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:07 smithi082 ceph-mon[98694]: pgmap v14931: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 533 KiB/s wr, 32 op/s 2023-09-10T07:01:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:07 smithi098 ceph-mon[103124]: pgmap v14931: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 533 KiB/s wr, 32 op/s 2023-09-10T07:01:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27699.490735 secs 2023-09-10T07:01:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27700.630077 secs 2023-09-10T07:01:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:09 smithi170 ceph-mon[102832]: osdmap e1001: 12 total, 12 up, 12 in 2023-09-10T07:01:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:09 smithi170 ceph-mon[102832]: pgmap v14933: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 284 KiB/s wr, 30 op/s 2023-09-10T07:01:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27699.490735 secs 2023-09-10T07:01:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27700.630077 secs 2023-09-10T07:01:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:09 smithi082 ceph-mon[98694]: osdmap e1001: 12 total, 12 up, 12 in 2023-09-10T07:01:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:09 smithi082 ceph-mon[98694]: pgmap v14933: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 284 KiB/s wr, 30 op/s 2023-09-10T07:01:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27699.490735 secs 2023-09-10T07:01:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27700.630077 secs 2023-09-10T07:01:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:09 smithi098 ceph-mon[103124]: osdmap e1001: 12 total, 12 up, 12 in 2023-09-10T07:01:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:09 smithi098 ceph-mon[103124]: pgmap v14933: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 284 KiB/s wr, 30 op/s 2023-09-10T07:01:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27701.107929 secs 2023-09-10T07:01:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27701.107929 secs 2023-09-10T07:01:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27701.107929 secs 2023-09-10T07:01:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:11 smithi170 ceph-mon[102832]: pgmap v14934: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 145 KiB/s wr, 28 op/s 2023-09-10T07:01:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:11 smithi082 ceph-mon[98694]: pgmap v14934: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 145 KiB/s wr, 28 op/s 2023-09-10T07:01:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:11 smithi098 ceph-mon[103124]: pgmap v14934: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 145 KiB/s wr, 28 op/s 2023-09-10T07:01:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:13 smithi170 ceph-mon[102832]: pgmap v14935: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 145 KiB/s wr, 28 op/s 2023-09-10T07:01:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27704.490775 secs 2023-09-10T07:01:13.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:13 smithi082 ceph-mon[98694]: pgmap v14935: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 145 KiB/s wr, 28 op/s 2023-09-10T07:01:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27704.490775 secs 2023-09-10T07:01:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:13 smithi098 ceph-mon[103124]: pgmap v14935: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 145 KiB/s wr, 28 op/s 2023-09-10T07:01:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27704.490775 secs 2023-09-10T07:01:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27705.630151 secs 2023-09-10T07:01:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27705.630151 secs 2023-09-10T07:01:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27705.630151 secs 2023-09-10T07:01:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27706.107993 secs 2023-09-10T07:01:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:15 smithi170 ceph-mon[102832]: pgmap v14936: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 145 KiB/s wr, 28 op/s 2023-09-10T07:01:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27706.107993 secs 2023-09-10T07:01:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:15 smithi082 ceph-mon[98694]: pgmap v14936: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 145 KiB/s wr, 28 op/s 2023-09-10T07:01:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27706.107993 secs 2023-09-10T07:01:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:15 smithi098 ceph-mon[103124]: pgmap v14936: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 145 KiB/s wr, 28 op/s 2023-09-10T07:01:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:17 smithi170 ceph-mon[102832]: osdmap e1002: 12 total, 12 up, 12 in 2023-09-10T07:01:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:17 smithi170 ceph-mon[102832]: pgmap v14938: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 205 KiB/s wr, 14 op/s 2023-09-10T07:01:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:17 smithi082 ceph-mon[98694]: osdmap e1002: 12 total, 12 up, 12 in 2023-09-10T07:01:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:17 smithi082 ceph-mon[98694]: pgmap v14938: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 205 KiB/s wr, 14 op/s 2023-09-10T07:01:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:17 smithi098 ceph-mon[103124]: osdmap e1002: 12 total, 12 up, 12 in 2023-09-10T07:01:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:17 smithi098 ceph-mon[103124]: pgmap v14938: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 205 KiB/s wr, 14 op/s 2023-09-10T07:01:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27709.490837 secs 2023-09-10T07:01:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27709.490837 secs 2023-09-10T07:01:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27709.490837 secs 2023-09-10T07:01:19.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27710.630209 secs 2023-09-10T07:01:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:19 smithi082 ceph-mon[98694]: pgmap v14939: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T07:01:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27710.630209 secs 2023-09-10T07:01:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:19 smithi098 ceph-mon[103124]: pgmap v14939: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T07:01:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27710.630209 secs 2023-09-10T07:01:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:19 smithi170 ceph-mon[102832]: pgmap v14939: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T07:01:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27711.108058 secs 2023-09-10T07:01:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27711.108058 secs 2023-09-10T07:01:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27711.108058 secs 2023-09-10T07:01:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:21 smithi098 ceph-mon[103124]: pgmap v14940: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 167 KiB/s wr, 12 op/s 2023-09-10T07:01:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:21 smithi170 ceph-mon[102832]: pgmap v14940: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 167 KiB/s wr, 12 op/s 2023-09-10T07:01:21.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:21 smithi082 ceph-mon[98694]: pgmap v14940: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 167 KiB/s wr, 12 op/s 2023-09-10T07:01:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:23 smithi170 ceph-mon[102832]: pgmap v14941: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 167 KiB/s wr, 12 op/s 2023-09-10T07:01:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27714.490948 secs 2023-09-10T07:01:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:23 smithi082 ceph-mon[98694]: pgmap v14941: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 167 KiB/s wr, 12 op/s 2023-09-10T07:01:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27714.490948 secs 2023-09-10T07:01:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:23 smithi098 ceph-mon[103124]: pgmap v14941: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 167 KiB/s wr, 12 op/s 2023-09-10T07:01:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27714.490948 secs 2023-09-10T07:01:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27715.630267 secs 2023-09-10T07:01:24.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27715.630267 secs 2023-09-10T07:01:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27715.630267 secs 2023-09-10T07:01:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27716.108144 secs 2023-09-10T07:01:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:25 smithi170 ceph-mon[102832]: pgmap v14942: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 167 KiB/s wr, 12 op/s 2023-09-10T07:01:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27716.108144 secs 2023-09-10T07:01:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:25 smithi082 ceph-mon[98694]: pgmap v14942: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 167 KiB/s wr, 12 op/s 2023-09-10T07:01:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27716.108144 secs 2023-09-10T07:01:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:25 smithi098 ceph-mon[103124]: pgmap v14942: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 167 KiB/s wr, 12 op/s 2023-09-10T07:01:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:27 smithi170 ceph-mon[102832]: pgmap v14943: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 705 B/s wr, 11 op/s 2023-09-10T07:01:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:27 smithi082 ceph-mon[98694]: pgmap v14943: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 705 B/s wr, 11 op/s 2023-09-10T07:01:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:27 smithi098 ceph-mon[103124]: pgmap v14943: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 705 B/s wr, 11 op/s 2023-09-10T07:01:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27720.630334 secs 2023-09-10T07:01:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:29 smithi170 ceph-mon[102832]: pgmap v14944: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2023-09-10T07:01:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27721.108210 secs 2023-09-10T07:01:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27720.630334 secs 2023-09-10T07:01:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:29 smithi082 ceph-mon[98694]: pgmap v14944: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2023-09-10T07:01:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27721.108210 secs 2023-09-10T07:01:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27720.630334 secs 2023-09-10T07:01:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:29 smithi098 ceph-mon[103124]: pgmap v14944: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2023-09-10T07:01:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27721.108210 secs 2023-09-10T07:01:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:01:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:01:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:01:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:01:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:01:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:01:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:31 smithi170 ceph-mon[102832]: pgmap v14945: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 190 KiB/s wr, 14 op/s 2023-09-10T07:01:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:01:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:31 smithi082 ceph-mon[98694]: pgmap v14945: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 190 KiB/s wr, 14 op/s 2023-09-10T07:01:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:01:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:31 smithi098 ceph-mon[103124]: pgmap v14945: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 190 KiB/s wr, 14 op/s 2023-09-10T07:01:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:01:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:01:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:01:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:01:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:01:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:01:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:01:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:01:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:01:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:01:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:33 smithi170 ceph-mon[102832]: pgmap v14946: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 190 KiB/s wr, 9 op/s 2023-09-10T07:01:33.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:33 smithi082 ceph-mon[98694]: pgmap v14946: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 190 KiB/s wr, 9 op/s 2023-09-10T07:01:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:33 smithi098 ceph-mon[103124]: pgmap v14946: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 190 KiB/s wr, 9 op/s 2023-09-10T07:01:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27725.630413 secs 2023-09-10T07:01:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27726.108271 secs 2023-09-10T07:01:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27725.630413 secs 2023-09-10T07:01:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27726.108271 secs 2023-09-10T07:01:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27725.630413 secs 2023-09-10T07:01:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27726.108271 secs 2023-09-10T07:01:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:35 smithi170 ceph-mon[102832]: pgmap v14947: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 190 KiB/s wr, 9 op/s 2023-09-10T07:01:35.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:35 smithi082 ceph-mon[98694]: pgmap v14947: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 190 KiB/s wr, 9 op/s 2023-09-10T07:01:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:35 smithi098 ceph-mon[103124]: pgmap v14947: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 190 KiB/s wr, 9 op/s 2023-09-10T07:01:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:01:36.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:01:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:01:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:37 smithi170 ceph-mon[102832]: pgmap v14948: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 338 KiB/s wr, 15 op/s 2023-09-10T07:01:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:01:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:01:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:01:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:37 smithi082 ceph-mon[98694]: pgmap v14948: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 338 KiB/s wr, 15 op/s 2023-09-10T07:01:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:01:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:01:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:01:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:37 smithi098 ceph-mon[103124]: pgmap v14948: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 338 KiB/s wr, 15 op/s 2023-09-10T07:01:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:01:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:01:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:01:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:01:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:01:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:01:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:01:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:01:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:01:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27730.630474 secs 2023-09-10T07:01:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:39 smithi170 ceph-mon[102832]: pgmap v14949: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 337 KiB/s wr, 10 op/s 2023-09-10T07:01:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27731.108336 secs 2023-09-10T07:01:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27730.630474 secs 2023-09-10T07:01:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:39 smithi082 ceph-mon[98694]: pgmap v14949: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 337 KiB/s wr, 10 op/s 2023-09-10T07:01:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27731.108336 secs 2023-09-10T07:01:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27730.630474 secs 2023-09-10T07:01:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:39 smithi098 ceph-mon[103124]: pgmap v14949: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 337 KiB/s wr, 10 op/s 2023-09-10T07:01:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27731.108336 secs 2023-09-10T07:01:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:01:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:01:40.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:01:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:01:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:01:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:01:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:41 smithi170 ceph-mon[102832]: pgmap v14950: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 464 KiB/s wr, 17 op/s 2023-09-10T07:01:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:41 smithi082 ceph-mon[98694]: pgmap v14950: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 464 KiB/s wr, 17 op/s 2023-09-10T07:01:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:41 smithi098 ceph-mon[103124]: pgmap v14950: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 464 KiB/s wr, 17 op/s 2023-09-10T07:01:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:43 smithi170 ceph-mon[102832]: pgmap v14951: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 275 KiB/s wr, 12 op/s 2023-09-10T07:01:43.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:43 smithi082 ceph-mon[98694]: pgmap v14951: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 275 KiB/s wr, 12 op/s 2023-09-10T07:01:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:43 smithi098 ceph-mon[103124]: pgmap v14951: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 275 KiB/s wr, 12 op/s 2023-09-10T07:01:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27735.630557 secs 2023-09-10T07:01:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27736.108398 secs 2023-09-10T07:01:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27735.630557 secs 2023-09-10T07:01:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27736.108398 secs 2023-09-10T07:01:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27735.630557 secs 2023-09-10T07:01:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27736.108398 secs 2023-09-10T07:01:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:45 smithi170 ceph-mon[102832]: pgmap v14952: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 275 KiB/s wr, 12 op/s 2023-09-10T07:01:45.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:45 smithi082 ceph-mon[98694]: pgmap v14952: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 275 KiB/s wr, 12 op/s 2023-09-10T07:01:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:45 smithi098 ceph-mon[103124]: pgmap v14952: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 275 KiB/s wr, 12 op/s 2023-09-10T07:01:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:47 smithi170 ceph-mon[102832]: pgmap v14953: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 522 KiB/s wr, 20 op/s 2023-09-10T07:01:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:47 smithi082 ceph-mon[98694]: pgmap v14953: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 522 KiB/s wr, 20 op/s 2023-09-10T07:01:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:47 smithi098 ceph-mon[103124]: pgmap v14953: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 522 KiB/s wr, 20 op/s 2023-09-10T07:01:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27740.630622 secs 2023-09-10T07:01:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:49 smithi170 ceph-mon[102832]: pgmap v14954: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 375 KiB/s wr, 14 op/s 2023-09-10T07:01:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27741.108463 secs 2023-09-10T07:01:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27740.630622 secs 2023-09-10T07:01:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:49 smithi082 ceph-mon[98694]: pgmap v14954: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 375 KiB/s wr, 14 op/s 2023-09-10T07:01:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27741.108463 secs 2023-09-10T07:01:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27740.630622 secs 2023-09-10T07:01:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:49 smithi098 ceph-mon[103124]: pgmap v14954: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 375 KiB/s wr, 14 op/s 2023-09-10T07:01:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27741.108463 secs 2023-09-10T07:01:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:51 smithi170 ceph-mon[102832]: pgmap v14955: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 495 KiB/s wr, 21 op/s 2023-09-10T07:01:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:51 smithi082 ceph-mon[98694]: pgmap v14955: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 495 KiB/s wr, 21 op/s 2023-09-10T07:01:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:51 smithi098 ceph-mon[103124]: pgmap v14955: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 495 KiB/s wr, 21 op/s 2023-09-10T07:01:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:53 smithi170 ceph-mon[102832]: pgmap v14956: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 369 KiB/s wr, 14 op/s 2023-09-10T07:01:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27744.491408 secs 2023-09-10T07:01:53.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:53 smithi082 ceph-mon[98694]: pgmap v14956: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 369 KiB/s wr, 14 op/s 2023-09-10T07:01:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27744.491408 secs 2023-09-10T07:01:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:53 smithi098 ceph-mon[103124]: pgmap v14956: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 369 KiB/s wr, 14 op/s 2023-09-10T07:01:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27744.491408 secs 2023-09-10T07:01:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27745.630713 secs 2023-09-10T07:01:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:01:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:01:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27746.108486 secs 2023-09-10T07:01:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27745.630713 secs 2023-09-10T07:01:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:01:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:01:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27746.108486 secs 2023-09-10T07:01:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27745.630713 secs 2023-09-10T07:01:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:01:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:01:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27746.108486 secs 2023-09-10T07:01:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:55 smithi170 ceph-mon[102832]: pgmap v14957: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 369 KiB/s wr, 14 op/s 2023-09-10T07:01:55.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:55 smithi082 ceph-mon[98694]: pgmap v14957: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 369 KiB/s wr, 14 op/s 2023-09-10T07:01:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:55 smithi098 ceph-mon[103124]: pgmap v14957: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 369 KiB/s wr, 14 op/s 2023-09-10T07:01:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:01:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:01:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:01:56.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:01:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:01:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:01:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:01:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:01:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:01:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:57 smithi170 ceph-mon[102832]: pgmap v14958: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 576 KiB/s wr, 20 op/s 2023-09-10T07:01:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:57 smithi082 ceph-mon[98694]: pgmap v14958: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 576 KiB/s wr, 20 op/s 2023-09-10T07:01:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:57 smithi098 ceph-mon[103124]: pgmap v14958: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 576 KiB/s wr, 20 op/s 2023-09-10T07:01:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27749.491548 secs 2023-09-10T07:01:58.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27749.491548 secs 2023-09-10T07:01:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27749.491548 secs 2023-09-10T07:01:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27750.630771 secs 2023-09-10T07:01:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:59 smithi170 ceph-mon[102832]: pgmap v14959: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 328 KiB/s wr, 12 op/s 2023-09-10T07:01:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:01:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27751.108554 secs 2023-09-10T07:01:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27750.630771 secs 2023-09-10T07:01:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:59 smithi082 ceph-mon[98694]: pgmap v14959: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 328 KiB/s wr, 12 op/s 2023-09-10T07:01:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:01:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27751.108554 secs 2023-09-10T07:02:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27750.630771 secs 2023-09-10T07:02:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:59 smithi098 ceph-mon[103124]: pgmap v14959: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 328 KiB/s wr, 12 op/s 2023-09-10T07:02:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:01:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27751.108554 secs 2023-09-10T07:02:01.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:01 smithi170 ceph-mon[102832]: pgmap v14960: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 403 KiB/s wr, 17 op/s 2023-09-10T07:02:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:01 smithi082 ceph-mon[98694]: pgmap v14960: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 403 KiB/s wr, 17 op/s 2023-09-10T07:02:02.010 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:01 smithi098 ceph-mon[103124]: pgmap v14960: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 403 KiB/s wr, 17 op/s 2023-09-10T07:02:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:03 smithi170 ceph-mon[102832]: pgmap v14961: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 282 KiB/s wr, 10 op/s 2023-09-10T07:02:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27754.491608 secs 2023-09-10T07:02:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:03 smithi082 ceph-mon[98694]: pgmap v14961: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 282 KiB/s wr, 10 op/s 2023-09-10T07:02:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27754.491608 secs 2023-09-10T07:02:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:03 smithi098 ceph-mon[103124]: pgmap v14961: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 282 KiB/s wr, 10 op/s 2023-09-10T07:02:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27754.491608 secs 2023-09-10T07:02:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27755.630828 secs 2023-09-10T07:02:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:02:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27756.108614 secs 2023-09-10T07:02:04.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27755.630828 secs 2023-09-10T07:02:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:02:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27756.108614 secs 2023-09-10T07:02:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27755.630828 secs 2023-09-10T07:02:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:02:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27756.108614 secs 2023-09-10T07:02:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:05 smithi170 ceph-mon[102832]: pgmap v14962: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 282 KiB/s wr, 10 op/s 2023-09-10T07:02:05.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:05 smithi082 ceph-mon[98694]: pgmap v14962: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 282 KiB/s wr, 10 op/s 2023-09-10T07:02:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:05 smithi098 ceph-mon[103124]: pgmap v14962: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 282 KiB/s wr, 10 op/s 2023-09-10T07:02:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:02:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:06.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:02:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:02:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:07 smithi170 ceph-mon[102832]: pgmap v14963: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 386 KiB/s wr, 30 op/s 2023-09-10T07:02:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:07 smithi082 ceph-mon[98694]: pgmap v14963: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 386 KiB/s wr, 30 op/s 2023-09-10T07:02:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:07 smithi098 ceph-mon[103124]: pgmap v14963: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 386 KiB/s wr, 30 op/s 2023-09-10T07:02:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27759.491674 secs 2023-09-10T07:02:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27759.491674 secs 2023-09-10T07:02:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27759.491674 secs 2023-09-10T07:02:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27760.630898 secs 2023-09-10T07:02:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:09 smithi170 ceph-mon[102832]: pgmap v14964: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 179 KiB/s wr, 24 op/s 2023-09-10T07:02:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:09 smithi170 ceph-mon[102832]: osdmap e1003: 12 total, 12 up, 12 in 2023-09-10T07:02:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27761.108681 secs 2023-09-10T07:02:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27760.630898 secs 2023-09-10T07:02:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:09 smithi082 ceph-mon[98694]: pgmap v14964: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 179 KiB/s wr, 24 op/s 2023-09-10T07:02:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:09 smithi082 ceph-mon[98694]: osdmap e1003: 12 total, 12 up, 12 in 2023-09-10T07:02:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27761.108681 secs 2023-09-10T07:02:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27760.630898 secs 2023-09-10T07:02:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:09 smithi098 ceph-mon[103124]: pgmap v14964: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 179 KiB/s wr, 24 op/s 2023-09-10T07:02:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:09 smithi098 ceph-mon[103124]: osdmap e1003: 12 total, 12 up, 12 in 2023-09-10T07:02:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27761.108681 secs 2023-09-10T07:02:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:11 smithi170 ceph-mon[102832]: pgmap v14966: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 125 KiB/s wr, 29 op/s 2023-09-10T07:02:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:11 smithi170 ceph-mon[102832]: osdmap e1004: 12 total, 12 up, 12 in 2023-09-10T07:02:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:11 smithi082 ceph-mon[98694]: pgmap v14966: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 125 KiB/s wr, 29 op/s 2023-09-10T07:02:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:11 smithi082 ceph-mon[98694]: osdmap e1004: 12 total, 12 up, 12 in 2023-09-10T07:02:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:11 smithi098 ceph-mon[103124]: pgmap v14966: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 125 KiB/s wr, 29 op/s 2023-09-10T07:02:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:11 smithi098 ceph-mon[103124]: osdmap e1004: 12 total, 12 up, 12 in 2023-09-10T07:02:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:13 smithi170 ceph-mon[102832]: pgmap v14968: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 157 KiB/s wr, 36 op/s 2023-09-10T07:02:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27764.491736 secs 2023-09-10T07:02:13.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:13 smithi082 ceph-mon[98694]: pgmap v14968: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 157 KiB/s wr, 36 op/s 2023-09-10T07:02:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27764.491736 secs 2023-09-10T07:02:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:13 smithi098 ceph-mon[103124]: pgmap v14968: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 157 KiB/s wr, 36 op/s 2023-09-10T07:02:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27764.491736 secs 2023-09-10T07:02:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27765.630958 secs 2023-09-10T07:02:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27766.108744 secs 2023-09-10T07:02:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27765.630958 secs 2023-09-10T07:02:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27766.108744 secs 2023-09-10T07:02:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27765.630958 secs 2023-09-10T07:02:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27766.108744 secs 2023-09-10T07:02:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:15 smithi170 ceph-mon[102832]: pgmap v14969: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:02:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:15 smithi082 ceph-mon[98694]: pgmap v14969: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:02:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:15 smithi098 ceph-mon[103124]: pgmap v14969: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:02:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:17 smithi170 ceph-mon[102832]: pgmap v14970: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 14 op/s 2023-09-10T07:02:17.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:17 smithi082 ceph-mon[98694]: pgmap v14970: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 14 op/s 2023-09-10T07:02:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:17 smithi098 ceph-mon[103124]: pgmap v14970: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 14 op/s 2023-09-10T07:02:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27769.491772 secs 2023-09-10T07:02:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27769.491772 secs 2023-09-10T07:02:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27769.491772 secs 2023-09-10T07:02:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27770.631016 secs 2023-09-10T07:02:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:19 smithi098 ceph-mon[103124]: pgmap v14971: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 207 B/s wr, 11 op/s 2023-09-10T07:02:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27770.631016 secs 2023-09-10T07:02:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:19 smithi170 ceph-mon[102832]: pgmap v14971: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 207 B/s wr, 11 op/s 2023-09-10T07:02:19.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27770.631016 secs 2023-09-10T07:02:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:19 smithi082 ceph-mon[98694]: pgmap v14971: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 207 B/s wr, 11 op/s 2023-09-10T07:02:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27771.108810 secs 2023-09-10T07:02:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27771.108810 secs 2023-09-10T07:02:20.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27771.108810 secs 2023-09-10T07:02:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:21 smithi098 ceph-mon[103124]: pgmap v14972: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T07:02:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:21 smithi170 ceph-mon[102832]: pgmap v14972: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T07:02:21.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:21 smithi082 ceph-mon[98694]: pgmap v14972: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T07:02:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:23 smithi170 ceph-mon[102832]: pgmap v14973: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T07:02:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27774.491844 secs 2023-09-10T07:02:23.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:23 smithi082 ceph-mon[98694]: pgmap v14973: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T07:02:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27774.491844 secs 2023-09-10T07:02:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:23 smithi098 ceph-mon[103124]: pgmap v14973: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T07:02:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27774.491844 secs 2023-09-10T07:02:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27775.631075 secs 2023-09-10T07:02:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27776.108868 secs 2023-09-10T07:02:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27775.631075 secs 2023-09-10T07:02:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27776.108868 secs 2023-09-10T07:02:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27775.631075 secs 2023-09-10T07:02:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27776.108868 secs 2023-09-10T07:02:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:25 smithi170 ceph-mon[102832]: pgmap v14974: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-10T07:02:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:25 smithi082 ceph-mon[98694]: pgmap v14974: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-10T07:02:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:25 smithi098 ceph-mon[103124]: pgmap v14974: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-10T07:02:27.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:27 smithi170 ceph-mon[102832]: pgmap v14975: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 91 KiB/s wr, 14 op/s 2023-09-10T07:02:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:27 smithi082 ceph-mon[98694]: pgmap v14975: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 91 KiB/s wr, 14 op/s 2023-09-10T07:02:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:27 smithi098 ceph-mon[103124]: pgmap v14975: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 91 KiB/s wr, 14 op/s 2023-09-10T07:02:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27780.631136 secs 2023-09-10T07:02:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:29 smithi170 ceph-mon[102832]: pgmap v14976: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 91 KiB/s wr, 9 op/s 2023-09-10T07:02:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27781.108933 secs 2023-09-10T07:02:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27780.631136 secs 2023-09-10T07:02:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:29 smithi082 ceph-mon[98694]: pgmap v14976: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 91 KiB/s wr, 9 op/s 2023-09-10T07:02:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27781.108933 secs 2023-09-10T07:02:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27780.631136 secs 2023-09-10T07:02:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:29 smithi098 ceph-mon[103124]: pgmap v14976: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 91 KiB/s wr, 9 op/s 2023-09-10T07:02:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27781.108933 secs 2023-09-10T07:02:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:31 smithi170 ceph-mon[102832]: pgmap v14977: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 281 KiB/s wr, 15 op/s 2023-09-10T07:02:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:02:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:31 smithi082 ceph-mon[98694]: pgmap v14977: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 281 KiB/s wr, 15 op/s 2023-09-10T07:02:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:02:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:31 smithi098 ceph-mon[103124]: pgmap v14977: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 281 KiB/s wr, 15 op/s 2023-09-10T07:02:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:02:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:02:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:02:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:02:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:02:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:02:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:02:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:33 smithi170 ceph-mon[102832]: pgmap v14978: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 281 KiB/s wr, 10 op/s 2023-09-10T07:02:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:33 smithi082 ceph-mon[98694]: pgmap v14978: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 281 KiB/s wr, 10 op/s 2023-09-10T07:02:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:33 smithi098 ceph-mon[103124]: pgmap v14978: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 281 KiB/s wr, 10 op/s 2023-09-10T07:02:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27785.631223 secs 2023-09-10T07:02:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:02:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27786.108999 secs 2023-09-10T07:02:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27785.631223 secs 2023-09-10T07:02:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:02:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27786.108999 secs 2023-09-10T07:02:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27785.631223 secs 2023-09-10T07:02:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:02:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27786.108999 secs 2023-09-10T07:02:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:35 smithi170 ceph-mon[102832]: pgmap v14979: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 281 KiB/s wr, 10 op/s 2023-09-10T07:02:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:35 smithi082 ceph-mon[98694]: pgmap v14979: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 281 KiB/s wr, 10 op/s 2023-09-10T07:02:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:35 smithi098 ceph-mon[103124]: pgmap v14979: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 281 KiB/s wr, 10 op/s 2023-09-10T07:02:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:02:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:36.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:02:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:02:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:37 smithi170 ceph-mon[102832]: pgmap v14980: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 695 KiB/s wr, 18 op/s 2023-09-10T07:02:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:02:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:02:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:02:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:02:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:37 smithi082 ceph-mon[98694]: pgmap v14980: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 695 KiB/s wr, 18 op/s 2023-09-10T07:02:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:02:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:02:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:02:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:02:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:37 smithi098 ceph-mon[103124]: pgmap v14980: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 695 KiB/s wr, 18 op/s 2023-09-10T07:02:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:02:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:02:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:02:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:02:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:02:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:02:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:02:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27790.631354 secs 2023-09-10T07:02:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:39 smithi170 ceph-mon[102832]: pgmap v14981: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 604 KiB/s wr, 13 op/s 2023-09-10T07:02:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27791.109065 secs 2023-09-10T07:02:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27790.631354 secs 2023-09-10T07:02:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:39 smithi082 ceph-mon[98694]: pgmap v14981: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 604 KiB/s wr, 13 op/s 2023-09-10T07:02:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27791.109065 secs 2023-09-10T07:02:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27790.631354 secs 2023-09-10T07:02:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:39 smithi098 ceph-mon[103124]: pgmap v14981: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 604 KiB/s wr, 13 op/s 2023-09-10T07:02:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27791.109065 secs 2023-09-10T07:02:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:02:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:40.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:02:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:02:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:41 smithi170 ceph-mon[102832]: pgmap v14982: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 625 KiB/s wr, 18 op/s 2023-09-10T07:02:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:41 smithi082 ceph-mon[98694]: pgmap v14982: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 625 KiB/s wr, 18 op/s 2023-09-10T07:02:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:41 smithi098 ceph-mon[103124]: pgmap v14982: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 625 KiB/s wr, 18 op/s 2023-09-10T07:02:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:43 smithi170 ceph-mon[102832]: pgmap v14983: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 435 KiB/s wr, 13 op/s 2023-09-10T07:02:43.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:43 smithi082 ceph-mon[98694]: pgmap v14983: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 435 KiB/s wr, 13 op/s 2023-09-10T07:02:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:43 smithi098 ceph-mon[103124]: pgmap v14983: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 435 KiB/s wr, 13 op/s 2023-09-10T07:02:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27795.631413 secs 2023-09-10T07:02:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27796.109127 secs 2023-09-10T07:02:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27795.631413 secs 2023-09-10T07:02:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27796.109127 secs 2023-09-10T07:02:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27795.631413 secs 2023-09-10T07:02:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27796.109127 secs 2023-09-10T07:02:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:45 smithi170 ceph-mon[102832]: pgmap v14984: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 435 KiB/s wr, 13 op/s 2023-09-10T07:02:45.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:45 smithi082 ceph-mon[98694]: pgmap v14984: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 435 KiB/s wr, 13 op/s 2023-09-10T07:02:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:45 smithi098 ceph-mon[103124]: pgmap v14984: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 435 KiB/s wr, 13 op/s 2023-09-10T07:02:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:47 smithi170 ceph-mon[102832]: pgmap v14985: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 777 KiB/s wr, 19 op/s 2023-09-10T07:02:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:47 smithi082 ceph-mon[98694]: pgmap v14985: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 777 KiB/s wr, 19 op/s 2023-09-10T07:02:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:47 smithi098 ceph-mon[103124]: pgmap v14985: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 777 KiB/s wr, 19 op/s 2023-09-10T07:02:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27800.631469 secs 2023-09-10T07:02:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:49 smithi170 ceph-mon[102832]: pgmap v14986: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 363 KiB/s wr, 12 op/s 2023-09-10T07:02:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27801.109193 secs 2023-09-10T07:02:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27800.631469 secs 2023-09-10T07:02:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:49 smithi082 ceph-mon[98694]: pgmap v14986: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 363 KiB/s wr, 12 op/s 2023-09-10T07:02:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27801.109193 secs 2023-09-10T07:02:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27800.631469 secs 2023-09-10T07:02:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:49 smithi098 ceph-mon[103124]: pgmap v14986: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 363 KiB/s wr, 12 op/s 2023-09-10T07:02:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27801.109193 secs 2023-09-10T07:02:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:51 smithi170 ceph-mon[102832]: pgmap v14987: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 692 KiB/s wr, 18 op/s 2023-09-10T07:02:51.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:51 smithi082 ceph-mon[98694]: pgmap v14987: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 692 KiB/s wr, 18 op/s 2023-09-10T07:02:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:51 smithi098 ceph-mon[103124]: pgmap v14987: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 692 KiB/s wr, 18 op/s 2023-09-10T07:02:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:02:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:52.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:02:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:02:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:53 smithi170 ceph-mon[102832]: pgmap v14988: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 671 KiB/s wr, 13 op/s 2023-09-10T07:02:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27804.492299 secs 2023-09-10T07:02:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:53 smithi082 ceph-mon[98694]: pgmap v14988: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 671 KiB/s wr, 13 op/s 2023-09-10T07:02:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27804.492299 secs 2023-09-10T07:02:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:53 smithi098 ceph-mon[103124]: pgmap v14988: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 671 KiB/s wr, 13 op/s 2023-09-10T07:02:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27804.492299 secs 2023-09-10T07:02:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27805.631522 secs 2023-09-10T07:02:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:02:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27806.109254 secs 2023-09-10T07:02:54.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27805.631522 secs 2023-09-10T07:02:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:02:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27806.109254 secs 2023-09-10T07:02:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27805.631522 secs 2023-09-10T07:02:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:02:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27806.109254 secs 2023-09-10T07:02:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:55 smithi170 ceph-mon[102832]: pgmap v14989: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 671 KiB/s wr, 13 op/s 2023-09-10T07:02:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:55 smithi082 ceph-mon[98694]: pgmap v14989: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 671 KiB/s wr, 13 op/s 2023-09-10T07:02:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:55 smithi098 ceph-mon[103124]: pgmap v14989: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 671 KiB/s wr, 13 op/s 2023-09-10T07:02:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:02:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:02:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:56.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:02:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:02:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:02:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:02:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:57 smithi170 ceph-mon[102832]: pgmap v14990: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 939 KiB/s wr, 19 op/s 2023-09-10T07:02:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:57 smithi082 ceph-mon[98694]: pgmap v14990: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 939 KiB/s wr, 19 op/s 2023-09-10T07:02:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:57 smithi098 ceph-mon[103124]: pgmap v14990: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 939 KiB/s wr, 19 op/s 2023-09-10T07:02:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:02:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27809.492359 secs 2023-09-10T07:02:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:02:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27809.492359 secs 2023-09-10T07:02:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:02:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:02:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27809.492359 secs 2023-09-10T07:02:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27810.631621 secs 2023-09-10T07:02:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:59 smithi170 ceph-mon[102832]: pgmap v14991: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 597 KiB/s wr, 13 op/s 2023-09-10T07:02:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:02:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27811.109321 secs 2023-09-10T07:02:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27810.631621 secs 2023-09-10T07:02:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:59 smithi082 ceph-mon[98694]: pgmap v14991: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 597 KiB/s wr, 13 op/s 2023-09-10T07:02:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:02:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27811.109321 secs 2023-09-10T07:03:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27810.631621 secs 2023-09-10T07:03:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:59 smithi098 ceph-mon[103124]: pgmap v14991: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 597 KiB/s wr, 13 op/s 2023-09-10T07:03:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:02:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27811.109321 secs 2023-09-10T07:03:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:03:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:03:00.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:03:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:03:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:03:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:03:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:01 smithi170 ceph-mon[102832]: pgmap v14992: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 804 KiB/s wr, 18 op/s 2023-09-10T07:03:01.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:01 smithi082 ceph-mon[98694]: pgmap v14992: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 804 KiB/s wr, 18 op/s 2023-09-10T07:03:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:01 smithi098 ceph-mon[103124]: pgmap v14992: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 804 KiB/s wr, 18 op/s 2023-09-10T07:03:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:03 smithi170 ceph-mon[102832]: pgmap v14993: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 476 KiB/s wr, 12 op/s 2023-09-10T07:03:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27814.492418 secs 2023-09-10T07:03:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:03 smithi082 ceph-mon[98694]: pgmap v14993: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 476 KiB/s wr, 12 op/s 2023-09-10T07:03:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27814.492418 secs 2023-09-10T07:03:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:03 smithi098 ceph-mon[103124]: pgmap v14993: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 476 KiB/s wr, 12 op/s 2023-09-10T07:03:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27814.492418 secs 2023-09-10T07:03:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27815.631707 secs 2023-09-10T07:03:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27816.109382 secs 2023-09-10T07:03:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27815.631707 secs 2023-09-10T07:03:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27816.109382 secs 2023-09-10T07:03:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27815.631707 secs 2023-09-10T07:03:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27816.109382 secs 2023-09-10T07:03:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:05 smithi082 ceph-mon[98694]: pgmap v14994: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 476 KiB/s wr, 12 op/s 2023-09-10T07:03:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:05 smithi098 ceph-mon[103124]: pgmap v14994: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 476 KiB/s wr, 12 op/s 2023-09-10T07:03:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:05 smithi170 ceph-mon[102832]: pgmap v14994: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 476 KiB/s wr, 12 op/s 2023-09-10T07:03:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:07 smithi170 ceph-mon[102832]: pgmap v14995: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 596 KiB/s wr, 31 op/s 2023-09-10T07:03:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:07 smithi082 ceph-mon[98694]: pgmap v14995: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 596 KiB/s wr, 31 op/s 2023-09-10T07:03:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:07 smithi098 ceph-mon[103124]: pgmap v14995: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 596 KiB/s wr, 31 op/s 2023-09-10T07:03:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27819.492480 secs 2023-09-10T07:03:08.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27819.492480 secs 2023-09-10T07:03:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27819.492480 secs 2023-09-10T07:03:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27820.631767 secs 2023-09-10T07:03:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:09 smithi170 ceph-mon[102832]: osdmap e1005: 12 total, 12 up, 12 in 2023-09-10T07:03:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:09 smithi170 ceph-mon[102832]: pgmap v14997: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 393 KiB/s wr, 30 op/s 2023-09-10T07:03:09.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27820.631767 secs 2023-09-10T07:03:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:09 smithi082 ceph-mon[98694]: osdmap e1005: 12 total, 12 up, 12 in 2023-09-10T07:03:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:09 smithi082 ceph-mon[98694]: pgmap v14997: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 393 KiB/s wr, 30 op/s 2023-09-10T07:03:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27820.631767 secs 2023-09-10T07:03:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:09 smithi098 ceph-mon[103124]: osdmap e1005: 12 total, 12 up, 12 in 2023-09-10T07:03:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:09 smithi098 ceph-mon[103124]: pgmap v14997: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 393 KiB/s wr, 30 op/s 2023-09-10T07:03:10.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27821.109450 secs 2023-09-10T07:03:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27821.109450 secs 2023-09-10T07:03:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27821.109450 secs 2023-09-10T07:03:11.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:11 smithi082 ceph-mon[98694]: pgmap v14998: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 145 KiB/s wr, 29 op/s 2023-09-10T07:03:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:11 smithi098 ceph-mon[103124]: pgmap v14998: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 145 KiB/s wr, 29 op/s 2023-09-10T07:03:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:11 smithi170 ceph-mon[102832]: pgmap v14998: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 145 KiB/s wr, 29 op/s 2023-09-10T07:03:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:13 smithi170 ceph-mon[102832]: pgmap v14999: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 145 KiB/s wr, 29 op/s 2023-09-10T07:03:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27824.492575 secs 2023-09-10T07:03:13.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:13 smithi082 ceph-mon[98694]: pgmap v14999: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 145 KiB/s wr, 29 op/s 2023-09-10T07:03:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27824.492575 secs 2023-09-10T07:03:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:13 smithi098 ceph-mon[103124]: pgmap v14999: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 145 KiB/s wr, 29 op/s 2023-09-10T07:03:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27824.492575 secs 2023-09-10T07:03:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27825.631826 secs 2023-09-10T07:03:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27826.109484 secs 2023-09-10T07:03:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27825.631826 secs 2023-09-10T07:03:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27826.109484 secs 2023-09-10T07:03:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27825.631826 secs 2023-09-10T07:03:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27826.109484 secs 2023-09-10T07:03:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:15 smithi170 ceph-mon[102832]: pgmap v15000: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 145 KiB/s wr, 29 op/s 2023-09-10T07:03:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:15 smithi082 ceph-mon[98694]: pgmap v15000: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 145 KiB/s wr, 29 op/s 2023-09-10T07:03:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:15 smithi098 ceph-mon[103124]: pgmap v15000: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 145 KiB/s wr, 29 op/s 2023-09-10T07:03:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:17 smithi170 ceph-mon[102832]: osdmap e1006: 12 total, 12 up, 12 in 2023-09-10T07:03:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:17 smithi170 ceph-mon[102832]: pgmap v15002: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 139 KiB/s wr, 14 op/s 2023-09-10T07:03:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:17 smithi082 ceph-mon[98694]: osdmap e1006: 12 total, 12 up, 12 in 2023-09-10T07:03:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:17 smithi082 ceph-mon[98694]: pgmap v15002: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 139 KiB/s wr, 14 op/s 2023-09-10T07:03:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:17 smithi098 ceph-mon[103124]: osdmap e1006: 12 total, 12 up, 12 in 2023-09-10T07:03:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:17 smithi098 ceph-mon[103124]: pgmap v15002: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 139 KiB/s wr, 14 op/s 2023-09-10T07:03:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27829.492637 secs 2023-09-10T07:03:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27829.492637 secs 2023-09-10T07:03:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27829.492637 secs 2023-09-10T07:03:19.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27830.631880 secs 2023-09-10T07:03:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:19 smithi082 ceph-mon[98694]: pgmap v15003: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T07:03:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27830.631880 secs 2023-09-10T07:03:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:19 smithi098 ceph-mon[103124]: pgmap v15003: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T07:03:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27830.631880 secs 2023-09-10T07:03:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:19 smithi170 ceph-mon[102832]: pgmap v15003: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T07:03:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27831.109550 secs 2023-09-10T07:03:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27831.109550 secs 2023-09-10T07:03:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27831.109550 secs 2023-09-10T07:03:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:21 smithi082 ceph-mon[98694]: pgmap v15004: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T07:03:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:21 smithi098 ceph-mon[103124]: pgmap v15004: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T07:03:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:21 smithi170 ceph-mon[102832]: pgmap v15004: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T07:03:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:23 smithi170 ceph-mon[102832]: pgmap v15005: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T07:03:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27834.492768 secs 2023-09-10T07:03:23.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:23 smithi082 ceph-mon[98694]: pgmap v15005: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T07:03:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27834.492768 secs 2023-09-10T07:03:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:23 smithi098 ceph-mon[103124]: pgmap v15005: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T07:03:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27834.492768 secs 2023-09-10T07:03:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27835.631939 secs 2023-09-10T07:03:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27836.109615 secs 2023-09-10T07:03:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27835.631939 secs 2023-09-10T07:03:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27836.109615 secs 2023-09-10T07:03:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27835.631939 secs 2023-09-10T07:03:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27836.109615 secs 2023-09-10T07:03:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:25 smithi170 ceph-mon[102832]: pgmap v15006: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T07:03:25.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:25 smithi082 ceph-mon[98694]: pgmap v15006: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T07:03:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:25 smithi098 ceph-mon[103124]: pgmap v15006: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T07:03:27.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:27 smithi170 ceph-mon[102832]: pgmap v15007: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 806 B/s wr, 10 op/s 2023-09-10T07:03:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:27 smithi082 ceph-mon[98694]: pgmap v15007: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 806 B/s wr, 10 op/s 2023-09-10T07:03:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:27 smithi098 ceph-mon[103124]: pgmap v15007: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 806 B/s wr, 10 op/s 2023-09-10T07:03:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27840.631995 secs 2023-09-10T07:03:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:29 smithi170 ceph-mon[102832]: pgmap v15008: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-10T07:03:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27841.109683 secs 2023-09-10T07:03:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27840.631995 secs 2023-09-10T07:03:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:29 smithi082 ceph-mon[98694]: pgmap v15008: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-10T07:03:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27841.109683 secs 2023-09-10T07:03:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27840.631995 secs 2023-09-10T07:03:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:29 smithi098 ceph-mon[103124]: pgmap v15008: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-10T07:03:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27841.109683 secs 2023-09-10T07:03:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:31 smithi170 ceph-mon[102832]: pgmap v15009: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 192 KiB/s wr, 14 op/s 2023-09-10T07:03:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:03:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:31 smithi082 ceph-mon[98694]: pgmap v15009: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 192 KiB/s wr, 14 op/s 2023-09-10T07:03:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:03:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:31 smithi098 ceph-mon[103124]: pgmap v15009: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 192 KiB/s wr, 14 op/s 2023-09-10T07:03:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:03:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:03:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:03:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:03:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:03:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:03:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:03:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:03:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:03:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:03:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:33 smithi170 ceph-mon[102832]: pgmap v15010: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 191 KiB/s wr, 10 op/s 2023-09-10T07:03:33.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:33 smithi082 ceph-mon[98694]: pgmap v15010: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 191 KiB/s wr, 10 op/s 2023-09-10T07:03:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:33 smithi098 ceph-mon[103124]: pgmap v15010: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 191 KiB/s wr, 10 op/s 2023-09-10T07:03:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27845.632054 secs 2023-09-10T07:03:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:03:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:03:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27846.109746 secs 2023-09-10T07:03:34.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27845.632054 secs 2023-09-10T07:03:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:03:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:03:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27846.109746 secs 2023-09-10T07:03:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27845.632054 secs 2023-09-10T07:03:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:03:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:03:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27846.109746 secs 2023-09-10T07:03:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:35 smithi170 ceph-mon[102832]: pgmap v15011: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 191 KiB/s wr, 10 op/s 2023-09-10T07:03:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:35 smithi082 ceph-mon[98694]: pgmap v15011: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 191 KiB/s wr, 10 op/s 2023-09-10T07:03:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:35 smithi098 ceph-mon[103124]: pgmap v15011: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 191 KiB/s wr, 10 op/s 2023-09-10T07:03:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:37 smithi170 ceph-mon[102832]: pgmap v15012: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 307 KiB/s wr, 16 op/s 2023-09-10T07:03:37.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:37 smithi082 ceph-mon[98694]: pgmap v15012: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 307 KiB/s wr, 16 op/s 2023-09-10T07:03:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:37 smithi098 ceph-mon[103124]: pgmap v15012: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 307 KiB/s wr, 16 op/s 2023-09-10T07:03:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:03:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:03:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:03:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:03:38.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:03:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:03:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:03:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:03:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:03:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:03:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:03:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:03:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27850.632115 secs 2023-09-10T07:03:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:39 smithi082 ceph-mon[98694]: pgmap v15013: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 307 KiB/s wr, 11 op/s 2023-09-10T07:03:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27850.632115 secs 2023-09-10T07:03:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:39 smithi098 ceph-mon[103124]: pgmap v15013: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 307 KiB/s wr, 11 op/s 2023-09-10T07:03:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27850.632115 secs 2023-09-10T07:03:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:39 smithi170 ceph-mon[102832]: pgmap v15013: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 307 KiB/s wr, 11 op/s 2023-09-10T07:03:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27851.109815 secs 2023-09-10T07:03:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27851.109815 secs 2023-09-10T07:03:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27851.109815 secs 2023-09-10T07:03:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:41 smithi082 ceph-mon[98694]: pgmap v15014: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 429 KiB/s wr, 18 op/s 2023-09-10T07:03:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:41 smithi098 ceph-mon[103124]: pgmap v15014: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 429 KiB/s wr, 18 op/s 2023-09-10T07:03:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:41 smithi170 ceph-mon[102832]: pgmap v15014: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 429 KiB/s wr, 18 op/s 2023-09-10T07:03:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:43 smithi170 ceph-mon[102832]: pgmap v15015: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 238 KiB/s wr, 12 op/s 2023-09-10T07:03:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:43 smithi082 ceph-mon[98694]: pgmap v15015: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 238 KiB/s wr, 12 op/s 2023-09-10T07:03:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:43 smithi098 ceph-mon[103124]: pgmap v15015: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 238 KiB/s wr, 12 op/s 2023-09-10T07:03:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27855.632188 secs 2023-09-10T07:03:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:03:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:03:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27856.109880 secs 2023-09-10T07:03:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27855.632188 secs 2023-09-10T07:03:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:03:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:03:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27856.109880 secs 2023-09-10T07:03:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27855.632188 secs 2023-09-10T07:03:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:03:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:03:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27856.109880 secs 2023-09-10T07:03:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:45 smithi170 ceph-mon[102832]: pgmap v15016: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 238 KiB/s wr, 12 op/s 2023-09-10T07:03:45.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:45 smithi082 ceph-mon[98694]: pgmap v15016: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 238 KiB/s wr, 12 op/s 2023-09-10T07:03:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:45 smithi098 ceph-mon[103124]: pgmap v15016: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 238 KiB/s wr, 12 op/s 2023-09-10T07:03:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:03:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:03:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:03:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:03:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:03:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:03:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:47 smithi170 ceph-mon[102832]: pgmap v15017: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 542 KiB/s wr, 20 op/s 2023-09-10T07:03:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:47 smithi082 ceph-mon[98694]: pgmap v15017: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 542 KiB/s wr, 20 op/s 2023-09-10T07:03:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:47 smithi098 ceph-mon[103124]: pgmap v15017: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 542 KiB/s wr, 20 op/s 2023-09-10T07:03:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27860.632249 secs 2023-09-10T07:03:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:49 smithi170 ceph-mon[102832]: pgmap v15018: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 426 KiB/s wr, 14 op/s 2023-09-10T07:03:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27861.109947 secs 2023-09-10T07:03:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27860.632249 secs 2023-09-10T07:03:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:49 smithi082 ceph-mon[98694]: pgmap v15018: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 426 KiB/s wr, 14 op/s 2023-09-10T07:03:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27861.109947 secs 2023-09-10T07:03:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27860.632249 secs 2023-09-10T07:03:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:49 smithi098 ceph-mon[103124]: pgmap v15018: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 426 KiB/s wr, 14 op/s 2023-09-10T07:03:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27861.109947 secs 2023-09-10T07:03:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:51 smithi170 ceph-mon[102832]: pgmap v15019: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 451 KiB/s wr, 19 op/s 2023-09-10T07:03:51.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:51 smithi082 ceph-mon[98694]: pgmap v15019: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 451 KiB/s wr, 19 op/s 2023-09-10T07:03:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:51 smithi098 ceph-mon[103124]: pgmap v15019: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 451 KiB/s wr, 19 op/s 2023-09-10T07:03:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:53 smithi170 ceph-mon[102832]: pgmap v15020: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 329 KiB/s wr, 13 op/s 2023-09-10T07:03:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27864.493356 secs 2023-09-10T07:03:53.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:53 smithi082 ceph-mon[98694]: pgmap v15020: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 329 KiB/s wr, 13 op/s 2023-09-10T07:03:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27864.493356 secs 2023-09-10T07:03:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:53 smithi098 ceph-mon[103124]: pgmap v15020: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 329 KiB/s wr, 13 op/s 2023-09-10T07:03:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27864.493356 secs 2023-09-10T07:03:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27865.632302 secs 2023-09-10T07:03:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:03:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:03:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27866.110011 secs 2023-09-10T07:03:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27865.632302 secs 2023-09-10T07:03:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:03:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:03:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27866.110011 secs 2023-09-10T07:03:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27865.632302 secs 2023-09-10T07:03:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:03:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:03:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27866.110011 secs 2023-09-10T07:03:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:55 smithi170 ceph-mon[102832]: pgmap v15021: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 329 KiB/s wr, 13 op/s 2023-09-10T07:03:55.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:55 smithi082 ceph-mon[98694]: pgmap v15021: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 329 KiB/s wr, 13 op/s 2023-09-10T07:03:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:55 smithi098 ceph-mon[103124]: pgmap v15021: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 329 KiB/s wr, 13 op/s 2023-09-10T07:03:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:03:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:03:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:03:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:03:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:03:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:03:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:03:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:03:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:03:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:57 smithi082 ceph-mon[98694]: pgmap v15022: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 571 KiB/s wr, 19 op/s 2023-09-10T07:03:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:57 smithi098 ceph-mon[103124]: pgmap v15022: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 571 KiB/s wr, 19 op/s 2023-09-10T07:03:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:57 smithi170 ceph-mon[102832]: pgmap v15022: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 571 KiB/s wr, 19 op/s 2023-09-10T07:03:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27869.493415 secs 2023-09-10T07:03:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27869.493415 secs 2023-09-10T07:03:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27869.493415 secs 2023-09-10T07:03:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27870.632361 secs 2023-09-10T07:03:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:59 smithi082 ceph-mon[98694]: pgmap v15023: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 267 KiB/s wr, 11 op/s 2023-09-10T07:03:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:03:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27871.110099 secs 2023-09-10T07:04:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27870.632361 secs 2023-09-10T07:04:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:59 smithi098 ceph-mon[103124]: pgmap v15023: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 267 KiB/s wr, 11 op/s 2023-09-10T07:04:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:03:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27871.110099 secs 2023-09-10T07:04:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27870.632361 secs 2023-09-10T07:04:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:59 smithi170 ceph-mon[102832]: pgmap v15023: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 267 KiB/s wr, 11 op/s 2023-09-10T07:04:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:03:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27871.110099 secs 2023-09-10T07:04:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:01 smithi082 ceph-mon[98694]: pgmap v15024: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 342 KiB/s wr, 16 op/s 2023-09-10T07:04:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:01 smithi098 ceph-mon[103124]: pgmap v15024: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 342 KiB/s wr, 16 op/s 2023-09-10T07:04:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:01 smithi170 ceph-mon[102832]: pgmap v15024: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 342 KiB/s wr, 16 op/s 2023-09-10T07:04:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:03 smithi082 ceph-mon[98694]: pgmap v15025: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 317 KiB/s wr, 10 op/s 2023-09-10T07:04:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27874.493481 secs 2023-09-10T07:04:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:03 smithi098 ceph-mon[103124]: pgmap v15025: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 317 KiB/s wr, 10 op/s 2023-09-10T07:04:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27874.493481 secs 2023-09-10T07:04:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:03 smithi170 ceph-mon[102832]: pgmap v15025: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 317 KiB/s wr, 10 op/s 2023-09-10T07:04:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27874.493481 secs 2023-09-10T07:04:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27875.632427 secs 2023-09-10T07:04:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27876.110161 secs 2023-09-10T07:04:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27875.632427 secs 2023-09-10T07:04:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27876.110161 secs 2023-09-10T07:04:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27875.632427 secs 2023-09-10T07:04:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27876.110161 secs 2023-09-10T07:04:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:05 smithi082 ceph-mon[98694]: pgmap v15026: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 317 KiB/s wr, 10 op/s 2023-09-10T07:04:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:05 smithi098 ceph-mon[103124]: pgmap v15026: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 317 KiB/s wr, 10 op/s 2023-09-10T07:04:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:05 smithi170 ceph-mon[102832]: pgmap v15026: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 317 KiB/s wr, 10 op/s 2023-09-10T07:04:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:07 smithi082 ceph-mon[98694]: pgmap v15027: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 362 KiB/s wr, 29 op/s 2023-09-10T07:04:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:07 smithi098 ceph-mon[103124]: pgmap v15027: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 362 KiB/s wr, 29 op/s 2023-09-10T07:04:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:07 smithi170 ceph-mon[102832]: pgmap v15027: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 362 KiB/s wr, 29 op/s 2023-09-10T07:04:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27879.493616 secs 2023-09-10T07:04:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27879.493616 secs 2023-09-10T07:04:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27879.493616 secs 2023-09-10T07:04:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27880.632504 secs 2023-09-10T07:04:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:09 smithi082 ceph-mon[98694]: osdmap e1007: 12 total, 12 up, 12 in 2023-09-10T07:04:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:09 smithi082 ceph-mon[98694]: pgmap v15029: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 144 KiB/s wr, 28 op/s 2023-09-10T07:04:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27880.632504 secs 2023-09-10T07:04:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:09 smithi098 ceph-mon[103124]: osdmap e1007: 12 total, 12 up, 12 in 2023-09-10T07:04:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:09 smithi098 ceph-mon[103124]: pgmap v15029: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 144 KiB/s wr, 28 op/s 2023-09-10T07:04:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27880.632504 secs 2023-09-10T07:04:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:09 smithi170 ceph-mon[102832]: osdmap e1007: 12 total, 12 up, 12 in 2023-09-10T07:04:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:09 smithi170 ceph-mon[102832]: pgmap v15029: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 144 KiB/s wr, 28 op/s 2023-09-10T07:04:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27881.110225 secs 2023-09-10T07:04:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27881.110225 secs 2023-09-10T07:04:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27881.110225 secs 2023-09-10T07:04:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:11 smithi098 ceph-mon[103124]: pgmap v15030: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T07:04:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:11 smithi170 ceph-mon[102832]: pgmap v15030: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T07:04:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:11 smithi082 ceph-mon[98694]: pgmap v15030: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T07:04:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:13 smithi170 ceph-mon[102832]: pgmap v15031: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T07:04:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27884.493673 secs 2023-09-10T07:04:13.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:13 smithi082 ceph-mon[98694]: pgmap v15031: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T07:04:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27884.493673 secs 2023-09-10T07:04:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:13 smithi098 ceph-mon[103124]: pgmap v15031: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T07:04:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27884.493673 secs 2023-09-10T07:04:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27885.632583 secs 2023-09-10T07:04:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27886.110286 secs 2023-09-10T07:04:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27885.632583 secs 2023-09-10T07:04:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27886.110286 secs 2023-09-10T07:04:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27885.632583 secs 2023-09-10T07:04:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27886.110286 secs 2023-09-10T07:04:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:15 smithi170 ceph-mon[102832]: pgmap v15032: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T07:04:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:15 smithi082 ceph-mon[98694]: pgmap v15032: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T07:04:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:15 smithi098 ceph-mon[103124]: pgmap v15032: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T07:04:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:17 smithi082 ceph-mon[98694]: osdmap e1008: 12 total, 12 up, 12 in 2023-09-10T07:04:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:17 smithi082 ceph-mon[98694]: pgmap v15034: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 251 B/s wr, 12 op/s 2023-09-10T07:04:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:17 smithi098 ceph-mon[103124]: osdmap e1008: 12 total, 12 up, 12 in 2023-09-10T07:04:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:17 smithi098 ceph-mon[103124]: pgmap v15034: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 251 B/s wr, 12 op/s 2023-09-10T07:04:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:17 smithi170 ceph-mon[102832]: osdmap e1008: 12 total, 12 up, 12 in 2023-09-10T07:04:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:17 smithi170 ceph-mon[102832]: pgmap v15034: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 251 B/s wr, 12 op/s 2023-09-10T07:04:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27889.493732 secs 2023-09-10T07:04:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27889.493732 secs 2023-09-10T07:04:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27889.493732 secs 2023-09-10T07:04:19.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27890.632641 secs 2023-09-10T07:04:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:19 smithi082 ceph-mon[98694]: pgmap v15035: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:04:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27890.632641 secs 2023-09-10T07:04:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:19 smithi098 ceph-mon[103124]: pgmap v15035: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:04:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27890.632641 secs 2023-09-10T07:04:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:19 smithi170 ceph-mon[102832]: pgmap v15035: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:04:20.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27891.110351 secs 2023-09-10T07:04:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27891.110351 secs 2023-09-10T07:04:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27891.110351 secs 2023-09-10T07:04:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:21 smithi098 ceph-mon[103124]: pgmap v15036: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:04:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:21 smithi170 ceph-mon[102832]: pgmap v15036: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:04:21.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:21 smithi082 ceph-mon[98694]: pgmap v15036: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:04:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:23 smithi170 ceph-mon[102832]: pgmap v15037: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:04:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27894.493809 secs 2023-09-10T07:04:23.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:23 smithi082 ceph-mon[98694]: pgmap v15037: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:04:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27894.493809 secs 2023-09-10T07:04:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:23 smithi098 ceph-mon[103124]: pgmap v15037: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:04:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27894.493809 secs 2023-09-10T07:04:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27895.632703 secs 2023-09-10T07:04:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27896.110417 secs 2023-09-10T07:04:24.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27895.632703 secs 2023-09-10T07:04:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27896.110417 secs 2023-09-10T07:04:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27895.632703 secs 2023-09-10T07:04:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27896.110417 secs 2023-09-10T07:04:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:25 smithi170 ceph-mon[102832]: pgmap v15038: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:04:25.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:25 smithi082 ceph-mon[98694]: pgmap v15038: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:04:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:25 smithi098 ceph-mon[103124]: pgmap v15038: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:04:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:27 smithi082 ceph-mon[98694]: pgmap v15039: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 100 B/s wr, 10 op/s 2023-09-10T07:04:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:27 smithi098 ceph-mon[103124]: pgmap v15039: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 100 B/s wr, 10 op/s 2023-09-10T07:04:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:27 smithi170 ceph-mon[102832]: pgmap v15039: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 100 B/s wr, 10 op/s 2023-09-10T07:04:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27900.632791 secs 2023-09-10T07:04:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:29 smithi170 ceph-mon[102832]: pgmap v15040: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:04:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27901.110484 secs 2023-09-10T07:04:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27900.632791 secs 2023-09-10T07:04:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:29 smithi082 ceph-mon[98694]: pgmap v15040: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:04:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27901.110484 secs 2023-09-10T07:04:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27900.632791 secs 2023-09-10T07:04:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:29 smithi098 ceph-mon[103124]: pgmap v15040: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:04:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27901.110484 secs 2023-09-10T07:04:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:31 smithi170 ceph-mon[102832]: pgmap v15041: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 188 KiB/s wr, 13 op/s 2023-09-10T07:04:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:04:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:31 smithi082 ceph-mon[98694]: pgmap v15041: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 188 KiB/s wr, 13 op/s 2023-09-10T07:04:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:04:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:31 smithi098 ceph-mon[103124]: pgmap v15041: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 188 KiB/s wr, 13 op/s 2023-09-10T07:04:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:04:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:04:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:04:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:04:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:04:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:04:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:04:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:04:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:04:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:04:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:33 smithi170 ceph-mon[102832]: pgmap v15042: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 188 KiB/s wr, 9 op/s 2023-09-10T07:04:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:33 smithi082 ceph-mon[98694]: pgmap v15042: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 188 KiB/s wr, 9 op/s 2023-09-10T07:04:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:33 smithi098 ceph-mon[103124]: pgmap v15042: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 188 KiB/s wr, 9 op/s 2023-09-10T07:04:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27905.632851 secs 2023-09-10T07:04:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:04:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:04:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27906.110544 secs 2023-09-10T07:04:34.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27905.632851 secs 2023-09-10T07:04:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:04:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:04:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27906.110544 secs 2023-09-10T07:04:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27905.632851 secs 2023-09-10T07:04:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:04:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:04:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27906.110544 secs 2023-09-10T07:04:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:35 smithi170 ceph-mon[102832]: pgmap v15043: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 188 KiB/s wr, 9 op/s 2023-09-10T07:04:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:35 smithi082 ceph-mon[98694]: pgmap v15043: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 188 KiB/s wr, 9 op/s 2023-09-10T07:04:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:35 smithi098 ceph-mon[103124]: pgmap v15043: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 188 KiB/s wr, 9 op/s 2023-09-10T07:04:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:04:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:04:36.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:04:36.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:04:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:04:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:04:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:37 smithi170 ceph-mon[102832]: pgmap v15044: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 641 KiB/s wr, 18 op/s 2023-09-10T07:04:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:37 smithi082 ceph-mon[98694]: pgmap v15044: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 641 KiB/s wr, 18 op/s 2023-09-10T07:04:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:37 smithi098 ceph-mon[103124]: pgmap v15044: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 641 KiB/s wr, 18 op/s 2023-09-10T07:04:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:04:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:04:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:04:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:04:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:04:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:04:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:04:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:04:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:04:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27910.632910 secs 2023-09-10T07:04:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:39 smithi082 ceph-mon[98694]: pgmap v15045: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 641 KiB/s wr, 13 op/s 2023-09-10T07:04:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27911.110612 secs 2023-09-10T07:04:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27910.632910 secs 2023-09-10T07:04:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:39 smithi098 ceph-mon[103124]: pgmap v15045: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 641 KiB/s wr, 13 op/s 2023-09-10T07:04:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27911.110612 secs 2023-09-10T07:04:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27910.632910 secs 2023-09-10T07:04:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:39 smithi170 ceph-mon[102832]: pgmap v15045: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 641 KiB/s wr, 13 op/s 2023-09-10T07:04:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27911.110612 secs 2023-09-10T07:04:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:42 smithi170 ceph-mon[102832]: pgmap v15046: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 644 KiB/s wr, 19 op/s 2023-09-10T07:04:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:04:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:04:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:04:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:04:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:04:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:04:42.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:04:42.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:04:42.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:04:42.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:04:42.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:04:42.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:04:42.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:04:42.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:04:42.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:04:42.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:04:42.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:04:42.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:04:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:42 smithi082 ceph-mon[98694]: pgmap v15046: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 644 KiB/s wr, 19 op/s 2023-09-10T07:04:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:04:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:04:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:04:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:04:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:04:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:04:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:04:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:04:42.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:04:42.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:04:42.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:04:42.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:04:42.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:04:42.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:04:42.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:04:42.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:04:42.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:04:42.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:04:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:42 smithi098 ceph-mon[103124]: pgmap v15046: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 644 KiB/s wr, 19 op/s 2023-09-10T07:04:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:04:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:04:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:04:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:04:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:04:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:04:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:04:42.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:04:42.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:04:42.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:04:42.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:04:42.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:04:42.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:04:42.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:04:42.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:04:42.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:04:42.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:04:42.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:04:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:43 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T07:04:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:43 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T07:04:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:43 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T07:04:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:43 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T07:04:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:43 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T07:04:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:43 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T07:04:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:43 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T07:04:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:43 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T07:04:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:43 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T07:04:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:43 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T07:04:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:43 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T07:04:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:43 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T07:04:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:44 smithi170 ceph-mon[102832]: pgmap v15047: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 456 KiB/s wr, 14 op/s 2023-09-10T07:04:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:04:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:04:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:44 smithi082 ceph-mon[98694]: pgmap v15047: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 456 KiB/s wr, 14 op/s 2023-09-10T07:04:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:04:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:04:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:44 smithi098 ceph-mon[103124]: pgmap v15047: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 456 KiB/s wr, 14 op/s 2023-09-10T07:04:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:04:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:04:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27915.632971 secs 2023-09-10T07:04:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27916.110674 secs 2023-09-10T07:04:45.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27915.632971 secs 2023-09-10T07:04:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27916.110674 secs 2023-09-10T07:04:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27915.632971 secs 2023-09-10T07:04:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27916.110674 secs 2023-09-10T07:04:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:46 smithi170 ceph-mon[102832]: pgmap v15048: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 456 KiB/s wr, 14 op/s 2023-09-10T07:04:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:04:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:04:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:46 smithi082 ceph-mon[98694]: pgmap v15048: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 456 KiB/s wr, 14 op/s 2023-09-10T07:04:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:04:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:04:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:46 smithi098 ceph-mon[103124]: pgmap v15048: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 456 KiB/s wr, 14 op/s 2023-09-10T07:04:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:04:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:04:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:47 smithi082 ceph-mon[98694]: pgmap v15049: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 826 KiB/s wr, 22 op/s 2023-09-10T07:04:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:47 smithi098 ceph-mon[103124]: pgmap v15049: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 826 KiB/s wr, 22 op/s 2023-09-10T07:04:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:47 smithi170 ceph-mon[102832]: pgmap v15049: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 826 KiB/s wr, 22 op/s 2023-09-10T07:04:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27919.494113 secs 2023-09-10T07:04:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27919.494113 secs 2023-09-10T07:04:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27919.494113 secs 2023-09-10T07:04:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27920.633029 secs 2023-09-10T07:04:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:49 smithi082 ceph-mon[98694]: pgmap v15050: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 373 KiB/s wr, 13 op/s 2023-09-10T07:04:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27920.633029 secs 2023-09-10T07:04:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:49 smithi098 ceph-mon[103124]: pgmap v15050: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 373 KiB/s wr, 13 op/s 2023-09-10T07:04:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27920.633029 secs 2023-09-10T07:04:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:49 smithi170 ceph-mon[102832]: pgmap v15050: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 373 KiB/s wr, 13 op/s 2023-09-10T07:04:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27921.110737 secs 2023-09-10T07:04:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27921.110737 secs 2023-09-10T07:04:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27921.110737 secs 2023-09-10T07:04:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:51 smithi098 ceph-mon[103124]: pgmap v15051: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 374 KiB/s wr, 19 op/s 2023-09-10T07:04:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:51 smithi170 ceph-mon[102832]: pgmap v15051: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 374 KiB/s wr, 19 op/s 2023-09-10T07:04:51.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:51 smithi082 ceph-mon[98694]: pgmap v15051: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 374 KiB/s wr, 19 op/s 2023-09-10T07:04:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:04:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:04:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:04:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:04:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:04:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:04:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:04:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:04:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:04:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:53 smithi098 ceph-mon[103124]: pgmap v15052: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 371 KiB/s wr, 13 op/s 2023-09-10T07:04:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27924.494173 secs 2023-09-10T07:04:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:53 smithi170 ceph-mon[102832]: pgmap v15052: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 371 KiB/s wr, 13 op/s 2023-09-10T07:04:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27924.494173 secs 2023-09-10T07:04:53.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:53 smithi082 ceph-mon[98694]: pgmap v15052: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 371 KiB/s wr, 13 op/s 2023-09-10T07:04:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27924.494173 secs 2023-09-10T07:04:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27925.633088 secs 2023-09-10T07:04:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27925.633088 secs 2023-09-10T07:04:54.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27925.633088 secs 2023-09-10T07:04:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27926.110803 secs 2023-09-10T07:04:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:55 smithi098 ceph-mon[103124]: pgmap v15053: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 371 KiB/s wr, 13 op/s 2023-09-10T07:04:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27926.110803 secs 2023-09-10T07:04:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:55 smithi170 ceph-mon[102832]: pgmap v15053: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 371 KiB/s wr, 13 op/s 2023-09-10T07:04:55.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27926.110803 secs 2023-09-10T07:04:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:55 smithi082 ceph-mon[98694]: pgmap v15053: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 371 KiB/s wr, 13 op/s 2023-09-10T07:04:57.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:57 smithi082 ceph-mon[98694]: pgmap v15054: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 579 KiB/s wr, 18 op/s 2023-09-10T07:04:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:57 smithi098 ceph-mon[103124]: pgmap v15054: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 579 KiB/s wr, 18 op/s 2023-09-10T07:04:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:57 smithi170 ceph-mon[102832]: pgmap v15054: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 579 KiB/s wr, 18 op/s 2023-09-10T07:04:58.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27929.494244 secs 2023-09-10T07:04:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27929.494244 secs 2023-09-10T07:04:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27929.494244 secs 2023-09-10T07:04:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27930.633173 secs 2023-09-10T07:04:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:04:59 smithi082 ceph-mon[98694]: pgmap v15055: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 209 KiB/s wr, 10 op/s 2023-09-10T07:04:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27930.633173 secs 2023-09-10T07:04:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:04:59 smithi098 ceph-mon[103124]: pgmap v15055: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 209 KiB/s wr, 10 op/s 2023-09-10T07:04:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27930.633173 secs 2023-09-10T07:04:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:04:59 smithi170 ceph-mon[102832]: pgmap v15055: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 209 KiB/s wr, 10 op/s 2023-09-10T07:05:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27931.110870 secs 2023-09-10T07:05:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27931.110870 secs 2023-09-10T07:05:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27931.110870 secs 2023-09-10T07:05:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:01 smithi098 ceph-mon[103124]: pgmap v15056: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 254 KiB/s wr, 29 op/s 2023-09-10T07:05:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:01 smithi170 ceph-mon[102832]: pgmap v15056: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 254 KiB/s wr, 29 op/s 2023-09-10T07:05:01.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:01 smithi082 ceph-mon[98694]: pgmap v15056: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 254 KiB/s wr, 29 op/s 2023-09-10T07:05:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:03 smithi170 ceph-mon[102832]: pgmap v15057: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 254 KiB/s wr, 24 op/s 2023-09-10T07:05:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27934.494303 secs 2023-09-10T07:05:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:03 smithi082 ceph-mon[98694]: pgmap v15057: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 254 KiB/s wr, 24 op/s 2023-09-10T07:05:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27934.494303 secs 2023-09-10T07:05:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:03 smithi098 ceph-mon[103124]: pgmap v15057: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 254 KiB/s wr, 24 op/s 2023-09-10T07:05:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27934.494303 secs 2023-09-10T07:05:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27935.633231 secs 2023-09-10T07:05:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27936.110930 secs 2023-09-10T07:05:04.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27935.633231 secs 2023-09-10T07:05:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27936.110930 secs 2023-09-10T07:05:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27935.633231 secs 2023-09-10T07:05:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27936.110930 secs 2023-09-10T07:05:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:05 smithi170 ceph-mon[102832]: pgmap v15058: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 254 KiB/s wr, 24 op/s 2023-09-10T07:05:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:05 smithi082 ceph-mon[98694]: pgmap v15058: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 254 KiB/s wr, 24 op/s 2023-09-10T07:05:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:05 smithi098 ceph-mon[103124]: pgmap v15058: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 254 KiB/s wr, 24 op/s 2023-09-10T07:05:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:07 smithi082 ceph-mon[98694]: pgmap v15059: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 344 KiB/s wr, 29 op/s 2023-09-10T07:05:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:07 smithi098 ceph-mon[103124]: pgmap v15059: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 344 KiB/s wr, 29 op/s 2023-09-10T07:05:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:07 smithi170 ceph-mon[102832]: pgmap v15059: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 344 KiB/s wr, 29 op/s 2023-09-10T07:05:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27939.494467 secs 2023-09-10T07:05:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27939.494467 secs 2023-09-10T07:05:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27939.494467 secs 2023-09-10T07:05:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27940.633290 secs 2023-09-10T07:05:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:09 smithi098 ceph-mon[103124]: osdmap e1009: 12 total, 12 up, 12 in 2023-09-10T07:05:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:09 smithi098 ceph-mon[103124]: pgmap v15061: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 163 KiB/s wr, 29 op/s 2023-09-10T07:05:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27940.633290 secs 2023-09-10T07:05:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:09 smithi170 ceph-mon[102832]: osdmap e1009: 12 total, 12 up, 12 in 2023-09-10T07:05:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:09 smithi170 ceph-mon[102832]: pgmap v15061: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 163 KiB/s wr, 29 op/s 2023-09-10T07:05:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27940.633290 secs 2023-09-10T07:05:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:09 smithi082 ceph-mon[98694]: osdmap e1009: 12 total, 12 up, 12 in 2023-09-10T07:05:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:09 smithi082 ceph-mon[98694]: pgmap v15061: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 163 KiB/s wr, 29 op/s 2023-09-10T07:05:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27941.110996 secs 2023-09-10T07:05:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27941.110996 secs 2023-09-10T07:05:10.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27941.110996 secs 2023-09-10T07:05:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:11 smithi098 ceph-mon[103124]: pgmap v15062: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T07:05:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:11 smithi170 ceph-mon[102832]: pgmap v15062: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T07:05:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:11 smithi082 ceph-mon[98694]: pgmap v15062: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T07:05:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:13 smithi170 ceph-mon[102832]: pgmap v15063: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T07:05:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27944.494529 secs 2023-09-10T07:05:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:13 smithi082 ceph-mon[98694]: pgmap v15063: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T07:05:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27944.494529 secs 2023-09-10T07:05:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:13 smithi098 ceph-mon[103124]: pgmap v15063: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T07:05:14.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27944.494529 secs 2023-09-10T07:05:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27945.633363 secs 2023-09-10T07:05:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27946.111062 secs 2023-09-10T07:05:14.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27945.633363 secs 2023-09-10T07:05:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27946.111062 secs 2023-09-10T07:05:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27945.633363 secs 2023-09-10T07:05:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27946.111062 secs 2023-09-10T07:05:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:15 smithi170 ceph-mon[102832]: pgmap v15064: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T07:05:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:15 smithi082 ceph-mon[98694]: pgmap v15064: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T07:05:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:15 smithi098 ceph-mon[103124]: pgmap v15064: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T07:05:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:05:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:05:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:16 smithi170 ceph-mon[102832]: osdmap e1010: 12 total, 12 up, 12 in 2023-09-10T07:05:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:05:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:05:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:16 smithi082 ceph-mon[98694]: osdmap e1010: 12 total, 12 up, 12 in 2023-09-10T07:05:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:05:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:05:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:16 smithi098 ceph-mon[103124]: osdmap e1010: 12 total, 12 up, 12 in 2023-09-10T07:05:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:17 smithi170 ceph-mon[102832]: pgmap v15066: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 377 B/s wr, 15 op/s 2023-09-10T07:05:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:17 smithi082 ceph-mon[98694]: pgmap v15066: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 377 B/s wr, 15 op/s 2023-09-10T07:05:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:17 smithi098 ceph-mon[103124]: pgmap v15066: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 377 B/s wr, 15 op/s 2023-09-10T07:05:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:05:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:05:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27949.494589 secs 2023-09-10T07:05:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:05:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:05:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27949.494589 secs 2023-09-10T07:05:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:05:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:05:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27949.494589 secs 2023-09-10T07:05:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27950.633436 secs 2023-09-10T07:05:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:19 smithi082 ceph-mon[98694]: pgmap v15067: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 12 op/s 2023-09-10T07:05:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27951.111124 secs 2023-09-10T07:05:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27950.633436 secs 2023-09-10T07:05:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:19 smithi098 ceph-mon[103124]: pgmap v15067: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 12 op/s 2023-09-10T07:05:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27951.111124 secs 2023-09-10T07:05:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27950.633436 secs 2023-09-10T07:05:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:19 smithi170 ceph-mon[102832]: pgmap v15067: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 12 op/s 2023-09-10T07:05:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27951.111124 secs 2023-09-10T07:05:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:21 smithi082 ceph-mon[98694]: pgmap v15068: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 716 B/s wr, 12 op/s 2023-09-10T07:05:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:21 smithi098 ceph-mon[103124]: pgmap v15068: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 716 B/s wr, 12 op/s 2023-09-10T07:05:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:21 smithi170 ceph-mon[102832]: pgmap v15068: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 716 B/s wr, 12 op/s 2023-09-10T07:05:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:23 smithi082 ceph-mon[98694]: pgmap v15069: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 716 B/s wr, 12 op/s 2023-09-10T07:05:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27954.494650 secs 2023-09-10T07:05:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:23 smithi098 ceph-mon[103124]: pgmap v15069: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 716 B/s wr, 12 op/s 2023-09-10T07:05:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27954.494650 secs 2023-09-10T07:05:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:23 smithi170 ceph-mon[102832]: pgmap v15069: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 716 B/s wr, 12 op/s 2023-09-10T07:05:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27954.494650 secs 2023-09-10T07:05:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27955.633492 secs 2023-09-10T07:05:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27956.111184 secs 2023-09-10T07:05:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27955.633492 secs 2023-09-10T07:05:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27956.111184 secs 2023-09-10T07:05:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27955.633492 secs 2023-09-10T07:05:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27956.111184 secs 2023-09-10T07:05:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:25 smithi082 ceph-mon[98694]: pgmap v15070: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 716 B/s wr, 12 op/s 2023-09-10T07:05:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:25 smithi098 ceph-mon[103124]: pgmap v15070: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 716 B/s wr, 12 op/s 2023-09-10T07:05:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:25 smithi170 ceph-mon[102832]: pgmap v15070: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 716 B/s wr, 12 op/s 2023-09-10T07:05:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:27 smithi098 ceph-mon[103124]: pgmap v15071: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T07:05:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:27 smithi170 ceph-mon[102832]: pgmap v15071: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T07:05:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:27 smithi082 ceph-mon[98694]: pgmap v15071: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T07:05:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27960.633550 secs 2023-09-10T07:05:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:29 smithi170 ceph-mon[102832]: pgmap v15072: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 10 op/s 2023-09-10T07:05:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27961.111252 secs 2023-09-10T07:05:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27960.633550 secs 2023-09-10T07:05:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:29 smithi082 ceph-mon[98694]: pgmap v15072: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 10 op/s 2023-09-10T07:05:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27961.111252 secs 2023-09-10T07:05:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27960.633550 secs 2023-09-10T07:05:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:29 smithi098 ceph-mon[103124]: pgmap v15072: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 10 op/s 2023-09-10T07:05:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27961.111252 secs 2023-09-10T07:05:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:31 smithi170 ceph-mon[102832]: pgmap v15073: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 193 KiB/s wr, 14 op/s 2023-09-10T07:05:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:05:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:31 smithi082 ceph-mon[98694]: pgmap v15073: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 193 KiB/s wr, 14 op/s 2023-09-10T07:05:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:05:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:31 smithi098 ceph-mon[103124]: pgmap v15073: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 193 KiB/s wr, 14 op/s 2023-09-10T07:05:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:05:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:05:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:05:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:05:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:05:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:05:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:05:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:05:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:05:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:05:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:33 smithi170 ceph-mon[102832]: pgmap v15074: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 9 op/s 2023-09-10T07:05:33.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:33 smithi082 ceph-mon[98694]: pgmap v15074: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 9 op/s 2023-09-10T07:05:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:33 smithi098 ceph-mon[103124]: pgmap v15074: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 9 op/s 2023-09-10T07:05:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27965.633633 secs 2023-09-10T07:05:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27966.111310 secs 2023-09-10T07:05:34.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27965.633633 secs 2023-09-10T07:05:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27966.111310 secs 2023-09-10T07:05:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27965.633633 secs 2023-09-10T07:05:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27966.111310 secs 2023-09-10T07:05:35.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:35 smithi082 ceph-mon[98694]: pgmap v15075: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 9 op/s 2023-09-10T07:05:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:35 smithi098 ceph-mon[103124]: pgmap v15075: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 9 op/s 2023-09-10T07:05:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:35 smithi170 ceph-mon[102832]: pgmap v15075: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 9 op/s 2023-09-10T07:05:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:37 smithi098 ceph-mon[103124]: pgmap v15076: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 309 KiB/s wr, 15 op/s 2023-09-10T07:05:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:37 smithi170 ceph-mon[102832]: pgmap v15076: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 309 KiB/s wr, 15 op/s 2023-09-10T07:05:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:37 smithi082 ceph-mon[98694]: pgmap v15076: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 309 KiB/s wr, 15 op/s 2023-09-10T07:05:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:05:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:05:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:05:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:05:38.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:05:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:05:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27970.633721 secs 2023-09-10T07:05:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:39 smithi098 ceph-mon[103124]: pgmap v15077: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 212 KiB/s wr, 10 op/s 2023-09-10T07:05:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27970.633721 secs 2023-09-10T07:05:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:39 smithi170 ceph-mon[102832]: pgmap v15077: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 212 KiB/s wr, 10 op/s 2023-09-10T07:05:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27970.633721 secs 2023-09-10T07:05:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:39 smithi082 ceph-mon[98694]: pgmap v15077: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 212 KiB/s wr, 10 op/s 2023-09-10T07:05:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27971.111378 secs 2023-09-10T07:05:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:05:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:05:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27971.111378 secs 2023-09-10T07:05:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:05:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:05:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27971.111378 secs 2023-09-10T07:05:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:05:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:05:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:41 smithi098 ceph-mon[103124]: pgmap v15078: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 344 KiB/s wr, 16 op/s 2023-09-10T07:05:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:41 smithi170 ceph-mon[102832]: pgmap v15078: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 344 KiB/s wr, 16 op/s 2023-09-10T07:05:41.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:41 smithi082 ceph-mon[98694]: pgmap v15078: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 344 KiB/s wr, 16 op/s 2023-09-10T07:05:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:05:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:05:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:05:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:05:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:05:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:05:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:05:42.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:05:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:05:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:05:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:05:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:05:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:43 smithi098 ceph-mon[103124]: pgmap v15079: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 249 KiB/s wr, 11 op/s 2023-09-10T07:05:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:43 smithi170 ceph-mon[102832]: pgmap v15079: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 249 KiB/s wr, 11 op/s 2023-09-10T07:05:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:43 smithi082 ceph-mon[98694]: pgmap v15079: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 249 KiB/s wr, 11 op/s 2023-09-10T07:05:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27975.633786 secs 2023-09-10T07:05:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27975.633786 secs 2023-09-10T07:05:44.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27975.633786 secs 2023-09-10T07:05:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27976.111462 secs 2023-09-10T07:05:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:45 smithi098 ceph-mon[103124]: pgmap v15080: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 249 KiB/s wr, 11 op/s 2023-09-10T07:05:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27976.111462 secs 2023-09-10T07:05:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:45 smithi170 ceph-mon[102832]: pgmap v15080: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 249 KiB/s wr, 11 op/s 2023-09-10T07:05:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27976.111462 secs 2023-09-10T07:05:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:45 smithi082 ceph-mon[98694]: pgmap v15080: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 249 KiB/s wr, 11 op/s 2023-09-10T07:05:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:47 smithi098 ceph-mon[103124]: pgmap v15081: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 545 KiB/s wr, 19 op/s 2023-09-10T07:05:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:47 smithi170 ceph-mon[102832]: pgmap v15081: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 545 KiB/s wr, 19 op/s 2023-09-10T07:05:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:47 smithi082 ceph-mon[98694]: pgmap v15081: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 545 KiB/s wr, 19 op/s 2023-09-10T07:05:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27980.633843 secs 2023-09-10T07:05:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:49 smithi170 ceph-mon[102832]: pgmap v15082: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 428 KiB/s wr, 13 op/s 2023-09-10T07:05:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27981.111487 secs 2023-09-10T07:05:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27980.633843 secs 2023-09-10T07:05:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:49 smithi082 ceph-mon[98694]: pgmap v15082: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 428 KiB/s wr, 13 op/s 2023-09-10T07:05:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27981.111487 secs 2023-09-10T07:05:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27980.633843 secs 2023-09-10T07:05:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:49 smithi098 ceph-mon[103124]: pgmap v15082: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 428 KiB/s wr, 13 op/s 2023-09-10T07:05:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27981.111487 secs 2023-09-10T07:05:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:05:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:05:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:05:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:05:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:05:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:05:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:51 smithi082 ceph-mon[98694]: pgmap v15083: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 454 KiB/s wr, 19 op/s 2023-09-10T07:05:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:51 smithi098 ceph-mon[103124]: pgmap v15083: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 454 KiB/s wr, 19 op/s 2023-09-10T07:05:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:51 smithi170 ceph-mon[102832]: pgmap v15083: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 454 KiB/s wr, 19 op/s 2023-09-10T07:05:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:05:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:05:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:05:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:05:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:05:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:05:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:53 smithi082 ceph-mon[98694]: pgmap v15084: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 322 KiB/s wr, 13 op/s 2023-09-10T07:05:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27984.495123 secs 2023-09-10T07:05:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:53 smithi098 ceph-mon[103124]: pgmap v15084: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 322 KiB/s wr, 13 op/s 2023-09-10T07:05:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27984.495123 secs 2023-09-10T07:05:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:53 smithi170 ceph-mon[102832]: pgmap v15084: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 322 KiB/s wr, 13 op/s 2023-09-10T07:05:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27984.495123 secs 2023-09-10T07:05:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27985.633901 secs 2023-09-10T07:05:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27986.111546 secs 2023-09-10T07:05:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27985.633901 secs 2023-09-10T07:05:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27986.111546 secs 2023-09-10T07:05:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27985.633901 secs 2023-09-10T07:05:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27986.111546 secs 2023-09-10T07:05:55.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:55 smithi082 ceph-mon[98694]: pgmap v15085: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 322 KiB/s wr, 13 op/s 2023-09-10T07:05:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:55 smithi098 ceph-mon[103124]: pgmap v15085: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 322 KiB/s wr, 13 op/s 2023-09-10T07:05:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:55 smithi170 ceph-mon[102832]: pgmap v15085: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 322 KiB/s wr, 13 op/s 2023-09-10T07:05:56.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:05:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:05:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:05:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:05:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:05:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:05:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:05:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:05:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:05:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:57 smithi082 ceph-mon[98694]: pgmap v15086: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 566 KiB/s wr, 19 op/s 2023-09-10T07:05:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:57 smithi098 ceph-mon[103124]: pgmap v15086: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 566 KiB/s wr, 19 op/s 2023-09-10T07:05:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:57 smithi170 ceph-mon[102832]: pgmap v15086: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 566 KiB/s wr, 19 op/s 2023-09-10T07:05:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27989.495181 secs 2023-09-10T07:05:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27989.495181 secs 2023-09-10T07:05:59.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27989.495181 secs 2023-09-10T07:06:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27990.633963 secs 2023-09-10T07:06:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:59 smithi098 ceph-mon[103124]: pgmap v15087: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T07:06:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:05:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27991.111614 secs 2023-09-10T07:06:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27990.633963 secs 2023-09-10T07:06:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:59 smithi170 ceph-mon[102832]: pgmap v15087: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T07:06:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:05:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27991.111614 secs 2023-09-10T07:06:00.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27990.633963 secs 2023-09-10T07:06:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:59 smithi082 ceph-mon[98694]: pgmap v15087: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T07:06:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:05:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27991.111614 secs 2023-09-10T07:06:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:01 smithi098 ceph-mon[103124]: pgmap v15088: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 378 KiB/s wr, 30 op/s 2023-09-10T07:06:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:01 smithi170 ceph-mon[102832]: pgmap v15088: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 378 KiB/s wr, 30 op/s 2023-09-10T07:06:02.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:01 smithi082 ceph-mon[98694]: pgmap v15088: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 378 KiB/s wr, 30 op/s 2023-09-10T07:06:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:03 smithi098 ceph-mon[103124]: pgmap v15089: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 352 KiB/s wr, 24 op/s 2023-09-10T07:06:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27994.495242 secs 2023-09-10T07:06:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:03 smithi170 ceph-mon[102832]: pgmap v15089: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 352 KiB/s wr, 24 op/s 2023-09-10T07:06:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27994.495242 secs 2023-09-10T07:06:04.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:03 smithi082 ceph-mon[98694]: pgmap v15089: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 352 KiB/s wr, 24 op/s 2023-09-10T07:06:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27994.495242 secs 2023-09-10T07:06:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27995.634021 secs 2023-09-10T07:06:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:06:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:06:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 27996.111674 secs 2023-09-10T07:06:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27995.634021 secs 2023-09-10T07:06:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:06:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:06:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 27996.111674 secs 2023-09-10T07:06:05.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27995.634021 secs 2023-09-10T07:06:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:06:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:06:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 27996.111674 secs 2023-09-10T07:06:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:05 smithi098 ceph-mon[103124]: pgmap v15090: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 352 KiB/s wr, 24 op/s 2023-09-10T07:06:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:05 smithi170 ceph-mon[102832]: pgmap v15090: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 352 KiB/s wr, 24 op/s 2023-09-10T07:06:06.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:05 smithi082 ceph-mon[98694]: pgmap v15090: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 352 KiB/s wr, 24 op/s 2023-09-10T07:06:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:06:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:06:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:06:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:06:07.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:06:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:06:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:07 smithi098 ceph-mon[103124]: pgmap v15091: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 364 KiB/s wr, 30 op/s 2023-09-10T07:06:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:07 smithi170 ceph-mon[102832]: pgmap v15091: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 364 KiB/s wr, 30 op/s 2023-09-10T07:06:08.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:07 smithi082 ceph-mon[98694]: pgmap v15091: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 364 KiB/s wr, 30 op/s 2023-09-10T07:06:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 27999.495303 secs 2023-09-10T07:06:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 27999.495303 secs 2023-09-10T07:06:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 27999.495303 secs 2023-09-10T07:06:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28000.634078 secs 2023-09-10T07:06:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:09 smithi098 ceph-mon[103124]: pgmap v15092: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 121 KiB/s wr, 24 op/s 2023-09-10T07:06:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:09 smithi098 ceph-mon[103124]: osdmap e1011: 12 total, 12 up, 12 in 2023-09-10T07:06:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28001.111744 secs 2023-09-10T07:06:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28000.634078 secs 2023-09-10T07:06:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:09 smithi170 ceph-mon[102832]: pgmap v15092: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 121 KiB/s wr, 24 op/s 2023-09-10T07:06:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:09 smithi170 ceph-mon[102832]: osdmap e1011: 12 total, 12 up, 12 in 2023-09-10T07:06:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28001.111744 secs 2023-09-10T07:06:10.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28000.634078 secs 2023-09-10T07:06:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:09 smithi082 ceph-mon[98694]: pgmap v15092: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 121 KiB/s wr, 24 op/s 2023-09-10T07:06:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:09 smithi082 ceph-mon[98694]: osdmap e1011: 12 total, 12 up, 12 in 2023-09-10T07:06:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28001.111744 secs 2023-09-10T07:06:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:11 smithi098 ceph-mon[103124]: pgmap v15094: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 14 KiB/s wr, 13 op/s 2023-09-10T07:06:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:11 smithi098 ceph-mon[103124]: osdmap e1012: 12 total, 12 up, 12 in 2023-09-10T07:06:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:11 smithi170 ceph-mon[102832]: pgmap v15094: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 14 KiB/s wr, 13 op/s 2023-09-10T07:06:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:11 smithi170 ceph-mon[102832]: osdmap e1012: 12 total, 12 up, 12 in 2023-09-10T07:06:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:11 smithi082 ceph-mon[98694]: pgmap v15094: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 14 KiB/s wr, 13 op/s 2023-09-10T07:06:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:11 smithi082 ceph-mon[98694]: osdmap e1012: 12 total, 12 up, 12 in 2023-09-10T07:06:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:13 smithi098 ceph-mon[103124]: pgmap v15096: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 18 KiB/s wr, 16 op/s 2023-09-10T07:06:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28004.495361 secs 2023-09-10T07:06:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:13 smithi170 ceph-mon[102832]: pgmap v15096: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 18 KiB/s wr, 16 op/s 2023-09-10T07:06:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28004.495361 secs 2023-09-10T07:06:14.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:13 smithi082 ceph-mon[98694]: pgmap v15096: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 18 KiB/s wr, 16 op/s 2023-09-10T07:06:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28004.495361 secs 2023-09-10T07:06:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28005.634148 secs 2023-09-10T07:06:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28006.111805 secs 2023-09-10T07:06:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28005.634148 secs 2023-09-10T07:06:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28006.111805 secs 2023-09-10T07:06:15.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28005.634148 secs 2023-09-10T07:06:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28006.111805 secs 2023-09-10T07:06:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:15 smithi098 ceph-mon[103124]: pgmap v15097: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:06:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:15 smithi170 ceph-mon[102832]: pgmap v15097: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:06:16.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:15 smithi082 ceph-mon[98694]: pgmap v15097: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:06:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:17 smithi098 ceph-mon[103124]: pgmap v15098: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 14 op/s 2023-09-10T07:06:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:17 smithi170 ceph-mon[102832]: pgmap v15098: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 14 op/s 2023-09-10T07:06:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:17 smithi082 ceph-mon[98694]: pgmap v15098: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 14 op/s 2023-09-10T07:06:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28009.495422 secs 2023-09-10T07:06:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28009.495422 secs 2023-09-10T07:06:18.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28009.495422 secs 2023-09-10T07:06:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28010.634207 secs 2023-09-10T07:06:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:19 smithi098 ceph-mon[103124]: pgmap v15099: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 310 B/s wr, 11 op/s 2023-09-10T07:06:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28010.634207 secs 2023-09-10T07:06:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:19 smithi170 ceph-mon[102832]: pgmap v15099: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 310 B/s wr, 11 op/s 2023-09-10T07:06:19.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28010.634207 secs 2023-09-10T07:06:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:19 smithi082 ceph-mon[98694]: pgmap v15099: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 310 B/s wr, 11 op/s 2023-09-10T07:06:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28011.111870 secs 2023-09-10T07:06:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28011.111870 secs 2023-09-10T07:06:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28011.111870 secs 2023-09-10T07:06:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:21 smithi098 ceph-mon[103124]: pgmap v15100: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T07:06:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:21 smithi170 ceph-mon[102832]: pgmap v15100: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T07:06:21.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:21 smithi082 ceph-mon[98694]: pgmap v15100: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T07:06:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:23 smithi170 ceph-mon[102832]: pgmap v15101: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 275 B/s wr, 9 op/s 2023-09-10T07:06:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28014.495535 secs 2023-09-10T07:06:23.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:23 smithi082 ceph-mon[98694]: pgmap v15101: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 275 B/s wr, 9 op/s 2023-09-10T07:06:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28014.495535 secs 2023-09-10T07:06:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:23 smithi098 ceph-mon[103124]: pgmap v15101: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 275 B/s wr, 9 op/s 2023-09-10T07:06:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28014.495535 secs 2023-09-10T07:06:24.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28015.634295 secs 2023-09-10T07:06:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28016.111930 secs 2023-09-10T07:06:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28015.634295 secs 2023-09-10T07:06:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28016.111930 secs 2023-09-10T07:06:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28015.634295 secs 2023-09-10T07:06:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28016.111930 secs 2023-09-10T07:06:25.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:25 smithi082 ceph-mon[98694]: pgmap v15102: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2023-09-10T07:06:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:25 smithi098 ceph-mon[103124]: pgmap v15102: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2023-09-10T07:06:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:25 smithi170 ceph-mon[102832]: pgmap v15102: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2023-09-10T07:06:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:27 smithi098 ceph-mon[103124]: pgmap v15103: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 13 op/s 2023-09-10T07:06:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:27 smithi170 ceph-mon[102832]: pgmap v15103: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 13 op/s 2023-09-10T07:06:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:27 smithi082 ceph-mon[98694]: pgmap v15103: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 13 op/s 2023-09-10T07:06:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28020.634382 secs 2023-09-10T07:06:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:29 smithi170 ceph-mon[102832]: pgmap v15104: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-10T07:06:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28021.111998 secs 2023-09-10T07:06:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28020.634382 secs 2023-09-10T07:06:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:29 smithi082 ceph-mon[98694]: pgmap v15104: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-10T07:06:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28021.111998 secs 2023-09-10T07:06:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28020.634382 secs 2023-09-10T07:06:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:29 smithi098 ceph-mon[103124]: pgmap v15104: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-10T07:06:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28021.111998 secs 2023-09-10T07:06:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:31 smithi170 ceph-mon[102832]: pgmap v15105: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 193 KiB/s wr, 13 op/s 2023-09-10T07:06:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:06:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:31 smithi082 ceph-mon[98694]: pgmap v15105: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 193 KiB/s wr, 13 op/s 2023-09-10T07:06:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:06:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:31 smithi098 ceph-mon[103124]: pgmap v15105: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 193 KiB/s wr, 13 op/s 2023-09-10T07:06:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:06:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:06:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:06:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:06:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:06:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:06:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:06:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:06:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:06:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:06:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:33 smithi082 ceph-mon[98694]: pgmap v15106: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 9 op/s 2023-09-10T07:06:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:33 smithi098 ceph-mon[103124]: pgmap v15106: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 9 op/s 2023-09-10T07:06:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:33 smithi170 ceph-mon[102832]: pgmap v15106: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 9 op/s 2023-09-10T07:06:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28025.634452 secs 2023-09-10T07:06:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28026.112062 secs 2023-09-10T07:06:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28025.634452 secs 2023-09-10T07:06:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28026.112062 secs 2023-09-10T07:06:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28025.634452 secs 2023-09-10T07:06:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28026.112062 secs 2023-09-10T07:06:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:35 smithi082 ceph-mon[98694]: pgmap v15107: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 9 op/s 2023-09-10T07:06:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:35 smithi098 ceph-mon[103124]: pgmap v15107: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 9 op/s 2023-09-10T07:06:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:35 smithi170 ceph-mon[102832]: pgmap v15107: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 9 op/s 2023-09-10T07:06:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:37 smithi098 ceph-mon[103124]: pgmap v15108: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 434 KiB/s wr, 15 op/s 2023-09-10T07:06:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:37 smithi170 ceph-mon[102832]: pgmap v15108: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 434 KiB/s wr, 15 op/s 2023-09-10T07:06:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:37 smithi082 ceph-mon[98694]: pgmap v15108: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 434 KiB/s wr, 15 op/s 2023-09-10T07:06:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:06:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:06:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:06:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:06:38.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:06:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:06:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28030.634494 secs 2023-09-10T07:06:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:39 smithi098 ceph-mon[103124]: pgmap v15109: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 434 KiB/s wr, 10 op/s 2023-09-10T07:06:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28030.634494 secs 2023-09-10T07:06:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:39 smithi170 ceph-mon[102832]: pgmap v15109: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 434 KiB/s wr, 10 op/s 2023-09-10T07:06:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28030.634494 secs 2023-09-10T07:06:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:39 smithi082 ceph-mon[98694]: pgmap v15109: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 434 KiB/s wr, 10 op/s 2023-09-10T07:06:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28031.112128 secs 2023-09-10T07:06:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:06:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:06:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28031.112128 secs 2023-09-10T07:06:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:06:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:06:40.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28031.112128 secs 2023-09-10T07:06:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:06:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:06:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:41 smithi098 ceph-mon[103124]: pgmap v15110: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 654 KiB/s wr, 17 op/s 2023-09-10T07:06:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:41 smithi170 ceph-mon[102832]: pgmap v15110: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 654 KiB/s wr, 17 op/s 2023-09-10T07:06:41.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:41 smithi082 ceph-mon[98694]: pgmap v15110: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 654 KiB/s wr, 17 op/s 2023-09-10T07:06:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:06:42.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:06:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:06:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:44 smithi170 ceph-mon[102832]: pgmap v15111: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 461 KiB/s wr, 12 op/s 2023-09-10T07:06:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:06:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:06:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:06:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:06:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:06:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:06:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:06:44.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:44 smithi082 ceph-mon[98694]: pgmap v15111: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 461 KiB/s wr, 12 op/s 2023-09-10T07:06:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:06:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:06:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:06:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:06:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:06:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:06:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:06:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:44 smithi098 ceph-mon[103124]: pgmap v15111: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 461 KiB/s wr, 12 op/s 2023-09-10T07:06:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:06:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:06:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:06:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:06:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:06:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:06:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:06:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28035.634570 secs 2023-09-10T07:06:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28036.112194 secs 2023-09-10T07:06:45.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28035.634570 secs 2023-09-10T07:06:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28036.112194 secs 2023-09-10T07:06:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28035.634570 secs 2023-09-10T07:06:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28036.112194 secs 2023-09-10T07:06:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:46 smithi170 ceph-mon[102832]: pgmap v15112: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 461 KiB/s wr, 12 op/s 2023-09-10T07:06:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:06:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:06:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:46 smithi082 ceph-mon[98694]: pgmap v15112: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 461 KiB/s wr, 12 op/s 2023-09-10T07:06:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:06:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:06:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:46 smithi098 ceph-mon[103124]: pgmap v15112: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 461 KiB/s wr, 12 op/s 2023-09-10T07:06:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:06:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:06:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:47 smithi098 ceph-mon[103124]: pgmap v15113: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 669 KiB/s wr, 19 op/s 2023-09-10T07:06:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:47 smithi170 ceph-mon[102832]: pgmap v15113: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 669 KiB/s wr, 19 op/s 2023-09-10T07:06:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:47 smithi082 ceph-mon[98694]: pgmap v15113: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 669 KiB/s wr, 19 op/s 2023-09-10T07:06:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28040.634631 secs 2023-09-10T07:06:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:49 smithi082 ceph-mon[98694]: pgmap v15114: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 428 KiB/s wr, 13 op/s 2023-09-10T07:06:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28041.112261 secs 2023-09-10T07:06:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28040.634631 secs 2023-09-10T07:06:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:49 smithi098 ceph-mon[103124]: pgmap v15114: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 428 KiB/s wr, 13 op/s 2023-09-10T07:06:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28041.112261 secs 2023-09-10T07:06:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28040.634631 secs 2023-09-10T07:06:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:49 smithi170 ceph-mon[102832]: pgmap v15114: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 428 KiB/s wr, 13 op/s 2023-09-10T07:06:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28041.112261 secs 2023-09-10T07:06:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:51 smithi082 ceph-mon[98694]: pgmap v15115: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 624 KiB/s wr, 19 op/s 2023-09-10T07:06:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:51 smithi098 ceph-mon[103124]: pgmap v15115: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 624 KiB/s wr, 19 op/s 2023-09-10T07:06:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:51 smithi170 ceph-mon[102832]: pgmap v15115: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 624 KiB/s wr, 19 op/s 2023-09-10T07:06:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:53 smithi082 ceph-mon[98694]: pgmap v15116: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 403 KiB/s wr, 13 op/s 2023-09-10T07:06:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28044.496077 secs 2023-09-10T07:06:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:53 smithi098 ceph-mon[103124]: pgmap v15116: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 403 KiB/s wr, 13 op/s 2023-09-10T07:06:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28044.496077 secs 2023-09-10T07:06:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:53 smithi170 ceph-mon[102832]: pgmap v15116: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 403 KiB/s wr, 13 op/s 2023-09-10T07:06:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28044.496077 secs 2023-09-10T07:06:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28045.634686 secs 2023-09-10T07:06:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:06:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:06:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28046.112323 secs 2023-09-10T07:06:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28045.634686 secs 2023-09-10T07:06:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:06:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:06:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28046.112323 secs 2023-09-10T07:06:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28045.634686 secs 2023-09-10T07:06:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:06:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:06:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28046.112323 secs 2023-09-10T07:06:55.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:55 smithi082 ceph-mon[98694]: pgmap v15117: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 403 KiB/s wr, 13 op/s 2023-09-10T07:06:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:55 smithi098 ceph-mon[103124]: pgmap v15117: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 403 KiB/s wr, 13 op/s 2023-09-10T07:06:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:55 smithi170 ceph-mon[102832]: pgmap v15117: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 403 KiB/s wr, 13 op/s 2023-09-10T07:06:56.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:06:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:06:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:06:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:06:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:06:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:06:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:06:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:06:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:06:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:57 smithi082 ceph-mon[98694]: pgmap v15118: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 569 KiB/s wr, 19 op/s 2023-09-10T07:06:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:57 smithi098 ceph-mon[103124]: pgmap v15118: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 569 KiB/s wr, 19 op/s 2023-09-10T07:06:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:57 smithi170 ceph-mon[102832]: pgmap v15118: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 569 KiB/s wr, 19 op/s 2023-09-10T07:06:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28049.496138 secs 2023-09-10T07:06:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28049.496138 secs 2023-09-10T07:06:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28049.496138 secs 2023-09-10T07:07:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28050.634771 secs 2023-09-10T07:07:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:59 smithi098 ceph-mon[103124]: pgmap v15119: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 361 KiB/s wr, 12 op/s 2023-09-10T07:07:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:06:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28051.112389 secs 2023-09-10T07:07:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28050.634771 secs 2023-09-10T07:07:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:59 smithi170 ceph-mon[102832]: pgmap v15119: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 361 KiB/s wr, 12 op/s 2023-09-10T07:07:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:06:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28051.112389 secs 2023-09-10T07:07:00.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28050.634771 secs 2023-09-10T07:07:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:59 smithi082 ceph-mon[98694]: pgmap v15119: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 361 KiB/s wr, 12 op/s 2023-09-10T07:07:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:06:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28051.112389 secs 2023-09-10T07:07:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:01 smithi098 ceph-mon[103124]: pgmap v15120: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 479 KiB/s wr, 19 op/s 2023-09-10T07:07:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:01 smithi170 ceph-mon[102832]: pgmap v15120: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 479 KiB/s wr, 19 op/s 2023-09-10T07:07:02.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:01 smithi082 ceph-mon[98694]: pgmap v15120: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 479 KiB/s wr, 19 op/s 2023-09-10T07:07:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:03 smithi098 ceph-mon[103124]: pgmap v15121: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 283 KiB/s wr, 13 op/s 2023-09-10T07:07:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28054.496218 secs 2023-09-10T07:07:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:03 smithi170 ceph-mon[102832]: pgmap v15121: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 283 KiB/s wr, 13 op/s 2023-09-10T07:07:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28054.496218 secs 2023-09-10T07:07:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:03 smithi082 ceph-mon[98694]: pgmap v15121: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 283 KiB/s wr, 13 op/s 2023-09-10T07:07:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28054.496218 secs 2023-09-10T07:07:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28055.634869 secs 2023-09-10T07:07:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28056.112463 secs 2023-09-10T07:07:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28055.634869 secs 2023-09-10T07:07:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28056.112463 secs 2023-09-10T07:07:05.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28055.634869 secs 2023-09-10T07:07:05.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28056.112463 secs 2023-09-10T07:07:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:05 smithi098 ceph-mon[103124]: pgmap v15122: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 283 KiB/s wr, 13 op/s 2023-09-10T07:07:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:05 smithi170 ceph-mon[102832]: pgmap v15122: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 283 KiB/s wr, 13 op/s 2023-09-10T07:07:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:05 smithi082 ceph-mon[98694]: pgmap v15122: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 283 KiB/s wr, 13 op/s 2023-09-10T07:07:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:07 smithi098 ceph-mon[103124]: pgmap v15123: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 317 KiB/s wr, 30 op/s 2023-09-10T07:07:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:07 smithi170 ceph-mon[102832]: pgmap v15123: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 317 KiB/s wr, 30 op/s 2023-09-10T07:07:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:07 smithi082 ceph-mon[98694]: pgmap v15123: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 317 KiB/s wr, 30 op/s 2023-09-10T07:07:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28059.496278 secs 2023-09-10T07:07:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28059.496278 secs 2023-09-10T07:07:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28059.496278 secs 2023-09-10T07:07:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28060.634928 secs 2023-09-10T07:07:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:09 smithi098 ceph-mon[103124]: osdmap e1013: 12 total, 12 up, 12 in 2023-09-10T07:07:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:09 smithi098 ceph-mon[103124]: pgmap v15125: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 182 KiB/s wr, 28 op/s 2023-09-10T07:07:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28060.634928 secs 2023-09-10T07:07:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:09 smithi170 ceph-mon[102832]: osdmap e1013: 12 total, 12 up, 12 in 2023-09-10T07:07:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:09 smithi170 ceph-mon[102832]: pgmap v15125: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 182 KiB/s wr, 28 op/s 2023-09-10T07:07:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28060.634928 secs 2023-09-10T07:07:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:09 smithi082 ceph-mon[98694]: osdmap e1013: 12 total, 12 up, 12 in 2023-09-10T07:07:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:09 smithi082 ceph-mon[98694]: pgmap v15125: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 182 KiB/s wr, 28 op/s 2023-09-10T07:07:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28061.112486 secs 2023-09-10T07:07:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28061.112486 secs 2023-09-10T07:07:10.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28061.112486 secs 2023-09-10T07:07:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:11 smithi098 ceph-mon[103124]: pgmap v15126: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 41 KiB/s wr, 25 op/s 2023-09-10T07:07:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:11 smithi170 ceph-mon[102832]: pgmap v15126: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 41 KiB/s wr, 25 op/s 2023-09-10T07:07:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:11 smithi082 ceph-mon[98694]: pgmap v15126: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 41 KiB/s wr, 25 op/s 2023-09-10T07:07:13.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:13 smithi082 ceph-mon[98694]: pgmap v15127: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 41 KiB/s wr, 25 op/s 2023-09-10T07:07:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28064.496339 secs 2023-09-10T07:07:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:13 smithi098 ceph-mon[103124]: pgmap v15127: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 41 KiB/s wr, 25 op/s 2023-09-10T07:07:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28064.496339 secs 2023-09-10T07:07:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:13 smithi170 ceph-mon[102832]: pgmap v15127: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 41 KiB/s wr, 25 op/s 2023-09-10T07:07:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28064.496339 secs 2023-09-10T07:07:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28065.634984 secs 2023-09-10T07:07:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28066.112549 secs 2023-09-10T07:07:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28065.634984 secs 2023-09-10T07:07:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28066.112549 secs 2023-09-10T07:07:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28065.634984 secs 2023-09-10T07:07:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28066.112549 secs 2023-09-10T07:07:15.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:15 smithi082 ceph-mon[98694]: pgmap v15128: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 41 KiB/s wr, 25 op/s 2023-09-10T07:07:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:15 smithi098 ceph-mon[103124]: pgmap v15128: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 41 KiB/s wr, 25 op/s 2023-09-10T07:07:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:15 smithi170 ceph-mon[102832]: pgmap v15128: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 41 KiB/s wr, 25 op/s 2023-09-10T07:07:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:17 smithi098 ceph-mon[103124]: osdmap e1014: 12 total, 12 up, 12 in 2023-09-10T07:07:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:17 smithi098 ceph-mon[103124]: pgmap v15130: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 251 B/s wr, 12 op/s 2023-09-10T07:07:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:17 smithi170 ceph-mon[102832]: osdmap e1014: 12 total, 12 up, 12 in 2023-09-10T07:07:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:17 smithi170 ceph-mon[102832]: pgmap v15130: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 251 B/s wr, 12 op/s 2023-09-10T07:07:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:17 smithi082 ceph-mon[98694]: osdmap e1014: 12 total, 12 up, 12 in 2023-09-10T07:07:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:17 smithi082 ceph-mon[98694]: pgmap v15130: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 251 B/s wr, 12 op/s 2023-09-10T07:07:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28069.496427 secs 2023-09-10T07:07:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28069.496427 secs 2023-09-10T07:07:18.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28069.496427 secs 2023-09-10T07:07:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28070.635043 secs 2023-09-10T07:07:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:19 smithi098 ceph-mon[103124]: pgmap v15131: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:07:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28070.635043 secs 2023-09-10T07:07:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:19 smithi170 ceph-mon[102832]: pgmap v15131: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:07:19.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28070.635043 secs 2023-09-10T07:07:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:19 smithi082 ceph-mon[98694]: pgmap v15131: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:07:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28071.112613 secs 2023-09-10T07:07:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28071.112613 secs 2023-09-10T07:07:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28071.112613 secs 2023-09-10T07:07:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:21 smithi098 ceph-mon[103124]: pgmap v15132: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T07:07:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:21 smithi170 ceph-mon[102832]: pgmap v15132: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T07:07:21.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:21 smithi082 ceph-mon[98694]: pgmap v15132: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T07:07:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:23 smithi082 ceph-mon[98694]: pgmap v15133: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T07:07:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28074.496489 secs 2023-09-10T07:07:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:23 smithi098 ceph-mon[103124]: pgmap v15133: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T07:07:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28074.496489 secs 2023-09-10T07:07:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:23 smithi170 ceph-mon[102832]: pgmap v15133: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T07:07:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28074.496489 secs 2023-09-10T07:07:24.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28075.635127 secs 2023-09-10T07:07:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28076.112677 secs 2023-09-10T07:07:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28075.635127 secs 2023-09-10T07:07:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28076.112677 secs 2023-09-10T07:07:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28075.635127 secs 2023-09-10T07:07:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28076.112677 secs 2023-09-10T07:07:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:25 smithi082 ceph-mon[98694]: pgmap v15134: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T07:07:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:25 smithi098 ceph-mon[103124]: pgmap v15134: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T07:07:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:25 smithi170 ceph-mon[102832]: pgmap v15134: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T07:07:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:27 smithi098 ceph-mon[103124]: pgmap v15135: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 224 KiB/s wr, 12 op/s 2023-09-10T07:07:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:27 smithi170 ceph-mon[102832]: pgmap v15135: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 224 KiB/s wr, 12 op/s 2023-09-10T07:07:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:27 smithi082 ceph-mon[98694]: pgmap v15135: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 224 KiB/s wr, 12 op/s 2023-09-10T07:07:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28080.635187 secs 2023-09-10T07:07:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:29 smithi082 ceph-mon[98694]: pgmap v15136: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 189 KiB/s wr, 10 op/s 2023-09-10T07:07:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28081.112740 secs 2023-09-10T07:07:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28080.635187 secs 2023-09-10T07:07:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:29 smithi098 ceph-mon[103124]: pgmap v15136: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 189 KiB/s wr, 10 op/s 2023-09-10T07:07:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28081.112740 secs 2023-09-10T07:07:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28080.635187 secs 2023-09-10T07:07:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:29 smithi170 ceph-mon[102832]: pgmap v15136: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 189 KiB/s wr, 10 op/s 2023-09-10T07:07:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28081.112740 secs 2023-09-10T07:07:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:31 smithi082 ceph-mon[98694]: pgmap v15137: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 285 KiB/s wr, 15 op/s 2023-09-10T07:07:31.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:31 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:07:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:31 smithi098 ceph-mon[103124]: pgmap v15137: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 285 KiB/s wr, 15 op/s 2023-09-10T07:07:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:31 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:07:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:31 smithi170 ceph-mon[102832]: pgmap v15137: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 285 KiB/s wr, 15 op/s 2023-09-10T07:07:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:31 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:07:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:32 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:07:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:07:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:07:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:32 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:07:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:07:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:07:33.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:32 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:07:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:07:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:07:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:33 smithi098 ceph-mon[103124]: pgmap v15138: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 10 op/s 2023-09-10T07:07:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:33 smithi170 ceph-mon[102832]: pgmap v15138: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 10 op/s 2023-09-10T07:07:34.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:33 smithi082 ceph-mon[98694]: pgmap v15138: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 10 op/s 2023-09-10T07:07:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28085.635244 secs 2023-09-10T07:07:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:07:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:07:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28086.112803 secs 2023-09-10T07:07:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28085.635244 secs 2023-09-10T07:07:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:07:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:07:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28086.112803 secs 2023-09-10T07:07:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28085.635244 secs 2023-09-10T07:07:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:07:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:07:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28086.112803 secs 2023-09-10T07:07:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:35 smithi098 ceph-mon[103124]: pgmap v15139: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 10 op/s 2023-09-10T07:07:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:35 smithi170 ceph-mon[102832]: pgmap v15139: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 10 op/s 2023-09-10T07:07:36.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:35 smithi082 ceph-mon[98694]: pgmap v15139: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 10 op/s 2023-09-10T07:07:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:37 smithi098 ceph-mon[103124]: pgmap v15140: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 313 KiB/s wr, 16 op/s 2023-09-10T07:07:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:37 smithi170 ceph-mon[102832]: pgmap v15140: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 313 KiB/s wr, 16 op/s 2023-09-10T07:07:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:37 smithi082 ceph-mon[98694]: pgmap v15140: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 313 KiB/s wr, 16 op/s 2023-09-10T07:07:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28090.635306 secs 2023-09-10T07:07:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:39 smithi082 ceph-mon[98694]: pgmap v15141: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 216 KiB/s wr, 10 op/s 2023-09-10T07:07:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28091.112866 secs 2023-09-10T07:07:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28090.635306 secs 2023-09-10T07:07:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:39 smithi098 ceph-mon[103124]: pgmap v15141: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 216 KiB/s wr, 10 op/s 2023-09-10T07:07:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28091.112866 secs 2023-09-10T07:07:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28090.635306 secs 2023-09-10T07:07:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:39 smithi170 ceph-mon[102832]: pgmap v15141: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 216 KiB/s wr, 10 op/s 2023-09-10T07:07:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28091.112866 secs 2023-09-10T07:07:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:41 smithi082 ceph-mon[98694]: pgmap v15142: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 349 KiB/s wr, 17 op/s 2023-09-10T07:07:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:41 smithi098 ceph-mon[103124]: pgmap v15142: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 349 KiB/s wr, 17 op/s 2023-09-10T07:07:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:41 smithi170 ceph-mon[102832]: pgmap v15142: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 349 KiB/s wr, 17 op/s 2023-09-10T07:07:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:43 smithi098 ceph-mon[103124]: pgmap v15143: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 253 KiB/s wr, 11 op/s 2023-09-10T07:07:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:07:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:43 smithi170 ceph-mon[102832]: pgmap v15143: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 253 KiB/s wr, 11 op/s 2023-09-10T07:07:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:07:44.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:43 smithi082 ceph-mon[98694]: pgmap v15143: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 253 KiB/s wr, 11 op/s 2023-09-10T07:07:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:07:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28095.635368 secs 2023-09-10T07:07:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:07:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:07:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28096.112933 secs 2023-09-10T07:07:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:07:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:07:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:07:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:07:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28095.635368 secs 2023-09-10T07:07:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:07:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:07:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28096.112933 secs 2023-09-10T07:07:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:07:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:07:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:07:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:07:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28095.635368 secs 2023-09-10T07:07:45.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:07:45.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:07:45.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28096.112933 secs 2023-09-10T07:07:45.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:07:45.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:07:45.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:07:45.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:07:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:45 smithi098 ceph-mon[103124]: pgmap v15144: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 253 KiB/s wr, 11 op/s 2023-09-10T07:07:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:07:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:07:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:07:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:45 smithi170 ceph-mon[102832]: pgmap v15144: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 253 KiB/s wr, 11 op/s 2023-09-10T07:07:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:07:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:07:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:07:46.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:45 smithi082 ceph-mon[98694]: pgmap v15144: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 253 KiB/s wr, 11 op/s 2023-09-10T07:07:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:07:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:07:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:07:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:07:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:07:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:07:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:07:47.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:07:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:07:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:47 smithi098 ceph-mon[103124]: pgmap v15145: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 520 KiB/s wr, 19 op/s 2023-09-10T07:07:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:47 smithi170 ceph-mon[102832]: pgmap v15145: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 520 KiB/s wr, 19 op/s 2023-09-10T07:07:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:47 smithi082 ceph-mon[98694]: pgmap v15145: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 520 KiB/s wr, 19 op/s 2023-09-10T07:07:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28100.635426 secs 2023-09-10T07:07:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:49 smithi098 ceph-mon[103124]: pgmap v15146: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 400 KiB/s wr, 13 op/s 2023-09-10T07:07:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28101.112995 secs 2023-09-10T07:07:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28100.635426 secs 2023-09-10T07:07:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:49 smithi170 ceph-mon[102832]: pgmap v15146: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 400 KiB/s wr, 13 op/s 2023-09-10T07:07:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28101.112995 secs 2023-09-10T07:07:50.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28100.635426 secs 2023-09-10T07:07:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:49 smithi082 ceph-mon[98694]: pgmap v15146: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 400 KiB/s wr, 13 op/s 2023-09-10T07:07:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28101.112995 secs 2023-09-10T07:07:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:51 smithi098 ceph-mon[103124]: pgmap v15147: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 518 KiB/s wr, 20 op/s 2023-09-10T07:07:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:51 smithi170 ceph-mon[102832]: pgmap v15147: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 518 KiB/s wr, 20 op/s 2023-09-10T07:07:52.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:51 smithi082 ceph-mon[98694]: pgmap v15147: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 518 KiB/s wr, 20 op/s 2023-09-10T07:07:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:53 smithi098 ceph-mon[103124]: pgmap v15148: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 385 KiB/s wr, 13 op/s 2023-09-10T07:07:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28104.496920 secs 2023-09-10T07:07:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:53 smithi170 ceph-mon[102832]: pgmap v15148: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 385 KiB/s wr, 13 op/s 2023-09-10T07:07:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28104.496920 secs 2023-09-10T07:07:54.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:53 smithi082 ceph-mon[98694]: pgmap v15148: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 385 KiB/s wr, 13 op/s 2023-09-10T07:07:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28104.496920 secs 2023-09-10T07:07:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28105.635489 secs 2023-09-10T07:07:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:07:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:07:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28106.113057 secs 2023-09-10T07:07:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28105.635489 secs 2023-09-10T07:07:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:07:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:07:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28106.113057 secs 2023-09-10T07:07:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28105.635489 secs 2023-09-10T07:07:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:07:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:07:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28106.113057 secs 2023-09-10T07:07:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:55 smithi098 ceph-mon[103124]: pgmap v15149: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 385 KiB/s wr, 13 op/s 2023-09-10T07:07:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:55 smithi170 ceph-mon[102832]: pgmap v15149: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 385 KiB/s wr, 13 op/s 2023-09-10T07:07:56.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:55 smithi082 ceph-mon[98694]: pgmap v15149: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 385 KiB/s wr, 13 op/s 2023-09-10T07:07:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:07:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:07:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:07:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:07:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:07:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:07:57.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:07:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:07:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:07:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:57 smithi098 ceph-mon[103124]: pgmap v15150: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 583 KiB/s wr, 20 op/s 2023-09-10T07:07:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:57 smithi170 ceph-mon[102832]: pgmap v15150: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 583 KiB/s wr, 20 op/s 2023-09-10T07:07:58.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:57 smithi082 ceph-mon[98694]: pgmap v15150: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 583 KiB/s wr, 20 op/s 2023-09-10T07:07:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28109.496979 secs 2023-09-10T07:07:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28109.496979 secs 2023-09-10T07:07:59.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28109.496979 secs 2023-09-10T07:08:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28110.635550 secs 2023-09-10T07:08:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:59 smithi098 ceph-mon[103124]: pgmap v15151: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 316 KiB/s wr, 13 op/s 2023-09-10T07:08:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:07:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28111.113122 secs 2023-09-10T07:08:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28110.635550 secs 2023-09-10T07:08:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:59 smithi170 ceph-mon[102832]: pgmap v15151: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 316 KiB/s wr, 13 op/s 2023-09-10T07:08:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:07:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28111.113122 secs 2023-09-10T07:08:00.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28110.635550 secs 2023-09-10T07:08:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:59 smithi082 ceph-mon[98694]: pgmap v15151: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 316 KiB/s wr, 13 op/s 2023-09-10T07:08:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:07:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28111.113122 secs 2023-09-10T07:08:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:01 smithi098 ceph-mon[103124]: pgmap v15152: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 423 KiB/s wr, 18 op/s 2023-09-10T07:08:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:01 smithi170 ceph-mon[102832]: pgmap v15152: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 423 KiB/s wr, 18 op/s 2023-09-10T07:08:02.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:01 smithi082 ceph-mon[98694]: pgmap v15152: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 423 KiB/s wr, 18 op/s 2023-09-10T07:08:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:03 smithi098 ceph-mon[103124]: pgmap v15153: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 305 KiB/s wr, 12 op/s 2023-09-10T07:08:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28114.497089 secs 2023-09-10T07:08:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:03 smithi170 ceph-mon[102832]: pgmap v15153: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 305 KiB/s wr, 12 op/s 2023-09-10T07:08:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28114.497089 secs 2023-09-10T07:08:04.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:03 smithi082 ceph-mon[98694]: pgmap v15153: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 305 KiB/s wr, 12 op/s 2023-09-10T07:08:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28114.497089 secs 2023-09-10T07:08:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28115.635634 secs 2023-09-10T07:08:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28116.113185 secs 2023-09-10T07:08:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28115.635634 secs 2023-09-10T07:08:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28116.113185 secs 2023-09-10T07:08:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28115.635634 secs 2023-09-10T07:08:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28116.113185 secs 2023-09-10T07:08:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:05 smithi170 ceph-mon[102832]: pgmap v15154: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 305 KiB/s wr, 12 op/s 2023-09-10T07:08:06.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:05 smithi082 ceph-mon[98694]: pgmap v15154: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 305 KiB/s wr, 12 op/s 2023-09-10T07:08:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:05 smithi098 ceph-mon[103124]: pgmap v15154: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 305 KiB/s wr, 12 op/s 2023-09-10T07:08:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:07 smithi098 ceph-mon[103124]: pgmap v15155: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 350 KiB/s wr, 31 op/s 2023-09-10T07:08:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:07 smithi170 ceph-mon[102832]: pgmap v15155: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 350 KiB/s wr, 31 op/s 2023-09-10T07:08:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:07 smithi082 ceph-mon[98694]: pgmap v15155: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 350 KiB/s wr, 31 op/s 2023-09-10T07:08:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28119.497177 secs 2023-09-10T07:08:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28119.497177 secs 2023-09-10T07:08:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28119.497177 secs 2023-09-10T07:08:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28120.635690 secs 2023-09-10T07:08:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:09 smithi170 ceph-mon[102832]: osdmap e1015: 12 total, 12 up, 12 in 2023-09-10T07:08:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:09 smithi170 ceph-mon[102832]: pgmap v15157: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 182 KiB/s wr, 30 op/s 2023-09-10T07:08:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28120.635690 secs 2023-09-10T07:08:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:09 smithi082 ceph-mon[98694]: osdmap e1015: 12 total, 12 up, 12 in 2023-09-10T07:08:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:09 smithi082 ceph-mon[98694]: pgmap v15157: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 182 KiB/s wr, 30 op/s 2023-09-10T07:08:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28120.635690 secs 2023-09-10T07:08:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:09 smithi098 ceph-mon[103124]: osdmap e1015: 12 total, 12 up, 12 in 2023-09-10T07:08:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:09 smithi098 ceph-mon[103124]: pgmap v15157: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 182 KiB/s wr, 30 op/s 2023-09-10T07:08:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28121.113246 secs 2023-09-10T07:08:10.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28121.113246 secs 2023-09-10T07:08:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28121.113246 secs 2023-09-10T07:08:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:11 smithi170 ceph-mon[102832]: pgmap v15158: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-09-10T07:08:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:11 smithi082 ceph-mon[98694]: pgmap v15158: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-09-10T07:08:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:11 smithi098 ceph-mon[103124]: pgmap v15158: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-09-10T07:08:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:13 smithi082 ceph-mon[98694]: pgmap v15159: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-09-10T07:08:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28124.497240 secs 2023-09-10T07:08:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:13 smithi098 ceph-mon[103124]: pgmap v15159: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-09-10T07:08:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28124.497240 secs 2023-09-10T07:08:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:13 smithi170 ceph-mon[102832]: pgmap v15159: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-09-10T07:08:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28124.497240 secs 2023-09-10T07:08:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28125.635747 secs 2023-09-10T07:08:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28126.113315 secs 2023-09-10T07:08:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28125.635747 secs 2023-09-10T07:08:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28126.113315 secs 2023-09-10T07:08:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28125.635747 secs 2023-09-10T07:08:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28126.113315 secs 2023-09-10T07:08:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:15 smithi098 ceph-mon[103124]: pgmap v15160: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-09-10T07:08:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:15 smithi170 ceph-mon[102832]: pgmap v15160: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-09-10T07:08:16.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:15 smithi082 ceph-mon[98694]: pgmap v15160: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-09-10T07:08:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:17 smithi098 ceph-mon[103124]: osdmap e1016: 12 total, 12 up, 12 in 2023-09-10T07:08:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:17 smithi098 ceph-mon[103124]: pgmap v15162: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 251 B/s wr, 13 op/s 2023-09-10T07:08:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:17 smithi170 ceph-mon[102832]: osdmap e1016: 12 total, 12 up, 12 in 2023-09-10T07:08:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:17 smithi170 ceph-mon[102832]: pgmap v15162: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 251 B/s wr, 13 op/s 2023-09-10T07:08:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:17 smithi082 ceph-mon[98694]: osdmap e1016: 12 total, 12 up, 12 in 2023-09-10T07:08:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:17 smithi082 ceph-mon[98694]: pgmap v15162: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 251 B/s wr, 13 op/s 2023-09-10T07:08:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28129.497299 secs 2023-09-10T07:08:18.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28129.497299 secs 2023-09-10T07:08:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28129.497299 secs 2023-09-10T07:08:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28130.635807 secs 2023-09-10T07:08:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:19 smithi170 ceph-mon[102832]: pgmap v15163: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T07:08:19.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28130.635807 secs 2023-09-10T07:08:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:19 smithi082 ceph-mon[98694]: pgmap v15163: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T07:08:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28130.635807 secs 2023-09-10T07:08:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:19 smithi098 ceph-mon[103124]: pgmap v15163: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T07:08:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28131.113375 secs 2023-09-10T07:08:20.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28131.113375 secs 2023-09-10T07:08:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28131.113375 secs 2023-09-10T07:08:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:21 smithi170 ceph-mon[102832]: pgmap v15164: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T07:08:21.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:21 smithi082 ceph-mon[98694]: pgmap v15164: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T07:08:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:21 smithi098 ceph-mon[103124]: pgmap v15164: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T07:08:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:23 smithi098 ceph-mon[103124]: pgmap v15165: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T07:08:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28134.497358 secs 2023-09-10T07:08:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:23 smithi170 ceph-mon[102832]: pgmap v15165: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T07:08:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28134.497358 secs 2023-09-10T07:08:24.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:23 smithi082 ceph-mon[98694]: pgmap v15165: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T07:08:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28134.497358 secs 2023-09-10T07:08:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28135.635865 secs 2023-09-10T07:08:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28136.113435 secs 2023-09-10T07:08:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28135.635865 secs 2023-09-10T07:08:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28136.113435 secs 2023-09-10T07:08:25.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28135.635865 secs 2023-09-10T07:08:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28136.113435 secs 2023-09-10T07:08:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:25 smithi098 ceph-mon[103124]: pgmap v15166: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T07:08:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:25 smithi170 ceph-mon[102832]: pgmap v15166: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T07:08:26.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:25 smithi082 ceph-mon[98694]: pgmap v15166: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T07:08:27.933 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:27 smithi170 ceph-mon[102832]: pgmap v15167: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 806 B/s wr, 11 op/s 2023-09-10T07:08:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:27 smithi082 ceph-mon[98694]: pgmap v15167: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 806 B/s wr, 11 op/s 2023-09-10T07:08:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:27 smithi098 ceph-mon[103124]: pgmap v15167: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 806 B/s wr, 11 op/s 2023-09-10T07:08:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28139.497418 secs 2023-09-10T07:08:28.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28139.497418 secs 2023-09-10T07:08:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28139.497418 secs 2023-09-10T07:08:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28140.635922 secs 2023-09-10T07:08:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:29 smithi170 ceph-mon[102832]: pgmap v15168: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-10T07:08:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28140.635922 secs 2023-09-10T07:08:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:29 smithi082 ceph-mon[98694]: pgmap v15168: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-10T07:08:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28140.635922 secs 2023-09-10T07:08:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:29 smithi098 ceph-mon[103124]: pgmap v15168: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-10T07:08:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28141.113484 secs 2023-09-10T07:08:30.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28141.113484 secs 2023-09-10T07:08:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28141.113484 secs 2023-09-10T07:08:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:31 smithi170 ceph-mon[102832]: pgmap v15169: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 98 KiB/s wr, 14 op/s 2023-09-10T07:08:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:31 smithi082 ceph-mon[98694]: pgmap v15169: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 98 KiB/s wr, 14 op/s 2023-09-10T07:08:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:31 smithi098 ceph-mon[103124]: pgmap v15169: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 98 KiB/s wr, 14 op/s 2023-09-10T07:08:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:33 smithi098 ceph-mon[103124]: pgmap v15170: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:08:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:33 smithi170 ceph-mon[102832]: pgmap v15170: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:08:34.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:33 smithi082 ceph-mon[98694]: pgmap v15170: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:08:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28145.636007 secs 2023-09-10T07:08:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28146.113549 secs 2023-09-10T07:08:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28145.636007 secs 2023-09-10T07:08:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28146.113549 secs 2023-09-10T07:08:35.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28145.636007 secs 2023-09-10T07:08:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28146.113549 secs 2023-09-10T07:08:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:35 smithi098 ceph-mon[103124]: pgmap v15171: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:08:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:08:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:35 smithi170 ceph-mon[102832]: pgmap v15171: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:08:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:08:36.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:35 smithi082 ceph-mon[98694]: pgmap v15171: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:08:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:08:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:08:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:08:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:08:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:08:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:08:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:08:37.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:08:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:08:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:08:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:37 smithi098 ceph-mon[103124]: pgmap v15172: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 195 KiB/s wr, 15 op/s 2023-09-10T07:08:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:37 smithi170 ceph-mon[102832]: pgmap v15172: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 195 KiB/s wr, 15 op/s 2023-09-10T07:08:38.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:37 smithi082 ceph-mon[98694]: pgmap v15172: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 195 KiB/s wr, 15 op/s 2023-09-10T07:08:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:08:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:08:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:08:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:08:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:08:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:08:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28150.636090 secs 2023-09-10T07:08:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:39 smithi098 ceph-mon[103124]: pgmap v15173: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 10 op/s 2023-09-10T07:08:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28151.113610 secs 2023-09-10T07:08:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28150.636090 secs 2023-09-10T07:08:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:39 smithi170 ceph-mon[102832]: pgmap v15173: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 10 op/s 2023-09-10T07:08:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28151.113610 secs 2023-09-10T07:08:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28150.636090 secs 2023-09-10T07:08:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:39 smithi082 ceph-mon[98694]: pgmap v15173: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 10 op/s 2023-09-10T07:08:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28151.113610 secs 2023-09-10T07:08:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:41 smithi098 ceph-mon[103124]: pgmap v15174: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 406 KiB/s wr, 16 op/s 2023-09-10T07:08:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:41 smithi170 ceph-mon[102832]: pgmap v15174: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 406 KiB/s wr, 16 op/s 2023-09-10T07:08:42.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:41 smithi082 ceph-mon[98694]: pgmap v15174: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 406 KiB/s wr, 16 op/s 2023-09-10T07:08:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:43 smithi098 ceph-mon[103124]: pgmap v15175: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 308 KiB/s wr, 11 op/s 2023-09-10T07:08:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:43 smithi170 ceph-mon[102832]: pgmap v15175: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 308 KiB/s wr, 11 op/s 2023-09-10T07:08:44.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:43 smithi082 ceph-mon[98694]: pgmap v15175: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 308 KiB/s wr, 11 op/s 2023-09-10T07:08:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28155.636152 secs 2023-09-10T07:08:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:08:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:08:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28156.113670 secs 2023-09-10T07:08:45.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28155.636152 secs 2023-09-10T07:08:45.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:08:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:08:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28156.113670 secs 2023-09-10T07:08:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28155.636152 secs 2023-09-10T07:08:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:08:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:08:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28156.113670 secs 2023-09-10T07:08:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:45 smithi170 ceph-mon[102832]: pgmap v15176: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 308 KiB/s wr, 11 op/s 2023-09-10T07:08:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:08:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:08:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:08:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:08:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:08:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:08:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:45 smithi082 ceph-mon[98694]: pgmap v15176: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 308 KiB/s wr, 11 op/s 2023-09-10T07:08:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:08:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:08:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:08:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:08:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:08:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:08:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:45 smithi098 ceph-mon[103124]: pgmap v15176: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 308 KiB/s wr, 11 op/s 2023-09-10T07:08:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:08:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:08:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:08:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:08:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:08:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:08:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:47 smithi170 ceph-mon[102832]: pgmap v15177: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 637 KiB/s wr, 18 op/s 2023-09-10T07:08:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:47 smithi082 ceph-mon[98694]: pgmap v15177: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 637 KiB/s wr, 18 op/s 2023-09-10T07:08:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:47 smithi098 ceph-mon[103124]: pgmap v15177: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 637 KiB/s wr, 18 op/s 2023-09-10T07:08:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28160.636209 secs 2023-09-10T07:08:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:49 smithi098 ceph-mon[103124]: pgmap v15178: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 540 KiB/s wr, 12 op/s 2023-09-10T07:08:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28161.113733 secs 2023-09-10T07:08:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28160.636209 secs 2023-09-10T07:08:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:49 smithi170 ceph-mon[102832]: pgmap v15178: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 540 KiB/s wr, 12 op/s 2023-09-10T07:08:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28161.113733 secs 2023-09-10T07:08:50.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28160.636209 secs 2023-09-10T07:08:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:49 smithi082 ceph-mon[98694]: pgmap v15178: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 540 KiB/s wr, 12 op/s 2023-09-10T07:08:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28161.113733 secs 2023-09-10T07:08:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:08:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:08:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:08:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:08:51.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:08:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:08:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:51 smithi098 ceph-mon[103124]: pgmap v15179: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 796 KiB/s wr, 20 op/s 2023-09-10T07:08:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:51 smithi170 ceph-mon[102832]: pgmap v15179: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 796 KiB/s wr, 20 op/s 2023-09-10T07:08:52.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:51 smithi082 ceph-mon[98694]: pgmap v15179: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 796 KiB/s wr, 20 op/s 2023-09-10T07:08:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:08:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:08:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:08:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:08:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:08:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:08:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:53 smithi098 ceph-mon[103124]: pgmap v15180: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 585 KiB/s wr, 14 op/s 2023-09-10T07:08:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:53 smithi170 ceph-mon[102832]: pgmap v15180: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 585 KiB/s wr, 14 op/s 2023-09-10T07:08:54.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:53 smithi082 ceph-mon[98694]: pgmap v15180: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 585 KiB/s wr, 14 op/s 2023-09-10T07:08:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28165.636282 secs 2023-09-10T07:08:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28166.113792 secs 2023-09-10T07:08:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28165.636282 secs 2023-09-10T07:08:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28166.113792 secs 2023-09-10T07:08:55.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28165.636282 secs 2023-09-10T07:08:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28166.113792 secs 2023-09-10T07:08:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:55 smithi098 ceph-mon[103124]: pgmap v15181: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 585 KiB/s wr, 14 op/s 2023-09-10T07:08:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:55 smithi170 ceph-mon[102832]: pgmap v15181: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 585 KiB/s wr, 14 op/s 2023-09-10T07:08:56.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:55 smithi082 ceph-mon[98694]: pgmap v15181: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 585 KiB/s wr, 14 op/s 2023-09-10T07:08:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:57 smithi170 ceph-mon[102832]: pgmap v15182: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 661 KiB/s wr, 20 op/s 2023-09-10T07:08:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:57 smithi082 ceph-mon[98694]: pgmap v15182: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 661 KiB/s wr, 20 op/s 2023-09-10T07:08:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:57 smithi098 ceph-mon[103124]: pgmap v15182: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 661 KiB/s wr, 20 op/s 2023-09-10T07:08:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:08:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:08:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28169.497795 secs 2023-09-10T07:08:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:08:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:08:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28169.497795 secs 2023-09-10T07:08:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:08:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:08:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28169.497795 secs 2023-09-10T07:08:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28170.636340 secs 2023-09-10T07:08:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:08:59 smithi170 ceph-mon[102832]: pgmap v15183: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 332 KiB/s wr, 13 op/s 2023-09-10T07:08:59.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28170.636340 secs 2023-09-10T07:08:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:08:59 smithi082 ceph-mon[98694]: pgmap v15183: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 332 KiB/s wr, 13 op/s 2023-09-10T07:09:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28170.636340 secs 2023-09-10T07:09:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:08:59 smithi098 ceph-mon[103124]: pgmap v15183: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 332 KiB/s wr, 13 op/s 2023-09-10T07:09:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28171.113857 secs 2023-09-10T07:09:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:09:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:09:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:09:00.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28171.113857 secs 2023-09-10T07:09:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:09:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:09:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:09:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28171.113857 secs 2023-09-10T07:09:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:09:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:09:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:09:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:01 smithi170 ceph-mon[102832]: pgmap v15184: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 558 KiB/s wr, 19 op/s 2023-09-10T07:09:01.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:01 smithi082 ceph-mon[98694]: pgmap v15184: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 558 KiB/s wr, 19 op/s 2023-09-10T07:09:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:01 smithi098 ceph-mon[103124]: pgmap v15184: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 558 KiB/s wr, 19 op/s 2023-09-10T07:09:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:03 smithi098 ceph-mon[103124]: pgmap v15185: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 303 KiB/s wr, 12 op/s 2023-09-10T07:09:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28174.497856 secs 2023-09-10T07:09:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:03 smithi170 ceph-mon[102832]: pgmap v15185: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 303 KiB/s wr, 12 op/s 2023-09-10T07:09:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28174.497856 secs 2023-09-10T07:09:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:03 smithi082 ceph-mon[98694]: pgmap v15185: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 303 KiB/s wr, 12 op/s 2023-09-10T07:09:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28174.497856 secs 2023-09-10T07:09:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28175.636398 secs 2023-09-10T07:09:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28176.113919 secs 2023-09-10T07:09:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28175.636398 secs 2023-09-10T07:09:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28176.113919 secs 2023-09-10T07:09:05.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28175.636398 secs 2023-09-10T07:09:05.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28176.113919 secs 2023-09-10T07:09:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:05 smithi098 ceph-mon[103124]: pgmap v15186: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 303 KiB/s wr, 12 op/s 2023-09-10T07:09:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:05 smithi170 ceph-mon[102832]: pgmap v15186: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 303 KiB/s wr, 12 op/s 2023-09-10T07:09:06.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:05 smithi082 ceph-mon[98694]: pgmap v15186: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 303 KiB/s wr, 12 op/s 2023-09-10T07:09:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:07 smithi170 ceph-mon[102832]: pgmap v15187: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 424 KiB/s wr, 31 op/s 2023-09-10T07:09:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:07 smithi082 ceph-mon[98694]: pgmap v15187: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 424 KiB/s wr, 31 op/s 2023-09-10T07:09:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:07 smithi098 ceph-mon[103124]: pgmap v15187: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 424 KiB/s wr, 31 op/s 2023-09-10T07:09:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28179.497913 secs 2023-09-10T07:09:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28179.497913 secs 2023-09-10T07:09:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28179.497913 secs 2023-09-10T07:09:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28180.636493 secs 2023-09-10T07:09:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:09 smithi170 ceph-mon[102832]: osdmap e1017: 12 total, 12 up, 12 in 2023-09-10T07:09:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:09 smithi170 ceph-mon[102832]: pgmap v15189: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 418 KiB/s wr, 30 op/s 2023-09-10T07:09:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28180.636493 secs 2023-09-10T07:09:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:09 smithi082 ceph-mon[98694]: osdmap e1017: 12 total, 12 up, 12 in 2023-09-10T07:09:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:09 smithi082 ceph-mon[98694]: pgmap v15189: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 418 KiB/s wr, 30 op/s 2023-09-10T07:09:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28180.636493 secs 2023-09-10T07:09:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:09 smithi098 ceph-mon[103124]: osdmap e1017: 12 total, 12 up, 12 in 2023-09-10T07:09:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:09 smithi098 ceph-mon[103124]: pgmap v15189: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 418 KiB/s wr, 30 op/s 2023-09-10T07:09:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28181.113979 secs 2023-09-10T07:09:10.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28181.113979 secs 2023-09-10T07:09:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28181.113979 secs 2023-09-10T07:09:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:11 smithi170 ceph-mon[102832]: pgmap v15190: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 146 KiB/s wr, 27 op/s 2023-09-10T07:09:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:11 smithi082 ceph-mon[98694]: pgmap v15190: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 146 KiB/s wr, 27 op/s 2023-09-10T07:09:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:11 smithi098 ceph-mon[103124]: pgmap v15190: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 146 KiB/s wr, 27 op/s 2023-09-10T07:09:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:13 smithi098 ceph-mon[103124]: pgmap v15191: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 146 KiB/s wr, 27 op/s 2023-09-10T07:09:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28184.497986 secs 2023-09-10T07:09:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:13 smithi170 ceph-mon[102832]: pgmap v15191: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 146 KiB/s wr, 27 op/s 2023-09-10T07:09:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28184.497986 secs 2023-09-10T07:09:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:13 smithi082 ceph-mon[98694]: pgmap v15191: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 146 KiB/s wr, 27 op/s 2023-09-10T07:09:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28184.497986 secs 2023-09-10T07:09:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28185.636554 secs 2023-09-10T07:09:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28186.114044 secs 2023-09-10T07:09:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28185.636554 secs 2023-09-10T07:09:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28186.114044 secs 2023-09-10T07:09:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28185.636554 secs 2023-09-10T07:09:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28186.114044 secs 2023-09-10T07:09:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:15 smithi098 ceph-mon[103124]: pgmap v15192: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 146 KiB/s wr, 27 op/s 2023-09-10T07:09:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:15 smithi170 ceph-mon[102832]: pgmap v15192: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 146 KiB/s wr, 27 op/s 2023-09-10T07:09:16.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:15 smithi082 ceph-mon[98694]: pgmap v15192: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 146 KiB/s wr, 27 op/s 2023-09-10T07:09:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:17 smithi170 ceph-mon[102832]: osdmap e1018: 12 total, 12 up, 12 in 2023-09-10T07:09:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:17 smithi170 ceph-mon[102832]: pgmap v15194: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 125 B/s wr, 12 op/s 2023-09-10T07:09:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:17 smithi082 ceph-mon[98694]: osdmap e1018: 12 total, 12 up, 12 in 2023-09-10T07:09:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:17 smithi082 ceph-mon[98694]: pgmap v15194: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 125 B/s wr, 12 op/s 2023-09-10T07:09:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:17 smithi098 ceph-mon[103124]: osdmap e1018: 12 total, 12 up, 12 in 2023-09-10T07:09:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:17 smithi098 ceph-mon[103124]: pgmap v15194: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 125 B/s wr, 12 op/s 2023-09-10T07:09:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28189.498081 secs 2023-09-10T07:09:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28189.498081 secs 2023-09-10T07:09:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28189.498081 secs 2023-09-10T07:09:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28190.636614 secs 2023-09-10T07:09:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:19 smithi170 ceph-mon[102832]: pgmap v15195: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:09:19.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28190.636614 secs 2023-09-10T07:09:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:19 smithi082 ceph-mon[98694]: pgmap v15195: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:09:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28190.636614 secs 2023-09-10T07:09:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:19 smithi098 ceph-mon[103124]: pgmap v15195: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:09:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28191.114109 secs 2023-09-10T07:09:20.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28191.114109 secs 2023-09-10T07:09:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28191.114109 secs 2023-09-10T07:09:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:21 smithi170 ceph-mon[102832]: pgmap v15196: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:09:21.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:21 smithi082 ceph-mon[98694]: pgmap v15196: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:09:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:21 smithi098 ceph-mon[103124]: pgmap v15196: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:09:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:23 smithi098 ceph-mon[103124]: pgmap v15197: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:09:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28194.498177 secs 2023-09-10T07:09:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:23 smithi170 ceph-mon[102832]: pgmap v15197: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:09:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28194.498177 secs 2023-09-10T07:09:24.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:23 smithi082 ceph-mon[98694]: pgmap v15197: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:09:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28194.498177 secs 2023-09-10T07:09:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28195.636671 secs 2023-09-10T07:09:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28196.114173 secs 2023-09-10T07:09:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28195.636671 secs 2023-09-10T07:09:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28196.114173 secs 2023-09-10T07:09:25.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28195.636671 secs 2023-09-10T07:09:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28196.114173 secs 2023-09-10T07:09:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:25 smithi098 ceph-mon[103124]: pgmap v15198: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:09:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:25 smithi170 ceph-mon[102832]: pgmap v15198: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:09:26.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:25 smithi082 ceph-mon[98694]: pgmap v15198: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:09:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:27 smithi170 ceph-mon[102832]: pgmap v15199: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2023-09-10T07:09:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:27 smithi082 ceph-mon[98694]: pgmap v15199: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2023-09-10T07:09:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:27 smithi098 ceph-mon[103124]: pgmap v15199: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2023-09-10T07:09:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28199.498236 secs 2023-09-10T07:09:28.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28199.498236 secs 2023-09-10T07:09:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28199.498236 secs 2023-09-10T07:09:29.932 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28200.636736 secs 2023-09-10T07:09:29.932 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:29 smithi170 ceph-mon[102832]: pgmap v15200: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-10T07:09:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28200.636736 secs 2023-09-10T07:09:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:29 smithi082 ceph-mon[98694]: pgmap v15200: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-10T07:09:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28200.636736 secs 2023-09-10T07:09:30.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:29 smithi098 ceph-mon[103124]: pgmap v15200: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-10T07:09:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28201.114239 secs 2023-09-10T07:09:30.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28201.114239 secs 2023-09-10T07:09:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28201.114239 secs 2023-09-10T07:09:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:31 smithi170 ceph-mon[102832]: pgmap v15201: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 13 op/s 2023-09-10T07:09:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:31 smithi082 ceph-mon[98694]: pgmap v15201: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 13 op/s 2023-09-10T07:09:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:31 smithi098 ceph-mon[103124]: pgmap v15201: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 13 op/s 2023-09-10T07:09:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:33 smithi098 ceph-mon[103124]: pgmap v15202: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2023-09-10T07:09:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:33 smithi170 ceph-mon[102832]: pgmap v15202: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2023-09-10T07:09:34.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:33 smithi082 ceph-mon[98694]: pgmap v15202: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2023-09-10T07:09:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28205.636794 secs 2023-09-10T07:09:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28206.114302 secs 2023-09-10T07:09:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28205.636794 secs 2023-09-10T07:09:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28206.114302 secs 2023-09-10T07:09:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28205.636794 secs 2023-09-10T07:09:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28206.114302 secs 2023-09-10T07:09:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:35 smithi098 ceph-mon[103124]: pgmap v15203: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2023-09-10T07:09:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:09:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:35 smithi170 ceph-mon[102832]: pgmap v15203: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2023-09-10T07:09:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:09:36.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:35 smithi082 ceph-mon[98694]: pgmap v15203: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2023-09-10T07:09:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:09:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:09:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:09:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:09:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:09:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:09:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:09:37.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:09:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:09:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:09:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:37 smithi098 ceph-mon[103124]: pgmap v15204: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 222 KiB/s wr, 15 op/s 2023-09-10T07:09:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:09:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:09:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:37 smithi170 ceph-mon[102832]: pgmap v15204: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 222 KiB/s wr, 15 op/s 2023-09-10T07:09:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:09:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:09:38.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:37 smithi082 ceph-mon[98694]: pgmap v15204: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 222 KiB/s wr, 15 op/s 2023-09-10T07:09:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:09:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:09:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28210.636855 secs 2023-09-10T07:09:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:39 smithi098 ceph-mon[103124]: pgmap v15205: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 222 KiB/s wr, 11 op/s 2023-09-10T07:09:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28211.114364 secs 2023-09-10T07:09:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28210.636855 secs 2023-09-10T07:09:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:39 smithi170 ceph-mon[102832]: pgmap v15205: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 222 KiB/s wr, 11 op/s 2023-09-10T07:09:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28211.114364 secs 2023-09-10T07:09:40.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28210.636855 secs 2023-09-10T07:09:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:39 smithi082 ceph-mon[98694]: pgmap v15205: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 222 KiB/s wr, 11 op/s 2023-09-10T07:09:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28211.114364 secs 2023-09-10T07:09:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:41 smithi098 ceph-mon[103124]: pgmap v15206: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 432 KiB/s wr, 17 op/s 2023-09-10T07:09:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:41 smithi170 ceph-mon[102832]: pgmap v15206: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 432 KiB/s wr, 17 op/s 2023-09-10T07:09:42.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:41 smithi082 ceph-mon[98694]: pgmap v15206: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 432 KiB/s wr, 17 op/s 2023-09-10T07:09:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:43 smithi098 ceph-mon[103124]: pgmap v15207: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 431 KiB/s wr, 12 op/s 2023-09-10T07:09:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:43 smithi170 ceph-mon[102832]: pgmap v15207: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 431 KiB/s wr, 12 op/s 2023-09-10T07:09:44.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:43 smithi082 ceph-mon[98694]: pgmap v15207: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 431 KiB/s wr, 12 op/s 2023-09-10T07:09:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28215.636916 secs 2023-09-10T07:09:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:09:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:09:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28216.114425 secs 2023-09-10T07:09:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28215.636916 secs 2023-09-10T07:09:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:09:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:09:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28216.114425 secs 2023-09-10T07:09:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28215.636916 secs 2023-09-10T07:09:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:09:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:09:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28216.114425 secs 2023-09-10T07:09:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:45 smithi170 ceph-mon[102832]: pgmap v15208: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 431 KiB/s wr, 12 op/s 2023-09-10T07:09:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:09:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:09:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:09:46.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:45 smithi082 ceph-mon[98694]: pgmap v15208: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 431 KiB/s wr, 12 op/s 2023-09-10T07:09:46.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:09:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:09:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:09:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:45 smithi098 ceph-mon[103124]: pgmap v15208: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 431 KiB/s wr, 12 op/s 2023-09-10T07:09:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:09:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:09:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:09:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:09:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:09:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:09:47.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:09:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:09:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:09:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:09:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:09:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:09:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:47 smithi170 ceph-mon[102832]: pgmap v15209: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 719 KiB/s wr, 20 op/s 2023-09-10T07:09:48.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:47 smithi082 ceph-mon[98694]: pgmap v15209: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 719 KiB/s wr, 20 op/s 2023-09-10T07:09:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:47 smithi098 ceph-mon[103124]: pgmap v15209: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 719 KiB/s wr, 20 op/s 2023-09-10T07:09:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28220.636974 secs 2023-09-10T07:09:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:49 smithi170 ceph-mon[102832]: pgmap v15210: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 498 KiB/s wr, 14 op/s 2023-09-10T07:09:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28221.114482 secs 2023-09-10T07:09:50.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28220.636974 secs 2023-09-10T07:09:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:49 smithi082 ceph-mon[98694]: pgmap v15210: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 498 KiB/s wr, 14 op/s 2023-09-10T07:09:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28221.114482 secs 2023-09-10T07:09:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28220.636974 secs 2023-09-10T07:09:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:49 smithi098 ceph-mon[103124]: pgmap v15210: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 498 KiB/s wr, 14 op/s 2023-09-10T07:09:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28221.114482 secs 2023-09-10T07:09:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:51 smithi170 ceph-mon[102832]: pgmap v15211: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 663 KiB/s wr, 21 op/s 2023-09-10T07:09:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:51 smithi082 ceph-mon[98694]: pgmap v15211: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 663 KiB/s wr, 21 op/s 2023-09-10T07:09:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:51 smithi098 ceph-mon[103124]: pgmap v15211: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 663 KiB/s wr, 21 op/s 2023-09-10T07:09:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:53 smithi170 ceph-mon[102832]: pgmap v15212: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 454 KiB/s wr, 14 op/s 2023-09-10T07:09:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:09:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:09:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28224.498606 secs 2023-09-10T07:09:54.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:53 smithi082 ceph-mon[98694]: pgmap v15212: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 454 KiB/s wr, 14 op/s 2023-09-10T07:09:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:09:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:09:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28224.498606 secs 2023-09-10T07:09:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:53 smithi098 ceph-mon[103124]: pgmap v15212: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 454 KiB/s wr, 14 op/s 2023-09-10T07:09:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:09:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:09:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28224.498606 secs 2023-09-10T07:09:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28225.637029 secs 2023-09-10T07:09:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28226.114545 secs 2023-09-10T07:09:55.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28225.637029 secs 2023-09-10T07:09:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28226.114545 secs 2023-09-10T07:09:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28225.637029 secs 2023-09-10T07:09:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28226.114545 secs 2023-09-10T07:09:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:55 smithi170 ceph-mon[102832]: pgmap v15213: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 454 KiB/s wr, 14 op/s 2023-09-10T07:09:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:55 smithi082 ceph-mon[98694]: pgmap v15213: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 454 KiB/s wr, 14 op/s 2023-09-10T07:09:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:55 smithi098 ceph-mon[103124]: pgmap v15213: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 454 KiB/s wr, 14 op/s 2023-09-10T07:09:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:09:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:09:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:09:57.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:09:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:09:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:09:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:09:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:09:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:09:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:57 smithi170 ceph-mon[102832]: pgmap v15214: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 592 KiB/s wr, 20 op/s 2023-09-10T07:09:58.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:57 smithi082 ceph-mon[98694]: pgmap v15214: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 592 KiB/s wr, 20 op/s 2023-09-10T07:09:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:57 smithi098 ceph-mon[103124]: pgmap v15214: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 592 KiB/s wr, 20 op/s 2023-09-10T07:09:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28229.498666 secs 2023-09-10T07:09:59.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28229.498666 secs 2023-09-10T07:09:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28229.498666 secs 2023-09-10T07:10:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28230.637090 secs 2023-09-10T07:10:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:59 smithi170 ceph-mon[102832]: pgmap v15215: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 304 KiB/s wr, 12 op/s 2023-09-10T07:10:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:09:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28231.114608 secs 2023-09-10T07:10:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28230.637090 secs 2023-09-10T07:10:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:59 smithi082 ceph-mon[98694]: pgmap v15215: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 304 KiB/s wr, 12 op/s 2023-09-10T07:10:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:09:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28231.114608 secs 2023-09-10T07:10:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28230.637090 secs 2023-09-10T07:10:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:59 smithi098 ceph-mon[103124]: pgmap v15215: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 304 KiB/s wr, 12 op/s 2023-09-10T07:10:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:09:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28231.114608 secs 2023-09-10T07:10:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T07:10:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T07:10:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T07:10:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:01 smithi170 ceph-mon[102832]: pgmap v15216: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 412 KiB/s wr, 17 op/s 2023-09-10T07:10:02.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:01 smithi082 ceph-mon[98694]: pgmap v15216: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 412 KiB/s wr, 17 op/s 2023-09-10T07:10:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:01 smithi098 ceph-mon[103124]: pgmap v15216: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 412 KiB/s wr, 17 op/s 2023-09-10T07:10:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:03 smithi170 ceph-mon[102832]: pgmap v15217: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 246 KiB/s wr, 10 op/s 2023-09-10T07:10:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28234.498724 secs 2023-09-10T07:10:04.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:03 smithi082 ceph-mon[98694]: pgmap v15217: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 246 KiB/s wr, 10 op/s 2023-09-10T07:10:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28234.498724 secs 2023-09-10T07:10:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:03 smithi098 ceph-mon[103124]: pgmap v15217: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 246 KiB/s wr, 10 op/s 2023-09-10T07:10:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28234.498724 secs 2023-09-10T07:10:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28235.637176 secs 2023-09-10T07:10:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28236.114672 secs 2023-09-10T07:10:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28235.637176 secs 2023-09-10T07:10:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28236.114672 secs 2023-09-10T07:10:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28235.637176 secs 2023-09-10T07:10:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28236.114672 secs 2023-09-10T07:10:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:05 smithi170 ceph-mon[102832]: pgmap v15218: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 246 KiB/s wr, 10 op/s 2023-09-10T07:10:06.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:05 smithi082 ceph-mon[98694]: pgmap v15218: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 246 KiB/s wr, 10 op/s 2023-09-10T07:10:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:05 smithi098 ceph-mon[103124]: pgmap v15218: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 246 KiB/s wr, 10 op/s 2023-09-10T07:10:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:07 smithi170 ceph-mon[102832]: pgmap v15219: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 291 KiB/s wr, 29 op/s 2023-09-10T07:10:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:07 smithi082 ceph-mon[98694]: pgmap v15219: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 291 KiB/s wr, 29 op/s 2023-09-10T07:10:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:07 smithi098 ceph-mon[103124]: pgmap v15219: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 291 KiB/s wr, 29 op/s 2023-09-10T07:10:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28239.498771 secs 2023-09-10T07:10:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28239.498771 secs 2023-09-10T07:10:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28239.498771 secs 2023-09-10T07:10:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28240.637233 secs 2023-09-10T07:10:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:09 smithi170 ceph-mon[102832]: osdmap e1019: 12 total, 12 up, 12 in 2023-09-10T07:10:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:09 smithi170 ceph-mon[102832]: pgmap v15221: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 183 KiB/s wr, 29 op/s 2023-09-10T07:10:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28240.637233 secs 2023-09-10T07:10:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:09 smithi082 ceph-mon[98694]: osdmap e1019: 12 total, 12 up, 12 in 2023-09-10T07:10:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:09 smithi082 ceph-mon[98694]: pgmap v15221: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 183 KiB/s wr, 29 op/s 2023-09-10T07:10:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28240.637233 secs 2023-09-10T07:10:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:09 smithi098 ceph-mon[103124]: osdmap e1019: 12 total, 12 up, 12 in 2023-09-10T07:10:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:09 smithi098 ceph-mon[103124]: pgmap v15221: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 183 KiB/s wr, 29 op/s 2023-09-10T07:10:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28241.114737 secs 2023-09-10T07:10:10.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28241.114737 secs 2023-09-10T07:10:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28241.114737 secs 2023-09-10T07:10:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:11 smithi170 ceph-mon[102832]: pgmap v15222: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 145 KiB/s wr, 28 op/s 2023-09-10T07:10:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:11 smithi082 ceph-mon[98694]: pgmap v15222: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 145 KiB/s wr, 28 op/s 2023-09-10T07:10:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:11 smithi098 ceph-mon[103124]: pgmap v15222: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 145 KiB/s wr, 28 op/s 2023-09-10T07:10:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:13 smithi098 ceph-mon[103124]: pgmap v15223: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 145 KiB/s wr, 28 op/s 2023-09-10T07:10:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28244.498796 secs 2023-09-10T07:10:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:13 smithi170 ceph-mon[102832]: pgmap v15223: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 145 KiB/s wr, 28 op/s 2023-09-10T07:10:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28244.498796 secs 2023-09-10T07:10:14.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:13 smithi082 ceph-mon[98694]: pgmap v15223: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 145 KiB/s wr, 28 op/s 2023-09-10T07:10:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28244.498796 secs 2023-09-10T07:10:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28245.637291 secs 2023-09-10T07:10:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28246.114799 secs 2023-09-10T07:10:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28245.637291 secs 2023-09-10T07:10:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28246.114799 secs 2023-09-10T07:10:15.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28245.637291 secs 2023-09-10T07:10:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28246.114799 secs 2023-09-10T07:10:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:15 smithi098 ceph-mon[103124]: pgmap v15224: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 145 KiB/s wr, 28 op/s 2023-09-10T07:10:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:15 smithi170 ceph-mon[102832]: pgmap v15224: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 145 KiB/s wr, 28 op/s 2023-09-10T07:10:16.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:15 smithi082 ceph-mon[98694]: pgmap v15224: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 145 KiB/s wr, 28 op/s 2023-09-10T07:10:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:17 smithi170 ceph-mon[102832]: osdmap e1020: 12 total, 12 up, 12 in 2023-09-10T07:10:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:17 smithi170 ceph-mon[102832]: pgmap v15226: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T07:10:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:17 smithi082 ceph-mon[98694]: osdmap e1020: 12 total, 12 up, 12 in 2023-09-10T07:10:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:17 smithi082 ceph-mon[98694]: pgmap v15226: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T07:10:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:17 smithi098 ceph-mon[103124]: osdmap e1020: 12 total, 12 up, 12 in 2023-09-10T07:10:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:17 smithi098 ceph-mon[103124]: pgmap v15226: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 112 KiB/s wr, 12 op/s 2023-09-10T07:10:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28249.498904 secs 2023-09-10T07:10:18.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28249.498904 secs 2023-09-10T07:10:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28249.498904 secs 2023-09-10T07:10:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28250.637376 secs 2023-09-10T07:10:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:19 smithi170 ceph-mon[102832]: pgmap v15227: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T07:10:19.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28250.637376 secs 2023-09-10T07:10:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:19 smithi082 ceph-mon[98694]: pgmap v15227: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T07:10:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28250.637376 secs 2023-09-10T07:10:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:19 smithi098 ceph-mon[103124]: pgmap v15227: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 10 op/s 2023-09-10T07:10:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28251.114863 secs 2023-09-10T07:10:20.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28251.114863 secs 2023-09-10T07:10:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28251.114863 secs 2023-09-10T07:10:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:21 smithi170 ceph-mon[102832]: pgmap v15228: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:10:21.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:21 smithi082 ceph-mon[98694]: pgmap v15228: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:10:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:21 smithi098 ceph-mon[103124]: pgmap v15228: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:10:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:23 smithi098 ceph-mon[103124]: pgmap v15229: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:10:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28254.499001 secs 2023-09-10T07:10:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:23 smithi170 ceph-mon[102832]: pgmap v15229: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:10:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28254.499001 secs 2023-09-10T07:10:24.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:23 smithi082 ceph-mon[98694]: pgmap v15229: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:10:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28254.499001 secs 2023-09-10T07:10:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28255.637436 secs 2023-09-10T07:10:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28256.114924 secs 2023-09-10T07:10:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28255.637436 secs 2023-09-10T07:10:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28256.114924 secs 2023-09-10T07:10:25.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28255.637436 secs 2023-09-10T07:10:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28256.114924 secs 2023-09-10T07:10:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:25 smithi098 ceph-mon[103124]: pgmap v15230: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:10:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:25 smithi170 ceph-mon[102832]: pgmap v15230: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:10:26.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:25 smithi082 ceph-mon[98694]: pgmap v15230: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:10:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:27 smithi170 ceph-mon[102832]: pgmap v15231: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 100 B/s wr, 10 op/s 2023-09-10T07:10:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:27 smithi082 ceph-mon[98694]: pgmap v15231: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 100 B/s wr, 10 op/s 2023-09-10T07:10:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:27 smithi098 ceph-mon[103124]: pgmap v15231: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 100 B/s wr, 10 op/s 2023-09-10T07:10:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28259.499061 secs 2023-09-10T07:10:28.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28259.499061 secs 2023-09-10T07:10:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28259.499061 secs 2023-09-10T07:10:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28260.637491 secs 2023-09-10T07:10:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:29 smithi170 ceph-mon[102832]: pgmap v15232: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:10:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28260.637491 secs 2023-09-10T07:10:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:29 smithi082 ceph-mon[98694]: pgmap v15232: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:10:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28260.637491 secs 2023-09-10T07:10:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:29 smithi098 ceph-mon[103124]: pgmap v15232: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:10:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28261.114985 secs 2023-09-10T07:10:30.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28261.114985 secs 2023-09-10T07:10:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28261.114985 secs 2023-09-10T07:10:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:31 smithi170 ceph-mon[102832]: pgmap v15233: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 51 KiB/s wr, 13 op/s 2023-09-10T07:10:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:31 smithi082 ceph-mon[98694]: pgmap v15233: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 51 KiB/s wr, 13 op/s 2023-09-10T07:10:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:31 smithi098 ceph-mon[103124]: pgmap v15233: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 51 KiB/s wr, 13 op/s 2023-09-10T07:10:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:33 smithi170 ceph-mon[102832]: pgmap v15234: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 51 KiB/s wr, 9 op/s 2023-09-10T07:10:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:10:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:10:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:33 smithi082 ceph-mon[98694]: pgmap v15234: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 51 KiB/s wr, 9 op/s 2023-09-10T07:10:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:10:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:10:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:33 smithi098 ceph-mon[103124]: pgmap v15234: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 51 KiB/s wr, 9 op/s 2023-09-10T07:10:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:10:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:10:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28265.637577 secs 2023-09-10T07:10:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28266.115053 secs 2023-09-10T07:10:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28265.637577 secs 2023-09-10T07:10:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28266.115053 secs 2023-09-10T07:10:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28265.637577 secs 2023-09-10T07:10:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28266.115053 secs 2023-09-10T07:10:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:35 smithi170 ceph-mon[102832]: pgmap v15235: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 51 KiB/s wr, 9 op/s 2023-09-10T07:10:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:10:36.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:35 smithi082 ceph-mon[98694]: pgmap v15235: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 51 KiB/s wr, 9 op/s 2023-09-10T07:10:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:10:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:35 smithi098 ceph-mon[103124]: pgmap v15235: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 51 KiB/s wr, 9 op/s 2023-09-10T07:10:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:10:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:10:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:10:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:10:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:10:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:10:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:10:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:10:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:10:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:10:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:37 smithi170 ceph-mon[102832]: pgmap v15236: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 225 KiB/s wr, 15 op/s 2023-09-10T07:10:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:37 smithi082 ceph-mon[98694]: pgmap v15236: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 225 KiB/s wr, 15 op/s 2023-09-10T07:10:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:37 smithi098 ceph-mon[103124]: pgmap v15236: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 225 KiB/s wr, 15 op/s 2023-09-10T07:10:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28270.637634 secs 2023-09-10T07:10:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:39 smithi170 ceph-mon[102832]: pgmap v15237: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 225 KiB/s wr, 11 op/s 2023-09-10T07:10:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28271.115123 secs 2023-09-10T07:10:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28270.637634 secs 2023-09-10T07:10:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:39 smithi082 ceph-mon[98694]: pgmap v15237: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 225 KiB/s wr, 11 op/s 2023-09-10T07:10:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28271.115123 secs 2023-09-10T07:10:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28270.637634 secs 2023-09-10T07:10:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:39 smithi098 ceph-mon[103124]: pgmap v15237: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 225 KiB/s wr, 11 op/s 2023-09-10T07:10:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28271.115123 secs 2023-09-10T07:10:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:41 smithi170 ceph-mon[102832]: pgmap v15238: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 436 KiB/s wr, 17 op/s 2023-09-10T07:10:42.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:41 smithi082 ceph-mon[98694]: pgmap v15238: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 436 KiB/s wr, 17 op/s 2023-09-10T07:10:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:41 smithi098 ceph-mon[103124]: pgmap v15238: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 436 KiB/s wr, 17 op/s 2023-09-10T07:10:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:43 smithi170 ceph-mon[102832]: pgmap v15239: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 385 KiB/s wr, 12 op/s 2023-09-10T07:10:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:43 smithi082 ceph-mon[98694]: pgmap v15239: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 385 KiB/s wr, 12 op/s 2023-09-10T07:10:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:43 smithi098 ceph-mon[103124]: pgmap v15239: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 385 KiB/s wr, 12 op/s 2023-09-10T07:10:45.104 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28275.637691 secs 2023-09-10T07:10:45.104 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:10:45.104 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:10:45.104 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28276.115185 secs 2023-09-10T07:10:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28275.637691 secs 2023-09-10T07:10:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:10:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:10:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28276.115185 secs 2023-09-10T07:10:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28275.637691 secs 2023-09-10T07:10:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:10:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:10:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28276.115185 secs 2023-09-10T07:10:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:45 smithi170 ceph-mon[102832]: pgmap v15240: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 385 KiB/s wr, 12 op/s 2023-09-10T07:10:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:10:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:10:46.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:45 smithi082 ceph-mon[98694]: pgmap v15240: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 385 KiB/s wr, 12 op/s 2023-09-10T07:10:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:10:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:10:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:45 smithi098 ceph-mon[103124]: pgmap v15240: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 385 KiB/s wr, 12 op/s 2023-09-10T07:10:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:10:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:10:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:10:47.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:10:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:10:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:47 smithi170 ceph-mon[102832]: pgmap v15241: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 655 KiB/s wr, 21 op/s 2023-09-10T07:10:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:10:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:10:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:10:48.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:47 smithi082 ceph-mon[98694]: pgmap v15241: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 655 KiB/s wr, 21 op/s 2023-09-10T07:10:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:10:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:10:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:10:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:47 smithi098 ceph-mon[103124]: pgmap v15241: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 655 KiB/s wr, 21 op/s 2023-09-10T07:10:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:10:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:10:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:10:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28280.637760 secs 2023-09-10T07:10:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:49 smithi170 ceph-mon[102832]: pgmap v15242: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 482 KiB/s wr, 15 op/s 2023-09-10T07:10:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28281.115249 secs 2023-09-10T07:10:50.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28280.637760 secs 2023-09-10T07:10:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:49 smithi082 ceph-mon[98694]: pgmap v15242: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 482 KiB/s wr, 15 op/s 2023-09-10T07:10:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28281.115249 secs 2023-09-10T07:10:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28280.637760 secs 2023-09-10T07:10:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:49 smithi098 ceph-mon[103124]: pgmap v15242: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 482 KiB/s wr, 15 op/s 2023-09-10T07:10:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28281.115249 secs 2023-09-10T07:10:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:51 smithi170 ceph-mon[102832]: pgmap v15243: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 616 KiB/s wr, 21 op/s 2023-09-10T07:10:52.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:51 smithi082 ceph-mon[98694]: pgmap v15243: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 616 KiB/s wr, 21 op/s 2023-09-10T07:10:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:51 smithi098 ceph-mon[103124]: pgmap v15243: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 616 KiB/s wr, 21 op/s 2023-09-10T07:10:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:53 smithi170 ceph-mon[102832]: pgmap v15244: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 404 KiB/s wr, 15 op/s 2023-09-10T07:10:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28284.499595 secs 2023-09-10T07:10:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:10:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:10:54.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:53 smithi082 ceph-mon[98694]: pgmap v15244: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 404 KiB/s wr, 15 op/s 2023-09-10T07:10:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28284.499595 secs 2023-09-10T07:10:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:10:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:10:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:53 smithi098 ceph-mon[103124]: pgmap v15244: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 404 KiB/s wr, 15 op/s 2023-09-10T07:10:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28284.499595 secs 2023-09-10T07:10:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:10:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:10:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28285.637842 secs 2023-09-10T07:10:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28286.115312 secs 2023-09-10T07:10:55.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28285.637842 secs 2023-09-10T07:10:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28286.115312 secs 2023-09-10T07:10:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28285.637842 secs 2023-09-10T07:10:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28286.115312 secs 2023-09-10T07:10:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:55 smithi170 ceph-mon[102832]: pgmap v15245: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 404 KiB/s wr, 15 op/s 2023-09-10T07:10:56.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:55 smithi082 ceph-mon[98694]: pgmap v15245: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 404 KiB/s wr, 15 op/s 2023-09-10T07:10:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:55 smithi098 ceph-mon[103124]: pgmap v15245: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 404 KiB/s wr, 15 op/s 2023-09-10T07:10:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:10:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:10:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:10:57.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:10:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:10:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:10:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:10:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:10:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:10:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:57 smithi170 ceph-mon[102832]: pgmap v15246: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 543 KiB/s wr, 20 op/s 2023-09-10T07:10:58.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:57 smithi082 ceph-mon[98694]: pgmap v15246: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 543 KiB/s wr, 20 op/s 2023-09-10T07:10:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:57 smithi098 ceph-mon[103124]: pgmap v15246: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 543 KiB/s wr, 20 op/s 2023-09-10T07:10:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28289.499654 secs 2023-09-10T07:10:59.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28289.499654 secs 2023-09-10T07:10:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28289.499654 secs 2023-09-10T07:11:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28290.637900 secs 2023-09-10T07:11:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:59 smithi170 ceph-mon[102832]: pgmap v15247: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 273 KiB/s wr, 12 op/s 2023-09-10T07:11:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:10:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28291.115375 secs 2023-09-10T07:11:00.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28290.637900 secs 2023-09-10T07:11:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:59 smithi082 ceph-mon[98694]: pgmap v15247: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 273 KiB/s wr, 12 op/s 2023-09-10T07:11:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:10:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28291.115375 secs 2023-09-10T07:11:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28290.637900 secs 2023-09-10T07:11:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:59 smithi098 ceph-mon[103124]: pgmap v15247: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 273 KiB/s wr, 12 op/s 2023-09-10T07:11:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:10:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28291.115375 secs 2023-09-10T07:11:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:01 smithi170 ceph-mon[102832]: pgmap v15248: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 381 KiB/s wr, 17 op/s 2023-09-10T07:11:02.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:01 smithi082 ceph-mon[98694]: pgmap v15248: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 381 KiB/s wr, 17 op/s 2023-09-10T07:11:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:01 smithi098 ceph-mon[103124]: pgmap v15248: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 381 KiB/s wr, 17 op/s 2023-09-10T07:11:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:03 smithi170 ceph-mon[102832]: pgmap v15249: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 247 KiB/s wr, 10 op/s 2023-09-10T07:11:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28294.499711 secs 2023-09-10T07:11:04.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:03 smithi082 ceph-mon[98694]: pgmap v15249: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 247 KiB/s wr, 10 op/s 2023-09-10T07:11:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28294.499711 secs 2023-09-10T07:11:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:03 smithi098 ceph-mon[103124]: pgmap v15249: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 247 KiB/s wr, 10 op/s 2023-09-10T07:11:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28294.499711 secs 2023-09-10T07:11:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28295.637959 secs 2023-09-10T07:11:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28296.115437 secs 2023-09-10T07:11:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28295.637959 secs 2023-09-10T07:11:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28296.115437 secs 2023-09-10T07:11:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28295.637959 secs 2023-09-10T07:11:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28296.115437 secs 2023-09-10T07:11:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:05 smithi170 ceph-mon[102832]: pgmap v15250: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 247 KiB/s wr, 10 op/s 2023-09-10T07:11:06.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:05 smithi082 ceph-mon[98694]: pgmap v15250: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 247 KiB/s wr, 10 op/s 2023-09-10T07:11:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:05 smithi098 ceph-mon[103124]: pgmap v15250: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 247 KiB/s wr, 10 op/s 2023-09-10T07:11:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:07 smithi170 ceph-mon[102832]: pgmap v15251: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 292 KiB/s wr, 29 op/s 2023-09-10T07:11:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:07 smithi082 ceph-mon[98694]: pgmap v15251: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 292 KiB/s wr, 29 op/s 2023-09-10T07:11:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:07 smithi098 ceph-mon[103124]: pgmap v15251: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 292 KiB/s wr, 29 op/s 2023-09-10T07:11:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28299.499777 secs 2023-09-10T07:11:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28299.499777 secs 2023-09-10T07:11:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28299.499777 secs 2023-09-10T07:11:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28300.638016 secs 2023-09-10T07:11:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:09 smithi170 ceph-mon[102832]: osdmap e1021: 12 total, 12 up, 12 in 2023-09-10T07:11:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:09 smithi170 ceph-mon[102832]: pgmap v15253: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 184 KiB/s wr, 28 op/s 2023-09-10T07:11:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28300.638016 secs 2023-09-10T07:11:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:09 smithi082 ceph-mon[98694]: osdmap e1021: 12 total, 12 up, 12 in 2023-09-10T07:11:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:09 smithi082 ceph-mon[98694]: pgmap v15253: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 184 KiB/s wr, 28 op/s 2023-09-10T07:11:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28300.638016 secs 2023-09-10T07:11:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:09 smithi098 ceph-mon[103124]: osdmap e1021: 12 total, 12 up, 12 in 2023-09-10T07:11:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:09 smithi098 ceph-mon[103124]: pgmap v15253: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 184 KiB/s wr, 28 op/s 2023-09-10T07:11:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28301.115487 secs 2023-09-10T07:11:10.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28301.115487 secs 2023-09-10T07:11:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28301.115487 secs 2023-09-10T07:11:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:11 smithi170 ceph-mon[102832]: pgmap v15254: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:11:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:11 smithi082 ceph-mon[98694]: pgmap v15254: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:11:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:11 smithi098 ceph-mon[103124]: pgmap v15254: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:11:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:13 smithi170 ceph-mon[102832]: pgmap v15255: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:11:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28304.499851 secs 2023-09-10T07:11:14.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:13 smithi082 ceph-mon[98694]: pgmap v15255: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:11:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28304.499851 secs 2023-09-10T07:11:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:13 smithi098 ceph-mon[103124]: pgmap v15255: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:11:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28304.499851 secs 2023-09-10T07:11:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28305.638076 secs 2023-09-10T07:11:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28306.115552 secs 2023-09-10T07:11:15.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28305.638076 secs 2023-09-10T07:11:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28306.115552 secs 2023-09-10T07:11:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28305.638076 secs 2023-09-10T07:11:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28306.115552 secs 2023-09-10T07:11:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:15 smithi170 ceph-mon[102832]: pgmap v15256: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:11:16.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:15 smithi082 ceph-mon[98694]: pgmap v15256: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:11:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:15 smithi098 ceph-mon[103124]: pgmap v15256: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:11:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:17 smithi170 ceph-mon[102832]: osdmap e1022: 12 total, 12 up, 12 in 2023-09-10T07:11:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:17 smithi170 ceph-mon[102832]: pgmap v15258: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 251 B/s wr, 12 op/s 2023-09-10T07:11:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:17 smithi082 ceph-mon[98694]: osdmap e1022: 12 total, 12 up, 12 in 2023-09-10T07:11:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:17 smithi082 ceph-mon[98694]: pgmap v15258: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 251 B/s wr, 12 op/s 2023-09-10T07:11:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:17 smithi098 ceph-mon[103124]: osdmap e1022: 12 total, 12 up, 12 in 2023-09-10T07:11:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:17 smithi098 ceph-mon[103124]: pgmap v15258: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 251 B/s wr, 12 op/s 2023-09-10T07:11:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28309.499912 secs 2023-09-10T07:11:18.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28309.499912 secs 2023-09-10T07:11:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28309.499912 secs 2023-09-10T07:11:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28310.638134 secs 2023-09-10T07:11:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:19 smithi170 ceph-mon[102832]: pgmap v15259: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:11:19.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28310.638134 secs 2023-09-10T07:11:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:19 smithi082 ceph-mon[98694]: pgmap v15259: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:11:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28310.638134 secs 2023-09-10T07:11:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:19 smithi098 ceph-mon[103124]: pgmap v15259: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:11:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28311.115616 secs 2023-09-10T07:11:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28311.115616 secs 2023-09-10T07:11:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28311.115616 secs 2023-09-10T07:11:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:21 smithi170 ceph-mon[102832]: pgmap v15260: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:11:21.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:21 smithi082 ceph-mon[98694]: pgmap v15260: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:11:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:21 smithi098 ceph-mon[103124]: pgmap v15260: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:11:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:23 smithi170 ceph-mon[102832]: pgmap v15261: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:11:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28314.500042 secs 2023-09-10T07:11:24.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:23 smithi082 ceph-mon[98694]: pgmap v15261: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:11:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28314.500042 secs 2023-09-10T07:11:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:23 smithi098 ceph-mon[103124]: pgmap v15261: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:11:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28314.500042 secs 2023-09-10T07:11:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28315.638195 secs 2023-09-10T07:11:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28316.115677 secs 2023-09-10T07:11:25.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28315.638195 secs 2023-09-10T07:11:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28316.115677 secs 2023-09-10T07:11:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28315.638195 secs 2023-09-10T07:11:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28316.115677 secs 2023-09-10T07:11:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:25 smithi170 ceph-mon[102832]: pgmap v15262: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:11:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:25 smithi082 ceph-mon[98694]: pgmap v15262: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:11:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:25 smithi098 ceph-mon[103124]: pgmap v15262: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:11:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:27 smithi170 ceph-mon[102832]: pgmap v15263: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T07:11:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:27 smithi082 ceph-mon[98694]: pgmap v15263: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T07:11:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:27 smithi098 ceph-mon[103124]: pgmap v15263: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T07:11:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28319.500156 secs 2023-09-10T07:11:28.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28319.500156 secs 2023-09-10T07:11:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28319.500156 secs 2023-09-10T07:11:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28320.638280 secs 2023-09-10T07:11:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:29 smithi170 ceph-mon[102832]: pgmap v15264: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 8 op/s 2023-09-10T07:11:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28320.638280 secs 2023-09-10T07:11:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:29 smithi082 ceph-mon[98694]: pgmap v15264: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 8 op/s 2023-09-10T07:11:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28320.638280 secs 2023-09-10T07:11:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:29 smithi098 ceph-mon[103124]: pgmap v15264: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 92 KiB/s wr, 8 op/s 2023-09-10T07:11:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28321.115742 secs 2023-09-10T07:11:30.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28321.115742 secs 2023-09-10T07:11:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28321.115742 secs 2023-09-10T07:11:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:31 smithi170 ceph-mon[102832]: pgmap v15265: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 98 KiB/s wr, 13 op/s 2023-09-10T07:11:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:31 smithi082 ceph-mon[98694]: pgmap v15265: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 98 KiB/s wr, 13 op/s 2023-09-10T07:11:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:31 smithi098 ceph-mon[103124]: pgmap v15265: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 98 KiB/s wr, 13 op/s 2023-09-10T07:11:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:33 smithi170 ceph-mon[102832]: pgmap v15266: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:11:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:11:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:11:34.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:33 smithi082 ceph-mon[98694]: pgmap v15266: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:11:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:11:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:11:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:33 smithi098 ceph-mon[103124]: pgmap v15266: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:11:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:11:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:11:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28325.638339 secs 2023-09-10T07:11:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28326.115802 secs 2023-09-10T07:11:35.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28325.638339 secs 2023-09-10T07:11:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28326.115802 secs 2023-09-10T07:11:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28325.638339 secs 2023-09-10T07:11:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28326.115802 secs 2023-09-10T07:11:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:35 smithi170 ceph-mon[102832]: pgmap v15267: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:11:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:11:36.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:35 smithi082 ceph-mon[98694]: pgmap v15267: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:11:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:11:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:35 smithi098 ceph-mon[103124]: pgmap v15267: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:11:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:11:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:11:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:11:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:11:37.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:11:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:11:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:11:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:11:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:11:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:11:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:37 smithi170 ceph-mon[102832]: pgmap v15268: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 315 KiB/s wr, 16 op/s 2023-09-10T07:11:38.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:37 smithi082 ceph-mon[98694]: pgmap v15268: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 315 KiB/s wr, 16 op/s 2023-09-10T07:11:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:37 smithi098 ceph-mon[103124]: pgmap v15268: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 315 KiB/s wr, 16 op/s 2023-09-10T07:11:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28330.638397 secs 2023-09-10T07:11:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:39 smithi170 ceph-mon[102832]: pgmap v15269: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T07:11:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28331.115870 secs 2023-09-10T07:11:40.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28330.638397 secs 2023-09-10T07:11:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:39 smithi082 ceph-mon[98694]: pgmap v15269: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T07:11:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28331.115870 secs 2023-09-10T07:11:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28330.638397 secs 2023-09-10T07:11:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:39 smithi098 ceph-mon[103124]: pgmap v15269: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T07:11:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28331.115870 secs 2023-09-10T07:11:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:41 smithi170 ceph-mon[102832]: pgmap v15270: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 434 KiB/s wr, 18 op/s 2023-09-10T07:11:42.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:41 smithi082 ceph-mon[98694]: pgmap v15270: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 434 KiB/s wr, 18 op/s 2023-09-10T07:11:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:41 smithi098 ceph-mon[103124]: pgmap v15270: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 434 KiB/s wr, 18 op/s 2023-09-10T07:11:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:43 smithi170 ceph-mon[102832]: pgmap v15271: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 428 KiB/s wr, 13 op/s 2023-09-10T07:11:44.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:43 smithi082 ceph-mon[98694]: pgmap v15271: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 428 KiB/s wr, 13 op/s 2023-09-10T07:11:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:43 smithi098 ceph-mon[103124]: pgmap v15271: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 428 KiB/s wr, 13 op/s 2023-09-10T07:11:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28335.638479 secs 2023-09-10T07:11:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:11:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:11:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28336.115930 secs 2023-09-10T07:11:45.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28335.638479 secs 2023-09-10T07:11:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:11:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:11:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28336.115930 secs 2023-09-10T07:11:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28335.638479 secs 2023-09-10T07:11:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:11:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:11:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28336.115930 secs 2023-09-10T07:11:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:45 smithi170 ceph-mon[102832]: pgmap v15272: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 428 KiB/s wr, 13 op/s 2023-09-10T07:11:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:11:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:11:46.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:45 smithi082 ceph-mon[98694]: pgmap v15272: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 428 KiB/s wr, 13 op/s 2023-09-10T07:11:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:11:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:11:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:45 smithi098 ceph-mon[103124]: pgmap v15272: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 428 KiB/s wr, 13 op/s 2023-09-10T07:11:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:11:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:11:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:47 smithi170 ceph-mon[102832]: pgmap v15273: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 597 KiB/s wr, 19 op/s 2023-09-10T07:11:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:11:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:47 smithi082 ceph-mon[98694]: pgmap v15273: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 597 KiB/s wr, 19 op/s 2023-09-10T07:11:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:11:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:47 smithi098 ceph-mon[103124]: pgmap v15273: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 597 KiB/s wr, 19 op/s 2023-09-10T07:11:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:11:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:11:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:11:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:11:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:11:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:11:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:11:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:11:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:11:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:11:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28340.638497 secs 2023-09-10T07:11:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:49 smithi170 ceph-mon[102832]: pgmap v15274: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 380 KiB/s wr, 13 op/s 2023-09-10T07:11:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28340.638497 secs 2023-09-10T07:11:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:49 smithi082 ceph-mon[98694]: pgmap v15274: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 380 KiB/s wr, 13 op/s 2023-09-10T07:11:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28340.638497 secs 2023-09-10T07:11:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:49 smithi098 ceph-mon[103124]: pgmap v15274: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 380 KiB/s wr, 13 op/s 2023-09-10T07:11:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28341.115991 secs 2023-09-10T07:11:50.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28341.115991 secs 2023-09-10T07:11:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28341.115991 secs 2023-09-10T07:11:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:51 smithi170 ceph-mon[102832]: pgmap v15275: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 583 KiB/s wr, 19 op/s 2023-09-10T07:11:51.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:51 smithi082 ceph-mon[98694]: pgmap v15275: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 583 KiB/s wr, 19 op/s 2023-09-10T07:11:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:51 smithi098 ceph-mon[103124]: pgmap v15275: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 583 KiB/s wr, 19 op/s 2023-09-10T07:11:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:53 smithi170 ceph-mon[102832]: pgmap v15276: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 372 KiB/s wr, 12 op/s 2023-09-10T07:11:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:11:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:11:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:53 smithi082 ceph-mon[98694]: pgmap v15276: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 372 KiB/s wr, 12 op/s 2023-09-10T07:11:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:11:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:11:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:53 smithi098 ceph-mon[103124]: pgmap v15276: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 372 KiB/s wr, 12 op/s 2023-09-10T07:11:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:11:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:11:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28345.638556 secs 2023-09-10T07:11:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28346.116056 secs 2023-09-10T07:11:55.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28345.638556 secs 2023-09-10T07:11:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28346.116056 secs 2023-09-10T07:11:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28345.638556 secs 2023-09-10T07:11:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28346.116056 secs 2023-09-10T07:11:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:55 smithi170 ceph-mon[102832]: pgmap v15277: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 372 KiB/s wr, 12 op/s 2023-09-10T07:11:56.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:55 smithi082 ceph-mon[98694]: pgmap v15277: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 372 KiB/s wr, 12 op/s 2023-09-10T07:11:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:55 smithi098 ceph-mon[103124]: pgmap v15277: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 372 KiB/s wr, 12 op/s 2023-09-10T07:11:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:11:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:11:57.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:11:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:11:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:11:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:11:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:57 smithi170 ceph-mon[102832]: pgmap v15278: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 578 KiB/s wr, 19 op/s 2023-09-10T07:11:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:57 smithi082 ceph-mon[98694]: pgmap v15278: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 578 KiB/s wr, 19 op/s 2023-09-10T07:11:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:57 smithi098 ceph-mon[103124]: pgmap v15278: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 578 KiB/s wr, 19 op/s 2023-09-10T07:11:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28349.500586 secs 2023-09-10T07:11:59.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28349.500586 secs 2023-09-10T07:11:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28349.500586 secs 2023-09-10T07:12:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28350.638616 secs 2023-09-10T07:12:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:59 smithi170 ceph-mon[102832]: pgmap v15279: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 410 KiB/s wr, 12 op/s 2023-09-10T07:12:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:11:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28351.116117 secs 2023-09-10T07:12:00.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28350.638616 secs 2023-09-10T07:12:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:59 smithi082 ceph-mon[98694]: pgmap v15279: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 410 KiB/s wr, 12 op/s 2023-09-10T07:12:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:11:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28351.116117 secs 2023-09-10T07:12:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28350.638616 secs 2023-09-10T07:12:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:59 smithi098 ceph-mon[103124]: pgmap v15279: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 410 KiB/s wr, 12 op/s 2023-09-10T07:12:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:11:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28351.116117 secs 2023-09-10T07:12:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:12:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:12:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:01.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:12:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:12:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:12:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:12:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:01 smithi170 ceph-mon[102832]: pgmap v15280: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 659 KiB/s wr, 33 op/s 2023-09-10T07:12:02.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:01 smithi082 ceph-mon[98694]: pgmap v15280: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 659 KiB/s wr, 33 op/s 2023-09-10T07:12:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:01 smithi098 ceph-mon[103124]: pgmap v15280: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 659 KiB/s wr, 33 op/s 2023-09-10T07:12:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:03 smithi170 ceph-mon[102832]: pgmap v15281: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 456 KiB/s wr, 27 op/s 2023-09-10T07:12:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28354.500644 secs 2023-09-10T07:12:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:03 smithi170 ceph-mon[102832]: osdmap e1023: 12 total, 12 up, 12 in 2023-09-10T07:12:04.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:03 smithi082 ceph-mon[98694]: pgmap v15281: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 456 KiB/s wr, 27 op/s 2023-09-10T07:12:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28354.500644 secs 2023-09-10T07:12:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:03 smithi082 ceph-mon[98694]: osdmap e1023: 12 total, 12 up, 12 in 2023-09-10T07:12:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:03 smithi098 ceph-mon[103124]: pgmap v15281: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 456 KiB/s wr, 27 op/s 2023-09-10T07:12:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28354.500644 secs 2023-09-10T07:12:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:03 smithi098 ceph-mon[103124]: osdmap e1023: 12 total, 12 up, 12 in 2023-09-10T07:12:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28355.638675 secs 2023-09-10T07:12:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:12:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28356.116181 secs 2023-09-10T07:12:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28355.638675 secs 2023-09-10T07:12:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:12:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28356.116181 secs 2023-09-10T07:12:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28355.638675 secs 2023-09-10T07:12:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:12:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28356.116181 secs 2023-09-10T07:12:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:05 smithi170 ceph-mon[102832]: pgmap v15283: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 547 KiB/s wr, 32 op/s 2023-09-10T07:12:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:12:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:05 smithi082 ceph-mon[98694]: pgmap v15283: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 547 KiB/s wr, 32 op/s 2023-09-10T07:12:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:12:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:05 smithi098 ceph-mon[103124]: pgmap v15283: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 547 KiB/s wr, 32 op/s 2023-09-10T07:12:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:12:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:07 smithi170 ceph-mon[102832]: pgmap v15284: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 445 KiB/s wr, 31 op/s 2023-09-10T07:12:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:07 smithi082 ceph-mon[98694]: pgmap v15284: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 445 KiB/s wr, 31 op/s 2023-09-10T07:12:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:07 smithi098 ceph-mon[103124]: pgmap v15284: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 445 KiB/s wr, 31 op/s 2023-09-10T07:12:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28359.500736 secs 2023-09-10T07:12:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28359.500736 secs 2023-09-10T07:12:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28359.500736 secs 2023-09-10T07:12:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28360.638732 secs 2023-09-10T07:12:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:09 smithi170 ceph-mon[102832]: pgmap v15285: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 445 KiB/s wr, 31 op/s 2023-09-10T07:12:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28360.638732 secs 2023-09-10T07:12:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:09 smithi082 ceph-mon[98694]: pgmap v15285: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 445 KiB/s wr, 31 op/s 2023-09-10T07:12:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28360.638732 secs 2023-09-10T07:12:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:09 smithi098 ceph-mon[103124]: pgmap v15285: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 445 KiB/s wr, 31 op/s 2023-09-10T07:12:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28361.116245 secs 2023-09-10T07:12:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28361.116245 secs 2023-09-10T07:12:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28361.116245 secs 2023-09-10T07:12:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:11 smithi170 ceph-mon[102832]: pgmap v15286: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 239 KiB/s wr, 12 op/s 2023-09-10T07:12:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:11 smithi170 ceph-mon[102832]: osdmap e1024: 12 total, 12 up, 12 in 2023-09-10T07:12:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:11 smithi082 ceph-mon[98694]: pgmap v15286: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 239 KiB/s wr, 12 op/s 2023-09-10T07:12:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:11 smithi082 ceph-mon[98694]: osdmap e1024: 12 total, 12 up, 12 in 2023-09-10T07:12:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:11 smithi098 ceph-mon[103124]: pgmap v15286: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 239 KiB/s wr, 12 op/s 2023-09-10T07:12:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:11 smithi098 ceph-mon[103124]: osdmap e1024: 12 total, 12 up, 12 in 2023-09-10T07:12:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:13 smithi170 ceph-mon[102832]: pgmap v15288: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 242 KiB/s wr, 12 op/s 2023-09-10T07:12:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28364.500776 secs 2023-09-10T07:12:14.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:13 smithi082 ceph-mon[98694]: pgmap v15288: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 242 KiB/s wr, 12 op/s 2023-09-10T07:12:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28364.500776 secs 2023-09-10T07:12:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:13 smithi098 ceph-mon[103124]: pgmap v15288: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 242 KiB/s wr, 12 op/s 2023-09-10T07:12:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28364.500776 secs 2023-09-10T07:12:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28365.638804 secs 2023-09-10T07:12:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28366.116307 secs 2023-09-10T07:12:15.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28365.638804 secs 2023-09-10T07:12:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28366.116307 secs 2023-09-10T07:12:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28365.638804 secs 2023-09-10T07:12:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28366.116307 secs 2023-09-10T07:12:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:15 smithi170 ceph-mon[102832]: pgmap v15289: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 239 KiB/s wr, 12 op/s 2023-09-10T07:12:16.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:15 smithi082 ceph-mon[98694]: pgmap v15289: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 239 KiB/s wr, 12 op/s 2023-09-10T07:12:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:15 smithi098 ceph-mon[103124]: pgmap v15289: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 239 KiB/s wr, 12 op/s 2023-09-10T07:12:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:17 smithi170 ceph-mon[102832]: pgmap v15290: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T07:12:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:17 smithi082 ceph-mon[98694]: pgmap v15290: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T07:12:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:17 smithi098 ceph-mon[103124]: pgmap v15290: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T07:12:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28369.500849 secs 2023-09-10T07:12:18.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28369.500849 secs 2023-09-10T07:12:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28369.500849 secs 2023-09-10T07:12:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28370.638891 secs 2023-09-10T07:12:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:19 smithi170 ceph-mon[102832]: pgmap v15291: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T07:12:19.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28370.638891 secs 2023-09-10T07:12:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:19 smithi082 ceph-mon[98694]: pgmap v15291: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T07:12:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28370.638891 secs 2023-09-10T07:12:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:19 smithi098 ceph-mon[103124]: pgmap v15291: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T07:12:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28371.116368 secs 2023-09-10T07:12:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28371.116368 secs 2023-09-10T07:12:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28371.116368 secs 2023-09-10T07:12:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:21 smithi170 ceph-mon[102832]: pgmap v15292: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T07:12:21.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:21 smithi082 ceph-mon[98694]: pgmap v15292: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T07:12:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:21 smithi098 ceph-mon[103124]: pgmap v15292: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T07:12:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:23 smithi170 ceph-mon[102832]: pgmap v15293: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 107 KiB/s wr, 10 op/s 2023-09-10T07:12:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28374.500910 secs 2023-09-10T07:12:24.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:23 smithi082 ceph-mon[98694]: pgmap v15293: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 107 KiB/s wr, 10 op/s 2023-09-10T07:12:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28374.500910 secs 2023-09-10T07:12:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:23 smithi098 ceph-mon[103124]: pgmap v15293: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 107 KiB/s wr, 10 op/s 2023-09-10T07:12:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28374.500910 secs 2023-09-10T07:12:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28375.638975 secs 2023-09-10T07:12:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28376.116429 secs 2023-09-10T07:12:25.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28375.638975 secs 2023-09-10T07:12:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28376.116429 secs 2023-09-10T07:12:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28375.638975 secs 2023-09-10T07:12:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28376.116429 secs 2023-09-10T07:12:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:25 smithi170 ceph-mon[102832]: pgmap v15294: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 10 op/s 2023-09-10T07:12:26.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:25 smithi082 ceph-mon[98694]: pgmap v15294: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 10 op/s 2023-09-10T07:12:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:25 smithi098 ceph-mon[103124]: pgmap v15294: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 10 op/s 2023-09-10T07:12:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:27 smithi170 ceph-mon[102832]: pgmap v15295: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 100 KiB/s wr, 14 op/s 2023-09-10T07:12:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:27 smithi082 ceph-mon[98694]: pgmap v15295: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 100 KiB/s wr, 14 op/s 2023-09-10T07:12:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:27 smithi098 ceph-mon[103124]: pgmap v15295: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 100 KiB/s wr, 14 op/s 2023-09-10T07:12:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28379.500969 secs 2023-09-10T07:12:28.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28379.500969 secs 2023-09-10T07:12:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28379.500969 secs 2023-09-10T07:12:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28380.639034 secs 2023-09-10T07:12:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:29 smithi170 ceph-mon[102832]: pgmap v15296: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 10 op/s 2023-09-10T07:12:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28380.639034 secs 2023-09-10T07:12:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:29 smithi082 ceph-mon[98694]: pgmap v15296: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 10 op/s 2023-09-10T07:12:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28380.639034 secs 2023-09-10T07:12:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:29 smithi098 ceph-mon[103124]: pgmap v15296: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 10 op/s 2023-09-10T07:12:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28381.116482 secs 2023-09-10T07:12:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:12:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:30.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28381.116482 secs 2023-09-10T07:12:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:12:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28381.116482 secs 2023-09-10T07:12:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:12:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:31 smithi170 ceph-mon[102832]: pgmap v15297: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 196 KiB/s wr, 15 op/s 2023-09-10T07:12:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:31 smithi082 ceph-mon[98694]: pgmap v15297: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 196 KiB/s wr, 15 op/s 2023-09-10T07:12:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:31 smithi098 ceph-mon[103124]: pgmap v15297: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 196 KiB/s wr, 15 op/s 2023-09-10T07:12:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:12:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:12:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:12:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:33 smithi170 ceph-mon[102832]: pgmap v15298: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 9 op/s 2023-09-10T07:12:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28384.501027 secs 2023-09-10T07:12:33.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:33 smithi082 ceph-mon[98694]: pgmap v15298: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 9 op/s 2023-09-10T07:12:33.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28384.501027 secs 2023-09-10T07:12:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:33 smithi098 ceph-mon[103124]: pgmap v15298: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 9 op/s 2023-09-10T07:12:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28384.501027 secs 2023-09-10T07:12:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28385.639093 secs 2023-09-10T07:12:34.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28385.639093 secs 2023-09-10T07:12:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28385.639093 secs 2023-09-10T07:12:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28386.116542 secs 2023-09-10T07:12:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:35 smithi170 ceph-mon[102832]: pgmap v15299: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 9 op/s 2023-09-10T07:12:35.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28386.116542 secs 2023-09-10T07:12:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:35 smithi082 ceph-mon[98694]: pgmap v15299: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 9 op/s 2023-09-10T07:12:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28386.116542 secs 2023-09-10T07:12:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:35 smithi098 ceph-mon[103124]: pgmap v15299: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 9 op/s 2023-09-10T07:12:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:37 smithi170 ceph-mon[102832]: pgmap v15300: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 333 KiB/s wr, 16 op/s 2023-09-10T07:12:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:37 smithi082 ceph-mon[98694]: pgmap v15300: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 333 KiB/s wr, 16 op/s 2023-09-10T07:12:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:37 smithi098 ceph-mon[103124]: pgmap v15300: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 333 KiB/s wr, 16 op/s 2023-09-10T07:12:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28390.639154 secs 2023-09-10T07:12:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:39 smithi170 ceph-mon[102832]: pgmap v15301: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 333 KiB/s wr, 11 op/s 2023-09-10T07:12:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28391.116601 secs 2023-09-10T07:12:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:12:40.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28390.639154 secs 2023-09-10T07:12:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:39 smithi082 ceph-mon[98694]: pgmap v15301: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 333 KiB/s wr, 11 op/s 2023-09-10T07:12:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28391.116601 secs 2023-09-10T07:12:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:12:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28390.639154 secs 2023-09-10T07:12:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:39 smithi098 ceph-mon[103124]: pgmap v15301: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 333 KiB/s wr, 11 op/s 2023-09-10T07:12:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28391.116601 secs 2023-09-10T07:12:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:12:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:12:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:12:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:41.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:12:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:12:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:12:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:12:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:41 smithi170 ceph-mon[102832]: pgmap v15302: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 532 KiB/s wr, 17 op/s 2023-09-10T07:12:42.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:41 smithi082 ceph-mon[98694]: pgmap v15302: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 532 KiB/s wr, 17 op/s 2023-09-10T07:12:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:41 smithi098 ceph-mon[103124]: pgmap v15302: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 532 KiB/s wr, 17 op/s 2023-09-10T07:12:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:43 smithi170 ceph-mon[102832]: pgmap v15303: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 436 KiB/s wr, 12 op/s 2023-09-10T07:12:44.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:43 smithi082 ceph-mon[98694]: pgmap v15303: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 436 KiB/s wr, 12 op/s 2023-09-10T07:12:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:43 smithi098 ceph-mon[103124]: pgmap v15303: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 436 KiB/s wr, 12 op/s 2023-09-10T07:12:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28395.639223 secs 2023-09-10T07:12:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:12:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28396.116662 secs 2023-09-10T07:12:45.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28395.639223 secs 2023-09-10T07:12:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:12:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28396.116662 secs 2023-09-10T07:12:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28395.639223 secs 2023-09-10T07:12:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:12:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28396.116662 secs 2023-09-10T07:12:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:45 smithi170 ceph-mon[102832]: pgmap v15304: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 436 KiB/s wr, 12 op/s 2023-09-10T07:12:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:12:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:46.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:45 smithi082 ceph-mon[98694]: pgmap v15304: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 436 KiB/s wr, 12 op/s 2023-09-10T07:12:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:12:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:45 smithi098 ceph-mon[103124]: pgmap v15304: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 436 KiB/s wr, 12 op/s 2023-09-10T07:12:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:12:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:47 smithi170 ceph-mon[102832]: pgmap v15305: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 625 KiB/s wr, 18 op/s 2023-09-10T07:12:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:47 smithi082 ceph-mon[98694]: pgmap v15305: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 625 KiB/s wr, 18 op/s 2023-09-10T07:12:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:47 smithi098 ceph-mon[103124]: pgmap v15305: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 625 KiB/s wr, 18 op/s 2023-09-10T07:12:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:12:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:12:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:12:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:12:48.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:12:48.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:12:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:12:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:12:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:12:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:12:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:12:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:12:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28400.639320 secs 2023-09-10T07:12:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:49 smithi170 ceph-mon[102832]: pgmap v15306: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 388 KiB/s wr, 12 op/s 2023-09-10T07:12:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28400.639320 secs 2023-09-10T07:12:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:49 smithi082 ceph-mon[98694]: pgmap v15306: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 388 KiB/s wr, 12 op/s 2023-09-10T07:12:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28400.639320 secs 2023-09-10T07:12:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:49 smithi098 ceph-mon[103124]: pgmap v15306: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 388 KiB/s wr, 12 op/s 2023-09-10T07:12:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28401.116725 secs 2023-09-10T07:12:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:12:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:50.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28401.116725 secs 2023-09-10T07:12:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:12:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28401.116725 secs 2023-09-10T07:12:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:12:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:51 smithi170 ceph-mon[102832]: pgmap v15307: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 592 KiB/s wr, 18 op/s 2023-09-10T07:12:51.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:51 smithi082 ceph-mon[98694]: pgmap v15307: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 592 KiB/s wr, 18 op/s 2023-09-10T07:12:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:51 smithi098 ceph-mon[103124]: pgmap v15307: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 592 KiB/s wr, 18 op/s 2023-09-10T07:12:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:12:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:52.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:12:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:12:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:53 smithi170 ceph-mon[102832]: pgmap v15308: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 393 KiB/s wr, 12 op/s 2023-09-10T07:12:53.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:53 smithi082 ceph-mon[98694]: pgmap v15308: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 393 KiB/s wr, 12 op/s 2023-09-10T07:12:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:53 smithi098 ceph-mon[103124]: pgmap v15308: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 393 KiB/s wr, 12 op/s 2023-09-10T07:12:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28405.639377 secs 2023-09-10T07:12:54.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28405.639377 secs 2023-09-10T07:12:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28405.639377 secs 2023-09-10T07:12:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28406.116787 secs 2023-09-10T07:12:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:55 smithi170 ceph-mon[102832]: pgmap v15309: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 393 KiB/s wr, 12 op/s 2023-09-10T07:12:55.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28406.116787 secs 2023-09-10T07:12:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:55 smithi082 ceph-mon[98694]: pgmap v15309: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 393 KiB/s wr, 12 op/s 2023-09-10T07:12:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28406.116787 secs 2023-09-10T07:12:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:55 smithi098 ceph-mon[103124]: pgmap v15309: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 393 KiB/s wr, 12 op/s 2023-09-10T07:12:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:12:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:12:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:12:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:57 smithi170 ceph-mon[102832]: pgmap v15310: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 583 KiB/s wr, 18 op/s 2023-09-10T07:12:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:57 smithi082 ceph-mon[98694]: pgmap v15310: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 583 KiB/s wr, 18 op/s 2023-09-10T07:12:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:57 smithi098 ceph-mon[103124]: pgmap v15310: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 583 KiB/s wr, 18 op/s 2023-09-10T07:12:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:12:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28409.501410 secs 2023-09-10T07:12:58.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:12:58.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28409.501410 secs 2023-09-10T07:12:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:12:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:12:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28409.501410 secs 2023-09-10T07:12:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28410.639436 secs 2023-09-10T07:12:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:12:59 smithi170 ceph-mon[102832]: pgmap v15311: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 394 KiB/s wr, 12 op/s 2023-09-10T07:12:59.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28410.639436 secs 2023-09-10T07:12:59.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:12:59 smithi082 ceph-mon[98694]: pgmap v15311: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 394 KiB/s wr, 12 op/s 2023-09-10T07:13:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28410.639436 secs 2023-09-10T07:13:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:12:59 smithi098 ceph-mon[103124]: pgmap v15311: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 394 KiB/s wr, 12 op/s 2023-09-10T07:13:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28411.116852 secs 2023-09-10T07:13:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:13:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:13:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:00.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28411.116852 secs 2023-09-10T07:13:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:13:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:13:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28411.116852 secs 2023-09-10T07:13:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:13:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:13:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:01 smithi170 ceph-mon[102832]: pgmap v15312: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 521 KiB/s wr, 31 op/s 2023-09-10T07:13:01.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:01 smithi082 ceph-mon[98694]: pgmap v15312: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 521 KiB/s wr, 31 op/s 2023-09-10T07:13:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:01 smithi098 ceph-mon[103124]: pgmap v15312: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 521 KiB/s wr, 31 op/s 2023-09-10T07:13:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:03 smithi170 ceph-mon[102832]: pgmap v15313: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 317 KiB/s wr, 25 op/s 2023-09-10T07:13:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28414.501471 secs 2023-09-10T07:13:04.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:03 smithi082 ceph-mon[98694]: pgmap v15313: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 317 KiB/s wr, 25 op/s 2023-09-10T07:13:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28414.501471 secs 2023-09-10T07:13:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:03 smithi098 ceph-mon[103124]: pgmap v15313: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 317 KiB/s wr, 25 op/s 2023-09-10T07:13:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28414.501471 secs 2023-09-10T07:13:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28415.639490 secs 2023-09-10T07:13:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:04 smithi170 ceph-mon[102832]: osdmap e1025: 12 total, 12 up, 12 in 2023-09-10T07:13:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28416.116915 secs 2023-09-10T07:13:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28415.639490 secs 2023-09-10T07:13:05.220 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:04 smithi082 ceph-mon[98694]: osdmap e1025: 12 total, 12 up, 12 in 2023-09-10T07:13:05.220 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28416.116915 secs 2023-09-10T07:13:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28415.639490 secs 2023-09-10T07:13:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:04 smithi098 ceph-mon[103124]: osdmap e1025: 12 total, 12 up, 12 in 2023-09-10T07:13:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28416.116915 secs 2023-09-10T07:13:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:05 smithi170 ceph-mon[102832]: pgmap v15315: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 381 KiB/s wr, 31 op/s 2023-09-10T07:13:06.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:05 smithi082 ceph-mon[98694]: pgmap v15315: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 381 KiB/s wr, 31 op/s 2023-09-10T07:13:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:05 smithi098 ceph-mon[103124]: pgmap v15315: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 381 KiB/s wr, 31 op/s 2023-09-10T07:13:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:07 smithi170 ceph-mon[102832]: pgmap v15316: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 337 KiB/s wr, 30 op/s 2023-09-10T07:13:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:07 smithi082 ceph-mon[98694]: pgmap v15316: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 337 KiB/s wr, 30 op/s 2023-09-10T07:13:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:07 smithi098 ceph-mon[103124]: pgmap v15316: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 337 KiB/s wr, 30 op/s 2023-09-10T07:13:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28419.501530 secs 2023-09-10T07:13:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28419.501530 secs 2023-09-10T07:13:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28419.501530 secs 2023-09-10T07:13:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28420.639571 secs 2023-09-10T07:13:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:09 smithi170 ceph-mon[102832]: pgmap v15317: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 337 KiB/s wr, 30 op/s 2023-09-10T07:13:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28420.639571 secs 2023-09-10T07:13:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:09 smithi082 ceph-mon[98694]: pgmap v15317: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 337 KiB/s wr, 30 op/s 2023-09-10T07:13:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28420.639571 secs 2023-09-10T07:13:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:09 smithi098 ceph-mon[103124]: pgmap v15317: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 337 KiB/s wr, 30 op/s 2023-09-10T07:13:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28421.116982 secs 2023-09-10T07:13:10.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28421.116982 secs 2023-09-10T07:13:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28421.116982 secs 2023-09-10T07:13:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:11 smithi170 ceph-mon[102832]: pgmap v15318: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 304 KiB/s wr, 13 op/s 2023-09-10T07:13:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:11 smithi170 ceph-mon[102832]: osdmap e1026: 12 total, 12 up, 12 in 2023-09-10T07:13:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:11 smithi082 ceph-mon[98694]: pgmap v15318: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 304 KiB/s wr, 13 op/s 2023-09-10T07:13:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:11 smithi082 ceph-mon[98694]: osdmap e1026: 12 total, 12 up, 12 in 2023-09-10T07:13:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:11 smithi098 ceph-mon[103124]: pgmap v15318: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 304 KiB/s wr, 13 op/s 2023-09-10T07:13:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:11 smithi098 ceph-mon[103124]: osdmap e1026: 12 total, 12 up, 12 in 2023-09-10T07:13:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:13 smithi170 ceph-mon[102832]: pgmap v15320: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 340 KiB/s wr, 15 op/s 2023-09-10T07:13:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28424.501673 secs 2023-09-10T07:13:14.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:13 smithi082 ceph-mon[98694]: pgmap v15320: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 340 KiB/s wr, 15 op/s 2023-09-10T07:13:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28424.501673 secs 2023-09-10T07:13:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:13 smithi098 ceph-mon[103124]: pgmap v15320: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 340 KiB/s wr, 15 op/s 2023-09-10T07:13:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28424.501673 secs 2023-09-10T07:13:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28425.639655 secs 2023-09-10T07:13:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28426.117050 secs 2023-09-10T07:13:15.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28425.639655 secs 2023-09-10T07:13:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28426.117050 secs 2023-09-10T07:13:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28425.639655 secs 2023-09-10T07:13:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28426.117050 secs 2023-09-10T07:13:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:15 smithi170 ceph-mon[102832]: pgmap v15321: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 304 KiB/s wr, 13 op/s 2023-09-10T07:13:16.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:15 smithi082 ceph-mon[98694]: pgmap v15321: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 304 KiB/s wr, 13 op/s 2023-09-10T07:13:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:15 smithi098 ceph-mon[103124]: pgmap v15321: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 304 KiB/s wr, 13 op/s 2023-09-10T07:13:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:17 smithi170 ceph-mon[102832]: pgmap v15322: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T07:13:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:17 smithi082 ceph-mon[98694]: pgmap v15322: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T07:13:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:17 smithi098 ceph-mon[103124]: pgmap v15322: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T07:13:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28429.501737 secs 2023-09-10T07:13:18.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28429.501737 secs 2023-09-10T07:13:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28429.501737 secs 2023-09-10T07:13:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28430.639713 secs 2023-09-10T07:13:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:19 smithi170 ceph-mon[102832]: pgmap v15323: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T07:13:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28430.639713 secs 2023-09-10T07:13:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:19 smithi082 ceph-mon[98694]: pgmap v15323: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T07:13:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28430.639713 secs 2023-09-10T07:13:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:19 smithi098 ceph-mon[103124]: pgmap v15323: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T07:13:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28431.117116 secs 2023-09-10T07:13:20.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28431.117116 secs 2023-09-10T07:13:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28431.117116 secs 2023-09-10T07:13:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:21 smithi170 ceph-mon[102832]: pgmap v15324: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T07:13:21.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:21 smithi082 ceph-mon[98694]: pgmap v15324: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T07:13:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:21 smithi098 ceph-mon[103124]: pgmap v15324: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T07:13:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:23 smithi170 ceph-mon[102832]: pgmap v15325: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 11 op/s 2023-09-10T07:13:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28434.501787 secs 2023-09-10T07:13:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:23 smithi082 ceph-mon[98694]: pgmap v15325: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 11 op/s 2023-09-10T07:13:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28434.501787 secs 2023-09-10T07:13:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:23 smithi098 ceph-mon[103124]: pgmap v15325: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 11 op/s 2023-09-10T07:13:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28434.501787 secs 2023-09-10T07:13:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28435.639772 secs 2023-09-10T07:13:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28436.117179 secs 2023-09-10T07:13:25.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28435.639772 secs 2023-09-10T07:13:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28436.117179 secs 2023-09-10T07:13:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28435.639772 secs 2023-09-10T07:13:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28436.117179 secs 2023-09-10T07:13:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:25 smithi170 ceph-mon[102832]: pgmap v15326: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 10 op/s 2023-09-10T07:13:26.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:25 smithi082 ceph-mon[98694]: pgmap v15326: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 10 op/s 2023-09-10T07:13:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:25 smithi098 ceph-mon[103124]: pgmap v15326: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 10 op/s 2023-09-10T07:13:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:27 smithi170 ceph-mon[102832]: pgmap v15327: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 100 KiB/s wr, 15 op/s 2023-09-10T07:13:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:27 smithi082 ceph-mon[98694]: pgmap v15327: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 100 KiB/s wr, 15 op/s 2023-09-10T07:13:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:27 smithi098 ceph-mon[103124]: pgmap v15327: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 100 KiB/s wr, 15 op/s 2023-09-10T07:13:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28439.501874 secs 2023-09-10T07:13:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28439.501874 secs 2023-09-10T07:13:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28439.501874 secs 2023-09-10T07:13:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28440.639841 secs 2023-09-10T07:13:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:29 smithi170 ceph-mon[102832]: pgmap v15328: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 10 op/s 2023-09-10T07:13:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28440.639841 secs 2023-09-10T07:13:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:29 smithi082 ceph-mon[98694]: pgmap v15328: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 10 op/s 2023-09-10T07:13:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28440.639841 secs 2023-09-10T07:13:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:29 smithi098 ceph-mon[103124]: pgmap v15328: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 10 op/s 2023-09-10T07:13:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28441.117245 secs 2023-09-10T07:13:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:13:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:30.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28441.117245 secs 2023-09-10T07:13:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:13:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28441.117245 secs 2023-09-10T07:13:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:13:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:31 smithi170 ceph-mon[102832]: pgmap v15329: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 196 KiB/s wr, 15 op/s 2023-09-10T07:13:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:31 smithi082 ceph-mon[98694]: pgmap v15329: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 196 KiB/s wr, 15 op/s 2023-09-10T07:13:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:31 smithi098 ceph-mon[103124]: pgmap v15329: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 196 KiB/s wr, 15 op/s 2023-09-10T07:13:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:13:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:13:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:13:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:33 smithi170 ceph-mon[102832]: pgmap v15330: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 10 op/s 2023-09-10T07:13:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28444.501933 secs 2023-09-10T07:13:33.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:33 smithi082 ceph-mon[98694]: pgmap v15330: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 10 op/s 2023-09-10T07:13:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28444.501933 secs 2023-09-10T07:13:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:33 smithi098 ceph-mon[103124]: pgmap v15330: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 10 op/s 2023-09-10T07:13:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28444.501933 secs 2023-09-10T07:13:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28445.639898 secs 2023-09-10T07:13:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28445.639898 secs 2023-09-10T07:13:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28445.639898 secs 2023-09-10T07:13:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28446.117309 secs 2023-09-10T07:13:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:35 smithi170 ceph-mon[102832]: pgmap v15331: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 10 op/s 2023-09-10T07:13:35.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28446.117309 secs 2023-09-10T07:13:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:35 smithi082 ceph-mon[98694]: pgmap v15331: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 10 op/s 2023-09-10T07:13:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28446.117309 secs 2023-09-10T07:13:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:35 smithi098 ceph-mon[103124]: pgmap v15331: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 10 op/s 2023-09-10T07:13:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:13:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:13:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:13:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:37 smithi170 ceph-mon[102832]: pgmap v15332: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 324 KiB/s wr, 16 op/s 2023-09-10T07:13:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:37 smithi082 ceph-mon[98694]: pgmap v15332: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 324 KiB/s wr, 16 op/s 2023-09-10T07:13:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:37 smithi098 ceph-mon[103124]: pgmap v15332: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 324 KiB/s wr, 16 op/s 2023-09-10T07:13:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:13:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:13:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:13:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28450.639965 secs 2023-09-10T07:13:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:39 smithi170 ceph-mon[102832]: pgmap v15333: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 323 KiB/s wr, 11 op/s 2023-09-10T07:13:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:13:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28450.639965 secs 2023-09-10T07:13:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:39 smithi082 ceph-mon[98694]: pgmap v15333: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 323 KiB/s wr, 11 op/s 2023-09-10T07:13:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:13:40.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28450.639965 secs 2023-09-10T07:13:40.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:39 smithi098 ceph-mon[103124]: pgmap v15333: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 323 KiB/s wr, 11 op/s 2023-09-10T07:13:40.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:13:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28451.117399 secs 2023-09-10T07:13:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:13:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:13:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:40.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28451.117399 secs 2023-09-10T07:13:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:13:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:13:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28451.117399 secs 2023-09-10T07:13:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:13:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:13:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:41 smithi170 ceph-mon[102832]: pgmap v15334: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 536 KiB/s wr, 17 op/s 2023-09-10T07:13:41.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:41 smithi082 ceph-mon[98694]: pgmap v15334: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 536 KiB/s wr, 17 op/s 2023-09-10T07:13:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:41 smithi098 ceph-mon[103124]: pgmap v15334: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 536 KiB/s wr, 17 op/s 2023-09-10T07:13:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:43 smithi170 ceph-mon[102832]: pgmap v15335: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 440 KiB/s wr, 12 op/s 2023-09-10T07:13:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:43 smithi082 ceph-mon[98694]: pgmap v15335: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 440 KiB/s wr, 12 op/s 2023-09-10T07:13:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:43 smithi098 ceph-mon[103124]: pgmap v15335: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 440 KiB/s wr, 12 op/s 2023-09-10T07:13:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28455.640022 secs 2023-09-10T07:13:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:13:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28456.117465 secs 2023-09-10T07:13:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28455.640022 secs 2023-09-10T07:13:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:13:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28456.117465 secs 2023-09-10T07:13:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28455.640022 secs 2023-09-10T07:13:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:13:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28456.117465 secs 2023-09-10T07:13:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:45 smithi170 ceph-mon[102832]: pgmap v15336: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 440 KiB/s wr, 12 op/s 2023-09-10T07:13:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:13:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:46.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:45 smithi082 ceph-mon[98694]: pgmap v15336: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 440 KiB/s wr, 12 op/s 2023-09-10T07:13:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:13:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:45 smithi098 ceph-mon[103124]: pgmap v15336: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 440 KiB/s wr, 12 op/s 2023-09-10T07:13:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:13:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:47 smithi170 ceph-mon[102832]: pgmap v15337: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 630 KiB/s wr, 19 op/s 2023-09-10T07:13:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:47 smithi082 ceph-mon[98694]: pgmap v15337: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 630 KiB/s wr, 19 op/s 2023-09-10T07:13:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:47 smithi098 ceph-mon[103124]: pgmap v15337: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 630 KiB/s wr, 19 op/s 2023-09-10T07:13:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:13:48.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:13:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:13:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28460.640107 secs 2023-09-10T07:13:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:49 smithi170 ceph-mon[102832]: pgmap v15338: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 403 KiB/s wr, 13 op/s 2023-09-10T07:13:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:13:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:13:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:13:49.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28460.640107 secs 2023-09-10T07:13:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:49 smithi082 ceph-mon[98694]: pgmap v15338: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 403 KiB/s wr, 13 op/s 2023-09-10T07:13:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:13:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:13:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:13:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28460.640107 secs 2023-09-10T07:13:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:49 smithi098 ceph-mon[103124]: pgmap v15338: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 403 KiB/s wr, 13 op/s 2023-09-10T07:13:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:13:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:13:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:13:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28461.117489 secs 2023-09-10T07:13:50.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28461.117489 secs 2023-09-10T07:13:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28461.117489 secs 2023-09-10T07:13:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:51 smithi170 ceph-mon[102832]: pgmap v15339: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 607 KiB/s wr, 19 op/s 2023-09-10T07:13:51.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:51 smithi082 ceph-mon[98694]: pgmap v15339: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 607 KiB/s wr, 19 op/s 2023-09-10T07:13:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:51 smithi098 ceph-mon[103124]: pgmap v15339: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 607 KiB/s wr, 19 op/s 2023-09-10T07:13:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:53 smithi170 ceph-mon[102832]: pgmap v15340: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 394 KiB/s wr, 13 op/s 2023-09-10T07:13:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:13:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:54.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:53 smithi082 ceph-mon[98694]: pgmap v15340: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 394 KiB/s wr, 13 op/s 2023-09-10T07:13:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:13:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:53 smithi098 ceph-mon[103124]: pgmap v15340: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 394 KiB/s wr, 13 op/s 2023-09-10T07:13:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:13:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28465.640191 secs 2023-09-10T07:13:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28466.117555 secs 2023-09-10T07:13:55.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28465.640191 secs 2023-09-10T07:13:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28466.117555 secs 2023-09-10T07:13:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28465.640191 secs 2023-09-10T07:13:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28466.117555 secs 2023-09-10T07:13:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:55 smithi170 ceph-mon[102832]: pgmap v15341: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 394 KiB/s wr, 13 op/s 2023-09-10T07:13:56.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:55 smithi082 ceph-mon[98694]: pgmap v15341: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 394 KiB/s wr, 13 op/s 2023-09-10T07:13:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:55 smithi098 ceph-mon[103124]: pgmap v15341: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 394 KiB/s wr, 13 op/s 2023-09-10T07:13:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:13:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:57.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:13:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:13:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:13:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:57 smithi170 ceph-mon[102832]: pgmap v15342: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 586 KiB/s wr, 19 op/s 2023-09-10T07:13:58.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:57 smithi082 ceph-mon[98694]: pgmap v15342: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 586 KiB/s wr, 19 op/s 2023-09-10T07:13:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:57 smithi098 ceph-mon[103124]: pgmap v15342: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 586 KiB/s wr, 19 op/s 2023-09-10T07:13:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28469.502248 secs 2023-09-10T07:13:59.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28469.502248 secs 2023-09-10T07:13:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28469.502248 secs 2023-09-10T07:14:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28470.640275 secs 2023-09-10T07:14:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:59 smithi170 ceph-mon[102832]: pgmap v15343: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 396 KiB/s wr, 12 op/s 2023-09-10T07:14:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:13:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28471.117620 secs 2023-09-10T07:14:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28470.640275 secs 2023-09-10T07:14:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:59 smithi082 ceph-mon[98694]: pgmap v15343: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 396 KiB/s wr, 12 op/s 2023-09-10T07:14:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:13:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28471.117620 secs 2023-09-10T07:14:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28470.640275 secs 2023-09-10T07:14:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:59 smithi098 ceph-mon[103124]: pgmap v15343: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 396 KiB/s wr, 12 op/s 2023-09-10T07:14:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:13:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28471.117620 secs 2023-09-10T07:14:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:14:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:14:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:01.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:14:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:14:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:14:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:14:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:01 smithi170 ceph-mon[102832]: pgmap v15344: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 489 KiB/s wr, 18 op/s 2023-09-10T07:14:02.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:01 smithi082 ceph-mon[98694]: pgmap v15344: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 489 KiB/s wr, 18 op/s 2023-09-10T07:14:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:01 smithi098 ceph-mon[103124]: pgmap v15344: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 489 KiB/s wr, 18 op/s 2023-09-10T07:14:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:03 smithi170 ceph-mon[102832]: pgmap v15345: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T07:14:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28474.502357 secs 2023-09-10T07:14:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:03 smithi082 ceph-mon[98694]: pgmap v15345: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T07:14:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28474.502357 secs 2023-09-10T07:14:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:03 smithi098 ceph-mon[103124]: pgmap v15345: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T07:14:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28474.502357 secs 2023-09-10T07:14:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28475.640358 secs 2023-09-10T07:14:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28476.117683 secs 2023-09-10T07:14:05.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28475.640358 secs 2023-09-10T07:14:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28476.117683 secs 2023-09-10T07:14:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28475.640358 secs 2023-09-10T07:14:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28476.117683 secs 2023-09-10T07:14:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:05 smithi170 ceph-mon[102832]: pgmap v15346: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T07:14:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:05 smithi082 ceph-mon[98694]: pgmap v15346: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T07:14:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:05 smithi098 ceph-mon[103124]: pgmap v15346: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 285 KiB/s wr, 12 op/s 2023-09-10T07:14:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:07 smithi170 ceph-mon[102832]: pgmap v15347: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 473 KiB/s wr, 32 op/s 2023-09-10T07:14:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:07 smithi082 ceph-mon[98694]: pgmap v15347: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 473 KiB/s wr, 32 op/s 2023-09-10T07:14:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:07 smithi098 ceph-mon[103124]: pgmap v15347: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 473 KiB/s wr, 32 op/s 2023-09-10T07:14:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28479.502463 secs 2023-09-10T07:14:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28479.502463 secs 2023-09-10T07:14:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28479.502463 secs 2023-09-10T07:14:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28480.640420 secs 2023-09-10T07:14:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:09 smithi170 ceph-mon[102832]: osdmap e1027: 12 total, 12 up, 12 in 2023-09-10T07:14:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:09 smithi170 ceph-mon[102832]: pgmap v15349: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 338 KiB/s wr, 31 op/s 2023-09-10T07:14:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28480.640420 secs 2023-09-10T07:14:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:09 smithi082 ceph-mon[98694]: osdmap e1027: 12 total, 12 up, 12 in 2023-09-10T07:14:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:09 smithi082 ceph-mon[98694]: pgmap v15349: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 338 KiB/s wr, 31 op/s 2023-09-10T07:14:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28480.640420 secs 2023-09-10T07:14:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:09 smithi098 ceph-mon[103124]: osdmap e1027: 12 total, 12 up, 12 in 2023-09-10T07:14:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:09 smithi098 ceph-mon[103124]: pgmap v15349: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 338 KiB/s wr, 31 op/s 2023-09-10T07:14:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28481.117745 secs 2023-09-10T07:14:10.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28481.117745 secs 2023-09-10T07:14:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28481.117745 secs 2023-09-10T07:14:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:11 smithi170 ceph-mon[102832]: pgmap v15350: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 320 KiB/s wr, 29 op/s 2023-09-10T07:14:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:11 smithi082 ceph-mon[98694]: pgmap v15350: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 320 KiB/s wr, 29 op/s 2023-09-10T07:14:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:11 smithi098 ceph-mon[103124]: pgmap v15350: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 320 KiB/s wr, 29 op/s 2023-09-10T07:14:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:13 smithi170 ceph-mon[102832]: pgmap v15351: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 320 KiB/s wr, 29 op/s 2023-09-10T07:14:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28484.502521 secs 2023-09-10T07:14:14.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:13 smithi082 ceph-mon[98694]: pgmap v15351: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 320 KiB/s wr, 29 op/s 2023-09-10T07:14:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28484.502521 secs 2023-09-10T07:14:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:13 smithi098 ceph-mon[103124]: pgmap v15351: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 320 KiB/s wr, 29 op/s 2023-09-10T07:14:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28484.502521 secs 2023-09-10T07:14:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28485.640475 secs 2023-09-10T07:14:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28486.117813 secs 2023-09-10T07:14:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28485.640475 secs 2023-09-10T07:14:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28486.117813 secs 2023-09-10T07:14:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28485.640475 secs 2023-09-10T07:14:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28486.117813 secs 2023-09-10T07:14:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:15 smithi170 ceph-mon[102832]: pgmap v15352: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 320 KiB/s wr, 29 op/s 2023-09-10T07:14:16.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:15 smithi082 ceph-mon[98694]: pgmap v15352: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 320 KiB/s wr, 29 op/s 2023-09-10T07:14:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:15 smithi098 ceph-mon[103124]: pgmap v15352: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 320 KiB/s wr, 29 op/s 2023-09-10T07:14:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:17 smithi170 ceph-mon[102832]: osdmap e1028: 12 total, 12 up, 12 in 2023-09-10T07:14:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:17 smithi170 ceph-mon[102832]: pgmap v15354: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 116 KiB/s wr, 14 op/s 2023-09-10T07:14:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:17 smithi082 ceph-mon[98694]: osdmap e1028: 12 total, 12 up, 12 in 2023-09-10T07:14:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:17 smithi082 ceph-mon[98694]: pgmap v15354: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 116 KiB/s wr, 14 op/s 2023-09-10T07:14:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:17 smithi098 ceph-mon[103124]: osdmap e1028: 12 total, 12 up, 12 in 2023-09-10T07:14:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:17 smithi098 ceph-mon[103124]: pgmap v15354: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 116 KiB/s wr, 14 op/s 2023-09-10T07:14:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28489.502584 secs 2023-09-10T07:14:18.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28489.502584 secs 2023-09-10T07:14:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28489.502584 secs 2023-09-10T07:14:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28490.640494 secs 2023-09-10T07:14:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:19 smithi170 ceph-mon[102832]: pgmap v15355: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T07:14:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28490.640494 secs 2023-09-10T07:14:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:19 smithi082 ceph-mon[98694]: pgmap v15355: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T07:14:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28490.640494 secs 2023-09-10T07:14:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:19 smithi098 ceph-mon[103124]: pgmap v15355: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T07:14:20.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28491.117875 secs 2023-09-10T07:14:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28491.117875 secs 2023-09-10T07:14:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28491.117875 secs 2023-09-10T07:14:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:21 smithi170 ceph-mon[102832]: pgmap v15356: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T07:14:21.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:21 smithi082 ceph-mon[98694]: pgmap v15356: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T07:14:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:21 smithi098 ceph-mon[103124]: pgmap v15356: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T07:14:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:23 smithi170 ceph-mon[102832]: pgmap v15357: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T07:14:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28494.502643 secs 2023-09-10T07:14:24.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:23 smithi082 ceph-mon[98694]: pgmap v15357: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T07:14:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28494.502643 secs 2023-09-10T07:14:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:23 smithi098 ceph-mon[103124]: pgmap v15357: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T07:14:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28494.502643 secs 2023-09-10T07:14:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28495.640565 secs 2023-09-10T07:14:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28496.117941 secs 2023-09-10T07:14:25.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28495.640565 secs 2023-09-10T07:14:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28496.117941 secs 2023-09-10T07:14:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28495.640565 secs 2023-09-10T07:14:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28496.117941 secs 2023-09-10T07:14:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:25 smithi170 ceph-mon[102832]: pgmap v15358: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T07:14:26.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:25 smithi082 ceph-mon[98694]: pgmap v15358: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T07:14:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:25 smithi098 ceph-mon[103124]: pgmap v15358: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T07:14:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:27 smithi170 ceph-mon[102832]: pgmap v15359: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T07:14:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:27 smithi082 ceph-mon[98694]: pgmap v15359: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T07:14:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:27 smithi098 ceph-mon[103124]: pgmap v15359: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T07:14:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28499.502704 secs 2023-09-10T07:14:28.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28499.502704 secs 2023-09-10T07:14:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28499.502704 secs 2023-09-10T07:14:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28500.640629 secs 2023-09-10T07:14:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:29 smithi170 ceph-mon[102832]: pgmap v15360: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 10 op/s 2023-09-10T07:14:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28500.640629 secs 2023-09-10T07:14:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:29 smithi082 ceph-mon[98694]: pgmap v15360: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 10 op/s 2023-09-10T07:14:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28500.640629 secs 2023-09-10T07:14:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:29 smithi098 ceph-mon[103124]: pgmap v15360: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 10 op/s 2023-09-10T07:14:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28501.118005 secs 2023-09-10T07:14:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:14:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:30.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28501.118005 secs 2023-09-10T07:14:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:14:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28501.118005 secs 2023-09-10T07:14:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:14:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:31 smithi170 ceph-mon[102832]: pgmap v15361: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 197 KiB/s wr, 15 op/s 2023-09-10T07:14:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:31 smithi082 ceph-mon[98694]: pgmap v15361: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 197 KiB/s wr, 15 op/s 2023-09-10T07:14:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:31 smithi098 ceph-mon[103124]: pgmap v15361: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 197 KiB/s wr, 15 op/s 2023-09-10T07:14:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:14:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:14:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:14:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:33 smithi170 ceph-mon[102832]: pgmap v15362: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 9 op/s 2023-09-10T07:14:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28504.502770 secs 2023-09-10T07:14:33.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:33 smithi082 ceph-mon[98694]: pgmap v15362: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 9 op/s 2023-09-10T07:14:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28504.502770 secs 2023-09-10T07:14:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:33 smithi098 ceph-mon[103124]: pgmap v15362: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 9 op/s 2023-09-10T07:14:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28504.502770 secs 2023-09-10T07:14:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28505.640687 secs 2023-09-10T07:14:34.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28505.640687 secs 2023-09-10T07:14:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28505.640687 secs 2023-09-10T07:14:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28506.118069 secs 2023-09-10T07:14:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:35 smithi170 ceph-mon[102832]: pgmap v15363: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 9 op/s 2023-09-10T07:14:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28506.118069 secs 2023-09-10T07:14:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:35 smithi082 ceph-mon[98694]: pgmap v15363: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 9 op/s 2023-09-10T07:14:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28506.118069 secs 2023-09-10T07:14:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:35 smithi098 ceph-mon[103124]: pgmap v15363: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 9 op/s 2023-09-10T07:14:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:14:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:36.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:14:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:14:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:37 smithi170 ceph-mon[102832]: pgmap v15364: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 326 KiB/s wr, 15 op/s 2023-09-10T07:14:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:37 smithi082 ceph-mon[98694]: pgmap v15364: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 326 KiB/s wr, 15 op/s 2023-09-10T07:14:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:37 smithi098 ceph-mon[103124]: pgmap v15364: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 326 KiB/s wr, 15 op/s 2023-09-10T07:14:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:14:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:38.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:14:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:14:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28510.640769 secs 2023-09-10T07:14:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:39 smithi170 ceph-mon[102832]: pgmap v15365: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 326 KiB/s wr, 10 op/s 2023-09-10T07:14:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:14:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28510.640769 secs 2023-09-10T07:14:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:39 smithi082 ceph-mon[98694]: pgmap v15365: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 326 KiB/s wr, 10 op/s 2023-09-10T07:14:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:14:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28510.640769 secs 2023-09-10T07:14:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:39 smithi098 ceph-mon[103124]: pgmap v15365: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 326 KiB/s wr, 10 op/s 2023-09-10T07:14:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:14:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28511.118133 secs 2023-09-10T07:14:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:14:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:14:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28511.118133 secs 2023-09-10T07:14:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:14:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:14:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28511.118133 secs 2023-09-10T07:14:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:14:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:14:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:41 smithi170 ceph-mon[102832]: pgmap v15366: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 539 KiB/s wr, 17 op/s 2023-09-10T07:14:41.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:41 smithi082 ceph-mon[98694]: pgmap v15366: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 539 KiB/s wr, 17 op/s 2023-09-10T07:14:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:41 smithi098 ceph-mon[103124]: pgmap v15366: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 539 KiB/s wr, 17 op/s 2023-09-10T07:14:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:43 smithi170 ceph-mon[102832]: pgmap v15367: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 442 KiB/s wr, 12 op/s 2023-09-10T07:14:44.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:43 smithi082 ceph-mon[98694]: pgmap v15367: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 442 KiB/s wr, 12 op/s 2023-09-10T07:14:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:43 smithi098 ceph-mon[103124]: pgmap v15367: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 442 KiB/s wr, 12 op/s 2023-09-10T07:14:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28515.640828 secs 2023-09-10T07:14:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:14:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28516.118195 secs 2023-09-10T07:14:45.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28515.640828 secs 2023-09-10T07:14:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:14:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28516.118195 secs 2023-09-10T07:14:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28515.640828 secs 2023-09-10T07:14:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:14:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28516.118195 secs 2023-09-10T07:14:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:45 smithi170 ceph-mon[102832]: pgmap v15368: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 442 KiB/s wr, 12 op/s 2023-09-10T07:14:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:14:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:45 smithi082 ceph-mon[98694]: pgmap v15368: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 442 KiB/s wr, 12 op/s 2023-09-10T07:14:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:14:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:45 smithi098 ceph-mon[103124]: pgmap v15368: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 442 KiB/s wr, 12 op/s 2023-09-10T07:14:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:14:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:47 smithi170 ceph-mon[102832]: pgmap v15369: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 633 KiB/s wr, 18 op/s 2023-09-10T07:14:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:47 smithi082 ceph-mon[98694]: pgmap v15369: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 633 KiB/s wr, 18 op/s 2023-09-10T07:14:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:47 smithi098 ceph-mon[103124]: pgmap v15369: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 633 KiB/s wr, 18 op/s 2023-09-10T07:14:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28520.640884 secs 2023-09-10T07:14:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:49 smithi170 ceph-mon[102832]: pgmap v15370: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 405 KiB/s wr, 13 op/s 2023-09-10T07:14:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28521.118276 secs 2023-09-10T07:14:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:14:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28520.640884 secs 2023-09-10T07:14:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:49 smithi082 ceph-mon[98694]: pgmap v15370: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 405 KiB/s wr, 13 op/s 2023-09-10T07:14:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28521.118276 secs 2023-09-10T07:14:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:14:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28520.640884 secs 2023-09-10T07:14:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:49 smithi098 ceph-mon[103124]: pgmap v15370: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 405 KiB/s wr, 13 op/s 2023-09-10T07:14:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28521.118276 secs 2023-09-10T07:14:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:14:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:51 smithi170 ceph-mon[102832]: pgmap v15371: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 610 KiB/s wr, 19 op/s 2023-09-10T07:14:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:14:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:14:52.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:51 smithi082 ceph-mon[98694]: pgmap v15371: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 610 KiB/s wr, 19 op/s 2023-09-10T07:14:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:14:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:14:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:51 smithi098 ceph-mon[103124]: pgmap v15371: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 610 KiB/s wr, 19 op/s 2023-09-10T07:14:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:14:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:14:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:14:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:14:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:14:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:14:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:14:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:14:53.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:14:53.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:14:53.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:14:53.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:14:53.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:14:53.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:14:53.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:14:53.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:14:53.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:14:53.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:14:53.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:14:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:14:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:14:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:14:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:14:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:14:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:14:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:14:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:14:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:14:53.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:14:53.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:14:53.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:14:53.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:14:53.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:14:53.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:14:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:14:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:14:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:14:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:14:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:14:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:14:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:14:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:14:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:14:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:14:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:14:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:14:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:14:53.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:14:53.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:14:53.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:14:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:53 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T07:14:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:53 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T07:14:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:53 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T07:14:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:53 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T07:14:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:53 smithi170 ceph-mon[102832]: pgmap v15372: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 397 KiB/s wr, 12 op/s 2023-09-10T07:14:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:14:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:54.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:53 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T07:14:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:53 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T07:14:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:53 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T07:14:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:53 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T07:14:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:53 smithi082 ceph-mon[98694]: pgmap v15372: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 397 KiB/s wr, 12 op/s 2023-09-10T07:14:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:14:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:53 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T07:14:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:53 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T07:14:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:53 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T07:14:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:53 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T07:14:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:53 smithi098 ceph-mon[103124]: pgmap v15372: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 397 KiB/s wr, 12 op/s 2023-09-10T07:14:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:14:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28525.640943 secs 2023-09-10T07:14:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28526.118335 secs 2023-09-10T07:14:55.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28525.640943 secs 2023-09-10T07:14:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28526.118335 secs 2023-09-10T07:14:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28525.640943 secs 2023-09-10T07:14:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28526.118335 secs 2023-09-10T07:14:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:55 smithi170 ceph-mon[102832]: pgmap v15373: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 397 KiB/s wr, 12 op/s 2023-09-10T07:14:56.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:55 smithi082 ceph-mon[98694]: pgmap v15373: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 397 KiB/s wr, 12 op/s 2023-09-10T07:14:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:55 smithi098 ceph-mon[103124]: pgmap v15373: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 397 KiB/s wr, 12 op/s 2023-09-10T07:14:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:14:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:14:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:14:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:14:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:57 smithi082 ceph-mon[98694]: pgmap v15374: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 589 KiB/s wr, 19 op/s 2023-09-10T07:14:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:57 smithi098 ceph-mon[103124]: pgmap v15374: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 589 KiB/s wr, 19 op/s 2023-09-10T07:14:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:57 smithi170 ceph-mon[102832]: pgmap v15374: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 589 KiB/s wr, 19 op/s 2023-09-10T07:14:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28529.503102 secs 2023-09-10T07:14:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28529.503102 secs 2023-09-10T07:14:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28529.503102 secs 2023-09-10T07:15:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28530.640997 secs 2023-09-10T07:15:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:59 smithi082 ceph-mon[98694]: pgmap v15375: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 398 KiB/s wr, 12 op/s 2023-09-10T07:15:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:14:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28531.118403 secs 2023-09-10T07:15:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28530.640997 secs 2023-09-10T07:15:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:59 smithi098 ceph-mon[103124]: pgmap v15375: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 398 KiB/s wr, 12 op/s 2023-09-10T07:15:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:14:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28531.118403 secs 2023-09-10T07:15:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28530.640997 secs 2023-09-10T07:15:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:59 smithi170 ceph-mon[102832]: pgmap v15375: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 398 KiB/s wr, 12 op/s 2023-09-10T07:15:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:14:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28531.118403 secs 2023-09-10T07:15:01.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:15:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:15:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:15:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:15:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:15:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:15:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:15:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:15:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:15:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:01 smithi082 ceph-mon[98694]: pgmap v15376: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 537 KiB/s wr, 18 op/s 2023-09-10T07:15:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:01 smithi098 ceph-mon[103124]: pgmap v15376: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 537 KiB/s wr, 18 op/s 2023-09-10T07:15:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:01 smithi170 ceph-mon[102832]: pgmap v15376: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 537 KiB/s wr, 18 op/s 2023-09-10T07:15:04.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:03 smithi082 ceph-mon[98694]: pgmap v15377: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T07:15:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28534.503164 secs 2023-09-10T07:15:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:03 smithi098 ceph-mon[103124]: pgmap v15377: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T07:15:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28534.503164 secs 2023-09-10T07:15:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:03 smithi170 ceph-mon[102832]: pgmap v15377: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T07:15:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28534.503164 secs 2023-09-10T07:15:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28535.641069 secs 2023-09-10T07:15:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28536.118463 secs 2023-09-10T07:15:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:15:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:15:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28535.641069 secs 2023-09-10T07:15:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28536.118463 secs 2023-09-10T07:15:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:15:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:15:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28535.641069 secs 2023-09-10T07:15:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28536.118463 secs 2023-09-10T07:15:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:15:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:15:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:05 smithi098 ceph-mon[103124]: pgmap v15378: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T07:15:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:15:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:15:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:05 smithi170 ceph-mon[102832]: pgmap v15378: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T07:15:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:15:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:15:06.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:05 smithi082 ceph-mon[98694]: pgmap v15378: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T07:15:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:15:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:15:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:07 smithi170 ceph-mon[102832]: pgmap v15379: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 520 KiB/s wr, 33 op/s 2023-09-10T07:15:07.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:07 smithi082 ceph-mon[98694]: pgmap v15379: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 520 KiB/s wr, 33 op/s 2023-09-10T07:15:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:07 smithi098 ceph-mon[103124]: pgmap v15379: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 520 KiB/s wr, 33 op/s 2023-09-10T07:15:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28539.503303 secs 2023-09-10T07:15:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28539.503303 secs 2023-09-10T07:15:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28539.503303 secs 2023-09-10T07:15:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28540.641153 secs 2023-09-10T07:15:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:09 smithi170 ceph-mon[102832]: osdmap e1029: 12 total, 12 up, 12 in 2023-09-10T07:15:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:09 smithi170 ceph-mon[102832]: pgmap v15381: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 394 KiB/s wr, 31 op/s 2023-09-10T07:15:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28540.641153 secs 2023-09-10T07:15:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:09 smithi082 ceph-mon[98694]: osdmap e1029: 12 total, 12 up, 12 in 2023-09-10T07:15:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:09 smithi082 ceph-mon[98694]: pgmap v15381: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 394 KiB/s wr, 31 op/s 2023-09-10T07:15:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28540.641153 secs 2023-09-10T07:15:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:09 smithi098 ceph-mon[103124]: osdmap e1029: 12 total, 12 up, 12 in 2023-09-10T07:15:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:09 smithi098 ceph-mon[103124]: pgmap v15381: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 394 KiB/s wr, 31 op/s 2023-09-10T07:15:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28541.118485 secs 2023-09-10T07:15:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28541.118485 secs 2023-09-10T07:15:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28541.118485 secs 2023-09-10T07:15:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:11 smithi170 ceph-mon[102832]: pgmap v15382: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 320 KiB/s wr, 30 op/s 2023-09-10T07:15:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:11 smithi082 ceph-mon[98694]: pgmap v15382: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 320 KiB/s wr, 30 op/s 2023-09-10T07:15:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:11 smithi098 ceph-mon[103124]: pgmap v15382: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 320 KiB/s wr, 30 op/s 2023-09-10T07:15:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:13 smithi170 ceph-mon[102832]: pgmap v15383: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 320 KiB/s wr, 30 op/s 2023-09-10T07:15:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28544.503364 secs 2023-09-10T07:15:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:13 smithi082 ceph-mon[98694]: pgmap v15383: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 320 KiB/s wr, 30 op/s 2023-09-10T07:15:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28544.503364 secs 2023-09-10T07:15:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:13 smithi098 ceph-mon[103124]: pgmap v15383: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 320 KiB/s wr, 30 op/s 2023-09-10T07:15:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28544.503364 secs 2023-09-10T07:15:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28545.641211 secs 2023-09-10T07:15:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28546.118550 secs 2023-09-10T07:15:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28545.641211 secs 2023-09-10T07:15:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28546.118550 secs 2023-09-10T07:15:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28545.641211 secs 2023-09-10T07:15:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28546.118550 secs 2023-09-10T07:15:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:15 smithi170 ceph-mon[102832]: pgmap v15384: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 320 KiB/s wr, 30 op/s 2023-09-10T07:15:16.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:15 smithi082 ceph-mon[98694]: pgmap v15384: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 320 KiB/s wr, 30 op/s 2023-09-10T07:15:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:15 smithi098 ceph-mon[103124]: pgmap v15384: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 320 KiB/s wr, 30 op/s 2023-09-10T07:15:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:17 smithi170 ceph-mon[102832]: osdmap e1030: 12 total, 12 up, 12 in 2023-09-10T07:15:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:17 smithi170 ceph-mon[102832]: pgmap v15386: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 15 op/s 2023-09-10T07:15:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:17 smithi082 ceph-mon[98694]: osdmap e1030: 12 total, 12 up, 12 in 2023-09-10T07:15:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:17 smithi082 ceph-mon[98694]: pgmap v15386: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 15 op/s 2023-09-10T07:15:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:17 smithi098 ceph-mon[103124]: osdmap e1030: 12 total, 12 up, 12 in 2023-09-10T07:15:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:17 smithi098 ceph-mon[103124]: pgmap v15386: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 15 op/s 2023-09-10T07:15:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28549.503451 secs 2023-09-10T07:15:18.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28549.503451 secs 2023-09-10T07:15:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28549.503451 secs 2023-09-10T07:15:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28550.641308 secs 2023-09-10T07:15:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:19 smithi170 ceph-mon[102832]: pgmap v15387: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T07:15:19.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28550.641308 secs 2023-09-10T07:15:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:19 smithi082 ceph-mon[98694]: pgmap v15387: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T07:15:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28550.641308 secs 2023-09-10T07:15:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:19 smithi098 ceph-mon[103124]: pgmap v15387: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T07:15:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28551.118615 secs 2023-09-10T07:15:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28551.118615 secs 2023-09-10T07:15:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28551.118615 secs 2023-09-10T07:15:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:21 smithi170 ceph-mon[102832]: pgmap v15388: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 178 KiB/s wr, 12 op/s 2023-09-10T07:15:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:21 smithi082 ceph-mon[98694]: pgmap v15388: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 178 KiB/s wr, 12 op/s 2023-09-10T07:15:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:21 smithi098 ceph-mon[103124]: pgmap v15388: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 178 KiB/s wr, 12 op/s 2023-09-10T07:15:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:23 smithi170 ceph-mon[102832]: pgmap v15389: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 178 KiB/s wr, 12 op/s 2023-09-10T07:15:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28554.503509 secs 2023-09-10T07:15:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:23 smithi082 ceph-mon[98694]: pgmap v15389: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 178 KiB/s wr, 12 op/s 2023-09-10T07:15:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28554.503509 secs 2023-09-10T07:15:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:23 smithi098 ceph-mon[103124]: pgmap v15389: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 178 KiB/s wr, 12 op/s 2023-09-10T07:15:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28554.503509 secs 2023-09-10T07:15:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28555.641367 secs 2023-09-10T07:15:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28556.118683 secs 2023-09-10T07:15:25.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28555.641367 secs 2023-09-10T07:15:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28556.118683 secs 2023-09-10T07:15:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28555.641367 secs 2023-09-10T07:15:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28556.118683 secs 2023-09-10T07:15:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:25 smithi170 ceph-mon[102832]: pgmap v15390: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 178 KiB/s wr, 12 op/s 2023-09-10T07:15:26.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:25 smithi082 ceph-mon[98694]: pgmap v15390: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 178 KiB/s wr, 12 op/s 2023-09-10T07:15:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:25 smithi098 ceph-mon[103124]: pgmap v15390: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 178 KiB/s wr, 12 op/s 2023-09-10T07:15:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:27 smithi170 ceph-mon[102832]: pgmap v15391: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 175 KiB/s wr, 11 op/s 2023-09-10T07:15:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:27 smithi082 ceph-mon[98694]: pgmap v15391: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 175 KiB/s wr, 11 op/s 2023-09-10T07:15:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:27 smithi098 ceph-mon[103124]: pgmap v15391: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 175 KiB/s wr, 11 op/s 2023-09-10T07:15:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28559.503613 secs 2023-09-10T07:15:28.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28559.503613 secs 2023-09-10T07:15:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28559.503613 secs 2023-09-10T07:15:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28560.641425 secs 2023-09-10T07:15:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:29 smithi170 ceph-mon[102832]: pgmap v15392: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 9 op/s 2023-09-10T07:15:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28560.641425 secs 2023-09-10T07:15:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:29 smithi082 ceph-mon[98694]: pgmap v15392: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 9 op/s 2023-09-10T07:15:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28560.641425 secs 2023-09-10T07:15:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:29 smithi098 ceph-mon[103124]: pgmap v15392: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 9 op/s 2023-09-10T07:15:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28561.118749 secs 2023-09-10T07:15:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28561.118749 secs 2023-09-10T07:15:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28561.118749 secs 2023-09-10T07:15:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:31 smithi170 ceph-mon[102832]: pgmap v15393: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 257 KiB/s wr, 14 op/s 2023-09-10T07:15:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:31 smithi082 ceph-mon[98694]: pgmap v15393: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 257 KiB/s wr, 14 op/s 2023-09-10T07:15:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:31 smithi098 ceph-mon[103124]: pgmap v15393: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 257 KiB/s wr, 14 op/s 2023-09-10T07:15:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:33 smithi170 ceph-mon[102832]: pgmap v15394: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 109 KiB/s wr, 9 op/s 2023-09-10T07:15:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:15:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:15:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:33 smithi082 ceph-mon[98694]: pgmap v15394: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 109 KiB/s wr, 9 op/s 2023-09-10T07:15:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:15:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:15:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:33 smithi098 ceph-mon[103124]: pgmap v15394: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 109 KiB/s wr, 9 op/s 2023-09-10T07:15:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:15:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:15:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28565.641512 secs 2023-09-10T07:15:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28566.118830 secs 2023-09-10T07:15:35.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28565.641512 secs 2023-09-10T07:15:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28566.118830 secs 2023-09-10T07:15:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28565.641512 secs 2023-09-10T07:15:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28566.118830 secs 2023-09-10T07:15:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:35 smithi170 ceph-mon[102832]: pgmap v15395: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 109 KiB/s wr, 9 op/s 2023-09-10T07:15:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:15:36.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:35 smithi082 ceph-mon[98694]: pgmap v15395: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 109 KiB/s wr, 9 op/s 2023-09-10T07:15:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:15:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:35 smithi098 ceph-mon[103124]: pgmap v15395: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 109 KiB/s wr, 9 op/s 2023-09-10T07:15:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:15:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:15:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:15:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:15:37.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:15:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:15:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:15:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:15:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:15:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:15:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:37 smithi170 ceph-mon[102832]: pgmap v15396: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 324 KiB/s wr, 15 op/s 2023-09-10T07:15:38.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:37 smithi082 ceph-mon[98694]: pgmap v15396: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 324 KiB/s wr, 15 op/s 2023-09-10T07:15:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:37 smithi098 ceph-mon[103124]: pgmap v15396: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 324 KiB/s wr, 15 op/s 2023-09-10T07:15:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28570.641573 secs 2023-09-10T07:15:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:39 smithi170 ceph-mon[102832]: pgmap v15397: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 324 KiB/s wr, 11 op/s 2023-09-10T07:15:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28571.118897 secs 2023-09-10T07:15:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28570.641573 secs 2023-09-10T07:15:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:39 smithi082 ceph-mon[98694]: pgmap v15397: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 324 KiB/s wr, 11 op/s 2023-09-10T07:15:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28571.118897 secs 2023-09-10T07:15:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28570.641573 secs 2023-09-10T07:15:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:39 smithi098 ceph-mon[103124]: pgmap v15397: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 324 KiB/s wr, 11 op/s 2023-09-10T07:15:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28571.118897 secs 2023-09-10T07:15:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:41 smithi170 ceph-mon[102832]: pgmap v15398: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 538 KiB/s wr, 17 op/s 2023-09-10T07:15:42.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:41 smithi082 ceph-mon[98694]: pgmap v15398: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 538 KiB/s wr, 17 op/s 2023-09-10T07:15:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:41 smithi098 ceph-mon[103124]: pgmap v15398: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 538 KiB/s wr, 17 op/s 2023-09-10T07:15:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:43 smithi170 ceph-mon[102832]: pgmap v15399: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 429 KiB/s wr, 12 op/s 2023-09-10T07:15:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:43 smithi082 ceph-mon[98694]: pgmap v15399: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 429 KiB/s wr, 12 op/s 2023-09-10T07:15:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:43 smithi098 ceph-mon[103124]: pgmap v15399: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 429 KiB/s wr, 12 op/s 2023-09-10T07:15:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28575.641653 secs 2023-09-10T07:15:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:15:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:15:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28576.118959 secs 2023-09-10T07:15:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28575.641653 secs 2023-09-10T07:15:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:15:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:15:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28576.118959 secs 2023-09-10T07:15:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28575.641653 secs 2023-09-10T07:15:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:15:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:15:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28576.118959 secs 2023-09-10T07:15:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:45 smithi170 ceph-mon[102832]: pgmap v15400: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 429 KiB/s wr, 12 op/s 2023-09-10T07:15:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:15:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:15:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:45 smithi082 ceph-mon[98694]: pgmap v15400: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 429 KiB/s wr, 12 op/s 2023-09-10T07:15:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:15:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:15:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:45 smithi098 ceph-mon[103124]: pgmap v15400: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 429 KiB/s wr, 12 op/s 2023-09-10T07:15:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:15:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:15:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:47 smithi170 ceph-mon[102832]: pgmap v15401: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 621 KiB/s wr, 19 op/s 2023-09-10T07:15:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:47 smithi082 ceph-mon[98694]: pgmap v15401: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 621 KiB/s wr, 19 op/s 2023-09-10T07:15:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:47 smithi098 ceph-mon[103124]: pgmap v15401: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 621 KiB/s wr, 19 op/s 2023-09-10T07:15:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28580.641711 secs 2023-09-10T07:15:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:49 smithi170 ceph-mon[102832]: pgmap v15402: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 406 KiB/s wr, 13 op/s 2023-09-10T07:15:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28581.119026 secs 2023-09-10T07:15:50.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28580.641711 secs 2023-09-10T07:15:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:49 smithi082 ceph-mon[98694]: pgmap v15402: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 406 KiB/s wr, 13 op/s 2023-09-10T07:15:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28581.119026 secs 2023-09-10T07:15:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28580.641711 secs 2023-09-10T07:15:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:49 smithi098 ceph-mon[103124]: pgmap v15402: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 406 KiB/s wr, 13 op/s 2023-09-10T07:15:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28581.119026 secs 2023-09-10T07:15:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:51 smithi170 ceph-mon[102832]: pgmap v15403: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 612 KiB/s wr, 20 op/s 2023-09-10T07:15:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:51 smithi082 ceph-mon[98694]: pgmap v15403: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 612 KiB/s wr, 20 op/s 2023-09-10T07:15:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:51 smithi098 ceph-mon[103124]: pgmap v15403: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 612 KiB/s wr, 20 op/s 2023-09-10T07:15:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:15:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:15:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:15:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:53 smithi170 ceph-mon[102832]: pgmap v15404: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 398 KiB/s wr, 13 op/s 2023-09-10T07:15:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:15:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:15:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:15:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:15:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:15:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:53 smithi082 ceph-mon[98694]: pgmap v15404: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 398 KiB/s wr, 13 op/s 2023-09-10T07:15:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:15:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:15:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:15:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:15:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:15:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:53 smithi098 ceph-mon[103124]: pgmap v15404: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 398 KiB/s wr, 13 op/s 2023-09-10T07:15:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:15:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:15:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:15:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:15:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:15:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28585.641771 secs 2023-09-10T07:15:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28586.119091 secs 2023-09-10T07:15:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28585.641771 secs 2023-09-10T07:15:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28586.119091 secs 2023-09-10T07:15:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28585.641771 secs 2023-09-10T07:15:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28586.119091 secs 2023-09-10T07:15:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:55 smithi170 ceph-mon[102832]: pgmap v15405: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 398 KiB/s wr, 13 op/s 2023-09-10T07:15:56.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:55 smithi082 ceph-mon[98694]: pgmap v15405: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 398 KiB/s wr, 13 op/s 2023-09-10T07:15:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:55 smithi098 ceph-mon[103124]: pgmap v15405: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 398 KiB/s wr, 13 op/s 2023-09-10T07:15:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:15:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:15:57.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:15:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:15:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:15:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:15:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:57 smithi170 ceph-mon[102832]: pgmap v15406: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 606 KiB/s wr, 19 op/s 2023-09-10T07:15:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:57 smithi082 ceph-mon[98694]: pgmap v15406: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 606 KiB/s wr, 19 op/s 2023-09-10T07:15:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:57 smithi098 ceph-mon[103124]: pgmap v15406: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 606 KiB/s wr, 19 op/s 2023-09-10T07:15:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28589.504042 secs 2023-09-10T07:15:59.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28589.504042 secs 2023-09-10T07:15:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28589.504042 secs 2023-09-10T07:16:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28590.641854 secs 2023-09-10T07:16:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:59 smithi170 ceph-mon[102832]: pgmap v15407: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 415 KiB/s wr, 12 op/s 2023-09-10T07:16:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:15:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28591.119156 secs 2023-09-10T07:16:00.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28590.641854 secs 2023-09-10T07:16:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:59 smithi082 ceph-mon[98694]: pgmap v15407: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 415 KiB/s wr, 12 op/s 2023-09-10T07:16:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:15:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28591.119156 secs 2023-09-10T07:16:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28590.641854 secs 2023-09-10T07:16:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:59 smithi098 ceph-mon[103124]: pgmap v15407: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 415 KiB/s wr, 12 op/s 2023-09-10T07:16:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:15:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28591.119156 secs 2023-09-10T07:16:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:16:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:16:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:16:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:16:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:16:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:16:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:16:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:16:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:16:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:01 smithi170 ceph-mon[102832]: pgmap v15408: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 509 KiB/s wr, 18 op/s 2023-09-10T07:16:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:01 smithi082 ceph-mon[98694]: pgmap v15408: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 509 KiB/s wr, 18 op/s 2023-09-10T07:16:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:01 smithi098 ceph-mon[103124]: pgmap v15408: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 509 KiB/s wr, 18 op/s 2023-09-10T07:16:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:03 smithi170 ceph-mon[102832]: pgmap v15409: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 303 KiB/s wr, 12 op/s 2023-09-10T07:16:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28594.504147 secs 2023-09-10T07:16:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:03 smithi082 ceph-mon[98694]: pgmap v15409: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 303 KiB/s wr, 12 op/s 2023-09-10T07:16:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28594.504147 secs 2023-09-10T07:16:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:03 smithi098 ceph-mon[103124]: pgmap v15409: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 303 KiB/s wr, 12 op/s 2023-09-10T07:16:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28594.504147 secs 2023-09-10T07:16:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28595.641911 secs 2023-09-10T07:16:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28596.119213 secs 2023-09-10T07:16:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:16:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:16:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28595.641911 secs 2023-09-10T07:16:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28596.119213 secs 2023-09-10T07:16:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:16:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:16:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28595.641911 secs 2023-09-10T07:16:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28596.119213 secs 2023-09-10T07:16:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:16:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:16:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:05 smithi170 ceph-mon[102832]: pgmap v15410: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 303 KiB/s wr, 12 op/s 2023-09-10T07:16:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:16:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:16:06.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:05 smithi082 ceph-mon[98694]: pgmap v15410: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 303 KiB/s wr, 12 op/s 2023-09-10T07:16:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:16:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:16:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:05 smithi098 ceph-mon[103124]: pgmap v15410: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 303 KiB/s wr, 12 op/s 2023-09-10T07:16:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:16:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:16:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:07 smithi170 ceph-mon[102832]: pgmap v15411: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 492 KiB/s wr, 32 op/s 2023-09-10T07:16:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:07 smithi082 ceph-mon[98694]: pgmap v15411: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 492 KiB/s wr, 32 op/s 2023-09-10T07:16:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:07 smithi098 ceph-mon[103124]: pgmap v15411: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 492 KiB/s wr, 32 op/s 2023-09-10T07:16:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28599.504212 secs 2023-09-10T07:16:08.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28599.504212 secs 2023-09-10T07:16:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28599.504212 secs 2023-09-10T07:16:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28600.641970 secs 2023-09-10T07:16:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:09 smithi170 ceph-mon[102832]: osdmap e1031: 12 total, 12 up, 12 in 2023-09-10T07:16:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:09 smithi170 ceph-mon[102832]: pgmap v15413: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 339 KiB/s wr, 31 op/s 2023-09-10T07:16:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28600.641970 secs 2023-09-10T07:16:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:09 smithi082 ceph-mon[98694]: osdmap e1031: 12 total, 12 up, 12 in 2023-09-10T07:16:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:09 smithi082 ceph-mon[98694]: pgmap v15413: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 339 KiB/s wr, 31 op/s 2023-09-10T07:16:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28600.641970 secs 2023-09-10T07:16:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:09 smithi098 ceph-mon[103124]: osdmap e1031: 12 total, 12 up, 12 in 2023-09-10T07:16:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:09 smithi098 ceph-mon[103124]: pgmap v15413: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 339 KiB/s wr, 31 op/s 2023-09-10T07:16:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28601.119278 secs 2023-09-10T07:16:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28601.119278 secs 2023-09-10T07:16:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28601.119278 secs 2023-09-10T07:16:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:11 smithi170 ceph-mon[102832]: pgmap v15414: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 321 KiB/s wr, 30 op/s 2023-09-10T07:16:11.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:11 smithi082 ceph-mon[98694]: pgmap v15414: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 321 KiB/s wr, 30 op/s 2023-09-10T07:16:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:11 smithi098 ceph-mon[103124]: pgmap v15414: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 321 KiB/s wr, 30 op/s 2023-09-10T07:16:14.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:13 smithi170 ceph-mon[102832]: pgmap v15415: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 321 KiB/s wr, 30 op/s 2023-09-10T07:16:14.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28604.504297 secs 2023-09-10T07:16:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:13 smithi082 ceph-mon[98694]: pgmap v15415: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 321 KiB/s wr, 30 op/s 2023-09-10T07:16:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28604.504297 secs 2023-09-10T07:16:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:13 smithi098 ceph-mon[103124]: pgmap v15415: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 321 KiB/s wr, 30 op/s 2023-09-10T07:16:14.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28604.504297 secs 2023-09-10T07:16:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28605.642030 secs 2023-09-10T07:16:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28606.119341 secs 2023-09-10T07:16:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28605.642030 secs 2023-09-10T07:16:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28606.119341 secs 2023-09-10T07:16:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28605.642030 secs 2023-09-10T07:16:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28606.119341 secs 2023-09-10T07:16:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:15 smithi170 ceph-mon[102832]: pgmap v15416: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 321 KiB/s wr, 30 op/s 2023-09-10T07:16:16.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:15 smithi082 ceph-mon[98694]: pgmap v15416: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 321 KiB/s wr, 30 op/s 2023-09-10T07:16:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:15 smithi098 ceph-mon[103124]: pgmap v15416: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 321 KiB/s wr, 30 op/s 2023-09-10T07:16:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:17 smithi170 ceph-mon[102832]: osdmap e1032: 12 total, 12 up, 12 in 2023-09-10T07:16:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:17 smithi170 ceph-mon[102832]: pgmap v15418: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 14 op/s 2023-09-10T07:16:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:17 smithi082 ceph-mon[98694]: osdmap e1032: 12 total, 12 up, 12 in 2023-09-10T07:16:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:17 smithi082 ceph-mon[98694]: pgmap v15418: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 14 op/s 2023-09-10T07:16:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:17 smithi098 ceph-mon[103124]: osdmap e1032: 12 total, 12 up, 12 in 2023-09-10T07:16:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:17 smithi098 ceph-mon[103124]: pgmap v15418: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 14 op/s 2023-09-10T07:16:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28609.504356 secs 2023-09-10T07:16:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28609.504356 secs 2023-09-10T07:16:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28609.504356 secs 2023-09-10T07:16:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28610.642088 secs 2023-09-10T07:16:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:19 smithi170 ceph-mon[102832]: pgmap v15419: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T07:16:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28610.642088 secs 2023-09-10T07:16:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:19 smithi082 ceph-mon[98694]: pgmap v15419: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T07:16:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28610.642088 secs 2023-09-10T07:16:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:19 smithi098 ceph-mon[103124]: pgmap v15419: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T07:16:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28611.119405 secs 2023-09-10T07:16:20.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28611.119405 secs 2023-09-10T07:16:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28611.119405 secs 2023-09-10T07:16:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:21 smithi170 ceph-mon[102832]: pgmap v15420: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T07:16:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:21 smithi082 ceph-mon[98694]: pgmap v15420: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T07:16:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:21 smithi098 ceph-mon[103124]: pgmap v15420: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T07:16:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:23 smithi170 ceph-mon[102832]: pgmap v15421: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T07:16:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28614.504450 secs 2023-09-10T07:16:24.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:23 smithi082 ceph-mon[98694]: pgmap v15421: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T07:16:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28614.504450 secs 2023-09-10T07:16:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:23 smithi098 ceph-mon[103124]: pgmap v15421: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T07:16:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28614.504450 secs 2023-09-10T07:16:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28615.642148 secs 2023-09-10T07:16:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28616.119477 secs 2023-09-10T07:16:25.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28615.642148 secs 2023-09-10T07:16:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28616.119477 secs 2023-09-10T07:16:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28615.642148 secs 2023-09-10T07:16:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28616.119477 secs 2023-09-10T07:16:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:25 smithi170 ceph-mon[102832]: pgmap v15422: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T07:16:26.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:25 smithi082 ceph-mon[98694]: pgmap v15422: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T07:16:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:25 smithi098 ceph-mon[103124]: pgmap v15422: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T07:16:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:27 smithi170 ceph-mon[102832]: pgmap v15423: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T07:16:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:27 smithi082 ceph-mon[98694]: pgmap v15423: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T07:16:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:27 smithi098 ceph-mon[103124]: pgmap v15423: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T07:16:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28619.504510 secs 2023-09-10T07:16:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28619.504510 secs 2023-09-10T07:16:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28619.504510 secs 2023-09-10T07:16:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28620.642207 secs 2023-09-10T07:16:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:29 smithi170 ceph-mon[102832]: pgmap v15424: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T07:16:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28620.642207 secs 2023-09-10T07:16:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:29 smithi082 ceph-mon[98694]: pgmap v15424: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T07:16:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28620.642207 secs 2023-09-10T07:16:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:29 smithi098 ceph-mon[103124]: pgmap v15424: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T07:16:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28621.119504 secs 2023-09-10T07:16:30.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28621.119504 secs 2023-09-10T07:16:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28621.119504 secs 2023-09-10T07:16:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:31 smithi170 ceph-mon[102832]: pgmap v15425: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 199 KiB/s wr, 14 op/s 2023-09-10T07:16:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:31 smithi082 ceph-mon[98694]: pgmap v15425: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 199 KiB/s wr, 14 op/s 2023-09-10T07:16:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:31 smithi098 ceph-mon[103124]: pgmap v15425: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 199 KiB/s wr, 14 op/s 2023-09-10T07:16:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:16:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:16:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:33 smithi170 ceph-mon[102832]: pgmap v15426: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:16:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28624.504620 secs 2023-09-10T07:16:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:16:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:16:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:33 smithi082 ceph-mon[98694]: pgmap v15426: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:16:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28624.504620 secs 2023-09-10T07:16:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:16:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:16:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:33 smithi098 ceph-mon[103124]: pgmap v15426: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:16:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28624.504620 secs 2023-09-10T07:16:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28625.642291 secs 2023-09-10T07:16:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28626.119570 secs 2023-09-10T07:16:35.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28625.642291 secs 2023-09-10T07:16:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28626.119570 secs 2023-09-10T07:16:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28625.642291 secs 2023-09-10T07:16:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28626.119570 secs 2023-09-10T07:16:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:35 smithi170 ceph-mon[102832]: pgmap v15427: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:16:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): Behind on trimming (843/128) 2023-09-10T07:16:36.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:35 smithi082 ceph-mon[98694]: pgmap v15427: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:16:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): Behind on trimming (843/128) 2023-09-10T07:16:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:35 smithi098 ceph-mon[103124]: pgmap v15427: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:16:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): Behind on trimming (843/128) 2023-09-10T07:16:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:36 smithi170 ceph-mon[102832]: Health check update: 1 MDSs behind on trimming (MDS_TRIM) 2023-09-10T07:16:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:16:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:16:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:36 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): Behind on trimming (844/128) 2023-09-10T07:16:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:36 smithi082 ceph-mon[98694]: Health check update: 1 MDSs behind on trimming (MDS_TRIM) 2023-09-10T07:16:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:16:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:16:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:36 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): Behind on trimming (844/128) 2023-09-10T07:16:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:36 smithi098 ceph-mon[103124]: Health check update: 1 MDSs behind on trimming (MDS_TRIM) 2023-09-10T07:16:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:16:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:16:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:36 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): Behind on trimming (844/128) 2023-09-10T07:16:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:37 smithi170 ceph-mon[102832]: pgmap v15428: 97 pgs: 97 active+clean; 9.7 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 411 KiB/s wr, 75 op/s 2023-09-10T07:16:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:37 smithi170 ceph-mon[102832]: Health check cleared: MDS_TRIM (was: 1 MDSs behind on trimming) 2023-09-10T07:16:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:16:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:16:38.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:37 smithi082 ceph-mon[98694]: pgmap v15428: 97 pgs: 97 active+clean; 9.7 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 411 KiB/s wr, 75 op/s 2023-09-10T07:16:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:37 smithi082 ceph-mon[98694]: Health check cleared: MDS_TRIM (was: 1 MDSs behind on trimming) 2023-09-10T07:16:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:16:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:16:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:37 smithi098 ceph-mon[103124]: pgmap v15428: 97 pgs: 97 active+clean; 9.7 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 411 KiB/s wr, 75 op/s 2023-09-10T07:16:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:37 smithi098 ceph-mon[103124]: Health check cleared: MDS_TRIM (was: 1 MDSs behind on trimming) 2023-09-10T07:16:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:16:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:16:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28630.642372 secs 2023-09-10T07:16:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:39 smithi170 ceph-mon[102832]: pgmap v15429: 97 pgs: 97 active+clean; 9.7 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 411 KiB/s wr, 70 op/s 2023-09-10T07:16:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28631.119636 secs 2023-09-10T07:16:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:16:40.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28630.642372 secs 2023-09-10T07:16:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:39 smithi082 ceph-mon[98694]: pgmap v15429: 97 pgs: 97 active+clean; 9.7 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 411 KiB/s wr, 70 op/s 2023-09-10T07:16:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28631.119636 secs 2023-09-10T07:16:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:16:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28630.642372 secs 2023-09-10T07:16:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:39 smithi098 ceph-mon[103124]: pgmap v15429: 97 pgs: 97 active+clean; 9.7 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 411 KiB/s wr, 70 op/s 2023-09-10T07:16:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28631.119636 secs 2023-09-10T07:16:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:16:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:16:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:16:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:16:41.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:16:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:16:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:16:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:16:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:16:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:16:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:41 smithi170 ceph-mon[102832]: pgmap v15430: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 540 KiB/s wr, 77 op/s 2023-09-10T07:16:42.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:41 smithi082 ceph-mon[98694]: pgmap v15430: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 540 KiB/s wr, 77 op/s 2023-09-10T07:16:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:41 smithi098 ceph-mon[103124]: pgmap v15430: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 540 KiB/s wr, 77 op/s 2023-09-10T07:16:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:43 smithi170 ceph-mon[102832]: pgmap v15431: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 442 KiB/s wr, 71 op/s 2023-09-10T07:16:44.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:43 smithi082 ceph-mon[98694]: pgmap v15431: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 442 KiB/s wr, 71 op/s 2023-09-10T07:16:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:43 smithi098 ceph-mon[103124]: pgmap v15431: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 442 KiB/s wr, 71 op/s 2023-09-10T07:16:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28635.642430 secs 2023-09-10T07:16:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28636.119696 secs 2023-09-10T07:16:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28635.642430 secs 2023-09-10T07:16:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28636.119696 secs 2023-09-10T07:16:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28635.642430 secs 2023-09-10T07:16:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28636.119696 secs 2023-09-10T07:16:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:45 smithi170 ceph-mon[102832]: pgmap v15432: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 442 KiB/s wr, 71 op/s 2023-09-10T07:16:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:45 smithi082 ceph-mon[98694]: pgmap v15432: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 442 KiB/s wr, 71 op/s 2023-09-10T07:16:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:45 smithi098 ceph-mon[103124]: pgmap v15432: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 442 KiB/s wr, 71 op/s 2023-09-10T07:16:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:47 smithi170 ceph-mon[102832]: pgmap v15433: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 699 KiB/s wr, 78 op/s 2023-09-10T07:16:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:47 smithi082 ceph-mon[98694]: pgmap v15433: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 699 KiB/s wr, 78 op/s 2023-09-10T07:16:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:47 smithi098 ceph-mon[103124]: pgmap v15433: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 699 KiB/s wr, 78 op/s 2023-09-10T07:16:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28640.642489 secs 2023-09-10T07:16:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:49 smithi170 ceph-mon[102832]: pgmap v15434: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 386 KiB/s wr, 13 op/s 2023-09-10T07:16:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28641.119763 secs 2023-09-10T07:16:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28640.642489 secs 2023-09-10T07:16:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:49 smithi082 ceph-mon[98694]: pgmap v15434: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 386 KiB/s wr, 13 op/s 2023-09-10T07:16:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28641.119763 secs 2023-09-10T07:16:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28640.642489 secs 2023-09-10T07:16:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:49 smithi098 ceph-mon[103124]: pgmap v15434: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 386 KiB/s wr, 13 op/s 2023-09-10T07:16:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28641.119763 secs 2023-09-10T07:16:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:51 smithi170 ceph-mon[102832]: pgmap v15435: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 528 KiB/s wr, 20 op/s 2023-09-10T07:16:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:51 smithi082 ceph-mon[98694]: pgmap v15435: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 528 KiB/s wr, 20 op/s 2023-09-10T07:16:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:51 smithi098 ceph-mon[103124]: pgmap v15435: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 528 KiB/s wr, 20 op/s 2023-09-10T07:16:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:53 smithi170 ceph-mon[102832]: pgmap v15436: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 399 KiB/s wr, 13 op/s 2023-09-10T07:16:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:16:54.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:53 smithi082 ceph-mon[98694]: pgmap v15436: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 399 KiB/s wr, 13 op/s 2023-09-10T07:16:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:16:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:53 smithi098 ceph-mon[103124]: pgmap v15436: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 399 KiB/s wr, 13 op/s 2023-09-10T07:16:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:16:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28645.642575 secs 2023-09-10T07:16:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28646.119826 secs 2023-09-10T07:16:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:16:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:16:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:16:55.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:16:55.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:16:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28645.642575 secs 2023-09-10T07:16:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28646.119826 secs 2023-09-10T07:16:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:16:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:16:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:16:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:16:55.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:16:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28645.642575 secs 2023-09-10T07:16:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28646.119826 secs 2023-09-10T07:16:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:16:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:16:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:16:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:16:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:16:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:55 smithi170 ceph-mon[102832]: pgmap v15437: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 399 KiB/s wr, 13 op/s 2023-09-10T07:16:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:55 smithi082 ceph-mon[98694]: pgmap v15437: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 399 KiB/s wr, 13 op/s 2023-09-10T07:16:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:55 smithi098 ceph-mon[103124]: pgmap v15437: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 399 KiB/s wr, 13 op/s 2023-09-10T07:16:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:57 smithi170 ceph-mon[102832]: pgmap v15438: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 592 KiB/s wr, 20 op/s 2023-09-10T07:16:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:57 smithi082 ceph-mon[98694]: pgmap v15438: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 592 KiB/s wr, 20 op/s 2023-09-10T07:16:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:57 smithi098 ceph-mon[103124]: pgmap v15438: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 592 KiB/s wr, 20 op/s 2023-09-10T07:16:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28649.505007 secs 2023-09-10T07:16:58.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28649.505007 secs 2023-09-10T07:16:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28649.505007 secs 2023-09-10T07:16:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28650.642634 secs 2023-09-10T07:16:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:16:59 smithi170 ceph-mon[102832]: pgmap v15439: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 335 KiB/s wr, 13 op/s 2023-09-10T07:16:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28650.642634 secs 2023-09-10T07:16:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:16:59 smithi082 ceph-mon[98694]: pgmap v15439: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 335 KiB/s wr, 13 op/s 2023-09-10T07:17:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28650.642634 secs 2023-09-10T07:17:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:16:59 smithi098 ceph-mon[103124]: pgmap v15439: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 335 KiB/s wr, 13 op/s 2023-09-10T07:17:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28651.119893 secs 2023-09-10T07:17:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:17:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:17:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:17:00.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28651.119893 secs 2023-09-10T07:17:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:17:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:17:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:17:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28651.119893 secs 2023-09-10T07:17:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:17:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:17:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:17:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:01 smithi170 ceph-mon[102832]: pgmap v15440: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 429 KiB/s wr, 19 op/s 2023-09-10T07:17:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:01 smithi082 ceph-mon[98694]: pgmap v15440: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 429 KiB/s wr, 19 op/s 2023-09-10T07:17:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:01 smithi098 ceph-mon[103124]: pgmap v15440: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 429 KiB/s wr, 19 op/s 2023-09-10T07:17:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28654.505087 secs 2023-09-10T07:17:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:03 smithi170 ceph-mon[102832]: pgmap v15441: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 288 KiB/s wr, 12 op/s 2023-09-10T07:17:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28654.505087 secs 2023-09-10T07:17:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:03 smithi082 ceph-mon[98694]: pgmap v15441: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 288 KiB/s wr, 12 op/s 2023-09-10T07:17:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28654.505087 secs 2023-09-10T07:17:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:03 smithi098 ceph-mon[103124]: pgmap v15441: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 288 KiB/s wr, 12 op/s 2023-09-10T07:17:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28655.642692 secs 2023-09-10T07:17:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28656.119952 secs 2023-09-10T07:17:05.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28655.642692 secs 2023-09-10T07:17:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28656.119952 secs 2023-09-10T07:17:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28655.642692 secs 2023-09-10T07:17:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28656.119952 secs 2023-09-10T07:17:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:05 smithi170 ceph-mon[102832]: pgmap v15442: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 288 KiB/s wr, 12 op/s 2023-09-10T07:17:06.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:05 smithi082 ceph-mon[98694]: pgmap v15442: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 288 KiB/s wr, 12 op/s 2023-09-10T07:17:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:05 smithi098 ceph-mon[103124]: pgmap v15442: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 288 KiB/s wr, 12 op/s 2023-09-10T07:17:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:07 smithi170 ceph-mon[102832]: pgmap v15443: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 477 KiB/s wr, 33 op/s 2023-09-10T07:17:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:07 smithi082 ceph-mon[98694]: pgmap v15443: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 477 KiB/s wr, 33 op/s 2023-09-10T07:17:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:07 smithi098 ceph-mon[103124]: pgmap v15443: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 477 KiB/s wr, 33 op/s 2023-09-10T07:17:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28659.505162 secs 2023-09-10T07:17:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28659.505162 secs 2023-09-10T07:17:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28659.505162 secs 2023-09-10T07:17:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28660.642750 secs 2023-09-10T07:17:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:09 smithi170 ceph-mon[102832]: osdmap e1033: 12 total, 12 up, 12 in 2023-09-10T07:17:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:09 smithi170 ceph-mon[102832]: pgmap v15445: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 340 KiB/s wr, 32 op/s 2023-09-10T07:17:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28660.642750 secs 2023-09-10T07:17:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:09 smithi082 ceph-mon[98694]: osdmap e1033: 12 total, 12 up, 12 in 2023-09-10T07:17:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:09 smithi082 ceph-mon[98694]: pgmap v15445: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 340 KiB/s wr, 32 op/s 2023-09-10T07:17:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28660.642750 secs 2023-09-10T07:17:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:09 smithi098 ceph-mon[103124]: osdmap e1033: 12 total, 12 up, 12 in 2023-09-10T07:17:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:09 smithi098 ceph-mon[103124]: pgmap v15445: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 340 KiB/s wr, 32 op/s 2023-09-10T07:17:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28661.120017 secs 2023-09-10T07:17:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28661.120017 secs 2023-09-10T07:17:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28661.120017 secs 2023-09-10T07:17:11.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:11 smithi170 ceph-mon[102832]: pgmap v15446: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 347 KiB/s wr, 31 op/s 2023-09-10T07:17:11.962 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:11 smithi082 ceph-mon[98694]: pgmap v15446: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 347 KiB/s wr, 31 op/s 2023-09-10T07:17:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:11 smithi098 ceph-mon[103124]: pgmap v15446: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 347 KiB/s wr, 31 op/s 2023-09-10T07:17:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28664.505225 secs 2023-09-10T07:17:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:13 smithi170 ceph-mon[102832]: pgmap v15447: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 347 KiB/s wr, 31 op/s 2023-09-10T07:17:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28664.505225 secs 2023-09-10T07:17:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:13 smithi082 ceph-mon[98694]: pgmap v15447: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 347 KiB/s wr, 31 op/s 2023-09-10T07:17:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28664.505225 secs 2023-09-10T07:17:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:13 smithi098 ceph-mon[103124]: pgmap v15447: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 347 KiB/s wr, 31 op/s 2023-09-10T07:17:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28665.642806 secs 2023-09-10T07:17:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28666.120077 secs 2023-09-10T07:17:15.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28665.642806 secs 2023-09-10T07:17:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28666.120077 secs 2023-09-10T07:17:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28665.642806 secs 2023-09-10T07:17:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28666.120077 secs 2023-09-10T07:17:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:15 smithi170 ceph-mon[102832]: pgmap v15448: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 347 KiB/s wr, 31 op/s 2023-09-10T07:17:16.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:15 smithi082 ceph-mon[98694]: pgmap v15448: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 347 KiB/s wr, 31 op/s 2023-09-10T07:17:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:15 smithi098 ceph-mon[103124]: pgmap v15448: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 347 KiB/s wr, 31 op/s 2023-09-10T07:17:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:17 smithi170 ceph-mon[102832]: osdmap e1034: 12 total, 12 up, 12 in 2023-09-10T07:17:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:17 smithi170 ceph-mon[102832]: pgmap v15450: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 15 op/s 2023-09-10T07:17:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:17 smithi082 ceph-mon[98694]: osdmap e1034: 12 total, 12 up, 12 in 2023-09-10T07:17:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:17 smithi082 ceph-mon[98694]: pgmap v15450: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 15 op/s 2023-09-10T07:17:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:17 smithi098 ceph-mon[103124]: osdmap e1034: 12 total, 12 up, 12 in 2023-09-10T07:17:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:17 smithi098 ceph-mon[103124]: pgmap v15450: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 15 op/s 2023-09-10T07:17:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28669.505286 secs 2023-09-10T07:17:18.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28669.505286 secs 2023-09-10T07:17:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28669.505286 secs 2023-09-10T07:17:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28670.642864 secs 2023-09-10T07:17:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:19 smithi170 ceph-mon[102832]: pgmap v15451: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T07:17:19.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28670.642864 secs 2023-09-10T07:17:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:19 smithi082 ceph-mon[98694]: pgmap v15451: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T07:17:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28670.642864 secs 2023-09-10T07:17:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:19 smithi098 ceph-mon[103124]: pgmap v15451: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T07:17:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28671.120142 secs 2023-09-10T07:17:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28671.120142 secs 2023-09-10T07:17:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28671.120142 secs 2023-09-10T07:17:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:21 smithi170 ceph-mon[102832]: pgmap v15452: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T07:17:21.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:21 smithi082 ceph-mon[98694]: pgmap v15452: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T07:17:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:21 smithi098 ceph-mon[103124]: pgmap v15452: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T07:17:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28674.505367 secs 2023-09-10T07:17:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:23 smithi170 ceph-mon[102832]: pgmap v15453: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T07:17:24.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28674.505367 secs 2023-09-10T07:17:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:23 smithi082 ceph-mon[98694]: pgmap v15453: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T07:17:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28674.505367 secs 2023-09-10T07:17:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:23 smithi098 ceph-mon[103124]: pgmap v15453: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T07:17:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28675.642924 secs 2023-09-10T07:17:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28676.120204 secs 2023-09-10T07:17:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28675.642924 secs 2023-09-10T07:17:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28676.120204 secs 2023-09-10T07:17:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28675.642924 secs 2023-09-10T07:17:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28676.120204 secs 2023-09-10T07:17:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:25 smithi170 ceph-mon[102832]: pgmap v15454: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T07:17:26.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:25 smithi082 ceph-mon[98694]: pgmap v15454: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T07:17:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:25 smithi098 ceph-mon[103124]: pgmap v15454: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T07:17:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:27 smithi170 ceph-mon[102832]: pgmap v15455: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T07:17:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:27 smithi082 ceph-mon[98694]: pgmap v15455: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T07:17:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:27 smithi098 ceph-mon[103124]: pgmap v15455: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 12 op/s 2023-09-10T07:17:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28679.505427 secs 2023-09-10T07:17:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28679.505427 secs 2023-09-10T07:17:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28679.505427 secs 2023-09-10T07:17:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28680.643002 secs 2023-09-10T07:17:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:29 smithi170 ceph-mon[102832]: pgmap v15456: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 10 op/s 2023-09-10T07:17:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28680.643002 secs 2023-09-10T07:17:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:29 smithi082 ceph-mon[98694]: pgmap v15456: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 10 op/s 2023-09-10T07:17:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28680.643002 secs 2023-09-10T07:17:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:29 smithi098 ceph-mon[103124]: pgmap v15456: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 10 op/s 2023-09-10T07:17:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28681.120276 secs 2023-09-10T07:17:30.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28681.120276 secs 2023-09-10T07:17:31.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28681.120276 secs 2023-09-10T07:17:31.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:31 smithi082 ceph-mon[98694]: pgmap v15457: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 199 KiB/s wr, 15 op/s 2023-09-10T07:17:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:31 smithi098 ceph-mon[103124]: pgmap v15457: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 199 KiB/s wr, 15 op/s 2023-09-10T07:17:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:31 smithi170 ceph-mon[102832]: pgmap v15457: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 199 KiB/s wr, 15 op/s 2023-09-10T07:17:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28684.505521 secs 2023-09-10T07:17:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:33 smithi170 ceph-mon[102832]: pgmap v15458: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:17:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28684.505521 secs 2023-09-10T07:17:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:33 smithi082 ceph-mon[98694]: pgmap v15458: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:17:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28684.505521 secs 2023-09-10T07:17:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:33 smithi098 ceph-mon[103124]: pgmap v15458: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:17:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28685.643080 secs 2023-09-10T07:17:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28686.120331 secs 2023-09-10T07:17:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28685.643080 secs 2023-09-10T07:17:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28686.120331 secs 2023-09-10T07:17:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28685.643080 secs 2023-09-10T07:17:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28686.120331 secs 2023-09-10T07:17:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:35 smithi170 ceph-mon[102832]: pgmap v15459: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:17:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:35 smithi082 ceph-mon[98694]: pgmap v15459: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:17:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:35 smithi098 ceph-mon[103124]: pgmap v15459: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:17:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:37 smithi170 ceph-mon[102832]: pgmap v15460: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 361 KiB/s wr, 15 op/s 2023-09-10T07:17:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:37 smithi082 ceph-mon[98694]: pgmap v15460: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 361 KiB/s wr, 15 op/s 2023-09-10T07:17:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:37 smithi098 ceph-mon[103124]: pgmap v15460: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 361 KiB/s wr, 15 op/s 2023-09-10T07:17:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28690.643137 secs 2023-09-10T07:17:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:39 smithi170 ceph-mon[102832]: pgmap v15461: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 360 KiB/s wr, 10 op/s 2023-09-10T07:17:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28691.120398 secs 2023-09-10T07:17:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:17:40.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28690.643137 secs 2023-09-10T07:17:40.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:39 smithi082 ceph-mon[98694]: pgmap v15461: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 360 KiB/s wr, 10 op/s 2023-09-10T07:17:40.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28691.120398 secs 2023-09-10T07:17:40.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:17:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28690.643137 secs 2023-09-10T07:17:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:39 smithi098 ceph-mon[103124]: pgmap v15461: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 360 KiB/s wr, 10 op/s 2023-09-10T07:17:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28691.120398 secs 2023-09-10T07:17:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:17:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:17:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:17:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:17:41.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:17:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:17:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:17:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:17:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:17:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:17:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:41 smithi170 ceph-mon[102832]: pgmap v15462: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 544 KiB/s wr, 17 op/s 2023-09-10T07:17:42.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:41 smithi082 ceph-mon[98694]: pgmap v15462: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 544 KiB/s wr, 17 op/s 2023-09-10T07:17:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:41 smithi098 ceph-mon[103124]: pgmap v15462: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 544 KiB/s wr, 17 op/s 2023-09-10T07:17:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:43 smithi170 ceph-mon[102832]: pgmap v15463: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 446 KiB/s wr, 11 op/s 2023-09-10T07:17:44.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:43 smithi082 ceph-mon[98694]: pgmap v15463: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 446 KiB/s wr, 11 op/s 2023-09-10T07:17:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:43 smithi098 ceph-mon[103124]: pgmap v15463: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 446 KiB/s wr, 11 op/s 2023-09-10T07:17:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28695.643221 secs 2023-09-10T07:17:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28696.120461 secs 2023-09-10T07:17:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28695.643221 secs 2023-09-10T07:17:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28696.120461 secs 2023-09-10T07:17:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28695.643221 secs 2023-09-10T07:17:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28696.120461 secs 2023-09-10T07:17:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:45 smithi170 ceph-mon[102832]: pgmap v15464: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 446 KiB/s wr, 11 op/s 2023-09-10T07:17:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:45 smithi082 ceph-mon[98694]: pgmap v15464: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 446 KiB/s wr, 11 op/s 2023-09-10T07:17:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:45 smithi098 ceph-mon[103124]: pgmap v15464: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 446 KiB/s wr, 11 op/s 2023-09-10T07:17:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:47 smithi170 ceph-mon[102832]: pgmap v15465: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 649 KiB/s wr, 18 op/s 2023-09-10T07:17:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:47 smithi082 ceph-mon[98694]: pgmap v15465: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 649 KiB/s wr, 18 op/s 2023-09-10T07:17:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:47 smithi098 ceph-mon[103124]: pgmap v15465: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 649 KiB/s wr, 18 op/s 2023-09-10T07:17:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28700.643279 secs 2023-09-10T07:17:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:49 smithi170 ceph-mon[102832]: pgmap v15466: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 386 KiB/s wr, 12 op/s 2023-09-10T07:17:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28701.120487 secs 2023-09-10T07:17:50.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28700.643279 secs 2023-09-10T07:17:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:49 smithi082 ceph-mon[98694]: pgmap v15466: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 386 KiB/s wr, 12 op/s 2023-09-10T07:17:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28701.120487 secs 2023-09-10T07:17:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28700.643279 secs 2023-09-10T07:17:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:49 smithi098 ceph-mon[103124]: pgmap v15466: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 386 KiB/s wr, 12 op/s 2023-09-10T07:17:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28701.120487 secs 2023-09-10T07:17:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:51 smithi170 ceph-mon[102832]: pgmap v15467: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 584 KiB/s wr, 18 op/s 2023-09-10T07:17:52.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:51 smithi082 ceph-mon[98694]: pgmap v15467: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 584 KiB/s wr, 18 op/s 2023-09-10T07:17:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:51 smithi098 ceph-mon[103124]: pgmap v15467: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 584 KiB/s wr, 18 op/s 2023-09-10T07:17:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:53 smithi170 ceph-mon[102832]: pgmap v15468: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 400 KiB/s wr, 12 op/s 2023-09-10T07:17:54.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:53 smithi082 ceph-mon[98694]: pgmap v15468: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 400 KiB/s wr, 12 op/s 2023-09-10T07:17:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:53 smithi098 ceph-mon[103124]: pgmap v15468: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 400 KiB/s wr, 12 op/s 2023-09-10T07:17:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28705.643340 secs 2023-09-10T07:17:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28706.120546 secs 2023-09-10T07:17:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:17:55.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28705.643340 secs 2023-09-10T07:17:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28706.120546 secs 2023-09-10T07:17:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:17:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28705.643340 secs 2023-09-10T07:17:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28706.120546 secs 2023-09-10T07:17:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:17:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:56 smithi170 ceph-mon[102832]: pgmap v15469: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 400 KiB/s wr, 12 op/s 2023-09-10T07:17:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:17:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:17:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:17:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:17:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:17:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:17:56.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:17:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:56 smithi082 ceph-mon[98694]: pgmap v15469: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 400 KiB/s wr, 12 op/s 2023-09-10T07:17:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:17:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:17:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:17:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:17:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:17:56.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:17:56.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:17:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:56 smithi098 ceph-mon[103124]: pgmap v15469: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 400 KiB/s wr, 12 op/s 2023-09-10T07:17:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:17:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:17:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:17:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:17:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:17:56.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:17:56.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:17:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:57 smithi170 ceph-mon[102832]: pgmap v15470: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 595 KiB/s wr, 19 op/s 2023-09-10T07:17:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:57 smithi082 ceph-mon[98694]: pgmap v15470: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 595 KiB/s wr, 19 op/s 2023-09-10T07:17:58.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:57 smithi098 ceph-mon[103124]: pgmap v15470: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 595 KiB/s wr, 19 op/s 2023-09-10T07:17:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28709.505869 secs 2023-09-10T07:17:58.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28709.505869 secs 2023-09-10T07:17:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28709.505869 secs 2023-09-10T07:17:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28710.643398 secs 2023-09-10T07:17:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:17:59 smithi082 ceph-mon[98694]: pgmap v15471: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 392 KiB/s wr, 13 op/s 2023-09-10T07:18:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28710.643398 secs 2023-09-10T07:18:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:17:59 smithi098 ceph-mon[103124]: pgmap v15471: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 392 KiB/s wr, 13 op/s 2023-09-10T07:18:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28710.643398 secs 2023-09-10T07:18:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:17:59 smithi170 ceph-mon[102832]: pgmap v15471: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 392 KiB/s wr, 13 op/s 2023-09-10T07:18:00.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28711.120611 secs 2023-09-10T07:18:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:18:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:18:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:18:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28711.120611 secs 2023-09-10T07:18:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:18:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:18:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:18:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28711.120611 secs 2023-09-10T07:18:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:18:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:18:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:18:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:01 smithi082 ceph-mon[98694]: pgmap v15472: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 531 KiB/s wr, 33 op/s 2023-09-10T07:18:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:01 smithi098 ceph-mon[103124]: pgmap v15472: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 531 KiB/s wr, 33 op/s 2023-09-10T07:18:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:01 smithi170 ceph-mon[102832]: pgmap v15472: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 531 KiB/s wr, 33 op/s 2023-09-10T07:18:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28714.505965 secs 2023-09-10T07:18:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:03 smithi170 ceph-mon[102832]: pgmap v15473: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 334 KiB/s wr, 27 op/s 2023-09-10T07:18:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:03 smithi170 ceph-mon[102832]: osdmap e1035: 12 total, 12 up, 12 in 2023-09-10T07:18:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28714.505965 secs 2023-09-10T07:18:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:03 smithi082 ceph-mon[98694]: pgmap v15473: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 334 KiB/s wr, 27 op/s 2023-09-10T07:18:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:03 smithi082 ceph-mon[98694]: osdmap e1035: 12 total, 12 up, 12 in 2023-09-10T07:18:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28714.505965 secs 2023-09-10T07:18:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:03 smithi098 ceph-mon[103124]: pgmap v15473: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 334 KiB/s wr, 27 op/s 2023-09-10T07:18:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:03 smithi098 ceph-mon[103124]: osdmap e1035: 12 total, 12 up, 12 in 2023-09-10T07:18:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28715.643456 secs 2023-09-10T07:18:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28716.120675 secs 2023-09-10T07:18:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28715.643456 secs 2023-09-10T07:18:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28716.120675 secs 2023-09-10T07:18:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28715.643456 secs 2023-09-10T07:18:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28716.120675 secs 2023-09-10T07:18:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:05 smithi170 ceph-mon[102832]: pgmap v15475: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 400 KiB/s wr, 32 op/s 2023-09-10T07:18:06.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:05 smithi082 ceph-mon[98694]: pgmap v15475: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 400 KiB/s wr, 32 op/s 2023-09-10T07:18:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:05 smithi098 ceph-mon[103124]: pgmap v15475: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 400 KiB/s wr, 32 op/s 2023-09-10T07:18:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:07 smithi170 ceph-mon[102832]: pgmap v15476: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 341 KiB/s wr, 30 op/s 2023-09-10T07:18:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:07 smithi082 ceph-mon[98694]: pgmap v15476: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 341 KiB/s wr, 30 op/s 2023-09-10T07:18:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:07 smithi098 ceph-mon[103124]: pgmap v15476: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 341 KiB/s wr, 30 op/s 2023-09-10T07:18:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28719.506027 secs 2023-09-10T07:18:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28719.506027 secs 2023-09-10T07:18:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28719.506027 secs 2023-09-10T07:18:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28720.643491 secs 2023-09-10T07:18:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:09 smithi170 ceph-mon[102832]: pgmap v15477: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 341 KiB/s wr, 30 op/s 2023-09-10T07:18:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28720.643491 secs 2023-09-10T07:18:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:09 smithi082 ceph-mon[98694]: pgmap v15477: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 341 KiB/s wr, 30 op/s 2023-09-10T07:18:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28720.643491 secs 2023-09-10T07:18:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:09 smithi098 ceph-mon[103124]: pgmap v15477: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 341 KiB/s wr, 30 op/s 2023-09-10T07:18:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28721.120738 secs 2023-09-10T07:18:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28721.120738 secs 2023-09-10T07:18:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28721.120738 secs 2023-09-10T07:18:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:11 smithi082 ceph-mon[98694]: pgmap v15478: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T07:18:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:11 smithi082 ceph-mon[98694]: osdmap e1036: 12 total, 12 up, 12 in 2023-09-10T07:18:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:11 smithi098 ceph-mon[103124]: pgmap v15478: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T07:18:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:11 smithi098 ceph-mon[103124]: osdmap e1036: 12 total, 12 up, 12 in 2023-09-10T07:18:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:11 smithi170 ceph-mon[102832]: pgmap v15478: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T07:18:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:11 smithi170 ceph-mon[102832]: osdmap e1036: 12 total, 12 up, 12 in 2023-09-10T07:18:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28724.506085 secs 2023-09-10T07:18:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:13 smithi170 ceph-mon[102832]: pgmap v15480: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T07:18:14.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28724.506085 secs 2023-09-10T07:18:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:13 smithi082 ceph-mon[98694]: pgmap v15480: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T07:18:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28724.506085 secs 2023-09-10T07:18:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:13 smithi098 ceph-mon[103124]: pgmap v15480: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 270 KiB/s wr, 12 op/s 2023-09-10T07:18:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28725.643549 secs 2023-09-10T07:18:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28726.120796 secs 2023-09-10T07:18:15.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28725.643549 secs 2023-09-10T07:18:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28726.120796 secs 2023-09-10T07:18:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28725.643549 secs 2023-09-10T07:18:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28726.120796 secs 2023-09-10T07:18:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:15 smithi170 ceph-mon[102832]: pgmap v15481: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T07:18:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:15 smithi082 ceph-mon[98694]: pgmap v15481: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T07:18:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:15 smithi098 ceph-mon[103124]: pgmap v15481: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 269 KiB/s wr, 12 op/s 2023-09-10T07:18:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:17 smithi082 ceph-mon[98694]: pgmap v15482: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T07:18:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:17 smithi098 ceph-mon[103124]: pgmap v15482: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T07:18:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:17 smithi170 ceph-mon[102832]: pgmap v15482: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T07:18:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28729.506162 secs 2023-09-10T07:18:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28729.506162 secs 2023-09-10T07:18:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28729.506162 secs 2023-09-10T07:18:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28730.643609 secs 2023-09-10T07:18:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:19 smithi082 ceph-mon[98694]: pgmap v15483: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T07:18:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28730.643609 secs 2023-09-10T07:18:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:19 smithi098 ceph-mon[103124]: pgmap v15483: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T07:18:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28730.643609 secs 2023-09-10T07:18:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:19 smithi170 ceph-mon[102832]: pgmap v15483: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T07:18:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28731.120863 secs 2023-09-10T07:18:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28731.120863 secs 2023-09-10T07:18:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28731.120863 secs 2023-09-10T07:18:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:21 smithi082 ceph-mon[98694]: pgmap v15484: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T07:18:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:21 smithi098 ceph-mon[103124]: pgmap v15484: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T07:18:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:21 smithi170 ceph-mon[102832]: pgmap v15484: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T07:18:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28734.506233 secs 2023-09-10T07:18:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:23 smithi170 ceph-mon[102832]: pgmap v15485: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T07:18:24.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28734.506233 secs 2023-09-10T07:18:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:23 smithi082 ceph-mon[98694]: pgmap v15485: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T07:18:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28734.506233 secs 2023-09-10T07:18:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:23 smithi098 ceph-mon[103124]: pgmap v15485: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T07:18:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28735.643700 secs 2023-09-10T07:18:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28736.120924 secs 2023-09-10T07:18:25.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28735.643700 secs 2023-09-10T07:18:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28736.120924 secs 2023-09-10T07:18:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28735.643700 secs 2023-09-10T07:18:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28736.120924 secs 2023-09-10T07:18:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:25 smithi170 ceph-mon[102832]: pgmap v15486: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 10 op/s 2023-09-10T07:18:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:25 smithi082 ceph-mon[98694]: pgmap v15486: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 10 op/s 2023-09-10T07:18:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:25 smithi098 ceph-mon[103124]: pgmap v15486: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 10 op/s 2023-09-10T07:18:27.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:27 smithi082 ceph-mon[98694]: pgmap v15487: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 102 KiB/s wr, 15 op/s 2023-09-10T07:18:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:27 smithi098 ceph-mon[103124]: pgmap v15487: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 102 KiB/s wr, 15 op/s 2023-09-10T07:18:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:27 smithi170 ceph-mon[102832]: pgmap v15487: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 102 KiB/s wr, 15 op/s 2023-09-10T07:18:28.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28739.506291 secs 2023-09-10T07:18:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28739.506291 secs 2023-09-10T07:18:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28739.506291 secs 2023-09-10T07:18:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28740.643757 secs 2023-09-10T07:18:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:29 smithi082 ceph-mon[98694]: pgmap v15488: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 102 KiB/s wr, 10 op/s 2023-09-10T07:18:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28741.120988 secs 2023-09-10T07:18:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28740.643757 secs 2023-09-10T07:18:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:29 smithi098 ceph-mon[103124]: pgmap v15488: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 102 KiB/s wr, 10 op/s 2023-09-10T07:18:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28741.120988 secs 2023-09-10T07:18:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28740.643757 secs 2023-09-10T07:18:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:29 smithi170 ceph-mon[102832]: pgmap v15488: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 102 KiB/s wr, 10 op/s 2023-09-10T07:18:30.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28741.120988 secs 2023-09-10T07:18:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:31 smithi170 ceph-mon[102832]: pgmap v15489: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 200 KiB/s wr, 15 op/s 2023-09-10T07:18:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:31 smithi082 ceph-mon[98694]: pgmap v15489: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 200 KiB/s wr, 15 op/s 2023-09-10T07:18:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:31 smithi098 ceph-mon[103124]: pgmap v15489: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 200 KiB/s wr, 15 op/s 2023-09-10T07:18:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28744.506389 secs 2023-09-10T07:18:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:33 smithi170 ceph-mon[102832]: pgmap v15490: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 10 op/s 2023-09-10T07:18:34.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28744.506389 secs 2023-09-10T07:18:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:33 smithi082 ceph-mon[98694]: pgmap v15490: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 10 op/s 2023-09-10T07:18:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28744.506389 secs 2023-09-10T07:18:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:33 smithi098 ceph-mon[103124]: pgmap v15490: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 10 op/s 2023-09-10T07:18:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28745.643817 secs 2023-09-10T07:18:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28746.121048 secs 2023-09-10T07:18:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28745.643817 secs 2023-09-10T07:18:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28746.121048 secs 2023-09-10T07:18:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28745.643817 secs 2023-09-10T07:18:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28746.121048 secs 2023-09-10T07:18:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:35 smithi170 ceph-mon[102832]: pgmap v15491: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 10 op/s 2023-09-10T07:18:36.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:35 smithi082 ceph-mon[98694]: pgmap v15491: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 10 op/s 2023-09-10T07:18:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:35 smithi098 ceph-mon[103124]: pgmap v15491: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 10 op/s 2023-09-10T07:18:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:37 smithi082 ceph-mon[98694]: pgmap v15492: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 327 KiB/s wr, 16 op/s 2023-09-10T07:18:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:37 smithi098 ceph-mon[103124]: pgmap v15492: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 327 KiB/s wr, 16 op/s 2023-09-10T07:18:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:37 smithi170 ceph-mon[102832]: pgmap v15492: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 327 KiB/s wr, 16 op/s 2023-09-10T07:18:38.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28749.506447 secs 2023-09-10T07:18:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28749.506447 secs 2023-09-10T07:18:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28749.506447 secs 2023-09-10T07:18:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28750.643926 secs 2023-09-10T07:18:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:39 smithi082 ceph-mon[98694]: pgmap v15493: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 327 KiB/s wr, 12 op/s 2023-09-10T07:18:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28751.121113 secs 2023-09-10T07:18:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28750.643926 secs 2023-09-10T07:18:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:39 smithi098 ceph-mon[103124]: pgmap v15493: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 327 KiB/s wr, 12 op/s 2023-09-10T07:18:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28751.121113 secs 2023-09-10T07:18:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28750.643926 secs 2023-09-10T07:18:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:39 smithi170 ceph-mon[102832]: pgmap v15493: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 327 KiB/s wr, 12 op/s 2023-09-10T07:18:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28751.121113 secs 2023-09-10T07:18:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:41 smithi170 ceph-mon[102832]: pgmap v15494: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 543 KiB/s wr, 17 op/s 2023-09-10T07:18:42.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:41 smithi082 ceph-mon[98694]: pgmap v15494: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 543 KiB/s wr, 17 op/s 2023-09-10T07:18:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:41 smithi098 ceph-mon[103124]: pgmap v15494: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 543 KiB/s wr, 17 op/s 2023-09-10T07:18:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:43 smithi170 ceph-mon[102832]: pgmap v15495: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 12 op/s 2023-09-10T07:18:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:18:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:43 smithi082 ceph-mon[98694]: pgmap v15495: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 12 op/s 2023-09-10T07:18:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:18:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:43 smithi098 ceph-mon[103124]: pgmap v15495: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 12 op/s 2023-09-10T07:18:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:18:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28755.643985 secs 2023-09-10T07:18:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28756.121172 secs 2023-09-10T07:18:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:44 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:18:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:18:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:18:45.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28755.643985 secs 2023-09-10T07:18:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28756.121172 secs 2023-09-10T07:18:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:44 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:18:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:18:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:18:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28755.643985 secs 2023-09-10T07:18:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28756.121172 secs 2023-09-10T07:18:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:44 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:18:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:18:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:18:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:45 smithi170 ceph-mon[102832]: pgmap v15496: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 12 op/s 2023-09-10T07:18:46.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:45 smithi082 ceph-mon[98694]: pgmap v15496: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 12 op/s 2023-09-10T07:18:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:45 smithi098 ceph-mon[103124]: pgmap v15496: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 12 op/s 2023-09-10T07:18:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:47 smithi082 ceph-mon[98694]: pgmap v15497: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 639 KiB/s wr, 19 op/s 2023-09-10T07:18:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:47 smithi098 ceph-mon[103124]: pgmap v15497: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 639 KiB/s wr, 19 op/s 2023-09-10T07:18:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:47 smithi170 ceph-mon[102832]: pgmap v15497: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 639 KiB/s wr, 19 op/s 2023-09-10T07:18:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28760.644055 secs 2023-09-10T07:18:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:49 smithi170 ceph-mon[102832]: pgmap v15498: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 410 KiB/s wr, 12 op/s 2023-09-10T07:18:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28761.121238 secs 2023-09-10T07:18:50.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28760.644055 secs 2023-09-10T07:18:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:49 smithi082 ceph-mon[98694]: pgmap v15498: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 410 KiB/s wr, 12 op/s 2023-09-10T07:18:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28761.121238 secs 2023-09-10T07:18:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28760.644055 secs 2023-09-10T07:18:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:49 smithi098 ceph-mon[103124]: pgmap v15498: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 410 KiB/s wr, 12 op/s 2023-09-10T07:18:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28761.121238 secs 2023-09-10T07:18:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:51 smithi170 ceph-mon[102832]: pgmap v15499: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 617 KiB/s wr, 17 op/s 2023-09-10T07:18:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:51 smithi082 ceph-mon[98694]: pgmap v15499: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 617 KiB/s wr, 17 op/s 2023-09-10T07:18:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:51 smithi098 ceph-mon[103124]: pgmap v15499: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 617 KiB/s wr, 17 op/s 2023-09-10T07:18:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:53 smithi170 ceph-mon[102832]: pgmap v15500: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 401 KiB/s wr, 12 op/s 2023-09-10T07:18:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:53 smithi082 ceph-mon[98694]: pgmap v15500: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 401 KiB/s wr, 12 op/s 2023-09-10T07:18:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:53 smithi098 ceph-mon[103124]: pgmap v15500: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 401 KiB/s wr, 12 op/s 2023-09-10T07:18:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28765.644115 secs 2023-09-10T07:18:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28766.121297 secs 2023-09-10T07:18:55.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28765.644115 secs 2023-09-10T07:18:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28766.121297 secs 2023-09-10T07:18:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28765.644115 secs 2023-09-10T07:18:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28766.121297 secs 2023-09-10T07:18:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:55 smithi170 ceph-mon[102832]: pgmap v15501: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 401 KiB/s wr, 12 op/s 2023-09-10T07:18:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:18:56.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:55 smithi082 ceph-mon[98694]: pgmap v15501: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 401 KiB/s wr, 12 op/s 2023-09-10T07:18:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:18:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:55 smithi098 ceph-mon[103124]: pgmap v15501: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 401 KiB/s wr, 12 op/s 2023-09-10T07:18:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:18:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:18:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:18:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:18:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:18:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:18:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:18:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:18:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:18:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:18:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:57 smithi170 ceph-mon[102832]: pgmap v15502: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 612 KiB/s wr, 18 op/s 2023-09-10T07:18:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:57 smithi082 ceph-mon[98694]: pgmap v15502: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 612 KiB/s wr, 18 op/s 2023-09-10T07:18:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:57 smithi098 ceph-mon[103124]: pgmap v15502: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 612 KiB/s wr, 18 op/s 2023-09-10T07:18:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28769.506689 secs 2023-09-10T07:18:59.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28769.506689 secs 2023-09-10T07:18:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28769.506689 secs 2023-09-10T07:19:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28770.644183 secs 2023-09-10T07:19:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:59 smithi170 ceph-mon[102832]: pgmap v15503: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 418 KiB/s wr, 11 op/s 2023-09-10T07:19:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:18:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28771.121369 secs 2023-09-10T07:19:00.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28770.644183 secs 2023-09-10T07:19:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:59 smithi082 ceph-mon[98694]: pgmap v15503: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 418 KiB/s wr, 11 op/s 2023-09-10T07:19:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:18:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28771.121369 secs 2023-09-10T07:19:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28770.644183 secs 2023-09-10T07:19:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:59 smithi098 ceph-mon[103124]: pgmap v15503: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 418 KiB/s wr, 11 op/s 2023-09-10T07:19:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:18:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28771.121369 secs 2023-09-10T07:19:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:19:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:19:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:19:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:19:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:19:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:19:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:19:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:19:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:19:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:01 smithi170 ceph-mon[102832]: pgmap v15504: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 557 KiB/s wr, 20 op/s 2023-09-10T07:19:02.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:01 smithi082 ceph-mon[98694]: pgmap v15504: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 557 KiB/s wr, 20 op/s 2023-09-10T07:19:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:01 smithi098 ceph-mon[103124]: pgmap v15504: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 557 KiB/s wr, 20 op/s 2023-09-10T07:19:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28774.506770 secs 2023-09-10T07:19:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:03 smithi082 ceph-mon[98694]: pgmap v15505: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 349 KiB/s wr, 15 op/s 2023-09-10T07:19:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:03 smithi082 ceph-mon[98694]: osdmap e1037: 12 total, 12 up, 12 in 2023-09-10T07:19:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28774.506770 secs 2023-09-10T07:19:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:03 smithi098 ceph-mon[103124]: pgmap v15505: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 349 KiB/s wr, 15 op/s 2023-09-10T07:19:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:03 smithi098 ceph-mon[103124]: osdmap e1037: 12 total, 12 up, 12 in 2023-09-10T07:19:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28774.506770 secs 2023-09-10T07:19:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:03 smithi170 ceph-mon[102832]: pgmap v15505: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 349 KiB/s wr, 15 op/s 2023-09-10T07:19:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:03 smithi170 ceph-mon[102832]: osdmap e1037: 12 total, 12 up, 12 in 2023-09-10T07:19:05.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28775.644242 secs 2023-09-10T07:19:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28776.121427 secs 2023-09-10T07:19:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28775.644242 secs 2023-09-10T07:19:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28776.121427 secs 2023-09-10T07:19:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28775.644242 secs 2023-09-10T07:19:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28776.121427 secs 2023-09-10T07:19:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:05 smithi082 ceph-mon[98694]: pgmap v15507: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 419 KiB/s wr, 18 op/s 2023-09-10T07:19:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:05 smithi098 ceph-mon[103124]: pgmap v15507: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 419 KiB/s wr, 18 op/s 2023-09-10T07:19:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:05 smithi170 ceph-mon[102832]: pgmap v15507: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 419 KiB/s wr, 18 op/s 2023-09-10T07:19:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:07 smithi082 ceph-mon[98694]: pgmap v15508: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 361 KiB/s wr, 31 op/s 2023-09-10T07:19:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:07 smithi098 ceph-mon[103124]: pgmap v15508: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 361 KiB/s wr, 31 op/s 2023-09-10T07:19:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:07 smithi170 ceph-mon[102832]: pgmap v15508: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 361 KiB/s wr, 31 op/s 2023-09-10T07:19:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28779.506858 secs 2023-09-10T07:19:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28779.506858 secs 2023-09-10T07:19:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28779.506858 secs 2023-09-10T07:19:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28780.644328 secs 2023-09-10T07:19:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:09 smithi082 ceph-mon[98694]: pgmap v15509: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 361 KiB/s wr, 31 op/s 2023-09-10T07:19:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28781.121494 secs 2023-09-10T07:19:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28780.644328 secs 2023-09-10T07:19:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:09 smithi098 ceph-mon[103124]: pgmap v15509: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 361 KiB/s wr, 31 op/s 2023-09-10T07:19:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28781.121494 secs 2023-09-10T07:19:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28780.644328 secs 2023-09-10T07:19:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:09 smithi170 ceph-mon[102832]: pgmap v15509: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 361 KiB/s wr, 31 op/s 2023-09-10T07:19:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28781.121494 secs 2023-09-10T07:19:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:11 smithi170 ceph-mon[102832]: pgmap v15510: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 316 KiB/s wr, 28 op/s 2023-09-10T07:19:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:11 smithi170 ceph-mon[102832]: osdmap e1038: 12 total, 12 up, 12 in 2023-09-10T07:19:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:11 smithi082 ceph-mon[98694]: pgmap v15510: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 316 KiB/s wr, 28 op/s 2023-09-10T07:19:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:11 smithi082 ceph-mon[98694]: osdmap e1038: 12 total, 12 up, 12 in 2023-09-10T07:19:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:11 smithi098 ceph-mon[103124]: pgmap v15510: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 316 KiB/s wr, 28 op/s 2023-09-10T07:19:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:11 smithi098 ceph-mon[103124]: osdmap e1038: 12 total, 12 up, 12 in 2023-09-10T07:19:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28784.506936 secs 2023-09-10T07:19:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:13 smithi170 ceph-mon[102832]: pgmap v15512: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 319 KiB/s wr, 29 op/s 2023-09-10T07:19:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28784.506936 secs 2023-09-10T07:19:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:13 smithi082 ceph-mon[98694]: pgmap v15512: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 319 KiB/s wr, 29 op/s 2023-09-10T07:19:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28784.506936 secs 2023-09-10T07:19:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:13 smithi098 ceph-mon[103124]: pgmap v15512: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 319 KiB/s wr, 29 op/s 2023-09-10T07:19:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28785.644386 secs 2023-09-10T07:19:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28786.121555 secs 2023-09-10T07:19:15.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28785.644386 secs 2023-09-10T07:19:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28786.121555 secs 2023-09-10T07:19:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28785.644386 secs 2023-09-10T07:19:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28786.121555 secs 2023-09-10T07:19:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:15 smithi170 ceph-mon[102832]: pgmap v15513: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 316 KiB/s wr, 28 op/s 2023-09-10T07:19:16.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:15 smithi082 ceph-mon[98694]: pgmap v15513: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 316 KiB/s wr, 28 op/s 2023-09-10T07:19:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:15 smithi098 ceph-mon[103124]: pgmap v15513: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 316 KiB/s wr, 28 op/s 2023-09-10T07:19:17.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:17 smithi082 ceph-mon[98694]: pgmap v15514: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 13 op/s 2023-09-10T07:19:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:17 smithi098 ceph-mon[103124]: pgmap v15514: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 13 op/s 2023-09-10T07:19:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:17 smithi170 ceph-mon[102832]: pgmap v15514: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 13 op/s 2023-09-10T07:19:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28789.506999 secs 2023-09-10T07:19:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28789.506999 secs 2023-09-10T07:19:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28789.506999 secs 2023-09-10T07:19:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28790.644451 secs 2023-09-10T07:19:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:19 smithi098 ceph-mon[103124]: pgmap v15515: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 13 op/s 2023-09-10T07:19:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28791.121621 secs 2023-09-10T07:19:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28790.644451 secs 2023-09-10T07:19:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:19 smithi170 ceph-mon[102832]: pgmap v15515: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 13 op/s 2023-09-10T07:19:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28791.121621 secs 2023-09-10T07:19:20.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28790.644451 secs 2023-09-10T07:19:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:19 smithi082 ceph-mon[98694]: pgmap v15515: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 13 op/s 2023-09-10T07:19:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28791.121621 secs 2023-09-10T07:19:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:21 smithi170 ceph-mon[102832]: pgmap v15516: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 197 KiB/s wr, 12 op/s 2023-09-10T07:19:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:21 smithi082 ceph-mon[98694]: pgmap v15516: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 197 KiB/s wr, 12 op/s 2023-09-10T07:19:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:21 smithi098 ceph-mon[103124]: pgmap v15516: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 197 KiB/s wr, 12 op/s 2023-09-10T07:19:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28794.507058 secs 2023-09-10T07:19:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:23 smithi170 ceph-mon[102832]: pgmap v15517: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T07:19:24.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28794.507058 secs 2023-09-10T07:19:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:23 smithi082 ceph-mon[98694]: pgmap v15517: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T07:19:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28794.507058 secs 2023-09-10T07:19:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:23 smithi098 ceph-mon[103124]: pgmap v15517: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T07:19:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28795.644496 secs 2023-09-10T07:19:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28796.121679 secs 2023-09-10T07:19:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28795.644496 secs 2023-09-10T07:19:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28796.121679 secs 2023-09-10T07:19:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28795.644496 secs 2023-09-10T07:19:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28796.121679 secs 2023-09-10T07:19:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:25 smithi170 ceph-mon[102832]: pgmap v15518: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 10 op/s 2023-09-10T07:19:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:25 smithi082 ceph-mon[98694]: pgmap v15518: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 10 op/s 2023-09-10T07:19:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:25 smithi098 ceph-mon[103124]: pgmap v15518: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 10 op/s 2023-09-10T07:19:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:27 smithi082 ceph-mon[98694]: pgmap v15519: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 164 KiB/s wr, 15 op/s 2023-09-10T07:19:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:27 smithi098 ceph-mon[103124]: pgmap v15519: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 164 KiB/s wr, 15 op/s 2023-09-10T07:19:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:27 smithi170 ceph-mon[102832]: pgmap v15519: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 164 KiB/s wr, 15 op/s 2023-09-10T07:19:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28799.507137 secs 2023-09-10T07:19:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28799.507137 secs 2023-09-10T07:19:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28799.507137 secs 2023-09-10T07:19:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28800.644554 secs 2023-09-10T07:19:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:29 smithi082 ceph-mon[98694]: pgmap v15520: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 10 op/s 2023-09-10T07:19:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28801.121746 secs 2023-09-10T07:19:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28800.644554 secs 2023-09-10T07:19:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:29 smithi098 ceph-mon[103124]: pgmap v15520: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 10 op/s 2023-09-10T07:19:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28801.121746 secs 2023-09-10T07:19:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28800.644554 secs 2023-09-10T07:19:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:29 smithi170 ceph-mon[102832]: pgmap v15520: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 10 op/s 2023-09-10T07:19:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28801.121746 secs 2023-09-10T07:19:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:31 smithi170 ceph-mon[102832]: pgmap v15521: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 291 KiB/s wr, 15 op/s 2023-09-10T07:19:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:31 smithi082 ceph-mon[98694]: pgmap v15521: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 291 KiB/s wr, 15 op/s 2023-09-10T07:19:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:31 smithi098 ceph-mon[103124]: pgmap v15521: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 291 KiB/s wr, 15 op/s 2023-09-10T07:19:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28804.507198 secs 2023-09-10T07:19:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:33 smithi170 ceph-mon[102832]: pgmap v15522: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 127 KiB/s wr, 10 op/s 2023-09-10T07:19:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28804.507198 secs 2023-09-10T07:19:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:33 smithi082 ceph-mon[98694]: pgmap v15522: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 127 KiB/s wr, 10 op/s 2023-09-10T07:19:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28804.507198 secs 2023-09-10T07:19:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:33 smithi098 ceph-mon[103124]: pgmap v15522: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 127 KiB/s wr, 10 op/s 2023-09-10T07:19:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28805.644617 secs 2023-09-10T07:19:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28806.121814 secs 2023-09-10T07:19:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28805.644617 secs 2023-09-10T07:19:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28806.121814 secs 2023-09-10T07:19:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28805.644617 secs 2023-09-10T07:19:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28806.121814 secs 2023-09-10T07:19:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:35 smithi170 ceph-mon[102832]: pgmap v15523: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 127 KiB/s wr, 10 op/s 2023-09-10T07:19:36.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:35 smithi082 ceph-mon[98694]: pgmap v15523: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 127 KiB/s wr, 10 op/s 2023-09-10T07:19:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:35 smithi098 ceph-mon[103124]: pgmap v15523: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 127 KiB/s wr, 10 op/s 2023-09-10T07:19:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:37 smithi082 ceph-mon[98694]: pgmap v15524: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 328 KiB/s wr, 16 op/s 2023-09-10T07:19:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:37 smithi098 ceph-mon[103124]: pgmap v15524: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 328 KiB/s wr, 16 op/s 2023-09-10T07:19:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:37 smithi170 ceph-mon[102832]: pgmap v15524: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 328 KiB/s wr, 16 op/s 2023-09-10T07:19:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28809.507260 secs 2023-09-10T07:19:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28809.507260 secs 2023-09-10T07:19:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28809.507260 secs 2023-09-10T07:19:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28810.644679 secs 2023-09-10T07:19:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:39 smithi098 ceph-mon[103124]: pgmap v15525: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 328 KiB/s wr, 11 op/s 2023-09-10T07:19:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28811.121878 secs 2023-09-10T07:19:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28810.644679 secs 2023-09-10T07:19:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:39 smithi170 ceph-mon[102832]: pgmap v15525: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 328 KiB/s wr, 11 op/s 2023-09-10T07:19:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28811.121878 secs 2023-09-10T07:19:40.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28810.644679 secs 2023-09-10T07:19:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:39 smithi082 ceph-mon[98694]: pgmap v15525: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 328 KiB/s wr, 11 op/s 2023-09-10T07:19:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28811.121878 secs 2023-09-10T07:19:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:41 smithi170 ceph-mon[102832]: pgmap v15526: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 544 KiB/s wr, 18 op/s 2023-09-10T07:19:42.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:41 smithi082 ceph-mon[98694]: pgmap v15526: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 544 KiB/s wr, 18 op/s 2023-09-10T07:19:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:41 smithi098 ceph-mon[103124]: pgmap v15526: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 544 KiB/s wr, 18 op/s 2023-09-10T07:19:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:43 smithi170 ceph-mon[102832]: pgmap v15527: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 417 KiB/s wr, 13 op/s 2023-09-10T07:19:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:19:44.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:43 smithi082 ceph-mon[98694]: pgmap v15527: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 417 KiB/s wr, 13 op/s 2023-09-10T07:19:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:19:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:43 smithi098 ceph-mon[103124]: pgmap v15527: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 417 KiB/s wr, 13 op/s 2023-09-10T07:19:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:19:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28815.644749 secs 2023-09-10T07:19:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28816.121935 secs 2023-09-10T07:19:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:44 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:19:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:19:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:19:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28815.644749 secs 2023-09-10T07:19:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28816.121935 secs 2023-09-10T07:19:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:44 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:19:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:19:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:19:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28815.644749 secs 2023-09-10T07:19:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28816.121935 secs 2023-09-10T07:19:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:44 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:19:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:19:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:19:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:45 smithi170 ceph-mon[102832]: pgmap v15528: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 417 KiB/s wr, 13 op/s 2023-09-10T07:19:46.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:45 smithi082 ceph-mon[98694]: pgmap v15528: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 417 KiB/s wr, 13 op/s 2023-09-10T07:19:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:45 smithi098 ceph-mon[103124]: pgmap v15528: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 417 KiB/s wr, 13 op/s 2023-09-10T07:19:47.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:47 smithi082 ceph-mon[98694]: pgmap v15529: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 612 KiB/s wr, 19 op/s 2023-09-10T07:19:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:47 smithi098 ceph-mon[103124]: pgmap v15529: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 612 KiB/s wr, 19 op/s 2023-09-10T07:19:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:47 smithi170 ceph-mon[102832]: pgmap v15529: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 612 KiB/s wr, 19 op/s 2023-09-10T07:19:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28820.644808 secs 2023-09-10T07:19:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28821.122001 secs 2023-09-10T07:19:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:49 smithi170 ceph-mon[102832]: pgmap v15530: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 411 KiB/s wr, 13 op/s 2023-09-10T07:19:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28820.644808 secs 2023-09-10T07:19:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28821.122001 secs 2023-09-10T07:19:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:49 smithi082 ceph-mon[98694]: pgmap v15530: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 411 KiB/s wr, 13 op/s 2023-09-10T07:19:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28820.644808 secs 2023-09-10T07:19:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28821.122001 secs 2023-09-10T07:19:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:49 smithi098 ceph-mon[103124]: pgmap v15530: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 411 KiB/s wr, 13 op/s 2023-09-10T07:19:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:51 smithi170 ceph-mon[102832]: pgmap v15531: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 619 KiB/s wr, 18 op/s 2023-09-10T07:19:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:51 smithi082 ceph-mon[98694]: pgmap v15531: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 619 KiB/s wr, 18 op/s 2023-09-10T07:19:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:51 smithi098 ceph-mon[103124]: pgmap v15531: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 619 KiB/s wr, 18 op/s 2023-09-10T07:19:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:53 smithi170 ceph-mon[102832]: pgmap v15532: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 402 KiB/s wr, 11 op/s 2023-09-10T07:19:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:53 smithi082 ceph-mon[98694]: pgmap v15532: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 402 KiB/s wr, 11 op/s 2023-09-10T07:19:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:53 smithi098 ceph-mon[103124]: pgmap v15532: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 402 KiB/s wr, 11 op/s 2023-09-10T07:19:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28825.644871 secs 2023-09-10T07:19:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28826.122060 secs 2023-09-10T07:19:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28825.644871 secs 2023-09-10T07:19:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28826.122060 secs 2023-09-10T07:19:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28825.644871 secs 2023-09-10T07:19:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28826.122060 secs 2023-09-10T07:19:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:55 smithi082 ceph-mon[98694]: pgmap v15533: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 402 KiB/s wr, 11 op/s 2023-09-10T07:19:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:55 smithi098 ceph-mon[103124]: pgmap v15533: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 402 KiB/s wr, 11 op/s 2023-09-10T07:19:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:55 smithi170 ceph-mon[102832]: pgmap v15533: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 402 KiB/s wr, 11 op/s 2023-09-10T07:19:57.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:19:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:19:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:19:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:57 smithi082 ceph-mon[98694]: pgmap v15534: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 613 KiB/s wr, 17 op/s 2023-09-10T07:19:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:19:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:19:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:19:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:57 smithi098 ceph-mon[103124]: pgmap v15534: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 613 KiB/s wr, 17 op/s 2023-09-10T07:19:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:19:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:19:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:19:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:57 smithi170 ceph-mon[102832]: pgmap v15534: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 613 KiB/s wr, 17 op/s 2023-09-10T07:19:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:19:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:19:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:19:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28829.507591 secs 2023-09-10T07:19:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28829.507591 secs 2023-09-10T07:19:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28829.507591 secs 2023-09-10T07:20:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28830.644930 secs 2023-09-10T07:20:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28831.122124 secs 2023-09-10T07:20:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:19:59 smithi082 ceph-mon[98694]: pgmap v15535: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 419 KiB/s wr, 11 op/s 2023-09-10T07:20:00.259 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28830.644930 secs 2023-09-10T07:20:00.259 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28831.122124 secs 2023-09-10T07:20:00.259 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:19:59 smithi098 ceph-mon[103124]: pgmap v15535: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 419 KiB/s wr, 11 op/s 2023-09-10T07:20:00.436 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28830.644930 secs 2023-09-10T07:20:00.437 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28831.122124 secs 2023-09-10T07:20:00.437 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:19:59 smithi170 ceph-mon[102832]: pgmap v15535: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 419 KiB/s wr, 11 op/s 2023-09-10T07:20:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:20:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:20:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:20:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:00 smithi082 ceph-mon[98694]: Health detail: HEALTH_WARN 3 MDSs report slow requests 2023-09-10T07:20:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:00 smithi082 ceph-mon[98694]: [WRN] MDS_SLOW_REQUEST: 3 MDSs report slow requests 2023-09-10T07:20:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:00 smithi082 ceph-mon[98694]: mds.l(mds.4): 3 slow requests are blocked > 180 secs 2023-09-10T07:20:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:00 smithi082 ceph-mon[98694]: mds.h(mds.1): 3 slow requests are blocked > 180 secs 2023-09-10T07:20:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:00 smithi082 ceph-mon[98694]: mds.b(mds.0): 1 slow requests are blocked > 180 secs 2023-09-10T07:20:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:20:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:20:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:20:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:00 smithi098 ceph-mon[103124]: Health detail: HEALTH_WARN 3 MDSs report slow requests 2023-09-10T07:20:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:00 smithi098 ceph-mon[103124]: [WRN] MDS_SLOW_REQUEST: 3 MDSs report slow requests 2023-09-10T07:20:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:00 smithi098 ceph-mon[103124]: mds.l(mds.4): 3 slow requests are blocked > 180 secs 2023-09-10T07:20:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:00 smithi098 ceph-mon[103124]: mds.h(mds.1): 3 slow requests are blocked > 180 secs 2023-09-10T07:20:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:00 smithi098 ceph-mon[103124]: mds.b(mds.0): 1 slow requests are blocked > 180 secs 2023-09-10T07:20:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:20:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:20:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:20:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:00 smithi170 ceph-mon[102832]: Health detail: HEALTH_WARN 3 MDSs report slow requests 2023-09-10T07:20:01.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:00 smithi170 ceph-mon[102832]: [WRN] MDS_SLOW_REQUEST: 3 MDSs report slow requests 2023-09-10T07:20:01.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:00 smithi170 ceph-mon[102832]: mds.l(mds.4): 3 slow requests are blocked > 180 secs 2023-09-10T07:20:01.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:00 smithi170 ceph-mon[102832]: mds.h(mds.1): 3 slow requests are blocked > 180 secs 2023-09-10T07:20:01.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:00 smithi170 ceph-mon[102832]: mds.b(mds.0): 1 slow requests are blocked > 180 secs 2023-09-10T07:20:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:01 smithi082 ceph-mon[98694]: pgmap v15536: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 624 KiB/s wr, 17 op/s 2023-09-10T07:20:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:01 smithi098 ceph-mon[103124]: pgmap v15536: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 624 KiB/s wr, 17 op/s 2023-09-10T07:20:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:01 smithi170 ceph-mon[102832]: pgmap v15536: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 624 KiB/s wr, 17 op/s 2023-09-10T07:20:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28834.507651 secs 2023-09-10T07:20:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:03 smithi082 ceph-mon[98694]: pgmap v15537: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 417 KiB/s wr, 12 op/s 2023-09-10T07:20:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:03 smithi082 ceph-mon[98694]: osdmap e1039: 12 total, 12 up, 12 in 2023-09-10T07:20:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28834.507651 secs 2023-09-10T07:20:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:03 smithi098 ceph-mon[103124]: pgmap v15537: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 417 KiB/s wr, 12 op/s 2023-09-10T07:20:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:03 smithi098 ceph-mon[103124]: osdmap e1039: 12 total, 12 up, 12 in 2023-09-10T07:20:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28834.507651 secs 2023-09-10T07:20:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:03 smithi170 ceph-mon[102832]: pgmap v15537: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 417 KiB/s wr, 12 op/s 2023-09-10T07:20:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:03 smithi170 ceph-mon[102832]: osdmap e1039: 12 total, 12 up, 12 in 2023-09-10T07:20:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28835.644989 secs 2023-09-10T07:20:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28836.122185 secs 2023-09-10T07:20:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28835.644989 secs 2023-09-10T07:20:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28836.122185 secs 2023-09-10T07:20:05.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28835.644989 secs 2023-09-10T07:20:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28836.122185 secs 2023-09-10T07:20:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:05 smithi098 ceph-mon[103124]: pgmap v15539: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 500 KiB/s wr, 14 op/s 2023-09-10T07:20:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:05 smithi170 ceph-mon[102832]: pgmap v15539: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 500 KiB/s wr, 14 op/s 2023-09-10T07:20:06.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:05 smithi082 ceph-mon[98694]: pgmap v15539: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 500 KiB/s wr, 14 op/s 2023-09-10T07:20:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:07 smithi098 ceph-mon[103124]: pgmap v15540: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 477 KiB/s wr, 32 op/s 2023-09-10T07:20:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:07 smithi170 ceph-mon[102832]: pgmap v15540: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 477 KiB/s wr, 32 op/s 2023-09-10T07:20:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:07 smithi082 ceph-mon[98694]: pgmap v15540: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 477 KiB/s wr, 32 op/s 2023-09-10T07:20:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28839.507709 secs 2023-09-10T07:20:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28839.507709 secs 2023-09-10T07:20:09.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28839.507709 secs 2023-09-10T07:20:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28840.645067 secs 2023-09-10T07:20:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28841.122249 secs 2023-09-10T07:20:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:09 smithi098 ceph-mon[103124]: pgmap v15541: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 477 KiB/s wr, 32 op/s 2023-09-10T07:20:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28840.645067 secs 2023-09-10T07:20:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28841.122249 secs 2023-09-10T07:20:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:09 smithi170 ceph-mon[102832]: pgmap v15541: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 477 KiB/s wr, 32 op/s 2023-09-10T07:20:10.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28840.645067 secs 2023-09-10T07:20:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28841.122249 secs 2023-09-10T07:20:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:09 smithi082 ceph-mon[98694]: pgmap v15541: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 477 KiB/s wr, 32 op/s 2023-09-10T07:20:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:11 smithi170 ceph-mon[102832]: pgmap v15542: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 326 KiB/s wr, 31 op/s 2023-09-10T07:20:12.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:11 smithi170 ceph-mon[102832]: osdmap e1040: 12 total, 12 up, 12 in 2023-09-10T07:20:12.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:11 smithi082 ceph-mon[98694]: pgmap v15542: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 326 KiB/s wr, 31 op/s 2023-09-10T07:20:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:11 smithi082 ceph-mon[98694]: osdmap e1040: 12 total, 12 up, 12 in 2023-09-10T07:20:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:11 smithi098 ceph-mon[103124]: pgmap v15542: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 326 KiB/s wr, 31 op/s 2023-09-10T07:20:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:11 smithi098 ceph-mon[103124]: osdmap e1040: 12 total, 12 up, 12 in 2023-09-10T07:20:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28844.507808 secs 2023-09-10T07:20:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:13 smithi170 ceph-mon[102832]: pgmap v15544: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 329 KiB/s wr, 31 op/s 2023-09-10T07:20:14.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28844.507808 secs 2023-09-10T07:20:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:13 smithi082 ceph-mon[98694]: pgmap v15544: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 329 KiB/s wr, 31 op/s 2023-09-10T07:20:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28844.507808 secs 2023-09-10T07:20:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:13 smithi098 ceph-mon[103124]: pgmap v15544: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 329 KiB/s wr, 31 op/s 2023-09-10T07:20:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28845.645125 secs 2023-09-10T07:20:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28846.122309 secs 2023-09-10T07:20:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28845.645125 secs 2023-09-10T07:20:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28846.122309 secs 2023-09-10T07:20:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28845.645125 secs 2023-09-10T07:20:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28846.122309 secs 2023-09-10T07:20:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:15 smithi082 ceph-mon[98694]: pgmap v15545: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 326 KiB/s wr, 31 op/s 2023-09-10T07:20:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:15 smithi098 ceph-mon[103124]: pgmap v15545: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 326 KiB/s wr, 31 op/s 2023-09-10T07:20:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:15 smithi170 ceph-mon[102832]: pgmap v15545: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 326 KiB/s wr, 31 op/s 2023-09-10T07:20:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:17 smithi098 ceph-mon[103124]: pgmap v15546: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T07:20:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:17 smithi170 ceph-mon[102832]: pgmap v15546: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T07:20:18.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:17 smithi082 ceph-mon[98694]: pgmap v15546: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T07:20:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28849.507868 secs 2023-09-10T07:20:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28849.507868 secs 2023-09-10T07:20:19.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28849.507868 secs 2023-09-10T07:20:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28850.645185 secs 2023-09-10T07:20:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28851.122371 secs 2023-09-10T07:20:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:19 smithi098 ceph-mon[103124]: pgmap v15547: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T07:20:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28850.645185 secs 2023-09-10T07:20:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28851.122371 secs 2023-09-10T07:20:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:19 smithi170 ceph-mon[102832]: pgmap v15547: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T07:20:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28850.645185 secs 2023-09-10T07:20:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28851.122371 secs 2023-09-10T07:20:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:19 smithi082 ceph-mon[98694]: pgmap v15547: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 12 op/s 2023-09-10T07:20:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:21 smithi170 ceph-mon[102832]: pgmap v15548: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 12 op/s 2023-09-10T07:20:22.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:21 smithi082 ceph-mon[98694]: pgmap v15548: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 12 op/s 2023-09-10T07:20:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:21 smithi098 ceph-mon[103124]: pgmap v15548: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 12 op/s 2023-09-10T07:20:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28854.507947 secs 2023-09-10T07:20:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:23 smithi082 ceph-mon[98694]: pgmap v15549: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 175 KiB/s wr, 11 op/s 2023-09-10T07:20:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28854.507947 secs 2023-09-10T07:20:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:23 smithi098 ceph-mon[103124]: pgmap v15549: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 175 KiB/s wr, 11 op/s 2023-09-10T07:20:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28854.507947 secs 2023-09-10T07:20:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:23 smithi170 ceph-mon[102832]: pgmap v15549: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 175 KiB/s wr, 11 op/s 2023-09-10T07:20:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28855.645275 secs 2023-09-10T07:20:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28856.122431 secs 2023-09-10T07:20:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28855.645275 secs 2023-09-10T07:20:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28856.122431 secs 2023-09-10T07:20:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28855.645275 secs 2023-09-10T07:20:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28856.122431 secs 2023-09-10T07:20:26.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:25 smithi082 ceph-mon[98694]: pgmap v15550: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T07:20:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:25 smithi098 ceph-mon[103124]: pgmap v15550: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T07:20:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:25 smithi170 ceph-mon[102832]: pgmap v15550: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T07:20:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:27 smithi098 ceph-mon[103124]: pgmap v15551: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 163 KiB/s wr, 15 op/s 2023-09-10T07:20:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:27 smithi170 ceph-mon[102832]: pgmap v15551: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 163 KiB/s wr, 15 op/s 2023-09-10T07:20:28.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:27 smithi082 ceph-mon[98694]: pgmap v15551: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 163 KiB/s wr, 15 op/s 2023-09-10T07:20:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28859.508039 secs 2023-09-10T07:20:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28859.508039 secs 2023-09-10T07:20:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28859.508039 secs 2023-09-10T07:20:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28860.645359 secs 2023-09-10T07:20:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28861.122483 secs 2023-09-10T07:20:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:29 smithi098 ceph-mon[103124]: pgmap v15552: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T07:20:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28860.645359 secs 2023-09-10T07:20:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28861.122483 secs 2023-09-10T07:20:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:29 smithi170 ceph-mon[102832]: pgmap v15552: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T07:20:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28860.645359 secs 2023-09-10T07:20:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28861.122483 secs 2023-09-10T07:20:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:29 smithi082 ceph-mon[98694]: pgmap v15552: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T07:20:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:31 smithi082 ceph-mon[98694]: pgmap v15553: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 259 KiB/s wr, 16 op/s 2023-09-10T07:20:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:31 smithi098 ceph-mon[103124]: pgmap v15553: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 259 KiB/s wr, 16 op/s 2023-09-10T07:20:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:31 smithi170 ceph-mon[102832]: pgmap v15553: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 259 KiB/s wr, 16 op/s 2023-09-10T07:20:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:33 smithi082 ceph-mon[98694]: pgmap v15554: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 10 op/s 2023-09-10T07:20:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:33 smithi098 ceph-mon[103124]: pgmap v15554: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 10 op/s 2023-09-10T07:20:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:33 smithi170 ceph-mon[102832]: pgmap v15554: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 10 op/s 2023-09-10T07:20:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28865.645442 secs 2023-09-10T07:20:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28866.122542 secs 2023-09-10T07:20:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28865.645442 secs 2023-09-10T07:20:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28866.122542 secs 2023-09-10T07:20:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28865.645442 secs 2023-09-10T07:20:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28866.122542 secs 2023-09-10T07:20:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:35 smithi082 ceph-mon[98694]: pgmap v15555: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 10 op/s 2023-09-10T07:20:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:20:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:35 smithi098 ceph-mon[103124]: pgmap v15555: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 10 op/s 2023-09-10T07:20:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:20:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:35 smithi170 ceph-mon[102832]: pgmap v15555: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 10 op/s 2023-09-10T07:20:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:20:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:20:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:20:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:20:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:20:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:20:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:20:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:20:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:20:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:20:38.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:37 smithi082 ceph-mon[98694]: pgmap v15556: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 315 KiB/s wr, 16 op/s 2023-09-10T07:20:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:37 smithi098 ceph-mon[103124]: pgmap v15556: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 315 KiB/s wr, 16 op/s 2023-09-10T07:20:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:37 smithi170 ceph-mon[102832]: pgmap v15556: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 315 KiB/s wr, 16 op/s 2023-09-10T07:20:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28870.645501 secs 2023-09-10T07:20:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28871.122603 secs 2023-09-10T07:20:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:39 smithi082 ceph-mon[98694]: pgmap v15557: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 315 KiB/s wr, 11 op/s 2023-09-10T07:20:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28870.645501 secs 2023-09-10T07:20:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28871.122603 secs 2023-09-10T07:20:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:39 smithi098 ceph-mon[103124]: pgmap v15557: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 315 KiB/s wr, 11 op/s 2023-09-10T07:20:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28870.645501 secs 2023-09-10T07:20:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28871.122603 secs 2023-09-10T07:20:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:39 smithi170 ceph-mon[102832]: pgmap v15557: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 315 KiB/s wr, 11 op/s 2023-09-10T07:20:42.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:41 smithi082 ceph-mon[98694]: pgmap v15558: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 527 KiB/s wr, 17 op/s 2023-09-10T07:20:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:41 smithi098 ceph-mon[103124]: pgmap v15558: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 527 KiB/s wr, 17 op/s 2023-09-10T07:20:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:41 smithi170 ceph-mon[102832]: pgmap v15558: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 527 KiB/s wr, 17 op/s 2023-09-10T07:20:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28874.508337 secs 2023-09-10T07:20:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:43 smithi082 ceph-mon[98694]: pgmap v15559: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 430 KiB/s wr, 11 op/s 2023-09-10T07:20:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28874.508337 secs 2023-09-10T07:20:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:43 smithi098 ceph-mon[103124]: pgmap v15559: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 430 KiB/s wr, 11 op/s 2023-09-10T07:20:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28874.508337 secs 2023-09-10T07:20:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:43 smithi170 ceph-mon[102832]: pgmap v15559: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 430 KiB/s wr, 11 op/s 2023-09-10T07:20:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28875.645558 secs 2023-09-10T07:20:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28876.122662 secs 2023-09-10T07:20:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:20:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:20:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:20:45.215 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28875.645558 secs 2023-09-10T07:20:45.215 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28876.122662 secs 2023-09-10T07:20:45.216 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:20:45.216 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:20:45.216 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:20:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28875.645558 secs 2023-09-10T07:20:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28876.122662 secs 2023-09-10T07:20:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:20:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:20:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:20:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:45 smithi098 ceph-mon[103124]: pgmap v15560: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 430 KiB/s wr, 11 op/s 2023-09-10T07:20:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:45 smithi170 ceph-mon[102832]: pgmap v15560: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 430 KiB/s wr, 11 op/s 2023-09-10T07:20:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:45 smithi082 ceph-mon[98694]: pgmap v15560: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 430 KiB/s wr, 11 op/s 2023-09-10T07:20:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:47 smithi098 ceph-mon[103124]: pgmap v15561: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 609 KiB/s wr, 18 op/s 2023-09-10T07:20:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:47 smithi170 ceph-mon[102832]: pgmap v15561: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 609 KiB/s wr, 18 op/s 2023-09-10T07:20:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:47 smithi082 ceph-mon[98694]: pgmap v15561: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 609 KiB/s wr, 18 op/s 2023-09-10T07:20:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28879.508442 secs 2023-09-10T07:20:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28879.508442 secs 2023-09-10T07:20:49.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28879.508442 secs 2023-09-10T07:20:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28880.645646 secs 2023-09-10T07:20:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28881.122730 secs 2023-09-10T07:20:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:49 smithi098 ceph-mon[103124]: pgmap v15562: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 391 KiB/s wr, 12 op/s 2023-09-10T07:20:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28880.645646 secs 2023-09-10T07:20:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28881.122730 secs 2023-09-10T07:20:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:49 smithi170 ceph-mon[102832]: pgmap v15562: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 391 KiB/s wr, 12 op/s 2023-09-10T07:20:50.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28880.645646 secs 2023-09-10T07:20:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28881.122730 secs 2023-09-10T07:20:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:49 smithi082 ceph-mon[98694]: pgmap v15562: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 391 KiB/s wr, 12 op/s 2023-09-10T07:20:52.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:51 smithi082 ceph-mon[98694]: pgmap v15563: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 597 KiB/s wr, 18 op/s 2023-09-10T07:20:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:51 smithi098 ceph-mon[103124]: pgmap v15563: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 597 KiB/s wr, 18 op/s 2023-09-10T07:20:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:51 smithi170 ceph-mon[102832]: pgmap v15563: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 597 KiB/s wr, 18 op/s 2023-09-10T07:20:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:53 smithi082 ceph-mon[98694]: pgmap v15564: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 385 KiB/s wr, 12 op/s 2023-09-10T07:20:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:53 smithi098 ceph-mon[103124]: pgmap v15564: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 385 KiB/s wr, 12 op/s 2023-09-10T07:20:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:53 smithi170 ceph-mon[102832]: pgmap v15564: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 385 KiB/s wr, 12 op/s 2023-09-10T07:20:55.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28885.645707 secs 2023-09-10T07:20:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28886.122789 secs 2023-09-10T07:20:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28885.645707 secs 2023-09-10T07:20:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28886.122789 secs 2023-09-10T07:20:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28885.645707 secs 2023-09-10T07:20:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28886.122789 secs 2023-09-10T07:20:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:55 smithi082 ceph-mon[98694]: pgmap v15565: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 385 KiB/s wr, 12 op/s 2023-09-10T07:20:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:55 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:20:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:55 smithi098 ceph-mon[103124]: pgmap v15565: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 385 KiB/s wr, 12 op/s 2023-09-10T07:20:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:55 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:20:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:55 smithi170 ceph-mon[102832]: pgmap v15565: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 385 KiB/s wr, 12 op/s 2023-09-10T07:20:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:55 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:20:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:56 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:20:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:20:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:20:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:56 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:20:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:20:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:20:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:56 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:20:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:20:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:20:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:57 smithi082 ceph-mon[98694]: pgmap v15566: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 582 KiB/s wr, 18 op/s 2023-09-10T07:20:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:20:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:20:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:20:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:20:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:57 smithi098 ceph-mon[103124]: pgmap v15566: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 582 KiB/s wr, 18 op/s 2023-09-10T07:20:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:20:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:20:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:20:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:20:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:57 smithi170 ceph-mon[102832]: pgmap v15566: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 582 KiB/s wr, 18 op/s 2023-09-10T07:20:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:20:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:20:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:20:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:20:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28889.508583 secs 2023-09-10T07:20:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28889.508583 secs 2023-09-10T07:20:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28889.508583 secs 2023-09-10T07:21:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28890.645765 secs 2023-09-10T07:21:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28891.122853 secs 2023-09-10T07:21:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:20:59 smithi098 ceph-mon[103124]: pgmap v15567: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 404 KiB/s wr, 12 op/s 2023-09-10T07:21:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28890.645765 secs 2023-09-10T07:21:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28891.122853 secs 2023-09-10T07:21:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:20:59 smithi170 ceph-mon[102832]: pgmap v15567: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 404 KiB/s wr, 12 op/s 2023-09-10T07:21:00.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28890.645765 secs 2023-09-10T07:21:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28891.122853 secs 2023-09-10T07:21:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:20:59 smithi082 ceph-mon[98694]: pgmap v15567: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 404 KiB/s wr, 12 op/s 2023-09-10T07:21:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:21:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:21:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:21:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:21:01.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:21:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:21:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:01 smithi098 ceph-mon[103124]: pgmap v15568: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 499 KiB/s wr, 17 op/s 2023-09-10T07:21:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:01 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:21:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:01 smithi170 ceph-mon[102832]: pgmap v15568: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 499 KiB/s wr, 17 op/s 2023-09-10T07:21:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:01 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:21:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:01 smithi082 ceph-mon[98694]: pgmap v15568: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 499 KiB/s wr, 17 op/s 2023-09-10T07:21:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:01 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:21:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28894.508641 secs 2023-09-10T07:21:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:03 smithi098 ceph-mon[103124]: pgmap v15569: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T07:21:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28894.508641 secs 2023-09-10T07:21:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:03 smithi170 ceph-mon[102832]: pgmap v15569: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T07:21:04.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28894.508641 secs 2023-09-10T07:21:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:03 smithi082 ceph-mon[98694]: pgmap v15569: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T07:21:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28895.645847 secs 2023-09-10T07:21:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28896.122912 secs 2023-09-10T07:21:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28895.645847 secs 2023-09-10T07:21:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28896.122912 secs 2023-09-10T07:21:05.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28895.645847 secs 2023-09-10T07:21:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28896.122912 secs 2023-09-10T07:21:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:05 smithi098 ceph-mon[103124]: pgmap v15570: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T07:21:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:05 smithi170 ceph-mon[102832]: pgmap v15570: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T07:21:06.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:05 smithi082 ceph-mon[98694]: pgmap v15570: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T07:21:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:07 smithi098 ceph-mon[103124]: pgmap v15571: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 479 KiB/s wr, 32 op/s 2023-09-10T07:21:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:07 smithi170 ceph-mon[102832]: pgmap v15571: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 479 KiB/s wr, 32 op/s 2023-09-10T07:21:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:07 smithi082 ceph-mon[98694]: pgmap v15571: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 479 KiB/s wr, 32 op/s 2023-09-10T07:21:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28899.508701 secs 2023-09-10T07:21:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28899.508701 secs 2023-09-10T07:21:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28899.508701 secs 2023-09-10T07:21:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28900.645905 secs 2023-09-10T07:21:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:09 smithi098 ceph-mon[103124]: osdmap e1041: 12 total, 12 up, 12 in 2023-09-10T07:21:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28901.122976 secs 2023-09-10T07:21:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:09 smithi098 ceph-mon[103124]: pgmap v15573: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 338 KiB/s wr, 30 op/s 2023-09-10T07:21:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28900.645905 secs 2023-09-10T07:21:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:09 smithi170 ceph-mon[102832]: osdmap e1041: 12 total, 12 up, 12 in 2023-09-10T07:21:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28901.122976 secs 2023-09-10T07:21:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:09 smithi170 ceph-mon[102832]: pgmap v15573: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 338 KiB/s wr, 30 op/s 2023-09-10T07:21:10.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28900.645905 secs 2023-09-10T07:21:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:09 smithi082 ceph-mon[98694]: osdmap e1041: 12 total, 12 up, 12 in 2023-09-10T07:21:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28901.122976 secs 2023-09-10T07:21:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:09 smithi082 ceph-mon[98694]: pgmap v15573: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 338 KiB/s wr, 30 op/s 2023-09-10T07:21:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:11 smithi082 ceph-mon[98694]: pgmap v15574: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 345 KiB/s wr, 30 op/s 2023-09-10T07:21:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:11 smithi082 ceph-mon[98694]: osdmap e1042: 12 total, 12 up, 12 in 2023-09-10T07:21:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:11 smithi098 ceph-mon[103124]: pgmap v15574: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 345 KiB/s wr, 30 op/s 2023-09-10T07:21:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:11 smithi098 ceph-mon[103124]: osdmap e1042: 12 total, 12 up, 12 in 2023-09-10T07:21:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:11 smithi170 ceph-mon[102832]: pgmap v15574: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 345 KiB/s wr, 30 op/s 2023-09-10T07:21:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:11 smithi170 ceph-mon[102832]: osdmap e1042: 12 total, 12 up, 12 in 2023-09-10T07:21:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28904.508781 secs 2023-09-10T07:21:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:13 smithi082 ceph-mon[98694]: pgmap v15576: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 432 KiB/s wr, 38 op/s 2023-09-10T07:21:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28904.508781 secs 2023-09-10T07:21:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:13 smithi098 ceph-mon[103124]: pgmap v15576: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 432 KiB/s wr, 38 op/s 2023-09-10T07:21:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28904.508781 secs 2023-09-10T07:21:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:13 smithi170 ceph-mon[102832]: pgmap v15576: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 432 KiB/s wr, 38 op/s 2023-09-10T07:21:15.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28905.645963 secs 2023-09-10T07:21:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28906.123038 secs 2023-09-10T07:21:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28905.645963 secs 2023-09-10T07:21:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28906.123038 secs 2023-09-10T07:21:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28905.645963 secs 2023-09-10T07:21:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28906.123038 secs 2023-09-10T07:21:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:15 smithi082 ceph-mon[98694]: pgmap v15577: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 152 KiB/s wr, 8 op/s 2023-09-10T07:21:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:15 smithi098 ceph-mon[103124]: pgmap v15577: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 152 KiB/s wr, 8 op/s 2023-09-10T07:21:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:15 smithi170 ceph-mon[102832]: pgmap v15577: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 152 KiB/s wr, 8 op/s 2023-09-10T07:21:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:17 smithi098 ceph-mon[103124]: pgmap v15578: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 16 op/s 2023-09-10T07:21:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:17 smithi170 ceph-mon[102832]: pgmap v15578: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 16 op/s 2023-09-10T07:21:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:17 smithi082 ceph-mon[98694]: pgmap v15578: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 16 op/s 2023-09-10T07:21:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28909.508854 secs 2023-09-10T07:21:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28909.508854 secs 2023-09-10T07:21:19.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28909.508854 secs 2023-09-10T07:21:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28910.646023 secs 2023-09-10T07:21:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28911.123104 secs 2023-09-10T07:21:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:19 smithi098 ceph-mon[103124]: pgmap v15579: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 13 op/s 2023-09-10T07:21:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28910.646023 secs 2023-09-10T07:21:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28911.123104 secs 2023-09-10T07:21:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:19 smithi170 ceph-mon[102832]: pgmap v15579: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 13 op/s 2023-09-10T07:21:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28910.646023 secs 2023-09-10T07:21:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28911.123104 secs 2023-09-10T07:21:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:19 smithi082 ceph-mon[98694]: pgmap v15579: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 13 op/s 2023-09-10T07:21:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:21 smithi082 ceph-mon[98694]: pgmap v15580: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T07:21:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:21 smithi098 ceph-mon[103124]: pgmap v15580: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T07:21:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:21 smithi170 ceph-mon[102832]: pgmap v15580: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T07:21:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28914.508918 secs 2023-09-10T07:21:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:23 smithi082 ceph-mon[98694]: pgmap v15581: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 11 op/s 2023-09-10T07:21:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28914.508918 secs 2023-09-10T07:21:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:23 smithi098 ceph-mon[103124]: pgmap v15581: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 11 op/s 2023-09-10T07:21:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28914.508918 secs 2023-09-10T07:21:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:23 smithi170 ceph-mon[102832]: pgmap v15581: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 11 op/s 2023-09-10T07:21:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28915.646079 secs 2023-09-10T07:21:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28916.123163 secs 2023-09-10T07:21:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28915.646079 secs 2023-09-10T07:21:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28916.123163 secs 2023-09-10T07:21:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28915.646079 secs 2023-09-10T07:21:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28916.123163 secs 2023-09-10T07:21:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:25 smithi082 ceph-mon[98694]: pgmap v15582: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 101 KiB/s wr, 10 op/s 2023-09-10T07:21:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:25 smithi098 ceph-mon[103124]: pgmap v15582: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 101 KiB/s wr, 10 op/s 2023-09-10T07:21:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:25 smithi170 ceph-mon[102832]: pgmap v15582: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 101 KiB/s wr, 10 op/s 2023-09-10T07:21:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:27 smithi098 ceph-mon[103124]: pgmap v15583: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 101 KiB/s wr, 15 op/s 2023-09-10T07:21:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:27 smithi170 ceph-mon[102832]: pgmap v15583: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 101 KiB/s wr, 15 op/s 2023-09-10T07:21:28.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:27 smithi082 ceph-mon[98694]: pgmap v15583: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 101 KiB/s wr, 15 op/s 2023-09-10T07:21:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28919.508979 secs 2023-09-10T07:21:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28919.508979 secs 2023-09-10T07:21:29.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28919.508979 secs 2023-09-10T07:21:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28920.646142 secs 2023-09-10T07:21:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28921.123227 secs 2023-09-10T07:21:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:29 smithi098 ceph-mon[103124]: pgmap v15584: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 10 op/s 2023-09-10T07:21:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28920.646142 secs 2023-09-10T07:21:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28921.123227 secs 2023-09-10T07:21:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:29 smithi170 ceph-mon[102832]: pgmap v15584: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 10 op/s 2023-09-10T07:21:30.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28920.646142 secs 2023-09-10T07:21:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28921.123227 secs 2023-09-10T07:21:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:29 smithi082 ceph-mon[98694]: pgmap v15584: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 10 op/s 2023-09-10T07:21:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:31 smithi082 ceph-mon[98694]: pgmap v15585: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 293 KiB/s wr, 15 op/s 2023-09-10T07:21:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:31 smithi098 ceph-mon[103124]: pgmap v15585: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 293 KiB/s wr, 15 op/s 2023-09-10T07:21:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:31 smithi170 ceph-mon[102832]: pgmap v15585: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 293 KiB/s wr, 15 op/s 2023-09-10T07:21:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:33 smithi082 ceph-mon[98694]: pgmap v15586: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 192 KiB/s wr, 10 op/s 2023-09-10T07:21:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:33 smithi098 ceph-mon[103124]: pgmap v15586: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 192 KiB/s wr, 10 op/s 2023-09-10T07:21:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:33 smithi170 ceph-mon[102832]: pgmap v15586: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 192 KiB/s wr, 10 op/s 2023-09-10T07:21:35.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28925.646225 secs 2023-09-10T07:21:35.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28926.123286 secs 2023-09-10T07:21:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28925.646225 secs 2023-09-10T07:21:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28926.123286 secs 2023-09-10T07:21:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28925.646225 secs 2023-09-10T07:21:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28926.123286 secs 2023-09-10T07:21:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:35 smithi082 ceph-mon[98694]: pgmap v15587: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 192 KiB/s wr, 10 op/s 2023-09-10T07:21:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:21:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:35 smithi098 ceph-mon[103124]: pgmap v15587: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 192 KiB/s wr, 10 op/s 2023-09-10T07:21:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:21:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:35 smithi170 ceph-mon[102832]: pgmap v15587: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 192 KiB/s wr, 10 op/s 2023-09-10T07:21:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:21:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:21:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:21:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:21:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:21:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:21:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:21:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:21:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:21:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:21:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:37 smithi098 ceph-mon[103124]: pgmap v15588: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 317 KiB/s wr, 16 op/s 2023-09-10T07:21:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:37 smithi170 ceph-mon[102832]: pgmap v15588: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 317 KiB/s wr, 16 op/s 2023-09-10T07:21:38.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:37 smithi082 ceph-mon[98694]: pgmap v15588: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 317 KiB/s wr, 16 op/s 2023-09-10T07:21:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28930.646289 secs 2023-09-10T07:21:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28931.123353 secs 2023-09-10T07:21:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:39 smithi098 ceph-mon[103124]: pgmap v15589: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 317 KiB/s wr, 11 op/s 2023-09-10T07:21:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28930.646289 secs 2023-09-10T07:21:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28931.123353 secs 2023-09-10T07:21:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:39 smithi170 ceph-mon[102832]: pgmap v15589: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 317 KiB/s wr, 11 op/s 2023-09-10T07:21:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28930.646289 secs 2023-09-10T07:21:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28931.123353 secs 2023-09-10T07:21:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:39 smithi082 ceph-mon[98694]: pgmap v15589: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 317 KiB/s wr, 11 op/s 2023-09-10T07:21:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:41 smithi098 ceph-mon[103124]: pgmap v15590: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 530 KiB/s wr, 18 op/s 2023-09-10T07:21:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:41 smithi170 ceph-mon[102832]: pgmap v15590: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 530 KiB/s wr, 18 op/s 2023-09-10T07:21:42.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:41 smithi082 ceph-mon[98694]: pgmap v15590: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 530 KiB/s wr, 18 op/s 2023-09-10T07:21:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28934.509159 secs 2023-09-10T07:21:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:43 smithi098 ceph-mon[103124]: pgmap v15591: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 338 KiB/s wr, 12 op/s 2023-09-10T07:21:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28934.509159 secs 2023-09-10T07:21:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:43 smithi170 ceph-mon[102832]: pgmap v15591: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 338 KiB/s wr, 12 op/s 2023-09-10T07:21:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28934.509159 secs 2023-09-10T07:21:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:43 smithi082 ceph-mon[98694]: pgmap v15591: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 338 KiB/s wr, 12 op/s 2023-09-10T07:21:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28935.646348 secs 2023-09-10T07:21:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28936.123411 secs 2023-09-10T07:21:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:21:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:21:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:21:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28935.646348 secs 2023-09-10T07:21:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28936.123411 secs 2023-09-10T07:21:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:21:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:21:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:21:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28935.646348 secs 2023-09-10T07:21:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28936.123411 secs 2023-09-10T07:21:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:21:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:21:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:21:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:45 smithi098 ceph-mon[103124]: pgmap v15592: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 338 KiB/s wr, 12 op/s 2023-09-10T07:21:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:45 smithi170 ceph-mon[102832]: pgmap v15592: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 338 KiB/s wr, 12 op/s 2023-09-10T07:21:46.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:45 smithi082 ceph-mon[98694]: pgmap v15592: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 338 KiB/s wr, 12 op/s 2023-09-10T07:21:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:47 smithi098 ceph-mon[103124]: pgmap v15593: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 518 KiB/s wr, 19 op/s 2023-09-10T07:21:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:47 smithi170 ceph-mon[102832]: pgmap v15593: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 518 KiB/s wr, 19 op/s 2023-09-10T07:21:48.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:47 smithi082 ceph-mon[98694]: pgmap v15593: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 518 KiB/s wr, 19 op/s 2023-09-10T07:21:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28939.509220 secs 2023-09-10T07:21:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28939.509220 secs 2023-09-10T07:21:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28939.509220 secs 2023-09-10T07:21:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28940.646409 secs 2023-09-10T07:21:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28941.123479 secs 2023-09-10T07:21:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:49 smithi098 ceph-mon[103124]: pgmap v15594: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T07:21:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28940.646409 secs 2023-09-10T07:21:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28941.123479 secs 2023-09-10T07:21:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:49 smithi170 ceph-mon[102832]: pgmap v15594: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T07:21:50.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28940.646409 secs 2023-09-10T07:21:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28941.123479 secs 2023-09-10T07:21:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:49 smithi082 ceph-mon[98694]: pgmap v15594: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T07:21:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:51 smithi082 ceph-mon[98694]: pgmap v15595: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 600 KiB/s wr, 19 op/s 2023-09-10T07:21:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:51 smithi098 ceph-mon[103124]: pgmap v15595: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 600 KiB/s wr, 19 op/s 2023-09-10T07:21:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:51 smithi170 ceph-mon[102832]: pgmap v15595: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 600 KiB/s wr, 19 op/s 2023-09-10T07:21:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:53 smithi082 ceph-mon[98694]: pgmap v15596: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 387 KiB/s wr, 12 op/s 2023-09-10T07:21:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:53 smithi098 ceph-mon[103124]: pgmap v15596: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 387 KiB/s wr, 12 op/s 2023-09-10T07:21:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:53 smithi170 ceph-mon[102832]: pgmap v15596: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 387 KiB/s wr, 12 op/s 2023-09-10T07:21:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28945.646501 secs 2023-09-10T07:21:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28946.123537 secs 2023-09-10T07:21:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28945.646501 secs 2023-09-10T07:21:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28946.123537 secs 2023-09-10T07:21:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28945.646501 secs 2023-09-10T07:21:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28946.123537 secs 2023-09-10T07:21:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:55 smithi098 ceph-mon[103124]: pgmap v15597: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 387 KiB/s wr, 12 op/s 2023-09-10T07:21:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:55 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:21:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:55 smithi170 ceph-mon[102832]: pgmap v15597: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 387 KiB/s wr, 12 op/s 2023-09-10T07:21:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:55 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:21:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:55 smithi082 ceph-mon[98694]: pgmap v15597: 97 pgs: 97 active+clean; 9.7 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 387 KiB/s wr, 12 op/s 2023-09-10T07:21:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:55 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:21:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:56 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:21:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:21:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:21:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:56 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:21:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:21:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:21:57.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:56 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:21:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:21:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:21:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:57 smithi098 ceph-mon[103124]: pgmap v15598: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 570 KiB/s wr, 19 op/s 2023-09-10T07:21:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:57 smithi170 ceph-mon[102832]: pgmap v15598: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 570 KiB/s wr, 19 op/s 2023-09-10T07:21:58.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:57 smithi082 ceph-mon[98694]: pgmap v15598: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 570 KiB/s wr, 19 op/s 2023-09-10T07:21:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28949.509379 secs 2023-09-10T07:21:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:21:59.371 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28949.509379 secs 2023-09-10T07:21:59.372 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:21:59.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28949.509379 secs 2023-09-10T07:21:59.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:22:00.071 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28950.646560 secs 2023-09-10T07:22:00.071 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28951.123605 secs 2023-09-10T07:22:00.072 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:21:59 smithi098 ceph-mon[103124]: pgmap v15599: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 390 KiB/s wr, 12 op/s 2023-09-10T07:22:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28950.646560 secs 2023-09-10T07:22:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28951.123605 secs 2023-09-10T07:22:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:21:59 smithi170 ceph-mon[102832]: pgmap v15599: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 390 KiB/s wr, 12 op/s 2023-09-10T07:22:00.263 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28950.646560 secs 2023-09-10T07:22:00.264 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28951.123605 secs 2023-09-10T07:22:00.264 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:21:59 smithi082 ceph-mon[98694]: pgmap v15599: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 390 KiB/s wr, 12 op/s 2023-09-10T07:22:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:22:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:22:01.401 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:22:01.402 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:22:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:22:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:22:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:01 smithi170 ceph-mon[102832]: pgmap v15600: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 485 KiB/s wr, 18 op/s 2023-09-10T07:22:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:22:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:22:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:01 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:22:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:22:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:22:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:01 smithi098 ceph-mon[103124]: pgmap v15600: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 485 KiB/s wr, 18 op/s 2023-09-10T07:22:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:22:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:22:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:01 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:22:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:22:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:22:02.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:01 smithi082 ceph-mon[98694]: pgmap v15600: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 485 KiB/s wr, 18 op/s 2023-09-10T07:22:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:22:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:22:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:01 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:22:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:22:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:22:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:22:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:22:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:22:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:22:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:22:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:22:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:22:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:22:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:22:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:22:03.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:22:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:22:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:22:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:22:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:22:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28954.509438 secs 2023-09-10T07:22:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:04 smithi170 ceph-mon[102832]: pgmap v15601: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T07:22:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28954.509438 secs 2023-09-10T07:22:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:04 smithi082 ceph-mon[98694]: pgmap v15601: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T07:22:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28954.509438 secs 2023-09-10T07:22:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:04 smithi098 ceph-mon[103124]: pgmap v15601: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T07:22:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28955.646618 secs 2023-09-10T07:22:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28956.123663 secs 2023-09-10T07:22:05.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28955.646618 secs 2023-09-10T07:22:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28956.123663 secs 2023-09-10T07:22:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28955.646618 secs 2023-09-10T07:22:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28956.123663 secs 2023-09-10T07:22:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:06 smithi170 ceph-mon[102832]: pgmap v15602: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T07:22:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:06 smithi082 ceph-mon[98694]: pgmap v15602: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T07:22:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:06 smithi098 ceph-mon[103124]: pgmap v15602: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T07:22:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:07 smithi098 ceph-mon[103124]: pgmap v15603: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 465 KiB/s wr, 32 op/s 2023-09-10T07:22:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:07 smithi170 ceph-mon[102832]: pgmap v15603: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 465 KiB/s wr, 32 op/s 2023-09-10T07:22:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:07 smithi082 ceph-mon[98694]: pgmap v15603: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 465 KiB/s wr, 32 op/s 2023-09-10T07:22:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28959.509540 secs 2023-09-10T07:22:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28959.509540 secs 2023-09-10T07:22:09.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28959.509540 secs 2023-09-10T07:22:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28960.646684 secs 2023-09-10T07:22:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:09 smithi098 ceph-mon[103124]: osdmap e1043: 12 total, 12 up, 12 in 2023-09-10T07:22:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28961.123732 secs 2023-09-10T07:22:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:09 smithi098 ceph-mon[103124]: pgmap v15605: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 339 KiB/s wr, 30 op/s 2023-09-10T07:22:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28960.646684 secs 2023-09-10T07:22:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:09 smithi170 ceph-mon[102832]: osdmap e1043: 12 total, 12 up, 12 in 2023-09-10T07:22:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28961.123732 secs 2023-09-10T07:22:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:09 smithi170 ceph-mon[102832]: pgmap v15605: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 339 KiB/s wr, 30 op/s 2023-09-10T07:22:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28960.646684 secs 2023-09-10T07:22:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:09 smithi082 ceph-mon[98694]: osdmap e1043: 12 total, 12 up, 12 in 2023-09-10T07:22:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28961.123732 secs 2023-09-10T07:22:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:09 smithi082 ceph-mon[98694]: pgmap v15605: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 339 KiB/s wr, 30 op/s 2023-09-10T07:22:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:11 smithi098 ceph-mon[103124]: pgmap v15606: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 346 KiB/s wr, 30 op/s 2023-09-10T07:22:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:11 smithi098 ceph-mon[103124]: osdmap e1044: 12 total, 12 up, 12 in 2023-09-10T07:22:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:11 smithi170 ceph-mon[102832]: pgmap v15606: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 346 KiB/s wr, 30 op/s 2023-09-10T07:22:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:11 smithi170 ceph-mon[102832]: osdmap e1044: 12 total, 12 up, 12 in 2023-09-10T07:22:12.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:11 smithi082 ceph-mon[98694]: pgmap v15606: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 346 KiB/s wr, 30 op/s 2023-09-10T07:22:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:11 smithi082 ceph-mon[98694]: osdmap e1044: 12 total, 12 up, 12 in 2023-09-10T07:22:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28964.509684 secs 2023-09-10T07:22:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:13 smithi098 ceph-mon[103124]: pgmap v15608: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 432 KiB/s wr, 37 op/s 2023-09-10T07:22:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28964.509684 secs 2023-09-10T07:22:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:13 smithi170 ceph-mon[102832]: pgmap v15608: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 432 KiB/s wr, 37 op/s 2023-09-10T07:22:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28964.509684 secs 2023-09-10T07:22:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:13 smithi082 ceph-mon[98694]: pgmap v15608: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 432 KiB/s wr, 37 op/s 2023-09-10T07:22:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28965.646740 secs 2023-09-10T07:22:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28966.123790 secs 2023-09-10T07:22:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28965.646740 secs 2023-09-10T07:22:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28966.123790 secs 2023-09-10T07:22:15.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28965.646740 secs 2023-09-10T07:22:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28966.123790 secs 2023-09-10T07:22:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:15 smithi098 ceph-mon[103124]: pgmap v15609: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 152 KiB/s wr, 8 op/s 2023-09-10T07:22:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:15 smithi170 ceph-mon[102832]: pgmap v15609: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 152 KiB/s wr, 8 op/s 2023-09-10T07:22:16.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:15 smithi082 ceph-mon[98694]: pgmap v15609: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 152 KiB/s wr, 8 op/s 2023-09-10T07:22:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:17 smithi098 ceph-mon[103124]: pgmap v15610: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 15 op/s 2023-09-10T07:22:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:17 smithi170 ceph-mon[102832]: pgmap v15610: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 15 op/s 2023-09-10T07:22:18.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:17 smithi082 ceph-mon[98694]: pgmap v15610: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 15 op/s 2023-09-10T07:22:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28969.509806 secs 2023-09-10T07:22:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28969.509806 secs 2023-09-10T07:22:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28969.509806 secs 2023-09-10T07:22:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28970.646799 secs 2023-09-10T07:22:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28971.123873 secs 2023-09-10T07:22:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:19 smithi170 ceph-mon[102832]: pgmap v15611: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T07:22:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28970.646799 secs 2023-09-10T07:22:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28971.123873 secs 2023-09-10T07:22:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:19 smithi082 ceph-mon[98694]: pgmap v15611: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T07:22:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28970.646799 secs 2023-09-10T07:22:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28971.123873 secs 2023-09-10T07:22:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:19 smithi098 ceph-mon[103124]: pgmap v15611: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T07:22:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:21 smithi098 ceph-mon[103124]: pgmap v15612: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T07:22:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:21 smithi170 ceph-mon[102832]: pgmap v15612: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T07:22:22.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:21 smithi082 ceph-mon[98694]: pgmap v15612: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T07:22:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28974.509869 secs 2023-09-10T07:22:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:23 smithi098 ceph-mon[103124]: pgmap v15613: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 109 KiB/s wr, 11 op/s 2023-09-10T07:22:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28974.509869 secs 2023-09-10T07:22:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:23 smithi170 ceph-mon[102832]: pgmap v15613: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 109 KiB/s wr, 11 op/s 2023-09-10T07:22:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28974.509869 secs 2023-09-10T07:22:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:23 smithi082 ceph-mon[98694]: pgmap v15613: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 109 KiB/s wr, 11 op/s 2023-09-10T07:22:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28975.646858 secs 2023-09-10T07:22:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28976.123934 secs 2023-09-10T07:22:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28975.646858 secs 2023-09-10T07:22:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28976.123934 secs 2023-09-10T07:22:25.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28975.646858 secs 2023-09-10T07:22:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28976.123934 secs 2023-09-10T07:22:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:25 smithi098 ceph-mon[103124]: pgmap v15614: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 10 op/s 2023-09-10T07:22:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:25 smithi170 ceph-mon[102832]: pgmap v15614: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 10 op/s 2023-09-10T07:22:26.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:25 smithi082 ceph-mon[98694]: pgmap v15614: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 10 op/s 2023-09-10T07:22:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:27 smithi098 ceph-mon[103124]: pgmap v15615: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 101 KiB/s wr, 14 op/s 2023-09-10T07:22:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:27 smithi170 ceph-mon[102832]: pgmap v15615: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 101 KiB/s wr, 14 op/s 2023-09-10T07:22:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:27 smithi082 ceph-mon[98694]: pgmap v15615: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 101 KiB/s wr, 14 op/s 2023-09-10T07:22:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28979.509981 secs 2023-09-10T07:22:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28979.509981 secs 2023-09-10T07:22:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28979.509981 secs 2023-09-10T07:22:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28980.646917 secs 2023-09-10T07:22:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28981.124002 secs 2023-09-10T07:22:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:29 smithi170 ceph-mon[102832]: pgmap v15616: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T07:22:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28980.646917 secs 2023-09-10T07:22:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28981.124002 secs 2023-09-10T07:22:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:29 smithi082 ceph-mon[98694]: pgmap v15616: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T07:22:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28980.646917 secs 2023-09-10T07:22:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28981.124002 secs 2023-09-10T07:22:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:29 smithi098 ceph-mon[103124]: pgmap v15616: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T07:22:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:31 smithi098 ceph-mon[103124]: pgmap v15617: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 275 KiB/s wr, 14 op/s 2023-09-10T07:22:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:31 smithi170 ceph-mon[102832]: pgmap v15617: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 275 KiB/s wr, 14 op/s 2023-09-10T07:22:32.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:31 smithi082 ceph-mon[98694]: pgmap v15617: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 275 KiB/s wr, 14 op/s 2023-09-10T07:22:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:33 smithi098 ceph-mon[103124]: pgmap v15618: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 175 KiB/s wr, 9 op/s 2023-09-10T07:22:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:33 smithi170 ceph-mon[102832]: pgmap v15618: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 175 KiB/s wr, 9 op/s 2023-09-10T07:22:34.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:33 smithi082 ceph-mon[98694]: pgmap v15618: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 175 KiB/s wr, 9 op/s 2023-09-10T07:22:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28985.646992 secs 2023-09-10T07:22:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28986.124061 secs 2023-09-10T07:22:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28985.646992 secs 2023-09-10T07:22:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28986.124061 secs 2023-09-10T07:22:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28985.646992 secs 2023-09-10T07:22:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28986.124061 secs 2023-09-10T07:22:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:35 smithi098 ceph-mon[103124]: pgmap v15619: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 175 KiB/s wr, 9 op/s 2023-09-10T07:22:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:22:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:35 smithi170 ceph-mon[102832]: pgmap v15619: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 175 KiB/s wr, 9 op/s 2023-09-10T07:22:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:22:36.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:35 smithi082 ceph-mon[98694]: pgmap v15619: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 175 KiB/s wr, 9 op/s 2023-09-10T07:22:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:22:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:22:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:22:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:22:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:22:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:22:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:22:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:22:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:22:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:22:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:37 smithi098 ceph-mon[103124]: pgmap v15620: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 318 KiB/s wr, 15 op/s 2023-09-10T07:22:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:37 smithi170 ceph-mon[102832]: pgmap v15620: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 318 KiB/s wr, 15 op/s 2023-09-10T07:22:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:37 smithi082 ceph-mon[98694]: pgmap v15620: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 318 KiB/s wr, 15 op/s 2023-09-10T07:22:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28990.647052 secs 2023-09-10T07:22:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28991.124124 secs 2023-09-10T07:22:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:39 smithi098 ceph-mon[103124]: pgmap v15621: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 317 KiB/s wr, 10 op/s 2023-09-10T07:22:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28990.647052 secs 2023-09-10T07:22:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28991.124124 secs 2023-09-10T07:22:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:39 smithi170 ceph-mon[102832]: pgmap v15621: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 317 KiB/s wr, 10 op/s 2023-09-10T07:22:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28990.647052 secs 2023-09-10T07:22:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28991.124124 secs 2023-09-10T07:22:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:39 smithi082 ceph-mon[98694]: pgmap v15621: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 317 KiB/s wr, 10 op/s 2023-09-10T07:22:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:41 smithi098 ceph-mon[103124]: pgmap v15622: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 531 KiB/s wr, 16 op/s 2023-09-10T07:22:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:41 smithi170 ceph-mon[102832]: pgmap v15622: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 531 KiB/s wr, 16 op/s 2023-09-10T07:22:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:41 smithi082 ceph-mon[98694]: pgmap v15622: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 531 KiB/s wr, 16 op/s 2023-09-10T07:22:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:43 smithi098 ceph-mon[103124]: pgmap v15623: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 357 KiB/s wr, 11 op/s 2023-09-10T07:22:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:43 smithi170 ceph-mon[102832]: pgmap v15623: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 357 KiB/s wr, 11 op/s 2023-09-10T07:22:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:43 smithi082 ceph-mon[98694]: pgmap v15623: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 357 KiB/s wr, 11 op/s 2023-09-10T07:22:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28995.647113 secs 2023-09-10T07:22:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 28996.124186 secs 2023-09-10T07:22:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28995.647113 secs 2023-09-10T07:22:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 28996.124186 secs 2023-09-10T07:22:45.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28995.647113 secs 2023-09-10T07:22:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 28996.124186 secs 2023-09-10T07:22:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:45 smithi098 ceph-mon[103124]: pgmap v15624: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 357 KiB/s wr, 11 op/s 2023-09-10T07:22:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:45 smithi170 ceph-mon[102832]: pgmap v15624: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 357 KiB/s wr, 11 op/s 2023-09-10T07:22:46.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:45 smithi082 ceph-mon[98694]: pgmap v15624: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 357 KiB/s wr, 11 op/s 2023-09-10T07:22:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:47 smithi170 ceph-mon[102832]: pgmap v15625: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 537 KiB/s wr, 18 op/s 2023-09-10T07:22:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:47 smithi082 ceph-mon[98694]: pgmap v15625: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 537 KiB/s wr, 18 op/s 2023-09-10T07:22:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:47 smithi098 ceph-mon[103124]: pgmap v15625: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 537 KiB/s wr, 18 op/s 2023-09-10T07:22:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 28999.510403 secs 2023-09-10T07:22:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 28999.510403 secs 2023-09-10T07:22:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 28999.510403 secs 2023-09-10T07:22:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29000.647173 secs 2023-09-10T07:22:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29001.124251 secs 2023-09-10T07:22:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:49 smithi170 ceph-mon[102832]: pgmap v15626: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 394 KiB/s wr, 12 op/s 2023-09-10T07:22:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29000.647173 secs 2023-09-10T07:22:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29001.124251 secs 2023-09-10T07:22:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:49 smithi082 ceph-mon[98694]: pgmap v15626: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 394 KiB/s wr, 12 op/s 2023-09-10T07:22:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29000.647173 secs 2023-09-10T07:22:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29001.124251 secs 2023-09-10T07:22:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:49 smithi098 ceph-mon[103124]: pgmap v15626: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 394 KiB/s wr, 12 op/s 2023-09-10T07:22:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:51 smithi098 ceph-mon[103124]: pgmap v15627: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 607 KiB/s wr, 18 op/s 2023-09-10T07:22:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:51 smithi170 ceph-mon[102832]: pgmap v15627: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 607 KiB/s wr, 18 op/s 2023-09-10T07:22:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:51 smithi082 ceph-mon[98694]: pgmap v15627: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 607 KiB/s wr, 18 op/s 2023-09-10T07:22:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:22:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:22:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:22:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:22:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:22:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:22:53.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:22:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:22:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:22:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29004.510466 secs 2023-09-10T07:22:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:53 smithi098 ceph-mon[103124]: pgmap v15628: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 394 KiB/s wr, 12 op/s 2023-09-10T07:22:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29004.510466 secs 2023-09-10T07:22:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:53 smithi170 ceph-mon[102832]: pgmap v15628: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 394 KiB/s wr, 12 op/s 2023-09-10T07:22:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29004.510466 secs 2023-09-10T07:22:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:53 smithi082 ceph-mon[98694]: pgmap v15628: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 394 KiB/s wr, 12 op/s 2023-09-10T07:22:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29005.647230 secs 2023-09-10T07:22:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29006.124315 secs 2023-09-10T07:22:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29005.647230 secs 2023-09-10T07:22:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29006.124315 secs 2023-09-10T07:22:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29005.647230 secs 2023-09-10T07:22:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29006.124315 secs 2023-09-10T07:22:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:55 smithi098 ceph-mon[103124]: pgmap v15629: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 394 KiB/s wr, 12 op/s 2023-09-10T07:22:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:55 smithi170 ceph-mon[102832]: pgmap v15629: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 394 KiB/s wr, 12 op/s 2023-09-10T07:22:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:55 smithi082 ceph-mon[98694]: pgmap v15629: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 394 KiB/s wr, 12 op/s 2023-09-10T07:22:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:57 smithi170 ceph-mon[102832]: pgmap v15630: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 571 KiB/s wr, 18 op/s 2023-09-10T07:22:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:57 smithi082 ceph-mon[98694]: pgmap v15630: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 571 KiB/s wr, 18 op/s 2023-09-10T07:22:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:57 smithi098 ceph-mon[103124]: pgmap v15630: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 571 KiB/s wr, 18 op/s 2023-09-10T07:22:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29009.510526 secs 2023-09-10T07:22:59.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29009.510526 secs 2023-09-10T07:22:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29009.510526 secs 2023-09-10T07:23:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29010.647288 secs 2023-09-10T07:23:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29011.124379 secs 2023-09-10T07:23:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:22:59 smithi170 ceph-mon[102832]: pgmap v15631: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 391 KiB/s wr, 12 op/s 2023-09-10T07:23:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29010.647288 secs 2023-09-10T07:23:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29011.124379 secs 2023-09-10T07:23:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:22:59 smithi082 ceph-mon[98694]: pgmap v15631: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 391 KiB/s wr, 12 op/s 2023-09-10T07:23:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29010.647288 secs 2023-09-10T07:23:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29011.124379 secs 2023-09-10T07:23:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:22:59 smithi098 ceph-mon[103124]: pgmap v15631: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 391 KiB/s wr, 12 op/s 2023-09-10T07:23:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:01 smithi098 ceph-mon[103124]: pgmap v15632: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 487 KiB/s wr, 18 op/s 2023-09-10T07:23:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:01 smithi170 ceph-mon[102832]: pgmap v15632: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 487 KiB/s wr, 18 op/s 2023-09-10T07:23:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:01 smithi082 ceph-mon[98694]: pgmap v15632: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 487 KiB/s wr, 18 op/s 2023-09-10T07:23:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:23:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:23:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:23:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:23:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:23:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:23:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:23:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:23:03.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:23:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:23:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:23:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:23:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29014.510589 secs 2023-09-10T07:23:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:03 smithi098 ceph-mon[103124]: pgmap v15633: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 274 KiB/s wr, 11 op/s 2023-09-10T07:23:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29014.510589 secs 2023-09-10T07:23:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:03 smithi170 ceph-mon[102832]: pgmap v15633: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 274 KiB/s wr, 11 op/s 2023-09-10T07:23:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29014.510589 secs 2023-09-10T07:23:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:03 smithi082 ceph-mon[98694]: pgmap v15633: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 274 KiB/s wr, 11 op/s 2023-09-10T07:23:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29015.647358 secs 2023-09-10T07:23:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29016.124441 secs 2023-09-10T07:23:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29015.647358 secs 2023-09-10T07:23:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29016.124441 secs 2023-09-10T07:23:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29015.647358 secs 2023-09-10T07:23:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29016.124441 secs 2023-09-10T07:23:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:05 smithi098 ceph-mon[103124]: pgmap v15634: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 274 KiB/s wr, 11 op/s 2023-09-10T07:23:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:05 smithi170 ceph-mon[102832]: pgmap v15634: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 274 KiB/s wr, 11 op/s 2023-09-10T07:23:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:05 smithi082 ceph-mon[98694]: pgmap v15634: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 274 KiB/s wr, 11 op/s 2023-09-10T07:23:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:07 smithi170 ceph-mon[102832]: pgmap v15635: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 460 KiB/s wr, 32 op/s 2023-09-10T07:23:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:07 smithi082 ceph-mon[98694]: pgmap v15635: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 460 KiB/s wr, 32 op/s 2023-09-10T07:23:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:07 smithi098 ceph-mon[103124]: pgmap v15635: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 460 KiB/s wr, 32 op/s 2023-09-10T07:23:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29019.510649 secs 2023-09-10T07:23:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29019.510649 secs 2023-09-10T07:23:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29019.510649 secs 2023-09-10T07:23:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29020.647416 secs 2023-09-10T07:23:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:09 smithi170 ceph-mon[102832]: osdmap e1045: 12 total, 12 up, 12 in 2023-09-10T07:23:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29021.124509 secs 2023-09-10T07:23:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:09 smithi170 ceph-mon[102832]: pgmap v15637: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 339 KiB/s wr, 31 op/s 2023-09-10T07:23:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29020.647416 secs 2023-09-10T07:23:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:09 smithi082 ceph-mon[98694]: osdmap e1045: 12 total, 12 up, 12 in 2023-09-10T07:23:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29021.124509 secs 2023-09-10T07:23:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:09 smithi082 ceph-mon[98694]: pgmap v15637: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 339 KiB/s wr, 31 op/s 2023-09-10T07:23:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29020.647416 secs 2023-09-10T07:23:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:09 smithi098 ceph-mon[103124]: osdmap e1045: 12 total, 12 up, 12 in 2023-09-10T07:23:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29021.124509 secs 2023-09-10T07:23:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:09 smithi098 ceph-mon[103124]: pgmap v15637: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 339 KiB/s wr, 31 op/s 2023-09-10T07:23:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:11 smithi098 ceph-mon[103124]: pgmap v15638: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 346 KiB/s wr, 31 op/s 2023-09-10T07:23:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:11 smithi098 ceph-mon[103124]: osdmap e1046: 12 total, 12 up, 12 in 2023-09-10T07:23:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:11 smithi170 ceph-mon[102832]: pgmap v15638: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 346 KiB/s wr, 31 op/s 2023-09-10T07:23:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:11 smithi170 ceph-mon[102832]: osdmap e1046: 12 total, 12 up, 12 in 2023-09-10T07:23:12.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:11 smithi082 ceph-mon[98694]: pgmap v15638: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 346 KiB/s wr, 31 op/s 2023-09-10T07:23:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:11 smithi082 ceph-mon[98694]: osdmap e1046: 12 total, 12 up, 12 in 2023-09-10T07:23:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29024.510744 secs 2023-09-10T07:23:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:13 smithi098 ceph-mon[103124]: pgmap v15640: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 433 KiB/s wr, 39 op/s 2023-09-10T07:23:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29024.510744 secs 2023-09-10T07:23:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:13 smithi170 ceph-mon[102832]: pgmap v15640: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 433 KiB/s wr, 39 op/s 2023-09-10T07:23:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29024.510744 secs 2023-09-10T07:23:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:13 smithi082 ceph-mon[98694]: pgmap v15640: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 433 KiB/s wr, 39 op/s 2023-09-10T07:23:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29025.647476 secs 2023-09-10T07:23:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29026.124568 secs 2023-09-10T07:23:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29025.647476 secs 2023-09-10T07:23:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29026.124568 secs 2023-09-10T07:23:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29025.647476 secs 2023-09-10T07:23:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29026.124568 secs 2023-09-10T07:23:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:15 smithi098 ceph-mon[103124]: pgmap v15641: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 153 KiB/s wr, 8 op/s 2023-09-10T07:23:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:15 smithi170 ceph-mon[102832]: pgmap v15641: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 153 KiB/s wr, 8 op/s 2023-09-10T07:23:16.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:15 smithi082 ceph-mon[98694]: pgmap v15641: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 153 KiB/s wr, 8 op/s 2023-09-10T07:23:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:17 smithi170 ceph-mon[102832]: pgmap v15642: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T07:23:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:17 smithi082 ceph-mon[98694]: pgmap v15642: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T07:23:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:17 smithi098 ceph-mon[103124]: pgmap v15642: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T07:23:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29029.510804 secs 2023-09-10T07:23:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29029.510804 secs 2023-09-10T07:23:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29029.510804 secs 2023-09-10T07:23:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29030.647537 secs 2023-09-10T07:23:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29031.124631 secs 2023-09-10T07:23:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:19 smithi170 ceph-mon[102832]: pgmap v15643: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T07:23:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29030.647537 secs 2023-09-10T07:23:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29031.124631 secs 2023-09-10T07:23:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:19 smithi082 ceph-mon[98694]: pgmap v15643: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T07:23:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29030.647537 secs 2023-09-10T07:23:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29031.124631 secs 2023-09-10T07:23:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:19 smithi098 ceph-mon[103124]: pgmap v15643: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T07:23:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:21 smithi098 ceph-mon[103124]: pgmap v15644: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T07:23:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:21 smithi170 ceph-mon[102832]: pgmap v15644: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T07:23:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:21 smithi082 ceph-mon[98694]: pgmap v15644: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T07:23:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29034.510864 secs 2023-09-10T07:23:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:23 smithi098 ceph-mon[103124]: pgmap v15645: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T07:23:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29034.510864 secs 2023-09-10T07:23:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:23 smithi170 ceph-mon[102832]: pgmap v15645: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T07:23:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29034.510864 secs 2023-09-10T07:23:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:23 smithi082 ceph-mon[98694]: pgmap v15645: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T07:23:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29035.647619 secs 2023-09-10T07:23:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29036.124690 secs 2023-09-10T07:23:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29035.647619 secs 2023-09-10T07:23:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29036.124690 secs 2023-09-10T07:23:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29035.647619 secs 2023-09-10T07:23:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29036.124690 secs 2023-09-10T07:23:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:25 smithi098 ceph-mon[103124]: pgmap v15646: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T07:23:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:25 smithi170 ceph-mon[102832]: pgmap v15646: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T07:23:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:25 smithi082 ceph-mon[98694]: pgmap v15646: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T07:23:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:27 smithi170 ceph-mon[102832]: pgmap v15647: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 101 KiB/s wr, 14 op/s 2023-09-10T07:23:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:27 smithi082 ceph-mon[98694]: pgmap v15647: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 101 KiB/s wr, 14 op/s 2023-09-10T07:23:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:27 smithi098 ceph-mon[103124]: pgmap v15647: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 101 KiB/s wr, 14 op/s 2023-09-10T07:23:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29039.510924 secs 2023-09-10T07:23:29.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29039.510924 secs 2023-09-10T07:23:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29039.510924 secs 2023-09-10T07:23:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29040.647676 secs 2023-09-10T07:23:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29041.124756 secs 2023-09-10T07:23:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:29 smithi170 ceph-mon[102832]: pgmap v15648: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T07:23:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29040.647676 secs 2023-09-10T07:23:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29041.124756 secs 2023-09-10T07:23:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:29 smithi082 ceph-mon[98694]: pgmap v15648: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T07:23:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29040.647676 secs 2023-09-10T07:23:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29041.124756 secs 2023-09-10T07:23:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:29 smithi098 ceph-mon[103124]: pgmap v15648: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T07:23:32.258 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:31 smithi098 ceph-mon[103124]: pgmap v15649: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 254 KiB/s wr, 14 op/s 2023-09-10T07:23:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:31 smithi170 ceph-mon[102832]: pgmap v15649: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 254 KiB/s wr, 14 op/s 2023-09-10T07:23:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:31 smithi082 ceph-mon[98694]: pgmap v15649: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 254 KiB/s wr, 14 op/s 2023-09-10T07:23:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:33 smithi098 ceph-mon[103124]: pgmap v15650: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 153 KiB/s wr, 9 op/s 2023-09-10T07:23:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:33 smithi170 ceph-mon[102832]: pgmap v15650: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 153 KiB/s wr, 9 op/s 2023-09-10T07:23:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:33 smithi082 ceph-mon[98694]: pgmap v15650: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 153 KiB/s wr, 9 op/s 2023-09-10T07:23:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29045.647734 secs 2023-09-10T07:23:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29046.124821 secs 2023-09-10T07:23:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29045.647734 secs 2023-09-10T07:23:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29046.124821 secs 2023-09-10T07:23:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29045.647734 secs 2023-09-10T07:23:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29046.124821 secs 2023-09-10T07:23:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:35 smithi098 ceph-mon[103124]: pgmap v15651: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 153 KiB/s wr, 9 op/s 2023-09-10T07:23:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:23:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:35 smithi170 ceph-mon[102832]: pgmap v15651: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 153 KiB/s wr, 9 op/s 2023-09-10T07:23:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:23:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:35 smithi082 ceph-mon[98694]: pgmap v15651: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 153 KiB/s wr, 9 op/s 2023-09-10T07:23:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:23:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:23:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:23:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:23:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:23:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:23:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:23:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:23:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:23:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:23:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:37 smithi098 ceph-mon[103124]: pgmap v15652: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 319 KiB/s wr, 15 op/s 2023-09-10T07:23:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:37 smithi170 ceph-mon[102832]: pgmap v15652: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 319 KiB/s wr, 15 op/s 2023-09-10T07:23:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:37 smithi082 ceph-mon[98694]: pgmap v15652: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 319 KiB/s wr, 15 op/s 2023-09-10T07:23:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29050.647794 secs 2023-09-10T07:23:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29051.124885 secs 2023-09-10T07:23:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:39 smithi098 ceph-mon[103124]: pgmap v15653: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 319 KiB/s wr, 11 op/s 2023-09-10T07:23:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29050.647794 secs 2023-09-10T07:23:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29051.124885 secs 2023-09-10T07:23:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:39 smithi170 ceph-mon[102832]: pgmap v15653: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 319 KiB/s wr, 11 op/s 2023-09-10T07:23:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29050.647794 secs 2023-09-10T07:23:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29051.124885 secs 2023-09-10T07:23:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:39 smithi082 ceph-mon[98694]: pgmap v15653: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 319 KiB/s wr, 11 op/s 2023-09-10T07:23:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:42 smithi098 ceph-mon[103124]: pgmap v15654: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 539 KiB/s wr, 17 op/s 2023-09-10T07:23:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:42 smithi170 ceph-mon[102832]: pgmap v15654: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 539 KiB/s wr, 17 op/s 2023-09-10T07:23:42.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:42 smithi082 ceph-mon[98694]: pgmap v15654: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 539 KiB/s wr, 17 op/s 2023-09-10T07:23:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:44 smithi170 ceph-mon[102832]: pgmap v15655: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 386 KiB/s wr, 12 op/s 2023-09-10T07:23:44.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:44 smithi082 ceph-mon[98694]: pgmap v15655: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 386 KiB/s wr, 12 op/s 2023-09-10T07:23:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:44 smithi098 ceph-mon[103124]: pgmap v15655: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 386 KiB/s wr, 12 op/s 2023-09-10T07:23:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29055.647853 secs 2023-09-10T07:23:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29056.124945 secs 2023-09-10T07:23:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29055.647853 secs 2023-09-10T07:23:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29056.124945 secs 2023-09-10T07:23:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29055.647853 secs 2023-09-10T07:23:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29056.124945 secs 2023-09-10T07:23:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:46 smithi170 ceph-mon[102832]: pgmap v15656: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 386 KiB/s wr, 12 op/s 2023-09-10T07:23:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:46 smithi082 ceph-mon[98694]: pgmap v15656: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 386 KiB/s wr, 12 op/s 2023-09-10T07:23:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:46 smithi098 ceph-mon[103124]: pgmap v15656: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 386 KiB/s wr, 12 op/s 2023-09-10T07:23:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:47 smithi170 ceph-mon[102832]: pgmap v15657: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 562 KiB/s wr, 19 op/s 2023-09-10T07:23:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:47 smithi082 ceph-mon[98694]: pgmap v15657: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 562 KiB/s wr, 19 op/s 2023-09-10T07:23:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:47 smithi098 ceph-mon[103124]: pgmap v15657: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 562 KiB/s wr, 19 op/s 2023-09-10T07:23:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29060.647953 secs 2023-09-10T07:23:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29061.125011 secs 2023-09-10T07:23:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:49 smithi098 ceph-mon[103124]: pgmap v15658: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 396 KiB/s wr, 13 op/s 2023-09-10T07:23:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29060.647953 secs 2023-09-10T07:23:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29061.125011 secs 2023-09-10T07:23:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:49 smithi170 ceph-mon[102832]: pgmap v15658: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 396 KiB/s wr, 13 op/s 2023-09-10T07:23:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29060.647953 secs 2023-09-10T07:23:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29061.125011 secs 2023-09-10T07:23:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:49 smithi082 ceph-mon[98694]: pgmap v15658: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 396 KiB/s wr, 13 op/s 2023-09-10T07:23:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:51 smithi098 ceph-mon[103124]: pgmap v15659: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 604 KiB/s wr, 19 op/s 2023-09-10T07:23:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:51 smithi170 ceph-mon[102832]: pgmap v15659: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 604 KiB/s wr, 19 op/s 2023-09-10T07:23:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:51 smithi082 ceph-mon[98694]: pgmap v15659: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 604 KiB/s wr, 19 op/s 2023-09-10T07:23:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29064.511311 secs 2023-09-10T07:23:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:53 smithi098 ceph-mon[103124]: pgmap v15660: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 384 KiB/s wr, 13 op/s 2023-09-10T07:23:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29064.511311 secs 2023-09-10T07:23:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:53 smithi170 ceph-mon[102832]: pgmap v15660: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 384 KiB/s wr, 13 op/s 2023-09-10T07:23:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29064.511311 secs 2023-09-10T07:23:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:53 smithi082 ceph-mon[98694]: pgmap v15660: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 384 KiB/s wr, 13 op/s 2023-09-10T07:23:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29065.648012 secs 2023-09-10T07:23:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29066.125069 secs 2023-09-10T07:23:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29065.648012 secs 2023-09-10T07:23:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29066.125069 secs 2023-09-10T07:23:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29065.648012 secs 2023-09-10T07:23:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29066.125069 secs 2023-09-10T07:23:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:56 smithi170 ceph-mon[102832]: pgmap v15661: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 384 KiB/s wr, 13 op/s 2023-09-10T07:23:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:23:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:56 smithi082 ceph-mon[98694]: pgmap v15661: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 384 KiB/s wr, 13 op/s 2023-09-10T07:23:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:23:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:56 smithi098 ceph-mon[103124]: pgmap v15661: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 384 KiB/s wr, 13 op/s 2023-09-10T07:23:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:23:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:23:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:23:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:23:57.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:23:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:23:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:23:58.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:58 smithi170 ceph-mon[102832]: pgmap v15662: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 584 KiB/s wr, 20 op/s 2023-09-10T07:23:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:58 smithi082 ceph-mon[98694]: pgmap v15662: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 584 KiB/s wr, 20 op/s 2023-09-10T07:23:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:58 smithi098 ceph-mon[103124]: pgmap v15662: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 584 KiB/s wr, 20 op/s 2023-09-10T07:23:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:23:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29069.511371 secs 2023-09-10T07:23:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:23:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29069.511371 secs 2023-09-10T07:23:59.511 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:23:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29069.511371 secs 2023-09-10T07:24:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29070.648069 secs 2023-09-10T07:24:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29071.125132 secs 2023-09-10T07:24:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:00 smithi170 ceph-mon[102832]: pgmap v15663: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 408 KiB/s wr, 13 op/s 2023-09-10T07:24:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29070.648069 secs 2023-09-10T07:24:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29071.125132 secs 2023-09-10T07:24:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:00 smithi082 ceph-mon[98694]: pgmap v15663: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 408 KiB/s wr, 13 op/s 2023-09-10T07:24:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29070.648069 secs 2023-09-10T07:24:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29071.125132 secs 2023-09-10T07:24:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:00 smithi098 ceph-mon[103124]: pgmap v15663: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 408 KiB/s wr, 13 op/s 2023-09-10T07:24:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:02 smithi170 ceph-mon[102832]: pgmap v15664: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 521 KiB/s wr, 18 op/s 2023-09-10T07:24:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:02 smithi082 ceph-mon[98694]: pgmap v15664: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 521 KiB/s wr, 18 op/s 2023-09-10T07:24:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:02 smithi098 ceph-mon[103124]: pgmap v15664: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 521 KiB/s wr, 18 op/s 2023-09-10T07:24:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:24:03.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:24:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:24:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29074.511435 secs 2023-09-10T07:24:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:04 smithi170 ceph-mon[102832]: pgmap v15665: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 312 KiB/s wr, 12 op/s 2023-09-10T07:24:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:24:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:24:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:24:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29074.511435 secs 2023-09-10T07:24:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:04 smithi082 ceph-mon[98694]: pgmap v15665: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 312 KiB/s wr, 12 op/s 2023-09-10T07:24:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:24:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:24:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:24:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29074.511435 secs 2023-09-10T07:24:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:04 smithi098 ceph-mon[103124]: pgmap v15665: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 312 KiB/s wr, 12 op/s 2023-09-10T07:24:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:24:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:24:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:24:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29075.648156 secs 2023-09-10T07:24:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29076.125191 secs 2023-09-10T07:24:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29075.648156 secs 2023-09-10T07:24:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29076.125191 secs 2023-09-10T07:24:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29075.648156 secs 2023-09-10T07:24:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29076.125191 secs 2023-09-10T07:24:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:06 smithi170 ceph-mon[102832]: pgmap v15666: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 312 KiB/s wr, 12 op/s 2023-09-10T07:24:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:06 smithi082 ceph-mon[98694]: pgmap v15666: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 312 KiB/s wr, 12 op/s 2023-09-10T07:24:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:06 smithi098 ceph-mon[103124]: pgmap v15666: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 312 KiB/s wr, 12 op/s 2023-09-10T07:24:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:07 smithi170 ceph-mon[102832]: pgmap v15667: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 500 KiB/s wr, 33 op/s 2023-09-10T07:24:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:07 smithi082 ceph-mon[98694]: pgmap v15667: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 500 KiB/s wr, 33 op/s 2023-09-10T07:24:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:07 smithi098 ceph-mon[103124]: pgmap v15667: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 500 KiB/s wr, 33 op/s 2023-09-10T07:24:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29079.511495 secs 2023-09-10T07:24:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29079.511495 secs 2023-09-10T07:24:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29079.511495 secs 2023-09-10T07:24:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29080.648229 secs 2023-09-10T07:24:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:09 smithi170 ceph-mon[102832]: osdmap e1047: 12 total, 12 up, 12 in 2023-09-10T07:24:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29081.125255 secs 2023-09-10T07:24:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:09 smithi170 ceph-mon[102832]: pgmap v15669: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 359 KiB/s wr, 31 op/s 2023-09-10T07:24:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29080.648229 secs 2023-09-10T07:24:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:09 smithi082 ceph-mon[98694]: osdmap e1047: 12 total, 12 up, 12 in 2023-09-10T07:24:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29081.125255 secs 2023-09-10T07:24:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:09 smithi082 ceph-mon[98694]: pgmap v15669: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 359 KiB/s wr, 31 op/s 2023-09-10T07:24:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29080.648229 secs 2023-09-10T07:24:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:09 smithi098 ceph-mon[103124]: osdmap e1047: 12 total, 12 up, 12 in 2023-09-10T07:24:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29081.125255 secs 2023-09-10T07:24:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:09 smithi098 ceph-mon[103124]: pgmap v15669: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 359 KiB/s wr, 31 op/s 2023-09-10T07:24:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:11 smithi098 ceph-mon[103124]: pgmap v15670: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 321 KiB/s wr, 30 op/s 2023-09-10T07:24:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:11 smithi098 ceph-mon[103124]: osdmap e1048: 12 total, 12 up, 12 in 2023-09-10T07:24:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:11 smithi170 ceph-mon[102832]: pgmap v15670: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 321 KiB/s wr, 30 op/s 2023-09-10T07:24:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:11 smithi170 ceph-mon[102832]: osdmap e1048: 12 total, 12 up, 12 in 2023-09-10T07:24:12.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:11 smithi082 ceph-mon[98694]: pgmap v15670: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 321 KiB/s wr, 30 op/s 2023-09-10T07:24:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:11 smithi082 ceph-mon[98694]: osdmap e1048: 12 total, 12 up, 12 in 2023-09-10T07:24:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29084.511556 secs 2023-09-10T07:24:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:13 smithi098 ceph-mon[103124]: pgmap v15672: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 401 KiB/s wr, 38 op/s 2023-09-10T07:24:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29084.511556 secs 2023-09-10T07:24:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:13 smithi170 ceph-mon[102832]: pgmap v15672: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 401 KiB/s wr, 38 op/s 2023-09-10T07:24:14.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29084.511556 secs 2023-09-10T07:24:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:13 smithi082 ceph-mon[98694]: pgmap v15672: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 401 KiB/s wr, 38 op/s 2023-09-10T07:24:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29085.648287 secs 2023-09-10T07:24:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29086.125314 secs 2023-09-10T07:24:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29085.648287 secs 2023-09-10T07:24:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29086.125314 secs 2023-09-10T07:24:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29085.648287 secs 2023-09-10T07:24:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29086.125314 secs 2023-09-10T07:24:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:16 smithi170 ceph-mon[102832]: pgmap v15673: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 121 KiB/s wr, 7 op/s 2023-09-10T07:24:16.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:16 smithi082 ceph-mon[98694]: pgmap v15673: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 121 KiB/s wr, 7 op/s 2023-09-10T07:24:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:16 smithi098 ceph-mon[103124]: pgmap v15673: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 121 KiB/s wr, 7 op/s 2023-09-10T07:24:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:17 smithi170 ceph-mon[102832]: pgmap v15674: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 15 op/s 2023-09-10T07:24:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:17 smithi082 ceph-mon[98694]: pgmap v15674: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 15 op/s 2023-09-10T07:24:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:17 smithi098 ceph-mon[103124]: pgmap v15674: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 15 op/s 2023-09-10T07:24:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29089.511617 secs 2023-09-10T07:24:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29089.511617 secs 2023-09-10T07:24:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29089.511617 secs 2023-09-10T07:24:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29090.648344 secs 2023-09-10T07:24:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29091.125378 secs 2023-09-10T07:24:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:19 smithi170 ceph-mon[102832]: pgmap v15675: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T07:24:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29090.648344 secs 2023-09-10T07:24:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29091.125378 secs 2023-09-10T07:24:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:19 smithi082 ceph-mon[98694]: pgmap v15675: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T07:24:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29090.648344 secs 2023-09-10T07:24:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29091.125378 secs 2023-09-10T07:24:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:19 smithi098 ceph-mon[103124]: pgmap v15675: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 12 op/s 2023-09-10T07:24:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:21 smithi098 ceph-mon[103124]: pgmap v15676: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 12 op/s 2023-09-10T07:24:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:21 smithi170 ceph-mon[102832]: pgmap v15676: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 12 op/s 2023-09-10T07:24:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:21 smithi082 ceph-mon[98694]: pgmap v15676: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 12 op/s 2023-09-10T07:24:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29094.511677 secs 2023-09-10T07:24:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:24 smithi098 ceph-mon[103124]: pgmap v15677: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T07:24:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29094.511677 secs 2023-09-10T07:24:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:24 smithi170 ceph-mon[102832]: pgmap v15677: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T07:24:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29094.511677 secs 2023-09-10T07:24:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:24 smithi082 ceph-mon[98694]: pgmap v15677: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 11 op/s 2023-09-10T07:24:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29095.648425 secs 2023-09-10T07:24:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29096.125437 secs 2023-09-10T07:24:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29095.648425 secs 2023-09-10T07:24:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29096.125437 secs 2023-09-10T07:24:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29095.648425 secs 2023-09-10T07:24:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29096.125437 secs 2023-09-10T07:24:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:26 smithi170 ceph-mon[102832]: pgmap v15678: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 165 KiB/s wr, 10 op/s 2023-09-10T07:24:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:26 smithi082 ceph-mon[98694]: pgmap v15678: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 165 KiB/s wr, 10 op/s 2023-09-10T07:24:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:26 smithi098 ceph-mon[103124]: pgmap v15678: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 165 KiB/s wr, 10 op/s 2023-09-10T07:24:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:27 smithi170 ceph-mon[102832]: pgmap v15679: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 165 KiB/s wr, 14 op/s 2023-09-10T07:24:28.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:27 smithi082 ceph-mon[98694]: pgmap v15679: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 165 KiB/s wr, 14 op/s 2023-09-10T07:24:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:27 smithi098 ceph-mon[103124]: pgmap v15679: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 165 KiB/s wr, 14 op/s 2023-09-10T07:24:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29099.511778 secs 2023-09-10T07:24:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29099.511778 secs 2023-09-10T07:24:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29099.511778 secs 2023-09-10T07:24:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29100.648489 secs 2023-09-10T07:24:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29101.125490 secs 2023-09-10T07:24:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:29 smithi170 ceph-mon[102832]: pgmap v15680: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 9 op/s 2023-09-10T07:24:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29100.648489 secs 2023-09-10T07:24:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29101.125490 secs 2023-09-10T07:24:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:29 smithi082 ceph-mon[98694]: pgmap v15680: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 9 op/s 2023-09-10T07:24:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29100.648489 secs 2023-09-10T07:24:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29101.125490 secs 2023-09-10T07:24:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:29 smithi098 ceph-mon[103124]: pgmap v15680: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 9 op/s 2023-09-10T07:24:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:32 smithi170 ceph-mon[102832]: pgmap v15681: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 263 KiB/s wr, 15 op/s 2023-09-10T07:24:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:32 smithi082 ceph-mon[98694]: pgmap v15681: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 263 KiB/s wr, 15 op/s 2023-09-10T07:24:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:32 smithi098 ceph-mon[103124]: pgmap v15681: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 263 KiB/s wr, 15 op/s 2023-09-10T07:24:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:34 smithi170 ceph-mon[102832]: pgmap v15682: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:24:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:34 smithi082 ceph-mon[98694]: pgmap v15682: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:24:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:34 smithi098 ceph-mon[103124]: pgmap v15682: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:24:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29105.648547 secs 2023-09-10T07:24:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29106.125547 secs 2023-09-10T07:24:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:24:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29105.648547 secs 2023-09-10T07:24:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29106.125547 secs 2023-09-10T07:24:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:24:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29105.648547 secs 2023-09-10T07:24:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29106.125547 secs 2023-09-10T07:24:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:24:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:36 smithi170 ceph-mon[102832]: pgmap v15683: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:24:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:24:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:24:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:24:36.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:36 smithi082 ceph-mon[98694]: pgmap v15683: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:24:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:24:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:24:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:24:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:36 smithi098 ceph-mon[103124]: pgmap v15683: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 98 KiB/s wr, 9 op/s 2023-09-10T07:24:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:24:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:24:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:24:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:37 smithi170 ceph-mon[102832]: pgmap v15684: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 395 KiB/s wr, 15 op/s 2023-09-10T07:24:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:37 smithi082 ceph-mon[98694]: pgmap v15684: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 395 KiB/s wr, 15 op/s 2023-09-10T07:24:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:37 smithi098 ceph-mon[103124]: pgmap v15684: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 395 KiB/s wr, 15 op/s 2023-09-10T07:24:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29110.648605 secs 2023-09-10T07:24:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29111.125611 secs 2023-09-10T07:24:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:40 smithi170 ceph-mon[102832]: pgmap v15685: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 395 KiB/s wr, 11 op/s 2023-09-10T07:24:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29110.648605 secs 2023-09-10T07:24:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29111.125611 secs 2023-09-10T07:24:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:40 smithi082 ceph-mon[98694]: pgmap v15685: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 395 KiB/s wr, 11 op/s 2023-09-10T07:24:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29110.648605 secs 2023-09-10T07:24:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29111.125611 secs 2023-09-10T07:24:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:40 smithi098 ceph-mon[103124]: pgmap v15685: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 395 KiB/s wr, 11 op/s 2023-09-10T07:24:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:42 smithi170 ceph-mon[102832]: pgmap v15686: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 534 KiB/s wr, 17 op/s 2023-09-10T07:24:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:42 smithi082 ceph-mon[98694]: pgmap v15686: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 534 KiB/s wr, 17 op/s 2023-09-10T07:24:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:42 smithi098 ceph-mon[103124]: pgmap v15686: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 534 KiB/s wr, 17 op/s 2023-09-10T07:24:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:44 smithi170 ceph-mon[102832]: pgmap v15687: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 436 KiB/s wr, 12 op/s 2023-09-10T07:24:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:44 smithi082 ceph-mon[98694]: pgmap v15687: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 436 KiB/s wr, 12 op/s 2023-09-10T07:24:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:44 smithi098 ceph-mon[103124]: pgmap v15687: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 436 KiB/s wr, 12 op/s 2023-09-10T07:24:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29115.648679 secs 2023-09-10T07:24:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29116.125673 secs 2023-09-10T07:24:45.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29115.648679 secs 2023-09-10T07:24:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29116.125673 secs 2023-09-10T07:24:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29115.648679 secs 2023-09-10T07:24:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29116.125673 secs 2023-09-10T07:24:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:46 smithi170 ceph-mon[102832]: pgmap v15688: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 436 KiB/s wr, 12 op/s 2023-09-10T07:24:46.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:46 smithi082 ceph-mon[98694]: pgmap v15688: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 436 KiB/s wr, 12 op/s 2023-09-10T07:24:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:46 smithi098 ceph-mon[103124]: pgmap v15688: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 436 KiB/s wr, 12 op/s 2023-09-10T07:24:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:47 smithi170 ceph-mon[102832]: pgmap v15689: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 683 KiB/s wr, 18 op/s 2023-09-10T07:24:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:47 smithi082 ceph-mon[98694]: pgmap v15689: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 683 KiB/s wr, 18 op/s 2023-09-10T07:24:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:47 smithi098 ceph-mon[103124]: pgmap v15689: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 683 KiB/s wr, 18 op/s 2023-09-10T07:24:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29120.648736 secs 2023-09-10T07:24:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29121.125738 secs 2023-09-10T07:24:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:50 smithi170 ceph-mon[102832]: pgmap v15690: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 386 KiB/s wr, 12 op/s 2023-09-10T07:24:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29120.648736 secs 2023-09-10T07:24:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29121.125738 secs 2023-09-10T07:24:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:50 smithi082 ceph-mon[98694]: pgmap v15690: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 386 KiB/s wr, 12 op/s 2023-09-10T07:24:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29120.648736 secs 2023-09-10T07:24:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29121.125738 secs 2023-09-10T07:24:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:50 smithi098 ceph-mon[103124]: pgmap v15690: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 386 KiB/s wr, 12 op/s 2023-09-10T07:24:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:52 smithi170 ceph-mon[102832]: pgmap v15691: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 530 KiB/s wr, 19 op/s 2023-09-10T07:24:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:52 smithi082 ceph-mon[98694]: pgmap v15691: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 530 KiB/s wr, 19 op/s 2023-09-10T07:24:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:52 smithi098 ceph-mon[103124]: pgmap v15691: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 530 KiB/s wr, 19 op/s 2023-09-10T07:24:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:54 smithi170 ceph-mon[102832]: pgmap v15692: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 391 KiB/s wr, 12 op/s 2023-09-10T07:24:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:54 smithi082 ceph-mon[98694]: pgmap v15692: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 391 KiB/s wr, 12 op/s 2023-09-10T07:24:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:54 smithi098 ceph-mon[103124]: pgmap v15692: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 391 KiB/s wr, 12 op/s 2023-09-10T07:24:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29125.648796 secs 2023-09-10T07:24:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29126.125795 secs 2023-09-10T07:24:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29125.648796 secs 2023-09-10T07:24:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29126.125795 secs 2023-09-10T07:24:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29125.648796 secs 2023-09-10T07:24:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29126.125795 secs 2023-09-10T07:24:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:56 smithi170 ceph-mon[102832]: pgmap v15693: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 391 KiB/s wr, 12 op/s 2023-09-10T07:24:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:56 smithi082 ceph-mon[98694]: pgmap v15693: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 391 KiB/s wr, 12 op/s 2023-09-10T07:24:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:56 smithi098 ceph-mon[103124]: pgmap v15693: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 391 KiB/s wr, 12 op/s 2023-09-10T07:24:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:57 smithi170 ceph-mon[102832]: pgmap v15694: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 575 KiB/s wr, 19 op/s 2023-09-10T07:24:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:57 smithi082 ceph-mon[98694]: pgmap v15694: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 575 KiB/s wr, 19 op/s 2023-09-10T07:24:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:57 smithi098 ceph-mon[103124]: pgmap v15694: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 575 KiB/s wr, 19 op/s 2023-09-10T07:24:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29129.512206 secs 2023-09-10T07:24:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29129.512206 secs 2023-09-10T07:24:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29129.512206 secs 2023-09-10T07:25:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29130.648866 secs 2023-09-10T07:25:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29131.125859 secs 2023-09-10T07:25:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:24:59 smithi170 ceph-mon[102832]: pgmap v15695: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 328 KiB/s wr, 12 op/s 2023-09-10T07:25:00.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29130.648866 secs 2023-09-10T07:25:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29131.125859 secs 2023-09-10T07:25:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:24:59 smithi082 ceph-mon[98694]: pgmap v15695: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 328 KiB/s wr, 12 op/s 2023-09-10T07:25:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29130.648866 secs 2023-09-10T07:25:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29131.125859 secs 2023-09-10T07:25:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:24:59 smithi098 ceph-mon[103124]: pgmap v15695: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 328 KiB/s wr, 12 op/s 2023-09-10T07:25:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:25:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:25:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:25:01.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:25:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:25:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:25:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:25:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:25:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:25:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:01 smithi170 ceph-mon[102832]: pgmap v15696: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 448 KiB/s wr, 31 op/s 2023-09-10T07:25:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:01 smithi082 ceph-mon[98694]: pgmap v15696: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 448 KiB/s wr, 31 op/s 2023-09-10T07:25:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:01 smithi098 ceph-mon[103124]: pgmap v15696: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 448 KiB/s wr, 31 op/s 2023-09-10T07:25:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29134.512277 secs 2023-09-10T07:25:04.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:03 smithi170 ceph-mon[102832]: osdmap e1049: 12 total, 12 up, 12 in 2023-09-10T07:25:04.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:03 smithi170 ceph-mon[102832]: pgmap v15698: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 364 KiB/s wr, 30 op/s 2023-09-10T07:25:04.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:25:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29134.512277 secs 2023-09-10T07:25:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:03 smithi082 ceph-mon[98694]: osdmap e1049: 12 total, 12 up, 12 in 2023-09-10T07:25:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:03 smithi082 ceph-mon[98694]: pgmap v15698: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 364 KiB/s wr, 30 op/s 2023-09-10T07:25:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:25:04.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29134.512277 secs 2023-09-10T07:25:04.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:03 smithi098 ceph-mon[103124]: osdmap e1049: 12 total, 12 up, 12 in 2023-09-10T07:25:04.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:03 smithi098 ceph-mon[103124]: pgmap v15698: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 364 KiB/s wr, 30 op/s 2023-09-10T07:25:04.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:25:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29135.648954 secs 2023-09-10T07:25:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29136.125928 secs 2023-09-10T07:25:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29135.648954 secs 2023-09-10T07:25:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29136.125928 secs 2023-09-10T07:25:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29135.648954 secs 2023-09-10T07:25:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29136.125928 secs 2023-09-10T07:25:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:05 smithi170 ceph-mon[102832]: pgmap v15699: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 364 KiB/s wr, 30 op/s 2023-09-10T07:25:06.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:05 smithi082 ceph-mon[98694]: pgmap v15699: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 364 KiB/s wr, 30 op/s 2023-09-10T07:25:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:05 smithi098 ceph-mon[103124]: pgmap v15699: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 364 KiB/s wr, 30 op/s 2023-09-10T07:25:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:07.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:07.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:07.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:25:07.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:07.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:07.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:07.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:07.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:07.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:07.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:07.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:25:07.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:07.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:07.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:07.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:07.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:07.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:25:07.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:08 smithi098 ceph-mon[103124]: pgmap v15700: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 341 KiB/s wr, 31 op/s 2023-09-10T07:25:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:25:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:25:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:25:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:25:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:08 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T07:25:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:08 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T07:25:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:25:08.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:25:08.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:25:08.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:25:08.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:25:08.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:25:08.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:08 smithi170 ceph-mon[102832]: pgmap v15700: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 341 KiB/s wr, 31 op/s 2023-09-10T07:25:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:25:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:25:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:25:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:25:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:08 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T07:25:08.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:08 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T07:25:08.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:08.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:08.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:25:08.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:25:08.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:25:08.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:25:08.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:25:08.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:25:08.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:08 smithi082 ceph-mon[98694]: pgmap v15700: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 341 KiB/s wr, 31 op/s 2023-09-10T07:25:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:25:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:25:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:25:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:25:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:08 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T07:25:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:08 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T07:25:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:08.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:25:08.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:25:08.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:25:08.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:25:08.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:25:08.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:25:08.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:25:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:09 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T07:25:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:09 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T07:25:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29139.512336 secs 2023-09-10T07:25:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:09 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T07:25:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:09 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T07:25:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29139.512336 secs 2023-09-10T07:25:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:09 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T07:25:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:09 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T07:25:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29139.512336 secs 2023-09-10T07:25:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29140.649012 secs 2023-09-10T07:25:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29141.125991 secs 2023-09-10T07:25:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:10 smithi098 ceph-mon[103124]: pgmap v15701: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 341 KiB/s wr, 31 op/s 2023-09-10T07:25:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29140.649012 secs 2023-09-10T07:25:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29141.125991 secs 2023-09-10T07:25:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:10 smithi170 ceph-mon[102832]: pgmap v15701: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 341 KiB/s wr, 31 op/s 2023-09-10T07:25:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29140.649012 secs 2023-09-10T07:25:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29141.125991 secs 2023-09-10T07:25:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:10 smithi082 ceph-mon[98694]: pgmap v15701: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 341 KiB/s wr, 31 op/s 2023-09-10T07:25:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:12 smithi098 ceph-mon[103124]: pgmap v15702: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 321 KiB/s wr, 14 op/s 2023-09-10T07:25:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:12 smithi098 ceph-mon[103124]: osdmap e1050: 12 total, 12 up, 12 in 2023-09-10T07:25:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:12 smithi170 ceph-mon[102832]: pgmap v15702: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 321 KiB/s wr, 14 op/s 2023-09-10T07:25:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:12 smithi170 ceph-mon[102832]: osdmap e1050: 12 total, 12 up, 12 in 2023-09-10T07:25:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:12 smithi082 ceph-mon[98694]: pgmap v15702: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 321 KiB/s wr, 14 op/s 2023-09-10T07:25:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:12 smithi082 ceph-mon[98694]: osdmap e1050: 12 total, 12 up, 12 in 2023-09-10T07:25:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29144.512407 secs 2023-09-10T07:25:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:14 smithi170 ceph-mon[102832]: pgmap v15704: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 321 KiB/s wr, 14 op/s 2023-09-10T07:25:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29144.512407 secs 2023-09-10T07:25:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:14 smithi082 ceph-mon[98694]: pgmap v15704: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 321 KiB/s wr, 14 op/s 2023-09-10T07:25:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29144.512407 secs 2023-09-10T07:25:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:14 smithi098 ceph-mon[103124]: pgmap v15704: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 321 KiB/s wr, 14 op/s 2023-09-10T07:25:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29145.649071 secs 2023-09-10T07:25:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29146.126053 secs 2023-09-10T07:25:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29145.649071 secs 2023-09-10T07:25:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29146.126053 secs 2023-09-10T07:25:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29145.649071 secs 2023-09-10T07:25:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29146.126053 secs 2023-09-10T07:25:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:16 smithi170 ceph-mon[102832]: pgmap v15705: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 321 KiB/s wr, 14 op/s 2023-09-10T07:25:16.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:16 smithi082 ceph-mon[98694]: pgmap v15705: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 321 KiB/s wr, 14 op/s 2023-09-10T07:25:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:16 smithi098 ceph-mon[103124]: pgmap v15705: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 321 KiB/s wr, 14 op/s 2023-09-10T07:25:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:17 smithi170 ceph-mon[102832]: pgmap v15706: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T07:25:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:17 smithi082 ceph-mon[98694]: pgmap v15706: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T07:25:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:17 smithi098 ceph-mon[103124]: pgmap v15706: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T07:25:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29149.512466 secs 2023-09-10T07:25:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29149.512466 secs 2023-09-10T07:25:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29149.512466 secs 2023-09-10T07:25:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29150.649125 secs 2023-09-10T07:25:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29151.126117 secs 2023-09-10T07:25:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:19 smithi170 ceph-mon[102832]: pgmap v15707: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T07:25:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29150.649125 secs 2023-09-10T07:25:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29151.126117 secs 2023-09-10T07:25:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:19 smithi082 ceph-mon[98694]: pgmap v15707: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T07:25:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29150.649125 secs 2023-09-10T07:25:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29151.126117 secs 2023-09-10T07:25:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:19 smithi098 ceph-mon[103124]: pgmap v15707: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T07:25:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:22 smithi170 ceph-mon[102832]: pgmap v15708: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T07:25:22.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:22 smithi082 ceph-mon[98694]: pgmap v15708: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T07:25:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:22 smithi098 ceph-mon[103124]: pgmap v15708: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T07:25:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29154.512527 secs 2023-09-10T07:25:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:24 smithi170 ceph-mon[102832]: pgmap v15709: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T07:25:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29154.512527 secs 2023-09-10T07:25:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:24 smithi082 ceph-mon[98694]: pgmap v15709: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T07:25:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29154.512527 secs 2023-09-10T07:25:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:24 smithi098 ceph-mon[103124]: pgmap v15709: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T07:25:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29155.649184 secs 2023-09-10T07:25:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29156.126178 secs 2023-09-10T07:25:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29155.649184 secs 2023-09-10T07:25:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29156.126178 secs 2023-09-10T07:25:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29155.649184 secs 2023-09-10T07:25:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29156.126178 secs 2023-09-10T07:25:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:26 smithi170 ceph-mon[102832]: pgmap v15710: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 102 KiB/s wr, 9 op/s 2023-09-10T07:25:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:26 smithi082 ceph-mon[98694]: pgmap v15710: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 102 KiB/s wr, 9 op/s 2023-09-10T07:25:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:26 smithi098 ceph-mon[103124]: pgmap v15710: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 102 KiB/s wr, 9 op/s 2023-09-10T07:25:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:27 smithi170 ceph-mon[102832]: pgmap v15711: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 102 KiB/s wr, 14 op/s 2023-09-10T07:25:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:27 smithi082 ceph-mon[98694]: pgmap v15711: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 102 KiB/s wr, 14 op/s 2023-09-10T07:25:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:27 smithi098 ceph-mon[103124]: pgmap v15711: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 102 KiB/s wr, 14 op/s 2023-09-10T07:25:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29159.512617 secs 2023-09-10T07:25:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29159.512617 secs 2023-09-10T07:25:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29159.512617 secs 2023-09-10T07:25:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29160.649250 secs 2023-09-10T07:25:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29161.126244 secs 2023-09-10T07:25:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:29 smithi170 ceph-mon[102832]: pgmap v15712: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T07:25:30.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29160.649250 secs 2023-09-10T07:25:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29161.126244 secs 2023-09-10T07:25:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:29 smithi082 ceph-mon[98694]: pgmap v15712: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T07:25:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29160.649250 secs 2023-09-10T07:25:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29161.126244 secs 2023-09-10T07:25:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:29 smithi098 ceph-mon[103124]: pgmap v15712: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 9 op/s 2023-09-10T07:25:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:32 smithi170 ceph-mon[102832]: pgmap v15713: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 200 KiB/s wr, 14 op/s 2023-09-10T07:25:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:32 smithi082 ceph-mon[98694]: pgmap v15713: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 200 KiB/s wr, 14 op/s 2023-09-10T07:25:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:32 smithi098 ceph-mon[103124]: pgmap v15713: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 200 KiB/s wr, 14 op/s 2023-09-10T07:25:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:34 smithi170 ceph-mon[102832]: pgmap v15714: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T07:25:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:34 smithi082 ceph-mon[98694]: pgmap v15714: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T07:25:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:34 smithi098 ceph-mon[103124]: pgmap v15714: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T07:25:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29165.649308 secs 2023-09-10T07:25:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29166.126302 secs 2023-09-10T07:25:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:25:35.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29165.649308 secs 2023-09-10T07:25:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29166.126302 secs 2023-09-10T07:25:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:25:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29165.649308 secs 2023-09-10T07:25:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29166.126302 secs 2023-09-10T07:25:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:25:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:36 smithi170 ceph-mon[102832]: pgmap v15715: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T07:25:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:25:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:25:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:25:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:36 smithi082 ceph-mon[98694]: pgmap v15715: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T07:25:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:25:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:25:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:25:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:36 smithi098 ceph-mon[103124]: pgmap v15715: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T07:25:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:25:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:25:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:25:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:37 smithi170 ceph-mon[102832]: pgmap v15716: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 365 KiB/s wr, 16 op/s 2023-09-10T07:25:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:37 smithi082 ceph-mon[98694]: pgmap v15716: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 365 KiB/s wr, 16 op/s 2023-09-10T07:25:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:37 smithi098 ceph-mon[103124]: pgmap v15716: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 365 KiB/s wr, 16 op/s 2023-09-10T07:25:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29170.649364 secs 2023-09-10T07:25:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29171.126368 secs 2023-09-10T07:25:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:40 smithi170 ceph-mon[102832]: pgmap v15717: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 365 KiB/s wr, 11 op/s 2023-09-10T07:25:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29170.649364 secs 2023-09-10T07:25:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29171.126368 secs 2023-09-10T07:25:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:40 smithi082 ceph-mon[98694]: pgmap v15717: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 365 KiB/s wr, 11 op/s 2023-09-10T07:25:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29170.649364 secs 2023-09-10T07:25:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29171.126368 secs 2023-09-10T07:25:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:40 smithi098 ceph-mon[103124]: pgmap v15717: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 365 KiB/s wr, 11 op/s 2023-09-10T07:25:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:42 smithi170 ceph-mon[102832]: pgmap v15718: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 537 KiB/s wr, 18 op/s 2023-09-10T07:25:42.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:42 smithi082 ceph-mon[98694]: pgmap v15718: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 537 KiB/s wr, 18 op/s 2023-09-10T07:25:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:42 smithi098 ceph-mon[103124]: pgmap v15718: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 537 KiB/s wr, 18 op/s 2023-09-10T07:25:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:44 smithi170 ceph-mon[102832]: pgmap v15719: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 438 KiB/s wr, 12 op/s 2023-09-10T07:25:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:44 smithi082 ceph-mon[98694]: pgmap v15719: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 438 KiB/s wr, 12 op/s 2023-09-10T07:25:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:44 smithi098 ceph-mon[103124]: pgmap v15719: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 438 KiB/s wr, 12 op/s 2023-09-10T07:25:45.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29175.649422 secs 2023-09-10T07:25:45.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29176.126428 secs 2023-09-10T07:25:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29175.649422 secs 2023-09-10T07:25:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29176.126428 secs 2023-09-10T07:25:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29175.649422 secs 2023-09-10T07:25:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29176.126428 secs 2023-09-10T07:25:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:46 smithi170 ceph-mon[102832]: pgmap v15720: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 438 KiB/s wr, 12 op/s 2023-09-10T07:25:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:46 smithi082 ceph-mon[98694]: pgmap v15720: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 438 KiB/s wr, 12 op/s 2023-09-10T07:25:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:46 smithi098 ceph-mon[103124]: pgmap v15720: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 438 KiB/s wr, 12 op/s 2023-09-10T07:25:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:47 smithi170 ceph-mon[102832]: pgmap v15721: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 723 KiB/s wr, 20 op/s 2023-09-10T07:25:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:47 smithi082 ceph-mon[98694]: pgmap v15721: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 723 KiB/s wr, 20 op/s 2023-09-10T07:25:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:47 smithi098 ceph-mon[103124]: pgmap v15721: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 723 KiB/s wr, 20 op/s 2023-09-10T07:25:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29180.649481 secs 2023-09-10T07:25:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29181.126483 secs 2023-09-10T07:25:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:50 smithi170 ceph-mon[102832]: pgmap v15722: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 457 KiB/s wr, 14 op/s 2023-09-10T07:25:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29180.649481 secs 2023-09-10T07:25:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29181.126483 secs 2023-09-10T07:25:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:50 smithi082 ceph-mon[98694]: pgmap v15722: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 457 KiB/s wr, 14 op/s 2023-09-10T07:25:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29180.649481 secs 2023-09-10T07:25:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29181.126483 secs 2023-09-10T07:25:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:50 smithi098 ceph-mon[103124]: pgmap v15722: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 457 KiB/s wr, 14 op/s 2023-09-10T07:25:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:52 smithi170 ceph-mon[102832]: pgmap v15723: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 594 KiB/s wr, 21 op/s 2023-09-10T07:25:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:52 smithi082 ceph-mon[98694]: pgmap v15723: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 594 KiB/s wr, 21 op/s 2023-09-10T07:25:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:52 smithi098 ceph-mon[103124]: pgmap v15723: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 594 KiB/s wr, 21 op/s 2023-09-10T07:25:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29184.513087 secs 2023-09-10T07:25:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:54 smithi170 ceph-mon[102832]: pgmap v15724: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 422 KiB/s wr, 14 op/s 2023-09-10T07:25:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29184.513087 secs 2023-09-10T07:25:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:54 smithi082 ceph-mon[98694]: pgmap v15724: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 422 KiB/s wr, 14 op/s 2023-09-10T07:25:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29184.513087 secs 2023-09-10T07:25:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:54 smithi098 ceph-mon[103124]: pgmap v15724: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 422 KiB/s wr, 14 op/s 2023-09-10T07:25:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29185.649566 secs 2023-09-10T07:25:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29186.126543 secs 2023-09-10T07:25:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29185.649566 secs 2023-09-10T07:25:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29186.126543 secs 2023-09-10T07:25:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29185.649566 secs 2023-09-10T07:25:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29186.126543 secs 2023-09-10T07:25:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:56 smithi170 ceph-mon[102832]: pgmap v15725: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 422 KiB/s wr, 14 op/s 2023-09-10T07:25:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:25:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:25:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:25:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:56 smithi082 ceph-mon[98694]: pgmap v15725: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 422 KiB/s wr, 14 op/s 2023-09-10T07:25:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:25:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:25:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:25:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:56 smithi098 ceph-mon[103124]: pgmap v15725: 97 pgs: 97 active+clean; 9.8 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 422 KiB/s wr, 14 op/s 2023-09-10T07:25:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:25:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:25:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:25:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:57 smithi170 ceph-mon[102832]: pgmap v15726: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 562 KiB/s wr, 19 op/s 2023-09-10T07:25:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:57 smithi082 ceph-mon[98694]: pgmap v15726: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 562 KiB/s wr, 19 op/s 2023-09-10T07:25:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:57 smithi098 ceph-mon[103124]: pgmap v15726: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 562 KiB/s wr, 19 op/s 2023-09-10T07:25:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29189.513144 secs 2023-09-10T07:25:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29189.513144 secs 2023-09-10T07:25:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29189.513144 secs 2023-09-10T07:26:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29190.649624 secs 2023-09-10T07:26:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29191.126611 secs 2023-09-10T07:26:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:25:59 smithi170 ceph-mon[102832]: pgmap v15727: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T07:26:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29190.649624 secs 2023-09-10T07:26:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29191.126611 secs 2023-09-10T07:26:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:25:59 smithi082 ceph-mon[98694]: pgmap v15727: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T07:26:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29190.649624 secs 2023-09-10T07:26:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29191.126611 secs 2023-09-10T07:26:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:25:59 smithi098 ceph-mon[103124]: pgmap v15727: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 12 op/s 2023-09-10T07:26:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:02 smithi170 ceph-mon[102832]: pgmap v15728: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 391 KiB/s wr, 19 op/s 2023-09-10T07:26:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:02 smithi082 ceph-mon[98694]: pgmap v15728: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 391 KiB/s wr, 19 op/s 2023-09-10T07:26:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:02 smithi098 ceph-mon[103124]: pgmap v15728: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 391 KiB/s wr, 19 op/s 2023-09-10T07:26:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29194.513231 secs 2023-09-10T07:26:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:04 smithi170 ceph-mon[102832]: pgmap v15729: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 254 KiB/s wr, 13 op/s 2023-09-10T07:26:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29194.513231 secs 2023-09-10T07:26:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:04 smithi082 ceph-mon[98694]: pgmap v15729: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 254 KiB/s wr, 13 op/s 2023-09-10T07:26:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29194.513231 secs 2023-09-10T07:26:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:04 smithi098 ceph-mon[103124]: pgmap v15729: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 254 KiB/s wr, 13 op/s 2023-09-10T07:26:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29195.649681 secs 2023-09-10T07:26:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29196.126670 secs 2023-09-10T07:26:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29195.649681 secs 2023-09-10T07:26:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29196.126670 secs 2023-09-10T07:26:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29195.649681 secs 2023-09-10T07:26:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29196.126670 secs 2023-09-10T07:26:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:06 smithi170 ceph-mon[102832]: pgmap v15730: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 254 KiB/s wr, 13 op/s 2023-09-10T07:26:06.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:06 smithi082 ceph-mon[98694]: pgmap v15730: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 254 KiB/s wr, 13 op/s 2023-09-10T07:26:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:06 smithi098 ceph-mon[103124]: pgmap v15730: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 254 KiB/s wr, 13 op/s 2023-09-10T07:26:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:07 smithi170 ceph-mon[102832]: pgmap v15731: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 299 KiB/s wr, 30 op/s 2023-09-10T07:26:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:26:08.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:07 smithi082 ceph-mon[98694]: pgmap v15731: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 299 KiB/s wr, 30 op/s 2023-09-10T07:26:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:26:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:07 smithi098 ceph-mon[103124]: pgmap v15731: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 299 KiB/s wr, 30 op/s 2023-09-10T07:26:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:26:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29199.513294 secs 2023-09-10T07:26:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:26:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:26:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:26:09.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29199.513294 secs 2023-09-10T07:26:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:26:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:26:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:26:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29199.513294 secs 2023-09-10T07:26:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:26:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:26:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:26:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29200.649740 secs 2023-09-10T07:26:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:09 smithi170 ceph-mon[102832]: osdmap e1051: 12 total, 12 up, 12 in 2023-09-10T07:26:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29201.126736 secs 2023-09-10T07:26:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:09 smithi170 ceph-mon[102832]: pgmap v15733: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 189 KiB/s wr, 29 op/s 2023-09-10T07:26:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29200.649740 secs 2023-09-10T07:26:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:09 smithi082 ceph-mon[98694]: osdmap e1051: 12 total, 12 up, 12 in 2023-09-10T07:26:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29201.126736 secs 2023-09-10T07:26:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:09 smithi082 ceph-mon[98694]: pgmap v15733: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 189 KiB/s wr, 29 op/s 2023-09-10T07:26:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29200.649740 secs 2023-09-10T07:26:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:09 smithi098 ceph-mon[103124]: osdmap e1051: 12 total, 12 up, 12 in 2023-09-10T07:26:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29201.126736 secs 2023-09-10T07:26:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:09 smithi098 ceph-mon[103124]: pgmap v15733: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 189 KiB/s wr, 29 op/s 2023-09-10T07:26:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:12 smithi170 ceph-mon[102832]: pgmap v15734: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2023-09-10T07:26:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:12 smithi170 ceph-mon[102832]: osdmap e1052: 12 total, 12 up, 12 in 2023-09-10T07:26:12.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:12 smithi082 ceph-mon[98694]: pgmap v15734: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2023-09-10T07:26:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:12 smithi082 ceph-mon[98694]: osdmap e1052: 12 total, 12 up, 12 in 2023-09-10T07:26:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:12 smithi098 ceph-mon[103124]: pgmap v15734: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2023-09-10T07:26:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:12 smithi098 ceph-mon[103124]: osdmap e1052: 12 total, 12 up, 12 in 2023-09-10T07:26:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29204.513357 secs 2023-09-10T07:26:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:14 smithi170 ceph-mon[102832]: pgmap v15736: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 67 KiB/s wr, 32 op/s 2023-09-10T07:26:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29204.513357 secs 2023-09-10T07:26:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:14 smithi082 ceph-mon[98694]: pgmap v15736: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 67 KiB/s wr, 32 op/s 2023-09-10T07:26:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29204.513357 secs 2023-09-10T07:26:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:14 smithi098 ceph-mon[103124]: pgmap v15736: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 67 KiB/s wr, 32 op/s 2023-09-10T07:26:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29205.649797 secs 2023-09-10T07:26:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29206.126795 secs 2023-09-10T07:26:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29205.649797 secs 2023-09-10T07:26:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29206.126795 secs 2023-09-10T07:26:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29205.649797 secs 2023-09-10T07:26:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29206.126795 secs 2023-09-10T07:26:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:16 smithi170 ceph-mon[102832]: pgmap v15737: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:26:16.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:16 smithi082 ceph-mon[98694]: pgmap v15737: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:26:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:16 smithi098 ceph-mon[103124]: pgmap v15737: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:26:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:17 smithi170 ceph-mon[102832]: pgmap v15738: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 250 B/s wr, 13 op/s 2023-09-10T07:26:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:17 smithi082 ceph-mon[98694]: pgmap v15738: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 250 B/s wr, 13 op/s 2023-09-10T07:26:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:17 smithi098 ceph-mon[103124]: pgmap v15738: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 250 B/s wr, 13 op/s 2023-09-10T07:26:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29209.513418 secs 2023-09-10T07:26:19.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29209.513418 secs 2023-09-10T07:26:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29209.513418 secs 2023-09-10T07:26:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29210.649856 secs 2023-09-10T07:26:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29211.126860 secs 2023-09-10T07:26:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:19 smithi170 ceph-mon[102832]: pgmap v15739: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T07:26:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29210.649856 secs 2023-09-10T07:26:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29211.126860 secs 2023-09-10T07:26:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:19 smithi082 ceph-mon[98694]: pgmap v15739: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T07:26:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29210.649856 secs 2023-09-10T07:26:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29211.126860 secs 2023-09-10T07:26:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:19 smithi098 ceph-mon[103124]: pgmap v15739: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T07:26:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:22 smithi170 ceph-mon[102832]: pgmap v15740: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T07:26:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:22 smithi082 ceph-mon[98694]: pgmap v15740: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T07:26:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:22 smithi098 ceph-mon[103124]: pgmap v15740: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T07:26:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29214.513480 secs 2023-09-10T07:26:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:24 smithi170 ceph-mon[102832]: pgmap v15741: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T07:26:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29214.513480 secs 2023-09-10T07:26:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:24 smithi082 ceph-mon[98694]: pgmap v15741: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T07:26:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29214.513480 secs 2023-09-10T07:26:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:24 smithi098 ceph-mon[103124]: pgmap v15741: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T07:26:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29215.649915 secs 2023-09-10T07:26:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29216.126918 secs 2023-09-10T07:26:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29215.649915 secs 2023-09-10T07:26:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29216.126918 secs 2023-09-10T07:26:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29215.649915 secs 2023-09-10T07:26:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29216.126918 secs 2023-09-10T07:26:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:26 smithi170 ceph-mon[102832]: pgmap v15742: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-10T07:26:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:26 smithi082 ceph-mon[98694]: pgmap v15742: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-10T07:26:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:26 smithi098 ceph-mon[103124]: pgmap v15742: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-10T07:26:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:27 smithi170 ceph-mon[102832]: pgmap v15743: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 96 KiB/s wr, 14 op/s 2023-09-10T07:26:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:27 smithi082 ceph-mon[98694]: pgmap v15743: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 96 KiB/s wr, 14 op/s 2023-09-10T07:26:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:27 smithi098 ceph-mon[103124]: pgmap v15743: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 96 KiB/s wr, 14 op/s 2023-09-10T07:26:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29219.513578 secs 2023-09-10T07:26:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29219.513578 secs 2023-09-10T07:26:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29219.513578 secs 2023-09-10T07:26:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29220.649972 secs 2023-09-10T07:26:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29221.126985 secs 2023-09-10T07:26:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:29 smithi170 ceph-mon[102832]: pgmap v15744: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 9 op/s 2023-09-10T07:26:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29220.649972 secs 2023-09-10T07:26:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29221.126985 secs 2023-09-10T07:26:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:29 smithi082 ceph-mon[98694]: pgmap v15744: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 9 op/s 2023-09-10T07:26:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29220.649972 secs 2023-09-10T07:26:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29221.126985 secs 2023-09-10T07:26:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:29 smithi098 ceph-mon[103124]: pgmap v15744: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 9 op/s 2023-09-10T07:26:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:32 smithi170 ceph-mon[102832]: pgmap v15745: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 97 KiB/s wr, 14 op/s 2023-09-10T07:26:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:32 smithi082 ceph-mon[98694]: pgmap v15745: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 97 KiB/s wr, 14 op/s 2023-09-10T07:26:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:32 smithi098 ceph-mon[103124]: pgmap v15745: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 97 KiB/s wr, 14 op/s 2023-09-10T07:26:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:34 smithi170 ceph-mon[102832]: pgmap v15746: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 9 op/s 2023-09-10T07:26:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:34 smithi082 ceph-mon[98694]: pgmap v15746: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 9 op/s 2023-09-10T07:26:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:34 smithi098 ceph-mon[103124]: pgmap v15746: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 9 op/s 2023-09-10T07:26:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29225.650030 secs 2023-09-10T07:26:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29226.127044 secs 2023-09-10T07:26:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:26:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29225.650030 secs 2023-09-10T07:26:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29226.127044 secs 2023-09-10T07:26:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:26:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29225.650030 secs 2023-09-10T07:26:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29226.127044 secs 2023-09-10T07:26:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:26:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:36 smithi170 ceph-mon[102832]: pgmap v15747: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 9 op/s 2023-09-10T07:26:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:26:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:26:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:26:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:36 smithi082 ceph-mon[98694]: pgmap v15747: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 9 op/s 2023-09-10T07:26:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:26:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:26:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:26:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:36 smithi098 ceph-mon[103124]: pgmap v15747: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 9 op/s 2023-09-10T07:26:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:26:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:26:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:26:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:37 smithi170 ceph-mon[102832]: pgmap v15748: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 339 KiB/s wr, 16 op/s 2023-09-10T07:26:38.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:37 smithi082 ceph-mon[98694]: pgmap v15748: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 339 KiB/s wr, 16 op/s 2023-09-10T07:26:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:37 smithi098 ceph-mon[103124]: pgmap v15748: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 339 KiB/s wr, 16 op/s 2023-09-10T07:26:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29230.650089 secs 2023-09-10T07:26:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29231.127109 secs 2023-09-10T07:26:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:40 smithi170 ceph-mon[102832]: pgmap v15749: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 242 KiB/s wr, 11 op/s 2023-09-10T07:26:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29230.650089 secs 2023-09-10T07:26:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29231.127109 secs 2023-09-10T07:26:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:40 smithi082 ceph-mon[98694]: pgmap v15749: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 242 KiB/s wr, 11 op/s 2023-09-10T07:26:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29230.650089 secs 2023-09-10T07:26:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29231.127109 secs 2023-09-10T07:26:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:40 smithi098 ceph-mon[103124]: pgmap v15749: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 242 KiB/s wr, 11 op/s 2023-09-10T07:26:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:42 smithi170 ceph-mon[102832]: pgmap v15750: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 508 KiB/s wr, 18 op/s 2023-09-10T07:26:42.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:42 smithi082 ceph-mon[98694]: pgmap v15750: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 508 KiB/s wr, 18 op/s 2023-09-10T07:26:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:42 smithi098 ceph-mon[103124]: pgmap v15750: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 508 KiB/s wr, 18 op/s 2023-09-10T07:26:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:44 smithi170 ceph-mon[102832]: pgmap v15751: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 508 KiB/s wr, 13 op/s 2023-09-10T07:26:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:44 smithi082 ceph-mon[98694]: pgmap v15751: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 508 KiB/s wr, 13 op/s 2023-09-10T07:26:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:44 smithi098 ceph-mon[103124]: pgmap v15751: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 508 KiB/s wr, 13 op/s 2023-09-10T07:26:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29235.650170 secs 2023-09-10T07:26:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29236.127166 secs 2023-09-10T07:26:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29235.650170 secs 2023-09-10T07:26:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29236.127166 secs 2023-09-10T07:26:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29235.650170 secs 2023-09-10T07:26:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29236.127166 secs 2023-09-10T07:26:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:46 smithi170 ceph-mon[102832]: pgmap v15752: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 508 KiB/s wr, 13 op/s 2023-09-10T07:26:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:46 smithi082 ceph-mon[98694]: pgmap v15752: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 508 KiB/s wr, 13 op/s 2023-09-10T07:26:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:46 smithi098 ceph-mon[103124]: pgmap v15752: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 508 KiB/s wr, 13 op/s 2023-09-10T07:26:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:47 smithi170 ceph-mon[102832]: pgmap v15753: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 758 KiB/s wr, 21 op/s 2023-09-10T07:26:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:47 smithi082 ceph-mon[98694]: pgmap v15753: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 758 KiB/s wr, 21 op/s 2023-09-10T07:26:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:47 smithi098 ceph-mon[103124]: pgmap v15753: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 758 KiB/s wr, 21 op/s 2023-09-10T07:26:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29240.650230 secs 2023-09-10T07:26:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29241.127230 secs 2023-09-10T07:26:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:50 smithi170 ceph-mon[102832]: pgmap v15754: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 517 KiB/s wr, 14 op/s 2023-09-10T07:26:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29240.650230 secs 2023-09-10T07:26:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29241.127230 secs 2023-09-10T07:26:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:50 smithi082 ceph-mon[98694]: pgmap v15754: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 517 KiB/s wr, 14 op/s 2023-09-10T07:26:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29240.650230 secs 2023-09-10T07:26:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29241.127230 secs 2023-09-10T07:26:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:50 smithi098 ceph-mon[103124]: pgmap v15754: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 517 KiB/s wr, 14 op/s 2023-09-10T07:26:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:52 smithi170 ceph-mon[102832]: pgmap v15755: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 653 KiB/s wr, 21 op/s 2023-09-10T07:26:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:52 smithi082 ceph-mon[98694]: pgmap v15755: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 653 KiB/s wr, 21 op/s 2023-09-10T07:26:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:52 smithi098 ceph-mon[103124]: pgmap v15755: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 653 KiB/s wr, 21 op/s 2023-09-10T07:26:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29244.513902 secs 2023-09-10T07:26:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:54 smithi170 ceph-mon[102832]: pgmap v15756: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 387 KiB/s wr, 14 op/s 2023-09-10T07:26:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29244.513902 secs 2023-09-10T07:26:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:54 smithi082 ceph-mon[98694]: pgmap v15756: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 387 KiB/s wr, 14 op/s 2023-09-10T07:26:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29244.513902 secs 2023-09-10T07:26:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:54 smithi098 ceph-mon[103124]: pgmap v15756: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 387 KiB/s wr, 14 op/s 2023-09-10T07:26:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29245.650290 secs 2023-09-10T07:26:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29246.127289 secs 2023-09-10T07:26:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29245.650290 secs 2023-09-10T07:26:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29246.127289 secs 2023-09-10T07:26:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29245.650290 secs 2023-09-10T07:26:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29246.127289 secs 2023-09-10T07:26:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:56 smithi170 ceph-mon[102832]: pgmap v15757: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 387 KiB/s wr, 14 op/s 2023-09-10T07:26:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:26:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:26:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:26:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:56 smithi082 ceph-mon[98694]: pgmap v15757: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 387 KiB/s wr, 14 op/s 2023-09-10T07:26:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:26:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:26:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:26:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:56 smithi098 ceph-mon[103124]: pgmap v15757: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 387 KiB/s wr, 14 op/s 2023-09-10T07:26:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:26:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:26:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:26:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:57 smithi170 ceph-mon[102832]: pgmap v15758: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 529 KiB/s wr, 20 op/s 2023-09-10T07:26:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:57 smithi082 ceph-mon[98694]: pgmap v15758: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 529 KiB/s wr, 20 op/s 2023-09-10T07:26:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:57 smithi098 ceph-mon[103124]: pgmap v15758: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 529 KiB/s wr, 20 op/s 2023-09-10T07:26:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29249.513962 secs 2023-09-10T07:26:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29249.513962 secs 2023-09-10T07:26:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29249.513962 secs 2023-09-10T07:27:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29250.650349 secs 2023-09-10T07:27:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29251.127355 secs 2023-09-10T07:27:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:26:59 smithi170 ceph-mon[102832]: pgmap v15759: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T07:27:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29250.650349 secs 2023-09-10T07:27:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29251.127355 secs 2023-09-10T07:27:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:26:59 smithi082 ceph-mon[98694]: pgmap v15759: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T07:27:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29250.650349 secs 2023-09-10T07:27:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29251.127355 secs 2023-09-10T07:27:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:26:59 smithi098 ceph-mon[103124]: pgmap v15759: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 278 KiB/s wr, 12 op/s 2023-09-10T07:27:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:02 smithi170 ceph-mon[102832]: pgmap v15760: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 391 KiB/s wr, 17 op/s 2023-09-10T07:27:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:02 smithi082 ceph-mon[98694]: pgmap v15760: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 391 KiB/s wr, 17 op/s 2023-09-10T07:27:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:02 smithi098 ceph-mon[103124]: pgmap v15760: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 391 KiB/s wr, 17 op/s 2023-09-10T07:27:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29254.514022 secs 2023-09-10T07:27:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:04 smithi170 ceph-mon[102832]: pgmap v15761: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 255 KiB/s wr, 11 op/s 2023-09-10T07:27:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29254.514022 secs 2023-09-10T07:27:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:04 smithi082 ceph-mon[98694]: pgmap v15761: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 255 KiB/s wr, 11 op/s 2023-09-10T07:27:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29254.514022 secs 2023-09-10T07:27:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:04 smithi098 ceph-mon[103124]: pgmap v15761: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 255 KiB/s wr, 11 op/s 2023-09-10T07:27:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29255.650407 secs 2023-09-10T07:27:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29256.127416 secs 2023-09-10T07:27:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29255.650407 secs 2023-09-10T07:27:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29256.127416 secs 2023-09-10T07:27:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29255.650407 secs 2023-09-10T07:27:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29256.127416 secs 2023-09-10T07:27:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:06 smithi170 ceph-mon[102832]: pgmap v15762: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 255 KiB/s wr, 11 op/s 2023-09-10T07:27:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:06 smithi082 ceph-mon[98694]: pgmap v15762: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 255 KiB/s wr, 11 op/s 2023-09-10T07:27:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:06 smithi098 ceph-mon[103124]: pgmap v15762: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 255 KiB/s wr, 11 op/s 2023-09-10T07:27:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:07 smithi170 ceph-mon[102832]: pgmap v15763: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 300 KiB/s wr, 30 op/s 2023-09-10T07:27:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:07 smithi082 ceph-mon[98694]: pgmap v15763: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 300 KiB/s wr, 30 op/s 2023-09-10T07:27:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:07 smithi098 ceph-mon[103124]: pgmap v15763: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 300 KiB/s wr, 30 op/s 2023-09-10T07:27:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29259.514099 secs 2023-09-10T07:27:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:08 smithi170 ceph-mon[102832]: osdmap e1053: 12 total, 12 up, 12 in 2023-09-10T07:27:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:27:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29259.514099 secs 2023-09-10T07:27:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:08 smithi082 ceph-mon[98694]: osdmap e1053: 12 total, 12 up, 12 in 2023-09-10T07:27:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:27:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29259.514099 secs 2023-09-10T07:27:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:08 smithi098 ceph-mon[103124]: osdmap e1053: 12 total, 12 up, 12 in 2023-09-10T07:27:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:27:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29260.650494 secs 2023-09-10T07:27:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29261.127487 secs 2023-09-10T07:27:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:27:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:27:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:09 smithi170 ceph-mon[102832]: pgmap v15765: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 190 KiB/s wr, 29 op/s 2023-09-10T07:27:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:27:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:27:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:27:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29260.650494 secs 2023-09-10T07:27:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29261.127487 secs 2023-09-10T07:27:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:27:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:27:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:09 smithi082 ceph-mon[98694]: pgmap v15765: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 190 KiB/s wr, 29 op/s 2023-09-10T07:27:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:27:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:27:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:27:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29260.650494 secs 2023-09-10T07:27:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29261.127487 secs 2023-09-10T07:27:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:27:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:27:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:09 smithi098 ceph-mon[103124]: pgmap v15765: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 190 KiB/s wr, 29 op/s 2023-09-10T07:27:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:27:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:27:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:27:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:12 smithi170 ceph-mon[102832]: pgmap v15766: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 150 KiB/s wr, 27 op/s 2023-09-10T07:27:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:12 smithi170 ceph-mon[102832]: osdmap e1054: 12 total, 12 up, 12 in 2023-09-10T07:27:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:12 smithi082 ceph-mon[98694]: pgmap v15766: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 150 KiB/s wr, 27 op/s 2023-09-10T07:27:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:12 smithi082 ceph-mon[98694]: osdmap e1054: 12 total, 12 up, 12 in 2023-09-10T07:27:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:12 smithi098 ceph-mon[103124]: pgmap v15766: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 150 KiB/s wr, 27 op/s 2023-09-10T07:27:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:12 smithi098 ceph-mon[103124]: osdmap e1054: 12 total, 12 up, 12 in 2023-09-10T07:27:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29264.514159 secs 2023-09-10T07:27:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:14 smithi170 ceph-mon[102832]: pgmap v15768: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 187 KiB/s wr, 34 op/s 2023-09-10T07:27:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29264.514159 secs 2023-09-10T07:27:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:14 smithi082 ceph-mon[98694]: pgmap v15768: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 187 KiB/s wr, 34 op/s 2023-09-10T07:27:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29264.514159 secs 2023-09-10T07:27:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:14 smithi098 ceph-mon[103124]: pgmap v15768: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 187 KiB/s wr, 34 op/s 2023-09-10T07:27:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29265.650553 secs 2023-09-10T07:27:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29266.127545 secs 2023-09-10T07:27:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29265.650553 secs 2023-09-10T07:27:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29266.127545 secs 2023-09-10T07:27:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29265.650553 secs 2023-09-10T07:27:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29266.127545 secs 2023-09-10T07:27:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:16 smithi170 ceph-mon[102832]: pgmap v15769: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 6 op/s 2023-09-10T07:27:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:16 smithi082 ceph-mon[98694]: pgmap v15769: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 6 op/s 2023-09-10T07:27:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:16 smithi098 ceph-mon[103124]: pgmap v15769: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 6 op/s 2023-09-10T07:27:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:17 smithi170 ceph-mon[102832]: pgmap v15770: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T07:27:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:17 smithi082 ceph-mon[98694]: pgmap v15770: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T07:27:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:17 smithi098 ceph-mon[103124]: pgmap v15770: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 105 KiB/s wr, 11 op/s 2023-09-10T07:27:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29269.514221 secs 2023-09-10T07:27:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29269.514221 secs 2023-09-10T07:27:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29269.514221 secs 2023-09-10T07:27:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29270.650648 secs 2023-09-10T07:27:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29271.127612 secs 2023-09-10T07:27:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:19 smithi170 ceph-mon[102832]: pgmap v15771: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T07:27:20.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29270.650648 secs 2023-09-10T07:27:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29271.127612 secs 2023-09-10T07:27:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:19 smithi082 ceph-mon[98694]: pgmap v15771: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T07:27:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29270.650648 secs 2023-09-10T07:27:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29271.127612 secs 2023-09-10T07:27:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:19 smithi098 ceph-mon[103124]: pgmap v15771: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T07:27:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:22 smithi170 ceph-mon[102832]: pgmap v15772: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:27:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:22 smithi082 ceph-mon[98694]: pgmap v15772: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:27:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:22 smithi098 ceph-mon[103124]: pgmap v15772: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:27:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29274.514281 secs 2023-09-10T07:27:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:24 smithi170 ceph-mon[102832]: pgmap v15773: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:27:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29274.514281 secs 2023-09-10T07:27:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:24 smithi082 ceph-mon[98694]: pgmap v15773: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:27:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29274.514281 secs 2023-09-10T07:27:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:24 smithi098 ceph-mon[103124]: pgmap v15773: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:27:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29275.650707 secs 2023-09-10T07:27:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29276.127675 secs 2023-09-10T07:27:25.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29275.650707 secs 2023-09-10T07:27:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29276.127675 secs 2023-09-10T07:27:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29275.650707 secs 2023-09-10T07:27:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29276.127675 secs 2023-09-10T07:27:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:26 smithi170 ceph-mon[102832]: pgmap v15774: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:27:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:26 smithi082 ceph-mon[98694]: pgmap v15774: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:27:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:26 smithi098 ceph-mon[103124]: pgmap v15774: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:27:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:27 smithi170 ceph-mon[102832]: pgmap v15775: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2023-09-10T07:27:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:27 smithi082 ceph-mon[98694]: pgmap v15775: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2023-09-10T07:27:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:27 smithi098 ceph-mon[103124]: pgmap v15775: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2023-09-10T07:27:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29279.514368 secs 2023-09-10T07:27:29.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29279.514368 secs 2023-09-10T07:27:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29279.514368 secs 2023-09-10T07:27:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29280.651141 secs 2023-09-10T07:27:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29281.127741 secs 2023-09-10T07:27:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:29 smithi170 ceph-mon[102832]: pgmap v15776: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:27:30.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29280.651141 secs 2023-09-10T07:27:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29281.127741 secs 2023-09-10T07:27:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:29 smithi082 ceph-mon[98694]: pgmap v15776: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:27:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29280.651141 secs 2023-09-10T07:27:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29281.127741 secs 2023-09-10T07:27:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:29 smithi098 ceph-mon[103124]: pgmap v15776: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:27:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:32 smithi170 ceph-mon[102832]: pgmap v15777: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 79 KiB/s wr, 13 op/s 2023-09-10T07:27:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:32 smithi082 ceph-mon[98694]: pgmap v15777: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 79 KiB/s wr, 13 op/s 2023-09-10T07:27:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:32 smithi098 ceph-mon[103124]: pgmap v15777: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 79 KiB/s wr, 13 op/s 2023-09-10T07:27:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:34 smithi170 ceph-mon[102832]: pgmap v15778: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 79 KiB/s wr, 9 op/s 2023-09-10T07:27:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:34 smithi082 ceph-mon[98694]: pgmap v15778: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 79 KiB/s wr, 9 op/s 2023-09-10T07:27:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:34 smithi098 ceph-mon[103124]: pgmap v15778: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 79 KiB/s wr, 9 op/s 2023-09-10T07:27:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29285.651199 secs 2023-09-10T07:27:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29286.127805 secs 2023-09-10T07:27:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:27:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29285.651199 secs 2023-09-10T07:27:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29286.127805 secs 2023-09-10T07:27:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:27:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29285.651199 secs 2023-09-10T07:27:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29286.127805 secs 2023-09-10T07:27:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:27:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:36 smithi170 ceph-mon[102832]: pgmap v15779: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 79 KiB/s wr, 9 op/s 2023-09-10T07:27:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:27:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:27:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:27:36.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:36 smithi082 ceph-mon[98694]: pgmap v15779: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 79 KiB/s wr, 9 op/s 2023-09-10T07:27:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:27:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:27:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:27:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:36 smithi098 ceph-mon[103124]: pgmap v15779: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 79 KiB/s wr, 9 op/s 2023-09-10T07:27:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:27:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:27:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:27:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:37 smithi170 ceph-mon[102832]: pgmap v15780: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 225 KiB/s wr, 15 op/s 2023-09-10T07:27:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:37 smithi082 ceph-mon[98694]: pgmap v15780: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 225 KiB/s wr, 15 op/s 2023-09-10T07:27:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:37 smithi098 ceph-mon[103124]: pgmap v15780: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 225 KiB/s wr, 15 op/s 2023-09-10T07:27:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29290.651260 secs 2023-09-10T07:27:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29291.127880 secs 2023-09-10T07:27:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:40 smithi170 ceph-mon[102832]: pgmap v15781: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 225 KiB/s wr, 11 op/s 2023-09-10T07:27:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29290.651260 secs 2023-09-10T07:27:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29291.127880 secs 2023-09-10T07:27:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:40 smithi082 ceph-mon[98694]: pgmap v15781: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 225 KiB/s wr, 11 op/s 2023-09-10T07:27:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29290.651260 secs 2023-09-10T07:27:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29291.127880 secs 2023-09-10T07:27:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:40 smithi098 ceph-mon[103124]: pgmap v15781: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 225 KiB/s wr, 11 op/s 2023-09-10T07:27:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:42 smithi170 ceph-mon[102832]: pgmap v15782: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 442 KiB/s wr, 17 op/s 2023-09-10T07:27:42.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:42 smithi082 ceph-mon[98694]: pgmap v15782: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 442 KiB/s wr, 17 op/s 2023-09-10T07:27:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:42 smithi098 ceph-mon[103124]: pgmap v15782: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 442 KiB/s wr, 17 op/s 2023-09-10T07:27:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:44 smithi170 ceph-mon[102832]: pgmap v15783: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 363 KiB/s wr, 12 op/s 2023-09-10T07:27:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:44 smithi082 ceph-mon[98694]: pgmap v15783: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 363 KiB/s wr, 12 op/s 2023-09-10T07:27:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:44 smithi098 ceph-mon[103124]: pgmap v15783: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 363 KiB/s wr, 12 op/s 2023-09-10T07:27:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29295.651318 secs 2023-09-10T07:27:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29296.127938 secs 2023-09-10T07:27:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29295.651318 secs 2023-09-10T07:27:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29296.127938 secs 2023-09-10T07:27:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29295.651318 secs 2023-09-10T07:27:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29296.127938 secs 2023-09-10T07:27:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:46 smithi170 ceph-mon[102832]: pgmap v15784: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 363 KiB/s wr, 12 op/s 2023-09-10T07:27:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:46 smithi082 ceph-mon[98694]: pgmap v15784: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 363 KiB/s wr, 12 op/s 2023-09-10T07:27:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:46 smithi098 ceph-mon[103124]: pgmap v15784: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 363 KiB/s wr, 12 op/s 2023-09-10T07:27:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:47 smithi170 ceph-mon[102832]: pgmap v15785: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 631 KiB/s wr, 20 op/s 2023-09-10T07:27:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:47 smithi082 ceph-mon[98694]: pgmap v15785: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 631 KiB/s wr, 20 op/s 2023-09-10T07:27:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:47 smithi098 ceph-mon[103124]: pgmap v15785: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 631 KiB/s wr, 20 op/s 2023-09-10T07:27:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29300.651376 secs 2023-09-10T07:27:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29301.128005 secs 2023-09-10T07:27:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:50 smithi170 ceph-mon[102832]: pgmap v15786: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 485 KiB/s wr, 14 op/s 2023-09-10T07:27:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29300.651376 secs 2023-09-10T07:27:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29301.128005 secs 2023-09-10T07:27:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:50 smithi082 ceph-mon[98694]: pgmap v15786: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 485 KiB/s wr, 14 op/s 2023-09-10T07:27:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29300.651376 secs 2023-09-10T07:27:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29301.128005 secs 2023-09-10T07:27:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:50 smithi098 ceph-mon[103124]: pgmap v15786: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 485 KiB/s wr, 14 op/s 2023-09-10T07:27:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:52 smithi170 ceph-mon[102832]: pgmap v15787: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 622 KiB/s wr, 20 op/s 2023-09-10T07:27:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:52 smithi082 ceph-mon[98694]: pgmap v15787: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 622 KiB/s wr, 20 op/s 2023-09-10T07:27:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:52 smithi098 ceph-mon[103124]: pgmap v15787: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 622 KiB/s wr, 20 op/s 2023-09-10T07:27:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29304.514695 secs 2023-09-10T07:27:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:54 smithi170 ceph-mon[102832]: pgmap v15788: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 405 KiB/s wr, 13 op/s 2023-09-10T07:27:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29304.514695 secs 2023-09-10T07:27:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:54 smithi082 ceph-mon[98694]: pgmap v15788: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 405 KiB/s wr, 13 op/s 2023-09-10T07:27:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29304.514695 secs 2023-09-10T07:27:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:54 smithi098 ceph-mon[103124]: pgmap v15788: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 405 KiB/s wr, 13 op/s 2023-09-10T07:27:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29305.651433 secs 2023-09-10T07:27:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29306.128068 secs 2023-09-10T07:27:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29305.651433 secs 2023-09-10T07:27:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29306.128068 secs 2023-09-10T07:27:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29305.651433 secs 2023-09-10T07:27:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29306.128068 secs 2023-09-10T07:27:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:56 smithi170 ceph-mon[102832]: pgmap v15789: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 405 KiB/s wr, 13 op/s 2023-09-10T07:27:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:27:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:27:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:27:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:56 smithi082 ceph-mon[98694]: pgmap v15789: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 405 KiB/s wr, 13 op/s 2023-09-10T07:27:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:27:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:27:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:27:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:56 smithi098 ceph-mon[103124]: pgmap v15789: 97 pgs: 97 active+clean; 9.8 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 405 KiB/s wr, 13 op/s 2023-09-10T07:27:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:27:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:27:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:27:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:57 smithi170 ceph-mon[102832]: pgmap v15790: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 547 KiB/s wr, 19 op/s 2023-09-10T07:27:58.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:57 smithi082 ceph-mon[98694]: pgmap v15790: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 547 KiB/s wr, 19 op/s 2023-09-10T07:27:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:57 smithi098 ceph-mon[103124]: pgmap v15790: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 547 KiB/s wr, 19 op/s 2023-09-10T07:27:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29309.514755 secs 2023-09-10T07:27:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29309.514755 secs 2023-09-10T07:27:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29309.514755 secs 2023-09-10T07:28:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29310.651490 secs 2023-09-10T07:28:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29311.128133 secs 2023-09-10T07:28:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:27:59 smithi170 ceph-mon[102832]: pgmap v15791: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T07:28:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29310.651490 secs 2023-09-10T07:28:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29311.128133 secs 2023-09-10T07:28:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:27:59 smithi082 ceph-mon[98694]: pgmap v15791: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T07:28:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29310.651490 secs 2023-09-10T07:28:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29311.128133 secs 2023-09-10T07:28:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:27:59 smithi098 ceph-mon[103124]: pgmap v15791: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T07:28:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:02 smithi170 ceph-mon[102832]: pgmap v15792: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 392 KiB/s wr, 17 op/s 2023-09-10T07:28:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:02 smithi082 ceph-mon[98694]: pgmap v15792: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 392 KiB/s wr, 17 op/s 2023-09-10T07:28:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:02 smithi098 ceph-mon[103124]: pgmap v15792: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 392 KiB/s wr, 17 op/s 2023-09-10T07:28:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29314.514831 secs 2023-09-10T07:28:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:04 smithi170 ceph-mon[102832]: pgmap v15793: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 255 KiB/s wr, 11 op/s 2023-09-10T07:28:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29314.514831 secs 2023-09-10T07:28:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:04 smithi082 ceph-mon[98694]: pgmap v15793: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 255 KiB/s wr, 11 op/s 2023-09-10T07:28:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29314.514831 secs 2023-09-10T07:28:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:04 smithi098 ceph-mon[103124]: pgmap v15793: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 255 KiB/s wr, 11 op/s 2023-09-10T07:28:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29315.651551 secs 2023-09-10T07:28:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29316.128194 secs 2023-09-10T07:28:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29315.651551 secs 2023-09-10T07:28:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29316.128194 secs 2023-09-10T07:28:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29315.651551 secs 2023-09-10T07:28:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29316.128194 secs 2023-09-10T07:28:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:06 smithi170 ceph-mon[102832]: pgmap v15794: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 255 KiB/s wr, 11 op/s 2023-09-10T07:28:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:06 smithi082 ceph-mon[98694]: pgmap v15794: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 255 KiB/s wr, 11 op/s 2023-09-10T07:28:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:06 smithi098 ceph-mon[103124]: pgmap v15794: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 255 KiB/s wr, 11 op/s 2023-09-10T07:28:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:07 smithi170 ceph-mon[102832]: pgmap v15795: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 300 KiB/s wr, 30 op/s 2023-09-10T07:28:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:07 smithi082 ceph-mon[98694]: pgmap v15795: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 300 KiB/s wr, 30 op/s 2023-09-10T07:28:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:07 smithi098 ceph-mon[103124]: pgmap v15795: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 300 KiB/s wr, 30 op/s 2023-09-10T07:28:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29319.514945 secs 2023-09-10T07:28:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:08 smithi170 ceph-mon[102832]: osdmap e1055: 12 total, 12 up, 12 in 2023-09-10T07:28:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29319.514945 secs 2023-09-10T07:28:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:08 smithi082 ceph-mon[98694]: osdmap e1055: 12 total, 12 up, 12 in 2023-09-10T07:28:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29319.514945 secs 2023-09-10T07:28:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:08 smithi098 ceph-mon[103124]: osdmap e1055: 12 total, 12 up, 12 in 2023-09-10T07:28:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29320.651635 secs 2023-09-10T07:28:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29321.128259 secs 2023-09-10T07:28:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:09 smithi170 ceph-mon[102832]: pgmap v15797: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 190 KiB/s wr, 29 op/s 2023-09-10T07:28:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:28:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29320.651635 secs 2023-09-10T07:28:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29321.128259 secs 2023-09-10T07:28:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:09 smithi082 ceph-mon[98694]: pgmap v15797: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 190 KiB/s wr, 29 op/s 2023-09-10T07:28:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:28:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29320.651635 secs 2023-09-10T07:28:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29321.128259 secs 2023-09-10T07:28:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:09 smithi098 ceph-mon[103124]: pgmap v15797: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 190 KiB/s wr, 29 op/s 2023-09-10T07:28:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:28:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:28:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:28:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:28:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:28:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:28:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:28:11.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:28:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:28:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:28:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:28:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:28:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:28:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:28:11.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:28:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:28:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:28:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:28:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:28:11.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:28:11.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:28:11.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:28:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:11 smithi098 ceph-mon[103124]: pgmap v15798: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:28:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:11 smithi170 ceph-mon[102832]: pgmap v15798: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:28:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:11 smithi082 ceph-mon[98694]: pgmap v15798: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:28:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29324.515060 secs 2023-09-10T07:28:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:14 smithi170 ceph-mon[102832]: pgmap v15799: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:28:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29324.515060 secs 2023-09-10T07:28:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:14 smithi082 ceph-mon[98694]: pgmap v15799: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:28:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29324.515060 secs 2023-09-10T07:28:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:14 smithi098 ceph-mon[103124]: pgmap v15799: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:28:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29325.651693 secs 2023-09-10T07:28:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29326.128319 secs 2023-09-10T07:28:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29325.651693 secs 2023-09-10T07:28:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29326.128319 secs 2023-09-10T07:28:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29325.651693 secs 2023-09-10T07:28:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29326.128319 secs 2023-09-10T07:28:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:16 smithi170 ceph-mon[102832]: pgmap v15800: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:28:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:16 smithi082 ceph-mon[98694]: pgmap v15800: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:28:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:16 smithi098 ceph-mon[103124]: pgmap v15800: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:28:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:17 smithi170 ceph-mon[102832]: osdmap e1056: 12 total, 12 up, 12 in 2023-09-10T07:28:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:17 smithi170 ceph-mon[102832]: pgmap v15802: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:28:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:17 smithi082 ceph-mon[98694]: osdmap e1056: 12 total, 12 up, 12 in 2023-09-10T07:28:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:17 smithi082 ceph-mon[98694]: pgmap v15802: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:28:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:17 smithi098 ceph-mon[103124]: osdmap e1056: 12 total, 12 up, 12 in 2023-09-10T07:28:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:17 smithi098 ceph-mon[103124]: pgmap v15802: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:28:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29329.515200 secs 2023-09-10T07:28:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29329.515200 secs 2023-09-10T07:28:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29329.515200 secs 2023-09-10T07:28:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29330.651776 secs 2023-09-10T07:28:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29331.128388 secs 2023-09-10T07:28:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:19 smithi170 ceph-mon[102832]: pgmap v15803: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:28:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29330.651776 secs 2023-09-10T07:28:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29331.128388 secs 2023-09-10T07:28:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:19 smithi082 ceph-mon[98694]: pgmap v15803: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:28:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29330.651776 secs 2023-09-10T07:28:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29331.128388 secs 2023-09-10T07:28:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:19 smithi098 ceph-mon[103124]: pgmap v15803: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:28:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:22 smithi170 ceph-mon[102832]: pgmap v15804: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2023-09-10T07:28:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:22 smithi082 ceph-mon[98694]: pgmap v15804: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2023-09-10T07:28:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:22 smithi098 ceph-mon[103124]: pgmap v15804: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2023-09-10T07:28:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29334.515259 secs 2023-09-10T07:28:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:24 smithi170 ceph-mon[102832]: pgmap v15805: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2023-09-10T07:28:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29334.515259 secs 2023-09-10T07:28:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:24 smithi082 ceph-mon[98694]: pgmap v15805: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2023-09-10T07:28:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29334.515259 secs 2023-09-10T07:28:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:24 smithi098 ceph-mon[103124]: pgmap v15805: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2023-09-10T07:28:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29335.651835 secs 2023-09-10T07:28:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29336.128448 secs 2023-09-10T07:28:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29335.651835 secs 2023-09-10T07:28:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29336.128448 secs 2023-09-10T07:28:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29335.651835 secs 2023-09-10T07:28:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29336.128448 secs 2023-09-10T07:28:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:26 smithi170 ceph-mon[102832]: pgmap v15806: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2023-09-10T07:28:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:26 smithi082 ceph-mon[98694]: pgmap v15806: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2023-09-10T07:28:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:26 smithi098 ceph-mon[103124]: pgmap v15806: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2023-09-10T07:28:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:27 smithi170 ceph-mon[102832]: pgmap v15807: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 114 KiB/s wr, 10 op/s 2023-09-10T07:28:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:27 smithi082 ceph-mon[98694]: pgmap v15807: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 114 KiB/s wr, 10 op/s 2023-09-10T07:28:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:27 smithi098 ceph-mon[103124]: pgmap v15807: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 114 KiB/s wr, 10 op/s 2023-09-10T07:28:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29339.515354 secs 2023-09-10T07:28:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29339.515354 secs 2023-09-10T07:28:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29339.515354 secs 2023-09-10T07:28:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29340.651892 secs 2023-09-10T07:28:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29341.128489 secs 2023-09-10T07:28:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:29 smithi170 ceph-mon[102832]: pgmap v15808: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 9 op/s 2023-09-10T07:28:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29340.651892 secs 2023-09-10T07:28:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29341.128489 secs 2023-09-10T07:28:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:29 smithi082 ceph-mon[98694]: pgmap v15808: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 9 op/s 2023-09-10T07:28:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29340.651892 secs 2023-09-10T07:28:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29341.128489 secs 2023-09-10T07:28:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:29 smithi098 ceph-mon[103124]: pgmap v15808: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 9 op/s 2023-09-10T07:28:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:32 smithi170 ceph-mon[102832]: pgmap v15809: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 114 KiB/s wr, 14 op/s 2023-09-10T07:28:32.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:32 smithi082 ceph-mon[98694]: pgmap v15809: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 114 KiB/s wr, 14 op/s 2023-09-10T07:28:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:32 smithi098 ceph-mon[103124]: pgmap v15809: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 114 KiB/s wr, 14 op/s 2023-09-10T07:28:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:34 smithi170 ceph-mon[102832]: pgmap v15810: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 9 op/s 2023-09-10T07:28:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:34 smithi082 ceph-mon[98694]: pgmap v15810: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 9 op/s 2023-09-10T07:28:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:34 smithi098 ceph-mon[103124]: pgmap v15810: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 9 op/s 2023-09-10T07:28:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29345.651950 secs 2023-09-10T07:28:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29346.128547 secs 2023-09-10T07:28:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:28:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29345.651950 secs 2023-09-10T07:28:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29346.128547 secs 2023-09-10T07:28:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:28:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29345.651950 secs 2023-09-10T07:28:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29346.128547 secs 2023-09-10T07:28:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:28:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:36 smithi170 ceph-mon[102832]: pgmap v15811: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 9 op/s 2023-09-10T07:28:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:28:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:28:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:28:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:36 smithi082 ceph-mon[98694]: pgmap v15811: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 9 op/s 2023-09-10T07:28:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:28:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:28:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:28:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:36 smithi098 ceph-mon[103124]: pgmap v15811: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 9 op/s 2023-09-10T07:28:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:28:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:28:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:28:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:37 smithi170 ceph-mon[102832]: pgmap v15812: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 322 KiB/s wr, 16 op/s 2023-09-10T07:28:38.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:37 smithi082 ceph-mon[98694]: pgmap v15812: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 322 KiB/s wr, 16 op/s 2023-09-10T07:28:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:37 smithi098 ceph-mon[103124]: pgmap v15812: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 322 KiB/s wr, 16 op/s 2023-09-10T07:28:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29350.652047 secs 2023-09-10T07:28:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29351.128613 secs 2023-09-10T07:28:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:40 smithi170 ceph-mon[102832]: pgmap v15813: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 225 KiB/s wr, 11 op/s 2023-09-10T07:28:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29350.652047 secs 2023-09-10T07:28:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29351.128613 secs 2023-09-10T07:28:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:40 smithi082 ceph-mon[98694]: pgmap v15813: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 225 KiB/s wr, 11 op/s 2023-09-10T07:28:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29350.652047 secs 2023-09-10T07:28:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29351.128613 secs 2023-09-10T07:28:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:40 smithi098 ceph-mon[103124]: pgmap v15813: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 225 KiB/s wr, 11 op/s 2023-09-10T07:28:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:42 smithi170 ceph-mon[102832]: pgmap v15814: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 459 KiB/s wr, 18 op/s 2023-09-10T07:28:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:42 smithi082 ceph-mon[98694]: pgmap v15814: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 459 KiB/s wr, 18 op/s 2023-09-10T07:28:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:42 smithi098 ceph-mon[103124]: pgmap v15814: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 459 KiB/s wr, 18 op/s 2023-09-10T07:28:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:44 smithi170 ceph-mon[102832]: pgmap v15815: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 442 KiB/s wr, 12 op/s 2023-09-10T07:28:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:44 smithi082 ceph-mon[98694]: pgmap v15815: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 442 KiB/s wr, 12 op/s 2023-09-10T07:28:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:44 smithi098 ceph-mon[103124]: pgmap v15815: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 442 KiB/s wr, 12 op/s 2023-09-10T07:28:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29355.652104 secs 2023-09-10T07:28:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29356.128670 secs 2023-09-10T07:28:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29355.652104 secs 2023-09-10T07:28:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29356.128670 secs 2023-09-10T07:28:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29355.652104 secs 2023-09-10T07:28:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29356.128670 secs 2023-09-10T07:28:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:46 smithi170 ceph-mon[102832]: pgmap v15816: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 442 KiB/s wr, 12 op/s 2023-09-10T07:28:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:46 smithi082 ceph-mon[98694]: pgmap v15816: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 442 KiB/s wr, 12 op/s 2023-09-10T07:28:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:46 smithi098 ceph-mon[103124]: pgmap v15816: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 442 KiB/s wr, 12 op/s 2023-09-10T07:28:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:47 smithi170 ceph-mon[102832]: pgmap v15817: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 694 KiB/s wr, 19 op/s 2023-09-10T07:28:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:47 smithi082 ceph-mon[98694]: pgmap v15817: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 694 KiB/s wr, 19 op/s 2023-09-10T07:28:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:47 smithi098 ceph-mon[103124]: pgmap v15817: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 694 KiB/s wr, 19 op/s 2023-09-10T07:28:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29360.652182 secs 2023-09-10T07:28:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29361.128733 secs 2023-09-10T07:28:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:50 smithi170 ceph-mon[102832]: pgmap v15818: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 486 KiB/s wr, 13 op/s 2023-09-10T07:28:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29360.652182 secs 2023-09-10T07:28:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29361.128733 secs 2023-09-10T07:28:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:50 smithi082 ceph-mon[98694]: pgmap v15818: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 486 KiB/s wr, 13 op/s 2023-09-10T07:28:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29360.652182 secs 2023-09-10T07:28:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29361.128733 secs 2023-09-10T07:28:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:50 smithi098 ceph-mon[103124]: pgmap v15818: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 486 KiB/s wr, 13 op/s 2023-09-10T07:28:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:52 smithi170 ceph-mon[102832]: pgmap v15819: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 624 KiB/s wr, 20 op/s 2023-09-10T07:28:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:52 smithi082 ceph-mon[98694]: pgmap v15819: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 624 KiB/s wr, 20 op/s 2023-09-10T07:28:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:52 smithi098 ceph-mon[103124]: pgmap v15819: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 624 KiB/s wr, 20 op/s 2023-09-10T07:28:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29364.515688 secs 2023-09-10T07:28:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:54 smithi170 ceph-mon[102832]: pgmap v15820: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 390 KiB/s wr, 13 op/s 2023-09-10T07:28:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29364.515688 secs 2023-09-10T07:28:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:54 smithi082 ceph-mon[98694]: pgmap v15820: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 390 KiB/s wr, 13 op/s 2023-09-10T07:28:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29364.515688 secs 2023-09-10T07:28:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:54 smithi098 ceph-mon[103124]: pgmap v15820: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 390 KiB/s wr, 13 op/s 2023-09-10T07:28:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29365.652251 secs 2023-09-10T07:28:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29366.128791 secs 2023-09-10T07:28:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29365.652251 secs 2023-09-10T07:28:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29366.128791 secs 2023-09-10T07:28:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29365.652251 secs 2023-09-10T07:28:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29366.128791 secs 2023-09-10T07:28:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:56 smithi170 ceph-mon[102832]: pgmap v15821: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 390 KiB/s wr, 13 op/s 2023-09-10T07:28:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:28:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:28:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:28:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:56 smithi082 ceph-mon[98694]: pgmap v15821: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 390 KiB/s wr, 13 op/s 2023-09-10T07:28:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:28:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:28:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:28:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:56 smithi098 ceph-mon[103124]: pgmap v15821: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 390 KiB/s wr, 13 op/s 2023-09-10T07:28:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:28:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:28:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:28:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:57 smithi170 ceph-mon[102832]: pgmap v15822: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 532 KiB/s wr, 19 op/s 2023-09-10T07:28:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:57 smithi082 ceph-mon[98694]: pgmap v15822: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 532 KiB/s wr, 19 op/s 2023-09-10T07:28:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:57 smithi098 ceph-mon[103124]: pgmap v15822: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 532 KiB/s wr, 19 op/s 2023-09-10T07:28:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29369.515798 secs 2023-09-10T07:28:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29369.515798 secs 2023-09-10T07:28:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29369.515798 secs 2023-09-10T07:29:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29370.652310 secs 2023-09-10T07:29:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29371.128859 secs 2023-09-10T07:29:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:28:59 smithi170 ceph-mon[102832]: pgmap v15823: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 280 KiB/s wr, 13 op/s 2023-09-10T07:29:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29370.652310 secs 2023-09-10T07:29:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29371.128859 secs 2023-09-10T07:29:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:28:59 smithi082 ceph-mon[98694]: pgmap v15823: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 280 KiB/s wr, 13 op/s 2023-09-10T07:29:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29370.652310 secs 2023-09-10T07:29:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29371.128859 secs 2023-09-10T07:29:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:28:59 smithi098 ceph-mon[103124]: pgmap v15823: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 280 KiB/s wr, 13 op/s 2023-09-10T07:29:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:02 smithi170 ceph-mon[102832]: pgmap v15824: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 394 KiB/s wr, 18 op/s 2023-09-10T07:29:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:02 smithi082 ceph-mon[98694]: pgmap v15824: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 394 KiB/s wr, 18 op/s 2023-09-10T07:29:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:02 smithi098 ceph-mon[103124]: pgmap v15824: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 394 KiB/s wr, 18 op/s 2023-09-10T07:29:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29374.515898 secs 2023-09-10T07:29:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:04 smithi170 ceph-mon[102832]: pgmap v15825: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 256 KiB/s wr, 11 op/s 2023-09-10T07:29:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29374.515898 secs 2023-09-10T07:29:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:04 smithi082 ceph-mon[98694]: pgmap v15825: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 256 KiB/s wr, 11 op/s 2023-09-10T07:29:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29374.515898 secs 2023-09-10T07:29:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:04 smithi098 ceph-mon[103124]: pgmap v15825: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 256 KiB/s wr, 11 op/s 2023-09-10T07:29:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29375.652397 secs 2023-09-10T07:29:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29376.128916 secs 2023-09-10T07:29:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29375.652397 secs 2023-09-10T07:29:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29376.128916 secs 2023-09-10T07:29:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29375.652397 secs 2023-09-10T07:29:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29376.128916 secs 2023-09-10T07:29:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:06 smithi170 ceph-mon[102832]: pgmap v15826: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 256 KiB/s wr, 11 op/s 2023-09-10T07:29:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:06 smithi082 ceph-mon[98694]: pgmap v15826: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 256 KiB/s wr, 11 op/s 2023-09-10T07:29:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:06 smithi098 ceph-mon[103124]: pgmap v15826: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 256 KiB/s wr, 11 op/s 2023-09-10T07:29:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:07 smithi170 ceph-mon[102832]: pgmap v15827: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 301 KiB/s wr, 30 op/s 2023-09-10T07:29:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:07 smithi082 ceph-mon[98694]: pgmap v15827: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 301 KiB/s wr, 30 op/s 2023-09-10T07:29:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:07 smithi098 ceph-mon[103124]: pgmap v15827: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 301 KiB/s wr, 30 op/s 2023-09-10T07:29:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29379.515957 secs 2023-09-10T07:29:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:08 smithi170 ceph-mon[102832]: osdmap e1057: 12 total, 12 up, 12 in 2023-09-10T07:29:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29379.515957 secs 2023-09-10T07:29:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:08 smithi082 ceph-mon[98694]: osdmap e1057: 12 total, 12 up, 12 in 2023-09-10T07:29:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29379.515957 secs 2023-09-10T07:29:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:08 smithi098 ceph-mon[103124]: osdmap e1057: 12 total, 12 up, 12 in 2023-09-10T07:29:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29380.652454 secs 2023-09-10T07:29:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29381.128980 secs 2023-09-10T07:29:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:09 smithi170 ceph-mon[102832]: pgmap v15829: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 190 KiB/s wr, 29 op/s 2023-09-10T07:29:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29380.652454 secs 2023-09-10T07:29:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29381.128980 secs 2023-09-10T07:29:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:09 smithi082 ceph-mon[98694]: pgmap v15829: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 190 KiB/s wr, 29 op/s 2023-09-10T07:29:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29380.652454 secs 2023-09-10T07:29:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29381.128980 secs 2023-09-10T07:29:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:09 smithi098 ceph-mon[103124]: pgmap v15829: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 190 KiB/s wr, 29 op/s 2023-09-10T07:29:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:29:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:29:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:29:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:11 smithi170 ceph-mon[102832]: pgmap v15830: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:29:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:29:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:29:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:29:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:11 smithi082 ceph-mon[98694]: pgmap v15830: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:29:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:29:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:29:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:29:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:11 smithi098 ceph-mon[103124]: pgmap v15830: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:29:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:29:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:29:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:29:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29384.516017 secs 2023-09-10T07:29:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:14 smithi170 ceph-mon[102832]: pgmap v15831: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:29:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29384.516017 secs 2023-09-10T07:29:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:14 smithi082 ceph-mon[98694]: pgmap v15831: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:29:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29384.516017 secs 2023-09-10T07:29:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:14 smithi098 ceph-mon[103124]: pgmap v15831: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:29:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29385.652514 secs 2023-09-10T07:29:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29386.129039 secs 2023-09-10T07:29:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29385.652514 secs 2023-09-10T07:29:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29386.129039 secs 2023-09-10T07:29:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29385.652514 secs 2023-09-10T07:29:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29386.129039 secs 2023-09-10T07:29:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:16 smithi170 ceph-mon[102832]: pgmap v15832: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:29:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:16 smithi082 ceph-mon[98694]: pgmap v15832: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:29:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:16 smithi098 ceph-mon[103124]: pgmap v15832: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:29:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:17 smithi170 ceph-mon[102832]: osdmap e1058: 12 total, 12 up, 12 in 2023-09-10T07:29:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:17 smithi170 ceph-mon[102832]: pgmap v15834: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:29:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:17 smithi082 ceph-mon[98694]: osdmap e1058: 12 total, 12 up, 12 in 2023-09-10T07:29:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:17 smithi082 ceph-mon[98694]: pgmap v15834: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:29:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:17 smithi098 ceph-mon[103124]: osdmap e1058: 12 total, 12 up, 12 in 2023-09-10T07:29:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:17 smithi098 ceph-mon[103124]: pgmap v15834: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:29:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29389.516106 secs 2023-09-10T07:29:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29389.516106 secs 2023-09-10T07:29:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29389.516106 secs 2023-09-10T07:29:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29390.652590 secs 2023-09-10T07:29:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29391.129105 secs 2023-09-10T07:29:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:19 smithi170 ceph-mon[102832]: pgmap v15835: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:29:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29390.652590 secs 2023-09-10T07:29:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29391.129105 secs 2023-09-10T07:29:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:19 smithi082 ceph-mon[98694]: pgmap v15835: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:29:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29390.652590 secs 2023-09-10T07:29:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29391.129105 secs 2023-09-10T07:29:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:19 smithi098 ceph-mon[103124]: pgmap v15835: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:29:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:22 smithi170 ceph-mon[102832]: pgmap v15836: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:29:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:22 smithi082 ceph-mon[98694]: pgmap v15836: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:29:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:22 smithi098 ceph-mon[103124]: pgmap v15836: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:29:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29394.516168 secs 2023-09-10T07:29:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:24 smithi170 ceph-mon[102832]: pgmap v15837: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:29:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29394.516168 secs 2023-09-10T07:29:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:24 smithi082 ceph-mon[98694]: pgmap v15837: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:29:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29394.516168 secs 2023-09-10T07:29:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:24 smithi098 ceph-mon[103124]: pgmap v15837: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:29:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29395.652652 secs 2023-09-10T07:29:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29396.129164 secs 2023-09-10T07:29:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29395.652652 secs 2023-09-10T07:29:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29396.129164 secs 2023-09-10T07:29:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29395.652652 secs 2023-09-10T07:29:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29396.129164 secs 2023-09-10T07:29:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:26 smithi170 ceph-mon[102832]: pgmap v15838: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:29:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:26 smithi082 ceph-mon[98694]: pgmap v15838: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:29:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:26 smithi098 ceph-mon[103124]: pgmap v15838: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:29:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:27 smithi170 ceph-mon[102832]: pgmap v15839: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 704 B/s wr, 10 op/s 2023-09-10T07:29:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:27 smithi082 ceph-mon[98694]: pgmap v15839: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 704 B/s wr, 10 op/s 2023-09-10T07:29:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:27 smithi098 ceph-mon[103124]: pgmap v15839: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 704 B/s wr, 10 op/s 2023-09-10T07:29:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29399.516228 secs 2023-09-10T07:29:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29399.516228 secs 2023-09-10T07:29:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29399.516228 secs 2023-09-10T07:29:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29400.652712 secs 2023-09-10T07:29:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29401.129226 secs 2023-09-10T07:29:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:29 smithi170 ceph-mon[102832]: pgmap v15840: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 8 op/s 2023-09-10T07:29:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29400.652712 secs 2023-09-10T07:29:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29401.129226 secs 2023-09-10T07:29:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:29 smithi082 ceph-mon[98694]: pgmap v15840: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 8 op/s 2023-09-10T07:29:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29400.652712 secs 2023-09-10T07:29:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29401.129226 secs 2023-09-10T07:29:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:29 smithi098 ceph-mon[103124]: pgmap v15840: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 8 op/s 2023-09-10T07:29:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:32 smithi170 ceph-mon[102832]: pgmap v15841: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 102 KiB/s wr, 13 op/s 2023-09-10T07:29:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:32 smithi082 ceph-mon[98694]: pgmap v15841: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 102 KiB/s wr, 13 op/s 2023-09-10T07:29:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:32 smithi098 ceph-mon[103124]: pgmap v15841: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 102 KiB/s wr, 13 op/s 2023-09-10T07:29:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:34 smithi170 ceph-mon[102832]: pgmap v15842: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 9 op/s 2023-09-10T07:29:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:34 smithi082 ceph-mon[98694]: pgmap v15842: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 9 op/s 2023-09-10T07:29:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:34 smithi098 ceph-mon[103124]: pgmap v15842: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 9 op/s 2023-09-10T07:29:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29405.652800 secs 2023-09-10T07:29:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29406.129285 secs 2023-09-10T07:29:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:29:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29405.652800 secs 2023-09-10T07:29:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29406.129285 secs 2023-09-10T07:29:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:29:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29405.652800 secs 2023-09-10T07:29:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29406.129285 secs 2023-09-10T07:29:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:29:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:36 smithi170 ceph-mon[102832]: pgmap v15843: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 9 op/s 2023-09-10T07:29:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:29:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:29:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:29:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:36 smithi082 ceph-mon[98694]: pgmap v15843: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 9 op/s 2023-09-10T07:29:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:29:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:29:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:29:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:36 smithi098 ceph-mon[103124]: pgmap v15843: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 9 op/s 2023-09-10T07:29:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:29:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:29:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:29:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:37 smithi170 ceph-mon[102832]: pgmap v15844: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 202 KiB/s wr, 15 op/s 2023-09-10T07:29:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:37 smithi082 ceph-mon[98694]: pgmap v15844: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 202 KiB/s wr, 15 op/s 2023-09-10T07:29:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:37 smithi098 ceph-mon[103124]: pgmap v15844: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 202 KiB/s wr, 15 op/s 2023-09-10T07:29:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29410.652860 secs 2023-09-10T07:29:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29411.129351 secs 2023-09-10T07:29:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:40 smithi170 ceph-mon[102832]: pgmap v15845: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 201 KiB/s wr, 10 op/s 2023-09-10T07:29:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29410.652860 secs 2023-09-10T07:29:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29411.129351 secs 2023-09-10T07:29:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:40 smithi082 ceph-mon[98694]: pgmap v15845: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 201 KiB/s wr, 10 op/s 2023-09-10T07:29:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29410.652860 secs 2023-09-10T07:29:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29411.129351 secs 2023-09-10T07:29:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:40 smithi098 ceph-mon[103124]: pgmap v15845: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 201 KiB/s wr, 10 op/s 2023-09-10T07:29:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:42 smithi170 ceph-mon[102832]: pgmap v15846: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 420 KiB/s wr, 16 op/s 2023-09-10T07:29:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:42 smithi082 ceph-mon[98694]: pgmap v15846: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 420 KiB/s wr, 16 op/s 2023-09-10T07:29:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:42 smithi098 ceph-mon[103124]: pgmap v15846: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 420 KiB/s wr, 16 op/s 2023-09-10T07:29:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:44 smithi170 ceph-mon[102832]: pgmap v15847: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 318 KiB/s wr, 11 op/s 2023-09-10T07:29:44.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:44 smithi082 ceph-mon[98694]: pgmap v15847: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 318 KiB/s wr, 11 op/s 2023-09-10T07:29:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:44 smithi098 ceph-mon[103124]: pgmap v15847: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 318 KiB/s wr, 11 op/s 2023-09-10T07:29:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29415.652918 secs 2023-09-10T07:29:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29416.129411 secs 2023-09-10T07:29:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29415.652918 secs 2023-09-10T07:29:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29416.129411 secs 2023-09-10T07:29:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29415.652918 secs 2023-09-10T07:29:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29416.129411 secs 2023-09-10T07:29:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:46 smithi170 ceph-mon[102832]: pgmap v15848: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 318 KiB/s wr, 11 op/s 2023-09-10T07:29:46.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:46 smithi082 ceph-mon[98694]: pgmap v15848: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 318 KiB/s wr, 11 op/s 2023-09-10T07:29:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:46 smithi098 ceph-mon[103124]: pgmap v15848: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 318 KiB/s wr, 11 op/s 2023-09-10T07:29:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:47 smithi170 ceph-mon[102832]: pgmap v15849: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 657 KiB/s wr, 18 op/s 2023-09-10T07:29:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:47 smithi082 ceph-mon[98694]: pgmap v15849: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 657 KiB/s wr, 18 op/s 2023-09-10T07:29:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:47 smithi098 ceph-mon[103124]: pgmap v15849: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 657 KiB/s wr, 18 op/s 2023-09-10T07:29:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29420.653004 secs 2023-09-10T07:29:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29421.129480 secs 2023-09-10T07:29:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:50 smithi170 ceph-mon[102832]: pgmap v15850: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 558 KiB/s wr, 13 op/s 2023-09-10T07:29:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29420.653004 secs 2023-09-10T07:29:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29421.129480 secs 2023-09-10T07:29:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:50 smithi082 ceph-mon[98694]: pgmap v15850: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 558 KiB/s wr, 13 op/s 2023-09-10T07:29:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29420.653004 secs 2023-09-10T07:29:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29421.129480 secs 2023-09-10T07:29:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:50 smithi098 ceph-mon[103124]: pgmap v15850: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 558 KiB/s wr, 13 op/s 2023-09-10T07:29:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:52 smithi170 ceph-mon[102832]: pgmap v15851: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 821 KiB/s wr, 20 op/s 2023-09-10T07:29:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:52 smithi082 ceph-mon[98694]: pgmap v15851: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 821 KiB/s wr, 20 op/s 2023-09-10T07:29:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:52 smithi098 ceph-mon[103124]: pgmap v15851: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 821 KiB/s wr, 20 op/s 2023-09-10T07:29:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:54 smithi170 ceph-mon[102832]: pgmap v15852: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 602 KiB/s wr, 15 op/s 2023-09-10T07:29:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:54 smithi082 ceph-mon[98694]: pgmap v15852: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 602 KiB/s wr, 15 op/s 2023-09-10T07:29:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:54 smithi098 ceph-mon[103124]: pgmap v15852: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 602 KiB/s wr, 15 op/s 2023-09-10T07:29:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29425.653063 secs 2023-09-10T07:29:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29426.129538 secs 2023-09-10T07:29:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29425.653063 secs 2023-09-10T07:29:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29426.129538 secs 2023-09-10T07:29:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29425.653063 secs 2023-09-10T07:29:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29426.129538 secs 2023-09-10T07:29:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:56 smithi170 ceph-mon[102832]: pgmap v15853: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 602 KiB/s wr, 15 op/s 2023-09-10T07:29:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:56 smithi082 ceph-mon[98694]: pgmap v15853: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 602 KiB/s wr, 15 op/s 2023-09-10T07:29:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:56 smithi098 ceph-mon[103124]: pgmap v15853: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 602 KiB/s wr, 15 op/s 2023-09-10T07:29:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:57 smithi170 ceph-mon[102832]: pgmap v15854: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 678 KiB/s wr, 21 op/s 2023-09-10T07:29:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:57 smithi082 ceph-mon[98694]: pgmap v15854: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 678 KiB/s wr, 21 op/s 2023-09-10T07:29:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:57 smithi098 ceph-mon[103124]: pgmap v15854: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 678 KiB/s wr, 21 op/s 2023-09-10T07:29:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29429.516774 secs 2023-09-10T07:29:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29429.516774 secs 2023-09-10T07:29:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29429.516774 secs 2023-09-10T07:30:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29430.653147 secs 2023-09-10T07:30:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29431.129600 secs 2023-09-10T07:30:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:29:59 smithi170 ceph-mon[102832]: pgmap v15855: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 339 KiB/s wr, 14 op/s 2023-09-10T07:30:00.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29430.653147 secs 2023-09-10T07:30:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29431.129600 secs 2023-09-10T07:30:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:29:59 smithi082 ceph-mon[98694]: pgmap v15855: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 339 KiB/s wr, 14 op/s 2023-09-10T07:30:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29430.653147 secs 2023-09-10T07:30:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29431.129600 secs 2023-09-10T07:30:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:29:59 smithi098 ceph-mon[103124]: pgmap v15855: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 339 KiB/s wr, 14 op/s 2023-09-10T07:30:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:30:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:30:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:30:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T07:30:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:30:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:30:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:30:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T07:30:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:30:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:30:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:30:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T07:30:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:01 smithi170 ceph-mon[102832]: pgmap v15856: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 576 KiB/s wr, 20 op/s 2023-09-10T07:30:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:01 smithi082 ceph-mon[98694]: pgmap v15856: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 576 KiB/s wr, 20 op/s 2023-09-10T07:30:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:01 smithi098 ceph-mon[103124]: pgmap v15856: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 576 KiB/s wr, 20 op/s 2023-09-10T07:30:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29434.516835 secs 2023-09-10T07:30:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:04 smithi170 ceph-mon[102832]: pgmap v15857: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 314 KiB/s wr, 12 op/s 2023-09-10T07:30:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29434.516835 secs 2023-09-10T07:30:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:04 smithi082 ceph-mon[98694]: pgmap v15857: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 314 KiB/s wr, 12 op/s 2023-09-10T07:30:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29434.516835 secs 2023-09-10T07:30:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:04 smithi098 ceph-mon[103124]: pgmap v15857: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 314 KiB/s wr, 12 op/s 2023-09-10T07:30:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29435.653206 secs 2023-09-10T07:30:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29436.129658 secs 2023-09-10T07:30:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29435.653206 secs 2023-09-10T07:30:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29436.129658 secs 2023-09-10T07:30:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29435.653206 secs 2023-09-10T07:30:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29436.129658 secs 2023-09-10T07:30:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:06 smithi170 ceph-mon[102832]: pgmap v15858: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 314 KiB/s wr, 12 op/s 2023-09-10T07:30:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:06 smithi082 ceph-mon[98694]: pgmap v15858: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 314 KiB/s wr, 12 op/s 2023-09-10T07:30:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:06 smithi098 ceph-mon[103124]: pgmap v15858: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 314 KiB/s wr, 12 op/s 2023-09-10T07:30:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:07 smithi170 ceph-mon[102832]: pgmap v15859: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 440 KiB/s wr, 32 op/s 2023-09-10T07:30:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:07 smithi082 ceph-mon[98694]: pgmap v15859: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 440 KiB/s wr, 32 op/s 2023-09-10T07:30:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:07 smithi098 ceph-mon[103124]: pgmap v15859: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 440 KiB/s wr, 32 op/s 2023-09-10T07:30:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29439.516893 secs 2023-09-10T07:30:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:08 smithi170 ceph-mon[102832]: osdmap e1059: 12 total, 12 up, 12 in 2023-09-10T07:30:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29439.516893 secs 2023-09-10T07:30:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:08 smithi082 ceph-mon[98694]: osdmap e1059: 12 total, 12 up, 12 in 2023-09-10T07:30:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29439.516893 secs 2023-09-10T07:30:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:08 smithi098 ceph-mon[103124]: osdmap e1059: 12 total, 12 up, 12 in 2023-09-10T07:30:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29440.653263 secs 2023-09-10T07:30:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29441.129723 secs 2023-09-10T07:30:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:09 smithi170 ceph-mon[102832]: pgmap v15861: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 435 KiB/s wr, 30 op/s 2023-09-10T07:30:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29440.653263 secs 2023-09-10T07:30:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29441.129723 secs 2023-09-10T07:30:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:09 smithi082 ceph-mon[98694]: pgmap v15861: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 435 KiB/s wr, 30 op/s 2023-09-10T07:30:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29440.653263 secs 2023-09-10T07:30:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29441.129723 secs 2023-09-10T07:30:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:09 smithi098 ceph-mon[103124]: pgmap v15861: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 435 KiB/s wr, 30 op/s 2023-09-10T07:30:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:12 smithi170 ceph-mon[102832]: pgmap v15862: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 151 KiB/s wr, 28 op/s 2023-09-10T07:30:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:30:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:12 smithi170 ceph-mon[102832]: osdmap e1060: 12 total, 12 up, 12 in 2023-09-10T07:30:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:30:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:30:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:30:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:12 smithi082 ceph-mon[98694]: pgmap v15862: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 151 KiB/s wr, 28 op/s 2023-09-10T07:30:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:30:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:12 smithi082 ceph-mon[98694]: osdmap e1060: 12 total, 12 up, 12 in 2023-09-10T07:30:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:30:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:30:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:30:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:12 smithi098 ceph-mon[103124]: pgmap v15862: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 151 KiB/s wr, 28 op/s 2023-09-10T07:30:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:30:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:12 smithi098 ceph-mon[103124]: osdmap e1060: 12 total, 12 up, 12 in 2023-09-10T07:30:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:30:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:30:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:30:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29444.516953 secs 2023-09-10T07:30:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:14 smithi170 ceph-mon[102832]: pgmap v15864: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 189 KiB/s wr, 35 op/s 2023-09-10T07:30:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29444.516953 secs 2023-09-10T07:30:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:14 smithi082 ceph-mon[98694]: pgmap v15864: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 189 KiB/s wr, 35 op/s 2023-09-10T07:30:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29444.516953 secs 2023-09-10T07:30:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:14 smithi098 ceph-mon[103124]: pgmap v15864: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 189 KiB/s wr, 35 op/s 2023-09-10T07:30:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29445.653322 secs 2023-09-10T07:30:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29446.129781 secs 2023-09-10T07:30:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29445.653322 secs 2023-09-10T07:30:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29446.129781 secs 2023-09-10T07:30:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29445.653322 secs 2023-09-10T07:30:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29446.129781 secs 2023-09-10T07:30:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:16 smithi170 ceph-mon[102832]: pgmap v15865: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:30:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:16 smithi082 ceph-mon[98694]: pgmap v15865: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:30:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:16 smithi098 ceph-mon[103124]: pgmap v15865: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:30:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:17 smithi170 ceph-mon[102832]: pgmap v15866: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:30:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:17 smithi082 ceph-mon[98694]: pgmap v15866: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:30:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:17 smithi098 ceph-mon[103124]: pgmap v15866: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:30:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29449.517012 secs 2023-09-10T07:30:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29449.517012 secs 2023-09-10T07:30:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29449.517012 secs 2023-09-10T07:30:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29450.653381 secs 2023-09-10T07:30:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29451.129846 secs 2023-09-10T07:30:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:19 smithi170 ceph-mon[102832]: pgmap v15867: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:30:20.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29450.653381 secs 2023-09-10T07:30:20.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29451.129846 secs 2023-09-10T07:30:20.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:19 smithi082 ceph-mon[98694]: pgmap v15867: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:30:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29450.653381 secs 2023-09-10T07:30:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29451.129846 secs 2023-09-10T07:30:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:19 smithi098 ceph-mon[103124]: pgmap v15867: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:30:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:22 smithi170 ceph-mon[102832]: pgmap v15868: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:30:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:22 smithi082 ceph-mon[98694]: pgmap v15868: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:30:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:22 smithi098 ceph-mon[103124]: pgmap v15868: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:30:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29454.517071 secs 2023-09-10T07:30:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:24 smithi170 ceph-mon[102832]: pgmap v15869: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:30:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29454.517071 secs 2023-09-10T07:30:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:24 smithi082 ceph-mon[98694]: pgmap v15869: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:30:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29454.517071 secs 2023-09-10T07:30:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:24 smithi098 ceph-mon[103124]: pgmap v15869: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:30:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29455.653439 secs 2023-09-10T07:30:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29456.129905 secs 2023-09-10T07:30:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29455.653439 secs 2023-09-10T07:30:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29456.129905 secs 2023-09-10T07:30:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29455.653439 secs 2023-09-10T07:30:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29456.129905 secs 2023-09-10T07:30:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:26 smithi170 ceph-mon[102832]: pgmap v15870: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:30:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:26 smithi082 ceph-mon[98694]: pgmap v15870: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:30:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:26 smithi098 ceph-mon[103124]: pgmap v15870: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:30:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:27 smithi170 ceph-mon[102832]: pgmap v15871: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 511 B/s wr, 13 op/s 2023-09-10T07:30:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:27 smithi082 ceph-mon[98694]: pgmap v15871: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 511 B/s wr, 13 op/s 2023-09-10T07:30:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:27 smithi098 ceph-mon[103124]: pgmap v15871: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 511 B/s wr, 13 op/s 2023-09-10T07:30:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29459.517131 secs 2023-09-10T07:30:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29459.517131 secs 2023-09-10T07:30:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29459.517131 secs 2023-09-10T07:30:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29460.653498 secs 2023-09-10T07:30:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29461.129989 secs 2023-09-10T07:30:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:29 smithi170 ceph-mon[102832]: pgmap v15872: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:30:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29460.653498 secs 2023-09-10T07:30:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29461.129989 secs 2023-09-10T07:30:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:29 smithi082 ceph-mon[98694]: pgmap v15872: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:30:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29460.653498 secs 2023-09-10T07:30:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29461.129989 secs 2023-09-10T07:30:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:29 smithi098 ceph-mon[103124]: pgmap v15872: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:30:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:32 smithi170 ceph-mon[102832]: pgmap v15873: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 112 KiB/s wr, 13 op/s 2023-09-10T07:30:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:32 smithi082 ceph-mon[98694]: pgmap v15873: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 112 KiB/s wr, 13 op/s 2023-09-10T07:30:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:32 smithi098 ceph-mon[103124]: pgmap v15873: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 112 KiB/s wr, 13 op/s 2023-09-10T07:30:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:34 smithi170 ceph-mon[102832]: pgmap v15874: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 112 KiB/s wr, 9 op/s 2023-09-10T07:30:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:34 smithi082 ceph-mon[98694]: pgmap v15874: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 112 KiB/s wr, 9 op/s 2023-09-10T07:30:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:34 smithi098 ceph-mon[103124]: pgmap v15874: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 112 KiB/s wr, 9 op/s 2023-09-10T07:30:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29465.653558 secs 2023-09-10T07:30:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29466.130054 secs 2023-09-10T07:30:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:30:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29465.653558 secs 2023-09-10T07:30:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29466.130054 secs 2023-09-10T07:30:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:30:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29465.653558 secs 2023-09-10T07:30:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29466.130054 secs 2023-09-10T07:30:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:30:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:36 smithi170 ceph-mon[102832]: pgmap v15875: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 112 KiB/s wr, 9 op/s 2023-09-10T07:30:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:30:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:30:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:30:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:36 smithi082 ceph-mon[98694]: pgmap v15875: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 112 KiB/s wr, 9 op/s 2023-09-10T07:30:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:30:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:30:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:30:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:36 smithi098 ceph-mon[103124]: pgmap v15875: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 112 KiB/s wr, 9 op/s 2023-09-10T07:30:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:30:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:30:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:30:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:37 smithi170 ceph-mon[102832]: pgmap v15876: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 202 KiB/s wr, 15 op/s 2023-09-10T07:30:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:37 smithi082 ceph-mon[98694]: pgmap v15876: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 202 KiB/s wr, 15 op/s 2023-09-10T07:30:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:37 smithi098 ceph-mon[103124]: pgmap v15876: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 202 KiB/s wr, 15 op/s 2023-09-10T07:30:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29470.653617 secs 2023-09-10T07:30:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29471.130116 secs 2023-09-10T07:30:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:40 smithi170 ceph-mon[102832]: pgmap v15877: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 202 KiB/s wr, 10 op/s 2023-09-10T07:30:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29470.653617 secs 2023-09-10T07:30:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29471.130116 secs 2023-09-10T07:30:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:40 smithi082 ceph-mon[98694]: pgmap v15877: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 202 KiB/s wr, 10 op/s 2023-09-10T07:30:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29470.653617 secs 2023-09-10T07:30:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29471.130116 secs 2023-09-10T07:30:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:40 smithi098 ceph-mon[103124]: pgmap v15877: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 202 KiB/s wr, 10 op/s 2023-09-10T07:30:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:42 smithi170 ceph-mon[102832]: pgmap v15878: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 422 KiB/s wr, 16 op/s 2023-09-10T07:30:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:42 smithi082 ceph-mon[98694]: pgmap v15878: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 422 KiB/s wr, 16 op/s 2023-09-10T07:30:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:42 smithi098 ceph-mon[103124]: pgmap v15878: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 422 KiB/s wr, 16 op/s 2023-09-10T07:30:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:44 smithi170 ceph-mon[102832]: pgmap v15879: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 310 KiB/s wr, 11 op/s 2023-09-10T07:30:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:44 smithi082 ceph-mon[98694]: pgmap v15879: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 310 KiB/s wr, 11 op/s 2023-09-10T07:30:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:44 smithi098 ceph-mon[103124]: pgmap v15879: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 310 KiB/s wr, 11 op/s 2023-09-10T07:30:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29475.653677 secs 2023-09-10T07:30:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29476.130174 secs 2023-09-10T07:30:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29475.653677 secs 2023-09-10T07:30:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29476.130174 secs 2023-09-10T07:30:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29475.653677 secs 2023-09-10T07:30:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29476.130174 secs 2023-09-10T07:30:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:46 smithi170 ceph-mon[102832]: pgmap v15880: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 310 KiB/s wr, 11 op/s 2023-09-10T07:30:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:46 smithi082 ceph-mon[98694]: pgmap v15880: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 310 KiB/s wr, 11 op/s 2023-09-10T07:30:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:46 smithi098 ceph-mon[103124]: pgmap v15880: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 310 KiB/s wr, 11 op/s 2023-09-10T07:30:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:47 smithi170 ceph-mon[102832]: pgmap v15881: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 456 KiB/s wr, 17 op/s 2023-09-10T07:30:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:47 smithi082 ceph-mon[98694]: pgmap v15881: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 456 KiB/s wr, 17 op/s 2023-09-10T07:30:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:47 smithi098 ceph-mon[103124]: pgmap v15881: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 456 KiB/s wr, 17 op/s 2023-09-10T07:30:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29480.653734 secs 2023-09-10T07:30:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29481.130237 secs 2023-09-10T07:30:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:50 smithi170 ceph-mon[102832]: pgmap v15882: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 365 KiB/s wr, 12 op/s 2023-09-10T07:30:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29480.653734 secs 2023-09-10T07:30:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29481.130237 secs 2023-09-10T07:30:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:50 smithi082 ceph-mon[98694]: pgmap v15882: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 365 KiB/s wr, 12 op/s 2023-09-10T07:30:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29480.653734 secs 2023-09-10T07:30:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29481.130237 secs 2023-09-10T07:30:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:50 smithi098 ceph-mon[103124]: pgmap v15882: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 365 KiB/s wr, 12 op/s 2023-09-10T07:30:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:52 smithi170 ceph-mon[102832]: pgmap v15883: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 629 KiB/s wr, 19 op/s 2023-09-10T07:30:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:52 smithi082 ceph-mon[98694]: pgmap v15883: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 629 KiB/s wr, 19 op/s 2023-09-10T07:30:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:52 smithi098 ceph-mon[103124]: pgmap v15883: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 629 KiB/s wr, 19 op/s 2023-09-10T07:30:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:54 smithi170 ceph-mon[102832]: pgmap v15884: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 409 KiB/s wr, 13 op/s 2023-09-10T07:30:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:54 smithi082 ceph-mon[98694]: pgmap v15884: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 409 KiB/s wr, 13 op/s 2023-09-10T07:30:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:54 smithi098 ceph-mon[103124]: pgmap v15884: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 409 KiB/s wr, 13 op/s 2023-09-10T07:30:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29485.653791 secs 2023-09-10T07:30:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29486.130297 secs 2023-09-10T07:30:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29485.653791 secs 2023-09-10T07:30:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29486.130297 secs 2023-09-10T07:30:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29485.653791 secs 2023-09-10T07:30:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29486.130297 secs 2023-09-10T07:30:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:56 smithi170 ceph-mon[102832]: pgmap v15885: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 409 KiB/s wr, 13 op/s 2023-09-10T07:30:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:56 smithi082 ceph-mon[98694]: pgmap v15885: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 409 KiB/s wr, 13 op/s 2023-09-10T07:30:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:56 smithi098 ceph-mon[103124]: pgmap v15885: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 409 KiB/s wr, 13 op/s 2023-09-10T07:30:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:57 smithi170 ceph-mon[102832]: pgmap v15886: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 486 KiB/s wr, 20 op/s 2023-09-10T07:30:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:57 smithi082 ceph-mon[98694]: pgmap v15886: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 486 KiB/s wr, 20 op/s 2023-09-10T07:30:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:57 smithi098 ceph-mon[103124]: pgmap v15886: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 486 KiB/s wr, 20 op/s 2023-09-10T07:30:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29489.517540 secs 2023-09-10T07:30:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29489.517540 secs 2023-09-10T07:30:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29489.517540 secs 2023-09-10T07:31:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29490.653850 secs 2023-09-10T07:31:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29491.130364 secs 2023-09-10T07:31:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:30:59 smithi170 ceph-mon[102832]: pgmap v15887: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 340 KiB/s wr, 13 op/s 2023-09-10T07:31:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29490.653850 secs 2023-09-10T07:31:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29491.130364 secs 2023-09-10T07:31:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:30:59 smithi082 ceph-mon[98694]: pgmap v15887: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 340 KiB/s wr, 13 op/s 2023-09-10T07:31:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29490.653850 secs 2023-09-10T07:31:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29491.130364 secs 2023-09-10T07:31:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:30:59 smithi098 ceph-mon[103124]: pgmap v15887: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 340 KiB/s wr, 13 op/s 2023-09-10T07:31:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:31:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:31:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:31:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:31:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:31:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:31:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:31:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:31:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:31:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:01 smithi170 ceph-mon[102832]: pgmap v15888: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 623 KiB/s wr, 34 op/s 2023-09-10T07:31:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:01 smithi082 ceph-mon[98694]: pgmap v15888: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 623 KiB/s wr, 34 op/s 2023-09-10T07:31:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:01 smithi098 ceph-mon[103124]: pgmap v15888: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 623 KiB/s wr, 34 op/s 2023-09-10T07:31:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29494.517647 secs 2023-09-10T07:31:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:04 smithi170 ceph-mon[102832]: pgmap v15889: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 359 KiB/s wr, 27 op/s 2023-09-10T07:31:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29494.517647 secs 2023-09-10T07:31:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:04 smithi082 ceph-mon[98694]: pgmap v15889: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 359 KiB/s wr, 27 op/s 2023-09-10T07:31:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29494.517647 secs 2023-09-10T07:31:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:04 smithi098 ceph-mon[103124]: pgmap v15889: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 359 KiB/s wr, 27 op/s 2023-09-10T07:31:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29495.653908 secs 2023-09-10T07:31:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29496.130428 secs 2023-09-10T07:31:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29495.653908 secs 2023-09-10T07:31:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29496.130428 secs 2023-09-10T07:31:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29495.653908 secs 2023-09-10T07:31:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29496.130428 secs 2023-09-10T07:31:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:06 smithi170 ceph-mon[102832]: pgmap v15890: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 359 KiB/s wr, 27 op/s 2023-09-10T07:31:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:06 smithi082 ceph-mon[98694]: pgmap v15890: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 359 KiB/s wr, 27 op/s 2023-09-10T07:31:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:06 smithi098 ceph-mon[103124]: pgmap v15890: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 359 KiB/s wr, 27 op/s 2023-09-10T07:31:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:07 smithi170 ceph-mon[102832]: pgmap v15891: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 441 KiB/s wr, 32 op/s 2023-09-10T07:31:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:07 smithi082 ceph-mon[98694]: pgmap v15891: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 441 KiB/s wr, 32 op/s 2023-09-10T07:31:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:07 smithi098 ceph-mon[103124]: pgmap v15891: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 441 KiB/s wr, 32 op/s 2023-09-10T07:31:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29499.517706 secs 2023-09-10T07:31:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:08 smithi170 ceph-mon[102832]: osdmap e1061: 12 total, 12 up, 12 in 2023-09-10T07:31:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29499.517706 secs 2023-09-10T07:31:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:08 smithi082 ceph-mon[98694]: osdmap e1061: 12 total, 12 up, 12 in 2023-09-10T07:31:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29499.517706 secs 2023-09-10T07:31:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:08 smithi098 ceph-mon[103124]: osdmap e1061: 12 total, 12 up, 12 in 2023-09-10T07:31:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29500.653971 secs 2023-09-10T07:31:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29501.130482 secs 2023-09-10T07:31:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:09 smithi170 ceph-mon[102832]: pgmap v15893: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 436 KiB/s wr, 30 op/s 2023-09-10T07:31:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29500.653971 secs 2023-09-10T07:31:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29501.130482 secs 2023-09-10T07:31:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:09 smithi082 ceph-mon[98694]: pgmap v15893: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 436 KiB/s wr, 30 op/s 2023-09-10T07:31:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29500.653971 secs 2023-09-10T07:31:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29501.130482 secs 2023-09-10T07:31:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:09 smithi098 ceph-mon[103124]: pgmap v15893: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 436 KiB/s wr, 30 op/s 2023-09-10T07:31:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:12 smithi170 ceph-mon[102832]: pgmap v15894: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 98 KiB/s wr, 11 op/s 2023-09-10T07:31:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:12 smithi170 ceph-mon[102832]: osdmap e1062: 12 total, 12 up, 12 in 2023-09-10T07:31:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:12 smithi082 ceph-mon[98694]: pgmap v15894: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 98 KiB/s wr, 11 op/s 2023-09-10T07:31:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:12 smithi082 ceph-mon[98694]: osdmap e1062: 12 total, 12 up, 12 in 2023-09-10T07:31:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:12 smithi098 ceph-mon[103124]: pgmap v15894: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 98 KiB/s wr, 11 op/s 2023-09-10T07:31:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:12 smithi098 ceph-mon[103124]: osdmap e1062: 12 total, 12 up, 12 in 2023-09-10T07:31:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:31:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:31:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:31:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:31:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:31:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:31:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:31:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:31:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:31:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:31:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:31:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:31:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29504.517785 secs 2023-09-10T07:31:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:14 smithi170 ceph-mon[102832]: pgmap v15896: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 122 KiB/s wr, 14 op/s 2023-09-10T07:31:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29504.517785 secs 2023-09-10T07:31:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:14 smithi082 ceph-mon[98694]: pgmap v15896: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 122 KiB/s wr, 14 op/s 2023-09-10T07:31:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29504.517785 secs 2023-09-10T07:31:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:14 smithi098 ceph-mon[103124]: pgmap v15896: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 122 KiB/s wr, 14 op/s 2023-09-10T07:31:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29505.654030 secs 2023-09-10T07:31:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29506.130541 secs 2023-09-10T07:31:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29505.654030 secs 2023-09-10T07:31:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29506.130541 secs 2023-09-10T07:31:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29505.654030 secs 2023-09-10T07:31:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29506.130541 secs 2023-09-10T07:31:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:16 smithi170 ceph-mon[102832]: pgmap v15897: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:31:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:16 smithi082 ceph-mon[98694]: pgmap v15897: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:31:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:16 smithi098 ceph-mon[103124]: pgmap v15897: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:31:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:17 smithi170 ceph-mon[102832]: pgmap v15898: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:31:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:17 smithi082 ceph-mon[98694]: pgmap v15898: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:31:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:17 smithi098 ceph-mon[103124]: pgmap v15898: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:31:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29509.517847 secs 2023-09-10T07:31:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29509.517847 secs 2023-09-10T07:31:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29509.517847 secs 2023-09-10T07:31:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29510.654089 secs 2023-09-10T07:31:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29511.130607 secs 2023-09-10T07:31:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:19 smithi170 ceph-mon[102832]: pgmap v15899: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:31:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29510.654089 secs 2023-09-10T07:31:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29511.130607 secs 2023-09-10T07:31:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:19 smithi082 ceph-mon[98694]: pgmap v15899: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:31:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29510.654089 secs 2023-09-10T07:31:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29511.130607 secs 2023-09-10T07:31:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:19 smithi098 ceph-mon[103124]: pgmap v15899: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:31:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:22 smithi170 ceph-mon[102832]: pgmap v15900: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-10T07:31:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:22 smithi082 ceph-mon[98694]: pgmap v15900: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-10T07:31:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:22 smithi098 ceph-mon[103124]: pgmap v15900: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-10T07:31:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29514.517928 secs 2023-09-10T07:31:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:24 smithi170 ceph-mon[102832]: pgmap v15901: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 9 op/s 2023-09-10T07:31:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29514.517928 secs 2023-09-10T07:31:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:24 smithi082 ceph-mon[98694]: pgmap v15901: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 9 op/s 2023-09-10T07:31:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29514.517928 secs 2023-09-10T07:31:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:24 smithi098 ceph-mon[103124]: pgmap v15901: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 9 op/s 2023-09-10T07:31:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29515.654148 secs 2023-09-10T07:31:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29516.130663 secs 2023-09-10T07:31:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29515.654148 secs 2023-09-10T07:31:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29516.130663 secs 2023-09-10T07:31:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29515.654148 secs 2023-09-10T07:31:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29516.130663 secs 2023-09-10T07:31:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:26 smithi170 ceph-mon[102832]: pgmap v15902: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 8 op/s 2023-09-10T07:31:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:26 smithi082 ceph-mon[98694]: pgmap v15902: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 8 op/s 2023-09-10T07:31:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:26 smithi098 ceph-mon[103124]: pgmap v15902: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 8 op/s 2023-09-10T07:31:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:27 smithi170 ceph-mon[102832]: pgmap v15903: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 98 KiB/s wr, 14 op/s 2023-09-10T07:31:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:27 smithi082 ceph-mon[98694]: pgmap v15903: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 98 KiB/s wr, 14 op/s 2023-09-10T07:31:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:27 smithi098 ceph-mon[103124]: pgmap v15903: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 98 KiB/s wr, 14 op/s 2023-09-10T07:31:29.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29519.517987 secs 2023-09-10T07:31:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29519.517987 secs 2023-09-10T07:31:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29519.517987 secs 2023-09-10T07:31:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29520.654211 secs 2023-09-10T07:31:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29521.130727 secs 2023-09-10T07:31:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:29 smithi170 ceph-mon[102832]: pgmap v15904: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 98 KiB/s wr, 10 op/s 2023-09-10T07:31:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29520.654211 secs 2023-09-10T07:31:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29521.130727 secs 2023-09-10T07:31:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:29 smithi082 ceph-mon[98694]: pgmap v15904: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 98 KiB/s wr, 10 op/s 2023-09-10T07:31:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29520.654211 secs 2023-09-10T07:31:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29521.130727 secs 2023-09-10T07:31:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:29 smithi098 ceph-mon[103124]: pgmap v15904: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 98 KiB/s wr, 10 op/s 2023-09-10T07:31:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:32 smithi170 ceph-mon[102832]: pgmap v15905: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 201 KiB/s wr, 15 op/s 2023-09-10T07:31:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:32 smithi082 ceph-mon[98694]: pgmap v15905: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 201 KiB/s wr, 15 op/s 2023-09-10T07:31:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:32 smithi098 ceph-mon[103124]: pgmap v15905: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 201 KiB/s wr, 15 op/s 2023-09-10T07:31:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29524.518049 secs 2023-09-10T07:31:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:34 smithi170 ceph-mon[102832]: pgmap v15906: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T07:31:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29524.518049 secs 2023-09-10T07:31:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:34 smithi082 ceph-mon[98694]: pgmap v15906: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T07:31:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29524.518049 secs 2023-09-10T07:31:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:34 smithi098 ceph-mon[103124]: pgmap v15906: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T07:31:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29525.654289 secs 2023-09-10T07:31:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29526.130784 secs 2023-09-10T07:31:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29525.654289 secs 2023-09-10T07:31:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29526.130784 secs 2023-09-10T07:31:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29525.654289 secs 2023-09-10T07:31:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29526.130784 secs 2023-09-10T07:31:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:36 smithi170 ceph-mon[102832]: pgmap v15907: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T07:31:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:36 smithi082 ceph-mon[98694]: pgmap v15907: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T07:31:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:36 smithi098 ceph-mon[103124]: pgmap v15907: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T07:31:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:37 smithi170 ceph-mon[102832]: pgmap v15908: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 343 KiB/s wr, 16 op/s 2023-09-10T07:31:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:37 smithi082 ceph-mon[98694]: pgmap v15908: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 343 KiB/s wr, 16 op/s 2023-09-10T07:31:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:37 smithi098 ceph-mon[103124]: pgmap v15908: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 343 KiB/s wr, 16 op/s 2023-09-10T07:31:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29529.518110 secs 2023-09-10T07:31:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29529.518110 secs 2023-09-10T07:31:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29529.518110 secs 2023-09-10T07:31:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29530.654347 secs 2023-09-10T07:31:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29531.130852 secs 2023-09-10T07:31:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:39 smithi170 ceph-mon[102832]: pgmap v15909: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 267 KiB/s wr, 10 op/s 2023-09-10T07:31:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29530.654347 secs 2023-09-10T07:31:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29531.130852 secs 2023-09-10T07:31:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:39 smithi082 ceph-mon[98694]: pgmap v15909: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 267 KiB/s wr, 10 op/s 2023-09-10T07:31:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29530.654347 secs 2023-09-10T07:31:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29531.130852 secs 2023-09-10T07:31:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:39 smithi098 ceph-mon[103124]: pgmap v15909: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 267 KiB/s wr, 10 op/s 2023-09-10T07:31:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:42 smithi170 ceph-mon[102832]: pgmap v15910: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 521 KiB/s wr, 16 op/s 2023-09-10T07:31:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:42 smithi082 ceph-mon[98694]: pgmap v15910: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 521 KiB/s wr, 16 op/s 2023-09-10T07:31:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:42 smithi098 ceph-mon[103124]: pgmap v15910: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 521 KiB/s wr, 16 op/s 2023-09-10T07:31:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:31:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:31:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:31:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:44 smithi170 ceph-mon[102832]: pgmap v15911: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 419 KiB/s wr, 11 op/s 2023-09-10T07:31:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:44 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:31:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:31:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:31:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:44 smithi082 ceph-mon[98694]: pgmap v15911: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 419 KiB/s wr, 11 op/s 2023-09-10T07:31:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:44 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:31:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:31:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:31:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:44 smithi098 ceph-mon[103124]: pgmap v15911: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 419 KiB/s wr, 11 op/s 2023-09-10T07:31:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:44 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:31:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:31:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:31:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29535.654407 secs 2023-09-10T07:31:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29536.130918 secs 2023-09-10T07:31:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29535.654407 secs 2023-09-10T07:31:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29536.130918 secs 2023-09-10T07:31:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29535.654407 secs 2023-09-10T07:31:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29536.130918 secs 2023-09-10T07:31:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:46 smithi170 ceph-mon[102832]: pgmap v15912: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 419 KiB/s wr, 11 op/s 2023-09-10T07:31:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:46 smithi082 ceph-mon[98694]: pgmap v15912: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 419 KiB/s wr, 11 op/s 2023-09-10T07:31:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:46 smithi098 ceph-mon[103124]: pgmap v15912: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 419 KiB/s wr, 11 op/s 2023-09-10T07:31:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:47 smithi170 ceph-mon[102832]: pgmap v15913: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 565 KiB/s wr, 18 op/s 2023-09-10T07:31:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:47 smithi082 ceph-mon[98694]: pgmap v15913: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 565 KiB/s wr, 18 op/s 2023-09-10T07:31:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:47 smithi098 ceph-mon[103124]: pgmap v15913: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 565 KiB/s wr, 18 op/s 2023-09-10T07:31:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29540.654465 secs 2023-09-10T07:31:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29541.130983 secs 2023-09-10T07:31:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:50 smithi170 ceph-mon[102832]: pgmap v15914: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 401 KiB/s wr, 13 op/s 2023-09-10T07:31:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29540.654465 secs 2023-09-10T07:31:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29541.130983 secs 2023-09-10T07:31:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:50 smithi082 ceph-mon[98694]: pgmap v15914: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 401 KiB/s wr, 13 op/s 2023-09-10T07:31:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29540.654465 secs 2023-09-10T07:31:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29541.130983 secs 2023-09-10T07:31:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:50 smithi098 ceph-mon[103124]: pgmap v15914: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 401 KiB/s wr, 13 op/s 2023-09-10T07:31:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:52 smithi170 ceph-mon[102832]: pgmap v15915: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 665 KiB/s wr, 20 op/s 2023-09-10T07:31:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:52 smithi082 ceph-mon[98694]: pgmap v15915: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 665 KiB/s wr, 20 op/s 2023-09-10T07:31:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:52 smithi098 ceph-mon[103124]: pgmap v15915: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 665 KiB/s wr, 20 op/s 2023-09-10T07:31:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:54 smithi170 ceph-mon[102832]: pgmap v15916: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 411 KiB/s wr, 14 op/s 2023-09-10T07:31:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:54 smithi082 ceph-mon[98694]: pgmap v15916: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 411 KiB/s wr, 14 op/s 2023-09-10T07:31:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:54 smithi098 ceph-mon[103124]: pgmap v15916: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 411 KiB/s wr, 14 op/s 2023-09-10T07:31:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29545.654494 secs 2023-09-10T07:31:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29546.131041 secs 2023-09-10T07:31:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29545.654494 secs 2023-09-10T07:31:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29546.131041 secs 2023-09-10T07:31:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29545.654494 secs 2023-09-10T07:31:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29546.131041 secs 2023-09-10T07:31:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:56 smithi170 ceph-mon[102832]: pgmap v15917: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 411 KiB/s wr, 14 op/s 2023-09-10T07:31:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:56 smithi082 ceph-mon[98694]: pgmap v15917: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 411 KiB/s wr, 14 op/s 2023-09-10T07:31:56.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:56 smithi098 ceph-mon[103124]: pgmap v15917: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 411 KiB/s wr, 14 op/s 2023-09-10T07:31:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:57 smithi170 ceph-mon[102832]: pgmap v15918: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 488 KiB/s wr, 21 op/s 2023-09-10T07:31:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:57 smithi082 ceph-mon[98694]: pgmap v15918: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 488 KiB/s wr, 21 op/s 2023-09-10T07:31:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:57 smithi098 ceph-mon[103124]: pgmap v15918: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 488 KiB/s wr, 21 op/s 2023-09-10T07:31:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29549.518370 secs 2023-09-10T07:31:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29549.518370 secs 2023-09-10T07:31:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29549.518370 secs 2023-09-10T07:32:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29550.654551 secs 2023-09-10T07:32:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29551.131103 secs 2023-09-10T07:32:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:31:59 smithi082 ceph-mon[98694]: pgmap v15919: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 342 KiB/s wr, 13 op/s 2023-09-10T07:32:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29550.654551 secs 2023-09-10T07:32:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29551.131103 secs 2023-09-10T07:32:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:31:59 smithi098 ceph-mon[103124]: pgmap v15919: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 342 KiB/s wr, 13 op/s 2023-09-10T07:32:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29550.654551 secs 2023-09-10T07:32:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29551.131103 secs 2023-09-10T07:32:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:31:59 smithi170 ceph-mon[102832]: pgmap v15919: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 342 KiB/s wr, 13 op/s 2023-09-10T07:32:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:32:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:32:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:32:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:32:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:32:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:32:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:32:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:32:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:32:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:01 smithi082 ceph-mon[98694]: pgmap v15920: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 597 KiB/s wr, 31 op/s 2023-09-10T07:32:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:01 smithi098 ceph-mon[103124]: pgmap v15920: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 597 KiB/s wr, 31 op/s 2023-09-10T07:32:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:01 smithi170 ceph-mon[102832]: pgmap v15920: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 597 KiB/s wr, 31 op/s 2023-09-10T07:32:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29554.518459 secs 2023-09-10T07:32:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:04 smithi170 ceph-mon[102832]: pgmap v15921: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 332 KiB/s wr, 23 op/s 2023-09-10T07:32:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29554.518459 secs 2023-09-10T07:32:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:04 smithi082 ceph-mon[98694]: pgmap v15921: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 332 KiB/s wr, 23 op/s 2023-09-10T07:32:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29554.518459 secs 2023-09-10T07:32:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:04 smithi098 ceph-mon[103124]: pgmap v15921: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 332 KiB/s wr, 23 op/s 2023-09-10T07:32:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29555.654610 secs 2023-09-10T07:32:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29556.131160 secs 2023-09-10T07:32:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29555.654610 secs 2023-09-10T07:32:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29556.131160 secs 2023-09-10T07:32:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29555.654610 secs 2023-09-10T07:32:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29556.131160 secs 2023-09-10T07:32:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:06 smithi170 ceph-mon[102832]: pgmap v15922: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 332 KiB/s wr, 23 op/s 2023-09-10T07:32:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:06 smithi082 ceph-mon[98694]: pgmap v15922: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 332 KiB/s wr, 23 op/s 2023-09-10T07:32:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:06 smithi098 ceph-mon[103124]: pgmap v15922: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 332 KiB/s wr, 23 op/s 2023-09-10T07:32:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:07 smithi170 ceph-mon[102832]: pgmap v15923: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 442 KiB/s wr, 32 op/s 2023-09-10T07:32:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:07 smithi082 ceph-mon[98694]: pgmap v15923: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 442 KiB/s wr, 32 op/s 2023-09-10T07:32:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:07 smithi098 ceph-mon[103124]: pgmap v15923: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 442 KiB/s wr, 32 op/s 2023-09-10T07:32:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29559.518518 secs 2023-09-10T07:32:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:08 smithi170 ceph-mon[102832]: osdmap e1063: 12 total, 12 up, 12 in 2023-09-10T07:32:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29559.518518 secs 2023-09-10T07:32:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:08 smithi082 ceph-mon[98694]: osdmap e1063: 12 total, 12 up, 12 in 2023-09-10T07:32:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29559.518518 secs 2023-09-10T07:32:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:08 smithi098 ceph-mon[103124]: osdmap e1063: 12 total, 12 up, 12 in 2023-09-10T07:32:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29560.654668 secs 2023-09-10T07:32:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29561.131225 secs 2023-09-10T07:32:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:09 smithi082 ceph-mon[98694]: pgmap v15925: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 437 KiB/s wr, 31 op/s 2023-09-10T07:32:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29560.654668 secs 2023-09-10T07:32:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29561.131225 secs 2023-09-10T07:32:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:09 smithi098 ceph-mon[103124]: pgmap v15925: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 437 KiB/s wr, 31 op/s 2023-09-10T07:32:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29560.654668 secs 2023-09-10T07:32:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29561.131225 secs 2023-09-10T07:32:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:09 smithi170 ceph-mon[102832]: pgmap v15925: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 437 KiB/s wr, 31 op/s 2023-09-10T07:32:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:12 smithi170 ceph-mon[102832]: pgmap v15926: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 131 KiB/s wr, 15 op/s 2023-09-10T07:32:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:12 smithi170 ceph-mon[102832]: osdmap e1064: 12 total, 12 up, 12 in 2023-09-10T07:32:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:12 smithi082 ceph-mon[98694]: pgmap v15926: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 131 KiB/s wr, 15 op/s 2023-09-10T07:32:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:12 smithi082 ceph-mon[98694]: osdmap e1064: 12 total, 12 up, 12 in 2023-09-10T07:32:12.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:12 smithi098 ceph-mon[103124]: pgmap v15926: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 131 KiB/s wr, 15 op/s 2023-09-10T07:32:12.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:12 smithi098 ceph-mon[103124]: osdmap e1064: 12 total, 12 up, 12 in 2023-09-10T07:32:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:32:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:32:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:32:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29564.518609 secs 2023-09-10T07:32:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:14 smithi170 ceph-mon[102832]: pgmap v15928: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 164 KiB/s wr, 19 op/s 2023-09-10T07:32:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:32:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:32:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:32:14.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29564.518609 secs 2023-09-10T07:32:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:14 smithi082 ceph-mon[98694]: pgmap v15928: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 164 KiB/s wr, 19 op/s 2023-09-10T07:32:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:32:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:32:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:32:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29564.518609 secs 2023-09-10T07:32:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:14 smithi098 ceph-mon[103124]: pgmap v15928: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 164 KiB/s wr, 19 op/s 2023-09-10T07:32:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:32:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:32:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:32:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29565.654732 secs 2023-09-10T07:32:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29566.131286 secs 2023-09-10T07:32:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29565.654732 secs 2023-09-10T07:32:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29566.131286 secs 2023-09-10T07:32:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29565.654732 secs 2023-09-10T07:32:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29566.131286 secs 2023-09-10T07:32:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:16 smithi170 ceph-mon[102832]: pgmap v15929: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-10T07:32:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:16 smithi082 ceph-mon[98694]: pgmap v15929: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-10T07:32:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:16 smithi098 ceph-mon[103124]: pgmap v15929: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-10T07:32:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:17 smithi170 ceph-mon[102832]: pgmap v15930: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 335 B/s wr, 11 op/s 2023-09-10T07:32:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:17 smithi082 ceph-mon[98694]: pgmap v15930: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 335 B/s wr, 11 op/s 2023-09-10T07:32:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:17 smithi098 ceph-mon[103124]: pgmap v15930: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 335 B/s wr, 11 op/s 2023-09-10T07:32:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29569.518690 secs 2023-09-10T07:32:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29569.518690 secs 2023-09-10T07:32:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29569.518690 secs 2023-09-10T07:32:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29570.654791 secs 2023-09-10T07:32:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29571.131350 secs 2023-09-10T07:32:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:19 smithi170 ceph-mon[102832]: pgmap v15931: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T07:32:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29570.654791 secs 2023-09-10T07:32:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29571.131350 secs 2023-09-10T07:32:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:19 smithi082 ceph-mon[98694]: pgmap v15931: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T07:32:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29570.654791 secs 2023-09-10T07:32:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29571.131350 secs 2023-09-10T07:32:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:19 smithi098 ceph-mon[103124]: pgmap v15931: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T07:32:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:22 smithi170 ceph-mon[102832]: pgmap v15932: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:32:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:22 smithi082 ceph-mon[98694]: pgmap v15932: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:32:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:22 smithi098 ceph-mon[103124]: pgmap v15932: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:32:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29574.518775 secs 2023-09-10T07:32:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:24 smithi170 ceph-mon[102832]: pgmap v15933: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:32:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29574.518775 secs 2023-09-10T07:32:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:24 smithi082 ceph-mon[98694]: pgmap v15933: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:32:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29574.518775 secs 2023-09-10T07:32:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:24 smithi098 ceph-mon[103124]: pgmap v15933: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:32:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29575.654852 secs 2023-09-10T07:32:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29576.131407 secs 2023-09-10T07:32:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29575.654852 secs 2023-09-10T07:32:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29576.131407 secs 2023-09-10T07:32:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29575.654852 secs 2023-09-10T07:32:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29576.131407 secs 2023-09-10T07:32:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:26 smithi170 ceph-mon[102832]: pgmap v15934: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:32:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:26 smithi082 ceph-mon[98694]: pgmap v15934: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:32:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:26 smithi098 ceph-mon[103124]: pgmap v15934: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:32:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:27 smithi170 ceph-mon[102832]: pgmap v15935: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-10T07:32:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:27 smithi082 ceph-mon[98694]: pgmap v15935: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-10T07:32:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:27 smithi098 ceph-mon[103124]: pgmap v15935: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-10T07:32:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29579.518834 secs 2023-09-10T07:32:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29579.518834 secs 2023-09-10T07:32:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29579.518834 secs 2023-09-10T07:32:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29580.654899 secs 2023-09-10T07:32:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29581.131473 secs 2023-09-10T07:32:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:29 smithi082 ceph-mon[98694]: pgmap v15936: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-10T07:32:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29580.654899 secs 2023-09-10T07:32:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29581.131473 secs 2023-09-10T07:32:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:29 smithi098 ceph-mon[103124]: pgmap v15936: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-10T07:32:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29580.654899 secs 2023-09-10T07:32:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29581.131473 secs 2023-09-10T07:32:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:29 smithi170 ceph-mon[102832]: pgmap v15936: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-10T07:32:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:32 smithi170 ceph-mon[102832]: pgmap v15937: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 76 KiB/s wr, 13 op/s 2023-09-10T07:32:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:32 smithi082 ceph-mon[98694]: pgmap v15937: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 76 KiB/s wr, 13 op/s 2023-09-10T07:32:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:32 smithi098 ceph-mon[103124]: pgmap v15937: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 76 KiB/s wr, 13 op/s 2023-09-10T07:32:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29584.518897 secs 2023-09-10T07:32:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:34 smithi170 ceph-mon[102832]: pgmap v15938: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T07:32:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29584.518897 secs 2023-09-10T07:32:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:34 smithi082 ceph-mon[98694]: pgmap v15938: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T07:32:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29584.518897 secs 2023-09-10T07:32:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:34 smithi098 ceph-mon[103124]: pgmap v15938: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T07:32:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29585.654959 secs 2023-09-10T07:32:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29586.131484 secs 2023-09-10T07:32:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29585.654959 secs 2023-09-10T07:32:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29586.131484 secs 2023-09-10T07:32:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29585.654959 secs 2023-09-10T07:32:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29586.131484 secs 2023-09-10T07:32:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:36 smithi170 ceph-mon[102832]: pgmap v15939: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T07:32:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:36 smithi082 ceph-mon[98694]: pgmap v15939: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T07:32:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:36 smithi098 ceph-mon[103124]: pgmap v15939: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 76 KiB/s wr, 9 op/s 2023-09-10T07:32:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:37 smithi082 ceph-mon[98694]: pgmap v15940: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 228 KiB/s wr, 15 op/s 2023-09-10T07:32:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:37 smithi098 ceph-mon[103124]: pgmap v15940: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 228 KiB/s wr, 15 op/s 2023-09-10T07:32:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:37 smithi170 ceph-mon[102832]: pgmap v15940: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 228 KiB/s wr, 15 op/s 2023-09-10T07:32:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:32:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:32:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:32:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29590.655014 secs 2023-09-10T07:32:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29591.131545 secs 2023-09-10T07:32:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:32:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:32:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:32:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:40 smithi170 ceph-mon[102832]: pgmap v15941: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 228 KiB/s wr, 11 op/s 2023-09-10T07:32:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29590.655014 secs 2023-09-10T07:32:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29591.131545 secs 2023-09-10T07:32:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:32:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:32:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:32:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:40 smithi082 ceph-mon[98694]: pgmap v15941: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 228 KiB/s wr, 11 op/s 2023-09-10T07:32:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29590.655014 secs 2023-09-10T07:32:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29591.131545 secs 2023-09-10T07:32:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:32:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:32:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:32:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:40 smithi098 ceph-mon[103124]: pgmap v15941: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 228 KiB/s wr, 11 op/s 2023-09-10T07:32:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:42 smithi170 ceph-mon[102832]: pgmap v15942: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 462 KiB/s wr, 18 op/s 2023-09-10T07:32:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:42 smithi082 ceph-mon[98694]: pgmap v15942: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 462 KiB/s wr, 18 op/s 2023-09-10T07:32:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:42 smithi098 ceph-mon[103124]: pgmap v15942: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 462 KiB/s wr, 18 op/s 2023-09-10T07:32:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:44 smithi170 ceph-mon[102832]: pgmap v15943: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 386 KiB/s wr, 12 op/s 2023-09-10T07:32:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:44 smithi082 ceph-mon[98694]: pgmap v15943: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 386 KiB/s wr, 12 op/s 2023-09-10T07:32:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:44 smithi098 ceph-mon[103124]: pgmap v15943: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 386 KiB/s wr, 12 op/s 2023-09-10T07:32:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29595.655071 secs 2023-09-10T07:32:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29596.131602 secs 2023-09-10T07:32:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29595.655071 secs 2023-09-10T07:32:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29596.131602 secs 2023-09-10T07:32:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29595.655071 secs 2023-09-10T07:32:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29596.131602 secs 2023-09-10T07:32:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:46 smithi170 ceph-mon[102832]: pgmap v15944: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 386 KiB/s wr, 12 op/s 2023-09-10T07:32:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:46 smithi082 ceph-mon[98694]: pgmap v15944: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 386 KiB/s wr, 12 op/s 2023-09-10T07:32:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:46 smithi098 ceph-mon[103124]: pgmap v15944: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 386 KiB/s wr, 12 op/s 2023-09-10T07:32:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:47 smithi082 ceph-mon[98694]: pgmap v15945: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 645 KiB/s wr, 20 op/s 2023-09-10T07:32:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:47 smithi098 ceph-mon[103124]: pgmap v15945: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 645 KiB/s wr, 20 op/s 2023-09-10T07:32:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:47 smithi170 ceph-mon[102832]: pgmap v15945: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 645 KiB/s wr, 20 op/s 2023-09-10T07:32:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29600.655130 secs 2023-09-10T07:32:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29601.131690 secs 2023-09-10T07:32:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:50 smithi170 ceph-mon[102832]: pgmap v15946: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 493 KiB/s wr, 14 op/s 2023-09-10T07:32:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29600.655130 secs 2023-09-10T07:32:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29601.131690 secs 2023-09-10T07:32:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:50 smithi082 ceph-mon[98694]: pgmap v15946: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 493 KiB/s wr, 14 op/s 2023-09-10T07:32:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29600.655130 secs 2023-09-10T07:32:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29601.131690 secs 2023-09-10T07:32:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:50 smithi098 ceph-mon[103124]: pgmap v15946: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 493 KiB/s wr, 14 op/s 2023-09-10T07:32:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:52 smithi170 ceph-mon[102832]: pgmap v15947: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 632 KiB/s wr, 21 op/s 2023-09-10T07:32:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:52 smithi082 ceph-mon[98694]: pgmap v15947: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 632 KiB/s wr, 21 op/s 2023-09-10T07:32:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:52 smithi098 ceph-mon[103124]: pgmap v15947: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 632 KiB/s wr, 21 op/s 2023-09-10T07:32:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29604.519136 secs 2023-09-10T07:32:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:54 smithi170 ceph-mon[102832]: pgmap v15948: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 397 KiB/s wr, 14 op/s 2023-09-10T07:32:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29604.519136 secs 2023-09-10T07:32:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:54 smithi082 ceph-mon[98694]: pgmap v15948: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 397 KiB/s wr, 14 op/s 2023-09-10T07:32:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29604.519136 secs 2023-09-10T07:32:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:54 smithi098 ceph-mon[103124]: pgmap v15948: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 397 KiB/s wr, 14 op/s 2023-09-10T07:32:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29605.655189 secs 2023-09-10T07:32:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29606.131749 secs 2023-09-10T07:32:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29605.655189 secs 2023-09-10T07:32:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29606.131749 secs 2023-09-10T07:32:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29605.655189 secs 2023-09-10T07:32:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29606.131749 secs 2023-09-10T07:32:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:56 smithi170 ceph-mon[102832]: pgmap v15949: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 397 KiB/s wr, 14 op/s 2023-09-10T07:32:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:32:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:32:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:32:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:56 smithi082 ceph-mon[98694]: pgmap v15949: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 397 KiB/s wr, 14 op/s 2023-09-10T07:32:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:32:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:32:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:32:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:56 smithi098 ceph-mon[103124]: pgmap v15949: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 397 KiB/s wr, 14 op/s 2023-09-10T07:32:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:32:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:32:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:32:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:57 smithi082 ceph-mon[98694]: pgmap v15950: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 541 KiB/s wr, 20 op/s 2023-09-10T07:32:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:57 smithi098 ceph-mon[103124]: pgmap v15950: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 541 KiB/s wr, 20 op/s 2023-09-10T07:32:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:57 smithi170 ceph-mon[102832]: pgmap v15950: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 541 KiB/s wr, 20 op/s 2023-09-10T07:32:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29609.519230 secs 2023-09-10T07:32:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29609.519230 secs 2023-09-10T07:32:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29609.519230 secs 2023-09-10T07:33:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29610.655273 secs 2023-09-10T07:33:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29611.131814 secs 2023-09-10T07:33:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:32:59 smithi082 ceph-mon[98694]: pgmap v15951: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 283 KiB/s wr, 12 op/s 2023-09-10T07:33:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29610.655273 secs 2023-09-10T07:33:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29611.131814 secs 2023-09-10T07:33:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:32:59 smithi098 ceph-mon[103124]: pgmap v15951: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 283 KiB/s wr, 12 op/s 2023-09-10T07:33:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29610.655273 secs 2023-09-10T07:33:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29611.131814 secs 2023-09-10T07:33:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:32:59 smithi170 ceph-mon[102832]: pgmap v15951: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 283 KiB/s wr, 12 op/s 2023-09-10T07:33:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:02 smithi170 ceph-mon[102832]: pgmap v15952: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 398 KiB/s wr, 18 op/s 2023-09-10T07:33:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:02 smithi082 ceph-mon[98694]: pgmap v15952: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 398 KiB/s wr, 18 op/s 2023-09-10T07:33:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:02 smithi098 ceph-mon[103124]: pgmap v15952: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 398 KiB/s wr, 18 op/s 2023-09-10T07:33:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29614.519297 secs 2023-09-10T07:33:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:04 smithi170 ceph-mon[102832]: pgmap v15953: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 259 KiB/s wr, 11 op/s 2023-09-10T07:33:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29614.519297 secs 2023-09-10T07:33:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:04 smithi082 ceph-mon[98694]: pgmap v15953: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 259 KiB/s wr, 11 op/s 2023-09-10T07:33:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29614.519297 secs 2023-09-10T07:33:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:04 smithi098 ceph-mon[103124]: pgmap v15953: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 259 KiB/s wr, 11 op/s 2023-09-10T07:33:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29615.655330 secs 2023-09-10T07:33:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29616.131871 secs 2023-09-10T07:33:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29615.655330 secs 2023-09-10T07:33:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29616.131871 secs 2023-09-10T07:33:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29615.655330 secs 2023-09-10T07:33:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29616.131871 secs 2023-09-10T07:33:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:06 smithi170 ceph-mon[102832]: pgmap v15954: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 259 KiB/s wr, 11 op/s 2023-09-10T07:33:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:06 smithi082 ceph-mon[98694]: pgmap v15954: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 259 KiB/s wr, 11 op/s 2023-09-10T07:33:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:06 smithi098 ceph-mon[103124]: pgmap v15954: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 259 KiB/s wr, 11 op/s 2023-09-10T07:33:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:07 smithi082 ceph-mon[98694]: pgmap v15955: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 304 KiB/s wr, 29 op/s 2023-09-10T07:33:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:07 smithi098 ceph-mon[103124]: pgmap v15955: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 304 KiB/s wr, 29 op/s 2023-09-10T07:33:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:07 smithi170 ceph-mon[102832]: pgmap v15955: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 304 KiB/s wr, 29 op/s 2023-09-10T07:33:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29619.519359 secs 2023-09-10T07:33:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:08 smithi082 ceph-mon[98694]: osdmap e1065: 12 total, 12 up, 12 in 2023-09-10T07:33:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29619.519359 secs 2023-09-10T07:33:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:08 smithi098 ceph-mon[103124]: osdmap e1065: 12 total, 12 up, 12 in 2023-09-10T07:33:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29619.519359 secs 2023-09-10T07:33:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:08 smithi170 ceph-mon[102832]: osdmap e1065: 12 total, 12 up, 12 in 2023-09-10T07:33:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29620.655413 secs 2023-09-10T07:33:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29621.131934 secs 2023-09-10T07:33:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:09 smithi082 ceph-mon[98694]: pgmap v15957: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 192 KiB/s wr, 29 op/s 2023-09-10T07:33:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29620.655413 secs 2023-09-10T07:33:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29621.131934 secs 2023-09-10T07:33:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:09 smithi098 ceph-mon[103124]: pgmap v15957: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 192 KiB/s wr, 29 op/s 2023-09-10T07:33:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29620.655413 secs 2023-09-10T07:33:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29621.131934 secs 2023-09-10T07:33:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:09 smithi170 ceph-mon[102832]: pgmap v15957: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 192 KiB/s wr, 29 op/s 2023-09-10T07:33:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:12 smithi170 ceph-mon[102832]: pgmap v15958: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T07:33:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:12 smithi170 ceph-mon[102832]: osdmap e1066: 12 total, 12 up, 12 in 2023-09-10T07:33:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:12 smithi082 ceph-mon[98694]: pgmap v15958: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T07:33:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:12 smithi082 ceph-mon[98694]: osdmap e1066: 12 total, 12 up, 12 in 2023-09-10T07:33:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:12 smithi098 ceph-mon[103124]: pgmap v15958: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T07:33:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:12 smithi098 ceph-mon[103124]: osdmap e1066: 12 total, 12 up, 12 in 2023-09-10T07:33:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29624.519418 secs 2023-09-10T07:33:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:14 smithi170 ceph-mon[102832]: pgmap v15960: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 68 KiB/s wr, 33 op/s 2023-09-10T07:33:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:33:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29624.519418 secs 2023-09-10T07:33:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:14 smithi082 ceph-mon[98694]: pgmap v15960: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 68 KiB/s wr, 33 op/s 2023-09-10T07:33:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:33:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29624.519418 secs 2023-09-10T07:33:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:14 smithi098 ceph-mon[103124]: pgmap v15960: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 68 KiB/s wr, 33 op/s 2023-09-10T07:33:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:33:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29625.655471 secs 2023-09-10T07:33:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29626.131999 secs 2023-09-10T07:33:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:33:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:33:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:33:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29625.655471 secs 2023-09-10T07:33:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29626.131999 secs 2023-09-10T07:33:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:33:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:33:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:33:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29625.655471 secs 2023-09-10T07:33:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29626.131999 secs 2023-09-10T07:33:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:33:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:33:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:33:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:16 smithi098 ceph-mon[103124]: pgmap v15961: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:33:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:16 smithi170 ceph-mon[102832]: pgmap v15961: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:33:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:16 smithi082 ceph-mon[98694]: pgmap v15961: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:33:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:17 smithi082 ceph-mon[98694]: pgmap v15962: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:33:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:17 smithi098 ceph-mon[103124]: pgmap v15962: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:33:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:17 smithi170 ceph-mon[102832]: pgmap v15962: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:33:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29629.519526 secs 2023-09-10T07:33:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29629.519526 secs 2023-09-10T07:33:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29629.519526 secs 2023-09-10T07:33:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29630.655495 secs 2023-09-10T07:33:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29631.132063 secs 2023-09-10T07:33:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:19 smithi082 ceph-mon[98694]: pgmap v15963: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:33:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29630.655495 secs 2023-09-10T07:33:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29631.132063 secs 2023-09-10T07:33:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:19 smithi098 ceph-mon[103124]: pgmap v15963: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:33:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29630.655495 secs 2023-09-10T07:33:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29631.132063 secs 2023-09-10T07:33:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:19 smithi170 ceph-mon[102832]: pgmap v15963: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:33:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:22 smithi170 ceph-mon[102832]: pgmap v15964: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:33:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:22 smithi082 ceph-mon[98694]: pgmap v15964: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:33:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:22 smithi098 ceph-mon[103124]: pgmap v15964: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:33:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29634.519586 secs 2023-09-10T07:33:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:24 smithi170 ceph-mon[102832]: pgmap v15965: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 183 B/s wr, 9 op/s 2023-09-10T07:33:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29634.519586 secs 2023-09-10T07:33:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:24 smithi082 ceph-mon[98694]: pgmap v15965: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 183 B/s wr, 9 op/s 2023-09-10T07:33:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29634.519586 secs 2023-09-10T07:33:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:24 smithi098 ceph-mon[103124]: pgmap v15965: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 183 B/s wr, 9 op/s 2023-09-10T07:33:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29635.655554 secs 2023-09-10T07:33:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29636.132124 secs 2023-09-10T07:33:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29635.655554 secs 2023-09-10T07:33:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29636.132124 secs 2023-09-10T07:33:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29635.655554 secs 2023-09-10T07:33:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29636.132124 secs 2023-09-10T07:33:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:26 smithi170 ceph-mon[102832]: pgmap v15966: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-10T07:33:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:26 smithi082 ceph-mon[98694]: pgmap v15966: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-10T07:33:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:26 smithi098 ceph-mon[103124]: pgmap v15966: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-10T07:33:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:27 smithi082 ceph-mon[98694]: pgmap v15967: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 99 KiB/s wr, 14 op/s 2023-09-10T07:33:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:27 smithi098 ceph-mon[103124]: pgmap v15967: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 99 KiB/s wr, 14 op/s 2023-09-10T07:33:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:27 smithi170 ceph-mon[102832]: pgmap v15967: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 99 KiB/s wr, 14 op/s 2023-09-10T07:33:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29639.519648 secs 2023-09-10T07:33:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29639.519648 secs 2023-09-10T07:33:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29639.519648 secs 2023-09-10T07:33:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29640.655615 secs 2023-09-10T07:33:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29641.132189 secs 2023-09-10T07:33:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:29 smithi082 ceph-mon[98694]: pgmap v15968: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T07:33:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29640.655615 secs 2023-09-10T07:33:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29641.132189 secs 2023-09-10T07:33:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:29 smithi098 ceph-mon[103124]: pgmap v15968: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T07:33:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29640.655615 secs 2023-09-10T07:33:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29641.132189 secs 2023-09-10T07:33:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:29 smithi170 ceph-mon[102832]: pgmap v15968: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 99 KiB/s wr, 10 op/s 2023-09-10T07:33:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:32 smithi170 ceph-mon[102832]: pgmap v15969: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 201 KiB/s wr, 15 op/s 2023-09-10T07:33:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:32 smithi082 ceph-mon[98694]: pgmap v15969: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 201 KiB/s wr, 15 op/s 2023-09-10T07:33:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:32 smithi098 ceph-mon[103124]: pgmap v15969: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 201 KiB/s wr, 15 op/s 2023-09-10T07:33:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:34 smithi170 ceph-mon[102832]: pgmap v15970: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 201 KiB/s wr, 10 op/s 2023-09-10T07:33:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:34 smithi082 ceph-mon[98694]: pgmap v15970: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 201 KiB/s wr, 10 op/s 2023-09-10T07:33:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:34 smithi098 ceph-mon[103124]: pgmap v15970: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 201 KiB/s wr, 10 op/s 2023-09-10T07:33:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29645.655716 secs 2023-09-10T07:33:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29646.132249 secs 2023-09-10T07:33:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:33:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29645.655716 secs 2023-09-10T07:33:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29646.132249 secs 2023-09-10T07:33:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:33:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29645.655716 secs 2023-09-10T07:33:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29646.132249 secs 2023-09-10T07:33:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:33:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:36 smithi170 ceph-mon[102832]: pgmap v15971: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 201 KiB/s wr, 10 op/s 2023-09-10T07:33:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:33:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:33:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:33:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:36 smithi082 ceph-mon[98694]: pgmap v15971: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 201 KiB/s wr, 10 op/s 2023-09-10T07:33:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:33:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:33:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:33:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:36 smithi098 ceph-mon[103124]: pgmap v15971: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 201 KiB/s wr, 10 op/s 2023-09-10T07:33:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:33:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:33:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:33:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:37 smithi082 ceph-mon[98694]: pgmap v15972: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 302 KiB/s wr, 16 op/s 2023-09-10T07:33:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:37 smithi098 ceph-mon[103124]: pgmap v15972: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 302 KiB/s wr, 16 op/s 2023-09-10T07:33:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:37 smithi170 ceph-mon[102832]: pgmap v15972: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 302 KiB/s wr, 16 op/s 2023-09-10T07:33:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29649.519772 secs 2023-09-10T07:33:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29649.519772 secs 2023-09-10T07:33:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29649.519772 secs 2023-09-10T07:33:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29650.655792 secs 2023-09-10T07:33:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29651.132312 secs 2023-09-10T07:33:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:39 smithi082 ceph-mon[98694]: pgmap v15973: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 203 KiB/s wr, 10 op/s 2023-09-10T07:33:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29650.655792 secs 2023-09-10T07:33:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29651.132312 secs 2023-09-10T07:33:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:39 smithi098 ceph-mon[103124]: pgmap v15973: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 203 KiB/s wr, 10 op/s 2023-09-10T07:33:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29650.655792 secs 2023-09-10T07:33:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29651.132312 secs 2023-09-10T07:33:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:39 smithi170 ceph-mon[102832]: pgmap v15973: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 203 KiB/s wr, 10 op/s 2023-09-10T07:33:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:33:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:33:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:33:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:33:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:33:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:33:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:41 smithi098 ceph-mon[103124]: pgmap v15974: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 425 KiB/s wr, 16 op/s 2023-09-10T07:33:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:41 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:33:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:41 smithi170 ceph-mon[102832]: pgmap v15974: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 425 KiB/s wr, 16 op/s 2023-09-10T07:33:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:41 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:33:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:41 smithi082 ceph-mon[98694]: pgmap v15974: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 425 KiB/s wr, 16 op/s 2023-09-10T07:33:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:41 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:33:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:44 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29654.519830 secs 2023-09-10T07:33:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:44 smithi170 ceph-mon[102832]: pgmap v15975: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 322 KiB/s wr, 11 op/s 2023-09-10T07:33:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:44 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29654.519830 secs 2023-09-10T07:33:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:44 smithi082 ceph-mon[98694]: pgmap v15975: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 322 KiB/s wr, 11 op/s 2023-09-10T07:33:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:44 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29654.519830 secs 2023-09-10T07:33:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:44 smithi098 ceph-mon[103124]: pgmap v15975: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 322 KiB/s wr, 11 op/s 2023-09-10T07:33:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29655.655855 secs 2023-09-10T07:33:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29656.132374 secs 2023-09-10T07:33:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29655.655855 secs 2023-09-10T07:33:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29656.132374 secs 2023-09-10T07:33:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29655.655855 secs 2023-09-10T07:33:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29656.132374 secs 2023-09-10T07:33:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:46 smithi170 ceph-mon[102832]: pgmap v15976: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 322 KiB/s wr, 11 op/s 2023-09-10T07:33:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:46 smithi082 ceph-mon[98694]: pgmap v15976: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 322 KiB/s wr, 11 op/s 2023-09-10T07:33:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:46 smithi098 ceph-mon[103124]: pgmap v15976: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 322 KiB/s wr, 11 op/s 2023-09-10T07:33:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:47 smithi082 ceph-mon[98694]: pgmap v15977: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 470 KiB/s wr, 17 op/s 2023-09-10T07:33:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:47 smithi098 ceph-mon[103124]: pgmap v15977: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 470 KiB/s wr, 17 op/s 2023-09-10T07:33:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:47 smithi170 ceph-mon[102832]: pgmap v15977: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 470 KiB/s wr, 17 op/s 2023-09-10T07:33:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29660.655914 secs 2023-09-10T07:33:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29661.132442 secs 2023-09-10T07:33:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:50 smithi170 ceph-mon[102832]: pgmap v15978: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 369 KiB/s wr, 12 op/s 2023-09-10T07:33:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29660.655914 secs 2023-09-10T07:33:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29661.132442 secs 2023-09-10T07:33:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:50 smithi082 ceph-mon[98694]: pgmap v15978: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 369 KiB/s wr, 12 op/s 2023-09-10T07:33:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29660.655914 secs 2023-09-10T07:33:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29661.132442 secs 2023-09-10T07:33:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:50 smithi098 ceph-mon[103124]: pgmap v15978: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 369 KiB/s wr, 12 op/s 2023-09-10T07:33:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:51 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:33:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:51 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:33:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:51 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:33:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:52 smithi170 ceph-mon[102832]: pgmap v15979: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 635 KiB/s wr, 20 op/s 2023-09-10T07:33:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:52 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:33:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:33:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:33:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:52 smithi082 ceph-mon[98694]: pgmap v15979: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 635 KiB/s wr, 20 op/s 2023-09-10T07:33:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:52 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:33:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:33:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:33:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:52 smithi098 ceph-mon[103124]: pgmap v15979: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 635 KiB/s wr, 20 op/s 2023-09-10T07:33:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:52 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:33:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:33:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:33:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:54 smithi170 ceph-mon[102832]: pgmap v15980: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 413 KiB/s wr, 14 op/s 2023-09-10T07:33:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:54 smithi082 ceph-mon[98694]: pgmap v15980: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 413 KiB/s wr, 14 op/s 2023-09-10T07:33:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:54 smithi098 ceph-mon[103124]: pgmap v15980: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 413 KiB/s wr, 14 op/s 2023-09-10T07:33:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29665.655971 secs 2023-09-10T07:33:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29666.132483 secs 2023-09-10T07:33:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29665.655971 secs 2023-09-10T07:33:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29666.132483 secs 2023-09-10T07:33:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29665.655971 secs 2023-09-10T07:33:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29666.132483 secs 2023-09-10T07:33:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:56 smithi082 ceph-mon[98694]: pgmap v15981: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 413 KiB/s wr, 14 op/s 2023-09-10T07:33:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:56 smithi098 ceph-mon[103124]: pgmap v15981: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 413 KiB/s wr, 14 op/s 2023-09-10T07:33:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:56 smithi170 ceph-mon[102832]: pgmap v15981: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 413 KiB/s wr, 14 op/s 2023-09-10T07:33:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:57 smithi082 ceph-mon[98694]: pgmap v15982: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 589 KiB/s wr, 20 op/s 2023-09-10T07:33:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:57 smithi098 ceph-mon[103124]: pgmap v15982: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 589 KiB/s wr, 20 op/s 2023-09-10T07:33:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:57 smithi170 ceph-mon[102832]: pgmap v15982: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 589 KiB/s wr, 20 op/s 2023-09-10T07:33:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29669.520065 secs 2023-09-10T07:33:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29669.520065 secs 2023-09-10T07:33:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29669.520065 secs 2023-09-10T07:34:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29670.656028 secs 2023-09-10T07:34:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29671.132545 secs 2023-09-10T07:34:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:33:59 smithi098 ceph-mon[103124]: pgmap v15983: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 442 KiB/s wr, 14 op/s 2023-09-10T07:34:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29670.656028 secs 2023-09-10T07:34:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29671.132545 secs 2023-09-10T07:34:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:33:59 smithi170 ceph-mon[102832]: pgmap v15983: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 442 KiB/s wr, 14 op/s 2023-09-10T07:34:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29670.656028 secs 2023-09-10T07:34:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29671.132545 secs 2023-09-10T07:34:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:33:59 smithi082 ceph-mon[98694]: pgmap v15983: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 442 KiB/s wr, 14 op/s 2023-09-10T07:34:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:34:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:34:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:34:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:34:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:34:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:34:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:34:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:34:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:34:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:01 smithi098 ceph-mon[103124]: pgmap v15984: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 681 KiB/s wr, 20 op/s 2023-09-10T07:34:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:01 smithi170 ceph-mon[102832]: pgmap v15984: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 681 KiB/s wr, 20 op/s 2023-09-10T07:34:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:01 smithi082 ceph-mon[98694]: pgmap v15984: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 681 KiB/s wr, 20 op/s 2023-09-10T07:34:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29674.520166 secs 2023-09-10T07:34:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:04 smithi170 ceph-mon[102832]: pgmap v15985: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 415 KiB/s wr, 12 op/s 2023-09-10T07:34:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29674.520166 secs 2023-09-10T07:34:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:04 smithi082 ceph-mon[98694]: pgmap v15985: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 415 KiB/s wr, 12 op/s 2023-09-10T07:34:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29674.520166 secs 2023-09-10T07:34:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:04 smithi098 ceph-mon[103124]: pgmap v15985: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 415 KiB/s wr, 12 op/s 2023-09-10T07:34:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29675.656084 secs 2023-09-10T07:34:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29676.132602 secs 2023-09-10T07:34:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29675.656084 secs 2023-09-10T07:34:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29676.132602 secs 2023-09-10T07:34:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29675.656084 secs 2023-09-10T07:34:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29676.132602 secs 2023-09-10T07:34:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:06 smithi082 ceph-mon[98694]: pgmap v15986: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 415 KiB/s wr, 12 op/s 2023-09-10T07:34:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:06 smithi098 ceph-mon[103124]: pgmap v15986: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 415 KiB/s wr, 12 op/s 2023-09-10T07:34:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:06 smithi170 ceph-mon[102832]: pgmap v15986: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 415 KiB/s wr, 12 op/s 2023-09-10T07:34:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:07 smithi082 ceph-mon[98694]: pgmap v15987: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 542 KiB/s wr, 31 op/s 2023-09-10T07:34:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:07 smithi098 ceph-mon[103124]: pgmap v15987: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 542 KiB/s wr, 31 op/s 2023-09-10T07:34:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:07 smithi170 ceph-mon[102832]: pgmap v15987: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 542 KiB/s wr, 31 op/s 2023-09-10T07:34:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29679.520229 secs 2023-09-10T07:34:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:08 smithi082 ceph-mon[98694]: osdmap e1067: 12 total, 12 up, 12 in 2023-09-10T07:34:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29679.520229 secs 2023-09-10T07:34:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:08 smithi098 ceph-mon[103124]: osdmap e1067: 12 total, 12 up, 12 in 2023-09-10T07:34:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29679.520229 secs 2023-09-10T07:34:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:08 smithi170 ceph-mon[102832]: osdmap e1067: 12 total, 12 up, 12 in 2023-09-10T07:34:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29680.656142 secs 2023-09-10T07:34:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29681.132666 secs 2023-09-10T07:34:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:09 smithi082 ceph-mon[98694]: pgmap v15989: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 439 KiB/s wr, 30 op/s 2023-09-10T07:34:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29680.656142 secs 2023-09-10T07:34:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29681.132666 secs 2023-09-10T07:34:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:09 smithi098 ceph-mon[103124]: pgmap v15989: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 439 KiB/s wr, 30 op/s 2023-09-10T07:34:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29680.656142 secs 2023-09-10T07:34:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29681.132666 secs 2023-09-10T07:34:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:09 smithi170 ceph-mon[102832]: pgmap v15989: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 439 KiB/s wr, 30 op/s 2023-09-10T07:34:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:12 smithi082 ceph-mon[98694]: pgmap v15990: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 152 KiB/s wr, 28 op/s 2023-09-10T07:34:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:12 smithi082 ceph-mon[98694]: osdmap e1068: 12 total, 12 up, 12 in 2023-09-10T07:34:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:12 smithi098 ceph-mon[103124]: pgmap v15990: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 152 KiB/s wr, 28 op/s 2023-09-10T07:34:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:12 smithi098 ceph-mon[103124]: osdmap e1068: 12 total, 12 up, 12 in 2023-09-10T07:34:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:12 smithi170 ceph-mon[102832]: pgmap v15990: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 152 KiB/s wr, 28 op/s 2023-09-10T07:34:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:12 smithi170 ceph-mon[102832]: osdmap e1068: 12 total, 12 up, 12 in 2023-09-10T07:34:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29684.520314 secs 2023-09-10T07:34:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:14 smithi082 ceph-mon[98694]: pgmap v15992: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 190 KiB/s wr, 36 op/s 2023-09-10T07:34:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29684.520314 secs 2023-09-10T07:34:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:14 smithi098 ceph-mon[103124]: pgmap v15992: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 190 KiB/s wr, 36 op/s 2023-09-10T07:34:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29684.520314 secs 2023-09-10T07:34:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:14 smithi170 ceph-mon[102832]: pgmap v15992: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 190 KiB/s wr, 36 op/s 2023-09-10T07:34:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29685.656201 secs 2023-09-10T07:34:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29686.132726 secs 2023-09-10T07:34:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:34:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:34:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:34:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:34:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29685.656201 secs 2023-09-10T07:34:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29686.132726 secs 2023-09-10T07:34:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:34:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:34:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:34:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:34:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29685.656201 secs 2023-09-10T07:34:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29686.132726 secs 2023-09-10T07:34:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:34:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:34:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:34:15.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:34:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:16 smithi082 ceph-mon[98694]: pgmap v15993: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-10T07:34:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:16 smithi098 ceph-mon[103124]: pgmap v15993: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-10T07:34:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:16 smithi170 ceph-mon[102832]: pgmap v15993: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-10T07:34:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:17 smithi082 ceph-mon[98694]: pgmap v15994: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 334 B/s wr, 11 op/s 2023-09-10T07:34:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:17 smithi098 ceph-mon[103124]: pgmap v15994: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 334 B/s wr, 11 op/s 2023-09-10T07:34:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:17 smithi170 ceph-mon[102832]: pgmap v15994: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 334 B/s wr, 11 op/s 2023-09-10T07:34:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29689.520376 secs 2023-09-10T07:34:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29689.520376 secs 2023-09-10T07:34:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29689.520376 secs 2023-09-10T07:34:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29690.656257 secs 2023-09-10T07:34:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29691.132793 secs 2023-09-10T07:34:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:19 smithi098 ceph-mon[103124]: pgmap v15995: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T07:34:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29690.656257 secs 2023-09-10T07:34:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29691.132793 secs 2023-09-10T07:34:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:19 smithi170 ceph-mon[102832]: pgmap v15995: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T07:34:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29690.656257 secs 2023-09-10T07:34:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29691.132793 secs 2023-09-10T07:34:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:19 smithi082 ceph-mon[98694]: pgmap v15995: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T07:34:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:22 smithi082 ceph-mon[98694]: pgmap v15996: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:34:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:22 smithi098 ceph-mon[103124]: pgmap v15996: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:34:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:22 smithi170 ceph-mon[102832]: pgmap v15996: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:34:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29694.520435 secs 2023-09-10T07:34:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:24 smithi082 ceph-mon[98694]: pgmap v15997: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:34:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29694.520435 secs 2023-09-10T07:34:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:24 smithi098 ceph-mon[103124]: pgmap v15997: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:34:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29694.520435 secs 2023-09-10T07:34:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:24 smithi170 ceph-mon[102832]: pgmap v15997: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:34:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29695.656327 secs 2023-09-10T07:34:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29696.132853 secs 2023-09-10T07:34:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29695.656327 secs 2023-09-10T07:34:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29696.132853 secs 2023-09-10T07:34:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29695.656327 secs 2023-09-10T07:34:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29696.132853 secs 2023-09-10T07:34:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:26 smithi082 ceph-mon[98694]: pgmap v15998: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:34:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:26 smithi098 ceph-mon[103124]: pgmap v15998: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:34:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:26 smithi170 ceph-mon[102832]: pgmap v15998: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:34:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:27 smithi098 ceph-mon[103124]: pgmap v15999: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-10T07:34:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:27 smithi170 ceph-mon[102832]: pgmap v15999: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-10T07:34:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:27 smithi082 ceph-mon[98694]: pgmap v15999: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-10T07:34:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29699.520494 secs 2023-09-10T07:34:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29699.520494 secs 2023-09-10T07:34:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29699.520494 secs 2023-09-10T07:34:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29700.656384 secs 2023-09-10T07:34:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29701.132918 secs 2023-09-10T07:34:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:29 smithi098 ceph-mon[103124]: pgmap v16000: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-10T07:34:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29700.656384 secs 2023-09-10T07:34:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29701.132918 secs 2023-09-10T07:34:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:29 smithi170 ceph-mon[102832]: pgmap v16000: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-10T07:34:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29700.656384 secs 2023-09-10T07:34:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29701.132918 secs 2023-09-10T07:34:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:29 smithi082 ceph-mon[98694]: pgmap v16000: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-10T07:34:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:32 smithi082 ceph-mon[98694]: pgmap v16001: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 682 B/s wr, 13 op/s 2023-09-10T07:34:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:32 smithi098 ceph-mon[103124]: pgmap v16001: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 682 B/s wr, 13 op/s 2023-09-10T07:34:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:32 smithi170 ceph-mon[102832]: pgmap v16001: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 682 B/s wr, 13 op/s 2023-09-10T07:34:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29704.520593 secs 2023-09-10T07:34:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:34 smithi082 ceph-mon[98694]: pgmap v16002: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2023-09-10T07:34:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29704.520593 secs 2023-09-10T07:34:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:34 smithi098 ceph-mon[103124]: pgmap v16002: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2023-09-10T07:34:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29704.520593 secs 2023-09-10T07:34:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:34 smithi170 ceph-mon[102832]: pgmap v16002: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2023-09-10T07:34:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29705.656441 secs 2023-09-10T07:34:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29706.132977 secs 2023-09-10T07:34:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29705.656441 secs 2023-09-10T07:34:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29706.132977 secs 2023-09-10T07:34:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29705.656441 secs 2023-09-10T07:34:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29706.132977 secs 2023-09-10T07:34:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:36 smithi082 ceph-mon[98694]: pgmap v16003: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2023-09-10T07:34:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:36 smithi098 ceph-mon[103124]: pgmap v16003: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2023-09-10T07:34:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:36 smithi170 ceph-mon[102832]: pgmap v16003: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2023-09-10T07:34:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:37 smithi098 ceph-mon[103124]: pgmap v16004: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 231 KiB/s wr, 15 op/s 2023-09-10T07:34:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:37 smithi170 ceph-mon[102832]: pgmap v16004: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 231 KiB/s wr, 15 op/s 2023-09-10T07:34:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:37 smithi082 ceph-mon[98694]: pgmap v16004: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 231 KiB/s wr, 15 op/s 2023-09-10T07:34:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:34:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:34:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:34:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29710.656499 secs 2023-09-10T07:34:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29711.133040 secs 2023-09-10T07:34:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:34:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:34:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:34:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:40 smithi170 ceph-mon[102832]: pgmap v16005: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T07:34:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29710.656499 secs 2023-09-10T07:34:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29711.133040 secs 2023-09-10T07:34:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:34:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:34:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:34:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:40 smithi082 ceph-mon[98694]: pgmap v16005: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T07:34:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29710.656499 secs 2023-09-10T07:34:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29711.133040 secs 2023-09-10T07:34:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:34:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:34:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:34:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:40 smithi098 ceph-mon[103124]: pgmap v16005: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T07:34:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:42 smithi082 ceph-mon[98694]: pgmap v16006: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 452 KiB/s wr, 17 op/s 2023-09-10T07:34:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:42 smithi098 ceph-mon[103124]: pgmap v16006: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 452 KiB/s wr, 17 op/s 2023-09-10T07:34:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:42 smithi170 ceph-mon[102832]: pgmap v16006: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 452 KiB/s wr, 17 op/s 2023-09-10T07:34:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:44 smithi082 ceph-mon[98694]: pgmap v16007: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 451 KiB/s wr, 12 op/s 2023-09-10T07:34:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:44 smithi098 ceph-mon[103124]: pgmap v16007: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 451 KiB/s wr, 12 op/s 2023-09-10T07:34:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:44 smithi170 ceph-mon[102832]: pgmap v16007: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 451 KiB/s wr, 12 op/s 2023-09-10T07:34:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29715.656558 secs 2023-09-10T07:34:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29716.133100 secs 2023-09-10T07:34:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29715.656558 secs 2023-09-10T07:34:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29716.133100 secs 2023-09-10T07:34:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29715.656558 secs 2023-09-10T07:34:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29716.133100 secs 2023-09-10T07:34:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:46 smithi082 ceph-mon[98694]: pgmap v16008: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 451 KiB/s wr, 12 op/s 2023-09-10T07:34:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:46 smithi098 ceph-mon[103124]: pgmap v16008: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 451 KiB/s wr, 12 op/s 2023-09-10T07:34:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:46 smithi170 ceph-mon[102832]: pgmap v16008: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 451 KiB/s wr, 12 op/s 2023-09-10T07:34:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:47 smithi098 ceph-mon[103124]: pgmap v16009: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 726 KiB/s wr, 20 op/s 2023-09-10T07:34:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:47 smithi170 ceph-mon[102832]: pgmap v16009: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 726 KiB/s wr, 20 op/s 2023-09-10T07:34:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:47 smithi082 ceph-mon[98694]: pgmap v16009: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 726 KiB/s wr, 20 op/s 2023-09-10T07:34:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29720.656618 secs 2023-09-10T07:34:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29721.133162 secs 2023-09-10T07:34:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:50 smithi082 ceph-mon[98694]: pgmap v16010: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 496 KiB/s wr, 14 op/s 2023-09-10T07:34:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29720.656618 secs 2023-09-10T07:34:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29721.133162 secs 2023-09-10T07:34:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:50 smithi098 ceph-mon[103124]: pgmap v16010: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 496 KiB/s wr, 14 op/s 2023-09-10T07:34:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29720.656618 secs 2023-09-10T07:34:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29721.133162 secs 2023-09-10T07:34:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:50 smithi170 ceph-mon[102832]: pgmap v16010: 97 pgs: 97 active+clean; 9.9 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 496 KiB/s wr, 14 op/s 2023-09-10T07:34:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:52 smithi082 ceph-mon[98694]: pgmap v16011: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 636 KiB/s wr, 20 op/s 2023-09-10T07:34:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:52 smithi098 ceph-mon[103124]: pgmap v16011: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 636 KiB/s wr, 20 op/s 2023-09-10T07:34:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:52 smithi170 ceph-mon[102832]: pgmap v16011: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 636 KiB/s wr, 20 op/s 2023-09-10T07:34:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29724.520836 secs 2023-09-10T07:34:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:54 smithi082 ceph-mon[98694]: pgmap v16012: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 415 KiB/s wr, 14 op/s 2023-09-10T07:34:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29724.520836 secs 2023-09-10T07:34:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:54 smithi098 ceph-mon[103124]: pgmap v16012: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 415 KiB/s wr, 14 op/s 2023-09-10T07:34:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29724.520836 secs 2023-09-10T07:34:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:54 smithi170 ceph-mon[102832]: pgmap v16012: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 415 KiB/s wr, 14 op/s 2023-09-10T07:34:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29725.656677 secs 2023-09-10T07:34:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29726.133225 secs 2023-09-10T07:34:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29725.656677 secs 2023-09-10T07:34:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29726.133225 secs 2023-09-10T07:34:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29725.656677 secs 2023-09-10T07:34:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29726.133225 secs 2023-09-10T07:34:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:56 smithi082 ceph-mon[98694]: pgmap v16013: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 415 KiB/s wr, 14 op/s 2023-09-10T07:34:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:34:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:34:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:34:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:56 smithi098 ceph-mon[103124]: pgmap v16013: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 415 KiB/s wr, 14 op/s 2023-09-10T07:34:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:34:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:34:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:34:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:56 smithi170 ceph-mon[102832]: pgmap v16013: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 415 KiB/s wr, 14 op/s 2023-09-10T07:34:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:34:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:34:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:34:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:57 smithi098 ceph-mon[103124]: pgmap v16014: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 559 KiB/s wr, 19 op/s 2023-09-10T07:34:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:57 smithi170 ceph-mon[102832]: pgmap v16014: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 559 KiB/s wr, 19 op/s 2023-09-10T07:34:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:57 smithi082 ceph-mon[98694]: pgmap v16014: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 559 KiB/s wr, 19 op/s 2023-09-10T07:34:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29729.520897 secs 2023-09-10T07:34:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29729.520897 secs 2023-09-10T07:34:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29729.520897 secs 2023-09-10T07:35:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29730.656762 secs 2023-09-10T07:35:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29731.133287 secs 2023-09-10T07:35:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:34:59 smithi098 ceph-mon[103124]: pgmap v16015: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-09-10T07:35:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29730.656762 secs 2023-09-10T07:35:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29731.133287 secs 2023-09-10T07:35:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:34:59 smithi170 ceph-mon[102832]: pgmap v16015: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-09-10T07:35:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29730.656762 secs 2023-09-10T07:35:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29731.133287 secs 2023-09-10T07:35:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:34:59 smithi082 ceph-mon[98694]: pgmap v16015: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 284 KiB/s wr, 12 op/s 2023-09-10T07:35:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:02 smithi082 ceph-mon[98694]: pgmap v16016: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 400 KiB/s wr, 17 op/s 2023-09-10T07:35:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:02 smithi098 ceph-mon[103124]: pgmap v16016: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 400 KiB/s wr, 17 op/s 2023-09-10T07:35:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:02 smithi170 ceph-mon[102832]: pgmap v16016: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 400 KiB/s wr, 17 op/s 2023-09-10T07:35:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29734.520958 secs 2023-09-10T07:35:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:04 smithi082 ceph-mon[98694]: pgmap v16017: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 260 KiB/s wr, 10 op/s 2023-09-10T07:35:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29734.520958 secs 2023-09-10T07:35:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:04 smithi098 ceph-mon[103124]: pgmap v16017: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 260 KiB/s wr, 10 op/s 2023-09-10T07:35:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29734.520958 secs 2023-09-10T07:35:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:04 smithi170 ceph-mon[102832]: pgmap v16017: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 260 KiB/s wr, 10 op/s 2023-09-10T07:35:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29735.656819 secs 2023-09-10T07:35:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29736.133347 secs 2023-09-10T07:35:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29735.656819 secs 2023-09-10T07:35:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29736.133347 secs 2023-09-10T07:35:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29735.656819 secs 2023-09-10T07:35:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29736.133347 secs 2023-09-10T07:35:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:06 smithi082 ceph-mon[98694]: pgmap v16018: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 260 KiB/s wr, 10 op/s 2023-09-10T07:35:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:06 smithi098 ceph-mon[103124]: pgmap v16018: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 260 KiB/s wr, 10 op/s 2023-09-10T07:35:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:06 smithi170 ceph-mon[102832]: pgmap v16018: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 260 KiB/s wr, 10 op/s 2023-09-10T07:35:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:07 smithi098 ceph-mon[103124]: pgmap v16019: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 305 KiB/s wr, 30 op/s 2023-09-10T07:35:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:07 smithi170 ceph-mon[102832]: pgmap v16019: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 305 KiB/s wr, 30 op/s 2023-09-10T07:35:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:07 smithi082 ceph-mon[98694]: pgmap v16019: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 305 KiB/s wr, 30 op/s 2023-09-10T07:35:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29739.521019 secs 2023-09-10T07:35:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:08 smithi098 ceph-mon[103124]: osdmap e1069: 12 total, 12 up, 12 in 2023-09-10T07:35:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29739.521019 secs 2023-09-10T07:35:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:08 smithi170 ceph-mon[102832]: osdmap e1069: 12 total, 12 up, 12 in 2023-09-10T07:35:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29739.521019 secs 2023-09-10T07:35:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:08 smithi082 ceph-mon[98694]: osdmap e1069: 12 total, 12 up, 12 in 2023-09-10T07:35:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29740.656903 secs 2023-09-10T07:35:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29741.133411 secs 2023-09-10T07:35:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:09 smithi098 ceph-mon[103124]: pgmap v16021: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 192 KiB/s wr, 29 op/s 2023-09-10T07:35:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29740.656903 secs 2023-09-10T07:35:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29741.133411 secs 2023-09-10T07:35:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:09 smithi170 ceph-mon[102832]: pgmap v16021: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 192 KiB/s wr, 29 op/s 2023-09-10T07:35:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29740.656903 secs 2023-09-10T07:35:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29741.133411 secs 2023-09-10T07:35:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:09 smithi082 ceph-mon[98694]: pgmap v16021: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 192 KiB/s wr, 29 op/s 2023-09-10T07:35:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:12 smithi082 ceph-mon[98694]: pgmap v16022: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:35:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:12 smithi082 ceph-mon[98694]: osdmap e1070: 12 total, 12 up, 12 in 2023-09-10T07:35:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:12 smithi098 ceph-mon[103124]: pgmap v16022: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:35:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:12 smithi098 ceph-mon[103124]: osdmap e1070: 12 total, 12 up, 12 in 2023-09-10T07:35:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:12 smithi170 ceph-mon[102832]: pgmap v16022: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:35:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:12 smithi170 ceph-mon[102832]: osdmap e1070: 12 total, 12 up, 12 in 2023-09-10T07:35:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29744.521080 secs 2023-09-10T07:35:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:14 smithi082 ceph-mon[98694]: pgmap v16024: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 68 KiB/s wr, 35 op/s 2023-09-10T07:35:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29744.521080 secs 2023-09-10T07:35:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:14 smithi098 ceph-mon[103124]: pgmap v16024: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 68 KiB/s wr, 35 op/s 2023-09-10T07:35:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29744.521080 secs 2023-09-10T07:35:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:14 smithi170 ceph-mon[102832]: pgmap v16024: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 68 KiB/s wr, 35 op/s 2023-09-10T07:35:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29745.656962 secs 2023-09-10T07:35:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29746.133472 secs 2023-09-10T07:35:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:35:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29745.656962 secs 2023-09-10T07:35:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29746.133472 secs 2023-09-10T07:35:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:35:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29745.656962 secs 2023-09-10T07:35:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29746.133472 secs 2023-09-10T07:35:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:35:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:16 smithi082 ceph-mon[98694]: pgmap v16025: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:35:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:16 smithi098 ceph-mon[103124]: pgmap v16025: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:35:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:16 smithi170 ceph-mon[102832]: pgmap v16025: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:35:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:17 smithi098 ceph-mon[103124]: pgmap v16026: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:35:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:35:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:35:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:35:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:35:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:17 smithi170 ceph-mon[102832]: pgmap v16026: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:35:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:35:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:35:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:35:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:35:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:17 smithi082 ceph-mon[98694]: pgmap v16026: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:35:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:35:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:35:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:35:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:35:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29749.521142 secs 2023-09-10T07:35:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:35:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:35:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:35:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:35:19.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:35:19.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:35:19.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:35:19.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:19 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T07:35:19.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:19 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T07:35:19.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:35:19.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:35:19.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:35:19.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:35:19.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:35:19.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:35:19.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:35:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29749.521142 secs 2023-09-10T07:35:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:35:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:35:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:35:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:35:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:35:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:35:19.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:35:19.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:19 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T07:35:19.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:19 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T07:35:19.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:35:19.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:35:19.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:35:19.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:35:19.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:35:19.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:35:19.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:35:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29749.521142 secs 2023-09-10T07:35:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:35:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:35:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:35:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:35:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:35:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:35:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:35:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:19 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T07:35:19.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:19 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T07:35:19.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:35:19.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:35:19.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:35:19.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:35:19.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:35:19.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:35:19.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:35:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29750.657061 secs 2023-09-10T07:35:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:20 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T07:35:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:20 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T07:35:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29751.133533 secs 2023-09-10T07:35:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:20 smithi170 ceph-mon[102832]: pgmap v16027: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:35:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29750.657061 secs 2023-09-10T07:35:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:20 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T07:35:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:20 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T07:35:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29751.133533 secs 2023-09-10T07:35:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:20 smithi082 ceph-mon[98694]: pgmap v16027: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:35:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29750.657061 secs 2023-09-10T07:35:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:20 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T07:35:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:20 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T07:35:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29751.133533 secs 2023-09-10T07:35:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:20 smithi098 ceph-mon[103124]: pgmap v16027: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:35:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:22 smithi082 ceph-mon[98694]: pgmap v16028: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:35:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:22 smithi098 ceph-mon[103124]: pgmap v16028: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:35:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:22 smithi170 ceph-mon[102832]: pgmap v16028: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:35:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29754.521201 secs 2023-09-10T07:35:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:24 smithi082 ceph-mon[98694]: pgmap v16029: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:35:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29754.521201 secs 2023-09-10T07:35:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:24 smithi098 ceph-mon[103124]: pgmap v16029: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:35:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29754.521201 secs 2023-09-10T07:35:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:24 smithi170 ceph-mon[102832]: pgmap v16029: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:35:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29755.657144 secs 2023-09-10T07:35:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29756.133593 secs 2023-09-10T07:35:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29755.657144 secs 2023-09-10T07:35:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29756.133593 secs 2023-09-10T07:35:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29755.657144 secs 2023-09-10T07:35:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29756.133593 secs 2023-09-10T07:35:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:26 smithi098 ceph-mon[103124]: pgmap v16030: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:35:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:26 smithi170 ceph-mon[102832]: pgmap v16030: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:35:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:26 smithi082 ceph-mon[98694]: pgmap v16030: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:35:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:27 smithi098 ceph-mon[103124]: pgmap v16031: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 99 KiB/s wr, 13 op/s 2023-09-10T07:35:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:27 smithi170 ceph-mon[102832]: pgmap v16031: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 99 KiB/s wr, 13 op/s 2023-09-10T07:35:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:27 smithi082 ceph-mon[98694]: pgmap v16031: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 99 KiB/s wr, 13 op/s 2023-09-10T07:35:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29759.521261 secs 2023-09-10T07:35:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29759.521261 secs 2023-09-10T07:35:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29759.521261 secs 2023-09-10T07:35:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29760.657201 secs 2023-09-10T07:35:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29761.133657 secs 2023-09-10T07:35:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:29 smithi098 ceph-mon[103124]: pgmap v16032: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 9 op/s 2023-09-10T07:35:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29760.657201 secs 2023-09-10T07:35:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29761.133657 secs 2023-09-10T07:35:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:29 smithi170 ceph-mon[102832]: pgmap v16032: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 9 op/s 2023-09-10T07:35:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29760.657201 secs 2023-09-10T07:35:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29761.133657 secs 2023-09-10T07:35:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:29 smithi082 ceph-mon[98694]: pgmap v16032: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 9 op/s 2023-09-10T07:35:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:32 smithi082 ceph-mon[98694]: pgmap v16033: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 153 KiB/s wr, 14 op/s 2023-09-10T07:35:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:32 smithi098 ceph-mon[103124]: pgmap v16033: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 153 KiB/s wr, 14 op/s 2023-09-10T07:35:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:32 smithi170 ceph-mon[102832]: pgmap v16033: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 153 KiB/s wr, 14 op/s 2023-09-10T07:35:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29764.521326 secs 2023-09-10T07:35:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:34 smithi082 ceph-mon[98694]: pgmap v16034: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 9 op/s 2023-09-10T07:35:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29764.521326 secs 2023-09-10T07:35:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:34 smithi098 ceph-mon[103124]: pgmap v16034: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 9 op/s 2023-09-10T07:35:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29764.521326 secs 2023-09-10T07:35:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:34 smithi170 ceph-mon[102832]: pgmap v16034: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 9 op/s 2023-09-10T07:35:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29765.657258 secs 2023-09-10T07:35:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29766.133719 secs 2023-09-10T07:35:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29765.657258 secs 2023-09-10T07:35:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29766.133719 secs 2023-09-10T07:35:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29765.657258 secs 2023-09-10T07:35:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29766.133719 secs 2023-09-10T07:35:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:36 smithi098 ceph-mon[103124]: pgmap v16035: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 9 op/s 2023-09-10T07:35:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:36 smithi170 ceph-mon[102832]: pgmap v16035: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 9 op/s 2023-09-10T07:35:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:36 smithi082 ceph-mon[98694]: pgmap v16035: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 9 op/s 2023-09-10T07:35:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:37 smithi098 ceph-mon[103124]: pgmap v16036: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 329 KiB/s wr, 16 op/s 2023-09-10T07:35:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:37 smithi170 ceph-mon[102832]: pgmap v16036: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 329 KiB/s wr, 16 op/s 2023-09-10T07:35:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:37 smithi082 ceph-mon[98694]: pgmap v16036: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 329 KiB/s wr, 16 op/s 2023-09-10T07:35:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:35:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:35:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:35:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29770.657317 secs 2023-09-10T07:35:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29771.133783 secs 2023-09-10T07:35:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:35:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:35:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:35:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:40 smithi170 ceph-mon[102832]: pgmap v16037: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T07:35:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29770.657317 secs 2023-09-10T07:35:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29771.133783 secs 2023-09-10T07:35:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:35:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:35:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:35:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:40 smithi082 ceph-mon[98694]: pgmap v16037: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T07:35:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29770.657317 secs 2023-09-10T07:35:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29771.133783 secs 2023-09-10T07:35:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:35:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:35:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:35:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:40 smithi098 ceph-mon[103124]: pgmap v16037: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T07:35:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:42 smithi082 ceph-mon[98694]: pgmap v16038: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 469 KiB/s wr, 17 op/s 2023-09-10T07:35:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:42 smithi098 ceph-mon[103124]: pgmap v16038: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 469 KiB/s wr, 17 op/s 2023-09-10T07:35:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:42 smithi170 ceph-mon[102832]: pgmap v16038: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 469 KiB/s wr, 17 op/s 2023-09-10T07:35:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:44 smithi098 ceph-mon[103124]: pgmap v16039: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 414 KiB/s wr, 12 op/s 2023-09-10T07:35:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:44 smithi170 ceph-mon[102832]: pgmap v16039: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 414 KiB/s wr, 12 op/s 2023-09-10T07:35:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:44 smithi082 ceph-mon[98694]: pgmap v16039: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 414 KiB/s wr, 12 op/s 2023-09-10T07:35:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29775.657388 secs 2023-09-10T07:35:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29776.133842 secs 2023-09-10T07:35:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29775.657388 secs 2023-09-10T07:35:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29776.133842 secs 2023-09-10T07:35:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29775.657388 secs 2023-09-10T07:35:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29776.133842 secs 2023-09-10T07:35:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:46 smithi098 ceph-mon[103124]: pgmap v16040: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 414 KiB/s wr, 12 op/s 2023-09-10T07:35:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:46 smithi170 ceph-mon[102832]: pgmap v16040: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 414 KiB/s wr, 12 op/s 2023-09-10T07:35:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:46 smithi082 ceph-mon[98694]: pgmap v16040: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 414 KiB/s wr, 12 op/s 2023-09-10T07:35:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:47 smithi098 ceph-mon[103124]: pgmap v16041: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 673 KiB/s wr, 20 op/s 2023-09-10T07:35:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:47 smithi170 ceph-mon[102832]: pgmap v16041: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 673 KiB/s wr, 20 op/s 2023-09-10T07:35:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:47 smithi082 ceph-mon[98694]: pgmap v16041: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 673 KiB/s wr, 20 op/s 2023-09-10T07:35:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29780.657445 secs 2023-09-10T07:35:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29781.133905 secs 2023-09-10T07:35:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:50 smithi098 ceph-mon[103124]: pgmap v16042: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 498 KiB/s wr, 14 op/s 2023-09-10T07:35:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29780.657445 secs 2023-09-10T07:35:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29781.133905 secs 2023-09-10T07:35:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:50 smithi170 ceph-mon[102832]: pgmap v16042: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 498 KiB/s wr, 14 op/s 2023-09-10T07:35:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29780.657445 secs 2023-09-10T07:35:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29781.133905 secs 2023-09-10T07:35:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:50 smithi082 ceph-mon[98694]: pgmap v16042: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 498 KiB/s wr, 14 op/s 2023-09-10T07:35:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:52 smithi098 ceph-mon[103124]: pgmap v16043: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 638 KiB/s wr, 21 op/s 2023-09-10T07:35:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:52 smithi170 ceph-mon[102832]: pgmap v16043: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 638 KiB/s wr, 21 op/s 2023-09-10T07:35:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:52 smithi082 ceph-mon[98694]: pgmap v16043: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 638 KiB/s wr, 21 op/s 2023-09-10T07:35:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29784.521624 secs 2023-09-10T07:35:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:54 smithi098 ceph-mon[103124]: pgmap v16044: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 399 KiB/s wr, 14 op/s 2023-09-10T07:35:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29784.521624 secs 2023-09-10T07:35:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:54 smithi170 ceph-mon[102832]: pgmap v16044: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 399 KiB/s wr, 14 op/s 2023-09-10T07:35:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29784.521624 secs 2023-09-10T07:35:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:54 smithi082 ceph-mon[98694]: pgmap v16044: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 399 KiB/s wr, 14 op/s 2023-09-10T07:35:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29785.657495 secs 2023-09-10T07:35:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29786.133962 secs 2023-09-10T07:35:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29785.657495 secs 2023-09-10T07:35:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29786.133962 secs 2023-09-10T07:35:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29785.657495 secs 2023-09-10T07:35:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29786.133962 secs 2023-09-10T07:35:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:56 smithi098 ceph-mon[103124]: pgmap v16045: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 399 KiB/s wr, 14 op/s 2023-09-10T07:35:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:35:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:35:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:35:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:56 smithi170 ceph-mon[102832]: pgmap v16045: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 399 KiB/s wr, 14 op/s 2023-09-10T07:35:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:35:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:35:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:35:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:56 smithi082 ceph-mon[98694]: pgmap v16045: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 399 KiB/s wr, 14 op/s 2023-09-10T07:35:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:35:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:35:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:35:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:57 smithi098 ceph-mon[103124]: pgmap v16046: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 544 KiB/s wr, 21 op/s 2023-09-10T07:35:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:57 smithi170 ceph-mon[102832]: pgmap v16046: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 544 KiB/s wr, 21 op/s 2023-09-10T07:35:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:57 smithi082 ceph-mon[98694]: pgmap v16046: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 544 KiB/s wr, 21 op/s 2023-09-10T07:35:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:35:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29789.521729 secs 2023-09-10T07:35:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29789.521729 secs 2023-09-10T07:35:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29789.521729 secs 2023-09-10T07:36:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29790.657517 secs 2023-09-10T07:36:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29791.134026 secs 2023-09-10T07:36:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:00 smithi098 ceph-mon[103124]: pgmap v16047: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 285 KiB/s wr, 13 op/s 2023-09-10T07:36:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29790.657517 secs 2023-09-10T07:36:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29791.134026 secs 2023-09-10T07:36:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:35:59 smithi170 ceph-mon[102832]: pgmap v16047: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 285 KiB/s wr, 13 op/s 2023-09-10T07:36:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29790.657517 secs 2023-09-10T07:36:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29791.134026 secs 2023-09-10T07:36:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:35:59 smithi082 ceph-mon[98694]: pgmap v16047: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 285 KiB/s wr, 13 op/s 2023-09-10T07:36:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:02 smithi098 ceph-mon[103124]: pgmap v16048: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 404 KiB/s wr, 18 op/s 2023-09-10T07:36:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:02 smithi170 ceph-mon[102832]: pgmap v16048: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 404 KiB/s wr, 18 op/s 2023-09-10T07:36:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:02 smithi082 ceph-mon[98694]: pgmap v16048: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 404 KiB/s wr, 18 op/s 2023-09-10T07:36:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29794.521775 secs 2023-09-10T07:36:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:04 smithi098 ceph-mon[103124]: pgmap v16049: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 263 KiB/s wr, 11 op/s 2023-09-10T07:36:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29794.521775 secs 2023-09-10T07:36:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:04 smithi170 ceph-mon[102832]: pgmap v16049: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 263 KiB/s wr, 11 op/s 2023-09-10T07:36:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29794.521775 secs 2023-09-10T07:36:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:04 smithi082 ceph-mon[98694]: pgmap v16049: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 263 KiB/s wr, 11 op/s 2023-09-10T07:36:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29795.657576 secs 2023-09-10T07:36:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29796.134110 secs 2023-09-10T07:36:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29795.657576 secs 2023-09-10T07:36:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29796.134110 secs 2023-09-10T07:36:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29795.657576 secs 2023-09-10T07:36:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29796.134110 secs 2023-09-10T07:36:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:06 smithi098 ceph-mon[103124]: pgmap v16050: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 263 KiB/s wr, 11 op/s 2023-09-10T07:36:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:06 smithi170 ceph-mon[102832]: pgmap v16050: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 263 KiB/s wr, 11 op/s 2023-09-10T07:36:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:06 smithi082 ceph-mon[98694]: pgmap v16050: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 263 KiB/s wr, 11 op/s 2023-09-10T07:36:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:07 smithi098 ceph-mon[103124]: pgmap v16051: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 308 KiB/s wr, 30 op/s 2023-09-10T07:36:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:07 smithi170 ceph-mon[102832]: pgmap v16051: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 308 KiB/s wr, 30 op/s 2023-09-10T07:36:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:07 smithi082 ceph-mon[98694]: pgmap v16051: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 308 KiB/s wr, 30 op/s 2023-09-10T07:36:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29799.521875 secs 2023-09-10T07:36:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:08 smithi098 ceph-mon[103124]: osdmap e1071: 12 total, 12 up, 12 in 2023-09-10T07:36:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29799.521875 secs 2023-09-10T07:36:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:08 smithi170 ceph-mon[102832]: osdmap e1071: 12 total, 12 up, 12 in 2023-09-10T07:36:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29799.521875 secs 2023-09-10T07:36:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:08 smithi082 ceph-mon[98694]: osdmap e1071: 12 total, 12 up, 12 in 2023-09-10T07:36:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29800.657635 secs 2023-09-10T07:36:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29801.134171 secs 2023-09-10T07:36:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:09 smithi098 ceph-mon[103124]: pgmap v16053: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 196 KiB/s wr, 29 op/s 2023-09-10T07:36:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29800.657635 secs 2023-09-10T07:36:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29801.134171 secs 2023-09-10T07:36:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:09 smithi170 ceph-mon[102832]: pgmap v16053: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 196 KiB/s wr, 29 op/s 2023-09-10T07:36:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29800.657635 secs 2023-09-10T07:36:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29801.134171 secs 2023-09-10T07:36:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:09 smithi082 ceph-mon[98694]: pgmap v16053: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 196 KiB/s wr, 29 op/s 2023-09-10T07:36:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:12 smithi098 ceph-mon[103124]: pgmap v16054: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T07:36:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:12 smithi098 ceph-mon[103124]: osdmap e1072: 12 total, 12 up, 12 in 2023-09-10T07:36:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:12 smithi170 ceph-mon[102832]: pgmap v16054: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T07:36:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:12 smithi170 ceph-mon[102832]: osdmap e1072: 12 total, 12 up, 12 in 2023-09-10T07:36:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:12 smithi082 ceph-mon[98694]: pgmap v16054: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2023-09-10T07:36:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:12 smithi082 ceph-mon[98694]: osdmap e1072: 12 total, 12 up, 12 in 2023-09-10T07:36:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29804.521935 secs 2023-09-10T07:36:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:14 smithi098 ceph-mon[103124]: pgmap v16056: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 67 KiB/s wr, 34 op/s 2023-09-10T07:36:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29804.521935 secs 2023-09-10T07:36:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:14 smithi170 ceph-mon[102832]: pgmap v16056: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 67 KiB/s wr, 34 op/s 2023-09-10T07:36:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29804.521935 secs 2023-09-10T07:36:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:14 smithi082 ceph-mon[98694]: pgmap v16056: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 67 KiB/s wr, 34 op/s 2023-09-10T07:36:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29805.657692 secs 2023-09-10T07:36:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29806.134232 secs 2023-09-10T07:36:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29805.657692 secs 2023-09-10T07:36:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29806.134232 secs 2023-09-10T07:36:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29805.657692 secs 2023-09-10T07:36:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29806.134232 secs 2023-09-10T07:36:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:16 smithi098 ceph-mon[103124]: pgmap v16057: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:36:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:16 smithi170 ceph-mon[102832]: pgmap v16057: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:36:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:16 smithi082 ceph-mon[98694]: pgmap v16057: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:36:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:17 smithi098 ceph-mon[103124]: pgmap v16058: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:36:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:17 smithi170 ceph-mon[102832]: pgmap v16058: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:36:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:17 smithi082 ceph-mon[98694]: pgmap v16058: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:36:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29809.521996 secs 2023-09-10T07:36:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:36:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29809.521996 secs 2023-09-10T07:36:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:36:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29809.521996 secs 2023-09-10T07:36:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:36:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29810.657750 secs 2023-09-10T07:36:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29811.134299 secs 2023-09-10T07:36:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:19 smithi098 ceph-mon[103124]: pgmap v16059: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:36:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:36:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:36:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:36:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29810.657750 secs 2023-09-10T07:36:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29811.134299 secs 2023-09-10T07:36:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:19 smithi170 ceph-mon[102832]: pgmap v16059: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:36:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:36:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:36:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:36:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29810.657750 secs 2023-09-10T07:36:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29811.134299 secs 2023-09-10T07:36:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:19 smithi082 ceph-mon[98694]: pgmap v16059: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:36:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:36:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:36:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:36:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:22 smithi098 ceph-mon[103124]: pgmap v16060: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:36:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:22 smithi170 ceph-mon[102832]: pgmap v16060: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:36:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:22 smithi082 ceph-mon[98694]: pgmap v16060: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:36:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29814.522110 secs 2023-09-10T07:36:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:24 smithi098 ceph-mon[103124]: pgmap v16061: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:36:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29814.522110 secs 2023-09-10T07:36:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:24 smithi170 ceph-mon[102832]: pgmap v16061: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:36:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29814.522110 secs 2023-09-10T07:36:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:24 smithi082 ceph-mon[98694]: pgmap v16061: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:36:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29815.657808 secs 2023-09-10T07:36:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29816.134359 secs 2023-09-10T07:36:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29815.657808 secs 2023-09-10T07:36:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29816.134359 secs 2023-09-10T07:36:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29815.657808 secs 2023-09-10T07:36:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29816.134359 secs 2023-09-10T07:36:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:26 smithi098 ceph-mon[103124]: pgmap v16062: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:36:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:26 smithi170 ceph-mon[102832]: pgmap v16062: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:36:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:26 smithi082 ceph-mon[98694]: pgmap v16062: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:36:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:27 smithi098 ceph-mon[103124]: pgmap v16063: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2023-09-10T07:36:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:27 smithi170 ceph-mon[102832]: pgmap v16063: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2023-09-10T07:36:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:27 smithi082 ceph-mon[98694]: pgmap v16063: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2023-09-10T07:36:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29819.522171 secs 2023-09-10T07:36:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29819.522171 secs 2023-09-10T07:36:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29819.522171 secs 2023-09-10T07:36:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29820.657868 secs 2023-09-10T07:36:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29821.134423 secs 2023-09-10T07:36:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:30 smithi170 ceph-mon[102832]: pgmap v16064: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:36:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29820.657868 secs 2023-09-10T07:36:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29821.134423 secs 2023-09-10T07:36:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:30 smithi082 ceph-mon[98694]: pgmap v16064: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:36:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29820.657868 secs 2023-09-10T07:36:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29821.134423 secs 2023-09-10T07:36:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:30 smithi098 ceph-mon[103124]: pgmap v16064: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:36:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:32 smithi098 ceph-mon[103124]: pgmap v16065: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 64 KiB/s wr, 13 op/s 2023-09-10T07:36:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:32 smithi170 ceph-mon[102832]: pgmap v16065: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 64 KiB/s wr, 13 op/s 2023-09-10T07:36:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:32 smithi082 ceph-mon[98694]: pgmap v16065: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 64 KiB/s wr, 13 op/s 2023-09-10T07:36:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29824.522232 secs 2023-09-10T07:36:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:34 smithi098 ceph-mon[103124]: pgmap v16066: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T07:36:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29824.522232 secs 2023-09-10T07:36:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:34 smithi170 ceph-mon[102832]: pgmap v16066: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T07:36:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29824.522232 secs 2023-09-10T07:36:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:34 smithi082 ceph-mon[98694]: pgmap v16066: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T07:36:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29825.657925 secs 2023-09-10T07:36:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29826.134484 secs 2023-09-10T07:36:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29825.657925 secs 2023-09-10T07:36:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29826.134484 secs 2023-09-10T07:36:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29825.657925 secs 2023-09-10T07:36:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29826.134484 secs 2023-09-10T07:36:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:36 smithi098 ceph-mon[103124]: pgmap v16067: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T07:36:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:36 smithi170 ceph-mon[102832]: pgmap v16067: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T07:36:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:36 smithi082 ceph-mon[98694]: pgmap v16067: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 9 op/s 2023-09-10T07:36:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:37 smithi098 ceph-mon[103124]: pgmap v16068: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 234 KiB/s wr, 15 op/s 2023-09-10T07:36:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:37 smithi170 ceph-mon[102832]: pgmap v16068: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 234 KiB/s wr, 15 op/s 2023-09-10T07:36:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:37 smithi082 ceph-mon[98694]: pgmap v16068: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 234 KiB/s wr, 15 op/s 2023-09-10T07:36:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29829.522348 secs 2023-09-10T07:36:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29829.522348 secs 2023-09-10T07:36:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29829.522348 secs 2023-09-10T07:36:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29830.657983 secs 2023-09-10T07:36:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29831.134548 secs 2023-09-10T07:36:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:40 smithi098 ceph-mon[103124]: pgmap v16069: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 234 KiB/s wr, 11 op/s 2023-09-10T07:36:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29830.657983 secs 2023-09-10T07:36:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29831.134548 secs 2023-09-10T07:36:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:40 smithi170 ceph-mon[102832]: pgmap v16069: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 234 KiB/s wr, 11 op/s 2023-09-10T07:36:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29830.657983 secs 2023-09-10T07:36:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29831.134548 secs 2023-09-10T07:36:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:40 smithi082 ceph-mon[98694]: pgmap v16069: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 234 KiB/s wr, 11 op/s 2023-09-10T07:36:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:42 smithi098 ceph-mon[103124]: pgmap v16070: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 457 KiB/s wr, 17 op/s 2023-09-10T07:36:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:42 smithi170 ceph-mon[102832]: pgmap v16070: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 457 KiB/s wr, 17 op/s 2023-09-10T07:36:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:42 smithi082 ceph-mon[98694]: pgmap v16070: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 457 KiB/s wr, 17 op/s 2023-09-10T07:36:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:36:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:36:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:36:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:44 smithi098 ceph-mon[103124]: pgmap v16071: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T07:36:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:44 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:36:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:36:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:36:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:44 smithi170 ceph-mon[102832]: pgmap v16071: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T07:36:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:44 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:36:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:36:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:36:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:44 smithi082 ceph-mon[98694]: pgmap v16071: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T07:36:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:44 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:36:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:36:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:36:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29835.658041 secs 2023-09-10T07:36:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29836.134607 secs 2023-09-10T07:36:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29835.658041 secs 2023-09-10T07:36:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29836.134607 secs 2023-09-10T07:36:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29835.658041 secs 2023-09-10T07:36:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29836.134607 secs 2023-09-10T07:36:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:46 smithi098 ceph-mon[103124]: pgmap v16072: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T07:36:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:46 smithi170 ceph-mon[102832]: pgmap v16072: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T07:36:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:46 smithi082 ceph-mon[98694]: pgmap v16072: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T07:36:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:48 smithi098 ceph-mon[103124]: pgmap v16073: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 671 KiB/s wr, 20 op/s 2023-09-10T07:36:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:47 smithi170 ceph-mon[102832]: pgmap v16073: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 671 KiB/s wr, 20 op/s 2023-09-10T07:36:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:47 smithi082 ceph-mon[98694]: pgmap v16073: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 671 KiB/s wr, 20 op/s 2023-09-10T07:36:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29840.658098 secs 2023-09-10T07:36:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29841.134668 secs 2023-09-10T07:36:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:50 smithi098 ceph-mon[103124]: pgmap v16074: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 501 KiB/s wr, 13 op/s 2023-09-10T07:36:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29840.658098 secs 2023-09-10T07:36:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29841.134668 secs 2023-09-10T07:36:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:50 smithi170 ceph-mon[102832]: pgmap v16074: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 501 KiB/s wr, 13 op/s 2023-09-10T07:36:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29840.658098 secs 2023-09-10T07:36:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29841.134668 secs 2023-09-10T07:36:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:50 smithi082 ceph-mon[98694]: pgmap v16074: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 501 KiB/s wr, 13 op/s 2023-09-10T07:36:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:52 smithi098 ceph-mon[103124]: pgmap v16075: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 642 KiB/s wr, 20 op/s 2023-09-10T07:36:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:52 smithi170 ceph-mon[102832]: pgmap v16075: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 642 KiB/s wr, 20 op/s 2023-09-10T07:36:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:52 smithi082 ceph-mon[98694]: pgmap v16075: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 642 KiB/s wr, 20 op/s 2023-09-10T07:36:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29844.522613 secs 2023-09-10T07:36:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:54 smithi098 ceph-mon[103124]: pgmap v16076: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 419 KiB/s wr, 14 op/s 2023-09-10T07:36:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29844.522613 secs 2023-09-10T07:36:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:54 smithi170 ceph-mon[102832]: pgmap v16076: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 419 KiB/s wr, 14 op/s 2023-09-10T07:36:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29844.522613 secs 2023-09-10T07:36:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:54 smithi082 ceph-mon[98694]: pgmap v16076: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 419 KiB/s wr, 14 op/s 2023-09-10T07:36:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29845.658187 secs 2023-09-10T07:36:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29846.134726 secs 2023-09-10T07:36:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29845.658187 secs 2023-09-10T07:36:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29846.134726 secs 2023-09-10T07:36:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29845.658187 secs 2023-09-10T07:36:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29846.134726 secs 2023-09-10T07:36:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:56 smithi098 ceph-mon[103124]: pgmap v16077: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 419 KiB/s wr, 14 op/s 2023-09-10T07:36:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:36:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:36:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:36:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:56 smithi170 ceph-mon[102832]: pgmap v16077: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 419 KiB/s wr, 14 op/s 2023-09-10T07:36:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:36:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:36:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:36:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:56 smithi082 ceph-mon[98694]: pgmap v16077: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 419 KiB/s wr, 14 op/s 2023-09-10T07:36:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:36:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:36:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:36:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:58 smithi098 ceph-mon[103124]: pgmap v16078: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 565 KiB/s wr, 20 op/s 2023-09-10T07:36:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:58 smithi170 ceph-mon[102832]: pgmap v16078: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 565 KiB/s wr, 20 op/s 2023-09-10T07:36:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:58 smithi082 ceph-mon[98694]: pgmap v16078: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 565 KiB/s wr, 20 op/s 2023-09-10T07:36:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:36:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29849.522673 secs 2023-09-10T07:36:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:36:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29849.522673 secs 2023-09-10T07:36:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:36:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29849.522673 secs 2023-09-10T07:37:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29850.658245 secs 2023-09-10T07:37:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29851.134794 secs 2023-09-10T07:37:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:00 smithi170 ceph-mon[102832]: pgmap v16079: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 287 KiB/s wr, 12 op/s 2023-09-10T07:37:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29850.658245 secs 2023-09-10T07:37:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29851.134794 secs 2023-09-10T07:37:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:00 smithi082 ceph-mon[98694]: pgmap v16079: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 287 KiB/s wr, 12 op/s 2023-09-10T07:37:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29850.658245 secs 2023-09-10T07:37:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29851.134794 secs 2023-09-10T07:37:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:00 smithi098 ceph-mon[103124]: pgmap v16079: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 287 KiB/s wr, 12 op/s 2023-09-10T07:37:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:02 smithi098 ceph-mon[103124]: pgmap v16080: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 403 KiB/s wr, 17 op/s 2023-09-10T07:37:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:02 smithi170 ceph-mon[102832]: pgmap v16080: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 403 KiB/s wr, 17 op/s 2023-09-10T07:37:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:02 smithi082 ceph-mon[98694]: pgmap v16080: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 403 KiB/s wr, 17 op/s 2023-09-10T07:37:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29854.522734 secs 2023-09-10T07:37:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:04 smithi098 ceph-mon[103124]: pgmap v16081: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 261 KiB/s wr, 10 op/s 2023-09-10T07:37:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29854.522734 secs 2023-09-10T07:37:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:04 smithi170 ceph-mon[102832]: pgmap v16081: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 261 KiB/s wr, 10 op/s 2023-09-10T07:37:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29854.522734 secs 2023-09-10T07:37:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:04 smithi082 ceph-mon[98694]: pgmap v16081: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 261 KiB/s wr, 10 op/s 2023-09-10T07:37:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29855.658302 secs 2023-09-10T07:37:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29856.134853 secs 2023-09-10T07:37:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29855.658302 secs 2023-09-10T07:37:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29856.134853 secs 2023-09-10T07:37:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29855.658302 secs 2023-09-10T07:37:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29856.134853 secs 2023-09-10T07:37:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:06 smithi098 ceph-mon[103124]: pgmap v16082: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 261 KiB/s wr, 10 op/s 2023-09-10T07:37:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:06 smithi170 ceph-mon[102832]: pgmap v16082: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 261 KiB/s wr, 10 op/s 2023-09-10T07:37:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:06 smithi082 ceph-mon[98694]: pgmap v16082: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 261 KiB/s wr, 10 op/s 2023-09-10T07:37:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:08 smithi170 ceph-mon[102832]: pgmap v16083: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 306 KiB/s wr, 30 op/s 2023-09-10T07:37:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:08 smithi082 ceph-mon[98694]: pgmap v16083: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 306 KiB/s wr, 30 op/s 2023-09-10T07:37:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:08 smithi098 ceph-mon[103124]: pgmap v16083: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 306 KiB/s wr, 30 op/s 2023-09-10T07:37:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29859.522788 secs 2023-09-10T07:37:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:09 smithi170 ceph-mon[102832]: osdmap e1073: 12 total, 12 up, 12 in 2023-09-10T07:37:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29859.522788 secs 2023-09-10T07:37:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:09 smithi082 ceph-mon[98694]: osdmap e1073: 12 total, 12 up, 12 in 2023-09-10T07:37:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29859.522788 secs 2023-09-10T07:37:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:09 smithi098 ceph-mon[103124]: osdmap e1073: 12 total, 12 up, 12 in 2023-09-10T07:37:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29860.658360 secs 2023-09-10T07:37:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29861.134917 secs 2023-09-10T07:37:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:10 smithi170 ceph-mon[102832]: pgmap v16085: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 193 KiB/s wr, 29 op/s 2023-09-10T07:37:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29860.658360 secs 2023-09-10T07:37:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29861.134917 secs 2023-09-10T07:37:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:10 smithi082 ceph-mon[98694]: pgmap v16085: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 193 KiB/s wr, 29 op/s 2023-09-10T07:37:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29860.658360 secs 2023-09-10T07:37:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29861.134917 secs 2023-09-10T07:37:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:10 smithi098 ceph-mon[103124]: pgmap v16085: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 193 KiB/s wr, 29 op/s 2023-09-10T07:37:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:12 smithi098 ceph-mon[103124]: pgmap v16086: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:37:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:12 smithi098 ceph-mon[103124]: osdmap e1074: 12 total, 12 up, 12 in 2023-09-10T07:37:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:12 smithi170 ceph-mon[102832]: pgmap v16086: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:37:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:12 smithi170 ceph-mon[102832]: osdmap e1074: 12 total, 12 up, 12 in 2023-09-10T07:37:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:12 smithi082 ceph-mon[98694]: pgmap v16086: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:37:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:12 smithi082 ceph-mon[98694]: osdmap e1074: 12 total, 12 up, 12 in 2023-09-10T07:37:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29864.522862 secs 2023-09-10T07:37:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:14 smithi098 ceph-mon[103124]: pgmap v16088: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 68 KiB/s wr, 35 op/s 2023-09-10T07:37:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29864.522862 secs 2023-09-10T07:37:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:14 smithi170 ceph-mon[102832]: pgmap v16088: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 68 KiB/s wr, 35 op/s 2023-09-10T07:37:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29864.522862 secs 2023-09-10T07:37:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:14 smithi082 ceph-mon[98694]: pgmap v16088: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 68 KiB/s wr, 35 op/s 2023-09-10T07:37:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29865.658431 secs 2023-09-10T07:37:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29866.134974 secs 2023-09-10T07:37:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29865.658431 secs 2023-09-10T07:37:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29866.134974 secs 2023-09-10T07:37:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29865.658431 secs 2023-09-10T07:37:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29866.134974 secs 2023-09-10T07:37:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:16 smithi098 ceph-mon[103124]: pgmap v16089: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:37:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:16 smithi170 ceph-mon[102832]: pgmap v16089: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:37:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:16 smithi082 ceph-mon[98694]: pgmap v16089: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:37:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:18 smithi170 ceph-mon[102832]: pgmap v16090: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:37:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:18 smithi082 ceph-mon[98694]: pgmap v16090: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:37:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:18 smithi098 ceph-mon[103124]: pgmap v16090: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:37:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29869.522930 secs 2023-09-10T07:37:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29869.522930 secs 2023-09-10T07:37:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29869.522930 secs 2023-09-10T07:37:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29870.658491 secs 2023-09-10T07:37:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29871.135038 secs 2023-09-10T07:37:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:20 smithi170 ceph-mon[102832]: pgmap v16091: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:37:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:37:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:37:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:37:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29870.658491 secs 2023-09-10T07:37:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29871.135038 secs 2023-09-10T07:37:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:20 smithi082 ceph-mon[98694]: pgmap v16091: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:37:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:37:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:37:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:37:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29870.658491 secs 2023-09-10T07:37:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29871.135038 secs 2023-09-10T07:37:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:20 smithi098 ceph-mon[103124]: pgmap v16091: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:37:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:37:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:37:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:37:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:37:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:37:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:37:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:37:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:37:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:37:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:37:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:37:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:37:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:22 smithi170 ceph-mon[102832]: pgmap v16092: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:37:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:22 smithi082 ceph-mon[98694]: pgmap v16092: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:37:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:22 smithi098 ceph-mon[103124]: pgmap v16092: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:37:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29874.522991 secs 2023-09-10T07:37:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:24 smithi098 ceph-mon[103124]: pgmap v16093: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:37:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29874.522991 secs 2023-09-10T07:37:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:24 smithi170 ceph-mon[102832]: pgmap v16093: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:37:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29874.522991 secs 2023-09-10T07:37:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:24 smithi082 ceph-mon[98694]: pgmap v16093: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:37:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29875.658552 secs 2023-09-10T07:37:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29876.135099 secs 2023-09-10T07:37:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29875.658552 secs 2023-09-10T07:37:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29876.135099 secs 2023-09-10T07:37:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29875.658552 secs 2023-09-10T07:37:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29876.135099 secs 2023-09-10T07:37:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:26 smithi098 ceph-mon[103124]: pgmap v16094: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:37:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:26 smithi170 ceph-mon[102832]: pgmap v16094: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:37:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:26 smithi082 ceph-mon[98694]: pgmap v16094: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:37:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:28 smithi170 ceph-mon[102832]: pgmap v16095: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2023-09-10T07:37:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:28 smithi082 ceph-mon[98694]: pgmap v16095: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2023-09-10T07:37:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:28 smithi098 ceph-mon[103124]: pgmap v16095: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2023-09-10T07:37:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29879.523049 secs 2023-09-10T07:37:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29879.523049 secs 2023-09-10T07:37:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29879.523049 secs 2023-09-10T07:37:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29880.658611 secs 2023-09-10T07:37:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29881.135158 secs 2023-09-10T07:37:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:30 smithi170 ceph-mon[102832]: pgmap v16096: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:37:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29880.658611 secs 2023-09-10T07:37:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29881.135158 secs 2023-09-10T07:37:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:30 smithi082 ceph-mon[98694]: pgmap v16096: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:37:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29880.658611 secs 2023-09-10T07:37:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29881.135158 secs 2023-09-10T07:37:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:30 smithi098 ceph-mon[103124]: pgmap v16096: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:37:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:32 smithi098 ceph-mon[103124]: pgmap v16097: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 71 KiB/s wr, 13 op/s 2023-09-10T07:37:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:32 smithi170 ceph-mon[102832]: pgmap v16097: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 71 KiB/s wr, 13 op/s 2023-09-10T07:37:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:32 smithi082 ceph-mon[98694]: pgmap v16097: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 71 KiB/s wr, 13 op/s 2023-09-10T07:37:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29884.523110 secs 2023-09-10T07:37:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:34 smithi098 ceph-mon[103124]: pgmap v16098: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T07:37:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29884.523110 secs 2023-09-10T07:37:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:34 smithi170 ceph-mon[102832]: pgmap v16098: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T07:37:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29884.523110 secs 2023-09-10T07:37:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:34 smithi082 ceph-mon[98694]: pgmap v16098: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T07:37:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29885.658667 secs 2023-09-10T07:37:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29886.135219 secs 2023-09-10T07:37:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29885.658667 secs 2023-09-10T07:37:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29886.135219 secs 2023-09-10T07:37:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29885.658667 secs 2023-09-10T07:37:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29886.135219 secs 2023-09-10T07:37:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:36 smithi098 ceph-mon[103124]: pgmap v16099: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T07:37:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:36 smithi170 ceph-mon[102832]: pgmap v16099: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T07:37:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:36 smithi082 ceph-mon[98694]: pgmap v16099: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 9 op/s 2023-09-10T07:37:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:38 smithi170 ceph-mon[102832]: pgmap v16100: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 232 KiB/s wr, 15 op/s 2023-09-10T07:37:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:38 smithi082 ceph-mon[98694]: pgmap v16100: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 232 KiB/s wr, 15 op/s 2023-09-10T07:37:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:38 smithi098 ceph-mon[103124]: pgmap v16100: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 232 KiB/s wr, 15 op/s 2023-09-10T07:37:39.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:39 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29889.523220 secs 2023-09-10T07:37:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:39 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29889.523220 secs 2023-09-10T07:37:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:39 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29889.523220 secs 2023-09-10T07:37:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29890.658725 secs 2023-09-10T07:37:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29891.135284 secs 2023-09-10T07:37:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:40 smithi170 ceph-mon[102832]: pgmap v16101: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 232 KiB/s wr, 11 op/s 2023-09-10T07:37:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29890.658725 secs 2023-09-10T07:37:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29891.135284 secs 2023-09-10T07:37:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:40 smithi082 ceph-mon[98694]: pgmap v16101: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 232 KiB/s wr, 11 op/s 2023-09-10T07:37:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29890.658725 secs 2023-09-10T07:37:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29891.135284 secs 2023-09-10T07:37:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:40 smithi098 ceph-mon[103124]: pgmap v16101: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 232 KiB/s wr, 11 op/s 2023-09-10T07:37:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:42 smithi098 ceph-mon[103124]: pgmap v16102: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 459 KiB/s wr, 17 op/s 2023-09-10T07:37:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:42 smithi170 ceph-mon[102832]: pgmap v16102: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 459 KiB/s wr, 17 op/s 2023-09-10T07:37:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:42 smithi082 ceph-mon[98694]: pgmap v16102: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 459 KiB/s wr, 17 op/s 2023-09-10T07:37:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:37:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:37:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:37:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:44 smithi098 ceph-mon[103124]: pgmap v16103: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 388 KiB/s wr, 12 op/s 2023-09-10T07:37:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:44 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:37:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:37:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:37:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:44 smithi170 ceph-mon[102832]: pgmap v16103: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 388 KiB/s wr, 12 op/s 2023-09-10T07:37:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:44 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:37:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:37:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:37:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:44 smithi082 ceph-mon[98694]: pgmap v16103: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 388 KiB/s wr, 12 op/s 2023-09-10T07:37:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:44 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:37:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:37:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:37:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29895.658784 secs 2023-09-10T07:37:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29896.135341 secs 2023-09-10T07:37:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29895.658784 secs 2023-09-10T07:37:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29896.135341 secs 2023-09-10T07:37:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29895.658784 secs 2023-09-10T07:37:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29896.135341 secs 2023-09-10T07:37:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:46 smithi170 ceph-mon[102832]: pgmap v16104: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 388 KiB/s wr, 12 op/s 2023-09-10T07:37:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:46 smithi082 ceph-mon[98694]: pgmap v16104: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 388 KiB/s wr, 12 op/s 2023-09-10T07:37:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:46 smithi098 ceph-mon[103124]: pgmap v16104: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 388 KiB/s wr, 12 op/s 2023-09-10T07:37:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:48 smithi170 ceph-mon[102832]: pgmap v16105: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 664 KiB/s wr, 20 op/s 2023-09-10T07:37:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:48 smithi082 ceph-mon[98694]: pgmap v16105: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 664 KiB/s wr, 20 op/s 2023-09-10T07:37:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:48 smithi098 ceph-mon[103124]: pgmap v16105: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 664 KiB/s wr, 20 op/s 2023-09-10T07:37:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29900.658842 secs 2023-09-10T07:37:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29901.135403 secs 2023-09-10T07:37:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:50 smithi098 ceph-mon[103124]: pgmap v16106: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 502 KiB/s wr, 13 op/s 2023-09-10T07:37:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29900.658842 secs 2023-09-10T07:37:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29901.135403 secs 2023-09-10T07:37:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:50 smithi170 ceph-mon[102832]: pgmap v16106: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 502 KiB/s wr, 13 op/s 2023-09-10T07:37:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29900.658842 secs 2023-09-10T07:37:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29901.135403 secs 2023-09-10T07:37:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:50 smithi082 ceph-mon[98694]: pgmap v16106: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 502 KiB/s wr, 13 op/s 2023-09-10T07:37:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:52 smithi098 ceph-mon[103124]: pgmap v16107: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 644 KiB/s wr, 20 op/s 2023-09-10T07:37:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:52 smithi170 ceph-mon[102832]: pgmap v16107: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 644 KiB/s wr, 20 op/s 2023-09-10T07:37:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:52 smithi082 ceph-mon[98694]: pgmap v16107: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 644 KiB/s wr, 20 op/s 2023-09-10T07:37:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29904.523441 secs 2023-09-10T07:37:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:54 smithi170 ceph-mon[102832]: pgmap v16108: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 418 KiB/s wr, 14 op/s 2023-09-10T07:37:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29904.523441 secs 2023-09-10T07:37:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:54 smithi082 ceph-mon[98694]: pgmap v16108: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 418 KiB/s wr, 14 op/s 2023-09-10T07:37:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29904.523441 secs 2023-09-10T07:37:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:54 smithi098 ceph-mon[103124]: pgmap v16108: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 418 KiB/s wr, 14 op/s 2023-09-10T07:37:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29905.658902 secs 2023-09-10T07:37:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29906.135464 secs 2023-09-10T07:37:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29905.658902 secs 2023-09-10T07:37:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29906.135464 secs 2023-09-10T07:37:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29905.658902 secs 2023-09-10T07:37:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29906.135464 secs 2023-09-10T07:37:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:56 smithi170 ceph-mon[102832]: pgmap v16109: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 418 KiB/s wr, 14 op/s 2023-09-10T07:37:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:37:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:37:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:37:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:56 smithi082 ceph-mon[98694]: pgmap v16109: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 418 KiB/s wr, 14 op/s 2023-09-10T07:37:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:37:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:37:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:37:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:56 smithi098 ceph-mon[103124]: pgmap v16109: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 418 KiB/s wr, 14 op/s 2023-09-10T07:37:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:37:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:37:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:37:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:58 smithi170 ceph-mon[102832]: pgmap v16110: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 564 KiB/s wr, 20 op/s 2023-09-10T07:37:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:58 smithi082 ceph-mon[98694]: pgmap v16110: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 564 KiB/s wr, 20 op/s 2023-09-10T07:37:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:58 smithi098 ceph-mon[103124]: pgmap v16110: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 564 KiB/s wr, 20 op/s 2023-09-10T07:37:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:37:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29909.523502 secs 2023-09-10T07:37:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:37:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29909.523502 secs 2023-09-10T07:37:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:37:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29909.523502 secs 2023-09-10T07:38:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29910.658959 secs 2023-09-10T07:38:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29911.135490 secs 2023-09-10T07:38:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:00 smithi170 ceph-mon[102832]: pgmap v16111: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 288 KiB/s wr, 12 op/s 2023-09-10T07:38:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29910.658959 secs 2023-09-10T07:38:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29911.135490 secs 2023-09-10T07:38:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:00 smithi082 ceph-mon[98694]: pgmap v16111: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 288 KiB/s wr, 12 op/s 2023-09-10T07:38:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29910.658959 secs 2023-09-10T07:38:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29911.135490 secs 2023-09-10T07:38:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:00 smithi098 ceph-mon[103124]: pgmap v16111: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 288 KiB/s wr, 12 op/s 2023-09-10T07:38:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:02 smithi098 ceph-mon[103124]: pgmap v16112: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 438 KiB/s wr, 31 op/s 2023-09-10T07:38:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:02 smithi170 ceph-mon[102832]: pgmap v16112: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 438 KiB/s wr, 31 op/s 2023-09-10T07:38:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:02 smithi082 ceph-mon[98694]: pgmap v16112: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 438 KiB/s wr, 31 op/s 2023-09-10T07:38:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29914.523562 secs 2023-09-10T07:38:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:04 smithi098 ceph-mon[103124]: pgmap v16113: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 296 KiB/s wr, 24 op/s 2023-09-10T07:38:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29914.523562 secs 2023-09-10T07:38:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:04 smithi170 ceph-mon[102832]: pgmap v16113: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 296 KiB/s wr, 24 op/s 2023-09-10T07:38:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29914.523562 secs 2023-09-10T07:38:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:04 smithi082 ceph-mon[98694]: pgmap v16113: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 296 KiB/s wr, 24 op/s 2023-09-10T07:38:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29915.659048 secs 2023-09-10T07:38:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29916.135551 secs 2023-09-10T07:38:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29915.659048 secs 2023-09-10T07:38:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29916.135551 secs 2023-09-10T07:38:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29915.659048 secs 2023-09-10T07:38:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29916.135551 secs 2023-09-10T07:38:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:06 smithi170 ceph-mon[102832]: pgmap v16114: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 296 KiB/s wr, 24 op/s 2023-09-10T07:38:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:06 smithi082 ceph-mon[98694]: pgmap v16114: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 296 KiB/s wr, 24 op/s 2023-09-10T07:38:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:06 smithi098 ceph-mon[103124]: pgmap v16114: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 296 KiB/s wr, 24 op/s 2023-09-10T07:38:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:08 smithi170 ceph-mon[102832]: pgmap v16115: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 307 KiB/s wr, 30 op/s 2023-09-10T07:38:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:08 smithi082 ceph-mon[98694]: pgmap v16115: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 307 KiB/s wr, 30 op/s 2023-09-10T07:38:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:08 smithi098 ceph-mon[103124]: pgmap v16115: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 307 KiB/s wr, 30 op/s 2023-09-10T07:38:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29919.523669 secs 2023-09-10T07:38:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:09 smithi170 ceph-mon[102832]: osdmap e1075: 12 total, 12 up, 12 in 2023-09-10T07:38:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29919.523669 secs 2023-09-10T07:38:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:09 smithi082 ceph-mon[98694]: osdmap e1075: 12 total, 12 up, 12 in 2023-09-10T07:38:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29919.523669 secs 2023-09-10T07:38:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:09 smithi098 ceph-mon[103124]: osdmap e1075: 12 total, 12 up, 12 in 2023-09-10T07:38:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29920.659125 secs 2023-09-10T07:38:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29921.135616 secs 2023-09-10T07:38:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:10 smithi170 ceph-mon[102832]: pgmap v16117: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 194 KiB/s wr, 29 op/s 2023-09-10T07:38:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29920.659125 secs 2023-09-10T07:38:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29921.135616 secs 2023-09-10T07:38:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:10 smithi082 ceph-mon[98694]: pgmap v16117: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 194 KiB/s wr, 29 op/s 2023-09-10T07:38:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29920.659125 secs 2023-09-10T07:38:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29921.135616 secs 2023-09-10T07:38:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:10 smithi098 ceph-mon[103124]: pgmap v16117: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 194 KiB/s wr, 29 op/s 2023-09-10T07:38:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:12 smithi098 ceph-mon[103124]: pgmap v16118: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T07:38:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:12 smithi098 ceph-mon[103124]: osdmap e1076: 12 total, 12 up, 12 in 2023-09-10T07:38:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:12 smithi170 ceph-mon[102832]: pgmap v16118: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T07:38:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:12 smithi170 ceph-mon[102832]: osdmap e1076: 12 total, 12 up, 12 in 2023-09-10T07:38:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:12 smithi082 ceph-mon[98694]: pgmap v16118: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-10T07:38:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:12 smithi082 ceph-mon[98694]: osdmap e1076: 12 total, 12 up, 12 in 2023-09-10T07:38:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29924.523808 secs 2023-09-10T07:38:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:14 smithi170 ceph-mon[102832]: pgmap v16120: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 17 KiB/s wr, 14 op/s 2023-09-10T07:38:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29924.523808 secs 2023-09-10T07:38:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:14 smithi082 ceph-mon[98694]: pgmap v16120: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 17 KiB/s wr, 14 op/s 2023-09-10T07:38:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29924.523808 secs 2023-09-10T07:38:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:14 smithi098 ceph-mon[103124]: pgmap v16120: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 17 KiB/s wr, 14 op/s 2023-09-10T07:38:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29925.659188 secs 2023-09-10T07:38:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29926.135679 secs 2023-09-10T07:38:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29925.659188 secs 2023-09-10T07:38:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29926.135679 secs 2023-09-10T07:38:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29925.659188 secs 2023-09-10T07:38:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29926.135679 secs 2023-09-10T07:38:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:16 smithi170 ceph-mon[102832]: pgmap v16121: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:38:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:16 smithi082 ceph-mon[98694]: pgmap v16121: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:38:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:16 smithi098 ceph-mon[103124]: pgmap v16121: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:38:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:18 smithi170 ceph-mon[102832]: pgmap v16122: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:38:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:18 smithi082 ceph-mon[98694]: pgmap v16122: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:38:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:18 smithi098 ceph-mon[103124]: pgmap v16122: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:38:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29929.523869 secs 2023-09-10T07:38:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29929.523869 secs 2023-09-10T07:38:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29929.523869 secs 2023-09-10T07:38:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29930.659245 secs 2023-09-10T07:38:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29931.135744 secs 2023-09-10T07:38:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:20 smithi170 ceph-mon[102832]: pgmap v16123: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:38:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29930.659245 secs 2023-09-10T07:38:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29931.135744 secs 2023-09-10T07:38:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:20 smithi082 ceph-mon[98694]: pgmap v16123: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:38:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29930.659245 secs 2023-09-10T07:38:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29931.135744 secs 2023-09-10T07:38:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:20 smithi098 ceph-mon[103124]: pgmap v16123: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:38:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:38:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:38:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:38:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:38:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:38:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:38:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:38:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:38:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:38:21.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:38:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:38:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:38:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:38:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:38:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:38:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:22 smithi170 ceph-mon[102832]: pgmap v16124: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:38:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:38:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:38:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:38:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:22 smithi082 ceph-mon[98694]: pgmap v16124: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:38:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:38:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:38:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:38:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:22 smithi098 ceph-mon[103124]: pgmap v16124: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:38:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:38:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:38:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:38:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29934.523949 secs 2023-09-10T07:38:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:24 smithi170 ceph-mon[102832]: pgmap v16125: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 183 B/s wr, 9 op/s 2023-09-10T07:38:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29934.523949 secs 2023-09-10T07:38:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:24 smithi082 ceph-mon[98694]: pgmap v16125: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 183 B/s wr, 9 op/s 2023-09-10T07:38:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29934.523949 secs 2023-09-10T07:38:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:24 smithi098 ceph-mon[103124]: pgmap v16125: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 183 B/s wr, 9 op/s 2023-09-10T07:38:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29935.659302 secs 2023-09-10T07:38:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29936.135802 secs 2023-09-10T07:38:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29935.659302 secs 2023-09-10T07:38:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29936.135802 secs 2023-09-10T07:38:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29935.659302 secs 2023-09-10T07:38:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29936.135802 secs 2023-09-10T07:38:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:26 smithi170 ceph-mon[102832]: pgmap v16126: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-10T07:38:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:26 smithi082 ceph-mon[98694]: pgmap v16126: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-10T07:38:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:26 smithi098 ceph-mon[103124]: pgmap v16126: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-10T07:38:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:28 smithi170 ceph-mon[102832]: pgmap v16127: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 13 op/s 2023-09-10T07:38:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:28 smithi082 ceph-mon[98694]: pgmap v16127: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 13 op/s 2023-09-10T07:38:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:28 smithi098 ceph-mon[103124]: pgmap v16127: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 13 op/s 2023-09-10T07:38:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29939.524010 secs 2023-09-10T07:38:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29939.524010 secs 2023-09-10T07:38:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29939.524010 secs 2023-09-10T07:38:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29940.659361 secs 2023-09-10T07:38:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29941.135865 secs 2023-09-10T07:38:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:30 smithi170 ceph-mon[102832]: pgmap v16128: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-10T07:38:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29940.659361 secs 2023-09-10T07:38:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29941.135865 secs 2023-09-10T07:38:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:30 smithi082 ceph-mon[98694]: pgmap v16128: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-10T07:38:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29940.659361 secs 2023-09-10T07:38:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29941.135865 secs 2023-09-10T07:38:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:30 smithi098 ceph-mon[103124]: pgmap v16128: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2023-09-10T07:38:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:32 smithi170 ceph-mon[102832]: pgmap v16129: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 105 KiB/s wr, 14 op/s 2023-09-10T07:38:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:32 smithi082 ceph-mon[98694]: pgmap v16129: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 105 KiB/s wr, 14 op/s 2023-09-10T07:38:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:32 smithi098 ceph-mon[103124]: pgmap v16129: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 105 KiB/s wr, 14 op/s 2023-09-10T07:38:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:34 smithi170 ceph-mon[102832]: pgmap v16130: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 9 op/s 2023-09-10T07:38:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:34 smithi082 ceph-mon[98694]: pgmap v16130: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 9 op/s 2023-09-10T07:38:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:34 smithi098 ceph-mon[103124]: pgmap v16130: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 9 op/s 2023-09-10T07:38:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29945.659420 secs 2023-09-10T07:38:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29946.135927 secs 2023-09-10T07:38:35.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:38:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29945.659420 secs 2023-09-10T07:38:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29946.135927 secs 2023-09-10T07:38:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:38:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29945.659420 secs 2023-09-10T07:38:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29946.135927 secs 2023-09-10T07:38:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:38:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:36 smithi170 ceph-mon[102832]: pgmap v16131: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 9 op/s 2023-09-10T07:38:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:38:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:38:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:38:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:36 smithi082 ceph-mon[98694]: pgmap v16131: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 9 op/s 2023-09-10T07:38:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:38:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:38:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:38:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:36 smithi098 ceph-mon[103124]: pgmap v16131: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 9 op/s 2023-09-10T07:38:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:38:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:38:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:38:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:38 smithi170 ceph-mon[102832]: pgmap v16132: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 207 KiB/s wr, 15 op/s 2023-09-10T07:38:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:38 smithi082 ceph-mon[98694]: pgmap v16132: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 207 KiB/s wr, 15 op/s 2023-09-10T07:38:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:38 smithi098 ceph-mon[103124]: pgmap v16132: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 207 KiB/s wr, 15 op/s 2023-09-10T07:38:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:39 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29949.524170 secs 2023-09-10T07:38:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:39 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29949.524170 secs 2023-09-10T07:38:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:39 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29949.524170 secs 2023-09-10T07:38:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29950.659481 secs 2023-09-10T07:38:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29951.135991 secs 2023-09-10T07:38:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:38:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:38:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:40 smithi170 ceph-mon[102832]: pgmap v16133: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 207 KiB/s wr, 10 op/s 2023-09-10T07:38:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29950.659481 secs 2023-09-10T07:38:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29951.135991 secs 2023-09-10T07:38:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:38:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:38:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:40 smithi082 ceph-mon[98694]: pgmap v16133: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 207 KiB/s wr, 10 op/s 2023-09-10T07:38:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29950.659481 secs 2023-09-10T07:38:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29951.135991 secs 2023-09-10T07:38:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:38:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:38:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:40 smithi098 ceph-mon[103124]: pgmap v16133: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 207 KiB/s wr, 10 op/s 2023-09-10T07:38:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:42 smithi170 ceph-mon[102832]: pgmap v16134: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 438 KiB/s wr, 16 op/s 2023-09-10T07:38:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:42 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:38:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:42 smithi082 ceph-mon[98694]: pgmap v16134: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 438 KiB/s wr, 16 op/s 2023-09-10T07:38:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:42 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:38:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:42 smithi098 ceph-mon[103124]: pgmap v16134: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 438 KiB/s wr, 16 op/s 2023-09-10T07:38:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:42 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:38:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:44 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29954.524237 secs 2023-09-10T07:38:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:44 smithi170 ceph-mon[102832]: pgmap v16135: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 334 KiB/s wr, 11 op/s 2023-09-10T07:38:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:44 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29954.524237 secs 2023-09-10T07:38:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:44 smithi082 ceph-mon[98694]: pgmap v16135: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 334 KiB/s wr, 11 op/s 2023-09-10T07:38:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:44 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29954.524237 secs 2023-09-10T07:38:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:44 smithi098 ceph-mon[103124]: pgmap v16135: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 334 KiB/s wr, 11 op/s 2023-09-10T07:38:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29955.659504 secs 2023-09-10T07:38:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29956.136047 secs 2023-09-10T07:38:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29955.659504 secs 2023-09-10T07:38:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29956.136047 secs 2023-09-10T07:38:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29955.659504 secs 2023-09-10T07:38:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29956.136047 secs 2023-09-10T07:38:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:46 smithi170 ceph-mon[102832]: pgmap v16136: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 334 KiB/s wr, 11 op/s 2023-09-10T07:38:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:46 smithi082 ceph-mon[98694]: pgmap v16136: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 334 KiB/s wr, 11 op/s 2023-09-10T07:38:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:46 smithi098 ceph-mon[103124]: pgmap v16136: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 334 KiB/s wr, 11 op/s 2023-09-10T07:38:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:48 smithi170 ceph-mon[102832]: pgmap v16137: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 482 KiB/s wr, 18 op/s 2023-09-10T07:38:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:48 smithi082 ceph-mon[98694]: pgmap v16137: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 482 KiB/s wr, 18 op/s 2023-09-10T07:38:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:48 smithi098 ceph-mon[103124]: pgmap v16137: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 482 KiB/s wr, 18 op/s 2023-09-10T07:38:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29960.659588 secs 2023-09-10T07:38:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29961.136110 secs 2023-09-10T07:38:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:50 smithi170 ceph-mon[102832]: pgmap v16138: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 379 KiB/s wr, 13 op/s 2023-09-10T07:38:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29960.659588 secs 2023-09-10T07:38:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29961.136110 secs 2023-09-10T07:38:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:50 smithi082 ceph-mon[98694]: pgmap v16138: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 379 KiB/s wr, 13 op/s 2023-09-10T07:38:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29960.659588 secs 2023-09-10T07:38:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29961.136110 secs 2023-09-10T07:38:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:50 smithi098 ceph-mon[103124]: pgmap v16138: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 379 KiB/s wr, 13 op/s 2023-09-10T07:38:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:51 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:38:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:51 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:38:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:51 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:38:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:52 smithi170 ceph-mon[102832]: pgmap v16139: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 650 KiB/s wr, 21 op/s 2023-09-10T07:38:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:52 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:38:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:38:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:38:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:52 smithi082 ceph-mon[98694]: pgmap v16139: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 650 KiB/s wr, 21 op/s 2023-09-10T07:38:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:52 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:38:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:38:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:38:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:52 smithi098 ceph-mon[103124]: pgmap v16139: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 650 KiB/s wr, 21 op/s 2023-09-10T07:38:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:52 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:38:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:38:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:38:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:54 smithi170 ceph-mon[102832]: pgmap v16140: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 418 KiB/s wr, 15 op/s 2023-09-10T07:38:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:54 smithi082 ceph-mon[98694]: pgmap v16140: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 418 KiB/s wr, 15 op/s 2023-09-10T07:38:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:54 smithi098 ceph-mon[103124]: pgmap v16140: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 418 KiB/s wr, 15 op/s 2023-09-10T07:38:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29965.659647 secs 2023-09-10T07:38:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29966.136168 secs 2023-09-10T07:38:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29965.659647 secs 2023-09-10T07:38:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29966.136168 secs 2023-09-10T07:38:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29965.659647 secs 2023-09-10T07:38:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29966.136168 secs 2023-09-10T07:38:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:56 smithi170 ceph-mon[102832]: pgmap v16141: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 418 KiB/s wr, 15 op/s 2023-09-10T07:38:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:56 smithi082 ceph-mon[98694]: pgmap v16141: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 418 KiB/s wr, 15 op/s 2023-09-10T07:38:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:56 smithi098 ceph-mon[103124]: pgmap v16141: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 418 KiB/s wr, 15 op/s 2023-09-10T07:38:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:58 smithi170 ceph-mon[102832]: pgmap v16142: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 498 KiB/s wr, 22 op/s 2023-09-10T07:38:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:58 smithi082 ceph-mon[98694]: pgmap v16142: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 498 KiB/s wr, 22 op/s 2023-09-10T07:38:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:58 smithi098 ceph-mon[103124]: pgmap v16142: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 498 KiB/s wr, 22 op/s 2023-09-10T07:38:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:38:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29969.524444 secs 2023-09-10T07:38:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:38:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29969.524444 secs 2023-09-10T07:38:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:38:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29969.524444 secs 2023-09-10T07:39:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29970.659705 secs 2023-09-10T07:39:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29971.136236 secs 2023-09-10T07:39:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:39:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:39:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:39:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:00 smithi170 ceph-mon[102832]: pgmap v16143: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 350 KiB/s wr, 14 op/s 2023-09-10T07:39:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29970.659705 secs 2023-09-10T07:39:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29971.136236 secs 2023-09-10T07:39:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:39:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:39:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:39:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:00 smithi082 ceph-mon[98694]: pgmap v16143: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 350 KiB/s wr, 14 op/s 2023-09-10T07:39:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29970.659705 secs 2023-09-10T07:39:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29971.136236 secs 2023-09-10T07:39:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:39:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:39:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:39:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:00 smithi098 ceph-mon[103124]: pgmap v16143: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 350 KiB/s wr, 14 op/s 2023-09-10T07:39:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:02 smithi170 ceph-mon[102832]: pgmap v16144: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 604 KiB/s wr, 30 op/s 2023-09-10T07:39:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:02 smithi082 ceph-mon[98694]: pgmap v16144: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 604 KiB/s wr, 30 op/s 2023-09-10T07:39:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:02 smithi098 ceph-mon[103124]: pgmap v16144: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 604 KiB/s wr, 30 op/s 2023-09-10T07:39:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29974.524518 secs 2023-09-10T07:39:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:04 smithi170 ceph-mon[102832]: pgmap v16145: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 334 KiB/s wr, 22 op/s 2023-09-10T07:39:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29974.524518 secs 2023-09-10T07:39:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:04 smithi082 ceph-mon[98694]: pgmap v16145: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 334 KiB/s wr, 22 op/s 2023-09-10T07:39:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29974.524518 secs 2023-09-10T07:39:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:04 smithi098 ceph-mon[103124]: pgmap v16145: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 334 KiB/s wr, 22 op/s 2023-09-10T07:39:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29975.659762 secs 2023-09-10T07:39:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29976.136294 secs 2023-09-10T07:39:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29975.659762 secs 2023-09-10T07:39:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29976.136294 secs 2023-09-10T07:39:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29975.659762 secs 2023-09-10T07:39:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29976.136294 secs 2023-09-10T07:39:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:06 smithi170 ceph-mon[102832]: pgmap v16146: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 334 KiB/s wr, 22 op/s 2023-09-10T07:39:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:06 smithi082 ceph-mon[98694]: pgmap v16146: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 334 KiB/s wr, 22 op/s 2023-09-10T07:39:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:06 smithi098 ceph-mon[103124]: pgmap v16146: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 334 KiB/s wr, 22 op/s 2023-09-10T07:39:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:08 smithi170 ceph-mon[102832]: pgmap v16147: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 450 KiB/s wr, 32 op/s 2023-09-10T07:39:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:08 smithi082 ceph-mon[98694]: pgmap v16147: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 450 KiB/s wr, 32 op/s 2023-09-10T07:39:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:08 smithi098 ceph-mon[103124]: pgmap v16147: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 450 KiB/s wr, 32 op/s 2023-09-10T07:39:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29979.524579 secs 2023-09-10T07:39:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:09 smithi170 ceph-mon[102832]: osdmap e1077: 12 total, 12 up, 12 in 2023-09-10T07:39:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29979.524579 secs 2023-09-10T07:39:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:09 smithi082 ceph-mon[98694]: osdmap e1077: 12 total, 12 up, 12 in 2023-09-10T07:39:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29979.524579 secs 2023-09-10T07:39:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:09 smithi098 ceph-mon[103124]: osdmap e1077: 12 total, 12 up, 12 in 2023-09-10T07:39:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29980.659821 secs 2023-09-10T07:39:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29981.136357 secs 2023-09-10T07:39:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:10 smithi170 ceph-mon[102832]: pgmap v16149: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 445 KiB/s wr, 30 op/s 2023-09-10T07:39:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29980.659821 secs 2023-09-10T07:39:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29981.136357 secs 2023-09-10T07:39:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:10 smithi082 ceph-mon[98694]: pgmap v16149: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 445 KiB/s wr, 30 op/s 2023-09-10T07:39:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29980.659821 secs 2023-09-10T07:39:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29981.136357 secs 2023-09-10T07:39:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:10 smithi098 ceph-mon[103124]: pgmap v16149: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 445 KiB/s wr, 30 op/s 2023-09-10T07:39:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:12 smithi170 ceph-mon[102832]: pgmap v16150: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 140 KiB/s wr, 18 op/s 2023-09-10T07:39:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:12 smithi170 ceph-mon[102832]: osdmap e1078: 12 total, 12 up, 12 in 2023-09-10T07:39:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:12 smithi082 ceph-mon[98694]: pgmap v16150: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 140 KiB/s wr, 18 op/s 2023-09-10T07:39:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:12 smithi082 ceph-mon[98694]: osdmap e1078: 12 total, 12 up, 12 in 2023-09-10T07:39:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:12 smithi098 ceph-mon[103124]: pgmap v16150: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 140 KiB/s wr, 18 op/s 2023-09-10T07:39:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:12 smithi098 ceph-mon[103124]: osdmap e1078: 12 total, 12 up, 12 in 2023-09-10T07:39:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29984.524638 secs 2023-09-10T07:39:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:14 smithi170 ceph-mon[102832]: pgmap v16152: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 175 KiB/s wr, 22 op/s 2023-09-10T07:39:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29984.524638 secs 2023-09-10T07:39:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:14 smithi082 ceph-mon[98694]: pgmap v16152: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 175 KiB/s wr, 22 op/s 2023-09-10T07:39:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29984.524638 secs 2023-09-10T07:39:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:14 smithi098 ceph-mon[103124]: pgmap v16152: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 175 KiB/s wr, 22 op/s 2023-09-10T07:39:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29985.659905 secs 2023-09-10T07:39:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29986.136433 secs 2023-09-10T07:39:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29985.659905 secs 2023-09-10T07:39:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29986.136433 secs 2023-09-10T07:39:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29985.659905 secs 2023-09-10T07:39:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29986.136433 secs 2023-09-10T07:39:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:16 smithi170 ceph-mon[102832]: pgmap v16153: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:39:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:16 smithi082 ceph-mon[98694]: pgmap v16153: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:39:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:16 smithi098 ceph-mon[103124]: pgmap v16153: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:39:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:18 smithi170 ceph-mon[102832]: pgmap v16154: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 12 op/s 2023-09-10T07:39:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:18 smithi082 ceph-mon[98694]: pgmap v16154: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 12 op/s 2023-09-10T07:39:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:18 smithi098 ceph-mon[103124]: pgmap v16154: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 12 op/s 2023-09-10T07:39:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29989.524696 secs 2023-09-10T07:39:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29989.524696 secs 2023-09-10T07:39:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29989.524696 secs 2023-09-10T07:39:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29990.659964 secs 2023-09-10T07:39:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29991.136485 secs 2023-09-10T07:39:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:20 smithi170 ceph-mon[102832]: pgmap v16155: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T07:39:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29990.659964 secs 2023-09-10T07:39:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29991.136485 secs 2023-09-10T07:39:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:20 smithi082 ceph-mon[98694]: pgmap v16155: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T07:39:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29990.659964 secs 2023-09-10T07:39:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29991.136485 secs 2023-09-10T07:39:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:20 smithi098 ceph-mon[103124]: pgmap v16155: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T07:39:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:22 smithi170 ceph-mon[102832]: pgmap v16156: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T07:39:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:39:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:39:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:39:22.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:39:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:22 smithi082 ceph-mon[98694]: pgmap v16156: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T07:39:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:39:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:39:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:39:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:39:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:22 smithi098 ceph-mon[103124]: pgmap v16156: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T07:39:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:39:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:39:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:39:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:39:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29994.524755 secs 2023-09-10T07:39:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:24 smithi170 ceph-mon[102832]: pgmap v16157: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T07:39:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29994.524755 secs 2023-09-10T07:39:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:24 smithi082 ceph-mon[98694]: pgmap v16157: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T07:39:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29994.524755 secs 2023-09-10T07:39:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:24 smithi098 ceph-mon[103124]: pgmap v16157: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T07:39:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29995.660044 secs 2023-09-10T07:39:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 29996.136544 secs 2023-09-10T07:39:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29995.660044 secs 2023-09-10T07:39:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 29996.136544 secs 2023-09-10T07:39:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29995.660044 secs 2023-09-10T07:39:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 29996.136544 secs 2023-09-10T07:39:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:26 smithi170 ceph-mon[102832]: pgmap v16158: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-10T07:39:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:26 smithi082 ceph-mon[98694]: pgmap v16158: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-10T07:39:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:26 smithi098 ceph-mon[103124]: pgmap v16158: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-10T07:39:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:28 smithi170 ceph-mon[102832]: pgmap v16159: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 100 KiB/s wr, 14 op/s 2023-09-10T07:39:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:28 smithi082 ceph-mon[98694]: pgmap v16159: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 100 KiB/s wr, 14 op/s 2023-09-10T07:39:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:28 smithi098 ceph-mon[103124]: pgmap v16159: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 100 KiB/s wr, 14 op/s 2023-09-10T07:39:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 29999.524807 secs 2023-09-10T07:39:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 29999.524807 secs 2023-09-10T07:39:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 29999.524807 secs 2023-09-10T07:39:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30000.660103 secs 2023-09-10T07:39:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30001.136605 secs 2023-09-10T07:39:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:30 smithi170 ceph-mon[102832]: pgmap v16160: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 9 op/s 2023-09-10T07:39:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30000.660103 secs 2023-09-10T07:39:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30001.136605 secs 2023-09-10T07:39:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:30 smithi082 ceph-mon[98694]: pgmap v16160: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 9 op/s 2023-09-10T07:39:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30000.660103 secs 2023-09-10T07:39:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30001.136605 secs 2023-09-10T07:39:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:30 smithi098 ceph-mon[103124]: pgmap v16160: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 9 op/s 2023-09-10T07:39:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:32 smithi170 ceph-mon[102832]: pgmap v16161: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 298 KiB/s wr, 15 op/s 2023-09-10T07:39:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:32 smithi082 ceph-mon[98694]: pgmap v16161: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 298 KiB/s wr, 15 op/s 2023-09-10T07:39:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:32 smithi098 ceph-mon[103124]: pgmap v16161: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 298 KiB/s wr, 15 op/s 2023-09-10T07:39:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30004.524868 secs 2023-09-10T07:39:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:34 smithi170 ceph-mon[102832]: pgmap v16162: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 298 KiB/s wr, 10 op/s 2023-09-10T07:39:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30004.524868 secs 2023-09-10T07:39:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:34 smithi082 ceph-mon[98694]: pgmap v16162: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 298 KiB/s wr, 10 op/s 2023-09-10T07:39:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30004.524868 secs 2023-09-10T07:39:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:34 smithi098 ceph-mon[103124]: pgmap v16162: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 298 KiB/s wr, 10 op/s 2023-09-10T07:39:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30005.660202 secs 2023-09-10T07:39:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30006.136663 secs 2023-09-10T07:39:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30005.660202 secs 2023-09-10T07:39:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30006.136663 secs 2023-09-10T07:39:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30005.660202 secs 2023-09-10T07:39:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30006.136663 secs 2023-09-10T07:39:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:36 smithi170 ceph-mon[102832]: pgmap v16163: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 298 KiB/s wr, 10 op/s 2023-09-10T07:39:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:36 smithi082 ceph-mon[98694]: pgmap v16163: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 298 KiB/s wr, 10 op/s 2023-09-10T07:39:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:36 smithi098 ceph-mon[103124]: pgmap v16163: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 298 KiB/s wr, 10 op/s 2023-09-10T07:39:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:38 smithi170 ceph-mon[102832]: pgmap v16164: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 307 KiB/s wr, 15 op/s 2023-09-10T07:39:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:38 smithi082 ceph-mon[98694]: pgmap v16164: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 307 KiB/s wr, 15 op/s 2023-09-10T07:39:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:38 smithi098 ceph-mon[103124]: pgmap v16164: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 307 KiB/s wr, 15 op/s 2023-09-10T07:39:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:39 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30009.524936 secs 2023-09-10T07:39:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:39 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30009.524936 secs 2023-09-10T07:39:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:39 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30009.524936 secs 2023-09-10T07:39:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30010.660260 secs 2023-09-10T07:39:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30011.136724 secs 2023-09-10T07:39:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:40 smithi170 ceph-mon[102832]: pgmap v16165: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 207 KiB/s wr, 10 op/s 2023-09-10T07:39:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30010.660260 secs 2023-09-10T07:39:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30011.136724 secs 2023-09-10T07:39:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:40 smithi082 ceph-mon[98694]: pgmap v16165: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 207 KiB/s wr, 10 op/s 2023-09-10T07:39:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30010.660260 secs 2023-09-10T07:39:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30011.136724 secs 2023-09-10T07:39:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:40 smithi098 ceph-mon[103124]: pgmap v16165: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 207 KiB/s wr, 10 op/s 2023-09-10T07:39:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:42 smithi170 ceph-mon[102832]: pgmap v16166: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 532 KiB/s wr, 15 op/s 2023-09-10T07:39:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:42 smithi082 ceph-mon[98694]: pgmap v16166: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 532 KiB/s wr, 15 op/s 2023-09-10T07:39:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:42 smithi098 ceph-mon[103124]: pgmap v16166: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 532 KiB/s wr, 15 op/s 2023-09-10T07:39:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:39:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:39:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:39:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:44 smithi170 ceph-mon[102832]: pgmap v16167: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 334 KiB/s wr, 10 op/s 2023-09-10T07:39:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:44 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:39:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:39:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:39:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:44 smithi082 ceph-mon[98694]: pgmap v16167: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 334 KiB/s wr, 10 op/s 2023-09-10T07:39:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:44 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:39:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:39:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:39:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:44 smithi098 ceph-mon[103124]: pgmap v16167: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 334 KiB/s wr, 10 op/s 2023-09-10T07:39:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:44 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:39:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:39:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:39:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30015.660320 secs 2023-09-10T07:39:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30016.136780 secs 2023-09-10T07:39:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30015.660320 secs 2023-09-10T07:39:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30016.136780 secs 2023-09-10T07:39:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30015.660320 secs 2023-09-10T07:39:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30016.136780 secs 2023-09-10T07:39:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:46 smithi170 ceph-mon[102832]: pgmap v16168: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 334 KiB/s wr, 10 op/s 2023-09-10T07:39:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:46 smithi082 ceph-mon[98694]: pgmap v16168: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 334 KiB/s wr, 10 op/s 2023-09-10T07:39:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:46 smithi098 ceph-mon[103124]: pgmap v16168: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 334 KiB/s wr, 10 op/s 2023-09-10T07:39:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:48 smithi170 ceph-mon[102832]: pgmap v16169: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 487 KiB/s wr, 17 op/s 2023-09-10T07:39:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:48 smithi082 ceph-mon[98694]: pgmap v16169: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 487 KiB/s wr, 17 op/s 2023-09-10T07:39:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:48 smithi098 ceph-mon[103124]: pgmap v16169: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 487 KiB/s wr, 17 op/s 2023-09-10T07:39:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30020.660404 secs 2023-09-10T07:39:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30021.136846 secs 2023-09-10T07:39:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:50 smithi170 ceph-mon[102832]: pgmap v16170: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 477 KiB/s wr, 11 op/s 2023-09-10T07:39:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30020.660404 secs 2023-09-10T07:39:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30021.136846 secs 2023-09-10T07:39:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:50 smithi082 ceph-mon[98694]: pgmap v16170: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 477 KiB/s wr, 11 op/s 2023-09-10T07:39:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30020.660404 secs 2023-09-10T07:39:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30021.136846 secs 2023-09-10T07:39:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:50 smithi098 ceph-mon[103124]: pgmap v16170: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 477 KiB/s wr, 11 op/s 2023-09-10T07:39:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:52 smithi170 ceph-mon[102832]: pgmap v16171: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 749 KiB/s wr, 19 op/s 2023-09-10T07:39:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:52 smithi082 ceph-mon[98694]: pgmap v16171: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 749 KiB/s wr, 19 op/s 2023-09-10T07:39:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:52 smithi098 ceph-mon[103124]: pgmap v16171: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 749 KiB/s wr, 19 op/s 2023-09-10T07:39:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:54 smithi170 ceph-mon[102832]: pgmap v16172: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 424 KiB/s wr, 14 op/s 2023-09-10T07:39:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:54 smithi082 ceph-mon[98694]: pgmap v16172: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 424 KiB/s wr, 14 op/s 2023-09-10T07:39:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:54 smithi098 ceph-mon[103124]: pgmap v16172: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 424 KiB/s wr, 14 op/s 2023-09-10T07:39:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30025.660494 secs 2023-09-10T07:39:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30026.136905 secs 2023-09-10T07:39:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30025.660494 secs 2023-09-10T07:39:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30026.136905 secs 2023-09-10T07:39:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30025.660494 secs 2023-09-10T07:39:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30026.136905 secs 2023-09-10T07:39:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:56 smithi170 ceph-mon[102832]: pgmap v16173: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 424 KiB/s wr, 14 op/s 2023-09-10T07:39:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:56 smithi082 ceph-mon[98694]: pgmap v16173: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 424 KiB/s wr, 14 op/s 2023-09-10T07:39:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:56 smithi098 ceph-mon[103124]: pgmap v16173: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 424 KiB/s wr, 14 op/s 2023-09-10T07:39:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:58 smithi170 ceph-mon[102832]: pgmap v16174: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 504 KiB/s wr, 20 op/s 2023-09-10T07:39:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:58 smithi082 ceph-mon[98694]: pgmap v16174: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 504 KiB/s wr, 20 op/s 2023-09-10T07:39:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:58 smithi098 ceph-mon[103124]: pgmap v16174: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 504 KiB/s wr, 20 op/s 2023-09-10T07:39:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:39:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30029.525209 secs 2023-09-10T07:39:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:39:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30029.525209 secs 2023-09-10T07:39:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:39:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30029.525209 secs 2023-09-10T07:40:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30030.660553 secs 2023-09-10T07:40:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30031.136967 secs 2023-09-10T07:40:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:40:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:40:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:40:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:00 smithi170 ceph-mon[102832]: pgmap v16175: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 351 KiB/s wr, 14 op/s 2023-09-10T07:40:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T07:40:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30030.660553 secs 2023-09-10T07:40:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30031.136967 secs 2023-09-10T07:40:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:40:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:40:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:40:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:00 smithi082 ceph-mon[98694]: pgmap v16175: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 351 KiB/s wr, 14 op/s 2023-09-10T07:40:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T07:40:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30030.660553 secs 2023-09-10T07:40:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30031.136967 secs 2023-09-10T07:40:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:40:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:40:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:40:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:00 smithi098 ceph-mon[103124]: pgmap v16175: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 351 KiB/s wr, 14 op/s 2023-09-10T07:40:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T07:40:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:02 smithi170 ceph-mon[102832]: pgmap v16176: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 595 KiB/s wr, 19 op/s 2023-09-10T07:40:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:02 smithi082 ceph-mon[98694]: pgmap v16176: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 595 KiB/s wr, 19 op/s 2023-09-10T07:40:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:02 smithi098 ceph-mon[103124]: pgmap v16176: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 595 KiB/s wr, 19 op/s 2023-09-10T07:40:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30034.525320 secs 2023-09-10T07:40:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:04 smithi170 ceph-mon[102832]: pgmap v16177: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 323 KiB/s wr, 11 op/s 2023-09-10T07:40:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30034.525320 secs 2023-09-10T07:40:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:04 smithi082 ceph-mon[98694]: pgmap v16177: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 323 KiB/s wr, 11 op/s 2023-09-10T07:40:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30034.525320 secs 2023-09-10T07:40:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:04 smithi098 ceph-mon[103124]: pgmap v16177: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 323 KiB/s wr, 11 op/s 2023-09-10T07:40:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30035.660612 secs 2023-09-10T07:40:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30036.137025 secs 2023-09-10T07:40:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30035.660612 secs 2023-09-10T07:40:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30036.137025 secs 2023-09-10T07:40:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30035.660612 secs 2023-09-10T07:40:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30036.137025 secs 2023-09-10T07:40:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:06 smithi170 ceph-mon[102832]: pgmap v16178: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 323 KiB/s wr, 11 op/s 2023-09-10T07:40:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:06 smithi082 ceph-mon[98694]: pgmap v16178: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 323 KiB/s wr, 11 op/s 2023-09-10T07:40:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:06 smithi098 ceph-mon[103124]: pgmap v16178: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 323 KiB/s wr, 11 op/s 2023-09-10T07:40:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:08 smithi170 ceph-mon[102832]: pgmap v16179: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 451 KiB/s wr, 31 op/s 2023-09-10T07:40:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:08 smithi082 ceph-mon[98694]: pgmap v16179: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 451 KiB/s wr, 31 op/s 2023-09-10T07:40:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:08 smithi098 ceph-mon[103124]: pgmap v16179: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 451 KiB/s wr, 31 op/s 2023-09-10T07:40:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30039.525426 secs 2023-09-10T07:40:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:09 smithi170 ceph-mon[102832]: osdmap e1079: 12 total, 12 up, 12 in 2023-09-10T07:40:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30039.525426 secs 2023-09-10T07:40:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:09 smithi082 ceph-mon[98694]: osdmap e1079: 12 total, 12 up, 12 in 2023-09-10T07:40:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30039.525426 secs 2023-09-10T07:40:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:09 smithi098 ceph-mon[103124]: osdmap e1079: 12 total, 12 up, 12 in 2023-09-10T07:40:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30040.660672 secs 2023-09-10T07:40:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30041.137086 secs 2023-09-10T07:40:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:10 smithi170 ceph-mon[102832]: pgmap v16181: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 446 KiB/s wr, 30 op/s 2023-09-10T07:40:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30040.660672 secs 2023-09-10T07:40:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30041.137086 secs 2023-09-10T07:40:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:10 smithi082 ceph-mon[98694]: pgmap v16181: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 446 KiB/s wr, 30 op/s 2023-09-10T07:40:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30040.660672 secs 2023-09-10T07:40:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30041.137086 secs 2023-09-10T07:40:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:10 smithi098 ceph-mon[103124]: pgmap v16181: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 446 KiB/s wr, 30 op/s 2023-09-10T07:40:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:12 smithi170 ceph-mon[102832]: pgmap v16182: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 153 KiB/s wr, 29 op/s 2023-09-10T07:40:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:12 smithi170 ceph-mon[102832]: osdmap e1080: 12 total, 12 up, 12 in 2023-09-10T07:40:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:12 smithi082 ceph-mon[98694]: pgmap v16182: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 153 KiB/s wr, 29 op/s 2023-09-10T07:40:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:12 smithi082 ceph-mon[98694]: osdmap e1080: 12 total, 12 up, 12 in 2023-09-10T07:40:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:12 smithi098 ceph-mon[103124]: pgmap v16182: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 153 KiB/s wr, 29 op/s 2023-09-10T07:40:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:12 smithi098 ceph-mon[103124]: osdmap e1080: 12 total, 12 up, 12 in 2023-09-10T07:40:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30044.525511 secs 2023-09-10T07:40:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:14 smithi170 ceph-mon[102832]: pgmap v16184: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 192 KiB/s wr, 36 op/s 2023-09-10T07:40:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30044.525511 secs 2023-09-10T07:40:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:14 smithi082 ceph-mon[98694]: pgmap v16184: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 192 KiB/s wr, 36 op/s 2023-09-10T07:40:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30044.525511 secs 2023-09-10T07:40:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:14 smithi098 ceph-mon[103124]: pgmap v16184: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 192 KiB/s wr, 36 op/s 2023-09-10T07:40:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30045.660729 secs 2023-09-10T07:40:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30046.137144 secs 2023-09-10T07:40:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30045.660729 secs 2023-09-10T07:40:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30046.137144 secs 2023-09-10T07:40:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30045.660729 secs 2023-09-10T07:40:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30046.137144 secs 2023-09-10T07:40:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:16 smithi170 ceph-mon[102832]: pgmap v16185: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:40:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:16 smithi082 ceph-mon[98694]: pgmap v16185: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:40:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:16 smithi098 ceph-mon[103124]: pgmap v16185: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:40:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:18 smithi170 ceph-mon[102832]: pgmap v16186: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 334 B/s wr, 12 op/s 2023-09-10T07:40:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:18 smithi082 ceph-mon[98694]: pgmap v16186: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 334 B/s wr, 12 op/s 2023-09-10T07:40:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:18 smithi098 ceph-mon[103124]: pgmap v16186: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 334 B/s wr, 12 op/s 2023-09-10T07:40:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30049.525571 secs 2023-09-10T07:40:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30049.525571 secs 2023-09-10T07:40:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30049.525571 secs 2023-09-10T07:40:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30050.660786 secs 2023-09-10T07:40:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30051.137208 secs 2023-09-10T07:40:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:20 smithi170 ceph-mon[102832]: pgmap v16187: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T07:40:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30050.660786 secs 2023-09-10T07:40:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30051.137208 secs 2023-09-10T07:40:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:20 smithi082 ceph-mon[98694]: pgmap v16187: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T07:40:20.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30050.660786 secs 2023-09-10T07:40:20.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30051.137208 secs 2023-09-10T07:40:20.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:20 smithi098 ceph-mon[103124]: pgmap v16187: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T07:40:22.609 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:22 smithi170 ceph-mon[102832]: pgmap v16188: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:40:22.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:22 smithi082 ceph-mon[98694]: pgmap v16188: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:40:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:22 smithi098 ceph-mon[103124]: pgmap v16188: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:40:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:40:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:40:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:40:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:40:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:40:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:40:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:40:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:40:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:40:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:40:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:40:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:40:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30054.525629 secs 2023-09-10T07:40:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:24 smithi170 ceph-mon[102832]: pgmap v16189: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 183 B/s wr, 9 op/s 2023-09-10T07:40:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30054.525629 secs 2023-09-10T07:40:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:24 smithi082 ceph-mon[98694]: pgmap v16189: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 183 B/s wr, 9 op/s 2023-09-10T07:40:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30054.525629 secs 2023-09-10T07:40:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:24 smithi098 ceph-mon[103124]: pgmap v16189: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 183 B/s wr, 9 op/s 2023-09-10T07:40:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30055.660846 secs 2023-09-10T07:40:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30056.137266 secs 2023-09-10T07:40:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30055.660846 secs 2023-09-10T07:40:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30056.137266 secs 2023-09-10T07:40:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30055.660846 secs 2023-09-10T07:40:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30056.137266 secs 2023-09-10T07:40:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:26 smithi170 ceph-mon[102832]: pgmap v16190: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-10T07:40:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:26 smithi082 ceph-mon[98694]: pgmap v16190: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-10T07:40:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:26 smithi098 ceph-mon[103124]: pgmap v16190: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2023-09-10T07:40:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:28 smithi170 ceph-mon[102832]: pgmap v16191: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 597 B/s wr, 13 op/s 2023-09-10T07:40:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:28 smithi082 ceph-mon[98694]: pgmap v16191: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 597 B/s wr, 13 op/s 2023-09-10T07:40:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:28 smithi098 ceph-mon[103124]: pgmap v16191: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 597 B/s wr, 13 op/s 2023-09-10T07:40:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30059.525732 secs 2023-09-10T07:40:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30059.525732 secs 2023-09-10T07:40:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30059.525732 secs 2023-09-10T07:40:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30060.660927 secs 2023-09-10T07:40:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30061.137330 secs 2023-09-10T07:40:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:30 smithi170 ceph-mon[102832]: pgmap v16192: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:40:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30060.660927 secs 2023-09-10T07:40:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30061.137330 secs 2023-09-10T07:40:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:30 smithi082 ceph-mon[98694]: pgmap v16192: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:40:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30060.660927 secs 2023-09-10T07:40:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30061.137330 secs 2023-09-10T07:40:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:30 smithi098 ceph-mon[103124]: pgmap v16192: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:40:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:32 smithi170 ceph-mon[102832]: pgmap v16193: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 105 KiB/s wr, 13 op/s 2023-09-10T07:40:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:32 smithi082 ceph-mon[98694]: pgmap v16193: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 105 KiB/s wr, 13 op/s 2023-09-10T07:40:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:32 smithi098 ceph-mon[103124]: pgmap v16193: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 105 KiB/s wr, 13 op/s 2023-09-10T07:40:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:34 smithi170 ceph-mon[102832]: pgmap v16194: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 9 op/s 2023-09-10T07:40:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:34 smithi082 ceph-mon[98694]: pgmap v16194: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 9 op/s 2023-09-10T07:40:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:34 smithi098 ceph-mon[103124]: pgmap v16194: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 9 op/s 2023-09-10T07:40:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30065.660989 secs 2023-09-10T07:40:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30066.137390 secs 2023-09-10T07:40:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:40:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30065.660989 secs 2023-09-10T07:40:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30066.137390 secs 2023-09-10T07:40:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:40:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30065.660989 secs 2023-09-10T07:40:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30066.137390 secs 2023-09-10T07:40:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:40:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:36 smithi170 ceph-mon[102832]: pgmap v16195: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 9 op/s 2023-09-10T07:40:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:40:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:40:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:40:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:36 smithi082 ceph-mon[98694]: pgmap v16195: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 9 op/s 2023-09-10T07:40:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:40:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:40:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:40:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:36 smithi098 ceph-mon[103124]: pgmap v16195: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 9 op/s 2023-09-10T07:40:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:40:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:40:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:40:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:38 smithi170 ceph-mon[102832]: pgmap v16196: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 320 KiB/s wr, 14 op/s 2023-09-10T07:40:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:38 smithi082 ceph-mon[98694]: pgmap v16196: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 320 KiB/s wr, 14 op/s 2023-09-10T07:40:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:38 smithi098 ceph-mon[103124]: pgmap v16196: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 320 KiB/s wr, 14 op/s 2023-09-10T07:40:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:39 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30069.525849 secs 2023-09-10T07:40:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:39 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30069.525849 secs 2023-09-10T07:40:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:39 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30069.525849 secs 2023-09-10T07:40:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30070.661073 secs 2023-09-10T07:40:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30071.137450 secs 2023-09-10T07:40:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:40:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:40:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:40 smithi170 ceph-mon[102832]: pgmap v16197: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 319 KiB/s wr, 10 op/s 2023-09-10T07:40:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30070.661073 secs 2023-09-10T07:40:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30071.137450 secs 2023-09-10T07:40:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:40:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:40:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:40 smithi082 ceph-mon[98694]: pgmap v16197: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 319 KiB/s wr, 10 op/s 2023-09-10T07:40:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30070.661073 secs 2023-09-10T07:40:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30071.137450 secs 2023-09-10T07:40:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:40:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:40:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:40 smithi098 ceph-mon[103124]: pgmap v16197: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 319 KiB/s wr, 10 op/s 2023-09-10T07:40:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:42 smithi170 ceph-mon[102832]: pgmap v16198: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 534 KiB/s wr, 15 op/s 2023-09-10T07:40:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:42 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:40:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:42 smithi082 ceph-mon[98694]: pgmap v16198: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 534 KiB/s wr, 15 op/s 2023-09-10T07:40:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:42 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:40:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:42 smithi098 ceph-mon[103124]: pgmap v16198: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 534 KiB/s wr, 15 op/s 2023-09-10T07:40:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:42 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:40:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:44 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30074.525933 secs 2023-09-10T07:40:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:44 smithi170 ceph-mon[102832]: pgmap v16199: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 429 KiB/s wr, 10 op/s 2023-09-10T07:40:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:44 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30074.525933 secs 2023-09-10T07:40:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:44 smithi082 ceph-mon[98694]: pgmap v16199: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 429 KiB/s wr, 10 op/s 2023-09-10T07:40:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:44 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30074.525933 secs 2023-09-10T07:40:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:44 smithi098 ceph-mon[103124]: pgmap v16199: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 429 KiB/s wr, 10 op/s 2023-09-10T07:40:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30075.661132 secs 2023-09-10T07:40:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30076.137508 secs 2023-09-10T07:40:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30075.661132 secs 2023-09-10T07:40:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30076.137508 secs 2023-09-10T07:40:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30075.661132 secs 2023-09-10T07:40:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30076.137508 secs 2023-09-10T07:40:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:46 smithi170 ceph-mon[102832]: pgmap v16200: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 429 KiB/s wr, 10 op/s 2023-09-10T07:40:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:46 smithi082 ceph-mon[98694]: pgmap v16200: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 429 KiB/s wr, 10 op/s 2023-09-10T07:40:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:46 smithi098 ceph-mon[103124]: pgmap v16200: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 429 KiB/s wr, 10 op/s 2023-09-10T07:40:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:48 smithi170 ceph-mon[102832]: pgmap v16201: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 581 KiB/s wr, 17 op/s 2023-09-10T07:40:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:48 smithi082 ceph-mon[98694]: pgmap v16201: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 581 KiB/s wr, 17 op/s 2023-09-10T07:40:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:48 smithi098 ceph-mon[103124]: pgmap v16201: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 581 KiB/s wr, 17 op/s 2023-09-10T07:40:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30080.661213 secs 2023-09-10T07:40:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30081.137571 secs 2023-09-10T07:40:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:50 smithi170 ceph-mon[102832]: pgmap v16202: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 367 KiB/s wr, 12 op/s 2023-09-10T07:40:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30080.661213 secs 2023-09-10T07:40:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30081.137571 secs 2023-09-10T07:40:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:50 smithi082 ceph-mon[98694]: pgmap v16202: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 367 KiB/s wr, 12 op/s 2023-09-10T07:40:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30080.661213 secs 2023-09-10T07:40:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30081.137571 secs 2023-09-10T07:40:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:50 smithi098 ceph-mon[103124]: pgmap v16202: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 367 KiB/s wr, 12 op/s 2023-09-10T07:40:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:51 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:40:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:51 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:40:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:51 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:40:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:52 smithi170 ceph-mon[102832]: pgmap v16203: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 639 KiB/s wr, 19 op/s 2023-09-10T07:40:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:52 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:40:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:40:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:40:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:52 smithi082 ceph-mon[98694]: pgmap v16203: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 639 KiB/s wr, 19 op/s 2023-09-10T07:40:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:52 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:40:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:40:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:40:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:52 smithi098 ceph-mon[103124]: pgmap v16203: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 639 KiB/s wr, 19 op/s 2023-09-10T07:40:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:52 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:40:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:40:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:40:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:54 smithi170 ceph-mon[102832]: pgmap v16204: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 424 KiB/s wr, 13 op/s 2023-09-10T07:40:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:54 smithi082 ceph-mon[98694]: pgmap v16204: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 424 KiB/s wr, 13 op/s 2023-09-10T07:40:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:54 smithi098 ceph-mon[103124]: pgmap v16204: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 424 KiB/s wr, 13 op/s 2023-09-10T07:40:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30085.661270 secs 2023-09-10T07:40:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30086.137631 secs 2023-09-10T07:40:55.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30085.661270 secs 2023-09-10T07:40:55.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30086.137631 secs 2023-09-10T07:40:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30085.661270 secs 2023-09-10T07:40:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30086.137631 secs 2023-09-10T07:40:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:56 smithi170 ceph-mon[102832]: pgmap v16205: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 424 KiB/s wr, 13 op/s 2023-09-10T07:40:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:56 smithi082 ceph-mon[98694]: pgmap v16205: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 424 KiB/s wr, 13 op/s 2023-09-10T07:40:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:56 smithi098 ceph-mon[103124]: pgmap v16205: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 424 KiB/s wr, 13 op/s 2023-09-10T07:40:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:58 smithi170 ceph-mon[102832]: pgmap v16206: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 504 KiB/s wr, 19 op/s 2023-09-10T07:40:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:58 smithi082 ceph-mon[98694]: pgmap v16206: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 504 KiB/s wr, 19 op/s 2023-09-10T07:40:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:58 smithi098 ceph-mon[103124]: pgmap v16206: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 504 KiB/s wr, 19 op/s 2023-09-10T07:40:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:40:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30089.526168 secs 2023-09-10T07:40:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:40:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30089.526168 secs 2023-09-10T07:40:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:40:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30089.526168 secs 2023-09-10T07:41:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30090.661328 secs 2023-09-10T07:41:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30091.137692 secs 2023-09-10T07:41:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:41:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:41:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:41:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:00 smithi170 ceph-mon[102832]: pgmap v16207: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 352 KiB/s wr, 13 op/s 2023-09-10T07:41:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30090.661328 secs 2023-09-10T07:41:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30091.137692 secs 2023-09-10T07:41:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:41:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:41:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:41:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:00 smithi082 ceph-mon[98694]: pgmap v16207: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 352 KiB/s wr, 13 op/s 2023-09-10T07:41:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30090.661328 secs 2023-09-10T07:41:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30091.137692 secs 2023-09-10T07:41:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:41:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:41:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:41:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:00 smithi098 ceph-mon[103124]: pgmap v16207: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 352 KiB/s wr, 13 op/s 2023-09-10T07:41:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:02 smithi170 ceph-mon[102832]: pgmap v16208: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 596 KiB/s wr, 18 op/s 2023-09-10T07:41:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:02 smithi082 ceph-mon[98694]: pgmap v16208: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 596 KiB/s wr, 18 op/s 2023-09-10T07:41:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:02 smithi098 ceph-mon[103124]: pgmap v16208: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 596 KiB/s wr, 18 op/s 2023-09-10T07:41:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30094.526229 secs 2023-09-10T07:41:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:04 smithi170 ceph-mon[102832]: pgmap v16209: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 324 KiB/s wr, 11 op/s 2023-09-10T07:41:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30094.526229 secs 2023-09-10T07:41:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:04 smithi082 ceph-mon[98694]: pgmap v16209: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 324 KiB/s wr, 11 op/s 2023-09-10T07:41:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30094.526229 secs 2023-09-10T07:41:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:04 smithi098 ceph-mon[103124]: pgmap v16209: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 324 KiB/s wr, 11 op/s 2023-09-10T07:41:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30095.661390 secs 2023-09-10T07:41:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30096.137748 secs 2023-09-10T07:41:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30095.661390 secs 2023-09-10T07:41:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30096.137748 secs 2023-09-10T07:41:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30095.661390 secs 2023-09-10T07:41:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30096.137748 secs 2023-09-10T07:41:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:06 smithi170 ceph-mon[102832]: pgmap v16210: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 324 KiB/s wr, 11 op/s 2023-09-10T07:41:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:06 smithi082 ceph-mon[98694]: pgmap v16210: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 324 KiB/s wr, 11 op/s 2023-09-10T07:41:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:06 smithi098 ceph-mon[103124]: pgmap v16210: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 324 KiB/s wr, 11 op/s 2023-09-10T07:41:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:08 smithi170 ceph-mon[102832]: pgmap v16211: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 452 KiB/s wr, 31 op/s 2023-09-10T07:41:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:08 smithi082 ceph-mon[98694]: pgmap v16211: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 452 KiB/s wr, 31 op/s 2023-09-10T07:41:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:08 smithi098 ceph-mon[103124]: pgmap v16211: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 452 KiB/s wr, 31 op/s 2023-09-10T07:41:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30099.526290 secs 2023-09-10T07:41:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:09 smithi170 ceph-mon[102832]: osdmap e1081: 12 total, 12 up, 12 in 2023-09-10T07:41:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30099.526290 secs 2023-09-10T07:41:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:09 smithi082 ceph-mon[98694]: osdmap e1081: 12 total, 12 up, 12 in 2023-09-10T07:41:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30099.526290 secs 2023-09-10T07:41:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:09 smithi098 ceph-mon[103124]: osdmap e1081: 12 total, 12 up, 12 in 2023-09-10T07:41:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30100.661473 secs 2023-09-10T07:41:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30101.137810 secs 2023-09-10T07:41:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:10 smithi170 ceph-mon[102832]: pgmap v16213: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 447 KiB/s wr, 30 op/s 2023-09-10T07:41:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30100.661473 secs 2023-09-10T07:41:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30101.137810 secs 2023-09-10T07:41:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:10 smithi082 ceph-mon[98694]: pgmap v16213: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 447 KiB/s wr, 30 op/s 2023-09-10T07:41:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30100.661473 secs 2023-09-10T07:41:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30101.137810 secs 2023-09-10T07:41:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:10 smithi098 ceph-mon[103124]: pgmap v16213: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 447 KiB/s wr, 30 op/s 2023-09-10T07:41:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:12 smithi170 ceph-mon[102832]: pgmap v16214: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 154 KiB/s wr, 28 op/s 2023-09-10T07:41:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:12 smithi170 ceph-mon[102832]: osdmap e1082: 12 total, 12 up, 12 in 2023-09-10T07:41:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:12 smithi082 ceph-mon[98694]: pgmap v16214: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 154 KiB/s wr, 28 op/s 2023-09-10T07:41:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:12 smithi082 ceph-mon[98694]: osdmap e1082: 12 total, 12 up, 12 in 2023-09-10T07:41:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:12 smithi098 ceph-mon[103124]: pgmap v16214: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 154 KiB/s wr, 28 op/s 2023-09-10T07:41:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:12 smithi098 ceph-mon[103124]: osdmap e1082: 12 total, 12 up, 12 in 2023-09-10T07:41:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30104.526389 secs 2023-09-10T07:41:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:14 smithi170 ceph-mon[102832]: pgmap v16216: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 192 KiB/s wr, 35 op/s 2023-09-10T07:41:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30104.526389 secs 2023-09-10T07:41:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:14 smithi082 ceph-mon[98694]: pgmap v16216: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 192 KiB/s wr, 35 op/s 2023-09-10T07:41:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30104.526389 secs 2023-09-10T07:41:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:14 smithi098 ceph-mon[103124]: pgmap v16216: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 192 KiB/s wr, 35 op/s 2023-09-10T07:41:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30105.661559 secs 2023-09-10T07:41:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30106.137867 secs 2023-09-10T07:41:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30105.661559 secs 2023-09-10T07:41:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30106.137867 secs 2023-09-10T07:41:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30105.661559 secs 2023-09-10T07:41:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30106.137867 secs 2023-09-10T07:41:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:16 smithi170 ceph-mon[102832]: pgmap v16217: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:41:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:16 smithi082 ceph-mon[98694]: pgmap v16217: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:41:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:16 smithi098 ceph-mon[103124]: pgmap v16217: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:41:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:18 smithi170 ceph-mon[102832]: pgmap v16218: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:41:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:18 smithi082 ceph-mon[98694]: pgmap v16218: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:41:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:18 smithi098 ceph-mon[103124]: pgmap v16218: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:41:19.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30109.526450 secs 2023-09-10T07:41:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30109.526450 secs 2023-09-10T07:41:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30109.526450 secs 2023-09-10T07:41:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30110.661640 secs 2023-09-10T07:41:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30111.137927 secs 2023-09-10T07:41:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:20 smithi170 ceph-mon[102832]: pgmap v16219: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:41:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30110.661640 secs 2023-09-10T07:41:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30111.137927 secs 2023-09-10T07:41:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:20 smithi082 ceph-mon[98694]: pgmap v16219: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:41:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30110.661640 secs 2023-09-10T07:41:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30111.137927 secs 2023-09-10T07:41:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:20 smithi098 ceph-mon[103124]: pgmap v16219: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:41:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:22 smithi170 ceph-mon[102832]: pgmap v16220: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:41:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:22 smithi082 ceph-mon[98694]: pgmap v16220: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:41:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:22 smithi098 ceph-mon[103124]: pgmap v16220: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:41:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:41:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:41:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:41:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30114.526510 secs 2023-09-10T07:41:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:24 smithi170 ceph-mon[102832]: pgmap v16221: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:41:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:41:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:41:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:41:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30114.526510 secs 2023-09-10T07:41:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:24 smithi082 ceph-mon[98694]: pgmap v16221: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:41:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:41:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:41:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:41:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30114.526510 secs 2023-09-10T07:41:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:24 smithi098 ceph-mon[103124]: pgmap v16221: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:41:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:41:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:41:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:41:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30115.661699 secs 2023-09-10T07:41:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30116.137987 secs 2023-09-10T07:41:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30115.661699 secs 2023-09-10T07:41:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30116.137987 secs 2023-09-10T07:41:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30115.661699 secs 2023-09-10T07:41:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30116.137987 secs 2023-09-10T07:41:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:26 smithi170 ceph-mon[102832]: pgmap v16222: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:41:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:26 smithi082 ceph-mon[98694]: pgmap v16222: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:41:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:26 smithi098 ceph-mon[103124]: pgmap v16222: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:41:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:28 smithi170 ceph-mon[102832]: pgmap v16223: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 100 KiB/s wr, 13 op/s 2023-09-10T07:41:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:28 smithi082 ceph-mon[98694]: pgmap v16223: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 100 KiB/s wr, 13 op/s 2023-09-10T07:41:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:28 smithi098 ceph-mon[103124]: pgmap v16223: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 100 KiB/s wr, 13 op/s 2023-09-10T07:41:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30119.526568 secs 2023-09-10T07:41:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30119.526568 secs 2023-09-10T07:41:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30119.526568 secs 2023-09-10T07:41:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30120.661757 secs 2023-09-10T07:41:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30121.138048 secs 2023-09-10T07:41:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:30 smithi170 ceph-mon[102832]: pgmap v16224: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 100 KiB/s wr, 9 op/s 2023-09-10T07:41:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30120.661757 secs 2023-09-10T07:41:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30121.138048 secs 2023-09-10T07:41:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:30 smithi082 ceph-mon[98694]: pgmap v16224: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 100 KiB/s wr, 9 op/s 2023-09-10T07:41:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30120.661757 secs 2023-09-10T07:41:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30121.138048 secs 2023-09-10T07:41:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:30 smithi098 ceph-mon[103124]: pgmap v16224: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 100 KiB/s wr, 9 op/s 2023-09-10T07:41:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:32 smithi170 ceph-mon[102832]: pgmap v16225: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 101 KiB/s wr, 14 op/s 2023-09-10T07:41:32.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:32 smithi082 ceph-mon[98694]: pgmap v16225: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 101 KiB/s wr, 14 op/s 2023-09-10T07:41:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:32 smithi098 ceph-mon[103124]: pgmap v16225: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 101 KiB/s wr, 14 op/s 2023-09-10T07:41:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30124.526628 secs 2023-09-10T07:41:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:34 smithi170 ceph-mon[102832]: pgmap v16226: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 101 KiB/s wr, 10 op/s 2023-09-10T07:41:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30124.526628 secs 2023-09-10T07:41:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:34 smithi082 ceph-mon[98694]: pgmap v16226: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 101 KiB/s wr, 10 op/s 2023-09-10T07:41:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30124.526628 secs 2023-09-10T07:41:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:34 smithi098 ceph-mon[103124]: pgmap v16226: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 101 KiB/s wr, 10 op/s 2023-09-10T07:41:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30125.661815 secs 2023-09-10T07:41:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30126.138109 secs 2023-09-10T07:41:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30125.661815 secs 2023-09-10T07:41:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30126.138109 secs 2023-09-10T07:41:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30125.661815 secs 2023-09-10T07:41:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30126.138109 secs 2023-09-10T07:41:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:36 smithi170 ceph-mon[102832]: pgmap v16227: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 101 KiB/s wr, 10 op/s 2023-09-10T07:41:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:36 smithi082 ceph-mon[98694]: pgmap v16227: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 101 KiB/s wr, 10 op/s 2023-09-10T07:41:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:36 smithi098 ceph-mon[103124]: pgmap v16227: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 101 KiB/s wr, 10 op/s 2023-09-10T07:41:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:38 smithi170 ceph-mon[102832]: pgmap v16228: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 336 KiB/s wr, 16 op/s 2023-09-10T07:41:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:38 smithi082 ceph-mon[98694]: pgmap v16228: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 336 KiB/s wr, 16 op/s 2023-09-10T07:41:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:38 smithi098 ceph-mon[103124]: pgmap v16228: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 336 KiB/s wr, 16 op/s 2023-09-10T07:41:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:39 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30129.526736 secs 2023-09-10T07:41:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:39 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30129.526736 secs 2023-09-10T07:41:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:39 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30129.526736 secs 2023-09-10T07:41:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30130.661898 secs 2023-09-10T07:41:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30131.138172 secs 2023-09-10T07:41:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:40 smithi170 ceph-mon[102832]: pgmap v16229: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 236 KiB/s wr, 11 op/s 2023-09-10T07:41:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30130.661898 secs 2023-09-10T07:41:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30131.138172 secs 2023-09-10T07:41:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:40 smithi082 ceph-mon[98694]: pgmap v16229: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 236 KiB/s wr, 11 op/s 2023-09-10T07:41:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30130.661898 secs 2023-09-10T07:41:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30131.138172 secs 2023-09-10T07:41:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:40 smithi098 ceph-mon[103124]: pgmap v16229: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 236 KiB/s wr, 11 op/s 2023-09-10T07:41:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:42 smithi170 ceph-mon[102832]: pgmap v16230: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 463 KiB/s wr, 18 op/s 2023-09-10T07:41:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:42 smithi082 ceph-mon[98694]: pgmap v16230: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 463 KiB/s wr, 18 op/s 2023-09-10T07:41:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:42 smithi098 ceph-mon[103124]: pgmap v16230: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 463 KiB/s wr, 18 op/s 2023-09-10T07:41:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:41:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:41:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:41:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:44 smithi170 ceph-mon[102832]: pgmap v16231: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 462 KiB/s wr, 12 op/s 2023-09-10T07:41:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:44 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:41:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:41:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:41:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:44 smithi082 ceph-mon[98694]: pgmap v16231: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 462 KiB/s wr, 12 op/s 2023-09-10T07:41:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:44 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:41:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:41:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:41:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:44 smithi098 ceph-mon[103124]: pgmap v16231: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 462 KiB/s wr, 12 op/s 2023-09-10T07:41:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:44 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:41:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:41:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:41:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30135.661956 secs 2023-09-10T07:41:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30136.138234 secs 2023-09-10T07:41:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30135.661956 secs 2023-09-10T07:41:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30136.138234 secs 2023-09-10T07:41:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30135.661956 secs 2023-09-10T07:41:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30136.138234 secs 2023-09-10T07:41:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:46 smithi170 ceph-mon[102832]: pgmap v16232: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 462 KiB/s wr, 12 op/s 2023-09-10T07:41:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:46 smithi082 ceph-mon[98694]: pgmap v16232: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 462 KiB/s wr, 12 op/s 2023-09-10T07:41:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:46 smithi098 ceph-mon[103124]: pgmap v16232: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 462 KiB/s wr, 12 op/s 2023-09-10T07:41:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:48 smithi170 ceph-mon[102832]: pgmap v16233: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 747 KiB/s wr, 19 op/s 2023-09-10T07:41:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:48 smithi082 ceph-mon[98694]: pgmap v16233: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 747 KiB/s wr, 19 op/s 2023-09-10T07:41:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:48 smithi098 ceph-mon[103124]: pgmap v16233: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 747 KiB/s wr, 19 op/s 2023-09-10T07:41:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30140.662014 secs 2023-09-10T07:41:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30141.138295 secs 2023-09-10T07:41:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:50 smithi170 ceph-mon[102832]: pgmap v16234: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 511 KiB/s wr, 14 op/s 2023-09-10T07:41:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30140.662014 secs 2023-09-10T07:41:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30141.138295 secs 2023-09-10T07:41:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:50 smithi082 ceph-mon[98694]: pgmap v16234: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 511 KiB/s wr, 14 op/s 2023-09-10T07:41:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30140.662014 secs 2023-09-10T07:41:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30141.138295 secs 2023-09-10T07:41:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:50 smithi098 ceph-mon[103124]: pgmap v16234: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 511 KiB/s wr, 14 op/s 2023-09-10T07:41:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:52 smithi170 ceph-mon[102832]: pgmap v16235: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 689 KiB/s wr, 21 op/s 2023-09-10T07:41:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:52 smithi082 ceph-mon[98694]: pgmap v16235: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 689 KiB/s wr, 21 op/s 2023-09-10T07:41:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:52 smithi098 ceph-mon[103124]: pgmap v16235: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 689 KiB/s wr, 21 op/s 2023-09-10T07:41:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30144.526997 secs 2023-09-10T07:41:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:54 smithi170 ceph-mon[102832]: pgmap v16236: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 463 KiB/s wr, 14 op/s 2023-09-10T07:41:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30144.526997 secs 2023-09-10T07:41:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:54 smithi082 ceph-mon[98694]: pgmap v16236: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 463 KiB/s wr, 14 op/s 2023-09-10T07:41:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30144.526997 secs 2023-09-10T07:41:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:54 smithi098 ceph-mon[103124]: pgmap v16236: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 463 KiB/s wr, 14 op/s 2023-09-10T07:41:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30145.662072 secs 2023-09-10T07:41:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30146.138353 secs 2023-09-10T07:41:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30145.662072 secs 2023-09-10T07:41:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30146.138353 secs 2023-09-10T07:41:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30145.662072 secs 2023-09-10T07:41:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30146.138353 secs 2023-09-10T07:41:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:56 smithi170 ceph-mon[102832]: pgmap v16237: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 463 KiB/s wr, 14 op/s 2023-09-10T07:41:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:41:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:41:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:41:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:56 smithi082 ceph-mon[98694]: pgmap v16237: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 463 KiB/s wr, 14 op/s 2023-09-10T07:41:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:41:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:41:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:41:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:56 smithi098 ceph-mon[103124]: pgmap v16237: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 463 KiB/s wr, 14 op/s 2023-09-10T07:41:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:41:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:41:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:41:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:58 smithi170 ceph-mon[102832]: pgmap v16238: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 610 KiB/s wr, 20 op/s 2023-09-10T07:41:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:58 smithi082 ceph-mon[98694]: pgmap v16238: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 610 KiB/s wr, 20 op/s 2023-09-10T07:41:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:58 smithi098 ceph-mon[103124]: pgmap v16238: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 610 KiB/s wr, 20 op/s 2023-09-10T07:41:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:41:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30149.527061 secs 2023-09-10T07:41:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:41:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30149.527061 secs 2023-09-10T07:41:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:41:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30149.527061 secs 2023-09-10T07:42:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30150.662131 secs 2023-09-10T07:42:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30151.138416 secs 2023-09-10T07:42:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:00 smithi170 ceph-mon[102832]: pgmap v16239: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 325 KiB/s wr, 13 op/s 2023-09-10T07:42:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30150.662131 secs 2023-09-10T07:42:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30151.138416 secs 2023-09-10T07:42:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:00 smithi082 ceph-mon[98694]: pgmap v16239: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 325 KiB/s wr, 13 op/s 2023-09-10T07:42:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30150.662131 secs 2023-09-10T07:42:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30151.138416 secs 2023-09-10T07:42:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:00 smithi098 ceph-mon[103124]: pgmap v16239: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 325 KiB/s wr, 13 op/s 2023-09-10T07:42:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:02 smithi170 ceph-mon[102832]: pgmap v16240: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 443 KiB/s wr, 18 op/s 2023-09-10T07:42:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:02 smithi082 ceph-mon[98694]: pgmap v16240: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 443 KiB/s wr, 18 op/s 2023-09-10T07:42:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:02 smithi098 ceph-mon[103124]: pgmap v16240: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 443 KiB/s wr, 18 op/s 2023-09-10T07:42:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30154.527121 secs 2023-09-10T07:42:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:04 smithi170 ceph-mon[102832]: pgmap v16241: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T07:42:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30154.527121 secs 2023-09-10T07:42:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:04 smithi082 ceph-mon[98694]: pgmap v16241: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T07:42:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30154.527121 secs 2023-09-10T07:42:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:04 smithi098 ceph-mon[103124]: pgmap v16241: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T07:42:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30155.662191 secs 2023-09-10T07:42:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30156.138479 secs 2023-09-10T07:42:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30155.662191 secs 2023-09-10T07:42:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30156.138479 secs 2023-09-10T07:42:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30155.662191 secs 2023-09-10T07:42:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30156.138479 secs 2023-09-10T07:42:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:06 smithi170 ceph-mon[102832]: pgmap v16242: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T07:42:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:06 smithi082 ceph-mon[98694]: pgmap v16242: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T07:42:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:06 smithi098 ceph-mon[103124]: pgmap v16242: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T07:42:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:08 smithi170 ceph-mon[102832]: pgmap v16243: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 310 KiB/s wr, 31 op/s 2023-09-10T07:42:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:08 smithi082 ceph-mon[98694]: pgmap v16243: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 310 KiB/s wr, 31 op/s 2023-09-10T07:42:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:08 smithi098 ceph-mon[103124]: pgmap v16243: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 310 KiB/s wr, 31 op/s 2023-09-10T07:42:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30159.527182 secs 2023-09-10T07:42:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:09 smithi170 ceph-mon[102832]: osdmap e1083: 12 total, 12 up, 12 in 2023-09-10T07:42:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30159.527182 secs 2023-09-10T07:42:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:09 smithi082 ceph-mon[98694]: osdmap e1083: 12 total, 12 up, 12 in 2023-09-10T07:42:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30159.527182 secs 2023-09-10T07:42:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:09 smithi098 ceph-mon[103124]: osdmap e1083: 12 total, 12 up, 12 in 2023-09-10T07:42:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30160.662275 secs 2023-09-10T07:42:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30161.138546 secs 2023-09-10T07:42:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:10 smithi170 ceph-mon[102832]: pgmap v16245: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 195 KiB/s wr, 30 op/s 2023-09-10T07:42:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30160.662275 secs 2023-09-10T07:42:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30161.138546 secs 2023-09-10T07:42:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:10 smithi082 ceph-mon[98694]: pgmap v16245: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 195 KiB/s wr, 30 op/s 2023-09-10T07:42:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30160.662275 secs 2023-09-10T07:42:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30161.138546 secs 2023-09-10T07:42:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:10 smithi098 ceph-mon[103124]: pgmap v16245: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 195 KiB/s wr, 30 op/s 2023-09-10T07:42:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:12 smithi170 ceph-mon[102832]: pgmap v16246: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 153 KiB/s wr, 29 op/s 2023-09-10T07:42:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:12 smithi170 ceph-mon[102832]: osdmap e1084: 12 total, 12 up, 12 in 2023-09-10T07:42:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:12 smithi082 ceph-mon[98694]: pgmap v16246: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 153 KiB/s wr, 29 op/s 2023-09-10T07:42:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:12 smithi082 ceph-mon[98694]: osdmap e1084: 12 total, 12 up, 12 in 2023-09-10T07:42:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:12 smithi098 ceph-mon[103124]: pgmap v16246: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 153 KiB/s wr, 29 op/s 2023-09-10T07:42:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:12 smithi098 ceph-mon[103124]: osdmap e1084: 12 total, 12 up, 12 in 2023-09-10T07:42:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30164.527268 secs 2023-09-10T07:42:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:14 smithi170 ceph-mon[102832]: pgmap v16248: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 192 KiB/s wr, 36 op/s 2023-09-10T07:42:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30164.527268 secs 2023-09-10T07:42:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:14 smithi082 ceph-mon[98694]: pgmap v16248: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 192 KiB/s wr, 36 op/s 2023-09-10T07:42:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30164.527268 secs 2023-09-10T07:42:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:14 smithi098 ceph-mon[103124]: pgmap v16248: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 192 KiB/s wr, 36 op/s 2023-09-10T07:42:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30165.662359 secs 2023-09-10T07:42:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30166.138606 secs 2023-09-10T07:42:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30165.662359 secs 2023-09-10T07:42:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30166.138606 secs 2023-09-10T07:42:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30165.662359 secs 2023-09-10T07:42:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30166.138606 secs 2023-09-10T07:42:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:16 smithi170 ceph-mon[102832]: pgmap v16249: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 124 KiB/s wr, 7 op/s 2023-09-10T07:42:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:16 smithi082 ceph-mon[98694]: pgmap v16249: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 124 KiB/s wr, 7 op/s 2023-09-10T07:42:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:16 smithi098 ceph-mon[103124]: pgmap v16249: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 124 KiB/s wr, 7 op/s 2023-09-10T07:42:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:18 smithi170 ceph-mon[102832]: pgmap v16250: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T07:42:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:18 smithi082 ceph-mon[98694]: pgmap v16250: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T07:42:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:18 smithi098 ceph-mon[103124]: pgmap v16250: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 109 KiB/s wr, 12 op/s 2023-09-10T07:42:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30169.527330 secs 2023-09-10T07:42:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30169.527330 secs 2023-09-10T07:42:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30169.527330 secs 2023-09-10T07:42:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30170.662445 secs 2023-09-10T07:42:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30171.138672 secs 2023-09-10T07:42:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:20 smithi170 ceph-mon[102832]: pgmap v16251: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T07:42:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30170.662445 secs 2023-09-10T07:42:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30171.138672 secs 2023-09-10T07:42:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:20 smithi082 ceph-mon[98694]: pgmap v16251: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T07:42:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30170.662445 secs 2023-09-10T07:42:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30171.138672 secs 2023-09-10T07:42:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:20 smithi098 ceph-mon[103124]: pgmap v16251: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T07:42:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:22 smithi170 ceph-mon[102832]: pgmap v16252: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T07:42:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:22 smithi082 ceph-mon[98694]: pgmap v16252: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T07:42:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:22 smithi098 ceph-mon[103124]: pgmap v16252: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T07:42:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30174.527400 secs 2023-09-10T07:42:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:24 smithi170 ceph-mon[102832]: pgmap v16253: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T07:42:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:42:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30174.527400 secs 2023-09-10T07:42:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:24 smithi082 ceph-mon[98694]: pgmap v16253: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T07:42:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:42:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30174.527400 secs 2023-09-10T07:42:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:24 smithi098 ceph-mon[103124]: pgmap v16253: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T07:42:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:42:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30175.662496 secs 2023-09-10T07:42:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30176.138741 secs 2023-09-10T07:42:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:42:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:42:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:42:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30175.662496 secs 2023-09-10T07:42:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30176.138741 secs 2023-09-10T07:42:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:42:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:42:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:42:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30175.662496 secs 2023-09-10T07:42:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30176.138741 secs 2023-09-10T07:42:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:42:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:42:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:42:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:26 smithi170 ceph-mon[102832]: pgmap v16254: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-10T07:42:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:26 smithi082 ceph-mon[98694]: pgmap v16254: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-10T07:42:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:26 smithi098 ceph-mon[103124]: pgmap v16254: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-10T07:42:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:28 smithi170 ceph-mon[102832]: pgmap v16255: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 13 op/s 2023-09-10T07:42:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:28 smithi082 ceph-mon[98694]: pgmap v16255: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 13 op/s 2023-09-10T07:42:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:28 smithi098 ceph-mon[103124]: pgmap v16255: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 13 op/s 2023-09-10T07:42:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30179.527498 secs 2023-09-10T07:42:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30179.527498 secs 2023-09-10T07:42:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30179.527498 secs 2023-09-10T07:42:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30180.662554 secs 2023-09-10T07:42:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30181.138802 secs 2023-09-10T07:42:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:30 smithi170 ceph-mon[102832]: pgmap v16256: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-10T07:42:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30180.662554 secs 2023-09-10T07:42:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30181.138802 secs 2023-09-10T07:42:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:30 smithi082 ceph-mon[98694]: pgmap v16256: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-10T07:42:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30180.662554 secs 2023-09-10T07:42:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30181.138802 secs 2023-09-10T07:42:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:30 smithi098 ceph-mon[103124]: pgmap v16256: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-10T07:42:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:32 smithi170 ceph-mon[102832]: pgmap v16257: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 14 op/s 2023-09-10T07:42:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:32 smithi082 ceph-mon[98694]: pgmap v16257: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 14 op/s 2023-09-10T07:42:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:32 smithi098 ceph-mon[103124]: pgmap v16257: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 14 op/s 2023-09-10T07:42:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:34 smithi170 ceph-mon[102832]: pgmap v16258: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2023-09-10T07:42:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:34 smithi082 ceph-mon[98694]: pgmap v16258: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2023-09-10T07:42:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:34 smithi098 ceph-mon[103124]: pgmap v16258: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2023-09-10T07:42:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30185.662673 secs 2023-09-10T07:42:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30186.138862 secs 2023-09-10T07:42:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:42:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30185.662673 secs 2023-09-10T07:42:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30186.138862 secs 2023-09-10T07:42:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:42:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30185.662673 secs 2023-09-10T07:42:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30186.138862 secs 2023-09-10T07:42:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:42:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:36 smithi170 ceph-mon[102832]: pgmap v16259: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2023-09-10T07:42:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:42:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:42:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:42:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:36 smithi082 ceph-mon[98694]: pgmap v16259: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2023-09-10T07:42:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:42:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:42:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:42:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:36 smithi098 ceph-mon[103124]: pgmap v16259: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2023-09-10T07:42:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:42:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:42:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:42:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:38 smithi170 ceph-mon[102832]: pgmap v16260: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 236 KiB/s wr, 16 op/s 2023-09-10T07:42:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:38 smithi082 ceph-mon[98694]: pgmap v16260: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 236 KiB/s wr, 16 op/s 2023-09-10T07:42:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:38 smithi098 ceph-mon[103124]: pgmap v16260: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 236 KiB/s wr, 16 op/s 2023-09-10T07:42:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:39 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30189.527682 secs 2023-09-10T07:42:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30190.662733 secs 2023-09-10T07:42:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:39 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30189.527682 secs 2023-09-10T07:42:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30190.662733 secs 2023-09-10T07:42:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:39 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30189.527682 secs 2023-09-10T07:42:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30190.662733 secs 2023-09-10T07:42:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30191.138922 secs 2023-09-10T07:42:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:42:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:42:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:40 smithi170 ceph-mon[102832]: pgmap v16261: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 236 KiB/s wr, 11 op/s 2023-09-10T07:42:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30191.138922 secs 2023-09-10T07:42:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:42:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:42:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:40 smithi082 ceph-mon[98694]: pgmap v16261: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 236 KiB/s wr, 11 op/s 2023-09-10T07:42:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30191.138922 secs 2023-09-10T07:42:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:42:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:42:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:40 smithi098 ceph-mon[103124]: pgmap v16261: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 236 KiB/s wr, 11 op/s 2023-09-10T07:42:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:42 smithi170 ceph-mon[102832]: pgmap v16262: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 462 KiB/s wr, 17 op/s 2023-09-10T07:42:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:42 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:42:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:42 smithi082 ceph-mon[98694]: pgmap v16262: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 462 KiB/s wr, 17 op/s 2023-09-10T07:42:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:42 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:42:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:42 smithi098 ceph-mon[103124]: pgmap v16262: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 462 KiB/s wr, 17 op/s 2023-09-10T07:42:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:42 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:42:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:44 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30194.527793 secs 2023-09-10T07:42:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:44 smithi170 ceph-mon[102832]: pgmap v16263: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 462 KiB/s wr, 12 op/s 2023-09-10T07:42:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30195.662795 secs 2023-09-10T07:42:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:44 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30194.527793 secs 2023-09-10T07:42:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:44 smithi082 ceph-mon[98694]: pgmap v16263: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 462 KiB/s wr, 12 op/s 2023-09-10T07:42:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30195.662795 secs 2023-09-10T07:42:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:44 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30194.527793 secs 2023-09-10T07:42:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:44 smithi098 ceph-mon[103124]: pgmap v16263: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 462 KiB/s wr, 12 op/s 2023-09-10T07:42:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30195.662795 secs 2023-09-10T07:42:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30196.138979 secs 2023-09-10T07:42:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30196.138979 secs 2023-09-10T07:42:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30196.138979 secs 2023-09-10T07:42:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:46 smithi170 ceph-mon[102832]: pgmap v16264: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 462 KiB/s wr, 13 op/s 2023-09-10T07:42:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:46 smithi082 ceph-mon[98694]: pgmap v16264: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 462 KiB/s wr, 13 op/s 2023-09-10T07:42:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:46 smithi098 ceph-mon[103124]: pgmap v16264: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 462 KiB/s wr, 13 op/s 2023-09-10T07:42:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:48 smithi170 ceph-mon[102832]: pgmap v16265: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 746 KiB/s wr, 20 op/s 2023-09-10T07:42:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:48 smithi082 ceph-mon[98694]: pgmap v16265: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 746 KiB/s wr, 20 op/s 2023-09-10T07:42:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:48 smithi098 ceph-mon[103124]: pgmap v16265: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 746 KiB/s wr, 20 op/s 2023-09-10T07:42:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30200.662850 secs 2023-09-10T07:42:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30200.662850 secs 2023-09-10T07:42:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30200.662850 secs 2023-09-10T07:42:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30201.139042 secs 2023-09-10T07:42:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:50 smithi170 ceph-mon[102832]: pgmap v16266: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 510 KiB/s wr, 14 op/s 2023-09-10T07:42:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30201.139042 secs 2023-09-10T07:42:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:50 smithi082 ceph-mon[98694]: pgmap v16266: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 510 KiB/s wr, 14 op/s 2023-09-10T07:42:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30201.139042 secs 2023-09-10T07:42:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:50 smithi098 ceph-mon[103124]: pgmap v16266: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 510 KiB/s wr, 14 op/s 2023-09-10T07:42:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:51 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:42:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:51 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:42:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:51 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:42:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:52 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:42:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:42:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:42:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:52 smithi170 ceph-mon[102832]: pgmap v16267: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 654 KiB/s wr, 20 op/s 2023-09-10T07:42:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:52 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:42:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:42:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:42:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:52 smithi082 ceph-mon[98694]: pgmap v16267: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 654 KiB/s wr, 20 op/s 2023-09-10T07:42:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:52 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:42:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:42:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:42:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:52 smithi098 ceph-mon[103124]: pgmap v16267: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 654 KiB/s wr, 20 op/s 2023-09-10T07:42:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30204.527960 secs 2023-09-10T07:42:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:54 smithi170 ceph-mon[102832]: pgmap v16268: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 428 KiB/s wr, 14 op/s 2023-09-10T07:42:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30205.662908 secs 2023-09-10T07:42:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30204.527960 secs 2023-09-10T07:42:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:54 smithi082 ceph-mon[98694]: pgmap v16268: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 428 KiB/s wr, 14 op/s 2023-09-10T07:42:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30205.662908 secs 2023-09-10T07:42:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30204.527960 secs 2023-09-10T07:42:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:54 smithi098 ceph-mon[103124]: pgmap v16268: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 428 KiB/s wr, 14 op/s 2023-09-10T07:42:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30205.662908 secs 2023-09-10T07:42:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30206.139102 secs 2023-09-10T07:42:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30206.139102 secs 2023-09-10T07:42:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30206.139102 secs 2023-09-10T07:42:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:56 smithi170 ceph-mon[102832]: pgmap v16269: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 428 KiB/s wr, 14 op/s 2023-09-10T07:42:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:42:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:42:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:56 smithi082 ceph-mon[98694]: pgmap v16269: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 428 KiB/s wr, 14 op/s 2023-09-10T07:42:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:42:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:42:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:56 smithi098 ceph-mon[103124]: pgmap v16269: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 428 KiB/s wr, 14 op/s 2023-09-10T07:42:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:42:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:42:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:57 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:42:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:57 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:42:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:57 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:42:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:58 smithi170 ceph-mon[102832]: pgmap v16270: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 575 KiB/s wr, 19 op/s 2023-09-10T07:42:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:58 smithi082 ceph-mon[98694]: pgmap v16270: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 575 KiB/s wr, 19 op/s 2023-09-10T07:42:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:58 smithi098 ceph-mon[103124]: pgmap v16270: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 575 KiB/s wr, 19 op/s 2023-09-10T07:42:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30209.528021 secs 2023-09-10T07:42:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:42:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30210.663000 secs 2023-09-10T07:42:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30209.528021 secs 2023-09-10T07:42:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:42:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30210.663000 secs 2023-09-10T07:42:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30209.528021 secs 2023-09-10T07:42:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:42:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30210.663000 secs 2023-09-10T07:43:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30211.139162 secs 2023-09-10T07:43:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:00 smithi170 ceph-mon[102832]: pgmap v16271: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 291 KiB/s wr, 11 op/s 2023-09-10T07:43:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30211.139162 secs 2023-09-10T07:43:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:00 smithi082 ceph-mon[98694]: pgmap v16271: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 291 KiB/s wr, 11 op/s 2023-09-10T07:43:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30211.139162 secs 2023-09-10T07:43:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:00 smithi098 ceph-mon[103124]: pgmap v16271: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 291 KiB/s wr, 11 op/s 2023-09-10T07:43:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:02 smithi170 ceph-mon[102832]: pgmap v16272: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 409 KiB/s wr, 17 op/s 2023-09-10T07:43:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:02 smithi082 ceph-mon[98694]: pgmap v16272: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 409 KiB/s wr, 17 op/s 2023-09-10T07:43:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:02 smithi098 ceph-mon[103124]: pgmap v16272: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 409 KiB/s wr, 17 op/s 2023-09-10T07:43:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30214.528081 secs 2023-09-10T07:43:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:04 smithi170 ceph-mon[102832]: pgmap v16273: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 10 op/s 2023-09-10T07:43:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30215.663053 secs 2023-09-10T07:43:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30214.528081 secs 2023-09-10T07:43:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:04 smithi082 ceph-mon[98694]: pgmap v16273: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 10 op/s 2023-09-10T07:43:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30215.663053 secs 2023-09-10T07:43:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30214.528081 secs 2023-09-10T07:43:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:04 smithi098 ceph-mon[103124]: pgmap v16273: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 10 op/s 2023-09-10T07:43:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30215.663053 secs 2023-09-10T07:43:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30216.139220 secs 2023-09-10T07:43:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30216.139220 secs 2023-09-10T07:43:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30216.139220 secs 2023-09-10T07:43:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:06 smithi170 ceph-mon[102832]: pgmap v16274: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T07:43:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:06 smithi082 ceph-mon[98694]: pgmap v16274: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T07:43:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:06 smithi098 ceph-mon[103124]: pgmap v16274: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T07:43:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:08 smithi170 ceph-mon[102832]: pgmap v16275: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 310 KiB/s wr, 29 op/s 2023-09-10T07:43:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:08 smithi082 ceph-mon[98694]: pgmap v16275: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 310 KiB/s wr, 29 op/s 2023-09-10T07:43:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:08 smithi098 ceph-mon[103124]: pgmap v16275: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 310 KiB/s wr, 29 op/s 2023-09-10T07:43:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30219.528142 secs 2023-09-10T07:43:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:09 smithi170 ceph-mon[102832]: osdmap e1085: 12 total, 12 up, 12 in 2023-09-10T07:43:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30220.663136 secs 2023-09-10T07:43:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30219.528142 secs 2023-09-10T07:43:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:09 smithi082 ceph-mon[98694]: osdmap e1085: 12 total, 12 up, 12 in 2023-09-10T07:43:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30220.663136 secs 2023-09-10T07:43:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30219.528142 secs 2023-09-10T07:43:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:09 smithi098 ceph-mon[103124]: osdmap e1085: 12 total, 12 up, 12 in 2023-09-10T07:43:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30220.663136 secs 2023-09-10T07:43:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30221.139281 secs 2023-09-10T07:43:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:10 smithi170 ceph-mon[102832]: pgmap v16277: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 195 KiB/s wr, 29 op/s 2023-09-10T07:43:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30221.139281 secs 2023-09-10T07:43:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:10 smithi082 ceph-mon[98694]: pgmap v16277: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 195 KiB/s wr, 29 op/s 2023-09-10T07:43:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30221.139281 secs 2023-09-10T07:43:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:10 smithi098 ceph-mon[103124]: pgmap v16277: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 195 KiB/s wr, 29 op/s 2023-09-10T07:43:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:12 smithi170 ceph-mon[102832]: pgmap v16278: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:43:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:12 smithi170 ceph-mon[102832]: osdmap e1086: 12 total, 12 up, 12 in 2023-09-10T07:43:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:12 smithi082 ceph-mon[98694]: pgmap v16278: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:43:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:12 smithi082 ceph-mon[98694]: osdmap e1086: 12 total, 12 up, 12 in 2023-09-10T07:43:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:12 smithi098 ceph-mon[103124]: pgmap v16278: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:43:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:12 smithi098 ceph-mon[103124]: osdmap e1086: 12 total, 12 up, 12 in 2023-09-10T07:43:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30224.528206 secs 2023-09-10T07:43:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:14 smithi170 ceph-mon[102832]: pgmap v16280: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 68 KiB/s wr, 34 op/s 2023-09-10T07:43:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30225.663212 secs 2023-09-10T07:43:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30224.528206 secs 2023-09-10T07:43:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:14 smithi082 ceph-mon[98694]: pgmap v16280: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 68 KiB/s wr, 34 op/s 2023-09-10T07:43:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30225.663212 secs 2023-09-10T07:43:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30224.528206 secs 2023-09-10T07:43:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:14 smithi098 ceph-mon[103124]: pgmap v16280: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 68 KiB/s wr, 34 op/s 2023-09-10T07:43:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30225.663212 secs 2023-09-10T07:43:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30226.139339 secs 2023-09-10T07:43:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30226.139339 secs 2023-09-10T07:43:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30226.139339 secs 2023-09-10T07:43:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:16 smithi170 ceph-mon[102832]: pgmap v16281: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:43:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:16 smithi082 ceph-mon[98694]: pgmap v16281: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:43:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:16 smithi098 ceph-mon[103124]: pgmap v16281: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:43:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:18 smithi170 ceph-mon[102832]: pgmap v16282: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:43:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:18 smithi082 ceph-mon[98694]: pgmap v16282: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:43:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:18 smithi098 ceph-mon[103124]: pgmap v16282: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:43:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30229.528309 secs 2023-09-10T07:43:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30230.663271 secs 2023-09-10T07:43:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30229.528309 secs 2023-09-10T07:43:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30230.663271 secs 2023-09-10T07:43:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30229.528309 secs 2023-09-10T07:43:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30230.663271 secs 2023-09-10T07:43:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30231.139399 secs 2023-09-10T07:43:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:20 smithi170 ceph-mon[102832]: pgmap v16283: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:43:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30231.139399 secs 2023-09-10T07:43:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:20 smithi082 ceph-mon[98694]: pgmap v16283: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:43:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30231.139399 secs 2023-09-10T07:43:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:20 smithi098 ceph-mon[103124]: pgmap v16283: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:43:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:22 smithi170 ceph-mon[102832]: pgmap v16284: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:43:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:22 smithi082 ceph-mon[98694]: pgmap v16284: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:43:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:22 smithi098 ceph-mon[103124]: pgmap v16284: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:43:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30234.528412 secs 2023-09-10T07:43:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:24 smithi170 ceph-mon[102832]: pgmap v16285: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:43:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30235.663355 secs 2023-09-10T07:43:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30234.528412 secs 2023-09-10T07:43:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:24 smithi082 ceph-mon[98694]: pgmap v16285: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:43:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30235.663355 secs 2023-09-10T07:43:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30234.528412 secs 2023-09-10T07:43:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:24 smithi098 ceph-mon[103124]: pgmap v16285: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:43:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30235.663355 secs 2023-09-10T07:43:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30236.139457 secs 2023-09-10T07:43:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:43:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:43:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:43:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:43:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30236.139457 secs 2023-09-10T07:43:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:43:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:43:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:43:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:43:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30236.139457 secs 2023-09-10T07:43:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:43:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:43:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:43:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:43:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:26 smithi170 ceph-mon[102832]: pgmap v16286: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-10T07:43:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:26 smithi082 ceph-mon[98694]: pgmap v16286: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-10T07:43:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:26 smithi098 ceph-mon[103124]: pgmap v16286: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-10T07:43:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:28 smithi170 ceph-mon[102832]: pgmap v16287: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T07:43:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:28 smithi082 ceph-mon[98694]: pgmap v16287: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T07:43:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:28 smithi098 ceph-mon[103124]: pgmap v16287: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 101 KiB/s wr, 12 op/s 2023-09-10T07:43:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30239.528471 secs 2023-09-10T07:43:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30240.663439 secs 2023-09-10T07:43:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30239.528471 secs 2023-09-10T07:43:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30240.663439 secs 2023-09-10T07:43:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30239.528471 secs 2023-09-10T07:43:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30240.663439 secs 2023-09-10T07:43:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30241.139488 secs 2023-09-10T07:43:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:30 smithi170 ceph-mon[102832]: pgmap v16288: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 8 op/s 2023-09-10T07:43:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30241.139488 secs 2023-09-10T07:43:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:30 smithi082 ceph-mon[98694]: pgmap v16288: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 8 op/s 2023-09-10T07:43:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30241.139488 secs 2023-09-10T07:43:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:30 smithi098 ceph-mon[103124]: pgmap v16288: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 100 KiB/s wr, 8 op/s 2023-09-10T07:43:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:32 smithi170 ceph-mon[102832]: pgmap v16289: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 195 KiB/s wr, 13 op/s 2023-09-10T07:43:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:32 smithi082 ceph-mon[98694]: pgmap v16289: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 195 KiB/s wr, 13 op/s 2023-09-10T07:43:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:32 smithi098 ceph-mon[103124]: pgmap v16289: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 195 KiB/s wr, 13 op/s 2023-09-10T07:43:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:34 smithi170 ceph-mon[102832]: pgmap v16290: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 9 op/s 2023-09-10T07:43:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30245.663492 secs 2023-09-10T07:43:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:34 smithi082 ceph-mon[98694]: pgmap v16290: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 9 op/s 2023-09-10T07:43:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30245.663492 secs 2023-09-10T07:43:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:34 smithi098 ceph-mon[103124]: pgmap v16290: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 9 op/s 2023-09-10T07:43:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30245.663492 secs 2023-09-10T07:43:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30246.139548 secs 2023-09-10T07:43:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:43:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30246.139548 secs 2023-09-10T07:43:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:43:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30246.139548 secs 2023-09-10T07:43:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:43:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:36 smithi170 ceph-mon[102832]: pgmap v16291: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 10 op/s 2023-09-10T07:43:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:43:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:43:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:43:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:36 smithi082 ceph-mon[98694]: pgmap v16291: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 10 op/s 2023-09-10T07:43:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:43:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:43:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:43:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:36 smithi098 ceph-mon[103124]: pgmap v16291: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 10 op/s 2023-09-10T07:43:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:43:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:43:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:43:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:38 smithi170 ceph-mon[102832]: pgmap v16292: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 338 KiB/s wr, 15 op/s 2023-09-10T07:43:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:38 smithi082 ceph-mon[98694]: pgmap v16292: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 338 KiB/s wr, 15 op/s 2023-09-10T07:43:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:38 smithi098 ceph-mon[103124]: pgmap v16292: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 338 KiB/s wr, 15 op/s 2023-09-10T07:43:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30250.663551 secs 2023-09-10T07:43:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30250.663551 secs 2023-09-10T07:43:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30250.663551 secs 2023-09-10T07:43:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30251.139610 secs 2023-09-10T07:43:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:40 smithi170 ceph-mon[102832]: pgmap v16293: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 238 KiB/s wr, 11 op/s 2023-09-10T07:43:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30251.139610 secs 2023-09-10T07:43:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:40 smithi082 ceph-mon[98694]: pgmap v16293: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 238 KiB/s wr, 11 op/s 2023-09-10T07:43:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30251.139610 secs 2023-09-10T07:43:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:40 smithi098 ceph-mon[103124]: pgmap v16293: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 238 KiB/s wr, 11 op/s 2023-09-10T07:43:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:42 smithi170 ceph-mon[102832]: pgmap v16294: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 483 KiB/s wr, 17 op/s 2023-09-10T07:43:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:42 smithi082 ceph-mon[98694]: pgmap v16294: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 483 KiB/s wr, 17 op/s 2023-09-10T07:43:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:42 smithi098 ceph-mon[103124]: pgmap v16294: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 483 KiB/s wr, 17 op/s 2023-09-10T07:43:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:43:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:43:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:43:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:43:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:43:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:43:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:43:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:43:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:43:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:44 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30254.528675 secs 2023-09-10T07:43:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:44 smithi170 ceph-mon[102832]: pgmap v16295: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 388 KiB/s wr, 12 op/s 2023-09-10T07:43:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30255.663635 secs 2023-09-10T07:43:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:44 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30254.528675 secs 2023-09-10T07:43:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:44 smithi082 ceph-mon[98694]: pgmap v16295: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 388 KiB/s wr, 12 op/s 2023-09-10T07:43:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30255.663635 secs 2023-09-10T07:43:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:44 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30254.528675 secs 2023-09-10T07:43:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:44 smithi098 ceph-mon[103124]: pgmap v16295: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 388 KiB/s wr, 12 op/s 2023-09-10T07:43:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30255.663635 secs 2023-09-10T07:43:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30256.139669 secs 2023-09-10T07:43:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30256.139669 secs 2023-09-10T07:43:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30256.139669 secs 2023-09-10T07:43:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:46 smithi170 ceph-mon[102832]: pgmap v16296: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 388 KiB/s wr, 13 op/s 2023-09-10T07:43:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:46 smithi082 ceph-mon[98694]: pgmap v16296: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 388 KiB/s wr, 13 op/s 2023-09-10T07:43:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:46 smithi098 ceph-mon[103124]: pgmap v16296: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 388 KiB/s wr, 13 op/s 2023-09-10T07:43:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:48 smithi170 ceph-mon[102832]: pgmap v16297: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 655 KiB/s wr, 18 op/s 2023-09-10T07:43:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:48 smithi082 ceph-mon[98694]: pgmap v16297: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 655 KiB/s wr, 18 op/s 2023-09-10T07:43:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:48 smithi098 ceph-mon[103124]: pgmap v16297: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 655 KiB/s wr, 18 op/s 2023-09-10T07:43:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30260.663720 secs 2023-09-10T07:43:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30260.663720 secs 2023-09-10T07:43:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30260.663720 secs 2023-09-10T07:43:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30261.139730 secs 2023-09-10T07:43:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:50 smithi170 ceph-mon[102832]: pgmap v16298: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 512 KiB/s wr, 13 op/s 2023-09-10T07:43:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30261.139730 secs 2023-09-10T07:43:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:50 smithi082 ceph-mon[98694]: pgmap v16298: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 512 KiB/s wr, 13 op/s 2023-09-10T07:43:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30261.139730 secs 2023-09-10T07:43:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:50 smithi098 ceph-mon[103124]: pgmap v16298: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 512 KiB/s wr, 13 op/s 2023-09-10T07:43:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:51 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:43:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:51 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:43:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:51 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:43:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:52 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:43:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:43:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:43:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:52 smithi170 ceph-mon[102832]: pgmap v16299: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 690 KiB/s wr, 20 op/s 2023-09-10T07:43:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:52 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:43:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:43:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:43:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:52 smithi082 ceph-mon[98694]: pgmap v16299: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 690 KiB/s wr, 20 op/s 2023-09-10T07:43:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:52 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:43:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:43:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:43:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:52 smithi098 ceph-mon[103124]: pgmap v16299: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 690 KiB/s wr, 20 op/s 2023-09-10T07:43:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30264.528776 secs 2023-09-10T07:43:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:54 smithi170 ceph-mon[102832]: pgmap v16300: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 446 KiB/s wr, 13 op/s 2023-09-10T07:43:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30265.663818 secs 2023-09-10T07:43:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30264.528776 secs 2023-09-10T07:43:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:54 smithi082 ceph-mon[98694]: pgmap v16300: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 446 KiB/s wr, 13 op/s 2023-09-10T07:43:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30265.663818 secs 2023-09-10T07:43:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30264.528776 secs 2023-09-10T07:43:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:54 smithi098 ceph-mon[103124]: pgmap v16300: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 446 KiB/s wr, 13 op/s 2023-09-10T07:43:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30265.663818 secs 2023-09-10T07:43:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30266.139793 secs 2023-09-10T07:43:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30266.139793 secs 2023-09-10T07:43:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30266.139793 secs 2023-09-10T07:43:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:56 smithi170 ceph-mon[102832]: pgmap v16301: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 594 KiB/s wr, 19 op/s 2023-09-10T07:43:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:43:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:43:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:56 smithi082 ceph-mon[98694]: pgmap v16301: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 594 KiB/s wr, 19 op/s 2023-09-10T07:43:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:43:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:43:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:56 smithi098 ceph-mon[103124]: pgmap v16301: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 594 KiB/s wr, 19 op/s 2023-09-10T07:43:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:43:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:43:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:57 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:43:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:57 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:43:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:57 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:43:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:58 smithi170 ceph-mon[102832]: pgmap v16302: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 594 KiB/s wr, 19 op/s 2023-09-10T07:43:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:58 smithi082 ceph-mon[98694]: pgmap v16302: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 594 KiB/s wr, 19 op/s 2023-09-10T07:43:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:58 smithi098 ceph-mon[103124]: pgmap v16302: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 594 KiB/s wr, 19 op/s 2023-09-10T07:43:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30269.528836 secs 2023-09-10T07:43:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:43:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30270.663877 secs 2023-09-10T07:43:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30269.528836 secs 2023-09-10T07:43:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:43:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30270.663877 secs 2023-09-10T07:43:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30269.528836 secs 2023-09-10T07:43:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:43:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30270.663877 secs 2023-09-10T07:44:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30271.139860 secs 2023-09-10T07:44:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:00 smithi170 ceph-mon[102832]: pgmap v16303: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 327 KiB/s wr, 12 op/s 2023-09-10T07:44:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30271.139860 secs 2023-09-10T07:44:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:00 smithi082 ceph-mon[98694]: pgmap v16303: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 327 KiB/s wr, 12 op/s 2023-09-10T07:44:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30271.139860 secs 2023-09-10T07:44:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:00 smithi098 ceph-mon[103124]: pgmap v16303: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 327 KiB/s wr, 12 op/s 2023-09-10T07:44:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:02 smithi170 ceph-mon[102832]: pgmap v16304: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 490 KiB/s wr, 32 op/s 2023-09-10T07:44:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:02 smithi082 ceph-mon[98694]: pgmap v16304: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 490 KiB/s wr, 32 op/s 2023-09-10T07:44:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:02 smithi098 ceph-mon[103124]: pgmap v16304: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 490 KiB/s wr, 32 op/s 2023-09-10T07:44:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30274.528903 secs 2023-09-10T07:44:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:04 smithi170 ceph-mon[102832]: pgmap v16305: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 311 KiB/s wr, 25 op/s 2023-09-10T07:44:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30275.663935 secs 2023-09-10T07:44:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30274.528903 secs 2023-09-10T07:44:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:04 smithi082 ceph-mon[98694]: pgmap v16305: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 311 KiB/s wr, 25 op/s 2023-09-10T07:44:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30275.663935 secs 2023-09-10T07:44:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30274.528903 secs 2023-09-10T07:44:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:04 smithi098 ceph-mon[103124]: pgmap v16305: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 311 KiB/s wr, 25 op/s 2023-09-10T07:44:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30275.663935 secs 2023-09-10T07:44:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30276.139916 secs 2023-09-10T07:44:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30276.139916 secs 2023-09-10T07:44:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30276.139916 secs 2023-09-10T07:44:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:06 smithi170 ceph-mon[102832]: pgmap v16306: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 311 KiB/s wr, 30 op/s 2023-09-10T07:44:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:06 smithi082 ceph-mon[98694]: pgmap v16306: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 311 KiB/s wr, 30 op/s 2023-09-10T07:44:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:06 smithi098 ceph-mon[103124]: pgmap v16306: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 311 KiB/s wr, 30 op/s 2023-09-10T07:44:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:08 smithi170 ceph-mon[102832]: pgmap v16307: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 163 KiB/s wr, 24 op/s 2023-09-10T07:44:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:08 smithi082 ceph-mon[98694]: pgmap v16307: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 163 KiB/s wr, 24 op/s 2023-09-10T07:44:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:08 smithi098 ceph-mon[103124]: pgmap v16307: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 163 KiB/s wr, 24 op/s 2023-09-10T07:44:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30279.528964 secs 2023-09-10T07:44:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:09 smithi170 ceph-mon[102832]: osdmap e1087: 12 total, 12 up, 12 in 2023-09-10T07:44:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30280.663992 secs 2023-09-10T07:44:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30279.528964 secs 2023-09-10T07:44:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:09 smithi082 ceph-mon[98694]: osdmap e1087: 12 total, 12 up, 12 in 2023-09-10T07:44:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30280.663992 secs 2023-09-10T07:44:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30279.528964 secs 2023-09-10T07:44:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:09 smithi098 ceph-mon[103124]: osdmap e1087: 12 total, 12 up, 12 in 2023-09-10T07:44:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30280.663992 secs 2023-09-10T07:44:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30281.139979 secs 2023-09-10T07:44:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:10 smithi170 ceph-mon[102832]: pgmap v16309: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 196 KiB/s wr, 29 op/s 2023-09-10T07:44:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30281.139979 secs 2023-09-10T07:44:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:10 smithi082 ceph-mon[98694]: pgmap v16309: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 196 KiB/s wr, 29 op/s 2023-09-10T07:44:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30281.139979 secs 2023-09-10T07:44:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:10 smithi098 ceph-mon[103124]: pgmap v16309: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 196 KiB/s wr, 29 op/s 2023-09-10T07:44:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:12 smithi170 ceph-mon[102832]: pgmap v16310: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T07:44:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:12 smithi170 ceph-mon[102832]: osdmap e1088: 12 total, 12 up, 12 in 2023-09-10T07:44:12.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:12 smithi082 ceph-mon[98694]: pgmap v16310: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T07:44:12.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:12 smithi082 ceph-mon[98694]: osdmap e1088: 12 total, 12 up, 12 in 2023-09-10T07:44:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:12 smithi098 ceph-mon[103124]: pgmap v16310: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 100 KiB/s wr, 11 op/s 2023-09-10T07:44:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:12 smithi098 ceph-mon[103124]: osdmap e1088: 12 total, 12 up, 12 in 2023-09-10T07:44:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30284.529077 secs 2023-09-10T07:44:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:14 smithi170 ceph-mon[102832]: pgmap v16312: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 7 op/s 2023-09-10T07:44:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30284.529077 secs 2023-09-10T07:44:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:14 smithi082 ceph-mon[98694]: pgmap v16312: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 7 op/s 2023-09-10T07:44:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30284.529077 secs 2023-09-10T07:44:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:14 smithi098 ceph-mon[103124]: pgmap v16312: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 7 op/s 2023-09-10T07:44:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30285.694569 secs 2023-09-10T07:44:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30286.140039 secs 2023-09-10T07:44:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30285.694569 secs 2023-09-10T07:44:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30286.140039 secs 2023-09-10T07:44:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30285.694569 secs 2023-09-10T07:44:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30286.140039 secs 2023-09-10T07:44:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:16 smithi170 ceph-mon[102832]: pgmap v16313: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 130 KiB/s wr, 13 op/s 2023-09-10T07:44:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:16 smithi082 ceph-mon[98694]: pgmap v16313: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 130 KiB/s wr, 13 op/s 2023-09-10T07:44:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:16 smithi098 ceph-mon[103124]: pgmap v16313: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 130 KiB/s wr, 13 op/s 2023-09-10T07:44:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:18 smithi170 ceph-mon[102832]: pgmap v16314: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T07:44:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:18 smithi082 ceph-mon[98694]: pgmap v16314: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T07:44:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:18 smithi098 ceph-mon[103124]: pgmap v16314: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 113 KiB/s wr, 12 op/s 2023-09-10T07:44:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30289.529137 secs 2023-09-10T07:44:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30290.694629 secs 2023-09-10T07:44:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30289.529137 secs 2023-09-10T07:44:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30290.694629 secs 2023-09-10T07:44:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30289.529137 secs 2023-09-10T07:44:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30290.694629 secs 2023-09-10T07:44:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30291.140098 secs 2023-09-10T07:44:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:20 smithi170 ceph-mon[102832]: pgmap v16315: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T07:44:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30291.140098 secs 2023-09-10T07:44:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:20 smithi082 ceph-mon[98694]: pgmap v16315: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T07:44:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30291.140098 secs 2023-09-10T07:44:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:20 smithi098 ceph-mon[103124]: pgmap v16315: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 104 KiB/s wr, 11 op/s 2023-09-10T07:44:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:22 smithi170 ceph-mon[102832]: pgmap v16316: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2023-09-10T07:44:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:22 smithi082 ceph-mon[98694]: pgmap v16316: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2023-09-10T07:44:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:22 smithi098 ceph-mon[103124]: pgmap v16316: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2023-09-10T07:44:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30294.529195 secs 2023-09-10T07:44:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:24 smithi170 ceph-mon[102832]: pgmap v16317: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.5 KiB/s wr, 10 op/s 2023-09-10T07:44:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30295.694686 secs 2023-09-10T07:44:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30294.529195 secs 2023-09-10T07:44:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:24 smithi082 ceph-mon[98694]: pgmap v16317: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.5 KiB/s wr, 10 op/s 2023-09-10T07:44:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30295.694686 secs 2023-09-10T07:44:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30294.529195 secs 2023-09-10T07:44:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:24 smithi098 ceph-mon[103124]: pgmap v16317: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.5 KiB/s wr, 10 op/s 2023-09-10T07:44:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30295.694686 secs 2023-09-10T07:44:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30296.140157 secs 2023-09-10T07:44:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:44:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30296.140157 secs 2023-09-10T07:44:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:44:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30296.140157 secs 2023-09-10T07:44:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:44:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:26 smithi170 ceph-mon[102832]: pgmap v16318: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.3 KiB/s wr, 13 op/s 2023-09-10T07:44:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:44:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:44:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:44:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:26 smithi082 ceph-mon[98694]: pgmap v16318: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.3 KiB/s wr, 13 op/s 2023-09-10T07:44:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:44:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:44:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:44:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:26 smithi098 ceph-mon[103124]: pgmap v16318: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.3 KiB/s wr, 13 op/s 2023-09-10T07:44:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:44:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:44:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:44:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:28 smithi170 ceph-mon[102832]: pgmap v16319: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-10T07:44:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:28 smithi082 ceph-mon[98694]: pgmap v16319: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-10T07:44:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:28 smithi098 ceph-mon[103124]: pgmap v16319: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-10T07:44:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30299.529257 secs 2023-09-10T07:44:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30300.694745 secs 2023-09-10T07:44:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30299.529257 secs 2023-09-10T07:44:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30300.694745 secs 2023-09-10T07:44:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30299.529257 secs 2023-09-10T07:44:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30300.694745 secs 2023-09-10T07:44:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30301.140219 secs 2023-09-10T07:44:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:30 smithi170 ceph-mon[102832]: pgmap v16320: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-10T07:44:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30301.140219 secs 2023-09-10T07:44:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:30 smithi082 ceph-mon[98694]: pgmap v16320: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-10T07:44:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30301.140219 secs 2023-09-10T07:44:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:30 smithi098 ceph-mon[103124]: pgmap v16320: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-10T07:44:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:32 smithi170 ceph-mon[102832]: pgmap v16321: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 14 op/s 2023-09-10T07:44:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:32 smithi082 ceph-mon[98694]: pgmap v16321: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 14 op/s 2023-09-10T07:44:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:32 smithi098 ceph-mon[103124]: pgmap v16321: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 14 op/s 2023-09-10T07:44:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:34 smithi170 ceph-mon[102832]: pgmap v16322: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2023-09-10T07:44:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30305.694803 secs 2023-09-10T07:44:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:34 smithi082 ceph-mon[98694]: pgmap v16322: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2023-09-10T07:44:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30305.694803 secs 2023-09-10T07:44:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:34 smithi098 ceph-mon[103124]: pgmap v16322: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2023-09-10T07:44:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30305.694803 secs 2023-09-10T07:44:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30306.140274 secs 2023-09-10T07:44:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:44:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30306.140274 secs 2023-09-10T07:44:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:44:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30306.140274 secs 2023-09-10T07:44:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:44:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:36 smithi170 ceph-mon[102832]: pgmap v16323: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 324 KiB/s wr, 15 op/s 2023-09-10T07:44:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:44:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:44:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:44:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:36 smithi082 ceph-mon[98694]: pgmap v16323: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 324 KiB/s wr, 15 op/s 2023-09-10T07:44:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:44:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:44:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:44:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:36 smithi098 ceph-mon[103124]: pgmap v16323: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 324 KiB/s wr, 15 op/s 2023-09-10T07:44:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:44:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:44:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:44:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:38 smithi170 ceph-mon[102832]: pgmap v16324: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 324 KiB/s wr, 11 op/s 2023-09-10T07:44:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:38 smithi082 ceph-mon[98694]: pgmap v16324: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 324 KiB/s wr, 11 op/s 2023-09-10T07:44:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:38 smithi098 ceph-mon[103124]: pgmap v16324: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 324 KiB/s wr, 11 op/s 2023-09-10T07:44:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30310.694877 secs 2023-09-10T07:44:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30310.694877 secs 2023-09-10T07:44:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30310.694877 secs 2023-09-10T07:44:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30311.140343 secs 2023-09-10T07:44:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:40 smithi170 ceph-mon[102832]: pgmap v16325: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 324 KiB/s wr, 11 op/s 2023-09-10T07:44:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30311.140343 secs 2023-09-10T07:44:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:40 smithi082 ceph-mon[98694]: pgmap v16325: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 324 KiB/s wr, 11 op/s 2023-09-10T07:44:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30311.140343 secs 2023-09-10T07:44:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:40 smithi098 ceph-mon[103124]: pgmap v16325: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 324 KiB/s wr, 11 op/s 2023-09-10T07:44:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:42 smithi170 ceph-mon[102832]: pgmap v16326: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 467 KiB/s wr, 18 op/s 2023-09-10T07:44:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:42 smithi082 ceph-mon[98694]: pgmap v16326: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 467 KiB/s wr, 18 op/s 2023-09-10T07:44:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:42 smithi098 ceph-mon[103124]: pgmap v16326: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 467 KiB/s wr, 18 op/s 2023-09-10T07:44:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:44:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:43 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:44:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:43 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:44:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:44:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:43 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:44:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:43 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:44:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:44:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:43 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:44:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:43 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:44:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:44 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30314.529559 secs 2023-09-10T07:44:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:44 smithi170 ceph-mon[102832]: pgmap v16327: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 466 KiB/s wr, 12 op/s 2023-09-10T07:44:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30315.694963 secs 2023-09-10T07:44:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:44 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30314.529559 secs 2023-09-10T07:44:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:44 smithi082 ceph-mon[98694]: pgmap v16327: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 466 KiB/s wr, 12 op/s 2023-09-10T07:44:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30315.694963 secs 2023-09-10T07:44:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:44 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30314.529559 secs 2023-09-10T07:44:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:44 smithi098 ceph-mon[103124]: pgmap v16327: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 466 KiB/s wr, 12 op/s 2023-09-10T07:44:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30315.694963 secs 2023-09-10T07:44:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30316.140430 secs 2023-09-10T07:44:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30316.140430 secs 2023-09-10T07:44:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30316.140430 secs 2023-09-10T07:44:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:46 smithi082 ceph-mon[98694]: pgmap v16328: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 754 KiB/s wr, 19 op/s 2023-09-10T07:44:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:46 smithi098 ceph-mon[103124]: pgmap v16328: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 754 KiB/s wr, 19 op/s 2023-09-10T07:44:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:46 smithi170 ceph-mon[102832]: pgmap v16328: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 754 KiB/s wr, 19 op/s 2023-09-10T07:44:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:48 smithi170 ceph-mon[102832]: pgmap v16329: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 431 KiB/s wr, 14 op/s 2023-09-10T07:44:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:48 smithi082 ceph-mon[98694]: pgmap v16329: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 431 KiB/s wr, 14 op/s 2023-09-10T07:44:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:48 smithi098 ceph-mon[103124]: pgmap v16329: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 431 KiB/s wr, 14 op/s 2023-09-10T07:44:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:49 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30319.529624 secs 2023-09-10T07:44:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30320.695023 secs 2023-09-10T07:44:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:49 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30319.529624 secs 2023-09-10T07:44:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30320.695023 secs 2023-09-10T07:44:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:49 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30319.529624 secs 2023-09-10T07:44:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30320.695023 secs 2023-09-10T07:44:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30321.140482 secs 2023-09-10T07:44:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:50 smithi170 ceph-mon[102832]: pgmap v16330: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 430 KiB/s wr, 14 op/s 2023-09-10T07:44:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30321.140482 secs 2023-09-10T07:44:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:50 smithi082 ceph-mon[98694]: pgmap v16330: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 430 KiB/s wr, 14 op/s 2023-09-10T07:44:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30321.140482 secs 2023-09-10T07:44:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:50 smithi098 ceph-mon[103124]: pgmap v16330: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 430 KiB/s wr, 14 op/s 2023-09-10T07:44:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:52 smithi170 ceph-mon[102832]: pgmap v16331: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 575 KiB/s wr, 20 op/s 2023-09-10T07:44:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:52 smithi082 ceph-mon[98694]: pgmap v16331: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 575 KiB/s wr, 20 op/s 2023-09-10T07:44:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:52 smithi098 ceph-mon[103124]: pgmap v16331: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 575 KiB/s wr, 20 op/s 2023-09-10T07:44:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30324.529684 secs 2023-09-10T07:44:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:54 smithi170 ceph-mon[102832]: pgmap v16332: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 432 KiB/s wr, 13 op/s 2023-09-10T07:44:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30325.695090 secs 2023-09-10T07:44:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30324.529684 secs 2023-09-10T07:44:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:54 smithi082 ceph-mon[98694]: pgmap v16332: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 432 KiB/s wr, 13 op/s 2023-09-10T07:44:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30325.695090 secs 2023-09-10T07:44:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30324.529684 secs 2023-09-10T07:44:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:54 smithi098 ceph-mon[103124]: pgmap v16332: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 432 KiB/s wr, 13 op/s 2023-09-10T07:44:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30325.695090 secs 2023-09-10T07:44:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30326.140539 secs 2023-09-10T07:44:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30326.140539 secs 2023-09-10T07:44:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30326.140539 secs 2023-09-10T07:44:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:56 smithi170 ceph-mon[102832]: pgmap v16333: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 581 KiB/s wr, 18 op/s 2023-09-10T07:44:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:56 smithi082 ceph-mon[98694]: pgmap v16333: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 581 KiB/s wr, 18 op/s 2023-09-10T07:44:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:56 smithi098 ceph-mon[103124]: pgmap v16333: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 581 KiB/s wr, 18 op/s 2023-09-10T07:44:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:58 smithi170 ceph-mon[102832]: pgmap v16334: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 293 KiB/s wr, 12 op/s 2023-09-10T07:44:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:58 smithi082 ceph-mon[98694]: pgmap v16334: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 293 KiB/s wr, 12 op/s 2023-09-10T07:44:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:58 smithi098 ceph-mon[103124]: pgmap v16334: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 293 KiB/s wr, 12 op/s 2023-09-10T07:44:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30329.529768 secs 2023-09-10T07:44:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:44:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30330.695148 secs 2023-09-10T07:44:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30329.529768 secs 2023-09-10T07:44:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:44:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30330.695148 secs 2023-09-10T07:44:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30329.529768 secs 2023-09-10T07:44:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:44:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30330.695148 secs 2023-09-10T07:45:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30331.140600 secs 2023-09-10T07:45:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:00 smithi170 ceph-mon[102832]: pgmap v16335: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T07:45:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30331.140600 secs 2023-09-10T07:45:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:00 smithi082 ceph-mon[98694]: pgmap v16335: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T07:45:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30331.140600 secs 2023-09-10T07:45:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:00 smithi098 ceph-mon[103124]: pgmap v16335: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T07:45:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:02 smithi170 ceph-mon[102832]: pgmap v16336: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 423 KiB/s wr, 20 op/s 2023-09-10T07:45:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:02 smithi082 ceph-mon[98694]: pgmap v16336: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 423 KiB/s wr, 20 op/s 2023-09-10T07:45:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:02 smithi098 ceph-mon[103124]: pgmap v16336: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 423 KiB/s wr, 20 op/s 2023-09-10T07:45:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30334.529827 secs 2023-09-10T07:45:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:04 smithi170 ceph-mon[102832]: pgmap v16337: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 279 KiB/s wr, 14 op/s 2023-09-10T07:45:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30335.695233 secs 2023-09-10T07:45:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30334.529827 secs 2023-09-10T07:45:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:04 smithi082 ceph-mon[98694]: pgmap v16337: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 279 KiB/s wr, 14 op/s 2023-09-10T07:45:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30335.695233 secs 2023-09-10T07:45:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30334.529827 secs 2023-09-10T07:45:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:04 smithi098 ceph-mon[103124]: pgmap v16337: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 279 KiB/s wr, 14 op/s 2023-09-10T07:45:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30335.695233 secs 2023-09-10T07:45:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30336.140659 secs 2023-09-10T07:45:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30336.140659 secs 2023-09-10T07:45:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30336.140659 secs 2023-09-10T07:45:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:06 smithi170 ceph-mon[102832]: pgmap v16338: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 312 KiB/s wr, 29 op/s 2023-09-10T07:45:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:06 smithi082 ceph-mon[98694]: pgmap v16338: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 312 KiB/s wr, 29 op/s 2023-09-10T07:45:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:06 smithi098 ceph-mon[103124]: pgmap v16338: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 312 KiB/s wr, 29 op/s 2023-09-10T07:45:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:08 smithi170 ceph-mon[102832]: pgmap v16339: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 163 KiB/s wr, 24 op/s 2023-09-10T07:45:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:08 smithi082 ceph-mon[98694]: pgmap v16339: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 163 KiB/s wr, 24 op/s 2023-09-10T07:45:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:08 smithi098 ceph-mon[103124]: pgmap v16339: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 163 KiB/s wr, 24 op/s 2023-09-10T07:45:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30339.529885 secs 2023-09-10T07:45:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:09 smithi170 ceph-mon[102832]: osdmap e1089: 12 total, 12 up, 12 in 2023-09-10T07:45:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30340.695289 secs 2023-09-10T07:45:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30339.529885 secs 2023-09-10T07:45:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:09 smithi082 ceph-mon[98694]: osdmap e1089: 12 total, 12 up, 12 in 2023-09-10T07:45:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30340.695289 secs 2023-09-10T07:45:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30339.529885 secs 2023-09-10T07:45:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:09 smithi098 ceph-mon[103124]: osdmap e1089: 12 total, 12 up, 12 in 2023-09-10T07:45:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30340.695289 secs 2023-09-10T07:45:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30341.140723 secs 2023-09-10T07:45:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:10 smithi170 ceph-mon[102832]: pgmap v16341: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 196 KiB/s wr, 29 op/s 2023-09-10T07:45:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30341.140723 secs 2023-09-10T07:45:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:10 smithi082 ceph-mon[98694]: pgmap v16341: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 196 KiB/s wr, 29 op/s 2023-09-10T07:45:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30341.140723 secs 2023-09-10T07:45:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:10 smithi098 ceph-mon[103124]: pgmap v16341: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 196 KiB/s wr, 29 op/s 2023-09-10T07:45:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:12 smithi170 ceph-mon[102832]: pgmap v16342: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 40 KiB/s wr, 24 op/s 2023-09-10T07:45:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:12 smithi170 ceph-mon[102832]: osdmap e1090: 12 total, 12 up, 12 in 2023-09-10T07:45:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:12 smithi082 ceph-mon[98694]: pgmap v16342: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 40 KiB/s wr, 24 op/s 2023-09-10T07:45:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:12 smithi082 ceph-mon[98694]: osdmap e1090: 12 total, 12 up, 12 in 2023-09-10T07:45:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:12 smithi098 ceph-mon[103124]: pgmap v16342: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 40 KiB/s wr, 24 op/s 2023-09-10T07:45:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:12 smithi098 ceph-mon[103124]: osdmap e1090: 12 total, 12 up, 12 in 2023-09-10T07:45:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30344.529947 secs 2023-09-10T07:45:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:14 smithi170 ceph-mon[102832]: pgmap v16344: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:45:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30345.695348 secs 2023-09-10T07:45:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30344.529947 secs 2023-09-10T07:45:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:14 smithi082 ceph-mon[98694]: pgmap v16344: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:45:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30345.695348 secs 2023-09-10T07:45:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30344.529947 secs 2023-09-10T07:45:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:14 smithi098 ceph-mon[103124]: pgmap v16344: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:45:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30345.695348 secs 2023-09-10T07:45:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30346.140782 secs 2023-09-10T07:45:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30346.140782 secs 2023-09-10T07:45:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30346.140782 secs 2023-09-10T07:45:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:16 smithi170 ceph-mon[102832]: pgmap v16345: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T07:45:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:16 smithi082 ceph-mon[98694]: pgmap v16345: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T07:45:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:16 smithi098 ceph-mon[103124]: pgmap v16345: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T07:45:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:18 smithi170 ceph-mon[102832]: pgmap v16346: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 11 op/s 2023-09-10T07:45:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:18 smithi082 ceph-mon[98694]: pgmap v16346: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 11 op/s 2023-09-10T07:45:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:18 smithi098 ceph-mon[103124]: pgmap v16346: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 11 op/s 2023-09-10T07:45:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30349.530007 secs 2023-09-10T07:45:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30350.695434 secs 2023-09-10T07:45:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30349.530007 secs 2023-09-10T07:45:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30350.695434 secs 2023-09-10T07:45:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30349.530007 secs 2023-09-10T07:45:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30350.695434 secs 2023-09-10T07:45:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30351.140843 secs 2023-09-10T07:45:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:20 smithi170 ceph-mon[102832]: pgmap v16347: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:45:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30351.140843 secs 2023-09-10T07:45:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:20 smithi082 ceph-mon[98694]: pgmap v16347: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:45:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30351.140843 secs 2023-09-10T07:45:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:20 smithi098 ceph-mon[103124]: pgmap v16347: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:45:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:22 smithi170 ceph-mon[102832]: pgmap v16348: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:45:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:22 smithi082 ceph-mon[98694]: pgmap v16348: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:45:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:22 smithi098 ceph-mon[103124]: pgmap v16348: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:45:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30354.530119 secs 2023-09-10T07:45:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:24 smithi170 ceph-mon[102832]: pgmap v16349: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:45:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30355.695517 secs 2023-09-10T07:45:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30354.530119 secs 2023-09-10T07:45:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:24 smithi082 ceph-mon[98694]: pgmap v16349: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:45:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30355.695517 secs 2023-09-10T07:45:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30354.530119 secs 2023-09-10T07:45:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:24 smithi098 ceph-mon[103124]: pgmap v16349: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:45:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30355.695517 secs 2023-09-10T07:45:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30356.140903 secs 2023-09-10T07:45:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30356.140903 secs 2023-09-10T07:45:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30356.140903 secs 2023-09-10T07:45:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:26 smithi170 ceph-mon[102832]: pgmap v16350: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 102 KiB/s wr, 13 op/s 2023-09-10T07:45:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:45:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:26 smithi082 ceph-mon[98694]: pgmap v16350: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 102 KiB/s wr, 13 op/s 2023-09-10T07:45:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:45:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:26 smithi098 ceph-mon[103124]: pgmap v16350: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 102 KiB/s wr, 13 op/s 2023-09-10T07:45:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:45:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:28 smithi170 ceph-mon[102832]: pgmap v16351: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 9 op/s 2023-09-10T07:45:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:28 smithi082 ceph-mon[98694]: pgmap v16351: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 9 op/s 2023-09-10T07:45:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:28 smithi098 ceph-mon[103124]: pgmap v16351: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 9 op/s 2023-09-10T07:45:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30359.530209 secs 2023-09-10T07:45:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30360.695580 secs 2023-09-10T07:45:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:45:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:45:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:45:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:45:29.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:45:29.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:45:29.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:45:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30359.530209 secs 2023-09-10T07:45:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30360.695580 secs 2023-09-10T07:45:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:45:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:45:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:45:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:45:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:45:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:45:29.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:45:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30359.530209 secs 2023-09-10T07:45:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30360.695580 secs 2023-09-10T07:45:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:45:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:45:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:45:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:45:29.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:45:29.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:45:29.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:45:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30361.140968 secs 2023-09-10T07:45:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:45:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:45:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:45:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:45:30.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:30 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T07:45:30.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:30 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T07:45:30.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:30 smithi170 ceph-mon[102832]: pgmap v16352: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 102 KiB/s wr, 9 op/s 2023-09-10T07:45:30.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:45:30.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:45:30.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:45:30.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:45:30.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:30 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T07:45:30.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:30 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T07:45:30.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:45:30.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:45:30.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:45:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30361.140968 secs 2023-09-10T07:45:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:45:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:45:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:45:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:45:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:30 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T07:45:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:30 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T07:45:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:30 smithi082 ceph-mon[98694]: pgmap v16352: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 102 KiB/s wr, 9 op/s 2023-09-10T07:45:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:45:30.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:45:30.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:45:30.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:45:30.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:30 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T07:45:30.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:30 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T07:45:30.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:45:30.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:45:30.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:45:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30361.140968 secs 2023-09-10T07:45:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:45:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:45:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:45:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:45:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:30 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T07:45:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:30 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T07:45:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:30 smithi098 ceph-mon[103124]: pgmap v16352: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 102 KiB/s wr, 9 op/s 2023-09-10T07:45:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:45:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:45:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:45:30.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:45:30.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:30 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T07:45:30.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:30 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T07:45:30.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:45:30.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:45:30.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:45:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:32 smithi170 ceph-mon[102832]: pgmap v16353: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 280 KiB/s wr, 13 op/s 2023-09-10T07:45:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:32 smithi082 ceph-mon[98694]: pgmap v16353: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 280 KiB/s wr, 13 op/s 2023-09-10T07:45:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:32 smithi098 ceph-mon[103124]: pgmap v16353: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 280 KiB/s wr, 13 op/s 2023-09-10T07:45:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:34 smithi170 ceph-mon[102832]: pgmap v16354: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 279 KiB/s wr, 9 op/s 2023-09-10T07:45:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30365.695638 secs 2023-09-10T07:45:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:34 smithi082 ceph-mon[98694]: pgmap v16354: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 279 KiB/s wr, 9 op/s 2023-09-10T07:45:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30365.695638 secs 2023-09-10T07:45:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:34 smithi098 ceph-mon[103124]: pgmap v16354: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 279 KiB/s wr, 9 op/s 2023-09-10T07:45:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30365.695638 secs 2023-09-10T07:45:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30366.141026 secs 2023-09-10T07:45:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:45:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30366.141026 secs 2023-09-10T07:45:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:45:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30366.141026 secs 2023-09-10T07:45:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:45:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:36 smithi170 ceph-mon[102832]: pgmap v16355: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 312 KiB/s wr, 14 op/s 2023-09-10T07:45:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:45:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:45:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:45:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:36 smithi082 ceph-mon[98694]: pgmap v16355: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 312 KiB/s wr, 14 op/s 2023-09-10T07:45:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:45:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:45:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:45:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:36 smithi098 ceph-mon[103124]: pgmap v16355: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 312 KiB/s wr, 14 op/s 2023-09-10T07:45:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:45:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:45:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:45:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:38 smithi170 ceph-mon[102832]: pgmap v16356: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 211 KiB/s wr, 10 op/s 2023-09-10T07:45:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:38 smithi082 ceph-mon[98694]: pgmap v16356: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 211 KiB/s wr, 10 op/s 2023-09-10T07:45:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:38 smithi098 ceph-mon[103124]: pgmap v16356: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 211 KiB/s wr, 10 op/s 2023-09-10T07:45:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30370.695698 secs 2023-09-10T07:45:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30370.695698 secs 2023-09-10T07:45:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30370.695698 secs 2023-09-10T07:45:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30371.141089 secs 2023-09-10T07:45:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:40 smithi170 ceph-mon[102832]: pgmap v16357: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 211 KiB/s wr, 10 op/s 2023-09-10T07:45:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30371.141089 secs 2023-09-10T07:45:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:40 smithi082 ceph-mon[98694]: pgmap v16357: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 211 KiB/s wr, 10 op/s 2023-09-10T07:45:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30371.141089 secs 2023-09-10T07:45:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:40 smithi098 ceph-mon[103124]: pgmap v16357: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 211 KiB/s wr, 10 op/s 2023-09-10T07:45:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:42 smithi170 ceph-mon[102832]: pgmap v16358: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 543 KiB/s wr, 15 op/s 2023-09-10T07:45:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:42 smithi082 ceph-mon[98694]: pgmap v16358: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 543 KiB/s wr, 15 op/s 2023-09-10T07:45:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:42 smithi098 ceph-mon[103124]: pgmap v16358: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 543 KiB/s wr, 15 op/s 2023-09-10T07:45:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:44 smithi170 ceph-mon[102832]: pgmap v16359: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 365 KiB/s wr, 11 op/s 2023-09-10T07:45:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30375.695758 secs 2023-09-10T07:45:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:44 smithi082 ceph-mon[98694]: pgmap v16359: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 365 KiB/s wr, 11 op/s 2023-09-10T07:45:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30375.695758 secs 2023-09-10T07:45:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:44 smithi098 ceph-mon[103124]: pgmap v16359: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 365 KiB/s wr, 11 op/s 2023-09-10T07:45:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30375.695758 secs 2023-09-10T07:45:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30376.141146 secs 2023-09-10T07:45:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30376.141146 secs 2023-09-10T07:45:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30376.141146 secs 2023-09-10T07:45:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:46 smithi170 ceph-mon[102832]: pgmap v16360: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 518 KiB/s wr, 18 op/s 2023-09-10T07:45:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:46 smithi082 ceph-mon[98694]: pgmap v16360: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 518 KiB/s wr, 18 op/s 2023-09-10T07:45:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:46 smithi098 ceph-mon[103124]: pgmap v16360: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 518 KiB/s wr, 18 op/s 2023-09-10T07:45:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:48 smithi170 ceph-mon[102832]: pgmap v16361: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 485 KiB/s wr, 13 op/s 2023-09-10T07:45:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:48 smithi082 ceph-mon[98694]: pgmap v16361: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 485 KiB/s wr, 13 op/s 2023-09-10T07:45:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:48 smithi098 ceph-mon[103124]: pgmap v16361: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 485 KiB/s wr, 13 op/s 2023-09-10T07:45:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30380.695820 secs 2023-09-10T07:45:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30380.695820 secs 2023-09-10T07:45:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30380.695820 secs 2023-09-10T07:45:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30381.141205 secs 2023-09-10T07:45:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:50 smithi170 ceph-mon[102832]: pgmap v16362: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 485 KiB/s wr, 12 op/s 2023-09-10T07:45:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30381.141205 secs 2023-09-10T07:45:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:50 smithi082 ceph-mon[98694]: pgmap v16362: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 485 KiB/s wr, 12 op/s 2023-09-10T07:45:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30381.141205 secs 2023-09-10T07:45:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:50 smithi098 ceph-mon[103124]: pgmap v16362: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 485 KiB/s wr, 12 op/s 2023-09-10T07:45:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:52 smithi170 ceph-mon[102832]: pgmap v16363: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 761 KiB/s wr, 20 op/s 2023-09-10T07:45:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:52 smithi082 ceph-mon[98694]: pgmap v16363: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 761 KiB/s wr, 20 op/s 2023-09-10T07:45:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:52 smithi098 ceph-mon[103124]: pgmap v16363: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 761 KiB/s wr, 20 op/s 2023-09-10T07:45:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:54 smithi170 ceph-mon[102832]: pgmap v16364: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 429 KiB/s wr, 14 op/s 2023-09-10T07:45:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30385.695877 secs 2023-09-10T07:45:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:54 smithi082 ceph-mon[98694]: pgmap v16364: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 429 KiB/s wr, 14 op/s 2023-09-10T07:45:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30385.695877 secs 2023-09-10T07:45:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:54 smithi098 ceph-mon[103124]: pgmap v16364: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 429 KiB/s wr, 14 op/s 2023-09-10T07:45:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30385.695877 secs 2023-09-10T07:45:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30386.141269 secs 2023-09-10T07:45:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30386.141269 secs 2023-09-10T07:45:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30386.141269 secs 2023-09-10T07:45:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:56 smithi170 ceph-mon[102832]: pgmap v16365: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 611 KiB/s wr, 20 op/s 2023-09-10T07:45:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:56 smithi082 ceph-mon[98694]: pgmap v16365: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 611 KiB/s wr, 20 op/s 2023-09-10T07:45:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:56 smithi098 ceph-mon[103124]: pgmap v16365: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 611 KiB/s wr, 20 op/s 2023-09-10T07:45:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:58 smithi170 ceph-mon[102832]: pgmap v16366: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 458 KiB/s wr, 13 op/s 2023-09-10T07:45:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:58 smithi082 ceph-mon[98694]: pgmap v16366: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 458 KiB/s wr, 13 op/s 2023-09-10T07:45:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:58 smithi098 ceph-mon[103124]: pgmap v16366: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 458 KiB/s wr, 13 op/s 2023-09-10T07:45:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30389.530683 secs 2023-09-10T07:45:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:45:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30390.695937 secs 2023-09-10T07:45:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30389.530683 secs 2023-09-10T07:45:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:45:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30390.695937 secs 2023-09-10T07:45:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30389.530683 secs 2023-09-10T07:45:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:45:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30390.695937 secs 2023-09-10T07:46:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30391.141328 secs 2023-09-10T07:46:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:46:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:46:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:46:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:00 smithi170 ceph-mon[102832]: pgmap v16367: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 458 KiB/s wr, 13 op/s 2023-09-10T07:46:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30391.141328 secs 2023-09-10T07:46:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:46:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:46:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:46:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:00 smithi082 ceph-mon[98694]: pgmap v16367: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 458 KiB/s wr, 13 op/s 2023-09-10T07:46:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30391.141328 secs 2023-09-10T07:46:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:46:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:46:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:46:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:00 smithi098 ceph-mon[103124]: pgmap v16367: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 458 KiB/s wr, 13 op/s 2023-09-10T07:46:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:02 smithi170 ceph-mon[102832]: pgmap v16368: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 706 KiB/s wr, 19 op/s 2023-09-10T07:46:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:02 smithi082 ceph-mon[98694]: pgmap v16368: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 706 KiB/s wr, 19 op/s 2023-09-10T07:46:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:02 smithi098 ceph-mon[103124]: pgmap v16368: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 706 KiB/s wr, 19 op/s 2023-09-10T07:46:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30394.530771 secs 2023-09-10T07:46:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:04 smithi170 ceph-mon[102832]: pgmap v16369: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 429 KiB/s wr, 12 op/s 2023-09-10T07:46:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30395.696021 secs 2023-09-10T07:46:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30394.530771 secs 2023-09-10T07:46:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:04 smithi082 ceph-mon[98694]: pgmap v16369: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 429 KiB/s wr, 12 op/s 2023-09-10T07:46:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30395.696021 secs 2023-09-10T07:46:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30394.530771 secs 2023-09-10T07:46:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:04 smithi098 ceph-mon[103124]: pgmap v16369: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 429 KiB/s wr, 12 op/s 2023-09-10T07:46:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30395.696021 secs 2023-09-10T07:46:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30396.141386 secs 2023-09-10T07:46:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30396.141386 secs 2023-09-10T07:46:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30396.141386 secs 2023-09-10T07:46:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:06 smithi170 ceph-mon[102832]: pgmap v16370: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 559 KiB/s wr, 31 op/s 2023-09-10T07:46:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:06 smithi082 ceph-mon[98694]: pgmap v16370: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 559 KiB/s wr, 31 op/s 2023-09-10T07:46:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:06 smithi098 ceph-mon[103124]: pgmap v16370: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 559 KiB/s wr, 31 op/s 2023-09-10T07:46:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:08 smithi170 ceph-mon[102832]: pgmap v16371: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 376 KiB/s wr, 25 op/s 2023-09-10T07:46:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:08 smithi082 ceph-mon[98694]: pgmap v16371: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 376 KiB/s wr, 25 op/s 2023-09-10T07:46:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:08 smithi098 ceph-mon[103124]: pgmap v16371: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 376 KiB/s wr, 25 op/s 2023-09-10T07:46:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30399.530833 secs 2023-09-10T07:46:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:09 smithi170 ceph-mon[102832]: osdmap e1091: 12 total, 12 up, 12 in 2023-09-10T07:46:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30400.696080 secs 2023-09-10T07:46:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30399.530833 secs 2023-09-10T07:46:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:09 smithi082 ceph-mon[98694]: osdmap e1091: 12 total, 12 up, 12 in 2023-09-10T07:46:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30400.696080 secs 2023-09-10T07:46:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30399.530833 secs 2023-09-10T07:46:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:09 smithi098 ceph-mon[103124]: osdmap e1091: 12 total, 12 up, 12 in 2023-09-10T07:46:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30400.696080 secs 2023-09-10T07:46:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30401.141448 secs 2023-09-10T07:46:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:10 smithi170 ceph-mon[102832]: pgmap v16373: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 452 KiB/s wr, 30 op/s 2023-09-10T07:46:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30401.141448 secs 2023-09-10T07:46:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:10 smithi082 ceph-mon[98694]: pgmap v16373: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 452 KiB/s wr, 30 op/s 2023-09-10T07:46:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30401.141448 secs 2023-09-10T07:46:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:10 smithi098 ceph-mon[103124]: pgmap v16373: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 452 KiB/s wr, 30 op/s 2023-09-10T07:46:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:12 smithi170 ceph-mon[102832]: pgmap v16374: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 155 KiB/s wr, 29 op/s 2023-09-10T07:46:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:12 smithi170 ceph-mon[102832]: osdmap e1092: 12 total, 12 up, 12 in 2023-09-10T07:46:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:12 smithi082 ceph-mon[98694]: pgmap v16374: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 155 KiB/s wr, 29 op/s 2023-09-10T07:46:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:12 smithi082 ceph-mon[98694]: osdmap e1092: 12 total, 12 up, 12 in 2023-09-10T07:46:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:12 smithi098 ceph-mon[103124]: pgmap v16374: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 155 KiB/s wr, 29 op/s 2023-09-10T07:46:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:12 smithi098 ceph-mon[103124]: osdmap e1092: 12 total, 12 up, 12 in 2023-09-10T07:46:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30404.530893 secs 2023-09-10T07:46:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:14 smithi170 ceph-mon[102832]: pgmap v16376: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:46:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30405.696166 secs 2023-09-10T07:46:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30404.530893 secs 2023-09-10T07:46:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:14 smithi082 ceph-mon[98694]: pgmap v16376: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:46:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30405.696166 secs 2023-09-10T07:46:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30404.530893 secs 2023-09-10T07:46:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:14 smithi098 ceph-mon[103124]: pgmap v16376: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:46:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30405.696166 secs 2023-09-10T07:46:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30406.141482 secs 2023-09-10T07:46:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30406.141482 secs 2023-09-10T07:46:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30406.141482 secs 2023-09-10T07:46:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:16 smithi170 ceph-mon[102832]: pgmap v16377: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 14 op/s 2023-09-10T07:46:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:16 smithi082 ceph-mon[98694]: pgmap v16377: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 14 op/s 2023-09-10T07:46:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:16 smithi098 ceph-mon[103124]: pgmap v16377: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 14 op/s 2023-09-10T07:46:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:18 smithi170 ceph-mon[102832]: pgmap v16378: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 334 B/s wr, 12 op/s 2023-09-10T07:46:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:18 smithi082 ceph-mon[98694]: pgmap v16378: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 334 B/s wr, 12 op/s 2023-09-10T07:46:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:18 smithi098 ceph-mon[103124]: pgmap v16378: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 334 B/s wr, 12 op/s 2023-09-10T07:46:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30409.530955 secs 2023-09-10T07:46:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30410.696225 secs 2023-09-10T07:46:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30409.530955 secs 2023-09-10T07:46:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30410.696225 secs 2023-09-10T07:46:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30409.530955 secs 2023-09-10T07:46:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30410.696225 secs 2023-09-10T07:46:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30411.141544 secs 2023-09-10T07:46:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:20 smithi170 ceph-mon[102832]: pgmap v16379: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T07:46:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30411.141544 secs 2023-09-10T07:46:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:20 smithi082 ceph-mon[98694]: pgmap v16379: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T07:46:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30411.141544 secs 2023-09-10T07:46:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:20 smithi098 ceph-mon[103124]: pgmap v16379: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T07:46:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:22 smithi170 ceph-mon[102832]: pgmap v16380: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:46:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:22 smithi082 ceph-mon[98694]: pgmap v16380: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:46:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:22 smithi098 ceph-mon[103124]: pgmap v16380: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:46:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30414.531064 secs 2023-09-10T07:46:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:24 smithi170 ceph-mon[102832]: pgmap v16381: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 182 B/s wr, 9 op/s 2023-09-10T07:46:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30415.696309 secs 2023-09-10T07:46:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30414.531064 secs 2023-09-10T07:46:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:24 smithi082 ceph-mon[98694]: pgmap v16381: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 182 B/s wr, 9 op/s 2023-09-10T07:46:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30415.696309 secs 2023-09-10T07:46:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30414.531064 secs 2023-09-10T07:46:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:24 smithi098 ceph-mon[103124]: pgmap v16381: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 182 B/s wr, 9 op/s 2023-09-10T07:46:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30415.696309 secs 2023-09-10T07:46:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30416.141602 secs 2023-09-10T07:46:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30416.141602 secs 2023-09-10T07:46:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30416.141602 secs 2023-09-10T07:46:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:26 smithi170 ceph-mon[102832]: pgmap v16382: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 597 B/s wr, 13 op/s 2023-09-10T07:46:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:26 smithi082 ceph-mon[98694]: pgmap v16382: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 597 B/s wr, 13 op/s 2023-09-10T07:46:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:26 smithi098 ceph-mon[103124]: pgmap v16382: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 597 B/s wr, 13 op/s 2023-09-10T07:46:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:28 smithi170 ceph-mon[102832]: pgmap v16383: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:46:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:28 smithi082 ceph-mon[98694]: pgmap v16383: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:46:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:28 smithi098 ceph-mon[103124]: pgmap v16383: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:46:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30419.531137 secs 2023-09-10T07:46:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30420.696365 secs 2023-09-10T07:46:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30419.531137 secs 2023-09-10T07:46:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30420.696365 secs 2023-09-10T07:46:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30419.531137 secs 2023-09-10T07:46:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30420.696365 secs 2023-09-10T07:46:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30421.141664 secs 2023-09-10T07:46:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:30 smithi170 ceph-mon[102832]: pgmap v16384: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:46:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:46:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30421.141664 secs 2023-09-10T07:46:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:30 smithi082 ceph-mon[98694]: pgmap v16384: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:46:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:46:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30421.141664 secs 2023-09-10T07:46:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:30 smithi098 ceph-mon[103124]: pgmap v16384: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:46:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:46:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:46:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:46:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:46:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:46:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:46:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:46:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:46:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:46:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:46:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:32 smithi082 ceph-mon[98694]: pgmap v16385: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 176 KiB/s wr, 13 op/s 2023-09-10T07:46:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:32 smithi098 ceph-mon[103124]: pgmap v16385: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 176 KiB/s wr, 13 op/s 2023-09-10T07:46:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:32 smithi170 ceph-mon[102832]: pgmap v16385: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 176 KiB/s wr, 13 op/s 2023-09-10T07:46:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:34 smithi170 ceph-mon[102832]: pgmap v16386: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 176 KiB/s wr, 9 op/s 2023-09-10T07:46:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30425.696422 secs 2023-09-10T07:46:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:34 smithi082 ceph-mon[98694]: pgmap v16386: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 176 KiB/s wr, 9 op/s 2023-09-10T07:46:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30425.696422 secs 2023-09-10T07:46:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:34 smithi098 ceph-mon[103124]: pgmap v16386: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 176 KiB/s wr, 9 op/s 2023-09-10T07:46:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30425.696422 secs 2023-09-10T07:46:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30426.141721 secs 2023-09-10T07:46:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:46:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30426.141721 secs 2023-09-10T07:46:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:46:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30426.141721 secs 2023-09-10T07:46:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:46:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:36 smithi170 ceph-mon[102832]: pgmap v16387: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 212 KiB/s wr, 15 op/s 2023-09-10T07:46:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:46:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:46:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:46:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:36 smithi082 ceph-mon[98694]: pgmap v16387: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 212 KiB/s wr, 15 op/s 2023-09-10T07:46:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:46:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:46:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:46:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:36 smithi098 ceph-mon[103124]: pgmap v16387: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 212 KiB/s wr, 15 op/s 2023-09-10T07:46:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:46:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:46:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:46:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:38 smithi170 ceph-mon[102832]: pgmap v16388: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 212 KiB/s wr, 10 op/s 2023-09-10T07:46:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:38 smithi082 ceph-mon[98694]: pgmap v16388: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 212 KiB/s wr, 10 op/s 2023-09-10T07:46:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:38 smithi098 ceph-mon[103124]: pgmap v16388: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 212 KiB/s wr, 10 op/s 2023-09-10T07:46:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30430.696480 secs 2023-09-10T07:46:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30430.696480 secs 2023-09-10T07:46:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30430.696480 secs 2023-09-10T07:46:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30431.141786 secs 2023-09-10T07:46:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:40 smithi082 ceph-mon[98694]: pgmap v16389: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 212 KiB/s wr, 10 op/s 2023-09-10T07:46:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30431.141786 secs 2023-09-10T07:46:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:40 smithi098 ceph-mon[103124]: pgmap v16389: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 212 KiB/s wr, 10 op/s 2023-09-10T07:46:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30431.141786 secs 2023-09-10T07:46:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:40 smithi170 ceph-mon[102832]: pgmap v16389: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 212 KiB/s wr, 10 op/s 2023-09-10T07:46:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:42 smithi170 ceph-mon[102832]: pgmap v16390: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 543 KiB/s wr, 16 op/s 2023-09-10T07:46:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:42 smithi082 ceph-mon[98694]: pgmap v16390: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 543 KiB/s wr, 16 op/s 2023-09-10T07:46:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:42 smithi098 ceph-mon[103124]: pgmap v16390: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 543 KiB/s wr, 16 op/s 2023-09-10T07:46:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:44 smithi170 ceph-mon[102832]: pgmap v16391: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 368 KiB/s wr, 11 op/s 2023-09-10T07:46:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30435.696495 secs 2023-09-10T07:46:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:44 smithi082 ceph-mon[98694]: pgmap v16391: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 368 KiB/s wr, 11 op/s 2023-09-10T07:46:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30435.696495 secs 2023-09-10T07:46:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:44 smithi098 ceph-mon[103124]: pgmap v16391: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 368 KiB/s wr, 11 op/s 2023-09-10T07:46:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30435.696495 secs 2023-09-10T07:46:45.419 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30436.141845 secs 2023-09-10T07:46:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30436.141845 secs 2023-09-10T07:46:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30436.141845 secs 2023-09-10T07:46:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:46 smithi170 ceph-mon[102832]: pgmap v16392: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 523 KiB/s wr, 18 op/s 2023-09-10T07:46:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:46 smithi082 ceph-mon[98694]: pgmap v16392: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 523 KiB/s wr, 18 op/s 2023-09-10T07:46:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:46 smithi098 ceph-mon[103124]: pgmap v16392: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 523 KiB/s wr, 18 op/s 2023-09-10T07:46:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:48 smithi170 ceph-mon[102832]: pgmap v16393: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 487 KiB/s wr, 13 op/s 2023-09-10T07:46:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:48 smithi082 ceph-mon[98694]: pgmap v16393: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 487 KiB/s wr, 13 op/s 2023-09-10T07:46:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:48 smithi098 ceph-mon[103124]: pgmap v16393: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 487 KiB/s wr, 13 op/s 2023-09-10T07:46:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:49 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30439.531498 secs 2023-09-10T07:46:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30440.696559 secs 2023-09-10T07:46:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:49 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30439.531498 secs 2023-09-10T07:46:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30440.696559 secs 2023-09-10T07:46:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:49 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30439.531498 secs 2023-09-10T07:46:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30440.696559 secs 2023-09-10T07:46:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30441.141906 secs 2023-09-10T07:46:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:50 smithi082 ceph-mon[98694]: pgmap v16394: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 487 KiB/s wr, 13 op/s 2023-09-10T07:46:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30441.141906 secs 2023-09-10T07:46:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:50 smithi098 ceph-mon[103124]: pgmap v16394: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 487 KiB/s wr, 13 op/s 2023-09-10T07:46:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30441.141906 secs 2023-09-10T07:46:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:50 smithi170 ceph-mon[102832]: pgmap v16394: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 487 KiB/s wr, 13 op/s 2023-09-10T07:46:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:46:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:46:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:46:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:46:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:46:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:46:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:46:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:46:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:46:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:52 smithi098 ceph-mon[103124]: pgmap v16395: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 764 KiB/s wr, 20 op/s 2023-09-10T07:46:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:52 smithi170 ceph-mon[102832]: pgmap v16395: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 764 KiB/s wr, 20 op/s 2023-09-10T07:46:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:52 smithi082 ceph-mon[98694]: pgmap v16395: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 764 KiB/s wr, 20 op/s 2023-09-10T07:46:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30444.531613 secs 2023-09-10T07:46:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:54 smithi170 ceph-mon[102832]: pgmap v16396: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 433 KiB/s wr, 14 op/s 2023-09-10T07:46:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30445.696625 secs 2023-09-10T07:46:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30444.531613 secs 2023-09-10T07:46:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:54 smithi082 ceph-mon[98694]: pgmap v16396: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 433 KiB/s wr, 14 op/s 2023-09-10T07:46:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30445.696625 secs 2023-09-10T07:46:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30444.531613 secs 2023-09-10T07:46:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:54 smithi098 ceph-mon[103124]: pgmap v16396: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 433 KiB/s wr, 14 op/s 2023-09-10T07:46:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30445.696625 secs 2023-09-10T07:46:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30446.141965 secs 2023-09-10T07:46:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30446.141965 secs 2023-09-10T07:46:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30446.141965 secs 2023-09-10T07:46:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:56 smithi170 ceph-mon[102832]: pgmap v16397: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 514 KiB/s wr, 20 op/s 2023-09-10T07:46:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:56 smithi082 ceph-mon[98694]: pgmap v16397: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 514 KiB/s wr, 20 op/s 2023-09-10T07:46:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:56 smithi098 ceph-mon[103124]: pgmap v16397: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 514 KiB/s wr, 20 op/s 2023-09-10T07:46:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:58 smithi170 ceph-mon[102832]: pgmap v16398: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 358 KiB/s wr, 13 op/s 2023-09-10T07:46:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:58 smithi082 ceph-mon[98694]: pgmap v16398: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 358 KiB/s wr, 13 op/s 2023-09-10T07:46:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:58 smithi098 ceph-mon[103124]: pgmap v16398: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 358 KiB/s wr, 13 op/s 2023-09-10T07:46:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30449.531675 secs 2023-09-10T07:46:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:46:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30450.696688 secs 2023-09-10T07:46:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30449.531675 secs 2023-09-10T07:46:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:46:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30450.696688 secs 2023-09-10T07:46:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30449.531675 secs 2023-09-10T07:46:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:46:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30450.696688 secs 2023-09-10T07:47:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30451.142027 secs 2023-09-10T07:47:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:00 smithi082 ceph-mon[98694]: pgmap v16399: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 358 KiB/s wr, 13 op/s 2023-09-10T07:47:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30451.142027 secs 2023-09-10T07:47:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:00 smithi098 ceph-mon[103124]: pgmap v16399: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 358 KiB/s wr, 13 op/s 2023-09-10T07:47:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30451.142027 secs 2023-09-10T07:47:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:00 smithi170 ceph-mon[102832]: pgmap v16399: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 358 KiB/s wr, 13 op/s 2023-09-10T07:47:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:02 smithi170 ceph-mon[102832]: pgmap v16400: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 606 KiB/s wr, 19 op/s 2023-09-10T07:47:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:02 smithi082 ceph-mon[98694]: pgmap v16400: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 606 KiB/s wr, 19 op/s 2023-09-10T07:47:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:02 smithi098 ceph-mon[103124]: pgmap v16400: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 606 KiB/s wr, 19 op/s 2023-09-10T07:47:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30454.531743 secs 2023-09-10T07:47:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:04 smithi170 ceph-mon[102832]: pgmap v16401: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 329 KiB/s wr, 12 op/s 2023-09-10T07:47:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30455.696750 secs 2023-09-10T07:47:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30454.531743 secs 2023-09-10T07:47:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:04 smithi082 ceph-mon[98694]: pgmap v16401: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 329 KiB/s wr, 12 op/s 2023-09-10T07:47:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30455.696750 secs 2023-09-10T07:47:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30454.531743 secs 2023-09-10T07:47:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:04 smithi098 ceph-mon[103124]: pgmap v16401: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 329 KiB/s wr, 12 op/s 2023-09-10T07:47:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30455.696750 secs 2023-09-10T07:47:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30456.142083 secs 2023-09-10T07:47:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30456.142083 secs 2023-09-10T07:47:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30456.142083 secs 2023-09-10T07:47:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:06 smithi170 ceph-mon[102832]: pgmap v16402: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 459 KiB/s wr, 31 op/s 2023-09-10T07:47:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:06 smithi082 ceph-mon[98694]: pgmap v16402: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 459 KiB/s wr, 31 op/s 2023-09-10T07:47:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:06 smithi098 ceph-mon[103124]: pgmap v16402: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 459 KiB/s wr, 31 op/s 2023-09-10T07:47:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:08 smithi082 ceph-mon[98694]: pgmap v16403: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 377 KiB/s wr, 25 op/s 2023-09-10T07:47:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:08 smithi098 ceph-mon[103124]: pgmap v16403: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 377 KiB/s wr, 25 op/s 2023-09-10T07:47:08.508 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:08 smithi170 ceph-mon[102832]: pgmap v16403: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 377 KiB/s wr, 25 op/s 2023-09-10T07:47:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30459.531776 secs 2023-09-10T07:47:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:09 smithi082 ceph-mon[98694]: osdmap e1093: 12 total, 12 up, 12 in 2023-09-10T07:47:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30460.696809 secs 2023-09-10T07:47:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30459.531776 secs 2023-09-10T07:47:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:09 smithi098 ceph-mon[103124]: osdmap e1093: 12 total, 12 up, 12 in 2023-09-10T07:47:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30460.696809 secs 2023-09-10T07:47:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30459.531776 secs 2023-09-10T07:47:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:09 smithi170 ceph-mon[102832]: osdmap e1093: 12 total, 12 up, 12 in 2023-09-10T07:47:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30460.696809 secs 2023-09-10T07:47:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30461.142146 secs 2023-09-10T07:47:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:10 smithi082 ceph-mon[98694]: pgmap v16405: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 453 KiB/s wr, 31 op/s 2023-09-10T07:47:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30461.142146 secs 2023-09-10T07:47:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:10 smithi098 ceph-mon[103124]: pgmap v16405: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 453 KiB/s wr, 31 op/s 2023-09-10T07:47:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30461.142146 secs 2023-09-10T07:47:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:10 smithi170 ceph-mon[102832]: pgmap v16405: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 453 KiB/s wr, 31 op/s 2023-09-10T07:47:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:12 smithi170 ceph-mon[102832]: pgmap v16406: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 155 KiB/s wr, 29 op/s 2023-09-10T07:47:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:12 smithi170 ceph-mon[102832]: osdmap e1094: 12 total, 12 up, 12 in 2023-09-10T07:47:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:12 smithi082 ceph-mon[98694]: pgmap v16406: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 155 KiB/s wr, 29 op/s 2023-09-10T07:47:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:12 smithi082 ceph-mon[98694]: osdmap e1094: 12 total, 12 up, 12 in 2023-09-10T07:47:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:12 smithi098 ceph-mon[103124]: pgmap v16406: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 155 KiB/s wr, 29 op/s 2023-09-10T07:47:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:12 smithi098 ceph-mon[103124]: osdmap e1094: 12 total, 12 up, 12 in 2023-09-10T07:47:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30464.531843 secs 2023-09-10T07:47:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:14 smithi170 ceph-mon[102832]: pgmap v16408: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:47:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30465.696870 secs 2023-09-10T07:47:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30464.531843 secs 2023-09-10T07:47:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:14 smithi082 ceph-mon[98694]: pgmap v16408: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:47:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30465.696870 secs 2023-09-10T07:47:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30464.531843 secs 2023-09-10T07:47:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:14 smithi098 ceph-mon[103124]: pgmap v16408: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:47:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30465.696870 secs 2023-09-10T07:47:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30466.142204 secs 2023-09-10T07:47:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30466.142204 secs 2023-09-10T07:47:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30466.142204 secs 2023-09-10T07:47:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:16 smithi170 ceph-mon[102832]: pgmap v16409: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 14 op/s 2023-09-10T07:47:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:16 smithi082 ceph-mon[98694]: pgmap v16409: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 14 op/s 2023-09-10T07:47:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:16 smithi098 ceph-mon[103124]: pgmap v16409: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 14 op/s 2023-09-10T07:47:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:18 smithi170 ceph-mon[102832]: pgmap v16410: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 12 op/s 2023-09-10T07:47:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:18 smithi082 ceph-mon[98694]: pgmap v16410: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 12 op/s 2023-09-10T07:47:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:18 smithi098 ceph-mon[103124]: pgmap v16410: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 12 op/s 2023-09-10T07:47:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30469.531948 secs 2023-09-10T07:47:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30470.696929 secs 2023-09-10T07:47:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30469.531948 secs 2023-09-10T07:47:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30470.696929 secs 2023-09-10T07:47:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30469.531948 secs 2023-09-10T07:47:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30470.696929 secs 2023-09-10T07:47:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30471.142265 secs 2023-09-10T07:47:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:20 smithi082 ceph-mon[98694]: pgmap v16411: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T07:47:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30471.142265 secs 2023-09-10T07:47:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:20 smithi098 ceph-mon[103124]: pgmap v16411: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T07:47:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30471.142265 secs 2023-09-10T07:47:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:20 smithi170 ceph-mon[102832]: pgmap v16411: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T07:47:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:22 smithi170 ceph-mon[102832]: pgmap v16412: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T07:47:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:22 smithi082 ceph-mon[98694]: pgmap v16412: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T07:47:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:22 smithi098 ceph-mon[103124]: pgmap v16412: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T07:47:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30474.532047 secs 2023-09-10T07:47:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:24 smithi170 ceph-mon[102832]: pgmap v16413: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T07:47:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30475.696988 secs 2023-09-10T07:47:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30474.532047 secs 2023-09-10T07:47:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:24 smithi082 ceph-mon[98694]: pgmap v16413: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T07:47:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30475.696988 secs 2023-09-10T07:47:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30474.532047 secs 2023-09-10T07:47:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:24 smithi098 ceph-mon[103124]: pgmap v16413: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T07:47:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30475.696988 secs 2023-09-10T07:47:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30476.142324 secs 2023-09-10T07:47:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30476.142324 secs 2023-09-10T07:47:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30476.142324 secs 2023-09-10T07:47:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:26 smithi082 ceph-mon[98694]: pgmap v16414: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 102 KiB/s wr, 14 op/s 2023-09-10T07:47:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:26 smithi098 ceph-mon[103124]: pgmap v16414: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 102 KiB/s wr, 14 op/s 2023-09-10T07:47:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:26 smithi170 ceph-mon[102832]: pgmap v16414: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 102 KiB/s wr, 14 op/s 2023-09-10T07:47:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:28 smithi082 ceph-mon[98694]: pgmap v16415: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 9 op/s 2023-09-10T07:47:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:28 smithi098 ceph-mon[103124]: pgmap v16415: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 9 op/s 2023-09-10T07:47:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:28 smithi170 ceph-mon[102832]: pgmap v16415: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 9 op/s 2023-09-10T07:47:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30479.532118 secs 2023-09-10T07:47:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30480.697105 secs 2023-09-10T07:47:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30479.532118 secs 2023-09-10T07:47:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30480.697105 secs 2023-09-10T07:47:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30479.532118 secs 2023-09-10T07:47:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30480.697105 secs 2023-09-10T07:47:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30481.142385 secs 2023-09-10T07:47:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:30 smithi082 ceph-mon[98694]: pgmap v16416: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 9 op/s 2023-09-10T07:47:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30481.142385 secs 2023-09-10T07:47:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:30 smithi098 ceph-mon[103124]: pgmap v16416: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 9 op/s 2023-09-10T07:47:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30481.142385 secs 2023-09-10T07:47:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:30 smithi170 ceph-mon[102832]: pgmap v16416: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 102 KiB/s wr, 9 op/s 2023-09-10T07:47:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:47:31.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:47:31.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:47:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:47:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:47:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:47:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:47:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:47:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:47:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:47:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:47:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:47:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:32 smithi082 ceph-mon[98694]: pgmap v16417: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 270 KiB/s wr, 15 op/s 2023-09-10T07:47:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:47:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:47:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:47:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:32 smithi098 ceph-mon[103124]: pgmap v16417: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 270 KiB/s wr, 15 op/s 2023-09-10T07:47:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:47:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:47:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:47:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:32 smithi170 ceph-mon[102832]: pgmap v16417: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 270 KiB/s wr, 15 op/s 2023-09-10T07:47:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:34 smithi170 ceph-mon[102832]: pgmap v16418: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 270 KiB/s wr, 10 op/s 2023-09-10T07:47:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30485.697194 secs 2023-09-10T07:47:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:34 smithi082 ceph-mon[98694]: pgmap v16418: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 270 KiB/s wr, 10 op/s 2023-09-10T07:47:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30485.697194 secs 2023-09-10T07:47:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:34 smithi098 ceph-mon[103124]: pgmap v16418: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 270 KiB/s wr, 10 op/s 2023-09-10T07:47:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30485.697194 secs 2023-09-10T07:47:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30486.142442 secs 2023-09-10T07:47:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:47:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30486.142442 secs 2023-09-10T07:47:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:47:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30486.142442 secs 2023-09-10T07:47:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:47:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:36 smithi082 ceph-mon[98694]: pgmap v16419: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 314 KiB/s wr, 15 op/s 2023-09-10T07:47:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:47:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:47:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:47:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:36 smithi098 ceph-mon[103124]: pgmap v16419: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 314 KiB/s wr, 15 op/s 2023-09-10T07:47:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:47:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:47:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:47:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:36 smithi170 ceph-mon[102832]: pgmap v16419: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 314 KiB/s wr, 15 op/s 2023-09-10T07:47:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:47:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:47:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:47:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:38 smithi082 ceph-mon[98694]: pgmap v16420: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 212 KiB/s wr, 10 op/s 2023-09-10T07:47:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:38 smithi098 ceph-mon[103124]: pgmap v16420: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 212 KiB/s wr, 10 op/s 2023-09-10T07:47:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:38 smithi170 ceph-mon[102832]: pgmap v16420: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 212 KiB/s wr, 10 op/s 2023-09-10T07:47:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30490.697277 secs 2023-09-10T07:47:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30490.697277 secs 2023-09-10T07:47:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30490.697277 secs 2023-09-10T07:47:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30491.142488 secs 2023-09-10T07:47:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:40 smithi082 ceph-mon[98694]: pgmap v16421: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 212 KiB/s wr, 10 op/s 2023-09-10T07:47:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30491.142488 secs 2023-09-10T07:47:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:40 smithi098 ceph-mon[103124]: pgmap v16421: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 212 KiB/s wr, 10 op/s 2023-09-10T07:47:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30491.142488 secs 2023-09-10T07:47:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:40 smithi170 ceph-mon[102832]: pgmap v16421: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 212 KiB/s wr, 10 op/s 2023-09-10T07:47:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:42 smithi170 ceph-mon[102832]: pgmap v16422: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 545 KiB/s wr, 16 op/s 2023-09-10T07:47:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:42 smithi082 ceph-mon[98694]: pgmap v16422: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 545 KiB/s wr, 16 op/s 2023-09-10T07:47:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:42 smithi098 ceph-mon[103124]: pgmap v16422: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 545 KiB/s wr, 16 op/s 2023-09-10T07:47:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:44 smithi082 ceph-mon[98694]: pgmap v16423: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 376 KiB/s wr, 11 op/s 2023-09-10T07:47:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30495.697336 secs 2023-09-10T07:47:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:44 smithi098 ceph-mon[103124]: pgmap v16423: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 376 KiB/s wr, 11 op/s 2023-09-10T07:47:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30495.697336 secs 2023-09-10T07:47:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:44 smithi170 ceph-mon[102832]: pgmap v16423: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 376 KiB/s wr, 11 op/s 2023-09-10T07:47:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30495.697336 secs 2023-09-10T07:47:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30496.142547 secs 2023-09-10T07:47:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30496.142547 secs 2023-09-10T07:47:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30496.142547 secs 2023-09-10T07:47:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:46 smithi082 ceph-mon[98694]: pgmap v16424: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 534 KiB/s wr, 18 op/s 2023-09-10T07:47:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:46 smithi098 ceph-mon[103124]: pgmap v16424: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 534 KiB/s wr, 18 op/s 2023-09-10T07:47:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:46 smithi170 ceph-mon[102832]: pgmap v16424: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 534 KiB/s wr, 18 op/s 2023-09-10T07:47:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:48 smithi082 ceph-mon[98694]: pgmap v16425: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 490 KiB/s wr, 12 op/s 2023-09-10T07:47:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:48 smithi098 ceph-mon[103124]: pgmap v16425: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 490 KiB/s wr, 12 op/s 2023-09-10T07:47:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:48 smithi170 ceph-mon[102832]: pgmap v16425: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 490 KiB/s wr, 12 op/s 2023-09-10T07:47:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30500.697394 secs 2023-09-10T07:47:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30500.697394 secs 2023-09-10T07:47:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30500.697394 secs 2023-09-10T07:47:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30501.142612 secs 2023-09-10T07:47:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:50 smithi082 ceph-mon[98694]: pgmap v16426: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 490 KiB/s wr, 12 op/s 2023-09-10T07:47:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30501.142612 secs 2023-09-10T07:47:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:50 smithi098 ceph-mon[103124]: pgmap v16426: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 490 KiB/s wr, 12 op/s 2023-09-10T07:47:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30501.142612 secs 2023-09-10T07:47:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:50 smithi170 ceph-mon[102832]: pgmap v16426: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 490 KiB/s wr, 12 op/s 2023-09-10T07:47:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:52 smithi082 ceph-mon[98694]: pgmap v16427: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 768 KiB/s wr, 20 op/s 2023-09-10T07:47:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:52 smithi098 ceph-mon[103124]: pgmap v16427: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 768 KiB/s wr, 20 op/s 2023-09-10T07:47:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:52 smithi170 ceph-mon[102832]: pgmap v16427: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 768 KiB/s wr, 20 op/s 2023-09-10T07:47:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30504.532421 secs 2023-09-10T07:47:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:54 smithi082 ceph-mon[98694]: pgmap v16428: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 436 KiB/s wr, 14 op/s 2023-09-10T07:47:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30505.697452 secs 2023-09-10T07:47:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30504.532421 secs 2023-09-10T07:47:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:54 smithi098 ceph-mon[103124]: pgmap v16428: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 436 KiB/s wr, 14 op/s 2023-09-10T07:47:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30505.697452 secs 2023-09-10T07:47:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30504.532421 secs 2023-09-10T07:47:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:54 smithi170 ceph-mon[102832]: pgmap v16428: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 436 KiB/s wr, 14 op/s 2023-09-10T07:47:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30505.697452 secs 2023-09-10T07:47:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30506.142672 secs 2023-09-10T07:47:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30506.142672 secs 2023-09-10T07:47:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30506.142672 secs 2023-09-10T07:47:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:56 smithi082 ceph-mon[98694]: pgmap v16429: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 619 KiB/s wr, 21 op/s 2023-09-10T07:47:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:47:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:47:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:47:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:56 smithi098 ceph-mon[103124]: pgmap v16429: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 619 KiB/s wr, 21 op/s 2023-09-10T07:47:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:47:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:47:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:47:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:56 smithi170 ceph-mon[102832]: pgmap v16429: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 619 KiB/s wr, 21 op/s 2023-09-10T07:47:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:47:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:47:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:47:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:58 smithi082 ceph-mon[98694]: pgmap v16430: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 460 KiB/s wr, 14 op/s 2023-09-10T07:47:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:58 smithi098 ceph-mon[103124]: pgmap v16430: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 460 KiB/s wr, 14 op/s 2023-09-10T07:47:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:58 smithi170 ceph-mon[102832]: pgmap v16430: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 460 KiB/s wr, 14 op/s 2023-09-10T07:47:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30509.532483 secs 2023-09-10T07:47:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:47:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30510.697512 secs 2023-09-10T07:47:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30509.532483 secs 2023-09-10T07:47:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:47:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30510.697512 secs 2023-09-10T07:47:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30509.532483 secs 2023-09-10T07:47:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:47:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30510.697512 secs 2023-09-10T07:48:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30511.142731 secs 2023-09-10T07:48:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:00 smithi082 ceph-mon[98694]: pgmap v16431: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 460 KiB/s wr, 14 op/s 2023-09-10T07:48:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30511.142731 secs 2023-09-10T07:48:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:00 smithi098 ceph-mon[103124]: pgmap v16431: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 460 KiB/s wr, 14 op/s 2023-09-10T07:48:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30511.142731 secs 2023-09-10T07:48:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:00 smithi170 ceph-mon[102832]: pgmap v16431: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 460 KiB/s wr, 14 op/s 2023-09-10T07:48:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:02 smithi082 ceph-mon[98694]: pgmap v16432: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 709 KiB/s wr, 20 op/s 2023-09-10T07:48:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:02 smithi098 ceph-mon[103124]: pgmap v16432: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 709 KiB/s wr, 20 op/s 2023-09-10T07:48:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:02 smithi170 ceph-mon[102832]: pgmap v16432: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 709 KiB/s wr, 20 op/s 2023-09-10T07:48:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30514.532540 secs 2023-09-10T07:48:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:04 smithi082 ceph-mon[98694]: pgmap v16433: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 432 KiB/s wr, 12 op/s 2023-09-10T07:48:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30515.697574 secs 2023-09-10T07:48:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30514.532540 secs 2023-09-10T07:48:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:04 smithi098 ceph-mon[103124]: pgmap v16433: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 432 KiB/s wr, 12 op/s 2023-09-10T07:48:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30515.697574 secs 2023-09-10T07:48:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30514.532540 secs 2023-09-10T07:48:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:04 smithi170 ceph-mon[102832]: pgmap v16433: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 432 KiB/s wr, 12 op/s 2023-09-10T07:48:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30515.697574 secs 2023-09-10T07:48:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30516.142790 secs 2023-09-10T07:48:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30516.142790 secs 2023-09-10T07:48:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30516.142790 secs 2023-09-10T07:48:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:06 smithi082 ceph-mon[98694]: pgmap v16434: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 561 KiB/s wr, 32 op/s 2023-09-10T07:48:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:06 smithi098 ceph-mon[103124]: pgmap v16434: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 561 KiB/s wr, 32 op/s 2023-09-10T07:48:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:06 smithi170 ceph-mon[102832]: pgmap v16434: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 561 KiB/s wr, 32 op/s 2023-09-10T07:48:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:08 smithi082 ceph-mon[98694]: pgmap v16435: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 378 KiB/s wr, 25 op/s 2023-09-10T07:48:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:08 smithi098 ceph-mon[103124]: pgmap v16435: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 378 KiB/s wr, 25 op/s 2023-09-10T07:48:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:08 smithi170 ceph-mon[102832]: pgmap v16435: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 378 KiB/s wr, 25 op/s 2023-09-10T07:48:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30519.532601 secs 2023-09-10T07:48:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:09 smithi082 ceph-mon[98694]: osdmap e1095: 12 total, 12 up, 12 in 2023-09-10T07:48:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30520.697660 secs 2023-09-10T07:48:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30519.532601 secs 2023-09-10T07:48:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:09 smithi098 ceph-mon[103124]: osdmap e1095: 12 total, 12 up, 12 in 2023-09-10T07:48:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30520.697660 secs 2023-09-10T07:48:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30519.532601 secs 2023-09-10T07:48:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:09 smithi170 ceph-mon[102832]: osdmap e1095: 12 total, 12 up, 12 in 2023-09-10T07:48:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30520.697660 secs 2023-09-10T07:48:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30521.142853 secs 2023-09-10T07:48:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:10 smithi082 ceph-mon[98694]: pgmap v16437: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 454 KiB/s wr, 31 op/s 2023-09-10T07:48:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30521.142853 secs 2023-09-10T07:48:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:10 smithi098 ceph-mon[103124]: pgmap v16437: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 454 KiB/s wr, 31 op/s 2023-09-10T07:48:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30521.142853 secs 2023-09-10T07:48:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:10 smithi170 ceph-mon[102832]: pgmap v16437: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 454 KiB/s wr, 31 op/s 2023-09-10T07:48:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:12 smithi082 ceph-mon[98694]: pgmap v16438: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 155 KiB/s wr, 29 op/s 2023-09-10T07:48:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:12 smithi082 ceph-mon[98694]: osdmap e1096: 12 total, 12 up, 12 in 2023-09-10T07:48:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:12 smithi098 ceph-mon[103124]: pgmap v16438: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 155 KiB/s wr, 29 op/s 2023-09-10T07:48:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:12 smithi098 ceph-mon[103124]: osdmap e1096: 12 total, 12 up, 12 in 2023-09-10T07:48:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:12 smithi170 ceph-mon[102832]: pgmap v16438: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 155 KiB/s wr, 29 op/s 2023-09-10T07:48:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:12 smithi170 ceph-mon[102832]: osdmap e1096: 12 total, 12 up, 12 in 2023-09-10T07:48:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30524.532663 secs 2023-09-10T07:48:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:14 smithi082 ceph-mon[98694]: pgmap v16440: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:48:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30525.697719 secs 2023-09-10T07:48:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30524.532663 secs 2023-09-10T07:48:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:14 smithi098 ceph-mon[103124]: pgmap v16440: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:48:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30525.697719 secs 2023-09-10T07:48:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30524.532663 secs 2023-09-10T07:48:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:14 smithi170 ceph-mon[102832]: pgmap v16440: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:48:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30525.697719 secs 2023-09-10T07:48:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30526.142910 secs 2023-09-10T07:48:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30526.142910 secs 2023-09-10T07:48:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30526.142910 secs 2023-09-10T07:48:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:16 smithi082 ceph-mon[98694]: pgmap v16441: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 14 op/s 2023-09-10T07:48:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:16 smithi098 ceph-mon[103124]: pgmap v16441: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 14 op/s 2023-09-10T07:48:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:16 smithi170 ceph-mon[102832]: pgmap v16441: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 14 op/s 2023-09-10T07:48:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:18 smithi082 ceph-mon[98694]: pgmap v16442: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 334 B/s wr, 12 op/s 2023-09-10T07:48:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:18 smithi098 ceph-mon[103124]: pgmap v16442: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 334 B/s wr, 12 op/s 2023-09-10T07:48:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:18 smithi170 ceph-mon[102832]: pgmap v16442: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 334 B/s wr, 12 op/s 2023-09-10T07:48:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30529.532722 secs 2023-09-10T07:48:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30530.697778 secs 2023-09-10T07:48:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30529.532722 secs 2023-09-10T07:48:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30530.697778 secs 2023-09-10T07:48:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30529.532722 secs 2023-09-10T07:48:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30530.697778 secs 2023-09-10T07:48:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30531.142970 secs 2023-09-10T07:48:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:20 smithi082 ceph-mon[98694]: pgmap v16443: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T07:48:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30531.142970 secs 2023-09-10T07:48:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:20 smithi098 ceph-mon[103124]: pgmap v16443: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T07:48:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30531.142970 secs 2023-09-10T07:48:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:20 smithi170 ceph-mon[102832]: pgmap v16443: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T07:48:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:22 smithi082 ceph-mon[98694]: pgmap v16444: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:48:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:22 smithi098 ceph-mon[103124]: pgmap v16444: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:48:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:22 smithi170 ceph-mon[102832]: pgmap v16444: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:48:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30534.532780 secs 2023-09-10T07:48:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:24 smithi082 ceph-mon[98694]: pgmap v16445: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 182 B/s wr, 9 op/s 2023-09-10T07:48:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30535.697836 secs 2023-09-10T07:48:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30534.532780 secs 2023-09-10T07:48:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:24 smithi098 ceph-mon[103124]: pgmap v16445: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 182 B/s wr, 9 op/s 2023-09-10T07:48:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30535.697836 secs 2023-09-10T07:48:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30534.532780 secs 2023-09-10T07:48:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:24 smithi170 ceph-mon[102832]: pgmap v16445: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 182 B/s wr, 9 op/s 2023-09-10T07:48:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30535.697836 secs 2023-09-10T07:48:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30536.143029 secs 2023-09-10T07:48:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30536.143029 secs 2023-09-10T07:48:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30536.143029 secs 2023-09-10T07:48:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:26 smithi082 ceph-mon[98694]: pgmap v16446: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 597 B/s wr, 13 op/s 2023-09-10T07:48:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:26 smithi098 ceph-mon[103124]: pgmap v16446: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 597 B/s wr, 13 op/s 2023-09-10T07:48:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:26 smithi170 ceph-mon[102832]: pgmap v16446: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 597 B/s wr, 13 op/s 2023-09-10T07:48:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:28 smithi082 ceph-mon[98694]: pgmap v16447: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:48:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:28 smithi098 ceph-mon[103124]: pgmap v16447: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:48:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:28 smithi170 ceph-mon[102832]: pgmap v16447: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:48:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30539.532841 secs 2023-09-10T07:48:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30540.697895 secs 2023-09-10T07:48:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30539.532841 secs 2023-09-10T07:48:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30540.697895 secs 2023-09-10T07:48:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30539.532841 secs 2023-09-10T07:48:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30540.697895 secs 2023-09-10T07:48:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30541.143090 secs 2023-09-10T07:48:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:30 smithi098 ceph-mon[103124]: pgmap v16448: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:48:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30541.143090 secs 2023-09-10T07:48:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:30 smithi170 ceph-mon[102832]: pgmap v16448: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:48:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30541.143090 secs 2023-09-10T07:48:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:30 smithi082 ceph-mon[98694]: pgmap v16448: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:48:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:48:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:48:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:48:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:32 smithi098 ceph-mon[103124]: pgmap v16449: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 158 KiB/s wr, 13 op/s 2023-09-10T07:48:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:48:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:48:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:48:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:48:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:32 smithi170 ceph-mon[102832]: pgmap v16449: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 158 KiB/s wr, 13 op/s 2023-09-10T07:48:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:48:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:48:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:48:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:48:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:32 smithi082 ceph-mon[98694]: pgmap v16449: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 158 KiB/s wr, 13 op/s 2023-09-10T07:48:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:48:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:48:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:48:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:48:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:48:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:48:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:48:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30545.697973 secs 2023-09-10T07:48:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:48:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:48:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:48:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30545.697973 secs 2023-09-10T07:48:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:48:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:48:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:48:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30545.697973 secs 2023-09-10T07:48:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:34 smithi098 ceph-mon[103124]: pgmap v16450: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 9 op/s 2023-09-10T07:48:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:34 smithi170 ceph-mon[102832]: pgmap v16450: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 9 op/s 2023-09-10T07:48:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:34 smithi082 ceph-mon[98694]: pgmap v16450: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 9 op/s 2023-09-10T07:48:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30546.143150 secs 2023-09-10T07:48:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:48:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30546.143150 secs 2023-09-10T07:48:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:48:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30546.143150 secs 2023-09-10T07:48:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:48:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:36 smithi098 ceph-mon[103124]: pgmap v16451: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 213 KiB/s wr, 14 op/s 2023-09-10T07:48:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:48:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:48:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:48:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:36 smithi170 ceph-mon[102832]: pgmap v16451: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 213 KiB/s wr, 14 op/s 2023-09-10T07:48:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:48:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:48:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:48:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:36 smithi082 ceph-mon[98694]: pgmap v16451: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 213 KiB/s wr, 14 op/s 2023-09-10T07:48:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:48:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:48:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:48:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:38 smithi082 ceph-mon[98694]: pgmap v16452: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 213 KiB/s wr, 10 op/s 2023-09-10T07:48:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:38 smithi098 ceph-mon[103124]: pgmap v16452: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 213 KiB/s wr, 10 op/s 2023-09-10T07:48:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:38 smithi170 ceph-mon[102832]: pgmap v16452: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 213 KiB/s wr, 10 op/s 2023-09-10T07:48:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30550.698030 secs 2023-09-10T07:48:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30550.698030 secs 2023-09-10T07:48:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30550.698030 secs 2023-09-10T07:48:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30551.143212 secs 2023-09-10T07:48:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:40 smithi098 ceph-mon[103124]: pgmap v16453: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 213 KiB/s wr, 10 op/s 2023-09-10T07:48:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30551.143212 secs 2023-09-10T07:48:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:40 smithi170 ceph-mon[102832]: pgmap v16453: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 213 KiB/s wr, 10 op/s 2023-09-10T07:48:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30551.143212 secs 2023-09-10T07:48:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:40 smithi082 ceph-mon[98694]: pgmap v16453: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 213 KiB/s wr, 10 op/s 2023-09-10T07:48:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:42 smithi082 ceph-mon[98694]: pgmap v16454: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 781 KiB/s wr, 17 op/s 2023-09-10T07:48:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:42 smithi098 ceph-mon[103124]: pgmap v16454: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 781 KiB/s wr, 17 op/s 2023-09-10T07:48:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:42 smithi170 ceph-mon[102832]: pgmap v16454: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 781 KiB/s wr, 17 op/s 2023-09-10T07:48:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:44 smithi082 ceph-mon[98694]: pgmap v16455: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 624 KiB/s wr, 12 op/s 2023-09-10T07:48:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30555.698103 secs 2023-09-10T07:48:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:44 smithi098 ceph-mon[103124]: pgmap v16455: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 624 KiB/s wr, 12 op/s 2023-09-10T07:48:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30555.698103 secs 2023-09-10T07:48:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:44 smithi170 ceph-mon[102832]: pgmap v16455: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 624 KiB/s wr, 12 op/s 2023-09-10T07:48:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30555.698103 secs 2023-09-10T07:48:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30556.143270 secs 2023-09-10T07:48:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30556.143270 secs 2023-09-10T07:48:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30556.143270 secs 2023-09-10T07:48:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:46 smithi082 ceph-mon[98694]: pgmap v16456: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 776 KiB/s wr, 19 op/s 2023-09-10T07:48:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:46 smithi098 ceph-mon[103124]: pgmap v16456: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 776 KiB/s wr, 19 op/s 2023-09-10T07:48:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:46 smithi170 ceph-mon[102832]: pgmap v16456: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 776 KiB/s wr, 19 op/s 2023-09-10T07:48:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:48 smithi082 ceph-mon[98694]: pgmap v16457: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 720 KiB/s wr, 14 op/s 2023-09-10T07:48:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:48 smithi098 ceph-mon[103124]: pgmap v16457: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 720 KiB/s wr, 14 op/s 2023-09-10T07:48:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:48 smithi170 ceph-mon[102832]: pgmap v16457: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 720 KiB/s wr, 14 op/s 2023-09-10T07:48:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30560.698162 secs 2023-09-10T07:48:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30560.698162 secs 2023-09-10T07:48:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30560.698162 secs 2023-09-10T07:48:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30561.143330 secs 2023-09-10T07:48:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:50 smithi098 ceph-mon[103124]: pgmap v16458: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 720 KiB/s wr, 14 op/s 2023-09-10T07:48:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30561.143330 secs 2023-09-10T07:48:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:50 smithi170 ceph-mon[102832]: pgmap v16458: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 720 KiB/s wr, 14 op/s 2023-09-10T07:48:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30561.143330 secs 2023-09-10T07:48:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:50 smithi082 ceph-mon[98694]: pgmap v16458: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 720 KiB/s wr, 14 op/s 2023-09-10T07:48:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:52 smithi082 ceph-mon[98694]: pgmap v16459: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T07:48:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:52 smithi098 ceph-mon[103124]: pgmap v16459: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T07:48:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:52 smithi170 ceph-mon[102832]: pgmap v16459: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T07:48:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30564.533212 secs 2023-09-10T07:48:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:54 smithi082 ceph-mon[98694]: pgmap v16460: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 557 KiB/s wr, 15 op/s 2023-09-10T07:48:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30565.698254 secs 2023-09-10T07:48:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30564.533212 secs 2023-09-10T07:48:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:54 smithi098 ceph-mon[103124]: pgmap v16460: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 557 KiB/s wr, 15 op/s 2023-09-10T07:48:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30565.698254 secs 2023-09-10T07:48:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30564.533212 secs 2023-09-10T07:48:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:54 smithi170 ceph-mon[102832]: pgmap v16460: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 557 KiB/s wr, 15 op/s 2023-09-10T07:48:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30565.698254 secs 2023-09-10T07:48:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30566.143389 secs 2023-09-10T07:48:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30566.143389 secs 2023-09-10T07:48:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30566.143389 secs 2023-09-10T07:48:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:56 smithi082 ceph-mon[98694]: pgmap v16461: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 557 KiB/s wr, 20 op/s 2023-09-10T07:48:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:48:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:48:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:48:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:56 smithi098 ceph-mon[103124]: pgmap v16461: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 557 KiB/s wr, 20 op/s 2023-09-10T07:48:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:48:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:48:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:48:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:56 smithi170 ceph-mon[102832]: pgmap v16461: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 557 KiB/s wr, 20 op/s 2023-09-10T07:48:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:48:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:48:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:48:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:58 smithi098 ceph-mon[103124]: pgmap v16462: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 405 KiB/s wr, 13 op/s 2023-09-10T07:48:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:58 smithi170 ceph-mon[102832]: pgmap v16462: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 405 KiB/s wr, 13 op/s 2023-09-10T07:48:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:58 smithi082 ceph-mon[98694]: pgmap v16462: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 405 KiB/s wr, 13 op/s 2023-09-10T07:48:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30569.533272 secs 2023-09-10T07:48:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:48:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30570.698315 secs 2023-09-10T07:48:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30569.533272 secs 2023-09-10T07:48:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:48:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30570.698315 secs 2023-09-10T07:48:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30569.533272 secs 2023-09-10T07:48:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:48:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30570.698315 secs 2023-09-10T07:49:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30571.143452 secs 2023-09-10T07:49:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:00 smithi098 ceph-mon[103124]: pgmap v16463: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 405 KiB/s wr, 13 op/s 2023-09-10T07:49:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30571.143452 secs 2023-09-10T07:49:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:00 smithi170 ceph-mon[102832]: pgmap v16463: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 405 KiB/s wr, 13 op/s 2023-09-10T07:49:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30571.143452 secs 2023-09-10T07:49:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:00 smithi082 ceph-mon[98694]: pgmap v16463: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 405 KiB/s wr, 13 op/s 2023-09-10T07:49:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:02 smithi082 ceph-mon[98694]: pgmap v16464: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 637 KiB/s wr, 18 op/s 2023-09-10T07:49:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:02 smithi098 ceph-mon[103124]: pgmap v16464: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 637 KiB/s wr, 18 op/s 2023-09-10T07:49:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:02 smithi170 ceph-mon[102832]: pgmap v16464: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 637 KiB/s wr, 18 op/s 2023-09-10T07:49:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30574.533365 secs 2023-09-10T07:49:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:04 smithi082 ceph-mon[98694]: pgmap v16465: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 232 KiB/s wr, 10 op/s 2023-09-10T07:49:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30575.698399 secs 2023-09-10T07:49:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30574.533365 secs 2023-09-10T07:49:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:04 smithi098 ceph-mon[103124]: pgmap v16465: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 232 KiB/s wr, 10 op/s 2023-09-10T07:49:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30575.698399 secs 2023-09-10T07:49:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30574.533365 secs 2023-09-10T07:49:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:04 smithi170 ceph-mon[102832]: pgmap v16465: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 232 KiB/s wr, 10 op/s 2023-09-10T07:49:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30575.698399 secs 2023-09-10T07:49:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30576.143485 secs 2023-09-10T07:49:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30576.143485 secs 2023-09-10T07:49:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30576.143485 secs 2023-09-10T07:49:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:06 smithi098 ceph-mon[103124]: pgmap v16466: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 362 KiB/s wr, 30 op/s 2023-09-10T07:49:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:06 smithi170 ceph-mon[102832]: pgmap v16466: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 362 KiB/s wr, 30 op/s 2023-09-10T07:49:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:06 smithi082 ceph-mon[98694]: pgmap v16466: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 362 KiB/s wr, 30 op/s 2023-09-10T07:49:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:08 smithi082 ceph-mon[98694]: pgmap v16467: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 361 KiB/s wr, 24 op/s 2023-09-10T07:49:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:08 smithi098 ceph-mon[103124]: pgmap v16467: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 361 KiB/s wr, 24 op/s 2023-09-10T07:49:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:08 smithi170 ceph-mon[102832]: pgmap v16467: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 361 KiB/s wr, 24 op/s 2023-09-10T07:49:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30579.533428 secs 2023-09-10T07:49:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:09 smithi098 ceph-mon[103124]: osdmap e1097: 12 total, 12 up, 12 in 2023-09-10T07:49:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30580.698459 secs 2023-09-10T07:49:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30579.533428 secs 2023-09-10T07:49:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:09 smithi170 ceph-mon[102832]: osdmap e1097: 12 total, 12 up, 12 in 2023-09-10T07:49:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30580.698459 secs 2023-09-10T07:49:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30579.533428 secs 2023-09-10T07:49:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:09 smithi082 ceph-mon[98694]: osdmap e1097: 12 total, 12 up, 12 in 2023-09-10T07:49:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30580.698459 secs 2023-09-10T07:49:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30581.143545 secs 2023-09-10T07:49:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:10 smithi098 ceph-mon[103124]: pgmap v16469: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 433 KiB/s wr, 29 op/s 2023-09-10T07:49:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30581.143545 secs 2023-09-10T07:49:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:10 smithi170 ceph-mon[102832]: pgmap v16469: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 433 KiB/s wr, 29 op/s 2023-09-10T07:49:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30581.143545 secs 2023-09-10T07:49:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:10 smithi082 ceph-mon[98694]: pgmap v16469: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 433 KiB/s wr, 29 op/s 2023-09-10T07:49:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:12 smithi082 ceph-mon[98694]: pgmap v16470: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 156 KiB/s wr, 29 op/s 2023-09-10T07:49:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:12 smithi082 ceph-mon[98694]: osdmap e1098: 12 total, 12 up, 12 in 2023-09-10T07:49:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:12 smithi098 ceph-mon[103124]: pgmap v16470: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 156 KiB/s wr, 29 op/s 2023-09-10T07:49:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:12 smithi098 ceph-mon[103124]: osdmap e1098: 12 total, 12 up, 12 in 2023-09-10T07:49:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:12 smithi170 ceph-mon[102832]: pgmap v16470: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 156 KiB/s wr, 29 op/s 2023-09-10T07:49:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:12 smithi170 ceph-mon[102832]: osdmap e1098: 12 total, 12 up, 12 in 2023-09-10T07:49:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30584.533488 secs 2023-09-10T07:49:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:14 smithi082 ceph-mon[98694]: pgmap v16472: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:49:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30585.698509 secs 2023-09-10T07:49:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30584.533488 secs 2023-09-10T07:49:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:14 smithi098 ceph-mon[103124]: pgmap v16472: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:49:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30585.698509 secs 2023-09-10T07:49:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30584.533488 secs 2023-09-10T07:49:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:14 smithi170 ceph-mon[102832]: pgmap v16472: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:49:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30585.698509 secs 2023-09-10T07:49:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30586.143602 secs 2023-09-10T07:49:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30586.143602 secs 2023-09-10T07:49:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30586.143602 secs 2023-09-10T07:49:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:16 smithi098 ceph-mon[103124]: pgmap v16473: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 14 op/s 2023-09-10T07:49:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:16 smithi170 ceph-mon[102832]: pgmap v16473: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 14 op/s 2023-09-10T07:49:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:16 smithi082 ceph-mon[98694]: pgmap v16473: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 14 op/s 2023-09-10T07:49:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:18 smithi098 ceph-mon[103124]: pgmap v16474: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 334 B/s wr, 12 op/s 2023-09-10T07:49:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:18 smithi170 ceph-mon[102832]: pgmap v16474: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 334 B/s wr, 12 op/s 2023-09-10T07:49:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:18 smithi082 ceph-mon[98694]: pgmap v16474: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 334 B/s wr, 12 op/s 2023-09-10T07:49:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30589.533588 secs 2023-09-10T07:49:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30590.698568 secs 2023-09-10T07:49:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30589.533588 secs 2023-09-10T07:49:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30590.698568 secs 2023-09-10T07:49:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30589.533588 secs 2023-09-10T07:49:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30590.698568 secs 2023-09-10T07:49:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30591.143665 secs 2023-09-10T07:49:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:20 smithi098 ceph-mon[103124]: pgmap v16475: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T07:49:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30591.143665 secs 2023-09-10T07:49:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:20 smithi170 ceph-mon[102832]: pgmap v16475: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T07:49:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30591.143665 secs 2023-09-10T07:49:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:20 smithi082 ceph-mon[98694]: pgmap v16475: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T07:49:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:22 smithi082 ceph-mon[98694]: pgmap v16476: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T07:49:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:22 smithi098 ceph-mon[103124]: pgmap v16476: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T07:49:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:22 smithi170 ceph-mon[102832]: pgmap v16476: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 12 op/s 2023-09-10T07:49:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30594.533649 secs 2023-09-10T07:49:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:24 smithi098 ceph-mon[103124]: pgmap v16477: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T07:49:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30595.698626 secs 2023-09-10T07:49:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30594.533649 secs 2023-09-10T07:49:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:24 smithi170 ceph-mon[102832]: pgmap v16477: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T07:49:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30595.698626 secs 2023-09-10T07:49:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30594.533649 secs 2023-09-10T07:49:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:24 smithi082 ceph-mon[98694]: pgmap v16477: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-09-10T07:49:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30595.698626 secs 2023-09-10T07:49:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30596.143724 secs 2023-09-10T07:49:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30596.143724 secs 2023-09-10T07:49:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30596.143724 secs 2023-09-10T07:49:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:26 smithi098 ceph-mon[103124]: pgmap v16478: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 210 KiB/s wr, 14 op/s 2023-09-10T07:49:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:26 smithi170 ceph-mon[102832]: pgmap v16478: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 210 KiB/s wr, 14 op/s 2023-09-10T07:49:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:26 smithi082 ceph-mon[98694]: pgmap v16478: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 210 KiB/s wr, 14 op/s 2023-09-10T07:49:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:28 smithi098 ceph-mon[103124]: pgmap v16479: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 210 KiB/s wr, 10 op/s 2023-09-10T07:49:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:28 smithi170 ceph-mon[102832]: pgmap v16479: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 210 KiB/s wr, 10 op/s 2023-09-10T07:49:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:28 smithi082 ceph-mon[98694]: pgmap v16479: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 210 KiB/s wr, 10 op/s 2023-09-10T07:49:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30599.533708 secs 2023-09-10T07:49:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30600.698685 secs 2023-09-10T07:49:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30599.533708 secs 2023-09-10T07:49:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30600.698685 secs 2023-09-10T07:49:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30599.533708 secs 2023-09-10T07:49:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30600.698685 secs 2023-09-10T07:49:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30601.143784 secs 2023-09-10T07:49:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:30 smithi098 ceph-mon[103124]: pgmap v16480: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 210 KiB/s wr, 10 op/s 2023-09-10T07:49:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30601.143784 secs 2023-09-10T07:49:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:30 smithi170 ceph-mon[102832]: pgmap v16480: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 210 KiB/s wr, 10 op/s 2023-09-10T07:49:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30601.143784 secs 2023-09-10T07:49:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:30 smithi082 ceph-mon[98694]: pgmap v16480: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 210 KiB/s wr, 10 op/s 2023-09-10T07:49:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:32 smithi098 ceph-mon[103124]: pgmap v16481: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 210 KiB/s wr, 14 op/s 2023-09-10T07:49:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:32 smithi170 ceph-mon[102832]: pgmap v16481: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 210 KiB/s wr, 14 op/s 2023-09-10T07:49:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:32 smithi082 ceph-mon[98694]: pgmap v16481: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 210 KiB/s wr, 14 op/s 2023-09-10T07:49:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:49:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:49:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:49:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:49:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:49:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:49:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:49:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:49:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:49:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:49:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:49:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:49:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:34 smithi098 ceph-mon[103124]: pgmap v16482: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 9 op/s 2023-09-10T07:49:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30605.698749 secs 2023-09-10T07:49:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:34 smithi170 ceph-mon[102832]: pgmap v16482: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 9 op/s 2023-09-10T07:49:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30605.698749 secs 2023-09-10T07:49:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:34 smithi082 ceph-mon[98694]: pgmap v16482: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 9 op/s 2023-09-10T07:49:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30605.698749 secs 2023-09-10T07:49:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30606.143841 secs 2023-09-10T07:49:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:49:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30606.143841 secs 2023-09-10T07:49:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:49:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30606.143841 secs 2023-09-10T07:49:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:49:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:36 smithi098 ceph-mon[103124]: pgmap v16483: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 226 KiB/s wr, 14 op/s 2023-09-10T07:49:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:49:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:49:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:49:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:36 smithi170 ceph-mon[102832]: pgmap v16483: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 226 KiB/s wr, 14 op/s 2023-09-10T07:49:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:49:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:49:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:49:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:36 smithi082 ceph-mon[98694]: pgmap v16483: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 226 KiB/s wr, 14 op/s 2023-09-10T07:49:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:49:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:49:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:49:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:38 smithi098 ceph-mon[103124]: pgmap v16484: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 119 KiB/s wr, 9 op/s 2023-09-10T07:49:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:38 smithi170 ceph-mon[102832]: pgmap v16484: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 119 KiB/s wr, 9 op/s 2023-09-10T07:49:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:38 smithi082 ceph-mon[98694]: pgmap v16484: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 119 KiB/s wr, 9 op/s 2023-09-10T07:49:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30610.698807 secs 2023-09-10T07:49:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30610.698807 secs 2023-09-10T07:49:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30610.698807 secs 2023-09-10T07:49:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30611.143991 secs 2023-09-10T07:49:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:40 smithi098 ceph-mon[103124]: pgmap v16485: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 119 KiB/s wr, 9 op/s 2023-09-10T07:49:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30611.143991 secs 2023-09-10T07:49:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:40 smithi170 ceph-mon[102832]: pgmap v16485: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 119 KiB/s wr, 9 op/s 2023-09-10T07:49:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30611.143991 secs 2023-09-10T07:49:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:40 smithi082 ceph-mon[98694]: pgmap v16485: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 119 KiB/s wr, 9 op/s 2023-09-10T07:49:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:42 smithi098 ceph-mon[103124]: pgmap v16486: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 693 KiB/s wr, 16 op/s 2023-09-10T07:49:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:42 smithi170 ceph-mon[102832]: pgmap v16486: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 693 KiB/s wr, 16 op/s 2023-09-10T07:49:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:42 smithi082 ceph-mon[98694]: pgmap v16486: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 693 KiB/s wr, 16 op/s 2023-09-10T07:49:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:44 smithi098 ceph-mon[103124]: pgmap v16487: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 693 KiB/s wr, 12 op/s 2023-09-10T07:49:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30615.698897 secs 2023-09-10T07:49:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:44 smithi170 ceph-mon[102832]: pgmap v16487: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 693 KiB/s wr, 12 op/s 2023-09-10T07:49:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30615.698897 secs 2023-09-10T07:49:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:44 smithi082 ceph-mon[98694]: pgmap v16487: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 693 KiB/s wr, 12 op/s 2023-09-10T07:49:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30615.698897 secs 2023-09-10T07:49:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30616.144051 secs 2023-09-10T07:49:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30616.144051 secs 2023-09-10T07:49:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30616.144051 secs 2023-09-10T07:49:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:46 smithi098 ceph-mon[103124]: pgmap v16488: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 871 KiB/s wr, 19 op/s 2023-09-10T07:49:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:46 smithi170 ceph-mon[102832]: pgmap v16488: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 871 KiB/s wr, 19 op/s 2023-09-10T07:49:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:46 smithi082 ceph-mon[98694]: pgmap v16488: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 871 KiB/s wr, 19 op/s 2023-09-10T07:49:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:48 smithi098 ceph-mon[103124]: pgmap v16489: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 753 KiB/s wr, 14 op/s 2023-09-10T07:49:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:48 smithi170 ceph-mon[102832]: pgmap v16489: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 753 KiB/s wr, 14 op/s 2023-09-10T07:49:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:48 smithi082 ceph-mon[98694]: pgmap v16489: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 753 KiB/s wr, 14 op/s 2023-09-10T07:49:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30620.698955 secs 2023-09-10T07:49:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30620.698955 secs 2023-09-10T07:49:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30620.698955 secs 2023-09-10T07:49:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30621.144115 secs 2023-09-10T07:49:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:50 smithi098 ceph-mon[103124]: pgmap v16490: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 753 KiB/s wr, 14 op/s 2023-09-10T07:49:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30621.144115 secs 2023-09-10T07:49:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:50 smithi170 ceph-mon[102832]: pgmap v16490: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 753 KiB/s wr, 14 op/s 2023-09-10T07:49:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30621.144115 secs 2023-09-10T07:49:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:50 smithi082 ceph-mon[98694]: pgmap v16490: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 753 KiB/s wr, 14 op/s 2023-09-10T07:49:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:52 smithi098 ceph-mon[103124]: pgmap v16491: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T07:49:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:52 smithi170 ceph-mon[102832]: pgmap v16491: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T07:49:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:52 smithi082 ceph-mon[98694]: pgmap v16491: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T07:49:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30625.699014 secs 2023-09-10T07:49:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30625.699014 secs 2023-09-10T07:49:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30625.699014 secs 2023-09-10T07:49:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30624.534159 secs 2023-09-10T07:49:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:54 smithi098 ceph-mon[103124]: pgmap v16492: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 541 KiB/s wr, 15 op/s 2023-09-10T07:49:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30624.534159 secs 2023-09-10T07:49:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:54 smithi170 ceph-mon[102832]: pgmap v16492: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 541 KiB/s wr, 15 op/s 2023-09-10T07:49:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30624.534159 secs 2023-09-10T07:49:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:54 smithi082 ceph-mon[98694]: pgmap v16492: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 541 KiB/s wr, 15 op/s 2023-09-10T07:49:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30626.144171 secs 2023-09-10T07:49:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30626.144171 secs 2023-09-10T07:49:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30626.144171 secs 2023-09-10T07:49:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:56 smithi098 ceph-mon[103124]: pgmap v16493: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 541 KiB/s wr, 19 op/s 2023-09-10T07:49:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:49:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:49:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:49:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:56 smithi170 ceph-mon[102832]: pgmap v16493: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 541 KiB/s wr, 19 op/s 2023-09-10T07:49:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:49:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:49:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:49:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:56 smithi082 ceph-mon[98694]: pgmap v16493: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 541 KiB/s wr, 19 op/s 2023-09-10T07:49:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:49:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:49:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:49:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:58 smithi098 ceph-mon[103124]: pgmap v16494: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 363 KiB/s wr, 12 op/s 2023-09-10T07:49:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:58 smithi170 ceph-mon[102832]: pgmap v16494: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 363 KiB/s wr, 12 op/s 2023-09-10T07:49:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:58 smithi082 ceph-mon[98694]: pgmap v16494: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 363 KiB/s wr, 12 op/s 2023-09-10T07:49:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30629.534220 secs 2023-09-10T07:49:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:49:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30630.699072 secs 2023-09-10T07:49:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30629.534220 secs 2023-09-10T07:49:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:49:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30630.699072 secs 2023-09-10T07:49:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30629.534220 secs 2023-09-10T07:49:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:49:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30630.699072 secs 2023-09-10T07:50:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30631.144232 secs 2023-09-10T07:50:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:00 smithi098 ceph-mon[103124]: pgmap v16495: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 363 KiB/s wr, 12 op/s 2023-09-10T07:50:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T07:50:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30631.144232 secs 2023-09-10T07:50:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:00 smithi170 ceph-mon[102832]: pgmap v16495: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 363 KiB/s wr, 12 op/s 2023-09-10T07:50:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T07:50:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30631.144232 secs 2023-09-10T07:50:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:00 smithi082 ceph-mon[98694]: pgmap v16495: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 363 KiB/s wr, 12 op/s 2023-09-10T07:50:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests 2023-09-10T07:50:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:02 smithi098 ceph-mon[103124]: pgmap v16496: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 595 KiB/s wr, 17 op/s 2023-09-10T07:50:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:02 smithi170 ceph-mon[102832]: pgmap v16496: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 595 KiB/s wr, 17 op/s 2023-09-10T07:50:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:02 smithi082 ceph-mon[98694]: pgmap v16496: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 595 KiB/s wr, 17 op/s 2023-09-10T07:50:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30635.699131 secs 2023-09-10T07:50:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30635.699131 secs 2023-09-10T07:50:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30635.699131 secs 2023-09-10T07:50:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30634.534284 secs 2023-09-10T07:50:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:04 smithi098 ceph-mon[103124]: pgmap v16497: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 233 KiB/s wr, 10 op/s 2023-09-10T07:50:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30634.534284 secs 2023-09-10T07:50:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:04 smithi170 ceph-mon[102832]: pgmap v16497: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 233 KiB/s wr, 10 op/s 2023-09-10T07:50:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30634.534284 secs 2023-09-10T07:50:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:04 smithi082 ceph-mon[98694]: pgmap v16497: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 233 KiB/s wr, 10 op/s 2023-09-10T07:50:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30636.144291 secs 2023-09-10T07:50:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30636.144291 secs 2023-09-10T07:50:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30636.144291 secs 2023-09-10T07:50:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:06 smithi098 ceph-mon[103124]: pgmap v16498: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 362 KiB/s wr, 29 op/s 2023-09-10T07:50:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:06 smithi170 ceph-mon[102832]: pgmap v16498: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 362 KiB/s wr, 29 op/s 2023-09-10T07:50:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:06 smithi082 ceph-mon[98694]: pgmap v16498: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 362 KiB/s wr, 29 op/s 2023-09-10T07:50:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:08 smithi098 ceph-mon[103124]: pgmap v16499: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 362 KiB/s wr, 24 op/s 2023-09-10T07:50:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:08 smithi170 ceph-mon[102832]: pgmap v16499: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 362 KiB/s wr, 24 op/s 2023-09-10T07:50:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:08 smithi082 ceph-mon[98694]: pgmap v16499: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 362 KiB/s wr, 24 op/s 2023-09-10T07:50:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30639.534388 secs 2023-09-10T07:50:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:09 smithi098 ceph-mon[103124]: osdmap e1099: 12 total, 12 up, 12 in 2023-09-10T07:50:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30640.699188 secs 2023-09-10T07:50:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30639.534388 secs 2023-09-10T07:50:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:09 smithi170 ceph-mon[102832]: osdmap e1099: 12 total, 12 up, 12 in 2023-09-10T07:50:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30640.699188 secs 2023-09-10T07:50:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30639.534388 secs 2023-09-10T07:50:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:09 smithi082 ceph-mon[98694]: osdmap e1099: 12 total, 12 up, 12 in 2023-09-10T07:50:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30640.699188 secs 2023-09-10T07:50:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30641.144353 secs 2023-09-10T07:50:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:10 smithi098 ceph-mon[103124]: pgmap v16501: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 434 KiB/s wr, 29 op/s 2023-09-10T07:50:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30641.144353 secs 2023-09-10T07:50:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:10 smithi170 ceph-mon[102832]: pgmap v16501: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 434 KiB/s wr, 29 op/s 2023-09-10T07:50:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30641.144353 secs 2023-09-10T07:50:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:10 smithi082 ceph-mon[98694]: pgmap v16501: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 434 KiB/s wr, 29 op/s 2023-09-10T07:50:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:12 smithi098 ceph-mon[103124]: pgmap v16502: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 156 KiB/s wr, 28 op/s 2023-09-10T07:50:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:12 smithi098 ceph-mon[103124]: osdmap e1100: 12 total, 12 up, 12 in 2023-09-10T07:50:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:12 smithi170 ceph-mon[102832]: pgmap v16502: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 156 KiB/s wr, 28 op/s 2023-09-10T07:50:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:12 smithi170 ceph-mon[102832]: osdmap e1100: 12 total, 12 up, 12 in 2023-09-10T07:50:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:12 smithi082 ceph-mon[98694]: pgmap v16502: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 156 KiB/s wr, 28 op/s 2023-09-10T07:50:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:12 smithi082 ceph-mon[98694]: osdmap e1100: 12 total, 12 up, 12 in 2023-09-10T07:50:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30645.699273 secs 2023-09-10T07:50:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30645.699273 secs 2023-09-10T07:50:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30645.699273 secs 2023-09-10T07:50:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30644.534453 secs 2023-09-10T07:50:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:14 smithi098 ceph-mon[103124]: pgmap v16504: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:50:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30644.534453 secs 2023-09-10T07:50:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:14 smithi170 ceph-mon[102832]: pgmap v16504: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:50:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30644.534453 secs 2023-09-10T07:50:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:14 smithi082 ceph-mon[98694]: pgmap v16504: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:50:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30646.144410 secs 2023-09-10T07:50:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30646.144410 secs 2023-09-10T07:50:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30646.144410 secs 2023-09-10T07:50:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:16 smithi098 ceph-mon[103124]: pgmap v16505: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T07:50:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:16 smithi170 ceph-mon[102832]: pgmap v16505: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T07:50:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:16 smithi082 ceph-mon[98694]: pgmap v16505: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T07:50:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:18 smithi098 ceph-mon[103124]: pgmap v16506: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 11 op/s 2023-09-10T07:50:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:18 smithi170 ceph-mon[102832]: pgmap v16506: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 11 op/s 2023-09-10T07:50:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:18 smithi082 ceph-mon[98694]: pgmap v16506: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 11 op/s 2023-09-10T07:50:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30649.534564 secs 2023-09-10T07:50:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30650.699332 secs 2023-09-10T07:50:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30649.534564 secs 2023-09-10T07:50:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30650.699332 secs 2023-09-10T07:50:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30649.534564 secs 2023-09-10T07:50:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30650.699332 secs 2023-09-10T07:50:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30651.144480 secs 2023-09-10T07:50:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:20 smithi098 ceph-mon[103124]: pgmap v16507: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:50:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30651.144480 secs 2023-09-10T07:50:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:20 smithi170 ceph-mon[102832]: pgmap v16507: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:50:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30651.144480 secs 2023-09-10T07:50:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:20 smithi082 ceph-mon[98694]: pgmap v16507: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:50:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:22 smithi098 ceph-mon[103124]: pgmap v16508: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:50:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:22 smithi170 ceph-mon[102832]: pgmap v16508: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:50:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:22 smithi082 ceph-mon[98694]: pgmap v16508: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:50:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30655.699390 secs 2023-09-10T07:50:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30655.699390 secs 2023-09-10T07:50:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30655.699390 secs 2023-09-10T07:50:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30654.534625 secs 2023-09-10T07:50:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:24 smithi098 ceph-mon[103124]: pgmap v16509: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:50:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30654.534625 secs 2023-09-10T07:50:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:24 smithi170 ceph-mon[102832]: pgmap v16509: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:50:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30654.534625 secs 2023-09-10T07:50:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:24 smithi082 ceph-mon[98694]: pgmap v16509: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:50:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30656.144537 secs 2023-09-10T07:50:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30656.144537 secs 2023-09-10T07:50:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30656.144537 secs 2023-09-10T07:50:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:26 smithi098 ceph-mon[103124]: pgmap v16510: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 511 B/s wr, 13 op/s 2023-09-10T07:50:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:26 smithi170 ceph-mon[102832]: pgmap v16510: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 511 B/s wr, 13 op/s 2023-09-10T07:50:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:26 smithi082 ceph-mon[98694]: pgmap v16510: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 511 B/s wr, 13 op/s 2023-09-10T07:50:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:28 smithi098 ceph-mon[103124]: pgmap v16511: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:50:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:28 smithi170 ceph-mon[102832]: pgmap v16511: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:50:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:28 smithi082 ceph-mon[98694]: pgmap v16511: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:50:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30659.534686 secs 2023-09-10T07:50:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30660.699450 secs 2023-09-10T07:50:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30659.534686 secs 2023-09-10T07:50:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30660.699450 secs 2023-09-10T07:50:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30659.534686 secs 2023-09-10T07:50:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30660.699450 secs 2023-09-10T07:50:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30661.144601 secs 2023-09-10T07:50:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:30 smithi098 ceph-mon[103124]: pgmap v16512: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:50:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30661.144601 secs 2023-09-10T07:50:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:30 smithi170 ceph-mon[102832]: pgmap v16512: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:50:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30661.144601 secs 2023-09-10T07:50:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:30 smithi082 ceph-mon[98694]: pgmap v16512: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:50:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:32 smithi098 ceph-mon[103124]: pgmap v16513: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 108 KiB/s wr, 14 op/s 2023-09-10T07:50:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:32 smithi170 ceph-mon[102832]: pgmap v16513: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 108 KiB/s wr, 14 op/s 2023-09-10T07:50:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:32 smithi082 ceph-mon[98694]: pgmap v16513: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 108 KiB/s wr, 14 op/s 2023-09-10T07:50:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:50:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30665.699500 secs 2023-09-10T07:50:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:50:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30665.699500 secs 2023-09-10T07:50:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:50:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30665.699500 secs 2023-09-10T07:50:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:34 smithi098 ceph-mon[103124]: pgmap v16514: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 10 op/s 2023-09-10T07:50:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:50:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:50:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:50:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:34 smithi170 ceph-mon[102832]: pgmap v16514: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 10 op/s 2023-09-10T07:50:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:50:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:50:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:50:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:34 smithi082 ceph-mon[98694]: pgmap v16514: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 108 KiB/s wr, 10 op/s 2023-09-10T07:50:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:50:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:50:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:50:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30666.144664 secs 2023-09-10T07:50:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:50:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30666.144664 secs 2023-09-10T07:50:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:50:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30666.144664 secs 2023-09-10T07:50:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:50:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:36 smithi098 ceph-mon[103124]: pgmap v16515: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 392 KiB/s wr, 15 op/s 2023-09-10T07:50:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:50:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:50:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:50:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:36 smithi170 ceph-mon[102832]: pgmap v16515: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 392 KiB/s wr, 15 op/s 2023-09-10T07:50:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:50:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:50:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:50:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:36 smithi082 ceph-mon[98694]: pgmap v16515: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 392 KiB/s wr, 15 op/s 2023-09-10T07:50:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:50:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:50:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:50:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:38 smithi098 ceph-mon[103124]: pgmap v16516: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 392 KiB/s wr, 11 op/s 2023-09-10T07:50:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:38 smithi170 ceph-mon[102832]: pgmap v16516: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 392 KiB/s wr, 11 op/s 2023-09-10T07:50:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:38 smithi082 ceph-mon[98694]: pgmap v16516: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 392 KiB/s wr, 11 op/s 2023-09-10T07:50:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30670.699586 secs 2023-09-10T07:50:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30670.699586 secs 2023-09-10T07:50:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30670.699586 secs 2023-09-10T07:50:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30671.144723 secs 2023-09-10T07:50:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:40 smithi098 ceph-mon[103124]: pgmap v16517: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 392 KiB/s wr, 11 op/s 2023-09-10T07:50:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30671.144723 secs 2023-09-10T07:50:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:40 smithi170 ceph-mon[102832]: pgmap v16517: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 392 KiB/s wr, 11 op/s 2023-09-10T07:50:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30671.144723 secs 2023-09-10T07:50:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:40 smithi082 ceph-mon[98694]: pgmap v16517: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 392 KiB/s wr, 11 op/s 2023-09-10T07:50:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:42 smithi098 ceph-mon[103124]: pgmap v16518: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 549 KiB/s wr, 17 op/s 2023-09-10T07:50:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:42 smithi170 ceph-mon[102832]: pgmap v16518: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 549 KiB/s wr, 17 op/s 2023-09-10T07:50:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:42 smithi082 ceph-mon[98694]: pgmap v16518: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 549 KiB/s wr, 17 op/s 2023-09-10T07:50:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30675.701144 secs 2023-09-10T07:50:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30675.701144 secs 2023-09-10T07:50:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30675.701144 secs 2023-09-10T07:50:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:44 smithi098 ceph-mon[103124]: pgmap v16519: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 441 KiB/s wr, 11 op/s 2023-09-10T07:50:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:44 smithi170 ceph-mon[102832]: pgmap v16519: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 441 KiB/s wr, 11 op/s 2023-09-10T07:50:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:44 smithi082 ceph-mon[98694]: pgmap v16519: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 441 KiB/s wr, 11 op/s 2023-09-10T07:50:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30676.144782 secs 2023-09-10T07:50:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30676.144782 secs 2023-09-10T07:50:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30676.144782 secs 2023-09-10T07:50:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:46 smithi098 ceph-mon[103124]: pgmap v16520: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 623 KiB/s wr, 19 op/s 2023-09-10T07:50:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:46 smithi170 ceph-mon[102832]: pgmap v16520: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 623 KiB/s wr, 19 op/s 2023-09-10T07:50:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:46 smithi082 ceph-mon[98694]: pgmap v16520: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 623 KiB/s wr, 19 op/s 2023-09-10T07:50:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:48 smithi098 ceph-mon[103124]: pgmap v16521: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 339 KiB/s wr, 13 op/s 2023-09-10T07:50:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:48 smithi170 ceph-mon[102832]: pgmap v16521: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 339 KiB/s wr, 13 op/s 2023-09-10T07:50:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:48 smithi082 ceph-mon[98694]: pgmap v16521: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 339 KiB/s wr, 13 op/s 2023-09-10T07:50:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30680.701202 secs 2023-09-10T07:50:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30680.701202 secs 2023-09-10T07:50:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30680.701202 secs 2023-09-10T07:50:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30681.144845 secs 2023-09-10T07:50:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:50 smithi098 ceph-mon[103124]: Health check failed: 1 MDSs behind on trimming (MDS_TRIM) 2023-09-10T07:50:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:50:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:50:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:50 smithi098 ceph-mon[103124]: pgmap v16522: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 339 KiB/s wr, 13 op/s 2023-09-10T07:50:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30681.144845 secs 2023-09-10T07:50:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:50 smithi170 ceph-mon[102832]: Health check failed: 1 MDSs behind on trimming (MDS_TRIM) 2023-09-10T07:50:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:50:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:50:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:50 smithi170 ceph-mon[102832]: pgmap v16522: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 339 KiB/s wr, 13 op/s 2023-09-10T07:50:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30681.144845 secs 2023-09-10T07:50:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:50 smithi082 ceph-mon[98694]: Health check failed: 1 MDSs behind on trimming (MDS_TRIM) 2023-09-10T07:50:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:50:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:50:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:50 smithi082 ceph-mon[98694]: pgmap v16522: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 339 KiB/s wr, 13 op/s 2023-09-10T07:50:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:50:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:50:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:52 smithi170 ceph-mon[102832]: pgmap v16523: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 668 KiB/s wr, 21 op/s 2023-09-10T07:50:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:50:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:50:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:52 smithi082 ceph-mon[98694]: pgmap v16523: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 668 KiB/s wr, 21 op/s 2023-09-10T07:50:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:50:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:50:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:52 smithi098 ceph-mon[103124]: pgmap v16523: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 668 KiB/s wr, 21 op/s 2023-09-10T07:50:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30685.701285 secs 2023-09-10T07:50:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30685.701285 secs 2023-09-10T07:50:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30685.701285 secs 2023-09-10T07:50:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30684.534969 secs 2023-09-10T07:50:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:54 smithi170 ceph-mon[102832]: pgmap v16524: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 510 KiB/s wr, 15 op/s 2023-09-10T07:50:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30684.534969 secs 2023-09-10T07:50:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:54 smithi082 ceph-mon[98694]: pgmap v16524: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 510 KiB/s wr, 15 op/s 2023-09-10T07:50:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30684.534969 secs 2023-09-10T07:50:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:54 smithi098 ceph-mon[103124]: pgmap v16524: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 510 KiB/s wr, 15 op/s 2023-09-10T07:50:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30686.144902 secs 2023-09-10T07:50:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30686.144902 secs 2023-09-10T07:50:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30686.144902 secs 2023-09-10T07:50:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:56 smithi170 ceph-mon[102832]: pgmap v16525: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 537 KiB/s wr, 21 op/s 2023-09-10T07:50:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:50:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:50:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:50:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:56 smithi082 ceph-mon[98694]: pgmap v16525: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 537 KiB/s wr, 21 op/s 2023-09-10T07:50:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:50:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:50:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:50:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:56 smithi098 ceph-mon[103124]: pgmap v16525: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 537 KiB/s wr, 21 op/s 2023-09-10T07:50:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:50:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:50:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:50:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:57 smithi170 ceph-mon[102832]: Health check update: 2 MDSs behind on trimming (MDS_TRIM) 2023-09-10T07:50:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:57 smithi082 ceph-mon[98694]: Health check update: 2 MDSs behind on trimming (MDS_TRIM) 2023-09-10T07:50:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:57 smithi098 ceph-mon[103124]: Health check update: 2 MDSs behind on trimming (MDS_TRIM) 2023-09-10T07:50:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:58 smithi170 ceph-mon[102832]: pgmap v16526: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 356 KiB/s wr, 14 op/s 2023-09-10T07:50:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30690.701368 secs 2023-09-10T07:50:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:58 smithi082 ceph-mon[98694]: pgmap v16526: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 356 KiB/s wr, 14 op/s 2023-09-10T07:50:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30690.701368 secs 2023-09-10T07:50:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:58 smithi098 ceph-mon[103124]: pgmap v16526: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 356 KiB/s wr, 14 op/s 2023-09-10T07:50:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30690.701368 secs 2023-09-10T07:50:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:50:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30689.535031 secs 2023-09-10T07:50:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:50:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30689.535031 secs 2023-09-10T07:51:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:50:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30689.535031 secs 2023-09-10T07:51:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30691.144962 secs 2023-09-10T07:51:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:00 smithi170 ceph-mon[102832]: pgmap v16527: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 356 KiB/s wr, 14 op/s 2023-09-10T07:51:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30691.144962 secs 2023-09-10T07:51:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:00 smithi082 ceph-mon[98694]: pgmap v16527: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 356 KiB/s wr, 14 op/s 2023-09-10T07:51:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30691.144962 secs 2023-09-10T07:51:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:00 smithi098 ceph-mon[103124]: pgmap v16527: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 356 KiB/s wr, 14 op/s 2023-09-10T07:51:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:02 smithi170 ceph-mon[102832]: pgmap v16528: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 660 KiB/s wr, 34 op/s 2023-09-10T07:51:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:02 smithi082 ceph-mon[98694]: pgmap v16528: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 660 KiB/s wr, 34 op/s 2023-09-10T07:51:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:02 smithi098 ceph-mon[103124]: pgmap v16528: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 660 KiB/s wr, 34 op/s 2023-09-10T07:51:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30695.701455 secs 2023-09-10T07:51:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30695.701455 secs 2023-09-10T07:51:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30695.701455 secs 2023-09-10T07:51:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30694.535099 secs 2023-09-10T07:51:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:04 smithi170 ceph-mon[102832]: pgmap v16529: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 331 KiB/s wr, 26 op/s 2023-09-10T07:51:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:51:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30694.535099 secs 2023-09-10T07:51:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:04 smithi082 ceph-mon[98694]: pgmap v16529: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 331 KiB/s wr, 26 op/s 2023-09-10T07:51:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:51:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30694.535099 secs 2023-09-10T07:51:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:04 smithi098 ceph-mon[103124]: pgmap v16529: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 331 KiB/s wr, 26 op/s 2023-09-10T07:51:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:51:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30696.145019 secs 2023-09-10T07:51:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30696.145019 secs 2023-09-10T07:51:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30696.145019 secs 2023-09-10T07:51:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:06 smithi170 ceph-mon[102832]: pgmap v16530: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 428 KiB/s wr, 31 op/s 2023-09-10T07:51:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:51:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:06 smithi082 ceph-mon[98694]: pgmap v16530: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 428 KiB/s wr, 31 op/s 2023-09-10T07:51:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:51:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:06 smithi098 ceph-mon[103124]: pgmap v16530: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 428 KiB/s wr, 31 op/s 2023-09-10T07:51:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:51:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:08 smithi098 ceph-mon[103124]: pgmap v16531: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 401 KiB/s wr, 25 op/s 2023-09-10T07:51:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:08 smithi170 ceph-mon[102832]: pgmap v16531: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 401 KiB/s wr, 25 op/s 2023-09-10T07:51:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:08 smithi082 ceph-mon[98694]: pgmap v16531: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 401 KiB/s wr, 25 op/s 2023-09-10T07:51:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30699.535167 secs 2023-09-10T07:51:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:09 smithi098 ceph-mon[103124]: osdmap e1101: 12 total, 12 up, 12 in 2023-09-10T07:51:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30700.701520 secs 2023-09-10T07:51:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30699.535167 secs 2023-09-10T07:51:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:09 smithi170 ceph-mon[102832]: osdmap e1101: 12 total, 12 up, 12 in 2023-09-10T07:51:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30700.701520 secs 2023-09-10T07:51:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30699.535167 secs 2023-09-10T07:51:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:09 smithi082 ceph-mon[98694]: osdmap e1101: 12 total, 12 up, 12 in 2023-09-10T07:51:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30700.701520 secs 2023-09-10T07:51:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30701.145079 secs 2023-09-10T07:51:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:10 smithi098 ceph-mon[103124]: pgmap v16533: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 481 KiB/s wr, 30 op/s 2023-09-10T07:51:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30701.145079 secs 2023-09-10T07:51:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:10 smithi170 ceph-mon[102832]: pgmap v16533: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 481 KiB/s wr, 30 op/s 2023-09-10T07:51:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30701.145079 secs 2023-09-10T07:51:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:10 smithi082 ceph-mon[98694]: pgmap v16533: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 481 KiB/s wr, 30 op/s 2023-09-10T07:51:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:12 smithi098 ceph-mon[103124]: pgmap v16534: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T07:51:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:12 smithi098 ceph-mon[103124]: osdmap e1102: 12 total, 12 up, 12 in 2023-09-10T07:51:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:12 smithi170 ceph-mon[102832]: pgmap v16534: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T07:51:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:12 smithi170 ceph-mon[102832]: osdmap e1102: 12 total, 12 up, 12 in 2023-09-10T07:51:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:12 smithi082 ceph-mon[98694]: pgmap v16534: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T07:51:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:12 smithi082 ceph-mon[98694]: osdmap e1102: 12 total, 12 up, 12 in 2023-09-10T07:51:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30705.701575 secs 2023-09-10T07:51:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30705.701575 secs 2023-09-10T07:51:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30705.701575 secs 2023-09-10T07:51:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30704.535227 secs 2023-09-10T07:51:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:14 smithi098 ceph-mon[103124]: pgmap v16536: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:51:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30704.535227 secs 2023-09-10T07:51:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:14 smithi170 ceph-mon[102832]: pgmap v16536: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:51:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30704.535227 secs 2023-09-10T07:51:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:14 smithi082 ceph-mon[98694]: pgmap v16536: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:51:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30706.145139 secs 2023-09-10T07:51:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30706.145139 secs 2023-09-10T07:51:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30706.145139 secs 2023-09-10T07:51:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:16 smithi098 ceph-mon[103124]: pgmap v16537: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T07:51:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:16 smithi170 ceph-mon[102832]: pgmap v16537: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T07:51:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:16 smithi082 ceph-mon[98694]: pgmap v16537: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T07:51:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:18 smithi098 ceph-mon[103124]: pgmap v16538: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 11 op/s 2023-09-10T07:51:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:18 smithi170 ceph-mon[102832]: pgmap v16538: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 11 op/s 2023-09-10T07:51:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:18 smithi082 ceph-mon[98694]: pgmap v16538: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 11 op/s 2023-09-10T07:51:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30709.535292 secs 2023-09-10T07:51:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30710.701634 secs 2023-09-10T07:51:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30709.535292 secs 2023-09-10T07:51:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30710.701634 secs 2023-09-10T07:51:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30709.535292 secs 2023-09-10T07:51:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30710.701634 secs 2023-09-10T07:51:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30711.145203 secs 2023-09-10T07:51:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:20 smithi170 ceph-mon[102832]: pgmap v16539: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:51:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30711.145203 secs 2023-09-10T07:51:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:20 smithi082 ceph-mon[98694]: pgmap v16539: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:51:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30711.145203 secs 2023-09-10T07:51:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:20 smithi098 ceph-mon[103124]: pgmap v16539: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:51:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:22 smithi098 ceph-mon[103124]: pgmap v16540: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:51:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:22 smithi170 ceph-mon[102832]: pgmap v16540: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:51:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:22 smithi082 ceph-mon[98694]: pgmap v16540: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:51:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30715.701693 secs 2023-09-10T07:51:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30715.701693 secs 2023-09-10T07:51:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30715.701693 secs 2023-09-10T07:51:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30714.535394 secs 2023-09-10T07:51:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:24 smithi098 ceph-mon[103124]: pgmap v16541: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:51:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30714.535394 secs 2023-09-10T07:51:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:24 smithi170 ceph-mon[102832]: pgmap v16541: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:51:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30714.535394 secs 2023-09-10T07:51:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:24 smithi082 ceph-mon[98694]: pgmap v16541: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:51:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30716.145262 secs 2023-09-10T07:51:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30716.145262 secs 2023-09-10T07:51:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30716.145262 secs 2023-09-10T07:51:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:26 smithi098 ceph-mon[103124]: pgmap v16542: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-10T07:51:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:26 smithi170 ceph-mon[102832]: pgmap v16542: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-10T07:51:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:26 smithi082 ceph-mon[98694]: pgmap v16542: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-10T07:51:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:28 smithi098 ceph-mon[103124]: pgmap v16543: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-10T07:51:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:28 smithi170 ceph-mon[102832]: pgmap v16543: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-10T07:51:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:28 smithi082 ceph-mon[98694]: pgmap v16543: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-10T07:51:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30719.535454 secs 2023-09-10T07:51:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30720.701752 secs 2023-09-10T07:51:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30719.535454 secs 2023-09-10T07:51:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30720.701752 secs 2023-09-10T07:51:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30719.535454 secs 2023-09-10T07:51:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30720.701752 secs 2023-09-10T07:51:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30721.145323 secs 2023-09-10T07:51:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:30 smithi170 ceph-mon[102832]: pgmap v16544: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-10T07:51:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30721.145323 secs 2023-09-10T07:51:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:30 smithi082 ceph-mon[98694]: pgmap v16544: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-10T07:51:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30721.145323 secs 2023-09-10T07:51:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:30 smithi098 ceph-mon[103124]: pgmap v16544: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-10T07:51:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:32 smithi098 ceph-mon[103124]: pgmap v16545: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 94 KiB/s wr, 13 op/s 2023-09-10T07:51:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:32 smithi170 ceph-mon[102832]: pgmap v16545: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 94 KiB/s wr, 13 op/s 2023-09-10T07:51:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:32 smithi082 ceph-mon[98694]: pgmap v16545: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 94 KiB/s wr, 13 op/s 2023-09-10T07:51:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30725.701811 secs 2023-09-10T07:51:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30725.701811 secs 2023-09-10T07:51:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30725.701811 secs 2023-09-10T07:51:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:34 smithi098 ceph-mon[103124]: pgmap v16546: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 9 op/s 2023-09-10T07:51:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:51:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:34 smithi170 ceph-mon[102832]: pgmap v16546: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 9 op/s 2023-09-10T07:51:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:51:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:34 smithi082 ceph-mon[98694]: pgmap v16546: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 9 op/s 2023-09-10T07:51:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:51:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30726.145383 secs 2023-09-10T07:51:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:51:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:51:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:51:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:51:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30726.145383 secs 2023-09-10T07:51:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:51:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:51:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:51:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:51:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30726.145383 secs 2023-09-10T07:51:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:51:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:51:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:51:35.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:51:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:36 smithi098 ceph-mon[103124]: pgmap v16547: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 107 KiB/s wr, 14 op/s 2023-09-10T07:51:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:51:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:51:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:36 smithi170 ceph-mon[102832]: pgmap v16547: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 107 KiB/s wr, 14 op/s 2023-09-10T07:51:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:51:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:51:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:36 smithi082 ceph-mon[98694]: pgmap v16547: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 107 KiB/s wr, 14 op/s 2023-09-10T07:51:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:51:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:51:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:38 smithi170 ceph-mon[102832]: pgmap v16548: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 107 KiB/s wr, 10 op/s 2023-09-10T07:51:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:51:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30730.701870 secs 2023-09-10T07:51:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:38 smithi082 ceph-mon[98694]: pgmap v16548: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 107 KiB/s wr, 10 op/s 2023-09-10T07:51:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:51:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30730.701870 secs 2023-09-10T07:51:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:38 smithi098 ceph-mon[103124]: pgmap v16548: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 107 KiB/s wr, 10 op/s 2023-09-10T07:51:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:51:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30730.701870 secs 2023-09-10T07:51:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30731.145447 secs 2023-09-10T07:51:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:40 smithi170 ceph-mon[102832]: pgmap v16549: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 107 KiB/s wr, 10 op/s 2023-09-10T07:51:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:51:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30731.145447 secs 2023-09-10T07:51:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:40 smithi082 ceph-mon[98694]: pgmap v16549: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 107 KiB/s wr, 10 op/s 2023-09-10T07:51:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:51:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30731.145447 secs 2023-09-10T07:51:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:40 smithi098 ceph-mon[103124]: pgmap v16549: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 107 KiB/s wr, 10 op/s 2023-09-10T07:51:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:51:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:42 smithi170 ceph-mon[102832]: pgmap v16550: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 680 KiB/s wr, 18 op/s 2023-09-10T07:51:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:42 smithi082 ceph-mon[98694]: pgmap v16550: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 680 KiB/s wr, 18 op/s 2023-09-10T07:51:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:42 smithi098 ceph-mon[103124]: pgmap v16550: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 680 KiB/s wr, 18 op/s 2023-09-10T07:51:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30735.701936 secs 2023-09-10T07:51:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30735.701936 secs 2023-09-10T07:51:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30735.701936 secs 2023-09-10T07:51:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:44 smithi170 ceph-mon[102832]: pgmap v16551: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 586 KiB/s wr, 12 op/s 2023-09-10T07:51:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:44 smithi082 ceph-mon[98694]: pgmap v16551: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 586 KiB/s wr, 12 op/s 2023-09-10T07:51:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:44 smithi098 ceph-mon[103124]: pgmap v16551: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 586 KiB/s wr, 12 op/s 2023-09-10T07:51:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30736.145481 secs 2023-09-10T07:51:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30736.145481 secs 2023-09-10T07:51:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30736.145481 secs 2023-09-10T07:51:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:46 smithi170 ceph-mon[102832]: pgmap v16552: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 753 KiB/s wr, 19 op/s 2023-09-10T07:51:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:51:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:46 smithi082 ceph-mon[98694]: pgmap v16552: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 753 KiB/s wr, 19 op/s 2023-09-10T07:51:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:51:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:46 smithi098 ceph-mon[103124]: pgmap v16552: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 753 KiB/s wr, 19 op/s 2023-09-10T07:51:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:51:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:48 smithi098 ceph-mon[103124]: pgmap v16553: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 740 KiB/s wr, 14 op/s 2023-09-10T07:51:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:48 smithi170 ceph-mon[102832]: pgmap v16553: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 740 KiB/s wr, 14 op/s 2023-09-10T07:51:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:48 smithi082 ceph-mon[98694]: pgmap v16553: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 740 KiB/s wr, 14 op/s 2023-09-10T07:51:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:49 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30739.535742 secs 2023-09-10T07:51:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30740.701993 secs 2023-09-10T07:51:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:49 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30739.535742 secs 2023-09-10T07:51:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30740.701993 secs 2023-09-10T07:51:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:49 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30739.535742 secs 2023-09-10T07:51:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30740.701993 secs 2023-09-10T07:51:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30741.145545 secs 2023-09-10T07:51:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:51:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:50 smithi170 ceph-mon[102832]: pgmap v16554: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 740 KiB/s wr, 14 op/s 2023-09-10T07:51:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30741.145545 secs 2023-09-10T07:51:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:51:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:50 smithi082 ceph-mon[98694]: pgmap v16554: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 740 KiB/s wr, 14 op/s 2023-09-10T07:51:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30741.145545 secs 2023-09-10T07:51:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:51:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:50 smithi098 ceph-mon[103124]: pgmap v16554: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 740 KiB/s wr, 14 op/s 2023-09-10T07:51:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:51:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:51:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:51:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:51:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:51:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:51:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:52 smithi170 ceph-mon[102832]: pgmap v16555: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T07:51:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:52 smithi082 ceph-mon[98694]: pgmap v16555: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T07:51:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:52 smithi098 ceph-mon[103124]: pgmap v16555: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T07:51:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30744.535776 secs 2023-09-10T07:51:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:51:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30745.702050 secs 2023-09-10T07:51:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:54 smithi170 ceph-mon[102832]: pgmap v16556: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 562 KiB/s wr, 14 op/s 2023-09-10T07:51:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30744.535776 secs 2023-09-10T07:51:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:51:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30745.702050 secs 2023-09-10T07:51:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:54 smithi082 ceph-mon[98694]: pgmap v16556: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 562 KiB/s wr, 14 op/s 2023-09-10T07:51:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30744.535776 secs 2023-09-10T07:51:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:51:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30745.702050 secs 2023-09-10T07:51:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:54 smithi098 ceph-mon[103124]: pgmap v16556: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 562 KiB/s wr, 14 op/s 2023-09-10T07:51:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30746.145603 secs 2023-09-10T07:51:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30746.145603 secs 2023-09-10T07:51:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30746.145603 secs 2023-09-10T07:51:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:51:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:56 smithi170 ceph-mon[102832]: pgmap v16557: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 562 KiB/s wr, 20 op/s 2023-09-10T07:51:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:51:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:56 smithi082 ceph-mon[98694]: pgmap v16557: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 562 KiB/s wr, 20 op/s 2023-09-10T07:51:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:51:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:51:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:56 smithi098 ceph-mon[103124]: pgmap v16557: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 562 KiB/s wr, 20 op/s 2023-09-10T07:51:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:58 smithi098 ceph-mon[103124]: pgmap v16558: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 394 KiB/s wr, 13 op/s 2023-09-10T07:51:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:58 smithi170 ceph-mon[102832]: pgmap v16558: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 394 KiB/s wr, 13 op/s 2023-09-10T07:51:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:58 smithi082 ceph-mon[98694]: pgmap v16558: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 394 KiB/s wr, 13 op/s 2023-09-10T07:51:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30749.535837 secs 2023-09-10T07:51:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:51:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30750.702111 secs 2023-09-10T07:51:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30749.535837 secs 2023-09-10T07:51:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:51:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30750.702111 secs 2023-09-10T07:51:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30749.535837 secs 2023-09-10T07:51:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:51:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30750.702111 secs 2023-09-10T07:52:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30751.145670 secs 2023-09-10T07:52:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:00 smithi170 ceph-mon[102832]: pgmap v16559: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 394 KiB/s wr, 13 op/s 2023-09-10T07:52:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30751.145670 secs 2023-09-10T07:52:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:00 smithi082 ceph-mon[98694]: pgmap v16559: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 394 KiB/s wr, 13 op/s 2023-09-10T07:52:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30751.145670 secs 2023-09-10T07:52:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:00 smithi098 ceph-mon[103124]: pgmap v16559: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 394 KiB/s wr, 13 op/s 2023-09-10T07:52:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:02 smithi170 ceph-mon[102832]: pgmap v16560: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 639 KiB/s wr, 21 op/s 2023-09-10T07:52:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:02 smithi082 ceph-mon[98694]: pgmap v16560: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 639 KiB/s wr, 21 op/s 2023-09-10T07:52:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:02 smithi098 ceph-mon[103124]: pgmap v16560: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 639 KiB/s wr, 21 op/s 2023-09-10T07:52:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30755.702170 secs 2023-09-10T07:52:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30755.702170 secs 2023-09-10T07:52:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30755.702170 secs 2023-09-10T07:52:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30754.535901 secs 2023-09-10T07:52:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:04 smithi170 ceph-mon[102832]: pgmap v16561: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 245 KiB/s wr, 13 op/s 2023-09-10T07:52:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30754.535901 secs 2023-09-10T07:52:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:04 smithi082 ceph-mon[98694]: pgmap v16561: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 245 KiB/s wr, 13 op/s 2023-09-10T07:52:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30754.535901 secs 2023-09-10T07:52:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:04 smithi098 ceph-mon[103124]: pgmap v16561: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 245 KiB/s wr, 13 op/s 2023-09-10T07:52:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30756.145727 secs 2023-09-10T07:52:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30756.145727 secs 2023-09-10T07:52:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30756.145727 secs 2023-09-10T07:52:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:06 smithi170 ceph-mon[102832]: pgmap v16562: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 364 KiB/s wr, 29 op/s 2023-09-10T07:52:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:06 smithi082 ceph-mon[98694]: pgmap v16562: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 364 KiB/s wr, 29 op/s 2023-09-10T07:52:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:06 smithi098 ceph-mon[103124]: pgmap v16562: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 364 KiB/s wr, 29 op/s 2023-09-10T07:52:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:08 smithi170 ceph-mon[102832]: pgmap v16563: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 364 KiB/s wr, 24 op/s 2023-09-10T07:52:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:08 smithi082 ceph-mon[98694]: pgmap v16563: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 364 KiB/s wr, 24 op/s 2023-09-10T07:52:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:08 smithi098 ceph-mon[103124]: pgmap v16563: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 364 KiB/s wr, 24 op/s 2023-09-10T07:52:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30759.535965 secs 2023-09-10T07:52:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:09 smithi170 ceph-mon[102832]: osdmap e1103: 12 total, 12 up, 12 in 2023-09-10T07:52:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30760.702232 secs 2023-09-10T07:52:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30759.535965 secs 2023-09-10T07:52:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:09 smithi082 ceph-mon[98694]: osdmap e1103: 12 total, 12 up, 12 in 2023-09-10T07:52:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30760.702232 secs 2023-09-10T07:52:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30759.535965 secs 2023-09-10T07:52:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:09 smithi098 ceph-mon[103124]: osdmap e1103: 12 total, 12 up, 12 in 2023-09-10T07:52:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30760.702232 secs 2023-09-10T07:52:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30761.145793 secs 2023-09-10T07:52:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:10 smithi170 ceph-mon[102832]: pgmap v16565: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 436 KiB/s wr, 29 op/s 2023-09-10T07:52:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30761.145793 secs 2023-09-10T07:52:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:10 smithi082 ceph-mon[98694]: pgmap v16565: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 436 KiB/s wr, 29 op/s 2023-09-10T07:52:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30761.145793 secs 2023-09-10T07:52:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:10 smithi098 ceph-mon[103124]: pgmap v16565: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 436 KiB/s wr, 29 op/s 2023-09-10T07:52:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:12 smithi170 ceph-mon[102832]: pgmap v16566: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 142 KiB/s wr, 24 op/s 2023-09-10T07:52:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:12 smithi170 ceph-mon[102832]: osdmap e1104: 12 total, 12 up, 12 in 2023-09-10T07:52:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:12 smithi082 ceph-mon[98694]: pgmap v16566: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 142 KiB/s wr, 24 op/s 2023-09-10T07:52:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:12 smithi082 ceph-mon[98694]: osdmap e1104: 12 total, 12 up, 12 in 2023-09-10T07:52:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:12 smithi098 ceph-mon[103124]: pgmap v16566: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 142 KiB/s wr, 24 op/s 2023-09-10T07:52:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:12 smithi098 ceph-mon[103124]: osdmap e1104: 12 total, 12 up, 12 in 2023-09-10T07:52:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30765.702315 secs 2023-09-10T07:52:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30765.702315 secs 2023-09-10T07:52:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30765.702315 secs 2023-09-10T07:52:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30764.536023 secs 2023-09-10T07:52:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:14 smithi170 ceph-mon[102832]: pgmap v16568: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:52:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30764.536023 secs 2023-09-10T07:52:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:14 smithi082 ceph-mon[98694]: pgmap v16568: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:52:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30764.536023 secs 2023-09-10T07:52:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:14 smithi098 ceph-mon[103124]: pgmap v16568: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:52:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30766.145850 secs 2023-09-10T07:52:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30766.145850 secs 2023-09-10T07:52:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30766.145850 secs 2023-09-10T07:52:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:16 smithi170 ceph-mon[102832]: pgmap v16569: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T07:52:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:16 smithi082 ceph-mon[98694]: pgmap v16569: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T07:52:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:16 smithi098 ceph-mon[103124]: pgmap v16569: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T07:52:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:18 smithi170 ceph-mon[102832]: pgmap v16570: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:52:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:18 smithi082 ceph-mon[98694]: pgmap v16570: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:52:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:18 smithi098 ceph-mon[103124]: pgmap v16570: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2023-09-10T07:52:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30769.536086 secs 2023-09-10T07:52:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30770.702374 secs 2023-09-10T07:52:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30769.536086 secs 2023-09-10T07:52:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30770.702374 secs 2023-09-10T07:52:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30769.536086 secs 2023-09-10T07:52:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30770.702374 secs 2023-09-10T07:52:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30771.145911 secs 2023-09-10T07:52:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:20 smithi170 ceph-mon[102832]: pgmap v16571: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:52:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30771.145911 secs 2023-09-10T07:52:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:20 smithi082 ceph-mon[98694]: pgmap v16571: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:52:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30771.145911 secs 2023-09-10T07:52:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:20 smithi098 ceph-mon[103124]: pgmap v16571: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:52:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:22 smithi170 ceph-mon[102832]: pgmap v16572: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T07:52:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:22 smithi082 ceph-mon[98694]: pgmap v16572: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T07:52:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:22 smithi098 ceph-mon[103124]: pgmap v16572: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T07:52:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30775.702434 secs 2023-09-10T07:52:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30775.702434 secs 2023-09-10T07:52:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30775.702434 secs 2023-09-10T07:52:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30774.536155 secs 2023-09-10T07:52:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:24 smithi170 ceph-mon[102832]: pgmap v16573: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 10 op/s 2023-09-10T07:52:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30774.536155 secs 2023-09-10T07:52:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:24 smithi082 ceph-mon[98694]: pgmap v16573: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 10 op/s 2023-09-10T07:52:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30774.536155 secs 2023-09-10T07:52:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:24 smithi098 ceph-mon[103124]: pgmap v16573: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 10 op/s 2023-09-10T07:52:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30776.145971 secs 2023-09-10T07:52:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30776.145971 secs 2023-09-10T07:52:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30776.145971 secs 2023-09-10T07:52:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:26 smithi170 ceph-mon[102832]: pgmap v16574: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 103 KiB/s wr, 14 op/s 2023-09-10T07:52:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:26 smithi082 ceph-mon[98694]: pgmap v16574: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 103 KiB/s wr, 14 op/s 2023-09-10T07:52:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:26 smithi098 ceph-mon[103124]: pgmap v16574: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 103 KiB/s wr, 14 op/s 2023-09-10T07:52:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:28 smithi170 ceph-mon[102832]: pgmap v16575: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 10 op/s 2023-09-10T07:52:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:28 smithi082 ceph-mon[98694]: pgmap v16575: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 10 op/s 2023-09-10T07:52:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:28 smithi098 ceph-mon[103124]: pgmap v16575: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 10 op/s 2023-09-10T07:52:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30779.536217 secs 2023-09-10T07:52:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30780.702493 secs 2023-09-10T07:52:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30779.536217 secs 2023-09-10T07:52:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30780.702493 secs 2023-09-10T07:52:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30779.536217 secs 2023-09-10T07:52:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30780.702493 secs 2023-09-10T07:52:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30781.146032 secs 2023-09-10T07:52:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:52:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:52:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:30 smithi170 ceph-mon[102832]: pgmap v16576: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 10 op/s 2023-09-10T07:52:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30781.146032 secs 2023-09-10T07:52:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:52:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:52:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:30 smithi082 ceph-mon[98694]: pgmap v16576: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 10 op/s 2023-09-10T07:52:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30781.146032 secs 2023-09-10T07:52:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:52:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:52:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:30 smithi098 ceph-mon[103124]: pgmap v16576: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 10 op/s 2023-09-10T07:52:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:52:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:52:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:32 smithi170 ceph-mon[102832]: pgmap v16577: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 212 KiB/s wr, 15 op/s 2023-09-10T07:52:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:52:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:52:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:32 smithi082 ceph-mon[98694]: pgmap v16577: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 212 KiB/s wr, 15 op/s 2023-09-10T07:52:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:52:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:52:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:32 smithi098 ceph-mon[103124]: pgmap v16577: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 212 KiB/s wr, 15 op/s 2023-09-10T07:52:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30785.702552 secs 2023-09-10T07:52:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30785.702552 secs 2023-09-10T07:52:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30785.702552 secs 2023-09-10T07:52:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:34 smithi170 ceph-mon[102832]: pgmap v16578: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T07:52:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:34 smithi082 ceph-mon[98694]: pgmap v16578: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T07:52:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:34 smithi098 ceph-mon[103124]: pgmap v16578: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 109 KiB/s wr, 10 op/s 2023-09-10T07:52:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30786.146093 secs 2023-09-10T07:52:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:52:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:52:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30786.146093 secs 2023-09-10T07:52:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:52:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:52:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30786.146093 secs 2023-09-10T07:52:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:52:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:52:36.558 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:36 smithi170 ceph-mon[102832]: pgmap v16579: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 257 KiB/s wr, 16 op/s 2023-09-10T07:52:36.559 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:52:36.559 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:52:36.559 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:52:36.596 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:36 smithi098 ceph-mon[103124]: pgmap v16579: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 257 KiB/s wr, 16 op/s 2023-09-10T07:52:36.596 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:52:36.596 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:52:36.597 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:52:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:36 smithi082 ceph-mon[98694]: pgmap v16579: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 257 KiB/s wr, 16 op/s 2023-09-10T07:52:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:52:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:52:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:52:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:38 smithi082 ceph-mon[98694]: pgmap v16580: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 256 KiB/s wr, 10 op/s 2023-09-10T07:52:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:52:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:52:38.536 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:38 smithi170 ceph-mon[102832]: pgmap v16580: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 256 KiB/s wr, 10 op/s 2023-09-10T07:52:38.536 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:52:38.536 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:52:38.546 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:38 smithi098 ceph-mon[103124]: pgmap v16580: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 256 KiB/s wr, 10 op/s 2023-09-10T07:52:38.546 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:52:38.547 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:52:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30790.702612 secs 2023-09-10T07:52:39.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:52:39.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:52:39.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:52:39.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:52:39.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:52:39.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:52:39.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:52:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30790.702612 secs 2023-09-10T07:52:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:52:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:52:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:52:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:52:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:52:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:52:39.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:52:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30790.702612 secs 2023-09-10T07:52:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:52:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:52:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:52:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:52:39.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:52:39.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:52:39.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:52:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30791.146162 secs 2023-09-10T07:52:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:40 smithi170 ceph-mon[102832]: pgmap v16581: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 256 KiB/s wr, 10 op/s 2023-09-10T07:52:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:52:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:52:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30791.146162 secs 2023-09-10T07:52:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:40 smithi082 ceph-mon[98694]: pgmap v16581: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 256 KiB/s wr, 10 op/s 2023-09-10T07:52:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:52:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:52:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30791.146162 secs 2023-09-10T07:52:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:40 smithi098 ceph-mon[103124]: pgmap v16581: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 256 KiB/s wr, 10 op/s 2023-09-10T07:52:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:52:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:52:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:42 smithi170 ceph-mon[102832]: pgmap v16582: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 554 KiB/s wr, 16 op/s 2023-09-10T07:52:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:42 smithi082 ceph-mon[98694]: pgmap v16582: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 554 KiB/s wr, 16 op/s 2023-09-10T07:52:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:42 smithi098 ceph-mon[103124]: pgmap v16582: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 554 KiB/s wr, 16 op/s 2023-09-10T07:52:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30795.702677 secs 2023-09-10T07:52:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30795.702677 secs 2023-09-10T07:52:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30795.702677 secs 2023-09-10T07:52:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:44 smithi170 ceph-mon[102832]: pgmap v16583: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 445 KiB/s wr, 11 op/s 2023-09-10T07:52:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:44 smithi082 ceph-mon[98694]: pgmap v16583: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 445 KiB/s wr, 11 op/s 2023-09-10T07:52:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:44 smithi098 ceph-mon[103124]: pgmap v16583: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 445 KiB/s wr, 11 op/s 2023-09-10T07:52:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30796.146221 secs 2023-09-10T07:52:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30796.146221 secs 2023-09-10T07:52:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30796.146221 secs 2023-09-10T07:52:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:46 smithi170 ceph-mon[102832]: pgmap v16584: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 605 KiB/s wr, 18 op/s 2023-09-10T07:52:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:52:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:52:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:46 smithi082 ceph-mon[98694]: pgmap v16584: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 605 KiB/s wr, 18 op/s 2023-09-10T07:52:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:52:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:52:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:46 smithi098 ceph-mon[103124]: pgmap v16584: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 605 KiB/s wr, 18 op/s 2023-09-10T07:52:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:52:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:52:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:48 smithi170 ceph-mon[102832]: pgmap v16585: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 457 KiB/s wr, 13 op/s 2023-09-10T07:52:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:48 smithi082 ceph-mon[98694]: pgmap v16585: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 457 KiB/s wr, 13 op/s 2023-09-10T07:52:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:48 smithi098 ceph-mon[103124]: pgmap v16585: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 457 KiB/s wr, 13 op/s 2023-09-10T07:52:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:49 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30799.536458 secs 2023-09-10T07:52:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30800.702733 secs 2023-09-10T07:52:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:49 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30799.536458 secs 2023-09-10T07:52:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30800.702733 secs 2023-09-10T07:52:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:49 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30799.536458 secs 2023-09-10T07:52:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30800.702733 secs 2023-09-10T07:52:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30801.146284 secs 2023-09-10T07:52:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:52:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:52:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:50 smithi170 ceph-mon[102832]: pgmap v16586: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 457 KiB/s wr, 13 op/s 2023-09-10T07:52:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30801.146284 secs 2023-09-10T07:52:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:52:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:52:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:50 smithi082 ceph-mon[98694]: pgmap v16586: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 457 KiB/s wr, 13 op/s 2023-09-10T07:52:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30801.146284 secs 2023-09-10T07:52:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:52:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:52:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:50 smithi098 ceph-mon[103124]: pgmap v16586: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 457 KiB/s wr, 13 op/s 2023-09-10T07:52:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:52:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:52:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:52:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:52:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:52:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:52:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:52:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:52:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:52:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:52 smithi170 ceph-mon[102832]: pgmap v16587: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 820 KiB/s wr, 21 op/s 2023-09-10T07:52:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:52 smithi082 ceph-mon[98694]: pgmap v16587: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 820 KiB/s wr, 21 op/s 2023-09-10T07:52:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:52 smithi098 ceph-mon[103124]: pgmap v16587: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 820 KiB/s wr, 21 op/s 2023-09-10T07:52:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30804.536520 secs 2023-09-10T07:52:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:52:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:52:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30805.702794 secs 2023-09-10T07:52:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:54 smithi170 ceph-mon[102832]: pgmap v16588: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 522 KiB/s wr, 15 op/s 2023-09-10T07:52:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30804.536520 secs 2023-09-10T07:52:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:52:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:52:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30805.702794 secs 2023-09-10T07:52:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:54 smithi082 ceph-mon[98694]: pgmap v16588: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 522 KiB/s wr, 15 op/s 2023-09-10T07:52:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30804.536520 secs 2023-09-10T07:52:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:52:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:52:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30805.702794 secs 2023-09-10T07:52:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:54 smithi098 ceph-mon[103124]: pgmap v16588: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 522 KiB/s wr, 15 op/s 2023-09-10T07:52:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30806.146341 secs 2023-09-10T07:52:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30806.146341 secs 2023-09-10T07:52:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30806.146341 secs 2023-09-10T07:52:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:52:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:52:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:56 smithi170 ceph-mon[102832]: pgmap v16589: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 543 KiB/s wr, 22 op/s 2023-09-10T07:52:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:52:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:52:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:56 smithi082 ceph-mon[98694]: pgmap v16589: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 543 KiB/s wr, 22 op/s 2023-09-10T07:52:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:52:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:52:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:56 smithi098 ceph-mon[103124]: pgmap v16589: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 543 KiB/s wr, 22 op/s 2023-09-10T07:52:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:58 smithi170 ceph-mon[102832]: pgmap v16590: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 384 KiB/s wr, 14 op/s 2023-09-10T07:52:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:58 smithi082 ceph-mon[98694]: pgmap v16590: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 384 KiB/s wr, 14 op/s 2023-09-10T07:52:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:58 smithi098 ceph-mon[103124]: pgmap v16590: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 384 KiB/s wr, 14 op/s 2023-09-10T07:52:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30809.536592 secs 2023-09-10T07:52:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:52:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30810.702852 secs 2023-09-10T07:52:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30809.536592 secs 2023-09-10T07:52:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:52:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30810.702852 secs 2023-09-10T07:52:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30809.536592 secs 2023-09-10T07:52:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:52:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30810.702852 secs 2023-09-10T07:53:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30811.146404 secs 2023-09-10T07:53:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:00 smithi170 ceph-mon[102832]: pgmap v16591: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 384 KiB/s wr, 14 op/s 2023-09-10T07:53:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30811.146404 secs 2023-09-10T07:53:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:00 smithi082 ceph-mon[98694]: pgmap v16591: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 384 KiB/s wr, 14 op/s 2023-09-10T07:53:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30811.146404 secs 2023-09-10T07:53:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:00 smithi098 ceph-mon[103124]: pgmap v16591: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 384 KiB/s wr, 14 op/s 2023-09-10T07:53:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:02 smithi170 ceph-mon[102832]: pgmap v16592: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 635 KiB/s wr, 19 op/s 2023-09-10T07:53:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:02 smithi082 ceph-mon[98694]: pgmap v16592: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 635 KiB/s wr, 19 op/s 2023-09-10T07:53:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:02 smithi098 ceph-mon[103124]: pgmap v16592: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 635 KiB/s wr, 19 op/s 2023-09-10T07:53:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30815.702909 secs 2023-09-10T07:53:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30815.702909 secs 2023-09-10T07:53:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30815.702909 secs 2023-09-10T07:53:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30814.536664 secs 2023-09-10T07:53:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:04 smithi170 ceph-mon[102832]: pgmap v16593: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 273 KiB/s wr, 11 op/s 2023-09-10T07:53:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30814.536664 secs 2023-09-10T07:53:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:04 smithi082 ceph-mon[98694]: pgmap v16593: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 273 KiB/s wr, 11 op/s 2023-09-10T07:53:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30814.536664 secs 2023-09-10T07:53:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:04 smithi098 ceph-mon[103124]: pgmap v16593: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 273 KiB/s wr, 11 op/s 2023-09-10T07:53:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30816.146463 secs 2023-09-10T07:53:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30816.146463 secs 2023-09-10T07:53:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30816.146463 secs 2023-09-10T07:53:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:06 smithi170 ceph-mon[102832]: pgmap v16594: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 403 KiB/s wr, 31 op/s 2023-09-10T07:53:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:06 smithi082 ceph-mon[98694]: pgmap v16594: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 403 KiB/s wr, 31 op/s 2023-09-10T07:53:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:06 smithi098 ceph-mon[103124]: pgmap v16594: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 403 KiB/s wr, 31 op/s 2023-09-10T07:53:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:08 smithi170 ceph-mon[102832]: pgmap v16595: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 382 KiB/s wr, 24 op/s 2023-09-10T07:53:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:08 smithi082 ceph-mon[98694]: pgmap v16595: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 382 KiB/s wr, 24 op/s 2023-09-10T07:53:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:08 smithi098 ceph-mon[103124]: pgmap v16595: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 382 KiB/s wr, 24 op/s 2023-09-10T07:53:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30819.536726 secs 2023-09-10T07:53:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:09 smithi170 ceph-mon[102832]: osdmap e1105: 12 total, 12 up, 12 in 2023-09-10T07:53:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30820.702968 secs 2023-09-10T07:53:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30819.536726 secs 2023-09-10T07:53:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:09 smithi082 ceph-mon[98694]: osdmap e1105: 12 total, 12 up, 12 in 2023-09-10T07:53:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30820.702968 secs 2023-09-10T07:53:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30819.536726 secs 2023-09-10T07:53:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:09 smithi098 ceph-mon[103124]: osdmap e1105: 12 total, 12 up, 12 in 2023-09-10T07:53:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30820.702968 secs 2023-09-10T07:53:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30821.146482 secs 2023-09-10T07:53:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:10 smithi170 ceph-mon[102832]: pgmap v16597: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 459 KiB/s wr, 29 op/s 2023-09-10T07:53:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30821.146482 secs 2023-09-10T07:53:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:10 smithi082 ceph-mon[98694]: pgmap v16597: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 459 KiB/s wr, 29 op/s 2023-09-10T07:53:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30821.146482 secs 2023-09-10T07:53:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:10 smithi098 ceph-mon[103124]: pgmap v16597: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 459 KiB/s wr, 29 op/s 2023-09-10T07:53:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:12 smithi170 ceph-mon[102832]: pgmap v16598: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 157 KiB/s wr, 29 op/s 2023-09-10T07:53:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:12 smithi170 ceph-mon[102832]: osdmap e1106: 12 total, 12 up, 12 in 2023-09-10T07:53:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:12 smithi082 ceph-mon[98694]: pgmap v16598: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 157 KiB/s wr, 29 op/s 2023-09-10T07:53:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:12 smithi082 ceph-mon[98694]: osdmap e1106: 12 total, 12 up, 12 in 2023-09-10T07:53:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:12 smithi098 ceph-mon[103124]: pgmap v16598: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 157 KiB/s wr, 29 op/s 2023-09-10T07:53:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:12 smithi098 ceph-mon[103124]: osdmap e1106: 12 total, 12 up, 12 in 2023-09-10T07:53:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30825.703027 secs 2023-09-10T07:53:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30825.703027 secs 2023-09-10T07:53:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30825.703027 secs 2023-09-10T07:53:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30824.536794 secs 2023-09-10T07:53:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:14 smithi170 ceph-mon[102832]: pgmap v16600: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:53:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30824.536794 secs 2023-09-10T07:53:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:14 smithi082 ceph-mon[98694]: pgmap v16600: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:53:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30824.536794 secs 2023-09-10T07:53:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:14 smithi098 ceph-mon[103124]: pgmap v16600: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:53:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30826.146541 secs 2023-09-10T07:53:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30826.146541 secs 2023-09-10T07:53:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30826.146541 secs 2023-09-10T07:53:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:16 smithi170 ceph-mon[102832]: pgmap v16601: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 14 op/s 2023-09-10T07:53:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:53:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:53:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:16 smithi082 ceph-mon[98694]: pgmap v16601: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 14 op/s 2023-09-10T07:53:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:53:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:53:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:16 smithi098 ceph-mon[103124]: pgmap v16601: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 14 op/s 2023-09-10T07:53:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:53:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:53:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:18 smithi170 ceph-mon[102832]: pgmap v16602: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 334 B/s wr, 12 op/s 2023-09-10T07:53:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:53:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:53:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30830.703087 secs 2023-09-10T07:53:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:18 smithi082 ceph-mon[98694]: pgmap v16602: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 334 B/s wr, 12 op/s 2023-09-10T07:53:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:53:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:53:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30830.703087 secs 2023-09-10T07:53:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:18 smithi098 ceph-mon[103124]: pgmap v16602: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 334 B/s wr, 12 op/s 2023-09-10T07:53:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:53:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:53:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30830.703087 secs 2023-09-10T07:53:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30829.536855 secs 2023-09-10T07:53:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30829.536855 secs 2023-09-10T07:53:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30829.536855 secs 2023-09-10T07:53:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30831.146602 secs 2023-09-10T07:53:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:20 smithi170 ceph-mon[102832]: pgmap v16603: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T07:53:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30831.146602 secs 2023-09-10T07:53:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:20 smithi082 ceph-mon[98694]: pgmap v16603: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T07:53:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30831.146602 secs 2023-09-10T07:53:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:20 smithi098 ceph-mon[103124]: pgmap v16603: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T07:53:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:22 smithi170 ceph-mon[102832]: pgmap v16604: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T07:53:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:22 smithi082 ceph-mon[98694]: pgmap v16604: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T07:53:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:22 smithi098 ceph-mon[103124]: pgmap v16604: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T07:53:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30835.703171 secs 2023-09-10T07:53:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30835.703171 secs 2023-09-10T07:53:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30835.703171 secs 2023-09-10T07:53:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30834.536926 secs 2023-09-10T07:53:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:24 smithi170 ceph-mon[102832]: pgmap v16605: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 182 B/s wr, 10 op/s 2023-09-10T07:53:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30834.536926 secs 2023-09-10T07:53:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:24 smithi082 ceph-mon[98694]: pgmap v16605: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 182 B/s wr, 10 op/s 2023-09-10T07:53:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30834.536926 secs 2023-09-10T07:53:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:24 smithi098 ceph-mon[103124]: pgmap v16605: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 182 B/s wr, 10 op/s 2023-09-10T07:53:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30836.146659 secs 2023-09-10T07:53:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30836.146659 secs 2023-09-10T07:53:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30836.146659 secs 2023-09-10T07:53:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:26 smithi170 ceph-mon[102832]: pgmap v16606: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 14 op/s 2023-09-10T07:53:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:26 smithi082 ceph-mon[98694]: pgmap v16606: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 14 op/s 2023-09-10T07:53:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:26 smithi098 ceph-mon[103124]: pgmap v16606: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 14 op/s 2023-09-10T07:53:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:28 smithi170 ceph-mon[102832]: pgmap v16607: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T07:53:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:28 smithi082 ceph-mon[98694]: pgmap v16607: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T07:53:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:28 smithi098 ceph-mon[103124]: pgmap v16607: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T07:53:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30839.536988 secs 2023-09-10T07:53:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30840.703230 secs 2023-09-10T07:53:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30839.536988 secs 2023-09-10T07:53:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30840.703230 secs 2023-09-10T07:53:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30839.536988 secs 2023-09-10T07:53:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30840.703230 secs 2023-09-10T07:53:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30841.146720 secs 2023-09-10T07:53:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:30 smithi170 ceph-mon[102832]: pgmap v16608: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T07:53:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30841.146720 secs 2023-09-10T07:53:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:30 smithi082 ceph-mon[98694]: pgmap v16608: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T07:53:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30841.146720 secs 2023-09-10T07:53:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:30 smithi098 ceph-mon[103124]: pgmap v16608: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T07:53:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:32 smithi170 ceph-mon[102832]: pgmap v16609: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 767 B/s wr, 14 op/s 2023-09-10T07:53:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:32 smithi082 ceph-mon[98694]: pgmap v16609: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 767 B/s wr, 14 op/s 2023-09-10T07:53:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:32 smithi098 ceph-mon[103124]: pgmap v16609: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 767 B/s wr, 14 op/s 2023-09-10T07:53:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30845.703280 secs 2023-09-10T07:53:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30845.703280 secs 2023-09-10T07:53:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30845.703280 secs 2023-09-10T07:53:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:34 smithi170 ceph-mon[102832]: pgmap v16610: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 9 op/s 2023-09-10T07:53:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:34 smithi082 ceph-mon[98694]: pgmap v16610: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 9 op/s 2023-09-10T07:53:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:34 smithi098 ceph-mon[103124]: pgmap v16610: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 9 op/s 2023-09-10T07:53:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30846.146778 secs 2023-09-10T07:53:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:53:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30846.146778 secs 2023-09-10T07:53:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:53:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30846.146778 secs 2023-09-10T07:53:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:53:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:36 smithi170 ceph-mon[102832]: pgmap v16611: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 331 KiB/s wr, 15 op/s 2023-09-10T07:53:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:53:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:53:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:53:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:36 smithi082 ceph-mon[98694]: pgmap v16611: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 331 KiB/s wr, 15 op/s 2023-09-10T07:53:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:53:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:53:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:53:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:36 smithi098 ceph-mon[103124]: pgmap v16611: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 331 KiB/s wr, 15 op/s 2023-09-10T07:53:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:53:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:53:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:53:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:38 smithi170 ceph-mon[102832]: pgmap v16612: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 331 KiB/s wr, 11 op/s 2023-09-10T07:53:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:38 smithi082 ceph-mon[98694]: pgmap v16612: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 331 KiB/s wr, 11 op/s 2023-09-10T07:53:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:38 smithi098 ceph-mon[103124]: pgmap v16612: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 331 KiB/s wr, 11 op/s 2023-09-10T07:53:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30850.703358 secs 2023-09-10T07:53:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:53:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30850.703358 secs 2023-09-10T07:53:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:53:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30850.703358 secs 2023-09-10T07:53:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:53:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30851.146839 secs 2023-09-10T07:53:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:40 smithi170 ceph-mon[102832]: pgmap v16613: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 331 KiB/s wr, 11 op/s 2023-09-10T07:53:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:53:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:53:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:53:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30851.146839 secs 2023-09-10T07:53:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:40 smithi082 ceph-mon[98694]: pgmap v16613: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 331 KiB/s wr, 11 op/s 2023-09-10T07:53:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:53:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:53:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:53:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30851.146839 secs 2023-09-10T07:53:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:40 smithi098 ceph-mon[103124]: pgmap v16613: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 331 KiB/s wr, 11 op/s 2023-09-10T07:53:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:53:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:53:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:53:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:42 smithi170 ceph-mon[102832]: pgmap v16614: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 477 KiB/s wr, 17 op/s 2023-09-10T07:53:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:42 smithi082 ceph-mon[98694]: pgmap v16614: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 477 KiB/s wr, 17 op/s 2023-09-10T07:53:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:42 smithi098 ceph-mon[103124]: pgmap v16614: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 477 KiB/s wr, 17 op/s 2023-09-10T07:53:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30855.703443 secs 2023-09-10T07:53:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30855.703443 secs 2023-09-10T07:53:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30855.703443 secs 2023-09-10T07:53:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:44 smithi170 ceph-mon[102832]: pgmap v16615: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 476 KiB/s wr, 12 op/s 2023-09-10T07:53:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:53:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:53:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:44 smithi082 ceph-mon[98694]: pgmap v16615: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 476 KiB/s wr, 12 op/s 2023-09-10T07:53:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:53:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:53:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:44 smithi098 ceph-mon[103124]: pgmap v16615: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 476 KiB/s wr, 12 op/s 2023-09-10T07:53:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:53:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:53:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30856.146898 secs 2023-09-10T07:53:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30856.146898 secs 2023-09-10T07:53:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30856.146898 secs 2023-09-10T07:53:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:46 smithi170 ceph-mon[102832]: pgmap v16616: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 772 KiB/s wr, 19 op/s 2023-09-10T07:53:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:53:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:53:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:46 smithi082 ceph-mon[98694]: pgmap v16616: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 772 KiB/s wr, 19 op/s 2023-09-10T07:53:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:53:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:53:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:46 smithi098 ceph-mon[103124]: pgmap v16616: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 772 KiB/s wr, 19 op/s 2023-09-10T07:53:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:53:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:53:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:48 smithi170 ceph-mon[102832]: pgmap v16617: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 442 KiB/s wr, 13 op/s 2023-09-10T07:53:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:48 smithi082 ceph-mon[98694]: pgmap v16617: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 442 KiB/s wr, 13 op/s 2023-09-10T07:53:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:48 smithi098 ceph-mon[103124]: pgmap v16617: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 442 KiB/s wr, 13 op/s 2023-09-10T07:53:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:49 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30859.537296 secs 2023-09-10T07:53:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30860.703494 secs 2023-09-10T07:53:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:49 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30859.537296 secs 2023-09-10T07:53:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30860.703494 secs 2023-09-10T07:53:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:49 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30859.537296 secs 2023-09-10T07:53:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30860.703494 secs 2023-09-10T07:53:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30861.146960 secs 2023-09-10T07:53:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:53:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:53:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:50 smithi170 ceph-mon[102832]: pgmap v16618: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 442 KiB/s wr, 13 op/s 2023-09-10T07:53:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30861.146960 secs 2023-09-10T07:53:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:53:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:53:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:50 smithi082 ceph-mon[98694]: pgmap v16618: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 442 KiB/s wr, 13 op/s 2023-09-10T07:53:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30861.146960 secs 2023-09-10T07:53:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:53:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:53:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:50 smithi098 ceph-mon[103124]: pgmap v16618: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 442 KiB/s wr, 13 op/s 2023-09-10T07:53:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:53:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:53:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:53:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:53:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:53:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:53:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:53:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:53:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:53:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:52 smithi170 ceph-mon[102832]: pgmap v16619: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 590 KiB/s wr, 20 op/s 2023-09-10T07:53:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:52 smithi082 ceph-mon[98694]: pgmap v16619: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 590 KiB/s wr, 20 op/s 2023-09-10T07:53:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:52 smithi098 ceph-mon[103124]: pgmap v16619: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 590 KiB/s wr, 20 op/s 2023-09-10T07:53:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30865.703551 secs 2023-09-10T07:53:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30865.703551 secs 2023-09-10T07:53:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30865.703551 secs 2023-09-10T07:53:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30864.537356 secs 2023-09-10T07:53:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:54 smithi170 ceph-mon[102832]: pgmap v16620: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 444 KiB/s wr, 14 op/s 2023-09-10T07:53:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30864.537356 secs 2023-09-10T07:53:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:54 smithi082 ceph-mon[98694]: pgmap v16620: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 444 KiB/s wr, 14 op/s 2023-09-10T07:53:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30864.537356 secs 2023-09-10T07:53:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:54 smithi098 ceph-mon[103124]: pgmap v16620: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 444 KiB/s wr, 14 op/s 2023-09-10T07:53:55.933 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30866.147019 secs 2023-09-10T07:53:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30866.147019 secs 2023-09-10T07:53:56.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30866.147019 secs 2023-09-10T07:53:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:56 smithi170 ceph-mon[102832]: pgmap v16621: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 596 KiB/s wr, 19 op/s 2023-09-10T07:53:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:56 smithi082 ceph-mon[98694]: pgmap v16621: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 596 KiB/s wr, 19 op/s 2023-09-10T07:53:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:56 smithi098 ceph-mon[103124]: pgmap v16621: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 596 KiB/s wr, 19 op/s 2023-09-10T07:53:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:58 smithi170 ceph-mon[102832]: pgmap v16622: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 300 KiB/s wr, 12 op/s 2023-09-10T07:53:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:58 smithi082 ceph-mon[98694]: pgmap v16622: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 300 KiB/s wr, 12 op/s 2023-09-10T07:53:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:58 smithi098 ceph-mon[103124]: pgmap v16622: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 300 KiB/s wr, 12 op/s 2023-09-10T07:53:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30869.537416 secs 2023-09-10T07:53:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:53:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30870.703611 secs 2023-09-10T07:53:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30869.537416 secs 2023-09-10T07:53:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:53:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30870.703611 secs 2023-09-10T07:53:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30869.537416 secs 2023-09-10T07:53:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:53:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30870.703611 secs 2023-09-10T07:54:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30871.147079 secs 2023-09-10T07:54:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:00 smithi170 ceph-mon[102832]: pgmap v16623: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 300 KiB/s wr, 12 op/s 2023-09-10T07:54:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30871.147079 secs 2023-09-10T07:54:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:00 smithi082 ceph-mon[98694]: pgmap v16623: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 300 KiB/s wr, 12 op/s 2023-09-10T07:54:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30871.147079 secs 2023-09-10T07:54:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:00 smithi098 ceph-mon[103124]: pgmap v16623: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 300 KiB/s wr, 12 op/s 2023-09-10T07:54:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:02 smithi170 ceph-mon[102832]: pgmap v16624: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 421 KiB/s wr, 17 op/s 2023-09-10T07:54:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:02 smithi082 ceph-mon[98694]: pgmap v16624: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 421 KiB/s wr, 17 op/s 2023-09-10T07:54:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:02 smithi098 ceph-mon[103124]: pgmap v16624: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 421 KiB/s wr, 17 op/s 2023-09-10T07:54:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30875.703668 secs 2023-09-10T07:54:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30875.703668 secs 2023-09-10T07:54:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30875.703668 secs 2023-09-10T07:54:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30874.537476 secs 2023-09-10T07:54:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:04 smithi170 ceph-mon[102832]: pgmap v16625: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 273 KiB/s wr, 10 op/s 2023-09-10T07:54:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:54:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:54:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30874.537476 secs 2023-09-10T07:54:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:04 smithi082 ceph-mon[98694]: pgmap v16625: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 273 KiB/s wr, 10 op/s 2023-09-10T07:54:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:54:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:54:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30874.537476 secs 2023-09-10T07:54:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:04 smithi098 ceph-mon[103124]: pgmap v16625: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 273 KiB/s wr, 10 op/s 2023-09-10T07:54:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:54:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:54:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30876.147137 secs 2023-09-10T07:54:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30876.147137 secs 2023-09-10T07:54:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30876.147137 secs 2023-09-10T07:54:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:06 smithi170 ceph-mon[102832]: pgmap v16626: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 318 KiB/s wr, 29 op/s 2023-09-10T07:54:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:54:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:54:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:06 smithi082 ceph-mon[98694]: pgmap v16626: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 318 KiB/s wr, 29 op/s 2023-09-10T07:54:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:54:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:54:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:06 smithi098 ceph-mon[103124]: pgmap v16626: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 318 KiB/s wr, 29 op/s 2023-09-10T07:54:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:54:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:54:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:08 smithi170 ceph-mon[102832]: pgmap v16627: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 166 KiB/s wr, 24 op/s 2023-09-10T07:54:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:08 smithi082 ceph-mon[98694]: pgmap v16627: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 166 KiB/s wr, 24 op/s 2023-09-10T07:54:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:08 smithi098 ceph-mon[103124]: pgmap v16627: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 166 KiB/s wr, 24 op/s 2023-09-10T07:54:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30879.537576 secs 2023-09-10T07:54:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:09 smithi170 ceph-mon[102832]: osdmap e1107: 12 total, 12 up, 12 in 2023-09-10T07:54:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30880.703727 secs 2023-09-10T07:54:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30879.537576 secs 2023-09-10T07:54:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:09 smithi082 ceph-mon[98694]: osdmap e1107: 12 total, 12 up, 12 in 2023-09-10T07:54:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30880.703727 secs 2023-09-10T07:54:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30879.537576 secs 2023-09-10T07:54:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:09 smithi098 ceph-mon[103124]: osdmap e1107: 12 total, 12 up, 12 in 2023-09-10T07:54:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30880.703727 secs 2023-09-10T07:54:10.309 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30881.147199 secs 2023-09-10T07:54:10.309 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:10 smithi082 ceph-mon[98694]: pgmap v16629: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 199 KiB/s wr, 29 op/s 2023-09-10T07:54:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30881.147199 secs 2023-09-10T07:54:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:10 smithi170 ceph-mon[102832]: pgmap v16629: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 199 KiB/s wr, 29 op/s 2023-09-10T07:54:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30881.147199 secs 2023-09-10T07:54:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:10 smithi098 ceph-mon[103124]: pgmap v16629: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 199 KiB/s wr, 29 op/s 2023-09-10T07:54:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:12 smithi170 ceph-mon[102832]: pgmap v16630: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:54:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:12 smithi170 ceph-mon[102832]: osdmap e1108: 12 total, 12 up, 12 in 2023-09-10T07:54:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:12 smithi082 ceph-mon[98694]: pgmap v16630: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:54:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:12 smithi082 ceph-mon[98694]: osdmap e1108: 12 total, 12 up, 12 in 2023-09-10T07:54:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:12 smithi098 ceph-mon[103124]: pgmap v16630: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T07:54:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:12 smithi098 ceph-mon[103124]: osdmap e1108: 12 total, 12 up, 12 in 2023-09-10T07:54:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30885.703786 secs 2023-09-10T07:54:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30885.703786 secs 2023-09-10T07:54:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30885.703786 secs 2023-09-10T07:54:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30884.537637 secs 2023-09-10T07:54:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:14 smithi170 ceph-mon[102832]: pgmap v16632: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:54:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30884.537637 secs 2023-09-10T07:54:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:14 smithi082 ceph-mon[98694]: pgmap v16632: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:54:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30884.537637 secs 2023-09-10T07:54:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:14 smithi098 ceph-mon[103124]: pgmap v16632: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:54:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30886.147257 secs 2023-09-10T07:54:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30886.147257 secs 2023-09-10T07:54:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30886.147257 secs 2023-09-10T07:54:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:16 smithi170 ceph-mon[102832]: pgmap v16633: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T07:54:16.941 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:16 smithi082 ceph-mon[98694]: pgmap v16633: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T07:54:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:16 smithi098 ceph-mon[103124]: pgmap v16633: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T07:54:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:18 smithi170 ceph-mon[102832]: pgmap v16634: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 11 op/s 2023-09-10T07:54:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:18 smithi082 ceph-mon[98694]: pgmap v16634: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 11 op/s 2023-09-10T07:54:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:18 smithi098 ceph-mon[103124]: pgmap v16634: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 11 op/s 2023-09-10T07:54:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30889.537696 secs 2023-09-10T07:54:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30890.703846 secs 2023-09-10T07:54:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30889.537696 secs 2023-09-10T07:54:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30890.703846 secs 2023-09-10T07:54:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30889.537696 secs 2023-09-10T07:54:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30890.703846 secs 2023-09-10T07:54:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30891.147320 secs 2023-09-10T07:54:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:20 smithi170 ceph-mon[102832]: pgmap v16635: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:54:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30891.147320 secs 2023-09-10T07:54:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:20 smithi082 ceph-mon[98694]: pgmap v16635: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:54:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30891.147320 secs 2023-09-10T07:54:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:20 smithi098 ceph-mon[103124]: pgmap v16635: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:54:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:22 smithi170 ceph-mon[102832]: pgmap v16636: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:54:22.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:22 smithi082 ceph-mon[98694]: pgmap v16636: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:54:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:22 smithi098 ceph-mon[103124]: pgmap v16636: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:54:23.880 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30895.703902 secs 2023-09-10T07:54:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30895.703902 secs 2023-09-10T07:54:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30895.703902 secs 2023-09-10T07:54:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30894.537756 secs 2023-09-10T07:54:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:24 smithi170 ceph-mon[102832]: pgmap v16637: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 182 B/s wr, 9 op/s 2023-09-10T07:54:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30894.537756 secs 2023-09-10T07:54:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:24 smithi082 ceph-mon[98694]: pgmap v16637: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 182 B/s wr, 9 op/s 2023-09-10T07:54:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30894.537756 secs 2023-09-10T07:54:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:24 smithi098 ceph-mon[103124]: pgmap v16637: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 182 B/s wr, 9 op/s 2023-09-10T07:54:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30896.147376 secs 2023-09-10T07:54:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30896.147376 secs 2023-09-10T07:54:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30896.147376 secs 2023-09-10T07:54:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:26 smithi170 ceph-mon[102832]: pgmap v16638: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 13 op/s 2023-09-10T07:54:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:26 smithi082 ceph-mon[98694]: pgmap v16638: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 13 op/s 2023-09-10T07:54:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:26 smithi098 ceph-mon[103124]: pgmap v16638: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 13 op/s 2023-09-10T07:54:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:28 smithi170 ceph-mon[102832]: pgmap v16639: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 9 op/s 2023-09-10T07:54:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:28 smithi082 ceph-mon[98694]: pgmap v16639: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 9 op/s 2023-09-10T07:54:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:28 smithi098 ceph-mon[103124]: pgmap v16639: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 9 op/s 2023-09-10T07:54:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30899.537776 secs 2023-09-10T07:54:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30900.703982 secs 2023-09-10T07:54:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30899.537776 secs 2023-09-10T07:54:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30900.703982 secs 2023-09-10T07:54:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30899.537776 secs 2023-09-10T07:54:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30900.703982 secs 2023-09-10T07:54:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30901.147441 secs 2023-09-10T07:54:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:30 smithi170 ceph-mon[102832]: pgmap v16640: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 9 op/s 2023-09-10T07:54:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30901.147441 secs 2023-09-10T07:54:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:30 smithi082 ceph-mon[98694]: pgmap v16640: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 9 op/s 2023-09-10T07:54:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30901.147441 secs 2023-09-10T07:54:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:30 smithi098 ceph-mon[103124]: pgmap v16640: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 9 op/s 2023-09-10T07:54:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:32 smithi170 ceph-mon[102832]: pgmap v16641: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 258 KiB/s wr, 14 op/s 2023-09-10T07:54:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:32 smithi082 ceph-mon[98694]: pgmap v16641: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 258 KiB/s wr, 14 op/s 2023-09-10T07:54:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:32 smithi098 ceph-mon[103124]: pgmap v16641: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 258 KiB/s wr, 14 op/s 2023-09-10T07:54:33.887 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30905.704034 secs 2023-09-10T07:54:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30905.704034 secs 2023-09-10T07:54:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30905.704034 secs 2023-09-10T07:54:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:34 smithi170 ceph-mon[102832]: pgmap v16642: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 258 KiB/s wr, 9 op/s 2023-09-10T07:54:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30906.147480 secs 2023-09-10T07:54:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:34 smithi082 ceph-mon[98694]: pgmap v16642: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 258 KiB/s wr, 9 op/s 2023-09-10T07:54:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30906.147480 secs 2023-09-10T07:54:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:34 smithi098 ceph-mon[103124]: pgmap v16642: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 258 KiB/s wr, 9 op/s 2023-09-10T07:54:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30906.147480 secs 2023-09-10T07:54:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:54:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:54:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:54:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:36 smithi170 ceph-mon[102832]: pgmap v16643: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 320 KiB/s wr, 15 op/s 2023-09-10T07:54:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:54:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:54:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:54:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:36 smithi082 ceph-mon[98694]: pgmap v16643: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 320 KiB/s wr, 15 op/s 2023-09-10T07:54:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:54:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:54:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:54:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:36 smithi098 ceph-mon[103124]: pgmap v16643: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 320 KiB/s wr, 15 op/s 2023-09-10T07:54:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:54:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:54:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:54:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:38 smithi170 ceph-mon[102832]: pgmap v16644: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 216 KiB/s wr, 9 op/s 2023-09-10T07:54:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:38 smithi082 ceph-mon[98694]: pgmap v16644: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 216 KiB/s wr, 9 op/s 2023-09-10T07:54:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:38 smithi098 ceph-mon[103124]: pgmap v16644: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 216 KiB/s wr, 9 op/s 2023-09-10T07:54:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30910.704115 secs 2023-09-10T07:54:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30911.147544 secs 2023-09-10T07:54:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30910.704115 secs 2023-09-10T07:54:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30911.147544 secs 2023-09-10T07:54:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30910.704115 secs 2023-09-10T07:54:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30911.147544 secs 2023-09-10T07:54:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:54:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:54:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:40 smithi170 ceph-mon[102832]: pgmap v16645: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 216 KiB/s wr, 9 op/s 2023-09-10T07:54:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:54:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:54:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:54:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:40 smithi082 ceph-mon[98694]: pgmap v16645: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 216 KiB/s wr, 9 op/s 2023-09-10T07:54:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:54:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:54:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:54:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:40 smithi098 ceph-mon[103124]: pgmap v16645: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 216 KiB/s wr, 9 op/s 2023-09-10T07:54:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:54:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:54:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:54:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:54:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:54:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:54:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:54:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:54:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:54:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:54:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:42 smithi170 ceph-mon[102832]: pgmap v16646: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 554 KiB/s wr, 15 op/s 2023-09-10T07:54:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:42 smithi082 ceph-mon[98694]: pgmap v16646: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 554 KiB/s wr, 15 op/s 2023-09-10T07:54:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:42 smithi098 ceph-mon[103124]: pgmap v16646: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 554 KiB/s wr, 15 op/s 2023-09-10T07:54:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:54:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:54:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30915.704172 secs 2023-09-10T07:54:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:44 smithi170 ceph-mon[102832]: pgmap v16647: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 400 KiB/s wr, 11 op/s 2023-09-10T07:54:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30916.147601 secs 2023-09-10T07:54:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:54:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:54:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30915.704172 secs 2023-09-10T07:54:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:44 smithi082 ceph-mon[98694]: pgmap v16647: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 400 KiB/s wr, 11 op/s 2023-09-10T07:54:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30916.147601 secs 2023-09-10T07:54:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:54:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:54:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30915.704172 secs 2023-09-10T07:54:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:44 smithi098 ceph-mon[103124]: pgmap v16647: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 400 KiB/s wr, 11 op/s 2023-09-10T07:54:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30916.147601 secs 2023-09-10T07:54:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:54:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:54:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:46 smithi170 ceph-mon[102832]: pgmap v16648: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 560 KiB/s wr, 18 op/s 2023-09-10T07:54:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:54:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:54:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:46 smithi082 ceph-mon[98694]: pgmap v16648: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 560 KiB/s wr, 18 op/s 2023-09-10T07:54:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:54:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:54:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:46 smithi098 ceph-mon[103124]: pgmap v16648: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 560 KiB/s wr, 18 op/s 2023-09-10T07:54:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:48 smithi170 ceph-mon[102832]: pgmap v16649: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 498 KiB/s wr, 12 op/s 2023-09-10T07:54:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:48 smithi082 ceph-mon[98694]: pgmap v16649: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 498 KiB/s wr, 12 op/s 2023-09-10T07:54:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:48 smithi098 ceph-mon[103124]: pgmap v16649: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 498 KiB/s wr, 12 op/s 2023-09-10T07:54:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:49 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30919.538030 secs 2023-09-10T07:54:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30920.704231 secs 2023-09-10T07:54:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30921.147669 secs 2023-09-10T07:54:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:49 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30919.538030 secs 2023-09-10T07:54:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30920.704231 secs 2023-09-10T07:54:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30921.147669 secs 2023-09-10T07:54:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:49 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30919.538030 secs 2023-09-10T07:54:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30920.704231 secs 2023-09-10T07:54:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30921.147669 secs 2023-09-10T07:54:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:50 smithi170 ceph-mon[102832]: pgmap v16650: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 498 KiB/s wr, 12 op/s 2023-09-10T07:54:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:50 smithi082 ceph-mon[98694]: pgmap v16650: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 498 KiB/s wr, 12 op/s 2023-09-10T07:54:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:50 smithi098 ceph-mon[103124]: pgmap v16650: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 498 KiB/s wr, 12 op/s 2023-09-10T07:54:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:54:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:54:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:54:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:54:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:54:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:54:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:54:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:54:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:54:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:52 smithi170 ceph-mon[102832]: pgmap v16651: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 782 KiB/s wr, 20 op/s 2023-09-10T07:54:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:52 smithi082 ceph-mon[98694]: pgmap v16651: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 782 KiB/s wr, 20 op/s 2023-09-10T07:54:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:52 smithi098 ceph-mon[103124]: pgmap v16651: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 782 KiB/s wr, 20 op/s 2023-09-10T07:54:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30924.538090 secs 2023-09-10T07:54:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:54:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:54:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30925.704312 secs 2023-09-10T07:54:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:54 smithi170 ceph-mon[102832]: pgmap v16652: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 444 KiB/s wr, 14 op/s 2023-09-10T07:54:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30926.147725 secs 2023-09-10T07:54:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30924.538090 secs 2023-09-10T07:54:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:54:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:54:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30925.704312 secs 2023-09-10T07:54:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:54 smithi082 ceph-mon[98694]: pgmap v16652: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 444 KiB/s wr, 14 op/s 2023-09-10T07:54:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30926.147725 secs 2023-09-10T07:54:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30924.538090 secs 2023-09-10T07:54:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:54:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:54:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30925.704312 secs 2023-09-10T07:54:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:54 smithi098 ceph-mon[103124]: pgmap v16652: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 444 KiB/s wr, 14 op/s 2023-09-10T07:54:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30926.147725 secs 2023-09-10T07:54:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:54:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:54:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:56 smithi170 ceph-mon[102832]: pgmap v16653: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 630 KiB/s wr, 21 op/s 2023-09-10T07:54:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:54:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:54:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:56 smithi082 ceph-mon[98694]: pgmap v16653: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 630 KiB/s wr, 21 op/s 2023-09-10T07:54:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:54:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:54:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:56 smithi098 ceph-mon[103124]: pgmap v16653: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 630 KiB/s wr, 21 op/s 2023-09-10T07:54:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:58 smithi170 ceph-mon[102832]: pgmap v16654: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 470 KiB/s wr, 13 op/s 2023-09-10T07:54:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:58 smithi082 ceph-mon[98694]: pgmap v16654: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 470 KiB/s wr, 13 op/s 2023-09-10T07:54:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:58 smithi098 ceph-mon[103124]: pgmap v16654: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 470 KiB/s wr, 13 op/s 2023-09-10T07:54:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30929.538149 secs 2023-09-10T07:54:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30930.704369 secs 2023-09-10T07:54:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:54:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30931.147791 secs 2023-09-10T07:54:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30929.538149 secs 2023-09-10T07:54:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30930.704369 secs 2023-09-10T07:54:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:54:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30931.147791 secs 2023-09-10T07:54:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30929.538149 secs 2023-09-10T07:54:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30930.704369 secs 2023-09-10T07:54:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:54:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30931.147791 secs 2023-09-10T07:55:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:00 smithi170 ceph-mon[102832]: pgmap v16655: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 470 KiB/s wr, 13 op/s 2023-09-10T07:55:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:00 smithi082 ceph-mon[98694]: pgmap v16655: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 470 KiB/s wr, 13 op/s 2023-09-10T07:55:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:00 smithi098 ceph-mon[103124]: pgmap v16655: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 470 KiB/s wr, 13 op/s 2023-09-10T07:55:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:02 smithi170 ceph-mon[102832]: pgmap v16656: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 723 KiB/s wr, 20 op/s 2023-09-10T07:55:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:02 smithi082 ceph-mon[98694]: pgmap v16656: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 723 KiB/s wr, 20 op/s 2023-09-10T07:55:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:02 smithi098 ceph-mon[103124]: pgmap v16656: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 723 KiB/s wr, 20 op/s 2023-09-10T07:55:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30935.704425 secs 2023-09-10T07:55:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30935.704425 secs 2023-09-10T07:55:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30935.704425 secs 2023-09-10T07:55:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30934.538210 secs 2023-09-10T07:55:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:04 smithi170 ceph-mon[102832]: pgmap v16657: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 440 KiB/s wr, 12 op/s 2023-09-10T07:55:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:55:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:55:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30936.147873 secs 2023-09-10T07:55:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30934.538210 secs 2023-09-10T07:55:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:04 smithi082 ceph-mon[98694]: pgmap v16657: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 440 KiB/s wr, 12 op/s 2023-09-10T07:55:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:55:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:55:04.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30936.147873 secs 2023-09-10T07:55:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30934.538210 secs 2023-09-10T07:55:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:04 smithi098 ceph-mon[103124]: pgmap v16657: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 440 KiB/s wr, 12 op/s 2023-09-10T07:55:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:55:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:55:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30936.147873 secs 2023-09-10T07:55:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:06 smithi170 ceph-mon[102832]: pgmap v16658: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 570 KiB/s wr, 32 op/s 2023-09-10T07:55:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:55:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:55:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:06 smithi082 ceph-mon[98694]: pgmap v16658: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 570 KiB/s wr, 32 op/s 2023-09-10T07:55:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:55:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:55:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:06 smithi098 ceph-mon[103124]: pgmap v16658: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 570 KiB/s wr, 32 op/s 2023-09-10T07:55:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:55:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:55:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:08 smithi170 ceph-mon[102832]: pgmap v16659: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 384 KiB/s wr, 25 op/s 2023-09-10T07:55:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:08 smithi082 ceph-mon[98694]: pgmap v16659: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 384 KiB/s wr, 25 op/s 2023-09-10T07:55:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:08 smithi098 ceph-mon[103124]: pgmap v16659: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 384 KiB/s wr, 25 op/s 2023-09-10T07:55:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30939.538271 secs 2023-09-10T07:55:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:09 smithi170 ceph-mon[102832]: osdmap e1109: 12 total, 12 up, 12 in 2023-09-10T07:55:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30940.704492 secs 2023-09-10T07:55:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30941.147965 secs 2023-09-10T07:55:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30939.538271 secs 2023-09-10T07:55:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:09 smithi082 ceph-mon[98694]: osdmap e1109: 12 total, 12 up, 12 in 2023-09-10T07:55:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30940.704492 secs 2023-09-10T07:55:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30941.147965 secs 2023-09-10T07:55:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30939.538271 secs 2023-09-10T07:55:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:09 smithi098 ceph-mon[103124]: osdmap e1109: 12 total, 12 up, 12 in 2023-09-10T07:55:09.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30940.704492 secs 2023-09-10T07:55:09.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30941.147965 secs 2023-09-10T07:55:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:10 smithi170 ceph-mon[102832]: pgmap v16661: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 461 KiB/s wr, 31 op/s 2023-09-10T07:55:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:10 smithi082 ceph-mon[98694]: pgmap v16661: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 461 KiB/s wr, 31 op/s 2023-09-10T07:55:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:10 smithi098 ceph-mon[103124]: pgmap v16661: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 461 KiB/s wr, 31 op/s 2023-09-10T07:55:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:12 smithi170 ceph-mon[102832]: pgmap v16662: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 157 KiB/s wr, 29 op/s 2023-09-10T07:55:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:12 smithi170 ceph-mon[102832]: osdmap e1110: 12 total, 12 up, 12 in 2023-09-10T07:55:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:12 smithi082 ceph-mon[98694]: pgmap v16662: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 157 KiB/s wr, 29 op/s 2023-09-10T07:55:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:12 smithi082 ceph-mon[98694]: osdmap e1110: 12 total, 12 up, 12 in 2023-09-10T07:55:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:12 smithi098 ceph-mon[103124]: pgmap v16662: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 157 KiB/s wr, 29 op/s 2023-09-10T07:55:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:12 smithi098 ceph-mon[103124]: osdmap e1110: 12 total, 12 up, 12 in 2023-09-10T07:55:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30945.704552 secs 2023-09-10T07:55:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30945.704552 secs 2023-09-10T07:55:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30945.704552 secs 2023-09-10T07:55:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30944.538382 secs 2023-09-10T07:55:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:14 smithi170 ceph-mon[102832]: pgmap v16664: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:55:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30946.148025 secs 2023-09-10T07:55:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30944.538382 secs 2023-09-10T07:55:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:14 smithi082 ceph-mon[98694]: pgmap v16664: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:55:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30946.148025 secs 2023-09-10T07:55:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30944.538382 secs 2023-09-10T07:55:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:14 smithi098 ceph-mon[103124]: pgmap v16664: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T07:55:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30946.148025 secs 2023-09-10T07:55:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:16 smithi170 ceph-mon[102832]: pgmap v16665: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 14 op/s 2023-09-10T07:55:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:16 smithi082 ceph-mon[98694]: pgmap v16665: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 14 op/s 2023-09-10T07:55:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:16 smithi098 ceph-mon[103124]: pgmap v16665: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 14 op/s 2023-09-10T07:55:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:18 smithi170 ceph-mon[102832]: pgmap v16666: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 334 B/s wr, 12 op/s 2023-09-10T07:55:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:18 smithi082 ceph-mon[98694]: pgmap v16666: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 334 B/s wr, 12 op/s 2023-09-10T07:55:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:18 smithi098 ceph-mon[103124]: pgmap v16666: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 334 B/s wr, 12 op/s 2023-09-10T07:55:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30949.538442 secs 2023-09-10T07:55:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30950.704611 secs 2023-09-10T07:55:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30951.148090 secs 2023-09-10T07:55:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30949.538442 secs 2023-09-10T07:55:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30950.704611 secs 2023-09-10T07:55:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30951.148090 secs 2023-09-10T07:55:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30949.538442 secs 2023-09-10T07:55:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30950.704611 secs 2023-09-10T07:55:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30951.148090 secs 2023-09-10T07:55:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:20 smithi170 ceph-mon[102832]: pgmap v16667: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T07:55:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:20 smithi082 ceph-mon[98694]: pgmap v16667: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T07:55:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:20 smithi098 ceph-mon[103124]: pgmap v16667: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T07:55:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:22 smithi170 ceph-mon[102832]: pgmap v16668: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:55:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:22 smithi082 ceph-mon[98694]: pgmap v16668: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:55:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:22 smithi098 ceph-mon[103124]: pgmap v16668: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:55:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30955.704670 secs 2023-09-10T07:55:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30955.704670 secs 2023-09-10T07:55:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30955.704670 secs 2023-09-10T07:55:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30954.538499 secs 2023-09-10T07:55:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:24 smithi170 ceph-mon[102832]: pgmap v16669: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 182 B/s wr, 9 op/s 2023-09-10T07:55:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30956.148148 secs 2023-09-10T07:55:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30954.538499 secs 2023-09-10T07:55:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:24 smithi082 ceph-mon[98694]: pgmap v16669: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 182 B/s wr, 9 op/s 2023-09-10T07:55:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30956.148148 secs 2023-09-10T07:55:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30954.538499 secs 2023-09-10T07:55:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:24 smithi098 ceph-mon[103124]: pgmap v16669: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 182 B/s wr, 9 op/s 2023-09-10T07:55:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30956.148148 secs 2023-09-10T07:55:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:26 smithi170 ceph-mon[102832]: pgmap v16670: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 597 B/s wr, 13 op/s 2023-09-10T07:55:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:26 smithi082 ceph-mon[98694]: pgmap v16670: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 597 B/s wr, 13 op/s 2023-09-10T07:55:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:26 smithi098 ceph-mon[103124]: pgmap v16670: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 597 B/s wr, 13 op/s 2023-09-10T07:55:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:28 smithi170 ceph-mon[102832]: pgmap v16671: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:55:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:28 smithi082 ceph-mon[98694]: pgmap v16671: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:55:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:28 smithi098 ceph-mon[103124]: pgmap v16671: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:55:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30959.538570 secs 2023-09-10T07:55:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30960.704730 secs 2023-09-10T07:55:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30961.148214 secs 2023-09-10T07:55:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30959.538570 secs 2023-09-10T07:55:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30960.704730 secs 2023-09-10T07:55:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30961.148214 secs 2023-09-10T07:55:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30959.538570 secs 2023-09-10T07:55:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30960.704730 secs 2023-09-10T07:55:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30961.148214 secs 2023-09-10T07:55:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:30 smithi170 ceph-mon[102832]: pgmap v16672: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:55:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:30 smithi082 ceph-mon[98694]: pgmap v16672: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:55:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:30 smithi098 ceph-mon[103124]: pgmap v16672: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2023-09-10T07:55:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:32 smithi170 ceph-mon[102832]: pgmap v16673: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 116 KiB/s wr, 14 op/s 2023-09-10T07:55:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:32 smithi082 ceph-mon[98694]: pgmap v16673: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 116 KiB/s wr, 14 op/s 2023-09-10T07:55:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:32 smithi098 ceph-mon[103124]: pgmap v16673: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 116 KiB/s wr, 14 op/s 2023-09-10T07:55:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30965.704871 secs 2023-09-10T07:55:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30965.704871 secs 2023-09-10T07:55:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30965.704871 secs 2023-09-10T07:55:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:34 smithi170 ceph-mon[102832]: pgmap v16674: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 9 op/s 2023-09-10T07:55:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30966.148269 secs 2023-09-10T07:55:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:34 smithi082 ceph-mon[98694]: pgmap v16674: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 9 op/s 2023-09-10T07:55:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30966.148269 secs 2023-09-10T07:55:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:34 smithi098 ceph-mon[103124]: pgmap v16674: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 9 op/s 2023-09-10T07:55:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30966.148269 secs 2023-09-10T07:55:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:55:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:55:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:55:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:36 smithi170 ceph-mon[102832]: pgmap v16675: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 218 KiB/s wr, 15 op/s 2023-09-10T07:55:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:55:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:55:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:55:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:36 smithi082 ceph-mon[98694]: pgmap v16675: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 218 KiB/s wr, 15 op/s 2023-09-10T07:55:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:55:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:55:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:55:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:36 smithi098 ceph-mon[103124]: pgmap v16675: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 218 KiB/s wr, 15 op/s 2023-09-10T07:55:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:55:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:55:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:55:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:38 smithi170 ceph-mon[102832]: pgmap v16676: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 217 KiB/s wr, 10 op/s 2023-09-10T07:55:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:38 smithi082 ceph-mon[98694]: pgmap v16676: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 217 KiB/s wr, 10 op/s 2023-09-10T07:55:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:38 smithi098 ceph-mon[103124]: pgmap v16676: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 217 KiB/s wr, 10 op/s 2023-09-10T07:55:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30970.704952 secs 2023-09-10T07:55:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30971.148335 secs 2023-09-10T07:55:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30970.704952 secs 2023-09-10T07:55:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30971.148335 secs 2023-09-10T07:55:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30970.704952 secs 2023-09-10T07:55:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30971.148335 secs 2023-09-10T07:55:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:55:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:55:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:40 smithi170 ceph-mon[102832]: pgmap v16677: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 217 KiB/s wr, 10 op/s 2023-09-10T07:55:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:55:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:55:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:40 smithi082 ceph-mon[98694]: pgmap v16677: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 217 KiB/s wr, 10 op/s 2023-09-10T07:55:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:55:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:55:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:40 smithi098 ceph-mon[103124]: pgmap v16677: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 217 KiB/s wr, 10 op/s 2023-09-10T07:55:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:55:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:55:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:55:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:42 smithi170 ceph-mon[102832]: pgmap v16678: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 556 KiB/s wr, 16 op/s 2023-09-10T07:55:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:42 smithi082 ceph-mon[98694]: pgmap v16678: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 556 KiB/s wr, 16 op/s 2023-09-10T07:55:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:42 smithi098 ceph-mon[103124]: pgmap v16678: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 556 KiB/s wr, 16 op/s 2023-09-10T07:55:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:55:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:55:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30975.705011 secs 2023-09-10T07:55:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:44 smithi170 ceph-mon[102832]: pgmap v16679: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 441 KiB/s wr, 11 op/s 2023-09-10T07:55:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:44.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:44.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:44.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:44.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30976.148391 secs 2023-09-10T07:55:44.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:44.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:44.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:55:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:55:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:55:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30975.705011 secs 2023-09-10T07:55:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:44 smithi082 ceph-mon[98694]: pgmap v16679: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 441 KiB/s wr, 11 op/s 2023-09-10T07:55:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:44.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:44.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30976.148391 secs 2023-09-10T07:55:44.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:44.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:44.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:55:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:55:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:55:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30975.705011 secs 2023-09-10T07:55:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:44 smithi098 ceph-mon[103124]: pgmap v16679: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 441 KiB/s wr, 11 op/s 2023-09-10T07:55:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:44.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:44.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:44.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30976.148391 secs 2023-09-10T07:55:44.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:44.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:44.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:55:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:55:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:55:45.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:55:45.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:55:45.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:45 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T07:55:45.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:45 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T07:55:45.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:45.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:45.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:55:45.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:55:45.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:55:45.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:55:45.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:45 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T07:55:45.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:45 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T07:55:45.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:55:45.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:55:45.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:55:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:55:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:55:45.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:55:45.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:45 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T07:55:45.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:45 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T07:55:45.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:45.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:45.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:55:45.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:55:45.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:55:45.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:55:45.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:45 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T07:55:45.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:45 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T07:55:45.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:55:45.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:55:45.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:55:45.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:55:45.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:55:45.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:55:45.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:45 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T07:55:45.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:45 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T07:55:45.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:45.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:45.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:55:45.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:55:45.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:55:45.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T07:55:45.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:45 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T07:55:45.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:45 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T07:55:45.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:55:45.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:55:45.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:55:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:46 smithi170 ceph-mon[102832]: pgmap v16680: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 601 KiB/s wr, 18 op/s 2023-09-10T07:55:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:55:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:55:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:46 smithi082 ceph-mon[98694]: pgmap v16680: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 601 KiB/s wr, 18 op/s 2023-09-10T07:55:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:55:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:55:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:46 smithi098 ceph-mon[103124]: pgmap v16680: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 601 KiB/s wr, 18 op/s 2023-09-10T07:55:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:55:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:55:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:48 smithi170 ceph-mon[102832]: pgmap v16681: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 499 KiB/s wr, 12 op/s 2023-09-10T07:55:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:48 smithi082 ceph-mon[98694]: pgmap v16681: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 499 KiB/s wr, 12 op/s 2023-09-10T07:55:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:48 smithi098 ceph-mon[103124]: pgmap v16681: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 499 KiB/s wr, 12 op/s 2023-09-10T07:55:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:49 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30979.538839 secs 2023-09-10T07:55:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30980.705072 secs 2023-09-10T07:55:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30981.148457 secs 2023-09-10T07:55:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:49 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30979.538839 secs 2023-09-10T07:55:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30980.705072 secs 2023-09-10T07:55:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30981.148457 secs 2023-09-10T07:55:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:49 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30979.538839 secs 2023-09-10T07:55:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30980.705072 secs 2023-09-10T07:55:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30981.148457 secs 2023-09-10T07:55:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:50 smithi170 ceph-mon[102832]: pgmap v16682: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 499 KiB/s wr, 12 op/s 2023-09-10T07:55:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:50 smithi082 ceph-mon[98694]: pgmap v16682: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 499 KiB/s wr, 12 op/s 2023-09-10T07:55:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:50 smithi098 ceph-mon[103124]: pgmap v16682: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 499 KiB/s wr, 12 op/s 2023-09-10T07:55:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:55:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:55:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:55:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:55:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:55:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:55:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:55:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:55:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:55:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:52 smithi170 ceph-mon[102832]: pgmap v16683: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 783 KiB/s wr, 20 op/s 2023-09-10T07:55:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:52 smithi082 ceph-mon[98694]: pgmap v16683: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 783 KiB/s wr, 20 op/s 2023-09-10T07:55:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:52 smithi098 ceph-mon[103124]: pgmap v16683: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 783 KiB/s wr, 20 op/s 2023-09-10T07:55:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30984.538899 secs 2023-09-10T07:55:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:55:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:55:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30985.705169 secs 2023-09-10T07:55:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:54 smithi170 ceph-mon[102832]: pgmap v16684: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 445 KiB/s wr, 14 op/s 2023-09-10T07:55:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30986.148482 secs 2023-09-10T07:55:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30984.538899 secs 2023-09-10T07:55:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:55:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:55:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30985.705169 secs 2023-09-10T07:55:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:54 smithi082 ceph-mon[98694]: pgmap v16684: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 445 KiB/s wr, 14 op/s 2023-09-10T07:55:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30986.148482 secs 2023-09-10T07:55:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30984.538899 secs 2023-09-10T07:55:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:55:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:55:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30985.705169 secs 2023-09-10T07:55:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:54 smithi098 ceph-mon[103124]: pgmap v16684: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 445 KiB/s wr, 14 op/s 2023-09-10T07:55:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30986.148482 secs 2023-09-10T07:55:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:56 smithi170 ceph-mon[102832]: pgmap v16685: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 528 KiB/s wr, 20 op/s 2023-09-10T07:55:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:56 smithi082 ceph-mon[98694]: pgmap v16685: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 528 KiB/s wr, 20 op/s 2023-09-10T07:55:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:56 smithi098 ceph-mon[103124]: pgmap v16685: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 528 KiB/s wr, 20 op/s 2023-09-10T07:55:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:58 smithi170 ceph-mon[102832]: pgmap v16686: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 368 KiB/s wr, 13 op/s 2023-09-10T07:55:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:58 smithi082 ceph-mon[98694]: pgmap v16686: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 368 KiB/s wr, 13 op/s 2023-09-10T07:55:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:58 smithi098 ceph-mon[103124]: pgmap v16686: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 368 KiB/s wr, 13 op/s 2023-09-10T07:55:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30989.539005 secs 2023-09-10T07:55:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30990.705247 secs 2023-09-10T07:55:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:55:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30991.148549 secs 2023-09-10T07:55:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30989.539005 secs 2023-09-10T07:55:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30990.705247 secs 2023-09-10T07:55:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:55:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30991.148549 secs 2023-09-10T07:55:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30989.539005 secs 2023-09-10T07:55:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30990.705247 secs 2023-09-10T07:55:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:55:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30991.148549 secs 2023-09-10T07:56:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:00 smithi170 ceph-mon[102832]: pgmap v16687: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 368 KiB/s wr, 13 op/s 2023-09-10T07:56:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:00 smithi082 ceph-mon[98694]: pgmap v16687: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 368 KiB/s wr, 13 op/s 2023-09-10T07:56:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:00 smithi098 ceph-mon[103124]: pgmap v16687: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 368 KiB/s wr, 13 op/s 2023-09-10T07:56:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:02 smithi170 ceph-mon[102832]: pgmap v16688: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 621 KiB/s wr, 19 op/s 2023-09-10T07:56:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:02 smithi082 ceph-mon[98694]: pgmap v16688: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 621 KiB/s wr, 19 op/s 2023-09-10T07:56:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:02 smithi098 ceph-mon[103124]: pgmap v16688: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 621 KiB/s wr, 19 op/s 2023-09-10T07:56:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30995.705332 secs 2023-09-10T07:56:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30995.705332 secs 2023-09-10T07:56:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30995.705332 secs 2023-09-10T07:56:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30994.539109 secs 2023-09-10T07:56:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:04 smithi170 ceph-mon[102832]: pgmap v16689: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 337 KiB/s wr, 11 op/s 2023-09-10T07:56:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:56:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:56:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 30996.148610 secs 2023-09-10T07:56:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30994.539109 secs 2023-09-10T07:56:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:04 smithi082 ceph-mon[98694]: pgmap v16689: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 337 KiB/s wr, 11 op/s 2023-09-10T07:56:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:56:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:56:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 30996.148610 secs 2023-09-10T07:56:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30994.539109 secs 2023-09-10T07:56:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:04 smithi098 ceph-mon[103124]: pgmap v16689: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 337 KiB/s wr, 11 op/s 2023-09-10T07:56:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:56:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:56:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 30996.148610 secs 2023-09-10T07:56:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:06 smithi170 ceph-mon[102832]: pgmap v16690: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 468 KiB/s wr, 31 op/s 2023-09-10T07:56:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:06 smithi082 ceph-mon[98694]: pgmap v16690: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 468 KiB/s wr, 31 op/s 2023-09-10T07:56:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:06 smithi098 ceph-mon[103124]: pgmap v16690: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 468 KiB/s wr, 31 op/s 2023-09-10T07:56:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:08 smithi170 ceph-mon[102832]: pgmap v16691: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 385 KiB/s wr, 25 op/s 2023-09-10T07:56:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:08 smithi082 ceph-mon[98694]: pgmap v16691: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 385 KiB/s wr, 25 op/s 2023-09-10T07:56:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:08 smithi098 ceph-mon[103124]: pgmap v16691: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 385 KiB/s wr, 25 op/s 2023-09-10T07:56:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 30999.539168 secs 2023-09-10T07:56:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:09 smithi170 ceph-mon[102832]: osdmap e1111: 12 total, 12 up, 12 in 2023-09-10T07:56:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31000.705415 secs 2023-09-10T07:56:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31001.148675 secs 2023-09-10T07:56:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 30999.539168 secs 2023-09-10T07:56:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:09 smithi082 ceph-mon[98694]: osdmap e1111: 12 total, 12 up, 12 in 2023-09-10T07:56:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31000.705415 secs 2023-09-10T07:56:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31001.148675 secs 2023-09-10T07:56:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 30999.539168 secs 2023-09-10T07:56:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:09 smithi098 ceph-mon[103124]: osdmap e1111: 12 total, 12 up, 12 in 2023-09-10T07:56:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31000.705415 secs 2023-09-10T07:56:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31001.148675 secs 2023-09-10T07:56:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:56:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:56:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:10 smithi170 ceph-mon[102832]: pgmap v16693: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 462 KiB/s wr, 30 op/s 2023-09-10T07:56:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:56:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:56:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:10 smithi082 ceph-mon[98694]: pgmap v16693: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 462 KiB/s wr, 30 op/s 2023-09-10T07:56:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:56:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:56:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:10 smithi098 ceph-mon[103124]: pgmap v16693: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 462 KiB/s wr, 30 op/s 2023-09-10T07:56:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:12 smithi170 ceph-mon[102832]: pgmap v16694: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 157 KiB/s wr, 28 op/s 2023-09-10T07:56:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:12 smithi170 ceph-mon[102832]: osdmap e1112: 12 total, 12 up, 12 in 2023-09-10T07:56:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:12 smithi082 ceph-mon[98694]: pgmap v16694: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 157 KiB/s wr, 28 op/s 2023-09-10T07:56:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:12 smithi082 ceph-mon[98694]: osdmap e1112: 12 total, 12 up, 12 in 2023-09-10T07:56:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:12 smithi098 ceph-mon[103124]: pgmap v16694: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 157 KiB/s wr, 28 op/s 2023-09-10T07:56:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:12 smithi098 ceph-mon[103124]: osdmap e1112: 12 total, 12 up, 12 in 2023-09-10T07:56:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31005.705474 secs 2023-09-10T07:56:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31005.705474 secs 2023-09-10T07:56:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31005.705474 secs 2023-09-10T07:56:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31004.539260 secs 2023-09-10T07:56:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:14 smithi170 ceph-mon[102832]: pgmap v16696: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:56:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31006.148729 secs 2023-09-10T07:56:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31004.539260 secs 2023-09-10T07:56:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:14 smithi082 ceph-mon[98694]: pgmap v16696: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:56:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31006.148729 secs 2023-09-10T07:56:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31004.539260 secs 2023-09-10T07:56:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:14 smithi098 ceph-mon[103124]: pgmap v16696: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:56:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31006.148729 secs 2023-09-10T07:56:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:16 smithi170 ceph-mon[102832]: pgmap v16697: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T07:56:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:16 smithi082 ceph-mon[98694]: pgmap v16697: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T07:56:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:16 smithi098 ceph-mon[103124]: pgmap v16697: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T07:56:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:18 smithi170 ceph-mon[102832]: pgmap v16698: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 11 op/s 2023-09-10T07:56:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:18 smithi082 ceph-mon[98694]: pgmap v16698: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 11 op/s 2023-09-10T07:56:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:18 smithi098 ceph-mon[103124]: pgmap v16698: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 11 op/s 2023-09-10T07:56:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31009.539317 secs 2023-09-10T07:56:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31010.705534 secs 2023-09-10T07:56:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31011.148793 secs 2023-09-10T07:56:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31009.539317 secs 2023-09-10T07:56:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31010.705534 secs 2023-09-10T07:56:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31011.148793 secs 2023-09-10T07:56:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31009.539317 secs 2023-09-10T07:56:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31010.705534 secs 2023-09-10T07:56:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31011.148793 secs 2023-09-10T07:56:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:20 smithi170 ceph-mon[102832]: pgmap v16699: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:56:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:20 smithi082 ceph-mon[98694]: pgmap v16699: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:56:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:20 smithi098 ceph-mon[103124]: pgmap v16699: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:56:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:22 smithi170 ceph-mon[102832]: pgmap v16700: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:56:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:22 smithi082 ceph-mon[98694]: pgmap v16700: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:56:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:22 smithi098 ceph-mon[103124]: pgmap v16700: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:56:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31015.705611 secs 2023-09-10T07:56:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31015.705611 secs 2023-09-10T07:56:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31015.705611 secs 2023-09-10T07:56:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31014.539383 secs 2023-09-10T07:56:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:24 smithi170 ceph-mon[102832]: pgmap v16701: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:56:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31016.148857 secs 2023-09-10T07:56:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31014.539383 secs 2023-09-10T07:56:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:24 smithi082 ceph-mon[98694]: pgmap v16701: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:56:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31016.148857 secs 2023-09-10T07:56:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31014.539383 secs 2023-09-10T07:56:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:24 smithi098 ceph-mon[103124]: pgmap v16701: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:56:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31016.148857 secs 2023-09-10T07:56:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:26 smithi170 ceph-mon[102832]: pgmap v16702: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 13 op/s 2023-09-10T07:56:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:26 smithi082 ceph-mon[98694]: pgmap v16702: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 13 op/s 2023-09-10T07:56:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:26 smithi098 ceph-mon[103124]: pgmap v16702: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 13 op/s 2023-09-10T07:56:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:28 smithi170 ceph-mon[102832]: pgmap v16703: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 9 op/s 2023-09-10T07:56:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:28 smithi082 ceph-mon[98694]: pgmap v16703: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 9 op/s 2023-09-10T07:56:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:28 smithi098 ceph-mon[103124]: pgmap v16703: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 9 op/s 2023-09-10T07:56:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31019.539435 secs 2023-09-10T07:56:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31020.705671 secs 2023-09-10T07:56:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31021.148921 secs 2023-09-10T07:56:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31019.539435 secs 2023-09-10T07:56:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31020.705671 secs 2023-09-10T07:56:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31021.148921 secs 2023-09-10T07:56:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31019.539435 secs 2023-09-10T07:56:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31020.705671 secs 2023-09-10T07:56:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31021.148921 secs 2023-09-10T07:56:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:30 smithi170 ceph-mon[102832]: pgmap v16704: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 9 op/s 2023-09-10T07:56:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:30 smithi082 ceph-mon[98694]: pgmap v16704: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 9 op/s 2023-09-10T07:56:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:30 smithi098 ceph-mon[103124]: pgmap v16704: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 9 op/s 2023-09-10T07:56:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:32 smithi170 ceph-mon[102832]: pgmap v16705: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 214 KiB/s wr, 14 op/s 2023-09-10T07:56:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:32 smithi082 ceph-mon[98694]: pgmap v16705: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 214 KiB/s wr, 14 op/s 2023-09-10T07:56:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:32 smithi098 ceph-mon[103124]: pgmap v16705: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 214 KiB/s wr, 14 op/s 2023-09-10T07:56:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31025.705754 secs 2023-09-10T07:56:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31025.705754 secs 2023-09-10T07:56:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31025.705754 secs 2023-09-10T07:56:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:34 smithi170 ceph-mon[102832]: pgmap v16706: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 214 KiB/s wr, 10 op/s 2023-09-10T07:56:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31026.148986 secs 2023-09-10T07:56:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:34 smithi082 ceph-mon[98694]: pgmap v16706: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 214 KiB/s wr, 10 op/s 2023-09-10T07:56:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31026.148986 secs 2023-09-10T07:56:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:34 smithi098 ceph-mon[103124]: pgmap v16706: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 214 KiB/s wr, 10 op/s 2023-09-10T07:56:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31026.148986 secs 2023-09-10T07:56:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:56:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:56:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:56:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:36 smithi170 ceph-mon[102832]: pgmap v16707: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 518 KiB/s wr, 16 op/s 2023-09-10T07:56:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:56:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:56:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:56:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:36 smithi082 ceph-mon[98694]: pgmap v16707: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 518 KiB/s wr, 16 op/s 2023-09-10T07:56:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:56:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:56:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:56:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:36 smithi098 ceph-mon[103124]: pgmap v16707: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 518 KiB/s wr, 16 op/s 2023-09-10T07:56:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:56:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:56:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:56:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:38 smithi170 ceph-mon[102832]: pgmap v16708: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 414 KiB/s wr, 10 op/s 2023-09-10T07:56:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:56:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:56:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31030.705826 secs 2023-09-10T07:56:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:38 smithi082 ceph-mon[98694]: pgmap v16708: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 414 KiB/s wr, 10 op/s 2023-09-10T07:56:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:56:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:56:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31030.705826 secs 2023-09-10T07:56:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:38 smithi098 ceph-mon[103124]: pgmap v16708: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 414 KiB/s wr, 10 op/s 2023-09-10T07:56:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:56:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:56:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31030.705826 secs 2023-09-10T07:56:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31031.149053 secs 2023-09-10T07:56:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31031.149053 secs 2023-09-10T07:56:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31031.149053 secs 2023-09-10T07:56:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:40 smithi170 ceph-mon[102832]: pgmap v16709: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 414 KiB/s wr, 10 op/s 2023-09-10T07:56:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:40 smithi082 ceph-mon[98694]: pgmap v16709: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 414 KiB/s wr, 10 op/s 2023-09-10T07:56:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:40 smithi098 ceph-mon[103124]: pgmap v16709: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 414 KiB/s wr, 10 op/s 2023-09-10T07:56:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:42 smithi170 ceph-mon[102832]: pgmap v16710: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 558 KiB/s wr, 17 op/s 2023-09-10T07:56:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:42 smithi082 ceph-mon[98694]: pgmap v16710: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 558 KiB/s wr, 17 op/s 2023-09-10T07:56:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:42 smithi098 ceph-mon[103124]: pgmap v16710: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 558 KiB/s wr, 17 op/s 2023-09-10T07:56:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31035.705884 secs 2023-09-10T07:56:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31035.705884 secs 2023-09-10T07:56:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31035.705884 secs 2023-09-10T07:56:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:44 smithi170 ceph-mon[102832]: pgmap v16711: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 448 KiB/s wr, 11 op/s 2023-09-10T07:56:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:56:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:56:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31036.149120 secs 2023-09-10T07:56:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:44 smithi082 ceph-mon[98694]: pgmap v16711: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 448 KiB/s wr, 11 op/s 2023-09-10T07:56:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:56:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:56:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31036.149120 secs 2023-09-10T07:56:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:44 smithi098 ceph-mon[103124]: pgmap v16711: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 448 KiB/s wr, 11 op/s 2023-09-10T07:56:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:56:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:56:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31036.149120 secs 2023-09-10T07:56:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:56:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:56:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:56:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:56:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:56:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:56:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:56:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:56:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:56:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:56:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:56:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:56:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:46 smithi170 ceph-mon[102832]: pgmap v16712: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 612 KiB/s wr, 19 op/s 2023-09-10T07:56:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:56:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:56:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:46 smithi082 ceph-mon[98694]: pgmap v16712: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 612 KiB/s wr, 19 op/s 2023-09-10T07:56:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:56:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:56:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:46 smithi098 ceph-mon[103124]: pgmap v16712: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 612 KiB/s wr, 19 op/s 2023-09-10T07:56:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:56:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:56:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:48 smithi170 ceph-mon[102832]: pgmap v16713: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 308 KiB/s wr, 13 op/s 2023-09-10T07:56:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:48 smithi082 ceph-mon[98694]: pgmap v16713: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 308 KiB/s wr, 13 op/s 2023-09-10T07:56:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:48 smithi098 ceph-mon[103124]: pgmap v16713: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 308 KiB/s wr, 13 op/s 2023-09-10T07:56:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:49 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31039.539716 secs 2023-09-10T07:56:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31040.705965 secs 2023-09-10T07:56:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31041.149184 secs 2023-09-10T07:56:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:49 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31039.539716 secs 2023-09-10T07:56:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31040.705965 secs 2023-09-10T07:56:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31041.149184 secs 2023-09-10T07:56:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:49 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31039.539716 secs 2023-09-10T07:56:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31040.705965 secs 2023-09-10T07:56:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31041.149184 secs 2023-09-10T07:56:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:56:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:56:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:50 smithi170 ceph-mon[102832]: pgmap v16714: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 308 KiB/s wr, 13 op/s 2023-09-10T07:56:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:56:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:56:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:50 smithi082 ceph-mon[98694]: pgmap v16714: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 308 KiB/s wr, 13 op/s 2023-09-10T07:56:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:56:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:56:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:50 smithi098 ceph-mon[103124]: pgmap v16714: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 308 KiB/s wr, 13 op/s 2023-09-10T07:56:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:56:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:56:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:56:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:56:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:56:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:56:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:56:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:56:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:56:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:52 smithi170 ceph-mon[102832]: pgmap v16715: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 593 KiB/s wr, 21 op/s 2023-09-10T07:56:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:52 smithi082 ceph-mon[98694]: pgmap v16715: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 593 KiB/s wr, 21 op/s 2023-09-10T07:56:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:52 smithi098 ceph-mon[103124]: pgmap v16715: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 593 KiB/s wr, 21 op/s 2023-09-10T07:56:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31045.706044 secs 2023-09-10T07:56:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31045.706044 secs 2023-09-10T07:56:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31045.706044 secs 2023-09-10T07:56:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31044.539774 secs 2023-09-10T07:56:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:54 smithi170 ceph-mon[102832]: pgmap v16716: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 449 KiB/s wr, 15 op/s 2023-09-10T07:56:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31046.149248 secs 2023-09-10T07:56:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31044.539774 secs 2023-09-10T07:56:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:54 smithi082 ceph-mon[98694]: pgmap v16716: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 449 KiB/s wr, 15 op/s 2023-09-10T07:56:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31046.149248 secs 2023-09-10T07:56:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31044.539774 secs 2023-09-10T07:56:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:54 smithi098 ceph-mon[103124]: pgmap v16716: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 449 KiB/s wr, 15 op/s 2023-09-10T07:56:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31046.149248 secs 2023-09-10T07:56:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:56 smithi170 ceph-mon[102832]: pgmap v16717: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 636 KiB/s wr, 22 op/s 2023-09-10T07:56:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:56 smithi082 ceph-mon[98694]: pgmap v16717: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 636 KiB/s wr, 22 op/s 2023-09-10T07:56:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:56 smithi098 ceph-mon[103124]: pgmap v16717: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 636 KiB/s wr, 22 op/s 2023-09-10T07:56:58.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:58 smithi170 ceph-mon[102832]: pgmap v16718: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 472 KiB/s wr, 15 op/s 2023-09-10T07:56:58.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:56:58.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:56:58.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31050.706102 secs 2023-09-10T07:56:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:58 smithi082 ceph-mon[98694]: pgmap v16718: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 472 KiB/s wr, 15 op/s 2023-09-10T07:56:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:56:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:56:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31050.706102 secs 2023-09-10T07:56:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:58 smithi098 ceph-mon[103124]: pgmap v16718: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 472 KiB/s wr, 15 op/s 2023-09-10T07:56:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:56:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:56:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31050.706102 secs 2023-09-10T07:56:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31049.539836 secs 2023-09-10T07:56:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:56:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31051.149311 secs 2023-09-10T07:56:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31049.539836 secs 2023-09-10T07:56:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:56:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31051.149311 secs 2023-09-10T07:57:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31049.539836 secs 2023-09-10T07:57:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:56:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31051.149311 secs 2023-09-10T07:57:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:00 smithi170 ceph-mon[102832]: pgmap v16719: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 472 KiB/s wr, 15 op/s 2023-09-10T07:57:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:57:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:57:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:00 smithi082 ceph-mon[98694]: pgmap v16719: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 472 KiB/s wr, 15 op/s 2023-09-10T07:57:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:57:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:57:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:00 smithi098 ceph-mon[103124]: pgmap v16719: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 472 KiB/s wr, 15 op/s 2023-09-10T07:57:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:57:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:57:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:02 smithi170 ceph-mon[102832]: pgmap v16720: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 727 KiB/s wr, 21 op/s 2023-09-10T07:57:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:02 smithi082 ceph-mon[98694]: pgmap v16720: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 727 KiB/s wr, 21 op/s 2023-09-10T07:57:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:02 smithi098 ceph-mon[103124]: pgmap v16720: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 727 KiB/s wr, 21 op/s 2023-09-10T07:57:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31055.706159 secs 2023-09-10T07:57:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31055.706159 secs 2023-09-10T07:57:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31055.706159 secs 2023-09-10T07:57:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31054.539944 secs 2023-09-10T07:57:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:04 smithi170 ceph-mon[102832]: pgmap v16721: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 442 KiB/s wr, 12 op/s 2023-09-10T07:57:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31056.149375 secs 2023-09-10T07:57:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31054.539944 secs 2023-09-10T07:57:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:04 smithi082 ceph-mon[98694]: pgmap v16721: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 442 KiB/s wr, 12 op/s 2023-09-10T07:57:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31056.149375 secs 2023-09-10T07:57:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31054.539944 secs 2023-09-10T07:57:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:04 smithi098 ceph-mon[103124]: pgmap v16721: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 442 KiB/s wr, 12 op/s 2023-09-10T07:57:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31056.149375 secs 2023-09-10T07:57:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:06 smithi170 ceph-mon[102832]: pgmap v16722: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 573 KiB/s wr, 32 op/s 2023-09-10T07:57:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:06 smithi082 ceph-mon[98694]: pgmap v16722: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 573 KiB/s wr, 32 op/s 2023-09-10T07:57:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:06 smithi098 ceph-mon[103124]: pgmap v16722: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 573 KiB/s wr, 32 op/s 2023-09-10T07:57:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:08 smithi170 ceph-mon[102832]: pgmap v16723: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 386 KiB/s wr, 25 op/s 2023-09-10T07:57:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:08 smithi082 ceph-mon[98694]: pgmap v16723: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 386 KiB/s wr, 25 op/s 2023-09-10T07:57:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:08 smithi098 ceph-mon[103124]: pgmap v16723: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 386 KiB/s wr, 25 op/s 2023-09-10T07:57:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31059.540038 secs 2023-09-10T07:57:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:09 smithi170 ceph-mon[102832]: osdmap e1113: 12 total, 12 up, 12 in 2023-09-10T07:57:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31060.706217 secs 2023-09-10T07:57:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31061.149443 secs 2023-09-10T07:57:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31059.540038 secs 2023-09-10T07:57:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:09 smithi082 ceph-mon[98694]: osdmap e1113: 12 total, 12 up, 12 in 2023-09-10T07:57:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31060.706217 secs 2023-09-10T07:57:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31061.149443 secs 2023-09-10T07:57:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31059.540038 secs 2023-09-10T07:57:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:09 smithi098 ceph-mon[103124]: osdmap e1113: 12 total, 12 up, 12 in 2023-09-10T07:57:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31060.706217 secs 2023-09-10T07:57:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31061.149443 secs 2023-09-10T07:57:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:10 smithi170 ceph-mon[102832]: pgmap v16725: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 463 KiB/s wr, 30 op/s 2023-09-10T07:57:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:10 smithi082 ceph-mon[98694]: pgmap v16725: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 463 KiB/s wr, 30 op/s 2023-09-10T07:57:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:10 smithi098 ceph-mon[103124]: pgmap v16725: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 463 KiB/s wr, 30 op/s 2023-09-10T07:57:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:12 smithi170 ceph-mon[102832]: pgmap v16726: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 158 KiB/s wr, 28 op/s 2023-09-10T07:57:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:12 smithi170 ceph-mon[102832]: osdmap e1114: 12 total, 12 up, 12 in 2023-09-10T07:57:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:12 smithi082 ceph-mon[98694]: pgmap v16726: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 158 KiB/s wr, 28 op/s 2023-09-10T07:57:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:12 smithi082 ceph-mon[98694]: osdmap e1114: 12 total, 12 up, 12 in 2023-09-10T07:57:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:12 smithi098 ceph-mon[103124]: pgmap v16726: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 158 KiB/s wr, 28 op/s 2023-09-10T07:57:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:12 smithi098 ceph-mon[103124]: osdmap e1114: 12 total, 12 up, 12 in 2023-09-10T07:57:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31065.706278 secs 2023-09-10T07:57:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31065.706278 secs 2023-09-10T07:57:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31065.706278 secs 2023-09-10T07:57:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31064.540151 secs 2023-09-10T07:57:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:14 smithi170 ceph-mon[102832]: pgmap v16728: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-10T07:57:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31066.149490 secs 2023-09-10T07:57:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31064.540151 secs 2023-09-10T07:57:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:14 smithi082 ceph-mon[98694]: pgmap v16728: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-10T07:57:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31066.149490 secs 2023-09-10T07:57:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31064.540151 secs 2023-09-10T07:57:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:14 smithi098 ceph-mon[103124]: pgmap v16728: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-09-10T07:57:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31066.149490 secs 2023-09-10T07:57:16.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:16 smithi170 ceph-mon[102832]: pgmap v16729: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 13 op/s 2023-09-10T07:57:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:16 smithi082 ceph-mon[98694]: pgmap v16729: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 13 op/s 2023-09-10T07:57:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:16 smithi098 ceph-mon[103124]: pgmap v16729: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 13 op/s 2023-09-10T07:57:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:18 smithi170 ceph-mon[102832]: pgmap v16730: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 334 B/s wr, 11 op/s 2023-09-10T07:57:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:18 smithi082 ceph-mon[98694]: pgmap v16730: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 334 B/s wr, 11 op/s 2023-09-10T07:57:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:18 smithi098 ceph-mon[103124]: pgmap v16730: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 334 B/s wr, 11 op/s 2023-09-10T07:57:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31069.540213 secs 2023-09-10T07:57:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31070.706336 secs 2023-09-10T07:57:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31071.149555 secs 2023-09-10T07:57:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31069.540213 secs 2023-09-10T07:57:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31070.706336 secs 2023-09-10T07:57:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31071.149555 secs 2023-09-10T07:57:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31069.540213 secs 2023-09-10T07:57:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31070.706336 secs 2023-09-10T07:57:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31071.149555 secs 2023-09-10T07:57:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:20 smithi170 ceph-mon[102832]: pgmap v16731: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T07:57:20.910 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:20 smithi098 ceph-mon[103124]: pgmap v16731: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T07:57:20.910 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:20 smithi082 ceph-mon[98694]: pgmap v16731: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2023-09-10T07:57:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:22 smithi170 ceph-mon[102832]: pgmap v16732: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:57:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:22 smithi082 ceph-mon[98694]: pgmap v16732: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:57:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:22 smithi098 ceph-mon[103124]: pgmap v16732: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:57:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31075.706397 secs 2023-09-10T07:57:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31075.706397 secs 2023-09-10T07:57:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31075.706397 secs 2023-09-10T07:57:24.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31074.540274 secs 2023-09-10T07:57:24.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:24 smithi170 ceph-mon[102832]: pgmap v16733: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:57:24.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31076.149619 secs 2023-09-10T07:57:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31074.540274 secs 2023-09-10T07:57:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:24 smithi082 ceph-mon[98694]: pgmap v16733: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:57:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31076.149619 secs 2023-09-10T07:57:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31074.540274 secs 2023-09-10T07:57:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:24 smithi098 ceph-mon[103124]: pgmap v16733: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:57:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31076.149619 secs 2023-09-10T07:57:26.932 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:26 smithi170 ceph-mon[102832]: pgmap v16734: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-10T07:57:26.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:26 smithi082 ceph-mon[98694]: pgmap v16734: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-10T07:57:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:26 smithi098 ceph-mon[103124]: pgmap v16734: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2023-09-10T07:57:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:28 smithi170 ceph-mon[102832]: pgmap v16735: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-10T07:57:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:28 smithi082 ceph-mon[98694]: pgmap v16735: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-10T07:57:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:28 smithi098 ceph-mon[103124]: pgmap v16735: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-10T07:57:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31079.540336 secs 2023-09-10T07:57:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31080.706455 secs 2023-09-10T07:57:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31081.149683 secs 2023-09-10T07:57:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31079.540336 secs 2023-09-10T07:57:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31080.706455 secs 2023-09-10T07:57:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31081.149683 secs 2023-09-10T07:57:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31079.540336 secs 2023-09-10T07:57:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31080.706455 secs 2023-09-10T07:57:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31081.149683 secs 2023-09-10T07:57:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:30 smithi170 ceph-mon[102832]: pgmap v16736: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-10T07:57:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:30 smithi082 ceph-mon[98694]: pgmap v16736: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-10T07:57:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:30 smithi098 ceph-mon[103124]: pgmap v16736: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-09-10T07:57:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:32 smithi170 ceph-mon[102832]: pgmap v16737: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 13 op/s 2023-09-10T07:57:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:32 smithi082 ceph-mon[98694]: pgmap v16737: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 13 op/s 2023-09-10T07:57:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:32 smithi098 ceph-mon[103124]: pgmap v16737: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 13 op/s 2023-09-10T07:57:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31085.706514 secs 2023-09-10T07:57:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31085.706514 secs 2023-09-10T07:57:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31085.706514 secs 2023-09-10T07:57:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:34 smithi170 ceph-mon[102832]: pgmap v16738: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 9 op/s 2023-09-10T07:57:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:57:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:57:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31086.149746 secs 2023-09-10T07:57:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:34 smithi082 ceph-mon[98694]: pgmap v16738: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 9 op/s 2023-09-10T07:57:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:57:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:57:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31086.149746 secs 2023-09-10T07:57:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:34 smithi098 ceph-mon[103124]: pgmap v16738: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 9 op/s 2023-09-10T07:57:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:57:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:57:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31086.149746 secs 2023-09-10T07:57:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:57:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:57:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:57:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:36 smithi170 ceph-mon[102832]: pgmap v16739: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 247 KiB/s wr, 15 op/s 2023-09-10T07:57:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:57:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:57:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:57:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:36 smithi082 ceph-mon[98694]: pgmap v16739: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 247 KiB/s wr, 15 op/s 2023-09-10T07:57:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:57:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:57:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:57:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:36 smithi098 ceph-mon[103124]: pgmap v16739: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 247 KiB/s wr, 15 op/s 2023-09-10T07:57:37.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:57:37.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:57:37.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:57:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:38 smithi170 ceph-mon[102832]: pgmap v16740: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 247 KiB/s wr, 11 op/s 2023-09-10T07:57:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:38 smithi082 ceph-mon[98694]: pgmap v16740: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 247 KiB/s wr, 11 op/s 2023-09-10T07:57:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:38 smithi098 ceph-mon[103124]: pgmap v16740: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 247 KiB/s wr, 11 op/s 2023-09-10T07:57:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31090.706597 secs 2023-09-10T07:57:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31091.149811 secs 2023-09-10T07:57:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31090.706597 secs 2023-09-10T07:57:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31091.149811 secs 2023-09-10T07:57:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31090.706597 secs 2023-09-10T07:57:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31091.149811 secs 2023-09-10T07:57:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:40 smithi170 ceph-mon[102832]: pgmap v16741: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 247 KiB/s wr, 11 op/s 2023-09-10T07:57:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:40 smithi082 ceph-mon[98694]: pgmap v16741: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 247 KiB/s wr, 11 op/s 2023-09-10T07:57:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:40 smithi098 ceph-mon[103124]: pgmap v16741: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 247 KiB/s wr, 11 op/s 2023-09-10T07:57:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:42 smithi170 ceph-mon[102832]: pgmap v16742: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 482 KiB/s wr, 18 op/s 2023-09-10T07:57:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:42 smithi082 ceph-mon[98694]: pgmap v16742: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 482 KiB/s wr, 18 op/s 2023-09-10T07:57:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:42 smithi098 ceph-mon[103124]: pgmap v16742: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 482 KiB/s wr, 18 op/s 2023-09-10T07:57:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31095.706678 secs 2023-09-10T07:57:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31095.706678 secs 2023-09-10T07:57:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31095.706678 secs 2023-09-10T07:57:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:44 smithi170 ceph-mon[102832]: pgmap v16743: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 457 KiB/s wr, 12 op/s 2023-09-10T07:57:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:57:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:57:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31096.149877 secs 2023-09-10T07:57:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:44 smithi082 ceph-mon[98694]: pgmap v16743: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 457 KiB/s wr, 12 op/s 2023-09-10T07:57:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:57:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:57:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31096.149877 secs 2023-09-10T07:57:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:44 smithi098 ceph-mon[103124]: pgmap v16743: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 457 KiB/s wr, 12 op/s 2023-09-10T07:57:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:57:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:57:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31096.149877 secs 2023-09-10T07:57:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:57:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:57:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:57:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:46 smithi170 ceph-mon[102832]: pgmap v16744: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 755 KiB/s wr, 20 op/s 2023-09-10T07:57:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:57:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:57:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:57:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:57:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:57:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:57:46.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:57:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:46 smithi082 ceph-mon[98694]: pgmap v16744: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 755 KiB/s wr, 20 op/s 2023-09-10T07:57:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:57:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:57:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:57:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:57:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:57:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:57:46.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:57:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:46 smithi098 ceph-mon[103124]: pgmap v16744: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 755 KiB/s wr, 20 op/s 2023-09-10T07:57:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:57:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:57:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:57:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:57:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:57:47.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:57:47.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:57:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:48 smithi170 ceph-mon[102832]: pgmap v16745: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 533 KiB/s wr, 14 op/s 2023-09-10T07:57:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:48 smithi082 ceph-mon[98694]: pgmap v16745: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 533 KiB/s wr, 14 op/s 2023-09-10T07:57:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:48 smithi098 ceph-mon[103124]: pgmap v16745: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 533 KiB/s wr, 14 op/s 2023-09-10T07:57:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:49 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31099.540685 secs 2023-09-10T07:57:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31100.706736 secs 2023-09-10T07:57:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31101.149941 secs 2023-09-10T07:57:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:49 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31099.540685 secs 2023-09-10T07:57:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31100.706736 secs 2023-09-10T07:57:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31101.149941 secs 2023-09-10T07:57:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:49 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31099.540685 secs 2023-09-10T07:57:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31100.706736 secs 2023-09-10T07:57:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31101.149941 secs 2023-09-10T07:57:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:50 smithi170 ceph-mon[102832]: pgmap v16746: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 533 KiB/s wr, 14 op/s 2023-09-10T07:57:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:50 smithi082 ceph-mon[98694]: pgmap v16746: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 533 KiB/s wr, 14 op/s 2023-09-10T07:57:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:50 smithi098 ceph-mon[103124]: pgmap v16746: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 533 KiB/s wr, 14 op/s 2023-09-10T07:57:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:57:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:57:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:57:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:57:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:57:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:57:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:57:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:57:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:57:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:52 smithi082 ceph-mon[98694]: pgmap v16747: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 683 KiB/s wr, 21 op/s 2023-09-10T07:57:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:52 smithi098 ceph-mon[103124]: pgmap v16747: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 683 KiB/s wr, 21 op/s 2023-09-10T07:57:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:52 smithi170 ceph-mon[102832]: pgmap v16747: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 683 KiB/s wr, 21 op/s 2023-09-10T07:57:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31104.540795 secs 2023-09-10T07:57:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:57:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:57:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31105.706794 secs 2023-09-10T07:57:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:54 smithi082 ceph-mon[98694]: pgmap v16748: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 448 KiB/s wr, 15 op/s 2023-09-10T07:57:54.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31106.150005 secs 2023-09-10T07:57:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31104.540795 secs 2023-09-10T07:57:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:57:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:57:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31105.706794 secs 2023-09-10T07:57:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:54 smithi098 ceph-mon[103124]: pgmap v16748: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 448 KiB/s wr, 15 op/s 2023-09-10T07:57:54.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31106.150005 secs 2023-09-10T07:57:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31104.540795 secs 2023-09-10T07:57:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:57:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:57:54.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31105.706794 secs 2023-09-10T07:57:54.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:54 smithi170 ceph-mon[102832]: pgmap v16748: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 448 KiB/s wr, 15 op/s 2023-09-10T07:57:54.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31106.150005 secs 2023-09-10T07:57:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:57:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:57:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:56 smithi082 ceph-mon[98694]: pgmap v16749: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 602 KiB/s wr, 20 op/s 2023-09-10T07:57:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:57:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:57:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:56 smithi098 ceph-mon[103124]: pgmap v16749: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 602 KiB/s wr, 20 op/s 2023-09-10T07:57:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:57:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:57:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:56 smithi170 ceph-mon[102832]: pgmap v16749: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 602 KiB/s wr, 20 op/s 2023-09-10T07:57:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:58 smithi170 ceph-mon[102832]: pgmap v16750: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 304 KiB/s wr, 12 op/s 2023-09-10T07:57:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:58 smithi082 ceph-mon[98694]: pgmap v16750: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 304 KiB/s wr, 12 op/s 2023-09-10T07:57:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:58 smithi098 ceph-mon[103124]: pgmap v16750: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 304 KiB/s wr, 12 op/s 2023-09-10T07:57:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31109.540855 secs 2023-09-10T07:57:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31110.706852 secs 2023-09-10T07:57:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:57:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31111.150067 secs 2023-09-10T07:57:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31109.540855 secs 2023-09-10T07:57:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31110.706852 secs 2023-09-10T07:57:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:57:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31111.150067 secs 2023-09-10T07:57:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31109.540855 secs 2023-09-10T07:57:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31110.706852 secs 2023-09-10T07:57:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:57:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31111.150067 secs 2023-09-10T07:58:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:00 smithi170 ceph-mon[102832]: pgmap v16751: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 304 KiB/s wr, 12 op/s 2023-09-10T07:58:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:00 smithi082 ceph-mon[98694]: pgmap v16751: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 304 KiB/s wr, 12 op/s 2023-09-10T07:58:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:00 smithi098 ceph-mon[103124]: pgmap v16751: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 304 KiB/s wr, 12 op/s 2023-09-10T07:58:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:02 smithi170 ceph-mon[102832]: pgmap v16752: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 459 KiB/s wr, 31 op/s 2023-09-10T07:58:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:02 smithi082 ceph-mon[98694]: pgmap v16752: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 459 KiB/s wr, 31 op/s 2023-09-10T07:58:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:02 smithi098 ceph-mon[103124]: pgmap v16752: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 459 KiB/s wr, 31 op/s 2023-09-10T07:58:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31115.706911 secs 2023-09-10T07:58:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31115.706911 secs 2023-09-10T07:58:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31115.706911 secs 2023-09-10T07:58:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31114.540919 secs 2023-09-10T07:58:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:04 smithi170 ceph-mon[102832]: pgmap v16753: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 309 KiB/s wr, 24 op/s 2023-09-10T07:58:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:58:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:58:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31116.150132 secs 2023-09-10T07:58:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31114.540919 secs 2023-09-10T07:58:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:04 smithi082 ceph-mon[98694]: pgmap v16753: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 309 KiB/s wr, 24 op/s 2023-09-10T07:58:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:58:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:58:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31116.150132 secs 2023-09-10T07:58:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31114.540919 secs 2023-09-10T07:58:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:04 smithi098 ceph-mon[103124]: pgmap v16753: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 309 KiB/s wr, 24 op/s 2023-09-10T07:58:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:58:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:58:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31116.150132 secs 2023-09-10T07:58:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:06 smithi170 ceph-mon[102832]: pgmap v16754: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 321 KiB/s wr, 30 op/s 2023-09-10T07:58:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:58:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:58:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:06 smithi082 ceph-mon[98694]: pgmap v16754: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 321 KiB/s wr, 30 op/s 2023-09-10T07:58:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:58:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:58:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:06 smithi098 ceph-mon[103124]: pgmap v16754: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 321 KiB/s wr, 30 op/s 2023-09-10T07:58:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:58:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:58:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:08 smithi170 ceph-mon[102832]: pgmap v16755: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 167 KiB/s wr, 25 op/s 2023-09-10T07:58:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:08 smithi082 ceph-mon[98694]: pgmap v16755: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 167 KiB/s wr, 25 op/s 2023-09-10T07:58:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:08 smithi098 ceph-mon[103124]: pgmap v16755: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 167 KiB/s wr, 25 op/s 2023-09-10T07:58:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31119.541024 secs 2023-09-10T07:58:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:09 smithi170 ceph-mon[102832]: osdmap e1115: 12 total, 12 up, 12 in 2023-09-10T07:58:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31120.706974 secs 2023-09-10T07:58:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31121.150198 secs 2023-09-10T07:58:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31119.541024 secs 2023-09-10T07:58:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:09 smithi082 ceph-mon[98694]: osdmap e1115: 12 total, 12 up, 12 in 2023-09-10T07:58:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31120.706974 secs 2023-09-10T07:58:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31121.150198 secs 2023-09-10T07:58:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31119.541024 secs 2023-09-10T07:58:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:09 smithi098 ceph-mon[103124]: osdmap e1115: 12 total, 12 up, 12 in 2023-09-10T07:58:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31120.706974 secs 2023-09-10T07:58:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31121.150198 secs 2023-09-10T07:58:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:10 smithi170 ceph-mon[102832]: pgmap v16757: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 200 KiB/s wr, 30 op/s 2023-09-10T07:58:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:10 smithi082 ceph-mon[98694]: pgmap v16757: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 200 KiB/s wr, 30 op/s 2023-09-10T07:58:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:10 smithi098 ceph-mon[103124]: pgmap v16757: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 200 KiB/s wr, 30 op/s 2023-09-10T07:58:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:12 smithi170 ceph-mon[102832]: pgmap v16758: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-09-10T07:58:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:12 smithi170 ceph-mon[102832]: osdmap e1116: 12 total, 12 up, 12 in 2023-09-10T07:58:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:12 smithi082 ceph-mon[98694]: pgmap v16758: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-09-10T07:58:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:12 smithi082 ceph-mon[98694]: osdmap e1116: 12 total, 12 up, 12 in 2023-09-10T07:58:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:12 smithi098 ceph-mon[103124]: pgmap v16758: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-09-10T07:58:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:12 smithi098 ceph-mon[103124]: osdmap e1116: 12 total, 12 up, 12 in 2023-09-10T07:58:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31125.707056 secs 2023-09-10T07:58:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31125.707056 secs 2023-09-10T07:58:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31125.707056 secs 2023-09-10T07:58:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31124.541128 secs 2023-09-10T07:58:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:14 smithi170 ceph-mon[102832]: pgmap v16760: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:58:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31126.150261 secs 2023-09-10T07:58:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31124.541128 secs 2023-09-10T07:58:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:14 smithi082 ceph-mon[98694]: pgmap v16760: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:58:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31126.150261 secs 2023-09-10T07:58:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31124.541128 secs 2023-09-10T07:58:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:14 smithi098 ceph-mon[103124]: pgmap v16760: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:58:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31126.150261 secs 2023-09-10T07:58:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:16 smithi170 ceph-mon[102832]: pgmap v16761: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T07:58:16.976 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:16 smithi082 ceph-mon[98694]: pgmap v16761: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T07:58:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:16 smithi098 ceph-mon[103124]: pgmap v16761: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T07:58:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:18 smithi170 ceph-mon[102832]: pgmap v16762: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 11 op/s 2023-09-10T07:58:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:18 smithi082 ceph-mon[98694]: pgmap v16762: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 11 op/s 2023-09-10T07:58:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:18 smithi098 ceph-mon[103124]: pgmap v16762: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 11 op/s 2023-09-10T07:58:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31129.541191 secs 2023-09-10T07:58:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31130.707118 secs 2023-09-10T07:58:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31131.150329 secs 2023-09-10T07:58:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31129.541191 secs 2023-09-10T07:58:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31130.707118 secs 2023-09-10T07:58:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31131.150329 secs 2023-09-10T07:58:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31129.541191 secs 2023-09-10T07:58:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31130.707118 secs 2023-09-10T07:58:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31131.150329 secs 2023-09-10T07:58:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:20 smithi170 ceph-mon[102832]: pgmap v16763: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:58:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:20 smithi082 ceph-mon[98694]: pgmap v16763: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:58:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:20 smithi098 ceph-mon[103124]: pgmap v16763: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:58:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:22 smithi170 ceph-mon[102832]: pgmap v16764: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:58:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:22 smithi082 ceph-mon[98694]: pgmap v16764: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:58:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:22 smithi098 ceph-mon[103124]: pgmap v16764: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:58:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31135.707218 secs 2023-09-10T07:58:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31135.707218 secs 2023-09-10T07:58:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31135.707218 secs 2023-09-10T07:58:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31134.541250 secs 2023-09-10T07:58:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:24 smithi170 ceph-mon[102832]: pgmap v16765: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:58:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31136.150392 secs 2023-09-10T07:58:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31134.541250 secs 2023-09-10T07:58:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:24 smithi082 ceph-mon[98694]: pgmap v16765: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:58:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31136.150392 secs 2023-09-10T07:58:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31134.541250 secs 2023-09-10T07:58:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:24 smithi098 ceph-mon[103124]: pgmap v16765: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:58:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31136.150392 secs 2023-09-10T07:58:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:26 smithi170 ceph-mon[102832]: pgmap v16766: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 13 op/s 2023-09-10T07:58:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:26 smithi082 ceph-mon[98694]: pgmap v16766: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 13 op/s 2023-09-10T07:58:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:26 smithi098 ceph-mon[103124]: pgmap v16766: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 104 KiB/s wr, 13 op/s 2023-09-10T07:58:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:28 smithi170 ceph-mon[102832]: pgmap v16767: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 8 op/s 2023-09-10T07:58:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:28 smithi082 ceph-mon[98694]: pgmap v16767: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 8 op/s 2023-09-10T07:58:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:28 smithi098 ceph-mon[103124]: pgmap v16767: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 8 op/s 2023-09-10T07:58:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31139.541356 secs 2023-09-10T07:58:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31140.707302 secs 2023-09-10T07:58:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31141.150456 secs 2023-09-10T07:58:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31139.541356 secs 2023-09-10T07:58:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31140.707302 secs 2023-09-10T07:58:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31141.150456 secs 2023-09-10T07:58:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31139.541356 secs 2023-09-10T07:58:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31140.707302 secs 2023-09-10T07:58:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31141.150456 secs 2023-09-10T07:58:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:30 smithi170 ceph-mon[102832]: pgmap v16768: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 8 op/s 2023-09-10T07:58:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:30 smithi082 ceph-mon[98694]: pgmap v16768: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 8 op/s 2023-09-10T07:58:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:30 smithi098 ceph-mon[103124]: pgmap v16768: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 8 op/s 2023-09-10T07:58:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:32 smithi170 ceph-mon[102832]: pgmap v16769: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 105 KiB/s wr, 13 op/s 2023-09-10T07:58:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:32 smithi082 ceph-mon[98694]: pgmap v16769: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 105 KiB/s wr, 13 op/s 2023-09-10T07:58:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:32 smithi098 ceph-mon[103124]: pgmap v16769: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 105 KiB/s wr, 13 op/s 2023-09-10T07:58:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31145.707386 secs 2023-09-10T07:58:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31145.707386 secs 2023-09-10T07:58:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31145.707386 secs 2023-09-10T07:58:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:34 smithi170 ceph-mon[102832]: pgmap v16770: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 9 op/s 2023-09-10T07:58:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31146.150482 secs 2023-09-10T07:58:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:34 smithi082 ceph-mon[98694]: pgmap v16770: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 9 op/s 2023-09-10T07:58:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31146.150482 secs 2023-09-10T07:58:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:34 smithi098 ceph-mon[103124]: pgmap v16770: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 9 op/s 2023-09-10T07:58:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31146.150482 secs 2023-09-10T07:58:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:58:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:58:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:58:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:36 smithi170 ceph-mon[102832]: pgmap v16771: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 351 KiB/s wr, 15 op/s 2023-09-10T07:58:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:58:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:58:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:58:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:36 smithi082 ceph-mon[98694]: pgmap v16771: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 351 KiB/s wr, 15 op/s 2023-09-10T07:58:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:58:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:58:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:58:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:36 smithi098 ceph-mon[103124]: pgmap v16771: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 351 KiB/s wr, 15 op/s 2023-09-10T07:58:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:58:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:58:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:58:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:38 smithi170 ceph-mon[102832]: pgmap v16772: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 247 KiB/s wr, 10 op/s 2023-09-10T07:58:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:58:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:58:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31150.707468 secs 2023-09-10T07:58:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:38 smithi082 ceph-mon[98694]: pgmap v16772: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 247 KiB/s wr, 10 op/s 2023-09-10T07:58:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:58:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:58:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31150.707468 secs 2023-09-10T07:58:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:38 smithi098 ceph-mon[103124]: pgmap v16772: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 247 KiB/s wr, 10 op/s 2023-09-10T07:58:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:58:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:58:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31150.707468 secs 2023-09-10T07:58:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31151.150547 secs 2023-09-10T07:58:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31151.150547 secs 2023-09-10T07:58:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31151.150547 secs 2023-09-10T07:58:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:40 smithi170 ceph-mon[102832]: pgmap v16773: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 247 KiB/s wr, 10 op/s 2023-09-10T07:58:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:40 smithi082 ceph-mon[98694]: pgmap v16773: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 247 KiB/s wr, 10 op/s 2023-09-10T07:58:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:40 smithi098 ceph-mon[103124]: pgmap v16773: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 247 KiB/s wr, 10 op/s 2023-09-10T07:58:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:42 smithi170 ceph-mon[102832]: pgmap v16774: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 501 KiB/s wr, 17 op/s 2023-09-10T07:58:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:42 smithi082 ceph-mon[98694]: pgmap v16774: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 501 KiB/s wr, 17 op/s 2023-09-10T07:58:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:42 smithi098 ceph-mon[103124]: pgmap v16774: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 501 KiB/s wr, 17 op/s 2023-09-10T07:58:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31155.707551 secs 2023-09-10T07:58:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31155.707551 secs 2023-09-10T07:58:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31155.707551 secs 2023-09-10T07:58:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:44 smithi170 ceph-mon[102832]: pgmap v16775: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 501 KiB/s wr, 12 op/s 2023-09-10T07:58:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:58:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:58:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31156.150614 secs 2023-09-10T07:58:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:44 smithi082 ceph-mon[98694]: pgmap v16775: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 501 KiB/s wr, 12 op/s 2023-09-10T07:58:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:58:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:58:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31156.150614 secs 2023-09-10T07:58:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:44 smithi098 ceph-mon[103124]: pgmap v16775: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 501 KiB/s wr, 12 op/s 2023-09-10T07:58:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:58:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:58:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31156.150614 secs 2023-09-10T07:58:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:46 smithi170 ceph-mon[102832]: pgmap v16776: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 782 KiB/s wr, 20 op/s 2023-09-10T07:58:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:58:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:58:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:58:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:46 smithi082 ceph-mon[98694]: pgmap v16776: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 782 KiB/s wr, 20 op/s 2023-09-10T07:58:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:58:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:58:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:58:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:46 smithi098 ceph-mon[103124]: pgmap v16776: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 782 KiB/s wr, 20 op/s 2023-09-10T07:58:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:58:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:58:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:58:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:58:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:58:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:58:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:58:48.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:48 smithi170 ceph-mon[102832]: pgmap v16777: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 536 KiB/s wr, 14 op/s 2023-09-10T07:58:48.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:58:48.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:58:48.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:58:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:58:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:58:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:58:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:58:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:48 smithi082 ceph-mon[98694]: pgmap v16777: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 536 KiB/s wr, 14 op/s 2023-09-10T07:58:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:58:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:58:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:58:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:58:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:58:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:58:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:58:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:48 smithi098 ceph-mon[103124]: pgmap v16777: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 536 KiB/s wr, 14 op/s 2023-09-10T07:58:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:58:48.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:58:48.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:58:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31160.707609 secs 2023-09-10T07:58:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31161.150681 secs 2023-09-10T07:58:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31160.707609 secs 2023-09-10T07:58:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31161.150681 secs 2023-09-10T07:58:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31160.707609 secs 2023-09-10T07:58:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31161.150681 secs 2023-09-10T07:58:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:50 smithi170 ceph-mon[102832]: pgmap v16778: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 536 KiB/s wr, 14 op/s 2023-09-10T07:58:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:50 smithi082 ceph-mon[98694]: pgmap v16778: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 536 KiB/s wr, 14 op/s 2023-09-10T07:58:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:50 smithi098 ceph-mon[103124]: pgmap v16778: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 536 KiB/s wr, 14 op/s 2023-09-10T07:58:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:52 smithi170 ceph-mon[102832]: pgmap v16779: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 686 KiB/s wr, 20 op/s 2023-09-10T07:58:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:52 smithi082 ceph-mon[98694]: pgmap v16779: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 686 KiB/s wr, 20 op/s 2023-09-10T07:58:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:52 smithi098 ceph-mon[103124]: pgmap v16779: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 686 KiB/s wr, 20 op/s 2023-09-10T07:58:53.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31165.707674 secs 2023-09-10T07:58:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31165.707674 secs 2023-09-10T07:58:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31165.707674 secs 2023-09-10T07:58:54.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31164.541678 secs 2023-09-10T07:58:54.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:54 smithi170 ceph-mon[102832]: pgmap v16780: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 432 KiB/s wr, 13 op/s 2023-09-10T07:58:54.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:58:54.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:58:54.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31166.150744 secs 2023-09-10T07:58:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31164.541678 secs 2023-09-10T07:58:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:54 smithi082 ceph-mon[98694]: pgmap v16780: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 432 KiB/s wr, 13 op/s 2023-09-10T07:58:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:58:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:58:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31166.150744 secs 2023-09-10T07:58:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31164.541678 secs 2023-09-10T07:58:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:54 smithi098 ceph-mon[103124]: pgmap v16780: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 432 KiB/s wr, 13 op/s 2023-09-10T07:58:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:58:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:58:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31166.150744 secs 2023-09-10T07:58:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:58:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:58:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:58:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:58:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:58:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:58:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:58:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:58:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:58:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:56 smithi082 ceph-mon[98694]: pgmap v16781: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 586 KiB/s wr, 19 op/s 2023-09-10T07:58:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:56 smithi098 ceph-mon[103124]: pgmap v16781: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 586 KiB/s wr, 19 op/s 2023-09-10T07:58:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:56 smithi170 ceph-mon[102832]: pgmap v16781: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 586 KiB/s wr, 19 op/s 2023-09-10T07:58:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:58 smithi170 ceph-mon[102832]: pgmap v16782: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 304 KiB/s wr, 12 op/s 2023-09-10T07:58:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:58 smithi082 ceph-mon[98694]: pgmap v16782: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 304 KiB/s wr, 12 op/s 2023-09-10T07:58:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:58 smithi098 ceph-mon[103124]: pgmap v16782: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 304 KiB/s wr, 12 op/s 2023-09-10T07:58:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31169.541739 secs 2023-09-10T07:58:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31170.707726 secs 2023-09-10T07:58:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:58:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31171.150808 secs 2023-09-10T07:58:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31169.541739 secs 2023-09-10T07:58:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31170.707726 secs 2023-09-10T07:58:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:58:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31171.150808 secs 2023-09-10T07:58:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31169.541739 secs 2023-09-10T07:58:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31170.707726 secs 2023-09-10T07:58:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:58:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31171.150808 secs 2023-09-10T07:59:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:00 smithi170 ceph-mon[102832]: pgmap v16783: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 304 KiB/s wr, 12 op/s 2023-09-10T07:59:00.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:00 smithi082 ceph-mon[98694]: pgmap v16783: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 304 KiB/s wr, 12 op/s 2023-09-10T07:59:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:00 smithi098 ceph-mon[103124]: pgmap v16783: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 304 KiB/s wr, 12 op/s 2023-09-10T07:59:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:02 smithi170 ceph-mon[102832]: pgmap v16784: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 427 KiB/s wr, 18 op/s 2023-09-10T07:59:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:02 smithi082 ceph-mon[98694]: pgmap v16784: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 427 KiB/s wr, 18 op/s 2023-09-10T07:59:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:02 smithi098 ceph-mon[103124]: pgmap v16784: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 427 KiB/s wr, 18 op/s 2023-09-10T07:59:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31175.707793 secs 2023-09-10T07:59:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31175.707793 secs 2023-09-10T07:59:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31175.707793 secs 2023-09-10T07:59:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31174.541843 secs 2023-09-10T07:59:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:04 smithi170 ceph-mon[102832]: pgmap v16785: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 11 op/s 2023-09-10T07:59:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31176.150870 secs 2023-09-10T07:59:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31174.541843 secs 2023-09-10T07:59:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:04 smithi082 ceph-mon[98694]: pgmap v16785: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 11 op/s 2023-09-10T07:59:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31176.150870 secs 2023-09-10T07:59:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31174.541843 secs 2023-09-10T07:59:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:04 smithi098 ceph-mon[103124]: pgmap v16785: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 11 op/s 2023-09-10T07:59:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31176.150870 secs 2023-09-10T07:59:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:06 smithi170 ceph-mon[102832]: pgmap v16786: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 322 KiB/s wr, 30 op/s 2023-09-10T07:59:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:06 smithi082 ceph-mon[98694]: pgmap v16786: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 322 KiB/s wr, 30 op/s 2023-09-10T07:59:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:06 smithi098 ceph-mon[103124]: pgmap v16786: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 322 KiB/s wr, 30 op/s 2023-09-10T07:59:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:08 smithi170 ceph-mon[102832]: pgmap v16787: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 167 KiB/s wr, 24 op/s 2023-09-10T07:59:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:08 smithi082 ceph-mon[98694]: pgmap v16787: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 167 KiB/s wr, 24 op/s 2023-09-10T07:59:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:08 smithi098 ceph-mon[103124]: pgmap v16787: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 167 KiB/s wr, 24 op/s 2023-09-10T07:59:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31179.541907 secs 2023-09-10T07:59:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:09 smithi170 ceph-mon[102832]: osdmap e1117: 12 total, 12 up, 12 in 2023-09-10T07:59:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31180.707853 secs 2023-09-10T07:59:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31181.150934 secs 2023-09-10T07:59:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31179.541907 secs 2023-09-10T07:59:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:09 smithi082 ceph-mon[98694]: osdmap e1117: 12 total, 12 up, 12 in 2023-09-10T07:59:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31180.707853 secs 2023-09-10T07:59:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31181.150934 secs 2023-09-10T07:59:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31179.541907 secs 2023-09-10T07:59:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:09 smithi098 ceph-mon[103124]: osdmap e1117: 12 total, 12 up, 12 in 2023-09-10T07:59:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31180.707853 secs 2023-09-10T07:59:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31181.150934 secs 2023-09-10T07:59:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:10 smithi170 ceph-mon[102832]: pgmap v16789: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 201 KiB/s wr, 28 op/s 2023-09-10T07:59:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:10 smithi082 ceph-mon[98694]: pgmap v16789: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 201 KiB/s wr, 28 op/s 2023-09-10T07:59:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:10 smithi098 ceph-mon[103124]: pgmap v16789: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 201 KiB/s wr, 28 op/s 2023-09-10T07:59:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:12 smithi170 ceph-mon[102832]: pgmap v16790: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2023-09-10T07:59:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:12 smithi170 ceph-mon[102832]: osdmap e1118: 12 total, 12 up, 12 in 2023-09-10T07:59:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:12 smithi082 ceph-mon[98694]: pgmap v16790: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2023-09-10T07:59:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:12 smithi082 ceph-mon[98694]: osdmap e1118: 12 total, 12 up, 12 in 2023-09-10T07:59:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:12 smithi098 ceph-mon[103124]: pgmap v16790: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2023-09-10T07:59:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:12 smithi098 ceph-mon[103124]: osdmap e1118: 12 total, 12 up, 12 in 2023-09-10T07:59:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31185.707943 secs 2023-09-10T07:59:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31185.707943 secs 2023-09-10T07:59:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31185.707943 secs 2023-09-10T07:59:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31184.541974 secs 2023-09-10T07:59:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:14 smithi170 ceph-mon[102832]: pgmap v16792: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:59:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31186.150998 secs 2023-09-10T07:59:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31184.541974 secs 2023-09-10T07:59:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:14 smithi082 ceph-mon[98694]: pgmap v16792: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:59:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31186.150998 secs 2023-09-10T07:59:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31184.541974 secs 2023-09-10T07:59:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:14 smithi098 ceph-mon[103124]: pgmap v16792: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T07:59:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31186.150998 secs 2023-09-10T07:59:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:16 smithi170 ceph-mon[102832]: pgmap v16793: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T07:59:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:16 smithi082 ceph-mon[98694]: pgmap v16793: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T07:59:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:16 smithi098 ceph-mon[103124]: pgmap v16793: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T07:59:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:18 smithi170 ceph-mon[102832]: pgmap v16794: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 11 op/s 2023-09-10T07:59:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:18 smithi082 ceph-mon[98694]: pgmap v16794: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 11 op/s 2023-09-10T07:59:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:18 smithi098 ceph-mon[103124]: pgmap v16794: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 11 op/s 2023-09-10T07:59:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31189.542061 secs 2023-09-10T07:59:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31190.708002 secs 2023-09-10T07:59:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31191.151063 secs 2023-09-10T07:59:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31189.542061 secs 2023-09-10T07:59:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31190.708002 secs 2023-09-10T07:59:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31191.151063 secs 2023-09-10T07:59:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31189.542061 secs 2023-09-10T07:59:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31190.708002 secs 2023-09-10T07:59:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31191.151063 secs 2023-09-10T07:59:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:20 smithi170 ceph-mon[102832]: pgmap v16795: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:59:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:20 smithi082 ceph-mon[98694]: pgmap v16795: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:59:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:20 smithi098 ceph-mon[103124]: pgmap v16795: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T07:59:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:22 smithi170 ceph-mon[102832]: pgmap v16796: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:59:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:22 smithi082 ceph-mon[98694]: pgmap v16796: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:59:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:22 smithi098 ceph-mon[103124]: pgmap v16796: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T07:59:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31195.708059 secs 2023-09-10T07:59:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31195.708059 secs 2023-09-10T07:59:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31195.708059 secs 2023-09-10T07:59:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31194.542120 secs 2023-09-10T07:59:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:24 smithi170 ceph-mon[102832]: pgmap v16797: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:59:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31196.151127 secs 2023-09-10T07:59:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31194.542120 secs 2023-09-10T07:59:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:24 smithi082 ceph-mon[98694]: pgmap v16797: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:59:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31196.151127 secs 2023-09-10T07:59:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31194.542120 secs 2023-09-10T07:59:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:24 smithi098 ceph-mon[103124]: pgmap v16797: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T07:59:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31196.151127 secs 2023-09-10T07:59:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:26 smithi170 ceph-mon[102832]: pgmap v16798: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2023-09-10T07:59:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:26 smithi082 ceph-mon[98694]: pgmap v16798: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2023-09-10T07:59:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:26 smithi098 ceph-mon[103124]: pgmap v16798: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2023-09-10T07:59:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:28 smithi170 ceph-mon[102832]: pgmap v16799: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:59:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:28 smithi082 ceph-mon[98694]: pgmap v16799: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:59:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:28 smithi098 ceph-mon[103124]: pgmap v16799: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:59:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31199.542180 secs 2023-09-10T07:59:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31200.708115 secs 2023-09-10T07:59:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31201.151192 secs 2023-09-10T07:59:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31199.542180 secs 2023-09-10T07:59:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31200.708115 secs 2023-09-10T07:59:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31201.151192 secs 2023-09-10T07:59:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31199.542180 secs 2023-09-10T07:59:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31200.708115 secs 2023-09-10T07:59:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31201.151192 secs 2023-09-10T07:59:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:30 smithi170 ceph-mon[102832]: pgmap v16800: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:59:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:30 smithi082 ceph-mon[98694]: pgmap v16800: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:59:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:30 smithi098 ceph-mon[103124]: pgmap v16800: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-09-10T07:59:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:32 smithi170 ceph-mon[102832]: pgmap v16801: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 13 op/s 2023-09-10T07:59:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:32 smithi082 ceph-mon[98694]: pgmap v16801: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 13 op/s 2023-09-10T07:59:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:32 smithi098 ceph-mon[103124]: pgmap v16801: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 13 op/s 2023-09-10T07:59:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31205.708174 secs 2023-09-10T07:59:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31205.708174 secs 2023-09-10T07:59:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31205.708174 secs 2023-09-10T07:59:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:34 smithi170 ceph-mon[102832]: pgmap v16802: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 8 op/s 2023-09-10T07:59:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31206.151256 secs 2023-09-10T07:59:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:34 smithi082 ceph-mon[98694]: pgmap v16802: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 8 op/s 2023-09-10T07:59:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31206.151256 secs 2023-09-10T07:59:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:34 smithi098 ceph-mon[103124]: pgmap v16802: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 8 op/s 2023-09-10T07:59:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31206.151256 secs 2023-09-10T07:59:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:59:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:59:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T07:59:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:36 smithi170 ceph-mon[102832]: pgmap v16803: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 250 KiB/s wr, 15 op/s 2023-09-10T07:59:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:59:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:59:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:59:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:36 smithi082 ceph-mon[98694]: pgmap v16803: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 250 KiB/s wr, 15 op/s 2023-09-10T07:59:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:59:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:59:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:59:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:36 smithi098 ceph-mon[103124]: pgmap v16803: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 250 KiB/s wr, 15 op/s 2023-09-10T07:59:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:59:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:59:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:59:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:38 smithi170 ceph-mon[102832]: pgmap v16804: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 250 KiB/s wr, 11 op/s 2023-09-10T07:59:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:59:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:59:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31210.708231 secs 2023-09-10T07:59:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:38 smithi082 ceph-mon[98694]: pgmap v16804: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 250 KiB/s wr, 11 op/s 2023-09-10T07:59:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:59:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:59:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31210.708231 secs 2023-09-10T07:59:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:38 smithi098 ceph-mon[103124]: pgmap v16804: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 250 KiB/s wr, 11 op/s 2023-09-10T07:59:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:59:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:59:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31210.708231 secs 2023-09-10T07:59:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31211.151324 secs 2023-09-10T07:59:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31211.151324 secs 2023-09-10T07:59:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31211.151324 secs 2023-09-10T07:59:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:40 smithi170 ceph-mon[102832]: pgmap v16805: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 250 KiB/s wr, 11 op/s 2023-09-10T07:59:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:40 smithi082 ceph-mon[98694]: pgmap v16805: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 250 KiB/s wr, 11 op/s 2023-09-10T07:59:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:40 smithi098 ceph-mon[103124]: pgmap v16805: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 250 KiB/s wr, 11 op/s 2023-09-10T07:59:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:42 smithi170 ceph-mon[102832]: pgmap v16806: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 487 KiB/s wr, 17 op/s 2023-09-10T07:59:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:42 smithi082 ceph-mon[98694]: pgmap v16806: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 487 KiB/s wr, 17 op/s 2023-09-10T07:59:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:42 smithi098 ceph-mon[103124]: pgmap v16806: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 487 KiB/s wr, 17 op/s 2023-09-10T07:59:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31215.708288 secs 2023-09-10T07:59:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31215.708288 secs 2023-09-10T07:59:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31215.708288 secs 2023-09-10T07:59:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:44 smithi170 ceph-mon[102832]: pgmap v16807: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 487 KiB/s wr, 13 op/s 2023-09-10T07:59:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:59:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:59:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31216.151385 secs 2023-09-10T07:59:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:44 smithi082 ceph-mon[98694]: pgmap v16807: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 487 KiB/s wr, 13 op/s 2023-09-10T07:59:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:59:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:59:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31216.151385 secs 2023-09-10T07:59:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:44 smithi098 ceph-mon[103124]: pgmap v16807: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 487 KiB/s wr, 13 op/s 2023-09-10T07:59:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:59:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:59:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31216.151385 secs 2023-09-10T07:59:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:46 smithi170 ceph-mon[102832]: pgmap v16808: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 789 KiB/s wr, 20 op/s 2023-09-10T07:59:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:59:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:59:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:46 smithi082 ceph-mon[98694]: pgmap v16808: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 789 KiB/s wr, 20 op/s 2023-09-10T07:59:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:59:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:59:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:46 smithi098 ceph-mon[103124]: pgmap v16808: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 789 KiB/s wr, 20 op/s 2023-09-10T07:59:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:59:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:59:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:48 smithi170 ceph-mon[102832]: pgmap v16809: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 539 KiB/s wr, 13 op/s 2023-09-10T07:59:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:59:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:59:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:59:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:59:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:48 smithi082 ceph-mon[98694]: pgmap v16809: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 539 KiB/s wr, 13 op/s 2023-09-10T07:59:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:59:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:59:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:59:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:59:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:48 smithi098 ceph-mon[103124]: pgmap v16809: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 539 KiB/s wr, 13 op/s 2023-09-10T07:59:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T07:59:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T07:59:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T07:59:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T07:59:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31220.708346 secs 2023-09-10T07:59:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31221.151449 secs 2023-09-10T07:59:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31220.708346 secs 2023-09-10T07:59:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31221.151449 secs 2023-09-10T07:59:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31220.708346 secs 2023-09-10T07:59:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31221.151449 secs 2023-09-10T07:59:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:50 smithi170 ceph-mon[102832]: pgmap v16810: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 539 KiB/s wr, 13 op/s 2023-09-10T07:59:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:50 smithi082 ceph-mon[98694]: pgmap v16810: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 539 KiB/s wr, 13 op/s 2023-09-10T07:59:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:50 smithi098 ceph-mon[103124]: pgmap v16810: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 539 KiB/s wr, 13 op/s 2023-09-10T07:59:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:52 smithi170 ceph-mon[102832]: pgmap v16811: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 689 KiB/s wr, 20 op/s 2023-09-10T07:59:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:52 smithi082 ceph-mon[98694]: pgmap v16811: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 689 KiB/s wr, 20 op/s 2023-09-10T07:59:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:52 smithi098 ceph-mon[103124]: pgmap v16811: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 689 KiB/s wr, 20 op/s 2023-09-10T07:59:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31225.708406 secs 2023-09-10T07:59:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31225.708406 secs 2023-09-10T07:59:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31225.708406 secs 2023-09-10T07:59:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31224.542548 secs 2023-09-10T07:59:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:54 smithi170 ceph-mon[102832]: pgmap v16812: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 453 KiB/s wr, 14 op/s 2023-09-10T07:59:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:59:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:59:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31226.151488 secs 2023-09-10T07:59:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31224.542548 secs 2023-09-10T07:59:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:54 smithi082 ceph-mon[98694]: pgmap v16812: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 453 KiB/s wr, 14 op/s 2023-09-10T07:59:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:59:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:59:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31226.151488 secs 2023-09-10T07:59:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31224.542548 secs 2023-09-10T07:59:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:54 smithi098 ceph-mon[103124]: pgmap v16812: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 453 KiB/s wr, 14 op/s 2023-09-10T07:59:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T07:59:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:59:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31226.151488 secs 2023-09-10T07:59:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:59:55.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:59:55.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:59:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:59:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:59:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:59:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T07:59:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T07:59:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T07:59:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:56 smithi082 ceph-mon[98694]: pgmap v16813: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 608 KiB/s wr, 20 op/s 2023-09-10T07:59:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:56 smithi098 ceph-mon[103124]: pgmap v16813: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 608 KiB/s wr, 20 op/s 2023-09-10T07:59:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:56 smithi170 ceph-mon[102832]: pgmap v16813: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 608 KiB/s wr, 20 op/s 2023-09-10T07:59:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:58 smithi170 ceph-mon[102832]: pgmap v16814: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 305 KiB/s wr, 12 op/s 2023-09-10T07:59:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:58 smithi082 ceph-mon[98694]: pgmap v16814: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 305 KiB/s wr, 12 op/s 2023-09-10T07:59:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:58 smithi098 ceph-mon[103124]: pgmap v16814: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 305 KiB/s wr, 12 op/s 2023-09-10T07:59:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31229.542639 secs 2023-09-10T07:59:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31230.708464 secs 2023-09-10T07:59:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 07:59:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31231.151550 secs 2023-09-10T07:59:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31229.542639 secs 2023-09-10T07:59:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31230.708464 secs 2023-09-10T07:59:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 07:59:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31231.151550 secs 2023-09-10T07:59:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31229.542639 secs 2023-09-10T07:59:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31230.708464 secs 2023-09-10T07:59:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 07:59:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31231.151550 secs 2023-09-10T08:00:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:00 smithi170 ceph-mon[102832]: pgmap v16815: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 305 KiB/s wr, 12 op/s 2023-09-10T08:00:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:00 smithi170 ceph-mon[102832]: Health detail: HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T08:00:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:00 smithi170 ceph-mon[102832]: [WRN] MDS_SLOW_REQUEST: 3 MDSs report slow requests 2023-09-10T08:00:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:00 smithi170 ceph-mon[102832]: mds.l(mds.4): 3 slow requests are blocked > 180 secs 2023-09-10T08:00:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:00 smithi170 ceph-mon[102832]: mds.h(mds.1): 3 slow requests are blocked > 180 secs 2023-09-10T08:00:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:00 smithi170 ceph-mon[102832]: mds.b(mds.0): 1 slow requests are blocked > 180 secs 2023-09-10T08:00:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:00 smithi170 ceph-mon[102832]: [WRN] MDS_TRIM: 2 MDSs behind on trimming 2023-09-10T08:00:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:00 smithi170 ceph-mon[102832]: mds.e(mds.0): Behind on trimming (292/128) max_segments: 128, num_segments: 292 2023-09-10T08:00:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:00 smithi170 ceph-mon[102832]: mds.b(mds.0): Behind on trimming (292/128) max_segments: 128, num_segments: 292 2023-09-10T08:00:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:00 smithi082 ceph-mon[98694]: pgmap v16815: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 305 KiB/s wr, 12 op/s 2023-09-10T08:00:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:00 smithi082 ceph-mon[98694]: Health detail: HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T08:00:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:00 smithi082 ceph-mon[98694]: [WRN] MDS_SLOW_REQUEST: 3 MDSs report slow requests 2023-09-10T08:00:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:00 smithi082 ceph-mon[98694]: mds.l(mds.4): 3 slow requests are blocked > 180 secs 2023-09-10T08:00:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:00 smithi082 ceph-mon[98694]: mds.h(mds.1): 3 slow requests are blocked > 180 secs 2023-09-10T08:00:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:00 smithi082 ceph-mon[98694]: mds.b(mds.0): 1 slow requests are blocked > 180 secs 2023-09-10T08:00:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:00 smithi082 ceph-mon[98694]: [WRN] MDS_TRIM: 2 MDSs behind on trimming 2023-09-10T08:00:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:00 smithi082 ceph-mon[98694]: mds.e(mds.0): Behind on trimming (292/128) max_segments: 128, num_segments: 292 2023-09-10T08:00:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:00 smithi082 ceph-mon[98694]: mds.b(mds.0): Behind on trimming (292/128) max_segments: 128, num_segments: 292 2023-09-10T08:00:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:00 smithi098 ceph-mon[103124]: pgmap v16815: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 305 KiB/s wr, 12 op/s 2023-09-10T08:00:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:00 smithi098 ceph-mon[103124]: Health detail: HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T08:00:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:00 smithi098 ceph-mon[103124]: [WRN] MDS_SLOW_REQUEST: 3 MDSs report slow requests 2023-09-10T08:00:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:00 smithi098 ceph-mon[103124]: mds.l(mds.4): 3 slow requests are blocked > 180 secs 2023-09-10T08:00:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:00 smithi098 ceph-mon[103124]: mds.h(mds.1): 3 slow requests are blocked > 180 secs 2023-09-10T08:00:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:00 smithi098 ceph-mon[103124]: mds.b(mds.0): 1 slow requests are blocked > 180 secs 2023-09-10T08:00:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:00 smithi098 ceph-mon[103124]: [WRN] MDS_TRIM: 2 MDSs behind on trimming 2023-09-10T08:00:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:00 smithi098 ceph-mon[103124]: mds.e(mds.0): Behind on trimming (292/128) max_segments: 128, num_segments: 292 2023-09-10T08:00:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:00 smithi098 ceph-mon[103124]: mds.b(mds.0): Behind on trimming (292/128) max_segments: 128, num_segments: 292 2023-09-10T08:00:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:02 smithi170 ceph-mon[102832]: pgmap v16816: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 428 KiB/s wr, 17 op/s 2023-09-10T08:00:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:02 smithi082 ceph-mon[98694]: pgmap v16816: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 428 KiB/s wr, 17 op/s 2023-09-10T08:00:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:02 smithi098 ceph-mon[103124]: pgmap v16816: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 428 KiB/s wr, 17 op/s 2023-09-10T08:00:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31235.708523 secs 2023-09-10T08:00:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31235.708523 secs 2023-09-10T08:00:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31235.708523 secs 2023-09-10T08:00:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31234.542700 secs 2023-09-10T08:00:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:04 smithi170 ceph-mon[102832]: pgmap v16817: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 10 op/s 2023-09-10T08:00:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31236.151613 secs 2023-09-10T08:00:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31234.542700 secs 2023-09-10T08:00:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:04 smithi082 ceph-mon[98694]: pgmap v16817: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 10 op/s 2023-09-10T08:00:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31236.151613 secs 2023-09-10T08:00:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31234.542700 secs 2023-09-10T08:00:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:04 smithi098 ceph-mon[103124]: pgmap v16817: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 10 op/s 2023-09-10T08:00:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31236.151613 secs 2023-09-10T08:00:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:06 smithi170 ceph-mon[102832]: pgmap v16818: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 322 KiB/s wr, 30 op/s 2023-09-10T08:00:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:06 smithi082 ceph-mon[98694]: pgmap v16818: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 322 KiB/s wr, 30 op/s 2023-09-10T08:00:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:06 smithi098 ceph-mon[103124]: pgmap v16818: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 322 KiB/s wr, 30 op/s 2023-09-10T08:00:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:08 smithi170 ceph-mon[102832]: pgmap v16819: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 167 KiB/s wr, 24 op/s 2023-09-10T08:00:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:08 smithi082 ceph-mon[98694]: pgmap v16819: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 167 KiB/s wr, 24 op/s 2023-09-10T08:00:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:08 smithi098 ceph-mon[103124]: pgmap v16819: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 167 KiB/s wr, 24 op/s 2023-09-10T08:00:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31239.542759 secs 2023-09-10T08:00:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:09 smithi170 ceph-mon[102832]: osdmap e1119: 12 total, 12 up, 12 in 2023-09-10T08:00:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31240.708615 secs 2023-09-10T08:00:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31241.151676 secs 2023-09-10T08:00:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31239.542759 secs 2023-09-10T08:00:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:09 smithi082 ceph-mon[98694]: osdmap e1119: 12 total, 12 up, 12 in 2023-09-10T08:00:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31240.708615 secs 2023-09-10T08:00:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31241.151676 secs 2023-09-10T08:00:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31239.542759 secs 2023-09-10T08:00:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:09 smithi098 ceph-mon[103124]: osdmap e1119: 12 total, 12 up, 12 in 2023-09-10T08:00:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31240.708615 secs 2023-09-10T08:00:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31241.151676 secs 2023-09-10T08:00:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:10 smithi170 ceph-mon[102832]: pgmap v16821: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 201 KiB/s wr, 29 op/s 2023-09-10T08:00:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:10 smithi082 ceph-mon[98694]: pgmap v16821: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 201 KiB/s wr, 29 op/s 2023-09-10T08:00:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:10 smithi098 ceph-mon[103124]: pgmap v16821: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 201 KiB/s wr, 29 op/s 2023-09-10T08:00:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:12 smithi170 ceph-mon[102832]: pgmap v16822: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T08:00:12.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:12 smithi170 ceph-mon[102832]: osdmap e1120: 12 total, 12 up, 12 in 2023-09-10T08:00:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:12 smithi082 ceph-mon[98694]: pgmap v16822: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T08:00:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:12 smithi082 ceph-mon[98694]: osdmap e1120: 12 total, 12 up, 12 in 2023-09-10T08:00:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:12 smithi098 ceph-mon[103124]: pgmap v16822: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-09-10T08:00:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:12 smithi098 ceph-mon[103124]: osdmap e1120: 12 total, 12 up, 12 in 2023-09-10T08:00:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31245.708671 secs 2023-09-10T08:00:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31245.708671 secs 2023-09-10T08:00:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31245.708671 secs 2023-09-10T08:00:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31244.542776 secs 2023-09-10T08:00:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:14 smithi170 ceph-mon[102832]: pgmap v16824: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T08:00:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31246.151739 secs 2023-09-10T08:00:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31244.542776 secs 2023-09-10T08:00:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:14 smithi082 ceph-mon[98694]: pgmap v16824: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T08:00:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31246.151739 secs 2023-09-10T08:00:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31244.542776 secs 2023-09-10T08:00:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:14 smithi098 ceph-mon[103124]: pgmap v16824: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2023-09-10T08:00:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31246.151739 secs 2023-09-10T08:00:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:16 smithi170 ceph-mon[102832]: pgmap v16825: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T08:00:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:16 smithi082 ceph-mon[98694]: pgmap v16825: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T08:00:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:16 smithi098 ceph-mon[103124]: pgmap v16825: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-09-10T08:00:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:18 smithi170 ceph-mon[102832]: pgmap v16826: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 11 op/s 2023-09-10T08:00:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:18 smithi082 ceph-mon[98694]: pgmap v16826: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 11 op/s 2023-09-10T08:00:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:18 smithi098 ceph-mon[103124]: pgmap v16826: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 11 op/s 2023-09-10T08:00:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31249.542855 secs 2023-09-10T08:00:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31250.708730 secs 2023-09-10T08:00:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31251.151803 secs 2023-09-10T08:00:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31249.542855 secs 2023-09-10T08:00:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31250.708730 secs 2023-09-10T08:00:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31251.151803 secs 2023-09-10T08:00:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31249.542855 secs 2023-09-10T08:00:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31250.708730 secs 2023-09-10T08:00:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31251.151803 secs 2023-09-10T08:00:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:20 smithi170 ceph-mon[102832]: pgmap v16827: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T08:00:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:20 smithi082 ceph-mon[98694]: pgmap v16827: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T08:00:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:20 smithi098 ceph-mon[103124]: pgmap v16827: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2023-09-10T08:00:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:22 smithi170 ceph-mon[102832]: pgmap v16828: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T08:00:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:22 smithi082 ceph-mon[98694]: pgmap v16828: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T08:00:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:22 smithi098 ceph-mon[103124]: pgmap v16828: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2023-09-10T08:00:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31255.708787 secs 2023-09-10T08:00:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31255.708787 secs 2023-09-10T08:00:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31255.708787 secs 2023-09-10T08:00:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31254.542952 secs 2023-09-10T08:00:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:24 smithi170 ceph-mon[102832]: pgmap v16829: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T08:00:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31256.151864 secs 2023-09-10T08:00:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31254.542952 secs 2023-09-10T08:00:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:24 smithi082 ceph-mon[98694]: pgmap v16829: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T08:00:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31256.151864 secs 2023-09-10T08:00:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31254.542952 secs 2023-09-10T08:00:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:24 smithi098 ceph-mon[103124]: pgmap v16829: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T08:00:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31256.151864 secs 2023-09-10T08:00:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:26 smithi170 ceph-mon[102832]: pgmap v16830: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 105 KiB/s wr, 13 op/s 2023-09-10T08:00:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:26 smithi082 ceph-mon[98694]: pgmap v16830: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 105 KiB/s wr, 13 op/s 2023-09-10T08:00:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:26 smithi098 ceph-mon[103124]: pgmap v16830: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 105 KiB/s wr, 13 op/s 2023-09-10T08:00:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:28 smithi170 ceph-mon[102832]: pgmap v16831: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 9 op/s 2023-09-10T08:00:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:28 smithi082 ceph-mon[98694]: pgmap v16831: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 9 op/s 2023-09-10T08:00:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:28 smithi098 ceph-mon[103124]: pgmap v16831: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 9 op/s 2023-09-10T08:00:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31259.543014 secs 2023-09-10T08:00:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31260.708895 secs 2023-09-10T08:00:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31261.151926 secs 2023-09-10T08:00:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31259.543014 secs 2023-09-10T08:00:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31260.708895 secs 2023-09-10T08:00:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31261.151926 secs 2023-09-10T08:00:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31259.543014 secs 2023-09-10T08:00:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31260.708895 secs 2023-09-10T08:00:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31261.151926 secs 2023-09-10T08:00:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:30 smithi170 ceph-mon[102832]: pgmap v16832: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 9 op/s 2023-09-10T08:00:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:30 smithi082 ceph-mon[98694]: pgmap v16832: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 9 op/s 2023-09-10T08:00:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:30 smithi098 ceph-mon[103124]: pgmap v16832: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 9 op/s 2023-09-10T08:00:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:32 smithi170 ceph-mon[102832]: pgmap v16833: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 105 KiB/s wr, 14 op/s 2023-09-10T08:00:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:32 smithi082 ceph-mon[98694]: pgmap v16833: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 105 KiB/s wr, 14 op/s 2023-09-10T08:00:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:32 smithi098 ceph-mon[103124]: pgmap v16833: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 105 KiB/s wr, 14 op/s 2023-09-10T08:00:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31265.708950 secs 2023-09-10T08:00:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31265.708950 secs 2023-09-10T08:00:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31265.708950 secs 2023-09-10T08:00:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:34 smithi170 ceph-mon[102832]: pgmap v16834: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T08:00:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:00:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:00:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31266.151986 secs 2023-09-10T08:00:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:34 smithi082 ceph-mon[98694]: pgmap v16834: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T08:00:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:00:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:00:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31266.151986 secs 2023-09-10T08:00:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:34 smithi098 ceph-mon[103124]: pgmap v16834: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 10 op/s 2023-09-10T08:00:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:00:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:00:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31266.151986 secs 2023-09-10T08:00:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:00:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:00:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:00:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:36 smithi170 ceph-mon[102832]: pgmap v16835: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 430 KiB/s wr, 16 op/s 2023-09-10T08:00:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:00:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:00:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:00:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:36 smithi082 ceph-mon[98694]: pgmap v16835: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 430 KiB/s wr, 16 op/s 2023-09-10T08:00:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:00:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:00:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:00:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:36 smithi098 ceph-mon[103124]: pgmap v16835: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 430 KiB/s wr, 16 op/s 2023-09-10T08:00:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:00:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:00:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:00:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:38 smithi170 ceph-mon[102832]: pgmap v16836: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 325 KiB/s wr, 11 op/s 2023-09-10T08:00:38.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:38 smithi082 ceph-mon[98694]: pgmap v16836: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 325 KiB/s wr, 11 op/s 2023-09-10T08:00:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:38 smithi098 ceph-mon[103124]: pgmap v16836: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 325 KiB/s wr, 11 op/s 2023-09-10T08:00:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31270.709010 secs 2023-09-10T08:00:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31271.152047 secs 2023-09-10T08:00:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31270.709010 secs 2023-09-10T08:00:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31271.152047 secs 2023-09-10T08:00:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31270.709010 secs 2023-09-10T08:00:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31271.152047 secs 2023-09-10T08:00:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:40 smithi170 ceph-mon[102832]: pgmap v16837: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 325 KiB/s wr, 11 op/s 2023-09-10T08:00:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:40 smithi082 ceph-mon[98694]: pgmap v16837: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 325 KiB/s wr, 11 op/s 2023-09-10T08:00:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:40 smithi098 ceph-mon[103124]: pgmap v16837: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 325 KiB/s wr, 11 op/s 2023-09-10T08:00:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:42 smithi170 ceph-mon[102832]: pgmap v16838: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 504 KiB/s wr, 18 op/s 2023-09-10T08:00:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:42 smithi082 ceph-mon[98694]: pgmap v16838: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 504 KiB/s wr, 18 op/s 2023-09-10T08:00:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:42 smithi098 ceph-mon[103124]: pgmap v16838: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 504 KiB/s wr, 18 op/s 2023-09-10T08:00:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31275.709069 secs 2023-09-10T08:00:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31275.709069 secs 2023-09-10T08:00:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31275.709069 secs 2023-09-10T08:00:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:44 smithi170 ceph-mon[102832]: pgmap v16839: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 503 KiB/s wr, 13 op/s 2023-09-10T08:00:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:00:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:00:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31276.152113 secs 2023-09-10T08:00:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:44 smithi082 ceph-mon[98694]: pgmap v16839: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 503 KiB/s wr, 13 op/s 2023-09-10T08:00:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:00:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:00:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31276.152113 secs 2023-09-10T08:00:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:44 smithi098 ceph-mon[103124]: pgmap v16839: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 503 KiB/s wr, 13 op/s 2023-09-10T08:00:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:00:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:00:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31276.152113 secs 2023-09-10T08:00:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:46 smithi170 ceph-mon[102832]: pgmap v16840: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 786 KiB/s wr, 20 op/s 2023-09-10T08:00:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:00:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:00:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:46 smithi082 ceph-mon[98694]: pgmap v16840: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 786 KiB/s wr, 20 op/s 2023-09-10T08:00:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:00:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:00:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:46 smithi098 ceph-mon[103124]: pgmap v16840: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 786 KiB/s wr, 20 op/s 2023-09-10T08:00:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:00:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:00:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:48 smithi170 ceph-mon[102832]: pgmap v16841: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 462 KiB/s wr, 13 op/s 2023-09-10T08:00:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:48 smithi082 ceph-mon[98694]: pgmap v16841: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 462 KiB/s wr, 13 op/s 2023-09-10T08:00:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:48 smithi098 ceph-mon[103124]: pgmap v16841: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 462 KiB/s wr, 13 op/s 2023-09-10T08:00:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31280.709155 secs 2023-09-10T08:00:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:00:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31281.152177 secs 2023-09-10T08:00:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:00:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:00:49.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:00:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31280.709155 secs 2023-09-10T08:00:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:00:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31281.152177 secs 2023-09-10T08:00:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:00:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:00:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:00:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31280.709155 secs 2023-09-10T08:00:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:00:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31281.152177 secs 2023-09-10T08:00:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:00:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:00:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:00:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:00:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:00:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:50 smithi170 ceph-mon[102832]: pgmap v16842: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 462 KiB/s wr, 13 op/s 2023-09-10T08:00:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:00:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:00:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:50 smithi082 ceph-mon[98694]: pgmap v16842: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 462 KiB/s wr, 13 op/s 2023-09-10T08:00:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:00:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:00:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:50 smithi098 ceph-mon[103124]: pgmap v16842: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 462 KiB/s wr, 13 op/s 2023-09-10T08:00:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:00:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:00:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:52 smithi082 ceph-mon[98694]: pgmap v16843: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 613 KiB/s wr, 20 op/s 2023-09-10T08:00:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:00:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:00:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:52 smithi098 ceph-mon[103124]: pgmap v16843: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 613 KiB/s wr, 20 op/s 2023-09-10T08:00:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:00:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:00:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:52 smithi170 ceph-mon[102832]: pgmap v16843: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 613 KiB/s wr, 20 op/s 2023-09-10T08:00:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31285.709212 secs 2023-09-10T08:00:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31285.709212 secs 2023-09-10T08:00:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31285.709212 secs 2023-09-10T08:00:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31284.543379 secs 2023-09-10T08:00:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:54 smithi170 ceph-mon[102832]: pgmap v16844: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 434 KiB/s wr, 14 op/s 2023-09-10T08:00:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31286.152240 secs 2023-09-10T08:00:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31284.543379 secs 2023-09-10T08:00:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:54 smithi082 ceph-mon[98694]: pgmap v16844: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 434 KiB/s wr, 14 op/s 2023-09-10T08:00:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31286.152240 secs 2023-09-10T08:00:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31284.543379 secs 2023-09-10T08:00:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:54 smithi098 ceph-mon[103124]: pgmap v16844: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 434 KiB/s wr, 14 op/s 2023-09-10T08:00:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31286.152240 secs 2023-09-10T08:00:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:00:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:00:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:00:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:00:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:00:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:00:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:00:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:00:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:00:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:56 smithi170 ceph-mon[102832]: pgmap v16845: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 589 KiB/s wr, 20 op/s 2023-09-10T08:00:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:56 smithi082 ceph-mon[98694]: pgmap v16845: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 589 KiB/s wr, 20 op/s 2023-09-10T08:00:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:56 smithi098 ceph-mon[103124]: pgmap v16845: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 589 KiB/s wr, 20 op/s 2023-09-10T08:00:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:58 smithi170 ceph-mon[102832]: pgmap v16846: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 306 KiB/s wr, 13 op/s 2023-09-10T08:00:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:58 smithi082 ceph-mon[98694]: pgmap v16846: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 306 KiB/s wr, 13 op/s 2023-09-10T08:00:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:58 smithi098 ceph-mon[103124]: pgmap v16846: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 306 KiB/s wr, 13 op/s 2023-09-10T08:00:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31289.543451 secs 2023-09-10T08:00:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31290.709273 secs 2023-09-10T08:00:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:00:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31291.152300 secs 2023-09-10T08:00:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31289.543451 secs 2023-09-10T08:00:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31290.709273 secs 2023-09-10T08:00:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:00:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31291.152300 secs 2023-09-10T08:00:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31289.543451 secs 2023-09-10T08:00:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31290.709273 secs 2023-09-10T08:00:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:00:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31291.152300 secs 2023-09-10T08:01:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:00 smithi170 ceph-mon[102832]: pgmap v16847: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 306 KiB/s wr, 13 op/s 2023-09-10T08:01:00.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:00 smithi082 ceph-mon[98694]: pgmap v16847: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 306 KiB/s wr, 13 op/s 2023-09-10T08:01:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:00 smithi098 ceph-mon[103124]: pgmap v16847: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 306 KiB/s wr, 13 op/s 2023-09-10T08:01:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:02 smithi170 ceph-mon[102832]: pgmap v16848: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 429 KiB/s wr, 18 op/s 2023-09-10T08:01:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:02 smithi082 ceph-mon[98694]: pgmap v16848: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 429 KiB/s wr, 18 op/s 2023-09-10T08:01:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:02 smithi098 ceph-mon[103124]: pgmap v16848: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 429 KiB/s wr, 18 op/s 2023-09-10T08:01:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31295.709329 secs 2023-09-10T08:01:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31295.709329 secs 2023-09-10T08:01:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31295.709329 secs 2023-09-10T08:01:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31294.543510 secs 2023-09-10T08:01:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:04 smithi170 ceph-mon[102832]: pgmap v16849: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 278 KiB/s wr, 11 op/s 2023-09-10T08:01:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:01:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31296.152363 secs 2023-09-10T08:01:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31294.543510 secs 2023-09-10T08:01:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:04 smithi082 ceph-mon[98694]: pgmap v16849: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 278 KiB/s wr, 11 op/s 2023-09-10T08:01:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:01:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31296.152363 secs 2023-09-10T08:01:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31294.543510 secs 2023-09-10T08:01:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:04 smithi098 ceph-mon[103124]: pgmap v16849: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 278 KiB/s wr, 11 op/s 2023-09-10T08:01:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:01:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31296.152363 secs 2023-09-10T08:01:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:06 smithi170 ceph-mon[102832]: pgmap v16850: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 323 KiB/s wr, 31 op/s 2023-09-10T08:01:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:01:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:06 smithi082 ceph-mon[98694]: pgmap v16850: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 323 KiB/s wr, 31 op/s 2023-09-10T08:01:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:01:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:06 smithi098 ceph-mon[103124]: pgmap v16850: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 323 KiB/s wr, 31 op/s 2023-09-10T08:01:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:01:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:08 smithi170 ceph-mon[102832]: pgmap v16851: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 168 KiB/s wr, 25 op/s 2023-09-10T08:01:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:08 smithi082 ceph-mon[98694]: pgmap v16851: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 168 KiB/s wr, 25 op/s 2023-09-10T08:01:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:08 smithi098 ceph-mon[103124]: pgmap v16851: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 168 KiB/s wr, 25 op/s 2023-09-10T08:01:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31299.543572 secs 2023-09-10T08:01:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:09 smithi170 ceph-mon[102832]: osdmap e1121: 12 total, 12 up, 12 in 2023-09-10T08:01:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31300.709389 secs 2023-09-10T08:01:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31301.152428 secs 2023-09-10T08:01:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31299.543572 secs 2023-09-10T08:01:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:09 smithi082 ceph-mon[98694]: osdmap e1121: 12 total, 12 up, 12 in 2023-09-10T08:01:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31300.709389 secs 2023-09-10T08:01:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31301.152428 secs 2023-09-10T08:01:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31299.543572 secs 2023-09-10T08:01:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:09 smithi098 ceph-mon[103124]: osdmap e1121: 12 total, 12 up, 12 in 2023-09-10T08:01:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31300.709389 secs 2023-09-10T08:01:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31301.152428 secs 2023-09-10T08:01:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:10 smithi170 ceph-mon[102832]: pgmap v16853: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 201 KiB/s wr, 30 op/s 2023-09-10T08:01:10.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:10 smithi082 ceph-mon[98694]: pgmap v16853: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 201 KiB/s wr, 30 op/s 2023-09-10T08:01:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:10 smithi098 ceph-mon[103124]: pgmap v16853: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 201 KiB/s wr, 30 op/s 2023-09-10T08:01:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:12 smithi170 ceph-mon[102832]: pgmap v16854: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-09-10T08:01:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:12 smithi170 ceph-mon[102832]: osdmap e1122: 12 total, 12 up, 12 in 2023-09-10T08:01:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:12 smithi082 ceph-mon[98694]: pgmap v16854: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-09-10T08:01:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:12 smithi082 ceph-mon[98694]: osdmap e1122: 12 total, 12 up, 12 in 2023-09-10T08:01:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:12 smithi098 ceph-mon[103124]: pgmap v16854: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-09-10T08:01:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:12 smithi098 ceph-mon[103124]: osdmap e1122: 12 total, 12 up, 12 in 2023-09-10T08:01:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31305.709448 secs 2023-09-10T08:01:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31305.709448 secs 2023-09-10T08:01:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31305.709448 secs 2023-09-10T08:01:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31304.543642 secs 2023-09-10T08:01:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:14 smithi170 ceph-mon[102832]: pgmap v16856: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T08:01:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31306.152485 secs 2023-09-10T08:01:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31304.543642 secs 2023-09-10T08:01:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:14 smithi082 ceph-mon[98694]: pgmap v16856: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T08:01:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31306.152485 secs 2023-09-10T08:01:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31304.543642 secs 2023-09-10T08:01:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:14 smithi098 ceph-mon[103124]: pgmap v16856: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T08:01:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31306.152485 secs 2023-09-10T08:01:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:16 smithi170 ceph-mon[102832]: pgmap v16857: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 14 op/s 2023-09-10T08:01:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:16 smithi082 ceph-mon[98694]: pgmap v16857: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 14 op/s 2023-09-10T08:01:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:16 smithi098 ceph-mon[103124]: pgmap v16857: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 14 op/s 2023-09-10T08:01:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:18 smithi170 ceph-mon[102832]: pgmap v16858: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 12 op/s 2023-09-10T08:01:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:18 smithi082 ceph-mon[98694]: pgmap v16858: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 12 op/s 2023-09-10T08:01:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:18 smithi098 ceph-mon[103124]: pgmap v16858: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 12 op/s 2023-09-10T08:01:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31309.543700 secs 2023-09-10T08:01:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31310.709508 secs 2023-09-10T08:01:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31311.152548 secs 2023-09-10T08:01:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31309.543700 secs 2023-09-10T08:01:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31310.709508 secs 2023-09-10T08:01:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31311.152548 secs 2023-09-10T08:01:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31309.543700 secs 2023-09-10T08:01:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31310.709508 secs 2023-09-10T08:01:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31311.152548 secs 2023-09-10T08:01:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:20 smithi170 ceph-mon[102832]: pgmap v16859: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T08:01:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:20 smithi082 ceph-mon[98694]: pgmap v16859: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T08:01:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:20 smithi098 ceph-mon[103124]: pgmap v16859: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T08:01:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:22 smithi170 ceph-mon[102832]: pgmap v16860: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T08:01:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:22 smithi082 ceph-mon[98694]: pgmap v16860: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T08:01:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:22 smithi098 ceph-mon[103124]: pgmap v16860: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T08:01:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31315.709566 secs 2023-09-10T08:01:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31315.709566 secs 2023-09-10T08:01:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31315.709566 secs 2023-09-10T08:01:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31314.543765 secs 2023-09-10T08:01:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:24 smithi170 ceph-mon[102832]: pgmap v16861: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T08:01:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31316.152609 secs 2023-09-10T08:01:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31314.543765 secs 2023-09-10T08:01:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:24 smithi082 ceph-mon[98694]: pgmap v16861: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T08:01:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31316.152609 secs 2023-09-10T08:01:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31314.543765 secs 2023-09-10T08:01:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:24 smithi098 ceph-mon[103124]: pgmap v16861: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T08:01:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31316.152609 secs 2023-09-10T08:01:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:26 smithi082 ceph-mon[98694]: pgmap v16862: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 14 op/s 2023-09-10T08:01:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:26 smithi098 ceph-mon[103124]: pgmap v16862: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 14 op/s 2023-09-10T08:01:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:26 smithi170 ceph-mon[102832]: pgmap v16862: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 14 op/s 2023-09-10T08:01:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:28 smithi082 ceph-mon[98694]: pgmap v16863: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2023-09-10T08:01:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:28 smithi098 ceph-mon[103124]: pgmap v16863: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2023-09-10T08:01:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:28 smithi170 ceph-mon[102832]: pgmap v16863: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2023-09-10T08:01:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31319.543775 secs 2023-09-10T08:01:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31320.709626 secs 2023-09-10T08:01:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31321.152670 secs 2023-09-10T08:01:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31319.543775 secs 2023-09-10T08:01:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31320.709626 secs 2023-09-10T08:01:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31321.152670 secs 2023-09-10T08:01:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31319.543775 secs 2023-09-10T08:01:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31320.709626 secs 2023-09-10T08:01:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31321.152670 secs 2023-09-10T08:01:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:30 smithi082 ceph-mon[98694]: pgmap v16864: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2023-09-10T08:01:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:30 smithi098 ceph-mon[103124]: pgmap v16864: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2023-09-10T08:01:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:30 smithi170 ceph-mon[102832]: pgmap v16864: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2023-09-10T08:01:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:32 smithi170 ceph-mon[102832]: pgmap v16865: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 111 KiB/s wr, 14 op/s 2023-09-10T08:01:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:32 smithi082 ceph-mon[98694]: pgmap v16865: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 111 KiB/s wr, 14 op/s 2023-09-10T08:01:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:32 smithi098 ceph-mon[103124]: pgmap v16865: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 111 KiB/s wr, 14 op/s 2023-09-10T08:01:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31325.709684 secs 2023-09-10T08:01:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31325.709684 secs 2023-09-10T08:01:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31325.709684 secs 2023-09-10T08:01:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:34 smithi082 ceph-mon[98694]: pgmap v16866: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 9 op/s 2023-09-10T08:01:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31326.152731 secs 2023-09-10T08:01:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:34 smithi098 ceph-mon[103124]: pgmap v16866: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 9 op/s 2023-09-10T08:01:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31326.152731 secs 2023-09-10T08:01:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:34 smithi170 ceph-mon[102832]: pgmap v16866: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 9 op/s 2023-09-10T08:01:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31326.152731 secs 2023-09-10T08:01:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:01:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:01:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:01:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:36 smithi082 ceph-mon[98694]: pgmap v16867: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 230 KiB/s wr, 14 op/s 2023-09-10T08:01:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:01:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:01:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:36 smithi098 ceph-mon[103124]: pgmap v16867: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 230 KiB/s wr, 14 op/s 2023-09-10T08:01:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:01:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:01:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:36 smithi170 ceph-mon[102832]: pgmap v16867: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 230 KiB/s wr, 14 op/s 2023-09-10T08:01:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:01:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:01:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:38 smithi170 ceph-mon[102832]: pgmap v16868: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 229 KiB/s wr, 9 op/s 2023-09-10T08:01:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:38 smithi082 ceph-mon[98694]: pgmap v16868: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 229 KiB/s wr, 9 op/s 2023-09-10T08:01:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:38 smithi098 ceph-mon[103124]: pgmap v16868: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 229 KiB/s wr, 9 op/s 2023-09-10T08:01:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31330.709742 secs 2023-09-10T08:01:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31331.152793 secs 2023-09-10T08:01:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31330.709742 secs 2023-09-10T08:01:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31331.152793 secs 2023-09-10T08:01:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31330.709742 secs 2023-09-10T08:01:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31331.152793 secs 2023-09-10T08:01:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:01:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:40 smithi082 ceph-mon[98694]: pgmap v16869: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 229 KiB/s wr, 9 op/s 2023-09-10T08:01:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:01:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:40 smithi098 ceph-mon[103124]: pgmap v16869: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 229 KiB/s wr, 9 op/s 2023-09-10T08:01:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:01:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:40 smithi170 ceph-mon[102832]: pgmap v16869: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 229 KiB/s wr, 9 op/s 2023-09-10T08:01:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:42 smithi170 ceph-mon[102832]: pgmap v16870: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 564 KiB/s wr, 24 op/s 2023-09-10T08:01:42.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:42 smithi082 ceph-mon[98694]: pgmap v16870: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 564 KiB/s wr, 24 op/s 2023-09-10T08:01:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:42 smithi098 ceph-mon[103124]: pgmap v16870: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 564 KiB/s wr, 24 op/s 2023-09-10T08:01:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31335.709802 secs 2023-09-10T08:01:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31335.709802 secs 2023-09-10T08:01:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31335.709802 secs 2023-09-10T08:01:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:44 smithi082 ceph-mon[98694]: pgmap v16871: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 453 KiB/s wr, 20 op/s 2023-09-10T08:01:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:01:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31336.152855 secs 2023-09-10T08:01:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:44 smithi098 ceph-mon[103124]: pgmap v16871: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 453 KiB/s wr, 20 op/s 2023-09-10T08:01:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:01:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31336.152855 secs 2023-09-10T08:01:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:44 smithi170 ceph-mon[102832]: pgmap v16871: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 453 KiB/s wr, 20 op/s 2023-09-10T08:01:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:01:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31336.152855 secs 2023-09-10T08:01:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:46 smithi082 ceph-mon[98694]: pgmap v16872: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 29 op/s 2023-09-10T08:01:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:01:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:46 smithi098 ceph-mon[103124]: pgmap v16872: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 29 op/s 2023-09-10T08:01:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:01:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:46 smithi170 ceph-mon[102832]: pgmap v16872: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 29 op/s 2023-09-10T08:01:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:01:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:48 smithi082 ceph-mon[98694]: pgmap v16873: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 24 op/s 2023-09-10T08:01:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:48 smithi098 ceph-mon[103124]: pgmap v16873: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 24 op/s 2023-09-10T08:01:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:48 smithi170 ceph-mon[102832]: pgmap v16873: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 24 op/s 2023-09-10T08:01:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:49 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31339.544016 secs 2023-09-10T08:01:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31340.709885 secs 2023-09-10T08:01:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31341.152916 secs 2023-09-10T08:01:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:01:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:49 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31339.544016 secs 2023-09-10T08:01:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31340.709885 secs 2023-09-10T08:01:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31341.152916 secs 2023-09-10T08:01:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:01:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:49 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31339.544016 secs 2023-09-10T08:01:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31340.709885 secs 2023-09-10T08:01:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31341.152916 secs 2023-09-10T08:01:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:01:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:01:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:50 smithi082 ceph-mon[98694]: pgmap v16874: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 24 op/s 2023-09-10T08:01:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:01:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:01:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:01:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:01:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:50 smithi098 ceph-mon[103124]: pgmap v16874: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 24 op/s 2023-09-10T08:01:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:01:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:01:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:01:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:01:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:50 smithi170 ceph-mon[102832]: pgmap v16874: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 24 op/s 2023-09-10T08:01:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:01:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:01:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:01:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:01:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:01:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:01:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:01:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:01:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:01:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:52 smithi098 ceph-mon[103124]: pgmap v16875: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 35 op/s 2023-09-10T08:01:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:52 smithi170 ceph-mon[102832]: pgmap v16875: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 35 op/s 2023-09-10T08:01:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:52 smithi082 ceph-mon[98694]: pgmap v16875: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 35 op/s 2023-09-10T08:01:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31345.709942 secs 2023-09-10T08:01:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:01:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:54 smithi170 ceph-mon[102832]: pgmap v16876: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 20 op/s 2023-09-10T08:01:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:54 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 3 slow requests are blocked > 180 secs 2023-09-10T08:01:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31345.709942 secs 2023-09-10T08:01:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:01:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:54 smithi082 ceph-mon[98694]: pgmap v16876: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 20 op/s 2023-09-10T08:01:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:54 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 3 slow requests are blocked > 180 secs 2023-09-10T08:01:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31345.709942 secs 2023-09-10T08:01:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:01:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:54 smithi098 ceph-mon[103124]: pgmap v16876: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 20 op/s 2023-09-10T08:01:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:54 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 3 slow requests are blocked > 180 secs 2023-09-10T08:01:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-10T08:01:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:55 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:01:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-10T08:01:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:55 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:01:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-10T08:01:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:55 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:01:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:01:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:56 smithi170 ceph-mon[102832]: pgmap v16877: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 31 op/s 2023-09-10T08:01:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:01:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:56 smithi082 ceph-mon[98694]: pgmap v16877: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 31 op/s 2023-09-10T08:01:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:01:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:56 smithi098 ceph-mon[103124]: pgmap v16877: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 31 op/s 2023-09-10T08:01:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:57 smithi170 ceph-mon[102832]: Health check update: 1 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:01:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:57 smithi082 ceph-mon[98694]: Health check update: 1 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:01:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:57 smithi098 ceph-mon[103124]: Health check update: 1 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:01:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:01:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:58 smithi170 ceph-mon[102832]: pgmap v16878: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 21 op/s 2023-09-10T08:01:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:01:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:58 smithi082 ceph-mon[98694]: pgmap v16878: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 21 op/s 2023-09-10T08:01:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:01:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:58 smithi098 ceph-mon[103124]: pgmap v16878: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 21 op/s 2023-09-10T08:01:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31350.710004 secs 2023-09-10T08:01:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:59 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:01:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31351.153040 secs 2023-09-10T08:01:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31350.710004 secs 2023-09-10T08:01:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:59 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:01:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:01:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31351.153040 secs 2023-09-10T08:02:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31350.710004 secs 2023-09-10T08:02:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:59 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:01:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31351.153040 secs 2023-09-10T08:02:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:02:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:00 smithi170 ceph-mon[102832]: pgmap v16879: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 21 op/s 2023-09-10T08:02:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:02:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:00 smithi082 ceph-mon[98694]: pgmap v16879: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 21 op/s 2023-09-10T08:02:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:02:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:00 smithi098 ceph-mon[103124]: pgmap v16879: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 21 op/s 2023-09-10T08:02:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:02 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:02 smithi170 ceph-mon[102832]: pgmap v16880: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 31 op/s 2023-09-10T08:02:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:02 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:02 smithi082 ceph-mon[98694]: pgmap v16880: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 31 op/s 2023-09-10T08:02:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:02 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:02 smithi098 ceph-mon[103124]: pgmap v16880: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 31 op/s 2023-09-10T08:02:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:03 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:02:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:03 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-10T08:02:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:03 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:03 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:02:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:03 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-10T08:02:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:03 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:03 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:02:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:03 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6836/2019449840,v1:172.21.15.98:6837/2019449840] up:active 2023-09-10T08:02:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:03 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31355.710089 secs 2023-09-10T08:02:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:02:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:04 smithi170 ceph-mon[102832]: osdmap e1123: 12 total, 12 up, 12 in 2023-09-10T08:02:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:04 smithi170 ceph-mon[102832]: pgmap v16882: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 24 op/s 2023-09-10T08:02:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31356.153106 secs 2023-09-10T08:02:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31355.710089 secs 2023-09-10T08:02:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:02:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:04 smithi082 ceph-mon[98694]: osdmap e1123: 12 total, 12 up, 12 in 2023-09-10T08:02:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:04 smithi082 ceph-mon[98694]: pgmap v16882: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 24 op/s 2023-09-10T08:02:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31356.153106 secs 2023-09-10T08:02:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31355.710089 secs 2023-09-10T08:02:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:02:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:04 smithi098 ceph-mon[103124]: osdmap e1123: 12 total, 12 up, 12 in 2023-09-10T08:02:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:04 smithi098 ceph-mon[103124]: pgmap v16882: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 24 op/s 2023-09-10T08:02:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31356.153106 secs 2023-09-10T08:02:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:02:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:06 smithi170 ceph-mon[102832]: pgmap v16883: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 41 op/s 2023-09-10T08:02:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:02:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:06 smithi082 ceph-mon[98694]: pgmap v16883: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 41 op/s 2023-09-10T08:02:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:02:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:06 smithi098 ceph-mon[103124]: pgmap v16883: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 41 op/s 2023-09-10T08:02:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:08 smithi082 ceph-mon[98694]: osdmap e1124: 12 total, 12 up, 12 in 2023-09-10T08:02:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:08 smithi082 ceph-mon[98694]: pgmap v16885: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 52 op/s 2023-09-10T08:02:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:08 smithi098 ceph-mon[103124]: osdmap e1124: 12 total, 12 up, 12 in 2023-09-10T08:02:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:08 smithi098 ceph-mon[103124]: pgmap v16885: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 52 op/s 2023-09-10T08:02:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:08 smithi170 ceph-mon[102832]: osdmap e1124: 12 total, 12 up, 12 in 2023-09-10T08:02:08.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:08 smithi170 ceph-mon[102832]: pgmap v16885: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 52 op/s 2023-09-10T08:02:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31359.544308 secs 2023-09-10T08:02:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31360.710147 secs 2023-09-10T08:02:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31361.153168 secs 2023-09-10T08:02:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31359.544308 secs 2023-09-10T08:02:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31360.710147 secs 2023-09-10T08:02:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31361.153168 secs 2023-09-10T08:02:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31359.544308 secs 2023-09-10T08:02:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31360.710147 secs 2023-09-10T08:02:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31361.153168 secs 2023-09-10T08:02:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:02:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:10 smithi082 ceph-mon[98694]: pgmap v16886: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 778 KiB/s wr, 36 op/s 2023-09-10T08:02:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:02:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:10 smithi098 ceph-mon[103124]: pgmap v16886: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 778 KiB/s wr, 36 op/s 2023-09-10T08:02:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:02:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:10 smithi170 ceph-mon[102832]: pgmap v16886: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 778 KiB/s wr, 36 op/s 2023-09-10T08:02:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:11 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:02:11.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:11 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:02:11.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:11 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:11 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:02:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:11 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:02:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:11 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:11 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:02:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:11 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:02:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:11 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:12 smithi170 ceph-mon[102832]: pgmap v16887: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 904 KiB/s wr, 48 op/s 2023-09-10T08:02:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:12 smithi082 ceph-mon[98694]: pgmap v16887: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 904 KiB/s wr, 48 op/s 2023-09-10T08:02:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:12 smithi098 ceph-mon[103124]: pgmap v16887: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 904 KiB/s wr, 48 op/s 2023-09-10T08:02:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31365.710236 secs 2023-09-10T08:02:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31365.710236 secs 2023-09-10T08:02:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31365.710236 secs 2023-09-10T08:02:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31364.544372 secs 2023-09-10T08:02:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:14 smithi170 ceph-mon[102832]: pgmap v16888: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 729 KiB/s wr, 38 op/s 2023-09-10T08:02:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31366.153229 secs 2023-09-10T08:02:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31364.544372 secs 2023-09-10T08:02:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:14 smithi082 ceph-mon[98694]: pgmap v16888: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 729 KiB/s wr, 38 op/s 2023-09-10T08:02:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31366.153229 secs 2023-09-10T08:02:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31364.544372 secs 2023-09-10T08:02:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:14 smithi098 ceph-mon[103124]: pgmap v16888: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 729 KiB/s wr, 38 op/s 2023-09-10T08:02:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31366.153229 secs 2023-09-10T08:02:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:16 smithi082 ceph-mon[98694]: pgmap v16889: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 107 KiB/s wr, 17 op/s 2023-09-10T08:02:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:16 smithi098 ceph-mon[103124]: pgmap v16889: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 107 KiB/s wr, 17 op/s 2023-09-10T08:02:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:16 smithi170 ceph-mon[102832]: pgmap v16889: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 107 KiB/s wr, 17 op/s 2023-09-10T08:02:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:18 smithi082 ceph-mon[98694]: pgmap v16890: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 105 KiB/s wr, 16 op/s 2023-09-10T08:02:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:18 smithi098 ceph-mon[103124]: pgmap v16890: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 105 KiB/s wr, 16 op/s 2023-09-10T08:02:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:18 smithi170 ceph-mon[102832]: pgmap v16890: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 105 KiB/s wr, 16 op/s 2023-09-10T08:02:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31369.544488 secs 2023-09-10T08:02:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31370.710325 secs 2023-09-10T08:02:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31371.153289 secs 2023-09-10T08:02:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31369.544488 secs 2023-09-10T08:02:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31370.710325 secs 2023-09-10T08:02:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31371.153289 secs 2023-09-10T08:02:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31369.544488 secs 2023-09-10T08:02:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31370.710325 secs 2023-09-10T08:02:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31371.153289 secs 2023-09-10T08:02:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:20 smithi082 ceph-mon[98694]: pgmap v16891: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 14 op/s 2023-09-10T08:02:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:20 smithi098 ceph-mon[103124]: pgmap v16891: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 14 op/s 2023-09-10T08:02:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:20 smithi170 ceph-mon[102832]: pgmap v16891: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 14 op/s 2023-09-10T08:02:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:22 smithi082 ceph-mon[98694]: pgmap v16892: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 195 KiB/s wr, 19 op/s 2023-09-10T08:02:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:22 smithi098 ceph-mon[103124]: pgmap v16892: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 195 KiB/s wr, 19 op/s 2023-09-10T08:02:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:22 smithi170 ceph-mon[102832]: pgmap v16892: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 195 KiB/s wr, 19 op/s 2023-09-10T08:02:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31374.544558 secs 2023-09-10T08:02:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31375.710406 secs 2023-09-10T08:02:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31374.544558 secs 2023-09-10T08:02:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31375.710406 secs 2023-09-10T08:02:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31374.544558 secs 2023-09-10T08:02:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31375.710406 secs 2023-09-10T08:02:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:24 smithi082 ceph-mon[98694]: pgmap v16893: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T08:02:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31376.153354 secs 2023-09-10T08:02:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:24 smithi098 ceph-mon[103124]: pgmap v16893: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T08:02:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31376.153354 secs 2023-09-10T08:02:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:24 smithi170 ceph-mon[102832]: pgmap v16893: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 107 KiB/s wr, 11 op/s 2023-09-10T08:02:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31376.153354 secs 2023-09-10T08:02:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:26 smithi082 ceph-mon[98694]: pgmap v16894: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 107 KiB/s wr, 16 op/s 2023-09-10T08:02:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:26 smithi098 ceph-mon[103124]: pgmap v16894: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 107 KiB/s wr, 16 op/s 2023-09-10T08:02:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:26 smithi170 ceph-mon[102832]: pgmap v16894: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 107 KiB/s wr, 16 op/s 2023-09-10T08:02:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:28 smithi082 ceph-mon[98694]: pgmap v16895: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 9 op/s 2023-09-10T08:02:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:28 smithi098 ceph-mon[103124]: pgmap v16895: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 9 op/s 2023-09-10T08:02:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:28 smithi170 ceph-mon[102832]: pgmap v16895: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 9 op/s 2023-09-10T08:02:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31379.544617 secs 2023-09-10T08:02:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31380.710469 secs 2023-09-10T08:02:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31381.153417 secs 2023-09-10T08:02:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31379.544617 secs 2023-09-10T08:02:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31380.710469 secs 2023-09-10T08:02:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31381.153417 secs 2023-09-10T08:02:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31379.544617 secs 2023-09-10T08:02:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31380.710469 secs 2023-09-10T08:02:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31381.153417 secs 2023-09-10T08:02:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:30 smithi082 ceph-mon[98694]: pgmap v16896: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 9 op/s 2023-09-10T08:02:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:30 smithi098 ceph-mon[103124]: pgmap v16896: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 9 op/s 2023-09-10T08:02:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:30 smithi170 ceph-mon[102832]: pgmap v16896: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 9 op/s 2023-09-10T08:02:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:32 smithi082 ceph-mon[98694]: pgmap v16897: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 207 KiB/s wr, 15 op/s 2023-09-10T08:02:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:32 smithi098 ceph-mon[103124]: pgmap v16897: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 207 KiB/s wr, 15 op/s 2023-09-10T08:02:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:32 smithi170 ceph-mon[102832]: pgmap v16897: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 207 KiB/s wr, 15 op/s 2023-09-10T08:02:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31385.710496 secs 2023-09-10T08:02:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31385.710496 secs 2023-09-10T08:02:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31385.710496 secs 2023-09-10T08:02:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:34 smithi082 ceph-mon[98694]: pgmap v16898: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 10 op/s 2023-09-10T08:02:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:02:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31386.153481 secs 2023-09-10T08:02:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:34 smithi098 ceph-mon[103124]: pgmap v16898: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 10 op/s 2023-09-10T08:02:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:02:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31386.153481 secs 2023-09-10T08:02:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:34 smithi170 ceph-mon[102832]: pgmap v16898: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 101 KiB/s wr, 10 op/s 2023-09-10T08:02:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:02:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31386.153481 secs 2023-09-10T08:02:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:02:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:02:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:02:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:36 smithi082 ceph-mon[98694]: pgmap v16899: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 243 KiB/s wr, 16 op/s 2023-09-10T08:02:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:02:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:02:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:36 smithi098 ceph-mon[103124]: pgmap v16899: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 243 KiB/s wr, 16 op/s 2023-09-10T08:02:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:02:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:02:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:36 smithi170 ceph-mon[102832]: pgmap v16899: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 243 KiB/s wr, 16 op/s 2023-09-10T08:02:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:02:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:02:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:38 smithi082 ceph-mon[98694]: pgmap v16900: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 243 KiB/s wr, 11 op/s 2023-09-10T08:02:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:38 smithi098 ceph-mon[103124]: pgmap v16900: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 243 KiB/s wr, 11 op/s 2023-09-10T08:02:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:38 smithi170 ceph-mon[102832]: pgmap v16900: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 243 KiB/s wr, 11 op/s 2023-09-10T08:02:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31390.710554 secs 2023-09-10T08:02:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31391.153542 secs 2023-09-10T08:02:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31390.710554 secs 2023-09-10T08:02:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31391.153542 secs 2023-09-10T08:02:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31390.710554 secs 2023-09-10T08:02:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31391.153542 secs 2023-09-10T08:02:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:40 smithi082 ceph-mon[98694]: pgmap v16901: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 243 KiB/s wr, 11 op/s 2023-09-10T08:02:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:40 smithi098 ceph-mon[103124]: pgmap v16901: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 243 KiB/s wr, 11 op/s 2023-09-10T08:02:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:40 smithi170 ceph-mon[102832]: pgmap v16901: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 243 KiB/s wr, 11 op/s 2023-09-10T08:02:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:42 smithi082 ceph-mon[98694]: pgmap v16902: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 492 KiB/s wr, 17 op/s 2023-09-10T08:02:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:42 smithi098 ceph-mon[103124]: pgmap v16902: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 492 KiB/s wr, 17 op/s 2023-09-10T08:02:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:42 smithi170 ceph-mon[102832]: pgmap v16902: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 492 KiB/s wr, 17 op/s 2023-09-10T08:02:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31395.710617 secs 2023-09-10T08:02:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31395.710617 secs 2023-09-10T08:02:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31395.710617 secs 2023-09-10T08:02:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:44 smithi082 ceph-mon[98694]: pgmap v16903: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 391 KiB/s wr, 12 op/s 2023-09-10T08:02:44.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:02:44.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:44.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31396.153608 secs 2023-09-10T08:02:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:44 smithi098 ceph-mon[103124]: pgmap v16903: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 391 KiB/s wr, 12 op/s 2023-09-10T08:02:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:02:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31396.153608 secs 2023-09-10T08:02:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:44 smithi170 ceph-mon[102832]: pgmap v16903: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 391 KiB/s wr, 12 op/s 2023-09-10T08:02:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:02:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31396.153608 secs 2023-09-10T08:02:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:46 smithi098 ceph-mon[103124]: pgmap v16904: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 663 KiB/s wr, 20 op/s 2023-09-10T08:02:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:02:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:46 smithi170 ceph-mon[102832]: pgmap v16904: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 663 KiB/s wr, 20 op/s 2023-09-10T08:02:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:02:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:46 smithi082 ceph-mon[98694]: pgmap v16904: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 663 KiB/s wr, 20 op/s 2023-09-10T08:02:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:02:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:48 smithi082 ceph-mon[98694]: pgmap v16905: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 521 KiB/s wr, 14 op/s 2023-09-10T08:02:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:48 smithi098 ceph-mon[103124]: pgmap v16905: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 521 KiB/s wr, 14 op/s 2023-09-10T08:02:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:48 smithi170 ceph-mon[102832]: pgmap v16905: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 521 KiB/s wr, 14 op/s 2023-09-10T08:02:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31400.710677 secs 2023-09-10T08:02:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31401.153667 secs 2023-09-10T08:02:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31400.710677 secs 2023-09-10T08:02:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31401.153667 secs 2023-09-10T08:02:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31400.710677 secs 2023-09-10T08:02:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31401.153667 secs 2023-09-10T08:02:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:50 smithi082 ceph-mon[98694]: pgmap v16906: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 521 KiB/s wr, 14 op/s 2023-09-10T08:02:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:02:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:50 smithi098 ceph-mon[103124]: pgmap v16906: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 521 KiB/s wr, 14 op/s 2023-09-10T08:02:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:02:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:50 smithi170 ceph-mon[102832]: pgmap v16906: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 521 KiB/s wr, 14 op/s 2023-09-10T08:02:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:02:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:02:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:02:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:02:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:02:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:02:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:02:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:02:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:02:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:02:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:52 smithi098 ceph-mon[103124]: pgmap v16907: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 669 KiB/s wr, 21 op/s 2023-09-10T08:02:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:52 smithi170 ceph-mon[102832]: pgmap v16907: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 669 KiB/s wr, 21 op/s 2023-09-10T08:02:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:52 smithi082 ceph-mon[98694]: pgmap v16907: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 669 KiB/s wr, 21 op/s 2023-09-10T08:02:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31404.544894 secs 2023-09-10T08:02:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31405.710735 secs 2023-09-10T08:02:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:02:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:54 smithi098 ceph-mon[103124]: pgmap v16908: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 420 KiB/s wr, 14 op/s 2023-09-10T08:02:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31406.153732 secs 2023-09-10T08:02:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31404.544894 secs 2023-09-10T08:02:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31405.710735 secs 2023-09-10T08:02:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:02:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:54 smithi170 ceph-mon[102832]: pgmap v16908: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 420 KiB/s wr, 14 op/s 2023-09-10T08:02:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31406.153732 secs 2023-09-10T08:02:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31404.544894 secs 2023-09-10T08:02:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31405.710735 secs 2023-09-10T08:02:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:02:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:54 smithi082 ceph-mon[98694]: pgmap v16908: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 420 KiB/s wr, 14 op/s 2023-09-10T08:02:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31406.153732 secs 2023-09-10T08:02:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:02:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:02:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:02:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:02:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:02:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:02:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:02:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:56 smithi170 ceph-mon[102832]: pgmap v16909: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 573 KiB/s wr, 21 op/s 2023-09-10T08:02:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:56 smithi082 ceph-mon[98694]: pgmap v16909: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 573 KiB/s wr, 21 op/s 2023-09-10T08:02:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:56 smithi098 ceph-mon[103124]: pgmap v16909: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 573 KiB/s wr, 21 op/s 2023-09-10T08:02:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:58 smithi082 ceph-mon[98694]: pgmap v16910: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 301 KiB/s wr, 13 op/s 2023-09-10T08:02:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:58 smithi098 ceph-mon[103124]: pgmap v16910: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 301 KiB/s wr, 13 op/s 2023-09-10T08:02:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:58 smithi170 ceph-mon[102832]: pgmap v16910: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 301 KiB/s wr, 13 op/s 2023-09-10T08:02:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31409.545002 secs 2023-09-10T08:02:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31410.710796 secs 2023-09-10T08:02:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:02:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31411.153795 secs 2023-09-10T08:02:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31409.545002 secs 2023-09-10T08:02:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31410.710796 secs 2023-09-10T08:02:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:02:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31411.153795 secs 2023-09-10T08:02:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31409.545002 secs 2023-09-10T08:02:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31410.710796 secs 2023-09-10T08:02:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:02:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31411.153795 secs 2023-09-10T08:03:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:00 smithi082 ceph-mon[98694]: pgmap v16911: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 301 KiB/s wr, 13 op/s 2023-09-10T08:03:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:00 smithi098 ceph-mon[103124]: pgmap v16911: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 301 KiB/s wr, 13 op/s 2023-09-10T08:03:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:00 smithi170 ceph-mon[102832]: pgmap v16911: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 301 KiB/s wr, 13 op/s 2023-09-10T08:03:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:02 smithi082 ceph-mon[98694]: pgmap v16912: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 424 KiB/s wr, 19 op/s 2023-09-10T08:03:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:02 smithi098 ceph-mon[103124]: pgmap v16912: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 424 KiB/s wr, 19 op/s 2023-09-10T08:03:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:02 smithi170 ceph-mon[102832]: pgmap v16912: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 424 KiB/s wr, 19 op/s 2023-09-10T08:03:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31414.545062 secs 2023-09-10T08:03:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31415.710854 secs 2023-09-10T08:03:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31414.545062 secs 2023-09-10T08:03:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31415.710854 secs 2023-09-10T08:03:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31414.545062 secs 2023-09-10T08:03:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31415.710854 secs 2023-09-10T08:03:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:04 smithi082 ceph-mon[98694]: pgmap v16913: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 276 KiB/s wr, 11 op/s 2023-09-10T08:03:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31416.153857 secs 2023-09-10T08:03:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:04 smithi098 ceph-mon[103124]: pgmap v16913: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 276 KiB/s wr, 11 op/s 2023-09-10T08:03:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31416.153857 secs 2023-09-10T08:03:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:04 smithi170 ceph-mon[102832]: pgmap v16913: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 276 KiB/s wr, 11 op/s 2023-09-10T08:03:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31416.153857 secs 2023-09-10T08:03:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:06 smithi082 ceph-mon[98694]: pgmap v16914: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 321 KiB/s wr, 31 op/s 2023-09-10T08:03:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:06 smithi098 ceph-mon[103124]: pgmap v16914: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 321 KiB/s wr, 31 op/s 2023-09-10T08:03:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:06 smithi170 ceph-mon[102832]: pgmap v16914: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 321 KiB/s wr, 31 op/s 2023-09-10T08:03:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:08 smithi082 ceph-mon[98694]: pgmap v16915: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 168 KiB/s wr, 25 op/s 2023-09-10T08:03:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:08 smithi098 ceph-mon[103124]: pgmap v16915: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 168 KiB/s wr, 25 op/s 2023-09-10T08:03:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:08 smithi170 ceph-mon[102832]: pgmap v16915: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 168 KiB/s wr, 25 op/s 2023-09-10T08:03:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31419.545162 secs 2023-09-10T08:03:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31420.710913 secs 2023-09-10T08:03:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:09 smithi082 ceph-mon[98694]: osdmap e1125: 12 total, 12 up, 12 in 2023-09-10T08:03:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31421.153918 secs 2023-09-10T08:03:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31419.545162 secs 2023-09-10T08:03:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31420.710913 secs 2023-09-10T08:03:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:09 smithi098 ceph-mon[103124]: osdmap e1125: 12 total, 12 up, 12 in 2023-09-10T08:03:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31421.153918 secs 2023-09-10T08:03:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31419.545162 secs 2023-09-10T08:03:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31420.710913 secs 2023-09-10T08:03:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:09 smithi170 ceph-mon[102832]: osdmap e1125: 12 total, 12 up, 12 in 2023-09-10T08:03:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31421.153918 secs 2023-09-10T08:03:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:10 smithi082 ceph-mon[98694]: pgmap v16917: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 202 KiB/s wr, 30 op/s 2023-09-10T08:03:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:10 smithi098 ceph-mon[103124]: pgmap v16917: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 202 KiB/s wr, 30 op/s 2023-09-10T08:03:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:10 smithi170 ceph-mon[102832]: pgmap v16917: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 202 KiB/s wr, 30 op/s 2023-09-10T08:03:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:12 smithi082 ceph-mon[98694]: pgmap v16918: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-09-10T08:03:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:12 smithi082 ceph-mon[98694]: osdmap e1126: 12 total, 12 up, 12 in 2023-09-10T08:03:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:12 smithi098 ceph-mon[103124]: pgmap v16918: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-09-10T08:03:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:12 smithi098 ceph-mon[103124]: osdmap e1126: 12 total, 12 up, 12 in 2023-09-10T08:03:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:12 smithi170 ceph-mon[102832]: pgmap v16918: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-09-10T08:03:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:12 smithi170 ceph-mon[102832]: osdmap e1126: 12 total, 12 up, 12 in 2023-09-10T08:03:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31424.545222 secs 2023-09-10T08:03:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31425.710999 secs 2023-09-10T08:03:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31424.545222 secs 2023-09-10T08:03:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31425.710999 secs 2023-09-10T08:03:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31424.545222 secs 2023-09-10T08:03:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31425.710999 secs 2023-09-10T08:03:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:14 smithi082 ceph-mon[98694]: pgmap v16920: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T08:03:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31426.153981 secs 2023-09-10T08:03:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:14 smithi098 ceph-mon[103124]: pgmap v16920: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T08:03:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31426.153981 secs 2023-09-10T08:03:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:14 smithi170 ceph-mon[102832]: pgmap v16920: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T08:03:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31426.153981 secs 2023-09-10T08:03:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:16 smithi098 ceph-mon[103124]: pgmap v16921: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 14 op/s 2023-09-10T08:03:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:16 smithi170 ceph-mon[102832]: pgmap v16921: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 14 op/s 2023-09-10T08:03:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:16 smithi082 ceph-mon[98694]: pgmap v16921: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 14 op/s 2023-09-10T08:03:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:18 smithi082 ceph-mon[98694]: pgmap v16922: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 12 op/s 2023-09-10T08:03:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:18 smithi098 ceph-mon[103124]: pgmap v16922: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 12 op/s 2023-09-10T08:03:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:18 smithi170 ceph-mon[102832]: pgmap v16922: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 12 op/s 2023-09-10T08:03:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31429.545283 secs 2023-09-10T08:03:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31430.711064 secs 2023-09-10T08:03:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31431.154042 secs 2023-09-10T08:03:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31429.545283 secs 2023-09-10T08:03:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31430.711064 secs 2023-09-10T08:03:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31431.154042 secs 2023-09-10T08:03:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31429.545283 secs 2023-09-10T08:03:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31430.711064 secs 2023-09-10T08:03:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31431.154042 secs 2023-09-10T08:03:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:20 smithi098 ceph-mon[103124]: pgmap v16923: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T08:03:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:20 smithi170 ceph-mon[102832]: pgmap v16923: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T08:03:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:20 smithi082 ceph-mon[98694]: pgmap v16923: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T08:03:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:22 smithi082 ceph-mon[98694]: pgmap v16924: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T08:03:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:22 smithi098 ceph-mon[103124]: pgmap v16924: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T08:03:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:22 smithi170 ceph-mon[102832]: pgmap v16924: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 71 KiB/s wr, 11 op/s 2023-09-10T08:03:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31434.545386 secs 2023-09-10T08:03:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31435.711128 secs 2023-09-10T08:03:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31434.545386 secs 2023-09-10T08:03:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31435.711128 secs 2023-09-10T08:03:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31434.545386 secs 2023-09-10T08:03:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31435.711128 secs 2023-09-10T08:03:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:24 smithi098 ceph-mon[103124]: pgmap v16925: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 63 KiB/s wr, 10 op/s 2023-09-10T08:03:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31436.154103 secs 2023-09-10T08:03:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:24 smithi170 ceph-mon[102832]: pgmap v16925: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 63 KiB/s wr, 10 op/s 2023-09-10T08:03:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31436.154103 secs 2023-09-10T08:03:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:24 smithi082 ceph-mon[98694]: pgmap v16925: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 63 KiB/s wr, 10 op/s 2023-09-10T08:03:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31436.154103 secs 2023-09-10T08:03:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:26 smithi098 ceph-mon[103124]: pgmap v16926: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 106 KiB/s wr, 15 op/s 2023-09-10T08:03:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:26 smithi170 ceph-mon[102832]: pgmap v16926: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 106 KiB/s wr, 15 op/s 2023-09-10T08:03:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:26 smithi082 ceph-mon[98694]: pgmap v16926: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 106 KiB/s wr, 15 op/s 2023-09-10T08:03:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:28 smithi098 ceph-mon[103124]: pgmap v16927: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T08:03:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:28 smithi170 ceph-mon[102832]: pgmap v16927: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T08:03:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:28 smithi082 ceph-mon[98694]: pgmap v16927: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T08:03:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31439.545485 secs 2023-09-10T08:03:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31440.711232 secs 2023-09-10T08:03:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31441.154164 secs 2023-09-10T08:03:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31439.545485 secs 2023-09-10T08:03:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31440.711232 secs 2023-09-10T08:03:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31441.154164 secs 2023-09-10T08:03:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31439.545485 secs 2023-09-10T08:03:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31440.711232 secs 2023-09-10T08:03:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31441.154164 secs 2023-09-10T08:03:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:03:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:03:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:30 smithi098 ceph-mon[103124]: pgmap v16928: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T08:03:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:03:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:03:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:30 smithi170 ceph-mon[102832]: pgmap v16928: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T08:03:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:03:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:03:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:30 smithi082 ceph-mon[98694]: pgmap v16928: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 106 KiB/s wr, 11 op/s 2023-09-10T08:03:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:03:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:03:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:32 smithi098 ceph-mon[103124]: pgmap v16929: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 235 KiB/s wr, 16 op/s 2023-09-10T08:03:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:03:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:03:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:32 smithi170 ceph-mon[102832]: pgmap v16929: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 235 KiB/s wr, 16 op/s 2023-09-10T08:03:32.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:03:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:03:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:32 smithi082 ceph-mon[98694]: pgmap v16929: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 235 KiB/s wr, 16 op/s 2023-09-10T08:03:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31445.711292 secs 2023-09-10T08:03:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31445.711292 secs 2023-09-10T08:03:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31445.711292 secs 2023-09-10T08:03:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:34 smithi170 ceph-mon[102832]: pgmap v16930: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 176 KiB/s wr, 11 op/s 2023-09-10T08:03:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31446.154225 secs 2023-09-10T08:03:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:34 smithi082 ceph-mon[98694]: pgmap v16930: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 176 KiB/s wr, 11 op/s 2023-09-10T08:03:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31446.154225 secs 2023-09-10T08:03:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:34 smithi098 ceph-mon[103124]: pgmap v16930: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 176 KiB/s wr, 11 op/s 2023-09-10T08:03:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31446.154225 secs 2023-09-10T08:03:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:03:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:03:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:03:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:03:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:03:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:03:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:36 smithi170 ceph-mon[102832]: pgmap v16931: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 370 KiB/s wr, 17 op/s 2023-09-10T08:03:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:03:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:03:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:03:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:36 smithi082 ceph-mon[98694]: pgmap v16931: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 370 KiB/s wr, 17 op/s 2023-09-10T08:03:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:03:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:03:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:03:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:36 smithi098 ceph-mon[103124]: pgmap v16931: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 370 KiB/s wr, 17 op/s 2023-09-10T08:03:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:38 smithi098 ceph-mon[103124]: pgmap v16932: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 323 KiB/s wr, 11 op/s 2023-09-10T08:03:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:38 smithi170 ceph-mon[102832]: pgmap v16932: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 323 KiB/s wr, 11 op/s 2023-09-10T08:03:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:38 smithi082 ceph-mon[98694]: pgmap v16932: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 323 KiB/s wr, 11 op/s 2023-09-10T08:03:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31450.711352 secs 2023-09-10T08:03:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31451.154285 secs 2023-09-10T08:03:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31450.711352 secs 2023-09-10T08:03:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31451.154285 secs 2023-09-10T08:03:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31450.711352 secs 2023-09-10T08:03:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31451.154285 secs 2023-09-10T08:03:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:03:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:03:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:40 smithi098 ceph-mon[103124]: pgmap v16933: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 323 KiB/s wr, 11 op/s 2023-09-10T08:03:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:03:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:03:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:40 smithi170 ceph-mon[102832]: pgmap v16933: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 323 KiB/s wr, 11 op/s 2023-09-10T08:03:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:03:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:03:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:40 smithi082 ceph-mon[98694]: pgmap v16933: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 323 KiB/s wr, 11 op/s 2023-09-10T08:03:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:42 smithi098 ceph-mon[103124]: pgmap v16934: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 594 KiB/s wr, 17 op/s 2023-09-10T08:03:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:42 smithi170 ceph-mon[102832]: pgmap v16934: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 594 KiB/s wr, 17 op/s 2023-09-10T08:03:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:42 smithi082 ceph-mon[98694]: pgmap v16934: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 594 KiB/s wr, 17 op/s 2023-09-10T08:03:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31455.711415 secs 2023-09-10T08:03:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31455.711415 secs 2023-09-10T08:03:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31455.711415 secs 2023-09-10T08:03:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:44 smithi098 ceph-mon[103124]: pgmap v16935: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 466 KiB/s wr, 12 op/s 2023-09-10T08:03:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31456.154350 secs 2023-09-10T08:03:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:44 smithi170 ceph-mon[102832]: pgmap v16935: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 466 KiB/s wr, 12 op/s 2023-09-10T08:03:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31456.154350 secs 2023-09-10T08:03:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:44 smithi082 ceph-mon[98694]: pgmap v16935: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 466 KiB/s wr, 12 op/s 2023-09-10T08:03:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31456.154350 secs 2023-09-10T08:03:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:46 smithi098 ceph-mon[103124]: pgmap v16936: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 636 KiB/s wr, 19 op/s 2023-09-10T08:03:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:03:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:03:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:46 smithi170 ceph-mon[102832]: pgmap v16936: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 636 KiB/s wr, 19 op/s 2023-09-10T08:03:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:03:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:03:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:46 smithi082 ceph-mon[98694]: pgmap v16936: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 636 KiB/s wr, 19 op/s 2023-09-10T08:03:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:03:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:03:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:48 smithi098 ceph-mon[103124]: pgmap v16937: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 442 KiB/s wr, 13 op/s 2023-09-10T08:03:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:48 smithi170 ceph-mon[102832]: pgmap v16937: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 442 KiB/s wr, 13 op/s 2023-09-10T08:03:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:48 smithi082 ceph-mon[98694]: pgmap v16937: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 442 KiB/s wr, 13 op/s 2023-09-10T08:03:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31460.711475 secs 2023-09-10T08:03:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31461.154413 secs 2023-09-10T08:03:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31460.711475 secs 2023-09-10T08:03:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31461.154413 secs 2023-09-10T08:03:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31460.711475 secs 2023-09-10T08:03:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31461.154413 secs 2023-09-10T08:03:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:03:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:03:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:50 smithi098 ceph-mon[103124]: pgmap v16938: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 442 KiB/s wr, 13 op/s 2023-09-10T08:03:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:03:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:03:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:50 smithi170 ceph-mon[102832]: pgmap v16938: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 442 KiB/s wr, 13 op/s 2023-09-10T08:03:50.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:03:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:03:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:50 smithi082 ceph-mon[98694]: pgmap v16938: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 442 KiB/s wr, 13 op/s 2023-09-10T08:03:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:03:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:03:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:03:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:03:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:03:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:03:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:03:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:03:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:03:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:03:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:03:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:03:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:03:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:03:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:52 smithi098 ceph-mon[103124]: pgmap v16939: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 731 KiB/s wr, 20 op/s 2023-09-10T08:03:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:03:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:03:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:52 smithi170 ceph-mon[102832]: pgmap v16939: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 731 KiB/s wr, 20 op/s 2023-09-10T08:03:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:03:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:03:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:52 smithi082 ceph-mon[98694]: pgmap v16939: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 731 KiB/s wr, 20 op/s 2023-09-10T08:03:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31465.711493 secs 2023-09-10T08:03:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31465.711493 secs 2023-09-10T08:03:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31465.711493 secs 2023-09-10T08:03:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:54 smithi170 ceph-mon[102832]: pgmap v16940: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 460 KiB/s wr, 15 op/s 2023-09-10T08:03:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31466.154481 secs 2023-09-10T08:03:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:54 smithi082 ceph-mon[98694]: pgmap v16940: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 460 KiB/s wr, 15 op/s 2023-09-10T08:03:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31466.154481 secs 2023-09-10T08:03:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:54 smithi098 ceph-mon[103124]: pgmap v16940: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 460 KiB/s wr, 15 op/s 2023-09-10T08:03:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31466.154481 secs 2023-09-10T08:03:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:56 smithi098 ceph-mon[103124]: pgmap v16941: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 655 KiB/s wr, 21 op/s 2023-09-10T08:03:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:56 smithi170 ceph-mon[102832]: pgmap v16941: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 655 KiB/s wr, 21 op/s 2023-09-10T08:03:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:56 smithi082 ceph-mon[98694]: pgmap v16941: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 655 KiB/s wr, 21 op/s 2023-09-10T08:03:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:58 smithi098 ceph-mon[103124]: pgmap v16942: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 485 KiB/s wr, 14 op/s 2023-09-10T08:03:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:03:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:03:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31469.545966 secs 2023-09-10T08:03:59.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31470.711551 secs 2023-09-10T08:03:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:58 smithi170 ceph-mon[102832]: pgmap v16942: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 485 KiB/s wr, 14 op/s 2023-09-10T08:03:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:03:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:03:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31469.545966 secs 2023-09-10T08:03:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31470.711551 secs 2023-09-10T08:03:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:58 smithi082 ceph-mon[98694]: pgmap v16942: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 485 KiB/s wr, 14 op/s 2023-09-10T08:03:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:03:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:03:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31469.545966 secs 2023-09-10T08:03:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31470.711551 secs 2023-09-10T08:04:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:03:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31471.154542 secs 2023-09-10T08:04:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:03:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31471.154542 secs 2023-09-10T08:04:00.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:03:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31471.154542 secs 2023-09-10T08:04:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:00 smithi098 ceph-mon[103124]: pgmap v16943: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 485 KiB/s wr, 14 op/s 2023-09-10T08:04:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:04:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:04:01.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:00 smithi170 ceph-mon[102832]: pgmap v16943: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 485 KiB/s wr, 14 op/s 2023-09-10T08:04:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:04:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:04:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:00 smithi082 ceph-mon[98694]: pgmap v16943: 97 pgs: 97 active+clean; 10 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 485 KiB/s wr, 14 op/s 2023-09-10T08:04:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:04:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:04:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:02 smithi098 ceph-mon[103124]: pgmap v16944: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 774 KiB/s wr, 34 op/s 2023-09-10T08:04:03.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:02 smithi170 ceph-mon[102832]: pgmap v16944: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 774 KiB/s wr, 34 op/s 2023-09-10T08:04:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:02 smithi082 ceph-mon[98694]: pgmap v16944: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 774 KiB/s wr, 34 op/s 2023-09-10T08:04:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31474.546071 secs 2023-09-10T08:04:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31475.711609 secs 2023-09-10T08:04:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31474.546071 secs 2023-09-10T08:04:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31475.711609 secs 2023-09-10T08:04:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31474.546071 secs 2023-09-10T08:04:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31475.711609 secs 2023-09-10T08:04:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:04 smithi098 ceph-mon[103124]: pgmap v16945: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 485 KiB/s wr, 27 op/s 2023-09-10T08:04:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:04:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31476.154602 secs 2023-09-10T08:04:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:04 smithi170 ceph-mon[102832]: pgmap v16945: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 485 KiB/s wr, 27 op/s 2023-09-10T08:04:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:04:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31476.154602 secs 2023-09-10T08:04:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:04 smithi082 ceph-mon[98694]: pgmap v16945: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 485 KiB/s wr, 27 op/s 2023-09-10T08:04:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:04:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31476.154602 secs 2023-09-10T08:04:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:06 smithi098 ceph-mon[103124]: pgmap v16946: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 584 KiB/s wr, 33 op/s 2023-09-10T08:04:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:04:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:06 smithi170 ceph-mon[102832]: pgmap v16946: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 584 KiB/s wr, 33 op/s 2023-09-10T08:04:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:04:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:06 smithi082 ceph-mon[98694]: pgmap v16946: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 584 KiB/s wr, 33 op/s 2023-09-10T08:04:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:04:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:08 smithi098 ceph-mon[103124]: pgmap v16947: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 388 KiB/s wr, 26 op/s 2023-09-10T08:04:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:08 smithi170 ceph-mon[102832]: pgmap v16947: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 388 KiB/s wr, 26 op/s 2023-09-10T08:04:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:08 smithi082 ceph-mon[98694]: pgmap v16947: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 388 KiB/s wr, 26 op/s 2023-09-10T08:04:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31479.546132 secs 2023-09-10T08:04:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31480.711666 secs 2023-09-10T08:04:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:09 smithi098 ceph-mon[103124]: osdmap e1127: 12 total, 12 up, 12 in 2023-09-10T08:04:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31481.154665 secs 2023-09-10T08:04:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31479.546132 secs 2023-09-10T08:04:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31480.711666 secs 2023-09-10T08:04:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:09 smithi170 ceph-mon[102832]: osdmap e1127: 12 total, 12 up, 12 in 2023-09-10T08:04:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31481.154665 secs 2023-09-10T08:04:09.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31479.546132 secs 2023-09-10T08:04:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31480.711666 secs 2023-09-10T08:04:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:09 smithi082 ceph-mon[98694]: osdmap e1127: 12 total, 12 up, 12 in 2023-09-10T08:04:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31481.154665 secs 2023-09-10T08:04:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:10 smithi098 ceph-mon[103124]: pgmap v16949: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 466 KiB/s wr, 32 op/s 2023-09-10T08:04:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:10 smithi170 ceph-mon[102832]: pgmap v16949: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 466 KiB/s wr, 32 op/s 2023-09-10T08:04:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:10 smithi082 ceph-mon[98694]: pgmap v16949: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 466 KiB/s wr, 32 op/s 2023-09-10T08:04:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:12 smithi098 ceph-mon[103124]: pgmap v16950: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 119 KiB/s wr, 14 op/s 2023-09-10T08:04:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:12 smithi098 ceph-mon[103124]: osdmap e1128: 12 total, 12 up, 12 in 2023-09-10T08:04:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:12 smithi170 ceph-mon[102832]: pgmap v16950: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 119 KiB/s wr, 14 op/s 2023-09-10T08:04:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:12 smithi170 ceph-mon[102832]: osdmap e1128: 12 total, 12 up, 12 in 2023-09-10T08:04:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:12 smithi082 ceph-mon[98694]: pgmap v16950: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 119 KiB/s wr, 14 op/s 2023-09-10T08:04:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:12 smithi082 ceph-mon[98694]: osdmap e1128: 12 total, 12 up, 12 in 2023-09-10T08:04:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31484.546192 secs 2023-09-10T08:04:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31485.711728 secs 2023-09-10T08:04:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31484.546192 secs 2023-09-10T08:04:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31485.711728 secs 2023-09-10T08:04:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31484.546192 secs 2023-09-10T08:04:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31485.711728 secs 2023-09-10T08:04:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:14 smithi098 ceph-mon[103124]: pgmap v16952: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T08:04:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:14 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31486.154730 secs 2023-09-10T08:04:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:14 smithi170 ceph-mon[102832]: pgmap v16952: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T08:04:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:14 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31486.154730 secs 2023-09-10T08:04:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:14 smithi082 ceph-mon[98694]: pgmap v16952: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T08:04:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:14 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31486.154730 secs 2023-09-10T08:04:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:16 smithi098 ceph-mon[103124]: pgmap v16953: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 80 KiB/s wr, 15 op/s 2023-09-10T08:04:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:16 smithi170 ceph-mon[102832]: pgmap v16953: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 80 KiB/s wr, 15 op/s 2023-09-10T08:04:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:16 smithi082 ceph-mon[98694]: pgmap v16953: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 80 KiB/s wr, 15 op/s 2023-09-10T08:04:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:18.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:18 smithi098 ceph-mon[103124]: pgmap v16954: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 70 KiB/s wr, 13 op/s 2023-09-10T08:04:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:18 smithi170 ceph-mon[102832]: pgmap v16954: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 70 KiB/s wr, 13 op/s 2023-09-10T08:04:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:18 smithi082 ceph-mon[98694]: pgmap v16954: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 70 KiB/s wr, 13 op/s 2023-09-10T08:04:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31489.546252 secs 2023-09-10T08:04:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31490.711790 secs 2023-09-10T08:04:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:19 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31491.154790 secs 2023-09-10T08:04:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31489.546252 secs 2023-09-10T08:04:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31490.711790 secs 2023-09-10T08:04:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:19 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31491.154790 secs 2023-09-10T08:04:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31489.546252 secs 2023-09-10T08:04:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31490.711790 secs 2023-09-10T08:04:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:19 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31491.154790 secs 2023-09-10T08:04:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:20 smithi098 ceph-mon[103124]: pgmap v16955: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T08:04:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:20 smithi170 ceph-mon[102832]: pgmap v16955: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T08:04:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:20 smithi082 ceph-mon[98694]: pgmap v16955: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 12 op/s 2023-09-10T08:04:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:22 smithi098 ceph-mon[103124]: pgmap v16956: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T08:04:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:22 smithi170 ceph-mon[102832]: pgmap v16956: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T08:04:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:22 smithi082 ceph-mon[98694]: pgmap v16956: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 11 op/s 2023-09-10T08:04:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31494.546313 secs 2023-09-10T08:04:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31495.711847 secs 2023-09-10T08:04:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31494.546313 secs 2023-09-10T08:04:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31495.711847 secs 2023-09-10T08:04:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31494.546313 secs 2023-09-10T08:04:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31495.711847 secs 2023-09-10T08:04:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:24 smithi098 ceph-mon[103124]: pgmap v16957: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 10 op/s 2023-09-10T08:04:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31496.154852 secs 2023-09-10T08:04:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:24 smithi170 ceph-mon[102832]: pgmap v16957: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 10 op/s 2023-09-10T08:04:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31496.154852 secs 2023-09-10T08:04:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:24 smithi082 ceph-mon[98694]: pgmap v16957: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 57 KiB/s wr, 10 op/s 2023-09-10T08:04:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31496.154852 secs 2023-09-10T08:04:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:26 smithi098 ceph-mon[103124]: pgmap v16958: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 54 KiB/s wr, 14 op/s 2023-09-10T08:04:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:26 smithi170 ceph-mon[102832]: pgmap v16958: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 54 KiB/s wr, 14 op/s 2023-09-10T08:04:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:26 smithi082 ceph-mon[98694]: pgmap v16958: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 54 KiB/s wr, 14 op/s 2023-09-10T08:04:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:28 smithi098 ceph-mon[103124]: pgmap v16959: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-10T08:04:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:28 smithi170 ceph-mon[102832]: pgmap v16959: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-10T08:04:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:28 smithi082 ceph-mon[98694]: pgmap v16959: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-10T08:04:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31499.546428 secs 2023-09-10T08:04:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31500.711906 secs 2023-09-10T08:04:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31501.154913 secs 2023-09-10T08:04:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31499.546428 secs 2023-09-10T08:04:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31500.711906 secs 2023-09-10T08:04:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31501.154913 secs 2023-09-10T08:04:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31499.546428 secs 2023-09-10T08:04:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31500.711906 secs 2023-09-10T08:04:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31501.154913 secs 2023-09-10T08:04:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:30 smithi098 ceph-mon[103124]: pgmap v16960: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-10T08:04:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:30 smithi170 ceph-mon[102832]: pgmap v16960: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-10T08:04:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:30 smithi082 ceph-mon[98694]: pgmap v16960: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-10T08:04:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:32 smithi098 ceph-mon[103124]: pgmap v16961: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 136 KiB/s wr, 14 op/s 2023-09-10T08:04:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:32 smithi170 ceph-mon[102832]: pgmap v16961: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 136 KiB/s wr, 14 op/s 2023-09-10T08:04:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:32 smithi082 ceph-mon[98694]: pgmap v16961: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 136 KiB/s wr, 14 op/s 2023-09-10T08:04:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31505.711963 secs 2023-09-10T08:04:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31505.711963 secs 2023-09-10T08:04:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31505.711963 secs 2023-09-10T08:04:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:34 smithi098 ceph-mon[103124]: pgmap v16962: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 10 op/s 2023-09-10T08:04:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31506.155222 secs 2023-09-10T08:04:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:34 smithi170 ceph-mon[102832]: pgmap v16962: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 10 op/s 2023-09-10T08:04:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31506.155222 secs 2023-09-10T08:04:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:34 smithi082 ceph-mon[98694]: pgmap v16962: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 10 op/s 2023-09-10T08:04:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31506.155222 secs 2023-09-10T08:04:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:04:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:04:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:04:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:36 smithi098 ceph-mon[103124]: pgmap v16963: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 218 KiB/s wr, 15 op/s 2023-09-10T08:04:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:04:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:04:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:36 smithi170 ceph-mon[102832]: pgmap v16963: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 218 KiB/s wr, 15 op/s 2023-09-10T08:04:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:04:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:04:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:36 smithi082 ceph-mon[98694]: pgmap v16963: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 218 KiB/s wr, 15 op/s 2023-09-10T08:04:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:04:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:04:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:38 smithi098 ceph-mon[103124]: pgmap v16964: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 217 KiB/s wr, 10 op/s 2023-09-10T08:04:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:04:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31510.712023 secs 2023-09-10T08:04:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:38 smithi170 ceph-mon[102832]: pgmap v16964: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 217 KiB/s wr, 10 op/s 2023-09-10T08:04:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:04:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31510.712023 secs 2023-09-10T08:04:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:38 smithi082 ceph-mon[98694]: pgmap v16964: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 217 KiB/s wr, 10 op/s 2023-09-10T08:04:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:04:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31510.712023 secs 2023-09-10T08:04:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:39 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31511.155284 secs 2023-09-10T08:04:40.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:39 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:40.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31511.155284 secs 2023-09-10T08:04:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:39 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31511.155284 secs 2023-09-10T08:04:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:40 smithi170 ceph-mon[102832]: pgmap v16965: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 217 KiB/s wr, 10 op/s 2023-09-10T08:04:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:04:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:40 smithi082 ceph-mon[98694]: pgmap v16965: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 217 KiB/s wr, 10 op/s 2023-09-10T08:04:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:04:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:40 smithi098 ceph-mon[103124]: pgmap v16965: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 217 KiB/s wr, 10 op/s 2023-09-10T08:04:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:04:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:42 smithi170 ceph-mon[102832]: pgmap v16966: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 799 KiB/s wr, 18 op/s 2023-09-10T08:04:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:42 smithi082 ceph-mon[98694]: pgmap v16966: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 799 KiB/s wr, 18 op/s 2023-09-10T08:04:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:42 smithi098 ceph-mon[103124]: pgmap v16966: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 799 KiB/s wr, 18 op/s 2023-09-10T08:04:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31515.712082 secs 2023-09-10T08:04:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31515.712082 secs 2023-09-10T08:04:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31515.712082 secs 2023-09-10T08:04:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:44 smithi170 ceph-mon[102832]: pgmap v16967: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 663 KiB/s wr, 13 op/s 2023-09-10T08:04:45.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:04:45.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:45.184 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31516.155349 secs 2023-09-10T08:04:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:44 smithi082 ceph-mon[98694]: pgmap v16967: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 663 KiB/s wr, 13 op/s 2023-09-10T08:04:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:04:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31516.155349 secs 2023-09-10T08:04:45.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:44 smithi098 ceph-mon[103124]: pgmap v16967: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 663 KiB/s wr, 13 op/s 2023-09-10T08:04:45.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:04:45.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:45.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31516.155349 secs 2023-09-10T08:04:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:46 smithi170 ceph-mon[102832]: pgmap v16968: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 815 KiB/s wr, 21 op/s 2023-09-10T08:04:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:04:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:46 smithi082 ceph-mon[98694]: pgmap v16968: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 815 KiB/s wr, 21 op/s 2023-09-10T08:04:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:04:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:46 smithi098 ceph-mon[103124]: pgmap v16968: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 815 KiB/s wr, 21 op/s 2023-09-10T08:04:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:04:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:48 smithi098 ceph-mon[103124]: pgmap v16969: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 733 KiB/s wr, 15 op/s 2023-09-10T08:04:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:48 smithi170 ceph-mon[102832]: pgmap v16969: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 733 KiB/s wr, 15 op/s 2023-09-10T08:04:48.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:48 smithi082 ceph-mon[98694]: pgmap v16969: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 733 KiB/s wr, 15 op/s 2023-09-10T08:04:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31520.712152 secs 2023-09-10T08:04:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31521.155415 secs 2023-09-10T08:04:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31520.712152 secs 2023-09-10T08:04:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31521.155415 secs 2023-09-10T08:04:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31520.712152 secs 2023-09-10T08:04:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31521.155415 secs 2023-09-10T08:04:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:50 smithi098 ceph-mon[103124]: pgmap v16970: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 733 KiB/s wr, 15 op/s 2023-09-10T08:04:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:50 smithi170 ceph-mon[102832]: pgmap v16970: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 733 KiB/s wr, 15 op/s 2023-09-10T08:04:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:50 smithi082 ceph-mon[98694]: pgmap v16970: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 733 KiB/s wr, 15 op/s 2023-09-10T08:04:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:04:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:04:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:04:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:52 smithi098 ceph-mon[103124]: pgmap v16971: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 24 op/s 2023-09-10T08:04:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:04:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:04:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:04:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:52 smithi170 ceph-mon[102832]: pgmap v16971: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 24 op/s 2023-09-10T08:04:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:04:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:04:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:04:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:52 smithi082 ceph-mon[98694]: pgmap v16971: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 24 op/s 2023-09-10T08:04:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:04:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:04:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:04:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31524.546765 secs 2023-09-10T08:04:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31525.712239 secs 2023-09-10T08:04:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:04:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:54 smithi170 ceph-mon[102832]: pgmap v16972: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 563 KiB/s wr, 16 op/s 2023-09-10T08:04:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31526.155487 secs 2023-09-10T08:04:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31524.546765 secs 2023-09-10T08:04:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31525.712239 secs 2023-09-10T08:04:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:04:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:54 smithi082 ceph-mon[98694]: pgmap v16972: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 563 KiB/s wr, 16 op/s 2023-09-10T08:04:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31526.155487 secs 2023-09-10T08:04:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31524.546765 secs 2023-09-10T08:04:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31525.712239 secs 2023-09-10T08:04:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:04:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:54 smithi098 ceph-mon[103124]: pgmap v16972: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 563 KiB/s wr, 16 op/s 2023-09-10T08:04:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31526.155487 secs 2023-09-10T08:04:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:04:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:04:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:04:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:04:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:04:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:04:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:04:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:56 smithi170 ceph-mon[102832]: pgmap v16973: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 564 KiB/s wr, 22 op/s 2023-09-10T08:04:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:56 smithi082 ceph-mon[98694]: pgmap v16973: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 564 KiB/s wr, 22 op/s 2023-09-10T08:04:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:56 smithi098 ceph-mon[103124]: pgmap v16973: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 564 KiB/s wr, 22 op/s 2023-09-10T08:04:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:58 smithi098 ceph-mon[103124]: pgmap v16974: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 412 KiB/s wr, 14 op/s 2023-09-10T08:04:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:58 smithi170 ceph-mon[102832]: pgmap v16974: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 412 KiB/s wr, 14 op/s 2023-09-10T08:04:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:58 smithi082 ceph-mon[98694]: pgmap v16974: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 412 KiB/s wr, 14 op/s 2023-09-10T08:04:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31529.546776 secs 2023-09-10T08:04:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31530.712297 secs 2023-09-10T08:04:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:04:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31531.155553 secs 2023-09-10T08:04:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31529.546776 secs 2023-09-10T08:04:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31530.712297 secs 2023-09-10T08:04:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:04:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31531.155553 secs 2023-09-10T08:04:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31529.546776 secs 2023-09-10T08:04:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31530.712297 secs 2023-09-10T08:04:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:04:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31531.155553 secs 2023-09-10T08:05:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:00 smithi098 ceph-mon[103124]: pgmap v16975: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 412 KiB/s wr, 14 op/s 2023-09-10T08:05:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:00 smithi170 ceph-mon[102832]: pgmap v16975: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 412 KiB/s wr, 14 op/s 2023-09-10T08:05:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:00 smithi082 ceph-mon[98694]: pgmap v16975: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 412 KiB/s wr, 14 op/s 2023-09-10T08:05:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:02 smithi098 ceph-mon[103124]: pgmap v16976: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 661 KiB/s wr, 27 op/s 2023-09-10T08:05:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:02 smithi170 ceph-mon[102832]: pgmap v16976: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 661 KiB/s wr, 27 op/s 2023-09-10T08:05:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:02 smithi082 ceph-mon[98694]: pgmap v16976: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 661 KiB/s wr, 27 op/s 2023-09-10T08:05:04.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31534.546886 secs 2023-09-10T08:05:04.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31535.712358 secs 2023-09-10T08:05:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31534.546886 secs 2023-09-10T08:05:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31535.712358 secs 2023-09-10T08:05:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31534.546886 secs 2023-09-10T08:05:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31535.712358 secs 2023-09-10T08:05:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:04 smithi098 ceph-mon[103124]: pgmap v16977: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 249 KiB/s wr, 18 op/s 2023-09-10T08:05:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31536.155618 secs 2023-09-10T08:05:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:04 smithi170 ceph-mon[102832]: pgmap v16977: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 249 KiB/s wr, 18 op/s 2023-09-10T08:05:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31536.155618 secs 2023-09-10T08:05:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:04 smithi082 ceph-mon[98694]: pgmap v16977: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 249 KiB/s wr, 18 op/s 2023-09-10T08:05:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31536.155618 secs 2023-09-10T08:05:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:06 smithi098 ceph-mon[103124]: pgmap v16978: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 371 KiB/s wr, 32 op/s 2023-09-10T08:05:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:06 smithi170 ceph-mon[102832]: pgmap v16978: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 371 KiB/s wr, 32 op/s 2023-09-10T08:05:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:06 smithi082 ceph-mon[98694]: pgmap v16978: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 371 KiB/s wr, 32 op/s 2023-09-10T08:05:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:08 smithi098 ceph-mon[103124]: pgmap v16979: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 370 KiB/s wr, 26 op/s 2023-09-10T08:05:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:08 smithi170 ceph-mon[102832]: pgmap v16979: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 370 KiB/s wr, 26 op/s 2023-09-10T08:05:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:08 smithi082 ceph-mon[98694]: pgmap v16979: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 370 KiB/s wr, 26 op/s 2023-09-10T08:05:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31539.546945 secs 2023-09-10T08:05:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31540.712443 secs 2023-09-10T08:05:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:09 smithi098 ceph-mon[103124]: osdmap e1129: 12 total, 12 up, 12 in 2023-09-10T08:05:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31541.155682 secs 2023-09-10T08:05:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31539.546945 secs 2023-09-10T08:05:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31540.712443 secs 2023-09-10T08:05:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:09 smithi170 ceph-mon[102832]: osdmap e1129: 12 total, 12 up, 12 in 2023-09-10T08:05:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31541.155682 secs 2023-09-10T08:05:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31539.546945 secs 2023-09-10T08:05:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31540.712443 secs 2023-09-10T08:05:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:09 smithi082 ceph-mon[98694]: osdmap e1129: 12 total, 12 up, 12 in 2023-09-10T08:05:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31541.155682 secs 2023-09-10T08:05:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:10 smithi098 ceph-mon[103124]: pgmap v16981: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 445 KiB/s wr, 31 op/s 2023-09-10T08:05:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:10 smithi170 ceph-mon[102832]: pgmap v16981: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 445 KiB/s wr, 31 op/s 2023-09-10T08:05:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:10 smithi082 ceph-mon[98694]: pgmap v16981: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 445 KiB/s wr, 31 op/s 2023-09-10T08:05:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:12 smithi098 ceph-mon[103124]: pgmap v16982: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 146 KiB/s wr, 22 op/s 2023-09-10T08:05:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:12 smithi098 ceph-mon[103124]: osdmap e1130: 12 total, 12 up, 12 in 2023-09-10T08:05:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:12 smithi170 ceph-mon[102832]: pgmap v16982: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 146 KiB/s wr, 22 op/s 2023-09-10T08:05:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:12 smithi170 ceph-mon[102832]: osdmap e1130: 12 total, 12 up, 12 in 2023-09-10T08:05:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:12 smithi082 ceph-mon[98694]: pgmap v16982: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 146 KiB/s wr, 22 op/s 2023-09-10T08:05:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:12 smithi082 ceph-mon[98694]: osdmap e1130: 12 total, 12 up, 12 in 2023-09-10T08:05:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31544.547007 secs 2023-09-10T08:05:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31545.712495 secs 2023-09-10T08:05:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31544.547007 secs 2023-09-10T08:05:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31545.712495 secs 2023-09-10T08:05:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31544.547007 secs 2023-09-10T08:05:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31545.712495 secs 2023-09-10T08:05:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:14 smithi098 ceph-mon[103124]: pgmap v16984: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T08:05:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31546.155748 secs 2023-09-10T08:05:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:14 smithi170 ceph-mon[102832]: pgmap v16984: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T08:05:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31546.155748 secs 2023-09-10T08:05:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:14 smithi082 ceph-mon[98694]: pgmap v16984: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T08:05:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31546.155748 secs 2023-09-10T08:05:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:16 smithi098 ceph-mon[103124]: pgmap v16985: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 15 op/s 2023-09-10T08:05:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:05:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:05:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:16 smithi170 ceph-mon[102832]: pgmap v16985: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 15 op/s 2023-09-10T08:05:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:05:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:05:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:16 smithi082 ceph-mon[98694]: pgmap v16985: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 15 op/s 2023-09-10T08:05:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:05:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:05:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:18 smithi098 ceph-mon[103124]: pgmap v16986: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 13 op/s 2023-09-10T08:05:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:05:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:05:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31549.547066 secs 2023-09-10T08:05:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31550.712547 secs 2023-09-10T08:05:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:18 smithi170 ceph-mon[102832]: pgmap v16986: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 13 op/s 2023-09-10T08:05:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:05:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:05:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31549.547066 secs 2023-09-10T08:05:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31550.712547 secs 2023-09-10T08:05:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:18 smithi082 ceph-mon[98694]: pgmap v16986: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 222 B/s wr, 13 op/s 2023-09-10T08:05:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:05:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:05:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31549.547066 secs 2023-09-10T08:05:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31550.712547 secs 2023-09-10T08:05:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31551.155814 secs 2023-09-10T08:05:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31551.155814 secs 2023-09-10T08:05:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31551.155814 secs 2023-09-10T08:05:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:20 smithi170 ceph-mon[102832]: pgmap v16987: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 12 op/s 2023-09-10T08:05:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:20 smithi082 ceph-mon[98694]: pgmap v16987: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 12 op/s 2023-09-10T08:05:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:20 smithi098 ceph-mon[103124]: pgmap v16987: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 12 op/s 2023-09-10T08:05:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:22 smithi170 ceph-mon[102832]: pgmap v16988: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 13 op/s 2023-09-10T08:05:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:22 smithi082 ceph-mon[98694]: pgmap v16988: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 13 op/s 2023-09-10T08:05:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:22 smithi098 ceph-mon[103124]: pgmap v16988: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 13 op/s 2023-09-10T08:05:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31554.547127 secs 2023-09-10T08:05:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31555.712607 secs 2023-09-10T08:05:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31554.547127 secs 2023-09-10T08:05:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31555.712607 secs 2023-09-10T08:05:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31554.547127 secs 2023-09-10T08:05:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31555.712607 secs 2023-09-10T08:05:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:24 smithi170 ceph-mon[102832]: pgmap v16989: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T08:05:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31556.155882 secs 2023-09-10T08:05:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:24 smithi082 ceph-mon[98694]: pgmap v16989: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T08:05:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31556.155882 secs 2023-09-10T08:05:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:24 smithi098 ceph-mon[103124]: pgmap v16989: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T08:05:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31556.155882 secs 2023-09-10T08:05:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:26 smithi170 ceph-mon[102832]: pgmap v16990: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 216 KiB/s wr, 16 op/s 2023-09-10T08:05:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:26 smithi082 ceph-mon[98694]: pgmap v16990: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 216 KiB/s wr, 16 op/s 2023-09-10T08:05:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:26 smithi098 ceph-mon[103124]: pgmap v16990: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 216 KiB/s wr, 16 op/s 2023-09-10T08:05:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:28 smithi098 ceph-mon[103124]: pgmap v16991: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 216 KiB/s wr, 11 op/s 2023-09-10T08:05:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:28 smithi170 ceph-mon[102832]: pgmap v16991: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 216 KiB/s wr, 11 op/s 2023-09-10T08:05:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:28 smithi082 ceph-mon[98694]: pgmap v16991: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 216 KiB/s wr, 11 op/s 2023-09-10T08:05:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31559.547223 secs 2023-09-10T08:05:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31560.712666 secs 2023-09-10T08:05:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31561.155970 secs 2023-09-10T08:05:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31559.547223 secs 2023-09-10T08:05:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31560.712666 secs 2023-09-10T08:05:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31561.155970 secs 2023-09-10T08:05:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31559.547223 secs 2023-09-10T08:05:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31560.712666 secs 2023-09-10T08:05:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31561.155970 secs 2023-09-10T08:05:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:30 smithi170 ceph-mon[102832]: pgmap v16992: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 216 KiB/s wr, 11 op/s 2023-09-10T08:05:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:30 smithi082 ceph-mon[98694]: pgmap v16992: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 216 KiB/s wr, 11 op/s 2023-09-10T08:05:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:30 smithi098 ceph-mon[103124]: pgmap v16992: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 216 KiB/s wr, 11 op/s 2023-09-10T08:05:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:32 smithi098 ceph-mon[103124]: pgmap v16993: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 275 KiB/s wr, 16 op/s 2023-09-10T08:05:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:32 smithi170 ceph-mon[102832]: pgmap v16993: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 275 KiB/s wr, 16 op/s 2023-09-10T08:05:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:32 smithi082 ceph-mon[98694]: pgmap v16993: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 275 KiB/s wr, 16 op/s 2023-09-10T08:05:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31565.712727 secs 2023-09-10T08:05:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31565.712727 secs 2023-09-10T08:05:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31565.712727 secs 2023-09-10T08:05:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:34 smithi098 ceph-mon[103124]: pgmap v16994: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 169 KiB/s wr, 10 op/s 2023-09-10T08:05:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31566.156038 secs 2023-09-10T08:05:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:34 smithi170 ceph-mon[102832]: pgmap v16994: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 169 KiB/s wr, 10 op/s 2023-09-10T08:05:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31566.156038 secs 2023-09-10T08:05:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:34 smithi082 ceph-mon[98694]: pgmap v16994: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 169 KiB/s wr, 10 op/s 2023-09-10T08:05:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31566.156038 secs 2023-09-10T08:05:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:05:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:05:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:05:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:36 smithi098 ceph-mon[103124]: pgmap v16995: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 218 KiB/s wr, 15 op/s 2023-09-10T08:05:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:05:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:05:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:05:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:36 smithi170 ceph-mon[102832]: pgmap v16995: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 218 KiB/s wr, 15 op/s 2023-09-10T08:05:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:05:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:05:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:05:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:36 smithi082 ceph-mon[98694]: pgmap v16995: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 218 KiB/s wr, 15 op/s 2023-09-10T08:05:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:05:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:05:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:05:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:38 smithi098 ceph-mon[103124]: pgmap v16996: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 108 KiB/s wr, 10 op/s 2023-09-10T08:05:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:05:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:05:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31570.712784 secs 2023-09-10T08:05:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:38 smithi170 ceph-mon[102832]: pgmap v16996: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 108 KiB/s wr, 10 op/s 2023-09-10T08:05:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:05:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:05:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31570.712784 secs 2023-09-10T08:05:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:38 smithi082 ceph-mon[98694]: pgmap v16996: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 108 KiB/s wr, 10 op/s 2023-09-10T08:05:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:05:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:05:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31570.712784 secs 2023-09-10T08:05:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31571.156103 secs 2023-09-10T08:05:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31571.156103 secs 2023-09-10T08:05:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31571.156103 secs 2023-09-10T08:05:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:40 smithi170 ceph-mon[102832]: pgmap v16997: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 108 KiB/s wr, 10 op/s 2023-09-10T08:05:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:05:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:05:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:40 smithi082 ceph-mon[98694]: pgmap v16997: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 108 KiB/s wr, 10 op/s 2023-09-10T08:05:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:05:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:05:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:40 smithi098 ceph-mon[103124]: pgmap v16997: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 108 KiB/s wr, 10 op/s 2023-09-10T08:05:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:05:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:05:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:42 smithi170 ceph-mon[102832]: pgmap v16998: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 710 KiB/s wr, 18 op/s 2023-09-10T08:05:43.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:42 smithi082 ceph-mon[98694]: pgmap v16998: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 710 KiB/s wr, 18 op/s 2023-09-10T08:05:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:42 smithi098 ceph-mon[103124]: pgmap v16998: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 710 KiB/s wr, 18 op/s 2023-09-10T08:05:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31575.712844 secs 2023-09-10T08:05:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31575.712844 secs 2023-09-10T08:05:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31575.712844 secs 2023-09-10T08:05:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:44 smithi170 ceph-mon[102832]: pgmap v16999: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 652 KiB/s wr, 13 op/s 2023-09-10T08:05:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:05:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:05:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31576.156168 secs 2023-09-10T08:05:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:44 smithi082 ceph-mon[98694]: pgmap v16999: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 652 KiB/s wr, 13 op/s 2023-09-10T08:05:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:05:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:05:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31576.156168 secs 2023-09-10T08:05:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:44 smithi098 ceph-mon[103124]: pgmap v16999: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 652 KiB/s wr, 13 op/s 2023-09-10T08:05:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:05:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:05:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31576.156168 secs 2023-09-10T08:05:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:46 smithi170 ceph-mon[102832]: pgmap v17000: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 786 KiB/s wr, 20 op/s 2023-09-10T08:05:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:05:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:05:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:46 smithi082 ceph-mon[98694]: pgmap v17000: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 786 KiB/s wr, 20 op/s 2023-09-10T08:05:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:05:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:05:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:46 smithi098 ceph-mon[103124]: pgmap v17000: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 786 KiB/s wr, 20 op/s 2023-09-10T08:05:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:05:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:05:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:48 smithi098 ceph-mon[103124]: pgmap v17001: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 737 KiB/s wr, 14 op/s 2023-09-10T08:05:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:48 smithi170 ceph-mon[102832]: pgmap v17001: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 737 KiB/s wr, 14 op/s 2023-09-10T08:05:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:48 smithi082 ceph-mon[98694]: pgmap v17001: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 737 KiB/s wr, 14 op/s 2023-09-10T08:05:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31580.712903 secs 2023-09-10T08:05:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31581.156235 secs 2023-09-10T08:05:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31580.712903 secs 2023-09-10T08:05:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31581.156235 secs 2023-09-10T08:05:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31580.712903 secs 2023-09-10T08:05:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31581.156235 secs 2023-09-10T08:05:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:50 smithi170 ceph-mon[102832]: pgmap v17002: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 737 KiB/s wr, 14 op/s 2023-09-10T08:05:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:50 smithi082 ceph-mon[98694]: pgmap v17002: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 737 KiB/s wr, 14 op/s 2023-09-10T08:05:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:50 smithi098 ceph-mon[103124]: pgmap v17002: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 737 KiB/s wr, 14 op/s 2023-09-10T08:05:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:52 smithi098 ceph-mon[103124]: pgmap v17003: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 23 op/s 2023-09-10T08:05:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:05:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:52 smithi170 ceph-mon[102832]: pgmap v17003: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 23 op/s 2023-09-10T08:05:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:05:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:52 smithi082 ceph-mon[98694]: pgmap v17003: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 23 op/s 2023-09-10T08:05:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:05:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31584.547564 secs 2023-09-10T08:05:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31585.712965 secs 2023-09-10T08:05:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31584.547564 secs 2023-09-10T08:05:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31585.712965 secs 2023-09-10T08:05:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31584.547564 secs 2023-09-10T08:05:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31585.712965 secs 2023-09-10T08:05:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:54 smithi170 ceph-mon[102832]: pgmap v17004: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 548 KiB/s wr, 15 op/s 2023-09-10T08:05:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:05:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:05:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31586.156303 secs 2023-09-10T08:05:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:05:55.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:05:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:54 smithi082 ceph-mon[98694]: pgmap v17004: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 548 KiB/s wr, 15 op/s 2023-09-10T08:05:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:05:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:05:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31586.156303 secs 2023-09-10T08:05:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:05:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:05:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:54 smithi098 ceph-mon[103124]: pgmap v17004: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 548 KiB/s wr, 15 op/s 2023-09-10T08:05:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:05:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:05:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31586.156303 secs 2023-09-10T08:05:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:05:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:05:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:05:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:05:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:05:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:05:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:05:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:05:56.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:05:56.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:05:56.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:05:56.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:55 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T08:05:56.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:55 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T08:05:56.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:05:56.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:05:56.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:05:56.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:05:56.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:55 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T08:05:56.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:55 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T08:05:56.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:05:56.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:05:56.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:05:56.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:55 smithi170 ceph-mon[102832]: pgmap v17005: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 549 KiB/s wr, 22 op/s 2023-09-10T08:05:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:05:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:05:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:05:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:05:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:05:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:05:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:05:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:05:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:05:56.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:55 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T08:05:56.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:55 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T08:05:56.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:05:56.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:05:56.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:05:56.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:05:56.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:55 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T08:05:56.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:55 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T08:05:56.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:05:56.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:05:56.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:05:56.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:55 smithi082 ceph-mon[98694]: pgmap v17005: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 549 KiB/s wr, 22 op/s 2023-09-10T08:05:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:05:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:05:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:05:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:05:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:05:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:05:56.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:05:56.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:05:56.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:05:56.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:55 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T08:05:56.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:55 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T08:05:56.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:05:56.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:05:56.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:05:56.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:05:56.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:55 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T08:05:56.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:55 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T08:05:56.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:05:56.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:05:56.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:05:56.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:55 smithi098 ceph-mon[103124]: pgmap v17005: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 549 KiB/s wr, 22 op/s 2023-09-10T08:05:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:05:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:05:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:05:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:05:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:05:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:05:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:05:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:05:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:05:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:58 smithi098 ceph-mon[103124]: pgmap v17006: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 414 KiB/s wr, 14 op/s 2023-09-10T08:05:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:58 smithi170 ceph-mon[102832]: pgmap v17006: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 414 KiB/s wr, 14 op/s 2023-09-10T08:05:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:58 smithi082 ceph-mon[98694]: pgmap v17006: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 414 KiB/s wr, 14 op/s 2023-09-10T08:05:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31589.547661 secs 2023-09-10T08:05:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31590.713023 secs 2023-09-10T08:05:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:05:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31591.156369 secs 2023-09-10T08:05:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31589.547661 secs 2023-09-10T08:05:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31590.713023 secs 2023-09-10T08:05:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:05:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31591.156369 secs 2023-09-10T08:05:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31589.547661 secs 2023-09-10T08:05:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31590.713023 secs 2023-09-10T08:05:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:05:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31591.156369 secs 2023-09-10T08:06:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:00 smithi170 ceph-mon[102832]: pgmap v17007: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 414 KiB/s wr, 14 op/s 2023-09-10T08:06:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:00 smithi082 ceph-mon[98694]: pgmap v17007: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 414 KiB/s wr, 14 op/s 2023-09-10T08:06:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:00 smithi098 ceph-mon[103124]: pgmap v17007: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 414 KiB/s wr, 14 op/s 2023-09-10T08:06:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:02 smithi098 ceph-mon[103124]: pgmap v17008: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 652 KiB/s wr, 20 op/s 2023-09-10T08:06:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:02 smithi170 ceph-mon[102832]: pgmap v17008: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 652 KiB/s wr, 20 op/s 2023-09-10T08:06:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:02 smithi082 ceph-mon[98694]: pgmap v17008: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 652 KiB/s wr, 20 op/s 2023-09-10T08:06:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31594.547723 secs 2023-09-10T08:06:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31595.713100 secs 2023-09-10T08:06:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31594.547723 secs 2023-09-10T08:06:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31595.713100 secs 2023-09-10T08:06:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31594.547723 secs 2023-09-10T08:06:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31595.713100 secs 2023-09-10T08:06:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:04 smithi170 ceph-mon[102832]: pgmap v17009: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 239 KiB/s wr, 11 op/s 2023-09-10T08:06:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31596.156434 secs 2023-09-10T08:06:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:04 smithi082 ceph-mon[98694]: pgmap v17009: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 239 KiB/s wr, 11 op/s 2023-09-10T08:06:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31596.156434 secs 2023-09-10T08:06:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:04 smithi098 ceph-mon[103124]: pgmap v17009: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 239 KiB/s wr, 11 op/s 2023-09-10T08:06:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31596.156434 secs 2023-09-10T08:06:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:06 smithi170 ceph-mon[102832]: pgmap v17010: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 372 KiB/s wr, 31 op/s 2023-09-10T08:06:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:06 smithi082 ceph-mon[98694]: pgmap v17010: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 372 KiB/s wr, 31 op/s 2023-09-10T08:06:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:06 smithi098 ceph-mon[103124]: pgmap v17010: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 372 KiB/s wr, 31 op/s 2023-09-10T08:06:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:08 smithi098 ceph-mon[103124]: pgmap v17011: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 371 KiB/s wr, 25 op/s 2023-09-10T08:06:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:08 smithi170 ceph-mon[102832]: pgmap v17011: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 371 KiB/s wr, 25 op/s 2023-09-10T08:06:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:08 smithi082 ceph-mon[98694]: pgmap v17011: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 371 KiB/s wr, 25 op/s 2023-09-10T08:06:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31599.547778 secs 2023-09-10T08:06:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31600.713159 secs 2023-09-10T08:06:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:09 smithi170 ceph-mon[102832]: osdmap e1131: 12 total, 12 up, 12 in 2023-09-10T08:06:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31601.156490 secs 2023-09-10T08:06:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31599.547778 secs 2023-09-10T08:06:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31600.713159 secs 2023-09-10T08:06:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:09 smithi082 ceph-mon[98694]: osdmap e1131: 12 total, 12 up, 12 in 2023-09-10T08:06:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31601.156490 secs 2023-09-10T08:06:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31599.547778 secs 2023-09-10T08:06:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31600.713159 secs 2023-09-10T08:06:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:09 smithi098 ceph-mon[103124]: osdmap e1131: 12 total, 12 up, 12 in 2023-09-10T08:06:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31601.156490 secs 2023-09-10T08:06:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:10 smithi170 ceph-mon[102832]: pgmap v17013: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 446 KiB/s wr, 30 op/s 2023-09-10T08:06:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:10 smithi082 ceph-mon[98694]: pgmap v17013: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 446 KiB/s wr, 30 op/s 2023-09-10T08:06:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:10 smithi098 ceph-mon[103124]: pgmap v17013: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 446 KiB/s wr, 30 op/s 2023-09-10T08:06:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:12 smithi170 ceph-mon[102832]: pgmap v17014: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 160 KiB/s wr, 30 op/s 2023-09-10T08:06:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:12 smithi170 ceph-mon[102832]: osdmap e1132: 12 total, 12 up, 12 in 2023-09-10T08:06:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:12 smithi082 ceph-mon[98694]: pgmap v17014: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 160 KiB/s wr, 30 op/s 2023-09-10T08:06:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:12 smithi082 ceph-mon[98694]: osdmap e1132: 12 total, 12 up, 12 in 2023-09-10T08:06:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:12 smithi098 ceph-mon[103124]: pgmap v17014: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 160 KiB/s wr, 30 op/s 2023-09-10T08:06:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:12 smithi098 ceph-mon[103124]: osdmap e1132: 12 total, 12 up, 12 in 2023-09-10T08:06:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31604.547860 secs 2023-09-10T08:06:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31605.713218 secs 2023-09-10T08:06:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31604.547860 secs 2023-09-10T08:06:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31605.713218 secs 2023-09-10T08:06:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31604.547860 secs 2023-09-10T08:06:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31605.713218 secs 2023-09-10T08:06:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:14 smithi170 ceph-mon[102832]: pgmap v17016: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T08:06:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31606.156557 secs 2023-09-10T08:06:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:14 smithi082 ceph-mon[98694]: pgmap v17016: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T08:06:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31606.156557 secs 2023-09-10T08:06:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:14 smithi098 ceph-mon[103124]: pgmap v17016: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T08:06:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31606.156557 secs 2023-09-10T08:06:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:16 smithi170 ceph-mon[102832]: pgmap v17017: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 15 op/s 2023-09-10T08:06:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:06:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:06:17.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:16 smithi082 ceph-mon[98694]: pgmap v17017: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 15 op/s 2023-09-10T08:06:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:06:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:06:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:16 smithi098 ceph-mon[103124]: pgmap v17017: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 15 op/s 2023-09-10T08:06:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:06:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:06:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:18 smithi170 ceph-mon[102832]: pgmap v17018: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 668 B/s wr, 13 op/s 2023-09-10T08:06:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:06:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:06:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31609.547919 secs 2023-09-10T08:06:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31610.713275 secs 2023-09-10T08:06:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:18 smithi082 ceph-mon[98694]: pgmap v17018: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 668 B/s wr, 13 op/s 2023-09-10T08:06:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:06:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:06:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31609.547919 secs 2023-09-10T08:06:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31610.713275 secs 2023-09-10T08:06:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:18 smithi098 ceph-mon[103124]: pgmap v17018: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 668 B/s wr, 13 op/s 2023-09-10T08:06:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:06:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:06:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31609.547919 secs 2023-09-10T08:06:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31610.713275 secs 2023-09-10T08:06:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31611.156623 secs 2023-09-10T08:06:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31611.156623 secs 2023-09-10T08:06:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31611.156623 secs 2023-09-10T08:06:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:20 smithi170 ceph-mon[102832]: pgmap v17019: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 614 B/s wr, 12 op/s 2023-09-10T08:06:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:20 smithi082 ceph-mon[98694]: pgmap v17019: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 614 B/s wr, 12 op/s 2023-09-10T08:06:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:20 smithi098 ceph-mon[103124]: pgmap v17019: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 614 B/s wr, 12 op/s 2023-09-10T08:06:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:22 smithi170 ceph-mon[102832]: pgmap v17020: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 921 B/s wr, 12 op/s 2023-09-10T08:06:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:22 smithi082 ceph-mon[98694]: pgmap v17020: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 921 B/s wr, 12 op/s 2023-09-10T08:06:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:22 smithi098 ceph-mon[103124]: pgmap v17020: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 921 B/s wr, 12 op/s 2023-09-10T08:06:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31614.547980 secs 2023-09-10T08:06:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31615.713337 secs 2023-09-10T08:06:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31614.547980 secs 2023-09-10T08:06:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31615.713337 secs 2023-09-10T08:06:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31614.547980 secs 2023-09-10T08:06:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31615.713337 secs 2023-09-10T08:06:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:24 smithi170 ceph-mon[102832]: pgmap v17021: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 822 B/s wr, 11 op/s 2023-09-10T08:06:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31616.156711 secs 2023-09-10T08:06:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:24 smithi082 ceph-mon[98694]: pgmap v17021: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 822 B/s wr, 11 op/s 2023-09-10T08:06:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31616.156711 secs 2023-09-10T08:06:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:24 smithi098 ceph-mon[103124]: pgmap v17021: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 822 B/s wr, 11 op/s 2023-09-10T08:06:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31616.156711 secs 2023-09-10T08:06:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:26 smithi170 ceph-mon[102832]: pgmap v17022: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 111 KiB/s wr, 15 op/s 2023-09-10T08:06:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:26 smithi082 ceph-mon[98694]: pgmap v17022: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 111 KiB/s wr, 15 op/s 2023-09-10T08:06:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:26 smithi098 ceph-mon[103124]: pgmap v17022: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 111 KiB/s wr, 15 op/s 2023-09-10T08:06:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:28 smithi170 ceph-mon[102832]: pgmap v17023: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 10 op/s 2023-09-10T08:06:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:28 smithi082 ceph-mon[98694]: pgmap v17023: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 10 op/s 2023-09-10T08:06:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:28 smithi098 ceph-mon[103124]: pgmap v17023: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 10 op/s 2023-09-10T08:06:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31619.548095 secs 2023-09-10T08:06:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31620.713390 secs 2023-09-10T08:06:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31621.156778 secs 2023-09-10T08:06:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31619.548095 secs 2023-09-10T08:06:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31620.713390 secs 2023-09-10T08:06:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31621.156778 secs 2023-09-10T08:06:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31619.548095 secs 2023-09-10T08:06:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31620.713390 secs 2023-09-10T08:06:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31621.156778 secs 2023-09-10T08:06:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:30 smithi170 ceph-mon[102832]: pgmap v17024: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 10 op/s 2023-09-10T08:06:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:30 smithi082 ceph-mon[98694]: pgmap v17024: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 10 op/s 2023-09-10T08:06:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:30 smithi098 ceph-mon[103124]: pgmap v17024: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 10 op/s 2023-09-10T08:06:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:32 smithi170 ceph-mon[102832]: pgmap v17025: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 111 KiB/s wr, 15 op/s 2023-09-10T08:06:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:32 smithi082 ceph-mon[98694]: pgmap v17025: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 111 KiB/s wr, 15 op/s 2023-09-10T08:06:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:32 smithi098 ceph-mon[103124]: pgmap v17025: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 111 KiB/s wr, 15 op/s 2023-09-10T08:06:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31625.713450 secs 2023-09-10T08:06:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31625.713450 secs 2023-09-10T08:06:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31625.713450 secs 2023-09-10T08:06:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:34 smithi170 ceph-mon[102832]: pgmap v17026: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 10 op/s 2023-09-10T08:06:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31626.156845 secs 2023-09-10T08:06:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:34 smithi082 ceph-mon[98694]: pgmap v17026: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 10 op/s 2023-09-10T08:06:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31626.156845 secs 2023-09-10T08:06:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:34 smithi098 ceph-mon[103124]: pgmap v17026: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 111 KiB/s wr, 10 op/s 2023-09-10T08:06:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31626.156845 secs 2023-09-10T08:06:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:06:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:06:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:06:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:36 smithi170 ceph-mon[102832]: pgmap v17027: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 304 KiB/s wr, 15 op/s 2023-09-10T08:06:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:06:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:06:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:06:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:36 smithi082 ceph-mon[98694]: pgmap v17027: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 304 KiB/s wr, 15 op/s 2023-09-10T08:06:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:06:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:06:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:06:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:36 smithi098 ceph-mon[103124]: pgmap v17027: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 304 KiB/s wr, 15 op/s 2023-09-10T08:06:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:06:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:06:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:06:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:38 smithi170 ceph-mon[102832]: pgmap v17028: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 193 KiB/s wr, 10 op/s 2023-09-10T08:06:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:38 smithi082 ceph-mon[98694]: pgmap v17028: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 193 KiB/s wr, 10 op/s 2023-09-10T08:06:39.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:38 smithi098 ceph-mon[103124]: pgmap v17028: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 193 KiB/s wr, 10 op/s 2023-09-10T08:06:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31630.713525 secs 2023-09-10T08:06:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31631.156914 secs 2023-09-10T08:06:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31630.713525 secs 2023-09-10T08:06:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31631.156914 secs 2023-09-10T08:06:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31630.713525 secs 2023-09-10T08:06:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31631.156914 secs 2023-09-10T08:06:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:40 smithi170 ceph-mon[102832]: pgmap v17029: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 193 KiB/s wr, 10 op/s 2023-09-10T08:06:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:40 smithi082 ceph-mon[98694]: pgmap v17029: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 193 KiB/s wr, 10 op/s 2023-09-10T08:06:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:40 smithi098 ceph-mon[103124]: pgmap v17029: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 193 KiB/s wr, 10 op/s 2023-09-10T08:06:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:42 smithi170 ceph-mon[102832]: pgmap v17030: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 345 KiB/s wr, 16 op/s 2023-09-10T08:06:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:42 smithi082 ceph-mon[98694]: pgmap v17030: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 345 KiB/s wr, 16 op/s 2023-09-10T08:06:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:42 smithi098 ceph-mon[103124]: pgmap v17030: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 345 KiB/s wr, 16 op/s 2023-09-10T08:06:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31635.713595 secs 2023-09-10T08:06:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31635.713595 secs 2023-09-10T08:06:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31635.713595 secs 2023-09-10T08:06:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:44 smithi170 ceph-mon[102832]: pgmap v17031: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 345 KiB/s wr, 11 op/s 2023-09-10T08:06:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:06:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:06:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31636.156979 secs 2023-09-10T08:06:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:44 smithi082 ceph-mon[98694]: pgmap v17031: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 345 KiB/s wr, 11 op/s 2023-09-10T08:06:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:06:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:06:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31636.156979 secs 2023-09-10T08:06:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:44 smithi098 ceph-mon[103124]: pgmap v17031: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 345 KiB/s wr, 11 op/s 2023-09-10T08:06:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:06:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:06:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31636.156979 secs 2023-09-10T08:06:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:46 smithi170 ceph-mon[102832]: pgmap v17032: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 531 KiB/s wr, 18 op/s 2023-09-10T08:06:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:06:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:06:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:46 smithi082 ceph-mon[98694]: pgmap v17032: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 531 KiB/s wr, 18 op/s 2023-09-10T08:06:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:06:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:06:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:46 smithi098 ceph-mon[103124]: pgmap v17032: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 531 KiB/s wr, 18 op/s 2023-09-10T08:06:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:06:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:06:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:48 smithi170 ceph-mon[102832]: pgmap v17033: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 339 KiB/s wr, 13 op/s 2023-09-10T08:06:48.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:48 smithi082 ceph-mon[98694]: pgmap v17033: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 339 KiB/s wr, 13 op/s 2023-09-10T08:06:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:48 smithi098 ceph-mon[103124]: pgmap v17033: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 339 KiB/s wr, 13 op/s 2023-09-10T08:06:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31640.713653 secs 2023-09-10T08:06:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31641.157043 secs 2023-09-10T08:06:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31640.713653 secs 2023-09-10T08:06:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31641.157043 secs 2023-09-10T08:06:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31640.713653 secs 2023-09-10T08:06:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31641.157043 secs 2023-09-10T08:06:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:06:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:06:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:50 smithi170 ceph-mon[102832]: pgmap v17034: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 339 KiB/s wr, 13 op/s 2023-09-10T08:06:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:06:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:06:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:50 smithi082 ceph-mon[98694]: pgmap v17034: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 339 KiB/s wr, 13 op/s 2023-09-10T08:06:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:06:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:06:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:50 smithi098 ceph-mon[103124]: pgmap v17034: 97 pgs: 97 active+clean; 10 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 339 KiB/s wr, 13 op/s 2023-09-10T08:06:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:06:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:06:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:52 smithi170 ceph-mon[102832]: pgmap v17035: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 624 KiB/s wr, 20 op/s 2023-09-10T08:06:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:06:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:06:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:52 smithi082 ceph-mon[98694]: pgmap v17035: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 624 KiB/s wr, 20 op/s 2023-09-10T08:06:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:06:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:06:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:52 smithi098 ceph-mon[103124]: pgmap v17035: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 624 KiB/s wr, 20 op/s 2023-09-10T08:06:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31645.713742 secs 2023-09-10T08:06:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31645.713742 secs 2023-09-10T08:06:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31645.713742 secs 2023-09-10T08:06:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:54 smithi170 ceph-mon[102832]: pgmap v17036: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 471 KiB/s wr, 13 op/s 2023-09-10T08:06:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31646.157109 secs 2023-09-10T08:06:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:54 smithi082 ceph-mon[98694]: pgmap v17036: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 471 KiB/s wr, 13 op/s 2023-09-10T08:06:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31646.157109 secs 2023-09-10T08:06:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:54 smithi098 ceph-mon[103124]: pgmap v17036: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 471 KiB/s wr, 13 op/s 2023-09-10T08:06:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31646.157109 secs 2023-09-10T08:06:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:56 smithi170 ceph-mon[102832]: pgmap v17037: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 604 KiB/s wr, 20 op/s 2023-09-10T08:06:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:06:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:06:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:06:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:06:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:56 smithi082 ceph-mon[98694]: pgmap v17037: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 604 KiB/s wr, 20 op/s 2023-09-10T08:06:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:06:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:06:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:06:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:06:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:56 smithi098 ceph-mon[103124]: pgmap v17037: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 604 KiB/s wr, 20 op/s 2023-09-10T08:06:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:06:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:06:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:06:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:06:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:58 smithi170 ceph-mon[102832]: pgmap v17038: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 418 KiB/s wr, 13 op/s 2023-09-10T08:06:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:06:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:06:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31650.713800 secs 2023-09-10T08:06:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:58 smithi082 ceph-mon[98694]: pgmap v17038: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 418 KiB/s wr, 13 op/s 2023-09-10T08:06:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:06:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:06:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31650.713800 secs 2023-09-10T08:06:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:58 smithi098 ceph-mon[103124]: pgmap v17038: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 418 KiB/s wr, 13 op/s 2023-09-10T08:06:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:06:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:06:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31650.713800 secs 2023-09-10T08:07:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:06:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31651.157177 secs 2023-09-10T08:07:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:06:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31651.157177 secs 2023-09-10T08:07:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:06:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31651.157177 secs 2023-09-10T08:07:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:00 smithi170 ceph-mon[102832]: pgmap v17039: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 418 KiB/s wr, 13 op/s 2023-09-10T08:07:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:07:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:00 smithi082 ceph-mon[98694]: pgmap v17039: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 418 KiB/s wr, 13 op/s 2023-09-10T08:07:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:07:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:00 smithi098 ceph-mon[103124]: pgmap v17039: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 418 KiB/s wr, 13 op/s 2023-09-10T08:07:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:07:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:02 smithi170 ceph-mon[102832]: pgmap v17040: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 949 KiB/s wr, 20 op/s 2023-09-10T08:07:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:02 smithi082 ceph-mon[98694]: pgmap v17040: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 949 KiB/s wr, 20 op/s 2023-09-10T08:07:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:02 smithi098 ceph-mon[103124]: pgmap v17040: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 949 KiB/s wr, 20 op/s 2023-09-10T08:07:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31654.548604 secs 2023-09-10T08:07:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31655.713863 secs 2023-09-10T08:07:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31654.548604 secs 2023-09-10T08:07:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31655.713863 secs 2023-09-10T08:07:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31654.548604 secs 2023-09-10T08:07:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31655.713863 secs 2023-09-10T08:07:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:04 smithi170 ceph-mon[102832]: pgmap v17041: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 665 KiB/s wr, 13 op/s 2023-09-10T08:07:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:07:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:07:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31656.157241 secs 2023-09-10T08:07:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:04 smithi082 ceph-mon[98694]: pgmap v17041: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 665 KiB/s wr, 13 op/s 2023-09-10T08:07:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:07:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:07:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31656.157241 secs 2023-09-10T08:07:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:04 smithi098 ceph-mon[103124]: pgmap v17041: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 665 KiB/s wr, 13 op/s 2023-09-10T08:07:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:07:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:07:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31656.157241 secs 2023-09-10T08:07:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:07:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:07:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:07:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:06 smithi170 ceph-mon[102832]: pgmap v17042: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 928 KiB/s wr, 34 op/s 2023-09-10T08:07:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:06 smithi082 ceph-mon[98694]: pgmap v17042: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 928 KiB/s wr, 34 op/s 2023-09-10T08:07:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:06 smithi098 ceph-mon[103124]: pgmap v17042: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 928 KiB/s wr, 34 op/s 2023-09-10T08:07:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:08 smithi170 ceph-mon[102832]: pgmap v17043: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 795 KiB/s wr, 28 op/s 2023-09-10T08:07:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:08 smithi082 ceph-mon[98694]: pgmap v17043: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 795 KiB/s wr, 28 op/s 2023-09-10T08:07:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:08 smithi098 ceph-mon[103124]: pgmap v17043: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 795 KiB/s wr, 28 op/s 2023-09-10T08:07:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31659.548704 secs 2023-09-10T08:07:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31660.713924 secs 2023-09-10T08:07:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:09 smithi170 ceph-mon[102832]: osdmap e1133: 12 total, 12 up, 12 in 2023-09-10T08:07:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31661.157331 secs 2023-09-10T08:07:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31659.548704 secs 2023-09-10T08:07:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31660.713924 secs 2023-09-10T08:07:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:09 smithi082 ceph-mon[98694]: osdmap e1133: 12 total, 12 up, 12 in 2023-09-10T08:07:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31661.157331 secs 2023-09-10T08:07:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31659.548704 secs 2023-09-10T08:07:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31660.713924 secs 2023-09-10T08:07:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:09 smithi098 ceph-mon[103124]: osdmap e1133: 12 total, 12 up, 12 in 2023-09-10T08:07:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31661.157331 secs 2023-09-10T08:07:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:07:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:10 smithi170 ceph-mon[102832]: pgmap v17045: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 954 KiB/s wr, 34 op/s 2023-09-10T08:07:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:07:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:10 smithi082 ceph-mon[98694]: pgmap v17045: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 954 KiB/s wr, 34 op/s 2023-09-10T08:07:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:07:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:10 smithi098 ceph-mon[103124]: pgmap v17045: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 954 KiB/s wr, 34 op/s 2023-09-10T08:07:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:07:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:12 smithi170 ceph-mon[102832]: pgmap v17046: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 627 KiB/s wr, 32 op/s 2023-09-10T08:07:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:07:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:12 smithi082 ceph-mon[98694]: pgmap v17046: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 627 KiB/s wr, 32 op/s 2023-09-10T08:07:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:07:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:12 smithi098 ceph-mon[103124]: pgmap v17046: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 627 KiB/s wr, 32 op/s 2023-09-10T08:07:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31665.714009 secs 2023-09-10T08:07:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31665.714009 secs 2023-09-10T08:07:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31665.714009 secs 2023-09-10T08:07:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31664.548818 secs 2023-09-10T08:07:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:14 smithi170 ceph-mon[102832]: pgmap v17047: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 627 KiB/s wr, 32 op/s 2023-09-10T08:07:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31666.157395 secs 2023-09-10T08:07:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31664.548818 secs 2023-09-10T08:07:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:14 smithi082 ceph-mon[98694]: pgmap v17047: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 627 KiB/s wr, 32 op/s 2023-09-10T08:07:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31666.157395 secs 2023-09-10T08:07:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31664.548818 secs 2023-09-10T08:07:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:14 smithi098 ceph-mon[103124]: pgmap v17047: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 627 KiB/s wr, 32 op/s 2023-09-10T08:07:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31666.157395 secs 2023-09-10T08:07:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:16 smithi170 ceph-mon[102832]: pgmap v17048: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 14 op/s 2023-09-10T08:07:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:16 smithi082 ceph-mon[98694]: pgmap v17048: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 14 op/s 2023-09-10T08:07:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:16 smithi098 ceph-mon[103124]: pgmap v17048: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 14 op/s 2023-09-10T08:07:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:18 smithi170 ceph-mon[102832]: osdmap e1134: 12 total, 12 up, 12 in 2023-09-10T08:07:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:18 smithi170 ceph-mon[102832]: pgmap v17050: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 484 KiB/s wr, 16 op/s 2023-09-10T08:07:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:18 smithi082 ceph-mon[98694]: osdmap e1134: 12 total, 12 up, 12 in 2023-09-10T08:07:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:18 smithi082 ceph-mon[98694]: pgmap v17050: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 484 KiB/s wr, 16 op/s 2023-09-10T08:07:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:18 smithi098 ceph-mon[103124]: osdmap e1134: 12 total, 12 up, 12 in 2023-09-10T08:07:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:18 smithi098 ceph-mon[103124]: pgmap v17050: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 484 KiB/s wr, 16 op/s 2023-09-10T08:07:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31669.548876 secs 2023-09-10T08:07:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31670.714070 secs 2023-09-10T08:07:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31671.157462 secs 2023-09-10T08:07:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31669.548876 secs 2023-09-10T08:07:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31670.714070 secs 2023-09-10T08:07:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31671.157462 secs 2023-09-10T08:07:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31669.548876 secs 2023-09-10T08:07:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31670.714070 secs 2023-09-10T08:07:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31671.157462 secs 2023-09-10T08:07:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:20 smithi170 ceph-mon[102832]: pgmap v17051: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 14 op/s 2023-09-10T08:07:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:20 smithi082 ceph-mon[98694]: pgmap v17051: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 14 op/s 2023-09-10T08:07:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:20 smithi098 ceph-mon[103124]: pgmap v17051: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 14 op/s 2023-09-10T08:07:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:22 smithi170 ceph-mon[102832]: pgmap v17052: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 14 op/s 2023-09-10T08:07:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:22 smithi082 ceph-mon[98694]: pgmap v17052: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 14 op/s 2023-09-10T08:07:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:22 smithi098 ceph-mon[103124]: pgmap v17052: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 14 op/s 2023-09-10T08:07:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31674.548942 secs 2023-09-10T08:07:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31675.714152 secs 2023-09-10T08:07:24.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31674.548942 secs 2023-09-10T08:07:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31675.714152 secs 2023-09-10T08:07:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31674.548942 secs 2023-09-10T08:07:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31675.714152 secs 2023-09-10T08:07:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:24 smithi170 ceph-mon[102832]: pgmap v17053: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 14 op/s 2023-09-10T08:07:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31676.157492 secs 2023-09-10T08:07:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:24 smithi082 ceph-mon[98694]: pgmap v17053: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 14 op/s 2023-09-10T08:07:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31676.157492 secs 2023-09-10T08:07:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:24 smithi098 ceph-mon[103124]: pgmap v17053: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 14 op/s 2023-09-10T08:07:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31676.157492 secs 2023-09-10T08:07:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:26 smithi170 ceph-mon[102832]: pgmap v17054: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2023-09-10T08:07:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:26 smithi082 ceph-mon[98694]: pgmap v17054: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2023-09-10T08:07:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:26 smithi098 ceph-mon[103124]: pgmap v17054: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2023-09-10T08:07:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:28 smithi170 ceph-mon[102832]: pgmap v17055: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2023-09-10T08:07:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:28 smithi082 ceph-mon[98694]: pgmap v17055: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2023-09-10T08:07:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:28 smithi098 ceph-mon[103124]: pgmap v17055: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2023-09-10T08:07:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31679.549003 secs 2023-09-10T08:07:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31680.714204 secs 2023-09-10T08:07:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31681.157556 secs 2023-09-10T08:07:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31679.549003 secs 2023-09-10T08:07:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31680.714204 secs 2023-09-10T08:07:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31681.157556 secs 2023-09-10T08:07:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31679.549003 secs 2023-09-10T08:07:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31680.714204 secs 2023-09-10T08:07:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31681.157556 secs 2023-09-10T08:07:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:07:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:30 smithi170 ceph-mon[102832]: pgmap v17056: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2023-09-10T08:07:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:07:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:30 smithi082 ceph-mon[98694]: pgmap v17056: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2023-09-10T08:07:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:07:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:30 smithi098 ceph-mon[103124]: pgmap v17056: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2023-09-10T08:07:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:07:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:32 smithi170 ceph-mon[102832]: pgmap v17057: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 17 op/s 2023-09-10T08:07:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:07:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:32 smithi082 ceph-mon[98694]: pgmap v17057: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 17 op/s 2023-09-10T08:07:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:07:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:32 smithi098 ceph-mon[103124]: pgmap v17057: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 17 op/s 2023-09-10T08:07:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31685.714280 secs 2023-09-10T08:07:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31685.714280 secs 2023-09-10T08:07:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31685.714280 secs 2023-09-10T08:07:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31684.549064 secs 2023-09-10T08:07:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:34 smithi170 ceph-mon[102832]: pgmap v17058: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T08:07:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31686.157620 secs 2023-09-10T08:07:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31684.549064 secs 2023-09-10T08:07:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:34 smithi082 ceph-mon[98694]: pgmap v17058: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T08:07:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31686.157620 secs 2023-09-10T08:07:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31684.549064 secs 2023-09-10T08:07:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:34 smithi098 ceph-mon[103124]: pgmap v17058: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 11 op/s 2023-09-10T08:07:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31686.157620 secs 2023-09-10T08:07:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:36 smithi170 ceph-mon[102832]: pgmap v17059: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 221 KiB/s wr, 16 op/s 2023-09-10T08:07:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:36 smithi082 ceph-mon[98694]: pgmap v17059: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 221 KiB/s wr, 16 op/s 2023-09-10T08:07:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:36 smithi098 ceph-mon[103124]: pgmap v17059: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 221 KiB/s wr, 16 op/s 2023-09-10T08:07:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:07:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:07:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:07:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:38 smithi170 ceph-mon[102832]: pgmap v17060: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 221 KiB/s wr, 10 op/s 2023-09-10T08:07:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31690.714348 secs 2023-09-10T08:07:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:38 smithi082 ceph-mon[98694]: pgmap v17060: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 221 KiB/s wr, 10 op/s 2023-09-10T08:07:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31690.714348 secs 2023-09-10T08:07:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:38 smithi098 ceph-mon[103124]: pgmap v17060: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 221 KiB/s wr, 10 op/s 2023-09-10T08:07:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31690.714348 secs 2023-09-10T08:07:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31691.157687 secs 2023-09-10T08:07:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:07:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31691.157687 secs 2023-09-10T08:07:40.235 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:07:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31691.157687 secs 2023-09-10T08:07:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:07:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:40 smithi170 ceph-mon[102832]: pgmap v17061: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 221 KiB/s wr, 10 op/s 2023-09-10T08:07:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:07:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:07:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:40 smithi082 ceph-mon[98694]: pgmap v17061: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 221 KiB/s wr, 10 op/s 2023-09-10T08:07:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:07:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:07:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:40 smithi098 ceph-mon[103124]: pgmap v17061: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 221 KiB/s wr, 10 op/s 2023-09-10T08:07:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:07:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:07:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:42 smithi170 ceph-mon[102832]: pgmap v17062: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 809 KiB/s wr, 19 op/s 2023-09-10T08:07:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:42 smithi082 ceph-mon[98694]: pgmap v17062: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 809 KiB/s wr, 19 op/s 2023-09-10T08:07:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:42 smithi098 ceph-mon[103124]: pgmap v17062: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 809 KiB/s wr, 19 op/s 2023-09-10T08:07:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31695.714431 secs 2023-09-10T08:07:44.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31695.714431 secs 2023-09-10T08:07:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31695.714431 secs 2023-09-10T08:07:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:44 smithi170 ceph-mon[102832]: pgmap v17063: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 697 KiB/s wr, 13 op/s 2023-09-10T08:07:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:07:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31696.157751 secs 2023-09-10T08:07:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:44 smithi082 ceph-mon[98694]: pgmap v17063: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 697 KiB/s wr, 13 op/s 2023-09-10T08:07:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:07:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31696.157751 secs 2023-09-10T08:07:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:44 smithi098 ceph-mon[103124]: pgmap v17063: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 697 KiB/s wr, 13 op/s 2023-09-10T08:07:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:07:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31696.157751 secs 2023-09-10T08:07:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:07:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:45 smithi170 ceph-mon[102832]: pgmap v17064: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 717 KiB/s wr, 19 op/s 2023-09-10T08:07:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:07:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:45 smithi082 ceph-mon[98694]: pgmap v17064: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 717 KiB/s wr, 19 op/s 2023-09-10T08:07:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:07:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:45 smithi098 ceph-mon[103124]: pgmap v17064: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 717 KiB/s wr, 19 op/s 2023-09-10T08:07:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:48 smithi170 ceph-mon[102832]: pgmap v17065: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 609 KiB/s wr, 14 op/s 2023-09-10T08:07:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:48 smithi082 ceph-mon[98694]: pgmap v17065: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 609 KiB/s wr, 14 op/s 2023-09-10T08:07:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:48 smithi098 ceph-mon[103124]: pgmap v17065: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 609 KiB/s wr, 14 op/s 2023-09-10T08:07:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31700.714520 secs 2023-09-10T08:07:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31701.157817 secs 2023-09-10T08:07:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31700.714520 secs 2023-09-10T08:07:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31701.157817 secs 2023-09-10T08:07:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31700.714520 secs 2023-09-10T08:07:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31701.157817 secs 2023-09-10T08:07:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:50 smithi170 ceph-mon[102832]: pgmap v17066: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 609 KiB/s wr, 14 op/s 2023-09-10T08:07:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:50 smithi082 ceph-mon[98694]: pgmap v17066: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 609 KiB/s wr, 14 op/s 2023-09-10T08:07:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:50 smithi098 ceph-mon[103124]: pgmap v17066: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 609 KiB/s wr, 14 op/s 2023-09-10T08:07:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:52 smithi170 ceph-mon[102832]: pgmap v17067: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1000 KiB/s wr, 21 op/s 2023-09-10T08:07:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:52 smithi082 ceph-mon[98694]: pgmap v17067: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1000 KiB/s wr, 21 op/s 2023-09-10T08:07:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:52 smithi098 ceph-mon[103124]: pgmap v17067: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1000 KiB/s wr, 21 op/s 2023-09-10T08:07:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31705.714579 secs 2023-09-10T08:07:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:07:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:54 smithi170 ceph-mon[102832]: pgmap v17068: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 411 KiB/s wr, 12 op/s 2023-09-10T08:07:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31706.157882 secs 2023-09-10T08:07:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31705.714579 secs 2023-09-10T08:07:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:07:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:54 smithi082 ceph-mon[98694]: pgmap v17068: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 411 KiB/s wr, 12 op/s 2023-09-10T08:07:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31706.157882 secs 2023-09-10T08:07:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31705.714579 secs 2023-09-10T08:07:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:07:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:54 smithi098 ceph-mon[103124]: pgmap v17068: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 411 KiB/s wr, 12 op/s 2023-09-10T08:07:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31706.157882 secs 2023-09-10T08:07:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:07:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:56 smithi170 ceph-mon[102832]: pgmap v17069: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 906 KiB/s wr, 20 op/s 2023-09-10T08:07:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:07:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:07:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:56 smithi082 ceph-mon[98694]: pgmap v17069: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 906 KiB/s wr, 20 op/s 2023-09-10T08:07:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:07:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:07:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:07:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:56 smithi098 ceph-mon[103124]: pgmap v17069: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 906 KiB/s wr, 20 op/s 2023-09-10T08:07:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:07:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:07:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:07:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:07:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:07:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:07:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:57 smithi170 ceph-mon[102832]: pgmap v17070: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 886 KiB/s wr, 14 op/s 2023-09-10T08:07:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:07:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:07:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:07:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:07:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:07:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:57 smithi082 ceph-mon[98694]: pgmap v17070: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 886 KiB/s wr, 14 op/s 2023-09-10T08:07:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:07:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:07:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:07:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:07:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:07:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:57 smithi098 ceph-mon[103124]: pgmap v17070: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 886 KiB/s wr, 14 op/s 2023-09-10T08:07:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31709.549448 secs 2023-09-10T08:07:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31710.714636 secs 2023-09-10T08:07:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:07:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31711.157947 secs 2023-09-10T08:07:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31709.549448 secs 2023-09-10T08:07:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31710.714636 secs 2023-09-10T08:07:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:07:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31711.157947 secs 2023-09-10T08:08:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31709.549448 secs 2023-09-10T08:08:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31710.714636 secs 2023-09-10T08:08:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:07:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31711.157947 secs 2023-09-10T08:08:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:08:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:08:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:00 smithi170 ceph-mon[102832]: pgmap v17071: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 886 KiB/s wr, 14 op/s 2023-09-10T08:08:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:08:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:08:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:00 smithi082 ceph-mon[98694]: pgmap v17071: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 886 KiB/s wr, 14 op/s 2023-09-10T08:08:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:08:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:08:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:00 smithi098 ceph-mon[103124]: pgmap v17071: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 886 KiB/s wr, 14 op/s 2023-09-10T08:08:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:02 smithi170 ceph-mon[102832]: pgmap v17072: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 993 KiB/s wr, 20 op/s 2023-09-10T08:08:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:02 smithi082 ceph-mon[98694]: pgmap v17072: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 993 KiB/s wr, 20 op/s 2023-09-10T08:08:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:02 smithi098 ceph-mon[103124]: pgmap v17072: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 993 KiB/s wr, 20 op/s 2023-09-10T08:08:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31714.549551 secs 2023-09-10T08:08:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31715.714723 secs 2023-09-10T08:08:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31714.549551 secs 2023-09-10T08:08:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31715.714723 secs 2023-09-10T08:08:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31714.549551 secs 2023-09-10T08:08:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31715.714723 secs 2023-09-10T08:08:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:04 smithi170 ceph-mon[102832]: pgmap v17073: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 602 KiB/s wr, 13 op/s 2023-09-10T08:08:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:08:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31716.158013 secs 2023-09-10T08:08:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:04 smithi082 ceph-mon[98694]: pgmap v17073: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 602 KiB/s wr, 13 op/s 2023-09-10T08:08:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:08:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31716.158013 secs 2023-09-10T08:08:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:04 smithi098 ceph-mon[103124]: pgmap v17073: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 602 KiB/s wr, 13 op/s 2023-09-10T08:08:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:08:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31716.158013 secs 2023-09-10T08:08:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:08:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:08:06.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:08:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:06 smithi170 ceph-mon[102832]: pgmap v17074: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 804 KiB/s wr, 34 op/s 2023-09-10T08:08:07.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:06 smithi082 ceph-mon[98694]: pgmap v17074: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 804 KiB/s wr, 34 op/s 2023-09-10T08:08:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:06 smithi098 ceph-mon[103124]: pgmap v17074: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 804 KiB/s wr, 34 op/s 2023-09-10T08:08:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:08 smithi170 ceph-mon[102832]: pgmap v17075: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 309 KiB/s wr, 26 op/s 2023-09-10T08:08:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:08 smithi082 ceph-mon[98694]: pgmap v17075: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 309 KiB/s wr, 26 op/s 2023-09-10T08:08:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:08 smithi098 ceph-mon[103124]: pgmap v17075: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 309 KiB/s wr, 26 op/s 2023-09-10T08:08:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31719.549610 secs 2023-09-10T08:08:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31720.714784 secs 2023-09-10T08:08:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:09 smithi170 ceph-mon[102832]: osdmap e1135: 12 total, 12 up, 12 in 2023-09-10T08:08:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31721.158081 secs 2023-09-10T08:08:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31719.549610 secs 2023-09-10T08:08:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31720.714784 secs 2023-09-10T08:08:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:09 smithi082 ceph-mon[98694]: osdmap e1135: 12 total, 12 up, 12 in 2023-09-10T08:08:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31721.158081 secs 2023-09-10T08:08:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31719.549610 secs 2023-09-10T08:08:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31720.714784 secs 2023-09-10T08:08:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:09 smithi098 ceph-mon[103124]: osdmap e1135: 12 total, 12 up, 12 in 2023-09-10T08:08:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31721.158081 secs 2023-09-10T08:08:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:10 smithi170 ceph-mon[102832]: pgmap v17077: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 370 KiB/s wr, 32 op/s 2023-09-10T08:08:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:10 smithi082 ceph-mon[98694]: pgmap v17077: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 370 KiB/s wr, 32 op/s 2023-09-10T08:08:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:10 smithi098 ceph-mon[103124]: pgmap v17077: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 370 KiB/s wr, 32 op/s 2023-09-10T08:08:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:12 smithi170 ceph-mon[102832]: pgmap v17078: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 349 KiB/s wr, 31 op/s 2023-09-10T08:08:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:12 smithi170 ceph-mon[102832]: osdmap e1136: 12 total, 12 up, 12 in 2023-09-10T08:08:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:12 smithi082 ceph-mon[98694]: pgmap v17078: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 349 KiB/s wr, 31 op/s 2023-09-10T08:08:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:12 smithi082 ceph-mon[98694]: osdmap e1136: 12 total, 12 up, 12 in 2023-09-10T08:08:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:12 smithi098 ceph-mon[103124]: pgmap v17078: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 349 KiB/s wr, 31 op/s 2023-09-10T08:08:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:12 smithi098 ceph-mon[103124]: osdmap e1136: 12 total, 12 up, 12 in 2023-09-10T08:08:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31724.549718 secs 2023-09-10T08:08:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31725.714861 secs 2023-09-10T08:08:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31724.549718 secs 2023-09-10T08:08:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31725.714861 secs 2023-09-10T08:08:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31724.549718 secs 2023-09-10T08:08:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31725.714861 secs 2023-09-10T08:08:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:14 smithi170 ceph-mon[102832]: pgmap v17080: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 8 op/s 2023-09-10T08:08:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31726.158149 secs 2023-09-10T08:08:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:14 smithi082 ceph-mon[98694]: pgmap v17080: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 8 op/s 2023-09-10T08:08:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31726.158149 secs 2023-09-10T08:08:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:14 smithi098 ceph-mon[103124]: pgmap v17080: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 8 op/s 2023-09-10T08:08:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31726.158149 secs 2023-09-10T08:08:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:16 smithi170 ceph-mon[102832]: pgmap v17081: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 170 KiB/s wr, 17 op/s 2023-09-10T08:08:17.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:16 smithi082 ceph-mon[98694]: pgmap v17081: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 170 KiB/s wr, 17 op/s 2023-09-10T08:08:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:16 smithi098 ceph-mon[103124]: pgmap v17081: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 170 KiB/s wr, 17 op/s 2023-09-10T08:08:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:18 smithi170 ceph-mon[102832]: pgmap v17082: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 14 op/s 2023-09-10T08:08:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:18 smithi082 ceph-mon[98694]: pgmap v17082: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 14 op/s 2023-09-10T08:08:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:18 smithi098 ceph-mon[103124]: pgmap v17082: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 14 op/s 2023-09-10T08:08:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31729.549823 secs 2023-09-10T08:08:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31730.714922 secs 2023-09-10T08:08:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31731.158212 secs 2023-09-10T08:08:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31729.549823 secs 2023-09-10T08:08:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31730.714922 secs 2023-09-10T08:08:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31731.158212 secs 2023-09-10T08:08:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31729.549823 secs 2023-09-10T08:08:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31730.714922 secs 2023-09-10T08:08:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31731.158212 secs 2023-09-10T08:08:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:20 smithi170 ceph-mon[102832]: pgmap v17083: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 13 op/s 2023-09-10T08:08:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:20 smithi082 ceph-mon[98694]: pgmap v17083: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 13 op/s 2023-09-10T08:08:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:20 smithi098 ceph-mon[103124]: pgmap v17083: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 13 op/s 2023-09-10T08:08:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:22 smithi170 ceph-mon[102832]: pgmap v17084: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 163 KiB/s wr, 13 op/s 2023-09-10T08:08:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:22 smithi082 ceph-mon[98694]: pgmap v17084: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 163 KiB/s wr, 13 op/s 2023-09-10T08:08:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:22 smithi098 ceph-mon[103124]: pgmap v17084: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 163 KiB/s wr, 13 op/s 2023-09-10T08:08:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31734.549886 secs 2023-09-10T08:08:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31735.714982 secs 2023-09-10T08:08:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31734.549886 secs 2023-09-10T08:08:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31735.714982 secs 2023-09-10T08:08:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31734.549886 secs 2023-09-10T08:08:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31735.714982 secs 2023-09-10T08:08:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:24 smithi170 ceph-mon[102832]: pgmap v17085: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 12 op/s 2023-09-10T08:08:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31736.158277 secs 2023-09-10T08:08:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:24 smithi082 ceph-mon[98694]: pgmap v17085: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 12 op/s 2023-09-10T08:08:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31736.158277 secs 2023-09-10T08:08:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:24 smithi098 ceph-mon[103124]: pgmap v17085: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 12 op/s 2023-09-10T08:08:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31736.158277 secs 2023-09-10T08:08:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:26 smithi170 ceph-mon[102832]: pgmap v17086: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 137 KiB/s wr, 16 op/s 2023-09-10T08:08:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:26 smithi082 ceph-mon[98694]: pgmap v17086: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 137 KiB/s wr, 16 op/s 2023-09-10T08:08:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:26 smithi098 ceph-mon[103124]: pgmap v17086: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 137 KiB/s wr, 16 op/s 2023-09-10T08:08:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:28 smithi170 ceph-mon[102832]: pgmap v17087: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 10 op/s 2023-09-10T08:08:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:28 smithi082 ceph-mon[98694]: pgmap v17087: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 10 op/s 2023-09-10T08:08:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:28 smithi098 ceph-mon[103124]: pgmap v17087: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 10 op/s 2023-09-10T08:08:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31739.549952 secs 2023-09-10T08:08:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31740.715043 secs 2023-09-10T08:08:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31741.158345 secs 2023-09-10T08:08:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31739.549952 secs 2023-09-10T08:08:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31740.715043 secs 2023-09-10T08:08:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31741.158345 secs 2023-09-10T08:08:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31739.549952 secs 2023-09-10T08:08:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31740.715043 secs 2023-09-10T08:08:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31741.158345 secs 2023-09-10T08:08:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:30 smithi170 ceph-mon[102832]: pgmap v17088: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 10 op/s 2023-09-10T08:08:30.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:30 smithi082 ceph-mon[98694]: pgmap v17088: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 10 op/s 2023-09-10T08:08:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:30 smithi098 ceph-mon[103124]: pgmap v17088: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 10 op/s 2023-09-10T08:08:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:32 smithi170 ceph-mon[102832]: pgmap v17089: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 130 KiB/s wr, 15 op/s 2023-09-10T08:08:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:32 smithi082 ceph-mon[98694]: pgmap v17089: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 130 KiB/s wr, 15 op/s 2023-09-10T08:08:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:32 smithi098 ceph-mon[103124]: pgmap v17089: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 130 KiB/s wr, 15 op/s 2023-09-10T08:08:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31744.550051 secs 2023-09-10T08:08:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31745.715126 secs 2023-09-10T08:08:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:08:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:34.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:34 smithi170 ceph-mon[102832]: pgmap v17090: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 10 op/s 2023-09-10T08:08:34.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31746.158411 secs 2023-09-10T08:08:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31744.550051 secs 2023-09-10T08:08:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31745.715126 secs 2023-09-10T08:08:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:08:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:34 smithi082 ceph-mon[98694]: pgmap v17090: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 10 op/s 2023-09-10T08:08:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31746.158411 secs 2023-09-10T08:08:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31744.550051 secs 2023-09-10T08:08:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31745.715126 secs 2023-09-10T08:08:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:08:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:34 smithi098 ceph-mon[103124]: pgmap v17090: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 10 op/s 2023-09-10T08:08:35.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31746.158411 secs 2023-09-10T08:08:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:08:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:36 smithi170 ceph-mon[102832]: pgmap v17091: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 266 KiB/s wr, 16 op/s 2023-09-10T08:08:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:08:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:36 smithi082 ceph-mon[98694]: pgmap v17091: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 266 KiB/s wr, 16 op/s 2023-09-10T08:08:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:08:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:36 smithi098 ceph-mon[103124]: pgmap v17091: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 266 KiB/s wr, 16 op/s 2023-09-10T08:08:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:38 smithi170 ceph-mon[102832]: pgmap v17092: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T08:08:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:38 smithi082 ceph-mon[98694]: pgmap v17092: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T08:08:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:38 smithi098 ceph-mon[103124]: pgmap v17092: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T08:08:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:39 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31749.550110 secs 2023-09-10T08:08:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31750.715184 secs 2023-09-10T08:08:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31751.158484 secs 2023-09-10T08:08:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:39 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31749.550110 secs 2023-09-10T08:08:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31750.715184 secs 2023-09-10T08:08:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31751.158484 secs 2023-09-10T08:08:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:39 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31749.550110 secs 2023-09-10T08:08:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31750.715184 secs 2023-09-10T08:08:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31751.158484 secs 2023-09-10T08:08:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:40 smithi170 ceph-mon[102832]: pgmap v17093: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T08:08:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:40 smithi082 ceph-mon[98694]: pgmap v17093: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T08:08:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:40 smithi098 ceph-mon[103124]: pgmap v17093: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T08:08:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:42 smithi170 ceph-mon[102832]: pgmap v17094: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 522 KiB/s wr, 18 op/s 2023-09-10T08:08:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:42 smithi082 ceph-mon[98694]: pgmap v17094: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 522 KiB/s wr, 18 op/s 2023-09-10T08:08:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:42 smithi098 ceph-mon[103124]: pgmap v17094: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 522 KiB/s wr, 18 op/s 2023-09-10T08:08:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:08:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31755.715240 secs 2023-09-10T08:08:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:08:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31755.715240 secs 2023-09-10T08:08:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:08:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31755.715240 secs 2023-09-10T08:08:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:44 smithi170 ceph-mon[102832]: pgmap v17095: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 504 KiB/s wr, 13 op/s 2023-09-10T08:08:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:44 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:08:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:08:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31756.158545 secs 2023-09-10T08:08:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:44 smithi082 ceph-mon[98694]: pgmap v17095: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 504 KiB/s wr, 13 op/s 2023-09-10T08:08:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:44 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:08:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:08:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31756.158545 secs 2023-09-10T08:08:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:44 smithi098 ceph-mon[103124]: pgmap v17095: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 504 KiB/s wr, 13 op/s 2023-09-10T08:08:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:44 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:08:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:08:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31756.158545 secs 2023-09-10T08:08:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:08:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:08:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:08:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:46 smithi170 ceph-mon[102832]: pgmap v17096: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 757 KiB/s wr, 20 op/s 2023-09-10T08:08:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:46 smithi082 ceph-mon[98694]: pgmap v17096: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 757 KiB/s wr, 20 op/s 2023-09-10T08:08:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:46 smithi098 ceph-mon[103124]: pgmap v17096: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 757 KiB/s wr, 20 op/s 2023-09-10T08:08:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:48 smithi170 ceph-mon[102832]: pgmap v17097: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 509 KiB/s wr, 13 op/s 2023-09-10T08:08:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:48 smithi082 ceph-mon[98694]: pgmap v17097: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 509 KiB/s wr, 13 op/s 2023-09-10T08:08:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:48 smithi098 ceph-mon[103124]: pgmap v17097: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 509 KiB/s wr, 13 op/s 2023-09-10T08:08:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31760.715316 secs 2023-09-10T08:08:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31761.158609 secs 2023-09-10T08:08:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31760.715316 secs 2023-09-10T08:08:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31761.158609 secs 2023-09-10T08:08:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31760.715316 secs 2023-09-10T08:08:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31761.158609 secs 2023-09-10T08:08:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:08:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:50 smithi170 ceph-mon[102832]: pgmap v17098: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 509 KiB/s wr, 13 op/s 2023-09-10T08:08:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:08:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:50 smithi082 ceph-mon[98694]: pgmap v17098: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 509 KiB/s wr, 13 op/s 2023-09-10T08:08:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:08:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:50 smithi098 ceph-mon[103124]: pgmap v17098: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 509 KiB/s wr, 13 op/s 2023-09-10T08:08:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:08:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:52 smithi170 ceph-mon[102832]: pgmap v17099: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 821 KiB/s wr, 20 op/s 2023-09-10T08:08:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:08:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:52 smithi082 ceph-mon[98694]: pgmap v17099: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 821 KiB/s wr, 20 op/s 2023-09-10T08:08:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:08:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:52 smithi098 ceph-mon[103124]: pgmap v17099: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 821 KiB/s wr, 20 op/s 2023-09-10T08:08:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31764.550398 secs 2023-09-10T08:08:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31765.715375 secs 2023-09-10T08:08:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:08:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:54 smithi170 ceph-mon[102832]: pgmap v17100: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 564 KiB/s wr, 13 op/s 2023-09-10T08:08:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31766.158676 secs 2023-09-10T08:08:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31764.550398 secs 2023-09-10T08:08:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31765.715375 secs 2023-09-10T08:08:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:08:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:54 smithi082 ceph-mon[98694]: pgmap v17100: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 564 KiB/s wr, 13 op/s 2023-09-10T08:08:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31766.158676 secs 2023-09-10T08:08:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31764.550398 secs 2023-09-10T08:08:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31765.715375 secs 2023-09-10T08:08:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:08:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:54 smithi098 ceph-mon[103124]: pgmap v17100: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 564 KiB/s wr, 13 op/s 2023-09-10T08:08:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31766.158676 secs 2023-09-10T08:08:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:08:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:08:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:08:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:08:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:08:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:08:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:08:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:56 smithi170 ceph-mon[102832]: pgmap v17101: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 776 KiB/s wr, 20 op/s 2023-09-10T08:08:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:56 smithi082 ceph-mon[98694]: pgmap v17101: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 776 KiB/s wr, 20 op/s 2023-09-10T08:08:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:56 smithi098 ceph-mon[103124]: pgmap v17101: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 776 KiB/s wr, 20 op/s 2023-09-10T08:08:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:08:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:58 smithi170 ceph-mon[102832]: pgmap v17102: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 524 KiB/s wr, 13 op/s 2023-09-10T08:08:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:08:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:08:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:08:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:08:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:08:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:58 smithi082 ceph-mon[98694]: pgmap v17102: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 524 KiB/s wr, 13 op/s 2023-09-10T08:08:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:08:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:08:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:08:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:08:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:08:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:58 smithi098 ceph-mon[103124]: pgmap v17102: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 524 KiB/s wr, 13 op/s 2023-09-10T08:08:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:08:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:08:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:08:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:08:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31769.550498 secs 2023-09-10T08:08:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31770.715463 secs 2023-09-10T08:08:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:08:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:08:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:08:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:08:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31771.158739 secs 2023-09-10T08:08:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31769.550498 secs 2023-09-10T08:08:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31770.715463 secs 2023-09-10T08:08:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:08:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:08:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:08:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:08:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31771.158739 secs 2023-09-10T08:09:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31769.550498 secs 2023-09-10T08:09:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31770.715463 secs 2023-09-10T08:09:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:09:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:09:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:09:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:08:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31771.158739 secs 2023-09-10T08:09:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:00 smithi170 ceph-mon[102832]: pgmap v17103: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 524 KiB/s wr, 13 op/s 2023-09-10T08:09:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:00 smithi082 ceph-mon[98694]: pgmap v17103: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 524 KiB/s wr, 13 op/s 2023-09-10T08:09:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:00 smithi098 ceph-mon[103124]: pgmap v17103: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 524 KiB/s wr, 13 op/s 2023-09-10T08:09:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:02 smithi170 ceph-mon[102832]: pgmap v17104: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 808 KiB/s wr, 20 op/s 2023-09-10T08:09:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:02 smithi082 ceph-mon[98694]: pgmap v17104: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 808 KiB/s wr, 20 op/s 2023-09-10T08:09:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:02 smithi098 ceph-mon[103124]: pgmap v17104: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 808 KiB/s wr, 20 op/s 2023-09-10T08:09:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31774.550557 secs 2023-09-10T08:09:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31775.715493 secs 2023-09-10T08:09:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31774.550557 secs 2023-09-10T08:09:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31775.715493 secs 2023-09-10T08:09:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31774.550557 secs 2023-09-10T08:09:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31775.715493 secs 2023-09-10T08:09:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:04 smithi170 ceph-mon[102832]: pgmap v17105: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 496 KiB/s wr, 13 op/s 2023-09-10T08:09:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:09:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:09:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31776.158802 secs 2023-09-10T08:09:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:04 smithi082 ceph-mon[98694]: pgmap v17105: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 496 KiB/s wr, 13 op/s 2023-09-10T08:09:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:09:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:09:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31776.158802 secs 2023-09-10T08:09:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:04 smithi098 ceph-mon[103124]: pgmap v17105: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 496 KiB/s wr, 13 op/s 2023-09-10T08:09:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:09:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:09:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31776.158802 secs 2023-09-10T08:09:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:09:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:09:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:09:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:09:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:09:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:09:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:06 smithi170 ceph-mon[102832]: pgmap v17106: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 541 KiB/s wr, 33 op/s 2023-09-10T08:09:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:06 smithi082 ceph-mon[98694]: pgmap v17106: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 541 KiB/s wr, 33 op/s 2023-09-10T08:09:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:06 smithi098 ceph-mon[103124]: pgmap v17106: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 541 KiB/s wr, 33 op/s 2023-09-10T08:09:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:08 smithi170 ceph-mon[102832]: pgmap v17107: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 329 KiB/s wr, 27 op/s 2023-09-10T08:09:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:08 smithi082 ceph-mon[98694]: pgmap v17107: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 329 KiB/s wr, 27 op/s 2023-09-10T08:09:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:08 smithi098 ceph-mon[103124]: pgmap v17107: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 329 KiB/s wr, 27 op/s 2023-09-10T08:09:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31779.550619 secs 2023-09-10T08:09:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31780.715551 secs 2023-09-10T08:09:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:09 smithi170 ceph-mon[102832]: osdmap e1137: 12 total, 12 up, 12 in 2023-09-10T08:09:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31781.158870 secs 2023-09-10T08:09:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31779.550619 secs 2023-09-10T08:09:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31780.715551 secs 2023-09-10T08:09:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:09 smithi082 ceph-mon[98694]: osdmap e1137: 12 total, 12 up, 12 in 2023-09-10T08:09:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31781.158870 secs 2023-09-10T08:09:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31779.550619 secs 2023-09-10T08:09:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31780.715551 secs 2023-09-10T08:09:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:09 smithi098 ceph-mon[103124]: osdmap e1137: 12 total, 12 up, 12 in 2023-09-10T08:09:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31781.158870 secs 2023-09-10T08:09:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:10 smithi170 ceph-mon[102832]: pgmap v17109: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 394 KiB/s wr, 32 op/s 2023-09-10T08:09:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:10 smithi082 ceph-mon[98694]: pgmap v17109: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 394 KiB/s wr, 32 op/s 2023-09-10T08:09:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:10 smithi098 ceph-mon[103124]: pgmap v17109: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 394 KiB/s wr, 32 op/s 2023-09-10T08:09:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:12 smithi170 ceph-mon[102832]: pgmap v17110: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 167 KiB/s wr, 30 op/s 2023-09-10T08:09:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:12 smithi170 ceph-mon[102832]: osdmap e1138: 12 total, 12 up, 12 in 2023-09-10T08:09:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:12 smithi082 ceph-mon[98694]: pgmap v17110: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 167 KiB/s wr, 30 op/s 2023-09-10T08:09:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:12 smithi082 ceph-mon[98694]: osdmap e1138: 12 total, 12 up, 12 in 2023-09-10T08:09:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:12 smithi098 ceph-mon[103124]: pgmap v17110: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 167 KiB/s wr, 30 op/s 2023-09-10T08:09:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:12 smithi098 ceph-mon[103124]: osdmap e1138: 12 total, 12 up, 12 in 2023-09-10T08:09:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31784.550681 secs 2023-09-10T08:09:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31785.715609 secs 2023-09-10T08:09:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31784.550681 secs 2023-09-10T08:09:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31785.715609 secs 2023-09-10T08:09:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31784.550681 secs 2023-09-10T08:09:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31785.715609 secs 2023-09-10T08:09:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:14 smithi170 ceph-mon[102832]: pgmap v17112: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 141 KiB/s wr, 8 op/s 2023-09-10T08:09:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31786.158936 secs 2023-09-10T08:09:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:14 smithi082 ceph-mon[98694]: pgmap v17112: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 141 KiB/s wr, 8 op/s 2023-09-10T08:09:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31786.158936 secs 2023-09-10T08:09:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:14 smithi098 ceph-mon[103124]: pgmap v17112: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 141 KiB/s wr, 8 op/s 2023-09-10T08:09:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31786.158936 secs 2023-09-10T08:09:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:16 smithi170 ceph-mon[102832]: pgmap v17113: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 142 KiB/s wr, 15 op/s 2023-09-10T08:09:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:16 smithi082 ceph-mon[98694]: pgmap v17113: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 142 KiB/s wr, 15 op/s 2023-09-10T08:09:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:16 smithi098 ceph-mon[103124]: pgmap v17113: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 142 KiB/s wr, 15 op/s 2023-09-10T08:09:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:18 smithi170 ceph-mon[102832]: pgmap v17114: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 13 op/s 2023-09-10T08:09:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:18 smithi082 ceph-mon[98694]: pgmap v17114: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 13 op/s 2023-09-10T08:09:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:18 smithi098 ceph-mon[103124]: pgmap v17114: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 13 op/s 2023-09-10T08:09:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31789.550744 secs 2023-09-10T08:09:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31790.715686 secs 2023-09-10T08:09:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31791.159008 secs 2023-09-10T08:09:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31789.550744 secs 2023-09-10T08:09:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31790.715686 secs 2023-09-10T08:09:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31791.159008 secs 2023-09-10T08:09:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31789.550744 secs 2023-09-10T08:09:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31790.715686 secs 2023-09-10T08:09:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31791.159008 secs 2023-09-10T08:09:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:20 smithi170 ceph-mon[102832]: pgmap v17115: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T08:09:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:20 smithi082 ceph-mon[98694]: pgmap v17115: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T08:09:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:20 smithi098 ceph-mon[103124]: pgmap v17115: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 114 KiB/s wr, 12 op/s 2023-09-10T08:09:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:22 smithi170 ceph-mon[102832]: pgmap v17116: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2023-09-10T08:09:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:22 smithi082 ceph-mon[98694]: pgmap v17116: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2023-09-10T08:09:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:22 smithi098 ceph-mon[103124]: pgmap v17116: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2023-09-10T08:09:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31794.550778 secs 2023-09-10T08:09:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31795.715745 secs 2023-09-10T08:09:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31794.550778 secs 2023-09-10T08:09:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31795.715745 secs 2023-09-10T08:09:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31794.550778 secs 2023-09-10T08:09:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31795.715745 secs 2023-09-10T08:09:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:24 smithi170 ceph-mon[102832]: pgmap v17117: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-10T08:09:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31796.159095 secs 2023-09-10T08:09:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:24 smithi082 ceph-mon[98694]: pgmap v17117: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-10T08:09:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31796.159095 secs 2023-09-10T08:09:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:24 smithi098 ceph-mon[103124]: pgmap v17117: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-10T08:09:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31796.159095 secs 2023-09-10T08:09:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:26 smithi170 ceph-mon[102832]: pgmap v17118: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 15 op/s 2023-09-10T08:09:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:26 smithi082 ceph-mon[98694]: pgmap v17118: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 15 op/s 2023-09-10T08:09:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:26 smithi098 ceph-mon[103124]: pgmap v17118: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 15 op/s 2023-09-10T08:09:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:28 smithi170 ceph-mon[102832]: pgmap v17119: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 10 op/s 2023-09-10T08:09:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:28 smithi082 ceph-mon[98694]: pgmap v17119: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 10 op/s 2023-09-10T08:09:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:28 smithi098 ceph-mon[103124]: pgmap v17119: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 10 op/s 2023-09-10T08:09:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31799.550900 secs 2023-09-10T08:09:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31800.715814 secs 2023-09-10T08:09:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31801.159161 secs 2023-09-10T08:09:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31799.550900 secs 2023-09-10T08:09:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31800.715814 secs 2023-09-10T08:09:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31801.159161 secs 2023-09-10T08:09:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31799.550900 secs 2023-09-10T08:09:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31800.715814 secs 2023-09-10T08:09:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31801.159161 secs 2023-09-10T08:09:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:30 smithi170 ceph-mon[102832]: pgmap v17120: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 10 op/s 2023-09-10T08:09:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:30 smithi082 ceph-mon[98694]: pgmap v17120: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 10 op/s 2023-09-10T08:09:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:30 smithi098 ceph-mon[103124]: pgmap v17120: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 112 KiB/s wr, 10 op/s 2023-09-10T08:09:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:32 smithi170 ceph-mon[102832]: pgmap v17121: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 196 KiB/s wr, 15 op/s 2023-09-10T08:09:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:32 smithi082 ceph-mon[98694]: pgmap v17121: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 196 KiB/s wr, 15 op/s 2023-09-10T08:09:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:32 smithi098 ceph-mon[103124]: pgmap v17121: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 196 KiB/s wr, 15 op/s 2023-09-10T08:09:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31804.550959 secs 2023-09-10T08:09:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31805.715871 secs 2023-09-10T08:09:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:09:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:09:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:34 smithi170 ceph-mon[102832]: pgmap v17122: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 10 op/s 2023-09-10T08:09:34.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31806.159227 secs 2023-09-10T08:09:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31804.550959 secs 2023-09-10T08:09:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31805.715871 secs 2023-09-10T08:09:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:09:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:09:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:34 smithi082 ceph-mon[98694]: pgmap v17122: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 10 op/s 2023-09-10T08:09:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31806.159227 secs 2023-09-10T08:09:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31804.550959 secs 2023-09-10T08:09:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31805.715871 secs 2023-09-10T08:09:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:09:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:09:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:34 smithi098 ceph-mon[103124]: pgmap v17122: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 10 op/s 2023-09-10T08:09:35.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31806.159227 secs 2023-09-10T08:09:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:09:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:09:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:36 smithi170 ceph-mon[102832]: pgmap v17123: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 270 KiB/s wr, 17 op/s 2023-09-10T08:09:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:09:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:09:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:36 smithi082 ceph-mon[98694]: pgmap v17123: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 270 KiB/s wr, 17 op/s 2023-09-10T08:09:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:09:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:09:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:36 smithi098 ceph-mon[103124]: pgmap v17123: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 270 KiB/s wr, 17 op/s 2023-09-10T08:09:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:38 smithi170 ceph-mon[102832]: pgmap v17124: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T08:09:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:38 smithi082 ceph-mon[98694]: pgmap v17124: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T08:09:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:38 smithi098 ceph-mon[103124]: pgmap v17124: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T08:09:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:39 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31809.551027 secs 2023-09-10T08:09:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31810.715929 secs 2023-09-10T08:09:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31811.159290 secs 2023-09-10T08:09:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:39 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31809.551027 secs 2023-09-10T08:09:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31810.715929 secs 2023-09-10T08:09:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31811.159290 secs 2023-09-10T08:09:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:39 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31809.551027 secs 2023-09-10T08:09:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31810.715929 secs 2023-09-10T08:09:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31811.159290 secs 2023-09-10T08:09:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:40 smithi170 ceph-mon[102832]: pgmap v17125: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T08:09:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:40 smithi082 ceph-mon[98694]: pgmap v17125: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T08:09:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:40 smithi098 ceph-mon[103124]: pgmap v17125: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 158 KiB/s wr, 12 op/s 2023-09-10T08:09:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:42 smithi170 ceph-mon[102832]: pgmap v17126: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 416 KiB/s wr, 18 op/s 2023-09-10T08:09:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:42 smithi082 ceph-mon[98694]: pgmap v17126: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 416 KiB/s wr, 18 op/s 2023-09-10T08:09:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:42 smithi098 ceph-mon[103124]: pgmap v17126: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 416 KiB/s wr, 18 op/s 2023-09-10T08:09:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:09:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31815.715987 secs 2023-09-10T08:09:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:09:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31815.715987 secs 2023-09-10T08:09:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:09:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31815.715987 secs 2023-09-10T08:09:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:44 smithi170 ceph-mon[102832]: pgmap v17127: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 331 KiB/s wr, 12 op/s 2023-09-10T08:09:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:44 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:09:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:09:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:09:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31816.159352 secs 2023-09-10T08:09:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:44 smithi082 ceph-mon[98694]: pgmap v17127: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 331 KiB/s wr, 12 op/s 2023-09-10T08:09:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:44 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:09:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:09:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:09:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31816.159352 secs 2023-09-10T08:09:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:44 smithi098 ceph-mon[103124]: pgmap v17127: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 331 KiB/s wr, 12 op/s 2023-09-10T08:09:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:44 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:09:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:09:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:09:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31816.159352 secs 2023-09-10T08:09:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:09:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:09:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:09:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:09:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:09:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:09:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:46 smithi170 ceph-mon[102832]: pgmap v17128: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 531 KiB/s wr, 20 op/s 2023-09-10T08:09:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:46 smithi082 ceph-mon[98694]: pgmap v17128: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 531 KiB/s wr, 20 op/s 2023-09-10T08:09:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:46 smithi098 ceph-mon[103124]: pgmap v17128: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 531 KiB/s wr, 20 op/s 2023-09-10T08:09:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:48 smithi170 ceph-mon[102832]: pgmap v17129: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 457 KiB/s wr, 13 op/s 2023-09-10T08:09:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:48 smithi082 ceph-mon[98694]: pgmap v17129: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 457 KiB/s wr, 13 op/s 2023-09-10T08:09:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:48 smithi098 ceph-mon[103124]: pgmap v17129: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 457 KiB/s wr, 13 op/s 2023-09-10T08:09:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31820.716049 secs 2023-09-10T08:09:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31821.159416 secs 2023-09-10T08:09:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31820.716049 secs 2023-09-10T08:09:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31821.159416 secs 2023-09-10T08:09:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31820.716049 secs 2023-09-10T08:09:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31821.159416 secs 2023-09-10T08:09:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:50 smithi170 ceph-mon[102832]: pgmap v17130: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 457 KiB/s wr, 13 op/s 2023-09-10T08:09:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:50 smithi082 ceph-mon[98694]: pgmap v17130: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 457 KiB/s wr, 13 op/s 2023-09-10T08:09:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:50 smithi098 ceph-mon[103124]: pgmap v17130: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 457 KiB/s wr, 13 op/s 2023-09-10T08:09:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:52 smithi170 ceph-mon[102832]: pgmap v17131: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 690 KiB/s wr, 20 op/s 2023-09-10T08:09:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:52 smithi082 ceph-mon[98694]: pgmap v17131: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 690 KiB/s wr, 20 op/s 2023-09-10T08:09:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:52 smithi098 ceph-mon[103124]: pgmap v17131: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 690 KiB/s wr, 20 op/s 2023-09-10T08:09:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31825.716109 secs 2023-09-10T08:09:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:09:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:09:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:54 smithi170 ceph-mon[102832]: pgmap v17132: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 432 KiB/s wr, 14 op/s 2023-09-10T08:09:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31826.159485 secs 2023-09-10T08:09:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31825.716109 secs 2023-09-10T08:09:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:09:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:09:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:54 smithi082 ceph-mon[98694]: pgmap v17132: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 432 KiB/s wr, 14 op/s 2023-09-10T08:09:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31826.159485 secs 2023-09-10T08:09:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31825.716109 secs 2023-09-10T08:09:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:09:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:09:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:54 smithi098 ceph-mon[103124]: pgmap v17132: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 432 KiB/s wr, 14 op/s 2023-09-10T08:09:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31826.159485 secs 2023-09-10T08:09:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:09:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:09:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:56 smithi170 ceph-mon[102832]: pgmap v17133: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 690 KiB/s wr, 21 op/s 2023-09-10T08:09:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:09:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:09:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:56 smithi082 ceph-mon[98694]: pgmap v17133: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 690 KiB/s wr, 21 op/s 2023-09-10T08:09:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:09:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:09:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:56 smithi098 ceph-mon[103124]: pgmap v17133: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 690 KiB/s wr, 21 op/s 2023-09-10T08:09:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:58 smithi170 ceph-mon[102832]: pgmap v17134: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 491 KiB/s wr, 14 op/s 2023-09-10T08:09:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:58 smithi082 ceph-mon[98694]: pgmap v17134: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 491 KiB/s wr, 14 op/s 2023-09-10T08:09:59.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:58 smithi098 ceph-mon[103124]: pgmap v17134: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 491 KiB/s wr, 14 op/s 2023-09-10T08:09:59.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31829.551315 secs 2023-09-10T08:09:59.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31830.716191 secs 2023-09-10T08:09:59.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:09:59.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31831.159567 secs 2023-09-10T08:09:59.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:09:59.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:09:59.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:09:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:09:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31829.551315 secs 2023-09-10T08:09:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31830.716191 secs 2023-09-10T08:09:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:09:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31831.159567 secs 2023-09-10T08:09:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:09:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:09:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:09:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:10:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31829.551315 secs 2023-09-10T08:10:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31830.716191 secs 2023-09-10T08:10:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:10:00.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31831.159567 secs 2023-09-10T08:10:00.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:10:00.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:10:00.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:09:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:10:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:10:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:10:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:00 smithi170 ceph-mon[102832]: pgmap v17135: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 491 KiB/s wr, 14 op/s 2023-09-10T08:10:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T08:10:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:10:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:10:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:00 smithi082 ceph-mon[98694]: pgmap v17135: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 491 KiB/s wr, 14 op/s 2023-09-10T08:10:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T08:10:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:10:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:10:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:00 smithi098 ceph-mon[103124]: pgmap v17135: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 491 KiB/s wr, 14 op/s 2023-09-10T08:10:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T08:10:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:02 smithi170 ceph-mon[102832]: pgmap v17136: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 801 KiB/s wr, 34 op/s 2023-09-10T08:10:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:02 smithi082 ceph-mon[98694]: pgmap v17136: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 801 KiB/s wr, 34 op/s 2023-09-10T08:10:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:02 smithi098 ceph-mon[103124]: pgmap v17136: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 801 KiB/s wr, 34 op/s 2023-09-10T08:10:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31834.551374 secs 2023-09-10T08:10:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31835.716256 secs 2023-09-10T08:10:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31834.551374 secs 2023-09-10T08:10:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31835.716256 secs 2023-09-10T08:10:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31834.551374 secs 2023-09-10T08:10:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31835.716256 secs 2023-09-10T08:10:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:04 smithi170 ceph-mon[102832]: pgmap v17137: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 568 KiB/s wr, 27 op/s 2023-09-10T08:10:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:10:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:04 smithi170 ceph-mon[102832]: osdmap e1139: 12 total, 12 up, 12 in 2023-09-10T08:10:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31836.159635 secs 2023-09-10T08:10:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:04 smithi082 ceph-mon[98694]: pgmap v17137: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 568 KiB/s wr, 27 op/s 2023-09-10T08:10:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:10:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:04 smithi082 ceph-mon[98694]: osdmap e1139: 12 total, 12 up, 12 in 2023-09-10T08:10:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31836.159635 secs 2023-09-10T08:10:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:04 smithi098 ceph-mon[103124]: pgmap v17137: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 568 KiB/s wr, 27 op/s 2023-09-10T08:10:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:10:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:04 smithi098 ceph-mon[103124]: osdmap e1139: 12 total, 12 up, 12 in 2023-09-10T08:10:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31836.159635 secs 2023-09-10T08:10:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:10:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:10:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:10:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:06 smithi170 ceph-mon[102832]: pgmap v17139: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 563 KiB/s wr, 32 op/s 2023-09-10T08:10:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:06 smithi082 ceph-mon[98694]: pgmap v17139: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 563 KiB/s wr, 32 op/s 2023-09-10T08:10:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:06 smithi098 ceph-mon[103124]: pgmap v17139: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 563 KiB/s wr, 32 op/s 2023-09-10T08:10:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:08 smithi170 ceph-mon[102832]: osdmap e1140: 12 total, 12 up, 12 in 2023-09-10T08:10:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:08 smithi170 ceph-mon[102832]: pgmap v17141: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 703 KiB/s wr, 40 op/s 2023-09-10T08:10:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:08 smithi082 ceph-mon[98694]: osdmap e1140: 12 total, 12 up, 12 in 2023-09-10T08:10:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:08 smithi082 ceph-mon[98694]: pgmap v17141: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 703 KiB/s wr, 40 op/s 2023-09-10T08:10:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:08 smithi098 ceph-mon[103124]: osdmap e1140: 12 total, 12 up, 12 in 2023-09-10T08:10:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:08 smithi098 ceph-mon[103124]: pgmap v17141: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 703 KiB/s wr, 40 op/s 2023-09-10T08:10:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31839.551460 secs 2023-09-10T08:10:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31840.716326 secs 2023-09-10T08:10:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31841.159702 secs 2023-09-10T08:10:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31839.551460 secs 2023-09-10T08:10:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31840.716326 secs 2023-09-10T08:10:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31841.159702 secs 2023-09-10T08:10:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31839.551460 secs 2023-09-10T08:10:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31840.716326 secs 2023-09-10T08:10:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31841.159702 secs 2023-09-10T08:10:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:10 smithi170 ceph-mon[102832]: pgmap v17142: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 238 KiB/s wr, 9 op/s 2023-09-10T08:10:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:10 smithi082 ceph-mon[98694]: pgmap v17142: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 238 KiB/s wr, 9 op/s 2023-09-10T08:10:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:10 smithi098 ceph-mon[103124]: pgmap v17142: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 238 KiB/s wr, 9 op/s 2023-09-10T08:10:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:12 smithi170 ceph-mon[102832]: pgmap v17143: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 372 KiB/s wr, 18 op/s 2023-09-10T08:10:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:12 smithi082 ceph-mon[98694]: pgmap v17143: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 372 KiB/s wr, 18 op/s 2023-09-10T08:10:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:12 smithi098 ceph-mon[103124]: pgmap v17143: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 372 KiB/s wr, 18 op/s 2023-09-10T08:10:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31844.551566 secs 2023-09-10T08:10:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31845.716410 secs 2023-09-10T08:10:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31844.551566 secs 2023-09-10T08:10:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31845.716410 secs 2023-09-10T08:10:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31844.551566 secs 2023-09-10T08:10:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31845.716410 secs 2023-09-10T08:10:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:14 smithi170 ceph-mon[102832]: pgmap v17144: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 299 KiB/s wr, 14 op/s 2023-09-10T08:10:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31846.159765 secs 2023-09-10T08:10:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:14 smithi082 ceph-mon[98694]: pgmap v17144: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 299 KiB/s wr, 14 op/s 2023-09-10T08:10:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31846.159765 secs 2023-09-10T08:10:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:14 smithi098 ceph-mon[103124]: pgmap v17144: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 299 KiB/s wr, 14 op/s 2023-09-10T08:10:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31846.159765 secs 2023-09-10T08:10:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:16 smithi170 ceph-mon[102832]: pgmap v17145: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 13 op/s 2023-09-10T08:10:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:16 smithi082 ceph-mon[98694]: pgmap v17145: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 13 op/s 2023-09-10T08:10:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:16 smithi098 ceph-mon[103124]: pgmap v17145: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 13 op/s 2023-09-10T08:10:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:18 smithi170 ceph-mon[102832]: pgmap v17146: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 134 KiB/s wr, 13 op/s 2023-09-10T08:10:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:18 smithi082 ceph-mon[98694]: pgmap v17146: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 134 KiB/s wr, 13 op/s 2023-09-10T08:10:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:18 smithi098 ceph-mon[103124]: pgmap v17146: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 134 KiB/s wr, 13 op/s 2023-09-10T08:10:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31849.551628 secs 2023-09-10T08:10:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31850.716481 secs 2023-09-10T08:10:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31851.159831 secs 2023-09-10T08:10:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31849.551628 secs 2023-09-10T08:10:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31850.716481 secs 2023-09-10T08:10:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31851.159831 secs 2023-09-10T08:10:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31849.551628 secs 2023-09-10T08:10:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31850.716481 secs 2023-09-10T08:10:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31851.159831 secs 2023-09-10T08:10:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:20 smithi170 ceph-mon[102832]: pgmap v17147: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T08:10:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:20 smithi082 ceph-mon[98694]: pgmap v17147: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T08:10:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:20 smithi098 ceph-mon[103124]: pgmap v17147: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T08:10:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:22 smithi170 ceph-mon[102832]: pgmap v17148: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 184 KiB/s wr, 17 op/s 2023-09-10T08:10:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:22 smithi082 ceph-mon[98694]: pgmap v17148: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 184 KiB/s wr, 17 op/s 2023-09-10T08:10:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:22 smithi098 ceph-mon[103124]: pgmap v17148: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 184 KiB/s wr, 17 op/s 2023-09-10T08:10:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31854.551731 secs 2023-09-10T08:10:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31855.716539 secs 2023-09-10T08:10:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31854.551731 secs 2023-09-10T08:10:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31855.716539 secs 2023-09-10T08:10:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31854.551731 secs 2023-09-10T08:10:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31855.716539 secs 2023-09-10T08:10:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:24 smithi170 ceph-mon[102832]: pgmap v17149: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T08:10:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:10:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31856.159896 secs 2023-09-10T08:10:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:24 smithi082 ceph-mon[98694]: pgmap v17149: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T08:10:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:10:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31856.159896 secs 2023-09-10T08:10:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:24 smithi098 ceph-mon[103124]: pgmap v17149: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 95 KiB/s wr, 11 op/s 2023-09-10T08:10:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:10:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31856.159896 secs 2023-09-10T08:10:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:25 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:10:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:25 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:25 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:10:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:25 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:25 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:10:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:25 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:27.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:26 smithi170 ceph-mon[102832]: pgmap v17150: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 208 KiB/s wr, 17 op/s 2023-09-10T08:10:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:26 smithi082 ceph-mon[98694]: pgmap v17150: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 208 KiB/s wr, 17 op/s 2023-09-10T08:10:27.258 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:26 smithi098 ceph-mon[103124]: pgmap v17150: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 208 KiB/s wr, 17 op/s 2023-09-10T08:10:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:28 smithi170 ceph-mon[102832]: pgmap v17151: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T08:10:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:28 smithi082 ceph-mon[98694]: pgmap v17151: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T08:10:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:28 smithi098 ceph-mon[103124]: pgmap v17151: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T08:10:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31859.551773 secs 2023-09-10T08:10:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31860.716599 secs 2023-09-10T08:10:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31861.159961 secs 2023-09-10T08:10:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31859.551773 secs 2023-09-10T08:10:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31860.716599 secs 2023-09-10T08:10:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31861.159961 secs 2023-09-10T08:10:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31859.551773 secs 2023-09-10T08:10:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31860.716599 secs 2023-09-10T08:10:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31861.159961 secs 2023-09-10T08:10:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:30 smithi170 ceph-mon[102832]: pgmap v17152: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T08:10:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:30 smithi082 ceph-mon[98694]: pgmap v17152: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T08:10:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:30 smithi098 ceph-mon[103124]: pgmap v17152: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T08:10:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:32 smithi170 ceph-mon[102832]: pgmap v17153: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 183 KiB/s wr, 16 op/s 2023-09-10T08:10:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:32 smithi082 ceph-mon[98694]: pgmap v17153: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 183 KiB/s wr, 16 op/s 2023-09-10T08:10:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:32 smithi098 ceph-mon[103124]: pgmap v17153: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 183 KiB/s wr, 16 op/s 2023-09-10T08:10:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31865.716683 secs 2023-09-10T08:10:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31865.716683 secs 2023-09-10T08:10:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31865.716683 secs 2023-09-10T08:10:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:34 smithi098 ceph-mon[103124]: pgmap v17154: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T08:10:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31866.160027 secs 2023-09-10T08:10:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:34 smithi170 ceph-mon[102832]: pgmap v17154: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T08:10:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31866.160027 secs 2023-09-10T08:10:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:34 smithi082 ceph-mon[98694]: pgmap v17154: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T08:10:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31866.160027 secs 2023-09-10T08:10:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:10:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:10:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:10:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:36 smithi170 ceph-mon[102832]: pgmap v17155: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 290 KiB/s wr, 17 op/s 2023-09-10T08:10:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:10:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:10:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:36 smithi082 ceph-mon[98694]: pgmap v17155: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 290 KiB/s wr, 17 op/s 2023-09-10T08:10:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:10:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:10:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:36 smithi098 ceph-mon[103124]: pgmap v17155: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 290 KiB/s wr, 17 op/s 2023-09-10T08:10:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:10:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:10:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:10:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:10:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:10:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:38 smithi170 ceph-mon[102832]: pgmap v17156: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T08:10:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31870.716745 secs 2023-09-10T08:10:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:38 smithi082 ceph-mon[98694]: pgmap v17156: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T08:10:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31870.716745 secs 2023-09-10T08:10:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:38 smithi098 ceph-mon[103124]: pgmap v17156: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T08:10:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31870.716745 secs 2023-09-10T08:10:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31871.160090 secs 2023-09-10T08:10:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31871.160090 secs 2023-09-10T08:10:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31871.160090 secs 2023-09-10T08:10:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:40 smithi170 ceph-mon[102832]: pgmap v17157: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T08:10:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:40 smithi082 ceph-mon[98694]: pgmap v17157: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T08:10:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:40 smithi098 ceph-mon[103124]: pgmap v17157: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 11 op/s 2023-09-10T08:10:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:42 smithi170 ceph-mon[102832]: pgmap v17158: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 436 KiB/s wr, 18 op/s 2023-09-10T08:10:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:42 smithi082 ceph-mon[98694]: pgmap v17158: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 436 KiB/s wr, 18 op/s 2023-09-10T08:10:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:42 smithi098 ceph-mon[103124]: pgmap v17158: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 436 KiB/s wr, 18 op/s 2023-09-10T08:10:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31874.551953 secs 2023-09-10T08:10:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31875.716801 secs 2023-09-10T08:10:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31874.551953 secs 2023-09-10T08:10:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31875.716801 secs 2023-09-10T08:10:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31874.551953 secs 2023-09-10T08:10:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31875.716801 secs 2023-09-10T08:10:45.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:44 smithi170 ceph-mon[102832]: pgmap v17159: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 436 KiB/s wr, 13 op/s 2023-09-10T08:10:45.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:10:45.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:10:45.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:45.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31876.160153 secs 2023-09-10T08:10:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:44 smithi082 ceph-mon[98694]: pgmap v17159: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 436 KiB/s wr, 13 op/s 2023-09-10T08:10:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:10:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:10:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:45.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31876.160153 secs 2023-09-10T08:10:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:44 smithi098 ceph-mon[103124]: pgmap v17159: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 436 KiB/s wr, 13 op/s 2023-09-10T08:10:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:10:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:10:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31876.160153 secs 2023-09-10T08:10:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:10:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:10:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:10:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:46 smithi170 ceph-mon[102832]: pgmap v17160: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 636 KiB/s wr, 20 op/s 2023-09-10T08:10:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:46 smithi082 ceph-mon[98694]: pgmap v17160: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 636 KiB/s wr, 20 op/s 2023-09-10T08:10:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:46 smithi098 ceph-mon[103124]: pgmap v17160: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 636 KiB/s wr, 20 op/s 2023-09-10T08:10:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:48 smithi170 ceph-mon[102832]: pgmap v17161: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 458 KiB/s wr, 13 op/s 2023-09-10T08:10:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:48 smithi082 ceph-mon[98694]: pgmap v17161: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 458 KiB/s wr, 13 op/s 2023-09-10T08:10:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:48 smithi098 ceph-mon[103124]: pgmap v17161: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 458 KiB/s wr, 13 op/s 2023-09-10T08:10:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:49 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31879.552012 secs 2023-09-10T08:10:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31880.716858 secs 2023-09-10T08:10:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31881.160218 secs 2023-09-10T08:10:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:49 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31879.552012 secs 2023-09-10T08:10:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31880.716858 secs 2023-09-10T08:10:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31881.160218 secs 2023-09-10T08:10:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:49 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31879.552012 secs 2023-09-10T08:10:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31880.716858 secs 2023-09-10T08:10:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31881.160218 secs 2023-09-10T08:10:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:50 smithi170 ceph-mon[102832]: pgmap v17162: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 458 KiB/s wr, 13 op/s 2023-09-10T08:10:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:50 smithi082 ceph-mon[98694]: pgmap v17162: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 458 KiB/s wr, 13 op/s 2023-09-10T08:10:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:50 smithi098 ceph-mon[103124]: pgmap v17162: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 458 KiB/s wr, 13 op/s 2023-09-10T08:10:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:52 smithi170 ceph-mon[102832]: pgmap v17163: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 826 KiB/s wr, 21 op/s 2023-09-10T08:10:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:52 smithi082 ceph-mon[98694]: pgmap v17163: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 826 KiB/s wr, 21 op/s 2023-09-10T08:10:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:52 smithi098 ceph-mon[103124]: pgmap v17163: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 826 KiB/s wr, 21 op/s 2023-09-10T08:10:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31884.552070 secs 2023-09-10T08:10:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31885.716917 secs 2023-09-10T08:10:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:10:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:54 smithi170 ceph-mon[102832]: pgmap v17164: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 568 KiB/s wr, 14 op/s 2023-09-10T08:10:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31886.160283 secs 2023-09-10T08:10:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31884.552070 secs 2023-09-10T08:10:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31885.716917 secs 2023-09-10T08:10:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:10:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:54 smithi082 ceph-mon[98694]: pgmap v17164: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 568 KiB/s wr, 14 op/s 2023-09-10T08:10:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31886.160283 secs 2023-09-10T08:10:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31884.552070 secs 2023-09-10T08:10:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31885.716917 secs 2023-09-10T08:10:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:10:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:54 smithi098 ceph-mon[103124]: pgmap v17164: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 568 KiB/s wr, 14 op/s 2023-09-10T08:10:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31886.160283 secs 2023-09-10T08:10:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:10:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:56 smithi170 ceph-mon[102832]: pgmap v17165: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 781 KiB/s wr, 20 op/s 2023-09-10T08:10:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:10:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:56 smithi082 ceph-mon[98694]: pgmap v17165: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 781 KiB/s wr, 20 op/s 2023-09-10T08:10:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:10:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:10:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:56 smithi098 ceph-mon[103124]: pgmap v17165: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 781 KiB/s wr, 20 op/s 2023-09-10T08:10:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:58 smithi170 ceph-mon[102832]: pgmap v17166: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 581 KiB/s wr, 13 op/s 2023-09-10T08:10:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:58 smithi082 ceph-mon[98694]: pgmap v17166: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 581 KiB/s wr, 13 op/s 2023-09-10T08:10:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:58 smithi098 ceph-mon[103124]: pgmap v17166: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 581 KiB/s wr, 13 op/s 2023-09-10T08:10:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31889.552158 secs 2023-09-10T08:10:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31890.716973 secs 2023-09-10T08:10:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31891.160347 secs 2023-09-10T08:10:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:10:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:10:59.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31889.552158 secs 2023-09-10T08:10:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31890.716973 secs 2023-09-10T08:10:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31891.160347 secs 2023-09-10T08:10:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:10:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:11:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31889.552158 secs 2023-09-10T08:11:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31890.716973 secs 2023-09-10T08:11:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31891.160347 secs 2023-09-10T08:11:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:10:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:11:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:00 smithi170 ceph-mon[102832]: pgmap v17167: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 581 KiB/s wr, 13 op/s 2023-09-10T08:11:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:11:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:11:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:11:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:00 smithi082 ceph-mon[98694]: pgmap v17167: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 581 KiB/s wr, 13 op/s 2023-09-10T08:11:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:11:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:11:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:11:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:00 smithi098 ceph-mon[103124]: pgmap v17167: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 581 KiB/s wr, 13 op/s 2023-09-10T08:11:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:11:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:11:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:11:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:02 smithi170 ceph-mon[102832]: pgmap v17168: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 773 KiB/s wr, 32 op/s 2023-09-10T08:11:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:02 smithi082 ceph-mon[98694]: pgmap v17168: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 773 KiB/s wr, 32 op/s 2023-09-10T08:11:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:02 smithi098 ceph-mon[103124]: pgmap v17168: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 773 KiB/s wr, 32 op/s 2023-09-10T08:11:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31894.552216 secs 2023-09-10T08:11:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31895.717035 secs 2023-09-10T08:11:04.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31894.552216 secs 2023-09-10T08:11:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31895.717035 secs 2023-09-10T08:11:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31894.552216 secs 2023-09-10T08:11:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31895.717035 secs 2023-09-10T08:11:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:04 smithi170 ceph-mon[102832]: pgmap v17169: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 405 KiB/s wr, 25 op/s 2023-09-10T08:11:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:04 smithi170 ceph-mon[102832]: osdmap e1141: 12 total, 12 up, 12 in 2023-09-10T08:11:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:11:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:11:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31896.160414 secs 2023-09-10T08:11:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:04 smithi082 ceph-mon[98694]: pgmap v17169: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 405 KiB/s wr, 25 op/s 2023-09-10T08:11:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:04 smithi082 ceph-mon[98694]: osdmap e1141: 12 total, 12 up, 12 in 2023-09-10T08:11:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:11:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:11:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31896.160414 secs 2023-09-10T08:11:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:04 smithi098 ceph-mon[103124]: pgmap v17169: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 405 KiB/s wr, 25 op/s 2023-09-10T08:11:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:04 smithi098 ceph-mon[103124]: osdmap e1141: 12 total, 12 up, 12 in 2023-09-10T08:11:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:11:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:11:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31896.160414 secs 2023-09-10T08:11:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:11:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:11:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:11:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:11:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:11:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:11:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:06 smithi170 ceph-mon[102832]: pgmap v17171: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 403 KiB/s wr, 32 op/s 2023-09-10T08:11:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:06 smithi082 ceph-mon[98694]: pgmap v17171: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 403 KiB/s wr, 32 op/s 2023-09-10T08:11:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:06 smithi098 ceph-mon[103124]: pgmap v17171: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 403 KiB/s wr, 32 op/s 2023-09-10T08:11:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:08 smithi170 ceph-mon[102832]: osdmap e1142: 12 total, 12 up, 12 in 2023-09-10T08:11:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:08 smithi170 ceph-mon[102832]: pgmap v17173: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 503 KiB/s wr, 40 op/s 2023-09-10T08:11:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:08 smithi082 ceph-mon[98694]: osdmap e1142: 12 total, 12 up, 12 in 2023-09-10T08:11:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:08 smithi082 ceph-mon[98694]: pgmap v17173: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 503 KiB/s wr, 40 op/s 2023-09-10T08:11:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:08 smithi098 ceph-mon[103124]: osdmap e1142: 12 total, 12 up, 12 in 2023-09-10T08:11:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:08 smithi098 ceph-mon[103124]: pgmap v17173: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 503 KiB/s wr, 40 op/s 2023-09-10T08:11:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31899.552278 secs 2023-09-10T08:11:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31900.717093 secs 2023-09-10T08:11:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31901.160484 secs 2023-09-10T08:11:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31899.552278 secs 2023-09-10T08:11:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31900.717093 secs 2023-09-10T08:11:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31901.160484 secs 2023-09-10T08:11:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31899.552278 secs 2023-09-10T08:11:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31900.717093 secs 2023-09-10T08:11:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31901.160484 secs 2023-09-10T08:11:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:10 smithi170 ceph-mon[102832]: pgmap v17174: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 215 KiB/s wr, 11 op/s 2023-09-10T08:11:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:10 smithi082 ceph-mon[98694]: pgmap v17174: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 215 KiB/s wr, 11 op/s 2023-09-10T08:11:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:10 smithi098 ceph-mon[103124]: pgmap v17174: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 215 KiB/s wr, 11 op/s 2023-09-10T08:11:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:12 smithi170 ceph-mon[102832]: pgmap v17175: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 333 KiB/s wr, 19 op/s 2023-09-10T08:11:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:12 smithi082 ceph-mon[98694]: pgmap v17175: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 333 KiB/s wr, 19 op/s 2023-09-10T08:11:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:12 smithi098 ceph-mon[103124]: pgmap v17175: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 333 KiB/s wr, 19 op/s 2023-09-10T08:11:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31904.552368 secs 2023-09-10T08:11:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31905.717183 secs 2023-09-10T08:11:14.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31904.552368 secs 2023-09-10T08:11:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31905.717183 secs 2023-09-10T08:11:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31904.552368 secs 2023-09-10T08:11:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31905.717183 secs 2023-09-10T08:11:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:14 smithi170 ceph-mon[102832]: pgmap v17176: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 268 KiB/s wr, 15 op/s 2023-09-10T08:11:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31906.160551 secs 2023-09-10T08:11:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:14 smithi082 ceph-mon[98694]: pgmap v17176: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 268 KiB/s wr, 15 op/s 2023-09-10T08:11:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31906.160551 secs 2023-09-10T08:11:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:14 smithi098 ceph-mon[103124]: pgmap v17176: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 268 KiB/s wr, 15 op/s 2023-09-10T08:11:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31906.160551 secs 2023-09-10T08:11:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:16 smithi170 ceph-mon[102832]: pgmap v17177: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T08:11:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:16 smithi082 ceph-mon[98694]: pgmap v17177: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T08:11:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:16 smithi098 ceph-mon[103124]: pgmap v17177: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 95 KiB/s wr, 12 op/s 2023-09-10T08:11:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:18 smithi170 ceph-mon[102832]: pgmap v17178: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T08:11:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:18 smithi082 ceph-mon[98694]: pgmap v17178: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T08:11:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:18 smithi098 ceph-mon[103124]: pgmap v17178: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 12 op/s 2023-09-10T08:11:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31909.552427 secs 2023-09-10T08:11:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31910.717286 secs 2023-09-10T08:11:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31911.160617 secs 2023-09-10T08:11:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31909.552427 secs 2023-09-10T08:11:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31910.717286 secs 2023-09-10T08:11:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31911.160617 secs 2023-09-10T08:11:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31909.552427 secs 2023-09-10T08:11:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31910.717286 secs 2023-09-10T08:11:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31911.160617 secs 2023-09-10T08:11:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:20 smithi170 ceph-mon[102832]: pgmap v17179: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 79 KiB/s wr, 10 op/s 2023-09-10T08:11:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:20 smithi082 ceph-mon[98694]: pgmap v17179: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 79 KiB/s wr, 10 op/s 2023-09-10T08:11:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:20 smithi098 ceph-mon[103124]: pgmap v17179: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 79 KiB/s wr, 10 op/s 2023-09-10T08:11:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:22 smithi170 ceph-mon[102832]: pgmap v17180: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 79 KiB/s wr, 15 op/s 2023-09-10T08:11:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:22 smithi082 ceph-mon[98694]: pgmap v17180: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 79 KiB/s wr, 15 op/s 2023-09-10T08:11:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:22 smithi098 ceph-mon[103124]: pgmap v17180: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 79 KiB/s wr, 15 op/s 2023-09-10T08:11:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31914.552492 secs 2023-09-10T08:11:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31915.717344 secs 2023-09-10T08:11:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31914.552492 secs 2023-09-10T08:11:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31915.717344 secs 2023-09-10T08:11:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31914.552492 secs 2023-09-10T08:11:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31915.717344 secs 2023-09-10T08:11:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:24 smithi170 ceph-mon[102832]: pgmap v17181: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-10T08:11:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31916.160685 secs 2023-09-10T08:11:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:24 smithi082 ceph-mon[98694]: pgmap v17181: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-10T08:11:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31916.160685 secs 2023-09-10T08:11:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:24 smithi098 ceph-mon[103124]: pgmap v17181: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-10T08:11:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31916.160685 secs 2023-09-10T08:11:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:26 smithi170 ceph-mon[102832]: pgmap v17182: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 15 op/s 2023-09-10T08:11:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:26 smithi082 ceph-mon[98694]: pgmap v17182: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 15 op/s 2023-09-10T08:11:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:26 smithi098 ceph-mon[103124]: pgmap v17182: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 113 KiB/s wr, 15 op/s 2023-09-10T08:11:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:28 smithi170 ceph-mon[102832]: pgmap v17183: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T08:11:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:28 smithi082 ceph-mon[98694]: pgmap v17183: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T08:11:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:28 smithi098 ceph-mon[103124]: pgmap v17183: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T08:11:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31919.552551 secs 2023-09-10T08:11:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31920.717404 secs 2023-09-10T08:11:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31921.160752 secs 2023-09-10T08:11:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31919.552551 secs 2023-09-10T08:11:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31920.717404 secs 2023-09-10T08:11:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31921.160752 secs 2023-09-10T08:11:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31919.552551 secs 2023-09-10T08:11:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31920.717404 secs 2023-09-10T08:11:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31921.160752 secs 2023-09-10T08:11:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:30 smithi170 ceph-mon[102832]: pgmap v17184: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T08:11:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:30 smithi082 ceph-mon[98694]: pgmap v17184: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T08:11:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:30 smithi098 ceph-mon[103124]: pgmap v17184: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T08:11:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:32 smithi170 ceph-mon[102832]: pgmap v17185: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 113 KiB/s wr, 16 op/s 2023-09-10T08:11:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:32 smithi082 ceph-mon[98694]: pgmap v17185: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 113 KiB/s wr, 16 op/s 2023-09-10T08:11:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:32 smithi098 ceph-mon[103124]: pgmap v17185: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 113 KiB/s wr, 16 op/s 2023-09-10T08:11:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31925.717465 secs 2023-09-10T08:11:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:11:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:11:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:34 smithi170 ceph-mon[102832]: pgmap v17186: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T08:11:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31926.160816 secs 2023-09-10T08:11:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31925.717465 secs 2023-09-10T08:11:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:11:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:11:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:34 smithi082 ceph-mon[98694]: pgmap v17186: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T08:11:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31926.160816 secs 2023-09-10T08:11:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31925.717465 secs 2023-09-10T08:11:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:11:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:11:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:34 smithi098 ceph-mon[103124]: pgmap v17186: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 113 KiB/s wr, 11 op/s 2023-09-10T08:11:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31926.160816 secs 2023-09-10T08:11:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:11:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:11:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:11:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:11:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:11:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:11:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:36 smithi170 ceph-mon[102832]: pgmap v17187: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 271 KiB/s wr, 17 op/s 2023-09-10T08:11:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:11:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:11:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:11:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:36 smithi082 ceph-mon[98694]: pgmap v17187: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 271 KiB/s wr, 17 op/s 2023-09-10T08:11:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:11:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:11:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:11:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:36 smithi098 ceph-mon[103124]: pgmap v17187: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 271 KiB/s wr, 17 op/s 2023-09-10T08:11:38.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:38 smithi170 ceph-mon[102832]: pgmap v17188: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T08:11:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:38 smithi082 ceph-mon[98694]: pgmap v17188: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T08:11:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:38 smithi098 ceph-mon[103124]: pgmap v17188: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T08:11:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31930.717513 secs 2023-09-10T08:11:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31931.160880 secs 2023-09-10T08:11:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31930.717513 secs 2023-09-10T08:11:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31931.160880 secs 2023-09-10T08:11:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31930.717513 secs 2023-09-10T08:11:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31931.160880 secs 2023-09-10T08:11:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:40 smithi170 ceph-mon[102832]: pgmap v17189: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T08:11:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:40 smithi082 ceph-mon[98694]: pgmap v17189: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T08:11:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:40 smithi098 ceph-mon[103124]: pgmap v17189: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 11 op/s 2023-09-10T08:11:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:42 smithi170 ceph-mon[102832]: pgmap v17190: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 418 KiB/s wr, 18 op/s 2023-09-10T08:11:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:42 smithi082 ceph-mon[98694]: pgmap v17190: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 418 KiB/s wr, 18 op/s 2023-09-10T08:11:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:42 smithi098 ceph-mon[103124]: pgmap v17190: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 418 KiB/s wr, 18 op/s 2023-09-10T08:11:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31934.552778 secs 2023-09-10T08:11:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31935.717573 secs 2023-09-10T08:11:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31934.552778 secs 2023-09-10T08:11:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31935.717573 secs 2023-09-10T08:11:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31934.552778 secs 2023-09-10T08:11:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31935.717573 secs 2023-09-10T08:11:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:44 smithi170 ceph-mon[102832]: pgmap v17191: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 418 KiB/s wr, 13 op/s 2023-09-10T08:11:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:11:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:11:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:11:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31936.160942 secs 2023-09-10T08:11:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:44 smithi082 ceph-mon[98694]: pgmap v17191: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 418 KiB/s wr, 13 op/s 2023-09-10T08:11:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:11:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:11:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:11:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31936.160942 secs 2023-09-10T08:11:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:44 smithi098 ceph-mon[103124]: pgmap v17191: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 418 KiB/s wr, 13 op/s 2023-09-10T08:11:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:11:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:11:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:11:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31936.160942 secs 2023-09-10T08:11:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:11:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:11:46.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:11:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:11:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:11:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:11:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:46 smithi170 ceph-mon[102832]: pgmap v17192: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 618 KiB/s wr, 20 op/s 2023-09-10T08:11:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:46 smithi082 ceph-mon[98694]: pgmap v17192: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 618 KiB/s wr, 20 op/s 2023-09-10T08:11:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:46 smithi098 ceph-mon[103124]: pgmap v17192: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 618 KiB/s wr, 20 op/s 2023-09-10T08:11:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:48 smithi170 ceph-mon[102832]: pgmap v17193: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 460 KiB/s wr, 14 op/s 2023-09-10T08:11:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:48 smithi082 ceph-mon[98694]: pgmap v17193: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 460 KiB/s wr, 14 op/s 2023-09-10T08:11:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:48 smithi098 ceph-mon[103124]: pgmap v17193: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 460 KiB/s wr, 14 op/s 2023-09-10T08:11:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:49 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31939.552870 secs 2023-09-10T08:11:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31940.717630 secs 2023-09-10T08:11:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31941.161007 secs 2023-09-10T08:11:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:49 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31939.552870 secs 2023-09-10T08:11:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31940.717630 secs 2023-09-10T08:11:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31941.161007 secs 2023-09-10T08:11:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:49 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31939.552870 secs 2023-09-10T08:11:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31940.717630 secs 2023-09-10T08:11:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31941.161007 secs 2023-09-10T08:11:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:50 smithi170 ceph-mon[102832]: pgmap v17194: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 460 KiB/s wr, 14 op/s 2023-09-10T08:11:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:50 smithi082 ceph-mon[98694]: pgmap v17194: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 460 KiB/s wr, 14 op/s 2023-09-10T08:11:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:50 smithi098 ceph-mon[103124]: pgmap v17194: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 460 KiB/s wr, 14 op/s 2023-09-10T08:11:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:52 smithi170 ceph-mon[102832]: pgmap v17195: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 694 KiB/s wr, 21 op/s 2023-09-10T08:11:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:52 smithi082 ceph-mon[98694]: pgmap v17195: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 694 KiB/s wr, 21 op/s 2023-09-10T08:11:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:52 smithi098 ceph-mon[103124]: pgmap v17195: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 694 KiB/s wr, 21 op/s 2023-09-10T08:11:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31944.552981 secs 2023-09-10T08:11:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31945.717690 secs 2023-09-10T08:11:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:11:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:11:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:54 smithi170 ceph-mon[102832]: pgmap v17196: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 435 KiB/s wr, 14 op/s 2023-09-10T08:11:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31946.161074 secs 2023-09-10T08:11:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31944.552981 secs 2023-09-10T08:11:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31945.717690 secs 2023-09-10T08:11:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:11:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:11:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:54 smithi082 ceph-mon[98694]: pgmap v17196: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 435 KiB/s wr, 14 op/s 2023-09-10T08:11:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31946.161074 secs 2023-09-10T08:11:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31944.552981 secs 2023-09-10T08:11:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31945.717690 secs 2023-09-10T08:11:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:11:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:11:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:54 smithi098 ceph-mon[103124]: pgmap v17196: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 435 KiB/s wr, 14 op/s 2023-09-10T08:11:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31946.161074 secs 2023-09-10T08:11:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:11:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:11:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:56 smithi170 ceph-mon[102832]: pgmap v17197: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 675 KiB/s wr, 21 op/s 2023-09-10T08:11:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:11:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:11:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:56 smithi082 ceph-mon[98694]: pgmap v17197: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 675 KiB/s wr, 21 op/s 2023-09-10T08:11:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:11:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:11:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:56 smithi098 ceph-mon[103124]: pgmap v17197: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 675 KiB/s wr, 21 op/s 2023-09-10T08:11:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:58 smithi170 ceph-mon[102832]: pgmap v17198: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 474 KiB/s wr, 14 op/s 2023-09-10T08:11:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:58 smithi082 ceph-mon[98694]: pgmap v17198: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 474 KiB/s wr, 14 op/s 2023-09-10T08:11:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:58 smithi098 ceph-mon[103124]: pgmap v17198: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 474 KiB/s wr, 14 op/s 2023-09-10T08:11:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31949.553051 secs 2023-09-10T08:11:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31950.717778 secs 2023-09-10T08:11:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:11:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31951.161141 secs 2023-09-10T08:11:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31949.553051 secs 2023-09-10T08:11:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31950.717778 secs 2023-09-10T08:11:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:11:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31951.161141 secs 2023-09-10T08:12:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31949.553051 secs 2023-09-10T08:12:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31950.717778 secs 2023-09-10T08:12:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:11:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31951.161141 secs 2023-09-10T08:12:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:00 smithi170 ceph-mon[102832]: pgmap v17199: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 474 KiB/s wr, 14 op/s 2023-09-10T08:12:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:12:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:00 smithi082 ceph-mon[98694]: pgmap v17199: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 474 KiB/s wr, 14 op/s 2023-09-10T08:12:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:12:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:00 smithi098 ceph-mon[103124]: pgmap v17199: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 474 KiB/s wr, 14 op/s 2023-09-10T08:12:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:12:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:12:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:12:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:12:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:12:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:12:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:12:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:12:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:12:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:12:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:02 smithi170 ceph-mon[102832]: pgmap v17200: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 752 KiB/s wr, 23 op/s 2023-09-10T08:12:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:02 smithi082 ceph-mon[98694]: pgmap v17200: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 752 KiB/s wr, 23 op/s 2023-09-10T08:12:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:02 smithi098 ceph-mon[103124]: pgmap v17200: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 752 KiB/s wr, 23 op/s 2023-09-10T08:12:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31954.553115 secs 2023-09-10T08:12:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31955.717837 secs 2023-09-10T08:12:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:12:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:04 smithi170 ceph-mon[102832]: osdmap e1143: 12 total, 12 up, 12 in 2023-09-10T08:12:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:04 smithi170 ceph-mon[102832]: pgmap v17202: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 621 KiB/s wr, 18 op/s 2023-09-10T08:12:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31956.161205 secs 2023-09-10T08:12:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31954.553115 secs 2023-09-10T08:12:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31955.717837 secs 2023-09-10T08:12:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:12:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:04 smithi082 ceph-mon[98694]: osdmap e1143: 12 total, 12 up, 12 in 2023-09-10T08:12:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:04 smithi082 ceph-mon[98694]: pgmap v17202: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 621 KiB/s wr, 18 op/s 2023-09-10T08:12:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31956.161205 secs 2023-09-10T08:12:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31954.553115 secs 2023-09-10T08:12:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31955.717837 secs 2023-09-10T08:12:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:12:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:04 smithi098 ceph-mon[103124]: osdmap e1143: 12 total, 12 up, 12 in 2023-09-10T08:12:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:04 smithi098 ceph-mon[103124]: pgmap v17202: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 621 KiB/s wr, 18 op/s 2023-09-10T08:12:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31956.161205 secs 2023-09-10T08:12:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:12:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:06 smithi170 ceph-mon[102832]: pgmap v17203: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 565 KiB/s wr, 33 op/s 2023-09-10T08:12:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:12:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:06 smithi082 ceph-mon[98694]: pgmap v17203: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 565 KiB/s wr, 33 op/s 2023-09-10T08:12:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:12:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:06 smithi098 ceph-mon[103124]: pgmap v17203: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 565 KiB/s wr, 33 op/s 2023-09-10T08:12:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:08 smithi170 ceph-mon[102832]: osdmap e1144: 12 total, 12 up, 12 in 2023-09-10T08:12:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:08 smithi170 ceph-mon[102832]: pgmap v17205: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 706 KiB/s wr, 41 op/s 2023-09-10T08:12:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:08 smithi082 ceph-mon[98694]: osdmap e1144: 12 total, 12 up, 12 in 2023-09-10T08:12:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:08 smithi082 ceph-mon[98694]: pgmap v17205: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 706 KiB/s wr, 41 op/s 2023-09-10T08:12:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:08 smithi098 ceph-mon[103124]: osdmap e1144: 12 total, 12 up, 12 in 2023-09-10T08:12:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:08 smithi098 ceph-mon[103124]: pgmap v17205: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 706 KiB/s wr, 41 op/s 2023-09-10T08:12:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31959.553176 secs 2023-09-10T08:12:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31960.717896 secs 2023-09-10T08:12:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31961.161272 secs 2023-09-10T08:12:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31959.553176 secs 2023-09-10T08:12:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31960.717896 secs 2023-09-10T08:12:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31961.161272 secs 2023-09-10T08:12:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31959.553176 secs 2023-09-10T08:12:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31960.717896 secs 2023-09-10T08:12:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31961.161272 secs 2023-09-10T08:12:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:10 smithi170 ceph-mon[102832]: pgmap v17206: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 290 KiB/s wr, 28 op/s 2023-09-10T08:12:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:10 smithi082 ceph-mon[98694]: pgmap v17206: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 290 KiB/s wr, 28 op/s 2023-09-10T08:12:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:10 smithi098 ceph-mon[103124]: pgmap v17206: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 290 KiB/s wr, 28 op/s 2023-09-10T08:12:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:12 smithi170 ceph-mon[102832]: pgmap v17207: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 450 KiB/s wr, 36 op/s 2023-09-10T08:12:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:12 smithi082 ceph-mon[98694]: pgmap v17207: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 450 KiB/s wr, 36 op/s 2023-09-10T08:12:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:12 smithi098 ceph-mon[103124]: pgmap v17207: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 450 KiB/s wr, 36 op/s 2023-09-10T08:12:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31964.553235 secs 2023-09-10T08:12:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31965.717957 secs 2023-09-10T08:12:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31964.553235 secs 2023-09-10T08:12:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31965.717957 secs 2023-09-10T08:12:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31964.553235 secs 2023-09-10T08:12:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31965.717957 secs 2023-09-10T08:12:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:14 smithi170 ceph-mon[102832]: pgmap v17208: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 368 KiB/s wr, 30 op/s 2023-09-10T08:12:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31966.161340 secs 2023-09-10T08:12:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:14 smithi082 ceph-mon[98694]: pgmap v17208: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 368 KiB/s wr, 30 op/s 2023-09-10T08:12:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31966.161340 secs 2023-09-10T08:12:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:14 smithi098 ceph-mon[103124]: pgmap v17208: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 368 KiB/s wr, 30 op/s 2023-09-10T08:12:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31966.161340 secs 2023-09-10T08:12:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:16 smithi170 ceph-mon[102832]: pgmap v17209: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 13 op/s 2023-09-10T08:12:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:16 smithi082 ceph-mon[98694]: pgmap v17209: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 13 op/s 2023-09-10T08:12:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:16 smithi098 ceph-mon[103124]: pgmap v17209: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 13 op/s 2023-09-10T08:12:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:18 smithi170 ceph-mon[102832]: pgmap v17210: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 13 op/s 2023-09-10T08:12:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:18 smithi082 ceph-mon[98694]: pgmap v17210: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 13 op/s 2023-09-10T08:12:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:18 smithi098 ceph-mon[103124]: pgmap v17210: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 13 op/s 2023-09-10T08:12:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31969.553295 secs 2023-09-10T08:12:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31970.718036 secs 2023-09-10T08:12:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31971.161404 secs 2023-09-10T08:12:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31969.553295 secs 2023-09-10T08:12:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31970.718036 secs 2023-09-10T08:12:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31971.161404 secs 2023-09-10T08:12:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31969.553295 secs 2023-09-10T08:12:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31970.718036 secs 2023-09-10T08:12:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31971.161404 secs 2023-09-10T08:12:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:20 smithi170 ceph-mon[102832]: pgmap v17211: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T08:12:20.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:20 smithi082 ceph-mon[98694]: pgmap v17211: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T08:12:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:20 smithi098 ceph-mon[103124]: pgmap v17211: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T08:12:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:22 smithi170 ceph-mon[102832]: pgmap v17212: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 281 KiB/s wr, 16 op/s 2023-09-10T08:12:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:22 smithi082 ceph-mon[98694]: pgmap v17212: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 281 KiB/s wr, 16 op/s 2023-09-10T08:12:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:22 smithi098 ceph-mon[103124]: pgmap v17212: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 281 KiB/s wr, 16 op/s 2023-09-10T08:12:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31974.553429 secs 2023-09-10T08:12:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31975.718093 secs 2023-09-10T08:12:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31974.553429 secs 2023-09-10T08:12:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31975.718093 secs 2023-09-10T08:12:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31974.553429 secs 2023-09-10T08:12:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31975.718093 secs 2023-09-10T08:12:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:24 smithi170 ceph-mon[102832]: pgmap v17213: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 167 KiB/s wr, 10 op/s 2023-09-10T08:12:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31976.161475 secs 2023-09-10T08:12:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:24 smithi082 ceph-mon[98694]: pgmap v17213: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 167 KiB/s wr, 10 op/s 2023-09-10T08:12:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31976.161475 secs 2023-09-10T08:12:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:24 smithi098 ceph-mon[103124]: pgmap v17213: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 167 KiB/s wr, 10 op/s 2023-09-10T08:12:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31976.161475 secs 2023-09-10T08:12:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:26 smithi170 ceph-mon[102832]: pgmap v17214: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 168 KiB/s wr, 15 op/s 2023-09-10T08:12:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:26 smithi082 ceph-mon[98694]: pgmap v17214: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 168 KiB/s wr, 15 op/s 2023-09-10T08:12:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:26 smithi098 ceph-mon[103124]: pgmap v17214: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 168 KiB/s wr, 15 op/s 2023-09-10T08:12:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:28 smithi170 ceph-mon[102832]: pgmap v17215: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 168 KiB/s wr, 10 op/s 2023-09-10T08:12:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:28 smithi082 ceph-mon[98694]: pgmap v17215: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 168 KiB/s wr, 10 op/s 2023-09-10T08:12:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:28 smithi098 ceph-mon[103124]: pgmap v17215: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 168 KiB/s wr, 10 op/s 2023-09-10T08:12:29.621 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 31979.553518 secs 2023-09-10T08:12:29.621 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31980.718155 secs 2023-09-10T08:12:29.622 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31981.161540 secs 2023-09-10T08:12:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 31979.553518 secs 2023-09-10T08:12:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31980.718155 secs 2023-09-10T08:12:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31981.161540 secs 2023-09-10T08:12:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 31979.553518 secs 2023-09-10T08:12:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31980.718155 secs 2023-09-10T08:12:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31981.161540 secs 2023-09-10T08:12:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:30 smithi170 ceph-mon[102832]: pgmap v17216: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 168 KiB/s wr, 10 op/s 2023-09-10T08:12:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:30 smithi082 ceph-mon[98694]: pgmap v17216: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 168 KiB/s wr, 10 op/s 2023-09-10T08:12:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:30 smithi098 ceph-mon[103124]: pgmap v17216: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 168 KiB/s wr, 10 op/s 2023-09-10T08:12:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:32 smithi170 ceph-mon[102832]: pgmap v17217: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 280 KiB/s wr, 16 op/s 2023-09-10T08:12:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:32 smithi082 ceph-mon[98694]: pgmap v17217: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 280 KiB/s wr, 16 op/s 2023-09-10T08:12:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:32 smithi098 ceph-mon[103124]: pgmap v17217: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 280 KiB/s wr, 16 op/s 2023-09-10T08:12:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31985.718218 secs 2023-09-10T08:12:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:12:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:34 smithi170 ceph-mon[102832]: pgmap v17218: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T08:12:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31986.161605 secs 2023-09-10T08:12:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31985.718218 secs 2023-09-10T08:12:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:12:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:34 smithi082 ceph-mon[98694]: pgmap v17218: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T08:12:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31986.161605 secs 2023-09-10T08:12:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31985.718218 secs 2023-09-10T08:12:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:12:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:34 smithi098 ceph-mon[103124]: pgmap v17218: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T08:12:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31986.161605 secs 2023-09-10T08:12:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:12:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:12:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:12:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:12:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:12:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:36 smithi170 ceph-mon[102832]: pgmap v17219: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 269 KiB/s wr, 17 op/s 2023-09-10T08:12:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:12:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:12:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:36 smithi082 ceph-mon[98694]: pgmap v17219: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 269 KiB/s wr, 17 op/s 2023-09-10T08:12:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:12:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:12:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:36 smithi098 ceph-mon[103124]: pgmap v17219: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 269 KiB/s wr, 17 op/s 2023-09-10T08:12:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:38 smithi170 ceph-mon[102832]: pgmap v17220: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T08:12:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:38 smithi082 ceph-mon[98694]: pgmap v17220: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T08:12:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:38 smithi098 ceph-mon[103124]: pgmap v17220: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T08:12:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31990.718276 secs 2023-09-10T08:12:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31991.161672 secs 2023-09-10T08:12:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31990.718276 secs 2023-09-10T08:12:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31991.161672 secs 2023-09-10T08:12:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31990.718276 secs 2023-09-10T08:12:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31991.161672 secs 2023-09-10T08:12:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:40 smithi170 ceph-mon[102832]: pgmap v17221: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T08:12:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:40 smithi082 ceph-mon[98694]: pgmap v17221: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T08:12:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:40 smithi098 ceph-mon[103124]: pgmap v17221: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 268 KiB/s wr, 12 op/s 2023-09-10T08:12:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:42 smithi170 ceph-mon[102832]: pgmap v17222: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 527 KiB/s wr, 18 op/s 2023-09-10T08:12:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:42 smithi082 ceph-mon[98694]: pgmap v17222: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 527 KiB/s wr, 18 op/s 2023-09-10T08:12:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:42 smithi098 ceph-mon[103124]: pgmap v17222: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 527 KiB/s wr, 18 op/s 2023-09-10T08:12:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31995.718337 secs 2023-09-10T08:12:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31995.718337 secs 2023-09-10T08:12:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31995.718337 secs 2023-09-10T08:12:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:44 smithi170 ceph-mon[102832]: pgmap v17223: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 415 KiB/s wr, 13 op/s 2023-09-10T08:12:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:12:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 31996.161736 secs 2023-09-10T08:12:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:44 smithi082 ceph-mon[98694]: pgmap v17223: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 415 KiB/s wr, 13 op/s 2023-09-10T08:12:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:12:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 31996.161736 secs 2023-09-10T08:12:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:44 smithi098 ceph-mon[103124]: pgmap v17223: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 415 KiB/s wr, 13 op/s 2023-09-10T08:12:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:12:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 31996.161736 secs 2023-09-10T08:12:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:12:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:12:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:12:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:46 smithi170 ceph-mon[102832]: pgmap v17224: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 700 KiB/s wr, 19 op/s 2023-09-10T08:12:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:46 smithi082 ceph-mon[98694]: pgmap v17224: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 700 KiB/s wr, 19 op/s 2023-09-10T08:12:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:46 smithi098 ceph-mon[103124]: pgmap v17224: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 700 KiB/s wr, 19 op/s 2023-09-10T08:12:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:48 smithi170 ceph-mon[102832]: pgmap v17225: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 545 KiB/s wr, 13 op/s 2023-09-10T08:12:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:48 smithi082 ceph-mon[98694]: pgmap v17225: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 545 KiB/s wr, 13 op/s 2023-09-10T08:12:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:48 smithi098 ceph-mon[103124]: pgmap v17225: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 545 KiB/s wr, 13 op/s 2023-09-10T08:12:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32000.718421 secs 2023-09-10T08:12:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32001.161801 secs 2023-09-10T08:12:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32000.718421 secs 2023-09-10T08:12:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32001.161801 secs 2023-09-10T08:12:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32000.718421 secs 2023-09-10T08:12:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32001.161801 secs 2023-09-10T08:12:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:12:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:50 smithi170 ceph-mon[102832]: pgmap v17226: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 545 KiB/s wr, 13 op/s 2023-09-10T08:12:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:12:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:50 smithi082 ceph-mon[98694]: pgmap v17226: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 545 KiB/s wr, 13 op/s 2023-09-10T08:12:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:12:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:50 smithi098 ceph-mon[103124]: pgmap v17226: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 545 KiB/s wr, 13 op/s 2023-09-10T08:12:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:12:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:52 smithi170 ceph-mon[102832]: pgmap v17227: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 830 KiB/s wr, 20 op/s 2023-09-10T08:12:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:12:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:52 smithi082 ceph-mon[98694]: pgmap v17227: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 830 KiB/s wr, 20 op/s 2023-09-10T08:12:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:12:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:52 smithi098 ceph-mon[103124]: pgmap v17227: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 830 KiB/s wr, 20 op/s 2023-09-10T08:12:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32005.718498 secs 2023-09-10T08:12:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32005.718498 secs 2023-09-10T08:12:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32005.718498 secs 2023-09-10T08:12:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32004.553911 secs 2023-09-10T08:12:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:12:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:54 smithi170 ceph-mon[102832]: pgmap v17228: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 571 KiB/s wr, 13 op/s 2023-09-10T08:12:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32006.161865 secs 2023-09-10T08:12:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32004.553911 secs 2023-09-10T08:12:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:12:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:54 smithi082 ceph-mon[98694]: pgmap v17228: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 571 KiB/s wr, 13 op/s 2023-09-10T08:12:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32006.161865 secs 2023-09-10T08:12:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32004.553911 secs 2023-09-10T08:12:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:12:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:54 smithi098 ceph-mon[103124]: pgmap v17228: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 571 KiB/s wr, 13 op/s 2023-09-10T08:12:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32006.161865 secs 2023-09-10T08:12:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:12:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:12:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:12:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:12:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:12:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:12:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:12:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:56 smithi098 ceph-mon[103124]: pgmap v17229: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 789 KiB/s wr, 20 op/s 2023-09-10T08:12:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:56 smithi170 ceph-mon[102832]: pgmap v17229: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 789 KiB/s wr, 20 op/s 2023-09-10T08:12:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:56 smithi082 ceph-mon[98694]: pgmap v17229: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 789 KiB/s wr, 20 op/s 2023-09-10T08:12:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:58 smithi170 ceph-mon[102832]: pgmap v17230: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 503 KiB/s wr, 14 op/s 2023-09-10T08:12:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:58 smithi082 ceph-mon[98694]: pgmap v17230: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 503 KiB/s wr, 14 op/s 2023-09-10T08:12:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:58 smithi098 ceph-mon[103124]: pgmap v17230: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 503 KiB/s wr, 14 op/s 2023-09-10T08:12:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32009.553970 secs 2023-09-10T08:12:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32010.718558 secs 2023-09-10T08:12:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:12:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32011.161932 secs 2023-09-10T08:12:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32009.553970 secs 2023-09-10T08:12:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32010.718558 secs 2023-09-10T08:12:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:12:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32011.161932 secs 2023-09-10T08:13:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32009.553970 secs 2023-09-10T08:13:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32010.718558 secs 2023-09-10T08:13:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:12:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32011.161932 secs 2023-09-10T08:13:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:00 smithi170 ceph-mon[102832]: pgmap v17231: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 503 KiB/s wr, 14 op/s 2023-09-10T08:13:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:00 smithi082 ceph-mon[98694]: pgmap v17231: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 503 KiB/s wr, 14 op/s 2023-09-10T08:13:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:00 smithi098 ceph-mon[103124]: pgmap v17231: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 503 KiB/s wr, 14 op/s 2023-09-10T08:13:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:13:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:13:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:13:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:13:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:13:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:13:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:13:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:13:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:13:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:13:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:13:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:13:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:02 smithi170 ceph-mon[102832]: pgmap v17232: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 662 KiB/s wr, 20 op/s 2023-09-10T08:13:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:02 smithi082 ceph-mon[98694]: pgmap v17232: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 662 KiB/s wr, 20 op/s 2023-09-10T08:13:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:02 smithi098 ceph-mon[103124]: pgmap v17232: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 662 KiB/s wr, 20 op/s 2023-09-10T08:13:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32015.718615 secs 2023-09-10T08:13:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:03 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:13:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:03 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:13:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32015.718615 secs 2023-09-10T08:13:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:03 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:13:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:03 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:13:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32015.718615 secs 2023-09-10T08:13:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:03 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:13:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:03 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:13:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32014.554027 secs 2023-09-10T08:13:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:04 smithi082 ceph-mon[98694]: pgmap v17233: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 377 KiB/s wr, 12 op/s 2023-09-10T08:13:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32016.161996 secs 2023-09-10T08:13:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32014.554027 secs 2023-09-10T08:13:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:04 smithi098 ceph-mon[103124]: pgmap v17233: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 377 KiB/s wr, 12 op/s 2023-09-10T08:13:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32016.161996 secs 2023-09-10T08:13:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32014.554027 secs 2023-09-10T08:13:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:04 smithi170 ceph-mon[102832]: pgmap v17233: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 377 KiB/s wr, 12 op/s 2023-09-10T08:13:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32016.161996 secs 2023-09-10T08:13:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:13:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:13:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:06 smithi082 ceph-mon[98694]: pgmap v17234: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 531 KiB/s wr, 33 op/s 2023-09-10T08:13:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:13:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:13:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:06 smithi098 ceph-mon[103124]: pgmap v17234: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 531 KiB/s wr, 33 op/s 2023-09-10T08:13:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:13:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:13:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:06 smithi170 ceph-mon[102832]: pgmap v17234: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 531 KiB/s wr, 33 op/s 2023-09-10T08:13:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:08 smithi170 ceph-mon[102832]: pgmap v17235: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 313 KiB/s wr, 26 op/s 2023-09-10T08:13:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:08 smithi082 ceph-mon[98694]: pgmap v17235: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 313 KiB/s wr, 26 op/s 2023-09-10T08:13:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:08 smithi098 ceph-mon[103124]: pgmap v17235: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 313 KiB/s wr, 26 op/s 2023-09-10T08:13:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32019.554088 secs 2023-09-10T08:13:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32020.718693 secs 2023-09-10T08:13:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:09 smithi170 ceph-mon[102832]: osdmap e1145: 12 total, 12 up, 12 in 2023-09-10T08:13:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32021.162065 secs 2023-09-10T08:13:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32019.554088 secs 2023-09-10T08:13:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32020.718693 secs 2023-09-10T08:13:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:09 smithi082 ceph-mon[98694]: osdmap e1145: 12 total, 12 up, 12 in 2023-09-10T08:13:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32021.162065 secs 2023-09-10T08:13:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32019.554088 secs 2023-09-10T08:13:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32020.718693 secs 2023-09-10T08:13:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:09 smithi098 ceph-mon[103124]: osdmap e1145: 12 total, 12 up, 12 in 2023-09-10T08:13:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32021.162065 secs 2023-09-10T08:13:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:10 smithi170 ceph-mon[102832]: pgmap v17237: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 376 KiB/s wr, 31 op/s 2023-09-10T08:13:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:10 smithi082 ceph-mon[98694]: pgmap v17237: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 376 KiB/s wr, 31 op/s 2023-09-10T08:13:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:10 smithi098 ceph-mon[103124]: pgmap v17237: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 376 KiB/s wr, 31 op/s 2023-09-10T08:13:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:12 smithi170 ceph-mon[102832]: pgmap v17238: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 279 KiB/s wr, 31 op/s 2023-09-10T08:13:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:12 smithi170 ceph-mon[102832]: osdmap e1146: 12 total, 12 up, 12 in 2023-09-10T08:13:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:12 smithi082 ceph-mon[98694]: pgmap v17238: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 279 KiB/s wr, 31 op/s 2023-09-10T08:13:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:12 smithi082 ceph-mon[98694]: osdmap e1146: 12 total, 12 up, 12 in 2023-09-10T08:13:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:12 smithi098 ceph-mon[103124]: pgmap v17238: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 279 KiB/s wr, 31 op/s 2023-09-10T08:13:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:12 smithi098 ceph-mon[103124]: osdmap e1146: 12 total, 12 up, 12 in 2023-09-10T08:13:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32024.554148 secs 2023-09-10T08:13:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32025.718752 secs 2023-09-10T08:13:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32024.554148 secs 2023-09-10T08:13:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32025.718752 secs 2023-09-10T08:13:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32024.554148 secs 2023-09-10T08:13:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32025.718752 secs 2023-09-10T08:13:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:14 smithi170 ceph-mon[102832]: pgmap v17240: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 118 KiB/s wr, 9 op/s 2023-09-10T08:13:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32026.162130 secs 2023-09-10T08:13:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:14 smithi082 ceph-mon[98694]: pgmap v17240: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 118 KiB/s wr, 9 op/s 2023-09-10T08:13:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32026.162130 secs 2023-09-10T08:13:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:14 smithi098 ceph-mon[103124]: pgmap v17240: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 118 KiB/s wr, 9 op/s 2023-09-10T08:13:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32026.162130 secs 2023-09-10T08:13:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:16 smithi170 ceph-mon[102832]: pgmap v17241: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 17 op/s 2023-09-10T08:13:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:16 smithi082 ceph-mon[98694]: pgmap v17241: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 17 op/s 2023-09-10T08:13:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:16 smithi098 ceph-mon[103124]: pgmap v17241: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 17 op/s 2023-09-10T08:13:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:18 smithi170 ceph-mon[102832]: pgmap v17242: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 15 op/s 2023-09-10T08:13:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:18 smithi082 ceph-mon[98694]: pgmap v17242: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 15 op/s 2023-09-10T08:13:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:18 smithi098 ceph-mon[103124]: pgmap v17242: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 15 op/s 2023-09-10T08:13:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32029.554217 secs 2023-09-10T08:13:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32030.718811 secs 2023-09-10T08:13:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32031.162196 secs 2023-09-10T08:13:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32029.554217 secs 2023-09-10T08:13:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32030.718811 secs 2023-09-10T08:13:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32031.162196 secs 2023-09-10T08:13:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32029.554217 secs 2023-09-10T08:13:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32030.718811 secs 2023-09-10T08:13:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32031.162196 secs 2023-09-10T08:13:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:20 smithi170 ceph-mon[102832]: pgmap v17243: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 14 op/s 2023-09-10T08:13:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:20 smithi082 ceph-mon[98694]: pgmap v17243: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 14 op/s 2023-09-10T08:13:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:20 smithi098 ceph-mon[103124]: pgmap v17243: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 14 op/s 2023-09-10T08:13:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:22 smithi170 ceph-mon[102832]: pgmap v17244: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 13 op/s 2023-09-10T08:13:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:22 smithi082 ceph-mon[98694]: pgmap v17244: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 13 op/s 2023-09-10T08:13:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:22 smithi098 ceph-mon[103124]: pgmap v17244: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 13 op/s 2023-09-10T08:13:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32034.554275 secs 2023-09-10T08:13:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32035.718873 secs 2023-09-10T08:13:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32034.554275 secs 2023-09-10T08:13:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32035.718873 secs 2023-09-10T08:13:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32034.554275 secs 2023-09-10T08:13:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32035.718873 secs 2023-09-10T08:13:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:24 smithi170 ceph-mon[102832]: pgmap v17245: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 12 op/s 2023-09-10T08:13:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32036.162260 secs 2023-09-10T08:13:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:24 smithi082 ceph-mon[98694]: pgmap v17245: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 12 op/s 2023-09-10T08:13:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32036.162260 secs 2023-09-10T08:13:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:24 smithi098 ceph-mon[103124]: pgmap v17245: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 12 op/s 2023-09-10T08:13:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32036.162260 secs 2023-09-10T08:13:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:26 smithi170 ceph-mon[102832]: pgmap v17246: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 16 op/s 2023-09-10T08:13:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:26 smithi082 ceph-mon[98694]: pgmap v17246: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 16 op/s 2023-09-10T08:13:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:26 smithi098 ceph-mon[103124]: pgmap v17246: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 16 op/s 2023-09-10T08:13:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:28 smithi170 ceph-mon[102832]: pgmap v17247: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T08:13:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:28 smithi082 ceph-mon[98694]: pgmap v17247: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T08:13:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:28 smithi098 ceph-mon[103124]: pgmap v17247: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T08:13:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32039.554356 secs 2023-09-10T08:13:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32040.718955 secs 2023-09-10T08:13:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32041.162325 secs 2023-09-10T08:13:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32039.554356 secs 2023-09-10T08:13:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32040.718955 secs 2023-09-10T08:13:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32041.162325 secs 2023-09-10T08:13:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32039.554356 secs 2023-09-10T08:13:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32040.718955 secs 2023-09-10T08:13:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32041.162325 secs 2023-09-10T08:13:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:30 smithi170 ceph-mon[102832]: pgmap v17248: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T08:13:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:30 smithi082 ceph-mon[98694]: pgmap v17248: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T08:13:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:30 smithi098 ceph-mon[103124]: pgmap v17248: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T08:13:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:32 smithi170 ceph-mon[102832]: pgmap v17249: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 114 KiB/s wr, 15 op/s 2023-09-10T08:13:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:32 smithi082 ceph-mon[98694]: pgmap v17249: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 114 KiB/s wr, 15 op/s 2023-09-10T08:13:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:32 smithi098 ceph-mon[103124]: pgmap v17249: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 114 KiB/s wr, 15 op/s 2023-09-10T08:13:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32045.719013 secs 2023-09-10T08:13:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32045.719013 secs 2023-09-10T08:13:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32045.719013 secs 2023-09-10T08:13:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:34 smithi170 ceph-mon[102832]: pgmap v17250: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T08:13:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32046.162392 secs 2023-09-10T08:13:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:34 smithi082 ceph-mon[98694]: pgmap v17250: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T08:13:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32046.162392 secs 2023-09-10T08:13:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:34 smithi098 ceph-mon[103124]: pgmap v17250: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 113 KiB/s wr, 10 op/s 2023-09-10T08:13:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32046.162392 secs 2023-09-10T08:13:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:13:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:13:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:13:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:36 smithi170 ceph-mon[102832]: pgmap v17251: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 275 KiB/s wr, 16 op/s 2023-09-10T08:13:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:13:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:13:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:13:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:36 smithi082 ceph-mon[98694]: pgmap v17251: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 275 KiB/s wr, 16 op/s 2023-09-10T08:13:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:13:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:13:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:13:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:36 smithi098 ceph-mon[103124]: pgmap v17251: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 275 KiB/s wr, 16 op/s 2023-09-10T08:13:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:13:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:13:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:13:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:38 smithi170 ceph-mon[102832]: pgmap v17252: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T08:13:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:38 smithi082 ceph-mon[98694]: pgmap v17252: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T08:13:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:38 smithi098 ceph-mon[103124]: pgmap v17252: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T08:13:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32050.719117 secs 2023-09-10T08:13:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32051.162461 secs 2023-09-10T08:13:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32050.719117 secs 2023-09-10T08:13:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32051.162461 secs 2023-09-10T08:13:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32050.719117 secs 2023-09-10T08:13:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32051.162461 secs 2023-09-10T08:13:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:40 smithi170 ceph-mon[102832]: pgmap v17253: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T08:13:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:40 smithi082 ceph-mon[98694]: pgmap v17253: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T08:13:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:40 smithi098 ceph-mon[103124]: pgmap v17253: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T08:13:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:42 smithi170 ceph-mon[102832]: pgmap v17254: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 421 KiB/s wr, 17 op/s 2023-09-10T08:13:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:42 smithi082 ceph-mon[98694]: pgmap v17254: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 421 KiB/s wr, 17 op/s 2023-09-10T08:13:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:42 smithi098 ceph-mon[103124]: pgmap v17254: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 421 KiB/s wr, 17 op/s 2023-09-10T08:13:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32055.719181 secs 2023-09-10T08:13:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32055.719181 secs 2023-09-10T08:13:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32055.719181 secs 2023-09-10T08:13:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:44 smithi170 ceph-mon[102832]: pgmap v17255: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 421 KiB/s wr, 12 op/s 2023-09-10T08:13:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:13:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:13:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32056.162486 secs 2023-09-10T08:13:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:44 smithi082 ceph-mon[98694]: pgmap v17255: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 421 KiB/s wr, 12 op/s 2023-09-10T08:13:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:13:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:13:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32056.162486 secs 2023-09-10T08:13:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:44 smithi098 ceph-mon[103124]: pgmap v17255: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 421 KiB/s wr, 12 op/s 2023-09-10T08:13:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:13:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:13:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32056.162486 secs 2023-09-10T08:13:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:13:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:13:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:13:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:13:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:13:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:13:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:46 smithi170 ceph-mon[102832]: pgmap v17256: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 623 KiB/s wr, 20 op/s 2023-09-10T08:13:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:46 smithi082 ceph-mon[98694]: pgmap v17256: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 623 KiB/s wr, 20 op/s 2023-09-10T08:13:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:46 smithi098 ceph-mon[103124]: pgmap v17256: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 623 KiB/s wr, 20 op/s 2023-09-10T08:13:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:48 smithi170 ceph-mon[102832]: pgmap v17257: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 461 KiB/s wr, 14 op/s 2023-09-10T08:13:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:48 smithi082 ceph-mon[98694]: pgmap v17257: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 461 KiB/s wr, 14 op/s 2023-09-10T08:13:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:48 smithi098 ceph-mon[103124]: pgmap v17257: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 461 KiB/s wr, 14 op/s 2023-09-10T08:13:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32060.719239 secs 2023-09-10T08:13:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32061.162553 secs 2023-09-10T08:13:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32060.719239 secs 2023-09-10T08:13:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32061.162553 secs 2023-09-10T08:13:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32060.719239 secs 2023-09-10T08:13:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32061.162553 secs 2023-09-10T08:13:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:50 smithi170 ceph-mon[102832]: pgmap v17258: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 461 KiB/s wr, 14 op/s 2023-09-10T08:13:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:50 smithi082 ceph-mon[98694]: pgmap v17258: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 461 KiB/s wr, 14 op/s 2023-09-10T08:13:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:50 smithi098 ceph-mon[103124]: pgmap v17258: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 461 KiB/s wr, 14 op/s 2023-09-10T08:13:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:52 smithi170 ceph-mon[102832]: pgmap v17259: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 733 KiB/s wr, 21 op/s 2023-09-10T08:13:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:52 smithi082 ceph-mon[98694]: pgmap v17259: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 733 KiB/s wr, 21 op/s 2023-09-10T08:13:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:52 smithi098 ceph-mon[103124]: pgmap v17259: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 733 KiB/s wr, 21 op/s 2023-09-10T08:13:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32064.554697 secs 2023-09-10T08:13:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32065.719317 secs 2023-09-10T08:13:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:13:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:13:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:54 smithi170 ceph-mon[102832]: pgmap v17260: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 474 KiB/s wr, 14 op/s 2023-09-10T08:13:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32066.162618 secs 2023-09-10T08:13:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32064.554697 secs 2023-09-10T08:13:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32065.719317 secs 2023-09-10T08:13:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:13:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:13:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:54 smithi082 ceph-mon[98694]: pgmap v17260: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 474 KiB/s wr, 14 op/s 2023-09-10T08:13:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32066.162618 secs 2023-09-10T08:13:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32064.554697 secs 2023-09-10T08:13:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32065.719317 secs 2023-09-10T08:13:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:13:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:13:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:54 smithi098 ceph-mon[103124]: pgmap v17260: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 474 KiB/s wr, 14 op/s 2023-09-10T08:13:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32066.162618 secs 2023-09-10T08:13:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:13:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:13:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:13:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:13:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:13:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:13:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:13:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:13:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:13:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:56 smithi082 ceph-mon[98694]: pgmap v17261: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 681 KiB/s wr, 22 op/s 2023-09-10T08:13:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:56 smithi098 ceph-mon[103124]: pgmap v17261: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 681 KiB/s wr, 22 op/s 2023-09-10T08:13:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:56 smithi170 ceph-mon[102832]: pgmap v17261: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 681 KiB/s wr, 22 op/s 2023-09-10T08:13:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:58 smithi170 ceph-mon[102832]: pgmap v17262: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 478 KiB/s wr, 14 op/s 2023-09-10T08:13:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:58 smithi082 ceph-mon[98694]: pgmap v17262: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 478 KiB/s wr, 14 op/s 2023-09-10T08:13:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:58 smithi098 ceph-mon[103124]: pgmap v17262: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 478 KiB/s wr, 14 op/s 2023-09-10T08:13:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32069.554758 secs 2023-09-10T08:13:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32070.719378 secs 2023-09-10T08:13:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:13:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32071.162682 secs 2023-09-10T08:13:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32069.554758 secs 2023-09-10T08:13:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32070.719378 secs 2023-09-10T08:13:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:13:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32071.162682 secs 2023-09-10T08:14:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32069.554758 secs 2023-09-10T08:14:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32070.719378 secs 2023-09-10T08:14:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:13:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32071.162682 secs 2023-09-10T08:14:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:00 smithi170 ceph-mon[102832]: pgmap v17263: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 478 KiB/s wr, 14 op/s 2023-09-10T08:14:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:00 smithi082 ceph-mon[98694]: pgmap v17263: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 478 KiB/s wr, 14 op/s 2023-09-10T08:14:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:00 smithi098 ceph-mon[103124]: pgmap v17263: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 478 KiB/s wr, 14 op/s 2023-09-10T08:14:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:14:02.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:14:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:14:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:02 smithi170 ceph-mon[102832]: pgmap v17264: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 747 KiB/s wr, 21 op/s 2023-09-10T08:14:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:14:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:14:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:14:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:02 smithi082 ceph-mon[98694]: pgmap v17264: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 747 KiB/s wr, 21 op/s 2023-09-10T08:14:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:14:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:14:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:14:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:02 smithi098 ceph-mon[103124]: pgmap v17264: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 747 KiB/s wr, 21 op/s 2023-09-10T08:14:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:14:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:14:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:14:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32074.554788 secs 2023-09-10T08:14:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32075.719437 secs 2023-09-10T08:14:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32074.554788 secs 2023-09-10T08:14:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32075.719437 secs 2023-09-10T08:14:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32074.554788 secs 2023-09-10T08:14:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32075.719437 secs 2023-09-10T08:14:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:14:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:14:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:04 smithi170 ceph-mon[102832]: pgmap v17265: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 475 KiB/s wr, 14 op/s 2023-09-10T08:14:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32076.162746 secs 2023-09-10T08:14:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:14:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:14:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:04 smithi082 ceph-mon[98694]: pgmap v17265: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 475 KiB/s wr, 14 op/s 2023-09-10T08:14:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32076.162746 secs 2023-09-10T08:14:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:14:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:14:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:04 smithi098 ceph-mon[103124]: pgmap v17265: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 475 KiB/s wr, 14 op/s 2023-09-10T08:14:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32076.162746 secs 2023-09-10T08:14:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:14:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:14:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:14:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:14:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:14:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:14:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:06 smithi170 ceph-mon[102832]: pgmap v17266: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 675 KiB/s wr, 34 op/s 2023-09-10T08:14:07.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:06 smithi082 ceph-mon[98694]: pgmap v17266: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 675 KiB/s wr, 34 op/s 2023-09-10T08:14:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:06 smithi098 ceph-mon[103124]: pgmap v17266: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 675 KiB/s wr, 34 op/s 2023-09-10T08:14:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:08 smithi170 ceph-mon[102832]: pgmap v17267: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 469 KiB/s wr, 27 op/s 2023-09-10T08:14:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:08 smithi082 ceph-mon[98694]: pgmap v17267: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 469 KiB/s wr, 27 op/s 2023-09-10T08:14:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:08 smithi098 ceph-mon[103124]: pgmap v17267: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 469 KiB/s wr, 27 op/s 2023-09-10T08:14:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32079.554896 secs 2023-09-10T08:14:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32080.719539 secs 2023-09-10T08:14:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:09 smithi170 ceph-mon[102832]: osdmap e1147: 12 total, 12 up, 12 in 2023-09-10T08:14:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32081.162812 secs 2023-09-10T08:14:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32079.554896 secs 2023-09-10T08:14:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32080.719539 secs 2023-09-10T08:14:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:09 smithi082 ceph-mon[98694]: osdmap e1147: 12 total, 12 up, 12 in 2023-09-10T08:14:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32081.162812 secs 2023-09-10T08:14:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32079.554896 secs 2023-09-10T08:14:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32080.719539 secs 2023-09-10T08:14:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:09 smithi098 ceph-mon[103124]: osdmap e1147: 12 total, 12 up, 12 in 2023-09-10T08:14:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32081.162812 secs 2023-09-10T08:14:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:10 smithi170 ceph-mon[102832]: pgmap v17269: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 562 KiB/s wr, 32 op/s 2023-09-10T08:14:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:10 smithi082 ceph-mon[98694]: pgmap v17269: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 562 KiB/s wr, 32 op/s 2023-09-10T08:14:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:10 smithi098 ceph-mon[103124]: pgmap v17269: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 562 KiB/s wr, 32 op/s 2023-09-10T08:14:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:12 smithi170 ceph-mon[102832]: pgmap v17270: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 377 KiB/s wr, 32 op/s 2023-09-10T08:14:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:12 smithi170 ceph-mon[102832]: osdmap e1148: 12 total, 12 up, 12 in 2023-09-10T08:14:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:12 smithi082 ceph-mon[98694]: pgmap v17270: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 377 KiB/s wr, 32 op/s 2023-09-10T08:14:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:12 smithi082 ceph-mon[98694]: osdmap e1148: 12 total, 12 up, 12 in 2023-09-10T08:14:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:12 smithi098 ceph-mon[103124]: pgmap v17270: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 377 KiB/s wr, 32 op/s 2023-09-10T08:14:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:12 smithi098 ceph-mon[103124]: osdmap e1148: 12 total, 12 up, 12 in 2023-09-10T08:14:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32084.554966 secs 2023-09-10T08:14:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32085.719627 secs 2023-09-10T08:14:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32084.554966 secs 2023-09-10T08:14:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32085.719627 secs 2023-09-10T08:14:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32084.554966 secs 2023-09-10T08:14:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32085.719627 secs 2023-09-10T08:14:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:14 smithi170 ceph-mon[102832]: pgmap v17272: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 171 KiB/s wr, 9 op/s 2023-09-10T08:14:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32086.162885 secs 2023-09-10T08:14:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:14 smithi082 ceph-mon[98694]: pgmap v17272: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 171 KiB/s wr, 9 op/s 2023-09-10T08:14:15.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32086.162885 secs 2023-09-10T08:14:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:14 smithi098 ceph-mon[103124]: pgmap v17272: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 171 KiB/s wr, 9 op/s 2023-09-10T08:14:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32086.162885 secs 2023-09-10T08:14:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:15 smithi098 ceph-mon[103124]: pgmap v17273: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 172 KiB/s wr, 16 op/s 2023-09-10T08:14:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:15 smithi170 ceph-mon[102832]: pgmap v17273: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 172 KiB/s wr, 16 op/s 2023-09-10T08:14:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:15 smithi082 ceph-mon[98694]: pgmap v17273: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 172 KiB/s wr, 16 op/s 2023-09-10T08:14:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:18 smithi170 ceph-mon[102832]: pgmap v17274: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 14 op/s 2023-09-10T08:14:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:18 smithi082 ceph-mon[98694]: pgmap v17274: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 14 op/s 2023-09-10T08:14:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:18 smithi098 ceph-mon[103124]: pgmap v17274: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 14 op/s 2023-09-10T08:14:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32089.555041 secs 2023-09-10T08:14:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32090.719686 secs 2023-09-10T08:14:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32091.162967 secs 2023-09-10T08:14:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32089.555041 secs 2023-09-10T08:14:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32090.719686 secs 2023-09-10T08:14:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32091.162967 secs 2023-09-10T08:14:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32089.555041 secs 2023-09-10T08:14:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32090.719686 secs 2023-09-10T08:14:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32091.162967 secs 2023-09-10T08:14:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:20 smithi170 ceph-mon[102832]: pgmap v17275: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 138 KiB/s wr, 13 op/s 2023-09-10T08:14:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:20 smithi082 ceph-mon[98694]: pgmap v17275: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 138 KiB/s wr, 13 op/s 2023-09-10T08:14:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:20 smithi098 ceph-mon[103124]: pgmap v17275: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 138 KiB/s wr, 13 op/s 2023-09-10T08:14:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:22 smithi170 ceph-mon[102832]: pgmap v17276: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 12 op/s 2023-09-10T08:14:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:22 smithi082 ceph-mon[98694]: pgmap v17276: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 12 op/s 2023-09-10T08:14:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:22 smithi098 ceph-mon[103124]: pgmap v17276: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 12 op/s 2023-09-10T08:14:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32094.555123 secs 2023-09-10T08:14:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32095.719772 secs 2023-09-10T08:14:24.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32094.555123 secs 2023-09-10T08:14:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32095.719772 secs 2023-09-10T08:14:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32094.555123 secs 2023-09-10T08:14:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32095.719772 secs 2023-09-10T08:14:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:24 smithi170 ceph-mon[102832]: pgmap v17277: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T08:14:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32096.163035 secs 2023-09-10T08:14:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:14:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:14:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:24 smithi082 ceph-mon[98694]: pgmap v17277: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T08:14:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32096.163035 secs 2023-09-10T08:14:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:14:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:14:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:24 smithi098 ceph-mon[103124]: pgmap v17277: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T08:14:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32096.163035 secs 2023-09-10T08:14:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:14:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:14:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:25 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:14:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:25 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:14:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:25 smithi170 ceph-mon[102832]: pgmap v17278: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 186 KiB/s wr, 16 op/s 2023-09-10T08:14:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:25 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:14:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:25 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:14:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:25 smithi082 ceph-mon[98694]: pgmap v17278: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 186 KiB/s wr, 16 op/s 2023-09-10T08:14:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:25 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:14:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:25 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:14:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:25 smithi098 ceph-mon[103124]: pgmap v17278: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 186 KiB/s wr, 16 op/s 2023-09-10T08:14:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:28 smithi170 ceph-mon[102832]: pgmap v17279: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T08:14:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:28 smithi082 ceph-mon[98694]: pgmap v17279: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T08:14:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:28 smithi098 ceph-mon[103124]: pgmap v17279: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T08:14:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32099.555181 secs 2023-09-10T08:14:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32100.719832 secs 2023-09-10T08:14:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32101.163098 secs 2023-09-10T08:14:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32099.555181 secs 2023-09-10T08:14:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32100.719832 secs 2023-09-10T08:14:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32101.163098 secs 2023-09-10T08:14:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32099.555181 secs 2023-09-10T08:14:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32100.719832 secs 2023-09-10T08:14:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32101.163098 secs 2023-09-10T08:14:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:30 smithi170 ceph-mon[102832]: pgmap v17280: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T08:14:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:30 smithi082 ceph-mon[98694]: pgmap v17280: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T08:14:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:30 smithi098 ceph-mon[103124]: pgmap v17280: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T08:14:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:32 smithi170 ceph-mon[102832]: pgmap v17281: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 186 KiB/s wr, 17 op/s 2023-09-10T08:14:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:32 smithi082 ceph-mon[98694]: pgmap v17281: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 186 KiB/s wr, 17 op/s 2023-09-10T08:14:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:32 smithi098 ceph-mon[103124]: pgmap v17281: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 186 KiB/s wr, 17 op/s 2023-09-10T08:14:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32105.719921 secs 2023-09-10T08:14:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32105.719921 secs 2023-09-10T08:14:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32105.719921 secs 2023-09-10T08:14:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:34 smithi170 ceph-mon[102832]: pgmap v17282: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T08:14:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32106.163165 secs 2023-09-10T08:14:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:34 smithi082 ceph-mon[98694]: pgmap v17282: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T08:14:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32106.163165 secs 2023-09-10T08:14:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:34 smithi098 ceph-mon[103124]: pgmap v17282: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T08:14:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32106.163165 secs 2023-09-10T08:14:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:14:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:35 smithi170 ceph-mon[102832]: pgmap v17283: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 308 KiB/s wr, 17 op/s 2023-09-10T08:14:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:14:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:35 smithi082 ceph-mon[98694]: pgmap v17283: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 308 KiB/s wr, 17 op/s 2023-09-10T08:14:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:14:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:35 smithi098 ceph-mon[103124]: pgmap v17283: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 308 KiB/s wr, 17 op/s 2023-09-10T08:14:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:14:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:14:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:14:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:14:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:14:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:14:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:14:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:14:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:14:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:38 smithi170 ceph-mon[102832]: pgmap v17284: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T08:14:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:38 smithi082 ceph-mon[98694]: pgmap v17284: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T08:14:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:38 smithi098 ceph-mon[103124]: pgmap v17284: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T08:14:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32110.720005 secs 2023-09-10T08:14:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32111.163229 secs 2023-09-10T08:14:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32110.720005 secs 2023-09-10T08:14:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32111.163229 secs 2023-09-10T08:14:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32110.720005 secs 2023-09-10T08:14:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32111.163229 secs 2023-09-10T08:14:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:40 smithi170 ceph-mon[102832]: pgmap v17285: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T08:14:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:40 smithi082 ceph-mon[98694]: pgmap v17285: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T08:14:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:40 smithi098 ceph-mon[103124]: pgmap v17285: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T08:14:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:42 smithi170 ceph-mon[102832]: pgmap v17286: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 442 KiB/s wr, 18 op/s 2023-09-10T08:14:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:42 smithi082 ceph-mon[98694]: pgmap v17286: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 442 KiB/s wr, 18 op/s 2023-09-10T08:14:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:42 smithi098 ceph-mon[103124]: pgmap v17286: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 442 KiB/s wr, 18 op/s 2023-09-10T08:14:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32115.720062 secs 2023-09-10T08:14:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32115.720062 secs 2023-09-10T08:14:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32115.720062 secs 2023-09-10T08:14:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:44 smithi170 ceph-mon[102832]: pgmap v17287: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 442 KiB/s wr, 13 op/s 2023-09-10T08:14:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32116.163295 secs 2023-09-10T08:14:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:14:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:14:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:44 smithi082 ceph-mon[98694]: pgmap v17287: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 442 KiB/s wr, 13 op/s 2023-09-10T08:14:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32116.163295 secs 2023-09-10T08:14:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:14:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:14:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:44 smithi098 ceph-mon[103124]: pgmap v17287: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 442 KiB/s wr, 13 op/s 2023-09-10T08:14:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32116.163295 secs 2023-09-10T08:14:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:14:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:14:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:14:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:14:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:45 smithi170 ceph-mon[102832]: pgmap v17288: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 645 KiB/s wr, 20 op/s 2023-09-10T08:14:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:14:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:14:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:45 smithi082 ceph-mon[98694]: pgmap v17288: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 645 KiB/s wr, 20 op/s 2023-09-10T08:14:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:14:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:14:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:45 smithi098 ceph-mon[103124]: pgmap v17288: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 645 KiB/s wr, 20 op/s 2023-09-10T08:14:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:48 smithi170 ceph-mon[102832]: pgmap v17289: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 451 KiB/s wr, 14 op/s 2023-09-10T08:14:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32120.720122 secs 2023-09-10T08:14:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:48 smithi082 ceph-mon[98694]: pgmap v17289: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 451 KiB/s wr, 14 op/s 2023-09-10T08:14:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32120.720122 secs 2023-09-10T08:14:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:48 smithi098 ceph-mon[103124]: pgmap v17289: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 451 KiB/s wr, 14 op/s 2023-09-10T08:14:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32120.720122 secs 2023-09-10T08:14:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32121.163358 secs 2023-09-10T08:14:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32121.163358 secs 2023-09-10T08:14:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32121.163358 secs 2023-09-10T08:14:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:50 smithi170 ceph-mon[102832]: pgmap v17290: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 451 KiB/s wr, 14 op/s 2023-09-10T08:14:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:50 smithi082 ceph-mon[98694]: pgmap v17290: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 451 KiB/s wr, 14 op/s 2023-09-10T08:14:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:50 smithi098 ceph-mon[103124]: pgmap v17290: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 451 KiB/s wr, 14 op/s 2023-09-10T08:14:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:52 smithi170 ceph-mon[102832]: pgmap v17291: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 823 KiB/s wr, 21 op/s 2023-09-10T08:14:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:52 smithi082 ceph-mon[98694]: pgmap v17291: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 823 KiB/s wr, 21 op/s 2023-09-10T08:14:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:52 smithi098 ceph-mon[103124]: pgmap v17291: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 823 KiB/s wr, 21 op/s 2023-09-10T08:14:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32124.555550 secs 2023-09-10T08:14:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32125.720184 secs 2023-09-10T08:14:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:14:54.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:14:54.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:53 smithi170 ceph-mon[102832]: pgmap v17292: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 575 KiB/s wr, 14 op/s 2023-09-10T08:14:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32124.555550 secs 2023-09-10T08:14:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32125.720184 secs 2023-09-10T08:14:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:14:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:14:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:53 smithi082 ceph-mon[98694]: pgmap v17292: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 575 KiB/s wr, 14 op/s 2023-09-10T08:14:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32124.555550 secs 2023-09-10T08:14:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32125.720184 secs 2023-09-10T08:14:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:14:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:14:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:53 smithi098 ceph-mon[103124]: pgmap v17292: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 575 KiB/s wr, 14 op/s 2023-09-10T08:14:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32126.163429 secs 2023-09-10T08:14:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32126.163429 secs 2023-09-10T08:14:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32126.163429 secs 2023-09-10T08:14:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:55 smithi170 ceph-mon[102832]: pgmap v17293: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 810 KiB/s wr, 20 op/s 2023-09-10T08:14:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:55 smithi082 ceph-mon[98694]: pgmap v17293: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 810 KiB/s wr, 20 op/s 2023-09-10T08:14:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:55 smithi098 ceph-mon[103124]: pgmap v17293: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 810 KiB/s wr, 20 op/s 2023-09-10T08:14:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:14:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:14:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:14:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:14:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:14:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:14:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:14:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:14:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:14:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:58 smithi170 ceph-mon[102832]: pgmap v17294: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 607 KiB/s wr, 13 op/s 2023-09-10T08:14:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32130.720245 secs 2023-09-10T08:14:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:58 smithi082 ceph-mon[98694]: pgmap v17294: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 607 KiB/s wr, 13 op/s 2023-09-10T08:14:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32130.720245 secs 2023-09-10T08:14:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:58 smithi098 ceph-mon[103124]: pgmap v17294: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 607 KiB/s wr, 13 op/s 2023-09-10T08:14:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32130.720245 secs 2023-09-10T08:14:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32129.555636 secs 2023-09-10T08:14:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:14:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32131.163493 secs 2023-09-10T08:14:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32129.555636 secs 2023-09-10T08:14:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:14:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32131.163493 secs 2023-09-10T08:15:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32129.555636 secs 2023-09-10T08:15:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:14:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32131.163493 secs 2023-09-10T08:15:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:00 smithi170 ceph-mon[102832]: pgmap v17295: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 607 KiB/s wr, 13 op/s 2023-09-10T08:15:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:00 smithi082 ceph-mon[98694]: pgmap v17295: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 607 KiB/s wr, 13 op/s 2023-09-10T08:15:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:00 smithi098 ceph-mon[103124]: pgmap v17295: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 607 KiB/s wr, 13 op/s 2023-09-10T08:15:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:01 smithi098 ceph-mon[103124]: pgmap v17296: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 876 KiB/s wr, 19 op/s 2023-09-10T08:15:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:01 smithi170 ceph-mon[102832]: pgmap v17296: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 876 KiB/s wr, 19 op/s 2023-09-10T08:15:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:01 smithi082 ceph-mon[98694]: pgmap v17296: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 876 KiB/s wr, 19 op/s 2023-09-10T08:15:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:15:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:15:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:15:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32134.555699 secs 2023-09-10T08:15:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:15:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:15:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:15:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32135.720304 secs 2023-09-10T08:15:04.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:03 smithi098 ceph-mon[103124]: pgmap v17297: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 504 KiB/s wr, 12 op/s 2023-09-10T08:15:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32134.555699 secs 2023-09-10T08:15:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:15:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:15:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:15:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32135.720304 secs 2023-09-10T08:15:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:03 smithi170 ceph-mon[102832]: pgmap v17297: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 504 KiB/s wr, 12 op/s 2023-09-10T08:15:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32134.555699 secs 2023-09-10T08:15:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:15:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:15:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:15:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32135.720304 secs 2023-09-10T08:15:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:03 smithi082 ceph-mon[98694]: pgmap v17297: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 504 KiB/s wr, 12 op/s 2023-09-10T08:15:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32136.163558 secs 2023-09-10T08:15:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:15:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:15:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32136.163558 secs 2023-09-10T08:15:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:15:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:15:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32136.163558 secs 2023-09-10T08:15:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:15:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:15:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:15:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:15:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:06 smithi170 ceph-mon[102832]: pgmap v17298: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 573 KiB/s wr, 33 op/s 2023-09-10T08:15:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:15:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:15:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:06 smithi082 ceph-mon[98694]: pgmap v17298: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 573 KiB/s wr, 33 op/s 2023-09-10T08:15:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:15:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:15:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:06 smithi098 ceph-mon[103124]: pgmap v17298: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 573 KiB/s wr, 33 op/s 2023-09-10T08:15:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:08 smithi170 ceph-mon[102832]: pgmap v17299: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 339 KiB/s wr, 26 op/s 2023-09-10T08:15:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:08 smithi082 ceph-mon[98694]: pgmap v17299: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 339 KiB/s wr, 26 op/s 2023-09-10T08:15:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:08 smithi098 ceph-mon[103124]: pgmap v17299: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 339 KiB/s wr, 26 op/s 2023-09-10T08:15:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32139.555813 secs 2023-09-10T08:15:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32140.720371 secs 2023-09-10T08:15:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:09 smithi170 ceph-mon[102832]: osdmap e1149: 12 total, 12 up, 12 in 2023-09-10T08:15:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32141.163629 secs 2023-09-10T08:15:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32139.555813 secs 2023-09-10T08:15:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32140.720371 secs 2023-09-10T08:15:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:09 smithi082 ceph-mon[98694]: osdmap e1149: 12 total, 12 up, 12 in 2023-09-10T08:15:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32141.163629 secs 2023-09-10T08:15:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32139.555813 secs 2023-09-10T08:15:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32140.720371 secs 2023-09-10T08:15:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:09 smithi098 ceph-mon[103124]: osdmap e1149: 12 total, 12 up, 12 in 2023-09-10T08:15:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32141.163629 secs 2023-09-10T08:15:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:10 smithi170 ceph-mon[102832]: pgmap v17301: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 406 KiB/s wr, 32 op/s 2023-09-10T08:15:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:10 smithi082 ceph-mon[98694]: pgmap v17301: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 406 KiB/s wr, 32 op/s 2023-09-10T08:15:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:10 smithi098 ceph-mon[103124]: pgmap v17301: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 406 KiB/s wr, 32 op/s 2023-09-10T08:15:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:11 smithi098 ceph-mon[103124]: pgmap v17302: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 168 KiB/s wr, 30 op/s 2023-09-10T08:15:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:11 smithi170 ceph-mon[102832]: pgmap v17302: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 168 KiB/s wr, 30 op/s 2023-09-10T08:15:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:11 smithi082 ceph-mon[98694]: pgmap v17302: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 168 KiB/s wr, 30 op/s 2023-09-10T08:15:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:13 smithi170 ceph-mon[102832]: osdmap e1150: 12 total, 12 up, 12 in 2023-09-10T08:15:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:13 smithi082 ceph-mon[98694]: osdmap e1150: 12 total, 12 up, 12 in 2023-09-10T08:15:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:13 smithi098 ceph-mon[103124]: osdmap e1150: 12 total, 12 up, 12 in 2023-09-10T08:15:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32144.555871 secs 2023-09-10T08:15:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32145.720430 secs 2023-09-10T08:15:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:14 smithi170 ceph-mon[102832]: pgmap v17304: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 107 KiB/s wr, 7 op/s 2023-09-10T08:15:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32146.163695 secs 2023-09-10T08:15:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32144.555871 secs 2023-09-10T08:15:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32145.720430 secs 2023-09-10T08:15:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:14 smithi082 ceph-mon[98694]: pgmap v17304: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 107 KiB/s wr, 7 op/s 2023-09-10T08:15:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32146.163695 secs 2023-09-10T08:15:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32144.555871 secs 2023-09-10T08:15:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32145.720430 secs 2023-09-10T08:15:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:14 smithi098 ceph-mon[103124]: pgmap v17304: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 107 KiB/s wr, 7 op/s 2023-09-10T08:15:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32146.163695 secs 2023-09-10T08:15:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:15 smithi098 ceph-mon[103124]: pgmap v17305: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 15 op/s 2023-09-10T08:15:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:15 smithi170 ceph-mon[102832]: pgmap v17305: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 15 op/s 2023-09-10T08:15:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:15 smithi082 ceph-mon[98694]: pgmap v17305: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 15 op/s 2023-09-10T08:15:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:18 smithi170 ceph-mon[102832]: pgmap v17306: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 13 op/s 2023-09-10T08:15:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:18 smithi082 ceph-mon[98694]: pgmap v17306: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 13 op/s 2023-09-10T08:15:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:18 smithi098 ceph-mon[103124]: pgmap v17306: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 13 op/s 2023-09-10T08:15:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32149.555932 secs 2023-09-10T08:15:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32150.720489 secs 2023-09-10T08:15:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32151.163762 secs 2023-09-10T08:15:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32149.555932 secs 2023-09-10T08:15:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32150.720489 secs 2023-09-10T08:15:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32151.163762 secs 2023-09-10T08:15:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32149.555932 secs 2023-09-10T08:15:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32150.720489 secs 2023-09-10T08:15:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32151.163762 secs 2023-09-10T08:15:20.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:20 smithi170 ceph-mon[102832]: pgmap v17307: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T08:15:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:20 smithi082 ceph-mon[98694]: pgmap v17307: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T08:15:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:20 smithi098 ceph-mon[103124]: pgmap v17307: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 12 op/s 2023-09-10T08:15:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:21 smithi098 ceph-mon[103124]: pgmap v17308: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T08:15:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:21 smithi170 ceph-mon[102832]: pgmap v17308: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T08:15:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:21 smithi082 ceph-mon[98694]: pgmap v17308: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T08:15:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32155.720549 secs 2023-09-10T08:15:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32155.720549 secs 2023-09-10T08:15:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32155.720549 secs 2023-09-10T08:15:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32154.555992 secs 2023-09-10T08:15:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:24 smithi098 ceph-mon[103124]: pgmap v17309: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T08:15:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32156.163829 secs 2023-09-10T08:15:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32154.555992 secs 2023-09-10T08:15:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:24 smithi170 ceph-mon[102832]: pgmap v17309: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T08:15:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32156.163829 secs 2023-09-10T08:15:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32154.555992 secs 2023-09-10T08:15:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:24 smithi082 ceph-mon[98694]: pgmap v17309: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T08:15:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32156.163829 secs 2023-09-10T08:15:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:25 smithi098 ceph-mon[103124]: pgmap v17310: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 16 op/s 2023-09-10T08:15:26.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:25 smithi170 ceph-mon[102832]: pgmap v17310: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 16 op/s 2023-09-10T08:15:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:25 smithi082 ceph-mon[98694]: pgmap v17310: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 16 op/s 2023-09-10T08:15:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:28 smithi170 ceph-mon[102832]: pgmap v17311: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 10 op/s 2023-09-10T08:15:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:28 smithi082 ceph-mon[98694]: pgmap v17311: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 10 op/s 2023-09-10T08:15:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:28 smithi098 ceph-mon[103124]: pgmap v17311: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 10 op/s 2023-09-10T08:15:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32159.556095 secs 2023-09-10T08:15:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32160.720608 secs 2023-09-10T08:15:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32161.163896 secs 2023-09-10T08:15:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32159.556095 secs 2023-09-10T08:15:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32160.720608 secs 2023-09-10T08:15:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32161.163896 secs 2023-09-10T08:15:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32159.556095 secs 2023-09-10T08:15:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32160.720608 secs 2023-09-10T08:15:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32161.163896 secs 2023-09-10T08:15:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:30 smithi082 ceph-mon[98694]: pgmap v17312: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 10 op/s 2023-09-10T08:15:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:30 smithi098 ceph-mon[103124]: pgmap v17312: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 10 op/s 2023-09-10T08:15:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:30 smithi170 ceph-mon[102832]: pgmap v17312: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 10 op/s 2023-09-10T08:15:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:31 smithi098 ceph-mon[103124]: pgmap v17313: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 114 KiB/s wr, 16 op/s 2023-09-10T08:15:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:31 smithi170 ceph-mon[102832]: pgmap v17313: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 114 KiB/s wr, 16 op/s 2023-09-10T08:15:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:31 smithi082 ceph-mon[98694]: pgmap v17313: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 114 KiB/s wr, 16 op/s 2023-09-10T08:15:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32165.720669 secs 2023-09-10T08:15:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:15:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:15:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:34 smithi170 ceph-mon[102832]: pgmap v17314: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T08:15:34.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32166.163959 secs 2023-09-10T08:15:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32165.720669 secs 2023-09-10T08:15:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:15:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:15:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:34 smithi082 ceph-mon[98694]: pgmap v17314: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T08:15:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32166.163959 secs 2023-09-10T08:15:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32165.720669 secs 2023-09-10T08:15:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:15:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:15:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:34 smithi098 ceph-mon[103124]: pgmap v17314: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T08:15:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32166.163959 secs 2023-09-10T08:15:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:15:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:15:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:15:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:15:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:15:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:15:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:36 smithi170 ceph-mon[102832]: pgmap v17315: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 275 KiB/s wr, 17 op/s 2023-09-10T08:15:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:15:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:15:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:15:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:36 smithi082 ceph-mon[98694]: pgmap v17315: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 275 KiB/s wr, 17 op/s 2023-09-10T08:15:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:15:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:15:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:15:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:36 smithi098 ceph-mon[103124]: pgmap v17315: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 275 KiB/s wr, 17 op/s 2023-09-10T08:15:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:38 smithi170 ceph-mon[102832]: pgmap v17316: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 161 KiB/s wr, 11 op/s 2023-09-10T08:15:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:38 smithi082 ceph-mon[98694]: pgmap v17316: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 161 KiB/s wr, 11 op/s 2023-09-10T08:15:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:38 smithi098 ceph-mon[103124]: pgmap v17316: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 161 KiB/s wr, 11 op/s 2023-09-10T08:15:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32170.720727 secs 2023-09-10T08:15:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32171.164028 secs 2023-09-10T08:15:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32170.720727 secs 2023-09-10T08:15:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32171.164028 secs 2023-09-10T08:15:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32170.720727 secs 2023-09-10T08:15:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32171.164028 secs 2023-09-10T08:15:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:40 smithi170 ceph-mon[102832]: pgmap v17317: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 161 KiB/s wr, 11 op/s 2023-09-10T08:15:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:40 smithi082 ceph-mon[98694]: pgmap v17317: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 161 KiB/s wr, 11 op/s 2023-09-10T08:15:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:40 smithi098 ceph-mon[103124]: pgmap v17317: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 161 KiB/s wr, 11 op/s 2023-09-10T08:15:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:41 smithi098 ceph-mon[103124]: pgmap v17318: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 423 KiB/s wr, 19 op/s 2023-09-10T08:15:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:41 smithi170 ceph-mon[102832]: pgmap v17318: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 423 KiB/s wr, 19 op/s 2023-09-10T08:15:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:41 smithi082 ceph-mon[98694]: pgmap v17318: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 423 KiB/s wr, 19 op/s 2023-09-10T08:15:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32175.720810 secs 2023-09-10T08:15:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:15:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:15:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:44 smithi170 ceph-mon[102832]: pgmap v17319: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 423 KiB/s wr, 13 op/s 2023-09-10T08:15:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32176.164092 secs 2023-09-10T08:15:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32175.720810 secs 2023-09-10T08:15:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:15:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:15:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:44 smithi082 ceph-mon[98694]: pgmap v17319: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 423 KiB/s wr, 13 op/s 2023-09-10T08:15:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32176.164092 secs 2023-09-10T08:15:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32175.720810 secs 2023-09-10T08:15:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:15:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:15:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:44 smithi098 ceph-mon[103124]: pgmap v17319: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 423 KiB/s wr, 13 op/s 2023-09-10T08:15:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32176.164092 secs 2023-09-10T08:15:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:15:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:15:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:46 smithi082 ceph-mon[98694]: pgmap v17320: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 708 KiB/s wr, 20 op/s 2023-09-10T08:15:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:15:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:15:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:46 smithi098 ceph-mon[103124]: pgmap v17320: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 708 KiB/s wr, 20 op/s 2023-09-10T08:15:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:15:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:15:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:46 smithi170 ceph-mon[102832]: pgmap v17320: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 708 KiB/s wr, 20 op/s 2023-09-10T08:15:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:48 smithi170 ceph-mon[102832]: pgmap v17321: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 547 KiB/s wr, 14 op/s 2023-09-10T08:15:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:48 smithi082 ceph-mon[98694]: pgmap v17321: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 547 KiB/s wr, 14 op/s 2023-09-10T08:15:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:48 smithi098 ceph-mon[103124]: pgmap v17321: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 547 KiB/s wr, 14 op/s 2023-09-10T08:15:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32180.720871 secs 2023-09-10T08:15:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32181.164158 secs 2023-09-10T08:15:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32180.720871 secs 2023-09-10T08:15:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32181.164158 secs 2023-09-10T08:15:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32180.720871 secs 2023-09-10T08:15:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32181.164158 secs 2023-09-10T08:15:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:50 smithi170 ceph-mon[102832]: pgmap v17322: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 547 KiB/s wr, 14 op/s 2023-09-10T08:15:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:50 smithi082 ceph-mon[98694]: pgmap v17322: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 547 KiB/s wr, 14 op/s 2023-09-10T08:15:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:50 smithi098 ceph-mon[103124]: pgmap v17322: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 547 KiB/s wr, 14 op/s 2023-09-10T08:15:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:51 smithi098 ceph-mon[103124]: pgmap v17323: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 702 KiB/s wr, 20 op/s 2023-09-10T08:15:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:51 smithi170 ceph-mon[102832]: pgmap v17323: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 702 KiB/s wr, 20 op/s 2023-09-10T08:15:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:51 smithi082 ceph-mon[98694]: pgmap v17323: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 702 KiB/s wr, 20 op/s 2023-09-10T08:15:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32185.720930 secs 2023-09-10T08:15:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:15:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:15:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:54 smithi170 ceph-mon[102832]: pgmap v17324: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 440 KiB/s wr, 13 op/s 2023-09-10T08:15:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32186.164223 secs 2023-09-10T08:15:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32185.720930 secs 2023-09-10T08:15:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:15:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:15:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:54 smithi082 ceph-mon[98694]: pgmap v17324: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 440 KiB/s wr, 13 op/s 2023-09-10T08:15:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32186.164223 secs 2023-09-10T08:15:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32185.720930 secs 2023-09-10T08:15:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:15:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:15:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:54 smithi098 ceph-mon[103124]: pgmap v17324: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 440 KiB/s wr, 13 op/s 2023-09-10T08:15:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32186.164223 secs 2023-09-10T08:15:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:15:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:15:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:56 smithi082 ceph-mon[98694]: pgmap v17325: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 684 KiB/s wr, 21 op/s 2023-09-10T08:15:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:15:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:15:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:56 smithi098 ceph-mon[103124]: pgmap v17325: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 684 KiB/s wr, 21 op/s 2023-09-10T08:15:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:15:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:15:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:56 smithi170 ceph-mon[102832]: pgmap v17325: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 684 KiB/s wr, 21 op/s 2023-09-10T08:15:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:15:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:15:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:58 smithi082 ceph-mon[98694]: pgmap v17326: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 399 KiB/s wr, 14 op/s 2023-09-10T08:15:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:15:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:15:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:58 smithi098 ceph-mon[103124]: pgmap v17326: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 399 KiB/s wr, 14 op/s 2023-09-10T08:15:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:15:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:15:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:58 smithi170 ceph-mon[102832]: pgmap v17326: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 399 KiB/s wr, 14 op/s 2023-09-10T08:15:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32189.556536 secs 2023-09-10T08:15:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32190.721020 secs 2023-09-10T08:15:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:15:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32191.164291 secs 2023-09-10T08:16:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32189.556536 secs 2023-09-10T08:16:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32190.721020 secs 2023-09-10T08:16:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:15:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32191.164291 secs 2023-09-10T08:16:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32189.556536 secs 2023-09-10T08:16:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32190.721020 secs 2023-09-10T08:16:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:15:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32191.164291 secs 2023-09-10T08:16:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:16:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:16:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:00 smithi082 ceph-mon[98694]: pgmap v17327: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 399 KiB/s wr, 14 op/s 2023-09-10T08:16:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:16:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:16:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:00 smithi098 ceph-mon[103124]: pgmap v17327: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 399 KiB/s wr, 14 op/s 2023-09-10T08:16:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:16:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:16:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:00 smithi170 ceph-mon[102832]: pgmap v17327: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 399 KiB/s wr, 14 op/s 2023-09-10T08:16:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:01 smithi098 ceph-mon[103124]: pgmap v17328: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 669 KiB/s wr, 20 op/s 2023-09-10T08:16:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:01 smithi170 ceph-mon[102832]: pgmap v17328: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 669 KiB/s wr, 20 op/s 2023-09-10T08:16:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:01 smithi082 ceph-mon[98694]: pgmap v17328: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 669 KiB/s wr, 20 op/s 2023-09-10T08:16:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:16:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:16:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:16:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32194.556598 secs 2023-09-10T08:16:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32195.721078 secs 2023-09-10T08:16:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:16:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:04 smithi082 ceph-mon[98694]: pgmap v17329: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 513 KiB/s wr, 13 op/s 2023-09-10T08:16:04.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32196.164356 secs 2023-09-10T08:16:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32194.556598 secs 2023-09-10T08:16:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32195.721078 secs 2023-09-10T08:16:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:16:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:04 smithi098 ceph-mon[103124]: pgmap v17329: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 513 KiB/s wr, 13 op/s 2023-09-10T08:16:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32196.164356 secs 2023-09-10T08:16:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32194.556598 secs 2023-09-10T08:16:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32195.721078 secs 2023-09-10T08:16:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:16:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:04 smithi170 ceph-mon[102832]: pgmap v17329: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 513 KiB/s wr, 13 op/s 2023-09-10T08:16:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32196.164356 secs 2023-09-10T08:16:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:16:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:16:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:16:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:06 smithi082 ceph-mon[98694]: pgmap v17330: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 719 KiB/s wr, 35 op/s 2023-09-10T08:16:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:16:06.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:16:06.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:16:06.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:16:06.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:16:06.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:16:06.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:16:06.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:16:06.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:16:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:16:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:16:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:16:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:06 smithi098 ceph-mon[103124]: pgmap v17330: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 719 KiB/s wr, 35 op/s 2023-09-10T08:16:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:16:07.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:16:07.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:16:07.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:16:07.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:16:07.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:16:07.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:16:07.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:16:07.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:16:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:16:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:16:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:16:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:06 smithi170 ceph-mon[102832]: pgmap v17330: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 719 KiB/s wr, 35 op/s 2023-09-10T08:16:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:16:07.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:16:07.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:16:07.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:16:07.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:16:07.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:16:07.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:16:07.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:16:07.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:16:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:07 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T08:16:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:07 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T08:16:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:16:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:16:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:16:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:16:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:07 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T08:16:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:07 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T08:16:07.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:16:07.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:16:07.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:16:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:07 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T08:16:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:07 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T08:16:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:16:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:16:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:16:08.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:16:08.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:07 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T08:16:08.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:07 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T08:16:08.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:16:08.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:16:08.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:16:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:07 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T08:16:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:07 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T08:16:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:16:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:16:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:16:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:16:08.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:07 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T08:16:08.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:07 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T08:16:08.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:16:08.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:16:08.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:16:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:08 smithi082 ceph-mon[98694]: pgmap v17331: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 476 KiB/s wr, 27 op/s 2023-09-10T08:16:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32200.721135 secs 2023-09-10T08:16:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:08 smithi098 ceph-mon[103124]: pgmap v17331: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 476 KiB/s wr, 27 op/s 2023-09-10T08:16:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32200.721135 secs 2023-09-10T08:16:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:08 smithi170 ceph-mon[102832]: pgmap v17331: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 476 KiB/s wr, 27 op/s 2023-09-10T08:16:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32200.721135 secs 2023-09-10T08:16:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32199.556680 secs 2023-09-10T08:16:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:09 smithi082 ceph-mon[98694]: osdmap e1151: 12 total, 12 up, 12 in 2023-09-10T08:16:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32201.164423 secs 2023-09-10T08:16:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32199.556680 secs 2023-09-10T08:16:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:09 smithi098 ceph-mon[103124]: osdmap e1151: 12 total, 12 up, 12 in 2023-09-10T08:16:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32201.164423 secs 2023-09-10T08:16:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32199.556680 secs 2023-09-10T08:16:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:09 smithi170 ceph-mon[102832]: osdmap e1151: 12 total, 12 up, 12 in 2023-09-10T08:16:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32201.164423 secs 2023-09-10T08:16:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:10 smithi082 ceph-mon[98694]: pgmap v17333: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 571 KiB/s wr, 32 op/s 2023-09-10T08:16:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:10 smithi098 ceph-mon[103124]: pgmap v17333: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 571 KiB/s wr, 32 op/s 2023-09-10T08:16:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:10 smithi170 ceph-mon[102832]: pgmap v17333: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 571 KiB/s wr, 32 op/s 2023-09-10T08:16:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:11 smithi098 ceph-mon[103124]: pgmap v17334: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 384 KiB/s wr, 33 op/s 2023-09-10T08:16:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:11 smithi170 ceph-mon[102832]: pgmap v17334: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 384 KiB/s wr, 33 op/s 2023-09-10T08:16:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:11 smithi082 ceph-mon[98694]: pgmap v17334: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 384 KiB/s wr, 33 op/s 2023-09-10T08:16:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:13 smithi170 ceph-mon[102832]: osdmap e1152: 12 total, 12 up, 12 in 2023-09-10T08:16:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:13 smithi082 ceph-mon[98694]: osdmap e1152: 12 total, 12 up, 12 in 2023-09-10T08:16:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:13 smithi098 ceph-mon[103124]: osdmap e1152: 12 total, 12 up, 12 in 2023-09-10T08:16:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32204.556750 secs 2023-09-10T08:16:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32205.721224 secs 2023-09-10T08:16:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:14 smithi170 ceph-mon[102832]: pgmap v17336: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 172 KiB/s wr, 9 op/s 2023-09-10T08:16:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32206.164488 secs 2023-09-10T08:16:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32204.556750 secs 2023-09-10T08:16:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32205.721224 secs 2023-09-10T08:16:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:14 smithi082 ceph-mon[98694]: pgmap v17336: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 172 KiB/s wr, 9 op/s 2023-09-10T08:16:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32206.164488 secs 2023-09-10T08:16:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32204.556750 secs 2023-09-10T08:16:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32205.721224 secs 2023-09-10T08:16:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:14 smithi098 ceph-mon[103124]: pgmap v17336: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 172 KiB/s wr, 9 op/s 2023-09-10T08:16:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32206.164488 secs 2023-09-10T08:16:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:15 smithi098 ceph-mon[103124]: pgmap v17337: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 173 KiB/s wr, 17 op/s 2023-09-10T08:16:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:15 smithi170 ceph-mon[102832]: pgmap v17337: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 173 KiB/s wr, 17 op/s 2023-09-10T08:16:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:15 smithi082 ceph-mon[98694]: pgmap v17337: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 173 KiB/s wr, 17 op/s 2023-09-10T08:16:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:18 smithi082 ceph-mon[98694]: pgmap v17338: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T08:16:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:18 smithi098 ceph-mon[103124]: pgmap v17338: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T08:16:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:18 smithi170 ceph-mon[102832]: pgmap v17338: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T08:16:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32209.556776 secs 2023-09-10T08:16:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32210.721283 secs 2023-09-10T08:16:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32211.164554 secs 2023-09-10T08:16:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32209.556776 secs 2023-09-10T08:16:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32210.721283 secs 2023-09-10T08:16:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32211.164554 secs 2023-09-10T08:16:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32209.556776 secs 2023-09-10T08:16:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32210.721283 secs 2023-09-10T08:16:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32211.164554 secs 2023-09-10T08:16:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:20 smithi082 ceph-mon[98694]: pgmap v17339: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 13 op/s 2023-09-10T08:16:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:20 smithi098 ceph-mon[103124]: pgmap v17339: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 13 op/s 2023-09-10T08:16:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:20 smithi170 ceph-mon[102832]: pgmap v17339: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 13 op/s 2023-09-10T08:16:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:21 smithi098 ceph-mon[103124]: pgmap v17340: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 12 op/s 2023-09-10T08:16:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:21 smithi170 ceph-mon[102832]: pgmap v17340: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 12 op/s 2023-09-10T08:16:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:21 smithi082 ceph-mon[98694]: pgmap v17340: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 12 op/s 2023-09-10T08:16:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32215.721342 secs 2023-09-10T08:16:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32215.721342 secs 2023-09-10T08:16:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32215.721342 secs 2023-09-10T08:16:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32214.556911 secs 2023-09-10T08:16:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:24 smithi098 ceph-mon[103124]: pgmap v17341: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T08:16:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32216.164619 secs 2023-09-10T08:16:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32214.556911 secs 2023-09-10T08:16:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:24 smithi170 ceph-mon[102832]: pgmap v17341: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T08:16:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32216.164619 secs 2023-09-10T08:16:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32214.556911 secs 2023-09-10T08:16:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:24 smithi082 ceph-mon[98694]: pgmap v17341: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 77 KiB/s wr, 11 op/s 2023-09-10T08:16:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32216.164619 secs 2023-09-10T08:16:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:25 smithi098 ceph-mon[103124]: pgmap v17342: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 187 KiB/s wr, 16 op/s 2023-09-10T08:16:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:25 smithi170 ceph-mon[102832]: pgmap v17342: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 187 KiB/s wr, 16 op/s 2023-09-10T08:16:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:25 smithi082 ceph-mon[98694]: pgmap v17342: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 187 KiB/s wr, 16 op/s 2023-09-10T08:16:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:28 smithi082 ceph-mon[98694]: pgmap v17343: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T08:16:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:28 smithi098 ceph-mon[103124]: pgmap v17343: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T08:16:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:28 smithi170 ceph-mon[102832]: pgmap v17343: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T08:16:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32219.556971 secs 2023-09-10T08:16:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32220.721427 secs 2023-09-10T08:16:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32221.164688 secs 2023-09-10T08:16:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32219.556971 secs 2023-09-10T08:16:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32220.721427 secs 2023-09-10T08:16:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32221.164688 secs 2023-09-10T08:16:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32219.556971 secs 2023-09-10T08:16:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32220.721427 secs 2023-09-10T08:16:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32221.164688 secs 2023-09-10T08:16:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:30 smithi082 ceph-mon[98694]: pgmap v17344: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T08:16:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:30 smithi098 ceph-mon[103124]: pgmap v17344: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T08:16:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:30 smithi170 ceph-mon[102832]: pgmap v17344: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T08:16:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:31 smithi098 ceph-mon[103124]: pgmap v17345: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 186 KiB/s wr, 16 op/s 2023-09-10T08:16:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:31 smithi170 ceph-mon[102832]: pgmap v17345: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 186 KiB/s wr, 16 op/s 2023-09-10T08:16:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:31 smithi082 ceph-mon[98694]: pgmap v17345: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 186 KiB/s wr, 16 op/s 2023-09-10T08:16:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32225.721461 secs 2023-09-10T08:16:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:16:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:34 smithi082 ceph-mon[98694]: pgmap v17346: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T08:16:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32226.164755 secs 2023-09-10T08:16:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32225.721461 secs 2023-09-10T08:16:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:16:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:34 smithi098 ceph-mon[103124]: pgmap v17346: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T08:16:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32226.164755 secs 2023-09-10T08:16:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32225.721461 secs 2023-09-10T08:16:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:16:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:34 smithi170 ceph-mon[102832]: pgmap v17346: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 115 KiB/s wr, 11 op/s 2023-09-10T08:16:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32226.164755 secs 2023-09-10T08:16:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:16:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:16:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:16:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:16:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:16:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:36 smithi082 ceph-mon[98694]: pgmap v17347: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 297 KiB/s wr, 17 op/s 2023-09-10T08:16:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:16:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:16:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:36 smithi098 ceph-mon[103124]: pgmap v17347: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 297 KiB/s wr, 17 op/s 2023-09-10T08:16:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:16:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:16:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:36 smithi170 ceph-mon[102832]: pgmap v17347: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 297 KiB/s wr, 17 op/s 2023-09-10T08:16:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:38 smithi082 ceph-mon[98694]: pgmap v17348: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 182 KiB/s wr, 11 op/s 2023-09-10T08:16:39.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:38 smithi098 ceph-mon[103124]: pgmap v17348: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 182 KiB/s wr, 11 op/s 2023-09-10T08:16:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:38 smithi170 ceph-mon[102832]: pgmap v17348: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 182 KiB/s wr, 11 op/s 2023-09-10T08:16:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32230.721496 secs 2023-09-10T08:16:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32231.164819 secs 2023-09-10T08:16:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32230.721496 secs 2023-09-10T08:16:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32231.164819 secs 2023-09-10T08:16:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32230.721496 secs 2023-09-10T08:16:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32231.164819 secs 2023-09-10T08:16:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:40 smithi082 ceph-mon[98694]: pgmap v17349: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 182 KiB/s wr, 11 op/s 2023-09-10T08:16:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:40 smithi098 ceph-mon[103124]: pgmap v17349: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 182 KiB/s wr, 11 op/s 2023-09-10T08:16:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:40 smithi170 ceph-mon[102832]: pgmap v17349: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 182 KiB/s wr, 11 op/s 2023-09-10T08:16:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:41 smithi098 ceph-mon[103124]: pgmap v17350: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 445 KiB/s wr, 18 op/s 2023-09-10T08:16:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:41 smithi170 ceph-mon[102832]: pgmap v17350: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 445 KiB/s wr, 18 op/s 2023-09-10T08:16:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:41 smithi082 ceph-mon[98694]: pgmap v17350: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 445 KiB/s wr, 18 op/s 2023-09-10T08:16:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32235.721554 secs 2023-09-10T08:16:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:16:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:44 smithi082 ceph-mon[98694]: pgmap v17351: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 444 KiB/s wr, 12 op/s 2023-09-10T08:16:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32236.164886 secs 2023-09-10T08:16:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32235.721554 secs 2023-09-10T08:16:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:16:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:44 smithi098 ceph-mon[103124]: pgmap v17351: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 444 KiB/s wr, 12 op/s 2023-09-10T08:16:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32236.164886 secs 2023-09-10T08:16:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32235.721554 secs 2023-09-10T08:16:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:16:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:44 smithi170 ceph-mon[102832]: pgmap v17351: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 444 KiB/s wr, 12 op/s 2023-09-10T08:16:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32236.164886 secs 2023-09-10T08:16:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:16:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:46 smithi082 ceph-mon[98694]: pgmap v17352: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 750 KiB/s wr, 19 op/s 2023-09-10T08:16:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:16:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:46 smithi098 ceph-mon[103124]: pgmap v17352: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 750 KiB/s wr, 19 op/s 2023-09-10T08:16:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:16:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:46 smithi170 ceph-mon[102832]: pgmap v17352: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 750 KiB/s wr, 19 op/s 2023-09-10T08:16:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:48 smithi082 ceph-mon[98694]: pgmap v17353: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 568 KiB/s wr, 13 op/s 2023-09-10T08:16:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:48 smithi098 ceph-mon[103124]: pgmap v17353: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 568 KiB/s wr, 13 op/s 2023-09-10T08:16:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:48 smithi170 ceph-mon[102832]: pgmap v17353: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 568 KiB/s wr, 13 op/s 2023-09-10T08:16:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32240.721625 secs 2023-09-10T08:16:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32241.164949 secs 2023-09-10T08:16:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32240.721625 secs 2023-09-10T08:16:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32241.164949 secs 2023-09-10T08:16:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32240.721625 secs 2023-09-10T08:16:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32241.164949 secs 2023-09-10T08:16:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:16:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:50 smithi082 ceph-mon[98694]: pgmap v17354: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 568 KiB/s wr, 13 op/s 2023-09-10T08:16:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:16:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:50 smithi098 ceph-mon[103124]: pgmap v17354: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 568 KiB/s wr, 13 op/s 2023-09-10T08:16:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:16:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:50 smithi170 ceph-mon[102832]: pgmap v17354: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 568 KiB/s wr, 13 op/s 2023-09-10T08:16:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:16:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:52 smithi098 ceph-mon[103124]: pgmap v17355: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 840 KiB/s wr, 20 op/s 2023-09-10T08:16:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:16:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:52 smithi170 ceph-mon[102832]: pgmap v17355: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 840 KiB/s wr, 20 op/s 2023-09-10T08:16:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:16:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:52 smithi082 ceph-mon[98694]: pgmap v17355: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 840 KiB/s wr, 20 op/s 2023-09-10T08:16:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32245.721683 secs 2023-09-10T08:16:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32245.721683 secs 2023-09-10T08:16:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32245.721683 secs 2023-09-10T08:16:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32244.557357 secs 2023-09-10T08:16:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:16:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:54 smithi098 ceph-mon[103124]: pgmap v17356: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 578 KiB/s wr, 13 op/s 2023-09-10T08:16:55.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32246.165016 secs 2023-09-10T08:16:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32244.557357 secs 2023-09-10T08:16:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:16:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:54 smithi170 ceph-mon[102832]: pgmap v17356: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 578 KiB/s wr, 13 op/s 2023-09-10T08:16:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32246.165016 secs 2023-09-10T08:16:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32244.557357 secs 2023-09-10T08:16:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:16:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:54 smithi082 ceph-mon[98694]: pgmap v17356: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 578 KiB/s wr, 13 op/s 2023-09-10T08:16:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32246.165016 secs 2023-09-10T08:16:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:16:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:16:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:16:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:16:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:16:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:16:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:16:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:56 smithi170 ceph-mon[102832]: pgmap v17357: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 795 KiB/s wr, 20 op/s 2023-09-10T08:16:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:56 smithi082 ceph-mon[98694]: pgmap v17357: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 795 KiB/s wr, 20 op/s 2023-09-10T08:16:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:56 smithi098 ceph-mon[103124]: pgmap v17357: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 795 KiB/s wr, 20 op/s 2023-09-10T08:16:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:58 smithi082 ceph-mon[98694]: pgmap v17358: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 490 KiB/s wr, 13 op/s 2023-09-10T08:16:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:58 smithi098 ceph-mon[103124]: pgmap v17358: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 490 KiB/s wr, 13 op/s 2023-09-10T08:16:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:58 smithi170 ceph-mon[102832]: pgmap v17358: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 490 KiB/s wr, 13 op/s 2023-09-10T08:16:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32249.557481 secs 2023-09-10T08:16:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32250.721742 secs 2023-09-10T08:16:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:16:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32251.165080 secs 2023-09-10T08:17:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32249.557481 secs 2023-09-10T08:17:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32250.721742 secs 2023-09-10T08:17:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:16:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32251.165080 secs 2023-09-10T08:17:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32249.557481 secs 2023-09-10T08:17:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32250.721742 secs 2023-09-10T08:17:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:16:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32251.165080 secs 2023-09-10T08:17:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:00 smithi082 ceph-mon[98694]: pgmap v17359: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 490 KiB/s wr, 13 op/s 2023-09-10T08:17:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:00 smithi098 ceph-mon[103124]: pgmap v17359: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 490 KiB/s wr, 13 op/s 2023-09-10T08:17:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:00 smithi170 ceph-mon[102832]: pgmap v17359: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 490 KiB/s wr, 13 op/s 2023-09-10T08:17:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:01 smithi098 ceph-mon[103124]: pgmap v17360: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 788 KiB/s wr, 28 op/s 2023-09-10T08:17:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:01 smithi170 ceph-mon[102832]: pgmap v17360: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 788 KiB/s wr, 28 op/s 2023-09-10T08:17:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:01 smithi082 ceph-mon[98694]: pgmap v17360: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 788 KiB/s wr, 28 op/s 2023-09-10T08:17:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32255.721800 secs 2023-09-10T08:17:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:03 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:17:04.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:03 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:17:04.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:03 smithi098 ceph-mon[103124]: osdmap e1153: 12 total, 12 up, 12 in 2023-09-10T08:17:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32255.721800 secs 2023-09-10T08:17:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:03 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:17:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:03 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:17:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:03 smithi170 ceph-mon[102832]: osdmap e1153: 12 total, 12 up, 12 in 2023-09-10T08:17:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32255.721800 secs 2023-09-10T08:17:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:03 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:17:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:03 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:17:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:03 smithi082 ceph-mon[98694]: osdmap e1153: 12 total, 12 up, 12 in 2023-09-10T08:17:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32254.557540 secs 2023-09-10T08:17:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:04 smithi098 ceph-mon[103124]: pgmap v17362: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 618 KiB/s wr, 26 op/s 2023-09-10T08:17:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32256.165145 secs 2023-09-10T08:17:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32254.557540 secs 2023-09-10T08:17:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:04 smithi170 ceph-mon[102832]: pgmap v17362: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 618 KiB/s wr, 26 op/s 2023-09-10T08:17:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32256.165145 secs 2023-09-10T08:17:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32254.557540 secs 2023-09-10T08:17:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:04 smithi082 ceph-mon[98694]: pgmap v17362: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 618 KiB/s wr, 26 op/s 2023-09-10T08:17:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32256.165145 secs 2023-09-10T08:17:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:17:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:17:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:06 smithi170 ceph-mon[102832]: pgmap v17363: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 510 KiB/s wr, 32 op/s 2023-09-10T08:17:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:17:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:17:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:06 smithi082 ceph-mon[98694]: pgmap v17363: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 510 KiB/s wr, 32 op/s 2023-09-10T08:17:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:17:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:17:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:06 smithi098 ceph-mon[103124]: pgmap v17363: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 510 KiB/s wr, 32 op/s 2023-09-10T08:17:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:17:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:17:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:17:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:17:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:07 smithi170 ceph-mon[102832]: osdmap e1154: 12 total, 12 up, 12 in 2023-09-10T08:17:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:17:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:17:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:17:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:17:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:07 smithi082 ceph-mon[98694]: osdmap e1154: 12 total, 12 up, 12 in 2023-09-10T08:17:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:17:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:17:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:17:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:17:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:07 smithi098 ceph-mon[103124]: osdmap e1154: 12 total, 12 up, 12 in 2023-09-10T08:17:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32259.557599 secs 2023-09-10T08:17:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:08 smithi170 ceph-mon[102832]: pgmap v17365: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 637 KiB/s wr, 40 op/s 2023-09-10T08:17:09.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32260.721884 secs 2023-09-10T08:17:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32259.557599 secs 2023-09-10T08:17:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:08 smithi082 ceph-mon[98694]: pgmap v17365: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 637 KiB/s wr, 40 op/s 2023-09-10T08:17:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32260.721884 secs 2023-09-10T08:17:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32259.557599 secs 2023-09-10T08:17:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:08 smithi098 ceph-mon[103124]: pgmap v17365: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 637 KiB/s wr, 40 op/s 2023-09-10T08:17:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32260.721884 secs 2023-09-10T08:17:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32261.165210 secs 2023-09-10T08:17:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32261.165210 secs 2023-09-10T08:17:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32261.165210 secs 2023-09-10T08:17:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:10 smithi170 ceph-mon[102832]: pgmap v17366: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 191 KiB/s wr, 17 op/s 2023-09-10T08:17:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:10 smithi082 ceph-mon[98694]: pgmap v17366: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 191 KiB/s wr, 17 op/s 2023-09-10T08:17:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:10 smithi098 ceph-mon[103124]: pgmap v17366: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 191 KiB/s wr, 17 op/s 2023-09-10T08:17:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:11 smithi098 ceph-mon[103124]: pgmap v17367: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 302 KiB/s wr, 25 op/s 2023-09-10T08:17:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:11 smithi170 ceph-mon[102832]: pgmap v17367: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 302 KiB/s wr, 25 op/s 2023-09-10T08:17:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:11 smithi082 ceph-mon[98694]: pgmap v17367: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 302 KiB/s wr, 25 op/s 2023-09-10T08:17:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32265.721943 secs 2023-09-10T08:17:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32265.721943 secs 2023-09-10T08:17:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32265.721943 secs 2023-09-10T08:17:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32264.557660 secs 2023-09-10T08:17:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:14 smithi082 ceph-mon[98694]: pgmap v17368: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 248 KiB/s wr, 20 op/s 2023-09-10T08:17:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32266.165279 secs 2023-09-10T08:17:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32264.557660 secs 2023-09-10T08:17:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:14 smithi098 ceph-mon[103124]: pgmap v17368: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 248 KiB/s wr, 20 op/s 2023-09-10T08:17:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32266.165279 secs 2023-09-10T08:17:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32264.557660 secs 2023-09-10T08:17:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:14 smithi170 ceph-mon[102832]: pgmap v17368: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 248 KiB/s wr, 20 op/s 2023-09-10T08:17:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32266.165279 secs 2023-09-10T08:17:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:15 smithi098 ceph-mon[103124]: pgmap v17369: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 13 op/s 2023-09-10T08:17:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:15 smithi170 ceph-mon[102832]: pgmap v17369: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 13 op/s 2023-09-10T08:17:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:15 smithi082 ceph-mon[98694]: pgmap v17369: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 13 op/s 2023-09-10T08:17:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:18 smithi082 ceph-mon[98694]: pgmap v17370: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T08:17:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32270.722003 secs 2023-09-10T08:17:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:18 smithi098 ceph-mon[103124]: pgmap v17370: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T08:17:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32270.722003 secs 2023-09-10T08:17:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:18 smithi170 ceph-mon[102832]: pgmap v17370: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T08:17:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32270.722003 secs 2023-09-10T08:17:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32269.557747 secs 2023-09-10T08:17:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32271.165342 secs 2023-09-10T08:17:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32269.557747 secs 2023-09-10T08:17:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32271.165342 secs 2023-09-10T08:17:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32269.557747 secs 2023-09-10T08:17:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32271.165342 secs 2023-09-10T08:17:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:20 smithi082 ceph-mon[98694]: pgmap v17371: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 104 KiB/s wr, 10 op/s 2023-09-10T08:17:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:20 smithi098 ceph-mon[103124]: pgmap v17371: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 104 KiB/s wr, 10 op/s 2023-09-10T08:17:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:20 smithi170 ceph-mon[102832]: pgmap v17371: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 104 KiB/s wr, 10 op/s 2023-09-10T08:17:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:21 smithi098 ceph-mon[103124]: pgmap v17372: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 15 op/s 2023-09-10T08:17:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:21 smithi170 ceph-mon[102832]: pgmap v17372: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 15 op/s 2023-09-10T08:17:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:21 smithi082 ceph-mon[98694]: pgmap v17372: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 104 KiB/s wr, 15 op/s 2023-09-10T08:17:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32275.722060 secs 2023-09-10T08:17:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32275.722060 secs 2023-09-10T08:17:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32275.722060 secs 2023-09-10T08:17:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32274.557806 secs 2023-09-10T08:17:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:24 smithi098 ceph-mon[103124]: pgmap v17373: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 10 op/s 2023-09-10T08:17:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32276.165407 secs 2023-09-10T08:17:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32274.557806 secs 2023-09-10T08:17:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:24 smithi170 ceph-mon[102832]: pgmap v17373: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 10 op/s 2023-09-10T08:17:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32276.165407 secs 2023-09-10T08:17:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32274.557806 secs 2023-09-10T08:17:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:24 smithi082 ceph-mon[98694]: pgmap v17373: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 10 op/s 2023-09-10T08:17:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32276.165407 secs 2023-09-10T08:17:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:25 smithi098 ceph-mon[103124]: pgmap v17374: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 15 op/s 2023-09-10T08:17:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:25 smithi170 ceph-mon[102832]: pgmap v17374: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 15 op/s 2023-09-10T08:17:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:25 smithi082 ceph-mon[98694]: pgmap v17374: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 15 op/s 2023-09-10T08:17:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:28 smithi082 ceph-mon[98694]: pgmap v17375: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 10 op/s 2023-09-10T08:17:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32280.722122 secs 2023-09-10T08:17:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:28 smithi098 ceph-mon[103124]: pgmap v17375: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 10 op/s 2023-09-10T08:17:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32280.722122 secs 2023-09-10T08:17:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:28 smithi170 ceph-mon[102832]: pgmap v17375: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 10 op/s 2023-09-10T08:17:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32280.722122 secs 2023-09-10T08:17:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32279.557914 secs 2023-09-10T08:17:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32281.165477 secs 2023-09-10T08:17:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32279.557914 secs 2023-09-10T08:17:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32281.165477 secs 2023-09-10T08:17:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32279.557914 secs 2023-09-10T08:17:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32281.165477 secs 2023-09-10T08:17:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:30 smithi098 ceph-mon[103124]: pgmap v17376: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 10 op/s 2023-09-10T08:17:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:30 smithi170 ceph-mon[102832]: pgmap v17376: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 10 op/s 2023-09-10T08:17:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:30 smithi082 ceph-mon[98694]: pgmap v17376: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 114 KiB/s wr, 10 op/s 2023-09-10T08:17:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:31 smithi098 ceph-mon[103124]: pgmap v17377: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 149 KiB/s wr, 15 op/s 2023-09-10T08:17:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:31 smithi170 ceph-mon[102832]: pgmap v17377: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 149 KiB/s wr, 15 op/s 2023-09-10T08:17:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:31 smithi082 ceph-mon[98694]: pgmap v17377: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 149 KiB/s wr, 15 op/s 2023-09-10T08:17:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32285.722210 secs 2023-09-10T08:17:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32285.722210 secs 2023-09-10T08:17:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32285.722210 secs 2023-09-10T08:17:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:17:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:17:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32286.165543 secs 2023-09-10T08:17:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:34 smithi098 ceph-mon[103124]: pgmap v17378: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 149 KiB/s wr, 10 op/s 2023-09-10T08:17:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:17:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:17:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32286.165543 secs 2023-09-10T08:17:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:34 smithi170 ceph-mon[102832]: pgmap v17378: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 149 KiB/s wr, 10 op/s 2023-09-10T08:17:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:17:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:17:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32286.165543 secs 2023-09-10T08:17:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:34 smithi082 ceph-mon[98694]: pgmap v17378: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 149 KiB/s wr, 10 op/s 2023-09-10T08:17:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:17:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:17:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:17:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:17:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:17:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:17:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:36 smithi098 ceph-mon[103124]: pgmap v17379: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 16 op/s 2023-09-10T08:17:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:17:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:17:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:17:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:36 smithi170 ceph-mon[102832]: pgmap v17379: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 16 op/s 2023-09-10T08:17:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:17:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:17:37.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:17:37.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:36 smithi082 ceph-mon[98694]: pgmap v17379: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 16 op/s 2023-09-10T08:17:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:38 smithi082 ceph-mon[98694]: pgmap v17380: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T08:17:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32290.722316 secs 2023-09-10T08:17:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:38 smithi098 ceph-mon[103124]: pgmap v17380: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T08:17:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32290.722316 secs 2023-09-10T08:17:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:38 smithi170 ceph-mon[102832]: pgmap v17380: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T08:17:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32290.722316 secs 2023-09-10T08:17:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32291.165607 secs 2023-09-10T08:17:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32291.165607 secs 2023-09-10T08:17:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32291.165607 secs 2023-09-10T08:17:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:40 smithi098 ceph-mon[103124]: pgmap v17381: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T08:17:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:40 smithi170 ceph-mon[102832]: pgmap v17381: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T08:17:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:40 smithi082 ceph-mon[98694]: pgmap v17381: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T08:17:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:41 smithi098 ceph-mon[103124]: pgmap v17382: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 425 KiB/s wr, 18 op/s 2023-09-10T08:17:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:41 smithi170 ceph-mon[102832]: pgmap v17382: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 425 KiB/s wr, 18 op/s 2023-09-10T08:17:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:41 smithi082 ceph-mon[98694]: pgmap v17382: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 425 KiB/s wr, 18 op/s 2023-09-10T08:17:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32295.722377 secs 2023-09-10T08:17:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32295.722377 secs 2023-09-10T08:17:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32295.722377 secs 2023-09-10T08:17:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:17:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:17:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32296.165674 secs 2023-09-10T08:17:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:44 smithi098 ceph-mon[103124]: pgmap v17383: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 391 KiB/s wr, 13 op/s 2023-09-10T08:17:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:17:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:17:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32296.165674 secs 2023-09-10T08:17:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:44 smithi170 ceph-mon[102832]: pgmap v17383: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 391 KiB/s wr, 13 op/s 2023-09-10T08:17:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:17:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:17:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32296.165674 secs 2023-09-10T08:17:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:44 smithi082 ceph-mon[98694]: pgmap v17383: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 391 KiB/s wr, 13 op/s 2023-09-10T08:17:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:17:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:17:47.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:46 smithi098 ceph-mon[103124]: pgmap v17384: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 595 KiB/s wr, 20 op/s 2023-09-10T08:17:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:17:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:17:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:46 smithi170 ceph-mon[102832]: pgmap v17384: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 595 KiB/s wr, 20 op/s 2023-09-10T08:17:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:17:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:17:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:46 smithi082 ceph-mon[98694]: pgmap v17384: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 595 KiB/s wr, 20 op/s 2023-09-10T08:17:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:48 smithi082 ceph-mon[98694]: pgmap v17385: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 468 KiB/s wr, 13 op/s 2023-09-10T08:17:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32300.722458 secs 2023-09-10T08:17:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:48 smithi098 ceph-mon[103124]: pgmap v17385: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 468 KiB/s wr, 13 op/s 2023-09-10T08:17:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32300.722458 secs 2023-09-10T08:17:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:48 smithi170 ceph-mon[102832]: pgmap v17385: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 468 KiB/s wr, 13 op/s 2023-09-10T08:17:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32300.722458 secs 2023-09-10T08:17:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32301.165740 secs 2023-09-10T08:17:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32301.165740 secs 2023-09-10T08:17:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32301.165740 secs 2023-09-10T08:17:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:50 smithi098 ceph-mon[103124]: pgmap v17386: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 468 KiB/s wr, 13 op/s 2023-09-10T08:17:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:50 smithi170 ceph-mon[102832]: pgmap v17386: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 468 KiB/s wr, 13 op/s 2023-09-10T08:17:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:50 smithi082 ceph-mon[98694]: pgmap v17386: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 468 KiB/s wr, 13 op/s 2023-09-10T08:17:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:51 smithi098 ceph-mon[103124]: pgmap v17387: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 706 KiB/s wr, 20 op/s 2023-09-10T08:17:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:51 smithi170 ceph-mon[102832]: pgmap v17387: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 706 KiB/s wr, 20 op/s 2023-09-10T08:17:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:51 smithi082 ceph-mon[98694]: pgmap v17387: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 706 KiB/s wr, 20 op/s 2023-09-10T08:17:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32305.722496 secs 2023-09-10T08:17:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32305.722496 secs 2023-09-10T08:17:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32305.722496 secs 2023-09-10T08:17:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:17:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:17:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32306.165825 secs 2023-09-10T08:17:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:54 smithi098 ceph-mon[103124]: pgmap v17388: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 443 KiB/s wr, 13 op/s 2023-09-10T08:17:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:17:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:17:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32306.165825 secs 2023-09-10T08:17:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:54 smithi170 ceph-mon[102832]: pgmap v17388: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 443 KiB/s wr, 13 op/s 2023-09-10T08:17:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:17:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:17:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32306.165825 secs 2023-09-10T08:17:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:54 smithi082 ceph-mon[98694]: pgmap v17388: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 443 KiB/s wr, 13 op/s 2023-09-10T08:17:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:17:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:17:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:56 smithi098 ceph-mon[103124]: pgmap v17389: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 707 KiB/s wr, 21 op/s 2023-09-10T08:17:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:17:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:17:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:56 smithi170 ceph-mon[102832]: pgmap v17389: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 707 KiB/s wr, 21 op/s 2023-09-10T08:17:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:17:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:17:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:56 smithi082 ceph-mon[98694]: pgmap v17389: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 707 KiB/s wr, 21 op/s 2023-09-10T08:17:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:58 smithi098 ceph-mon[103124]: pgmap v17390: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 502 KiB/s wr, 14 op/s 2023-09-10T08:17:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32310.722556 secs 2023-09-10T08:17:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:58 smithi170 ceph-mon[102832]: pgmap v17390: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 502 KiB/s wr, 14 op/s 2023-09-10T08:17:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32310.722556 secs 2023-09-10T08:17:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:58 smithi082 ceph-mon[98694]: pgmap v17390: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 502 KiB/s wr, 14 op/s 2023-09-10T08:17:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32310.722556 secs 2023-09-10T08:18:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32309.558272 secs 2023-09-10T08:18:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:17:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32311.165890 secs 2023-09-10T08:18:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32309.558272 secs 2023-09-10T08:18:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:17:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32311.165890 secs 2023-09-10T08:18:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32309.558272 secs 2023-09-10T08:18:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:17:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32311.165890 secs 2023-09-10T08:18:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:18:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:18:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:00 smithi098 ceph-mon[103124]: pgmap v17391: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 502 KiB/s wr, 14 op/s 2023-09-10T08:18:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:18:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:18:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:00 smithi170 ceph-mon[102832]: pgmap v17391: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 502 KiB/s wr, 14 op/s 2023-09-10T08:18:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:18:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:18:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:00 smithi082 ceph-mon[98694]: pgmap v17391: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 502 KiB/s wr, 14 op/s 2023-09-10T08:18:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:01 smithi098 ceph-mon[103124]: pgmap v17392: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 773 KiB/s wr, 20 op/s 2023-09-10T08:18:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:01 smithi170 ceph-mon[102832]: pgmap v17392: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 773 KiB/s wr, 20 op/s 2023-09-10T08:18:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:01 smithi082 ceph-mon[98694]: pgmap v17392: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 773 KiB/s wr, 20 op/s 2023-09-10T08:18:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32315.722657 secs 2023-09-10T08:18:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32315.722657 secs 2023-09-10T08:18:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32315.722657 secs 2023-09-10T08:18:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32314.558331 secs 2023-09-10T08:18:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:18:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:04 smithi098 ceph-mon[103124]: osdmap e1155: 12 total, 12 up, 12 in 2023-09-10T08:18:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32316.165959 secs 2023-09-10T08:18:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:04 smithi098 ceph-mon[103124]: pgmap v17394: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 642 KiB/s wr, 16 op/s 2023-09-10T08:18:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32314.558331 secs 2023-09-10T08:18:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:18:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:04 smithi170 ceph-mon[102832]: osdmap e1155: 12 total, 12 up, 12 in 2023-09-10T08:18:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32316.165959 secs 2023-09-10T08:18:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:04 smithi170 ceph-mon[102832]: pgmap v17394: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 642 KiB/s wr, 16 op/s 2023-09-10T08:18:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32314.558331 secs 2023-09-10T08:18:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:18:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:04 smithi082 ceph-mon[98694]: osdmap e1155: 12 total, 12 up, 12 in 2023-09-10T08:18:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32316.165959 secs 2023-09-10T08:18:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:04 smithi082 ceph-mon[98694]: pgmap v17394: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 642 KiB/s wr, 16 op/s 2023-09-10T08:18:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:18:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:06 smithi098 ceph-mon[103124]: pgmap v17395: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 545 KiB/s wr, 32 op/s 2023-09-10T08:18:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:18:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:06 smithi170 ceph-mon[102832]: pgmap v17395: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 545 KiB/s wr, 32 op/s 2023-09-10T08:18:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:18:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:06 smithi082 ceph-mon[98694]: pgmap v17395: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 545 KiB/s wr, 32 op/s 2023-09-10T08:18:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:18:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:07 smithi098 ceph-mon[103124]: osdmap e1156: 12 total, 12 up, 12 in 2023-09-10T08:18:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:18:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:07 smithi170 ceph-mon[102832]: osdmap e1156: 12 total, 12 up, 12 in 2023-09-10T08:18:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:18:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:07 smithi082 ceph-mon[98694]: osdmap e1156: 12 total, 12 up, 12 in 2023-09-10T08:18:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:08 smithi098 ceph-mon[103124]: pgmap v17397: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 682 KiB/s wr, 41 op/s 2023-09-10T08:18:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:18:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:18:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32320.722743 secs 2023-09-10T08:18:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:18:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:18:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:18:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:08 smithi170 ceph-mon[102832]: pgmap v17397: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 682 KiB/s wr, 41 op/s 2023-09-10T08:18:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:18:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:18:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32320.722743 secs 2023-09-10T08:18:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:18:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:18:09.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:18:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:08 smithi082 ceph-mon[98694]: pgmap v17397: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 682 KiB/s wr, 41 op/s 2023-09-10T08:18:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:18:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:18:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32320.722743 secs 2023-09-10T08:18:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:18:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:18:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:18:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32319.558390 secs 2023-09-10T08:18:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32321.166025 secs 2023-09-10T08:18:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32319.558390 secs 2023-09-10T08:18:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32321.166025 secs 2023-09-10T08:18:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32319.558390 secs 2023-09-10T08:18:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32321.166025 secs 2023-09-10T08:18:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:10 smithi098 ceph-mon[103124]: pgmap v17398: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 275 KiB/s wr, 31 op/s 2023-09-10T08:18:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:10 smithi170 ceph-mon[102832]: pgmap v17398: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 275 KiB/s wr, 31 op/s 2023-09-10T08:18:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:10 smithi082 ceph-mon[98694]: pgmap v17398: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 275 KiB/s wr, 31 op/s 2023-09-10T08:18:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:18:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:12 smithi098 ceph-mon[103124]: pgmap v17399: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 436 KiB/s wr, 39 op/s 2023-09-10T08:18:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:18:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:12 smithi170 ceph-mon[102832]: pgmap v17399: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 436 KiB/s wr, 39 op/s 2023-09-10T08:18:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:18:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:12 smithi082 ceph-mon[98694]: pgmap v17399: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 436 KiB/s wr, 39 op/s 2023-09-10T08:18:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32325.722802 secs 2023-09-10T08:18:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32325.722802 secs 2023-09-10T08:18:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32325.722802 secs 2023-09-10T08:18:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32324.558453 secs 2023-09-10T08:18:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:14 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:18:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:14 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32326.166092 secs 2023-09-10T08:18:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:14 smithi170 ceph-mon[102832]: pgmap v17400: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 357 KiB/s wr, 32 op/s 2023-09-10T08:18:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32324.558453 secs 2023-09-10T08:18:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:14 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:18:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:14 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32326.166092 secs 2023-09-10T08:18:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:14 smithi082 ceph-mon[98694]: pgmap v17400: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 357 KiB/s wr, 32 op/s 2023-09-10T08:18:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32324.558453 secs 2023-09-10T08:18:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:14 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:18:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:14 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32326.166092 secs 2023-09-10T08:18:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:14 smithi098 ceph-mon[103124]: pgmap v17400: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 357 KiB/s wr, 32 op/s 2023-09-10T08:18:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:15 smithi098 ceph-mon[103124]: pgmap v17401: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 167 KiB/s wr, 14 op/s 2023-09-10T08:18:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:15 smithi170 ceph-mon[102832]: pgmap v17401: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 167 KiB/s wr, 14 op/s 2023-09-10T08:18:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:15 smithi082 ceph-mon[98694]: pgmap v17401: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 167 KiB/s wr, 14 op/s 2023-09-10T08:18:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:18 smithi098 ceph-mon[103124]: pgmap v17402: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 13 op/s 2023-09-10T08:18:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32330.722861 secs 2023-09-10T08:18:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:18 smithi170 ceph-mon[102832]: pgmap v17402: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 13 op/s 2023-09-10T08:18:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32330.722861 secs 2023-09-10T08:18:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:18 smithi082 ceph-mon[98694]: pgmap v17402: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 13 op/s 2023-09-10T08:18:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32330.722861 secs 2023-09-10T08:18:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32329.558562 secs 2023-09-10T08:18:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32331.166156 secs 2023-09-10T08:18:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32329.558562 secs 2023-09-10T08:18:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32331.166156 secs 2023-09-10T08:18:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32329.558562 secs 2023-09-10T08:18:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32331.166156 secs 2023-09-10T08:18:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:20 smithi098 ceph-mon[103124]: pgmap v17403: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T08:18:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:20 smithi170 ceph-mon[102832]: pgmap v17403: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T08:18:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:20 smithi082 ceph-mon[98694]: pgmap v17403: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T08:18:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:21 smithi098 ceph-mon[103124]: pgmap v17404: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 211 KiB/s wr, 16 op/s 2023-09-10T08:18:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:21 smithi170 ceph-mon[102832]: pgmap v17404: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 211 KiB/s wr, 16 op/s 2023-09-10T08:18:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:21 smithi082 ceph-mon[98694]: pgmap v17404: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 211 KiB/s wr, 16 op/s 2023-09-10T08:18:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32335.722920 secs 2023-09-10T08:18:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32335.722920 secs 2023-09-10T08:18:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32335.722920 secs 2023-09-10T08:18:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32334.558647 secs 2023-09-10T08:18:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32336.166222 secs 2023-09-10T08:18:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:24 smithi098 ceph-mon[103124]: pgmap v17405: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T08:18:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32334.558647 secs 2023-09-10T08:18:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32336.166222 secs 2023-09-10T08:18:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:24 smithi170 ceph-mon[102832]: pgmap v17405: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T08:18:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32334.558647 secs 2023-09-10T08:18:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32336.166222 secs 2023-09-10T08:18:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:24 smithi082 ceph-mon[98694]: pgmap v17405: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T08:18:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:25 smithi098 ceph-mon[103124]: pgmap v17406: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 212 KiB/s wr, 16 op/s 2023-09-10T08:18:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:25 smithi170 ceph-mon[102832]: pgmap v17406: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 212 KiB/s wr, 16 op/s 2023-09-10T08:18:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:25 smithi082 ceph-mon[98694]: pgmap v17406: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 212 KiB/s wr, 16 op/s 2023-09-10T08:18:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:28 smithi098 ceph-mon[103124]: pgmap v17407: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 10 op/s 2023-09-10T08:18:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32340.723004 secs 2023-09-10T08:18:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:28 smithi170 ceph-mon[102832]: pgmap v17407: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 10 op/s 2023-09-10T08:18:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32340.723004 secs 2023-09-10T08:18:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:28 smithi082 ceph-mon[98694]: pgmap v17407: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 10 op/s 2023-09-10T08:18:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32340.723004 secs 2023-09-10T08:18:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32339.558706 secs 2023-09-10T08:18:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32341.166288 secs 2023-09-10T08:18:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32339.558706 secs 2023-09-10T08:18:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32341.166288 secs 2023-09-10T08:18:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32339.558706 secs 2023-09-10T08:18:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32341.166288 secs 2023-09-10T08:18:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:30 smithi098 ceph-mon[103124]: pgmap v17408: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 10 op/s 2023-09-10T08:18:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:30 smithi170 ceph-mon[102832]: pgmap v17408: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 10 op/s 2023-09-10T08:18:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:30 smithi082 ceph-mon[98694]: pgmap v17408: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 10 op/s 2023-09-10T08:18:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:31 smithi098 ceph-mon[103124]: pgmap v17409: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 234 KiB/s wr, 15 op/s 2023-09-10T08:18:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:31 smithi170 ceph-mon[102832]: pgmap v17409: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 234 KiB/s wr, 15 op/s 2023-09-10T08:18:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:31 smithi082 ceph-mon[98694]: pgmap v17409: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 234 KiB/s wr, 15 op/s 2023-09-10T08:18:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32345.723064 secs 2023-09-10T08:18:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32345.723064 secs 2023-09-10T08:18:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32345.723064 secs 2023-09-10T08:18:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:18:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32346.166355 secs 2023-09-10T08:18:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:34 smithi098 ceph-mon[103124]: pgmap v17410: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T08:18:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:18:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32346.166355 secs 2023-09-10T08:18:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:34 smithi170 ceph-mon[102832]: pgmap v17410: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T08:18:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:18:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32346.166355 secs 2023-09-10T08:18:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:34 smithi082 ceph-mon[98694]: pgmap v17410: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T08:18:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:18:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:18:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:18:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:18:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:18:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:36 smithi098 ceph-mon[103124]: pgmap v17411: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 298 KiB/s wr, 16 op/s 2023-09-10T08:18:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:18:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:18:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:36 smithi170 ceph-mon[102832]: pgmap v17411: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 298 KiB/s wr, 16 op/s 2023-09-10T08:18:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:18:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:18:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:36 smithi082 ceph-mon[98694]: pgmap v17411: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 298 KiB/s wr, 16 op/s 2023-09-10T08:18:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:38 smithi098 ceph-mon[103124]: pgmap v17412: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T08:18:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32350.723123 secs 2023-09-10T08:18:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:38 smithi170 ceph-mon[102832]: pgmap v17412: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T08:18:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32350.723123 secs 2023-09-10T08:18:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:38 smithi082 ceph-mon[98694]: pgmap v17412: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T08:18:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32350.723123 secs 2023-09-10T08:18:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32351.166424 secs 2023-09-10T08:18:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32351.166424 secs 2023-09-10T08:18:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32351.166424 secs 2023-09-10T08:18:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:40 smithi098 ceph-mon[103124]: pgmap v17413: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T08:18:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:40 smithi170 ceph-mon[102832]: pgmap v17413: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T08:18:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:40 smithi082 ceph-mon[98694]: pgmap v17413: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T08:18:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:41 smithi098 ceph-mon[103124]: pgmap v17414: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 447 KiB/s wr, 18 op/s 2023-09-10T08:18:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:41 smithi170 ceph-mon[102832]: pgmap v17414: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 447 KiB/s wr, 18 op/s 2023-09-10T08:18:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:41 smithi082 ceph-mon[98694]: pgmap v17414: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 447 KiB/s wr, 18 op/s 2023-09-10T08:18:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32355.723183 secs 2023-09-10T08:18:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32355.723183 secs 2023-09-10T08:18:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32355.723183 secs 2023-09-10T08:18:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:18:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32356.166487 secs 2023-09-10T08:18:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:44 smithi098 ceph-mon[103124]: pgmap v17415: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 400 KiB/s wr, 13 op/s 2023-09-10T08:18:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:18:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32356.166487 secs 2023-09-10T08:18:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:44 smithi170 ceph-mon[102832]: pgmap v17415: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 400 KiB/s wr, 13 op/s 2023-09-10T08:18:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:18:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32356.166487 secs 2023-09-10T08:18:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:44 smithi082 ceph-mon[98694]: pgmap v17415: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 400 KiB/s wr, 13 op/s 2023-09-10T08:18:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:18:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:46 smithi098 ceph-mon[103124]: pgmap v17416: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 605 KiB/s wr, 20 op/s 2023-09-10T08:18:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:18:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:46 smithi170 ceph-mon[102832]: pgmap v17416: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 605 KiB/s wr, 20 op/s 2023-09-10T08:18:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:18:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:46 smithi082 ceph-mon[98694]: pgmap v17416: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 605 KiB/s wr, 20 op/s 2023-09-10T08:18:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:48 smithi098 ceph-mon[103124]: pgmap v17417: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 469 KiB/s wr, 14 op/s 2023-09-10T08:18:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32360.723242 secs 2023-09-10T08:18:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:48 smithi170 ceph-mon[102832]: pgmap v17417: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 469 KiB/s wr, 14 op/s 2023-09-10T08:18:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32360.723242 secs 2023-09-10T08:18:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:48 smithi082 ceph-mon[98694]: pgmap v17417: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 469 KiB/s wr, 14 op/s 2023-09-10T08:18:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32360.723242 secs 2023-09-10T08:18:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32361.166551 secs 2023-09-10T08:18:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32361.166551 secs 2023-09-10T08:18:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32361.166551 secs 2023-09-10T08:18:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:50 smithi098 ceph-mon[103124]: pgmap v17418: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 469 KiB/s wr, 14 op/s 2023-09-10T08:18:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:50 smithi170 ceph-mon[102832]: pgmap v17418: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 469 KiB/s wr, 14 op/s 2023-09-10T08:18:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:50 smithi082 ceph-mon[98694]: pgmap v17418: 97 pgs: 97 active+clean; 11 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 469 KiB/s wr, 14 op/s 2023-09-10T08:18:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:18:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:52 smithi098 ceph-mon[103124]: pgmap v17419: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 845 KiB/s wr, 21 op/s 2023-09-10T08:18:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:18:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:52 smithi170 ceph-mon[102832]: pgmap v17419: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 845 KiB/s wr, 21 op/s 2023-09-10T08:18:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:18:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:52 smithi082 ceph-mon[98694]: pgmap v17419: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 845 KiB/s wr, 21 op/s 2023-09-10T08:18:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32365.723326 secs 2023-09-10T08:18:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32365.723326 secs 2023-09-10T08:18:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32365.723326 secs 2023-09-10T08:18:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32364.559159 secs 2023-09-10T08:18:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:18:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32366.166616 secs 2023-09-10T08:18:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:54 smithi098 ceph-mon[103124]: pgmap v17420: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 581 KiB/s wr, 14 op/s 2023-09-10T08:18:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32364.559159 secs 2023-09-10T08:18:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:18:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32366.166616 secs 2023-09-10T08:18:55.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:54 smithi170 ceph-mon[102832]: pgmap v17420: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 581 KiB/s wr, 14 op/s 2023-09-10T08:18:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32364.559159 secs 2023-09-10T08:18:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:18:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32366.166616 secs 2023-09-10T08:18:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:54 smithi082 ceph-mon[98694]: pgmap v17420: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 581 KiB/s wr, 14 op/s 2023-09-10T08:18:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:18:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:18:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:18:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:18:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:18:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:18:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:18:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:56 smithi170 ceph-mon[102832]: pgmap v17421: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 800 KiB/s wr, 21 op/s 2023-09-10T08:18:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:56 smithi082 ceph-mon[98694]: pgmap v17421: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 800 KiB/s wr, 21 op/s 2023-09-10T08:18:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:56 smithi098 ceph-mon[103124]: pgmap v17421: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 800 KiB/s wr, 21 op/s 2023-09-10T08:18:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:58 smithi098 ceph-mon[103124]: pgmap v17422: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 594 KiB/s wr, 14 op/s 2023-09-10T08:18:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32370.723387 secs 2023-09-10T08:18:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:58 smithi170 ceph-mon[102832]: pgmap v17422: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 594 KiB/s wr, 14 op/s 2023-09-10T08:18:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32370.723387 secs 2023-09-10T08:18:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:58 smithi082 ceph-mon[98694]: pgmap v17422: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 594 KiB/s wr, 14 op/s 2023-09-10T08:18:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32370.723387 secs 2023-09-10T08:19:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32369.559221 secs 2023-09-10T08:19:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:18:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32371.166682 secs 2023-09-10T08:19:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32369.559221 secs 2023-09-10T08:19:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:18:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32371.166682 secs 2023-09-10T08:19:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32369.559221 secs 2023-09-10T08:19:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:18:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32371.166682 secs 2023-09-10T08:19:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:00 smithi098 ceph-mon[103124]: pgmap v17423: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 594 KiB/s wr, 14 op/s 2023-09-10T08:19:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:00 smithi170 ceph-mon[102832]: pgmap v17423: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 594 KiB/s wr, 14 op/s 2023-09-10T08:19:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:00 smithi082 ceph-mon[98694]: pgmap v17423: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 594 KiB/s wr, 14 op/s 2023-09-10T08:19:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:02 smithi170 ceph-mon[102832]: pgmap v17424: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 756 KiB/s wr, 20 op/s 2023-09-10T08:19:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:02 smithi082 ceph-mon[98694]: pgmap v17424: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 756 KiB/s wr, 20 op/s 2023-09-10T08:19:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:02 smithi098 ceph-mon[103124]: pgmap v17424: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 756 KiB/s wr, 20 op/s 2023-09-10T08:19:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32375.723472 secs 2023-09-10T08:19:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32375.723472 secs 2023-09-10T08:19:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32375.723472 secs 2023-09-10T08:19:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32374.559281 secs 2023-09-10T08:19:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32376.166749 secs 2023-09-10T08:19:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:04 smithi098 ceph-mon[103124]: pgmap v17425: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 380 KiB/s wr, 13 op/s 2023-09-10T08:19:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32374.559281 secs 2023-09-10T08:19:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32376.166749 secs 2023-09-10T08:19:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:04 smithi170 ceph-mon[102832]: pgmap v17425: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 380 KiB/s wr, 13 op/s 2023-09-10T08:19:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32374.559281 secs 2023-09-10T08:19:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32376.166749 secs 2023-09-10T08:19:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:04 smithi082 ceph-mon[98694]: pgmap v17425: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 380 KiB/s wr, 13 op/s 2023-09-10T08:19:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:06 smithi170 ceph-mon[102832]: pgmap v17426: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 536 KiB/s wr, 32 op/s 2023-09-10T08:19:06.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:06 smithi082 ceph-mon[98694]: pgmap v17426: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 536 KiB/s wr, 32 op/s 2023-09-10T08:19:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:06 smithi098 ceph-mon[103124]: pgmap v17426: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 536 KiB/s wr, 32 op/s 2023-09-10T08:19:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:08 smithi098 ceph-mon[103124]: pgmap v17427: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 318 KiB/s wr, 26 op/s 2023-09-10T08:19:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32380.723494 secs 2023-09-10T08:19:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:19:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:08 smithi170 ceph-mon[102832]: pgmap v17427: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 318 KiB/s wr, 26 op/s 2023-09-10T08:19:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32380.723494 secs 2023-09-10T08:19:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:19:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:08 smithi082 ceph-mon[98694]: pgmap v17427: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 318 KiB/s wr, 26 op/s 2023-09-10T08:19:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32380.723494 secs 2023-09-10T08:19:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:19:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32379.559343 secs 2023-09-10T08:19:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:09 smithi098 ceph-mon[103124]: osdmap e1157: 12 total, 12 up, 12 in 2023-09-10T08:19:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32381.166817 secs 2023-09-10T08:19:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:19:10.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:19:10.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:19:10.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:19:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32379.559343 secs 2023-09-10T08:19:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:09 smithi170 ceph-mon[102832]: osdmap e1157: 12 total, 12 up, 12 in 2023-09-10T08:19:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32381.166817 secs 2023-09-10T08:19:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:19:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:19:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:19:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:19:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32379.559343 secs 2023-09-10T08:19:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:09 smithi082 ceph-mon[98694]: osdmap e1157: 12 total, 12 up, 12 in 2023-09-10T08:19:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32381.166817 secs 2023-09-10T08:19:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:19:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:19:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:19:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:19:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:19:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:19:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:19:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:10 smithi098 ceph-mon[103124]: pgmap v17429: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 381 KiB/s wr, 31 op/s 2023-09-10T08:19:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:19:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:19:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:19:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:10 smithi170 ceph-mon[102832]: pgmap v17429: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 381 KiB/s wr, 31 op/s 2023-09-10T08:19:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:19:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:19:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:19:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:10 smithi082 ceph-mon[98694]: pgmap v17429: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 381 KiB/s wr, 31 op/s 2023-09-10T08:19:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:12 smithi170 ceph-mon[102832]: pgmap v17430: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 283 KiB/s wr, 30 op/s 2023-09-10T08:19:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:12 smithi082 ceph-mon[98694]: pgmap v17430: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 283 KiB/s wr, 30 op/s 2023-09-10T08:19:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:12 smithi098 ceph-mon[103124]: pgmap v17430: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 283 KiB/s wr, 30 op/s 2023-09-10T08:19:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:13 smithi098 ceph-mon[103124]: osdmap e1158: 12 total, 12 up, 12 in 2023-09-10T08:19:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32385.723578 secs 2023-09-10T08:19:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:13 smithi170 ceph-mon[102832]: osdmap e1158: 12 total, 12 up, 12 in 2023-09-10T08:19:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32385.723578 secs 2023-09-10T08:19:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:13 smithi082 ceph-mon[98694]: osdmap e1158: 12 total, 12 up, 12 in 2023-09-10T08:19:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32385.723578 secs 2023-09-10T08:19:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32384.559409 secs 2023-09-10T08:19:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32386.166880 secs 2023-09-10T08:19:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:14 smithi098 ceph-mon[103124]: pgmap v17432: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 119 KiB/s wr, 9 op/s 2023-09-10T08:19:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32384.559409 secs 2023-09-10T08:19:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32386.166880 secs 2023-09-10T08:19:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:14 smithi170 ceph-mon[102832]: pgmap v17432: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 119 KiB/s wr, 9 op/s 2023-09-10T08:19:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32384.559409 secs 2023-09-10T08:19:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32386.166880 secs 2023-09-10T08:19:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:14 smithi082 ceph-mon[98694]: pgmap v17432: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 119 KiB/s wr, 9 op/s 2023-09-10T08:19:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:19:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:19:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:16 smithi098 ceph-mon[103124]: pgmap v17433: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 156 KiB/s wr, 18 op/s 2023-09-10T08:19:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:19:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:19:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:16 smithi170 ceph-mon[102832]: pgmap v17433: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 156 KiB/s wr, 18 op/s 2023-09-10T08:19:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:19:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:19:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:16 smithi082 ceph-mon[98694]: pgmap v17433: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 156 KiB/s wr, 18 op/s 2023-09-10T08:19:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:19:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:19:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:18 smithi098 ceph-mon[103124]: pgmap v17434: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 15 op/s 2023-09-10T08:19:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32390.723638 secs 2023-09-10T08:19:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:19:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:19:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:18 smithi170 ceph-mon[102832]: pgmap v17434: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 15 op/s 2023-09-10T08:19:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32390.723638 secs 2023-09-10T08:19:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:19:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:19:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:18 smithi082 ceph-mon[98694]: pgmap v17434: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 15 op/s 2023-09-10T08:19:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32390.723638 secs 2023-09-10T08:19:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32389.559471 secs 2023-09-10T08:19:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32391.166948 secs 2023-09-10T08:19:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32389.559471 secs 2023-09-10T08:19:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32391.166948 secs 2023-09-10T08:19:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32389.559471 secs 2023-09-10T08:19:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32391.166948 secs 2023-09-10T08:19:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:20 smithi098 ceph-mon[103124]: pgmap v17435: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 14 op/s 2023-09-10T08:19:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:20 smithi170 ceph-mon[102832]: pgmap v17435: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 14 op/s 2023-09-10T08:19:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:20 smithi082 ceph-mon[98694]: pgmap v17435: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 14 op/s 2023-09-10T08:19:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:22 smithi170 ceph-mon[102832]: pgmap v17436: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 13 op/s 2023-09-10T08:19:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:22 smithi082 ceph-mon[98694]: pgmap v17436: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 13 op/s 2023-09-10T08:19:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:22 smithi098 ceph-mon[103124]: pgmap v17436: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 13 op/s 2023-09-10T08:19:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32395.723696 secs 2023-09-10T08:19:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32395.723696 secs 2023-09-10T08:19:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32395.723696 secs 2023-09-10T08:19:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32394.559554 secs 2023-09-10T08:19:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32396.167013 secs 2023-09-10T08:19:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:24 smithi098 ceph-mon[103124]: pgmap v17437: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 12 op/s 2023-09-10T08:19:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32394.559554 secs 2023-09-10T08:19:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32396.167013 secs 2023-09-10T08:19:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:24 smithi170 ceph-mon[102832]: pgmap v17437: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 12 op/s 2023-09-10T08:19:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32394.559554 secs 2023-09-10T08:19:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32396.167013 secs 2023-09-10T08:19:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:24 smithi082 ceph-mon[98694]: pgmap v17437: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 12 op/s 2023-09-10T08:19:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:26 smithi170 ceph-mon[102832]: pgmap v17438: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 140 KiB/s wr, 16 op/s 2023-09-10T08:19:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:26 smithi082 ceph-mon[98694]: pgmap v17438: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 140 KiB/s wr, 16 op/s 2023-09-10T08:19:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:26 smithi098 ceph-mon[103124]: pgmap v17438: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 140 KiB/s wr, 16 op/s 2023-09-10T08:19:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:28 smithi098 ceph-mon[103124]: pgmap v17439: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 10 op/s 2023-09-10T08:19:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32400.723756 secs 2023-09-10T08:19:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:28 smithi170 ceph-mon[102832]: pgmap v17439: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 10 op/s 2023-09-10T08:19:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32400.723756 secs 2023-09-10T08:19:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:28 smithi082 ceph-mon[98694]: pgmap v17439: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 10 op/s 2023-09-10T08:19:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32400.723756 secs 2023-09-10T08:19:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32399.559636 secs 2023-09-10T08:19:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32401.167080 secs 2023-09-10T08:19:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32399.559636 secs 2023-09-10T08:19:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32401.167080 secs 2023-09-10T08:19:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32399.559636 secs 2023-09-10T08:19:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32401.167080 secs 2023-09-10T08:19:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:30 smithi098 ceph-mon[103124]: pgmap v17440: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 10 op/s 2023-09-10T08:19:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:30 smithi170 ceph-mon[102832]: pgmap v17440: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 10 op/s 2023-09-10T08:19:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:30 smithi082 ceph-mon[98694]: pgmap v17440: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 10 op/s 2023-09-10T08:19:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:32 smithi170 ceph-mon[102832]: pgmap v17441: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 116 KiB/s wr, 15 op/s 2023-09-10T08:19:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:32 smithi082 ceph-mon[98694]: pgmap v17441: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 116 KiB/s wr, 15 op/s 2023-09-10T08:19:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:32 smithi098 ceph-mon[103124]: pgmap v17441: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 116 KiB/s wr, 15 op/s 2023-09-10T08:19:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32405.723836 secs 2023-09-10T08:19:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32405.723836 secs 2023-09-10T08:19:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32405.723836 secs 2023-09-10T08:19:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32404.559727 secs 2023-09-10T08:19:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32406.167146 secs 2023-09-10T08:19:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:34 smithi098 ceph-mon[103124]: pgmap v17442: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 10 op/s 2023-09-10T08:19:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32404.559727 secs 2023-09-10T08:19:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32406.167146 secs 2023-09-10T08:19:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:34 smithi170 ceph-mon[102832]: pgmap v17442: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 10 op/s 2023-09-10T08:19:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32404.559727 secs 2023-09-10T08:19:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32406.167146 secs 2023-09-10T08:19:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:34 smithi082 ceph-mon[98694]: pgmap v17442: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 115 KiB/s wr, 10 op/s 2023-09-10T08:19:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:36 smithi170 ceph-mon[102832]: pgmap v17443: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 327 KiB/s wr, 16 op/s 2023-09-10T08:19:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:36 smithi082 ceph-mon[98694]: pgmap v17443: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 327 KiB/s wr, 16 op/s 2023-09-10T08:19:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:36 smithi098 ceph-mon[103124]: pgmap v17443: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 327 KiB/s wr, 16 op/s 2023-09-10T08:19:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:38 smithi098 ceph-mon[103124]: pgmap v17444: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 212 KiB/s wr, 11 op/s 2023-09-10T08:19:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32410.723894 secs 2023-09-10T08:19:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:38 smithi170 ceph-mon[102832]: pgmap v17444: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 212 KiB/s wr, 11 op/s 2023-09-10T08:19:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32410.723894 secs 2023-09-10T08:19:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:38 smithi082 ceph-mon[98694]: pgmap v17444: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 212 KiB/s wr, 11 op/s 2023-09-10T08:19:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32410.723894 secs 2023-09-10T08:19:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32411.167211 secs 2023-09-10T08:19:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:19:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32411.167211 secs 2023-09-10T08:19:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:19:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32411.167211 secs 2023-09-10T08:19:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:19:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:19:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:19:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:19:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:40 smithi098 ceph-mon[103124]: pgmap v17445: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 212 KiB/s wr, 11 op/s 2023-09-10T08:19:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:19:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:19:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:19:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:40 smithi170 ceph-mon[102832]: pgmap v17445: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 212 KiB/s wr, 11 op/s 2023-09-10T08:19:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:19:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:19:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:19:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:40 smithi082 ceph-mon[98694]: pgmap v17445: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 212 KiB/s wr, 11 op/s 2023-09-10T08:19:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:42 smithi170 ceph-mon[102832]: pgmap v17446: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 428 KiB/s wr, 17 op/s 2023-09-10T08:19:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:42 smithi082 ceph-mon[98694]: pgmap v17446: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 428 KiB/s wr, 17 op/s 2023-09-10T08:19:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:42 smithi098 ceph-mon[103124]: pgmap v17446: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 428 KiB/s wr, 17 op/s 2023-09-10T08:19:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32415.723953 secs 2023-09-10T08:19:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32415.723953 secs 2023-09-10T08:19:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32415.723953 secs 2023-09-10T08:19:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:19:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:19:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32416.167278 secs 2023-09-10T08:19:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:44 smithi098 ceph-mon[103124]: pgmap v17447: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 428 KiB/s wr, 12 op/s 2023-09-10T08:19:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:19:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:19:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32416.167278 secs 2023-09-10T08:19:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:44 smithi170 ceph-mon[102832]: pgmap v17447: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 428 KiB/s wr, 12 op/s 2023-09-10T08:19:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:19:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:19:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32416.167278 secs 2023-09-10T08:19:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:44 smithi082 ceph-mon[98694]: pgmap v17447: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 428 KiB/s wr, 12 op/s 2023-09-10T08:19:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:19:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:19:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:46 smithi170 ceph-mon[102832]: pgmap v17448: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 635 KiB/s wr, 20 op/s 2023-09-10T08:19:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:19:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:19:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:46 smithi082 ceph-mon[98694]: pgmap v17448: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 635 KiB/s wr, 20 op/s 2023-09-10T08:19:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:19:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:19:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:46 smithi098 ceph-mon[103124]: pgmap v17448: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 635 KiB/s wr, 20 op/s 2023-09-10T08:19:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:48 smithi098 ceph-mon[103124]: pgmap v17449: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 423 KiB/s wr, 14 op/s 2023-09-10T08:19:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32420.724011 secs 2023-09-10T08:19:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:48 smithi170 ceph-mon[102832]: pgmap v17449: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 423 KiB/s wr, 14 op/s 2023-09-10T08:19:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32420.724011 secs 2023-09-10T08:19:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:48 smithi082 ceph-mon[98694]: pgmap v17449: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 423 KiB/s wr, 14 op/s 2023-09-10T08:19:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32420.724011 secs 2023-09-10T08:19:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32421.167343 secs 2023-09-10T08:19:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32421.167343 secs 2023-09-10T08:19:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32421.167343 secs 2023-09-10T08:19:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:50 smithi170 ceph-mon[102832]: pgmap v17450: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 423 KiB/s wr, 14 op/s 2023-09-10T08:19:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:50 smithi082 ceph-mon[98694]: pgmap v17450: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 423 KiB/s wr, 14 op/s 2023-09-10T08:19:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:50 smithi098 ceph-mon[103124]: pgmap v17450: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 423 KiB/s wr, 14 op/s 2023-09-10T08:19:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:52 smithi170 ceph-mon[102832]: pgmap v17451: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 662 KiB/s wr, 21 op/s 2023-09-10T08:19:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:52 smithi082 ceph-mon[98694]: pgmap v17451: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 662 KiB/s wr, 21 op/s 2023-09-10T08:19:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:52 smithi098 ceph-mon[103124]: pgmap v17451: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 662 KiB/s wr, 21 op/s 2023-09-10T08:19:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32425.724089 secs 2023-09-10T08:19:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32425.724089 secs 2023-09-10T08:19:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32425.724089 secs 2023-09-10T08:19:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:19:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:19:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32426.167410 secs 2023-09-10T08:19:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:54 smithi098 ceph-mon[103124]: pgmap v17452: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 446 KiB/s wr, 14 op/s 2023-09-10T08:19:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:19:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:19:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32426.167410 secs 2023-09-10T08:19:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:54 smithi170 ceph-mon[102832]: pgmap v17452: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 446 KiB/s wr, 14 op/s 2023-09-10T08:19:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:19:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:19:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32426.167410 secs 2023-09-10T08:19:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:54 smithi082 ceph-mon[98694]: pgmap v17452: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 446 KiB/s wr, 14 op/s 2023-09-10T08:19:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:19:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:19:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:56 smithi170 ceph-mon[102832]: pgmap v17453: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 693 KiB/s wr, 21 op/s 2023-09-10T08:19:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:19:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:19:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:56 smithi082 ceph-mon[98694]: pgmap v17453: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 693 KiB/s wr, 21 op/s 2023-09-10T08:19:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:19:57.269 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:19:57.269 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:56 smithi098 ceph-mon[103124]: pgmap v17453: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 693 KiB/s wr, 21 op/s 2023-09-10T08:19:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:58 smithi098 ceph-mon[103124]: pgmap v17454: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 487 KiB/s wr, 14 op/s 2023-09-10T08:19:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32430.724146 secs 2023-09-10T08:19:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:58 smithi170 ceph-mon[102832]: pgmap v17454: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 487 KiB/s wr, 14 op/s 2023-09-10T08:19:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32430.724146 secs 2023-09-10T08:19:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:58 smithi082 ceph-mon[98694]: pgmap v17454: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 487 KiB/s wr, 14 op/s 2023-09-10T08:19:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32430.724146 secs 2023-09-10T08:20:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32429.560041 secs 2023-09-10T08:20:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:19:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32431.167476 secs 2023-09-10T08:20:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32429.560041 secs 2023-09-10T08:20:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:19:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32431.167476 secs 2023-09-10T08:20:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32429.560041 secs 2023-09-10T08:20:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:19:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32431.167476 secs 2023-09-10T08:20:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:20:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:20:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:20:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:00 smithi170 ceph-mon[102832]: pgmap v17455: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 487 KiB/s wr, 14 op/s 2023-09-10T08:20:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T08:20:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:20:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:20:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:20:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:00 smithi082 ceph-mon[98694]: pgmap v17455: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 487 KiB/s wr, 14 op/s 2023-09-10T08:20:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T08:20:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:20:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:20:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:20:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:00 smithi098 ceph-mon[103124]: pgmap v17455: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 487 KiB/s wr, 14 op/s 2023-09-10T08:20:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T08:20:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:02 smithi170 ceph-mon[102832]: pgmap v17456: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 781 KiB/s wr, 63 op/s 2023-09-10T08:20:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:02 smithi082 ceph-mon[98694]: pgmap v17456: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 781 KiB/s wr, 63 op/s 2023-09-10T08:20:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:02 smithi098 ceph-mon[103124]: pgmap v17456: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 781 KiB/s wr, 63 op/s 2023-09-10T08:20:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32435.724205 secs 2023-09-10T08:20:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32435.724205 secs 2023-09-10T08:20:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32435.724205 secs 2023-09-10T08:20:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32434.560139 secs 2023-09-10T08:20:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:20:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:20:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32436.167543 secs 2023-09-10T08:20:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:04 smithi170 ceph-mon[102832]: pgmap v17457: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 541 KiB/s wr, 56 op/s 2023-09-10T08:20:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32434.560139 secs 2023-09-10T08:20:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:20:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:20:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32436.167543 secs 2023-09-10T08:20:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:04 smithi082 ceph-mon[98694]: pgmap v17457: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 541 KiB/s wr, 56 op/s 2023-09-10T08:20:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32434.560139 secs 2023-09-10T08:20:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:20:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:20:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32436.167543 secs 2023-09-10T08:20:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:04 smithi098 ceph-mon[103124]: pgmap v17457: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 541 KiB/s wr, 56 op/s 2023-09-10T08:20:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:20:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:20:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:06 smithi170 ceph-mon[102832]: pgmap v17458: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 749 KiB/s wr, 77 op/s 2023-09-10T08:20:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:20:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:20:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:06 smithi082 ceph-mon[98694]: pgmap v17458: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 749 KiB/s wr, 77 op/s 2023-09-10T08:20:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:20:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:20:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:06 smithi098 ceph-mon[103124]: pgmap v17458: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 749 KiB/s wr, 77 op/s 2023-09-10T08:20:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:08 smithi098 ceph-mon[103124]: pgmap v17459: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 502 KiB/s wr, 70 op/s 2023-09-10T08:20:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32440.724265 secs 2023-09-10T08:20:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:08 smithi170 ceph-mon[102832]: pgmap v17459: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 502 KiB/s wr, 70 op/s 2023-09-10T08:20:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32440.724265 secs 2023-09-10T08:20:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:08 smithi082 ceph-mon[98694]: pgmap v17459: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 502 KiB/s wr, 70 op/s 2023-09-10T08:20:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32440.724265 secs 2023-09-10T08:20:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32439.560234 secs 2023-09-10T08:20:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:09 smithi098 ceph-mon[103124]: osdmap e1159: 12 total, 12 up, 12 in 2023-09-10T08:20:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32441.167607 secs 2023-09-10T08:20:10.119 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32439.560234 secs 2023-09-10T08:20:10.120 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:09 smithi170 ceph-mon[102832]: osdmap e1159: 12 total, 12 up, 12 in 2023-09-10T08:20:10.120 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32441.167607 secs 2023-09-10T08:20:10.122 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32439.560234 secs 2023-09-10T08:20:10.122 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:09 smithi082 ceph-mon[98694]: osdmap e1159: 12 total, 12 up, 12 in 2023-09-10T08:20:10.123 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32441.167607 secs 2023-09-10T08:20:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:20:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:10 smithi170 ceph-mon[102832]: pgmap v17461: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 603 KiB/s wr, 84 op/s 2023-09-10T08:20:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:20:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:20:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:20:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:20:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:10 smithi082 ceph-mon[98694]: pgmap v17461: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 603 KiB/s wr, 84 op/s 2023-09-10T08:20:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:20:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:20:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:20:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:20:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:10 smithi098 ceph-mon[103124]: pgmap v17461: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 603 KiB/s wr, 84 op/s 2023-09-10T08:20:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:20:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:20:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:20:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:12 smithi170 ceph-mon[102832]: pgmap v17462: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 359 KiB/s wr, 31 op/s 2023-09-10T08:20:12.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:12 smithi082 ceph-mon[98694]: pgmap v17462: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 359 KiB/s wr, 31 op/s 2023-09-10T08:20:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:12 smithi098 ceph-mon[103124]: pgmap v17462: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 359 KiB/s wr, 31 op/s 2023-09-10T08:20:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:13 smithi098 ceph-mon[103124]: osdmap e1160: 12 total, 12 up, 12 in 2023-09-10T08:20:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32445.724327 secs 2023-09-10T08:20:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:13 smithi170 ceph-mon[102832]: osdmap e1160: 12 total, 12 up, 12 in 2023-09-10T08:20:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32445.724327 secs 2023-09-10T08:20:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:13 smithi082 ceph-mon[98694]: osdmap e1160: 12 total, 12 up, 12 in 2023-09-10T08:20:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32445.724327 secs 2023-09-10T08:20:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32444.560324 secs 2023-09-10T08:20:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32446.167674 secs 2023-09-10T08:20:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:14 smithi170 ceph-mon[102832]: pgmap v17464: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 8 op/s 2023-09-10T08:20:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32444.560324 secs 2023-09-10T08:20:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32446.167674 secs 2023-09-10T08:20:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:14 smithi082 ceph-mon[98694]: pgmap v17464: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 8 op/s 2023-09-10T08:20:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32444.560324 secs 2023-09-10T08:20:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32446.167674 secs 2023-09-10T08:20:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:14 smithi098 ceph-mon[103124]: pgmap v17464: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 8 op/s 2023-09-10T08:20:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:16 smithi170 ceph-mon[102832]: pgmap v17465: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 174 KiB/s wr, 16 op/s 2023-09-10T08:20:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:16 smithi082 ceph-mon[98694]: pgmap v17465: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 174 KiB/s wr, 16 op/s 2023-09-10T08:20:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:16 smithi098 ceph-mon[103124]: pgmap v17465: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 174 KiB/s wr, 16 op/s 2023-09-10T08:20:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:18 smithi098 ceph-mon[103124]: pgmap v17466: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 14 op/s 2023-09-10T08:20:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32450.724384 secs 2023-09-10T08:20:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:18 smithi170 ceph-mon[102832]: pgmap v17466: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 14 op/s 2023-09-10T08:20:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32450.724384 secs 2023-09-10T08:20:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:18 smithi082 ceph-mon[98694]: pgmap v17466: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 14 op/s 2023-09-10T08:20:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32450.724384 secs 2023-09-10T08:20:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32449.560436 secs 2023-09-10T08:20:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32451.167739 secs 2023-09-10T08:20:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32449.560436 secs 2023-09-10T08:20:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32451.167739 secs 2023-09-10T08:20:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32449.560436 secs 2023-09-10T08:20:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32451.167739 secs 2023-09-10T08:20:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:20 smithi170 ceph-mon[102832]: pgmap v17467: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 140 KiB/s wr, 13 op/s 2023-09-10T08:20:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:20 smithi082 ceph-mon[98694]: pgmap v17467: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 140 KiB/s wr, 13 op/s 2023-09-10T08:20:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:20 smithi098 ceph-mon[103124]: pgmap v17467: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 140 KiB/s wr, 13 op/s 2023-09-10T08:20:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:22 smithi170 ceph-mon[102832]: pgmap v17468: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 117 KiB/s wr, 13 op/s 2023-09-10T08:20:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:22 smithi082 ceph-mon[98694]: pgmap v17468: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 117 KiB/s wr, 13 op/s 2023-09-10T08:20:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:22 smithi098 ceph-mon[103124]: pgmap v17468: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 117 KiB/s wr, 13 op/s 2023-09-10T08:20:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32455.724469 secs 2023-09-10T08:20:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32455.724469 secs 2023-09-10T08:20:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32455.724469 secs 2023-09-10T08:20:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32454.560499 secs 2023-09-10T08:20:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:20:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:20:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32456.167808 secs 2023-09-10T08:20:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:24 smithi170 ceph-mon[102832]: pgmap v17469: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T08:20:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32454.560499 secs 2023-09-10T08:20:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:20:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:20:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32456.167808 secs 2023-09-10T08:20:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:24 smithi082 ceph-mon[98694]: pgmap v17469: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T08:20:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32454.560499 secs 2023-09-10T08:20:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:20:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:20:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32456.167808 secs 2023-09-10T08:20:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:24 smithi098 ceph-mon[103124]: pgmap v17469: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 104 KiB/s wr, 12 op/s 2023-09-10T08:20:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:26 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:20:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:20:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:26 smithi170 ceph-mon[102832]: pgmap v17470: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 213 KiB/s wr, 17 op/s 2023-09-10T08:20:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:26 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:20:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:20:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:26 smithi082 ceph-mon[98694]: pgmap v17470: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 213 KiB/s wr, 17 op/s 2023-09-10T08:20:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:26 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:20:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:20:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:26 smithi098 ceph-mon[103124]: pgmap v17470: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 213 KiB/s wr, 17 op/s 2023-09-10T08:20:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:28 smithi170 ceph-mon[102832]: pgmap v17471: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T08:20:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32460.724527 secs 2023-09-10T08:20:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:28 smithi082 ceph-mon[98694]: pgmap v17471: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T08:20:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32460.724527 secs 2023-09-10T08:20:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:28 smithi098 ceph-mon[103124]: pgmap v17471: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T08:20:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32460.724527 secs 2023-09-10T08:20:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32459.560559 secs 2023-09-10T08:20:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32461.167871 secs 2023-09-10T08:20:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32459.560559 secs 2023-09-10T08:20:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32461.167871 secs 2023-09-10T08:20:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32459.560559 secs 2023-09-10T08:20:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32461.167871 secs 2023-09-10T08:20:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:30 smithi170 ceph-mon[102832]: pgmap v17472: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T08:20:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:30 smithi082 ceph-mon[98694]: pgmap v17472: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T08:20:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:30 smithi098 ceph-mon[103124]: pgmap v17472: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T08:20:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:32 smithi170 ceph-mon[102832]: pgmap v17473: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 189 KiB/s wr, 16 op/s 2023-09-10T08:20:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:32 smithi082 ceph-mon[98694]: pgmap v17473: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 189 KiB/s wr, 16 op/s 2023-09-10T08:20:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:32 smithi098 ceph-mon[103124]: pgmap v17473: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 189 KiB/s wr, 16 op/s 2023-09-10T08:20:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32465.724586 secs 2023-09-10T08:20:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32465.724586 secs 2023-09-10T08:20:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32465.724586 secs 2023-09-10T08:20:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32464.560624 secs 2023-09-10T08:20:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32466.167936 secs 2023-09-10T08:20:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:34 smithi170 ceph-mon[102832]: pgmap v17474: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T08:20:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32464.560624 secs 2023-09-10T08:20:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32466.167936 secs 2023-09-10T08:20:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:34 smithi082 ceph-mon[98694]: pgmap v17474: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T08:20:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32464.560624 secs 2023-09-10T08:20:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32466.167936 secs 2023-09-10T08:20:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:34 smithi098 ceph-mon[103124]: pgmap v17474: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T08:20:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:36 smithi170 ceph-mon[102832]: pgmap v17475: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 347 KiB/s wr, 17 op/s 2023-09-10T08:20:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:36 smithi082 ceph-mon[98694]: pgmap v17475: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 347 KiB/s wr, 17 op/s 2023-09-10T08:20:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:36 smithi098 ceph-mon[103124]: pgmap v17475: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 347 KiB/s wr, 17 op/s 2023-09-10T08:20:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:38 smithi170 ceph-mon[102832]: pgmap v17476: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T08:20:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32470.724644 secs 2023-09-10T08:20:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:38 smithi082 ceph-mon[98694]: pgmap v17476: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T08:20:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32470.724644 secs 2023-09-10T08:20:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:38 smithi098 ceph-mon[103124]: pgmap v17476: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T08:20:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32470.724644 secs 2023-09-10T08:20:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:39 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32469.560685 secs 2023-09-10T08:20:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32471.168001 secs 2023-09-10T08:20:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:39 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32469.560685 secs 2023-09-10T08:20:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32471.168001 secs 2023-09-10T08:20:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:39 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32469.560685 secs 2023-09-10T08:20:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32471.168001 secs 2023-09-10T08:20:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:40 smithi170 ceph-mon[102832]: pgmap v17477: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T08:20:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:40 smithi082 ceph-mon[98694]: pgmap v17477: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T08:20:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:40 smithi098 ceph-mon[103124]: pgmap v17477: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 231 KiB/s wr, 11 op/s 2023-09-10T08:20:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:42 smithi170 ceph-mon[102832]: pgmap v17478: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 450 KiB/s wr, 18 op/s 2023-09-10T08:20:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:42 smithi082 ceph-mon[98694]: pgmap v17478: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 450 KiB/s wr, 18 op/s 2023-09-10T08:20:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:42 smithi098 ceph-mon[103124]: pgmap v17478: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 450 KiB/s wr, 18 op/s 2023-09-10T08:20:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:20:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32475.724704 secs 2023-09-10T08:20:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:20:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32475.724704 secs 2023-09-10T08:20:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:20:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32475.724704 secs 2023-09-10T08:20:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:44 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:20:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:20:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:20:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32476.168069 secs 2023-09-10T08:20:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:44 smithi170 ceph-mon[102832]: pgmap v17479: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 450 KiB/s wr, 12 op/s 2023-09-10T08:20:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:44 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:20:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:20:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:20:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32476.168069 secs 2023-09-10T08:20:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:44 smithi082 ceph-mon[98694]: pgmap v17479: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 450 KiB/s wr, 12 op/s 2023-09-10T08:20:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:44 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:20:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:20:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:20:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32476.168069 secs 2023-09-10T08:20:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:44 smithi098 ceph-mon[103124]: pgmap v17479: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 450 KiB/s wr, 12 op/s 2023-09-10T08:20:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:20:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:20:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:46 smithi170 ceph-mon[102832]: pgmap v17480: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 657 KiB/s wr, 20 op/s 2023-09-10T08:20:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:20:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:20:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:46 smithi082 ceph-mon[98694]: pgmap v17480: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 657 KiB/s wr, 20 op/s 2023-09-10T08:20:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:20:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:20:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:46 smithi098 ceph-mon[103124]: pgmap v17480: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 657 KiB/s wr, 20 op/s 2023-09-10T08:20:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:48 smithi170 ceph-mon[102832]: pgmap v17481: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 426 KiB/s wr, 13 op/s 2023-09-10T08:20:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32480.724763 secs 2023-09-10T08:20:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:48 smithi082 ceph-mon[98694]: pgmap v17481: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 426 KiB/s wr, 13 op/s 2023-09-10T08:20:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32480.724763 secs 2023-09-10T08:20:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:48 smithi098 ceph-mon[103124]: pgmap v17481: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 426 KiB/s wr, 13 op/s 2023-09-10T08:20:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32480.724763 secs 2023-09-10T08:20:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32481.168135 secs 2023-09-10T08:20:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32481.168135 secs 2023-09-10T08:20:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32481.168135 secs 2023-09-10T08:20:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:50 smithi170 ceph-mon[102832]: pgmap v17482: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 426 KiB/s wr, 13 op/s 2023-09-10T08:20:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:50 smithi082 ceph-mon[98694]: pgmap v17482: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 426 KiB/s wr, 13 op/s 2023-09-10T08:20:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:50 smithi098 ceph-mon[103124]: pgmap v17482: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 426 KiB/s wr, 13 op/s 2023-09-10T08:20:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:52 smithi170 ceph-mon[102832]: pgmap v17483: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 805 KiB/s wr, 21 op/s 2023-09-10T08:20:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:52 smithi082 ceph-mon[98694]: pgmap v17483: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 805 KiB/s wr, 21 op/s 2023-09-10T08:20:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:52 smithi098 ceph-mon[103124]: pgmap v17483: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 805 KiB/s wr, 21 op/s 2023-09-10T08:20:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32485.724821 secs 2023-09-10T08:20:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32485.724821 secs 2023-09-10T08:20:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32485.724821 secs 2023-09-10T08:20:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32484.560876 secs 2023-09-10T08:20:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:20:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:20:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32486.168203 secs 2023-09-10T08:20:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:54 smithi170 ceph-mon[102832]: pgmap v17484: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 586 KiB/s wr, 14 op/s 2023-09-10T08:20:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32484.560876 secs 2023-09-10T08:20:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:20:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:20:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32486.168203 secs 2023-09-10T08:20:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:54 smithi082 ceph-mon[98694]: pgmap v17484: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 586 KiB/s wr, 14 op/s 2023-09-10T08:20:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32484.560876 secs 2023-09-10T08:20:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:20:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:20:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32486.168203 secs 2023-09-10T08:20:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:54 smithi098 ceph-mon[103124]: pgmap v17484: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 586 KiB/s wr, 14 op/s 2023-09-10T08:20:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:20:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:20:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:20:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:20:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:20:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:20:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:20:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:20:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:20:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:56 smithi170 ceph-mon[102832]: pgmap v17485: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 806 KiB/s wr, 20 op/s 2023-09-10T08:20:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:56 smithi082 ceph-mon[98694]: pgmap v17485: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 806 KiB/s wr, 20 op/s 2023-09-10T08:20:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:56 smithi098 ceph-mon[103124]: pgmap v17485: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 806 KiB/s wr, 20 op/s 2023-09-10T08:20:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:58 smithi170 ceph-mon[102832]: pgmap v17486: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 599 KiB/s wr, 13 op/s 2023-09-10T08:20:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32490.724883 secs 2023-09-10T08:20:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:58 smithi082 ceph-mon[98694]: pgmap v17486: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 599 KiB/s wr, 13 op/s 2023-09-10T08:20:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32490.724883 secs 2023-09-10T08:20:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:58 smithi098 ceph-mon[103124]: pgmap v17486: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 599 KiB/s wr, 13 op/s 2023-09-10T08:20:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32490.724883 secs 2023-09-10T08:21:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32489.560972 secs 2023-09-10T08:21:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:20:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32491.168269 secs 2023-09-10T08:21:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32489.560972 secs 2023-09-10T08:21:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:20:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32491.168269 secs 2023-09-10T08:21:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32489.560972 secs 2023-09-10T08:21:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:20:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32491.168269 secs 2023-09-10T08:21:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:00 smithi170 ceph-mon[102832]: pgmap v17487: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 599 KiB/s wr, 13 op/s 2023-09-10T08:21:01.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:00 smithi082 ceph-mon[98694]: pgmap v17487: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 599 KiB/s wr, 13 op/s 2023-09-10T08:21:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:00 smithi098 ceph-mon[103124]: pgmap v17487: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 599 KiB/s wr, 13 op/s 2023-09-10T08:21:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:02 smithi170 ceph-mon[102832]: pgmap v17488: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 873 KiB/s wr, 20 op/s 2023-09-10T08:21:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:02 smithi082 ceph-mon[98694]: pgmap v17488: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 873 KiB/s wr, 20 op/s 2023-09-10T08:21:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:02 smithi098 ceph-mon[103124]: pgmap v17488: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 873 KiB/s wr, 20 op/s 2023-09-10T08:21:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32495.724943 secs 2023-09-10T08:21:04.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32495.724943 secs 2023-09-10T08:21:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32495.724943 secs 2023-09-10T08:21:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32494.561082 secs 2023-09-10T08:21:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:21:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:21:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32496.168336 secs 2023-09-10T08:21:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32494.561082 secs 2023-09-10T08:21:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:21:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:21:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32496.168336 secs 2023-09-10T08:21:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32494.561082 secs 2023-09-10T08:21:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:21:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:21:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32496.168336 secs 2023-09-10T08:21:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:05 smithi170 ceph-mon[102832]: pgmap v17489: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 494 KiB/s wr, 12 op/s 2023-09-10T08:21:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:05 smithi082 ceph-mon[98694]: pgmap v17489: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 494 KiB/s wr, 12 op/s 2023-09-10T08:21:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:05 smithi098 ceph-mon[103124]: pgmap v17489: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 494 KiB/s wr, 12 op/s 2023-09-10T08:21:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:21:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:21:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:21:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:21:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:21:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:21:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:07 smithi170 ceph-mon[102832]: pgmap v17490: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 540 KiB/s wr, 32 op/s 2023-09-10T08:21:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:07 smithi082 ceph-mon[98694]: pgmap v17490: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 540 KiB/s wr, 32 op/s 2023-09-10T08:21:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:07 smithi098 ceph-mon[103124]: pgmap v17490: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 540 KiB/s wr, 32 op/s 2023-09-10T08:21:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32499.561142 secs 2023-09-10T08:21:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32500.725027 secs 2023-09-10T08:21:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32499.561142 secs 2023-09-10T08:21:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32500.725027 secs 2023-09-10T08:21:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32499.561142 secs 2023-09-10T08:21:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32500.725027 secs 2023-09-10T08:21:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:09 smithi170 ceph-mon[102832]: pgmap v17491: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 319 KiB/s wr, 26 op/s 2023-09-10T08:21:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:09 smithi170 ceph-mon[102832]: osdmap e1161: 12 total, 12 up, 12 in 2023-09-10T08:21:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32501.168405 secs 2023-09-10T08:21:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:09 smithi082 ceph-mon[98694]: pgmap v17491: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 319 KiB/s wr, 26 op/s 2023-09-10T08:21:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:09 smithi082 ceph-mon[98694]: osdmap e1161: 12 total, 12 up, 12 in 2023-09-10T08:21:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32501.168405 secs 2023-09-10T08:21:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:09 smithi098 ceph-mon[103124]: pgmap v17491: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 319 KiB/s wr, 26 op/s 2023-09-10T08:21:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:09 smithi098 ceph-mon[103124]: osdmap e1161: 12 total, 12 up, 12 in 2023-09-10T08:21:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32501.168405 secs 2023-09-10T08:21:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:21:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:21:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:21:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:11 smithi170 ceph-mon[102832]: pgmap v17493: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 383 KiB/s wr, 31 op/s 2023-09-10T08:21:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:21:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:21:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:21:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:11 smithi082 ceph-mon[98694]: pgmap v17493: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 383 KiB/s wr, 31 op/s 2023-09-10T08:21:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:21:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:21:12.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:21:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:11 smithi098 ceph-mon[103124]: pgmap v17493: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 383 KiB/s wr, 31 op/s 2023-09-10T08:21:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:21:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:21:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:21:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:13 smithi170 ceph-mon[102832]: pgmap v17494: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 170 KiB/s wr, 30 op/s 2023-09-10T08:21:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32504.561201 secs 2023-09-10T08:21:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32505.725085 secs 2023-09-10T08:21:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:13 smithi082 ceph-mon[98694]: pgmap v17494: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 170 KiB/s wr, 30 op/s 2023-09-10T08:21:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32504.561201 secs 2023-09-10T08:21:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32505.725085 secs 2023-09-10T08:21:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:13 smithi098 ceph-mon[103124]: pgmap v17494: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 170 KiB/s wr, 30 op/s 2023-09-10T08:21:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32504.561201 secs 2023-09-10T08:21:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32505.725085 secs 2023-09-10T08:21:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32506.168475 secs 2023-09-10T08:21:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32506.168475 secs 2023-09-10T08:21:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32506.168475 secs 2023-09-10T08:21:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:15 smithi170 ceph-mon[102832]: pgmap v17495: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 170 KiB/s wr, 30 op/s 2023-09-10T08:21:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:15 smithi082 ceph-mon[98694]: pgmap v17495: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 170 KiB/s wr, 30 op/s 2023-09-10T08:21:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:15 smithi098 ceph-mon[103124]: pgmap v17495: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 170 KiB/s wr, 30 op/s 2023-09-10T08:21:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:17 smithi170 ceph-mon[102832]: pgmap v17496: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 13 op/s 2023-09-10T08:21:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:17 smithi170 ceph-mon[102832]: osdmap e1162: 12 total, 12 up, 12 in 2023-09-10T08:21:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:17 smithi082 ceph-mon[98694]: pgmap v17496: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 13 op/s 2023-09-10T08:21:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:17 smithi082 ceph-mon[98694]: osdmap e1162: 12 total, 12 up, 12 in 2023-09-10T08:21:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:17 smithi098 ceph-mon[103124]: pgmap v17496: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 13 op/s 2023-09-10T08:21:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:17 smithi098 ceph-mon[103124]: osdmap e1162: 12 total, 12 up, 12 in 2023-09-10T08:21:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32509.561264 secs 2023-09-10T08:21:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32510.725145 secs 2023-09-10T08:21:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32509.561264 secs 2023-09-10T08:21:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32510.725145 secs 2023-09-10T08:21:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32509.561264 secs 2023-09-10T08:21:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32510.725145 secs 2023-09-10T08:21:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:19 smithi170 ceph-mon[102832]: pgmap v17498: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 14 op/s 2023-09-10T08:21:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32511.168542 secs 2023-09-10T08:21:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:19 smithi082 ceph-mon[98694]: pgmap v17498: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 14 op/s 2023-09-10T08:21:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32511.168542 secs 2023-09-10T08:21:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:19 smithi098 ceph-mon[103124]: pgmap v17498: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 14 op/s 2023-09-10T08:21:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32511.168542 secs 2023-09-10T08:21:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:21 smithi170 ceph-mon[102832]: pgmap v17499: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 13 op/s 2023-09-10T08:21:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:21 smithi082 ceph-mon[98694]: pgmap v17499: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 13 op/s 2023-09-10T08:21:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:21 smithi098 ceph-mon[103124]: pgmap v17499: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 13 op/s 2023-09-10T08:21:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:23 smithi170 ceph-mon[102832]: pgmap v17500: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T08:21:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32514.561323 secs 2023-09-10T08:21:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32515.725230 secs 2023-09-10T08:21:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:23 smithi082 ceph-mon[98694]: pgmap v17500: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T08:21:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32514.561323 secs 2023-09-10T08:21:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32515.725230 secs 2023-09-10T08:21:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:23 smithi098 ceph-mon[103124]: pgmap v17500: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T08:21:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32514.561323 secs 2023-09-10T08:21:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32515.725230 secs 2023-09-10T08:21:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32516.168608 secs 2023-09-10T08:21:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32516.168608 secs 2023-09-10T08:21:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32516.168608 secs 2023-09-10T08:21:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:25 smithi170 ceph-mon[102832]: pgmap v17501: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T08:21:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:25 smithi082 ceph-mon[98694]: pgmap v17501: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T08:21:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:25 smithi098 ceph-mon[103124]: pgmap v17501: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T08:21:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:27 smithi170 ceph-mon[102832]: pgmap v17502: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 13 op/s 2023-09-10T08:21:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:27 smithi082 ceph-mon[98694]: pgmap v17502: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 13 op/s 2023-09-10T08:21:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:27 smithi098 ceph-mon[103124]: pgmap v17502: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 13 op/s 2023-09-10T08:21:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32519.561407 secs 2023-09-10T08:21:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32520.725302 secs 2023-09-10T08:21:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32519.561407 secs 2023-09-10T08:21:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32520.725302 secs 2023-09-10T08:21:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32519.561407 secs 2023-09-10T08:21:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32520.725302 secs 2023-09-10T08:21:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:29 smithi170 ceph-mon[102832]: pgmap v17503: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 136 KiB/s wr, 12 op/s 2023-09-10T08:21:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32521.168676 secs 2023-09-10T08:21:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:29 smithi082 ceph-mon[98694]: pgmap v17503: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 136 KiB/s wr, 12 op/s 2023-09-10T08:21:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32521.168676 secs 2023-09-10T08:21:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:29 smithi098 ceph-mon[103124]: pgmap v17503: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 136 KiB/s wr, 12 op/s 2023-09-10T08:21:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32521.168676 secs 2023-09-10T08:21:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:31 smithi170 ceph-mon[102832]: pgmap v17504: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 10 op/s 2023-09-10T08:21:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:31 smithi082 ceph-mon[98694]: pgmap v17504: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 10 op/s 2023-09-10T08:21:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:31 smithi098 ceph-mon[103124]: pgmap v17504: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 10 op/s 2023-09-10T08:21:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:33 smithi170 ceph-mon[102832]: pgmap v17505: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 116 KiB/s wr, 16 op/s 2023-09-10T08:21:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32524.561467 secs 2023-09-10T08:21:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32525.725365 secs 2023-09-10T08:21:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:21:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:21:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:33 smithi082 ceph-mon[98694]: pgmap v17505: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 116 KiB/s wr, 16 op/s 2023-09-10T08:21:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32524.561467 secs 2023-09-10T08:21:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32525.725365 secs 2023-09-10T08:21:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:21:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:21:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:33 smithi098 ceph-mon[103124]: pgmap v17505: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 116 KiB/s wr, 16 op/s 2023-09-10T08:21:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32524.561467 secs 2023-09-10T08:21:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32525.725365 secs 2023-09-10T08:21:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:21:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:21:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32526.168741 secs 2023-09-10T08:21:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32526.168741 secs 2023-09-10T08:21:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32526.168741 secs 2023-09-10T08:21:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:35 smithi170 ceph-mon[102832]: pgmap v17506: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T08:21:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:35 smithi082 ceph-mon[98694]: pgmap v17506: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T08:21:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:35 smithi098 ceph-mon[103124]: pgmap v17506: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 11 op/s 2023-09-10T08:21:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:21:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:21:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:21:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:21:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:21:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:21:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:37 smithi170 ceph-mon[102832]: pgmap v17507: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 280 KiB/s wr, 17 op/s 2023-09-10T08:21:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:37 smithi082 ceph-mon[98694]: pgmap v17507: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 280 KiB/s wr, 17 op/s 2023-09-10T08:21:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:37 smithi098 ceph-mon[103124]: pgmap v17507: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 280 KiB/s wr, 17 op/s 2023-09-10T08:21:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32529.561525 secs 2023-09-10T08:21:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32530.725422 secs 2023-09-10T08:21:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32529.561525 secs 2023-09-10T08:21:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32530.725422 secs 2023-09-10T08:21:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32529.561525 secs 2023-09-10T08:21:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32530.725422 secs 2023-09-10T08:21:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:39 smithi170 ceph-mon[102832]: pgmap v17508: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 11 op/s 2023-09-10T08:21:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32531.168806 secs 2023-09-10T08:21:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:39 smithi082 ceph-mon[98694]: pgmap v17508: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 11 op/s 2023-09-10T08:21:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32531.168806 secs 2023-09-10T08:21:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:39 smithi098 ceph-mon[103124]: pgmap v17508: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 11 op/s 2023-09-10T08:21:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32531.168806 secs 2023-09-10T08:21:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:41 smithi170 ceph-mon[102832]: pgmap v17509: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 11 op/s 2023-09-10T08:21:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:41 smithi082 ceph-mon[98694]: pgmap v17509: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 11 op/s 2023-09-10T08:21:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:41 smithi098 ceph-mon[103124]: pgmap v17509: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 164 KiB/s wr, 11 op/s 2023-09-10T08:21:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:43 smithi170 ceph-mon[102832]: pgmap v17510: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 431 KiB/s wr, 18 op/s 2023-09-10T08:21:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:21:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32535.725497 secs 2023-09-10T08:21:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:43 smithi082 ceph-mon[98694]: pgmap v17510: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 431 KiB/s wr, 18 op/s 2023-09-10T08:21:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:21:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32535.725497 secs 2023-09-10T08:21:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:43 smithi098 ceph-mon[103124]: pgmap v17510: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 431 KiB/s wr, 18 op/s 2023-09-10T08:21:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:21:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32535.725497 secs 2023-09-10T08:21:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:44 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:21:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:21:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:21:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32536.168869 secs 2023-09-10T08:21:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:44 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:21:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:21:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:21:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32536.168869 secs 2023-09-10T08:21:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:44 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:21:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:21:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:21:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32536.168869 secs 2023-09-10T08:21:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:45 smithi170 ceph-mon[102832]: pgmap v17511: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 431 KiB/s wr, 13 op/s 2023-09-10T08:21:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:21:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:21:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:45 smithi082 ceph-mon[98694]: pgmap v17511: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 431 KiB/s wr, 13 op/s 2023-09-10T08:21:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:21:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:21:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:45 smithi098 ceph-mon[103124]: pgmap v17511: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 431 KiB/s wr, 13 op/s 2023-09-10T08:21:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:21:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:21:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:47 smithi170 ceph-mon[102832]: pgmap v17512: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 734 KiB/s wr, 20 op/s 2023-09-10T08:21:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:47 smithi082 ceph-mon[98694]: pgmap v17512: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 734 KiB/s wr, 20 op/s 2023-09-10T08:21:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:47 smithi098 ceph-mon[103124]: pgmap v17512: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 734 KiB/s wr, 20 op/s 2023-09-10T08:21:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32540.725553 secs 2023-09-10T08:21:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32540.725553 secs 2023-09-10T08:21:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32540.725553 secs 2023-09-10T08:21:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:49 smithi170 ceph-mon[102832]: pgmap v17513: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 570 KiB/s wr, 14 op/s 2023-09-10T08:21:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32541.168933 secs 2023-09-10T08:21:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:49 smithi082 ceph-mon[98694]: pgmap v17513: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 570 KiB/s wr, 14 op/s 2023-09-10T08:21:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32541.168933 secs 2023-09-10T08:21:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:49 smithi098 ceph-mon[103124]: pgmap v17513: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 570 KiB/s wr, 14 op/s 2023-09-10T08:21:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32541.168933 secs 2023-09-10T08:21:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:51 smithi170 ceph-mon[102832]: pgmap v17514: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 570 KiB/s wr, 14 op/s 2023-09-10T08:21:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:51 smithi082 ceph-mon[98694]: pgmap v17514: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 570 KiB/s wr, 14 op/s 2023-09-10T08:21:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:51 smithi098 ceph-mon[103124]: pgmap v17514: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 570 KiB/s wr, 14 op/s 2023-09-10T08:21:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:53 smithi170 ceph-mon[102832]: pgmap v17515: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 715 KiB/s wr, 21 op/s 2023-09-10T08:21:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32545.725613 secs 2023-09-10T08:21:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:21:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:21:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:53 smithi082 ceph-mon[98694]: pgmap v17515: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 715 KiB/s wr, 21 op/s 2023-09-10T08:21:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32545.725613 secs 2023-09-10T08:21:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:21:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:21:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:53 smithi098 ceph-mon[103124]: pgmap v17515: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 715 KiB/s wr, 21 op/s 2023-09-10T08:21:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32545.725613 secs 2023-09-10T08:21:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:21:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:21:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32546.168999 secs 2023-09-10T08:21:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32546.168999 secs 2023-09-10T08:21:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32546.168999 secs 2023-09-10T08:21:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:55 smithi170 ceph-mon[102832]: pgmap v17516: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 449 KiB/s wr, 14 op/s 2023-09-10T08:21:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:55 smithi082 ceph-mon[98694]: pgmap v17516: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 449 KiB/s wr, 14 op/s 2023-09-10T08:21:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:55 smithi098 ceph-mon[103124]: pgmap v17516: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 449 KiB/s wr, 14 op/s 2023-09-10T08:21:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:21:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:21:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:21:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:21:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:21:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:21:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:57 smithi170 ceph-mon[102832]: pgmap v17517: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 697 KiB/s wr, 21 op/s 2023-09-10T08:21:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:21:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:21:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:57 smithi082 ceph-mon[98694]: pgmap v17517: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 697 KiB/s wr, 21 op/s 2023-09-10T08:21:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:21:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:21:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:57 smithi098 ceph-mon[103124]: pgmap v17517: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 697 KiB/s wr, 21 op/s 2023-09-10T08:21:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:21:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:21:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32549.561773 secs 2023-09-10T08:21:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32550.725698 secs 2023-09-10T08:21:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32549.561773 secs 2023-09-10T08:21:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32550.725698 secs 2023-09-10T08:21:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32549.561773 secs 2023-09-10T08:21:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32550.725698 secs 2023-09-10T08:22:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:59 smithi082 ceph-mon[98694]: pgmap v17518: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 393 KiB/s wr, 14 op/s 2023-09-10T08:22:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:21:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32551.169067 secs 2023-09-10T08:22:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:59 smithi098 ceph-mon[103124]: pgmap v17518: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 393 KiB/s wr, 14 op/s 2023-09-10T08:22:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:21:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32551.169067 secs 2023-09-10T08:22:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:59 smithi170 ceph-mon[102832]: pgmap v17518: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 393 KiB/s wr, 14 op/s 2023-09-10T08:22:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:21:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32551.169067 secs 2023-09-10T08:22:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:22:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:22:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:22:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:22:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:22:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:22:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:01 smithi082 ceph-mon[98694]: pgmap v17519: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 393 KiB/s wr, 14 op/s 2023-09-10T08:22:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:01 smithi098 ceph-mon[103124]: pgmap v17519: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 393 KiB/s wr, 14 op/s 2023-09-10T08:22:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:01 smithi170 ceph-mon[102832]: pgmap v17519: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 393 KiB/s wr, 14 op/s 2023-09-10T08:22:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:03 smithi082 ceph-mon[98694]: pgmap v17520: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 668 KiB/s wr, 20 op/s 2023-09-10T08:22:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32554.561832 secs 2023-09-10T08:22:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32555.725753 secs 2023-09-10T08:22:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:03 smithi098 ceph-mon[103124]: pgmap v17520: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 668 KiB/s wr, 20 op/s 2023-09-10T08:22:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32554.561832 secs 2023-09-10T08:22:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32555.725753 secs 2023-09-10T08:22:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:03 smithi170 ceph-mon[102832]: pgmap v17520: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 668 KiB/s wr, 20 op/s 2023-09-10T08:22:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32554.561832 secs 2023-09-10T08:22:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32555.725753 secs 2023-09-10T08:22:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32556.169131 secs 2023-09-10T08:22:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:22:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32556.169131 secs 2023-09-10T08:22:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:22:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32556.169131 secs 2023-09-10T08:22:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:22:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:05 smithi082 ceph-mon[98694]: pgmap v17521: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 523 KiB/s wr, 13 op/s 2023-09-10T08:22:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:22:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:05 smithi098 ceph-mon[103124]: pgmap v17521: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 523 KiB/s wr, 13 op/s 2023-09-10T08:22:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:22:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:05 smithi170 ceph-mon[102832]: pgmap v17521: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 523 KiB/s wr, 13 op/s 2023-09-10T08:22:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:22:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:07 smithi098 ceph-mon[103124]: pgmap v17522: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 708 KiB/s wr, 34 op/s 2023-09-10T08:22:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:07 smithi170 ceph-mon[102832]: pgmap v17522: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 708 KiB/s wr, 34 op/s 2023-09-10T08:22:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:07 smithi082 ceph-mon[98694]: pgmap v17522: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 708 KiB/s wr, 34 op/s 2023-09-10T08:22:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32559.561906 secs 2023-09-10T08:22:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32560.725809 secs 2023-09-10T08:22:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:08 smithi098 ceph-mon[103124]: osdmap e1163: 12 total, 12 up, 12 in 2023-09-10T08:22:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32559.561906 secs 2023-09-10T08:22:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32560.725809 secs 2023-09-10T08:22:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:08 smithi170 ceph-mon[102832]: osdmap e1163: 12 total, 12 up, 12 in 2023-09-10T08:22:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32559.561906 secs 2023-09-10T08:22:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32560.725809 secs 2023-09-10T08:22:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:08 smithi082 ceph-mon[98694]: osdmap e1163: 12 total, 12 up, 12 in 2023-09-10T08:22:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:09 smithi098 ceph-mon[103124]: pgmap v17523: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 460 KiB/s wr, 26 op/s 2023-09-10T08:22:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32561.169198 secs 2023-09-10T08:22:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:09 smithi170 ceph-mon[102832]: pgmap v17523: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 460 KiB/s wr, 26 op/s 2023-09-10T08:22:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32561.169198 secs 2023-09-10T08:22:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:09 smithi082 ceph-mon[98694]: pgmap v17523: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 460 KiB/s wr, 26 op/s 2023-09-10T08:22:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32561.169198 secs 2023-09-10T08:22:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:11 smithi098 ceph-mon[103124]: pgmap v17525: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 552 KiB/s wr, 31 op/s 2023-09-10T08:22:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:22:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:22:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:22:12.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:22:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:11 smithi170 ceph-mon[102832]: pgmap v17525: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 552 KiB/s wr, 31 op/s 2023-09-10T08:22:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:22:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:22:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:22:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:22:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:11 smithi082 ceph-mon[98694]: pgmap v17525: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 552 KiB/s wr, 31 op/s 2023-09-10T08:22:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:22:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:22:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:22:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:22:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:13 smithi098 ceph-mon[103124]: pgmap v17526: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 360 KiB/s wr, 32 op/s 2023-09-10T08:22:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32564.562044 secs 2023-09-10T08:22:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32565.725867 secs 2023-09-10T08:22:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:13 smithi170 ceph-mon[102832]: pgmap v17526: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 360 KiB/s wr, 32 op/s 2023-09-10T08:22:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32564.562044 secs 2023-09-10T08:22:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32565.725867 secs 2023-09-10T08:22:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:13 smithi082 ceph-mon[98694]: pgmap v17526: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 360 KiB/s wr, 32 op/s 2023-09-10T08:22:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32564.562044 secs 2023-09-10T08:22:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32565.725867 secs 2023-09-10T08:22:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32566.169288 secs 2023-09-10T08:22:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32566.169288 secs 2023-09-10T08:22:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32566.169288 secs 2023-09-10T08:22:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:15 smithi098 ceph-mon[103124]: pgmap v17527: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 361 KiB/s wr, 32 op/s 2023-09-10T08:22:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:15 smithi170 ceph-mon[102832]: pgmap v17527: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 361 KiB/s wr, 32 op/s 2023-09-10T08:22:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:15 smithi082 ceph-mon[98694]: pgmap v17527: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 361 KiB/s wr, 32 op/s 2023-09-10T08:22:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:17 smithi098 ceph-mon[103124]: pgmap v17528: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 14 op/s 2023-09-10T08:22:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:17 smithi098 ceph-mon[103124]: osdmap e1164: 12 total, 12 up, 12 in 2023-09-10T08:22:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:17 smithi170 ceph-mon[102832]: pgmap v17528: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 14 op/s 2023-09-10T08:22:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:17 smithi170 ceph-mon[102832]: osdmap e1164: 12 total, 12 up, 12 in 2023-09-10T08:22:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:17 smithi082 ceph-mon[98694]: pgmap v17528: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 14 op/s 2023-09-10T08:22:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:17 smithi082 ceph-mon[98694]: osdmap e1164: 12 total, 12 up, 12 in 2023-09-10T08:22:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32569.562105 secs 2023-09-10T08:22:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32570.725957 secs 2023-09-10T08:22:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32569.562105 secs 2023-09-10T08:22:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32570.725957 secs 2023-09-10T08:22:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32569.562105 secs 2023-09-10T08:22:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32570.725957 secs 2023-09-10T08:22:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:19 smithi098 ceph-mon[103124]: pgmap v17530: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T08:22:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32571.169353 secs 2023-09-10T08:22:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:19 smithi170 ceph-mon[102832]: pgmap v17530: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T08:22:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32571.169353 secs 2023-09-10T08:22:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:19 smithi082 ceph-mon[98694]: pgmap v17530: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T08:22:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32571.169353 secs 2023-09-10T08:22:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:20 smithi098 ceph-mon[103124]: pgmap v17531: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 14 op/s 2023-09-10T08:22:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:20 smithi170 ceph-mon[102832]: pgmap v17531: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 14 op/s 2023-09-10T08:22:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:20 smithi082 ceph-mon[98694]: pgmap v17531: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 14 op/s 2023-09-10T08:22:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:23 smithi170 ceph-mon[102832]: pgmap v17532: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 13 op/s 2023-09-10T08:22:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32574.562165 secs 2023-09-10T08:22:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32575.726046 secs 2023-09-10T08:22:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:23 smithi082 ceph-mon[98694]: pgmap v17532: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 13 op/s 2023-09-10T08:22:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32574.562165 secs 2023-09-10T08:22:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32575.726046 secs 2023-09-10T08:22:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:23 smithi098 ceph-mon[103124]: pgmap v17532: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 13 op/s 2023-09-10T08:22:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32574.562165 secs 2023-09-10T08:22:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32575.726046 secs 2023-09-10T08:22:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32576.169421 secs 2023-09-10T08:22:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32576.169421 secs 2023-09-10T08:22:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32576.169421 secs 2023-09-10T08:22:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:25 smithi170 ceph-mon[102832]: pgmap v17533: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 13 op/s 2023-09-10T08:22:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:25 smithi082 ceph-mon[98694]: pgmap v17533: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 13 op/s 2023-09-10T08:22:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:25 smithi098 ceph-mon[103124]: pgmap v17533: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 88 KiB/s wr, 13 op/s 2023-09-10T08:22:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:27 smithi170 ceph-mon[102832]: pgmap v17534: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 13 op/s 2023-09-10T08:22:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:27 smithi082 ceph-mon[98694]: pgmap v17534: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 13 op/s 2023-09-10T08:22:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:27 smithi098 ceph-mon[103124]: pgmap v17534: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 13 op/s 2023-09-10T08:22:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32579.562253 secs 2023-09-10T08:22:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32580.726104 secs 2023-09-10T08:22:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32579.562253 secs 2023-09-10T08:22:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32580.726104 secs 2023-09-10T08:22:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32579.562253 secs 2023-09-10T08:22:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32580.726104 secs 2023-09-10T08:22:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:29 smithi170 ceph-mon[102832]: pgmap v17535: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 222 KiB/s wr, 13 op/s 2023-09-10T08:22:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32581.169490 secs 2023-09-10T08:22:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:29 smithi082 ceph-mon[98694]: pgmap v17535: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 222 KiB/s wr, 13 op/s 2023-09-10T08:22:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32581.169490 secs 2023-09-10T08:22:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:29 smithi098 ceph-mon[103124]: pgmap v17535: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 222 KiB/s wr, 13 op/s 2023-09-10T08:22:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32581.169490 secs 2023-09-10T08:22:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:31 smithi170 ceph-mon[102832]: pgmap v17536: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T08:22:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:31 smithi082 ceph-mon[98694]: pgmap v17536: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T08:22:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:31 smithi098 ceph-mon[103124]: pgmap v17536: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 189 KiB/s wr, 11 op/s 2023-09-10T08:22:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:33 smithi170 ceph-mon[102832]: pgmap v17537: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 189 KiB/s wr, 16 op/s 2023-09-10T08:22:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32585.726166 secs 2023-09-10T08:22:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:22:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:33 smithi082 ceph-mon[98694]: pgmap v17537: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 189 KiB/s wr, 16 op/s 2023-09-10T08:22:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32585.726166 secs 2023-09-10T08:22:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:22:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:33 smithi098 ceph-mon[103124]: pgmap v17537: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 189 KiB/s wr, 16 op/s 2023-09-10T08:22:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32585.726166 secs 2023-09-10T08:22:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:22:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32586.169557 secs 2023-09-10T08:22:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32586.169557 secs 2023-09-10T08:22:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32586.169557 secs 2023-09-10T08:22:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:35 smithi170 ceph-mon[102832]: pgmap v17538: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T08:22:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:22:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:35 smithi082 ceph-mon[98694]: pgmap v17538: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T08:22:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:22:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:35 smithi098 ceph-mon[103124]: pgmap v17538: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T08:22:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:22:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:22:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:22:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:22:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:22:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:22:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:22:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:37 smithi170 ceph-mon[102832]: pgmap v17539: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 303 KiB/s wr, 17 op/s 2023-09-10T08:22:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:37 smithi082 ceph-mon[98694]: pgmap v17539: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 303 KiB/s wr, 17 op/s 2023-09-10T08:22:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:37 smithi098 ceph-mon[103124]: pgmap v17539: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 303 KiB/s wr, 17 op/s 2023-09-10T08:22:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32589.562407 secs 2023-09-10T08:22:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32590.726251 secs 2023-09-10T08:22:39.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32589.562407 secs 2023-09-10T08:22:39.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32590.726251 secs 2023-09-10T08:22:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32589.562407 secs 2023-09-10T08:22:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32590.726251 secs 2023-09-10T08:22:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:39 smithi170 ceph-mon[102832]: pgmap v17540: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T08:22:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32591.169622 secs 2023-09-10T08:22:40.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:39 smithi082 ceph-mon[98694]: pgmap v17540: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T08:22:40.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32591.169622 secs 2023-09-10T08:22:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:39 smithi098 ceph-mon[103124]: pgmap v17540: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T08:22:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32591.169622 secs 2023-09-10T08:22:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:22:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:22:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:22:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:41 smithi170 ceph-mon[102832]: pgmap v17541: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T08:22:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:41 smithi082 ceph-mon[98694]: pgmap v17541: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T08:22:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:41 smithi098 ceph-mon[103124]: pgmap v17541: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T08:22:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:42 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:22:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:42 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:22:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:42 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:22:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:43 smithi170 ceph-mon[102832]: pgmap v17542: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 453 KiB/s wr, 18 op/s 2023-09-10T08:22:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:22:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32595.726309 secs 2023-09-10T08:22:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:43 smithi082 ceph-mon[98694]: pgmap v17542: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 453 KiB/s wr, 18 op/s 2023-09-10T08:22:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:22:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32595.726309 secs 2023-09-10T08:22:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:43 smithi098 ceph-mon[103124]: pgmap v17542: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 453 KiB/s wr, 18 op/s 2023-09-10T08:22:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:22:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32595.726309 secs 2023-09-10T08:22:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:22:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32596.169687 secs 2023-09-10T08:22:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:22:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32596.169687 secs 2023-09-10T08:22:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:22:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32596.169687 secs 2023-09-10T08:22:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:45 smithi170 ceph-mon[102832]: pgmap v17543: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 453 KiB/s wr, 12 op/s 2023-09-10T08:22:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:22:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:45 smithi082 ceph-mon[98694]: pgmap v17543: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 453 KiB/s wr, 12 op/s 2023-09-10T08:22:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:22:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:45 smithi098 ceph-mon[103124]: pgmap v17543: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 453 KiB/s wr, 12 op/s 2023-09-10T08:22:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:22:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:47 smithi170 ceph-mon[102832]: pgmap v17544: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 745 KiB/s wr, 19 op/s 2023-09-10T08:22:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:47 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:22:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:47 smithi082 ceph-mon[98694]: pgmap v17544: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 745 KiB/s wr, 19 op/s 2023-09-10T08:22:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:47 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:22:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:47 smithi098 ceph-mon[103124]: pgmap v17544: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 745 KiB/s wr, 19 op/s 2023-09-10T08:22:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:47 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:22:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32600.726368 secs 2023-09-10T08:22:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32600.726368 secs 2023-09-10T08:22:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32600.726368 secs 2023-09-10T08:22:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:49 smithi170 ceph-mon[102832]: pgmap v17545: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 559 KiB/s wr, 13 op/s 2023-09-10T08:22:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:22:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32601.169749 secs 2023-09-10T08:22:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:49 smithi082 ceph-mon[98694]: pgmap v17545: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 559 KiB/s wr, 13 op/s 2023-09-10T08:22:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:22:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32601.169749 secs 2023-09-10T08:22:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:49 smithi098 ceph-mon[103124]: pgmap v17545: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 559 KiB/s wr, 13 op/s 2023-09-10T08:22:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:22:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32601.169749 secs 2023-09-10T08:22:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:51 smithi170 ceph-mon[102832]: pgmap v17546: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 559 KiB/s wr, 13 op/s 2023-09-10T08:22:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:51 smithi082 ceph-mon[98694]: pgmap v17546: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 559 KiB/s wr, 13 op/s 2023-09-10T08:22:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:51 smithi098 ceph-mon[103124]: pgmap v17546: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 559 KiB/s wr, 13 op/s 2023-09-10T08:22:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:22:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:22:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:22:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:53 smithi170 ceph-mon[102832]: pgmap v17547: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 856 KiB/s wr, 20 op/s 2023-09-10T08:22:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32604.562640 secs 2023-09-10T08:22:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32605.726426 secs 2023-09-10T08:22:54.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:22:54.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:53 smithi082 ceph-mon[98694]: pgmap v17547: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 856 KiB/s wr, 20 op/s 2023-09-10T08:22:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32604.562640 secs 2023-09-10T08:22:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32605.726426 secs 2023-09-10T08:22:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:22:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:53 smithi098 ceph-mon[103124]: pgmap v17547: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 856 KiB/s wr, 20 op/s 2023-09-10T08:22:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32604.562640 secs 2023-09-10T08:22:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32605.726426 secs 2023-09-10T08:22:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:22:54.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32606.169816 secs 2023-09-10T08:22:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32606.169816 secs 2023-09-10T08:22:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32606.169816 secs 2023-09-10T08:22:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:55 smithi170 ceph-mon[102832]: pgmap v17548: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 589 KiB/s wr, 13 op/s 2023-09-10T08:22:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:55 smithi082 ceph-mon[98694]: pgmap v17548: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 589 KiB/s wr, 13 op/s 2023-09-10T08:22:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:55 smithi098 ceph-mon[103124]: pgmap v17548: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 589 KiB/s wr, 13 op/s 2023-09-10T08:22:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:22:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:22:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:22:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:22:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:22:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:22:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:22:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:57 smithi170 ceph-mon[102832]: pgmap v17549: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 811 KiB/s wr, 21 op/s 2023-09-10T08:22:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:57 smithi082 ceph-mon[98694]: pgmap v17549: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 811 KiB/s wr, 21 op/s 2023-09-10T08:22:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:57 smithi098 ceph-mon[103124]: pgmap v17549: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 811 KiB/s wr, 21 op/s 2023-09-10T08:22:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32609.562703 secs 2023-09-10T08:22:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32610.726489 secs 2023-09-10T08:22:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32609.562703 secs 2023-09-10T08:22:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32610.726489 secs 2023-09-10T08:22:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32609.562703 secs 2023-09-10T08:22:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32610.726489 secs 2023-09-10T08:23:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:59 smithi170 ceph-mon[102832]: pgmap v17550: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 519 KiB/s wr, 14 op/s 2023-09-10T08:23:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:22:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32611.169881 secs 2023-09-10T08:23:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:59 smithi082 ceph-mon[98694]: pgmap v17550: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 519 KiB/s wr, 14 op/s 2023-09-10T08:23:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:22:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32611.169881 secs 2023-09-10T08:23:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:59 smithi098 ceph-mon[103124]: pgmap v17550: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 519 KiB/s wr, 14 op/s 2023-09-10T08:23:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:22:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32611.169881 secs 2023-09-10T08:23:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:01 smithi170 ceph-mon[102832]: pgmap v17551: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 519 KiB/s wr, 14 op/s 2023-09-10T08:23:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:01 smithi082 ceph-mon[98694]: pgmap v17551: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 519 KiB/s wr, 14 op/s 2023-09-10T08:23:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:01 smithi098 ceph-mon[103124]: pgmap v17551: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 519 KiB/s wr, 14 op/s 2023-09-10T08:23:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:03 smithi170 ceph-mon[102832]: pgmap v17552: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 839 KiB/s wr, 35 op/s 2023-09-10T08:23:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32614.562780 secs 2023-09-10T08:23:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32615.726550 secs 2023-09-10T08:23:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:03 smithi082 ceph-mon[98694]: pgmap v17552: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 839 KiB/s wr, 35 op/s 2023-09-10T08:23:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32614.562780 secs 2023-09-10T08:23:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32615.726550 secs 2023-09-10T08:23:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:03 smithi098 ceph-mon[103124]: pgmap v17552: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 839 KiB/s wr, 35 op/s 2023-09-10T08:23:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32614.562780 secs 2023-09-10T08:23:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32615.726550 secs 2023-09-10T08:23:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32616.169949 secs 2023-09-10T08:23:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:23:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32616.169949 secs 2023-09-10T08:23:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:23:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32616.169949 secs 2023-09-10T08:23:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:23:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:05 smithi082 ceph-mon[98694]: pgmap v17553: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 542 KiB/s wr, 28 op/s 2023-09-10T08:23:06.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:23:06.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:05 smithi098 ceph-mon[103124]: pgmap v17553: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 542 KiB/s wr, 28 op/s 2023-09-10T08:23:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:23:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:05 smithi170 ceph-mon[102832]: pgmap v17553: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 542 KiB/s wr, 28 op/s 2023-09-10T08:23:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:23:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:07 smithi082 ceph-mon[98694]: pgmap v17554: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 543 KiB/s wr, 34 op/s 2023-09-10T08:23:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:07 smithi098 ceph-mon[103124]: pgmap v17554: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 543 KiB/s wr, 34 op/s 2023-09-10T08:23:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:07 smithi170 ceph-mon[102832]: pgmap v17554: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 543 KiB/s wr, 34 op/s 2023-09-10T08:23:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32619.562852 secs 2023-09-10T08:23:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32620.726607 secs 2023-09-10T08:23:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:08 smithi082 ceph-mon[98694]: osdmap e1165: 12 total, 12 up, 12 in 2023-09-10T08:23:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32619.562852 secs 2023-09-10T08:23:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32620.726607 secs 2023-09-10T08:23:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:08 smithi098 ceph-mon[103124]: osdmap e1165: 12 total, 12 up, 12 in 2023-09-10T08:23:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32619.562852 secs 2023-09-10T08:23:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32620.726607 secs 2023-09-10T08:23:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:08 smithi170 ceph-mon[102832]: osdmap e1165: 12 total, 12 up, 12 in 2023-09-10T08:23:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:09 smithi082 ceph-mon[98694]: pgmap v17555: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 321 KiB/s wr, 27 op/s 2023-09-10T08:23:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32621.170016 secs 2023-09-10T08:23:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:09 smithi098 ceph-mon[103124]: pgmap v17555: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 321 KiB/s wr, 27 op/s 2023-09-10T08:23:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32621.170016 secs 2023-09-10T08:23:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:09 smithi170 ceph-mon[102832]: pgmap v17555: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 321 KiB/s wr, 27 op/s 2023-09-10T08:23:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32621.170016 secs 2023-09-10T08:23:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:11 smithi082 ceph-mon[98694]: pgmap v17557: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 385 KiB/s wr, 32 op/s 2023-09-10T08:23:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:11 smithi098 ceph-mon[103124]: pgmap v17557: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 385 KiB/s wr, 32 op/s 2023-09-10T08:23:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:11 smithi170 ceph-mon[102832]: pgmap v17557: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 385 KiB/s wr, 32 op/s 2023-09-10T08:23:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:23:13.405 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:23:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:23:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:13 smithi098 ceph-mon[103124]: pgmap v17558: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 116 KiB/s wr, 14 op/s 2023-09-10T08:23:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32624.562916 secs 2023-09-10T08:23:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32625.726675 secs 2023-09-10T08:23:14.335 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:13 smithi170 ceph-mon[102832]: pgmap v17558: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 116 KiB/s wr, 14 op/s 2023-09-10T08:23:14.335 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32624.562916 secs 2023-09-10T08:23:14.335 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32625.726675 secs 2023-09-10T08:23:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:13 smithi082 ceph-mon[98694]: pgmap v17558: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 116 KiB/s wr, 14 op/s 2023-09-10T08:23:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32624.562916 secs 2023-09-10T08:23:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32625.726675 secs 2023-09-10T08:23:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32626.170084 secs 2023-09-10T08:23:15.375 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32626.170084 secs 2023-09-10T08:23:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32626.170084 secs 2023-09-10T08:23:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:15 smithi098 ceph-mon[103124]: pgmap v17559: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 116 KiB/s wr, 14 op/s 2023-09-10T08:23:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:23:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:23:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:15 smithi170 ceph-mon[102832]: pgmap v17559: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 116 KiB/s wr, 14 op/s 2023-09-10T08:23:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:23:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:23:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:15 smithi082 ceph-mon[98694]: pgmap v17559: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 116 KiB/s wr, 14 op/s 2023-09-10T08:23:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:23:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:23:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:23:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:23:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:23:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:23:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:23:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:23:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:23:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:23:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:23:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:23:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:23:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:23:17.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:23:17.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:23:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:23:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:23:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:23:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:23:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:23:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:23:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:23:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:17 smithi098 ceph-mon[103124]: pgmap v17560: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T08:23:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:17 smithi098 ceph-mon[103124]: osdmap e1166: 12 total, 12 up, 12 in 2023-09-10T08:23:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:17 smithi170 ceph-mon[102832]: pgmap v17560: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T08:23:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:17 smithi170 ceph-mon[102832]: osdmap e1166: 12 total, 12 up, 12 in 2023-09-10T08:23:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:17 smithi082 ceph-mon[98694]: pgmap v17560: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T08:23:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:17 smithi082 ceph-mon[98694]: osdmap e1166: 12 total, 12 up, 12 in 2023-09-10T08:23:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32629.562977 secs 2023-09-10T08:23:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:18 smithi098 ceph-mon[103124]: pgmap v17562: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 13 op/s 2023-09-10T08:23:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32630.726733 secs 2023-09-10T08:23:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32629.562977 secs 2023-09-10T08:23:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:18 smithi170 ceph-mon[102832]: pgmap v17562: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 13 op/s 2023-09-10T08:23:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32630.726733 secs 2023-09-10T08:23:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32629.562977 secs 2023-09-10T08:23:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:18 smithi082 ceph-mon[98694]: pgmap v17562: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 13 op/s 2023-09-10T08:23:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32630.726733 secs 2023-09-10T08:23:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32631.170151 secs 2023-09-10T08:23:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32631.170151 secs 2023-09-10T08:23:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32631.170151 secs 2023-09-10T08:23:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:21 smithi098 ceph-mon[103124]: pgmap v17563: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T08:23:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:21 smithi170 ceph-mon[102832]: pgmap v17563: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T08:23:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:21 smithi082 ceph-mon[98694]: pgmap v17563: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 12 op/s 2023-09-10T08:23:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:23 smithi170 ceph-mon[102832]: pgmap v17564: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2023-09-10T08:23:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32634.563055 secs 2023-09-10T08:23:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32635.726788 secs 2023-09-10T08:23:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:23 smithi082 ceph-mon[98694]: pgmap v17564: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2023-09-10T08:23:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32634.563055 secs 2023-09-10T08:23:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32635.726788 secs 2023-09-10T08:23:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:23 smithi098 ceph-mon[103124]: pgmap v17564: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2023-09-10T08:23:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32634.563055 secs 2023-09-10T08:23:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32635.726788 secs 2023-09-10T08:23:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32636.170218 secs 2023-09-10T08:23:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32636.170218 secs 2023-09-10T08:23:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32636.170218 secs 2023-09-10T08:23:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:25 smithi170 ceph-mon[102832]: pgmap v17565: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2023-09-10T08:23:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:25 smithi082 ceph-mon[98694]: pgmap v17565: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2023-09-10T08:23:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:25 smithi098 ceph-mon[103124]: pgmap v17565: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2023-09-10T08:23:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:27 smithi170 ceph-mon[102832]: pgmap v17566: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 12 op/s 2023-09-10T08:23:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:27 smithi082 ceph-mon[98694]: pgmap v17566: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 12 op/s 2023-09-10T08:23:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:27 smithi098 ceph-mon[103124]: pgmap v17566: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 12 op/s 2023-09-10T08:23:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32639.563113 secs 2023-09-10T08:23:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32640.726847 secs 2023-09-10T08:23:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32639.563113 secs 2023-09-10T08:23:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32640.726847 secs 2023-09-10T08:23:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32639.563113 secs 2023-09-10T08:23:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32640.726847 secs 2023-09-10T08:23:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:29 smithi170 ceph-mon[102832]: pgmap v17567: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T08:23:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32641.170283 secs 2023-09-10T08:23:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:29 smithi082 ceph-mon[98694]: pgmap v17567: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T08:23:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32641.170283 secs 2023-09-10T08:23:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:29 smithi098 ceph-mon[103124]: pgmap v17567: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T08:23:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32641.170283 secs 2023-09-10T08:23:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:31 smithi170 ceph-mon[102832]: pgmap v17568: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 10 op/s 2023-09-10T08:23:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:31 smithi082 ceph-mon[98694]: pgmap v17568: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 10 op/s 2023-09-10T08:23:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:31 smithi098 ceph-mon[103124]: pgmap v17568: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 10 op/s 2023-09-10T08:23:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:33 smithi170 ceph-mon[102832]: pgmap v17569: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 208 KiB/s wr, 15 op/s 2023-09-10T08:23:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32645.726916 secs 2023-09-10T08:23:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:33 smithi082 ceph-mon[98694]: pgmap v17569: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 208 KiB/s wr, 15 op/s 2023-09-10T08:23:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32645.726916 secs 2023-09-10T08:23:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:33 smithi098 ceph-mon[103124]: pgmap v17569: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 208 KiB/s wr, 15 op/s 2023-09-10T08:23:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32645.726916 secs 2023-09-10T08:23:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32646.170348 secs 2023-09-10T08:23:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32646.170348 secs 2023-09-10T08:23:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32646.170348 secs 2023-09-10T08:23:36.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:35 smithi170 ceph-mon[102832]: pgmap v17570: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 208 KiB/s wr, 10 op/s 2023-09-10T08:23:36.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:23:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:35 smithi082 ceph-mon[98694]: pgmap v17570: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 208 KiB/s wr, 10 op/s 2023-09-10T08:23:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:23:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:35 smithi098 ceph-mon[103124]: pgmap v17570: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 208 KiB/s wr, 10 op/s 2023-09-10T08:23:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:23:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:23:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:23:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:23:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:23:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:23:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:23:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:37 smithi170 ceph-mon[102832]: pgmap v17571: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 230 KiB/s wr, 16 op/s 2023-09-10T08:23:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:23:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:37 smithi082 ceph-mon[98694]: pgmap v17571: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 230 KiB/s wr, 16 op/s 2023-09-10T08:23:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:23:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:37 smithi098 ceph-mon[103124]: pgmap v17571: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 230 KiB/s wr, 16 op/s 2023-09-10T08:23:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:23:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32649.563234 secs 2023-09-10T08:23:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32650.727000 secs 2023-09-10T08:23:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32649.563234 secs 2023-09-10T08:23:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32650.727000 secs 2023-09-10T08:23:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32649.563234 secs 2023-09-10T08:23:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32650.727000 secs 2023-09-10T08:23:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:39 smithi170 ceph-mon[102832]: pgmap v17572: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T08:23:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32651.170416 secs 2023-09-10T08:23:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:39 smithi082 ceph-mon[98694]: pgmap v17572: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T08:23:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32651.170416 secs 2023-09-10T08:23:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:39 smithi098 ceph-mon[103124]: pgmap v17572: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T08:23:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32651.170416 secs 2023-09-10T08:23:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:23:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:23:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:23:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:41 smithi170 ceph-mon[102832]: pgmap v17573: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T08:23:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:41 smithi082 ceph-mon[98694]: pgmap v17573: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T08:23:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:41 smithi098 ceph-mon[103124]: pgmap v17573: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T08:23:43.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:42 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:23:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:42 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:23:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:42 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:23:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:43 smithi170 ceph-mon[102832]: pgmap v17574: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 506 KiB/s wr, 18 op/s 2023-09-10T08:23:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32654.563322 secs 2023-09-10T08:23:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32655.727060 secs 2023-09-10T08:23:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:43 smithi082 ceph-mon[98694]: pgmap v17574: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 506 KiB/s wr, 18 op/s 2023-09-10T08:23:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32654.563322 secs 2023-09-10T08:23:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32655.727060 secs 2023-09-10T08:23:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:43 smithi098 ceph-mon[103124]: pgmap v17574: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 506 KiB/s wr, 18 op/s 2023-09-10T08:23:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32654.563322 secs 2023-09-10T08:23:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32655.727060 secs 2023-09-10T08:23:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:23:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32656.170482 secs 2023-09-10T08:23:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:23:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32656.170482 secs 2023-09-10T08:23:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:23:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32656.170482 secs 2023-09-10T08:23:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:45 smithi170 ceph-mon[102832]: pgmap v17575: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 414 KiB/s wr, 12 op/s 2023-09-10T08:23:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:23:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:45 smithi082 ceph-mon[98694]: pgmap v17575: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 414 KiB/s wr, 12 op/s 2023-09-10T08:23:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:23:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:45 smithi098 ceph-mon[103124]: pgmap v17575: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 414 KiB/s wr, 12 op/s 2023-09-10T08:23:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:23:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:47 smithi170 ceph-mon[102832]: pgmap v17576: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 437 KiB/s wr, 18 op/s 2023-09-10T08:23:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:47 smithi082 ceph-mon[98694]: pgmap v17576: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 437 KiB/s wr, 18 op/s 2023-09-10T08:23:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:47 smithi098 ceph-mon[103124]: pgmap v17576: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 437 KiB/s wr, 18 op/s 2023-09-10T08:23:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32660.727111 secs 2023-09-10T08:23:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32660.727111 secs 2023-09-10T08:23:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32660.727111 secs 2023-09-10T08:23:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:49 smithi170 ceph-mon[102832]: pgmap v17577: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 414 KiB/s wr, 13 op/s 2023-09-10T08:23:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32661.170548 secs 2023-09-10T08:23:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:49 smithi082 ceph-mon[98694]: pgmap v17577: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 414 KiB/s wr, 13 op/s 2023-09-10T08:23:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32661.170548 secs 2023-09-10T08:23:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:49 smithi098 ceph-mon[103124]: pgmap v17577: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 414 KiB/s wr, 13 op/s 2023-09-10T08:23:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32661.170548 secs 2023-09-10T08:23:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:51 smithi170 ceph-mon[102832]: pgmap v17578: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 414 KiB/s wr, 13 op/s 2023-09-10T08:23:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:51 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:23:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:51 smithi082 ceph-mon[98694]: pgmap v17578: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 414 KiB/s wr, 13 op/s 2023-09-10T08:23:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:51 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:23:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:51 smithi098 ceph-mon[103124]: pgmap v17578: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 414 KiB/s wr, 13 op/s 2023-09-10T08:23:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:51 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:23:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:52 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:23:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:23:53.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:52 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:23:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:23:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:52 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:23:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:23:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:53 smithi170 ceph-mon[102832]: pgmap v17579: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 823 KiB/s wr, 20 op/s 2023-09-10T08:23:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32665.727173 secs 2023-09-10T08:23:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:53 smithi082 ceph-mon[98694]: pgmap v17579: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 823 KiB/s wr, 20 op/s 2023-09-10T08:23:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32665.727173 secs 2023-09-10T08:23:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:53 smithi098 ceph-mon[103124]: pgmap v17579: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 823 KiB/s wr, 20 op/s 2023-09-10T08:23:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32665.727173 secs 2023-09-10T08:23:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32666.170616 secs 2023-09-10T08:23:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32666.170616 secs 2023-09-10T08:23:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32666.170616 secs 2023-09-10T08:23:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:55 smithi170 ceph-mon[102832]: pgmap v17580: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 432 KiB/s wr, 13 op/s 2023-09-10T08:23:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:55 smithi082 ceph-mon[98694]: pgmap v17580: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 432 KiB/s wr, 13 op/s 2023-09-10T08:23:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:55 smithi098 ceph-mon[103124]: pgmap v17580: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 432 KiB/s wr, 13 op/s 2023-09-10T08:23:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:23:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:23:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:23:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:57 smithi170 ceph-mon[102832]: pgmap v17581: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 478 KiB/s wr, 19 op/s 2023-09-10T08:23:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:23:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:57 smithi082 ceph-mon[98694]: pgmap v17581: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 478 KiB/s wr, 19 op/s 2023-09-10T08:23:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:23:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:57 smithi098 ceph-mon[103124]: pgmap v17581: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 478 KiB/s wr, 19 op/s 2023-09-10T08:23:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:23:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:23:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32669.563501 secs 2023-09-10T08:23:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32670.727255 secs 2023-09-10T08:23:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32669.563501 secs 2023-09-10T08:23:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32670.727255 secs 2023-09-10T08:23:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32669.563501 secs 2023-09-10T08:23:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32670.727255 secs 2023-09-10T08:24:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:59 smithi082 ceph-mon[98694]: pgmap v17582: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 456 KiB/s wr, 13 op/s 2023-09-10T08:24:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:23:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32671.170682 secs 2023-09-10T08:24:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:59 smithi098 ceph-mon[103124]: pgmap v17582: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 456 KiB/s wr, 13 op/s 2023-09-10T08:24:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:23:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32671.170682 secs 2023-09-10T08:24:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:59 smithi170 ceph-mon[102832]: pgmap v17582: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 456 KiB/s wr, 13 op/s 2023-09-10T08:24:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:23:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32671.170682 secs 2023-09-10T08:24:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:24:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:24:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:24:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:24:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:24:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:24:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:01 smithi082 ceph-mon[98694]: pgmap v17583: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 456 KiB/s wr, 13 op/s 2023-09-10T08:24:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:01 smithi098 ceph-mon[103124]: pgmap v17583: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 456 KiB/s wr, 13 op/s 2023-09-10T08:24:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:01 smithi170 ceph-mon[102832]: pgmap v17583: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 456 KiB/s wr, 13 op/s 2023-09-10T08:24:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:03 smithi082 ceph-mon[98694]: pgmap v17584: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 930 KiB/s wr, 23 op/s 2023-09-10T08:24:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32674.563562 secs 2023-09-10T08:24:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32675.727296 secs 2023-09-10T08:24:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:03 smithi082 ceph-mon[98694]: osdmap e1167: 12 total, 12 up, 12 in 2023-09-10T08:24:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:03 smithi098 ceph-mon[103124]: pgmap v17584: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 930 KiB/s wr, 23 op/s 2023-09-10T08:24:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32674.563562 secs 2023-09-10T08:24:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32675.727296 secs 2023-09-10T08:24:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:03 smithi098 ceph-mon[103124]: osdmap e1167: 12 total, 12 up, 12 in 2023-09-10T08:24:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:03 smithi170 ceph-mon[102832]: pgmap v17584: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 930 KiB/s wr, 23 op/s 2023-09-10T08:24:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32674.563562 secs 2023-09-10T08:24:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32675.727296 secs 2023-09-10T08:24:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:03 smithi170 ceph-mon[102832]: osdmap e1167: 12 total, 12 up, 12 in 2023-09-10T08:24:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32676.170748 secs 2023-09-10T08:24:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32676.170748 secs 2023-09-10T08:24:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32676.170748 secs 2023-09-10T08:24:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:05 smithi082 ceph-mon[98694]: pgmap v17586: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 625 KiB/s wr, 19 op/s 2023-09-10T08:24:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:05 smithi098 ceph-mon[103124]: pgmap v17586: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 625 KiB/s wr, 19 op/s 2023-09-10T08:24:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:05 smithi170 ceph-mon[102832]: pgmap v17586: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 625 KiB/s wr, 19 op/s 2023-09-10T08:24:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:07 smithi082 ceph-mon[98694]: pgmap v17587: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 898 KiB/s wr, 32 op/s 2023-09-10T08:24:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:07 smithi082 ceph-mon[98694]: osdmap e1168: 12 total, 12 up, 12 in 2023-09-10T08:24:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:07 smithi098 ceph-mon[103124]: pgmap v17587: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 898 KiB/s wr, 32 op/s 2023-09-10T08:24:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:07 smithi098 ceph-mon[103124]: osdmap e1168: 12 total, 12 up, 12 in 2023-09-10T08:24:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:07 smithi170 ceph-mon[102832]: pgmap v17587: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 898 KiB/s wr, 32 op/s 2023-09-10T08:24:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:07 smithi170 ceph-mon[102832]: osdmap e1168: 12 total, 12 up, 12 in 2023-09-10T08:24:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32679.563639 secs 2023-09-10T08:24:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32680.727358 secs 2023-09-10T08:24:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32679.563639 secs 2023-09-10T08:24:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32680.727358 secs 2023-09-10T08:24:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32679.563639 secs 2023-09-10T08:24:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32680.727358 secs 2023-09-10T08:24:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:09 smithi082 ceph-mon[98694]: pgmap v17589: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 40 op/s 2023-09-10T08:24:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32681.170815 secs 2023-09-10T08:24:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:09 smithi098 ceph-mon[103124]: pgmap v17589: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 40 op/s 2023-09-10T08:24:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32681.170815 secs 2023-09-10T08:24:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:09 smithi170 ceph-mon[102832]: pgmap v17589: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 40 op/s 2023-09-10T08:24:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32681.170815 secs 2023-09-10T08:24:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:11 smithi098 ceph-mon[103124]: pgmap v17590: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 411 KiB/s wr, 26 op/s 2023-09-10T08:24:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:11 smithi170 ceph-mon[102832]: pgmap v17590: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 411 KiB/s wr, 26 op/s 2023-09-10T08:24:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:11 smithi082 ceph-mon[98694]: pgmap v17590: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 411 KiB/s wr, 26 op/s 2023-09-10T08:24:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:24:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:24:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:24:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:13 smithi098 ceph-mon[103124]: pgmap v17591: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 570 KiB/s wr, 34 op/s 2023-09-10T08:24:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32684.563790 secs 2023-09-10T08:24:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:24:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:14.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32685.727417 secs 2023-09-10T08:24:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:13 smithi170 ceph-mon[102832]: pgmap v17591: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 570 KiB/s wr, 34 op/s 2023-09-10T08:24:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32684.563790 secs 2023-09-10T08:24:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:24:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32685.727417 secs 2023-09-10T08:24:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:13 smithi082 ceph-mon[98694]: pgmap v17591: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 570 KiB/s wr, 34 op/s 2023-09-10T08:24:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32684.563790 secs 2023-09-10T08:24:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:24:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32685.727417 secs 2023-09-10T08:24:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32686.170883 secs 2023-09-10T08:24:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32686.170883 secs 2023-09-10T08:24:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32686.170883 secs 2023-09-10T08:24:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:15 smithi098 ceph-mon[103124]: pgmap v17592: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 468 KiB/s wr, 28 op/s 2023-09-10T08:24:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:15 smithi170 ceph-mon[102832]: pgmap v17592: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 468 KiB/s wr, 28 op/s 2023-09-10T08:24:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:15 smithi082 ceph-mon[98694]: pgmap v17592: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 468 KiB/s wr, 28 op/s 2023-09-10T08:24:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:24:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:24:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:24:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:24:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:24:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:24:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:24:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:24:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:24:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:24:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:24:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:24:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:17 smithi098 ceph-mon[103124]: pgmap v17593: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 13 op/s 2023-09-10T08:24:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:17 smithi170 ceph-mon[102832]: pgmap v17593: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 13 op/s 2023-09-10T08:24:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:17 smithi082 ceph-mon[98694]: pgmap v17593: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 13 op/s 2023-09-10T08:24:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32689.563847 secs 2023-09-10T08:24:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32690.727477 secs 2023-09-10T08:24:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32689.563847 secs 2023-09-10T08:24:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32690.727477 secs 2023-09-10T08:24:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32689.563847 secs 2023-09-10T08:24:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32690.727477 secs 2023-09-10T08:24:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:20 smithi098 ceph-mon[103124]: pgmap v17594: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 166 KiB/s wr, 13 op/s 2023-09-10T08:24:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32691.170947 secs 2023-09-10T08:24:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:20 smithi170 ceph-mon[102832]: pgmap v17594: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 166 KiB/s wr, 13 op/s 2023-09-10T08:24:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32691.170947 secs 2023-09-10T08:24:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:20 smithi082 ceph-mon[98694]: pgmap v17594: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 166 KiB/s wr, 13 op/s 2023-09-10T08:24:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32691.170947 secs 2023-09-10T08:24:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:22 smithi098 ceph-mon[103124]: pgmap v17595: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T08:24:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:22 smithi170 ceph-mon[102832]: pgmap v17595: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T08:24:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:22 smithi082 ceph-mon[98694]: pgmap v17595: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T08:24:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:23 smithi170 ceph-mon[102832]: pgmap v17596: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 16 op/s 2023-09-10T08:24:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32694.563964 secs 2023-09-10T08:24:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32695.727546 secs 2023-09-10T08:24:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:23 smithi082 ceph-mon[98694]: pgmap v17596: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 16 op/s 2023-09-10T08:24:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32694.563964 secs 2023-09-10T08:24:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32695.727546 secs 2023-09-10T08:24:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:23 smithi098 ceph-mon[103124]: pgmap v17596: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 141 KiB/s wr, 16 op/s 2023-09-10T08:24:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32694.563964 secs 2023-09-10T08:24:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32695.727546 secs 2023-09-10T08:24:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32696.171012 secs 2023-09-10T08:24:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32696.171012 secs 2023-09-10T08:24:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32696.171012 secs 2023-09-10T08:24:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:25 smithi170 ceph-mon[102832]: pgmap v17597: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 10 op/s 2023-09-10T08:24:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:25 smithi082 ceph-mon[98694]: pgmap v17597: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 10 op/s 2023-09-10T08:24:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:25 smithi098 ceph-mon[103124]: pgmap v17597: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 10 op/s 2023-09-10T08:24:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:27 smithi170 ceph-mon[102832]: pgmap v17598: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 142 KiB/s wr, 15 op/s 2023-09-10T08:24:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:27 smithi082 ceph-mon[98694]: pgmap v17598: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 142 KiB/s wr, 15 op/s 2023-09-10T08:24:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:27 smithi098 ceph-mon[103124]: pgmap v17598: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 142 KiB/s wr, 15 op/s 2023-09-10T08:24:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32699.564024 secs 2023-09-10T08:24:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32700.727610 secs 2023-09-10T08:24:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32699.564024 secs 2023-09-10T08:24:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32700.727610 secs 2023-09-10T08:24:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32699.564024 secs 2023-09-10T08:24:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32700.727610 secs 2023-09-10T08:24:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:29 smithi170 ceph-mon[102832]: pgmap v17599: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 10 op/s 2023-09-10T08:24:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32701.171077 secs 2023-09-10T08:24:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:29 smithi082 ceph-mon[98694]: pgmap v17599: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 10 op/s 2023-09-10T08:24:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32701.171077 secs 2023-09-10T08:24:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:29 smithi098 ceph-mon[103124]: pgmap v17599: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 10 op/s 2023-09-10T08:24:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32701.171077 secs 2023-09-10T08:24:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:31 smithi170 ceph-mon[102832]: pgmap v17600: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 10 op/s 2023-09-10T08:24:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:31 smithi082 ceph-mon[98694]: pgmap v17600: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 10 op/s 2023-09-10T08:24:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:31 smithi098 ceph-mon[103124]: pgmap v17600: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 10 op/s 2023-09-10T08:24:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:33 smithi170 ceph-mon[102832]: pgmap v17601: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 144 KiB/s wr, 15 op/s 2023-09-10T08:24:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:24:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32705.727663 secs 2023-09-10T08:24:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:33 smithi082 ceph-mon[98694]: pgmap v17601: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 144 KiB/s wr, 15 op/s 2023-09-10T08:24:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:24:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32705.727663 secs 2023-09-10T08:24:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:33 smithi098 ceph-mon[103124]: pgmap v17601: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 144 KiB/s wr, 15 op/s 2023-09-10T08:24:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:24:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32705.727663 secs 2023-09-10T08:24:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32706.171143 secs 2023-09-10T08:24:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32706.171143 secs 2023-09-10T08:24:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32706.171143 secs 2023-09-10T08:24:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:35 smithi170 ceph-mon[102832]: pgmap v17602: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T08:24:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:24:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:35 smithi082 ceph-mon[98694]: pgmap v17602: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T08:24:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:24:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:35 smithi098 ceph-mon[103124]: pgmap v17602: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T08:24:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:24:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:24:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:24:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:24:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:24:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:24:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:24:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:37 smithi170 ceph-mon[102832]: pgmap v17603: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 280 KiB/s wr, 17 op/s 2023-09-10T08:24:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:37 smithi082 ceph-mon[98694]: pgmap v17603: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 280 KiB/s wr, 17 op/s 2023-09-10T08:24:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:37 smithi098 ceph-mon[103124]: pgmap v17603: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 280 KiB/s wr, 17 op/s 2023-09-10T08:24:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32709.564167 secs 2023-09-10T08:24:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32710.727748 secs 2023-09-10T08:24:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32709.564167 secs 2023-09-10T08:24:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32710.727748 secs 2023-09-10T08:24:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32709.564167 secs 2023-09-10T08:24:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32710.727748 secs 2023-09-10T08:24:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:39 smithi170 ceph-mon[102832]: pgmap v17604: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T08:24:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32711.171210 secs 2023-09-10T08:24:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:39 smithi082 ceph-mon[98694]: pgmap v17604: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T08:24:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32711.171210 secs 2023-09-10T08:24:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:39 smithi098 ceph-mon[103124]: pgmap v17604: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T08:24:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32711.171210 secs 2023-09-10T08:24:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:24:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:24:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:24:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:41 smithi170 ceph-mon[102832]: pgmap v17605: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T08:24:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:41 smithi082 ceph-mon[98694]: pgmap v17605: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T08:24:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:41 smithi098 ceph-mon[103124]: pgmap v17605: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 163 KiB/s wr, 12 op/s 2023-09-10T08:24:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:42 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:24:43.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:42 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:24:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:42 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:24:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:43 smithi170 ceph-mon[102832]: pgmap v17606: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 550 KiB/s wr, 20 op/s 2023-09-10T08:24:44.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:24:44.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32715.727809 secs 2023-09-10T08:24:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:43 smithi082 ceph-mon[98694]: pgmap v17606: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 550 KiB/s wr, 20 op/s 2023-09-10T08:24:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:24:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32715.727809 secs 2023-09-10T08:24:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:43 smithi098 ceph-mon[103124]: pgmap v17606: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 550 KiB/s wr, 20 op/s 2023-09-10T08:24:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:24:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32715.727809 secs 2023-09-10T08:24:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:24:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32716.171272 secs 2023-09-10T08:24:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:24:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32716.171272 secs 2023-09-10T08:24:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:24:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32716.171272 secs 2023-09-10T08:24:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:45 smithi170 ceph-mon[102832]: pgmap v17607: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 523 KiB/s wr, 15 op/s 2023-09-10T08:24:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:24:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:45 smithi082 ceph-mon[98694]: pgmap v17607: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 523 KiB/s wr, 15 op/s 2023-09-10T08:24:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:24:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:45 smithi098 ceph-mon[103124]: pgmap v17607: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 523 KiB/s wr, 15 op/s 2023-09-10T08:24:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:24:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:47 smithi170 ceph-mon[102832]: pgmap v17608: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 757 KiB/s wr, 22 op/s 2023-09-10T08:24:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:47 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:24:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:47 smithi082 ceph-mon[98694]: pgmap v17608: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 757 KiB/s wr, 22 op/s 2023-09-10T08:24:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:47 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:24:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:47 smithi098 ceph-mon[103124]: pgmap v17608: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 757 KiB/s wr, 22 op/s 2023-09-10T08:24:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:47 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:24:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32720.727865 secs 2023-09-10T08:24:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32720.727865 secs 2023-09-10T08:24:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32720.727865 secs 2023-09-10T08:24:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:49 smithi170 ceph-mon[102832]: pgmap v17609: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 621 KiB/s wr, 15 op/s 2023-09-10T08:24:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32721.171342 secs 2023-09-10T08:24:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:24:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:49 smithi082 ceph-mon[98694]: pgmap v17609: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 621 KiB/s wr, 15 op/s 2023-09-10T08:24:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32721.171342 secs 2023-09-10T08:24:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:24:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:49 smithi098 ceph-mon[103124]: pgmap v17609: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 621 KiB/s wr, 15 op/s 2023-09-10T08:24:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32721.171342 secs 2023-09-10T08:24:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:24:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:51 smithi170 ceph-mon[102832]: pgmap v17610: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 621 KiB/s wr, 15 op/s 2023-09-10T08:24:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:51 smithi082 ceph-mon[98694]: pgmap v17610: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 621 KiB/s wr, 15 op/s 2023-09-10T08:24:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:51 smithi098 ceph-mon[103124]: pgmap v17610: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 621 KiB/s wr, 15 op/s 2023-09-10T08:24:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:24:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:24:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:24:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:53 smithi082 ceph-mon[98694]: pgmap v17611: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 23 op/s 2023-09-10T08:24:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32724.564440 secs 2023-09-10T08:24:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:24:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32725.727950 secs 2023-09-10T08:24:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:53 smithi098 ceph-mon[103124]: pgmap v17611: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 23 op/s 2023-09-10T08:24:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32724.564440 secs 2023-09-10T08:24:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:24:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:54.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32725.727950 secs 2023-09-10T08:24:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:53 smithi170 ceph-mon[102832]: pgmap v17611: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 23 op/s 2023-09-10T08:24:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32724.564440 secs 2023-09-10T08:24:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:24:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32725.727950 secs 2023-09-10T08:24:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32726.171406 secs 2023-09-10T08:24:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32726.171406 secs 2023-09-10T08:24:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32726.171406 secs 2023-09-10T08:24:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:55 smithi082 ceph-mon[98694]: pgmap v17612: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 775 KiB/s wr, 14 op/s 2023-09-10T08:24:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:55 smithi098 ceph-mon[103124]: pgmap v17612: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 775 KiB/s wr, 14 op/s 2023-09-10T08:24:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:55 smithi170 ceph-mon[102832]: pgmap v17612: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 775 KiB/s wr, 14 op/s 2023-09-10T08:24:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:24:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:24:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:24:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:24:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:24:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:24:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:57 smithi082 ceph-mon[98694]: pgmap v17613: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 891 KiB/s wr, 20 op/s 2023-09-10T08:24:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:24:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:57 smithi098 ceph-mon[103124]: pgmap v17613: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 891 KiB/s wr, 20 op/s 2023-09-10T08:24:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:24:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:57 smithi170 ceph-mon[102832]: pgmap v17613: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 891 KiB/s wr, 20 op/s 2023-09-10T08:24:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:24:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:24:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32729.564501 secs 2023-09-10T08:24:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32730.728011 secs 2023-09-10T08:24:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32729.564501 secs 2023-09-10T08:24:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32730.728011 secs 2023-09-10T08:24:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32729.564501 secs 2023-09-10T08:24:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32730.728011 secs 2023-09-10T08:25:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:59 smithi082 ceph-mon[98694]: pgmap v17614: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 657 KiB/s wr, 13 op/s 2023-09-10T08:25:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:24:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32731.171479 secs 2023-09-10T08:25:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:59 smithi098 ceph-mon[103124]: pgmap v17614: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 657 KiB/s wr, 13 op/s 2023-09-10T08:25:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:24:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32731.171479 secs 2023-09-10T08:25:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:59 smithi170 ceph-mon[102832]: pgmap v17614: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 657 KiB/s wr, 13 op/s 2023-09-10T08:25:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:24:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32731.171479 secs 2023-09-10T08:25:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:25:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:25:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:25:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:25:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:25:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:25:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:01 smithi098 ceph-mon[103124]: pgmap v17615: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 657 KiB/s wr, 13 op/s 2023-09-10T08:25:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:01 smithi170 ceph-mon[102832]: pgmap v17615: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 657 KiB/s wr, 13 op/s 2023-09-10T08:25:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:01 smithi082 ceph-mon[98694]: pgmap v17615: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 657 KiB/s wr, 13 op/s 2023-09-10T08:25:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:03 smithi098 ceph-mon[103124]: pgmap v17616: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 924 KiB/s wr, 19 op/s 2023-09-10T08:25:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32734.564576 secs 2023-09-10T08:25:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32735.728071 secs 2023-09-10T08:25:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:03 smithi170 ceph-mon[102832]: pgmap v17616: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 924 KiB/s wr, 19 op/s 2023-09-10T08:25:04.482 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32734.564576 secs 2023-09-10T08:25:04.483 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32735.728071 secs 2023-09-10T08:25:04.484 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:03 smithi082 ceph-mon[98694]: pgmap v17616: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 924 KiB/s wr, 19 op/s 2023-09-10T08:25:04.484 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32734.564576 secs 2023-09-10T08:25:04.484 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32735.728071 secs 2023-09-10T08:25:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32736.171543 secs 2023-09-10T08:25:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32736.171543 secs 2023-09-10T08:25:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32736.171543 secs 2023-09-10T08:25:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:05 smithi098 ceph-mon[103124]: pgmap v17617: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 383 KiB/s wr, 12 op/s 2023-09-10T08:25:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:05 smithi170 ceph-mon[102832]: pgmap v17617: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 383 KiB/s wr, 12 op/s 2023-09-10T08:25:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:05 smithi082 ceph-mon[98694]: pgmap v17617: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 383 KiB/s wr, 12 op/s 2023-09-10T08:25:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:07 smithi098 ceph-mon[103124]: pgmap v17618: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 429 KiB/s wr, 32 op/s 2023-09-10T08:25:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:07 smithi170 ceph-mon[102832]: pgmap v17618: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 429 KiB/s wr, 32 op/s 2023-09-10T08:25:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:07 smithi082 ceph-mon[98694]: pgmap v17618: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 429 KiB/s wr, 32 op/s 2023-09-10T08:25:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32739.564636 secs 2023-09-10T08:25:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32740.728161 secs 2023-09-10T08:25:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:08 smithi098 ceph-mon[103124]: osdmap e1169: 12 total, 12 up, 12 in 2023-09-10T08:25:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32739.564636 secs 2023-09-10T08:25:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32740.728161 secs 2023-09-10T08:25:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:08 smithi170 ceph-mon[102832]: osdmap e1169: 12 total, 12 up, 12 in 2023-09-10T08:25:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32739.564636 secs 2023-09-10T08:25:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32740.728161 secs 2023-09-10T08:25:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:08 smithi082 ceph-mon[98694]: osdmap e1169: 12 total, 12 up, 12 in 2023-09-10T08:25:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:09 smithi098 ceph-mon[103124]: pgmap v17619: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 312 KiB/s wr, 26 op/s 2023-09-10T08:25:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32741.171608 secs 2023-09-10T08:25:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:09 smithi170 ceph-mon[102832]: pgmap v17619: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 312 KiB/s wr, 26 op/s 2023-09-10T08:25:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32741.171608 secs 2023-09-10T08:25:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:09 smithi082 ceph-mon[98694]: pgmap v17619: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 312 KiB/s wr, 26 op/s 2023-09-10T08:25:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32741.171608 secs 2023-09-10T08:25:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:11 smithi098 ceph-mon[103124]: pgmap v17621: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 375 KiB/s wr, 31 op/s 2023-09-10T08:25:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:11 smithi170 ceph-mon[102832]: pgmap v17621: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 375 KiB/s wr, 31 op/s 2023-09-10T08:25:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:11 smithi082 ceph-mon[98694]: pgmap v17621: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 375 KiB/s wr, 31 op/s 2023-09-10T08:25:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:13 smithi098 ceph-mon[103124]: pgmap v17622: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 217 KiB/s wr, 30 op/s 2023-09-10T08:25:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32744.564697 secs 2023-09-10T08:25:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32745.728216 secs 2023-09-10T08:25:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:13 smithi170 ceph-mon[102832]: pgmap v17622: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 217 KiB/s wr, 30 op/s 2023-09-10T08:25:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32744.564697 secs 2023-09-10T08:25:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32745.728216 secs 2023-09-10T08:25:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:13 smithi082 ceph-mon[98694]: pgmap v17622: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 217 KiB/s wr, 30 op/s 2023-09-10T08:25:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32744.564697 secs 2023-09-10T08:25:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32745.728216 secs 2023-09-10T08:25:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32746.171674 secs 2023-09-10T08:25:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32746.171674 secs 2023-09-10T08:25:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32746.171674 secs 2023-09-10T08:25:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:16 smithi098 ceph-mon[103124]: pgmap v17623: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 217 KiB/s wr, 30 op/s 2023-09-10T08:25:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:16 smithi170 ceph-mon[102832]: pgmap v17623: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 217 KiB/s wr, 30 op/s 2023-09-10T08:25:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:16 smithi082 ceph-mon[98694]: pgmap v17623: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 217 KiB/s wr, 30 op/s 2023-09-10T08:25:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:18 smithi098 ceph-mon[103124]: pgmap v17624: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 192 KiB/s wr, 13 op/s 2023-09-10T08:25:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:25:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:25:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:25:18.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:25:18.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:18 smithi098 ceph-mon[103124]: osdmap e1170: 12 total, 12 up, 12 in 2023-09-10T08:25:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:18 smithi170 ceph-mon[102832]: pgmap v17624: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 192 KiB/s wr, 13 op/s 2023-09-10T08:25:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:25:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:25:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:25:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:25:18.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:18 smithi170 ceph-mon[102832]: osdmap e1170: 12 total, 12 up, 12 in 2023-09-10T08:25:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:18 smithi082 ceph-mon[98694]: pgmap v17624: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 192 KiB/s wr, 13 op/s 2023-09-10T08:25:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:25:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:25:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:25:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:25:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:18 smithi082 ceph-mon[98694]: osdmap e1170: 12 total, 12 up, 12 in 2023-09-10T08:25:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32749.564757 secs 2023-09-10T08:25:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32750.728275 secs 2023-09-10T08:25:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32749.564757 secs 2023-09-10T08:25:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32750.728275 secs 2023-09-10T08:25:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32749.564757 secs 2023-09-10T08:25:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32750.728275 secs 2023-09-10T08:25:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:20 smithi170 ceph-mon[102832]: pgmap v17626: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 208 KiB/s wr, 14 op/s 2023-09-10T08:25:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32751.171739 secs 2023-09-10T08:25:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:20 smithi082 ceph-mon[98694]: pgmap v17626: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 208 KiB/s wr, 14 op/s 2023-09-10T08:25:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32751.171739 secs 2023-09-10T08:25:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:20 smithi098 ceph-mon[103124]: pgmap v17626: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 208 KiB/s wr, 14 op/s 2023-09-10T08:25:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32751.171739 secs 2023-09-10T08:25:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:22 smithi170 ceph-mon[102832]: pgmap v17627: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 192 KiB/s wr, 13 op/s 2023-09-10T08:25:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:22 smithi082 ceph-mon[98694]: pgmap v17627: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 192 KiB/s wr, 13 op/s 2023-09-10T08:25:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:22 smithi098 ceph-mon[103124]: pgmap v17627: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 192 KiB/s wr, 13 op/s 2023-09-10T08:25:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:23 smithi170 ceph-mon[102832]: pgmap v17628: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 169 KiB/s wr, 13 op/s 2023-09-10T08:25:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32754.564777 secs 2023-09-10T08:25:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32755.728329 secs 2023-09-10T08:25:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:23 smithi082 ceph-mon[98694]: pgmap v17628: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 169 KiB/s wr, 13 op/s 2023-09-10T08:25:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32754.564777 secs 2023-09-10T08:25:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32755.728329 secs 2023-09-10T08:25:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:23 smithi098 ceph-mon[103124]: pgmap v17628: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 169 KiB/s wr, 13 op/s 2023-09-10T08:25:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32754.564777 secs 2023-09-10T08:25:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32755.728329 secs 2023-09-10T08:25:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32756.171807 secs 2023-09-10T08:25:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32756.171807 secs 2023-09-10T08:25:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32756.171807 secs 2023-09-10T08:25:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:25 smithi170 ceph-mon[102832]: pgmap v17629: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 169 KiB/s wr, 13 op/s 2023-09-10T08:25:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:25 smithi082 ceph-mon[98694]: pgmap v17629: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 169 KiB/s wr, 13 op/s 2023-09-10T08:25:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:25 smithi098 ceph-mon[103124]: pgmap v17629: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 169 KiB/s wr, 13 op/s 2023-09-10T08:25:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:27 smithi170 ceph-mon[102832]: pgmap v17630: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 140 KiB/s wr, 13 op/s 2023-09-10T08:25:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:27 smithi082 ceph-mon[98694]: pgmap v17630: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 140 KiB/s wr, 13 op/s 2023-09-10T08:25:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:27 smithi098 ceph-mon[103124]: pgmap v17630: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 140 KiB/s wr, 13 op/s 2023-09-10T08:25:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32759.564844 secs 2023-09-10T08:25:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32760.728416 secs 2023-09-10T08:25:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32759.564844 secs 2023-09-10T08:25:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32760.728416 secs 2023-09-10T08:25:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32759.564844 secs 2023-09-10T08:25:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32760.728416 secs 2023-09-10T08:25:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:29 smithi170 ceph-mon[102832]: pgmap v17631: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 137 KiB/s wr, 13 op/s 2023-09-10T08:25:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:29 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:25:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:29 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:25:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32761.171870 secs 2023-09-10T08:25:30.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:29 smithi082 ceph-mon[98694]: pgmap v17631: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 137 KiB/s wr, 13 op/s 2023-09-10T08:25:30.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:29 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:25:30.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:29 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:25:30.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32761.171870 secs 2023-09-10T08:25:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:29 smithi098 ceph-mon[103124]: pgmap v17631: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 137 KiB/s wr, 13 op/s 2023-09-10T08:25:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:29 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:25:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:29 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:25:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32761.171870 secs 2023-09-10T08:25:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:31 smithi170 ceph-mon[102832]: pgmap v17632: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T08:25:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:31 smithi082 ceph-mon[98694]: pgmap v17632: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T08:25:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:31 smithi098 ceph-mon[103124]: pgmap v17632: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T08:25:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:25:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:25:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:25:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:25:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:25:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:25:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:33 smithi170 ceph-mon[102832]: pgmap v17633: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 233 KiB/s wr, 16 op/s 2023-09-10T08:25:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32765.728473 secs 2023-09-10T08:25:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:33 smithi082 ceph-mon[98694]: pgmap v17633: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 233 KiB/s wr, 16 op/s 2023-09-10T08:25:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32765.728473 secs 2023-09-10T08:25:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:33 smithi098 ceph-mon[103124]: pgmap v17633: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 233 KiB/s wr, 16 op/s 2023-09-10T08:25:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32765.728473 secs 2023-09-10T08:25:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32766.171938 secs 2023-09-10T08:25:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32766.171938 secs 2023-09-10T08:25:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32766.171938 secs 2023-09-10T08:25:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:35 smithi170 ceph-mon[102832]: pgmap v17634: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 10 op/s 2023-09-10T08:25:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:25:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:35 smithi082 ceph-mon[98694]: pgmap v17634: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 10 op/s 2023-09-10T08:25:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:25:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:35 smithi098 ceph-mon[103124]: pgmap v17634: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 10 op/s 2023-09-10T08:25:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:25:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:25:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:25:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:25:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:25:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:25:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:25:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:25:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:25:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:25:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:37 smithi170 ceph-mon[102832]: pgmap v17635: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 264 KiB/s wr, 16 op/s 2023-09-10T08:25:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:37 smithi082 ceph-mon[98694]: pgmap v17635: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 264 KiB/s wr, 16 op/s 2023-09-10T08:25:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:37 smithi098 ceph-mon[103124]: pgmap v17635: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 264 KiB/s wr, 16 op/s 2023-09-10T08:25:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32770.728554 secs 2023-09-10T08:25:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32770.728554 secs 2023-09-10T08:25:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32770.728554 secs 2023-09-10T08:25:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:39 smithi170 ceph-mon[102832]: pgmap v17636: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 263 KiB/s wr, 10 op/s 2023-09-10T08:25:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32771.172005 secs 2023-09-10T08:25:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:39 smithi082 ceph-mon[98694]: pgmap v17636: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 263 KiB/s wr, 10 op/s 2023-09-10T08:25:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32771.172005 secs 2023-09-10T08:25:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:39 smithi098 ceph-mon[103124]: pgmap v17636: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 263 KiB/s wr, 10 op/s 2023-09-10T08:25:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32771.172005 secs 2023-09-10T08:25:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:41 smithi170 ceph-mon[102832]: pgmap v17637: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 263 KiB/s wr, 10 op/s 2023-09-10T08:25:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:41 smithi082 ceph-mon[98694]: pgmap v17637: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 263 KiB/s wr, 10 op/s 2023-09-10T08:25:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:41 smithi098 ceph-mon[103124]: pgmap v17637: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 263 KiB/s wr, 10 op/s 2023-09-10T08:25:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:43 smithi170 ceph-mon[102832]: pgmap v17638: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 480 KiB/s wr, 16 op/s 2023-09-10T08:25:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32775.728615 secs 2023-09-10T08:25:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:43 smithi082 ceph-mon[98694]: pgmap v17638: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 480 KiB/s wr, 16 op/s 2023-09-10T08:25:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32775.728615 secs 2023-09-10T08:25:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:43 smithi098 ceph-mon[103124]: pgmap v17638: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 480 KiB/s wr, 16 op/s 2023-09-10T08:25:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32775.728615 secs 2023-09-10T08:25:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32776.172069 secs 2023-09-10T08:25:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:25:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:25:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32776.172069 secs 2023-09-10T08:25:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:25:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:25:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32776.172069 secs 2023-09-10T08:25:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:25:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:25:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:45 smithi170 ceph-mon[102832]: pgmap v17639: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 364 KiB/s wr, 11 op/s 2023-09-10T08:25:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:25:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:25:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:45 smithi082 ceph-mon[98694]: pgmap v17639: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 364 KiB/s wr, 11 op/s 2023-09-10T08:25:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:25:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:25:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:45 smithi098 ceph-mon[103124]: pgmap v17639: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 364 KiB/s wr, 11 op/s 2023-09-10T08:25:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:25:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:25:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:47 smithi170 ceph-mon[102832]: pgmap v17640: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 522 KiB/s wr, 17 op/s 2023-09-10T08:25:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:47 smithi082 ceph-mon[98694]: pgmap v17640: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 522 KiB/s wr, 17 op/s 2023-09-10T08:25:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:47 smithi098 ceph-mon[103124]: pgmap v17640: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 522 KiB/s wr, 17 op/s 2023-09-10T08:25:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32779.565185 secs 2023-09-10T08:25:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32780.728703 secs 2023-09-10T08:25:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32779.565185 secs 2023-09-10T08:25:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32780.728703 secs 2023-09-10T08:25:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32779.565185 secs 2023-09-10T08:25:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32780.728703 secs 2023-09-10T08:25:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:49 smithi082 ceph-mon[98694]: pgmap v17641: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 375 KiB/s wr, 12 op/s 2023-09-10T08:25:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32781.172135 secs 2023-09-10T08:25:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:25:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:25:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:49 smithi098 ceph-mon[103124]: pgmap v17641: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 375 KiB/s wr, 12 op/s 2023-09-10T08:25:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32781.172135 secs 2023-09-10T08:25:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:25:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:25:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:49 smithi170 ceph-mon[102832]: pgmap v17641: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 375 KiB/s wr, 12 op/s 2023-09-10T08:25:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32781.172135 secs 2023-09-10T08:25:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:25:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:25:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:51 smithi082 ceph-mon[98694]: pgmap v17642: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 375 KiB/s wr, 12 op/s 2023-09-10T08:25:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:51 smithi098 ceph-mon[103124]: pgmap v17642: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 375 KiB/s wr, 12 op/s 2023-09-10T08:25:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:51 smithi170 ceph-mon[102832]: pgmap v17642: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 375 KiB/s wr, 12 op/s 2023-09-10T08:25:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:25:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:25:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:25:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:25:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:25:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:25:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:25:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:25:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:25:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:53 smithi082 ceph-mon[98694]: pgmap v17643: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 677 KiB/s wr, 19 op/s 2023-09-10T08:25:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32784.565245 secs 2023-09-10T08:25:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32785.728789 secs 2023-09-10T08:25:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:53 smithi098 ceph-mon[103124]: pgmap v17643: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 677 KiB/s wr, 19 op/s 2023-09-10T08:25:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32784.565245 secs 2023-09-10T08:25:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32785.728789 secs 2023-09-10T08:25:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:53 smithi170 ceph-mon[102832]: pgmap v17643: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 677 KiB/s wr, 19 op/s 2023-09-10T08:25:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32784.565245 secs 2023-09-10T08:25:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32785.728789 secs 2023-09-10T08:25:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32786.172201 secs 2023-09-10T08:25:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32786.172201 secs 2023-09-10T08:25:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32786.172201 secs 2023-09-10T08:25:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:55 smithi082 ceph-mon[98694]: pgmap v17644: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 460 KiB/s wr, 13 op/s 2023-09-10T08:25:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:55 smithi098 ceph-mon[103124]: pgmap v17644: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 460 KiB/s wr, 13 op/s 2023-09-10T08:25:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:55 smithi170 ceph-mon[102832]: pgmap v17644: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 460 KiB/s wr, 13 op/s 2023-09-10T08:25:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:57 smithi082 ceph-mon[98694]: pgmap v17645: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 541 KiB/s wr, 20 op/s 2023-09-10T08:25:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:25:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:25:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:57 smithi098 ceph-mon[103124]: pgmap v17645: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 541 KiB/s wr, 20 op/s 2023-09-10T08:25:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:25:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:25:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:57 smithi170 ceph-mon[102832]: pgmap v17645: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 541 KiB/s wr, 20 op/s 2023-09-10T08:25:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:25:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:25:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32790.728848 secs 2023-09-10T08:25:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32790.728848 secs 2023-09-10T08:25:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32790.728848 secs 2023-09-10T08:26:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:59 smithi098 ceph-mon[103124]: pgmap v17646: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 383 KiB/s wr, 13 op/s 2023-09-10T08:26:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32791.172264 secs 2023-09-10T08:26:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:25:59 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:26:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:59 smithi170 ceph-mon[102832]: pgmap v17646: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 383 KiB/s wr, 13 op/s 2023-09-10T08:26:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32791.172264 secs 2023-09-10T08:26:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:25:59 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:26:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:59 smithi082 ceph-mon[98694]: pgmap v17646: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 383 KiB/s wr, 13 op/s 2023-09-10T08:26:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32791.172264 secs 2023-09-10T08:26:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:25:59 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:26:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:00 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:26:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:26:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:00 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:26:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:26:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:00 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:26:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:26:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:01 smithi098 ceph-mon[103124]: pgmap v17647: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 383 KiB/s wr, 13 op/s 2023-09-10T08:26:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:01 smithi170 ceph-mon[102832]: pgmap v17647: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 383 KiB/s wr, 13 op/s 2023-09-10T08:26:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:01 smithi082 ceph-mon[98694]: pgmap v17647: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 383 KiB/s wr, 13 op/s 2023-09-10T08:26:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:03 smithi098 ceph-mon[103124]: pgmap v17648: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 780 KiB/s wr, 21 op/s 2023-09-10T08:26:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32794.565473 secs 2023-09-10T08:26:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32795.728937 secs 2023-09-10T08:26:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:03 smithi170 ceph-mon[102832]: pgmap v17648: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 780 KiB/s wr, 21 op/s 2023-09-10T08:26:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32794.565473 secs 2023-09-10T08:26:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32795.728937 secs 2023-09-10T08:26:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:03 smithi082 ceph-mon[98694]: pgmap v17648: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 780 KiB/s wr, 21 op/s 2023-09-10T08:26:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32794.565473 secs 2023-09-10T08:26:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32795.728937 secs 2023-09-10T08:26:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32796.172330 secs 2023-09-10T08:26:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:26:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32796.172330 secs 2023-09-10T08:26:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:26:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32796.172330 secs 2023-09-10T08:26:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:26:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:05 smithi098 ceph-mon[103124]: pgmap v17649: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 477 KiB/s wr, 13 op/s 2023-09-10T08:26:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:26:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:05 smithi170 ceph-mon[102832]: pgmap v17649: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 477 KiB/s wr, 13 op/s 2023-09-10T08:26:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:26:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:05 smithi082 ceph-mon[98694]: pgmap v17649: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 477 KiB/s wr, 13 op/s 2023-09-10T08:26:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:26:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:07 smithi098 ceph-mon[103124]: pgmap v17650: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 728 KiB/s wr, 34 op/s 2023-09-10T08:26:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:07 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:26:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:07 smithi170 ceph-mon[102832]: pgmap v17650: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 728 KiB/s wr, 34 op/s 2023-09-10T08:26:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:07 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:26:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:07 smithi082 ceph-mon[98694]: pgmap v17650: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 728 KiB/s wr, 34 op/s 2023-09-10T08:26:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:07 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:26:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32799.565533 secs 2023-09-10T08:26:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32800.728998 secs 2023-09-10T08:26:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:08 smithi098 ceph-mon[103124]: osdmap e1171: 12 total, 12 up, 12 in 2023-09-10T08:26:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32799.565533 secs 2023-09-10T08:26:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32800.728998 secs 2023-09-10T08:26:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:08 smithi170 ceph-mon[102832]: osdmap e1171: 12 total, 12 up, 12 in 2023-09-10T08:26:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32799.565533 secs 2023-09-10T08:26:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32800.728998 secs 2023-09-10T08:26:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:08 smithi082 ceph-mon[98694]: osdmap e1171: 12 total, 12 up, 12 in 2023-09-10T08:26:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:10 smithi098 ceph-mon[103124]: pgmap v17651: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 648 KiB/s wr, 27 op/s 2023-09-10T08:26:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32801.172395 secs 2023-09-10T08:26:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:10 smithi170 ceph-mon[102832]: pgmap v17651: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 648 KiB/s wr, 27 op/s 2023-09-10T08:26:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32801.172395 secs 2023-09-10T08:26:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:10 smithi082 ceph-mon[98694]: pgmap v17651: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 648 KiB/s wr, 27 op/s 2023-09-10T08:26:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32801.172395 secs 2023-09-10T08:26:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:12 smithi098 ceph-mon[103124]: pgmap v17653: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 777 KiB/s wr, 33 op/s 2023-09-10T08:26:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:12 smithi170 ceph-mon[102832]: pgmap v17653: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 777 KiB/s wr, 33 op/s 2023-09-10T08:26:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:12 smithi082 ceph-mon[98694]: pgmap v17653: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 777 KiB/s wr, 33 op/s 2023-09-10T08:26:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:14 smithi170 ceph-mon[102832]: pgmap v17654: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 633 KiB/s wr, 32 op/s 2023-09-10T08:26:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32804.565595 secs 2023-09-10T08:26:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:14 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:26:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:14 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32805.729060 secs 2023-09-10T08:26:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:14 smithi082 ceph-mon[98694]: pgmap v17654: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 633 KiB/s wr, 32 op/s 2023-09-10T08:26:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32804.565595 secs 2023-09-10T08:26:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:14 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:26:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:14 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32805.729060 secs 2023-09-10T08:26:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:14 smithi098 ceph-mon[103124]: pgmap v17654: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 633 KiB/s wr, 32 op/s 2023-09-10T08:26:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32804.565595 secs 2023-09-10T08:26:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:14 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:26:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:14 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32805.729060 secs 2023-09-10T08:26:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32806.172461 secs 2023-09-10T08:26:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32806.172461 secs 2023-09-10T08:26:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32806.172461 secs 2023-09-10T08:26:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:16 smithi170 ceph-mon[102832]: pgmap v17655: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 633 KiB/s wr, 32 op/s 2023-09-10T08:26:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:16 smithi082 ceph-mon[98694]: pgmap v17655: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 633 KiB/s wr, 32 op/s 2023-09-10T08:26:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:16 smithi098 ceph-mon[103124]: pgmap v17655: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 633 KiB/s wr, 32 op/s 2023-09-10T08:26:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:17 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:26:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:17 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:17 smithi170 ceph-mon[102832]: pgmap v17656: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 444 KiB/s wr, 14 op/s 2023-09-10T08:26:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:17 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:26:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:17 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:17 smithi082 ceph-mon[98694]: pgmap v17656: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 444 KiB/s wr, 14 op/s 2023-09-10T08:26:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:17 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:26:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:17 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:17 smithi098 ceph-mon[103124]: pgmap v17656: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 444 KiB/s wr, 14 op/s 2023-09-10T08:26:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:26:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:18 smithi170 ceph-mon[102832]: osdmap e1172: 12 total, 12 up, 12 in 2023-09-10T08:26:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:26:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:18 smithi082 ceph-mon[98694]: osdmap e1172: 12 total, 12 up, 12 in 2023-09-10T08:26:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:26:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:18 smithi098 ceph-mon[103124]: osdmap e1172: 12 total, 12 up, 12 in 2023-09-10T08:26:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32809.565659 secs 2023-09-10T08:26:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:19 smithi170 ceph-mon[102832]: pgmap v17658: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 482 KiB/s wr, 16 op/s 2023-09-10T08:26:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32810.729118 secs 2023-09-10T08:26:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32809.565659 secs 2023-09-10T08:26:19.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:19 smithi082 ceph-mon[98694]: pgmap v17658: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 482 KiB/s wr, 16 op/s 2023-09-10T08:26:19.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32810.729118 secs 2023-09-10T08:26:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32809.565659 secs 2023-09-10T08:26:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:19 smithi098 ceph-mon[103124]: pgmap v17658: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 482 KiB/s wr, 16 op/s 2023-09-10T08:26:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32810.729118 secs 2023-09-10T08:26:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32811.172489 secs 2023-09-10T08:26:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32811.172489 secs 2023-09-10T08:26:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32811.172489 secs 2023-09-10T08:26:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:21 smithi170 ceph-mon[102832]: pgmap v17659: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 444 KiB/s wr, 14 op/s 2023-09-10T08:26:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:21.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:21.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:21.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:21.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:21.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:21.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:21.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:26:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:21 smithi082 ceph-mon[98694]: pgmap v17659: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 444 KiB/s wr, 14 op/s 2023-09-10T08:26:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:21.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:21.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:21.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:21.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:21.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:21.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:26:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:21 smithi098 ceph-mon[103124]: pgmap v17659: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 444 KiB/s wr, 14 op/s 2023-09-10T08:26:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:21.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:21.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:21.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:21.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:26:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:26:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:26:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:26:22.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:26:22.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:22 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T08:26:22.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:22 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T08:26:22.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:22.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:22.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:26:22.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:26:22.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:26:22.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:26:22.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:22 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T08:26:22.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:22 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T08:26:22.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:26:22.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:26:22.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:26:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:26:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:26:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:26:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:22 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T08:26:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:22 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T08:26:22.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:22.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:22.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:26:22.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:26:22.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:26:22.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:26:22.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:22 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T08:26:22.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:22 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T08:26:22.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:26:22.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:26:22.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:26:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:26:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:26:22.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:26:22.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:22 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T08:26:22.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:22 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T08:26:22.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:22.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:22.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:26:22.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:26:22.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:26:22.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:26:22.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:22 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T08:26:22.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:22 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T08:26:22.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:26:22.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:26:22.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:26:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:23 smithi170 ceph-mon[102832]: pgmap v17660: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 13 op/s 2023-09-10T08:26:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32814.565721 secs 2023-09-10T08:26:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32815.729213 secs 2023-09-10T08:26:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:23 smithi082 ceph-mon[98694]: pgmap v17660: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 13 op/s 2023-09-10T08:26:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32814.565721 secs 2023-09-10T08:26:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32815.729213 secs 2023-09-10T08:26:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:23 smithi098 ceph-mon[103124]: pgmap v17660: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 13 op/s 2023-09-10T08:26:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32814.565721 secs 2023-09-10T08:26:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32815.729213 secs 2023-09-10T08:26:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32816.172550 secs 2023-09-10T08:26:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32816.172550 secs 2023-09-10T08:26:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32816.172550 secs 2023-09-10T08:26:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:25 smithi170 ceph-mon[102832]: pgmap v17661: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 13 op/s 2023-09-10T08:26:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:25 smithi082 ceph-mon[98694]: pgmap v17661: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 13 op/s 2023-09-10T08:26:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:25 smithi098 ceph-mon[103124]: pgmap v17661: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 13 op/s 2023-09-10T08:26:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:27 smithi170 ceph-mon[102832]: pgmap v17662: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T08:26:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:27 smithi082 ceph-mon[98694]: pgmap v17662: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T08:26:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:27 smithi098 ceph-mon[103124]: pgmap v17662: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T08:26:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32819.565781 secs 2023-09-10T08:26:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32820.729296 secs 2023-09-10T08:26:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32819.565781 secs 2023-09-10T08:26:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32820.729296 secs 2023-09-10T08:26:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32819.565781 secs 2023-09-10T08:26:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32820.729296 secs 2023-09-10T08:26:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:29 smithi170 ceph-mon[102832]: pgmap v17663: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T08:26:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32821.172619 secs 2023-09-10T08:26:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:29 smithi082 ceph-mon[98694]: pgmap v17663: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T08:26:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32821.172619 secs 2023-09-10T08:26:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:29 smithi098 ceph-mon[103124]: pgmap v17663: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 12 op/s 2023-09-10T08:26:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32821.172619 secs 2023-09-10T08:26:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:31 smithi170 ceph-mon[102832]: pgmap v17664: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 10 op/s 2023-09-10T08:26:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:31 smithi082 ceph-mon[98694]: pgmap v17664: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 10 op/s 2023-09-10T08:26:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:31 smithi098 ceph-mon[103124]: pgmap v17664: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 10 op/s 2023-09-10T08:26:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:33 smithi170 ceph-mon[102832]: pgmap v17665: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 151 KiB/s wr, 16 op/s 2023-09-10T08:26:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32825.729354 secs 2023-09-10T08:26:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:33 smithi082 ceph-mon[98694]: pgmap v17665: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 151 KiB/s wr, 16 op/s 2023-09-10T08:26:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32825.729354 secs 2023-09-10T08:26:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:33 smithi098 ceph-mon[103124]: pgmap v17665: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 151 KiB/s wr, 16 op/s 2023-09-10T08:26:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32825.729354 secs 2023-09-10T08:26:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32826.172685 secs 2023-09-10T08:26:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32826.172685 secs 2023-09-10T08:26:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32826.172685 secs 2023-09-10T08:26:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:35 smithi170 ceph-mon[102832]: pgmap v17666: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 126 KiB/s wr, 10 op/s 2023-09-10T08:26:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:26:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:35 smithi082 ceph-mon[98694]: pgmap v17666: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 126 KiB/s wr, 10 op/s 2023-09-10T08:26:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:26:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:35 smithi098 ceph-mon[103124]: pgmap v17666: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 126 KiB/s wr, 10 op/s 2023-09-10T08:26:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:26:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:26:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:26:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:26:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:26:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:26:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:26:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:37 smithi170 ceph-mon[102832]: pgmap v17667: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 233 KiB/s wr, 16 op/s 2023-09-10T08:26:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:26:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:37 smithi082 ceph-mon[98694]: pgmap v17667: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 233 KiB/s wr, 16 op/s 2023-09-10T08:26:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:26:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:37 smithi098 ceph-mon[103124]: pgmap v17667: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 233 KiB/s wr, 16 op/s 2023-09-10T08:26:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:26:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32830.729414 secs 2023-09-10T08:26:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32830.729414 secs 2023-09-10T08:26:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32830.729414 secs 2023-09-10T08:26:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:39 smithi170 ceph-mon[102832]: pgmap v17668: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 233 KiB/s wr, 11 op/s 2023-09-10T08:26:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32831.172750 secs 2023-09-10T08:26:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:39 smithi082 ceph-mon[98694]: pgmap v17668: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 233 KiB/s wr, 11 op/s 2023-09-10T08:26:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32831.172750 secs 2023-09-10T08:26:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:39 smithi098 ceph-mon[103124]: pgmap v17668: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 233 KiB/s wr, 11 op/s 2023-09-10T08:26:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32831.172750 secs 2023-09-10T08:26:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:26:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:26:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:26:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:41 smithi170 ceph-mon[102832]: pgmap v17669: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 233 KiB/s wr, 11 op/s 2023-09-10T08:26:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:41 smithi082 ceph-mon[98694]: pgmap v17669: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 233 KiB/s wr, 11 op/s 2023-09-10T08:26:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:41 smithi098 ceph-mon[103124]: pgmap v17669: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 233 KiB/s wr, 11 op/s 2023-09-10T08:26:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:43 smithi170 ceph-mon[102832]: pgmap v17670: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 858 KiB/s wr, 19 op/s 2023-09-10T08:26:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32835.729495 secs 2023-09-10T08:26:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:43 smithi082 ceph-mon[98694]: pgmap v17670: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 858 KiB/s wr, 19 op/s 2023-09-10T08:26:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32835.729495 secs 2023-09-10T08:26:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:43 smithi098 ceph-mon[103124]: pgmap v17670: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 858 KiB/s wr, 19 op/s 2023-09-10T08:26:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32835.729495 secs 2023-09-10T08:26:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32836.172815 secs 2023-09-10T08:26:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:26:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32836.172815 secs 2023-09-10T08:26:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:26:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32836.172815 secs 2023-09-10T08:26:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:26:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:45 smithi082 ceph-mon[98694]: pgmap v17671: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 732 KiB/s wr, 13 op/s 2023-09-10T08:26:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:45 smithi098 ceph-mon[103124]: pgmap v17671: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 732 KiB/s wr, 13 op/s 2023-09-10T08:26:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:45 smithi170 ceph-mon[102832]: pgmap v17671: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 732 KiB/s wr, 13 op/s 2023-09-10T08:26:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:47 smithi082 ceph-mon[98694]: pgmap v17672: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 751 KiB/s wr, 21 op/s 2023-09-10T08:26:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:47 smithi098 ceph-mon[103124]: pgmap v17672: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 751 KiB/s wr, 21 op/s 2023-09-10T08:26:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:47 smithi170 ceph-mon[102832]: pgmap v17672: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 751 KiB/s wr, 21 op/s 2023-09-10T08:26:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32839.566117 secs 2023-09-10T08:26:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32840.729554 secs 2023-09-10T08:26:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32839.566117 secs 2023-09-10T08:26:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32840.729554 secs 2023-09-10T08:26:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32839.566117 secs 2023-09-10T08:26:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32840.729554 secs 2023-09-10T08:26:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:49 smithi082 ceph-mon[98694]: pgmap v17673: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 644 KiB/s wr, 15 op/s 2023-09-10T08:26:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32841.172882 secs 2023-09-10T08:26:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:26:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:49 smithi098 ceph-mon[103124]: pgmap v17673: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 644 KiB/s wr, 15 op/s 2023-09-10T08:26:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32841.172882 secs 2023-09-10T08:26:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:26:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:49 smithi170 ceph-mon[102832]: pgmap v17673: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 644 KiB/s wr, 15 op/s 2023-09-10T08:26:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32841.172882 secs 2023-09-10T08:26:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:26:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:51 smithi082 ceph-mon[98694]: pgmap v17674: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 644 KiB/s wr, 15 op/s 2023-09-10T08:26:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:51 smithi098 ceph-mon[103124]: pgmap v17674: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 644 KiB/s wr, 15 op/s 2023-09-10T08:26:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:51 smithi170 ceph-mon[102832]: pgmap v17674: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 644 KiB/s wr, 15 op/s 2023-09-10T08:26:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:26:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:26:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:26:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:26:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:26:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:26:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:53 smithi098 ceph-mon[103124]: pgmap v17675: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 23 op/s 2023-09-10T08:26:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32844.566178 secs 2023-09-10T08:26:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:26:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32845.729618 secs 2023-09-10T08:26:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:53 smithi170 ceph-mon[102832]: pgmap v17675: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 23 op/s 2023-09-10T08:26:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32844.566178 secs 2023-09-10T08:26:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:26:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32845.729618 secs 2023-09-10T08:26:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:53 smithi082 ceph-mon[98694]: pgmap v17675: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 23 op/s 2023-09-10T08:26:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32844.566178 secs 2023-09-10T08:26:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:26:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32845.729618 secs 2023-09-10T08:26:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32846.172950 secs 2023-09-10T08:26:55.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32846.172950 secs 2023-09-10T08:26:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32846.172950 secs 2023-09-10T08:26:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:55 smithi098 ceph-mon[103124]: pgmap v17676: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 432 KiB/s wr, 15 op/s 2023-09-10T08:26:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:55 smithi170 ceph-mon[102832]: pgmap v17676: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 432 KiB/s wr, 15 op/s 2023-09-10T08:26:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:55 smithi082 ceph-mon[98694]: pgmap v17676: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 432 KiB/s wr, 15 op/s 2023-09-10T08:26:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:26:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:26:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:26:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:57 smithi098 ceph-mon[103124]: pgmap v17677: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 844 KiB/s wr, 22 op/s 2023-09-10T08:26:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:26:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:57 smithi170 ceph-mon[102832]: pgmap v17677: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 844 KiB/s wr, 22 op/s 2023-09-10T08:26:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:26:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:57 smithi082 ceph-mon[98694]: pgmap v17677: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 844 KiB/s wr, 22 op/s 2023-09-10T08:26:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:26:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:26:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32849.566239 secs 2023-09-10T08:26:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32850.729676 secs 2023-09-10T08:26:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32849.566239 secs 2023-09-10T08:26:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32850.729676 secs 2023-09-10T08:26:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32849.566239 secs 2023-09-10T08:26:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32850.729676 secs 2023-09-10T08:27:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:59 smithi098 ceph-mon[103124]: pgmap v17678: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 826 KiB/s wr, 14 op/s 2023-09-10T08:27:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:26:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32851.173018 secs 2023-09-10T08:27:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:59 smithi170 ceph-mon[102832]: pgmap v17678: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 826 KiB/s wr, 14 op/s 2023-09-10T08:27:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:26:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32851.173018 secs 2023-09-10T08:27:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:59 smithi082 ceph-mon[98694]: pgmap v17678: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 826 KiB/s wr, 14 op/s 2023-09-10T08:27:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:26:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32851.173018 secs 2023-09-10T08:27:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:27:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:27:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:27:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:01 smithi098 ceph-mon[103124]: pgmap v17679: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 826 KiB/s wr, 14 op/s 2023-09-10T08:27:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:01 smithi170 ceph-mon[102832]: pgmap v17679: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 826 KiB/s wr, 14 op/s 2023-09-10T08:27:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:01 smithi082 ceph-mon[98694]: pgmap v17679: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 826 KiB/s wr, 14 op/s 2023-09-10T08:27:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:03 smithi098 ceph-mon[103124]: pgmap v17680: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 918 KiB/s wr, 21 op/s 2023-09-10T08:27:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32854.566301 secs 2023-09-10T08:27:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32855.729733 secs 2023-09-10T08:27:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:03 smithi170 ceph-mon[102832]: pgmap v17680: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 918 KiB/s wr, 21 op/s 2023-09-10T08:27:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32854.566301 secs 2023-09-10T08:27:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32855.729733 secs 2023-09-10T08:27:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:03 smithi082 ceph-mon[98694]: pgmap v17680: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 918 KiB/s wr, 21 op/s 2023-09-10T08:27:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32854.566301 secs 2023-09-10T08:27:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32855.729733 secs 2023-09-10T08:27:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32856.173081 secs 2023-09-10T08:27:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32856.173081 secs 2023-09-10T08:27:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32856.173081 secs 2023-09-10T08:27:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:06 smithi098 ceph-mon[103124]: pgmap v17681: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 504 KiB/s wr, 13 op/s 2023-09-10T08:27:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:06 smithi170 ceph-mon[102832]: pgmap v17681: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 504 KiB/s wr, 13 op/s 2023-09-10T08:27:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:06 smithi082 ceph-mon[98694]: pgmap v17681: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 504 KiB/s wr, 13 op/s 2023-09-10T08:27:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:08 smithi170 ceph-mon[102832]: pgmap v17682: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 802 KiB/s wr, 33 op/s 2023-09-10T08:27:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:08 smithi082 ceph-mon[98694]: pgmap v17682: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 802 KiB/s wr, 33 op/s 2023-09-10T08:27:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:08 smithi098 ceph-mon[103124]: pgmap v17682: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 802 KiB/s wr, 33 op/s 2023-09-10T08:27:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32859.566355 secs 2023-09-10T08:27:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32860.729789 secs 2023-09-10T08:27:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:09 smithi170 ceph-mon[102832]: osdmap e1173: 12 total, 12 up, 12 in 2023-09-10T08:27:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32859.566355 secs 2023-09-10T08:27:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32860.729789 secs 2023-09-10T08:27:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:09 smithi082 ceph-mon[98694]: osdmap e1173: 12 total, 12 up, 12 in 2023-09-10T08:27:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32859.566355 secs 2023-09-10T08:27:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32860.729789 secs 2023-09-10T08:27:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:09 smithi098 ceph-mon[103124]: osdmap e1173: 12 total, 12 up, 12 in 2023-09-10T08:27:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:10 smithi170 ceph-mon[102832]: pgmap v17683: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 391 KiB/s wr, 26 op/s 2023-09-10T08:27:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32861.173150 secs 2023-09-10T08:27:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:10 smithi082 ceph-mon[98694]: pgmap v17683: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 391 KiB/s wr, 26 op/s 2023-09-10T08:27:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32861.173150 secs 2023-09-10T08:27:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:10 smithi098 ceph-mon[103124]: pgmap v17683: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 391 KiB/s wr, 26 op/s 2023-09-10T08:27:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32861.173150 secs 2023-09-10T08:27:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:12 smithi170 ceph-mon[102832]: pgmap v17685: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 469 KiB/s wr, 32 op/s 2023-09-10T08:27:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:12 smithi082 ceph-mon[98694]: pgmap v17685: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 469 KiB/s wr, 32 op/s 2023-09-10T08:27:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:12 smithi098 ceph-mon[103124]: pgmap v17685: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 469 KiB/s wr, 32 op/s 2023-09-10T08:27:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:14 smithi170 ceph-mon[102832]: pgmap v17686: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 499 KiB/s wr, 31 op/s 2023-09-10T08:27:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32864.566454 secs 2023-09-10T08:27:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:14 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:27:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:14 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32865.729847 secs 2023-09-10T08:27:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:14 smithi082 ceph-mon[98694]: pgmap v17686: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 499 KiB/s wr, 31 op/s 2023-09-10T08:27:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32864.566454 secs 2023-09-10T08:27:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:14 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:27:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:14 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32865.729847 secs 2023-09-10T08:27:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:14 smithi098 ceph-mon[103124]: pgmap v17686: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 499 KiB/s wr, 31 op/s 2023-09-10T08:27:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32864.566454 secs 2023-09-10T08:27:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:14 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:27:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:14 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32865.729847 secs 2023-09-10T08:27:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32866.173214 secs 2023-09-10T08:27:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32866.173214 secs 2023-09-10T08:27:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32866.173214 secs 2023-09-10T08:27:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:16 smithi170 ceph-mon[102832]: pgmap v17687: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 499 KiB/s wr, 31 op/s 2023-09-10T08:27:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:16 smithi082 ceph-mon[98694]: pgmap v17687: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 499 KiB/s wr, 31 op/s 2023-09-10T08:27:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:16 smithi098 ceph-mon[103124]: pgmap v17687: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 499 KiB/s wr, 31 op/s 2023-09-10T08:27:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:17 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:27:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:17 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:17 smithi170 ceph-mon[102832]: pgmap v17688: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 172 KiB/s wr, 14 op/s 2023-09-10T08:27:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:17 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:27:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:17 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:17 smithi082 ceph-mon[98694]: pgmap v17688: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 172 KiB/s wr, 14 op/s 2023-09-10T08:27:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:17 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:27:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:17 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:17 smithi098 ceph-mon[103124]: pgmap v17688: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 172 KiB/s wr, 14 op/s 2023-09-10T08:27:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32869.566518 secs 2023-09-10T08:27:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:18 smithi170 ceph-mon[102832]: osdmap e1174: 12 total, 12 up, 12 in 2023-09-10T08:27:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32870.729905 secs 2023-09-10T08:27:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32869.566518 secs 2023-09-10T08:27:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:18 smithi082 ceph-mon[98694]: osdmap e1174: 12 total, 12 up, 12 in 2023-09-10T08:27:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32870.729905 secs 2023-09-10T08:27:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32869.566518 secs 2023-09-10T08:27:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:18 smithi098 ceph-mon[103124]: osdmap e1174: 12 total, 12 up, 12 in 2023-09-10T08:27:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32870.729905 secs 2023-09-10T08:27:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:19 smithi170 ceph-mon[102832]: pgmap v17690: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 186 KiB/s wr, 15 op/s 2023-09-10T08:27:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32871.173279 secs 2023-09-10T08:27:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:19 smithi082 ceph-mon[98694]: pgmap v17690: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 186 KiB/s wr, 15 op/s 2023-09-10T08:27:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32871.173279 secs 2023-09-10T08:27:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:19 smithi098 ceph-mon[103124]: pgmap v17690: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 186 KiB/s wr, 15 op/s 2023-09-10T08:27:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32871.173279 secs 2023-09-10T08:27:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:21 smithi170 ceph-mon[102832]: pgmap v17691: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 172 KiB/s wr, 14 op/s 2023-09-10T08:27:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:27:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:21 smithi082 ceph-mon[98694]: pgmap v17691: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 172 KiB/s wr, 14 op/s 2023-09-10T08:27:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:27:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:21 smithi098 ceph-mon[103124]: pgmap v17691: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 172 KiB/s wr, 14 op/s 2023-09-10T08:27:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:27:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:27:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:27:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:27:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:27:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:27:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:27:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:27:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:27:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:27:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:23 smithi170 ceph-mon[102832]: pgmap v17692: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:27:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32874.566626 secs 2023-09-10T08:27:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32875.729963 secs 2023-09-10T08:27:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:23 smithi082 ceph-mon[98694]: pgmap v17692: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:27:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32874.566626 secs 2023-09-10T08:27:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32875.729963 secs 2023-09-10T08:27:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:23 smithi098 ceph-mon[103124]: pgmap v17692: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:27:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32874.566626 secs 2023-09-10T08:27:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32875.729963 secs 2023-09-10T08:27:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32876.173346 secs 2023-09-10T08:27:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32876.173346 secs 2023-09-10T08:27:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32876.173346 secs 2023-09-10T08:27:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:25 smithi170 ceph-mon[102832]: pgmap v17693: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:27:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:25 smithi082 ceph-mon[98694]: pgmap v17693: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:27:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:25 smithi098 ceph-mon[103124]: pgmap v17693: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:27:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:27 smithi170 ceph-mon[102832]: pgmap v17694: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 13 op/s 2023-09-10T08:27:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:27 smithi082 ceph-mon[98694]: pgmap v17694: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 13 op/s 2023-09-10T08:27:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:27 smithi098 ceph-mon[103124]: pgmap v17694: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 13 op/s 2023-09-10T08:27:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32879.566687 secs 2023-09-10T08:27:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32880.730024 secs 2023-09-10T08:27:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32879.566687 secs 2023-09-10T08:27:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32880.730024 secs 2023-09-10T08:27:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32879.566687 secs 2023-09-10T08:27:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32880.730024 secs 2023-09-10T08:27:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:29 smithi170 ceph-mon[102832]: pgmap v17695: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 13 op/s 2023-09-10T08:27:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32881.173425 secs 2023-09-10T08:27:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:29 smithi082 ceph-mon[98694]: pgmap v17695: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 13 op/s 2023-09-10T08:27:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32881.173425 secs 2023-09-10T08:27:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:29 smithi098 ceph-mon[103124]: pgmap v17695: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 13 op/s 2023-09-10T08:27:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32881.173425 secs 2023-09-10T08:27:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:31 smithi170 ceph-mon[102832]: pgmap v17696: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T08:27:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:31 smithi082 ceph-mon[98694]: pgmap v17696: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T08:27:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:31 smithi098 ceph-mon[103124]: pgmap v17696: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T08:27:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:33 smithi170 ceph-mon[102832]: pgmap v17697: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 161 KiB/s wr, 16 op/s 2023-09-10T08:27:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:27:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32885.730086 secs 2023-09-10T08:27:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:33 smithi082 ceph-mon[98694]: pgmap v17697: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 161 KiB/s wr, 16 op/s 2023-09-10T08:27:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:27:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32885.730086 secs 2023-09-10T08:27:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:33 smithi098 ceph-mon[103124]: pgmap v17697: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 161 KiB/s wr, 16 op/s 2023-09-10T08:27:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:27:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32885.730086 secs 2023-09-10T08:27:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32886.173484 secs 2023-09-10T08:27:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32886.173484 secs 2023-09-10T08:27:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32886.173484 secs 2023-09-10T08:27:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:35 smithi170 ceph-mon[102832]: pgmap v17698: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 160 KiB/s wr, 10 op/s 2023-09-10T08:27:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:27:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:35 smithi082 ceph-mon[98694]: pgmap v17698: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 160 KiB/s wr, 10 op/s 2023-09-10T08:27:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:27:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:35 smithi098 ceph-mon[103124]: pgmap v17698: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 160 KiB/s wr, 10 op/s 2023-09-10T08:27:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:27:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:27:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:27:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:27:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:27:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:27:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:27:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:37 smithi082 ceph-mon[98694]: pgmap v17699: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 308 KiB/s wr, 16 op/s 2023-09-10T08:27:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:37 smithi098 ceph-mon[103124]: pgmap v17699: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 308 KiB/s wr, 16 op/s 2023-09-10T08:27:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:37 smithi170 ceph-mon[102832]: pgmap v17699: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 308 KiB/s wr, 16 op/s 2023-09-10T08:27:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32890.730144 secs 2023-09-10T08:27:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32890.730144 secs 2023-09-10T08:27:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32890.730144 secs 2023-09-10T08:27:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:39 smithi082 ceph-mon[98694]: pgmap v17700: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 190 KiB/s wr, 11 op/s 2023-09-10T08:27:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32891.173548 secs 2023-09-10T08:27:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:39 smithi098 ceph-mon[103124]: pgmap v17700: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 190 KiB/s wr, 11 op/s 2023-09-10T08:27:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32891.173548 secs 2023-09-10T08:27:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:39 smithi170 ceph-mon[102832]: pgmap v17700: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 190 KiB/s wr, 11 op/s 2023-09-10T08:27:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32891.173548 secs 2023-09-10T08:27:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:41 smithi082 ceph-mon[98694]: pgmap v17701: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 190 KiB/s wr, 11 op/s 2023-09-10T08:27:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:41 smithi098 ceph-mon[103124]: pgmap v17701: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 190 KiB/s wr, 11 op/s 2023-09-10T08:27:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:41 smithi170 ceph-mon[102832]: pgmap v17701: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 190 KiB/s wr, 11 op/s 2023-09-10T08:27:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:43 smithi082 ceph-mon[98694]: pgmap v17702: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 461 KiB/s wr, 18 op/s 2023-09-10T08:27:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32895.730202 secs 2023-09-10T08:27:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:43 smithi098 ceph-mon[103124]: pgmap v17702: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 461 KiB/s wr, 18 op/s 2023-09-10T08:27:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32895.730202 secs 2023-09-10T08:27:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:43 smithi170 ceph-mon[102832]: pgmap v17702: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 461 KiB/s wr, 18 op/s 2023-09-10T08:27:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32895.730202 secs 2023-09-10T08:27:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32896.173614 secs 2023-09-10T08:27:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:27:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32896.173614 secs 2023-09-10T08:27:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:27:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32896.173614 secs 2023-09-10T08:27:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:27:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:45 smithi082 ceph-mon[98694]: pgmap v17703: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 419 KiB/s wr, 13 op/s 2023-09-10T08:27:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:27:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:45 smithi098 ceph-mon[103124]: pgmap v17703: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 419 KiB/s wr, 13 op/s 2023-09-10T08:27:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:27:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:45 smithi170 ceph-mon[102832]: pgmap v17703: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 419 KiB/s wr, 13 op/s 2023-09-10T08:27:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:27:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:47 smithi082 ceph-mon[98694]: pgmap v17704: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 632 KiB/s wr, 20 op/s 2023-09-10T08:27:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:47 smithi098 ceph-mon[103124]: pgmap v17704: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 632 KiB/s wr, 20 op/s 2023-09-10T08:27:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:47 smithi170 ceph-mon[102832]: pgmap v17704: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 632 KiB/s wr, 20 op/s 2023-09-10T08:27:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32900.730261 secs 2023-09-10T08:27:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32900.730261 secs 2023-09-10T08:27:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32900.730261 secs 2023-09-10T08:27:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:49 smithi098 ceph-mon[103124]: pgmap v17705: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 484 KiB/s wr, 13 op/s 2023-09-10T08:27:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32901.173677 secs 2023-09-10T08:27:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:49 smithi170 ceph-mon[102832]: pgmap v17705: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 484 KiB/s wr, 13 op/s 2023-09-10T08:27:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32901.173677 secs 2023-09-10T08:27:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:49 smithi082 ceph-mon[98694]: pgmap v17705: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 484 KiB/s wr, 13 op/s 2023-09-10T08:27:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32901.173677 secs 2023-09-10T08:27:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:51 smithi098 ceph-mon[103124]: pgmap v17706: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 484 KiB/s wr, 13 op/s 2023-09-10T08:27:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:51 smithi170 ceph-mon[102832]: pgmap v17706: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 484 KiB/s wr, 13 op/s 2023-09-10T08:27:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:51 smithi082 ceph-mon[98694]: pgmap v17706: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 484 KiB/s wr, 13 op/s 2023-09-10T08:27:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:27:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:27:53.549 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:53.550 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:27:53.550 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:53 smithi098 ceph-mon[103124]: pgmap v17707: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 871 KiB/s wr, 21 op/s 2023-09-10T08:27:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32904.567017 secs 2023-09-10T08:27:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:27:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:54.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32905.730320 secs 2023-09-10T08:27:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:53 smithi170 ceph-mon[102832]: pgmap v17707: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 871 KiB/s wr, 21 op/s 2023-09-10T08:27:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32904.567017 secs 2023-09-10T08:27:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:27:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32905.730320 secs 2023-09-10T08:27:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:53 smithi082 ceph-mon[98694]: pgmap v17707: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 871 KiB/s wr, 21 op/s 2023-09-10T08:27:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32904.567017 secs 2023-09-10T08:27:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:27:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32905.730320 secs 2023-09-10T08:27:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32906.173741 secs 2023-09-10T08:27:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32906.173741 secs 2023-09-10T08:27:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32906.173741 secs 2023-09-10T08:27:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:55 smithi098 ceph-mon[103124]: pgmap v17708: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 599 KiB/s wr, 14 op/s 2023-09-10T08:27:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:55 smithi170 ceph-mon[102832]: pgmap v17708: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 599 KiB/s wr, 14 op/s 2023-09-10T08:27:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:55 smithi082 ceph-mon[98694]: pgmap v17708: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 599 KiB/s wr, 14 op/s 2023-09-10T08:27:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:27:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:27:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:57.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:27:57.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:27:57.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:27:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:27:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:27:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:57 smithi098 ceph-mon[103124]: pgmap v17709: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 825 KiB/s wr, 21 op/s 2023-09-10T08:27:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:57 smithi170 ceph-mon[102832]: pgmap v17709: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 825 KiB/s wr, 21 op/s 2023-09-10T08:27:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:57 smithi082 ceph-mon[98694]: pgmap v17709: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 825 KiB/s wr, 21 op/s 2023-09-10T08:27:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32909.567077 secs 2023-09-10T08:27:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:27:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32910.730378 secs 2023-09-10T08:27:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32909.567077 secs 2023-09-10T08:27:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:27:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32910.730378 secs 2023-09-10T08:27:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32909.567077 secs 2023-09-10T08:27:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:27:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32910.730378 secs 2023-09-10T08:28:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:00 smithi098 ceph-mon[103124]: pgmap v17710: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 612 KiB/s wr, 14 op/s 2023-09-10T08:28:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32911.173806 secs 2023-09-10T08:28:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:00 smithi170 ceph-mon[102832]: pgmap v17710: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 612 KiB/s wr, 14 op/s 2023-09-10T08:28:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32911.173806 secs 2023-09-10T08:28:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:00 smithi082 ceph-mon[98694]: pgmap v17710: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 612 KiB/s wr, 14 op/s 2023-09-10T08:28:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32911.173806 secs 2023-09-10T08:28:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:02 smithi170 ceph-mon[102832]: pgmap v17711: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 612 KiB/s wr, 14 op/s 2023-09-10T08:28:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:02 smithi082 ceph-mon[98694]: pgmap v17711: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 612 KiB/s wr, 14 op/s 2023-09-10T08:28:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:02 smithi098 ceph-mon[103124]: pgmap v17711: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 612 KiB/s wr, 14 op/s 2023-09-10T08:28:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:04 smithi170 ceph-mon[102832]: pgmap v17712: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 778 KiB/s wr, 21 op/s 2023-09-10T08:28:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32914.567140 secs 2023-09-10T08:28:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32915.730435 secs 2023-09-10T08:28:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:04 smithi082 ceph-mon[98694]: pgmap v17712: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 778 KiB/s wr, 21 op/s 2023-09-10T08:28:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32914.567140 secs 2023-09-10T08:28:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32915.730435 secs 2023-09-10T08:28:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:04 smithi098 ceph-mon[103124]: pgmap v17712: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 778 KiB/s wr, 21 op/s 2023-09-10T08:28:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32914.567140 secs 2023-09-10T08:28:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32915.730435 secs 2023-09-10T08:28:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32916.173872 secs 2023-09-10T08:28:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:28:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:28:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32916.173872 secs 2023-09-10T08:28:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:28:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:28:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32916.173872 secs 2023-09-10T08:28:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:28:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:28:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:06 smithi170 ceph-mon[102832]: pgmap v17713: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 391 KiB/s wr, 13 op/s 2023-09-10T08:28:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:06 smithi082 ceph-mon[98694]: pgmap v17713: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 391 KiB/s wr, 13 op/s 2023-09-10T08:28:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:06 smithi098 ceph-mon[103124]: pgmap v17713: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 391 KiB/s wr, 13 op/s 2023-09-10T08:28:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:08 smithi170 ceph-mon[102832]: pgmap v17714: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 550 KiB/s wr, 33 op/s 2023-09-10T08:28:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:08 smithi082 ceph-mon[98694]: pgmap v17714: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 550 KiB/s wr, 33 op/s 2023-09-10T08:28:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:08 smithi098 ceph-mon[103124]: pgmap v17714: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 550 KiB/s wr, 33 op/s 2023-09-10T08:28:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32919.567279 secs 2023-09-10T08:28:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:09 smithi170 ceph-mon[102832]: osdmap e1175: 12 total, 12 up, 12 in 2023-09-10T08:28:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32920.730492 secs 2023-09-10T08:28:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32919.567279 secs 2023-09-10T08:28:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:09 smithi082 ceph-mon[98694]: osdmap e1175: 12 total, 12 up, 12 in 2023-09-10T08:28:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32920.730492 secs 2023-09-10T08:28:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32919.567279 secs 2023-09-10T08:28:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:09 smithi098 ceph-mon[103124]: osdmap e1175: 12 total, 12 up, 12 in 2023-09-10T08:28:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32920.730492 secs 2023-09-10T08:28:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:10 smithi170 ceph-mon[102832]: pgmap v17716: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 389 KiB/s wr, 31 op/s 2023-09-10T08:28:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32921.173939 secs 2023-09-10T08:28:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:28:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:28:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:10 smithi082 ceph-mon[98694]: pgmap v17716: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 389 KiB/s wr, 31 op/s 2023-09-10T08:28:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32921.173939 secs 2023-09-10T08:28:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:28:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:28:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:10 smithi098 ceph-mon[103124]: pgmap v17716: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 389 KiB/s wr, 31 op/s 2023-09-10T08:28:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32921.173939 secs 2023-09-10T08:28:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:28:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:28:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:11 smithi170 ceph-mon[102832]: pgmap v17717: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 389 KiB/s wr, 31 op/s 2023-09-10T08:28:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:11 smithi082 ceph-mon[98694]: pgmap v17717: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 389 KiB/s wr, 31 op/s 2023-09-10T08:28:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:11 smithi098 ceph-mon[103124]: pgmap v17717: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 389 KiB/s wr, 31 op/s 2023-09-10T08:28:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:13 smithi170 ceph-mon[102832]: pgmap v17718: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 288 KiB/s wr, 31 op/s 2023-09-10T08:28:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32924.567340 secs 2023-09-10T08:28:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:13 smithi170 ceph-mon[102832]: osdmap e1176: 12 total, 12 up, 12 in 2023-09-10T08:28:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32925.730549 secs 2023-09-10T08:28:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:13 smithi082 ceph-mon[98694]: pgmap v17718: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 288 KiB/s wr, 31 op/s 2023-09-10T08:28:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32924.567340 secs 2023-09-10T08:28:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:13 smithi082 ceph-mon[98694]: osdmap e1176: 12 total, 12 up, 12 in 2023-09-10T08:28:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32925.730549 secs 2023-09-10T08:28:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:13 smithi098 ceph-mon[103124]: pgmap v17718: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 288 KiB/s wr, 31 op/s 2023-09-10T08:28:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32924.567340 secs 2023-09-10T08:28:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:13 smithi098 ceph-mon[103124]: osdmap e1176: 12 total, 12 up, 12 in 2023-09-10T08:28:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32925.730549 secs 2023-09-10T08:28:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32926.174004 secs 2023-09-10T08:28:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32926.174004 secs 2023-09-10T08:28:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32926.174004 secs 2023-09-10T08:28:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:15 smithi170 ceph-mon[102832]: pgmap v17720: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 9 op/s 2023-09-10T08:28:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:15 smithi082 ceph-mon[98694]: pgmap v17720: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 9 op/s 2023-09-10T08:28:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:15 smithi098 ceph-mon[103124]: pgmap v17720: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 9 op/s 2023-09-10T08:28:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:17 smithi170 ceph-mon[102832]: pgmap v17721: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 159 KiB/s wr, 17 op/s 2023-09-10T08:28:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:17 smithi082 ceph-mon[98694]: pgmap v17721: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 159 KiB/s wr, 17 op/s 2023-09-10T08:28:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:17 smithi098 ceph-mon[103124]: pgmap v17721: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 159 KiB/s wr, 17 op/s 2023-09-10T08:28:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32929.567430 secs 2023-09-10T08:28:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32930.731269 secs 2023-09-10T08:28:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32929.567430 secs 2023-09-10T08:28:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32930.731269 secs 2023-09-10T08:28:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32929.567430 secs 2023-09-10T08:28:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32930.731269 secs 2023-09-10T08:28:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:19 smithi170 ceph-mon[102832]: pgmap v17722: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 14 op/s 2023-09-10T08:28:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32931.174072 secs 2023-09-10T08:28:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:19 smithi082 ceph-mon[98694]: pgmap v17722: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 14 op/s 2023-09-10T08:28:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32931.174072 secs 2023-09-10T08:28:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:19 smithi098 ceph-mon[103124]: pgmap v17722: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 14 op/s 2023-09-10T08:28:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32931.174072 secs 2023-09-10T08:28:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:21 smithi170 ceph-mon[102832]: pgmap v17723: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 14 op/s 2023-09-10T08:28:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:21 smithi082 ceph-mon[98694]: pgmap v17723: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 14 op/s 2023-09-10T08:28:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:21 smithi098 ceph-mon[103124]: pgmap v17723: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 14 op/s 2023-09-10T08:28:23.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:28:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:28:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:28:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:23 smithi098 ceph-mon[103124]: pgmap v17724: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:28:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32934.567515 secs 2023-09-10T08:28:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:28:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:28:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32935.731325 secs 2023-09-10T08:28:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:28:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:28:24.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:28:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:23 smithi170 ceph-mon[102832]: pgmap v17724: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:28:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32934.567515 secs 2023-09-10T08:28:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:28:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:28:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32935.731325 secs 2023-09-10T08:28:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:28:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:28:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:28:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:23 smithi082 ceph-mon[98694]: pgmap v17724: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:28:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32934.567515 secs 2023-09-10T08:28:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:28:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:28:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32935.731325 secs 2023-09-10T08:28:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:28:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:28:24.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:28:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32936.174136 secs 2023-09-10T08:28:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32936.174136 secs 2023-09-10T08:28:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32936.174136 secs 2023-09-10T08:28:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:25 smithi098 ceph-mon[103124]: pgmap v17725: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 12 op/s 2023-09-10T08:28:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:25 smithi170 ceph-mon[102832]: pgmap v17725: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 12 op/s 2023-09-10T08:28:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:25 smithi082 ceph-mon[98694]: pgmap v17725: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 12 op/s 2023-09-10T08:28:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:27 smithi098 ceph-mon[103124]: pgmap v17726: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 17 op/s 2023-09-10T08:28:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:27 smithi170 ceph-mon[102832]: pgmap v17726: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 17 op/s 2023-09-10T08:28:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:27 smithi082 ceph-mon[98694]: pgmap v17726: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 17 op/s 2023-09-10T08:28:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32939.567577 secs 2023-09-10T08:28:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32940.731410 secs 2023-09-10T08:28:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32939.567577 secs 2023-09-10T08:28:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32940.731410 secs 2023-09-10T08:28:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32939.567577 secs 2023-09-10T08:28:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32940.731410 secs 2023-09-10T08:28:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:30 smithi098 ceph-mon[103124]: pgmap v17727: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T08:28:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32941.174202 secs 2023-09-10T08:28:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:30 smithi170 ceph-mon[102832]: pgmap v17727: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T08:28:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32941.174202 secs 2023-09-10T08:28:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:30 smithi082 ceph-mon[98694]: pgmap v17727: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T08:28:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32941.174202 secs 2023-09-10T08:28:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:32 smithi170 ceph-mon[102832]: pgmap v17728: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T08:28:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:32 smithi082 ceph-mon[98694]: pgmap v17728: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T08:28:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:32 smithi098 ceph-mon[103124]: pgmap v17728: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T08:28:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:34 smithi170 ceph-mon[102832]: pgmap v17729: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 16 op/s 2023-09-10T08:28:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32945.731472 secs 2023-09-10T08:28:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:34 smithi082 ceph-mon[98694]: pgmap v17729: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 16 op/s 2023-09-10T08:28:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32945.731472 secs 2023-09-10T08:28:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:34 smithi098 ceph-mon[103124]: pgmap v17729: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 16 op/s 2023-09-10T08:28:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32945.731472 secs 2023-09-10T08:28:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32946.174265 secs 2023-09-10T08:28:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32946.174265 secs 2023-09-10T08:28:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32946.174265 secs 2023-09-10T08:28:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:36 smithi170 ceph-mon[102832]: pgmap v17730: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T08:28:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:36 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:28:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:36 smithi082 ceph-mon[98694]: pgmap v17730: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T08:28:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:36 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:28:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:36 smithi098 ceph-mon[103124]: pgmap v17730: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 11 op/s 2023-09-10T08:28:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:36 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:28:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:37 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:28:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:28:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:28:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:37 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:28:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:28:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:28:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:37 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:28:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:28:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:28:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:38 smithi170 ceph-mon[102832]: pgmap v17731: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 287 KiB/s wr, 18 op/s 2023-09-10T08:28:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:28:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:28:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:38 smithi082 ceph-mon[98694]: pgmap v17731: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 287 KiB/s wr, 18 op/s 2023-09-10T08:28:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:28:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:28:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:38 smithi098 ceph-mon[103124]: pgmap v17731: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 287 KiB/s wr, 18 op/s 2023-09-10T08:28:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:28:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:28:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32950.731492 secs 2023-09-10T08:28:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32950.731492 secs 2023-09-10T08:28:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32950.731492 secs 2023-09-10T08:28:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:40 smithi170 ceph-mon[102832]: pgmap v17732: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 12 op/s 2023-09-10T08:28:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32951.174332 secs 2023-09-10T08:28:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:40 smithi082 ceph-mon[98694]: pgmap v17732: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 12 op/s 2023-09-10T08:28:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32951.174332 secs 2023-09-10T08:28:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:40 smithi098 ceph-mon[103124]: pgmap v17732: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 12 op/s 2023-09-10T08:28:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32951.174332 secs 2023-09-10T08:28:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:42 smithi170 ceph-mon[102832]: pgmap v17733: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 12 op/s 2023-09-10T08:28:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:42 smithi082 ceph-mon[98694]: pgmap v17733: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 12 op/s 2023-09-10T08:28:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:42 smithi098 ceph-mon[103124]: pgmap v17733: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 12 op/s 2023-09-10T08:28:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:44 smithi170 ceph-mon[102832]: pgmap v17734: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 441 KiB/s wr, 19 op/s 2023-09-10T08:28:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32955.731551 secs 2023-09-10T08:28:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:44 smithi082 ceph-mon[98694]: pgmap v17734: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 441 KiB/s wr, 19 op/s 2023-09-10T08:28:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32955.731551 secs 2023-09-10T08:28:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:44 smithi098 ceph-mon[103124]: pgmap v17734: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 441 KiB/s wr, 19 op/s 2023-09-10T08:28:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32955.731551 secs 2023-09-10T08:28:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32956.174399 secs 2023-09-10T08:28:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:28:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:28:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32956.174399 secs 2023-09-10T08:28:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:28:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:28:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32956.174399 secs 2023-09-10T08:28:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:28:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:28:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:46 smithi170 ceph-mon[102832]: pgmap v17735: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 440 KiB/s wr, 13 op/s 2023-09-10T08:28:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:28:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:28:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:46 smithi082 ceph-mon[98694]: pgmap v17735: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 440 KiB/s wr, 13 op/s 2023-09-10T08:28:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:28:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:28:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:46 smithi098 ceph-mon[103124]: pgmap v17735: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 440 KiB/s wr, 13 op/s 2023-09-10T08:28:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:28:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:28:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:47 smithi170 ceph-mon[102832]: pgmap v17736: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 654 KiB/s wr, 20 op/s 2023-09-10T08:28:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:47 smithi082 ceph-mon[98694]: pgmap v17736: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 654 KiB/s wr, 20 op/s 2023-09-10T08:28:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:47 smithi098 ceph-mon[103124]: pgmap v17736: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 654 KiB/s wr, 20 op/s 2023-09-10T08:28:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32960.731608 secs 2023-09-10T08:28:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32960.731608 secs 2023-09-10T08:28:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32960.731608 secs 2023-09-10T08:28:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:49 smithi170 ceph-mon[102832]: pgmap v17737: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 485 KiB/s wr, 13 op/s 2023-09-10T08:28:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32961.174464 secs 2023-09-10T08:28:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:49 smithi082 ceph-mon[98694]: pgmap v17737: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 485 KiB/s wr, 13 op/s 2023-09-10T08:28:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32961.174464 secs 2023-09-10T08:28:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:49 smithi098 ceph-mon[103124]: pgmap v17737: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 485 KiB/s wr, 13 op/s 2023-09-10T08:28:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32961.174464 secs 2023-09-10T08:28:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:51 smithi170 ceph-mon[102832]: pgmap v17738: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 485 KiB/s wr, 13 op/s 2023-09-10T08:28:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:51 smithi082 ceph-mon[98694]: pgmap v17738: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 485 KiB/s wr, 13 op/s 2023-09-10T08:28:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:51 smithi098 ceph-mon[103124]: pgmap v17738: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 485 KiB/s wr, 13 op/s 2023-09-10T08:28:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:53 smithi170 ceph-mon[102832]: pgmap v17739: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 730 KiB/s wr, 20 op/s 2023-09-10T08:28:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:28:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:28:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32965.731666 secs 2023-09-10T08:28:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:53 smithi082 ceph-mon[98694]: pgmap v17739: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 730 KiB/s wr, 20 op/s 2023-09-10T08:28:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:28:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:28:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32965.731666 secs 2023-09-10T08:28:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:53 smithi098 ceph-mon[103124]: pgmap v17739: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 730 KiB/s wr, 20 op/s 2023-09-10T08:28:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:28:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:28:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32965.731666 secs 2023-09-10T08:28:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32966.174491 secs 2023-09-10T08:28:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32966.174491 secs 2023-09-10T08:28:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32966.174491 secs 2023-09-10T08:28:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:55 smithi170 ceph-mon[102832]: pgmap v17740: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 459 KiB/s wr, 13 op/s 2023-09-10T08:28:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:55 smithi082 ceph-mon[98694]: pgmap v17740: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 459 KiB/s wr, 13 op/s 2023-09-10T08:28:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:55 smithi098 ceph-mon[103124]: pgmap v17740: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 459 KiB/s wr, 13 op/s 2023-09-10T08:28:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:28:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:28:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:28:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:28:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:28:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:28:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:57 smithi170 ceph-mon[102832]: pgmap v17741: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 713 KiB/s wr, 20 op/s 2023-09-10T08:28:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:57 smithi082 ceph-mon[98694]: pgmap v17741: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 713 KiB/s wr, 20 op/s 2023-09-10T08:28:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:57 smithi098 ceph-mon[103124]: pgmap v17741: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 713 KiB/s wr, 20 op/s 2023-09-10T08:28:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32969.568035 secs 2023-09-10T08:28:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32970.731724 secs 2023-09-10T08:28:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32969.568035 secs 2023-09-10T08:28:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32970.731724 secs 2023-09-10T08:28:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32969.568035 secs 2023-09-10T08:28:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32970.731724 secs 2023-09-10T08:29:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:59 smithi082 ceph-mon[98694]: pgmap v17742: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 500 KiB/s wr, 13 op/s 2023-09-10T08:29:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:28:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32971.174555 secs 2023-09-10T08:29:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:59 smithi098 ceph-mon[103124]: pgmap v17742: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 500 KiB/s wr, 13 op/s 2023-09-10T08:29:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:28:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32971.174555 secs 2023-09-10T08:29:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:59 smithi170 ceph-mon[102832]: pgmap v17742: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 500 KiB/s wr, 13 op/s 2023-09-10T08:29:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:28:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32971.174555 secs 2023-09-10T08:29:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:29:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:29:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:29:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:29:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:29:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:29:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:01 smithi082 ceph-mon[98694]: pgmap v17743: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 500 KiB/s wr, 13 op/s 2023-09-10T08:29:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:01 smithi098 ceph-mon[103124]: pgmap v17743: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 500 KiB/s wr, 13 op/s 2023-09-10T08:29:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:01 smithi170 ceph-mon[102832]: pgmap v17743: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 500 KiB/s wr, 13 op/s 2023-09-10T08:29:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:03 smithi098 ceph-mon[103124]: pgmap v17744: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 803 KiB/s wr, 32 op/s 2023-09-10T08:29:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32974.568097 secs 2023-09-10T08:29:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:03 smithi098 ceph-mon[103124]: osdmap e1177: 12 total, 12 up, 12 in 2023-09-10T08:29:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32975.731799 secs 2023-09-10T08:29:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:03 smithi170 ceph-mon[102832]: pgmap v17744: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 803 KiB/s wr, 32 op/s 2023-09-10T08:29:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32974.568097 secs 2023-09-10T08:29:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:03 smithi170 ceph-mon[102832]: osdmap e1177: 12 total, 12 up, 12 in 2023-09-10T08:29:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32975.731799 secs 2023-09-10T08:29:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:03 smithi082 ceph-mon[98694]: pgmap v17744: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 803 KiB/s wr, 32 op/s 2023-09-10T08:29:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32974.568097 secs 2023-09-10T08:29:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:03 smithi082 ceph-mon[98694]: osdmap e1177: 12 total, 12 up, 12 in 2023-09-10T08:29:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32975.731799 secs 2023-09-10T08:29:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32976.174620 secs 2023-09-10T08:29:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:29:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32976.174620 secs 2023-09-10T08:29:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:29:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32976.174620 secs 2023-09-10T08:29:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:29:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:05 smithi098 ceph-mon[103124]: pgmap v17746: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 669 KiB/s wr, 31 op/s 2023-09-10T08:29:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:29:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:05 smithi170 ceph-mon[102832]: pgmap v17746: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 669 KiB/s wr, 31 op/s 2023-09-10T08:29:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:29:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:05 smithi082 ceph-mon[98694]: pgmap v17746: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 669 KiB/s wr, 31 op/s 2023-09-10T08:29:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:29:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:07 smithi098 ceph-mon[103124]: pgmap v17747: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 561 KiB/s wr, 31 op/s 2023-09-10T08:29:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:07 smithi098 ceph-mon[103124]: osdmap e1178: 12 total, 12 up, 12 in 2023-09-10T08:29:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:07 smithi170 ceph-mon[102832]: pgmap v17747: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 561 KiB/s wr, 31 op/s 2023-09-10T08:29:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:07 smithi170 ceph-mon[102832]: osdmap e1178: 12 total, 12 up, 12 in 2023-09-10T08:29:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:07 smithi082 ceph-mon[98694]: pgmap v17747: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 561 KiB/s wr, 31 op/s 2023-09-10T08:29:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:07 smithi082 ceph-mon[98694]: osdmap e1178: 12 total, 12 up, 12 in 2023-09-10T08:29:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32979.568209 secs 2023-09-10T08:29:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32980.731857 secs 2023-09-10T08:29:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32979.568209 secs 2023-09-10T08:29:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32980.731857 secs 2023-09-10T08:29:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32979.568209 secs 2023-09-10T08:29:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32980.731857 secs 2023-09-10T08:29:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:09 smithi098 ceph-mon[103124]: pgmap v17749: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 701 KiB/s wr, 39 op/s 2023-09-10T08:29:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32981.174685 secs 2023-09-10T08:29:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:09 smithi170 ceph-mon[102832]: pgmap v17749: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 701 KiB/s wr, 39 op/s 2023-09-10T08:29:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32981.174685 secs 2023-09-10T08:29:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:09 smithi082 ceph-mon[98694]: pgmap v17749: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 701 KiB/s wr, 39 op/s 2023-09-10T08:29:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32981.174685 secs 2023-09-10T08:29:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:11 smithi098 ceph-mon[103124]: pgmap v17750: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 247 KiB/s wr, 10 op/s 2023-09-10T08:29:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:11 smithi170 ceph-mon[102832]: pgmap v17750: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 247 KiB/s wr, 10 op/s 2023-09-10T08:29:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:11 smithi082 ceph-mon[98694]: pgmap v17750: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 247 KiB/s wr, 10 op/s 2023-09-10T08:29:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:13 smithi098 ceph-mon[103124]: pgmap v17751: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 371 KiB/s wr, 17 op/s 2023-09-10T08:29:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32984.568268 secs 2023-09-10T08:29:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:29:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32985.731917 secs 2023-09-10T08:29:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:13 smithi170 ceph-mon[102832]: pgmap v17751: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 371 KiB/s wr, 17 op/s 2023-09-10T08:29:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32984.568268 secs 2023-09-10T08:29:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:29:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32985.731917 secs 2023-09-10T08:29:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:13 smithi082 ceph-mon[98694]: pgmap v17751: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 371 KiB/s wr, 17 op/s 2023-09-10T08:29:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32984.568268 secs 2023-09-10T08:29:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:29:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32985.731917 secs 2023-09-10T08:29:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32986.174750 secs 2023-09-10T08:29:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32986.174750 secs 2023-09-10T08:29:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32986.174750 secs 2023-09-10T08:29:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:15 smithi098 ceph-mon[103124]: pgmap v17752: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 339 KiB/s wr, 15 op/s 2023-09-10T08:29:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:15 smithi170 ceph-mon[102832]: pgmap v17752: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 339 KiB/s wr, 15 op/s 2023-09-10T08:29:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:15 smithi082 ceph-mon[98694]: pgmap v17752: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 339 KiB/s wr, 15 op/s 2023-09-10T08:29:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:17 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:29:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:17 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:17 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:29:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:17 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:17 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:29:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:17 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:18 smithi098 ceph-mon[103124]: pgmap v17753: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 172 KiB/s wr, 14 op/s 2023-09-10T08:29:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:18 smithi170 ceph-mon[102832]: pgmap v17753: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 172 KiB/s wr, 14 op/s 2023-09-10T08:29:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:18 smithi082 ceph-mon[98694]: pgmap v17753: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 172 KiB/s wr, 14 op/s 2023-09-10T08:29:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32989.568387 secs 2023-09-10T08:29:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32990.731972 secs 2023-09-10T08:29:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32989.568387 secs 2023-09-10T08:29:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32990.731972 secs 2023-09-10T08:29:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32989.568387 secs 2023-09-10T08:29:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32990.731972 secs 2023-09-10T08:29:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:20 smithi170 ceph-mon[102832]: pgmap v17754: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 169 KiB/s wr, 14 op/s 2023-09-10T08:29:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32991.174816 secs 2023-09-10T08:29:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:20 smithi082 ceph-mon[98694]: pgmap v17754: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 169 KiB/s wr, 14 op/s 2023-09-10T08:29:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32991.174816 secs 2023-09-10T08:29:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:20 smithi098 ceph-mon[103124]: pgmap v17754: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 169 KiB/s wr, 14 op/s 2023-09-10T08:29:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32991.174816 secs 2023-09-10T08:29:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:22 smithi170 ceph-mon[102832]: pgmap v17755: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 144 KiB/s wr, 12 op/s 2023-09-10T08:29:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:22 smithi082 ceph-mon[98694]: pgmap v17755: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 144 KiB/s wr, 12 op/s 2023-09-10T08:29:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:22 smithi098 ceph-mon[103124]: pgmap v17755: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 144 KiB/s wr, 12 op/s 2023-09-10T08:29:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:24 smithi170 ceph-mon[102832]: pgmap v17756: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 217 KiB/s wr, 17 op/s 2023-09-10T08:29:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32994.568448 secs 2023-09-10T08:29:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32995.732075 secs 2023-09-10T08:29:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:29:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:24 smithi082 ceph-mon[98694]: pgmap v17756: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 217 KiB/s wr, 17 op/s 2023-09-10T08:29:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32994.568448 secs 2023-09-10T08:29:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32995.732075 secs 2023-09-10T08:29:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:29:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:24 smithi098 ceph-mon[103124]: pgmap v17756: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 217 KiB/s wr, 17 op/s 2023-09-10T08:29:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32994.568448 secs 2023-09-10T08:29:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32995.732075 secs 2023-09-10T08:29:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:29:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 32996.174883 secs 2023-09-10T08:29:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:29:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:29:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:25 smithi170 ceph-mon[102832]: pgmap v17757: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T08:29:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:29:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:29:25.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:29:25.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:29:25.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:29:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 32996.174883 secs 2023-09-10T08:29:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:29:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:29:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:25 smithi082 ceph-mon[98694]: pgmap v17757: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T08:29:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:29:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:29:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:29:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:29:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:29:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 32996.174883 secs 2023-09-10T08:29:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:29:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:29:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:25 smithi098 ceph-mon[103124]: pgmap v17757: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 99 KiB/s wr, 11 op/s 2023-09-10T08:29:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:29:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:29:25.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:29:25.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:29:25.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:29:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:27 smithi170 ceph-mon[102832]: pgmap v17758: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 218 KiB/s wr, 17 op/s 2023-09-10T08:29:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:27 smithi082 ceph-mon[98694]: pgmap v17758: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 218 KiB/s wr, 17 op/s 2023-09-10T08:29:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:27 smithi098 ceph-mon[103124]: pgmap v17758: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 218 KiB/s wr, 17 op/s 2023-09-10T08:29:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 32999.568510 secs 2023-09-10T08:29:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33000.732134 secs 2023-09-10T08:29:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 32999.568510 secs 2023-09-10T08:29:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33000.732134 secs 2023-09-10T08:29:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 32999.568510 secs 2023-09-10T08:29:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33000.732134 secs 2023-09-10T08:29:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:29 smithi170 ceph-mon[102832]: pgmap v17759: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T08:29:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33001.174950 secs 2023-09-10T08:29:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:29 smithi082 ceph-mon[98694]: pgmap v17759: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T08:29:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33001.174950 secs 2023-09-10T08:29:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:29 smithi098 ceph-mon[103124]: pgmap v17759: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T08:29:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33001.174950 secs 2023-09-10T08:29:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:31 smithi170 ceph-mon[102832]: pgmap v17760: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T08:29:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:31 smithi082 ceph-mon[98694]: pgmap v17760: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T08:29:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:31 smithi098 ceph-mon[103124]: pgmap v17760: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T08:29:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:33 smithi170 ceph-mon[102832]: pgmap v17761: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 303 KiB/s wr, 16 op/s 2023-09-10T08:29:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:29:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33005.732193 secs 2023-09-10T08:29:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:33 smithi082 ceph-mon[98694]: pgmap v17761: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 303 KiB/s wr, 16 op/s 2023-09-10T08:29:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:29:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33005.732193 secs 2023-09-10T08:29:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:33 smithi098 ceph-mon[103124]: pgmap v17761: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 303 KiB/s wr, 16 op/s 2023-09-10T08:29:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:29:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33005.732193 secs 2023-09-10T08:29:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33006.175017 secs 2023-09-10T08:29:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33006.175017 secs 2023-09-10T08:29:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33006.175017 secs 2023-09-10T08:29:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:35 smithi082 ceph-mon[98694]: pgmap v17762: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 229 KiB/s wr, 11 op/s 2023-09-10T08:29:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:29:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:35 smithi098 ceph-mon[103124]: pgmap v17762: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 229 KiB/s wr, 11 op/s 2023-09-10T08:29:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:29:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:35 smithi170 ceph-mon[102832]: pgmap v17762: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 229 KiB/s wr, 11 op/s 2023-09-10T08:29:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:29:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:29:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:29:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:29:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:29:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:29:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:29:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:37 smithi082 ceph-mon[98694]: pgmap v17763: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 310 KiB/s wr, 17 op/s 2023-09-10T08:29:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:37 smithi098 ceph-mon[103124]: pgmap v17763: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 310 KiB/s wr, 17 op/s 2023-09-10T08:29:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:37 smithi170 ceph-mon[102832]: pgmap v17763: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 310 KiB/s wr, 17 op/s 2023-09-10T08:29:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33010.732264 secs 2023-09-10T08:29:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33010.732264 secs 2023-09-10T08:29:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33010.732264 secs 2023-09-10T08:29:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:39 smithi082 ceph-mon[98694]: pgmap v17764: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T08:29:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33011.175081 secs 2023-09-10T08:29:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:39 smithi098 ceph-mon[103124]: pgmap v17764: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T08:29:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33011.175081 secs 2023-09-10T08:29:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:39 smithi170 ceph-mon[102832]: pgmap v17764: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T08:29:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33011.175081 secs 2023-09-10T08:29:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:41 smithi082 ceph-mon[98694]: pgmap v17765: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T08:29:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:41 smithi098 ceph-mon[103124]: pgmap v17765: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T08:29:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:41 smithi170 ceph-mon[102832]: pgmap v17765: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 191 KiB/s wr, 11 op/s 2023-09-10T08:29:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:43 smithi098 ceph-mon[103124]: pgmap v17766: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 503 KiB/s wr, 18 op/s 2023-09-10T08:29:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33015.732353 secs 2023-09-10T08:29:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:43 smithi170 ceph-mon[102832]: pgmap v17766: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 503 KiB/s wr, 18 op/s 2023-09-10T08:29:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33015.732353 secs 2023-09-10T08:29:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:43 smithi082 ceph-mon[98694]: pgmap v17766: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 503 KiB/s wr, 18 op/s 2023-09-10T08:29:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33015.732353 secs 2023-09-10T08:29:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33016.175146 secs 2023-09-10T08:29:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:29:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33016.175146 secs 2023-09-10T08:29:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:29:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33016.175146 secs 2023-09-10T08:29:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:29:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:45 smithi098 ceph-mon[103124]: pgmap v17767: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 392 KiB/s wr, 13 op/s 2023-09-10T08:29:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:29:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:45 smithi170 ceph-mon[102832]: pgmap v17767: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 392 KiB/s wr, 13 op/s 2023-09-10T08:29:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:29:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:45 smithi082 ceph-mon[98694]: pgmap v17767: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 392 KiB/s wr, 13 op/s 2023-09-10T08:29:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:29:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:47 smithi098 ceph-mon[103124]: pgmap v17768: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 567 KiB/s wr, 20 op/s 2023-09-10T08:29:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:47 smithi170 ceph-mon[102832]: pgmap v17768: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 567 KiB/s wr, 20 op/s 2023-09-10T08:29:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:47 smithi082 ceph-mon[98694]: pgmap v17768: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 567 KiB/s wr, 20 op/s 2023-09-10T08:29:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33020.732416 secs 2023-09-10T08:29:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33020.732416 secs 2023-09-10T08:29:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33020.732416 secs 2023-09-10T08:29:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:49 smithi098 ceph-mon[103124]: pgmap v17769: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 486 KiB/s wr, 14 op/s 2023-09-10T08:29:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33021.175213 secs 2023-09-10T08:29:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:49 smithi170 ceph-mon[102832]: pgmap v17769: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 486 KiB/s wr, 14 op/s 2023-09-10T08:29:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33021.175213 secs 2023-09-10T08:29:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:49 smithi082 ceph-mon[98694]: pgmap v17769: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 486 KiB/s wr, 14 op/s 2023-09-10T08:29:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33021.175213 secs 2023-09-10T08:29:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:51 smithi098 ceph-mon[103124]: pgmap v17770: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 486 KiB/s wr, 14 op/s 2023-09-10T08:29:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:51 smithi170 ceph-mon[102832]: pgmap v17770: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 486 KiB/s wr, 14 op/s 2023-09-10T08:29:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:51 smithi082 ceph-mon[98694]: pgmap v17770: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 486 KiB/s wr, 14 op/s 2023-09-10T08:29:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:29:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:29:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:29:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:54 smithi098 ceph-mon[103124]: pgmap v17771: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 875 KiB/s wr, 21 op/s 2023-09-10T08:29:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33024.568884 secs 2023-09-10T08:29:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:29:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:54.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33025.732474 secs 2023-09-10T08:29:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:54 smithi170 ceph-mon[102832]: pgmap v17771: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 875 KiB/s wr, 21 op/s 2023-09-10T08:29:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33024.568884 secs 2023-09-10T08:29:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:29:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33025.732474 secs 2023-09-10T08:29:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:54 smithi082 ceph-mon[98694]: pgmap v17771: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 875 KiB/s wr, 21 op/s 2023-09-10T08:29:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33024.568884 secs 2023-09-10T08:29:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:29:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33025.732474 secs 2023-09-10T08:29:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33026.175278 secs 2023-09-10T08:29:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33026.175278 secs 2023-09-10T08:29:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33026.175278 secs 2023-09-10T08:29:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:56 smithi170 ceph-mon[102832]: pgmap v17772: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 563 KiB/s wr, 14 op/s 2023-09-10T08:29:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:56 smithi082 ceph-mon[98694]: pgmap v17772: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 563 KiB/s wr, 14 op/s 2023-09-10T08:29:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:56 smithi098 ceph-mon[103124]: pgmap v17772: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 563 KiB/s wr, 14 op/s 2023-09-10T08:29:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:57 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:29:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:29:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:57 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:29:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:29:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:57 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:29:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:29:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:29:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:58 smithi170 ceph-mon[102832]: pgmap v17773: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 790 KiB/s wr, 21 op/s 2023-09-10T08:29:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:58 smithi082 ceph-mon[98694]: pgmap v17773: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 790 KiB/s wr, 21 op/s 2023-09-10T08:29:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:58 smithi098 ceph-mon[103124]: pgmap v17773: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 790 KiB/s wr, 21 op/s 2023-09-10T08:29:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33029.568945 secs 2023-09-10T08:29:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:29:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33030.732561 secs 2023-09-10T08:29:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33029.568945 secs 2023-09-10T08:29:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:29:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33030.732561 secs 2023-09-10T08:29:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33029.568945 secs 2023-09-10T08:29:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:29:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33030.732561 secs 2023-09-10T08:30:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:00 smithi170 ceph-mon[102832]: pgmap v17774: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 615 KiB/s wr, 14 op/s 2023-09-10T08:30:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33031.175341 secs 2023-09-10T08:30:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T08:30:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:00 smithi082 ceph-mon[98694]: pgmap v17774: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 615 KiB/s wr, 14 op/s 2023-09-10T08:30:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33031.175341 secs 2023-09-10T08:30:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T08:30:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:00 smithi098 ceph-mon[103124]: pgmap v17774: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 615 KiB/s wr, 14 op/s 2023-09-10T08:30:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33031.175341 secs 2023-09-10T08:30:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T08:30:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:02 smithi170 ceph-mon[102832]: pgmap v17775: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 615 KiB/s wr, 14 op/s 2023-09-10T08:30:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:02 smithi082 ceph-mon[98694]: pgmap v17775: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 615 KiB/s wr, 14 op/s 2023-09-10T08:30:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:02 smithi098 ceph-mon[103124]: pgmap v17775: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 615 KiB/s wr, 14 op/s 2023-09-10T08:30:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:04 smithi170 ceph-mon[102832]: pgmap v17776: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 1008 KiB/s wr, 23 op/s 2023-09-10T08:30:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33034.569045 secs 2023-09-10T08:30:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33035.732619 secs 2023-09-10T08:30:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:04 smithi082 ceph-mon[98694]: pgmap v17776: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 1008 KiB/s wr, 23 op/s 2023-09-10T08:30:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33034.569045 secs 2023-09-10T08:30:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33035.732619 secs 2023-09-10T08:30:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:04 smithi098 ceph-mon[103124]: pgmap v17776: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 1008 KiB/s wr, 23 op/s 2023-09-10T08:30:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33034.569045 secs 2023-09-10T08:30:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33035.732619 secs 2023-09-10T08:30:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33036.175410 secs 2023-09-10T08:30:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:30:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:30:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33036.175410 secs 2023-09-10T08:30:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:30:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:30:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33036.175410 secs 2023-09-10T08:30:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:30:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:30:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:06 smithi170 ceph-mon[102832]: pgmap v17777: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 620 KiB/s wr, 16 op/s 2023-09-10T08:30:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:30:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:30:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:06 smithi082 ceph-mon[98694]: pgmap v17777: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 620 KiB/s wr, 16 op/s 2023-09-10T08:30:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:30:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:30:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:06 smithi098 ceph-mon[103124]: pgmap v17777: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 620 KiB/s wr, 16 op/s 2023-09-10T08:30:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:30:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:30:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:08 smithi170 ceph-mon[102832]: pgmap v17778: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 665 KiB/s wr, 34 op/s 2023-09-10T08:30:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:08 smithi082 ceph-mon[98694]: pgmap v17778: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 665 KiB/s wr, 34 op/s 2023-09-10T08:30:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:08 smithi098 ceph-mon[103124]: pgmap v17778: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 665 KiB/s wr, 34 op/s 2023-09-10T08:30:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33039.569105 secs 2023-09-10T08:30:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:09 smithi170 ceph-mon[102832]: osdmap e1179: 12 total, 12 up, 12 in 2023-09-10T08:30:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33040.732674 secs 2023-09-10T08:30:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33039.569105 secs 2023-09-10T08:30:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:09 smithi082 ceph-mon[98694]: osdmap e1179: 12 total, 12 up, 12 in 2023-09-10T08:30:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33040.732674 secs 2023-09-10T08:30:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33039.569105 secs 2023-09-10T08:30:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:09 smithi098 ceph-mon[103124]: osdmap e1179: 12 total, 12 up, 12 in 2023-09-10T08:30:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33040.732674 secs 2023-09-10T08:30:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:10 smithi170 ceph-mon[102832]: pgmap v17780: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 525 KiB/s wr, 32 op/s 2023-09-10T08:30:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33041.175481 secs 2023-09-10T08:30:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:10 smithi082 ceph-mon[98694]: pgmap v17780: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 525 KiB/s wr, 32 op/s 2023-09-10T08:30:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33041.175481 secs 2023-09-10T08:30:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:10 smithi098 ceph-mon[103124]: pgmap v17780: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 525 KiB/s wr, 32 op/s 2023-09-10T08:30:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33041.175481 secs 2023-09-10T08:30:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:11 smithi170 ceph-mon[102832]: pgmap v17781: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 525 KiB/s wr, 32 op/s 2023-09-10T08:30:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:11 smithi082 ceph-mon[98694]: pgmap v17781: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 525 KiB/s wr, 32 op/s 2023-09-10T08:30:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:11 smithi098 ceph-mon[103124]: pgmap v17781: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 525 KiB/s wr, 32 op/s 2023-09-10T08:30:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:13 smithi170 ceph-mon[102832]: pgmap v17782: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 151 KiB/s wr, 28 op/s 2023-09-10T08:30:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33044.569164 secs 2023-09-10T08:30:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:13 smithi170 ceph-mon[102832]: osdmap e1180: 12 total, 12 up, 12 in 2023-09-10T08:30:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33045.732736 secs 2023-09-10T08:30:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:13 smithi082 ceph-mon[98694]: pgmap v17782: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 151 KiB/s wr, 28 op/s 2023-09-10T08:30:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33044.569164 secs 2023-09-10T08:30:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:13 smithi082 ceph-mon[98694]: osdmap e1180: 12 total, 12 up, 12 in 2023-09-10T08:30:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33045.732736 secs 2023-09-10T08:30:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:13 smithi098 ceph-mon[103124]: pgmap v17782: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 151 KiB/s wr, 28 op/s 2023-09-10T08:30:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33044.569164 secs 2023-09-10T08:30:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:13 smithi098 ceph-mon[103124]: osdmap e1180: 12 total, 12 up, 12 in 2023-09-10T08:30:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33045.732736 secs 2023-09-10T08:30:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33046.175548 secs 2023-09-10T08:30:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33046.175548 secs 2023-09-10T08:30:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33046.175548 secs 2023-09-10T08:30:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:15 smithi170 ceph-mon[102832]: pgmap v17784: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 8 op/s 2023-09-10T08:30:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:15 smithi082 ceph-mon[98694]: pgmap v17784: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 8 op/s 2023-09-10T08:30:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:15 smithi098 ceph-mon[103124]: pgmap v17784: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 8 op/s 2023-09-10T08:30:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:17 smithi170 ceph-mon[102832]: pgmap v17785: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 122 KiB/s wr, 15 op/s 2023-09-10T08:30:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:17 smithi082 ceph-mon[98694]: pgmap v17785: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 122 KiB/s wr, 15 op/s 2023-09-10T08:30:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:17 smithi098 ceph-mon[103124]: pgmap v17785: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 122 KiB/s wr, 15 op/s 2023-09-10T08:30:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33049.569226 secs 2023-09-10T08:30:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33050.732794 secs 2023-09-10T08:30:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33049.569226 secs 2023-09-10T08:30:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33050.732794 secs 2023-09-10T08:30:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33049.569226 secs 2023-09-10T08:30:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33050.732794 secs 2023-09-10T08:30:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:19 smithi082 ceph-mon[98694]: pgmap v17786: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T08:30:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33051.175616 secs 2023-09-10T08:30:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:19 smithi098 ceph-mon[103124]: pgmap v17786: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T08:30:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33051.175616 secs 2023-09-10T08:30:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:19 smithi170 ceph-mon[102832]: pgmap v17786: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T08:30:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33051.175616 secs 2023-09-10T08:30:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:21 smithi082 ceph-mon[98694]: pgmap v17787: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T08:30:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:21 smithi098 ceph-mon[103124]: pgmap v17787: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T08:30:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:21 smithi170 ceph-mon[102832]: pgmap v17787: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 99 KiB/s wr, 12 op/s 2023-09-10T08:30:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:23 smithi082 ceph-mon[98694]: pgmap v17788: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2023-09-10T08:30:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33054.569314 secs 2023-09-10T08:30:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:23 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33055.732853 secs 2023-09-10T08:30:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:23 smithi098 ceph-mon[103124]: pgmap v17788: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2023-09-10T08:30:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33054.569314 secs 2023-09-10T08:30:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:23 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33055.732853 secs 2023-09-10T08:30:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:23 smithi170 ceph-mon[102832]: pgmap v17788: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2023-09-10T08:30:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33054.569314 secs 2023-09-10T08:30:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:23 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33055.732853 secs 2023-09-10T08:30:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33056.175681 secs 2023-09-10T08:30:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:30:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33056.175681 secs 2023-09-10T08:30:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:30:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33056.175681 secs 2023-09-10T08:30:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:30:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:25 smithi082 ceph-mon[98694]: pgmap v17789: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-10T08:30:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:30:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:30:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:30:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:25 smithi098 ceph-mon[103124]: pgmap v17789: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-10T08:30:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:30:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:30:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:30:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:25 smithi170 ceph-mon[102832]: pgmap v17789: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-10T08:30:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:30:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:30:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:30:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:27 smithi082 ceph-mon[98694]: pgmap v17790: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 15 op/s 2023-09-10T08:30:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:27 smithi098 ceph-mon[103124]: pgmap v17790: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 15 op/s 2023-09-10T08:30:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:27 smithi170 ceph-mon[102832]: pgmap v17790: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 15 op/s 2023-09-10T08:30:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33059.569375 secs 2023-09-10T08:30:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:28 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33060.732935 secs 2023-09-10T08:30:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33059.569375 secs 2023-09-10T08:30:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:28 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33060.732935 secs 2023-09-10T08:30:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33059.569375 secs 2023-09-10T08:30:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:28 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33060.732935 secs 2023-09-10T08:30:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:29 smithi098 ceph-mon[103124]: pgmap v17791: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T08:30:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33061.175750 secs 2023-09-10T08:30:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:29 smithi170 ceph-mon[102832]: pgmap v17791: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T08:30:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33061.175750 secs 2023-09-10T08:30:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:29 smithi082 ceph-mon[98694]: pgmap v17791: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T08:30:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33061.175750 secs 2023-09-10T08:30:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:31 smithi098 ceph-mon[103124]: pgmap v17792: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T08:30:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:31 smithi170 ceph-mon[102832]: pgmap v17792: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T08:30:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:31 smithi082 ceph-mon[98694]: pgmap v17792: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T08:30:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:33 smithi098 ceph-mon[103124]: pgmap v17793: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 119 KiB/s wr, 15 op/s 2023-09-10T08:30:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:30:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:30:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33065.733017 secs 2023-09-10T08:30:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:33 smithi170 ceph-mon[102832]: pgmap v17793: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 119 KiB/s wr, 15 op/s 2023-09-10T08:30:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:30:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:30:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33065.733017 secs 2023-09-10T08:30:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:33 smithi082 ceph-mon[98694]: pgmap v17793: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 119 KiB/s wr, 15 op/s 2023-09-10T08:30:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:30:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:30:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33065.733017 secs 2023-09-10T08:30:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33066.175814 secs 2023-09-10T08:30:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33066.175814 secs 2023-09-10T08:30:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33066.175814 secs 2023-09-10T08:30:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:35 smithi098 ceph-mon[103124]: pgmap v17794: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T08:30:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:30:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:35 smithi170 ceph-mon[102832]: pgmap v17794: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T08:30:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:30:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:35 smithi082 ceph-mon[98694]: pgmap v17794: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T08:30:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:30:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:30:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:30:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:30:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:30:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:30:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:30:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:30:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:30:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:30:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:37 smithi098 ceph-mon[103124]: pgmap v17795: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 289 KiB/s wr, 17 op/s 2023-09-10T08:30:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:37 smithi170 ceph-mon[102832]: pgmap v17795: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 289 KiB/s wr, 17 op/s 2023-09-10T08:30:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:37 smithi082 ceph-mon[98694]: pgmap v17795: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 289 KiB/s wr, 17 op/s 2023-09-10T08:30:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33070.733074 secs 2023-09-10T08:30:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33070.733074 secs 2023-09-10T08:30:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33070.733074 secs 2023-09-10T08:30:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:40 smithi098 ceph-mon[103124]: pgmap v17796: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 170 KiB/s wr, 11 op/s 2023-09-10T08:30:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33071.175885 secs 2023-09-10T08:30:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:40 smithi170 ceph-mon[102832]: pgmap v17796: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 170 KiB/s wr, 11 op/s 2023-09-10T08:30:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33071.175885 secs 2023-09-10T08:30:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:40 smithi082 ceph-mon[98694]: pgmap v17796: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 170 KiB/s wr, 11 op/s 2023-09-10T08:30:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33071.175885 secs 2023-09-10T08:30:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:42 smithi170 ceph-mon[102832]: pgmap v17797: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 170 KiB/s wr, 11 op/s 2023-09-10T08:30:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:42 smithi082 ceph-mon[98694]: pgmap v17797: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 170 KiB/s wr, 11 op/s 2023-09-10T08:30:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:42 smithi098 ceph-mon[103124]: pgmap v17797: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 170 KiB/s wr, 11 op/s 2023-09-10T08:30:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:44 smithi170 ceph-mon[102832]: pgmap v17798: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 477 KiB/s wr, 19 op/s 2023-09-10T08:30:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33075.733161 secs 2023-09-10T08:30:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:44 smithi082 ceph-mon[98694]: pgmap v17798: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 477 KiB/s wr, 19 op/s 2023-09-10T08:30:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33075.733161 secs 2023-09-10T08:30:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:44 smithi098 ceph-mon[103124]: pgmap v17798: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 477 KiB/s wr, 19 op/s 2023-09-10T08:30:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33075.733161 secs 2023-09-10T08:30:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33076.175949 secs 2023-09-10T08:30:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:30:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:30:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33076.175949 secs 2023-09-10T08:30:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:30:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:30:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33076.175949 secs 2023-09-10T08:30:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:30:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:30:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:46 smithi170 ceph-mon[102832]: pgmap v17799: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 477 KiB/s wr, 14 op/s 2023-09-10T08:30:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:30:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:30:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:46 smithi082 ceph-mon[98694]: pgmap v17799: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 477 KiB/s wr, 14 op/s 2023-09-10T08:30:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:30:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:30:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:46 smithi098 ceph-mon[103124]: pgmap v17799: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 477 KiB/s wr, 14 op/s 2023-09-10T08:30:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:30:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:30:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:48 smithi170 ceph-mon[102832]: pgmap v17800: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 658 KiB/s wr, 21 op/s 2023-09-10T08:30:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:48 smithi082 ceph-mon[98694]: pgmap v17800: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 658 KiB/s wr, 21 op/s 2023-09-10T08:30:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:48 smithi098 ceph-mon[103124]: pgmap v17800: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 658 KiB/s wr, 21 op/s 2023-09-10T08:30:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33080.733219 secs 2023-09-10T08:30:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33080.733219 secs 2023-09-10T08:30:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33080.733219 secs 2023-09-10T08:30:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:50 smithi170 ceph-mon[102832]: pgmap v17801: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 488 KiB/s wr, 14 op/s 2023-09-10T08:30:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33081.176015 secs 2023-09-10T08:30:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:50 smithi082 ceph-mon[98694]: pgmap v17801: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 488 KiB/s wr, 14 op/s 2023-09-10T08:30:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33081.176015 secs 2023-09-10T08:30:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:50 smithi098 ceph-mon[103124]: pgmap v17801: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 488 KiB/s wr, 14 op/s 2023-09-10T08:30:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33081.176015 secs 2023-09-10T08:30:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:52 smithi170 ceph-mon[102832]: pgmap v17802: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 488 KiB/s wr, 14 op/s 2023-09-10T08:30:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:52 smithi082 ceph-mon[98694]: pgmap v17802: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 488 KiB/s wr, 14 op/s 2023-09-10T08:30:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:52 smithi098 ceph-mon[103124]: pgmap v17802: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 488 KiB/s wr, 14 op/s 2023-09-10T08:30:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:54 smithi170 ceph-mon[102832]: pgmap v17803: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 735 KiB/s wr, 21 op/s 2023-09-10T08:30:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:30:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:30:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33085.733279 secs 2023-09-10T08:30:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:54 smithi082 ceph-mon[98694]: pgmap v17803: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 735 KiB/s wr, 21 op/s 2023-09-10T08:30:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:30:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:30:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33085.733279 secs 2023-09-10T08:30:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:54 smithi098 ceph-mon[103124]: pgmap v17803: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 735 KiB/s wr, 21 op/s 2023-09-10T08:30:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:30:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:30:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33085.733279 secs 2023-09-10T08:30:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33086.176081 secs 2023-09-10T08:30:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33086.176081 secs 2023-09-10T08:30:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33086.176081 secs 2023-09-10T08:30:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:56 smithi170 ceph-mon[102832]: pgmap v17804: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 428 KiB/s wr, 13 op/s 2023-09-10T08:30:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:56 smithi082 ceph-mon[98694]: pgmap v17804: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 428 KiB/s wr, 13 op/s 2023-09-10T08:30:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:56 smithi098 ceph-mon[103124]: pgmap v17804: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 428 KiB/s wr, 13 op/s 2023-09-10T08:30:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:30:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:30:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:30:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:30:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:30:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:30:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:58 smithi170 ceph-mon[102832]: pgmap v17805: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 667 KiB/s wr, 20 op/s 2023-09-10T08:30:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:58 smithi082 ceph-mon[98694]: pgmap v17805: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 667 KiB/s wr, 20 op/s 2023-09-10T08:30:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:58 smithi098 ceph-mon[103124]: pgmap v17805: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 667 KiB/s wr, 20 op/s 2023-09-10T08:30:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33089.569773 secs 2023-09-10T08:30:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:30:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33090.733337 secs 2023-09-10T08:30:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33089.569773 secs 2023-09-10T08:30:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:30:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33090.733337 secs 2023-09-10T08:30:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33089.569773 secs 2023-09-10T08:30:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:30:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33090.733337 secs 2023-09-10T08:31:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:00 smithi170 ceph-mon[102832]: pgmap v17806: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 486 KiB/s wr, 13 op/s 2023-09-10T08:31:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33091.176145 secs 2023-09-10T08:31:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:00 smithi082 ceph-mon[98694]: pgmap v17806: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 486 KiB/s wr, 13 op/s 2023-09-10T08:31:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33091.176145 secs 2023-09-10T08:31:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:00 smithi098 ceph-mon[103124]: pgmap v17806: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 486 KiB/s wr, 13 op/s 2023-09-10T08:31:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33091.176145 secs 2023-09-10T08:31:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:01 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:31:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:01 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:31:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:01 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:01 smithi170 ceph-mon[102832]: pgmap v17807: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 486 KiB/s wr, 13 op/s 2023-09-10T08:31:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:01 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:31:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:01 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:31:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:01 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:01 smithi082 ceph-mon[98694]: pgmap v17807: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 486 KiB/s wr, 13 op/s 2023-09-10T08:31:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:01 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:31:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:01 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:31:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:01 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:01 smithi098 ceph-mon[103124]: pgmap v17807: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 486 KiB/s wr, 13 op/s 2023-09-10T08:31:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:03 smithi082 ceph-mon[98694]: pgmap v17808: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 713 KiB/s wr, 19 op/s 2023-09-10T08:31:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33094.569831 secs 2023-09-10T08:31:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:03 smithi082 ceph-mon[98694]: osdmap e1181: 12 total, 12 up, 12 in 2023-09-10T08:31:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33095.733426 secs 2023-09-10T08:31:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:03 smithi098 ceph-mon[103124]: pgmap v17808: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 713 KiB/s wr, 19 op/s 2023-09-10T08:31:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33094.569831 secs 2023-09-10T08:31:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:03 smithi098 ceph-mon[103124]: osdmap e1181: 12 total, 12 up, 12 in 2023-09-10T08:31:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33095.733426 secs 2023-09-10T08:31:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:03 smithi170 ceph-mon[102832]: pgmap v17808: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 713 KiB/s wr, 19 op/s 2023-09-10T08:31:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33094.569831 secs 2023-09-10T08:31:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:03 smithi170 ceph-mon[102832]: osdmap e1181: 12 total, 12 up, 12 in 2023-09-10T08:31:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33095.733426 secs 2023-09-10T08:31:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33096.176212 secs 2023-09-10T08:31:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:31:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33096.176212 secs 2023-09-10T08:31:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:31:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33096.176212 secs 2023-09-10T08:31:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:31:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:05 smithi082 ceph-mon[98694]: pgmap v17810: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 560 KiB/s wr, 15 op/s 2023-09-10T08:31:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:31:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:05 smithi098 ceph-mon[103124]: pgmap v17810: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 560 KiB/s wr, 15 op/s 2023-09-10T08:31:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:31:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:05 smithi170 ceph-mon[102832]: pgmap v17810: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 560 KiB/s wr, 15 op/s 2023-09-10T08:31:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:31:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:07 smithi082 ceph-mon[98694]: pgmap v17811: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 499 KiB/s wr, 31 op/s 2023-09-10T08:31:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:07 smithi082 ceph-mon[98694]: osdmap e1182: 12 total, 12 up, 12 in 2023-09-10T08:31:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:07 smithi098 ceph-mon[103124]: pgmap v17811: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 499 KiB/s wr, 31 op/s 2023-09-10T08:31:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:07 smithi098 ceph-mon[103124]: osdmap e1182: 12 total, 12 up, 12 in 2023-09-10T08:31:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:07 smithi170 ceph-mon[102832]: pgmap v17811: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 499 KiB/s wr, 31 op/s 2023-09-10T08:31:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:07 smithi170 ceph-mon[102832]: osdmap e1182: 12 total, 12 up, 12 in 2023-09-10T08:31:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33099.569893 secs 2023-09-10T08:31:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:08 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33100.733488 secs 2023-09-10T08:31:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33099.569893 secs 2023-09-10T08:31:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:08 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33100.733488 secs 2023-09-10T08:31:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33099.569893 secs 2023-09-10T08:31:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:08 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33100.733488 secs 2023-09-10T08:31:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:09 smithi082 ceph-mon[98694]: pgmap v17813: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 624 KiB/s wr, 39 op/s 2023-09-10T08:31:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33101.176277 secs 2023-09-10T08:31:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:09 smithi098 ceph-mon[103124]: pgmap v17813: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 624 KiB/s wr, 39 op/s 2023-09-10T08:31:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33101.176277 secs 2023-09-10T08:31:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:09 smithi170 ceph-mon[102832]: pgmap v17813: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 624 KiB/s wr, 39 op/s 2023-09-10T08:31:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33101.176277 secs 2023-09-10T08:31:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:11 smithi098 ceph-mon[103124]: pgmap v17814: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 283 KiB/s wr, 30 op/s 2023-09-10T08:31:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:11 smithi170 ceph-mon[102832]: pgmap v17814: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 283 KiB/s wr, 30 op/s 2023-09-10T08:31:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:11 smithi082 ceph-mon[98694]: pgmap v17814: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 283 KiB/s wr, 30 op/s 2023-09-10T08:31:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:13 smithi098 ceph-mon[103124]: pgmap v17815: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 412 KiB/s wr, 35 op/s 2023-09-10T08:31:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33104.569958 secs 2023-09-10T08:31:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:31:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33105.733577 secs 2023-09-10T08:31:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:13 smithi170 ceph-mon[102832]: pgmap v17815: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 412 KiB/s wr, 35 op/s 2023-09-10T08:31:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33104.569958 secs 2023-09-10T08:31:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:31:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33105.733577 secs 2023-09-10T08:31:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:13 smithi082 ceph-mon[98694]: pgmap v17815: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 412 KiB/s wr, 35 op/s 2023-09-10T08:31:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33104.569958 secs 2023-09-10T08:31:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:31:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33105.733577 secs 2023-09-10T08:31:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33106.176346 secs 2023-09-10T08:31:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33106.176346 secs 2023-09-10T08:31:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33106.176346 secs 2023-09-10T08:31:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:15 smithi098 ceph-mon[103124]: pgmap v17816: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 368 KiB/s wr, 31 op/s 2023-09-10T08:31:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:15 smithi170 ceph-mon[102832]: pgmap v17816: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 368 KiB/s wr, 31 op/s 2023-09-10T08:31:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:15 smithi082 ceph-mon[98694]: pgmap v17816: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 368 KiB/s wr, 31 op/s 2023-09-10T08:31:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:31:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:31:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:31:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:17 smithi098 ceph-mon[103124]: pgmap v17817: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 14 op/s 2023-09-10T08:31:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:17 smithi170 ceph-mon[102832]: pgmap v17817: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 14 op/s 2023-09-10T08:31:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:17 smithi082 ceph-mon[98694]: pgmap v17817: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 14 op/s 2023-09-10T08:31:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33109.570019 secs 2023-09-10T08:31:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:18 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33110.733635 secs 2023-09-10T08:31:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33109.570019 secs 2023-09-10T08:31:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:18 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33110.733635 secs 2023-09-10T08:31:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33109.570019 secs 2023-09-10T08:31:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:18 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33110.733635 secs 2023-09-10T08:31:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:20 smithi098 ceph-mon[103124]: pgmap v17818: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 13 op/s 2023-09-10T08:31:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33111.176408 secs 2023-09-10T08:31:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:20 smithi170 ceph-mon[102832]: pgmap v17818: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 13 op/s 2023-09-10T08:31:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33111.176408 secs 2023-09-10T08:31:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:20 smithi082 ceph-mon[98694]: pgmap v17818: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 13 op/s 2023-09-10T08:31:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33111.176408 secs 2023-09-10T08:31:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:22 smithi098 ceph-mon[103124]: pgmap v17819: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T08:31:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:22 smithi170 ceph-mon[102832]: pgmap v17819: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T08:31:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:22 smithi082 ceph-mon[98694]: pgmap v17819: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T08:31:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:24 smithi170 ceph-mon[102832]: pgmap v17820: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 213 KiB/s wr, 17 op/s 2023-09-10T08:31:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33114.570079 secs 2023-09-10T08:31:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33115.733694 secs 2023-09-10T08:31:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:24 smithi082 ceph-mon[98694]: pgmap v17820: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 213 KiB/s wr, 17 op/s 2023-09-10T08:31:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33114.570079 secs 2023-09-10T08:31:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33115.733694 secs 2023-09-10T08:31:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:24 smithi098 ceph-mon[103124]: pgmap v17820: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 213 KiB/s wr, 17 op/s 2023-09-10T08:31:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33114.570079 secs 2023-09-10T08:31:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33115.733694 secs 2023-09-10T08:31:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33116.176477 secs 2023-09-10T08:31:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33116.176477 secs 2023-09-10T08:31:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33116.176477 secs 2023-09-10T08:31:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:26 smithi170 ceph-mon[102832]: pgmap v17821: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T08:31:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:31:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:26 smithi082 ceph-mon[98694]: pgmap v17821: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T08:31:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:31:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:26 smithi098 ceph-mon[103124]: pgmap v17821: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 11 op/s 2023-09-10T08:31:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:31:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:31:27.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:31:27.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:31:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:31:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:31:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:31:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:31:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:31:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:31:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:28 smithi170 ceph-mon[102832]: pgmap v17822: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 214 KiB/s wr, 17 op/s 2023-09-10T08:31:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:28 smithi082 ceph-mon[98694]: pgmap v17822: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 214 KiB/s wr, 17 op/s 2023-09-10T08:31:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:28 smithi098 ceph-mon[103124]: pgmap v17822: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 214 KiB/s wr, 17 op/s 2023-09-10T08:31:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33119.570138 secs 2023-09-10T08:31:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33120.733765 secs 2023-09-10T08:31:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33119.570138 secs 2023-09-10T08:31:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33120.733765 secs 2023-09-10T08:31:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33119.570138 secs 2023-09-10T08:31:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33120.733765 secs 2023-09-10T08:31:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:30 smithi170 ceph-mon[102832]: pgmap v17823: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 213 KiB/s wr, 11 op/s 2023-09-10T08:31:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33121.176615 secs 2023-09-10T08:31:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:30 smithi082 ceph-mon[98694]: pgmap v17823: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 213 KiB/s wr, 11 op/s 2023-09-10T08:31:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33121.176615 secs 2023-09-10T08:31:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:30 smithi098 ceph-mon[103124]: pgmap v17823: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 213 KiB/s wr, 11 op/s 2023-09-10T08:31:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33121.176615 secs 2023-09-10T08:31:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:32 smithi170 ceph-mon[102832]: pgmap v17824: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 213 KiB/s wr, 11 op/s 2023-09-10T08:31:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:32 smithi082 ceph-mon[98694]: pgmap v17824: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 213 KiB/s wr, 11 op/s 2023-09-10T08:31:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:32 smithi098 ceph-mon[103124]: pgmap v17824: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 213 KiB/s wr, 11 op/s 2023-09-10T08:31:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:34 smithi170 ceph-mon[102832]: pgmap v17825: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 213 KiB/s wr, 16 op/s 2023-09-10T08:31:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:31:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33125.733847 secs 2023-09-10T08:31:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:34 smithi082 ceph-mon[98694]: pgmap v17825: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 213 KiB/s wr, 16 op/s 2023-09-10T08:31:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:31:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33125.733847 secs 2023-09-10T08:31:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:34 smithi098 ceph-mon[103124]: pgmap v17825: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 213 KiB/s wr, 16 op/s 2023-09-10T08:31:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:31:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33125.733847 secs 2023-09-10T08:31:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33126.176679 secs 2023-09-10T08:31:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33126.176679 secs 2023-09-10T08:31:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33126.176679 secs 2023-09-10T08:31:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:36 smithi170 ceph-mon[102832]: pgmap v17826: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T08:31:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:36 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:31:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:36 smithi082 ceph-mon[98694]: pgmap v17826: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T08:31:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:36 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:31:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:36 smithi098 ceph-mon[103124]: pgmap v17826: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T08:31:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:36 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:31:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:37 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:31:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:31:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:37 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:31:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:31:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:37 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:31:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:31:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:38 smithi170 ceph-mon[102832]: pgmap v17827: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 312 KiB/s wr, 18 op/s 2023-09-10T08:31:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:38 smithi082 ceph-mon[98694]: pgmap v17827: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 312 KiB/s wr, 18 op/s 2023-09-10T08:31:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:38 smithi098 ceph-mon[103124]: pgmap v17827: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 312 KiB/s wr, 18 op/s 2023-09-10T08:31:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33130.733910 secs 2023-09-10T08:31:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33130.733910 secs 2023-09-10T08:31:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33130.733910 secs 2023-09-10T08:31:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:40 smithi170 ceph-mon[102832]: pgmap v17828: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 192 KiB/s wr, 12 op/s 2023-09-10T08:31:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33131.176747 secs 2023-09-10T08:31:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:40 smithi082 ceph-mon[98694]: pgmap v17828: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 192 KiB/s wr, 12 op/s 2023-09-10T08:31:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33131.176747 secs 2023-09-10T08:31:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:40 smithi098 ceph-mon[103124]: pgmap v17828: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 192 KiB/s wr, 12 op/s 2023-09-10T08:31:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33131.176747 secs 2023-09-10T08:31:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:42 smithi170 ceph-mon[102832]: pgmap v17829: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 192 KiB/s wr, 12 op/s 2023-09-10T08:31:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:42 smithi082 ceph-mon[98694]: pgmap v17829: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 192 KiB/s wr, 12 op/s 2023-09-10T08:31:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:42 smithi098 ceph-mon[103124]: pgmap v17829: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 192 KiB/s wr, 12 op/s 2023-09-10T08:31:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:44 smithi170 ceph-mon[102832]: pgmap v17830: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 492 KiB/s wr, 19 op/s 2023-09-10T08:31:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33135.733995 secs 2023-09-10T08:31:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:44 smithi082 ceph-mon[98694]: pgmap v17830: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 492 KiB/s wr, 19 op/s 2023-09-10T08:31:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33135.733995 secs 2023-09-10T08:31:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:44 smithi098 ceph-mon[103124]: pgmap v17830: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 492 KiB/s wr, 19 op/s 2023-09-10T08:31:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33135.733995 secs 2023-09-10T08:31:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33136.176811 secs 2023-09-10T08:31:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:31:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33136.176811 secs 2023-09-10T08:31:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:31:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33136.176811 secs 2023-09-10T08:31:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:31:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:46 smithi170 ceph-mon[102832]: pgmap v17831: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 492 KiB/s wr, 13 op/s 2023-09-10T08:31:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:31:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:46 smithi082 ceph-mon[98694]: pgmap v17831: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 492 KiB/s wr, 13 op/s 2023-09-10T08:31:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:31:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:46 smithi098 ceph-mon[103124]: pgmap v17831: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 492 KiB/s wr, 13 op/s 2023-09-10T08:31:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:31:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:47 smithi170 ceph-mon[102832]: pgmap v17832: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 681 KiB/s wr, 20 op/s 2023-09-10T08:31:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:47 smithi082 ceph-mon[98694]: pgmap v17832: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 681 KiB/s wr, 20 op/s 2023-09-10T08:31:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:47 smithi098 ceph-mon[103124]: pgmap v17832: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 681 KiB/s wr, 20 op/s 2023-09-10T08:31:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33140.734079 secs 2023-09-10T08:31:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33140.734079 secs 2023-09-10T08:31:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33140.734079 secs 2023-09-10T08:31:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:49 smithi082 ceph-mon[98694]: pgmap v17833: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 489 KiB/s wr, 13 op/s 2023-09-10T08:31:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33141.176877 secs 2023-09-10T08:31:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:49 smithi098 ceph-mon[103124]: pgmap v17833: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 489 KiB/s wr, 13 op/s 2023-09-10T08:31:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33141.176877 secs 2023-09-10T08:31:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:49 smithi170 ceph-mon[102832]: pgmap v17833: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 489 KiB/s wr, 13 op/s 2023-09-10T08:31:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33141.176877 secs 2023-09-10T08:31:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:51 smithi082 ceph-mon[98694]: pgmap v17834: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 489 KiB/s wr, 13 op/s 2023-09-10T08:31:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:51 smithi098 ceph-mon[103124]: pgmap v17834: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 489 KiB/s wr, 13 op/s 2023-09-10T08:31:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:51 smithi170 ceph-mon[102832]: pgmap v17834: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 489 KiB/s wr, 13 op/s 2023-09-10T08:31:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:31:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:31:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:31:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:53 smithi098 ceph-mon[103124]: pgmap v17835: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 879 KiB/s wr, 21 op/s 2023-09-10T08:31:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33144.570480 secs 2023-09-10T08:31:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:31:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33145.734137 secs 2023-09-10T08:31:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:53 smithi170 ceph-mon[102832]: pgmap v17835: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 879 KiB/s wr, 21 op/s 2023-09-10T08:31:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33144.570480 secs 2023-09-10T08:31:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:31:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33145.734137 secs 2023-09-10T08:31:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:53 smithi082 ceph-mon[98694]: pgmap v17835: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 879 KiB/s wr, 21 op/s 2023-09-10T08:31:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33144.570480 secs 2023-09-10T08:31:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:31:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33145.734137 secs 2023-09-10T08:31:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33146.176943 secs 2023-09-10T08:31:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33146.176943 secs 2023-09-10T08:31:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33146.176943 secs 2023-09-10T08:31:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:55 smithi098 ceph-mon[103124]: pgmap v17836: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 579 KiB/s wr, 14 op/s 2023-09-10T08:31:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:55 smithi170 ceph-mon[102832]: pgmap v17836: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 579 KiB/s wr, 14 op/s 2023-09-10T08:31:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:55 smithi082 ceph-mon[98694]: pgmap v17836: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 579 KiB/s wr, 14 op/s 2023-09-10T08:31:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:31:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:31:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:31:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:31:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:31:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:31:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:31:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:57 smithi098 ceph-mon[103124]: pgmap v17837: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 807 KiB/s wr, 21 op/s 2023-09-10T08:31:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:57 smithi170 ceph-mon[102832]: pgmap v17837: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 807 KiB/s wr, 21 op/s 2023-09-10T08:31:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:57 smithi082 ceph-mon[98694]: pgmap v17837: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 807 KiB/s wr, 21 op/s 2023-09-10T08:31:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33149.570539 secs 2023-09-10T08:31:59.273 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33150.734214 secs 2023-09-10T08:31:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33149.570539 secs 2023-09-10T08:31:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33150.734214 secs 2023-09-10T08:31:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33149.570539 secs 2023-09-10T08:31:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33150.734214 secs 2023-09-10T08:32:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:59 smithi098 ceph-mon[103124]: pgmap v17838: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 619 KiB/s wr, 14 op/s 2023-09-10T08:32:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:31:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33151.177014 secs 2023-09-10T08:32:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:59 smithi170 ceph-mon[102832]: pgmap v17838: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 619 KiB/s wr, 14 op/s 2023-09-10T08:32:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:31:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33151.177014 secs 2023-09-10T08:32:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:59 smithi082 ceph-mon[98694]: pgmap v17838: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 619 KiB/s wr, 14 op/s 2023-09-10T08:32:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:31:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33151.177014 secs 2023-09-10T08:32:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:01 smithi098 ceph-mon[103124]: pgmap v17839: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 619 KiB/s wr, 14 op/s 2023-09-10T08:32:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:01 smithi170 ceph-mon[102832]: pgmap v17839: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 619 KiB/s wr, 14 op/s 2023-09-10T08:32:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:01 smithi082 ceph-mon[98694]: pgmap v17839: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 619 KiB/s wr, 14 op/s 2023-09-10T08:32:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:03 smithi098 ceph-mon[103124]: pgmap v17840: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 900 KiB/s wr, 20 op/s 2023-09-10T08:32:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33154.570602 secs 2023-09-10T08:32:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33155.734278 secs 2023-09-10T08:32:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:03 smithi170 ceph-mon[102832]: pgmap v17840: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 900 KiB/s wr, 20 op/s 2023-09-10T08:32:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33154.570602 secs 2023-09-10T08:32:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33155.734278 secs 2023-09-10T08:32:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:03 smithi082 ceph-mon[98694]: pgmap v17840: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 900 KiB/s wr, 20 op/s 2023-09-10T08:32:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33154.570602 secs 2023-09-10T08:32:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33155.734278 secs 2023-09-10T08:32:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33156.177075 secs 2023-09-10T08:32:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33156.177075 secs 2023-09-10T08:32:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33156.177075 secs 2023-09-10T08:32:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:05 smithi098 ceph-mon[103124]: pgmap v17841: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 510 KiB/s wr, 13 op/s 2023-09-10T08:32:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:05 smithi170 ceph-mon[102832]: pgmap v17841: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 510 KiB/s wr, 13 op/s 2023-09-10T08:32:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:05 smithi082 ceph-mon[98694]: pgmap v17841: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 510 KiB/s wr, 13 op/s 2023-09-10T08:32:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:08 smithi098 ceph-mon[103124]: pgmap v17842: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 555 KiB/s wr, 33 op/s 2023-09-10T08:32:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:08 smithi170 ceph-mon[102832]: pgmap v17842: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 555 KiB/s wr, 33 op/s 2023-09-10T08:32:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:07 smithi082 ceph-mon[98694]: pgmap v17842: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 555 KiB/s wr, 33 op/s 2023-09-10T08:32:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33159.570663 secs 2023-09-10T08:32:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:09 smithi170 ceph-mon[102832]: osdmap e1183: 12 total, 12 up, 12 in 2023-09-10T08:32:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33160.734336 secs 2023-09-10T08:32:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33159.570663 secs 2023-09-10T08:32:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:09 smithi082 ceph-mon[98694]: osdmap e1183: 12 total, 12 up, 12 in 2023-09-10T08:32:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33160.734336 secs 2023-09-10T08:32:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33159.570663 secs 2023-09-10T08:32:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:09 smithi098 ceph-mon[103124]: osdmap e1183: 12 total, 12 up, 12 in 2023-09-10T08:32:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33160.734336 secs 2023-09-10T08:32:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:10 smithi170 ceph-mon[102832]: pgmap v17844: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 392 KiB/s wr, 31 op/s 2023-09-10T08:32:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33161.177143 secs 2023-09-10T08:32:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:10 smithi082 ceph-mon[98694]: pgmap v17844: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 392 KiB/s wr, 31 op/s 2023-09-10T08:32:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33161.177143 secs 2023-09-10T08:32:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:10 smithi098 ceph-mon[103124]: pgmap v17844: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 392 KiB/s wr, 31 op/s 2023-09-10T08:32:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33161.177143 secs 2023-09-10T08:32:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:12 smithi170 ceph-mon[102832]: pgmap v17845: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 392 KiB/s wr, 31 op/s 2023-09-10T08:32:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:12 smithi082 ceph-mon[98694]: pgmap v17845: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 392 KiB/s wr, 31 op/s 2023-09-10T08:32:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:12 smithi098 ceph-mon[103124]: pgmap v17845: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 392 KiB/s wr, 31 op/s 2023-09-10T08:32:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:14 smithi170 ceph-mon[102832]: pgmap v17846: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 173 KiB/s wr, 31 op/s 2023-09-10T08:32:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33164.570722 secs 2023-09-10T08:32:14.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33165.734395 secs 2023-09-10T08:32:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:14 smithi082 ceph-mon[98694]: pgmap v17846: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 173 KiB/s wr, 31 op/s 2023-09-10T08:32:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33164.570722 secs 2023-09-10T08:32:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33165.734395 secs 2023-09-10T08:32:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:14 smithi098 ceph-mon[103124]: pgmap v17846: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 173 KiB/s wr, 31 op/s 2023-09-10T08:32:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33164.570722 secs 2023-09-10T08:32:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33165.734395 secs 2023-09-10T08:32:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33166.177209 secs 2023-09-10T08:32:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33166.177209 secs 2023-09-10T08:32:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33166.177209 secs 2023-09-10T08:32:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:16 smithi170 ceph-mon[102832]: pgmap v17847: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 173 KiB/s wr, 31 op/s 2023-09-10T08:32:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:16 smithi082 ceph-mon[98694]: pgmap v17847: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 173 KiB/s wr, 31 op/s 2023-09-10T08:32:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:16 smithi098 ceph-mon[103124]: pgmap v17847: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 173 KiB/s wr, 31 op/s 2023-09-10T08:32:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:17 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:32:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:17 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:32:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:17 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:32:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:17 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:32:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:17 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:32:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:17 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:32:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:18 smithi170 ceph-mon[102832]: pgmap v17848: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 14 op/s 2023-09-10T08:32:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:32:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:32:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:18 smithi170 ceph-mon[102832]: osdmap e1184: 12 total, 12 up, 12 in 2023-09-10T08:32:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:18 smithi082 ceph-mon[98694]: pgmap v17848: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 14 op/s 2023-09-10T08:32:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:32:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:32:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:18 smithi082 ceph-mon[98694]: osdmap e1184: 12 total, 12 up, 12 in 2023-09-10T08:32:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:18 smithi098 ceph-mon[103124]: pgmap v17848: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 14 op/s 2023-09-10T08:32:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:32:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:32:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:18 smithi098 ceph-mon[103124]: osdmap e1184: 12 total, 12 up, 12 in 2023-09-10T08:32:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33169.570777 secs 2023-09-10T08:32:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33170.734453 secs 2023-09-10T08:32:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33169.570777 secs 2023-09-10T08:32:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33170.734453 secs 2023-09-10T08:32:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33169.570777 secs 2023-09-10T08:32:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33170.734453 secs 2023-09-10T08:32:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:20 smithi170 ceph-mon[102832]: pgmap v17850: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 14 op/s 2023-09-10T08:32:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33171.177271 secs 2023-09-10T08:32:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:20 smithi082 ceph-mon[98694]: pgmap v17850: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 14 op/s 2023-09-10T08:32:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33171.177271 secs 2023-09-10T08:32:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:20 smithi098 ceph-mon[103124]: pgmap v17850: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 14 op/s 2023-09-10T08:32:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33171.177271 secs 2023-09-10T08:32:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:22 smithi170 ceph-mon[102832]: pgmap v17851: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 14 op/s 2023-09-10T08:32:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:22 smithi082 ceph-mon[98694]: pgmap v17851: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 14 op/s 2023-09-10T08:32:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:22 smithi098 ceph-mon[103124]: pgmap v17851: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 14 op/s 2023-09-10T08:32:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:24 smithi170 ceph-mon[102832]: pgmap v17852: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:32:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33174.570837 secs 2023-09-10T08:32:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33175.734536 secs 2023-09-10T08:32:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:24 smithi082 ceph-mon[98694]: pgmap v17852: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:32:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33174.570837 secs 2023-09-10T08:32:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33175.734536 secs 2023-09-10T08:32:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:24 smithi098 ceph-mon[103124]: pgmap v17852: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:32:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33174.570837 secs 2023-09-10T08:32:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33175.734536 secs 2023-09-10T08:32:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33176.177339 secs 2023-09-10T08:32:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33176.177339 secs 2023-09-10T08:32:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33176.177339 secs 2023-09-10T08:32:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:26 smithi170 ceph-mon[102832]: pgmap v17853: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:32:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:26 smithi082 ceph-mon[98694]: pgmap v17853: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:32:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:26 smithi098 ceph-mon[103124]: pgmap v17853: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:32:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:32:27.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:32:27.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:32:27.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:32:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:32:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:32:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:32:27.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:32:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:32:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:32:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:32:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:32:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:28 smithi170 ceph-mon[102832]: pgmap v17854: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 13 op/s 2023-09-10T08:32:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:28 smithi082 ceph-mon[98694]: pgmap v17854: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 13 op/s 2023-09-10T08:32:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:28 smithi098 ceph-mon[103124]: pgmap v17854: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 13 op/s 2023-09-10T08:32:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33179.570896 secs 2023-09-10T08:32:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33180.734596 secs 2023-09-10T08:32:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33179.570896 secs 2023-09-10T08:32:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33180.734596 secs 2023-09-10T08:32:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33179.570896 secs 2023-09-10T08:32:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33180.734596 secs 2023-09-10T08:32:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:30 smithi170 ceph-mon[102832]: pgmap v17855: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 12 op/s 2023-09-10T08:32:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33181.177404 secs 2023-09-10T08:32:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:30 smithi082 ceph-mon[98694]: pgmap v17855: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 12 op/s 2023-09-10T08:32:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33181.177404 secs 2023-09-10T08:32:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:30 smithi098 ceph-mon[103124]: pgmap v17855: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 12 op/s 2023-09-10T08:32:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33181.177404 secs 2023-09-10T08:32:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:31 smithi170 ceph-mon[102832]: pgmap v17856: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T08:32:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:31 smithi082 ceph-mon[98694]: pgmap v17856: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T08:32:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:31 smithi098 ceph-mon[103124]: pgmap v17856: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T08:32:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:33 smithi082 ceph-mon[98694]: pgmap v17857: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 15 op/s 2023-09-10T08:32:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:33 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33185.734655 secs 2023-09-10T08:32:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:33 smithi098 ceph-mon[103124]: pgmap v17857: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 15 op/s 2023-09-10T08:32:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:33 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33185.734655 secs 2023-09-10T08:32:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:33 smithi170 ceph-mon[102832]: pgmap v17857: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 15 op/s 2023-09-10T08:32:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:33 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33185.734655 secs 2023-09-10T08:32:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33186.177472 secs 2023-09-10T08:32:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33186.177472 secs 2023-09-10T08:32:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33186.177472 secs 2023-09-10T08:32:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:35 smithi098 ceph-mon[103124]: pgmap v17858: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T08:32:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:32:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:35 smithi170 ceph-mon[102832]: pgmap v17858: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T08:32:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:32:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:35 smithi082 ceph-mon[98694]: pgmap v17858: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T08:32:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:32:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:32:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:32:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:32:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:32:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:32:37.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:32:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:32:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:32:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:32:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:37 smithi098 ceph-mon[103124]: pgmap v17859: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 305 KiB/s wr, 16 op/s 2023-09-10T08:32:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:37 smithi170 ceph-mon[102832]: pgmap v17859: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 305 KiB/s wr, 16 op/s 2023-09-10T08:32:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:37 smithi082 ceph-mon[98694]: pgmap v17859: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 305 KiB/s wr, 16 op/s 2023-09-10T08:32:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:38 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33190.772751 secs 2023-09-10T08:32:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:38 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33190.772751 secs 2023-09-10T08:32:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:38 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33190.772751 secs 2023-09-10T08:32:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:39 smithi098 ceph-mon[103124]: pgmap v17860: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T08:32:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33191.177487 secs 2023-09-10T08:32:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:39 smithi170 ceph-mon[102832]: pgmap v17860: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T08:32:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33191.177487 secs 2023-09-10T08:32:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:39 smithi082 ceph-mon[98694]: pgmap v17860: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T08:32:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33191.177487 secs 2023-09-10T08:32:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:41 smithi098 ceph-mon[103124]: pgmap v17861: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T08:32:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:41 smithi170 ceph-mon[102832]: pgmap v17861: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T08:32:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:41 smithi082 ceph-mon[98694]: pgmap v17861: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 186 KiB/s wr, 11 op/s 2023-09-10T08:32:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:43 smithi098 ceph-mon[103124]: pgmap v17862: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 446 KiB/s wr, 18 op/s 2023-09-10T08:32:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33195.772811 secs 2023-09-10T08:32:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:43 smithi170 ceph-mon[102832]: pgmap v17862: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 446 KiB/s wr, 18 op/s 2023-09-10T08:32:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33195.772811 secs 2023-09-10T08:32:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:43 smithi082 ceph-mon[98694]: pgmap v17862: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 446 KiB/s wr, 18 op/s 2023-09-10T08:32:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33195.772811 secs 2023-09-10T08:32:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33196.177551 secs 2023-09-10T08:32:45.292 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:32:45.292 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:32:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33196.177551 secs 2023-09-10T08:32:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:32:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:32:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33196.177551 secs 2023-09-10T08:32:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:32:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:32:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:45 smithi098 ceph-mon[103124]: pgmap v17863: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 446 KiB/s wr, 13 op/s 2023-09-10T08:32:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:32:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:32:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:45 smithi170 ceph-mon[102832]: pgmap v17863: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 446 KiB/s wr, 13 op/s 2023-09-10T08:32:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:32:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:32:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:45 smithi082 ceph-mon[98694]: pgmap v17863: 97 pgs: 97 active+clean; 11 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 446 KiB/s wr, 13 op/s 2023-09-10T08:32:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:32:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:32:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:47 smithi098 ceph-mon[103124]: pgmap v17864: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 661 KiB/s wr, 21 op/s 2023-09-10T08:32:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:47 smithi170 ceph-mon[102832]: pgmap v17864: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 661 KiB/s wr, 21 op/s 2023-09-10T08:32:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:47 smithi082 ceph-mon[98694]: pgmap v17864: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 661 KiB/s wr, 21 op/s 2023-09-10T08:32:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:48 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33200.772881 secs 2023-09-10T08:32:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:48 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33200.772881 secs 2023-09-10T08:32:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:48 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33200.772881 secs 2023-09-10T08:32:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:50 smithi098 ceph-mon[103124]: pgmap v17865: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 475 KiB/s wr, 15 op/s 2023-09-10T08:32:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33201.177617 secs 2023-09-10T08:32:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:50 smithi170 ceph-mon[102832]: pgmap v17865: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 475 KiB/s wr, 15 op/s 2023-09-10T08:32:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33201.177617 secs 2023-09-10T08:32:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:50 smithi082 ceph-mon[98694]: pgmap v17865: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 475 KiB/s wr, 15 op/s 2023-09-10T08:32:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33201.177617 secs 2023-09-10T08:32:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:52 smithi170 ceph-mon[102832]: pgmap v17866: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 475 KiB/s wr, 15 op/s 2023-09-10T08:32:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:52 smithi082 ceph-mon[98694]: pgmap v17866: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 475 KiB/s wr, 15 op/s 2023-09-10T08:32:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:52 smithi098 ceph-mon[103124]: pgmap v17866: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 475 KiB/s wr, 15 op/s 2023-09-10T08:32:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:54 smithi170 ceph-mon[102832]: pgmap v17867: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 736 KiB/s wr, 22 op/s 2023-09-10T08:32:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:32:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:32:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33205.772945 secs 2023-09-10T08:32:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:54 smithi082 ceph-mon[98694]: pgmap v17867: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 736 KiB/s wr, 22 op/s 2023-09-10T08:32:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:32:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:32:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33205.772945 secs 2023-09-10T08:32:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:54 smithi098 ceph-mon[103124]: pgmap v17867: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 736 KiB/s wr, 22 op/s 2023-09-10T08:32:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:32:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:32:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33205.772945 secs 2023-09-10T08:32:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33206.177686 secs 2023-09-10T08:32:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33206.177686 secs 2023-09-10T08:32:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33206.177686 secs 2023-09-10T08:32:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:56 smithi170 ceph-mon[102832]: pgmap v17868: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 476 KiB/s wr, 15 op/s 2023-09-10T08:32:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:56 smithi082 ceph-mon[98694]: pgmap v17868: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 476 KiB/s wr, 15 op/s 2023-09-10T08:32:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:56 smithi098 ceph-mon[103124]: pgmap v17868: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 476 KiB/s wr, 15 op/s 2023-09-10T08:32:57.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:32:57.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:32:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:32:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:32:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:32:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:32:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:58 smithi170 ceph-mon[102832]: pgmap v17869: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 711 KiB/s wr, 22 op/s 2023-09-10T08:32:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:58 smithi082 ceph-mon[98694]: pgmap v17869: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 711 KiB/s wr, 22 op/s 2023-09-10T08:32:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:58 smithi098 ceph-mon[103124]: pgmap v17869: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 711 KiB/s wr, 22 op/s 2023-09-10T08:32:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33209.571376 secs 2023-09-10T08:32:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:32:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33210.773003 secs 2023-09-10T08:32:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33209.571376 secs 2023-09-10T08:32:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:32:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33210.773003 secs 2023-09-10T08:32:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33209.571376 secs 2023-09-10T08:32:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:32:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33210.773003 secs 2023-09-10T08:33:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:00 smithi170 ceph-mon[102832]: pgmap v17870: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 495 KiB/s wr, 14 op/s 2023-09-10T08:33:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33211.177751 secs 2023-09-10T08:33:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:00 smithi082 ceph-mon[98694]: pgmap v17870: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 495 KiB/s wr, 14 op/s 2023-09-10T08:33:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33211.177751 secs 2023-09-10T08:33:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:00 smithi098 ceph-mon[103124]: pgmap v17870: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 495 KiB/s wr, 14 op/s 2023-09-10T08:33:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33211.177751 secs 2023-09-10T08:33:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:01 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:33:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:01 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:33:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:01 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:01 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:33:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:01 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:33:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:01 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:01 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:33:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:01 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:33:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:01 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:02 smithi170 ceph-mon[102832]: pgmap v17871: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 496 KiB/s wr, 14 op/s 2023-09-10T08:33:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:02 smithi082 ceph-mon[98694]: pgmap v17871: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 496 KiB/s wr, 14 op/s 2023-09-10T08:33:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:02 smithi098 ceph-mon[103124]: pgmap v17871: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 496 KiB/s wr, 14 op/s 2023-09-10T08:33:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:04 smithi170 ceph-mon[102832]: pgmap v17872: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 796 KiB/s wr, 20 op/s 2023-09-10T08:33:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33214.571435 secs 2023-09-10T08:33:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33215.773061 secs 2023-09-10T08:33:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:04 smithi082 ceph-mon[98694]: pgmap v17872: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 796 KiB/s wr, 20 op/s 2023-09-10T08:33:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33214.571435 secs 2023-09-10T08:33:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33215.773061 secs 2023-09-10T08:33:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:04 smithi098 ceph-mon[103124]: pgmap v17872: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 796 KiB/s wr, 20 op/s 2023-09-10T08:33:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33214.571435 secs 2023-09-10T08:33:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33215.773061 secs 2023-09-10T08:33:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33216.177821 secs 2023-09-10T08:33:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:33:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33216.177821 secs 2023-09-10T08:33:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:33:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33216.177821 secs 2023-09-10T08:33:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:33:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:06 smithi170 ceph-mon[102832]: pgmap v17873: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 536 KiB/s wr, 13 op/s 2023-09-10T08:33:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:33:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:06 smithi082 ceph-mon[98694]: pgmap v17873: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 536 KiB/s wr, 13 op/s 2023-09-10T08:33:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:33:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:06 smithi098 ceph-mon[103124]: pgmap v17873: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 536 KiB/s wr, 13 op/s 2023-09-10T08:33:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:33:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:08 smithi170 ceph-mon[102832]: pgmap v17874: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 722 KiB/s wr, 33 op/s 2023-09-10T08:33:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:08 smithi082 ceph-mon[98694]: pgmap v17874: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 722 KiB/s wr, 33 op/s 2023-09-10T08:33:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:08 smithi098 ceph-mon[103124]: pgmap v17874: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 722 KiB/s wr, 33 op/s 2023-09-10T08:33:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33219.571495 secs 2023-09-10T08:33:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:09 smithi170 ceph-mon[102832]: osdmap e1185: 12 total, 12 up, 12 in 2023-09-10T08:33:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33220.773143 secs 2023-09-10T08:33:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33219.571495 secs 2023-09-10T08:33:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:09 smithi082 ceph-mon[98694]: osdmap e1185: 12 total, 12 up, 12 in 2023-09-10T08:33:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33220.773143 secs 2023-09-10T08:33:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33219.571495 secs 2023-09-10T08:33:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:09 smithi098 ceph-mon[103124]: osdmap e1185: 12 total, 12 up, 12 in 2023-09-10T08:33:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33220.773143 secs 2023-09-10T08:33:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:10 smithi170 ceph-mon[102832]: pgmap v17876: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 584 KiB/s wr, 31 op/s 2023-09-10T08:33:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33221.177887 secs 2023-09-10T08:33:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:10 smithi082 ceph-mon[98694]: pgmap v17876: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 584 KiB/s wr, 31 op/s 2023-09-10T08:33:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33221.177887 secs 2023-09-10T08:33:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:10 smithi098 ceph-mon[103124]: pgmap v17876: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 584 KiB/s wr, 31 op/s 2023-09-10T08:33:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33221.177887 secs 2023-09-10T08:33:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:11 smithi170 ceph-mon[102832]: pgmap v17877: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 584 KiB/s wr, 31 op/s 2023-09-10T08:33:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:11 smithi082 ceph-mon[98694]: pgmap v17877: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 584 KiB/s wr, 31 op/s 2023-09-10T08:33:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:11 smithi098 ceph-mon[103124]: pgmap v17877: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 584 KiB/s wr, 31 op/s 2023-09-10T08:33:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:13 smithi098 ceph-mon[103124]: pgmap v17878: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 366 KiB/s wr, 31 op/s 2023-09-10T08:33:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33224.571556 secs 2023-09-10T08:33:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:13 smithi098 ceph-mon[103124]: osdmap e1186: 12 total, 12 up, 12 in 2023-09-10T08:33:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:33:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33225.773205 secs 2023-09-10T08:33:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:13 smithi170 ceph-mon[102832]: pgmap v17878: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 366 KiB/s wr, 31 op/s 2023-09-10T08:33:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33224.571556 secs 2023-09-10T08:33:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:13 smithi170 ceph-mon[102832]: osdmap e1186: 12 total, 12 up, 12 in 2023-09-10T08:33:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:33:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33225.773205 secs 2023-09-10T08:33:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:13 smithi082 ceph-mon[98694]: pgmap v17878: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 366 KiB/s wr, 31 op/s 2023-09-10T08:33:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33224.571556 secs 2023-09-10T08:33:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:13 smithi082 ceph-mon[98694]: osdmap e1186: 12 total, 12 up, 12 in 2023-09-10T08:33:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:33:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33225.773205 secs 2023-09-10T08:33:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33226.177954 secs 2023-09-10T08:33:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33226.177954 secs 2023-09-10T08:33:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33226.177954 secs 2023-09-10T08:33:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:15 smithi098 ceph-mon[103124]: pgmap v17880: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 178 KiB/s wr, 9 op/s 2023-09-10T08:33:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:15 smithi170 ceph-mon[102832]: pgmap v17880: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 178 KiB/s wr, 9 op/s 2023-09-10T08:33:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:15 smithi082 ceph-mon[98694]: pgmap v17880: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 178 KiB/s wr, 9 op/s 2023-09-10T08:33:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:33:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:33:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:33:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:17 smithi098 ceph-mon[103124]: pgmap v17881: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 214 KiB/s wr, 18 op/s 2023-09-10T08:33:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:17 smithi170 ceph-mon[102832]: pgmap v17881: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 214 KiB/s wr, 18 op/s 2023-09-10T08:33:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:17 smithi082 ceph-mon[98694]: pgmap v17881: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 214 KiB/s wr, 18 op/s 2023-09-10T08:33:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33229.571620 secs 2023-09-10T08:33:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33230.773263 secs 2023-09-10T08:33:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33229.571620 secs 2023-09-10T08:33:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33230.773263 secs 2023-09-10T08:33:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33229.571620 secs 2023-09-10T08:33:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33230.773263 secs 2023-09-10T08:33:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:20 smithi170 ceph-mon[102832]: pgmap v17882: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 173 KiB/s wr, 14 op/s 2023-09-10T08:33:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33231.178019 secs 2023-09-10T08:33:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:20 smithi082 ceph-mon[98694]: pgmap v17882: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 173 KiB/s wr, 14 op/s 2023-09-10T08:33:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33231.178019 secs 2023-09-10T08:33:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:20 smithi098 ceph-mon[103124]: pgmap v17882: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 173 KiB/s wr, 14 op/s 2023-09-10T08:33:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33231.178019 secs 2023-09-10T08:33:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:22 smithi170 ceph-mon[102832]: pgmap v17883: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 173 KiB/s wr, 14 op/s 2023-09-10T08:33:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:22 smithi082 ceph-mon[98694]: pgmap v17883: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 173 KiB/s wr, 14 op/s 2023-09-10T08:33:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:22 smithi098 ceph-mon[103124]: pgmap v17883: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 173 KiB/s wr, 14 op/s 2023-09-10T08:33:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:24 smithi170 ceph-mon[102832]: pgmap v17884: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T08:33:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33234.571679 secs 2023-09-10T08:33:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33235.773351 secs 2023-09-10T08:33:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:24 smithi082 ceph-mon[98694]: pgmap v17884: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T08:33:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33234.571679 secs 2023-09-10T08:33:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33235.773351 secs 2023-09-10T08:33:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:24 smithi098 ceph-mon[103124]: pgmap v17884: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T08:33:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33234.571679 secs 2023-09-10T08:33:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33235.773351 secs 2023-09-10T08:33:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33236.178086 secs 2023-09-10T08:33:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33236.178086 secs 2023-09-10T08:33:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33236.178086 secs 2023-09-10T08:33:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:26 smithi170 ceph-mon[102832]: pgmap v17885: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 107 KiB/s wr, 12 op/s 2023-09-10T08:33:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:26 smithi082 ceph-mon[98694]: pgmap v17885: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 107 KiB/s wr, 12 op/s 2023-09-10T08:33:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:26 smithi098 ceph-mon[103124]: pgmap v17885: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 107 KiB/s wr, 12 op/s 2023-09-10T08:33:27.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:33:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:33:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:33:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:28 smithi170 ceph-mon[102832]: pgmap v17886: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 219 KiB/s wr, 17 op/s 2023-09-10T08:33:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:33:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:33:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:33:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:28 smithi082 ceph-mon[98694]: pgmap v17886: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 219 KiB/s wr, 17 op/s 2023-09-10T08:33:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:33:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:33:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:33:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:28 smithi098 ceph-mon[103124]: pgmap v17886: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 219 KiB/s wr, 17 op/s 2023-09-10T08:33:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:33:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:33:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:33:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33239.571742 secs 2023-09-10T08:33:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33240.773413 secs 2023-09-10T08:33:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33239.571742 secs 2023-09-10T08:33:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33240.773413 secs 2023-09-10T08:33:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33239.571742 secs 2023-09-10T08:33:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33240.773413 secs 2023-09-10T08:33:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:30 smithi170 ceph-mon[102832]: pgmap v17887: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T08:33:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33241.178152 secs 2023-09-10T08:33:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:30 smithi082 ceph-mon[98694]: pgmap v17887: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T08:33:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33241.178152 secs 2023-09-10T08:33:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:30 smithi098 ceph-mon[103124]: pgmap v17887: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T08:33:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33241.178152 secs 2023-09-10T08:33:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:32 smithi170 ceph-mon[102832]: pgmap v17888: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T08:33:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:32 smithi082 ceph-mon[98694]: pgmap v17888: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T08:33:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:32 smithi098 ceph-mon[103124]: pgmap v17888: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T08:33:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:34 smithi170 ceph-mon[102832]: pgmap v17889: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 193 KiB/s wr, 16 op/s 2023-09-10T08:33:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:33:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33245.773474 secs 2023-09-10T08:33:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:34 smithi082 ceph-mon[98694]: pgmap v17889: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 193 KiB/s wr, 16 op/s 2023-09-10T08:33:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:33:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33245.773474 secs 2023-09-10T08:33:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:34 smithi098 ceph-mon[103124]: pgmap v17889: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 193 KiB/s wr, 16 op/s 2023-09-10T08:33:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:33:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33245.773474 secs 2023-09-10T08:33:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33246.178222 secs 2023-09-10T08:33:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33246.178222 secs 2023-09-10T08:33:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33246.178222 secs 2023-09-10T08:33:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:36 smithi170 ceph-mon[102832]: pgmap v17890: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T08:33:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:36 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:33:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:36 smithi082 ceph-mon[98694]: pgmap v17890: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T08:33:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:36 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:33:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:36 smithi098 ceph-mon[103124]: pgmap v17890: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T08:33:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:36 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:33:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:37 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:33:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:33:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:37 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:33:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:33:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:37 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:33:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:33:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:38 smithi170 ceph-mon[102832]: pgmap v17891: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 304 KiB/s wr, 17 op/s 2023-09-10T08:33:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:38 smithi082 ceph-mon[98694]: pgmap v17891: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 304 KiB/s wr, 17 op/s 2023-09-10T08:33:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:38 smithi098 ceph-mon[103124]: pgmap v17891: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 304 KiB/s wr, 17 op/s 2023-09-10T08:33:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33250.773501 secs 2023-09-10T08:33:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33250.773501 secs 2023-09-10T08:33:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33250.773501 secs 2023-09-10T08:33:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:40 smithi170 ceph-mon[102832]: pgmap v17892: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T08:33:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33251.178289 secs 2023-09-10T08:33:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:40 smithi082 ceph-mon[98694]: pgmap v17892: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T08:33:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33251.178289 secs 2023-09-10T08:33:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:40 smithi098 ceph-mon[103124]: pgmap v17892: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T08:33:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33251.178289 secs 2023-09-10T08:33:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:42 smithi170 ceph-mon[102832]: pgmap v17893: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T08:33:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:42 smithi082 ceph-mon[98694]: pgmap v17893: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T08:33:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:42 smithi098 ceph-mon[103124]: pgmap v17893: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 11 op/s 2023-09-10T08:33:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:44 smithi170 ceph-mon[102832]: pgmap v17894: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 458 KiB/s wr, 18 op/s 2023-09-10T08:33:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33255.773559 secs 2023-09-10T08:33:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:33:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:44 smithi082 ceph-mon[98694]: pgmap v17894: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 458 KiB/s wr, 18 op/s 2023-09-10T08:33:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33255.773559 secs 2023-09-10T08:33:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:33:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:44 smithi098 ceph-mon[103124]: pgmap v17894: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 458 KiB/s wr, 18 op/s 2023-09-10T08:33:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33255.773559 secs 2023-09-10T08:33:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:33:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33256.178355 secs 2023-09-10T08:33:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33256.178355 secs 2023-09-10T08:33:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33256.178355 secs 2023-09-10T08:33:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:46 smithi170 ceph-mon[102832]: pgmap v17895: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 458 KiB/s wr, 13 op/s 2023-09-10T08:33:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:33:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:46 smithi082 ceph-mon[98694]: pgmap v17895: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 458 KiB/s wr, 13 op/s 2023-09-10T08:33:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:33:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:46 smithi098 ceph-mon[103124]: pgmap v17895: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 458 KiB/s wr, 13 op/s 2023-09-10T08:33:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:33:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:48 smithi170 ceph-mon[102832]: pgmap v17896: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 666 KiB/s wr, 20 op/s 2023-09-10T08:33:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:48 smithi082 ceph-mon[98694]: pgmap v17896: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 666 KiB/s wr, 20 op/s 2023-09-10T08:33:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:48 smithi098 ceph-mon[103124]: pgmap v17896: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 666 KiB/s wr, 20 op/s 2023-09-10T08:33:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33260.773618 secs 2023-09-10T08:33:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33260.773618 secs 2023-09-10T08:33:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33260.773618 secs 2023-09-10T08:33:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:50 smithi170 ceph-mon[102832]: pgmap v17897: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 481 KiB/s wr, 14 op/s 2023-09-10T08:33:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33261.178423 secs 2023-09-10T08:33:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:50 smithi082 ceph-mon[98694]: pgmap v17897: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 481 KiB/s wr, 14 op/s 2023-09-10T08:33:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33261.178423 secs 2023-09-10T08:33:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:50 smithi098 ceph-mon[103124]: pgmap v17897: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 481 KiB/s wr, 14 op/s 2023-09-10T08:33:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33261.178423 secs 2023-09-10T08:33:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:52 smithi170 ceph-mon[102832]: pgmap v17898: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 481 KiB/s wr, 14 op/s 2023-09-10T08:33:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:33:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:52 smithi082 ceph-mon[98694]: pgmap v17898: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 481 KiB/s wr, 14 op/s 2023-09-10T08:33:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:33:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:52 smithi098 ceph-mon[103124]: pgmap v17898: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 481 KiB/s wr, 14 op/s 2023-09-10T08:33:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:33:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:54 smithi170 ceph-mon[102832]: pgmap v17899: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 868 KiB/s wr, 21 op/s 2023-09-10T08:33:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33264.572057 secs 2023-09-10T08:33:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:33:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33265.773679 secs 2023-09-10T08:33:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:54 smithi082 ceph-mon[98694]: pgmap v17899: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 868 KiB/s wr, 21 op/s 2023-09-10T08:33:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33264.572057 secs 2023-09-10T08:33:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:33:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33265.773679 secs 2023-09-10T08:33:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:54 smithi098 ceph-mon[103124]: pgmap v17899: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 868 KiB/s wr, 21 op/s 2023-09-10T08:33:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33264.572057 secs 2023-09-10T08:33:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:33:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33265.773679 secs 2023-09-10T08:33:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33266.178488 secs 2023-09-10T08:33:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:55 smithi170 ceph-mon[102832]: pgmap v17900: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 596 KiB/s wr, 14 op/s 2023-09-10T08:33:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33266.178488 secs 2023-09-10T08:33:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:55 smithi082 ceph-mon[98694]: pgmap v17900: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 596 KiB/s wr, 14 op/s 2023-09-10T08:33:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33266.178488 secs 2023-09-10T08:33:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:55 smithi098 ceph-mon[103124]: pgmap v17900: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 596 KiB/s wr, 14 op/s 2023-09-10T08:33:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:33:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:33:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:33:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:33:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:33:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:33:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:33:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:57 smithi170 ceph-mon[102832]: pgmap v17901: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 839 KiB/s wr, 21 op/s 2023-09-10T08:33:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:57 smithi082 ceph-mon[98694]: pgmap v17901: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 839 KiB/s wr, 21 op/s 2023-09-10T08:33:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:57 smithi098 ceph-mon[103124]: pgmap v17901: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 839 KiB/s wr, 21 op/s 2023-09-10T08:33:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33269.572149 secs 2023-09-10T08:33:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:58 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33270.773737 secs 2023-09-10T08:33:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33269.572149 secs 2023-09-10T08:33:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:58 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33270.773737 secs 2023-09-10T08:33:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33269.572149 secs 2023-09-10T08:33:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:58 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33270.773737 secs 2023-09-10T08:34:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:59 smithi098 ceph-mon[103124]: pgmap v17902: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 631 KiB/s wr, 14 op/s 2023-09-10T08:34:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:33:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33271.178556 secs 2023-09-10T08:34:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:59 smithi170 ceph-mon[102832]: pgmap v17902: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 631 KiB/s wr, 14 op/s 2023-09-10T08:34:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:33:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33271.178556 secs 2023-09-10T08:34:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:59 smithi082 ceph-mon[98694]: pgmap v17902: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 631 KiB/s wr, 14 op/s 2023-09-10T08:34:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:33:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33271.178556 secs 2023-09-10T08:34:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:01 smithi098 ceph-mon[103124]: pgmap v17903: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 631 KiB/s wr, 14 op/s 2023-09-10T08:34:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:01 smithi170 ceph-mon[102832]: pgmap v17903: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 631 KiB/s wr, 14 op/s 2023-09-10T08:34:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:01 smithi082 ceph-mon[98694]: pgmap v17903: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 631 KiB/s wr, 14 op/s 2023-09-10T08:34:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:03 smithi098 ceph-mon[103124]: pgmap v17904: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 799 KiB/s wr, 20 op/s 2023-09-10T08:34:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33274.572211 secs 2023-09-10T08:34:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:03 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33275.773822 secs 2023-09-10T08:34:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:03 smithi170 ceph-mon[102832]: pgmap v17904: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 799 KiB/s wr, 20 op/s 2023-09-10T08:34:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33274.572211 secs 2023-09-10T08:34:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:03 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33275.773822 secs 2023-09-10T08:34:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:03 smithi082 ceph-mon[98694]: pgmap v17904: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 799 KiB/s wr, 20 op/s 2023-09-10T08:34:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33274.572211 secs 2023-09-10T08:34:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:03 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33275.773822 secs 2023-09-10T08:34:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33276.178619 secs 2023-09-10T08:34:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33276.178619 secs 2023-09-10T08:34:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33276.178619 secs 2023-09-10T08:34:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:06 smithi098 ceph-mon[103124]: pgmap v17905: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 411 KiB/s wr, 13 op/s 2023-09-10T08:34:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:06 smithi170 ceph-mon[102832]: pgmap v17905: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 411 KiB/s wr, 13 op/s 2023-09-10T08:34:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:06 smithi082 ceph-mon[98694]: pgmap v17905: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 411 KiB/s wr, 13 op/s 2023-09-10T08:34:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:08 smithi170 ceph-mon[102832]: pgmap v17906: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 570 KiB/s wr, 32 op/s 2023-09-10T08:34:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:08 smithi082 ceph-mon[98694]: pgmap v17906: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 570 KiB/s wr, 32 op/s 2023-09-10T08:34:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:08 smithi098 ceph-mon[103124]: pgmap v17906: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 570 KiB/s wr, 32 op/s 2023-09-10T08:34:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33279.572335 secs 2023-09-10T08:34:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:09 smithi170 ceph-mon[102832]: osdmap e1187: 12 total, 12 up, 12 in 2023-09-10T08:34:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33280.773904 secs 2023-09-10T08:34:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33279.572335 secs 2023-09-10T08:34:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:09 smithi082 ceph-mon[98694]: osdmap e1187: 12 total, 12 up, 12 in 2023-09-10T08:34:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33280.773904 secs 2023-09-10T08:34:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33279.572335 secs 2023-09-10T08:34:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:09 smithi098 ceph-mon[103124]: osdmap e1187: 12 total, 12 up, 12 in 2023-09-10T08:34:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33280.773904 secs 2023-09-10T08:34:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:10 smithi170 ceph-mon[102832]: pgmap v17908: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 393 KiB/s wr, 31 op/s 2023-09-10T08:34:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33281.178686 secs 2023-09-10T08:34:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:10 smithi082 ceph-mon[98694]: pgmap v17908: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 393 KiB/s wr, 31 op/s 2023-09-10T08:34:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33281.178686 secs 2023-09-10T08:34:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:10 smithi098 ceph-mon[103124]: pgmap v17908: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 393 KiB/s wr, 31 op/s 2023-09-10T08:34:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33281.178686 secs 2023-09-10T08:34:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:12 smithi170 ceph-mon[102832]: pgmap v17909: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 393 KiB/s wr, 31 op/s 2023-09-10T08:34:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:12 smithi082 ceph-mon[98694]: pgmap v17909: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 393 KiB/s wr, 31 op/s 2023-09-10T08:34:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:12 smithi098 ceph-mon[103124]: pgmap v17909: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 393 KiB/s wr, 31 op/s 2023-09-10T08:34:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:14 smithi170 ceph-mon[102832]: pgmap v17910: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 290 KiB/s wr, 30 op/s 2023-09-10T08:34:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33284.572394 secs 2023-09-10T08:34:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33285.773966 secs 2023-09-10T08:34:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:14 smithi082 ceph-mon[98694]: pgmap v17910: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 290 KiB/s wr, 30 op/s 2023-09-10T08:34:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33284.572394 secs 2023-09-10T08:34:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33285.773966 secs 2023-09-10T08:34:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:14 smithi098 ceph-mon[103124]: pgmap v17910: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 290 KiB/s wr, 30 op/s 2023-09-10T08:34:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33284.572394 secs 2023-09-10T08:34:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33285.773966 secs 2023-09-10T08:34:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33286.178750 secs 2023-09-10T08:34:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33286.178750 secs 2023-09-10T08:34:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33286.178750 secs 2023-09-10T08:34:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:16 smithi170 ceph-mon[102832]: pgmap v17911: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 290 KiB/s wr, 30 op/s 2023-09-10T08:34:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:16 smithi082 ceph-mon[98694]: pgmap v17911: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 290 KiB/s wr, 30 op/s 2023-09-10T08:34:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:16 smithi098 ceph-mon[103124]: pgmap v17911: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 290 KiB/s wr, 30 op/s 2023-09-10T08:34:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:17 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:34:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:17 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:34:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:17 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:34:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:17 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:34:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:17 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:34:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:17 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:34:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:18 smithi170 ceph-mon[102832]: pgmap v17912: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 14 op/s 2023-09-10T08:34:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:34:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:34:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:18 smithi170 ceph-mon[102832]: osdmap e1188: 12 total, 12 up, 12 in 2023-09-10T08:34:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:18 smithi082 ceph-mon[98694]: pgmap v17912: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 14 op/s 2023-09-10T08:34:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:34:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:34:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:18 smithi082 ceph-mon[98694]: osdmap e1188: 12 total, 12 up, 12 in 2023-09-10T08:34:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:18 smithi098 ceph-mon[103124]: pgmap v17912: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 14 op/s 2023-09-10T08:34:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:34:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:34:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:18 smithi098 ceph-mon[103124]: osdmap e1188: 12 total, 12 up, 12 in 2023-09-10T08:34:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33289.572453 secs 2023-09-10T08:34:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33290.774024 secs 2023-09-10T08:34:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33289.572453 secs 2023-09-10T08:34:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33290.774024 secs 2023-09-10T08:34:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33289.572453 secs 2023-09-10T08:34:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33290.774024 secs 2023-09-10T08:34:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:20 smithi170 ceph-mon[102832]: pgmap v17914: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 14 op/s 2023-09-10T08:34:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33291.178817 secs 2023-09-10T08:34:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:20 smithi082 ceph-mon[98694]: pgmap v17914: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 14 op/s 2023-09-10T08:34:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33291.178817 secs 2023-09-10T08:34:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:20 smithi098 ceph-mon[103124]: pgmap v17914: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 14 op/s 2023-09-10T08:34:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33291.178817 secs 2023-09-10T08:34:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:22 smithi170 ceph-mon[102832]: pgmap v17915: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 14 op/s 2023-09-10T08:34:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:22 smithi082 ceph-mon[98694]: pgmap v17915: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 14 op/s 2023-09-10T08:34:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:22 smithi098 ceph-mon[103124]: pgmap v17915: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 14 op/s 2023-09-10T08:34:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:24 smithi170 ceph-mon[102832]: pgmap v17916: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:34:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33294.572563 secs 2023-09-10T08:34:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33295.774103 secs 2023-09-10T08:34:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:24 smithi082 ceph-mon[98694]: pgmap v17916: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:34:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33294.572563 secs 2023-09-10T08:34:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33295.774103 secs 2023-09-10T08:34:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:24 smithi098 ceph-mon[103124]: pgmap v17916: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:34:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33294.572563 secs 2023-09-10T08:34:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33295.774103 secs 2023-09-10T08:34:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33296.178881 secs 2023-09-10T08:34:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33296.178881 secs 2023-09-10T08:34:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33296.178881 secs 2023-09-10T08:34:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:26 smithi170 ceph-mon[102832]: pgmap v17917: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:34:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:26 smithi082 ceph-mon[98694]: pgmap v17917: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:34:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:26 smithi098 ceph-mon[103124]: pgmap v17917: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:34:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:28 smithi170 ceph-mon[102832]: pgmap v17918: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 142 KiB/s wr, 12 op/s 2023-09-10T08:34:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:34:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:28 smithi082 ceph-mon[98694]: pgmap v17918: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 142 KiB/s wr, 12 op/s 2023-09-10T08:34:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:34:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:28 smithi098 ceph-mon[103124]: pgmap v17918: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 142 KiB/s wr, 12 op/s 2023-09-10T08:34:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:34:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33299.572622 secs 2023-09-10T08:34:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:34:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:34:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:34:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33300.774156 secs 2023-09-10T08:34:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33299.572622 secs 2023-09-10T08:34:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:34:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:34:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:34:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33300.774156 secs 2023-09-10T08:34:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33299.572622 secs 2023-09-10T08:34:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:34:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:34:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:34:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33300.774156 secs 2023-09-10T08:34:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:30 smithi170 ceph-mon[102832]: pgmap v17919: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 12 op/s 2023-09-10T08:34:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33301.178946 secs 2023-09-10T08:34:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:30 smithi082 ceph-mon[98694]: pgmap v17919: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 12 op/s 2023-09-10T08:34:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33301.178946 secs 2023-09-10T08:34:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:30 smithi098 ceph-mon[103124]: pgmap v17919: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 12 op/s 2023-09-10T08:34:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33301.178946 secs 2023-09-10T08:34:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:32 smithi170 ceph-mon[102832]: pgmap v17920: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T08:34:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:32 smithi082 ceph-mon[98694]: pgmap v17920: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T08:34:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:32 smithi098 ceph-mon[103124]: pgmap v17920: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T08:34:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:34 smithi170 ceph-mon[102832]: pgmap v17921: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 15 op/s 2023-09-10T08:34:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33305.774216 secs 2023-09-10T08:34:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:34 smithi082 ceph-mon[98694]: pgmap v17921: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 15 op/s 2023-09-10T08:34:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33305.774216 secs 2023-09-10T08:34:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:34 smithi098 ceph-mon[103124]: pgmap v17921: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 119 KiB/s wr, 15 op/s 2023-09-10T08:34:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33305.774216 secs 2023-09-10T08:34:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33306.179009 secs 2023-09-10T08:34:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:34:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33306.179009 secs 2023-09-10T08:34:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:34:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33306.179009 secs 2023-09-10T08:34:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:34:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:36 smithi170 ceph-mon[102832]: pgmap v17922: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T08:34:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:34:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:34:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:34:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:36 smithi082 ceph-mon[98694]: pgmap v17922: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T08:34:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:34:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:34:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:34:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:36 smithi098 ceph-mon[103124]: pgmap v17922: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 118 KiB/s wr, 10 op/s 2023-09-10T08:34:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:34:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:34:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:34:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:38 smithi170 ceph-mon[102832]: pgmap v17923: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 336 KiB/s wr, 16 op/s 2023-09-10T08:34:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:38 smithi082 ceph-mon[98694]: pgmap v17923: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 336 KiB/s wr, 16 op/s 2023-09-10T08:34:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:38 smithi098 ceph-mon[103124]: pgmap v17923: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 336 KiB/s wr, 16 op/s 2023-09-10T08:34:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33310.774300 secs 2023-09-10T08:34:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33310.774300 secs 2023-09-10T08:34:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33310.774300 secs 2023-09-10T08:34:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:40 smithi170 ceph-mon[102832]: pgmap v17924: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 218 KiB/s wr, 10 op/s 2023-09-10T08:34:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33311.179074 secs 2023-09-10T08:34:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:40 smithi082 ceph-mon[98694]: pgmap v17924: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 218 KiB/s wr, 10 op/s 2023-09-10T08:34:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33311.179074 secs 2023-09-10T08:34:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:40 smithi098 ceph-mon[103124]: pgmap v17924: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 218 KiB/s wr, 10 op/s 2023-09-10T08:34:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33311.179074 secs 2023-09-10T08:34:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:41 smithi170 ceph-mon[102832]: pgmap v17925: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 218 KiB/s wr, 10 op/s 2023-09-10T08:34:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:41 smithi082 ceph-mon[98694]: pgmap v17925: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 218 KiB/s wr, 10 op/s 2023-09-10T08:34:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:41 smithi098 ceph-mon[103124]: pgmap v17925: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 218 KiB/s wr, 10 op/s 2023-09-10T08:34:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:43 smithi098 ceph-mon[103124]: pgmap v17926: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 438 KiB/s wr, 18 op/s 2023-09-10T08:34:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:43 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33315.774357 secs 2023-09-10T08:34:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:43 smithi170 ceph-mon[102832]: pgmap v17926: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 438 KiB/s wr, 18 op/s 2023-09-10T08:34:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:43 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33315.774357 secs 2023-09-10T08:34:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:43 smithi082 ceph-mon[98694]: pgmap v17926: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 438 KiB/s wr, 18 op/s 2023-09-10T08:34:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:43 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33315.774357 secs 2023-09-10T08:34:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33316.179140 secs 2023-09-10T08:34:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:34:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:34:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33316.179140 secs 2023-09-10T08:34:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:34:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:34:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33316.179140 secs 2023-09-10T08:34:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:34:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:34:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:46 smithi170 ceph-mon[102832]: pgmap v17927: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 438 KiB/s wr, 13 op/s 2023-09-10T08:34:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:34:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:34:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:46 smithi082 ceph-mon[98694]: pgmap v17927: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 438 KiB/s wr, 13 op/s 2023-09-10T08:34:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:34:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:34:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:46 smithi098 ceph-mon[103124]: pgmap v17927: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 438 KiB/s wr, 13 op/s 2023-09-10T08:34:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:34:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:34:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:48 smithi170 ceph-mon[102832]: pgmap v17928: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 713 KiB/s wr, 22 op/s 2023-09-10T08:34:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:48 smithi082 ceph-mon[98694]: pgmap v17928: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 713 KiB/s wr, 22 op/s 2023-09-10T08:34:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:48 smithi098 ceph-mon[103124]: pgmap v17928: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 713 KiB/s wr, 22 op/s 2023-09-10T08:34:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33320.774422 secs 2023-09-10T08:34:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33320.774422 secs 2023-09-10T08:34:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33320.774422 secs 2023-09-10T08:34:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:50 smithi170 ceph-mon[102832]: pgmap v17929: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 496 KiB/s wr, 16 op/s 2023-09-10T08:34:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33321.179207 secs 2023-09-10T08:34:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:50 smithi082 ceph-mon[98694]: pgmap v17929: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 496 KiB/s wr, 16 op/s 2023-09-10T08:34:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33321.179207 secs 2023-09-10T08:34:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:50 smithi098 ceph-mon[103124]: pgmap v17929: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 496 KiB/s wr, 16 op/s 2023-09-10T08:34:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33321.179207 secs 2023-09-10T08:34:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:52 smithi170 ceph-mon[102832]: pgmap v17930: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 496 KiB/s wr, 16 op/s 2023-09-10T08:34:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:52 smithi082 ceph-mon[98694]: pgmap v17930: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 496 KiB/s wr, 16 op/s 2023-09-10T08:34:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:52 smithi098 ceph-mon[103124]: pgmap v17930: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 496 KiB/s wr, 16 op/s 2023-09-10T08:34:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:54 smithi170 ceph-mon[102832]: pgmap v17931: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 743 KiB/s wr, 23 op/s 2023-09-10T08:34:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:34:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:34:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33325.774481 secs 2023-09-10T08:34:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:54 smithi082 ceph-mon[98694]: pgmap v17931: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 743 KiB/s wr, 23 op/s 2023-09-10T08:34:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:34:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:34:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33325.774481 secs 2023-09-10T08:34:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:54 smithi098 ceph-mon[103124]: pgmap v17931: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 743 KiB/s wr, 23 op/s 2023-09-10T08:34:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:34:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:34:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33325.774481 secs 2023-09-10T08:34:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33326.179269 secs 2023-09-10T08:34:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33326.179269 secs 2023-09-10T08:34:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33326.179269 secs 2023-09-10T08:34:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:56 smithi170 ceph-mon[102832]: pgmap v17932: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 523 KiB/s wr, 16 op/s 2023-09-10T08:34:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:56 smithi082 ceph-mon[98694]: pgmap v17932: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 523 KiB/s wr, 16 op/s 2023-09-10T08:34:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:56 smithi098 ceph-mon[103124]: pgmap v17932: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 523 KiB/s wr, 16 op/s 2023-09-10T08:34:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:34:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:34:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:34:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:34:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:34:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:34:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:58 smithi170 ceph-mon[102832]: pgmap v17933: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 773 KiB/s wr, 23 op/s 2023-09-10T08:34:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:58 smithi082 ceph-mon[98694]: pgmap v17933: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 773 KiB/s wr, 23 op/s 2023-09-10T08:34:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:58 smithi098 ceph-mon[103124]: pgmap v17933: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 773 KiB/s wr, 23 op/s 2023-09-10T08:34:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33329.573055 secs 2023-09-10T08:34:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:34:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33330.774571 secs 2023-09-10T08:34:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33329.573055 secs 2023-09-10T08:34:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:34:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33330.774571 secs 2023-09-10T08:34:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33329.573055 secs 2023-09-10T08:34:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:34:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33330.774571 secs 2023-09-10T08:35:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:00 smithi170 ceph-mon[102832]: pgmap v17934: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 498 KiB/s wr, 14 op/s 2023-09-10T08:35:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33331.179333 secs 2023-09-10T08:35:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:00 smithi082 ceph-mon[98694]: pgmap v17934: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 498 KiB/s wr, 14 op/s 2023-09-10T08:35:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33331.179333 secs 2023-09-10T08:35:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:00 smithi098 ceph-mon[103124]: pgmap v17934: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 498 KiB/s wr, 14 op/s 2023-09-10T08:35:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33331.179333 secs 2023-09-10T08:35:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:01 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:35:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:01 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:35:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:01 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:01 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:35:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:01 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:35:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:01 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:01 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:35:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:01 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:35:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:01 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:02 smithi170 ceph-mon[102832]: pgmap v17935: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 498 KiB/s wr, 14 op/s 2023-09-10T08:35:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:02 smithi082 ceph-mon[98694]: pgmap v17935: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 498 KiB/s wr, 14 op/s 2023-09-10T08:35:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:02 smithi098 ceph-mon[103124]: pgmap v17935: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 498 KiB/s wr, 14 op/s 2023-09-10T08:35:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:04 smithi170 ceph-mon[102832]: pgmap v17936: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 781 KiB/s wr, 20 op/s 2023-09-10T08:35:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33334.573114 secs 2023-09-10T08:35:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33335.774628 secs 2023-09-10T08:35:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:04 smithi082 ceph-mon[98694]: pgmap v17936: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 781 KiB/s wr, 20 op/s 2023-09-10T08:35:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33334.573114 secs 2023-09-10T08:35:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33335.774628 secs 2023-09-10T08:35:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:04 smithi098 ceph-mon[103124]: pgmap v17936: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 781 KiB/s wr, 20 op/s 2023-09-10T08:35:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33334.573114 secs 2023-09-10T08:35:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33335.774628 secs 2023-09-10T08:35:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33336.179399 secs 2023-09-10T08:35:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:35:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33336.179399 secs 2023-09-10T08:35:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:35:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33336.179399 secs 2023-09-10T08:35:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:35:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:06 smithi170 ceph-mon[102832]: pgmap v17937: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 533 KiB/s wr, 13 op/s 2023-09-10T08:35:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:35:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:06 smithi082 ceph-mon[98694]: pgmap v17937: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 533 KiB/s wr, 13 op/s 2023-09-10T08:35:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:35:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:06 smithi098 ceph-mon[103124]: pgmap v17937: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 533 KiB/s wr, 13 op/s 2023-09-10T08:35:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:35:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:08 smithi170 ceph-mon[102832]: pgmap v17938: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 720 KiB/s wr, 34 op/s 2023-09-10T08:35:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:08 smithi082 ceph-mon[98694]: pgmap v17938: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 720 KiB/s wr, 34 op/s 2023-09-10T08:35:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:08 smithi098 ceph-mon[103124]: pgmap v17938: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 720 KiB/s wr, 34 op/s 2023-09-10T08:35:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33339.573174 secs 2023-09-10T08:35:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:09 smithi170 ceph-mon[102832]: osdmap e1189: 12 total, 12 up, 12 in 2023-09-10T08:35:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33340.774687 secs 2023-09-10T08:35:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33339.573174 secs 2023-09-10T08:35:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:09 smithi082 ceph-mon[98694]: osdmap e1189: 12 total, 12 up, 12 in 2023-09-10T08:35:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33340.774687 secs 2023-09-10T08:35:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33339.573174 secs 2023-09-10T08:35:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:09 smithi098 ceph-mon[103124]: osdmap e1189: 12 total, 12 up, 12 in 2023-09-10T08:35:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33340.774687 secs 2023-09-10T08:35:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:10 smithi170 ceph-mon[102832]: pgmap v17940: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 563 KiB/s wr, 32 op/s 2023-09-10T08:35:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33341.179464 secs 2023-09-10T08:35:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:10 smithi082 ceph-mon[98694]: pgmap v17940: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 563 KiB/s wr, 32 op/s 2023-09-10T08:35:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33341.179464 secs 2023-09-10T08:35:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:10 smithi098 ceph-mon[103124]: pgmap v17940: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 563 KiB/s wr, 32 op/s 2023-09-10T08:35:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33341.179464 secs 2023-09-10T08:35:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:12 smithi170 ceph-mon[102832]: pgmap v17941: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 563 KiB/s wr, 32 op/s 2023-09-10T08:35:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:12 smithi082 ceph-mon[98694]: pgmap v17941: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 563 KiB/s wr, 32 op/s 2023-09-10T08:35:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:12 smithi098 ceph-mon[103124]: pgmap v17941: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 563 KiB/s wr, 32 op/s 2023-09-10T08:35:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:14 smithi170 ceph-mon[102832]: pgmap v17942: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 367 KiB/s wr, 31 op/s 2023-09-10T08:35:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33344.573233 secs 2023-09-10T08:35:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33345.774745 secs 2023-09-10T08:35:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:14 smithi082 ceph-mon[98694]: pgmap v17942: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 367 KiB/s wr, 31 op/s 2023-09-10T08:35:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33344.573233 secs 2023-09-10T08:35:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33345.774745 secs 2023-09-10T08:35:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:14 smithi098 ceph-mon[103124]: pgmap v17942: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 367 KiB/s wr, 31 op/s 2023-09-10T08:35:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33344.573233 secs 2023-09-10T08:35:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33345.774745 secs 2023-09-10T08:35:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33346.179490 secs 2023-09-10T08:35:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33346.179490 secs 2023-09-10T08:35:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33346.179490 secs 2023-09-10T08:35:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:16 smithi170 ceph-mon[102832]: pgmap v17943: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 367 KiB/s wr, 31 op/s 2023-09-10T08:35:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:16 smithi082 ceph-mon[98694]: pgmap v17943: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 367 KiB/s wr, 31 op/s 2023-09-10T08:35:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:16 smithi098 ceph-mon[103124]: pgmap v17943: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 367 KiB/s wr, 31 op/s 2023-09-10T08:35:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:18 smithi170 ceph-mon[102832]: pgmap v17944: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 13 op/s 2023-09-10T08:35:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:18 smithi170 ceph-mon[102832]: osdmap e1190: 12 total, 12 up, 12 in 2023-09-10T08:35:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:18 smithi082 ceph-mon[98694]: pgmap v17944: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 13 op/s 2023-09-10T08:35:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:18 smithi082 ceph-mon[98694]: osdmap e1190: 12 total, 12 up, 12 in 2023-09-10T08:35:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:18 smithi098 ceph-mon[103124]: pgmap v17944: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 13 op/s 2023-09-10T08:35:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:18 smithi098 ceph-mon[103124]: osdmap e1190: 12 total, 12 up, 12 in 2023-09-10T08:35:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33349.573292 secs 2023-09-10T08:35:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33350.774836 secs 2023-09-10T08:35:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33349.573292 secs 2023-09-10T08:35:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33350.774836 secs 2023-09-10T08:35:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33349.573292 secs 2023-09-10T08:35:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33350.774836 secs 2023-09-10T08:35:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:20 smithi170 ceph-mon[102832]: pgmap v17946: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 13 op/s 2023-09-10T08:35:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33351.179555 secs 2023-09-10T08:35:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:20 smithi082 ceph-mon[98694]: pgmap v17946: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 13 op/s 2023-09-10T08:35:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33351.179555 secs 2023-09-10T08:35:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:20 smithi098 ceph-mon[103124]: pgmap v17946: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 13 op/s 2023-09-10T08:35:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33351.179555 secs 2023-09-10T08:35:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:21 smithi170 ceph-mon[102832]: pgmap v17947: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 13 op/s 2023-09-10T08:35:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:21 smithi082 ceph-mon[98694]: pgmap v17947: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 13 op/s 2023-09-10T08:35:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:21 smithi098 ceph-mon[103124]: pgmap v17947: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 13 op/s 2023-09-10T08:35:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:24 smithi170 ceph-mon[102832]: pgmap v17948: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T08:35:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33354.573350 secs 2023-09-10T08:35:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33355.774940 secs 2023-09-10T08:35:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:24 smithi082 ceph-mon[98694]: pgmap v17948: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T08:35:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33354.573350 secs 2023-09-10T08:35:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33355.774940 secs 2023-09-10T08:35:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:24 smithi098 ceph-mon[103124]: pgmap v17948: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T08:35:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33354.573350 secs 2023-09-10T08:35:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33355.774940 secs 2023-09-10T08:35:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33356.179620 secs 2023-09-10T08:35:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:25 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:35:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:25 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33356.179620 secs 2023-09-10T08:35:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:25 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:35:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:25 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33356.179620 secs 2023-09-10T08:35:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:25 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:35:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:25 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:26 smithi170 ceph-mon[102832]: pgmap v17949: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T08:35:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:26 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:35:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:26 smithi082 ceph-mon[98694]: pgmap v17949: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T08:35:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:26 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:35:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:26 smithi098 ceph-mon[103124]: pgmap v17949: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T08:35:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:26 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:35:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:28 smithi170 ceph-mon[102832]: pgmap v17950: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 233 KiB/s wr, 13 op/s 2023-09-10T08:35:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:28 smithi082 ceph-mon[98694]: pgmap v17950: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 233 KiB/s wr, 13 op/s 2023-09-10T08:35:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:28 smithi098 ceph-mon[103124]: pgmap v17950: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 233 KiB/s wr, 13 op/s 2023-09-10T08:35:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33359.573417 secs 2023-09-10T08:35:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:35:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33360.775000 secs 2023-09-10T08:35:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:35:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:35:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:35:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33359.573417 secs 2023-09-10T08:35:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:35:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33360.775000 secs 2023-09-10T08:35:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:35:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:35:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:35:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33359.573417 secs 2023-09-10T08:35:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:35:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33360.775000 secs 2023-09-10T08:35:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:35:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:35:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:35:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:30 smithi170 ceph-mon[102832]: pgmap v17951: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 227 KiB/s wr, 13 op/s 2023-09-10T08:35:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33361.179698 secs 2023-09-10T08:35:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:30 smithi082 ceph-mon[98694]: pgmap v17951: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 227 KiB/s wr, 13 op/s 2023-09-10T08:35:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33361.179698 secs 2023-09-10T08:35:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:30 smithi098 ceph-mon[103124]: pgmap v17951: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 227 KiB/s wr, 13 op/s 2023-09-10T08:35:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33361.179698 secs 2023-09-10T08:35:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:32 smithi170 ceph-mon[102832]: pgmap v17952: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T08:35:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:32 smithi082 ceph-mon[98694]: pgmap v17952: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T08:35:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:32 smithi098 ceph-mon[103124]: pgmap v17952: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 194 KiB/s wr, 11 op/s 2023-09-10T08:35:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:34 smithi170 ceph-mon[102832]: pgmap v17953: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 194 KiB/s wr, 16 op/s 2023-09-10T08:35:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33365.775058 secs 2023-09-10T08:35:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:34 smithi082 ceph-mon[98694]: pgmap v17953: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 194 KiB/s wr, 16 op/s 2023-09-10T08:35:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33365.775058 secs 2023-09-10T08:35:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:34 smithi098 ceph-mon[103124]: pgmap v17953: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 194 KiB/s wr, 16 op/s 2023-09-10T08:35:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33365.775058 secs 2023-09-10T08:35:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33366.179764 secs 2023-09-10T08:35:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33366.179764 secs 2023-09-10T08:35:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33366.179764 secs 2023-09-10T08:35:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:36 smithi170 ceph-mon[102832]: pgmap v17954: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T08:35:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:36 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:35:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:36 smithi082 ceph-mon[98694]: pgmap v17954: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T08:35:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:36 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:35:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:36 smithi098 ceph-mon[103124]: pgmap v17954: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T08:35:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:36 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:35:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:37 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:35:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:35:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:37 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:35:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:35:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:37 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:35:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:35:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:38 smithi170 ceph-mon[102832]: pgmap v17955: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 315 KiB/s wr, 17 op/s 2023-09-10T08:35:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:38 smithi082 ceph-mon[98694]: pgmap v17955: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 315 KiB/s wr, 17 op/s 2023-09-10T08:35:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:38 smithi098 ceph-mon[103124]: pgmap v17955: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 315 KiB/s wr, 17 op/s 2023-09-10T08:35:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33370.775117 secs 2023-09-10T08:35:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33370.775117 secs 2023-09-10T08:35:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33370.775117 secs 2023-09-10T08:35:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:40 smithi170 ceph-mon[102832]: pgmap v17956: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 196 KiB/s wr, 11 op/s 2023-09-10T08:35:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33371.179829 secs 2023-09-10T08:35:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:40 smithi082 ceph-mon[98694]: pgmap v17956: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 196 KiB/s wr, 11 op/s 2023-09-10T08:35:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33371.179829 secs 2023-09-10T08:35:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:40 smithi098 ceph-mon[103124]: pgmap v17956: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 196 KiB/s wr, 11 op/s 2023-09-10T08:35:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33371.179829 secs 2023-09-10T08:35:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:42 smithi170 ceph-mon[102832]: pgmap v17957: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 196 KiB/s wr, 11 op/s 2023-09-10T08:35:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:42 smithi082 ceph-mon[98694]: pgmap v17957: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 196 KiB/s wr, 11 op/s 2023-09-10T08:35:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:42 smithi098 ceph-mon[103124]: pgmap v17957: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 196 KiB/s wr, 11 op/s 2023-09-10T08:35:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:44 smithi170 ceph-mon[102832]: pgmap v17958: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 460 KiB/s wr, 18 op/s 2023-09-10T08:35:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33375.775202 secs 2023-09-10T08:35:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:44 smithi082 ceph-mon[98694]: pgmap v17958: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 460 KiB/s wr, 18 op/s 2023-09-10T08:35:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33375.775202 secs 2023-09-10T08:35:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:44 smithi098 ceph-mon[103124]: pgmap v17958: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 460 KiB/s wr, 18 op/s 2023-09-10T08:35:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33375.775202 secs 2023-09-10T08:35:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33376.179896 secs 2023-09-10T08:35:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:35:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33376.179896 secs 2023-09-10T08:35:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:35:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33376.179896 secs 2023-09-10T08:35:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:35:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:46.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:46 smithi170 ceph-mon[102832]: pgmap v17959: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 460 KiB/s wr, 12 op/s 2023-09-10T08:35:46.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:35:46.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:46 smithi082 ceph-mon[98694]: pgmap v17959: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 460 KiB/s wr, 12 op/s 2023-09-10T08:35:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:35:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:46 smithi098 ceph-mon[103124]: pgmap v17959: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 460 KiB/s wr, 12 op/s 2023-09-10T08:35:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:35:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:48 smithi170 ceph-mon[102832]: pgmap v17960: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 670 KiB/s wr, 20 op/s 2023-09-10T08:35:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:48 smithi082 ceph-mon[98694]: pgmap v17960: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 670 KiB/s wr, 20 op/s 2023-09-10T08:35:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:48 smithi098 ceph-mon[103124]: pgmap v17960: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 670 KiB/s wr, 20 op/s 2023-09-10T08:35:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33380.775261 secs 2023-09-10T08:35:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33380.775261 secs 2023-09-10T08:35:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33380.775261 secs 2023-09-10T08:35:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:50 smithi170 ceph-mon[102832]: pgmap v17961: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 474 KiB/s wr, 13 op/s 2023-09-10T08:35:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33381.179962 secs 2023-09-10T08:35:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:50 smithi082 ceph-mon[98694]: pgmap v17961: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 474 KiB/s wr, 13 op/s 2023-09-10T08:35:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33381.179962 secs 2023-09-10T08:35:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:50 smithi098 ceph-mon[103124]: pgmap v17961: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 474 KiB/s wr, 13 op/s 2023-09-10T08:35:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33381.179962 secs 2023-09-10T08:35:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:52 smithi170 ceph-mon[102832]: pgmap v17962: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 474 KiB/s wr, 13 op/s 2023-09-10T08:35:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:52 smithi082 ceph-mon[98694]: pgmap v17962: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 474 KiB/s wr, 13 op/s 2023-09-10T08:35:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:52 smithi098 ceph-mon[103124]: pgmap v17962: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 474 KiB/s wr, 13 op/s 2023-09-10T08:35:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:54 smithi170 ceph-mon[102832]: pgmap v17963: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 863 KiB/s wr, 21 op/s 2023-09-10T08:35:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33384.573785 secs 2023-09-10T08:35:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:35:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33385.775346 secs 2023-09-10T08:35:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:54 smithi082 ceph-mon[98694]: pgmap v17963: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 863 KiB/s wr, 21 op/s 2023-09-10T08:35:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33384.573785 secs 2023-09-10T08:35:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:35:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33385.775346 secs 2023-09-10T08:35:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:54 smithi098 ceph-mon[103124]: pgmap v17963: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 863 KiB/s wr, 21 op/s 2023-09-10T08:35:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33384.573785 secs 2023-09-10T08:35:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:35:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33385.775346 secs 2023-09-10T08:35:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33386.180029 secs 2023-09-10T08:35:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33386.180029 secs 2023-09-10T08:35:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33386.180029 secs 2023-09-10T08:35:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:56 smithi170 ceph-mon[102832]: pgmap v17964: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 599 KiB/s wr, 14 op/s 2023-09-10T08:35:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:35:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:35:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:56 smithi082 ceph-mon[98694]: pgmap v17964: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 599 KiB/s wr, 14 op/s 2023-09-10T08:35:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:35:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:35:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:56 smithi098 ceph-mon[103124]: pgmap v17964: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 599 KiB/s wr, 14 op/s 2023-09-10T08:35:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:35:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:35:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:35:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:58 smithi170 ceph-mon[102832]: pgmap v17965: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 824 KiB/s wr, 20 op/s 2023-09-10T08:35:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:58 smithi082 ceph-mon[98694]: pgmap v17965: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 824 KiB/s wr, 20 op/s 2023-09-10T08:35:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:58 smithi098 ceph-mon[103124]: pgmap v17965: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 824 KiB/s wr, 20 op/s 2023-09-10T08:35:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33389.573846 secs 2023-09-10T08:35:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:35:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33390.775403 secs 2023-09-10T08:35:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33389.573846 secs 2023-09-10T08:35:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:35:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33390.775403 secs 2023-09-10T08:35:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33389.573846 secs 2023-09-10T08:35:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:35:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33390.775403 secs 2023-09-10T08:36:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:00 smithi170 ceph-mon[102832]: pgmap v17966: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 614 KiB/s wr, 13 op/s 2023-09-10T08:36:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33391.180091 secs 2023-09-10T08:36:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:00 smithi082 ceph-mon[98694]: pgmap v17966: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 614 KiB/s wr, 13 op/s 2023-09-10T08:36:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33391.180091 secs 2023-09-10T08:36:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:00 smithi098 ceph-mon[103124]: pgmap v17966: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 614 KiB/s wr, 13 op/s 2023-09-10T08:36:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33391.180091 secs 2023-09-10T08:36:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:02 smithi170 ceph-mon[102832]: pgmap v17967: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 614 KiB/s wr, 13 op/s 2023-09-10T08:36:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:02 smithi082 ceph-mon[98694]: pgmap v17967: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 614 KiB/s wr, 13 op/s 2023-09-10T08:36:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:02 smithi098 ceph-mon[103124]: pgmap v17967: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 614 KiB/s wr, 13 op/s 2023-09-10T08:36:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:04 smithi170 ceph-mon[102832]: pgmap v17968: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 805 KiB/s wr, 30 op/s 2023-09-10T08:36:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33394.573904 secs 2023-09-10T08:36:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:04 smithi170 ceph-mon[102832]: osdmap e1191: 12 total, 12 up, 12 in 2023-09-10T08:36:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33395.775462 secs 2023-09-10T08:36:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:04 smithi082 ceph-mon[98694]: pgmap v17968: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 805 KiB/s wr, 30 op/s 2023-09-10T08:36:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33394.573904 secs 2023-09-10T08:36:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:04 smithi082 ceph-mon[98694]: osdmap e1191: 12 total, 12 up, 12 in 2023-09-10T08:36:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33395.775462 secs 2023-09-10T08:36:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:04 smithi098 ceph-mon[103124]: pgmap v17968: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 805 KiB/s wr, 30 op/s 2023-09-10T08:36:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33394.573904 secs 2023-09-10T08:36:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:04 smithi098 ceph-mon[103124]: osdmap e1191: 12 total, 12 up, 12 in 2023-09-10T08:36:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33395.775462 secs 2023-09-10T08:36:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33396.180159 secs 2023-09-10T08:36:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:36:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:36:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33396.180159 secs 2023-09-10T08:36:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:36:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:36:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33396.180159 secs 2023-09-10T08:36:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:36:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:36:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:06 smithi170 ceph-mon[102832]: pgmap v17970: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 499 KiB/s wr, 27 op/s 2023-09-10T08:36:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:36:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:36:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:06 smithi082 ceph-mon[98694]: pgmap v17970: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 499 KiB/s wr, 27 op/s 2023-09-10T08:36:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:36:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:36:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:06 smithi098 ceph-mon[103124]: pgmap v17970: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 499 KiB/s wr, 27 op/s 2023-09-10T08:36:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:36:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:36:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:08 smithi170 ceph-mon[102832]: pgmap v17971: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 394 KiB/s wr, 31 op/s 2023-09-10T08:36:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:08 smithi170 ceph-mon[102832]: osdmap e1192: 12 total, 12 up, 12 in 2023-09-10T08:36:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:08 smithi082 ceph-mon[98694]: pgmap v17971: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 394 KiB/s wr, 31 op/s 2023-09-10T08:36:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:08 smithi082 ceph-mon[98694]: osdmap e1192: 12 total, 12 up, 12 in 2023-09-10T08:36:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:08 smithi098 ceph-mon[103124]: pgmap v17971: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 394 KiB/s wr, 31 op/s 2023-09-10T08:36:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:08 smithi098 ceph-mon[103124]: osdmap e1192: 12 total, 12 up, 12 in 2023-09-10T08:36:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33399.574017 secs 2023-09-10T08:36:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33400.775492 secs 2023-09-10T08:36:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33399.574017 secs 2023-09-10T08:36:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33400.775492 secs 2023-09-10T08:36:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33399.574017 secs 2023-09-10T08:36:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33400.775492 secs 2023-09-10T08:36:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:10 smithi170 ceph-mon[102832]: pgmap v17973: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 493 KiB/s wr, 38 op/s 2023-09-10T08:36:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33401.180227 secs 2023-09-10T08:36:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:10 smithi082 ceph-mon[98694]: pgmap v17973: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 493 KiB/s wr, 38 op/s 2023-09-10T08:36:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33401.180227 secs 2023-09-10T08:36:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:10 smithi098 ceph-mon[103124]: pgmap v17973: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 493 KiB/s wr, 38 op/s 2023-09-10T08:36:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33401.180227 secs 2023-09-10T08:36:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:12 smithi170 ceph-mon[102832]: pgmap v17974: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 207 KiB/s wr, 14 op/s 2023-09-10T08:36:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:12 smithi082 ceph-mon[98694]: pgmap v17974: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 207 KiB/s wr, 14 op/s 2023-09-10T08:36:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:12 smithi098 ceph-mon[103124]: pgmap v17974: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 207 KiB/s wr, 14 op/s 2023-09-10T08:36:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:14 smithi170 ceph-mon[102832]: pgmap v17975: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 290 KiB/s wr, 19 op/s 2023-09-10T08:36:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33404.574076 secs 2023-09-10T08:36:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33405.775519 secs 2023-09-10T08:36:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:14 smithi082 ceph-mon[98694]: pgmap v17975: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 290 KiB/s wr, 19 op/s 2023-09-10T08:36:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33404.574076 secs 2023-09-10T08:36:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33405.775519 secs 2023-09-10T08:36:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:14 smithi098 ceph-mon[103124]: pgmap v17975: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 290 KiB/s wr, 19 op/s 2023-09-10T08:36:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33404.574076 secs 2023-09-10T08:36:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33405.775519 secs 2023-09-10T08:36:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33406.180294 secs 2023-09-10T08:36:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33406.180294 secs 2023-09-10T08:36:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33406.180294 secs 2023-09-10T08:36:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:16 smithi170 ceph-mon[102832]: pgmap v17976: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 264 KiB/s wr, 18 op/s 2023-09-10T08:36:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:16 smithi082 ceph-mon[98694]: pgmap v17976: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 264 KiB/s wr, 18 op/s 2023-09-10T08:36:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:16 smithi098 ceph-mon[103124]: pgmap v17976: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 264 KiB/s wr, 18 op/s 2023-09-10T08:36:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:18 smithi170 ceph-mon[102832]: pgmap v17977: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 13 op/s 2023-09-10T08:36:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:18 smithi082 ceph-mon[98694]: pgmap v17977: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 13 op/s 2023-09-10T08:36:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:18 smithi098 ceph-mon[103124]: pgmap v17977: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 13 op/s 2023-09-10T08:36:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33409.574135 secs 2023-09-10T08:36:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33410.775578 secs 2023-09-10T08:36:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33409.574135 secs 2023-09-10T08:36:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33410.775578 secs 2023-09-10T08:36:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33409.574135 secs 2023-09-10T08:36:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33410.775578 secs 2023-09-10T08:36:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:20 smithi170 ceph-mon[102832]: pgmap v17978: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T08:36:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33411.180363 secs 2023-09-10T08:36:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:20 smithi082 ceph-mon[98694]: pgmap v17978: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T08:36:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33411.180363 secs 2023-09-10T08:36:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:20 smithi098 ceph-mon[103124]: pgmap v17978: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T08:36:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33411.180363 secs 2023-09-10T08:36:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:22 smithi170 ceph-mon[102832]: pgmap v17979: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 108 KiB/s wr, 11 op/s 2023-09-10T08:36:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:22 smithi082 ceph-mon[98694]: pgmap v17979: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 108 KiB/s wr, 11 op/s 2023-09-10T08:36:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:22 smithi098 ceph-mon[103124]: pgmap v17979: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 108 KiB/s wr, 11 op/s 2023-09-10T08:36:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:24 smithi170 ceph-mon[102832]: pgmap v17980: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 108 KiB/s wr, 15 op/s 2023-09-10T08:36:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33414.574213 secs 2023-09-10T08:36:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33415.775648 secs 2023-09-10T08:36:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:24 smithi082 ceph-mon[98694]: pgmap v17980: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 108 KiB/s wr, 15 op/s 2023-09-10T08:36:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33414.574213 secs 2023-09-10T08:36:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33415.775648 secs 2023-09-10T08:36:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:24 smithi098 ceph-mon[103124]: pgmap v17980: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 108 KiB/s wr, 15 op/s 2023-09-10T08:36:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33414.574213 secs 2023-09-10T08:36:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33415.775648 secs 2023-09-10T08:36:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33416.180431 secs 2023-09-10T08:36:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33416.180431 secs 2023-09-10T08:36:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33416.180431 secs 2023-09-10T08:36:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:26 smithi170 ceph-mon[102832]: pgmap v17981: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 10 op/s 2023-09-10T08:36:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:26 smithi082 ceph-mon[98694]: pgmap v17981: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 10 op/s 2023-09-10T08:36:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:26 smithi098 ceph-mon[103124]: pgmap v17981: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 10 op/s 2023-09-10T08:36:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:28 smithi170 ceph-mon[102832]: pgmap v17982: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 16 op/s 2023-09-10T08:36:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:28 smithi082 ceph-mon[98694]: pgmap v17982: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 16 op/s 2023-09-10T08:36:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:28 smithi098 ceph-mon[103124]: pgmap v17982: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 16 op/s 2023-09-10T08:36:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33419.574274 secs 2023-09-10T08:36:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33420.775729 secs 2023-09-10T08:36:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33419.574274 secs 2023-09-10T08:36:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33420.775729 secs 2023-09-10T08:36:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33419.574274 secs 2023-09-10T08:36:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33420.775729 secs 2023-09-10T08:36:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:30 smithi170 ceph-mon[102832]: pgmap v17983: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T08:36:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33421.180490 secs 2023-09-10T08:36:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:36:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:30 smithi082 ceph-mon[98694]: pgmap v17983: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T08:36:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33421.180490 secs 2023-09-10T08:36:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:36:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:30 smithi098 ceph-mon[103124]: pgmap v17983: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T08:36:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33421.180490 secs 2023-09-10T08:36:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:36:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:32 smithi170 ceph-mon[102832]: pgmap v17984: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T08:36:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:36:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:36:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:36:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:36:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:36:32.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:36:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:32 smithi082 ceph-mon[98694]: pgmap v17984: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T08:36:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:36:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:36:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:36:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:36:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:36:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:36:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:32 smithi098 ceph-mon[103124]: pgmap v17984: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T08:36:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:36:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:36:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:36:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:36:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:36:32.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:36:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:36:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:36:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:36:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:36:33.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:36:33.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:36:33.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:36:33.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:36:33.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:36:33.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:36:33.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:36:33.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:36:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:36:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:36:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:36:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:36:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:36:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:36:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:36:33.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:36:33.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:36:33.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:36:33.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:36:33.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:36:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:36:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:36:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:36:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:36:33.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:36:33.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:36:33.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:36:33.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:36:33.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:36:33.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:36:33.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:36:33.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:36:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:34 smithi170 ceph-mon[102832]: pgmap v17985: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 119 KiB/s wr, 15 op/s 2023-09-10T08:36:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:34 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T08:36:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:34 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T08:36:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:34 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T08:36:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:34 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T08:36:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:36:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:36:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33425.775815 secs 2023-09-10T08:36:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:34 smithi082 ceph-mon[98694]: pgmap v17985: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 119 KiB/s wr, 15 op/s 2023-09-10T08:36:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:34 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T08:36:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:34 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T08:36:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:34 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T08:36:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:34 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T08:36:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:36:34.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:36:34.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33425.775815 secs 2023-09-10T08:36:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:34 smithi098 ceph-mon[103124]: pgmap v17985: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 119 KiB/s wr, 15 op/s 2023-09-10T08:36:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:34 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T08:36:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:34 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T08:36:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:34 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T08:36:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:34 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T08:36:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:36:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:36:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33425.775815 secs 2023-09-10T08:36:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33426.180555 secs 2023-09-10T08:36:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:36:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33426.180555 secs 2023-09-10T08:36:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:36:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33426.180555 secs 2023-09-10T08:36:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:36:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:36 smithi170 ceph-mon[102832]: pgmap v17986: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T08:36:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:36:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:36:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:36:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:36 smithi082 ceph-mon[98694]: pgmap v17986: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T08:36:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:36:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:36:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:36:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:36 smithi098 ceph-mon[103124]: pgmap v17986: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 11 op/s 2023-09-10T08:36:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:36:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:36:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:36:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:38 smithi170 ceph-mon[102832]: pgmap v17987: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 285 KiB/s wr, 17 op/s 2023-09-10T08:36:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:38 smithi082 ceph-mon[98694]: pgmap v17987: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 285 KiB/s wr, 17 op/s 2023-09-10T08:36:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:38 smithi098 ceph-mon[103124]: pgmap v17987: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 285 KiB/s wr, 17 op/s 2023-09-10T08:36:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33430.775882 secs 2023-09-10T08:36:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33430.775882 secs 2023-09-10T08:36:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33430.775882 secs 2023-09-10T08:36:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:40 smithi170 ceph-mon[102832]: pgmap v17988: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T08:36:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33431.180622 secs 2023-09-10T08:36:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:40 smithi082 ceph-mon[98694]: pgmap v17988: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T08:36:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33431.180622 secs 2023-09-10T08:36:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:40 smithi098 ceph-mon[103124]: pgmap v17988: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 167 KiB/s wr, 11 op/s 2023-09-10T08:36:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33431.180622 secs 2023-09-10T08:36:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:41 smithi170 ceph-mon[102832]: pgmap v17989: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 14 op/s 2023-09-10T08:36:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:41 smithi082 ceph-mon[98694]: pgmap v17989: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 14 op/s 2023-09-10T08:36:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:41 smithi098 ceph-mon[103124]: pgmap v17989: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 14 op/s 2023-09-10T08:36:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:44 smithi170 ceph-mon[102832]: pgmap v17990: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 441 KiB/s wr, 18 op/s 2023-09-10T08:36:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33435.775940 secs 2023-09-10T08:36:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:44 smithi082 ceph-mon[98694]: pgmap v17990: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 441 KiB/s wr, 18 op/s 2023-09-10T08:36:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33435.775940 secs 2023-09-10T08:36:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:44 smithi098 ceph-mon[103124]: pgmap v17990: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 441 KiB/s wr, 18 op/s 2023-09-10T08:36:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33435.775940 secs 2023-09-10T08:36:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33436.180686 secs 2023-09-10T08:36:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:36:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:36:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33436.180686 secs 2023-09-10T08:36:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:36:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:36:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33436.180686 secs 2023-09-10T08:36:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:36:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:36:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:46 smithi170 ceph-mon[102832]: pgmap v17991: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 440 KiB/s wr, 13 op/s 2023-09-10T08:36:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:36:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:36:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:46 smithi082 ceph-mon[98694]: pgmap v17991: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 440 KiB/s wr, 13 op/s 2023-09-10T08:36:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:36:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:36:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:46 smithi098 ceph-mon[103124]: pgmap v17991: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 440 KiB/s wr, 13 op/s 2023-09-10T08:36:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:36:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:36:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:48 smithi170 ceph-mon[102832]: pgmap v17992: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 680 KiB/s wr, 21 op/s 2023-09-10T08:36:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:48 smithi082 ceph-mon[98694]: pgmap v17992: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 680 KiB/s wr, 21 op/s 2023-09-10T08:36:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:48 smithi098 ceph-mon[103124]: pgmap v17992: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 680 KiB/s wr, 21 op/s 2023-09-10T08:36:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33440.775999 secs 2023-09-10T08:36:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33440.775999 secs 2023-09-10T08:36:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33440.775999 secs 2023-09-10T08:36:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:50 smithi170 ceph-mon[102832]: pgmap v17993: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 513 KiB/s wr, 14 op/s 2023-09-10T08:36:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33441.180750 secs 2023-09-10T08:36:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:50 smithi082 ceph-mon[98694]: pgmap v17993: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 513 KiB/s wr, 14 op/s 2023-09-10T08:36:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33441.180750 secs 2023-09-10T08:36:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:50 smithi098 ceph-mon[103124]: pgmap v17993: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 513 KiB/s wr, 14 op/s 2023-09-10T08:36:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33441.180750 secs 2023-09-10T08:36:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:52 smithi170 ceph-mon[102832]: pgmap v17994: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 733 KiB/s wr, 18 op/s 2023-09-10T08:36:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:52 smithi082 ceph-mon[98694]: pgmap v17994: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 733 KiB/s wr, 18 op/s 2023-09-10T08:36:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:52 smithi098 ceph-mon[103124]: pgmap v17994: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 733 KiB/s wr, 18 op/s 2023-09-10T08:36:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:54 smithi170 ceph-mon[102832]: pgmap v17995: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 708 KiB/s wr, 19 op/s 2023-09-10T08:36:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:36:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:36:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33445.776057 secs 2023-09-10T08:36:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:54 smithi082 ceph-mon[98694]: pgmap v17995: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 708 KiB/s wr, 19 op/s 2023-09-10T08:36:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:36:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:36:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33445.776057 secs 2023-09-10T08:36:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:54 smithi098 ceph-mon[103124]: pgmap v17995: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 708 KiB/s wr, 19 op/s 2023-09-10T08:36:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:36:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:36:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33445.776057 secs 2023-09-10T08:36:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33446.180820 secs 2023-09-10T08:36:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33446.180820 secs 2023-09-10T08:36:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33446.180820 secs 2023-09-10T08:36:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:56 smithi170 ceph-mon[102832]: pgmap v17996: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 459 KiB/s wr, 15 op/s 2023-09-10T08:36:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:56 smithi082 ceph-mon[98694]: pgmap v17996: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 459 KiB/s wr, 15 op/s 2023-09-10T08:36:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:56 smithi098 ceph-mon[103124]: pgmap v17996: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 459 KiB/s wr, 15 op/s 2023-09-10T08:36:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:36:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:36:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:36:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:36:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:36:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:36:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:58 smithi170 ceph-mon[102832]: pgmap v17997: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 674 KiB/s wr, 22 op/s 2023-09-10T08:36:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:58 smithi082 ceph-mon[98694]: pgmap v17997: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 674 KiB/s wr, 22 op/s 2023-09-10T08:36:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:58 smithi098 ceph-mon[103124]: pgmap v17997: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 674 KiB/s wr, 22 op/s 2023-09-10T08:36:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33449.574738 secs 2023-09-10T08:36:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:36:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33450.776163 secs 2023-09-10T08:36:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33449.574738 secs 2023-09-10T08:36:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:36:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33450.776163 secs 2023-09-10T08:36:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33449.574738 secs 2023-09-10T08:36:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:36:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33450.776163 secs 2023-09-10T08:37:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:00 smithi170 ceph-mon[102832]: pgmap v17998: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 435 KiB/s wr, 15 op/s 2023-09-10T08:37:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33451.180887 secs 2023-09-10T08:37:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:37:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:37:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:00 smithi082 ceph-mon[98694]: pgmap v17998: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 435 KiB/s wr, 15 op/s 2023-09-10T08:37:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33451.180887 secs 2023-09-10T08:37:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:37:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:37:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:00 smithi098 ceph-mon[103124]: pgmap v17998: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 435 KiB/s wr, 15 op/s 2023-09-10T08:37:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33451.180887 secs 2023-09-10T08:37:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:37:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:37:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:02 smithi170 ceph-mon[102832]: pgmap v17999: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 664 KiB/s wr, 18 op/s 2023-09-10T08:37:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:02 smithi082 ceph-mon[98694]: pgmap v17999: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 664 KiB/s wr, 18 op/s 2023-09-10T08:37:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:02 smithi098 ceph-mon[103124]: pgmap v17999: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 664 KiB/s wr, 18 op/s 2023-09-10T08:37:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:04 smithi170 ceph-mon[102832]: pgmap v18000: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 445 KiB/s wr, 17 op/s 2023-09-10T08:37:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33454.574773 secs 2023-09-10T08:37:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:04 smithi170 ceph-mon[102832]: osdmap e1193: 12 total, 12 up, 12 in 2023-09-10T08:37:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33455.776223 secs 2023-09-10T08:37:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:37:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:04 smithi082 ceph-mon[98694]: pgmap v18000: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 445 KiB/s wr, 17 op/s 2023-09-10T08:37:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33454.574773 secs 2023-09-10T08:37:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:04 smithi082 ceph-mon[98694]: osdmap e1193: 12 total, 12 up, 12 in 2023-09-10T08:37:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33455.776223 secs 2023-09-10T08:37:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:37:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:04 smithi098 ceph-mon[103124]: pgmap v18000: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 445 KiB/s wr, 17 op/s 2023-09-10T08:37:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33454.574773 secs 2023-09-10T08:37:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:04 smithi098 ceph-mon[103124]: osdmap e1193: 12 total, 12 up, 12 in 2023-09-10T08:37:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33455.776223 secs 2023-09-10T08:37:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:37:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33456.180953 secs 2023-09-10T08:37:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33456.180953 secs 2023-09-10T08:37:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33456.180953 secs 2023-09-10T08:37:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:06 smithi170 ceph-mon[102832]: pgmap v18002: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 533 KiB/s wr, 15 op/s 2023-09-10T08:37:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:37:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:06 smithi082 ceph-mon[98694]: pgmap v18002: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 533 KiB/s wr, 15 op/s 2023-09-10T08:37:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:37:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:06 smithi098 ceph-mon[103124]: pgmap v18002: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 533 KiB/s wr, 15 op/s 2023-09-10T08:37:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:37:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:08 smithi170 ceph-mon[102832]: pgmap v18003: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 501 KiB/s wr, 31 op/s 2023-09-10T08:37:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:08 smithi170 ceph-mon[102832]: osdmap e1194: 12 total, 12 up, 12 in 2023-09-10T08:37:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:08 smithi082 ceph-mon[98694]: pgmap v18003: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 501 KiB/s wr, 31 op/s 2023-09-10T08:37:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:08 smithi082 ceph-mon[98694]: osdmap e1194: 12 total, 12 up, 12 in 2023-09-10T08:37:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:08 smithi098 ceph-mon[103124]: pgmap v18003: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 501 KiB/s wr, 31 op/s 2023-09-10T08:37:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:08 smithi098 ceph-mon[103124]: osdmap e1194: 12 total, 12 up, 12 in 2023-09-10T08:37:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33459.574835 secs 2023-09-10T08:37:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33460.776297 secs 2023-09-10T08:37:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33459.574835 secs 2023-09-10T08:37:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33460.776297 secs 2023-09-10T08:37:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33459.574835 secs 2023-09-10T08:37:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33460.776297 secs 2023-09-10T08:37:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:10 smithi170 ceph-mon[102832]: pgmap v18005: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 282 KiB/s wr, 34 op/s 2023-09-10T08:37:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33461.181017 secs 2023-09-10T08:37:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:10 smithi082 ceph-mon[98694]: pgmap v18005: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 282 KiB/s wr, 34 op/s 2023-09-10T08:37:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33461.181017 secs 2023-09-10T08:37:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:10 smithi098 ceph-mon[103124]: pgmap v18005: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 282 KiB/s wr, 34 op/s 2023-09-10T08:37:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33461.181017 secs 2023-09-10T08:37:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:12 smithi170 ceph-mon[102832]: pgmap v18006: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 461 KiB/s wr, 35 op/s 2023-09-10T08:37:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:12 smithi082 ceph-mon[98694]: pgmap v18006: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 461 KiB/s wr, 35 op/s 2023-09-10T08:37:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:12 smithi098 ceph-mon[103124]: pgmap v18006: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 461 KiB/s wr, 35 op/s 2023-09-10T08:37:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:14 smithi170 ceph-mon[102832]: pgmap v18007: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 404 KiB/s wr, 34 op/s 2023-09-10T08:37:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33464.574893 secs 2023-09-10T08:37:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33465.776355 secs 2023-09-10T08:37:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33466.181083 secs 2023-09-10T08:37:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:14 smithi082 ceph-mon[98694]: pgmap v18007: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 404 KiB/s wr, 34 op/s 2023-09-10T08:37:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33464.574893 secs 2023-09-10T08:37:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33465.776355 secs 2023-09-10T08:37:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33466.181083 secs 2023-09-10T08:37:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:14 smithi098 ceph-mon[103124]: pgmap v18007: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 404 KiB/s wr, 34 op/s 2023-09-10T08:37:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33464.574893 secs 2023-09-10T08:37:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33465.776355 secs 2023-09-10T08:37:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33466.181083 secs 2023-09-10T08:37:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:15 smithi170 ceph-mon[102832]: pgmap v18008: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 369 KiB/s wr, 31 op/s 2023-09-10T08:37:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:15 smithi082 ceph-mon[98694]: pgmap v18008: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 369 KiB/s wr, 31 op/s 2023-09-10T08:37:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:15 smithi098 ceph-mon[103124]: pgmap v18008: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 369 KiB/s wr, 31 op/s 2023-09-10T08:37:17.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:17 smithi170 ceph-mon[102832]: pgmap v18009: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 13 op/s 2023-09-10T08:37:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:17 smithi082 ceph-mon[98694]: pgmap v18009: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 13 op/s 2023-09-10T08:37:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:17 smithi098 ceph-mon[103124]: pgmap v18009: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 13 op/s 2023-09-10T08:37:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33469.574953 secs 2023-09-10T08:37:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33470.776418 secs 2023-09-10T08:37:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33469.574953 secs 2023-09-10T08:37:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33470.776418 secs 2023-09-10T08:37:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33469.574953 secs 2023-09-10T08:37:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33470.776418 secs 2023-09-10T08:37:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:20 smithi170 ceph-mon[102832]: pgmap v18010: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 170 KiB/s wr, 13 op/s 2023-09-10T08:37:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33471.181147 secs 2023-09-10T08:37:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:20 smithi082 ceph-mon[98694]: pgmap v18010: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 170 KiB/s wr, 13 op/s 2023-09-10T08:37:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33471.181147 secs 2023-09-10T08:37:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:20 smithi098 ceph-mon[103124]: pgmap v18010: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 170 KiB/s wr, 13 op/s 2023-09-10T08:37:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33471.181147 secs 2023-09-10T08:37:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:22 smithi170 ceph-mon[102832]: pgmap v18011: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 240 KiB/s wr, 14 op/s 2023-09-10T08:37:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:22 smithi082 ceph-mon[98694]: pgmap v18011: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 240 KiB/s wr, 14 op/s 2023-09-10T08:37:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:22 smithi098 ceph-mon[103124]: pgmap v18011: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 240 KiB/s wr, 14 op/s 2023-09-10T08:37:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:24 smithi170 ceph-mon[102832]: pgmap v18012: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T08:37:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33474.575035 secs 2023-09-10T08:37:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33475.776490 secs 2023-09-10T08:37:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:24 smithi082 ceph-mon[98694]: pgmap v18012: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T08:37:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33474.575035 secs 2023-09-10T08:37:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33475.776490 secs 2023-09-10T08:37:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:24 smithi098 ceph-mon[103124]: pgmap v18012: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T08:37:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33474.575035 secs 2023-09-10T08:37:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33475.776490 secs 2023-09-10T08:37:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33476.181212 secs 2023-09-10T08:37:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33476.181212 secs 2023-09-10T08:37:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33476.181212 secs 2023-09-10T08:37:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:26 smithi170 ceph-mon[102832]: pgmap v18013: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 10 op/s 2023-09-10T08:37:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:26 smithi082 ceph-mon[98694]: pgmap v18013: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 10 op/s 2023-09-10T08:37:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:26 smithi098 ceph-mon[103124]: pgmap v18013: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 10 op/s 2023-09-10T08:37:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:28 smithi170 ceph-mon[102832]: pgmap v18014: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 240 KiB/s wr, 16 op/s 2023-09-10T08:37:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:28 smithi082 ceph-mon[98694]: pgmap v18014: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 240 KiB/s wr, 16 op/s 2023-09-10T08:37:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:28 smithi098 ceph-mon[103124]: pgmap v18014: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 240 KiB/s wr, 16 op/s 2023-09-10T08:37:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33479.575099 secs 2023-09-10T08:37:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33480.776550 secs 2023-09-10T08:37:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33479.575099 secs 2023-09-10T08:37:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33480.776550 secs 2023-09-10T08:37:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33479.575099 secs 2023-09-10T08:37:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33480.776550 secs 2023-09-10T08:37:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:30 smithi170 ceph-mon[102832]: pgmap v18015: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 214 KiB/s wr, 11 op/s 2023-09-10T08:37:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33481.181278 secs 2023-09-10T08:37:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:30 smithi082 ceph-mon[98694]: pgmap v18015: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 214 KiB/s wr, 11 op/s 2023-09-10T08:37:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33481.181278 secs 2023-09-10T08:37:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:30 smithi098 ceph-mon[103124]: pgmap v18015: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 214 KiB/s wr, 11 op/s 2023-09-10T08:37:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33481.181278 secs 2023-09-10T08:37:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:32 smithi170 ceph-mon[102832]: pgmap v18016: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 214 KiB/s wr, 13 op/s 2023-09-10T08:37:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:32 smithi082 ceph-mon[98694]: pgmap v18016: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 214 KiB/s wr, 13 op/s 2023-09-10T08:37:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:32 smithi098 ceph-mon[103124]: pgmap v18016: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 214 KiB/s wr, 13 op/s 2023-09-10T08:37:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:33 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:37:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:33 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:37:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:33 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:37:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:34 smithi170 ceph-mon[102832]: pgmap v18017: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T08:37:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:37:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:37:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:37:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:37:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33485.776609 secs 2023-09-10T08:37:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:34 smithi082 ceph-mon[98694]: pgmap v18017: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T08:37:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:37:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:37:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:37:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:37:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33485.776609 secs 2023-09-10T08:37:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:34 smithi098 ceph-mon[103124]: pgmap v18017: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T08:37:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:37:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:37:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:37:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:37:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33485.776609 secs 2023-09-10T08:37:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33486.181350 secs 2023-09-10T08:37:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:37:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33486.181350 secs 2023-09-10T08:37:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:37:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33486.181350 secs 2023-09-10T08:37:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:37:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:36 smithi170 ceph-mon[102832]: pgmap v18018: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T08:37:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:37:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:37:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:36 smithi082 ceph-mon[98694]: pgmap v18018: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T08:37:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:37:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:37:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:36 smithi098 ceph-mon[103124]: pgmap v18018: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T08:37:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:37:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:37:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:38 smithi170 ceph-mon[102832]: pgmap v18019: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 307 KiB/s wr, 17 op/s 2023-09-10T08:37:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:38 smithi082 ceph-mon[98694]: pgmap v18019: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 307 KiB/s wr, 17 op/s 2023-09-10T08:37:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:38 smithi098 ceph-mon[103124]: pgmap v18019: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 307 KiB/s wr, 17 op/s 2023-09-10T08:37:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33490.776666 secs 2023-09-10T08:37:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33490.776666 secs 2023-09-10T08:37:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33490.776666 secs 2023-09-10T08:37:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:40 smithi170 ceph-mon[102832]: pgmap v18020: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 188 KiB/s wr, 11 op/s 2023-09-10T08:37:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33491.181416 secs 2023-09-10T08:37:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:40 smithi082 ceph-mon[98694]: pgmap v18020: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 188 KiB/s wr, 11 op/s 2023-09-10T08:37:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33491.181416 secs 2023-09-10T08:37:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:40 smithi098 ceph-mon[103124]: pgmap v18020: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 188 KiB/s wr, 11 op/s 2023-09-10T08:37:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33491.181416 secs 2023-09-10T08:37:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:42 smithi170 ceph-mon[102832]: pgmap v18021: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 213 KiB/s wr, 13 op/s 2023-09-10T08:37:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:42 smithi082 ceph-mon[98694]: pgmap v18021: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 213 KiB/s wr, 13 op/s 2023-09-10T08:37:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:42 smithi098 ceph-mon[103124]: pgmap v18021: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 213 KiB/s wr, 13 op/s 2023-09-10T08:37:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:44 smithi170 ceph-mon[102832]: pgmap v18022: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 463 KiB/s wr, 15 op/s 2023-09-10T08:37:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33495.776737 secs 2023-09-10T08:37:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:37:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:44 smithi082 ceph-mon[98694]: pgmap v18022: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 463 KiB/s wr, 15 op/s 2023-09-10T08:37:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33495.776737 secs 2023-09-10T08:37:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:37:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:44 smithi098 ceph-mon[103124]: pgmap v18022: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 463 KiB/s wr, 15 op/s 2023-09-10T08:37:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33495.776737 secs 2023-09-10T08:37:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:37:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33496.181485 secs 2023-09-10T08:37:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33496.181485 secs 2023-09-10T08:37:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33496.181485 secs 2023-09-10T08:37:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:46 smithi170 ceph-mon[102832]: pgmap v18023: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 463 KiB/s wr, 12 op/s 2023-09-10T08:37:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:37:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:46 smithi082 ceph-mon[98694]: pgmap v18023: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 463 KiB/s wr, 12 op/s 2023-09-10T08:37:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:37:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:46 smithi098 ceph-mon[103124]: pgmap v18023: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 463 KiB/s wr, 12 op/s 2023-09-10T08:37:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:37:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:48 smithi170 ceph-mon[102832]: pgmap v18024: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 720 KiB/s wr, 19 op/s 2023-09-10T08:37:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:48 smithi082 ceph-mon[98694]: pgmap v18024: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 720 KiB/s wr, 19 op/s 2023-09-10T08:37:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:48 smithi098 ceph-mon[103124]: pgmap v18024: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 720 KiB/s wr, 19 op/s 2023-09-10T08:37:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33500.776840 secs 2023-09-10T08:37:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33500.776840 secs 2023-09-10T08:37:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33500.776840 secs 2023-09-10T08:37:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:50 smithi170 ceph-mon[102832]: pgmap v18025: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 532 KiB/s wr, 13 op/s 2023-09-10T08:37:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33501.181550 secs 2023-09-10T08:37:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:37:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:50 smithi082 ceph-mon[98694]: pgmap v18025: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 532 KiB/s wr, 13 op/s 2023-09-10T08:37:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33501.181550 secs 2023-09-10T08:37:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:37:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:50 smithi098 ceph-mon[103124]: pgmap v18025: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 532 KiB/s wr, 13 op/s 2023-09-10T08:37:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33501.181550 secs 2023-09-10T08:37:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:37:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:52 smithi170 ceph-mon[102832]: pgmap v18026: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 899 KiB/s wr, 17 op/s 2023-09-10T08:37:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:37:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:52 smithi082 ceph-mon[98694]: pgmap v18026: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 899 KiB/s wr, 17 op/s 2023-09-10T08:37:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:37:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:52 smithi098 ceph-mon[103124]: pgmap v18026: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 899 KiB/s wr, 17 op/s 2023-09-10T08:37:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:37:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:54 smithi170 ceph-mon[102832]: pgmap v18027: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 874 KiB/s wr, 18 op/s 2023-09-10T08:37:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33504.575447 secs 2023-09-10T08:37:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:37:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33505.776896 secs 2023-09-10T08:37:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33506.181615 secs 2023-09-10T08:37:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:54 smithi082 ceph-mon[98694]: pgmap v18027: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 874 KiB/s wr, 18 op/s 2023-09-10T08:37:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33504.575447 secs 2023-09-10T08:37:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:37:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33505.776896 secs 2023-09-10T08:37:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33506.181615 secs 2023-09-10T08:37:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:54 smithi098 ceph-mon[103124]: pgmap v18027: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 874 KiB/s wr, 18 op/s 2023-09-10T08:37:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33504.575447 secs 2023-09-10T08:37:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:37:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33505.776896 secs 2023-09-10T08:37:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33506.181615 secs 2023-09-10T08:37:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:55 smithi082 ceph-mon[98694]: pgmap v18028: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 624 KiB/s wr, 14 op/s 2023-09-10T08:37:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:37:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:37:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:55 smithi098 ceph-mon[103124]: pgmap v18028: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 624 KiB/s wr, 14 op/s 2023-09-10T08:37:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:37:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:37:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:55 smithi170 ceph-mon[102832]: pgmap v18028: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 624 KiB/s wr, 14 op/s 2023-09-10T08:37:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:37:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:37:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:37:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:57 smithi170 ceph-mon[102832]: pgmap v18029: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 829 KiB/s wr, 21 op/s 2023-09-10T08:37:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:57 smithi082 ceph-mon[98694]: pgmap v18029: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 829 KiB/s wr, 21 op/s 2023-09-10T08:37:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:57 smithi098 ceph-mon[103124]: pgmap v18029: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 829 KiB/s wr, 21 op/s 2023-09-10T08:37:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33509.575550 secs 2023-09-10T08:37:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:37:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33510.776960 secs 2023-09-10T08:37:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33509.575550 secs 2023-09-10T08:37:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:37:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33510.776960 secs 2023-09-10T08:37:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33509.575550 secs 2023-09-10T08:37:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:37:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33510.776960 secs 2023-09-10T08:38:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:00 smithi170 ceph-mon[102832]: pgmap v18030: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 571 KiB/s wr, 14 op/s 2023-09-10T08:38:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33511.181684 secs 2023-09-10T08:38:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:00 smithi082 ceph-mon[98694]: pgmap v18030: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 571 KiB/s wr, 14 op/s 2023-09-10T08:38:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33511.181684 secs 2023-09-10T08:38:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:00 smithi098 ceph-mon[103124]: pgmap v18030: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 571 KiB/s wr, 14 op/s 2023-09-10T08:38:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33511.181684 secs 2023-09-10T08:38:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:02 smithi170 ceph-mon[102832]: pgmap v18031: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 734 KiB/s wr, 18 op/s 2023-09-10T08:38:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:02 smithi082 ceph-mon[98694]: pgmap v18031: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 734 KiB/s wr, 18 op/s 2023-09-10T08:38:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:02 smithi098 ceph-mon[103124]: pgmap v18031: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 734 KiB/s wr, 18 op/s 2023-09-10T08:38:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:04 smithi170 ceph-mon[102832]: pgmap v18032: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 489 KiB/s wr, 18 op/s 2023-09-10T08:38:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33514.575608 secs 2023-09-10T08:38:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33515.777023 secs 2023-09-10T08:38:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:04 smithi082 ceph-mon[98694]: pgmap v18032: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 489 KiB/s wr, 18 op/s 2023-09-10T08:38:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33514.575608 secs 2023-09-10T08:38:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33515.777023 secs 2023-09-10T08:38:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:04 smithi098 ceph-mon[103124]: pgmap v18032: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 489 KiB/s wr, 18 op/s 2023-09-10T08:38:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33514.575608 secs 2023-09-10T08:38:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33515.777023 secs 2023-09-10T08:38:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33516.181747 secs 2023-09-10T08:38:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:38:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:38:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33516.181747 secs 2023-09-10T08:38:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:38:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:38:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33516.181747 secs 2023-09-10T08:38:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:38:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:38:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:06 smithi170 ceph-mon[102832]: pgmap v18033: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 489 KiB/s wr, 14 op/s 2023-09-10T08:38:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:38:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:38:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:06 smithi082 ceph-mon[98694]: pgmap v18033: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 489 KiB/s wr, 14 op/s 2023-09-10T08:38:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:38:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:38:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:06 smithi098 ceph-mon[103124]: pgmap v18033: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 489 KiB/s wr, 14 op/s 2023-09-10T08:38:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:38:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:38:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:08 smithi170 ceph-mon[102832]: pgmap v18034: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 534 KiB/s wr, 34 op/s 2023-09-10T08:38:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:08 smithi082 ceph-mon[98694]: pgmap v18034: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 534 KiB/s wr, 34 op/s 2023-09-10T08:38:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:08 smithi098 ceph-mon[103124]: pgmap v18034: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 534 KiB/s wr, 34 op/s 2023-09-10T08:38:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33519.575666 secs 2023-09-10T08:38:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:09 smithi170 ceph-mon[102832]: osdmap e1195: 12 total, 12 up, 12 in 2023-09-10T08:38:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33520.777079 secs 2023-09-10T08:38:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33519.575666 secs 2023-09-10T08:38:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:09 smithi082 ceph-mon[98694]: osdmap e1195: 12 total, 12 up, 12 in 2023-09-10T08:38:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33520.777079 secs 2023-09-10T08:38:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33519.575666 secs 2023-09-10T08:38:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:09 smithi098 ceph-mon[103124]: osdmap e1195: 12 total, 12 up, 12 in 2023-09-10T08:38:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33520.777079 secs 2023-09-10T08:38:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:10 smithi170 ceph-mon[102832]: pgmap v18036: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 396 KiB/s wr, 31 op/s 2023-09-10T08:38:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33521.181814 secs 2023-09-10T08:38:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:10 smithi082 ceph-mon[98694]: pgmap v18036: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 396 KiB/s wr, 31 op/s 2023-09-10T08:38:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33521.181814 secs 2023-09-10T08:38:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:10 smithi098 ceph-mon[103124]: pgmap v18036: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 396 KiB/s wr, 31 op/s 2023-09-10T08:38:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33521.181814 secs 2023-09-10T08:38:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:12 smithi170 ceph-mon[102832]: pgmap v18037: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 230 KiB/s wr, 30 op/s 2023-09-10T08:38:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:12 smithi082 ceph-mon[98694]: pgmap v18037: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 230 KiB/s wr, 30 op/s 2023-09-10T08:38:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:12 smithi098 ceph-mon[103124]: pgmap v18037: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 230 KiB/s wr, 30 op/s 2023-09-10T08:38:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:14 smithi170 ceph-mon[102832]: pgmap v18038: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 174 KiB/s wr, 30 op/s 2023-09-10T08:38:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33524.575725 secs 2023-09-10T08:38:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33525.777140 secs 2023-09-10T08:38:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:14 smithi082 ceph-mon[98694]: pgmap v18038: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 174 KiB/s wr, 30 op/s 2023-09-10T08:38:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33524.575725 secs 2023-09-10T08:38:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33525.777140 secs 2023-09-10T08:38:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:14 smithi098 ceph-mon[103124]: pgmap v18038: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 174 KiB/s wr, 30 op/s 2023-09-10T08:38:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33524.575725 secs 2023-09-10T08:38:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33525.777140 secs 2023-09-10T08:38:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33526.181881 secs 2023-09-10T08:38:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33526.181881 secs 2023-09-10T08:38:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33526.181881 secs 2023-09-10T08:38:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:16 smithi170 ceph-mon[102832]: pgmap v18039: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 174 KiB/s wr, 30 op/s 2023-09-10T08:38:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:16 smithi082 ceph-mon[98694]: pgmap v18039: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 174 KiB/s wr, 30 op/s 2023-09-10T08:38:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:16 smithi098 ceph-mon[103124]: pgmap v18039: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 174 KiB/s wr, 30 op/s 2023-09-10T08:38:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:18 smithi170 ceph-mon[102832]: pgmap v18040: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T08:38:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:18 smithi170 ceph-mon[102832]: osdmap e1196: 12 total, 12 up, 12 in 2023-09-10T08:38:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:18 smithi082 ceph-mon[98694]: pgmap v18040: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T08:38:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:18 smithi082 ceph-mon[98694]: osdmap e1196: 12 total, 12 up, 12 in 2023-09-10T08:38:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:18 smithi098 ceph-mon[103124]: pgmap v18040: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T08:38:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:18 smithi098 ceph-mon[103124]: osdmap e1196: 12 total, 12 up, 12 in 2023-09-10T08:38:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33529.575828 secs 2023-09-10T08:38:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33530.777197 secs 2023-09-10T08:38:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33529.575828 secs 2023-09-10T08:38:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33530.777197 secs 2023-09-10T08:38:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33529.575828 secs 2023-09-10T08:38:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33530.777197 secs 2023-09-10T08:38:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:20 smithi170 ceph-mon[102832]: pgmap v18042: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T08:38:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33531.181945 secs 2023-09-10T08:38:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:20 smithi082 ceph-mon[98694]: pgmap v18042: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T08:38:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33531.181945 secs 2023-09-10T08:38:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:20 smithi098 ceph-mon[103124]: pgmap v18042: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 12 op/s 2023-09-10T08:38:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33531.181945 secs 2023-09-10T08:38:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:22 smithi170 ceph-mon[102832]: pgmap v18043: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T08:38:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:22 smithi082 ceph-mon[98694]: pgmap v18043: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T08:38:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:22 smithi098 ceph-mon[103124]: pgmap v18043: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 91 KiB/s wr, 12 op/s 2023-09-10T08:38:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:24 smithi170 ceph-mon[102832]: pgmap v18044: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2023-09-10T08:38:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33534.575917 secs 2023-09-10T08:38:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33535.777256 secs 2023-09-10T08:38:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:24 smithi082 ceph-mon[98694]: pgmap v18044: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2023-09-10T08:38:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33534.575917 secs 2023-09-10T08:38:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33535.777256 secs 2023-09-10T08:38:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:24 smithi098 ceph-mon[103124]: pgmap v18044: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2023-09-10T08:38:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33534.575917 secs 2023-09-10T08:38:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33535.777256 secs 2023-09-10T08:38:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33536.182011 secs 2023-09-10T08:38:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33536.182011 secs 2023-09-10T08:38:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33536.182011 secs 2023-09-10T08:38:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:26 smithi170 ceph-mon[102832]: pgmap v18045: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2023-09-10T08:38:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:26 smithi082 ceph-mon[98694]: pgmap v18045: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2023-09-10T08:38:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:26 smithi098 ceph-mon[103124]: pgmap v18045: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2023-09-10T08:38:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:28 smithi170 ceph-mon[102832]: pgmap v18046: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 143 KiB/s wr, 12 op/s 2023-09-10T08:38:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:28 smithi082 ceph-mon[98694]: pgmap v18046: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 143 KiB/s wr, 12 op/s 2023-09-10T08:38:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:28 smithi098 ceph-mon[103124]: pgmap v18046: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 143 KiB/s wr, 12 op/s 2023-09-10T08:38:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33539.575984 secs 2023-09-10T08:38:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33540.777338 secs 2023-09-10T08:38:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33541.182078 secs 2023-09-10T08:38:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33539.575984 secs 2023-09-10T08:38:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33540.777338 secs 2023-09-10T08:38:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33541.182078 secs 2023-09-10T08:38:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33539.575984 secs 2023-09-10T08:38:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33540.777338 secs 2023-09-10T08:38:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33541.182078 secs 2023-09-10T08:38:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:30 smithi170 ceph-mon[102832]: pgmap v18047: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 140 KiB/s wr, 12 op/s 2023-09-10T08:38:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:30 smithi082 ceph-mon[98694]: pgmap v18047: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 140 KiB/s wr, 12 op/s 2023-09-10T08:38:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:30 smithi098 ceph-mon[103124]: pgmap v18047: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 140 KiB/s wr, 12 op/s 2023-09-10T08:38:31.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:31 smithi170 ceph-mon[102832]: pgmap v18048: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T08:38:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:31 smithi082 ceph-mon[98694]: pgmap v18048: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T08:38:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:31 smithi098 ceph-mon[103124]: pgmap v18048: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T08:38:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:34 smithi170 ceph-mon[102832]: pgmap v18049: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T08:38:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33544.576042 secs 2023-09-10T08:38:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:38:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33545.777425 secs 2023-09-10T08:38:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:38:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:34 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:38:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:34 smithi082 ceph-mon[98694]: pgmap v18049: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T08:38:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33544.576042 secs 2023-09-10T08:38:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:38:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33545.777425 secs 2023-09-10T08:38:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:38:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:34 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:38:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:34 smithi098 ceph-mon[103124]: pgmap v18049: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T08:38:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33544.576042 secs 2023-09-10T08:38:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:38:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33545.777425 secs 2023-09-10T08:38:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:38:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:34 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:38:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33546.182145 secs 2023-09-10T08:38:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:38:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:38:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:38:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33546.182145 secs 2023-09-10T08:38:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:38:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:38:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:38:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33546.182145 secs 2023-09-10T08:38:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:38:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:38:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:38:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:36 smithi170 ceph-mon[102832]: pgmap v18050: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 10 op/s 2023-09-10T08:38:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:36 smithi082 ceph-mon[98694]: pgmap v18050: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 10 op/s 2023-09-10T08:38:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:36 smithi098 ceph-mon[103124]: pgmap v18050: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 10 op/s 2023-09-10T08:38:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:38 smithi170 ceph-mon[102832]: pgmap v18051: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 236 KiB/s wr, 16 op/s 2023-09-10T08:38:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:38:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:38:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:38 smithi082 ceph-mon[98694]: pgmap v18051: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 236 KiB/s wr, 16 op/s 2023-09-10T08:38:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:38:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:38:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:38 smithi098 ceph-mon[103124]: pgmap v18051: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 236 KiB/s wr, 16 op/s 2023-09-10T08:38:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:38:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:38:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33550.777488 secs 2023-09-10T08:38:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:38:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33550.777488 secs 2023-09-10T08:38:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:38:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33550.777488 secs 2023-09-10T08:38:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:38:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:40 smithi170 ceph-mon[102832]: pgmap v18052: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T08:38:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33551.182210 secs 2023-09-10T08:38:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:38:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:38:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:38:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:40 smithi082 ceph-mon[98694]: pgmap v18052: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T08:38:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33551.182210 secs 2023-09-10T08:38:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:38:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:38:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:38:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:40 smithi098 ceph-mon[103124]: pgmap v18052: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 117 KiB/s wr, 11 op/s 2023-09-10T08:38:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33551.182210 secs 2023-09-10T08:38:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:38:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:38:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:38:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:42 smithi170 ceph-mon[102832]: pgmap v18053: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 418 KiB/s wr, 15 op/s 2023-09-10T08:38:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:42 smithi082 ceph-mon[98694]: pgmap v18053: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 418 KiB/s wr, 15 op/s 2023-09-10T08:38:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:42 smithi098 ceph-mon[103124]: pgmap v18053: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 418 KiB/s wr, 15 op/s 2023-09-10T08:38:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:44 smithi170 ceph-mon[102832]: pgmap v18054: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 518 KiB/s wr, 15 op/s 2023-09-10T08:38:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33555.777545 secs 2023-09-10T08:38:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:38:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:38:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:44 smithi082 ceph-mon[98694]: pgmap v18054: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 518 KiB/s wr, 15 op/s 2023-09-10T08:38:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33555.777545 secs 2023-09-10T08:38:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:38:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:38:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:44 smithi098 ceph-mon[103124]: pgmap v18054: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 518 KiB/s wr, 15 op/s 2023-09-10T08:38:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33555.777545 secs 2023-09-10T08:38:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:38:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:38:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33556.182278 secs 2023-09-10T08:38:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33556.182278 secs 2023-09-10T08:38:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33556.182278 secs 2023-09-10T08:38:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:46 smithi170 ceph-mon[102832]: pgmap v18055: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 518 KiB/s wr, 13 op/s 2023-09-10T08:38:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:38:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:38:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:46 smithi082 ceph-mon[98694]: pgmap v18055: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 518 KiB/s wr, 13 op/s 2023-09-10T08:38:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:38:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:38:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:46 smithi098 ceph-mon[103124]: pgmap v18055: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 518 KiB/s wr, 13 op/s 2023-09-10T08:38:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:38:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:38:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:48 smithi170 ceph-mon[102832]: pgmap v18056: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 539 KiB/s wr, 19 op/s 2023-09-10T08:38:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:48 smithi082 ceph-mon[98694]: pgmap v18056: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 539 KiB/s wr, 19 op/s 2023-09-10T08:38:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:48 smithi098 ceph-mon[103124]: pgmap v18056: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 539 KiB/s wr, 19 op/s 2023-09-10T08:38:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33560.777629 secs 2023-09-10T08:38:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33560.777629 secs 2023-09-10T08:38:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33560.777629 secs 2023-09-10T08:38:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:50 smithi170 ceph-mon[102832]: pgmap v18057: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 423 KiB/s wr, 13 op/s 2023-09-10T08:38:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33561.182340 secs 2023-09-10T08:38:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:38:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:38:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:50 smithi082 ceph-mon[98694]: pgmap v18057: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 423 KiB/s wr, 13 op/s 2023-09-10T08:38:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33561.182340 secs 2023-09-10T08:38:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:38:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:38:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:50 smithi098 ceph-mon[103124]: pgmap v18057: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 423 KiB/s wr, 13 op/s 2023-09-10T08:38:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33561.182340 secs 2023-09-10T08:38:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:38:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:38:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:52 smithi170 ceph-mon[102832]: pgmap v18058: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 610 KiB/s wr, 17 op/s 2023-09-10T08:38:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:38:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:38:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:52 smithi082 ceph-mon[98694]: pgmap v18058: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 610 KiB/s wr, 17 op/s 2023-09-10T08:38:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:38:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:38:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:52 smithi098 ceph-mon[103124]: pgmap v18058: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 610 KiB/s wr, 17 op/s 2023-09-10T08:38:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:38:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:38:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:54 smithi170 ceph-mon[102832]: pgmap v18059: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 541 KiB/s wr, 16 op/s 2023-09-10T08:38:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33565.777687 secs 2023-09-10T08:38:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:54 smithi082 ceph-mon[98694]: pgmap v18059: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 541 KiB/s wr, 16 op/s 2023-09-10T08:38:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33565.777687 secs 2023-09-10T08:38:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:54 smithi098 ceph-mon[103124]: pgmap v18059: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 541 KiB/s wr, 16 op/s 2023-09-10T08:38:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33565.777687 secs 2023-09-10T08:38:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33566.182405 secs 2023-09-10T08:38:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33566.182405 secs 2023-09-10T08:38:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33566.182405 secs 2023-09-10T08:38:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:56 smithi170 ceph-mon[102832]: pgmap v18060: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 441 KiB/s wr, 13 op/s 2023-09-10T08:38:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:56 smithi082 ceph-mon[98694]: pgmap v18060: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 441 KiB/s wr, 13 op/s 2023-09-10T08:38:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:56 smithi098 ceph-mon[103124]: pgmap v18060: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 441 KiB/s wr, 13 op/s 2023-09-10T08:38:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:58 smithi170 ceph-mon[102832]: pgmap v18061: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 488 KiB/s wr, 20 op/s 2023-09-10T08:38:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:38:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:38:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:58 smithi082 ceph-mon[98694]: pgmap v18061: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 488 KiB/s wr, 20 op/s 2023-09-10T08:38:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:38:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:38:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:58 smithi098 ceph-mon[103124]: pgmap v18061: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 488 KiB/s wr, 20 op/s 2023-09-10T08:38:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:38:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:38:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33569.576407 secs 2023-09-10T08:38:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:38:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33570.777787 secs 2023-09-10T08:38:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33569.576407 secs 2023-09-10T08:38:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:38:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33570.777787 secs 2023-09-10T08:38:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33569.576407 secs 2023-09-10T08:38:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:38:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33570.777787 secs 2023-09-10T08:39:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:00 smithi170 ceph-mon[102832]: pgmap v18062: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 466 KiB/s wr, 14 op/s 2023-09-10T08:39:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33571.182478 secs 2023-09-10T08:39:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:39:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:39:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:00 smithi082 ceph-mon[98694]: pgmap v18062: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 466 KiB/s wr, 14 op/s 2023-09-10T08:39:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33571.182478 secs 2023-09-10T08:39:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:39:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:39:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:00 smithi098 ceph-mon[103124]: pgmap v18062: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 466 KiB/s wr, 14 op/s 2023-09-10T08:39:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33571.182478 secs 2023-09-10T08:39:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:39:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:39:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:02 smithi170 ceph-mon[102832]: pgmap v18063: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 910 KiB/s wr, 18 op/s 2023-09-10T08:39:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:02 smithi082 ceph-mon[98694]: pgmap v18063: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 910 KiB/s wr, 18 op/s 2023-09-10T08:39:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:02 smithi098 ceph-mon[103124]: pgmap v18063: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 910 KiB/s wr, 18 op/s 2023-09-10T08:39:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:04 smithi170 ceph-mon[102832]: pgmap v18064: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 748 KiB/s wr, 16 op/s 2023-09-10T08:39:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33574.576468 secs 2023-09-10T08:39:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33575.777876 secs 2023-09-10T08:39:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33576.182546 secs 2023-09-10T08:39:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:39:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:04 smithi082 ceph-mon[98694]: pgmap v18064: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 748 KiB/s wr, 16 op/s 2023-09-10T08:39:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33574.576468 secs 2023-09-10T08:39:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33575.777876 secs 2023-09-10T08:39:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33576.182546 secs 2023-09-10T08:39:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:39:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:04 smithi098 ceph-mon[103124]: pgmap v18064: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 748 KiB/s wr, 16 op/s 2023-09-10T08:39:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33574.576468 secs 2023-09-10T08:39:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33575.777876 secs 2023-09-10T08:39:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33576.182546 secs 2023-09-10T08:39:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:39:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:05 smithi082 ceph-mon[98694]: pgmap v18065: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 515 KiB/s wr, 13 op/s 2023-09-10T08:39:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:39:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:05 smithi098 ceph-mon[103124]: pgmap v18065: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 515 KiB/s wr, 13 op/s 2023-09-10T08:39:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:39:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:05 smithi170 ceph-mon[102832]: pgmap v18065: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 515 KiB/s wr, 13 op/s 2023-09-10T08:39:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:39:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:07 smithi170 ceph-mon[102832]: pgmap v18066: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 818 KiB/s wr, 34 op/s 2023-09-10T08:39:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:07 smithi082 ceph-mon[98694]: pgmap v18066: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 818 KiB/s wr, 34 op/s 2023-09-10T08:39:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:07 smithi098 ceph-mon[103124]: pgmap v18066: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 818 KiB/s wr, 34 op/s 2023-09-10T08:39:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33579.576563 secs 2023-09-10T08:39:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:09 smithi170 ceph-mon[102832]: pgmap v18067: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 772 KiB/s wr, 27 op/s 2023-09-10T08:39:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:09 smithi170 ceph-mon[102832]: osdmap e1197: 12 total, 12 up, 12 in 2023-09-10T08:39:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33580.777939 secs 2023-09-10T08:39:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33581.182612 secs 2023-09-10T08:39:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33579.576563 secs 2023-09-10T08:39:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:09 smithi082 ceph-mon[98694]: pgmap v18067: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 772 KiB/s wr, 27 op/s 2023-09-10T08:39:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:09 smithi082 ceph-mon[98694]: osdmap e1197: 12 total, 12 up, 12 in 2023-09-10T08:39:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33580.777939 secs 2023-09-10T08:39:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33581.182612 secs 2023-09-10T08:39:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33579.576563 secs 2023-09-10T08:39:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:09 smithi098 ceph-mon[103124]: pgmap v18067: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 772 KiB/s wr, 27 op/s 2023-09-10T08:39:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:09 smithi098 ceph-mon[103124]: osdmap e1197: 12 total, 12 up, 12 in 2023-09-10T08:39:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33580.777939 secs 2023-09-10T08:39:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33581.182612 secs 2023-09-10T08:39:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:11 smithi170 ceph-mon[102832]: pgmap v18069: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 393 KiB/s wr, 31 op/s 2023-09-10T08:39:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:11 smithi082 ceph-mon[98694]: pgmap v18069: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 393 KiB/s wr, 31 op/s 2023-09-10T08:39:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:11 smithi098 ceph-mon[103124]: pgmap v18069: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 393 KiB/s wr, 31 op/s 2023-09-10T08:39:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:14 smithi170 ceph-mon[102832]: pgmap v18070: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 507 KiB/s wr, 32 op/s 2023-09-10T08:39:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33584.576624 secs 2023-09-10T08:39:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:14 smithi170 ceph-mon[102832]: osdmap e1198: 12 total, 12 up, 12 in 2023-09-10T08:39:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33585.777996 secs 2023-09-10T08:39:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:14 smithi082 ceph-mon[98694]: pgmap v18070: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 507 KiB/s wr, 32 op/s 2023-09-10T08:39:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33584.576624 secs 2023-09-10T08:39:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:14 smithi082 ceph-mon[98694]: osdmap e1198: 12 total, 12 up, 12 in 2023-09-10T08:39:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33585.777996 secs 2023-09-10T08:39:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:14 smithi098 ceph-mon[103124]: pgmap v18070: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 507 KiB/s wr, 32 op/s 2023-09-10T08:39:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33584.576624 secs 2023-09-10T08:39:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:14 smithi098 ceph-mon[103124]: osdmap e1198: 12 total, 12 up, 12 in 2023-09-10T08:39:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33585.777996 secs 2023-09-10T08:39:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33586.182678 secs 2023-09-10T08:39:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33586.182678 secs 2023-09-10T08:39:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33586.182678 secs 2023-09-10T08:39:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:16 smithi170 ceph-mon[102832]: pgmap v18072: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 9 op/s 2023-09-10T08:39:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:16 smithi082 ceph-mon[98694]: pgmap v18072: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 9 op/s 2023-09-10T08:39:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:16 smithi098 ceph-mon[103124]: pgmap v18072: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 9 op/s 2023-09-10T08:39:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:18 smithi170 ceph-mon[102832]: pgmap v18073: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 218 KiB/s wr, 17 op/s 2023-09-10T08:39:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:18 smithi082 ceph-mon[98694]: pgmap v18073: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 218 KiB/s wr, 17 op/s 2023-09-10T08:39:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:18 smithi098 ceph-mon[103124]: pgmap v18073: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 218 KiB/s wr, 17 op/s 2023-09-10T08:39:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33589.576707 secs 2023-09-10T08:39:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33590.778058 secs 2023-09-10T08:39:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33589.576707 secs 2023-09-10T08:39:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33590.778058 secs 2023-09-10T08:39:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33589.576707 secs 2023-09-10T08:39:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33590.778058 secs 2023-09-10T08:39:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:20 smithi170 ceph-mon[102832]: pgmap v18074: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 176 KiB/s wr, 14 op/s 2023-09-10T08:39:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33591.182745 secs 2023-09-10T08:39:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:20 smithi082 ceph-mon[98694]: pgmap v18074: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 176 KiB/s wr, 14 op/s 2023-09-10T08:39:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33591.182745 secs 2023-09-10T08:39:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:20 smithi098 ceph-mon[103124]: pgmap v18074: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 176 KiB/s wr, 14 op/s 2023-09-10T08:39:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33591.182745 secs 2023-09-10T08:39:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:22 smithi170 ceph-mon[102832]: pgmap v18075: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 175 KiB/s wr, 13 op/s 2023-09-10T08:39:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:22 smithi082 ceph-mon[98694]: pgmap v18075: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 175 KiB/s wr, 13 op/s 2023-09-10T08:39:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:22 smithi098 ceph-mon[103124]: pgmap v18075: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 175 KiB/s wr, 13 op/s 2023-09-10T08:39:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:24 smithi170 ceph-mon[102832]: pgmap v18076: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-10T08:39:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33594.576767 secs 2023-09-10T08:39:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33595.778116 secs 2023-09-10T08:39:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:24 smithi082 ceph-mon[98694]: pgmap v18076: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-10T08:39:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33594.576767 secs 2023-09-10T08:39:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33595.778116 secs 2023-09-10T08:39:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:24 smithi098 ceph-mon[103124]: pgmap v18076: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-10T08:39:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33594.576767 secs 2023-09-10T08:39:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33595.778116 secs 2023-09-10T08:39:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33596.182812 secs 2023-09-10T08:39:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33596.182812 secs 2023-09-10T08:39:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33596.182812 secs 2023-09-10T08:39:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:26 smithi170 ceph-mon[102832]: pgmap v18077: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T08:39:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:26 smithi082 ceph-mon[98694]: pgmap v18077: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T08:39:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:26 smithi098 ceph-mon[103124]: pgmap v18077: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T08:39:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:28 smithi170 ceph-mon[102832]: pgmap v18078: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 16 op/s 2023-09-10T08:39:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:28 smithi082 ceph-mon[98694]: pgmap v18078: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 16 op/s 2023-09-10T08:39:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:28 smithi098 ceph-mon[103124]: pgmap v18078: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 16 op/s 2023-09-10T08:39:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33599.576874 secs 2023-09-10T08:39:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33600.778204 secs 2023-09-10T08:39:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33599.576874 secs 2023-09-10T08:39:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33600.778204 secs 2023-09-10T08:39:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33599.576874 secs 2023-09-10T08:39:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33600.778204 secs 2023-09-10T08:39:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:30 smithi170 ceph-mon[102832]: pgmap v18079: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 10 op/s 2023-09-10T08:39:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33601.182880 secs 2023-09-10T08:39:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:30 smithi082 ceph-mon[98694]: pgmap v18079: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 10 op/s 2023-09-10T08:39:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33601.182880 secs 2023-09-10T08:39:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:30 smithi098 ceph-mon[103124]: pgmap v18079: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 10 op/s 2023-09-10T08:39:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33601.182880 secs 2023-09-10T08:39:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:32 smithi170 ceph-mon[102832]: pgmap v18080: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T08:39:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:32 smithi082 ceph-mon[98694]: pgmap v18080: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T08:39:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:32 smithi098 ceph-mon[103124]: pgmap v18080: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T08:39:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:34 smithi170 ceph-mon[102832]: pgmap v18081: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T08:39:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33604.576934 secs 2023-09-10T08:39:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:39:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33605.778245 secs 2023-09-10T08:39:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:34 smithi082 ceph-mon[98694]: pgmap v18081: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T08:39:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33604.576934 secs 2023-09-10T08:39:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:39:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33605.778245 secs 2023-09-10T08:39:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:34 smithi098 ceph-mon[103124]: pgmap v18081: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T08:39:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33604.576934 secs 2023-09-10T08:39:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:39:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33605.778245 secs 2023-09-10T08:39:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33606.182947 secs 2023-09-10T08:39:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:39:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:39:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:39:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:39:35.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:39:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33606.182947 secs 2023-09-10T08:39:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:39:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:39:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:39:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:39:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:39:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33606.182947 secs 2023-09-10T08:39:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:39:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:39:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:39:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:39:35.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:39:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:36 smithi170 ceph-mon[102832]: pgmap v18082: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T08:39:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:39:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:39:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:39:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:39:36.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:36 smithi082 ceph-mon[98694]: pgmap v18082: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T08:39:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:39:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:39:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:39:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:39:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:36 smithi098 ceph-mon[103124]: pgmap v18082: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T08:39:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:39:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:39:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:39:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:39:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:38 smithi170 ceph-mon[102832]: pgmap v18083: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 285 KiB/s wr, 17 op/s 2023-09-10T08:39:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:38 smithi082 ceph-mon[98694]: pgmap v18083: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 285 KiB/s wr, 17 op/s 2023-09-10T08:39:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:38 smithi098 ceph-mon[103124]: pgmap v18083: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 285 KiB/s wr, 17 op/s 2023-09-10T08:39:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33610.778304 secs 2023-09-10T08:39:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33611.183016 secs 2023-09-10T08:39:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:39:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33610.778304 secs 2023-09-10T08:39:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33611.183016 secs 2023-09-10T08:39:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:39:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33610.778304 secs 2023-09-10T08:39:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33611.183016 secs 2023-09-10T08:39:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:39:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:40 smithi170 ceph-mon[102832]: pgmap v18084: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T08:39:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:39:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:39:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:40 smithi082 ceph-mon[98694]: pgmap v18084: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T08:39:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:39:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:39:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:40 smithi098 ceph-mon[103124]: pgmap v18084: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T08:39:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:39:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:39:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:42 smithi170 ceph-mon[102832]: pgmap v18085: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 537 KiB/s wr, 16 op/s 2023-09-10T08:39:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:42 smithi082 ceph-mon[98694]: pgmap v18085: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 537 KiB/s wr, 16 op/s 2023-09-10T08:39:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:42 smithi098 ceph-mon[103124]: pgmap v18085: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 537 KiB/s wr, 16 op/s 2023-09-10T08:39:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:44 smithi170 ceph-mon[102832]: pgmap v18086: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 562 KiB/s wr, 38 op/s 2023-09-10T08:39:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:39:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33615.778380 secs 2023-09-10T08:39:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:44 smithi082 ceph-mon[98694]: pgmap v18086: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 562 KiB/s wr, 38 op/s 2023-09-10T08:39:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:39:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33615.778380 secs 2023-09-10T08:39:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:44 smithi098 ceph-mon[103124]: pgmap v18086: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 562 KiB/s wr, 38 op/s 2023-09-10T08:39:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:39:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33615.778380 secs 2023-09-10T08:39:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33616.183081 secs 2023-09-10T08:39:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33616.183081 secs 2023-09-10T08:39:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33616.183081 secs 2023-09-10T08:39:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:46 smithi170 ceph-mon[102832]: pgmap v18087: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 562 KiB/s wr, 35 op/s 2023-09-10T08:39:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:39:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:46 smithi082 ceph-mon[98694]: pgmap v18087: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 562 KiB/s wr, 35 op/s 2023-09-10T08:39:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:39:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:46 smithi098 ceph-mon[103124]: pgmap v18087: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 562 KiB/s wr, 35 op/s 2023-09-10T08:39:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:39:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:48 smithi170 ceph-mon[102832]: pgmap v18088: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 729 KiB/s wr, 42 op/s 2023-09-10T08:39:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:48 smithi082 ceph-mon[98694]: pgmap v18088: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 729 KiB/s wr, 42 op/s 2023-09-10T08:39:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:48 smithi098 ceph-mon[103124]: pgmap v18088: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 729 KiB/s wr, 42 op/s 2023-09-10T08:39:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33620.778423 secs 2023-09-10T08:39:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33620.778423 secs 2023-09-10T08:39:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33620.778423 secs 2023-09-10T08:39:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:50 smithi170 ceph-mon[102832]: pgmap v18089: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 564 KiB/s wr, 36 op/s 2023-09-10T08:39:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33621.183146 secs 2023-09-10T08:39:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:39:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:50 smithi082 ceph-mon[98694]: pgmap v18089: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 564 KiB/s wr, 36 op/s 2023-09-10T08:39:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33621.183146 secs 2023-09-10T08:39:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:39:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:50 smithi098 ceph-mon[103124]: pgmap v18089: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 564 KiB/s wr, 36 op/s 2023-09-10T08:39:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33621.183146 secs 2023-09-10T08:39:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:39:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:52 smithi170 ceph-mon[102832]: pgmap v18090: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 41 op/s 2023-09-10T08:39:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:39:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:52 smithi082 ceph-mon[98694]: pgmap v18090: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 41 op/s 2023-09-10T08:39:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:39:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:52 smithi098 ceph-mon[103124]: pgmap v18090: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 41 op/s 2023-09-10T08:39:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:39:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:54 smithi170 ceph-mon[102832]: pgmap v18091: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 825 KiB/s wr, 39 op/s 2023-09-10T08:39:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33624.577170 secs 2023-09-10T08:39:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33625.778487 secs 2023-09-10T08:39:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:54 smithi082 ceph-mon[98694]: pgmap v18091: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 825 KiB/s wr, 39 op/s 2023-09-10T08:39:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33624.577170 secs 2023-09-10T08:39:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33625.778487 secs 2023-09-10T08:39:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:54 smithi098 ceph-mon[103124]: pgmap v18091: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 825 KiB/s wr, 39 op/s 2023-09-10T08:39:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33624.577170 secs 2023-09-10T08:39:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33625.778487 secs 2023-09-10T08:39:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33626.183214 secs 2023-09-10T08:39:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33626.183214 secs 2023-09-10T08:39:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33626.183214 secs 2023-09-10T08:39:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:56 smithi170 ceph-mon[102832]: pgmap v18092: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 799 KiB/s wr, 14 op/s 2023-09-10T08:39:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:39:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:39:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:56 smithi082 ceph-mon[98694]: pgmap v18092: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 799 KiB/s wr, 14 op/s 2023-09-10T08:39:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:39:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:39:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:56 smithi098 ceph-mon[103124]: pgmap v18092: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 799 KiB/s wr, 14 op/s 2023-09-10T08:39:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:39:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:39:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:58 smithi170 ceph-mon[102832]: pgmap v18093: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 800 KiB/s wr, 20 op/s 2023-09-10T08:39:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:39:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:58 smithi082 ceph-mon[98694]: pgmap v18093: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 800 KiB/s wr, 20 op/s 2023-09-10T08:39:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:39:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:58 smithi098 ceph-mon[103124]: pgmap v18093: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 800 KiB/s wr, 20 op/s 2023-09-10T08:39:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:39:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:39:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33629.577231 secs 2023-09-10T08:39:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:39:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33630.778540 secs 2023-09-10T08:39:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33629.577231 secs 2023-09-10T08:39:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:39:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33630.778540 secs 2023-09-10T08:39:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33629.577231 secs 2023-09-10T08:39:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:39:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33630.778540 secs 2023-09-10T08:40:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:00 smithi170 ceph-mon[102832]: pgmap v18094: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 633 KiB/s wr, 13 op/s 2023-09-10T08:40:00.481 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33631.183279 secs 2023-09-10T08:40:00.481 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:40:00.481 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:00.481 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T08:40:00.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:00 smithi082 ceph-mon[98694]: pgmap v18094: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 633 KiB/s wr, 13 op/s 2023-09-10T08:40:00.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33631.183279 secs 2023-09-10T08:40:00.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:40:00.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:00.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T08:40:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:00 smithi098 ceph-mon[103124]: pgmap v18094: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 633 KiB/s wr, 13 op/s 2023-09-10T08:40:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33631.183279 secs 2023-09-10T08:40:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:40:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T08:40:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:02 smithi170 ceph-mon[102832]: pgmap v18095: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 681 KiB/s wr, 16 op/s 2023-09-10T08:40:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:02 smithi082 ceph-mon[98694]: pgmap v18095: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 681 KiB/s wr, 16 op/s 2023-09-10T08:40:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:02 smithi098 ceph-mon[103124]: pgmap v18095: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 681 KiB/s wr, 16 op/s 2023-09-10T08:40:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:04 smithi170 ceph-mon[102832]: pgmap v18096: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 350 KiB/s wr, 15 op/s 2023-09-10T08:40:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33634.577292 secs 2023-09-10T08:40:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33635.778599 secs 2023-09-10T08:40:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33636.183342 secs 2023-09-10T08:40:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:04 smithi082 ceph-mon[98694]: pgmap v18096: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 350 KiB/s wr, 15 op/s 2023-09-10T08:40:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33634.577292 secs 2023-09-10T08:40:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33635.778599 secs 2023-09-10T08:40:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33636.183342 secs 2023-09-10T08:40:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:04 smithi098 ceph-mon[103124]: pgmap v18096: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 350 KiB/s wr, 15 op/s 2023-09-10T08:40:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33634.577292 secs 2023-09-10T08:40:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33635.778599 secs 2023-09-10T08:40:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33636.183342 secs 2023-09-10T08:40:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:05 smithi170 ceph-mon[102832]: pgmap v18097: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 300 KiB/s wr, 12 op/s 2023-09-10T08:40:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:05 smithi082 ceph-mon[98694]: pgmap v18097: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 300 KiB/s wr, 12 op/s 2023-09-10T08:40:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:05 smithi098 ceph-mon[103124]: pgmap v18097: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 300 KiB/s wr, 12 op/s 2023-09-10T08:40:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:07 smithi170 ceph-mon[102832]: pgmap v18098: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 345 KiB/s wr, 31 op/s 2023-09-10T08:40:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:07 smithi082 ceph-mon[98694]: pgmap v18098: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 345 KiB/s wr, 31 op/s 2023-09-10T08:40:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:07 smithi098 ceph-mon[103124]: pgmap v18098: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 345 KiB/s wr, 31 op/s 2023-09-10T08:40:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33639.577433 secs 2023-09-10T08:40:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:09 smithi170 ceph-mon[102832]: pgmap v18099: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 345 KiB/s wr, 26 op/s 2023-09-10T08:40:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:09 smithi170 ceph-mon[102832]: osdmap e1199: 12 total, 12 up, 12 in 2023-09-10T08:40:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33640.778660 secs 2023-09-10T08:40:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33641.183407 secs 2023-09-10T08:40:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33639.577433 secs 2023-09-10T08:40:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:09 smithi082 ceph-mon[98694]: pgmap v18099: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 345 KiB/s wr, 26 op/s 2023-09-10T08:40:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:09 smithi082 ceph-mon[98694]: osdmap e1199: 12 total, 12 up, 12 in 2023-09-10T08:40:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33640.778660 secs 2023-09-10T08:40:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33641.183407 secs 2023-09-10T08:40:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33639.577433 secs 2023-09-10T08:40:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:09 smithi098 ceph-mon[103124]: pgmap v18099: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 345 KiB/s wr, 26 op/s 2023-09-10T08:40:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:09 smithi098 ceph-mon[103124]: osdmap e1199: 12 total, 12 up, 12 in 2023-09-10T08:40:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33640.778660 secs 2023-09-10T08:40:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33641.183407 secs 2023-09-10T08:40:11.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:11 smithi170 ceph-mon[102832]: pgmap v18101: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 357 KiB/s wr, 30 op/s 2023-09-10T08:40:11.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:11 smithi082 ceph-mon[98694]: pgmap v18101: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 357 KiB/s wr, 30 op/s 2023-09-10T08:40:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:11 smithi098 ceph-mon[103124]: pgmap v18101: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 357 KiB/s wr, 30 op/s 2023-09-10T08:40:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:14 smithi170 ceph-mon[102832]: pgmap v18102: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 223 KiB/s wr, 30 op/s 2023-09-10T08:40:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33644.577540 secs 2023-09-10T08:40:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:14 smithi170 ceph-mon[102832]: osdmap e1200: 12 total, 12 up, 12 in 2023-09-10T08:40:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33645.778719 secs 2023-09-10T08:40:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:14 smithi082 ceph-mon[98694]: pgmap v18102: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 223 KiB/s wr, 30 op/s 2023-09-10T08:40:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33644.577540 secs 2023-09-10T08:40:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:14 smithi082 ceph-mon[98694]: osdmap e1200: 12 total, 12 up, 12 in 2023-09-10T08:40:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33645.778719 secs 2023-09-10T08:40:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:14 smithi098 ceph-mon[103124]: pgmap v18102: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 223 KiB/s wr, 30 op/s 2023-09-10T08:40:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33644.577540 secs 2023-09-10T08:40:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:14 smithi098 ceph-mon[103124]: osdmap e1200: 12 total, 12 up, 12 in 2023-09-10T08:40:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33645.778719 secs 2023-09-10T08:40:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33646.183495 secs 2023-09-10T08:40:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33646.183495 secs 2023-09-10T08:40:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33646.183495 secs 2023-09-10T08:40:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:16 smithi170 ceph-mon[102832]: pgmap v18104: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 210 KiB/s wr, 8 op/s 2023-09-10T08:40:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:40:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:16 smithi082 ceph-mon[98694]: pgmap v18104: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 210 KiB/s wr, 8 op/s 2023-09-10T08:40:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:40:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:16 smithi098 ceph-mon[103124]: pgmap v18104: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 210 KiB/s wr, 8 op/s 2023-09-10T08:40:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:40:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:18 smithi170 ceph-mon[102832]: pgmap v18105: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 211 KiB/s wr, 17 op/s 2023-09-10T08:40:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:40:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:18 smithi082 ceph-mon[98694]: pgmap v18105: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 211 KiB/s wr, 17 op/s 2023-09-10T08:40:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:40:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:18 smithi098 ceph-mon[103124]: pgmap v18105: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 211 KiB/s wr, 17 op/s 2023-09-10T08:40:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:40:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33649.577642 secs 2023-09-10T08:40:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33650.778803 secs 2023-09-10T08:40:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33649.577642 secs 2023-09-10T08:40:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33650.778803 secs 2023-09-10T08:40:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33649.577642 secs 2023-09-10T08:40:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33650.778803 secs 2023-09-10T08:40:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:20 smithi170 ceph-mon[102832]: pgmap v18106: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 170 KiB/s wr, 14 op/s 2023-09-10T08:40:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33651.183559 secs 2023-09-10T08:40:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:20 smithi082 ceph-mon[98694]: pgmap v18106: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 170 KiB/s wr, 14 op/s 2023-09-10T08:40:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33651.183559 secs 2023-09-10T08:40:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:20 smithi098 ceph-mon[103124]: pgmap v18106: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 170 KiB/s wr, 14 op/s 2023-09-10T08:40:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33651.183559 secs 2023-09-10T08:40:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:22 smithi170 ceph-mon[102832]: pgmap v18107: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 168 KiB/s wr, 13 op/s 2023-09-10T08:40:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:22 smithi082 ceph-mon[98694]: pgmap v18107: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 168 KiB/s wr, 13 op/s 2023-09-10T08:40:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:22 smithi098 ceph-mon[103124]: pgmap v18107: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 168 KiB/s wr, 13 op/s 2023-09-10T08:40:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:24 smithi170 ceph-mon[102832]: pgmap v18108: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 13 op/s 2023-09-10T08:40:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33654.577743 secs 2023-09-10T08:40:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33655.778891 secs 2023-09-10T08:40:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:24 smithi082 ceph-mon[98694]: pgmap v18108: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 13 op/s 2023-09-10T08:40:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33654.577743 secs 2023-09-10T08:40:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33655.778891 secs 2023-09-10T08:40:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:24 smithi098 ceph-mon[103124]: pgmap v18108: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 13 op/s 2023-09-10T08:40:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33654.577743 secs 2023-09-10T08:40:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33655.778891 secs 2023-09-10T08:40:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33656.183625 secs 2023-09-10T08:40:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33656.183625 secs 2023-09-10T08:40:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33656.183625 secs 2023-09-10T08:40:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:26 smithi170 ceph-mon[102832]: pgmap v18109: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 12 op/s 2023-09-10T08:40:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:26 smithi082 ceph-mon[98694]: pgmap v18109: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 12 op/s 2023-09-10T08:40:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:26 smithi098 ceph-mon[103124]: pgmap v18109: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 12 op/s 2023-09-10T08:40:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:28 smithi170 ceph-mon[102832]: pgmap v18110: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 16 op/s 2023-09-10T08:40:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:28 smithi082 ceph-mon[98694]: pgmap v18110: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 16 op/s 2023-09-10T08:40:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:28 smithi098 ceph-mon[103124]: pgmap v18110: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 16 op/s 2023-09-10T08:40:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33659.577779 secs 2023-09-10T08:40:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33660.778949 secs 2023-09-10T08:40:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33659.577779 secs 2023-09-10T08:40:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33660.778949 secs 2023-09-10T08:40:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33659.577779 secs 2023-09-10T08:40:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33660.778949 secs 2023-09-10T08:40:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:30 smithi170 ceph-mon[102832]: pgmap v18111: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T08:40:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33661.183689 secs 2023-09-10T08:40:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:30 smithi082 ceph-mon[98694]: pgmap v18111: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T08:40:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33661.183689 secs 2023-09-10T08:40:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:30 smithi098 ceph-mon[103124]: pgmap v18111: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T08:40:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33661.183689 secs 2023-09-10T08:40:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:32 smithi170 ceph-mon[102832]: pgmap v18112: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T08:40:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:32 smithi082 ceph-mon[98694]: pgmap v18112: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T08:40:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:32 smithi098 ceph-mon[103124]: pgmap v18112: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 120 KiB/s wr, 13 op/s 2023-09-10T08:40:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:34 smithi170 ceph-mon[102832]: pgmap v18113: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 239 KiB/s wr, 13 op/s 2023-09-10T08:40:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33665.779039 secs 2023-09-10T08:40:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:34 smithi082 ceph-mon[98694]: pgmap v18113: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 239 KiB/s wr, 13 op/s 2023-09-10T08:40:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33665.779039 secs 2023-09-10T08:40:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:34 smithi098 ceph-mon[103124]: pgmap v18113: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 239 KiB/s wr, 13 op/s 2023-09-10T08:40:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33665.779039 secs 2023-09-10T08:40:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33666.183753 secs 2023-09-10T08:40:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:40:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33666.183753 secs 2023-09-10T08:40:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:40:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33666.183753 secs 2023-09-10T08:40:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:40:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:36 smithi170 ceph-mon[102832]: pgmap v18114: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 10 op/s 2023-09-10T08:40:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:40:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:40:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:40:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:36 smithi082 ceph-mon[98694]: pgmap v18114: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 10 op/s 2023-09-10T08:40:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:40:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:40:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:40:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:36 smithi098 ceph-mon[103124]: pgmap v18114: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 10 op/s 2023-09-10T08:40:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:40:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:40:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:40:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:40:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:40:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:40:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:40:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:40:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:40:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:40:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:40:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:40:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:38 smithi170 ceph-mon[102832]: pgmap v18115: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 237 KiB/s wr, 16 op/s 2023-09-10T08:40:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:38 smithi082 ceph-mon[98694]: pgmap v18115: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 237 KiB/s wr, 16 op/s 2023-09-10T08:40:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:38 smithi098 ceph-mon[103124]: pgmap v18115: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 237 KiB/s wr, 16 op/s 2023-09-10T08:40:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:39 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33669.577898 secs 2023-09-10T08:40:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33670.779122 secs 2023-09-10T08:40:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:39 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33669.577898 secs 2023-09-10T08:40:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33670.779122 secs 2023-09-10T08:40:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:39 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33669.577898 secs 2023-09-10T08:40:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33670.779122 secs 2023-09-10T08:40:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:40 smithi170 ceph-mon[102832]: pgmap v18116: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 237 KiB/s wr, 10 op/s 2023-09-10T08:40:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33671.183818 secs 2023-09-10T08:40:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:40:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:40 smithi082 ceph-mon[98694]: pgmap v18116: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 237 KiB/s wr, 10 op/s 2023-09-10T08:40:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33671.183818 secs 2023-09-10T08:40:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:40:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:40 smithi098 ceph-mon[103124]: pgmap v18116: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 237 KiB/s wr, 10 op/s 2023-09-10T08:40:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33671.183818 secs 2023-09-10T08:40:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:40:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:42 smithi170 ceph-mon[102832]: pgmap v18117: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 237 KiB/s wr, 13 op/s 2023-09-10T08:40:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:42 smithi082 ceph-mon[98694]: pgmap v18117: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 237 KiB/s wr, 13 op/s 2023-09-10T08:40:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:42 smithi098 ceph-mon[103124]: pgmap v18117: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 237 KiB/s wr, 13 op/s 2023-09-10T08:40:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:40:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:40:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:40:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:44 smithi170 ceph-mon[102832]: pgmap v18118: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 493 KiB/s wr, 14 op/s 2023-09-10T08:40:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:44 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33674.577960 secs 2023-09-10T08:40:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33675.779181 secs 2023-09-10T08:40:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33676.183885 secs 2023-09-10T08:40:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:44 smithi082 ceph-mon[98694]: pgmap v18118: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 493 KiB/s wr, 14 op/s 2023-09-10T08:40:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:44 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33674.577960 secs 2023-09-10T08:40:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33675.779181 secs 2023-09-10T08:40:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33676.183885 secs 2023-09-10T08:40:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:44 smithi098 ceph-mon[103124]: pgmap v18118: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 493 KiB/s wr, 14 op/s 2023-09-10T08:40:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:44 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33674.577960 secs 2023-09-10T08:40:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33675.779181 secs 2023-09-10T08:40:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33676.183885 secs 2023-09-10T08:40:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:46 smithi170 ceph-mon[102832]: pgmap v18119: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 373 KiB/s wr, 11 op/s 2023-09-10T08:40:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:40:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:46 smithi082 ceph-mon[98694]: pgmap v18119: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 373 KiB/s wr, 11 op/s 2023-09-10T08:40:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:40:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:46 smithi098 ceph-mon[103124]: pgmap v18119: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 373 KiB/s wr, 11 op/s 2023-09-10T08:40:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:40:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:47.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:47 smithi170 ceph-mon[102832]: pgmap v18120: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 535 KiB/s wr, 19 op/s 2023-09-10T08:40:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:47 smithi082 ceph-mon[98694]: pgmap v18120: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 535 KiB/s wr, 19 op/s 2023-09-10T08:40:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:47 smithi098 ceph-mon[103124]: pgmap v18120: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 535 KiB/s wr, 19 op/s 2023-09-10T08:40:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33680.779239 secs 2023-09-10T08:40:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33680.779239 secs 2023-09-10T08:40:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33680.779239 secs 2023-09-10T08:40:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:50 smithi170 ceph-mon[102832]: pgmap v18121: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 418 KiB/s wr, 13 op/s 2023-09-10T08:40:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33681.183954 secs 2023-09-10T08:40:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:40:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:50 smithi082 ceph-mon[98694]: pgmap v18121: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 418 KiB/s wr, 13 op/s 2023-09-10T08:40:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33681.183954 secs 2023-09-10T08:40:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:40:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:50 smithi098 ceph-mon[103124]: pgmap v18121: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 418 KiB/s wr, 13 op/s 2023-09-10T08:40:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33681.183954 secs 2023-09-10T08:40:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:40:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:51 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:40:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:51 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:40:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:51 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:40:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:52 smithi170 ceph-mon[102832]: pgmap v18122: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 695 KiB/s wr, 17 op/s 2023-09-10T08:40:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:52 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:40:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:40:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:52 smithi082 ceph-mon[98694]: pgmap v18122: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 695 KiB/s wr, 17 op/s 2023-09-10T08:40:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:52 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:40:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:40:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:52 smithi098 ceph-mon[103124]: pgmap v18122: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 695 KiB/s wr, 17 op/s 2023-09-10T08:40:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:52 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:40:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:40:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:54 smithi170 ceph-mon[102832]: pgmap v18123: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 727 KiB/s wr, 18 op/s 2023-09-10T08:40:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:40:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33685.779299 secs 2023-09-10T08:40:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:54 smithi082 ceph-mon[98694]: pgmap v18123: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 727 KiB/s wr, 18 op/s 2023-09-10T08:40:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:40:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33685.779299 secs 2023-09-10T08:40:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:54 smithi098 ceph-mon[103124]: pgmap v18123: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 727 KiB/s wr, 18 op/s 2023-09-10T08:40:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:40:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33685.779299 secs 2023-09-10T08:40:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33686.184020 secs 2023-09-10T08:40:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33686.184020 secs 2023-09-10T08:40:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33686.184020 secs 2023-09-10T08:40:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:56 smithi170 ceph-mon[102832]: pgmap v18124: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 470 KiB/s wr, 15 op/s 2023-09-10T08:40:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:40:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:56 smithi082 ceph-mon[98694]: pgmap v18124: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 470 KiB/s wr, 15 op/s 2023-09-10T08:40:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:40:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:56 smithi098 ceph-mon[103124]: pgmap v18124: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 470 KiB/s wr, 15 op/s 2023-09-10T08:40:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:40:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:40:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:58 smithi170 ceph-mon[102832]: pgmap v18125: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 512 KiB/s wr, 21 op/s 2023-09-10T08:40:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:58 smithi082 ceph-mon[98694]: pgmap v18125: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 512 KiB/s wr, 21 op/s 2023-09-10T08:40:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:58 smithi098 ceph-mon[103124]: pgmap v18125: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 512 KiB/s wr, 21 op/s 2023-09-10T08:40:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:40:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33690.779383 secs 2023-09-10T08:40:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:40:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33690.779383 secs 2023-09-10T08:40:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:40:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33690.779383 secs 2023-09-10T08:41:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:00 smithi170 ceph-mon[102832]: pgmap v18126: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 349 KiB/s wr, 14 op/s 2023-09-10T08:41:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33691.184108 secs 2023-09-10T08:41:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:00 smithi082 ceph-mon[98694]: pgmap v18126: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 349 KiB/s wr, 14 op/s 2023-09-10T08:41:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33691.184108 secs 2023-09-10T08:41:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:00 smithi098 ceph-mon[103124]: pgmap v18126: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 349 KiB/s wr, 14 op/s 2023-09-10T08:41:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33691.184108 secs 2023-09-10T08:41:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:02 smithi170 ceph-mon[102832]: pgmap v18127: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 496 KiB/s wr, 18 op/s 2023-09-10T08:41:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:02 smithi082 ceph-mon[98694]: pgmap v18127: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 496 KiB/s wr, 18 op/s 2023-09-10T08:41:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:02 smithi098 ceph-mon[103124]: pgmap v18127: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 496 KiB/s wr, 18 op/s 2023-09-10T08:41:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:04 smithi170 ceph-mon[102832]: pgmap v18128: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 520 KiB/s wr, 17 op/s 2023-09-10T08:41:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33694.578249 secs 2023-09-10T08:41:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:41:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:41:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33695.779443 secs 2023-09-10T08:41:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:04 smithi082 ceph-mon[98694]: pgmap v18128: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 520 KiB/s wr, 17 op/s 2023-09-10T08:41:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33694.578249 secs 2023-09-10T08:41:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:41:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:41:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33695.779443 secs 2023-09-10T08:41:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:04 smithi098 ceph-mon[103124]: pgmap v18128: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 520 KiB/s wr, 17 op/s 2023-09-10T08:41:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33694.578249 secs 2023-09-10T08:41:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:41:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:41:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33695.779443 secs 2023-09-10T08:41:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33696.184175 secs 2023-09-10T08:41:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33696.184175 secs 2023-09-10T08:41:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33696.184175 secs 2023-09-10T08:41:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:06 smithi170 ceph-mon[102832]: pgmap v18129: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 488 KiB/s wr, 14 op/s 2023-09-10T08:41:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:41:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:06 smithi082 ceph-mon[98694]: pgmap v18129: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 488 KiB/s wr, 14 op/s 2023-09-10T08:41:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:41:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:06 smithi098 ceph-mon[103124]: pgmap v18129: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 488 KiB/s wr, 14 op/s 2023-09-10T08:41:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:41:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:08 smithi170 ceph-mon[102832]: pgmap v18130: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 745 KiB/s wr, 35 op/s 2023-09-10T08:41:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:08 smithi082 ceph-mon[98694]: pgmap v18130: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 745 KiB/s wr, 35 op/s 2023-09-10T08:41:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:08 smithi098 ceph-mon[103124]: pgmap v18130: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 745 KiB/s wr, 35 op/s 2023-09-10T08:41:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33699.578356 secs 2023-09-10T08:41:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:09 smithi170 ceph-mon[102832]: osdmap e1201: 12 total, 12 up, 12 in 2023-09-10T08:41:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33700.779496 secs 2023-09-10T08:41:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33699.578356 secs 2023-09-10T08:41:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:09 smithi082 ceph-mon[98694]: osdmap e1201: 12 total, 12 up, 12 in 2023-09-10T08:41:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33700.779496 secs 2023-09-10T08:41:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33699.578356 secs 2023-09-10T08:41:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:09 smithi098 ceph-mon[103124]: osdmap e1201: 12 total, 12 up, 12 in 2023-09-10T08:41:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33700.779496 secs 2023-09-10T08:41:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:10 smithi170 ceph-mon[102832]: pgmap v18132: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 845 KiB/s wr, 34 op/s 2023-09-10T08:41:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33701.184241 secs 2023-09-10T08:41:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:41:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:10 smithi082 ceph-mon[98694]: pgmap v18132: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 845 KiB/s wr, 34 op/s 2023-09-10T08:41:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33701.184241 secs 2023-09-10T08:41:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:41:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:10 smithi098 ceph-mon[103124]: pgmap v18132: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 845 KiB/s wr, 34 op/s 2023-09-10T08:41:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33701.184241 secs 2023-09-10T08:41:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:41:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:12 smithi170 ceph-mon[102832]: pgmap v18133: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 731 KiB/s wr, 34 op/s 2023-09-10T08:41:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:41:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:12 smithi082 ceph-mon[98694]: pgmap v18133: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 731 KiB/s wr, 34 op/s 2023-09-10T08:41:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:41:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:12 smithi098 ceph-mon[103124]: pgmap v18133: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 731 KiB/s wr, 34 op/s 2023-09-10T08:41:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:41:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:14 smithi170 ceph-mon[102832]: pgmap v18134: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 649 KiB/s wr, 33 op/s 2023-09-10T08:41:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33704.578417 secs 2023-09-10T08:41:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33705.779580 secs 2023-09-10T08:41:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:14 smithi082 ceph-mon[98694]: pgmap v18134: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 649 KiB/s wr, 33 op/s 2023-09-10T08:41:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33704.578417 secs 2023-09-10T08:41:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33705.779580 secs 2023-09-10T08:41:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:14 smithi098 ceph-mon[103124]: pgmap v18134: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 649 KiB/s wr, 33 op/s 2023-09-10T08:41:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33704.578417 secs 2023-09-10T08:41:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33705.779580 secs 2023-09-10T08:41:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33706.184309 secs 2023-09-10T08:41:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33706.184309 secs 2023-09-10T08:41:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33706.184309 secs 2023-09-10T08:41:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:16 smithi170 ceph-mon[102832]: pgmap v18135: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 649 KiB/s wr, 33 op/s 2023-09-10T08:41:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:16 smithi082 ceph-mon[98694]: pgmap v18135: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 649 KiB/s wr, 33 op/s 2023-09-10T08:41:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:16 smithi098 ceph-mon[103124]: pgmap v18135: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 649 KiB/s wr, 33 op/s 2023-09-10T08:41:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:18 smithi170 ceph-mon[102832]: pgmap v18136: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 456 KiB/s wr, 14 op/s 2023-09-10T08:41:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:18 smithi170 ceph-mon[102832]: osdmap e1202: 12 total, 12 up, 12 in 2023-09-10T08:41:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:18 smithi082 ceph-mon[98694]: pgmap v18136: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 456 KiB/s wr, 14 op/s 2023-09-10T08:41:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:18 smithi082 ceph-mon[98694]: osdmap e1202: 12 total, 12 up, 12 in 2023-09-10T08:41:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:18 smithi098 ceph-mon[103124]: pgmap v18136: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 456 KiB/s wr, 14 op/s 2023-09-10T08:41:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:18 smithi098 ceph-mon[103124]: osdmap e1202: 12 total, 12 up, 12 in 2023-09-10T08:41:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33709.578476 secs 2023-09-10T08:41:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33710.779638 secs 2023-09-10T08:41:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33709.578476 secs 2023-09-10T08:41:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33710.779638 secs 2023-09-10T08:41:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33709.578476 secs 2023-09-10T08:41:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33710.779638 secs 2023-09-10T08:41:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:20 smithi170 ceph-mon[102832]: pgmap v18138: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 456 KiB/s wr, 14 op/s 2023-09-10T08:41:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33711.184374 secs 2023-09-10T08:41:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:20 smithi082 ceph-mon[98694]: pgmap v18138: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 456 KiB/s wr, 14 op/s 2023-09-10T08:41:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33711.184374 secs 2023-09-10T08:41:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:20 smithi098 ceph-mon[103124]: pgmap v18138: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 456 KiB/s wr, 14 op/s 2023-09-10T08:41:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33711.184374 secs 2023-09-10T08:41:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:22 smithi170 ceph-mon[102832]: pgmap v18139: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 394 KiB/s wr, 13 op/s 2023-09-10T08:41:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:22 smithi082 ceph-mon[98694]: pgmap v18139: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 394 KiB/s wr, 13 op/s 2023-09-10T08:41:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:22 smithi098 ceph-mon[103124]: pgmap v18139: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 394 KiB/s wr, 13 op/s 2023-09-10T08:41:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:24 smithi170 ceph-mon[102832]: pgmap v18140: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T08:41:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33714.578593 secs 2023-09-10T08:41:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33715.779697 secs 2023-09-10T08:41:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33716.184439 secs 2023-09-10T08:41:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:24 smithi082 ceph-mon[98694]: pgmap v18140: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T08:41:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33714.578593 secs 2023-09-10T08:41:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33715.779697 secs 2023-09-10T08:41:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33716.184439 secs 2023-09-10T08:41:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:24 smithi098 ceph-mon[103124]: pgmap v18140: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T08:41:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33714.578593 secs 2023-09-10T08:41:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33715.779697 secs 2023-09-10T08:41:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33716.184439 secs 2023-09-10T08:41:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:26 smithi170 ceph-mon[102832]: pgmap v18141: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T08:41:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:26 smithi082 ceph-mon[98694]: pgmap v18141: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T08:41:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:26 smithi098 ceph-mon[103124]: pgmap v18141: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 116 KiB/s wr, 12 op/s 2023-09-10T08:41:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:28 smithi170 ceph-mon[102832]: pgmap v18142: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-09-10T08:41:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:28 smithi082 ceph-mon[98694]: pgmap v18142: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-09-10T08:41:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:28 smithi098 ceph-mon[103124]: pgmap v18142: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-09-10T08:41:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33719.578655 secs 2023-09-10T08:41:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33720.779758 secs 2023-09-10T08:41:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33721.184488 secs 2023-09-10T08:41:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33719.578655 secs 2023-09-10T08:41:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33720.779758 secs 2023-09-10T08:41:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33721.184488 secs 2023-09-10T08:41:29.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33719.578655 secs 2023-09-10T08:41:29.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33720.779758 secs 2023-09-10T08:41:29.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33721.184488 secs 2023-09-10T08:41:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:30 smithi170 ceph-mon[102832]: pgmap v18143: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-09-10T08:41:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:30 smithi082 ceph-mon[98694]: pgmap v18143: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-09-10T08:41:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:30 smithi098 ceph-mon[103124]: pgmap v18143: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-09-10T08:41:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:32 smithi170 ceph-mon[102832]: pgmap v18144: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 108 KiB/s wr, 13 op/s 2023-09-10T08:41:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:32 smithi082 ceph-mon[98694]: pgmap v18144: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 108 KiB/s wr, 13 op/s 2023-09-10T08:41:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:32 smithi098 ceph-mon[103124]: pgmap v18144: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 108 KiB/s wr, 13 op/s 2023-09-10T08:41:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:34 smithi170 ceph-mon[102832]: pgmap v18145: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 13 op/s 2023-09-10T08:41:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33724.578755 secs 2023-09-10T08:41:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33725.779826 secs 2023-09-10T08:41:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33726.184556 secs 2023-09-10T08:41:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:34 smithi082 ceph-mon[98694]: pgmap v18145: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 13 op/s 2023-09-10T08:41:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33724.578755 secs 2023-09-10T08:41:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33725.779826 secs 2023-09-10T08:41:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33726.184556 secs 2023-09-10T08:41:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:34 smithi098 ceph-mon[103124]: pgmap v18145: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 13 op/s 2023-09-10T08:41:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33724.578755 secs 2023-09-10T08:41:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33725.779826 secs 2023-09-10T08:41:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33726.184556 secs 2023-09-10T08:41:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:35 smithi098 ceph-mon[103124]: pgmap v18146: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T08:41:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:35 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:41:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:35 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:35 smithi170 ceph-mon[102832]: pgmap v18146: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T08:41:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:35 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:41:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:35 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:35 smithi082 ceph-mon[98694]: pgmap v18146: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T08:41:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:35 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:41:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:35 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:41:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:41:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:41:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:37 smithi082 ceph-mon[98694]: pgmap v18147: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 238 KiB/s wr, 16 op/s 2023-09-10T08:41:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:41:37.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:41:37.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:41:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:37 smithi098 ceph-mon[103124]: pgmap v18147: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 238 KiB/s wr, 16 op/s 2023-09-10T08:41:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:41:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:41:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:41:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:37 smithi170 ceph-mon[102832]: pgmap v18147: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 238 KiB/s wr, 16 op/s 2023-09-10T08:41:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:41:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:41:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:41:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:41:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:41:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:41:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:39 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33729.578780 secs 2023-09-10T08:41:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:39 smithi082 ceph-mon[98694]: pgmap v18148: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 238 KiB/s wr, 11 op/s 2023-09-10T08:41:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33730.779889 secs 2023-09-10T08:41:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33731.184619 secs 2023-09-10T08:41:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:39 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33729.578780 secs 2023-09-10T08:41:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:39 smithi098 ceph-mon[103124]: pgmap v18148: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 238 KiB/s wr, 11 op/s 2023-09-10T08:41:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33730.779889 secs 2023-09-10T08:41:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33731.184619 secs 2023-09-10T08:41:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:39 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33729.578780 secs 2023-09-10T08:41:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:39 smithi170 ceph-mon[102832]: pgmap v18148: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 238 KiB/s wr, 11 op/s 2023-09-10T08:41:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33730.779889 secs 2023-09-10T08:41:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33731.184619 secs 2023-09-10T08:41:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:41:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:41:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:41:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:41 smithi082 ceph-mon[98694]: pgmap v18149: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 873 KiB/s wr, 17 op/s 2023-09-10T08:41:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:41 smithi098 ceph-mon[103124]: pgmap v18149: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 873 KiB/s wr, 17 op/s 2023-09-10T08:41:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:41 smithi170 ceph-mon[102832]: pgmap v18149: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 873 KiB/s wr, 17 op/s 2023-09-10T08:41:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:43 smithi098 ceph-mon[103124]: pgmap v18150: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 767 KiB/s wr, 15 op/s 2023-09-10T08:41:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:41:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:43 smithi170 ceph-mon[102832]: pgmap v18150: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 767 KiB/s wr, 15 op/s 2023-09-10T08:41:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:41:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:43 smithi082 ceph-mon[98694]: pgmap v18150: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 767 KiB/s wr, 15 op/s 2023-09-10T08:41:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:41:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:44 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:41:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:41:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33735.779968 secs 2023-09-10T08:41:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33736.184684 secs 2023-09-10T08:41:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:44 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:41:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:41:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33735.779968 secs 2023-09-10T08:41:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33736.184684 secs 2023-09-10T08:41:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:44 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:41:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:41:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33735.779968 secs 2023-09-10T08:41:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33736.184684 secs 2023-09-10T08:41:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:45 smithi098 ceph-mon[103124]: pgmap v18151: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 752 KiB/s wr, 12 op/s 2023-09-10T08:41:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:45 smithi170 ceph-mon[102832]: pgmap v18151: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 752 KiB/s wr, 12 op/s 2023-09-10T08:41:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:45 smithi082 ceph-mon[98694]: pgmap v18151: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 752 KiB/s wr, 12 op/s 2023-09-10T08:41:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:41:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:41:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:41:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:47 smithi098 ceph-mon[103124]: pgmap v18152: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 775 KiB/s wr, 19 op/s 2023-09-10T08:41:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:47 smithi170 ceph-mon[102832]: pgmap v18152: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 775 KiB/s wr, 19 op/s 2023-09-10T08:41:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:47 smithi082 ceph-mon[98694]: pgmap v18152: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 775 KiB/s wr, 19 op/s 2023-09-10T08:41:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33740.780026 secs 2023-09-10T08:41:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33740.780026 secs 2023-09-10T08:41:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33740.780026 secs 2023-09-10T08:41:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:50 smithi170 ceph-mon[102832]: pgmap v18153: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 658 KiB/s wr, 14 op/s 2023-09-10T08:41:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33741.184748 secs 2023-09-10T08:41:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:41:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:50 smithi082 ceph-mon[98694]: pgmap v18153: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 658 KiB/s wr, 14 op/s 2023-09-10T08:41:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33741.184748 secs 2023-09-10T08:41:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:41:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:50 smithi098 ceph-mon[103124]: pgmap v18153: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 658 KiB/s wr, 14 op/s 2023-09-10T08:41:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33741.184748 secs 2023-09-10T08:41:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:41:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:52 smithi170 ceph-mon[102832]: pgmap v18154: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 19 op/s 2023-09-10T08:41:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:41:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:52 smithi082 ceph-mon[98694]: pgmap v18154: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 19 op/s 2023-09-10T08:41:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:41:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:52 smithi098 ceph-mon[103124]: pgmap v18154: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 19 op/s 2023-09-10T08:41:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:41:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:54 smithi170 ceph-mon[102832]: pgmap v18155: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 15 op/s 2023-09-10T08:41:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:41:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33745.780085 secs 2023-09-10T08:41:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:54 smithi082 ceph-mon[98694]: pgmap v18155: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 15 op/s 2023-09-10T08:41:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:41:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33745.780085 secs 2023-09-10T08:41:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:54 smithi098 ceph-mon[103124]: pgmap v18155: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 15 op/s 2023-09-10T08:41:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:41:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33745.780085 secs 2023-09-10T08:41:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33746.184812 secs 2023-09-10T08:41:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33746.184812 secs 2023-09-10T08:41:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33746.184812 secs 2023-09-10T08:41:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:56 smithi170 ceph-mon[102832]: pgmap v18156: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 445 KiB/s wr, 14 op/s 2023-09-10T08:41:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:41:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:56 smithi082 ceph-mon[98694]: pgmap v18156: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 445 KiB/s wr, 14 op/s 2023-09-10T08:41:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:41:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:56 smithi098 ceph-mon[103124]: pgmap v18156: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 445 KiB/s wr, 14 op/s 2023-09-10T08:41:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:41:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:41:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:58 smithi170 ceph-mon[102832]: pgmap v18157: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 866 KiB/s wr, 20 op/s 2023-09-10T08:41:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:58 smithi082 ceph-mon[98694]: pgmap v18157: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 866 KiB/s wr, 20 op/s 2023-09-10T08:41:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:58 smithi098 ceph-mon[103124]: pgmap v18157: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 866 KiB/s wr, 20 op/s 2023-09-10T08:41:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33749.579128 secs 2023-09-10T08:41:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33750.780144 secs 2023-09-10T08:41:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:41:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33751.184879 secs 2023-09-10T08:41:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33749.579128 secs 2023-09-10T08:41:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33750.780144 secs 2023-09-10T08:41:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:41:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33751.184879 secs 2023-09-10T08:41:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33749.579128 secs 2023-09-10T08:41:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33750.780144 secs 2023-09-10T08:41:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:41:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33751.184879 secs 2023-09-10T08:42:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:00 smithi170 ceph-mon[102832]: pgmap v18158: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 843 KiB/s wr, 13 op/s 2023-09-10T08:42:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:42:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:42:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:00 smithi082 ceph-mon[98694]: pgmap v18158: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 843 KiB/s wr, 13 op/s 2023-09-10T08:42:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:42:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:42:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:00 smithi098 ceph-mon[103124]: pgmap v18158: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 843 KiB/s wr, 13 op/s 2023-09-10T08:42:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:42:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:42:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:02 smithi170 ceph-mon[102832]: pgmap v18159: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 983 KiB/s wr, 32 op/s 2023-09-10T08:42:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:02 smithi082 ceph-mon[98694]: pgmap v18159: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 983 KiB/s wr, 32 op/s 2023-09-10T08:42:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:02 smithi098 ceph-mon[103124]: pgmap v18159: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 983 KiB/s wr, 32 op/s 2023-09-10T08:42:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:04 smithi170 ceph-mon[102832]: pgmap v18160: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 560 KiB/s wr, 28 op/s 2023-09-10T08:42:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33754.579195 secs 2023-09-10T08:42:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:04 smithi170 ceph-mon[102832]: osdmap e1203: 12 total, 12 up, 12 in 2023-09-10T08:42:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:42:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33755.780205 secs 2023-09-10T08:42:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33756.184947 secs 2023-09-10T08:42:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:04 smithi082 ceph-mon[98694]: pgmap v18160: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 560 KiB/s wr, 28 op/s 2023-09-10T08:42:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33754.579195 secs 2023-09-10T08:42:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:04 smithi082 ceph-mon[98694]: osdmap e1203: 12 total, 12 up, 12 in 2023-09-10T08:42:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:42:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33755.780205 secs 2023-09-10T08:42:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33756.184947 secs 2023-09-10T08:42:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:04 smithi098 ceph-mon[103124]: pgmap v18160: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 560 KiB/s wr, 28 op/s 2023-09-10T08:42:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33754.579195 secs 2023-09-10T08:42:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:04 smithi098 ceph-mon[103124]: osdmap e1203: 12 total, 12 up, 12 in 2023-09-10T08:42:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:42:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33755.780205 secs 2023-09-10T08:42:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33756.184947 secs 2023-09-10T08:42:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:06 smithi170 ceph-mon[102832]: pgmap v18162: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 672 KiB/s wr, 32 op/s 2023-09-10T08:42:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:42:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:06 smithi082 ceph-mon[98694]: pgmap v18162: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 672 KiB/s wr, 32 op/s 2023-09-10T08:42:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:42:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:06 smithi098 ceph-mon[103124]: pgmap v18162: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 672 KiB/s wr, 32 op/s 2023-09-10T08:42:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:42:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:07 smithi098 ceph-mon[103124]: pgmap v18163: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 481 KiB/s wr, 31 op/s 2023-09-10T08:42:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:07 smithi170 ceph-mon[102832]: pgmap v18163: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 481 KiB/s wr, 31 op/s 2023-09-10T08:42:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:07 smithi082 ceph-mon[98694]: pgmap v18163: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 481 KiB/s wr, 31 op/s 2023-09-10T08:42:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33759.579257 secs 2023-09-10T08:42:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:09 smithi170 ceph-mon[102832]: osdmap e1204: 12 total, 12 up, 12 in 2023-09-10T08:42:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33760.780293 secs 2023-09-10T08:42:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33759.579257 secs 2023-09-10T08:42:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:09 smithi082 ceph-mon[98694]: osdmap e1204: 12 total, 12 up, 12 in 2023-09-10T08:42:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33760.780293 secs 2023-09-10T08:42:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33759.579257 secs 2023-09-10T08:42:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:09 smithi098 ceph-mon[103124]: osdmap e1204: 12 total, 12 up, 12 in 2023-09-10T08:42:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33760.780293 secs 2023-09-10T08:42:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:10 smithi170 ceph-mon[102832]: pgmap v18165: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 391 KiB/s wr, 11 op/s 2023-09-10T08:42:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33761.185011 secs 2023-09-10T08:42:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:10 smithi082 ceph-mon[98694]: pgmap v18165: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 391 KiB/s wr, 11 op/s 2023-09-10T08:42:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33761.185011 secs 2023-09-10T08:42:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:10 smithi098 ceph-mon[103124]: pgmap v18165: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 391 KiB/s wr, 11 op/s 2023-09-10T08:42:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33761.185011 secs 2023-09-10T08:42:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:12 smithi170 ceph-mon[102832]: pgmap v18166: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 428 KiB/s wr, 13 op/s 2023-09-10T08:42:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:12 smithi082 ceph-mon[98694]: pgmap v18166: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 428 KiB/s wr, 13 op/s 2023-09-10T08:42:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:12 smithi098 ceph-mon[103124]: pgmap v18166: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 428 KiB/s wr, 13 op/s 2023-09-10T08:42:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:14 smithi170 ceph-mon[102832]: pgmap v18167: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 498 KiB/s wr, 15 op/s 2023-09-10T08:42:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33764.579378 secs 2023-09-10T08:42:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33765.780354 secs 2023-09-10T08:42:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:14 smithi082 ceph-mon[98694]: pgmap v18167: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 498 KiB/s wr, 15 op/s 2023-09-10T08:42:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33764.579378 secs 2023-09-10T08:42:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33765.780354 secs 2023-09-10T08:42:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:14 smithi098 ceph-mon[103124]: pgmap v18167: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 498 KiB/s wr, 15 op/s 2023-09-10T08:42:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33764.579378 secs 2023-09-10T08:42:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33765.780354 secs 2023-09-10T08:42:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33766.185078 secs 2023-09-10T08:42:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33766.185078 secs 2023-09-10T08:42:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33766.185078 secs 2023-09-10T08:42:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:16 smithi170 ceph-mon[102832]: pgmap v18168: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 457 KiB/s wr, 14 op/s 2023-09-10T08:42:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:16 smithi082 ceph-mon[98694]: pgmap v18168: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 457 KiB/s wr, 14 op/s 2023-09-10T08:42:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:16 smithi098 ceph-mon[103124]: pgmap v18168: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 457 KiB/s wr, 14 op/s 2023-09-10T08:42:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:18 smithi170 ceph-mon[102832]: pgmap v18169: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 176 KiB/s wr, 13 op/s 2023-09-10T08:42:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:18 smithi082 ceph-mon[98694]: pgmap v18169: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 176 KiB/s wr, 13 op/s 2023-09-10T08:42:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:18 smithi098 ceph-mon[103124]: pgmap v18169: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 176 KiB/s wr, 13 op/s 2023-09-10T08:42:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33769.579480 secs 2023-09-10T08:42:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33770.780411 secs 2023-09-10T08:42:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33769.579480 secs 2023-09-10T08:42:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33770.780411 secs 2023-09-10T08:42:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33769.579480 secs 2023-09-10T08:42:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33770.780411 secs 2023-09-10T08:42:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:20 smithi170 ceph-mon[102832]: pgmap v18170: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 172 KiB/s wr, 12 op/s 2023-09-10T08:42:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33771.185142 secs 2023-09-10T08:42:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:20 smithi082 ceph-mon[98694]: pgmap v18170: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 172 KiB/s wr, 12 op/s 2023-09-10T08:42:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33771.185142 secs 2023-09-10T08:42:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:20 smithi098 ceph-mon[103124]: pgmap v18170: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 172 KiB/s wr, 12 op/s 2023-09-10T08:42:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33771.185142 secs 2023-09-10T08:42:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:22 smithi170 ceph-mon[102832]: pgmap v18171: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 13 op/s 2023-09-10T08:42:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:22 smithi082 ceph-mon[98694]: pgmap v18171: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 13 op/s 2023-09-10T08:42:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:22 smithi098 ceph-mon[103124]: pgmap v18171: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 13 op/s 2023-09-10T08:42:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:24 smithi170 ceph-mon[102832]: pgmap v18172: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T08:42:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33774.579542 secs 2023-09-10T08:42:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33775.780470 secs 2023-09-10T08:42:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33776.185207 secs 2023-09-10T08:42:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:24 smithi082 ceph-mon[98694]: pgmap v18172: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T08:42:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33774.579542 secs 2023-09-10T08:42:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33775.780470 secs 2023-09-10T08:42:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33776.185207 secs 2023-09-10T08:42:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:24 smithi098 ceph-mon[103124]: pgmap v18172: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 12 op/s 2023-09-10T08:42:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33774.579542 secs 2023-09-10T08:42:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33775.780470 secs 2023-09-10T08:42:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33776.185207 secs 2023-09-10T08:42:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:26 smithi170 ceph-mon[102832]: pgmap v18173: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 10 op/s 2023-09-10T08:42:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:26 smithi082 ceph-mon[98694]: pgmap v18173: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 10 op/s 2023-09-10T08:42:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:26 smithi098 ceph-mon[103124]: pgmap v18173: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 10 op/s 2023-09-10T08:42:28.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:28 smithi170 ceph-mon[102832]: pgmap v18174: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T08:42:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:28 smithi082 ceph-mon[98694]: pgmap v18174: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T08:42:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:28 smithi098 ceph-mon[103124]: pgmap v18174: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T08:42:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33779.579600 secs 2023-09-10T08:42:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33780.780522 secs 2023-09-10T08:42:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33781.185275 secs 2023-09-10T08:42:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33779.579600 secs 2023-09-10T08:42:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33780.780522 secs 2023-09-10T08:42:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33781.185275 secs 2023-09-10T08:42:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33779.579600 secs 2023-09-10T08:42:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33780.780522 secs 2023-09-10T08:42:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33781.185275 secs 2023-09-10T08:42:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:30 smithi170 ceph-mon[102832]: pgmap v18175: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T08:42:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:30 smithi082 ceph-mon[98694]: pgmap v18175: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T08:42:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:30 smithi098 ceph-mon[103124]: pgmap v18175: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T08:42:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:32 smithi082 ceph-mon[98694]: pgmap v18176: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 14 op/s 2023-09-10T08:42:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:32 smithi098 ceph-mon[103124]: pgmap v18176: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 14 op/s 2023-09-10T08:42:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:32 smithi170 ceph-mon[102832]: pgmap v18176: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 14 op/s 2023-09-10T08:42:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:34 smithi170 ceph-mon[102832]: pgmap v18177: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 13 op/s 2023-09-10T08:42:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:42:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33785.780592 secs 2023-09-10T08:42:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:34 smithi082 ceph-mon[98694]: pgmap v18177: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 13 op/s 2023-09-10T08:42:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:42:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33785.780592 secs 2023-09-10T08:42:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:34 smithi098 ceph-mon[103124]: pgmap v18177: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 121 KiB/s wr, 13 op/s 2023-09-10T08:42:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:42:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33785.780592 secs 2023-09-10T08:42:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33786.185346 secs 2023-09-10T08:42:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:42:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33786.185346 secs 2023-09-10T08:42:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:42:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33786.185346 secs 2023-09-10T08:42:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:42:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:36 smithi170 ceph-mon[102832]: pgmap v18178: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T08:42:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:42:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:42:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:36 smithi082 ceph-mon[98694]: pgmap v18178: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T08:42:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:42:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:42:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:36 smithi098 ceph-mon[103124]: pgmap v18178: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T08:42:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:42:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:42:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:42:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:42:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:42:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:38 smithi170 ceph-mon[102832]: pgmap v18179: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 344 KiB/s wr, 17 op/s 2023-09-10T08:42:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:42:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:42:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:42:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:38 smithi082 ceph-mon[98694]: pgmap v18179: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 344 KiB/s wr, 17 op/s 2023-09-10T08:42:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:42:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:42:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:42:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:38 smithi098 ceph-mon[103124]: pgmap v18179: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 344 KiB/s wr, 17 op/s 2023-09-10T08:42:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:42:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:42:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:42:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:39 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33789.579765 secs 2023-09-10T08:42:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33790.780652 secs 2023-09-10T08:42:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:39 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33789.579765 secs 2023-09-10T08:42:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33790.780652 secs 2023-09-10T08:42:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:39 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33789.579765 secs 2023-09-10T08:42:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33790.780652 secs 2023-09-10T08:42:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:40 smithi170 ceph-mon[102832]: pgmap v18180: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T08:42:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33791.185413 secs 2023-09-10T08:42:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:42:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:40 smithi082 ceph-mon[98694]: pgmap v18180: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T08:42:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33791.185413 secs 2023-09-10T08:42:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:42:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:40 smithi098 ceph-mon[103124]: pgmap v18180: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T08:42:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33791.185413 secs 2023-09-10T08:42:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:42:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:42 smithi170 ceph-mon[102832]: pgmap v18181: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 248 KiB/s wr, 15 op/s 2023-09-10T08:42:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:42 smithi082 ceph-mon[98694]: pgmap v18181: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 248 KiB/s wr, 15 op/s 2023-09-10T08:42:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:42 smithi098 ceph-mon[103124]: pgmap v18181: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 248 KiB/s wr, 15 op/s 2023-09-10T08:42:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:42:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:42:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:42:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:44 smithi170 ceph-mon[102832]: pgmap v18182: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 403 KiB/s wr, 14 op/s 2023-09-10T08:42:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:44 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33794.579828 secs 2023-09-10T08:42:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:42:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33795.780711 secs 2023-09-10T08:42:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33796.185486 secs 2023-09-10T08:42:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:44 smithi082 ceph-mon[98694]: pgmap v18182: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 403 KiB/s wr, 14 op/s 2023-09-10T08:42:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:44 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33794.579828 secs 2023-09-10T08:42:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:42:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33795.780711 secs 2023-09-10T08:42:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33796.185486 secs 2023-09-10T08:42:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:44 smithi098 ceph-mon[103124]: pgmap v18182: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 403 KiB/s wr, 14 op/s 2023-09-10T08:42:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:44 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33794.579828 secs 2023-09-10T08:42:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:42:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33795.780711 secs 2023-09-10T08:42:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33796.185486 secs 2023-09-10T08:42:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:46 smithi082 ceph-mon[98694]: pgmap v18183: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 403 KiB/s wr, 12 op/s 2023-09-10T08:42:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:42:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:46 smithi098 ceph-mon[103124]: pgmap v18183: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 403 KiB/s wr, 12 op/s 2023-09-10T08:42:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:42:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:46 smithi170 ceph-mon[102832]: pgmap v18183: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 403 KiB/s wr, 12 op/s 2023-09-10T08:42:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:42:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:47 smithi098 ceph-mon[103124]: pgmap v18184: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 470 KiB/s wr, 19 op/s 2023-09-10T08:42:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:47 smithi170 ceph-mon[102832]: pgmap v18184: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 470 KiB/s wr, 19 op/s 2023-09-10T08:42:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:47 smithi082 ceph-mon[98694]: pgmap v18184: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 470 KiB/s wr, 19 op/s 2023-09-10T08:42:49.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33800.780773 secs 2023-09-10T08:42:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33800.780773 secs 2023-09-10T08:42:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33800.780773 secs 2023-09-10T08:42:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:50 smithi170 ceph-mon[102832]: pgmap v18185: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 248 KiB/s wr, 13 op/s 2023-09-10T08:42:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33801.185550 secs 2023-09-10T08:42:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:42:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:50 smithi082 ceph-mon[98694]: pgmap v18185: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 248 KiB/s wr, 13 op/s 2023-09-10T08:42:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33801.185550 secs 2023-09-10T08:42:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:42:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:50 smithi098 ceph-mon[103124]: pgmap v18185: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 248 KiB/s wr, 13 op/s 2023-09-10T08:42:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33801.185550 secs 2023-09-10T08:42:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:42:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:51 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:42:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:51 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:42:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:51 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:42:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:52 smithi170 ceph-mon[102832]: pgmap v18186: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 273 KiB/s wr, 17 op/s 2023-09-10T08:42:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:52 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:42:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:42:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:52 smithi082 ceph-mon[98694]: pgmap v18186: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 273 KiB/s wr, 17 op/s 2023-09-10T08:42:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:52 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:42:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:42:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:52 smithi098 ceph-mon[103124]: pgmap v18186: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 273 KiB/s wr, 17 op/s 2023-09-10T08:42:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:52 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:42:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:42:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:54 smithi170 ceph-mon[102832]: pgmap v18187: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 649 KiB/s wr, 17 op/s 2023-09-10T08:42:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33805.780832 secs 2023-09-10T08:42:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:54 smithi082 ceph-mon[98694]: pgmap v18187: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 649 KiB/s wr, 17 op/s 2023-09-10T08:42:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33805.780832 secs 2023-09-10T08:42:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:54 smithi098 ceph-mon[103124]: pgmap v18187: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 649 KiB/s wr, 17 op/s 2023-09-10T08:42:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33805.780832 secs 2023-09-10T08:42:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33806.185616 secs 2023-09-10T08:42:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33806.185616 secs 2023-09-10T08:42:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33806.185616 secs 2023-09-10T08:42:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:56 smithi170 ceph-mon[102832]: pgmap v18188: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 494 KiB/s wr, 14 op/s 2023-09-10T08:42:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:42:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:56 smithi082 ceph-mon[98694]: pgmap v18188: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 494 KiB/s wr, 14 op/s 2023-09-10T08:42:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:42:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:56 smithi098 ceph-mon[103124]: pgmap v18188: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 494 KiB/s wr, 14 op/s 2023-09-10T08:42:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:42:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:58 smithi170 ceph-mon[102832]: pgmap v18189: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 735 KiB/s wr, 21 op/s 2023-09-10T08:42:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:42:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:58 smithi082 ceph-mon[98694]: pgmap v18189: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 735 KiB/s wr, 21 op/s 2023-09-10T08:42:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:42:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:58 smithi098 ceph-mon[103124]: pgmap v18189: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 735 KiB/s wr, 21 op/s 2023-09-10T08:42:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:42:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:42:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33809.580008 secs 2023-09-10T08:42:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33810.780914 secs 2023-09-10T08:42:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:42:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33811.185679 secs 2023-09-10T08:42:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33809.580008 secs 2023-09-10T08:42:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33810.780914 secs 2023-09-10T08:42:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:42:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33811.185679 secs 2023-09-10T08:42:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33809.580008 secs 2023-09-10T08:42:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33810.780914 secs 2023-09-10T08:42:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:42:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33811.185679 secs 2023-09-10T08:43:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:00 smithi170 ceph-mon[102832]: pgmap v18190: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 668 KiB/s wr, 14 op/s 2023-09-10T08:43:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:43:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:43:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:00 smithi082 ceph-mon[98694]: pgmap v18190: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 668 KiB/s wr, 14 op/s 2023-09-10T08:43:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:43:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:43:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:00 smithi098 ceph-mon[103124]: pgmap v18190: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 668 KiB/s wr, 14 op/s 2023-09-10T08:43:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:43:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:43:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:02 smithi082 ceph-mon[98694]: pgmap v18191: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 715 KiB/s wr, 23 op/s 2023-09-10T08:43:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:02 smithi098 ceph-mon[103124]: pgmap v18191: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 715 KiB/s wr, 23 op/s 2023-09-10T08:43:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:02 smithi170 ceph-mon[102832]: pgmap v18191: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 715 KiB/s wr, 23 op/s 2023-09-10T08:43:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:04 smithi082 ceph-mon[98694]: pgmap v18192: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 915 KiB/s wr, 23 op/s 2023-09-10T08:43:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33814.580092 secs 2023-09-10T08:43:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:43:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:04 smithi082 ceph-mon[98694]: osdmap e1205: 12 total, 12 up, 12 in 2023-09-10T08:43:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33815.780972 secs 2023-09-10T08:43:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33816.185742 secs 2023-09-10T08:43:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:04 smithi098 ceph-mon[103124]: pgmap v18192: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 915 KiB/s wr, 23 op/s 2023-09-10T08:43:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33814.580092 secs 2023-09-10T08:43:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:43:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:04 smithi098 ceph-mon[103124]: osdmap e1205: 12 total, 12 up, 12 in 2023-09-10T08:43:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33815.780972 secs 2023-09-10T08:43:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33816.185742 secs 2023-09-10T08:43:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:04 smithi170 ceph-mon[102832]: pgmap v18192: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 915 KiB/s wr, 23 op/s 2023-09-10T08:43:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33814.580092 secs 2023-09-10T08:43:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:43:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:04 smithi170 ceph-mon[102832]: osdmap e1205: 12 total, 12 up, 12 in 2023-09-10T08:43:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33815.780972 secs 2023-09-10T08:43:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33816.185742 secs 2023-09-10T08:43:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:06 smithi082 ceph-mon[98694]: pgmap v18194: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 617 KiB/s wr, 23 op/s 2023-09-10T08:43:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:43:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:06 smithi098 ceph-mon[103124]: pgmap v18194: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 617 KiB/s wr, 23 op/s 2023-09-10T08:43:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:43:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:06 smithi170 ceph-mon[102832]: pgmap v18194: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 617 KiB/s wr, 23 op/s 2023-09-10T08:43:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:43:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:07 smithi082 ceph-mon[98694]: pgmap v18195: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 659 KiB/s wr, 35 op/s 2023-09-10T08:43:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:07 smithi098 ceph-mon[103124]: pgmap v18195: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 659 KiB/s wr, 35 op/s 2023-09-10T08:43:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:07 smithi170 ceph-mon[102832]: pgmap v18195: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 659 KiB/s wr, 35 op/s 2023-09-10T08:43:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33819.580191 secs 2023-09-10T08:43:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:09 smithi170 ceph-mon[102832]: osdmap e1206: 12 total, 12 up, 12 in 2023-09-10T08:43:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33820.781031 secs 2023-09-10T08:43:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33819.580191 secs 2023-09-10T08:43:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:09 smithi082 ceph-mon[98694]: osdmap e1206: 12 total, 12 up, 12 in 2023-09-10T08:43:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33820.781031 secs 2023-09-10T08:43:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33819.580191 secs 2023-09-10T08:43:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:09 smithi098 ceph-mon[103124]: osdmap e1206: 12 total, 12 up, 12 in 2023-09-10T08:43:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33820.781031 secs 2023-09-10T08:43:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:10 smithi170 ceph-mon[102832]: pgmap v18197: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 753 KiB/s wr, 32 op/s 2023-09-10T08:43:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33821.185810 secs 2023-09-10T08:43:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:10 smithi082 ceph-mon[98694]: pgmap v18197: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 753 KiB/s wr, 32 op/s 2023-09-10T08:43:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33821.185810 secs 2023-09-10T08:43:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:10 smithi098 ceph-mon[103124]: pgmap v18197: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 753 KiB/s wr, 32 op/s 2023-09-10T08:43:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33821.185810 secs 2023-09-10T08:43:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:12 smithi170 ceph-mon[102832]: pgmap v18198: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 453 KiB/s wr, 33 op/s 2023-09-10T08:43:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:12 smithi082 ceph-mon[98694]: pgmap v18198: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 453 KiB/s wr, 33 op/s 2023-09-10T08:43:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:12 smithi098 ceph-mon[103124]: pgmap v18198: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 453 KiB/s wr, 33 op/s 2023-09-10T08:43:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:14 smithi170 ceph-mon[102832]: pgmap v18199: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 395 KiB/s wr, 31 op/s 2023-09-10T08:43:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33824.580251 secs 2023-09-10T08:43:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33825.781105 secs 2023-09-10T08:43:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:14 smithi082 ceph-mon[98694]: pgmap v18199: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 395 KiB/s wr, 31 op/s 2023-09-10T08:43:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33824.580251 secs 2023-09-10T08:43:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33825.781105 secs 2023-09-10T08:43:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:14 smithi098 ceph-mon[103124]: pgmap v18199: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 395 KiB/s wr, 31 op/s 2023-09-10T08:43:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33824.580251 secs 2023-09-10T08:43:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33825.781105 secs 2023-09-10T08:43:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33826.185872 secs 2023-09-10T08:43:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33826.185872 secs 2023-09-10T08:43:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33826.185872 secs 2023-09-10T08:43:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:16 smithi170 ceph-mon[102832]: pgmap v18200: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 362 KiB/s wr, 28 op/s 2023-09-10T08:43:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:16 smithi082 ceph-mon[98694]: pgmap v18200: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 362 KiB/s wr, 28 op/s 2023-09-10T08:43:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:16 smithi098 ceph-mon[103124]: pgmap v18200: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 362 KiB/s wr, 28 op/s 2023-09-10T08:43:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:18 smithi170 ceph-mon[102832]: pgmap v18201: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T08:43:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:18 smithi082 ceph-mon[98694]: pgmap v18201: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T08:43:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:18 smithi098 ceph-mon[103124]: pgmap v18201: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T08:43:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33829.580313 secs 2023-09-10T08:43:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33830.781163 secs 2023-09-10T08:43:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33831.185936 secs 2023-09-10T08:43:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33829.580313 secs 2023-09-10T08:43:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33830.781163 secs 2023-09-10T08:43:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33831.185936 secs 2023-09-10T08:43:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33829.580313 secs 2023-09-10T08:43:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33830.781163 secs 2023-09-10T08:43:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33831.185936 secs 2023-09-10T08:43:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:20 smithi170 ceph-mon[102832]: pgmap v18202: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 142 KiB/s wr, 13 op/s 2023-09-10T08:43:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:20 smithi082 ceph-mon[98694]: pgmap v18202: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 142 KiB/s wr, 13 op/s 2023-09-10T08:43:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:20 smithi098 ceph-mon[103124]: pgmap v18202: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 142 KiB/s wr, 13 op/s 2023-09-10T08:43:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:22 smithi170 ceph-mon[102832]: pgmap v18203: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 121 KiB/s wr, 16 op/s 2023-09-10T08:43:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:22 smithi082 ceph-mon[98694]: pgmap v18203: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 121 KiB/s wr, 16 op/s 2023-09-10T08:43:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:22 smithi098 ceph-mon[103124]: pgmap v18203: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 121 KiB/s wr, 16 op/s 2023-09-10T08:43:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:24 smithi170 ceph-mon[102832]: pgmap v18204: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T08:43:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33834.580371 secs 2023-09-10T08:43:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33835.781221 secs 2023-09-10T08:43:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33836.186000 secs 2023-09-10T08:43:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:24 smithi082 ceph-mon[98694]: pgmap v18204: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T08:43:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33834.580371 secs 2023-09-10T08:43:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33835.781221 secs 2023-09-10T08:43:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33836.186000 secs 2023-09-10T08:43:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:24 smithi098 ceph-mon[103124]: pgmap v18204: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 11 op/s 2023-09-10T08:43:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33834.580371 secs 2023-09-10T08:43:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33835.781221 secs 2023-09-10T08:43:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33836.186000 secs 2023-09-10T08:43:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:26 smithi170 ceph-mon[102832]: pgmap v18205: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T08:43:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:26 smithi082 ceph-mon[98694]: pgmap v18205: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T08:43:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:26 smithi098 ceph-mon[103124]: pgmap v18205: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 96 KiB/s wr, 10 op/s 2023-09-10T08:43:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:28 smithi082 ceph-mon[98694]: pgmap v18206: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 97 KiB/s wr, 15 op/s 2023-09-10T08:43:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:28 smithi098 ceph-mon[103124]: pgmap v18206: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 97 KiB/s wr, 15 op/s 2023-09-10T08:43:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:28 smithi170 ceph-mon[102832]: pgmap v18206: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 97 KiB/s wr, 15 op/s 2023-09-10T08:43:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33839.580429 secs 2023-09-10T08:43:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33840.781280 secs 2023-09-10T08:43:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33841.186063 secs 2023-09-10T08:43:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33839.580429 secs 2023-09-10T08:43:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33840.781280 secs 2023-09-10T08:43:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33841.186063 secs 2023-09-10T08:43:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33839.580429 secs 2023-09-10T08:43:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33840.781280 secs 2023-09-10T08:43:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33841.186063 secs 2023-09-10T08:43:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:30 smithi082 ceph-mon[98694]: pgmap v18207: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-10T08:43:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:43:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:30 smithi098 ceph-mon[103124]: pgmap v18207: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-10T08:43:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:43:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:30 smithi170 ceph-mon[102832]: pgmap v18207: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-10T08:43:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:43:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:31 smithi098 ceph-mon[103124]: pgmap v18208: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 122 KiB/s wr, 15 op/s 2023-09-10T08:43:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:31 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:43:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:31 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:31 smithi170 ceph-mon[102832]: pgmap v18208: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 122 KiB/s wr, 15 op/s 2023-09-10T08:43:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:31 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:43:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:31 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:31 smithi082 ceph-mon[98694]: pgmap v18208: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 122 KiB/s wr, 15 op/s 2023-09-10T08:43:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:31 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:43:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:31 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:34 smithi170 ceph-mon[102832]: pgmap v18209: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T08:43:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33844.580557 secs 2023-09-10T08:43:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33845.781364 secs 2023-09-10T08:43:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:34 smithi082 ceph-mon[98694]: pgmap v18209: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T08:43:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33844.580557 secs 2023-09-10T08:43:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33845.781364 secs 2023-09-10T08:43:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:34 smithi098 ceph-mon[103124]: pgmap v18209: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T08:43:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33844.580557 secs 2023-09-10T08:43:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33845.781364 secs 2023-09-10T08:43:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33846.186131 secs 2023-09-10T08:43:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33846.186131 secs 2023-09-10T08:43:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33846.186131 secs 2023-09-10T08:43:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:36 smithi170 ceph-mon[102832]: pgmap v18210: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 10 op/s 2023-09-10T08:43:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:36 smithi082 ceph-mon[98694]: pgmap v18210: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 10 op/s 2023-09-10T08:43:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:36 smithi098 ceph-mon[103124]: pgmap v18210: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 10 op/s 2023-09-10T08:43:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:38 smithi170 ceph-mon[102832]: pgmap v18211: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 322 KiB/s wr, 16 op/s 2023-09-10T08:43:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:43:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:38 smithi082 ceph-mon[98694]: pgmap v18211: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 322 KiB/s wr, 16 op/s 2023-09-10T08:43:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:43:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:38 smithi098 ceph-mon[103124]: pgmap v18211: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 322 KiB/s wr, 16 op/s 2023-09-10T08:43:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:43:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:39 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33849.580618 secs 2023-09-10T08:43:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:43:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:43:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:43:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33850.781425 secs 2023-09-10T08:43:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:39 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33849.580618 secs 2023-09-10T08:43:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:43:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:43:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:43:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33850.781425 secs 2023-09-10T08:43:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:39 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33849.580618 secs 2023-09-10T08:43:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:43:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:43:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:43:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33850.781425 secs 2023-09-10T08:43:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:40 smithi170 ceph-mon[102832]: pgmap v18212: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 322 KiB/s wr, 10 op/s 2023-09-10T08:43:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33851.186198 secs 2023-09-10T08:43:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:40 smithi082 ceph-mon[98694]: pgmap v18212: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 322 KiB/s wr, 10 op/s 2023-09-10T08:43:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33851.186198 secs 2023-09-10T08:43:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:40 smithi098 ceph-mon[103124]: pgmap v18212: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 322 KiB/s wr, 10 op/s 2023-09-10T08:43:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33851.186198 secs 2023-09-10T08:43:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:42 smithi170 ceph-mon[102832]: pgmap v18213: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 498 KiB/s wr, 17 op/s 2023-09-10T08:43:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:42 smithi082 ceph-mon[98694]: pgmap v18213: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 498 KiB/s wr, 17 op/s 2023-09-10T08:43:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:42 smithi098 ceph-mon[103124]: pgmap v18213: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 498 KiB/s wr, 17 op/s 2023-09-10T08:43:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:43:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:43:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:43:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:44 smithi170 ceph-mon[102832]: pgmap v18214: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 377 KiB/s wr, 11 op/s 2023-09-10T08:43:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:44 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:43:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:43:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33855.781491 secs 2023-09-10T08:43:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33856.186263 secs 2023-09-10T08:43:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:44 smithi082 ceph-mon[98694]: pgmap v18214: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 377 KiB/s wr, 11 op/s 2023-09-10T08:43:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:44 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:43:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:43:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33855.781491 secs 2023-09-10T08:43:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33856.186263 secs 2023-09-10T08:43:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:44 smithi098 ceph-mon[103124]: pgmap v18214: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 377 KiB/s wr, 11 op/s 2023-09-10T08:43:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:44 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:43:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:43:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33855.781491 secs 2023-09-10T08:43:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33856.186263 secs 2023-09-10T08:43:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:46 smithi170 ceph-mon[102832]: pgmap v18215: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 377 KiB/s wr, 11 op/s 2023-09-10T08:43:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:43:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:46 smithi082 ceph-mon[98694]: pgmap v18215: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 377 KiB/s wr, 11 op/s 2023-09-10T08:43:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:43:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:46 smithi098 ceph-mon[103124]: pgmap v18215: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 377 KiB/s wr, 11 op/s 2023-09-10T08:43:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:43:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:48 smithi170 ceph-mon[102832]: pgmap v18216: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 593 KiB/s wr, 18 op/s 2023-09-10T08:43:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:48 smithi082 ceph-mon[98694]: pgmap v18216: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 593 KiB/s wr, 18 op/s 2023-09-10T08:43:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:48 smithi098 ceph-mon[103124]: pgmap v18216: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 593 KiB/s wr, 18 op/s 2023-09-10T08:43:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33860.781549 secs 2023-09-10T08:43:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33861.186330 secs 2023-09-10T08:43:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33860.781549 secs 2023-09-10T08:43:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33861.186330 secs 2023-09-10T08:43:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33860.781549 secs 2023-09-10T08:43:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33861.186330 secs 2023-09-10T08:43:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:50 smithi082 ceph-mon[98694]: pgmap v18217: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 392 KiB/s wr, 12 op/s 2023-09-10T08:43:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:43:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:50 smithi098 ceph-mon[103124]: pgmap v18217: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 392 KiB/s wr, 12 op/s 2023-09-10T08:43:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:43:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:50 smithi170 ceph-mon[102832]: pgmap v18217: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 392 KiB/s wr, 12 op/s 2023-09-10T08:43:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:43:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:52 smithi082 ceph-mon[98694]: pgmap v18218: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 759 KiB/s wr, 19 op/s 2023-09-10T08:43:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:43:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:52 smithi098 ceph-mon[103124]: pgmap v18218: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 759 KiB/s wr, 19 op/s 2023-09-10T08:43:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:43:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:52 smithi170 ceph-mon[102832]: pgmap v18218: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 759 KiB/s wr, 19 op/s 2023-09-10T08:43:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:43:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:54 smithi082 ceph-mon[98694]: pgmap v18219: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 583 KiB/s wr, 13 op/s 2023-09-10T08:43:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33865.781606 secs 2023-09-10T08:43:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33866.186394 secs 2023-09-10T08:43:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:54 smithi098 ceph-mon[103124]: pgmap v18219: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 583 KiB/s wr, 13 op/s 2023-09-10T08:43:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33865.781606 secs 2023-09-10T08:43:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33866.186394 secs 2023-09-10T08:43:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:54 smithi170 ceph-mon[102832]: pgmap v18219: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 583 KiB/s wr, 13 op/s 2023-09-10T08:43:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33865.781606 secs 2023-09-10T08:43:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33866.186394 secs 2023-09-10T08:43:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:55 smithi082 ceph-mon[98694]: pgmap v18220: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 583 KiB/s wr, 13 op/s 2023-09-10T08:43:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:55 smithi098 ceph-mon[103124]: pgmap v18220: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 583 KiB/s wr, 13 op/s 2023-09-10T08:43:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:55 smithi170 ceph-mon[102832]: pgmap v18220: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 583 KiB/s wr, 13 op/s 2023-09-10T08:43:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:57 smithi082 ceph-mon[98694]: pgmap v18221: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 617 KiB/s wr, 20 op/s 2023-09-10T08:43:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:57 smithi098 ceph-mon[103124]: pgmap v18221: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 617 KiB/s wr, 20 op/s 2023-09-10T08:43:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:57 smithi170 ceph-mon[102832]: pgmap v18221: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 617 KiB/s wr, 20 op/s 2023-09-10T08:43:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:43:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:43:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:43:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:43:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33869.580907 secs 2023-09-10T08:43:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:59 smithi082 ceph-mon[98694]: pgmap v18222: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 401 KiB/s wr, 13 op/s 2023-09-10T08:43:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33870.781685 secs 2023-09-10T08:43:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:43:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33871.186464 secs 2023-09-10T08:43:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33869.580907 secs 2023-09-10T08:43:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:59 smithi098 ceph-mon[103124]: pgmap v18222: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 401 KiB/s wr, 13 op/s 2023-09-10T08:43:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33870.781685 secs 2023-09-10T08:43:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:43:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33871.186464 secs 2023-09-10T08:43:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33869.580907 secs 2023-09-10T08:43:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:59 smithi170 ceph-mon[102832]: pgmap v18222: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 401 KiB/s wr, 13 op/s 2023-09-10T08:43:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33870.781685 secs 2023-09-10T08:43:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:43:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33871.186464 secs 2023-09-10T08:44:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:44:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:44:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:44:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:44:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:44:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:44:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:01 smithi098 ceph-mon[103124]: pgmap v18223: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 760 KiB/s wr, 20 op/s 2023-09-10T08:44:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:01 smithi170 ceph-mon[102832]: pgmap v18223: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 760 KiB/s wr, 20 op/s 2023-09-10T08:44:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:01 smithi082 ceph-mon[98694]: pgmap v18223: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 760 KiB/s wr, 20 op/s 2023-09-10T08:44:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:04 smithi170 ceph-mon[102832]: pgmap v18224: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T08:44:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33874.580967 secs 2023-09-10T08:44:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:44:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:04 smithi170 ceph-mon[102832]: osdmap e1207: 12 total, 12 up, 12 in 2023-09-10T08:44:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33875.781752 secs 2023-09-10T08:44:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:04 smithi082 ceph-mon[98694]: pgmap v18224: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T08:44:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33874.580967 secs 2023-09-10T08:44:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:44:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:04 smithi082 ceph-mon[98694]: osdmap e1207: 12 total, 12 up, 12 in 2023-09-10T08:44:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33875.781752 secs 2023-09-10T08:44:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:04 smithi098 ceph-mon[103124]: pgmap v18224: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T08:44:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33874.580967 secs 2023-09-10T08:44:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:44:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:04 smithi098 ceph-mon[103124]: osdmap e1207: 12 total, 12 up, 12 in 2023-09-10T08:44:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33875.781752 secs 2023-09-10T08:44:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33876.186489 secs 2023-09-10T08:44:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33876.186489 secs 2023-09-10T08:44:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33876.186489 secs 2023-09-10T08:44:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:06 smithi170 ceph-mon[102832]: pgmap v18226: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 471 KiB/s wr, 15 op/s 2023-09-10T08:44:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:44:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:06 smithi082 ceph-mon[98694]: pgmap v18226: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 471 KiB/s wr, 15 op/s 2023-09-10T08:44:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:44:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:06 smithi098 ceph-mon[103124]: pgmap v18226: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 471 KiB/s wr, 15 op/s 2023-09-10T08:44:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:44:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:08 smithi170 ceph-mon[102832]: pgmap v18227: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 842 KiB/s wr, 32 op/s 2023-09-10T08:44:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:08 smithi170 ceph-mon[102832]: osdmap e1208: 12 total, 12 up, 12 in 2023-09-10T08:44:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:08 smithi082 ceph-mon[98694]: pgmap v18227: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 842 KiB/s wr, 32 op/s 2023-09-10T08:44:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:08 smithi082 ceph-mon[98694]: osdmap e1208: 12 total, 12 up, 12 in 2023-09-10T08:44:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:08 smithi098 ceph-mon[103124]: pgmap v18227: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 842 KiB/s wr, 32 op/s 2023-09-10T08:44:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:08 smithi098 ceph-mon[103124]: osdmap e1208: 12 total, 12 up, 12 in 2023-09-10T08:44:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33879.581086 secs 2023-09-10T08:44:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33880.781810 secs 2023-09-10T08:44:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33881.186554 secs 2023-09-10T08:44:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33879.581086 secs 2023-09-10T08:44:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33880.781810 secs 2023-09-10T08:44:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33881.186554 secs 2023-09-10T08:44:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33879.581086 secs 2023-09-10T08:44:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33880.781810 secs 2023-09-10T08:44:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33881.186554 secs 2023-09-10T08:44:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:10 smithi170 ceph-mon[102832]: pgmap v18229: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 515 KiB/s wr, 30 op/s 2023-09-10T08:44:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:10 smithi082 ceph-mon[98694]: pgmap v18229: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 515 KiB/s wr, 30 op/s 2023-09-10T08:44:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:10 smithi098 ceph-mon[103124]: pgmap v18229: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 515 KiB/s wr, 30 op/s 2023-09-10T08:44:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:12 smithi170 ceph-mon[102832]: pgmap v18230: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 696 KiB/s wr, 40 op/s 2023-09-10T08:44:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:12 smithi082 ceph-mon[98694]: pgmap v18230: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 696 KiB/s wr, 40 op/s 2023-09-10T08:44:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:12 smithi098 ceph-mon[103124]: pgmap v18230: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 696 KiB/s wr, 40 op/s 2023-09-10T08:44:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:14 smithi082 ceph-mon[98694]: pgmap v18231: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 626 KiB/s wr, 36 op/s 2023-09-10T08:44:14.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33884.581145 secs 2023-09-10T08:44:14.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:14 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:44:14.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:14 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:14.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33885.781886 secs 2023-09-10T08:44:14.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33886.186620 secs 2023-09-10T08:44:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:14 smithi098 ceph-mon[103124]: pgmap v18231: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 626 KiB/s wr, 36 op/s 2023-09-10T08:44:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33884.581145 secs 2023-09-10T08:44:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:14 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:44:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:14 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33885.781886 secs 2023-09-10T08:44:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33886.186620 secs 2023-09-10T08:44:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:14 smithi170 ceph-mon[102832]: pgmap v18231: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 626 KiB/s wr, 36 op/s 2023-09-10T08:44:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33884.581145 secs 2023-09-10T08:44:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:14 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:44:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:14 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33885.781886 secs 2023-09-10T08:44:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33886.186620 secs 2023-09-10T08:44:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:16 smithi082 ceph-mon[98694]: pgmap v18232: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 557 KiB/s wr, 32 op/s 2023-09-10T08:44:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:44:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:16 smithi098 ceph-mon[103124]: pgmap v18232: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 557 KiB/s wr, 32 op/s 2023-09-10T08:44:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:44:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:16 smithi170 ceph-mon[102832]: pgmap v18232: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 557 KiB/s wr, 32 op/s 2023-09-10T08:44:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:44:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:18 smithi082 ceph-mon[98694]: pgmap v18233: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 14 op/s 2023-09-10T08:44:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:18 smithi098 ceph-mon[103124]: pgmap v18233: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 14 op/s 2023-09-10T08:44:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:18 smithi170 ceph-mon[102832]: pgmap v18233: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 14 op/s 2023-09-10T08:44:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33889.581246 secs 2023-09-10T08:44:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33890.781949 secs 2023-09-10T08:44:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33891.186685 secs 2023-09-10T08:44:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33889.581246 secs 2023-09-10T08:44:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33890.781949 secs 2023-09-10T08:44:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33891.186685 secs 2023-09-10T08:44:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33889.581246 secs 2023-09-10T08:44:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33890.781949 secs 2023-09-10T08:44:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33891.186685 secs 2023-09-10T08:44:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:20 smithi082 ceph-mon[98694]: pgmap v18234: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 14 op/s 2023-09-10T08:44:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:20 smithi098 ceph-mon[103124]: pgmap v18234: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 14 op/s 2023-09-10T08:44:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:20 smithi170 ceph-mon[102832]: pgmap v18234: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 14 op/s 2023-09-10T08:44:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:21 smithi082 ceph-mon[98694]: pgmap v18235: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 243 KiB/s wr, 18 op/s 2023-09-10T08:44:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:21 smithi098 ceph-mon[103124]: pgmap v18235: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 243 KiB/s wr, 18 op/s 2023-09-10T08:44:21.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:21 smithi170 ceph-mon[102832]: pgmap v18235: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 243 KiB/s wr, 18 op/s 2023-09-10T08:44:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:24 smithi170 ceph-mon[102832]: pgmap v18236: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T08:44:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33894.581310 secs 2023-09-10T08:44:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33895.782049 secs 2023-09-10T08:44:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:24 smithi082 ceph-mon[98694]: pgmap v18236: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T08:44:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33894.581310 secs 2023-09-10T08:44:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33895.782049 secs 2023-09-10T08:44:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:24 smithi098 ceph-mon[103124]: pgmap v18236: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T08:44:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33894.581310 secs 2023-09-10T08:44:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33895.782049 secs 2023-09-10T08:44:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33896.186748 secs 2023-09-10T08:44:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33896.186748 secs 2023-09-10T08:44:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33896.186748 secs 2023-09-10T08:44:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:26 smithi170 ceph-mon[102832]: pgmap v18237: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T08:44:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:26 smithi082 ceph-mon[98694]: pgmap v18237: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T08:44:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:26 smithi098 ceph-mon[103124]: pgmap v18237: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T08:44:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:28 smithi170 ceph-mon[102832]: pgmap v18238: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 243 KiB/s wr, 17 op/s 2023-09-10T08:44:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:28 smithi082 ceph-mon[98694]: pgmap v18238: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 243 KiB/s wr, 17 op/s 2023-09-10T08:44:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:28 smithi098 ceph-mon[103124]: pgmap v18238: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 243 KiB/s wr, 17 op/s 2023-09-10T08:44:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33899.581372 secs 2023-09-10T08:44:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33900.782105 secs 2023-09-10T08:44:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33901.186810 secs 2023-09-10T08:44:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33899.581372 secs 2023-09-10T08:44:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33900.782105 secs 2023-09-10T08:44:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33901.186810 secs 2023-09-10T08:44:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33899.581372 secs 2023-09-10T08:44:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33900.782105 secs 2023-09-10T08:44:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33901.186810 secs 2023-09-10T08:44:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:30 smithi082 ceph-mon[98694]: pgmap v18239: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 242 KiB/s wr, 11 op/s 2023-09-10T08:44:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:30 smithi098 ceph-mon[103124]: pgmap v18239: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 242 KiB/s wr, 11 op/s 2023-09-10T08:44:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:30 smithi170 ceph-mon[102832]: pgmap v18239: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 242 KiB/s wr, 11 op/s 2023-09-10T08:44:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:32 smithi082 ceph-mon[98694]: pgmap v18240: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 242 KiB/s wr, 16 op/s 2023-09-10T08:44:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:32 smithi098 ceph-mon[103124]: pgmap v18240: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 242 KiB/s wr, 16 op/s 2023-09-10T08:44:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:32 smithi170 ceph-mon[102832]: pgmap v18240: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 242 KiB/s wr, 16 op/s 2023-09-10T08:44:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:34 smithi082 ceph-mon[98694]: pgmap v18241: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 10 op/s 2023-09-10T08:44:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33905.782166 secs 2023-09-10T08:44:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33906.186874 secs 2023-09-10T08:44:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:34 smithi098 ceph-mon[103124]: pgmap v18241: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 10 op/s 2023-09-10T08:44:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33905.782166 secs 2023-09-10T08:44:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33906.186874 secs 2023-09-10T08:44:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:34 smithi170 ceph-mon[102832]: pgmap v18241: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 10 op/s 2023-09-10T08:44:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33905.782166 secs 2023-09-10T08:44:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33906.186874 secs 2023-09-10T08:44:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:44:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:44:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:44:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:36 smithi082 ceph-mon[98694]: pgmap v18242: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 10 op/s 2023-09-10T08:44:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:44:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:44:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:36 smithi098 ceph-mon[103124]: pgmap v18242: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 10 op/s 2023-09-10T08:44:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:44:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:44:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:36 smithi170 ceph-mon[102832]: pgmap v18242: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 10 op/s 2023-09-10T08:44:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:44:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:44:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:38 smithi082 ceph-mon[98694]: pgmap v18243: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 241 KiB/s wr, 16 op/s 2023-09-10T08:44:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:44:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:38 smithi098 ceph-mon[103124]: pgmap v18243: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 241 KiB/s wr, 16 op/s 2023-09-10T08:44:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:44:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:38 smithi170 ceph-mon[102832]: pgmap v18243: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 241 KiB/s wr, 16 op/s 2023-09-10T08:44:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:44:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:39 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33909.581555 secs 2023-09-10T08:44:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:44:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33910.782229 secs 2023-09-10T08:44:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33911.186943 secs 2023-09-10T08:44:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:44:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:44:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:44:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:39 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33909.581555 secs 2023-09-10T08:44:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:44:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33910.782229 secs 2023-09-10T08:44:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33911.186943 secs 2023-09-10T08:44:39.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:44:39.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:44:39.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:44:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:39 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33909.581555 secs 2023-09-10T08:44:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:44:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33910.782229 secs 2023-09-10T08:44:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33911.186943 secs 2023-09-10T08:44:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:44:39.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:44:39.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:44:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:40 smithi082 ceph-mon[98694]: pgmap v18244: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 10 op/s 2023-09-10T08:44:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:44:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:40 smithi098 ceph-mon[103124]: pgmap v18244: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 10 op/s 2023-09-10T08:44:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:44:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:40 smithi170 ceph-mon[102832]: pgmap v18244: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 120 KiB/s wr, 10 op/s 2023-09-10T08:44:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:44:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:41 smithi098 ceph-mon[103124]: pgmap v18245: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 548 KiB/s wr, 18 op/s 2023-09-10T08:44:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:41 smithi170 ceph-mon[102832]: pgmap v18245: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 548 KiB/s wr, 18 op/s 2023-09-10T08:44:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:41 smithi082 ceph-mon[98694]: pgmap v18245: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 548 KiB/s wr, 18 op/s 2023-09-10T08:44:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:43 smithi082 ceph-mon[98694]: pgmap v18246: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 547 KiB/s wr, 13 op/s 2023-09-10T08:44:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:44:43.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:44:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:43 smithi098 ceph-mon[103124]: pgmap v18246: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 547 KiB/s wr, 13 op/s 2023-09-10T08:44:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:44:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:44:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:43 smithi170 ceph-mon[102832]: pgmap v18246: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 547 KiB/s wr, 13 op/s 2023-09-10T08:44:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:44:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:44:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:44:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33915.782290 secs 2023-09-10T08:44:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33916.187008 secs 2023-09-10T08:44:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:44:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33915.782290 secs 2023-09-10T08:44:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33916.187008 secs 2023-09-10T08:44:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:44:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33915.782290 secs 2023-09-10T08:44:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33916.187008 secs 2023-09-10T08:44:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:45 smithi098 ceph-mon[103124]: pgmap v18247: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 547 KiB/s wr, 13 op/s 2023-09-10T08:44:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:45 smithi170 ceph-mon[102832]: pgmap v18247: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 547 KiB/s wr, 13 op/s 2023-09-10T08:44:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:45 smithi082 ceph-mon[98694]: pgmap v18247: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 547 KiB/s wr, 13 op/s 2023-09-10T08:44:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:44:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:44:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:44:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:47 smithi098 ceph-mon[103124]: pgmap v18248: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 550 KiB/s wr, 19 op/s 2023-09-10T08:44:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:47 smithi170 ceph-mon[102832]: pgmap v18248: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 550 KiB/s wr, 19 op/s 2023-09-10T08:44:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:47 smithi082 ceph-mon[98694]: pgmap v18248: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 550 KiB/s wr, 19 op/s 2023-09-10T08:44:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:48 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:44:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:48 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:44:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:48 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:44:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:49 smithi098 ceph-mon[103124]: pgmap v18249: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 430 KiB/s wr, 13 op/s 2023-09-10T08:44:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33920.782374 secs 2023-09-10T08:44:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33921.187073 secs 2023-09-10T08:44:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:49 smithi170 ceph-mon[102832]: pgmap v18249: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 430 KiB/s wr, 13 op/s 2023-09-10T08:44:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33920.782374 secs 2023-09-10T08:44:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33921.187073 secs 2023-09-10T08:44:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:49 smithi082 ceph-mon[98694]: pgmap v18249: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 430 KiB/s wr, 13 op/s 2023-09-10T08:44:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33920.782374 secs 2023-09-10T08:44:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33921.187073 secs 2023-09-10T08:44:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:44:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:44:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:44:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:51 smithi098 ceph-mon[103124]: pgmap v18250: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T08:44:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:51 smithi170 ceph-mon[102832]: pgmap v18250: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T08:44:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:51 smithi082 ceph-mon[98694]: pgmap v18250: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T08:44:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:44:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:44:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:44:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:54 smithi170 ceph-mon[102832]: pgmap v18251: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 662 KiB/s wr, 14 op/s 2023-09-10T08:44:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33925.782430 secs 2023-09-10T08:44:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33926.187140 secs 2023-09-10T08:44:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:54 smithi082 ceph-mon[98694]: pgmap v18251: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 662 KiB/s wr, 14 op/s 2023-09-10T08:44:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33925.782430 secs 2023-09-10T08:44:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33926.187140 secs 2023-09-10T08:44:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:54 smithi098 ceph-mon[103124]: pgmap v18251: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 662 KiB/s wr, 14 op/s 2023-09-10T08:44:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33925.782430 secs 2023-09-10T08:44:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33926.187140 secs 2023-09-10T08:44:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:56 smithi082 ceph-mon[98694]: pgmap v18252: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 662 KiB/s wr, 14 op/s 2023-09-10T08:44:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:44:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:56 smithi098 ceph-mon[103124]: pgmap v18252: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 662 KiB/s wr, 14 op/s 2023-09-10T08:44:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:44:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:56 smithi170 ceph-mon[102832]: pgmap v18252: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 662 KiB/s wr, 14 op/s 2023-09-10T08:44:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:44:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:58 smithi082 ceph-mon[98694]: pgmap v18253: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 709 KiB/s wr, 20 op/s 2023-09-10T08:44:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:44:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:58 smithi098 ceph-mon[103124]: pgmap v18253: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 709 KiB/s wr, 20 op/s 2023-09-10T08:44:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:44:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:58 smithi170 ceph-mon[102832]: pgmap v18253: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 709 KiB/s wr, 20 op/s 2023-09-10T08:44:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:44:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:44:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33930.782494 secs 2023-09-10T08:44:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:44:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33931.187203 secs 2023-09-10T08:44:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33930.782494 secs 2023-09-10T08:44:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:44:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33931.187203 secs 2023-09-10T08:44:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33930.782494 secs 2023-09-10T08:44:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:44:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33931.187203 secs 2023-09-10T08:45:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:00 smithi082 ceph-mon[98694]: pgmap v18254: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 706 KiB/s wr, 14 op/s 2023-09-10T08:45:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:00 smithi098 ceph-mon[103124]: pgmap v18254: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 706 KiB/s wr, 14 op/s 2023-09-10T08:45:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:00 smithi170 ceph-mon[102832]: pgmap v18254: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 706 KiB/s wr, 14 op/s 2023-09-10T08:45:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:02 smithi082 ceph-mon[98694]: pgmap v18255: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 945 KiB/s wr, 20 op/s 2023-09-10T08:45:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:02 smithi098 ceph-mon[103124]: pgmap v18255: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 945 KiB/s wr, 20 op/s 2023-09-10T08:45:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:02 smithi170 ceph-mon[102832]: pgmap v18255: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 945 KiB/s wr, 20 op/s 2023-09-10T08:45:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:04 smithi170 ceph-mon[102832]: pgmap v18256: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 286 KiB/s wr, 12 op/s 2023-09-10T08:45:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33934.581936 secs 2023-09-10T08:45:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:45:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:45:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:45:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33935.782577 secs 2023-09-10T08:45:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33936.187270 secs 2023-09-10T08:45:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:04 smithi082 ceph-mon[98694]: pgmap v18256: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 286 KiB/s wr, 12 op/s 2023-09-10T08:45:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33934.581936 secs 2023-09-10T08:45:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:45:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:45:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:45:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33935.782577 secs 2023-09-10T08:45:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33936.187270 secs 2023-09-10T08:45:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:04 smithi098 ceph-mon[103124]: pgmap v18256: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 286 KiB/s wr, 12 op/s 2023-09-10T08:45:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33934.581936 secs 2023-09-10T08:45:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:45:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:45:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:45:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33935.782577 secs 2023-09-10T08:45:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33936.187270 secs 2023-09-10T08:45:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:06 smithi082 ceph-mon[98694]: pgmap v18257: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 286 KiB/s wr, 12 op/s 2023-09-10T08:45:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:45:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:45:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:06 smithi098 ceph-mon[103124]: pgmap v18257: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 286 KiB/s wr, 12 op/s 2023-09-10T08:45:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:45:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:45:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:06 smithi170 ceph-mon[102832]: pgmap v18257: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 286 KiB/s wr, 12 op/s 2023-09-10T08:45:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:45:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:45:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:08 smithi082 ceph-mon[98694]: pgmap v18258: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 612 KiB/s wr, 33 op/s 2023-09-10T08:45:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:08 smithi098 ceph-mon[103124]: pgmap v18258: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 612 KiB/s wr, 33 op/s 2023-09-10T08:45:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:08 smithi170 ceph-mon[102832]: pgmap v18258: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 612 KiB/s wr, 33 op/s 2023-09-10T08:45:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33939.581995 secs 2023-09-10T08:45:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:09 smithi082 ceph-mon[98694]: osdmap e1209: 12 total, 12 up, 12 in 2023-09-10T08:45:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33940.782671 secs 2023-09-10T08:45:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33941.187337 secs 2023-09-10T08:45:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33939.581995 secs 2023-09-10T08:45:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:09 smithi098 ceph-mon[103124]: osdmap e1209: 12 total, 12 up, 12 in 2023-09-10T08:45:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33940.782671 secs 2023-09-10T08:45:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33941.187337 secs 2023-09-10T08:45:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33939.581995 secs 2023-09-10T08:45:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:09 smithi170 ceph-mon[102832]: osdmap e1209: 12 total, 12 up, 12 in 2023-09-10T08:45:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33940.782671 secs 2023-09-10T08:45:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33941.187337 secs 2023-09-10T08:45:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:10 smithi082 ceph-mon[98694]: pgmap v18260: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 677 KiB/s wr, 32 op/s 2023-09-10T08:45:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:10 smithi098 ceph-mon[103124]: pgmap v18260: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 677 KiB/s wr, 32 op/s 2023-09-10T08:45:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:10 smithi170 ceph-mon[102832]: pgmap v18260: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 677 KiB/s wr, 32 op/s 2023-09-10T08:45:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:12 smithi082 ceph-mon[98694]: pgmap v18261: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 537 KiB/s wr, 32 op/s 2023-09-10T08:45:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:12 smithi098 ceph-mon[103124]: pgmap v18261: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 537 KiB/s wr, 32 op/s 2023-09-10T08:45:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:12 smithi170 ceph-mon[102832]: pgmap v18261: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 537 KiB/s wr, 32 op/s 2023-09-10T08:45:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:14 smithi082 ceph-mon[98694]: pgmap v18262: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 537 KiB/s wr, 32 op/s 2023-09-10T08:45:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33944.582059 secs 2023-09-10T08:45:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:14 smithi082 ceph-mon[98694]: osdmap e1210: 12 total, 12 up, 12 in 2023-09-10T08:45:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33945.782729 secs 2023-09-10T08:45:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33946.187402 secs 2023-09-10T08:45:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:14 smithi098 ceph-mon[103124]: pgmap v18262: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 537 KiB/s wr, 32 op/s 2023-09-10T08:45:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33944.582059 secs 2023-09-10T08:45:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:14 smithi098 ceph-mon[103124]: osdmap e1210: 12 total, 12 up, 12 in 2023-09-10T08:45:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33945.782729 secs 2023-09-10T08:45:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33946.187402 secs 2023-09-10T08:45:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:14 smithi170 ceph-mon[102832]: pgmap v18262: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 537 KiB/s wr, 32 op/s 2023-09-10T08:45:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33944.582059 secs 2023-09-10T08:45:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:14 smithi170 ceph-mon[102832]: osdmap e1210: 12 total, 12 up, 12 in 2023-09-10T08:45:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33945.782729 secs 2023-09-10T08:45:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33946.187402 secs 2023-09-10T08:45:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:16 smithi082 ceph-mon[98694]: pgmap v18264: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 183 KiB/s wr, 8 op/s 2023-09-10T08:45:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:16 smithi098 ceph-mon[103124]: pgmap v18264: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 183 KiB/s wr, 8 op/s 2023-09-10T08:45:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:16 smithi170 ceph-mon[102832]: pgmap v18264: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 183 KiB/s wr, 8 op/s 2023-09-10T08:45:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:18 smithi082 ceph-mon[98694]: pgmap v18265: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 215 KiB/s wr, 16 op/s 2023-09-10T08:45:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:18 smithi098 ceph-mon[103124]: pgmap v18265: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 215 KiB/s wr, 16 op/s 2023-09-10T08:45:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:18 smithi170 ceph-mon[102832]: pgmap v18265: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 215 KiB/s wr, 16 op/s 2023-09-10T08:45:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33949.582151 secs 2023-09-10T08:45:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33950.782787 secs 2023-09-10T08:45:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33951.187465 secs 2023-09-10T08:45:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33949.582151 secs 2023-09-10T08:45:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33950.782787 secs 2023-09-10T08:45:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33951.187465 secs 2023-09-10T08:45:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33949.582151 secs 2023-09-10T08:45:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33950.782787 secs 2023-09-10T08:45:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33951.187465 secs 2023-09-10T08:45:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:20 smithi082 ceph-mon[98694]: pgmap v18266: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 13 op/s 2023-09-10T08:45:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:20 smithi098 ceph-mon[103124]: pgmap v18266: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 13 op/s 2023-09-10T08:45:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:20 smithi170 ceph-mon[102832]: pgmap v18266: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 13 op/s 2023-09-10T08:45:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:22 smithi082 ceph-mon[98694]: pgmap v18267: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-10T08:45:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:22 smithi098 ceph-mon[103124]: pgmap v18267: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-10T08:45:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:22 smithi170 ceph-mon[102832]: pgmap v18267: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-10T08:45:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:24 smithi082 ceph-mon[98694]: pgmap v18268: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-10T08:45:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33954.582294 secs 2023-09-10T08:45:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33955.782844 secs 2023-09-10T08:45:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33956.187486 secs 2023-09-10T08:45:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:24 smithi098 ceph-mon[103124]: pgmap v18268: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-10T08:45:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33954.582294 secs 2023-09-10T08:45:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33955.782844 secs 2023-09-10T08:45:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33956.187486 secs 2023-09-10T08:45:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:24 smithi170 ceph-mon[102832]: pgmap v18268: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-10T08:45:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33954.582294 secs 2023-09-10T08:45:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33955.782844 secs 2023-09-10T08:45:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33956.187486 secs 2023-09-10T08:45:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:26 smithi082 ceph-mon[98694]: pgmap v18269: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T08:45:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:26 smithi098 ceph-mon[103124]: pgmap v18269: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T08:45:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:26 smithi170 ceph-mon[102832]: pgmap v18269: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T08:45:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:28 smithi082 ceph-mon[98694]: pgmap v18270: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 16 op/s 2023-09-10T08:45:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:28 smithi098 ceph-mon[103124]: pgmap v18270: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 16 op/s 2023-09-10T08:45:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:28 smithi170 ceph-mon[102832]: pgmap v18270: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 16 op/s 2023-09-10T08:45:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33959.582364 secs 2023-09-10T08:45:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33960.782903 secs 2023-09-10T08:45:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33961.187551 secs 2023-09-10T08:45:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33959.582364 secs 2023-09-10T08:45:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33960.782903 secs 2023-09-10T08:45:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33961.187551 secs 2023-09-10T08:45:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33959.582364 secs 2023-09-10T08:45:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33960.782903 secs 2023-09-10T08:45:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33961.187551 secs 2023-09-10T08:45:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:30 smithi098 ceph-mon[103124]: pgmap v18271: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T08:45:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:30 smithi170 ceph-mon[102832]: pgmap v18271: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T08:45:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:30 smithi082 ceph-mon[98694]: pgmap v18271: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T08:45:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:31 smithi098 ceph-mon[103124]: pgmap v18272: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 122 KiB/s wr, 16 op/s 2023-09-10T08:45:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:31 smithi170 ceph-mon[102832]: pgmap v18272: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 122 KiB/s wr, 16 op/s 2023-09-10T08:45:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:31 smithi082 ceph-mon[98694]: pgmap v18272: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 122 KiB/s wr, 16 op/s 2023-09-10T08:45:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:34 smithi082 ceph-mon[98694]: pgmap v18273: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T08:45:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:45:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:45:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33965.832167 secs 2023-09-10T08:45:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33966.187616 secs 2023-09-10T08:45:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:34 smithi098 ceph-mon[103124]: pgmap v18273: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T08:45:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:45:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:45:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33965.832167 secs 2023-09-10T08:45:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33966.187616 secs 2023-09-10T08:45:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:34 smithi170 ceph-mon[102832]: pgmap v18273: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T08:45:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:45:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:45:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33965.832167 secs 2023-09-10T08:45:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33966.187616 secs 2023-09-10T08:45:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:45:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:45:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:45:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:36 smithi082 ceph-mon[98694]: pgmap v18274: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T08:45:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:45:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:45:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:45:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:36 smithi098 ceph-mon[103124]: pgmap v18274: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T08:45:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:45:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:45:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:45:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:36 smithi170 ceph-mon[102832]: pgmap v18274: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 10 op/s 2023-09-10T08:45:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:45:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:45:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:45:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:38 smithi082 ceph-mon[98694]: pgmap v18275: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 243 KiB/s wr, 16 op/s 2023-09-10T08:45:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:38 smithi098 ceph-mon[103124]: pgmap v18275: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 243 KiB/s wr, 16 op/s 2023-09-10T08:45:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:38 smithi170 ceph-mon[102832]: pgmap v18275: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 243 KiB/s wr, 16 op/s 2023-09-10T08:45:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:39 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33969.582553 secs 2023-09-10T08:45:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33970.832241 secs 2023-09-10T08:45:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33971.187684 secs 2023-09-10T08:45:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:39 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33969.582553 secs 2023-09-10T08:45:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33970.832241 secs 2023-09-10T08:45:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33971.187684 secs 2023-09-10T08:45:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:39 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33969.582553 secs 2023-09-10T08:45:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33970.832241 secs 2023-09-10T08:45:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33971.187684 secs 2023-09-10T08:45:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:40 smithi082 ceph-mon[98694]: pgmap v18276: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 10 op/s 2023-09-10T08:45:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:45:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:45:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:45:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:45:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:45:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:45:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:40 smithi098 ceph-mon[103124]: pgmap v18276: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 10 op/s 2023-09-10T08:45:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:45:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:45:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:45:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:45:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:45:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:45:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:40 smithi170 ceph-mon[102832]: pgmap v18276: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 10 op/s 2023-09-10T08:45:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:45:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:45:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:45:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:45:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:45:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:45:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:42 smithi082 ceph-mon[98694]: pgmap v18277: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 530 KiB/s wr, 18 op/s 2023-09-10T08:45:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:42 smithi098 ceph-mon[103124]: pgmap v18277: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 530 KiB/s wr, 18 op/s 2023-09-10T08:45:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:42 smithi170 ceph-mon[102832]: pgmap v18277: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 530 KiB/s wr, 18 op/s 2023-09-10T08:45:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:45:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:45:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:45:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:45:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:45:43.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:45:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:44 smithi082 ceph-mon[98694]: pgmap v18278: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 530 KiB/s wr, 13 op/s 2023-09-10T08:45:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:45:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:45:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33975.832303 secs 2023-09-10T08:45:44.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33976.187752 secs 2023-09-10T08:45:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:44 smithi098 ceph-mon[103124]: pgmap v18278: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 530 KiB/s wr, 13 op/s 2023-09-10T08:45:44.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:45:44.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:45:44.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33975.832303 secs 2023-09-10T08:45:44.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33976.187752 secs 2023-09-10T08:45:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:44 smithi170 ceph-mon[102832]: pgmap v18278: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 530 KiB/s wr, 13 op/s 2023-09-10T08:45:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:45:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:45:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33975.832303 secs 2023-09-10T08:45:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33976.187752 secs 2023-09-10T08:45:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:46 smithi098 ceph-mon[103124]: pgmap v18279: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 530 KiB/s wr, 13 op/s 2023-09-10T08:45:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:45:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:45:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:46 smithi170 ceph-mon[102832]: pgmap v18279: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 530 KiB/s wr, 13 op/s 2023-09-10T08:45:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:45:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:45:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:46 smithi082 ceph-mon[98694]: pgmap v18279: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 530 KiB/s wr, 13 op/s 2023-09-10T08:45:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:45:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:45:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:47 smithi170 ceph-mon[102832]: pgmap v18280: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 666 KiB/s wr, 19 op/s 2023-09-10T08:45:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:47 smithi082 ceph-mon[98694]: pgmap v18280: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 666 KiB/s wr, 19 op/s 2023-09-10T08:45:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:47 smithi098 ceph-mon[103124]: pgmap v18280: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 666 KiB/s wr, 19 op/s 2023-09-10T08:45:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:48 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:45:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:48 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:45:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:48 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:45:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:49 smithi170 ceph-mon[102832]: pgmap v18281: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 545 KiB/s wr, 13 op/s 2023-09-10T08:45:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33980.832383 secs 2023-09-10T08:45:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33981.187817 secs 2023-09-10T08:45:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:49 smithi082 ceph-mon[98694]: pgmap v18281: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 545 KiB/s wr, 13 op/s 2023-09-10T08:45:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33980.832383 secs 2023-09-10T08:45:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33981.187817 secs 2023-09-10T08:45:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:49 smithi098 ceph-mon[103124]: pgmap v18281: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 545 KiB/s wr, 13 op/s 2023-09-10T08:45:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33980.832383 secs 2023-09-10T08:45:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33981.187817 secs 2023-09-10T08:45:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:45:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:45:50.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:45:50.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:45:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:45:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:45:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:51 smithi170 ceph-mon[102832]: pgmap v18282: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T08:45:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:51 smithi082 ceph-mon[98694]: pgmap v18282: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T08:45:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:51 smithi098 ceph-mon[103124]: pgmap v18282: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T08:45:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:45:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:45:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:45:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:45:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:45:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:45:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:54 smithi082 ceph-mon[98694]: pgmap v18283: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 685 KiB/s wr, 13 op/s 2023-09-10T08:45:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33985.832468 secs 2023-09-10T08:45:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33986.187880 secs 2023-09-10T08:45:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:54 smithi098 ceph-mon[103124]: pgmap v18283: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 685 KiB/s wr, 13 op/s 2023-09-10T08:45:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33985.832468 secs 2023-09-10T08:45:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33986.187880 secs 2023-09-10T08:45:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:54 smithi170 ceph-mon[102832]: pgmap v18283: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 685 KiB/s wr, 13 op/s 2023-09-10T08:45:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33985.832468 secs 2023-09-10T08:45:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33986.187880 secs 2023-09-10T08:45:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:56 smithi082 ceph-mon[98694]: pgmap v18284: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 685 KiB/s wr, 13 op/s 2023-09-10T08:45:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:56 smithi098 ceph-mon[103124]: pgmap v18284: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 685 KiB/s wr, 13 op/s 2023-09-10T08:45:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:56 smithi170 ceph-mon[102832]: pgmap v18284: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 685 KiB/s wr, 13 op/s 2023-09-10T08:45:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:58 smithi082 ceph-mon[98694]: pgmap v18285: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 870 KiB/s wr, 20 op/s 2023-09-10T08:45:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:58 smithi098 ceph-mon[103124]: pgmap v18285: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 870 KiB/s wr, 20 op/s 2023-09-10T08:45:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:58 smithi170 ceph-mon[102832]: pgmap v18285: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 870 KiB/s wr, 20 op/s 2023-09-10T08:45:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33990.832524 secs 2023-09-10T08:45:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:45:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33991.187947 secs 2023-09-10T08:45:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33990.832524 secs 2023-09-10T08:45:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:45:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33991.187947 secs 2023-09-10T08:45:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33990.832524 secs 2023-09-10T08:45:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:45:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33991.187947 secs 2023-09-10T08:46:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:00 smithi082 ceph-mon[98694]: pgmap v18286: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 734 KiB/s wr, 14 op/s 2023-09-10T08:46:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:00 smithi098 ceph-mon[103124]: pgmap v18286: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 734 KiB/s wr, 14 op/s 2023-09-10T08:46:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:00 smithi170 ceph-mon[102832]: pgmap v18286: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 734 KiB/s wr, 14 op/s 2023-09-10T08:46:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:02 smithi098 ceph-mon[103124]: pgmap v18287: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 937 KiB/s wr, 20 op/s 2023-09-10T08:46:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:02 smithi170 ceph-mon[102832]: pgmap v18287: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 937 KiB/s wr, 20 op/s 2023-09-10T08:46:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:02 smithi082 ceph-mon[98694]: pgmap v18287: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 937 KiB/s wr, 20 op/s 2023-09-10T08:46:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33994.582872 secs 2023-09-10T08:46:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33994.582872 secs 2023-09-10T08:46:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33994.582872 secs 2023-09-10T08:46:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:04 smithi098 ceph-mon[103124]: pgmap v18288: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 388 KiB/s wr, 12 op/s 2023-09-10T08:46:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:46:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:46:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33995.832581 secs 2023-09-10T08:46:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 33996.188010 secs 2023-09-10T08:46:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:04 smithi170 ceph-mon[102832]: pgmap v18288: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 388 KiB/s wr, 12 op/s 2023-09-10T08:46:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:46:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:46:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33995.832581 secs 2023-09-10T08:46:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 33996.188010 secs 2023-09-10T08:46:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:04 smithi082 ceph-mon[98694]: pgmap v18288: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 388 KiB/s wr, 12 op/s 2023-09-10T08:46:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:46:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:46:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33995.832581 secs 2023-09-10T08:46:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 33996.188010 secs 2023-09-10T08:46:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:05 smithi098 ceph-mon[103124]: pgmap v18289: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 388 KiB/s wr, 12 op/s 2023-09-10T08:46:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:05 smithi170 ceph-mon[102832]: pgmap v18289: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 388 KiB/s wr, 12 op/s 2023-09-10T08:46:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:05 smithi082 ceph-mon[98694]: pgmap v18289: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 388 KiB/s wr, 12 op/s 2023-09-10T08:46:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:46:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:46:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:46:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:07 smithi098 ceph-mon[103124]: pgmap v18290: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 580 KiB/s wr, 33 op/s 2023-09-10T08:46:07.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:07 smithi170 ceph-mon[102832]: pgmap v18290: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 580 KiB/s wr, 33 op/s 2023-09-10T08:46:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:07 smithi082 ceph-mon[98694]: pgmap v18290: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 580 KiB/s wr, 33 op/s 2023-09-10T08:46:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 33999.582935 secs 2023-09-10T08:46:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 33999.582935 secs 2023-09-10T08:46:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 33999.582935 secs 2023-09-10T08:46:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:09 smithi098 ceph-mon[103124]: osdmap e1211: 12 total, 12 up, 12 in 2023-09-10T08:46:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:09 smithi098 ceph-mon[103124]: pgmap v18292: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 474 KiB/s wr, 32 op/s 2023-09-10T08:46:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34000.832641 secs 2023-09-10T08:46:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34001.188097 secs 2023-09-10T08:46:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:09 smithi170 ceph-mon[102832]: osdmap e1211: 12 total, 12 up, 12 in 2023-09-10T08:46:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:09 smithi170 ceph-mon[102832]: pgmap v18292: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 474 KiB/s wr, 32 op/s 2023-09-10T08:46:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34000.832641 secs 2023-09-10T08:46:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34001.188097 secs 2023-09-10T08:46:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:09 smithi082 ceph-mon[98694]: osdmap e1211: 12 total, 12 up, 12 in 2023-09-10T08:46:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:09 smithi082 ceph-mon[98694]: pgmap v18292: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 474 KiB/s wr, 32 op/s 2023-09-10T08:46:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34000.832641 secs 2023-09-10T08:46:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34001.188097 secs 2023-09-10T08:46:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:11 smithi098 ceph-mon[103124]: pgmap v18293: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 424 KiB/s wr, 31 op/s 2023-09-10T08:46:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:11 smithi170 ceph-mon[102832]: pgmap v18293: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 424 KiB/s wr, 31 op/s 2023-09-10T08:46:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:11 smithi082 ceph-mon[98694]: pgmap v18293: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 424 KiB/s wr, 31 op/s 2023-09-10T08:46:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:13 smithi098 ceph-mon[103124]: pgmap v18294: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 424 KiB/s wr, 31 op/s 2023-09-10T08:46:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34004.582993 secs 2023-09-10T08:46:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:13 smithi098 ceph-mon[103124]: osdmap e1212: 12 total, 12 up, 12 in 2023-09-10T08:46:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:13 smithi170 ceph-mon[102832]: pgmap v18294: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 424 KiB/s wr, 31 op/s 2023-09-10T08:46:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34004.582993 secs 2023-09-10T08:46:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:13 smithi170 ceph-mon[102832]: osdmap e1212: 12 total, 12 up, 12 in 2023-09-10T08:46:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:13 smithi082 ceph-mon[98694]: pgmap v18294: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 424 KiB/s wr, 31 op/s 2023-09-10T08:46:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34004.582993 secs 2023-09-10T08:46:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:13 smithi082 ceph-mon[98694]: osdmap e1212: 12 total, 12 up, 12 in 2023-09-10T08:46:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:14 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:46:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:14 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34005.832699 secs 2023-09-10T08:46:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34006.188160 secs 2023-09-10T08:46:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:14 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:46:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:14 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34005.832699 secs 2023-09-10T08:46:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34006.188160 secs 2023-09-10T08:46:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:14 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:46:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:14 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34005.832699 secs 2023-09-10T08:46:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34006.188160 secs 2023-09-10T08:46:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:15 smithi098 ceph-mon[103124]: pgmap v18296: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 243 KiB/s wr, 8 op/s 2023-09-10T08:46:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:15 smithi170 ceph-mon[102832]: pgmap v18296: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 243 KiB/s wr, 8 op/s 2023-09-10T08:46:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:15 smithi082 ceph-mon[98694]: pgmap v18296: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 243 KiB/s wr, 8 op/s 2023-09-10T08:46:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:46:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:46:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:46:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:17 smithi098 ceph-mon[103124]: pgmap v18297: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 275 KiB/s wr, 17 op/s 2023-09-10T08:46:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:17 smithi170 ceph-mon[102832]: pgmap v18297: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 275 KiB/s wr, 17 op/s 2023-09-10T08:46:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:17 smithi082 ceph-mon[98694]: pgmap v18297: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 275 KiB/s wr, 17 op/s 2023-09-10T08:46:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34009.583090 secs 2023-09-10T08:46:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34009.583090 secs 2023-09-10T08:46:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34009.583090 secs 2023-09-10T08:46:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:19 smithi170 ceph-mon[102832]: pgmap v18298: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 225 KiB/s wr, 14 op/s 2023-09-10T08:46:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34010.832756 secs 2023-09-10T08:46:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34011.188224 secs 2023-09-10T08:46:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:19 smithi082 ceph-mon[98694]: pgmap v18298: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 225 KiB/s wr, 14 op/s 2023-09-10T08:46:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34010.832756 secs 2023-09-10T08:46:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34011.188224 secs 2023-09-10T08:46:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:19 smithi098 ceph-mon[103124]: pgmap v18298: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 225 KiB/s wr, 14 op/s 2023-09-10T08:46:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34010.832756 secs 2023-09-10T08:46:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34011.188224 secs 2023-09-10T08:46:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:21 smithi170 ceph-mon[102832]: pgmap v18299: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:46:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:21 smithi082 ceph-mon[98694]: pgmap v18299: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:46:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:21 smithi098 ceph-mon[103124]: pgmap v18299: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:46:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:23 smithi170 ceph-mon[102832]: pgmap v18300: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:46:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34014.583150 secs 2023-09-10T08:46:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:23 smithi082 ceph-mon[98694]: pgmap v18300: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:46:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34014.583150 secs 2023-09-10T08:46:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:23 smithi098 ceph-mon[103124]: pgmap v18300: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:46:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34014.583150 secs 2023-09-10T08:46:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34015.832875 secs 2023-09-10T08:46:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34016.188288 secs 2023-09-10T08:46:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34015.832875 secs 2023-09-10T08:46:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34016.188288 secs 2023-09-10T08:46:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34015.832875 secs 2023-09-10T08:46:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34016.188288 secs 2023-09-10T08:46:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:25 smithi170 ceph-mon[102832]: pgmap v18301: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 12 op/s 2023-09-10T08:46:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:25 smithi082 ceph-mon[98694]: pgmap v18301: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 12 op/s 2023-09-10T08:46:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:25 smithi098 ceph-mon[103124]: pgmap v18301: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 12 op/s 2023-09-10T08:46:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:27 smithi170 ceph-mon[102832]: pgmap v18302: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 17 op/s 2023-09-10T08:46:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:27 smithi082 ceph-mon[98694]: pgmap v18302: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 17 op/s 2023-09-10T08:46:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:27 smithi098 ceph-mon[103124]: pgmap v18302: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 17 op/s 2023-09-10T08:46:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34019.583263 secs 2023-09-10T08:46:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34019.583263 secs 2023-09-10T08:46:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34019.583263 secs 2023-09-10T08:46:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:29 smithi170 ceph-mon[102832]: pgmap v18303: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T08:46:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34020.832945 secs 2023-09-10T08:46:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34021.188352 secs 2023-09-10T08:46:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:29 smithi082 ceph-mon[98694]: pgmap v18303: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T08:46:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34020.832945 secs 2023-09-10T08:46:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34021.188352 secs 2023-09-10T08:46:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:29 smithi098 ceph-mon[103124]: pgmap v18303: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T08:46:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34020.832945 secs 2023-09-10T08:46:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34021.188352 secs 2023-09-10T08:46:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:31 smithi170 ceph-mon[102832]: pgmap v18304: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 16 op/s 2023-09-10T08:46:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:31 smithi082 ceph-mon[98694]: pgmap v18304: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 16 op/s 2023-09-10T08:46:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:31 smithi098 ceph-mon[103124]: pgmap v18304: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 16 op/s 2023-09-10T08:46:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:34 smithi082 ceph-mon[98694]: pgmap v18305: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T08:46:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34025.832999 secs 2023-09-10T08:46:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34026.188417 secs 2023-09-10T08:46:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:34 smithi098 ceph-mon[103124]: pgmap v18305: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T08:46:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34025.832999 secs 2023-09-10T08:46:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34026.188417 secs 2023-09-10T08:46:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:34 smithi170 ceph-mon[102832]: pgmap v18305: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T08:46:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34025.832999 secs 2023-09-10T08:46:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34026.188417 secs 2023-09-10T08:46:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:46:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:46:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:46:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:36 smithi082 ceph-mon[98694]: pgmap v18306: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T08:46:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:46:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:46:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:36 smithi098 ceph-mon[103124]: pgmap v18306: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T08:46:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:46:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:46:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:36 smithi170 ceph-mon[102832]: pgmap v18306: 97 pgs: 97 active+clean; 11 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T08:46:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:46:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:46:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:38 smithi098 ceph-mon[103124]: pgmap v18307: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 244 KiB/s wr, 16 op/s 2023-09-10T08:46:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:46:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:38 smithi170 ceph-mon[102832]: pgmap v18307: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 244 KiB/s wr, 16 op/s 2023-09-10T08:46:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:46:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:38 smithi082 ceph-mon[98694]: pgmap v18307: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 244 KiB/s wr, 16 op/s 2023-09-10T08:46:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:46:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34030.833057 secs 2023-09-10T08:46:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34031.188483 secs 2023-09-10T08:46:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34030.833057 secs 2023-09-10T08:46:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34031.188483 secs 2023-09-10T08:46:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34030.833057 secs 2023-09-10T08:46:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34031.188483 secs 2023-09-10T08:46:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:40 smithi098 ceph-mon[103124]: pgmap v18308: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T08:46:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:46:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:40 smithi170 ceph-mon[102832]: pgmap v18308: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T08:46:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:46:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:40 smithi082 ceph-mon[98694]: pgmap v18308: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-09-10T08:46:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:46:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:42 smithi098 ceph-mon[103124]: pgmap v18309: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 552 KiB/s wr, 18 op/s 2023-09-10T08:46:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:42 smithi170 ceph-mon[102832]: pgmap v18309: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 552 KiB/s wr, 18 op/s 2023-09-10T08:46:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:42 smithi082 ceph-mon[98694]: pgmap v18309: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 552 KiB/s wr, 18 op/s 2023-09-10T08:46:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:43 smithi170 ceph-mon[102832]: pgmap v18310: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 552 KiB/s wr, 13 op/s 2023-09-10T08:46:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:46:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:46:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:46:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:46:43.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:46:43.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:46:43.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34034.583480 secs 2023-09-10T08:46:43.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:46:43.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:46:43.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:46:43.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:46:43.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:46:43.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:43 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T08:46:43.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:43 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T08:46:43.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:46:43.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:46:43.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:46:43.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:46:43.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:46:43.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:46:43.932 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:46:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:43 smithi082 ceph-mon[98694]: pgmap v18310: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 552 KiB/s wr, 13 op/s 2023-09-10T08:46:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:46:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:46:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:46:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:46:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:46:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:46:43.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34034.583480 secs 2023-09-10T08:46:43.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:46:43.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:46:43.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:46:43.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:46:43.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:46:43.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:43 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T08:46:43.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:43 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T08:46:43.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:46:43.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:46:43.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:46:43.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:46:43.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:46:43.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:46:43.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:46:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:43 smithi098 ceph-mon[103124]: pgmap v18310: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 552 KiB/s wr, 13 op/s 2023-09-10T08:46:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:46:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:46:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:46:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:46:44.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:46:44.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:46:44.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34034.583480 secs 2023-09-10T08:46:44.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:46:44.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:46:44.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:46:44.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:46:44.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:46:44.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:43 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T08:46:44.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:43 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T08:46:44.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:46:44.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:46:44.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:46:44.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:46:44.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:46:44.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:46:44.008 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:46:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:44 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T08:46:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:44 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T08:46:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34035.833120 secs 2023-09-10T08:46:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:46:44.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:46:44.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:44.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34036.188546 secs 2023-09-10T08:46:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:44 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T08:46:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:44 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T08:46:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34035.833120 secs 2023-09-10T08:46:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:46:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:46:44.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:44.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34036.188546 secs 2023-09-10T08:46:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:44 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T08:46:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:44 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T08:46:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34035.833120 secs 2023-09-10T08:46:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:46:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:46:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34036.188546 secs 2023-09-10T08:46:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:45 smithi170 ceph-mon[102832]: pgmap v18311: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 552 KiB/s wr, 13 op/s 2023-09-10T08:46:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:45 smithi082 ceph-mon[98694]: pgmap v18311: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 552 KiB/s wr, 13 op/s 2023-09-10T08:46:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:45 smithi098 ceph-mon[103124]: pgmap v18311: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 552 KiB/s wr, 13 op/s 2023-09-10T08:46:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:46:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:46:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:46:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:47 smithi170 ceph-mon[102832]: pgmap v18312: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 554 KiB/s wr, 19 op/s 2023-09-10T08:46:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:47 smithi082 ceph-mon[98694]: pgmap v18312: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 554 KiB/s wr, 19 op/s 2023-09-10T08:46:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:47 smithi098 ceph-mon[103124]: pgmap v18312: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 554 KiB/s wr, 19 op/s 2023-09-10T08:46:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34039.583543 secs 2023-09-10T08:46:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34039.583543 secs 2023-09-10T08:46:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34039.583543 secs 2023-09-10T08:46:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:49 smithi170 ceph-mon[102832]: pgmap v18313: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 433 KiB/s wr, 13 op/s 2023-09-10T08:46:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34040.833178 secs 2023-09-10T08:46:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34041.188615 secs 2023-09-10T08:46:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:49 smithi082 ceph-mon[98694]: pgmap v18313: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 433 KiB/s wr, 13 op/s 2023-09-10T08:46:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34040.833178 secs 2023-09-10T08:46:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34041.188615 secs 2023-09-10T08:46:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:49 smithi098 ceph-mon[103124]: pgmap v18313: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 433 KiB/s wr, 13 op/s 2023-09-10T08:46:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34040.833178 secs 2023-09-10T08:46:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34041.188615 secs 2023-09-10T08:46:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:51 smithi170 ceph-mon[102832]: pgmap v18314: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 862 KiB/s wr, 20 op/s 2023-09-10T08:46:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:51 smithi082 ceph-mon[98694]: pgmap v18314: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 862 KiB/s wr, 20 op/s 2023-09-10T08:46:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:51 smithi098 ceph-mon[103124]: pgmap v18314: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 862 KiB/s wr, 20 op/s 2023-09-10T08:46:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:53 smithi170 ceph-mon[102832]: pgmap v18315: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 432 KiB/s wr, 13 op/s 2023-09-10T08:46:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:46:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:53 smithi082 ceph-mon[98694]: pgmap v18315: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 432 KiB/s wr, 13 op/s 2023-09-10T08:46:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:46:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:53 smithi098 ceph-mon[103124]: pgmap v18315: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 432 KiB/s wr, 13 op/s 2023-09-10T08:46:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:46:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34045.833236 secs 2023-09-10T08:46:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34046.188681 secs 2023-09-10T08:46:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34045.833236 secs 2023-09-10T08:46:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34046.188681 secs 2023-09-10T08:46:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34045.833236 secs 2023-09-10T08:46:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34046.188681 secs 2023-09-10T08:46:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:55 smithi170 ceph-mon[102832]: pgmap v18316: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 432 KiB/s wr, 13 op/s 2023-09-10T08:46:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:55 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:46:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:55 smithi082 ceph-mon[98694]: pgmap v18316: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 432 KiB/s wr, 13 op/s 2023-09-10T08:46:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:55 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:46:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:55 smithi098 ceph-mon[103124]: pgmap v18316: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 432 KiB/s wr, 13 op/s 2023-09-10T08:46:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:55 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:46:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:56 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:46:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:46:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:56 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:46:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:46:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:56 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:46:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:46:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:46:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:57 smithi170 ceph-mon[102832]: pgmap v18317: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 480 KiB/s wr, 19 op/s 2023-09-10T08:46:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:57 smithi082 ceph-mon[98694]: pgmap v18317: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 480 KiB/s wr, 19 op/s 2023-09-10T08:46:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:57 smithi098 ceph-mon[103124]: pgmap v18317: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 480 KiB/s wr, 19 op/s 2023-09-10T08:46:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34050.833313 secs 2023-09-10T08:46:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:46:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34051.188746 secs 2023-09-10T08:46:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34050.833313 secs 2023-09-10T08:46:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:46:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34051.188746 secs 2023-09-10T08:46:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34050.833313 secs 2023-09-10T08:46:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:46:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34051.188746 secs 2023-09-10T08:47:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:00 smithi098 ceph-mon[103124]: pgmap v18318: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 477 KiB/s wr, 13 op/s 2023-09-10T08:47:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:00 smithi170 ceph-mon[102832]: pgmap v18318: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 477 KiB/s wr, 13 op/s 2023-09-10T08:47:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:00 smithi082 ceph-mon[98694]: pgmap v18318: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 477 KiB/s wr, 13 op/s 2023-09-10T08:47:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:02 smithi098 ceph-mon[103124]: pgmap v18319: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 718 KiB/s wr, 19 op/s 2023-09-10T08:47:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:02 smithi170 ceph-mon[102832]: pgmap v18319: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 718 KiB/s wr, 19 op/s 2023-09-10T08:47:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:02 smithi082 ceph-mon[98694]: pgmap v18319: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 718 KiB/s wr, 19 op/s 2023-09-10T08:47:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34054.583722 secs 2023-09-10T08:47:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34054.583722 secs 2023-09-10T08:47:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34054.583722 secs 2023-09-10T08:47:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:04 smithi098 ceph-mon[103124]: pgmap v18320: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 289 KiB/s wr, 12 op/s 2023-09-10T08:47:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:47:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:47:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34055.833372 secs 2023-09-10T08:47:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34056.188816 secs 2023-09-10T08:47:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:04 smithi170 ceph-mon[102832]: pgmap v18320: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 289 KiB/s wr, 12 op/s 2023-09-10T08:47:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:47:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:47:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34055.833372 secs 2023-09-10T08:47:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34056.188816 secs 2023-09-10T08:47:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:04 smithi082 ceph-mon[98694]: pgmap v18320: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 289 KiB/s wr, 12 op/s 2023-09-10T08:47:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:47:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:47:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34055.833372 secs 2023-09-10T08:47:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34056.188816 secs 2023-09-10T08:47:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:05 smithi098 ceph-mon[103124]: pgmap v18321: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 289 KiB/s wr, 12 op/s 2023-09-10T08:47:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:05 smithi170 ceph-mon[102832]: pgmap v18321: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 289 KiB/s wr, 12 op/s 2023-09-10T08:47:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:05 smithi082 ceph-mon[98694]: pgmap v18321: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 289 KiB/s wr, 12 op/s 2023-09-10T08:47:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:47:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:47:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:47:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:07 smithi098 ceph-mon[103124]: pgmap v18322: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 597 KiB/s wr, 33 op/s 2023-09-10T08:47:07.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:07 smithi170 ceph-mon[102832]: pgmap v18322: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 597 KiB/s wr, 33 op/s 2023-09-10T08:47:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:07 smithi082 ceph-mon[98694]: pgmap v18322: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 597 KiB/s wr, 33 op/s 2023-09-10T08:47:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34059.583803 secs 2023-09-10T08:47:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34059.583803 secs 2023-09-10T08:47:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34059.583803 secs 2023-09-10T08:47:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:09 smithi170 ceph-mon[102832]: osdmap e1213: 12 total, 12 up, 12 in 2023-09-10T08:47:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:09 smithi170 ceph-mon[102832]: pgmap v18324: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 659 KiB/s wr, 33 op/s 2023-09-10T08:47:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34060.833429 secs 2023-09-10T08:47:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34061.188879 secs 2023-09-10T08:47:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:09 smithi082 ceph-mon[98694]: osdmap e1213: 12 total, 12 up, 12 in 2023-09-10T08:47:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:09 smithi082 ceph-mon[98694]: pgmap v18324: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 659 KiB/s wr, 33 op/s 2023-09-10T08:47:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34060.833429 secs 2023-09-10T08:47:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34061.188879 secs 2023-09-10T08:47:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:09 smithi098 ceph-mon[103124]: osdmap e1213: 12 total, 12 up, 12 in 2023-09-10T08:47:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:09 smithi098 ceph-mon[103124]: pgmap v18324: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 659 KiB/s wr, 33 op/s 2023-09-10T08:47:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34060.833429 secs 2023-09-10T08:47:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34061.188879 secs 2023-09-10T08:47:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:11 smithi098 ceph-mon[103124]: pgmap v18325: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 517 KiB/s wr, 32 op/s 2023-09-10T08:47:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:11 smithi170 ceph-mon[102832]: pgmap v18325: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 517 KiB/s wr, 32 op/s 2023-09-10T08:47:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:11 smithi082 ceph-mon[98694]: pgmap v18325: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 517 KiB/s wr, 32 op/s 2023-09-10T08:47:13.503 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:13 smithi170 ceph-mon[102832]: pgmap v18326: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 517 KiB/s wr, 32 op/s 2023-09-10T08:47:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:13 smithi098 ceph-mon[103124]: pgmap v18326: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 517 KiB/s wr, 32 op/s 2023-09-10T08:47:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34064.583863 secs 2023-09-10T08:47:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:13 smithi098 ceph-mon[103124]: osdmap e1214: 12 total, 12 up, 12 in 2023-09-10T08:47:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34064.583863 secs 2023-09-10T08:47:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:13 smithi170 ceph-mon[102832]: osdmap e1214: 12 total, 12 up, 12 in 2023-09-10T08:47:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:13 smithi082 ceph-mon[98694]: pgmap v18326: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 517 KiB/s wr, 32 op/s 2023-09-10T08:47:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34064.583863 secs 2023-09-10T08:47:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:13 smithi082 ceph-mon[98694]: osdmap e1214: 12 total, 12 up, 12 in 2023-09-10T08:47:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34065.833497 secs 2023-09-10T08:47:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34066.188945 secs 2023-09-10T08:47:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34065.833497 secs 2023-09-10T08:47:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34066.188945 secs 2023-09-10T08:47:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34065.833497 secs 2023-09-10T08:47:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34066.188945 secs 2023-09-10T08:47:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:15 smithi170 ceph-mon[102832]: pgmap v18328: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 183 KiB/s wr, 9 op/s 2023-09-10T08:47:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:15 smithi082 ceph-mon[98694]: pgmap v18328: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 183 KiB/s wr, 9 op/s 2023-09-10T08:47:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:15 smithi098 ceph-mon[103124]: pgmap v18328: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 183 KiB/s wr, 9 op/s 2023-09-10T08:47:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:47:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:47:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:47:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:17 smithi170 ceph-mon[102832]: pgmap v18329: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 217 KiB/s wr, 18 op/s 2023-09-10T08:47:18.223 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:17 smithi098 ceph-mon[103124]: pgmap v18329: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 217 KiB/s wr, 18 op/s 2023-09-10T08:47:18.224 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:17 smithi082 ceph-mon[98694]: pgmap v18329: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 217 KiB/s wr, 18 op/s 2023-09-10T08:47:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:47:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34069.583966 secs 2023-09-10T08:47:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:47:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34069.583966 secs 2023-09-10T08:47:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:47:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34069.583966 secs 2023-09-10T08:47:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:19 smithi170 ceph-mon[102832]: pgmap v18330: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 178 KiB/s wr, 14 op/s 2023-09-10T08:47:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34070.833569 secs 2023-09-10T08:47:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34071.189012 secs 2023-09-10T08:47:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:19 smithi082 ceph-mon[98694]: pgmap v18330: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 178 KiB/s wr, 14 op/s 2023-09-10T08:47:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34070.833569 secs 2023-09-10T08:47:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34071.189012 secs 2023-09-10T08:47:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:19 smithi098 ceph-mon[103124]: pgmap v18330: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 178 KiB/s wr, 14 op/s 2023-09-10T08:47:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34070.833569 secs 2023-09-10T08:47:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34071.189012 secs 2023-09-10T08:47:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:21 smithi170 ceph-mon[102832]: pgmap v18331: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:47:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:21 smithi082 ceph-mon[98694]: pgmap v18331: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:47:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:21 smithi098 ceph-mon[103124]: pgmap v18331: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:47:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:23 smithi170 ceph-mon[102832]: pgmap v18332: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:47:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34074.584024 secs 2023-09-10T08:47:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:23 smithi082 ceph-mon[98694]: pgmap v18332: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:47:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34074.584024 secs 2023-09-10T08:47:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:23 smithi098 ceph-mon[103124]: pgmap v18332: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2023-09-10T08:47:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34074.584024 secs 2023-09-10T08:47:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34075.833629 secs 2023-09-10T08:47:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34076.189074 secs 2023-09-10T08:47:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34075.833629 secs 2023-09-10T08:47:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34076.189074 secs 2023-09-10T08:47:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34075.833629 secs 2023-09-10T08:47:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34076.189074 secs 2023-09-10T08:47:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:25 smithi170 ceph-mon[102832]: pgmap v18333: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 12 op/s 2023-09-10T08:47:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:25 smithi082 ceph-mon[98694]: pgmap v18333: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 12 op/s 2023-09-10T08:47:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:25 smithi098 ceph-mon[103124]: pgmap v18333: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 12 op/s 2023-09-10T08:47:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:27 smithi170 ceph-mon[102832]: pgmap v18334: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 149 KiB/s wr, 17 op/s 2023-09-10T08:47:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:27 smithi082 ceph-mon[98694]: pgmap v18334: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 149 KiB/s wr, 17 op/s 2023-09-10T08:47:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:27 smithi098 ceph-mon[103124]: pgmap v18334: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 149 KiB/s wr, 17 op/s 2023-09-10T08:47:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34079.584158 secs 2023-09-10T08:47:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34079.584158 secs 2023-09-10T08:47:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34079.584158 secs 2023-09-10T08:47:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:29 smithi170 ceph-mon[102832]: pgmap v18335: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T08:47:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34080.833713 secs 2023-09-10T08:47:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34081.189140 secs 2023-09-10T08:47:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:29 smithi082 ceph-mon[98694]: pgmap v18335: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T08:47:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34080.833713 secs 2023-09-10T08:47:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34081.189140 secs 2023-09-10T08:47:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:29 smithi098 ceph-mon[103124]: pgmap v18335: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T08:47:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34080.833713 secs 2023-09-10T08:47:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34081.189140 secs 2023-09-10T08:47:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:31 smithi170 ceph-mon[102832]: pgmap v18336: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 123 KiB/s wr, 16 op/s 2023-09-10T08:47:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:31 smithi082 ceph-mon[98694]: pgmap v18336: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 123 KiB/s wr, 16 op/s 2023-09-10T08:47:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:31 smithi098 ceph-mon[103124]: pgmap v18336: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 123 KiB/s wr, 16 op/s 2023-09-10T08:47:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:34 smithi098 ceph-mon[103124]: pgmap v18337: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T08:47:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34085.833769 secs 2023-09-10T08:47:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34086.189204 secs 2023-09-10T08:47:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:34 smithi170 ceph-mon[102832]: pgmap v18337: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T08:47:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34085.833769 secs 2023-09-10T08:47:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34086.189204 secs 2023-09-10T08:47:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:34 smithi082 ceph-mon[98694]: pgmap v18337: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T08:47:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34085.833769 secs 2023-09-10T08:47:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34086.189204 secs 2023-09-10T08:47:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:47:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:47:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:47:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:36 smithi098 ceph-mon[103124]: pgmap v18338: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T08:47:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:47:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:47:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:36 smithi170 ceph-mon[102832]: pgmap v18338: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T08:47:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:47:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:47:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:36 smithi082 ceph-mon[98694]: pgmap v18338: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T08:47:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:47:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:47:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:38 smithi098 ceph-mon[103124]: pgmap v18339: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 304 KiB/s wr, 16 op/s 2023-09-10T08:47:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:47:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:38 smithi170 ceph-mon[102832]: pgmap v18339: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 304 KiB/s wr, 16 op/s 2023-09-10T08:47:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:47:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:38 smithi082 ceph-mon[98694]: pgmap v18339: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 304 KiB/s wr, 16 op/s 2023-09-10T08:47:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:47:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34090.833857 secs 2023-09-10T08:47:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34091.189266 secs 2023-09-10T08:47:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34090.833857 secs 2023-09-10T08:47:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34091.189266 secs 2023-09-10T08:47:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34090.833857 secs 2023-09-10T08:47:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34091.189266 secs 2023-09-10T08:47:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:40 smithi098 ceph-mon[103124]: pgmap v18340: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 182 KiB/s wr, 11 op/s 2023-09-10T08:47:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:47:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:40 smithi170 ceph-mon[102832]: pgmap v18340: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 182 KiB/s wr, 11 op/s 2023-09-10T08:47:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:47:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:40 smithi082 ceph-mon[98694]: pgmap v18340: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 182 KiB/s wr, 11 op/s 2023-09-10T08:47:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:47:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:41 smithi098 ceph-mon[103124]: pgmap v18341: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 534 KiB/s wr, 18 op/s 2023-09-10T08:47:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:41 smithi170 ceph-mon[102832]: pgmap v18341: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 534 KiB/s wr, 18 op/s 2023-09-10T08:47:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:41 smithi082 ceph-mon[98694]: pgmap v18341: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 534 KiB/s wr, 18 op/s 2023-09-10T08:47:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:44 smithi098 ceph-mon[103124]: pgmap v18342: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 534 KiB/s wr, 13 op/s 2023-09-10T08:47:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:47:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34095.833917 secs 2023-09-10T08:47:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34096.189336 secs 2023-09-10T08:47:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:47:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:47:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:47:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:44 smithi170 ceph-mon[102832]: pgmap v18342: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 534 KiB/s wr, 13 op/s 2023-09-10T08:47:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:47:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34095.833917 secs 2023-09-10T08:47:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34096.189336 secs 2023-09-10T08:47:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:47:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:47:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:47:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:44 smithi082 ceph-mon[98694]: pgmap v18342: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 534 KiB/s wr, 13 op/s 2023-09-10T08:47:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:47:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34095.833917 secs 2023-09-10T08:47:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34096.189336 secs 2023-09-10T08:47:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:47:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:47:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:47:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:46 smithi098 ceph-mon[103124]: pgmap v18343: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 534 KiB/s wr, 13 op/s 2023-09-10T08:47:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:46 smithi170 ceph-mon[102832]: pgmap v18343: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 534 KiB/s wr, 13 op/s 2023-09-10T08:47:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:46 smithi082 ceph-mon[98694]: pgmap v18343: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 534 KiB/s wr, 13 op/s 2023-09-10T08:47:48.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:48 smithi098 ceph-mon[103124]: pgmap v18344: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 576 KiB/s wr, 19 op/s 2023-09-10T08:47:48.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34099.584394 secs 2023-09-10T08:47:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:48 smithi170 ceph-mon[102832]: pgmap v18344: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 576 KiB/s wr, 19 op/s 2023-09-10T08:47:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34099.584394 secs 2023-09-10T08:47:48.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:48 smithi082 ceph-mon[98694]: pgmap v18344: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 576 KiB/s wr, 19 op/s 2023-09-10T08:47:48.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34099.584394 secs 2023-09-10T08:47:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34100.833976 secs 2023-09-10T08:47:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34101.189400 secs 2023-09-10T08:47:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34100.833976 secs 2023-09-10T08:47:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34101.189400 secs 2023-09-10T08:47:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34100.833976 secs 2023-09-10T08:47:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34101.189400 secs 2023-09-10T08:47:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:50 smithi098 ceph-mon[103124]: pgmap v18345: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 394 KiB/s wr, 13 op/s 2023-09-10T08:47:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:47:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:50 smithi170 ceph-mon[102832]: pgmap v18345: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 394 KiB/s wr, 13 op/s 2023-09-10T08:47:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:47:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:50 smithi082 ceph-mon[98694]: pgmap v18345: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 394 KiB/s wr, 13 op/s 2023-09-10T08:47:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:47:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:51 smithi098 ceph-mon[103124]: pgmap v18346: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 20 op/s 2023-09-10T08:47:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:51 smithi170 ceph-mon[102832]: pgmap v18346: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 20 op/s 2023-09-10T08:47:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:51 smithi082 ceph-mon[98694]: pgmap v18346: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 20 op/s 2023-09-10T08:47:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:47:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:47:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:47:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:47:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:47:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:47:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:53 smithi098 ceph-mon[103124]: pgmap v18347: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 945 KiB/s wr, 13 op/s 2023-09-10T08:47:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34104.584474 secs 2023-09-10T08:47:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:53 smithi170 ceph-mon[102832]: pgmap v18347: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 945 KiB/s wr, 13 op/s 2023-09-10T08:47:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34104.584474 secs 2023-09-10T08:47:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:53 smithi082 ceph-mon[98694]: pgmap v18347: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 945 KiB/s wr, 13 op/s 2023-09-10T08:47:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34104.584474 secs 2023-09-10T08:47:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:47:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34105.834034 secs 2023-09-10T08:47:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34106.189464 secs 2023-09-10T08:47:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:47:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34105.834034 secs 2023-09-10T08:47:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34106.189464 secs 2023-09-10T08:47:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:47:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34105.834034 secs 2023-09-10T08:47:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34106.189464 secs 2023-09-10T08:47:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:55 smithi170 ceph-mon[102832]: pgmap v18348: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 945 KiB/s wr, 13 op/s 2023-09-10T08:47:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:55 smithi082 ceph-mon[98694]: pgmap v18348: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 945 KiB/s wr, 13 op/s 2023-09-10T08:47:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:55 smithi098 ceph-mon[103124]: pgmap v18348: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 945 KiB/s wr, 13 op/s 2023-09-10T08:47:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:47:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:47:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:47:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:57 smithi170 ceph-mon[102832]: pgmap v18349: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 20 op/s 2023-09-10T08:47:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:57 smithi082 ceph-mon[98694]: pgmap v18349: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 20 op/s 2023-09-10T08:47:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:57 smithi098 ceph-mon[103124]: pgmap v18349: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 20 op/s 2023-09-10T08:47:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:47:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34109.584536 secs 2023-09-10T08:47:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:47:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34109.584536 secs 2023-09-10T08:47:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:47:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:47:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34109.584536 secs 2023-09-10T08:47:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:59 smithi170 ceph-mon[102832]: pgmap v18350: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 14 op/s 2023-09-10T08:47:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34110.834093 secs 2023-09-10T08:47:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:47:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34111.189492 secs 2023-09-10T08:47:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:59 smithi082 ceph-mon[98694]: pgmap v18350: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 14 op/s 2023-09-10T08:47:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34110.834093 secs 2023-09-10T08:47:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:47:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34111.189492 secs 2023-09-10T08:47:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:59 smithi098 ceph-mon[103124]: pgmap v18350: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 14 op/s 2023-09-10T08:47:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34110.834093 secs 2023-09-10T08:47:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:47:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34111.189492 secs 2023-09-10T08:48:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:48:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:48:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:48:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:48:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:48:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:48:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:01 smithi170 ceph-mon[102832]: pgmap v18351: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 35 op/s 2023-09-10T08:48:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:01 smithi082 ceph-mon[98694]: pgmap v18351: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 35 op/s 2023-09-10T08:48:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:01 smithi098 ceph-mon[103124]: pgmap v18351: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 35 op/s 2023-09-10T08:48:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:03 smithi170 ceph-mon[102832]: pgmap v18352: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 459 KiB/s wr, 28 op/s 2023-09-10T08:48:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34114.584597 secs 2023-09-10T08:48:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:03 smithi082 ceph-mon[98694]: pgmap v18352: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 459 KiB/s wr, 28 op/s 2023-09-10T08:48:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34114.584597 secs 2023-09-10T08:48:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:03 smithi098 ceph-mon[103124]: pgmap v18352: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 459 KiB/s wr, 28 op/s 2023-09-10T08:48:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34114.584597 secs 2023-09-10T08:48:04.525 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34115.834152 secs 2023-09-10T08:48:04.526 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34116.189559 secs 2023-09-10T08:48:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34115.834152 secs 2023-09-10T08:48:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34116.189559 secs 2023-09-10T08:48:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34115.834152 secs 2023-09-10T08:48:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34116.189559 secs 2023-09-10T08:48:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:05 smithi170 ceph-mon[102832]: pgmap v18353: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 459 KiB/s wr, 28 op/s 2023-09-10T08:48:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:05 smithi082 ceph-mon[98694]: pgmap v18353: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 459 KiB/s wr, 28 op/s 2023-09-10T08:48:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:05 smithi098 ceph-mon[103124]: pgmap v18353: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 459 KiB/s wr, 28 op/s 2023-09-10T08:48:07.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:07 smithi170 ceph-mon[102832]: pgmap v18354: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 577 KiB/s wr, 34 op/s 2023-09-10T08:48:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:07 smithi082 ceph-mon[98694]: pgmap v18354: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 577 KiB/s wr, 34 op/s 2023-09-10T08:48:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:07 smithi098 ceph-mon[103124]: pgmap v18354: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 577 KiB/s wr, 34 op/s 2023-09-10T08:48:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34119.584654 secs 2023-09-10T08:48:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34119.584654 secs 2023-09-10T08:48:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34119.584654 secs 2023-09-10T08:48:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:09 smithi170 ceph-mon[102832]: pgmap v18355: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 406 KiB/s wr, 27 op/s 2023-09-10T08:48:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:09 smithi170 ceph-mon[102832]: osdmap e1215: 12 total, 12 up, 12 in 2023-09-10T08:48:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34120.834212 secs 2023-09-10T08:48:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34121.189621 secs 2023-09-10T08:48:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:09 smithi082 ceph-mon[98694]: pgmap v18355: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 406 KiB/s wr, 27 op/s 2023-09-10T08:48:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:09 smithi082 ceph-mon[98694]: osdmap e1215: 12 total, 12 up, 12 in 2023-09-10T08:48:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34120.834212 secs 2023-09-10T08:48:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34121.189621 secs 2023-09-10T08:48:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:09 smithi098 ceph-mon[103124]: pgmap v18355: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 406 KiB/s wr, 27 op/s 2023-09-10T08:48:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:09 smithi098 ceph-mon[103124]: osdmap e1215: 12 total, 12 up, 12 in 2023-09-10T08:48:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34120.834212 secs 2023-09-10T08:48:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34121.189621 secs 2023-09-10T08:48:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:11 smithi170 ceph-mon[102832]: pgmap v18357: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 172 KiB/s wr, 13 op/s 2023-09-10T08:48:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:11 smithi082 ceph-mon[98694]: pgmap v18357: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 172 KiB/s wr, 13 op/s 2023-09-10T08:48:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:11 smithi098 ceph-mon[103124]: pgmap v18357: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 172 KiB/s wr, 13 op/s 2023-09-10T08:48:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:13 smithi170 ceph-mon[102832]: pgmap v18358: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 172 KiB/s wr, 13 op/s 2023-09-10T08:48:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34124.584717 secs 2023-09-10T08:48:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:13 smithi170 ceph-mon[102832]: osdmap e1216: 12 total, 12 up, 12 in 2023-09-10T08:48:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:13 smithi082 ceph-mon[98694]: pgmap v18358: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 172 KiB/s wr, 13 op/s 2023-09-10T08:48:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34124.584717 secs 2023-09-10T08:48:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:13 smithi082 ceph-mon[98694]: osdmap e1216: 12 total, 12 up, 12 in 2023-09-10T08:48:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:13 smithi098 ceph-mon[103124]: pgmap v18358: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 172 KiB/s wr, 13 op/s 2023-09-10T08:48:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34124.584717 secs 2023-09-10T08:48:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:13 smithi098 ceph-mon[103124]: osdmap e1216: 12 total, 12 up, 12 in 2023-09-10T08:48:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34125.834289 secs 2023-09-10T08:48:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34126.189690 secs 2023-09-10T08:48:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34125.834289 secs 2023-09-10T08:48:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34126.189690 secs 2023-09-10T08:48:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34125.834289 secs 2023-09-10T08:48:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34126.189690 secs 2023-09-10T08:48:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:15 smithi170 ceph-mon[102832]: pgmap v18360: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 8 op/s 2023-09-10T08:48:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:15 smithi082 ceph-mon[98694]: pgmap v18360: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 8 op/s 2023-09-10T08:48:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:15 smithi098 ceph-mon[103124]: pgmap v18360: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 8 op/s 2023-09-10T08:48:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:17 smithi170 ceph-mon[102832]: pgmap v18361: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 16 op/s 2023-09-10T08:48:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:17 smithi082 ceph-mon[98694]: pgmap v18361: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 16 op/s 2023-09-10T08:48:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:17 smithi098 ceph-mon[103124]: pgmap v18361: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 16 op/s 2023-09-10T08:48:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34129.584785 secs 2023-09-10T08:48:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34129.584785 secs 2023-09-10T08:48:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34129.584785 secs 2023-09-10T08:48:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:19 smithi170 ceph-mon[102832]: pgmap v18362: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 13 op/s 2023-09-10T08:48:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34130.834349 secs 2023-09-10T08:48:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34131.189755 secs 2023-09-10T08:48:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:19 smithi082 ceph-mon[98694]: pgmap v18362: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 13 op/s 2023-09-10T08:48:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34130.834349 secs 2023-09-10T08:48:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34131.189755 secs 2023-09-10T08:48:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:19 smithi098 ceph-mon[103124]: pgmap v18362: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 63 KiB/s wr, 13 op/s 2023-09-10T08:48:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34130.834349 secs 2023-09-10T08:48:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34131.189755 secs 2023-09-10T08:48:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:21 smithi170 ceph-mon[102832]: pgmap v18363: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T08:48:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:21 smithi082 ceph-mon[98694]: pgmap v18363: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T08:48:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:21 smithi098 ceph-mon[103124]: pgmap v18363: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T08:48:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:23 smithi170 ceph-mon[102832]: pgmap v18364: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T08:48:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34134.584844 secs 2023-09-10T08:48:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:23 smithi082 ceph-mon[98694]: pgmap v18364: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T08:48:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34134.584844 secs 2023-09-10T08:48:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:23 smithi098 ceph-mon[103124]: pgmap v18364: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T08:48:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34134.584844 secs 2023-09-10T08:48:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34135.834441 secs 2023-09-10T08:48:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34136.189820 secs 2023-09-10T08:48:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34135.834441 secs 2023-09-10T08:48:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34136.189820 secs 2023-09-10T08:48:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34135.834441 secs 2023-09-10T08:48:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34136.189820 secs 2023-09-10T08:48:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:25 smithi170 ceph-mon[102832]: pgmap v18365: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T08:48:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:25 smithi082 ceph-mon[98694]: pgmap v18365: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T08:48:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:25 smithi098 ceph-mon[103124]: pgmap v18365: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T08:48:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:27 smithi170 ceph-mon[102832]: pgmap v18366: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 16 op/s 2023-09-10T08:48:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:27 smithi082 ceph-mon[98694]: pgmap v18366: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 16 op/s 2023-09-10T08:48:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:27 smithi098 ceph-mon[103124]: pgmap v18366: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 16 op/s 2023-09-10T08:48:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34139.584920 secs 2023-09-10T08:48:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34139.584920 secs 2023-09-10T08:48:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34139.584920 secs 2023-09-10T08:48:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:29 smithi170 ceph-mon[102832]: pgmap v18367: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T08:48:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34140.834508 secs 2023-09-10T08:48:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34141.189884 secs 2023-09-10T08:48:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:29 smithi082 ceph-mon[98694]: pgmap v18367: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T08:48:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34140.834508 secs 2023-09-10T08:48:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34141.189884 secs 2023-09-10T08:48:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:29 smithi098 ceph-mon[103124]: pgmap v18367: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T08:48:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34140.834508 secs 2023-09-10T08:48:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34141.189884 secs 2023-09-10T08:48:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:31 smithi170 ceph-mon[102832]: pgmap v18368: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 16 op/s 2023-09-10T08:48:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:31 smithi082 ceph-mon[98694]: pgmap v18368: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 16 op/s 2023-09-10T08:48:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:31 smithi098 ceph-mon[103124]: pgmap v18368: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 16 op/s 2023-09-10T08:48:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:34 smithi098 ceph-mon[103124]: pgmap v18369: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T08:48:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34145.834578 secs 2023-09-10T08:48:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:48:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:48:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34146.189947 secs 2023-09-10T08:48:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:34 smithi170 ceph-mon[102832]: pgmap v18369: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T08:48:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34145.834578 secs 2023-09-10T08:48:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:48:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:48:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34146.189947 secs 2023-09-10T08:48:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:34 smithi082 ceph-mon[98694]: pgmap v18369: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T08:48:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34145.834578 secs 2023-09-10T08:48:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:48:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:48:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34146.189947 secs 2023-09-10T08:48:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:35 smithi098 ceph-mon[103124]: pgmap v18370: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T08:48:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:48:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:35 smithi170 ceph-mon[102832]: pgmap v18370: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T08:48:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:48:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:35 smithi082 ceph-mon[98694]: pgmap v18370: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 123 KiB/s wr, 11 op/s 2023-09-10T08:48:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:48:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:48:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:48:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:48:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:48:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:48:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:48:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:48:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:48:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:48:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:37 smithi098 ceph-mon[103124]: pgmap v18371: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 352 KiB/s wr, 17 op/s 2023-09-10T08:48:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:37 smithi170 ceph-mon[102832]: pgmap v18371: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 352 KiB/s wr, 17 op/s 2023-09-10T08:48:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:37 smithi082 ceph-mon[98694]: pgmap v18371: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 352 KiB/s wr, 17 op/s 2023-09-10T08:48:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34150.834636 secs 2023-09-10T08:48:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34151.190010 secs 2023-09-10T08:48:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34150.834636 secs 2023-09-10T08:48:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34151.190010 secs 2023-09-10T08:48:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34150.834636 secs 2023-09-10T08:48:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34151.190010 secs 2023-09-10T08:48:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:40 smithi098 ceph-mon[103124]: pgmap v18372: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 229 KiB/s wr, 11 op/s 2023-09-10T08:48:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:40 smithi170 ceph-mon[102832]: pgmap v18372: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 229 KiB/s wr, 11 op/s 2023-09-10T08:48:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:40 smithi082 ceph-mon[98694]: pgmap v18372: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 229 KiB/s wr, 11 op/s 2023-09-10T08:48:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:41 smithi098 ceph-mon[103124]: pgmap v18373: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 410 KiB/s wr, 18 op/s 2023-09-10T08:48:41.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:41 smithi170 ceph-mon[102832]: pgmap v18373: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 410 KiB/s wr, 18 op/s 2023-09-10T08:48:41.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:41 smithi082 ceph-mon[98694]: pgmap v18373: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 410 KiB/s wr, 18 op/s 2023-09-10T08:48:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:44 smithi098 ceph-mon[103124]: pgmap v18374: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 410 KiB/s wr, 12 op/s 2023-09-10T08:48:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:48:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:48:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34155.834706 secs 2023-09-10T08:48:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34156.190122 secs 2023-09-10T08:48:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:44 smithi170 ceph-mon[102832]: pgmap v18374: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 410 KiB/s wr, 12 op/s 2023-09-10T08:48:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:48:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:48:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34155.834706 secs 2023-09-10T08:48:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34156.190122 secs 2023-09-10T08:48:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:44 smithi082 ceph-mon[98694]: pgmap v18374: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 410 KiB/s wr, 12 op/s 2023-09-10T08:48:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:48:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:48:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34155.834706 secs 2023-09-10T08:48:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34156.190122 secs 2023-09-10T08:48:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:48:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:45 smithi098 ceph-mon[103124]: pgmap v18375: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 410 KiB/s wr, 12 op/s 2023-09-10T08:48:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:48:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:48:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:48:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:45 smithi170 ceph-mon[102832]: pgmap v18375: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 410 KiB/s wr, 12 op/s 2023-09-10T08:48:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:48:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:48:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:48:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:45 smithi082 ceph-mon[98694]: pgmap v18375: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 410 KiB/s wr, 12 op/s 2023-09-10T08:48:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:48:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:48:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:48:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:48:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:48:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:48:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:48:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:48:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:48:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:48:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:48:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:48:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:48:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:48:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:48:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:48:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:48:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:47 smithi098 ceph-mon[103124]: pgmap v18376: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 596 KiB/s wr, 19 op/s 2023-09-10T08:48:47.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:47 smithi170 ceph-mon[102832]: pgmap v18376: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 596 KiB/s wr, 19 op/s 2023-09-10T08:48:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:47 smithi082 ceph-mon[98694]: pgmap v18376: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 596 KiB/s wr, 19 op/s 2023-09-10T08:48:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34159.585217 secs 2023-09-10T08:48:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34159.585217 secs 2023-09-10T08:48:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34159.585217 secs 2023-09-10T08:48:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:49 smithi170 ceph-mon[102832]: pgmap v18377: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 368 KiB/s wr, 13 op/s 2023-09-10T08:48:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34160.834762 secs 2023-09-10T08:48:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34161.190188 secs 2023-09-10T08:48:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:49 smithi082 ceph-mon[98694]: pgmap v18377: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 368 KiB/s wr, 13 op/s 2023-09-10T08:48:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34160.834762 secs 2023-09-10T08:48:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34161.190188 secs 2023-09-10T08:48:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:49 smithi098 ceph-mon[103124]: pgmap v18377: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 368 KiB/s wr, 13 op/s 2023-09-10T08:48:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34160.834762 secs 2023-09-10T08:48:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34161.190188 secs 2023-09-10T08:48:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:48:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:48:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:48:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:48:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:48:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:48:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:51 smithi170 ceph-mon[102832]: pgmap v18378: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 763 KiB/s wr, 21 op/s 2023-09-10T08:48:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:51 smithi082 ceph-mon[98694]: pgmap v18378: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 763 KiB/s wr, 21 op/s 2023-09-10T08:48:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:51 smithi098 ceph-mon[103124]: pgmap v18378: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 763 KiB/s wr, 21 op/s 2023-09-10T08:48:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:48:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:48:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:48:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:48:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:48:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:48:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:48:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:48:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:48:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:53 smithi170 ceph-mon[102832]: pgmap v18379: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 582 KiB/s wr, 14 op/s 2023-09-10T08:48:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:53 smithi082 ceph-mon[98694]: pgmap v18379: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 582 KiB/s wr, 14 op/s 2023-09-10T08:48:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:53 smithi098 ceph-mon[103124]: pgmap v18379: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 582 KiB/s wr, 14 op/s 2023-09-10T08:48:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34165.834847 secs 2023-09-10T08:48:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34166.190255 secs 2023-09-10T08:48:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34165.834847 secs 2023-09-10T08:48:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34166.190255 secs 2023-09-10T08:48:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34165.834847 secs 2023-09-10T08:48:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34166.190255 secs 2023-09-10T08:48:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:55 smithi170 ceph-mon[102832]: pgmap v18380: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 582 KiB/s wr, 14 op/s 2023-09-10T08:48:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:55 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:48:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:55 smithi082 ceph-mon[98694]: pgmap v18380: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 582 KiB/s wr, 14 op/s 2023-09-10T08:48:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:55 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:48:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:55 smithi098 ceph-mon[103124]: pgmap v18380: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 582 KiB/s wr, 14 op/s 2023-09-10T08:48:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:55 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:48:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:48:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:48:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:48:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:48:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:48:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:48:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:57 smithi170 ceph-mon[102832]: pgmap v18381: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 585 KiB/s wr, 21 op/s 2023-09-10T08:48:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:57 smithi082 ceph-mon[98694]: pgmap v18381: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 585 KiB/s wr, 21 op/s 2023-09-10T08:48:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:57 smithi098 ceph-mon[103124]: pgmap v18381: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 585 KiB/s wr, 21 op/s 2023-09-10T08:48:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:48:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:48:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:58 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:48:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:48:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:48:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:58 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:48:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:48:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:48:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:58 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:48:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:59 smithi170 ceph-mon[102832]: pgmap v18382: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 399 KiB/s wr, 14 op/s 2023-09-10T08:48:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34170.834907 secs 2023-09-10T08:48:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:48:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34171.190322 secs 2023-09-10T08:48:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:59 smithi082 ceph-mon[98694]: pgmap v18382: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 399 KiB/s wr, 14 op/s 2023-09-10T08:48:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34170.834907 secs 2023-09-10T08:48:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:48:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34171.190322 secs 2023-09-10T08:48:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:59 smithi098 ceph-mon[103124]: pgmap v18382: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 399 KiB/s wr, 14 op/s 2023-09-10T08:48:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34170.834907 secs 2023-09-10T08:48:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:48:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34171.190322 secs 2023-09-10T08:49:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:49:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:49:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:49:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:01 smithi170 ceph-mon[102832]: pgmap v18383: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 878 KiB/s wr, 27 op/s 2023-09-10T08:49:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:01 smithi082 ceph-mon[98694]: pgmap v18383: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 878 KiB/s wr, 27 op/s 2023-09-10T08:49:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:01 smithi098 ceph-mon[103124]: pgmap v18383: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 878 KiB/s wr, 27 op/s 2023-09-10T08:49:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:03 smithi170 ceph-mon[102832]: pgmap v18384: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 483 KiB/s wr, 19 op/s 2023-09-10T08:49:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34174.585475 secs 2023-09-10T08:49:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:03 smithi082 ceph-mon[98694]: pgmap v18384: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 483 KiB/s wr, 19 op/s 2023-09-10T08:49:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34174.585475 secs 2023-09-10T08:49:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:03 smithi098 ceph-mon[103124]: pgmap v18384: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 483 KiB/s wr, 19 op/s 2023-09-10T08:49:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34174.585475 secs 2023-09-10T08:49:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:49:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:49:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:04 smithi170 ceph-mon[102832]: osdmap e1217: 12 total, 12 up, 12 in 2023-09-10T08:49:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34175.834965 secs 2023-09-10T08:49:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34176.190387 secs 2023-09-10T08:49:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:49:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:49:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:04 smithi082 ceph-mon[98694]: osdmap e1217: 12 total, 12 up, 12 in 2023-09-10T08:49:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34175.834965 secs 2023-09-10T08:49:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34176.190387 secs 2023-09-10T08:49:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:49:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:49:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:04 smithi098 ceph-mon[103124]: osdmap e1217: 12 total, 12 up, 12 in 2023-09-10T08:49:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34175.834965 secs 2023-09-10T08:49:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34176.190387 secs 2023-09-10T08:49:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:05 smithi170 ceph-mon[102832]: pgmap v18386: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 580 KiB/s wr, 23 op/s 2023-09-10T08:49:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:05 smithi082 ceph-mon[98694]: pgmap v18386: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 580 KiB/s wr, 23 op/s 2023-09-10T08:49:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:05 smithi098 ceph-mon[103124]: pgmap v18386: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 580 KiB/s wr, 23 op/s 2023-09-10T08:49:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:49:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:49:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:49:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:07 smithi170 ceph-mon[102832]: pgmap v18387: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 746 KiB/s wr, 35 op/s 2023-09-10T08:49:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:07 smithi082 ceph-mon[98694]: pgmap v18387: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 746 KiB/s wr, 35 op/s 2023-09-10T08:49:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:07 smithi098 ceph-mon[103124]: pgmap v18387: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 746 KiB/s wr, 35 op/s 2023-09-10T08:49:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34179.585536 secs 2023-09-10T08:49:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:08 smithi170 ceph-mon[102832]: osdmap e1218: 12 total, 12 up, 12 in 2023-09-10T08:49:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34179.585536 secs 2023-09-10T08:49:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:08 smithi082 ceph-mon[98694]: osdmap e1218: 12 total, 12 up, 12 in 2023-09-10T08:49:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34179.585536 secs 2023-09-10T08:49:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:08 smithi098 ceph-mon[103124]: osdmap e1218: 12 total, 12 up, 12 in 2023-09-10T08:49:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:09 smithi170 ceph-mon[102832]: pgmap v18389: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 213 KiB/s wr, 25 op/s 2023-09-10T08:49:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34180.835024 secs 2023-09-10T08:49:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34181.190455 secs 2023-09-10T08:49:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:09 smithi082 ceph-mon[98694]: pgmap v18389: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 213 KiB/s wr, 25 op/s 2023-09-10T08:49:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34180.835024 secs 2023-09-10T08:49:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34181.190455 secs 2023-09-10T08:49:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:09 smithi098 ceph-mon[103124]: pgmap v18389: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 213 KiB/s wr, 25 op/s 2023-09-10T08:49:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34180.835024 secs 2023-09-10T08:49:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34181.190455 secs 2023-09-10T08:49:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:11 smithi170 ceph-mon[102832]: pgmap v18390: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 619 KiB/s wr, 35 op/s 2023-09-10T08:49:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:11 smithi082 ceph-mon[98694]: pgmap v18390: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 619 KiB/s wr, 35 op/s 2023-09-10T08:49:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:11 smithi098 ceph-mon[103124]: pgmap v18390: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 619 KiB/s wr, 35 op/s 2023-09-10T08:49:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:13 smithi170 ceph-mon[102832]: pgmap v18391: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 554 KiB/s wr, 31 op/s 2023-09-10T08:49:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34184.585597 secs 2023-09-10T08:49:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:13 smithi082 ceph-mon[98694]: pgmap v18391: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 554 KiB/s wr, 31 op/s 2023-09-10T08:49:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34184.585597 secs 2023-09-10T08:49:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:13 smithi098 ceph-mon[103124]: pgmap v18391: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 554 KiB/s wr, 31 op/s 2023-09-10T08:49:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34184.585597 secs 2023-09-10T08:49:14.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34185.835109 secs 2023-09-10T08:49:14.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34186.190483 secs 2023-09-10T08:49:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34185.835109 secs 2023-09-10T08:49:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34186.190483 secs 2023-09-10T08:49:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34185.835109 secs 2023-09-10T08:49:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34186.190483 secs 2023-09-10T08:49:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:15 smithi170 ceph-mon[102832]: pgmap v18392: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 495 KiB/s wr, 28 op/s 2023-09-10T08:49:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:15 smithi082 ceph-mon[98694]: pgmap v18392: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 495 KiB/s wr, 28 op/s 2023-09-10T08:49:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:15 smithi098 ceph-mon[103124]: pgmap v18392: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 495 KiB/s wr, 28 op/s 2023-09-10T08:49:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:17 smithi170 ceph-mon[102832]: pgmap v18393: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 325 KiB/s wr, 14 op/s 2023-09-10T08:49:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:17 smithi082 ceph-mon[98694]: pgmap v18393: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 325 KiB/s wr, 14 op/s 2023-09-10T08:49:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:17 smithi098 ceph-mon[103124]: pgmap v18393: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 325 KiB/s wr, 14 op/s 2023-09-10T08:49:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34189.585656 secs 2023-09-10T08:49:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34189.585656 secs 2023-09-10T08:49:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34189.585656 secs 2023-09-10T08:49:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:19 smithi170 ceph-mon[102832]: pgmap v18394: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 318 KiB/s wr, 14 op/s 2023-09-10T08:49:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34190.835166 secs 2023-09-10T08:49:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34191.190547 secs 2023-09-10T08:49:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:19 smithi082 ceph-mon[98694]: pgmap v18394: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 318 KiB/s wr, 14 op/s 2023-09-10T08:49:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34190.835166 secs 2023-09-10T08:49:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34191.190547 secs 2023-09-10T08:49:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:19 smithi098 ceph-mon[103124]: pgmap v18394: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 318 KiB/s wr, 14 op/s 2023-09-10T08:49:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34190.835166 secs 2023-09-10T08:49:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34191.190547 secs 2023-09-10T08:49:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:21 smithi170 ceph-mon[102832]: pgmap v18395: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 272 KiB/s wr, 17 op/s 2023-09-10T08:49:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:21 smithi082 ceph-mon[98694]: pgmap v18395: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 272 KiB/s wr, 17 op/s 2023-09-10T08:49:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:21 smithi098 ceph-mon[103124]: pgmap v18395: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 272 KiB/s wr, 17 op/s 2023-09-10T08:49:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:23 smithi170 ceph-mon[102832]: pgmap v18396: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1023 B/s wr, 10 op/s 2023-09-10T08:49:23.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34194.585736 secs 2023-09-10T08:49:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:23 smithi082 ceph-mon[98694]: pgmap v18396: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1023 B/s wr, 10 op/s 2023-09-10T08:49:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34194.585736 secs 2023-09-10T08:49:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:23 smithi098 ceph-mon[103124]: pgmap v18396: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1023 B/s wr, 10 op/s 2023-09-10T08:49:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34194.585736 secs 2023-09-10T08:49:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34195.835253 secs 2023-09-10T08:49:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34196.190612 secs 2023-09-10T08:49:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34195.835253 secs 2023-09-10T08:49:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34196.190612 secs 2023-09-10T08:49:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34195.835253 secs 2023-09-10T08:49:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34196.190612 secs 2023-09-10T08:49:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:25 smithi170 ceph-mon[102832]: pgmap v18397: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1023 B/s wr, 10 op/s 2023-09-10T08:49:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:25 smithi082 ceph-mon[98694]: pgmap v18397: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1023 B/s wr, 10 op/s 2023-09-10T08:49:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:25 smithi098 ceph-mon[103124]: pgmap v18397: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1023 B/s wr, 10 op/s 2023-09-10T08:49:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:27 smithi170 ceph-mon[102832]: pgmap v18398: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2023-09-10T08:49:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:27 smithi082 ceph-mon[98694]: pgmap v18398: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2023-09-10T08:49:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:27 smithi098 ceph-mon[103124]: pgmap v18398: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2023-09-10T08:49:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34199.585808 secs 2023-09-10T08:49:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34199.585808 secs 2023-09-10T08:49:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34199.585808 secs 2023-09-10T08:49:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:29 smithi170 ceph-mon[102832]: pgmap v18399: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-10T08:49:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34200.835348 secs 2023-09-10T08:49:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34201.190679 secs 2023-09-10T08:49:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:29 smithi082 ceph-mon[98694]: pgmap v18399: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-10T08:49:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34200.835348 secs 2023-09-10T08:49:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34201.190679 secs 2023-09-10T08:49:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:29 smithi098 ceph-mon[103124]: pgmap v18399: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-10T08:49:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34200.835348 secs 2023-09-10T08:49:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34201.190679 secs 2023-09-10T08:49:30.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:49:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:49:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:49:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:31 smithi170 ceph-mon[102832]: pgmap v18400: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 16 op/s 2023-09-10T08:49:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:31 smithi082 ceph-mon[98694]: pgmap v18400: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 16 op/s 2023-09-10T08:49:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:31 smithi098 ceph-mon[103124]: pgmap v18400: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 124 KiB/s wr, 16 op/s 2023-09-10T08:49:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:49:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:49:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:49:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:33 smithi170 ceph-mon[102832]: pgmap v18401: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T08:49:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:33 smithi082 ceph-mon[98694]: pgmap v18401: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T08:49:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:33 smithi098 ceph-mon[103124]: pgmap v18401: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T08:49:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34205.835405 secs 2023-09-10T08:49:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34206.190743 secs 2023-09-10T08:49:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34205.835405 secs 2023-09-10T08:49:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34206.190743 secs 2023-09-10T08:49:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34205.835405 secs 2023-09-10T08:49:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34206.190743 secs 2023-09-10T08:49:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:35 smithi170 ceph-mon[102832]: pgmap v18402: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T08:49:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:49:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:35 smithi082 ceph-mon[98694]: pgmap v18402: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T08:49:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:49:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:35 smithi098 ceph-mon[103124]: pgmap v18402: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 124 KiB/s wr, 11 op/s 2023-09-10T08:49:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:49:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:49:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:49:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:49:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:49:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:37.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:49:37.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:49:37.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:37 smithi170 ceph-mon[102832]: pgmap v18403: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 253 KiB/s wr, 16 op/s 2023-09-10T08:49:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:37 smithi082 ceph-mon[98694]: pgmap v18403: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 253 KiB/s wr, 16 op/s 2023-09-10T08:49:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:37 smithi098 ceph-mon[103124]: pgmap v18403: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 253 KiB/s wr, 16 op/s 2023-09-10T08:49:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:49:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:49:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:39.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:49:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:39 smithi170 ceph-mon[102832]: pgmap v18404: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 253 KiB/s wr, 10 op/s 2023-09-10T08:49:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34210.835464 secs 2023-09-10T08:49:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34211.190809 secs 2023-09-10T08:49:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:39 smithi082 ceph-mon[98694]: pgmap v18404: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 253 KiB/s wr, 10 op/s 2023-09-10T08:49:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34210.835464 secs 2023-09-10T08:49:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34211.190809 secs 2023-09-10T08:49:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:39 smithi098 ceph-mon[103124]: pgmap v18404: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 253 KiB/s wr, 10 op/s 2023-09-10T08:49:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34210.835464 secs 2023-09-10T08:49:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34211.190809 secs 2023-09-10T08:49:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:49:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:49:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:49:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:41 smithi170 ceph-mon[102832]: pgmap v18405: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 894 KiB/s wr, 18 op/s 2023-09-10T08:49:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:41 smithi082 ceph-mon[98694]: pgmap v18405: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 894 KiB/s wr, 18 op/s 2023-09-10T08:49:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:41 smithi098 ceph-mon[103124]: pgmap v18405: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 894 KiB/s wr, 18 op/s 2023-09-10T08:49:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:43 smithi170 ceph-mon[102832]: pgmap v18406: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 771 KiB/s wr, 12 op/s 2023-09-10T08:49:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:43 smithi082 ceph-mon[98694]: pgmap v18406: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 771 KiB/s wr, 12 op/s 2023-09-10T08:49:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:43 smithi098 ceph-mon[103124]: pgmap v18406: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 771 KiB/s wr, 12 op/s 2023-09-10T08:49:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34215.835540 secs 2023-09-10T08:49:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34216.190874 secs 2023-09-10T08:49:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34215.835540 secs 2023-09-10T08:49:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34216.190874 secs 2023-09-10T08:49:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34215.835540 secs 2023-09-10T08:49:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34216.190874 secs 2023-09-10T08:49:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:45 smithi170 ceph-mon[102832]: pgmap v18407: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 771 KiB/s wr, 12 op/s 2023-09-10T08:49:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:49:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:45 smithi082 ceph-mon[98694]: pgmap v18407: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 771 KiB/s wr, 12 op/s 2023-09-10T08:49:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:49:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:45 smithi098 ceph-mon[103124]: pgmap v18407: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 771 KiB/s wr, 12 op/s 2023-09-10T08:49:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:49:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:49:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:49:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:49:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:49:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:49:47.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:49:47.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:49:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:49:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:49:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:49:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:49:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:49:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:49:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:49:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:49:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:49:47.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:49:47.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:49:47.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:49:47.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:49:47.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:49:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:47 smithi170 ceph-mon[102832]: pgmap v18408: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 865 KiB/s wr, 19 op/s 2023-09-10T08:49:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:47 smithi082 ceph-mon[98694]: pgmap v18408: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 865 KiB/s wr, 19 op/s 2023-09-10T08:49:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:47 smithi098 ceph-mon[103124]: pgmap v18408: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 865 KiB/s wr, 19 op/s 2023-09-10T08:49:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34219.586100 secs 2023-09-10T08:49:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34219.586100 secs 2023-09-10T08:49:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34219.586100 secs 2023-09-10T08:49:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:49 smithi170 ceph-mon[102832]: pgmap v18409: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 735 KiB/s wr, 14 op/s 2023-09-10T08:49:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34220.835599 secs 2023-09-10T08:49:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34221.190939 secs 2023-09-10T08:49:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:49 smithi082 ceph-mon[98694]: pgmap v18409: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 735 KiB/s wr, 14 op/s 2023-09-10T08:49:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34220.835599 secs 2023-09-10T08:49:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34221.190939 secs 2023-09-10T08:49:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:49 smithi098 ceph-mon[103124]: pgmap v18409: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 735 KiB/s wr, 14 op/s 2023-09-10T08:49:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34220.835599 secs 2023-09-10T08:49:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34221.190939 secs 2023-09-10T08:49:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:49:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:49:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:49:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:51 smithi170 ceph-mon[102832]: pgmap v18410: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T08:49:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:51 smithi082 ceph-mon[98694]: pgmap v18410: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T08:49:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:51 smithi098 ceph-mon[103124]: pgmap v18410: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T08:49:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:49:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:49:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:49:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:49:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:49:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:49:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:53 smithi170 ceph-mon[102832]: pgmap v18411: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 461 KiB/s wr, 13 op/s 2023-09-10T08:49:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:49:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:53 smithi082 ceph-mon[98694]: pgmap v18411: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 461 KiB/s wr, 13 op/s 2023-09-10T08:49:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:49:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:53 smithi098 ceph-mon[103124]: pgmap v18411: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 461 KiB/s wr, 13 op/s 2023-09-10T08:49:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:49:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34225.835686 secs 2023-09-10T08:49:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34226.191002 secs 2023-09-10T08:49:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34225.835686 secs 2023-09-10T08:49:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34226.191002 secs 2023-09-10T08:49:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34225.835686 secs 2023-09-10T08:49:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34226.191002 secs 2023-09-10T08:49:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:55 smithi170 ceph-mon[102832]: pgmap v18412: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 461 KiB/s wr, 13 op/s 2023-09-10T08:49:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:55 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:49:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:55 smithi082 ceph-mon[98694]: pgmap v18412: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 461 KiB/s wr, 13 op/s 2023-09-10T08:49:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:55 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:49:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:55 smithi098 ceph-mon[103124]: pgmap v18412: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 461 KiB/s wr, 13 op/s 2023-09-10T08:49:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:55 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:49:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:49:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:49:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:49:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:57 smithi170 ceph-mon[102832]: pgmap v18413: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 887 KiB/s wr, 21 op/s 2023-09-10T08:49:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:49:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:57 smithi082 ceph-mon[98694]: pgmap v18413: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 887 KiB/s wr, 21 op/s 2023-09-10T08:49:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:49:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:57 smithi098 ceph-mon[103124]: pgmap v18413: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 887 KiB/s wr, 21 op/s 2023-09-10T08:49:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:49:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:49:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34229.586219 secs 2023-09-10T08:49:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:58 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:49:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34229.586219 secs 2023-09-10T08:49:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:58 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:49:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34229.586219 secs 2023-09-10T08:49:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:58 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:50:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:59 smithi170 ceph-mon[102832]: pgmap v18414: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 793 KiB/s wr, 14 op/s 2023-09-10T08:50:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34230.835744 secs 2023-09-10T08:50:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:49:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34231.191066 secs 2023-09-10T08:50:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:59 smithi082 ceph-mon[98694]: pgmap v18414: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 793 KiB/s wr, 14 op/s 2023-09-10T08:50:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34230.835744 secs 2023-09-10T08:50:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:49:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34231.191066 secs 2023-09-10T08:50:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:59 smithi098 ceph-mon[103124]: pgmap v18414: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 793 KiB/s wr, 14 op/s 2023-09-10T08:50:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34230.835744 secs 2023-09-10T08:50:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:49:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34231.191066 secs 2023-09-10T08:50:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:50:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T08:50:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:50:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T08:50:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:50:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T08:50:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:01 smithi170 ceph-mon[102832]: pgmap v18415: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 890 KiB/s wr, 20 op/s 2023-09-10T08:50:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:01 smithi082 ceph-mon[98694]: pgmap v18415: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 890 KiB/s wr, 20 op/s 2023-09-10T08:50:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:01 smithi098 ceph-mon[103124]: pgmap v18415: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 890 KiB/s wr, 20 op/s 2023-09-10T08:50:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:03 smithi170 ceph-mon[102832]: pgmap v18416: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 523 KiB/s wr, 13 op/s 2023-09-10T08:50:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34234.586302 secs 2023-09-10T08:50:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:03 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:50:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:03 smithi082 ceph-mon[98694]: pgmap v18416: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 523 KiB/s wr, 13 op/s 2023-09-10T08:50:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34234.586302 secs 2023-09-10T08:50:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:03 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:50:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:03 smithi098 ceph-mon[103124]: pgmap v18416: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 523 KiB/s wr, 13 op/s 2023-09-10T08:50:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34234.586302 secs 2023-09-10T08:50:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:03 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:50:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34235.835803 secs 2023-09-10T08:50:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:04 smithi170 ceph-mon[102832]: osdmap e1219: 12 total, 12 up, 12 in 2023-09-10T08:50:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34236.191132 secs 2023-09-10T08:50:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34235.835803 secs 2023-09-10T08:50:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:04 smithi082 ceph-mon[98694]: osdmap e1219: 12 total, 12 up, 12 in 2023-09-10T08:50:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34236.191132 secs 2023-09-10T08:50:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34235.835803 secs 2023-09-10T08:50:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:04 smithi098 ceph-mon[103124]: osdmap e1219: 12 total, 12 up, 12 in 2023-09-10T08:50:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34236.191132 secs 2023-09-10T08:50:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:05 smithi170 ceph-mon[102832]: pgmap v18418: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 627 KiB/s wr, 16 op/s 2023-09-10T08:50:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:05 smithi082 ceph-mon[98694]: pgmap v18418: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 627 KiB/s wr, 16 op/s 2023-09-10T08:50:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:05 smithi098 ceph-mon[103124]: pgmap v18418: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 627 KiB/s wr, 16 op/s 2023-09-10T08:50:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:07 smithi170 ceph-mon[102832]: pgmap v18419: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 489 KiB/s wr, 31 op/s 2023-09-10T08:50:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:07 smithi082 ceph-mon[98694]: pgmap v18419: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 489 KiB/s wr, 31 op/s 2023-09-10T08:50:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:07 smithi098 ceph-mon[103124]: pgmap v18419: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 489 KiB/s wr, 31 op/s 2023-09-10T08:50:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34239.586365 secs 2023-09-10T08:50:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:08 smithi170 ceph-mon[102832]: osdmap e1220: 12 total, 12 up, 12 in 2023-09-10T08:50:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34239.586365 secs 2023-09-10T08:50:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:08 smithi082 ceph-mon[98694]: osdmap e1220: 12 total, 12 up, 12 in 2023-09-10T08:50:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34239.586365 secs 2023-09-10T08:50:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:08 smithi098 ceph-mon[103124]: osdmap e1220: 12 total, 12 up, 12 in 2023-09-10T08:50:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:09 smithi170 ceph-mon[102832]: pgmap v18421: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 466 KiB/s wr, 30 op/s 2023-09-10T08:50:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34240.835864 secs 2023-09-10T08:50:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34241.191198 secs 2023-09-10T08:50:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:09 smithi082 ceph-mon[98694]: pgmap v18421: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 466 KiB/s wr, 30 op/s 2023-09-10T08:50:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34240.835864 secs 2023-09-10T08:50:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34241.191198 secs 2023-09-10T08:50:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:09 smithi098 ceph-mon[103124]: pgmap v18421: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 466 KiB/s wr, 30 op/s 2023-09-10T08:50:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34240.835864 secs 2023-09-10T08:50:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34241.191198 secs 2023-09-10T08:50:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:11 smithi170 ceph-mon[102832]: pgmap v18422: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 650 KiB/s wr, 39 op/s 2023-09-10T08:50:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:11 smithi082 ceph-mon[98694]: pgmap v18422: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 650 KiB/s wr, 39 op/s 2023-09-10T08:50:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:11 smithi098 ceph-mon[103124]: pgmap v18422: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 650 KiB/s wr, 39 op/s 2023-09-10T08:50:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:50:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:50:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:50:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:13 smithi170 ceph-mon[102832]: pgmap v18423: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 604 KiB/s wr, 36 op/s 2023-09-10T08:50:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34244.586426 secs 2023-09-10T08:50:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:50:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:13 smithi082 ceph-mon[98694]: pgmap v18423: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 604 KiB/s wr, 36 op/s 2023-09-10T08:50:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34244.586426 secs 2023-09-10T08:50:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:50:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:13 smithi098 ceph-mon[103124]: pgmap v18423: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 604 KiB/s wr, 36 op/s 2023-09-10T08:50:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34244.586426 secs 2023-09-10T08:50:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:50:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34245.835938 secs 2023-09-10T08:50:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34246.191264 secs 2023-09-10T08:50:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34245.835938 secs 2023-09-10T08:50:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34246.191264 secs 2023-09-10T08:50:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34245.835938 secs 2023-09-10T08:50:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34246.191264 secs 2023-09-10T08:50:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:15 smithi170 ceph-mon[102832]: pgmap v18424: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 520 KiB/s wr, 31 op/s 2023-09-10T08:50:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:15 smithi082 ceph-mon[98694]: pgmap v18424: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 520 KiB/s wr, 31 op/s 2023-09-10T08:50:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:15 smithi098 ceph-mon[103124]: pgmap v18424: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 520 KiB/s wr, 31 op/s 2023-09-10T08:50:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:17 smithi170 ceph-mon[102832]: pgmap v18425: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 14 op/s 2023-09-10T08:50:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:17 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34249.586488 secs 2023-09-10T08:50:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:17 smithi082 ceph-mon[98694]: pgmap v18425: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 14 op/s 2023-09-10T08:50:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:17 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34249.586488 secs 2023-09-10T08:50:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:17 smithi098 ceph-mon[103124]: pgmap v18425: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 14 op/s 2023-09-10T08:50:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:17 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34249.586488 secs 2023-09-10T08:50:19.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:19 smithi170 ceph-mon[102832]: pgmap v18426: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 176 KiB/s wr, 13 op/s 2023-09-10T08:50:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34250.836022 secs 2023-09-10T08:50:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34251.191328 secs 2023-09-10T08:50:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:19 smithi082 ceph-mon[98694]: pgmap v18426: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 176 KiB/s wr, 13 op/s 2023-09-10T08:50:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34250.836022 secs 2023-09-10T08:50:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34251.191328 secs 2023-09-10T08:50:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:19 smithi098 ceph-mon[103124]: pgmap v18426: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 176 KiB/s wr, 13 op/s 2023-09-10T08:50:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34250.836022 secs 2023-09-10T08:50:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34251.191328 secs 2023-09-10T08:50:21.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:21 smithi170 ceph-mon[102832]: pgmap v18427: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 17 op/s 2023-09-10T08:50:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:21 smithi082 ceph-mon[98694]: pgmap v18427: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 17 op/s 2023-09-10T08:50:22.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:21 smithi098 ceph-mon[103124]: pgmap v18427: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 17 op/s 2023-09-10T08:50:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:23 smithi170 ceph-mon[102832]: pgmap v18428: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T08:50:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34254.586601 secs 2023-09-10T08:50:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:23 smithi082 ceph-mon[98694]: pgmap v18428: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T08:50:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34254.586601 secs 2023-09-10T08:50:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:23 smithi098 ceph-mon[103124]: pgmap v18428: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T08:50:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34254.586601 secs 2023-09-10T08:50:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34255.836080 secs 2023-09-10T08:50:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34256.191390 secs 2023-09-10T08:50:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34255.836080 secs 2023-09-10T08:50:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34256.191390 secs 2023-09-10T08:50:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34255.836080 secs 2023-09-10T08:50:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34256.191390 secs 2023-09-10T08:50:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:25 smithi170 ceph-mon[102832]: pgmap v18429: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T08:50:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:25 smithi082 ceph-mon[98694]: pgmap v18429: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T08:50:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:25 smithi098 ceph-mon[103124]: pgmap v18429: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T08:50:27.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:27 smithi170 ceph-mon[102832]: pgmap v18430: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 16 op/s 2023-09-10T08:50:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:27 smithi082 ceph-mon[98694]: pgmap v18430: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 16 op/s 2023-09-10T08:50:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:27 smithi098 ceph-mon[103124]: pgmap v18430: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 150 KiB/s wr, 16 op/s 2023-09-10T08:50:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34259.586695 secs 2023-09-10T08:50:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34259.586695 secs 2023-09-10T08:50:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34259.586695 secs 2023-09-10T08:50:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:29 smithi170 ceph-mon[102832]: pgmap v18431: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T08:50:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34260.836141 secs 2023-09-10T08:50:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34261.191454 secs 2023-09-10T08:50:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:29 smithi082 ceph-mon[98694]: pgmap v18431: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T08:50:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34260.836141 secs 2023-09-10T08:50:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34261.191454 secs 2023-09-10T08:50:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:29 smithi098 ceph-mon[103124]: pgmap v18431: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T08:50:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34260.836141 secs 2023-09-10T08:50:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34261.191454 secs 2023-09-10T08:50:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:31 smithi170 ceph-mon[102832]: pgmap v18432: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 124 KiB/s wr, 15 op/s 2023-09-10T08:50:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:31 smithi082 ceph-mon[98694]: pgmap v18432: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 124 KiB/s wr, 15 op/s 2023-09-10T08:50:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:31 smithi098 ceph-mon[103124]: pgmap v18432: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 124 KiB/s wr, 15 op/s 2023-09-10T08:50:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:33 smithi170 ceph-mon[102832]: pgmap v18433: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T08:50:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:33 smithi082 ceph-mon[98694]: pgmap v18433: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T08:50:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:33 smithi098 ceph-mon[103124]: pgmap v18433: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T08:50:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34265.836202 secs 2023-09-10T08:50:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:50:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34266.191487 secs 2023-09-10T08:50:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34265.836202 secs 2023-09-10T08:50:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:50:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34266.191487 secs 2023-09-10T08:50:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34265.836202 secs 2023-09-10T08:50:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:50:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34266.191487 secs 2023-09-10T08:50:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:35 smithi170 ceph-mon[102832]: pgmap v18434: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T08:50:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:50:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:35 smithi082 ceph-mon[98694]: pgmap v18434: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T08:50:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:50:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:35 smithi098 ceph-mon[103124]: pgmap v18434: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T08:50:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:50:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:50:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:50:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:50:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:50:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:50:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:50:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:37 smithi170 ceph-mon[102832]: pgmap v18435: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 272 KiB/s wr, 16 op/s 2023-09-10T08:50:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:37 smithi082 ceph-mon[98694]: pgmap v18435: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 272 KiB/s wr, 16 op/s 2023-09-10T08:50:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:37 smithi098 ceph-mon[103124]: pgmap v18435: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 272 KiB/s wr, 16 op/s 2023-09-10T08:50:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:39 smithi170 ceph-mon[102832]: pgmap v18436: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T08:50:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34270.836259 secs 2023-09-10T08:50:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34271.191549 secs 2023-09-10T08:50:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:39 smithi082 ceph-mon[98694]: pgmap v18436: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T08:50:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34270.836259 secs 2023-09-10T08:50:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34271.191549 secs 2023-09-10T08:50:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:39 smithi098 ceph-mon[103124]: pgmap v18436: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T08:50:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34270.836259 secs 2023-09-10T08:50:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34271.191549 secs 2023-09-10T08:50:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:41 smithi170 ceph-mon[102832]: pgmap v18437: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 436 KiB/s wr, 17 op/s 2023-09-10T08:50:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:41 smithi082 ceph-mon[98694]: pgmap v18437: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 436 KiB/s wr, 17 op/s 2023-09-10T08:50:42.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:41 smithi098 ceph-mon[103124]: pgmap v18437: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 436 KiB/s wr, 17 op/s 2023-09-10T08:50:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:43 smithi170 ceph-mon[102832]: pgmap v18438: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 436 KiB/s wr, 12 op/s 2023-09-10T08:50:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:43 smithi082 ceph-mon[98694]: pgmap v18438: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 436 KiB/s wr, 12 op/s 2023-09-10T08:50:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:43 smithi098 ceph-mon[103124]: pgmap v18438: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 436 KiB/s wr, 12 op/s 2023-09-10T08:50:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34275.836320 secs 2023-09-10T08:50:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34276.191614 secs 2023-09-10T08:50:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34275.836320 secs 2023-09-10T08:50:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34276.191614 secs 2023-09-10T08:50:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34275.836320 secs 2023-09-10T08:50:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34276.191614 secs 2023-09-10T08:50:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:45 smithi170 ceph-mon[102832]: pgmap v18439: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 436 KiB/s wr, 12 op/s 2023-09-10T08:50:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:45 smithi082 ceph-mon[98694]: pgmap v18439: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 436 KiB/s wr, 12 op/s 2023-09-10T08:50:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:45 smithi098 ceph-mon[103124]: pgmap v18439: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 436 KiB/s wr, 12 op/s 2023-09-10T08:50:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:47 smithi170 ceph-mon[102832]: pgmap v18440: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 482 KiB/s wr, 19 op/s 2023-09-10T08:50:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:50:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:50:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:50:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:50:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:47 smithi082 ceph-mon[98694]: pgmap v18440: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 482 KiB/s wr, 19 op/s 2023-09-10T08:50:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:50:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:50:47.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:50:47.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:50:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:47 smithi098 ceph-mon[103124]: pgmap v18440: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 482 KiB/s wr, 19 op/s 2023-09-10T08:50:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:50:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:50:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:50:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:50:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34279.586981 secs 2023-09-10T08:50:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34279.586981 secs 2023-09-10T08:50:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34279.586981 secs 2023-09-10T08:50:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:49 smithi170 ceph-mon[102832]: pgmap v18441: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 334 KiB/s wr, 12 op/s 2023-09-10T08:50:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34280.836382 secs 2023-09-10T08:50:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34281.191677 secs 2023-09-10T08:50:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:49 smithi082 ceph-mon[98694]: pgmap v18441: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 334 KiB/s wr, 12 op/s 2023-09-10T08:50:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34280.836382 secs 2023-09-10T08:50:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34281.191677 secs 2023-09-10T08:50:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:49 smithi098 ceph-mon[103124]: pgmap v18441: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 334 KiB/s wr, 12 op/s 2023-09-10T08:50:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34280.836382 secs 2023-09-10T08:50:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34281.191677 secs 2023-09-10T08:50:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:50:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:50:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:50:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:51 smithi170 ceph-mon[102832]: pgmap v18442: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 791 KiB/s wr, 20 op/s 2023-09-10T08:50:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:51 smithi082 ceph-mon[98694]: pgmap v18442: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 791 KiB/s wr, 20 op/s 2023-09-10T08:50:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:51 smithi098 ceph-mon[103124]: pgmap v18442: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 791 KiB/s wr, 20 op/s 2023-09-10T08:50:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:50:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:50:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:50:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:50:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:50:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:50:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:53 smithi170 ceph-mon[102832]: pgmap v18443: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 503 KiB/s wr, 14 op/s 2023-09-10T08:50:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34284.587054 secs 2023-09-10T08:50:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:53 smithi082 ceph-mon[98694]: pgmap v18443: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 503 KiB/s wr, 14 op/s 2023-09-10T08:50:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34284.587054 secs 2023-09-10T08:50:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:53 smithi098 ceph-mon[103124]: pgmap v18443: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 503 KiB/s wr, 14 op/s 2023-09-10T08:50:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34284.587054 secs 2023-09-10T08:50:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34285.836465 secs 2023-09-10T08:50:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:50:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34286.191739 secs 2023-09-10T08:50:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34285.836465 secs 2023-09-10T08:50:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:50:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34286.191739 secs 2023-09-10T08:50:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34285.836465 secs 2023-09-10T08:50:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:50:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34286.191739 secs 2023-09-10T08:50:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:55 smithi170 ceph-mon[102832]: pgmap v18444: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 503 KiB/s wr, 14 op/s 2023-09-10T08:50:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:55 smithi082 ceph-mon[98694]: pgmap v18444: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 503 KiB/s wr, 14 op/s 2023-09-10T08:50:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:55 smithi098 ceph-mon[103124]: pgmap v18444: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 503 KiB/s wr, 14 op/s 2023-09-10T08:50:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:50:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:50:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:50:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:57 smithi170 ceph-mon[102832]: pgmap v18445: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 564 KiB/s wr, 20 op/s 2023-09-10T08:50:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:57 smithi082 ceph-mon[98694]: pgmap v18445: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 564 KiB/s wr, 20 op/s 2023-09-10T08:50:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:57 smithi098 ceph-mon[103124]: pgmap v18445: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 564 KiB/s wr, 20 op/s 2023-09-10T08:50:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:50:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34289.587115 secs 2023-09-10T08:50:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:50:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34289.587115 secs 2023-09-10T08:50:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:50:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:50:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34289.587115 secs 2023-09-10T08:50:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:59 smithi170 ceph-mon[102832]: pgmap v18446: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 518 KiB/s wr, 13 op/s 2023-09-10T08:50:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34290.836499 secs 2023-09-10T08:50:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:50:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34291.191801 secs 2023-09-10T08:50:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:59 smithi082 ceph-mon[98694]: pgmap v18446: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 518 KiB/s wr, 13 op/s 2023-09-10T08:50:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34290.836499 secs 2023-09-10T08:50:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:50:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34291.191801 secs 2023-09-10T08:51:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:59 smithi098 ceph-mon[103124]: pgmap v18446: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 518 KiB/s wr, 13 op/s 2023-09-10T08:51:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34290.836499 secs 2023-09-10T08:51:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:50:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34291.191801 secs 2023-09-10T08:51:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:51:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:51:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:51:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:01 smithi170 ceph-mon[102832]: pgmap v18447: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 978 KiB/s wr, 21 op/s 2023-09-10T08:51:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:01 smithi082 ceph-mon[98694]: pgmap v18447: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 978 KiB/s wr, 21 op/s 2023-09-10T08:51:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:01 smithi098 ceph-mon[103124]: pgmap v18447: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 978 KiB/s wr, 21 op/s 2023-09-10T08:51:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:03 smithi170 ceph-mon[102832]: pgmap v18448: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 520 KiB/s wr, 13 op/s 2023-09-10T08:51:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34294.587172 secs 2023-09-10T08:51:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:03 smithi082 ceph-mon[98694]: pgmap v18448: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 520 KiB/s wr, 13 op/s 2023-09-10T08:51:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34294.587172 secs 2023-09-10T08:51:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:03 smithi098 ceph-mon[103124]: pgmap v18448: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 520 KiB/s wr, 13 op/s 2023-09-10T08:51:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34294.587172 secs 2023-09-10T08:51:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34295.836557 secs 2023-09-10T08:51:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:51:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34296.191863 secs 2023-09-10T08:51:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34295.836557 secs 2023-09-10T08:51:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:51:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34296.191863 secs 2023-09-10T08:51:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34295.836557 secs 2023-09-10T08:51:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:51:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34296.191863 secs 2023-09-10T08:51:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:05 smithi170 ceph-mon[102832]: pgmap v18449: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 520 KiB/s wr, 13 op/s 2023-09-10T08:51:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:05 smithi082 ceph-mon[98694]: pgmap v18449: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 520 KiB/s wr, 13 op/s 2023-09-10T08:51:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:05 smithi098 ceph-mon[103124]: pgmap v18449: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 520 KiB/s wr, 13 op/s 2023-09-10T08:51:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:51:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:51:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:51:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:07 smithi082 ceph-mon[98694]: pgmap v18450: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 685 KiB/s wr, 35 op/s 2023-09-10T08:51:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:07 smithi098 ceph-mon[103124]: pgmap v18450: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 685 KiB/s wr, 35 op/s 2023-09-10T08:51:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:07 smithi170 ceph-mon[102832]: pgmap v18450: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 685 KiB/s wr, 35 op/s 2023-09-10T08:51:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34299.587291 secs 2023-09-10T08:51:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34299.587291 secs 2023-09-10T08:51:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34299.587291 secs 2023-09-10T08:51:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:09 smithi082 ceph-mon[98694]: pgmap v18451: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 625 KiB/s wr, 29 op/s 2023-09-10T08:51:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34300.836618 secs 2023-09-10T08:51:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:09 smithi082 ceph-mon[98694]: osdmap e1221: 12 total, 12 up, 12 in 2023-09-10T08:51:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34301.191929 secs 2023-09-10T08:51:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:09 smithi098 ceph-mon[103124]: pgmap v18451: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 625 KiB/s wr, 29 op/s 2023-09-10T08:51:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34300.836618 secs 2023-09-10T08:51:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:09 smithi098 ceph-mon[103124]: osdmap e1221: 12 total, 12 up, 12 in 2023-09-10T08:51:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34301.191929 secs 2023-09-10T08:51:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:09 smithi170 ceph-mon[102832]: pgmap v18451: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 625 KiB/s wr, 29 op/s 2023-09-10T08:51:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34300.836618 secs 2023-09-10T08:51:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:09 smithi170 ceph-mon[102832]: osdmap e1221: 12 total, 12 up, 12 in 2023-09-10T08:51:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34301.191929 secs 2023-09-10T08:51:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:11 smithi082 ceph-mon[98694]: pgmap v18453: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 525 KiB/s wr, 33 op/s 2023-09-10T08:51:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:11 smithi098 ceph-mon[103124]: pgmap v18453: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 525 KiB/s wr, 33 op/s 2023-09-10T08:51:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:11 smithi170 ceph-mon[102832]: pgmap v18453: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 525 KiB/s wr, 33 op/s 2023-09-10T08:51:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:13 smithi082 ceph-mon[98694]: pgmap v18454: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 525 KiB/s wr, 33 op/s 2023-09-10T08:51:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34304.587351 secs 2023-09-10T08:51:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:13 smithi082 ceph-mon[98694]: osdmap e1222: 12 total, 12 up, 12 in 2023-09-10T08:51:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:13 smithi098 ceph-mon[103124]: pgmap v18454: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 525 KiB/s wr, 33 op/s 2023-09-10T08:51:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34304.587351 secs 2023-09-10T08:51:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:13 smithi098 ceph-mon[103124]: osdmap e1222: 12 total, 12 up, 12 in 2023-09-10T08:51:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:13 smithi170 ceph-mon[102832]: pgmap v18454: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 525 KiB/s wr, 33 op/s 2023-09-10T08:51:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34304.587351 secs 2023-09-10T08:51:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:13 smithi170 ceph-mon[102832]: osdmap e1222: 12 total, 12 up, 12 in 2023-09-10T08:51:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34305.836681 secs 2023-09-10T08:51:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34306.191992 secs 2023-09-10T08:51:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34305.836681 secs 2023-09-10T08:51:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34306.191992 secs 2023-09-10T08:51:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34305.836681 secs 2023-09-10T08:51:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34306.191992 secs 2023-09-10T08:51:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:15 smithi082 ceph-mon[98694]: pgmap v18456: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 408 KiB/s wr, 9 op/s 2023-09-10T08:51:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:15 smithi098 ceph-mon[103124]: pgmap v18456: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 408 KiB/s wr, 9 op/s 2023-09-10T08:51:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:15 smithi170 ceph-mon[102832]: pgmap v18456: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 408 KiB/s wr, 9 op/s 2023-09-10T08:51:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:17 smithi082 ceph-mon[98694]: pgmap v18457: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 409 KiB/s wr, 16 op/s 2023-09-10T08:51:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:17 smithi098 ceph-mon[103124]: pgmap v18457: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 409 KiB/s wr, 16 op/s 2023-09-10T08:51:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:17 smithi170 ceph-mon[102832]: pgmap v18457: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 409 KiB/s wr, 16 op/s 2023-09-10T08:51:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34309.587491 secs 2023-09-10T08:51:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34309.587491 secs 2023-09-10T08:51:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34309.587491 secs 2023-09-10T08:51:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:19 smithi082 ceph-mon[98694]: pgmap v18458: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 333 KiB/s wr, 13 op/s 2023-09-10T08:51:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34310.836763 secs 2023-09-10T08:51:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34311.192053 secs 2023-09-10T08:51:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:19 smithi098 ceph-mon[103124]: pgmap v18458: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 333 KiB/s wr, 13 op/s 2023-09-10T08:51:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34310.836763 secs 2023-09-10T08:51:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34311.192053 secs 2023-09-10T08:51:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:19 smithi170 ceph-mon[102832]: pgmap v18458: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 333 KiB/s wr, 13 op/s 2023-09-10T08:51:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34310.836763 secs 2023-09-10T08:51:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34311.192053 secs 2023-09-10T08:51:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:21 smithi082 ceph-mon[98694]: pgmap v18459: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T08:51:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:21 smithi098 ceph-mon[103124]: pgmap v18459: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T08:51:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:21 smithi170 ceph-mon[102832]: pgmap v18459: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T08:51:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:23 smithi098 ceph-mon[103124]: pgmap v18460: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T08:51:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34314.587550 secs 2023-09-10T08:51:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:23 smithi170 ceph-mon[102832]: pgmap v18460: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T08:51:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34314.587550 secs 2023-09-10T08:51:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:23 smithi082 ceph-mon[98694]: pgmap v18460: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T08:51:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34314.587550 secs 2023-09-10T08:51:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34315.836821 secs 2023-09-10T08:51:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34316.192116 secs 2023-09-10T08:51:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34315.836821 secs 2023-09-10T08:51:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34316.192116 secs 2023-09-10T08:51:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34315.836821 secs 2023-09-10T08:51:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34316.192116 secs 2023-09-10T08:51:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:25 smithi098 ceph-mon[103124]: pgmap v18461: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 10 op/s 2023-09-10T08:51:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:25 smithi170 ceph-mon[102832]: pgmap v18461: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 10 op/s 2023-09-10T08:51:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:25 smithi082 ceph-mon[98694]: pgmap v18461: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 10 op/s 2023-09-10T08:51:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:27 smithi098 ceph-mon[103124]: pgmap v18462: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 15 op/s 2023-09-10T08:51:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:27 smithi170 ceph-mon[102832]: pgmap v18462: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 15 op/s 2023-09-10T08:51:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:27 smithi082 ceph-mon[98694]: pgmap v18462: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 15 op/s 2023-09-10T08:51:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34319.587609 secs 2023-09-10T08:51:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34319.587609 secs 2023-09-10T08:51:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34319.587609 secs 2023-09-10T08:51:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:29 smithi098 ceph-mon[103124]: pgmap v18463: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-10T08:51:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34320.836878 secs 2023-09-10T08:51:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34321.192179 secs 2023-09-10T08:51:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:29 smithi170 ceph-mon[102832]: pgmap v18463: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-10T08:51:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34320.836878 secs 2023-09-10T08:51:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34321.192179 secs 2023-09-10T08:51:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:29 smithi082 ceph-mon[98694]: pgmap v18463: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-10T08:51:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34320.836878 secs 2023-09-10T08:51:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34321.192179 secs 2023-09-10T08:51:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:31 smithi098 ceph-mon[103124]: pgmap v18464: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 16 op/s 2023-09-10T08:51:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:31 smithi170 ceph-mon[102832]: pgmap v18464: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 16 op/s 2023-09-10T08:51:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:31 smithi082 ceph-mon[98694]: pgmap v18464: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 16 op/s 2023-09-10T08:51:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:33 smithi098 ceph-mon[103124]: pgmap v18465: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T08:51:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:33 smithi170 ceph-mon[102832]: pgmap v18465: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T08:51:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:33 smithi082 ceph-mon[98694]: pgmap v18465: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T08:51:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34325.836937 secs 2023-09-10T08:51:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34326.192242 secs 2023-09-10T08:51:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34325.836937 secs 2023-09-10T08:51:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34326.192242 secs 2023-09-10T08:51:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34325.836937 secs 2023-09-10T08:51:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34326.192242 secs 2023-09-10T08:51:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:35 smithi098 ceph-mon[103124]: pgmap v18466: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T08:51:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:51:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:35 smithi170 ceph-mon[102832]: pgmap v18466: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T08:51:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:51:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:35 smithi082 ceph-mon[98694]: pgmap v18466: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T08:51:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:51:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:51:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:51:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:51:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:51:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:51:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:51:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:37 smithi098 ceph-mon[103124]: pgmap v18467: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 246 KiB/s wr, 16 op/s 2023-09-10T08:51:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:37 smithi170 ceph-mon[102832]: pgmap v18467: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 246 KiB/s wr, 16 op/s 2023-09-10T08:51:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:37 smithi082 ceph-mon[98694]: pgmap v18467: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 246 KiB/s wr, 16 op/s 2023-09-10T08:51:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:51:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:51:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:51:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:39 smithi170 ceph-mon[102832]: pgmap v18468: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 245 KiB/s wr, 11 op/s 2023-09-10T08:51:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34330.837035 secs 2023-09-10T08:51:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34331.192303 secs 2023-09-10T08:51:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:39 smithi082 ceph-mon[98694]: pgmap v18468: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 245 KiB/s wr, 11 op/s 2023-09-10T08:51:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34330.837035 secs 2023-09-10T08:51:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34331.192303 secs 2023-09-10T08:51:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:39 smithi098 ceph-mon[103124]: pgmap v18468: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 245 KiB/s wr, 11 op/s 2023-09-10T08:51:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34330.837035 secs 2023-09-10T08:51:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34331.192303 secs 2023-09-10T08:51:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:51:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:51:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:51:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:41 smithi170 ceph-mon[102832]: pgmap v18469: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 900 KiB/s wr, 19 op/s 2023-09-10T08:51:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:41 smithi082 ceph-mon[98694]: pgmap v18469: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 900 KiB/s wr, 19 op/s 2023-09-10T08:51:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:41 smithi098 ceph-mon[103124]: pgmap v18469: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 900 KiB/s wr, 19 op/s 2023-09-10T08:51:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:43 smithi170 ceph-mon[102832]: pgmap v18470: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 775 KiB/s wr, 13 op/s 2023-09-10T08:51:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:43 smithi082 ceph-mon[98694]: pgmap v18470: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 775 KiB/s wr, 13 op/s 2023-09-10T08:51:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:43 smithi098 ceph-mon[103124]: pgmap v18470: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 775 KiB/s wr, 13 op/s 2023-09-10T08:51:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34335.837143 secs 2023-09-10T08:51:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:51:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34336.192366 secs 2023-09-10T08:51:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34335.837143 secs 2023-09-10T08:51:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:51:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34336.192366 secs 2023-09-10T08:51:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34335.837143 secs 2023-09-10T08:51:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:51:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34336.192366 secs 2023-09-10T08:51:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:45 smithi170 ceph-mon[102832]: pgmap v18471: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 775 KiB/s wr, 13 op/s 2023-09-10T08:51:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:45 smithi082 ceph-mon[98694]: pgmap v18471: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 775 KiB/s wr, 13 op/s 2023-09-10T08:51:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:45 smithi098 ceph-mon[103124]: pgmap v18471: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 775 KiB/s wr, 13 op/s 2023-09-10T08:51:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:51:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:51:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:51:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:47 smithi170 ceph-mon[102832]: pgmap v18472: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 799 KiB/s wr, 20 op/s 2023-09-10T08:51:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:51:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:47 smithi082 ceph-mon[98694]: pgmap v18472: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 799 KiB/s wr, 20 op/s 2023-09-10T08:51:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:51:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:47 smithi098 ceph-mon[103124]: pgmap v18472: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 799 KiB/s wr, 20 op/s 2023-09-10T08:51:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:51:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:51:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:51:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:51:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:51:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:51:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:51:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:51:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:51:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:51:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:49 smithi170 ceph-mon[102832]: pgmap v18473: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 678 KiB/s wr, 14 op/s 2023-09-10T08:51:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34340.837199 secs 2023-09-10T08:51:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34341.192429 secs 2023-09-10T08:51:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:49 smithi082 ceph-mon[98694]: pgmap v18473: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 678 KiB/s wr, 14 op/s 2023-09-10T08:51:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34340.837199 secs 2023-09-10T08:51:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34341.192429 secs 2023-09-10T08:51:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:49 smithi098 ceph-mon[103124]: pgmap v18473: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 678 KiB/s wr, 14 op/s 2023-09-10T08:51:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34340.837199 secs 2023-09-10T08:51:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34341.192429 secs 2023-09-10T08:51:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:51:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:51:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:51:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:51 smithi170 ceph-mon[102832]: pgmap v18474: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T08:51:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:51 smithi082 ceph-mon[98694]: pgmap v18474: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T08:51:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:51 smithi098 ceph-mon[103124]: pgmap v18474: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T08:51:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:51:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:51:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:51:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:53 smithi170 ceph-mon[102832]: pgmap v18475: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 460 KiB/s wr, 13 op/s 2023-09-10T08:51:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34344.587923 secs 2023-09-10T08:51:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:53 smithi082 ceph-mon[98694]: pgmap v18475: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 460 KiB/s wr, 13 op/s 2023-09-10T08:51:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34344.587923 secs 2023-09-10T08:51:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:53 smithi098 ceph-mon[103124]: pgmap v18475: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 460 KiB/s wr, 13 op/s 2023-09-10T08:51:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34344.587923 secs 2023-09-10T08:51:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34345.837257 secs 2023-09-10T08:51:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:51:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34346.192487 secs 2023-09-10T08:51:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34345.837257 secs 2023-09-10T08:51:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:51:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34346.192487 secs 2023-09-10T08:51:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34345.837257 secs 2023-09-10T08:51:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:51:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34346.192487 secs 2023-09-10T08:51:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:55 smithi170 ceph-mon[102832]: pgmap v18476: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 460 KiB/s wr, 13 op/s 2023-09-10T08:51:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:55 smithi082 ceph-mon[98694]: pgmap v18476: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 460 KiB/s wr, 13 op/s 2023-09-10T08:51:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:55 smithi098 ceph-mon[103124]: pgmap v18476: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 460 KiB/s wr, 13 op/s 2023-09-10T08:51:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:51:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:51:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:51:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:51:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:51:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:51:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:51:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:57 smithi170 ceph-mon[102832]: pgmap v18477: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 893 KiB/s wr, 20 op/s 2023-09-10T08:51:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:57 smithi082 ceph-mon[98694]: pgmap v18477: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 893 KiB/s wr, 20 op/s 2023-09-10T08:51:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:57 smithi098 ceph-mon[103124]: pgmap v18477: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 893 KiB/s wr, 20 op/s 2023-09-10T08:51:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34349.588015 secs 2023-09-10T08:51:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34349.588015 secs 2023-09-10T08:51:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34349.588015 secs 2023-09-10T08:52:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:59 smithi170 ceph-mon[102832]: pgmap v18478: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 869 KiB/s wr, 14 op/s 2023-09-10T08:52:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34350.837342 secs 2023-09-10T08:52:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:51:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34351.192555 secs 2023-09-10T08:52:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:59 smithi082 ceph-mon[98694]: pgmap v18478: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 869 KiB/s wr, 14 op/s 2023-09-10T08:52:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34350.837342 secs 2023-09-10T08:52:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:51:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34351.192555 secs 2023-09-10T08:52:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:59 smithi098 ceph-mon[103124]: pgmap v18478: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 869 KiB/s wr, 14 op/s 2023-09-10T08:52:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34350.837342 secs 2023-09-10T08:52:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:51:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34351.192555 secs 2023-09-10T08:52:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:01 smithi170 ceph-mon[102832]: pgmap v18479: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 997 KiB/s wr, 20 op/s 2023-09-10T08:52:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:01 smithi082 ceph-mon[98694]: pgmap v18479: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 997 KiB/s wr, 20 op/s 2023-09-10T08:52:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:01 smithi098 ceph-mon[103124]: pgmap v18479: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 997 KiB/s wr, 20 op/s 2023-09-10T08:52:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:03 smithi170 ceph-mon[102832]: pgmap v18480: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 562 KiB/s wr, 13 op/s 2023-09-10T08:52:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34354.588076 secs 2023-09-10T08:52:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:03 smithi082 ceph-mon[98694]: pgmap v18480: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 562 KiB/s wr, 13 op/s 2023-09-10T08:52:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34354.588076 secs 2023-09-10T08:52:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:03 smithi098 ceph-mon[103124]: pgmap v18480: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 562 KiB/s wr, 13 op/s 2023-09-10T08:52:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34354.588076 secs 2023-09-10T08:52:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34355.837399 secs 2023-09-10T08:52:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:52:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34356.192617 secs 2023-09-10T08:52:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34355.837399 secs 2023-09-10T08:52:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:52:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34356.192617 secs 2023-09-10T08:52:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34355.837399 secs 2023-09-10T08:52:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:52:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34356.192617 secs 2023-09-10T08:52:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:05 smithi170 ceph-mon[102832]: pgmap v18481: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 562 KiB/s wr, 13 op/s 2023-09-10T08:52:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:52:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:05 smithi082 ceph-mon[98694]: pgmap v18481: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 562 KiB/s wr, 13 op/s 2023-09-10T08:52:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:52:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:05 smithi098 ceph-mon[103124]: pgmap v18481: 97 pgs: 97 active+clean; 11 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 562 KiB/s wr, 13 op/s 2023-09-10T08:52:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:52:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:07 smithi170 ceph-mon[102832]: pgmap v18482: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 884 KiB/s wr, 33 op/s 2023-09-10T08:52:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:07 smithi082 ceph-mon[98694]: pgmap v18482: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 884 KiB/s wr, 33 op/s 2023-09-10T08:52:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:07 smithi098 ceph-mon[103124]: pgmap v18482: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 884 KiB/s wr, 33 op/s 2023-09-10T08:52:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34359.588159 secs 2023-09-10T08:52:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34359.588159 secs 2023-09-10T08:52:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34359.588159 secs 2023-09-10T08:52:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:09 smithi170 ceph-mon[102832]: pgmap v18483: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 451 KiB/s wr, 26 op/s 2023-09-10T08:52:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34360.837490 secs 2023-09-10T08:52:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:09 smithi170 ceph-mon[102832]: osdmap e1223: 12 total, 12 up, 12 in 2023-09-10T08:52:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34361.192683 secs 2023-09-10T08:52:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:09 smithi082 ceph-mon[98694]: pgmap v18483: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 451 KiB/s wr, 26 op/s 2023-09-10T08:52:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34360.837490 secs 2023-09-10T08:52:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:09 smithi082 ceph-mon[98694]: osdmap e1223: 12 total, 12 up, 12 in 2023-09-10T08:52:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34361.192683 secs 2023-09-10T08:52:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:09 smithi098 ceph-mon[103124]: pgmap v18483: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 451 KiB/s wr, 26 op/s 2023-09-10T08:52:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34360.837490 secs 2023-09-10T08:52:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:09 smithi098 ceph-mon[103124]: osdmap e1223: 12 total, 12 up, 12 in 2023-09-10T08:52:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34361.192683 secs 2023-09-10T08:52:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:11 smithi170 ceph-mon[102832]: pgmap v18485: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 536 KiB/s wr, 32 op/s 2023-09-10T08:52:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:11 smithi082 ceph-mon[98694]: pgmap v18485: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 536 KiB/s wr, 32 op/s 2023-09-10T08:52:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:11 smithi098 ceph-mon[103124]: pgmap v18485: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 536 KiB/s wr, 32 op/s 2023-09-10T08:52:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:13 smithi170 ceph-mon[102832]: pgmap v18486: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 536 KiB/s wr, 32 op/s 2023-09-10T08:52:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:52:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34364.588257 secs 2023-09-10T08:52:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:13 smithi170 ceph-mon[102832]: osdmap e1224: 12 total, 12 up, 12 in 2023-09-10T08:52:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:13 smithi082 ceph-mon[98694]: pgmap v18486: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 536 KiB/s wr, 32 op/s 2023-09-10T08:52:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:52:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34364.588257 secs 2023-09-10T08:52:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:13 smithi082 ceph-mon[98694]: osdmap e1224: 12 total, 12 up, 12 in 2023-09-10T08:52:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:13 smithi098 ceph-mon[103124]: pgmap v18486: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 536 KiB/s wr, 32 op/s 2023-09-10T08:52:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:52:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34364.588257 secs 2023-09-10T08:52:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:13 smithi098 ceph-mon[103124]: osdmap e1224: 12 total, 12 up, 12 in 2023-09-10T08:52:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34365.837548 secs 2023-09-10T08:52:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34366.192744 secs 2023-09-10T08:52:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34365.837548 secs 2023-09-10T08:52:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34366.192744 secs 2023-09-10T08:52:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34365.837548 secs 2023-09-10T08:52:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34366.192744 secs 2023-09-10T08:52:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:15 smithi170 ceph-mon[102832]: pgmap v18488: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 186 KiB/s wr, 9 op/s 2023-09-10T08:52:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:15 smithi082 ceph-mon[98694]: pgmap v18488: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 186 KiB/s wr, 9 op/s 2023-09-10T08:52:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:15 smithi098 ceph-mon[103124]: pgmap v18488: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 186 KiB/s wr, 9 op/s 2023-09-10T08:52:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:52:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:52:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:52:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:17 smithi170 ceph-mon[102832]: pgmap v18489: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 226 KiB/s wr, 19 op/s 2023-09-10T08:52:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:17 smithi082 ceph-mon[98694]: pgmap v18489: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 226 KiB/s wr, 19 op/s 2023-09-10T08:52:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:17 smithi098 ceph-mon[103124]: pgmap v18489: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 226 KiB/s wr, 19 op/s 2023-09-10T08:52:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34369.588329 secs 2023-09-10T08:52:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34369.588329 secs 2023-09-10T08:52:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34369.588329 secs 2023-09-10T08:52:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:19 smithi170 ceph-mon[102832]: pgmap v18490: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 16 op/s 2023-09-10T08:52:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34370.838466 secs 2023-09-10T08:52:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34371.192809 secs 2023-09-10T08:52:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:19 smithi082 ceph-mon[98694]: pgmap v18490: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 16 op/s 2023-09-10T08:52:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34370.838466 secs 2023-09-10T08:52:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34371.192809 secs 2023-09-10T08:52:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:19 smithi098 ceph-mon[103124]: pgmap v18490: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 16 op/s 2023-09-10T08:52:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34370.838466 secs 2023-09-10T08:52:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34371.192809 secs 2023-09-10T08:52:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:21 smithi170 ceph-mon[102832]: pgmap v18491: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 150 KiB/s wr, 14 op/s 2023-09-10T08:52:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:21 smithi082 ceph-mon[98694]: pgmap v18491: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 150 KiB/s wr, 14 op/s 2023-09-10T08:52:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:21 smithi098 ceph-mon[103124]: pgmap v18491: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 150 KiB/s wr, 14 op/s 2023-09-10T08:52:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:23 smithi170 ceph-mon[102832]: pgmap v18492: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 150 KiB/s wr, 14 op/s 2023-09-10T08:52:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34374.588389 secs 2023-09-10T08:52:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:23 smithi082 ceph-mon[98694]: pgmap v18492: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 150 KiB/s wr, 14 op/s 2023-09-10T08:52:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34374.588389 secs 2023-09-10T08:52:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:23 smithi098 ceph-mon[103124]: pgmap v18492: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 150 KiB/s wr, 14 op/s 2023-09-10T08:52:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34374.588389 secs 2023-09-10T08:52:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34375.838525 secs 2023-09-10T08:52:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34376.192873 secs 2023-09-10T08:52:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34375.838525 secs 2023-09-10T08:52:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34376.192873 secs 2023-09-10T08:52:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34375.838525 secs 2023-09-10T08:52:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34376.192873 secs 2023-09-10T08:52:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:25 smithi170 ceph-mon[102832]: pgmap v18493: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 13 op/s 2023-09-10T08:52:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:25 smithi082 ceph-mon[98694]: pgmap v18493: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 13 op/s 2023-09-10T08:52:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:25 smithi098 ceph-mon[103124]: pgmap v18493: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 13 op/s 2023-09-10T08:52:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:27 smithi170 ceph-mon[102832]: pgmap v18494: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 249 KiB/s wr, 17 op/s 2023-09-10T08:52:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:27 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34379.588464 secs 2023-09-10T08:52:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:27 smithi082 ceph-mon[98694]: pgmap v18494: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 249 KiB/s wr, 17 op/s 2023-09-10T08:52:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:27 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34379.588464 secs 2023-09-10T08:52:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:27 smithi098 ceph-mon[103124]: pgmap v18494: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 249 KiB/s wr, 17 op/s 2023-09-10T08:52:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:27 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34379.588464 secs 2023-09-10T08:52:29.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:29 smithi170 ceph-mon[102832]: pgmap v18495: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 222 KiB/s wr, 11 op/s 2023-09-10T08:52:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34380.838609 secs 2023-09-10T08:52:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34381.192935 secs 2023-09-10T08:52:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:29 smithi082 ceph-mon[98694]: pgmap v18495: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 222 KiB/s wr, 11 op/s 2023-09-10T08:52:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34380.838609 secs 2023-09-10T08:52:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34381.192935 secs 2023-09-10T08:52:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:29 smithi098 ceph-mon[103124]: pgmap v18495: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 222 KiB/s wr, 11 op/s 2023-09-10T08:52:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34380.838609 secs 2023-09-10T08:52:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34381.192935 secs 2023-09-10T08:52:31.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:31 smithi170 ceph-mon[102832]: pgmap v18496: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 222 KiB/s wr, 15 op/s 2023-09-10T08:52:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:31 smithi082 ceph-mon[98694]: pgmap v18496: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 222 KiB/s wr, 15 op/s 2023-09-10T08:52:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:31 smithi098 ceph-mon[103124]: pgmap v18496: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 222 KiB/s wr, 15 op/s 2023-09-10T08:52:33.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:33 smithi170 ceph-mon[102832]: pgmap v18497: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T08:52:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:33 smithi082 ceph-mon[98694]: pgmap v18497: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T08:52:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:33 smithi098 ceph-mon[103124]: pgmap v18497: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T08:52:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34385.838682 secs 2023-09-10T08:52:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:52:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34386.192997 secs 2023-09-10T08:52:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34385.838682 secs 2023-09-10T08:52:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:52:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34386.192997 secs 2023-09-10T08:52:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34385.838682 secs 2023-09-10T08:52:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:52:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34386.192997 secs 2023-09-10T08:52:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:35 smithi170 ceph-mon[102832]: pgmap v18498: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T08:52:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:52:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:35 smithi082 ceph-mon[98694]: pgmap v18498: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T08:52:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:52:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:35 smithi098 ceph-mon[103124]: pgmap v18498: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 124 KiB/s wr, 10 op/s 2023-09-10T08:52:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:52:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:52:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:52:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:52:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:52:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:52:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:52:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:37 smithi170 ceph-mon[102832]: pgmap v18499: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 273 KiB/s wr, 16 op/s 2023-09-10T08:52:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:37 smithi082 ceph-mon[98694]: pgmap v18499: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 273 KiB/s wr, 16 op/s 2023-09-10T08:52:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:37 smithi098 ceph-mon[103124]: pgmap v18499: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 273 KiB/s wr, 16 op/s 2023-09-10T08:52:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:39 smithi170 ceph-mon[102832]: pgmap v18500: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 10 op/s 2023-09-10T08:52:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34390.838744 secs 2023-09-10T08:52:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34391.193060 secs 2023-09-10T08:52:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:39 smithi082 ceph-mon[98694]: pgmap v18500: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 10 op/s 2023-09-10T08:52:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34390.838744 secs 2023-09-10T08:52:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34391.193060 secs 2023-09-10T08:52:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:39 smithi098 ceph-mon[103124]: pgmap v18500: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 10 op/s 2023-09-10T08:52:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34390.838744 secs 2023-09-10T08:52:39.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34391.193060 secs 2023-09-10T08:52:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:41 smithi170 ceph-mon[102832]: pgmap v18501: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 454 KiB/s wr, 17 op/s 2023-09-10T08:52:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:41 smithi082 ceph-mon[98694]: pgmap v18501: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 454 KiB/s wr, 17 op/s 2023-09-10T08:52:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:41 smithi098 ceph-mon[103124]: pgmap v18501: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 454 KiB/s wr, 17 op/s 2023-09-10T08:52:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:43 smithi170 ceph-mon[102832]: pgmap v18502: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 454 KiB/s wr, 12 op/s 2023-09-10T08:52:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:43 smithi082 ceph-mon[98694]: pgmap v18502: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 454 KiB/s wr, 12 op/s 2023-09-10T08:52:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:43 smithi098 ceph-mon[103124]: pgmap v18502: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 454 KiB/s wr, 12 op/s 2023-09-10T08:52:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34395.838803 secs 2023-09-10T08:52:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34396.193121 secs 2023-09-10T08:52:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34395.838803 secs 2023-09-10T08:52:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34396.193121 secs 2023-09-10T08:52:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34395.838803 secs 2023-09-10T08:52:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34396.193121 secs 2023-09-10T08:52:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:45 smithi170 ceph-mon[102832]: pgmap v18503: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 454 KiB/s wr, 12 op/s 2023-09-10T08:52:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:45 smithi082 ceph-mon[98694]: pgmap v18503: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 454 KiB/s wr, 12 op/s 2023-09-10T08:52:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:45 smithi098 ceph-mon[103124]: pgmap v18503: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 454 KiB/s wr, 12 op/s 2023-09-10T08:52:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:47 smithi170 ceph-mon[102832]: pgmap v18504: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 603 KiB/s wr, 18 op/s 2023-09-10T08:52:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:47 smithi082 ceph-mon[98694]: pgmap v18504: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 603 KiB/s wr, 18 op/s 2023-09-10T08:52:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:47 smithi098 ceph-mon[103124]: pgmap v18504: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 603 KiB/s wr, 18 op/s 2023-09-10T08:52:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:52:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:52:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:52:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:52:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:52:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:52:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:52:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:52:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:52:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:52:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:52:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:52:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:49 smithi170 ceph-mon[102832]: pgmap v18505: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 454 KiB/s wr, 12 op/s 2023-09-10T08:52:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34400.838883 secs 2023-09-10T08:52:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34401.193184 secs 2023-09-10T08:52:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:49 smithi082 ceph-mon[98694]: pgmap v18505: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 454 KiB/s wr, 12 op/s 2023-09-10T08:52:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34400.838883 secs 2023-09-10T08:52:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34401.193184 secs 2023-09-10T08:52:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:49 smithi098 ceph-mon[103124]: pgmap v18505: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 454 KiB/s wr, 12 op/s 2023-09-10T08:52:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34400.838883 secs 2023-09-10T08:52:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34401.193184 secs 2023-09-10T08:52:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:52:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:52:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:52:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:51 smithi170 ceph-mon[102832]: pgmap v18506: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 797 KiB/s wr, 21 op/s 2023-09-10T08:52:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:51 smithi082 ceph-mon[98694]: pgmap v18506: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 797 KiB/s wr, 21 op/s 2023-09-10T08:52:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:51 smithi098 ceph-mon[103124]: pgmap v18506: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 797 KiB/s wr, 21 op/s 2023-09-10T08:52:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:52:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:52:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:52:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:53 smithi170 ceph-mon[102832]: pgmap v18507: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 492 KiB/s wr, 14 op/s 2023-09-10T08:52:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:53 smithi082 ceph-mon[98694]: pgmap v18507: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 492 KiB/s wr, 14 op/s 2023-09-10T08:52:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:53 smithi098 ceph-mon[103124]: pgmap v18507: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 492 KiB/s wr, 14 op/s 2023-09-10T08:52:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34405.838967 secs 2023-09-10T08:52:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:52:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34406.193247 secs 2023-09-10T08:52:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34405.838967 secs 2023-09-10T08:52:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:52:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34406.193247 secs 2023-09-10T08:52:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34405.838967 secs 2023-09-10T08:52:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:52:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34406.193247 secs 2023-09-10T08:52:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:55 smithi170 ceph-mon[102832]: pgmap v18508: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 492 KiB/s wr, 14 op/s 2023-09-10T08:52:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:55 smithi082 ceph-mon[98694]: pgmap v18508: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 492 KiB/s wr, 14 op/s 2023-09-10T08:52:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:55 smithi098 ceph-mon[103124]: pgmap v18508: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 492 KiB/s wr, 14 op/s 2023-09-10T08:52:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:52:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:52:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:52:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:57 smithi170 ceph-mon[102832]: pgmap v18509: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 553 KiB/s wr, 21 op/s 2023-09-10T08:52:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:57 smithi082 ceph-mon[98694]: pgmap v18509: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 553 KiB/s wr, 21 op/s 2023-09-10T08:52:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:57 smithi098 ceph-mon[103124]: pgmap v18509: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 553 KiB/s wr, 21 op/s 2023-09-10T08:52:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:52:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:52:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:52:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:52:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:59 smithi170 ceph-mon[102832]: pgmap v18510: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 404 KiB/s wr, 14 op/s 2023-09-10T08:52:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34410.839027 secs 2023-09-10T08:52:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:52:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34411.193312 secs 2023-09-10T08:52:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:59 smithi082 ceph-mon[98694]: pgmap v18510: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 404 KiB/s wr, 14 op/s 2023-09-10T08:52:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34410.839027 secs 2023-09-10T08:52:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:52:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34411.193312 secs 2023-09-10T08:53:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:59 smithi098 ceph-mon[103124]: pgmap v18510: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 404 KiB/s wr, 14 op/s 2023-09-10T08:53:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34410.839027 secs 2023-09-10T08:53:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:52:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34411.193312 secs 2023-09-10T08:53:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:53:01.019 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:01.020 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:53:01.020 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:53:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:01 smithi082 ceph-mon[98694]: pgmap v18511: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T08:53:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:01 smithi098 ceph-mon[103124]: pgmap v18511: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T08:53:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:01 smithi170 ceph-mon[102832]: pgmap v18511: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T08:53:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:03 smithi082 ceph-mon[98694]: pgmap v18512: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 760 KiB/s wr, 13 op/s 2023-09-10T08:53:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34414.588898 secs 2023-09-10T08:53:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:03 smithi098 ceph-mon[103124]: pgmap v18512: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 760 KiB/s wr, 13 op/s 2023-09-10T08:53:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34414.588898 secs 2023-09-10T08:53:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:03 smithi170 ceph-mon[102832]: pgmap v18512: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 760 KiB/s wr, 13 op/s 2023-09-10T08:53:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34414.588898 secs 2023-09-10T08:53:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34415.839087 secs 2023-09-10T08:53:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:53:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:53:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34416.193373 secs 2023-09-10T08:53:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34415.839087 secs 2023-09-10T08:53:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:53:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:53:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34416.193373 secs 2023-09-10T08:53:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34415.839087 secs 2023-09-10T08:53:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:53:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:53:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34416.193373 secs 2023-09-10T08:53:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:05 smithi098 ceph-mon[103124]: pgmap v18513: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 760 KiB/s wr, 13 op/s 2023-09-10T08:53:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:05 smithi170 ceph-mon[102832]: pgmap v18513: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 760 KiB/s wr, 13 op/s 2023-09-10T08:53:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:05 smithi082 ceph-mon[98694]: pgmap v18513: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 760 KiB/s wr, 13 op/s 2023-09-10T08:53:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:53:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:53:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:53:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:07 smithi098 ceph-mon[103124]: pgmap v18514: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 806 KiB/s wr, 34 op/s 2023-09-10T08:53:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:07 smithi170 ceph-mon[102832]: pgmap v18514: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 806 KiB/s wr, 34 op/s 2023-09-10T08:53:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:07 smithi082 ceph-mon[98694]: pgmap v18514: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 806 KiB/s wr, 34 op/s 2023-09-10T08:53:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34419.588970 secs 2023-09-10T08:53:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34419.588970 secs 2023-09-10T08:53:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34419.588970 secs 2023-09-10T08:53:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:09 smithi098 ceph-mon[103124]: pgmap v18515: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 744 KiB/s wr, 28 op/s 2023-09-10T08:53:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34420.839146 secs 2023-09-10T08:53:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:09 smithi098 ceph-mon[103124]: osdmap e1225: 12 total, 12 up, 12 in 2023-09-10T08:53:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34421.193437 secs 2023-09-10T08:53:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:09 smithi170 ceph-mon[102832]: pgmap v18515: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 744 KiB/s wr, 28 op/s 2023-09-10T08:53:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34420.839146 secs 2023-09-10T08:53:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:09 smithi170 ceph-mon[102832]: osdmap e1225: 12 total, 12 up, 12 in 2023-09-10T08:53:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34421.193437 secs 2023-09-10T08:53:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:09 smithi082 ceph-mon[98694]: pgmap v18515: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 744 KiB/s wr, 28 op/s 2023-09-10T08:53:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34420.839146 secs 2023-09-10T08:53:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:09 smithi082 ceph-mon[98694]: osdmap e1225: 12 total, 12 up, 12 in 2023-09-10T08:53:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34421.193437 secs 2023-09-10T08:53:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:11 smithi098 ceph-mon[103124]: pgmap v18517: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 407 KiB/s wr, 33 op/s 2023-09-10T08:53:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:11 smithi170 ceph-mon[102832]: pgmap v18517: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 407 KiB/s wr, 33 op/s 2023-09-10T08:53:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:11 smithi082 ceph-mon[98694]: pgmap v18517: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 407 KiB/s wr, 33 op/s 2023-09-10T08:53:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:13 smithi098 ceph-mon[103124]: pgmap v18518: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 407 KiB/s wr, 33 op/s 2023-09-10T08:53:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34424.589035 secs 2023-09-10T08:53:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:13 smithi098 ceph-mon[103124]: osdmap e1226: 12 total, 12 up, 12 in 2023-09-10T08:53:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:13 smithi170 ceph-mon[102832]: pgmap v18518: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 407 KiB/s wr, 33 op/s 2023-09-10T08:53:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34424.589035 secs 2023-09-10T08:53:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:13 smithi170 ceph-mon[102832]: osdmap e1226: 12 total, 12 up, 12 in 2023-09-10T08:53:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:13 smithi082 ceph-mon[98694]: pgmap v18518: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 407 KiB/s wr, 33 op/s 2023-09-10T08:53:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34424.589035 secs 2023-09-10T08:53:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:13 smithi082 ceph-mon[98694]: osdmap e1226: 12 total, 12 up, 12 in 2023-09-10T08:53:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34425.839208 secs 2023-09-10T08:53:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:14 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:53:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:14 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34426.193487 secs 2023-09-10T08:53:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34425.839208 secs 2023-09-10T08:53:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:14 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:53:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:14 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34426.193487 secs 2023-09-10T08:53:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34425.839208 secs 2023-09-10T08:53:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:14 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:53:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:14 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34426.193487 secs 2023-09-10T08:53:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:15 smithi098 ceph-mon[103124]: pgmap v18520: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 440 KiB/s wr, 10 op/s 2023-09-10T08:53:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:15 smithi170 ceph-mon[102832]: pgmap v18520: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 440 KiB/s wr, 10 op/s 2023-09-10T08:53:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:15 smithi082 ceph-mon[98694]: pgmap v18520: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 440 KiB/s wr, 10 op/s 2023-09-10T08:53:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:53:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:53:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:53:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:17 smithi170 ceph-mon[102832]: pgmap v18521: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 441 KiB/s wr, 18 op/s 2023-09-10T08:53:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:17 smithi082 ceph-mon[98694]: pgmap v18521: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 441 KiB/s wr, 18 op/s 2023-09-10T08:53:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:17 smithi098 ceph-mon[103124]: pgmap v18521: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 441 KiB/s wr, 18 op/s 2023-09-10T08:53:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34429.589094 secs 2023-09-10T08:53:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34429.589094 secs 2023-09-10T08:53:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34429.589094 secs 2023-09-10T08:53:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:19 smithi170 ceph-mon[102832]: pgmap v18522: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 359 KiB/s wr, 15 op/s 2023-09-10T08:53:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34430.839266 secs 2023-09-10T08:53:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34431.193550 secs 2023-09-10T08:53:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:19 smithi082 ceph-mon[98694]: pgmap v18522: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 359 KiB/s wr, 15 op/s 2023-09-10T08:53:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34430.839266 secs 2023-09-10T08:53:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34431.193550 secs 2023-09-10T08:53:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:19 smithi098 ceph-mon[103124]: pgmap v18522: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 359 KiB/s wr, 15 op/s 2023-09-10T08:53:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34430.839266 secs 2023-09-10T08:53:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34431.193550 secs 2023-09-10T08:53:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:21 smithi170 ceph-mon[102832]: pgmap v18523: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 13 op/s 2023-09-10T08:53:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:21 smithi082 ceph-mon[98694]: pgmap v18523: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 13 op/s 2023-09-10T08:53:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:21 smithi098 ceph-mon[103124]: pgmap v18523: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 13 op/s 2023-09-10T08:53:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:23 smithi170 ceph-mon[102832]: pgmap v18524: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 13 op/s 2023-09-10T08:53:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34434.589155 secs 2023-09-10T08:53:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:23 smithi082 ceph-mon[98694]: pgmap v18524: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 13 op/s 2023-09-10T08:53:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34434.589155 secs 2023-09-10T08:53:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:23 smithi098 ceph-mon[103124]: pgmap v18524: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 13 op/s 2023-09-10T08:53:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34434.589155 secs 2023-09-10T08:53:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34435.839349 secs 2023-09-10T08:53:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34436.193616 secs 2023-09-10T08:53:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34435.839349 secs 2023-09-10T08:53:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34436.193616 secs 2023-09-10T08:53:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34435.839349 secs 2023-09-10T08:53:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34436.193616 secs 2023-09-10T08:53:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:25 smithi170 ceph-mon[102832]: pgmap v18525: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 12 op/s 2023-09-10T08:53:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:25 smithi082 ceph-mon[98694]: pgmap v18525: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 12 op/s 2023-09-10T08:53:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:25 smithi098 ceph-mon[103124]: pgmap v18525: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 12 op/s 2023-09-10T08:53:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:27 smithi170 ceph-mon[102832]: pgmap v18526: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 16 op/s 2023-09-10T08:53:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:27 smithi082 ceph-mon[98694]: pgmap v18526: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 16 op/s 2023-09-10T08:53:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:27 smithi098 ceph-mon[103124]: pgmap v18526: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 16 op/s 2023-09-10T08:53:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34439.589259 secs 2023-09-10T08:53:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34439.589259 secs 2023-09-10T08:53:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34439.589259 secs 2023-09-10T08:53:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:29 smithi170 ceph-mon[102832]: pgmap v18527: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T08:53:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34440.839406 secs 2023-09-10T08:53:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34441.193677 secs 2023-09-10T08:53:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:29 smithi082 ceph-mon[98694]: pgmap v18527: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T08:53:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34440.839406 secs 2023-09-10T08:53:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34441.193677 secs 2023-09-10T08:53:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:29 smithi098 ceph-mon[103124]: pgmap v18527: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T08:53:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34440.839406 secs 2023-09-10T08:53:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34441.193677 secs 2023-09-10T08:53:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:31 smithi170 ceph-mon[102832]: pgmap v18528: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 270 KiB/s wr, 16 op/s 2023-09-10T08:53:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:31 smithi082 ceph-mon[98694]: pgmap v18528: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 270 KiB/s wr, 16 op/s 2023-09-10T08:53:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:31 smithi098 ceph-mon[103124]: pgmap v18528: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 270 KiB/s wr, 16 op/s 2023-09-10T08:53:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:33 smithi170 ceph-mon[102832]: pgmap v18529: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T08:53:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:33 smithi082 ceph-mon[98694]: pgmap v18529: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T08:53:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:33 smithi098 ceph-mon[103124]: pgmap v18529: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T08:53:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34445.839465 secs 2023-09-10T08:53:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34446.193738 secs 2023-09-10T08:53:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34445.839465 secs 2023-09-10T08:53:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34446.193738 secs 2023-09-10T08:53:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34445.839465 secs 2023-09-10T08:53:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34446.193738 secs 2023-09-10T08:53:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:35 smithi170 ceph-mon[102832]: pgmap v18530: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T08:53:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:53:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:35 smithi082 ceph-mon[98694]: pgmap v18530: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T08:53:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:53:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:35 smithi098 ceph-mon[103124]: pgmap v18530: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T08:53:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:53:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:53:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:53:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:53:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:53:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:53:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:53:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:37 smithi170 ceph-mon[102832]: pgmap v18531: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 247 KiB/s wr, 16 op/s 2023-09-10T08:53:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:37 smithi082 ceph-mon[98694]: pgmap v18531: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 247 KiB/s wr, 16 op/s 2023-09-10T08:53:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:37 smithi098 ceph-mon[103124]: pgmap v18531: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 247 KiB/s wr, 16 op/s 2023-09-10T08:53:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:53:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:53:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:53:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:39 smithi170 ceph-mon[102832]: pgmap v18532: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 246 KiB/s wr, 11 op/s 2023-09-10T08:53:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34450.839495 secs 2023-09-10T08:53:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34451.193797 secs 2023-09-10T08:53:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:39 smithi082 ceph-mon[98694]: pgmap v18532: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 246 KiB/s wr, 11 op/s 2023-09-10T08:53:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34450.839495 secs 2023-09-10T08:53:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34451.193797 secs 2023-09-10T08:53:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:39 smithi098 ceph-mon[103124]: pgmap v18532: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 246 KiB/s wr, 11 op/s 2023-09-10T08:53:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34450.839495 secs 2023-09-10T08:53:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34451.193797 secs 2023-09-10T08:53:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:53:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:53:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:53:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:41 smithi170 ceph-mon[102832]: pgmap v18533: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 909 KiB/s wr, 19 op/s 2023-09-10T08:53:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:41 smithi082 ceph-mon[98694]: pgmap v18533: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 909 KiB/s wr, 19 op/s 2023-09-10T08:53:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:41 smithi098 ceph-mon[103124]: pgmap v18533: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 909 KiB/s wr, 19 op/s 2023-09-10T08:53:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:43 smithi170 ceph-mon[102832]: pgmap v18534: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 785 KiB/s wr, 13 op/s 2023-09-10T08:53:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:43 smithi082 ceph-mon[98694]: pgmap v18534: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 785 KiB/s wr, 13 op/s 2023-09-10T08:53:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:43 smithi098 ceph-mon[103124]: pgmap v18534: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 785 KiB/s wr, 13 op/s 2023-09-10T08:53:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34455.839554 secs 2023-09-10T08:53:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:53:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34456.193861 secs 2023-09-10T08:53:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34455.839554 secs 2023-09-10T08:53:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:53:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34456.193861 secs 2023-09-10T08:53:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34455.839554 secs 2023-09-10T08:53:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:53:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34456.193861 secs 2023-09-10T08:53:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:45 smithi170 ceph-mon[102832]: pgmap v18535: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 785 KiB/s wr, 13 op/s 2023-09-10T08:53:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:45 smithi082 ceph-mon[98694]: pgmap v18535: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 785 KiB/s wr, 13 op/s 2023-09-10T08:53:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:45 smithi098 ceph-mon[103124]: pgmap v18535: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 785 KiB/s wr, 13 op/s 2023-09-10T08:53:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:47 smithi170 ceph-mon[102832]: pgmap v18536: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 803 KiB/s wr, 20 op/s 2023-09-10T08:53:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:47 smithi082 ceph-mon[98694]: pgmap v18536: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 803 KiB/s wr, 20 op/s 2023-09-10T08:53:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:47 smithi098 ceph-mon[103124]: pgmap v18536: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 803 KiB/s wr, 20 op/s 2023-09-10T08:53:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:49 smithi170 ceph-mon[102832]: pgmap v18537: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 681 KiB/s wr, 14 op/s 2023-09-10T08:53:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34460.839609 secs 2023-09-10T08:53:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:53:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34461.193928 secs 2023-09-10T08:53:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:49 smithi082 ceph-mon[98694]: pgmap v18537: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 681 KiB/s wr, 14 op/s 2023-09-10T08:53:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34460.839609 secs 2023-09-10T08:53:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:53:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34461.193928 secs 2023-09-10T08:53:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:49 smithi098 ceph-mon[103124]: pgmap v18537: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 681 KiB/s wr, 14 op/s 2023-09-10T08:53:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34460.839609 secs 2023-09-10T08:53:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:53:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34461.193928 secs 2023-09-10T08:53:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:53:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:53:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:53:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:51 smithi098 ceph-mon[103124]: pgmap v18538: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T08:53:51.653 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:51 smithi082 ceph-mon[98694]: pgmap v18538: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T08:53:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:51 smithi170 ceph-mon[102832]: pgmap v18538: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T08:53:52.983 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:53:52.983 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:53:52.983 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:53:52.984 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:53:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:53:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:53:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:53:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:53:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:53:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:53:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:53:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:53:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:53 smithi098 ceph-mon[103124]: pgmap v18539: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 456 KiB/s wr, 14 op/s 2023-09-10T08:53:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:53:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:53:54.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:53:54.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:53:54.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:53:54.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:53:54.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:53 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34465.839669 secs 2023-09-10T08:53:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:53 smithi170 ceph-mon[102832]: pgmap v18539: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 456 KiB/s wr, 14 op/s 2023-09-10T08:53:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:53:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:53:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:53:54.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:53:54.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:53:54.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:53:54.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:53 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34465.839669 secs 2023-09-10T08:53:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:53 smithi082 ceph-mon[98694]: pgmap v18539: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 456 KiB/s wr, 14 op/s 2023-09-10T08:53:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:53:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:53:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:53:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:53:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:53:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:53:54.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:53 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34465.839669 secs 2023-09-10T08:53:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34466.193990 secs 2023-09-10T08:53:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34466.193990 secs 2023-09-10T08:53:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34466.193990 secs 2023-09-10T08:53:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:55 smithi098 ceph-mon[103124]: pgmap v18540: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 456 KiB/s wr, 14 op/s 2023-09-10T08:53:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:55 smithi170 ceph-mon[102832]: pgmap v18540: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 456 KiB/s wr, 14 op/s 2023-09-10T08:53:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:55 smithi082 ceph-mon[98694]: pgmap v18540: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 456 KiB/s wr, 14 op/s 2023-09-10T08:53:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:53:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:53:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:53:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:53:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:57 smithi098 ceph-mon[103124]: pgmap v18541: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 932 KiB/s wr, 21 op/s 2023-09-10T08:53:58.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:57 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34469.589680 secs 2023-09-10T08:53:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:57 smithi170 ceph-mon[102832]: pgmap v18541: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 932 KiB/s wr, 21 op/s 2023-09-10T08:53:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:57 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34469.589680 secs 2023-09-10T08:53:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:57 smithi082 ceph-mon[98694]: pgmap v18541: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 932 KiB/s wr, 21 op/s 2023-09-10T08:53:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:57 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34469.589680 secs 2023-09-10T08:53:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:59 smithi170 ceph-mon[102832]: pgmap v18542: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 914 KiB/s wr, 15 op/s 2023-09-10T08:53:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34470.839729 secs 2023-09-10T08:53:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:53:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34471.194052 secs 2023-09-10T08:53:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:59 smithi082 ceph-mon[98694]: pgmap v18542: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 914 KiB/s wr, 15 op/s 2023-09-10T08:53:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34470.839729 secs 2023-09-10T08:53:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:53:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34471.194052 secs 2023-09-10T08:53:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:59 smithi098 ceph-mon[103124]: pgmap v18542: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 914 KiB/s wr, 15 op/s 2023-09-10T08:53:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34470.839729 secs 2023-09-10T08:53:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:53:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34471.194052 secs 2023-09-10T08:54:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:54:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:54:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:54:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:54:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:54:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:54:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:01.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:01 smithi170 ceph-mon[102832]: pgmap v18543: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T08:54:01.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:01 smithi082 ceph-mon[98694]: pgmap v18543: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T08:54:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:01 smithi098 ceph-mon[103124]: pgmap v18543: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T08:54:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:03 smithi170 ceph-mon[102832]: pgmap v18544: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 636 KiB/s wr, 13 op/s 2023-09-10T08:54:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34474.589770 secs 2023-09-10T08:54:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:03 smithi082 ceph-mon[98694]: pgmap v18544: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 636 KiB/s wr, 13 op/s 2023-09-10T08:54:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34474.589770 secs 2023-09-10T08:54:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:03 smithi098 ceph-mon[103124]: pgmap v18544: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 636 KiB/s wr, 13 op/s 2023-09-10T08:54:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34474.589770 secs 2023-09-10T08:54:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34475.839785 secs 2023-09-10T08:54:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:54:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34476.194112 secs 2023-09-10T08:54:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34475.839785 secs 2023-09-10T08:54:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:54:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34476.194112 secs 2023-09-10T08:54:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34475.839785 secs 2023-09-10T08:54:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:54:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34476.194112 secs 2023-09-10T08:54:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:05 smithi170 ceph-mon[102832]: pgmap v18545: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 636 KiB/s wr, 13 op/s 2023-09-10T08:54:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:05 smithi082 ceph-mon[98694]: pgmap v18545: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 636 KiB/s wr, 13 op/s 2023-09-10T08:54:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:05 smithi098 ceph-mon[103124]: pgmap v18545: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 636 KiB/s wr, 13 op/s 2023-09-10T08:54:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:54:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:54:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:54:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:07 smithi170 ceph-mon[102832]: pgmap v18546: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 830 KiB/s wr, 34 op/s 2023-09-10T08:54:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:07 smithi082 ceph-mon[98694]: pgmap v18546: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 830 KiB/s wr, 34 op/s 2023-09-10T08:54:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:07 smithi098 ceph-mon[103124]: pgmap v18546: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 830 KiB/s wr, 34 op/s 2023-09-10T08:54:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34479.589831 secs 2023-09-10T08:54:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34479.589831 secs 2023-09-10T08:54:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34479.589831 secs 2023-09-10T08:54:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:09 smithi170 ceph-mon[102832]: pgmap v18547: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 354 KiB/s wr, 26 op/s 2023-09-10T08:54:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34480.839843 secs 2023-09-10T08:54:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:09 smithi170 ceph-mon[102832]: osdmap e1227: 12 total, 12 up, 12 in 2023-09-10T08:54:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34481.194172 secs 2023-09-10T08:54:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:09 smithi082 ceph-mon[98694]: pgmap v18547: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 354 KiB/s wr, 26 op/s 2023-09-10T08:54:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34480.839843 secs 2023-09-10T08:54:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:09 smithi082 ceph-mon[98694]: osdmap e1227: 12 total, 12 up, 12 in 2023-09-10T08:54:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34481.194172 secs 2023-09-10T08:54:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:09 smithi098 ceph-mon[103124]: pgmap v18547: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 354 KiB/s wr, 26 op/s 2023-09-10T08:54:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34480.839843 secs 2023-09-10T08:54:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:09 smithi098 ceph-mon[103124]: osdmap e1227: 12 total, 12 up, 12 in 2023-09-10T08:54:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34481.194172 secs 2023-09-10T08:54:11.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:11 smithi170 ceph-mon[102832]: pgmap v18549: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 382 KiB/s wr, 31 op/s 2023-09-10T08:54:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:11 smithi082 ceph-mon[98694]: pgmap v18549: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 382 KiB/s wr, 31 op/s 2023-09-10T08:54:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:11 smithi098 ceph-mon[103124]: pgmap v18549: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 382 KiB/s wr, 31 op/s 2023-09-10T08:54:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:13 smithi170 ceph-mon[102832]: pgmap v18550: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 382 KiB/s wr, 31 op/s 2023-09-10T08:54:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34484.589889 secs 2023-09-10T08:54:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:13 smithi170 ceph-mon[102832]: osdmap e1228: 12 total, 12 up, 12 in 2023-09-10T08:54:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:13 smithi082 ceph-mon[98694]: pgmap v18550: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 382 KiB/s wr, 31 op/s 2023-09-10T08:54:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34484.589889 secs 2023-09-10T08:54:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:13 smithi082 ceph-mon[98694]: osdmap e1228: 12 total, 12 up, 12 in 2023-09-10T08:54:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:13 smithi098 ceph-mon[103124]: pgmap v18550: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 382 KiB/s wr, 31 op/s 2023-09-10T08:54:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34484.589889 secs 2023-09-10T08:54:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:13 smithi098 ceph-mon[103124]: osdmap e1228: 12 total, 12 up, 12 in 2023-09-10T08:54:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34485.839902 secs 2023-09-10T08:54:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34486.194232 secs 2023-09-10T08:54:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34485.839902 secs 2023-09-10T08:54:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34486.194232 secs 2023-09-10T08:54:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34485.839902 secs 2023-09-10T08:54:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34486.194232 secs 2023-09-10T08:54:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:15 smithi170 ceph-mon[102832]: pgmap v18552: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 186 KiB/s wr, 9 op/s 2023-09-10T08:54:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:15 smithi082 ceph-mon[98694]: pgmap v18552: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 186 KiB/s wr, 9 op/s 2023-09-10T08:54:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:15 smithi098 ceph-mon[103124]: pgmap v18552: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 186 KiB/s wr, 9 op/s 2023-09-10T08:54:17.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:17 smithi170 ceph-mon[102832]: pgmap v18553: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 226 KiB/s wr, 18 op/s 2023-09-10T08:54:17.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:17 smithi082 ceph-mon[98694]: pgmap v18553: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 226 KiB/s wr, 18 op/s 2023-09-10T08:54:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:17 smithi098 ceph-mon[103124]: pgmap v18553: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 226 KiB/s wr, 18 op/s 2023-09-10T08:54:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34489.589950 secs 2023-09-10T08:54:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34489.589950 secs 2023-09-10T08:54:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34489.589950 secs 2023-09-10T08:54:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:19 smithi170 ceph-mon[102832]: pgmap v18554: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 182 KiB/s wr, 14 op/s 2023-09-10T08:54:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34490.839961 secs 2023-09-10T08:54:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34491.194292 secs 2023-09-10T08:54:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:19 smithi082 ceph-mon[98694]: pgmap v18554: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 182 KiB/s wr, 14 op/s 2023-09-10T08:54:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34490.839961 secs 2023-09-10T08:54:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34491.194292 secs 2023-09-10T08:54:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:19 smithi098 ceph-mon[103124]: pgmap v18554: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 182 KiB/s wr, 14 op/s 2023-09-10T08:54:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34490.839961 secs 2023-09-10T08:54:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34491.194292 secs 2023-09-10T08:54:21.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:21 smithi170 ceph-mon[102832]: pgmap v18555: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 13 op/s 2023-09-10T08:54:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:21 smithi082 ceph-mon[98694]: pgmap v18555: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 13 op/s 2023-09-10T08:54:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:21 smithi098 ceph-mon[103124]: pgmap v18555: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 13 op/s 2023-09-10T08:54:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:23 smithi082 ceph-mon[98694]: pgmap v18556: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 13 op/s 2023-09-10T08:54:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34494.590045 secs 2023-09-10T08:54:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:23 smithi098 ceph-mon[103124]: pgmap v18556: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 13 op/s 2023-09-10T08:54:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34494.590045 secs 2023-09-10T08:54:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:23 smithi170 ceph-mon[102832]: pgmap v18556: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 13 op/s 2023-09-10T08:54:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34494.590045 secs 2023-09-10T08:54:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34495.840020 secs 2023-09-10T08:54:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34496.194376 secs 2023-09-10T08:54:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34495.840020 secs 2023-09-10T08:54:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34496.194376 secs 2023-09-10T08:54:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34495.840020 secs 2023-09-10T08:54:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34496.194376 secs 2023-09-10T08:54:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:25 smithi082 ceph-mon[98694]: pgmap v18557: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 134 KiB/s wr, 12 op/s 2023-09-10T08:54:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:25 smithi098 ceph-mon[103124]: pgmap v18557: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 134 KiB/s wr, 12 op/s 2023-09-10T08:54:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:25 smithi170 ceph-mon[102832]: pgmap v18557: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 134 KiB/s wr, 12 op/s 2023-09-10T08:54:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:27 smithi082 ceph-mon[98694]: pgmap v18558: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 250 KiB/s wr, 17 op/s 2023-09-10T08:54:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:27 smithi098 ceph-mon[103124]: pgmap v18558: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 250 KiB/s wr, 17 op/s 2023-09-10T08:54:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:27 smithi170 ceph-mon[102832]: pgmap v18558: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 250 KiB/s wr, 17 op/s 2023-09-10T08:54:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34499.590108 secs 2023-09-10T08:54:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34499.590108 secs 2023-09-10T08:54:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34499.590108 secs 2023-09-10T08:54:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:29 smithi082 ceph-mon[98694]: pgmap v18559: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 224 KiB/s wr, 11 op/s 2023-09-10T08:54:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34500.840080 secs 2023-09-10T08:54:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34501.194435 secs 2023-09-10T08:54:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:29 smithi098 ceph-mon[103124]: pgmap v18559: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 224 KiB/s wr, 11 op/s 2023-09-10T08:54:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34500.840080 secs 2023-09-10T08:54:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34501.194435 secs 2023-09-10T08:54:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:29 smithi170 ceph-mon[102832]: pgmap v18559: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 224 KiB/s wr, 11 op/s 2023-09-10T08:54:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34500.840080 secs 2023-09-10T08:54:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34501.194435 secs 2023-09-10T08:54:31.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:31 smithi082 ceph-mon[98694]: pgmap v18560: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 224 KiB/s wr, 16 op/s 2023-09-10T08:54:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:31 smithi098 ceph-mon[103124]: pgmap v18560: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 224 KiB/s wr, 16 op/s 2023-09-10T08:54:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:31 smithi170 ceph-mon[102832]: pgmap v18560: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 224 KiB/s wr, 16 op/s 2023-09-10T08:54:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:33 smithi082 ceph-mon[98694]: pgmap v18561: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T08:54:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:33 smithi098 ceph-mon[103124]: pgmap v18561: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T08:54:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:33 smithi170 ceph-mon[102832]: pgmap v18561: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T08:54:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34505.840143 secs 2023-09-10T08:54:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:54:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34506.194497 secs 2023-09-10T08:54:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34505.840143 secs 2023-09-10T08:54:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:54:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34506.194497 secs 2023-09-10T08:54:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34505.840143 secs 2023-09-10T08:54:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:54:35.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:35.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34506.194497 secs 2023-09-10T08:54:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:35 smithi098 ceph-mon[103124]: pgmap v18562: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T08:54:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:54:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:35 smithi170 ceph-mon[102832]: pgmap v18562: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T08:54:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:54:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:35 smithi082 ceph-mon[98694]: pgmap v18562: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 11 op/s 2023-09-10T08:54:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:54:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:54:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:54:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:54:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:54:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:54:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:54:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:38.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:37 smithi098 ceph-mon[103124]: pgmap v18563: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 349 KiB/s wr, 17 op/s 2023-09-10T08:54:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:37 smithi170 ceph-mon[102832]: pgmap v18563: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 349 KiB/s wr, 17 op/s 2023-09-10T08:54:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:37 smithi082 ceph-mon[98694]: pgmap v18563: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 349 KiB/s wr, 17 op/s 2023-09-10T08:54:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:39 smithi170 ceph-mon[102832]: pgmap v18564: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 225 KiB/s wr, 11 op/s 2023-09-10T08:54:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34510.840202 secs 2023-09-10T08:54:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34511.194558 secs 2023-09-10T08:54:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:39 smithi082 ceph-mon[98694]: pgmap v18564: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 225 KiB/s wr, 11 op/s 2023-09-10T08:54:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34510.840202 secs 2023-09-10T08:54:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34511.194558 secs 2023-09-10T08:54:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:39 smithi098 ceph-mon[103124]: pgmap v18564: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 225 KiB/s wr, 11 op/s 2023-09-10T08:54:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34510.840202 secs 2023-09-10T08:54:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34511.194558 secs 2023-09-10T08:54:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:41 smithi170 ceph-mon[102832]: pgmap v18565: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 419 KiB/s wr, 18 op/s 2023-09-10T08:54:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:41 smithi082 ceph-mon[98694]: pgmap v18565: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 419 KiB/s wr, 18 op/s 2023-09-10T08:54:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:41 smithi098 ceph-mon[103124]: pgmap v18565: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 419 KiB/s wr, 18 op/s 2023-09-10T08:54:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:43 smithi170 ceph-mon[102832]: pgmap v18566: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 418 KiB/s wr, 12 op/s 2023-09-10T08:54:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:43 smithi082 ceph-mon[98694]: pgmap v18566: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 418 KiB/s wr, 12 op/s 2023-09-10T08:54:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:43 smithi098 ceph-mon[103124]: pgmap v18566: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 418 KiB/s wr, 12 op/s 2023-09-10T08:54:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34515.840259 secs 2023-09-10T08:54:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:54:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34516.194623 secs 2023-09-10T08:54:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34515.840259 secs 2023-09-10T08:54:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:54:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34516.194623 secs 2023-09-10T08:54:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34515.840259 secs 2023-09-10T08:54:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:54:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34516.194623 secs 2023-09-10T08:54:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:45 smithi170 ceph-mon[102832]: pgmap v18567: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 418 KiB/s wr, 12 op/s 2023-09-10T08:54:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:45 smithi082 ceph-mon[98694]: pgmap v18567: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 418 KiB/s wr, 12 op/s 2023-09-10T08:54:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:45 smithi098 ceph-mon[103124]: pgmap v18567: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 418 KiB/s wr, 12 op/s 2023-09-10T08:54:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:54:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:54:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:54:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:47 smithi170 ceph-mon[102832]: pgmap v18568: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 489 KiB/s wr, 19 op/s 2023-09-10T08:54:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:47 smithi082 ceph-mon[98694]: pgmap v18568: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 489 KiB/s wr, 19 op/s 2023-09-10T08:54:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:47 smithi098 ceph-mon[103124]: pgmap v18568: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 489 KiB/s wr, 19 op/s 2023-09-10T08:54:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:49 smithi170 ceph-mon[102832]: pgmap v18569: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 265 KiB/s wr, 13 op/s 2023-09-10T08:54:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34520.840342 secs 2023-09-10T08:54:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34521.194685 secs 2023-09-10T08:54:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:49 smithi082 ceph-mon[98694]: pgmap v18569: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 265 KiB/s wr, 13 op/s 2023-09-10T08:54:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34520.840342 secs 2023-09-10T08:54:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34521.194685 secs 2023-09-10T08:54:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:49 smithi098 ceph-mon[103124]: pgmap v18569: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 265 KiB/s wr, 13 op/s 2023-09-10T08:54:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34520.840342 secs 2023-09-10T08:54:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34521.194685 secs 2023-09-10T08:54:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:54:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:54:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:54:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:51 smithi170 ceph-mon[102832]: pgmap v18570: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 763 KiB/s wr, 22 op/s 2023-09-10T08:54:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:51 smithi082 ceph-mon[98694]: pgmap v18570: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 763 KiB/s wr, 22 op/s 2023-09-10T08:54:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:51 smithi098 ceph-mon[103124]: pgmap v18570: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 763 KiB/s wr, 22 op/s 2023-09-10T08:54:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:54:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:54:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:54:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:53 smithi170 ceph-mon[102832]: pgmap v18571: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 569 KiB/s wr, 15 op/s 2023-09-10T08:54:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:54:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:53 smithi082 ceph-mon[98694]: pgmap v18571: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 569 KiB/s wr, 15 op/s 2023-09-10T08:54:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:54:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:53 smithi098 ceph-mon[103124]: pgmap v18571: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 569 KiB/s wr, 15 op/s 2023-09-10T08:54:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:54:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34525.840401 secs 2023-09-10T08:54:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:54:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:54:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:54:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34526.194755 secs 2023-09-10T08:54:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34525.840401 secs 2023-09-10T08:54:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:54:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:54:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:54:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34526.194755 secs 2023-09-10T08:54:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34525.840401 secs 2023-09-10T08:54:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:54:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:54:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:54:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34526.194755 secs 2023-09-10T08:54:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:55 smithi170 ceph-mon[102832]: pgmap v18572: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 569 KiB/s wr, 15 op/s 2023-09-10T08:54:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:55 smithi082 ceph-mon[98694]: pgmap v18572: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 569 KiB/s wr, 15 op/s 2023-09-10T08:54:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:55 smithi098 ceph-mon[103124]: pgmap v18572: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 569 KiB/s wr, 15 op/s 2023-09-10T08:54:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:57 smithi170 ceph-mon[102832]: pgmap v18573: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 693 KiB/s wr, 22 op/s 2023-09-10T08:54:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:57 smithi082 ceph-mon[98694]: pgmap v18573: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 693 KiB/s wr, 22 op/s 2023-09-10T08:54:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:57 smithi098 ceph-mon[103124]: pgmap v18573: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 693 KiB/s wr, 22 op/s 2023-09-10T08:54:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:54:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34529.590520 secs 2023-09-10T08:54:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:54:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34529.590520 secs 2023-09-10T08:54:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:54:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:54:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34529.590520 secs 2023-09-10T08:54:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34530.840492 secs 2023-09-10T08:54:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:59 smithi170 ceph-mon[102832]: pgmap v18574: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 623 KiB/s wr, 15 op/s 2023-09-10T08:54:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:54:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34531.194816 secs 2023-09-10T08:54:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34530.840492 secs 2023-09-10T08:54:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:59 smithi082 ceph-mon[98694]: pgmap v18574: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 623 KiB/s wr, 15 op/s 2023-09-10T08:54:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:54:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34531.194816 secs 2023-09-10T08:55:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34530.840492 secs 2023-09-10T08:55:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:59 smithi098 ceph-mon[103124]: pgmap v18574: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 623 KiB/s wr, 15 op/s 2023-09-10T08:55:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:54:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34531.194816 secs 2023-09-10T08:55:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:55:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:55:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:55:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:55:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:55:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:55:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:01 smithi170 ceph-mon[102832]: pgmap v18575: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 36 op/s 2023-09-10T08:55:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:01 smithi082 ceph-mon[98694]: pgmap v18575: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 36 op/s 2023-09-10T08:55:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:01 smithi098 ceph-mon[103124]: pgmap v18575: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 36 op/s 2023-09-10T08:55:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:03 smithi170 ceph-mon[102832]: pgmap v18576: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 705 KiB/s wr, 27 op/s 2023-09-10T08:55:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34534.590605 secs 2023-09-10T08:55:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:03 smithi082 ceph-mon[98694]: pgmap v18576: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 705 KiB/s wr, 27 op/s 2023-09-10T08:55:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34534.590605 secs 2023-09-10T08:55:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:03 smithi098 ceph-mon[103124]: pgmap v18576: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 705 KiB/s wr, 27 op/s 2023-09-10T08:55:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34534.590605 secs 2023-09-10T08:55:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34535.840523 secs 2023-09-10T08:55:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34536.194877 secs 2023-09-10T08:55:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34535.840523 secs 2023-09-10T08:55:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34536.194877 secs 2023-09-10T08:55:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34535.840523 secs 2023-09-10T08:55:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34536.194877 secs 2023-09-10T08:55:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:05 smithi170 ceph-mon[102832]: pgmap v18577: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 705 KiB/s wr, 27 op/s 2023-09-10T08:55:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:05 smithi082 ceph-mon[98694]: pgmap v18577: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 705 KiB/s wr, 27 op/s 2023-09-10T08:55:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:05 smithi098 ceph-mon[103124]: pgmap v18577: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 705 KiB/s wr, 27 op/s 2023-09-10T08:55:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:07 smithi170 ceph-mon[102832]: pgmap v18578: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 857 KiB/s wr, 35 op/s 2023-09-10T08:55:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:07 smithi082 ceph-mon[98694]: pgmap v18578: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 857 KiB/s wr, 35 op/s 2023-09-10T08:55:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:07 smithi098 ceph-mon[103124]: pgmap v18578: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 857 KiB/s wr, 35 op/s 2023-09-10T08:55:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34539.590669 secs 2023-09-10T08:55:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34539.590669 secs 2023-09-10T08:55:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34539.590669 secs 2023-09-10T08:55:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34540.840605 secs 2023-09-10T08:55:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:09 smithi082 ceph-mon[98694]: pgmap v18579: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 732 KiB/s wr, 29 op/s 2023-09-10T08:55:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:09 smithi082 ceph-mon[98694]: osdmap e1229: 12 total, 12 up, 12 in 2023-09-10T08:55:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34541.194938 secs 2023-09-10T08:55:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34540.840605 secs 2023-09-10T08:55:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:09 smithi098 ceph-mon[103124]: pgmap v18579: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 732 KiB/s wr, 29 op/s 2023-09-10T08:55:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:09 smithi098 ceph-mon[103124]: osdmap e1229: 12 total, 12 up, 12 in 2023-09-10T08:55:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34541.194938 secs 2023-09-10T08:55:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34540.840605 secs 2023-09-10T08:55:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:09 smithi170 ceph-mon[102832]: pgmap v18579: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 732 KiB/s wr, 29 op/s 2023-09-10T08:55:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:09 smithi170 ceph-mon[102832]: osdmap e1229: 12 total, 12 up, 12 in 2023-09-10T08:55:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34541.194938 secs 2023-09-10T08:55:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:55:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:55:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:55:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:11 smithi082 ceph-mon[98694]: pgmap v18581: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 392 KiB/s wr, 17 op/s 2023-09-10T08:55:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:11 smithi098 ceph-mon[103124]: pgmap v18581: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 392 KiB/s wr, 17 op/s 2023-09-10T08:55:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:11 smithi170 ceph-mon[102832]: pgmap v18581: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 392 KiB/s wr, 17 op/s 2023-09-10T08:55:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:55:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:55:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:55:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:13 smithi082 ceph-mon[98694]: pgmap v18582: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 392 KiB/s wr, 17 op/s 2023-09-10T08:55:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34544.590756 secs 2023-09-10T08:55:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:13 smithi082 ceph-mon[98694]: osdmap e1230: 12 total, 12 up, 12 in 2023-09-10T08:55:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:13 smithi098 ceph-mon[103124]: pgmap v18582: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 392 KiB/s wr, 17 op/s 2023-09-10T08:55:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34544.590756 secs 2023-09-10T08:55:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:13 smithi098 ceph-mon[103124]: osdmap e1230: 12 total, 12 up, 12 in 2023-09-10T08:55:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:13 smithi170 ceph-mon[102832]: pgmap v18582: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 392 KiB/s wr, 17 op/s 2023-09-10T08:55:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34544.590756 secs 2023-09-10T08:55:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:13 smithi170 ceph-mon[102832]: osdmap e1230: 12 total, 12 up, 12 in 2023-09-10T08:55:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34545.840664 secs 2023-09-10T08:55:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34546.195002 secs 2023-09-10T08:55:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34545.840664 secs 2023-09-10T08:55:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34546.195002 secs 2023-09-10T08:55:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34545.840664 secs 2023-09-10T08:55:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34546.195002 secs 2023-09-10T08:55:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:15 smithi098 ceph-mon[103124]: pgmap v18584: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 264 KiB/s wr, 10 op/s 2023-09-10T08:55:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:15 smithi170 ceph-mon[102832]: pgmap v18584: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 264 KiB/s wr, 10 op/s 2023-09-10T08:55:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:15 smithi082 ceph-mon[98694]: pgmap v18584: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 264 KiB/s wr, 10 op/s 2023-09-10T08:55:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:17 smithi098 ceph-mon[103124]: pgmap v18585: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 412 KiB/s wr, 18 op/s 2023-09-10T08:55:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:17 smithi170 ceph-mon[102832]: pgmap v18585: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 412 KiB/s wr, 18 op/s 2023-09-10T08:55:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:17 smithi082 ceph-mon[98694]: pgmap v18585: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 412 KiB/s wr, 18 op/s 2023-09-10T08:55:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34549.590775 secs 2023-09-10T08:55:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34549.590775 secs 2023-09-10T08:55:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34549.590775 secs 2023-09-10T08:55:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34550.840749 secs 2023-09-10T08:55:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:19 smithi098 ceph-mon[103124]: pgmap v18586: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 332 KiB/s wr, 15 op/s 2023-09-10T08:55:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34551.195063 secs 2023-09-10T08:55:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34550.840749 secs 2023-09-10T08:55:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:19 smithi170 ceph-mon[102832]: pgmap v18586: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 332 KiB/s wr, 15 op/s 2023-09-10T08:55:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34551.195063 secs 2023-09-10T08:55:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34550.840749 secs 2023-09-10T08:55:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:19 smithi082 ceph-mon[98694]: pgmap v18586: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 332 KiB/s wr, 15 op/s 2023-09-10T08:55:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34551.195063 secs 2023-09-10T08:55:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:21 smithi098 ceph-mon[103124]: pgmap v18587: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T08:55:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:21 smithi170 ceph-mon[102832]: pgmap v18587: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T08:55:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:21 smithi082 ceph-mon[98694]: pgmap v18587: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T08:55:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:23 smithi098 ceph-mon[103124]: pgmap v18588: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T08:55:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34554.590835 secs 2023-09-10T08:55:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:23 smithi170 ceph-mon[102832]: pgmap v18588: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T08:55:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34554.590835 secs 2023-09-10T08:55:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:23 smithi082 ceph-mon[98694]: pgmap v18588: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 12 op/s 2023-09-10T08:55:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34554.590835 secs 2023-09-10T08:55:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34555.840807 secs 2023-09-10T08:55:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34556.195127 secs 2023-09-10T08:55:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34555.840807 secs 2023-09-10T08:55:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34556.195127 secs 2023-09-10T08:55:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34555.840807 secs 2023-09-10T08:55:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34556.195127 secs 2023-09-10T08:55:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:25 smithi098 ceph-mon[103124]: pgmap v18589: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 10 op/s 2023-09-10T08:55:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:25 smithi170 ceph-mon[102832]: pgmap v18589: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 10 op/s 2023-09-10T08:55:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:25 smithi082 ceph-mon[98694]: pgmap v18589: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 106 KiB/s wr, 10 op/s 2023-09-10T08:55:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:27 smithi098 ceph-mon[103124]: pgmap v18590: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 100 KiB/s wr, 15 op/s 2023-09-10T08:55:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:27 smithi170 ceph-mon[102832]: pgmap v18590: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 100 KiB/s wr, 15 op/s 2023-09-10T08:55:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:27 smithi082 ceph-mon[98694]: pgmap v18590: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 100 KiB/s wr, 15 op/s 2023-09-10T08:55:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34559.590893 secs 2023-09-10T08:55:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34559.590893 secs 2023-09-10T08:55:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34559.590893 secs 2023-09-10T08:55:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34560.840911 secs 2023-09-10T08:55:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:29 smithi098 ceph-mon[103124]: pgmap v18591: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2023-09-10T08:55:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34561.195190 secs 2023-09-10T08:55:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34560.840911 secs 2023-09-10T08:55:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:29 smithi170 ceph-mon[102832]: pgmap v18591: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2023-09-10T08:55:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34561.195190 secs 2023-09-10T08:55:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34560.840911 secs 2023-09-10T08:55:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:29 smithi082 ceph-mon[98694]: pgmap v18591: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2023-09-10T08:55:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34561.195190 secs 2023-09-10T08:55:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:31 smithi098 ceph-mon[103124]: pgmap v18592: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 126 KiB/s wr, 15 op/s 2023-09-10T08:55:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:31 smithi170 ceph-mon[102832]: pgmap v18592: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 126 KiB/s wr, 15 op/s 2023-09-10T08:55:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:31 smithi082 ceph-mon[98694]: pgmap v18592: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 126 KiB/s wr, 15 op/s 2023-09-10T08:55:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:33 smithi098 ceph-mon[103124]: pgmap v18593: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T08:55:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:33 smithi170 ceph-mon[102832]: pgmap v18593: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T08:55:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:33 smithi082 ceph-mon[98694]: pgmap v18593: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T08:55:35.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34565.840969 secs 2023-09-10T08:55:35.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34566.195252 secs 2023-09-10T08:55:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34565.840969 secs 2023-09-10T08:55:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34566.195252 secs 2023-09-10T08:55:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34565.840969 secs 2023-09-10T08:55:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34566.195252 secs 2023-09-10T08:55:36.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:35 smithi098 ceph-mon[103124]: pgmap v18594: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T08:55:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:55:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:35 smithi170 ceph-mon[102832]: pgmap v18594: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T08:55:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:55:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:35 smithi082 ceph-mon[98694]: pgmap v18594: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T08:55:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:55:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:55:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:55:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:55:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:55:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:55:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:55:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:37 smithi170 ceph-mon[102832]: pgmap v18595: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 359 KiB/s wr, 17 op/s 2023-09-10T08:55:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:37 smithi082 ceph-mon[98694]: pgmap v18595: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 359 KiB/s wr, 17 op/s 2023-09-10T08:55:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:37 smithi098 ceph-mon[103124]: pgmap v18595: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 359 KiB/s wr, 17 op/s 2023-09-10T08:55:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:55:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:55:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:55:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34570.841027 secs 2023-09-10T08:55:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:39 smithi170 ceph-mon[102832]: pgmap v18596: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 359 KiB/s wr, 11 op/s 2023-09-10T08:55:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34571.195313 secs 2023-09-10T08:55:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34570.841027 secs 2023-09-10T08:55:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:39 smithi082 ceph-mon[98694]: pgmap v18596: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 359 KiB/s wr, 11 op/s 2023-09-10T08:55:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34571.195313 secs 2023-09-10T08:55:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34570.841027 secs 2023-09-10T08:55:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:39 smithi098 ceph-mon[103124]: pgmap v18596: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 359 KiB/s wr, 11 op/s 2023-09-10T08:55:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34571.195313 secs 2023-09-10T08:55:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:55:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:55:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:55:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:41 smithi170 ceph-mon[102832]: pgmap v18597: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 908 KiB/s wr, 20 op/s 2023-09-10T08:55:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:41 smithi082 ceph-mon[98694]: pgmap v18597: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 908 KiB/s wr, 20 op/s 2023-09-10T08:55:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:41 smithi098 ceph-mon[103124]: pgmap v18597: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 908 KiB/s wr, 20 op/s 2023-09-10T08:55:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:43 smithi170 ceph-mon[102832]: pgmap v18598: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 783 KiB/s wr, 14 op/s 2023-09-10T08:55:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:43 smithi082 ceph-mon[98694]: pgmap v18598: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 783 KiB/s wr, 14 op/s 2023-09-10T08:55:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:43 smithi098 ceph-mon[103124]: pgmap v18598: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 783 KiB/s wr, 14 op/s 2023-09-10T08:55:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34575.841115 secs 2023-09-10T08:55:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34576.195375 secs 2023-09-10T08:55:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34575.841115 secs 2023-09-10T08:55:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34576.195375 secs 2023-09-10T08:55:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34575.841115 secs 2023-09-10T08:55:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34576.195375 secs 2023-09-10T08:55:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:45 smithi170 ceph-mon[102832]: pgmap v18599: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 783 KiB/s wr, 14 op/s 2023-09-10T08:55:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:45 smithi082 ceph-mon[98694]: pgmap v18599: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 783 KiB/s wr, 14 op/s 2023-09-10T08:55:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:45 smithi098 ceph-mon[103124]: pgmap v18599: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 783 KiB/s wr, 14 op/s 2023-09-10T08:55:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:55:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:55:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:55:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:47 smithi170 ceph-mon[102832]: pgmap v18600: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 807 KiB/s wr, 20 op/s 2023-09-10T08:55:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:47 smithi082 ceph-mon[98694]: pgmap v18600: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 807 KiB/s wr, 20 op/s 2023-09-10T08:55:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:47 smithi098 ceph-mon[103124]: pgmap v18600: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 807 KiB/s wr, 20 op/s 2023-09-10T08:55:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34580.841175 secs 2023-09-10T08:55:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:49 smithi170 ceph-mon[102832]: pgmap v18601: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 573 KiB/s wr, 14 op/s 2023-09-10T08:55:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34581.195436 secs 2023-09-10T08:55:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34580.841175 secs 2023-09-10T08:55:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:49 smithi082 ceph-mon[98694]: pgmap v18601: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 573 KiB/s wr, 14 op/s 2023-09-10T08:55:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34581.195436 secs 2023-09-10T08:55:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34580.841175 secs 2023-09-10T08:55:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:49 smithi098 ceph-mon[103124]: pgmap v18601: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 573 KiB/s wr, 14 op/s 2023-09-10T08:55:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34581.195436 secs 2023-09-10T08:55:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:55:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:55:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:55:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:51 smithi170 ceph-mon[102832]: pgmap v18602: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1013 KiB/s wr, 21 op/s 2023-09-10T08:55:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:51 smithi082 ceph-mon[98694]: pgmap v18602: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1013 KiB/s wr, 21 op/s 2023-09-10T08:55:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:51 smithi098 ceph-mon[103124]: pgmap v18602: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1013 KiB/s wr, 21 op/s 2023-09-10T08:55:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:55:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:55:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:55:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:53 smithi170 ceph-mon[102832]: pgmap v18603: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 464 KiB/s wr, 13 op/s 2023-09-10T08:55:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:53 smithi082 ceph-mon[98694]: pgmap v18603: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 464 KiB/s wr, 13 op/s 2023-09-10T08:55:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:53 smithi098 ceph-mon[103124]: pgmap v18603: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 464 KiB/s wr, 13 op/s 2023-09-10T08:55:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:55:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34585.841233 secs 2023-09-10T08:55:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34586.195485 secs 2023-09-10T08:55:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:55:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:55:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34585.841233 secs 2023-09-10T08:55:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34586.195485 secs 2023-09-10T08:55:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:55:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:55:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34585.841233 secs 2023-09-10T08:55:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34586.195485 secs 2023-09-10T08:55:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:55:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:55 smithi170 ceph-mon[102832]: pgmap v18604: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 464 KiB/s wr, 13 op/s 2023-09-10T08:55:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:55:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:55:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:55:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:55 smithi082 ceph-mon[98694]: pgmap v18604: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 464 KiB/s wr, 13 op/s 2023-09-10T08:55:55.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:55:55.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:55:55.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:55:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:55 smithi098 ceph-mon[103124]: pgmap v18604: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 464 KiB/s wr, 13 op/s 2023-09-10T08:55:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:55:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:55:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:55:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:55:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:55:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:55:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:55:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:57 smithi170 ceph-mon[102832]: pgmap v18605: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 901 KiB/s wr, 20 op/s 2023-09-10T08:55:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:57 smithi082 ceph-mon[98694]: pgmap v18605: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 901 KiB/s wr, 20 op/s 2023-09-10T08:55:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:57 smithi098 ceph-mon[103124]: pgmap v18605: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 901 KiB/s wr, 20 op/s 2023-09-10T08:55:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34589.591293 secs 2023-09-10T08:55:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34589.591293 secs 2023-09-10T08:55:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34589.591293 secs 2023-09-10T08:55:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34590.841293 secs 2023-09-10T08:55:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:59 smithi170 ceph-mon[102832]: pgmap v18606: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 877 KiB/s wr, 14 op/s 2023-09-10T08:55:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:55:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34591.195545 secs 2023-09-10T08:55:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34590.841293 secs 2023-09-10T08:55:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:59 smithi082 ceph-mon[98694]: pgmap v18606: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 877 KiB/s wr, 14 op/s 2023-09-10T08:55:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:55:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34591.195545 secs 2023-09-10T08:56:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34590.841293 secs 2023-09-10T08:56:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:59 smithi098 ceph-mon[103124]: pgmap v18606: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 877 KiB/s wr, 14 op/s 2023-09-10T08:56:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:55:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34591.195545 secs 2023-09-10T08:56:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:56:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:56:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:56:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:56:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:56:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:56:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:01 smithi170 ceph-mon[102832]: pgmap v18607: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 987 KiB/s wr, 22 op/s 2023-09-10T08:56:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:01 smithi082 ceph-mon[98694]: pgmap v18607: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 987 KiB/s wr, 22 op/s 2023-09-10T08:56:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:01 smithi098 ceph-mon[103124]: pgmap v18607: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 987 KiB/s wr, 22 op/s 2023-09-10T08:56:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:03 smithi170 ceph-mon[102832]: pgmap v18608: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 547 KiB/s wr, 14 op/s 2023-09-10T08:56:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34594.591361 secs 2023-09-10T08:56:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:03 smithi082 ceph-mon[98694]: pgmap v18608: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 547 KiB/s wr, 14 op/s 2023-09-10T08:56:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34594.591361 secs 2023-09-10T08:56:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:03 smithi098 ceph-mon[103124]: pgmap v18608: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 547 KiB/s wr, 14 op/s 2023-09-10T08:56:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34594.591361 secs 2023-09-10T08:56:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34595.841349 secs 2023-09-10T08:56:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:56:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:04 smithi170 ceph-mon[102832]: osdmap e1231: 12 total, 12 up, 12 in 2023-09-10T08:56:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34596.195610 secs 2023-09-10T08:56:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34595.841349 secs 2023-09-10T08:56:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:56:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:04 smithi082 ceph-mon[98694]: osdmap e1231: 12 total, 12 up, 12 in 2023-09-10T08:56:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34596.195610 secs 2023-09-10T08:56:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34595.841349 secs 2023-09-10T08:56:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:56:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:04 smithi098 ceph-mon[103124]: osdmap e1231: 12 total, 12 up, 12 in 2023-09-10T08:56:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34596.195610 secs 2023-09-10T08:56:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:05 smithi170 ceph-mon[102832]: pgmap v18610: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 657 KiB/s wr, 17 op/s 2023-09-10T08:56:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:05 smithi082 ceph-mon[98694]: pgmap v18610: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 657 KiB/s wr, 17 op/s 2023-09-10T08:56:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:05 smithi098 ceph-mon[103124]: pgmap v18610: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 657 KiB/s wr, 17 op/s 2023-09-10T08:56:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:56:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:56:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:56:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:07 smithi082 ceph-mon[98694]: pgmap v18611: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 521 KiB/s wr, 32 op/s 2023-09-10T08:56:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:07 smithi098 ceph-mon[103124]: pgmap v18611: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 521 KiB/s wr, 32 op/s 2023-09-10T08:56:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:07 smithi170 ceph-mon[102832]: pgmap v18611: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 521 KiB/s wr, 32 op/s 2023-09-10T08:56:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34599.591422 secs 2023-09-10T08:56:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:08 smithi082 ceph-mon[98694]: osdmap e1232: 12 total, 12 up, 12 in 2023-09-10T08:56:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34599.591422 secs 2023-09-10T08:56:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:08 smithi098 ceph-mon[103124]: osdmap e1232: 12 total, 12 up, 12 in 2023-09-10T08:56:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34599.591422 secs 2023-09-10T08:56:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:08 smithi170 ceph-mon[102832]: osdmap e1232: 12 total, 12 up, 12 in 2023-09-10T08:56:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34600.841408 secs 2023-09-10T08:56:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:09 smithi082 ceph-mon[98694]: pgmap v18613: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 486 KiB/s wr, 28 op/s 2023-09-10T08:56:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34601.195673 secs 2023-09-10T08:56:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34600.841408 secs 2023-09-10T08:56:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:09 smithi098 ceph-mon[103124]: pgmap v18613: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 486 KiB/s wr, 28 op/s 2023-09-10T08:56:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34601.195673 secs 2023-09-10T08:56:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34600.841408 secs 2023-09-10T08:56:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:09 smithi170 ceph-mon[102832]: pgmap v18613: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 486 KiB/s wr, 28 op/s 2023-09-10T08:56:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34601.195673 secs 2023-09-10T08:56:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:11 smithi082 ceph-mon[98694]: pgmap v18614: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 673 KiB/s wr, 37 op/s 2023-09-10T08:56:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:11 smithi098 ceph-mon[103124]: pgmap v18614: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 673 KiB/s wr, 37 op/s 2023-09-10T08:56:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:11 smithi170 ceph-mon[102832]: pgmap v18614: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 673 KiB/s wr, 37 op/s 2023-09-10T08:56:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:56:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:56:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:56:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:13 smithi082 ceph-mon[98694]: pgmap v18615: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 602 KiB/s wr, 33 op/s 2023-09-10T08:56:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34604.591523 secs 2023-09-10T08:56:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:13 smithi098 ceph-mon[103124]: pgmap v18615: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 602 KiB/s wr, 33 op/s 2023-09-10T08:56:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34604.591523 secs 2023-09-10T08:56:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:13 smithi170 ceph-mon[102832]: pgmap v18615: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 602 KiB/s wr, 33 op/s 2023-09-10T08:56:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34604.591523 secs 2023-09-10T08:56:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34605.841467 secs 2023-09-10T08:56:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:14 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:56:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:14 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34606.195735 secs 2023-09-10T08:56:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34605.841467 secs 2023-09-10T08:56:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:14 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:56:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:14 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34606.195735 secs 2023-09-10T08:56:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34605.841467 secs 2023-09-10T08:56:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:14 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:56:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:14 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34606.195735 secs 2023-09-10T08:56:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:15 smithi082 ceph-mon[98694]: pgmap v18616: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 539 KiB/s wr, 29 op/s 2023-09-10T08:56:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:15 smithi098 ceph-mon[103124]: pgmap v18616: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 539 KiB/s wr, 29 op/s 2023-09-10T08:56:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:15 smithi170 ceph-mon[102832]: pgmap v18616: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 539 KiB/s wr, 29 op/s 2023-09-10T08:56:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:17 smithi098 ceph-mon[103124]: pgmap v18617: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 181 KiB/s wr, 14 op/s 2023-09-10T08:56:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:17 smithi170 ceph-mon[102832]: pgmap v18617: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 181 KiB/s wr, 14 op/s 2023-09-10T08:56:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:17 smithi082 ceph-mon[98694]: pgmap v18617: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 181 KiB/s wr, 14 op/s 2023-09-10T08:56:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34609.591600 secs 2023-09-10T08:56:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34609.591600 secs 2023-09-10T08:56:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34609.591600 secs 2023-09-10T08:56:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34610.842085 secs 2023-09-10T08:56:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:19 smithi098 ceph-mon[103124]: pgmap v18618: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 14 op/s 2023-09-10T08:56:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34611.195801 secs 2023-09-10T08:56:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34610.842085 secs 2023-09-10T08:56:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:19 smithi170 ceph-mon[102832]: pgmap v18618: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 14 op/s 2023-09-10T08:56:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34611.195801 secs 2023-09-10T08:56:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34610.842085 secs 2023-09-10T08:56:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:19 smithi082 ceph-mon[98694]: pgmap v18618: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 14 op/s 2023-09-10T08:56:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34611.195801 secs 2023-09-10T08:56:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:21 smithi098 ceph-mon[103124]: pgmap v18619: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 17 op/s 2023-09-10T08:56:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:21 smithi170 ceph-mon[102832]: pgmap v18619: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 17 op/s 2023-09-10T08:56:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:21 smithi082 ceph-mon[98694]: pgmap v18619: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 17 op/s 2023-09-10T08:56:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:23 smithi098 ceph-mon[103124]: pgmap v18620: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T08:56:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34614.591660 secs 2023-09-10T08:56:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:23 smithi170 ceph-mon[102832]: pgmap v18620: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T08:56:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34614.591660 secs 2023-09-10T08:56:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:23 smithi082 ceph-mon[98694]: pgmap v18620: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T08:56:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34614.591660 secs 2023-09-10T08:56:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34615.842138 secs 2023-09-10T08:56:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34616.195861 secs 2023-09-10T08:56:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34615.842138 secs 2023-09-10T08:56:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34616.195861 secs 2023-09-10T08:56:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34615.842138 secs 2023-09-10T08:56:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34616.195861 secs 2023-09-10T08:56:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:25 smithi098 ceph-mon[103124]: pgmap v18621: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T08:56:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:25 smithi170 ceph-mon[102832]: pgmap v18621: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T08:56:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:25 smithi082 ceph-mon[98694]: pgmap v18621: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T08:56:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:27 smithi098 ceph-mon[103124]: pgmap v18622: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 152 KiB/s wr, 16 op/s 2023-09-10T08:56:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:27 smithi170 ceph-mon[102832]: pgmap v18622: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 152 KiB/s wr, 16 op/s 2023-09-10T08:56:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:27 smithi082 ceph-mon[98694]: pgmap v18622: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 152 KiB/s wr, 16 op/s 2023-09-10T08:56:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34619.591718 secs 2023-09-10T08:56:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34619.591718 secs 2023-09-10T08:56:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34619.591718 secs 2023-09-10T08:56:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34620.842510 secs 2023-09-10T08:56:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:29 smithi098 ceph-mon[103124]: pgmap v18623: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T08:56:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34621.195924 secs 2023-09-10T08:56:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34620.842510 secs 2023-09-10T08:56:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:29 smithi170 ceph-mon[102832]: pgmap v18623: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T08:56:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34621.195924 secs 2023-09-10T08:56:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34620.842510 secs 2023-09-10T08:56:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:29 smithi082 ceph-mon[98694]: pgmap v18623: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T08:56:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34621.195924 secs 2023-09-10T08:56:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:31 smithi098 ceph-mon[103124]: pgmap v18624: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 126 KiB/s wr, 15 op/s 2023-09-10T08:56:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:31 smithi170 ceph-mon[102832]: pgmap v18624: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 126 KiB/s wr, 15 op/s 2023-09-10T08:56:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:31 smithi082 ceph-mon[98694]: pgmap v18624: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 126 KiB/s wr, 15 op/s 2023-09-10T08:56:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:33 smithi098 ceph-mon[103124]: pgmap v18625: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 126 KiB/s wr, 10 op/s 2023-09-10T08:56:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:33 smithi170 ceph-mon[102832]: pgmap v18625: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 126 KiB/s wr, 10 op/s 2023-09-10T08:56:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:33 smithi082 ceph-mon[98694]: pgmap v18625: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 126 KiB/s wr, 10 op/s 2023-09-10T08:56:35.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34625.842572 secs 2023-09-10T08:56:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:56:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34626.195985 secs 2023-09-10T08:56:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34625.842572 secs 2023-09-10T08:56:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:56:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34626.195985 secs 2023-09-10T08:56:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34625.842572 secs 2023-09-10T08:56:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:56:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34626.195985 secs 2023-09-10T08:56:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:35 smithi098 ceph-mon[103124]: pgmap v18626: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 126 KiB/s wr, 10 op/s 2023-09-10T08:56:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:56:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:35 smithi170 ceph-mon[102832]: pgmap v18626: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 126 KiB/s wr, 10 op/s 2023-09-10T08:56:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:56:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:35 smithi082 ceph-mon[98694]: pgmap v18626: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 126 KiB/s wr, 10 op/s 2023-09-10T08:56:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:56:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:56:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:56:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:56:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:56:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:56:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:56:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:37 smithi170 ceph-mon[102832]: pgmap v18627: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 16 op/s 2023-09-10T08:56:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:37 smithi082 ceph-mon[98694]: pgmap v18627: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 16 op/s 2023-09-10T08:56:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:37 smithi098 ceph-mon[103124]: pgmap v18627: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 16 op/s 2023-09-10T08:56:39.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34630.842632 secs 2023-09-10T08:56:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:39 smithi170 ceph-mon[102832]: pgmap v18628: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T08:56:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34631.196045 secs 2023-09-10T08:56:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34630.842632 secs 2023-09-10T08:56:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:39 smithi082 ceph-mon[98694]: pgmap v18628: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T08:56:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34631.196045 secs 2023-09-10T08:56:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34630.842632 secs 2023-09-10T08:56:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:39 smithi098 ceph-mon[103124]: pgmap v18628: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T08:56:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34631.196045 secs 2023-09-10T08:56:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:41 smithi170 ceph-mon[102832]: pgmap v18629: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 419 KiB/s wr, 16 op/s 2023-09-10T08:56:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:41 smithi082 ceph-mon[98694]: pgmap v18629: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 419 KiB/s wr, 16 op/s 2023-09-10T08:56:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:41 smithi098 ceph-mon[103124]: pgmap v18629: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 419 KiB/s wr, 16 op/s 2023-09-10T08:56:43.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:43 smithi170 ceph-mon[102832]: pgmap v18630: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 419 KiB/s wr, 12 op/s 2023-09-10T08:56:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:43 smithi082 ceph-mon[98694]: pgmap v18630: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 419 KiB/s wr, 12 op/s 2023-09-10T08:56:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:43 smithi098 ceph-mon[103124]: pgmap v18630: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 419 KiB/s wr, 12 op/s 2023-09-10T08:56:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34635.842690 secs 2023-09-10T08:56:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34636.196105 secs 2023-09-10T08:56:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34635.842690 secs 2023-09-10T08:56:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34636.196105 secs 2023-09-10T08:56:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34635.842690 secs 2023-09-10T08:56:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34636.196105 secs 2023-09-10T08:56:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:45 smithi170 ceph-mon[102832]: pgmap v18631: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 419 KiB/s wr, 12 op/s 2023-09-10T08:56:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:45 smithi082 ceph-mon[98694]: pgmap v18631: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 419 KiB/s wr, 12 op/s 2023-09-10T08:56:46.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:45 smithi098 ceph-mon[103124]: pgmap v18631: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 419 KiB/s wr, 12 op/s 2023-09-10T08:56:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:47 smithi170 ceph-mon[102832]: pgmap v18632: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 609 KiB/s wr, 18 op/s 2023-09-10T08:56:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:47 smithi082 ceph-mon[98694]: pgmap v18632: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 609 KiB/s wr, 18 op/s 2023-09-10T08:56:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:47 smithi098 ceph-mon[103124]: pgmap v18632: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 609 KiB/s wr, 18 op/s 2023-09-10T08:56:49.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34640.842773 secs 2023-09-10T08:56:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:49 smithi170 ceph-mon[102832]: pgmap v18633: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 458 KiB/s wr, 12 op/s 2023-09-10T08:56:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34641.196168 secs 2023-09-10T08:56:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34640.842773 secs 2023-09-10T08:56:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:49 smithi082 ceph-mon[98694]: pgmap v18633: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 458 KiB/s wr, 12 op/s 2023-09-10T08:56:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34641.196168 secs 2023-09-10T08:56:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34640.842773 secs 2023-09-10T08:56:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:49 smithi098 ceph-mon[103124]: pgmap v18633: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 458 KiB/s wr, 12 op/s 2023-09-10T08:56:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34641.196168 secs 2023-09-10T08:56:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:56:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:56:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:56:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:51 smithi170 ceph-mon[102832]: pgmap v18634: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 805 KiB/s wr, 20 op/s 2023-09-10T08:56:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:51 smithi082 ceph-mon[98694]: pgmap v18634: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 805 KiB/s wr, 20 op/s 2023-09-10T08:56:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:51 smithi098 ceph-mon[103124]: pgmap v18634: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 805 KiB/s wr, 20 op/s 2023-09-10T08:56:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:56:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:56:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:56:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:53 smithi170 ceph-mon[102832]: pgmap v18635: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 536 KiB/s wr, 14 op/s 2023-09-10T08:56:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:53 smithi082 ceph-mon[98694]: pgmap v18635: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 536 KiB/s wr, 14 op/s 2023-09-10T08:56:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:53 smithi098 ceph-mon[103124]: pgmap v18635: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 536 KiB/s wr, 14 op/s 2023-09-10T08:56:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:56:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34645.842864 secs 2023-09-10T08:56:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34646.196234 secs 2023-09-10T08:56:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:56:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34645.842864 secs 2023-09-10T08:56:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34646.196234 secs 2023-09-10T08:56:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:56:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34645.842864 secs 2023-09-10T08:56:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34646.196234 secs 2023-09-10T08:56:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:55 smithi170 ceph-mon[102832]: pgmap v18636: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 536 KiB/s wr, 14 op/s 2023-09-10T08:56:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:56:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:55 smithi082 ceph-mon[98694]: pgmap v18636: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 536 KiB/s wr, 14 op/s 2023-09-10T08:56:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:56:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:55 smithi098 ceph-mon[103124]: pgmap v18636: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 536 KiB/s wr, 14 op/s 2023-09-10T08:56:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:56:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:56:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:56:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:56:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:57.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:57 smithi170 ceph-mon[102832]: pgmap v18637: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 598 KiB/s wr, 21 op/s 2023-09-10T08:56:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:57 smithi082 ceph-mon[98694]: pgmap v18637: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 598 KiB/s wr, 21 op/s 2023-09-10T08:56:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:57 smithi098 ceph-mon[103124]: pgmap v18637: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 598 KiB/s wr, 21 op/s 2023-09-10T08:56:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:56:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:56:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:58.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:56:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:56:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:56:59.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:59.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:56:59.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:56:59.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:56:59.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:56:59.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:59 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T08:56:59.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:59 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T08:56:59.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34650.842923 secs 2023-09-10T08:56:59.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:59 smithi170 ceph-mon[102832]: pgmap v18638: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 408 KiB/s wr, 14 op/s 2023-09-10T08:56:59.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:59.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:59.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:56:59.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:56:59.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:56:59.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:56:59.682 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:59 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T08:56:59.682 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:59 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T08:56:59.682 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:56:59.682 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:56:59.682 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:59.683 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:56:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34651.196297 secs 2023-09-10T08:56:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:56:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:56:59.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:56:59.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:56:59.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:56:59.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:59 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T08:56:59.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:59 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T08:56:59.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34650.842923 secs 2023-09-10T08:56:59.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:59 smithi082 ceph-mon[98694]: pgmap v18638: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 408 KiB/s wr, 14 op/s 2023-09-10T08:56:59.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:59.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:59.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:56:59.711 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:56:59.711 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:56:59.711 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:56:59.711 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:59 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T08:56:59.711 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:59 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T08:56:59.712 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:56:59.712 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:56:59.712 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:59.712 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:56:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34651.196297 secs 2023-09-10T08:56:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:56:59.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:59.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:56:59.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:56:59.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:56:59.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:56:59.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:59 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T08:56:59.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:59 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T08:56:59.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34650.842923 secs 2023-09-10T08:56:59.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:59 smithi098 ceph-mon[103124]: pgmap v18638: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 408 KiB/s wr, 14 op/s 2023-09-10T08:56:59.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:59.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:59.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:56:59.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:56:59.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:56:59.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T08:56:59.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:59 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T08:56:59.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:59 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T08:56:59.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:56:59.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:56:59.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:56:59.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:56:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34651.196297 secs 2023-09-10T08:57:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:57:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:57:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:57:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:01 smithi170 ceph-mon[102832]: pgmap v18639: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T08:57:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:01 smithi082 ceph-mon[98694]: pgmap v18639: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T08:57:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:01 smithi098 ceph-mon[103124]: pgmap v18639: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T08:57:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:03 smithi170 ceph-mon[102832]: pgmap v18640: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 769 KiB/s wr, 13 op/s 2023-09-10T08:57:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34654.592229 secs 2023-09-10T08:57:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:03 smithi082 ceph-mon[98694]: pgmap v18640: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 769 KiB/s wr, 13 op/s 2023-09-10T08:57:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34654.592229 secs 2023-09-10T08:57:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:03 smithi098 ceph-mon[103124]: pgmap v18640: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 769 KiB/s wr, 13 op/s 2023-09-10T08:57:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34654.592229 secs 2023-09-10T08:57:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34655.842985 secs 2023-09-10T08:57:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:57:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:57:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34656.196363 secs 2023-09-10T08:57:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34655.842985 secs 2023-09-10T08:57:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:57:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:57:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34656.196363 secs 2023-09-10T08:57:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34655.842985 secs 2023-09-10T08:57:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:57:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:57:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34656.196363 secs 2023-09-10T08:57:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:05 smithi170 ceph-mon[102832]: pgmap v18641: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 769 KiB/s wr, 13 op/s 2023-09-10T08:57:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:05 smithi082 ceph-mon[98694]: pgmap v18641: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 769 KiB/s wr, 13 op/s 2023-09-10T08:57:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:05 smithi098 ceph-mon[103124]: pgmap v18641: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 769 KiB/s wr, 13 op/s 2023-09-10T08:57:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:57:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:57:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:57:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:07 smithi082 ceph-mon[98694]: pgmap v18642: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 815 KiB/s wr, 34 op/s 2023-09-10T08:57:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:07 smithi098 ceph-mon[103124]: pgmap v18642: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 815 KiB/s wr, 34 op/s 2023-09-10T08:57:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:07 smithi170 ceph-mon[102832]: pgmap v18642: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 815 KiB/s wr, 34 op/s 2023-09-10T08:57:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34659.592290 secs 2023-09-10T08:57:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34659.592290 secs 2023-09-10T08:57:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34659.592290 secs 2023-09-10T08:57:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34660.843052 secs 2023-09-10T08:57:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:09 smithi082 ceph-mon[98694]: pgmap v18643: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 753 KiB/s wr, 28 op/s 2023-09-10T08:57:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:09 smithi082 ceph-mon[98694]: osdmap e1233: 12 total, 12 up, 12 in 2023-09-10T08:57:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34661.196429 secs 2023-09-10T08:57:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34660.843052 secs 2023-09-10T08:57:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:09 smithi098 ceph-mon[103124]: pgmap v18643: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 753 KiB/s wr, 28 op/s 2023-09-10T08:57:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:09 smithi098 ceph-mon[103124]: osdmap e1233: 12 total, 12 up, 12 in 2023-09-10T08:57:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34661.196429 secs 2023-09-10T08:57:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34660.843052 secs 2023-09-10T08:57:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:09 smithi170 ceph-mon[102832]: pgmap v18643: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 753 KiB/s wr, 28 op/s 2023-09-10T08:57:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:09 smithi170 ceph-mon[102832]: osdmap e1233: 12 total, 12 up, 12 in 2023-09-10T08:57:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34661.196429 secs 2023-09-10T08:57:11.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:11 smithi082 ceph-mon[98694]: pgmap v18645: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 385 KiB/s wr, 33 op/s 2023-09-10T08:57:12.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:11 smithi098 ceph-mon[103124]: pgmap v18645: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 385 KiB/s wr, 33 op/s 2023-09-10T08:57:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:11 smithi170 ceph-mon[102832]: pgmap v18645: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 385 KiB/s wr, 33 op/s 2023-09-10T08:57:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:13 smithi082 ceph-mon[98694]: pgmap v18646: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 385 KiB/s wr, 33 op/s 2023-09-10T08:57:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34664.592367 secs 2023-09-10T08:57:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:13 smithi082 ceph-mon[98694]: osdmap e1234: 12 total, 12 up, 12 in 2023-09-10T08:57:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:13 smithi098 ceph-mon[103124]: pgmap v18646: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 385 KiB/s wr, 33 op/s 2023-09-10T08:57:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34664.592367 secs 2023-09-10T08:57:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:13 smithi098 ceph-mon[103124]: osdmap e1234: 12 total, 12 up, 12 in 2023-09-10T08:57:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:13 smithi170 ceph-mon[102832]: pgmap v18646: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 385 KiB/s wr, 33 op/s 2023-09-10T08:57:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34664.592367 secs 2023-09-10T08:57:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:13 smithi170 ceph-mon[102832]: osdmap e1234: 12 total, 12 up, 12 in 2023-09-10T08:57:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34665.843133 secs 2023-09-10T08:57:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34666.196486 secs 2023-09-10T08:57:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34665.843133 secs 2023-09-10T08:57:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34666.196486 secs 2023-09-10T08:57:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34665.843133 secs 2023-09-10T08:57:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34666.196486 secs 2023-09-10T08:57:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:15 smithi098 ceph-mon[103124]: pgmap v18648: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 413 KiB/s wr, 10 op/s 2023-09-10T08:57:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:15 smithi170 ceph-mon[102832]: pgmap v18648: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 413 KiB/s wr, 10 op/s 2023-09-10T08:57:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:15 smithi082 ceph-mon[98694]: pgmap v18648: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 413 KiB/s wr, 10 op/s 2023-09-10T08:57:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:57:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:57:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:57:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:17 smithi098 ceph-mon[103124]: pgmap v18649: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 414 KiB/s wr, 18 op/s 2023-09-10T08:57:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:17 smithi170 ceph-mon[102832]: pgmap v18649: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 414 KiB/s wr, 18 op/s 2023-09-10T08:57:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:17 smithi082 ceph-mon[98694]: pgmap v18649: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 414 KiB/s wr, 18 op/s 2023-09-10T08:57:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:57:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34669.592464 secs 2023-09-10T08:57:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:57:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34669.592464 secs 2023-09-10T08:57:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:57:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34669.592464 secs 2023-09-10T08:57:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34670.843207 secs 2023-09-10T08:57:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:19 smithi098 ceph-mon[103124]: pgmap v18650: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 333 KiB/s wr, 14 op/s 2023-09-10T08:57:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34671.196552 secs 2023-09-10T08:57:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34670.843207 secs 2023-09-10T08:57:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:19 smithi170 ceph-mon[102832]: pgmap v18650: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 333 KiB/s wr, 14 op/s 2023-09-10T08:57:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34671.196552 secs 2023-09-10T08:57:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34670.843207 secs 2023-09-10T08:57:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:19 smithi082 ceph-mon[98694]: pgmap v18650: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 333 KiB/s wr, 14 op/s 2023-09-10T08:57:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34671.196552 secs 2023-09-10T08:57:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:21 smithi098 ceph-mon[103124]: pgmap v18651: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 13 op/s 2023-09-10T08:57:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:21 smithi170 ceph-mon[102832]: pgmap v18651: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 13 op/s 2023-09-10T08:57:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:21 smithi082 ceph-mon[98694]: pgmap v18651: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 13 op/s 2023-09-10T08:57:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:23 smithi098 ceph-mon[103124]: pgmap v18652: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 13 op/s 2023-09-10T08:57:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34674.592564 secs 2023-09-10T08:57:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:23 smithi170 ceph-mon[102832]: pgmap v18652: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 13 op/s 2023-09-10T08:57:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34674.592564 secs 2023-09-10T08:57:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:23 smithi082 ceph-mon[98694]: pgmap v18652: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 13 op/s 2023-09-10T08:57:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34674.592564 secs 2023-09-10T08:57:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34675.843295 secs 2023-09-10T08:57:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34676.196622 secs 2023-09-10T08:57:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34675.843295 secs 2023-09-10T08:57:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34676.196622 secs 2023-09-10T08:57:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34675.843295 secs 2023-09-10T08:57:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34676.196622 secs 2023-09-10T08:57:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:25 smithi098 ceph-mon[103124]: pgmap v18653: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 12 op/s 2023-09-10T08:57:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:25 smithi170 ceph-mon[102832]: pgmap v18653: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 12 op/s 2023-09-10T08:57:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:25 smithi082 ceph-mon[98694]: pgmap v18653: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 12 op/s 2023-09-10T08:57:28.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:27 smithi098 ceph-mon[103124]: pgmap v18654: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 16 op/s 2023-09-10T08:57:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:27 smithi170 ceph-mon[102832]: pgmap v18654: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 16 op/s 2023-09-10T08:57:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:27 smithi082 ceph-mon[98694]: pgmap v18654: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 16 op/s 2023-09-10T08:57:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34679.592624 secs 2023-09-10T08:57:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34679.592624 secs 2023-09-10T08:57:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34679.592624 secs 2023-09-10T08:57:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34680.843352 secs 2023-09-10T08:57:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:29 smithi098 ceph-mon[103124]: pgmap v18655: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T08:57:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34681.196688 secs 2023-09-10T08:57:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34680.843352 secs 2023-09-10T08:57:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:29 smithi170 ceph-mon[102832]: pgmap v18655: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T08:57:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34681.196688 secs 2023-09-10T08:57:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34680.843352 secs 2023-09-10T08:57:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:29 smithi082 ceph-mon[98694]: pgmap v18655: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 11 op/s 2023-09-10T08:57:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34681.196688 secs 2023-09-10T08:57:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:31 smithi098 ceph-mon[103124]: pgmap v18656: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 16 op/s 2023-09-10T08:57:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:31 smithi170 ceph-mon[102832]: pgmap v18656: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 16 op/s 2023-09-10T08:57:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:31 smithi082 ceph-mon[98694]: pgmap v18656: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 16 op/s 2023-09-10T08:57:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:33 smithi098 ceph-mon[103124]: pgmap v18657: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T08:57:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:33 smithi170 ceph-mon[102832]: pgmap v18657: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T08:57:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:33 smithi082 ceph-mon[98694]: pgmap v18657: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T08:57:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34685.843414 secs 2023-09-10T08:57:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34686.196777 secs 2023-09-10T08:57:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34685.843414 secs 2023-09-10T08:57:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34686.196777 secs 2023-09-10T08:57:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34685.843414 secs 2023-09-10T08:57:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34686.196777 secs 2023-09-10T08:57:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:35 smithi170 ceph-mon[102832]: pgmap v18658: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T08:57:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:57:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:35 smithi082 ceph-mon[98694]: pgmap v18658: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T08:57:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:57:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:35 smithi098 ceph-mon[103124]: pgmap v18658: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T08:57:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:57:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:57:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:57:37.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:57:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:57:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:57:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:57:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:37 smithi170 ceph-mon[102832]: pgmap v18659: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 324 KiB/s wr, 16 op/s 2023-09-10T08:57:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:37 smithi082 ceph-mon[98694]: pgmap v18659: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 324 KiB/s wr, 16 op/s 2023-09-10T08:57:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:37 smithi098 ceph-mon[103124]: pgmap v18659: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 324 KiB/s wr, 16 op/s 2023-09-10T08:57:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:57:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:57:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:57:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34690.843502 secs 2023-09-10T08:57:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:39 smithi170 ceph-mon[102832]: pgmap v18660: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 324 KiB/s wr, 10 op/s 2023-09-10T08:57:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34691.196839 secs 2023-09-10T08:57:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34690.843502 secs 2023-09-10T08:57:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:39 smithi082 ceph-mon[98694]: pgmap v18660: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 324 KiB/s wr, 10 op/s 2023-09-10T08:57:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34691.196839 secs 2023-09-10T08:57:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34690.843502 secs 2023-09-10T08:57:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:39 smithi098 ceph-mon[103124]: pgmap v18660: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 324 KiB/s wr, 10 op/s 2023-09-10T08:57:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34691.196839 secs 2023-09-10T08:57:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:57:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:57:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:57:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:41 smithi170 ceph-mon[102832]: pgmap v18661: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 911 KiB/s wr, 18 op/s 2023-09-10T08:57:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:41 smithi082 ceph-mon[98694]: pgmap v18661: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 911 KiB/s wr, 18 op/s 2023-09-10T08:57:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:41 smithi098 ceph-mon[103124]: pgmap v18661: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 911 KiB/s wr, 18 op/s 2023-09-10T08:57:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:43 smithi170 ceph-mon[102832]: pgmap v18662: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 786 KiB/s wr, 12 op/s 2023-09-10T08:57:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:43 smithi082 ceph-mon[98694]: pgmap v18662: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 786 KiB/s wr, 12 op/s 2023-09-10T08:57:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:43 smithi098 ceph-mon[103124]: pgmap v18662: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 786 KiB/s wr, 12 op/s 2023-09-10T08:57:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34695.843561 secs 2023-09-10T08:57:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34696.196908 secs 2023-09-10T08:57:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34695.843561 secs 2023-09-10T08:57:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34696.196908 secs 2023-09-10T08:57:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34695.843561 secs 2023-09-10T08:57:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34696.196908 secs 2023-09-10T08:57:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:45 smithi170 ceph-mon[102832]: pgmap v18663: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 786 KiB/s wr, 12 op/s 2023-09-10T08:57:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:45 smithi082 ceph-mon[98694]: pgmap v18663: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 786 KiB/s wr, 12 op/s 2023-09-10T08:57:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:45 smithi098 ceph-mon[103124]: pgmap v18663: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 786 KiB/s wr, 12 op/s 2023-09-10T08:57:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:57:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:57:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:57:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:47 smithi170 ceph-mon[102832]: pgmap v18664: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 811 KiB/s wr, 19 op/s 2023-09-10T08:57:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:47 smithi082 ceph-mon[98694]: pgmap v18664: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 811 KiB/s wr, 19 op/s 2023-09-10T08:57:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:47 smithi098 ceph-mon[103124]: pgmap v18664: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 811 KiB/s wr, 19 op/s 2023-09-10T08:57:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34700.843618 secs 2023-09-10T08:57:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:49 smithi170 ceph-mon[102832]: pgmap v18665: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 612 KiB/s wr, 14 op/s 2023-09-10T08:57:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34701.196975 secs 2023-09-10T08:57:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34700.843618 secs 2023-09-10T08:57:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:49 smithi082 ceph-mon[98694]: pgmap v18665: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 612 KiB/s wr, 14 op/s 2023-09-10T08:57:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34701.196975 secs 2023-09-10T08:57:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34700.843618 secs 2023-09-10T08:57:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:49 smithi098 ceph-mon[103124]: pgmap v18665: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 612 KiB/s wr, 14 op/s 2023-09-10T08:57:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34701.196975 secs 2023-09-10T08:57:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:57:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:57:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:57:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:51 smithi170 ceph-mon[102832]: pgmap v18666: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T08:57:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:51 smithi082 ceph-mon[98694]: pgmap v18666: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T08:57:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:51 smithi098 ceph-mon[103124]: pgmap v18666: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T08:57:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:57:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:57:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:57:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:53 smithi170 ceph-mon[102832]: pgmap v18667: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 467 KiB/s wr, 13 op/s 2023-09-10T08:57:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:53 smithi082 ceph-mon[98694]: pgmap v18667: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 467 KiB/s wr, 13 op/s 2023-09-10T08:57:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:53 smithi098 ceph-mon[103124]: pgmap v18667: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 467 KiB/s wr, 13 op/s 2023-09-10T08:57:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:57:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34705.843672 secs 2023-09-10T08:57:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34706.197043 secs 2023-09-10T08:57:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:57:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34705.843672 secs 2023-09-10T08:57:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34706.197043 secs 2023-09-10T08:57:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:57:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34705.843672 secs 2023-09-10T08:57:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34706.197043 secs 2023-09-10T08:57:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:55 smithi082 ceph-mon[98694]: pgmap v18668: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 467 KiB/s wr, 13 op/s 2023-09-10T08:57:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:55 smithi098 ceph-mon[103124]: pgmap v18668: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 467 KiB/s wr, 13 op/s 2023-09-10T08:57:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:55 smithi170 ceph-mon[102832]: pgmap v18668: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 467 KiB/s wr, 13 op/s 2023-09-10T08:57:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:57:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:57:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:57:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:57:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:57 smithi082 ceph-mon[98694]: pgmap v18669: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 906 KiB/s wr, 20 op/s 2023-09-10T08:57:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:57 smithi098 ceph-mon[103124]: pgmap v18669: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 906 KiB/s wr, 20 op/s 2023-09-10T08:57:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:57 smithi170 ceph-mon[102832]: pgmap v18669: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 906 KiB/s wr, 20 op/s 2023-09-10T08:57:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34709.593078 secs 2023-09-10T08:57:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34709.593078 secs 2023-09-10T08:57:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34709.593078 secs 2023-09-10T08:57:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34710.843730 secs 2023-09-10T08:57:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:59 smithi082 ceph-mon[98694]: pgmap v18670: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 881 KiB/s wr, 14 op/s 2023-09-10T08:57:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:57:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34711.197109 secs 2023-09-10T08:57:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:57:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:57:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:57:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:57:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34710.843730 secs 2023-09-10T08:57:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:59 smithi098 ceph-mon[103124]: pgmap v18670: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 881 KiB/s wr, 14 op/s 2023-09-10T08:57:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:57:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34711.197109 secs 2023-09-10T08:57:59.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:57:59.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:57:59.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:57:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:57:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34710.843730 secs 2023-09-10T08:57:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:59 smithi170 ceph-mon[102832]: pgmap v18670: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 881 KiB/s wr, 14 op/s 2023-09-10T08:57:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:57:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34711.197109 secs 2023-09-10T08:57:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:57:59.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:57:59.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:57:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:58:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:58:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:58:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:58:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:58:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:58:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:58:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:01 smithi098 ceph-mon[103124]: pgmap v18671: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 980 KiB/s wr, 20 op/s 2023-09-10T08:58:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:01 smithi170 ceph-mon[102832]: pgmap v18671: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 980 KiB/s wr, 20 op/s 2023-09-10T08:58:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:01 smithi082 ceph-mon[98694]: pgmap v18671: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 980 KiB/s wr, 20 op/s 2023-09-10T08:58:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:03 smithi082 ceph-mon[98694]: pgmap v18672: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 538 KiB/s wr, 13 op/s 2023-09-10T08:58:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34714.593186 secs 2023-09-10T08:58:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:03 smithi098 ceph-mon[103124]: pgmap v18672: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 538 KiB/s wr, 13 op/s 2023-09-10T08:58:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34714.593186 secs 2023-09-10T08:58:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:03 smithi170 ceph-mon[102832]: pgmap v18672: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 538 KiB/s wr, 13 op/s 2023-09-10T08:58:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34714.593186 secs 2023-09-10T08:58:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34715.843789 secs 2023-09-10T08:58:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:58:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:04.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34716.197173 secs 2023-09-10T08:58:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34715.843789 secs 2023-09-10T08:58:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:58:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34716.197173 secs 2023-09-10T08:58:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34715.843789 secs 2023-09-10T08:58:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:58:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34716.197173 secs 2023-09-10T08:58:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:05 smithi082 ceph-mon[98694]: pgmap v18673: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 538 KiB/s wr, 13 op/s 2023-09-10T08:58:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:05 smithi098 ceph-mon[103124]: pgmap v18673: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 538 KiB/s wr, 13 op/s 2023-09-10T08:58:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:05 smithi170 ceph-mon[102832]: pgmap v18673: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 538 KiB/s wr, 13 op/s 2023-09-10T08:58:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:58:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:58:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:58:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:07 smithi098 ceph-mon[103124]: pgmap v18674: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 854 KiB/s wr, 34 op/s 2023-09-10T08:58:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:07 smithi170 ceph-mon[102832]: pgmap v18674: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 854 KiB/s wr, 34 op/s 2023-09-10T08:58:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:07 smithi082 ceph-mon[98694]: pgmap v18674: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 854 KiB/s wr, 34 op/s 2023-09-10T08:58:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34719.593282 secs 2023-09-10T08:58:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34719.593282 secs 2023-09-10T08:58:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34719.593282 secs 2023-09-10T08:58:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34720.843849 secs 2023-09-10T08:58:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:09 smithi098 ceph-mon[103124]: pgmap v18675: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 415 KiB/s wr, 26 op/s 2023-09-10T08:58:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:09 smithi098 ceph-mon[103124]: osdmap e1235: 12 total, 12 up, 12 in 2023-09-10T08:58:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34721.197239 secs 2023-09-10T08:58:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34720.843849 secs 2023-09-10T08:58:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:09 smithi170 ceph-mon[102832]: pgmap v18675: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 415 KiB/s wr, 26 op/s 2023-09-10T08:58:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:09 smithi170 ceph-mon[102832]: osdmap e1235: 12 total, 12 up, 12 in 2023-09-10T08:58:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34721.197239 secs 2023-09-10T08:58:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34720.843849 secs 2023-09-10T08:58:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:09 smithi082 ceph-mon[98694]: pgmap v18675: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 415 KiB/s wr, 26 op/s 2023-09-10T08:58:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:09 smithi082 ceph-mon[98694]: osdmap e1235: 12 total, 12 up, 12 in 2023-09-10T08:58:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34721.197239 secs 2023-09-10T08:58:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:11 smithi098 ceph-mon[103124]: pgmap v18677: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 528 KiB/s wr, 32 op/s 2023-09-10T08:58:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:11 smithi170 ceph-mon[102832]: pgmap v18677: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 528 KiB/s wr, 32 op/s 2023-09-10T08:58:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:11 smithi082 ceph-mon[98694]: pgmap v18677: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 528 KiB/s wr, 32 op/s 2023-09-10T08:58:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:13 smithi098 ceph-mon[103124]: pgmap v18678: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 528 KiB/s wr, 32 op/s 2023-09-10T08:58:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34724.593342 secs 2023-09-10T08:58:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:13 smithi098 ceph-mon[103124]: osdmap e1236: 12 total, 12 up, 12 in 2023-09-10T08:58:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:13 smithi170 ceph-mon[102832]: pgmap v18678: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 528 KiB/s wr, 32 op/s 2023-09-10T08:58:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34724.593342 secs 2023-09-10T08:58:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:13 smithi170 ceph-mon[102832]: osdmap e1236: 12 total, 12 up, 12 in 2023-09-10T08:58:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:13 smithi082 ceph-mon[98694]: pgmap v18678: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 528 KiB/s wr, 32 op/s 2023-09-10T08:58:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34724.593342 secs 2023-09-10T08:58:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:13 smithi082 ceph-mon[98694]: osdmap e1236: 12 total, 12 up, 12 in 2023-09-10T08:58:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34725.843907 secs 2023-09-10T08:58:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:14 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:58:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:14 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34726.197311 secs 2023-09-10T08:58:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34725.843907 secs 2023-09-10T08:58:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:14 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:58:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:14 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34726.197311 secs 2023-09-10T08:58:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34725.843907 secs 2023-09-10T08:58:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:14 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:58:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:14 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34726.197311 secs 2023-09-10T08:58:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:15 smithi098 ceph-mon[103124]: pgmap v18680: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 187 KiB/s wr, 9 op/s 2023-09-10T08:58:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:15 smithi170 ceph-mon[102832]: pgmap v18680: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 187 KiB/s wr, 9 op/s 2023-09-10T08:58:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:15 smithi082 ceph-mon[98694]: pgmap v18680: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 187 KiB/s wr, 9 op/s 2023-09-10T08:58:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:58:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:17.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:58:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:58:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:17 smithi098 ceph-mon[103124]: pgmap v18681: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 227 KiB/s wr, 19 op/s 2023-09-10T08:58:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:17 smithi170 ceph-mon[102832]: pgmap v18681: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 227 KiB/s wr, 19 op/s 2023-09-10T08:58:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:17 smithi082 ceph-mon[98694]: pgmap v18681: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 227 KiB/s wr, 19 op/s 2023-09-10T08:58:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34729.593450 secs 2023-09-10T08:58:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34729.593450 secs 2023-09-10T08:58:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34729.593450 secs 2023-09-10T08:58:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34730.843966 secs 2023-09-10T08:58:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:19 smithi098 ceph-mon[103124]: pgmap v18682: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 15 op/s 2023-09-10T08:58:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34731.197372 secs 2023-09-10T08:58:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34730.843966 secs 2023-09-10T08:58:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:19 smithi170 ceph-mon[102832]: pgmap v18682: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 15 op/s 2023-09-10T08:58:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34731.197372 secs 2023-09-10T08:58:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34730.843966 secs 2023-09-10T08:58:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:19 smithi082 ceph-mon[98694]: pgmap v18682: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 15 op/s 2023-09-10T08:58:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34731.197372 secs 2023-09-10T08:58:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:21 smithi098 ceph-mon[103124]: pgmap v18683: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-09-10T08:58:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:21 smithi170 ceph-mon[102832]: pgmap v18683: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-09-10T08:58:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:21 smithi082 ceph-mon[98694]: pgmap v18683: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-09-10T08:58:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:23 smithi170 ceph-mon[102832]: pgmap v18684: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-09-10T08:58:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34734.593510 secs 2023-09-10T08:58:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:23 smithi082 ceph-mon[98694]: pgmap v18684: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-09-10T08:58:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34734.593510 secs 2023-09-10T08:58:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:23 smithi098 ceph-mon[103124]: pgmap v18684: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-09-10T08:58:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34734.593510 secs 2023-09-10T08:58:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34735.844024 secs 2023-09-10T08:58:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34736.197436 secs 2023-09-10T08:58:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34735.844024 secs 2023-09-10T08:58:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34736.197436 secs 2023-09-10T08:58:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34735.844024 secs 2023-09-10T08:58:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34736.197436 secs 2023-09-10T08:58:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:25 smithi170 ceph-mon[102832]: pgmap v18685: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 13 op/s 2023-09-10T08:58:26.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:25 smithi082 ceph-mon[98694]: pgmap v18685: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 13 op/s 2023-09-10T08:58:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:25 smithi098 ceph-mon[103124]: pgmap v18685: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 13 op/s 2023-09-10T08:58:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:27 smithi170 ceph-mon[102832]: pgmap v18686: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 17 op/s 2023-09-10T08:58:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:27 smithi082 ceph-mon[98694]: pgmap v18686: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 17 op/s 2023-09-10T08:58:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:27 smithi098 ceph-mon[103124]: pgmap v18686: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 17 op/s 2023-09-10T08:58:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34739.593601 secs 2023-09-10T08:58:29.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34739.593601 secs 2023-09-10T08:58:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34739.593601 secs 2023-09-10T08:58:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34740.844084 secs 2023-09-10T08:58:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:29 smithi170 ceph-mon[102832]: pgmap v18687: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T08:58:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34741.197489 secs 2023-09-10T08:58:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34740.844084 secs 2023-09-10T08:58:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:29 smithi082 ceph-mon[98694]: pgmap v18687: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T08:58:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34741.197489 secs 2023-09-10T08:58:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34740.844084 secs 2023-09-10T08:58:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:29 smithi098 ceph-mon[103124]: pgmap v18687: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 11 op/s 2023-09-10T08:58:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34741.197489 secs 2023-09-10T08:58:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:31 smithi170 ceph-mon[102832]: pgmap v18688: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 16 op/s 2023-09-10T08:58:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:31 smithi082 ceph-mon[98694]: pgmap v18688: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 16 op/s 2023-09-10T08:58:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:31 smithi098 ceph-mon[103124]: pgmap v18688: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 16 op/s 2023-09-10T08:58:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:33 smithi170 ceph-mon[102832]: pgmap v18689: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 10 op/s 2023-09-10T08:58:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:33 smithi082 ceph-mon[98694]: pgmap v18689: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 10 op/s 2023-09-10T08:58:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:33 smithi098 ceph-mon[103124]: pgmap v18689: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 10 op/s 2023-09-10T08:58:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34745.844144 secs 2023-09-10T08:58:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:58:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34746.197556 secs 2023-09-10T08:58:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34745.844144 secs 2023-09-10T08:58:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:58:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34746.197556 secs 2023-09-10T08:58:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34745.844144 secs 2023-09-10T08:58:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:58:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34746.197556 secs 2023-09-10T08:58:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:35 smithi170 ceph-mon[102832]: pgmap v18690: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 10 op/s 2023-09-10T08:58:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:58:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:35 smithi082 ceph-mon[98694]: pgmap v18690: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 10 op/s 2023-09-10T08:58:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:58:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:35 smithi098 ceph-mon[103124]: pgmap v18690: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 10 op/s 2023-09-10T08:58:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:58:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:58:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:58:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:58:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:58:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:58:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:58:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:37 smithi170 ceph-mon[102832]: pgmap v18691: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 278 KiB/s wr, 16 op/s 2023-09-10T08:58:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:37 smithi082 ceph-mon[98694]: pgmap v18691: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 278 KiB/s wr, 16 op/s 2023-09-10T08:58:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:37 smithi098 ceph-mon[103124]: pgmap v18691: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 278 KiB/s wr, 16 op/s 2023-09-10T08:58:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34750.844205 secs 2023-09-10T08:58:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:39 smithi170 ceph-mon[102832]: pgmap v18692: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-09-10T08:58:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34751.197623 secs 2023-09-10T08:58:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34750.844205 secs 2023-09-10T08:58:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:39 smithi082 ceph-mon[98694]: pgmap v18692: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-09-10T08:58:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34751.197623 secs 2023-09-10T08:58:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34750.844205 secs 2023-09-10T08:58:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:39 smithi098 ceph-mon[103124]: pgmap v18692: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 10 op/s 2023-09-10T08:58:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34751.197623 secs 2023-09-10T08:58:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:41 smithi082 ceph-mon[98694]: pgmap v18693: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 460 KiB/s wr, 17 op/s 2023-09-10T08:58:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:41 smithi098 ceph-mon[103124]: pgmap v18693: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 460 KiB/s wr, 17 op/s 2023-09-10T08:58:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:41 smithi170 ceph-mon[102832]: pgmap v18693: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 460 KiB/s wr, 17 op/s 2023-09-10T08:58:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:43 smithi082 ceph-mon[98694]: pgmap v18694: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 460 KiB/s wr, 12 op/s 2023-09-10T08:58:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:43 smithi098 ceph-mon[103124]: pgmap v18694: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 460 KiB/s wr, 12 op/s 2023-09-10T08:58:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:43 smithi170 ceph-mon[102832]: pgmap v18694: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 460 KiB/s wr, 12 op/s 2023-09-10T08:58:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34755.844278 secs 2023-09-10T08:58:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34756.197689 secs 2023-09-10T08:58:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34755.844278 secs 2023-09-10T08:58:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34756.197689 secs 2023-09-10T08:58:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34755.844278 secs 2023-09-10T08:58:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34756.197689 secs 2023-09-10T08:58:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:45 smithi098 ceph-mon[103124]: pgmap v18695: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 460 KiB/s wr, 12 op/s 2023-09-10T08:58:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:45 smithi170 ceph-mon[102832]: pgmap v18695: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 460 KiB/s wr, 12 op/s 2023-09-10T08:58:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:45 smithi082 ceph-mon[98694]: pgmap v18695: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 460 KiB/s wr, 12 op/s 2023-09-10T08:58:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:47 smithi098 ceph-mon[103124]: pgmap v18696: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 492 KiB/s wr, 19 op/s 2023-09-10T08:58:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:47 smithi170 ceph-mon[102832]: pgmap v18696: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 492 KiB/s wr, 19 op/s 2023-09-10T08:58:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:47 smithi082 ceph-mon[98694]: pgmap v18696: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 492 KiB/s wr, 19 op/s 2023-09-10T08:58:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34760.844339 secs 2023-09-10T08:58:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:49 smithi082 ceph-mon[98694]: pgmap v18697: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 340 KiB/s wr, 13 op/s 2023-09-10T08:58:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34761.197755 secs 2023-09-10T08:58:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34760.844339 secs 2023-09-10T08:58:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:49 smithi098 ceph-mon[103124]: pgmap v18697: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 340 KiB/s wr, 13 op/s 2023-09-10T08:58:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34761.197755 secs 2023-09-10T08:58:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34760.844339 secs 2023-09-10T08:58:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:49 smithi170 ceph-mon[102832]: pgmap v18697: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 340 KiB/s wr, 13 op/s 2023-09-10T08:58:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34761.197755 secs 2023-09-10T08:58:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:58:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:58:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:58:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:51 smithi082 ceph-mon[98694]: pgmap v18698: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 808 KiB/s wr, 21 op/s 2023-09-10T08:58:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:51 smithi098 ceph-mon[103124]: pgmap v18698: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 808 KiB/s wr, 21 op/s 2023-09-10T08:58:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:51 smithi170 ceph-mon[102832]: pgmap v18698: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 808 KiB/s wr, 21 op/s 2023-09-10T08:58:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:58:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:58:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:58:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:53 smithi082 ceph-mon[98694]: pgmap v18699: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 500 KiB/s wr, 15 op/s 2023-09-10T08:58:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:53 smithi098 ceph-mon[103124]: pgmap v18699: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 500 KiB/s wr, 15 op/s 2023-09-10T08:58:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:53 smithi170 ceph-mon[102832]: pgmap v18699: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 500 KiB/s wr, 15 op/s 2023-09-10T08:58:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:58:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34765.844394 secs 2023-09-10T08:58:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34766.197820 secs 2023-09-10T08:58:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:58:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34765.844394 secs 2023-09-10T08:58:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34766.197820 secs 2023-09-10T08:58:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:58:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34765.844394 secs 2023-09-10T08:58:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34766.197820 secs 2023-09-10T08:58:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:55 smithi082 ceph-mon[98694]: pgmap v18700: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 500 KiB/s wr, 15 op/s 2023-09-10T08:58:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:55 smithi098 ceph-mon[103124]: pgmap v18700: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 500 KiB/s wr, 15 op/s 2023-09-10T08:58:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:55 smithi170 ceph-mon[102832]: pgmap v18700: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 500 KiB/s wr, 15 op/s 2023-09-10T08:58:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:58:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:58:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:58:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:57 smithi098 ceph-mon[103124]: pgmap v18701: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 681 KiB/s wr, 21 op/s 2023-09-10T08:58:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:57 smithi170 ceph-mon[102832]: pgmap v18701: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 681 KiB/s wr, 21 op/s 2023-09-10T08:58:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:57 smithi082 ceph-mon[98694]: pgmap v18701: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 681 KiB/s wr, 21 op/s 2023-09-10T08:58:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:58:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:58:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:58:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:58:59.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34770.844452 secs 2023-09-10T08:58:59.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:59 smithi098 ceph-mon[103124]: pgmap v18702: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 649 KiB/s wr, 14 op/s 2023-09-10T08:58:59.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:58:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34771.197885 secs 2023-09-10T08:58:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34770.844452 secs 2023-09-10T08:58:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:59 smithi170 ceph-mon[102832]: pgmap v18702: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 649 KiB/s wr, 14 op/s 2023-09-10T08:58:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:58:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34771.197885 secs 2023-09-10T08:58:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34770.844452 secs 2023-09-10T08:58:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:59 smithi082 ceph-mon[98694]: pgmap v18702: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 649 KiB/s wr, 14 op/s 2023-09-10T08:58:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:58:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34771.197885 secs 2023-09-10T08:59:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:59:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:59:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:59:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:59:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:59:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:59:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:59:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:59:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:59:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:59:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:59:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:59:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:59:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:59:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:59:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T08:59:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:59:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:59:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T08:59:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T08:59:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T08:59:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:01 smithi098 ceph-mon[103124]: pgmap v18703: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1018 KiB/s wr, 21 op/s 2023-09-10T08:59:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:01 smithi170 ceph-mon[102832]: pgmap v18703: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1018 KiB/s wr, 21 op/s 2023-09-10T08:59:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:01 smithi082 ceph-mon[98694]: pgmap v18703: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1018 KiB/s wr, 21 op/s 2023-09-10T08:59:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:03 smithi098 ceph-mon[103124]: pgmap v18704: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 551 KiB/s wr, 13 op/s 2023-09-10T08:59:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34774.594145 secs 2023-09-10T08:59:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:03 smithi170 ceph-mon[102832]: pgmap v18704: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 551 KiB/s wr, 13 op/s 2023-09-10T08:59:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34774.594145 secs 2023-09-10T08:59:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:03 smithi082 ceph-mon[98694]: pgmap v18704: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 551 KiB/s wr, 13 op/s 2023-09-10T08:59:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34774.594145 secs 2023-09-10T08:59:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34775.844533 secs 2023-09-10T08:59:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:59:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:59:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34776.197951 secs 2023-09-10T08:59:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34775.844533 secs 2023-09-10T08:59:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:59:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:59:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34776.197951 secs 2023-09-10T08:59:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34775.844533 secs 2023-09-10T08:59:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:59:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:59:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34776.197951 secs 2023-09-10T08:59:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:05 smithi098 ceph-mon[103124]: pgmap v18705: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 551 KiB/s wr, 13 op/s 2023-09-10T08:59:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:05 smithi170 ceph-mon[102832]: pgmap v18705: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 551 KiB/s wr, 13 op/s 2023-09-10T08:59:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:05 smithi082 ceph-mon[98694]: pgmap v18705: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 551 KiB/s wr, 13 op/s 2023-09-10T08:59:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:59:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:59:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:59:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:07 smithi098 ceph-mon[103124]: pgmap v18706: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 718 KiB/s wr, 34 op/s 2023-09-10T08:59:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:07 smithi170 ceph-mon[102832]: pgmap v18706: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 718 KiB/s wr, 34 op/s 2023-09-10T08:59:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:07 smithi082 ceph-mon[98694]: pgmap v18706: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 718 KiB/s wr, 34 op/s 2023-09-10T08:59:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34779.594203 secs 2023-09-10T08:59:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34779.594203 secs 2023-09-10T08:59:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34779.594203 secs 2023-09-10T08:59:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34780.844590 secs 2023-09-10T08:59:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:09 smithi098 ceph-mon[103124]: pgmap v18707: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 536 KiB/s wr, 28 op/s 2023-09-10T08:59:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:09 smithi098 ceph-mon[103124]: osdmap e1237: 12 total, 12 up, 12 in 2023-09-10T08:59:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34781.198017 secs 2023-09-10T08:59:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34780.844590 secs 2023-09-10T08:59:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:09 smithi170 ceph-mon[102832]: pgmap v18707: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 536 KiB/s wr, 28 op/s 2023-09-10T08:59:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:09 smithi170 ceph-mon[102832]: osdmap e1237: 12 total, 12 up, 12 in 2023-09-10T08:59:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34781.198017 secs 2023-09-10T08:59:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34780.844590 secs 2023-09-10T08:59:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:09 smithi082 ceph-mon[98694]: pgmap v18707: 97 pgs: 97 active+clean; 12 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 536 KiB/s wr, 28 op/s 2023-09-10T08:59:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:09 smithi082 ceph-mon[98694]: osdmap e1237: 12 total, 12 up, 12 in 2023-09-10T08:59:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34781.198017 secs 2023-09-10T08:59:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:11 smithi098 ceph-mon[103124]: pgmap v18709: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 558 KiB/s wr, 32 op/s 2023-09-10T08:59:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:11 smithi170 ceph-mon[102832]: pgmap v18709: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 558 KiB/s wr, 32 op/s 2023-09-10T08:59:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:11 smithi082 ceph-mon[98694]: pgmap v18709: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 558 KiB/s wr, 32 op/s 2023-09-10T08:59:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:13 smithi098 ceph-mon[103124]: pgmap v18710: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 558 KiB/s wr, 32 op/s 2023-09-10T08:59:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34784.594264 secs 2023-09-10T08:59:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:13 smithi098 ceph-mon[103124]: osdmap e1238: 12 total, 12 up, 12 in 2023-09-10T08:59:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:13 smithi170 ceph-mon[102832]: pgmap v18710: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 558 KiB/s wr, 32 op/s 2023-09-10T08:59:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34784.594264 secs 2023-09-10T08:59:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:13 smithi170 ceph-mon[102832]: osdmap e1238: 12 total, 12 up, 12 in 2023-09-10T08:59:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:13 smithi082 ceph-mon[98694]: pgmap v18710: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 558 KiB/s wr, 32 op/s 2023-09-10T08:59:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34784.594264 secs 2023-09-10T08:59:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:13 smithi082 ceph-mon[98694]: osdmap e1238: 12 total, 12 up, 12 in 2023-09-10T08:59:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34785.844650 secs 2023-09-10T08:59:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34786.198080 secs 2023-09-10T08:59:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34785.844650 secs 2023-09-10T08:59:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34786.198080 secs 2023-09-10T08:59:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34785.844650 secs 2023-09-10T08:59:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34786.198080 secs 2023-09-10T08:59:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:15 smithi098 ceph-mon[103124]: pgmap v18712: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 446 KiB/s wr, 9 op/s 2023-09-10T08:59:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:15 smithi170 ceph-mon[102832]: pgmap v18712: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 446 KiB/s wr, 9 op/s 2023-09-10T08:59:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:15 smithi082 ceph-mon[98694]: pgmap v18712: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 446 KiB/s wr, 9 op/s 2023-09-10T08:59:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:17 smithi098 ceph-mon[103124]: pgmap v18713: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 447 KiB/s wr, 16 op/s 2023-09-10T08:59:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:17 smithi170 ceph-mon[102832]: pgmap v18713: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 447 KiB/s wr, 16 op/s 2023-09-10T08:59:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:17 smithi082 ceph-mon[98694]: pgmap v18713: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 447 KiB/s wr, 16 op/s 2023-09-10T08:59:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34789.594325 secs 2023-09-10T08:59:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34789.594325 secs 2023-09-10T08:59:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34789.594325 secs 2023-09-10T08:59:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34790.844711 secs 2023-09-10T08:59:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:19 smithi170 ceph-mon[102832]: pgmap v18714: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 360 KiB/s wr, 13 op/s 2023-09-10T08:59:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34791.198144 secs 2023-09-10T08:59:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34790.844711 secs 2023-09-10T08:59:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:19 smithi082 ceph-mon[98694]: pgmap v18714: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 360 KiB/s wr, 13 op/s 2023-09-10T08:59:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34791.198144 secs 2023-09-10T08:59:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34790.844711 secs 2023-09-10T08:59:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:19 smithi098 ceph-mon[103124]: pgmap v18714: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 360 KiB/s wr, 13 op/s 2023-09-10T08:59:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34791.198144 secs 2023-09-10T08:59:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:21 smithi170 ceph-mon[102832]: pgmap v18715: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 13 op/s 2023-09-10T08:59:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:21 smithi082 ceph-mon[98694]: pgmap v18715: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 13 op/s 2023-09-10T08:59:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:21 smithi098 ceph-mon[103124]: pgmap v18715: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 13 op/s 2023-09-10T08:59:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:23 smithi170 ceph-mon[102832]: pgmap v18716: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 13 op/s 2023-09-10T08:59:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34794.594404 secs 2023-09-10T08:59:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:23 smithi082 ceph-mon[98694]: pgmap v18716: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 13 op/s 2023-09-10T08:59:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34794.594404 secs 2023-09-10T08:59:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:23 smithi098 ceph-mon[103124]: pgmap v18716: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 13 op/s 2023-09-10T08:59:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34794.594404 secs 2023-09-10T08:59:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34795.844769 secs 2023-09-10T08:59:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34796.198209 secs 2023-09-10T08:59:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34795.844769 secs 2023-09-10T08:59:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34796.198209 secs 2023-09-10T08:59:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34795.844769 secs 2023-09-10T08:59:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34796.198209 secs 2023-09-10T08:59:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:25 smithi170 ceph-mon[102832]: pgmap v18717: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 157 KiB/s wr, 11 op/s 2023-09-10T08:59:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:25 smithi082 ceph-mon[98694]: pgmap v18717: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 157 KiB/s wr, 11 op/s 2023-09-10T08:59:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:25 smithi098 ceph-mon[103124]: pgmap v18717: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 157 KiB/s wr, 11 op/s 2023-09-10T08:59:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:27 smithi170 ceph-mon[102832]: pgmap v18718: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T08:59:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:27 smithi082 ceph-mon[98694]: pgmap v18718: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T08:59:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:27 smithi098 ceph-mon[103124]: pgmap v18718: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T08:59:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34799.594454 secs 2023-09-10T08:59:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34799.594454 secs 2023-09-10T08:59:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34799.594454 secs 2023-09-10T08:59:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34800.844858 secs 2023-09-10T08:59:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:29 smithi170 ceph-mon[102832]: pgmap v18719: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T08:59:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34801.198277 secs 2023-09-10T08:59:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34800.844858 secs 2023-09-10T08:59:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:29 smithi082 ceph-mon[98694]: pgmap v18719: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T08:59:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34801.198277 secs 2023-09-10T08:59:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34800.844858 secs 2023-09-10T08:59:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:29 smithi098 ceph-mon[103124]: pgmap v18719: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T08:59:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34801.198277 secs 2023-09-10T08:59:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:59:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:59:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:59:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:31 smithi170 ceph-mon[102832]: pgmap v18720: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 273 KiB/s wr, 17 op/s 2023-09-10T08:59:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:31 smithi082 ceph-mon[98694]: pgmap v18720: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 273 KiB/s wr, 17 op/s 2023-09-10T08:59:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:31 smithi098 ceph-mon[103124]: pgmap v18720: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 273 KiB/s wr, 17 op/s 2023-09-10T08:59:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:59:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:59:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:59:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:33 smithi170 ceph-mon[102832]: pgmap v18721: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T08:59:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:33 smithi082 ceph-mon[98694]: pgmap v18721: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T08:59:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:33 smithi098 ceph-mon[103124]: pgmap v18721: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T08:59:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34805.844915 secs 2023-09-10T08:59:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34806.198344 secs 2023-09-10T08:59:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34805.844915 secs 2023-09-10T08:59:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34806.198344 secs 2023-09-10T08:59:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34805.844915 secs 2023-09-10T08:59:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34806.198344 secs 2023-09-10T08:59:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:35 smithi170 ceph-mon[102832]: pgmap v18722: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T08:59:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:59:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:35 smithi082 ceph-mon[98694]: pgmap v18722: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T08:59:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:59:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:35 smithi098 ceph-mon[103124]: pgmap v18722: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T08:59:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T08:59:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:59:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:59:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:59:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:59:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T08:59:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:59:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:37 smithi170 ceph-mon[102832]: pgmap v18723: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 250 KiB/s wr, 16 op/s 2023-09-10T08:59:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:37 smithi082 ceph-mon[98694]: pgmap v18723: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 250 KiB/s wr, 16 op/s 2023-09-10T08:59:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:37 smithi098 ceph-mon[103124]: pgmap v18723: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 250 KiB/s wr, 16 op/s 2023-09-10T08:59:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:59:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:59:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:59:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34810.844973 secs 2023-09-10T08:59:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:39 smithi170 ceph-mon[102832]: pgmap v18724: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 250 KiB/s wr, 10 op/s 2023-09-10T08:59:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34811.198408 secs 2023-09-10T08:59:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34810.844973 secs 2023-09-10T08:59:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:39 smithi082 ceph-mon[98694]: pgmap v18724: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 250 KiB/s wr, 10 op/s 2023-09-10T08:59:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34811.198408 secs 2023-09-10T08:59:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34810.844973 secs 2023-09-10T08:59:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:39 smithi098 ceph-mon[103124]: pgmap v18724: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 250 KiB/s wr, 10 op/s 2023-09-10T08:59:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34811.198408 secs 2023-09-10T08:59:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:59:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:59:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:59:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:41 smithi170 ceph-mon[102832]: pgmap v18725: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 916 KiB/s wr, 18 op/s 2023-09-10T08:59:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:41 smithi082 ceph-mon[98694]: pgmap v18725: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 916 KiB/s wr, 18 op/s 2023-09-10T08:59:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:41 smithi098 ceph-mon[103124]: pgmap v18725: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 916 KiB/s wr, 18 op/s 2023-09-10T08:59:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:43 smithi170 ceph-mon[102832]: pgmap v18726: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 790 KiB/s wr, 13 op/s 2023-09-10T08:59:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:43 smithi082 ceph-mon[98694]: pgmap v18726: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 790 KiB/s wr, 13 op/s 2023-09-10T08:59:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:43 smithi098 ceph-mon[103124]: pgmap v18726: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 790 KiB/s wr, 13 op/s 2023-09-10T08:59:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34815.845031 secs 2023-09-10T08:59:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34816.198482 secs 2023-09-10T08:59:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34815.845031 secs 2023-09-10T08:59:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34816.198482 secs 2023-09-10T08:59:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34815.845031 secs 2023-09-10T08:59:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34816.198482 secs 2023-09-10T08:59:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:45 smithi170 ceph-mon[102832]: pgmap v18727: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 790 KiB/s wr, 13 op/s 2023-09-10T08:59:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:45 smithi082 ceph-mon[98694]: pgmap v18727: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 790 KiB/s wr, 13 op/s 2023-09-10T08:59:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:45 smithi098 ceph-mon[103124]: pgmap v18727: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 790 KiB/s wr, 13 op/s 2023-09-10T08:59:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:47 smithi170 ceph-mon[102832]: pgmap v18728: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 859 KiB/s wr, 19 op/s 2023-09-10T08:59:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:47 smithi082 ceph-mon[98694]: pgmap v18728: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 859 KiB/s wr, 19 op/s 2023-09-10T08:59:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:47 smithi098 ceph-mon[103124]: pgmap v18728: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 859 KiB/s wr, 19 op/s 2023-09-10T08:59:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34820.845089 secs 2023-09-10T08:59:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:49 smithi082 ceph-mon[98694]: pgmap v18729: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 736 KiB/s wr, 14 op/s 2023-09-10T08:59:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34821.198548 secs 2023-09-10T08:59:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34820.845089 secs 2023-09-10T08:59:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:49 smithi098 ceph-mon[103124]: pgmap v18729: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 736 KiB/s wr, 14 op/s 2023-09-10T08:59:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34821.198548 secs 2023-09-10T08:59:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34820.845089 secs 2023-09-10T08:59:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:49 smithi170 ceph-mon[102832]: pgmap v18729: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 736 KiB/s wr, 14 op/s 2023-09-10T08:59:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34821.198548 secs 2023-09-10T08:59:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:59:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:59:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:50.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:59:50.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:51 smithi082 ceph-mon[98694]: pgmap v18730: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T08:59:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:51 smithi098 ceph-mon[103124]: pgmap v18730: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T08:59:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:51 smithi170 ceph-mon[102832]: pgmap v18730: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T08:59:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:59:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:59:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:59:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:53 smithi098 ceph-mon[103124]: pgmap v18731: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 469 KiB/s wr, 13 op/s 2023-09-10T08:59:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:53 smithi170 ceph-mon[102832]: pgmap v18731: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 469 KiB/s wr, 13 op/s 2023-09-10T08:59:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:53 smithi082 ceph-mon[98694]: pgmap v18731: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 469 KiB/s wr, 13 op/s 2023-09-10T08:59:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:59:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34825.845146 secs 2023-09-10T08:59:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34826.198614 secs 2023-09-10T08:59:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:59:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34825.845146 secs 2023-09-10T08:59:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34826.198614 secs 2023-09-10T08:59:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:59:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34825.845146 secs 2023-09-10T08:59:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34826.198614 secs 2023-09-10T08:59:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:55 smithi098 ceph-mon[103124]: pgmap v18732: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 469 KiB/s wr, 13 op/s 2023-09-10T08:59:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:55 smithi170 ceph-mon[102832]: pgmap v18732: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 469 KiB/s wr, 13 op/s 2023-09-10T08:59:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:55 smithi082 ceph-mon[98694]: pgmap v18732: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 469 KiB/s wr, 13 op/s 2023-09-10T08:59:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:59:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:59:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T08:59:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:57 smithi098 ceph-mon[103124]: pgmap v18733: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 910 KiB/s wr, 20 op/s 2023-09-10T08:59:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:57 smithi170 ceph-mon[102832]: pgmap v18733: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 910 KiB/s wr, 20 op/s 2023-09-10T08:59:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:57 smithi082 ceph-mon[98694]: pgmap v18733: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 910 KiB/s wr, 20 op/s 2023-09-10T08:59:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:59:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34829.594903 secs 2023-09-10T08:59:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:59:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34829.594903 secs 2023-09-10T08:59:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T08:59:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T08:59:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34829.594903 secs 2023-09-10T08:59:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34830.845230 secs 2023-09-10T08:59:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:59 smithi098 ceph-mon[103124]: pgmap v18734: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 841 KiB/s wr, 14 op/s 2023-09-10T08:59:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 08:59:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34831.198677 secs 2023-09-10T08:59:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34830.845230 secs 2023-09-10T08:59:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:59 smithi170 ceph-mon[102832]: pgmap v18734: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 841 KiB/s wr, 14 op/s 2023-09-10T08:59:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 08:59:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34831.198677 secs 2023-09-10T08:59:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34830.845230 secs 2023-09-10T08:59:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:59 smithi082 ceph-mon[98694]: pgmap v18734: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 841 KiB/s wr, 14 op/s 2023-09-10T08:59:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 08:59:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34831.198677 secs 2023-09-10T09:00:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:00:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:00:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T09:00:00.797 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:00:00.797 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:00:00.797 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:00.797 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T09:00:00.810 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:00:00.810 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:00:00.810 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:00.810 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T09:00:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:00:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:01 smithi098 ceph-mon[103124]: pgmap v18735: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 940 KiB/s wr, 20 op/s 2023-09-10T09:00:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:00:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:00:01.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:00:01.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:01 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:00:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:00:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:01 smithi170 ceph-mon[102832]: pgmap v18735: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 940 KiB/s wr, 20 op/s 2023-09-10T09:00:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:00:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:00:01.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:00:01.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:01 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:00:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:00:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:01 smithi082 ceph-mon[98694]: pgmap v18735: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 940 KiB/s wr, 20 op/s 2023-09-10T09:00:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:00:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:00:01.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:00:01.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:01 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:00:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:00:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:00:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:00:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:00:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:00:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:00:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:00:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:00:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:00:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:03 smithi098 ceph-mon[103124]: pgmap v18736: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 541 KiB/s wr, 13 op/s 2023-09-10T09:00:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34834.594973 secs 2023-09-10T09:00:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:03 smithi170 ceph-mon[102832]: pgmap v18736: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 541 KiB/s wr, 13 op/s 2023-09-10T09:00:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34834.594973 secs 2023-09-10T09:00:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:03 smithi082 ceph-mon[98694]: pgmap v18736: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 541 KiB/s wr, 13 op/s 2023-09-10T09:00:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34834.594973 secs 2023-09-10T09:00:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:00:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34835.845315 secs 2023-09-10T09:00:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34836.198743 secs 2023-09-10T09:00:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:00:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34835.845315 secs 2023-09-10T09:00:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34836.198743 secs 2023-09-10T09:00:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:00:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34835.845315 secs 2023-09-10T09:00:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34836.198743 secs 2023-09-10T09:00:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:05 smithi170 ceph-mon[102832]: pgmap v18737: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 541 KiB/s wr, 13 op/s 2023-09-10T09:00:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:05 smithi082 ceph-mon[98694]: pgmap v18737: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 541 KiB/s wr, 13 op/s 2023-09-10T09:00:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:05 smithi098 ceph-mon[103124]: pgmap v18737: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 541 KiB/s wr, 13 op/s 2023-09-10T09:00:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:00:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:00:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:00:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:07.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:07 smithi170 ceph-mon[102832]: pgmap v18738: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 899 KiB/s wr, 34 op/s 2023-09-10T09:00:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:07 smithi082 ceph-mon[98694]: pgmap v18738: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 899 KiB/s wr, 34 op/s 2023-09-10T09:00:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:07 smithi098 ceph-mon[103124]: pgmap v18738: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 899 KiB/s wr, 34 op/s 2023-09-10T09:00:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34839.595031 secs 2023-09-10T09:00:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34839.595031 secs 2023-09-10T09:00:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34839.595031 secs 2023-09-10T09:00:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:09 smithi170 ceph-mon[102832]: osdmap e1239: 12 total, 12 up, 12 in 2023-09-10T09:00:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34840.845374 secs 2023-09-10T09:00:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:09 smithi170 ceph-mon[102832]: pgmap v18740: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 549 KiB/s wr, 31 op/s 2023-09-10T09:00:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34841.198807 secs 2023-09-10T09:00:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:09 smithi082 ceph-mon[98694]: osdmap e1239: 12 total, 12 up, 12 in 2023-09-10T09:00:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34840.845374 secs 2023-09-10T09:00:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:09 smithi082 ceph-mon[98694]: pgmap v18740: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 549 KiB/s wr, 31 op/s 2023-09-10T09:00:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34841.198807 secs 2023-09-10T09:00:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:09 smithi098 ceph-mon[103124]: osdmap e1239: 12 total, 12 up, 12 in 2023-09-10T09:00:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34840.845374 secs 2023-09-10T09:00:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:09 smithi098 ceph-mon[103124]: pgmap v18740: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 549 KiB/s wr, 31 op/s 2023-09-10T09:00:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34841.198807 secs 2023-09-10T09:00:12.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:11 smithi098 ceph-mon[103124]: pgmap v18741: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 580 KiB/s wr, 31 op/s 2023-09-10T09:00:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:11 smithi170 ceph-mon[102832]: pgmap v18741: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 580 KiB/s wr, 31 op/s 2023-09-10T09:00:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:11 smithi082 ceph-mon[98694]: pgmap v18741: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 580 KiB/s wr, 31 op/s 2023-09-10T09:00:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:13 smithi098 ceph-mon[103124]: pgmap v18742: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 580 KiB/s wr, 31 op/s 2023-09-10T09:00:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34844.595090 secs 2023-09-10T09:00:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:13 smithi098 ceph-mon[103124]: osdmap e1240: 12 total, 12 up, 12 in 2023-09-10T09:00:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:13 smithi170 ceph-mon[102832]: pgmap v18742: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 580 KiB/s wr, 31 op/s 2023-09-10T09:00:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34844.595090 secs 2023-09-10T09:00:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:13 smithi170 ceph-mon[102832]: osdmap e1240: 12 total, 12 up, 12 in 2023-09-10T09:00:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:13 smithi082 ceph-mon[98694]: pgmap v18742: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 580 KiB/s wr, 31 op/s 2023-09-10T09:00:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34844.595090 secs 2023-09-10T09:00:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:13 smithi082 ceph-mon[98694]: osdmap e1240: 12 total, 12 up, 12 in 2023-09-10T09:00:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34845.845433 secs 2023-09-10T09:00:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34846.198874 secs 2023-09-10T09:00:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34845.845433 secs 2023-09-10T09:00:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34846.198874 secs 2023-09-10T09:00:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34845.845433 secs 2023-09-10T09:00:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34846.198874 secs 2023-09-10T09:00:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:15 smithi098 ceph-mon[103124]: pgmap v18744: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 188 KiB/s wr, 9 op/s 2023-09-10T09:00:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:15 smithi170 ceph-mon[102832]: pgmap v18744: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 188 KiB/s wr, 9 op/s 2023-09-10T09:00:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:15 smithi082 ceph-mon[98694]: pgmap v18744: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 188 KiB/s wr, 9 op/s 2023-09-10T09:00:18.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:17 smithi098 ceph-mon[103124]: pgmap v18745: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 224 KiB/s wr, 17 op/s 2023-09-10T09:00:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:17 smithi170 ceph-mon[102832]: pgmap v18745: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 224 KiB/s wr, 17 op/s 2023-09-10T09:00:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:17 smithi082 ceph-mon[98694]: pgmap v18745: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 224 KiB/s wr, 17 op/s 2023-09-10T09:00:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34849.595220 secs 2023-09-10T09:00:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34849.595220 secs 2023-09-10T09:00:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34849.595220 secs 2023-09-10T09:00:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34850.845496 secs 2023-09-10T09:00:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:19 smithi098 ceph-mon[103124]: pgmap v18746: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T09:00:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34851.198939 secs 2023-09-10T09:00:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34850.845496 secs 2023-09-10T09:00:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:19 smithi170 ceph-mon[102832]: pgmap v18746: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T09:00:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34851.198939 secs 2023-09-10T09:00:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34850.845496 secs 2023-09-10T09:00:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:19 smithi082 ceph-mon[98694]: pgmap v18746: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T09:00:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34851.198939 secs 2023-09-10T09:00:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:21 smithi098 ceph-mon[103124]: pgmap v18747: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 14 op/s 2023-09-10T09:00:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:21 smithi170 ceph-mon[102832]: pgmap v18747: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 14 op/s 2023-09-10T09:00:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:21 smithi082 ceph-mon[98694]: pgmap v18747: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 14 op/s 2023-09-10T09:00:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:23 smithi098 ceph-mon[103124]: pgmap v18748: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 14 op/s 2023-09-10T09:00:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34854.595293 secs 2023-09-10T09:00:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:23 smithi170 ceph-mon[102832]: pgmap v18748: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 14 op/s 2023-09-10T09:00:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34854.595293 secs 2023-09-10T09:00:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:23 smithi082 ceph-mon[98694]: pgmap v18748: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 14 op/s 2023-09-10T09:00:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34854.595293 secs 2023-09-10T09:00:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34855.845556 secs 2023-09-10T09:00:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:00:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34856.199006 secs 2023-09-10T09:00:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34855.845556 secs 2023-09-10T09:00:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:00:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34856.199006 secs 2023-09-10T09:00:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34855.845556 secs 2023-09-10T09:00:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:00:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34856.199006 secs 2023-09-10T09:00:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:25 smithi170 ceph-mon[102832]: pgmap v18749: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 135 KiB/s wr, 12 op/s 2023-09-10T09:00:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:25 smithi082 ceph-mon[98694]: pgmap v18749: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 135 KiB/s wr, 12 op/s 2023-09-10T09:00:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:25 smithi098 ceph-mon[103124]: pgmap v18749: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 135 KiB/s wr, 12 op/s 2023-09-10T09:00:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:26 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:00:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:26 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:00:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:26 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:00:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:27 smithi170 ceph-mon[102832]: pgmap v18750: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 253 KiB/s wr, 17 op/s 2023-09-10T09:00:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:27 smithi082 ceph-mon[98694]: pgmap v18750: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 253 KiB/s wr, 17 op/s 2023-09-10T09:00:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:27 smithi098 ceph-mon[103124]: pgmap v18750: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 253 KiB/s wr, 17 op/s 2023-09-10T09:00:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34859.595379 secs 2023-09-10T09:00:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34859.595379 secs 2023-09-10T09:00:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34859.595379 secs 2023-09-10T09:00:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34860.845615 secs 2023-09-10T09:00:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:29 smithi170 ceph-mon[102832]: pgmap v18751: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T09:00:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34861.199071 secs 2023-09-10T09:00:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34860.845615 secs 2023-09-10T09:00:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:29 smithi082 ceph-mon[98694]: pgmap v18751: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T09:00:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34861.199071 secs 2023-09-10T09:00:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34860.845615 secs 2023-09-10T09:00:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:29 smithi098 ceph-mon[103124]: pgmap v18751: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 227 KiB/s wr, 11 op/s 2023-09-10T09:00:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34861.199071 secs 2023-09-10T09:00:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:31 smithi170 ceph-mon[102832]: pgmap v18752: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 227 KiB/s wr, 17 op/s 2023-09-10T09:00:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:31 smithi082 ceph-mon[98694]: pgmap v18752: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 227 KiB/s wr, 17 op/s 2023-09-10T09:00:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:31 smithi098 ceph-mon[103124]: pgmap v18752: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 227 KiB/s wr, 17 op/s 2023-09-10T09:00:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:33 smithi170 ceph-mon[102832]: pgmap v18753: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T09:00:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34864.595440 secs 2023-09-10T09:00:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:33 smithi082 ceph-mon[98694]: pgmap v18753: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T09:00:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34864.595440 secs 2023-09-10T09:00:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:33 smithi098 ceph-mon[103124]: pgmap v18753: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T09:00:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34864.595440 secs 2023-09-10T09:00:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34865.845703 secs 2023-09-10T09:00:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34866.199138 secs 2023-09-10T09:00:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34865.845703 secs 2023-09-10T09:00:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34866.199138 secs 2023-09-10T09:00:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34865.845703 secs 2023-09-10T09:00:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34866.199138 secs 2023-09-10T09:00:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:35 smithi170 ceph-mon[102832]: pgmap v18754: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T09:00:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:35 smithi082 ceph-mon[98694]: pgmap v18754: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T09:00:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:35 smithi098 ceph-mon[103124]: pgmap v18754: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T09:00:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:37 smithi170 ceph-mon[102832]: pgmap v18755: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 334 KiB/s wr, 17 op/s 2023-09-10T09:00:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:37 smithi082 ceph-mon[98694]: pgmap v18755: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 334 KiB/s wr, 17 op/s 2023-09-10T09:00:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:37 smithi098 ceph-mon[103124]: pgmap v18755: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 334 KiB/s wr, 17 op/s 2023-09-10T09:00:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34870.846343 secs 2023-09-10T09:00:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:39 smithi098 ceph-mon[103124]: pgmap v18756: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 208 KiB/s wr, 11 op/s 2023-09-10T09:00:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34871.199203 secs 2023-09-10T09:00:39.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:00:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34870.846343 secs 2023-09-10T09:00:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:39 smithi170 ceph-mon[102832]: pgmap v18756: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 208 KiB/s wr, 11 op/s 2023-09-10T09:00:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34871.199203 secs 2023-09-10T09:00:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:00:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34870.846343 secs 2023-09-10T09:00:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:39 smithi082 ceph-mon[98694]: pgmap v18756: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 208 KiB/s wr, 11 op/s 2023-09-10T09:00:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34871.199203 secs 2023-09-10T09:00:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:00:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:00:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:00:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:40.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:00:40.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:00:40.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:00:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:00:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:41 smithi098 ceph-mon[103124]: pgmap v18757: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 423 KiB/s wr, 18 op/s 2023-09-10T09:00:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:41 smithi170 ceph-mon[102832]: pgmap v18757: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 423 KiB/s wr, 18 op/s 2023-09-10T09:00:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:41 smithi082 ceph-mon[98694]: pgmap v18757: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 423 KiB/s wr, 18 op/s 2023-09-10T09:00:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:43 smithi098 ceph-mon[103124]: pgmap v18758: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 423 KiB/s wr, 12 op/s 2023-09-10T09:00:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:43 smithi170 ceph-mon[102832]: pgmap v18758: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 423 KiB/s wr, 12 op/s 2023-09-10T09:00:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:43 smithi082 ceph-mon[98694]: pgmap v18758: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 423 KiB/s wr, 12 op/s 2023-09-10T09:00:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34875.846401 secs 2023-09-10T09:00:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:00:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34876.199270 secs 2023-09-10T09:00:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34875.846401 secs 2023-09-10T09:00:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:00:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34876.199270 secs 2023-09-10T09:00:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34875.846401 secs 2023-09-10T09:00:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:00:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34876.199270 secs 2023-09-10T09:00:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:45 smithi170 ceph-mon[102832]: pgmap v18759: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 423 KiB/s wr, 12 op/s 2023-09-10T09:00:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:45 smithi082 ceph-mon[98694]: pgmap v18759: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 423 KiB/s wr, 12 op/s 2023-09-10T09:00:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:45 smithi098 ceph-mon[103124]: pgmap v18759: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 423 KiB/s wr, 12 op/s 2023-09-10T09:00:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:00:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:00:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:00:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:47 smithi170 ceph-mon[102832]: pgmap v18760: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 614 KiB/s wr, 20 op/s 2023-09-10T09:00:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:47 smithi082 ceph-mon[98694]: pgmap v18760: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 614 KiB/s wr, 20 op/s 2023-09-10T09:00:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:47 smithi098 ceph-mon[103124]: pgmap v18760: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 614 KiB/s wr, 20 op/s 2023-09-10T09:00:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34880.846461 secs 2023-09-10T09:00:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:49 smithi098 ceph-mon[103124]: pgmap v18761: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 407 KiB/s wr, 14 op/s 2023-09-10T09:00:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34881.199333 secs 2023-09-10T09:00:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34880.846461 secs 2023-09-10T09:00:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:49 smithi170 ceph-mon[102832]: pgmap v18761: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 407 KiB/s wr, 14 op/s 2023-09-10T09:00:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34881.199333 secs 2023-09-10T09:00:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34880.846461 secs 2023-09-10T09:00:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:49 smithi082 ceph-mon[98694]: pgmap v18761: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 407 KiB/s wr, 14 op/s 2023-09-10T09:00:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34881.199333 secs 2023-09-10T09:00:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:00:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:00:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:00:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:51 smithi098 ceph-mon[103124]: pgmap v18762: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 763 KiB/s wr, 22 op/s 2023-09-10T09:00:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:51 smithi170 ceph-mon[102832]: pgmap v18762: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 763 KiB/s wr, 22 op/s 2023-09-10T09:00:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:51 smithi082 ceph-mon[98694]: pgmap v18762: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 763 KiB/s wr, 22 op/s 2023-09-10T09:00:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:00:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:00:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:00:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:53 smithi098 ceph-mon[103124]: pgmap v18763: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 547 KiB/s wr, 15 op/s 2023-09-10T09:00:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:53 smithi170 ceph-mon[102832]: pgmap v18763: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 547 KiB/s wr, 15 op/s 2023-09-10T09:00:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:53 smithi082 ceph-mon[98694]: pgmap v18763: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 547 KiB/s wr, 15 op/s 2023-09-10T09:00:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34885.846491 secs 2023-09-10T09:00:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34886.199399 secs 2023-09-10T09:00:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34885.846491 secs 2023-09-10T09:00:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34886.199399 secs 2023-09-10T09:00:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34885.846491 secs 2023-09-10T09:00:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34886.199399 secs 2023-09-10T09:00:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:55 smithi098 ceph-mon[103124]: pgmap v18764: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 548 KiB/s wr, 15 op/s 2023-09-10T09:00:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:55 smithi170 ceph-mon[102832]: pgmap v18764: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 548 KiB/s wr, 15 op/s 2023-09-10T09:00:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:55 smithi082 ceph-mon[98694]: pgmap v18764: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 548 KiB/s wr, 15 op/s 2023-09-10T09:00:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:57 smithi098 ceph-mon[103124]: pgmap v18765: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 604 KiB/s wr, 22 op/s 2023-09-10T09:00:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:57 smithi170 ceph-mon[102832]: pgmap v18765: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 604 KiB/s wr, 22 op/s 2023-09-10T09:00:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:57 smithi082 ceph-mon[98694]: pgmap v18765: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 604 KiB/s wr, 22 op/s 2023-09-10T09:00:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:00:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:00:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:00:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:00:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34890.846550 secs 2023-09-10T09:01:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:59 smithi170 ceph-mon[102832]: pgmap v18766: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 413 KiB/s wr, 15 op/s 2023-09-10T09:01:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:00:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34891.199466 secs 2023-09-10T09:01:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34890.846550 secs 2023-09-10T09:01:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:59 smithi082 ceph-mon[98694]: pgmap v18766: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 413 KiB/s wr, 15 op/s 2023-09-10T09:01:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:00:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34891.199466 secs 2023-09-10T09:01:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34890.846550 secs 2023-09-10T09:01:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:59 smithi098 ceph-mon[103124]: pgmap v18766: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 413 KiB/s wr, 15 op/s 2023-09-10T09:01:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:00:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34891.199466 secs 2023-09-10T09:01:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:01:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:01:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:01:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:01 smithi170 ceph-mon[102832]: pgmap v18767: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 905 KiB/s wr, 25 op/s 2023-09-10T09:01:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:01 smithi082 ceph-mon[98694]: pgmap v18767: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 905 KiB/s wr, 25 op/s 2023-09-10T09:01:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:01 smithi098 ceph-mon[103124]: pgmap v18767: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 905 KiB/s wr, 25 op/s 2023-09-10T09:01:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:01:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:01:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:01:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:01:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:01:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:01:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:01:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:01:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:01:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:01:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:01:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:01:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:03 smithi170 ceph-mon[102832]: pgmap v18768: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 549 KiB/s wr, 17 op/s 2023-09-10T09:01:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34894.595997 secs 2023-09-10T09:01:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:03 smithi082 ceph-mon[98694]: pgmap v18768: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 549 KiB/s wr, 17 op/s 2023-09-10T09:01:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34894.595997 secs 2023-09-10T09:01:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:03 smithi098 ceph-mon[103124]: pgmap v18768: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 549 KiB/s wr, 17 op/s 2023-09-10T09:01:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34894.595997 secs 2023-09-10T09:01:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34895.846623 secs 2023-09-10T09:01:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:01:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:01:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34896.199489 secs 2023-09-10T09:01:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34895.846623 secs 2023-09-10T09:01:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:01:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:01:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:05.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34896.199489 secs 2023-09-10T09:01:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34895.846623 secs 2023-09-10T09:01:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:01:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:01:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34896.199489 secs 2023-09-10T09:01:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:05 smithi170 ceph-mon[102832]: pgmap v18769: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 549 KiB/s wr, 17 op/s 2023-09-10T09:01:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:05 smithi082 ceph-mon[98694]: pgmap v18769: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 549 KiB/s wr, 17 op/s 2023-09-10T09:01:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:05 smithi098 ceph-mon[103124]: pgmap v18769: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 549 KiB/s wr, 17 op/s 2023-09-10T09:01:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:01:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:01:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:01:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:07 smithi170 ceph-mon[102832]: pgmap v18770: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 572 KiB/s wr, 34 op/s 2023-09-10T09:01:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:07 smithi082 ceph-mon[98694]: pgmap v18770: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 572 KiB/s wr, 34 op/s 2023-09-10T09:01:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:07 smithi098 ceph-mon[103124]: pgmap v18770: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 572 KiB/s wr, 34 op/s 2023-09-10T09:01:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34899.596076 secs 2023-09-10T09:01:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34899.596076 secs 2023-09-10T09:01:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34899.596076 secs 2023-09-10T09:01:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34900.846684 secs 2023-09-10T09:01:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:09 smithi170 ceph-mon[102832]: pgmap v18771: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 515 KiB/s wr, 27 op/s 2023-09-10T09:01:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:09 smithi170 ceph-mon[102832]: osdmap e1241: 12 total, 12 up, 12 in 2023-09-10T09:01:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34901.199553 secs 2023-09-10T09:01:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34900.846684 secs 2023-09-10T09:01:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:09 smithi082 ceph-mon[98694]: pgmap v18771: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 515 KiB/s wr, 27 op/s 2023-09-10T09:01:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:09 smithi082 ceph-mon[98694]: osdmap e1241: 12 total, 12 up, 12 in 2023-09-10T09:01:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34901.199553 secs 2023-09-10T09:01:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34900.846684 secs 2023-09-10T09:01:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:09 smithi098 ceph-mon[103124]: pgmap v18771: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 515 KiB/s wr, 27 op/s 2023-09-10T09:01:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:09 smithi098 ceph-mon[103124]: osdmap e1241: 12 total, 12 up, 12 in 2023-09-10T09:01:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34901.199553 secs 2023-09-10T09:01:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:11 smithi170 ceph-mon[102832]: pgmap v18773: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 386 KiB/s wr, 29 op/s 2023-09-10T09:01:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:11 smithi082 ceph-mon[98694]: pgmap v18773: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 386 KiB/s wr, 29 op/s 2023-09-10T09:01:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:11 smithi098 ceph-mon[103124]: pgmap v18773: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 386 KiB/s wr, 29 op/s 2023-09-10T09:01:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:13 smithi170 ceph-mon[102832]: pgmap v18774: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 386 KiB/s wr, 29 op/s 2023-09-10T09:01:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34904.596140 secs 2023-09-10T09:01:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:13 smithi170 ceph-mon[102832]: osdmap e1242: 12 total, 12 up, 12 in 2023-09-10T09:01:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:13 smithi082 ceph-mon[98694]: pgmap v18774: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 386 KiB/s wr, 29 op/s 2023-09-10T09:01:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34904.596140 secs 2023-09-10T09:01:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:13 smithi082 ceph-mon[98694]: osdmap e1242: 12 total, 12 up, 12 in 2023-09-10T09:01:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:13 smithi098 ceph-mon[103124]: pgmap v18774: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 386 KiB/s wr, 29 op/s 2023-09-10T09:01:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34904.596140 secs 2023-09-10T09:01:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:13 smithi098 ceph-mon[103124]: osdmap e1242: 12 total, 12 up, 12 in 2023-09-10T09:01:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34905.846742 secs 2023-09-10T09:01:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:14 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:01:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:14 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34906.199618 secs 2023-09-10T09:01:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34905.846742 secs 2023-09-10T09:01:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:14 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:01:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:14 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34906.199618 secs 2023-09-10T09:01:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34905.846742 secs 2023-09-10T09:01:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:14 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:01:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:14 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34906.199618 secs 2023-09-10T09:01:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:15 smithi170 ceph-mon[102832]: pgmap v18776: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 448 KiB/s wr, 10 op/s 2023-09-10T09:01:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:15 smithi082 ceph-mon[98694]: pgmap v18776: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 448 KiB/s wr, 10 op/s 2023-09-10T09:01:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:15 smithi098 ceph-mon[103124]: pgmap v18776: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 448 KiB/s wr, 10 op/s 2023-09-10T09:01:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:01:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:01:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:01:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:17 smithi170 ceph-mon[102832]: pgmap v18777: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 454 KiB/s wr, 18 op/s 2023-09-10T09:01:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:17 smithi082 ceph-mon[98694]: pgmap v18777: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 454 KiB/s wr, 18 op/s 2023-09-10T09:01:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:17 smithi098 ceph-mon[103124]: pgmap v18777: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 454 KiB/s wr, 18 op/s 2023-09-10T09:01:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34909.596198 secs 2023-09-10T09:01:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34909.596198 secs 2023-09-10T09:01:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34909.596198 secs 2023-09-10T09:01:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34910.846803 secs 2023-09-10T09:01:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:19 smithi170 ceph-mon[102832]: pgmap v18778: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 367 KiB/s wr, 15 op/s 2023-09-10T09:01:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34911.199686 secs 2023-09-10T09:01:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34910.846803 secs 2023-09-10T09:01:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:19 smithi082 ceph-mon[98694]: pgmap v18778: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 367 KiB/s wr, 15 op/s 2023-09-10T09:01:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34911.199686 secs 2023-09-10T09:01:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34910.846803 secs 2023-09-10T09:01:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:19 smithi098 ceph-mon[103124]: pgmap v18778: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 367 KiB/s wr, 15 op/s 2023-09-10T09:01:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34911.199686 secs 2023-09-10T09:01:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:21 smithi170 ceph-mon[102832]: pgmap v18779: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 181 KiB/s wr, 13 op/s 2023-09-10T09:01:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:21 smithi082 ceph-mon[98694]: pgmap v18779: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 181 KiB/s wr, 13 op/s 2023-09-10T09:01:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:21 smithi098 ceph-mon[103124]: pgmap v18779: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 181 KiB/s wr, 13 op/s 2023-09-10T09:01:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:23 smithi170 ceph-mon[102832]: pgmap v18780: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 181 KiB/s wr, 13 op/s 2023-09-10T09:01:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34914.596310 secs 2023-09-10T09:01:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:23 smithi082 ceph-mon[98694]: pgmap v18780: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 181 KiB/s wr, 13 op/s 2023-09-10T09:01:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34914.596310 secs 2023-09-10T09:01:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:23 smithi098 ceph-mon[103124]: pgmap v18780: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 181 KiB/s wr, 13 op/s 2023-09-10T09:01:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34914.596310 secs 2023-09-10T09:01:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34915.846861 secs 2023-09-10T09:01:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34916.199751 secs 2023-09-10T09:01:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34915.846861 secs 2023-09-10T09:01:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34916.199751 secs 2023-09-10T09:01:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34915.846861 secs 2023-09-10T09:01:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34916.199751 secs 2023-09-10T09:01:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:25 smithi170 ceph-mon[102832]: pgmap v18781: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 161 KiB/s wr, 12 op/s 2023-09-10T09:01:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:25 smithi082 ceph-mon[98694]: pgmap v18781: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 161 KiB/s wr, 12 op/s 2023-09-10T09:01:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:25 smithi098 ceph-mon[103124]: pgmap v18781: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 161 KiB/s wr, 12 op/s 2023-09-10T09:01:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:27 smithi170 ceph-mon[102832]: pgmap v18782: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 151 KiB/s wr, 17 op/s 2023-09-10T09:01:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:27 smithi082 ceph-mon[98694]: pgmap v18782: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 151 KiB/s wr, 17 op/s 2023-09-10T09:01:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:27 smithi098 ceph-mon[103124]: pgmap v18782: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 151 KiB/s wr, 17 op/s 2023-09-10T09:01:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34919.596418 secs 2023-09-10T09:01:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34919.596418 secs 2023-09-10T09:01:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34919.596418 secs 2023-09-10T09:01:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34920.846920 secs 2023-09-10T09:01:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:29 smithi170 ceph-mon[102832]: pgmap v18783: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T09:01:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34921.199815 secs 2023-09-10T09:01:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34920.846920 secs 2023-09-10T09:01:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:29 smithi082 ceph-mon[98694]: pgmap v18783: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T09:01:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34921.199815 secs 2023-09-10T09:01:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34920.846920 secs 2023-09-10T09:01:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:29 smithi098 ceph-mon[103124]: pgmap v18783: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T09:01:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34921.199815 secs 2023-09-10T09:01:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:31 smithi170 ceph-mon[102832]: pgmap v18784: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 274 KiB/s wr, 17 op/s 2023-09-10T09:01:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:31 smithi082 ceph-mon[98694]: pgmap v18784: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 274 KiB/s wr, 17 op/s 2023-09-10T09:01:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:31 smithi098 ceph-mon[103124]: pgmap v18784: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 274 KiB/s wr, 17 op/s 2023-09-10T09:01:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:33 smithi170 ceph-mon[102832]: pgmap v18785: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:01:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34924.596478 secs 2023-09-10T09:01:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:33 smithi082 ceph-mon[98694]: pgmap v18785: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:01:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34924.596478 secs 2023-09-10T09:01:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:33 smithi098 ceph-mon[103124]: pgmap v18785: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:01:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34924.596478 secs 2023-09-10T09:01:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34925.846980 secs 2023-09-10T09:01:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34926.199882 secs 2023-09-10T09:01:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34925.846980 secs 2023-09-10T09:01:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34926.199882 secs 2023-09-10T09:01:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34925.846980 secs 2023-09-10T09:01:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34926.199882 secs 2023-09-10T09:01:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:35 smithi170 ceph-mon[102832]: pgmap v18786: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:01:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:35 smithi082 ceph-mon[98694]: pgmap v18786: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:01:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:35 smithi098 ceph-mon[103124]: pgmap v18786: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:01:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:37 smithi170 ceph-mon[102832]: pgmap v18787: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 252 KiB/s wr, 16 op/s 2023-09-10T09:01:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:01:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:37 smithi082 ceph-mon[98694]: pgmap v18787: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 252 KiB/s wr, 16 op/s 2023-09-10T09:01:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:01:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:37 smithi098 ceph-mon[103124]: pgmap v18787: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 252 KiB/s wr, 16 op/s 2023-09-10T09:01:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:01:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34930.847039 secs 2023-09-10T09:01:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:39 smithi098 ceph-mon[103124]: pgmap v18788: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 251 KiB/s wr, 11 op/s 2023-09-10T09:01:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34931.199946 secs 2023-09-10T09:01:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:01:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34930.847039 secs 2023-09-10T09:01:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:39 smithi170 ceph-mon[102832]: pgmap v18788: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 251 KiB/s wr, 11 op/s 2023-09-10T09:01:39.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34931.199946 secs 2023-09-10T09:01:39.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:01:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34930.847039 secs 2023-09-10T09:01:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:39 smithi082 ceph-mon[98694]: pgmap v18788: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 251 KiB/s wr, 11 op/s 2023-09-10T09:01:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34931.199946 secs 2023-09-10T09:01:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:01:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:01:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:01:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:01:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:01:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:01:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:01:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:41 smithi098 ceph-mon[103124]: pgmap v18789: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 921 KiB/s wr, 19 op/s 2023-09-10T09:01:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:41 smithi170 ceph-mon[102832]: pgmap v18789: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 921 KiB/s wr, 19 op/s 2023-09-10T09:01:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:41 smithi082 ceph-mon[98694]: pgmap v18789: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 921 KiB/s wr, 19 op/s 2023-09-10T09:01:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:43 smithi170 ceph-mon[102832]: pgmap v18790: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 793 KiB/s wr, 13 op/s 2023-09-10T09:01:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:43 smithi082 ceph-mon[98694]: pgmap v18790: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 793 KiB/s wr, 13 op/s 2023-09-10T09:01:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:43 smithi098 ceph-mon[103124]: pgmap v18790: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 793 KiB/s wr, 13 op/s 2023-09-10T09:01:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34935.847122 secs 2023-09-10T09:01:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:01:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34936.200013 secs 2023-09-10T09:01:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34935.847122 secs 2023-09-10T09:01:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:01:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34936.200013 secs 2023-09-10T09:01:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34935.847122 secs 2023-09-10T09:01:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:01:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34936.200013 secs 2023-09-10T09:01:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:45 smithi170 ceph-mon[102832]: pgmap v18791: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 794 KiB/s wr, 13 op/s 2023-09-10T09:01:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:45 smithi082 ceph-mon[98694]: pgmap v18791: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 794 KiB/s wr, 13 op/s 2023-09-10T09:01:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:45 smithi098 ceph-mon[103124]: pgmap v18791: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 794 KiB/s wr, 13 op/s 2023-09-10T09:01:47.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:01:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:01:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:01:47.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:47 smithi170 ceph-mon[102832]: pgmap v18792: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 819 KiB/s wr, 20 op/s 2023-09-10T09:01:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:47 smithi082 ceph-mon[98694]: pgmap v18792: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 819 KiB/s wr, 20 op/s 2023-09-10T09:01:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:47 smithi098 ceph-mon[103124]: pgmap v18792: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 819 KiB/s wr, 20 op/s 2023-09-10T09:01:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34940.847204 secs 2023-09-10T09:01:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:49 smithi098 ceph-mon[103124]: pgmap v18793: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 694 KiB/s wr, 14 op/s 2023-09-10T09:01:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34941.200078 secs 2023-09-10T09:01:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34940.847204 secs 2023-09-10T09:01:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:49 smithi170 ceph-mon[102832]: pgmap v18793: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 694 KiB/s wr, 14 op/s 2023-09-10T09:01:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34941.200078 secs 2023-09-10T09:01:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34940.847204 secs 2023-09-10T09:01:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:49 smithi082 ceph-mon[98694]: pgmap v18793: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 694 KiB/s wr, 14 op/s 2023-09-10T09:01:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34941.200078 secs 2023-09-10T09:01:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:51 smithi170 ceph-mon[102832]: pgmap v18794: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:01:52.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:51 smithi082 ceph-mon[98694]: pgmap v18794: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:01:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:51 smithi098 ceph-mon[103124]: pgmap v18794: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:01:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:53 smithi170 ceph-mon[102832]: pgmap v18795: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 471 KiB/s wr, 13 op/s 2023-09-10T09:01:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:53 smithi082 ceph-mon[98694]: pgmap v18795: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 471 KiB/s wr, 13 op/s 2023-09-10T09:01:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:53 smithi098 ceph-mon[103124]: pgmap v18795: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 471 KiB/s wr, 13 op/s 2023-09-10T09:01:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34945.847260 secs 2023-09-10T09:01:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:01:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34946.200145 secs 2023-09-10T09:01:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34945.847260 secs 2023-09-10T09:01:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:01:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34946.200145 secs 2023-09-10T09:01:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34945.847260 secs 2023-09-10T09:01:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:01:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34946.200145 secs 2023-09-10T09:01:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:55 smithi170 ceph-mon[102832]: pgmap v18796: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 471 KiB/s wr, 13 op/s 2023-09-10T09:01:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:55 smithi082 ceph-mon[98694]: pgmap v18796: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 471 KiB/s wr, 13 op/s 2023-09-10T09:01:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:55 smithi098 ceph-mon[103124]: pgmap v18796: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 471 KiB/s wr, 13 op/s 2023-09-10T09:01:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:01:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:01:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:01:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:01:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:57 smithi170 ceph-mon[102832]: pgmap v18797: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 956 KiB/s wr, 20 op/s 2023-09-10T09:01:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:57 smithi082 ceph-mon[98694]: pgmap v18797: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 956 KiB/s wr, 20 op/s 2023-09-10T09:01:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:57 smithi098 ceph-mon[103124]: pgmap v18797: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 956 KiB/s wr, 20 op/s 2023-09-10T09:01:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34949.596833 secs 2023-09-10T09:01:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34949.596833 secs 2023-09-10T09:01:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34949.596833 secs 2023-09-10T09:02:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34950.847319 secs 2023-09-10T09:02:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:59 smithi170 ceph-mon[102832]: pgmap v18798: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 931 KiB/s wr, 14 op/s 2023-09-10T09:02:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:01:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34951.200211 secs 2023-09-10T09:02:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34950.847319 secs 2023-09-10T09:02:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:59 smithi082 ceph-mon[98694]: pgmap v18798: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 931 KiB/s wr, 14 op/s 2023-09-10T09:02:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:01:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34951.200211 secs 2023-09-10T09:02:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34950.847319 secs 2023-09-10T09:02:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:59 smithi098 ceph-mon[103124]: pgmap v18798: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 931 KiB/s wr, 14 op/s 2023-09-10T09:02:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:01:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34951.200211 secs 2023-09-10T09:02:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:02:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:02:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:02:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:02:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:02:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:02:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:01 smithi170 ceph-mon[102832]: pgmap v18799: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 20 op/s 2023-09-10T09:02:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:01 smithi082 ceph-mon[98694]: pgmap v18799: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 20 op/s 2023-09-10T09:02:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:01 smithi098 ceph-mon[103124]: pgmap v18799: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 20 op/s 2023-09-10T09:02:03.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:02:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:02:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:02:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:03 smithi170 ceph-mon[102832]: pgmap v18800: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 647 KiB/s wr, 13 op/s 2023-09-10T09:02:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34954.596895 secs 2023-09-10T09:02:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:02:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:02:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:02:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:03 smithi082 ceph-mon[98694]: pgmap v18800: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 647 KiB/s wr, 13 op/s 2023-09-10T09:02:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34954.596895 secs 2023-09-10T09:02:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:02:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:02:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:02:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:03 smithi098 ceph-mon[103124]: pgmap v18800: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 647 KiB/s wr, 13 op/s 2023-09-10T09:02:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34954.596895 secs 2023-09-10T09:02:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:02:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:02:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:02:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34955.847380 secs 2023-09-10T09:02:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:04 smithi170 ceph-mon[102832]: osdmap e1243: 12 total, 12 up, 12 in 2023-09-10T09:02:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34956.200274 secs 2023-09-10T09:02:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:02:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34955.847380 secs 2023-09-10T09:02:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:04 smithi082 ceph-mon[98694]: osdmap e1243: 12 total, 12 up, 12 in 2023-09-10T09:02:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34956.200274 secs 2023-09-10T09:02:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:02:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34955.847380 secs 2023-09-10T09:02:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:04 smithi098 ceph-mon[103124]: osdmap e1243: 12 total, 12 up, 12 in 2023-09-10T09:02:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34956.200274 secs 2023-09-10T09:02:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:02:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:06 smithi170 ceph-mon[102832]: pgmap v18802: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 777 KiB/s wr, 16 op/s 2023-09-10T09:02:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:02:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:06 smithi082 ceph-mon[98694]: pgmap v18802: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 777 KiB/s wr, 16 op/s 2023-09-10T09:02:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:02:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:06 smithi098 ceph-mon[103124]: pgmap v18802: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 777 KiB/s wr, 16 op/s 2023-09-10T09:02:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:02:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:08 smithi170 ceph-mon[102832]: pgmap v18803: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 432 KiB/s wr, 31 op/s 2023-09-10T09:02:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34959.596985 secs 2023-09-10T09:02:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:08 smithi082 ceph-mon[98694]: pgmap v18803: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 432 KiB/s wr, 31 op/s 2023-09-10T09:02:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34959.596985 secs 2023-09-10T09:02:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:08 smithi098 ceph-mon[103124]: pgmap v18803: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 432 KiB/s wr, 31 op/s 2023-09-10T09:02:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34959.596985 secs 2023-09-10T09:02:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:09 smithi098 ceph-mon[103124]: osdmap e1244: 12 total, 12 up, 12 in 2023-09-10T09:02:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34960.847440 secs 2023-09-10T09:02:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:09 smithi098 ceph-mon[103124]: pgmap v18805: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 297 KiB/s wr, 30 op/s 2023-09-10T09:02:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34961.200339 secs 2023-09-10T09:02:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:09 smithi170 ceph-mon[102832]: osdmap e1244: 12 total, 12 up, 12 in 2023-09-10T09:02:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34960.847440 secs 2023-09-10T09:02:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:09 smithi170 ceph-mon[102832]: pgmap v18805: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 297 KiB/s wr, 30 op/s 2023-09-10T09:02:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34961.200339 secs 2023-09-10T09:02:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:09 smithi082 ceph-mon[98694]: osdmap e1244: 12 total, 12 up, 12 in 2023-09-10T09:02:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34960.847440 secs 2023-09-10T09:02:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:09 smithi082 ceph-mon[98694]: pgmap v18805: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 297 KiB/s wr, 30 op/s 2023-09-10T09:02:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34961.200339 secs 2023-09-10T09:02:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:11 smithi170 ceph-mon[102832]: pgmap v18806: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 519 KiB/s wr, 39 op/s 2023-09-10T09:02:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:11 smithi082 ceph-mon[98694]: pgmap v18806: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 519 KiB/s wr, 39 op/s 2023-09-10T09:02:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:11 smithi098 ceph-mon[103124]: pgmap v18806: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 519 KiB/s wr, 39 op/s 2023-09-10T09:02:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:13 smithi170 ceph-mon[102832]: pgmap v18807: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 466 KiB/s wr, 35 op/s 2023-09-10T09:02:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34964.597086 secs 2023-09-10T09:02:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:13 smithi082 ceph-mon[98694]: pgmap v18807: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 466 KiB/s wr, 35 op/s 2023-09-10T09:02:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34964.597086 secs 2023-09-10T09:02:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:13 smithi098 ceph-mon[103124]: pgmap v18807: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 466 KiB/s wr, 35 op/s 2023-09-10T09:02:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34964.597086 secs 2023-09-10T09:02:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34965.847496 secs 2023-09-10T09:02:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34966.200402 secs 2023-09-10T09:02:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34965.847496 secs 2023-09-10T09:02:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34966.200402 secs 2023-09-10T09:02:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34965.847496 secs 2023-09-10T09:02:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34966.200402 secs 2023-09-10T09:02:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:15 smithi170 ceph-mon[102832]: pgmap v18808: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 415 KiB/s wr, 31 op/s 2023-09-10T09:02:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:15 smithi082 ceph-mon[98694]: pgmap v18808: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 415 KiB/s wr, 31 op/s 2023-09-10T09:02:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:15 smithi098 ceph-mon[103124]: pgmap v18808: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 415 KiB/s wr, 31 op/s 2023-09-10T09:02:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:17 smithi170 ceph-mon[102832]: pgmap v18809: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 183 KiB/s wr, 15 op/s 2023-09-10T09:02:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:17 smithi082 ceph-mon[98694]: pgmap v18809: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 183 KiB/s wr, 15 op/s 2023-09-10T09:02:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:17 smithi098 ceph-mon[103124]: pgmap v18809: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 183 KiB/s wr, 15 op/s 2023-09-10T09:02:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34969.597160 secs 2023-09-10T09:02:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34969.597160 secs 2023-09-10T09:02:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34969.597160 secs 2023-09-10T09:02:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34970.847558 secs 2023-09-10T09:02:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:19 smithi170 ceph-mon[102832]: pgmap v18810: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 179 KiB/s wr, 14 op/s 2023-09-10T09:02:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34971.200472 secs 2023-09-10T09:02:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34970.847558 secs 2023-09-10T09:02:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:19 smithi082 ceph-mon[98694]: pgmap v18810: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 179 KiB/s wr, 14 op/s 2023-09-10T09:02:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34971.200472 secs 2023-09-10T09:02:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34970.847558 secs 2023-09-10T09:02:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:19 smithi098 ceph-mon[103124]: pgmap v18810: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 179 KiB/s wr, 14 op/s 2023-09-10T09:02:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34971.200472 secs 2023-09-10T09:02:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:21 smithi170 ceph-mon[102832]: pgmap v18811: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 253 KiB/s wr, 18 op/s 2023-09-10T09:02:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:21 smithi082 ceph-mon[98694]: pgmap v18811: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 253 KiB/s wr, 18 op/s 2023-09-10T09:02:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:21 smithi098 ceph-mon[103124]: pgmap v18811: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 253 KiB/s wr, 18 op/s 2023-09-10T09:02:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:23 smithi170 ceph-mon[102832]: pgmap v18812: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T09:02:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34974.597244 secs 2023-09-10T09:02:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:23 smithi082 ceph-mon[98694]: pgmap v18812: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T09:02:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34974.597244 secs 2023-09-10T09:02:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:23 smithi098 ceph-mon[103124]: pgmap v18812: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T09:02:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34974.597244 secs 2023-09-10T09:02:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34975.847616 secs 2023-09-10T09:02:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:02:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34976.200487 secs 2023-09-10T09:02:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34975.847616 secs 2023-09-10T09:02:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:02:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34976.200487 secs 2023-09-10T09:02:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34975.847616 secs 2023-09-10T09:02:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:02:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34976.200487 secs 2023-09-10T09:02:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:25 smithi170 ceph-mon[102832]: pgmap v18813: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T09:02:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:25 smithi082 ceph-mon[98694]: pgmap v18813: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T09:02:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:25 smithi098 ceph-mon[103124]: pgmap v18813: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-09-10T09:02:27.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:26 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:02:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:26 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:02:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:26 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:02:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:27 smithi170 ceph-mon[102832]: pgmap v18814: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 232 KiB/s wr, 18 op/s 2023-09-10T09:02:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:27 smithi082 ceph-mon[98694]: pgmap v18814: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 232 KiB/s wr, 18 op/s 2023-09-10T09:02:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:27 smithi098 ceph-mon[103124]: pgmap v18814: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 232 KiB/s wr, 18 op/s 2023-09-10T09:02:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34979.597307 secs 2023-09-10T09:02:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34979.597307 secs 2023-09-10T09:02:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34979.597307 secs 2023-09-10T09:02:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34980.847676 secs 2023-09-10T09:02:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:29 smithi170 ceph-mon[102832]: pgmap v18815: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 12 op/s 2023-09-10T09:02:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34981.200551 secs 2023-09-10T09:02:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34980.847676 secs 2023-09-10T09:02:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:29 smithi082 ceph-mon[98694]: pgmap v18815: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 12 op/s 2023-09-10T09:02:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34981.200551 secs 2023-09-10T09:02:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34980.847676 secs 2023-09-10T09:02:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:29 smithi098 ceph-mon[103124]: pgmap v18815: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 12 op/s 2023-09-10T09:02:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34981.200551 secs 2023-09-10T09:02:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:31 smithi170 ceph-mon[102832]: pgmap v18816: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 228 KiB/s wr, 18 op/s 2023-09-10T09:02:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:31 smithi082 ceph-mon[98694]: pgmap v18816: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 228 KiB/s wr, 18 op/s 2023-09-10T09:02:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:31 smithi098 ceph-mon[103124]: pgmap v18816: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 228 KiB/s wr, 18 op/s 2023-09-10T09:02:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:33 smithi170 ceph-mon[102832]: pgmap v18817: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T09:02:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34984.597368 secs 2023-09-10T09:02:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:33 smithi082 ceph-mon[98694]: pgmap v18817: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T09:02:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34984.597368 secs 2023-09-10T09:02:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:33 smithi098 ceph-mon[103124]: pgmap v18817: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T09:02:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34984.597368 secs 2023-09-10T09:02:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34985.847734 secs 2023-09-10T09:02:35.237 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34986.200620 secs 2023-09-10T09:02:35.238 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34985.847734 secs 2023-09-10T09:02:35.238 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34986.200620 secs 2023-09-10T09:02:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34985.847734 secs 2023-09-10T09:02:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34986.200620 secs 2023-09-10T09:02:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:35 smithi170 ceph-mon[102832]: pgmap v18818: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T09:02:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:35 smithi082 ceph-mon[98694]: pgmap v18818: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T09:02:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:35 smithi098 ceph-mon[103124]: pgmap v18818: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 12 op/s 2023-09-10T09:02:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:37 smithi170 ceph-mon[102832]: pgmap v18819: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 289 KiB/s wr, 18 op/s 2023-09-10T09:02:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:37 smithi082 ceph-mon[98694]: pgmap v18819: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 289 KiB/s wr, 18 op/s 2023-09-10T09:02:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:37 smithi098 ceph-mon[103124]: pgmap v18819: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 289 KiB/s wr, 18 op/s 2023-09-10T09:02:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34989.597431 secs 2023-09-10T09:02:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34989.597431 secs 2023-09-10T09:02:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34989.597431 secs 2023-09-10T09:02:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34990.847793 secs 2023-09-10T09:02:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:39 smithi170 ceph-mon[102832]: pgmap v18820: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 162 KiB/s wr, 12 op/s 2023-09-10T09:02:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34991.200686 secs 2023-09-10T09:02:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34990.847793 secs 2023-09-10T09:02:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:39 smithi082 ceph-mon[98694]: pgmap v18820: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 162 KiB/s wr, 12 op/s 2023-09-10T09:02:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34991.200686 secs 2023-09-10T09:02:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34990.847793 secs 2023-09-10T09:02:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:39 smithi098 ceph-mon[103124]: pgmap v18820: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 162 KiB/s wr, 12 op/s 2023-09-10T09:02:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34991.200686 secs 2023-09-10T09:02:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:41 smithi170 ceph-mon[102832]: pgmap v18821: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 425 KiB/s wr, 19 op/s 2023-09-10T09:02:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:41 smithi082 ceph-mon[98694]: pgmap v18821: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 425 KiB/s wr, 19 op/s 2023-09-10T09:02:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:41 smithi098 ceph-mon[103124]: pgmap v18821: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 425 KiB/s wr, 19 op/s 2023-09-10T09:02:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:43 smithi170 ceph-mon[102832]: pgmap v18822: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 425 KiB/s wr, 13 op/s 2023-09-10T09:02:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 34994.597491 secs 2023-09-10T09:02:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:43 smithi082 ceph-mon[98694]: pgmap v18822: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 425 KiB/s wr, 13 op/s 2023-09-10T09:02:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 34994.597491 secs 2023-09-10T09:02:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:43 smithi098 ceph-mon[103124]: pgmap v18822: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 425 KiB/s wr, 13 op/s 2023-09-10T09:02:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 34994.597491 secs 2023-09-10T09:02:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34995.847852 secs 2023-09-10T09:02:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:02:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 34996.200751 secs 2023-09-10T09:02:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34995.847852 secs 2023-09-10T09:02:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:02:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 34996.200751 secs 2023-09-10T09:02:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34995.847852 secs 2023-09-10T09:02:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:02:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 34996.200751 secs 2023-09-10T09:02:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:45 smithi170 ceph-mon[102832]: pgmap v18823: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 425 KiB/s wr, 13 op/s 2023-09-10T09:02:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:02:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:45 smithi082 ceph-mon[98694]: pgmap v18823: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 425 KiB/s wr, 13 op/s 2023-09-10T09:02:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:02:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:45 smithi098 ceph-mon[103124]: pgmap v18823: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 425 KiB/s wr, 13 op/s 2023-09-10T09:02:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:02:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:47 smithi170 ceph-mon[102832]: pgmap v18824: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 556 KiB/s wr, 20 op/s 2023-09-10T09:02:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:47 smithi082 ceph-mon[98694]: pgmap v18824: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 556 KiB/s wr, 20 op/s 2023-09-10T09:02:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:47 smithi098 ceph-mon[103124]: pgmap v18824: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 556 KiB/s wr, 20 op/s 2023-09-10T09:02:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35000.847914 secs 2023-09-10T09:02:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:49 smithi098 ceph-mon[103124]: pgmap v18825: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 394 KiB/s wr, 13 op/s 2023-09-10T09:02:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35001.200819 secs 2023-09-10T09:02:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35000.847914 secs 2023-09-10T09:02:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:49 smithi170 ceph-mon[102832]: pgmap v18825: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 394 KiB/s wr, 13 op/s 2023-09-10T09:02:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35001.200819 secs 2023-09-10T09:02:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35000.847914 secs 2023-09-10T09:02:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:49 smithi082 ceph-mon[98694]: pgmap v18825: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 394 KiB/s wr, 13 op/s 2023-09-10T09:02:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35001.200819 secs 2023-09-10T09:02:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:02:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:02:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:02:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:51 smithi098 ceph-mon[103124]: pgmap v18826: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 808 KiB/s wr, 21 op/s 2023-09-10T09:02:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:51 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:02:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:51 smithi170 ceph-mon[102832]: pgmap v18826: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 808 KiB/s wr, 21 op/s 2023-09-10T09:02:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:51 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:02:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:51 smithi082 ceph-mon[98694]: pgmap v18826: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 808 KiB/s wr, 21 op/s 2023-09-10T09:02:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:51 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:02:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:52 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:02:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:02:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:52 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:02:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:02:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:52 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:02:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:02:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:53 smithi170 ceph-mon[102832]: pgmap v18827: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 544 KiB/s wr, 14 op/s 2023-09-10T09:02:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:53 smithi082 ceph-mon[98694]: pgmap v18827: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 544 KiB/s wr, 14 op/s 2023-09-10T09:02:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:53 smithi098 ceph-mon[103124]: pgmap v18827: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 544 KiB/s wr, 14 op/s 2023-09-10T09:02:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:02:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35005.847987 secs 2023-09-10T09:02:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35006.200882 secs 2023-09-10T09:02:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:02:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35005.847987 secs 2023-09-10T09:02:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35006.200882 secs 2023-09-10T09:02:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:02:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35005.847987 secs 2023-09-10T09:02:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35006.200882 secs 2023-09-10T09:02:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:55 smithi170 ceph-mon[102832]: pgmap v18828: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 544 KiB/s wr, 14 op/s 2023-09-10T09:02:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:55 smithi082 ceph-mon[98694]: pgmap v18828: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 544 KiB/s wr, 14 op/s 2023-09-10T09:02:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:55 smithi098 ceph-mon[103124]: pgmap v18828: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 544 KiB/s wr, 14 op/s 2023-09-10T09:02:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:02:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:02:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:02:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:57 smithi170 ceph-mon[102832]: pgmap v18829: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:02:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:57 smithi082 ceph-mon[98694]: pgmap v18829: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:02:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:57 smithi098 ceph-mon[103124]: pgmap v18829: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:02:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:02:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35009.597672 secs 2023-09-10T09:02:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:02:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35009.597672 secs 2023-09-10T09:02:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:02:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:02:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35009.597672 secs 2023-09-10T09:02:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35010.848045 secs 2023-09-10T09:02:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:59 smithi170 ceph-mon[102832]: pgmap v18830: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 988 KiB/s wr, 15 op/s 2023-09-10T09:02:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:02:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35011.200947 secs 2023-09-10T09:02:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35010.848045 secs 2023-09-10T09:02:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:59 smithi082 ceph-mon[98694]: pgmap v18830: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 988 KiB/s wr, 15 op/s 2023-09-10T09:02:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:02:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35011.200947 secs 2023-09-10T09:03:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35010.848045 secs 2023-09-10T09:03:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:59 smithi098 ceph-mon[103124]: pgmap v18830: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 988 KiB/s wr, 15 op/s 2023-09-10T09:03:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:02:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35011.200947 secs 2023-09-10T09:03:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:03:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:03:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:03:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:03:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:03:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:03:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:01 smithi170 ceph-mon[102832]: pgmap v18831: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 22 op/s 2023-09-10T09:03:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:01 smithi082 ceph-mon[98694]: pgmap v18831: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 22 op/s 2023-09-10T09:03:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:01 smithi098 ceph-mon[103124]: pgmap v18831: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 22 op/s 2023-09-10T09:03:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:03 smithi170 ceph-mon[102832]: pgmap v18832: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 14 op/s 2023-09-10T09:03:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35014.597731 secs 2023-09-10T09:03:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:03:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:03:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:03:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:03 smithi082 ceph-mon[98694]: pgmap v18832: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 14 op/s 2023-09-10T09:03:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35014.597731 secs 2023-09-10T09:03:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:03:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:03:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:03:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:03 smithi098 ceph-mon[103124]: pgmap v18832: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 14 op/s 2023-09-10T09:03:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35014.597731 secs 2023-09-10T09:03:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:03:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:03:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:03:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35015.848105 secs 2023-09-10T09:03:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:03:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:03:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35016.201013 secs 2023-09-10T09:03:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35015.848105 secs 2023-09-10T09:03:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:03:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:03:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35016.201013 secs 2023-09-10T09:03:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35015.848105 secs 2023-09-10T09:03:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:03:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:03:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35016.201013 secs 2023-09-10T09:03:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:05 smithi170 ceph-mon[102832]: pgmap v18833: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 14 op/s 2023-09-10T09:03:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:05 smithi082 ceph-mon[98694]: pgmap v18833: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 14 op/s 2023-09-10T09:03:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:05 smithi098 ceph-mon[103124]: pgmap v18833: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 14 op/s 2023-09-10T09:03:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:03:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:03:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:03:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:07 smithi170 ceph-mon[102832]: pgmap v18834: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 35 op/s 2023-09-10T09:03:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:07 smithi082 ceph-mon[98694]: pgmap v18834: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 35 op/s 2023-09-10T09:03:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:07 smithi098 ceph-mon[103124]: pgmap v18834: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 35 op/s 2023-09-10T09:03:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35019.597818 secs 2023-09-10T09:03:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35019.597818 secs 2023-09-10T09:03:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35019.597818 secs 2023-09-10T09:03:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35020.848171 secs 2023-09-10T09:03:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:09 smithi170 ceph-mon[102832]: pgmap v18835: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 611 KiB/s wr, 28 op/s 2023-09-10T09:03:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:09 smithi170 ceph-mon[102832]: osdmap e1245: 12 total, 12 up, 12 in 2023-09-10T09:03:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35021.201078 secs 2023-09-10T09:03:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35020.848171 secs 2023-09-10T09:03:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:09 smithi082 ceph-mon[98694]: pgmap v18835: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 611 KiB/s wr, 28 op/s 2023-09-10T09:03:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:09 smithi082 ceph-mon[98694]: osdmap e1245: 12 total, 12 up, 12 in 2023-09-10T09:03:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35021.201078 secs 2023-09-10T09:03:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35020.848171 secs 2023-09-10T09:03:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:09 smithi098 ceph-mon[103124]: pgmap v18835: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 611 KiB/s wr, 28 op/s 2023-09-10T09:03:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:09 smithi098 ceph-mon[103124]: osdmap e1245: 12 total, 12 up, 12 in 2023-09-10T09:03:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35021.201078 secs 2023-09-10T09:03:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:11 smithi170 ceph-mon[102832]: pgmap v18837: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 231 KiB/s wr, 32 op/s 2023-09-10T09:03:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:11 smithi082 ceph-mon[98694]: pgmap v18837: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 231 KiB/s wr, 32 op/s 2023-09-10T09:03:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:11 smithi098 ceph-mon[103124]: pgmap v18837: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 231 KiB/s wr, 32 op/s 2023-09-10T09:03:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:13 smithi170 ceph-mon[102832]: pgmap v18838: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 231 KiB/s wr, 32 op/s 2023-09-10T09:03:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35024.597949 secs 2023-09-10T09:03:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:13 smithi170 ceph-mon[102832]: osdmap e1246: 12 total, 12 up, 12 in 2023-09-10T09:03:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:13 smithi082 ceph-mon[98694]: pgmap v18838: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 231 KiB/s wr, 32 op/s 2023-09-10T09:03:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35024.597949 secs 2023-09-10T09:03:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:13 smithi082 ceph-mon[98694]: osdmap e1246: 12 total, 12 up, 12 in 2023-09-10T09:03:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:13 smithi098 ceph-mon[103124]: pgmap v18838: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 231 KiB/s wr, 32 op/s 2023-09-10T09:03:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35024.597949 secs 2023-09-10T09:03:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:13 smithi098 ceph-mon[103124]: osdmap e1246: 12 total, 12 up, 12 in 2023-09-10T09:03:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35025.848228 secs 2023-09-10T09:03:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35026.201144 secs 2023-09-10T09:03:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35025.848228 secs 2023-09-10T09:03:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35026.201144 secs 2023-09-10T09:03:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35025.848228 secs 2023-09-10T09:03:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35026.201144 secs 2023-09-10T09:03:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:15 smithi170 ceph-mon[102832]: pgmap v18840: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 220 KiB/s wr, 9 op/s 2023-09-10T09:03:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:15 smithi082 ceph-mon[98694]: pgmap v18840: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 220 KiB/s wr, 9 op/s 2023-09-10T09:03:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:15 smithi098 ceph-mon[103124]: pgmap v18840: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 220 KiB/s wr, 9 op/s 2023-09-10T09:03:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:17 smithi170 ceph-mon[102832]: pgmap v18841: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 220 KiB/s wr, 16 op/s 2023-09-10T09:03:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:17 smithi082 ceph-mon[98694]: pgmap v18841: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 220 KiB/s wr, 16 op/s 2023-09-10T09:03:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:17 smithi098 ceph-mon[103124]: pgmap v18841: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 220 KiB/s wr, 16 op/s 2023-09-10T09:03:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35029.598009 secs 2023-09-10T09:03:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35029.598009 secs 2023-09-10T09:03:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35029.598009 secs 2023-09-10T09:03:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35030.848308 secs 2023-09-10T09:03:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:19 smithi170 ceph-mon[102832]: pgmap v18842: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 13 op/s 2023-09-10T09:03:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35031.201210 secs 2023-09-10T09:03:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35030.848308 secs 2023-09-10T09:03:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:19 smithi082 ceph-mon[98694]: pgmap v18842: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 13 op/s 2023-09-10T09:03:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35031.201210 secs 2023-09-10T09:03:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35030.848308 secs 2023-09-10T09:03:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:19 smithi098 ceph-mon[103124]: pgmap v18842: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 13 op/s 2023-09-10T09:03:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35031.201210 secs 2023-09-10T09:03:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:21 smithi170 ceph-mon[102832]: pgmap v18843: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T09:03:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:21 smithi082 ceph-mon[98694]: pgmap v18843: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T09:03:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:21 smithi098 ceph-mon[103124]: pgmap v18843: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T09:03:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:23 smithi170 ceph-mon[102832]: pgmap v18844: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T09:03:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35034.598071 secs 2023-09-10T09:03:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:23 smithi082 ceph-mon[98694]: pgmap v18844: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T09:03:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35034.598071 secs 2023-09-10T09:03:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:23 smithi098 ceph-mon[103124]: pgmap v18844: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T09:03:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35034.598071 secs 2023-09-10T09:03:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35035.848368 secs 2023-09-10T09:03:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35036.201277 secs 2023-09-10T09:03:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35035.848368 secs 2023-09-10T09:03:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35036.201277 secs 2023-09-10T09:03:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35035.848368 secs 2023-09-10T09:03:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35036.201277 secs 2023-09-10T09:03:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:25 smithi170 ceph-mon[102832]: pgmap v18845: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T09:03:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:25 smithi082 ceph-mon[98694]: pgmap v18845: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T09:03:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:25 smithi098 ceph-mon[103124]: pgmap v18845: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T09:03:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:27 smithi170 ceph-mon[102832]: pgmap v18846: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 15 op/s 2023-09-10T09:03:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:27 smithi082 ceph-mon[98694]: pgmap v18846: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 15 op/s 2023-09-10T09:03:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:27 smithi098 ceph-mon[103124]: pgmap v18846: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 15 op/s 2023-09-10T09:03:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35039.598156 secs 2023-09-10T09:03:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35039.598156 secs 2023-09-10T09:03:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35039.598156 secs 2023-09-10T09:03:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35040.848426 secs 2023-09-10T09:03:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:29 smithi170 ceph-mon[102832]: pgmap v18847: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-10T09:03:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35041.201337 secs 2023-09-10T09:03:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35040.848426 secs 2023-09-10T09:03:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:29 smithi082 ceph-mon[98694]: pgmap v18847: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-10T09:03:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35041.201337 secs 2023-09-10T09:03:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35040.848426 secs 2023-09-10T09:03:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:29 smithi098 ceph-mon[103124]: pgmap v18847: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-09-10T09:03:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35041.201337 secs 2023-09-10T09:03:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:31 smithi170 ceph-mon[102832]: pgmap v18848: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 16 op/s 2023-09-10T09:03:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:31 smithi082 ceph-mon[98694]: pgmap v18848: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 16 op/s 2023-09-10T09:03:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:31 smithi098 ceph-mon[103124]: pgmap v18848: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 16 op/s 2023-09-10T09:03:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:33 smithi170 ceph-mon[102832]: pgmap v18849: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:03:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:33 smithi082 ceph-mon[98694]: pgmap v18849: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:03:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:33 smithi098 ceph-mon[103124]: pgmap v18849: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:03:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35045.848509 secs 2023-09-10T09:03:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35046.201406 secs 2023-09-10T09:03:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35045.848509 secs 2023-09-10T09:03:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35046.201406 secs 2023-09-10T09:03:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35045.848509 secs 2023-09-10T09:03:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35046.201406 secs 2023-09-10T09:03:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:35 smithi170 ceph-mon[102832]: pgmap v18850: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:03:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:03:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:35 smithi082 ceph-mon[98694]: pgmap v18850: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:03:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:03:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:35 smithi098 ceph-mon[103124]: pgmap v18850: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:03:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:03:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:03:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:03:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:03:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:03:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:03:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:03:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:37 smithi170 ceph-mon[102832]: pgmap v18851: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 253 KiB/s wr, 16 op/s 2023-09-10T09:03:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:03:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:37 smithi082 ceph-mon[98694]: pgmap v18851: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 253 KiB/s wr, 16 op/s 2023-09-10T09:03:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:03:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:37 smithi098 ceph-mon[103124]: pgmap v18851: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 253 KiB/s wr, 16 op/s 2023-09-10T09:03:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:03:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35049.598264 secs 2023-09-10T09:03:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35049.598264 secs 2023-09-10T09:03:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35049.598264 secs 2023-09-10T09:03:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35050.848570 secs 2023-09-10T09:03:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:39 smithi170 ceph-mon[102832]: pgmap v18852: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 252 KiB/s wr, 11 op/s 2023-09-10T09:03:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35051.201476 secs 2023-09-10T09:03:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35050.848570 secs 2023-09-10T09:03:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:39 smithi082 ceph-mon[98694]: pgmap v18852: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 252 KiB/s wr, 11 op/s 2023-09-10T09:03:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35051.201476 secs 2023-09-10T09:03:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35050.848570 secs 2023-09-10T09:03:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:39 smithi098 ceph-mon[103124]: pgmap v18852: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 252 KiB/s wr, 11 op/s 2023-09-10T09:03:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35051.201476 secs 2023-09-10T09:03:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:03:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:03:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:03:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:41 smithi170 ceph-mon[102832]: pgmap v18853: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 925 KiB/s wr, 20 op/s 2023-09-10T09:03:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:41 smithi082 ceph-mon[98694]: pgmap v18853: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 925 KiB/s wr, 20 op/s 2023-09-10T09:03:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:41 smithi098 ceph-mon[103124]: pgmap v18853: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 925 KiB/s wr, 20 op/s 2023-09-10T09:03:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:43 smithi170 ceph-mon[102832]: pgmap v18854: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 797 KiB/s wr, 14 op/s 2023-09-10T09:03:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:03:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:03:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:43 smithi082 ceph-mon[98694]: pgmap v18854: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 797 KiB/s wr, 14 op/s 2023-09-10T09:03:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:03:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:03:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:43 smithi098 ceph-mon[103124]: pgmap v18854: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 797 KiB/s wr, 14 op/s 2023-09-10T09:03:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:03:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:03:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35055.848630 secs 2023-09-10T09:03:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:03:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35056.201536 secs 2023-09-10T09:03:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35055.848630 secs 2023-09-10T09:03:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:03:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35056.201536 secs 2023-09-10T09:03:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35055.848630 secs 2023-09-10T09:03:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:03:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35056.201536 secs 2023-09-10T09:03:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:45 smithi170 ceph-mon[102832]: pgmap v18855: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 797 KiB/s wr, 14 op/s 2023-09-10T09:03:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:03:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:45 smithi082 ceph-mon[98694]: pgmap v18855: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 797 KiB/s wr, 14 op/s 2023-09-10T09:03:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:03:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:45 smithi098 ceph-mon[103124]: pgmap v18855: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 797 KiB/s wr, 14 op/s 2023-09-10T09:03:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:03:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:47 smithi170 ceph-mon[102832]: pgmap v18856: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 822 KiB/s wr, 20 op/s 2023-09-10T09:03:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:47 smithi082 ceph-mon[98694]: pgmap v18856: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 822 KiB/s wr, 20 op/s 2023-09-10T09:03:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:47 smithi098 ceph-mon[103124]: pgmap v18856: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 822 KiB/s wr, 20 op/s 2023-09-10T09:03:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:48 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:03:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:48 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:03:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:48 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:03:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35060.848689 secs 2023-09-10T09:03:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:49 smithi170 ceph-mon[102832]: pgmap v18857: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 698 KiB/s wr, 14 op/s 2023-09-10T09:03:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35061.201604 secs 2023-09-10T09:03:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:03:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35060.848689 secs 2023-09-10T09:03:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:49 smithi082 ceph-mon[98694]: pgmap v18857: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 698 KiB/s wr, 14 op/s 2023-09-10T09:03:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35061.201604 secs 2023-09-10T09:03:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:03:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35060.848689 secs 2023-09-10T09:03:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:49 smithi098 ceph-mon[103124]: pgmap v18857: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 698 KiB/s wr, 14 op/s 2023-09-10T09:03:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35061.201604 secs 2023-09-10T09:03:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:03:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:51 smithi170 ceph-mon[102832]: pgmap v18858: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:03:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:51 smithi082 ceph-mon[98694]: pgmap v18858: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:03:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:51 smithi098 ceph-mon[103124]: pgmap v18858: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:03:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:03:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:03:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:03:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:53 smithi082 ceph-mon[98694]: pgmap v18859: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 473 KiB/s wr, 12 op/s 2023-09-10T09:03:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:03:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:53 smithi098 ceph-mon[103124]: pgmap v18859: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 473 KiB/s wr, 12 op/s 2023-09-10T09:03:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:03:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:53 smithi170 ceph-mon[102832]: pgmap v18859: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 473 KiB/s wr, 12 op/s 2023-09-10T09:03:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:03:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35065.848748 secs 2023-09-10T09:03:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35066.201669 secs 2023-09-10T09:03:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35065.848748 secs 2023-09-10T09:03:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35066.201669 secs 2023-09-10T09:03:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35065.848748 secs 2023-09-10T09:03:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35066.201669 secs 2023-09-10T09:03:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:55 smithi082 ceph-mon[98694]: pgmap v18860: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 473 KiB/s wr, 12 op/s 2023-09-10T09:03:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:55 smithi098 ceph-mon[103124]: pgmap v18860: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 473 KiB/s wr, 12 op/s 2023-09-10T09:03:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:55 smithi170 ceph-mon[102832]: pgmap v18860: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 473 KiB/s wr, 12 op/s 2023-09-10T09:03:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:03:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:03:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:03:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:03:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:57 smithi082 ceph-mon[98694]: pgmap v18861: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 918 KiB/s wr, 20 op/s 2023-09-10T09:03:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:57 smithi098 ceph-mon[103124]: pgmap v18861: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 918 KiB/s wr, 20 op/s 2023-09-10T09:03:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:57 smithi170 ceph-mon[102832]: pgmap v18861: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 918 KiB/s wr, 20 op/s 2023-09-10T09:03:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35069.598551 secs 2023-09-10T09:03:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35069.598551 secs 2023-09-10T09:03:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35069.598551 secs 2023-09-10T09:04:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35070.848808 secs 2023-09-10T09:04:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:59 smithi082 ceph-mon[98694]: pgmap v18862: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 893 KiB/s wr, 14 op/s 2023-09-10T09:04:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:03:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35071.201732 secs 2023-09-10T09:04:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35070.848808 secs 2023-09-10T09:04:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:59 smithi098 ceph-mon[103124]: pgmap v18862: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 893 KiB/s wr, 14 op/s 2023-09-10T09:04:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:03:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35071.201732 secs 2023-09-10T09:04:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35070.848808 secs 2023-09-10T09:04:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:59 smithi170 ceph-mon[102832]: pgmap v18862: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 893 KiB/s wr, 14 op/s 2023-09-10T09:04:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:03:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35071.201732 secs 2023-09-10T09:04:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:04:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:04:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:04:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:04:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:04:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:04:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:01 smithi098 ceph-mon[103124]: pgmap v18863: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 20 op/s 2023-09-10T09:04:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:01 smithi170 ceph-mon[102832]: pgmap v18863: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 20 op/s 2023-09-10T09:04:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:01 smithi082 ceph-mon[98694]: pgmap v18863: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 20 op/s 2023-09-10T09:04:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:03 smithi098 ceph-mon[103124]: pgmap v18864: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 667 KiB/s wr, 14 op/s 2023-09-10T09:04:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35074.598662 secs 2023-09-10T09:04:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:04:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:03 smithi170 ceph-mon[102832]: pgmap v18864: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 667 KiB/s wr, 14 op/s 2023-09-10T09:04:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35074.598662 secs 2023-09-10T09:04:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:04:04.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:03 smithi082 ceph-mon[98694]: pgmap v18864: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 667 KiB/s wr, 14 op/s 2023-09-10T09:04:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35074.598662 secs 2023-09-10T09:04:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:04:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35075.848866 secs 2023-09-10T09:04:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35076.201798 secs 2023-09-10T09:04:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:04:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:04:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:04:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:04:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35075.848866 secs 2023-09-10T09:04:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35076.201798 secs 2023-09-10T09:04:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:04:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:04:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:04:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:04:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35075.848866 secs 2023-09-10T09:04:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35076.201798 secs 2023-09-10T09:04:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:04:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:04:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:04:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:04:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:05 smithi098 ceph-mon[103124]: pgmap v18865: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 667 KiB/s wr, 14 op/s 2023-09-10T09:04:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:04:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:05 smithi170 ceph-mon[102832]: pgmap v18865: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 667 KiB/s wr, 14 op/s 2023-09-10T09:04:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:04:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:05 smithi082 ceph-mon[98694]: pgmap v18865: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 667 KiB/s wr, 14 op/s 2023-09-10T09:04:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:04:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:07 smithi098 ceph-mon[103124]: pgmap v18866: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 986 KiB/s wr, 34 op/s 2023-09-10T09:04:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:07 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35079.598724 secs 2023-09-10T09:04:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:07 smithi170 ceph-mon[102832]: pgmap v18866: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 986 KiB/s wr, 34 op/s 2023-09-10T09:04:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:07 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35079.598724 secs 2023-09-10T09:04:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:07 smithi082 ceph-mon[98694]: pgmap v18866: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 986 KiB/s wr, 34 op/s 2023-09-10T09:04:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:07 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35079.598724 secs 2023-09-10T09:04:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:08 smithi098 ceph-mon[103124]: osdmap e1247: 12 total, 12 up, 12 in 2023-09-10T09:04:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:08 smithi170 ceph-mon[102832]: osdmap e1247: 12 total, 12 up, 12 in 2023-09-10T09:04:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:08 smithi082 ceph-mon[98694]: osdmap e1247: 12 total, 12 up, 12 in 2023-09-10T09:04:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35080.848951 secs 2023-09-10T09:04:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:09 smithi098 ceph-mon[103124]: pgmap v18868: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 648 KiB/s wr, 32 op/s 2023-09-10T09:04:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35081.201867 secs 2023-09-10T09:04:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35080.848951 secs 2023-09-10T09:04:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:09 smithi170 ceph-mon[102832]: pgmap v18868: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 648 KiB/s wr, 32 op/s 2023-09-10T09:04:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35081.201867 secs 2023-09-10T09:04:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35080.848951 secs 2023-09-10T09:04:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:09 smithi082 ceph-mon[98694]: pgmap v18868: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 648 KiB/s wr, 32 op/s 2023-09-10T09:04:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35081.201867 secs 2023-09-10T09:04:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:11 smithi098 ceph-mon[103124]: pgmap v18869: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 535 KiB/s wr, 32 op/s 2023-09-10T09:04:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:11 smithi170 ceph-mon[102832]: pgmap v18869: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 535 KiB/s wr, 32 op/s 2023-09-10T09:04:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:11 smithi082 ceph-mon[98694]: pgmap v18869: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 535 KiB/s wr, 32 op/s 2023-09-10T09:04:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35084.598775 secs 2023-09-10T09:04:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35084.598775 secs 2023-09-10T09:04:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35084.598775 secs 2023-09-10T09:04:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:13 smithi098 ceph-mon[103124]: pgmap v18870: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 535 KiB/s wr, 32 op/s 2023-09-10T09:04:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:04:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:13 smithi098 ceph-mon[103124]: osdmap e1248: 12 total, 12 up, 12 in 2023-09-10T09:04:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:13 smithi170 ceph-mon[102832]: pgmap v18870: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 535 KiB/s wr, 32 op/s 2023-09-10T09:04:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:04:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:13 smithi170 ceph-mon[102832]: osdmap e1248: 12 total, 12 up, 12 in 2023-09-10T09:04:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:13 smithi082 ceph-mon[98694]: pgmap v18870: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 535 KiB/s wr, 32 op/s 2023-09-10T09:04:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:04:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:13 smithi082 ceph-mon[98694]: osdmap e1248: 12 total, 12 up, 12 in 2023-09-10T09:04:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35085.849041 secs 2023-09-10T09:04:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35086.201935 secs 2023-09-10T09:04:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35085.849041 secs 2023-09-10T09:04:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35086.201935 secs 2023-09-10T09:04:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35085.849041 secs 2023-09-10T09:04:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35086.201935 secs 2023-09-10T09:04:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:16 smithi170 ceph-mon[102832]: pgmap v18872: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 190 KiB/s wr, 9 op/s 2023-09-10T09:04:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:16 smithi082 ceph-mon[98694]: pgmap v18872: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 190 KiB/s wr, 9 op/s 2023-09-10T09:04:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:16 smithi098 ceph-mon[103124]: pgmap v18872: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 190 KiB/s wr, 9 op/s 2023-09-10T09:04:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:17 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:04:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:17 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:17 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:04:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:17 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:17 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:04:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:17 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:18 smithi170 ceph-mon[102832]: pgmap v18873: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 210 KiB/s wr, 18 op/s 2023-09-10T09:04:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35089.598883 secs 2023-09-10T09:04:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:18 smithi082 ceph-mon[98694]: pgmap v18873: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 210 KiB/s wr, 18 op/s 2023-09-10T09:04:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35089.598883 secs 2023-09-10T09:04:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:18 smithi098 ceph-mon[103124]: pgmap v18873: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 210 KiB/s wr, 18 op/s 2023-09-10T09:04:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35089.598883 secs 2023-09-10T09:04:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35090.849100 secs 2023-09-10T09:04:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:19 smithi170 ceph-mon[102832]: pgmap v18874: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 184 KiB/s wr, 15 op/s 2023-09-10T09:04:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35091.202001 secs 2023-09-10T09:04:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35090.849100 secs 2023-09-10T09:04:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:19 smithi082 ceph-mon[98694]: pgmap v18874: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 184 KiB/s wr, 15 op/s 2023-09-10T09:04:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35091.202001 secs 2023-09-10T09:04:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35090.849100 secs 2023-09-10T09:04:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:19 smithi098 ceph-mon[103124]: pgmap v18874: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 184 KiB/s wr, 15 op/s 2023-09-10T09:04:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35091.202001 secs 2023-09-10T09:04:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:21 smithi170 ceph-mon[102832]: pgmap v18875: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-09-10T09:04:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:21 smithi082 ceph-mon[98694]: pgmap v18875: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-09-10T09:04:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:21 smithi098 ceph-mon[103124]: pgmap v18875: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-09-10T09:04:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:23 smithi170 ceph-mon[102832]: pgmap v18876: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-09-10T09:04:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35094.598947 secs 2023-09-10T09:04:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:23 smithi082 ceph-mon[98694]: pgmap v18876: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-09-10T09:04:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35094.598947 secs 2023-09-10T09:04:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:23 smithi098 ceph-mon[103124]: pgmap v18876: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-09-10T09:04:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35094.598947 secs 2023-09-10T09:04:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35095.849159 secs 2023-09-10T09:04:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35096.202069 secs 2023-09-10T09:04:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35095.849159 secs 2023-09-10T09:04:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35096.202069 secs 2023-09-10T09:04:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35095.849159 secs 2023-09-10T09:04:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35096.202069 secs 2023-09-10T09:04:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:25 smithi170 ceph-mon[102832]: pgmap v18877: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 12 op/s 2023-09-10T09:04:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:25 smithi082 ceph-mon[98694]: pgmap v18877: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 12 op/s 2023-09-10T09:04:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:25 smithi098 ceph-mon[103124]: pgmap v18877: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 12 op/s 2023-09-10T09:04:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:27 smithi170 ceph-mon[102832]: pgmap v18878: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 17 op/s 2023-09-10T09:04:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:27 smithi082 ceph-mon[98694]: pgmap v18878: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 17 op/s 2023-09-10T09:04:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:27 smithi098 ceph-mon[103124]: pgmap v18878: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 17 op/s 2023-09-10T09:04:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35099.599007 secs 2023-09-10T09:04:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35099.599007 secs 2023-09-10T09:04:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35099.599007 secs 2023-09-10T09:04:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35100.849219 secs 2023-09-10T09:04:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:29 smithi170 ceph-mon[102832]: pgmap v18879: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:04:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35101.202134 secs 2023-09-10T09:04:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35100.849219 secs 2023-09-10T09:04:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:29 smithi082 ceph-mon[98694]: pgmap v18879: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:04:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35101.202134 secs 2023-09-10T09:04:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35100.849219 secs 2023-09-10T09:04:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:29 smithi098 ceph-mon[103124]: pgmap v18879: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:04:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35101.202134 secs 2023-09-10T09:04:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:31 smithi170 ceph-mon[102832]: pgmap v18880: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 128 KiB/s wr, 16 op/s 2023-09-10T09:04:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:31 smithi082 ceph-mon[98694]: pgmap v18880: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 128 KiB/s wr, 16 op/s 2023-09-10T09:04:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:31 smithi098 ceph-mon[103124]: pgmap v18880: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 128 KiB/s wr, 16 op/s 2023-09-10T09:04:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:33 smithi170 ceph-mon[102832]: pgmap v18881: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:04:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:33 smithi082 ceph-mon[98694]: pgmap v18881: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:04:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:33 smithi098 ceph-mon[103124]: pgmap v18881: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:04:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35105.849278 secs 2023-09-10T09:04:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:04:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35106.202198 secs 2023-09-10T09:04:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35105.849278 secs 2023-09-10T09:04:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:04:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35106.202198 secs 2023-09-10T09:04:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35105.849278 secs 2023-09-10T09:04:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:04:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35106.202198 secs 2023-09-10T09:04:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:35 smithi170 ceph-mon[102832]: pgmap v18882: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:04:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:04:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:35 smithi082 ceph-mon[98694]: pgmap v18882: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:04:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:04:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:35 smithi098 ceph-mon[103124]: pgmap v18882: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:04:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:04:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:04:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:04:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:04:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:04:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:04:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:04:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:37 smithi170 ceph-mon[102832]: pgmap v18883: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 17 op/s 2023-09-10T09:04:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:37 smithi082 ceph-mon[98694]: pgmap v18883: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 17 op/s 2023-09-10T09:04:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:37 smithi098 ceph-mon[103124]: pgmap v18883: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 17 op/s 2023-09-10T09:04:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35109.599131 secs 2023-09-10T09:04:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35109.599131 secs 2023-09-10T09:04:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35109.599131 secs 2023-09-10T09:04:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35110.849337 secs 2023-09-10T09:04:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:39 smithi170 ceph-mon[102832]: pgmap v18884: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T09:04:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35111.202266 secs 2023-09-10T09:04:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35110.849337 secs 2023-09-10T09:04:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:39 smithi082 ceph-mon[98694]: pgmap v18884: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T09:04:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35111.202266 secs 2023-09-10T09:04:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35110.849337 secs 2023-09-10T09:04:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:39 smithi098 ceph-mon[103124]: pgmap v18884: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T09:04:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35111.202266 secs 2023-09-10T09:04:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:04:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:04:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:04:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:41 smithi170 ceph-mon[102832]: pgmap v18885: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 437 KiB/s wr, 18 op/s 2023-09-10T09:04:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:41 smithi082 ceph-mon[98694]: pgmap v18885: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 437 KiB/s wr, 18 op/s 2023-09-10T09:04:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:41 smithi098 ceph-mon[103124]: pgmap v18885: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 437 KiB/s wr, 18 op/s 2023-09-10T09:04:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:43 smithi170 ceph-mon[102832]: pgmap v18886: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 437 KiB/s wr, 13 op/s 2023-09-10T09:04:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35114.599191 secs 2023-09-10T09:04:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:04:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:43 smithi082 ceph-mon[98694]: pgmap v18886: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 437 KiB/s wr, 13 op/s 2023-09-10T09:04:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35114.599191 secs 2023-09-10T09:04:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:04:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:43 smithi098 ceph-mon[103124]: pgmap v18886: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 437 KiB/s wr, 13 op/s 2023-09-10T09:04:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35114.599191 secs 2023-09-10T09:04:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:04:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35115.849426 secs 2023-09-10T09:04:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35116.202354 secs 2023-09-10T09:04:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35115.849426 secs 2023-09-10T09:04:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35116.202354 secs 2023-09-10T09:04:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35115.849426 secs 2023-09-10T09:04:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35116.202354 secs 2023-09-10T09:04:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:45 smithi170 ceph-mon[102832]: pgmap v18887: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 437 KiB/s wr, 13 op/s 2023-09-10T09:04:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:45 smithi082 ceph-mon[98694]: pgmap v18887: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 437 KiB/s wr, 13 op/s 2023-09-10T09:04:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:45 smithi098 ceph-mon[103124]: pgmap v18887: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 437 KiB/s wr, 13 op/s 2023-09-10T09:04:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:47 smithi170 ceph-mon[102832]: pgmap v18888: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 621 KiB/s wr, 20 op/s 2023-09-10T09:04:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:47 smithi082 ceph-mon[98694]: pgmap v18888: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 621 KiB/s wr, 20 op/s 2023-09-10T09:04:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:47 smithi098 ceph-mon[103124]: pgmap v18888: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 621 KiB/s wr, 20 op/s 2023-09-10T09:04:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35120.849496 secs 2023-09-10T09:04:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:49 smithi170 ceph-mon[102832]: pgmap v18889: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 466 KiB/s wr, 13 op/s 2023-09-10T09:04:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35121.202419 secs 2023-09-10T09:04:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35120.849496 secs 2023-09-10T09:04:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:49 smithi082 ceph-mon[98694]: pgmap v18889: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 466 KiB/s wr, 13 op/s 2023-09-10T09:04:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35121.202419 secs 2023-09-10T09:04:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35120.849496 secs 2023-09-10T09:04:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:49 smithi098 ceph-mon[103124]: pgmap v18889: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 466 KiB/s wr, 13 op/s 2023-09-10T09:04:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35121.202419 secs 2023-09-10T09:04:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:04:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:04:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:04:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:51 smithi170 ceph-mon[102832]: pgmap v18890: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 820 KiB/s wr, 21 op/s 2023-09-10T09:04:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:51 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:04:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:51 smithi082 ceph-mon[98694]: pgmap v18890: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 820 KiB/s wr, 21 op/s 2023-09-10T09:04:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:51 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:04:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:51 smithi098 ceph-mon[103124]: pgmap v18890: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 820 KiB/s wr, 21 op/s 2023-09-10T09:04:52.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:51 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:04:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:52 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:04:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:04:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:52 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:04:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:04:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:52 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:04:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:04:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:53 smithi170 ceph-mon[102832]: pgmap v18891: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 538 KiB/s wr, 14 op/s 2023-09-10T09:04:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:53 smithi082 ceph-mon[98694]: pgmap v18891: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 538 KiB/s wr, 14 op/s 2023-09-10T09:04:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:53 smithi098 ceph-mon[103124]: pgmap v18891: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 538 KiB/s wr, 14 op/s 2023-09-10T09:04:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:04:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35125.849557 secs 2023-09-10T09:04:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35126.202487 secs 2023-09-10T09:04:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:04:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35125.849557 secs 2023-09-10T09:04:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35126.202487 secs 2023-09-10T09:04:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:04:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35125.849557 secs 2023-09-10T09:04:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35126.202487 secs 2023-09-10T09:04:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:55 smithi170 ceph-mon[102832]: pgmap v18892: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 538 KiB/s wr, 14 op/s 2023-09-10T09:04:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:55 smithi082 ceph-mon[98694]: pgmap v18892: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 538 KiB/s wr, 14 op/s 2023-09-10T09:04:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:55 smithi098 ceph-mon[103124]: pgmap v18892: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 538 KiB/s wr, 14 op/s 2023-09-10T09:04:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:04:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:04:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:04:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:57 smithi170 ceph-mon[102832]: pgmap v18893: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 599 KiB/s wr, 21 op/s 2023-09-10T09:04:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:57 smithi082 ceph-mon[98694]: pgmap v18893: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 599 KiB/s wr, 21 op/s 2023-09-10T09:04:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:57 smithi098 ceph-mon[103124]: pgmap v18893: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 599 KiB/s wr, 21 op/s 2023-09-10T09:04:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:04:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:04:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:04:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:04:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35130.849617 secs 2023-09-10T09:04:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:59 smithi170 ceph-mon[102832]: pgmap v18894: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 416 KiB/s wr, 14 op/s 2023-09-10T09:04:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:04:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35131.202553 secs 2023-09-10T09:04:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35130.849617 secs 2023-09-10T09:04:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:59 smithi082 ceph-mon[98694]: pgmap v18894: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 416 KiB/s wr, 14 op/s 2023-09-10T09:04:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:04:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35131.202553 secs 2023-09-10T09:05:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35130.849617 secs 2023-09-10T09:05:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:59 smithi098 ceph-mon[103124]: pgmap v18894: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 416 KiB/s wr, 14 op/s 2023-09-10T09:05:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:04:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35131.202553 secs 2023-09-10T09:05:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:05:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:05:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:05:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:01 smithi170 ceph-mon[102832]: pgmap v18895: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 882 KiB/s wr, 21 op/s 2023-09-10T09:05:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:01 smithi082 ceph-mon[98694]: pgmap v18895: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 882 KiB/s wr, 21 op/s 2023-09-10T09:05:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:01 smithi098 ceph-mon[103124]: pgmap v18895: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 882 KiB/s wr, 21 op/s 2023-09-10T09:05:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:03 smithi170 ceph-mon[102832]: pgmap v18896: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 528 KiB/s wr, 13 op/s 2023-09-10T09:05:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35134.599490 secs 2023-09-10T09:05:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:03 smithi082 ceph-mon[98694]: pgmap v18896: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 528 KiB/s wr, 13 op/s 2023-09-10T09:05:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35134.599490 secs 2023-09-10T09:05:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:03 smithi098 ceph-mon[103124]: pgmap v18896: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 528 KiB/s wr, 13 op/s 2023-09-10T09:05:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35134.599490 secs 2023-09-10T09:05:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35135.849686 secs 2023-09-10T09:05:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:05:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:05:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35136.202617 secs 2023-09-10T09:05:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:05:05.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35135.849686 secs 2023-09-10T09:05:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:05:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:05:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35136.202617 secs 2023-09-10T09:05:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:05:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35135.849686 secs 2023-09-10T09:05:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:05:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:05:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35136.202617 secs 2023-09-10T09:05:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:05:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:05 smithi170 ceph-mon[102832]: pgmap v18897: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 528 KiB/s wr, 13 op/s 2023-09-10T09:05:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:05:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:05:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:05:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:05 smithi082 ceph-mon[98694]: pgmap v18897: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 528 KiB/s wr, 13 op/s 2023-09-10T09:05:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:05:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:05:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:05:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:05 smithi098 ceph-mon[103124]: pgmap v18897: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 528 KiB/s wr, 13 op/s 2023-09-10T09:05:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:05:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:05:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:05:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:05:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:05:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:05:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:07 smithi170 ceph-mon[102832]: pgmap v18898: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 696 KiB/s wr, 34 op/s 2023-09-10T09:05:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:07 smithi082 ceph-mon[98694]: pgmap v18898: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 696 KiB/s wr, 34 op/s 2023-09-10T09:05:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:07 smithi098 ceph-mon[103124]: pgmap v18898: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 696 KiB/s wr, 34 op/s 2023-09-10T09:05:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35139.599585 secs 2023-09-10T09:05:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35139.599585 secs 2023-09-10T09:05:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35139.599585 secs 2023-09-10T09:05:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35140.849776 secs 2023-09-10T09:05:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:09 smithi170 ceph-mon[102832]: pgmap v18899: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 634 KiB/s wr, 28 op/s 2023-09-10T09:05:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:09 smithi170 ceph-mon[102832]: osdmap e1249: 12 total, 12 up, 12 in 2023-09-10T09:05:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35141.202683 secs 2023-09-10T09:05:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35140.849776 secs 2023-09-10T09:05:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:09 smithi082 ceph-mon[98694]: pgmap v18899: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 634 KiB/s wr, 28 op/s 2023-09-10T09:05:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:09 smithi082 ceph-mon[98694]: osdmap e1249: 12 total, 12 up, 12 in 2023-09-10T09:05:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35141.202683 secs 2023-09-10T09:05:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35140.849776 secs 2023-09-10T09:05:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:09 smithi098 ceph-mon[103124]: pgmap v18899: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 634 KiB/s wr, 28 op/s 2023-09-10T09:05:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:09 smithi098 ceph-mon[103124]: osdmap e1249: 12 total, 12 up, 12 in 2023-09-10T09:05:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35141.202683 secs 2023-09-10T09:05:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:11 smithi170 ceph-mon[102832]: pgmap v18901: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 560 KiB/s wr, 32 op/s 2023-09-10T09:05:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:11 smithi082 ceph-mon[98694]: pgmap v18901: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 560 KiB/s wr, 32 op/s 2023-09-10T09:05:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:11 smithi098 ceph-mon[103124]: pgmap v18901: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 560 KiB/s wr, 32 op/s 2023-09-10T09:05:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:13 smithi170 ceph-mon[102832]: pgmap v18902: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 560 KiB/s wr, 32 op/s 2023-09-10T09:05:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35144.599646 secs 2023-09-10T09:05:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:13 smithi170 ceph-mon[102832]: osdmap e1250: 12 total, 12 up, 12 in 2023-09-10T09:05:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:13 smithi082 ceph-mon[98694]: pgmap v18902: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 560 KiB/s wr, 32 op/s 2023-09-10T09:05:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35144.599646 secs 2023-09-10T09:05:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:13 smithi082 ceph-mon[98694]: osdmap e1250: 12 total, 12 up, 12 in 2023-09-10T09:05:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:13 smithi098 ceph-mon[103124]: pgmap v18902: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 560 KiB/s wr, 32 op/s 2023-09-10T09:05:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35144.599646 secs 2023-09-10T09:05:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:13 smithi098 ceph-mon[103124]: osdmap e1250: 12 total, 12 up, 12 in 2023-09-10T09:05:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35145.849836 secs 2023-09-10T09:05:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35146.202750 secs 2023-09-10T09:05:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35145.849836 secs 2023-09-10T09:05:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35146.202750 secs 2023-09-10T09:05:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35145.849836 secs 2023-09-10T09:05:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35146.202750 secs 2023-09-10T09:05:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:15 smithi170 ceph-mon[102832]: pgmap v18904: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 448 KiB/s wr, 9 op/s 2023-09-10T09:05:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:15 smithi082 ceph-mon[98694]: pgmap v18904: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 448 KiB/s wr, 9 op/s 2023-09-10T09:05:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:15 smithi098 ceph-mon[103124]: pgmap v18904: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 448 KiB/s wr, 9 op/s 2023-09-10T09:05:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:17 smithi170 ceph-mon[102832]: pgmap v18905: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 449 KiB/s wr, 16 op/s 2023-09-10T09:05:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:17 smithi082 ceph-mon[98694]: pgmap v18905: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 449 KiB/s wr, 16 op/s 2023-09-10T09:05:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:17 smithi098 ceph-mon[103124]: pgmap v18905: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 449 KiB/s wr, 16 op/s 2023-09-10T09:05:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35149.599705 secs 2023-09-10T09:05:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35149.599705 secs 2023-09-10T09:05:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35149.599705 secs 2023-09-10T09:05:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35150.849910 secs 2023-09-10T09:05:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:19 smithi170 ceph-mon[102832]: pgmap v18906: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 362 KiB/s wr, 13 op/s 2023-09-10T09:05:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35151.202816 secs 2023-09-10T09:05:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35150.849910 secs 2023-09-10T09:05:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:19 smithi082 ceph-mon[98694]: pgmap v18906: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 362 KiB/s wr, 13 op/s 2023-09-10T09:05:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35151.202816 secs 2023-09-10T09:05:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35150.849910 secs 2023-09-10T09:05:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:19 smithi098 ceph-mon[103124]: pgmap v18906: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 362 KiB/s wr, 13 op/s 2023-09-10T09:05:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35151.202816 secs 2023-09-10T09:05:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:21 smithi170 ceph-mon[102832]: pgmap v18907: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 13 op/s 2023-09-10T09:05:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:21 smithi082 ceph-mon[98694]: pgmap v18907: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 13 op/s 2023-09-10T09:05:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:21 smithi098 ceph-mon[103124]: pgmap v18907: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 13 op/s 2023-09-10T09:05:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:23 smithi170 ceph-mon[102832]: pgmap v18908: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 13 op/s 2023-09-10T09:05:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35154.599803 secs 2023-09-10T09:05:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:23 smithi082 ceph-mon[98694]: pgmap v18908: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 13 op/s 2023-09-10T09:05:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35154.599803 secs 2023-09-10T09:05:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:23 smithi098 ceph-mon[103124]: pgmap v18908: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 178 KiB/s wr, 13 op/s 2023-09-10T09:05:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35154.599803 secs 2023-09-10T09:05:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35155.849968 secs 2023-09-10T09:05:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35156.202883 secs 2023-09-10T09:05:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35155.849968 secs 2023-09-10T09:05:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35156.202883 secs 2023-09-10T09:05:25.260 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35155.849968 secs 2023-09-10T09:05:25.261 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35156.202883 secs 2023-09-10T09:05:26.173 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:25 smithi098 ceph-mon[103124]: pgmap v18909: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T09:05:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:25 smithi170 ceph-mon[102832]: pgmap v18909: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T09:05:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:25 smithi082 ceph-mon[98694]: pgmap v18909: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T09:05:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:27 smithi170 ceph-mon[102832]: pgmap v18910: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 149 KiB/s wr, 16 op/s 2023-09-10T09:05:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:27 smithi082 ceph-mon[98694]: pgmap v18910: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 149 KiB/s wr, 16 op/s 2023-09-10T09:05:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:27 smithi098 ceph-mon[103124]: pgmap v18910: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 149 KiB/s wr, 16 op/s 2023-09-10T09:05:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35159.599864 secs 2023-09-10T09:05:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35159.599864 secs 2023-09-10T09:05:29.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35159.599864 secs 2023-09-10T09:05:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35160.850026 secs 2023-09-10T09:05:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:29 smithi170 ceph-mon[102832]: pgmap v18911: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T09:05:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35161.202949 secs 2023-09-10T09:05:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:29 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:05:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:29 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35160.850026 secs 2023-09-10T09:05:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:29 smithi082 ceph-mon[98694]: pgmap v18911: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T09:05:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35161.202949 secs 2023-09-10T09:05:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:29 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:05:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:29 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35160.850026 secs 2023-09-10T09:05:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:29 smithi098 ceph-mon[103124]: pgmap v18911: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T09:05:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35161.202949 secs 2023-09-10T09:05:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:29 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:05:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:29 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:31 smithi170 ceph-mon[102832]: pgmap v18912: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 275 KiB/s wr, 17 op/s 2023-09-10T09:05:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:31 smithi082 ceph-mon[98694]: pgmap v18912: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 275 KiB/s wr, 17 op/s 2023-09-10T09:05:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:31 smithi098 ceph-mon[103124]: pgmap v18912: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 275 KiB/s wr, 17 op/s 2023-09-10T09:05:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:05:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:05:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:05:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:33 smithi170 ceph-mon[102832]: pgmap v18913: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T09:05:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:33 smithi082 ceph-mon[98694]: pgmap v18913: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T09:05:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:33 smithi098 ceph-mon[103124]: pgmap v18913: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T09:05:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35165.850094 secs 2023-09-10T09:05:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35166.203016 secs 2023-09-10T09:05:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35165.850094 secs 2023-09-10T09:05:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35166.203016 secs 2023-09-10T09:05:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35165.850094 secs 2023-09-10T09:05:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35166.203016 secs 2023-09-10T09:05:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:35 smithi170 ceph-mon[102832]: pgmap v18914: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T09:05:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:05:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:35 smithi082 ceph-mon[98694]: pgmap v18914: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T09:05:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:05:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:35 smithi098 ceph-mon[103124]: pgmap v18914: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T09:05:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:05:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:05:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:05:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:05:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:05:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:05:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:05:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:37 smithi170 ceph-mon[102832]: pgmap v18915: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 250 KiB/s wr, 17 op/s 2023-09-10T09:05:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:05:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:37 smithi082 ceph-mon[98694]: pgmap v18915: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 250 KiB/s wr, 17 op/s 2023-09-10T09:05:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:05:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:37 smithi098 ceph-mon[103124]: pgmap v18915: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 250 KiB/s wr, 17 op/s 2023-09-10T09:05:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:05:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:39.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35169.599982 secs 2023-09-10T09:05:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35169.599982 secs 2023-09-10T09:05:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35169.599982 secs 2023-09-10T09:05:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35170.850152 secs 2023-09-10T09:05:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:39 smithi170 ceph-mon[102832]: pgmap v18916: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 250 KiB/s wr, 11 op/s 2023-09-10T09:05:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35171.203082 secs 2023-09-10T09:05:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35170.850152 secs 2023-09-10T09:05:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:39 smithi082 ceph-mon[98694]: pgmap v18916: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 250 KiB/s wr, 11 op/s 2023-09-10T09:05:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35171.203082 secs 2023-09-10T09:05:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35170.850152 secs 2023-09-10T09:05:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:39 smithi098 ceph-mon[103124]: pgmap v18916: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 250 KiB/s wr, 11 op/s 2023-09-10T09:05:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35171.203082 secs 2023-09-10T09:05:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:05:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:05:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:05:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:41 smithi082 ceph-mon[98694]: pgmap v18917: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 916 KiB/s wr, 19 op/s 2023-09-10T09:05:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:41 smithi098 ceph-mon[103124]: pgmap v18917: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 916 KiB/s wr, 19 op/s 2023-09-10T09:05:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:41 smithi170 ceph-mon[102832]: pgmap v18917: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 916 KiB/s wr, 19 op/s 2023-09-10T09:05:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:43 smithi082 ceph-mon[98694]: pgmap v18918: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 790 KiB/s wr, 13 op/s 2023-09-10T09:05:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35174.600046 secs 2023-09-10T09:05:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:05:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:43 smithi098 ceph-mon[103124]: pgmap v18918: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 790 KiB/s wr, 13 op/s 2023-09-10T09:05:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35174.600046 secs 2023-09-10T09:05:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:05:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:43 smithi170 ceph-mon[102832]: pgmap v18918: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 790 KiB/s wr, 13 op/s 2023-09-10T09:05:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35174.600046 secs 2023-09-10T09:05:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:05:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35175.850224 secs 2023-09-10T09:05:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35176.203149 secs 2023-09-10T09:05:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35175.850224 secs 2023-09-10T09:05:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35176.203149 secs 2023-09-10T09:05:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35175.850224 secs 2023-09-10T09:05:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35176.203149 secs 2023-09-10T09:05:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:45 smithi082 ceph-mon[98694]: pgmap v18919: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 790 KiB/s wr, 13 op/s 2023-09-10T09:05:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:45 smithi098 ceph-mon[103124]: pgmap v18919: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 790 KiB/s wr, 13 op/s 2023-09-10T09:05:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:45 smithi170 ceph-mon[102832]: pgmap v18919: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 790 KiB/s wr, 13 op/s 2023-09-10T09:05:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:47 smithi082 ceph-mon[98694]: pgmap v18920: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 843 KiB/s wr, 19 op/s 2023-09-10T09:05:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:47 smithi098 ceph-mon[103124]: pgmap v18920: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 843 KiB/s wr, 19 op/s 2023-09-10T09:05:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:47 smithi170 ceph-mon[102832]: pgmap v18920: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 843 KiB/s wr, 19 op/s 2023-09-10T09:05:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35180.850281 secs 2023-09-10T09:05:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:49 smithi170 ceph-mon[102832]: pgmap v18921: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 720 KiB/s wr, 13 op/s 2023-09-10T09:05:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35181.203215 secs 2023-09-10T09:05:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35180.850281 secs 2023-09-10T09:05:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:49 smithi082 ceph-mon[98694]: pgmap v18921: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 720 KiB/s wr, 13 op/s 2023-09-10T09:05:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35181.203215 secs 2023-09-10T09:05:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35180.850281 secs 2023-09-10T09:05:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:49 smithi098 ceph-mon[103124]: pgmap v18921: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 720 KiB/s wr, 13 op/s 2023-09-10T09:05:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35181.203215 secs 2023-09-10T09:05:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:05:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:05:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:05:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:51 smithi170 ceph-mon[102832]: pgmap v18922: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:05:51.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:51 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:05:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:51 smithi082 ceph-mon[98694]: pgmap v18922: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:05:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:51 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:05:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:51 smithi098 ceph-mon[103124]: pgmap v18922: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:05:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:51 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:05:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:52 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:05:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:05:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:52 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:05:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:05:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:52 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:05:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:05:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:53 smithi170 ceph-mon[102832]: pgmap v18923: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 465 KiB/s wr, 13 op/s 2023-09-10T09:05:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:53 smithi082 ceph-mon[98694]: pgmap v18923: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 465 KiB/s wr, 13 op/s 2023-09-10T09:05:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:53 smithi098 ceph-mon[103124]: pgmap v18923: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 465 KiB/s wr, 13 op/s 2023-09-10T09:05:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:05:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35185.850360 secs 2023-09-10T09:05:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35186.203303 secs 2023-09-10T09:05:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:05:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35185.850360 secs 2023-09-10T09:05:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35186.203303 secs 2023-09-10T09:05:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:05:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35185.850360 secs 2023-09-10T09:05:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35186.203303 secs 2023-09-10T09:05:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:55 smithi170 ceph-mon[102832]: pgmap v18924: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 466 KiB/s wr, 13 op/s 2023-09-10T09:05:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:55 smithi082 ceph-mon[98694]: pgmap v18924: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 466 KiB/s wr, 13 op/s 2023-09-10T09:05:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:55 smithi098 ceph-mon[103124]: pgmap v18924: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 466 KiB/s wr, 13 op/s 2023-09-10T09:05:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:05:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:05:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:05:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:57 smithi170 ceph-mon[102832]: pgmap v18925: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 906 KiB/s wr, 21 op/s 2023-09-10T09:05:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:57 smithi082 ceph-mon[98694]: pgmap v18925: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 906 KiB/s wr, 21 op/s 2023-09-10T09:05:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:57 smithi098 ceph-mon[103124]: pgmap v18925: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 906 KiB/s wr, 21 op/s 2023-09-10T09:05:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:05:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35189.600228 secs 2023-09-10T09:05:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:05:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35189.600228 secs 2023-09-10T09:05:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:05:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:05:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35189.600228 secs 2023-09-10T09:05:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35190.850422 secs 2023-09-10T09:05:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:59 smithi170 ceph-mon[102832]: pgmap v18926: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 852 KiB/s wr, 14 op/s 2023-09-10T09:05:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:05:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35191.203366 secs 2023-09-10T09:05:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35190.850422 secs 2023-09-10T09:05:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:59 smithi082 ceph-mon[98694]: pgmap v18926: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 852 KiB/s wr, 14 op/s 2023-09-10T09:05:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:05:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35191.203366 secs 2023-09-10T09:06:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35190.850422 secs 2023-09-10T09:06:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:59 smithi098 ceph-mon[103124]: pgmap v18926: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 852 KiB/s wr, 14 op/s 2023-09-10T09:06:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:05:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35191.203366 secs 2023-09-10T09:06:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:06:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:06:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:06:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:06:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:06:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:06:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:01 smithi170 ceph-mon[102832]: pgmap v18927: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 953 KiB/s wr, 20 op/s 2023-09-10T09:06:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:01 smithi082 ceph-mon[98694]: pgmap v18927: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 953 KiB/s wr, 20 op/s 2023-09-10T09:06:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:01 smithi098 ceph-mon[103124]: pgmap v18927: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 953 KiB/s wr, 20 op/s 2023-09-10T09:06:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:03 smithi170 ceph-mon[102832]: pgmap v18928: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 541 KiB/s wr, 13 op/s 2023-09-10T09:06:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35194.600285 secs 2023-09-10T09:06:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:03 smithi082 ceph-mon[98694]: pgmap v18928: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 541 KiB/s wr, 13 op/s 2023-09-10T09:06:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35194.600285 secs 2023-09-10T09:06:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:03 smithi098 ceph-mon[103124]: pgmap v18928: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 541 KiB/s wr, 13 op/s 2023-09-10T09:06:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35194.600285 secs 2023-09-10T09:06:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35195.850502 secs 2023-09-10T09:06:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:06:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35196.203430 secs 2023-09-10T09:06:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35195.850502 secs 2023-09-10T09:06:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:06:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35196.203430 secs 2023-09-10T09:06:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35195.850502 secs 2023-09-10T09:06:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:06:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35196.203430 secs 2023-09-10T09:06:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:05 smithi170 ceph-mon[102832]: pgmap v18929: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 541 KiB/s wr, 13 op/s 2023-09-10T09:06:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:06:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:06:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:05 smithi082 ceph-mon[98694]: pgmap v18929: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 541 KiB/s wr, 13 op/s 2023-09-10T09:06:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:06:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:06:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:05 smithi098 ceph-mon[103124]: pgmap v18929: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 541 KiB/s wr, 13 op/s 2023-09-10T09:06:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:06:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:06:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:06:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:06:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:06:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:06:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:06:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:06:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:06:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:06:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:06:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:07 smithi098 ceph-mon[103124]: pgmap v18930: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 862 KiB/s wr, 34 op/s 2023-09-10T09:06:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:07 smithi170 ceph-mon[102832]: pgmap v18930: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 862 KiB/s wr, 34 op/s 2023-09-10T09:06:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:07 smithi082 ceph-mon[98694]: pgmap v18930: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 862 KiB/s wr, 34 op/s 2023-09-10T09:06:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35199.600346 secs 2023-09-10T09:06:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:08 smithi098 ceph-mon[103124]: osdmap e1251: 12 total, 12 up, 12 in 2023-09-10T09:06:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35199.600346 secs 2023-09-10T09:06:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:08 smithi170 ceph-mon[102832]: osdmap e1251: 12 total, 12 up, 12 in 2023-09-10T09:06:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35199.600346 secs 2023-09-10T09:06:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:08 smithi082 ceph-mon[98694]: osdmap e1251: 12 total, 12 up, 12 in 2023-09-10T09:06:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35200.850564 secs 2023-09-10T09:06:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:09 smithi098 ceph-mon[103124]: pgmap v18932: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 506 KiB/s wr, 32 op/s 2023-09-10T09:06:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35201.203487 secs 2023-09-10T09:06:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35200.850564 secs 2023-09-10T09:06:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:09 smithi170 ceph-mon[102832]: pgmap v18932: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 506 KiB/s wr, 32 op/s 2023-09-10T09:06:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35201.203487 secs 2023-09-10T09:06:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35200.850564 secs 2023-09-10T09:06:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:09 smithi082 ceph-mon[98694]: pgmap v18932: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 506 KiB/s wr, 32 op/s 2023-09-10T09:06:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35201.203487 secs 2023-09-10T09:06:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:11 smithi098 ceph-mon[103124]: pgmap v18933: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 533 KiB/s wr, 33 op/s 2023-09-10T09:06:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:11 smithi170 ceph-mon[102832]: pgmap v18933: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 533 KiB/s wr, 33 op/s 2023-09-10T09:06:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:11 smithi082 ceph-mon[98694]: pgmap v18933: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 533 KiB/s wr, 33 op/s 2023-09-10T09:06:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:12 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35204.600409 secs 2023-09-10T09:06:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:12 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35204.600409 secs 2023-09-10T09:06:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:12 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35204.600409 secs 2023-09-10T09:06:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:13 smithi098 ceph-mon[103124]: pgmap v18934: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 533 KiB/s wr, 33 op/s 2023-09-10T09:06:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:13 smithi098 ceph-mon[103124]: osdmap e1252: 12 total, 12 up, 12 in 2023-09-10T09:06:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:13 smithi170 ceph-mon[102832]: pgmap v18934: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 533 KiB/s wr, 33 op/s 2023-09-10T09:06:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:13 smithi170 ceph-mon[102832]: osdmap e1252: 12 total, 12 up, 12 in 2023-09-10T09:06:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:13 smithi082 ceph-mon[98694]: pgmap v18934: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 533 KiB/s wr, 33 op/s 2023-09-10T09:06:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:13 smithi082 ceph-mon[98694]: osdmap e1252: 12 total, 12 up, 12 in 2023-09-10T09:06:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35205.850623 secs 2023-09-10T09:06:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35206.203556 secs 2023-09-10T09:06:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35205.850623 secs 2023-09-10T09:06:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35206.203556 secs 2023-09-10T09:06:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35205.850623 secs 2023-09-10T09:06:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35206.203556 secs 2023-09-10T09:06:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:15 smithi098 ceph-mon[103124]: pgmap v18936: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 185 KiB/s wr, 9 op/s 2023-09-10T09:06:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:15 smithi170 ceph-mon[102832]: pgmap v18936: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 185 KiB/s wr, 9 op/s 2023-09-10T09:06:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:15 smithi082 ceph-mon[98694]: pgmap v18936: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 185 KiB/s wr, 9 op/s 2023-09-10T09:06:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:18 smithi098 ceph-mon[103124]: pgmap v18937: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 204 KiB/s wr, 16 op/s 2023-09-10T09:06:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35209.600468 secs 2023-09-10T09:06:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:18 smithi170 ceph-mon[102832]: pgmap v18937: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 204 KiB/s wr, 16 op/s 2023-09-10T09:06:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35209.600468 secs 2023-09-10T09:06:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:18 smithi082 ceph-mon[98694]: pgmap v18937: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 204 KiB/s wr, 16 op/s 2023-09-10T09:06:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35209.600468 secs 2023-09-10T09:06:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35210.850681 secs 2023-09-10T09:06:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:19 smithi170 ceph-mon[102832]: pgmap v18938: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 180 KiB/s wr, 14 op/s 2023-09-10T09:06:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35211.203619 secs 2023-09-10T09:06:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35210.850681 secs 2023-09-10T09:06:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:19 smithi082 ceph-mon[98694]: pgmap v18938: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 180 KiB/s wr, 14 op/s 2023-09-10T09:06:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35211.203619 secs 2023-09-10T09:06:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35210.850681 secs 2023-09-10T09:06:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:19 smithi098 ceph-mon[103124]: pgmap v18938: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 180 KiB/s wr, 14 op/s 2023-09-10T09:06:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35211.203619 secs 2023-09-10T09:06:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:21 smithi170 ceph-mon[102832]: pgmap v18939: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T09:06:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:21 smithi082 ceph-mon[98694]: pgmap v18939: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T09:06:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:21 smithi098 ceph-mon[103124]: pgmap v18939: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T09:06:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:23 smithi170 ceph-mon[102832]: pgmap v18940: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T09:06:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35214.600531 secs 2023-09-10T09:06:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:23 smithi082 ceph-mon[98694]: pgmap v18940: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T09:06:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35214.600531 secs 2023-09-10T09:06:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:23 smithi098 ceph-mon[103124]: pgmap v18940: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 12 op/s 2023-09-10T09:06:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35214.600531 secs 2023-09-10T09:06:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35215.850743 secs 2023-09-10T09:06:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35216.203685 secs 2023-09-10T09:06:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35215.850743 secs 2023-09-10T09:06:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35216.203685 secs 2023-09-10T09:06:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35215.850743 secs 2023-09-10T09:06:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35216.203685 secs 2023-09-10T09:06:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:25 smithi170 ceph-mon[102832]: pgmap v18941: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T09:06:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:25 smithi082 ceph-mon[98694]: pgmap v18941: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T09:06:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:25 smithi098 ceph-mon[103124]: pgmap v18941: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T09:06:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:27 smithi170 ceph-mon[102832]: pgmap v18942: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 16 op/s 2023-09-10T09:06:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:27 smithi082 ceph-mon[98694]: pgmap v18942: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 16 op/s 2023-09-10T09:06:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:27 smithi098 ceph-mon[103124]: pgmap v18942: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 154 KiB/s wr, 16 op/s 2023-09-10T09:06:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35219.600591 secs 2023-09-10T09:06:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35219.600591 secs 2023-09-10T09:06:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35219.600591 secs 2023-09-10T09:06:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35220.850830 secs 2023-09-10T09:06:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:29 smithi170 ceph-mon[102832]: pgmap v18943: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T09:06:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35221.203753 secs 2023-09-10T09:06:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35220.850830 secs 2023-09-10T09:06:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:29 smithi082 ceph-mon[98694]: pgmap v18943: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T09:06:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35221.203753 secs 2023-09-10T09:06:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35220.850830 secs 2023-09-10T09:06:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:29 smithi098 ceph-mon[103124]: pgmap v18943: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T09:06:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35221.203753 secs 2023-09-10T09:06:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:31 smithi170 ceph-mon[102832]: pgmap v18944: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 127 KiB/s wr, 16 op/s 2023-09-10T09:06:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:31 smithi082 ceph-mon[98694]: pgmap v18944: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 127 KiB/s wr, 16 op/s 2023-09-10T09:06:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:31 smithi098 ceph-mon[103124]: pgmap v18944: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 127 KiB/s wr, 16 op/s 2023-09-10T09:06:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:33 smithi170 ceph-mon[102832]: pgmap v18945: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T09:06:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:06:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:33 smithi082 ceph-mon[98694]: pgmap v18945: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T09:06:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:06:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:33 smithi098 ceph-mon[103124]: pgmap v18945: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T09:06:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:06:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35225.850891 secs 2023-09-10T09:06:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35226.203819 secs 2023-09-10T09:06:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35225.850891 secs 2023-09-10T09:06:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35226.203819 secs 2023-09-10T09:06:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35225.850891 secs 2023-09-10T09:06:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35226.203819 secs 2023-09-10T09:06:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:35 smithi170 ceph-mon[102832]: pgmap v18946: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T09:06:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:06:36.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:35 smithi082 ceph-mon[98694]: pgmap v18946: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T09:06:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:06:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:35 smithi098 ceph-mon[103124]: pgmap v18946: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T09:06:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:06:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:06:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:06:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:06:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:06:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:06:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:06:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:37 smithi170 ceph-mon[102832]: pgmap v18947: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 16 op/s 2023-09-10T09:06:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:37 smithi082 ceph-mon[98694]: pgmap v18947: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 16 op/s 2023-09-10T09:06:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:37 smithi098 ceph-mon[103124]: pgmap v18947: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 16 op/s 2023-09-10T09:06:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35230.850950 secs 2023-09-10T09:06:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:39 smithi170 ceph-mon[102832]: pgmap v18948: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 10 op/s 2023-09-10T09:06:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35231.203882 secs 2023-09-10T09:06:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35230.850950 secs 2023-09-10T09:06:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:39 smithi082 ceph-mon[98694]: pgmap v18948: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 10 op/s 2023-09-10T09:06:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35231.203882 secs 2023-09-10T09:06:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35230.850950 secs 2023-09-10T09:06:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:39 smithi098 ceph-mon[103124]: pgmap v18948: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 10 op/s 2023-09-10T09:06:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35231.203882 secs 2023-09-10T09:06:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:41 smithi170 ceph-mon[102832]: pgmap v18949: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 421 KiB/s wr, 16 op/s 2023-09-10T09:06:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:41 smithi082 ceph-mon[98694]: pgmap v18949: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 421 KiB/s wr, 16 op/s 2023-09-10T09:06:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:41 smithi098 ceph-mon[103124]: pgmap v18949: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 421 KiB/s wr, 16 op/s 2023-09-10T09:06:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:43 smithi170 ceph-mon[102832]: pgmap v18950: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 421 KiB/s wr, 11 op/s 2023-09-10T09:06:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:43 smithi082 ceph-mon[98694]: pgmap v18950: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 421 KiB/s wr, 11 op/s 2023-09-10T09:06:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:43 smithi098 ceph-mon[103124]: pgmap v18950: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 421 KiB/s wr, 11 op/s 2023-09-10T09:06:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35235.851029 secs 2023-09-10T09:06:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:06:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35236.203950 secs 2023-09-10T09:06:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35235.851029 secs 2023-09-10T09:06:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:06:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35236.203950 secs 2023-09-10T09:06:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35235.851029 secs 2023-09-10T09:06:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:06:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35236.203950 secs 2023-09-10T09:06:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:45 smithi170 ceph-mon[102832]: pgmap v18951: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 421 KiB/s wr, 11 op/s 2023-09-10T09:06:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:06:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:45 smithi082 ceph-mon[98694]: pgmap v18951: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 421 KiB/s wr, 11 op/s 2023-09-10T09:06:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:06:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:45 smithi098 ceph-mon[103124]: pgmap v18951: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 421 KiB/s wr, 11 op/s 2023-09-10T09:06:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:06:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:47 smithi170 ceph-mon[102832]: pgmap v18952: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 492 KiB/s wr, 18 op/s 2023-09-10T09:06:48.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:47 smithi082 ceph-mon[98694]: pgmap v18952: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 492 KiB/s wr, 18 op/s 2023-09-10T09:06:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:47 smithi098 ceph-mon[103124]: pgmap v18952: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 492 KiB/s wr, 18 op/s 2023-09-10T09:06:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35240.851089 secs 2023-09-10T09:06:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:49 smithi170 ceph-mon[102832]: pgmap v18953: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 342 KiB/s wr, 13 op/s 2023-09-10T09:06:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35241.204003 secs 2023-09-10T09:06:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35240.851089 secs 2023-09-10T09:06:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:49 smithi082 ceph-mon[98694]: pgmap v18953: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 342 KiB/s wr, 13 op/s 2023-09-10T09:06:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35241.204003 secs 2023-09-10T09:06:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35240.851089 secs 2023-09-10T09:06:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:49 smithi098 ceph-mon[103124]: pgmap v18953: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 342 KiB/s wr, 13 op/s 2023-09-10T09:06:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35241.204003 secs 2023-09-10T09:06:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:06:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:06:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:06:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:51 smithi170 ceph-mon[102832]: pgmap v18954: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 810 KiB/s wr, 20 op/s 2023-09-10T09:06:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:51 smithi082 ceph-mon[98694]: pgmap v18954: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 810 KiB/s wr, 20 op/s 2023-09-10T09:06:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:51 smithi098 ceph-mon[103124]: pgmap v18954: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 810 KiB/s wr, 20 op/s 2023-09-10T09:06:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:06:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:06:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:06:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:53 smithi170 ceph-mon[102832]: pgmap v18955: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 538 KiB/s wr, 13 op/s 2023-09-10T09:06:53.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:53 smithi082 ceph-mon[98694]: pgmap v18955: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 538 KiB/s wr, 13 op/s 2023-09-10T09:06:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:53 smithi098 ceph-mon[103124]: pgmap v18955: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 538 KiB/s wr, 13 op/s 2023-09-10T09:06:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35245.851162 secs 2023-09-10T09:06:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35246.204068 secs 2023-09-10T09:06:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35245.851162 secs 2023-09-10T09:06:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35246.204068 secs 2023-09-10T09:06:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35245.851162 secs 2023-09-10T09:06:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35246.204068 secs 2023-09-10T09:06:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:55 smithi170 ceph-mon[102832]: pgmap v18956: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 538 KiB/s wr, 13 op/s 2023-09-10T09:06:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:55 smithi082 ceph-mon[98694]: pgmap v18956: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 538 KiB/s wr, 13 op/s 2023-09-10T09:06:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:55 smithi098 ceph-mon[103124]: pgmap v18956: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 538 KiB/s wr, 13 op/s 2023-09-10T09:06:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:57 smithi170 ceph-mon[102832]: pgmap v18957: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 678 KiB/s wr, 20 op/s 2023-09-10T09:06:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:06:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:57 smithi082 ceph-mon[98694]: pgmap v18957: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 678 KiB/s wr, 20 op/s 2023-09-10T09:06:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:06:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:57 smithi098 ceph-mon[103124]: pgmap v18957: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 678 KiB/s wr, 20 op/s 2023-09-10T09:06:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:06:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:06:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35250.851240 secs 2023-09-10T09:06:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:59 smithi170 ceph-mon[102832]: pgmap v18958: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 607 KiB/s wr, 13 op/s 2023-09-10T09:06:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:06:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35251.204132 secs 2023-09-10T09:06:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35250.851240 secs 2023-09-10T09:06:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:59 smithi082 ceph-mon[98694]: pgmap v18958: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 607 KiB/s wr, 13 op/s 2023-09-10T09:06:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:06:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35251.204132 secs 2023-09-10T09:07:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35250.851240 secs 2023-09-10T09:07:00.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:59 smithi098 ceph-mon[103124]: pgmap v18958: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 607 KiB/s wr, 13 op/s 2023-09-10T09:07:00.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:06:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35251.204132 secs 2023-09-10T09:07:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:07:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:07:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:07:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:01 smithi170 ceph-mon[102832]: pgmap v18959: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1021 KiB/s wr, 31 op/s 2023-09-10T09:07:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:01 smithi082 ceph-mon[98694]: pgmap v18959: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1021 KiB/s wr, 31 op/s 2023-09-10T09:07:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:01 smithi098 ceph-mon[103124]: pgmap v18959: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1021 KiB/s wr, 31 op/s 2023-09-10T09:07:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:03 smithi170 ceph-mon[102832]: pgmap v18960: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 553 KiB/s wr, 24 op/s 2023-09-10T09:07:04.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35254.601049 secs 2023-09-10T09:07:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:03 smithi082 ceph-mon[98694]: pgmap v18960: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 553 KiB/s wr, 24 op/s 2023-09-10T09:07:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35254.601049 secs 2023-09-10T09:07:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:03 smithi098 ceph-mon[103124]: pgmap v18960: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 553 KiB/s wr, 24 op/s 2023-09-10T09:07:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35254.601049 secs 2023-09-10T09:07:05.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35255.851298 secs 2023-09-10T09:07:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:07:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:07:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:04 smithi170 ceph-mon[102832]: osdmap e1253: 12 total, 12 up, 12 in 2023-09-10T09:07:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35256.204198 secs 2023-09-10T09:07:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35255.851298 secs 2023-09-10T09:07:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:07:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:07:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:04 smithi082 ceph-mon[98694]: osdmap e1253: 12 total, 12 up, 12 in 2023-09-10T09:07:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35256.204198 secs 2023-09-10T09:07:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35255.851298 secs 2023-09-10T09:07:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:07:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:07:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:04 smithi098 ceph-mon[103124]: osdmap e1253: 12 total, 12 up, 12 in 2023-09-10T09:07:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35256.204198 secs 2023-09-10T09:07:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:05 smithi170 ceph-mon[102832]: pgmap v18962: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 664 KiB/s wr, 29 op/s 2023-09-10T09:07:06.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:05 smithi082 ceph-mon[98694]: pgmap v18962: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 664 KiB/s wr, 29 op/s 2023-09-10T09:07:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:05 smithi098 ceph-mon[103124]: pgmap v18962: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 664 KiB/s wr, 29 op/s 2023-09-10T09:07:07.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:07:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:07:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:07:08.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:07 smithi170 ceph-mon[102832]: pgmap v18963: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 672 KiB/s wr, 33 op/s 2023-09-10T09:07:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:07 smithi082 ceph-mon[98694]: pgmap v18963: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 672 KiB/s wr, 33 op/s 2023-09-10T09:07:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:07 smithi098 ceph-mon[103124]: pgmap v18963: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 672 KiB/s wr, 33 op/s 2023-09-10T09:07:09.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35259.601109 secs 2023-09-10T09:07:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:08 smithi170 ceph-mon[102832]: osdmap e1254: 12 total, 12 up, 12 in 2023-09-10T09:07:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:07:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:07:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35259.601109 secs 2023-09-10T09:07:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:08 smithi082 ceph-mon[98694]: osdmap e1254: 12 total, 12 up, 12 in 2023-09-10T09:07:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:07:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:07:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35259.601109 secs 2023-09-10T09:07:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:08 smithi098 ceph-mon[103124]: osdmap e1254: 12 total, 12 up, 12 in 2023-09-10T09:07:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:07:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:07:10.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35260.851362 secs 2023-09-10T09:07:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:09 smithi170 ceph-mon[102832]: pgmap v18965: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 219 KiB/s wr, 14 op/s 2023-09-10T09:07:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:07:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:07:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35261.204264 secs 2023-09-10T09:07:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:07:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:07:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:07:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:07:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:07:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:07:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:07:10.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:09 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T09:07:10.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:09 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T09:07:10.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:07:10.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:07:10.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:07:10.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:07:10.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:07:10.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:07:10.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:07:10.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:09 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:07:10.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:09 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35260.851362 secs 2023-09-10T09:07:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:09 smithi082 ceph-mon[98694]: pgmap v18965: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 219 KiB/s wr, 14 op/s 2023-09-10T09:07:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:07:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:07:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35261.204264 secs 2023-09-10T09:07:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:07:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:07:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:07:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:07:10.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:07:10.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:07:10.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:07:10.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:09 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T09:07:10.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:09 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T09:07:10.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:07:10.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:07:10.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:07:10.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:07:10.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:07:10.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:07:10.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:07:10.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:09 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:07:10.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:09 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35260.851362 secs 2023-09-10T09:07:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:09 smithi098 ceph-mon[103124]: pgmap v18965: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 219 KiB/s wr, 14 op/s 2023-09-10T09:07:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:07:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:07:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35261.204264 secs 2023-09-10T09:07:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:07:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:07:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:07:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:07:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:07:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:07:10.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:07:10.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:09 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T09:07:10.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:09 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T09:07:10.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:07:10.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:07:10.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:07:10.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:07:10.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:07:10.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:07:10.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:07:10.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:09 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:07:10.258 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:09 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:11.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:10 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T09:07:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:10 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T09:07:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:10 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T09:07:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:10 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T09:07:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:10 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T09:07:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:10 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T09:07:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:11 smithi170 ceph-mon[102832]: pgmap v18966: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 670 KiB/s wr, 23 op/s 2023-09-10T09:07:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:11 smithi082 ceph-mon[98694]: pgmap v18966: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 670 KiB/s wr, 23 op/s 2023-09-10T09:07:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:11 smithi098 ceph-mon[103124]: pgmap v18966: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 670 KiB/s wr, 23 op/s 2023-09-10T09:07:13.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:07:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:07:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:07:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:14.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:13 smithi170 ceph-mon[102832]: pgmap v18967: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 599 KiB/s wr, 21 op/s 2023-09-10T09:07:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35264.601169 secs 2023-09-10T09:07:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:13 smithi082 ceph-mon[98694]: pgmap v18967: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 599 KiB/s wr, 21 op/s 2023-09-10T09:07:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35264.601169 secs 2023-09-10T09:07:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:13 smithi098 ceph-mon[103124]: pgmap v18967: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 599 KiB/s wr, 21 op/s 2023-09-10T09:07:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35264.601169 secs 2023-09-10T09:07:15.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35265.851441 secs 2023-09-10T09:07:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35266.204329 secs 2023-09-10T09:07:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35265.851441 secs 2023-09-10T09:07:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35266.204329 secs 2023-09-10T09:07:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35265.851441 secs 2023-09-10T09:07:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35266.204329 secs 2023-09-10T09:07:16.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:15 smithi170 ceph-mon[102832]: pgmap v18968: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 536 KiB/s wr, 19 op/s 2023-09-10T09:07:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:15 smithi082 ceph-mon[98694]: pgmap v18968: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 536 KiB/s wr, 19 op/s 2023-09-10T09:07:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:15 smithi098 ceph-mon[103124]: pgmap v18968: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 536 KiB/s wr, 19 op/s 2023-09-10T09:07:18.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:17 smithi170 ceph-mon[102832]: pgmap v18969: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 361 KiB/s wr, 13 op/s 2023-09-10T09:07:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:17 smithi082 ceph-mon[98694]: pgmap v18969: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 361 KiB/s wr, 13 op/s 2023-09-10T09:07:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:17 smithi098 ceph-mon[103124]: pgmap v18969: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 361 KiB/s wr, 13 op/s 2023-09-10T09:07:19.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35269.601237 secs 2023-09-10T09:07:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35269.601237 secs 2023-09-10T09:07:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35269.601237 secs 2023-09-10T09:07:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35270.851492 secs 2023-09-10T09:07:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:19 smithi170 ceph-mon[102832]: pgmap v18970: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 353 KiB/s wr, 13 op/s 2023-09-10T09:07:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35271.204395 secs 2023-09-10T09:07:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35270.851492 secs 2023-09-10T09:07:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:19 smithi082 ceph-mon[98694]: pgmap v18970: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 353 KiB/s wr, 13 op/s 2023-09-10T09:07:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35271.204395 secs 2023-09-10T09:07:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35270.851492 secs 2023-09-10T09:07:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:19 smithi098 ceph-mon[103124]: pgmap v18970: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 353 KiB/s wr, 13 op/s 2023-09-10T09:07:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35271.204395 secs 2023-09-10T09:07:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:21 smithi170 ceph-mon[102832]: pgmap v18971: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 423 KiB/s wr, 16 op/s 2023-09-10T09:07:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:21 smithi082 ceph-mon[98694]: pgmap v18971: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 423 KiB/s wr, 16 op/s 2023-09-10T09:07:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:21 smithi098 ceph-mon[103124]: pgmap v18971: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 423 KiB/s wr, 16 op/s 2023-09-10T09:07:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:23 smithi170 ceph-mon[102832]: pgmap v18972: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T09:07:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35274.601353 secs 2023-09-10T09:07:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:23 smithi082 ceph-mon[98694]: pgmap v18972: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T09:07:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35274.601353 secs 2023-09-10T09:07:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:23 smithi098 ceph-mon[103124]: pgmap v18972: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T09:07:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35274.601353 secs 2023-09-10T09:07:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35275.851551 secs 2023-09-10T09:07:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:07:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35276.204457 secs 2023-09-10T09:07:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35275.851551 secs 2023-09-10T09:07:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:07:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35276.204457 secs 2023-09-10T09:07:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35275.851551 secs 2023-09-10T09:07:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:07:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35276.204457 secs 2023-09-10T09:07:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:25 smithi082 ceph-mon[98694]: pgmap v18973: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T09:07:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:25 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:07:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:25 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:25 smithi098 ceph-mon[103124]: pgmap v18973: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T09:07:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:25 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:07:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:25 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:25 smithi170 ceph-mon[102832]: pgmap v18973: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T09:07:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:25 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:07:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:25 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:27 smithi082 ceph-mon[98694]: pgmap v18974: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 15 op/s 2023-09-10T09:07:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:27 smithi098 ceph-mon[103124]: pgmap v18974: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 15 op/s 2023-09-10T09:07:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:27 smithi170 ceph-mon[102832]: pgmap v18974: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 123 KiB/s wr, 15 op/s 2023-09-10T09:07:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35279.601421 secs 2023-09-10T09:07:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35279.601421 secs 2023-09-10T09:07:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35279.601421 secs 2023-09-10T09:07:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35280.851614 secs 2023-09-10T09:07:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:29 smithi082 ceph-mon[98694]: pgmap v18975: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T09:07:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35281.204492 secs 2023-09-10T09:07:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35280.851614 secs 2023-09-10T09:07:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:29 smithi098 ceph-mon[103124]: pgmap v18975: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T09:07:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35281.204492 secs 2023-09-10T09:07:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35280.851614 secs 2023-09-10T09:07:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:29 smithi170 ceph-mon[102832]: pgmap v18975: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 10 op/s 2023-09-10T09:07:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35281.204492 secs 2023-09-10T09:07:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:31 smithi082 ceph-mon[98694]: pgmap v18976: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 250 KiB/s wr, 15 op/s 2023-09-10T09:07:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:31 smithi098 ceph-mon[103124]: pgmap v18976: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 250 KiB/s wr, 15 op/s 2023-09-10T09:07:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:31 smithi170 ceph-mon[102832]: pgmap v18976: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 250 KiB/s wr, 15 op/s 2023-09-10T09:07:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:33 smithi082 ceph-mon[98694]: pgmap v18977: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 10 op/s 2023-09-10T09:07:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:33 smithi098 ceph-mon[103124]: pgmap v18977: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 10 op/s 2023-09-10T09:07:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:33 smithi170 ceph-mon[102832]: pgmap v18977: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 10 op/s 2023-09-10T09:07:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35285.851674 secs 2023-09-10T09:07:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35286.204561 secs 2023-09-10T09:07:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35285.851674 secs 2023-09-10T09:07:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35286.204561 secs 2023-09-10T09:07:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35285.851674 secs 2023-09-10T09:07:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35286.204561 secs 2023-09-10T09:07:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:35 smithi082 ceph-mon[98694]: pgmap v18978: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 10 op/s 2023-09-10T09:07:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:07:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:35 smithi098 ceph-mon[103124]: pgmap v18978: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 10 op/s 2023-09-10T09:07:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:07:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:35 smithi170 ceph-mon[102832]: pgmap v18978: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 128 KiB/s wr, 10 op/s 2023-09-10T09:07:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:07:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:07:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:07:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:07:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:07:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:07:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:07:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:37 smithi098 ceph-mon[103124]: pgmap v18979: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 251 KiB/s wr, 16 op/s 2023-09-10T09:07:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:07:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:37 smithi170 ceph-mon[102832]: pgmap v18979: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 251 KiB/s wr, 16 op/s 2023-09-10T09:07:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:07:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:37 smithi082 ceph-mon[98694]: pgmap v18979: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 251 KiB/s wr, 16 op/s 2023-09-10T09:07:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:07:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:39.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35289.601538 secs 2023-09-10T09:07:39.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35289.601538 secs 2023-09-10T09:07:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35289.601538 secs 2023-09-10T09:07:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35290.851747 secs 2023-09-10T09:07:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:39 smithi098 ceph-mon[103124]: pgmap v18980: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 250 KiB/s wr, 11 op/s 2023-09-10T09:07:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35291.204625 secs 2023-09-10T09:07:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35290.851747 secs 2023-09-10T09:07:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:39 smithi170 ceph-mon[102832]: pgmap v18980: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 250 KiB/s wr, 11 op/s 2023-09-10T09:07:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35291.204625 secs 2023-09-10T09:07:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35290.851747 secs 2023-09-10T09:07:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:39 smithi082 ceph-mon[98694]: pgmap v18980: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 250 KiB/s wr, 11 op/s 2023-09-10T09:07:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35291.204625 secs 2023-09-10T09:07:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:07:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:07:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:07:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:41 smithi098 ceph-mon[103124]: pgmap v18981: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 921 KiB/s wr, 19 op/s 2023-09-10T09:07:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:41 smithi170 ceph-mon[102832]: pgmap v18981: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 921 KiB/s wr, 19 op/s 2023-09-10T09:07:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:41 smithi082 ceph-mon[98694]: pgmap v18981: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 921 KiB/s wr, 19 op/s 2023-09-10T09:07:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:43 smithi098 ceph-mon[103124]: pgmap v18982: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 794 KiB/s wr, 13 op/s 2023-09-10T09:07:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35294.601639 secs 2023-09-10T09:07:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:43 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:07:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:43 smithi170 ceph-mon[102832]: pgmap v18982: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 794 KiB/s wr, 13 op/s 2023-09-10T09:07:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35294.601639 secs 2023-09-10T09:07:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:43 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:07:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:43 smithi082 ceph-mon[98694]: pgmap v18982: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 794 KiB/s wr, 13 op/s 2023-09-10T09:07:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35294.601639 secs 2023-09-10T09:07:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:43 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:07:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35295.851807 secs 2023-09-10T09:07:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35296.204690 secs 2023-09-10T09:07:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:07:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35295.851807 secs 2023-09-10T09:07:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35296.204690 secs 2023-09-10T09:07:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:07:45.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35295.851807 secs 2023-09-10T09:07:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35296.204690 secs 2023-09-10T09:07:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:07:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:45 smithi098 ceph-mon[103124]: pgmap v18983: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 794 KiB/s wr, 13 op/s 2023-09-10T09:07:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:07:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:45 smithi170 ceph-mon[102832]: pgmap v18983: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 794 KiB/s wr, 13 op/s 2023-09-10T09:07:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:07:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:45 smithi082 ceph-mon[98694]: pgmap v18983: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 794 KiB/s wr, 13 op/s 2023-09-10T09:07:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:07:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:47 smithi098 ceph-mon[103124]: pgmap v18984: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 817 KiB/s wr, 20 op/s 2023-09-10T09:07:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:47 smithi170 ceph-mon[102832]: pgmap v18984: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 817 KiB/s wr, 20 op/s 2023-09-10T09:07:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:47 smithi082 ceph-mon[98694]: pgmap v18984: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 817 KiB/s wr, 20 op/s 2023-09-10T09:07:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35300.851902 secs 2023-09-10T09:07:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:49 smithi170 ceph-mon[102832]: pgmap v18985: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 694 KiB/s wr, 14 op/s 2023-09-10T09:07:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35301.204760 secs 2023-09-10T09:07:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35300.851902 secs 2023-09-10T09:07:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:49 smithi082 ceph-mon[98694]: pgmap v18985: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 694 KiB/s wr, 14 op/s 2023-09-10T09:07:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35301.204760 secs 2023-09-10T09:07:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35300.851902 secs 2023-09-10T09:07:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:49 smithi098 ceph-mon[103124]: pgmap v18985: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 694 KiB/s wr, 14 op/s 2023-09-10T09:07:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35301.204760 secs 2023-09-10T09:07:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:51 smithi170 ceph-mon[102832]: pgmap v18986: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:07:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:51 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:07:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:51 smithi082 ceph-mon[98694]: pgmap v18986: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:07:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:51 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:07:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:51 smithi098 ceph-mon[103124]: pgmap v18986: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:07:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:51 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:07:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:52 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:07:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:07:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:52 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:07:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:07:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:52 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:07:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:07:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:53 smithi170 ceph-mon[102832]: pgmap v18987: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 469 KiB/s wr, 14 op/s 2023-09-10T09:07:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:07:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:53 smithi082 ceph-mon[98694]: pgmap v18987: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 469 KiB/s wr, 14 op/s 2023-09-10T09:07:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:07:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:53 smithi098 ceph-mon[103124]: pgmap v18987: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 469 KiB/s wr, 14 op/s 2023-09-10T09:07:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:07:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35305.851963 secs 2023-09-10T09:07:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35306.204827 secs 2023-09-10T09:07:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35305.851963 secs 2023-09-10T09:07:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35306.204827 secs 2023-09-10T09:07:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35305.851963 secs 2023-09-10T09:07:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35306.204827 secs 2023-09-10T09:07:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:55 smithi170 ceph-mon[102832]: pgmap v18988: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 469 KiB/s wr, 14 op/s 2023-09-10T09:07:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:55 smithi082 ceph-mon[98694]: pgmap v18988: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 469 KiB/s wr, 14 op/s 2023-09-10T09:07:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:55 smithi098 ceph-mon[103124]: pgmap v18988: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 469 KiB/s wr, 14 op/s 2023-09-10T09:07:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:07:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:07:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:07:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:07:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:57 smithi170 ceph-mon[102832]: pgmap v18989: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 912 KiB/s wr, 21 op/s 2023-09-10T09:07:58.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:57 smithi082 ceph-mon[98694]: pgmap v18989: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 912 KiB/s wr, 21 op/s 2023-09-10T09:07:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:57 smithi098 ceph-mon[103124]: pgmap v18989: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 912 KiB/s wr, 21 op/s 2023-09-10T09:07:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35309.601875 secs 2023-09-10T09:07:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35309.601875 secs 2023-09-10T09:07:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35309.601875 secs 2023-09-10T09:08:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35310.852022 secs 2023-09-10T09:08:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:59 smithi170 ceph-mon[102832]: pgmap v18990: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 889 KiB/s wr, 15 op/s 2023-09-10T09:08:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:07:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35311.204893 secs 2023-09-10T09:08:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35310.852022 secs 2023-09-10T09:08:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:59 smithi082 ceph-mon[98694]: pgmap v18990: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 889 KiB/s wr, 15 op/s 2023-09-10T09:08:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:07:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35311.204893 secs 2023-09-10T09:08:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35310.852022 secs 2023-09-10T09:08:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:59 smithi098 ceph-mon[103124]: pgmap v18990: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 889 KiB/s wr, 15 op/s 2023-09-10T09:08:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:07:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35311.204893 secs 2023-09-10T09:08:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:08:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:08:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:08:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:08:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:08:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:08:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:01 smithi170 ceph-mon[102832]: pgmap v18991: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 990 KiB/s wr, 21 op/s 2023-09-10T09:08:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:01 smithi082 ceph-mon[98694]: pgmap v18991: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 990 KiB/s wr, 21 op/s 2023-09-10T09:08:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:01 smithi098 ceph-mon[103124]: pgmap v18991: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 990 KiB/s wr, 21 op/s 2023-09-10T09:08:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:03 smithi170 ceph-mon[102832]: pgmap v18992: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 545 KiB/s wr, 13 op/s 2023-09-10T09:08:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35314.601934 secs 2023-09-10T09:08:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:03 smithi170 ceph-mon[102832]: osdmap e1255: 12 total, 12 up, 12 in 2023-09-10T09:08:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:03 smithi082 ceph-mon[98694]: pgmap v18992: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 545 KiB/s wr, 13 op/s 2023-09-10T09:08:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35314.601934 secs 2023-09-10T09:08:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:03 smithi082 ceph-mon[98694]: osdmap e1255: 12 total, 12 up, 12 in 2023-09-10T09:08:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:03 smithi098 ceph-mon[103124]: pgmap v18992: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 545 KiB/s wr, 13 op/s 2023-09-10T09:08:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35314.601934 secs 2023-09-10T09:08:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:03 smithi098 ceph-mon[103124]: osdmap e1255: 12 total, 12 up, 12 in 2023-09-10T09:08:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35315.852097 secs 2023-09-10T09:08:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:08:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35316.204955 secs 2023-09-10T09:08:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35315.852097 secs 2023-09-10T09:08:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:08:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35316.204955 secs 2023-09-10T09:08:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35315.852097 secs 2023-09-10T09:08:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:08:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35316.204955 secs 2023-09-10T09:08:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:05 smithi082 ceph-mon[98694]: pgmap v18994: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 654 KiB/s wr, 16 op/s 2023-09-10T09:08:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:08:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:05 smithi098 ceph-mon[103124]: pgmap v18994: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 654 KiB/s wr, 16 op/s 2023-09-10T09:08:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:08:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:05 smithi170 ceph-mon[102832]: pgmap v18994: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 654 KiB/s wr, 16 op/s 2023-09-10T09:08:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:08:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:07 smithi082 ceph-mon[98694]: pgmap v18995: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 555 KiB/s wr, 33 op/s 2023-09-10T09:08:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:07 smithi098 ceph-mon[103124]: pgmap v18995: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 555 KiB/s wr, 33 op/s 2023-09-10T09:08:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:07 smithi170 ceph-mon[102832]: pgmap v18995: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 555 KiB/s wr, 33 op/s 2023-09-10T09:08:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35319.601997 secs 2023-09-10T09:08:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:08 smithi082 ceph-mon[98694]: osdmap e1256: 12 total, 12 up, 12 in 2023-09-10T09:08:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35319.601997 secs 2023-09-10T09:08:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:08 smithi098 ceph-mon[103124]: osdmap e1256: 12 total, 12 up, 12 in 2023-09-10T09:08:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35319.601997 secs 2023-09-10T09:08:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:08 smithi170 ceph-mon[102832]: osdmap e1256: 12 total, 12 up, 12 in 2023-09-10T09:08:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35320.852159 secs 2023-09-10T09:08:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:09 smithi082 ceph-mon[98694]: pgmap v18997: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 542 KiB/s wr, 31 op/s 2023-09-10T09:08:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35321.205020 secs 2023-09-10T09:08:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:08:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35320.852159 secs 2023-09-10T09:08:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:09 smithi098 ceph-mon[103124]: pgmap v18997: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 542 KiB/s wr, 31 op/s 2023-09-10T09:08:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35321.205020 secs 2023-09-10T09:08:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:08:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35320.852159 secs 2023-09-10T09:08:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:09 smithi170 ceph-mon[102832]: pgmap v18997: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 542 KiB/s wr, 31 op/s 2023-09-10T09:08:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35321.205020 secs 2023-09-10T09:08:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:08:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:08:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:08:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:08:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:08:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:08:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:08:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:08:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:08:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:08:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:12 smithi170 ceph-mon[102832]: pgmap v18998: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 734 KiB/s wr, 42 op/s 2023-09-10T09:08:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:12 smithi082 ceph-mon[98694]: pgmap v18998: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 734 KiB/s wr, 42 op/s 2023-09-10T09:08:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:12 smithi098 ceph-mon[103124]: pgmap v18998: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 734 KiB/s wr, 42 op/s 2023-09-10T09:08:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35324.602057 secs 2023-09-10T09:08:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35324.602057 secs 2023-09-10T09:08:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35324.602057 secs 2023-09-10T09:08:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:14 smithi170 ceph-mon[102832]: pgmap v18999: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 593 KiB/s wr, 34 op/s 2023-09-10T09:08:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35325.852218 secs 2023-09-10T09:08:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:14 smithi082 ceph-mon[98694]: pgmap v18999: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 593 KiB/s wr, 34 op/s 2023-09-10T09:08:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35325.852218 secs 2023-09-10T09:08:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:14 smithi098 ceph-mon[103124]: pgmap v18999: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 593 KiB/s wr, 34 op/s 2023-09-10T09:08:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35325.852218 secs 2023-09-10T09:08:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35326.205087 secs 2023-09-10T09:08:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35326.205087 secs 2023-09-10T09:08:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35326.205087 secs 2023-09-10T09:08:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:16 smithi170 ceph-mon[102832]: pgmap v19000: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 587 KiB/s wr, 33 op/s 2023-09-10T09:08:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:16 smithi082 ceph-mon[98694]: pgmap v19000: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 587 KiB/s wr, 33 op/s 2023-09-10T09:08:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:16 smithi098 ceph-mon[103124]: pgmap v19000: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 587 KiB/s wr, 33 op/s 2023-09-10T09:08:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:18 smithi170 ceph-mon[102832]: pgmap v19001: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 186 KiB/s wr, 15 op/s 2023-09-10T09:08:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35329.602161 secs 2023-09-10T09:08:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:18 smithi082 ceph-mon[98694]: pgmap v19001: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 186 KiB/s wr, 15 op/s 2023-09-10T09:08:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35329.602161 secs 2023-09-10T09:08:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:18 smithi098 ceph-mon[103124]: pgmap v19001: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 186 KiB/s wr, 15 op/s 2023-09-10T09:08:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35329.602161 secs 2023-09-10T09:08:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35330.852275 secs 2023-09-10T09:08:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:19 smithi170 ceph-mon[102832]: pgmap v19002: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 182 KiB/s wr, 14 op/s 2023-09-10T09:08:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35331.205154 secs 2023-09-10T09:08:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35330.852275 secs 2023-09-10T09:08:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:19 smithi082 ceph-mon[98694]: pgmap v19002: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 182 KiB/s wr, 14 op/s 2023-09-10T09:08:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35331.205154 secs 2023-09-10T09:08:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35330.852275 secs 2023-09-10T09:08:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:19 smithi098 ceph-mon[103124]: pgmap v19002: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 182 KiB/s wr, 14 op/s 2023-09-10T09:08:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35331.205154 secs 2023-09-10T09:08:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:21 smithi170 ceph-mon[102832]: pgmap v19003: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 257 KiB/s wr, 18 op/s 2023-09-10T09:08:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:21 smithi082 ceph-mon[98694]: pgmap v19003: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 257 KiB/s wr, 18 op/s 2023-09-10T09:08:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:21 smithi098 ceph-mon[103124]: pgmap v19003: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 257 KiB/s wr, 18 op/s 2023-09-10T09:08:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:23 smithi170 ceph-mon[102832]: pgmap v19004: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T09:08:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35334.602267 secs 2023-09-10T09:08:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:23 smithi082 ceph-mon[98694]: pgmap v19004: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T09:08:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35334.602267 secs 2023-09-10T09:08:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:23 smithi098 ceph-mon[103124]: pgmap v19004: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T09:08:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35334.602267 secs 2023-09-10T09:08:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35335.852338 secs 2023-09-10T09:08:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:08:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35336.205218 secs 2023-09-10T09:08:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35335.852338 secs 2023-09-10T09:08:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:08:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35336.205218 secs 2023-09-10T09:08:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35335.852338 secs 2023-09-10T09:08:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:08:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35336.205218 secs 2023-09-10T09:08:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:25 smithi170 ceph-mon[102832]: pgmap v19005: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T09:08:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:25 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:08:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:25 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:25 smithi082 ceph-mon[98694]: pgmap v19005: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T09:08:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:25 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:08:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:25 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:25 smithi098 ceph-mon[103124]: pgmap v19005: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T09:08:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:25 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:08:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:25 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:27 smithi170 ceph-mon[102832]: pgmap v19006: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 256 KiB/s wr, 17 op/s 2023-09-10T09:08:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:27 smithi082 ceph-mon[98694]: pgmap v19006: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 256 KiB/s wr, 17 op/s 2023-09-10T09:08:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:27 smithi098 ceph-mon[103124]: pgmap v19006: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 256 KiB/s wr, 17 op/s 2023-09-10T09:08:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35339.602351 secs 2023-09-10T09:08:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35339.602351 secs 2023-09-10T09:08:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35339.602351 secs 2023-09-10T09:08:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35340.852398 secs 2023-09-10T09:08:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:29 smithi170 ceph-mon[102832]: pgmap v19007: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 229 KiB/s wr, 11 op/s 2023-09-10T09:08:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35341.205283 secs 2023-09-10T09:08:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35340.852398 secs 2023-09-10T09:08:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:29 smithi082 ceph-mon[98694]: pgmap v19007: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 229 KiB/s wr, 11 op/s 2023-09-10T09:08:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35341.205283 secs 2023-09-10T09:08:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35340.852398 secs 2023-09-10T09:08:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:29 smithi098 ceph-mon[103124]: pgmap v19007: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 229 KiB/s wr, 11 op/s 2023-09-10T09:08:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35341.205283 secs 2023-09-10T09:08:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:31 smithi170 ceph-mon[102832]: pgmap v19008: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 229 KiB/s wr, 16 op/s 2023-09-10T09:08:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:31 smithi082 ceph-mon[98694]: pgmap v19008: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 229 KiB/s wr, 16 op/s 2023-09-10T09:08:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:31 smithi098 ceph-mon[103124]: pgmap v19008: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 229 KiB/s wr, 16 op/s 2023-09-10T09:08:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:33 smithi170 ceph-mon[102832]: pgmap v19009: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T09:08:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:08:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:33 smithi082 ceph-mon[98694]: pgmap v19009: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T09:08:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:08:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:33 smithi098 ceph-mon[103124]: pgmap v19009: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T09:08:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:08:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35345.852494 secs 2023-09-10T09:08:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35346.205350 secs 2023-09-10T09:08:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35345.852494 secs 2023-09-10T09:08:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35346.205350 secs 2023-09-10T09:08:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35345.852494 secs 2023-09-10T09:08:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35346.205350 secs 2023-09-10T09:08:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:35 smithi170 ceph-mon[102832]: pgmap v19010: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T09:08:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:08:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:35 smithi082 ceph-mon[98694]: pgmap v19010: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T09:08:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:08:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:35 smithi098 ceph-mon[103124]: pgmap v19010: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 127 KiB/s wr, 11 op/s 2023-09-10T09:08:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:08:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:08:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:08:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:08:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:08:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:08:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:08:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:37 smithi170 ceph-mon[102832]: pgmap v19011: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 278 KiB/s wr, 17 op/s 2023-09-10T09:08:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:37 smithi082 ceph-mon[98694]: pgmap v19011: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 278 KiB/s wr, 17 op/s 2023-09-10T09:08:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:37 smithi098 ceph-mon[103124]: pgmap v19011: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 278 KiB/s wr, 17 op/s 2023-09-10T09:08:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35350.852553 secs 2023-09-10T09:08:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:39 smithi170 ceph-mon[102832]: pgmap v19012: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T09:08:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35351.205419 secs 2023-09-10T09:08:39.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35350.852553 secs 2023-09-10T09:08:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:39 smithi082 ceph-mon[98694]: pgmap v19012: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T09:08:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35351.205419 secs 2023-09-10T09:08:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35350.852553 secs 2023-09-10T09:08:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:39 smithi098 ceph-mon[103124]: pgmap v19012: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T09:08:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35351.205419 secs 2023-09-10T09:08:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:41 smithi170 ceph-mon[102832]: pgmap v19013: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 423 KiB/s wr, 17 op/s 2023-09-10T09:08:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:41 smithi082 ceph-mon[98694]: pgmap v19013: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 423 KiB/s wr, 17 op/s 2023-09-10T09:08:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:41 smithi098 ceph-mon[103124]: pgmap v19013: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 423 KiB/s wr, 17 op/s 2023-09-10T09:08:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:43 smithi170 ceph-mon[102832]: pgmap v19014: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 423 KiB/s wr, 12 op/s 2023-09-10T09:08:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:43 smithi082 ceph-mon[98694]: pgmap v19014: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 423 KiB/s wr, 12 op/s 2023-09-10T09:08:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:43 smithi098 ceph-mon[103124]: pgmap v19014: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 423 KiB/s wr, 12 op/s 2023-09-10T09:08:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35355.852628 secs 2023-09-10T09:08:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35356.205486 secs 2023-09-10T09:08:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35355.852628 secs 2023-09-10T09:08:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35356.205486 secs 2023-09-10T09:08:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35355.852628 secs 2023-09-10T09:08:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35356.205486 secs 2023-09-10T09:08:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:45 smithi170 ceph-mon[102832]: pgmap v19015: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 423 KiB/s wr, 12 op/s 2023-09-10T09:08:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:45 smithi082 ceph-mon[98694]: pgmap v19015: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 423 KiB/s wr, 12 op/s 2023-09-10T09:08:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:45 smithi098 ceph-mon[103124]: pgmap v19015: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 423 KiB/s wr, 12 op/s 2023-09-10T09:08:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:47 smithi170 ceph-mon[102832]: pgmap v19016: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 616 KiB/s wr, 19 op/s 2023-09-10T09:08:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:47 smithi082 ceph-mon[98694]: pgmap v19016: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 616 KiB/s wr, 19 op/s 2023-09-10T09:08:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:47 smithi098 ceph-mon[103124]: pgmap v19016: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 616 KiB/s wr, 19 op/s 2023-09-10T09:08:49.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35359.602599 secs 2023-09-10T09:08:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35359.602599 secs 2023-09-10T09:08:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35359.602599 secs 2023-09-10T09:08:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35360.852688 secs 2023-09-10T09:08:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:49 smithi170 ceph-mon[102832]: pgmap v19017: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 465 KiB/s wr, 13 op/s 2023-09-10T09:08:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35361.205560 secs 2023-09-10T09:08:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:08:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:50.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35360.852688 secs 2023-09-10T09:08:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:49 smithi082 ceph-mon[98694]: pgmap v19017: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 465 KiB/s wr, 13 op/s 2023-09-10T09:08:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35361.205560 secs 2023-09-10T09:08:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:08:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35360.852688 secs 2023-09-10T09:08:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:49 smithi098 ceph-mon[103124]: pgmap v19017: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 465 KiB/s wr, 13 op/s 2023-09-10T09:08:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35361.205560 secs 2023-09-10T09:08:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:08:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:51 smithi170 ceph-mon[102832]: pgmap v19018: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 814 KiB/s wr, 20 op/s 2023-09-10T09:08:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:51 smithi082 ceph-mon[98694]: pgmap v19018: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 814 KiB/s wr, 20 op/s 2023-09-10T09:08:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:51 smithi098 ceph-mon[103124]: pgmap v19018: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 814 KiB/s wr, 20 op/s 2023-09-10T09:08:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:08:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:08:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:08:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:08:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:08:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:08:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:53 smithi170 ceph-mon[102832]: pgmap v19019: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 542 KiB/s wr, 14 op/s 2023-09-10T09:08:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35364.602659 secs 2023-09-10T09:08:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:53 smithi082 ceph-mon[98694]: pgmap v19019: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 542 KiB/s wr, 14 op/s 2023-09-10T09:08:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35364.602659 secs 2023-09-10T09:08:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:53 smithi098 ceph-mon[103124]: pgmap v19019: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 542 KiB/s wr, 14 op/s 2023-09-10T09:08:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35364.602659 secs 2023-09-10T09:08:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35365.852746 secs 2023-09-10T09:08:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35366.205624 secs 2023-09-10T09:08:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35365.852746 secs 2023-09-10T09:08:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35366.205624 secs 2023-09-10T09:08:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35365.852746 secs 2023-09-10T09:08:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35366.205624 secs 2023-09-10T09:08:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:55 smithi170 ceph-mon[102832]: pgmap v19020: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 542 KiB/s wr, 14 op/s 2023-09-10T09:08:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:55 smithi082 ceph-mon[98694]: pgmap v19020: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 542 KiB/s wr, 14 op/s 2023-09-10T09:08:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:55 smithi098 ceph-mon[103124]: pgmap v19020: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 542 KiB/s wr, 14 op/s 2023-09-10T09:08:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:08:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:08:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:57.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:08:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:57 smithi082 ceph-mon[98694]: pgmap v19021: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 585 KiB/s wr, 20 op/s 2023-09-10T09:08:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:08:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:57 smithi098 ceph-mon[103124]: pgmap v19021: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 585 KiB/s wr, 20 op/s 2023-09-10T09:08:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:08:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:57 smithi170 ceph-mon[102832]: pgmap v19021: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 585 KiB/s wr, 20 op/s 2023-09-10T09:08:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:08:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:08:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35369.602804 secs 2023-09-10T09:08:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35369.602804 secs 2023-09-10T09:08:59.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35369.602804 secs 2023-09-10T09:09:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35370.852807 secs 2023-09-10T09:09:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:59 smithi082 ceph-mon[98694]: pgmap v19022: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T09:09:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:08:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35371.205692 secs 2023-09-10T09:09:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35370.852807 secs 2023-09-10T09:09:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:59 smithi098 ceph-mon[103124]: pgmap v19022: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T09:09:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:08:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35371.205692 secs 2023-09-10T09:09:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35370.852807 secs 2023-09-10T09:09:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:59 smithi170 ceph-mon[102832]: pgmap v19022: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T09:09:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:08:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35371.205692 secs 2023-09-10T09:09:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:09:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:09:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:09:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:01 smithi082 ceph-mon[98694]: pgmap v19023: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:09:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:01 smithi098 ceph-mon[103124]: pgmap v19023: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:09:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:01 smithi170 ceph-mon[102832]: pgmap v19023: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:09:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:03 smithi082 ceph-mon[98694]: pgmap v19024: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 741 KiB/s wr, 13 op/s 2023-09-10T09:09:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35374.602865 secs 2023-09-10T09:09:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:03 smithi098 ceph-mon[103124]: pgmap v19024: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 741 KiB/s wr, 13 op/s 2023-09-10T09:09:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35374.602865 secs 2023-09-10T09:09:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:03 smithi170 ceph-mon[102832]: pgmap v19024: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 741 KiB/s wr, 13 op/s 2023-09-10T09:09:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35374.602865 secs 2023-09-10T09:09:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35375.852868 secs 2023-09-10T09:09:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35376.205760 secs 2023-09-10T09:09:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:09:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35375.852868 secs 2023-09-10T09:09:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35376.205760 secs 2023-09-10T09:09:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:09:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35375.852868 secs 2023-09-10T09:09:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35376.205760 secs 2023-09-10T09:09:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:09:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:05 smithi098 ceph-mon[103124]: pgmap v19025: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 741 KiB/s wr, 13 op/s 2023-09-10T09:09:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:09:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:05 smithi170 ceph-mon[102832]: pgmap v19025: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 741 KiB/s wr, 13 op/s 2023-09-10T09:09:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:09:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:05 smithi082 ceph-mon[98694]: pgmap v19025: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 741 KiB/s wr, 13 op/s 2023-09-10T09:09:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:09:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:07 smithi098 ceph-mon[103124]: pgmap v19026: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 909 KiB/s wr, 34 op/s 2023-09-10T09:09:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:07 smithi170 ceph-mon[102832]: pgmap v19026: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 909 KiB/s wr, 34 op/s 2023-09-10T09:09:08.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:07 smithi082 ceph-mon[98694]: pgmap v19026: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 909 KiB/s wr, 34 op/s 2023-09-10T09:09:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35379.602926 secs 2023-09-10T09:09:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:08 smithi098 ceph-mon[103124]: osdmap e1257: 12 total, 12 up, 12 in 2023-09-10T09:09:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35379.602926 secs 2023-09-10T09:09:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:08 smithi170 ceph-mon[102832]: osdmap e1257: 12 total, 12 up, 12 in 2023-09-10T09:09:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35379.602926 secs 2023-09-10T09:09:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:08 smithi082 ceph-mon[98694]: osdmap e1257: 12 total, 12 up, 12 in 2023-09-10T09:09:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35380.852948 secs 2023-09-10T09:09:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:09 smithi098 ceph-mon[103124]: pgmap v19028: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.0 MiB/s wr, 34 op/s 2023-09-10T09:09:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35381.205825 secs 2023-09-10T09:09:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35380.852948 secs 2023-09-10T09:09:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:09 smithi170 ceph-mon[102832]: pgmap v19028: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.0 MiB/s wr, 34 op/s 2023-09-10T09:09:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35381.205825 secs 2023-09-10T09:09:10.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35380.852948 secs 2023-09-10T09:09:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:09 smithi082 ceph-mon[98694]: pgmap v19028: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.0 MiB/s wr, 34 op/s 2023-09-10T09:09:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35381.205825 secs 2023-09-10T09:09:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:09:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:09:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:10 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:09:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:09:11.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:09:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:10 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:09:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:09:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:09:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:10 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:09:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:11 smithi098 ceph-mon[103124]: pgmap v19029: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 416 KiB/s wr, 32 op/s 2023-09-10T09:09:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:09:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:09:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:09:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:11 smithi170 ceph-mon[102832]: pgmap v19029: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 416 KiB/s wr, 32 op/s 2023-09-10T09:09:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:09:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:09:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:09:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:11 smithi082 ceph-mon[98694]: pgmap v19029: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 416 KiB/s wr, 32 op/s 2023-09-10T09:09:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:09:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:09:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:09:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:13 smithi098 ceph-mon[103124]: pgmap v19030: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 416 KiB/s wr, 32 op/s 2023-09-10T09:09:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35384.602986 secs 2023-09-10T09:09:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:13 smithi098 ceph-mon[103124]: osdmap e1258: 12 total, 12 up, 12 in 2023-09-10T09:09:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:13 smithi170 ceph-mon[102832]: pgmap v19030: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 416 KiB/s wr, 32 op/s 2023-09-10T09:09:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35384.602986 secs 2023-09-10T09:09:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:13 smithi170 ceph-mon[102832]: osdmap e1258: 12 total, 12 up, 12 in 2023-09-10T09:09:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:13 smithi082 ceph-mon[98694]: pgmap v19030: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 416 KiB/s wr, 32 op/s 2023-09-10T09:09:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35384.602986 secs 2023-09-10T09:09:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:13 smithi082 ceph-mon[98694]: osdmap e1258: 12 total, 12 up, 12 in 2023-09-10T09:09:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35385.853006 secs 2023-09-10T09:09:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35386.205894 secs 2023-09-10T09:09:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35385.853006 secs 2023-09-10T09:09:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35386.205894 secs 2023-09-10T09:09:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35385.853006 secs 2023-09-10T09:09:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35386.205894 secs 2023-09-10T09:09:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:16 smithi098 ceph-mon[103124]: pgmap v19032: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 269 KiB/s wr, 9 op/s 2023-09-10T09:09:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:16 smithi170 ceph-mon[102832]: pgmap v19032: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 269 KiB/s wr, 9 op/s 2023-09-10T09:09:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:16 smithi082 ceph-mon[98694]: pgmap v19032: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 269 KiB/s wr, 9 op/s 2023-09-10T09:09:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:17 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:09:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:17 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:17 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:09:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:17 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:17 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:09:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:17 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:18 smithi170 ceph-mon[102832]: pgmap v19033: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 380 KiB/s wr, 15 op/s 2023-09-10T09:09:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:09:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35389.603048 secs 2023-09-10T09:09:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:18 smithi082 ceph-mon[98694]: pgmap v19033: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 380 KiB/s wr, 15 op/s 2023-09-10T09:09:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:09:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35389.603048 secs 2023-09-10T09:09:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:18 smithi098 ceph-mon[103124]: pgmap v19033: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 380 KiB/s wr, 15 op/s 2023-09-10T09:09:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:09:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35389.603048 secs 2023-09-10T09:09:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35390.853066 secs 2023-09-10T09:09:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:19 smithi170 ceph-mon[102832]: pgmap v19034: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 337 KiB/s wr, 13 op/s 2023-09-10T09:09:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35391.205957 secs 2023-09-10T09:09:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35390.853066 secs 2023-09-10T09:09:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:19 smithi082 ceph-mon[98694]: pgmap v19034: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 337 KiB/s wr, 13 op/s 2023-09-10T09:09:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35391.205957 secs 2023-09-10T09:09:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35390.853066 secs 2023-09-10T09:09:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:19 smithi098 ceph-mon[103124]: pgmap v19034: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 337 KiB/s wr, 13 op/s 2023-09-10T09:09:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35391.205957 secs 2023-09-10T09:09:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:21 smithi170 ceph-mon[102832]: pgmap v19035: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 154 KiB/s wr, 13 op/s 2023-09-10T09:09:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:21 smithi082 ceph-mon[98694]: pgmap v19035: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 154 KiB/s wr, 13 op/s 2023-09-10T09:09:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:21 smithi098 ceph-mon[103124]: pgmap v19035: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 154 KiB/s wr, 13 op/s 2023-09-10T09:09:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:23 smithi170 ceph-mon[102832]: pgmap v19036: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 154 KiB/s wr, 13 op/s 2023-09-10T09:09:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35394.603108 secs 2023-09-10T09:09:24.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:23 smithi082 ceph-mon[98694]: pgmap v19036: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 154 KiB/s wr, 13 op/s 2023-09-10T09:09:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35394.603108 secs 2023-09-10T09:09:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:23 smithi098 ceph-mon[103124]: pgmap v19036: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 154 KiB/s wr, 13 op/s 2023-09-10T09:09:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35394.603108 secs 2023-09-10T09:09:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35395.853128 secs 2023-09-10T09:09:25.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35396.206025 secs 2023-09-10T09:09:25.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35395.853128 secs 2023-09-10T09:09:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35396.206025 secs 2023-09-10T09:09:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35395.853128 secs 2023-09-10T09:09:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35396.206025 secs 2023-09-10T09:09:26.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:25 smithi170 ceph-mon[102832]: pgmap v19037: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T09:09:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:25 smithi082 ceph-mon[98694]: pgmap v19037: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T09:09:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:25 smithi098 ceph-mon[103124]: pgmap v19037: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 12 op/s 2023-09-10T09:09:28.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:27 smithi170 ceph-mon[102832]: pgmap v19038: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 129 KiB/s wr, 16 op/s 2023-09-10T09:09:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:27 smithi082 ceph-mon[98694]: pgmap v19038: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 129 KiB/s wr, 16 op/s 2023-09-10T09:09:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:27 smithi098 ceph-mon[103124]: pgmap v19038: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 129 KiB/s wr, 16 op/s 2023-09-10T09:09:29.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35399.603168 secs 2023-09-10T09:09:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35399.603168 secs 2023-09-10T09:09:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35399.603168 secs 2023-09-10T09:09:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35400.853187 secs 2023-09-10T09:09:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:29 smithi170 ceph-mon[102832]: pgmap v19039: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T09:09:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35401.206092 secs 2023-09-10T09:09:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35400.853187 secs 2023-09-10T09:09:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:29 smithi082 ceph-mon[98694]: pgmap v19039: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T09:09:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35401.206092 secs 2023-09-10T09:09:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35400.853187 secs 2023-09-10T09:09:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:29 smithi098 ceph-mon[103124]: pgmap v19039: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T09:09:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35401.206092 secs 2023-09-10T09:09:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:31 smithi170 ceph-mon[102832]: pgmap v19040: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 156 KiB/s wr, 16 op/s 2023-09-10T09:09:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:31 smithi082 ceph-mon[98694]: pgmap v19040: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 156 KiB/s wr, 16 op/s 2023-09-10T09:09:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:31 smithi098 ceph-mon[103124]: pgmap v19040: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 156 KiB/s wr, 16 op/s 2023-09-10T09:09:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:33 smithi170 ceph-mon[102832]: pgmap v19041: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:09:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:33 smithi082 ceph-mon[98694]: pgmap v19041: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:09:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:33 smithi098 ceph-mon[103124]: pgmap v19041: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:09:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35405.853247 secs 2023-09-10T09:09:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35406.206158 secs 2023-09-10T09:09:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35405.853247 secs 2023-09-10T09:09:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35406.206158 secs 2023-09-10T09:09:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35405.853247 secs 2023-09-10T09:09:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35406.206158 secs 2023-09-10T09:09:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:35 smithi170 ceph-mon[102832]: pgmap v19042: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:09:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:09:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:35 smithi082 ceph-mon[98694]: pgmap v19042: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:09:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:09:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:35 smithi098 ceph-mon[103124]: pgmap v19042: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:09:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:09:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:09:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:09:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:09:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:09:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:09:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:09:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:37 smithi082 ceph-mon[98694]: pgmap v19043: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 252 KiB/s wr, 16 op/s 2023-09-10T09:09:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:09:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:37 smithi098 ceph-mon[103124]: pgmap v19043: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 252 KiB/s wr, 16 op/s 2023-09-10T09:09:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:09:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:37 smithi170 ceph-mon[102832]: pgmap v19043: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 252 KiB/s wr, 16 op/s 2023-09-10T09:09:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:09:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35410.853308 secs 2023-09-10T09:09:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:39 smithi170 ceph-mon[102832]: pgmap v19044: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 252 KiB/s wr, 11 op/s 2023-09-10T09:09:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35411.206222 secs 2023-09-10T09:09:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35410.853308 secs 2023-09-10T09:09:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:39 smithi082 ceph-mon[98694]: pgmap v19044: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 252 KiB/s wr, 11 op/s 2023-09-10T09:09:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35411.206222 secs 2023-09-10T09:09:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35410.853308 secs 2023-09-10T09:09:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:39 smithi098 ceph-mon[103124]: pgmap v19044: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 252 KiB/s wr, 11 op/s 2023-09-10T09:09:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35411.206222 secs 2023-09-10T09:09:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:09:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:09:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:09:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:41 smithi170 ceph-mon[102832]: pgmap v19045: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 926 KiB/s wr, 19 op/s 2023-09-10T09:09:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:41 smithi082 ceph-mon[98694]: pgmap v19045: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 926 KiB/s wr, 19 op/s 2023-09-10T09:09:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:41 smithi098 ceph-mon[103124]: pgmap v19045: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 926 KiB/s wr, 19 op/s 2023-09-10T09:09:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:43 smithi170 ceph-mon[102832]: pgmap v19046: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 798 KiB/s wr, 13 op/s 2023-09-10T09:09:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:43 smithi082 ceph-mon[98694]: pgmap v19046: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 798 KiB/s wr, 13 op/s 2023-09-10T09:09:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:43 smithi098 ceph-mon[103124]: pgmap v19046: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 798 KiB/s wr, 13 op/s 2023-09-10T09:09:45.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35415.853366 secs 2023-09-10T09:09:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35416.206290 secs 2023-09-10T09:09:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35415.853366 secs 2023-09-10T09:09:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35416.206290 secs 2023-09-10T09:09:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35415.853366 secs 2023-09-10T09:09:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35416.206290 secs 2023-09-10T09:09:46.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:45 smithi170 ceph-mon[102832]: pgmap v19047: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 798 KiB/s wr, 13 op/s 2023-09-10T09:09:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:45 smithi082 ceph-mon[98694]: pgmap v19047: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 798 KiB/s wr, 13 op/s 2023-09-10T09:09:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:45 smithi098 ceph-mon[103124]: pgmap v19047: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 798 KiB/s wr, 13 op/s 2023-09-10T09:09:48.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:47 smithi170 ceph-mon[102832]: pgmap v19048: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 826 KiB/s wr, 19 op/s 2023-09-10T09:09:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:47 smithi082 ceph-mon[98694]: pgmap v19048: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 826 KiB/s wr, 19 op/s 2023-09-10T09:09:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:47 smithi098 ceph-mon[103124]: pgmap v19048: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 826 KiB/s wr, 19 op/s 2023-09-10T09:09:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35420.853425 secs 2023-09-10T09:09:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:49 smithi170 ceph-mon[102832]: pgmap v19049: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 702 KiB/s wr, 13 op/s 2023-09-10T09:09:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35421.206355 secs 2023-09-10T09:09:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35420.853425 secs 2023-09-10T09:09:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:49 smithi082 ceph-mon[98694]: pgmap v19049: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 702 KiB/s wr, 13 op/s 2023-09-10T09:09:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35421.206355 secs 2023-09-10T09:09:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35420.853425 secs 2023-09-10T09:09:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:49 smithi098 ceph-mon[103124]: pgmap v19049: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 702 KiB/s wr, 13 op/s 2023-09-10T09:09:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35421.206355 secs 2023-09-10T09:09:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:09:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:50.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:09:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:09:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:51 smithi170 ceph-mon[102832]: pgmap v19050: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:09:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:51 smithi082 ceph-mon[98694]: pgmap v19050: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:09:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:51 smithi098 ceph-mon[103124]: pgmap v19050: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:09:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:09:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:09:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:09:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:53 smithi170 ceph-mon[102832]: pgmap v19051: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 471 KiB/s wr, 13 op/s 2023-09-10T09:09:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35424.603590 secs 2023-09-10T09:09:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:53 smithi082 ceph-mon[98694]: pgmap v19051: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 471 KiB/s wr, 13 op/s 2023-09-10T09:09:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35424.603590 secs 2023-09-10T09:09:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:53 smithi098 ceph-mon[103124]: pgmap v19051: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 471 KiB/s wr, 13 op/s 2023-09-10T09:09:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35424.603590 secs 2023-09-10T09:09:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35425.853509 secs 2023-09-10T09:09:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:09:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35426.206420 secs 2023-09-10T09:09:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35425.853509 secs 2023-09-10T09:09:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:09:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35426.206420 secs 2023-09-10T09:09:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35425.853509 secs 2023-09-10T09:09:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:09:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35426.206420 secs 2023-09-10T09:09:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:55 smithi170 ceph-mon[102832]: pgmap v19052: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 471 KiB/s wr, 13 op/s 2023-09-10T09:09:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:55 smithi082 ceph-mon[98694]: pgmap v19052: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 471 KiB/s wr, 13 op/s 2023-09-10T09:09:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:55 smithi098 ceph-mon[103124]: pgmap v19052: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 471 KiB/s wr, 13 op/s 2023-09-10T09:09:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:09:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:09:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:09:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:09:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:09:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:09:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:09:57.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:57 smithi170 ceph-mon[102832]: pgmap v19053: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 959 KiB/s wr, 21 op/s 2023-09-10T09:09:57.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:57 smithi082 ceph-mon[98694]: pgmap v19053: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 959 KiB/s wr, 21 op/s 2023-09-10T09:09:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:57 smithi098 ceph-mon[103124]: pgmap v19053: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 959 KiB/s wr, 21 op/s 2023-09-10T09:09:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35429.603651 secs 2023-09-10T09:09:58.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35429.603651 secs 2023-09-10T09:09:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35429.603651 secs 2023-09-10T09:09:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35430.853568 secs 2023-09-10T09:09:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:59 smithi170 ceph-mon[102832]: pgmap v19054: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 931 KiB/s wr, 15 op/s 2023-09-10T09:09:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:09:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35431.206488 secs 2023-09-10T09:09:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35430.853568 secs 2023-09-10T09:09:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:59 smithi082 ceph-mon[98694]: pgmap v19054: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 931 KiB/s wr, 15 op/s 2023-09-10T09:09:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:09:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35431.206488 secs 2023-09-10T09:10:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35430.853568 secs 2023-09-10T09:10:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:59 smithi098 ceph-mon[103124]: pgmap v19054: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 931 KiB/s wr, 15 op/s 2023-09-10T09:10:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:09:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35431.206488 secs 2023-09-10T09:10:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T09:10:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T09:10:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T09:10:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:01 smithi170 ceph-mon[102832]: pgmap v19055: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:10:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:01 smithi082 ceph-mon[98694]: pgmap v19055: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:10:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:01 smithi098 ceph-mon[103124]: pgmap v19055: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:10:04.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:03 smithi170 ceph-mon[102832]: pgmap v19056: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 653 KiB/s wr, 14 op/s 2023-09-10T09:10:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35434.603714 secs 2023-09-10T09:10:04.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:03 smithi082 ceph-mon[98694]: pgmap v19056: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 653 KiB/s wr, 14 op/s 2023-09-10T09:10:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35434.603714 secs 2023-09-10T09:10:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:03 smithi098 ceph-mon[103124]: pgmap v19056: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 653 KiB/s wr, 14 op/s 2023-09-10T09:10:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35434.603714 secs 2023-09-10T09:10:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35435.853652 secs 2023-09-10T09:10:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:10:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35436.206553 secs 2023-09-10T09:10:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35435.853652 secs 2023-09-10T09:10:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:10:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35436.206553 secs 2023-09-10T09:10:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35435.853652 secs 2023-09-10T09:10:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:10:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35436.206553 secs 2023-09-10T09:10:06.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:05 smithi170 ceph-mon[102832]: pgmap v19057: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 653 KiB/s wr, 14 op/s 2023-09-10T09:10:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:10:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:05 smithi082 ceph-mon[98694]: pgmap v19057: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 653 KiB/s wr, 14 op/s 2023-09-10T09:10:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:10:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:05 smithi098 ceph-mon[103124]: pgmap v19057: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 653 KiB/s wr, 14 op/s 2023-09-10T09:10:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:10:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:07 smithi082 ceph-mon[98694]: pgmap v19058: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 851 KiB/s wr, 34 op/s 2023-09-10T09:10:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:07 smithi098 ceph-mon[103124]: pgmap v19058: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 851 KiB/s wr, 34 op/s 2023-09-10T09:10:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:07 smithi170 ceph-mon[102832]: pgmap v19058: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 851 KiB/s wr, 34 op/s 2023-09-10T09:10:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35439.603772 secs 2023-09-10T09:10:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:08 smithi082 ceph-mon[98694]: osdmap e1259: 12 total, 12 up, 12 in 2023-09-10T09:10:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35439.603772 secs 2023-09-10T09:10:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:08 smithi098 ceph-mon[103124]: osdmap e1259: 12 total, 12 up, 12 in 2023-09-10T09:10:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35439.603772 secs 2023-09-10T09:10:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:08 smithi170 ceph-mon[102832]: osdmap e1259: 12 total, 12 up, 12 in 2023-09-10T09:10:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35440.853737 secs 2023-09-10T09:10:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:09 smithi082 ceph-mon[98694]: pgmap v19060: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 435 KiB/s wr, 31 op/s 2023-09-10T09:10:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35441.206623 secs 2023-09-10T09:10:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35440.853737 secs 2023-09-10T09:10:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:09 smithi098 ceph-mon[103124]: pgmap v19060: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 435 KiB/s wr, 31 op/s 2023-09-10T09:10:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35441.206623 secs 2023-09-10T09:10:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35440.853737 secs 2023-09-10T09:10:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:09 smithi170 ceph-mon[102832]: pgmap v19060: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 435 KiB/s wr, 31 op/s 2023-09-10T09:10:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35441.206623 secs 2023-09-10T09:10:12.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:11 smithi170 ceph-mon[102832]: pgmap v19061: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 392 KiB/s wr, 31 op/s 2023-09-10T09:10:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:10:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:10:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:11 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:10:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:11 smithi082 ceph-mon[98694]: pgmap v19061: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 392 KiB/s wr, 31 op/s 2023-09-10T09:10:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:10:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:10:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:11 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:10:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:11 smithi098 ceph-mon[103124]: pgmap v19061: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 392 KiB/s wr, 31 op/s 2023-09-10T09:10:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:10:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:10:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:11 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:10:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:10:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:10:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:10:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:10:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:10:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:10:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:10:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:10:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:10:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:10:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:10:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:10:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:10:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:10:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:10:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:13 smithi098 ceph-mon[103124]: pgmap v19062: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 392 KiB/s wr, 31 op/s 2023-09-10T09:10:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35444.603854 secs 2023-09-10T09:10:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:13 smithi098 ceph-mon[103124]: osdmap e1260: 12 total, 12 up, 12 in 2023-09-10T09:10:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:13 smithi170 ceph-mon[102832]: pgmap v19062: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 392 KiB/s wr, 31 op/s 2023-09-10T09:10:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35444.603854 secs 2023-09-10T09:10:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:13 smithi170 ceph-mon[102832]: osdmap e1260: 12 total, 12 up, 12 in 2023-09-10T09:10:14.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:13 smithi082 ceph-mon[98694]: pgmap v19062: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 392 KiB/s wr, 31 op/s 2023-09-10T09:10:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35444.603854 secs 2023-09-10T09:10:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:13 smithi082 ceph-mon[98694]: osdmap e1260: 12 total, 12 up, 12 in 2023-09-10T09:10:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35445.853798 secs 2023-09-10T09:10:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35446.206689 secs 2023-09-10T09:10:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35445.853798 secs 2023-09-10T09:10:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35446.206689 secs 2023-09-10T09:10:15.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35445.853798 secs 2023-09-10T09:10:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35446.206689 secs 2023-09-10T09:10:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:15 smithi098 ceph-mon[103124]: pgmap v19064: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 9 op/s 2023-09-10T09:10:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:15 smithi170 ceph-mon[102832]: pgmap v19064: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 9 op/s 2023-09-10T09:10:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:15 smithi082 ceph-mon[98694]: pgmap v19064: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 9 op/s 2023-09-10T09:10:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:17 smithi098 ceph-mon[103124]: pgmap v19065: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 209 KiB/s wr, 16 op/s 2023-09-10T09:10:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:17 smithi170 ceph-mon[102832]: pgmap v19065: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 209 KiB/s wr, 16 op/s 2023-09-10T09:10:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:17 smithi082 ceph-mon[98694]: pgmap v19065: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 209 KiB/s wr, 16 op/s 2023-09-10T09:10:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35449.603950 secs 2023-09-10T09:10:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35449.603950 secs 2023-09-10T09:10:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35449.603950 secs 2023-09-10T09:10:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35450.853885 secs 2023-09-10T09:10:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:20 smithi098 ceph-mon[103124]: pgmap v19066: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 14 op/s 2023-09-10T09:10:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35451.206763 secs 2023-09-10T09:10:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35450.853885 secs 2023-09-10T09:10:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:20 smithi170 ceph-mon[102832]: pgmap v19066: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 14 op/s 2023-09-10T09:10:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35451.206763 secs 2023-09-10T09:10:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35450.853885 secs 2023-09-10T09:10:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:20 smithi082 ceph-mon[98694]: pgmap v19066: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 14 op/s 2023-09-10T09:10:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35451.206763 secs 2023-09-10T09:10:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:22 smithi098 ceph-mon[103124]: pgmap v19067: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 14 op/s 2023-09-10T09:10:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:22 smithi170 ceph-mon[102832]: pgmap v19067: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 14 op/s 2023-09-10T09:10:22.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:22 smithi082 ceph-mon[98694]: pgmap v19067: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 14 op/s 2023-09-10T09:10:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35454.604009 secs 2023-09-10T09:10:23.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35454.604009 secs 2023-09-10T09:10:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35454.604009 secs 2023-09-10T09:10:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:24 smithi170 ceph-mon[102832]: pgmap v19068: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 14 op/s 2023-09-10T09:10:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:24 smithi082 ceph-mon[98694]: pgmap v19068: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 14 op/s 2023-09-10T09:10:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:24 smithi098 ceph-mon[103124]: pgmap v19068: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 14 op/s 2023-09-10T09:10:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35455.853945 secs 2023-09-10T09:10:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35456.206827 secs 2023-09-10T09:10:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:25 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:10:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:25 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35455.853945 secs 2023-09-10T09:10:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35456.206827 secs 2023-09-10T09:10:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:25 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:10:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:25 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35455.853945 secs 2023-09-10T09:10:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35456.206827 secs 2023-09-10T09:10:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:25 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:10:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:25 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:26 smithi170 ceph-mon[102832]: pgmap v19069: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T09:10:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:26 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:10:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:26.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:26 smithi082 ceph-mon[98694]: pgmap v19069: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T09:10:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:26 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:10:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:26 smithi098 ceph-mon[103124]: pgmap v19069: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T09:10:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:26 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:10:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:28 smithi170 ceph-mon[102832]: pgmap v19070: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 258 KiB/s wr, 17 op/s 2023-09-10T09:10:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35459.604070 secs 2023-09-10T09:10:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:28 smithi082 ceph-mon[98694]: pgmap v19070: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 258 KiB/s wr, 17 op/s 2023-09-10T09:10:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35459.604070 secs 2023-09-10T09:10:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:28 smithi098 ceph-mon[103124]: pgmap v19070: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 258 KiB/s wr, 17 op/s 2023-09-10T09:10:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35459.604070 secs 2023-09-10T09:10:29.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35460.854005 secs 2023-09-10T09:10:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:29 smithi170 ceph-mon[102832]: pgmap v19071: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T09:10:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35461.206894 secs 2023-09-10T09:10:29.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35460.854005 secs 2023-09-10T09:10:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:29 smithi082 ceph-mon[98694]: pgmap v19071: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T09:10:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35461.206894 secs 2023-09-10T09:10:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35460.854005 secs 2023-09-10T09:10:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:29 smithi098 ceph-mon[103124]: pgmap v19071: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 230 KiB/s wr, 11 op/s 2023-09-10T09:10:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35461.206894 secs 2023-09-10T09:10:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:31 smithi170 ceph-mon[102832]: pgmap v19072: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 230 KiB/s wr, 16 op/s 2023-09-10T09:10:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:31 smithi082 ceph-mon[98694]: pgmap v19072: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 230 KiB/s wr, 16 op/s 2023-09-10T09:10:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:31 smithi098 ceph-mon[103124]: pgmap v19072: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 230 KiB/s wr, 16 op/s 2023-09-10T09:10:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:33 smithi170 ceph-mon[102832]: pgmap v19073: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:10:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:33 smithi082 ceph-mon[98694]: pgmap v19073: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:10:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:33 smithi098 ceph-mon[103124]: pgmap v19073: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:10:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35465.854063 secs 2023-09-10T09:10:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35466.206963 secs 2023-09-10T09:10:35.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35465.854063 secs 2023-09-10T09:10:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35466.206963 secs 2023-09-10T09:10:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35465.854063 secs 2023-09-10T09:10:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35466.206963 secs 2023-09-10T09:10:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:35 smithi170 ceph-mon[102832]: pgmap v19074: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:10:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:10:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:35 smithi082 ceph-mon[98694]: pgmap v19074: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:10:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:10:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:35 smithi098 ceph-mon[103124]: pgmap v19074: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 11 op/s 2023-09-10T09:10:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:10:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:10:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:10:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:10:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:10:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:10:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:10:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:37 smithi082 ceph-mon[98694]: pgmap v19075: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 337 KiB/s wr, 17 op/s 2023-09-10T09:10:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:37 smithi098 ceph-mon[103124]: pgmap v19075: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 337 KiB/s wr, 17 op/s 2023-09-10T09:10:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:37 smithi170 ceph-mon[102832]: pgmap v19075: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 337 KiB/s wr, 17 op/s 2023-09-10T09:10:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35470.854124 secs 2023-09-10T09:10:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:39 smithi170 ceph-mon[102832]: pgmap v19076: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 209 KiB/s wr, 11 op/s 2023-09-10T09:10:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35471.207029 secs 2023-09-10T09:10:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35470.854124 secs 2023-09-10T09:10:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:39 smithi082 ceph-mon[98694]: pgmap v19076: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 209 KiB/s wr, 11 op/s 2023-09-10T09:10:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35471.207029 secs 2023-09-10T09:10:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35470.854124 secs 2023-09-10T09:10:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:39 smithi098 ceph-mon[103124]: pgmap v19076: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 209 KiB/s wr, 11 op/s 2023-09-10T09:10:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35471.207029 secs 2023-09-10T09:10:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:41 smithi170 ceph-mon[102832]: pgmap v19077: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 427 KiB/s wr, 18 op/s 2023-09-10T09:10:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:41 smithi082 ceph-mon[98694]: pgmap v19077: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 427 KiB/s wr, 18 op/s 2023-09-10T09:10:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:41 smithi098 ceph-mon[103124]: pgmap v19077: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 427 KiB/s wr, 18 op/s 2023-09-10T09:10:44.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:43 smithi170 ceph-mon[102832]: pgmap v19078: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 427 KiB/s wr, 13 op/s 2023-09-10T09:10:44.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:43 smithi082 ceph-mon[98694]: pgmap v19078: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 427 KiB/s wr, 13 op/s 2023-09-10T09:10:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:43 smithi098 ceph-mon[103124]: pgmap v19078: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 427 KiB/s wr, 13 op/s 2023-09-10T09:10:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35475.854200 secs 2023-09-10T09:10:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:10:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35476.207092 secs 2023-09-10T09:10:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35475.854200 secs 2023-09-10T09:10:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:10:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35476.207092 secs 2023-09-10T09:10:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35475.854200 secs 2023-09-10T09:10:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:10:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35476.207092 secs 2023-09-10T09:10:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:45 smithi082 ceph-mon[98694]: pgmap v19079: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 427 KiB/s wr, 13 op/s 2023-09-10T09:10:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:10:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:45 smithi098 ceph-mon[103124]: pgmap v19079: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 427 KiB/s wr, 13 op/s 2023-09-10T09:10:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:10:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:45 smithi170 ceph-mon[102832]: pgmap v19079: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 427 KiB/s wr, 13 op/s 2023-09-10T09:10:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:10:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:47 smithi082 ceph-mon[98694]: pgmap v19080: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 605 KiB/s wr, 19 op/s 2023-09-10T09:10:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:47 smithi098 ceph-mon[103124]: pgmap v19080: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 605 KiB/s wr, 19 op/s 2023-09-10T09:10:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:47 smithi170 ceph-mon[102832]: pgmap v19080: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 605 KiB/s wr, 19 op/s 2023-09-10T09:10:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35480.854260 secs 2023-09-10T09:10:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:49 smithi170 ceph-mon[102832]: pgmap v19081: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 396 KiB/s wr, 13 op/s 2023-09-10T09:10:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35481.207157 secs 2023-09-10T09:10:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35480.854260 secs 2023-09-10T09:10:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:49 smithi082 ceph-mon[98694]: pgmap v19081: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 396 KiB/s wr, 13 op/s 2023-09-10T09:10:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35481.207157 secs 2023-09-10T09:10:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35480.854260 secs 2023-09-10T09:10:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:49 smithi098 ceph-mon[103124]: pgmap v19081: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 396 KiB/s wr, 13 op/s 2023-09-10T09:10:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35481.207157 secs 2023-09-10T09:10:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:10:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:10:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:10:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:51 smithi170 ceph-mon[102832]: pgmap v19082: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 765 KiB/s wr, 21 op/s 2023-09-10T09:10:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:51 smithi082 ceph-mon[98694]: pgmap v19082: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 765 KiB/s wr, 21 op/s 2023-09-10T09:10:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:51 smithi098 ceph-mon[103124]: pgmap v19082: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 765 KiB/s wr, 21 op/s 2023-09-10T09:10:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:10:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:10:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:10:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:53 smithi170 ceph-mon[102832]: pgmap v19083: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 546 KiB/s wr, 14 op/s 2023-09-10T09:10:53.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:53 smithi082 ceph-mon[98694]: pgmap v19083: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 546 KiB/s wr, 14 op/s 2023-09-10T09:10:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:53 smithi098 ceph-mon[103124]: pgmap v19083: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 546 KiB/s wr, 14 op/s 2023-09-10T09:10:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35485.854343 secs 2023-09-10T09:10:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35486.207223 secs 2023-09-10T09:10:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35485.854343 secs 2023-09-10T09:10:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35486.207223 secs 2023-09-10T09:10:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35485.854343 secs 2023-09-10T09:10:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35486.207223 secs 2023-09-10T09:10:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:55 smithi170 ceph-mon[102832]: pgmap v19084: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 546 KiB/s wr, 14 op/s 2023-09-10T09:10:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:55 smithi082 ceph-mon[98694]: pgmap v19084: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 546 KiB/s wr, 14 op/s 2023-09-10T09:10:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:55 smithi098 ceph-mon[103124]: pgmap v19084: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 546 KiB/s wr, 14 op/s 2023-09-10T09:10:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:57 smithi170 ceph-mon[102832]: pgmap v19085: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 709 KiB/s wr, 21 op/s 2023-09-10T09:10:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:10:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:57 smithi082 ceph-mon[98694]: pgmap v19085: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 709 KiB/s wr, 21 op/s 2023-09-10T09:10:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:10:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:57 smithi098 ceph-mon[103124]: pgmap v19085: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 709 KiB/s wr, 21 op/s 2023-09-10T09:10:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:10:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:10:59.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35490.854402 secs 2023-09-10T09:10:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:59 smithi170 ceph-mon[102832]: pgmap v19086: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 532 KiB/s wr, 14 op/s 2023-09-10T09:10:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:10:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35491.207291 secs 2023-09-10T09:10:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35490.854402 secs 2023-09-10T09:10:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:59 smithi082 ceph-mon[98694]: pgmap v19086: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 532 KiB/s wr, 14 op/s 2023-09-10T09:10:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:10:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35491.207291 secs 2023-09-10T09:11:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35490.854402 secs 2023-09-10T09:11:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:59 smithi098 ceph-mon[103124]: pgmap v19086: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 532 KiB/s wr, 14 op/s 2023-09-10T09:11:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:10:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35491.207291 secs 2023-09-10T09:11:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:11:00.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:00.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:11:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:11:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:01.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:01 smithi170 ceph-mon[102832]: pgmap v19087: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:11:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:01 smithi082 ceph-mon[98694]: pgmap v19087: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:11:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:01 smithi098 ceph-mon[103124]: pgmap v19087: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:11:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:03 smithi082 ceph-mon[98694]: pgmap v19088: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 724 KiB/s wr, 14 op/s 2023-09-10T09:11:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35494.604588 secs 2023-09-10T09:11:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:03 smithi098 ceph-mon[103124]: pgmap v19088: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 724 KiB/s wr, 14 op/s 2023-09-10T09:11:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35494.604588 secs 2023-09-10T09:11:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:03 smithi170 ceph-mon[102832]: pgmap v19088: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 724 KiB/s wr, 14 op/s 2023-09-10T09:11:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35494.604588 secs 2023-09-10T09:11:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35495.854491 secs 2023-09-10T09:11:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:11:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:11:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35496.207356 secs 2023-09-10T09:11:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35495.854491 secs 2023-09-10T09:11:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:11:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:11:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35496.207356 secs 2023-09-10T09:11:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35495.854491 secs 2023-09-10T09:11:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:11:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:11:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35496.207356 secs 2023-09-10T09:11:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:05 smithi082 ceph-mon[98694]: pgmap v19089: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 724 KiB/s wr, 14 op/s 2023-09-10T09:11:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:05 smithi098 ceph-mon[103124]: pgmap v19089: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 724 KiB/s wr, 14 op/s 2023-09-10T09:11:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:05 smithi170 ceph-mon[102832]: pgmap v19089: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 724 KiB/s wr, 14 op/s 2023-09-10T09:11:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:07 smithi082 ceph-mon[98694]: pgmap v19090: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 914 KiB/s wr, 35 op/s 2023-09-10T09:11:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:07 smithi098 ceph-mon[103124]: pgmap v19090: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 914 KiB/s wr, 35 op/s 2023-09-10T09:11:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:07 smithi170 ceph-mon[102832]: pgmap v19090: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 914 KiB/s wr, 35 op/s 2023-09-10T09:11:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35499.604649 secs 2023-09-10T09:11:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:08 smithi082 ceph-mon[98694]: osdmap e1261: 12 total, 12 up, 12 in 2023-09-10T09:11:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35499.604649 secs 2023-09-10T09:11:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:08 smithi098 ceph-mon[103124]: osdmap e1261: 12 total, 12 up, 12 in 2023-09-10T09:11:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35499.604649 secs 2023-09-10T09:11:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:08 smithi170 ceph-mon[102832]: osdmap e1261: 12 total, 12 up, 12 in 2023-09-10T09:11:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35500.854575 secs 2023-09-10T09:11:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:09 smithi082 ceph-mon[98694]: pgmap v19092: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 901 KiB/s wr, 34 op/s 2023-09-10T09:11:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35501.207423 secs 2023-09-10T09:11:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:09 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:11:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:09 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35500.854575 secs 2023-09-10T09:11:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:09 smithi098 ceph-mon[103124]: pgmap v19092: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 901 KiB/s wr, 34 op/s 2023-09-10T09:11:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35501.207423 secs 2023-09-10T09:11:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:09 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:11:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:09 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35500.854575 secs 2023-09-10T09:11:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:09 smithi170 ceph-mon[102832]: pgmap v19092: 97 pgs: 97 active+clean; 12 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 901 KiB/s wr, 34 op/s 2023-09-10T09:11:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35501.207423 secs 2023-09-10T09:11:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:09 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:11:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:09 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:11 smithi082 ceph-mon[98694]: pgmap v19093: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 444 KiB/s wr, 33 op/s 2023-09-10T09:11:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:11 smithi098 ceph-mon[103124]: pgmap v19093: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 444 KiB/s wr, 33 op/s 2023-09-10T09:11:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:11 smithi170 ceph-mon[102832]: pgmap v19093: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 444 KiB/s wr, 33 op/s 2023-09-10T09:11:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:11:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:11:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:11:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:11:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:11:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:11:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:13 smithi098 ceph-mon[103124]: pgmap v19094: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 444 KiB/s wr, 33 op/s 2023-09-10T09:11:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35504.604709 secs 2023-09-10T09:11:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:11:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:11:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:11:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:13 smithi098 ceph-mon[103124]: osdmap e1262: 12 total, 12 up, 12 in 2023-09-10T09:11:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:13 smithi170 ceph-mon[102832]: pgmap v19094: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 444 KiB/s wr, 33 op/s 2023-09-10T09:11:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35504.604709 secs 2023-09-10T09:11:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:11:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:11:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:11:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:13 smithi170 ceph-mon[102832]: osdmap e1262: 12 total, 12 up, 12 in 2023-09-10T09:11:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:13 smithi082 ceph-mon[98694]: pgmap v19094: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 444 KiB/s wr, 33 op/s 2023-09-10T09:11:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35504.604709 secs 2023-09-10T09:11:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:11:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:11:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:11:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:13 smithi082 ceph-mon[98694]: osdmap e1262: 12 total, 12 up, 12 in 2023-09-10T09:11:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35505.854633 secs 2023-09-10T09:11:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35506.207490 secs 2023-09-10T09:11:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35505.854633 secs 2023-09-10T09:11:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35506.207490 secs 2023-09-10T09:11:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35505.854633 secs 2023-09-10T09:11:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35506.207490 secs 2023-09-10T09:11:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:15 smithi098 ceph-mon[103124]: pgmap v19096: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 270 KiB/s wr, 10 op/s 2023-09-10T09:11:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:15 smithi170 ceph-mon[102832]: pgmap v19096: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 270 KiB/s wr, 10 op/s 2023-09-10T09:11:16.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:15 smithi082 ceph-mon[98694]: pgmap v19096: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 270 KiB/s wr, 10 op/s 2023-09-10T09:11:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:17 smithi098 ceph-mon[103124]: pgmap v19097: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 380 KiB/s wr, 17 op/s 2023-09-10T09:11:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:17 smithi170 ceph-mon[102832]: pgmap v19097: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 380 KiB/s wr, 17 op/s 2023-09-10T09:11:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:17 smithi082 ceph-mon[98694]: pgmap v19097: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 380 KiB/s wr, 17 op/s 2023-09-10T09:11:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35509.604833 secs 2023-09-10T09:11:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35509.604833 secs 2023-09-10T09:11:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35509.604833 secs 2023-09-10T09:11:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35510.854691 secs 2023-09-10T09:11:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:19 smithi098 ceph-mon[103124]: pgmap v19098: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 339 KiB/s wr, 15 op/s 2023-09-10T09:11:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35511.207556 secs 2023-09-10T09:11:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35510.854691 secs 2023-09-10T09:11:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:19 smithi170 ceph-mon[102832]: pgmap v19098: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 339 KiB/s wr, 15 op/s 2023-09-10T09:11:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35511.207556 secs 2023-09-10T09:11:20.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35510.854691 secs 2023-09-10T09:11:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:19 smithi082 ceph-mon[98694]: pgmap v19098: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 339 KiB/s wr, 15 op/s 2023-09-10T09:11:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35511.207556 secs 2023-09-10T09:11:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:21 smithi098 ceph-mon[103124]: pgmap v19099: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 14 op/s 2023-09-10T09:11:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:21 smithi170 ceph-mon[102832]: pgmap v19099: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 14 op/s 2023-09-10T09:11:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:21 smithi082 ceph-mon[98694]: pgmap v19099: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 14 op/s 2023-09-10T09:11:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:23 smithi098 ceph-mon[103124]: pgmap v19100: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 14 op/s 2023-09-10T09:11:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35514.604894 secs 2023-09-10T09:11:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:23 smithi170 ceph-mon[102832]: pgmap v19100: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 14 op/s 2023-09-10T09:11:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35514.604894 secs 2023-09-10T09:11:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:23 smithi082 ceph-mon[98694]: pgmap v19100: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 14 op/s 2023-09-10T09:11:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35514.604894 secs 2023-09-10T09:11:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35515.854780 secs 2023-09-10T09:11:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35516.207620 secs 2023-09-10T09:11:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35515.854780 secs 2023-09-10T09:11:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35516.207620 secs 2023-09-10T09:11:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35515.854780 secs 2023-09-10T09:11:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35516.207620 secs 2023-09-10T09:11:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:25 smithi098 ceph-mon[103124]: pgmap v19101: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T09:11:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:25 smithi170 ceph-mon[102832]: pgmap v19101: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T09:11:26.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:25 smithi082 ceph-mon[98694]: pgmap v19101: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 12 op/s 2023-09-10T09:11:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:27 smithi098 ceph-mon[103124]: pgmap v19102: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 130 KiB/s wr, 17 op/s 2023-09-10T09:11:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:27 smithi170 ceph-mon[102832]: pgmap v19102: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 130 KiB/s wr, 17 op/s 2023-09-10T09:11:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:27 smithi082 ceph-mon[98694]: pgmap v19102: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 130 KiB/s wr, 17 op/s 2023-09-10T09:11:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35519.604955 secs 2023-09-10T09:11:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35519.604955 secs 2023-09-10T09:11:29.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35519.604955 secs 2023-09-10T09:11:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35520.854837 secs 2023-09-10T09:11:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:29 smithi098 ceph-mon[103124]: pgmap v19103: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T09:11:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35521.207687 secs 2023-09-10T09:11:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:29 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:11:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:29 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35520.854837 secs 2023-09-10T09:11:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:29 smithi170 ceph-mon[102832]: pgmap v19103: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T09:11:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35521.207687 secs 2023-09-10T09:11:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:29 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:11:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:29 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35520.854837 secs 2023-09-10T09:11:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:29 smithi082 ceph-mon[98694]: pgmap v19103: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T09:11:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35521.207687 secs 2023-09-10T09:11:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:29 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:11:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:29 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:32 smithi098 ceph-mon[103124]: pgmap v19104: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 156 KiB/s wr, 17 op/s 2023-09-10T09:11:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:32 smithi170 ceph-mon[102832]: pgmap v19104: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 156 KiB/s wr, 17 op/s 2023-09-10T09:11:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:32 smithi082 ceph-mon[98694]: pgmap v19104: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 156 KiB/s wr, 17 op/s 2023-09-10T09:11:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:11:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:11:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:11:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:34 smithi170 ceph-mon[102832]: pgmap v19105: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T09:11:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:34 smithi082 ceph-mon[98694]: pgmap v19105: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T09:11:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:34 smithi098 ceph-mon[103124]: pgmap v19105: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T09:11:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35525.854918 secs 2023-09-10T09:11:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35526.207754 secs 2023-09-10T09:11:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35525.854918 secs 2023-09-10T09:11:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35526.207754 secs 2023-09-10T09:11:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35525.854918 secs 2023-09-10T09:11:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35526.207754 secs 2023-09-10T09:11:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:36 smithi170 ceph-mon[102832]: pgmap v19106: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T09:11:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:36 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:11:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:36 smithi082 ceph-mon[98694]: pgmap v19106: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T09:11:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:36 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:11:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:36 smithi098 ceph-mon[103124]: pgmap v19106: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T09:11:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:36 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:11:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:37 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:11:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:11:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:37 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:11:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:11:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:37 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:11:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:11:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:38 smithi170 ceph-mon[102832]: pgmap v19107: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 297 KiB/s wr, 17 op/s 2023-09-10T09:11:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:11:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:38 smithi082 ceph-mon[98694]: pgmap v19107: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 297 KiB/s wr, 17 op/s 2023-09-10T09:11:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:11:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:38 smithi098 ceph-mon[103124]: pgmap v19107: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 297 KiB/s wr, 17 op/s 2023-09-10T09:11:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:11:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35530.855006 secs 2023-09-10T09:11:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:39 smithi170 ceph-mon[102832]: pgmap v19108: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T09:11:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35531.207819 secs 2023-09-10T09:11:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35530.855006 secs 2023-09-10T09:11:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:39 smithi082 ceph-mon[98694]: pgmap v19108: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T09:11:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35531.207819 secs 2023-09-10T09:11:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35530.855006 secs 2023-09-10T09:11:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:39 smithi098 ceph-mon[103124]: pgmap v19108: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T09:11:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35531.207819 secs 2023-09-10T09:11:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:11:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:11:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:11:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:41.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:41 smithi170 ceph-mon[102832]: pgmap v19109: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 933 KiB/s wr, 19 op/s 2023-09-10T09:11:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:41 smithi082 ceph-mon[98694]: pgmap v19109: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 933 KiB/s wr, 19 op/s 2023-09-10T09:11:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:41 smithi098 ceph-mon[103124]: pgmap v19109: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 933 KiB/s wr, 19 op/s 2023-09-10T09:11:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:43 smithi082 ceph-mon[98694]: pgmap v19110: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 804 KiB/s wr, 13 op/s 2023-09-10T09:11:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:43 smithi098 ceph-mon[103124]: pgmap v19110: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 804 KiB/s wr, 13 op/s 2023-09-10T09:11:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:43 smithi170 ceph-mon[102832]: pgmap v19110: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 804 KiB/s wr, 13 op/s 2023-09-10T09:11:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35535.855069 secs 2023-09-10T09:11:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35536.207952 secs 2023-09-10T09:11:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35535.855069 secs 2023-09-10T09:11:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35536.207952 secs 2023-09-10T09:11:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35535.855069 secs 2023-09-10T09:11:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35536.207952 secs 2023-09-10T09:11:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:45 smithi082 ceph-mon[98694]: pgmap v19111: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 804 KiB/s wr, 13 op/s 2023-09-10T09:11:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:11:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:45 smithi098 ceph-mon[103124]: pgmap v19111: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 804 KiB/s wr, 13 op/s 2023-09-10T09:11:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:11:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:45 smithi170 ceph-mon[102832]: pgmap v19111: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 804 KiB/s wr, 13 op/s 2023-09-10T09:11:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:11:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:47 smithi082 ceph-mon[98694]: pgmap v19112: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 905 KiB/s wr, 20 op/s 2023-09-10T09:11:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:47 smithi098 ceph-mon[103124]: pgmap v19112: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 905 KiB/s wr, 20 op/s 2023-09-10T09:11:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:47 smithi170 ceph-mon[102832]: pgmap v19112: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 905 KiB/s wr, 20 op/s 2023-09-10T09:11:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35540.855127 secs 2023-09-10T09:11:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:49 smithi170 ceph-mon[102832]: pgmap v19113: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 737 KiB/s wr, 14 op/s 2023-09-10T09:11:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35541.208020 secs 2023-09-10T09:11:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35540.855127 secs 2023-09-10T09:11:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:49 smithi082 ceph-mon[98694]: pgmap v19113: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 737 KiB/s wr, 14 op/s 2023-09-10T09:11:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35541.208020 secs 2023-09-10T09:11:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35540.855127 secs 2023-09-10T09:11:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:49 smithi098 ceph-mon[103124]: pgmap v19113: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 737 KiB/s wr, 14 op/s 2023-09-10T09:11:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35541.208020 secs 2023-09-10T09:11:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:11:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:11:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:11:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:51.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:51 smithi170 ceph-mon[102832]: pgmap v19114: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:11:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:51 smithi082 ceph-mon[98694]: pgmap v19114: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:11:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:51 smithi098 ceph-mon[103124]: pgmap v19114: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:11:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:11:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:11:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:11:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:53.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:53 smithi170 ceph-mon[102832]: pgmap v19115: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 474 KiB/s wr, 14 op/s 2023-09-10T09:11:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:53 smithi082 ceph-mon[98694]: pgmap v19115: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 474 KiB/s wr, 14 op/s 2023-09-10T09:11:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:53 smithi098 ceph-mon[103124]: pgmap v19115: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 474 KiB/s wr, 14 op/s 2023-09-10T09:11:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35545.855183 secs 2023-09-10T09:11:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:11:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35546.208086 secs 2023-09-10T09:11:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35545.855183 secs 2023-09-10T09:11:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:11:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35546.208086 secs 2023-09-10T09:11:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35545.855183 secs 2023-09-10T09:11:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:11:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35546.208086 secs 2023-09-10T09:11:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:55 smithi082 ceph-mon[98694]: pgmap v19116: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 474 KiB/s wr, 14 op/s 2023-09-10T09:11:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:55 smithi098 ceph-mon[103124]: pgmap v19116: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 474 KiB/s wr, 14 op/s 2023-09-10T09:11:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:55 smithi170 ceph-mon[102832]: pgmap v19116: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 474 KiB/s wr, 14 op/s 2023-09-10T09:11:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:11:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:11:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:57.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:11:57.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:57.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:57 smithi082 ceph-mon[98694]: pgmap v19117: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 923 KiB/s wr, 21 op/s 2023-09-10T09:11:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:57 smithi098 ceph-mon[103124]: pgmap v19117: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 923 KiB/s wr, 21 op/s 2023-09-10T09:11:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:57 smithi170 ceph-mon[102832]: pgmap v19117: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 923 KiB/s wr, 21 op/s 2023-09-10T09:11:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:11:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35549.605454 secs 2023-09-10T09:11:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:11:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35549.605454 secs 2023-09-10T09:11:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:11:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:11:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35549.605454 secs 2023-09-10T09:11:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35550.855240 secs 2023-09-10T09:11:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:59 smithi082 ceph-mon[98694]: pgmap v19118: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 822 KiB/s wr, 15 op/s 2023-09-10T09:11:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:11:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35551.208151 secs 2023-09-10T09:12:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35550.855240 secs 2023-09-10T09:12:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:59 smithi098 ceph-mon[103124]: pgmap v19118: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 822 KiB/s wr, 15 op/s 2023-09-10T09:12:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:11:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35551.208151 secs 2023-09-10T09:12:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35550.855240 secs 2023-09-10T09:12:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:59 smithi170 ceph-mon[102832]: pgmap v19118: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 822 KiB/s wr, 15 op/s 2023-09-10T09:12:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:11:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35551.208151 secs 2023-09-10T09:12:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:12:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:12:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:12:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:12:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:12:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:12:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:01 smithi082 ceph-mon[98694]: pgmap v19119: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 924 KiB/s wr, 21 op/s 2023-09-10T09:12:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:01 smithi098 ceph-mon[103124]: pgmap v19119: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 924 KiB/s wr, 21 op/s 2023-09-10T09:12:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:01 smithi170 ceph-mon[102832]: pgmap v19119: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 924 KiB/s wr, 21 op/s 2023-09-10T09:12:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:03 smithi082 ceph-mon[98694]: pgmap v19120: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 550 KiB/s wr, 13 op/s 2023-09-10T09:12:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35554.605534 secs 2023-09-10T09:12:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:03 smithi098 ceph-mon[103124]: pgmap v19120: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 550 KiB/s wr, 13 op/s 2023-09-10T09:12:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35554.605534 secs 2023-09-10T09:12:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:03 smithi170 ceph-mon[102832]: pgmap v19120: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 550 KiB/s wr, 13 op/s 2023-09-10T09:12:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35554.605534 secs 2023-09-10T09:12:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35555.855298 secs 2023-09-10T09:12:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35556.208218 secs 2023-09-10T09:12:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35555.855298 secs 2023-09-10T09:12:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35556.208218 secs 2023-09-10T09:12:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35555.855298 secs 2023-09-10T09:12:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35556.208218 secs 2023-09-10T09:12:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:05 smithi082 ceph-mon[98694]: pgmap v19121: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 550 KiB/s wr, 13 op/s 2023-09-10T09:12:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:05 smithi098 ceph-mon[103124]: pgmap v19121: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 550 KiB/s wr, 13 op/s 2023-09-10T09:12:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:05 smithi170 ceph-mon[102832]: pgmap v19121: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 550 KiB/s wr, 13 op/s 2023-09-10T09:12:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:07 smithi082 ceph-mon[98694]: pgmap v19122: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 893 KiB/s wr, 34 op/s 2023-09-10T09:12:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:07 smithi098 ceph-mon[103124]: pgmap v19122: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 893 KiB/s wr, 34 op/s 2023-09-10T09:12:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:07 smithi170 ceph-mon[102832]: pgmap v19122: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 893 KiB/s wr, 34 op/s 2023-09-10T09:12:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35559.605638 secs 2023-09-10T09:12:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:08 smithi098 ceph-mon[103124]: osdmap e1263: 12 total, 12 up, 12 in 2023-09-10T09:12:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35559.605638 secs 2023-09-10T09:12:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:08 smithi170 ceph-mon[102832]: osdmap e1263: 12 total, 12 up, 12 in 2023-09-10T09:12:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35559.605638 secs 2023-09-10T09:12:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:08 smithi082 ceph-mon[98694]: osdmap e1263: 12 total, 12 up, 12 in 2023-09-10T09:12:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35560.855400 secs 2023-09-10T09:12:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:09 smithi098 ceph-mon[103124]: pgmap v19124: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 534 KiB/s wr, 32 op/s 2023-09-10T09:12:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35561.208283 secs 2023-09-10T09:12:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35560.855400 secs 2023-09-10T09:12:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:09 smithi170 ceph-mon[102832]: pgmap v19124: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 534 KiB/s wr, 32 op/s 2023-09-10T09:12:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35561.208283 secs 2023-09-10T09:12:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35560.855400 secs 2023-09-10T09:12:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:09 smithi082 ceph-mon[98694]: pgmap v19124: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 534 KiB/s wr, 32 op/s 2023-09-10T09:12:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35561.208283 secs 2023-09-10T09:12:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:11 smithi098 ceph-mon[103124]: pgmap v19125: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 566 KiB/s wr, 32 op/s 2023-09-10T09:12:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:11 smithi170 ceph-mon[102832]: pgmap v19125: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 566 KiB/s wr, 32 op/s 2023-09-10T09:12:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:11 smithi082 ceph-mon[98694]: pgmap v19125: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 566 KiB/s wr, 32 op/s 2023-09-10T09:12:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:13 smithi098 ceph-mon[103124]: pgmap v19126: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 566 KiB/s wr, 32 op/s 2023-09-10T09:12:14.346 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35564.605699 secs 2023-09-10T09:12:14.346 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:12:14.346 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:14.347 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:12:14.347 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:13 smithi098 ceph-mon[103124]: osdmap e1264: 12 total, 12 up, 12 in 2023-09-10T09:12:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:13 smithi170 ceph-mon[102832]: pgmap v19126: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 566 KiB/s wr, 32 op/s 2023-09-10T09:12:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35564.605699 secs 2023-09-10T09:12:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:12:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:12:14.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:13 smithi170 ceph-mon[102832]: osdmap e1264: 12 total, 12 up, 12 in 2023-09-10T09:12:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:13 smithi082 ceph-mon[98694]: pgmap v19126: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 566 KiB/s wr, 32 op/s 2023-09-10T09:12:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35564.605699 secs 2023-09-10T09:12:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:12:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:12:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:13 smithi082 ceph-mon[98694]: osdmap e1264: 12 total, 12 up, 12 in 2023-09-10T09:12:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35565.855462 secs 2023-09-10T09:12:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35566.208351 secs 2023-09-10T09:12:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:12:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:12:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:12:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35565.855462 secs 2023-09-10T09:12:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35566.208351 secs 2023-09-10T09:12:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:12:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:12:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:12:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35565.855462 secs 2023-09-10T09:12:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35566.208351 secs 2023-09-10T09:12:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:12:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:12:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:12:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:16 smithi170 ceph-mon[102832]: pgmap v19128: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 8 op/s 2023-09-10T09:12:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:16 smithi082 ceph-mon[98694]: pgmap v19128: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 8 op/s 2023-09-10T09:12:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:16 smithi098 ceph-mon[103124]: pgmap v19128: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 193 KiB/s wr, 8 op/s 2023-09-10T09:12:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:17 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:12:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:17 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:17 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:12:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:17 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:17 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:12:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:17 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:18 smithi170 ceph-mon[102832]: pgmap v19129: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 209 KiB/s wr, 16 op/s 2023-09-10T09:12:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35569.605788 secs 2023-09-10T09:12:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:18 smithi082 ceph-mon[98694]: pgmap v19129: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 209 KiB/s wr, 16 op/s 2023-09-10T09:12:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35569.605788 secs 2023-09-10T09:12:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:18 smithi098 ceph-mon[103124]: pgmap v19129: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 209 KiB/s wr, 16 op/s 2023-09-10T09:12:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35569.605788 secs 2023-09-10T09:12:19.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35570.855525 secs 2023-09-10T09:12:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:19 smithi170 ceph-mon[102832]: pgmap v19130: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 14 op/s 2023-09-10T09:12:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35571.208417 secs 2023-09-10T09:12:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35570.855525 secs 2023-09-10T09:12:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:19 smithi082 ceph-mon[98694]: pgmap v19130: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 14 op/s 2023-09-10T09:12:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35571.208417 secs 2023-09-10T09:12:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35570.855525 secs 2023-09-10T09:12:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:19 smithi098 ceph-mon[103124]: pgmap v19130: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 14 op/s 2023-09-10T09:12:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35571.208417 secs 2023-09-10T09:12:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:21 smithi082 ceph-mon[98694]: pgmap v19131: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 13 op/s 2023-09-10T09:12:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:21 smithi098 ceph-mon[103124]: pgmap v19131: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 13 op/s 2023-09-10T09:12:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:21 smithi170 ceph-mon[102832]: pgmap v19131: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 13 op/s 2023-09-10T09:12:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:23 smithi082 ceph-mon[98694]: pgmap v19132: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 13 op/s 2023-09-10T09:12:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35574.605891 secs 2023-09-10T09:12:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:23 smithi098 ceph-mon[103124]: pgmap v19132: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 13 op/s 2023-09-10T09:12:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35574.605891 secs 2023-09-10T09:12:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:23 smithi170 ceph-mon[102832]: pgmap v19132: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 13 op/s 2023-09-10T09:12:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35574.605891 secs 2023-09-10T09:12:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35575.855583 secs 2023-09-10T09:12:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35576.208487 secs 2023-09-10T09:12:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35575.855583 secs 2023-09-10T09:12:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35576.208487 secs 2023-09-10T09:12:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35575.855583 secs 2023-09-10T09:12:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35576.208487 secs 2023-09-10T09:12:26.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:25 smithi098 ceph-mon[103124]: pgmap v19133: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 12 op/s 2023-09-10T09:12:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:25 smithi170 ceph-mon[102832]: pgmap v19133: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 12 op/s 2023-09-10T09:12:26.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:25 smithi082 ceph-mon[98694]: pgmap v19133: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 12 op/s 2023-09-10T09:12:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:27 smithi098 ceph-mon[103124]: pgmap v19134: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 156 KiB/s wr, 17 op/s 2023-09-10T09:12:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:27 smithi170 ceph-mon[102832]: pgmap v19134: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 156 KiB/s wr, 17 op/s 2023-09-10T09:12:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:27 smithi082 ceph-mon[98694]: pgmap v19134: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 156 KiB/s wr, 17 op/s 2023-09-10T09:12:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35579.605954 secs 2023-09-10T09:12:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35579.605954 secs 2023-09-10T09:12:29.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35579.605954 secs 2023-09-10T09:12:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35580.855644 secs 2023-09-10T09:12:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:29 smithi098 ceph-mon[103124]: pgmap v19135: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T09:12:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35581.208551 secs 2023-09-10T09:12:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35580.855644 secs 2023-09-10T09:12:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:29 smithi170 ceph-mon[102832]: pgmap v19135: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T09:12:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35581.208551 secs 2023-09-10T09:12:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35580.855644 secs 2023-09-10T09:12:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:29 smithi082 ceph-mon[98694]: pgmap v19135: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T09:12:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35581.208551 secs 2023-09-10T09:12:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:31 smithi098 ceph-mon[103124]: pgmap v19136: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 129 KiB/s wr, 16 op/s 2023-09-10T09:12:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:31 smithi170 ceph-mon[102832]: pgmap v19136: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 129 KiB/s wr, 16 op/s 2023-09-10T09:12:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:31 smithi082 ceph-mon[98694]: pgmap v19136: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 129 KiB/s wr, 16 op/s 2023-09-10T09:12:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:33 smithi098 ceph-mon[103124]: pgmap v19137: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T09:12:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:12:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:33 smithi170 ceph-mon[102832]: pgmap v19137: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T09:12:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:12:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:33 smithi082 ceph-mon[98694]: pgmap v19137: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T09:12:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:12:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35585.855705 secs 2023-09-10T09:12:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35586.208619 secs 2023-09-10T09:12:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35585.855705 secs 2023-09-10T09:12:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35586.208619 secs 2023-09-10T09:12:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35585.855705 secs 2023-09-10T09:12:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35586.208619 secs 2023-09-10T09:12:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:35 smithi098 ceph-mon[103124]: pgmap v19138: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T09:12:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:12:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:35 smithi170 ceph-mon[102832]: pgmap v19138: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T09:12:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:12:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:35 smithi082 ceph-mon[98694]: pgmap v19138: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T09:12:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:12:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:12:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:12:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:12:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:12:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:12:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:12:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:37 smithi098 ceph-mon[103124]: pgmap v19139: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 282 KiB/s wr, 17 op/s 2023-09-10T09:12:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:37 smithi170 ceph-mon[102832]: pgmap v19139: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 282 KiB/s wr, 17 op/s 2023-09-10T09:12:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:37 smithi082 ceph-mon[98694]: pgmap v19139: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 282 KiB/s wr, 17 op/s 2023-09-10T09:12:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35590.855777 secs 2023-09-10T09:12:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:39 smithi170 ceph-mon[102832]: pgmap v19140: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T09:12:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35591.208685 secs 2023-09-10T09:12:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35590.855777 secs 2023-09-10T09:12:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:39 smithi082 ceph-mon[98694]: pgmap v19140: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T09:12:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35591.208685 secs 2023-09-10T09:12:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35590.855777 secs 2023-09-10T09:12:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:39 smithi098 ceph-mon[103124]: pgmap v19140: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T09:12:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35591.208685 secs 2023-09-10T09:12:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:41 smithi082 ceph-mon[98694]: pgmap v19141: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 429 KiB/s wr, 17 op/s 2023-09-10T09:12:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:41 smithi098 ceph-mon[103124]: pgmap v19141: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 429 KiB/s wr, 17 op/s 2023-09-10T09:12:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:41 smithi170 ceph-mon[102832]: pgmap v19141: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 429 KiB/s wr, 17 op/s 2023-09-10T09:12:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:43 smithi098 ceph-mon[103124]: pgmap v19142: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 428 KiB/s wr, 12 op/s 2023-09-10T09:12:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:43 smithi170 ceph-mon[102832]: pgmap v19142: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 428 KiB/s wr, 12 op/s 2023-09-10T09:12:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:43 smithi082 ceph-mon[98694]: pgmap v19142: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 428 KiB/s wr, 12 op/s 2023-09-10T09:12:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35595.855855 secs 2023-09-10T09:12:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35596.208752 secs 2023-09-10T09:12:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:12:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35595.855855 secs 2023-09-10T09:12:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35596.208752 secs 2023-09-10T09:12:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:12:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35595.855855 secs 2023-09-10T09:12:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35596.208752 secs 2023-09-10T09:12:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:12:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:45 smithi098 ceph-mon[103124]: pgmap v19143: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 428 KiB/s wr, 12 op/s 2023-09-10T09:12:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:12:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:45 smithi170 ceph-mon[102832]: pgmap v19143: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 428 KiB/s wr, 12 op/s 2023-09-10T09:12:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:12:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:45 smithi082 ceph-mon[98694]: pgmap v19143: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 428 KiB/s wr, 12 op/s 2023-09-10T09:12:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:12:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:47 smithi098 ceph-mon[103124]: pgmap v19144: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 624 KiB/s wr, 19 op/s 2023-09-10T09:12:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:47 smithi170 ceph-mon[102832]: pgmap v19144: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 624 KiB/s wr, 19 op/s 2023-09-10T09:12:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:47 smithi082 ceph-mon[98694]: pgmap v19144: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 624 KiB/s wr, 19 op/s 2023-09-10T09:12:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35600.855914 secs 2023-09-10T09:12:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:49 smithi170 ceph-mon[102832]: pgmap v19145: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 471 KiB/s wr, 13 op/s 2023-09-10T09:12:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35601.208819 secs 2023-09-10T09:12:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35600.855914 secs 2023-09-10T09:12:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:49 smithi082 ceph-mon[98694]: pgmap v19145: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 471 KiB/s wr, 13 op/s 2023-09-10T09:12:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35601.208819 secs 2023-09-10T09:12:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35600.855914 secs 2023-09-10T09:12:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:49 smithi098 ceph-mon[103124]: pgmap v19145: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 471 KiB/s wr, 13 op/s 2023-09-10T09:12:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35601.208819 secs 2023-09-10T09:12:50.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:12:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:12:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:12:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:51.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:51 smithi082 ceph-mon[98694]: pgmap v19146: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 823 KiB/s wr, 21 op/s 2023-09-10T09:12:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:51 smithi098 ceph-mon[103124]: pgmap v19146: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 823 KiB/s wr, 21 op/s 2023-09-10T09:12:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:51 smithi170 ceph-mon[102832]: pgmap v19146: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 823 KiB/s wr, 21 op/s 2023-09-10T09:12:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:12:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:12:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:12:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:53.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:53 smithi082 ceph-mon[98694]: pgmap v19147: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 548 KiB/s wr, 14 op/s 2023-09-10T09:12:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:53 smithi098 ceph-mon[103124]: pgmap v19147: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 548 KiB/s wr, 14 op/s 2023-09-10T09:12:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:53 smithi170 ceph-mon[102832]: pgmap v19147: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 548 KiB/s wr, 14 op/s 2023-09-10T09:12:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35605.855998 secs 2023-09-10T09:12:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35606.208881 secs 2023-09-10T09:12:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35605.855998 secs 2023-09-10T09:12:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35606.208881 secs 2023-09-10T09:12:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35605.855998 secs 2023-09-10T09:12:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35606.208881 secs 2023-09-10T09:12:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:55 smithi082 ceph-mon[98694]: pgmap v19148: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 548 KiB/s wr, 14 op/s 2023-09-10T09:12:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:55 smithi098 ceph-mon[103124]: pgmap v19148: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 548 KiB/s wr, 14 op/s 2023-09-10T09:12:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:55 smithi170 ceph-mon[102832]: pgmap v19148: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 548 KiB/s wr, 14 op/s 2023-09-10T09:12:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:12:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:57 smithi170 ceph-mon[102832]: pgmap v19149: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 673 KiB/s wr, 21 op/s 2023-09-10T09:12:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:12:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:57 smithi082 ceph-mon[98694]: pgmap v19149: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 673 KiB/s wr, 21 op/s 2023-09-10T09:12:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:12:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:12:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:57 smithi098 ceph-mon[103124]: pgmap v19149: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 673 KiB/s wr, 21 op/s 2023-09-10T09:12:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35610.856059 secs 2023-09-10T09:12:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:59 smithi082 ceph-mon[98694]: pgmap v19150: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 478 KiB/s wr, 14 op/s 2023-09-10T09:12:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:12:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35611.208946 secs 2023-09-10T09:13:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35610.856059 secs 2023-09-10T09:13:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:59 smithi098 ceph-mon[103124]: pgmap v19150: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 478 KiB/s wr, 14 op/s 2023-09-10T09:13:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:12:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35611.208946 secs 2023-09-10T09:13:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35610.856059 secs 2023-09-10T09:13:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:59 smithi170 ceph-mon[102832]: pgmap v19150: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 478 KiB/s wr, 14 op/s 2023-09-10T09:13:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:12:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35611.208946 secs 2023-09-10T09:13:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:13:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:13:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:13:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:01.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:01 smithi082 ceph-mon[98694]: pgmap v19151: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 36 op/s 2023-09-10T09:13:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:01 smithi098 ceph-mon[103124]: pgmap v19151: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 36 op/s 2023-09-10T09:13:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:01 smithi170 ceph-mon[102832]: pgmap v19151: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 36 op/s 2023-09-10T09:13:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:03 smithi098 ceph-mon[103124]: pgmap v19152: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 832 KiB/s wr, 28 op/s 2023-09-10T09:13:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35614.606467 secs 2023-09-10T09:13:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:03 smithi170 ceph-mon[102832]: pgmap v19152: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 832 KiB/s wr, 28 op/s 2023-09-10T09:13:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35614.606467 secs 2023-09-10T09:13:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:03 smithi082 ceph-mon[98694]: pgmap v19152: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 832 KiB/s wr, 28 op/s 2023-09-10T09:13:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35614.606467 secs 2023-09-10T09:13:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35615.856123 secs 2023-09-10T09:13:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35616.209015 secs 2023-09-10T09:13:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:13:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:13:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35615.856123 secs 2023-09-10T09:13:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35616.209015 secs 2023-09-10T09:13:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:13:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:13:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35615.856123 secs 2023-09-10T09:13:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35616.209015 secs 2023-09-10T09:13:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:13:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:13:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:05 smithi098 ceph-mon[103124]: pgmap v19153: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 832 KiB/s wr, 28 op/s 2023-09-10T09:13:06.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:05 smithi170 ceph-mon[102832]: pgmap v19153: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 832 KiB/s wr, 28 op/s 2023-09-10T09:13:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:05 smithi082 ceph-mon[98694]: pgmap v19153: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 832 KiB/s wr, 28 op/s 2023-09-10T09:13:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:07 smithi098 ceph-mon[103124]: pgmap v19154: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 833 KiB/s wr, 35 op/s 2023-09-10T09:13:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:07 smithi170 ceph-mon[102832]: pgmap v19154: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 833 KiB/s wr, 35 op/s 2023-09-10T09:13:08.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:07 smithi082 ceph-mon[98694]: pgmap v19154: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 833 KiB/s wr, 35 op/s 2023-09-10T09:13:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35619.606580 secs 2023-09-10T09:13:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:08 smithi098 ceph-mon[103124]: osdmap e1265: 12 total, 12 up, 12 in 2023-09-10T09:13:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35619.606580 secs 2023-09-10T09:13:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:08 smithi170 ceph-mon[102832]: osdmap e1265: 12 total, 12 up, 12 in 2023-09-10T09:13:09.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35619.606580 secs 2023-09-10T09:13:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:08 smithi082 ceph-mon[98694]: osdmap e1265: 12 total, 12 up, 12 in 2023-09-10T09:13:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35620.856206 secs 2023-09-10T09:13:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:09 smithi098 ceph-mon[103124]: pgmap v19156: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 849 KiB/s wr, 34 op/s 2023-09-10T09:13:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35621.209085 secs 2023-09-10T09:13:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:09 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:13:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:09 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35620.856206 secs 2023-09-10T09:13:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:09 smithi170 ceph-mon[102832]: pgmap v19156: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 849 KiB/s wr, 34 op/s 2023-09-10T09:13:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35621.209085 secs 2023-09-10T09:13:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:09 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:13:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:09 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35620.856206 secs 2023-09-10T09:13:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:09 smithi082 ceph-mon[98694]: pgmap v19156: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 849 KiB/s wr, 34 op/s 2023-09-10T09:13:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35621.209085 secs 2023-09-10T09:13:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:09 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:13:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:09 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:11 smithi098 ceph-mon[103124]: pgmap v19157: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 366 KiB/s wr, 16 op/s 2023-09-10T09:13:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:11 smithi170 ceph-mon[102832]: pgmap v19157: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 366 KiB/s wr, 16 op/s 2023-09-10T09:13:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:11 smithi082 ceph-mon[98694]: pgmap v19157: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 366 KiB/s wr, 16 op/s 2023-09-10T09:13:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:13:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:13:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:13:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35624.606641 secs 2023-09-10T09:13:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:14 smithi098 ceph-mon[103124]: pgmap v19158: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 366 KiB/s wr, 16 op/s 2023-09-10T09:13:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:14 smithi098 ceph-mon[103124]: osdmap e1266: 12 total, 12 up, 12 in 2023-09-10T09:13:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35624.606641 secs 2023-09-10T09:13:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:14 smithi170 ceph-mon[102832]: pgmap v19158: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 366 KiB/s wr, 16 op/s 2023-09-10T09:13:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:14 smithi170 ceph-mon[102832]: osdmap e1266: 12 total, 12 up, 12 in 2023-09-10T09:13:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35624.606641 secs 2023-09-10T09:13:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:14 smithi082 ceph-mon[98694]: pgmap v19158: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 366 KiB/s wr, 16 op/s 2023-09-10T09:13:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:14 smithi082 ceph-mon[98694]: osdmap e1266: 12 total, 12 up, 12 in 2023-09-10T09:13:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35625.856302 secs 2023-09-10T09:13:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35626.209149 secs 2023-09-10T09:13:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:13:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:13:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:13:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:13:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35625.856302 secs 2023-09-10T09:13:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35626.209149 secs 2023-09-10T09:13:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:13:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:13:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:13:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:13:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35625.856302 secs 2023-09-10T09:13:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35626.209149 secs 2023-09-10T09:13:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:13:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:13:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:13:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:13:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:16 smithi170 ceph-mon[102832]: pgmap v19160: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 457 KiB/s wr, 10 op/s 2023-09-10T09:13:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:16 smithi082 ceph-mon[98694]: pgmap v19160: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 457 KiB/s wr, 10 op/s 2023-09-10T09:13:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:16 smithi098 ceph-mon[103124]: pgmap v19160: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 457 KiB/s wr, 10 op/s 2023-09-10T09:13:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:18 smithi170 ceph-mon[102832]: pgmap v19161: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 410 KiB/s wr, 15 op/s 2023-09-10T09:13:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:18 smithi082 ceph-mon[98694]: pgmap v19161: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 410 KiB/s wr, 15 op/s 2023-09-10T09:13:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:18 smithi098 ceph-mon[103124]: pgmap v19161: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 410 KiB/s wr, 15 op/s 2023-09-10T09:13:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35629.606731 secs 2023-09-10T09:13:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35629.606731 secs 2023-09-10T09:13:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35629.606731 secs 2023-09-10T09:13:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35630.856363 secs 2023-09-10T09:13:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:20 smithi170 ceph-mon[102832]: pgmap v19162: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 366 KiB/s wr, 14 op/s 2023-09-10T09:13:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35631.209218 secs 2023-09-10T09:13:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35630.856363 secs 2023-09-10T09:13:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:20 smithi082 ceph-mon[98694]: pgmap v19162: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 366 KiB/s wr, 14 op/s 2023-09-10T09:13:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35631.209218 secs 2023-09-10T09:13:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35630.856363 secs 2023-09-10T09:13:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:20 smithi098 ceph-mon[103124]: pgmap v19162: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 366 KiB/s wr, 14 op/s 2023-09-10T09:13:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35631.209218 secs 2023-09-10T09:13:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:22 smithi170 ceph-mon[102832]: pgmap v19163: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T09:13:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:22 smithi082 ceph-mon[98694]: pgmap v19163: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T09:13:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:22 smithi098 ceph-mon[103124]: pgmap v19163: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T09:13:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35634.606777 secs 2023-09-10T09:13:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35634.606777 secs 2023-09-10T09:13:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35634.606777 secs 2023-09-10T09:13:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:24 smithi170 ceph-mon[102832]: pgmap v19164: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T09:13:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:24 smithi082 ceph-mon[98694]: pgmap v19164: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T09:13:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:24 smithi098 ceph-mon[103124]: pgmap v19164: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T09:13:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35635.856422 secs 2023-09-10T09:13:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35636.209284 secs 2023-09-10T09:13:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35635.856422 secs 2023-09-10T09:13:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35636.209284 secs 2023-09-10T09:13:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35635.856422 secs 2023-09-10T09:13:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35636.209284 secs 2023-09-10T09:13:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:26 smithi170 ceph-mon[102832]: pgmap v19165: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 10 op/s 2023-09-10T09:13:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:26 smithi082 ceph-mon[98694]: pgmap v19165: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 10 op/s 2023-09-10T09:13:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:26 smithi098 ceph-mon[103124]: pgmap v19165: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 10 op/s 2023-09-10T09:13:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:28 smithi170 ceph-mon[102832]: pgmap v19166: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 125 KiB/s wr, 15 op/s 2023-09-10T09:13:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35639.606838 secs 2023-09-10T09:13:28.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:28 smithi082 ceph-mon[98694]: pgmap v19166: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 125 KiB/s wr, 15 op/s 2023-09-10T09:13:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35639.606838 secs 2023-09-10T09:13:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:28 smithi098 ceph-mon[103124]: pgmap v19166: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 125 KiB/s wr, 15 op/s 2023-09-10T09:13:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35639.606838 secs 2023-09-10T09:13:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35640.856501 secs 2023-09-10T09:13:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:29 smithi082 ceph-mon[98694]: pgmap v19167: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 125 KiB/s wr, 10 op/s 2023-09-10T09:13:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35641.209348 secs 2023-09-10T09:13:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35640.856501 secs 2023-09-10T09:13:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:29 smithi098 ceph-mon[103124]: pgmap v19167: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 125 KiB/s wr, 10 op/s 2023-09-10T09:13:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35641.209348 secs 2023-09-10T09:13:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35640.856501 secs 2023-09-10T09:13:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:29 smithi170 ceph-mon[102832]: pgmap v19167: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 125 KiB/s wr, 10 op/s 2023-09-10T09:13:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35641.209348 secs 2023-09-10T09:13:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:31 smithi098 ceph-mon[103124]: pgmap v19168: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 253 KiB/s wr, 15 op/s 2023-09-10T09:13:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:31 smithi170 ceph-mon[102832]: pgmap v19168: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 253 KiB/s wr, 15 op/s 2023-09-10T09:13:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:31 smithi082 ceph-mon[98694]: pgmap v19168: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 253 KiB/s wr, 15 op/s 2023-09-10T09:13:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:33 smithi098 ceph-mon[103124]: pgmap v19169: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 10 op/s 2023-09-10T09:13:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:13:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:33 smithi170 ceph-mon[102832]: pgmap v19169: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 10 op/s 2023-09-10T09:13:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:13:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:33 smithi082 ceph-mon[98694]: pgmap v19169: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 10 op/s 2023-09-10T09:13:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:13:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35645.856562 secs 2023-09-10T09:13:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35646.209416 secs 2023-09-10T09:13:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35645.856562 secs 2023-09-10T09:13:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35646.209416 secs 2023-09-10T09:13:35.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35645.856562 secs 2023-09-10T09:13:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35646.209416 secs 2023-09-10T09:13:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:35 smithi098 ceph-mon[103124]: pgmap v19170: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 10 op/s 2023-09-10T09:13:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:13:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:35 smithi170 ceph-mon[102832]: pgmap v19170: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 10 op/s 2023-09-10T09:13:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:13:36.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:35 smithi082 ceph-mon[98694]: pgmap v19170: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 10 op/s 2023-09-10T09:13:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:13:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:13:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:13:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:13:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:13:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:13:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:13:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:38.259 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:37 smithi098 ceph-mon[103124]: pgmap v19171: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 380 KiB/s wr, 17 op/s 2023-09-10T09:13:38.260 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:13:38.260 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:38.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:37 smithi170 ceph-mon[102832]: pgmap v19171: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 380 KiB/s wr, 17 op/s 2023-09-10T09:13:38.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:13:38.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:38.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:37 smithi082 ceph-mon[98694]: pgmap v19171: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 380 KiB/s wr, 17 op/s 2023-09-10T09:13:38.462 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:13:38.462 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35650.856640 secs 2023-09-10T09:13:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:39 smithi082 ceph-mon[98694]: pgmap v19172: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 379 KiB/s wr, 12 op/s 2023-09-10T09:13:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35651.209485 secs 2023-09-10T09:13:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35650.856640 secs 2023-09-10T09:13:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:39 smithi098 ceph-mon[103124]: pgmap v19172: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 379 KiB/s wr, 12 op/s 2023-09-10T09:13:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35651.209485 secs 2023-09-10T09:13:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35650.856640 secs 2023-09-10T09:13:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:39 smithi170 ceph-mon[102832]: pgmap v19172: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 379 KiB/s wr, 12 op/s 2023-09-10T09:13:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35651.209485 secs 2023-09-10T09:13:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:13:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:13:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:13:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:41.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:41 smithi082 ceph-mon[98694]: pgmap v19173: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 812 KiB/s wr, 19 op/s 2023-09-10T09:13:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:41 smithi098 ceph-mon[103124]: pgmap v19173: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 812 KiB/s wr, 19 op/s 2023-09-10T09:13:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:41 smithi170 ceph-mon[102832]: pgmap v19173: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 812 KiB/s wr, 19 op/s 2023-09-10T09:13:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:43 smithi098 ceph-mon[103124]: pgmap v19174: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 684 KiB/s wr, 14 op/s 2023-09-10T09:13:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:43 smithi170 ceph-mon[102832]: pgmap v19174: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 684 KiB/s wr, 14 op/s 2023-09-10T09:13:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:43 smithi082 ceph-mon[98694]: pgmap v19174: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 684 KiB/s wr, 14 op/s 2023-09-10T09:13:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35655.856697 secs 2023-09-10T09:13:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35656.209549 secs 2023-09-10T09:13:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:13:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:45.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35655.856697 secs 2023-09-10T09:13:45.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35656.209549 secs 2023-09-10T09:13:45.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:13:45.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35655.856697 secs 2023-09-10T09:13:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35656.209549 secs 2023-09-10T09:13:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:13:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:45 smithi098 ceph-mon[103124]: pgmap v19175: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 684 KiB/s wr, 14 op/s 2023-09-10T09:13:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:13:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:45 smithi170 ceph-mon[102832]: pgmap v19175: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 684 KiB/s wr, 14 op/s 2023-09-10T09:13:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:13:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:45 smithi082 ceph-mon[98694]: pgmap v19175: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 684 KiB/s wr, 14 op/s 2023-09-10T09:13:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:13:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:47 smithi098 ceph-mon[103124]: pgmap v19176: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 708 KiB/s wr, 20 op/s 2023-09-10T09:13:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:47 smithi170 ceph-mon[102832]: pgmap v19176: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 708 KiB/s wr, 20 op/s 2023-09-10T09:13:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:47 smithi082 ceph-mon[98694]: pgmap v19176: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 708 KiB/s wr, 20 op/s 2023-09-10T09:13:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35660.856756 secs 2023-09-10T09:13:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:49 smithi082 ceph-mon[98694]: pgmap v19177: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 457 KiB/s wr, 14 op/s 2023-09-10T09:13:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35661.209614 secs 2023-09-10T09:13:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35660.856756 secs 2023-09-10T09:13:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:49 smithi098 ceph-mon[103124]: pgmap v19177: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 457 KiB/s wr, 14 op/s 2023-09-10T09:13:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35661.209614 secs 2023-09-10T09:13:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35660.856756 secs 2023-09-10T09:13:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:49 smithi170 ceph-mon[102832]: pgmap v19177: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 457 KiB/s wr, 14 op/s 2023-09-10T09:13:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35661.209614 secs 2023-09-10T09:13:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:13:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:13:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:13:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:51 smithi098 ceph-mon[103124]: pgmap v19178: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:13:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:51 smithi170 ceph-mon[102832]: pgmap v19178: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:13:52.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:51 smithi082 ceph-mon[98694]: pgmap v19178: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:13:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:13:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:13:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:13:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:53 smithi098 ceph-mon[103124]: pgmap v19179: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 725 KiB/s wr, 14 op/s 2023-09-10T09:13:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:53 smithi170 ceph-mon[102832]: pgmap v19179: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 725 KiB/s wr, 14 op/s 2023-09-10T09:13:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:53 smithi082 ceph-mon[98694]: pgmap v19179: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 725 KiB/s wr, 14 op/s 2023-09-10T09:13:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35665.856814 secs 2023-09-10T09:13:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35666.209691 secs 2023-09-10T09:13:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35665.856814 secs 2023-09-10T09:13:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35666.209691 secs 2023-09-10T09:13:55.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35665.856814 secs 2023-09-10T09:13:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35666.209691 secs 2023-09-10T09:13:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:55 smithi098 ceph-mon[103124]: pgmap v19180: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 725 KiB/s wr, 14 op/s 2023-09-10T09:13:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:55 smithi170 ceph-mon[102832]: pgmap v19180: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 725 KiB/s wr, 14 op/s 2023-09-10T09:13:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:55 smithi082 ceph-mon[98694]: pgmap v19180: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 725 KiB/s wr, 14 op/s 2023-09-10T09:13:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:13:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:13:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:13:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:57 smithi098 ceph-mon[103124]: pgmap v19181: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:13:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:57 smithi170 ceph-mon[102832]: pgmap v19181: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:13:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:57 smithi082 ceph-mon[98694]: pgmap v19181: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:13:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:13:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:13:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:13:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:13:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35670.856873 secs 2023-09-10T09:14:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:59 smithi098 ceph-mon[103124]: pgmap v19182: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 14 op/s 2023-09-10T09:14:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:13:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35671.209759 secs 2023-09-10T09:14:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35670.856873 secs 2023-09-10T09:14:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:59 smithi170 ceph-mon[102832]: pgmap v19182: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 14 op/s 2023-09-10T09:14:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:13:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35671.209759 secs 2023-09-10T09:14:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35670.856873 secs 2023-09-10T09:14:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:59 smithi082 ceph-mon[98694]: pgmap v19182: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 14 op/s 2023-09-10T09:14:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:13:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35671.209759 secs 2023-09-10T09:14:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:14:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:14:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:14:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:01 smithi098 ceph-mon[103124]: pgmap v19183: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 22 op/s 2023-09-10T09:14:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:01 smithi170 ceph-mon[102832]: pgmap v19183: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 22 op/s 2023-09-10T09:14:02.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:01 smithi082 ceph-mon[98694]: pgmap v19183: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 22 op/s 2023-09-10T09:14:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35674.607333 secs 2023-09-10T09:14:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:03 smithi098 ceph-mon[103124]: pgmap v19184: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 563 KiB/s wr, 14 op/s 2023-09-10T09:14:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:03 smithi098 ceph-mon[103124]: osdmap e1267: 12 total, 12 up, 12 in 2023-09-10T09:14:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35674.607333 secs 2023-09-10T09:14:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:03 smithi170 ceph-mon[102832]: pgmap v19184: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 563 KiB/s wr, 14 op/s 2023-09-10T09:14:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:03 smithi170 ceph-mon[102832]: osdmap e1267: 12 total, 12 up, 12 in 2023-09-10T09:14:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35674.607333 secs 2023-09-10T09:14:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:03 smithi082 ceph-mon[98694]: pgmap v19184: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 563 KiB/s wr, 14 op/s 2023-09-10T09:14:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:03 smithi082 ceph-mon[98694]: osdmap e1267: 12 total, 12 up, 12 in 2023-09-10T09:14:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35675.856931 secs 2023-09-10T09:14:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35676.209824 secs 2023-09-10T09:14:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:14:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:14:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35675.856931 secs 2023-09-10T09:14:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35676.209824 secs 2023-09-10T09:14:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:14:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:14:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35675.856931 secs 2023-09-10T09:14:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35676.209824 secs 2023-09-10T09:14:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:14:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:14:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:05 smithi098 ceph-mon[103124]: pgmap v19186: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 675 KiB/s wr, 17 op/s 2023-09-10T09:14:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:14:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:05 smithi170 ceph-mon[102832]: pgmap v19186: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 675 KiB/s wr, 17 op/s 2023-09-10T09:14:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:14:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:05 smithi082 ceph-mon[98694]: pgmap v19186: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 675 KiB/s wr, 17 op/s 2023-09-10T09:14:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:14:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:07 smithi098 ceph-mon[103124]: pgmap v19187: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 545 KiB/s wr, 33 op/s 2023-09-10T09:14:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:07 smithi170 ceph-mon[102832]: pgmap v19187: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 545 KiB/s wr, 33 op/s 2023-09-10T09:14:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:07 smithi082 ceph-mon[98694]: pgmap v19187: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 545 KiB/s wr, 33 op/s 2023-09-10T09:14:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35679.607392 secs 2023-09-10T09:14:09.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:08 smithi098 ceph-mon[103124]: osdmap e1268: 12 total, 12 up, 12 in 2023-09-10T09:14:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35679.607392 secs 2023-09-10T09:14:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:08 smithi170 ceph-mon[102832]: osdmap e1268: 12 total, 12 up, 12 in 2023-09-10T09:14:09.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35679.607392 secs 2023-09-10T09:14:09.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:08 smithi082 ceph-mon[98694]: osdmap e1268: 12 total, 12 up, 12 in 2023-09-10T09:14:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35680.856991 secs 2023-09-10T09:14:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:09 smithi098 ceph-mon[103124]: pgmap v19189: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 468 KiB/s wr, 29 op/s 2023-09-10T09:14:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35681.209888 secs 2023-09-10T09:14:10.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35680.856991 secs 2023-09-10T09:14:10.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:09 smithi170 ceph-mon[102832]: pgmap v19189: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 468 KiB/s wr, 29 op/s 2023-09-10T09:14:10.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35681.209888 secs 2023-09-10T09:14:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35680.856991 secs 2023-09-10T09:14:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:09 smithi082 ceph-mon[98694]: pgmap v19189: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 468 KiB/s wr, 29 op/s 2023-09-10T09:14:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35681.209888 secs 2023-09-10T09:14:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:11 smithi098 ceph-mon[103124]: pgmap v19190: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 662 KiB/s wr, 38 op/s 2023-09-10T09:14:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:11 smithi170 ceph-mon[102832]: pgmap v19190: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 662 KiB/s wr, 38 op/s 2023-09-10T09:14:12.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:11 smithi082 ceph-mon[98694]: pgmap v19190: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 662 KiB/s wr, 38 op/s 2023-09-10T09:14:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35684.607452 secs 2023-09-10T09:14:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:13 smithi098 ceph-mon[103124]: pgmap v19191: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 531 KiB/s wr, 31 op/s 2023-09-10T09:14:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35684.607452 secs 2023-09-10T09:14:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:13 smithi170 ceph-mon[102832]: pgmap v19191: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 531 KiB/s wr, 31 op/s 2023-09-10T09:14:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35684.607452 secs 2023-09-10T09:14:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:13 smithi082 ceph-mon[98694]: pgmap v19191: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 531 KiB/s wr, 31 op/s 2023-09-10T09:14:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35685.857049 secs 2023-09-10T09:14:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35686.209953 secs 2023-09-10T09:14:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:14:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35685.857049 secs 2023-09-10T09:14:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35686.209953 secs 2023-09-10T09:14:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:14:15.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35685.857049 secs 2023-09-10T09:14:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35686.209953 secs 2023-09-10T09:14:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:14:16.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:16 smithi098 ceph-mon[103124]: pgmap v19192: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 529 KiB/s wr, 31 op/s 2023-09-10T09:14:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:14:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:14:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:14:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:16 smithi170 ceph-mon[102832]: pgmap v19192: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 529 KiB/s wr, 31 op/s 2023-09-10T09:14:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:14:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:14:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:14:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:16 smithi082 ceph-mon[98694]: pgmap v19192: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 529 KiB/s wr, 31 op/s 2023-09-10T09:14:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:14:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:14:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:14:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:18 smithi170 ceph-mon[102832]: pgmap v19193: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 188 KiB/s wr, 14 op/s 2023-09-10T09:14:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:18 smithi082 ceph-mon[98694]: pgmap v19193: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 188 KiB/s wr, 14 op/s 2023-09-10T09:14:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:18 smithi098 ceph-mon[103124]: pgmap v19193: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 188 KiB/s wr, 14 op/s 2023-09-10T09:14:19.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35689.607535 secs 2023-09-10T09:14:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35689.607535 secs 2023-09-10T09:14:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35689.607535 secs 2023-09-10T09:14:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35690.857107 secs 2023-09-10T09:14:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:20 smithi170 ceph-mon[102832]: pgmap v19194: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 13 op/s 2023-09-10T09:14:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35691.210020 secs 2023-09-10T09:14:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35690.857107 secs 2023-09-10T09:14:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:20 smithi082 ceph-mon[98694]: pgmap v19194: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 13 op/s 2023-09-10T09:14:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35691.210020 secs 2023-09-10T09:14:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35690.857107 secs 2023-09-10T09:14:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:20 smithi098 ceph-mon[103124]: pgmap v19194: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 13 op/s 2023-09-10T09:14:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35691.210020 secs 2023-09-10T09:14:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:22 smithi170 ceph-mon[102832]: pgmap v19195: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 17 op/s 2023-09-10T09:14:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:22 smithi082 ceph-mon[98694]: pgmap v19195: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 17 op/s 2023-09-10T09:14:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:22 smithi098 ceph-mon[103124]: pgmap v19195: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 17 op/s 2023-09-10T09:14:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35694.607594 secs 2023-09-10T09:14:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:24 smithi170 ceph-mon[102832]: pgmap v19196: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T09:14:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35694.607594 secs 2023-09-10T09:14:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:24 smithi082 ceph-mon[98694]: pgmap v19196: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T09:14:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35694.607594 secs 2023-09-10T09:14:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:24 smithi098 ceph-mon[103124]: pgmap v19196: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T09:14:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35695.857189 secs 2023-09-10T09:14:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35696.210087 secs 2023-09-10T09:14:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35695.857189 secs 2023-09-10T09:14:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35696.210087 secs 2023-09-10T09:14:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35695.857189 secs 2023-09-10T09:14:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35696.210087 secs 2023-09-10T09:14:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:26 smithi170 ceph-mon[102832]: pgmap v19197: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T09:14:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:26 smithi082 ceph-mon[98694]: pgmap v19197: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T09:14:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:26 smithi098 ceph-mon[103124]: pgmap v19197: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-09-10T09:14:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:28 smithi170 ceph-mon[102832]: pgmap v19198: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 17 op/s 2023-09-10T09:14:28.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:28 smithi082 ceph-mon[98694]: pgmap v19198: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 17 op/s 2023-09-10T09:14:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:28 smithi098 ceph-mon[103124]: pgmap v19198: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 17 op/s 2023-09-10T09:14:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35699.607654 secs 2023-09-10T09:14:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35699.607654 secs 2023-09-10T09:14:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35699.607654 secs 2023-09-10T09:14:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35700.857277 secs 2023-09-10T09:14:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:30 smithi170 ceph-mon[102832]: pgmap v19199: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 130 KiB/s wr, 11 op/s 2023-09-10T09:14:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35701.210150 secs 2023-09-10T09:14:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35700.857277 secs 2023-09-10T09:14:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:30 smithi082 ceph-mon[98694]: pgmap v19199: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 130 KiB/s wr, 11 op/s 2023-09-10T09:14:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35701.210150 secs 2023-09-10T09:14:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35700.857277 secs 2023-09-10T09:14:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:30 smithi098 ceph-mon[103124]: pgmap v19199: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 130 KiB/s wr, 11 op/s 2023-09-10T09:14:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35701.210150 secs 2023-09-10T09:14:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:32 smithi170 ceph-mon[102832]: pgmap v19200: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 130 KiB/s wr, 16 op/s 2023-09-10T09:14:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:32 smithi082 ceph-mon[98694]: pgmap v19200: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 130 KiB/s wr, 16 op/s 2023-09-10T09:14:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:32 smithi098 ceph-mon[103124]: pgmap v19200: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 130 KiB/s wr, 16 op/s 2023-09-10T09:14:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:34 smithi170 ceph-mon[102832]: pgmap v19201: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T09:14:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:14:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:34 smithi082 ceph-mon[98694]: pgmap v19201: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T09:14:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:14:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:34 smithi098 ceph-mon[103124]: pgmap v19201: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T09:14:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:14:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35705.857361 secs 2023-09-10T09:14:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35706.210219 secs 2023-09-10T09:14:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35705.857361 secs 2023-09-10T09:14:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35706.210219 secs 2023-09-10T09:14:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35705.857361 secs 2023-09-10T09:14:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35706.210219 secs 2023-09-10T09:14:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:36 smithi170 ceph-mon[102832]: pgmap v19202: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T09:14:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:36 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:14:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:36 smithi082 ceph-mon[98694]: pgmap v19202: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T09:14:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:36 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:14:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:36 smithi098 ceph-mon[103124]: pgmap v19202: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 11 op/s 2023-09-10T09:14:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:36 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:14:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:37 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:14:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:14:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:37.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:37 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:14:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:14:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:37 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:14:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:14:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:38 smithi170 ceph-mon[102832]: pgmap v19203: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 412 KiB/s wr, 17 op/s 2023-09-10T09:14:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:38 smithi082 ceph-mon[98694]: pgmap v19203: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 412 KiB/s wr, 17 op/s 2023-09-10T09:14:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:38 smithi098 ceph-mon[103124]: pgmap v19203: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 412 KiB/s wr, 17 op/s 2023-09-10T09:14:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35710.857419 secs 2023-09-10T09:14:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:39 smithi098 ceph-mon[103124]: pgmap v19204: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 283 KiB/s wr, 11 op/s 2023-09-10T09:14:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35711.210284 secs 2023-09-10T09:14:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35710.857419 secs 2023-09-10T09:14:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:39 smithi170 ceph-mon[102832]: pgmap v19204: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 283 KiB/s wr, 11 op/s 2023-09-10T09:14:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35711.210284 secs 2023-09-10T09:14:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35710.857419 secs 2023-09-10T09:14:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:39 smithi082 ceph-mon[98694]: pgmap v19204: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 283 KiB/s wr, 11 op/s 2023-09-10T09:14:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35711.210284 secs 2023-09-10T09:14:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:41 smithi098 ceph-mon[103124]: pgmap v19205: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 555 KiB/s wr, 17 op/s 2023-09-10T09:14:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:41 smithi170 ceph-mon[102832]: pgmap v19205: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 555 KiB/s wr, 17 op/s 2023-09-10T09:14:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:41 smithi082 ceph-mon[98694]: pgmap v19205: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 555 KiB/s wr, 17 op/s 2023-09-10T09:14:44.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:43 smithi098 ceph-mon[103124]: pgmap v19206: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 555 KiB/s wr, 12 op/s 2023-09-10T09:14:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:43 smithi170 ceph-mon[102832]: pgmap v19206: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 555 KiB/s wr, 12 op/s 2023-09-10T09:14:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:43 smithi082 ceph-mon[98694]: pgmap v19206: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 555 KiB/s wr, 12 op/s 2023-09-10T09:14:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35715.857505 secs 2023-09-10T09:14:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35716.210348 secs 2023-09-10T09:14:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:14:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35715.857505 secs 2023-09-10T09:14:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35716.210348 secs 2023-09-10T09:14:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:14:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35715.857505 secs 2023-09-10T09:14:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35716.210348 secs 2023-09-10T09:14:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:14:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:46 smithi170 ceph-mon[102832]: pgmap v19207: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 555 KiB/s wr, 12 op/s 2023-09-10T09:14:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:14:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:46 smithi082 ceph-mon[98694]: pgmap v19207: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 555 KiB/s wr, 12 op/s 2023-09-10T09:14:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:14:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:46 smithi098 ceph-mon[103124]: pgmap v19207: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 555 KiB/s wr, 12 op/s 2023-09-10T09:14:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:14:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:48 smithi170 ceph-mon[102832]: pgmap v19208: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 678 KiB/s wr, 19 op/s 2023-09-10T09:14:48.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:48 smithi082 ceph-mon[98694]: pgmap v19208: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 678 KiB/s wr, 19 op/s 2023-09-10T09:14:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:48 smithi098 ceph-mon[103124]: pgmap v19208: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 678 KiB/s wr, 19 op/s 2023-09-10T09:14:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35720.857566 secs 2023-09-10T09:14:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:49 smithi098 ceph-mon[103124]: pgmap v19209: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 396 KiB/s wr, 13 op/s 2023-09-10T09:14:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35721.210417 secs 2023-09-10T09:14:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35720.857566 secs 2023-09-10T09:14:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:49 smithi170 ceph-mon[102832]: pgmap v19209: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 396 KiB/s wr, 13 op/s 2023-09-10T09:14:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35721.210417 secs 2023-09-10T09:14:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35720.857566 secs 2023-09-10T09:14:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:49 smithi082 ceph-mon[98694]: pgmap v19209: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 396 KiB/s wr, 13 op/s 2023-09-10T09:14:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35721.210417 secs 2023-09-10T09:14:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:14:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:14:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:14:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:51 smithi098 ceph-mon[103124]: pgmap v19210: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 801 KiB/s wr, 21 op/s 2023-09-10T09:14:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:51 smithi170 ceph-mon[102832]: pgmap v19210: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 801 KiB/s wr, 21 op/s 2023-09-10T09:14:52.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:51 smithi082 ceph-mon[98694]: pgmap v19210: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 801 KiB/s wr, 21 op/s 2023-09-10T09:14:53.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:14:53.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:14:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:53.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:14:53.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:53 smithi098 ceph-mon[103124]: pgmap v19211: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 529 KiB/s wr, 14 op/s 2023-09-10T09:14:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:53 smithi170 ceph-mon[102832]: pgmap v19211: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 529 KiB/s wr, 14 op/s 2023-09-10T09:14:54.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:53 smithi082 ceph-mon[98694]: pgmap v19211: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 529 KiB/s wr, 14 op/s 2023-09-10T09:14:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35725.857624 secs 2023-09-10T09:14:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:14:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35726.210483 secs 2023-09-10T09:14:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35725.857624 secs 2023-09-10T09:14:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:14:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35726.210483 secs 2023-09-10T09:14:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35725.857624 secs 2023-09-10T09:14:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:14:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35726.210483 secs 2023-09-10T09:14:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:55 smithi098 ceph-mon[103124]: pgmap v19212: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 529 KiB/s wr, 14 op/s 2023-09-10T09:14:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:55 smithi170 ceph-mon[102832]: pgmap v19212: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 529 KiB/s wr, 14 op/s 2023-09-10T09:14:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:55 smithi082 ceph-mon[98694]: pgmap v19212: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 529 KiB/s wr, 14 op/s 2023-09-10T09:14:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:14:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:14:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:14:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:58.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:57 smithi098 ceph-mon[103124]: pgmap v19213: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 716 KiB/s wr, 21 op/s 2023-09-10T09:14:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:57 smithi170 ceph-mon[102832]: pgmap v19213: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 716 KiB/s wr, 21 op/s 2023-09-10T09:14:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:57 smithi082 ceph-mon[98694]: pgmap v19213: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 716 KiB/s wr, 21 op/s 2023-09-10T09:14:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:14:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:14:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:14:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:14:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35730.857682 secs 2023-09-10T09:15:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:59 smithi098 ceph-mon[103124]: pgmap v19214: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 593 KiB/s wr, 14 op/s 2023-09-10T09:15:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:14:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35731.210548 secs 2023-09-10T09:15:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35730.857682 secs 2023-09-10T09:15:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:59 smithi170 ceph-mon[102832]: pgmap v19214: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 593 KiB/s wr, 14 op/s 2023-09-10T09:15:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:14:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35731.210548 secs 2023-09-10T09:15:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35730.857682 secs 2023-09-10T09:15:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:59 smithi082 ceph-mon[98694]: pgmap v19214: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 593 KiB/s wr, 14 op/s 2023-09-10T09:15:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:14:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35731.210548 secs 2023-09-10T09:15:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:15:01.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:01.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:15:01.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:15:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:01 smithi170 ceph-mon[102832]: pgmap v19215: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 893 KiB/s wr, 22 op/s 2023-09-10T09:15:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:01 smithi082 ceph-mon[98694]: pgmap v19215: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 893 KiB/s wr, 22 op/s 2023-09-10T09:15:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:01 smithi098 ceph-mon[103124]: pgmap v19215: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 893 KiB/s wr, 22 op/s 2023-09-10T09:15:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35734.608206 secs 2023-09-10T09:15:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:03 smithi098 ceph-mon[103124]: pgmap v19216: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 487 KiB/s wr, 14 op/s 2023-09-10T09:15:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:03 smithi098 ceph-mon[103124]: osdmap e1269: 12 total, 12 up, 12 in 2023-09-10T09:15:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35734.608206 secs 2023-09-10T09:15:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:03 smithi170 ceph-mon[102832]: pgmap v19216: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 487 KiB/s wr, 14 op/s 2023-09-10T09:15:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:03 smithi170 ceph-mon[102832]: osdmap e1269: 12 total, 12 up, 12 in 2023-09-10T09:15:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35734.608206 secs 2023-09-10T09:15:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:03 smithi082 ceph-mon[98694]: pgmap v19216: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 487 KiB/s wr, 14 op/s 2023-09-10T09:15:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:03 smithi082 ceph-mon[98694]: osdmap e1269: 12 total, 12 up, 12 in 2023-09-10T09:15:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35735.857741 secs 2023-09-10T09:15:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35736.210617 secs 2023-09-10T09:15:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:15:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:15:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35735.857741 secs 2023-09-10T09:15:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35736.210617 secs 2023-09-10T09:15:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:15:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:15:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35735.857741 secs 2023-09-10T09:15:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35736.210617 secs 2023-09-10T09:15:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:15:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:15:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:06 smithi170 ceph-mon[102832]: pgmap v19218: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 585 KiB/s wr, 17 op/s 2023-09-10T09:15:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:06 smithi082 ceph-mon[98694]: pgmap v19218: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 585 KiB/s wr, 17 op/s 2023-09-10T09:15:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:06 smithi098 ceph-mon[103124]: pgmap v19218: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 585 KiB/s wr, 17 op/s 2023-09-10T09:15:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:08 smithi170 ceph-mon[102832]: pgmap v19219: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 740 KiB/s wr, 35 op/s 2023-09-10T09:15:08.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:08 smithi082 ceph-mon[98694]: pgmap v19219: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 740 KiB/s wr, 35 op/s 2023-09-10T09:15:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:08 smithi098 ceph-mon[103124]: pgmap v19219: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 740 KiB/s wr, 35 op/s 2023-09-10T09:15:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35739.608266 secs 2023-09-10T09:15:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:09 smithi170 ceph-mon[102832]: osdmap e1270: 12 total, 12 up, 12 in 2023-09-10T09:15:09.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35739.608266 secs 2023-09-10T09:15:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:09 smithi082 ceph-mon[98694]: osdmap e1270: 12 total, 12 up, 12 in 2023-09-10T09:15:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35739.608266 secs 2023-09-10T09:15:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:09 smithi098 ceph-mon[103124]: osdmap e1270: 12 total, 12 up, 12 in 2023-09-10T09:15:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35740.857829 secs 2023-09-10T09:15:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:10 smithi170 ceph-mon[102832]: pgmap v19221: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 474 KiB/s wr, 32 op/s 2023-09-10T09:15:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35741.210680 secs 2023-09-10T09:15:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:15:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:10.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35740.857829 secs 2023-09-10T09:15:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:10 smithi082 ceph-mon[98694]: pgmap v19221: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 474 KiB/s wr, 32 op/s 2023-09-10T09:15:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35741.210680 secs 2023-09-10T09:15:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:15:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35740.857829 secs 2023-09-10T09:15:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:10 smithi098 ceph-mon[103124]: pgmap v19221: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 474 KiB/s wr, 32 op/s 2023-09-10T09:15:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35741.210680 secs 2023-09-10T09:15:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:15:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:12 smithi170 ceph-mon[102832]: pgmap v19222: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 743 KiB/s wr, 41 op/s 2023-09-10T09:15:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:12 smithi082 ceph-mon[98694]: pgmap v19222: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 743 KiB/s wr, 41 op/s 2023-09-10T09:15:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:12 smithi098 ceph-mon[103124]: pgmap v19222: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 743 KiB/s wr, 41 op/s 2023-09-10T09:15:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:15:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:15:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:15:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35744.608329 secs 2023-09-10T09:15:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:14 smithi170 ceph-mon[102832]: pgmap v19223: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 595 KiB/s wr, 33 op/s 2023-09-10T09:15:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35744.608329 secs 2023-09-10T09:15:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:14 smithi082 ceph-mon[98694]: pgmap v19223: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 595 KiB/s wr, 33 op/s 2023-09-10T09:15:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35744.608329 secs 2023-09-10T09:15:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:14 smithi098 ceph-mon[103124]: pgmap v19223: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 595 KiB/s wr, 33 op/s 2023-09-10T09:15:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35745.857888 secs 2023-09-10T09:15:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35746.210745 secs 2023-09-10T09:15:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35745.857888 secs 2023-09-10T09:15:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35746.210745 secs 2023-09-10T09:15:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35745.857888 secs 2023-09-10T09:15:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35746.210745 secs 2023-09-10T09:15:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:16 smithi170 ceph-mon[102832]: pgmap v19224: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 594 KiB/s wr, 33 op/s 2023-09-10T09:15:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:15:16.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:16 smithi082 ceph-mon[98694]: pgmap v19224: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 594 KiB/s wr, 33 op/s 2023-09-10T09:15:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:15:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:16 smithi098 ceph-mon[103124]: pgmap v19224: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 594 KiB/s wr, 33 op/s 2023-09-10T09:15:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:15:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:15:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:15:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:15:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:15:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:15:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:15:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:15:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:15:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:15:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:18 smithi082 ceph-mon[98694]: pgmap v19225: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 338 KiB/s wr, 14 op/s 2023-09-10T09:15:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35749.608392 secs 2023-09-10T09:15:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:18 smithi098 ceph-mon[103124]: pgmap v19225: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 338 KiB/s wr, 14 op/s 2023-09-10T09:15:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35749.608392 secs 2023-09-10T09:15:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:18 smithi170 ceph-mon[102832]: pgmap v19225: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 338 KiB/s wr, 14 op/s 2023-09-10T09:15:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35749.608392 secs 2023-09-10T09:15:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35750.857948 secs 2023-09-10T09:15:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:19 smithi098 ceph-mon[103124]: pgmap v19226: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 330 KiB/s wr, 14 op/s 2023-09-10T09:15:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35751.210810 secs 2023-09-10T09:15:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35750.857948 secs 2023-09-10T09:15:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:19 smithi170 ceph-mon[102832]: pgmap v19226: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 330 KiB/s wr, 14 op/s 2023-09-10T09:15:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35751.210810 secs 2023-09-10T09:15:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35750.857948 secs 2023-09-10T09:15:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:19 smithi082 ceph-mon[98694]: pgmap v19226: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 330 KiB/s wr, 14 op/s 2023-09-10T09:15:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35751.210810 secs 2023-09-10T09:15:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:22 smithi098 ceph-mon[103124]: pgmap v19227: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 282 KiB/s wr, 16 op/s 2023-09-10T09:15:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:22 smithi170 ceph-mon[102832]: pgmap v19227: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 282 KiB/s wr, 16 op/s 2023-09-10T09:15:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:22 smithi082 ceph-mon[98694]: pgmap v19227: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 282 KiB/s wr, 16 op/s 2023-09-10T09:15:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35754.608455 secs 2023-09-10T09:15:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:24 smithi170 ceph-mon[102832]: pgmap v19228: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 10 op/s 2023-09-10T09:15:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35754.608455 secs 2023-09-10T09:15:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:24 smithi082 ceph-mon[98694]: pgmap v19228: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 10 op/s 2023-09-10T09:15:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35754.608455 secs 2023-09-10T09:15:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:24 smithi098 ceph-mon[103124]: pgmap v19228: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 10 op/s 2023-09-10T09:15:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35755.858005 secs 2023-09-10T09:15:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35756.210877 secs 2023-09-10T09:15:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35755.858005 secs 2023-09-10T09:15:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35756.210877 secs 2023-09-10T09:15:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35755.858005 secs 2023-09-10T09:15:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35756.210877 secs 2023-09-10T09:15:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:26 smithi170 ceph-mon[102832]: pgmap v19229: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 10 op/s 2023-09-10T09:15:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:26 smithi082 ceph-mon[98694]: pgmap v19229: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 10 op/s 2023-09-10T09:15:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:26 smithi098 ceph-mon[103124]: pgmap v19229: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 103 KiB/s wr, 10 op/s 2023-09-10T09:15:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:28 smithi170 ceph-mon[102832]: pgmap v19230: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 103 KiB/s wr, 15 op/s 2023-09-10T09:15:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:28 smithi082 ceph-mon[98694]: pgmap v19230: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 103 KiB/s wr, 15 op/s 2023-09-10T09:15:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:28 smithi098 ceph-mon[103124]: pgmap v19230: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 103 KiB/s wr, 15 op/s 2023-09-10T09:15:29.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35759.608517 secs 2023-09-10T09:15:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35759.608517 secs 2023-09-10T09:15:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35759.608517 secs 2023-09-10T09:15:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35760.858063 secs 2023-09-10T09:15:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35761.210940 secs 2023-09-10T09:15:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:30 smithi170 ceph-mon[102832]: pgmap v19231: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-10T09:15:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35760.858063 secs 2023-09-10T09:15:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35761.210940 secs 2023-09-10T09:15:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:30 smithi082 ceph-mon[98694]: pgmap v19231: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-10T09:15:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35760.858063 secs 2023-09-10T09:15:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35761.210940 secs 2023-09-10T09:15:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:30 smithi098 ceph-mon[103124]: pgmap v19231: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-10T09:15:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:32 smithi170 ceph-mon[102832]: pgmap v19232: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 129 KiB/s wr, 15 op/s 2023-09-10T09:15:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:32 smithi082 ceph-mon[98694]: pgmap v19232: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 129 KiB/s wr, 15 op/s 2023-09-10T09:15:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:32 smithi098 ceph-mon[103124]: pgmap v19232: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 129 KiB/s wr, 15 op/s 2023-09-10T09:15:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:34 smithi170 ceph-mon[102832]: pgmap v19233: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 10 op/s 2023-09-10T09:15:34.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:34 smithi082 ceph-mon[98694]: pgmap v19233: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 10 op/s 2023-09-10T09:15:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:34 smithi098 ceph-mon[103124]: pgmap v19233: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 10 op/s 2023-09-10T09:15:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35765.858122 secs 2023-09-10T09:15:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35766.211006 secs 2023-09-10T09:15:35.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35765.858122 secs 2023-09-10T09:15:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35766.211006 secs 2023-09-10T09:15:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35765.858122 secs 2023-09-10T09:15:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35766.211006 secs 2023-09-10T09:15:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:36 smithi170 ceph-mon[102832]: pgmap v19234: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 10 op/s 2023-09-10T09:15:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:36 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:15:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:36 smithi082 ceph-mon[98694]: pgmap v19234: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 10 op/s 2023-09-10T09:15:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:36 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:15:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:36 smithi098 ceph-mon[103124]: pgmap v19234: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 129 KiB/s wr, 10 op/s 2023-09-10T09:15:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:36 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:15:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:37 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:15:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:15:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:37 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:15:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:15:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:37 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:15:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:15:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:38 smithi170 ceph-mon[102832]: pgmap v19235: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 373 KiB/s wr, 16 op/s 2023-09-10T09:15:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:15:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:38 smithi082 ceph-mon[98694]: pgmap v19235: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 373 KiB/s wr, 16 op/s 2023-09-10T09:15:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:15:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:38 smithi098 ceph-mon[103124]: pgmap v19235: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 373 KiB/s wr, 16 op/s 2023-09-10T09:15:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:15:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35770.858181 secs 2023-09-10T09:15:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35771.211070 secs 2023-09-10T09:15:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:39 smithi098 ceph-mon[103124]: pgmap v19236: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 372 KiB/s wr, 11 op/s 2023-09-10T09:15:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35770.858181 secs 2023-09-10T09:15:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35771.211070 secs 2023-09-10T09:15:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:39 smithi170 ceph-mon[102832]: pgmap v19236: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 372 KiB/s wr, 11 op/s 2023-09-10T09:15:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35770.858181 secs 2023-09-10T09:15:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35771.211070 secs 2023-09-10T09:15:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:39 smithi082 ceph-mon[98694]: pgmap v19236: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 372 KiB/s wr, 11 op/s 2023-09-10T09:15:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:15:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:15:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:15:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:41 smithi098 ceph-mon[103124]: pgmap v19237: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 657 KiB/s wr, 17 op/s 2023-09-10T09:15:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:41 smithi170 ceph-mon[102832]: pgmap v19237: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 657 KiB/s wr, 17 op/s 2023-09-10T09:15:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:41 smithi082 ceph-mon[98694]: pgmap v19237: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 657 KiB/s wr, 17 op/s 2023-09-10T09:15:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:44 smithi170 ceph-mon[102832]: pgmap v19238: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 528 KiB/s wr, 11 op/s 2023-09-10T09:15:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:44 smithi082 ceph-mon[98694]: pgmap v19238: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 528 KiB/s wr, 11 op/s 2023-09-10T09:15:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:44 smithi098 ceph-mon[103124]: pgmap v19238: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 528 KiB/s wr, 11 op/s 2023-09-10T09:15:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35775.858281 secs 2023-09-10T09:15:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35776.211138 secs 2023-09-10T09:15:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35775.858281 secs 2023-09-10T09:15:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35776.211138 secs 2023-09-10T09:15:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35775.858281 secs 2023-09-10T09:15:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35776.211138 secs 2023-09-10T09:15:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:46 smithi170 ceph-mon[102832]: pgmap v19239: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 528 KiB/s wr, 11 op/s 2023-09-10T09:15:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:15:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:46.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:46 smithi082 ceph-mon[98694]: pgmap v19239: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 528 KiB/s wr, 11 op/s 2023-09-10T09:15:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:15:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:46 smithi098 ceph-mon[103124]: pgmap v19239: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 528 KiB/s wr, 11 op/s 2023-09-10T09:15:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:15:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:48 smithi170 ceph-mon[102832]: pgmap v19240: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 711 KiB/s wr, 19 op/s 2023-09-10T09:15:48.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:48 smithi082 ceph-mon[98694]: pgmap v19240: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 711 KiB/s wr, 19 op/s 2023-09-10T09:15:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:48 smithi098 ceph-mon[103124]: pgmap v19240: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 711 KiB/s wr, 19 op/s 2023-09-10T09:15:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35780.858337 secs 2023-09-10T09:15:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35781.211208 secs 2023-09-10T09:15:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:49 smithi098 ceph-mon[103124]: pgmap v19241: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 468 KiB/s wr, 13 op/s 2023-09-10T09:15:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35780.858337 secs 2023-09-10T09:15:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35781.211208 secs 2023-09-10T09:15:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:49 smithi170 ceph-mon[102832]: pgmap v19241: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 468 KiB/s wr, 13 op/s 2023-09-10T09:15:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35780.858337 secs 2023-09-10T09:15:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35781.211208 secs 2023-09-10T09:15:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:49 smithi082 ceph-mon[98694]: pgmap v19241: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 468 KiB/s wr, 13 op/s 2023-09-10T09:15:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:15:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:15:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:15:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:51 smithi098 ceph-mon[103124]: pgmap v19242: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 800 KiB/s wr, 21 op/s 2023-09-10T09:15:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:51 smithi170 ceph-mon[102832]: pgmap v19242: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 800 KiB/s wr, 21 op/s 2023-09-10T09:15:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:51 smithi082 ceph-mon[98694]: pgmap v19242: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 800 KiB/s wr, 21 op/s 2023-09-10T09:15:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:15:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:15:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:53.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:15:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:54.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:53 smithi098 ceph-mon[103124]: pgmap v19243: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 515 KiB/s wr, 15 op/s 2023-09-10T09:15:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:53 smithi170 ceph-mon[102832]: pgmap v19243: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 515 KiB/s wr, 15 op/s 2023-09-10T09:15:54.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:53 smithi082 ceph-mon[98694]: pgmap v19243: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 515 KiB/s wr, 15 op/s 2023-09-10T09:15:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35785.858395 secs 2023-09-10T09:15:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:15:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35786.211274 secs 2023-09-10T09:15:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35785.858395 secs 2023-09-10T09:15:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:15:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35786.211274 secs 2023-09-10T09:15:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35785.858395 secs 2023-09-10T09:15:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:15:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35786.211274 secs 2023-09-10T09:15:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:55 smithi170 ceph-mon[102832]: pgmap v19244: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 515 KiB/s wr, 15 op/s 2023-09-10T09:15:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:55 smithi082 ceph-mon[98694]: pgmap v19244: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 515 KiB/s wr, 15 op/s 2023-09-10T09:15:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:55 smithi098 ceph-mon[103124]: pgmap v19244: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 515 KiB/s wr, 15 op/s 2023-09-10T09:15:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:15:57.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:15:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:15:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:15:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:57 smithi170 ceph-mon[102832]: pgmap v19245: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 610 KiB/s wr, 22 op/s 2023-09-10T09:15:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:57 smithi082 ceph-mon[98694]: pgmap v19245: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 610 KiB/s wr, 22 op/s 2023-09-10T09:15:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:57 smithi098 ceph-mon[103124]: pgmap v19245: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 610 KiB/s wr, 22 op/s 2023-09-10T09:15:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35789.609026 secs 2023-09-10T09:15:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35789.609026 secs 2023-09-10T09:15:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35789.609026 secs 2023-09-10T09:16:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35790.858454 secs 2023-09-10T09:16:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35791.211336 secs 2023-09-10T09:16:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:15:59 smithi170 ceph-mon[102832]: pgmap v19246: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 427 KiB/s wr, 15 op/s 2023-09-10T09:16:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35790.858454 secs 2023-09-10T09:16:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35791.211336 secs 2023-09-10T09:16:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:15:59 smithi082 ceph-mon[98694]: pgmap v19246: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 427 KiB/s wr, 15 op/s 2023-09-10T09:16:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35790.858454 secs 2023-09-10T09:16:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35791.211336 secs 2023-09-10T09:16:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:15:59 smithi098 ceph-mon[103124]: pgmap v19246: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 427 KiB/s wr, 15 op/s 2023-09-10T09:16:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:16:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:16:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:16:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:16:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:16:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:16:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:01 smithi170 ceph-mon[102832]: pgmap v19247: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 728 KiB/s wr, 22 op/s 2023-09-10T09:16:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:01 smithi082 ceph-mon[98694]: pgmap v19247: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 728 KiB/s wr, 22 op/s 2023-09-10T09:16:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:01 smithi098 ceph-mon[103124]: pgmap v19247: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 728 KiB/s wr, 22 op/s 2023-09-10T09:16:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35794.609162 secs 2023-09-10T09:16:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:04 smithi170 ceph-mon[102832]: pgmap v19248: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 397 KiB/s wr, 13 op/s 2023-09-10T09:16:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35794.609162 secs 2023-09-10T09:16:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:04 smithi082 ceph-mon[98694]: pgmap v19248: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 397 KiB/s wr, 13 op/s 2023-09-10T09:16:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35794.609162 secs 2023-09-10T09:16:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:04 smithi098 ceph-mon[103124]: pgmap v19248: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 397 KiB/s wr, 13 op/s 2023-09-10T09:16:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35795.858523 secs 2023-09-10T09:16:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35796.211402 secs 2023-09-10T09:16:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:16:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35795.858523 secs 2023-09-10T09:16:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35796.211402 secs 2023-09-10T09:16:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:16:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35795.858523 secs 2023-09-10T09:16:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35796.211402 secs 2023-09-10T09:16:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:16:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:06 smithi170 ceph-mon[102832]: pgmap v19249: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 397 KiB/s wr, 13 op/s 2023-09-10T09:16:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:16:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:06.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:06 smithi082 ceph-mon[98694]: pgmap v19249: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 397 KiB/s wr, 13 op/s 2023-09-10T09:16:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:16:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:06 smithi098 ceph-mon[103124]: pgmap v19249: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 397 KiB/s wr, 13 op/s 2023-09-10T09:16:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:16:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:08 smithi170 ceph-mon[102832]: pgmap v19250: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 545 KiB/s wr, 34 op/s 2023-09-10T09:16:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:08 smithi082 ceph-mon[98694]: pgmap v19250: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 545 KiB/s wr, 34 op/s 2023-09-10T09:16:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:08 smithi098 ceph-mon[103124]: pgmap v19250: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 545 KiB/s wr, 34 op/s 2023-09-10T09:16:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35799.609222 secs 2023-09-10T09:16:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:09 smithi170 ceph-mon[102832]: osdmap e1271: 12 total, 12 up, 12 in 2023-09-10T09:16:09.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35799.609222 secs 2023-09-10T09:16:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:09 smithi082 ceph-mon[98694]: osdmap e1271: 12 total, 12 up, 12 in 2023-09-10T09:16:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35799.609222 secs 2023-09-10T09:16:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:09 smithi098 ceph-mon[103124]: osdmap e1271: 12 total, 12 up, 12 in 2023-09-10T09:16:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35800.858581 secs 2023-09-10T09:16:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35801.211466 secs 2023-09-10T09:16:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:10 smithi170 ceph-mon[102832]: pgmap v19252: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 539 KiB/s wr, 32 op/s 2023-09-10T09:16:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35800.858581 secs 2023-09-10T09:16:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35801.211466 secs 2023-09-10T09:16:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:10 smithi082 ceph-mon[98694]: pgmap v19252: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 539 KiB/s wr, 32 op/s 2023-09-10T09:16:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35800.858581 secs 2023-09-10T09:16:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35801.211466 secs 2023-09-10T09:16:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:10 smithi098 ceph-mon[103124]: pgmap v19252: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 539 KiB/s wr, 32 op/s 2023-09-10T09:16:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:12 smithi170 ceph-mon[102832]: pgmap v19253: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 178 KiB/s wr, 29 op/s 2023-09-10T09:16:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:12 smithi082 ceph-mon[98694]: pgmap v19253: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 178 KiB/s wr, 29 op/s 2023-09-10T09:16:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:12 smithi098 ceph-mon[103124]: pgmap v19253: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 178 KiB/s wr, 29 op/s 2023-09-10T09:16:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35804.609282 secs 2023-09-10T09:16:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35804.609282 secs 2023-09-10T09:16:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35804.609282 secs 2023-09-10T09:16:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:14 smithi170 ceph-mon[102832]: pgmap v19254: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 178 KiB/s wr, 29 op/s 2023-09-10T09:16:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:14 smithi170 ceph-mon[102832]: osdmap e1272: 12 total, 12 up, 12 in 2023-09-10T09:16:14.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:14 smithi082 ceph-mon[98694]: pgmap v19254: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 178 KiB/s wr, 29 op/s 2023-09-10T09:16:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:14 smithi082 ceph-mon[98694]: osdmap e1272: 12 total, 12 up, 12 in 2023-09-10T09:16:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:14 smithi098 ceph-mon[103124]: pgmap v19254: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 178 KiB/s wr, 29 op/s 2023-09-10T09:16:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:14 smithi098 ceph-mon[103124]: osdmap e1272: 12 total, 12 up, 12 in 2023-09-10T09:16:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35805.858678 secs 2023-09-10T09:16:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35806.211492 secs 2023-09-10T09:16:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35805.858678 secs 2023-09-10T09:16:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35806.211492 secs 2023-09-10T09:16:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35805.858678 secs 2023-09-10T09:16:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35806.211492 secs 2023-09-10T09:16:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:16 smithi170 ceph-mon[102832]: pgmap v19256: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T09:16:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:16 smithi082 ceph-mon[98694]: pgmap v19256: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T09:16:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:16 smithi098 ceph-mon[103124]: pgmap v19256: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T09:16:17.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:16:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:16:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:16:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:16:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:16:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:16:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:16:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:16:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:16:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:16:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:16:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:16:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:18 smithi170 ceph-mon[102832]: pgmap v19257: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 12 op/s 2023-09-10T09:16:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35809.609342 secs 2023-09-10T09:16:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:18 smithi082 ceph-mon[98694]: pgmap v19257: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 12 op/s 2023-09-10T09:16:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35809.609342 secs 2023-09-10T09:16:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:18 smithi098 ceph-mon[103124]: pgmap v19257: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 12 op/s 2023-09-10T09:16:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35809.609342 secs 2023-09-10T09:16:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35810.858737 secs 2023-09-10T09:16:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35811.211557 secs 2023-09-10T09:16:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:19 smithi098 ceph-mon[103124]: pgmap v19258: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T09:16:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35810.858737 secs 2023-09-10T09:16:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35811.211557 secs 2023-09-10T09:16:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:19 smithi170 ceph-mon[102832]: pgmap v19258: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T09:16:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35810.858737 secs 2023-09-10T09:16:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35811.211557 secs 2023-09-10T09:16:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:19 smithi082 ceph-mon[98694]: pgmap v19258: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T09:16:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:22 smithi170 ceph-mon[102832]: pgmap v19259: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T09:16:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:22 smithi082 ceph-mon[98694]: pgmap v19259: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T09:16:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:22 smithi098 ceph-mon[103124]: pgmap v19259: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T09:16:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35814.609400 secs 2023-09-10T09:16:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:24 smithi170 ceph-mon[102832]: pgmap v19260: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T09:16:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35814.609400 secs 2023-09-10T09:16:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:24 smithi082 ceph-mon[98694]: pgmap v19260: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T09:16:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35814.609400 secs 2023-09-10T09:16:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:24 smithi098 ceph-mon[103124]: pgmap v19260: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T09:16:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35815.858810 secs 2023-09-10T09:16:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35816.211623 secs 2023-09-10T09:16:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35815.858810 secs 2023-09-10T09:16:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35816.211623 secs 2023-09-10T09:16:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35815.858810 secs 2023-09-10T09:16:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35816.211623 secs 2023-09-10T09:16:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:26 smithi170 ceph-mon[102832]: pgmap v19261: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T09:16:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:26 smithi082 ceph-mon[98694]: pgmap v19261: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T09:16:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:26 smithi098 ceph-mon[103124]: pgmap v19261: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T09:16:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:28 smithi170 ceph-mon[102832]: pgmap v19262: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 125 KiB/s wr, 14 op/s 2023-09-10T09:16:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35819.609467 secs 2023-09-10T09:16:28.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:28 smithi082 ceph-mon[98694]: pgmap v19262: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 125 KiB/s wr, 14 op/s 2023-09-10T09:16:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35819.609467 secs 2023-09-10T09:16:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:28 smithi098 ceph-mon[103124]: pgmap v19262: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 125 KiB/s wr, 14 op/s 2023-09-10T09:16:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35819.609467 secs 2023-09-10T09:16:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35820.858868 secs 2023-09-10T09:16:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35821.211689 secs 2023-09-10T09:16:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:29 smithi098 ceph-mon[103124]: pgmap v19263: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 10 op/s 2023-09-10T09:16:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35820.858868 secs 2023-09-10T09:16:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35821.211689 secs 2023-09-10T09:16:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:29 smithi170 ceph-mon[102832]: pgmap v19263: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 10 op/s 2023-09-10T09:16:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35820.858868 secs 2023-09-10T09:16:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35821.211689 secs 2023-09-10T09:16:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:29 smithi082 ceph-mon[98694]: pgmap v19263: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 10 op/s 2023-09-10T09:16:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:16:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:16:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:16:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:32.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:31 smithi098 ceph-mon[103124]: pgmap v19264: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 277 KiB/s wr, 15 op/s 2023-09-10T09:16:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:31 smithi170 ceph-mon[102832]: pgmap v19264: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 277 KiB/s wr, 15 op/s 2023-09-10T09:16:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:31 smithi082 ceph-mon[98694]: pgmap v19264: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 277 KiB/s wr, 15 op/s 2023-09-10T09:16:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:16:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:16:33.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:16:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:34.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:33 smithi170 ceph-mon[102832]: pgmap v19265: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 11 op/s 2023-09-10T09:16:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:33 smithi082 ceph-mon[98694]: pgmap v19265: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 11 op/s 2023-09-10T09:16:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:33 smithi098 ceph-mon[103124]: pgmap v19265: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 11 op/s 2023-09-10T09:16:35.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35825.858926 secs 2023-09-10T09:16:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35826.211756 secs 2023-09-10T09:16:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35825.858926 secs 2023-09-10T09:16:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35826.211756 secs 2023-09-10T09:16:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35825.858926 secs 2023-09-10T09:16:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35826.211756 secs 2023-09-10T09:16:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:35 smithi170 ceph-mon[102832]: pgmap v19266: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 11 op/s 2023-09-10T09:16:36.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:16:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:35 smithi082 ceph-mon[98694]: pgmap v19266: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 11 op/s 2023-09-10T09:16:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:16:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:35 smithi098 ceph-mon[103124]: pgmap v19266: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 277 KiB/s wr, 11 op/s 2023-09-10T09:16:36.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:16:37.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:16:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:16:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:16:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:16:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:16:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:16:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:38.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:37 smithi170 ceph-mon[102832]: pgmap v19267: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 507 KiB/s wr, 16 op/s 2023-09-10T09:16:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:37 smithi082 ceph-mon[98694]: pgmap v19267: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 507 KiB/s wr, 16 op/s 2023-09-10T09:16:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:37 smithi098 ceph-mon[103124]: pgmap v19267: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 507 KiB/s wr, 16 op/s 2023-09-10T09:16:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35830.858985 secs 2023-09-10T09:16:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35831.211820 secs 2023-09-10T09:16:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:39 smithi098 ceph-mon[103124]: pgmap v19268: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 383 KiB/s wr, 11 op/s 2023-09-10T09:16:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35830.858985 secs 2023-09-10T09:16:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35831.211820 secs 2023-09-10T09:16:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:39 smithi170 ceph-mon[102832]: pgmap v19268: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 383 KiB/s wr, 11 op/s 2023-09-10T09:16:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35830.858985 secs 2023-09-10T09:16:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35831.211820 secs 2023-09-10T09:16:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:39 smithi082 ceph-mon[98694]: pgmap v19268: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 383 KiB/s wr, 11 op/s 2023-09-10T09:16:42.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:42 smithi170 ceph-mon[102832]: pgmap v19269: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 602 KiB/s wr, 17 op/s 2023-09-10T09:16:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:42 smithi082 ceph-mon[98694]: pgmap v19269: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 602 KiB/s wr, 17 op/s 2023-09-10T09:16:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:42 smithi098 ceph-mon[103124]: pgmap v19269: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 602 KiB/s wr, 17 op/s 2023-09-10T09:16:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:44 smithi170 ceph-mon[102832]: pgmap v19270: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 449 KiB/s wr, 12 op/s 2023-09-10T09:16:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:44 smithi082 ceph-mon[98694]: pgmap v19270: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 449 KiB/s wr, 12 op/s 2023-09-10T09:16:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:44 smithi098 ceph-mon[103124]: pgmap v19270: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 449 KiB/s wr, 12 op/s 2023-09-10T09:16:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35835.859042 secs 2023-09-10T09:16:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35836.211888 secs 2023-09-10T09:16:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:16:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35835.859042 secs 2023-09-10T09:16:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35836.211888 secs 2023-09-10T09:16:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:16:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35835.859042 secs 2023-09-10T09:16:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35836.211888 secs 2023-09-10T09:16:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:16:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:46 smithi170 ceph-mon[102832]: pgmap v19271: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 449 KiB/s wr, 12 op/s 2023-09-10T09:16:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:16:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:46 smithi082 ceph-mon[98694]: pgmap v19271: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 449 KiB/s wr, 12 op/s 2023-09-10T09:16:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:16:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:46 smithi098 ceph-mon[103124]: pgmap v19271: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 449 KiB/s wr, 12 op/s 2023-09-10T09:16:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:16:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:48 smithi170 ceph-mon[102832]: pgmap v19272: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 633 KiB/s wr, 19 op/s 2023-09-10T09:16:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:48 smithi082 ceph-mon[98694]: pgmap v19272: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 633 KiB/s wr, 19 op/s 2023-09-10T09:16:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:48 smithi098 ceph-mon[103124]: pgmap v19272: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 633 KiB/s wr, 19 op/s 2023-09-10T09:16:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35840.859101 secs 2023-09-10T09:16:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35841.211954 secs 2023-09-10T09:16:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:49 smithi098 ceph-mon[103124]: pgmap v19273: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 403 KiB/s wr, 13 op/s 2023-09-10T09:16:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35840.859101 secs 2023-09-10T09:16:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35841.211954 secs 2023-09-10T09:16:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:49 smithi170 ceph-mon[102832]: pgmap v19273: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 403 KiB/s wr, 13 op/s 2023-09-10T09:16:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35840.859101 secs 2023-09-10T09:16:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35841.211954 secs 2023-09-10T09:16:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:49 smithi082 ceph-mon[98694]: pgmap v19273: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 403 KiB/s wr, 13 op/s 2023-09-10T09:16:51.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:16:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:16:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:16:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:51 smithi170 ceph-mon[102832]: pgmap v19274: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 714 KiB/s wr, 21 op/s 2023-09-10T09:16:52.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:51 smithi082 ceph-mon[98694]: pgmap v19274: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 714 KiB/s wr, 21 op/s 2023-09-10T09:16:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:51 smithi098 ceph-mon[103124]: pgmap v19274: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 714 KiB/s wr, 21 op/s 2023-09-10T09:16:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:16:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:16:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:16:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:53 smithi170 ceph-mon[102832]: pgmap v19275: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 495 KiB/s wr, 15 op/s 2023-09-10T09:16:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:53 smithi082 ceph-mon[98694]: pgmap v19275: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 495 KiB/s wr, 15 op/s 2023-09-10T09:16:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:53 smithi098 ceph-mon[103124]: pgmap v19275: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 495 KiB/s wr, 15 op/s 2023-09-10T09:16:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35845.859159 secs 2023-09-10T09:16:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35846.212020 secs 2023-09-10T09:16:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35845.859159 secs 2023-09-10T09:16:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35846.212020 secs 2023-09-10T09:16:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35845.859159 secs 2023-09-10T09:16:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35846.212020 secs 2023-09-10T09:16:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:55 smithi170 ceph-mon[102832]: pgmap v19276: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 495 KiB/s wr, 15 op/s 2023-09-10T09:16:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:55 smithi082 ceph-mon[98694]: pgmap v19276: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 495 KiB/s wr, 15 op/s 2023-09-10T09:16:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:55 smithi098 ceph-mon[103124]: pgmap v19276: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 495 KiB/s wr, 15 op/s 2023-09-10T09:16:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:16:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:57 smithi170 ceph-mon[102832]: pgmap v19277: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 672 KiB/s wr, 22 op/s 2023-09-10T09:16:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:16:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:57 smithi082 ceph-mon[98694]: pgmap v19277: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 672 KiB/s wr, 22 op/s 2023-09-10T09:16:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:16:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:16:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:57 smithi098 ceph-mon[103124]: pgmap v19277: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 672 KiB/s wr, 22 op/s 2023-09-10T09:16:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35849.609858 secs 2023-09-10T09:16:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35849.609858 secs 2023-09-10T09:16:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35849.609858 secs 2023-09-10T09:17:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35850.859223 secs 2023-09-10T09:17:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35851.212086 secs 2023-09-10T09:17:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:16:59 smithi170 ceph-mon[102832]: pgmap v19278: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 488 KiB/s wr, 15 op/s 2023-09-10T09:17:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35850.859223 secs 2023-09-10T09:17:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35851.212086 secs 2023-09-10T09:17:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:16:59 smithi082 ceph-mon[98694]: pgmap v19278: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 488 KiB/s wr, 15 op/s 2023-09-10T09:17:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35850.859223 secs 2023-09-10T09:17:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35851.212086 secs 2023-09-10T09:17:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:16:59 smithi098 ceph-mon[103124]: pgmap v19278: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 488 KiB/s wr, 15 op/s 2023-09-10T09:17:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:17:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:17:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:17:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:17:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:17:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:17:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:01 smithi170 ceph-mon[102832]: pgmap v19279: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 768 KiB/s wr, 21 op/s 2023-09-10T09:17:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:01 smithi082 ceph-mon[98694]: pgmap v19279: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 768 KiB/s wr, 21 op/s 2023-09-10T09:17:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:01 smithi098 ceph-mon[103124]: pgmap v19279: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 768 KiB/s wr, 21 op/s 2023-09-10T09:17:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35854.609922 secs 2023-09-10T09:17:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:04 smithi170 ceph-mon[102832]: pgmap v19280: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 457 KiB/s wr, 13 op/s 2023-09-10T09:17:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35854.609922 secs 2023-09-10T09:17:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:04 smithi082 ceph-mon[98694]: pgmap v19280: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 457 KiB/s wr, 13 op/s 2023-09-10T09:17:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35854.609922 secs 2023-09-10T09:17:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:04 smithi098 ceph-mon[103124]: pgmap v19280: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 457 KiB/s wr, 13 op/s 2023-09-10T09:17:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35855.859281 secs 2023-09-10T09:17:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35856.212149 secs 2023-09-10T09:17:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35855.859281 secs 2023-09-10T09:17:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35856.212149 secs 2023-09-10T09:17:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35855.859281 secs 2023-09-10T09:17:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35856.212149 secs 2023-09-10T09:17:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:06 smithi170 ceph-mon[102832]: pgmap v19281: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 457 KiB/s wr, 13 op/s 2023-09-10T09:17:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:06 smithi082 ceph-mon[98694]: pgmap v19281: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 457 KiB/s wr, 13 op/s 2023-09-10T09:17:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:06 smithi098 ceph-mon[103124]: pgmap v19281: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 457 KiB/s wr, 13 op/s 2023-09-10T09:17:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:08 smithi170 ceph-mon[102832]: pgmap v19282: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 605 KiB/s wr, 34 op/s 2023-09-10T09:17:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35859.610039 secs 2023-09-10T09:17:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:08 smithi082 ceph-mon[98694]: pgmap v19282: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 605 KiB/s wr, 34 op/s 2023-09-10T09:17:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35859.610039 secs 2023-09-10T09:17:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:08 smithi098 ceph-mon[103124]: pgmap v19282: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 605 KiB/s wr, 34 op/s 2023-09-10T09:17:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35859.610039 secs 2023-09-10T09:17:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:09 smithi170 ceph-mon[102832]: osdmap e1273: 12 total, 12 up, 12 in 2023-09-10T09:17:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:09 smithi082 ceph-mon[98694]: osdmap e1273: 12 total, 12 up, 12 in 2023-09-10T09:17:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:09 smithi098 ceph-mon[103124]: osdmap e1273: 12 total, 12 up, 12 in 2023-09-10T09:17:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35860.859342 secs 2023-09-10T09:17:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35861.212217 secs 2023-09-10T09:17:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:10 smithi170 ceph-mon[102832]: pgmap v19284: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 514 KiB/s wr, 31 op/s 2023-09-10T09:17:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35860.859342 secs 2023-09-10T09:17:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35861.212217 secs 2023-09-10T09:17:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:10 smithi082 ceph-mon[98694]: pgmap v19284: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 514 KiB/s wr, 31 op/s 2023-09-10T09:17:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35860.859342 secs 2023-09-10T09:17:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35861.212217 secs 2023-09-10T09:17:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:10 smithi098 ceph-mon[103124]: pgmap v19284: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 514 KiB/s wr, 31 op/s 2023-09-10T09:17:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:12 smithi170 ceph-mon[102832]: pgmap v19285: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 276 KiB/s wr, 30 op/s 2023-09-10T09:17:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:12 smithi082 ceph-mon[98694]: pgmap v19285: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 276 KiB/s wr, 30 op/s 2023-09-10T09:17:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:12 smithi098 ceph-mon[103124]: pgmap v19285: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 276 KiB/s wr, 30 op/s 2023-09-10T09:17:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35864.610100 secs 2023-09-10T09:17:13.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35864.610100 secs 2023-09-10T09:17:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35864.610100 secs 2023-09-10T09:17:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:14 smithi170 ceph-mon[102832]: pgmap v19286: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 276 KiB/s wr, 30 op/s 2023-09-10T09:17:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:14 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:17:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:14 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:14 smithi170 ceph-mon[102832]: osdmap e1274: 12 total, 12 up, 12 in 2023-09-10T09:17:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:14 smithi082 ceph-mon[98694]: pgmap v19286: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 276 KiB/s wr, 30 op/s 2023-09-10T09:17:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:14 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:17:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:14 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:14 smithi082 ceph-mon[98694]: osdmap e1274: 12 total, 12 up, 12 in 2023-09-10T09:17:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:14 smithi098 ceph-mon[103124]: pgmap v19286: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 276 KiB/s wr, 30 op/s 2023-09-10T09:17:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:14 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:17:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:14 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:14 smithi098 ceph-mon[103124]: osdmap e1274: 12 total, 12 up, 12 in 2023-09-10T09:17:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35865.859410 secs 2023-09-10T09:17:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35866.212282 secs 2023-09-10T09:17:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35865.859410 secs 2023-09-10T09:17:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35866.212282 secs 2023-09-10T09:17:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35865.859410 secs 2023-09-10T09:17:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35866.212282 secs 2023-09-10T09:17:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:16 smithi170 ceph-mon[102832]: pgmap v19288: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 7 op/s 2023-09-10T09:17:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:16 smithi082 ceph-mon[98694]: pgmap v19288: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 7 op/s 2023-09-10T09:17:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:16 smithi098 ceph-mon[103124]: pgmap v19288: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 122 KiB/s wr, 7 op/s 2023-09-10T09:17:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:17 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:17:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:17 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:17:17.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:17 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:17:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:17 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:17:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:17 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:17:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:17 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:17:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:18 smithi170 ceph-mon[102832]: pgmap v19289: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 13 op/s 2023-09-10T09:17:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35869.610163 secs 2023-09-10T09:17:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:18 smithi082 ceph-mon[98694]: pgmap v19289: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 13 op/s 2023-09-10T09:17:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35869.610163 secs 2023-09-10T09:17:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:18 smithi098 ceph-mon[103124]: pgmap v19289: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 110 KiB/s wr, 13 op/s 2023-09-10T09:17:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35869.610163 secs 2023-09-10T09:17:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35870.859494 secs 2023-09-10T09:17:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35871.212348 secs 2023-09-10T09:17:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:19 smithi170 ceph-mon[102832]: pgmap v19290: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T09:17:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:17:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:17:20.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35870.859494 secs 2023-09-10T09:17:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35871.212348 secs 2023-09-10T09:17:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:19 smithi082 ceph-mon[98694]: pgmap v19290: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T09:17:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:17:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:17:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35870.859494 secs 2023-09-10T09:17:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35871.212348 secs 2023-09-10T09:17:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:19 smithi098 ceph-mon[103124]: pgmap v19290: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T09:17:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:17:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:17:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:17:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:17:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:17:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:17:21.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:17:21.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:17:21.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:17:21.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:17:21.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:20 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T09:17:21.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:20 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T09:17:21.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:17:21.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:17:21.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:17:21.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:17:21.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:17:21.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:17:21.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:17:21.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:17:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:17:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:17:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:17:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:17:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:17:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:17:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:17:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:17:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:20 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T09:17:21.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:20 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T09:17:21.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:17:21.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:17:21.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:17:21.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:17:21.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:17:21.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:17:21.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:17:21.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:17:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:17:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:17:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:17:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:17:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:17:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:17:21.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:17:21.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:17:21.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:20 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T09:17:21.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:20 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T09:17:21.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:17:21.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:17:21.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:17:21.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:17:21.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:17:21.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:17:21.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:17:21.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:17:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:21 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T09:17:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:21 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T09:17:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:21 smithi170 ceph-mon[102832]: pgmap v19291: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T09:17:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:21 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T09:17:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:21 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T09:17:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:21 smithi082 ceph-mon[98694]: pgmap v19291: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T09:17:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:21 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T09:17:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:21 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T09:17:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:21 smithi098 ceph-mon[103124]: pgmap v19291: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T09:17:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35874.610222 secs 2023-09-10T09:17:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35874.610222 secs 2023-09-10T09:17:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35874.610222 secs 2023-09-10T09:17:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:24 smithi170 ceph-mon[102832]: pgmap v19292: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T09:17:24.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:24 smithi082 ceph-mon[98694]: pgmap v19292: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T09:17:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:24 smithi098 ceph-mon[103124]: pgmap v19292: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T09:17:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35875.859551 secs 2023-09-10T09:17:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35876.212416 secs 2023-09-10T09:17:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35875.859551 secs 2023-09-10T09:17:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35876.212416 secs 2023-09-10T09:17:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35875.859551 secs 2023-09-10T09:17:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35876.212416 secs 2023-09-10T09:17:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:26 smithi170 ceph-mon[102832]: pgmap v19293: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T09:17:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:26 smithi082 ceph-mon[98694]: pgmap v19293: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T09:17:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:26 smithi098 ceph-mon[103124]: pgmap v19293: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T09:17:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:28 smithi170 ceph-mon[102832]: pgmap v19294: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 13 op/s 2023-09-10T09:17:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35879.610284 secs 2023-09-10T09:17:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:28 smithi082 ceph-mon[98694]: pgmap v19294: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 13 op/s 2023-09-10T09:17:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35879.610284 secs 2023-09-10T09:17:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:28 smithi098 ceph-mon[103124]: pgmap v19294: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 13 op/s 2023-09-10T09:17:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35879.610284 secs 2023-09-10T09:17:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35880.859611 secs 2023-09-10T09:17:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35881.212489 secs 2023-09-10T09:17:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:29 smithi170 ceph-mon[102832]: pgmap v19295: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T09:17:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35880.859611 secs 2023-09-10T09:17:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35881.212489 secs 2023-09-10T09:17:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:29 smithi082 ceph-mon[98694]: pgmap v19295: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T09:17:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35880.859611 secs 2023-09-10T09:17:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35881.212489 secs 2023-09-10T09:17:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:29 smithi098 ceph-mon[103124]: pgmap v19295: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T09:17:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:32 smithi170 ceph-mon[102832]: pgmap v19296: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 511 B/s wr, 14 op/s 2023-09-10T09:17:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:32 smithi082 ceph-mon[98694]: pgmap v19296: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 511 B/s wr, 14 op/s 2023-09-10T09:17:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:32 smithi098 ceph-mon[103124]: pgmap v19296: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 511 B/s wr, 14 op/s 2023-09-10T09:17:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:34 smithi170 ceph-mon[102832]: pgmap v19297: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 9 op/s 2023-09-10T09:17:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:34 smithi082 ceph-mon[98694]: pgmap v19297: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 9 op/s 2023-09-10T09:17:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:34 smithi098 ceph-mon[103124]: pgmap v19297: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 9 op/s 2023-09-10T09:17:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35885.859670 secs 2023-09-10T09:17:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35886.212555 secs 2023-09-10T09:17:35.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35885.859670 secs 2023-09-10T09:17:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35886.212555 secs 2023-09-10T09:17:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35885.859670 secs 2023-09-10T09:17:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35886.212555 secs 2023-09-10T09:17:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:36 smithi170 ceph-mon[102832]: pgmap v19298: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 9 op/s 2023-09-10T09:17:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:36 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:17:36.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:36 smithi082 ceph-mon[98694]: pgmap v19298: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 9 op/s 2023-09-10T09:17:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:36 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:17:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:36 smithi098 ceph-mon[103124]: pgmap v19298: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 9 op/s 2023-09-10T09:17:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:36 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:17:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:37 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:17:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:17:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:37 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:17:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:17:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:37 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:17:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:17:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:38 smithi170 ceph-mon[102832]: pgmap v19299: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 257 KiB/s wr, 16 op/s 2023-09-10T09:17:38.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:38 smithi082 ceph-mon[98694]: pgmap v19299: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 257 KiB/s wr, 16 op/s 2023-09-10T09:17:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:38 smithi098 ceph-mon[103124]: pgmap v19299: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 257 KiB/s wr, 16 op/s 2023-09-10T09:17:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35890.859731 secs 2023-09-10T09:17:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35891.212620 secs 2023-09-10T09:17:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:39 smithi170 ceph-mon[102832]: pgmap v19300: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 257 KiB/s wr, 11 op/s 2023-09-10T09:17:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35890.859731 secs 2023-09-10T09:17:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35891.212620 secs 2023-09-10T09:17:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:39 smithi082 ceph-mon[98694]: pgmap v19300: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 257 KiB/s wr, 11 op/s 2023-09-10T09:17:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35890.859731 secs 2023-09-10T09:17:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35891.212620 secs 2023-09-10T09:17:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:39 smithi098 ceph-mon[103124]: pgmap v19300: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 257 KiB/s wr, 11 op/s 2023-09-10T09:17:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:17:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:17:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:17:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:41 smithi170 ceph-mon[102832]: pgmap v19301: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 704 KiB/s wr, 18 op/s 2023-09-10T09:17:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:41 smithi082 ceph-mon[98694]: pgmap v19301: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 704 KiB/s wr, 18 op/s 2023-09-10T09:17:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:41 smithi098 ceph-mon[103124]: pgmap v19301: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 704 KiB/s wr, 18 op/s 2023-09-10T09:17:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:44 smithi170 ceph-mon[102832]: pgmap v19302: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 703 KiB/s wr, 12 op/s 2023-09-10T09:17:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:44 smithi082 ceph-mon[98694]: pgmap v19302: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 703 KiB/s wr, 12 op/s 2023-09-10T09:17:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:44 smithi098 ceph-mon[103124]: pgmap v19302: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 703 KiB/s wr, 12 op/s 2023-09-10T09:17:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35895.859790 secs 2023-09-10T09:17:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35896.212685 secs 2023-09-10T09:17:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:17:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35895.859790 secs 2023-09-10T09:17:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35896.212685 secs 2023-09-10T09:17:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:17:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35895.859790 secs 2023-09-10T09:17:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35896.212685 secs 2023-09-10T09:17:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:17:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:46 smithi170 ceph-mon[102832]: pgmap v19303: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 703 KiB/s wr, 12 op/s 2023-09-10T09:17:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:17:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:46 smithi082 ceph-mon[98694]: pgmap v19303: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 703 KiB/s wr, 12 op/s 2023-09-10T09:17:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:17:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:46 smithi098 ceph-mon[103124]: pgmap v19303: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 703 KiB/s wr, 12 op/s 2023-09-10T09:17:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:17:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:48 smithi170 ceph-mon[102832]: pgmap v19304: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 990 KiB/s wr, 21 op/s 2023-09-10T09:17:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:48 smithi082 ceph-mon[98694]: pgmap v19304: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 990 KiB/s wr, 21 op/s 2023-09-10T09:17:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:48 smithi098 ceph-mon[103124]: pgmap v19304: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 990 KiB/s wr, 21 op/s 2023-09-10T09:17:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35900.859850 secs 2023-09-10T09:17:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35901.212751 secs 2023-09-10T09:17:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:49 smithi170 ceph-mon[102832]: pgmap v19305: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 734 KiB/s wr, 14 op/s 2023-09-10T09:17:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35900.859850 secs 2023-09-10T09:17:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35901.212751 secs 2023-09-10T09:17:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:49 smithi082 ceph-mon[98694]: pgmap v19305: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 734 KiB/s wr, 14 op/s 2023-09-10T09:17:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35900.859850 secs 2023-09-10T09:17:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35901.212751 secs 2023-09-10T09:17:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:49 smithi098 ceph-mon[103124]: pgmap v19305: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 734 KiB/s wr, 14 op/s 2023-09-10T09:17:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:17:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:17:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:17:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:51 smithi170 ceph-mon[102832]: pgmap v19306: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 22 op/s 2023-09-10T09:17:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:51 smithi082 ceph-mon[98694]: pgmap v19306: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 22 op/s 2023-09-10T09:17:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:51 smithi098 ceph-mon[103124]: pgmap v19306: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 22 op/s 2023-09-10T09:17:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:17:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:17:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:17:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:53 smithi170 ceph-mon[102832]: pgmap v19307: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 592 KiB/s wr, 15 op/s 2023-09-10T09:17:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:53 smithi082 ceph-mon[98694]: pgmap v19307: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 592 KiB/s wr, 15 op/s 2023-09-10T09:17:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:53 smithi098 ceph-mon[103124]: pgmap v19307: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 592 KiB/s wr, 15 op/s 2023-09-10T09:17:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35905.859909 secs 2023-09-10T09:17:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35906.212820 secs 2023-09-10T09:17:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35905.859909 secs 2023-09-10T09:17:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35906.212820 secs 2023-09-10T09:17:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35905.859909 secs 2023-09-10T09:17:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35906.212820 secs 2023-09-10T09:17:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:55 smithi170 ceph-mon[102832]: pgmap v19308: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 592 KiB/s wr, 15 op/s 2023-09-10T09:17:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:55 smithi082 ceph-mon[98694]: pgmap v19308: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 592 KiB/s wr, 15 op/s 2023-09-10T09:17:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:55 smithi098 ceph-mon[103124]: pgmap v19308: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 592 KiB/s wr, 15 op/s 2023-09-10T09:17:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:17:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:57 smithi170 ceph-mon[102832]: pgmap v19309: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 817 KiB/s wr, 23 op/s 2023-09-10T09:17:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:17:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:57 smithi082 ceph-mon[98694]: pgmap v19309: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 817 KiB/s wr, 23 op/s 2023-09-10T09:17:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:17:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:17:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:57 smithi098 ceph-mon[103124]: pgmap v19309: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 817 KiB/s wr, 23 op/s 2023-09-10T09:17:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35909.610832 secs 2023-09-10T09:17:59.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35909.610832 secs 2023-09-10T09:17:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35909.610832 secs 2023-09-10T09:18:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35910.860009 secs 2023-09-10T09:18:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35911.212890 secs 2023-09-10T09:18:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:17:59 smithi170 ceph-mon[102832]: pgmap v19310: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 530 KiB/s wr, 15 op/s 2023-09-10T09:18:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35910.860009 secs 2023-09-10T09:18:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35911.212890 secs 2023-09-10T09:18:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:17:59 smithi082 ceph-mon[98694]: pgmap v19310: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 530 KiB/s wr, 15 op/s 2023-09-10T09:18:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35910.860009 secs 2023-09-10T09:18:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35911.212890 secs 2023-09-10T09:18:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:17:59 smithi098 ceph-mon[103124]: pgmap v19310: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 530 KiB/s wr, 15 op/s 2023-09-10T09:18:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:18:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:18:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:18:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:18:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:18:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:18:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:01 smithi170 ceph-mon[102832]: pgmap v19311: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 748 KiB/s wr, 23 op/s 2023-09-10T09:18:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:01 smithi082 ceph-mon[98694]: pgmap v19311: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 748 KiB/s wr, 23 op/s 2023-09-10T09:18:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:01 smithi098 ceph-mon[103124]: pgmap v19311: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 748 KiB/s wr, 23 op/s 2023-09-10T09:18:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35914.610910 secs 2023-09-10T09:18:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35914.610910 secs 2023-09-10T09:18:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35914.610910 secs 2023-09-10T09:18:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:04 smithi170 ceph-mon[102832]: pgmap v19312: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 443 KiB/s wr, 16 op/s 2023-09-10T09:18:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:04 smithi082 ceph-mon[98694]: pgmap v19312: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 443 KiB/s wr, 16 op/s 2023-09-10T09:18:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:04 smithi098 ceph-mon[103124]: pgmap v19312: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 443 KiB/s wr, 16 op/s 2023-09-10T09:18:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35915.860098 secs 2023-09-10T09:18:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35916.212951 secs 2023-09-10T09:18:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:18:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35915.860098 secs 2023-09-10T09:18:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35916.212951 secs 2023-09-10T09:18:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:18:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35915.860098 secs 2023-09-10T09:18:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35916.212951 secs 2023-09-10T09:18:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:18:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:06 smithi170 ceph-mon[102832]: pgmap v19313: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 443 KiB/s wr, 16 op/s 2023-09-10T09:18:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:18:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:06 smithi082 ceph-mon[98694]: pgmap v19313: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 443 KiB/s wr, 16 op/s 2023-09-10T09:18:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:18:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:06 smithi098 ceph-mon[103124]: pgmap v19313: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 443 KiB/s wr, 16 op/s 2023-09-10T09:18:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:18:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:08 smithi170 ceph-mon[102832]: pgmap v19314: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 634 KiB/s wr, 36 op/s 2023-09-10T09:18:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35919.611015 secs 2023-09-10T09:18:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:08 smithi082 ceph-mon[98694]: pgmap v19314: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 634 KiB/s wr, 36 op/s 2023-09-10T09:18:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35919.611015 secs 2023-09-10T09:18:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:08 smithi098 ceph-mon[103124]: pgmap v19314: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 634 KiB/s wr, 36 op/s 2023-09-10T09:18:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35919.611015 secs 2023-09-10T09:18:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:09 smithi170 ceph-mon[102832]: osdmap e1275: 12 total, 12 up, 12 in 2023-09-10T09:18:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35920.860156 secs 2023-09-10T09:18:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:09 smithi082 ceph-mon[98694]: osdmap e1275: 12 total, 12 up, 12 in 2023-09-10T09:18:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35920.860156 secs 2023-09-10T09:18:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:09 smithi098 ceph-mon[103124]: osdmap e1275: 12 total, 12 up, 12 in 2023-09-10T09:18:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35920.860156 secs 2023-09-10T09:18:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35921.213018 secs 2023-09-10T09:18:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:10 smithi170 ceph-mon[102832]: pgmap v19316: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 492 KiB/s wr, 35 op/s 2023-09-10T09:18:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35921.213018 secs 2023-09-10T09:18:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:10 smithi082 ceph-mon[98694]: pgmap v19316: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 492 KiB/s wr, 35 op/s 2023-09-10T09:18:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35921.213018 secs 2023-09-10T09:18:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:10 smithi098 ceph-mon[103124]: pgmap v19316: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 492 KiB/s wr, 35 op/s 2023-09-10T09:18:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:12 smithi170 ceph-mon[102832]: pgmap v19317: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 230 KiB/s wr, 29 op/s 2023-09-10T09:18:12.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:12 smithi082 ceph-mon[98694]: pgmap v19317: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 230 KiB/s wr, 29 op/s 2023-09-10T09:18:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:12 smithi098 ceph-mon[103124]: pgmap v19317: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 230 KiB/s wr, 29 op/s 2023-09-10T09:18:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35924.611076 secs 2023-09-10T09:18:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35924.611076 secs 2023-09-10T09:18:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35924.611076 secs 2023-09-10T09:18:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:14 smithi170 ceph-mon[102832]: pgmap v19318: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 230 KiB/s wr, 29 op/s 2023-09-10T09:18:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35925.860212 secs 2023-09-10T09:18:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:14 smithi170 ceph-mon[102832]: osdmap e1276: 12 total, 12 up, 12 in 2023-09-10T09:18:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:14 smithi082 ceph-mon[98694]: pgmap v19318: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 230 KiB/s wr, 29 op/s 2023-09-10T09:18:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35925.860212 secs 2023-09-10T09:18:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:14 smithi082 ceph-mon[98694]: osdmap e1276: 12 total, 12 up, 12 in 2023-09-10T09:18:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:14 smithi098 ceph-mon[103124]: pgmap v19318: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 230 KiB/s wr, 29 op/s 2023-09-10T09:18:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35925.860212 secs 2023-09-10T09:18:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:14 smithi098 ceph-mon[103124]: osdmap e1276: 12 total, 12 up, 12 in 2023-09-10T09:18:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35926.213083 secs 2023-09-10T09:18:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35926.213083 secs 2023-09-10T09:18:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35926.213083 secs 2023-09-10T09:18:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:16 smithi170 ceph-mon[102832]: pgmap v19320: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2023-09-10T09:18:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:16 smithi082 ceph-mon[98694]: pgmap v19320: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2023-09-10T09:18:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:16 smithi098 ceph-mon[103124]: pgmap v19320: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2023-09-10T09:18:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:18 smithi170 ceph-mon[102832]: pgmap v19321: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 115 B/s wr, 12 op/s 2023-09-10T09:18:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35929.611136 secs 2023-09-10T09:18:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:18 smithi082 ceph-mon[98694]: pgmap v19321: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 115 B/s wr, 12 op/s 2023-09-10T09:18:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35929.611136 secs 2023-09-10T09:18:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:18 smithi098 ceph-mon[103124]: pgmap v19321: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 115 B/s wr, 12 op/s 2023-09-10T09:18:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35929.611136 secs 2023-09-10T09:18:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35930.860297 secs 2023-09-10T09:18:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35931.213146 secs 2023-09-10T09:18:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:19 smithi170 ceph-mon[102832]: pgmap v19322: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T09:18:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35930.860297 secs 2023-09-10T09:18:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35931.213146 secs 2023-09-10T09:18:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:19 smithi082 ceph-mon[98694]: pgmap v19322: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T09:18:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35930.860297 secs 2023-09-10T09:18:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35931.213146 secs 2023-09-10T09:18:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:19 smithi098 ceph-mon[103124]: pgmap v19322: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T09:18:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:18:21.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:18:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:18:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:21 smithi170 ceph-mon[102832]: pgmap v19323: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T09:18:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:18:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:18:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:18:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:21 smithi082 ceph-mon[98694]: pgmap v19323: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T09:18:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:18:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:18:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:18:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:21 smithi098 ceph-mon[103124]: pgmap v19323: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T09:18:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:18:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:18:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:18:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35934.611197 secs 2023-09-10T09:18:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35934.611197 secs 2023-09-10T09:18:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35934.611197 secs 2023-09-10T09:18:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:24 smithi170 ceph-mon[102832]: pgmap v19324: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T09:18:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:24 smithi082 ceph-mon[98694]: pgmap v19324: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T09:18:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:24 smithi098 ceph-mon[103124]: pgmap v19324: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T09:18:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35935.860356 secs 2023-09-10T09:18:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35936.213213 secs 2023-09-10T09:18:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35935.860356 secs 2023-09-10T09:18:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35936.213213 secs 2023-09-10T09:18:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35935.860356 secs 2023-09-10T09:18:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35936.213213 secs 2023-09-10T09:18:26.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:26 smithi170 ceph-mon[102832]: pgmap v19325: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T09:18:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:26 smithi082 ceph-mon[98694]: pgmap v19325: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T09:18:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:26 smithi098 ceph-mon[103124]: pgmap v19325: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 9 op/s 2023-09-10T09:18:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:28 smithi170 ceph-mon[102832]: pgmap v19326: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 13 op/s 2023-09-10T09:18:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35939.611273 secs 2023-09-10T09:18:28.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:28 smithi082 ceph-mon[98694]: pgmap v19326: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 13 op/s 2023-09-10T09:18:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35939.611273 secs 2023-09-10T09:18:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:28 smithi098 ceph-mon[103124]: pgmap v19326: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 13 op/s 2023-09-10T09:18:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35939.611273 secs 2023-09-10T09:18:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35940.860415 secs 2023-09-10T09:18:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35941.213277 secs 2023-09-10T09:18:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:29 smithi170 ceph-mon[102832]: pgmap v19327: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T09:18:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35940.860415 secs 2023-09-10T09:18:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35941.213277 secs 2023-09-10T09:18:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:29 smithi082 ceph-mon[98694]: pgmap v19327: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T09:18:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35940.860415 secs 2023-09-10T09:18:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35941.213277 secs 2023-09-10T09:18:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:29 smithi098 ceph-mon[103124]: pgmap v19327: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-10T09:18:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:32 smithi170 ceph-mon[102832]: pgmap v19328: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 426 B/s wr, 14 op/s 2023-09-10T09:18:32.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:32 smithi082 ceph-mon[98694]: pgmap v19328: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 426 B/s wr, 14 op/s 2023-09-10T09:18:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:32 smithi098 ceph-mon[103124]: pgmap v19328: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 426 B/s wr, 14 op/s 2023-09-10T09:18:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:34 smithi170 ceph-mon[102832]: pgmap v19329: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-10T09:18:34.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:34 smithi082 ceph-mon[98694]: pgmap v19329: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-10T09:18:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:34 smithi098 ceph-mon[103124]: pgmap v19329: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-10T09:18:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35945.860472 secs 2023-09-10T09:18:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35946.213344 secs 2023-09-10T09:18:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35945.860472 secs 2023-09-10T09:18:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35946.213344 secs 2023-09-10T09:18:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35945.860472 secs 2023-09-10T09:18:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35946.213344 secs 2023-09-10T09:18:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:36 smithi170 ceph-mon[102832]: pgmap v19330: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-10T09:18:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:36 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:18:36.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:36 smithi082 ceph-mon[98694]: pgmap v19330: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-10T09:18:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:36 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:18:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:36 smithi098 ceph-mon[103124]: pgmap v19330: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-10T09:18:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:36 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:18:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:37 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:18:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:18:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:37.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:37 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:18:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:18:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:37 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:18:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:18:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:38 smithi170 ceph-mon[102832]: pgmap v19331: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 258 KiB/s wr, 15 op/s 2023-09-10T09:18:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:38 smithi082 ceph-mon[98694]: pgmap v19331: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 258 KiB/s wr, 15 op/s 2023-09-10T09:18:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:38 smithi098 ceph-mon[103124]: pgmap v19331: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 258 KiB/s wr, 15 op/s 2023-09-10T09:18:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35950.860557 secs 2023-09-10T09:18:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35951.213408 secs 2023-09-10T09:18:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:39 smithi170 ceph-mon[102832]: pgmap v19332: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 257 KiB/s wr, 11 op/s 2023-09-10T09:18:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35950.860557 secs 2023-09-10T09:18:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35951.213408 secs 2023-09-10T09:18:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:39 smithi082 ceph-mon[98694]: pgmap v19332: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 257 KiB/s wr, 11 op/s 2023-09-10T09:18:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35950.860557 secs 2023-09-10T09:18:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35951.213408 secs 2023-09-10T09:18:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:39 smithi098 ceph-mon[103124]: pgmap v19332: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 257 KiB/s wr, 11 op/s 2023-09-10T09:18:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:18:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:18:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:18:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:41 smithi170 ceph-mon[102832]: pgmap v19333: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 581 KiB/s wr, 18 op/s 2023-09-10T09:18:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:41 smithi082 ceph-mon[98694]: pgmap v19333: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 581 KiB/s wr, 18 op/s 2023-09-10T09:18:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:41 smithi098 ceph-mon[103124]: pgmap v19333: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 581 KiB/s wr, 18 op/s 2023-09-10T09:18:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:44 smithi170 ceph-mon[102832]: pgmap v19334: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 580 KiB/s wr, 13 op/s 2023-09-10T09:18:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:44 smithi082 ceph-mon[98694]: pgmap v19334: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 580 KiB/s wr, 13 op/s 2023-09-10T09:18:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:44 smithi098 ceph-mon[103124]: pgmap v19334: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 580 KiB/s wr, 13 op/s 2023-09-10T09:18:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35955.860642 secs 2023-09-10T09:18:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35956.213479 secs 2023-09-10T09:18:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:18:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35955.860642 secs 2023-09-10T09:18:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35956.213479 secs 2023-09-10T09:18:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:18:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35955.860642 secs 2023-09-10T09:18:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35956.213479 secs 2023-09-10T09:18:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:18:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:46 smithi170 ceph-mon[102832]: pgmap v19335: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 580 KiB/s wr, 13 op/s 2023-09-10T09:18:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:18:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:46 smithi082 ceph-mon[98694]: pgmap v19335: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 580 KiB/s wr, 13 op/s 2023-09-10T09:18:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:18:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:46 smithi098 ceph-mon[103124]: pgmap v19335: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 580 KiB/s wr, 13 op/s 2023-09-10T09:18:46.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:18:46.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:48 smithi170 ceph-mon[102832]: pgmap v19336: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 744 KiB/s wr, 20 op/s 2023-09-10T09:18:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:48 smithi082 ceph-mon[98694]: pgmap v19336: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 744 KiB/s wr, 20 op/s 2023-09-10T09:18:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:48 smithi098 ceph-mon[103124]: pgmap v19336: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 744 KiB/s wr, 20 op/s 2023-09-10T09:18:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35960.860701 secs 2023-09-10T09:18:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35961.213544 secs 2023-09-10T09:18:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:49 smithi170 ceph-mon[102832]: pgmap v19337: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 486 KiB/s wr, 14 op/s 2023-09-10T09:18:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35960.860701 secs 2023-09-10T09:18:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35961.213544 secs 2023-09-10T09:18:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:49 smithi082 ceph-mon[98694]: pgmap v19337: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 486 KiB/s wr, 14 op/s 2023-09-10T09:18:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35960.860701 secs 2023-09-10T09:18:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35961.213544 secs 2023-09-10T09:18:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:49 smithi098 ceph-mon[103124]: pgmap v19337: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 486 KiB/s wr, 14 op/s 2023-09-10T09:18:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:52 smithi170 ceph-mon[102832]: pgmap v19338: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 799 KiB/s wr, 22 op/s 2023-09-10T09:18:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:52 smithi082 ceph-mon[98694]: pgmap v19338: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 799 KiB/s wr, 22 op/s 2023-09-10T09:18:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:52 smithi098 ceph-mon[103124]: pgmap v19338: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 799 KiB/s wr, 22 op/s 2023-09-10T09:18:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:18:53.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:53.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:18:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:18:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:54 smithi170 ceph-mon[102832]: pgmap v19339: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 476 KiB/s wr, 15 op/s 2023-09-10T09:18:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:18:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:54 smithi082 ceph-mon[98694]: pgmap v19339: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 476 KiB/s wr, 15 op/s 2023-09-10T09:18:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:18:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:54 smithi098 ceph-mon[103124]: pgmap v19339: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 476 KiB/s wr, 15 op/s 2023-09-10T09:18:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:18:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35965.860758 secs 2023-09-10T09:18:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35966.213611 secs 2023-09-10T09:18:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35965.860758 secs 2023-09-10T09:18:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35966.213611 secs 2023-09-10T09:18:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35965.860758 secs 2023-09-10T09:18:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35966.213611 secs 2023-09-10T09:18:56.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:56 smithi170 ceph-mon[102832]: pgmap v19340: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 476 KiB/s wr, 15 op/s 2023-09-10T09:18:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:56 smithi082 ceph-mon[98694]: pgmap v19340: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 476 KiB/s wr, 15 op/s 2023-09-10T09:18:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:56 smithi098 ceph-mon[103124]: pgmap v19340: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 476 KiB/s wr, 15 op/s 2023-09-10T09:18:58.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:58 smithi170 ceph-mon[102832]: pgmap v19341: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 653 KiB/s wr, 21 op/s 2023-09-10T09:18:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:18:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35969.611674 secs 2023-09-10T09:18:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:58 smithi082 ceph-mon[98694]: pgmap v19341: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 653 KiB/s wr, 21 op/s 2023-09-10T09:18:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:18:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35969.611674 secs 2023-09-10T09:18:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:58 smithi098 ceph-mon[103124]: pgmap v19341: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 653 KiB/s wr, 21 op/s 2023-09-10T09:18:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:18:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:18:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35969.611674 secs 2023-09-10T09:19:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35970.860842 secs 2023-09-10T09:19:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35971.213676 secs 2023-09-10T09:19:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:18:59 smithi170 ceph-mon[102832]: pgmap v19342: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 490 KiB/s wr, 13 op/s 2023-09-10T09:19:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35970.860842 secs 2023-09-10T09:19:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35971.213676 secs 2023-09-10T09:19:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:18:59 smithi082 ceph-mon[98694]: pgmap v19342: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 490 KiB/s wr, 13 op/s 2023-09-10T09:19:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35970.860842 secs 2023-09-10T09:19:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35971.213676 secs 2023-09-10T09:19:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:18:59 smithi098 ceph-mon[103124]: pgmap v19342: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 490 KiB/s wr, 13 op/s 2023-09-10T09:19:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:19:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:19:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:19:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:19:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:19:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:19:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:01 smithi170 ceph-mon[102832]: pgmap v19343: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 772 KiB/s wr, 20 op/s 2023-09-10T09:19:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:01 smithi082 ceph-mon[98694]: pgmap v19343: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 772 KiB/s wr, 20 op/s 2023-09-10T09:19:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:01 smithi098 ceph-mon[103124]: pgmap v19343: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 772 KiB/s wr, 20 op/s 2023-09-10T09:19:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35974.611739 secs 2023-09-10T09:19:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35974.611739 secs 2023-09-10T09:19:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35974.611739 secs 2023-09-10T09:19:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:04 smithi170 ceph-mon[102832]: pgmap v19344: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 459 KiB/s wr, 12 op/s 2023-09-10T09:19:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:04 smithi082 ceph-mon[98694]: pgmap v19344: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 459 KiB/s wr, 12 op/s 2023-09-10T09:19:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:04 smithi098 ceph-mon[103124]: pgmap v19344: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 459 KiB/s wr, 12 op/s 2023-09-10T09:19:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35975.860931 secs 2023-09-10T09:19:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35976.213741 secs 2023-09-10T09:19:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35975.860931 secs 2023-09-10T09:19:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35976.213741 secs 2023-09-10T09:19:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35975.860931 secs 2023-09-10T09:19:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35976.213741 secs 2023-09-10T09:19:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:06 smithi170 ceph-mon[102832]: pgmap v19345: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 459 KiB/s wr, 12 op/s 2023-09-10T09:19:06.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:06 smithi082 ceph-mon[98694]: pgmap v19345: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 459 KiB/s wr, 12 op/s 2023-09-10T09:19:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:06 smithi098 ceph-mon[103124]: pgmap v19345: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 459 KiB/s wr, 12 op/s 2023-09-10T09:19:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:08 smithi170 ceph-mon[102832]: pgmap v19346: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 608 KiB/s wr, 32 op/s 2023-09-10T09:19:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35979.611775 secs 2023-09-10T09:19:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:08 smithi082 ceph-mon[98694]: pgmap v19346: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 608 KiB/s wr, 32 op/s 2023-09-10T09:19:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35979.611775 secs 2023-09-10T09:19:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:08 smithi098 ceph-mon[103124]: pgmap v19346: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 608 KiB/s wr, 32 op/s 2023-09-10T09:19:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35979.611775 secs 2023-09-10T09:19:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:09 smithi170 ceph-mon[102832]: osdmap e1277: 12 total, 12 up, 12 in 2023-09-10T09:19:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35980.860994 secs 2023-09-10T09:19:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:09 smithi082 ceph-mon[98694]: osdmap e1277: 12 total, 12 up, 12 in 2023-09-10T09:19:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35980.860994 secs 2023-09-10T09:19:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:09 smithi098 ceph-mon[103124]: osdmap e1277: 12 total, 12 up, 12 in 2023-09-10T09:19:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35980.860994 secs 2023-09-10T09:19:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35981.213808 secs 2023-09-10T09:19:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:10 smithi170 ceph-mon[102832]: pgmap v19348: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 516 KiB/s wr, 31 op/s 2023-09-10T09:19:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35981.213808 secs 2023-09-10T09:19:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:10 smithi082 ceph-mon[98694]: pgmap v19348: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 516 KiB/s wr, 31 op/s 2023-09-10T09:19:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35981.213808 secs 2023-09-10T09:19:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:10 smithi098 ceph-mon[103124]: pgmap v19348: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 516 KiB/s wr, 31 op/s 2023-09-10T09:19:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:12 smithi170 ceph-mon[102832]: pgmap v19349: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 276 KiB/s wr, 30 op/s 2023-09-10T09:19:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:12 smithi082 ceph-mon[98694]: pgmap v19349: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 276 KiB/s wr, 30 op/s 2023-09-10T09:19:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:12 smithi098 ceph-mon[103124]: pgmap v19349: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 276 KiB/s wr, 30 op/s 2023-09-10T09:19:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35984.611836 secs 2023-09-10T09:19:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35984.611836 secs 2023-09-10T09:19:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35984.611836 secs 2023-09-10T09:19:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:14 smithi170 ceph-mon[102832]: pgmap v19350: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 276 KiB/s wr, 30 op/s 2023-09-10T09:19:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35985.861053 secs 2023-09-10T09:19:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:14 smithi170 ceph-mon[102832]: osdmap e1278: 12 total, 12 up, 12 in 2023-09-10T09:19:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:14 smithi082 ceph-mon[98694]: pgmap v19350: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 276 KiB/s wr, 30 op/s 2023-09-10T09:19:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35985.861053 secs 2023-09-10T09:19:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:14 smithi082 ceph-mon[98694]: osdmap e1278: 12 total, 12 up, 12 in 2023-09-10T09:19:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:14 smithi098 ceph-mon[103124]: pgmap v19350: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 276 KiB/s wr, 30 op/s 2023-09-10T09:19:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35985.861053 secs 2023-09-10T09:19:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:14 smithi098 ceph-mon[103124]: osdmap e1278: 12 total, 12 up, 12 in 2023-09-10T09:19:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35986.213876 secs 2023-09-10T09:19:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35986.213876 secs 2023-09-10T09:19:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35986.213876 secs 2023-09-10T09:19:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:16 smithi170 ceph-mon[102832]: pgmap v19352: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 7 op/s 2023-09-10T09:19:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:19:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:16 smithi082 ceph-mon[98694]: pgmap v19352: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 7 op/s 2023-09-10T09:19:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:19:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:16 smithi098 ceph-mon[103124]: pgmap v19352: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 7 op/s 2023-09-10T09:19:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:19:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:18 smithi170 ceph-mon[102832]: pgmap v19353: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 14 op/s 2023-09-10T09:19:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:19:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35989.611943 secs 2023-09-10T09:19:18.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:18 smithi082 ceph-mon[98694]: pgmap v19353: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 14 op/s 2023-09-10T09:19:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:19:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35989.611943 secs 2023-09-10T09:19:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:18 smithi098 ceph-mon[103124]: pgmap v19353: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 14 op/s 2023-09-10T09:19:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:19:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35989.611943 secs 2023-09-10T09:19:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35990.861113 secs 2023-09-10T09:19:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35991.213944 secs 2023-09-10T09:19:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:19 smithi170 ceph-mon[102832]: pgmap v19354: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T09:19:20.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35990.861113 secs 2023-09-10T09:19:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35991.213944 secs 2023-09-10T09:19:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:19 smithi082 ceph-mon[98694]: pgmap v19354: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T09:19:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35990.861113 secs 2023-09-10T09:19:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35991.213944 secs 2023-09-10T09:19:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:19 smithi098 ceph-mon[103124]: pgmap v19354: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 98 KiB/s wr, 12 op/s 2023-09-10T09:19:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:22 smithi170 ceph-mon[102832]: pgmap v19355: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 12 op/s 2023-09-10T09:19:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:19:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:19:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:22 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:19:22.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:22 smithi082 ceph-mon[98694]: pgmap v19355: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 12 op/s 2023-09-10T09:19:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:19:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:19:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:22 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:19:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:22 smithi098 ceph-mon[103124]: pgmap v19355: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 12 op/s 2023-09-10T09:19:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:19:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:19:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:22 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:19:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:19:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:19:23.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:19:23.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35994.612035 secs 2023-09-10T09:19:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:19:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:19:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:19:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35994.612035 secs 2023-09-10T09:19:23.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:19:23.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:19:23.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:19:23.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35994.612035 secs 2023-09-10T09:19:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:24 smithi098 ceph-mon[103124]: pgmap v19356: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 12 op/s 2023-09-10T09:19:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35995.861176 secs 2023-09-10T09:19:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 35996.214009 secs 2023-09-10T09:19:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:24 smithi170 ceph-mon[102832]: pgmap v19356: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 12 op/s 2023-09-10T09:19:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35995.861176 secs 2023-09-10T09:19:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 35996.214009 secs 2023-09-10T09:19:24.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:24 smithi082 ceph-mon[98694]: pgmap v19356: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 12 op/s 2023-09-10T09:19:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35995.861176 secs 2023-09-10T09:19:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 35996.214009 secs 2023-09-10T09:19:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:26 smithi098 ceph-mon[103124]: pgmap v19357: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 182 B/s wr, 11 op/s 2023-09-10T09:19:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:26 smithi170 ceph-mon[102832]: pgmap v19357: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 182 B/s wr, 11 op/s 2023-09-10T09:19:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:26 smithi082 ceph-mon[98694]: pgmap v19357: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 182 B/s wr, 11 op/s 2023-09-10T09:19:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:28 smithi098 ceph-mon[103124]: pgmap v19358: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 125 KiB/s wr, 15 op/s 2023-09-10T09:19:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 35999.612093 secs 2023-09-10T09:19:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:28 smithi170 ceph-mon[102832]: pgmap v19358: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 125 KiB/s wr, 15 op/s 2023-09-10T09:19:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 35999.612093 secs 2023-09-10T09:19:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:28 smithi082 ceph-mon[98694]: pgmap v19358: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 125 KiB/s wr, 15 op/s 2023-09-10T09:19:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 35999.612093 secs 2023-09-10T09:19:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36000.861235 secs 2023-09-10T09:19:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36001.214078 secs 2023-09-10T09:19:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:29 smithi170 ceph-mon[102832]: pgmap v19359: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 10 op/s 2023-09-10T09:19:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36000.861235 secs 2023-09-10T09:19:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36001.214078 secs 2023-09-10T09:19:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:29 smithi082 ceph-mon[98694]: pgmap v19359: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 10 op/s 2023-09-10T09:19:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36000.861235 secs 2023-09-10T09:19:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36001.214078 secs 2023-09-10T09:19:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:29 smithi098 ceph-mon[103124]: pgmap v19359: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 10 op/s 2023-09-10T09:19:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:32 smithi170 ceph-mon[102832]: pgmap v19360: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 125 KiB/s wr, 15 op/s 2023-09-10T09:19:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:32 smithi082 ceph-mon[98694]: pgmap v19360: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 125 KiB/s wr, 15 op/s 2023-09-10T09:19:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:32 smithi098 ceph-mon[103124]: pgmap v19360: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 125 KiB/s wr, 15 op/s 2023-09-10T09:19:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:34 smithi170 ceph-mon[102832]: pgmap v19361: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 10 op/s 2023-09-10T09:19:34.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:34 smithi082 ceph-mon[98694]: pgmap v19361: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 10 op/s 2023-09-10T09:19:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:34 smithi098 ceph-mon[103124]: pgmap v19361: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 10 op/s 2023-09-10T09:19:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36005.861295 secs 2023-09-10T09:19:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36006.214145 secs 2023-09-10T09:19:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36005.861295 secs 2023-09-10T09:19:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36006.214145 secs 2023-09-10T09:19:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36005.861295 secs 2023-09-10T09:19:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36006.214145 secs 2023-09-10T09:19:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:36 smithi170 ceph-mon[102832]: pgmap v19362: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 10 op/s 2023-09-10T09:19:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:36 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:19:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:36 smithi082 ceph-mon[98694]: pgmap v19362: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 10 op/s 2023-09-10T09:19:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:36 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:19:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:36 smithi098 ceph-mon[103124]: pgmap v19362: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 125 KiB/s wr, 10 op/s 2023-09-10T09:19:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:36 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:19:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:37 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:19:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:19:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:37.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:37 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:19:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:19:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:37 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:19:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:19:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:38 smithi170 ceph-mon[102832]: pgmap v19363: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 383 KiB/s wr, 16 op/s 2023-09-10T09:19:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:38 smithi082 ceph-mon[98694]: pgmap v19363: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 383 KiB/s wr, 16 op/s 2023-09-10T09:19:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:38 smithi098 ceph-mon[103124]: pgmap v19363: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 383 KiB/s wr, 16 op/s 2023-09-10T09:19:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36010.861384 secs 2023-09-10T09:19:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36011.214210 secs 2023-09-10T09:19:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:39 smithi170 ceph-mon[102832]: pgmap v19364: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 258 KiB/s wr, 11 op/s 2023-09-10T09:19:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36010.861384 secs 2023-09-10T09:19:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36011.214210 secs 2023-09-10T09:19:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:39 smithi082 ceph-mon[98694]: pgmap v19364: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 258 KiB/s wr, 11 op/s 2023-09-10T09:19:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36010.861384 secs 2023-09-10T09:19:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36011.214210 secs 2023-09-10T09:19:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:39 smithi098 ceph-mon[103124]: pgmap v19364: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 258 KiB/s wr, 11 op/s 2023-09-10T09:19:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:19:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:19:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:19:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:41 smithi170 ceph-mon[102832]: pgmap v19365: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 664 KiB/s wr, 18 op/s 2023-09-10T09:19:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:41 smithi082 ceph-mon[98694]: pgmap v19365: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 664 KiB/s wr, 18 op/s 2023-09-10T09:19:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:41 smithi098 ceph-mon[103124]: pgmap v19365: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 664 KiB/s wr, 18 op/s 2023-09-10T09:19:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:44 smithi170 ceph-mon[102832]: pgmap v19366: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 663 KiB/s wr, 12 op/s 2023-09-10T09:19:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:44 smithi082 ceph-mon[98694]: pgmap v19366: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 663 KiB/s wr, 12 op/s 2023-09-10T09:19:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:44 smithi098 ceph-mon[103124]: pgmap v19366: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 663 KiB/s wr, 12 op/s 2023-09-10T09:19:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36015.861462 secs 2023-09-10T09:19:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36016.214277 secs 2023-09-10T09:19:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:19:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36015.861462 secs 2023-09-10T09:19:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36016.214277 secs 2023-09-10T09:19:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:19:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36015.861462 secs 2023-09-10T09:19:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36016.214277 secs 2023-09-10T09:19:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:19:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:46 smithi170 ceph-mon[102832]: pgmap v19367: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 663 KiB/s wr, 12 op/s 2023-09-10T09:19:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:19:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:46 smithi082 ceph-mon[98694]: pgmap v19367: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 663 KiB/s wr, 12 op/s 2023-09-10T09:19:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:19:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:46 smithi098 ceph-mon[103124]: pgmap v19367: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 663 KiB/s wr, 12 op/s 2023-09-10T09:19:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:19:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:48 smithi170 ceph-mon[102832]: pgmap v19368: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 849 KiB/s wr, 19 op/s 2023-09-10T09:19:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:48 smithi082 ceph-mon[98694]: pgmap v19368: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 849 KiB/s wr, 19 op/s 2023-09-10T09:19:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:48 smithi098 ceph-mon[103124]: pgmap v19368: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 849 KiB/s wr, 19 op/s 2023-09-10T09:19:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36020.861495 secs 2023-09-10T09:19:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36021.214343 secs 2023-09-10T09:19:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:49 smithi170 ceph-mon[102832]: pgmap v19369: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 591 KiB/s wr, 13 op/s 2023-09-10T09:19:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36020.861495 secs 2023-09-10T09:19:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36021.214343 secs 2023-09-10T09:19:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:49 smithi082 ceph-mon[98694]: pgmap v19369: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 591 KiB/s wr, 13 op/s 2023-09-10T09:19:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36020.861495 secs 2023-09-10T09:19:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36021.214343 secs 2023-09-10T09:19:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:49 smithi098 ceph-mon[103124]: pgmap v19369: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 591 KiB/s wr, 13 op/s 2023-09-10T09:19:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:19:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:19:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:19:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:51 smithi170 ceph-mon[102832]: pgmap v19370: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 926 KiB/s wr, 20 op/s 2023-09-10T09:19:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:51 smithi082 ceph-mon[98694]: pgmap v19370: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 926 KiB/s wr, 20 op/s 2023-09-10T09:19:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:51 smithi098 ceph-mon[103124]: pgmap v19370: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 926 KiB/s wr, 20 op/s 2023-09-10T09:19:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:19:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:19:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:19:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:53 smithi170 ceph-mon[102832]: pgmap v19371: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 520 KiB/s wr, 14 op/s 2023-09-10T09:19:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:53 smithi082 ceph-mon[98694]: pgmap v19371: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 520 KiB/s wr, 14 op/s 2023-09-10T09:19:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:53 smithi098 ceph-mon[103124]: pgmap v19371: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 520 KiB/s wr, 14 op/s 2023-09-10T09:19:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36025.861522 secs 2023-09-10T09:19:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36026.214410 secs 2023-09-10T09:19:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36025.861522 secs 2023-09-10T09:19:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36026.214410 secs 2023-09-10T09:19:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36025.861522 secs 2023-09-10T09:19:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36026.214410 secs 2023-09-10T09:19:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:55 smithi170 ceph-mon[102832]: pgmap v19372: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 520 KiB/s wr, 14 op/s 2023-09-10T09:19:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:55 smithi082 ceph-mon[98694]: pgmap v19372: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 520 KiB/s wr, 14 op/s 2023-09-10T09:19:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:55 smithi098 ceph-mon[103124]: pgmap v19372: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 520 KiB/s wr, 14 op/s 2023-09-10T09:19:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:19:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:57 smithi170 ceph-mon[102832]: pgmap v19373: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 616 KiB/s wr, 20 op/s 2023-09-10T09:19:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:19:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:57 smithi082 ceph-mon[98694]: pgmap v19373: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 616 KiB/s wr, 20 op/s 2023-09-10T09:19:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:19:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:19:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:57 smithi098 ceph-mon[103124]: pgmap v19373: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 616 KiB/s wr, 20 op/s 2023-09-10T09:19:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36029.612524 secs 2023-09-10T09:19:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36029.612524 secs 2023-09-10T09:19:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36029.612524 secs 2023-09-10T09:20:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36030.861581 secs 2023-09-10T09:20:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36031.214481 secs 2023-09-10T09:20:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:19:59 smithi170 ceph-mon[102832]: pgmap v19374: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 431 KiB/s wr, 13 op/s 2023-09-10T09:20:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36030.861581 secs 2023-09-10T09:20:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36031.214481 secs 2023-09-10T09:20:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:19:59 smithi082 ceph-mon[98694]: pgmap v19374: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 431 KiB/s wr, 13 op/s 2023-09-10T09:20:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36030.861581 secs 2023-09-10T09:20:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36031.214481 secs 2023-09-10T09:20:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:19:59 smithi098 ceph-mon[103124]: pgmap v19374: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 431 KiB/s wr, 13 op/s 2023-09-10T09:20:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:20:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:20:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T09:20:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:20:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:20:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T09:20:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:20:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:20:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T09:20:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:01 smithi170 ceph-mon[102832]: pgmap v19375: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 757 KiB/s wr, 26 op/s 2023-09-10T09:20:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:01 smithi082 ceph-mon[98694]: pgmap v19375: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 757 KiB/s wr, 26 op/s 2023-09-10T09:20:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:01 smithi098 ceph-mon[103124]: pgmap v19375: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 757 KiB/s wr, 26 op/s 2023-09-10T09:20:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36034.612583 secs 2023-09-10T09:20:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36034.612583 secs 2023-09-10T09:20:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36034.612583 secs 2023-09-10T09:20:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:04 smithi170 ceph-mon[102832]: pgmap v19376: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 422 KiB/s wr, 19 op/s 2023-09-10T09:20:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:04 smithi082 ceph-mon[98694]: pgmap v19376: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 422 KiB/s wr, 19 op/s 2023-09-10T09:20:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:04 smithi098 ceph-mon[103124]: pgmap v19376: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 422 KiB/s wr, 19 op/s 2023-09-10T09:20:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36035.861649 secs 2023-09-10T09:20:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36036.214547 secs 2023-09-10T09:20:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36035.861649 secs 2023-09-10T09:20:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36036.214547 secs 2023-09-10T09:20:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36035.861649 secs 2023-09-10T09:20:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36036.214547 secs 2023-09-10T09:20:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:06 smithi170 ceph-mon[102832]: pgmap v19377: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 422 KiB/s wr, 19 op/s 2023-09-10T09:20:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:06 smithi082 ceph-mon[98694]: pgmap v19377: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 422 KiB/s wr, 19 op/s 2023-09-10T09:20:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:06 smithi098 ceph-mon[103124]: pgmap v19377: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 422 KiB/s wr, 19 op/s 2023-09-10T09:20:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:08 smithi170 ceph-mon[102832]: pgmap v19378: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 549 KiB/s wr, 33 op/s 2023-09-10T09:20:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36039.612642 secs 2023-09-10T09:20:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:08 smithi082 ceph-mon[98694]: pgmap v19378: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 549 KiB/s wr, 33 op/s 2023-09-10T09:20:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36039.612642 secs 2023-09-10T09:20:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:08 smithi098 ceph-mon[103124]: pgmap v19378: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 549 KiB/s wr, 33 op/s 2023-09-10T09:20:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36039.612642 secs 2023-09-10T09:20:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:09 smithi170 ceph-mon[102832]: osdmap e1279: 12 total, 12 up, 12 in 2023-09-10T09:20:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36040.861732 secs 2023-09-10T09:20:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:09 smithi082 ceph-mon[98694]: osdmap e1279: 12 total, 12 up, 12 in 2023-09-10T09:20:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36040.861732 secs 2023-09-10T09:20:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:09 smithi098 ceph-mon[103124]: osdmap e1279: 12 total, 12 up, 12 in 2023-09-10T09:20:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36040.861732 secs 2023-09-10T09:20:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36041.214615 secs 2023-09-10T09:20:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:10 smithi170 ceph-mon[102832]: pgmap v19380: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 543 KiB/s wr, 32 op/s 2023-09-10T09:20:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36041.214615 secs 2023-09-10T09:20:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:10 smithi082 ceph-mon[98694]: pgmap v19380: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 543 KiB/s wr, 32 op/s 2023-09-10T09:20:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36041.214615 secs 2023-09-10T09:20:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:10 smithi098 ceph-mon[103124]: pgmap v19380: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 543 KiB/s wr, 32 op/s 2023-09-10T09:20:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:12 smithi170 ceph-mon[102832]: pgmap v19381: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 152 KiB/s wr, 23 op/s 2023-09-10T09:20:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:12 smithi082 ceph-mon[98694]: pgmap v19381: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 152 KiB/s wr, 23 op/s 2023-09-10T09:20:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:12 smithi098 ceph-mon[103124]: pgmap v19381: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 152 KiB/s wr, 23 op/s 2023-09-10T09:20:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36044.612705 secs 2023-09-10T09:20:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36044.612705 secs 2023-09-10T09:20:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36044.612705 secs 2023-09-10T09:20:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:14 smithi170 ceph-mon[102832]: pgmap v19382: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 152 KiB/s wr, 23 op/s 2023-09-10T09:20:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36045.861792 secs 2023-09-10T09:20:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:14 smithi170 ceph-mon[102832]: osdmap e1280: 12 total, 12 up, 12 in 2023-09-10T09:20:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:14 smithi082 ceph-mon[98694]: pgmap v19382: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 152 KiB/s wr, 23 op/s 2023-09-10T09:20:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36045.861792 secs 2023-09-10T09:20:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:14 smithi082 ceph-mon[98694]: osdmap e1280: 12 total, 12 up, 12 in 2023-09-10T09:20:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:14 smithi098 ceph-mon[103124]: pgmap v19382: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 152 KiB/s wr, 23 op/s 2023-09-10T09:20:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36045.861792 secs 2023-09-10T09:20:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:14 smithi098 ceph-mon[103124]: osdmap e1280: 12 total, 12 up, 12 in 2023-09-10T09:20:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36046.214683 secs 2023-09-10T09:20:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36046.214683 secs 2023-09-10T09:20:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36046.214683 secs 2023-09-10T09:20:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:16 smithi170 ceph-mon[102832]: pgmap v19384: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 8 op/s 2023-09-10T09:20:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:20:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:16 smithi082 ceph-mon[98694]: pgmap v19384: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 8 op/s 2023-09-10T09:20:16.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:20:16.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:16 smithi098 ceph-mon[103124]: pgmap v19384: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 8 op/s 2023-09-10T09:20:16.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:20:16.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:18 smithi170 ceph-mon[102832]: pgmap v19385: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 459 B/s wr, 15 op/s 2023-09-10T09:20:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:20:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36049.612769 secs 2023-09-10T09:20:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:18 smithi082 ceph-mon[98694]: pgmap v19385: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 459 B/s wr, 15 op/s 2023-09-10T09:20:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:20:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36049.612769 secs 2023-09-10T09:20:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:18 smithi098 ceph-mon[103124]: pgmap v19385: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 459 B/s wr, 15 op/s 2023-09-10T09:20:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:20:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36049.612769 secs 2023-09-10T09:20:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36050.861851 secs 2023-09-10T09:20:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36051.214748 secs 2023-09-10T09:20:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:19 smithi170 ceph-mon[102832]: pgmap v19386: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 409 B/s wr, 13 op/s 2023-09-10T09:20:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36050.861851 secs 2023-09-10T09:20:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36051.214748 secs 2023-09-10T09:20:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:19 smithi082 ceph-mon[98694]: pgmap v19386: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 409 B/s wr, 13 op/s 2023-09-10T09:20:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36050.861851 secs 2023-09-10T09:20:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36051.214748 secs 2023-09-10T09:20:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:19 smithi098 ceph-mon[103124]: pgmap v19386: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 409 B/s wr, 13 op/s 2023-09-10T09:20:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:22 smithi170 ceph-mon[102832]: pgmap v19387: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T09:20:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:22 smithi082 ceph-mon[98694]: pgmap v19387: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T09:20:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:22 smithi098 ceph-mon[103124]: pgmap v19387: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T09:20:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:20:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36054.612870 secs 2023-09-10T09:20:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:20:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:20:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:23 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:20:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:20:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36054.612870 secs 2023-09-10T09:20:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:20:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:20:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:23 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:20:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:20:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36054.612870 secs 2023-09-10T09:20:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:20:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:20:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:23 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:20:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:24 smithi170 ceph-mon[102832]: pgmap v19388: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T09:20:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:20:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:20:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:20:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:20:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:24 smithi082 ceph-mon[98694]: pgmap v19388: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T09:20:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:20:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:20:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:20:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:20:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:24 smithi098 ceph-mon[103124]: pgmap v19388: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 11 op/s 2023-09-10T09:20:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:20:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:20:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:20:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:20:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36055.861912 secs 2023-09-10T09:20:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36056.214813 secs 2023-09-10T09:20:25.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36055.861912 secs 2023-09-10T09:20:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36056.214813 secs 2023-09-10T09:20:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36055.861912 secs 2023-09-10T09:20:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36056.214813 secs 2023-09-10T09:20:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:26 smithi170 ceph-mon[102832]: pgmap v19389: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 273 B/s wr, 10 op/s 2023-09-10T09:20:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:26 smithi082 ceph-mon[98694]: pgmap v19389: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 273 B/s wr, 10 op/s 2023-09-10T09:20:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:26 smithi098 ceph-mon[103124]: pgmap v19389: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 273 B/s wr, 10 op/s 2023-09-10T09:20:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:28 smithi170 ceph-mon[102832]: pgmap v19390: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 14 op/s 2023-09-10T09:20:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36059.612987 secs 2023-09-10T09:20:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:28 smithi082 ceph-mon[98694]: pgmap v19390: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 14 op/s 2023-09-10T09:20:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36059.612987 secs 2023-09-10T09:20:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:28 smithi098 ceph-mon[103124]: pgmap v19390: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 14 op/s 2023-09-10T09:20:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36059.612987 secs 2023-09-10T09:20:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36060.862001 secs 2023-09-10T09:20:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36061.214879 secs 2023-09-10T09:20:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:29 smithi170 ceph-mon[102832]: pgmap v19391: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2023-09-10T09:20:30.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36060.862001 secs 2023-09-10T09:20:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36061.214879 secs 2023-09-10T09:20:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:29 smithi082 ceph-mon[98694]: pgmap v19391: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2023-09-10T09:20:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36060.862001 secs 2023-09-10T09:20:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36061.214879 secs 2023-09-10T09:20:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:29 smithi098 ceph-mon[103124]: pgmap v19391: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2023-09-10T09:20:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:32 smithi170 ceph-mon[102832]: pgmap v19392: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 14 op/s 2023-09-10T09:20:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:32 smithi082 ceph-mon[98694]: pgmap v19392: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 14 op/s 2023-09-10T09:20:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:32 smithi098 ceph-mon[103124]: pgmap v19392: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 14 op/s 2023-09-10T09:20:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:34 smithi170 ceph-mon[102832]: pgmap v19393: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-10T09:20:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:34 smithi082 ceph-mon[98694]: pgmap v19393: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-10T09:20:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:34 smithi098 ceph-mon[103124]: pgmap v19393: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-10T09:20:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36065.862090 secs 2023-09-10T09:20:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36066.214946 secs 2023-09-10T09:20:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36065.862090 secs 2023-09-10T09:20:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36066.214946 secs 2023-09-10T09:20:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36065.862090 secs 2023-09-10T09:20:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36066.214946 secs 2023-09-10T09:20:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:36 smithi170 ceph-mon[102832]: pgmap v19394: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-10T09:20:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:36 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:20:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:36 smithi082 ceph-mon[98694]: pgmap v19394: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-10T09:20:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:36 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:20:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:36 smithi098 ceph-mon[103124]: pgmap v19394: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-10T09:20:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:36 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:20:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:37 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:20:37.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:20:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:37 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:20:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:20:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:37 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:20:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:20:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:38 smithi170 ceph-mon[102832]: pgmap v19395: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 321 KiB/s wr, 15 op/s 2023-09-10T09:20:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:38 smithi082 ceph-mon[98694]: pgmap v19395: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 321 KiB/s wr, 15 op/s 2023-09-10T09:20:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:38 smithi098 ceph-mon[103124]: pgmap v19395: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 321 KiB/s wr, 15 op/s 2023-09-10T09:20:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36070.862175 secs 2023-09-10T09:20:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36071.215014 secs 2023-09-10T09:20:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:39 smithi170 ceph-mon[102832]: pgmap v19396: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 321 KiB/s wr, 10 op/s 2023-09-10T09:20:40.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36070.862175 secs 2023-09-10T09:20:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36071.215014 secs 2023-09-10T09:20:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:39 smithi082 ceph-mon[98694]: pgmap v19396: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 321 KiB/s wr, 10 op/s 2023-09-10T09:20:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36070.862175 secs 2023-09-10T09:20:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36071.215014 secs 2023-09-10T09:20:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:39 smithi098 ceph-mon[103124]: pgmap v19396: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 321 KiB/s wr, 10 op/s 2023-09-10T09:20:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:20:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:20:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:20:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:41 smithi170 ceph-mon[102832]: pgmap v19397: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 708 KiB/s wr, 17 op/s 2023-09-10T09:20:42.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:41 smithi082 ceph-mon[98694]: pgmap v19397: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 708 KiB/s wr, 17 op/s 2023-09-10T09:20:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:41 smithi098 ceph-mon[103124]: pgmap v19397: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 708 KiB/s wr, 17 op/s 2023-09-10T09:20:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:44 smithi170 ceph-mon[102832]: pgmap v19398: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 708 KiB/s wr, 12 op/s 2023-09-10T09:20:44.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:44 smithi082 ceph-mon[98694]: pgmap v19398: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 708 KiB/s wr, 12 op/s 2023-09-10T09:20:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:44 smithi098 ceph-mon[103124]: pgmap v19398: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 708 KiB/s wr, 12 op/s 2023-09-10T09:20:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36075.862261 secs 2023-09-10T09:20:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36076.215078 secs 2023-09-10T09:20:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:20:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36075.862261 secs 2023-09-10T09:20:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36076.215078 secs 2023-09-10T09:20:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:20:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36075.862261 secs 2023-09-10T09:20:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36076.215078 secs 2023-09-10T09:20:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:20:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:46 smithi170 ceph-mon[102832]: pgmap v19399: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 708 KiB/s wr, 12 op/s 2023-09-10T09:20:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:20:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:46 smithi082 ceph-mon[98694]: pgmap v19399: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 708 KiB/s wr, 12 op/s 2023-09-10T09:20:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:20:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:46 smithi098 ceph-mon[103124]: pgmap v19399: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 708 KiB/s wr, 12 op/s 2023-09-10T09:20:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:20:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:48 smithi170 ceph-mon[102832]: pgmap v19400: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 997 KiB/s wr, 20 op/s 2023-09-10T09:20:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:48 smithi082 ceph-mon[98694]: pgmap v19400: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 997 KiB/s wr, 20 op/s 2023-09-10T09:20:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:48 smithi098 ceph-mon[103124]: pgmap v19400: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 997 KiB/s wr, 20 op/s 2023-09-10T09:20:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36080.862321 secs 2023-09-10T09:20:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36081.215140 secs 2023-09-10T09:20:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:49 smithi170 ceph-mon[102832]: pgmap v19401: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 676 KiB/s wr, 14 op/s 2023-09-10T09:20:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36080.862321 secs 2023-09-10T09:20:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36081.215140 secs 2023-09-10T09:20:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:49 smithi082 ceph-mon[98694]: pgmap v19401: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 676 KiB/s wr, 14 op/s 2023-09-10T09:20:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36080.862321 secs 2023-09-10T09:20:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36081.215140 secs 2023-09-10T09:20:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:49 smithi098 ceph-mon[103124]: pgmap v19401: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 676 KiB/s wr, 14 op/s 2023-09-10T09:20:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:20:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:20:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:20:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:51 smithi170 ceph-mon[102832]: pgmap v19402: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 947 KiB/s wr, 22 op/s 2023-09-10T09:20:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:51 smithi082 ceph-mon[98694]: pgmap v19402: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 947 KiB/s wr, 22 op/s 2023-09-10T09:20:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:51 smithi098 ceph-mon[103124]: pgmap v19402: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 947 KiB/s wr, 22 op/s 2023-09-10T09:20:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:20:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:20:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:20:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:53 smithi170 ceph-mon[102832]: pgmap v19403: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 560 KiB/s wr, 15 op/s 2023-09-10T09:20:54.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:53 smithi082 ceph-mon[98694]: pgmap v19403: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 560 KiB/s wr, 15 op/s 2023-09-10T09:20:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:53 smithi098 ceph-mon[103124]: pgmap v19403: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 560 KiB/s wr, 15 op/s 2023-09-10T09:20:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36085.862380 secs 2023-09-10T09:20:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36086.215206 secs 2023-09-10T09:20:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36085.862380 secs 2023-09-10T09:20:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36086.215206 secs 2023-09-10T09:20:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36085.862380 secs 2023-09-10T09:20:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36086.215206 secs 2023-09-10T09:20:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:55 smithi170 ceph-mon[102832]: pgmap v19404: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 560 KiB/s wr, 15 op/s 2023-09-10T09:20:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:55 smithi082 ceph-mon[98694]: pgmap v19404: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 560 KiB/s wr, 15 op/s 2023-09-10T09:20:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:55 smithi098 ceph-mon[103124]: pgmap v19404: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 560 KiB/s wr, 15 op/s 2023-09-10T09:20:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:20:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:58.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:57 smithi170 ceph-mon[102832]: pgmap v19405: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 823 KiB/s wr, 23 op/s 2023-09-10T09:20:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:20:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:57 smithi082 ceph-mon[98694]: pgmap v19405: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 823 KiB/s wr, 23 op/s 2023-09-10T09:20:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:20:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:20:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:57 smithi098 ceph-mon[103124]: pgmap v19405: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 823 KiB/s wr, 23 op/s 2023-09-10T09:20:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36089.613380 secs 2023-09-10T09:20:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36089.613380 secs 2023-09-10T09:20:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36089.613380 secs 2023-09-10T09:21:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36090.862445 secs 2023-09-10T09:21:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36091.215273 secs 2023-09-10T09:21:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:20:59 smithi170 ceph-mon[102832]: pgmap v19406: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 534 KiB/s wr, 15 op/s 2023-09-10T09:21:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36090.862445 secs 2023-09-10T09:21:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36091.215273 secs 2023-09-10T09:21:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:20:59 smithi082 ceph-mon[98694]: pgmap v19406: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 534 KiB/s wr, 15 op/s 2023-09-10T09:21:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36090.862445 secs 2023-09-10T09:21:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36091.215273 secs 2023-09-10T09:21:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:20:59 smithi098 ceph-mon[103124]: pgmap v19406: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 534 KiB/s wr, 15 op/s 2023-09-10T09:21:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:21:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:21:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:21:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:21:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:21:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:21:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:02.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:01 smithi170 ceph-mon[102832]: pgmap v19407: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 754 KiB/s wr, 22 op/s 2023-09-10T09:21:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:01 smithi082 ceph-mon[98694]: pgmap v19407: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 754 KiB/s wr, 22 op/s 2023-09-10T09:21:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:01 smithi098 ceph-mon[103124]: pgmap v19407: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 754 KiB/s wr, 22 op/s 2023-09-10T09:21:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36094.613456 secs 2023-09-10T09:21:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36094.613456 secs 2023-09-10T09:21:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36094.613456 secs 2023-09-10T09:21:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:04 smithi170 ceph-mon[102832]: pgmap v19408: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 483 KiB/s wr, 14 op/s 2023-09-10T09:21:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:04 smithi082 ceph-mon[98694]: pgmap v19408: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 483 KiB/s wr, 14 op/s 2023-09-10T09:21:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:04 smithi098 ceph-mon[103124]: pgmap v19408: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 483 KiB/s wr, 14 op/s 2023-09-10T09:21:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36095.862493 secs 2023-09-10T09:21:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36096.215340 secs 2023-09-10T09:21:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36095.862493 secs 2023-09-10T09:21:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36096.215340 secs 2023-09-10T09:21:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36095.862493 secs 2023-09-10T09:21:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36096.215340 secs 2023-09-10T09:21:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:06 smithi170 ceph-mon[102832]: pgmap v19409: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 483 KiB/s wr, 14 op/s 2023-09-10T09:21:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:06 smithi082 ceph-mon[98694]: pgmap v19409: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 483 KiB/s wr, 14 op/s 2023-09-10T09:21:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:06 smithi098 ceph-mon[103124]: pgmap v19409: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 483 KiB/s wr, 14 op/s 2023-09-10T09:21:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:08 smithi170 ceph-mon[102832]: pgmap v19410: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 675 KiB/s wr, 35 op/s 2023-09-10T09:21:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36099.613516 secs 2023-09-10T09:21:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:08 smithi082 ceph-mon[98694]: pgmap v19410: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 675 KiB/s wr, 35 op/s 2023-09-10T09:21:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36099.613516 secs 2023-09-10T09:21:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:08 smithi098 ceph-mon[103124]: pgmap v19410: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 675 KiB/s wr, 35 op/s 2023-09-10T09:21:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36099.613516 secs 2023-09-10T09:21:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:09 smithi170 ceph-mon[102832]: osdmap e1281: 12 total, 12 up, 12 in 2023-09-10T09:21:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36100.862555 secs 2023-09-10T09:21:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:09 smithi082 ceph-mon[98694]: osdmap e1281: 12 total, 12 up, 12 in 2023-09-10T09:21:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36100.862555 secs 2023-09-10T09:21:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:09 smithi098 ceph-mon[103124]: osdmap e1281: 12 total, 12 up, 12 in 2023-09-10T09:21:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36100.862555 secs 2023-09-10T09:21:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36101.215407 secs 2023-09-10T09:21:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:10 smithi170 ceph-mon[102832]: pgmap v19412: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 495 KiB/s wr, 33 op/s 2023-09-10T09:21:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36101.215407 secs 2023-09-10T09:21:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:10 smithi082 ceph-mon[98694]: pgmap v19412: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 495 KiB/s wr, 33 op/s 2023-09-10T09:21:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36101.215407 secs 2023-09-10T09:21:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:10 smithi098 ceph-mon[103124]: pgmap v19412: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 495 KiB/s wr, 33 op/s 2023-09-10T09:21:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:12 smithi170 ceph-mon[102832]: pgmap v19413: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 231 KiB/s wr, 31 op/s 2023-09-10T09:21:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:12 smithi082 ceph-mon[98694]: pgmap v19413: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 231 KiB/s wr, 31 op/s 2023-09-10T09:21:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:12 smithi098 ceph-mon[103124]: pgmap v19413: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 231 KiB/s wr, 31 op/s 2023-09-10T09:21:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36104.613608 secs 2023-09-10T09:21:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36104.613608 secs 2023-09-10T09:21:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36104.613608 secs 2023-09-10T09:21:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:14 smithi170 ceph-mon[102832]: pgmap v19414: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 231 KiB/s wr, 31 op/s 2023-09-10T09:21:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36105.862615 secs 2023-09-10T09:21:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:14 smithi170 ceph-mon[102832]: osdmap e1282: 12 total, 12 up, 12 in 2023-09-10T09:21:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:14 smithi082 ceph-mon[98694]: pgmap v19414: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 231 KiB/s wr, 31 op/s 2023-09-10T09:21:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36105.862615 secs 2023-09-10T09:21:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:14 smithi082 ceph-mon[98694]: osdmap e1282: 12 total, 12 up, 12 in 2023-09-10T09:21:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:14 smithi098 ceph-mon[103124]: pgmap v19414: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 231 KiB/s wr, 31 op/s 2023-09-10T09:21:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36105.862615 secs 2023-09-10T09:21:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:14 smithi098 ceph-mon[103124]: osdmap e1282: 12 total, 12 up, 12 in 2023-09-10T09:21:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36106.215478 secs 2023-09-10T09:21:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36106.215478 secs 2023-09-10T09:21:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36106.215478 secs 2023-09-10T09:21:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:16 smithi170 ceph-mon[102832]: pgmap v19416: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T09:21:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:21:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:16 smithi082 ceph-mon[98694]: pgmap v19416: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T09:21:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:21:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:16 smithi098 ceph-mon[103124]: pgmap v19416: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 7 op/s 2023-09-10T09:21:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:21:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:18 smithi170 ceph-mon[102832]: pgmap v19417: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 14 op/s 2023-09-10T09:21:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:21:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36109.613670 secs 2023-09-10T09:21:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:18 smithi082 ceph-mon[98694]: pgmap v19417: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 14 op/s 2023-09-10T09:21:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:21:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36109.613670 secs 2023-09-10T09:21:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:18 smithi098 ceph-mon[103124]: pgmap v19417: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 14 op/s 2023-09-10T09:21:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:21:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36109.613670 secs 2023-09-10T09:21:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36110.862674 secs 2023-09-10T09:21:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36111.215542 secs 2023-09-10T09:21:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:19 smithi170 ceph-mon[102832]: pgmap v19418: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 12 op/s 2023-09-10T09:21:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36110.862674 secs 2023-09-10T09:21:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36111.215542 secs 2023-09-10T09:21:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:19 smithi082 ceph-mon[98694]: pgmap v19418: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 12 op/s 2023-09-10T09:21:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36110.862674 secs 2023-09-10T09:21:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36111.215542 secs 2023-09-10T09:21:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:19 smithi098 ceph-mon[103124]: pgmap v19418: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 12 op/s 2023-09-10T09:21:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:22 smithi170 ceph-mon[102832]: pgmap v19419: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2023-09-10T09:21:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:22 smithi082 ceph-mon[98694]: pgmap v19419: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2023-09-10T09:21:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:22 smithi098 ceph-mon[103124]: pgmap v19419: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2023-09-10T09:21:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36114.613757 secs 2023-09-10T09:21:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36114.613757 secs 2023-09-10T09:21:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36114.613757 secs 2023-09-10T09:21:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:24 smithi170 ceph-mon[102832]: pgmap v19420: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2023-09-10T09:21:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:21:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:24 smithi082 ceph-mon[98694]: pgmap v19420: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2023-09-10T09:21:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:21:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:24 smithi098 ceph-mon[103124]: pgmap v19420: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2023-09-10T09:21:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:21:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36115.862734 secs 2023-09-10T09:21:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36116.215610 secs 2023-09-10T09:21:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:21:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:21:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:21:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36115.862734 secs 2023-09-10T09:21:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36116.215610 secs 2023-09-10T09:21:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:21:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:21:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:21:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36115.862734 secs 2023-09-10T09:21:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36116.215610 secs 2023-09-10T09:21:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:21:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:21:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:21:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:26 smithi170 ceph-mon[102832]: pgmap v19421: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T09:21:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:26 smithi082 ceph-mon[98694]: pgmap v19421: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T09:21:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:26 smithi098 ceph-mon[103124]: pgmap v19421: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T09:21:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:28 smithi170 ceph-mon[102832]: pgmap v19422: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 126 KiB/s wr, 16 op/s 2023-09-10T09:21:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36119.613772 secs 2023-09-10T09:21:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:28 smithi082 ceph-mon[98694]: pgmap v19422: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 126 KiB/s wr, 16 op/s 2023-09-10T09:21:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36119.613772 secs 2023-09-10T09:21:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:28 smithi098 ceph-mon[103124]: pgmap v19422: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 126 KiB/s wr, 16 op/s 2023-09-10T09:21:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36119.613772 secs 2023-09-10T09:21:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36120.862826 secs 2023-09-10T09:21:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36121.215674 secs 2023-09-10T09:21:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:29 smithi170 ceph-mon[102832]: pgmap v19423: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 10 op/s 2023-09-10T09:21:30.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36120.862826 secs 2023-09-10T09:21:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36121.215674 secs 2023-09-10T09:21:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:29 smithi082 ceph-mon[98694]: pgmap v19423: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 10 op/s 2023-09-10T09:21:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36120.862826 secs 2023-09-10T09:21:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36121.215674 secs 2023-09-10T09:21:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:29 smithi098 ceph-mon[103124]: pgmap v19423: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 10 op/s 2023-09-10T09:21:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:32 smithi170 ceph-mon[102832]: pgmap v19424: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 256 KiB/s wr, 16 op/s 2023-09-10T09:21:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:32 smithi082 ceph-mon[98694]: pgmap v19424: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 256 KiB/s wr, 16 op/s 2023-09-10T09:21:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:32 smithi098 ceph-mon[103124]: pgmap v19424: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 256 KiB/s wr, 16 op/s 2023-09-10T09:21:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:34 smithi170 ceph-mon[102832]: pgmap v19425: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 256 KiB/s wr, 11 op/s 2023-09-10T09:21:34.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:34 smithi082 ceph-mon[98694]: pgmap v19425: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 256 KiB/s wr, 11 op/s 2023-09-10T09:21:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:34 smithi098 ceph-mon[103124]: pgmap v19425: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 256 KiB/s wr, 11 op/s 2023-09-10T09:21:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36125.862910 secs 2023-09-10T09:21:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36126.215742 secs 2023-09-10T09:21:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36125.862910 secs 2023-09-10T09:21:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36126.215742 secs 2023-09-10T09:21:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36125.862910 secs 2023-09-10T09:21:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36126.215742 secs 2023-09-10T09:21:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:36 smithi170 ceph-mon[102832]: pgmap v19426: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 256 KiB/s wr, 11 op/s 2023-09-10T09:21:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:36 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:21:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:21:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:21:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:36 smithi082 ceph-mon[98694]: pgmap v19426: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 256 KiB/s wr, 11 op/s 2023-09-10T09:21:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:36 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:21:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:21:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:21:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:36 smithi098 ceph-mon[103124]: pgmap v19426: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 256 KiB/s wr, 11 op/s 2023-09-10T09:21:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:36 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:21:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:21:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:21:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:38.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:38 smithi170 ceph-mon[102832]: pgmap v19427: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 385 KiB/s wr, 16 op/s 2023-09-10T09:21:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:38 smithi082 ceph-mon[98694]: pgmap v19427: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 385 KiB/s wr, 16 op/s 2023-09-10T09:21:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:38 smithi098 ceph-mon[103124]: pgmap v19427: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 385 KiB/s wr, 16 op/s 2023-09-10T09:21:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36130.862971 secs 2023-09-10T09:21:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36131.215807 secs 2023-09-10T09:21:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:39 smithi170 ceph-mon[102832]: pgmap v19428: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 259 KiB/s wr, 10 op/s 2023-09-10T09:21:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36130.862971 secs 2023-09-10T09:21:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36131.215807 secs 2023-09-10T09:21:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:39 smithi082 ceph-mon[98694]: pgmap v19428: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 259 KiB/s wr, 10 op/s 2023-09-10T09:21:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36130.862971 secs 2023-09-10T09:21:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36131.215807 secs 2023-09-10T09:21:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:39 smithi098 ceph-mon[103124]: pgmap v19428: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 259 KiB/s wr, 10 op/s 2023-09-10T09:21:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:21:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:41.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:21:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:21:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:41 smithi170 ceph-mon[102832]: pgmap v19429: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 584 KiB/s wr, 17 op/s 2023-09-10T09:21:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:41 smithi082 ceph-mon[98694]: pgmap v19429: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 584 KiB/s wr, 17 op/s 2023-09-10T09:21:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:41 smithi098 ceph-mon[103124]: pgmap v19429: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 584 KiB/s wr, 17 op/s 2023-09-10T09:21:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:44 smithi170 ceph-mon[102832]: pgmap v19430: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 454 KiB/s wr, 12 op/s 2023-09-10T09:21:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:44 smithi082 ceph-mon[98694]: pgmap v19430: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 454 KiB/s wr, 12 op/s 2023-09-10T09:21:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:44 smithi098 ceph-mon[103124]: pgmap v19430: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 454 KiB/s wr, 12 op/s 2023-09-10T09:21:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36135.863028 secs 2023-09-10T09:21:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36136.215874 secs 2023-09-10T09:21:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:21:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36135.863028 secs 2023-09-10T09:21:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36136.215874 secs 2023-09-10T09:21:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:21:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36135.863028 secs 2023-09-10T09:21:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36136.215874 secs 2023-09-10T09:21:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:21:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:46 smithi170 ceph-mon[102832]: pgmap v19431: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 454 KiB/s wr, 12 op/s 2023-09-10T09:21:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:21:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:46.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:46 smithi082 ceph-mon[98694]: pgmap v19431: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 454 KiB/s wr, 12 op/s 2023-09-10T09:21:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:21:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:46 smithi098 ceph-mon[103124]: pgmap v19431: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 454 KiB/s wr, 12 op/s 2023-09-10T09:21:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:21:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:48 smithi170 ceph-mon[102832]: pgmap v19432: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 619 KiB/s wr, 19 op/s 2023-09-10T09:21:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:48 smithi082 ceph-mon[98694]: pgmap v19432: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 619 KiB/s wr, 19 op/s 2023-09-10T09:21:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:48 smithi098 ceph-mon[103124]: pgmap v19432: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 619 KiB/s wr, 19 op/s 2023-09-10T09:21:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36140.863113 secs 2023-09-10T09:21:50.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36141.215941 secs 2023-09-10T09:21:50.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:49 smithi170 ceph-mon[102832]: pgmap v19433: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 490 KiB/s wr, 13 op/s 2023-09-10T09:21:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36140.863113 secs 2023-09-10T09:21:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36141.215941 secs 2023-09-10T09:21:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:49 smithi082 ceph-mon[98694]: pgmap v19433: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 490 KiB/s wr, 13 op/s 2023-09-10T09:21:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36140.863113 secs 2023-09-10T09:21:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36141.215941 secs 2023-09-10T09:21:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:49 smithi098 ceph-mon[103124]: pgmap v19433: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 490 KiB/s wr, 13 op/s 2023-09-10T09:21:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:52 smithi170 ceph-mon[102832]: pgmap v19434: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 805 KiB/s wr, 21 op/s 2023-09-10T09:21:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:21:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:52 smithi082 ceph-mon[98694]: pgmap v19434: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 805 KiB/s wr, 21 op/s 2023-09-10T09:21:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:21:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:52 smithi098 ceph-mon[103124]: pgmap v19434: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 805 KiB/s wr, 21 op/s 2023-09-10T09:21:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:21:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:54 smithi170 ceph-mon[102832]: pgmap v19435: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 480 KiB/s wr, 15 op/s 2023-09-10T09:21:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:21:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36145.863173 secs 2023-09-10T09:21:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:54 smithi082 ceph-mon[98694]: pgmap v19435: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 480 KiB/s wr, 15 op/s 2023-09-10T09:21:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:21:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36145.863173 secs 2023-09-10T09:21:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:54 smithi098 ceph-mon[103124]: pgmap v19435: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 480 KiB/s wr, 15 op/s 2023-09-10T09:21:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:21:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36145.863173 secs 2023-09-10T09:21:55.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36146.216004 secs 2023-09-10T09:21:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36146.216004 secs 2023-09-10T09:21:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36146.216004 secs 2023-09-10T09:21:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:56 smithi082 ceph-mon[98694]: pgmap v19436: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 480 KiB/s wr, 15 op/s 2023-09-10T09:21:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:56 smithi098 ceph-mon[103124]: pgmap v19436: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 480 KiB/s wr, 15 op/s 2023-09-10T09:21:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:56 smithi170 ceph-mon[102832]: pgmap v19436: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 480 KiB/s wr, 15 op/s 2023-09-10T09:21:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:58 smithi082 ceph-mon[98694]: pgmap v19437: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 659 KiB/s wr, 22 op/s 2023-09-10T09:21:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:21:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36149.614176 secs 2023-09-10T09:21:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:58 smithi098 ceph-mon[103124]: pgmap v19437: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 659 KiB/s wr, 22 op/s 2023-09-10T09:21:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:21:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36149.614176 secs 2023-09-10T09:21:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:58 smithi170 ceph-mon[102832]: pgmap v19437: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 659 KiB/s wr, 22 op/s 2023-09-10T09:21:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:21:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:21:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36149.614176 secs 2023-09-10T09:22:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36150.863232 secs 2023-09-10T09:22:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36151.216074 secs 2023-09-10T09:22:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:21:59 smithi170 ceph-mon[102832]: pgmap v19438: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 494 KiB/s wr, 15 op/s 2023-09-10T09:22:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36150.863232 secs 2023-09-10T09:22:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36151.216074 secs 2023-09-10T09:22:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:21:59 smithi082 ceph-mon[98694]: pgmap v19438: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 494 KiB/s wr, 15 op/s 2023-09-10T09:22:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36150.863232 secs 2023-09-10T09:22:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36151.216074 secs 2023-09-10T09:22:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:21:59 smithi098 ceph-mon[103124]: pgmap v19438: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 494 KiB/s wr, 15 op/s 2023-09-10T09:22:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:22:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:22:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:22:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:22:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:22:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:22:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:01 smithi170 ceph-mon[102832]: pgmap v19439: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 777 KiB/s wr, 21 op/s 2023-09-10T09:22:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:01 smithi082 ceph-mon[98694]: pgmap v19439: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 777 KiB/s wr, 21 op/s 2023-09-10T09:22:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:01 smithi098 ceph-mon[103124]: pgmap v19439: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 777 KiB/s wr, 21 op/s 2023-09-10T09:22:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36154.614303 secs 2023-09-10T09:22:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36154.614303 secs 2023-09-10T09:22:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36154.614303 secs 2023-09-10T09:22:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:04 smithi170 ceph-mon[102832]: pgmap v19440: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 462 KiB/s wr, 13 op/s 2023-09-10T09:22:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36155.863290 secs 2023-09-10T09:22:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:04 smithi082 ceph-mon[98694]: pgmap v19440: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 462 KiB/s wr, 13 op/s 2023-09-10T09:22:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36155.863290 secs 2023-09-10T09:22:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:04 smithi098 ceph-mon[103124]: pgmap v19440: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 462 KiB/s wr, 13 op/s 2023-09-10T09:22:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36155.863290 secs 2023-09-10T09:22:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36156.216140 secs 2023-09-10T09:22:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36156.216140 secs 2023-09-10T09:22:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36156.216140 secs 2023-09-10T09:22:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:06 smithi170 ceph-mon[102832]: pgmap v19441: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 462 KiB/s wr, 13 op/s 2023-09-10T09:22:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:06 smithi082 ceph-mon[98694]: pgmap v19441: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 462 KiB/s wr, 13 op/s 2023-09-10T09:22:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:06 smithi098 ceph-mon[103124]: pgmap v19441: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 462 KiB/s wr, 13 op/s 2023-09-10T09:22:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:08 smithi170 ceph-mon[102832]: pgmap v19442: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 611 KiB/s wr, 33 op/s 2023-09-10T09:22:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36159.614362 secs 2023-09-10T09:22:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:08 smithi082 ceph-mon[98694]: pgmap v19442: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 611 KiB/s wr, 33 op/s 2023-09-10T09:22:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36159.614362 secs 2023-09-10T09:22:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:08 smithi098 ceph-mon[103124]: pgmap v19442: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 611 KiB/s wr, 33 op/s 2023-09-10T09:22:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36159.614362 secs 2023-09-10T09:22:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:09 smithi170 ceph-mon[102832]: osdmap e1283: 12 total, 12 up, 12 in 2023-09-10T09:22:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36160.863347 secs 2023-09-10T09:22:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:09 smithi082 ceph-mon[98694]: osdmap e1283: 12 total, 12 up, 12 in 2023-09-10T09:22:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36160.863347 secs 2023-09-10T09:22:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:09 smithi098 ceph-mon[103124]: osdmap e1283: 12 total, 12 up, 12 in 2023-09-10T09:22:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36160.863347 secs 2023-09-10T09:22:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36161.216205 secs 2023-09-10T09:22:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:10 smithi170 ceph-mon[102832]: pgmap v19444: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 519 KiB/s wr, 32 op/s 2023-09-10T09:22:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36161.216205 secs 2023-09-10T09:22:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:10 smithi082 ceph-mon[98694]: pgmap v19444: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 519 KiB/s wr, 32 op/s 2023-09-10T09:22:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36161.216205 secs 2023-09-10T09:22:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:10 smithi098 ceph-mon[103124]: pgmap v19444: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 519 KiB/s wr, 32 op/s 2023-09-10T09:22:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:12 smithi170 ceph-mon[102832]: pgmap v19445: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 278 KiB/s wr, 31 op/s 2023-09-10T09:22:12.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:12 smithi082 ceph-mon[98694]: pgmap v19445: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 278 KiB/s wr, 31 op/s 2023-09-10T09:22:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:12 smithi098 ceph-mon[103124]: pgmap v19445: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 278 KiB/s wr, 31 op/s 2023-09-10T09:22:13.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36164.614422 secs 2023-09-10T09:22:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36164.614422 secs 2023-09-10T09:22:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36164.614422 secs 2023-09-10T09:22:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:14 smithi170 ceph-mon[102832]: pgmap v19446: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 278 KiB/s wr, 31 op/s 2023-09-10T09:22:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36165.863405 secs 2023-09-10T09:22:14.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:14 smithi170 ceph-mon[102832]: osdmap e1284: 12 total, 12 up, 12 in 2023-09-10T09:22:14.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:14 smithi082 ceph-mon[98694]: pgmap v19446: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 278 KiB/s wr, 31 op/s 2023-09-10T09:22:14.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36165.863405 secs 2023-09-10T09:22:14.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:14 smithi082 ceph-mon[98694]: osdmap e1284: 12 total, 12 up, 12 in 2023-09-10T09:22:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:14 smithi098 ceph-mon[103124]: pgmap v19446: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 278 KiB/s wr, 31 op/s 2023-09-10T09:22:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36165.863405 secs 2023-09-10T09:22:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:14 smithi098 ceph-mon[103124]: osdmap e1284: 12 total, 12 up, 12 in 2023-09-10T09:22:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36166.216269 secs 2023-09-10T09:22:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36166.216269 secs 2023-09-10T09:22:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36166.216269 secs 2023-09-10T09:22:16.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:16 smithi170 ceph-mon[102832]: pgmap v19448: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 124 KiB/s wr, 8 op/s 2023-09-10T09:22:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:22:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:16 smithi082 ceph-mon[98694]: pgmap v19448: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 124 KiB/s wr, 8 op/s 2023-09-10T09:22:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:22:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:16 smithi098 ceph-mon[103124]: pgmap v19448: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 124 KiB/s wr, 8 op/s 2023-09-10T09:22:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:22:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:18 smithi170 ceph-mon[102832]: pgmap v19449: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 14 op/s 2023-09-10T09:22:18.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:22:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36169.614530 secs 2023-09-10T09:22:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:18 smithi082 ceph-mon[98694]: pgmap v19449: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 14 op/s 2023-09-10T09:22:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:22:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36169.614530 secs 2023-09-10T09:22:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:18 smithi098 ceph-mon[103124]: pgmap v19449: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 111 KiB/s wr, 14 op/s 2023-09-10T09:22:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:22:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36169.614530 secs 2023-09-10T09:22:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36170.863494 secs 2023-09-10T09:22:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36171.216338 secs 2023-09-10T09:22:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:19 smithi170 ceph-mon[102832]: pgmap v19450: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 99 KiB/s wr, 13 op/s 2023-09-10T09:22:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36170.863494 secs 2023-09-10T09:22:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36171.216338 secs 2023-09-10T09:22:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:19 smithi082 ceph-mon[98694]: pgmap v19450: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 99 KiB/s wr, 13 op/s 2023-09-10T09:22:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36170.863494 secs 2023-09-10T09:22:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36171.216338 secs 2023-09-10T09:22:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:19 smithi098 ceph-mon[103124]: pgmap v19450: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 99 KiB/s wr, 13 op/s 2023-09-10T09:22:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:22 smithi170 ceph-mon[102832]: pgmap v19451: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2023-09-10T09:22:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:22 smithi082 ceph-mon[98694]: pgmap v19451: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2023-09-10T09:22:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:22 smithi098 ceph-mon[103124]: pgmap v19451: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2023-09-10T09:22:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36174.614601 secs 2023-09-10T09:22:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36174.614601 secs 2023-09-10T09:22:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36174.614601 secs 2023-09-10T09:22:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:24 smithi170 ceph-mon[102832]: pgmap v19452: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2023-09-10T09:22:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36175.863552 secs 2023-09-10T09:22:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:24 smithi082 ceph-mon[98694]: pgmap v19452: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2023-09-10T09:22:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36175.863552 secs 2023-09-10T09:22:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:24 smithi098 ceph-mon[103124]: pgmap v19452: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2023-09-10T09:22:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36175.863552 secs 2023-09-10T09:22:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36176.216403 secs 2023-09-10T09:22:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:22:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:22:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:22:25.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:22:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36176.216403 secs 2023-09-10T09:22:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:22:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:22:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:22:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:22:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36176.216403 secs 2023-09-10T09:22:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:22:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:22:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:22:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:22:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:26 smithi170 ceph-mon[102832]: pgmap v19453: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T09:22:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:26 smithi082 ceph-mon[98694]: pgmap v19453: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T09:22:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:26 smithi098 ceph-mon[103124]: pgmap v19453: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T09:22:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:28 smithi170 ceph-mon[102832]: pgmap v19454: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 511 B/s wr, 15 op/s 2023-09-10T09:22:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36179.614663 secs 2023-09-10T09:22:28.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:28 smithi082 ceph-mon[98694]: pgmap v19454: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 511 B/s wr, 15 op/s 2023-09-10T09:22:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36179.614663 secs 2023-09-10T09:22:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:28 smithi098 ceph-mon[103124]: pgmap v19454: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 511 B/s wr, 15 op/s 2023-09-10T09:22:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36179.614663 secs 2023-09-10T09:22:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36180.863609 secs 2023-09-10T09:22:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36181.216475 secs 2023-09-10T09:22:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:29 smithi170 ceph-mon[102832]: pgmap v19455: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 10 op/s 2023-09-10T09:22:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36180.863609 secs 2023-09-10T09:22:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36181.216475 secs 2023-09-10T09:22:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:29 smithi082 ceph-mon[98694]: pgmap v19455: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 10 op/s 2023-09-10T09:22:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36180.863609 secs 2023-09-10T09:22:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36181.216475 secs 2023-09-10T09:22:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:29 smithi098 ceph-mon[103124]: pgmap v19455: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 10 op/s 2023-09-10T09:22:32.133 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:32 smithi170 ceph-mon[102832]: pgmap v19456: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 131 KiB/s wr, 15 op/s 2023-09-10T09:22:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:32 smithi082 ceph-mon[98694]: pgmap v19456: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 131 KiB/s wr, 15 op/s 2023-09-10T09:22:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:32 smithi098 ceph-mon[103124]: pgmap v19456: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 131 KiB/s wr, 15 op/s 2023-09-10T09:22:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:34 smithi170 ceph-mon[102832]: pgmap v19457: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 10 op/s 2023-09-10T09:22:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36185.863668 secs 2023-09-10T09:22:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:34 smithi082 ceph-mon[98694]: pgmap v19457: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 10 op/s 2023-09-10T09:22:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36185.863668 secs 2023-09-10T09:22:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:34 smithi098 ceph-mon[103124]: pgmap v19457: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 10 op/s 2023-09-10T09:22:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36185.863668 secs 2023-09-10T09:22:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36186.216490 secs 2023-09-10T09:22:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:22:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36186.216490 secs 2023-09-10T09:22:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:22:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36186.216490 secs 2023-09-10T09:22:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:22:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:36 smithi170 ceph-mon[102832]: pgmap v19458: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 10 op/s 2023-09-10T09:22:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:22:36.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:22:36.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:36 smithi082 ceph-mon[98694]: pgmap v19458: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 10 op/s 2023-09-10T09:22:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:22:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:22:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:36 smithi098 ceph-mon[103124]: pgmap v19458: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 131 KiB/s wr, 10 op/s 2023-09-10T09:22:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:22:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:22:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:38 smithi170 ceph-mon[102832]: pgmap v19459: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 275 KiB/s wr, 16 op/s 2023-09-10T09:22:38.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:38 smithi082 ceph-mon[98694]: pgmap v19459: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 275 KiB/s wr, 16 op/s 2023-09-10T09:22:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:38 smithi098 ceph-mon[103124]: pgmap v19459: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 275 KiB/s wr, 16 op/s 2023-09-10T09:22:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36190.863725 secs 2023-09-10T09:22:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36191.216554 secs 2023-09-10T09:22:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:39 smithi170 ceph-mon[102832]: pgmap v19460: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 274 KiB/s wr, 10 op/s 2023-09-10T09:22:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36190.863725 secs 2023-09-10T09:22:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36191.216554 secs 2023-09-10T09:22:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:39 smithi082 ceph-mon[98694]: pgmap v19460: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 274 KiB/s wr, 10 op/s 2023-09-10T09:22:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36190.863725 secs 2023-09-10T09:22:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36191.216554 secs 2023-09-10T09:22:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:39 smithi098 ceph-mon[103124]: pgmap v19460: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 274 KiB/s wr, 10 op/s 2023-09-10T09:22:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:22:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:41.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:22:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:22:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:41 smithi170 ceph-mon[102832]: pgmap v19461: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 711 KiB/s wr, 17 op/s 2023-09-10T09:22:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:41 smithi082 ceph-mon[98694]: pgmap v19461: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 711 KiB/s wr, 17 op/s 2023-09-10T09:22:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:41 smithi098 ceph-mon[103124]: pgmap v19461: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 711 KiB/s wr, 17 op/s 2023-09-10T09:22:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:44 smithi170 ceph-mon[102832]: pgmap v19462: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 580 KiB/s wr, 12 op/s 2023-09-10T09:22:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:22:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36195.863800 secs 2023-09-10T09:22:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:44 smithi082 ceph-mon[98694]: pgmap v19462: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 580 KiB/s wr, 12 op/s 2023-09-10T09:22:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:22:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36195.863800 secs 2023-09-10T09:22:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:44 smithi098 ceph-mon[103124]: pgmap v19462: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 580 KiB/s wr, 12 op/s 2023-09-10T09:22:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:22:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36195.863800 secs 2023-09-10T09:22:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36196.216622 secs 2023-09-10T09:22:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36196.216622 secs 2023-09-10T09:22:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36196.216622 secs 2023-09-10T09:22:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:46 smithi170 ceph-mon[102832]: pgmap v19463: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 580 KiB/s wr, 12 op/s 2023-09-10T09:22:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:22:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:46 smithi082 ceph-mon[98694]: pgmap v19463: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 580 KiB/s wr, 12 op/s 2023-09-10T09:22:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:22:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:46 smithi098 ceph-mon[103124]: pgmap v19463: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 580 KiB/s wr, 12 op/s 2023-09-10T09:22:46.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:22:46.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:48 smithi082 ceph-mon[98694]: pgmap v19464: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 871 KiB/s wr, 19 op/s 2023-09-10T09:22:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:48 smithi098 ceph-mon[103124]: pgmap v19464: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 871 KiB/s wr, 19 op/s 2023-09-10T09:22:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:48 smithi170 ceph-mon[102832]: pgmap v19464: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 871 KiB/s wr, 19 op/s 2023-09-10T09:22:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36200.863858 secs 2023-09-10T09:22:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36201.216686 secs 2023-09-10T09:22:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:49 smithi170 ceph-mon[102832]: pgmap v19465: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 727 KiB/s wr, 13 op/s 2023-09-10T09:22:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36200.863858 secs 2023-09-10T09:22:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36201.216686 secs 2023-09-10T09:22:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:49 smithi082 ceph-mon[98694]: pgmap v19465: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 727 KiB/s wr, 13 op/s 2023-09-10T09:22:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36200.863858 secs 2023-09-10T09:22:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36201.216686 secs 2023-09-10T09:22:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:49 smithi098 ceph-mon[103124]: pgmap v19465: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 727 KiB/s wr, 13 op/s 2023-09-10T09:22:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:22:51.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:51.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:22:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:22:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:52.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:51 smithi170 ceph-mon[102832]: pgmap v19466: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1000 KiB/s wr, 21 op/s 2023-09-10T09:22:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:51 smithi082 ceph-mon[98694]: pgmap v19466: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1000 KiB/s wr, 21 op/s 2023-09-10T09:22:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:51 smithi098 ceph-mon[103124]: pgmap v19466: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1000 KiB/s wr, 21 op/s 2023-09-10T09:22:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:22:53.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:53.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:22:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:22:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:54.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:53 smithi170 ceph-mon[102832]: pgmap v19467: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 563 KiB/s wr, 14 op/s 2023-09-10T09:22:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:53 smithi082 ceph-mon[98694]: pgmap v19467: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 563 KiB/s wr, 14 op/s 2023-09-10T09:22:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:53 smithi098 ceph-mon[103124]: pgmap v19467: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 563 KiB/s wr, 14 op/s 2023-09-10T09:22:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36205.863918 secs 2023-09-10T09:22:55.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36206.216749 secs 2023-09-10T09:22:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36205.863918 secs 2023-09-10T09:22:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36206.216749 secs 2023-09-10T09:22:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36205.863918 secs 2023-09-10T09:22:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36206.216749 secs 2023-09-10T09:22:56.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:55 smithi170 ceph-mon[102832]: pgmap v19468: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 563 KiB/s wr, 14 op/s 2023-09-10T09:22:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:55 smithi082 ceph-mon[98694]: pgmap v19468: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 563 KiB/s wr, 14 op/s 2023-09-10T09:22:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:55 smithi098 ceph-mon[103124]: pgmap v19468: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 563 KiB/s wr, 14 op/s 2023-09-10T09:22:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:22:58.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:57 smithi170 ceph-mon[102832]: pgmap v19469: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 827 KiB/s wr, 21 op/s 2023-09-10T09:22:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:22:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:57 smithi082 ceph-mon[98694]: pgmap v19469: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 827 KiB/s wr, 21 op/s 2023-09-10T09:22:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:22:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:22:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:57 smithi098 ceph-mon[103124]: pgmap v19469: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 827 KiB/s wr, 21 op/s 2023-09-10T09:22:59.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36209.615197 secs 2023-09-10T09:22:59.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36209.615197 secs 2023-09-10T09:22:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36209.615197 secs 2023-09-10T09:23:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36210.864001 secs 2023-09-10T09:23:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36211.216820 secs 2023-09-10T09:23:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:22:59 smithi170 ceph-mon[102832]: pgmap v19470: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 536 KiB/s wr, 14 op/s 2023-09-10T09:23:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36210.864001 secs 2023-09-10T09:23:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36211.216820 secs 2023-09-10T09:23:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:22:59 smithi082 ceph-mon[98694]: pgmap v19470: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 536 KiB/s wr, 14 op/s 2023-09-10T09:23:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36210.864001 secs 2023-09-10T09:23:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36211.216820 secs 2023-09-10T09:23:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:22:59 smithi098 ceph-mon[103124]: pgmap v19470: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 536 KiB/s wr, 14 op/s 2023-09-10T09:23:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:23:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:23:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:01.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:23:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:23:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:23:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:23:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:01 smithi170 ceph-mon[102832]: pgmap v19471: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 761 KiB/s wr, 21 op/s 2023-09-10T09:23:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:01 smithi082 ceph-mon[98694]: pgmap v19471: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 761 KiB/s wr, 21 op/s 2023-09-10T09:23:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:01 smithi098 ceph-mon[103124]: pgmap v19471: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 761 KiB/s wr, 21 op/s 2023-09-10T09:23:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36214.615259 secs 2023-09-10T09:23:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36214.615259 secs 2023-09-10T09:23:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36214.615259 secs 2023-09-10T09:23:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:04 smithi170 ceph-mon[102832]: pgmap v19472: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 489 KiB/s wr, 13 op/s 2023-09-10T09:23:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36215.864060 secs 2023-09-10T09:23:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:04 smithi082 ceph-mon[98694]: pgmap v19472: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 489 KiB/s wr, 13 op/s 2023-09-10T09:23:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36215.864060 secs 2023-09-10T09:23:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:04 smithi098 ceph-mon[103124]: pgmap v19472: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 489 KiB/s wr, 13 op/s 2023-09-10T09:23:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36215.864060 secs 2023-09-10T09:23:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36216.216888 secs 2023-09-10T09:23:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36216.216888 secs 2023-09-10T09:23:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36216.216888 secs 2023-09-10T09:23:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:06 smithi170 ceph-mon[102832]: pgmap v19473: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 489 KiB/s wr, 13 op/s 2023-09-10T09:23:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:06 smithi082 ceph-mon[98694]: pgmap v19473: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 489 KiB/s wr, 13 op/s 2023-09-10T09:23:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:06 smithi098 ceph-mon[103124]: pgmap v19473: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 489 KiB/s wr, 13 op/s 2023-09-10T09:23:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:08 smithi170 ceph-mon[102832]: pgmap v19474: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 682 KiB/s wr, 33 op/s 2023-09-10T09:23:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36219.615362 secs 2023-09-10T09:23:08.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:08 smithi082 ceph-mon[98694]: pgmap v19474: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 682 KiB/s wr, 33 op/s 2023-09-10T09:23:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36219.615362 secs 2023-09-10T09:23:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:08 smithi098 ceph-mon[103124]: pgmap v19474: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 682 KiB/s wr, 33 op/s 2023-09-10T09:23:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36219.615362 secs 2023-09-10T09:23:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:09 smithi170 ceph-mon[102832]: osdmap e1285: 12 total, 12 up, 12 in 2023-09-10T09:23:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36220.864119 secs 2023-09-10T09:23:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:09 smithi082 ceph-mon[98694]: osdmap e1285: 12 total, 12 up, 12 in 2023-09-10T09:23:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36220.864119 secs 2023-09-10T09:23:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:09 smithi098 ceph-mon[103124]: osdmap e1285: 12 total, 12 up, 12 in 2023-09-10T09:23:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36220.864119 secs 2023-09-10T09:23:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36221.216955 secs 2023-09-10T09:23:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:10 smithi170 ceph-mon[102832]: pgmap v19476: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 501 KiB/s wr, 32 op/s 2023-09-10T09:23:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:23:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36221.216955 secs 2023-09-10T09:23:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:10 smithi082 ceph-mon[98694]: pgmap v19476: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 501 KiB/s wr, 32 op/s 2023-09-10T09:23:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:23:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36221.216955 secs 2023-09-10T09:23:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:10 smithi098 ceph-mon[103124]: pgmap v19476: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 501 KiB/s wr, 32 op/s 2023-09-10T09:23:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:23:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:12 smithi170 ceph-mon[102832]: pgmap v19477: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 232 KiB/s wr, 30 op/s 2023-09-10T09:23:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:23:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:12 smithi082 ceph-mon[98694]: pgmap v19477: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 232 KiB/s wr, 30 op/s 2023-09-10T09:23:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:23:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:12 smithi098 ceph-mon[103124]: pgmap v19477: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 232 KiB/s wr, 30 op/s 2023-09-10T09:23:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:23:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36224.615439 secs 2023-09-10T09:23:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36224.615439 secs 2023-09-10T09:23:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36224.615439 secs 2023-09-10T09:23:14.183 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:14 smithi098 ceph-mon[103124]: pgmap v19478: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 232 KiB/s wr, 30 op/s 2023-09-10T09:23:14.184 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36225.864179 secs 2023-09-10T09:23:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:14 smithi082 ceph-mon[98694]: pgmap v19478: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 232 KiB/s wr, 30 op/s 2023-09-10T09:23:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36225.864179 secs 2023-09-10T09:23:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:14 smithi082 ceph-mon[98694]: osdmap e1286: 12 total, 12 up, 12 in 2023-09-10T09:23:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:14 smithi098 ceph-mon[103124]: osdmap e1286: 12 total, 12 up, 12 in 2023-09-10T09:23:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:14 smithi170 ceph-mon[102832]: pgmap v19478: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 232 KiB/s wr, 30 op/s 2023-09-10T09:23:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36225.864179 secs 2023-09-10T09:23:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:14 smithi170 ceph-mon[102832]: osdmap e1286: 12 total, 12 up, 12 in 2023-09-10T09:23:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36226.217022 secs 2023-09-10T09:23:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36226.217022 secs 2023-09-10T09:23:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36226.217022 secs 2023-09-10T09:23:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:16 smithi082 ceph-mon[98694]: pgmap v19480: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 8 op/s 2023-09-10T09:23:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:16 smithi098 ceph-mon[103124]: pgmap v19480: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 8 op/s 2023-09-10T09:23:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:16 smithi170 ceph-mon[102832]: pgmap v19480: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 8 op/s 2023-09-10T09:23:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:18 smithi082 ceph-mon[98694]: pgmap v19481: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 14 op/s 2023-09-10T09:23:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36229.615506 secs 2023-09-10T09:23:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:18 smithi098 ceph-mon[103124]: pgmap v19481: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 14 op/s 2023-09-10T09:23:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36229.615506 secs 2023-09-10T09:23:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:18 smithi170 ceph-mon[102832]: pgmap v19481: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 14 op/s 2023-09-10T09:23:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36229.615506 secs 2023-09-10T09:23:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36230.864264 secs 2023-09-10T09:23:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36231.217099 secs 2023-09-10T09:23:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:19 smithi170 ceph-mon[102832]: pgmap v19482: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 12 op/s 2023-09-10T09:23:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36230.864264 secs 2023-09-10T09:23:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36231.217099 secs 2023-09-10T09:23:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:19 smithi082 ceph-mon[98694]: pgmap v19482: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 12 op/s 2023-09-10T09:23:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36230.864264 secs 2023-09-10T09:23:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36231.217099 secs 2023-09-10T09:23:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:19 smithi098 ceph-mon[103124]: pgmap v19482: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 12 op/s 2023-09-10T09:23:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:22 smithi170 ceph-mon[102832]: pgmap v19483: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2023-09-10T09:23:22.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:22 smithi082 ceph-mon[98694]: pgmap v19483: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2023-09-10T09:23:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:22 smithi098 ceph-mon[103124]: pgmap v19483: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2023-09-10T09:23:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36234.615569 secs 2023-09-10T09:23:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36234.615569 secs 2023-09-10T09:23:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36234.615569 secs 2023-09-10T09:23:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:24 smithi170 ceph-mon[102832]: pgmap v19484: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2023-09-10T09:23:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36235.864346 secs 2023-09-10T09:23:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:24 smithi082 ceph-mon[98694]: pgmap v19484: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2023-09-10T09:23:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36235.864346 secs 2023-09-10T09:23:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:24 smithi098 ceph-mon[103124]: pgmap v19484: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2023-09-10T09:23:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36235.864346 secs 2023-09-10T09:23:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36236.217165 secs 2023-09-10T09:23:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36236.217165 secs 2023-09-10T09:23:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36236.217165 secs 2023-09-10T09:23:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:26 smithi170 ceph-mon[102832]: pgmap v19485: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T09:23:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:23:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:23:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:23:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:23:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:26 smithi082 ceph-mon[98694]: pgmap v19485: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T09:23:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:23:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:23:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:23:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:23:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:26 smithi098 ceph-mon[103124]: pgmap v19485: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 91 B/s wr, 10 op/s 2023-09-10T09:23:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:23:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:23:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:23:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:23:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:28 smithi170 ceph-mon[102832]: pgmap v19486: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 126 KiB/s wr, 15 op/s 2023-09-10T09:23:28.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36239.615629 secs 2023-09-10T09:23:28.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:28 smithi082 ceph-mon[98694]: pgmap v19486: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 126 KiB/s wr, 15 op/s 2023-09-10T09:23:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36239.615629 secs 2023-09-10T09:23:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:28 smithi098 ceph-mon[103124]: pgmap v19486: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 126 KiB/s wr, 15 op/s 2023-09-10T09:23:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36239.615629 secs 2023-09-10T09:23:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36240.864404 secs 2023-09-10T09:23:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36241.217230 secs 2023-09-10T09:23:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:29 smithi170 ceph-mon[102832]: pgmap v19487: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 10 op/s 2023-09-10T09:23:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36240.864404 secs 2023-09-10T09:23:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36241.217230 secs 2023-09-10T09:23:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:29 smithi082 ceph-mon[98694]: pgmap v19487: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 10 op/s 2023-09-10T09:23:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36240.864404 secs 2023-09-10T09:23:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36241.217230 secs 2023-09-10T09:23:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:29 smithi098 ceph-mon[103124]: pgmap v19487: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 126 KiB/s wr, 10 op/s 2023-09-10T09:23:32.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:32 smithi170 ceph-mon[102832]: pgmap v19488: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 258 KiB/s wr, 16 op/s 2023-09-10T09:23:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:32 smithi082 ceph-mon[98694]: pgmap v19488: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 258 KiB/s wr, 16 op/s 2023-09-10T09:23:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:32 smithi098 ceph-mon[103124]: pgmap v19488: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 258 KiB/s wr, 16 op/s 2023-09-10T09:23:33.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36244.615689 secs 2023-09-10T09:23:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36244.615689 secs 2023-09-10T09:23:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36244.615689 secs 2023-09-10T09:23:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:34 smithi170 ceph-mon[102832]: pgmap v19489: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 258 KiB/s wr, 11 op/s 2023-09-10T09:23:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36245.864495 secs 2023-09-10T09:23:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:34 smithi082 ceph-mon[98694]: pgmap v19489: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 258 KiB/s wr, 11 op/s 2023-09-10T09:23:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36245.864495 secs 2023-09-10T09:23:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:34 smithi098 ceph-mon[103124]: pgmap v19489: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 258 KiB/s wr, 11 op/s 2023-09-10T09:23:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36245.864495 secs 2023-09-10T09:23:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36246.217297 secs 2023-09-10T09:23:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36246.217297 secs 2023-09-10T09:23:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36246.217297 secs 2023-09-10T09:23:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:36 smithi170 ceph-mon[102832]: pgmap v19490: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 258 KiB/s wr, 11 op/s 2023-09-10T09:23:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:23:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:36 smithi082 ceph-mon[98694]: pgmap v19490: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 258 KiB/s wr, 11 op/s 2023-09-10T09:23:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:23:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:36 smithi098 ceph-mon[103124]: pgmap v19490: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 258 KiB/s wr, 11 op/s 2023-09-10T09:23:36.908 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:23:36.908 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:38 smithi082 ceph-mon[98694]: pgmap v19491: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 387 KiB/s wr, 17 op/s 2023-09-10T09:23:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:23:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36249.615766 secs 2023-09-10T09:23:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:38 smithi098 ceph-mon[103124]: pgmap v19491: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 387 KiB/s wr, 17 op/s 2023-09-10T09:23:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:23:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36249.615766 secs 2023-09-10T09:23:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:38 smithi170 ceph-mon[102832]: pgmap v19491: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 387 KiB/s wr, 17 op/s 2023-09-10T09:23:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:23:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36249.615766 secs 2023-09-10T09:23:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36250.864554 secs 2023-09-10T09:23:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36251.217365 secs 2023-09-10T09:23:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:39 smithi170 ceph-mon[102832]: pgmap v19492: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 260 KiB/s wr, 11 op/s 2023-09-10T09:23:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36250.864554 secs 2023-09-10T09:23:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36251.217365 secs 2023-09-10T09:23:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:39 smithi082 ceph-mon[98694]: pgmap v19492: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 260 KiB/s wr, 11 op/s 2023-09-10T09:23:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36250.864554 secs 2023-09-10T09:23:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36251.217365 secs 2023-09-10T09:23:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:39 smithi098 ceph-mon[103124]: pgmap v19492: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 260 KiB/s wr, 11 op/s 2023-09-10T09:23:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:42 smithi170 ceph-mon[102832]: pgmap v19493: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 587 KiB/s wr, 18 op/s 2023-09-10T09:23:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:42 smithi082 ceph-mon[98694]: pgmap v19493: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 587 KiB/s wr, 18 op/s 2023-09-10T09:23:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:42 smithi098 ceph-mon[103124]: pgmap v19493: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 587 KiB/s wr, 18 op/s 2023-09-10T09:23:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:23:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:23:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:23:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:44 smithi170 ceph-mon[102832]: pgmap v19494: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 456 KiB/s wr, 12 op/s 2023-09-10T09:23:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:44 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:23:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:23:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36255.864614 secs 2023-09-10T09:23:44.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:44 smithi082 ceph-mon[98694]: pgmap v19494: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 456 KiB/s wr, 12 op/s 2023-09-10T09:23:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:44 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:23:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:23:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36255.864614 secs 2023-09-10T09:23:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:44 smithi098 ceph-mon[103124]: pgmap v19494: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 456 KiB/s wr, 12 op/s 2023-09-10T09:23:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:44 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:23:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:23:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36255.864614 secs 2023-09-10T09:23:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36256.217432 secs 2023-09-10T09:23:45.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36256.217432 secs 2023-09-10T09:23:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36256.217432 secs 2023-09-10T09:23:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:46 smithi170 ceph-mon[102832]: pgmap v19495: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 456 KiB/s wr, 12 op/s 2023-09-10T09:23:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:23:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:46.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:46 smithi082 ceph-mon[98694]: pgmap v19495: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 456 KiB/s wr, 12 op/s 2023-09-10T09:23:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:23:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:46 smithi098 ceph-mon[103124]: pgmap v19495: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 456 KiB/s wr, 12 op/s 2023-09-10T09:23:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:23:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:48 smithi170 ceph-mon[102832]: pgmap v19496: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 747 KiB/s wr, 20 op/s 2023-09-10T09:23:48.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:48 smithi082 ceph-mon[98694]: pgmap v19496: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 747 KiB/s wr, 20 op/s 2023-09-10T09:23:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:48 smithi098 ceph-mon[103124]: pgmap v19496: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 747 KiB/s wr, 20 op/s 2023-09-10T09:23:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36260.864675 secs 2023-09-10T09:23:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36261.217486 secs 2023-09-10T09:23:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:49 smithi170 ceph-mon[102832]: pgmap v19497: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 618 KiB/s wr, 14 op/s 2023-09-10T09:23:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:23:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36260.864675 secs 2023-09-10T09:23:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36261.217486 secs 2023-09-10T09:23:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:49 smithi082 ceph-mon[98694]: pgmap v19497: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 618 KiB/s wr, 14 op/s 2023-09-10T09:23:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:23:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36260.864675 secs 2023-09-10T09:23:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36261.217486 secs 2023-09-10T09:23:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:49 smithi098 ceph-mon[103124]: pgmap v19497: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 618 KiB/s wr, 14 op/s 2023-09-10T09:23:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:23:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:52 smithi170 ceph-mon[102832]: pgmap v19498: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 937 KiB/s wr, 23 op/s 2023-09-10T09:23:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:23:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:52 smithi082 ceph-mon[98694]: pgmap v19498: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 937 KiB/s wr, 23 op/s 2023-09-10T09:23:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:23:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:52 smithi098 ceph-mon[103124]: pgmap v19498: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 937 KiB/s wr, 23 op/s 2023-09-10T09:23:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:23:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:54 smithi170 ceph-mon[102832]: pgmap v19499: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 611 KiB/s wr, 16 op/s 2023-09-10T09:23:54.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36265.864753 secs 2023-09-10T09:23:54.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:54 smithi082 ceph-mon[98694]: pgmap v19499: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 611 KiB/s wr, 16 op/s 2023-09-10T09:23:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36265.864753 secs 2023-09-10T09:23:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:54 smithi098 ceph-mon[103124]: pgmap v19499: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 611 KiB/s wr, 16 op/s 2023-09-10T09:23:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36265.864753 secs 2023-09-10T09:23:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36266.217553 secs 2023-09-10T09:23:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36266.217553 secs 2023-09-10T09:23:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36266.217553 secs 2023-09-10T09:23:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:56 smithi082 ceph-mon[98694]: pgmap v19500: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 611 KiB/s wr, 16 op/s 2023-09-10T09:23:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:56 smithi098 ceph-mon[103124]: pgmap v19500: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 611 KiB/s wr, 16 op/s 2023-09-10T09:23:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:56 smithi170 ceph-mon[102832]: pgmap v19500: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 611 KiB/s wr, 16 op/s 2023-09-10T09:23:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:58 smithi082 ceph-mon[98694]: pgmap v19501: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 829 KiB/s wr, 24 op/s 2023-09-10T09:23:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:23:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36269.616007 secs 2023-09-10T09:23:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:58 smithi098 ceph-mon[103124]: pgmap v19501: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 829 KiB/s wr, 24 op/s 2023-09-10T09:23:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:23:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36269.616007 secs 2023-09-10T09:23:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:58 smithi170 ceph-mon[102832]: pgmap v19501: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 829 KiB/s wr, 24 op/s 2023-09-10T09:23:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:23:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:23:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36269.616007 secs 2023-09-10T09:24:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36270.864814 secs 2023-09-10T09:24:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36271.217620 secs 2023-09-10T09:24:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:23:59 smithi170 ceph-mon[102832]: pgmap v19502: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 537 KiB/s wr, 16 op/s 2023-09-10T09:24:00.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36270.864814 secs 2023-09-10T09:24:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36271.217620 secs 2023-09-10T09:24:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:23:59 smithi082 ceph-mon[98694]: pgmap v19502: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 537 KiB/s wr, 16 op/s 2023-09-10T09:24:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36270.864814 secs 2023-09-10T09:24:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36271.217620 secs 2023-09-10T09:24:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:23:59 smithi098 ceph-mon[103124]: pgmap v19502: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 537 KiB/s wr, 16 op/s 2023-09-10T09:24:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:24:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:24:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:24:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:24:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:24:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:24:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:01 smithi170 ceph-mon[102832]: pgmap v19503: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 759 KiB/s wr, 22 op/s 2023-09-10T09:24:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:01 smithi082 ceph-mon[98694]: pgmap v19503: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 759 KiB/s wr, 22 op/s 2023-09-10T09:24:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:01 smithi098 ceph-mon[103124]: pgmap v19503: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 759 KiB/s wr, 22 op/s 2023-09-10T09:24:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36274.616107 secs 2023-09-10T09:24:03.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36274.616107 secs 2023-09-10T09:24:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36274.616107 secs 2023-09-10T09:24:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:04 smithi170 ceph-mon[102832]: pgmap v19504: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 440 KiB/s wr, 14 op/s 2023-09-10T09:24:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36275.864952 secs 2023-09-10T09:24:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:24:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:04 smithi082 ceph-mon[98694]: pgmap v19504: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 440 KiB/s wr, 14 op/s 2023-09-10T09:24:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36275.864952 secs 2023-09-10T09:24:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:24:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:04 smithi098 ceph-mon[103124]: pgmap v19504: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 440 KiB/s wr, 14 op/s 2023-09-10T09:24:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36275.864952 secs 2023-09-10T09:24:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:24:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36276.217685 secs 2023-09-10T09:24:05.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36276.217685 secs 2023-09-10T09:24:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36276.217685 secs 2023-09-10T09:24:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:06 smithi170 ceph-mon[102832]: pgmap v19505: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 440 KiB/s wr, 14 op/s 2023-09-10T09:24:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:24:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:06 smithi082 ceph-mon[98694]: pgmap v19505: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 440 KiB/s wr, 14 op/s 2023-09-10T09:24:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:24:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:06 smithi098 ceph-mon[103124]: pgmap v19505: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 440 KiB/s wr, 14 op/s 2023-09-10T09:24:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:24:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:08 smithi170 ceph-mon[102832]: pgmap v19506: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 633 KiB/s wr, 35 op/s 2023-09-10T09:24:08.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36279.616229 secs 2023-09-10T09:24:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:08 smithi082 ceph-mon[98694]: pgmap v19506: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 633 KiB/s wr, 35 op/s 2023-09-10T09:24:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36279.616229 secs 2023-09-10T09:24:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:08 smithi098 ceph-mon[103124]: pgmap v19506: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 633 KiB/s wr, 35 op/s 2023-09-10T09:24:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36279.616229 secs 2023-09-10T09:24:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:09 smithi170 ceph-mon[102832]: osdmap e1287: 12 total, 12 up, 12 in 2023-09-10T09:24:09.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36280.865027 secs 2023-09-10T09:24:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:09 smithi082 ceph-mon[98694]: osdmap e1287: 12 total, 12 up, 12 in 2023-09-10T09:24:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36280.865027 secs 2023-09-10T09:24:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:09 smithi098 ceph-mon[103124]: osdmap e1287: 12 total, 12 up, 12 in 2023-09-10T09:24:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36280.865027 secs 2023-09-10T09:24:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36281.217751 secs 2023-09-10T09:24:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:10 smithi082 ceph-mon[98694]: pgmap v19508: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 498 KiB/s wr, 32 op/s 2023-09-10T09:24:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36281.217751 secs 2023-09-10T09:24:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:10 smithi098 ceph-mon[103124]: pgmap v19508: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 498 KiB/s wr, 32 op/s 2023-09-10T09:24:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36281.217751 secs 2023-09-10T09:24:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:10 smithi170 ceph-mon[102832]: pgmap v19508: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 498 KiB/s wr, 32 op/s 2023-09-10T09:24:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:12 smithi082 ceph-mon[98694]: pgmap v19509: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 232 KiB/s wr, 31 op/s 2023-09-10T09:24:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:12 smithi098 ceph-mon[103124]: pgmap v19509: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 232 KiB/s wr, 31 op/s 2023-09-10T09:24:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:12 smithi170 ceph-mon[102832]: pgmap v19509: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 232 KiB/s wr, 31 op/s 2023-09-10T09:24:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36284.616287 secs 2023-09-10T09:24:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36284.616287 secs 2023-09-10T09:24:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36284.616287 secs 2023-09-10T09:24:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:14 smithi082 ceph-mon[98694]: pgmap v19510: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 232 KiB/s wr, 31 op/s 2023-09-10T09:24:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36285.865080 secs 2023-09-10T09:24:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:14 smithi082 ceph-mon[98694]: osdmap e1288: 12 total, 12 up, 12 in 2023-09-10T09:24:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:14 smithi098 ceph-mon[103124]: pgmap v19510: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 232 KiB/s wr, 31 op/s 2023-09-10T09:24:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36285.865080 secs 2023-09-10T09:24:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:14 smithi098 ceph-mon[103124]: osdmap e1288: 12 total, 12 up, 12 in 2023-09-10T09:24:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:14 smithi170 ceph-mon[102832]: pgmap v19510: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 232 KiB/s wr, 31 op/s 2023-09-10T09:24:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36285.865080 secs 2023-09-10T09:24:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:14 smithi170 ceph-mon[102832]: osdmap e1288: 12 total, 12 up, 12 in 2023-09-10T09:24:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36286.217816 secs 2023-09-10T09:24:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36286.217816 secs 2023-09-10T09:24:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36286.217816 secs 2023-09-10T09:24:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:16 smithi082 ceph-mon[98694]: pgmap v19512: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 0 B/s wr, 8 op/s 2023-09-10T09:24:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:16 smithi098 ceph-mon[103124]: pgmap v19512: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 0 B/s wr, 8 op/s 2023-09-10T09:24:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:16 smithi170 ceph-mon[102832]: pgmap v19512: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 0 B/s wr, 8 op/s 2023-09-10T09:24:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:18 smithi082 ceph-mon[98694]: pgmap v19513: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 229 B/s wr, 14 op/s 2023-09-10T09:24:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36289.616393 secs 2023-09-10T09:24:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:18 smithi098 ceph-mon[103124]: pgmap v19513: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 229 B/s wr, 14 op/s 2023-09-10T09:24:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36289.616393 secs 2023-09-10T09:24:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:18 smithi170 ceph-mon[102832]: pgmap v19513: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 229 B/s wr, 14 op/s 2023-09-10T09:24:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36289.616393 secs 2023-09-10T09:24:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36290.865137 secs 2023-09-10T09:24:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36291.217883 secs 2023-09-10T09:24:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:19 smithi170 ceph-mon[102832]: pgmap v19514: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 13 op/s 2023-09-10T09:24:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36290.865137 secs 2023-09-10T09:24:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36291.217883 secs 2023-09-10T09:24:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:19 smithi082 ceph-mon[98694]: pgmap v19514: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 13 op/s 2023-09-10T09:24:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36290.865137 secs 2023-09-10T09:24:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36291.217883 secs 2023-09-10T09:24:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:19 smithi098 ceph-mon[103124]: pgmap v19514: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 13 op/s 2023-09-10T09:24:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:22 smithi170 ceph-mon[102832]: pgmap v19515: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T09:24:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:22 smithi082 ceph-mon[98694]: pgmap v19515: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T09:24:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:22 smithi098 ceph-mon[103124]: pgmap v19515: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T09:24:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36294.616490 secs 2023-09-10T09:24:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36294.616490 secs 2023-09-10T09:24:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36294.616490 secs 2023-09-10T09:24:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:24 smithi170 ceph-mon[102832]: pgmap v19516: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T09:24:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36295.865195 secs 2023-09-10T09:24:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:24 smithi082 ceph-mon[98694]: pgmap v19516: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T09:24:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36295.865195 secs 2023-09-10T09:24:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:24 smithi098 ceph-mon[103124]: pgmap v19516: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T09:24:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36295.865195 secs 2023-09-10T09:24:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36296.217949 secs 2023-09-10T09:24:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36296.217949 secs 2023-09-10T09:24:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36296.217949 secs 2023-09-10T09:24:26.408 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:26 smithi082 ceph-mon[98694]: pgmap v19517: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 182 B/s wr, 10 op/s 2023-09-10T09:24:26.408 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:24:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:26 smithi098 ceph-mon[103124]: pgmap v19517: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 182 B/s wr, 10 op/s 2023-09-10T09:24:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:24:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:26 smithi170 ceph-mon[102832]: pgmap v19517: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 182 B/s wr, 10 op/s 2023-09-10T09:24:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:24:28.358 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:28 smithi098 ceph-mon[103124]: pgmap v19518: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 597 B/s wr, 14 op/s 2023-09-10T09:24:28.358 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36299.616551 secs 2023-09-10T09:24:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:28 smithi082 ceph-mon[98694]: pgmap v19518: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 597 B/s wr, 14 op/s 2023-09-10T09:24:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36299.616551 secs 2023-09-10T09:24:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:28 smithi170 ceph-mon[102832]: pgmap v19518: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 597 B/s wr, 14 op/s 2023-09-10T09:24:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36299.616551 secs 2023-09-10T09:24:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36300.865255 secs 2023-09-10T09:24:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36301.218016 secs 2023-09-10T09:24:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:29 smithi170 ceph-mon[102832]: pgmap v19519: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-10T09:24:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:24:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:24:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36300.865255 secs 2023-09-10T09:24:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36301.218016 secs 2023-09-10T09:24:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:29 smithi082 ceph-mon[98694]: pgmap v19519: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-10T09:24:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:24:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:24:30.237 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36300.865255 secs 2023-09-10T09:24:30.237 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36301.218016 secs 2023-09-10T09:24:30.238 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:29 smithi098 ceph-mon[103124]: pgmap v19519: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 9 op/s 2023-09-10T09:24:30.238 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:24:30.238 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:24:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:24:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:24:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:24:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:24:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:24:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:24:31.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:24:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:24:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:24:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:24:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:24:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:24:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:24:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:24:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:24:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:24:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:24:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:24:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:24:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:24:31.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:24:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:31 smithi170 ceph-mon[102832]: pgmap v19520: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 132 KiB/s wr, 14 op/s 2023-09-10T09:24:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:31 smithi082 ceph-mon[98694]: pgmap v19520: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 132 KiB/s wr, 14 op/s 2023-09-10T09:24:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:31 smithi098 ceph-mon[103124]: pgmap v19520: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 132 KiB/s wr, 14 op/s 2023-09-10T09:24:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:34 smithi170 ceph-mon[102832]: pgmap v19521: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 9 op/s 2023-09-10T09:24:34.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36305.865313 secs 2023-09-10T09:24:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:34 smithi082 ceph-mon[98694]: pgmap v19521: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 9 op/s 2023-09-10T09:24:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36305.865313 secs 2023-09-10T09:24:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:34 smithi098 ceph-mon[103124]: pgmap v19521: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 9 op/s 2023-09-10T09:24:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36305.865313 secs 2023-09-10T09:24:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36306.218083 secs 2023-09-10T09:24:35.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:24:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36306.218083 secs 2023-09-10T09:24:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:24:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36306.218083 secs 2023-09-10T09:24:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:24:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:36 smithi170 ceph-mon[102832]: pgmap v19522: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 9 op/s 2023-09-10T09:24:36.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:24:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:24:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:36.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:36 smithi082 ceph-mon[98694]: pgmap v19522: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 9 op/s 2023-09-10T09:24:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:24:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:24:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:36 smithi098 ceph-mon[103124]: pgmap v19522: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 9 op/s 2023-09-10T09:24:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:24:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:24:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:38 smithi082 ceph-mon[98694]: pgmap v19523: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 261 KiB/s wr, 15 op/s 2023-09-10T09:24:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:24:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36309.616677 secs 2023-09-10T09:24:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:38 smithi098 ceph-mon[103124]: pgmap v19523: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 261 KiB/s wr, 15 op/s 2023-09-10T09:24:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:24:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36309.616677 secs 2023-09-10T09:24:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:38 smithi170 ceph-mon[102832]: pgmap v19523: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 261 KiB/s wr, 15 op/s 2023-09-10T09:24:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:24:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36309.616677 secs 2023-09-10T09:24:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36310.865371 secs 2023-09-10T09:24:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36311.218148 secs 2023-09-10T09:24:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:39 smithi170 ceph-mon[102832]: pgmap v19524: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 261 KiB/s wr, 10 op/s 2023-09-10T09:24:40.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36310.865371 secs 2023-09-10T09:24:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36311.218148 secs 2023-09-10T09:24:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:39 smithi082 ceph-mon[98694]: pgmap v19524: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 261 KiB/s wr, 10 op/s 2023-09-10T09:24:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36310.865371 secs 2023-09-10T09:24:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36311.218148 secs 2023-09-10T09:24:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:39 smithi098 ceph-mon[103124]: pgmap v19524: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 261 KiB/s wr, 10 op/s 2023-09-10T09:24:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:24:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:24:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:24:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:41 smithi170 ceph-mon[102832]: pgmap v19525: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 579 KiB/s wr, 18 op/s 2023-09-10T09:24:42.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:41 smithi082 ceph-mon[98694]: pgmap v19525: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 579 KiB/s wr, 18 op/s 2023-09-10T09:24:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:41 smithi098 ceph-mon[103124]: pgmap v19525: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 579 KiB/s wr, 18 op/s 2023-09-10T09:24:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:24:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:24:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:24:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:44 smithi170 ceph-mon[102832]: pgmap v19526: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 448 KiB/s wr, 13 op/s 2023-09-10T09:24:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:44 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:24:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:24:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:44.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36315.865459 secs 2023-09-10T09:24:44.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:44 smithi082 ceph-mon[98694]: pgmap v19526: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 448 KiB/s wr, 13 op/s 2023-09-10T09:24:44.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:44 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:24:44.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:24:44.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:44.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36315.865459 secs 2023-09-10T09:24:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:44 smithi098 ceph-mon[103124]: pgmap v19526: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 448 KiB/s wr, 13 op/s 2023-09-10T09:24:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:44 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:24:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:24:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:44.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36315.865459 secs 2023-09-10T09:24:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36316.218219 secs 2023-09-10T09:24:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36316.218219 secs 2023-09-10T09:24:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36316.218219 secs 2023-09-10T09:24:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:46 smithi082 ceph-mon[98694]: pgmap v19527: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 448 KiB/s wr, 13 op/s 2023-09-10T09:24:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:24:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:46 smithi098 ceph-mon[103124]: pgmap v19527: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 448 KiB/s wr, 13 op/s 2023-09-10T09:24:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:24:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:46 smithi170 ceph-mon[102832]: pgmap v19527: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 448 KiB/s wr, 13 op/s 2023-09-10T09:24:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:24:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:48 smithi082 ceph-mon[98694]: pgmap v19528: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 473 KiB/s wr, 20 op/s 2023-09-10T09:24:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:48 smithi098 ceph-mon[103124]: pgmap v19528: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 473 KiB/s wr, 20 op/s 2023-09-10T09:24:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:48 smithi170 ceph-mon[102832]: pgmap v19528: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 473 KiB/s wr, 20 op/s 2023-09-10T09:24:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36320.865543 secs 2023-09-10T09:24:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36321.218284 secs 2023-09-10T09:24:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:49 smithi170 ceph-mon[102832]: pgmap v19529: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 344 KiB/s wr, 14 op/s 2023-09-10T09:24:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36320.865543 secs 2023-09-10T09:24:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36321.218284 secs 2023-09-10T09:24:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:49 smithi082 ceph-mon[98694]: pgmap v19529: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 344 KiB/s wr, 14 op/s 2023-09-10T09:24:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36320.865543 secs 2023-09-10T09:24:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36321.218284 secs 2023-09-10T09:24:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:49 smithi098 ceph-mon[103124]: pgmap v19529: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 344 KiB/s wr, 14 op/s 2023-09-10T09:24:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:52 smithi170 ceph-mon[102832]: pgmap v19530: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 810 KiB/s wr, 21 op/s 2023-09-10T09:24:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:24:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:52 smithi082 ceph-mon[98694]: pgmap v19530: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 810 KiB/s wr, 21 op/s 2023-09-10T09:24:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:24:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:52 smithi098 ceph-mon[103124]: pgmap v19530: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 810 KiB/s wr, 21 op/s 2023-09-10T09:24:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:24:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:54 smithi082 ceph-mon[98694]: pgmap v19531: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 492 KiB/s wr, 14 op/s 2023-09-10T09:24:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:24:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36325.865633 secs 2023-09-10T09:24:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:54 smithi098 ceph-mon[103124]: pgmap v19531: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 492 KiB/s wr, 14 op/s 2023-09-10T09:24:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:24:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36325.865633 secs 2023-09-10T09:24:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:54 smithi170 ceph-mon[102832]: pgmap v19531: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 492 KiB/s wr, 14 op/s 2023-09-10T09:24:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:24:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36325.865633 secs 2023-09-10T09:24:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36326.218351 secs 2023-09-10T09:24:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36326.218351 secs 2023-09-10T09:24:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36326.218351 secs 2023-09-10T09:24:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:56 smithi082 ceph-mon[98694]: pgmap v19532: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 492 KiB/s wr, 14 op/s 2023-09-10T09:24:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:56 smithi098 ceph-mon[103124]: pgmap v19532: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 492 KiB/s wr, 14 op/s 2023-09-10T09:24:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:56 smithi170 ceph-mon[102832]: pgmap v19532: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 492 KiB/s wr, 14 op/s 2023-09-10T09:24:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:58 smithi082 ceph-mon[98694]: pgmap v19533: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 968 KiB/s wr, 21 op/s 2023-09-10T09:24:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:24:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36329.616956 secs 2023-09-10T09:24:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:58 smithi098 ceph-mon[103124]: pgmap v19533: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 968 KiB/s wr, 21 op/s 2023-09-10T09:24:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:24:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36329.616956 secs 2023-09-10T09:24:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:58 smithi170 ceph-mon[102832]: pgmap v19533: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 968 KiB/s wr, 21 op/s 2023-09-10T09:24:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:24:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:24:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36329.616956 secs 2023-09-10T09:25:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36330.865696 secs 2023-09-10T09:25:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36331.218417 secs 2023-09-10T09:25:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:24:59 smithi170 ceph-mon[102832]: pgmap v19534: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 942 KiB/s wr, 14 op/s 2023-09-10T09:25:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36330.865696 secs 2023-09-10T09:25:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36331.218417 secs 2023-09-10T09:25:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:24:59 smithi082 ceph-mon[98694]: pgmap v19534: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 942 KiB/s wr, 14 op/s 2023-09-10T09:25:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36330.865696 secs 2023-09-10T09:25:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36331.218417 secs 2023-09-10T09:25:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:24:59 smithi098 ceph-mon[103124]: pgmap v19534: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 942 KiB/s wr, 14 op/s 2023-09-10T09:25:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:25:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:25:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:25:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:25:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:25:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:25:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:01 smithi170 ceph-mon[102832]: pgmap v19535: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 21 op/s 2023-09-10T09:25:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:01 smithi082 ceph-mon[98694]: pgmap v19535: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 21 op/s 2023-09-10T09:25:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:01 smithi098 ceph-mon[103124]: pgmap v19535: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 21 op/s 2023-09-10T09:25:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36334.617016 secs 2023-09-10T09:25:03.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36334.617016 secs 2023-09-10T09:25:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36334.617016 secs 2023-09-10T09:25:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:04 smithi170 ceph-mon[102832]: pgmap v19536: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 886 KiB/s wr, 14 op/s 2023-09-10T09:25:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:25:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36335.865768 secs 2023-09-10T09:25:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:04 smithi082 ceph-mon[98694]: pgmap v19536: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 886 KiB/s wr, 14 op/s 2023-09-10T09:25:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:25:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36335.865768 secs 2023-09-10T09:25:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:04 smithi098 ceph-mon[103124]: pgmap v19536: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 886 KiB/s wr, 14 op/s 2023-09-10T09:25:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:25:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36335.865768 secs 2023-09-10T09:25:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36336.218486 secs 2023-09-10T09:25:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36336.218486 secs 2023-09-10T09:25:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36336.218486 secs 2023-09-10T09:25:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:06 smithi082 ceph-mon[98694]: pgmap v19537: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 886 KiB/s wr, 14 op/s 2023-09-10T09:25:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:25:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:06 smithi098 ceph-mon[103124]: pgmap v19537: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 886 KiB/s wr, 14 op/s 2023-09-10T09:25:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:25:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:06 smithi170 ceph-mon[102832]: pgmap v19537: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 886 KiB/s wr, 14 op/s 2023-09-10T09:25:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:25:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:08 smithi082 ceph-mon[98694]: pgmap v19538: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 35 op/s 2023-09-10T09:25:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36339.617076 secs 2023-09-10T09:25:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:08 smithi098 ceph-mon[103124]: pgmap v19538: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 35 op/s 2023-09-10T09:25:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36339.617076 secs 2023-09-10T09:25:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:08 smithi170 ceph-mon[102832]: pgmap v19538: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 35 op/s 2023-09-10T09:25:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36339.617076 secs 2023-09-10T09:25:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:09 smithi082 ceph-mon[98694]: osdmap e1289: 12 total, 12 up, 12 in 2023-09-10T09:25:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36340.865847 secs 2023-09-10T09:25:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:09 smithi098 ceph-mon[103124]: osdmap e1289: 12 total, 12 up, 12 in 2023-09-10T09:25:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36340.865847 secs 2023-09-10T09:25:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:09 smithi170 ceph-mon[102832]: osdmap e1289: 12 total, 12 up, 12 in 2023-09-10T09:25:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36340.865847 secs 2023-09-10T09:25:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36341.218554 secs 2023-09-10T09:25:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:10 smithi082 ceph-mon[98694]: pgmap v19540: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 888 KiB/s wr, 33 op/s 2023-09-10T09:25:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36341.218554 secs 2023-09-10T09:25:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:10 smithi098 ceph-mon[103124]: pgmap v19540: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 888 KiB/s wr, 33 op/s 2023-09-10T09:25:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36341.218554 secs 2023-09-10T09:25:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:10 smithi170 ceph-mon[102832]: pgmap v19540: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 888 KiB/s wr, 33 op/s 2023-09-10T09:25:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:12 smithi082 ceph-mon[98694]: pgmap v19541: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 522 KiB/s wr, 31 op/s 2023-09-10T09:25:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:12 smithi098 ceph-mon[103124]: pgmap v19541: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 522 KiB/s wr, 31 op/s 2023-09-10T09:25:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:12 smithi170 ceph-mon[102832]: pgmap v19541: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 522 KiB/s wr, 31 op/s 2023-09-10T09:25:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36344.617138 secs 2023-09-10T09:25:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36344.617138 secs 2023-09-10T09:25:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36344.617138 secs 2023-09-10T09:25:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:14 smithi098 ceph-mon[103124]: pgmap v19542: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 522 KiB/s wr, 31 op/s 2023-09-10T09:25:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36345.865908 secs 2023-09-10T09:25:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:14 smithi098 ceph-mon[103124]: osdmap e1290: 12 total, 12 up, 12 in 2023-09-10T09:25:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:14 smithi170 ceph-mon[102832]: pgmap v19542: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 522 KiB/s wr, 31 op/s 2023-09-10T09:25:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36345.865908 secs 2023-09-10T09:25:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:14 smithi170 ceph-mon[102832]: osdmap e1290: 12 total, 12 up, 12 in 2023-09-10T09:25:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:14 smithi082 ceph-mon[98694]: pgmap v19542: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 522 KiB/s wr, 31 op/s 2023-09-10T09:25:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36345.865908 secs 2023-09-10T09:25:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:14 smithi082 ceph-mon[98694]: osdmap e1290: 12 total, 12 up, 12 in 2023-09-10T09:25:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36346.218617 secs 2023-09-10T09:25:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36346.218617 secs 2023-09-10T09:25:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36346.218617 secs 2023-09-10T09:25:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:16 smithi098 ceph-mon[103124]: pgmap v19544: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 157 KiB/s wr, 8 op/s 2023-09-10T09:25:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:25:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:16 smithi170 ceph-mon[102832]: pgmap v19544: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 157 KiB/s wr, 8 op/s 2023-09-10T09:25:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:25:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:16 smithi082 ceph-mon[98694]: pgmap v19544: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 157 KiB/s wr, 8 op/s 2023-09-10T09:25:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:25:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:18 smithi098 ceph-mon[103124]: pgmap v19545: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 142 KiB/s wr, 14 op/s 2023-09-10T09:25:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:25:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36349.617230 secs 2023-09-10T09:25:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:18 smithi170 ceph-mon[102832]: pgmap v19545: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 142 KiB/s wr, 14 op/s 2023-09-10T09:25:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:25:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36349.617230 secs 2023-09-10T09:25:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:18 smithi082 ceph-mon[98694]: pgmap v19545: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 142 KiB/s wr, 14 op/s 2023-09-10T09:25:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:25:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36349.617230 secs 2023-09-10T09:25:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36350.865966 secs 2023-09-10T09:25:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36351.218687 secs 2023-09-10T09:25:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:19 smithi170 ceph-mon[102832]: pgmap v19546: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 13 op/s 2023-09-10T09:25:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36350.865966 secs 2023-09-10T09:25:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36351.218687 secs 2023-09-10T09:25:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:19 smithi082 ceph-mon[98694]: pgmap v19546: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 13 op/s 2023-09-10T09:25:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36350.865966 secs 2023-09-10T09:25:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36351.218687 secs 2023-09-10T09:25:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:19 smithi098 ceph-mon[103124]: pgmap v19546: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 127 KiB/s wr, 13 op/s 2023-09-10T09:25:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:22 smithi082 ceph-mon[98694]: pgmap v19547: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 13 op/s 2023-09-10T09:25:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:22 smithi098 ceph-mon[103124]: pgmap v19547: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 13 op/s 2023-09-10T09:25:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:22 smithi170 ceph-mon[102832]: pgmap v19547: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 13 op/s 2023-09-10T09:25:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36354.617331 secs 2023-09-10T09:25:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36354.617331 secs 2023-09-10T09:25:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36354.617331 secs 2023-09-10T09:25:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:24 smithi082 ceph-mon[98694]: pgmap v19548: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 13 op/s 2023-09-10T09:25:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36355.866025 secs 2023-09-10T09:25:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:24 smithi098 ceph-mon[103124]: pgmap v19548: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 13 op/s 2023-09-10T09:25:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36355.866025 secs 2023-09-10T09:25:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:24 smithi170 ceph-mon[102832]: pgmap v19548: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 155 KiB/s wr, 13 op/s 2023-09-10T09:25:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36355.866025 secs 2023-09-10T09:25:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36356.218751 secs 2023-09-10T09:25:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36356.218751 secs 2023-09-10T09:25:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36356.218751 secs 2023-09-10T09:25:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:26 smithi082 ceph-mon[98694]: pgmap v19549: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T09:25:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:26 smithi098 ceph-mon[103124]: pgmap v19549: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T09:25:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:26 smithi170 ceph-mon[102832]: pgmap v19549: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T09:25:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:28 smithi082 ceph-mon[98694]: pgmap v19550: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 129 KiB/s wr, 16 op/s 2023-09-10T09:25:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36359.617408 secs 2023-09-10T09:25:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:28 smithi098 ceph-mon[103124]: pgmap v19550: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 129 KiB/s wr, 16 op/s 2023-09-10T09:25:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36359.617408 secs 2023-09-10T09:25:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:28 smithi170 ceph-mon[102832]: pgmap v19550: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 129 KiB/s wr, 16 op/s 2023-09-10T09:25:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36359.617408 secs 2023-09-10T09:25:30.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36360.866082 secs 2023-09-10T09:25:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36361.218817 secs 2023-09-10T09:25:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:29 smithi170 ceph-mon[102832]: pgmap v19551: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 10 op/s 2023-09-10T09:25:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36360.866082 secs 2023-09-10T09:25:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36361.218817 secs 2023-09-10T09:25:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:29 smithi082 ceph-mon[98694]: pgmap v19551: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 10 op/s 2023-09-10T09:25:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36360.866082 secs 2023-09-10T09:25:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36361.218817 secs 2023-09-10T09:25:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:29 smithi098 ceph-mon[103124]: pgmap v19551: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 10 op/s 2023-09-10T09:25:31.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:25:31.206 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:25:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:25:32.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:25:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:25:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:25:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:31 smithi170 ceph-mon[102832]: pgmap v19552: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 129 KiB/s wr, 16 op/s 2023-09-10T09:25:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:25:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:25:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:25:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:31 smithi082 ceph-mon[98694]: pgmap v19552: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 129 KiB/s wr, 16 op/s 2023-09-10T09:25:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:25:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:25:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:25:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:31 smithi098 ceph-mon[103124]: pgmap v19552: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 129 KiB/s wr, 16 op/s 2023-09-10T09:25:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:34 smithi082 ceph-mon[98694]: pgmap v19553: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 511 B/s wr, 10 op/s 2023-09-10T09:25:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36365.866140 secs 2023-09-10T09:25:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:34 smithi098 ceph-mon[103124]: pgmap v19553: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 511 B/s wr, 10 op/s 2023-09-10T09:25:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36365.866140 secs 2023-09-10T09:25:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:34 smithi170 ceph-mon[102832]: pgmap v19553: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 511 B/s wr, 10 op/s 2023-09-10T09:25:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36365.866140 secs 2023-09-10T09:25:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36366.218885 secs 2023-09-10T09:25:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:25:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36366.218885 secs 2023-09-10T09:25:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:25:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36366.218885 secs 2023-09-10T09:25:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:25:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:36 smithi082 ceph-mon[98694]: pgmap v19554: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 511 B/s wr, 10 op/s 2023-09-10T09:25:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:25:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:25:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:36 smithi098 ceph-mon[103124]: pgmap v19554: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 511 B/s wr, 10 op/s 2023-09-10T09:25:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:25:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:25:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:36 smithi170 ceph-mon[102832]: pgmap v19554: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 511 B/s wr, 10 op/s 2023-09-10T09:25:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:25:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:25:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:38 smithi082 ceph-mon[98694]: pgmap v19555: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 262 KiB/s wr, 16 op/s 2023-09-10T09:25:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:38 smithi098 ceph-mon[103124]: pgmap v19555: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 262 KiB/s wr, 16 op/s 2023-09-10T09:25:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:38 smithi170 ceph-mon[102832]: pgmap v19555: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 262 KiB/s wr, 16 op/s 2023-09-10T09:25:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36370.866200 secs 2023-09-10T09:25:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36371.218949 secs 2023-09-10T09:25:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:39 smithi170 ceph-mon[102832]: pgmap v19556: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 262 KiB/s wr, 11 op/s 2023-09-10T09:25:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36370.866200 secs 2023-09-10T09:25:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36371.218949 secs 2023-09-10T09:25:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:39 smithi082 ceph-mon[98694]: pgmap v19556: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 262 KiB/s wr, 11 op/s 2023-09-10T09:25:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36370.866200 secs 2023-09-10T09:25:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36371.218949 secs 2023-09-10T09:25:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:39 smithi098 ceph-mon[103124]: pgmap v19556: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 262 KiB/s wr, 11 op/s 2023-09-10T09:25:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:25:41.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:25:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:25:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:42.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:41 smithi170 ceph-mon[102832]: pgmap v19557: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 418 KiB/s wr, 17 op/s 2023-09-10T09:25:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:41 smithi082 ceph-mon[98694]: pgmap v19557: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 418 KiB/s wr, 17 op/s 2023-09-10T09:25:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:41 smithi098 ceph-mon[103124]: pgmap v19557: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 418 KiB/s wr, 17 op/s 2023-09-10T09:25:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36374.617701 secs 2023-09-10T09:25:43.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36374.617701 secs 2023-09-10T09:25:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36374.617701 secs 2023-09-10T09:25:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:44 smithi170 ceph-mon[102832]: pgmap v19558: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 418 KiB/s wr, 12 op/s 2023-09-10T09:25:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:25:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:25:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:44 smithi082 ceph-mon[98694]: pgmap v19558: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 418 KiB/s wr, 12 op/s 2023-09-10T09:25:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:25:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:25:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:44 smithi098 ceph-mon[103124]: pgmap v19558: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 418 KiB/s wr, 12 op/s 2023-09-10T09:25:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:25:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:25:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36375.866271 secs 2023-09-10T09:25:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36376.219018 secs 2023-09-10T09:25:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36375.866271 secs 2023-09-10T09:25:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36376.219018 secs 2023-09-10T09:25:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36375.866271 secs 2023-09-10T09:25:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36376.219018 secs 2023-09-10T09:25:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:46 smithi170 ceph-mon[102832]: pgmap v19559: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 418 KiB/s wr, 12 op/s 2023-09-10T09:25:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:25:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:46.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:46 smithi082 ceph-mon[98694]: pgmap v19559: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 418 KiB/s wr, 12 op/s 2023-09-10T09:25:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:25:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:46 smithi098 ceph-mon[103124]: pgmap v19559: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 418 KiB/s wr, 12 op/s 2023-09-10T09:25:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:25:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:48 smithi170 ceph-mon[102832]: pgmap v19560: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 596 KiB/s wr, 18 op/s 2023-09-10T09:25:48.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36379.617769 secs 2023-09-10T09:25:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:48 smithi082 ceph-mon[98694]: pgmap v19560: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 596 KiB/s wr, 18 op/s 2023-09-10T09:25:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36379.617769 secs 2023-09-10T09:25:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:48 smithi098 ceph-mon[103124]: pgmap v19560: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 596 KiB/s wr, 18 op/s 2023-09-10T09:25:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36379.617769 secs 2023-09-10T09:25:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36380.866328 secs 2023-09-10T09:25:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36381.219082 secs 2023-09-10T09:25:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:49 smithi170 ceph-mon[102832]: pgmap v19561: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 335 KiB/s wr, 12 op/s 2023-09-10T09:25:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:25:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36380.866328 secs 2023-09-10T09:25:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36381.219082 secs 2023-09-10T09:25:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:49 smithi082 ceph-mon[98694]: pgmap v19561: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 335 KiB/s wr, 12 op/s 2023-09-10T09:25:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:25:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36380.866328 secs 2023-09-10T09:25:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36381.219082 secs 2023-09-10T09:25:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:49 smithi098 ceph-mon[103124]: pgmap v19561: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 335 KiB/s wr, 12 op/s 2023-09-10T09:25:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:25:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:52.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:52 smithi170 ceph-mon[102832]: pgmap v19562: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 704 KiB/s wr, 20 op/s 2023-09-10T09:25:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:25:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:52 smithi082 ceph-mon[98694]: pgmap v19562: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 704 KiB/s wr, 20 op/s 2023-09-10T09:25:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:25:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:52 smithi098 ceph-mon[103124]: pgmap v19562: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 704 KiB/s wr, 20 op/s 2023-09-10T09:25:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:25:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36384.617830 secs 2023-09-10T09:25:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36384.617830 secs 2023-09-10T09:25:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36384.617830 secs 2023-09-10T09:25:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:54 smithi082 ceph-mon[98694]: pgmap v19563: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 547 KiB/s wr, 14 op/s 2023-09-10T09:25:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36385.866385 secs 2023-09-10T09:25:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:54 smithi098 ceph-mon[103124]: pgmap v19563: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 547 KiB/s wr, 14 op/s 2023-09-10T09:25:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36385.866385 secs 2023-09-10T09:25:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:54 smithi170 ceph-mon[102832]: pgmap v19563: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 547 KiB/s wr, 14 op/s 2023-09-10T09:25:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36385.866385 secs 2023-09-10T09:25:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36386.219148 secs 2023-09-10T09:25:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36386.219148 secs 2023-09-10T09:25:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36386.219148 secs 2023-09-10T09:25:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:56 smithi082 ceph-mon[98694]: pgmap v19564: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 548 KiB/s wr, 14 op/s 2023-09-10T09:25:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:56 smithi098 ceph-mon[103124]: pgmap v19564: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 548 KiB/s wr, 14 op/s 2023-09-10T09:25:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:56 smithi170 ceph-mon[102832]: pgmap v19564: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 548 KiB/s wr, 14 op/s 2023-09-10T09:25:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:58 smithi082 ceph-mon[98694]: pgmap v19565: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 565 KiB/s wr, 20 op/s 2023-09-10T09:25:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:25:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:58 smithi098 ceph-mon[103124]: pgmap v19565: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 565 KiB/s wr, 20 op/s 2023-09-10T09:25:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:25:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:25:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:58 smithi170 ceph-mon[102832]: pgmap v19565: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 565 KiB/s wr, 20 op/s 2023-09-10T09:25:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:25:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:00.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36390.866471 secs 2023-09-10T09:26:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36391.219215 secs 2023-09-10T09:26:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:59 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:26:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:25:59 smithi170 ceph-mon[102832]: pgmap v19566: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 387 KiB/s wr, 13 op/s 2023-09-10T09:26:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36390.866471 secs 2023-09-10T09:26:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36391.219215 secs 2023-09-10T09:26:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:59 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:26:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:25:59 smithi082 ceph-mon[98694]: pgmap v19566: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 387 KiB/s wr, 13 op/s 2023-09-10T09:26:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36390.866471 secs 2023-09-10T09:26:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36391.219215 secs 2023-09-10T09:26:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:59 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:26:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:25:59 smithi098 ceph-mon[103124]: pgmap v19566: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 387 KiB/s wr, 13 op/s 2023-09-10T09:26:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:00 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:26:01.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:26:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:00 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:26:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:26:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:00 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:26:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:26:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:02.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:01 smithi170 ceph-mon[102832]: pgmap v19567: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 36 op/s 2023-09-10T09:26:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:01 smithi082 ceph-mon[98694]: pgmap v19567: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 36 op/s 2023-09-10T09:26:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:01 smithi098 ceph-mon[103124]: pgmap v19567: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 36 op/s 2023-09-10T09:26:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36394.617951 secs 2023-09-10T09:26:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:03 smithi098 ceph-mon[103124]: osdmap e1291: 12 total, 12 up, 12 in 2023-09-10T09:26:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:03 smithi098 ceph-mon[103124]: pgmap v19569: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 34 op/s 2023-09-10T09:26:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36394.617951 secs 2023-09-10T09:26:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:03 smithi170 ceph-mon[102832]: osdmap e1291: 12 total, 12 up, 12 in 2023-09-10T09:26:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:03 smithi170 ceph-mon[102832]: pgmap v19569: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 34 op/s 2023-09-10T09:26:04.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36394.617951 secs 2023-09-10T09:26:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:03 smithi082 ceph-mon[98694]: osdmap e1291: 12 total, 12 up, 12 in 2023-09-10T09:26:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:03 smithi082 ceph-mon[98694]: pgmap v19569: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 34 op/s 2023-09-10T09:26:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36395.866498 secs 2023-09-10T09:26:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36396.219279 secs 2023-09-10T09:26:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:26:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36395.866498 secs 2023-09-10T09:26:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36396.219279 secs 2023-09-10T09:26:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:26:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:05.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36395.866498 secs 2023-09-10T09:26:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36396.219279 secs 2023-09-10T09:26:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:26:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:05 smithi098 ceph-mon[103124]: pgmap v19570: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 34 op/s 2023-09-10T09:26:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:05 smithi170 ceph-mon[102832]: pgmap v19570: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 34 op/s 2023-09-10T09:26:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:05 smithi082 ceph-mon[98694]: pgmap v19570: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 34 op/s 2023-09-10T09:26:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:08 smithi082 ceph-mon[98694]: pgmap v19571: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 36 op/s 2023-09-10T09:26:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36399.618016 secs 2023-09-10T09:26:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:08 smithi098 ceph-mon[103124]: pgmap v19571: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 36 op/s 2023-09-10T09:26:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36399.618016 secs 2023-09-10T09:26:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:08 smithi170 ceph-mon[102832]: pgmap v19571: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 36 op/s 2023-09-10T09:26:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36399.618016 secs 2023-09-10T09:26:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36400.866554 secs 2023-09-10T09:26:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:09 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:26:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:09 smithi082 ceph-mon[98694]: osdmap e1292: 12 total, 12 up, 12 in 2023-09-10T09:26:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36400.866554 secs 2023-09-10T09:26:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:09 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:26:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:09 smithi098 ceph-mon[103124]: osdmap e1292: 12 total, 12 up, 12 in 2023-09-10T09:26:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36400.866554 secs 2023-09-10T09:26:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:09 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:26:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:09 smithi170 ceph-mon[102832]: osdmap e1292: 12 total, 12 up, 12 in 2023-09-10T09:26:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36401.219345 secs 2023-09-10T09:26:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:10 smithi082 ceph-mon[98694]: pgmap v19573: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 364 KiB/s wr, 11 op/s 2023-09-10T09:26:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:26:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36401.219345 secs 2023-09-10T09:26:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:10 smithi098 ceph-mon[103124]: pgmap v19573: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 364 KiB/s wr, 11 op/s 2023-09-10T09:26:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:26:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36401.219345 secs 2023-09-10T09:26:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:10 smithi170 ceph-mon[102832]: pgmap v19573: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 364 KiB/s wr, 11 op/s 2023-09-10T09:26:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:26:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:12 smithi098 ceph-mon[103124]: pgmap v19574: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 816 KiB/s wr, 20 op/s 2023-09-10T09:26:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:26:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:12 smithi170 ceph-mon[102832]: pgmap v19574: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 816 KiB/s wr, 20 op/s 2023-09-10T09:26:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:26:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:12.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:12 smithi082 ceph-mon[98694]: pgmap v19574: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 816 KiB/s wr, 20 op/s 2023-09-10T09:26:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:26:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36404.618077 secs 2023-09-10T09:26:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36404.618077 secs 2023-09-10T09:26:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36404.618077 secs 2023-09-10T09:26:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:14 smithi098 ceph-mon[103124]: pgmap v19575: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 667 KiB/s wr, 16 op/s 2023-09-10T09:26:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36405.866613 secs 2023-09-10T09:26:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:14 smithi170 ceph-mon[102832]: pgmap v19575: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 667 KiB/s wr, 16 op/s 2023-09-10T09:26:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36405.866613 secs 2023-09-10T09:26:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:14 smithi082 ceph-mon[98694]: pgmap v19575: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 667 KiB/s wr, 16 op/s 2023-09-10T09:26:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36405.866613 secs 2023-09-10T09:26:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36406.219411 secs 2023-09-10T09:26:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36406.219411 secs 2023-09-10T09:26:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36406.219411 secs 2023-09-10T09:26:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:16 smithi098 ceph-mon[103124]: pgmap v19576: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 667 KiB/s wr, 16 op/s 2023-09-10T09:26:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:16 smithi170 ceph-mon[102832]: pgmap v19576: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 667 KiB/s wr, 16 op/s 2023-09-10T09:26:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:16 smithi082 ceph-mon[98694]: pgmap v19576: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 667 KiB/s wr, 16 op/s 2023-09-10T09:26:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:18 smithi098 ceph-mon[103124]: pgmap v19577: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 534 KiB/s wr, 15 op/s 2023-09-10T09:26:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:26:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36409.618138 secs 2023-09-10T09:26:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:18 smithi170 ceph-mon[102832]: pgmap v19577: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 534 KiB/s wr, 15 op/s 2023-09-10T09:26:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:26:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36409.618138 secs 2023-09-10T09:26:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:18 smithi082 ceph-mon[98694]: pgmap v19577: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 534 KiB/s wr, 15 op/s 2023-09-10T09:26:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:26:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36409.618138 secs 2023-09-10T09:26:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36410.866673 secs 2023-09-10T09:26:20.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36411.219480 secs 2023-09-10T09:26:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:19 smithi170 ceph-mon[102832]: pgmap v19578: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 522 KiB/s wr, 15 op/s 2023-09-10T09:26:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36410.866673 secs 2023-09-10T09:26:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36411.219480 secs 2023-09-10T09:26:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:19 smithi082 ceph-mon[98694]: pgmap v19578: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 522 KiB/s wr, 15 op/s 2023-09-10T09:26:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36410.866673 secs 2023-09-10T09:26:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36411.219480 secs 2023-09-10T09:26:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:19 smithi098 ceph-mon[103124]: pgmap v19578: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 522 KiB/s wr, 15 op/s 2023-09-10T09:26:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:26:21.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:26:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:26:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:22.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:21 smithi170 ceph-mon[102832]: pgmap v19579: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 446 KiB/s wr, 18 op/s 2023-09-10T09:26:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:21 smithi082 ceph-mon[98694]: pgmap v19579: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 446 KiB/s wr, 18 op/s 2023-09-10T09:26:22.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:21 smithi098 ceph-mon[103124]: pgmap v19579: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 446 KiB/s wr, 18 op/s 2023-09-10T09:26:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36414.618226 secs 2023-09-10T09:26:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36414.618226 secs 2023-09-10T09:26:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36414.618226 secs 2023-09-10T09:26:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:24 smithi170 ceph-mon[102832]: pgmap v19580: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 12 op/s 2023-09-10T09:26:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:24 smithi082 ceph-mon[98694]: pgmap v19580: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 12 op/s 2023-09-10T09:26:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:24 smithi098 ceph-mon[103124]: pgmap v19580: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 12 op/s 2023-09-10T09:26:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36415.866734 secs 2023-09-10T09:26:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36416.219547 secs 2023-09-10T09:26:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36415.866734 secs 2023-09-10T09:26:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36416.219547 secs 2023-09-10T09:26:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36415.866734 secs 2023-09-10T09:26:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36416.219547 secs 2023-09-10T09:26:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:26 smithi170 ceph-mon[102832]: pgmap v19581: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 12 op/s 2023-09-10T09:26:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:26 smithi082 ceph-mon[98694]: pgmap v19581: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 12 op/s 2023-09-10T09:26:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:26 smithi098 ceph-mon[103124]: pgmap v19581: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 12 op/s 2023-09-10T09:26:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:28 smithi082 ceph-mon[98694]: pgmap v19582: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 266 KiB/s wr, 17 op/s 2023-09-10T09:26:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36419.618351 secs 2023-09-10T09:26:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:28 smithi098 ceph-mon[103124]: pgmap v19582: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 266 KiB/s wr, 17 op/s 2023-09-10T09:26:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36419.618351 secs 2023-09-10T09:26:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:28 smithi170 ceph-mon[102832]: pgmap v19582: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 266 KiB/s wr, 17 op/s 2023-09-10T09:26:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36419.618351 secs 2023-09-10T09:26:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36420.866803 secs 2023-09-10T09:26:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36421.219617 secs 2023-09-10T09:26:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:29 smithi082 ceph-mon[98694]: pgmap v19583: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T09:26:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36420.866803 secs 2023-09-10T09:26:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36421.219617 secs 2023-09-10T09:26:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:29 smithi098 ceph-mon[103124]: pgmap v19583: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T09:26:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36420.866803 secs 2023-09-10T09:26:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36421.219617 secs 2023-09-10T09:26:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:29 smithi170 ceph-mon[102832]: pgmap v19583: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T09:26:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:26:31.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:26:31.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:26:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:32 smithi098 ceph-mon[103124]: pgmap v19584: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 133 KiB/s wr, 16 op/s 2023-09-10T09:26:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:26:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:26:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:26:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:32 smithi170 ceph-mon[102832]: pgmap v19584: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 133 KiB/s wr, 16 op/s 2023-09-10T09:26:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:26:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:26:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:26:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:32 smithi082 ceph-mon[98694]: pgmap v19584: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 133 KiB/s wr, 16 op/s 2023-09-10T09:26:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:26:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:26:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:26:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:34 smithi098 ceph-mon[103124]: pgmap v19585: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 10 op/s 2023-09-10T09:26:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36425.866867 secs 2023-09-10T09:26:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:34 smithi170 ceph-mon[102832]: pgmap v19585: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 10 op/s 2023-09-10T09:26:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36425.866867 secs 2023-09-10T09:26:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:34 smithi082 ceph-mon[98694]: pgmap v19585: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 10 op/s 2023-09-10T09:26:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36425.866867 secs 2023-09-10T09:26:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36426.219684 secs 2023-09-10T09:26:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:26:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36426.219684 secs 2023-09-10T09:26:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:26:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36426.219684 secs 2023-09-10T09:26:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:26:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:36 smithi098 ceph-mon[103124]: pgmap v19586: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 10 op/s 2023-09-10T09:26:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:26:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:26:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:36 smithi170 ceph-mon[102832]: pgmap v19586: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 10 op/s 2023-09-10T09:26:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:26:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:26:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:36 smithi082 ceph-mon[98694]: pgmap v19586: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 10 op/s 2023-09-10T09:26:36.742 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:26:36.742 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:26:36.742 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:38 smithi098 ceph-mon[103124]: pgmap v19587: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 391 KiB/s wr, 16 op/s 2023-09-10T09:26:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:26:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:38 smithi170 ceph-mon[102832]: pgmap v19587: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 391 KiB/s wr, 16 op/s 2023-09-10T09:26:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:26:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:38 smithi082 ceph-mon[98694]: pgmap v19587: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 391 KiB/s wr, 16 op/s 2023-09-10T09:26:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:26:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36430.866928 secs 2023-09-10T09:26:40.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36431.219747 secs 2023-09-10T09:26:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:39 smithi170 ceph-mon[102832]: pgmap v19588: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 259 KiB/s wr, 11 op/s 2023-09-10T09:26:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36430.866928 secs 2023-09-10T09:26:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36431.219747 secs 2023-09-10T09:26:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:39 smithi082 ceph-mon[98694]: pgmap v19588: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 259 KiB/s wr, 11 op/s 2023-09-10T09:26:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36430.866928 secs 2023-09-10T09:26:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36431.219747 secs 2023-09-10T09:26:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:39 smithi098 ceph-mon[103124]: pgmap v19588: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 259 KiB/s wr, 11 op/s 2023-09-10T09:26:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:42 smithi098 ceph-mon[103124]: pgmap v19589: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 578 KiB/s wr, 18 op/s 2023-09-10T09:26:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:42 smithi170 ceph-mon[102832]: pgmap v19589: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 578 KiB/s wr, 18 op/s 2023-09-10T09:26:42.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:42 smithi082 ceph-mon[98694]: pgmap v19589: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 578 KiB/s wr, 18 op/s 2023-09-10T09:26:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:44 smithi098 ceph-mon[103124]: pgmap v19590: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 578 KiB/s wr, 13 op/s 2023-09-10T09:26:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:26:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36435.867015 secs 2023-09-10T09:26:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:44 smithi170 ceph-mon[102832]: pgmap v19590: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 578 KiB/s wr, 13 op/s 2023-09-10T09:26:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:26:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36435.867015 secs 2023-09-10T09:26:44.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:44 smithi082 ceph-mon[98694]: pgmap v19590: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 578 KiB/s wr, 13 op/s 2023-09-10T09:26:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:26:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36435.867015 secs 2023-09-10T09:26:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36436.219814 secs 2023-09-10T09:26:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36436.219814 secs 2023-09-10T09:26:45.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36436.219814 secs 2023-09-10T09:26:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:46 smithi098 ceph-mon[103124]: pgmap v19591: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 578 KiB/s wr, 13 op/s 2023-09-10T09:26:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:26:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:46 smithi170 ceph-mon[102832]: pgmap v19591: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 578 KiB/s wr, 13 op/s 2023-09-10T09:26:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:26:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:46 smithi082 ceph-mon[98694]: pgmap v19591: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 578 KiB/s wr, 13 op/s 2023-09-10T09:26:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:26:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:48 smithi098 ceph-mon[103124]: pgmap v19592: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 603 KiB/s wr, 19 op/s 2023-09-10T09:26:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36439.618614 secs 2023-09-10T09:26:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:48 smithi170 ceph-mon[102832]: pgmap v19592: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 603 KiB/s wr, 19 op/s 2023-09-10T09:26:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36439.618614 secs 2023-09-10T09:26:48.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:48 smithi082 ceph-mon[98694]: pgmap v19592: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 603 KiB/s wr, 19 op/s 2023-09-10T09:26:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36439.618614 secs 2023-09-10T09:26:50.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36440.867069 secs 2023-09-10T09:26:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36441.219879 secs 2023-09-10T09:26:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:49 smithi170 ceph-mon[102832]: pgmap v19593: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 345 KiB/s wr, 13 op/s 2023-09-10T09:26:50.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:26:50.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36440.867069 secs 2023-09-10T09:26:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36441.219879 secs 2023-09-10T09:26:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:49 smithi082 ceph-mon[98694]: pgmap v19593: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 345 KiB/s wr, 13 op/s 2023-09-10T09:26:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:26:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36440.867069 secs 2023-09-10T09:26:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36441.219879 secs 2023-09-10T09:26:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:49 smithi098 ceph-mon[103124]: pgmap v19593: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 345 KiB/s wr, 13 op/s 2023-09-10T09:26:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:26:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:26:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:26:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:26:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:26:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:51.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:26:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:26:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:52 smithi098 ceph-mon[103124]: pgmap v19594: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 20 op/s 2023-09-10T09:26:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:52 smithi170 ceph-mon[102832]: pgmap v19594: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 20 op/s 2023-09-10T09:26:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:52 smithi082 ceph-mon[98694]: pgmap v19594: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 20 op/s 2023-09-10T09:26:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36444.618675 secs 2023-09-10T09:26:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36444.618675 secs 2023-09-10T09:26:53.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36444.618675 secs 2023-09-10T09:26:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:54 smithi098 ceph-mon[103124]: pgmap v19595: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 747 KiB/s wr, 13 op/s 2023-09-10T09:26:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36445.867146 secs 2023-09-10T09:26:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:54 smithi170 ceph-mon[102832]: pgmap v19595: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 747 KiB/s wr, 13 op/s 2023-09-10T09:26:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36445.867146 secs 2023-09-10T09:26:54.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:54 smithi082 ceph-mon[98694]: pgmap v19595: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 747 KiB/s wr, 13 op/s 2023-09-10T09:26:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36445.867146 secs 2023-09-10T09:26:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36446.219945 secs 2023-09-10T09:26:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36446.219945 secs 2023-09-10T09:26:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36446.219945 secs 2023-09-10T09:26:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:56 smithi098 ceph-mon[103124]: pgmap v19596: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 747 KiB/s wr, 13 op/s 2023-09-10T09:26:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:26:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:56 smithi170 ceph-mon[102832]: pgmap v19596: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 747 KiB/s wr, 13 op/s 2023-09-10T09:26:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:26:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:56 smithi082 ceph-mon[98694]: pgmap v19596: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 747 KiB/s wr, 13 op/s 2023-09-10T09:26:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:26:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:58 smithi098 ceph-mon[103124]: pgmap v19597: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 799 KiB/s wr, 20 op/s 2023-09-10T09:26:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:26:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:58 smithi170 ceph-mon[102832]: pgmap v19597: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 799 KiB/s wr, 20 op/s 2023-09-10T09:26:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:26:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:26:58.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:58 smithi082 ceph-mon[98694]: pgmap v19597: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 799 KiB/s wr, 20 op/s 2023-09-10T09:26:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:26:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36450.867229 secs 2023-09-10T09:27:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36451.220008 secs 2023-09-10T09:27:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:59 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:27:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:26:59 smithi082 ceph-mon[98694]: pgmap v19598: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 774 KiB/s wr, 14 op/s 2023-09-10T09:27:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36450.867229 secs 2023-09-10T09:27:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36451.220008 secs 2023-09-10T09:27:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:59 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:27:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:26:59 smithi098 ceph-mon[103124]: pgmap v19598: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 774 KiB/s wr, 14 op/s 2023-09-10T09:27:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36450.867229 secs 2023-09-10T09:27:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36451.220008 secs 2023-09-10T09:27:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:59 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:27:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:26:59 smithi170 ceph-mon[102832]: pgmap v19598: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 774 KiB/s wr, 14 op/s 2023-09-10T09:27:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:00 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:27:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:27:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:00 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:27:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:27:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:00 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:27:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:27:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:01 smithi082 ceph-mon[98694]: pgmap v19599: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 23 op/s 2023-09-10T09:27:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:01 smithi098 ceph-mon[103124]: pgmap v19599: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 23 op/s 2023-09-10T09:27:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:01 smithi170 ceph-mon[102832]: pgmap v19599: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 23 op/s 2023-09-10T09:27:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36454.618776 secs 2023-09-10T09:27:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:03 smithi082 ceph-mon[98694]: osdmap e1293: 12 total, 12 up, 12 in 2023-09-10T09:27:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:03 smithi082 ceph-mon[98694]: pgmap v19601: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 541 KiB/s wr, 18 op/s 2023-09-10T09:27:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36454.618776 secs 2023-09-10T09:27:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:03 smithi098 ceph-mon[103124]: osdmap e1293: 12 total, 12 up, 12 in 2023-09-10T09:27:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:03 smithi098 ceph-mon[103124]: pgmap v19601: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 541 KiB/s wr, 18 op/s 2023-09-10T09:27:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36454.618776 secs 2023-09-10T09:27:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:03 smithi170 ceph-mon[102832]: osdmap e1293: 12 total, 12 up, 12 in 2023-09-10T09:27:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:03 smithi170 ceph-mon[102832]: pgmap v19601: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 541 KiB/s wr, 18 op/s 2023-09-10T09:27:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36455.867288 secs 2023-09-10T09:27:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:27:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36456.220071 secs 2023-09-10T09:27:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36455.867288 secs 2023-09-10T09:27:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:27:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36456.220071 secs 2023-09-10T09:27:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36455.867288 secs 2023-09-10T09:27:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:27:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36456.220071 secs 2023-09-10T09:27:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:27:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:05 smithi082 ceph-mon[98694]: pgmap v19602: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 541 KiB/s wr, 18 op/s 2023-09-10T09:27:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:27:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:05 smithi098 ceph-mon[103124]: pgmap v19602: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 541 KiB/s wr, 18 op/s 2023-09-10T09:27:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:27:06.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:06.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:05 smithi170 ceph-mon[102832]: pgmap v19602: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 541 KiB/s wr, 18 op/s 2023-09-10T09:27:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:08 smithi098 ceph-mon[103124]: pgmap v19603: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 733 KiB/s wr, 32 op/s 2023-09-10T09:27:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36459.618842 secs 2023-09-10T09:27:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:08 smithi170 ceph-mon[102832]: pgmap v19603: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 733 KiB/s wr, 32 op/s 2023-09-10T09:27:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36459.618842 secs 2023-09-10T09:27:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:08 smithi082 ceph-mon[98694]: pgmap v19603: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 733 KiB/s wr, 32 op/s 2023-09-10T09:27:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36459.618842 secs 2023-09-10T09:27:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36460.867349 secs 2023-09-10T09:27:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:09 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:27:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:09 smithi098 ceph-mon[103124]: osdmap e1294: 12 total, 12 up, 12 in 2023-09-10T09:27:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36460.867349 secs 2023-09-10T09:27:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:09 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:27:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:09 smithi170 ceph-mon[102832]: osdmap e1294: 12 total, 12 up, 12 in 2023-09-10T09:27:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36460.867349 secs 2023-09-10T09:27:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:09 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:27:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:09 smithi082 ceph-mon[98694]: osdmap e1294: 12 total, 12 up, 12 in 2023-09-10T09:27:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36461.220137 secs 2023-09-10T09:27:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:10 smithi098 ceph-mon[103124]: pgmap v19605: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 320 KiB/s wr, 28 op/s 2023-09-10T09:27:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36461.220137 secs 2023-09-10T09:27:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:10 smithi170 ceph-mon[102832]: pgmap v19605: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 320 KiB/s wr, 28 op/s 2023-09-10T09:27:10.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36461.220137 secs 2023-09-10T09:27:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:10 smithi082 ceph-mon[98694]: pgmap v19605: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 320 KiB/s wr, 28 op/s 2023-09-10T09:27:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:12 smithi098 ceph-mon[103124]: pgmap v19606: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 504 KiB/s wr, 36 op/s 2023-09-10T09:27:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:12 smithi170 ceph-mon[102832]: pgmap v19606: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 504 KiB/s wr, 36 op/s 2023-09-10T09:27:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:12 smithi082 ceph-mon[98694]: pgmap v19606: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 504 KiB/s wr, 36 op/s 2023-09-10T09:27:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36464.618949 secs 2023-09-10T09:27:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36464.618949 secs 2023-09-10T09:27:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36464.618949 secs 2023-09-10T09:27:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:14 smithi098 ceph-mon[103124]: pgmap v19607: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 415 KiB/s wr, 30 op/s 2023-09-10T09:27:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:14 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:27:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:14 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36465.867408 secs 2023-09-10T09:27:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:14 smithi170 ceph-mon[102832]: pgmap v19607: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 415 KiB/s wr, 30 op/s 2023-09-10T09:27:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:14 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:27:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:14 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36465.867408 secs 2023-09-10T09:27:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:14 smithi082 ceph-mon[98694]: pgmap v19607: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 415 KiB/s wr, 30 op/s 2023-09-10T09:27:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:14 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:27:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:14 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36465.867408 secs 2023-09-10T09:27:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36466.220202 secs 2023-09-10T09:27:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36466.220202 secs 2023-09-10T09:27:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36466.220202 secs 2023-09-10T09:27:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:16 smithi098 ceph-mon[103124]: pgmap v19608: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 415 KiB/s wr, 30 op/s 2023-09-10T09:27:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:27:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:16 smithi170 ceph-mon[102832]: pgmap v19608: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 415 KiB/s wr, 30 op/s 2023-09-10T09:27:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:27:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:16 smithi082 ceph-mon[98694]: pgmap v19608: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 415 KiB/s wr, 30 op/s 2023-09-10T09:27:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:27:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:18 smithi098 ceph-mon[103124]: pgmap v19609: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 13 op/s 2023-09-10T09:27:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36469.619013 secs 2023-09-10T09:27:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:18 smithi170 ceph-mon[102832]: pgmap v19609: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 13 op/s 2023-09-10T09:27:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36469.619013 secs 2023-09-10T09:27:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:18 smithi082 ceph-mon[98694]: pgmap v19609: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 13 op/s 2023-09-10T09:27:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36469.619013 secs 2023-09-10T09:27:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36470.867512 secs 2023-09-10T09:27:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36471.220268 secs 2023-09-10T09:27:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:19 smithi082 ceph-mon[98694]: pgmap v19610: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 13 op/s 2023-09-10T09:27:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36470.867512 secs 2023-09-10T09:27:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36471.220268 secs 2023-09-10T09:27:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:19 smithi098 ceph-mon[103124]: pgmap v19610: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 13 op/s 2023-09-10T09:27:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36470.867512 secs 2023-09-10T09:27:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36471.220268 secs 2023-09-10T09:27:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:19 smithi170 ceph-mon[102832]: pgmap v19610: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 187 KiB/s wr, 13 op/s 2023-09-10T09:27:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:22 smithi098 ceph-mon[103124]: pgmap v19611: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 161 KiB/s wr, 16 op/s 2023-09-10T09:27:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:22 smithi170 ceph-mon[102832]: pgmap v19611: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 161 KiB/s wr, 16 op/s 2023-09-10T09:27:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:22 smithi082 ceph-mon[98694]: pgmap v19611: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 161 KiB/s wr, 16 op/s 2023-09-10T09:27:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36474.619083 secs 2023-09-10T09:27:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36474.619083 secs 2023-09-10T09:27:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36474.619083 secs 2023-09-10T09:27:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:24 smithi098 ceph-mon[103124]: pgmap v19612: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 10 op/s 2023-09-10T09:27:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36475.867568 secs 2023-09-10T09:27:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:24 smithi170 ceph-mon[102832]: pgmap v19612: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 10 op/s 2023-09-10T09:27:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36475.867568 secs 2023-09-10T09:27:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:24 smithi082 ceph-mon[98694]: pgmap v19612: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 10 op/s 2023-09-10T09:27:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36475.867568 secs 2023-09-10T09:27:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36476.220335 secs 2023-09-10T09:27:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36476.220335 secs 2023-09-10T09:27:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36476.220335 secs 2023-09-10T09:27:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:26 smithi098 ceph-mon[103124]: pgmap v19613: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 10 op/s 2023-09-10T09:27:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:26 smithi170 ceph-mon[102832]: pgmap v19613: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 10 op/s 2023-09-10T09:27:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:26 smithi082 ceph-mon[98694]: pgmap v19613: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 10 op/s 2023-09-10T09:27:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:28 smithi098 ceph-mon[103124]: pgmap v19614: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 160 KiB/s wr, 15 op/s 2023-09-10T09:27:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36479.619191 secs 2023-09-10T09:27:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:28 smithi170 ceph-mon[102832]: pgmap v19614: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 160 KiB/s wr, 15 op/s 2023-09-10T09:27:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36479.619191 secs 2023-09-10T09:27:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:28 smithi082 ceph-mon[98694]: pgmap v19614: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 160 KiB/s wr, 15 op/s 2023-09-10T09:27:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36479.619191 secs 2023-09-10T09:27:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36480.867672 secs 2023-09-10T09:27:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36481.220399 secs 2023-09-10T09:27:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:29 smithi082 ceph-mon[98694]: pgmap v19615: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 10 op/s 2023-09-10T09:27:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36480.867672 secs 2023-09-10T09:27:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36481.220399 secs 2023-09-10T09:27:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:29 smithi098 ceph-mon[103124]: pgmap v19615: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 10 op/s 2023-09-10T09:27:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36480.867672 secs 2023-09-10T09:27:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36481.220399 secs 2023-09-10T09:27:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:29 smithi170 ceph-mon[102832]: pgmap v19615: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 132 KiB/s wr, 10 op/s 2023-09-10T09:27:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:32 smithi098 ceph-mon[103124]: pgmap v19616: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 261 KiB/s wr, 16 op/s 2023-09-10T09:27:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:27:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:32 smithi170 ceph-mon[102832]: pgmap v19616: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 261 KiB/s wr, 16 op/s 2023-09-10T09:27:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:27:32.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:32 smithi082 ceph-mon[98694]: pgmap v19616: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 261 KiB/s wr, 16 op/s 2023-09-10T09:27:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:27:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:34 smithi098 ceph-mon[103124]: pgmap v19617: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 260 KiB/s wr, 10 op/s 2023-09-10T09:27:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:27:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36485.867729 secs 2023-09-10T09:27:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:34 smithi170 ceph-mon[102832]: pgmap v19617: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 260 KiB/s wr, 10 op/s 2023-09-10T09:27:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:27:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36485.867729 secs 2023-09-10T09:27:34.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:34 smithi082 ceph-mon[98694]: pgmap v19617: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 260 KiB/s wr, 10 op/s 2023-09-10T09:27:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:27:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36485.867729 secs 2023-09-10T09:27:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36486.220467 secs 2023-09-10T09:27:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:27:35.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:35.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36486.220467 secs 2023-09-10T09:27:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:35.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:27:35.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:35.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:35.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36486.220467 secs 2023-09-10T09:27:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:27:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:36 smithi098 ceph-mon[103124]: pgmap v19618: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 260 KiB/s wr, 10 op/s 2023-09-10T09:27:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:36.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:36.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:27:36.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:36.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:36.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:27:36.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:27:36.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:27:36.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:27:36.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:36 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T09:27:36.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:36 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T09:27:36.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:27:36.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:27:36.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:36.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:36.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:36.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:27:36.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:27:36.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:27:36.762 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:27:36.762 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:27:36.763 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:27:36.763 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:36 smithi170 ceph-mon[102832]: pgmap v19618: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 260 KiB/s wr, 10 op/s 2023-09-10T09:27:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:27:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:36.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:27:36.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:27:36.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:27:36.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:27:36.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:36 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T09:27:36.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:36 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T09:27:36.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:27:36.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:27:36.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:36.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:36.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:36.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:27:36.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:27:36.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:27:36.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:27:36.932 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:27:36.932 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:27:36.932 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:36 smithi082 ceph-mon[98694]: pgmap v19618: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 260 KiB/s wr, 10 op/s 2023-09-10T09:27:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:27:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:36.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:36.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:27:36.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:27:36.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:27:36.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:27:36.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:36 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T09:27:36.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:36 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T09:27:36.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:27:36.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:27:36.962 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:36.962 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:36.963 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:36.963 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:27:36.963 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:27:36.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:27:36.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:27:36.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:27:36.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:27:36.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:27:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:37 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T09:27:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:37 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T09:27:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:37 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T09:27:37.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:37 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T09:27:37.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:37 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T09:27:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:37 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T09:27:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:38 smithi098 ceph-mon[103124]: pgmap v19619: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 519 KiB/s wr, 16 op/s 2023-09-10T09:27:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:38 smithi170 ceph-mon[102832]: pgmap v19619: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 519 KiB/s wr, 16 op/s 2023-09-10T09:27:38.956 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:38 smithi082 ceph-mon[98694]: pgmap v19619: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 519 KiB/s wr, 16 op/s 2023-09-10T09:27:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36490.867790 secs 2023-09-10T09:27:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36491.220491 secs 2023-09-10T09:27:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:39 smithi082 ceph-mon[98694]: pgmap v19620: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 388 KiB/s wr, 11 op/s 2023-09-10T09:27:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36490.867790 secs 2023-09-10T09:27:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36491.220491 secs 2023-09-10T09:27:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:39 smithi098 ceph-mon[103124]: pgmap v19620: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 388 KiB/s wr, 11 op/s 2023-09-10T09:27:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36490.867790 secs 2023-09-10T09:27:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36491.220491 secs 2023-09-10T09:27:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:39 smithi170 ceph-mon[102832]: pgmap v19620: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 388 KiB/s wr, 11 op/s 2023-09-10T09:27:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:42 smithi098 ceph-mon[103124]: pgmap v19621: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 543 KiB/s wr, 18 op/s 2023-09-10T09:27:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:42 smithi170 ceph-mon[102832]: pgmap v19621: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 543 KiB/s wr, 18 op/s 2023-09-10T09:27:42.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:42 smithi082 ceph-mon[98694]: pgmap v19621: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 543 KiB/s wr, 18 op/s 2023-09-10T09:27:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36494.619381 secs 2023-09-10T09:27:43.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36494.619381 secs 2023-09-10T09:27:43.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36494.619381 secs 2023-09-10T09:27:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:44 smithi098 ceph-mon[103124]: pgmap v19622: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 415 KiB/s wr, 12 op/s 2023-09-10T09:27:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:27:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:27:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36495.867848 secs 2023-09-10T09:27:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:44 smithi170 ceph-mon[102832]: pgmap v19622: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 415 KiB/s wr, 12 op/s 2023-09-10T09:27:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:27:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:27:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36495.867848 secs 2023-09-10T09:27:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:44 smithi082 ceph-mon[98694]: pgmap v19622: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 415 KiB/s wr, 12 op/s 2023-09-10T09:27:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:27:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:27:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36495.867848 secs 2023-09-10T09:27:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36496.220562 secs 2023-09-10T09:27:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36496.220562 secs 2023-09-10T09:27:45.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36496.220562 secs 2023-09-10T09:27:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:46 smithi098 ceph-mon[103124]: pgmap v19623: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 415 KiB/s wr, 12 op/s 2023-09-10T09:27:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:27:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:46 smithi170 ceph-mon[102832]: pgmap v19623: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 415 KiB/s wr, 12 op/s 2023-09-10T09:27:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:27:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:46 smithi082 ceph-mon[98694]: pgmap v19623: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 415 KiB/s wr, 12 op/s 2023-09-10T09:27:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:27:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:48 smithi098 ceph-mon[103124]: pgmap v19624: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 849 KiB/s wr, 39 op/s 2023-09-10T09:27:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36499.619441 secs 2023-09-10T09:27:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:48 smithi170 ceph-mon[102832]: pgmap v19624: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 849 KiB/s wr, 39 op/s 2023-09-10T09:27:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36499.619441 secs 2023-09-10T09:27:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:48 smithi082 ceph-mon[98694]: pgmap v19624: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 849 KiB/s wr, 39 op/s 2023-09-10T09:27:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36499.619441 secs 2023-09-10T09:27:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36500.867907 secs 2023-09-10T09:27:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36501.220626 secs 2023-09-10T09:27:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:49 smithi082 ceph-mon[98694]: pgmap v19625: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 589 KiB/s wr, 33 op/s 2023-09-10T09:27:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:27:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36500.867907 secs 2023-09-10T09:27:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36501.220626 secs 2023-09-10T09:27:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:49 smithi098 ceph-mon[103124]: pgmap v19625: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 589 KiB/s wr, 33 op/s 2023-09-10T09:27:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:27:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36500.867907 secs 2023-09-10T09:27:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36501.220626 secs 2023-09-10T09:27:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:49 smithi170 ceph-mon[102832]: pgmap v19625: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 589 KiB/s wr, 33 op/s 2023-09-10T09:27:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:27:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:52 smithi082 ceph-mon[98694]: pgmap v19626: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 41 op/s 2023-09-10T09:27:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:27:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:52 smithi098 ceph-mon[103124]: pgmap v19626: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 41 op/s 2023-09-10T09:27:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:27:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:52 smithi170 ceph-mon[102832]: pgmap v19626: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 41 op/s 2023-09-10T09:27:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:27:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36504.619503 secs 2023-09-10T09:27:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36504.619503 secs 2023-09-10T09:27:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36504.619503 secs 2023-09-10T09:27:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:54 smithi082 ceph-mon[98694]: pgmap v19627: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 896 KiB/s wr, 34 op/s 2023-09-10T09:27:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36505.867966 secs 2023-09-10T09:27:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:54 smithi098 ceph-mon[103124]: pgmap v19627: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 896 KiB/s wr, 34 op/s 2023-09-10T09:27:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36505.867966 secs 2023-09-10T09:27:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:54 smithi170 ceph-mon[102832]: pgmap v19627: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 896 KiB/s wr, 34 op/s 2023-09-10T09:27:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36505.867966 secs 2023-09-10T09:27:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36506.220693 secs 2023-09-10T09:27:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36506.220693 secs 2023-09-10T09:27:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36506.220693 secs 2023-09-10T09:27:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:56 smithi082 ceph-mon[98694]: pgmap v19628: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 896 KiB/s wr, 34 op/s 2023-09-10T09:27:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:56 smithi098 ceph-mon[103124]: pgmap v19628: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 896 KiB/s wr, 34 op/s 2023-09-10T09:27:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:56 smithi170 ceph-mon[102832]: pgmap v19628: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 896 KiB/s wr, 34 op/s 2023-09-10T09:27:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:58 smithi082 ceph-mon[98694]: pgmap v19629: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 42 op/s 2023-09-10T09:27:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:27:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36509.619565 secs 2023-09-10T09:27:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:58 smithi098 ceph-mon[103124]: pgmap v19629: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 42 op/s 2023-09-10T09:27:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:27:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36509.619565 secs 2023-09-10T09:27:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:58 smithi170 ceph-mon[102832]: pgmap v19629: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 42 op/s 2023-09-10T09:27:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:27:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:27:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36509.619565 secs 2023-09-10T09:28:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36510.868003 secs 2023-09-10T09:28:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36511.220757 secs 2023-09-10T09:28:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:27:59 smithi082 ceph-mon[98694]: pgmap v19630: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 602 KiB/s wr, 15 op/s 2023-09-10T09:28:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36510.868003 secs 2023-09-10T09:28:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36511.220757 secs 2023-09-10T09:28:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:27:59 smithi098 ceph-mon[103124]: pgmap v19630: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 602 KiB/s wr, 15 op/s 2023-09-10T09:28:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36510.868003 secs 2023-09-10T09:28:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36511.220757 secs 2023-09-10T09:28:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:27:59 smithi170 ceph-mon[102832]: pgmap v19630: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 602 KiB/s wr, 15 op/s 2023-09-10T09:28:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:28:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:28:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:28:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:01 smithi082 ceph-mon[98694]: pgmap v19631: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T09:28:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:01 smithi098 ceph-mon[103124]: pgmap v19631: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T09:28:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:01 smithi170 ceph-mon[102832]: pgmap v19631: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T09:28:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36514.619637 secs 2023-09-10T09:28:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:03 smithi098 ceph-mon[103124]: osdmap e1295: 12 total, 12 up, 12 in 2023-09-10T09:28:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:03 smithi098 ceph-mon[103124]: pgmap v19633: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 680 KiB/s wr, 16 op/s 2023-09-10T09:28:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36514.619637 secs 2023-09-10T09:28:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:03 smithi170 ceph-mon[102832]: osdmap e1295: 12 total, 12 up, 12 in 2023-09-10T09:28:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:03 smithi170 ceph-mon[102832]: pgmap v19633: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 680 KiB/s wr, 16 op/s 2023-09-10T09:28:04.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36514.619637 secs 2023-09-10T09:28:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:03 smithi082 ceph-mon[98694]: osdmap e1295: 12 total, 12 up, 12 in 2023-09-10T09:28:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:03 smithi082 ceph-mon[98694]: pgmap v19633: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 680 KiB/s wr, 16 op/s 2023-09-10T09:28:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36515.868086 secs 2023-09-10T09:28:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36516.220821 secs 2023-09-10T09:28:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:28:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36515.868086 secs 2023-09-10T09:28:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36516.220821 secs 2023-09-10T09:28:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:28:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36515.868086 secs 2023-09-10T09:28:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36516.220821 secs 2023-09-10T09:28:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:28:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:28:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:06.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:05 smithi098 ceph-mon[103124]: pgmap v19634: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 680 KiB/s wr, 16 op/s 2023-09-10T09:28:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:28:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:05 smithi170 ceph-mon[102832]: pgmap v19634: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 680 KiB/s wr, 16 op/s 2023-09-10T09:28:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:28:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:05 smithi082 ceph-mon[98694]: pgmap v19634: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 680 KiB/s wr, 16 op/s 2023-09-10T09:28:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:08 smithi098 ceph-mon[103124]: pgmap v19635: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 909 KiB/s wr, 33 op/s 2023-09-10T09:28:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36519.619720 secs 2023-09-10T09:28:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:08 smithi170 ceph-mon[102832]: pgmap v19635: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 909 KiB/s wr, 33 op/s 2023-09-10T09:28:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36519.619720 secs 2023-09-10T09:28:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:08 smithi082 ceph-mon[98694]: pgmap v19635: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 909 KiB/s wr, 33 op/s 2023-09-10T09:28:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36519.619720 secs 2023-09-10T09:28:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36520.868144 secs 2023-09-10T09:28:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:09 smithi082 ceph-mon[98694]: osdmap e1296: 12 total, 12 up, 12 in 2023-09-10T09:28:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36521.220886 secs 2023-09-10T09:28:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:09 smithi082 ceph-mon[98694]: pgmap v19637: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 495 KiB/s wr, 32 op/s 2023-09-10T09:28:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36520.868144 secs 2023-09-10T09:28:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:09 smithi098 ceph-mon[103124]: osdmap e1296: 12 total, 12 up, 12 in 2023-09-10T09:28:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36521.220886 secs 2023-09-10T09:28:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:09 smithi098 ceph-mon[103124]: pgmap v19637: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 495 KiB/s wr, 32 op/s 2023-09-10T09:28:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36520.868144 secs 2023-09-10T09:28:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:09 smithi170 ceph-mon[102832]: osdmap e1296: 12 total, 12 up, 12 in 2023-09-10T09:28:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36521.220886 secs 2023-09-10T09:28:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:09 smithi170 ceph-mon[102832]: pgmap v19637: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 495 KiB/s wr, 32 op/s 2023-09-10T09:28:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:28:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:12 smithi082 ceph-mon[98694]: pgmap v19638: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 676 KiB/s wr, 40 op/s 2023-09-10T09:28:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:28:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:12 smithi098 ceph-mon[103124]: pgmap v19638: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 676 KiB/s wr, 40 op/s 2023-09-10T09:28:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:28:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:12 smithi170 ceph-mon[102832]: pgmap v19638: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 676 KiB/s wr, 40 op/s 2023-09-10T09:28:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36524.619776 secs 2023-09-10T09:28:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36524.619776 secs 2023-09-10T09:28:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36524.619776 secs 2023-09-10T09:28:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:14 smithi082 ceph-mon[98694]: pgmap v19639: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 556 KiB/s wr, 33 op/s 2023-09-10T09:28:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:14 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:28:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:14 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36525.868203 secs 2023-09-10T09:28:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:14 smithi098 ceph-mon[103124]: pgmap v19639: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 556 KiB/s wr, 33 op/s 2023-09-10T09:28:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:14 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:28:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:14 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36525.868203 secs 2023-09-10T09:28:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:14 smithi170 ceph-mon[102832]: pgmap v19639: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 556 KiB/s wr, 33 op/s 2023-09-10T09:28:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:14 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:28:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:14 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36525.868203 secs 2023-09-10T09:28:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36526.220959 secs 2023-09-10T09:28:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36526.220959 secs 2023-09-10T09:28:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36526.220959 secs 2023-09-10T09:28:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:16 smithi082 ceph-mon[98694]: pgmap v19640: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 556 KiB/s wr, 33 op/s 2023-09-10T09:28:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:16 smithi098 ceph-mon[103124]: pgmap v19640: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 556 KiB/s wr, 33 op/s 2023-09-10T09:28:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:16 smithi170 ceph-mon[102832]: pgmap v19640: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 556 KiB/s wr, 33 op/s 2023-09-10T09:28:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:18 smithi082 ceph-mon[98694]: pgmap v19641: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 14 op/s 2023-09-10T09:28:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36529.619837 secs 2023-09-10T09:28:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:18 smithi098 ceph-mon[103124]: pgmap v19641: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 14 op/s 2023-09-10T09:28:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36529.619837 secs 2023-09-10T09:28:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:18 smithi170 ceph-mon[102832]: pgmap v19641: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 160 KiB/s wr, 14 op/s 2023-09-10T09:28:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36529.619837 secs 2023-09-10T09:28:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36530.868286 secs 2023-09-10T09:28:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36531.221029 secs 2023-09-10T09:28:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:19 smithi082 ceph-mon[98694]: pgmap v19642: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 157 KiB/s wr, 14 op/s 2023-09-10T09:28:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36530.868286 secs 2023-09-10T09:28:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36531.221029 secs 2023-09-10T09:28:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:19 smithi098 ceph-mon[103124]: pgmap v19642: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 157 KiB/s wr, 14 op/s 2023-09-10T09:28:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36530.868286 secs 2023-09-10T09:28:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36531.221029 secs 2023-09-10T09:28:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:19 smithi170 ceph-mon[102832]: pgmap v19642: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 157 KiB/s wr, 14 op/s 2023-09-10T09:28:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:22 smithi098 ceph-mon[103124]: pgmap v19643: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 134 KiB/s wr, 18 op/s 2023-09-10T09:28:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:22 smithi170 ceph-mon[102832]: pgmap v19643: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 134 KiB/s wr, 18 op/s 2023-09-10T09:28:22.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:22 smithi082 ceph-mon[98694]: pgmap v19643: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 134 KiB/s wr, 18 op/s 2023-09-10T09:28:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36534.619907 secs 2023-09-10T09:28:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36534.619907 secs 2023-09-10T09:28:23.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36534.619907 secs 2023-09-10T09:28:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:24 smithi098 ceph-mon[103124]: pgmap v19644: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-09-10T09:28:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36535.868344 secs 2023-09-10T09:28:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:24 smithi170 ceph-mon[102832]: pgmap v19644: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-09-10T09:28:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36535.868344 secs 2023-09-10T09:28:24.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:24 smithi082 ceph-mon[98694]: pgmap v19644: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-09-10T09:28:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36535.868344 secs 2023-09-10T09:28:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36536.221094 secs 2023-09-10T09:28:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36536.221094 secs 2023-09-10T09:28:25.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36536.221094 secs 2023-09-10T09:28:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:26 smithi098 ceph-mon[103124]: pgmap v19645: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-09-10T09:28:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:26 smithi170 ceph-mon[102832]: pgmap v19645: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-09-10T09:28:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:26 smithi082 ceph-mon[98694]: pgmap v19645: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-09-10T09:28:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:28 smithi098 ceph-mon[103124]: pgmap v19646: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 134 KiB/s wr, 17 op/s 2023-09-10T09:28:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36539.619967 secs 2023-09-10T09:28:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:28 smithi170 ceph-mon[102832]: pgmap v19646: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 134 KiB/s wr, 17 op/s 2023-09-10T09:28:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36539.619967 secs 2023-09-10T09:28:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:28 smithi082 ceph-mon[98694]: pgmap v19646: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 134 KiB/s wr, 17 op/s 2023-09-10T09:28:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36539.619967 secs 2023-09-10T09:28:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36540.868404 secs 2023-09-10T09:28:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36541.221169 secs 2023-09-10T09:28:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:29 smithi082 ceph-mon[98694]: pgmap v19647: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T09:28:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36540.868404 secs 2023-09-10T09:28:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36541.221169 secs 2023-09-10T09:28:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:29 smithi098 ceph-mon[103124]: pgmap v19647: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T09:28:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36540.868404 secs 2023-09-10T09:28:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36541.221169 secs 2023-09-10T09:28:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:29 smithi170 ceph-mon[102832]: pgmap v19647: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T09:28:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:32 smithi098 ceph-mon[103124]: pgmap v19648: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 133 KiB/s wr, 16 op/s 2023-09-10T09:28:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:32 smithi170 ceph-mon[102832]: pgmap v19648: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 133 KiB/s wr, 16 op/s 2023-09-10T09:28:32.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:32 smithi082 ceph-mon[98694]: pgmap v19648: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 133 KiB/s wr, 16 op/s 2023-09-10T09:28:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:34 smithi098 ceph-mon[103124]: pgmap v19649: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 10 op/s 2023-09-10T09:28:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36545.868463 secs 2023-09-10T09:28:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:34 smithi170 ceph-mon[102832]: pgmap v19649: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 10 op/s 2023-09-10T09:28:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36545.868463 secs 2023-09-10T09:28:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:34 smithi082 ceph-mon[98694]: pgmap v19649: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 10 op/s 2023-09-10T09:28:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36545.868463 secs 2023-09-10T09:28:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36546.221234 secs 2023-09-10T09:28:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:28:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36546.221234 secs 2023-09-10T09:28:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:28:35.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36546.221234 secs 2023-09-10T09:28:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:28:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:36 smithi170 ceph-mon[102832]: pgmap v19650: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 10 op/s 2023-09-10T09:28:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:28:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:28:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:36.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:36 smithi082 ceph-mon[98694]: pgmap v19650: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 10 op/s 2023-09-10T09:28:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:28:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:28:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:36 smithi098 ceph-mon[103124]: pgmap v19650: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 10 op/s 2023-09-10T09:28:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:28:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:28:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:28:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:28:37.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:28:37.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:28:37.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:28:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:28:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:28:37.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:28:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:28:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:28:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:28:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:28:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:38 smithi170 ceph-mon[102832]: pgmap v19651: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 416 KiB/s wr, 16 op/s 2023-09-10T09:28:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:28:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:38 smithi082 ceph-mon[98694]: pgmap v19651: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 416 KiB/s wr, 16 op/s 2023-09-10T09:28:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:28:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:38 smithi098 ceph-mon[103124]: pgmap v19651: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 416 KiB/s wr, 16 op/s 2023-09-10T09:28:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:28:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36550.868501 secs 2023-09-10T09:28:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36551.221303 secs 2023-09-10T09:28:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:39 smithi082 ceph-mon[98694]: pgmap v19652: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 283 KiB/s wr, 10 op/s 2023-09-10T09:28:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36550.868501 secs 2023-09-10T09:28:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36551.221303 secs 2023-09-10T09:28:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:39 smithi098 ceph-mon[103124]: pgmap v19652: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 283 KiB/s wr, 10 op/s 2023-09-10T09:28:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36550.868501 secs 2023-09-10T09:28:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36551.221303 secs 2023-09-10T09:28:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:39 smithi170 ceph-mon[102832]: pgmap v19652: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 283 KiB/s wr, 10 op/s 2023-09-10T09:28:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:42 smithi098 ceph-mon[103124]: pgmap v19653: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 472 KiB/s wr, 17 op/s 2023-09-10T09:28:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:42 smithi170 ceph-mon[102832]: pgmap v19653: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 472 KiB/s wr, 17 op/s 2023-09-10T09:28:42.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:42 smithi082 ceph-mon[98694]: pgmap v19653: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 472 KiB/s wr, 17 op/s 2023-09-10T09:28:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:44 smithi098 ceph-mon[103124]: pgmap v19654: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 472 KiB/s wr, 12 op/s 2023-09-10T09:28:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36555.868583 secs 2023-09-10T09:28:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:44 smithi170 ceph-mon[102832]: pgmap v19654: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 472 KiB/s wr, 12 op/s 2023-09-10T09:28:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36555.868583 secs 2023-09-10T09:28:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:44 smithi082 ceph-mon[98694]: pgmap v19654: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 472 KiB/s wr, 12 op/s 2023-09-10T09:28:44.747 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36555.868583 secs 2023-09-10T09:28:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36556.221367 secs 2023-09-10T09:28:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36556.221367 secs 2023-09-10T09:28:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36556.221367 secs 2023-09-10T09:28:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:46 smithi098 ceph-mon[103124]: pgmap v19655: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 472 KiB/s wr, 12 op/s 2023-09-10T09:28:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:46 smithi170 ceph-mon[102832]: pgmap v19655: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 472 KiB/s wr, 12 op/s 2023-09-10T09:28:46.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:46 smithi082 ceph-mon[98694]: pgmap v19655: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 472 KiB/s wr, 12 op/s 2023-09-10T09:28:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:48 smithi170 ceph-mon[102832]: pgmap v19656: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 514 KiB/s wr, 19 op/s 2023-09-10T09:28:48.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:48 smithi082 ceph-mon[98694]: pgmap v19656: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 514 KiB/s wr, 19 op/s 2023-09-10T09:28:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:48 smithi098 ceph-mon[103124]: pgmap v19656: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 514 KiB/s wr, 19 op/s 2023-09-10T09:28:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36560.868645 secs 2023-09-10T09:28:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36561.221434 secs 2023-09-10T09:28:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:49 smithi082 ceph-mon[98694]: pgmap v19657: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 231 KiB/s wr, 13 op/s 2023-09-10T09:28:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:28:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36560.868645 secs 2023-09-10T09:28:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36561.221434 secs 2023-09-10T09:28:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:49 smithi098 ceph-mon[103124]: pgmap v19657: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 231 KiB/s wr, 13 op/s 2023-09-10T09:28:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:28:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36560.868645 secs 2023-09-10T09:28:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36561.221434 secs 2023-09-10T09:28:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:49 smithi170 ceph-mon[102832]: pgmap v19657: 97 pgs: 97 active+clean; 12 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 231 KiB/s wr, 13 op/s 2023-09-10T09:28:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:28:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:28:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:52 smithi082 ceph-mon[98694]: pgmap v19658: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 697 KiB/s wr, 20 op/s 2023-09-10T09:28:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:28:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:52 smithi098 ceph-mon[103124]: pgmap v19658: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 697 KiB/s wr, 20 op/s 2023-09-10T09:28:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:28:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:52 smithi170 ceph-mon[102832]: pgmap v19658: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 697 KiB/s wr, 20 op/s 2023-09-10T09:28:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:28:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:54 smithi082 ceph-mon[98694]: pgmap v19659: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 508 KiB/s wr, 14 op/s 2023-09-10T09:28:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36565.868703 secs 2023-09-10T09:28:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:28:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:54 smithi098 ceph-mon[103124]: pgmap v19659: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 508 KiB/s wr, 14 op/s 2023-09-10T09:28:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36565.868703 secs 2023-09-10T09:28:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:28:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:54 smithi170 ceph-mon[102832]: pgmap v19659: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 508 KiB/s wr, 14 op/s 2023-09-10T09:28:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36565.868703 secs 2023-09-10T09:28:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36566.221485 secs 2023-09-10T09:28:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36566.221485 secs 2023-09-10T09:28:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36566.221485 secs 2023-09-10T09:28:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:56 smithi082 ceph-mon[98694]: pgmap v19660: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 508 KiB/s wr, 14 op/s 2023-09-10T09:28:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:28:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:56 smithi098 ceph-mon[103124]: pgmap v19660: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 508 KiB/s wr, 14 op/s 2023-09-10T09:28:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:28:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:56 smithi170 ceph-mon[102832]: pgmap v19660: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 508 KiB/s wr, 14 op/s 2023-09-10T09:28:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:28:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:58 smithi082 ceph-mon[98694]: pgmap v19661: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 699 KiB/s wr, 20 op/s 2023-09-10T09:28:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:28:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:58 smithi098 ceph-mon[103124]: pgmap v19661: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 699 KiB/s wr, 20 op/s 2023-09-10T09:28:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:28:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:28:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:58 smithi170 ceph-mon[102832]: pgmap v19661: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 699 KiB/s wr, 20 op/s 2023-09-10T09:28:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:28:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36570.868763 secs 2023-09-10T09:29:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36571.221547 secs 2023-09-10T09:29:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:28:59 smithi098 ceph-mon[103124]: pgmap v19662: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 658 KiB/s wr, 13 op/s 2023-09-10T09:29:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36570.868763 secs 2023-09-10T09:29:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36571.221547 secs 2023-09-10T09:29:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:28:59 smithi170 ceph-mon[102832]: pgmap v19662: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 658 KiB/s wr, 13 op/s 2023-09-10T09:29:00.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36570.868763 secs 2023-09-10T09:29:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36571.221547 secs 2023-09-10T09:29:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:28:59 smithi082 ceph-mon[98694]: pgmap v19662: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 658 KiB/s wr, 13 op/s 2023-09-10T09:29:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:29:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:29:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:01.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:29:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:01 smithi098 ceph-mon[103124]: pgmap v19663: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1003 KiB/s wr, 20 op/s 2023-09-10T09:29:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:01 smithi170 ceph-mon[102832]: pgmap v19663: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1003 KiB/s wr, 20 op/s 2023-09-10T09:29:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:01 smithi082 ceph-mon[98694]: pgmap v19663: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1003 KiB/s wr, 20 op/s 2023-09-10T09:29:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36574.620523 secs 2023-09-10T09:29:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36574.620523 secs 2023-09-10T09:29:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36574.620523 secs 2023-09-10T09:29:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:29:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:29:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:04 smithi170 ceph-mon[102832]: pgmap v19664: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 537 KiB/s wr, 12 op/s 2023-09-10T09:29:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:29:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:29:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:04 smithi082 ceph-mon[98694]: pgmap v19664: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 537 KiB/s wr, 12 op/s 2023-09-10T09:29:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:29:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:29:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:04 smithi098 ceph-mon[103124]: pgmap v19664: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 537 KiB/s wr, 12 op/s 2023-09-10T09:29:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36575.868821 secs 2023-09-10T09:29:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36576.221616 secs 2023-09-10T09:29:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36575.868821 secs 2023-09-10T09:29:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36576.221616 secs 2023-09-10T09:29:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36575.868821 secs 2023-09-10T09:29:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36576.221616 secs 2023-09-10T09:29:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:06 smithi170 ceph-mon[102832]: pgmap v19665: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 537 KiB/s wr, 12 op/s 2023-09-10T09:29:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:06 smithi082 ceph-mon[98694]: pgmap v19665: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 537 KiB/s wr, 12 op/s 2023-09-10T09:29:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:06 smithi098 ceph-mon[103124]: pgmap v19665: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 537 KiB/s wr, 12 op/s 2023-09-10T09:29:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:08 smithi170 ceph-mon[102832]: pgmap v19666: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 732 KiB/s wr, 34 op/s 2023-09-10T09:29:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36579.620583 secs 2023-09-10T09:29:08.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:08 smithi082 ceph-mon[98694]: pgmap v19666: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 732 KiB/s wr, 34 op/s 2023-09-10T09:29:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36579.620583 secs 2023-09-10T09:29:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:08 smithi098 ceph-mon[103124]: pgmap v19666: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 732 KiB/s wr, 34 op/s 2023-09-10T09:29:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36579.620583 secs 2023-09-10T09:29:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:09 smithi170 ceph-mon[102832]: osdmap e1297: 12 total, 12 up, 12 in 2023-09-10T09:29:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36580.868881 secs 2023-09-10T09:29:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:09 smithi082 ceph-mon[98694]: osdmap e1297: 12 total, 12 up, 12 in 2023-09-10T09:29:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36580.868881 secs 2023-09-10T09:29:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:09 smithi098 ceph-mon[103124]: osdmap e1297: 12 total, 12 up, 12 in 2023-09-10T09:29:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36580.868881 secs 2023-09-10T09:29:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36581.221682 secs 2023-09-10T09:29:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:10 smithi170 ceph-mon[102832]: pgmap v19668: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 648 KiB/s wr, 33 op/s 2023-09-10T09:29:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:29:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36581.221682 secs 2023-09-10T09:29:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:10 smithi082 ceph-mon[98694]: pgmap v19668: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 648 KiB/s wr, 33 op/s 2023-09-10T09:29:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:29:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36581.221682 secs 2023-09-10T09:29:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:10 smithi098 ceph-mon[103124]: pgmap v19668: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 648 KiB/s wr, 33 op/s 2023-09-10T09:29:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:29:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:12 smithi170 ceph-mon[102832]: pgmap v19669: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 457 KiB/s wr, 33 op/s 2023-09-10T09:29:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:29:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:12 smithi082 ceph-mon[98694]: pgmap v19669: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 457 KiB/s wr, 33 op/s 2023-09-10T09:29:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:29:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:12 smithi098 ceph-mon[103124]: pgmap v19669: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 457 KiB/s wr, 33 op/s 2023-09-10T09:29:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:29:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36584.620643 secs 2023-09-10T09:29:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36584.620643 secs 2023-09-10T09:29:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36584.620643 secs 2023-09-10T09:29:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:14 smithi170 ceph-mon[102832]: pgmap v19670: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 457 KiB/s wr, 33 op/s 2023-09-10T09:29:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36585.868971 secs 2023-09-10T09:29:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:14 smithi170 ceph-mon[102832]: osdmap e1298: 12 total, 12 up, 12 in 2023-09-10T09:29:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:14 smithi082 ceph-mon[98694]: pgmap v19670: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 457 KiB/s wr, 33 op/s 2023-09-10T09:29:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36585.868971 secs 2023-09-10T09:29:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:14 smithi082 ceph-mon[98694]: osdmap e1298: 12 total, 12 up, 12 in 2023-09-10T09:29:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:14 smithi098 ceph-mon[103124]: pgmap v19670: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 457 KiB/s wr, 33 op/s 2023-09-10T09:29:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36585.868971 secs 2023-09-10T09:29:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:14 smithi098 ceph-mon[103124]: osdmap e1298: 12 total, 12 up, 12 in 2023-09-10T09:29:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36586.221750 secs 2023-09-10T09:29:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36586.221750 secs 2023-09-10T09:29:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36586.221750 secs 2023-09-10T09:29:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:16 smithi170 ceph-mon[102832]: pgmap v19672: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 279 KiB/s wr, 9 op/s 2023-09-10T09:29:16.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:16 smithi082 ceph-mon[98694]: pgmap v19672: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 279 KiB/s wr, 9 op/s 2023-09-10T09:29:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:16 smithi098 ceph-mon[103124]: pgmap v19672: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 279 KiB/s wr, 9 op/s 2023-09-10T09:29:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:18 smithi170 ceph-mon[102832]: pgmap v19673: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 392 KiB/s wr, 16 op/s 2023-09-10T09:29:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36589.620747 secs 2023-09-10T09:29:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:18 smithi082 ceph-mon[98694]: pgmap v19673: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 392 KiB/s wr, 16 op/s 2023-09-10T09:29:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36589.620747 secs 2023-09-10T09:29:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:18 smithi098 ceph-mon[103124]: pgmap v19673: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 392 KiB/s wr, 16 op/s 2023-09-10T09:29:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36589.620747 secs 2023-09-10T09:29:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36590.869030 secs 2023-09-10T09:29:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36591.221816 secs 2023-09-10T09:29:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:19 smithi098 ceph-mon[103124]: pgmap v19674: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 351 KiB/s wr, 14 op/s 2023-09-10T09:29:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36590.869030 secs 2023-09-10T09:29:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36591.221816 secs 2023-09-10T09:29:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:19 smithi170 ceph-mon[102832]: pgmap v19674: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 351 KiB/s wr, 14 op/s 2023-09-10T09:29:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36590.869030 secs 2023-09-10T09:29:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36591.221816 secs 2023-09-10T09:29:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:19 smithi082 ceph-mon[98694]: pgmap v19674: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 351 KiB/s wr, 14 op/s 2023-09-10T09:29:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:22 smithi170 ceph-mon[102832]: pgmap v19675: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 12 op/s 2023-09-10T09:29:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:22 smithi082 ceph-mon[98694]: pgmap v19675: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 12 op/s 2023-09-10T09:29:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:22 smithi098 ceph-mon[103124]: pgmap v19675: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 12 op/s 2023-09-10T09:29:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36594.620776 secs 2023-09-10T09:29:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36594.620776 secs 2023-09-10T09:29:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36594.620776 secs 2023-09-10T09:29:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:24 smithi170 ceph-mon[102832]: pgmap v19676: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 12 op/s 2023-09-10T09:29:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36595.869083 secs 2023-09-10T09:29:24.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:24 smithi082 ceph-mon[98694]: pgmap v19676: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 12 op/s 2023-09-10T09:29:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36595.869083 secs 2023-09-10T09:29:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:24 smithi098 ceph-mon[103124]: pgmap v19676: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 128 KiB/s wr, 12 op/s 2023-09-10T09:29:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36595.869083 secs 2023-09-10T09:29:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36596.221886 secs 2023-09-10T09:29:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36596.221886 secs 2023-09-10T09:29:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36596.221886 secs 2023-09-10T09:29:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:26 smithi170 ceph-mon[102832]: pgmap v19677: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T09:29:26.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:26 smithi082 ceph-mon[98694]: pgmap v19677: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T09:29:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:26 smithi098 ceph-mon[103124]: pgmap v19677: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 114 KiB/s wr, 11 op/s 2023-09-10T09:29:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:28 smithi170 ceph-mon[102832]: pgmap v19678: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 239 KiB/s wr, 16 op/s 2023-09-10T09:29:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36599.620839 secs 2023-09-10T09:29:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:28 smithi082 ceph-mon[98694]: pgmap v19678: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 239 KiB/s wr, 16 op/s 2023-09-10T09:29:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36599.620839 secs 2023-09-10T09:29:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:28 smithi098 ceph-mon[103124]: pgmap v19678: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 239 KiB/s wr, 16 op/s 2023-09-10T09:29:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36599.620839 secs 2023-09-10T09:29:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36600.869173 secs 2023-09-10T09:29:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36601.221952 secs 2023-09-10T09:29:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:29 smithi098 ceph-mon[103124]: pgmap v19679: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 10 op/s 2023-09-10T09:29:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36600.869173 secs 2023-09-10T09:29:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36601.221952 secs 2023-09-10T09:29:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:29 smithi170 ceph-mon[102832]: pgmap v19679: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 10 op/s 2023-09-10T09:29:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36600.869173 secs 2023-09-10T09:29:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36601.221952 secs 2023-09-10T09:29:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:29 smithi082 ceph-mon[98694]: pgmap v19679: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 10 op/s 2023-09-10T09:29:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:32 smithi170 ceph-mon[102832]: pgmap v19680: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 133 KiB/s wr, 16 op/s 2023-09-10T09:29:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:32 smithi082 ceph-mon[98694]: pgmap v19680: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 133 KiB/s wr, 16 op/s 2023-09-10T09:29:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:32 smithi098 ceph-mon[103124]: pgmap v19680: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 133 KiB/s wr, 16 op/s 2023-09-10T09:29:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:34 smithi170 ceph-mon[102832]: pgmap v19681: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T09:29:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:29:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36605.869230 secs 2023-09-10T09:29:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:34 smithi082 ceph-mon[98694]: pgmap v19681: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T09:29:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:29:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36605.869230 secs 2023-09-10T09:29:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:34 smithi098 ceph-mon[103124]: pgmap v19681: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T09:29:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:29:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36605.869230 secs 2023-09-10T09:29:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36606.222016 secs 2023-09-10T09:29:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:29:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36606.222016 secs 2023-09-10T09:29:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:29:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36606.222016 secs 2023-09-10T09:29:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:29:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:36 smithi170 ceph-mon[102832]: pgmap v19682: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 12 op/s 2023-09-10T09:29:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:29:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:29:36.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:36 smithi082 ceph-mon[98694]: pgmap v19682: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 12 op/s 2023-09-10T09:29:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:29:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:29:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:36 smithi098 ceph-mon[103124]: pgmap v19682: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 133 KiB/s wr, 12 op/s 2023-09-10T09:29:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:29:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:29:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:37.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:29:37.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:29:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:29:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:38 smithi170 ceph-mon[102832]: pgmap v19683: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 390 KiB/s wr, 17 op/s 2023-09-10T09:29:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:29:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:29:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:29:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36609.620968 secs 2023-09-10T09:29:38.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:29:38.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:29:38.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:29:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:38 smithi082 ceph-mon[98694]: pgmap v19683: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 390 KiB/s wr, 17 op/s 2023-09-10T09:29:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:29:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:29:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:29:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36609.620968 secs 2023-09-10T09:29:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:29:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:29:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:29:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:38 smithi098 ceph-mon[103124]: pgmap v19683: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 390 KiB/s wr, 17 op/s 2023-09-10T09:29:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:29:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:29:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:29:38.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36609.620968 secs 2023-09-10T09:29:38.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:29:38.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:29:38.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:29:40.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36610.869316 secs 2023-09-10T09:29:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36611.222079 secs 2023-09-10T09:29:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:39 smithi098 ceph-mon[103124]: pgmap v19684: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 258 KiB/s wr, 11 op/s 2023-09-10T09:29:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36610.869316 secs 2023-09-10T09:29:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36611.222079 secs 2023-09-10T09:29:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:39 smithi170 ceph-mon[102832]: pgmap v19684: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 258 KiB/s wr, 11 op/s 2023-09-10T09:29:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36610.869316 secs 2023-09-10T09:29:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36611.222079 secs 2023-09-10T09:29:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:39 smithi082 ceph-mon[98694]: pgmap v19684: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 258 KiB/s wr, 11 op/s 2023-09-10T09:29:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:29:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:29:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:29:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:42.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:41 smithi098 ceph-mon[103124]: pgmap v19685: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 575 KiB/s wr, 18 op/s 2023-09-10T09:29:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:41 smithi170 ceph-mon[102832]: pgmap v19685: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 575 KiB/s wr, 18 op/s 2023-09-10T09:29:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:41 smithi082 ceph-mon[98694]: pgmap v19685: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 575 KiB/s wr, 18 op/s 2023-09-10T09:29:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36614.621037 secs 2023-09-10T09:29:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36614.621037 secs 2023-09-10T09:29:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36614.621037 secs 2023-09-10T09:29:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:44 smithi170 ceph-mon[102832]: pgmap v19686: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 575 KiB/s wr, 13 op/s 2023-09-10T09:29:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:29:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:44 smithi082 ceph-mon[98694]: pgmap v19686: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 575 KiB/s wr, 13 op/s 2023-09-10T09:29:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:29:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:44 smithi098 ceph-mon[103124]: pgmap v19686: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 575 KiB/s wr, 13 op/s 2023-09-10T09:29:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:29:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36615.869374 secs 2023-09-10T09:29:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36616.222143 secs 2023-09-10T09:29:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36615.869374 secs 2023-09-10T09:29:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36616.222143 secs 2023-09-10T09:29:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36615.869374 secs 2023-09-10T09:29:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36616.222143 secs 2023-09-10T09:29:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:46 smithi170 ceph-mon[102832]: pgmap v19687: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 575 KiB/s wr, 14 op/s 2023-09-10T09:29:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:46 smithi082 ceph-mon[98694]: pgmap v19687: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 575 KiB/s wr, 14 op/s 2023-09-10T09:29:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:46 smithi098 ceph-mon[103124]: pgmap v19687: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 575 KiB/s wr, 14 op/s 2023-09-10T09:29:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:48 smithi170 ceph-mon[102832]: pgmap v19688: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 628 KiB/s wr, 19 op/s 2023-09-10T09:29:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:48 smithi082 ceph-mon[98694]: pgmap v19688: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 628 KiB/s wr, 19 op/s 2023-09-10T09:29:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:48 smithi098 ceph-mon[103124]: pgmap v19688: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 628 KiB/s wr, 19 op/s 2023-09-10T09:29:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36620.869431 secs 2023-09-10T09:29:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36621.222211 secs 2023-09-10T09:29:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:49 smithi098 ceph-mon[103124]: pgmap v19689: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 370 KiB/s wr, 14 op/s 2023-09-10T09:29:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:29:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36620.869431 secs 2023-09-10T09:29:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36621.222211 secs 2023-09-10T09:29:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:49 smithi170 ceph-mon[102832]: pgmap v19689: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 370 KiB/s wr, 14 op/s 2023-09-10T09:29:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:29:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36620.869431 secs 2023-09-10T09:29:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36621.222211 secs 2023-09-10T09:29:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:49 smithi082 ceph-mon[98694]: pgmap v19689: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 370 KiB/s wr, 14 op/s 2023-09-10T09:29:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:29:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:51 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:29:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:51 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:29:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:51 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:29:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:52 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:29:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:29:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:52 smithi082 ceph-mon[98694]: pgmap v19690: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:29:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:52 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:29:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:29:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:52 smithi098 ceph-mon[103124]: pgmap v19690: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:29:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:52 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:29:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:29:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:52 smithi170 ceph-mon[102832]: pgmap v19690: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:29:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:54 smithi170 ceph-mon[102832]: pgmap v19691: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 764 KiB/s wr, 14 op/s 2023-09-10T09:29:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36625.869510 secs 2023-09-10T09:29:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:54 smithi082 ceph-mon[98694]: pgmap v19691: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 764 KiB/s wr, 14 op/s 2023-09-10T09:29:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36625.869510 secs 2023-09-10T09:29:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:54 smithi098 ceph-mon[103124]: pgmap v19691: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 764 KiB/s wr, 14 op/s 2023-09-10T09:29:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36625.869510 secs 2023-09-10T09:29:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36626.222274 secs 2023-09-10T09:29:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36626.222274 secs 2023-09-10T09:29:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36626.222274 secs 2023-09-10T09:29:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:56 smithi170 ceph-mon[102832]: pgmap v19692: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 896 KiB/s wr, 17 op/s 2023-09-10T09:29:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:56 smithi082 ceph-mon[98694]: pgmap v19692: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 896 KiB/s wr, 17 op/s 2023-09-10T09:29:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:56 smithi098 ceph-mon[103124]: pgmap v19692: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 896 KiB/s wr, 17 op/s 2023-09-10T09:29:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:58 smithi170 ceph-mon[102832]: pgmap v19693: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 923 KiB/s wr, 20 op/s 2023-09-10T09:29:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:29:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:58 smithi082 ceph-mon[98694]: pgmap v19693: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 923 KiB/s wr, 20 op/s 2023-09-10T09:29:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:29:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:29:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:58 smithi098 ceph-mon[103124]: pgmap v19693: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 923 KiB/s wr, 20 op/s 2023-09-10T09:29:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:29:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36630.869571 secs 2023-09-10T09:30:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36631.222362 secs 2023-09-10T09:30:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:29:59 smithi098 ceph-mon[103124]: pgmap v19694: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 871 KiB/s wr, 14 op/s 2023-09-10T09:30:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36630.869571 secs 2023-09-10T09:30:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36631.222362 secs 2023-09-10T09:30:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:29:59 smithi170 ceph-mon[102832]: pgmap v19694: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 871 KiB/s wr, 14 op/s 2023-09-10T09:30:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36630.869571 secs 2023-09-10T09:30:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36631.222362 secs 2023-09-10T09:30:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:29:59 smithi082 ceph-mon[98694]: pgmap v19694: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 871 KiB/s wr, 14 op/s 2023-09-10T09:30:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:30:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:00 smithi098 ceph-mon[103124]: Health detail: HEALTH_WARN 2 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T09:30:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:00 smithi098 ceph-mon[103124]: [WRN] MDS_SLOW_REQUEST: 2 MDSs report slow requests 2023-09-10T09:30:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:00 smithi098 ceph-mon[103124]: mds.l(mds.4): 3 slow requests are blocked > 180 secs 2023-09-10T09:30:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:00 smithi098 ceph-mon[103124]: mds.h(mds.1): 3 slow requests are blocked > 180 secs 2023-09-10T09:30:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:00 smithi098 ceph-mon[103124]: [WRN] MDS_TRIM: 2 MDSs behind on trimming 2023-09-10T09:30:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:00 smithi098 ceph-mon[103124]: mds.e(mds.0): Behind on trimming (787/128) max_segments: 128, num_segments: 787 2023-09-10T09:30:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:00 smithi098 ceph-mon[103124]: mds.b(mds.0): Behind on trimming (787/128) max_segments: 128, num_segments: 787 2023-09-10T09:30:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:30:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:00 smithi170 ceph-mon[102832]: Health detail: HEALTH_WARN 2 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T09:30:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:00 smithi170 ceph-mon[102832]: [WRN] MDS_SLOW_REQUEST: 2 MDSs report slow requests 2023-09-10T09:30:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:00 smithi170 ceph-mon[102832]: mds.l(mds.4): 3 slow requests are blocked > 180 secs 2023-09-10T09:30:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:00 smithi170 ceph-mon[102832]: mds.h(mds.1): 3 slow requests are blocked > 180 secs 2023-09-10T09:30:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:00 smithi170 ceph-mon[102832]: [WRN] MDS_TRIM: 2 MDSs behind on trimming 2023-09-10T09:30:01.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:00 smithi170 ceph-mon[102832]: mds.e(mds.0): Behind on trimming (787/128) max_segments: 128, num_segments: 787 2023-09-10T09:30:01.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:00 smithi170 ceph-mon[102832]: mds.b(mds.0): Behind on trimming (787/128) max_segments: 128, num_segments: 787 2023-09-10T09:30:01.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:30:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:00 smithi082 ceph-mon[98694]: Health detail: HEALTH_WARN 2 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T09:30:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:00 smithi082 ceph-mon[98694]: [WRN] MDS_SLOW_REQUEST: 2 MDSs report slow requests 2023-09-10T09:30:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:00 smithi082 ceph-mon[98694]: mds.l(mds.4): 3 slow requests are blocked > 180 secs 2023-09-10T09:30:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:00 smithi082 ceph-mon[98694]: mds.h(mds.1): 3 slow requests are blocked > 180 secs 2023-09-10T09:30:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:00 smithi082 ceph-mon[98694]: [WRN] MDS_TRIM: 2 MDSs behind on trimming 2023-09-10T09:30:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:00 smithi082 ceph-mon[98694]: mds.e(mds.0): Behind on trimming (787/128) max_segments: 128, num_segments: 787 2023-09-10T09:30:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:00 smithi082 ceph-mon[98694]: mds.b(mds.0): Behind on trimming (787/128) max_segments: 128, num_segments: 787 2023-09-10T09:30:02.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:01 smithi098 ceph-mon[103124]: pgmap v19695: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1005 KiB/s wr, 21 op/s 2023-09-10T09:30:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:01 smithi170 ceph-mon[102832]: pgmap v19695: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1005 KiB/s wr, 21 op/s 2023-09-10T09:30:02.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:01 smithi082 ceph-mon[98694]: pgmap v19695: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1005 KiB/s wr, 21 op/s 2023-09-10T09:30:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36634.621355 secs 2023-09-10T09:30:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36634.621355 secs 2023-09-10T09:30:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36634.621355 secs 2023-09-10T09:30:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:30:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:30:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:04 smithi170 ceph-mon[102832]: pgmap v19696: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 294 KiB/s wr, 13 op/s 2023-09-10T09:30:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:30:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:30:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:04 smithi082 ceph-mon[98694]: pgmap v19696: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 294 KiB/s wr, 13 op/s 2023-09-10T09:30:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:30:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:30:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:04 smithi098 ceph-mon[103124]: pgmap v19696: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 294 KiB/s wr, 13 op/s 2023-09-10T09:30:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36635.869627 secs 2023-09-10T09:30:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36636.222427 secs 2023-09-10T09:30:05.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36635.869627 secs 2023-09-10T09:30:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36636.222427 secs 2023-09-10T09:30:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36635.869627 secs 2023-09-10T09:30:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36636.222427 secs 2023-09-10T09:30:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:30:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:06 smithi082 ceph-mon[98694]: pgmap v19697: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 419 KiB/s wr, 17 op/s 2023-09-10T09:30:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:30:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:06 smithi098 ceph-mon[103124]: pgmap v19697: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 419 KiB/s wr, 17 op/s 2023-09-10T09:30:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:30:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:06 smithi170 ceph-mon[102832]: pgmap v19697: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 419 KiB/s wr, 17 op/s 2023-09-10T09:30:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:08 smithi170 ceph-mon[102832]: pgmap v19698: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 493 KiB/s wr, 31 op/s 2023-09-10T09:30:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36639.621439 secs 2023-09-10T09:30:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:08 smithi082 ceph-mon[98694]: pgmap v19698: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 493 KiB/s wr, 31 op/s 2023-09-10T09:30:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36639.621439 secs 2023-09-10T09:30:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:08 smithi098 ceph-mon[103124]: pgmap v19698: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 493 KiB/s wr, 31 op/s 2023-09-10T09:30:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36639.621439 secs 2023-09-10T09:30:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:09 smithi170 ceph-mon[102832]: osdmap e1299: 12 total, 12 up, 12 in 2023-09-10T09:30:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36640.869686 secs 2023-09-10T09:30:09.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:09 smithi082 ceph-mon[98694]: osdmap e1299: 12 total, 12 up, 12 in 2023-09-10T09:30:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36640.869686 secs 2023-09-10T09:30:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:09 smithi098 ceph-mon[103124]: osdmap e1299: 12 total, 12 up, 12 in 2023-09-10T09:30:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36640.869686 secs 2023-09-10T09:30:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36641.222492 secs 2023-09-10T09:30:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:10 smithi170 ceph-mon[102832]: pgmap v19700: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 559 KiB/s wr, 33 op/s 2023-09-10T09:30:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36641.222492 secs 2023-09-10T09:30:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:10 smithi082 ceph-mon[98694]: pgmap v19700: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 559 KiB/s wr, 33 op/s 2023-09-10T09:30:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36641.222492 secs 2023-09-10T09:30:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:10 smithi098 ceph-mon[103124]: pgmap v19700: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 559 KiB/s wr, 33 op/s 2023-09-10T09:30:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:12 smithi170 ceph-mon[102832]: pgmap v19701: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 558 KiB/s wr, 32 op/s 2023-09-10T09:30:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:12 smithi082 ceph-mon[98694]: pgmap v19701: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 558 KiB/s wr, 32 op/s 2023-09-10T09:30:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:12 smithi098 ceph-mon[103124]: pgmap v19701: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 558 KiB/s wr, 32 op/s 2023-09-10T09:30:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36644.621502 secs 2023-09-10T09:30:13.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36644.621502 secs 2023-09-10T09:30:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36644.621502 secs 2023-09-10T09:30:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:14 smithi170 ceph-mon[102832]: pgmap v19702: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 558 KiB/s wr, 32 op/s 2023-09-10T09:30:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36645.869744 secs 2023-09-10T09:30:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:14 smithi170 ceph-mon[102832]: osdmap e1300: 12 total, 12 up, 12 in 2023-09-10T09:30:14.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:14 smithi082 ceph-mon[98694]: pgmap v19702: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 558 KiB/s wr, 32 op/s 2023-09-10T09:30:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36645.869744 secs 2023-09-10T09:30:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:14 smithi082 ceph-mon[98694]: osdmap e1300: 12 total, 12 up, 12 in 2023-09-10T09:30:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:14 smithi098 ceph-mon[103124]: pgmap v19702: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 558 KiB/s wr, 32 op/s 2023-09-10T09:30:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36645.869744 secs 2023-09-10T09:30:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:14 smithi098 ceph-mon[103124]: osdmap e1300: 12 total, 12 up, 12 in 2023-09-10T09:30:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36646.222559 secs 2023-09-10T09:30:15.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36646.222559 secs 2023-09-10T09:30:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36646.222559 secs 2023-09-10T09:30:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:16 smithi170 ceph-mon[102832]: pgmap v19704: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 201 KiB/s wr, 11 op/s 2023-09-10T09:30:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:16 smithi082 ceph-mon[98694]: pgmap v19704: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 201 KiB/s wr, 11 op/s 2023-09-10T09:30:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:16 smithi098 ceph-mon[103124]: pgmap v19704: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 201 KiB/s wr, 11 op/s 2023-09-10T09:30:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:18 smithi170 ceph-mon[102832]: pgmap v19705: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 217 KiB/s wr, 16 op/s 2023-09-10T09:30:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36649.621577 secs 2023-09-10T09:30:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:18 smithi082 ceph-mon[98694]: pgmap v19705: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 217 KiB/s wr, 16 op/s 2023-09-10T09:30:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36649.621577 secs 2023-09-10T09:30:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:18 smithi098 ceph-mon[103124]: pgmap v19705: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 217 KiB/s wr, 16 op/s 2023-09-10T09:30:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36649.621577 secs 2023-09-10T09:30:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36650.869816 secs 2023-09-10T09:30:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36651.222622 secs 2023-09-10T09:30:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:19 smithi098 ceph-mon[103124]: pgmap v19706: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 14 op/s 2023-09-10T09:30:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36650.869816 secs 2023-09-10T09:30:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36651.222622 secs 2023-09-10T09:30:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:19 smithi170 ceph-mon[102832]: pgmap v19706: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 14 op/s 2023-09-10T09:30:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36650.869816 secs 2023-09-10T09:30:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36651.222622 secs 2023-09-10T09:30:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:19 smithi082 ceph-mon[98694]: pgmap v19706: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 14 op/s 2023-09-10T09:30:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:22 smithi170 ceph-mon[102832]: pgmap v19707: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 14 op/s 2023-09-10T09:30:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:22 smithi082 ceph-mon[98694]: pgmap v19707: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 14 op/s 2023-09-10T09:30:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:22 smithi098 ceph-mon[103124]: pgmap v19707: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 14 op/s 2023-09-10T09:30:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36654.621669 secs 2023-09-10T09:30:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36654.621669 secs 2023-09-10T09:30:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36654.621669 secs 2023-09-10T09:30:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:24 smithi170 ceph-mon[102832]: pgmap v19708: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 14 op/s 2023-09-10T09:30:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36655.869875 secs 2023-09-10T09:30:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:24 smithi082 ceph-mon[98694]: pgmap v19708: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 14 op/s 2023-09-10T09:30:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36655.869875 secs 2023-09-10T09:30:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:24 smithi098 ceph-mon[103124]: pgmap v19708: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 14 op/s 2023-09-10T09:30:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36655.869875 secs 2023-09-10T09:30:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36656.222687 secs 2023-09-10T09:30:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36656.222687 secs 2023-09-10T09:30:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36656.222687 secs 2023-09-10T09:30:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:26 smithi170 ceph-mon[102832]: pgmap v19709: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 172 KiB/s wr, 14 op/s 2023-09-10T09:30:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:26 smithi082 ceph-mon[98694]: pgmap v19709: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 172 KiB/s wr, 14 op/s 2023-09-10T09:30:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:26 smithi098 ceph-mon[103124]: pgmap v19709: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 172 KiB/s wr, 14 op/s 2023-09-10T09:30:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:28 smithi170 ceph-mon[102832]: pgmap v19710: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 161 KiB/s wr, 15 op/s 2023-09-10T09:30:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36659.621727 secs 2023-09-10T09:30:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:28 smithi082 ceph-mon[98694]: pgmap v19710: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 161 KiB/s wr, 15 op/s 2023-09-10T09:30:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36659.621727 secs 2023-09-10T09:30:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:28 smithi098 ceph-mon[103124]: pgmap v19710: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 161 KiB/s wr, 15 op/s 2023-09-10T09:30:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36659.621727 secs 2023-09-10T09:30:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36660.869961 secs 2023-09-10T09:30:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36661.222754 secs 2023-09-10T09:30:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:29 smithi098 ceph-mon[103124]: pgmap v19711: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T09:30:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:29 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:30:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:29 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36660.869961 secs 2023-09-10T09:30:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36661.222754 secs 2023-09-10T09:30:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:29 smithi170 ceph-mon[102832]: pgmap v19711: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T09:30:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:29 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:30:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:29 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36660.869961 secs 2023-09-10T09:30:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36661.222754 secs 2023-09-10T09:30:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:29 smithi082 ceph-mon[98694]: pgmap v19711: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 11 op/s 2023-09-10T09:30:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:29 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:30:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:29 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:30:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:32 smithi082 ceph-mon[98694]: pgmap v19712: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 289 KiB/s wr, 17 op/s 2023-09-10T09:30:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:30:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:32 smithi098 ceph-mon[103124]: pgmap v19712: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 289 KiB/s wr, 17 op/s 2023-09-10T09:30:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:30:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:32 smithi170 ceph-mon[102832]: pgmap v19712: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 289 KiB/s wr, 17 op/s 2023-09-10T09:30:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:34 smithi170 ceph-mon[102832]: pgmap v19713: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 156 KiB/s wr, 11 op/s 2023-09-10T09:30:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36665.870019 secs 2023-09-10T09:30:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:34 smithi082 ceph-mon[98694]: pgmap v19713: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 156 KiB/s wr, 11 op/s 2023-09-10T09:30:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36665.870019 secs 2023-09-10T09:30:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:34 smithi098 ceph-mon[103124]: pgmap v19713: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 156 KiB/s wr, 11 op/s 2023-09-10T09:30:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36665.870019 secs 2023-09-10T09:30:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36666.222820 secs 2023-09-10T09:30:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:30:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36666.222820 secs 2023-09-10T09:30:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:30:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36666.222820 secs 2023-09-10T09:30:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:30:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:36 smithi170 ceph-mon[102832]: pgmap v19714: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 326 KiB/s wr, 13 op/s 2023-09-10T09:30:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:30:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:30:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:36 smithi082 ceph-mon[98694]: pgmap v19714: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 326 KiB/s wr, 13 op/s 2023-09-10T09:30:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:30:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:30:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:36 smithi098 ceph-mon[103124]: pgmap v19714: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 326 KiB/s wr, 13 op/s 2023-09-10T09:30:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:30:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:30:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:38 smithi170 ceph-mon[102832]: pgmap v19715: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 327 KiB/s wr, 15 op/s 2023-09-10T09:30:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:30:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:30:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:38 smithi082 ceph-mon[98694]: pgmap v19715: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 327 KiB/s wr, 15 op/s 2023-09-10T09:30:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:30:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:30:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:38 smithi098 ceph-mon[103124]: pgmap v19715: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 327 KiB/s wr, 15 op/s 2023-09-10T09:30:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:30:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:30:39.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:30:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:30:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36670.870080 secs 2023-09-10T09:30:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:30:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:30:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36671.222886 secs 2023-09-10T09:30:39.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:30:39.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:30:39.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:30:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:30:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:30:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36670.870080 secs 2023-09-10T09:30:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:30:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:30:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36671.222886 secs 2023-09-10T09:30:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:30:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:30:39.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:30:40.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:30:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:30:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36670.870080 secs 2023-09-10T09:30:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:30:40.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:30:40.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36671.222886 secs 2023-09-10T09:30:40.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:30:40.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:30:40.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:30:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:40 smithi170 ceph-mon[102832]: pgmap v19716: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 327 KiB/s wr, 11 op/s 2023-09-10T09:30:40.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:30:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:40 smithi082 ceph-mon[98694]: pgmap v19716: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 327 KiB/s wr, 11 op/s 2023-09-10T09:30:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:30:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:40 smithi098 ceph-mon[103124]: pgmap v19716: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 327 KiB/s wr, 11 op/s 2023-09-10T09:30:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:30:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:42 smithi170 ceph-mon[102832]: pgmap v19717: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 484 KiB/s wr, 18 op/s 2023-09-10T09:30:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:42 smithi082 ceph-mon[98694]: pgmap v19717: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 484 KiB/s wr, 18 op/s 2023-09-10T09:30:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:42 smithi098 ceph-mon[103124]: pgmap v19717: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 484 KiB/s wr, 18 op/s 2023-09-10T09:30:44.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:44 smithi170 ceph-mon[102832]: pgmap v19718: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 328 KiB/s wr, 12 op/s 2023-09-10T09:30:44.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:30:44.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:44.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36675.870138 secs 2023-09-10T09:30:44.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36676.222948 secs 2023-09-10T09:30:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:44 smithi082 ceph-mon[98694]: pgmap v19718: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 328 KiB/s wr, 12 op/s 2023-09-10T09:30:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:30:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36675.870138 secs 2023-09-10T09:30:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36676.222948 secs 2023-09-10T09:30:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:44 smithi098 ceph-mon[103124]: pgmap v19718: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 328 KiB/s wr, 12 op/s 2023-09-10T09:30:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:30:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36675.870138 secs 2023-09-10T09:30:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36676.222948 secs 2023-09-10T09:30:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:46 smithi170 ceph-mon[102832]: pgmap v19719: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 818 KiB/s wr, 16 op/s 2023-09-10T09:30:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:30:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:46 smithi082 ceph-mon[98694]: pgmap v19719: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 818 KiB/s wr, 16 op/s 2023-09-10T09:30:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:30:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:46 smithi098 ceph-mon[103124]: pgmap v19719: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 818 KiB/s wr, 16 op/s 2023-09-10T09:30:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:30:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:48 smithi170 ceph-mon[102832]: pgmap v19720: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 674 KiB/s wr, 17 op/s 2023-09-10T09:30:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36679.622010 secs 2023-09-10T09:30:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:48 smithi082 ceph-mon[98694]: pgmap v19720: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 674 KiB/s wr, 17 op/s 2023-09-10T09:30:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36679.622010 secs 2023-09-10T09:30:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:48 smithi098 ceph-mon[103124]: pgmap v19720: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 674 KiB/s wr, 17 op/s 2023-09-10T09:30:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36679.622010 secs 2023-09-10T09:30:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36680.870195 secs 2023-09-10T09:30:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36681.223014 secs 2023-09-10T09:30:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:49 smithi098 ceph-mon[103124]: pgmap v19721: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 674 KiB/s wr, 13 op/s 2023-09-10T09:30:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:30:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36680.870195 secs 2023-09-10T09:30:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36681.223014 secs 2023-09-10T09:30:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:49 smithi170 ceph-mon[102832]: pgmap v19721: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 674 KiB/s wr, 13 op/s 2023-09-10T09:30:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:30:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36680.870195 secs 2023-09-10T09:30:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36681.223014 secs 2023-09-10T09:30:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:49 smithi082 ceph-mon[98694]: pgmap v19721: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 674 KiB/s wr, 13 op/s 2023-09-10T09:30:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:30:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:30:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:30:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:30:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:30:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:30:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:30:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:52 smithi098 ceph-mon[103124]: pgmap v19722: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:30:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:52 smithi170 ceph-mon[102832]: pgmap v19722: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:30:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:52 smithi082 ceph-mon[98694]: pgmap v19722: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:30:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36684.622082 secs 2023-09-10T09:30:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36684.622082 secs 2023-09-10T09:30:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36684.622082 secs 2023-09-10T09:30:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:54 smithi098 ceph-mon[103124]: pgmap v19723: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 922 KiB/s wr, 15 op/s 2023-09-10T09:30:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36685.870299 secs 2023-09-10T09:30:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:54 smithi170 ceph-mon[102832]: pgmap v19723: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 922 KiB/s wr, 15 op/s 2023-09-10T09:30:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36685.870299 secs 2023-09-10T09:30:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:54 smithi082 ceph-mon[98694]: pgmap v19723: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 922 KiB/s wr, 15 op/s 2023-09-10T09:30:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36685.870299 secs 2023-09-10T09:30:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36686.223078 secs 2023-09-10T09:30:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36686.223078 secs 2023-09-10T09:30:55.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36686.223078 secs 2023-09-10T09:30:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:56 smithi098 ceph-mon[103124]: pgmap v19724: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 923 KiB/s wr, 17 op/s 2023-09-10T09:30:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:30:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:56 smithi170 ceph-mon[102832]: pgmap v19724: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 923 KiB/s wr, 17 op/s 2023-09-10T09:30:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:30:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:56 smithi082 ceph-mon[98694]: pgmap v19724: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 923 KiB/s wr, 17 op/s 2023-09-10T09:30:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:30:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:58 smithi170 ceph-mon[102832]: pgmap v19725: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 529 KiB/s wr, 17 op/s 2023-09-10T09:30:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:30:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36689.622143 secs 2023-09-10T09:30:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:58 smithi082 ceph-mon[98694]: pgmap v19725: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 529 KiB/s wr, 17 op/s 2023-09-10T09:30:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:30:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36689.622143 secs 2023-09-10T09:30:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:58 smithi098 ceph-mon[103124]: pgmap v19725: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 529 KiB/s wr, 17 op/s 2023-09-10T09:30:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:30:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:30:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36689.622143 secs 2023-09-10T09:31:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36690.870366 secs 2023-09-10T09:31:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36691.223144 secs 2023-09-10T09:31:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:30:59 smithi098 ceph-mon[103124]: pgmap v19726: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 501 KiB/s wr, 14 op/s 2023-09-10T09:31:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36690.870366 secs 2023-09-10T09:31:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36691.223144 secs 2023-09-10T09:31:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:30:59 smithi170 ceph-mon[102832]: pgmap v19726: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 501 KiB/s wr, 14 op/s 2023-09-10T09:31:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36690.870366 secs 2023-09-10T09:31:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36691.223144 secs 2023-09-10T09:31:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:30:59 smithi082 ceph-mon[98694]: pgmap v19726: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 501 KiB/s wr, 14 op/s 2023-09-10T09:31:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:02 smithi170 ceph-mon[102832]: pgmap v19727: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 739 KiB/s wr, 20 op/s 2023-09-10T09:31:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:02 smithi082 ceph-mon[98694]: pgmap v19727: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 739 KiB/s wr, 20 op/s 2023-09-10T09:31:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:02 smithi098 ceph-mon[103124]: pgmap v19727: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 739 KiB/s wr, 20 op/s 2023-09-10T09:31:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36694.622253 secs 2023-09-10T09:31:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36694.622253 secs 2023-09-10T09:31:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36694.622253 secs 2023-09-10T09:31:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:04 smithi170 ceph-mon[102832]: pgmap v19728: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 334 KiB/s wr, 12 op/s 2023-09-10T09:31:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:31:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36695.870434 secs 2023-09-10T09:31:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:04 smithi082 ceph-mon[98694]: pgmap v19728: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 334 KiB/s wr, 12 op/s 2023-09-10T09:31:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:31:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36695.870434 secs 2023-09-10T09:31:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:04 smithi098 ceph-mon[103124]: pgmap v19728: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 334 KiB/s wr, 12 op/s 2023-09-10T09:31:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:31:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36695.870434 secs 2023-09-10T09:31:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36696.223212 secs 2023-09-10T09:31:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36696.223212 secs 2023-09-10T09:31:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36696.223212 secs 2023-09-10T09:31:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:06 smithi170 ceph-mon[102832]: pgmap v19729: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 377 KiB/s wr, 28 op/s 2023-09-10T09:31:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:06 smithi082 ceph-mon[98694]: pgmap v19729: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 377 KiB/s wr, 28 op/s 2023-09-10T09:31:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:06 smithi098 ceph-mon[103124]: pgmap v19729: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 377 KiB/s wr, 28 op/s 2023-09-10T09:31:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:08 smithi170 ceph-mon[102832]: pgmap v19730: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 685 KiB/s wr, 31 op/s 2023-09-10T09:31:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36699.622313 secs 2023-09-10T09:31:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:08 smithi082 ceph-mon[98694]: pgmap v19730: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 685 KiB/s wr, 31 op/s 2023-09-10T09:31:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36699.622313 secs 2023-09-10T09:31:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:08 smithi098 ceph-mon[103124]: pgmap v19730: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 685 KiB/s wr, 31 op/s 2023-09-10T09:31:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36699.622313 secs 2023-09-10T09:31:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:09 smithi170 ceph-mon[102832]: osdmap e1301: 12 total, 12 up, 12 in 2023-09-10T09:31:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36700.870521 secs 2023-09-10T09:31:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:09 smithi082 ceph-mon[98694]: osdmap e1301: 12 total, 12 up, 12 in 2023-09-10T09:31:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36700.870521 secs 2023-09-10T09:31:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:09 smithi098 ceph-mon[103124]: osdmap e1301: 12 total, 12 up, 12 in 2023-09-10T09:31:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36700.870521 secs 2023-09-10T09:31:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36701.223277 secs 2023-09-10T09:31:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:10 smithi170 ceph-mon[102832]: pgmap v19732: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 707 KiB/s wr, 33 op/s 2023-09-10T09:31:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:31:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36701.223277 secs 2023-09-10T09:31:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:10 smithi082 ceph-mon[98694]: pgmap v19732: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 707 KiB/s wr, 33 op/s 2023-09-10T09:31:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:31:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36701.223277 secs 2023-09-10T09:31:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:10 smithi098 ceph-mon[103124]: pgmap v19732: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 707 KiB/s wr, 33 op/s 2023-09-10T09:31:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:31:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:12 smithi170 ceph-mon[102832]: pgmap v19733: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 582 KiB/s wr, 32 op/s 2023-09-10T09:31:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:12 smithi082 ceph-mon[98694]: pgmap v19733: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 582 KiB/s wr, 32 op/s 2023-09-10T09:31:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:12 smithi098 ceph-mon[103124]: pgmap v19733: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 582 KiB/s wr, 32 op/s 2023-09-10T09:31:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36704.622417 secs 2023-09-10T09:31:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36704.622417 secs 2023-09-10T09:31:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36704.622417 secs 2023-09-10T09:31:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:14 smithi170 ceph-mon[102832]: pgmap v19734: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 582 KiB/s wr, 32 op/s 2023-09-10T09:31:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36705.870575 secs 2023-09-10T09:31:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:14 smithi170 ceph-mon[102832]: osdmap e1302: 12 total, 12 up, 12 in 2023-09-10T09:31:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:14 smithi082 ceph-mon[98694]: pgmap v19734: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 582 KiB/s wr, 32 op/s 2023-09-10T09:31:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36705.870575 secs 2023-09-10T09:31:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:14 smithi082 ceph-mon[98694]: osdmap e1302: 12 total, 12 up, 12 in 2023-09-10T09:31:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:14 smithi098 ceph-mon[103124]: pgmap v19734: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 582 KiB/s wr, 32 op/s 2023-09-10T09:31:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36705.870575 secs 2023-09-10T09:31:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:14 smithi098 ceph-mon[103124]: osdmap e1302: 12 total, 12 up, 12 in 2023-09-10T09:31:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36706.223339 secs 2023-09-10T09:31:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36706.223339 secs 2023-09-10T09:31:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36706.223339 secs 2023-09-10T09:31:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:16 smithi170 ceph-mon[102832]: pgmap v19736: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 201 KiB/s wr, 13 op/s 2023-09-10T09:31:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:31:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:16 smithi082 ceph-mon[98694]: pgmap v19736: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 201 KiB/s wr, 13 op/s 2023-09-10T09:31:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:31:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:16 smithi098 ceph-mon[103124]: pgmap v19736: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 201 KiB/s wr, 13 op/s 2023-09-10T09:31:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:31:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:18 smithi170 ceph-mon[102832]: pgmap v19737: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 217 KiB/s wr, 16 op/s 2023-09-10T09:31:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:31:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36709.622507 secs 2023-09-10T09:31:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:18 smithi082 ceph-mon[98694]: pgmap v19737: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 217 KiB/s wr, 16 op/s 2023-09-10T09:31:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:31:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36709.622507 secs 2023-09-10T09:31:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:18 smithi098 ceph-mon[103124]: pgmap v19737: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 217 KiB/s wr, 16 op/s 2023-09-10T09:31:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:31:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36709.622507 secs 2023-09-10T09:31:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36710.870634 secs 2023-09-10T09:31:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36711.223405 secs 2023-09-10T09:31:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:19 smithi098 ceph-mon[103124]: pgmap v19738: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 14 op/s 2023-09-10T09:31:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36710.870634 secs 2023-09-10T09:31:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36711.223405 secs 2023-09-10T09:31:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:19 smithi170 ceph-mon[102832]: pgmap v19738: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 14 op/s 2023-09-10T09:31:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36710.870634 secs 2023-09-10T09:31:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36711.223405 secs 2023-09-10T09:31:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:19 smithi082 ceph-mon[98694]: pgmap v19738: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 194 KiB/s wr, 14 op/s 2023-09-10T09:31:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:22 smithi170 ceph-mon[102832]: pgmap v19739: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 13 op/s 2023-09-10T09:31:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:22 smithi082 ceph-mon[98694]: pgmap v19739: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 13 op/s 2023-09-10T09:31:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:22 smithi098 ceph-mon[103124]: pgmap v19739: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 13 op/s 2023-09-10T09:31:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36714.622568 secs 2023-09-10T09:31:23.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36714.622568 secs 2023-09-10T09:31:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36714.622568 secs 2023-09-10T09:31:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:24 smithi170 ceph-mon[102832]: pgmap v19740: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 13 op/s 2023-09-10T09:31:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36715.870693 secs 2023-09-10T09:31:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:24 smithi082 ceph-mon[98694]: pgmap v19740: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 13 op/s 2023-09-10T09:31:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36715.870693 secs 2023-09-10T09:31:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:24 smithi098 ceph-mon[103124]: pgmap v19740: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 13 op/s 2023-09-10T09:31:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36715.870693 secs 2023-09-10T09:31:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36716.223479 secs 2023-09-10T09:31:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36716.223479 secs 2023-09-10T09:31:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36716.223479 secs 2023-09-10T09:31:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:26 smithi170 ceph-mon[102832]: pgmap v19741: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 172 KiB/s wr, 15 op/s 2023-09-10T09:31:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:26 smithi082 ceph-mon[98694]: pgmap v19741: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 172 KiB/s wr, 15 op/s 2023-09-10T09:31:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:26 smithi098 ceph-mon[103124]: pgmap v19741: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 172 KiB/s wr, 15 op/s 2023-09-10T09:31:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:28 smithi170 ceph-mon[102832]: pgmap v19742: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 13 op/s 2023-09-10T09:31:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36719.622627 secs 2023-09-10T09:31:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:28 smithi082 ceph-mon[98694]: pgmap v19742: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 13 op/s 2023-09-10T09:31:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36719.622627 secs 2023-09-10T09:31:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:28 smithi098 ceph-mon[103124]: pgmap v19742: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 13 op/s 2023-09-10T09:31:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36719.622627 secs 2023-09-10T09:31:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36720.870753 secs 2023-09-10T09:31:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36721.223547 secs 2023-09-10T09:31:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:30 smithi170 ceph-mon[102832]: pgmap v19743: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 134 KiB/s wr, 11 op/s 2023-09-10T09:31:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36720.870753 secs 2023-09-10T09:31:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36721.223547 secs 2023-09-10T09:31:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:30 smithi082 ceph-mon[98694]: pgmap v19743: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 134 KiB/s wr, 11 op/s 2023-09-10T09:31:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36720.870753 secs 2023-09-10T09:31:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36721.223547 secs 2023-09-10T09:31:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:30 smithi098 ceph-mon[103124]: pgmap v19743: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 134 KiB/s wr, 11 op/s 2023-09-10T09:31:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:32 smithi170 ceph-mon[102832]: pgmap v19744: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 134 KiB/s wr, 15 op/s 2023-09-10T09:31:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:32 smithi082 ceph-mon[98694]: pgmap v19744: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 134 KiB/s wr, 15 op/s 2023-09-10T09:31:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:32 smithi098 ceph-mon[103124]: pgmap v19744: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 134 KiB/s wr, 15 op/s 2023-09-10T09:31:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:34 smithi170 ceph-mon[102832]: pgmap v19745: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 10 op/s 2023-09-10T09:31:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36725.870813 secs 2023-09-10T09:31:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:34 smithi082 ceph-mon[98694]: pgmap v19745: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 10 op/s 2023-09-10T09:31:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36725.870813 secs 2023-09-10T09:31:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:34 smithi098 ceph-mon[103124]: pgmap v19745: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 10 op/s 2023-09-10T09:31:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36725.870813 secs 2023-09-10T09:31:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36726.223611 secs 2023-09-10T09:31:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:31:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36726.223611 secs 2023-09-10T09:31:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:31:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36726.223611 secs 2023-09-10T09:31:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:31:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:36 smithi170 ceph-mon[102832]: pgmap v19746: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 418 KiB/s wr, 13 op/s 2023-09-10T09:31:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:31:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:31:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:36 smithi082 ceph-mon[98694]: pgmap v19746: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 418 KiB/s wr, 13 op/s 2023-09-10T09:31:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:31:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:31:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:36 smithi098 ceph-mon[103124]: pgmap v19746: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 418 KiB/s wr, 13 op/s 2023-09-10T09:31:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:31:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:31:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:38 smithi170 ceph-mon[102832]: pgmap v19747: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 286 KiB/s wr, 13 op/s 2023-09-10T09:31:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:38 smithi082 ceph-mon[98694]: pgmap v19747: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 286 KiB/s wr, 13 op/s 2023-09-10T09:31:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:38 smithi098 ceph-mon[103124]: pgmap v19747: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 286 KiB/s wr, 13 op/s 2023-09-10T09:31:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36730.870871 secs 2023-09-10T09:31:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36731.223680 secs 2023-09-10T09:31:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:40 smithi170 ceph-mon[102832]: pgmap v19748: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 285 KiB/s wr, 10 op/s 2023-09-10T09:31:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:31:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:31:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:31:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:31:40.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36730.870871 secs 2023-09-10T09:31:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36731.223680 secs 2023-09-10T09:31:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:40 smithi082 ceph-mon[98694]: pgmap v19748: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 285 KiB/s wr, 10 op/s 2023-09-10T09:31:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:31:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:31:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:31:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:31:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36730.870871 secs 2023-09-10T09:31:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36731.223680 secs 2023-09-10T09:31:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:40 smithi098 ceph-mon[103124]: pgmap v19748: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 285 KiB/s wr, 10 op/s 2023-09-10T09:31:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:31:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:31:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:31:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:31:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:41 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:31:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:41 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:41 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:31:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:41 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:41 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:31:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:41 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:42 smithi170 ceph-mon[102832]: pgmap v19749: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 443 KiB/s wr, 16 op/s 2023-09-10T09:31:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:42 smithi082 ceph-mon[98694]: pgmap v19749: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 443 KiB/s wr, 16 op/s 2023-09-10T09:31:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:42 smithi098 ceph-mon[103124]: pgmap v19749: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 443 KiB/s wr, 16 op/s 2023-09-10T09:31:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:44 smithi170 ceph-mon[102832]: pgmap v19750: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 443 KiB/s wr, 12 op/s 2023-09-10T09:31:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36735.870934 secs 2023-09-10T09:31:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:44 smithi082 ceph-mon[98694]: pgmap v19750: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 443 KiB/s wr, 12 op/s 2023-09-10T09:31:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36735.870934 secs 2023-09-10T09:31:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:44 smithi098 ceph-mon[103124]: pgmap v19750: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 443 KiB/s wr, 12 op/s 2023-09-10T09:31:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36735.870934 secs 2023-09-10T09:31:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36736.223746 secs 2023-09-10T09:31:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36736.223746 secs 2023-09-10T09:31:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36736.223746 secs 2023-09-10T09:31:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:46 smithi170 ceph-mon[102832]: pgmap v19751: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 564 KiB/s wr, 17 op/s 2023-09-10T09:31:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:31:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:46 smithi082 ceph-mon[98694]: pgmap v19751: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 564 KiB/s wr, 17 op/s 2023-09-10T09:31:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:31:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:46 smithi098 ceph-mon[103124]: pgmap v19751: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 564 KiB/s wr, 17 op/s 2023-09-10T09:31:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:31:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:48 smithi170 ceph-mon[102832]: pgmap v19752: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 279 KiB/s wr, 15 op/s 2023-09-10T09:31:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36739.622871 secs 2023-09-10T09:31:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:48 smithi082 ceph-mon[98694]: pgmap v19752: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 279 KiB/s wr, 15 op/s 2023-09-10T09:31:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36739.622871 secs 2023-09-10T09:31:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:48 smithi098 ceph-mon[103124]: pgmap v19752: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 279 KiB/s wr, 15 op/s 2023-09-10T09:31:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36739.622871 secs 2023-09-10T09:31:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36740.870993 secs 2023-09-10T09:31:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36741.223812 secs 2023-09-10T09:31:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:50 smithi170 ceph-mon[102832]: pgmap v19753: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 279 KiB/s wr, 13 op/s 2023-09-10T09:31:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:31:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36740.870993 secs 2023-09-10T09:31:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36741.223812 secs 2023-09-10T09:31:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:50 smithi082 ceph-mon[98694]: pgmap v19753: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 279 KiB/s wr, 13 op/s 2023-09-10T09:31:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:31:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:50.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36740.870993 secs 2023-09-10T09:31:50.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36741.223812 secs 2023-09-10T09:31:50.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:50 smithi098 ceph-mon[103124]: pgmap v19753: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 279 KiB/s wr, 13 op/s 2023-09-10T09:31:50.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:31:50.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:31:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:31:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:31:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:31:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:31:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:31:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:52 smithi098 ceph-mon[103124]: pgmap v19754: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 723 KiB/s wr, 21 op/s 2023-09-10T09:31:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:52 smithi170 ceph-mon[102832]: pgmap v19754: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 723 KiB/s wr, 21 op/s 2023-09-10T09:31:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:52 smithi082 ceph-mon[98694]: pgmap v19754: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 723 KiB/s wr, 21 op/s 2023-09-10T09:31:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:31:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:54 smithi098 ceph-mon[103124]: pgmap v19755: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 566 KiB/s wr, 15 op/s 2023-09-10T09:31:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36745.871054 secs 2023-09-10T09:31:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:31:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:54 smithi170 ceph-mon[102832]: pgmap v19755: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 566 KiB/s wr, 15 op/s 2023-09-10T09:31:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36745.871054 secs 2023-09-10T09:31:54.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:31:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:54 smithi082 ceph-mon[98694]: pgmap v19755: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 566 KiB/s wr, 15 op/s 2023-09-10T09:31:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36745.871054 secs 2023-09-10T09:31:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36746.223877 secs 2023-09-10T09:31:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:55 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:31:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36746.223877 secs 2023-09-10T09:31:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:55 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:31:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36746.223877 secs 2023-09-10T09:31:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:55 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:31:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:31:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:56 smithi098 ceph-mon[103124]: pgmap v19756: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 566 KiB/s wr, 18 op/s 2023-09-10T09:31:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:31:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:56 smithi170 ceph-mon[102832]: pgmap v19756: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 566 KiB/s wr, 18 op/s 2023-09-10T09:31:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:31:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:56 smithi082 ceph-mon[98694]: pgmap v19756: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 566 KiB/s wr, 18 op/s 2023-09-10T09:31:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:58 smithi170 ceph-mon[102832]: pgmap v19757: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 509 KiB/s wr, 16 op/s 2023-09-10T09:31:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:31:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:58 smithi082 ceph-mon[98694]: pgmap v19757: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 509 KiB/s wr, 16 op/s 2023-09-10T09:31:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:31:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:58 smithi098 ceph-mon[103124]: pgmap v19757: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 509 KiB/s wr, 16 op/s 2023-09-10T09:31:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:31:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:31:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36750.871138 secs 2023-09-10T09:31:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:31:59 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:31:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36750.871138 secs 2023-09-10T09:31:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:31:59 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:31:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36750.871138 secs 2023-09-10T09:31:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:31:59 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:32:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36751.223942 secs 2023-09-10T09:32:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:00 smithi170 ceph-mon[102832]: pgmap v19758: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 509 KiB/s wr, 14 op/s 2023-09-10T09:32:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:32:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36751.223942 secs 2023-09-10T09:32:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:00 smithi082 ceph-mon[98694]: pgmap v19758: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 509 KiB/s wr, 14 op/s 2023-09-10T09:32:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:32:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36751.223942 secs 2023-09-10T09:32:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:00 smithi098 ceph-mon[103124]: pgmap v19758: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 509 KiB/s wr, 14 op/s 2023-09-10T09:32:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:32:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:02 smithi170 ceph-mon[102832]: pgmap v19759: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 34 op/s 2023-09-10T09:32:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:02 smithi082 ceph-mon[98694]: pgmap v19759: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 34 op/s 2023-09-10T09:32:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:02 smithi098 ceph-mon[103124]: pgmap v19759: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 34 op/s 2023-09-10T09:32:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36754.623101 secs 2023-09-10T09:32:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36754.623101 secs 2023-09-10T09:32:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36754.623101 secs 2023-09-10T09:32:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:04 smithi170 ceph-mon[102832]: pgmap v19760: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 581 KiB/s wr, 26 op/s 2023-09-10T09:32:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:32:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:04 smithi170 ceph-mon[102832]: osdmap e1303: 12 total, 12 up, 12 in 2023-09-10T09:32:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36755.871198 secs 2023-09-10T09:32:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:32:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:04 smithi082 ceph-mon[98694]: pgmap v19760: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 581 KiB/s wr, 26 op/s 2023-09-10T09:32:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:32:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:04 smithi082 ceph-mon[98694]: osdmap e1303: 12 total, 12 up, 12 in 2023-09-10T09:32:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36755.871198 secs 2023-09-10T09:32:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:32:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:04 smithi098 ceph-mon[103124]: pgmap v19760: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 581 KiB/s wr, 26 op/s 2023-09-10T09:32:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:32:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:04 smithi098 ceph-mon[103124]: osdmap e1303: 12 total, 12 up, 12 in 2023-09-10T09:32:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36755.871198 secs 2023-09-10T09:32:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:32:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36756.224007 secs 2023-09-10T09:32:05.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36756.224007 secs 2023-09-10T09:32:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36756.224007 secs 2023-09-10T09:32:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:06 smithi170 ceph-mon[102832]: pgmap v19762: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 878 KiB/s wr, 35 op/s 2023-09-10T09:32:06.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:06 smithi082 ceph-mon[98694]: pgmap v19762: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 878 KiB/s wr, 35 op/s 2023-09-10T09:32:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:06 smithi098 ceph-mon[103124]: pgmap v19762: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 878 KiB/s wr, 35 op/s 2023-09-10T09:32:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:08 smithi170 ceph-mon[102832]: pgmap v19763: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 801 KiB/s wr, 34 op/s 2023-09-10T09:32:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36759.623214 secs 2023-09-10T09:32:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:08 smithi082 ceph-mon[98694]: pgmap v19763: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 801 KiB/s wr, 34 op/s 2023-09-10T09:32:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36759.623214 secs 2023-09-10T09:32:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:08 smithi098 ceph-mon[103124]: pgmap v19763: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 801 KiB/s wr, 34 op/s 2023-09-10T09:32:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36759.623214 secs 2023-09-10T09:32:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36760.871293 secs 2023-09-10T09:32:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36761.224074 secs 2023-09-10T09:32:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:10 smithi170 ceph-mon[102832]: osdmap e1304: 12 total, 12 up, 12 in 2023-09-10T09:32:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:10 smithi170 ceph-mon[102832]: pgmap v19765: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 227 KiB/s wr, 13 op/s 2023-09-10T09:32:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:32:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36760.871293 secs 2023-09-10T09:32:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36761.224074 secs 2023-09-10T09:32:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:10 smithi082 ceph-mon[98694]: osdmap e1304: 12 total, 12 up, 12 in 2023-09-10T09:32:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:10 smithi082 ceph-mon[98694]: pgmap v19765: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 227 KiB/s wr, 13 op/s 2023-09-10T09:32:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:32:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36760.871293 secs 2023-09-10T09:32:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36761.224074 secs 2023-09-10T09:32:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:10 smithi098 ceph-mon[103124]: osdmap e1304: 12 total, 12 up, 12 in 2023-09-10T09:32:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:10 smithi098 ceph-mon[103124]: pgmap v19765: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 227 KiB/s wr, 13 op/s 2023-09-10T09:32:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:32:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:32:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:12 smithi082 ceph-mon[98694]: pgmap v19766: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 667 KiB/s wr, 23 op/s 2023-09-10T09:32:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:32:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:12 smithi098 ceph-mon[103124]: pgmap v19766: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 667 KiB/s wr, 23 op/s 2023-09-10T09:32:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:32:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:12 smithi170 ceph-mon[102832]: pgmap v19766: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 667 KiB/s wr, 23 op/s 2023-09-10T09:32:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36764.623313 secs 2023-09-10T09:32:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36764.623313 secs 2023-09-10T09:32:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36764.623313 secs 2023-09-10T09:32:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:14 smithi082 ceph-mon[98694]: pgmap v19767: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 535 KiB/s wr, 18 op/s 2023-09-10T09:32:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36765.871369 secs 2023-09-10T09:32:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:14 smithi098 ceph-mon[103124]: pgmap v19767: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 535 KiB/s wr, 18 op/s 2023-09-10T09:32:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36765.871369 secs 2023-09-10T09:32:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:14 smithi170 ceph-mon[102832]: pgmap v19767: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 535 KiB/s wr, 18 op/s 2023-09-10T09:32:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36765.871369 secs 2023-09-10T09:32:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36766.224138 secs 2023-09-10T09:32:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36766.224138 secs 2023-09-10T09:32:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36766.224138 secs 2023-09-10T09:32:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:16 smithi082 ceph-mon[98694]: pgmap v19768: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 352 KiB/s wr, 13 op/s 2023-09-10T09:32:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:16 smithi098 ceph-mon[103124]: pgmap v19768: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 352 KiB/s wr, 13 op/s 2023-09-10T09:32:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:16 smithi170 ceph-mon[102832]: pgmap v19768: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 352 KiB/s wr, 13 op/s 2023-09-10T09:32:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:18 smithi170 ceph-mon[102832]: pgmap v19769: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 353 KiB/s wr, 13 op/s 2023-09-10T09:32:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36769.623373 secs 2023-09-10T09:32:18.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:18 smithi082 ceph-mon[98694]: pgmap v19769: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 353 KiB/s wr, 13 op/s 2023-09-10T09:32:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36769.623373 secs 2023-09-10T09:32:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:18 smithi098 ceph-mon[103124]: pgmap v19769: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 353 KiB/s wr, 13 op/s 2023-09-10T09:32:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36769.623373 secs 2023-09-10T09:32:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36770.871429 secs 2023-09-10T09:32:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36771.224234 secs 2023-09-10T09:32:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:20 smithi170 ceph-mon[102832]: pgmap v19770: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 345 KiB/s wr, 13 op/s 2023-09-10T09:32:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36770.871429 secs 2023-09-10T09:32:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36771.224234 secs 2023-09-10T09:32:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:20 smithi082 ceph-mon[98694]: pgmap v19770: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 345 KiB/s wr, 13 op/s 2023-09-10T09:32:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36770.871429 secs 2023-09-10T09:32:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36771.224234 secs 2023-09-10T09:32:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:20 smithi098 ceph-mon[103124]: pgmap v19770: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 345 KiB/s wr, 13 op/s 2023-09-10T09:32:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:22 smithi170 ceph-mon[102832]: pgmap v19771: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 295 KiB/s wr, 16 op/s 2023-09-10T09:32:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:22 smithi082 ceph-mon[98694]: pgmap v19771: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 295 KiB/s wr, 16 op/s 2023-09-10T09:32:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:22 smithi098 ceph-mon[103124]: pgmap v19771: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 295 KiB/s wr, 16 op/s 2023-09-10T09:32:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36774.623447 secs 2023-09-10T09:32:23.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36774.623447 secs 2023-09-10T09:32:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36774.623447 secs 2023-09-10T09:32:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:24 smithi170 ceph-mon[102832]: pgmap v19772: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2023-09-10T09:32:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36775.871498 secs 2023-09-10T09:32:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:24 smithi082 ceph-mon[98694]: pgmap v19772: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2023-09-10T09:32:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36775.871498 secs 2023-09-10T09:32:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:24 smithi098 ceph-mon[103124]: pgmap v19772: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2023-09-10T09:32:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36775.871498 secs 2023-09-10T09:32:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36776.224299 secs 2023-09-10T09:32:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36776.224299 secs 2023-09-10T09:32:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36776.224299 secs 2023-09-10T09:32:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:26 smithi170 ceph-mon[102832]: pgmap v19773: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 14 op/s 2023-09-10T09:32:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:26 smithi082 ceph-mon[98694]: pgmap v19773: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 14 op/s 2023-09-10T09:32:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:26 smithi098 ceph-mon[103124]: pgmap v19773: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 14 op/s 2023-09-10T09:32:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:28 smithi170 ceph-mon[102832]: pgmap v19774: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 134 KiB/s wr, 12 op/s 2023-09-10T09:32:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36779.623506 secs 2023-09-10T09:32:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:28 smithi082 ceph-mon[98694]: pgmap v19774: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 134 KiB/s wr, 12 op/s 2023-09-10T09:32:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36779.623506 secs 2023-09-10T09:32:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:28 smithi098 ceph-mon[103124]: pgmap v19774: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 134 KiB/s wr, 12 op/s 2023-09-10T09:32:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36779.623506 secs 2023-09-10T09:32:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36780.871559 secs 2023-09-10T09:32:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36781.224363 secs 2023-09-10T09:32:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:30 smithi170 ceph-mon[102832]: pgmap v19775: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 134 KiB/s wr, 11 op/s 2023-09-10T09:32:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36780.871559 secs 2023-09-10T09:32:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36781.224363 secs 2023-09-10T09:32:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:30 smithi082 ceph-mon[98694]: pgmap v19775: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 134 KiB/s wr, 11 op/s 2023-09-10T09:32:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36780.871559 secs 2023-09-10T09:32:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36781.224363 secs 2023-09-10T09:32:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:30 smithi098 ceph-mon[103124]: pgmap v19775: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 134 KiB/s wr, 11 op/s 2023-09-10T09:32:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:32 smithi170 ceph-mon[102832]: pgmap v19776: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 134 KiB/s wr, 16 op/s 2023-09-10T09:32:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:32 smithi082 ceph-mon[98694]: pgmap v19776: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 134 KiB/s wr, 16 op/s 2023-09-10T09:32:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:32 smithi098 ceph-mon[103124]: pgmap v19776: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 134 KiB/s wr, 16 op/s 2023-09-10T09:32:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:34 smithi170 ceph-mon[102832]: pgmap v19777: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 134 KiB/s wr, 11 op/s 2023-09-10T09:32:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:32:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36785.871619 secs 2023-09-10T09:32:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:34 smithi082 ceph-mon[98694]: pgmap v19777: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 134 KiB/s wr, 11 op/s 2023-09-10T09:32:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:32:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36785.871619 secs 2023-09-10T09:32:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:34 smithi098 ceph-mon[103124]: pgmap v19777: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 134 KiB/s wr, 11 op/s 2023-09-10T09:32:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:32:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36785.871619 secs 2023-09-10T09:32:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36786.224430 secs 2023-09-10T09:32:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:32:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36786.224430 secs 2023-09-10T09:32:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:32:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36786.224430 secs 2023-09-10T09:32:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:32:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:36 smithi170 ceph-mon[102832]: pgmap v19778: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 393 KiB/s wr, 15 op/s 2023-09-10T09:32:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:32:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:32:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:36 smithi082 ceph-mon[98694]: pgmap v19778: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 393 KiB/s wr, 15 op/s 2023-09-10T09:32:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:32:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:32:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:36 smithi098 ceph-mon[103124]: pgmap v19778: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 393 KiB/s wr, 15 op/s 2023-09-10T09:32:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:32:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:32:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:38 smithi170 ceph-mon[102832]: pgmap v19779: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T09:32:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:38 smithi082 ceph-mon[98694]: pgmap v19779: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T09:32:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:38 smithi098 ceph-mon[103124]: pgmap v19779: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 260 KiB/s wr, 12 op/s 2023-09-10T09:32:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36790.871678 secs 2023-09-10T09:32:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36791.224486 secs 2023-09-10T09:32:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:40 smithi170 ceph-mon[102832]: pgmap v19780: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 259 KiB/s wr, 10 op/s 2023-09-10T09:32:40.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36790.871678 secs 2023-09-10T09:32:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36791.224486 secs 2023-09-10T09:32:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:40 smithi082 ceph-mon[98694]: pgmap v19780: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 259 KiB/s wr, 10 op/s 2023-09-10T09:32:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36790.871678 secs 2023-09-10T09:32:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36791.224486 secs 2023-09-10T09:32:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:40 smithi098 ceph-mon[103124]: pgmap v19780: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 259 KiB/s wr, 10 op/s 2023-09-10T09:32:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:41 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:32:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:41 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:32:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:32:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:32:41.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:32:41.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:41 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:32:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:41 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:32:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:32:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:32:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:32:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:41 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:32:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:41 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:32:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:32:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:32:41.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:32:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:42 smithi170 ceph-mon[102832]: pgmap v19781: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 580 KiB/s wr, 18 op/s 2023-09-10T09:32:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:42 smithi082 ceph-mon[98694]: pgmap v19781: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 580 KiB/s wr, 18 op/s 2023-09-10T09:32:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:42 smithi098 ceph-mon[103124]: pgmap v19781: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 580 KiB/s wr, 18 op/s 2023-09-10T09:32:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36794.623745 secs 2023-09-10T09:32:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36794.623745 secs 2023-09-10T09:32:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36794.623745 secs 2023-09-10T09:32:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:44 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:32:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:32:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:44 smithi170 ceph-mon[102832]: pgmap v19782: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 580 KiB/s wr, 13 op/s 2023-09-10T09:32:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:44 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:32:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:32:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:44 smithi082 ceph-mon[98694]: pgmap v19782: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 580 KiB/s wr, 13 op/s 2023-09-10T09:32:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:44 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:32:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:32:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:44 smithi098 ceph-mon[103124]: pgmap v19782: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 580 KiB/s wr, 13 op/s 2023-09-10T09:32:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36795.871735 secs 2023-09-10T09:32:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36796.224552 secs 2023-09-10T09:32:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36795.871735 secs 2023-09-10T09:32:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36796.224552 secs 2023-09-10T09:32:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36795.871735 secs 2023-09-10T09:32:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36796.224552 secs 2023-09-10T09:32:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:32:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:46 smithi082 ceph-mon[98694]: pgmap v19783: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 580 KiB/s wr, 16 op/s 2023-09-10T09:32:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:32:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:46 smithi098 ceph-mon[103124]: pgmap v19783: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 580 KiB/s wr, 16 op/s 2023-09-10T09:32:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:32:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:46 smithi170 ceph-mon[102832]: pgmap v19783: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 580 KiB/s wr, 16 op/s 2023-09-10T09:32:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:48 smithi170 ceph-mon[102832]: pgmap v19784: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 401 KiB/s wr, 15 op/s 2023-09-10T09:32:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36799.623774 secs 2023-09-10T09:32:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:48 smithi082 ceph-mon[98694]: pgmap v19784: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 401 KiB/s wr, 15 op/s 2023-09-10T09:32:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36799.623774 secs 2023-09-10T09:32:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:48 smithi098 ceph-mon[103124]: pgmap v19784: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 401 KiB/s wr, 15 op/s 2023-09-10T09:32:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36799.623774 secs 2023-09-10T09:32:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36800.871796 secs 2023-09-10T09:32:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36801.224618 secs 2023-09-10T09:32:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:50 smithi170 ceph-mon[102832]: pgmap v19785: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 401 KiB/s wr, 13 op/s 2023-09-10T09:32:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:32:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36800.871796 secs 2023-09-10T09:32:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36801.224618 secs 2023-09-10T09:32:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:50 smithi082 ceph-mon[98694]: pgmap v19785: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 401 KiB/s wr, 13 op/s 2023-09-10T09:32:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:32:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36800.871796 secs 2023-09-10T09:32:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36801.224618 secs 2023-09-10T09:32:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:50 smithi098 ceph-mon[103124]: pgmap v19785: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 401 KiB/s wr, 13 op/s 2023-09-10T09:32:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:32:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:32:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:52 smithi082 ceph-mon[98694]: pgmap v19786: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T09:32:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:32:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:52 smithi098 ceph-mon[103124]: pgmap v19786: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T09:32:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:32:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:32:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:52 smithi170 ceph-mon[102832]: pgmap v19786: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T09:32:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36804.623835 secs 2023-09-10T09:32:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36804.623835 secs 2023-09-10T09:32:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36804.623835 secs 2023-09-10T09:32:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:54 smithi082 ceph-mon[98694]: pgmap v19787: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 728 KiB/s wr, 13 op/s 2023-09-10T09:32:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36805.871856 secs 2023-09-10T09:32:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:54 smithi098 ceph-mon[103124]: pgmap v19787: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 728 KiB/s wr, 13 op/s 2023-09-10T09:32:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36805.871856 secs 2023-09-10T09:32:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:54 smithi170 ceph-mon[102832]: pgmap v19787: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 728 KiB/s wr, 13 op/s 2023-09-10T09:32:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36805.871856 secs 2023-09-10T09:32:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36806.224684 secs 2023-09-10T09:32:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36806.224684 secs 2023-09-10T09:32:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36806.224684 secs 2023-09-10T09:32:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:56 smithi082 ceph-mon[98694]: pgmap v19788: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 819 KiB/s wr, 18 op/s 2023-09-10T09:32:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:56 smithi098 ceph-mon[103124]: pgmap v19788: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 819 KiB/s wr, 18 op/s 2023-09-10T09:32:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:56 smithi170 ceph-mon[102832]: pgmap v19788: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 819 KiB/s wr, 18 op/s 2023-09-10T09:32:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:32:58 smithi170 ceph-mon[102832]: pgmap v19789: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 819 KiB/s wr, 16 op/s 2023-09-10T09:32:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:32:58 smithi082 ceph-mon[98694]: pgmap v19789: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 819 KiB/s wr, 16 op/s 2023-09-10T09:32:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:32:58 smithi098 ceph-mon[103124]: pgmap v19789: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 819 KiB/s wr, 16 op/s 2023-09-10T09:33:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36810.871917 secs 2023-09-10T09:33:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36811.224749 secs 2023-09-10T09:33:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:00 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:33:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:00 smithi170 ceph-mon[102832]: pgmap v19790: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 739 KiB/s wr, 13 op/s 2023-09-10T09:33:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36810.871917 secs 2023-09-10T09:33:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36811.224749 secs 2023-09-10T09:33:00.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:00 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:33:00.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:00 smithi082 ceph-mon[98694]: pgmap v19790: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 739 KiB/s wr, 13 op/s 2023-09-10T09:33:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36810.871917 secs 2023-09-10T09:33:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36811.224749 secs 2023-09-10T09:33:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:00 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:33:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:00 smithi098 ceph-mon[103124]: pgmap v19790: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 739 KiB/s wr, 13 op/s 2023-09-10T09:33:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:01 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:33:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:01 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:33:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:01 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:01 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:33:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:01 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:33:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:01 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:01 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:33:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:01 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:33:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:01 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:02 smithi170 ceph-mon[102832]: pgmap v19791: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:33:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:02 smithi082 ceph-mon[98694]: pgmap v19791: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:33:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:02 smithi098 ceph-mon[103124]: pgmap v19791: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:33:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36814.623954 secs 2023-09-10T09:33:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:04 smithi170 ceph-mon[102832]: osdmap e1305: 12 total, 12 up, 12 in 2023-09-10T09:33:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:04 smithi170 ceph-mon[102832]: pgmap v19793: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 553 KiB/s wr, 17 op/s 2023-09-10T09:33:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36814.623954 secs 2023-09-10T09:33:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:04 smithi082 ceph-mon[98694]: osdmap e1305: 12 total, 12 up, 12 in 2023-09-10T09:33:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:04 smithi082 ceph-mon[98694]: pgmap v19793: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 553 KiB/s wr, 17 op/s 2023-09-10T09:33:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36814.623954 secs 2023-09-10T09:33:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:04 smithi098 ceph-mon[103124]: osdmap e1305: 12 total, 12 up, 12 in 2023-09-10T09:33:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:04 smithi098 ceph-mon[103124]: pgmap v19793: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 553 KiB/s wr, 17 op/s 2023-09-10T09:33:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36815.871978 secs 2023-09-10T09:33:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36816.224814 secs 2023-09-10T09:33:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:33:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36815.871978 secs 2023-09-10T09:33:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36816.224814 secs 2023-09-10T09:33:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:33:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36815.871978 secs 2023-09-10T09:33:05.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36816.224814 secs 2023-09-10T09:33:05.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:33:05.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:33:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:06 smithi170 ceph-mon[102832]: pgmap v19794: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 821 KiB/s wr, 32 op/s 2023-09-10T09:33:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:33:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:06 smithi082 ceph-mon[98694]: pgmap v19794: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 821 KiB/s wr, 32 op/s 2023-09-10T09:33:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:33:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:06 smithi098 ceph-mon[103124]: pgmap v19794: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 821 KiB/s wr, 32 op/s 2023-09-10T09:33:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:08 smithi170 ceph-mon[102832]: pgmap v19795: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 821 KiB/s wr, 33 op/s 2023-09-10T09:33:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36819.624049 secs 2023-09-10T09:33:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:08 smithi082 ceph-mon[98694]: pgmap v19795: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 821 KiB/s wr, 33 op/s 2023-09-10T09:33:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36819.624049 secs 2023-09-10T09:33:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:08 smithi098 ceph-mon[103124]: pgmap v19795: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 821 KiB/s wr, 33 op/s 2023-09-10T09:33:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36819.624049 secs 2023-09-10T09:33:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36820.872036 secs 2023-09-10T09:33:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:09 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:33:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:09 smithi170 ceph-mon[102832]: osdmap e1306: 12 total, 12 up, 12 in 2023-09-10T09:33:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36820.872036 secs 2023-09-10T09:33:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:09 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:33:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:09 smithi082 ceph-mon[98694]: osdmap e1306: 12 total, 12 up, 12 in 2023-09-10T09:33:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36820.872036 secs 2023-09-10T09:33:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:09 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:33:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:09 smithi098 ceph-mon[103124]: osdmap e1306: 12 total, 12 up, 12 in 2023-09-10T09:33:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36821.224876 secs 2023-09-10T09:33:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:10 smithi170 ceph-mon[102832]: pgmap v19797: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 473 KiB/s wr, 29 op/s 2023-09-10T09:33:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36821.224876 secs 2023-09-10T09:33:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:10 smithi082 ceph-mon[98694]: pgmap v19797: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 473 KiB/s wr, 29 op/s 2023-09-10T09:33:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36821.224876 secs 2023-09-10T09:33:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:10 smithi098 ceph-mon[103124]: pgmap v19797: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 473 KiB/s wr, 29 op/s 2023-09-10T09:33:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:12 smithi170 ceph-mon[102832]: pgmap v19798: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 718 KiB/s wr, 38 op/s 2023-09-10T09:33:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:12 smithi082 ceph-mon[98694]: pgmap v19798: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 718 KiB/s wr, 38 op/s 2023-09-10T09:33:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:12 smithi098 ceph-mon[103124]: pgmap v19798: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 718 KiB/s wr, 38 op/s 2023-09-10T09:33:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36824.624160 secs 2023-09-10T09:33:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36824.624160 secs 2023-09-10T09:33:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36824.624160 secs 2023-09-10T09:33:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:14 smithi170 ceph-mon[102832]: pgmap v19799: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 591 KiB/s wr, 31 op/s 2023-09-10T09:33:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36825.872093 secs 2023-09-10T09:33:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:14 smithi082 ceph-mon[98694]: pgmap v19799: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 591 KiB/s wr, 31 op/s 2023-09-10T09:33:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36825.872093 secs 2023-09-10T09:33:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:14 smithi098 ceph-mon[103124]: pgmap v19799: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 591 KiB/s wr, 31 op/s 2023-09-10T09:33:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36825.872093 secs 2023-09-10T09:33:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36826.224942 secs 2023-09-10T09:33:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36826.224942 secs 2023-09-10T09:33:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36826.224942 secs 2023-09-10T09:33:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:16 smithi170 ceph-mon[102832]: pgmap v19800: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 222 KiB/s wr, 15 op/s 2023-09-10T09:33:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:16 smithi082 ceph-mon[98694]: pgmap v19800: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 222 KiB/s wr, 15 op/s 2023-09-10T09:33:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:16 smithi098 ceph-mon[103124]: pgmap v19800: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 222 KiB/s wr, 15 op/s 2023-09-10T09:33:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:18 smithi170 ceph-mon[102832]: pgmap v19801: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 222 KiB/s wr, 15 op/s 2023-09-10T09:33:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:33:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36829.624219 secs 2023-09-10T09:33:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:18 smithi082 ceph-mon[98694]: pgmap v19801: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 222 KiB/s wr, 15 op/s 2023-09-10T09:33:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:33:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36829.624219 secs 2023-09-10T09:33:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:18 smithi098 ceph-mon[103124]: pgmap v19801: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 222 KiB/s wr, 15 op/s 2023-09-10T09:33:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:33:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36829.624219 secs 2023-09-10T09:33:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36830.872152 secs 2023-09-10T09:33:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36831.225007 secs 2023-09-10T09:33:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:20 smithi170 ceph-mon[102832]: pgmap v19802: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 217 KiB/s wr, 14 op/s 2023-09-10T09:33:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36830.872152 secs 2023-09-10T09:33:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36831.225007 secs 2023-09-10T09:33:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:20 smithi082 ceph-mon[98694]: pgmap v19802: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 217 KiB/s wr, 14 op/s 2023-09-10T09:33:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36830.872152 secs 2023-09-10T09:33:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36831.225007 secs 2023-09-10T09:33:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:20 smithi098 ceph-mon[103124]: pgmap v19802: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 217 KiB/s wr, 14 op/s 2023-09-10T09:33:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:21 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:33:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:21 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:21 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:33:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:21 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:21 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:33:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:21 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:22 smithi170 ceph-mon[102832]: pgmap v19803: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 425 KiB/s wr, 18 op/s 2023-09-10T09:33:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:22 smithi082 ceph-mon[98694]: pgmap v19803: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 425 KiB/s wr, 18 op/s 2023-09-10T09:33:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:22 smithi098 ceph-mon[103124]: pgmap v19803: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 425 KiB/s wr, 18 op/s 2023-09-10T09:33:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36834.624287 secs 2023-09-10T09:33:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36834.624287 secs 2023-09-10T09:33:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36834.624287 secs 2023-09-10T09:33:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:24 smithi170 ceph-mon[102832]: pgmap v19804: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 248 KiB/s wr, 12 op/s 2023-09-10T09:33:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:24 smithi082 ceph-mon[98694]: pgmap v19804: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 248 KiB/s wr, 12 op/s 2023-09-10T09:33:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:24 smithi098 ceph-mon[103124]: pgmap v19804: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 248 KiB/s wr, 12 op/s 2023-09-10T09:33:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36835.872237 secs 2023-09-10T09:33:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36836.225071 secs 2023-09-10T09:33:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36835.872237 secs 2023-09-10T09:33:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36836.225071 secs 2023-09-10T09:33:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36835.872237 secs 2023-09-10T09:33:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36836.225071 secs 2023-09-10T09:33:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:26 smithi082 ceph-mon[98694]: pgmap v19805: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 248 KiB/s wr, 15 op/s 2023-09-10T09:33:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:26 smithi098 ceph-mon[103124]: pgmap v19805: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 248 KiB/s wr, 15 op/s 2023-09-10T09:33:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:26 smithi170 ceph-mon[102832]: pgmap v19805: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 248 KiB/s wr, 15 op/s 2023-09-10T09:33:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:28 smithi170 ceph-mon[102832]: pgmap v19806: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 240 KiB/s wr, 13 op/s 2023-09-10T09:33:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36839.624347 secs 2023-09-10T09:33:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:28 smithi082 ceph-mon[98694]: pgmap v19806: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 240 KiB/s wr, 13 op/s 2023-09-10T09:33:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36839.624347 secs 2023-09-10T09:33:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:28 smithi098 ceph-mon[103124]: pgmap v19806: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 240 KiB/s wr, 13 op/s 2023-09-10T09:33:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36839.624347 secs 2023-09-10T09:33:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36840.872296 secs 2023-09-10T09:33:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36841.225137 secs 2023-09-10T09:33:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:30 smithi170 ceph-mon[102832]: pgmap v19807: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 240 KiB/s wr, 11 op/s 2023-09-10T09:33:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:33:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36840.872296 secs 2023-09-10T09:33:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36841.225137 secs 2023-09-10T09:33:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:30 smithi082 ceph-mon[98694]: pgmap v19807: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 240 KiB/s wr, 11 op/s 2023-09-10T09:33:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:33:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36840.872296 secs 2023-09-10T09:33:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36841.225137 secs 2023-09-10T09:33:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:30 smithi098 ceph-mon[103124]: pgmap v19807: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 240 KiB/s wr, 11 op/s 2023-09-10T09:33:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:33:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:33:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:32 smithi082 ceph-mon[98694]: pgmap v19808: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 397 KiB/s wr, 17 op/s 2023-09-10T09:33:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:33:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:32 smithi098 ceph-mon[103124]: pgmap v19808: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 397 KiB/s wr, 17 op/s 2023-09-10T09:33:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:33:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:32 smithi170 ceph-mon[102832]: pgmap v19808: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 397 KiB/s wr, 17 op/s 2023-09-10T09:33:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:34 smithi170 ceph-mon[102832]: pgmap v19809: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 157 KiB/s wr, 11 op/s 2023-09-10T09:33:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36845.872357 secs 2023-09-10T09:33:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:34 smithi082 ceph-mon[98694]: pgmap v19809: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 157 KiB/s wr, 11 op/s 2023-09-10T09:33:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36845.872357 secs 2023-09-10T09:33:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:34 smithi098 ceph-mon[103124]: pgmap v19809: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 157 KiB/s wr, 11 op/s 2023-09-10T09:33:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36845.872357 secs 2023-09-10T09:33:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36846.225204 secs 2023-09-10T09:33:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:33:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36846.225204 secs 2023-09-10T09:33:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:33:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36846.225204 secs 2023-09-10T09:33:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:33:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:36 smithi170 ceph-mon[102832]: pgmap v19810: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 14 op/s 2023-09-10T09:33:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:33:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:33:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:36 smithi082 ceph-mon[98694]: pgmap v19810: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 14 op/s 2023-09-10T09:33:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:33:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:33:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:36 smithi098 ceph-mon[103124]: pgmap v19810: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 157 KiB/s wr, 14 op/s 2023-09-10T09:33:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:33:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:33:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:38 smithi170 ceph-mon[102832]: pgmap v19811: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 330 KiB/s wr, 13 op/s 2023-09-10T09:33:38.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:38 smithi082 ceph-mon[98694]: pgmap v19811: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 330 KiB/s wr, 13 op/s 2023-09-10T09:33:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:38 smithi098 ceph-mon[103124]: pgmap v19811: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 330 KiB/s wr, 13 op/s 2023-09-10T09:33:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36850.872423 secs 2023-09-10T09:33:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36851.225269 secs 2023-09-10T09:33:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:40 smithi170 ceph-mon[102832]: pgmap v19812: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 330 KiB/s wr, 11 op/s 2023-09-10T09:33:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36850.872423 secs 2023-09-10T09:33:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36851.225269 secs 2023-09-10T09:33:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:40 smithi082 ceph-mon[98694]: pgmap v19812: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 330 KiB/s wr, 11 op/s 2023-09-10T09:33:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36850.872423 secs 2023-09-10T09:33:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36851.225269 secs 2023-09-10T09:33:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:40 smithi098 ceph-mon[103124]: pgmap v19812: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 330 KiB/s wr, 11 op/s 2023-09-10T09:33:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:33:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:33:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:33:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:42 smithi170 ceph-mon[102832]: pgmap v19813: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 487 KiB/s wr, 17 op/s 2023-09-10T09:33:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:33:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:33:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:33:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:42 smithi082 ceph-mon[98694]: pgmap v19813: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 487 KiB/s wr, 17 op/s 2023-09-10T09:33:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:33:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:33:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:33:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:42 smithi098 ceph-mon[103124]: pgmap v19813: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 487 KiB/s wr, 17 op/s 2023-09-10T09:33:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:33:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:33:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:33:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:33:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:44 smithi082 ceph-mon[98694]: pgmap v19814: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 331 KiB/s wr, 11 op/s 2023-09-10T09:33:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:33:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:44 smithi098 ceph-mon[103124]: pgmap v19814: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 331 KiB/s wr, 11 op/s 2023-09-10T09:33:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:33:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:44 smithi170 ceph-mon[102832]: pgmap v19814: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 331 KiB/s wr, 11 op/s 2023-09-10T09:33:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36855.872476 secs 2023-09-10T09:33:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36856.225333 secs 2023-09-10T09:33:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36855.872476 secs 2023-09-10T09:33:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36856.225333 secs 2023-09-10T09:33:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36855.872476 secs 2023-09-10T09:33:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36856.225333 secs 2023-09-10T09:33:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:33:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:46 smithi082 ceph-mon[98694]: pgmap v19815: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 359 KiB/s wr, 34 op/s 2023-09-10T09:33:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:33:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:46 smithi098 ceph-mon[103124]: pgmap v19815: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 359 KiB/s wr, 34 op/s 2023-09-10T09:33:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:33:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:46 smithi170 ceph-mon[102832]: pgmap v19815: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 359 KiB/s wr, 34 op/s 2023-09-10T09:33:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:48 smithi170 ceph-mon[102832]: pgmap v19816: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 835 KiB/s wr, 35 op/s 2023-09-10T09:33:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36859.624612 secs 2023-09-10T09:33:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:48 smithi082 ceph-mon[98694]: pgmap v19816: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 835 KiB/s wr, 35 op/s 2023-09-10T09:33:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36859.624612 secs 2023-09-10T09:33:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:48 smithi098 ceph-mon[103124]: pgmap v19816: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 835 KiB/s wr, 35 op/s 2023-09-10T09:33:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36859.624612 secs 2023-09-10T09:33:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36860.872561 secs 2023-09-10T09:33:50.481 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36861.225399 secs 2023-09-10T09:33:50.481 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:50 smithi170 ceph-mon[102832]: pgmap v19817: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 662 KiB/s wr, 33 op/s 2023-09-10T09:33:50.482 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:33:50.482 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:50.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36860.872561 secs 2023-09-10T09:33:50.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36861.225399 secs 2023-09-10T09:33:50.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:50 smithi082 ceph-mon[98694]: pgmap v19817: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 662 KiB/s wr, 33 op/s 2023-09-10T09:33:50.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:33:50.484 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36860.872561 secs 2023-09-10T09:33:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36861.225399 secs 2023-09-10T09:33:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:50 smithi098 ceph-mon[103124]: pgmap v19817: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 662 KiB/s wr, 33 op/s 2023-09-10T09:33:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:33:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:33:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:33:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:33:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:33:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:51.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:33:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:33:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:52 smithi098 ceph-mon[103124]: pgmap v19818: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 40 op/s 2023-09-10T09:33:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:52 smithi170 ceph-mon[102832]: pgmap v19818: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 40 op/s 2023-09-10T09:33:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:52 smithi082 ceph-mon[98694]: pgmap v19818: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 40 op/s 2023-09-10T09:33:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36864.624671 secs 2023-09-10T09:33:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36864.624671 secs 2023-09-10T09:33:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36864.624671 secs 2023-09-10T09:33:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:54 smithi098 ceph-mon[103124]: pgmap v19819: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 953 KiB/s rd, 929 KiB/s wr, 34 op/s 2023-09-10T09:33:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36865.872647 secs 2023-09-10T09:33:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:54 smithi170 ceph-mon[102832]: pgmap v19819: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 953 KiB/s rd, 929 KiB/s wr, 34 op/s 2023-09-10T09:33:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36865.872647 secs 2023-09-10T09:33:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:54 smithi082 ceph-mon[98694]: pgmap v19819: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 953 KiB/s rd, 929 KiB/s wr, 34 op/s 2023-09-10T09:33:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36865.872647 secs 2023-09-10T09:33:55.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36866.225464 secs 2023-09-10T09:33:55.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36866.225464 secs 2023-09-10T09:33:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36866.225464 secs 2023-09-10T09:33:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:56 smithi098 ceph-mon[103124]: pgmap v19820: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 957 KiB/s wr, 38 op/s 2023-09-10T09:33:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:56 smithi170 ceph-mon[102832]: pgmap v19820: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 957 KiB/s wr, 38 op/s 2023-09-10T09:33:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:56 smithi082 ceph-mon[98694]: pgmap v19820: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 957 KiB/s wr, 38 op/s 2023-09-10T09:33:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:33:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:58 smithi170 ceph-mon[102832]: pgmap v19821: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1008 KiB/s wr, 17 op/s 2023-09-10T09:33:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:33:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36869.624789 secs 2023-09-10T09:33:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:33:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:58 smithi082 ceph-mon[98694]: pgmap v19821: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1008 KiB/s wr, 17 op/s 2023-09-10T09:33:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:33:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36869.624789 secs 2023-09-10T09:33:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:33:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:33:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:58 smithi098 ceph-mon[103124]: pgmap v19821: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1008 KiB/s wr, 17 op/s 2023-09-10T09:33:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:33:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36869.624789 secs 2023-09-10T09:34:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36870.872707 secs 2023-09-10T09:34:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36871.225491 secs 2023-09-10T09:34:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:00 smithi170 ceph-mon[102832]: pgmap v19822: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 531 KiB/s wr, 13 op/s 2023-09-10T09:34:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36870.872707 secs 2023-09-10T09:34:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36871.225491 secs 2023-09-10T09:34:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:00 smithi082 ceph-mon[98694]: pgmap v19822: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 531 KiB/s wr, 13 op/s 2023-09-10T09:34:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36870.872707 secs 2023-09-10T09:34:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36871.225491 secs 2023-09-10T09:34:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:00 smithi098 ceph-mon[103124]: pgmap v19822: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 531 KiB/s wr, 13 op/s 2023-09-10T09:34:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:01 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:34:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:01 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:01 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:34:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:01 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:01 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:34:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:01 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:02 smithi170 ceph-mon[102832]: pgmap v19823: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1018 KiB/s wr, 20 op/s 2023-09-10T09:34:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:02 smithi082 ceph-mon[98694]: pgmap v19823: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1018 KiB/s wr, 20 op/s 2023-09-10T09:34:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:02 smithi098 ceph-mon[103124]: pgmap v19823: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1018 KiB/s wr, 20 op/s 2023-09-10T09:34:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36874.624850 secs 2023-09-10T09:34:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:04 smithi170 ceph-mon[102832]: osdmap e1307: 12 total, 12 up, 12 in 2023-09-10T09:34:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:04 smithi170 ceph-mon[102832]: pgmap v19825: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 712 KiB/s wr, 16 op/s 2023-09-10T09:34:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36874.624850 secs 2023-09-10T09:34:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:04 smithi082 ceph-mon[98694]: osdmap e1307: 12 total, 12 up, 12 in 2023-09-10T09:34:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:04 smithi082 ceph-mon[98694]: pgmap v19825: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 712 KiB/s wr, 16 op/s 2023-09-10T09:34:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36874.624850 secs 2023-09-10T09:34:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:04 smithi098 ceph-mon[103124]: osdmap e1307: 12 total, 12 up, 12 in 2023-09-10T09:34:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:04 smithi098 ceph-mon[103124]: pgmap v19825: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 712 KiB/s wr, 16 op/s 2023-09-10T09:34:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36875.872795 secs 2023-09-10T09:34:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36876.225557 secs 2023-09-10T09:34:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36875.872795 secs 2023-09-10T09:34:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36876.225557 secs 2023-09-10T09:34:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36875.872795 secs 2023-09-10T09:34:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36876.225557 secs 2023-09-10T09:34:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:06 smithi170 ceph-mon[102832]: pgmap v19826: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 731 KiB/s wr, 33 op/s 2023-09-10T09:34:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:06 smithi082 ceph-mon[98694]: pgmap v19826: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 731 KiB/s wr, 33 op/s 2023-09-10T09:34:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:06 smithi098 ceph-mon[103124]: pgmap v19826: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 731 KiB/s wr, 33 op/s 2023-09-10T09:34:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:08 smithi170 ceph-mon[102832]: pgmap v19827: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1008 KiB/s wr, 34 op/s 2023-09-10T09:34:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36879.624911 secs 2023-09-10T09:34:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:08 smithi082 ceph-mon[98694]: pgmap v19827: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1008 KiB/s wr, 34 op/s 2023-09-10T09:34:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36879.624911 secs 2023-09-10T09:34:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:08 smithi098 ceph-mon[103124]: pgmap v19827: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1008 KiB/s wr, 34 op/s 2023-09-10T09:34:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36879.624911 secs 2023-09-10T09:34:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36880.872895 secs 2023-09-10T09:34:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36881.225622 secs 2023-09-10T09:34:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:10 smithi170 ceph-mon[102832]: osdmap e1308: 12 total, 12 up, 12 in 2023-09-10T09:34:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:10 smithi170 ceph-mon[102832]: pgmap v19829: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 530 KiB/s wr, 31 op/s 2023-09-10T09:34:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:34:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36880.872895 secs 2023-09-10T09:34:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36881.225622 secs 2023-09-10T09:34:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:10 smithi082 ceph-mon[98694]: osdmap e1308: 12 total, 12 up, 12 in 2023-09-10T09:34:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:10 smithi082 ceph-mon[98694]: pgmap v19829: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 530 KiB/s wr, 31 op/s 2023-09-10T09:34:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:34:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36880.872895 secs 2023-09-10T09:34:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36881.225622 secs 2023-09-10T09:34:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:10 smithi098 ceph-mon[103124]: osdmap e1308: 12 total, 12 up, 12 in 2023-09-10T09:34:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:10 smithi098 ceph-mon[103124]: pgmap v19829: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 530 KiB/s wr, 31 op/s 2023-09-10T09:34:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:34:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:34:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:12 smithi082 ceph-mon[98694]: pgmap v19830: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 711 KiB/s wr, 39 op/s 2023-09-10T09:34:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:34:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:12 smithi098 ceph-mon[103124]: pgmap v19830: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 711 KiB/s wr, 39 op/s 2023-09-10T09:34:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:34:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:12 smithi170 ceph-mon[102832]: pgmap v19830: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 711 KiB/s wr, 39 op/s 2023-09-10T09:34:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36884.625009 secs 2023-09-10T09:34:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36884.625009 secs 2023-09-10T09:34:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36884.625009 secs 2023-09-10T09:34:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:14 smithi082 ceph-mon[98694]: pgmap v19831: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 585 KiB/s wr, 32 op/s 2023-09-10T09:34:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:14 smithi098 ceph-mon[103124]: pgmap v19831: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 585 KiB/s wr, 32 op/s 2023-09-10T09:34:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:14 smithi170 ceph-mon[102832]: pgmap v19831: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 585 KiB/s wr, 32 op/s 2023-09-10T09:34:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36885.872956 secs 2023-09-10T09:34:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36886.225686 secs 2023-09-10T09:34:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36885.872956 secs 2023-09-10T09:34:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36886.225686 secs 2023-09-10T09:34:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36885.872956 secs 2023-09-10T09:34:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36886.225686 secs 2023-09-10T09:34:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:16 smithi082 ceph-mon[98694]: pgmap v19832: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 533 KiB/s wr, 15 op/s 2023-09-10T09:34:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:16 smithi098 ceph-mon[103124]: pgmap v19832: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 533 KiB/s wr, 15 op/s 2023-09-10T09:34:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:16 smithi170 ceph-mon[102832]: pgmap v19832: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 533 KiB/s wr, 15 op/s 2023-09-10T09:34:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:18 smithi170 ceph-mon[102832]: pgmap v19833: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 13 op/s 2023-09-10T09:34:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36889.625065 secs 2023-09-10T09:34:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:18 smithi082 ceph-mon[98694]: pgmap v19833: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 13 op/s 2023-09-10T09:34:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36889.625065 secs 2023-09-10T09:34:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:18 smithi098 ceph-mon[103124]: pgmap v19833: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 13 op/s 2023-09-10T09:34:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36889.625065 secs 2023-09-10T09:34:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36890.873038 secs 2023-09-10T09:34:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36891.225751 secs 2023-09-10T09:34:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:20 smithi170 ceph-mon[102832]: pgmap v19834: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 13 op/s 2023-09-10T09:34:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36890.873038 secs 2023-09-10T09:34:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36891.225751 secs 2023-09-10T09:34:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:20 smithi082 ceph-mon[98694]: pgmap v19834: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 13 op/s 2023-09-10T09:34:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36890.873038 secs 2023-09-10T09:34:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36891.225751 secs 2023-09-10T09:34:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:20 smithi098 ceph-mon[103124]: pgmap v19834: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 13 op/s 2023-09-10T09:34:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:22 smithi170 ceph-mon[102832]: pgmap v19835: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 135 KiB/s wr, 16 op/s 2023-09-10T09:34:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:22 smithi082 ceph-mon[98694]: pgmap v19835: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 135 KiB/s wr, 16 op/s 2023-09-10T09:34:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:22 smithi098 ceph-mon[103124]: pgmap v19835: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 135 KiB/s wr, 16 op/s 2023-09-10T09:34:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36894.625154 secs 2023-09-10T09:34:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36894.625154 secs 2023-09-10T09:34:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36894.625154 secs 2023-09-10T09:34:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:24 smithi170 ceph-mon[102832]: pgmap v19836: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-10T09:34:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36895.873097 secs 2023-09-10T09:34:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:24 smithi082 ceph-mon[98694]: pgmap v19836: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-10T09:34:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36895.873097 secs 2023-09-10T09:34:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:24 smithi098 ceph-mon[103124]: pgmap v19836: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-10T09:34:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36895.873097 secs 2023-09-10T09:34:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36896.225816 secs 2023-09-10T09:34:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36896.225816 secs 2023-09-10T09:34:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36896.225816 secs 2023-09-10T09:34:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:26 smithi170 ceph-mon[102832]: pgmap v19837: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2023-09-10T09:34:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:26 smithi082 ceph-mon[98694]: pgmap v19837: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2023-09-10T09:34:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:26 smithi098 ceph-mon[103124]: pgmap v19837: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2023-09-10T09:34:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:28 smithi170 ceph-mon[102832]: pgmap v19838: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 134 KiB/s wr, 13 op/s 2023-09-10T09:34:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36899.625215 secs 2023-09-10T09:34:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:28 smithi082 ceph-mon[98694]: pgmap v19838: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 134 KiB/s wr, 13 op/s 2023-09-10T09:34:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36899.625215 secs 2023-09-10T09:34:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:28 smithi098 ceph-mon[103124]: pgmap v19838: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 134 KiB/s wr, 13 op/s 2023-09-10T09:34:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36899.625215 secs 2023-09-10T09:34:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36900.873153 secs 2023-09-10T09:34:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36901.225883 secs 2023-09-10T09:34:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:30 smithi170 ceph-mon[102832]: pgmap v19839: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 10 op/s 2023-09-10T09:34:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36900.873153 secs 2023-09-10T09:34:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36901.225883 secs 2023-09-10T09:34:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:30 smithi082 ceph-mon[98694]: pgmap v19839: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 10 op/s 2023-09-10T09:34:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36900.873153 secs 2023-09-10T09:34:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36901.225883 secs 2023-09-10T09:34:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:30 smithi098 ceph-mon[103124]: pgmap v19839: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 10 op/s 2023-09-10T09:34:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:32 smithi170 ceph-mon[102832]: pgmap v19840: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 135 KiB/s wr, 15 op/s 2023-09-10T09:34:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:32 smithi082 ceph-mon[98694]: pgmap v19840: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 135 KiB/s wr, 15 op/s 2023-09-10T09:34:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:32 smithi098 ceph-mon[103124]: pgmap v19840: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 135 KiB/s wr, 15 op/s 2023-09-10T09:34:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:34 smithi170 ceph-mon[102832]: pgmap v19841: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 10 op/s 2023-09-10T09:34:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36905.873209 secs 2023-09-10T09:34:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:34 smithi082 ceph-mon[98694]: pgmap v19841: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 10 op/s 2023-09-10T09:34:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36905.873209 secs 2023-09-10T09:34:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:34 smithi098 ceph-mon[103124]: pgmap v19841: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 10 op/s 2023-09-10T09:34:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36905.873209 secs 2023-09-10T09:34:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36906.225971 secs 2023-09-10T09:34:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:34:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36906.225971 secs 2023-09-10T09:34:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:34:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36906.225971 secs 2023-09-10T09:34:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:34:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:36 smithi170 ceph-mon[102832]: pgmap v19842: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 134 KiB/s wr, 13 op/s 2023-09-10T09:34:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:34:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:34:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:36 smithi082 ceph-mon[98694]: pgmap v19842: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 134 KiB/s wr, 13 op/s 2023-09-10T09:34:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:34:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:34:36.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:36 smithi098 ceph-mon[103124]: pgmap v19842: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 134 KiB/s wr, 13 op/s 2023-09-10T09:34:36.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:34:36.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:34:36.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:38 smithi170 ceph-mon[102832]: pgmap v19843: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 421 KiB/s wr, 13 op/s 2023-09-10T09:34:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:34:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:38 smithi082 ceph-mon[98694]: pgmap v19843: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 421 KiB/s wr, 13 op/s 2023-09-10T09:34:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:34:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:38 smithi098 ceph-mon[103124]: pgmap v19843: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 421 KiB/s wr, 13 op/s 2023-09-10T09:34:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:34:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36910.873267 secs 2023-09-10T09:34:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36911.226039 secs 2023-09-10T09:34:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:40 smithi170 ceph-mon[102832]: pgmap v19844: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 287 KiB/s wr, 10 op/s 2023-09-10T09:34:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36910.873267 secs 2023-09-10T09:34:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36911.226039 secs 2023-09-10T09:34:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:40 smithi082 ceph-mon[98694]: pgmap v19844: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 287 KiB/s wr, 10 op/s 2023-09-10T09:34:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36910.873267 secs 2023-09-10T09:34:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36911.226039 secs 2023-09-10T09:34:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:40 smithi098 ceph-mon[103124]: pgmap v19844: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 287 KiB/s wr, 10 op/s 2023-09-10T09:34:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:42 smithi170 ceph-mon[102832]: pgmap v19845: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 446 KiB/s wr, 17 op/s 2023-09-10T09:34:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:34:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:34:42.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:34:42.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:42 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:34:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:42 smithi082 ceph-mon[98694]: pgmap v19845: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 446 KiB/s wr, 17 op/s 2023-09-10T09:34:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:34:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:34:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:34:42.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:42 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:34:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:42 smithi098 ceph-mon[103124]: pgmap v19845: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 446 KiB/s wr, 17 op/s 2023-09-10T09:34:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:34:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:34:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:34:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:42 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:34:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:44 smithi170 ceph-mon[102832]: pgmap v19846: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 446 KiB/s wr, 12 op/s 2023-09-10T09:34:44.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:34:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36915.873352 secs 2023-09-10T09:34:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:44 smithi082 ceph-mon[98694]: pgmap v19846: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 446 KiB/s wr, 12 op/s 2023-09-10T09:34:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:34:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36915.873352 secs 2023-09-10T09:34:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:44 smithi098 ceph-mon[103124]: pgmap v19846: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 446 KiB/s wr, 12 op/s 2023-09-10T09:34:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:34:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36915.873352 secs 2023-09-10T09:34:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36916.226104 secs 2023-09-10T09:34:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36916.226104 secs 2023-09-10T09:34:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36916.226104 secs 2023-09-10T09:34:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:46 smithi170 ceph-mon[102832]: pgmap v19847: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 520 KiB/s wr, 16 op/s 2023-09-10T09:34:46.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:34:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:46 smithi082 ceph-mon[98694]: pgmap v19847: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 520 KiB/s wr, 16 op/s 2023-09-10T09:34:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:34:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:46 smithi098 ceph-mon[103124]: pgmap v19847: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 520 KiB/s wr, 16 op/s 2023-09-10T09:34:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:34:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:48 smithi170 ceph-mon[102832]: pgmap v19848: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 520 KiB/s wr, 16 op/s 2023-09-10T09:34:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36919.625488 secs 2023-09-10T09:34:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:48 smithi082 ceph-mon[98694]: pgmap v19848: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 520 KiB/s wr, 16 op/s 2023-09-10T09:34:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36919.625488 secs 2023-09-10T09:34:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:48 smithi098 ceph-mon[103124]: pgmap v19848: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 520 KiB/s wr, 16 op/s 2023-09-10T09:34:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36919.625488 secs 2023-09-10T09:34:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36920.873410 secs 2023-09-10T09:34:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36921.226172 secs 2023-09-10T09:34:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:50 smithi170 ceph-mon[102832]: pgmap v19849: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 233 KiB/s wr, 13 op/s 2023-09-10T09:34:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:34:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36920.873410 secs 2023-09-10T09:34:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36921.226172 secs 2023-09-10T09:34:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:50 smithi082 ceph-mon[98694]: pgmap v19849: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 233 KiB/s wr, 13 op/s 2023-09-10T09:34:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:34:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36920.873410 secs 2023-09-10T09:34:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36921.226172 secs 2023-09-10T09:34:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:50 smithi098 ceph-mon[103124]: pgmap v19849: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 233 KiB/s wr, 13 op/s 2023-09-10T09:34:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:34:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:34:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:34:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:34:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:34:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:34:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:34:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:52 smithi098 ceph-mon[103124]: pgmap v19850: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 728 KiB/s wr, 21 op/s 2023-09-10T09:34:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:52 smithi170 ceph-mon[102832]: pgmap v19850: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 728 KiB/s wr, 21 op/s 2023-09-10T09:34:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:52 smithi082 ceph-mon[98694]: pgmap v19850: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 728 KiB/s wr, 21 op/s 2023-09-10T09:34:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:54 smithi170 ceph-mon[102832]: pgmap v19851: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 569 KiB/s wr, 14 op/s 2023-09-10T09:34:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36925.873470 secs 2023-09-10T09:34:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:54 smithi082 ceph-mon[98694]: pgmap v19851: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 569 KiB/s wr, 14 op/s 2023-09-10T09:34:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36925.873470 secs 2023-09-10T09:34:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:54 smithi098 ceph-mon[103124]: pgmap v19851: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 569 KiB/s wr, 14 op/s 2023-09-10T09:34:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36925.873470 secs 2023-09-10T09:34:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36926.226235 secs 2023-09-10T09:34:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:55 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:34:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36926.226235 secs 2023-09-10T09:34:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:55 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:34:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36926.226235 secs 2023-09-10T09:34:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:55 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:34:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:56 smithi170 ceph-mon[102832]: pgmap v19852: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 680 KiB/s wr, 18 op/s 2023-09-10T09:34:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:34:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:56 smithi082 ceph-mon[98694]: pgmap v19852: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 680 KiB/s wr, 18 op/s 2023-09-10T09:34:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:34:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:56 smithi098 ceph-mon[103124]: pgmap v19852: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 680 KiB/s wr, 18 op/s 2023-09-10T09:34:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:34:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:58 smithi170 ceph-mon[102832]: pgmap v19853: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 606 KiB/s wr, 16 op/s 2023-09-10T09:34:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:34:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:58 smithi082 ceph-mon[98694]: pgmap v19853: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 606 KiB/s wr, 16 op/s 2023-09-10T09:34:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:34:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:58 smithi098 ceph-mon[103124]: pgmap v19853: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 606 KiB/s wr, 16 op/s 2023-09-10T09:34:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:34:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:34:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36930.873493 secs 2023-09-10T09:34:59.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:34:59 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:34:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36930.873493 secs 2023-09-10T09:34:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:34:59 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:34:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36930.873493 secs 2023-09-10T09:34:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:34:59 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:35:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36931.226304 secs 2023-09-10T09:35:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:00 smithi170 ceph-mon[102832]: pgmap v19854: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 606 KiB/s wr, 13 op/s 2023-09-10T09:35:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:35:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:00.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36931.226304 secs 2023-09-10T09:35:00.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:00 smithi082 ceph-mon[98694]: pgmap v19854: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 606 KiB/s wr, 13 op/s 2023-09-10T09:35:00.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:35:00.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36931.226304 secs 2023-09-10T09:35:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:00 smithi098 ceph-mon[103124]: pgmap v19854: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 606 KiB/s wr, 13 op/s 2023-09-10T09:35:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:35:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:02 smithi170 ceph-mon[102832]: pgmap v19855: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 21 op/s 2023-09-10T09:35:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:02 smithi082 ceph-mon[98694]: pgmap v19855: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 21 op/s 2023-09-10T09:35:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:02 smithi098 ceph-mon[103124]: pgmap v19855: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 21 op/s 2023-09-10T09:35:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36934.625718 secs 2023-09-10T09:35:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36934.625718 secs 2023-09-10T09:35:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36934.625718 secs 2023-09-10T09:35:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:04 smithi170 ceph-mon[102832]: pgmap v19856: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 818 KiB/s wr, 13 op/s 2023-09-10T09:35:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:35:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36935.873552 secs 2023-09-10T09:35:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:35:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:04 smithi082 ceph-mon[98694]: pgmap v19856: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 818 KiB/s wr, 13 op/s 2023-09-10T09:35:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:35:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36935.873552 secs 2023-09-10T09:35:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:35:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:04 smithi098 ceph-mon[103124]: pgmap v19856: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 818 KiB/s wr, 13 op/s 2023-09-10T09:35:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:35:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36935.873552 secs 2023-09-10T09:35:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:35:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36936.226371 secs 2023-09-10T09:35:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36936.226371 secs 2023-09-10T09:35:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36936.226371 secs 2023-09-10T09:35:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:06 smithi170 ceph-mon[102832]: pgmap v19857: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 864 KiB/s wr, 32 op/s 2023-09-10T09:35:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:06 smithi082 ceph-mon[98694]: pgmap v19857: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 864 KiB/s wr, 32 op/s 2023-09-10T09:35:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:06 smithi098 ceph-mon[103124]: pgmap v19857: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 864 KiB/s wr, 32 op/s 2023-09-10T09:35:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:08 smithi170 ceph-mon[102832]: pgmap v19858: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 753 KiB/s wr, 30 op/s 2023-09-10T09:35:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36939.625824 secs 2023-09-10T09:35:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:08 smithi082 ceph-mon[98694]: pgmap v19858: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 753 KiB/s wr, 30 op/s 2023-09-10T09:35:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36939.625824 secs 2023-09-10T09:35:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:08 smithi098 ceph-mon[103124]: pgmap v19858: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 753 KiB/s wr, 30 op/s 2023-09-10T09:35:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36939.625824 secs 2023-09-10T09:35:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:09 smithi170 ceph-mon[102832]: osdmap e1309: 12 total, 12 up, 12 in 2023-09-10T09:35:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36940.873609 secs 2023-09-10T09:35:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:09 smithi082 ceph-mon[98694]: osdmap e1309: 12 total, 12 up, 12 in 2023-09-10T09:35:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36940.873609 secs 2023-09-10T09:35:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:09 smithi098 ceph-mon[103124]: osdmap e1309: 12 total, 12 up, 12 in 2023-09-10T09:35:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36940.873609 secs 2023-09-10T09:35:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36941.226434 secs 2023-09-10T09:35:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:10 smithi170 ceph-mon[102832]: pgmap v19860: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 903 KiB/s wr, 34 op/s 2023-09-10T09:35:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:35:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36941.226434 secs 2023-09-10T09:35:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:10 smithi082 ceph-mon[98694]: pgmap v19860: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 903 KiB/s wr, 34 op/s 2023-09-10T09:35:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:35:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36941.226434 secs 2023-09-10T09:35:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:10 smithi098 ceph-mon[103124]: pgmap v19860: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 903 KiB/s wr, 34 op/s 2023-09-10T09:35:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:35:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:12 smithi170 ceph-mon[102832]: pgmap v19861: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 435 KiB/s wr, 33 op/s 2023-09-10T09:35:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:35:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:12 smithi082 ceph-mon[98694]: pgmap v19861: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 435 KiB/s wr, 33 op/s 2023-09-10T09:35:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:35:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:12 smithi098 ceph-mon[103124]: pgmap v19861: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 435 KiB/s wr, 33 op/s 2023-09-10T09:35:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:35:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36944.625881 secs 2023-09-10T09:35:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36944.625881 secs 2023-09-10T09:35:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36944.625881 secs 2023-09-10T09:35:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:14 smithi170 ceph-mon[102832]: pgmap v19862: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 435 KiB/s wr, 33 op/s 2023-09-10T09:35:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36945.873670 secs 2023-09-10T09:35:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:14 smithi170 ceph-mon[102832]: osdmap e1310: 12 total, 12 up, 12 in 2023-09-10T09:35:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:14 smithi082 ceph-mon[98694]: pgmap v19862: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 435 KiB/s wr, 33 op/s 2023-09-10T09:35:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36945.873670 secs 2023-09-10T09:35:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:14 smithi082 ceph-mon[98694]: osdmap e1310: 12 total, 12 up, 12 in 2023-09-10T09:35:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:14 smithi098 ceph-mon[103124]: pgmap v19862: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 435 KiB/s wr, 33 op/s 2023-09-10T09:35:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36945.873670 secs 2023-09-10T09:35:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:14 smithi098 ceph-mon[103124]: osdmap e1310: 12 total, 12 up, 12 in 2023-09-10T09:35:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36946.226493 secs 2023-09-10T09:35:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36946.226493 secs 2023-09-10T09:35:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36946.226493 secs 2023-09-10T09:35:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:16 smithi082 ceph-mon[98694]: pgmap v19864: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 477 KiB/s wr, 15 op/s 2023-09-10T09:35:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:16 smithi098 ceph-mon[103124]: pgmap v19864: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 477 KiB/s wr, 15 op/s 2023-09-10T09:35:16.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:16 smithi170 ceph-mon[102832]: pgmap v19864: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 477 KiB/s wr, 15 op/s 2023-09-10T09:35:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:18 smithi082 ceph-mon[98694]: pgmap v19865: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 428 KiB/s wr, 16 op/s 2023-09-10T09:35:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36949.625940 secs 2023-09-10T09:35:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:18 smithi098 ceph-mon[103124]: pgmap v19865: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 428 KiB/s wr, 16 op/s 2023-09-10T09:35:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36949.625940 secs 2023-09-10T09:35:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:18 smithi170 ceph-mon[102832]: pgmap v19865: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 428 KiB/s wr, 16 op/s 2023-09-10T09:35:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36949.625940 secs 2023-09-10T09:35:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36950.873749 secs 2023-09-10T09:35:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36951.226558 secs 2023-09-10T09:35:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:20 smithi170 ceph-mon[102832]: pgmap v19866: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 382 KiB/s wr, 14 op/s 2023-09-10T09:35:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36950.873749 secs 2023-09-10T09:35:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36951.226558 secs 2023-09-10T09:35:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:20 smithi082 ceph-mon[98694]: pgmap v19866: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 382 KiB/s wr, 14 op/s 2023-09-10T09:35:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36950.873749 secs 2023-09-10T09:35:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36951.226558 secs 2023-09-10T09:35:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:20 smithi098 ceph-mon[103124]: pgmap v19866: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 382 KiB/s wr, 14 op/s 2023-09-10T09:35:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:22 smithi170 ceph-mon[102832]: pgmap v19867: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T09:35:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:22 smithi082 ceph-mon[98694]: pgmap v19867: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T09:35:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:22 smithi098 ceph-mon[103124]: pgmap v19867: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T09:35:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36954.626045 secs 2023-09-10T09:35:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36954.626045 secs 2023-09-10T09:35:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36954.626045 secs 2023-09-10T09:35:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:24 smithi170 ceph-mon[102832]: pgmap v19868: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T09:35:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:35:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36955.873808 secs 2023-09-10T09:35:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:24 smithi082 ceph-mon[98694]: pgmap v19868: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T09:35:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:35:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36955.873808 secs 2023-09-10T09:35:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:24 smithi098 ceph-mon[103124]: pgmap v19868: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 156 KiB/s wr, 12 op/s 2023-09-10T09:35:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:35:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36955.873808 secs 2023-09-10T09:35:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36956.226626 secs 2023-09-10T09:35:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36956.226626 secs 2023-09-10T09:35:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36956.226626 secs 2023-09-10T09:35:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:26 smithi170 ceph-mon[102832]: pgmap v19869: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 282 KiB/s wr, 15 op/s 2023-09-10T09:35:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:26 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:35:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:26 smithi082 ceph-mon[98694]: pgmap v19869: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 282 KiB/s wr, 15 op/s 2023-09-10T09:35:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:26 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:35:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:26 smithi098 ceph-mon[103124]: pgmap v19869: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 282 KiB/s wr, 15 op/s 2023-09-10T09:35:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:26 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:35:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:28 smithi170 ceph-mon[102832]: pgmap v19870: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 263 KiB/s wr, 13 op/s 2023-09-10T09:35:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36959.626145 secs 2023-09-10T09:35:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:28 smithi082 ceph-mon[98694]: pgmap v19870: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 263 KiB/s wr, 13 op/s 2023-09-10T09:35:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36959.626145 secs 2023-09-10T09:35:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:28 smithi098 ceph-mon[103124]: pgmap v19870: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 263 KiB/s wr, 13 op/s 2023-09-10T09:35:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36959.626145 secs 2023-09-10T09:35:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36960.873890 secs 2023-09-10T09:35:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36961.226692 secs 2023-09-10T09:35:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:30 smithi170 ceph-mon[102832]: pgmap v19871: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 263 KiB/s wr, 11 op/s 2023-09-10T09:35:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36960.873890 secs 2023-09-10T09:35:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36961.226692 secs 2023-09-10T09:35:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:30 smithi082 ceph-mon[98694]: pgmap v19871: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 263 KiB/s wr, 11 op/s 2023-09-10T09:35:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36960.873890 secs 2023-09-10T09:35:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36961.226692 secs 2023-09-10T09:35:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:30 smithi098 ceph-mon[103124]: pgmap v19871: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 263 KiB/s wr, 11 op/s 2023-09-10T09:35:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:32 smithi170 ceph-mon[102832]: pgmap v19872: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 16 op/s 2023-09-10T09:35:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:32 smithi082 ceph-mon[98694]: pgmap v19872: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 16 op/s 2023-09-10T09:35:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:32 smithi098 ceph-mon[103124]: pgmap v19872: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 16 op/s 2023-09-10T09:35:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:34 smithi170 ceph-mon[102832]: pgmap v19873: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 11 op/s 2023-09-10T09:35:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36965.873948 secs 2023-09-10T09:35:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:34 smithi082 ceph-mon[98694]: pgmap v19873: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 11 op/s 2023-09-10T09:35:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36965.873948 secs 2023-09-10T09:35:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:34 smithi098 ceph-mon[103124]: pgmap v19873: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 134 KiB/s wr, 11 op/s 2023-09-10T09:35:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36965.873948 secs 2023-09-10T09:35:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36966.226756 secs 2023-09-10T09:35:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:35:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36966.226756 secs 2023-09-10T09:35:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:35:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36966.226756 secs 2023-09-10T09:35:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:35:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:36 smithi170 ceph-mon[102832]: pgmap v19874: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 296 KiB/s wr, 15 op/s 2023-09-10T09:35:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:35:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:35:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:36 smithi082 ceph-mon[98694]: pgmap v19874: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 296 KiB/s wr, 15 op/s 2023-09-10T09:35:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:35:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:35:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:36 smithi098 ceph-mon[103124]: pgmap v19874: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 296 KiB/s wr, 15 op/s 2023-09-10T09:35:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:35:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:35:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:38 smithi170 ceph-mon[102832]: pgmap v19875: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 262 KiB/s wr, 13 op/s 2023-09-10T09:35:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:35:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:38 smithi082 ceph-mon[98694]: pgmap v19875: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 262 KiB/s wr, 13 op/s 2023-09-10T09:35:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:35:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:38 smithi098 ceph-mon[103124]: pgmap v19875: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 262 KiB/s wr, 13 op/s 2023-09-10T09:35:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:35:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36970.874036 secs 2023-09-10T09:35:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36971.226820 secs 2023-09-10T09:35:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:40 smithi170 ceph-mon[102832]: pgmap v19876: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 262 KiB/s wr, 11 op/s 2023-09-10T09:35:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36970.874036 secs 2023-09-10T09:35:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36971.226820 secs 2023-09-10T09:35:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:40 smithi082 ceph-mon[98694]: pgmap v19876: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 262 KiB/s wr, 11 op/s 2023-09-10T09:35:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36970.874036 secs 2023-09-10T09:35:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36971.226820 secs 2023-09-10T09:35:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:40 smithi098 ceph-mon[103124]: pgmap v19876: 97 pgs: 97 active+clean; 12 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 262 KiB/s wr, 11 op/s 2023-09-10T09:35:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:41 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:35:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:41 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:41 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:35:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:41 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:41 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:35:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:41 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:42 smithi170 ceph-mon[102832]: pgmap v19877: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 584 KiB/s wr, 19 op/s 2023-09-10T09:35:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:42 smithi082 ceph-mon[98694]: pgmap v19877: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 584 KiB/s wr, 19 op/s 2023-09-10T09:35:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:42 smithi098 ceph-mon[103124]: pgmap v19877: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 584 KiB/s wr, 19 op/s 2023-09-10T09:35:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:35:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:35:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:35:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:35:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:35:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:35:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:35:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:35:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:35:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:35:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:35:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:35:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:44 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36974.626348 secs 2023-09-10T09:35:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:44 smithi170 ceph-mon[102832]: pgmap v19878: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 584 KiB/s wr, 14 op/s 2023-09-10T09:35:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:44 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36974.626348 secs 2023-09-10T09:35:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:44 smithi082 ceph-mon[98694]: pgmap v19878: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 584 KiB/s wr, 14 op/s 2023-09-10T09:35:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:44 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36974.626348 secs 2023-09-10T09:35:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:44 smithi098 ceph-mon[103124]: pgmap v19878: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 584 KiB/s wr, 14 op/s 2023-09-10T09:35:45.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36975.874095 secs 2023-09-10T09:35:45.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36976.226886 secs 2023-09-10T09:35:45.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:45 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:35:45.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:35:45.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:45.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36975.874095 secs 2023-09-10T09:35:45.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36976.226886 secs 2023-09-10T09:35:45.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:45 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:35:45.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:35:45.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36975.874095 secs 2023-09-10T09:35:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36976.226886 secs 2023-09-10T09:35:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:45 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:35:45.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:35:45.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:46.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:35:46.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:46.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:46 smithi170 ceph-mon[102832]: pgmap v19879: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 609 KiB/s wr, 19 op/s 2023-09-10T09:35:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:35:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:46.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:46 smithi082 ceph-mon[98694]: pgmap v19879: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 609 KiB/s wr, 19 op/s 2023-09-10T09:35:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:35:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:46 smithi098 ceph-mon[103124]: pgmap v19879: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 609 KiB/s wr, 19 op/s 2023-09-10T09:35:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:48 smithi170 ceph-mon[102832]: pgmap v19880: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 447 KiB/s wr, 17 op/s 2023-09-10T09:35:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36979.626451 secs 2023-09-10T09:35:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:48 smithi082 ceph-mon[98694]: pgmap v19880: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 447 KiB/s wr, 17 op/s 2023-09-10T09:35:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36979.626451 secs 2023-09-10T09:35:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:48 smithi098 ceph-mon[103124]: pgmap v19880: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 447 KiB/s wr, 17 op/s 2023-09-10T09:35:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36979.626451 secs 2023-09-10T09:35:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36980.874170 secs 2023-09-10T09:35:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36981.226948 secs 2023-09-10T09:35:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:50 smithi170 ceph-mon[102832]: pgmap v19881: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 347 KiB/s wr, 15 op/s 2023-09-10T09:35:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:35:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36980.874170 secs 2023-09-10T09:35:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36981.226948 secs 2023-09-10T09:35:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:50 smithi082 ceph-mon[98694]: pgmap v19881: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 347 KiB/s wr, 15 op/s 2023-09-10T09:35:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:35:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36980.874170 secs 2023-09-10T09:35:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36981.226948 secs 2023-09-10T09:35:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:50 smithi098 ceph-mon[103124]: pgmap v19881: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 347 KiB/s wr, 15 op/s 2023-09-10T09:35:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:35:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:35:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:52 smithi082 ceph-mon[98694]: pgmap v19882: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 23 op/s 2023-09-10T09:35:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:35:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:52 smithi098 ceph-mon[103124]: pgmap v19882: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 23 op/s 2023-09-10T09:35:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:35:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:52 smithi170 ceph-mon[102832]: pgmap v19882: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 23 op/s 2023-09-10T09:35:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:54 smithi170 ceph-mon[102832]: pgmap v19883: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 754 KiB/s wr, 14 op/s 2023-09-10T09:35:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36985.874229 secs 2023-09-10T09:35:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:54 smithi082 ceph-mon[98694]: pgmap v19883: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 754 KiB/s wr, 14 op/s 2023-09-10T09:35:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36985.874229 secs 2023-09-10T09:35:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:54 smithi098 ceph-mon[103124]: pgmap v19883: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 754 KiB/s wr, 14 op/s 2023-09-10T09:35:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36985.874229 secs 2023-09-10T09:35:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36986.227014 secs 2023-09-10T09:35:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:55 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:35:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36986.227014 secs 2023-09-10T09:35:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:55 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:35:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36986.227014 secs 2023-09-10T09:35:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:55 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:35:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:56 smithi170 ceph-mon[102832]: pgmap v19884: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 782 KiB/s wr, 18 op/s 2023-09-10T09:35:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:56 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:35:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:35:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:56 smithi082 ceph-mon[98694]: pgmap v19884: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 782 KiB/s wr, 18 op/s 2023-09-10T09:35:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:56 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:35:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:35:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:56 smithi098 ceph-mon[103124]: pgmap v19884: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 782 KiB/s wr, 18 op/s 2023-09-10T09:35:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:56 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:35:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:35:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:58 smithi170 ceph-mon[102832]: pgmap v19885: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 933 KiB/s wr, 16 op/s 2023-09-10T09:35:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:35:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:35:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:58 smithi082 ceph-mon[98694]: pgmap v19885: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 933 KiB/s wr, 16 op/s 2023-09-10T09:35:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:35:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:35:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:35:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:58 smithi098 ceph-mon[103124]: pgmap v19885: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 933 KiB/s wr, 16 op/s 2023-09-10T09:35:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:35:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:35:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36990.874315 secs 2023-09-10T09:36:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36991.227079 secs 2023-09-10T09:36:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:00 smithi170 ceph-mon[102832]: pgmap v19886: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 933 KiB/s wr, 14 op/s 2023-09-10T09:36:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36990.874315 secs 2023-09-10T09:36:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36991.227079 secs 2023-09-10T09:36:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:00 smithi082 ceph-mon[98694]: pgmap v19886: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 933 KiB/s wr, 14 op/s 2023-09-10T09:36:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36990.874315 secs 2023-09-10T09:36:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36991.227079 secs 2023-09-10T09:36:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:00 smithi098 ceph-mon[103124]: pgmap v19886: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 933 KiB/s wr, 14 op/s 2023-09-10T09:36:02.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:02 smithi170 ceph-mon[102832]: pgmap v19887: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 20 op/s 2023-09-10T09:36:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:02 smithi082 ceph-mon[98694]: pgmap v19887: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 20 op/s 2023-09-10T09:36:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:02 smithi098 ceph-mon[103124]: pgmap v19887: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 20 op/s 2023-09-10T09:36:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36994.626719 secs 2023-09-10T09:36:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36994.626719 secs 2023-09-10T09:36:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36994.626719 secs 2023-09-10T09:36:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:04 smithi170 ceph-mon[102832]: pgmap v19888: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 317 KiB/s wr, 13 op/s 2023-09-10T09:36:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:36:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:36:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36995.874397 secs 2023-09-10T09:36:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:04 smithi082 ceph-mon[98694]: pgmap v19888: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 317 KiB/s wr, 13 op/s 2023-09-10T09:36:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:36:04.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:36:04.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:04.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36995.874397 secs 2023-09-10T09:36:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:04 smithi098 ceph-mon[103124]: pgmap v19888: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 317 KiB/s wr, 13 op/s 2023-09-10T09:36:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:36:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:36:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36995.874397 secs 2023-09-10T09:36:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 36996.227143 secs 2023-09-10T09:36:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 36996.227143 secs 2023-09-10T09:36:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 36996.227143 secs 2023-09-10T09:36:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:06 smithi170 ceph-mon[102832]: pgmap v19889: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 639 KiB/s wr, 31 op/s 2023-09-10T09:36:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:36:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:06 smithi082 ceph-mon[98694]: pgmap v19889: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 639 KiB/s wr, 31 op/s 2023-09-10T09:36:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:36:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:06 smithi098 ceph-mon[103124]: pgmap v19889: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 639 KiB/s wr, 31 op/s 2023-09-10T09:36:06.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:36:06.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:08 smithi170 ceph-mon[102832]: pgmap v19890: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 624 KiB/s wr, 29 op/s 2023-09-10T09:36:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 36999.626799 secs 2023-09-10T09:36:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:08 smithi082 ceph-mon[98694]: pgmap v19890: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 624 KiB/s wr, 29 op/s 2023-09-10T09:36:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 36999.626799 secs 2023-09-10T09:36:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:08 smithi098 ceph-mon[103124]: pgmap v19890: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 624 KiB/s wr, 29 op/s 2023-09-10T09:36:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 36999.626799 secs 2023-09-10T09:36:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:09 smithi170 ceph-mon[102832]: osdmap e1311: 12 total, 12 up, 12 in 2023-09-10T09:36:09.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37000.874459 secs 2023-09-10T09:36:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:09 smithi082 ceph-mon[98694]: osdmap e1311: 12 total, 12 up, 12 in 2023-09-10T09:36:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37000.874459 secs 2023-09-10T09:36:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:09 smithi098 ceph-mon[103124]: osdmap e1311: 12 total, 12 up, 12 in 2023-09-10T09:36:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37000.874459 secs 2023-09-10T09:36:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37001.227208 secs 2023-09-10T09:36:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:10 smithi170 ceph-mon[102832]: pgmap v19892: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 538 KiB/s wr, 32 op/s 2023-09-10T09:36:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37001.227208 secs 2023-09-10T09:36:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:10 smithi082 ceph-mon[98694]: pgmap v19892: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 538 KiB/s wr, 32 op/s 2023-09-10T09:36:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37001.227208 secs 2023-09-10T09:36:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:10 smithi098 ceph-mon[103124]: pgmap v19892: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 538 KiB/s wr, 32 op/s 2023-09-10T09:36:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:12 smithi170 ceph-mon[102832]: pgmap v19893: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 564 KiB/s wr, 32 op/s 2023-09-10T09:36:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:12 smithi082 ceph-mon[98694]: pgmap v19893: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 564 KiB/s wr, 32 op/s 2023-09-10T09:36:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:12 smithi098 ceph-mon[103124]: pgmap v19893: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 564 KiB/s wr, 32 op/s 2023-09-10T09:36:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37004.626858 secs 2023-09-10T09:36:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37004.626858 secs 2023-09-10T09:36:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37004.626858 secs 2023-09-10T09:36:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:14 smithi170 ceph-mon[102832]: pgmap v19894: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 564 KiB/s wr, 32 op/s 2023-09-10T09:36:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37005.874515 secs 2023-09-10T09:36:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:14 smithi170 ceph-mon[102832]: osdmap e1312: 12 total, 12 up, 12 in 2023-09-10T09:36:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:14 smithi082 ceph-mon[98694]: pgmap v19894: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 564 KiB/s wr, 32 op/s 2023-09-10T09:36:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37005.874515 secs 2023-09-10T09:36:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:14 smithi082 ceph-mon[98694]: osdmap e1312: 12 total, 12 up, 12 in 2023-09-10T09:36:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:14 smithi098 ceph-mon[103124]: pgmap v19894: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 564 KiB/s wr, 32 op/s 2023-09-10T09:36:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37005.874515 secs 2023-09-10T09:36:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:14 smithi098 ceph-mon[103124]: osdmap e1312: 12 total, 12 up, 12 in 2023-09-10T09:36:15.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37006.227272 secs 2023-09-10T09:36:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37006.227272 secs 2023-09-10T09:36:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37006.227272 secs 2023-09-10T09:36:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:16 smithi170 ceph-mon[102832]: pgmap v19896: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 244 KiB/s wr, 15 op/s 2023-09-10T09:36:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:36:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:16 smithi082 ceph-mon[98694]: pgmap v19896: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 244 KiB/s wr, 15 op/s 2023-09-10T09:36:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:36:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:16 smithi098 ceph-mon[103124]: pgmap v19896: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 244 KiB/s wr, 15 op/s 2023-09-10T09:36:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:36:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:18.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:18 smithi170 ceph-mon[102832]: pgmap v19897: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 219 KiB/s wr, 16 op/s 2023-09-10T09:36:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:36:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37009.626917 secs 2023-09-10T09:36:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:18 smithi082 ceph-mon[98694]: pgmap v19897: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 219 KiB/s wr, 16 op/s 2023-09-10T09:36:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:36:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:18.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37009.626917 secs 2023-09-10T09:36:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:18 smithi098 ceph-mon[103124]: pgmap v19897: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 219 KiB/s wr, 16 op/s 2023-09-10T09:36:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:36:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37009.626917 secs 2023-09-10T09:36:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37010.874572 secs 2023-09-10T09:36:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37011.227338 secs 2023-09-10T09:36:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:20 smithi170 ceph-mon[102832]: pgmap v19898: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 14 op/s 2023-09-10T09:36:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37010.874572 secs 2023-09-10T09:36:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37011.227338 secs 2023-09-10T09:36:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:20 smithi082 ceph-mon[98694]: pgmap v19898: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 14 op/s 2023-09-10T09:36:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37010.874572 secs 2023-09-10T09:36:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37011.227338 secs 2023-09-10T09:36:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:20 smithi098 ceph-mon[103124]: pgmap v19898: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 14 op/s 2023-09-10T09:36:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:22 smithi170 ceph-mon[102832]: pgmap v19899: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 14 op/s 2023-09-10T09:36:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:22 smithi082 ceph-mon[98694]: pgmap v19899: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 14 op/s 2023-09-10T09:36:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:22 smithi098 ceph-mon[103124]: pgmap v19899: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 14 op/s 2023-09-10T09:36:23.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37014.626976 secs 2023-09-10T09:36:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37014.626976 secs 2023-09-10T09:36:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37014.626976 secs 2023-09-10T09:36:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:24 smithi170 ceph-mon[102832]: pgmap v19900: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 14 op/s 2023-09-10T09:36:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37015.874632 secs 2023-09-10T09:36:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:24 smithi082 ceph-mon[98694]: pgmap v19900: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 14 op/s 2023-09-10T09:36:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37015.874632 secs 2023-09-10T09:36:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:24 smithi098 ceph-mon[103124]: pgmap v19900: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 195 KiB/s wr, 14 op/s 2023-09-10T09:36:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37015.874632 secs 2023-09-10T09:36:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37016.227404 secs 2023-09-10T09:36:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37016.227404 secs 2023-09-10T09:36:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37016.227404 secs 2023-09-10T09:36:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:26 smithi170 ceph-mon[102832]: pgmap v19901: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 173 KiB/s wr, 16 op/s 2023-09-10T09:36:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:26 smithi082 ceph-mon[98694]: pgmap v19901: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 173 KiB/s wr, 16 op/s 2023-09-10T09:36:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:26 smithi098 ceph-mon[103124]: pgmap v19901: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 173 KiB/s wr, 16 op/s 2023-09-10T09:36:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:28 smithi170 ceph-mon[102832]: pgmap v19902: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 135 KiB/s wr, 13 op/s 2023-09-10T09:36:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37019.627052 secs 2023-09-10T09:36:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:28 smithi082 ceph-mon[98694]: pgmap v19902: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 135 KiB/s wr, 13 op/s 2023-09-10T09:36:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37019.627052 secs 2023-09-10T09:36:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:28 smithi098 ceph-mon[103124]: pgmap v19902: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 135 KiB/s wr, 13 op/s 2023-09-10T09:36:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37019.627052 secs 2023-09-10T09:36:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37020.874720 secs 2023-09-10T09:36:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37021.227474 secs 2023-09-10T09:36:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:30 smithi170 ceph-mon[102832]: pgmap v19903: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T09:36:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37020.874720 secs 2023-09-10T09:36:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37021.227474 secs 2023-09-10T09:36:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:30 smithi082 ceph-mon[98694]: pgmap v19903: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T09:36:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37020.874720 secs 2023-09-10T09:36:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37021.227474 secs 2023-09-10T09:36:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:30 smithi098 ceph-mon[103124]: pgmap v19903: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 135 KiB/s wr, 11 op/s 2023-09-10T09:36:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:32 smithi170 ceph-mon[102832]: pgmap v19904: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 16 op/s 2023-09-10T09:36:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:32 smithi082 ceph-mon[98694]: pgmap v19904: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 16 op/s 2023-09-10T09:36:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:32 smithi098 ceph-mon[103124]: pgmap v19904: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 16 op/s 2023-09-10T09:36:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:34 smithi170 ceph-mon[102832]: pgmap v19905: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 10 op/s 2023-09-10T09:36:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37025.874780 secs 2023-09-10T09:36:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:34 smithi082 ceph-mon[98694]: pgmap v19905: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 10 op/s 2023-09-10T09:36:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37025.874780 secs 2023-09-10T09:36:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:34 smithi098 ceph-mon[103124]: pgmap v19905: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 10 op/s 2023-09-10T09:36:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37025.874780 secs 2023-09-10T09:36:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37026.227490 secs 2023-09-10T09:36:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:36:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37026.227490 secs 2023-09-10T09:36:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:36:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37026.227490 secs 2023-09-10T09:36:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:36:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:36 smithi170 ceph-mon[102832]: pgmap v19906: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 332 KiB/s wr, 14 op/s 2023-09-10T09:36:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:36:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:36:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:36 smithi082 ceph-mon[98694]: pgmap v19906: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 332 KiB/s wr, 14 op/s 2023-09-10T09:36:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:36:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:36:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:36 smithi098 ceph-mon[103124]: pgmap v19906: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 332 KiB/s wr, 14 op/s 2023-09-10T09:36:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:36:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:36:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:38 smithi170 ceph-mon[102832]: pgmap v19907: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 331 KiB/s wr, 12 op/s 2023-09-10T09:36:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:36:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:38 smithi082 ceph-mon[98694]: pgmap v19907: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 331 KiB/s wr, 12 op/s 2023-09-10T09:36:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:36:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:38 smithi098 ceph-mon[103124]: pgmap v19907: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 331 KiB/s wr, 12 op/s 2023-09-10T09:36:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:36:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37030.874838 secs 2023-09-10T09:36:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37031.227556 secs 2023-09-10T09:36:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:40 smithi170 ceph-mon[102832]: pgmap v19908: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 331 KiB/s wr, 11 op/s 2023-09-10T09:36:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37030.874838 secs 2023-09-10T09:36:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37031.227556 secs 2023-09-10T09:36:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:40 smithi082 ceph-mon[98694]: pgmap v19908: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 331 KiB/s wr, 11 op/s 2023-09-10T09:36:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37030.874838 secs 2023-09-10T09:36:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37031.227556 secs 2023-09-10T09:36:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:40 smithi098 ceph-mon[103124]: pgmap v19908: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 331 KiB/s wr, 11 op/s 2023-09-10T09:36:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:41 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:36:41.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:41 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:41 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:36:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:41 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:41 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:36:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:41 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:42.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:42 smithi170 ceph-mon[102832]: pgmap v19909: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 490 KiB/s wr, 17 op/s 2023-09-10T09:36:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:42 smithi082 ceph-mon[98694]: pgmap v19909: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 490 KiB/s wr, 17 op/s 2023-09-10T09:36:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:42 smithi098 ceph-mon[103124]: pgmap v19909: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 490 KiB/s wr, 17 op/s 2023-09-10T09:36:43.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:43 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:36:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:43 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:36:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:43 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:36:44.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:44 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37034.627304 secs 2023-09-10T09:36:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:44 smithi170 ceph-mon[102832]: pgmap v19910: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T09:36:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:36:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:36:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:36:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:44 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37034.627304 secs 2023-09-10T09:36:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:44 smithi082 ceph-mon[98694]: pgmap v19910: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T09:36:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:36:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:36:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:36:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:44 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37034.627304 secs 2023-09-10T09:36:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:44 smithi098 ceph-mon[103124]: pgmap v19910: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T09:36:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:36:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:36:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:36:45.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37035.874898 secs 2023-09-10T09:36:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37036.227621 secs 2023-09-10T09:36:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:45 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:36:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:36:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37035.874898 secs 2023-09-10T09:36:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37036.227621 secs 2023-09-10T09:36:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:45 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:36:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:36:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37035.874898 secs 2023-09-10T09:36:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37036.227621 secs 2023-09-10T09:36:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:45 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:36:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:36:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:46.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:36:46.554 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:46.555 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:46 smithi170 ceph-mon[102832]: pgmap v19911: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 840 KiB/s wr, 37 op/s 2023-09-10T09:36:46.556 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:36:46.556 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:46.556 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:46 smithi098 ceph-mon[103124]: pgmap v19911: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 840 KiB/s wr, 37 op/s 2023-09-10T09:36:46.556 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:36:46.556 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:46.557 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:46 smithi082 ceph-mon[98694]: pgmap v19911: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 840 KiB/s wr, 37 op/s 2023-09-10T09:36:48.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:48 smithi170 ceph-mon[102832]: pgmap v19912: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 667 KiB/s wr, 33 op/s 2023-09-10T09:36:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37039.627388 secs 2023-09-10T09:36:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:48 smithi082 ceph-mon[98694]: pgmap v19912: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 667 KiB/s wr, 33 op/s 2023-09-10T09:36:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37039.627388 secs 2023-09-10T09:36:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:48 smithi098 ceph-mon[103124]: pgmap v19912: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 667 KiB/s wr, 33 op/s 2023-09-10T09:36:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37039.627388 secs 2023-09-10T09:36:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37040.874972 secs 2023-09-10T09:36:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37041.227686 secs 2023-09-10T09:36:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:50 smithi170 ceph-mon[102832]: pgmap v19913: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 667 KiB/s wr, 31 op/s 2023-09-10T09:36:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:36:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37040.874972 secs 2023-09-10T09:36:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37041.227686 secs 2023-09-10T09:36:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:50 smithi082 ceph-mon[98694]: pgmap v19913: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 667 KiB/s wr, 31 op/s 2023-09-10T09:36:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:36:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37040.874972 secs 2023-09-10T09:36:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37041.227686 secs 2023-09-10T09:36:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:50 smithi098 ceph-mon[103124]: pgmap v19913: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 667 KiB/s wr, 31 op/s 2023-09-10T09:36:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:36:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:36:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:52 smithi082 ceph-mon[98694]: pgmap v19914: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 39 op/s 2023-09-10T09:36:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:36:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:52 smithi098 ceph-mon[103124]: pgmap v19914: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 39 op/s 2023-09-10T09:36:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:36:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:52 smithi170 ceph-mon[102832]: pgmap v19914: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 39 op/s 2023-09-10T09:36:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37044.627455 secs 2023-09-10T09:36:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37044.627455 secs 2023-09-10T09:36:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37044.627455 secs 2023-09-10T09:36:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:36:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:54 smithi082 ceph-mon[98694]: pgmap v19915: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 915 KiB/s wr, 32 op/s 2023-09-10T09:36:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:36:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:54 smithi098 ceph-mon[103124]: pgmap v19915: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 915 KiB/s wr, 32 op/s 2023-09-10T09:36:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:36:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:54 smithi170 ceph-mon[102832]: pgmap v19915: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 915 KiB/s wr, 32 op/s 2023-09-10T09:36:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37045.875032 secs 2023-09-10T09:36:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37046.227752 secs 2023-09-10T09:36:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37045.875032 secs 2023-09-10T09:36:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37046.227752 secs 2023-09-10T09:36:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37045.875032 secs 2023-09-10T09:36:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37046.227752 secs 2023-09-10T09:36:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:36:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:56 smithi098 ceph-mon[103124]: pgmap v19916: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 944 KiB/s wr, 38 op/s 2023-09-10T09:36:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:36:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:56 smithi170 ceph-mon[102832]: pgmap v19916: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 944 KiB/s wr, 38 op/s 2023-09-10T09:36:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:36:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:36:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:56 smithi082 ceph-mon[98694]: pgmap v19916: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 944 KiB/s wr, 38 op/s 2023-09-10T09:36:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:36:58 smithi170 ceph-mon[102832]: pgmap v19917: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 525 KiB/s wr, 14 op/s 2023-09-10T09:36:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:36:58 smithi082 ceph-mon[98694]: pgmap v19917: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 525 KiB/s wr, 14 op/s 2023-09-10T09:36:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:36:58 smithi098 ceph-mon[103124]: pgmap v19917: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 525 KiB/s wr, 14 op/s 2023-09-10T09:37:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37050.875090 secs 2023-09-10T09:37:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37051.227817 secs 2023-09-10T09:37:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:00 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:37:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:00 smithi170 ceph-mon[102832]: pgmap v19918: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 525 KiB/s wr, 14 op/s 2023-09-10T09:37:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37050.875090 secs 2023-09-10T09:37:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37051.227817 secs 2023-09-10T09:37:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:00 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:37:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:00 smithi082 ceph-mon[98694]: pgmap v19918: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 525 KiB/s wr, 14 op/s 2023-09-10T09:37:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37050.875090 secs 2023-09-10T09:37:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37051.227817 secs 2023-09-10T09:37:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:00 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:37:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:00 smithi098 ceph-mon[103124]: pgmap v19918: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 525 KiB/s wr, 14 op/s 2023-09-10T09:37:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:01 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:37:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:01 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:37:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:01 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:01 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:37:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:01 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:37:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:01 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:01 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:37:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:01 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:37:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:01 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:02 smithi170 ceph-mon[102832]: pgmap v19919: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 746 KiB/s wr, 21 op/s 2023-09-10T09:37:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:02 smithi082 ceph-mon[98694]: pgmap v19919: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 746 KiB/s wr, 21 op/s 2023-09-10T09:37:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:02 smithi098 ceph-mon[103124]: pgmap v19919: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 746 KiB/s wr, 21 op/s 2023-09-10T09:37:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37054.627627 secs 2023-09-10T09:37:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37054.627627 secs 2023-09-10T09:37:03.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37054.627627 secs 2023-09-10T09:37:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:37:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:04 smithi082 ceph-mon[98694]: pgmap v19920: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 338 KiB/s wr, 13 op/s 2023-09-10T09:37:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:37:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:04 smithi098 ceph-mon[103124]: pgmap v19920: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 338 KiB/s wr, 13 op/s 2023-09-10T09:37:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:37:04.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:04 smithi170 ceph-mon[102832]: pgmap v19920: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 338 KiB/s wr, 13 op/s 2023-09-10T09:37:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37055.875147 secs 2023-09-10T09:37:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37056.227886 secs 2023-09-10T09:37:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37055.875147 secs 2023-09-10T09:37:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37056.227886 secs 2023-09-10T09:37:05.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37055.875147 secs 2023-09-10T09:37:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37056.227886 secs 2023-09-10T09:37:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:37:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:06 smithi082 ceph-mon[98694]: pgmap v19921: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 730 KiB/s wr, 35 op/s 2023-09-10T09:37:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:37:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:06 smithi098 ceph-mon[103124]: pgmap v19921: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 730 KiB/s wr, 35 op/s 2023-09-10T09:37:06.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:37:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:06 smithi170 ceph-mon[102832]: pgmap v19921: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 730 KiB/s wr, 35 op/s 2023-09-10T09:37:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:08 smithi170 ceph-mon[102832]: pgmap v19922: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 702 KiB/s wr, 29 op/s 2023-09-10T09:37:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37059.627747 secs 2023-09-10T09:37:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:08 smithi082 ceph-mon[98694]: pgmap v19922: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 702 KiB/s wr, 29 op/s 2023-09-10T09:37:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37059.627747 secs 2023-09-10T09:37:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:08 smithi098 ceph-mon[103124]: pgmap v19922: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 702 KiB/s wr, 29 op/s 2023-09-10T09:37:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37059.627747 secs 2023-09-10T09:37:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:09 smithi170 ceph-mon[102832]: osdmap e1313: 12 total, 12 up, 12 in 2023-09-10T09:37:09.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37060.875238 secs 2023-09-10T09:37:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:09 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:37:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:09 smithi082 ceph-mon[98694]: osdmap e1313: 12 total, 12 up, 12 in 2023-09-10T09:37:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37060.875238 secs 2023-09-10T09:37:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:09 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:37:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:09 smithi098 ceph-mon[103124]: osdmap e1313: 12 total, 12 up, 12 in 2023-09-10T09:37:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37060.875238 secs 2023-09-10T09:37:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:09 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:37:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37061.227949 secs 2023-09-10T09:37:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:10 smithi170 ceph-mon[102832]: pgmap v19924: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 735 KiB/s wr, 34 op/s 2023-09-10T09:37:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:37:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37061.227949 secs 2023-09-10T09:37:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:10 smithi082 ceph-mon[98694]: pgmap v19924: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 735 KiB/s wr, 34 op/s 2023-09-10T09:37:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:37:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37061.227949 secs 2023-09-10T09:37:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:10 smithi098 ceph-mon[103124]: pgmap v19924: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 735 KiB/s wr, 34 op/s 2023-09-10T09:37:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:37:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:12 smithi170 ceph-mon[102832]: pgmap v19925: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 617 KiB/s wr, 34 op/s 2023-09-10T09:37:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:37:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:12 smithi082 ceph-mon[98694]: pgmap v19925: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 617 KiB/s wr, 34 op/s 2023-09-10T09:37:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:37:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:12 smithi098 ceph-mon[103124]: pgmap v19925: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 617 KiB/s wr, 34 op/s 2023-09-10T09:37:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:37:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37064.627827 secs 2023-09-10T09:37:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37064.627827 secs 2023-09-10T09:37:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37064.627827 secs 2023-09-10T09:37:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:14 smithi082 ceph-mon[98694]: pgmap v19926: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 617 KiB/s wr, 34 op/s 2023-09-10T09:37:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37065.875296 secs 2023-09-10T09:37:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:14 smithi082 ceph-mon[98694]: osdmap e1314: 12 total, 12 up, 12 in 2023-09-10T09:37:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:14 smithi098 ceph-mon[103124]: pgmap v19926: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 617 KiB/s wr, 34 op/s 2023-09-10T09:37:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37065.875296 secs 2023-09-10T09:37:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:14 smithi098 ceph-mon[103124]: osdmap e1314: 12 total, 12 up, 12 in 2023-09-10T09:37:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:14 smithi170 ceph-mon[102832]: pgmap v19926: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 617 KiB/s wr, 34 op/s 2023-09-10T09:37:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37065.875296 secs 2023-09-10T09:37:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:14 smithi170 ceph-mon[102832]: osdmap e1314: 12 total, 12 up, 12 in 2023-09-10T09:37:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37066.228016 secs 2023-09-10T09:37:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37066.228016 secs 2023-09-10T09:37:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37066.228016 secs 2023-09-10T09:37:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:16 smithi082 ceph-mon[98694]: pgmap v19928: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 185 KiB/s wr, 18 op/s 2023-09-10T09:37:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:16 smithi098 ceph-mon[103124]: pgmap v19928: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 185 KiB/s wr, 18 op/s 2023-09-10T09:37:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:16 smithi170 ceph-mon[102832]: pgmap v19928: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 185 KiB/s wr, 18 op/s 2023-09-10T09:37:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:18 smithi082 ceph-mon[98694]: pgmap v19929: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 165 KiB/s wr, 16 op/s 2023-09-10T09:37:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37069.627937 secs 2023-09-10T09:37:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:18 smithi098 ceph-mon[103124]: pgmap v19929: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 165 KiB/s wr, 16 op/s 2023-09-10T09:37:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37069.627937 secs 2023-09-10T09:37:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:18 smithi170 ceph-mon[102832]: pgmap v19929: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 165 KiB/s wr, 16 op/s 2023-09-10T09:37:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37069.627937 secs 2023-09-10T09:37:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37070.875349 secs 2023-09-10T09:37:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37071.228082 secs 2023-09-10T09:37:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:20 smithi170 ceph-mon[102832]: pgmap v19930: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 148 KiB/s wr, 15 op/s 2023-09-10T09:37:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37070.875349 secs 2023-09-10T09:37:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37071.228082 secs 2023-09-10T09:37:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:20 smithi082 ceph-mon[98694]: pgmap v19930: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 148 KiB/s wr, 15 op/s 2023-09-10T09:37:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37070.875349 secs 2023-09-10T09:37:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37071.228082 secs 2023-09-10T09:37:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:20 smithi098 ceph-mon[103124]: pgmap v19930: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 148 KiB/s wr, 15 op/s 2023-09-10T09:37:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:22 smithi170 ceph-mon[102832]: pgmap v19931: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-09-10T09:37:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:22 smithi082 ceph-mon[98694]: pgmap v19931: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-09-10T09:37:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:22 smithi098 ceph-mon[103124]: pgmap v19931: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-09-10T09:37:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37074.628028 secs 2023-09-10T09:37:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37074.628028 secs 2023-09-10T09:37:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37074.628028 secs 2023-09-10T09:37:24.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:24 smithi170 ceph-mon[102832]: pgmap v19932: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-09-10T09:37:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37075.875439 secs 2023-09-10T09:37:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:24 smithi082 ceph-mon[98694]: pgmap v19932: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-09-10T09:37:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37075.875439 secs 2023-09-10T09:37:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:24 smithi098 ceph-mon[103124]: pgmap v19932: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-09-10T09:37:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37075.875439 secs 2023-09-10T09:37:25.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37076.228147 secs 2023-09-10T09:37:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37076.228147 secs 2023-09-10T09:37:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37076.228147 secs 2023-09-10T09:37:26.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:26 smithi170 ceph-mon[102832]: pgmap v19933: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 17 op/s 2023-09-10T09:37:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:26 smithi082 ceph-mon[98694]: pgmap v19933: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 17 op/s 2023-09-10T09:37:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:26 smithi098 ceph-mon[103124]: pgmap v19933: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 17 op/s 2023-09-10T09:37:28.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:28 smithi170 ceph-mon[102832]: pgmap v19934: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 10 op/s 2023-09-10T09:37:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37079.628087 secs 2023-09-10T09:37:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:28 smithi082 ceph-mon[98694]: pgmap v19934: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 10 op/s 2023-09-10T09:37:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37079.628087 secs 2023-09-10T09:37:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:28 smithi098 ceph-mon[103124]: pgmap v19934: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 10 op/s 2023-09-10T09:37:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37079.628087 secs 2023-09-10T09:37:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37080.875492 secs 2023-09-10T09:37:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37081.228210 secs 2023-09-10T09:37:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:30 smithi170 ceph-mon[102832]: pgmap v19935: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 10 op/s 2023-09-10T09:37:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37080.875492 secs 2023-09-10T09:37:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37081.228210 secs 2023-09-10T09:37:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:30 smithi082 ceph-mon[98694]: pgmap v19935: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 10 op/s 2023-09-10T09:37:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37080.875492 secs 2023-09-10T09:37:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37081.228210 secs 2023-09-10T09:37:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:30 smithi098 ceph-mon[103124]: pgmap v19935: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 10 op/s 2023-09-10T09:37:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:32 smithi170 ceph-mon[102832]: pgmap v19936: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 135 KiB/s wr, 15 op/s 2023-09-10T09:37:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:32 smithi082 ceph-mon[98694]: pgmap v19936: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 135 KiB/s wr, 15 op/s 2023-09-10T09:37:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:32 smithi098 ceph-mon[103124]: pgmap v19936: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 135 KiB/s wr, 15 op/s 2023-09-10T09:37:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:34 smithi170 ceph-mon[102832]: pgmap v19937: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 10 op/s 2023-09-10T09:37:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:37:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37085.875556 secs 2023-09-10T09:37:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:34 smithi082 ceph-mon[98694]: pgmap v19937: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 10 op/s 2023-09-10T09:37:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:37:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37085.875556 secs 2023-09-10T09:37:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:34 smithi098 ceph-mon[103124]: pgmap v19937: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 10 op/s 2023-09-10T09:37:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:37:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37085.875556 secs 2023-09-10T09:37:35.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37086.228277 secs 2023-09-10T09:37:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:37:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37086.228277 secs 2023-09-10T09:37:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:37:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37086.228277 secs 2023-09-10T09:37:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:37:36.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:36 smithi170 ceph-mon[102832]: pgmap v19938: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 424 KiB/s wr, 16 op/s 2023-09-10T09:37:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:37:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:37:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:36 smithi082 ceph-mon[98694]: pgmap v19938: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 424 KiB/s wr, 16 op/s 2023-09-10T09:37:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:37:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:37:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:36 smithi098 ceph-mon[103124]: pgmap v19938: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 424 KiB/s wr, 16 op/s 2023-09-10T09:37:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:37:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:37:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:38.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:38 smithi170 ceph-mon[102832]: pgmap v19939: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 289 KiB/s wr, 10 op/s 2023-09-10T09:37:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:38 smithi082 ceph-mon[98694]: pgmap v19939: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 289 KiB/s wr, 10 op/s 2023-09-10T09:37:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:38 smithi098 ceph-mon[103124]: pgmap v19939: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 289 KiB/s wr, 10 op/s 2023-09-10T09:37:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37090.875615 secs 2023-09-10T09:37:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37091.228345 secs 2023-09-10T09:37:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:40 smithi170 ceph-mon[102832]: pgmap v19940: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 289 KiB/s wr, 10 op/s 2023-09-10T09:37:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37090.875615 secs 2023-09-10T09:37:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37091.228345 secs 2023-09-10T09:37:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:40 smithi082 ceph-mon[98694]: pgmap v19940: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 289 KiB/s wr, 10 op/s 2023-09-10T09:37:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37090.875615 secs 2023-09-10T09:37:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37091.228345 secs 2023-09-10T09:37:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:40 smithi098 ceph-mon[103124]: pgmap v19940: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 289 KiB/s wr, 10 op/s 2023-09-10T09:37:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:42 smithi170 ceph-mon[102832]: pgmap v19941: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 449 KiB/s wr, 17 op/s 2023-09-10T09:37:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:42 smithi082 ceph-mon[98694]: pgmap v19941: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 449 KiB/s wr, 17 op/s 2023-09-10T09:37:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:42 smithi098 ceph-mon[103124]: pgmap v19941: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 449 KiB/s wr, 17 op/s 2023-09-10T09:37:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:44 smithi170 ceph-mon[102832]: pgmap v19942: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 449 KiB/s wr, 12 op/s 2023-09-10T09:37:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:37:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37095.875673 secs 2023-09-10T09:37:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:44 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:37:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:44 smithi082 ceph-mon[98694]: pgmap v19942: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 449 KiB/s wr, 12 op/s 2023-09-10T09:37:44.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:37:44.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:44.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37095.875673 secs 2023-09-10T09:37:44.711 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:44 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:37:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:44 smithi098 ceph-mon[103124]: pgmap v19942: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 449 KiB/s wr, 12 op/s 2023-09-10T09:37:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:37:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37095.875673 secs 2023-09-10T09:37:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:44 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:37:45.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37096.228411 secs 2023-09-10T09:37:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37096.228411 secs 2023-09-10T09:37:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37096.228411 secs 2023-09-10T09:37:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:46 smithi170 ceph-mon[102832]: pgmap v19943: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 524 KiB/s wr, 19 op/s 2023-09-10T09:37:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:37:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:37:46.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:37:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:46 smithi082 ceph-mon[98694]: pgmap v19943: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 524 KiB/s wr, 19 op/s 2023-09-10T09:37:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:37:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:37:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:37:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:46 smithi098 ceph-mon[103124]: pgmap v19943: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 524 KiB/s wr, 19 op/s 2023-09-10T09:37:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:37:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:37:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:37:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:37:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:37:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:37:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:37:47.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:37:47.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:37:47.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:37:47.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:37:47.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:37:47.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:37:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:37:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:37:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:37:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:37:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:37:47.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:37:47.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:37:47.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:37:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:37:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:37:47.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:37:47.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:37:47.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:37:47.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:37:47.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:37:47.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:37:47.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:37:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:48 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T09:37:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:48 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T09:37:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:48 smithi098 ceph-mon[103124]: pgmap v19944: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 234 KiB/s wr, 13 op/s 2023-09-10T09:37:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:37:48.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:37:48.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:37:48.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:37:48.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:48 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T09:37:48.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:48 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T09:37:48.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:37:48.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:37:48.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:37:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:48 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T09:37:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:48 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T09:37:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:48 smithi170 ceph-mon[102832]: pgmap v19944: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 234 KiB/s wr, 13 op/s 2023-09-10T09:37:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:37:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:37:48.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:37:48.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:37:48.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:48 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T09:37:48.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:48 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T09:37:48.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:37:48.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:37:48.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:37:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:48 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T09:37:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:48 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T09:37:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:48 smithi082 ceph-mon[98694]: pgmap v19944: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 234 KiB/s wr, 13 op/s 2023-09-10T09:37:48.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:37:48.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:37:48.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:37:48.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:37:48.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:48 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T09:37:48.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:48 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T09:37:48.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:37:48.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:37:48.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:37:50.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37100.875737 secs 2023-09-10T09:37:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37101.228485 secs 2023-09-10T09:37:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:50 smithi170 ceph-mon[102832]: pgmap v19945: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 234 KiB/s wr, 13 op/s 2023-09-10T09:37:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:37:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37100.875737 secs 2023-09-10T09:37:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37101.228485 secs 2023-09-10T09:37:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:50 smithi082 ceph-mon[98694]: pgmap v19945: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 234 KiB/s wr, 13 op/s 2023-09-10T09:37:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:37:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37100.875737 secs 2023-09-10T09:37:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37101.228485 secs 2023-09-10T09:37:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:50 smithi098 ceph-mon[103124]: pgmap v19945: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 234 KiB/s wr, 13 op/s 2023-09-10T09:37:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:37:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:37:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:52 smithi082 ceph-mon[98694]: pgmap v19946: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 733 KiB/s wr, 20 op/s 2023-09-10T09:37:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:37:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:52 smithi098 ceph-mon[103124]: pgmap v19946: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 733 KiB/s wr, 20 op/s 2023-09-10T09:37:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:37:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:52 smithi170 ceph-mon[102832]: pgmap v19946: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 733 KiB/s wr, 20 op/s 2023-09-10T09:37:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:54 smithi170 ceph-mon[102832]: pgmap v19947: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 573 KiB/s wr, 14 op/s 2023-09-10T09:37:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37105.875796 secs 2023-09-10T09:37:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:54 smithi082 ceph-mon[98694]: pgmap v19947: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 573 KiB/s wr, 14 op/s 2023-09-10T09:37:54.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37105.875796 secs 2023-09-10T09:37:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:54 smithi098 ceph-mon[103124]: pgmap v19947: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 573 KiB/s wr, 14 op/s 2023-09-10T09:37:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37105.875796 secs 2023-09-10T09:37:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37106.228547 secs 2023-09-10T09:37:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37106.228547 secs 2023-09-10T09:37:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37106.228547 secs 2023-09-10T09:37:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:56 smithi170 ceph-mon[102832]: pgmap v19948: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 696 KiB/s wr, 20 op/s 2023-09-10T09:37:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:56 smithi082 ceph-mon[98694]: pgmap v19948: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 696 KiB/s wr, 20 op/s 2023-09-10T09:37:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:56 smithi098 ceph-mon[103124]: pgmap v19948: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 696 KiB/s wr, 20 op/s 2023-09-10T09:37:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:58 smithi082 ceph-mon[98694]: pgmap v19949: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 621 KiB/s wr, 13 op/s 2023-09-10T09:37:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:37:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:37:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37109.628497 secs 2023-09-10T09:37:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:58 smithi098 ceph-mon[103124]: pgmap v19949: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 621 KiB/s wr, 13 op/s 2023-09-10T09:37:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:37:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:37:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37109.628497 secs 2023-09-10T09:37:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:58 smithi170 ceph-mon[102832]: pgmap v19949: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 621 KiB/s wr, 13 op/s 2023-09-10T09:37:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:37:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:37:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:37:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37109.628497 secs 2023-09-10T09:38:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37110.875858 secs 2023-09-10T09:38:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37111.228609 secs 2023-09-10T09:38:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:00 smithi170 ceph-mon[102832]: pgmap v19950: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 621 KiB/s wr, 13 op/s 2023-09-10T09:38:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37110.875858 secs 2023-09-10T09:38:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37111.228609 secs 2023-09-10T09:38:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:00 smithi082 ceph-mon[98694]: pgmap v19950: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 621 KiB/s wr, 13 op/s 2023-09-10T09:38:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37110.875858 secs 2023-09-10T09:38:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37111.228609 secs 2023-09-10T09:38:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:00 smithi098 ceph-mon[103124]: pgmap v19950: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 621 KiB/s wr, 13 op/s 2023-09-10T09:38:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:01 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:38:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:01 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:38:01.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:01 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:01 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:38:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:01 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:38:01.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:01 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:01 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:38:01.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:01 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:38:01.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:01 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:02 smithi170 ceph-mon[102832]: pgmap v19951: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 35 op/s 2023-09-10T09:38:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:02 smithi082 ceph-mon[98694]: pgmap v19951: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 35 op/s 2023-09-10T09:38:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:02 smithi098 ceph-mon[103124]: pgmap v19951: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 35 op/s 2023-09-10T09:38:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37114.628556 secs 2023-09-10T09:38:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:04 smithi170 ceph-mon[102832]: osdmap e1315: 12 total, 12 up, 12 in 2023-09-10T09:38:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:04 smithi170 ceph-mon[102832]: pgmap v19953: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 733 KiB/s wr, 33 op/s 2023-09-10T09:38:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37114.628556 secs 2023-09-10T09:38:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:04 smithi082 ceph-mon[98694]: osdmap e1315: 12 total, 12 up, 12 in 2023-09-10T09:38:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:04 smithi082 ceph-mon[98694]: pgmap v19953: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 733 KiB/s wr, 33 op/s 2023-09-10T09:38:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37114.628556 secs 2023-09-10T09:38:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:04 smithi098 ceph-mon[103124]: osdmap e1315: 12 total, 12 up, 12 in 2023-09-10T09:38:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:04 smithi098 ceph-mon[103124]: pgmap v19953: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 733 KiB/s wr, 33 op/s 2023-09-10T09:38:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37115.875918 secs 2023-09-10T09:38:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37116.228673 secs 2023-09-10T09:38:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:38:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37115.875918 secs 2023-09-10T09:38:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37116.228673 secs 2023-09-10T09:38:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:38:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37115.875918 secs 2023-09-10T09:38:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37116.228673 secs 2023-09-10T09:38:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:38:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:38:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:06 smithi170 ceph-mon[102832]: pgmap v19954: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 744 KiB/s wr, 33 op/s 2023-09-10T09:38:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:38:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:06 smithi082 ceph-mon[98694]: pgmap v19954: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 744 KiB/s wr, 33 op/s 2023-09-10T09:38:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:38:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:06.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:06 smithi098 ceph-mon[103124]: pgmap v19954: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 744 KiB/s wr, 33 op/s 2023-09-10T09:38:08.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:08 smithi170 ceph-mon[102832]: pgmap v19955: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 744 KiB/s wr, 33 op/s 2023-09-10T09:38:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37119.628630 secs 2023-09-10T09:38:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:08 smithi082 ceph-mon[98694]: pgmap v19955: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 744 KiB/s wr, 33 op/s 2023-09-10T09:38:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37119.628630 secs 2023-09-10T09:38:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:08 smithi098 ceph-mon[103124]: pgmap v19955: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 744 KiB/s wr, 33 op/s 2023-09-10T09:38:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37119.628630 secs 2023-09-10T09:38:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37120.875980 secs 2023-09-10T09:38:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37121.228746 secs 2023-09-10T09:38:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:10 smithi170 ceph-mon[102832]: osdmap e1316: 12 total, 12 up, 12 in 2023-09-10T09:38:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:10 smithi170 ceph-mon[102832]: pgmap v19957: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 197 KiB/s wr, 9 op/s 2023-09-10T09:38:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37120.875980 secs 2023-09-10T09:38:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37121.228746 secs 2023-09-10T09:38:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:10 smithi082 ceph-mon[98694]: osdmap e1316: 12 total, 12 up, 12 in 2023-09-10T09:38:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:10 smithi082 ceph-mon[98694]: pgmap v19957: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 197 KiB/s wr, 9 op/s 2023-09-10T09:38:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37120.875980 secs 2023-09-10T09:38:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37121.228746 secs 2023-09-10T09:38:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:10 smithi098 ceph-mon[103124]: osdmap e1316: 12 total, 12 up, 12 in 2023-09-10T09:38:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:10 smithi098 ceph-mon[103124]: pgmap v19957: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 197 KiB/s wr, 9 op/s 2023-09-10T09:38:12.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:12 smithi170 ceph-mon[102832]: pgmap v19958: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 624 KiB/s wr, 18 op/s 2023-09-10T09:38:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:12 smithi082 ceph-mon[98694]: pgmap v19958: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 624 KiB/s wr, 18 op/s 2023-09-10T09:38:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:12 smithi098 ceph-mon[103124]: pgmap v19958: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 624 KiB/s wr, 18 op/s 2023-09-10T09:38:13.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37124.628711 secs 2023-09-10T09:38:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37124.628711 secs 2023-09-10T09:38:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37124.628711 secs 2023-09-10T09:38:14.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:14 smithi170 ceph-mon[102832]: pgmap v19959: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 514 KiB/s wr, 14 op/s 2023-09-10T09:38:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37125.876040 secs 2023-09-10T09:38:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:14 smithi082 ceph-mon[98694]: pgmap v19959: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 514 KiB/s wr, 14 op/s 2023-09-10T09:38:14.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37125.876040 secs 2023-09-10T09:38:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:14 smithi098 ceph-mon[103124]: pgmap v19959: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 514 KiB/s wr, 14 op/s 2023-09-10T09:38:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37125.876040 secs 2023-09-10T09:38:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37126.228811 secs 2023-09-10T09:38:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37126.228811 secs 2023-09-10T09:38:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37126.228811 secs 2023-09-10T09:38:16.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:16 smithi082 ceph-mon[98694]: pgmap v19960: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 390 KiB/s wr, 14 op/s 2023-09-10T09:38:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:16 smithi098 ceph-mon[103124]: pgmap v19960: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 390 KiB/s wr, 14 op/s 2023-09-10T09:38:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:16 smithi170 ceph-mon[102832]: pgmap v19960: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 390 KiB/s wr, 14 op/s 2023-09-10T09:38:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:18 smithi082 ceph-mon[98694]: pgmap v19961: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 390 KiB/s wr, 14 op/s 2023-09-10T09:38:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37129.628775 secs 2023-09-10T09:38:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:18 smithi098 ceph-mon[103124]: pgmap v19961: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 390 KiB/s wr, 14 op/s 2023-09-10T09:38:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37129.628775 secs 2023-09-10T09:38:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:18 smithi170 ceph-mon[102832]: pgmap v19961: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 390 KiB/s wr, 14 op/s 2023-09-10T09:38:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37129.628775 secs 2023-09-10T09:38:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37130.876101 secs 2023-09-10T09:38:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37131.228875 secs 2023-09-10T09:38:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:20 smithi170 ceph-mon[102832]: pgmap v19962: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 380 KiB/s wr, 14 op/s 2023-09-10T09:38:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37130.876101 secs 2023-09-10T09:38:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37131.228875 secs 2023-09-10T09:38:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:20 smithi082 ceph-mon[98694]: pgmap v19962: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 380 KiB/s wr, 14 op/s 2023-09-10T09:38:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37130.876101 secs 2023-09-10T09:38:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37131.228875 secs 2023-09-10T09:38:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:20 smithi098 ceph-mon[103124]: pgmap v19962: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 380 KiB/s wr, 14 op/s 2023-09-10T09:38:22.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:22 smithi170 ceph-mon[102832]: pgmap v19963: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 325 KiB/s wr, 17 op/s 2023-09-10T09:38:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:22 smithi082 ceph-mon[98694]: pgmap v19963: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 325 KiB/s wr, 17 op/s 2023-09-10T09:38:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:22 smithi098 ceph-mon[103124]: pgmap v19963: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 325 KiB/s wr, 17 op/s 2023-09-10T09:38:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37134.628834 secs 2023-09-10T09:38:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37134.628834 secs 2023-09-10T09:38:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37134.628834 secs 2023-09-10T09:38:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:24 smithi170 ceph-mon[102832]: pgmap v19964: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T09:38:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:38:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37135.876163 secs 2023-09-10T09:38:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:24 smithi082 ceph-mon[98694]: pgmap v19964: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T09:38:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:38:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37135.876163 secs 2023-09-10T09:38:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:24 smithi098 ceph-mon[103124]: pgmap v19964: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T09:38:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:38:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37135.876163 secs 2023-09-10T09:38:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37136.228941 secs 2023-09-10T09:38:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37136.228941 secs 2023-09-10T09:38:25.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37136.228941 secs 2023-09-10T09:38:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:26 smithi082 ceph-mon[98694]: pgmap v19965: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 164 KiB/s wr, 16 op/s 2023-09-10T09:38:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:26 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:38:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:26 smithi098 ceph-mon[103124]: pgmap v19965: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 164 KiB/s wr, 16 op/s 2023-09-10T09:38:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:26 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:38:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:26 smithi170 ceph-mon[102832]: pgmap v19965: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 164 KiB/s wr, 16 op/s 2023-09-10T09:38:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:26 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:38:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:28 smithi082 ceph-mon[98694]: pgmap v19966: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 136 KiB/s wr, 10 op/s 2023-09-10T09:38:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37139.628913 secs 2023-09-10T09:38:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:28 smithi098 ceph-mon[103124]: pgmap v19966: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 136 KiB/s wr, 10 op/s 2023-09-10T09:38:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37139.628913 secs 2023-09-10T09:38:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:28 smithi170 ceph-mon[102832]: pgmap v19966: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 136 KiB/s wr, 10 op/s 2023-09-10T09:38:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37139.628913 secs 2023-09-10T09:38:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37140.876248 secs 2023-09-10T09:38:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37141.229004 secs 2023-09-10T09:38:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:30 smithi170 ceph-mon[102832]: pgmap v19967: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 136 KiB/s wr, 10 op/s 2023-09-10T09:38:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37140.876248 secs 2023-09-10T09:38:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37141.229004 secs 2023-09-10T09:38:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:30 smithi082 ceph-mon[98694]: pgmap v19967: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 136 KiB/s wr, 10 op/s 2023-09-10T09:38:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37140.876248 secs 2023-09-10T09:38:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37141.229004 secs 2023-09-10T09:38:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:30 smithi098 ceph-mon[103124]: pgmap v19967: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 136 KiB/s wr, 10 op/s 2023-09-10T09:38:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:32 smithi170 ceph-mon[102832]: pgmap v19968: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 136 KiB/s wr, 15 op/s 2023-09-10T09:38:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:32 smithi082 ceph-mon[98694]: pgmap v19968: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 136 KiB/s wr, 15 op/s 2023-09-10T09:38:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:32 smithi098 ceph-mon[103124]: pgmap v19968: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 136 KiB/s wr, 15 op/s 2023-09-10T09:38:34.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:34 smithi170 ceph-mon[102832]: pgmap v19969: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 10 op/s 2023-09-10T09:38:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37145.876307 secs 2023-09-10T09:38:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:34 smithi082 ceph-mon[98694]: pgmap v19969: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 10 op/s 2023-09-10T09:38:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37145.876307 secs 2023-09-10T09:38:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:34 smithi098 ceph-mon[103124]: pgmap v19969: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 10 op/s 2023-09-10T09:38:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37145.876307 secs 2023-09-10T09:38:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37146.229067 secs 2023-09-10T09:38:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:38:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37146.229067 secs 2023-09-10T09:38:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:38:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37146.229067 secs 2023-09-10T09:38:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:38:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:36 smithi082 ceph-mon[98694]: pgmap v19970: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 398 KiB/s wr, 16 op/s 2023-09-10T09:38:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:38:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:38:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:36 smithi098 ceph-mon[103124]: pgmap v19970: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 398 KiB/s wr, 16 op/s 2023-09-10T09:38:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:38:36.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:38:36.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:36 smithi170 ceph-mon[102832]: pgmap v19970: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 398 KiB/s wr, 16 op/s 2023-09-10T09:38:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:38:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:38:36.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:38 smithi082 ceph-mon[98694]: pgmap v19971: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 263 KiB/s wr, 10 op/s 2023-09-10T09:38:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:38:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:38 smithi098 ceph-mon[103124]: pgmap v19971: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 263 KiB/s wr, 10 op/s 2023-09-10T09:38:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:38:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:38 smithi170 ceph-mon[102832]: pgmap v19971: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 263 KiB/s wr, 10 op/s 2023-09-10T09:38:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:38:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37150.876366 secs 2023-09-10T09:38:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37151.229133 secs 2023-09-10T09:38:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:40 smithi170 ceph-mon[102832]: pgmap v19972: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 263 KiB/s wr, 10 op/s 2023-09-10T09:38:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37150.876366 secs 2023-09-10T09:38:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37151.229133 secs 2023-09-10T09:38:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:40 smithi082 ceph-mon[98694]: pgmap v19972: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 263 KiB/s wr, 10 op/s 2023-09-10T09:38:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37150.876366 secs 2023-09-10T09:38:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37151.229133 secs 2023-09-10T09:38:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:40 smithi098 ceph-mon[103124]: pgmap v19972: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 263 KiB/s wr, 10 op/s 2023-09-10T09:38:42.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:42 smithi170 ceph-mon[102832]: pgmap v19973: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 587 KiB/s wr, 17 op/s 2023-09-10T09:38:42.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:42 smithi082 ceph-mon[98694]: pgmap v19973: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 587 KiB/s wr, 17 op/s 2023-09-10T09:38:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:42 smithi098 ceph-mon[103124]: pgmap v19973: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 587 KiB/s wr, 17 op/s 2023-09-10T09:38:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:44 smithi082 ceph-mon[98694]: pgmap v19974: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 587 KiB/s wr, 13 op/s 2023-09-10T09:38:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:38:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:44.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37155.876426 secs 2023-09-10T09:38:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:44 smithi098 ceph-mon[103124]: pgmap v19974: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 587 KiB/s wr, 13 op/s 2023-09-10T09:38:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:38:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37155.876426 secs 2023-09-10T09:38:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:44 smithi170 ceph-mon[102832]: pgmap v19974: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 587 KiB/s wr, 13 op/s 2023-09-10T09:38:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:38:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37155.876426 secs 2023-09-10T09:38:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37156.229194 secs 2023-09-10T09:38:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37156.229194 secs 2023-09-10T09:38:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37156.229194 secs 2023-09-10T09:38:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:46 smithi082 ceph-mon[98694]: pgmap v19975: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 612 KiB/s wr, 19 op/s 2023-09-10T09:38:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:38:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:46 smithi098 ceph-mon[103124]: pgmap v19975: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 612 KiB/s wr, 19 op/s 2023-09-10T09:38:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:38:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:46 smithi170 ceph-mon[102832]: pgmap v19975: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 612 KiB/s wr, 19 op/s 2023-09-10T09:38:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:38:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:48 smithi082 ceph-mon[98694]: pgmap v19976: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 349 KiB/s wr, 13 op/s 2023-09-10T09:38:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:38:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:38:48.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:38:48.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:48 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:38:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:48 smithi098 ceph-mon[103124]: pgmap v19976: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 349 KiB/s wr, 13 op/s 2023-09-10T09:38:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:38:48.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:38:48.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:38:48.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:48 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:38:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:48 smithi170 ceph-mon[102832]: pgmap v19976: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 349 KiB/s wr, 13 op/s 2023-09-10T09:38:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:38:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:38:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:38:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:48 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:38:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37160.876517 secs 2023-09-10T09:38:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37161.229262 secs 2023-09-10T09:38:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:50 smithi170 ceph-mon[102832]: pgmap v19977: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 349 KiB/s wr, 13 op/s 2023-09-10T09:38:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:38:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37160.876517 secs 2023-09-10T09:38:50.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37161.229262 secs 2023-09-10T09:38:50.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:50 smithi082 ceph-mon[98694]: pgmap v19977: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 349 KiB/s wr, 13 op/s 2023-09-10T09:38:50.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:38:50.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:50.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37160.876517 secs 2023-09-10T09:38:50.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37161.229262 secs 2023-09-10T09:38:50.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:50 smithi098 ceph-mon[103124]: pgmap v19977: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 349 KiB/s wr, 13 op/s 2023-09-10T09:38:50.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:38:50.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:38:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:52.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:52 smithi082 ceph-mon[98694]: pgmap v19978: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:38:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:38:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:52 smithi098 ceph-mon[103124]: pgmap v19978: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:38:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:38:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:52 smithi170 ceph-mon[102832]: pgmap v19978: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:38:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:38:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:54 smithi082 ceph-mon[98694]: pgmap v19979: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 760 KiB/s wr, 14 op/s 2023-09-10T09:38:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:38:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:54 smithi098 ceph-mon[103124]: pgmap v19979: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 760 KiB/s wr, 14 op/s 2023-09-10T09:38:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:38:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:54 smithi170 ceph-mon[102832]: pgmap v19979: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 760 KiB/s wr, 14 op/s 2023-09-10T09:38:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37165.876577 secs 2023-09-10T09:38:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37166.229327 secs 2023-09-10T09:38:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37165.876577 secs 2023-09-10T09:38:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37166.229327 secs 2023-09-10T09:38:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37165.876577 secs 2023-09-10T09:38:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37166.229327 secs 2023-09-10T09:38:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:38:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:56 smithi098 ceph-mon[103124]: pgmap v19980: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 813 KiB/s wr, 20 op/s 2023-09-10T09:38:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:38:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:56 smithi170 ceph-mon[102832]: pgmap v19980: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 813 KiB/s wr, 20 op/s 2023-09-10T09:38:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:38:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:38:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:56 smithi082 ceph-mon[98694]: pgmap v19980: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 813 KiB/s wr, 20 op/s 2023-09-10T09:38:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:58 smithi082 ceph-mon[98694]: pgmap v19981: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 787 KiB/s wr, 14 op/s 2023-09-10T09:38:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:38:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37169.629428 secs 2023-09-10T09:38:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:58 smithi098 ceph-mon[103124]: pgmap v19981: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 787 KiB/s wr, 14 op/s 2023-09-10T09:38:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:38:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37169.629428 secs 2023-09-10T09:38:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:58 smithi170 ceph-mon[102832]: pgmap v19981: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 787 KiB/s wr, 14 op/s 2023-09-10T09:38:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:38:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37169.629428 secs 2023-09-10T09:39:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37170.876635 secs 2023-09-10T09:39:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37171.229390 secs 2023-09-10T09:39:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:00 smithi170 ceph-mon[102832]: pgmap v19982: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 787 KiB/s wr, 14 op/s 2023-09-10T09:39:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37170.876635 secs 2023-09-10T09:39:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37171.229390 secs 2023-09-10T09:39:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:00 smithi082 ceph-mon[98694]: pgmap v19982: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 787 KiB/s wr, 14 op/s 2023-09-10T09:39:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37170.876635 secs 2023-09-10T09:39:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37171.229390 secs 2023-09-10T09:39:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:00 smithi098 ceph-mon[103124]: pgmap v19982: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 787 KiB/s wr, 14 op/s 2023-09-10T09:39:01.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:01 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:39:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:01 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:39:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:01 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:01 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:39:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:01 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:39:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:01 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:01 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:39:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:01 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:39:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:01 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:02.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:02 smithi170 ceph-mon[102832]: pgmap v19983: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 27 op/s 2023-09-10T09:39:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:02 smithi082 ceph-mon[98694]: pgmap v19983: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 27 op/s 2023-09-10T09:39:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:02 smithi098 ceph-mon[103124]: pgmap v19983: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 27 op/s 2023-09-10T09:39:03.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37174.629540 secs 2023-09-10T09:39:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37174.629540 secs 2023-09-10T09:39:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37174.629540 secs 2023-09-10T09:39:04.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:39:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:04 smithi170 ceph-mon[102832]: osdmap e1317: 12 total, 12 up, 12 in 2023-09-10T09:39:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:04 smithi170 ceph-mon[102832]: pgmap v19985: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 720 KiB/s wr, 23 op/s 2023-09-10T09:39:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:39:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:04 smithi082 ceph-mon[98694]: osdmap e1317: 12 total, 12 up, 12 in 2023-09-10T09:39:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:04 smithi082 ceph-mon[98694]: pgmap v19985: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 720 KiB/s wr, 23 op/s 2023-09-10T09:39:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:39:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:04 smithi098 ceph-mon[103124]: osdmap e1317: 12 total, 12 up, 12 in 2023-09-10T09:39:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:04 smithi098 ceph-mon[103124]: pgmap v19985: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 720 KiB/s wr, 23 op/s 2023-09-10T09:39:05.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37175.876724 secs 2023-09-10T09:39:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37176.229455 secs 2023-09-10T09:39:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37175.876724 secs 2023-09-10T09:39:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37176.229455 secs 2023-09-10T09:39:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37175.876724 secs 2023-09-10T09:39:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37176.229455 secs 2023-09-10T09:39:06.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:39:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:06 smithi170 ceph-mon[102832]: pgmap v19986: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 712 KiB/s wr, 32 op/s 2023-09-10T09:39:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:39:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:06 smithi082 ceph-mon[98694]: pgmap v19986: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 712 KiB/s wr, 32 op/s 2023-09-10T09:39:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:39:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:06 smithi098 ceph-mon[103124]: pgmap v19986: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 712 KiB/s wr, 32 op/s 2023-09-10T09:39:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:08 smithi082 ceph-mon[98694]: pgmap v19987: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 1.0 MiB/s wr, 33 op/s 2023-09-10T09:39:08.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37179.629648 secs 2023-09-10T09:39:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:08 smithi098 ceph-mon[103124]: pgmap v19987: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 1.0 MiB/s wr, 33 op/s 2023-09-10T09:39:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37179.629648 secs 2023-09-10T09:39:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:08 smithi170 ceph-mon[102832]: pgmap v19987: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 1.0 MiB/s wr, 33 op/s 2023-09-10T09:39:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37179.629648 secs 2023-09-10T09:39:10.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37180.876809 secs 2023-09-10T09:39:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37181.229488 secs 2023-09-10T09:39:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:10 smithi170 ceph-mon[102832]: osdmap e1318: 12 total, 12 up, 12 in 2023-09-10T09:39:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:10 smithi170 ceph-mon[102832]: pgmap v19989: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 472 KiB/s wr, 21 op/s 2023-09-10T09:39:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37180.876809 secs 2023-09-10T09:39:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37181.229488 secs 2023-09-10T09:39:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:10 smithi082 ceph-mon[98694]: osdmap e1318: 12 total, 12 up, 12 in 2023-09-10T09:39:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:10 smithi082 ceph-mon[98694]: pgmap v19989: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 472 KiB/s wr, 21 op/s 2023-09-10T09:39:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37180.876809 secs 2023-09-10T09:39:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37181.229488 secs 2023-09-10T09:39:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:10 smithi098 ceph-mon[103124]: osdmap e1318: 12 total, 12 up, 12 in 2023-09-10T09:39:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:10 smithi098 ceph-mon[103124]: pgmap v19989: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 472 KiB/s wr, 21 op/s 2023-09-10T09:39:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:12 smithi082 ceph-mon[98694]: pgmap v19990: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 657 KiB/s wr, 31 op/s 2023-09-10T09:39:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:12 smithi098 ceph-mon[103124]: pgmap v19990: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 657 KiB/s wr, 31 op/s 2023-09-10T09:39:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:12 smithi170 ceph-mon[102832]: pgmap v19990: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 657 KiB/s wr, 31 op/s 2023-09-10T09:39:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37184.629749 secs 2023-09-10T09:39:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37184.629749 secs 2023-09-10T09:39:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37184.629749 secs 2023-09-10T09:39:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:14 smithi082 ceph-mon[98694]: pgmap v19991: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 540 KiB/s wr, 25 op/s 2023-09-10T09:39:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37185.876868 secs 2023-09-10T09:39:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:14 smithi098 ceph-mon[103124]: pgmap v19991: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 540 KiB/s wr, 25 op/s 2023-09-10T09:39:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37185.876868 secs 2023-09-10T09:39:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:14 smithi170 ceph-mon[102832]: pgmap v19991: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 540 KiB/s wr, 25 op/s 2023-09-10T09:39:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37185.876868 secs 2023-09-10T09:39:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37186.229552 secs 2023-09-10T09:39:15.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37186.229552 secs 2023-09-10T09:39:15.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37186.229552 secs 2023-09-10T09:39:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:16 smithi098 ceph-mon[103124]: pgmap v19992: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 518 KiB/s wr, 15 op/s 2023-09-10T09:39:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:16 smithi170 ceph-mon[102832]: pgmap v19992: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 518 KiB/s wr, 15 op/s 2023-09-10T09:39:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:16 smithi082 ceph-mon[98694]: pgmap v19992: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 518 KiB/s wr, 15 op/s 2023-09-10T09:39:18.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:18 smithi098 ceph-mon[103124]: pgmap v19993: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 15 op/s 2023-09-10T09:39:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:39:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37189.629775 secs 2023-09-10T09:39:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:18 smithi170 ceph-mon[102832]: pgmap v19993: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 15 op/s 2023-09-10T09:39:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:39:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37189.629775 secs 2023-09-10T09:39:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:18 smithi082 ceph-mon[98694]: pgmap v19993: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 15 op/s 2023-09-10T09:39:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:39:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37189.629775 secs 2023-09-10T09:39:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37190.876928 secs 2023-09-10T09:39:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37191.229615 secs 2023-09-10T09:39:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:20 smithi170 ceph-mon[102832]: pgmap v19994: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 14 op/s 2023-09-10T09:39:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37190.876928 secs 2023-09-10T09:39:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37191.229615 secs 2023-09-10T09:39:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:20 smithi082 ceph-mon[98694]: pgmap v19994: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 14 op/s 2023-09-10T09:39:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37190.876928 secs 2023-09-10T09:39:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37191.229615 secs 2023-09-10T09:39:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:20 smithi098 ceph-mon[103124]: pgmap v19994: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 14 op/s 2023-09-10T09:39:21.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:21 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:39:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:21 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:21 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:39:21.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:21 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:21 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:39:21.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:21 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:22.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:22 smithi170 ceph-mon[102832]: pgmap v19995: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 299 KiB/s wr, 18 op/s 2023-09-10T09:39:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:22 smithi082 ceph-mon[98694]: pgmap v19995: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 299 KiB/s wr, 18 op/s 2023-09-10T09:39:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:22 smithi098 ceph-mon[103124]: pgmap v19995: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 299 KiB/s wr, 18 op/s 2023-09-10T09:39:23.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37194.629835 secs 2023-09-10T09:39:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37194.629835 secs 2023-09-10T09:39:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37194.629835 secs 2023-09-10T09:39:24.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:24 smithi170 ceph-mon[102832]: pgmap v19996: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-09-10T09:39:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:24 smithi082 ceph-mon[98694]: pgmap v19996: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-09-10T09:39:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:24 smithi098 ceph-mon[103124]: pgmap v19996: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-09-10T09:39:25.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37195.877011 secs 2023-09-10T09:39:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37196.229679 secs 2023-09-10T09:39:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37195.877011 secs 2023-09-10T09:39:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37196.229679 secs 2023-09-10T09:39:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37195.877011 secs 2023-09-10T09:39:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37196.229679 secs 2023-09-10T09:39:26.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:26 smithi170 ceph-mon[102832]: pgmap v19997: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 163 KiB/s wr, 16 op/s 2023-09-10T09:39:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:26 smithi082 ceph-mon[98694]: pgmap v19997: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 163 KiB/s wr, 16 op/s 2023-09-10T09:39:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:26 smithi098 ceph-mon[103124]: pgmap v19997: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 163 KiB/s wr, 16 op/s 2023-09-10T09:39:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:28 smithi082 ceph-mon[98694]: pgmap v19998: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 136 KiB/s wr, 11 op/s 2023-09-10T09:39:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37199.629896 secs 2023-09-10T09:39:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:28 smithi098 ceph-mon[103124]: pgmap v19998: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 136 KiB/s wr, 11 op/s 2023-09-10T09:39:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37199.629896 secs 2023-09-10T09:39:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:28 smithi170 ceph-mon[102832]: pgmap v19998: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 136 KiB/s wr, 11 op/s 2023-09-10T09:39:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37199.629896 secs 2023-09-10T09:39:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37200.877073 secs 2023-09-10T09:39:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37201.229743 secs 2023-09-10T09:39:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:30 smithi170 ceph-mon[102832]: pgmap v19999: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 136 KiB/s wr, 11 op/s 2023-09-10T09:39:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37200.877073 secs 2023-09-10T09:39:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37201.229743 secs 2023-09-10T09:39:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:30 smithi082 ceph-mon[98694]: pgmap v19999: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 136 KiB/s wr, 11 op/s 2023-09-10T09:39:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37200.877073 secs 2023-09-10T09:39:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37201.229743 secs 2023-09-10T09:39:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:30 smithi098 ceph-mon[103124]: pgmap v19999: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 136 KiB/s wr, 11 op/s 2023-09-10T09:39:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:32 smithi082 ceph-mon[98694]: pgmap v20000: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 295 KiB/s wr, 16 op/s 2023-09-10T09:39:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:32 smithi098 ceph-mon[103124]: pgmap v20000: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 295 KiB/s wr, 16 op/s 2023-09-10T09:39:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:32 smithi170 ceph-mon[102832]: pgmap v20000: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 295 KiB/s wr, 16 op/s 2023-09-10T09:39:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:34 smithi082 ceph-mon[98694]: pgmap v20001: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 10 op/s 2023-09-10T09:39:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37205.877131 secs 2023-09-10T09:39:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:34 smithi098 ceph-mon[103124]: pgmap v20001: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 10 op/s 2023-09-10T09:39:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37205.877131 secs 2023-09-10T09:39:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:34 smithi170 ceph-mon[102832]: pgmap v20001: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 159 KiB/s wr, 10 op/s 2023-09-10T09:39:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37205.877131 secs 2023-09-10T09:39:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37206.229808 secs 2023-09-10T09:39:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:39:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37206.229808 secs 2023-09-10T09:39:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:39:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37206.229808 secs 2023-09-10T09:39:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:39:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:36 smithi082 ceph-mon[98694]: pgmap v20002: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 334 KiB/s wr, 16 op/s 2023-09-10T09:39:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:39:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:39:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:36 smithi098 ceph-mon[103124]: pgmap v20002: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 334 KiB/s wr, 16 op/s 2023-09-10T09:39:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:39:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:39:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:36 smithi170 ceph-mon[102832]: pgmap v20002: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 334 KiB/s wr, 16 op/s 2023-09-10T09:39:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:39:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:39:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:38 smithi082 ceph-mon[98694]: pgmap v20003: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 334 KiB/s wr, 11 op/s 2023-09-10T09:39:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:39:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:38 smithi098 ceph-mon[103124]: pgmap v20003: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 334 KiB/s wr, 11 op/s 2023-09-10T09:39:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:39:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:38 smithi170 ceph-mon[102832]: pgmap v20003: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 334 KiB/s wr, 11 op/s 2023-09-10T09:39:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:39:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37210.877191 secs 2023-09-10T09:39:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37211.229875 secs 2023-09-10T09:39:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:40 smithi170 ceph-mon[102832]: pgmap v20004: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 334 KiB/s wr, 11 op/s 2023-09-10T09:39:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:39:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37210.877191 secs 2023-09-10T09:39:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37211.229875 secs 2023-09-10T09:39:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:40 smithi082 ceph-mon[98694]: pgmap v20004: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 334 KiB/s wr, 11 op/s 2023-09-10T09:39:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:39:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37210.877191 secs 2023-09-10T09:39:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37211.229875 secs 2023-09-10T09:39:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:40 smithi098 ceph-mon[103124]: pgmap v20004: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 334 KiB/s wr, 11 op/s 2023-09-10T09:39:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:39:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:42.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:42 smithi082 ceph-mon[98694]: pgmap v20005: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 494 KiB/s wr, 17 op/s 2023-09-10T09:39:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:42 smithi098 ceph-mon[103124]: pgmap v20005: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 494 KiB/s wr, 17 op/s 2023-09-10T09:39:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:42 smithi170 ceph-mon[102832]: pgmap v20005: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 494 KiB/s wr, 17 op/s 2023-09-10T09:39:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:44 smithi082 ceph-mon[98694]: pgmap v20006: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 335 KiB/s wr, 11 op/s 2023-09-10T09:39:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:39:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37215.877249 secs 2023-09-10T09:39:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:44 smithi098 ceph-mon[103124]: pgmap v20006: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 335 KiB/s wr, 11 op/s 2023-09-10T09:39:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:39:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37215.877249 secs 2023-09-10T09:39:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:44 smithi170 ceph-mon[102832]: pgmap v20006: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 335 KiB/s wr, 11 op/s 2023-09-10T09:39:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:39:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37215.877249 secs 2023-09-10T09:39:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37216.229937 secs 2023-09-10T09:39:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37216.229937 secs 2023-09-10T09:39:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37216.229937 secs 2023-09-10T09:39:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:46 smithi082 ceph-mon[98694]: pgmap v20007: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 846 KiB/s wr, 19 op/s 2023-09-10T09:39:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:39:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:46 smithi098 ceph-mon[103124]: pgmap v20007: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 846 KiB/s wr, 19 op/s 2023-09-10T09:39:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:39:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:46 smithi170 ceph-mon[102832]: pgmap v20007: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 846 KiB/s wr, 19 op/s 2023-09-10T09:39:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:39:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:48 smithi098 ceph-mon[103124]: pgmap v20008: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 671 KiB/s wr, 14 op/s 2023-09-10T09:39:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:48 smithi170 ceph-mon[102832]: pgmap v20008: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 671 KiB/s wr, 14 op/s 2023-09-10T09:39:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:48 smithi082 ceph-mon[98694]: pgmap v20008: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 671 KiB/s wr, 14 op/s 2023-09-10T09:39:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:39:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37220.877307 secs 2023-09-10T09:39:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:39:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:39:49.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:39:49.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:39:49.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:49 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:39:49.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:39:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37220.877307 secs 2023-09-10T09:39:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:39:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:39:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:39:49.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:39:49.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:49 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:39:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:39:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37220.877307 secs 2023-09-10T09:39:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:39:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:39:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:39:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:39:49.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:49 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:39:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37221.230007 secs 2023-09-10T09:39:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:50 smithi098 ceph-mon[103124]: pgmap v20009: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 671 KiB/s wr, 14 op/s 2023-09-10T09:39:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:39:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37221.230007 secs 2023-09-10T09:39:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:50 smithi170 ceph-mon[102832]: pgmap v20009: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 671 KiB/s wr, 14 op/s 2023-09-10T09:39:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:39:50.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37221.230007 secs 2023-09-10T09:39:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:50 smithi082 ceph-mon[98694]: pgmap v20009: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 671 KiB/s wr, 14 op/s 2023-09-10T09:39:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:39:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:52 smithi098 ceph-mon[103124]: pgmap v20010: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:39:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:52 smithi170 ceph-mon[102832]: pgmap v20010: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:39:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:52 smithi082 ceph-mon[98694]: pgmap v20010: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:39:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:54 smithi098 ceph-mon[103124]: pgmap v20011: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 941 KiB/s wr, 15 op/s 2023-09-10T09:39:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:39:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37225.877414 secs 2023-09-10T09:39:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:54 smithi170 ceph-mon[102832]: pgmap v20011: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 941 KiB/s wr, 15 op/s 2023-09-10T09:39:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:39:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37225.877414 secs 2023-09-10T09:39:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:54 smithi082 ceph-mon[98694]: pgmap v20011: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 941 KiB/s wr, 15 op/s 2023-09-10T09:39:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:39:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37225.877414 secs 2023-09-10T09:39:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37226.230072 secs 2023-09-10T09:39:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37226.230072 secs 2023-09-10T09:39:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37226.230072 secs 2023-09-10T09:39:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:56 smithi098 ceph-mon[103124]: pgmap v20012: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 22 op/s 2023-09-10T09:39:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:39:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:56 smithi170 ceph-mon[102832]: pgmap v20012: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 22 op/s 2023-09-10T09:39:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:39:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:56 smithi082 ceph-mon[98694]: pgmap v20012: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 22 op/s 2023-09-10T09:39:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:39:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:39:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:39:58 smithi098 ceph-mon[103124]: pgmap v20013: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 528 KiB/s wr, 15 op/s 2023-09-10T09:39:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:39:58 smithi170 ceph-mon[102832]: pgmap v20013: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 528 KiB/s wr, 15 op/s 2023-09-10T09:39:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:39:58 smithi082 ceph-mon[98694]: pgmap v20013: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 528 KiB/s wr, 15 op/s 2023-09-10T09:40:00.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37230.877491 secs 2023-09-10T09:40:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37231.230137 secs 2023-09-10T09:40:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:00 smithi170 ceph-mon[102832]: pgmap v20014: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 528 KiB/s wr, 14 op/s 2023-09-10T09:40:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 2 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T09:40:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37230.877491 secs 2023-09-10T09:40:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37231.230137 secs 2023-09-10T09:40:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:00 smithi082 ceph-mon[98694]: pgmap v20014: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 528 KiB/s wr, 14 op/s 2023-09-10T09:40:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 2 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T09:40:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37230.877491 secs 2023-09-10T09:40:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37231.230137 secs 2023-09-10T09:40:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:00 smithi098 ceph-mon[103124]: pgmap v20014: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 528 KiB/s wr, 14 op/s 2023-09-10T09:40:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 2 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T09:40:02.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:02 smithi082 ceph-mon[98694]: pgmap v20015: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 770 KiB/s wr, 21 op/s 2023-09-10T09:40:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:02 smithi098 ceph-mon[103124]: pgmap v20015: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 770 KiB/s wr, 21 op/s 2023-09-10T09:40:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:02 smithi170 ceph-mon[102832]: pgmap v20015: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 770 KiB/s wr, 21 op/s 2023-09-10T09:40:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37234.630464 secs 2023-09-10T09:40:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37234.630464 secs 2023-09-10T09:40:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37234.630464 secs 2023-09-10T09:40:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:04 smithi098 ceph-mon[103124]: pgmap v20016: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 340 KiB/s wr, 13 op/s 2023-09-10T09:40:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:40:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:40:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:04 smithi098 ceph-mon[103124]: osdmap e1319: 12 total, 12 up, 12 in 2023-09-10T09:40:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37235.877549 secs 2023-09-10T09:40:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:04 smithi170 ceph-mon[102832]: pgmap v20016: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 340 KiB/s wr, 13 op/s 2023-09-10T09:40:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:40:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:40:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:04 smithi170 ceph-mon[102832]: osdmap e1319: 12 total, 12 up, 12 in 2023-09-10T09:40:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37235.877549 secs 2023-09-10T09:40:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:04 smithi082 ceph-mon[98694]: pgmap v20016: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 340 KiB/s wr, 13 op/s 2023-09-10T09:40:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:40:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:40:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:04 smithi082 ceph-mon[98694]: osdmap e1319: 12 total, 12 up, 12 in 2023-09-10T09:40:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37235.877549 secs 2023-09-10T09:40:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37236.230202 secs 2023-09-10T09:40:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37236.230202 secs 2023-09-10T09:40:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37236.230202 secs 2023-09-10T09:40:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:06 smithi098 ceph-mon[103124]: pgmap v20018: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 719 KiB/s wr, 33 op/s 2023-09-10T09:40:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:40:06.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:06 smithi170 ceph-mon[102832]: pgmap v20018: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 719 KiB/s wr, 33 op/s 2023-09-10T09:40:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:40:06.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:06 smithi082 ceph-mon[98694]: pgmap v20018: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 719 KiB/s wr, 33 op/s 2023-09-10T09:40:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:40:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:08 smithi098 ceph-mon[103124]: pgmap v20019: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 719 KiB/s wr, 33 op/s 2023-09-10T09:40:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37239.630572 secs 2023-09-10T09:40:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:08 smithi170 ceph-mon[102832]: pgmap v20019: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 719 KiB/s wr, 33 op/s 2023-09-10T09:40:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37239.630572 secs 2023-09-10T09:40:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:08 smithi082 ceph-mon[98694]: pgmap v20019: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 719 KiB/s wr, 33 op/s 2023-09-10T09:40:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37239.630572 secs 2023-09-10T09:40:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37240.877651 secs 2023-09-10T09:40:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37241.230264 secs 2023-09-10T09:40:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:10 smithi170 ceph-mon[102832]: osdmap e1320: 12 total, 12 up, 12 in 2023-09-10T09:40:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:10 smithi170 ceph-mon[102832]: pgmap v20021: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 536 KiB/s wr, 31 op/s 2023-09-10T09:40:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:40:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37240.877651 secs 2023-09-10T09:40:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37241.230264 secs 2023-09-10T09:40:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:10 smithi082 ceph-mon[98694]: osdmap e1320: 12 total, 12 up, 12 in 2023-09-10T09:40:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:10 smithi082 ceph-mon[98694]: pgmap v20021: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 536 KiB/s wr, 31 op/s 2023-09-10T09:40:10.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:40:10.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37240.877651 secs 2023-09-10T09:40:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37241.230264 secs 2023-09-10T09:40:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:10 smithi098 ceph-mon[103124]: osdmap e1320: 12 total, 12 up, 12 in 2023-09-10T09:40:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:10 smithi098 ceph-mon[103124]: pgmap v20021: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 536 KiB/s wr, 31 op/s 2023-09-10T09:40:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:40:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:12.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:40:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:12 smithi170 ceph-mon[102832]: pgmap v20022: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 739 KiB/s wr, 41 op/s 2023-09-10T09:40:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:40:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:12 smithi082 ceph-mon[98694]: pgmap v20022: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 739 KiB/s wr, 41 op/s 2023-09-10T09:40:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:40:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:12 smithi098 ceph-mon[103124]: pgmap v20022: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 739 KiB/s wr, 41 op/s 2023-09-10T09:40:13.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37244.630634 secs 2023-09-10T09:40:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37244.630634 secs 2023-09-10T09:40:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37244.630634 secs 2023-09-10T09:40:14.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:14 smithi170 ceph-mon[102832]: pgmap v20023: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 595 KiB/s wr, 33 op/s 2023-09-10T09:40:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:14 smithi082 ceph-mon[98694]: pgmap v20023: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 595 KiB/s wr, 33 op/s 2023-09-10T09:40:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:14 smithi098 ceph-mon[103124]: pgmap v20023: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 595 KiB/s wr, 33 op/s 2023-09-10T09:40:15.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37245.877734 secs 2023-09-10T09:40:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37246.230331 secs 2023-09-10T09:40:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37245.877734 secs 2023-09-10T09:40:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37246.230331 secs 2023-09-10T09:40:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37245.877734 secs 2023-09-10T09:40:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37246.230331 secs 2023-09-10T09:40:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:16 smithi082 ceph-mon[98694]: pgmap v20024: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 195 KiB/s wr, 15 op/s 2023-09-10T09:40:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:16 smithi098 ceph-mon[103124]: pgmap v20024: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 195 KiB/s wr, 15 op/s 2023-09-10T09:40:16.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:16 smithi170 ceph-mon[102832]: pgmap v20024: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 195 KiB/s wr, 15 op/s 2023-09-10T09:40:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:18 smithi082 ceph-mon[98694]: pgmap v20025: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 15 op/s 2023-09-10T09:40:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37249.630692 secs 2023-09-10T09:40:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:18 smithi098 ceph-mon[103124]: pgmap v20025: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 15 op/s 2023-09-10T09:40:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37249.630692 secs 2023-09-10T09:40:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:18 smithi170 ceph-mon[102832]: pgmap v20025: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 15 op/s 2023-09-10T09:40:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37249.630692 secs 2023-09-10T09:40:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37250.877794 secs 2023-09-10T09:40:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37251.230394 secs 2023-09-10T09:40:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:20 smithi170 ceph-mon[102832]: pgmap v20026: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 14 op/s 2023-09-10T09:40:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37250.877794 secs 2023-09-10T09:40:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37251.230394 secs 2023-09-10T09:40:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:20 smithi082 ceph-mon[98694]: pgmap v20026: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 14 op/s 2023-09-10T09:40:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37250.877794 secs 2023-09-10T09:40:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37251.230394 secs 2023-09-10T09:40:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:20 smithi098 ceph-mon[103124]: pgmap v20026: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 14 op/s 2023-09-10T09:40:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:22 smithi082 ceph-mon[98694]: pgmap v20027: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 164 KiB/s wr, 18 op/s 2023-09-10T09:40:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:22 smithi098 ceph-mon[103124]: pgmap v20027: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 164 KiB/s wr, 18 op/s 2023-09-10T09:40:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:22 smithi170 ceph-mon[102832]: pgmap v20027: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 164 KiB/s wr, 18 op/s 2023-09-10T09:40:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37254.630753 secs 2023-09-10T09:40:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37254.630753 secs 2023-09-10T09:40:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37254.630753 secs 2023-09-10T09:40:24.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:24 smithi098 ceph-mon[103124]: pgmap v20028: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T09:40:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37255.877858 secs 2023-09-10T09:40:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:24 smithi170 ceph-mon[102832]: pgmap v20028: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T09:40:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37255.877858 secs 2023-09-10T09:40:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:24 smithi082 ceph-mon[98694]: pgmap v20028: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T09:40:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37255.877858 secs 2023-09-10T09:40:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37256.230461 secs 2023-09-10T09:40:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37256.230461 secs 2023-09-10T09:40:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37256.230461 secs 2023-09-10T09:40:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:26 smithi098 ceph-mon[103124]: pgmap v20029: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 165 KiB/s wr, 17 op/s 2023-09-10T09:40:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:26 smithi170 ceph-mon[102832]: pgmap v20029: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 165 KiB/s wr, 17 op/s 2023-09-10T09:40:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:26 smithi082 ceph-mon[98694]: pgmap v20029: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 165 KiB/s wr, 17 op/s 2023-09-10T09:40:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:28 smithi098 ceph-mon[103124]: pgmap v20030: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 11 op/s 2023-09-10T09:40:28.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37259.630780 secs 2023-09-10T09:40:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:28 smithi170 ceph-mon[102832]: pgmap v20030: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 11 op/s 2023-09-10T09:40:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37259.630780 secs 2023-09-10T09:40:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:28 smithi082 ceph-mon[98694]: pgmap v20030: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 11 op/s 2023-09-10T09:40:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37259.630780 secs 2023-09-10T09:40:30.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37260.877918 secs 2023-09-10T09:40:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37261.230484 secs 2023-09-10T09:40:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:30 smithi170 ceph-mon[102832]: pgmap v20031: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 11 op/s 2023-09-10T09:40:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37260.877918 secs 2023-09-10T09:40:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37261.230484 secs 2023-09-10T09:40:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:30 smithi082 ceph-mon[98694]: pgmap v20031: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 11 op/s 2023-09-10T09:40:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37260.877918 secs 2023-09-10T09:40:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37261.230484 secs 2023-09-10T09:40:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:30 smithi098 ceph-mon[103124]: pgmap v20031: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 11 op/s 2023-09-10T09:40:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:32 smithi098 ceph-mon[103124]: pgmap v20032: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 136 KiB/s wr, 16 op/s 2023-09-10T09:40:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:32 smithi170 ceph-mon[102832]: pgmap v20032: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 136 KiB/s wr, 16 op/s 2023-09-10T09:40:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:32 smithi082 ceph-mon[98694]: pgmap v20032: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 136 KiB/s wr, 16 op/s 2023-09-10T09:40:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:34 smithi098 ceph-mon[103124]: pgmap v20033: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 10 op/s 2023-09-10T09:40:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:40:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37265.877989 secs 2023-09-10T09:40:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:34 smithi170 ceph-mon[102832]: pgmap v20033: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 10 op/s 2023-09-10T09:40:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:40:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37265.877989 secs 2023-09-10T09:40:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:34 smithi082 ceph-mon[98694]: pgmap v20033: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 136 KiB/s wr, 10 op/s 2023-09-10T09:40:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:40:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37265.877989 secs 2023-09-10T09:40:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37266.230547 secs 2023-09-10T09:40:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:40:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37266.230547 secs 2023-09-10T09:40:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:40:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37266.230547 secs 2023-09-10T09:40:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:40:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:36 smithi098 ceph-mon[103124]: pgmap v20034: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 427 KiB/s wr, 16 op/s 2023-09-10T09:40:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:40:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:40:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:36 smithi170 ceph-mon[102832]: pgmap v20034: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 427 KiB/s wr, 16 op/s 2023-09-10T09:40:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:40:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:40:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:36 smithi082 ceph-mon[98694]: pgmap v20034: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 427 KiB/s wr, 16 op/s 2023-09-10T09:40:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:40:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:40:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:38 smithi098 ceph-mon[103124]: pgmap v20035: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 291 KiB/s wr, 10 op/s 2023-09-10T09:40:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:38 smithi170 ceph-mon[102832]: pgmap v20035: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 291 KiB/s wr, 10 op/s 2023-09-10T09:40:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:38 smithi082 ceph-mon[98694]: pgmap v20035: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 291 KiB/s wr, 10 op/s 2023-09-10T09:40:40.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37270.878046 secs 2023-09-10T09:40:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37271.230612 secs 2023-09-10T09:40:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:40 smithi170 ceph-mon[102832]: pgmap v20036: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 291 KiB/s wr, 10 op/s 2023-09-10T09:40:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37270.878046 secs 2023-09-10T09:40:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37271.230612 secs 2023-09-10T09:40:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:40 smithi082 ceph-mon[98694]: pgmap v20036: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 291 KiB/s wr, 10 op/s 2023-09-10T09:40:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37270.878046 secs 2023-09-10T09:40:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37271.230612 secs 2023-09-10T09:40:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:40 smithi098 ceph-mon[103124]: pgmap v20036: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 291 KiB/s wr, 10 op/s 2023-09-10T09:40:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:42 smithi098 ceph-mon[103124]: pgmap v20037: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 452 KiB/s wr, 17 op/s 2023-09-10T09:40:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:42 smithi170 ceph-mon[102832]: pgmap v20037: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 452 KiB/s wr, 17 op/s 2023-09-10T09:40:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:42 smithi082 ceph-mon[98694]: pgmap v20037: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 452 KiB/s wr, 17 op/s 2023-09-10T09:40:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:44 smithi098 ceph-mon[103124]: pgmap v20038: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 452 KiB/s wr, 12 op/s 2023-09-10T09:40:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:40:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37275.878105 secs 2023-09-10T09:40:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:44 smithi170 ceph-mon[102832]: pgmap v20038: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 452 KiB/s wr, 12 op/s 2023-09-10T09:40:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:40:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:44.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37275.878105 secs 2023-09-10T09:40:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:44 smithi082 ceph-mon[98694]: pgmap v20038: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 452 KiB/s wr, 12 op/s 2023-09-10T09:40:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:40:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37275.878105 secs 2023-09-10T09:40:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37276.230683 secs 2023-09-10T09:40:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37276.230683 secs 2023-09-10T09:40:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37276.230683 secs 2023-09-10T09:40:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:46 smithi098 ceph-mon[103124]: pgmap v20039: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 528 KiB/s wr, 18 op/s 2023-09-10T09:40:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:40:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:46 smithi170 ceph-mon[102832]: pgmap v20039: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 528 KiB/s wr, 18 op/s 2023-09-10T09:40:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:40:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:46 smithi082 ceph-mon[98694]: pgmap v20039: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 528 KiB/s wr, 18 op/s 2023-09-10T09:40:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:40:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:48.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:48 smithi098 ceph-mon[103124]: pgmap v20040: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 236 KiB/s wr, 13 op/s 2023-09-10T09:40:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:48 smithi170 ceph-mon[102832]: pgmap v20040: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 236 KiB/s wr, 13 op/s 2023-09-10T09:40:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:48 smithi082 ceph-mon[98694]: pgmap v20040: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 236 KiB/s wr, 13 op/s 2023-09-10T09:40:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37280.878163 secs 2023-09-10T09:40:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37281.230745 secs 2023-09-10T09:40:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:50 smithi170 ceph-mon[102832]: pgmap v20041: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 236 KiB/s wr, 13 op/s 2023-09-10T09:40:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:40:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:40:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:40:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:40:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:40:50.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:40:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37280.878163 secs 2023-09-10T09:40:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37281.230745 secs 2023-09-10T09:40:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:50 smithi082 ceph-mon[98694]: pgmap v20041: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 236 KiB/s wr, 13 op/s 2023-09-10T09:40:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:40:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:40:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:40:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:40:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:40:50.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:40:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37280.878163 secs 2023-09-10T09:40:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37281.230745 secs 2023-09-10T09:40:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:50 smithi098 ceph-mon[103124]: pgmap v20041: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 236 KiB/s wr, 13 op/s 2023-09-10T09:40:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:40:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:40:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:40:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:40:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:40:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:40:51.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:40:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:40:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:40:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:40:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:40:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:40:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:40:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:40:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:40:52.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:52 smithi170 ceph-mon[102832]: pgmap v20042: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 738 KiB/s wr, 21 op/s 2023-09-10T09:40:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:40:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:52 smithi082 ceph-mon[98694]: pgmap v20042: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 738 KiB/s wr, 21 op/s 2023-09-10T09:40:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:40:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:52 smithi098 ceph-mon[103124]: pgmap v20042: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 738 KiB/s wr, 21 op/s 2023-09-10T09:40:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:40:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:54 smithi170 ceph-mon[102832]: pgmap v20043: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 578 KiB/s wr, 14 op/s 2023-09-10T09:40:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37285.878219 secs 2023-09-10T09:40:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37286.230810 secs 2023-09-10T09:40:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:54 smithi082 ceph-mon[98694]: pgmap v20043: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 578 KiB/s wr, 14 op/s 2023-09-10T09:40:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37285.878219 secs 2023-09-10T09:40:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37286.230810 secs 2023-09-10T09:40:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:54 smithi098 ceph-mon[103124]: pgmap v20043: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 578 KiB/s wr, 14 op/s 2023-09-10T09:40:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37285.878219 secs 2023-09-10T09:40:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37286.230810 secs 2023-09-10T09:40:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:56 smithi170 ceph-mon[102832]: pgmap v20044: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 690 KiB/s wr, 20 op/s 2023-09-10T09:40:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:56 smithi082 ceph-mon[98694]: pgmap v20044: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 690 KiB/s wr, 20 op/s 2023-09-10T09:40:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:56 smithi098 ceph-mon[103124]: pgmap v20044: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 690 KiB/s wr, 20 op/s 2023-09-10T09:40:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:58 smithi170 ceph-mon[102832]: pgmap v20045: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 615 KiB/s wr, 14 op/s 2023-09-10T09:40:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:40:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:40:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:58 smithi082 ceph-mon[98694]: pgmap v20045: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 615 KiB/s wr, 14 op/s 2023-09-10T09:40:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:40:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:40:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:40:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:58 smithi098 ceph-mon[103124]: pgmap v20045: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 615 KiB/s wr, 14 op/s 2023-09-10T09:40:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:40:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:40:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37290.878324 secs 2023-09-10T09:41:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37291.230875 secs 2023-09-10T09:41:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:00 smithi170 ceph-mon[102832]: pgmap v20046: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 615 KiB/s wr, 13 op/s 2023-09-10T09:41:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:41:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37290.878324 secs 2023-09-10T09:41:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37291.230875 secs 2023-09-10T09:41:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:00 smithi082 ceph-mon[98694]: pgmap v20046: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 615 KiB/s wr, 13 op/s 2023-09-10T09:41:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:41:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37290.878324 secs 2023-09-10T09:41:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37291.230875 secs 2023-09-10T09:41:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:00 smithi098 ceph-mon[103124]: pgmap v20046: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 615 KiB/s wr, 13 op/s 2023-09-10T09:41:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:41:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:02 smithi098 ceph-mon[103124]: pgmap v20047: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:41:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:02 smithi170 ceph-mon[102832]: pgmap v20047: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:41:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:02 smithi082 ceph-mon[98694]: pgmap v20047: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:41:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37294.631280 secs 2023-09-10T09:41:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37294.631280 secs 2023-09-10T09:41:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37294.631280 secs 2023-09-10T09:41:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:04 smithi098 ceph-mon[103124]: pgmap v20048: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 615 KiB/s wr, 14 op/s 2023-09-10T09:41:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:41:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:41:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37295.878412 secs 2023-09-10T09:41:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:04 smithi170 ceph-mon[102832]: pgmap v20048: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 615 KiB/s wr, 14 op/s 2023-09-10T09:41:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:41:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:41:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37295.878412 secs 2023-09-10T09:41:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:04 smithi082 ceph-mon[98694]: pgmap v20048: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 615 KiB/s wr, 14 op/s 2023-09-10T09:41:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:41:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:41:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37295.878412 secs 2023-09-10T09:41:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37296.230941 secs 2023-09-10T09:41:05.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37296.230941 secs 2023-09-10T09:41:05.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37296.230941 secs 2023-09-10T09:41:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:06 smithi098 ceph-mon[103124]: pgmap v20049: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 791 KiB/s wr, 36 op/s 2023-09-10T09:41:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:41:06.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:06 smithi170 ceph-mon[102832]: pgmap v20049: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 791 KiB/s wr, 36 op/s 2023-09-10T09:41:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:41:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:06 smithi082 ceph-mon[98694]: pgmap v20049: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 791 KiB/s wr, 36 op/s 2023-09-10T09:41:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:41:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:08 smithi098 ceph-mon[103124]: pgmap v20050: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 678 KiB/s wr, 30 op/s 2023-09-10T09:41:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37299.631340 secs 2023-09-10T09:41:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:08 smithi170 ceph-mon[102832]: pgmap v20050: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 678 KiB/s wr, 30 op/s 2023-09-10T09:41:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37299.631340 secs 2023-09-10T09:41:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:08 smithi082 ceph-mon[98694]: pgmap v20050: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 678 KiB/s wr, 30 op/s 2023-09-10T09:41:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37299.631340 secs 2023-09-10T09:41:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:09 smithi098 ceph-mon[103124]: osdmap e1321: 12 total, 12 up, 12 in 2023-09-10T09:41:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37300.878469 secs 2023-09-10T09:41:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:09 smithi170 ceph-mon[102832]: osdmap e1321: 12 total, 12 up, 12 in 2023-09-10T09:41:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37300.878469 secs 2023-09-10T09:41:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:09 smithi082 ceph-mon[98694]: osdmap e1321: 12 total, 12 up, 12 in 2023-09-10T09:41:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37300.878469 secs 2023-09-10T09:41:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37301.231006 secs 2023-09-10T09:41:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:10 smithi098 ceph-mon[103124]: pgmap v20052: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 814 KiB/s wr, 36 op/s 2023-09-10T09:41:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:41:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37301.231006 secs 2023-09-10T09:41:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:10 smithi170 ceph-mon[102832]: pgmap v20052: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 814 KiB/s wr, 36 op/s 2023-09-10T09:41:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:41:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37301.231006 secs 2023-09-10T09:41:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:10 smithi082 ceph-mon[98694]: pgmap v20052: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 814 KiB/s wr, 36 op/s 2023-09-10T09:41:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:41:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:12 smithi098 ceph-mon[103124]: pgmap v20053: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 597 KiB/s wr, 34 op/s 2023-09-10T09:41:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:41:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:12 smithi170 ceph-mon[102832]: pgmap v20053: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 597 KiB/s wr, 34 op/s 2023-09-10T09:41:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:41:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:12 smithi082 ceph-mon[98694]: pgmap v20053: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 597 KiB/s wr, 34 op/s 2023-09-10T09:41:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:41:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37304.631400 secs 2023-09-10T09:41:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37304.631400 secs 2023-09-10T09:41:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37304.631400 secs 2023-09-10T09:41:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:14 smithi170 ceph-mon[102832]: pgmap v20054: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 597 KiB/s wr, 34 op/s 2023-09-10T09:41:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37305.878495 secs 2023-09-10T09:41:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37306.231070 secs 2023-09-10T09:41:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:14 smithi170 ceph-mon[102832]: osdmap e1322: 12 total, 12 up, 12 in 2023-09-10T09:41:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:14 smithi082 ceph-mon[98694]: pgmap v20054: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 597 KiB/s wr, 34 op/s 2023-09-10T09:41:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37305.878495 secs 2023-09-10T09:41:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37306.231070 secs 2023-09-10T09:41:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:14 smithi082 ceph-mon[98694]: osdmap e1322: 12 total, 12 up, 12 in 2023-09-10T09:41:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:14 smithi098 ceph-mon[103124]: pgmap v20054: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 597 KiB/s wr, 34 op/s 2023-09-10T09:41:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37305.878495 secs 2023-09-10T09:41:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37306.231070 secs 2023-09-10T09:41:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:14 smithi098 ceph-mon[103124]: osdmap e1322: 12 total, 12 up, 12 in 2023-09-10T09:41:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:16 smithi170 ceph-mon[102832]: pgmap v20056: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 484 KiB/s wr, 16 op/s 2023-09-10T09:41:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:16 smithi082 ceph-mon[98694]: pgmap v20056: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 484 KiB/s wr, 16 op/s 2023-09-10T09:41:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:16 smithi098 ceph-mon[103124]: pgmap v20056: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 484 KiB/s wr, 16 op/s 2023-09-10T09:41:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:18 smithi170 ceph-mon[102832]: pgmap v20057: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 433 KiB/s wr, 15 op/s 2023-09-10T09:41:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37309.631460 secs 2023-09-10T09:41:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:18 smithi082 ceph-mon[98694]: pgmap v20057: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 433 KiB/s wr, 15 op/s 2023-09-10T09:41:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37309.631460 secs 2023-09-10T09:41:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:18 smithi098 ceph-mon[103124]: pgmap v20057: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 433 KiB/s wr, 15 op/s 2023-09-10T09:41:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37309.631460 secs 2023-09-10T09:41:20.427 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37310.878582 secs 2023-09-10T09:41:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37311.231137 secs 2023-09-10T09:41:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:20 smithi170 ceph-mon[102832]: pgmap v20058: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 387 KiB/s wr, 14 op/s 2023-09-10T09:41:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37310.878582 secs 2023-09-10T09:41:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37311.231137 secs 2023-09-10T09:41:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:20 smithi082 ceph-mon[98694]: pgmap v20058: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 387 KiB/s wr, 14 op/s 2023-09-10T09:41:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37310.878582 secs 2023-09-10T09:41:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37311.231137 secs 2023-09-10T09:41:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:20 smithi098 ceph-mon[103124]: pgmap v20058: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 387 KiB/s wr, 14 op/s 2023-09-10T09:41:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:22 smithi098 ceph-mon[103124]: pgmap v20059: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T09:41:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:22 smithi170 ceph-mon[102832]: pgmap v20059: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T09:41:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:22 smithi082 ceph-mon[98694]: pgmap v20059: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T09:41:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37314.631521 secs 2023-09-10T09:41:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37314.631521 secs 2023-09-10T09:41:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37314.631521 secs 2023-09-10T09:41:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:24 smithi098 ceph-mon[103124]: pgmap v20060: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T09:41:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37315.878642 secs 2023-09-10T09:41:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:24 smithi170 ceph-mon[102832]: pgmap v20060: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T09:41:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37315.878642 secs 2023-09-10T09:41:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:24 smithi082 ceph-mon[98694]: pgmap v20060: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 158 KiB/s wr, 11 op/s 2023-09-10T09:41:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37315.878642 secs 2023-09-10T09:41:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37316.231209 secs 2023-09-10T09:41:25.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37316.231209 secs 2023-09-10T09:41:25.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37316.231209 secs 2023-09-10T09:41:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:26 smithi098 ceph-mon[103124]: pgmap v20061: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 262 KiB/s wr, 16 op/s 2023-09-10T09:41:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:26 smithi170 ceph-mon[102832]: pgmap v20061: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 262 KiB/s wr, 16 op/s 2023-09-10T09:41:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:26 smithi082 ceph-mon[98694]: pgmap v20061: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 262 KiB/s wr, 16 op/s 2023-09-10T09:41:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:28 smithi098 ceph-mon[103124]: pgmap v20062: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 267 KiB/s wr, 11 op/s 2023-09-10T09:41:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37319.631623 secs 2023-09-10T09:41:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:28 smithi170 ceph-mon[102832]: pgmap v20062: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 267 KiB/s wr, 11 op/s 2023-09-10T09:41:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37319.631623 secs 2023-09-10T09:41:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:28 smithi082 ceph-mon[98694]: pgmap v20062: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 267 KiB/s wr, 11 op/s 2023-09-10T09:41:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37319.631623 secs 2023-09-10T09:41:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37320.878701 secs 2023-09-10T09:41:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37321.231269 secs 2023-09-10T09:41:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:30 smithi082 ceph-mon[98694]: pgmap v20063: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 267 KiB/s wr, 10 op/s 2023-09-10T09:41:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37320.878701 secs 2023-09-10T09:41:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37321.231269 secs 2023-09-10T09:41:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:30 smithi098 ceph-mon[103124]: pgmap v20063: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 267 KiB/s wr, 10 op/s 2023-09-10T09:41:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37320.878701 secs 2023-09-10T09:41:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37321.231269 secs 2023-09-10T09:41:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:30 smithi170 ceph-mon[102832]: pgmap v20063: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 267 KiB/s wr, 10 op/s 2023-09-10T09:41:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:32 smithi098 ceph-mon[103124]: pgmap v20064: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 267 KiB/s wr, 16 op/s 2023-09-10T09:41:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:32 smithi170 ceph-mon[102832]: pgmap v20064: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 267 KiB/s wr, 16 op/s 2023-09-10T09:41:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:32 smithi082 ceph-mon[98694]: pgmap v20064: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 267 KiB/s wr, 16 op/s 2023-09-10T09:41:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:34 smithi098 ceph-mon[103124]: pgmap v20065: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 11 op/s 2023-09-10T09:41:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:41:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37325.878764 secs 2023-09-10T09:41:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:34 smithi170 ceph-mon[102832]: pgmap v20065: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 11 op/s 2023-09-10T09:41:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:41:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37325.878764 secs 2023-09-10T09:41:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:34 smithi082 ceph-mon[98694]: pgmap v20065: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 137 KiB/s wr, 11 op/s 2023-09-10T09:41:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:41:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37325.878764 secs 2023-09-10T09:41:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37326.231336 secs 2023-09-10T09:41:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:41:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37326.231336 secs 2023-09-10T09:41:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:41:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37326.231336 secs 2023-09-10T09:41:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:41:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:36 smithi098 ceph-mon[103124]: pgmap v20066: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 402 KiB/s wr, 17 op/s 2023-09-10T09:41:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:41:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:41:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:36 smithi170 ceph-mon[102832]: pgmap v20066: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 402 KiB/s wr, 17 op/s 2023-09-10T09:41:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:41:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:41:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:36 smithi082 ceph-mon[98694]: pgmap v20066: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 402 KiB/s wr, 17 op/s 2023-09-10T09:41:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:41:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:41:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:38 smithi170 ceph-mon[102832]: pgmap v20067: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 288 KiB/s wr, 11 op/s 2023-09-10T09:41:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:41:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:38 smithi082 ceph-mon[98694]: pgmap v20067: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 288 KiB/s wr, 11 op/s 2023-09-10T09:41:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:41:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:38 smithi098 ceph-mon[103124]: pgmap v20067: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 288 KiB/s wr, 11 op/s 2023-09-10T09:41:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:41:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37330.878823 secs 2023-09-10T09:41:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37331.231398 secs 2023-09-10T09:41:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:40 smithi082 ceph-mon[98694]: pgmap v20068: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T09:41:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:41:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37330.878823 secs 2023-09-10T09:41:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37331.231398 secs 2023-09-10T09:41:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:40 smithi098 ceph-mon[103124]: pgmap v20068: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T09:41:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:41:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37330.878823 secs 2023-09-10T09:41:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37331.231398 secs 2023-09-10T09:41:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:40 smithi170 ceph-mon[102832]: pgmap v20068: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T09:41:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:41:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:42 smithi098 ceph-mon[103124]: pgmap v20069: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 594 KiB/s wr, 18 op/s 2023-09-10T09:41:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:42 smithi170 ceph-mon[102832]: pgmap v20069: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 594 KiB/s wr, 18 op/s 2023-09-10T09:41:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:42 smithi082 ceph-mon[98694]: pgmap v20069: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 594 KiB/s wr, 18 op/s 2023-09-10T09:41:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:44 smithi098 ceph-mon[103124]: pgmap v20070: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 593 KiB/s wr, 13 op/s 2023-09-10T09:41:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37335.878882 secs 2023-09-10T09:41:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:44 smithi170 ceph-mon[102832]: pgmap v20070: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 593 KiB/s wr, 13 op/s 2023-09-10T09:41:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37335.878882 secs 2023-09-10T09:41:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:44 smithi082 ceph-mon[98694]: pgmap v20070: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 593 KiB/s wr, 13 op/s 2023-09-10T09:41:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37335.878882 secs 2023-09-10T09:41:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37336.231466 secs 2023-09-10T09:41:45.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37336.231466 secs 2023-09-10T09:41:45.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37336.231466 secs 2023-09-10T09:41:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:46 smithi098 ceph-mon[103124]: pgmap v20071: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 627 KiB/s wr, 19 op/s 2023-09-10T09:41:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:46 smithi170 ceph-mon[102832]: pgmap v20071: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 627 KiB/s wr, 19 op/s 2023-09-10T09:41:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:46 smithi082 ceph-mon[98694]: pgmap v20071: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 627 KiB/s wr, 19 op/s 2023-09-10T09:41:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:48 smithi098 ceph-mon[103124]: pgmap v20072: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 362 KiB/s wr, 13 op/s 2023-09-10T09:41:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:48 smithi170 ceph-mon[102832]: pgmap v20072: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 362 KiB/s wr, 13 op/s 2023-09-10T09:41:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:48 smithi082 ceph-mon[98694]: pgmap v20072: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 362 KiB/s wr, 13 op/s 2023-09-10T09:41:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37340.878942 secs 2023-09-10T09:41:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37341.231490 secs 2023-09-10T09:41:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:50 smithi082 ceph-mon[98694]: pgmap v20073: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 362 KiB/s wr, 13 op/s 2023-09-10T09:41:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:41:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37340.878942 secs 2023-09-10T09:41:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37341.231490 secs 2023-09-10T09:41:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:50 smithi098 ceph-mon[103124]: pgmap v20073: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 362 KiB/s wr, 13 op/s 2023-09-10T09:41:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:41:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37340.878942 secs 2023-09-10T09:41:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37341.231490 secs 2023-09-10T09:41:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:50 smithi170 ceph-mon[102832]: pgmap v20073: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 362 KiB/s wr, 13 op/s 2023-09-10T09:41:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:41:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:41:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:41:51.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:41:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:41:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:41:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:41:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:41:52.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:52 smithi082 ceph-mon[98694]: pgmap v20074: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T09:41:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:41:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:41:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:41:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:41:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:52 smithi098 ceph-mon[103124]: pgmap v20074: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T09:41:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:41:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:41:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:41:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:41:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:52 smithi170 ceph-mon[102832]: pgmap v20074: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T09:41:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:54 smithi098 ceph-mon[103124]: pgmap v20075: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 745 KiB/s wr, 14 op/s 2023-09-10T09:41:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37345.879000 secs 2023-09-10T09:41:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:54 smithi170 ceph-mon[102832]: pgmap v20075: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 745 KiB/s wr, 14 op/s 2023-09-10T09:41:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37345.879000 secs 2023-09-10T09:41:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:54 smithi082 ceph-mon[98694]: pgmap v20075: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 745 KiB/s wr, 14 op/s 2023-09-10T09:41:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37345.879000 secs 2023-09-10T09:41:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37346.231554 secs 2023-09-10T09:41:55.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37346.231554 secs 2023-09-10T09:41:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37346.231554 secs 2023-09-10T09:41:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:56 smithi098 ceph-mon[103124]: pgmap v20076: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 950 KiB/s wr, 20 op/s 2023-09-10T09:41:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:56 smithi170 ceph-mon[102832]: pgmap v20076: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 950 KiB/s wr, 20 op/s 2023-09-10T09:41:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:56 smithi082 ceph-mon[98694]: pgmap v20076: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 950 KiB/s wr, 20 op/s 2023-09-10T09:41:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:58 smithi098 ceph-mon[103124]: pgmap v20077: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 916 KiB/s wr, 14 op/s 2023-09-10T09:41:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:41:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:41:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:58 smithi170 ceph-mon[102832]: pgmap v20077: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 916 KiB/s wr, 14 op/s 2023-09-10T09:41:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:41:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:41:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:41:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:58 smithi082 ceph-mon[98694]: pgmap v20077: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 916 KiB/s wr, 14 op/s 2023-09-10T09:41:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:41:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:41:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37350.879058 secs 2023-09-10T09:42:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37351.231621 secs 2023-09-10T09:42:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:00 smithi082 ceph-mon[98694]: pgmap v20078: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 916 KiB/s wr, 14 op/s 2023-09-10T09:42:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:42:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37350.879058 secs 2023-09-10T09:42:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37351.231621 secs 2023-09-10T09:42:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:00 smithi098 ceph-mon[103124]: pgmap v20078: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 916 KiB/s wr, 14 op/s 2023-09-10T09:42:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:42:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37350.879058 secs 2023-09-10T09:42:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37351.231621 secs 2023-09-10T09:42:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:00 smithi170 ceph-mon[102832]: pgmap v20078: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 916 KiB/s wr, 14 op/s 2023-09-10T09:42:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:42:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:02 smithi098 ceph-mon[103124]: pgmap v20079: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 20 op/s 2023-09-10T09:42:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:02 smithi170 ceph-mon[102832]: pgmap v20079: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 20 op/s 2023-09-10T09:42:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:02 smithi082 ceph-mon[98694]: pgmap v20079: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 20 op/s 2023-09-10T09:42:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37354.632111 secs 2023-09-10T09:42:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37354.632111 secs 2023-09-10T09:42:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37354.632111 secs 2023-09-10T09:42:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:04 smithi098 ceph-mon[103124]: pgmap v20080: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 366 KiB/s wr, 13 op/s 2023-09-10T09:42:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:42:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:42:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37355.879119 secs 2023-09-10T09:42:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37356.231683 secs 2023-09-10T09:42:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:04 smithi170 ceph-mon[102832]: pgmap v20080: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 366 KiB/s wr, 13 op/s 2023-09-10T09:42:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:42:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:42:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37355.879119 secs 2023-09-10T09:42:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37356.231683 secs 2023-09-10T09:42:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:04 smithi082 ceph-mon[98694]: pgmap v20080: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 366 KiB/s wr, 13 op/s 2023-09-10T09:42:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:42:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:42:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37355.879119 secs 2023-09-10T09:42:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37356.231683 secs 2023-09-10T09:42:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:06 smithi170 ceph-mon[102832]: pgmap v20081: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 660 KiB/s wr, 33 op/s 2023-09-10T09:42:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:42:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:06 smithi082 ceph-mon[98694]: pgmap v20081: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 660 KiB/s wr, 33 op/s 2023-09-10T09:42:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:42:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:06 smithi098 ceph-mon[103124]: pgmap v20081: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 660 KiB/s wr, 33 op/s 2023-09-10T09:42:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:42:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:08 smithi170 ceph-mon[102832]: pgmap v20082: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 455 KiB/s wr, 27 op/s 2023-09-10T09:42:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37359.632203 secs 2023-09-10T09:42:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:08 smithi082 ceph-mon[98694]: pgmap v20082: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 455 KiB/s wr, 27 op/s 2023-09-10T09:42:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37359.632203 secs 2023-09-10T09:42:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:08 smithi098 ceph-mon[103124]: pgmap v20082: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 455 KiB/s wr, 27 op/s 2023-09-10T09:42:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37359.632203 secs 2023-09-10T09:42:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:09 smithi170 ceph-mon[102832]: osdmap e1323: 12 total, 12 up, 12 in 2023-09-10T09:42:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37360.879178 secs 2023-09-10T09:42:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37361.231746 secs 2023-09-10T09:42:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:09 smithi082 ceph-mon[98694]: osdmap e1323: 12 total, 12 up, 12 in 2023-09-10T09:42:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37360.879178 secs 2023-09-10T09:42:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37361.231746 secs 2023-09-10T09:42:10.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:09 smithi098 ceph-mon[103124]: osdmap e1323: 12 total, 12 up, 12 in 2023-09-10T09:42:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37360.879178 secs 2023-09-10T09:42:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37361.231746 secs 2023-09-10T09:42:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:10 smithi170 ceph-mon[102832]: pgmap v20084: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 546 KiB/s wr, 32 op/s 2023-09-10T09:42:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:10 smithi082 ceph-mon[98694]: pgmap v20084: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 546 KiB/s wr, 32 op/s 2023-09-10T09:42:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:10 smithi098 ceph-mon[103124]: pgmap v20084: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 546 KiB/s wr, 32 op/s 2023-09-10T09:42:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:12 smithi170 ceph-mon[102832]: pgmap v20085: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 544 KiB/s wr, 32 op/s 2023-09-10T09:42:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:12 smithi082 ceph-mon[98694]: pgmap v20085: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 544 KiB/s wr, 32 op/s 2023-09-10T09:42:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:12 smithi098 ceph-mon[103124]: pgmap v20085: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 544 KiB/s wr, 32 op/s 2023-09-10T09:42:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37364.632270 secs 2023-09-10T09:42:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37364.632270 secs 2023-09-10T09:42:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37364.632270 secs 2023-09-10T09:42:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:14 smithi170 ceph-mon[102832]: pgmap v20086: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 544 KiB/s wr, 32 op/s 2023-09-10T09:42:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37365.879240 secs 2023-09-10T09:42:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37366.231811 secs 2023-09-10T09:42:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:14 smithi170 ceph-mon[102832]: osdmap e1324: 12 total, 12 up, 12 in 2023-09-10T09:42:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:14 smithi082 ceph-mon[98694]: pgmap v20086: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 544 KiB/s wr, 32 op/s 2023-09-10T09:42:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37365.879240 secs 2023-09-10T09:42:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37366.231811 secs 2023-09-10T09:42:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:14 smithi082 ceph-mon[98694]: osdmap e1324: 12 total, 12 up, 12 in 2023-09-10T09:42:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:14 smithi098 ceph-mon[103124]: pgmap v20086: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 544 KiB/s wr, 32 op/s 2023-09-10T09:42:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37365.879240 secs 2023-09-10T09:42:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37366.231811 secs 2023-09-10T09:42:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:14 smithi098 ceph-mon[103124]: osdmap e1324: 12 total, 12 up, 12 in 2023-09-10T09:42:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:16 smithi170 ceph-mon[102832]: pgmap v20088: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 281 KiB/s wr, 18 op/s 2023-09-10T09:42:16.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:42:16.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:16 smithi082 ceph-mon[98694]: pgmap v20088: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 281 KiB/s wr, 18 op/s 2023-09-10T09:42:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:42:16.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:16 smithi098 ceph-mon[103124]: pgmap v20088: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 281 KiB/s wr, 18 op/s 2023-09-10T09:42:17.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:42:17.043 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:18 smithi170 ceph-mon[102832]: pgmap v20089: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 252 KiB/s wr, 16 op/s 2023-09-10T09:42:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:42:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37369.632359 secs 2023-09-10T09:42:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:18 smithi082 ceph-mon[98694]: pgmap v20089: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 252 KiB/s wr, 16 op/s 2023-09-10T09:42:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:42:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37369.632359 secs 2023-09-10T09:42:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:18 smithi098 ceph-mon[103124]: pgmap v20089: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 252 KiB/s wr, 16 op/s 2023-09-10T09:42:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:42:19.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:19.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37369.632359 secs 2023-09-10T09:42:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37370.879297 secs 2023-09-10T09:42:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37371.231876 secs 2023-09-10T09:42:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:20 smithi082 ceph-mon[98694]: pgmap v20090: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 225 KiB/s wr, 14 op/s 2023-09-10T09:42:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37370.879297 secs 2023-09-10T09:42:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37371.231876 secs 2023-09-10T09:42:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:20 smithi098 ceph-mon[103124]: pgmap v20090: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 225 KiB/s wr, 14 op/s 2023-09-10T09:42:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37370.879297 secs 2023-09-10T09:42:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37371.231876 secs 2023-09-10T09:42:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:20 smithi170 ceph-mon[102832]: pgmap v20090: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 225 KiB/s wr, 14 op/s 2023-09-10T09:42:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:22 smithi098 ceph-mon[103124]: pgmap v20091: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 327 KiB/s wr, 14 op/s 2023-09-10T09:42:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:22 smithi170 ceph-mon[102832]: pgmap v20091: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 327 KiB/s wr, 14 op/s 2023-09-10T09:42:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:22 smithi082 ceph-mon[98694]: pgmap v20091: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 327 KiB/s wr, 14 op/s 2023-09-10T09:42:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37374.632420 secs 2023-09-10T09:42:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37374.632420 secs 2023-09-10T09:42:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37374.632420 secs 2023-09-10T09:42:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:24 smithi098 ceph-mon[103124]: pgmap v20092: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 327 KiB/s wr, 14 op/s 2023-09-10T09:42:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37375.879356 secs 2023-09-10T09:42:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37376.231950 secs 2023-09-10T09:42:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:24 smithi170 ceph-mon[102832]: pgmap v20092: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 327 KiB/s wr, 14 op/s 2023-09-10T09:42:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37375.879356 secs 2023-09-10T09:42:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37376.231950 secs 2023-09-10T09:42:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:24 smithi082 ceph-mon[98694]: pgmap v20092: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 327 KiB/s wr, 14 op/s 2023-09-10T09:42:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37375.879356 secs 2023-09-10T09:42:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37376.231950 secs 2023-09-10T09:42:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:26 smithi098 ceph-mon[103124]: pgmap v20093: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 291 KiB/s wr, 17 op/s 2023-09-10T09:42:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:26 smithi170 ceph-mon[102832]: pgmap v20093: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 291 KiB/s wr, 17 op/s 2023-09-10T09:42:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:26 smithi082 ceph-mon[98694]: pgmap v20093: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 291 KiB/s wr, 17 op/s 2023-09-10T09:42:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:28 smithi170 ceph-mon[102832]: pgmap v20094: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 245 KiB/s wr, 10 op/s 2023-09-10T09:42:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37379.632479 secs 2023-09-10T09:42:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:28 smithi082 ceph-mon[98694]: pgmap v20094: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 245 KiB/s wr, 10 op/s 2023-09-10T09:42:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37379.632479 secs 2023-09-10T09:42:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:28 smithi098 ceph-mon[103124]: pgmap v20094: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 245 KiB/s wr, 10 op/s 2023-09-10T09:42:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37379.632479 secs 2023-09-10T09:42:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37380.879445 secs 2023-09-10T09:42:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37381.232010 secs 2023-09-10T09:42:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:30 smithi082 ceph-mon[98694]: pgmap v20095: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 245 KiB/s wr, 10 op/s 2023-09-10T09:42:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:42:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37380.879445 secs 2023-09-10T09:42:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37381.232010 secs 2023-09-10T09:42:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:30 smithi098 ceph-mon[103124]: pgmap v20095: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 245 KiB/s wr, 10 op/s 2023-09-10T09:42:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:42:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37380.879445 secs 2023-09-10T09:42:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37381.232010 secs 2023-09-10T09:42:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:30 smithi170 ceph-mon[102832]: pgmap v20095: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 245 KiB/s wr, 10 op/s 2023-09-10T09:42:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:42:30.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:42:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:32 smithi082 ceph-mon[98694]: pgmap v20096: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 405 KiB/s wr, 16 op/s 2023-09-10T09:42:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:42:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:32 smithi098 ceph-mon[103124]: pgmap v20096: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 405 KiB/s wr, 16 op/s 2023-09-10T09:42:32.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:42:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:32 smithi170 ceph-mon[102832]: pgmap v20096: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 405 KiB/s wr, 16 op/s 2023-09-10T09:42:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:34 smithi098 ceph-mon[103124]: pgmap v20097: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 161 KiB/s wr, 10 op/s 2023-09-10T09:42:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37385.879504 secs 2023-09-10T09:42:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37386.232074 secs 2023-09-10T09:42:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:34 smithi170 ceph-mon[102832]: pgmap v20097: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 161 KiB/s wr, 10 op/s 2023-09-10T09:42:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37385.879504 secs 2023-09-10T09:42:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37386.232074 secs 2023-09-10T09:42:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:34 smithi082 ceph-mon[98694]: pgmap v20097: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 161 KiB/s wr, 10 op/s 2023-09-10T09:42:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37385.879504 secs 2023-09-10T09:42:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37386.232074 secs 2023-09-10T09:42:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:42:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:42:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:42:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:36 smithi098 ceph-mon[103124]: pgmap v20098: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 337 KiB/s wr, 16 op/s 2023-09-10T09:42:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:42:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:42:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:36 smithi170 ceph-mon[102832]: pgmap v20098: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 337 KiB/s wr, 16 op/s 2023-09-10T09:42:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:42:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:42:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:36 smithi082 ceph-mon[98694]: pgmap v20098: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 337 KiB/s wr, 16 op/s 2023-09-10T09:42:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:42:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:42:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:38 smithi098 ceph-mon[103124]: pgmap v20099: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 337 KiB/s wr, 11 op/s 2023-09-10T09:42:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:38 smithi170 ceph-mon[102832]: pgmap v20099: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 337 KiB/s wr, 11 op/s 2023-09-10T09:42:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:38 smithi082 ceph-mon[98694]: pgmap v20099: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 337 KiB/s wr, 11 op/s 2023-09-10T09:42:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37390.879562 secs 2023-09-10T09:42:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37391.232137 secs 2023-09-10T09:42:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:40 smithi082 ceph-mon[98694]: pgmap v20100: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 337 KiB/s wr, 11 op/s 2023-09-10T09:42:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37390.879562 secs 2023-09-10T09:42:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37391.232137 secs 2023-09-10T09:42:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:40 smithi098 ceph-mon[103124]: pgmap v20100: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 337 KiB/s wr, 11 op/s 2023-09-10T09:42:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37390.879562 secs 2023-09-10T09:42:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37391.232137 secs 2023-09-10T09:42:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:40 smithi170 ceph-mon[102832]: pgmap v20100: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 337 KiB/s wr, 11 op/s 2023-09-10T09:42:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:42 smithi098 ceph-mon[103124]: pgmap v20101: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 498 KiB/s wr, 18 op/s 2023-09-10T09:42:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:42 smithi170 ceph-mon[102832]: pgmap v20101: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 498 KiB/s wr, 18 op/s 2023-09-10T09:42:42.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:42 smithi082 ceph-mon[98694]: pgmap v20101: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 498 KiB/s wr, 18 op/s 2023-09-10T09:42:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:44 smithi098 ceph-mon[103124]: pgmap v20102: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 338 KiB/s wr, 12 op/s 2023-09-10T09:42:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:42:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37395.879622 secs 2023-09-10T09:42:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37396.232204 secs 2023-09-10T09:42:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:44 smithi170 ceph-mon[102832]: pgmap v20102: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 338 KiB/s wr, 12 op/s 2023-09-10T09:42:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:42:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37395.879622 secs 2023-09-10T09:42:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37396.232204 secs 2023-09-10T09:42:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:44 smithi082 ceph-mon[98694]: pgmap v20102: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 338 KiB/s wr, 12 op/s 2023-09-10T09:42:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:42:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37395.879622 secs 2023-09-10T09:42:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37396.232204 secs 2023-09-10T09:42:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:46 smithi170 ceph-mon[102832]: pgmap v20103: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 853 KiB/s wr, 19 op/s 2023-09-10T09:42:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:42:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:46 smithi082 ceph-mon[98694]: pgmap v20103: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 853 KiB/s wr, 19 op/s 2023-09-10T09:42:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:42:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:46 smithi098 ceph-mon[103124]: pgmap v20103: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 853 KiB/s wr, 19 op/s 2023-09-10T09:42:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:42:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:48 smithi170 ceph-mon[102832]: pgmap v20104: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 676 KiB/s wr, 13 op/s 2023-09-10T09:42:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:48 smithi082 ceph-mon[98694]: pgmap v20104: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 676 KiB/s wr, 13 op/s 2023-09-10T09:42:49.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:48 smithi098 ceph-mon[103124]: pgmap v20104: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 676 KiB/s wr, 13 op/s 2023-09-10T09:42:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37400.879683 secs 2023-09-10T09:42:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37401.232265 secs 2023-09-10T09:42:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:50 smithi082 ceph-mon[98694]: pgmap v20105: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 676 KiB/s wr, 13 op/s 2023-09-10T09:42:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:42:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37400.879683 secs 2023-09-10T09:42:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37401.232265 secs 2023-09-10T09:42:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:50 smithi098 ceph-mon[103124]: pgmap v20105: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 676 KiB/s wr, 13 op/s 2023-09-10T09:42:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:42:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37400.879683 secs 2023-09-10T09:42:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37401.232265 secs 2023-09-10T09:42:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:50 smithi170 ceph-mon[102832]: pgmap v20105: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 676 KiB/s wr, 13 op/s 2023-09-10T09:42:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:42:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:42:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:52 smithi098 ceph-mon[103124]: pgmap v20106: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:42:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:42:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:42:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:42:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:42:52.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:42:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:52 smithi170 ceph-mon[102832]: pgmap v20106: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:42:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:42:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:42:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:42:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:42:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:42:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:52 smithi082 ceph-mon[98694]: pgmap v20106: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:42:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:42:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:42:52.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:42:52.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:42:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:54 smithi098 ceph-mon[103124]: pgmap v20107: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 929 KiB/s wr, 15 op/s 2023-09-10T09:42:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37405.879740 secs 2023-09-10T09:42:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37406.232329 secs 2023-09-10T09:42:54.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:54 smithi170 ceph-mon[102832]: pgmap v20107: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 929 KiB/s wr, 15 op/s 2023-09-10T09:42:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37405.879740 secs 2023-09-10T09:42:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37406.232329 secs 2023-09-10T09:42:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:54 smithi082 ceph-mon[98694]: pgmap v20107: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 929 KiB/s wr, 15 op/s 2023-09-10T09:42:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37405.879740 secs 2023-09-10T09:42:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37406.232329 secs 2023-09-10T09:42:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:56 smithi098 ceph-mon[103124]: pgmap v20108: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 23 op/s 2023-09-10T09:42:56.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:56 smithi170 ceph-mon[102832]: pgmap v20108: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 23 op/s 2023-09-10T09:42:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:56 smithi082 ceph-mon[98694]: pgmap v20108: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 23 op/s 2023-09-10T09:42:58.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:58 smithi170 ceph-mon[102832]: pgmap v20109: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 557 KiB/s wr, 15 op/s 2023-09-10T09:42:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:42:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:42:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37409.632893 secs 2023-09-10T09:42:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:58 smithi082 ceph-mon[98694]: pgmap v20109: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 557 KiB/s wr, 15 op/s 2023-09-10T09:42:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:42:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:42:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37409.632893 secs 2023-09-10T09:42:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:58 smithi098 ceph-mon[103124]: pgmap v20109: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 557 KiB/s wr, 15 op/s 2023-09-10T09:42:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:42:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:42:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:42:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37409.632893 secs 2023-09-10T09:43:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37410.879800 secs 2023-09-10T09:43:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37411.232392 secs 2023-09-10T09:43:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:00 smithi082 ceph-mon[98694]: pgmap v20110: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 557 KiB/s wr, 15 op/s 2023-09-10T09:43:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:43:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:43:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37410.879800 secs 2023-09-10T09:43:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37411.232392 secs 2023-09-10T09:43:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:00 smithi098 ceph-mon[103124]: pgmap v20110: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 557 KiB/s wr, 15 op/s 2023-09-10T09:43:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:43:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:43:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37410.879800 secs 2023-09-10T09:43:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37411.232392 secs 2023-09-10T09:43:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:00 smithi170 ceph-mon[102832]: pgmap v20110: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 557 KiB/s wr, 15 op/s 2023-09-10T09:43:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:43:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:43:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:02 smithi098 ceph-mon[103124]: pgmap v20111: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1020 KiB/s wr, 22 op/s 2023-09-10T09:43:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:02 smithi170 ceph-mon[102832]: pgmap v20111: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1020 KiB/s wr, 22 op/s 2023-09-10T09:43:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:02 smithi082 ceph-mon[98694]: pgmap v20111: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1020 KiB/s wr, 22 op/s 2023-09-10T09:43:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37414.632951 secs 2023-09-10T09:43:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37414.632951 secs 2023-09-10T09:43:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37414.632951 secs 2023-09-10T09:43:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:04 smithi170 ceph-mon[102832]: pgmap v20112: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 606 KiB/s wr, 14 op/s 2023-09-10T09:43:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:43:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37415.879857 secs 2023-09-10T09:43:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37416.232457 secs 2023-09-10T09:43:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:04 smithi082 ceph-mon[98694]: pgmap v20112: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 606 KiB/s wr, 14 op/s 2023-09-10T09:43:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:43:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37415.879857 secs 2023-09-10T09:43:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37416.232457 secs 2023-09-10T09:43:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:04 smithi098 ceph-mon[103124]: pgmap v20112: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 606 KiB/s wr, 14 op/s 2023-09-10T09:43:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:43:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37415.879857 secs 2023-09-10T09:43:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37416.232457 secs 2023-09-10T09:43:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:06 smithi170 ceph-mon[102832]: pgmap v20113: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 965 KiB/s wr, 35 op/s 2023-09-10T09:43:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:43:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:06 smithi082 ceph-mon[98694]: pgmap v20113: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 965 KiB/s wr, 35 op/s 2023-09-10T09:43:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:43:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:06 smithi098 ceph-mon[103124]: pgmap v20113: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 965 KiB/s wr, 35 op/s 2023-09-10T09:43:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:43:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:08 smithi170 ceph-mon[102832]: pgmap v20114: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 822 KiB/s wr, 27 op/s 2023-09-10T09:43:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37419.633012 secs 2023-09-10T09:43:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:08 smithi082 ceph-mon[98694]: pgmap v20114: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 822 KiB/s wr, 27 op/s 2023-09-10T09:43:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37419.633012 secs 2023-09-10T09:43:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:08 smithi098 ceph-mon[103124]: pgmap v20114: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 822 KiB/s wr, 27 op/s 2023-09-10T09:43:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37419.633012 secs 2023-09-10T09:43:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:09 smithi170 ceph-mon[102832]: osdmap e1325: 12 total, 12 up, 12 in 2023-09-10T09:43:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37420.879915 secs 2023-09-10T09:43:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37421.232493 secs 2023-09-10T09:43:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:09 smithi082 ceph-mon[98694]: osdmap e1325: 12 total, 12 up, 12 in 2023-09-10T09:43:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37420.879915 secs 2023-09-10T09:43:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37421.232493 secs 2023-09-10T09:43:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:09 smithi098 ceph-mon[103124]: osdmap e1325: 12 total, 12 up, 12 in 2023-09-10T09:43:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37420.879915 secs 2023-09-10T09:43:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37421.232493 secs 2023-09-10T09:43:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:10 smithi170 ceph-mon[102832]: pgmap v20116: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 987 KiB/s wr, 33 op/s 2023-09-10T09:43:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:43:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:10 smithi082 ceph-mon[98694]: pgmap v20116: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 987 KiB/s wr, 33 op/s 2023-09-10T09:43:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:43:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:10 smithi098 ceph-mon[103124]: pgmap v20116: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 987 KiB/s wr, 33 op/s 2023-09-10T09:43:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:43:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:12 smithi170 ceph-mon[102832]: pgmap v20117: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 595 KiB/s wr, 33 op/s 2023-09-10T09:43:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:43:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:12 smithi082 ceph-mon[98694]: pgmap v20117: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 595 KiB/s wr, 33 op/s 2023-09-10T09:43:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:43:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:12 smithi098 ceph-mon[103124]: pgmap v20117: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 595 KiB/s wr, 33 op/s 2023-09-10T09:43:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:43:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37424.633121 secs 2023-09-10T09:43:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37424.633121 secs 2023-09-10T09:43:14.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37424.633121 secs 2023-09-10T09:43:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:14 smithi170 ceph-mon[102832]: pgmap v20118: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 595 KiB/s wr, 33 op/s 2023-09-10T09:43:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37425.879992 secs 2023-09-10T09:43:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37426.232555 secs 2023-09-10T09:43:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:14 smithi170 ceph-mon[102832]: osdmap e1326: 12 total, 12 up, 12 in 2023-09-10T09:43:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:14 smithi082 ceph-mon[98694]: pgmap v20118: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 595 KiB/s wr, 33 op/s 2023-09-10T09:43:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37425.879992 secs 2023-09-10T09:43:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37426.232555 secs 2023-09-10T09:43:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:14 smithi082 ceph-mon[98694]: osdmap e1326: 12 total, 12 up, 12 in 2023-09-10T09:43:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:14 smithi098 ceph-mon[103124]: pgmap v20118: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 595 KiB/s wr, 33 op/s 2023-09-10T09:43:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37425.879992 secs 2023-09-10T09:43:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37426.232555 secs 2023-09-10T09:43:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:14 smithi098 ceph-mon[103124]: osdmap e1326: 12 total, 12 up, 12 in 2023-09-10T09:43:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:16 smithi170 ceph-mon[102832]: pgmap v20120: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 247 KiB/s wr, 18 op/s 2023-09-10T09:43:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:16 smithi082 ceph-mon[98694]: pgmap v20120: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 247 KiB/s wr, 18 op/s 2023-09-10T09:43:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:16 smithi098 ceph-mon[103124]: pgmap v20120: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 247 KiB/s wr, 18 op/s 2023-09-10T09:43:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:18 smithi170 ceph-mon[102832]: pgmap v20121: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 222 KiB/s wr, 16 op/s 2023-09-10T09:43:18.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37429.633180 secs 2023-09-10T09:43:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:18 smithi082 ceph-mon[98694]: pgmap v20121: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 222 KiB/s wr, 16 op/s 2023-09-10T09:43:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37429.633180 secs 2023-09-10T09:43:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:18 smithi098 ceph-mon[103124]: pgmap v20121: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 222 KiB/s wr, 16 op/s 2023-09-10T09:43:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37429.633180 secs 2023-09-10T09:43:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37430.880070 secs 2023-09-10T09:43:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37431.232620 secs 2023-09-10T09:43:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:20 smithi098 ceph-mon[103124]: pgmap v20122: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 198 KiB/s wr, 14 op/s 2023-09-10T09:43:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37430.880070 secs 2023-09-10T09:43:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37431.232620 secs 2023-09-10T09:43:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:20 smithi170 ceph-mon[102832]: pgmap v20122: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 198 KiB/s wr, 14 op/s 2023-09-10T09:43:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37430.880070 secs 2023-09-10T09:43:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37431.232620 secs 2023-09-10T09:43:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:20 smithi082 ceph-mon[98694]: pgmap v20122: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 198 KiB/s wr, 14 op/s 2023-09-10T09:43:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:22 smithi170 ceph-mon[102832]: pgmap v20123: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 12 op/s 2023-09-10T09:43:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:22 smithi082 ceph-mon[98694]: pgmap v20123: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 12 op/s 2023-09-10T09:43:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:22 smithi098 ceph-mon[103124]: pgmap v20123: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 12 op/s 2023-09-10T09:43:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37434.633240 secs 2023-09-10T09:43:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37434.633240 secs 2023-09-10T09:43:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37434.633240 secs 2023-09-10T09:43:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:24 smithi170 ceph-mon[102832]: pgmap v20124: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 12 op/s 2023-09-10T09:43:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37435.880154 secs 2023-09-10T09:43:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37436.232685 secs 2023-09-10T09:43:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:24 smithi082 ceph-mon[98694]: pgmap v20124: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 12 op/s 2023-09-10T09:43:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37435.880154 secs 2023-09-10T09:43:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37436.232685 secs 2023-09-10T09:43:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:24 smithi098 ceph-mon[103124]: pgmap v20124: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 12 op/s 2023-09-10T09:43:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37435.880154 secs 2023-09-10T09:43:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37436.232685 secs 2023-09-10T09:43:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:26 smithi170 ceph-mon[102832]: pgmap v20125: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 177 KiB/s wr, 17 op/s 2023-09-10T09:43:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:26 smithi082 ceph-mon[98694]: pgmap v20125: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 177 KiB/s wr, 17 op/s 2023-09-10T09:43:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:26 smithi098 ceph-mon[103124]: pgmap v20125: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 177 KiB/s wr, 17 op/s 2023-09-10T09:43:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:28 smithi170 ceph-mon[102832]: pgmap v20126: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T09:43:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37439.633328 secs 2023-09-10T09:43:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:28 smithi082 ceph-mon[98694]: pgmap v20126: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T09:43:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37439.633328 secs 2023-09-10T09:43:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:28 smithi098 ceph-mon[103124]: pgmap v20126: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T09:43:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37439.633328 secs 2023-09-10T09:43:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37440.880213 secs 2023-09-10T09:43:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37441.232750 secs 2023-09-10T09:43:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:30 smithi098 ceph-mon[103124]: pgmap v20127: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T09:43:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37440.880213 secs 2023-09-10T09:43:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37441.232750 secs 2023-09-10T09:43:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:30 smithi170 ceph-mon[102832]: pgmap v20127: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T09:43:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37440.880213 secs 2023-09-10T09:43:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37441.232750 secs 2023-09-10T09:43:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:30 smithi082 ceph-mon[98694]: pgmap v20127: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T09:43:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:32 smithi170 ceph-mon[102832]: pgmap v20128: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 137 KiB/s wr, 15 op/s 2023-09-10T09:43:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:32 smithi082 ceph-mon[98694]: pgmap v20128: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 137 KiB/s wr, 15 op/s 2023-09-10T09:43:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:32 smithi098 ceph-mon[103124]: pgmap v20128: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 137 KiB/s wr, 15 op/s 2023-09-10T09:43:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:34 smithi170 ceph-mon[102832]: pgmap v20129: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T09:43:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:43:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37445.880268 secs 2023-09-10T09:43:34.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37446.232811 secs 2023-09-10T09:43:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:34 smithi082 ceph-mon[98694]: pgmap v20129: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T09:43:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:43:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37445.880268 secs 2023-09-10T09:43:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37446.232811 secs 2023-09-10T09:43:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:34 smithi098 ceph-mon[103124]: pgmap v20129: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T09:43:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:43:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37445.880268 secs 2023-09-10T09:43:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37446.232811 secs 2023-09-10T09:43:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:43:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:43:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:43:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:36 smithi170 ceph-mon[102832]: pgmap v20130: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 435 KiB/s wr, 16 op/s 2023-09-10T09:43:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:43:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:43:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:36 smithi082 ceph-mon[98694]: pgmap v20130: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 435 KiB/s wr, 16 op/s 2023-09-10T09:43:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:43:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:43:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:37.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:36 smithi098 ceph-mon[103124]: pgmap v20130: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 435 KiB/s wr, 16 op/s 2023-09-10T09:43:37.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:43:37.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:43:37.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:38 smithi170 ceph-mon[102832]: pgmap v20131: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 298 KiB/s wr, 10 op/s 2023-09-10T09:43:39.032 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:38 smithi098 ceph-mon[103124]: pgmap v20131: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 298 KiB/s wr, 10 op/s 2023-09-10T09:43:39.032 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:38 smithi082 ceph-mon[98694]: pgmap v20131: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 298 KiB/s wr, 10 op/s 2023-09-10T09:43:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37450.880340 secs 2023-09-10T09:43:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37451.232877 secs 2023-09-10T09:43:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:40 smithi098 ceph-mon[103124]: pgmap v20132: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 298 KiB/s wr, 10 op/s 2023-09-10T09:43:40.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37450.880340 secs 2023-09-10T09:43:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37451.232877 secs 2023-09-10T09:43:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:40 smithi170 ceph-mon[102832]: pgmap v20132: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 298 KiB/s wr, 10 op/s 2023-09-10T09:43:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37450.880340 secs 2023-09-10T09:43:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37451.232877 secs 2023-09-10T09:43:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:40 smithi082 ceph-mon[98694]: pgmap v20132: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 298 KiB/s wr, 10 op/s 2023-09-10T09:43:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:42 smithi170 ceph-mon[102832]: pgmap v20133: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 460 KiB/s wr, 17 op/s 2023-09-10T09:43:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:42 smithi082 ceph-mon[98694]: pgmap v20133: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 460 KiB/s wr, 17 op/s 2023-09-10T09:43:43.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:42 smithi098 ceph-mon[103124]: pgmap v20133: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 460 KiB/s wr, 17 op/s 2023-09-10T09:43:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:44 smithi170 ceph-mon[102832]: pgmap v20134: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 460 KiB/s wr, 12 op/s 2023-09-10T09:43:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:43:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37455.880425 secs 2023-09-10T09:43:44.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37456.232938 secs 2023-09-10T09:43:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:44 smithi082 ceph-mon[98694]: pgmap v20134: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 460 KiB/s wr, 12 op/s 2023-09-10T09:43:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:43:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37455.880425 secs 2023-09-10T09:43:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37456.232938 secs 2023-09-10T09:43:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:44 smithi098 ceph-mon[103124]: pgmap v20134: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 460 KiB/s wr, 12 op/s 2023-09-10T09:43:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:43:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37455.880425 secs 2023-09-10T09:43:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37456.232938 secs 2023-09-10T09:43:46.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:46 smithi170 ceph-mon[102832]: pgmap v20135: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 536 KiB/s wr, 19 op/s 2023-09-10T09:43:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:43:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:46 smithi082 ceph-mon[98694]: pgmap v20135: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 536 KiB/s wr, 19 op/s 2023-09-10T09:43:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:43:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:46 smithi098 ceph-mon[103124]: pgmap v20135: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 536 KiB/s wr, 19 op/s 2023-09-10T09:43:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:43:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:48 smithi170 ceph-mon[102832]: pgmap v20136: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 238 KiB/s wr, 13 op/s 2023-09-10T09:43:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:48 smithi082 ceph-mon[98694]: pgmap v20136: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 238 KiB/s wr, 13 op/s 2023-09-10T09:43:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:48 smithi098 ceph-mon[103124]: pgmap v20136: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 238 KiB/s wr, 13 op/s 2023-09-10T09:43:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37460.880487 secs 2023-09-10T09:43:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37461.233003 secs 2023-09-10T09:43:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:50 smithi098 ceph-mon[103124]: pgmap v20137: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 238 KiB/s wr, 13 op/s 2023-09-10T09:43:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:43:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37460.880487 secs 2023-09-10T09:43:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37461.233003 secs 2023-09-10T09:43:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:50 smithi170 ceph-mon[102832]: pgmap v20137: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 238 KiB/s wr, 13 op/s 2023-09-10T09:43:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:43:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37460.880487 secs 2023-09-10T09:43:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37461.233003 secs 2023-09-10T09:43:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:50 smithi082 ceph-mon[98694]: pgmap v20137: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 238 KiB/s wr, 13 op/s 2023-09-10T09:43:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:43:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:43:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:52 smithi098 ceph-mon[103124]: pgmap v20138: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 746 KiB/s wr, 21 op/s 2023-09-10T09:43:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:43:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:43:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:52 smithi170 ceph-mon[102832]: pgmap v20138: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 746 KiB/s wr, 21 op/s 2023-09-10T09:43:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:43:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:43:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:52 smithi082 ceph-mon[98694]: pgmap v20138: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 746 KiB/s wr, 21 op/s 2023-09-10T09:43:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:43:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:43:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:43:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:43:53.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:43:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:43:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:43:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:43:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:43:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:43:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:54 smithi098 ceph-mon[103124]: pgmap v20139: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 584 KiB/s wr, 14 op/s 2023-09-10T09:43:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:54 smithi170 ceph-mon[102832]: pgmap v20139: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 584 KiB/s wr, 14 op/s 2023-09-10T09:43:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:54 smithi082 ceph-mon[98694]: pgmap v20139: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 584 KiB/s wr, 14 op/s 2023-09-10T09:43:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37465.880545 secs 2023-09-10T09:43:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37466.233065 secs 2023-09-10T09:43:55.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37465.880545 secs 2023-09-10T09:43:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37466.233065 secs 2023-09-10T09:43:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37465.880545 secs 2023-09-10T09:43:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37466.233065 secs 2023-09-10T09:43:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:56 smithi098 ceph-mon[103124]: pgmap v20140: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 684 KiB/s wr, 19 op/s 2023-09-10T09:43:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:56 smithi170 ceph-mon[102832]: pgmap v20140: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 684 KiB/s wr, 19 op/s 2023-09-10T09:43:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:56 smithi082 ceph-mon[98694]: pgmap v20140: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 684 KiB/s wr, 19 op/s 2023-09-10T09:43:58.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:43:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:43:58 smithi170 ceph-mon[102832]: pgmap v20141: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 608 KiB/s wr, 12 op/s 2023-09-10T09:43:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:43:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:43:58 smithi082 ceph-mon[98694]: pgmap v20141: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 608 KiB/s wr, 12 op/s 2023-09-10T09:43:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:43:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:43:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:43:58 smithi098 ceph-mon[103124]: pgmap v20141: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 608 KiB/s wr, 12 op/s 2023-09-10T09:44:00.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37470.880606 secs 2023-09-10T09:44:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37471.233129 secs 2023-09-10T09:44:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:44:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:00 smithi170 ceph-mon[102832]: pgmap v20142: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 608 KiB/s wr, 12 op/s 2023-09-10T09:44:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37470.880606 secs 2023-09-10T09:44:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37471.233129 secs 2023-09-10T09:44:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:44:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:00 smithi082 ceph-mon[98694]: pgmap v20142: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 608 KiB/s wr, 12 op/s 2023-09-10T09:44:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37470.880606 secs 2023-09-10T09:44:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37471.233129 secs 2023-09-10T09:44:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:44:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:00 smithi098 ceph-mon[103124]: pgmap v20142: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 608 KiB/s wr, 12 op/s 2023-09-10T09:44:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:02 smithi170 ceph-mon[102832]: pgmap v20143: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 20 op/s 2023-09-10T09:44:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:02 smithi082 ceph-mon[98694]: pgmap v20143: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 20 op/s 2023-09-10T09:44:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:02 smithi098 ceph-mon[103124]: pgmap v20143: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 20 op/s 2023-09-10T09:44:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37474.633771 secs 2023-09-10T09:44:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37474.633771 secs 2023-09-10T09:44:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37474.633771 secs 2023-09-10T09:44:04.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:04 smithi170 ceph-mon[102832]: pgmap v20144: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 835 KiB/s wr, 12 op/s 2023-09-10T09:44:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:44:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:44:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37475.880666 secs 2023-09-10T09:44:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37476.233195 secs 2023-09-10T09:44:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:04 smithi082 ceph-mon[98694]: pgmap v20144: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 835 KiB/s wr, 12 op/s 2023-09-10T09:44:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:44:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:44:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37475.880666 secs 2023-09-10T09:44:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37476.233195 secs 2023-09-10T09:44:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:04 smithi098 ceph-mon[103124]: pgmap v20144: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 835 KiB/s wr, 12 op/s 2023-09-10T09:44:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:44:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:44:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37475.880666 secs 2023-09-10T09:44:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37476.233195 secs 2023-09-10T09:44:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:06 smithi170 ceph-mon[102832]: pgmap v20145: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1012 KiB/s wr, 33 op/s 2023-09-10T09:44:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:06 smithi082 ceph-mon[98694]: pgmap v20145: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1012 KiB/s wr, 33 op/s 2023-09-10T09:44:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:06 smithi098 ceph-mon[103124]: pgmap v20145: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1012 KiB/s wr, 33 op/s 2023-09-10T09:44:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:08 smithi170 ceph-mon[102832]: pgmap v20146: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 912 KiB/s wr, 27 op/s 2023-09-10T09:44:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37479.633830 secs 2023-09-10T09:44:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:08 smithi082 ceph-mon[98694]: pgmap v20146: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 912 KiB/s wr, 27 op/s 2023-09-10T09:44:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37479.633830 secs 2023-09-10T09:44:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:08 smithi098 ceph-mon[103124]: pgmap v20146: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 912 KiB/s wr, 27 op/s 2023-09-10T09:44:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37479.633830 secs 2023-09-10T09:44:09.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:09 smithi170 ceph-mon[102832]: osdmap e1327: 12 total, 12 up, 12 in 2023-09-10T09:44:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37480.880728 secs 2023-09-10T09:44:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37481.233258 secs 2023-09-10T09:44:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:09 smithi082 ceph-mon[98694]: osdmap e1327: 12 total, 12 up, 12 in 2023-09-10T09:44:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37480.880728 secs 2023-09-10T09:44:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37481.233258 secs 2023-09-10T09:44:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:09 smithi098 ceph-mon[103124]: osdmap e1327: 12 total, 12 up, 12 in 2023-09-10T09:44:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37480.880728 secs 2023-09-10T09:44:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37481.233258 secs 2023-09-10T09:44:10.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:10 smithi170 ceph-mon[102832]: pgmap v20148: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 33 op/s 2023-09-10T09:44:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:44:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:10 smithi082 ceph-mon[98694]: pgmap v20148: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 33 op/s 2023-09-10T09:44:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:44:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:10 smithi098 ceph-mon[103124]: pgmap v20148: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 33 op/s 2023-09-10T09:44:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:44:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:12.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:12 smithi170 ceph-mon[102832]: pgmap v20149: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 442 KiB/s wr, 32 op/s 2023-09-10T09:44:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:44:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:12 smithi082 ceph-mon[98694]: pgmap v20149: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 442 KiB/s wr, 32 op/s 2023-09-10T09:44:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:44:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:12 smithi098 ceph-mon[103124]: pgmap v20149: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 442 KiB/s wr, 32 op/s 2023-09-10T09:44:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:44:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37484.633893 secs 2023-09-10T09:44:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37484.633893 secs 2023-09-10T09:44:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37484.633893 secs 2023-09-10T09:44:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:14 smithi170 ceph-mon[102832]: pgmap v20150: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 442 KiB/s wr, 32 op/s 2023-09-10T09:44:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37485.880799 secs 2023-09-10T09:44:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37486.233319 secs 2023-09-10T09:44:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:14 smithi170 ceph-mon[102832]: osdmap e1328: 12 total, 12 up, 12 in 2023-09-10T09:44:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:14 smithi082 ceph-mon[98694]: pgmap v20150: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 442 KiB/s wr, 32 op/s 2023-09-10T09:44:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37485.880799 secs 2023-09-10T09:44:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37486.233319 secs 2023-09-10T09:44:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:14 smithi082 ceph-mon[98694]: osdmap e1328: 12 total, 12 up, 12 in 2023-09-10T09:44:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:14 smithi098 ceph-mon[103124]: pgmap v20150: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 442 KiB/s wr, 32 op/s 2023-09-10T09:44:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37485.880799 secs 2023-09-10T09:44:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37486.233319 secs 2023-09-10T09:44:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:14 smithi098 ceph-mon[103124]: osdmap e1328: 12 total, 12 up, 12 in 2023-09-10T09:44:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:16 smithi170 ceph-mon[102832]: pgmap v20152: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 494 KiB/s wr, 18 op/s 2023-09-10T09:44:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:16 smithi082 ceph-mon[98694]: pgmap v20152: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 494 KiB/s wr, 18 op/s 2023-09-10T09:44:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:16 smithi098 ceph-mon[103124]: pgmap v20152: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 494 KiB/s wr, 18 op/s 2023-09-10T09:44:18.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:18 smithi170 ceph-mon[102832]: pgmap v20153: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 443 KiB/s wr, 17 op/s 2023-09-10T09:44:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37489.633970 secs 2023-09-10T09:44:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:18 smithi082 ceph-mon[98694]: pgmap v20153: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 443 KiB/s wr, 17 op/s 2023-09-10T09:44:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37489.633970 secs 2023-09-10T09:44:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:18 smithi098 ceph-mon[103124]: pgmap v20153: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 443 KiB/s wr, 17 op/s 2023-09-10T09:44:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37489.633970 secs 2023-09-10T09:44:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37490.880856 secs 2023-09-10T09:44:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37491.233384 secs 2023-09-10T09:44:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:20 smithi098 ceph-mon[103124]: pgmap v20154: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 395 KiB/s wr, 15 op/s 2023-09-10T09:44:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37490.880856 secs 2023-09-10T09:44:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37491.233384 secs 2023-09-10T09:44:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:20 smithi170 ceph-mon[102832]: pgmap v20154: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 395 KiB/s wr, 15 op/s 2023-09-10T09:44:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37490.880856 secs 2023-09-10T09:44:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37491.233384 secs 2023-09-10T09:44:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:20 smithi082 ceph-mon[98694]: pgmap v20154: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 395 KiB/s wr, 15 op/s 2023-09-10T09:44:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:22 smithi170 ceph-mon[102832]: pgmap v20155: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 165 KiB/s wr, 14 op/s 2023-09-10T09:44:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:22 smithi082 ceph-mon[98694]: pgmap v20155: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 165 KiB/s wr, 14 op/s 2023-09-10T09:44:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:22 smithi098 ceph-mon[103124]: pgmap v20155: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 165 KiB/s wr, 14 op/s 2023-09-10T09:44:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37494.634032 secs 2023-09-10T09:44:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37494.634032 secs 2023-09-10T09:44:24.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37494.634032 secs 2023-09-10T09:44:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:24 smithi170 ceph-mon[102832]: pgmap v20156: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 165 KiB/s wr, 14 op/s 2023-09-10T09:44:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:44:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37495.880914 secs 2023-09-10T09:44:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37496.233444 secs 2023-09-10T09:44:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:24 smithi082 ceph-mon[98694]: pgmap v20156: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 165 KiB/s wr, 14 op/s 2023-09-10T09:44:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:44:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37495.880914 secs 2023-09-10T09:44:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37496.233444 secs 2023-09-10T09:44:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:24 smithi098 ceph-mon[103124]: pgmap v20156: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 165 KiB/s wr, 14 op/s 2023-09-10T09:44:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:44:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37495.880914 secs 2023-09-10T09:44:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37496.233444 secs 2023-09-10T09:44:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:26 smithi170 ceph-mon[102832]: pgmap v20157: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 19 op/s 2023-09-10T09:44:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:26 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:44:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:26 smithi082 ceph-mon[98694]: pgmap v20157: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 19 op/s 2023-09-10T09:44:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:26 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:44:26.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:26 smithi098 ceph-mon[103124]: pgmap v20157: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 19 op/s 2023-09-10T09:44:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:26 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:44:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:28.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:28 smithi170 ceph-mon[102832]: pgmap v20158: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T09:44:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37499.634124 secs 2023-09-10T09:44:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:28 smithi082 ceph-mon[98694]: pgmap v20158: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T09:44:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37499.634124 secs 2023-09-10T09:44:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:28 smithi098 ceph-mon[103124]: pgmap v20158: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T09:44:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37499.634124 secs 2023-09-10T09:44:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37500.880973 secs 2023-09-10T09:44:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37501.233482 secs 2023-09-10T09:44:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:30 smithi098 ceph-mon[103124]: pgmap v20159: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T09:44:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37500.880973 secs 2023-09-10T09:44:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37501.233482 secs 2023-09-10T09:44:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:30 smithi170 ceph-mon[102832]: pgmap v20159: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T09:44:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37500.880973 secs 2023-09-10T09:44:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37501.233482 secs 2023-09-10T09:44:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:30 smithi082 ceph-mon[98694]: pgmap v20159: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T09:44:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:32 smithi170 ceph-mon[102832]: pgmap v20160: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 16 op/s 2023-09-10T09:44:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:32 smithi082 ceph-mon[98694]: pgmap v20160: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 16 op/s 2023-09-10T09:44:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:32 smithi098 ceph-mon[103124]: pgmap v20160: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 138 KiB/s wr, 16 op/s 2023-09-10T09:44:34.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:34 smithi170 ceph-mon[102832]: pgmap v20161: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T09:44:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37505.881045 secs 2023-09-10T09:44:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37506.233544 secs 2023-09-10T09:44:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:34 smithi082 ceph-mon[98694]: pgmap v20161: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T09:44:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37505.881045 secs 2023-09-10T09:44:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37506.233544 secs 2023-09-10T09:44:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:34 smithi098 ceph-mon[103124]: pgmap v20161: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 137 KiB/s wr, 10 op/s 2023-09-10T09:44:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37505.881045 secs 2023-09-10T09:44:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37506.233544 secs 2023-09-10T09:44:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:44:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:44:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:44:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:36 smithi170 ceph-mon[102832]: pgmap v20162: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 404 KiB/s wr, 16 op/s 2023-09-10T09:44:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:44:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:44:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:36 smithi082 ceph-mon[98694]: pgmap v20162: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 404 KiB/s wr, 16 op/s 2023-09-10T09:44:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:44:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:44:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:36 smithi098 ceph-mon[103124]: pgmap v20162: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 404 KiB/s wr, 16 op/s 2023-09-10T09:44:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:44:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:44:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:38.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:38 smithi170 ceph-mon[102832]: pgmap v20163: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 267 KiB/s wr, 10 op/s 2023-09-10T09:44:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:44:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:38 smithi082 ceph-mon[98694]: pgmap v20163: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 267 KiB/s wr, 10 op/s 2023-09-10T09:44:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:44:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:38 smithi098 ceph-mon[103124]: pgmap v20163: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 267 KiB/s wr, 10 op/s 2023-09-10T09:44:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:44:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37510.881104 secs 2023-09-10T09:44:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37511.233608 secs 2023-09-10T09:44:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:40 smithi098 ceph-mon[103124]: pgmap v20164: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 267 KiB/s wr, 10 op/s 2023-09-10T09:44:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:44:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37510.881104 secs 2023-09-10T09:44:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37511.233608 secs 2023-09-10T09:44:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:40 smithi170 ceph-mon[102832]: pgmap v20164: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 267 KiB/s wr, 10 op/s 2023-09-10T09:44:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:44:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37510.881104 secs 2023-09-10T09:44:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37511.233608 secs 2023-09-10T09:44:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:40 smithi082 ceph-mon[98694]: pgmap v20164: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 267 KiB/s wr, 10 op/s 2023-09-10T09:44:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:44:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:42.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:42 smithi170 ceph-mon[102832]: pgmap v20165: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 596 KiB/s wr, 18 op/s 2023-09-10T09:44:42.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:42 smithi082 ceph-mon[98694]: pgmap v20165: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 596 KiB/s wr, 18 op/s 2023-09-10T09:44:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:42 smithi098 ceph-mon[103124]: pgmap v20165: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 596 KiB/s wr, 18 op/s 2023-09-10T09:44:44.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:44 smithi170 ceph-mon[102832]: pgmap v20166: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 596 KiB/s wr, 13 op/s 2023-09-10T09:44:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:44:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37515.881163 secs 2023-09-10T09:44:44.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37516.233671 secs 2023-09-10T09:44:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:44 smithi082 ceph-mon[98694]: pgmap v20166: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 596 KiB/s wr, 13 op/s 2023-09-10T09:44:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:44:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37515.881163 secs 2023-09-10T09:44:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37516.233671 secs 2023-09-10T09:44:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:44 smithi098 ceph-mon[103124]: pgmap v20166: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 596 KiB/s wr, 13 op/s 2023-09-10T09:44:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:44:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37515.881163 secs 2023-09-10T09:44:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37516.233671 secs 2023-09-10T09:44:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:46 smithi170 ceph-mon[102832]: pgmap v20167: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 622 KiB/s wr, 20 op/s 2023-09-10T09:44:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:44:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:46 smithi082 ceph-mon[98694]: pgmap v20167: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 622 KiB/s wr, 20 op/s 2023-09-10T09:44:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:44:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:46 smithi098 ceph-mon[103124]: pgmap v20167: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 622 KiB/s wr, 20 op/s 2023-09-10T09:44:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:44:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:48 smithi170 ceph-mon[102832]: pgmap v20168: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 355 KiB/s wr, 14 op/s 2023-09-10T09:44:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:48 smithi082 ceph-mon[98694]: pgmap v20168: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 355 KiB/s wr, 14 op/s 2023-09-10T09:44:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:48 smithi098 ceph-mon[103124]: pgmap v20168: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 355 KiB/s wr, 14 op/s 2023-09-10T09:44:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37520.881216 secs 2023-09-10T09:44:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37521.233736 secs 2023-09-10T09:44:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:50 smithi098 ceph-mon[103124]: pgmap v20169: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 355 KiB/s wr, 14 op/s 2023-09-10T09:44:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:44:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37520.881216 secs 2023-09-10T09:44:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37521.233736 secs 2023-09-10T09:44:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:50 smithi170 ceph-mon[102832]: pgmap v20169: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 355 KiB/s wr, 14 op/s 2023-09-10T09:44:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:44:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37520.881216 secs 2023-09-10T09:44:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37521.233736 secs 2023-09-10T09:44:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:50 smithi082 ceph-mon[98694]: pgmap v20169: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 355 KiB/s wr, 14 op/s 2023-09-10T09:44:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:44:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:44:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:52 smithi098 ceph-mon[103124]: pgmap v20170: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:44:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:44:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:52 smithi170 ceph-mon[102832]: pgmap v20170: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:44:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:44:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:52 smithi082 ceph-mon[98694]: pgmap v20170: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:44:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:44:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:44:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:44:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:54 smithi098 ceph-mon[103124]: pgmap v20171: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 771 KiB/s wr, 14 op/s 2023-09-10T09:44:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:44:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:44:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:44:54.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:54 smithi170 ceph-mon[102832]: pgmap v20171: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 771 KiB/s wr, 14 op/s 2023-09-10T09:44:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:44:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:44:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:44:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:54 smithi082 ceph-mon[98694]: pgmap v20171: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 771 KiB/s wr, 14 op/s 2023-09-10T09:44:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:44:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:44:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:44:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37525.881278 secs 2023-09-10T09:44:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37526.233797 secs 2023-09-10T09:44:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37525.881278 secs 2023-09-10T09:44:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37526.233797 secs 2023-09-10T09:44:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37525.881278 secs 2023-09-10T09:44:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37526.233797 secs 2023-09-10T09:44:56.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:44:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:56 smithi170 ceph-mon[102832]: pgmap v20172: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 825 KiB/s wr, 21 op/s 2023-09-10T09:44:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:44:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:56 smithi082 ceph-mon[98694]: pgmap v20172: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 825 KiB/s wr, 21 op/s 2023-09-10T09:44:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:44:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:56 smithi098 ceph-mon[103124]: pgmap v20172: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 825 KiB/s wr, 21 op/s 2023-09-10T09:44:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:44:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:58 smithi170 ceph-mon[102832]: pgmap v20173: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 799 KiB/s wr, 14 op/s 2023-09-10T09:44:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:44:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37529.634630 secs 2023-09-10T09:44:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:44:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:58 smithi082 ceph-mon[98694]: pgmap v20173: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 799 KiB/s wr, 14 op/s 2023-09-10T09:44:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:44:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37529.634630 secs 2023-09-10T09:44:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:44:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:44:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:58 smithi098 ceph-mon[103124]: pgmap v20173: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 799 KiB/s wr, 14 op/s 2023-09-10T09:44:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:44:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37529.634630 secs 2023-09-10T09:45:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37530.881334 secs 2023-09-10T09:45:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37531.233861 secs 2023-09-10T09:45:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:45:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:45:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:00 smithi098 ceph-mon[103124]: pgmap v20174: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 799 KiB/s wr, 14 op/s 2023-09-10T09:45:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37530.881334 secs 2023-09-10T09:45:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37531.233861 secs 2023-09-10T09:45:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:45:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:45:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:00 smithi170 ceph-mon[102832]: pgmap v20174: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 799 KiB/s wr, 14 op/s 2023-09-10T09:45:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37530.881334 secs 2023-09-10T09:45:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37531.233861 secs 2023-09-10T09:45:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:45:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:45:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:00 smithi082 ceph-mon[98694]: pgmap v20174: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 799 KiB/s wr, 14 op/s 2023-09-10T09:45:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:02 smithi170 ceph-mon[102832]: pgmap v20175: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 24 op/s 2023-09-10T09:45:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:02 smithi082 ceph-mon[98694]: pgmap v20175: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 24 op/s 2023-09-10T09:45:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:02 smithi098 ceph-mon[103124]: pgmap v20175: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 24 op/s 2023-09-10T09:45:03.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37534.634689 secs 2023-09-10T09:45:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37534.634689 secs 2023-09-10T09:45:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37534.634689 secs 2023-09-10T09:45:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:04 smithi170 ceph-mon[102832]: pgmap v20176: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 608 KiB/s wr, 16 op/s 2023-09-10T09:45:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:45:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:04 smithi170 ceph-mon[102832]: osdmap e1329: 12 total, 12 up, 12 in 2023-09-10T09:45:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37535.881392 secs 2023-09-10T09:45:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37536.233922 secs 2023-09-10T09:45:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:04 smithi082 ceph-mon[98694]: pgmap v20176: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 608 KiB/s wr, 16 op/s 2023-09-10T09:45:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:45:04.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:04.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:04 smithi082 ceph-mon[98694]: osdmap e1329: 12 total, 12 up, 12 in 2023-09-10T09:45:04.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37535.881392 secs 2023-09-10T09:45:04.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37536.233922 secs 2023-09-10T09:45:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:04 smithi098 ceph-mon[103124]: pgmap v20176: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 608 KiB/s wr, 16 op/s 2023-09-10T09:45:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:45:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:04 smithi098 ceph-mon[103124]: osdmap e1329: 12 total, 12 up, 12 in 2023-09-10T09:45:05.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37535.881392 secs 2023-09-10T09:45:05.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37536.233922 secs 2023-09-10T09:45:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:06 smithi170 ceph-mon[102832]: pgmap v20178: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 1.0 MiB/s wr, 32 op/s 2023-09-10T09:45:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:45:06.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:06 smithi082 ceph-mon[98694]: pgmap v20178: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 1.0 MiB/s wr, 32 op/s 2023-09-10T09:45:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:45:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:06 smithi098 ceph-mon[103124]: pgmap v20178: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 1.0 MiB/s wr, 32 op/s 2023-09-10T09:45:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:45:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:08 smithi170 ceph-mon[102832]: pgmap v20179: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 1.0 MiB/s wr, 32 op/s 2023-09-10T09:45:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37539.634778 secs 2023-09-10T09:45:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:08 smithi082 ceph-mon[98694]: pgmap v20179: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 1.0 MiB/s wr, 32 op/s 2023-09-10T09:45:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37539.634778 secs 2023-09-10T09:45:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:08 smithi098 ceph-mon[103124]: pgmap v20179: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 1.0 MiB/s wr, 32 op/s 2023-09-10T09:45:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37539.634778 secs 2023-09-10T09:45:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37540.881481 secs 2023-09-10T09:45:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37541.233985 secs 2023-09-10T09:45:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:10 smithi098 ceph-mon[103124]: osdmap e1330: 12 total, 12 up, 12 in 2023-09-10T09:45:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:10 smithi098 ceph-mon[103124]: pgmap v20181: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 478 KiB/s wr, 26 op/s 2023-09-10T09:45:10.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37540.881481 secs 2023-09-10T09:45:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37541.233985 secs 2023-09-10T09:45:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:10 smithi170 ceph-mon[102832]: osdmap e1330: 12 total, 12 up, 12 in 2023-09-10T09:45:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:10 smithi170 ceph-mon[102832]: pgmap v20181: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 478 KiB/s wr, 26 op/s 2023-09-10T09:45:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37540.881481 secs 2023-09-10T09:45:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37541.233985 secs 2023-09-10T09:45:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:10 smithi082 ceph-mon[98694]: osdmap e1330: 12 total, 12 up, 12 in 2023-09-10T09:45:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:10 smithi082 ceph-mon[98694]: pgmap v20181: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 478 KiB/s wr, 26 op/s 2023-09-10T09:45:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:12 smithi170 ceph-mon[102832]: pgmap v20182: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 684 KiB/s wr, 36 op/s 2023-09-10T09:45:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:12 smithi082 ceph-mon[98694]: pgmap v20182: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 684 KiB/s wr, 36 op/s 2023-09-10T09:45:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:12 smithi098 ceph-mon[103124]: pgmap v20182: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 684 KiB/s wr, 36 op/s 2023-09-10T09:45:13.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37544.634840 secs 2023-09-10T09:45:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37544.634840 secs 2023-09-10T09:45:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37544.634840 secs 2023-09-10T09:45:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:14 smithi170 ceph-mon[102832]: pgmap v20183: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 551 KiB/s wr, 29 op/s 2023-09-10T09:45:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37545.881562 secs 2023-09-10T09:45:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37546.234048 secs 2023-09-10T09:45:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:14 smithi082 ceph-mon[98694]: pgmap v20183: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 551 KiB/s wr, 29 op/s 2023-09-10T09:45:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37545.881562 secs 2023-09-10T09:45:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37546.234048 secs 2023-09-10T09:45:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:14 smithi098 ceph-mon[103124]: pgmap v20183: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 551 KiB/s wr, 29 op/s 2023-09-10T09:45:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37545.881562 secs 2023-09-10T09:45:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37546.234048 secs 2023-09-10T09:45:16.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:16 smithi170 ceph-mon[102832]: pgmap v20184: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 14 op/s 2023-09-10T09:45:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:16 smithi082 ceph-mon[98694]: pgmap v20184: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 14 op/s 2023-09-10T09:45:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:16 smithi098 ceph-mon[103124]: pgmap v20184: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 14 op/s 2023-09-10T09:45:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:18 smithi170 ceph-mon[102832]: pgmap v20185: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 14 op/s 2023-09-10T09:45:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:45:18.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:18.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37549.634922 secs 2023-09-10T09:45:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:18 smithi082 ceph-mon[98694]: pgmap v20185: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 14 op/s 2023-09-10T09:45:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:45:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37549.634922 secs 2023-09-10T09:45:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:18 smithi098 ceph-mon[103124]: pgmap v20185: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 198 KiB/s wr, 14 op/s 2023-09-10T09:45:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:45:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37549.634922 secs 2023-09-10T09:45:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37550.881620 secs 2023-09-10T09:45:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37551.234113 secs 2023-09-10T09:45:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:20 smithi098 ceph-mon[103124]: pgmap v20186: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 14 op/s 2023-09-10T09:45:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:45:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:20.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37550.881620 secs 2023-09-10T09:45:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37551.234113 secs 2023-09-10T09:45:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:20 smithi170 ceph-mon[102832]: pgmap v20186: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 14 op/s 2023-09-10T09:45:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:45:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37550.881620 secs 2023-09-10T09:45:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37551.234113 secs 2023-09-10T09:45:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:20 smithi082 ceph-mon[98694]: pgmap v20186: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 14 op/s 2023-09-10T09:45:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:45:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:22.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:22 smithi170 ceph-mon[102832]: pgmap v20187: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 303 KiB/s wr, 18 op/s 2023-09-10T09:45:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:22 smithi082 ceph-mon[98694]: pgmap v20187: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 303 KiB/s wr, 18 op/s 2023-09-10T09:45:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:22 smithi098 ceph-mon[103124]: pgmap v20187: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 303 KiB/s wr, 18 op/s 2023-09-10T09:45:23.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37554.634987 secs 2023-09-10T09:45:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37554.634987 secs 2023-09-10T09:45:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37554.634987 secs 2023-09-10T09:45:24.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:24 smithi170 ceph-mon[102832]: pgmap v20188: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T09:45:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37555.881689 secs 2023-09-10T09:45:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37556.234178 secs 2023-09-10T09:45:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:24 smithi082 ceph-mon[98694]: pgmap v20188: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T09:45:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37555.881689 secs 2023-09-10T09:45:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37556.234178 secs 2023-09-10T09:45:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:24 smithi098 ceph-mon[103124]: pgmap v20188: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T09:45:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37555.881689 secs 2023-09-10T09:45:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37556.234178 secs 2023-09-10T09:45:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:26 smithi170 ceph-mon[102832]: pgmap v20189: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 165 KiB/s wr, 16 op/s 2023-09-10T09:45:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:26 smithi082 ceph-mon[98694]: pgmap v20189: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 165 KiB/s wr, 16 op/s 2023-09-10T09:45:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:26 smithi098 ceph-mon[103124]: pgmap v20189: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 165 KiB/s wr, 16 op/s 2023-09-10T09:45:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:28 smithi170 ceph-mon[102832]: pgmap v20190: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T09:45:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37559.635045 secs 2023-09-10T09:45:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:28 smithi082 ceph-mon[98694]: pgmap v20190: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T09:45:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37559.635045 secs 2023-09-10T09:45:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:28 smithi098 ceph-mon[103124]: pgmap v20190: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T09:45:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37559.635045 secs 2023-09-10T09:45:30.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37560.881747 secs 2023-09-10T09:45:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37561.234241 secs 2023-09-10T09:45:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:30 smithi170 ceph-mon[102832]: pgmap v20191: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T09:45:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37560.881747 secs 2023-09-10T09:45:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37561.234241 secs 2023-09-10T09:45:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:30 smithi082 ceph-mon[98694]: pgmap v20191: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T09:45:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37560.881747 secs 2023-09-10T09:45:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37561.234241 secs 2023-09-10T09:45:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:30 smithi098 ceph-mon[103124]: pgmap v20191: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T09:45:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:32 smithi170 ceph-mon[102832]: pgmap v20192: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 299 KiB/s wr, 16 op/s 2023-09-10T09:45:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:32 smithi082 ceph-mon[98694]: pgmap v20192: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 299 KiB/s wr, 16 op/s 2023-09-10T09:45:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:32 smithi098 ceph-mon[103124]: pgmap v20192: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 299 KiB/s wr, 16 op/s 2023-09-10T09:45:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:34 smithi170 ceph-mon[102832]: pgmap v20193: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T09:45:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37565.881804 secs 2023-09-10T09:45:34.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37566.234309 secs 2023-09-10T09:45:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:34 smithi082 ceph-mon[98694]: pgmap v20193: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T09:45:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37565.881804 secs 2023-09-10T09:45:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37566.234309 secs 2023-09-10T09:45:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:34 smithi098 ceph-mon[103124]: pgmap v20193: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 162 KiB/s wr, 10 op/s 2023-09-10T09:45:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37565.881804 secs 2023-09-10T09:45:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37566.234309 secs 2023-09-10T09:45:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:45:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:45:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:45:36.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:36 smithi170 ceph-mon[102832]: pgmap v20194: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 339 KiB/s wr, 16 op/s 2023-09-10T09:45:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:45:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:45:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:36 smithi082 ceph-mon[98694]: pgmap v20194: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 339 KiB/s wr, 16 op/s 2023-09-10T09:45:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:45:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:45:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:36 smithi098 ceph-mon[103124]: pgmap v20194: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 339 KiB/s wr, 16 op/s 2023-09-10T09:45:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:45:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:45:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:38 smithi170 ceph-mon[102832]: pgmap v20195: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 339 KiB/s wr, 11 op/s 2023-09-10T09:45:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:45:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:38 smithi082 ceph-mon[98694]: pgmap v20195: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 339 KiB/s wr, 11 op/s 2023-09-10T09:45:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:45:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:38 smithi098 ceph-mon[103124]: pgmap v20195: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 339 KiB/s wr, 11 op/s 2023-09-10T09:45:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:45:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37570.881903 secs 2023-09-10T09:45:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37571.234373 secs 2023-09-10T09:45:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:40 smithi170 ceph-mon[102832]: pgmap v20196: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 339 KiB/s wr, 11 op/s 2023-09-10T09:45:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:45:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37570.881903 secs 2023-09-10T09:45:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37571.234373 secs 2023-09-10T09:45:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:40 smithi082 ceph-mon[98694]: pgmap v20196: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 339 KiB/s wr, 11 op/s 2023-09-10T09:45:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:45:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37570.881903 secs 2023-09-10T09:45:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37571.234373 secs 2023-09-10T09:45:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:40 smithi098 ceph-mon[103124]: pgmap v20196: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 339 KiB/s wr, 11 op/s 2023-09-10T09:45:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:45:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:42.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:42 smithi170 ceph-mon[102832]: pgmap v20197: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 501 KiB/s wr, 17 op/s 2023-09-10T09:45:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:42 smithi082 ceph-mon[98694]: pgmap v20197: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 501 KiB/s wr, 17 op/s 2023-09-10T09:45:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:42 smithi098 ceph-mon[103124]: pgmap v20197: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 501 KiB/s wr, 17 op/s 2023-09-10T09:45:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:44 smithi170 ceph-mon[102832]: pgmap v20198: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 340 KiB/s wr, 12 op/s 2023-09-10T09:45:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:45:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37575.881961 secs 2023-09-10T09:45:44.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37576.234434 secs 2023-09-10T09:45:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:44 smithi082 ceph-mon[98694]: pgmap v20198: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 340 KiB/s wr, 12 op/s 2023-09-10T09:45:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:45:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37575.881961 secs 2023-09-10T09:45:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37576.234434 secs 2023-09-10T09:45:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:44 smithi098 ceph-mon[103124]: pgmap v20198: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 340 KiB/s wr, 12 op/s 2023-09-10T09:45:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:45:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37575.881961 secs 2023-09-10T09:45:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37576.234434 secs 2023-09-10T09:45:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:46 smithi170 ceph-mon[102832]: pgmap v20199: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 858 KiB/s wr, 19 op/s 2023-09-10T09:45:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:45:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:46 smithi082 ceph-mon[98694]: pgmap v20199: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 858 KiB/s wr, 19 op/s 2023-09-10T09:45:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:45:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:46 smithi098 ceph-mon[103124]: pgmap v20199: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 858 KiB/s wr, 19 op/s 2023-09-10T09:45:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:45:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:48 smithi170 ceph-mon[102832]: pgmap v20200: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 681 KiB/s wr, 13 op/s 2023-09-10T09:45:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:48 smithi082 ceph-mon[98694]: pgmap v20200: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 681 KiB/s wr, 13 op/s 2023-09-10T09:45:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:48 smithi098 ceph-mon[103124]: pgmap v20200: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 681 KiB/s wr, 13 op/s 2023-09-10T09:45:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37580.882016 secs 2023-09-10T09:45:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37581.234489 secs 2023-09-10T09:45:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:50 smithi170 ceph-mon[102832]: pgmap v20201: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 681 KiB/s wr, 13 op/s 2023-09-10T09:45:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:45:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37580.882016 secs 2023-09-10T09:45:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37581.234489 secs 2023-09-10T09:45:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:50 smithi082 ceph-mon[98694]: pgmap v20201: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 681 KiB/s wr, 13 op/s 2023-09-10T09:45:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:45:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37580.882016 secs 2023-09-10T09:45:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37581.234489 secs 2023-09-10T09:45:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:50 smithi098 ceph-mon[103124]: pgmap v20201: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 681 KiB/s wr, 13 op/s 2023-09-10T09:45:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:45:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:52 smithi170 ceph-mon[102832]: pgmap v20202: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:45:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:52 smithi082 ceph-mon[98694]: pgmap v20202: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:45:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:52 smithi098 ceph-mon[103124]: pgmap v20202: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:45:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:54 smithi170 ceph-mon[102832]: pgmap v20203: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 935 KiB/s wr, 14 op/s 2023-09-10T09:45:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:45:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:54.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37585.882071 secs 2023-09-10T09:45:54.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:45:54.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37586.234549 secs 2023-09-10T09:45:54.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:45:54.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:45:54.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:45:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:54 smithi082 ceph-mon[98694]: pgmap v20203: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 935 KiB/s wr, 14 op/s 2023-09-10T09:45:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:45:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37585.882071 secs 2023-09-10T09:45:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:45:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37586.234549 secs 2023-09-10T09:45:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:45:54.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:45:54.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:45:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:54 smithi098 ceph-mon[103124]: pgmap v20203: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 935 KiB/s wr, 14 op/s 2023-09-10T09:45:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:45:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37585.882071 secs 2023-09-10T09:45:55.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:45:55.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37586.234549 secs 2023-09-10T09:45:55.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:45:55.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:45:55.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:45:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:56 smithi170 ceph-mon[102832]: pgmap v20204: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T09:45:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:45:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:56 smithi082 ceph-mon[98694]: pgmap v20204: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T09:45:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:45:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:56 smithi098 ceph-mon[103124]: pgmap v20204: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T09:45:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:45:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:45:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:45:58 smithi170 ceph-mon[102832]: pgmap v20205: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 537 KiB/s wr, 14 op/s 2023-09-10T09:45:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:45:58 smithi082 ceph-mon[98694]: pgmap v20205: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 537 KiB/s wr, 14 op/s 2023-09-10T09:45:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:45:58 smithi098 ceph-mon[103124]: pgmap v20205: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 537 KiB/s wr, 14 op/s 2023-09-10T09:46:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37590.882129 secs 2023-09-10T09:46:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37591.234611 secs 2023-09-10T09:46:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:00 smithi170 ceph-mon[102832]: pgmap v20206: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 537 KiB/s wr, 14 op/s 2023-09-10T09:46:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37590.882129 secs 2023-09-10T09:46:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37591.234611 secs 2023-09-10T09:46:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:00 smithi082 ceph-mon[98694]: pgmap v20206: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 537 KiB/s wr, 14 op/s 2023-09-10T09:46:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37590.882129 secs 2023-09-10T09:46:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37591.234611 secs 2023-09-10T09:46:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:00 smithi098 ceph-mon[103124]: pgmap v20206: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 537 KiB/s wr, 14 op/s 2023-09-10T09:46:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:02 smithi170 ceph-mon[102832]: pgmap v20207: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 762 KiB/s wr, 20 op/s 2023-09-10T09:46:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:02 smithi082 ceph-mon[98694]: pgmap v20207: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 762 KiB/s wr, 20 op/s 2023-09-10T09:46:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:02 smithi098 ceph-mon[103124]: pgmap v20207: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 762 KiB/s wr, 20 op/s 2023-09-10T09:46:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37594.635561 secs 2023-09-10T09:46:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37594.635561 secs 2023-09-10T09:46:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37594.635561 secs 2023-09-10T09:46:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:04 smithi170 ceph-mon[102832]: pgmap v20208: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 345 KiB/s wr, 13 op/s 2023-09-10T09:46:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:46:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:04 smithi170 ceph-mon[102832]: osdmap e1331: 12 total, 12 up, 12 in 2023-09-10T09:46:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:46:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37595.882187 secs 2023-09-10T09:46:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37596.234676 secs 2023-09-10T09:46:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:04 smithi082 ceph-mon[98694]: pgmap v20208: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 345 KiB/s wr, 13 op/s 2023-09-10T09:46:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:46:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:04 smithi082 ceph-mon[98694]: osdmap e1331: 12 total, 12 up, 12 in 2023-09-10T09:46:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:46:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37595.882187 secs 2023-09-10T09:46:04.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37596.234676 secs 2023-09-10T09:46:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:04 smithi098 ceph-mon[103124]: pgmap v20208: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 345 KiB/s wr, 13 op/s 2023-09-10T09:46:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:46:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:04 smithi098 ceph-mon[103124]: osdmap e1331: 12 total, 12 up, 12 in 2023-09-10T09:46:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:46:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37595.882187 secs 2023-09-10T09:46:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37596.234676 secs 2023-09-10T09:46:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:06 smithi170 ceph-mon[102832]: pgmap v20210: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 705 KiB/s wr, 33 op/s 2023-09-10T09:46:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:46:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:06 smithi082 ceph-mon[98694]: pgmap v20210: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 705 KiB/s wr, 33 op/s 2023-09-10T09:46:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:46:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:06 smithi098 ceph-mon[103124]: pgmap v20210: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 705 KiB/s wr, 33 op/s 2023-09-10T09:46:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:46:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:08 smithi170 ceph-mon[102832]: pgmap v20211: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 705 KiB/s wr, 33 op/s 2023-09-10T09:46:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37599.635623 secs 2023-09-10T09:46:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:08 smithi082 ceph-mon[98694]: pgmap v20211: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 705 KiB/s wr, 33 op/s 2023-09-10T09:46:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37599.635623 secs 2023-09-10T09:46:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:08 smithi098 ceph-mon[103124]: pgmap v20211: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 705 KiB/s wr, 33 op/s 2023-09-10T09:46:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37599.635623 secs 2023-09-10T09:46:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37600.882244 secs 2023-09-10T09:46:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37601.234739 secs 2023-09-10T09:46:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:10 smithi170 ceph-mon[102832]: osdmap e1332: 12 total, 12 up, 12 in 2023-09-10T09:46:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:10 smithi170 ceph-mon[102832]: pgmap v20213: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 543 KiB/s wr, 31 op/s 2023-09-10T09:46:10.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:46:10.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37600.882244 secs 2023-09-10T09:46:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37601.234739 secs 2023-09-10T09:46:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:10 smithi082 ceph-mon[98694]: osdmap e1332: 12 total, 12 up, 12 in 2023-09-10T09:46:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:10 smithi082 ceph-mon[98694]: pgmap v20213: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 543 KiB/s wr, 31 op/s 2023-09-10T09:46:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:46:10.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37600.882244 secs 2023-09-10T09:46:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37601.234739 secs 2023-09-10T09:46:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:10 smithi098 ceph-mon[103124]: osdmap e1332: 12 total, 12 up, 12 in 2023-09-10T09:46:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:10 smithi098 ceph-mon[103124]: pgmap v20213: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 543 KiB/s wr, 31 op/s 2023-09-10T09:46:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:46:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:46:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:12 smithi170 ceph-mon[102832]: pgmap v20214: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 750 KiB/s wr, 41 op/s 2023-09-10T09:46:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:46:12.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:12 smithi082 ceph-mon[98694]: pgmap v20214: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 750 KiB/s wr, 41 op/s 2023-09-10T09:46:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:46:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:12 smithi098 ceph-mon[103124]: pgmap v20214: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 750 KiB/s wr, 41 op/s 2023-09-10T09:46:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37604.635682 secs 2023-09-10T09:46:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37604.635682 secs 2023-09-10T09:46:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37604.635682 secs 2023-09-10T09:46:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:14 smithi170 ceph-mon[102832]: pgmap v20215: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 604 KiB/s wr, 33 op/s 2023-09-10T09:46:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:14 smithi082 ceph-mon[98694]: pgmap v20215: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 604 KiB/s wr, 33 op/s 2023-09-10T09:46:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:14 smithi098 ceph-mon[103124]: pgmap v20215: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 604 KiB/s wr, 33 op/s 2023-09-10T09:46:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37605.882304 secs 2023-09-10T09:46:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37606.234802 secs 2023-09-10T09:46:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37605.882304 secs 2023-09-10T09:46:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37606.234802 secs 2023-09-10T09:46:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37605.882304 secs 2023-09-10T09:46:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37606.234802 secs 2023-09-10T09:46:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:16 smithi170 ceph-mon[102832]: pgmap v20216: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 198 KiB/s wr, 16 op/s 2023-09-10T09:46:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:16 smithi082 ceph-mon[98694]: pgmap v20216: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 198 KiB/s wr, 16 op/s 2023-09-10T09:46:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:16 smithi098 ceph-mon[103124]: pgmap v20216: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 198 KiB/s wr, 16 op/s 2023-09-10T09:46:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:18 smithi170 ceph-mon[102832]: pgmap v20217: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 198 KiB/s wr, 16 op/s 2023-09-10T09:46:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37609.635740 secs 2023-09-10T09:46:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:18 smithi082 ceph-mon[98694]: pgmap v20217: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 198 KiB/s wr, 16 op/s 2023-09-10T09:46:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37609.635740 secs 2023-09-10T09:46:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:18 smithi098 ceph-mon[103124]: pgmap v20217: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 198 KiB/s wr, 16 op/s 2023-09-10T09:46:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37609.635740 secs 2023-09-10T09:46:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37610.882364 secs 2023-09-10T09:46:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37611.234867 secs 2023-09-10T09:46:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:20 smithi170 ceph-mon[102832]: pgmap v20218: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 193 KiB/s wr, 15 op/s 2023-09-10T09:46:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37610.882364 secs 2023-09-10T09:46:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37611.234867 secs 2023-09-10T09:46:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:20 smithi082 ceph-mon[98694]: pgmap v20218: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 193 KiB/s wr, 15 op/s 2023-09-10T09:46:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37610.882364 secs 2023-09-10T09:46:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37611.234867 secs 2023-09-10T09:46:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:20 smithi098 ceph-mon[103124]: pgmap v20218: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 193 KiB/s wr, 15 op/s 2023-09-10T09:46:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:22 smithi170 ceph-mon[102832]: pgmap v20219: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 166 KiB/s wr, 19 op/s 2023-09-10T09:46:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:22 smithi082 ceph-mon[98694]: pgmap v20219: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 166 KiB/s wr, 19 op/s 2023-09-10T09:46:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:22 smithi098 ceph-mon[103124]: pgmap v20219: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 166 KiB/s wr, 19 op/s 2023-09-10T09:46:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37614.635775 secs 2023-09-10T09:46:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37614.635775 secs 2023-09-10T09:46:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37614.635775 secs 2023-09-10T09:46:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:24 smithi170 ceph-mon[102832]: pgmap v20220: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 12 op/s 2023-09-10T09:46:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37615.882439 secs 2023-09-10T09:46:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37616.234927 secs 2023-09-10T09:46:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:24 smithi082 ceph-mon[98694]: pgmap v20220: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 12 op/s 2023-09-10T09:46:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37615.882439 secs 2023-09-10T09:46:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37616.234927 secs 2023-09-10T09:46:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:24 smithi098 ceph-mon[103124]: pgmap v20220: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 12 op/s 2023-09-10T09:46:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37615.882439 secs 2023-09-10T09:46:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37616.234927 secs 2023-09-10T09:46:26.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:26 smithi170 ceph-mon[102832]: pgmap v20221: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 166 KiB/s wr, 18 op/s 2023-09-10T09:46:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:26 smithi082 ceph-mon[98694]: pgmap v20221: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 166 KiB/s wr, 18 op/s 2023-09-10T09:46:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:26 smithi098 ceph-mon[103124]: pgmap v20221: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 166 KiB/s wr, 18 op/s 2023-09-10T09:46:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:28 smithi170 ceph-mon[102832]: pgmap v20222: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T09:46:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37619.635835 secs 2023-09-10T09:46:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:28 smithi082 ceph-mon[98694]: pgmap v20222: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T09:46:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37619.635835 secs 2023-09-10T09:46:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:28 smithi098 ceph-mon[103124]: pgmap v20222: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T09:46:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37619.635835 secs 2023-09-10T09:46:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37620.882528 secs 2023-09-10T09:46:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37621.234989 secs 2023-09-10T09:46:30.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:30 smithi170 ceph-mon[102832]: pgmap v20223: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T09:46:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37620.882528 secs 2023-09-10T09:46:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37621.234989 secs 2023-09-10T09:46:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:30 smithi082 ceph-mon[98694]: pgmap v20223: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T09:46:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37620.882528 secs 2023-09-10T09:46:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37621.234989 secs 2023-09-10T09:46:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:30 smithi098 ceph-mon[103124]: pgmap v20223: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T09:46:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:32 smithi170 ceph-mon[102832]: pgmap v20224: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 138 KiB/s wr, 16 op/s 2023-09-10T09:46:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:32 smithi082 ceph-mon[98694]: pgmap v20224: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 138 KiB/s wr, 16 op/s 2023-09-10T09:46:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:32 smithi098 ceph-mon[103124]: pgmap v20224: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 138 KiB/s wr, 16 op/s 2023-09-10T09:46:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:34 smithi170 ceph-mon[102832]: pgmap v20225: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 10 op/s 2023-09-10T09:46:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:46:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37625.882613 secs 2023-09-10T09:46:34.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37626.235052 secs 2023-09-10T09:46:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:34 smithi082 ceph-mon[98694]: pgmap v20225: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 10 op/s 2023-09-10T09:46:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:46:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37625.882613 secs 2023-09-10T09:46:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37626.235052 secs 2023-09-10T09:46:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:34 smithi098 ceph-mon[103124]: pgmap v20225: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 10 op/s 2023-09-10T09:46:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:46:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37625.882613 secs 2023-09-10T09:46:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37626.235052 secs 2023-09-10T09:46:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:46:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:46:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:46:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:36 smithi170 ceph-mon[102832]: pgmap v20226: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 434 KiB/s wr, 16 op/s 2023-09-10T09:46:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:46:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:46:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:36 smithi082 ceph-mon[98694]: pgmap v20226: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 434 KiB/s wr, 16 op/s 2023-09-10T09:46:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:46:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:46:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:36 smithi098 ceph-mon[103124]: pgmap v20226: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 434 KiB/s wr, 16 op/s 2023-09-10T09:46:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:46:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:46:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:38 smithi170 ceph-mon[102832]: pgmap v20227: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 296 KiB/s wr, 10 op/s 2023-09-10T09:46:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:38 smithi082 ceph-mon[98694]: pgmap v20227: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 296 KiB/s wr, 10 op/s 2023-09-10T09:46:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:38 smithi098 ceph-mon[103124]: pgmap v20227: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 296 KiB/s wr, 10 op/s 2023-09-10T09:46:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37630.882703 secs 2023-09-10T09:46:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37631.235113 secs 2023-09-10T09:46:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:40 smithi170 ceph-mon[102832]: pgmap v20228: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 296 KiB/s wr, 10 op/s 2023-09-10T09:46:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37630.882703 secs 2023-09-10T09:46:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37631.235113 secs 2023-09-10T09:46:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:40 smithi082 ceph-mon[98694]: pgmap v20228: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 296 KiB/s wr, 10 op/s 2023-09-10T09:46:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37630.882703 secs 2023-09-10T09:46:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37631.235113 secs 2023-09-10T09:46:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:40 smithi098 ceph-mon[103124]: pgmap v20228: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 296 KiB/s wr, 10 op/s 2023-09-10T09:46:42.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:42 smithi170 ceph-mon[102832]: pgmap v20229: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 460 KiB/s wr, 17 op/s 2023-09-10T09:46:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:42 smithi082 ceph-mon[98694]: pgmap v20229: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 460 KiB/s wr, 17 op/s 2023-09-10T09:46:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:42 smithi098 ceph-mon[103124]: pgmap v20229: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 460 KiB/s wr, 17 op/s 2023-09-10T09:46:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:44 smithi170 ceph-mon[102832]: pgmap v20230: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 460 KiB/s wr, 12 op/s 2023-09-10T09:46:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:46:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:44.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37635.882762 secs 2023-09-10T09:46:44.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37636.235175 secs 2023-09-10T09:46:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:44 smithi082 ceph-mon[98694]: pgmap v20230: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 460 KiB/s wr, 12 op/s 2023-09-10T09:46:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:46:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37635.882762 secs 2023-09-10T09:46:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37636.235175 secs 2023-09-10T09:46:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:44 smithi098 ceph-mon[103124]: pgmap v20230: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 460 KiB/s wr, 12 op/s 2023-09-10T09:46:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:46:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37635.882762 secs 2023-09-10T09:46:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37636.235175 secs 2023-09-10T09:46:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:46 smithi170 ceph-mon[102832]: pgmap v20231: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 537 KiB/s wr, 19 op/s 2023-09-10T09:46:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:46:46.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:46 smithi082 ceph-mon[98694]: pgmap v20231: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 537 KiB/s wr, 19 op/s 2023-09-10T09:46:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:46:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:46 smithi098 ceph-mon[103124]: pgmap v20231: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 537 KiB/s wr, 19 op/s 2023-09-10T09:46:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:46:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:48.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:48 smithi170 ceph-mon[102832]: pgmap v20232: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 241 KiB/s wr, 13 op/s 2023-09-10T09:46:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:48 smithi082 ceph-mon[98694]: pgmap v20232: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 241 KiB/s wr, 13 op/s 2023-09-10T09:46:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:48 smithi098 ceph-mon[103124]: pgmap v20232: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 241 KiB/s wr, 13 op/s 2023-09-10T09:46:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37640.882823 secs 2023-09-10T09:46:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37641.235239 secs 2023-09-10T09:46:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:50 smithi170 ceph-mon[102832]: pgmap v20233: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 241 KiB/s wr, 13 op/s 2023-09-10T09:46:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:46:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37640.882823 secs 2023-09-10T09:46:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37641.235239 secs 2023-09-10T09:46:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:50 smithi082 ceph-mon[98694]: pgmap v20233: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 241 KiB/s wr, 13 op/s 2023-09-10T09:46:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:46:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37640.882823 secs 2023-09-10T09:46:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37641.235239 secs 2023-09-10T09:46:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:50 smithi098 ceph-mon[103124]: pgmap v20233: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 241 KiB/s wr, 13 op/s 2023-09-10T09:46:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:46:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:46:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:52 smithi170 ceph-mon[102832]: pgmap v20234: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 752 KiB/s wr, 20 op/s 2023-09-10T09:46:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:46:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:52 smithi082 ceph-mon[98694]: pgmap v20234: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 752 KiB/s wr, 20 op/s 2023-09-10T09:46:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:46:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:52 smithi098 ceph-mon[103124]: pgmap v20234: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 752 KiB/s wr, 20 op/s 2023-09-10T09:46:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:54 smithi170 ceph-mon[102832]: pgmap v20235: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 588 KiB/s wr, 14 op/s 2023-09-10T09:46:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37645.882908 secs 2023-09-10T09:46:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37646.235300 secs 2023-09-10T09:46:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:46:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:54 smithi082 ceph-mon[98694]: pgmap v20235: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 588 KiB/s wr, 14 op/s 2023-09-10T09:46:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37645.882908 secs 2023-09-10T09:46:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37646.235300 secs 2023-09-10T09:46:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:46:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:54 smithi098 ceph-mon[103124]: pgmap v20235: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 588 KiB/s wr, 14 op/s 2023-09-10T09:46:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37645.882908 secs 2023-09-10T09:46:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37646.235300 secs 2023-09-10T09:46:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:46:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:46:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:46:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:46:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:46:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:46:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:46:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:46:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:46:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:46:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:56 smithi170 ceph-mon[102832]: pgmap v20236: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 819 KiB/s wr, 21 op/s 2023-09-10T09:46:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:46:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:56 smithi082 ceph-mon[98694]: pgmap v20236: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 819 KiB/s wr, 21 op/s 2023-09-10T09:46:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:46:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:56 smithi098 ceph-mon[103124]: pgmap v20236: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 819 KiB/s wr, 21 op/s 2023-09-10T09:46:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:46:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:58 smithi170 ceph-mon[102832]: pgmap v20237: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 741 KiB/s wr, 14 op/s 2023-09-10T09:46:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:46:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:46:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37649.636264 secs 2023-09-10T09:46:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:58 smithi082 ceph-mon[98694]: pgmap v20237: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 741 KiB/s wr, 14 op/s 2023-09-10T09:46:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:46:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:46:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37649.636264 secs 2023-09-10T09:46:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:58 smithi098 ceph-mon[103124]: pgmap v20237: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 741 KiB/s wr, 14 op/s 2023-09-10T09:46:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:46:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:46:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:46:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37649.636264 secs 2023-09-10T09:47:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37650.882965 secs 2023-09-10T09:47:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37651.235361 secs 2023-09-10T09:47:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:00 smithi170 ceph-mon[102832]: pgmap v20238: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 741 KiB/s wr, 14 op/s 2023-09-10T09:47:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:47:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:47:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37650.882965 secs 2023-09-10T09:47:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37651.235361 secs 2023-09-10T09:47:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:00 smithi082 ceph-mon[98694]: pgmap v20238: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 741 KiB/s wr, 14 op/s 2023-09-10T09:47:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:47:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:47:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37650.882965 secs 2023-09-10T09:47:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37651.235361 secs 2023-09-10T09:47:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:00 smithi098 ceph-mon[103124]: pgmap v20238: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 741 KiB/s wr, 14 op/s 2023-09-10T09:47:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:47:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:47:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:02.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:02 smithi170 ceph-mon[102832]: pgmap v20239: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:47:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:02 smithi082 ceph-mon[98694]: pgmap v20239: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:47:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:02 smithi098 ceph-mon[103124]: pgmap v20239: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:47:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37654.636360 secs 2023-09-10T09:47:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37654.636360 secs 2023-09-10T09:47:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37654.636360 secs 2023-09-10T09:47:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:04 smithi170 ceph-mon[102832]: pgmap v20240: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 579 KiB/s wr, 14 op/s 2023-09-10T09:47:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:47:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:04 smithi170 ceph-mon[102832]: osdmap e1333: 12 total, 12 up, 12 in 2023-09-10T09:47:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37655.883045 secs 2023-09-10T09:47:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37656.235424 secs 2023-09-10T09:47:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:04 smithi082 ceph-mon[98694]: pgmap v20240: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 579 KiB/s wr, 14 op/s 2023-09-10T09:47:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:47:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:04 smithi082 ceph-mon[98694]: osdmap e1333: 12 total, 12 up, 12 in 2023-09-10T09:47:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37655.883045 secs 2023-09-10T09:47:04.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37656.235424 secs 2023-09-10T09:47:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:04 smithi098 ceph-mon[103124]: pgmap v20240: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 579 KiB/s wr, 14 op/s 2023-09-10T09:47:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:47:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:04 smithi098 ceph-mon[103124]: osdmap e1333: 12 total, 12 up, 12 in 2023-09-10T09:47:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37655.883045 secs 2023-09-10T09:47:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37656.235424 secs 2023-09-10T09:47:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:06 smithi170 ceph-mon[102832]: pgmap v20242: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 633 KiB/s wr, 34 op/s 2023-09-10T09:47:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:47:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:06 smithi082 ceph-mon[98694]: pgmap v20242: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 633 KiB/s wr, 34 op/s 2023-09-10T09:47:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:47:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:06 smithi098 ceph-mon[103124]: pgmap v20242: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 633 KiB/s wr, 34 op/s 2023-09-10T09:47:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:47:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:08 smithi170 ceph-mon[102832]: pgmap v20243: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 633 KiB/s wr, 34 op/s 2023-09-10T09:47:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37659.636434 secs 2023-09-10T09:47:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:08 smithi082 ceph-mon[98694]: pgmap v20243: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 633 KiB/s wr, 34 op/s 2023-09-10T09:47:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37659.636434 secs 2023-09-10T09:47:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:08 smithi098 ceph-mon[103124]: pgmap v20243: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 633 KiB/s wr, 34 op/s 2023-09-10T09:47:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37659.636434 secs 2023-09-10T09:47:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37660.883105 secs 2023-09-10T09:47:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37661.235485 secs 2023-09-10T09:47:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:10 smithi170 ceph-mon[102832]: osdmap e1334: 12 total, 12 up, 12 in 2023-09-10T09:47:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:10 smithi170 ceph-mon[102832]: pgmap v20245: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 269 KiB/s wr, 31 op/s 2023-09-10T09:47:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37660.883105 secs 2023-09-10T09:47:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37661.235485 secs 2023-09-10T09:47:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:10 smithi082 ceph-mon[98694]: osdmap e1334: 12 total, 12 up, 12 in 2023-09-10T09:47:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:10 smithi082 ceph-mon[98694]: pgmap v20245: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 269 KiB/s wr, 31 op/s 2023-09-10T09:47:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37660.883105 secs 2023-09-10T09:47:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37661.235485 secs 2023-09-10T09:47:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:10 smithi098 ceph-mon[103124]: osdmap e1334: 12 total, 12 up, 12 in 2023-09-10T09:47:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:10 smithi098 ceph-mon[103124]: pgmap v20245: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 269 KiB/s wr, 31 op/s 2023-09-10T09:47:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:12 smithi170 ceph-mon[102832]: pgmap v20246: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 475 KiB/s wr, 40 op/s 2023-09-10T09:47:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:12 smithi082 ceph-mon[98694]: pgmap v20246: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 475 KiB/s wr, 40 op/s 2023-09-10T09:47:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:12 smithi098 ceph-mon[103124]: pgmap v20246: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 475 KiB/s wr, 40 op/s 2023-09-10T09:47:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37664.636502 secs 2023-09-10T09:47:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37664.636502 secs 2023-09-10T09:47:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37664.636502 secs 2023-09-10T09:47:14.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:14 smithi170 ceph-mon[102832]: pgmap v20247: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 383 KiB/s wr, 32 op/s 2023-09-10T09:47:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37665.883183 secs 2023-09-10T09:47:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37666.235550 secs 2023-09-10T09:47:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:14 smithi082 ceph-mon[98694]: pgmap v20247: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 383 KiB/s wr, 32 op/s 2023-09-10T09:47:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37665.883183 secs 2023-09-10T09:47:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37666.235550 secs 2023-09-10T09:47:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:14 smithi098 ceph-mon[103124]: pgmap v20247: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 383 KiB/s wr, 32 op/s 2023-09-10T09:47:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37665.883183 secs 2023-09-10T09:47:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37666.235550 secs 2023-09-10T09:47:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:16 smithi170 ceph-mon[102832]: pgmap v20248: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 166 KiB/s wr, 13 op/s 2023-09-10T09:47:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:16 smithi082 ceph-mon[98694]: pgmap v20248: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 166 KiB/s wr, 13 op/s 2023-09-10T09:47:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:16 smithi098 ceph-mon[103124]: pgmap v20248: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 166 KiB/s wr, 13 op/s 2023-09-10T09:47:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:18 smithi170 ceph-mon[102832]: pgmap v20249: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 166 KiB/s wr, 13 op/s 2023-09-10T09:47:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37669.636630 secs 2023-09-10T09:47:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:18 smithi082 ceph-mon[98694]: pgmap v20249: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 166 KiB/s wr, 13 op/s 2023-09-10T09:47:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37669.636630 secs 2023-09-10T09:47:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:18 smithi098 ceph-mon[103124]: pgmap v20249: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 166 KiB/s wr, 13 op/s 2023-09-10T09:47:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37669.636630 secs 2023-09-10T09:47:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37670.883251 secs 2023-09-10T09:47:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37671.235611 secs 2023-09-10T09:47:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:20 smithi170 ceph-mon[102832]: pgmap v20250: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 12 op/s 2023-09-10T09:47:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37670.883251 secs 2023-09-10T09:47:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37671.235611 secs 2023-09-10T09:47:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:20 smithi082 ceph-mon[98694]: pgmap v20250: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 12 op/s 2023-09-10T09:47:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37670.883251 secs 2023-09-10T09:47:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37671.235611 secs 2023-09-10T09:47:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:20 smithi098 ceph-mon[103124]: pgmap v20250: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 12 op/s 2023-09-10T09:47:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:22 smithi170 ceph-mon[102832]: pgmap v20251: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 15 op/s 2023-09-10T09:47:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:22 smithi082 ceph-mon[98694]: pgmap v20251: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 15 op/s 2023-09-10T09:47:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:22 smithi098 ceph-mon[103124]: pgmap v20251: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 15 op/s 2023-09-10T09:47:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37674.636693 secs 2023-09-10T09:47:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37674.636693 secs 2023-09-10T09:47:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37674.636693 secs 2023-09-10T09:47:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:24 smithi170 ceph-mon[102832]: pgmap v20252: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2023-09-10T09:47:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37675.883310 secs 2023-09-10T09:47:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37676.235673 secs 2023-09-10T09:47:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:24 smithi082 ceph-mon[98694]: pgmap v20252: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2023-09-10T09:47:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37675.883310 secs 2023-09-10T09:47:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37676.235673 secs 2023-09-10T09:47:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:24 smithi098 ceph-mon[103124]: pgmap v20252: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2023-09-10T09:47:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37675.883310 secs 2023-09-10T09:47:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37676.235673 secs 2023-09-10T09:47:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:26 smithi170 ceph-mon[102832]: pgmap v20253: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 138 KiB/s wr, 15 op/s 2023-09-10T09:47:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:26 smithi082 ceph-mon[98694]: pgmap v20253: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 138 KiB/s wr, 15 op/s 2023-09-10T09:47:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:26 smithi098 ceph-mon[103124]: pgmap v20253: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 138 KiB/s wr, 15 op/s 2023-09-10T09:47:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:28 smithi170 ceph-mon[102832]: pgmap v20254: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 10 op/s 2023-09-10T09:47:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37679.636751 secs 2023-09-10T09:47:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:28 smithi082 ceph-mon[98694]: pgmap v20254: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 10 op/s 2023-09-10T09:47:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37679.636751 secs 2023-09-10T09:47:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:28 smithi098 ceph-mon[103124]: pgmap v20254: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 10 op/s 2023-09-10T09:47:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37679.636751 secs 2023-09-10T09:47:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37680.883367 secs 2023-09-10T09:47:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37681.235733 secs 2023-09-10T09:47:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:30 smithi170 ceph-mon[102832]: pgmap v20255: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 10 op/s 2023-09-10T09:47:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37680.883367 secs 2023-09-10T09:47:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37681.235733 secs 2023-09-10T09:47:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:30 smithi082 ceph-mon[98694]: pgmap v20255: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 10 op/s 2023-09-10T09:47:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37680.883367 secs 2023-09-10T09:47:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37681.235733 secs 2023-09-10T09:47:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:30 smithi098 ceph-mon[103124]: pgmap v20255: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 10 op/s 2023-09-10T09:47:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:32 smithi170 ceph-mon[102832]: pgmap v20256: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 138 KiB/s wr, 16 op/s 2023-09-10T09:47:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:32 smithi082 ceph-mon[98694]: pgmap v20256: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 138 KiB/s wr, 16 op/s 2023-09-10T09:47:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:32 smithi098 ceph-mon[103124]: pgmap v20256: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 138 KiB/s wr, 16 op/s 2023-09-10T09:47:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:34 smithi170 ceph-mon[102832]: pgmap v20257: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T09:47:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37685.883451 secs 2023-09-10T09:47:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37686.235794 secs 2023-09-10T09:47:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:34 smithi082 ceph-mon[98694]: pgmap v20257: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T09:47:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37685.883451 secs 2023-09-10T09:47:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37686.235794 secs 2023-09-10T09:47:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:34 smithi098 ceph-mon[103124]: pgmap v20257: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 11 op/s 2023-09-10T09:47:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37685.883451 secs 2023-09-10T09:47:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37686.235794 secs 2023-09-10T09:47:35.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:47:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:47:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:47:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:36 smithi170 ceph-mon[102832]: pgmap v20258: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 273 KiB/s wr, 16 op/s 2023-09-10T09:47:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:47:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:47:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:36 smithi082 ceph-mon[98694]: pgmap v20258: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 273 KiB/s wr, 16 op/s 2023-09-10T09:47:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:47:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:47:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:36 smithi098 ceph-mon[103124]: pgmap v20258: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 273 KiB/s wr, 16 op/s 2023-09-10T09:47:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:47:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:47:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:38.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:38 smithi170 ceph-mon[102832]: pgmap v20259: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 10 op/s 2023-09-10T09:47:38.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:47:38.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:38 smithi082 ceph-mon[98694]: pgmap v20259: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 10 op/s 2023-09-10T09:47:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:47:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:38 smithi098 ceph-mon[103124]: pgmap v20259: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 10 op/s 2023-09-10T09:47:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:47:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37690.883533 secs 2023-09-10T09:47:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37691.235854 secs 2023-09-10T09:47:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:40 smithi170 ceph-mon[102832]: pgmap v20260: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 10 op/s 2023-09-10T09:47:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37690.883533 secs 2023-09-10T09:47:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37691.235854 secs 2023-09-10T09:47:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:40 smithi082 ceph-mon[98694]: pgmap v20260: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 10 op/s 2023-09-10T09:47:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37690.883533 secs 2023-09-10T09:47:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37691.235854 secs 2023-09-10T09:47:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:40 smithi098 ceph-mon[103124]: pgmap v20260: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 135 KiB/s wr, 10 op/s 2023-09-10T09:47:42.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:42 smithi170 ceph-mon[102832]: pgmap v20261: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 299 KiB/s wr, 16 op/s 2023-09-10T09:47:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:42 smithi082 ceph-mon[98694]: pgmap v20261: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 299 KiB/s wr, 16 op/s 2023-09-10T09:47:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:42 smithi098 ceph-mon[103124]: pgmap v20261: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 299 KiB/s wr, 16 op/s 2023-09-10T09:47:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:44 smithi170 ceph-mon[102832]: pgmap v20262: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 299 KiB/s wr, 11 op/s 2023-09-10T09:47:44.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:47:44.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:44.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37695.883591 secs 2023-09-10T09:47:44.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37696.235913 secs 2023-09-10T09:47:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:44 smithi082 ceph-mon[98694]: pgmap v20262: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 299 KiB/s wr, 11 op/s 2023-09-10T09:47:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:47:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37695.883591 secs 2023-09-10T09:47:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37696.235913 secs 2023-09-10T09:47:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:44 smithi098 ceph-mon[103124]: pgmap v20262: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 299 KiB/s wr, 11 op/s 2023-09-10T09:47:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:47:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37695.883591 secs 2023-09-10T09:47:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37696.235913 secs 2023-09-10T09:47:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:46 smithi170 ceph-mon[102832]: pgmap v20263: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 612 KiB/s wr, 18 op/s 2023-09-10T09:47:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:47:46.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:46 smithi082 ceph-mon[98694]: pgmap v20263: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 612 KiB/s wr, 18 op/s 2023-09-10T09:47:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:47:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:46 smithi098 ceph-mon[103124]: pgmap v20263: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 612 KiB/s wr, 18 op/s 2023-09-10T09:47:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:47:47.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:48 smithi170 ceph-mon[102832]: pgmap v20264: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 477 KiB/s wr, 13 op/s 2023-09-10T09:47:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:48 smithi082 ceph-mon[98694]: pgmap v20264: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 477 KiB/s wr, 13 op/s 2023-09-10T09:47:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:48 smithi098 ceph-mon[103124]: pgmap v20264: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 477 KiB/s wr, 13 op/s 2023-09-10T09:47:50.677 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37700.883650 secs 2023-09-10T09:47:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37701.235973 secs 2023-09-10T09:47:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:50 smithi170 ceph-mon[102832]: pgmap v20265: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 477 KiB/s wr, 13 op/s 2023-09-10T09:47:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:47:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37700.883650 secs 2023-09-10T09:47:50.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37701.235973 secs 2023-09-10T09:47:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:50 smithi082 ceph-mon[98694]: pgmap v20265: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 477 KiB/s wr, 13 op/s 2023-09-10T09:47:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:47:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37700.883650 secs 2023-09-10T09:47:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37701.235973 secs 2023-09-10T09:47:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:50 smithi098 ceph-mon[103124]: pgmap v20265: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 477 KiB/s wr, 13 op/s 2023-09-10T09:47:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:47:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:47:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:52 smithi170 ceph-mon[102832]: pgmap v20266: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 816 KiB/s wr, 20 op/s 2023-09-10T09:47:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:47:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:52 smithi082 ceph-mon[98694]: pgmap v20266: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 816 KiB/s wr, 20 op/s 2023-09-10T09:47:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:47:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:52 smithi098 ceph-mon[103124]: pgmap v20266: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 816 KiB/s wr, 20 op/s 2023-09-10T09:47:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:54 smithi170 ceph-mon[102832]: pgmap v20267: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 652 KiB/s wr, 14 op/s 2023-09-10T09:47:54.969 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37705.883709 secs 2023-09-10T09:47:54.970 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37706.236034 secs 2023-09-10T09:47:54.971 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:54 smithi082 ceph-mon[98694]: pgmap v20267: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 652 KiB/s wr, 14 op/s 2023-09-10T09:47:54.971 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37705.883709 secs 2023-09-10T09:47:54.971 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37706.236034 secs 2023-09-10T09:47:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:54 smithi098 ceph-mon[103124]: pgmap v20267: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 652 KiB/s wr, 14 op/s 2023-09-10T09:47:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37705.883709 secs 2023-09-10T09:47:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37706.236034 secs 2023-09-10T09:47:55.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:47:55.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:47:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:47:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:56 smithi170 ceph-mon[102832]: pgmap v20268: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 839 KiB/s wr, 20 op/s 2023-09-10T09:47:56.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:56 smithi082 ceph-mon[98694]: pgmap v20268: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 839 KiB/s wr, 20 op/s 2023-09-10T09:47:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:56 smithi098 ceph-mon[103124]: pgmap v20268: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 839 KiB/s wr, 20 op/s 2023-09-10T09:47:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:58 smithi170 ceph-mon[102832]: pgmap v20269: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 526 KiB/s wr, 13 op/s 2023-09-10T09:47:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:47:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:47:58.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:47:58.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:47:58.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:47:58.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37709.637129 secs 2023-09-10T09:47:58.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:47:58.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:47:58.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:47:58.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:47:58.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:47:58.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:47:58.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:47:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:58 smithi082 ceph-mon[98694]: pgmap v20269: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 526 KiB/s wr, 13 op/s 2023-09-10T09:47:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:47:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:47:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:47:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:47:58.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:47:58.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37709.637129 secs 2023-09-10T09:47:58.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:47:58.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:47:58.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:47:58.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:47:58.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:47:58.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:47:58.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:47:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:58 smithi098 ceph-mon[103124]: pgmap v20269: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 526 KiB/s wr, 13 op/s 2023-09-10T09:47:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:47:59.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:47:59.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:47:59.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:47:59.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:47:59.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:47:59.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37709.637129 secs 2023-09-10T09:47:59.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:47:59.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:47:59.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:47:59.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:47:59.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:47:59.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:47:59.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:47:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:59 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T09:47:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:59 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T09:47:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37710.883770 secs 2023-09-10T09:47:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:47:59.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:47:59.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:47:59.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:47:59.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:59 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T09:47:59.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:59 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T09:47:59.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:47:59.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:47:59.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:47:59.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:47:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37711.236098 secs 2023-09-10T09:47:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:59 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T09:47:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:59 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T09:47:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37710.883770 secs 2023-09-10T09:47:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:47:59.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:47:59.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:47:59.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:47:59.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:59 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T09:47:59.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:59 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T09:47:59.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:47:59.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:47:59.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:47:59.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:47:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37711.236098 secs 2023-09-10T09:48:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:59 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T09:48:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:59 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T09:48:00.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37710.883770 secs 2023-09-10T09:48:00.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:48:00.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:48:00.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:48:00.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:48:00.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:59 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T09:48:00.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:59 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T09:48:00.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:48:00.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:48:00.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:48:00.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:47:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37711.236098 secs 2023-09-10T09:48:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:00 smithi170 ceph-mon[102832]: pgmap v20270: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 526 KiB/s wr, 13 op/s 2023-09-10T09:48:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:48:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:48:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:00 smithi082 ceph-mon[98694]: pgmap v20270: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 526 KiB/s wr, 13 op/s 2023-09-10T09:48:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:48:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:48:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:00 smithi098 ceph-mon[103124]: pgmap v20270: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 526 KiB/s wr, 13 op/s 2023-09-10T09:48:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:48:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:48:01.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:02 smithi170 ceph-mon[102832]: pgmap v20271: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 803 KiB/s wr, 19 op/s 2023-09-10T09:48:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:02 smithi082 ceph-mon[98694]: pgmap v20271: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 803 KiB/s wr, 19 op/s 2023-09-10T09:48:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:02 smithi098 ceph-mon[103124]: pgmap v20271: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 803 KiB/s wr, 19 op/s 2023-09-10T09:48:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37714.637192 secs 2023-09-10T09:48:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37714.637192 secs 2023-09-10T09:48:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37714.637192 secs 2023-09-10T09:48:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:04 smithi170 ceph-mon[102832]: pgmap v20272: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 465 KiB/s wr, 12 op/s 2023-09-10T09:48:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:48:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37715.883856 secs 2023-09-10T09:48:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37716.236162 secs 2023-09-10T09:48:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:04 smithi082 ceph-mon[98694]: pgmap v20272: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 465 KiB/s wr, 12 op/s 2023-09-10T09:48:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:48:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37715.883856 secs 2023-09-10T09:48:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37716.236162 secs 2023-09-10T09:48:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:04 smithi098 ceph-mon[103124]: pgmap v20272: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 465 KiB/s wr, 12 op/s 2023-09-10T09:48:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:48:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37715.883856 secs 2023-09-10T09:48:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37716.236162 secs 2023-09-10T09:48:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:06 smithi170 ceph-mon[102832]: pgmap v20273: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 806 KiB/s wr, 33 op/s 2023-09-10T09:48:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:48:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:06 smithi082 ceph-mon[98694]: pgmap v20273: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 806 KiB/s wr, 33 op/s 2023-09-10T09:48:06.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:48:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:06 smithi098 ceph-mon[103124]: pgmap v20273: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 806 KiB/s wr, 33 op/s 2023-09-10T09:48:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:48:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:08.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:08 smithi170 ceph-mon[102832]: pgmap v20274: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 619 KiB/s wr, 27 op/s 2023-09-10T09:48:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37719.637253 secs 2023-09-10T09:48:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:08 smithi082 ceph-mon[98694]: pgmap v20274: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 619 KiB/s wr, 27 op/s 2023-09-10T09:48:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37719.637253 secs 2023-09-10T09:48:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:08 smithi098 ceph-mon[103124]: pgmap v20274: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 619 KiB/s wr, 27 op/s 2023-09-10T09:48:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37719.637253 secs 2023-09-10T09:48:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37720.883921 secs 2023-09-10T09:48:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:09 smithi170 ceph-mon[102832]: osdmap e1335: 12 total, 12 up, 12 in 2023-09-10T09:48:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37721.236223 secs 2023-09-10T09:48:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37720.883921 secs 2023-09-10T09:48:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:09 smithi082 ceph-mon[98694]: osdmap e1335: 12 total, 12 up, 12 in 2023-09-10T09:48:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37721.236223 secs 2023-09-10T09:48:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37720.883921 secs 2023-09-10T09:48:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:09 smithi098 ceph-mon[103124]: osdmap e1335: 12 total, 12 up, 12 in 2023-09-10T09:48:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37721.236223 secs 2023-09-10T09:48:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:10 smithi170 ceph-mon[102832]: pgmap v20276: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 743 KiB/s wr, 33 op/s 2023-09-10T09:48:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:10 smithi082 ceph-mon[98694]: pgmap v20276: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 743 KiB/s wr, 33 op/s 2023-09-10T09:48:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:10 smithi098 ceph-mon[103124]: pgmap v20276: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 743 KiB/s wr, 33 op/s 2023-09-10T09:48:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:12 smithi170 ceph-mon[102832]: pgmap v20277: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 573 KiB/s wr, 32 op/s 2023-09-10T09:48:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:12 smithi082 ceph-mon[98694]: pgmap v20277: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 573 KiB/s wr, 32 op/s 2023-09-10T09:48:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:12 smithi098 ceph-mon[103124]: pgmap v20277: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 573 KiB/s wr, 32 op/s 2023-09-10T09:48:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37724.637314 secs 2023-09-10T09:48:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37724.637314 secs 2023-09-10T09:48:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37724.637314 secs 2023-09-10T09:48:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:14 smithi170 ceph-mon[102832]: pgmap v20278: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 573 KiB/s wr, 32 op/s 2023-09-10T09:48:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37725.883998 secs 2023-09-10T09:48:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37726.236284 secs 2023-09-10T09:48:14.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:14 smithi170 ceph-mon[102832]: osdmap e1336: 12 total, 12 up, 12 in 2023-09-10T09:48:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:14 smithi082 ceph-mon[98694]: pgmap v20278: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 573 KiB/s wr, 32 op/s 2023-09-10T09:48:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37725.883998 secs 2023-09-10T09:48:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37726.236284 secs 2023-09-10T09:48:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:14 smithi082 ceph-mon[98694]: osdmap e1336: 12 total, 12 up, 12 in 2023-09-10T09:48:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:14 smithi098 ceph-mon[103124]: pgmap v20278: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 573 KiB/s wr, 32 op/s 2023-09-10T09:48:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37725.883998 secs 2023-09-10T09:48:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37726.236284 secs 2023-09-10T09:48:15.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:14 smithi098 ceph-mon[103124]: osdmap e1336: 12 total, 12 up, 12 in 2023-09-10T09:48:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:16 smithi170 ceph-mon[102832]: pgmap v20280: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 248 KiB/s wr, 18 op/s 2023-09-10T09:48:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:48:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:16 smithi082 ceph-mon[98694]: pgmap v20280: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 248 KiB/s wr, 18 op/s 2023-09-10T09:48:16.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:48:16.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:16 smithi098 ceph-mon[103124]: pgmap v20280: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 248 KiB/s wr, 18 op/s 2023-09-10T09:48:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:48:17.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:18 smithi170 ceph-mon[102832]: pgmap v20281: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 223 KiB/s wr, 16 op/s 2023-09-10T09:48:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:48:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37729.637373 secs 2023-09-10T09:48:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:18 smithi082 ceph-mon[98694]: pgmap v20281: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 223 KiB/s wr, 16 op/s 2023-09-10T09:48:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:48:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37729.637373 secs 2023-09-10T09:48:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:18 smithi098 ceph-mon[103124]: pgmap v20281: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 223 KiB/s wr, 16 op/s 2023-09-10T09:48:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:48:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:19.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37729.637373 secs 2023-09-10T09:48:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37730.884056 secs 2023-09-10T09:48:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37731.236348 secs 2023-09-10T09:48:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37730.884056 secs 2023-09-10T09:48:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37731.236348 secs 2023-09-10T09:48:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37730.884056 secs 2023-09-10T09:48:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37731.236348 secs 2023-09-10T09:48:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:20 smithi170 ceph-mon[102832]: pgmap v20282: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 199 KiB/s wr, 15 op/s 2023-09-10T09:48:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:20 smithi082 ceph-mon[98694]: pgmap v20282: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 199 KiB/s wr, 15 op/s 2023-09-10T09:48:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:20 smithi098 ceph-mon[103124]: pgmap v20282: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 199 KiB/s wr, 15 op/s 2023-09-10T09:48:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:22 smithi170 ceph-mon[102832]: pgmap v20283: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 13 op/s 2023-09-10T09:48:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:22 smithi082 ceph-mon[98694]: pgmap v20283: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 13 op/s 2023-09-10T09:48:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:22 smithi098 ceph-mon[103124]: pgmap v20283: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 13 op/s 2023-09-10T09:48:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37734.637434 secs 2023-09-10T09:48:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37734.637434 secs 2023-09-10T09:48:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37734.637434 secs 2023-09-10T09:48:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:24 smithi170 ceph-mon[102832]: pgmap v20284: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 13 op/s 2023-09-10T09:48:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37735.884115 secs 2023-09-10T09:48:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37736.236411 secs 2023-09-10T09:48:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:24 smithi082 ceph-mon[98694]: pgmap v20284: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 13 op/s 2023-09-10T09:48:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37735.884115 secs 2023-09-10T09:48:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37736.236411 secs 2023-09-10T09:48:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:24 smithi098 ceph-mon[103124]: pgmap v20284: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 13 op/s 2023-09-10T09:48:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37735.884115 secs 2023-09-10T09:48:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37736.236411 secs 2023-09-10T09:48:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:26 smithi170 ceph-mon[102832]: pgmap v20285: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 179 KiB/s wr, 18 op/s 2023-09-10T09:48:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:26 smithi082 ceph-mon[98694]: pgmap v20285: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 179 KiB/s wr, 18 op/s 2023-09-10T09:48:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:26 smithi098 ceph-mon[103124]: pgmap v20285: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 179 KiB/s wr, 18 op/s 2023-09-10T09:48:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:28 smithi170 ceph-mon[102832]: pgmap v20286: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T09:48:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37739.637537 secs 2023-09-10T09:48:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:28 smithi082 ceph-mon[98694]: pgmap v20286: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T09:48:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37739.637537 secs 2023-09-10T09:48:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:28 smithi098 ceph-mon[103124]: pgmap v20286: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T09:48:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37739.637537 secs 2023-09-10T09:48:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37740.884177 secs 2023-09-10T09:48:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37741.236480 secs 2023-09-10T09:48:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37740.884177 secs 2023-09-10T09:48:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37741.236480 secs 2023-09-10T09:48:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37740.884177 secs 2023-09-10T09:48:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37741.236480 secs 2023-09-10T09:48:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:30 smithi170 ceph-mon[102832]: pgmap v20287: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T09:48:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:30 smithi082 ceph-mon[98694]: pgmap v20287: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T09:48:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:30 smithi098 ceph-mon[103124]: pgmap v20287: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T09:48:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:32 smithi082 ceph-mon[98694]: pgmap v20288: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 15 op/s 2023-09-10T09:48:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:32 smithi098 ceph-mon[103124]: pgmap v20288: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 15 op/s 2023-09-10T09:48:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:32 smithi170 ceph-mon[102832]: pgmap v20288: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 139 KiB/s wr, 15 op/s 2023-09-10T09:48:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:34 smithi082 ceph-mon[98694]: pgmap v20289: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T09:48:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37745.884237 secs 2023-09-10T09:48:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:48:34.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:34.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37746.236543 secs 2023-09-10T09:48:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:34 smithi098 ceph-mon[103124]: pgmap v20289: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T09:48:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37745.884237 secs 2023-09-10T09:48:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:48:35.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:35.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37746.236543 secs 2023-09-10T09:48:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:34 smithi170 ceph-mon[102832]: pgmap v20289: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T09:48:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37745.884237 secs 2023-09-10T09:48:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:48:35.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:35.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37746.236543 secs 2023-09-10T09:48:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:48:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:48:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:48:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:36 smithi082 ceph-mon[98694]: pgmap v20290: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 436 KiB/s wr, 16 op/s 2023-09-10T09:48:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:48:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:48:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:36 smithi098 ceph-mon[103124]: pgmap v20290: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 436 KiB/s wr, 16 op/s 2023-09-10T09:48:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:48:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:48:37.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:36 smithi170 ceph-mon[102832]: pgmap v20290: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 436 KiB/s wr, 16 op/s 2023-09-10T09:48:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:48:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:48:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:38 smithi082 ceph-mon[98694]: pgmap v20291: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 297 KiB/s wr, 10 op/s 2023-09-10T09:48:39.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:38 smithi098 ceph-mon[103124]: pgmap v20291: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 297 KiB/s wr, 10 op/s 2023-09-10T09:48:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:38 smithi170 ceph-mon[102832]: pgmap v20291: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 297 KiB/s wr, 10 op/s 2023-09-10T09:48:39.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37750.884323 secs 2023-09-10T09:48:39.962 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37751.236605 secs 2023-09-10T09:48:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37750.884323 secs 2023-09-10T09:48:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37751.236605 secs 2023-09-10T09:48:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37750.884323 secs 2023-09-10T09:48:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37751.236605 secs 2023-09-10T09:48:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:40 smithi082 ceph-mon[98694]: pgmap v20292: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 298 KiB/s wr, 10 op/s 2023-09-10T09:48:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:40 smithi098 ceph-mon[103124]: pgmap v20292: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 298 KiB/s wr, 10 op/s 2023-09-10T09:48:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:40 smithi170 ceph-mon[102832]: pgmap v20292: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 298 KiB/s wr, 10 op/s 2023-09-10T09:48:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:42 smithi082 ceph-mon[98694]: pgmap v20293: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 478 KiB/s wr, 16 op/s 2023-09-10T09:48:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:42 smithi098 ceph-mon[103124]: pgmap v20293: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 478 KiB/s wr, 16 op/s 2023-09-10T09:48:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:42 smithi170 ceph-mon[102832]: pgmap v20293: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 478 KiB/s wr, 16 op/s 2023-09-10T09:48:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:44 smithi082 ceph-mon[98694]: pgmap v20294: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 478 KiB/s wr, 12 op/s 2023-09-10T09:48:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37755.884383 secs 2023-09-10T09:48:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37756.236666 secs 2023-09-10T09:48:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:44 smithi098 ceph-mon[103124]: pgmap v20294: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 478 KiB/s wr, 12 op/s 2023-09-10T09:48:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37755.884383 secs 2023-09-10T09:48:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37756.236666 secs 2023-09-10T09:48:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:44 smithi170 ceph-mon[102832]: pgmap v20294: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 478 KiB/s wr, 12 op/s 2023-09-10T09:48:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37755.884383 secs 2023-09-10T09:48:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37756.236666 secs 2023-09-10T09:48:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:46 smithi082 ceph-mon[98694]: pgmap v20295: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 539 KiB/s wr, 18 op/s 2023-09-10T09:48:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:46 smithi098 ceph-mon[103124]: pgmap v20295: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 539 KiB/s wr, 18 op/s 2023-09-10T09:48:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:46 smithi170 ceph-mon[102832]: pgmap v20295: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 539 KiB/s wr, 18 op/s 2023-09-10T09:48:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:48 smithi098 ceph-mon[103124]: pgmap v20296: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 242 KiB/s wr, 12 op/s 2023-09-10T09:48:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:48 smithi170 ceph-mon[102832]: pgmap v20296: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 242 KiB/s wr, 12 op/s 2023-09-10T09:48:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:48 smithi082 ceph-mon[98694]: pgmap v20296: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 242 KiB/s wr, 12 op/s 2023-09-10T09:48:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37760.884469 secs 2023-09-10T09:48:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37761.236728 secs 2023-09-10T09:48:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37760.884469 secs 2023-09-10T09:48:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37761.236728 secs 2023-09-10T09:48:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37760.884469 secs 2023-09-10T09:48:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37761.236728 secs 2023-09-10T09:48:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:50 smithi098 ceph-mon[103124]: pgmap v20297: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 242 KiB/s wr, 12 op/s 2023-09-10T09:48:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:48:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:50 smithi170 ceph-mon[102832]: pgmap v20297: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 242 KiB/s wr, 12 op/s 2023-09-10T09:48:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:48:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:50 smithi082 ceph-mon[98694]: pgmap v20297: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 242 KiB/s wr, 12 op/s 2023-09-10T09:48:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:48:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:52 smithi098 ceph-mon[103124]: pgmap v20298: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 755 KiB/s wr, 20 op/s 2023-09-10T09:48:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:48:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:52 smithi170 ceph-mon[102832]: pgmap v20298: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 755 KiB/s wr, 20 op/s 2023-09-10T09:48:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:48:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:52 smithi082 ceph-mon[98694]: pgmap v20298: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 755 KiB/s wr, 20 op/s 2023-09-10T09:48:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:48:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:54 smithi098 ceph-mon[103124]: pgmap v20299: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 574 KiB/s wr, 14 op/s 2023-09-10T09:48:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37765.884508 secs 2023-09-10T09:48:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:48:55.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:55.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37766.236791 secs 2023-09-10T09:48:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:54 smithi170 ceph-mon[102832]: pgmap v20299: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 574 KiB/s wr, 14 op/s 2023-09-10T09:48:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37765.884508 secs 2023-09-10T09:48:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:48:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:55.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37766.236791 secs 2023-09-10T09:48:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:54 smithi082 ceph-mon[98694]: pgmap v20299: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 574 KiB/s wr, 14 op/s 2023-09-10T09:48:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37765.884508 secs 2023-09-10T09:48:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:48:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37766.236791 secs 2023-09-10T09:48:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:56 smithi098 ceph-mon[103124]: pgmap v20300: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 741 KiB/s wr, 20 op/s 2023-09-10T09:48:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:56 smithi170 ceph-mon[102832]: pgmap v20300: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 741 KiB/s wr, 20 op/s 2023-09-10T09:48:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:56 smithi082 ceph-mon[98694]: pgmap v20300: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 741 KiB/s wr, 20 op/s 2023-09-10T09:48:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:58 smithi098 ceph-mon[103124]: pgmap v20301: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 680 KiB/s wr, 14 op/s 2023-09-10T09:48:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:48:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:58 smithi170 ceph-mon[102832]: pgmap v20301: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 680 KiB/s wr, 14 op/s 2023-09-10T09:48:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:48:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:48:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:58 smithi082 ceph-mon[98694]: pgmap v20301: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 680 KiB/s wr, 14 op/s 2023-09-10T09:48:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:48:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37770.884597 secs 2023-09-10T09:49:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:49:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37771.236855 secs 2023-09-10T09:49:00.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:49:00.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:49:00.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:48:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:49:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37770.884597 secs 2023-09-10T09:49:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:49:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37771.236855 secs 2023-09-10T09:49:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:49:00.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:49:00.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:48:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:49:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37770.884597 secs 2023-09-10T09:49:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:49:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37771.236855 secs 2023-09-10T09:49:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:49:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:49:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:48:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:49:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:00 smithi170 ceph-mon[102832]: pgmap v20302: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 680 KiB/s wr, 14 op/s 2023-09-10T09:49:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:49:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:00 smithi082 ceph-mon[98694]: pgmap v20302: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 680 KiB/s wr, 14 op/s 2023-09-10T09:49:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:49:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:00 smithi098 ceph-mon[103124]: pgmap v20302: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 680 KiB/s wr, 14 op/s 2023-09-10T09:49:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:49:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:02 smithi170 ceph-mon[102832]: pgmap v20303: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:49:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:02 smithi082 ceph-mon[98694]: pgmap v20303: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:49:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:02 smithi098 ceph-mon[103124]: pgmap v20303: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:49:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37774.638030 secs 2023-09-10T09:49:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37774.638030 secs 2023-09-10T09:49:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37774.638030 secs 2023-09-10T09:49:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:04 smithi170 ceph-mon[102832]: pgmap v20304: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 634 KiB/s wr, 14 op/s 2023-09-10T09:49:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37775.884655 secs 2023-09-10T09:49:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:49:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:49:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37776.236917 secs 2023-09-10T09:49:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:04 smithi082 ceph-mon[98694]: pgmap v20304: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 634 KiB/s wr, 14 op/s 2023-09-10T09:49:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37775.884655 secs 2023-09-10T09:49:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:49:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:49:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37776.236917 secs 2023-09-10T09:49:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:04 smithi098 ceph-mon[103124]: pgmap v20304: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 634 KiB/s wr, 14 op/s 2023-09-10T09:49:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37775.884655 secs 2023-09-10T09:49:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:49:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:49:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37776.236917 secs 2023-09-10T09:49:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:06 smithi170 ceph-mon[102832]: pgmap v20305: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 812 KiB/s wr, 35 op/s 2023-09-10T09:49:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:49:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:06 smithi082 ceph-mon[98694]: pgmap v20305: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 812 KiB/s wr, 35 op/s 2023-09-10T09:49:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:49:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:06 smithi098 ceph-mon[103124]: pgmap v20305: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 812 KiB/s wr, 35 op/s 2023-09-10T09:49:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:49:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:08 smithi170 ceph-mon[102832]: pgmap v20306: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 645 KiB/s wr, 28 op/s 2023-09-10T09:49:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37779.638089 secs 2023-09-10T09:49:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:08 smithi082 ceph-mon[98694]: pgmap v20306: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 645 KiB/s wr, 28 op/s 2023-09-10T09:49:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37779.638089 secs 2023-09-10T09:49:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:08 smithi098 ceph-mon[103124]: pgmap v20306: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 645 KiB/s wr, 28 op/s 2023-09-10T09:49:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37779.638089 secs 2023-09-10T09:49:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37780.884714 secs 2023-09-10T09:49:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:09 smithi170 ceph-mon[102832]: osdmap e1337: 12 total, 12 up, 12 in 2023-09-10T09:49:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37781.236976 secs 2023-09-10T09:49:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37780.884714 secs 2023-09-10T09:49:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:09 smithi082 ceph-mon[98694]: osdmap e1337: 12 total, 12 up, 12 in 2023-09-10T09:49:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37781.236976 secs 2023-09-10T09:49:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37780.884714 secs 2023-09-10T09:49:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:09 smithi098 ceph-mon[103124]: osdmap e1337: 12 total, 12 up, 12 in 2023-09-10T09:49:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37781.236976 secs 2023-09-10T09:49:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:10 smithi170 ceph-mon[102832]: pgmap v20308: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 774 KiB/s wr, 34 op/s 2023-09-10T09:49:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:10 smithi082 ceph-mon[98694]: pgmap v20308: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 774 KiB/s wr, 34 op/s 2023-09-10T09:49:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:10 smithi098 ceph-mon[103124]: pgmap v20308: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 774 KiB/s wr, 34 op/s 2023-09-10T09:49:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:12 smithi170 ceph-mon[102832]: pgmap v20309: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 579 KiB/s wr, 34 op/s 2023-09-10T09:49:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:12 smithi082 ceph-mon[98694]: pgmap v20309: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 579 KiB/s wr, 34 op/s 2023-09-10T09:49:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:12 smithi098 ceph-mon[103124]: pgmap v20309: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 579 KiB/s wr, 34 op/s 2023-09-10T09:49:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37784.638171 secs 2023-09-10T09:49:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37784.638171 secs 2023-09-10T09:49:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37784.638171 secs 2023-09-10T09:49:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:14 smithi170 ceph-mon[102832]: pgmap v20310: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 579 KiB/s wr, 34 op/s 2023-09-10T09:49:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37785.884798 secs 2023-09-10T09:49:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:14 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:49:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:14 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37786.237044 secs 2023-09-10T09:49:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:14 smithi170 ceph-mon[102832]: osdmap e1338: 12 total, 12 up, 12 in 2023-09-10T09:49:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:14 smithi082 ceph-mon[98694]: pgmap v20310: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 579 KiB/s wr, 34 op/s 2023-09-10T09:49:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37785.884798 secs 2023-09-10T09:49:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:14 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:49:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:14 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37786.237044 secs 2023-09-10T09:49:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:14 smithi082 ceph-mon[98694]: osdmap e1338: 12 total, 12 up, 12 in 2023-09-10T09:49:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:14 smithi098 ceph-mon[103124]: pgmap v20310: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 579 KiB/s wr, 34 op/s 2023-09-10T09:49:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37785.884798 secs 2023-09-10T09:49:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:14 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:49:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:14 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37786.237044 secs 2023-09-10T09:49:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:14 smithi098 ceph-mon[103124]: osdmap e1338: 12 total, 12 up, 12 in 2023-09-10T09:49:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:16 smithi170 ceph-mon[102832]: pgmap v20312: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 500 KiB/s wr, 19 op/s 2023-09-10T09:49:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:49:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:16 smithi082 ceph-mon[98694]: pgmap v20312: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 500 KiB/s wr, 19 op/s 2023-09-10T09:49:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:49:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:16 smithi098 ceph-mon[103124]: pgmap v20312: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 500 KiB/s wr, 19 op/s 2023-09-10T09:49:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:49:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:18 smithi170 ceph-mon[102832]: pgmap v20313: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 456 KiB/s wr, 17 op/s 2023-09-10T09:49:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37789.638261 secs 2023-09-10T09:49:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:18 smithi082 ceph-mon[98694]: pgmap v20313: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 456 KiB/s wr, 17 op/s 2023-09-10T09:49:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37789.638261 secs 2023-09-10T09:49:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:18 smithi098 ceph-mon[103124]: pgmap v20313: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 456 KiB/s wr, 17 op/s 2023-09-10T09:49:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37789.638261 secs 2023-09-10T09:49:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37790.884857 secs 2023-09-10T09:49:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37791.237111 secs 2023-09-10T09:49:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37790.884857 secs 2023-09-10T09:49:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37791.237111 secs 2023-09-10T09:49:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37790.884857 secs 2023-09-10T09:49:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37791.237111 secs 2023-09-10T09:49:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:20 smithi170 ceph-mon[102832]: pgmap v20314: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 400 KiB/s wr, 15 op/s 2023-09-10T09:49:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:20 smithi082 ceph-mon[98694]: pgmap v20314: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 400 KiB/s wr, 15 op/s 2023-09-10T09:49:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:20 smithi098 ceph-mon[103124]: pgmap v20314: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 400 KiB/s wr, 15 op/s 2023-09-10T09:49:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:22 smithi170 ceph-mon[102832]: pgmap v20315: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 12 op/s 2023-09-10T09:49:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:22 smithi082 ceph-mon[98694]: pgmap v20315: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 12 op/s 2023-09-10T09:49:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:22 smithi098 ceph-mon[103124]: pgmap v20315: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 12 op/s 2023-09-10T09:49:24.177 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37794.638362 secs 2023-09-10T09:49:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37794.638362 secs 2023-09-10T09:49:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37794.638362 secs 2023-09-10T09:49:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:24 smithi170 ceph-mon[102832]: pgmap v20316: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 12 op/s 2023-09-10T09:49:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37795.884944 secs 2023-09-10T09:49:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37796.237179 secs 2023-09-10T09:49:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:24 smithi082 ceph-mon[98694]: pgmap v20316: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 12 op/s 2023-09-10T09:49:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37795.884944 secs 2023-09-10T09:49:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37796.237179 secs 2023-09-10T09:49:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:24 smithi098 ceph-mon[103124]: pgmap v20316: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 12 op/s 2023-09-10T09:49:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37795.884944 secs 2023-09-10T09:49:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37796.237179 secs 2023-09-10T09:49:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:26 smithi170 ceph-mon[102832]: pgmap v20317: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 17 op/s 2023-09-10T09:49:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:26 smithi082 ceph-mon[98694]: pgmap v20317: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 17 op/s 2023-09-10T09:49:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:26 smithi098 ceph-mon[103124]: pgmap v20317: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 17 op/s 2023-09-10T09:49:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:28 smithi170 ceph-mon[102832]: pgmap v20318: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T09:49:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37799.638419 secs 2023-09-10T09:49:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:28 smithi082 ceph-mon[98694]: pgmap v20318: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T09:49:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37799.638419 secs 2023-09-10T09:49:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:28 smithi098 ceph-mon[103124]: pgmap v20318: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T09:49:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37799.638419 secs 2023-09-10T09:49:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37800.885022 secs 2023-09-10T09:49:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37801.237245 secs 2023-09-10T09:49:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:29 smithi170 ceph-mon[102832]: pgmap v20319: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T09:49:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37800.885022 secs 2023-09-10T09:49:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37801.237245 secs 2023-09-10T09:49:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:29 smithi082 ceph-mon[98694]: pgmap v20319: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T09:49:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37800.885022 secs 2023-09-10T09:49:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37801.237245 secs 2023-09-10T09:49:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:29 smithi098 ceph-mon[103124]: pgmap v20319: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T09:49:32.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:32 smithi170 ceph-mon[102832]: pgmap v20320: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 139 KiB/s wr, 15 op/s 2023-09-10T09:49:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:32 smithi082 ceph-mon[98694]: pgmap v20320: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 139 KiB/s wr, 15 op/s 2023-09-10T09:49:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:32 smithi098 ceph-mon[103124]: pgmap v20320: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 139 KiB/s wr, 15 op/s 2023-09-10T09:49:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37804.638492 secs 2023-09-10T09:49:33.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37804.638492 secs 2023-09-10T09:49:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37804.638492 secs 2023-09-10T09:49:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:34 smithi170 ceph-mon[102832]: pgmap v20321: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T09:49:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37805.885083 secs 2023-09-10T09:49:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37806.237310 secs 2023-09-10T09:49:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:34 smithi082 ceph-mon[98694]: pgmap v20321: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T09:49:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37805.885083 secs 2023-09-10T09:49:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37806.237310 secs 2023-09-10T09:49:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:34 smithi098 ceph-mon[103124]: pgmap v20321: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T09:49:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37805.885083 secs 2023-09-10T09:49:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37806.237310 secs 2023-09-10T09:49:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:36 smithi170 ceph-mon[102832]: pgmap v20322: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 409 KiB/s wr, 16 op/s 2023-09-10T09:49:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:49:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:36 smithi082 ceph-mon[98694]: pgmap v20322: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 409 KiB/s wr, 16 op/s 2023-09-10T09:49:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:49:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:36 smithi098 ceph-mon[103124]: pgmap v20322: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 409 KiB/s wr, 16 op/s 2023-09-10T09:49:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:49:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:38.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:38 smithi170 ceph-mon[102832]: pgmap v20323: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 270 KiB/s wr, 11 op/s 2023-09-10T09:49:38.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:49:38.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:38.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37809.638579 secs 2023-09-10T09:49:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:38 smithi082 ceph-mon[98694]: pgmap v20323: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 270 KiB/s wr, 11 op/s 2023-09-10T09:49:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:49:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37809.638579 secs 2023-09-10T09:49:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:38 smithi098 ceph-mon[103124]: pgmap v20323: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 270 KiB/s wr, 11 op/s 2023-09-10T09:49:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:49:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37809.638579 secs 2023-09-10T09:49:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37810.885167 secs 2023-09-10T09:49:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37811.237374 secs 2023-09-10T09:49:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37810.885167 secs 2023-09-10T09:49:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37811.237374 secs 2023-09-10T09:49:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37810.885167 secs 2023-09-10T09:49:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37811.237374 secs 2023-09-10T09:49:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:40 smithi170 ceph-mon[102832]: pgmap v20324: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 270 KiB/s wr, 11 op/s 2023-09-10T09:49:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:49:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:40 smithi082 ceph-mon[98694]: pgmap v20324: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 270 KiB/s wr, 11 op/s 2023-09-10T09:49:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:49:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:40 smithi098 ceph-mon[103124]: pgmap v20324: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 270 KiB/s wr, 11 op/s 2023-09-10T09:49:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:49:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:42.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:42 smithi170 ceph-mon[102832]: pgmap v20325: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 604 KiB/s wr, 19 op/s 2023-09-10T09:49:42.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:42 smithi082 ceph-mon[98694]: pgmap v20325: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 604 KiB/s wr, 19 op/s 2023-09-10T09:49:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:42 smithi098 ceph-mon[103124]: pgmap v20325: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 604 KiB/s wr, 19 op/s 2023-09-10T09:49:43.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:49:43.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:49:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:49:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:44 smithi170 ceph-mon[102832]: pgmap v20326: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 604 KiB/s wr, 14 op/s 2023-09-10T09:49:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37815.885229 secs 2023-09-10T09:49:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:44 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:49:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:49:44.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:44.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37816.237438 secs 2023-09-10T09:49:44.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:44 smithi082 ceph-mon[98694]: pgmap v20326: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 604 KiB/s wr, 14 op/s 2023-09-10T09:49:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37815.885229 secs 2023-09-10T09:49:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:44 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:49:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:49:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:44.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37816.237438 secs 2023-09-10T09:49:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:44 smithi098 ceph-mon[103124]: pgmap v20326: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 604 KiB/s wr, 14 op/s 2023-09-10T09:49:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37815.885229 secs 2023-09-10T09:49:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:44 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:49:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:49:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37816.237438 secs 2023-09-10T09:49:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:46 smithi170 ceph-mon[102832]: pgmap v20327: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 631 KiB/s wr, 20 op/s 2023-09-10T09:49:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:49:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:46 smithi082 ceph-mon[98694]: pgmap v20327: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 631 KiB/s wr, 20 op/s 2023-09-10T09:49:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:49:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:46 smithi098 ceph-mon[103124]: pgmap v20327: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 631 KiB/s wr, 20 op/s 2023-09-10T09:49:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:49:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:48.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:48 smithi170 ceph-mon[102832]: pgmap v20328: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 361 KiB/s wr, 14 op/s 2023-09-10T09:49:48.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:48 smithi082 ceph-mon[98694]: pgmap v20328: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 361 KiB/s wr, 14 op/s 2023-09-10T09:49:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:48 smithi098 ceph-mon[103124]: pgmap v20328: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 361 KiB/s wr, 14 op/s 2023-09-10T09:49:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37820.885319 secs 2023-09-10T09:49:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37821.237491 secs 2023-09-10T09:49:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37820.885319 secs 2023-09-10T09:49:49.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37821.237491 secs 2023-09-10T09:49:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37820.885319 secs 2023-09-10T09:49:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37821.237491 secs 2023-09-10T09:49:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:50 smithi170 ceph-mon[102832]: pgmap v20329: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 361 KiB/s wr, 14 op/s 2023-09-10T09:49:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:49:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:50 smithi082 ceph-mon[98694]: pgmap v20329: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 361 KiB/s wr, 14 op/s 2023-09-10T09:49:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:49:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:50 smithi098 ceph-mon[103124]: pgmap v20329: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 361 KiB/s wr, 14 op/s 2023-09-10T09:49:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:49:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:52.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:52 smithi082 ceph-mon[98694]: pgmap v20330: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:49:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:49:52.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:52 smithi098 ceph-mon[103124]: pgmap v20330: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:49:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:49:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:52 smithi170 ceph-mon[102832]: pgmap v20330: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T09:49:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:49:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:49:54.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:54 smithi082 ceph-mon[98694]: pgmap v20331: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 781 KiB/s wr, 14 op/s 2023-09-10T09:49:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37825.885378 secs 2023-09-10T09:49:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37826.237556 secs 2023-09-10T09:49:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:54 smithi098 ceph-mon[103124]: pgmap v20331: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 781 KiB/s wr, 14 op/s 2023-09-10T09:49:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37825.885378 secs 2023-09-10T09:49:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37826.237556 secs 2023-09-10T09:49:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:54 smithi170 ceph-mon[102832]: pgmap v20331: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 781 KiB/s wr, 14 op/s 2023-09-10T09:49:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37825.885378 secs 2023-09-10T09:49:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37826.237556 secs 2023-09-10T09:49:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:56 smithi082 ceph-mon[98694]: pgmap v20332: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 946 KiB/s wr, 20 op/s 2023-09-10T09:49:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:56 smithi098 ceph-mon[103124]: pgmap v20332: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 946 KiB/s wr, 20 op/s 2023-09-10T09:49:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:56 smithi170 ceph-mon[102832]: pgmap v20332: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 946 KiB/s wr, 20 op/s 2023-09-10T09:49:58.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:58 smithi082 ceph-mon[98694]: pgmap v20333: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 919 KiB/s wr, 14 op/s 2023-09-10T09:49:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:58 smithi098 ceph-mon[103124]: pgmap v20333: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 919 KiB/s wr, 14 op/s 2023-09-10T09:49:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:58 smithi170 ceph-mon[102832]: pgmap v20333: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 919 KiB/s wr, 14 op/s 2023-09-10T09:49:59.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37830.885436 secs 2023-09-10T09:49:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37831.237619 secs 2023-09-10T09:49:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:49:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:50:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37830.885436 secs 2023-09-10T09:50:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37831.237619 secs 2023-09-10T09:50:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:49:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:50:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37830.885436 secs 2023-09-10T09:50:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37831.237619 secs 2023-09-10T09:50:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:49:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:50:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:00 smithi082 ceph-mon[98694]: pgmap v20334: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 919 KiB/s wr, 14 op/s 2023-09-10T09:50:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 2 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T09:50:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:50:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:50:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:50:00.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:50:00.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:50:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:00 smithi098 ceph-mon[103124]: pgmap v20334: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 919 KiB/s wr, 14 op/s 2023-09-10T09:50:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 2 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T09:50:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:50:01.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:50:01.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:50:01.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:50:01.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:50:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:00 smithi170 ceph-mon[102832]: pgmap v20334: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 919 KiB/s wr, 14 op/s 2023-09-10T09:50:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 2 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T09:50:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:50:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:50:01.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:50:01.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:50:01.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:50:02.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:02 smithi082 ceph-mon[98694]: pgmap v20335: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:50:03.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:02 smithi098 ceph-mon[103124]: pgmap v20335: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:50:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:02 smithi170 ceph-mon[102832]: pgmap v20335: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:50:03.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37834.638933 secs 2023-09-10T09:50:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37834.638933 secs 2023-09-10T09:50:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37834.638933 secs 2023-09-10T09:50:04.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:04 smithi082 ceph-mon[98694]: pgmap v20336: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 328 KiB/s wr, 13 op/s 2023-09-10T09:50:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37835.885490 secs 2023-09-10T09:50:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:50:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:50:04.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:04.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37836.237683 secs 2023-09-10T09:50:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:04 smithi098 ceph-mon[103124]: pgmap v20336: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 328 KiB/s wr, 13 op/s 2023-09-10T09:50:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37835.885490 secs 2023-09-10T09:50:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:50:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:50:05.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:05.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37836.237683 secs 2023-09-10T09:50:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:04 smithi170 ceph-mon[102832]: pgmap v20336: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 328 KiB/s wr, 13 op/s 2023-09-10T09:50:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37835.885490 secs 2023-09-10T09:50:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:50:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:50:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37836.237683 secs 2023-09-10T09:50:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:06 smithi098 ceph-mon[103124]: pgmap v20337: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 805 KiB/s wr, 35 op/s 2023-09-10T09:50:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:50:07.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:06 smithi170 ceph-mon[102832]: pgmap v20337: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 805 KiB/s wr, 35 op/s 2023-09-10T09:50:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:50:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:06 smithi082 ceph-mon[98694]: pgmap v20337: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 805 KiB/s wr, 35 op/s 2023-09-10T09:50:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:50:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:08 smithi098 ceph-mon[103124]: pgmap v20338: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 639 KiB/s wr, 28 op/s 2023-09-10T09:50:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37839.638995 secs 2023-09-10T09:50:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:08 smithi170 ceph-mon[102832]: pgmap v20338: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 639 KiB/s wr, 28 op/s 2023-09-10T09:50:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37839.638995 secs 2023-09-10T09:50:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:08 smithi082 ceph-mon[98694]: pgmap v20338: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 639 KiB/s wr, 28 op/s 2023-09-10T09:50:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37839.638995 secs 2023-09-10T09:50:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37840.885548 secs 2023-09-10T09:50:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:09 smithi098 ceph-mon[103124]: osdmap e1339: 12 total, 12 up, 12 in 2023-09-10T09:50:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37841.237749 secs 2023-09-10T09:50:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37840.885548 secs 2023-09-10T09:50:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:09 smithi170 ceph-mon[102832]: osdmap e1339: 12 total, 12 up, 12 in 2023-09-10T09:50:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37841.237749 secs 2023-09-10T09:50:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37840.885548 secs 2023-09-10T09:50:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:09 smithi082 ceph-mon[98694]: osdmap e1339: 12 total, 12 up, 12 in 2023-09-10T09:50:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37841.237749 secs 2023-09-10T09:50:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:10 smithi098 ceph-mon[103124]: pgmap v20340: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 767 KiB/s wr, 34 op/s 2023-09-10T09:50:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:10 smithi170 ceph-mon[102832]: pgmap v20340: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 767 KiB/s wr, 34 op/s 2023-09-10T09:50:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:10 smithi082 ceph-mon[98694]: pgmap v20340: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 767 KiB/s wr, 34 op/s 2023-09-10T09:50:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:12 smithi098 ceph-mon[103124]: pgmap v20341: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 738 KiB/s wr, 33 op/s 2023-09-10T09:50:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:12 smithi170 ceph-mon[102832]: pgmap v20341: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 738 KiB/s wr, 33 op/s 2023-09-10T09:50:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:12 smithi082 ceph-mon[98694]: pgmap v20341: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 738 KiB/s wr, 33 op/s 2023-09-10T09:50:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37844.639092 secs 2023-09-10T09:50:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37844.639092 secs 2023-09-10T09:50:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37844.639092 secs 2023-09-10T09:50:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:14 smithi098 ceph-mon[103124]: pgmap v20342: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 738 KiB/s wr, 33 op/s 2023-09-10T09:50:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37845.885603 secs 2023-09-10T09:50:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37846.237813 secs 2023-09-10T09:50:15.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:14 smithi098 ceph-mon[103124]: osdmap e1340: 12 total, 12 up, 12 in 2023-09-10T09:50:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:14 smithi170 ceph-mon[102832]: pgmap v20342: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 738 KiB/s wr, 33 op/s 2023-09-10T09:50:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37845.885603 secs 2023-09-10T09:50:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37846.237813 secs 2023-09-10T09:50:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:14 smithi170 ceph-mon[102832]: osdmap e1340: 12 total, 12 up, 12 in 2023-09-10T09:50:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:14 smithi082 ceph-mon[98694]: pgmap v20342: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 738 KiB/s wr, 33 op/s 2023-09-10T09:50:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37845.885603 secs 2023-09-10T09:50:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37846.237813 secs 2023-09-10T09:50:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:14 smithi082 ceph-mon[98694]: osdmap e1340: 12 total, 12 up, 12 in 2023-09-10T09:50:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:16 smithi098 ceph-mon[103124]: pgmap v20344: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 251 KiB/s wr, 17 op/s 2023-09-10T09:50:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:16 smithi170 ceph-mon[102832]: pgmap v20344: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 251 KiB/s wr, 17 op/s 2023-09-10T09:50:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:16 smithi082 ceph-mon[98694]: pgmap v20344: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 251 KiB/s wr, 17 op/s 2023-09-10T09:50:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:18 smithi098 ceph-mon[103124]: pgmap v20345: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 16 op/s 2023-09-10T09:50:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37849.639151 secs 2023-09-10T09:50:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:18 smithi170 ceph-mon[102832]: pgmap v20345: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 16 op/s 2023-09-10T09:50:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37849.639151 secs 2023-09-10T09:50:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:18 smithi082 ceph-mon[98694]: pgmap v20345: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 227 KiB/s wr, 16 op/s 2023-09-10T09:50:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37849.639151 secs 2023-09-10T09:50:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37850.885664 secs 2023-09-10T09:50:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37851.237878 secs 2023-09-10T09:50:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37850.885664 secs 2023-09-10T09:50:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37851.237878 secs 2023-09-10T09:50:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37850.885664 secs 2023-09-10T09:50:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37851.237878 secs 2023-09-10T09:50:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:20 smithi098 ceph-mon[103124]: pgmap v20346: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 201 KiB/s wr, 14 op/s 2023-09-10T09:50:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:20 smithi170 ceph-mon[102832]: pgmap v20346: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 201 KiB/s wr, 14 op/s 2023-09-10T09:50:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:20 smithi082 ceph-mon[98694]: pgmap v20346: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 201 KiB/s wr, 14 op/s 2023-09-10T09:50:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:22 smithi098 ceph-mon[103124]: pgmap v20347: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 201 KiB/s wr, 14 op/s 2023-09-10T09:50:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:22 smithi170 ceph-mon[102832]: pgmap v20347: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 201 KiB/s wr, 14 op/s 2023-09-10T09:50:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:22 smithi082 ceph-mon[98694]: pgmap v20347: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 201 KiB/s wr, 14 op/s 2023-09-10T09:50:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37854.639243 secs 2023-09-10T09:50:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37854.639243 secs 2023-09-10T09:50:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37854.639243 secs 2023-09-10T09:50:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:24 smithi098 ceph-mon[103124]: pgmap v20348: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 201 KiB/s wr, 14 op/s 2023-09-10T09:50:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37855.885723 secs 2023-09-10T09:50:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37856.237944 secs 2023-09-10T09:50:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:24 smithi170 ceph-mon[102832]: pgmap v20348: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 201 KiB/s wr, 14 op/s 2023-09-10T09:50:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37855.885723 secs 2023-09-10T09:50:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37856.237944 secs 2023-09-10T09:50:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:24 smithi082 ceph-mon[98694]: pgmap v20348: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 201 KiB/s wr, 14 op/s 2023-09-10T09:50:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37855.885723 secs 2023-09-10T09:50:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37856.237944 secs 2023-09-10T09:50:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:26 smithi170 ceph-mon[102832]: pgmap v20349: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 179 KiB/s wr, 19 op/s 2023-09-10T09:50:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:26 smithi082 ceph-mon[98694]: pgmap v20349: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 179 KiB/s wr, 19 op/s 2023-09-10T09:50:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:26 smithi098 ceph-mon[103124]: pgmap v20349: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 179 KiB/s wr, 19 op/s 2023-09-10T09:50:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:28 smithi170 ceph-mon[102832]: pgmap v20350: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T09:50:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37859.639302 secs 2023-09-10T09:50:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:28 smithi082 ceph-mon[98694]: pgmap v20350: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T09:50:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37859.639302 secs 2023-09-10T09:50:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:28 smithi098 ceph-mon[103124]: pgmap v20350: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T09:50:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37859.639302 secs 2023-09-10T09:50:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37860.885779 secs 2023-09-10T09:50:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37861.238009 secs 2023-09-10T09:50:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37860.885779 secs 2023-09-10T09:50:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37861.238009 secs 2023-09-10T09:50:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37860.885779 secs 2023-09-10T09:50:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37861.238009 secs 2023-09-10T09:50:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:30 smithi170 ceph-mon[102832]: pgmap v20351: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T09:50:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:50:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:30 smithi082 ceph-mon[98694]: pgmap v20351: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T09:50:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:50:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:30 smithi098 ceph-mon[103124]: pgmap v20351: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T09:50:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:50:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:32 smithi170 ceph-mon[102832]: pgmap v20352: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 303 KiB/s wr, 17 op/s 2023-09-10T09:50:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:50:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:32 smithi082 ceph-mon[98694]: pgmap v20352: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 303 KiB/s wr, 17 op/s 2023-09-10T09:50:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:50:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:32 smithi098 ceph-mon[103124]: pgmap v20352: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 303 KiB/s wr, 17 op/s 2023-09-10T09:50:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:50:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:34 smithi170 ceph-mon[102832]: pgmap v20353: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 164 KiB/s wr, 11 op/s 2023-09-10T09:50:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37865.885841 secs 2023-09-10T09:50:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37866.238073 secs 2023-09-10T09:50:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:34 smithi082 ceph-mon[98694]: pgmap v20353: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 164 KiB/s wr, 11 op/s 2023-09-10T09:50:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37865.885841 secs 2023-09-10T09:50:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37866.238073 secs 2023-09-10T09:50:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:34 smithi098 ceph-mon[103124]: pgmap v20353: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 164 KiB/s wr, 11 op/s 2023-09-10T09:50:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37865.885841 secs 2023-09-10T09:50:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37866.238073 secs 2023-09-10T09:50:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:50:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:50:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:50:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:36 smithi170 ceph-mon[102832]: pgmap v20354: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 343 KiB/s wr, 16 op/s 2023-09-10T09:50:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:50:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:50:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:36 smithi082 ceph-mon[98694]: pgmap v20354: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 343 KiB/s wr, 16 op/s 2023-09-10T09:50:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:50:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:50:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:36 smithi098 ceph-mon[103124]: pgmap v20354: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 343 KiB/s wr, 16 op/s 2023-09-10T09:50:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:50:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:50:37.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:38 smithi170 ceph-mon[102832]: pgmap v20355: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 343 KiB/s wr, 11 op/s 2023-09-10T09:50:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:50:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:39.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37869.639423 secs 2023-09-10T09:50:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:38 smithi082 ceph-mon[98694]: pgmap v20355: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 343 KiB/s wr, 11 op/s 2023-09-10T09:50:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:50:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37869.639423 secs 2023-09-10T09:50:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:38 smithi098 ceph-mon[103124]: pgmap v20355: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 343 KiB/s wr, 11 op/s 2023-09-10T09:50:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:50:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:39.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37869.639423 secs 2023-09-10T09:50:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37870.885903 secs 2023-09-10T09:50:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37871.238138 secs 2023-09-10T09:50:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37870.885903 secs 2023-09-10T09:50:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37871.238138 secs 2023-09-10T09:50:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37870.885903 secs 2023-09-10T09:50:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37871.238138 secs 2023-09-10T09:50:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:40 smithi170 ceph-mon[102832]: pgmap v20356: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 343 KiB/s wr, 11 op/s 2023-09-10T09:50:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:50:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:40 smithi082 ceph-mon[98694]: pgmap v20356: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 343 KiB/s wr, 11 op/s 2023-09-10T09:50:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:50:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:40 smithi098 ceph-mon[103124]: pgmap v20356: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 343 KiB/s wr, 11 op/s 2023-09-10T09:50:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:50:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:42 smithi170 ceph-mon[102832]: pgmap v20357: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 508 KiB/s wr, 17 op/s 2023-09-10T09:50:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:42 smithi082 ceph-mon[98694]: pgmap v20357: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 508 KiB/s wr, 17 op/s 2023-09-10T09:50:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:42 smithi098 ceph-mon[103124]: pgmap v20357: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 508 KiB/s wr, 17 op/s 2023-09-10T09:50:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:50:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:50:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:50:45.104 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:44 smithi170 ceph-mon[102832]: pgmap v20358: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 344 KiB/s wr, 12 op/s 2023-09-10T09:50:45.104 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37875.885989 secs 2023-09-10T09:50:45.105 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:44 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:50:45.105 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:50:45.105 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:45.105 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37876.238201 secs 2023-09-10T09:50:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:44 smithi082 ceph-mon[98694]: pgmap v20358: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 344 KiB/s wr, 12 op/s 2023-09-10T09:50:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37875.885989 secs 2023-09-10T09:50:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:44 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:50:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:50:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37876.238201 secs 2023-09-10T09:50:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:44 smithi098 ceph-mon[103124]: pgmap v20358: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 344 KiB/s wr, 12 op/s 2023-09-10T09:50:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37875.885989 secs 2023-09-10T09:50:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:44 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:50:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:50:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37876.238201 secs 2023-09-10T09:50:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:46 smithi170 ceph-mon[102832]: pgmap v20359: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 869 KiB/s wr, 19 op/s 2023-09-10T09:50:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:50:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:46 smithi082 ceph-mon[98694]: pgmap v20359: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 869 KiB/s wr, 19 op/s 2023-09-10T09:50:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:50:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:46 smithi098 ceph-mon[103124]: pgmap v20359: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 869 KiB/s wr, 19 op/s 2023-09-10T09:50:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:50:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:48 smithi170 ceph-mon[102832]: pgmap v20360: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 689 KiB/s wr, 14 op/s 2023-09-10T09:50:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:48 smithi082 ceph-mon[98694]: pgmap v20360: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 689 KiB/s wr, 14 op/s 2023-09-10T09:50:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:48 smithi098 ceph-mon[103124]: pgmap v20360: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 689 KiB/s wr, 14 op/s 2023-09-10T09:50:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37880.886048 secs 2023-09-10T09:50:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37881.238268 secs 2023-09-10T09:50:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37880.886048 secs 2023-09-10T09:50:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37881.238268 secs 2023-09-10T09:50:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37880.886048 secs 2023-09-10T09:50:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37881.238268 secs 2023-09-10T09:50:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:50 smithi170 ceph-mon[102832]: pgmap v20361: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 689 KiB/s wr, 14 op/s 2023-09-10T09:50:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:50:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:50 smithi082 ceph-mon[98694]: pgmap v20361: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 689 KiB/s wr, 14 op/s 2023-09-10T09:50:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:50:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:50 smithi098 ceph-mon[103124]: pgmap v20361: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 689 KiB/s wr, 14 op/s 2023-09-10T09:50:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:50:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:52 smithi170 ceph-mon[102832]: pgmap v20362: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:50:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:52 smithi082 ceph-mon[98694]: pgmap v20362: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:50:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:52 smithi098 ceph-mon[103124]: pgmap v20362: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:50:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:54 smithi170 ceph-mon[102832]: pgmap v20363: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 967 KiB/s wr, 15 op/s 2023-09-10T09:50:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37885.886132 secs 2023-09-10T09:50:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:50:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:55.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37886.238334 secs 2023-09-10T09:50:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:54 smithi082 ceph-mon[98694]: pgmap v20363: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 967 KiB/s wr, 15 op/s 2023-09-10T09:50:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37885.886132 secs 2023-09-10T09:50:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:50:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37886.238334 secs 2023-09-10T09:50:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:54 smithi098 ceph-mon[103124]: pgmap v20363: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 967 KiB/s wr, 15 op/s 2023-09-10T09:50:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37885.886132 secs 2023-09-10T09:50:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:50:55.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:55.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37886.238334 secs 2023-09-10T09:50:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:56 smithi170 ceph-mon[102832]: pgmap v20364: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 22 op/s 2023-09-10T09:50:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:50:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:56 smithi082 ceph-mon[98694]: pgmap v20364: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 22 op/s 2023-09-10T09:50:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:50:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:56 smithi098 ceph-mon[103124]: pgmap v20364: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 22 op/s 2023-09-10T09:50:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:50:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:57 smithi170 ceph-mon[102832]: pgmap v20365: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 544 KiB/s wr, 15 op/s 2023-09-10T09:50:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:50:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:57 smithi082 ceph-mon[98694]: pgmap v20365: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 544 KiB/s wr, 15 op/s 2023-09-10T09:50:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:50:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:57 smithi098 ceph-mon[103124]: pgmap v20365: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 544 KiB/s wr, 15 op/s 2023-09-10T09:50:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:50:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:50:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37889.639693 secs 2023-09-10T09:50:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37889.639693 secs 2023-09-10T09:50:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37889.639693 secs 2023-09-10T09:51:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37890.886217 secs 2023-09-10T09:51:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37891.238399 secs 2023-09-10T09:51:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:50:59 smithi170 ceph-mon[102832]: pgmap v20366: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 544 KiB/s wr, 15 op/s 2023-09-10T09:51:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37890.886217 secs 2023-09-10T09:51:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37891.238399 secs 2023-09-10T09:51:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:50:59 smithi082 ceph-mon[98694]: pgmap v20366: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 544 KiB/s wr, 15 op/s 2023-09-10T09:51:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37890.886217 secs 2023-09-10T09:51:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37891.238399 secs 2023-09-10T09:51:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:50:59 smithi098 ceph-mon[103124]: pgmap v20366: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 544 KiB/s wr, 15 op/s 2023-09-10T09:51:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:51:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:51:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:01.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:00 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:51:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:51:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:51:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:00 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:51:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:51:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:51:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:00 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:51:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:51:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:51:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:51:02.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:51:02.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:02 smithi170 ceph-mon[102832]: pgmap v20367: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 34 op/s 2023-09-10T09:51:02.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:51:02.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:51:02.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:51:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:51:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:51:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:51:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:51:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:02 smithi082 ceph-mon[98694]: pgmap v20367: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 34 op/s 2023-09-10T09:51:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:51:02.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:51:02.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:51:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:51:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:51:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:51:02.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:51:02.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:02 smithi098 ceph-mon[103124]: pgmap v20367: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 34 op/s 2023-09-10T09:51:02.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:51:02.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:51:02.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:51:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37894.639754 secs 2023-09-10T09:51:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:03 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:51:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:03 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:03.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:03 smithi098 ceph-mon[103124]: osdmap e1341: 12 total, 12 up, 12 in 2023-09-10T09:51:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37894.639754 secs 2023-09-10T09:51:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:03 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:51:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:03 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:03 smithi170 ceph-mon[102832]: osdmap e1341: 12 total, 12 up, 12 in 2023-09-10T09:51:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37894.639754 secs 2023-09-10T09:51:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:03 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:51:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:03 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:03 smithi082 ceph-mon[98694]: osdmap e1341: 12 total, 12 up, 12 in 2023-09-10T09:51:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:04 smithi098 ceph-mon[103124]: pgmap v20369: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 781 KiB/s wr, 32 op/s 2023-09-10T09:51:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:04 smithi170 ceph-mon[102832]: pgmap v20369: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 781 KiB/s wr, 32 op/s 2023-09-10T09:51:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:04 smithi082 ceph-mon[98694]: pgmap v20369: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 781 KiB/s wr, 32 op/s 2023-09-10T09:51:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37895.886278 secs 2023-09-10T09:51:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37896.238463 secs 2023-09-10T09:51:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37895.886278 secs 2023-09-10T09:51:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37896.238463 secs 2023-09-10T09:51:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37895.886278 secs 2023-09-10T09:51:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37896.238463 secs 2023-09-10T09:51:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:51:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:06 smithi170 ceph-mon[102832]: pgmap v20370: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.0 MiB/s wr, 34 op/s 2023-09-10T09:51:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:51:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:06 smithi082 ceph-mon[98694]: pgmap v20370: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.0 MiB/s wr, 34 op/s 2023-09-10T09:51:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:51:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:06.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:06 smithi098 ceph-mon[103124]: pgmap v20370: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.0 MiB/s wr, 34 op/s 2023-09-10T09:51:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:08 smithi170 ceph-mon[102832]: pgmap v20371: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.0 MiB/s wr, 34 op/s 2023-09-10T09:51:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37899.639778 secs 2023-09-10T09:51:08.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:08 smithi082 ceph-mon[98694]: pgmap v20371: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.0 MiB/s wr, 34 op/s 2023-09-10T09:51:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37899.639778 secs 2023-09-10T09:51:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:08 smithi098 ceph-mon[103124]: pgmap v20371: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.0 MiB/s wr, 34 op/s 2023-09-10T09:51:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37899.639778 secs 2023-09-10T09:51:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37900.886360 secs 2023-09-10T09:51:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37901.238492 secs 2023-09-10T09:51:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:09 smithi170 ceph-mon[102832]: osdmap e1342: 12 total, 12 up, 12 in 2023-09-10T09:51:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37900.886360 secs 2023-09-10T09:51:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37901.238492 secs 2023-09-10T09:51:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:09 smithi082 ceph-mon[98694]: osdmap e1342: 12 total, 12 up, 12 in 2023-09-10T09:51:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37900.886360 secs 2023-09-10T09:51:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37901.238492 secs 2023-09-10T09:51:10.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:09 smithi098 ceph-mon[103124]: osdmap e1342: 12 total, 12 up, 12 in 2023-09-10T09:51:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:10 smithi170 ceph-mon[102832]: pgmap v20373: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 497 KiB/s wr, 12 op/s 2023-09-10T09:51:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:10 smithi082 ceph-mon[98694]: pgmap v20373: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 497 KiB/s wr, 12 op/s 2023-09-10T09:51:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:10 smithi098 ceph-mon[103124]: pgmap v20373: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 497 KiB/s wr, 12 op/s 2023-09-10T09:51:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:12 smithi170 ceph-mon[102832]: pgmap v20374: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 672 KiB/s wr, 22 op/s 2023-09-10T09:51:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:51:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:12 smithi082 ceph-mon[98694]: pgmap v20374: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 672 KiB/s wr, 22 op/s 2023-09-10T09:51:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:51:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:12 smithi098 ceph-mon[103124]: pgmap v20374: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 672 KiB/s wr, 22 op/s 2023-09-10T09:51:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:51:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37904.639838 secs 2023-09-10T09:51:13.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37904.639838 secs 2023-09-10T09:51:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37904.639838 secs 2023-09-10T09:51:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:14 smithi170 ceph-mon[102832]: pgmap v20375: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 564 KiB/s wr, 18 op/s 2023-09-10T09:51:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37905.886435 secs 2023-09-10T09:51:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:14 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:51:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:14 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:14.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37906.238559 secs 2023-09-10T09:51:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:14 smithi082 ceph-mon[98694]: pgmap v20375: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 564 KiB/s wr, 18 op/s 2023-09-10T09:51:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37905.886435 secs 2023-09-10T09:51:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:14 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:51:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:14 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37906.238559 secs 2023-09-10T09:51:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:14 smithi098 ceph-mon[103124]: pgmap v20375: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 564 KiB/s wr, 18 op/s 2023-09-10T09:51:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37905.886435 secs 2023-09-10T09:51:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:14 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:51:15.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:14 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:15.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37906.238559 secs 2023-09-10T09:51:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:16 smithi170 ceph-mon[102832]: pgmap v20376: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 201 KiB/s wr, 16 op/s 2023-09-10T09:51:16.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:16 smithi082 ceph-mon[98694]: pgmap v20376: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 201 KiB/s wr, 16 op/s 2023-09-10T09:51:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:16 smithi098 ceph-mon[103124]: pgmap v20376: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 201 KiB/s wr, 16 op/s 2023-09-10T09:51:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:18 smithi170 ceph-mon[102832]: pgmap v20377: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 201 KiB/s wr, 16 op/s 2023-09-10T09:51:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37909.639899 secs 2023-09-10T09:51:18.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:18 smithi082 ceph-mon[98694]: pgmap v20377: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 201 KiB/s wr, 16 op/s 2023-09-10T09:51:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37909.639899 secs 2023-09-10T09:51:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:18 smithi098 ceph-mon[103124]: pgmap v20377: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 201 KiB/s wr, 16 op/s 2023-09-10T09:51:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37909.639899 secs 2023-09-10T09:51:19.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37910.886511 secs 2023-09-10T09:51:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37911.238622 secs 2023-09-10T09:51:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37910.886511 secs 2023-09-10T09:51:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37911.238622 secs 2023-09-10T09:51:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37910.886511 secs 2023-09-10T09:51:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37911.238622 secs 2023-09-10T09:51:20.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:20 smithi082 ceph-mon[98694]: pgmap v20378: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 15 op/s 2023-09-10T09:51:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:20 smithi098 ceph-mon[103124]: pgmap v20378: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 15 op/s 2023-09-10T09:51:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:20 smithi170 ceph-mon[102832]: pgmap v20378: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 15 op/s 2023-09-10T09:51:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:22 smithi082 ceph-mon[98694]: pgmap v20379: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 168 KiB/s wr, 19 op/s 2023-09-10T09:51:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:22 smithi098 ceph-mon[103124]: pgmap v20379: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 168 KiB/s wr, 19 op/s 2023-09-10T09:51:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:22 smithi170 ceph-mon[102832]: pgmap v20379: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 168 KiB/s wr, 19 op/s 2023-09-10T09:51:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37914.639983 secs 2023-09-10T09:51:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37914.639983 secs 2023-09-10T09:51:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37914.639983 secs 2023-09-10T09:51:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:24 smithi082 ceph-mon[98694]: pgmap v20380: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 12 op/s 2023-09-10T09:51:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37915.886572 secs 2023-09-10T09:51:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37916.238686 secs 2023-09-10T09:51:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:24 smithi098 ceph-mon[103124]: pgmap v20380: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 12 op/s 2023-09-10T09:51:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37915.886572 secs 2023-09-10T09:51:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37916.238686 secs 2023-09-10T09:51:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:24 smithi170 ceph-mon[102832]: pgmap v20380: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 12 op/s 2023-09-10T09:51:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37915.886572 secs 2023-09-10T09:51:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37916.238686 secs 2023-09-10T09:51:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:26 smithi082 ceph-mon[98694]: pgmap v20381: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 18 op/s 2023-09-10T09:51:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:26 smithi098 ceph-mon[103124]: pgmap v20381: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 18 op/s 2023-09-10T09:51:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:26 smithi170 ceph-mon[102832]: pgmap v20381: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 18 op/s 2023-09-10T09:51:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:28 smithi082 ceph-mon[98694]: pgmap v20382: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T09:51:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37919.640100 secs 2023-09-10T09:51:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:28 smithi098 ceph-mon[103124]: pgmap v20382: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T09:51:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37919.640100 secs 2023-09-10T09:51:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:28 smithi170 ceph-mon[102832]: pgmap v20382: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T09:51:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37919.640100 secs 2023-09-10T09:51:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37920.886633 secs 2023-09-10T09:51:29.963 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37921.238754 secs 2023-09-10T09:51:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37920.886633 secs 2023-09-10T09:51:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37921.238754 secs 2023-09-10T09:51:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37920.886633 secs 2023-09-10T09:51:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37921.238754 secs 2023-09-10T09:51:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:30 smithi082 ceph-mon[98694]: pgmap v20383: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T09:51:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:30 smithi098 ceph-mon[103124]: pgmap v20383: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T09:51:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:30 smithi170 ceph-mon[102832]: pgmap v20383: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T09:51:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:32 smithi082 ceph-mon[98694]: pgmap v20384: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 140 KiB/s wr, 16 op/s 2023-09-10T09:51:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:32 smithi098 ceph-mon[103124]: pgmap v20384: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 140 KiB/s wr, 16 op/s 2023-09-10T09:51:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:32 smithi170 ceph-mon[102832]: pgmap v20384: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 140 KiB/s wr, 16 op/s 2023-09-10T09:51:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:34 smithi082 ceph-mon[98694]: pgmap v20385: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T09:51:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37925.886696 secs 2023-09-10T09:51:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:51:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37926.238818 secs 2023-09-10T09:51:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:34 smithi098 ceph-mon[103124]: pgmap v20385: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T09:51:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37925.886696 secs 2023-09-10T09:51:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:51:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:35.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37926.238818 secs 2023-09-10T09:51:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:34 smithi170 ceph-mon[102832]: pgmap v20385: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 10 op/s 2023-09-10T09:51:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37925.886696 secs 2023-09-10T09:51:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:51:35.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:35.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37926.238818 secs 2023-09-10T09:51:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:51:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:51:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:51:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:36 smithi082 ceph-mon[98694]: pgmap v20386: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 439 KiB/s wr, 15 op/s 2023-09-10T09:51:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:51:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:51:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:36 smithi098 ceph-mon[103124]: pgmap v20386: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 439 KiB/s wr, 15 op/s 2023-09-10T09:51:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:51:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:51:37.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:36 smithi170 ceph-mon[102832]: pgmap v20386: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 439 KiB/s wr, 15 op/s 2023-09-10T09:51:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:51:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:51:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:38 smithi098 ceph-mon[103124]: pgmap v20387: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 300 KiB/s wr, 10 op/s 2023-09-10T09:51:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:38 smithi170 ceph-mon[102832]: pgmap v20387: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 300 KiB/s wr, 10 op/s 2023-09-10T09:51:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:38 smithi082 ceph-mon[98694]: pgmap v20387: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 300 KiB/s wr, 10 op/s 2023-09-10T09:51:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37930.886754 secs 2023-09-10T09:51:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37931.238884 secs 2023-09-10T09:51:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37930.886754 secs 2023-09-10T09:51:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37931.238884 secs 2023-09-10T09:51:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37930.886754 secs 2023-09-10T09:51:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37931.238884 secs 2023-09-10T09:51:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:40 smithi098 ceph-mon[103124]: pgmap v20388: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 300 KiB/s wr, 10 op/s 2023-09-10T09:51:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:40 smithi170 ceph-mon[102832]: pgmap v20388: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 300 KiB/s wr, 10 op/s 2023-09-10T09:51:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:40 smithi082 ceph-mon[98694]: pgmap v20388: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 300 KiB/s wr, 10 op/s 2023-09-10T09:51:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:42 smithi098 ceph-mon[103124]: pgmap v20389: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 465 KiB/s wr, 17 op/s 2023-09-10T09:51:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:42 smithi170 ceph-mon[102832]: pgmap v20389: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 465 KiB/s wr, 17 op/s 2023-09-10T09:51:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:42 smithi082 ceph-mon[98694]: pgmap v20389: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 465 KiB/s wr, 17 op/s 2023-09-10T09:51:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:44 smithi098 ceph-mon[103124]: pgmap v20390: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 465 KiB/s wr, 12 op/s 2023-09-10T09:51:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37935.886817 secs 2023-09-10T09:51:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:51:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37936.238949 secs 2023-09-10T09:51:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:44 smithi170 ceph-mon[102832]: pgmap v20390: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 465 KiB/s wr, 12 op/s 2023-09-10T09:51:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37935.886817 secs 2023-09-10T09:51:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:51:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37936.238949 secs 2023-09-10T09:51:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:44 smithi082 ceph-mon[98694]: pgmap v20390: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 465 KiB/s wr, 12 op/s 2023-09-10T09:51:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37935.886817 secs 2023-09-10T09:51:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:51:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37936.238949 secs 2023-09-10T09:51:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:46 smithi098 ceph-mon[103124]: pgmap v20391: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 544 KiB/s wr, 19 op/s 2023-09-10T09:51:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:51:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:46 smithi170 ceph-mon[102832]: pgmap v20391: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 544 KiB/s wr, 19 op/s 2023-09-10T09:51:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:51:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:46 smithi082 ceph-mon[98694]: pgmap v20391: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 544 KiB/s wr, 19 op/s 2023-09-10T09:51:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:51:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:48 smithi098 ceph-mon[103124]: pgmap v20392: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 244 KiB/s wr, 13 op/s 2023-09-10T09:51:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37939.640384 secs 2023-09-10T09:51:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:48 smithi170 ceph-mon[102832]: pgmap v20392: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 244 KiB/s wr, 13 op/s 2023-09-10T09:51:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37939.640384 secs 2023-09-10T09:51:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:48 smithi082 ceph-mon[98694]: pgmap v20392: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 244 KiB/s wr, 13 op/s 2023-09-10T09:51:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37939.640384 secs 2023-09-10T09:51:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37940.886874 secs 2023-09-10T09:51:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37941.239014 secs 2023-09-10T09:51:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37940.886874 secs 2023-09-10T09:51:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37941.239014 secs 2023-09-10T09:51:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37940.886874 secs 2023-09-10T09:51:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37941.239014 secs 2023-09-10T09:51:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:50 smithi098 ceph-mon[103124]: pgmap v20393: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 244 KiB/s wr, 13 op/s 2023-09-10T09:51:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:51:51.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:50 smithi170 ceph-mon[102832]: pgmap v20393: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 244 KiB/s wr, 13 op/s 2023-09-10T09:51:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:51:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:50 smithi082 ceph-mon[98694]: pgmap v20393: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 244 KiB/s wr, 13 op/s 2023-09-10T09:51:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:51:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:51:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:51:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:51:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:51:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:51:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:51:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:52 smithi170 ceph-mon[102832]: pgmap v20394: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 760 KiB/s wr, 21 op/s 2023-09-10T09:51:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:52 smithi082 ceph-mon[98694]: pgmap v20394: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 760 KiB/s wr, 21 op/s 2023-09-10T09:51:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:52 smithi098 ceph-mon[103124]: pgmap v20394: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 760 KiB/s wr, 21 op/s 2023-09-10T09:51:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:54 smithi170 ceph-mon[102832]: pgmap v20395: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 594 KiB/s wr, 14 op/s 2023-09-10T09:51:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37945.886948 secs 2023-09-10T09:51:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37946.239077 secs 2023-09-10T09:51:55.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:54 smithi082 ceph-mon[98694]: pgmap v20395: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 594 KiB/s wr, 14 op/s 2023-09-10T09:51:55.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37945.886948 secs 2023-09-10T09:51:55.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37946.239077 secs 2023-09-10T09:51:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:54 smithi098 ceph-mon[103124]: pgmap v20395: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 594 KiB/s wr, 14 op/s 2023-09-10T09:51:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37945.886948 secs 2023-09-10T09:51:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37946.239077 secs 2023-09-10T09:51:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:55 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:51:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:55 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:51:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:55 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:51:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:56 smithi170 ceph-mon[102832]: pgmap v20396: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 733 KiB/s wr, 21 op/s 2023-09-10T09:51:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:51:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:56 smithi082 ceph-mon[98694]: pgmap v20396: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 733 KiB/s wr, 21 op/s 2023-09-10T09:51:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:51:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:56 smithi098 ceph-mon[103124]: pgmap v20396: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 733 KiB/s wr, 21 op/s 2023-09-10T09:51:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:51:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:58 smithi170 ceph-mon[102832]: pgmap v20397: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 654 KiB/s wr, 14 op/s 2023-09-10T09:51:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:51:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:58 smithi082 ceph-mon[98694]: pgmap v20397: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 654 KiB/s wr, 14 op/s 2023-09-10T09:51:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:51:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:51:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:58 smithi098 ceph-mon[103124]: pgmap v20397: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 654 KiB/s wr, 14 op/s 2023-09-10T09:51:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:51:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37950.887031 secs 2023-09-10T09:52:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37951.239143 secs 2023-09-10T09:52:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:51:59 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:52:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37950.887031 secs 2023-09-10T09:52:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37951.239143 secs 2023-09-10T09:52:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:51:59 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:52:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37950.887031 secs 2023-09-10T09:52:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37951.239143 secs 2023-09-10T09:52:00.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:51:59 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:52:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:00 smithi170 ceph-mon[102832]: pgmap v20398: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 654 KiB/s wr, 14 op/s 2023-09-10T09:52:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:52:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:00 smithi082 ceph-mon[98694]: pgmap v20398: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 654 KiB/s wr, 14 op/s 2023-09-10T09:52:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:52:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:00 smithi098 ceph-mon[103124]: pgmap v20398: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 654 KiB/s wr, 14 op/s 2023-09-10T09:52:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:52:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:02 smithi170 ceph-mon[102832]: pgmap v20399: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 22 op/s 2023-09-10T09:52:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:52:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:52:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:52:03.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:52:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:02 smithi082 ceph-mon[98694]: pgmap v20399: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 22 op/s 2023-09-10T09:52:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:52:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:52:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:52:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:52:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:02 smithi098 ceph-mon[103124]: pgmap v20399: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 22 op/s 2023-09-10T09:52:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:52:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:52:03.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:52:03.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:52:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37954.640660 secs 2023-09-10T09:52:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37954.640660 secs 2023-09-10T09:52:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37954.640660 secs 2023-09-10T09:52:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:04 smithi170 ceph-mon[102832]: pgmap v20400: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 857 KiB/s wr, 14 op/s 2023-09-10T09:52:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37955.887088 secs 2023-09-10T09:52:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:52:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:04 smithi170 ceph-mon[102832]: osdmap e1343: 12 total, 12 up, 12 in 2023-09-10T09:52:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37956.239209 secs 2023-09-10T09:52:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:52:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:04 smithi082 ceph-mon[98694]: pgmap v20400: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 857 KiB/s wr, 14 op/s 2023-09-10T09:52:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37955.887088 secs 2023-09-10T09:52:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:52:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:04 smithi082 ceph-mon[98694]: osdmap e1343: 12 total, 12 up, 12 in 2023-09-10T09:52:05.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37956.239209 secs 2023-09-10T09:52:05.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:52:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:04 smithi098 ceph-mon[103124]: pgmap v20400: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 857 KiB/s wr, 14 op/s 2023-09-10T09:52:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37955.887088 secs 2023-09-10T09:52:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:52:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:04 smithi098 ceph-mon[103124]: osdmap e1343: 12 total, 12 up, 12 in 2023-09-10T09:52:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37956.239209 secs 2023-09-10T09:52:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:52:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:06 smithi170 ceph-mon[102832]: pgmap v20402: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 34 op/s 2023-09-10T09:52:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:06 smithi082 ceph-mon[98694]: pgmap v20402: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 34 op/s 2023-09-10T09:52:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:06 smithi098 ceph-mon[103124]: pgmap v20402: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 34 op/s 2023-09-10T09:52:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:08 smithi170 ceph-mon[102832]: pgmap v20403: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 34 op/s 2023-09-10T09:52:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37959.640770 secs 2023-09-10T09:52:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:08 smithi082 ceph-mon[98694]: pgmap v20403: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 34 op/s 2023-09-10T09:52:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37959.640770 secs 2023-09-10T09:52:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:08 smithi098 ceph-mon[103124]: pgmap v20403: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 34 op/s 2023-09-10T09:52:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37959.640770 secs 2023-09-10T09:52:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37960.887171 secs 2023-09-10T09:52:10.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37961.239273 secs 2023-09-10T09:52:10.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:09 smithi170 ceph-mon[102832]: osdmap e1344: 12 total, 12 up, 12 in 2023-09-10T09:52:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37960.887171 secs 2023-09-10T09:52:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37961.239273 secs 2023-09-10T09:52:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:09 smithi082 ceph-mon[98694]: osdmap e1344: 12 total, 12 up, 12 in 2023-09-10T09:52:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37960.887171 secs 2023-09-10T09:52:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37961.239273 secs 2023-09-10T09:52:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:09 smithi098 ceph-mon[103124]: osdmap e1344: 12 total, 12 up, 12 in 2023-09-10T09:52:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:10 smithi170 ceph-mon[102832]: pgmap v20405: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 259 KiB/s wr, 31 op/s 2023-09-10T09:52:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:52:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:10 smithi082 ceph-mon[98694]: pgmap v20405: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 259 KiB/s wr, 31 op/s 2023-09-10T09:52:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:52:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:10 smithi098 ceph-mon[103124]: pgmap v20405: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 259 KiB/s wr, 31 op/s 2023-09-10T09:52:11.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:52:11.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:11 smithi170 ceph-mon[102832]: pgmap v20406: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 753 KiB/s wr, 42 op/s 2023-09-10T09:52:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:11 smithi082 ceph-mon[98694]: pgmap v20406: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 753 KiB/s wr, 42 op/s 2023-09-10T09:52:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:11 smithi098 ceph-mon[103124]: pgmap v20406: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 753 KiB/s wr, 42 op/s 2023-09-10T09:52:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:52:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:52:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:52:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37964.640827 secs 2023-09-10T09:52:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:13 smithi170 ceph-mon[102832]: pgmap v20407: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 616 KiB/s wr, 34 op/s 2023-09-10T09:52:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37964.640827 secs 2023-09-10T09:52:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:13 smithi082 ceph-mon[98694]: pgmap v20407: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 616 KiB/s wr, 34 op/s 2023-09-10T09:52:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37964.640827 secs 2023-09-10T09:52:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:13 smithi098 ceph-mon[103124]: pgmap v20407: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 616 KiB/s wr, 34 op/s 2023-09-10T09:52:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37965.887229 secs 2023-09-10T09:52:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:14 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:52:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:14 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37966.239339 secs 2023-09-10T09:52:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37965.887229 secs 2023-09-10T09:52:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:14 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:52:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:14 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37966.239339 secs 2023-09-10T09:52:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37965.887229 secs 2023-09-10T09:52:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:14 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:52:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:14 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37966.239339 secs 2023-09-10T09:52:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:15 smithi170 ceph-mon[102832]: pgmap v20408: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 430 KiB/s wr, 15 op/s 2023-09-10T09:52:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:15 smithi082 ceph-mon[98694]: pgmap v20408: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 430 KiB/s wr, 15 op/s 2023-09-10T09:52:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:15 smithi098 ceph-mon[103124]: pgmap v20408: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 430 KiB/s wr, 15 op/s 2023-09-10T09:52:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:52:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:52:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:52:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:17 smithi170 ceph-mon[102832]: pgmap v20409: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 430 KiB/s wr, 15 op/s 2023-09-10T09:52:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:17 smithi082 ceph-mon[98694]: pgmap v20409: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 430 KiB/s wr, 15 op/s 2023-09-10T09:52:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:17 smithi098 ceph-mon[103124]: pgmap v20409: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 430 KiB/s wr, 15 op/s 2023-09-10T09:52:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37969.640887 secs 2023-09-10T09:52:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37969.640887 secs 2023-09-10T09:52:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37969.640887 secs 2023-09-10T09:52:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37970.887287 secs 2023-09-10T09:52:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37971.239403 secs 2023-09-10T09:52:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:19 smithi170 ceph-mon[102832]: pgmap v20410: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 419 KiB/s wr, 15 op/s 2023-09-10T09:52:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37970.887287 secs 2023-09-10T09:52:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37971.239403 secs 2023-09-10T09:52:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:19 smithi082 ceph-mon[98694]: pgmap v20410: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 419 KiB/s wr, 15 op/s 2023-09-10T09:52:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37970.887287 secs 2023-09-10T09:52:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37971.239403 secs 2023-09-10T09:52:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:19 smithi098 ceph-mon[103124]: pgmap v20410: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 419 KiB/s wr, 15 op/s 2023-09-10T09:52:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:22 smithi170 ceph-mon[102832]: pgmap v20411: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 492 KiB/s wr, 18 op/s 2023-09-10T09:52:22.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:22 smithi082 ceph-mon[98694]: pgmap v20411: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 492 KiB/s wr, 18 op/s 2023-09-10T09:52:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:22 smithi098 ceph-mon[103124]: pgmap v20411: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 492 KiB/s wr, 18 op/s 2023-09-10T09:52:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37974.640970 secs 2023-09-10T09:52:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37974.640970 secs 2023-09-10T09:52:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37974.640970 secs 2023-09-10T09:52:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:24 smithi170 ceph-mon[102832]: pgmap v20412: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T09:52:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37975.887341 secs 2023-09-10T09:52:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37976.239467 secs 2023-09-10T09:52:24.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:24 smithi082 ceph-mon[98694]: pgmap v20412: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T09:52:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37975.887341 secs 2023-09-10T09:52:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37976.239467 secs 2023-09-10T09:52:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:24 smithi098 ceph-mon[103124]: pgmap v20412: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 11 op/s 2023-09-10T09:52:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37975.887341 secs 2023-09-10T09:52:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37976.239467 secs 2023-09-10T09:52:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:26 smithi082 ceph-mon[98694]: pgmap v20413: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 302 KiB/s wr, 17 op/s 2023-09-10T09:52:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:26 smithi098 ceph-mon[103124]: pgmap v20413: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 302 KiB/s wr, 17 op/s 2023-09-10T09:52:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:26 smithi170 ceph-mon[102832]: pgmap v20413: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 302 KiB/s wr, 17 op/s 2023-09-10T09:52:28.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:28 smithi082 ceph-mon[98694]: pgmap v20414: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T09:52:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37979.641054 secs 2023-09-10T09:52:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:28 smithi098 ceph-mon[103124]: pgmap v20414: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T09:52:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37979.641054 secs 2023-09-10T09:52:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:28 smithi170 ceph-mon[102832]: pgmap v20414: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T09:52:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37979.641054 secs 2023-09-10T09:52:29.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37980.887426 secs 2023-09-10T09:52:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37981.239487 secs 2023-09-10T09:52:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37980.887426 secs 2023-09-10T09:52:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37981.239487 secs 2023-09-10T09:52:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37980.887426 secs 2023-09-10T09:52:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37981.239487 secs 2023-09-10T09:52:30.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:30 smithi082 ceph-mon[98694]: pgmap v20415: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 273 KiB/s wr, 11 op/s 2023-09-10T09:52:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:30 smithi098 ceph-mon[103124]: pgmap v20415: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 273 KiB/s wr, 11 op/s 2023-09-10T09:52:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:30 smithi170 ceph-mon[102832]: pgmap v20415: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 273 KiB/s wr, 11 op/s 2023-09-10T09:52:32.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:32 smithi082 ceph-mon[98694]: pgmap v20416: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 273 KiB/s wr, 16 op/s 2023-09-10T09:52:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:32 smithi098 ceph-mon[103124]: pgmap v20416: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 273 KiB/s wr, 16 op/s 2023-09-10T09:52:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:32 smithi170 ceph-mon[102832]: pgmap v20416: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 273 KiB/s wr, 16 op/s 2023-09-10T09:52:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:34 smithi082 ceph-mon[98694]: pgmap v20417: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T09:52:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37985.887490 secs 2023-09-10T09:52:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37986.239553 secs 2023-09-10T09:52:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:34 smithi098 ceph-mon[103124]: pgmap v20417: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T09:52:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37985.887490 secs 2023-09-10T09:52:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37986.239553 secs 2023-09-10T09:52:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:34 smithi170 ceph-mon[102832]: pgmap v20417: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 11 op/s 2023-09-10T09:52:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37985.887490 secs 2023-09-10T09:52:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37986.239553 secs 2023-09-10T09:52:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:52:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:52:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:52:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:36 smithi082 ceph-mon[98694]: pgmap v20418: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 411 KiB/s wr, 17 op/s 2023-09-10T09:52:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:52:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:52:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:36 smithi098 ceph-mon[103124]: pgmap v20418: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 411 KiB/s wr, 17 op/s 2023-09-10T09:52:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:52:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:52:37.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:36 smithi170 ceph-mon[102832]: pgmap v20418: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 411 KiB/s wr, 17 op/s 2023-09-10T09:52:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:52:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:52:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:38 smithi082 ceph-mon[98694]: pgmap v20419: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T09:52:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:52:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:38 smithi098 ceph-mon[103124]: pgmap v20419: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T09:52:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:52:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:38 smithi170 ceph-mon[102832]: pgmap v20419: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T09:52:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:52:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:39.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37990.887573 secs 2023-09-10T09:52:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37991.239618 secs 2023-09-10T09:52:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37990.887573 secs 2023-09-10T09:52:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37991.239618 secs 2023-09-10T09:52:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37990.887573 secs 2023-09-10T09:52:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37991.239618 secs 2023-09-10T09:52:40.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:40 smithi082 ceph-mon[98694]: pgmap v20420: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T09:52:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:52:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:40 smithi098 ceph-mon[103124]: pgmap v20420: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T09:52:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:52:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:40 smithi170 ceph-mon[102832]: pgmap v20420: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T09:52:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:52:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:42 smithi098 ceph-mon[103124]: pgmap v20421: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 608 KiB/s wr, 19 op/s 2023-09-10T09:52:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:42 smithi170 ceph-mon[102832]: pgmap v20421: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 608 KiB/s wr, 19 op/s 2023-09-10T09:52:43.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:42 smithi082 ceph-mon[98694]: pgmap v20421: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 608 KiB/s wr, 19 op/s 2023-09-10T09:52:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:44 smithi098 ceph-mon[103124]: pgmap v20422: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 608 KiB/s wr, 14 op/s 2023-09-10T09:52:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37995.887632 secs 2023-09-10T09:52:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 37996.239682 secs 2023-09-10T09:52:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:44 smithi170 ceph-mon[102832]: pgmap v20422: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 608 KiB/s wr, 14 op/s 2023-09-10T09:52:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37995.887632 secs 2023-09-10T09:52:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 37996.239682 secs 2023-09-10T09:52:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:44 smithi082 ceph-mon[98694]: pgmap v20422: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 608 KiB/s wr, 14 op/s 2023-09-10T09:52:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37995.887632 secs 2023-09-10T09:52:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 37996.239682 secs 2023-09-10T09:52:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:46 smithi098 ceph-mon[103124]: pgmap v20423: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 745 KiB/s wr, 20 op/s 2023-09-10T09:52:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:52:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:46 smithi170 ceph-mon[102832]: pgmap v20423: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 745 KiB/s wr, 20 op/s 2023-09-10T09:52:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:52:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:46 smithi082 ceph-mon[98694]: pgmap v20423: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 745 KiB/s wr, 20 op/s 2023-09-10T09:52:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:52:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:48 smithi098 ceph-mon[103124]: pgmap v20424: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 473 KiB/s wr, 14 op/s 2023-09-10T09:52:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 37999.641295 secs 2023-09-10T09:52:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:48 smithi170 ceph-mon[102832]: pgmap v20424: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 473 KiB/s wr, 14 op/s 2023-09-10T09:52:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 37999.641295 secs 2023-09-10T09:52:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:48 smithi082 ceph-mon[98694]: pgmap v20424: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 473 KiB/s wr, 14 op/s 2023-09-10T09:52:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 37999.641295 secs 2023-09-10T09:52:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38000.887693 secs 2023-09-10T09:52:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38001.239749 secs 2023-09-10T09:52:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38000.887693 secs 2023-09-10T09:52:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38001.239749 secs 2023-09-10T09:52:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38000.887693 secs 2023-09-10T09:52:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38001.239749 secs 2023-09-10T09:52:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:50 smithi098 ceph-mon[103124]: pgmap v20425: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 473 KiB/s wr, 14 op/s 2023-09-10T09:52:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:52:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:50 smithi170 ceph-mon[102832]: pgmap v20425: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 473 KiB/s wr, 14 op/s 2023-09-10T09:52:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:52:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:50 smithi082 ceph-mon[98694]: pgmap v20425: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 473 KiB/s wr, 14 op/s 2023-09-10T09:52:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:52:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:52:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:52:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:52:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:52:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:52:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:52:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:52:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:52 smithi170 ceph-mon[102832]: pgmap v20426: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:52:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:52 smithi082 ceph-mon[98694]: pgmap v20426: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:52:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:52 smithi098 ceph-mon[103124]: pgmap v20426: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:52:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38004.641358 secs 2023-09-10T09:52:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38004.641358 secs 2023-09-10T09:52:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38004.641358 secs 2023-09-10T09:52:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38005.887753 secs 2023-09-10T09:52:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:54 smithi170 ceph-mon[102832]: pgmap v20427: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 785 KiB/s wr, 13 op/s 2023-09-10T09:52:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38006.239814 secs 2023-09-10T09:52:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38005.887753 secs 2023-09-10T09:52:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:54 smithi082 ceph-mon[98694]: pgmap v20427: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 785 KiB/s wr, 13 op/s 2023-09-10T09:52:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38006.239814 secs 2023-09-10T09:52:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38005.887753 secs 2023-09-10T09:52:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:54 smithi098 ceph-mon[103124]: pgmap v20427: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 785 KiB/s wr, 13 op/s 2023-09-10T09:52:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38006.239814 secs 2023-09-10T09:52:57.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:56 smithi170 ceph-mon[102832]: pgmap v20428: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 997 KiB/s wr, 20 op/s 2023-09-10T09:52:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:56 smithi082 ceph-mon[98694]: pgmap v20428: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 997 KiB/s wr, 20 op/s 2023-09-10T09:52:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:56 smithi098 ceph-mon[103124]: pgmap v20428: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 997 KiB/s wr, 20 op/s 2023-09-10T09:52:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:58 smithi170 ceph-mon[102832]: pgmap v20429: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 859 KiB/s wr, 13 op/s 2023-09-10T09:52:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38009.641418 secs 2023-09-10T09:52:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:58 smithi082 ceph-mon[98694]: pgmap v20429: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 859 KiB/s wr, 13 op/s 2023-09-10T09:52:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38009.641418 secs 2023-09-10T09:52:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:58 smithi098 ceph-mon[103124]: pgmap v20429: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 859 KiB/s wr, 13 op/s 2023-09-10T09:52:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38009.641418 secs 2023-09-10T09:53:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38010.887811 secs 2023-09-10T09:53:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:52:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38011.239878 secs 2023-09-10T09:53:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38010.887811 secs 2023-09-10T09:53:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:52:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38011.239878 secs 2023-09-10T09:53:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38010.887811 secs 2023-09-10T09:53:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:52:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38011.239878 secs 2023-09-10T09:53:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:00 smithi170 ceph-mon[102832]: pgmap v20430: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 859 KiB/s wr, 13 op/s 2023-09-10T09:53:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:53:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:00 smithi082 ceph-mon[98694]: pgmap v20430: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 859 KiB/s wr, 13 op/s 2023-09-10T09:53:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:53:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:00 smithi098 ceph-mon[103124]: pgmap v20430: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 859 KiB/s wr, 13 op/s 2023-09-10T09:53:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:53:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:02 smithi170 ceph-mon[102832]: pgmap v20431: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 978 KiB/s wr, 20 op/s 2023-09-10T09:53:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:53:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:02 smithi082 ceph-mon[98694]: pgmap v20431: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 978 KiB/s wr, 20 op/s 2023-09-10T09:53:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:53:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:02 smithi098 ceph-mon[103124]: pgmap v20431: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 978 KiB/s wr, 20 op/s 2023-09-10T09:53:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:53:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38014.641492 secs 2023-09-10T09:53:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:53:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:53:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:53:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38014.641492 secs 2023-09-10T09:53:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:53:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:53:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:53:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38014.641492 secs 2023-09-10T09:53:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:53:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:53:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:53:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38015.887872 secs 2023-09-10T09:53:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:04 smithi170 ceph-mon[102832]: pgmap v20432: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 330 KiB/s wr, 13 op/s 2023-09-10T09:53:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:53:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:04 smithi170 ceph-mon[102832]: osdmap e1345: 12 total, 12 up, 12 in 2023-09-10T09:53:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38016.239943 secs 2023-09-10T09:53:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38015.887872 secs 2023-09-10T09:53:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:04 smithi082 ceph-mon[98694]: pgmap v20432: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 330 KiB/s wr, 13 op/s 2023-09-10T09:53:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:53:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:04 smithi082 ceph-mon[98694]: osdmap e1345: 12 total, 12 up, 12 in 2023-09-10T09:53:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38016.239943 secs 2023-09-10T09:53:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38015.887872 secs 2023-09-10T09:53:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:04 smithi098 ceph-mon[103124]: pgmap v20432: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 330 KiB/s wr, 13 op/s 2023-09-10T09:53:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:53:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:04 smithi098 ceph-mon[103124]: osdmap e1345: 12 total, 12 up, 12 in 2023-09-10T09:53:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38016.239943 secs 2023-09-10T09:53:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:06 smithi170 ceph-mon[102832]: pgmap v20434: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 558 KiB/s wr, 33 op/s 2023-09-10T09:53:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:53:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:06 smithi082 ceph-mon[98694]: pgmap v20434: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 558 KiB/s wr, 33 op/s 2023-09-10T09:53:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:53:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:06 smithi098 ceph-mon[103124]: pgmap v20434: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 558 KiB/s wr, 33 op/s 2023-09-10T09:53:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:53:07.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:08 smithi170 ceph-mon[102832]: pgmap v20435: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 558 KiB/s wr, 33 op/s 2023-09-10T09:53:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38019.641548 secs 2023-09-10T09:53:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:08 smithi082 ceph-mon[98694]: pgmap v20435: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 558 KiB/s wr, 33 op/s 2023-09-10T09:53:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38019.641548 secs 2023-09-10T09:53:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:08 smithi098 ceph-mon[103124]: pgmap v20435: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 558 KiB/s wr, 33 op/s 2023-09-10T09:53:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38019.641548 secs 2023-09-10T09:53:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38020.887954 secs 2023-09-10T09:53:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38021.240008 secs 2023-09-10T09:53:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:09 smithi170 ceph-mon[102832]: osdmap e1346: 12 total, 12 up, 12 in 2023-09-10T09:53:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38020.887954 secs 2023-09-10T09:53:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38021.240008 secs 2023-09-10T09:53:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:09 smithi082 ceph-mon[98694]: osdmap e1346: 12 total, 12 up, 12 in 2023-09-10T09:53:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38020.887954 secs 2023-09-10T09:53:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38021.240008 secs 2023-09-10T09:53:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:09 smithi098 ceph-mon[103124]: osdmap e1346: 12 total, 12 up, 12 in 2023-09-10T09:53:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:10 smithi170 ceph-mon[102832]: pgmap v20437: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 519 KiB/s wr, 31 op/s 2023-09-10T09:53:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:10 smithi082 ceph-mon[98694]: pgmap v20437: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 519 KiB/s wr, 31 op/s 2023-09-10T09:53:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:10 smithi098 ceph-mon[103124]: pgmap v20437: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 519 KiB/s wr, 31 op/s 2023-09-10T09:53:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:12 smithi170 ceph-mon[102832]: pgmap v20438: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 732 KiB/s wr, 41 op/s 2023-09-10T09:53:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:12 smithi082 ceph-mon[98694]: pgmap v20438: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 732 KiB/s wr, 41 op/s 2023-09-10T09:53:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:12 smithi098 ceph-mon[103124]: pgmap v20438: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 732 KiB/s wr, 41 op/s 2023-09-10T09:53:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38024.641666 secs 2023-09-10T09:53:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38024.641666 secs 2023-09-10T09:53:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38024.641666 secs 2023-09-10T09:53:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38025.888012 secs 2023-09-10T09:53:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:14 smithi170 ceph-mon[102832]: pgmap v20439: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 596 KiB/s wr, 33 op/s 2023-09-10T09:53:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38026.240077 secs 2023-09-10T09:53:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38025.888012 secs 2023-09-10T09:53:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:14 smithi082 ceph-mon[98694]: pgmap v20439: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 596 KiB/s wr, 33 op/s 2023-09-10T09:53:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38026.240077 secs 2023-09-10T09:53:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38025.888012 secs 2023-09-10T09:53:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:14 smithi098 ceph-mon[103124]: pgmap v20439: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 596 KiB/s wr, 33 op/s 2023-09-10T09:53:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38026.240077 secs 2023-09-10T09:53:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:16 smithi170 ceph-mon[102832]: pgmap v20440: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 202 KiB/s wr, 14 op/s 2023-09-10T09:53:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:16 smithi082 ceph-mon[98694]: pgmap v20440: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 202 KiB/s wr, 14 op/s 2023-09-10T09:53:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:16 smithi098 ceph-mon[103124]: pgmap v20440: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 202 KiB/s wr, 14 op/s 2023-09-10T09:53:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:18 smithi170 ceph-mon[102832]: pgmap v20441: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 202 KiB/s wr, 14 op/s 2023-09-10T09:53:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38029.641776 secs 2023-09-10T09:53:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:18 smithi082 ceph-mon[98694]: pgmap v20441: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 202 KiB/s wr, 14 op/s 2023-09-10T09:53:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38029.641776 secs 2023-09-10T09:53:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:18 smithi098 ceph-mon[103124]: pgmap v20441: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 202 KiB/s wr, 14 op/s 2023-09-10T09:53:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38029.641776 secs 2023-09-10T09:53:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38030.888064 secs 2023-09-10T09:53:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38031.240144 secs 2023-09-10T09:53:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38030.888064 secs 2023-09-10T09:53:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38031.240144 secs 2023-09-10T09:53:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38030.888064 secs 2023-09-10T09:53:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38031.240144 secs 2023-09-10T09:53:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:20 smithi170 ceph-mon[102832]: pgmap v20442: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 197 KiB/s wr, 14 op/s 2023-09-10T09:53:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:20 smithi082 ceph-mon[98694]: pgmap v20442: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 197 KiB/s wr, 14 op/s 2023-09-10T09:53:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:20 smithi098 ceph-mon[103124]: pgmap v20442: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 197 KiB/s wr, 14 op/s 2023-09-10T09:53:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:22 smithi170 ceph-mon[102832]: pgmap v20443: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 307 KiB/s wr, 19 op/s 2023-09-10T09:53:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:22 smithi082 ceph-mon[98694]: pgmap v20443: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 307 KiB/s wr, 19 op/s 2023-09-10T09:53:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:22 smithi098 ceph-mon[103124]: pgmap v20443: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 307 KiB/s wr, 19 op/s 2023-09-10T09:53:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38034.641837 secs 2023-09-10T09:53:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38034.641837 secs 2023-09-10T09:53:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38034.641837 secs 2023-09-10T09:53:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38035.888134 secs 2023-09-10T09:53:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:24 smithi170 ceph-mon[102832]: pgmap v20444: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 165 KiB/s wr, 12 op/s 2023-09-10T09:53:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38036.240209 secs 2023-09-10T09:53:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38035.888134 secs 2023-09-10T09:53:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:24 smithi082 ceph-mon[98694]: pgmap v20444: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 165 KiB/s wr, 12 op/s 2023-09-10T09:53:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38036.240209 secs 2023-09-10T09:53:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38035.888134 secs 2023-09-10T09:53:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:24 smithi098 ceph-mon[103124]: pgmap v20444: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 165 KiB/s wr, 12 op/s 2023-09-10T09:53:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38036.240209 secs 2023-09-10T09:53:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:26 smithi170 ceph-mon[102832]: pgmap v20445: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 166 KiB/s wr, 19 op/s 2023-09-10T09:53:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:26 smithi082 ceph-mon[98694]: pgmap v20445: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 166 KiB/s wr, 19 op/s 2023-09-10T09:53:27.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:26 smithi098 ceph-mon[103124]: pgmap v20445: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 166 KiB/s wr, 19 op/s 2023-09-10T09:53:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:27 smithi170 ceph-mon[102832]: pgmap v20446: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 140 KiB/s wr, 12 op/s 2023-09-10T09:53:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:27 smithi082 ceph-mon[98694]: pgmap v20446: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 140 KiB/s wr, 12 op/s 2023-09-10T09:53:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:27 smithi098 ceph-mon[103124]: pgmap v20446: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 140 KiB/s wr, 12 op/s 2023-09-10T09:53:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38039.641935 secs 2023-09-10T09:53:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38039.641935 secs 2023-09-10T09:53:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38039.641935 secs 2023-09-10T09:53:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38040.888217 secs 2023-09-10T09:53:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:29 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:53:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:29 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38041.240272 secs 2023-09-10T09:53:30.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:29 smithi170 ceph-mon[102832]: pgmap v20447: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 140 KiB/s wr, 12 op/s 2023-09-10T09:53:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38040.888217 secs 2023-09-10T09:53:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:29 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:53:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:29 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38041.240272 secs 2023-09-10T09:53:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:29 smithi082 ceph-mon[98694]: pgmap v20447: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 140 KiB/s wr, 12 op/s 2023-09-10T09:53:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38040.888217 secs 2023-09-10T09:53:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:29 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:53:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:29 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:30.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38041.240272 secs 2023-09-10T09:53:30.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:29 smithi098 ceph-mon[103124]: pgmap v20447: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 140 KiB/s wr, 12 op/s 2023-09-10T09:53:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:53:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:32 smithi170 ceph-mon[102832]: pgmap v20448: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 304 KiB/s wr, 18 op/s 2023-09-10T09:53:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:53:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:32 smithi082 ceph-mon[98694]: pgmap v20448: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 304 KiB/s wr, 18 op/s 2023-09-10T09:53:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:53:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:32 smithi098 ceph-mon[103124]: pgmap v20448: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 304 KiB/s wr, 18 op/s 2023-09-10T09:53:34.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38045.888291 secs 2023-09-10T09:53:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:34 smithi082 ceph-mon[98694]: pgmap v20449: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T09:53:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38046.240337 secs 2023-09-10T09:53:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38045.888291 secs 2023-09-10T09:53:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:34 smithi098 ceph-mon[103124]: pgmap v20449: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T09:53:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38046.240337 secs 2023-09-10T09:53:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38045.888291 secs 2023-09-10T09:53:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:34 smithi170 ceph-mon[102832]: pgmap v20449: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 165 KiB/s wr, 11 op/s 2023-09-10T09:53:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38046.240337 secs 2023-09-10T09:53:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:53:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:53:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:53:36.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:36 smithi082 ceph-mon[98694]: pgmap v20450: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 345 KiB/s wr, 17 op/s 2023-09-10T09:53:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:53:36.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:53:36.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:36 smithi098 ceph-mon[103124]: pgmap v20450: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 345 KiB/s wr, 17 op/s 2023-09-10T09:53:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:53:37.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:53:37.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:36 smithi170 ceph-mon[102832]: pgmap v20450: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 345 KiB/s wr, 17 op/s 2023-09-10T09:53:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:53:37.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:53:37.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:38.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:38 smithi082 ceph-mon[98694]: pgmap v20451: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 345 KiB/s wr, 11 op/s 2023-09-10T09:53:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:53:38.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:38 smithi098 ceph-mon[103124]: pgmap v20451: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 345 KiB/s wr, 11 op/s 2023-09-10T09:53:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:53:39.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:38 smithi170 ceph-mon[102832]: pgmap v20451: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 345 KiB/s wr, 11 op/s 2023-09-10T09:53:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:53:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38050.888349 secs 2023-09-10T09:53:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38051.240402 secs 2023-09-10T09:53:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38050.888349 secs 2023-09-10T09:53:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38051.240402 secs 2023-09-10T09:53:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38050.888349 secs 2023-09-10T09:53:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38051.240402 secs 2023-09-10T09:53:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:40 smithi098 ceph-mon[103124]: pgmap v20452: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 345 KiB/s wr, 11 op/s 2023-09-10T09:53:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:53:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:40 smithi170 ceph-mon[102832]: pgmap v20452: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 345 KiB/s wr, 11 op/s 2023-09-10T09:53:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:53:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:40 smithi082 ceph-mon[98694]: pgmap v20452: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 345 KiB/s wr, 11 op/s 2023-09-10T09:53:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:53:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:43.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:42 smithi098 ceph-mon[103124]: pgmap v20453: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 512 KiB/s wr, 17 op/s 2023-09-10T09:53:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:42 smithi170 ceph-mon[102832]: pgmap v20453: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 512 KiB/s wr, 17 op/s 2023-09-10T09:53:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:42 smithi082 ceph-mon[98694]: pgmap v20453: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 512 KiB/s wr, 17 op/s 2023-09-10T09:53:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38055.888406 secs 2023-09-10T09:53:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:44 smithi098 ceph-mon[103124]: pgmap v20454: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 348 KiB/s wr, 11 op/s 2023-09-10T09:53:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:53:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38056.240473 secs 2023-09-10T09:53:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38055.888406 secs 2023-09-10T09:53:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:44 smithi170 ceph-mon[102832]: pgmap v20454: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 348 KiB/s wr, 11 op/s 2023-09-10T09:53:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:53:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38056.240473 secs 2023-09-10T09:53:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38055.888406 secs 2023-09-10T09:53:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:44 smithi082 ceph-mon[98694]: pgmap v20454: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 348 KiB/s wr, 11 op/s 2023-09-10T09:53:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:53:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38056.240473 secs 2023-09-10T09:53:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:46 smithi098 ceph-mon[103124]: pgmap v20455: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 876 KiB/s wr, 39 op/s 2023-09-10T09:53:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:53:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:46 smithi170 ceph-mon[102832]: pgmap v20455: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 876 KiB/s wr, 39 op/s 2023-09-10T09:53:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:53:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:46 smithi082 ceph-mon[98694]: pgmap v20455: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 876 KiB/s wr, 39 op/s 2023-09-10T09:53:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:53:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:48 smithi098 ceph-mon[103124]: pgmap v20456: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 695 KiB/s wr, 33 op/s 2023-09-10T09:53:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:48 smithi170 ceph-mon[102832]: pgmap v20456: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 695 KiB/s wr, 33 op/s 2023-09-10T09:53:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:48 smithi082 ceph-mon[98694]: pgmap v20456: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 695 KiB/s wr, 33 op/s 2023-09-10T09:53:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38060.888465 secs 2023-09-10T09:53:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38061.240533 secs 2023-09-10T09:53:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38060.888465 secs 2023-09-10T09:53:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38061.240533 secs 2023-09-10T09:53:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38060.888465 secs 2023-09-10T09:53:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38061.240533 secs 2023-09-10T09:53:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:50 smithi098 ceph-mon[103124]: pgmap v20457: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 695 KiB/s wr, 33 op/s 2023-09-10T09:53:51.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:53:51.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:50 smithi170 ceph-mon[102832]: pgmap v20457: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 695 KiB/s wr, 33 op/s 2023-09-10T09:53:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:53:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:50 smithi082 ceph-mon[98694]: pgmap v20457: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 695 KiB/s wr, 33 op/s 2023-09-10T09:53:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:53:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:52 smithi098 ceph-mon[103124]: pgmap v20458: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 41 op/s 2023-09-10T09:53:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:53:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:52 smithi170 ceph-mon[102832]: pgmap v20458: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 41 op/s 2023-09-10T09:53:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:53:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:52 smithi082 ceph-mon[98694]: pgmap v20458: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 41 op/s 2023-09-10T09:53:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:53:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38065.888492 secs 2023-09-10T09:53:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:54 smithi170 ceph-mon[102832]: pgmap v20459: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 950 KiB/s wr, 35 op/s 2023-09-10T09:53:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:53:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:55.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38066.240602 secs 2023-09-10T09:53:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38065.888492 secs 2023-09-10T09:53:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:54 smithi082 ceph-mon[98694]: pgmap v20459: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 950 KiB/s wr, 35 op/s 2023-09-10T09:53:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:53:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38066.240602 secs 2023-09-10T09:53:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38065.888492 secs 2023-09-10T09:53:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:54 smithi098 ceph-mon[103124]: pgmap v20459: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 950 KiB/s wr, 35 op/s 2023-09-10T09:53:55.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:53:55.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:55.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38066.240602 secs 2023-09-10T09:53:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:56 smithi170 ceph-mon[102832]: pgmap v20460: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 42 op/s 2023-09-10T09:53:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:53:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:56 smithi082 ceph-mon[98694]: pgmap v20460: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 42 op/s 2023-09-10T09:53:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:53:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:56 smithi098 ceph-mon[103124]: pgmap v20460: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 42 op/s 2023-09-10T09:53:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:53:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:53:59.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:58 smithi170 ceph-mon[102832]: pgmap v20461: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 547 KiB/s wr, 15 op/s 2023-09-10T09:53:59.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38069.642355 secs 2023-09-10T09:53:59.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:58 smithi082 ceph-mon[98694]: pgmap v20461: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 547 KiB/s wr, 15 op/s 2023-09-10T09:53:59.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38069.642355 secs 2023-09-10T09:53:59.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:58 smithi098 ceph-mon[103124]: pgmap v20461: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 547 KiB/s wr, 15 op/s 2023-09-10T09:53:59.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38069.642355 secs 2023-09-10T09:54:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38070.888584 secs 2023-09-10T09:54:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:53:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38071.240669 secs 2023-09-10T09:54:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38070.888584 secs 2023-09-10T09:54:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:53:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38071.240669 secs 2023-09-10T09:54:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38070.888584 secs 2023-09-10T09:54:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:53:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38071.240669 secs 2023-09-10T09:54:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:00 smithi170 ceph-mon[102832]: pgmap v20462: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 547 KiB/s wr, 15 op/s 2023-09-10T09:54:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:54:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:54:01.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:00 smithi082 ceph-mon[98694]: pgmap v20462: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 547 KiB/s wr, 15 op/s 2023-09-10T09:54:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:54:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:54:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:00 smithi098 ceph-mon[103124]: pgmap v20462: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 547 KiB/s wr, 15 op/s 2023-09-10T09:54:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:54:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:54:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:02 smithi170 ceph-mon[102832]: pgmap v20463: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 820 KiB/s wr, 22 op/s 2023-09-10T09:54:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:02 smithi082 ceph-mon[98694]: pgmap v20463: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 820 KiB/s wr, 22 op/s 2023-09-10T09:54:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:02 smithi098 ceph-mon[103124]: pgmap v20463: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 820 KiB/s wr, 22 op/s 2023-09-10T09:54:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38074.642417 secs 2023-09-10T09:54:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:54:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38074.642417 secs 2023-09-10T09:54:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:54:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38074.642417 secs 2023-09-10T09:54:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:54:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38075.888667 secs 2023-09-10T09:54:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:04 smithi170 ceph-mon[102832]: pgmap v20464: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 398 KiB/s wr, 14 op/s 2023-09-10T09:54:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:54:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:54:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:54:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:54:05.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38076.240733 secs 2023-09-10T09:54:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38075.888667 secs 2023-09-10T09:54:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:04 smithi082 ceph-mon[98694]: pgmap v20464: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 398 KiB/s wr, 14 op/s 2023-09-10T09:54:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:54:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:54:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:54:05.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:54:05.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38076.240733 secs 2023-09-10T09:54:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38075.888667 secs 2023-09-10T09:54:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:04 smithi098 ceph-mon[103124]: pgmap v20464: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 398 KiB/s wr, 14 op/s 2023-09-10T09:54:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:54:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:54:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:54:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:54:05.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38076.240733 secs 2023-09-10T09:54:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:06 smithi170 ceph-mon[102832]: pgmap v20465: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 720 KiB/s wr, 35 op/s 2023-09-10T09:54:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:54:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:06 smithi082 ceph-mon[98694]: pgmap v20465: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 720 KiB/s wr, 35 op/s 2023-09-10T09:54:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:54:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:06 smithi098 ceph-mon[103124]: pgmap v20465: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 720 KiB/s wr, 35 op/s 2023-09-10T09:54:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:54:07.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:08 smithi170 ceph-mon[102832]: pgmap v20466: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 595 KiB/s wr, 28 op/s 2023-09-10T09:54:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38079.642474 secs 2023-09-10T09:54:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:08 smithi082 ceph-mon[98694]: pgmap v20466: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 595 KiB/s wr, 28 op/s 2023-09-10T09:54:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38079.642474 secs 2023-09-10T09:54:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:08 smithi098 ceph-mon[103124]: pgmap v20466: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 595 KiB/s wr, 28 op/s 2023-09-10T09:54:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38079.642474 secs 2023-09-10T09:54:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38080.888725 secs 2023-09-10T09:54:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:09 smithi170 ceph-mon[102832]: osdmap e1347: 12 total, 12 up, 12 in 2023-09-10T09:54:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38081.240798 secs 2023-09-10T09:54:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38080.888725 secs 2023-09-10T09:54:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:09 smithi082 ceph-mon[98694]: osdmap e1347: 12 total, 12 up, 12 in 2023-09-10T09:54:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38081.240798 secs 2023-09-10T09:54:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38080.888725 secs 2023-09-10T09:54:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:09 smithi098 ceph-mon[103124]: osdmap e1347: 12 total, 12 up, 12 in 2023-09-10T09:54:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38081.240798 secs 2023-09-10T09:54:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:10 smithi170 ceph-mon[102832]: pgmap v20468: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 713 KiB/s wr, 34 op/s 2023-09-10T09:54:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:10 smithi082 ceph-mon[98694]: pgmap v20468: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 713 KiB/s wr, 34 op/s 2023-09-10T09:54:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:10 smithi098 ceph-mon[103124]: pgmap v20468: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 713 KiB/s wr, 34 op/s 2023-09-10T09:54:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:12 smithi170 ceph-mon[102832]: pgmap v20469: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 581 KiB/s wr, 33 op/s 2023-09-10T09:54:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:12 smithi082 ceph-mon[98694]: pgmap v20469: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 581 KiB/s wr, 33 op/s 2023-09-10T09:54:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:12 smithi098 ceph-mon[103124]: pgmap v20469: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 581 KiB/s wr, 33 op/s 2023-09-10T09:54:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38084.642547 secs 2023-09-10T09:54:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38084.642547 secs 2023-09-10T09:54:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38084.642547 secs 2023-09-10T09:54:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38085.888784 secs 2023-09-10T09:54:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:14 smithi170 ceph-mon[102832]: pgmap v20470: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 581 KiB/s wr, 33 op/s 2023-09-10T09:54:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:14 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:54:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:14 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38086.240866 secs 2023-09-10T09:54:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:14 smithi170 ceph-mon[102832]: osdmap e1348: 12 total, 12 up, 12 in 2023-09-10T09:54:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38085.888784 secs 2023-09-10T09:54:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:14 smithi082 ceph-mon[98694]: pgmap v20470: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 581 KiB/s wr, 33 op/s 2023-09-10T09:54:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:14 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:54:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:14 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38086.240866 secs 2023-09-10T09:54:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:14 smithi082 ceph-mon[98694]: osdmap e1348: 12 total, 12 up, 12 in 2023-09-10T09:54:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38085.888784 secs 2023-09-10T09:54:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:14 smithi098 ceph-mon[103124]: pgmap v20470: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 581 KiB/s wr, 33 op/s 2023-09-10T09:54:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:14 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:54:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:14 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38086.240866 secs 2023-09-10T09:54:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:14 smithi098 ceph-mon[103124]: osdmap e1348: 12 total, 12 up, 12 in 2023-09-10T09:54:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:16 smithi170 ceph-mon[102832]: pgmap v20472: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 245 KiB/s wr, 18 op/s 2023-09-10T09:54:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:54:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:16 smithi082 ceph-mon[98694]: pgmap v20472: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 245 KiB/s wr, 18 op/s 2023-09-10T09:54:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:54:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:16 smithi098 ceph-mon[103124]: pgmap v20472: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 245 KiB/s wr, 18 op/s 2023-09-10T09:54:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:54:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:18 smithi170 ceph-mon[102832]: pgmap v20473: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 221 KiB/s wr, 16 op/s 2023-09-10T09:54:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38089.642648 secs 2023-09-10T09:54:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:18 smithi082 ceph-mon[98694]: pgmap v20473: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 221 KiB/s wr, 16 op/s 2023-09-10T09:54:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38089.642648 secs 2023-09-10T09:54:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:18 smithi098 ceph-mon[103124]: pgmap v20473: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 221 KiB/s wr, 16 op/s 2023-09-10T09:54:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38089.642648 secs 2023-09-10T09:54:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38090.888842 secs 2023-09-10T09:54:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38091.240932 secs 2023-09-10T09:54:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38090.888842 secs 2023-09-10T09:54:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38091.240932 secs 2023-09-10T09:54:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38090.888842 secs 2023-09-10T09:54:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38091.240932 secs 2023-09-10T09:54:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:20 smithi170 ceph-mon[102832]: pgmap v20474: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 14 op/s 2023-09-10T09:54:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:20 smithi082 ceph-mon[98694]: pgmap v20474: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 14 op/s 2023-09-10T09:54:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:20 smithi098 ceph-mon[103124]: pgmap v20474: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 14 op/s 2023-09-10T09:54:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:22 smithi170 ceph-mon[102832]: pgmap v20475: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 14 op/s 2023-09-10T09:54:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:22 smithi082 ceph-mon[98694]: pgmap v20475: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 14 op/s 2023-09-10T09:54:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:22 smithi098 ceph-mon[103124]: pgmap v20475: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 14 op/s 2023-09-10T09:54:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38094.642709 secs 2023-09-10T09:54:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38094.642709 secs 2023-09-10T09:54:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38094.642709 secs 2023-09-10T09:54:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38095.888900 secs 2023-09-10T09:54:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:24 smithi170 ceph-mon[102832]: pgmap v20476: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 14 op/s 2023-09-10T09:54:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38096.240999 secs 2023-09-10T09:54:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38095.888900 secs 2023-09-10T09:54:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:24 smithi082 ceph-mon[98694]: pgmap v20476: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 14 op/s 2023-09-10T09:54:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38096.240999 secs 2023-09-10T09:54:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38095.888900 secs 2023-09-10T09:54:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:24 smithi098 ceph-mon[103124]: pgmap v20476: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 162 KiB/s wr, 14 op/s 2023-09-10T09:54:25.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38096.240999 secs 2023-09-10T09:54:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:26 smithi170 ceph-mon[102832]: pgmap v20477: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 294 KiB/s wr, 19 op/s 2023-09-10T09:54:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:26 smithi082 ceph-mon[98694]: pgmap v20477: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 294 KiB/s wr, 19 op/s 2023-09-10T09:54:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:26 smithi098 ceph-mon[103124]: pgmap v20477: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 294 KiB/s wr, 19 op/s 2023-09-10T09:54:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:28 smithi170 ceph-mon[102832]: pgmap v20478: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 274 KiB/s wr, 11 op/s 2023-09-10T09:54:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38099.642813 secs 2023-09-10T09:54:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:28 smithi082 ceph-mon[98694]: pgmap v20478: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 274 KiB/s wr, 11 op/s 2023-09-10T09:54:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38099.642813 secs 2023-09-10T09:54:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:28 smithi098 ceph-mon[103124]: pgmap v20478: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 274 KiB/s wr, 11 op/s 2023-09-10T09:54:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38099.642813 secs 2023-09-10T09:54:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38100.888959 secs 2023-09-10T09:54:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38101.241062 secs 2023-09-10T09:54:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38100.888959 secs 2023-09-10T09:54:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38101.241062 secs 2023-09-10T09:54:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38100.888959 secs 2023-09-10T09:54:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38101.241062 secs 2023-09-10T09:54:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:30 smithi170 ceph-mon[102832]: pgmap v20479: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 274 KiB/s wr, 11 op/s 2023-09-10T09:54:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:30 smithi082 ceph-mon[98694]: pgmap v20479: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 274 KiB/s wr, 11 op/s 2023-09-10T09:54:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:30 smithi098 ceph-mon[103124]: pgmap v20479: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 274 KiB/s wr, 11 op/s 2023-09-10T09:54:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:32 smithi170 ceph-mon[102832]: pgmap v20480: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 275 KiB/s wr, 17 op/s 2023-09-10T09:54:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:32 smithi082 ceph-mon[98694]: pgmap v20480: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 275 KiB/s wr, 17 op/s 2023-09-10T09:54:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:32 smithi098 ceph-mon[103124]: pgmap v20480: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 275 KiB/s wr, 17 op/s 2023-09-10T09:54:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38105.889045 secs 2023-09-10T09:54:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:34 smithi170 ceph-mon[102832]: pgmap v20481: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 11 op/s 2023-09-10T09:54:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:54:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:35.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38106.241127 secs 2023-09-10T09:54:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38105.889045 secs 2023-09-10T09:54:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:34 smithi082 ceph-mon[98694]: pgmap v20481: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 11 op/s 2023-09-10T09:54:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:54:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38106.241127 secs 2023-09-10T09:54:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38105.889045 secs 2023-09-10T09:54:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:34 smithi098 ceph-mon[103124]: pgmap v20481: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 11 op/s 2023-09-10T09:54:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:54:35.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:35.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38106.241127 secs 2023-09-10T09:54:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:54:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:54:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:54:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:36 smithi170 ceph-mon[102832]: pgmap v20482: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 443 KiB/s wr, 17 op/s 2023-09-10T09:54:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:54:37.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:54:37.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:36 smithi082 ceph-mon[98694]: pgmap v20482: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 443 KiB/s wr, 17 op/s 2023-09-10T09:54:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:54:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:54:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:36 smithi098 ceph-mon[103124]: pgmap v20482: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 443 KiB/s wr, 17 op/s 2023-09-10T09:54:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:54:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:54:37.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:38 smithi170 ceph-mon[102832]: pgmap v20483: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 302 KiB/s wr, 11 op/s 2023-09-10T09:54:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:38 smithi082 ceph-mon[98694]: pgmap v20483: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 302 KiB/s wr, 11 op/s 2023-09-10T09:54:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:38 smithi098 ceph-mon[103124]: pgmap v20483: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 302 KiB/s wr, 11 op/s 2023-09-10T09:54:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38110.889104 secs 2023-09-10T09:54:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38111.241193 secs 2023-09-10T09:54:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38110.889104 secs 2023-09-10T09:54:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38111.241193 secs 2023-09-10T09:54:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38110.889104 secs 2023-09-10T09:54:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38111.241193 secs 2023-09-10T09:54:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:40 smithi170 ceph-mon[102832]: pgmap v20484: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 302 KiB/s wr, 11 op/s 2023-09-10T09:54:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:54:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:40 smithi082 ceph-mon[98694]: pgmap v20484: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 302 KiB/s wr, 11 op/s 2023-09-10T09:54:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:54:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:40 smithi098 ceph-mon[103124]: pgmap v20484: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 302 KiB/s wr, 11 op/s 2023-09-10T09:54:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:54:41.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:42 smithi170 ceph-mon[102832]: pgmap v20485: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 469 KiB/s wr, 17 op/s 2023-09-10T09:54:43.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:42 smithi082 ceph-mon[98694]: pgmap v20485: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 469 KiB/s wr, 17 op/s 2023-09-10T09:54:43.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:42 smithi098 ceph-mon[103124]: pgmap v20485: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 469 KiB/s wr, 17 op/s 2023-09-10T09:54:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38115.889164 secs 2023-09-10T09:54:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:44 smithi170 ceph-mon[102832]: pgmap v20486: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 469 KiB/s wr, 12 op/s 2023-09-10T09:54:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38116.241260 secs 2023-09-10T09:54:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38115.889164 secs 2023-09-10T09:54:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:44 smithi082 ceph-mon[98694]: pgmap v20486: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 469 KiB/s wr, 12 op/s 2023-09-10T09:54:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38116.241260 secs 2023-09-10T09:54:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38115.889164 secs 2023-09-10T09:54:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:44 smithi098 ceph-mon[103124]: pgmap v20486: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 469 KiB/s wr, 12 op/s 2023-09-10T09:54:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38116.241260 secs 2023-09-10T09:54:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:45 smithi170 ceph-mon[102832]: pgmap v20487: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 655 KiB/s wr, 19 op/s 2023-09-10T09:54:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:54:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:45 smithi082 ceph-mon[98694]: pgmap v20487: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 655 KiB/s wr, 19 op/s 2023-09-10T09:54:46.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:54:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:45 smithi098 ceph-mon[103124]: pgmap v20487: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 655 KiB/s wr, 19 op/s 2023-09-10T09:54:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:54:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:49.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:48 smithi098 ceph-mon[103124]: pgmap v20488: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 352 KiB/s wr, 13 op/s 2023-09-10T09:54:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:48 smithi170 ceph-mon[102832]: pgmap v20488: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 352 KiB/s wr, 13 op/s 2023-09-10T09:54:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:48 smithi082 ceph-mon[98694]: pgmap v20488: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 352 KiB/s wr, 13 op/s 2023-09-10T09:54:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38120.889220 secs 2023-09-10T09:54:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38121.241323 secs 2023-09-10T09:54:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38120.889220 secs 2023-09-10T09:54:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38121.241323 secs 2023-09-10T09:54:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38120.889220 secs 2023-09-10T09:54:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38121.241323 secs 2023-09-10T09:54:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:50 smithi098 ceph-mon[103124]: pgmap v20489: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 352 KiB/s wr, 13 op/s 2023-09-10T09:54:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:54:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:50 smithi170 ceph-mon[102832]: pgmap v20489: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 352 KiB/s wr, 13 op/s 2023-09-10T09:54:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:54:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:50 smithi082 ceph-mon[98694]: pgmap v20489: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 352 KiB/s wr, 13 op/s 2023-09-10T09:54:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:54:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:52 smithi098 ceph-mon[103124]: pgmap v20490: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 817 KiB/s wr, 20 op/s 2023-09-10T09:54:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:54:53.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:52 smithi170 ceph-mon[102832]: pgmap v20490: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 817 KiB/s wr, 20 op/s 2023-09-10T09:54:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:54:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:52 smithi082 ceph-mon[98694]: pgmap v20490: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 817 KiB/s wr, 20 op/s 2023-09-10T09:54:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:54:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38124.643158 secs 2023-09-10T09:54:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38124.643158 secs 2023-09-10T09:54:54.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38124.643158 secs 2023-09-10T09:54:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38125.889277 secs 2023-09-10T09:54:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:54 smithi098 ceph-mon[103124]: pgmap v20491: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 650 KiB/s wr, 14 op/s 2023-09-10T09:54:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:54:55.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:55.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38126.241388 secs 2023-09-10T09:54:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38125.889277 secs 2023-09-10T09:54:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:54 smithi170 ceph-mon[102832]: pgmap v20491: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 650 KiB/s wr, 14 op/s 2023-09-10T09:54:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:54:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:55.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38126.241388 secs 2023-09-10T09:54:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38125.889277 secs 2023-09-10T09:54:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:54 smithi082 ceph-mon[98694]: pgmap v20491: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 650 KiB/s wr, 14 op/s 2023-09-10T09:54:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:54:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38126.241388 secs 2023-09-10T09:54:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:55 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:54:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:55 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:54:56.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:55 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:56.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:55 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:54:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:55 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:54:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:55 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:55 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:54:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:55 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:54:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:55 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:56 smithi170 ceph-mon[102832]: pgmap v20492: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 683 KiB/s wr, 20 op/s 2023-09-10T09:54:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:56 smithi082 ceph-mon[98694]: pgmap v20492: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 683 KiB/s wr, 20 op/s 2023-09-10T09:54:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:56 smithi098 ceph-mon[103124]: pgmap v20492: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 683 KiB/s wr, 20 op/s 2023-09-10T09:54:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:58 smithi170 ceph-mon[102832]: pgmap v20493: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 497 KiB/s wr, 14 op/s 2023-09-10T09:54:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:54:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38129.643240 secs 2023-09-10T09:54:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:58 smithi082 ceph-mon[98694]: pgmap v20493: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 497 KiB/s wr, 14 op/s 2023-09-10T09:54:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:54:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38129.643240 secs 2023-09-10T09:54:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:58 smithi098 ceph-mon[103124]: pgmap v20493: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 497 KiB/s wr, 14 op/s 2023-09-10T09:54:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:54:59.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:54:59.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38129.643240 secs 2023-09-10T09:55:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38130.889336 secs 2023-09-10T09:55:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:54:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38131.241452 secs 2023-09-10T09:55:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38130.889336 secs 2023-09-10T09:55:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:54:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38131.241452 secs 2023-09-10T09:55:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38130.889336 secs 2023-09-10T09:55:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:54:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38131.241452 secs 2023-09-10T09:55:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:00 smithi170 ceph-mon[102832]: pgmap v20494: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 497 KiB/s wr, 14 op/s 2023-09-10T09:55:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:55:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:00 smithi082 ceph-mon[98694]: pgmap v20494: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 497 KiB/s wr, 14 op/s 2023-09-10T09:55:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:55:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:00 smithi098 ceph-mon[103124]: pgmap v20494: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 497 KiB/s wr, 14 op/s 2023-09-10T09:55:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:55:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:02 smithi170 ceph-mon[102832]: pgmap v20495: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1012 KiB/s wr, 21 op/s 2023-09-10T09:55:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:02 smithi082 ceph-mon[98694]: pgmap v20495: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1012 KiB/s wr, 21 op/s 2023-09-10T09:55:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:02 smithi098 ceph-mon[103124]: pgmap v20495: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1012 KiB/s wr, 21 op/s 2023-09-10T09:55:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38134.643300 secs 2023-09-10T09:55:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38134.643300 secs 2023-09-10T09:55:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38134.643300 secs 2023-09-10T09:55:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38135.889394 secs 2023-09-10T09:55:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:04 smithi170 ceph-mon[102832]: pgmap v20496: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 547 KiB/s wr, 13 op/s 2023-09-10T09:55:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:55:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38136.241488 secs 2023-09-10T09:55:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:55:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38135.889394 secs 2023-09-10T09:55:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:04 smithi082 ceph-mon[98694]: pgmap v20496: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 547 KiB/s wr, 13 op/s 2023-09-10T09:55:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:55:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38136.241488 secs 2023-09-10T09:55:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:55:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38135.889394 secs 2023-09-10T09:55:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:04 smithi098 ceph-mon[103124]: pgmap v20496: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 547 KiB/s wr, 13 op/s 2023-09-10T09:55:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:55:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38136.241488 secs 2023-09-10T09:55:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:55:07.158 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:06 smithi082 ceph-mon[98694]: pgmap v20497: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 732 KiB/s wr, 35 op/s 2023-09-10T09:55:07.158 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:55:07.159 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:06 smithi170 ceph-mon[102832]: pgmap v20497: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 732 KiB/s wr, 35 op/s 2023-09-10T09:55:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:55:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:07.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:06 smithi098 ceph-mon[103124]: pgmap v20497: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 732 KiB/s wr, 35 op/s 2023-09-10T09:55:07.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:55:07.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:55:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:55:08.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:08 smithi170 ceph-mon[102832]: pgmap v20498: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 699 KiB/s wr, 28 op/s 2023-09-10T09:55:08.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38139.643414 secs 2023-09-10T09:55:08.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:55:08.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:55:08.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:55:08.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:55:08.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:55:08.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:55:08.931 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:55:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:55:08.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:55:08.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:08 smithi082 ceph-mon[98694]: pgmap v20498: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 699 KiB/s wr, 28 op/s 2023-09-10T09:55:08.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38139.643414 secs 2023-09-10T09:55:08.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:55:08.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:55:08.961 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:55:08.962 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:55:08.962 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:55:08.963 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:55:08.963 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:55:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:55:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:55:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:08 smithi098 ceph-mon[103124]: pgmap v20498: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 699 KiB/s wr, 28 op/s 2023-09-10T09:55:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38139.643414 secs 2023-09-10T09:55:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:55:09.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:55:09.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:55:09.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:55:09.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:55:09.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:55:09.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:55:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:09 smithi170 ceph-mon[102832]: osdmap e1349: 12 total, 12 up, 12 in 2023-09-10T09:55:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38140.889454 secs 2023-09-10T09:55:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38141.241555 secs 2023-09-10T09:55:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:09 smithi082 ceph-mon[98694]: osdmap e1349: 12 total, 12 up, 12 in 2023-09-10T09:55:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38140.889454 secs 2023-09-10T09:55:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38141.241555 secs 2023-09-10T09:55:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:09 smithi098 ceph-mon[103124]: osdmap e1349: 12 total, 12 up, 12 in 2023-09-10T09:55:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38140.889454 secs 2023-09-10T09:55:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38141.241555 secs 2023-09-10T09:55:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:10 smithi170 ceph-mon[102832]: pgmap v20500: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 839 KiB/s wr, 34 op/s 2023-09-10T09:55:10.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:10 smithi082 ceph-mon[98694]: pgmap v20500: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 839 KiB/s wr, 34 op/s 2023-09-10T09:55:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:10 smithi098 ceph-mon[103124]: pgmap v20500: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 839 KiB/s wr, 34 op/s 2023-09-10T09:55:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:12 smithi098 ceph-mon[103124]: pgmap v20501: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 358 KiB/s wr, 33 op/s 2023-09-10T09:55:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:12 smithi170 ceph-mon[102832]: pgmap v20501: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 358 KiB/s wr, 33 op/s 2023-09-10T09:55:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:12 smithi082 ceph-mon[98694]: pgmap v20501: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 358 KiB/s wr, 33 op/s 2023-09-10T09:55:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38144.643494 secs 2023-09-10T09:55:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38144.643494 secs 2023-09-10T09:55:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38144.643494 secs 2023-09-10T09:55:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38145.889492 secs 2023-09-10T09:55:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:14 smithi098 ceph-mon[103124]: pgmap v20502: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 358 KiB/s wr, 33 op/s 2023-09-10T09:55:15.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38146.241618 secs 2023-09-10T09:55:15.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:14 smithi098 ceph-mon[103124]: osdmap e1350: 12 total, 12 up, 12 in 2023-09-10T09:55:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38145.889492 secs 2023-09-10T09:55:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:14 smithi170 ceph-mon[102832]: pgmap v20502: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 358 KiB/s wr, 33 op/s 2023-09-10T09:55:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38146.241618 secs 2023-09-10T09:55:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:14 smithi170 ceph-mon[102832]: osdmap e1350: 12 total, 12 up, 12 in 2023-09-10T09:55:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38145.889492 secs 2023-09-10T09:55:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:14 smithi082 ceph-mon[98694]: pgmap v20502: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 358 KiB/s wr, 33 op/s 2023-09-10T09:55:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38146.241618 secs 2023-09-10T09:55:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:14 smithi082 ceph-mon[98694]: osdmap e1350: 12 total, 12 up, 12 in 2023-09-10T09:55:17.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:16 smithi098 ceph-mon[103124]: pgmap v20504: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 171 KiB/s wr, 18 op/s 2023-09-10T09:55:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:16 smithi170 ceph-mon[102832]: pgmap v20504: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 171 KiB/s wr, 18 op/s 2023-09-10T09:55:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:16 smithi082 ceph-mon[98694]: pgmap v20504: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 171 KiB/s wr, 18 op/s 2023-09-10T09:55:19.010 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:18 smithi098 ceph-mon[103124]: pgmap v20505: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T09:55:19.010 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38149.643555 secs 2023-09-10T09:55:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:18 smithi170 ceph-mon[102832]: pgmap v20505: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T09:55:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38149.643555 secs 2023-09-10T09:55:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:18 smithi082 ceph-mon[98694]: pgmap v20505: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 151 KiB/s wr, 15 op/s 2023-09-10T09:55:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38149.643555 secs 2023-09-10T09:55:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38150.889572 secs 2023-09-10T09:55:20.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38151.241682 secs 2023-09-10T09:55:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38150.889572 secs 2023-09-10T09:55:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38151.241682 secs 2023-09-10T09:55:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38150.889572 secs 2023-09-10T09:55:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38151.241682 secs 2023-09-10T09:55:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:20 smithi098 ceph-mon[103124]: pgmap v20506: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 137 KiB/s wr, 14 op/s 2023-09-10T09:55:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:20 smithi170 ceph-mon[102832]: pgmap v20506: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 137 KiB/s wr, 14 op/s 2023-09-10T09:55:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:20 smithi082 ceph-mon[98694]: pgmap v20506: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 137 KiB/s wr, 14 op/s 2023-09-10T09:55:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:22 smithi098 ceph-mon[103124]: pgmap v20507: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2023-09-10T09:55:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:22 smithi170 ceph-mon[102832]: pgmap v20507: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2023-09-10T09:55:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:22 smithi082 ceph-mon[98694]: pgmap v20507: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2023-09-10T09:55:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38154.643617 secs 2023-09-10T09:55:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38154.643617 secs 2023-09-10T09:55:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38154.643617 secs 2023-09-10T09:55:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38155.889630 secs 2023-09-10T09:55:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:24 smithi098 ceph-mon[103124]: pgmap v20508: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2023-09-10T09:55:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38156.241748 secs 2023-09-10T09:55:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38155.889630 secs 2023-09-10T09:55:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:24 smithi170 ceph-mon[102832]: pgmap v20508: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2023-09-10T09:55:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38156.241748 secs 2023-09-10T09:55:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38155.889630 secs 2023-09-10T09:55:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:24 smithi082 ceph-mon[98694]: pgmap v20508: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2023-09-10T09:55:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38156.241748 secs 2023-09-10T09:55:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:26 smithi098 ceph-mon[103124]: pgmap v20509: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 150 KiB/s wr, 17 op/s 2023-09-10T09:55:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:26 smithi170 ceph-mon[102832]: pgmap v20509: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 150 KiB/s wr, 17 op/s 2023-09-10T09:55:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:26 smithi082 ceph-mon[98694]: pgmap v20509: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 150 KiB/s wr, 17 op/s 2023-09-10T09:55:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:28 smithi170 ceph-mon[102832]: pgmap v20510: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 10 op/s 2023-09-10T09:55:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38159.643692 secs 2023-09-10T09:55:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:28 smithi082 ceph-mon[98694]: pgmap v20510: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 10 op/s 2023-09-10T09:55:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38159.643692 secs 2023-09-10T09:55:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:28 smithi098 ceph-mon[103124]: pgmap v20510: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 10 op/s 2023-09-10T09:55:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38159.643692 secs 2023-09-10T09:55:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38160.889688 secs 2023-09-10T09:55:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38161.241812 secs 2023-09-10T09:55:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38160.889688 secs 2023-09-10T09:55:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38161.241812 secs 2023-09-10T09:55:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38160.889688 secs 2023-09-10T09:55:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38161.241812 secs 2023-09-10T09:55:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:30 smithi170 ceph-mon[102832]: pgmap v20511: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 10 op/s 2023-09-10T09:55:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:30 smithi082 ceph-mon[98694]: pgmap v20511: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 10 op/s 2023-09-10T09:55:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:30 smithi098 ceph-mon[103124]: pgmap v20511: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 10 op/s 2023-09-10T09:55:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:32 smithi170 ceph-mon[102832]: pgmap v20512: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 140 KiB/s wr, 15 op/s 2023-09-10T09:55:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:32 smithi082 ceph-mon[98694]: pgmap v20512: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 140 KiB/s wr, 15 op/s 2023-09-10T09:55:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:32 smithi098 ceph-mon[103124]: pgmap v20512: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 140 KiB/s wr, 15 op/s 2023-09-10T09:55:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38165.889746 secs 2023-09-10T09:55:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:34 smithi170 ceph-mon[102832]: pgmap v20513: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 10 op/s 2023-09-10T09:55:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38166.241877 secs 2023-09-10T09:55:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38165.889746 secs 2023-09-10T09:55:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:34 smithi082 ceph-mon[98694]: pgmap v20513: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 10 op/s 2023-09-10T09:55:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38166.241877 secs 2023-09-10T09:55:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38165.889746 secs 2023-09-10T09:55:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:34 smithi098 ceph-mon[103124]: pgmap v20513: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 10 op/s 2023-09-10T09:55:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38166.241877 secs 2023-09-10T09:55:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:55:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:55:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:55:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:36 smithi170 ceph-mon[102832]: pgmap v20514: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 278 KiB/s wr, 16 op/s 2023-09-10T09:55:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:55:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:55:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:36 smithi082 ceph-mon[98694]: pgmap v20514: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 278 KiB/s wr, 16 op/s 2023-09-10T09:55:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:55:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:55:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:36 smithi098 ceph-mon[103124]: pgmap v20514: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 278 KiB/s wr, 16 op/s 2023-09-10T09:55:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:55:37.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:55:37.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:38 smithi170 ceph-mon[102832]: pgmap v20515: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 10 op/s 2023-09-10T09:55:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:55:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:38 smithi082 ceph-mon[98694]: pgmap v20515: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 10 op/s 2023-09-10T09:55:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:55:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:38 smithi098 ceph-mon[103124]: pgmap v20515: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 10 op/s 2023-09-10T09:55:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:55:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38170.889807 secs 2023-09-10T09:55:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38171.241945 secs 2023-09-10T09:55:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38170.889807 secs 2023-09-10T09:55:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38171.241945 secs 2023-09-10T09:55:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38170.889807 secs 2023-09-10T09:55:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38171.241945 secs 2023-09-10T09:55:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:40 smithi170 ceph-mon[102832]: pgmap v20516: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 10 op/s 2023-09-10T09:55:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:55:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:40 smithi082 ceph-mon[98694]: pgmap v20516: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 10 op/s 2023-09-10T09:55:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:55:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:40 smithi098 ceph-mon[103124]: pgmap v20516: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 10 op/s 2023-09-10T09:55:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:55:41.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:42 smithi170 ceph-mon[102832]: pgmap v20517: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 478 KiB/s wr, 18 op/s 2023-09-10T09:55:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:42 smithi082 ceph-mon[98694]: pgmap v20517: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 478 KiB/s wr, 18 op/s 2023-09-10T09:55:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:42 smithi098 ceph-mon[103124]: pgmap v20517: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 478 KiB/s wr, 18 op/s 2023-09-10T09:55:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38175.889893 secs 2023-09-10T09:55:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:44 smithi170 ceph-mon[102832]: pgmap v20518: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 478 KiB/s wr, 13 op/s 2023-09-10T09:55:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:55:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38176.242010 secs 2023-09-10T09:55:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38175.889893 secs 2023-09-10T09:55:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:44 smithi082 ceph-mon[98694]: pgmap v20518: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 478 KiB/s wr, 13 op/s 2023-09-10T09:55:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:55:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38176.242010 secs 2023-09-10T09:55:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38175.889893 secs 2023-09-10T09:55:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:44 smithi098 ceph-mon[103124]: pgmap v20518: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 478 KiB/s wr, 13 op/s 2023-09-10T09:55:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:55:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38176.242010 secs 2023-09-10T09:55:47.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:46 smithi170 ceph-mon[102832]: pgmap v20519: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 506 KiB/s wr, 20 op/s 2023-09-10T09:55:47.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:55:47.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:46 smithi082 ceph-mon[98694]: pgmap v20519: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 506 KiB/s wr, 20 op/s 2023-09-10T09:55:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:55:47.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:46 smithi098 ceph-mon[103124]: pgmap v20519: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 506 KiB/s wr, 20 op/s 2023-09-10T09:55:47.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:55:47.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:48 smithi170 ceph-mon[102832]: pgmap v20520: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 368 KiB/s wr, 14 op/s 2023-09-10T09:55:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:48 smithi082 ceph-mon[98694]: pgmap v20520: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 368 KiB/s wr, 14 op/s 2023-09-10T09:55:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:48 smithi098 ceph-mon[103124]: pgmap v20520: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 368 KiB/s wr, 14 op/s 2023-09-10T09:55:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38180.889957 secs 2023-09-10T09:55:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38181.242074 secs 2023-09-10T09:55:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38180.889957 secs 2023-09-10T09:55:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38181.242074 secs 2023-09-10T09:55:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38180.889957 secs 2023-09-10T09:55:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38181.242074 secs 2023-09-10T09:55:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:50 smithi170 ceph-mon[102832]: pgmap v20521: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 368 KiB/s wr, 14 op/s 2023-09-10T09:55:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:50 smithi082 ceph-mon[98694]: pgmap v20521: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 368 KiB/s wr, 14 op/s 2023-09-10T09:55:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:50 smithi098 ceph-mon[103124]: pgmap v20521: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 368 KiB/s wr, 14 op/s 2023-09-10T09:55:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:52 smithi170 ceph-mon[102832]: pgmap v20522: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 839 KiB/s wr, 21 op/s 2023-09-10T09:55:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:52 smithi082 ceph-mon[98694]: pgmap v20522: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 839 KiB/s wr, 21 op/s 2023-09-10T09:55:53.259 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:52 smithi098 ceph-mon[103124]: pgmap v20522: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 839 KiB/s wr, 21 op/s 2023-09-10T09:55:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38185.890046 secs 2023-09-10T09:55:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:54 smithi170 ceph-mon[102832]: pgmap v20523: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 499 KiB/s wr, 14 op/s 2023-09-10T09:55:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:55:55.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:55.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38186.242136 secs 2023-09-10T09:55:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38185.890046 secs 2023-09-10T09:55:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:54 smithi082 ceph-mon[98694]: pgmap v20523: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 499 KiB/s wr, 14 op/s 2023-09-10T09:55:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:55:55.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:55.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38186.242136 secs 2023-09-10T09:55:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38185.890046 secs 2023-09-10T09:55:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:54 smithi098 ceph-mon[103124]: pgmap v20523: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 499 KiB/s wr, 14 op/s 2023-09-10T09:55:55.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:55:55.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:55.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38186.242136 secs 2023-09-10T09:55:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:56 smithi170 ceph-mon[102832]: pgmap v20524: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 578 KiB/s wr, 19 op/s 2023-09-10T09:55:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:55:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:56 smithi082 ceph-mon[98694]: pgmap v20524: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 578 KiB/s wr, 19 op/s 2023-09-10T09:55:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:55:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:56 smithi098 ceph-mon[103124]: pgmap v20524: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 578 KiB/s wr, 19 op/s 2023-09-10T09:55:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:55:57.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:58 smithi170 ceph-mon[102832]: pgmap v20525: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 550 KiB/s wr, 13 op/s 2023-09-10T09:55:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:55:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:59.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38189.644203 secs 2023-09-10T09:55:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:58 smithi082 ceph-mon[98694]: pgmap v20525: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 550 KiB/s wr, 13 op/s 2023-09-10T09:55:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:55:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38189.644203 secs 2023-09-10T09:55:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:58 smithi098 ceph-mon[103124]: pgmap v20525: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 550 KiB/s wr, 13 op/s 2023-09-10T09:55:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:55:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:55:59.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38189.644203 secs 2023-09-10T09:56:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38190.890104 secs 2023-09-10T09:56:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:55:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38191.242200 secs 2023-09-10T09:56:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38190.890104 secs 2023-09-10T09:56:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:55:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38191.242200 secs 2023-09-10T09:56:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38190.890104 secs 2023-09-10T09:56:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:55:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38191.242200 secs 2023-09-10T09:56:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:00 smithi170 ceph-mon[102832]: pgmap v20526: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 550 KiB/s wr, 13 op/s 2023-09-10T09:56:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:56:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:56:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:00 smithi082 ceph-mon[98694]: pgmap v20526: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 550 KiB/s wr, 13 op/s 2023-09-10T09:56:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:56:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:56:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:00 smithi098 ceph-mon[103124]: pgmap v20526: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 550 KiB/s wr, 13 op/s 2023-09-10T09:56:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:56:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:56:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:02 smithi170 ceph-mon[102832]: pgmap v20527: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 20 op/s 2023-09-10T09:56:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:02 smithi082 ceph-mon[98694]: pgmap v20527: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 20 op/s 2023-09-10T09:56:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:02 smithi098 ceph-mon[103124]: pgmap v20527: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 20 op/s 2023-09-10T09:56:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38194.644264 secs 2023-09-10T09:56:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38194.644264 secs 2023-09-10T09:56:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38194.644264 secs 2023-09-10T09:56:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38195.890180 secs 2023-09-10T09:56:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:04 smithi170 ceph-mon[102832]: pgmap v20528: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 649 KiB/s wr, 13 op/s 2023-09-10T09:56:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:56:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38196.242265 secs 2023-09-10T09:56:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38195.890180 secs 2023-09-10T09:56:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:04 smithi082 ceph-mon[98694]: pgmap v20528: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 649 KiB/s wr, 13 op/s 2023-09-10T09:56:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:56:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38196.242265 secs 2023-09-10T09:56:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38195.890180 secs 2023-09-10T09:56:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:04 smithi098 ceph-mon[103124]: pgmap v20528: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 649 KiB/s wr, 13 op/s 2023-09-10T09:56:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:56:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38196.242265 secs 2023-09-10T09:56:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:06 smithi170 ceph-mon[102832]: pgmap v20529: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 997 KiB/s wr, 34 op/s 2023-09-10T09:56:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:56:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:06 smithi082 ceph-mon[98694]: pgmap v20529: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 997 KiB/s wr, 34 op/s 2023-09-10T09:56:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:56:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:06 smithi098 ceph-mon[103124]: pgmap v20529: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 997 KiB/s wr, 34 op/s 2023-09-10T09:56:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:56:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:08 smithi170 ceph-mon[102832]: pgmap v20530: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 919 KiB/s wr, 28 op/s 2023-09-10T09:56:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38199.644355 secs 2023-09-10T09:56:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:56:09.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:56:09.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:56:09.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:56:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:08 smithi082 ceph-mon[98694]: pgmap v20530: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 919 KiB/s wr, 28 op/s 2023-09-10T09:56:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38199.644355 secs 2023-09-10T09:56:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:56:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:56:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:56:09.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:56:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:08 smithi098 ceph-mon[103124]: pgmap v20530: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 919 KiB/s wr, 28 op/s 2023-09-10T09:56:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38199.644355 secs 2023-09-10T09:56:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:56:09.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:56:09.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:56:09.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:56:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38200.890240 secs 2023-09-10T09:56:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:09 smithi170 ceph-mon[102832]: osdmap e1351: 12 total, 12 up, 12 in 2023-09-10T09:56:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38201.242332 secs 2023-09-10T09:56:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38200.890240 secs 2023-09-10T09:56:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:09 smithi082 ceph-mon[98694]: osdmap e1351: 12 total, 12 up, 12 in 2023-09-10T09:56:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38201.242332 secs 2023-09-10T09:56:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38200.890240 secs 2023-09-10T09:56:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:09 smithi098 ceph-mon[103124]: osdmap e1351: 12 total, 12 up, 12 in 2023-09-10T09:56:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38201.242332 secs 2023-09-10T09:56:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:10 smithi170 ceph-mon[102832]: pgmap v20532: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 34 op/s 2023-09-10T09:56:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:10 smithi082 ceph-mon[98694]: pgmap v20532: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 34 op/s 2023-09-10T09:56:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:10 smithi098 ceph-mon[103124]: pgmap v20532: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 34 op/s 2023-09-10T09:56:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:12 smithi170 ceph-mon[102832]: pgmap v20533: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 586 KiB/s wr, 32 op/s 2023-09-10T09:56:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:12 smithi082 ceph-mon[98694]: pgmap v20533: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 586 KiB/s wr, 32 op/s 2023-09-10T09:56:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:12 smithi098 ceph-mon[103124]: pgmap v20533: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 586 KiB/s wr, 32 op/s 2023-09-10T09:56:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38204.644416 secs 2023-09-10T09:56:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38204.644416 secs 2023-09-10T09:56:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38204.644416 secs 2023-09-10T09:56:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38205.890307 secs 2023-09-10T09:56:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:14 smithi170 ceph-mon[102832]: pgmap v20534: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 586 KiB/s wr, 32 op/s 2023-09-10T09:56:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:14 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:56:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:14 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38206.242395 secs 2023-09-10T09:56:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:14 smithi170 ceph-mon[102832]: osdmap e1352: 12 total, 12 up, 12 in 2023-09-10T09:56:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38205.890307 secs 2023-09-10T09:56:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:14 smithi082 ceph-mon[98694]: pgmap v20534: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 586 KiB/s wr, 32 op/s 2023-09-10T09:56:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:14 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:56:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:14 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38206.242395 secs 2023-09-10T09:56:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:14 smithi082 ceph-mon[98694]: osdmap e1352: 12 total, 12 up, 12 in 2023-09-10T09:56:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38205.890307 secs 2023-09-10T09:56:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:14 smithi098 ceph-mon[103124]: pgmap v20534: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 586 KiB/s wr, 32 op/s 2023-09-10T09:56:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:14 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:56:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:14 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38206.242395 secs 2023-09-10T09:56:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:14 smithi098 ceph-mon[103124]: osdmap e1352: 12 total, 12 up, 12 in 2023-09-10T09:56:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:16 smithi170 ceph-mon[102832]: pgmap v20536: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 253 KiB/s wr, 17 op/s 2023-09-10T09:56:17.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:56:17.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:16 smithi082 ceph-mon[98694]: pgmap v20536: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 253 KiB/s wr, 17 op/s 2023-09-10T09:56:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:56:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:16 smithi098 ceph-mon[103124]: pgmap v20536: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 253 KiB/s wr, 17 op/s 2023-09-10T09:56:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:56:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:18 smithi170 ceph-mon[102832]: pgmap v20537: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 229 KiB/s wr, 16 op/s 2023-09-10T09:56:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38209.644476 secs 2023-09-10T09:56:19.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:18 smithi082 ceph-mon[98694]: pgmap v20537: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 229 KiB/s wr, 16 op/s 2023-09-10T09:56:19.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38209.644476 secs 2023-09-10T09:56:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:18 smithi098 ceph-mon[103124]: pgmap v20537: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 229 KiB/s wr, 16 op/s 2023-09-10T09:56:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38209.644476 secs 2023-09-10T09:56:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38210.890367 secs 2023-09-10T09:56:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38211.242460 secs 2023-09-10T09:56:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38210.890367 secs 2023-09-10T09:56:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38211.242460 secs 2023-09-10T09:56:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38210.890367 secs 2023-09-10T09:56:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38211.242460 secs 2023-09-10T09:56:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:20 smithi170 ceph-mon[102832]: pgmap v20538: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 203 KiB/s wr, 14 op/s 2023-09-10T09:56:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:20 smithi082 ceph-mon[98694]: pgmap v20538: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 203 KiB/s wr, 14 op/s 2023-09-10T09:56:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:20 smithi098 ceph-mon[103124]: pgmap v20538: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 203 KiB/s wr, 14 op/s 2023-09-10T09:56:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:22 smithi170 ceph-mon[102832]: pgmap v20539: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 203 KiB/s wr, 14 op/s 2023-09-10T09:56:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:22 smithi082 ceph-mon[98694]: pgmap v20539: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 203 KiB/s wr, 14 op/s 2023-09-10T09:56:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:22 smithi098 ceph-mon[103124]: pgmap v20539: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 203 KiB/s wr, 14 op/s 2023-09-10T09:56:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38214.644536 secs 2023-09-10T09:56:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38214.644536 secs 2023-09-10T09:56:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38214.644536 secs 2023-09-10T09:56:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38215.890425 secs 2023-09-10T09:56:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:24 smithi170 ceph-mon[102832]: pgmap v20540: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 203 KiB/s wr, 14 op/s 2023-09-10T09:56:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38216.242488 secs 2023-09-10T09:56:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38215.890425 secs 2023-09-10T09:56:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:24 smithi082 ceph-mon[98694]: pgmap v20540: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 203 KiB/s wr, 14 op/s 2023-09-10T09:56:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38216.242488 secs 2023-09-10T09:56:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38215.890425 secs 2023-09-10T09:56:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:24 smithi098 ceph-mon[103124]: pgmap v20540: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 203 KiB/s wr, 14 op/s 2023-09-10T09:56:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38216.242488 secs 2023-09-10T09:56:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:25 smithi170 ceph-mon[102832]: pgmap v20541: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 181 KiB/s wr, 18 op/s 2023-09-10T09:56:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:25 smithi082 ceph-mon[98694]: pgmap v20541: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 181 KiB/s wr, 18 op/s 2023-09-10T09:56:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:25 smithi098 ceph-mon[103124]: pgmap v20541: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 181 KiB/s wr, 18 op/s 2023-09-10T09:56:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:28 smithi098 ceph-mon[103124]: pgmap v20542: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T09:56:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38219.644595 secs 2023-09-10T09:56:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:28 smithi170 ceph-mon[102832]: pgmap v20542: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T09:56:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38219.644595 secs 2023-09-10T09:56:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:28 smithi082 ceph-mon[98694]: pgmap v20542: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T09:56:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38219.644595 secs 2023-09-10T09:56:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38220.890513 secs 2023-09-10T09:56:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38221.242552 secs 2023-09-10T09:56:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38220.890513 secs 2023-09-10T09:56:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38221.242552 secs 2023-09-10T09:56:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38220.890513 secs 2023-09-10T09:56:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38221.242552 secs 2023-09-10T09:56:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:30 smithi170 ceph-mon[102832]: pgmap v20543: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T09:56:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:56:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:30 smithi082 ceph-mon[98694]: pgmap v20543: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T09:56:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:56:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:30 smithi098 ceph-mon[103124]: pgmap v20543: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T09:56:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:56:31.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:32 smithi170 ceph-mon[102832]: pgmap v20544: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 307 KiB/s wr, 17 op/s 2023-09-10T09:56:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:56:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:32 smithi082 ceph-mon[98694]: pgmap v20544: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 307 KiB/s wr, 17 op/s 2023-09-10T09:56:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:56:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:32 smithi098 ceph-mon[103124]: pgmap v20544: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 307 KiB/s wr, 17 op/s 2023-09-10T09:56:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:56:33.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38225.890575 secs 2023-09-10T09:56:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:34 smithi170 ceph-mon[102832]: pgmap v20545: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 166 KiB/s wr, 11 op/s 2023-09-10T09:56:35.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38226.242621 secs 2023-09-10T09:56:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38225.890575 secs 2023-09-10T09:56:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:34 smithi082 ceph-mon[98694]: pgmap v20545: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 166 KiB/s wr, 11 op/s 2023-09-10T09:56:35.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38226.242621 secs 2023-09-10T09:56:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38225.890575 secs 2023-09-10T09:56:35.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:34 smithi098 ceph-mon[103124]: pgmap v20545: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 166 KiB/s wr, 11 op/s 2023-09-10T09:56:35.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38226.242621 secs 2023-09-10T09:56:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:56:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:56:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:56:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:36 smithi170 ceph-mon[102832]: pgmap v20546: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 348 KiB/s wr, 16 op/s 2023-09-10T09:56:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:56:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:56:37.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:36 smithi082 ceph-mon[98694]: pgmap v20546: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 348 KiB/s wr, 16 op/s 2023-09-10T09:56:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:56:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:56:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:36 smithi098 ceph-mon[103124]: pgmap v20546: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 348 KiB/s wr, 16 op/s 2023-09-10T09:56:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:56:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:56:37.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:39.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:38 smithi170 ceph-mon[102832]: pgmap v20547: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 348 KiB/s wr, 11 op/s 2023-09-10T09:56:39.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:38 smithi082 ceph-mon[98694]: pgmap v20547: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 348 KiB/s wr, 11 op/s 2023-09-10T09:56:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:38 smithi098 ceph-mon[103124]: pgmap v20547: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 348 KiB/s wr, 11 op/s 2023-09-10T09:56:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38230.890661 secs 2023-09-10T09:56:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38231.242682 secs 2023-09-10T09:56:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38230.890661 secs 2023-09-10T09:56:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38231.242682 secs 2023-09-10T09:56:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38230.890661 secs 2023-09-10T09:56:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38231.242682 secs 2023-09-10T09:56:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:40 smithi170 ceph-mon[102832]: pgmap v20548: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 348 KiB/s wr, 11 op/s 2023-09-10T09:56:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:40 smithi082 ceph-mon[98694]: pgmap v20548: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 348 KiB/s wr, 11 op/s 2023-09-10T09:56:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:40 smithi098 ceph-mon[103124]: pgmap v20548: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 348 KiB/s wr, 11 op/s 2023-09-10T09:56:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:42 smithi170 ceph-mon[102832]: pgmap v20549: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 515 KiB/s wr, 18 op/s 2023-09-10T09:56:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:42 smithi082 ceph-mon[98694]: pgmap v20549: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 515 KiB/s wr, 18 op/s 2023-09-10T09:56:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:42 smithi098 ceph-mon[103124]: pgmap v20549: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 515 KiB/s wr, 18 op/s 2023-09-10T09:56:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38235.890718 secs 2023-09-10T09:56:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:44 smithi170 ceph-mon[102832]: pgmap v20550: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 349 KiB/s wr, 12 op/s 2023-09-10T09:56:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:56:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38236.242747 secs 2023-09-10T09:56:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38235.890718 secs 2023-09-10T09:56:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:44 smithi082 ceph-mon[98694]: pgmap v20550: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 349 KiB/s wr, 12 op/s 2023-09-10T09:56:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:56:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:45.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38236.242747 secs 2023-09-10T09:56:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38235.890718 secs 2023-09-10T09:56:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:44 smithi098 ceph-mon[103124]: pgmap v20550: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 349 KiB/s wr, 12 op/s 2023-09-10T09:56:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:56:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38236.242747 secs 2023-09-10T09:56:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:46 smithi170 ceph-mon[102832]: pgmap v20551: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 882 KiB/s wr, 19 op/s 2023-09-10T09:56:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:56:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:46 smithi082 ceph-mon[98694]: pgmap v20551: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 882 KiB/s wr, 19 op/s 2023-09-10T09:56:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:56:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:46 smithi098 ceph-mon[103124]: pgmap v20551: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 882 KiB/s wr, 19 op/s 2023-09-10T09:56:47.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:56:47.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:48 smithi170 ceph-mon[102832]: pgmap v20552: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 699 KiB/s wr, 13 op/s 2023-09-10T09:56:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:48 smithi082 ceph-mon[98694]: pgmap v20552: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 699 KiB/s wr, 13 op/s 2023-09-10T09:56:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:48 smithi098 ceph-mon[103124]: pgmap v20552: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 699 KiB/s wr, 13 op/s 2023-09-10T09:56:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38240.890774 secs 2023-09-10T09:56:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38241.242836 secs 2023-09-10T09:56:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38240.890774 secs 2023-09-10T09:56:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38241.242836 secs 2023-09-10T09:56:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38240.890774 secs 2023-09-10T09:56:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38241.242836 secs 2023-09-10T09:56:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:50 smithi170 ceph-mon[102832]: pgmap v20553: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 699 KiB/s wr, 13 op/s 2023-09-10T09:56:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:56:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:50 smithi082 ceph-mon[98694]: pgmap v20553: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 699 KiB/s wr, 13 op/s 2023-09-10T09:56:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:56:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:50 smithi098 ceph-mon[103124]: pgmap v20553: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 699 KiB/s wr, 13 op/s 2023-09-10T09:56:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:56:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:52 smithi170 ceph-mon[102832]: pgmap v20554: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:56:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:56:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:52 smithi082 ceph-mon[98694]: pgmap v20554: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:56:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:56:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:52 smithi098 ceph-mon[103124]: pgmap v20554: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:56:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:56:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:55.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38245.890833 secs 2023-09-10T09:56:55.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:54 smithi170 ceph-mon[102832]: pgmap v20555: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 961 KiB/s wr, 14 op/s 2023-09-10T09:56:55.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38246.242903 secs 2023-09-10T09:56:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38245.890833 secs 2023-09-10T09:56:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:54 smithi082 ceph-mon[98694]: pgmap v20555: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 961 KiB/s wr, 14 op/s 2023-09-10T09:56:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38246.242903 secs 2023-09-10T09:56:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38245.890833 secs 2023-09-10T09:56:55.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:54 smithi098 ceph-mon[103124]: pgmap v20555: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 961 KiB/s wr, 14 op/s 2023-09-10T09:56:55.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38246.242903 secs 2023-09-10T09:56:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:56 smithi170 ceph-mon[102832]: pgmap v20556: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:56:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:56 smithi082 ceph-mon[98694]: pgmap v20556: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:56:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:56 smithi098 ceph-mon[103124]: pgmap v20556: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:56:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:58 smithi170 ceph-mon[102832]: pgmap v20557: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 552 KiB/s wr, 14 op/s 2023-09-10T09:56:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:56:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38249.645083 secs 2023-09-10T09:56:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:58 smithi082 ceph-mon[98694]: pgmap v20557: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 552 KiB/s wr, 14 op/s 2023-09-10T09:56:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:56:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38249.645083 secs 2023-09-10T09:56:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:58 smithi098 ceph-mon[103124]: pgmap v20557: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 552 KiB/s wr, 14 op/s 2023-09-10T09:56:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:56:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:56:59.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38249.645083 secs 2023-09-10T09:57:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38250.890917 secs 2023-09-10T09:57:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:56:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38251.242992 secs 2023-09-10T09:57:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38250.890917 secs 2023-09-10T09:57:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:56:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38251.242992 secs 2023-09-10T09:57:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38250.890917 secs 2023-09-10T09:57:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:56:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38251.242992 secs 2023-09-10T09:57:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:00 smithi170 ceph-mon[102832]: pgmap v20558: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 552 KiB/s wr, 14 op/s 2023-09-10T09:57:01.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:57:01.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:57:01.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:00 smithi082 ceph-mon[98694]: pgmap v20558: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 552 KiB/s wr, 14 op/s 2023-09-10T09:57:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:57:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:57:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:00 smithi098 ceph-mon[103124]: pgmap v20558: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 552 KiB/s wr, 14 op/s 2023-09-10T09:57:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:57:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:57:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:02 smithi170 ceph-mon[102832]: pgmap v20559: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 36 op/s 2023-09-10T09:57:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:02 smithi082 ceph-mon[98694]: pgmap v20559: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 36 op/s 2023-09-10T09:57:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:02 smithi098 ceph-mon[103124]: pgmap v20559: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 36 op/s 2023-09-10T09:57:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38254.645175 secs 2023-09-10T09:57:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38254.645175 secs 2023-09-10T09:57:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38254.645175 secs 2023-09-10T09:57:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38255.890975 secs 2023-09-10T09:57:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:04 smithi170 ceph-mon[102832]: pgmap v20560: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 670 KiB/s wr, 29 op/s 2023-09-10T09:57:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:04 smithi170 ceph-mon[102832]: osdmap e1353: 12 total, 12 up, 12 in 2023-09-10T09:57:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38256.243064 secs 2023-09-10T09:57:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38255.890975 secs 2023-09-10T09:57:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:04 smithi082 ceph-mon[98694]: pgmap v20560: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 670 KiB/s wr, 29 op/s 2023-09-10T09:57:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:04 smithi082 ceph-mon[98694]: osdmap e1353: 12 total, 12 up, 12 in 2023-09-10T09:57:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38256.243064 secs 2023-09-10T09:57:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38255.890975 secs 2023-09-10T09:57:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:04 smithi098 ceph-mon[103124]: pgmap v20560: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 670 KiB/s wr, 29 op/s 2023-09-10T09:57:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:04 smithi098 ceph-mon[103124]: osdmap e1353: 12 total, 12 up, 12 in 2023-09-10T09:57:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38256.243064 secs 2023-09-10T09:57:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:06 smithi170 ceph-mon[102832]: pgmap v20562: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.0 MiB/s wr, 34 op/s 2023-09-10T09:57:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:06 smithi082 ceph-mon[98694]: pgmap v20562: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.0 MiB/s wr, 34 op/s 2023-09-10T09:57:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:06 smithi098 ceph-mon[103124]: pgmap v20562: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.0 MiB/s wr, 34 op/s 2023-09-10T09:57:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:08 smithi170 ceph-mon[102832]: pgmap v20563: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.0 MiB/s wr, 34 op/s 2023-09-10T09:57:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38259.645272 secs 2023-09-10T09:57:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:08 smithi082 ceph-mon[98694]: pgmap v20563: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.0 MiB/s wr, 34 op/s 2023-09-10T09:57:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38259.645272 secs 2023-09-10T09:57:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:08 smithi098 ceph-mon[103124]: pgmap v20563: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.0 MiB/s wr, 34 op/s 2023-09-10T09:57:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38259.645272 secs 2023-09-10T09:57:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38260.891033 secs 2023-09-10T09:57:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:57:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38261.243130 secs 2023-09-10T09:57:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:09 smithi170 ceph-mon[102832]: osdmap e1354: 12 total, 12 up, 12 in 2023-09-10T09:57:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:57:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:57:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:57:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38260.891033 secs 2023-09-10T09:57:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:57:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38261.243130 secs 2023-09-10T09:57:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:09 smithi082 ceph-mon[98694]: osdmap e1354: 12 total, 12 up, 12 in 2023-09-10T09:57:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:57:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:57:10.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:57:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38260.891033 secs 2023-09-10T09:57:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:57:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38261.243130 secs 2023-09-10T09:57:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:09 smithi098 ceph-mon[103124]: osdmap e1354: 12 total, 12 up, 12 in 2023-09-10T09:57:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:57:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:57:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:57:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:10 smithi170 ceph-mon[102832]: pgmap v20565: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 487 KiB/s wr, 10 op/s 2023-09-10T09:57:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:57:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:10 smithi082 ceph-mon[98694]: pgmap v20565: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 487 KiB/s wr, 10 op/s 2023-09-10T09:57:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:57:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:10 smithi098 ceph-mon[103124]: pgmap v20565: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 487 KiB/s wr, 10 op/s 2023-09-10T09:57:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:57:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:12 smithi170 ceph-mon[102832]: pgmap v20566: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 697 KiB/s wr, 20 op/s 2023-09-10T09:57:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:57:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:12 smithi082 ceph-mon[98694]: pgmap v20566: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 697 KiB/s wr, 20 op/s 2023-09-10T09:57:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:57:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:12 smithi098 ceph-mon[103124]: pgmap v20566: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 697 KiB/s wr, 20 op/s 2023-09-10T09:57:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:57:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38264.645341 secs 2023-09-10T09:57:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38264.645341 secs 2023-09-10T09:57:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38264.645341 secs 2023-09-10T09:57:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38265.891120 secs 2023-09-10T09:57:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:14 smithi170 ceph-mon[102832]: pgmap v20567: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 566 KiB/s wr, 16 op/s 2023-09-10T09:57:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38266.243197 secs 2023-09-10T09:57:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38265.891120 secs 2023-09-10T09:57:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:14 smithi082 ceph-mon[98694]: pgmap v20567: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 566 KiB/s wr, 16 op/s 2023-09-10T09:57:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38266.243197 secs 2023-09-10T09:57:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38265.891120 secs 2023-09-10T09:57:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:14 smithi098 ceph-mon[103124]: pgmap v20567: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 566 KiB/s wr, 16 op/s 2023-09-10T09:57:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38266.243197 secs 2023-09-10T09:57:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:16 smithi170 ceph-mon[102832]: pgmap v20568: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 203 KiB/s wr, 16 op/s 2023-09-10T09:57:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:16 smithi082 ceph-mon[98694]: pgmap v20568: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 203 KiB/s wr, 16 op/s 2023-09-10T09:57:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:16 smithi098 ceph-mon[103124]: pgmap v20568: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 203 KiB/s wr, 16 op/s 2023-09-10T09:57:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:18 smithi170 ceph-mon[102832]: pgmap v20569: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 203 KiB/s wr, 16 op/s 2023-09-10T09:57:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38269.645399 secs 2023-09-10T09:57:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:18 smithi082 ceph-mon[98694]: pgmap v20569: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 203 KiB/s wr, 16 op/s 2023-09-10T09:57:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38269.645399 secs 2023-09-10T09:57:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:18 smithi098 ceph-mon[103124]: pgmap v20569: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 203 KiB/s wr, 16 op/s 2023-09-10T09:57:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38269.645399 secs 2023-09-10T09:57:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38270.891173 secs 2023-09-10T09:57:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38271.243264 secs 2023-09-10T09:57:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38270.891173 secs 2023-09-10T09:57:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38271.243264 secs 2023-09-10T09:57:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38270.891173 secs 2023-09-10T09:57:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38271.243264 secs 2023-09-10T09:57:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:20 smithi170 ceph-mon[102832]: pgmap v20570: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 198 KiB/s wr, 15 op/s 2023-09-10T09:57:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:20 smithi082 ceph-mon[98694]: pgmap v20570: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 198 KiB/s wr, 15 op/s 2023-09-10T09:57:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:20 smithi098 ceph-mon[103124]: pgmap v20570: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 198 KiB/s wr, 15 op/s 2023-09-10T09:57:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:22 smithi170 ceph-mon[102832]: pgmap v20571: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 169 KiB/s wr, 19 op/s 2023-09-10T09:57:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:22 smithi082 ceph-mon[98694]: pgmap v20571: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 169 KiB/s wr, 19 op/s 2023-09-10T09:57:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:22 smithi098 ceph-mon[103124]: pgmap v20571: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 169 KiB/s wr, 19 op/s 2023-09-10T09:57:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38274.645461 secs 2023-09-10T09:57:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38274.645461 secs 2023-09-10T09:57:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38274.645461 secs 2023-09-10T09:57:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38275.891234 secs 2023-09-10T09:57:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:24 smithi170 ceph-mon[102832]: pgmap v20572: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 12 op/s 2023-09-10T09:57:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38276.243334 secs 2023-09-10T09:57:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38275.891234 secs 2023-09-10T09:57:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:24 smithi082 ceph-mon[98694]: pgmap v20572: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 12 op/s 2023-09-10T09:57:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38276.243334 secs 2023-09-10T09:57:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38275.891234 secs 2023-09-10T09:57:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:24 smithi098 ceph-mon[103124]: pgmap v20572: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 12 op/s 2023-09-10T09:57:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38276.243334 secs 2023-09-10T09:57:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:26 smithi170 ceph-mon[102832]: pgmap v20573: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 171 KiB/s wr, 18 op/s 2023-09-10T09:57:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:26 smithi082 ceph-mon[98694]: pgmap v20573: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 171 KiB/s wr, 18 op/s 2023-09-10T09:57:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:26 smithi098 ceph-mon[103124]: pgmap v20573: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 171 KiB/s wr, 18 op/s 2023-09-10T09:57:28.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:27 smithi082 ceph-mon[98694]: pgmap v20574: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 142 KiB/s wr, 12 op/s 2023-09-10T09:57:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:27 smithi098 ceph-mon[103124]: pgmap v20574: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 142 KiB/s wr, 12 op/s 2023-09-10T09:57:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:27 smithi170 ceph-mon[102832]: pgmap v20574: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 142 KiB/s wr, 12 op/s 2023-09-10T09:57:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38279.645548 secs 2023-09-10T09:57:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38279.645548 secs 2023-09-10T09:57:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38279.645548 secs 2023-09-10T09:57:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38280.891316 secs 2023-09-10T09:57:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38281.243399 secs 2023-09-10T09:57:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:29 smithi082 ceph-mon[98694]: pgmap v20575: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 142 KiB/s wr, 12 op/s 2023-09-10T09:57:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38280.891316 secs 2023-09-10T09:57:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38281.243399 secs 2023-09-10T09:57:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:29 smithi098 ceph-mon[103124]: pgmap v20575: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 142 KiB/s wr, 12 op/s 2023-09-10T09:57:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38280.891316 secs 2023-09-10T09:57:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38281.243399 secs 2023-09-10T09:57:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:29 smithi170 ceph-mon[102832]: pgmap v20575: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 142 KiB/s wr, 12 op/s 2023-09-10T09:57:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:32 smithi170 ceph-mon[102832]: pgmap v20576: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 142 KiB/s wr, 17 op/s 2023-09-10T09:57:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:32 smithi082 ceph-mon[98694]: pgmap v20576: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 142 KiB/s wr, 17 op/s 2023-09-10T09:57:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:32 smithi098 ceph-mon[103124]: pgmap v20576: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 142 KiB/s wr, 17 op/s 2023-09-10T09:57:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38285.891398 secs 2023-09-10T09:57:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:34 smithi170 ceph-mon[102832]: pgmap v20577: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T09:57:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:57:35.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:35.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38286.243465 secs 2023-09-10T09:57:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38285.891398 secs 2023-09-10T09:57:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:34 smithi082 ceph-mon[98694]: pgmap v20577: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T09:57:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:57:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38286.243465 secs 2023-09-10T09:57:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38285.891398 secs 2023-09-10T09:57:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:34 smithi098 ceph-mon[103124]: pgmap v20577: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T09:57:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:57:35.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:35.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38286.243465 secs 2023-09-10T09:57:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:57:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:57:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:57:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:36 smithi170 ceph-mon[102832]: pgmap v20578: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 446 KiB/s wr, 17 op/s 2023-09-10T09:57:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:57:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:57:37.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:36 smithi082 ceph-mon[98694]: pgmap v20578: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 446 KiB/s wr, 17 op/s 2023-09-10T09:57:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:57:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:57:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:36 smithi098 ceph-mon[103124]: pgmap v20578: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 446 KiB/s wr, 17 op/s 2023-09-10T09:57:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:57:37.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:57:37.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:38 smithi170 ceph-mon[102832]: pgmap v20579: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 304 KiB/s wr, 11 op/s 2023-09-10T09:57:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:38 smithi082 ceph-mon[98694]: pgmap v20579: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 304 KiB/s wr, 11 op/s 2023-09-10T09:57:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:38 smithi098 ceph-mon[103124]: pgmap v20579: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 304 KiB/s wr, 11 op/s 2023-09-10T09:57:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38290.891491 secs 2023-09-10T09:57:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38291.243486 secs 2023-09-10T09:57:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38290.891491 secs 2023-09-10T09:57:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38291.243486 secs 2023-09-10T09:57:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38290.891491 secs 2023-09-10T09:57:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38291.243486 secs 2023-09-10T09:57:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:40 smithi170 ceph-mon[102832]: pgmap v20580: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 304 KiB/s wr, 11 op/s 2023-09-10T09:57:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:57:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:40 smithi082 ceph-mon[98694]: pgmap v20580: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 304 KiB/s wr, 11 op/s 2023-09-10T09:57:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:57:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:40 smithi098 ceph-mon[103124]: pgmap v20580: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 304 KiB/s wr, 11 op/s 2023-09-10T09:57:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:57:41.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:42 smithi170 ceph-mon[102832]: pgmap v20581: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 472 KiB/s wr, 17 op/s 2023-09-10T09:57:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:42 smithi082 ceph-mon[98694]: pgmap v20581: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 472 KiB/s wr, 17 op/s 2023-09-10T09:57:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:42 smithi098 ceph-mon[103124]: pgmap v20581: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 472 KiB/s wr, 17 op/s 2023-09-10T09:57:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38295.891550 secs 2023-09-10T09:57:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:44 smithi170 ceph-mon[102832]: pgmap v20582: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 472 KiB/s wr, 12 op/s 2023-09-10T09:57:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38296.243554 secs 2023-09-10T09:57:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38295.891550 secs 2023-09-10T09:57:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:44 smithi082 ceph-mon[98694]: pgmap v20582: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 472 KiB/s wr, 12 op/s 2023-09-10T09:57:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38296.243554 secs 2023-09-10T09:57:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38295.891550 secs 2023-09-10T09:57:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:44 smithi098 ceph-mon[103124]: pgmap v20582: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 472 KiB/s wr, 12 op/s 2023-09-10T09:57:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38296.243554 secs 2023-09-10T09:57:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:46 smithi170 ceph-mon[102832]: pgmap v20583: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 686 KiB/s wr, 18 op/s 2023-09-10T09:57:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:57:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:46 smithi082 ceph-mon[98694]: pgmap v20583: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 686 KiB/s wr, 18 op/s 2023-09-10T09:57:47.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:57:47.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:46 smithi098 ceph-mon[103124]: pgmap v20583: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 686 KiB/s wr, 18 op/s 2023-09-10T09:57:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:57:47.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:49.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:48 smithi170 ceph-mon[102832]: pgmap v20584: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 381 KiB/s wr, 12 op/s 2023-09-10T09:57:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:48 smithi082 ceph-mon[98694]: pgmap v20584: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 381 KiB/s wr, 12 op/s 2023-09-10T09:57:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:48 smithi098 ceph-mon[103124]: pgmap v20584: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 381 KiB/s wr, 12 op/s 2023-09-10T09:57:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38300.891631 secs 2023-09-10T09:57:50.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38301.243622 secs 2023-09-10T09:57:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38300.891631 secs 2023-09-10T09:57:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38301.243622 secs 2023-09-10T09:57:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38300.891631 secs 2023-09-10T09:57:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38301.243622 secs 2023-09-10T09:57:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:50 smithi170 ceph-mon[102832]: pgmap v20585: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 381 KiB/s wr, 12 op/s 2023-09-10T09:57:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:57:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:50 smithi082 ceph-mon[98694]: pgmap v20585: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 381 KiB/s wr, 12 op/s 2023-09-10T09:57:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:57:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:50 smithi098 ceph-mon[103124]: pgmap v20585: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 381 KiB/s wr, 12 op/s 2023-09-10T09:57:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:57:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:52 smithi170 ceph-mon[102832]: pgmap v20586: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 772 KiB/s wr, 20 op/s 2023-09-10T09:57:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:57:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:52 smithi082 ceph-mon[98694]: pgmap v20586: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 772 KiB/s wr, 20 op/s 2023-09-10T09:57:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:57:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:52 smithi098 ceph-mon[103124]: pgmap v20586: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 772 KiB/s wr, 20 op/s 2023-09-10T09:57:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:57:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38305.891715 secs 2023-09-10T09:57:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:54 smithi170 ceph-mon[102832]: pgmap v20587: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 604 KiB/s wr, 14 op/s 2023-09-10T09:57:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:57:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:55.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38306.243687 secs 2023-09-10T09:57:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38305.891715 secs 2023-09-10T09:57:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:54 smithi082 ceph-mon[98694]: pgmap v20587: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 604 KiB/s wr, 14 op/s 2023-09-10T09:57:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:57:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38306.243687 secs 2023-09-10T09:57:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38305.891715 secs 2023-09-10T09:57:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:54 smithi098 ceph-mon[103124]: pgmap v20587: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 604 KiB/s wr, 14 op/s 2023-09-10T09:57:55.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:57:55.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:55.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38306.243687 secs 2023-09-10T09:57:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:56 smithi170 ceph-mon[102832]: pgmap v20588: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:57:57.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:57:57.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:56 smithi082 ceph-mon[98694]: pgmap v20588: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:57:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:57:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:56 smithi098 ceph-mon[103124]: pgmap v20588: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:57:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:57:57.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:58 smithi170 ceph-mon[102832]: pgmap v20589: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 896 KiB/s wr, 14 op/s 2023-09-10T09:57:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:57:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38309.646006 secs 2023-09-10T09:57:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:58 smithi082 ceph-mon[98694]: pgmap v20589: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 896 KiB/s wr, 14 op/s 2023-09-10T09:57:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:57:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38309.646006 secs 2023-09-10T09:57:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:58 smithi098 ceph-mon[103124]: pgmap v20589: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 896 KiB/s wr, 14 op/s 2023-09-10T09:57:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:57:59.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:57:59.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38309.646006 secs 2023-09-10T09:58:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38310.891771 secs 2023-09-10T09:58:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:57:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38311.243750 secs 2023-09-10T09:58:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38310.891771 secs 2023-09-10T09:58:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:57:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38311.243750 secs 2023-09-10T09:58:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38310.891771 secs 2023-09-10T09:58:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:57:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38311.243750 secs 2023-09-10T09:58:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:00 smithi170 ceph-mon[102832]: pgmap v20590: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 896 KiB/s wr, 14 op/s 2023-09-10T09:58:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:58:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:58:01.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:00 smithi082 ceph-mon[98694]: pgmap v20590: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 896 KiB/s wr, 14 op/s 2023-09-10T09:58:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:58:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:58:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:00 smithi098 ceph-mon[103124]: pgmap v20590: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 896 KiB/s wr, 14 op/s 2023-09-10T09:58:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:58:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:58:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:02 smithi170 ceph-mon[102832]: pgmap v20591: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 29 op/s 2023-09-10T09:58:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:02 smithi082 ceph-mon[98694]: pgmap v20591: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 29 op/s 2023-09-10T09:58:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:02 smithi098 ceph-mon[103124]: pgmap v20591: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 29 op/s 2023-09-10T09:58:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38314.646068 secs 2023-09-10T09:58:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38314.646068 secs 2023-09-10T09:58:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38314.646068 secs 2023-09-10T09:58:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38315.891830 secs 2023-09-10T09:58:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:04 smithi170 ceph-mon[102832]: pgmap v20592: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 1020 KiB/s wr, 21 op/s 2023-09-10T09:58:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:04 smithi170 ceph-mon[102832]: osdmap e1355: 12 total, 12 up, 12 in 2023-09-10T09:58:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38316.243817 secs 2023-09-10T09:58:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38315.891830 secs 2023-09-10T09:58:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:04 smithi082 ceph-mon[98694]: pgmap v20592: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 1020 KiB/s wr, 21 op/s 2023-09-10T09:58:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:04 smithi082 ceph-mon[98694]: osdmap e1355: 12 total, 12 up, 12 in 2023-09-10T09:58:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38316.243817 secs 2023-09-10T09:58:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38315.891830 secs 2023-09-10T09:58:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:04 smithi098 ceph-mon[103124]: pgmap v20592: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 1020 KiB/s wr, 21 op/s 2023-09-10T09:58:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:04 smithi098 ceph-mon[103124]: osdmap e1355: 12 total, 12 up, 12 in 2023-09-10T09:58:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38316.243817 secs 2023-09-10T09:58:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:06 smithi170 ceph-mon[102832]: pgmap v20594: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 835 KiB/s wr, 34 op/s 2023-09-10T09:58:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:06 smithi082 ceph-mon[98694]: pgmap v20594: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 835 KiB/s wr, 34 op/s 2023-09-10T09:58:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:06 smithi098 ceph-mon[103124]: pgmap v20594: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 835 KiB/s wr, 34 op/s 2023-09-10T09:58:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:08 smithi170 ceph-mon[102832]: pgmap v20595: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 835 KiB/s wr, 34 op/s 2023-09-10T09:58:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38319.646182 secs 2023-09-10T09:58:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:08 smithi082 ceph-mon[98694]: pgmap v20595: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 835 KiB/s wr, 34 op/s 2023-09-10T09:58:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38319.646182 secs 2023-09-10T09:58:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:08 smithi098 ceph-mon[103124]: pgmap v20595: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 835 KiB/s wr, 34 op/s 2023-09-10T09:58:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38319.646182 secs 2023-09-10T09:58:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38320.891883 secs 2023-09-10T09:58:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38321.243882 secs 2023-09-10T09:58:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:09 smithi170 ceph-mon[102832]: osdmap e1356: 12 total, 12 up, 12 in 2023-09-10T09:58:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:58:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38320.891883 secs 2023-09-10T09:58:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38321.243882 secs 2023-09-10T09:58:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:09 smithi082 ceph-mon[98694]: osdmap e1356: 12 total, 12 up, 12 in 2023-09-10T09:58:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:58:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38320.891883 secs 2023-09-10T09:58:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38321.243882 secs 2023-09-10T09:58:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:09 smithi098 ceph-mon[103124]: osdmap e1356: 12 total, 12 up, 12 in 2023-09-10T09:58:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:58:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:10 smithi170 ceph-mon[102832]: pgmap v20597: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 274 KiB/s wr, 20 op/s 2023-09-10T09:58:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:10 smithi082 ceph-mon[98694]: pgmap v20597: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 274 KiB/s wr, 20 op/s 2023-09-10T09:58:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:10 smithi098 ceph-mon[103124]: pgmap v20597: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 274 KiB/s wr, 20 op/s 2023-09-10T09:58:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:12 smithi170 ceph-mon[102832]: pgmap v20598: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 485 KiB/s wr, 31 op/s 2023-09-10T09:58:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:58:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:13.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:13.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:13.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:13.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:13.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:13.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:12 smithi082 ceph-mon[98694]: pgmap v20598: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 485 KiB/s wr, 31 op/s 2023-09-10T09:58:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:58:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:13.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:13.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:12 smithi098 ceph-mon[103124]: pgmap v20598: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 485 KiB/s wr, 31 op/s 2023-09-10T09:58:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:58:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:13.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:13.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:13.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:13.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:13.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:13.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38324.646244 secs 2023-09-10T09:58:14.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:14.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:14.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:14.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:58:14.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:14.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:58:14.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:58:14.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:58:14.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:58:14.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:14 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T09:58:14.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:14 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T09:58:14.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:14.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:14.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:58:14.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:58:14.462 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:58:14.462 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:58:14.462 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:14 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T09:58:14.462 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:14 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T09:58:14.462 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:58:14.463 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:58:14.463 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:14.463 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:14 smithi082 ceph-mon[98694]: pgmap v20599: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 393 KiB/s wr, 25 op/s 2023-09-10T09:58:14.463 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:14 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:58:14.463 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:14 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38324.646244 secs 2023-09-10T09:58:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:58:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:58:14.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:58:14.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:58:14.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:58:14.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:14 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T09:58:14.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:14 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T09:58:14.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:14.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:14.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:58:14.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:58:14.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:58:14.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:58:14.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:14 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T09:58:14.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:14 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T09:58:14.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:58:14.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:58:14.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:14.509 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:14 smithi098 ceph-mon[103124]: pgmap v20599: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 393 KiB/s wr, 25 op/s 2023-09-10T09:58:14.509 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:14 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:58:14.509 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:14 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38324.646244 secs 2023-09-10T09:58:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:58:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:58:14.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:58:14.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:58:14.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:58:14.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:14 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T09:58:14.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:14 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T09:58:14.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:14.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:14.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:58:14.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:58:14.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:58:14.682 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T09:58:14.682 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:14 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T09:58:14.682 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:14 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T09:58:14.682 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:58:14.682 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:58:14.683 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:58:14.683 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:14 smithi170 ceph-mon[102832]: pgmap v20599: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 393 KiB/s wr, 25 op/s 2023-09-10T09:58:14.683 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:14 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:58:14.683 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:14 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38325.891965 secs 2023-09-10T09:58:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38326.243947 secs 2023-09-10T09:58:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38325.891965 secs 2023-09-10T09:58:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38326.243947 secs 2023-09-10T09:58:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38325.891965 secs 2023-09-10T09:58:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38326.243947 secs 2023-09-10T09:58:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:16 smithi098 ceph-mon[103124]: pgmap v20600: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 204 KiB/s wr, 14 op/s 2023-09-10T09:58:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:16 smithi170 ceph-mon[102832]: pgmap v20600: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 204 KiB/s wr, 14 op/s 2023-09-10T09:58:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:16 smithi082 ceph-mon[98694]: pgmap v20600: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 204 KiB/s wr, 14 op/s 2023-09-10T09:58:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:18 smithi170 ceph-mon[102832]: pgmap v20601: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 204 KiB/s wr, 14 op/s 2023-09-10T09:58:19.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38329.646372 secs 2023-09-10T09:58:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:18 smithi082 ceph-mon[98694]: pgmap v20601: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 204 KiB/s wr, 14 op/s 2023-09-10T09:58:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38329.646372 secs 2023-09-10T09:58:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:18 smithi098 ceph-mon[103124]: pgmap v20601: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 204 KiB/s wr, 14 op/s 2023-09-10T09:58:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38329.646372 secs 2023-09-10T09:58:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38330.892035 secs 2023-09-10T09:58:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38331.244012 secs 2023-09-10T09:58:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38330.892035 secs 2023-09-10T09:58:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38331.244012 secs 2023-09-10T09:58:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38330.892035 secs 2023-09-10T09:58:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38331.244012 secs 2023-09-10T09:58:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:20 smithi170 ceph-mon[102832]: pgmap v20602: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 199 KiB/s wr, 14 op/s 2023-09-10T09:58:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:20 smithi082 ceph-mon[98694]: pgmap v20602: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 199 KiB/s wr, 14 op/s 2023-09-10T09:58:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:20 smithi098 ceph-mon[103124]: pgmap v20602: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 199 KiB/s wr, 14 op/s 2023-09-10T09:58:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:22 smithi170 ceph-mon[102832]: pgmap v20603: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 170 KiB/s wr, 17 op/s 2023-09-10T09:58:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:22 smithi082 ceph-mon[98694]: pgmap v20603: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 170 KiB/s wr, 17 op/s 2023-09-10T09:58:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:22 smithi098 ceph-mon[103124]: pgmap v20603: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 170 KiB/s wr, 17 op/s 2023-09-10T09:58:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38334.646516 secs 2023-09-10T09:58:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38334.646516 secs 2023-09-10T09:58:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38334.646516 secs 2023-09-10T09:58:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38335.892092 secs 2023-09-10T09:58:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:24 smithi170 ceph-mon[102832]: pgmap v20604: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 10 op/s 2023-09-10T09:58:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38336.244076 secs 2023-09-10T09:58:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38335.892092 secs 2023-09-10T09:58:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:24 smithi082 ceph-mon[98694]: pgmap v20604: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 10 op/s 2023-09-10T09:58:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38336.244076 secs 2023-09-10T09:58:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38335.892092 secs 2023-09-10T09:58:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:24 smithi098 ceph-mon[103124]: pgmap v20604: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 10 op/s 2023-09-10T09:58:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38336.244076 secs 2023-09-10T09:58:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:26 smithi170 ceph-mon[102832]: pgmap v20605: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 171 KiB/s wr, 16 op/s 2023-09-10T09:58:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:26 smithi082 ceph-mon[98694]: pgmap v20605: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 171 KiB/s wr, 16 op/s 2023-09-10T09:58:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:26 smithi098 ceph-mon[103124]: pgmap v20605: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 171 KiB/s wr, 16 op/s 2023-09-10T09:58:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:28 smithi170 ceph-mon[102832]: pgmap v20606: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T09:58:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38339.646592 secs 2023-09-10T09:58:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:28 smithi082 ceph-mon[98694]: pgmap v20606: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T09:58:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38339.646592 secs 2023-09-10T09:58:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:28 smithi098 ceph-mon[103124]: pgmap v20606: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T09:58:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38339.646592 secs 2023-09-10T09:58:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38340.892158 secs 2023-09-10T09:58:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38341.244141 secs 2023-09-10T09:58:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38340.892158 secs 2023-09-10T09:58:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38341.244141 secs 2023-09-10T09:58:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38340.892158 secs 2023-09-10T09:58:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38341.244141 secs 2023-09-10T09:58:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:30 smithi170 ceph-mon[102832]: pgmap v20607: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T09:58:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:30 smithi082 ceph-mon[98694]: pgmap v20607: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T09:58:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:30 smithi098 ceph-mon[103124]: pgmap v20607: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T09:58:33.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:32 smithi170 ceph-mon[102832]: pgmap v20608: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 142 KiB/s wr, 16 op/s 2023-09-10T09:58:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:32 smithi082 ceph-mon[98694]: pgmap v20608: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 142 KiB/s wr, 16 op/s 2023-09-10T09:58:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:32 smithi098 ceph-mon[103124]: pgmap v20608: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 142 KiB/s wr, 16 op/s 2023-09-10T09:58:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38345.892230 secs 2023-09-10T09:58:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:34 smithi170 ceph-mon[102832]: pgmap v20609: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T09:58:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:58:35.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:35.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38346.244207 secs 2023-09-10T09:58:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38345.892230 secs 2023-09-10T09:58:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:34 smithi082 ceph-mon[98694]: pgmap v20609: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T09:58:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:58:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38346.244207 secs 2023-09-10T09:58:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38345.892230 secs 2023-09-10T09:58:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:34 smithi098 ceph-mon[103124]: pgmap v20609: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T09:58:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:58:35.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:35.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38346.244207 secs 2023-09-10T09:58:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:35 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:58:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:35 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:58:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:35 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:58:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:36 smithi170 ceph-mon[102832]: pgmap v20610: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 16 op/s 2023-09-10T09:58:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:36 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:58:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:58:37.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:36 smithi082 ceph-mon[98694]: pgmap v20610: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 16 op/s 2023-09-10T09:58:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:36 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:58:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:58:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:36 smithi098 ceph-mon[103124]: pgmap v20610: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 445 KiB/s wr, 16 op/s 2023-09-10T09:58:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:36 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:58:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:58:37.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:38 smithi170 ceph-mon[102832]: pgmap v20611: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 303 KiB/s wr, 10 op/s 2023-09-10T09:58:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:38 smithi082 ceph-mon[98694]: pgmap v20611: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 303 KiB/s wr, 10 op/s 2023-09-10T09:58:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:38 smithi098 ceph-mon[103124]: pgmap v20611: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 303 KiB/s wr, 10 op/s 2023-09-10T09:58:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38350.892286 secs 2023-09-10T09:58:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38351.244274 secs 2023-09-10T09:58:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38350.892286 secs 2023-09-10T09:58:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38351.244274 secs 2023-09-10T09:58:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38350.892286 secs 2023-09-10T09:58:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38351.244274 secs 2023-09-10T09:58:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:40 smithi170 ceph-mon[102832]: pgmap v20612: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 303 KiB/s wr, 10 op/s 2023-09-10T09:58:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:40 smithi082 ceph-mon[98694]: pgmap v20612: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 303 KiB/s wr, 10 op/s 2023-09-10T09:58:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:40 smithi098 ceph-mon[103124]: pgmap v20612: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 303 KiB/s wr, 10 op/s 2023-09-10T09:58:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:42 smithi170 ceph-mon[102832]: pgmap v20613: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 417 KiB/s wr, 16 op/s 2023-09-10T09:58:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:42 smithi082 ceph-mon[98694]: pgmap v20613: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 417 KiB/s wr, 16 op/s 2023-09-10T09:58:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:42 smithi098 ceph-mon[103124]: pgmap v20613: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 417 KiB/s wr, 16 op/s 2023-09-10T09:58:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38355.892355 secs 2023-09-10T09:58:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:44 smithi170 ceph-mon[102832]: pgmap v20614: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 417 KiB/s wr, 11 op/s 2023-09-10T09:58:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:58:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38356.244337 secs 2023-09-10T09:58:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38355.892355 secs 2023-09-10T09:58:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:44 smithi082 ceph-mon[98694]: pgmap v20614: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 417 KiB/s wr, 11 op/s 2023-09-10T09:58:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:58:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38356.244337 secs 2023-09-10T09:58:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38355.892355 secs 2023-09-10T09:58:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:44 smithi098 ceph-mon[103124]: pgmap v20614: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 417 KiB/s wr, 11 op/s 2023-09-10T09:58:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:58:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38356.244337 secs 2023-09-10T09:58:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:46 smithi170 ceph-mon[102832]: pgmap v20615: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 608 KiB/s wr, 17 op/s 2023-09-10T09:58:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:46 smithi082 ceph-mon[98694]: pgmap v20615: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 608 KiB/s wr, 17 op/s 2023-09-10T09:58:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:46 smithi098 ceph-mon[103124]: pgmap v20615: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 608 KiB/s wr, 17 op/s 2023-09-10T09:58:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:48 smithi170 ceph-mon[102832]: pgmap v20616: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 305 KiB/s wr, 12 op/s 2023-09-10T09:58:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:48 smithi082 ceph-mon[98694]: pgmap v20616: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 305 KiB/s wr, 12 op/s 2023-09-10T09:58:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:48 smithi098 ceph-mon[103124]: pgmap v20616: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 305 KiB/s wr, 12 op/s 2023-09-10T09:58:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38360.892435 secs 2023-09-10T09:58:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38361.244404 secs 2023-09-10T09:58:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38360.892435 secs 2023-09-10T09:58:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38361.244404 secs 2023-09-10T09:58:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38360.892435 secs 2023-09-10T09:58:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38361.244404 secs 2023-09-10T09:58:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:50 smithi170 ceph-mon[102832]: pgmap v20617: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 305 KiB/s wr, 12 op/s 2023-09-10T09:58:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:58:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:50 smithi082 ceph-mon[98694]: pgmap v20617: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 305 KiB/s wr, 12 op/s 2023-09-10T09:58:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:58:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:50 smithi098 ceph-mon[103124]: pgmap v20617: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 305 KiB/s wr, 12 op/s 2023-09-10T09:58:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:58:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:52 smithi170 ceph-mon[102832]: pgmap v20618: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 884 KiB/s wr, 19 op/s 2023-09-10T09:58:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:58:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:52 smithi082 ceph-mon[98694]: pgmap v20618: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 884 KiB/s wr, 19 op/s 2023-09-10T09:58:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:58:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:52 smithi098 ceph-mon[103124]: pgmap v20618: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 884 KiB/s wr, 19 op/s 2023-09-10T09:58:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:58:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38365.892517 secs 2023-09-10T09:58:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:54 smithi170 ceph-mon[102832]: pgmap v20619: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 770 KiB/s wr, 13 op/s 2023-09-10T09:58:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:58:55.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:55.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38366.244472 secs 2023-09-10T09:58:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38365.892517 secs 2023-09-10T09:58:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:54 smithi082 ceph-mon[98694]: pgmap v20619: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 770 KiB/s wr, 13 op/s 2023-09-10T09:58:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:58:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38366.244472 secs 2023-09-10T09:58:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38365.892517 secs 2023-09-10T09:58:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:54 smithi098 ceph-mon[103124]: pgmap v20619: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 770 KiB/s wr, 13 op/s 2023-09-10T09:58:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:58:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:55.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38366.244472 secs 2023-09-10T09:58:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:56 smithi170 ceph-mon[102832]: pgmap v20620: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:58:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:58:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:56 smithi082 ceph-mon[98694]: pgmap v20620: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:58:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:58:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:56 smithi098 ceph-mon[103124]: pgmap v20620: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:58:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:58:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:58:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:58 smithi170 ceph-mon[102832]: pgmap v20621: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 965 KiB/s wr, 14 op/s 2023-09-10T09:58:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:58 smithi082 ceph-mon[98694]: pgmap v20621: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 965 KiB/s wr, 14 op/s 2023-09-10T09:58:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:58 smithi098 ceph-mon[103124]: pgmap v20621: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 965 KiB/s wr, 14 op/s 2023-09-10T09:59:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38370.892599 secs 2023-09-10T09:59:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:58:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38371.244492 secs 2023-09-10T09:59:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38370.892599 secs 2023-09-10T09:59:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:58:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38371.244492 secs 2023-09-10T09:59:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38370.892599 secs 2023-09-10T09:59:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:58:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38371.244492 secs 2023-09-10T09:59:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:00 smithi170 ceph-mon[102832]: pgmap v20622: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 965 KiB/s wr, 14 op/s 2023-09-10T09:59:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:59:01.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:00 smithi082 ceph-mon[98694]: pgmap v20622: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 965 KiB/s wr, 14 op/s 2023-09-10T09:59:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:59:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:00 smithi098 ceph-mon[103124]: pgmap v20622: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 965 KiB/s wr, 14 op/s 2023-09-10T09:59:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:59:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:02 smithi170 ceph-mon[102832]: pgmap v20623: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 22 op/s 2023-09-10T09:59:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:02 smithi082 ceph-mon[98694]: pgmap v20623: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 22 op/s 2023-09-10T09:59:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:02 smithi098 ceph-mon[103124]: pgmap v20623: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 22 op/s 2023-09-10T09:59:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38374.661444 secs 2023-09-10T09:59:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38374.661444 secs 2023-09-10T09:59:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38374.661444 secs 2023-09-10T09:59:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38375.892654 secs 2023-09-10T09:59:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:04 smithi170 ceph-mon[102832]: pgmap v20624: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 633 KiB/s wr, 15 op/s 2023-09-10T09:59:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:59:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:59:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38376.244554 secs 2023-09-10T09:59:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38375.892654 secs 2023-09-10T09:59:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:04 smithi082 ceph-mon[98694]: pgmap v20624: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 633 KiB/s wr, 15 op/s 2023-09-10T09:59:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:59:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:59:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38376.244554 secs 2023-09-10T09:59:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38375.892654 secs 2023-09-10T09:59:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:04 smithi098 ceph-mon[103124]: pgmap v20624: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 633 KiB/s wr, 15 op/s 2023-09-10T09:59:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:59:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:59:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38376.244554 secs 2023-09-10T09:59:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:06 smithi170 ceph-mon[102832]: pgmap v20625: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 37 op/s 2023-09-10T09:59:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:59:07.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:06 smithi082 ceph-mon[98694]: pgmap v20625: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 37 op/s 2023-09-10T09:59:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:59:07.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:06 smithi098 ceph-mon[103124]: pgmap v20625: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 37 op/s 2023-09-10T09:59:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:59:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:08 smithi170 ceph-mon[102832]: pgmap v20626: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 880 KiB/s wr, 29 op/s 2023-09-10T09:59:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38379.661504 secs 2023-09-10T09:59:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:08 smithi082 ceph-mon[98694]: pgmap v20626: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 880 KiB/s wr, 29 op/s 2023-09-10T09:59:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38379.661504 secs 2023-09-10T09:59:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:08 smithi098 ceph-mon[103124]: pgmap v20626: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 880 KiB/s wr, 29 op/s 2023-09-10T09:59:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38379.661504 secs 2023-09-10T09:59:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38380.892712 secs 2023-09-10T09:59:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:09 smithi170 ceph-mon[102832]: osdmap e1357: 12 total, 12 up, 12 in 2023-09-10T09:59:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:09 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:59:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:09 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38381.244619 secs 2023-09-10T09:59:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38380.892712 secs 2023-09-10T09:59:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:09 smithi082 ceph-mon[98694]: osdmap e1357: 12 total, 12 up, 12 in 2023-09-10T09:59:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:09 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:59:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:09 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38381.244619 secs 2023-09-10T09:59:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38380.892712 secs 2023-09-10T09:59:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:09 smithi098 ceph-mon[103124]: osdmap e1357: 12 total, 12 up, 12 in 2023-09-10T09:59:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:09 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:59:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:09 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38381.244619 secs 2023-09-10T09:59:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:10 smithi170 ceph-mon[102832]: pgmap v20628: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 35 op/s 2023-09-10T09:59:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:10 smithi082 ceph-mon[98694]: pgmap v20628: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 35 op/s 2023-09-10T09:59:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:10 smithi098 ceph-mon[103124]: pgmap v20628: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 35 op/s 2023-09-10T09:59:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:11 smithi170 ceph-mon[102832]: pgmap v20629: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 34 op/s 2023-09-10T09:59:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:11 smithi082 ceph-mon[98694]: pgmap v20629: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 34 op/s 2023-09-10T09:59:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:11 smithi098 ceph-mon[103124]: pgmap v20629: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 34 op/s 2023-09-10T09:59:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:59:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:59:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:59:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38384.661562 secs 2023-09-10T09:59:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:59:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:13 smithi170 ceph-mon[102832]: pgmap v20630: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 34 op/s 2023-09-10T09:59:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38384.661562 secs 2023-09-10T09:59:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:59:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:13 smithi082 ceph-mon[98694]: pgmap v20630: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 34 op/s 2023-09-10T09:59:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38384.661562 secs 2023-09-10T09:59:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T09:59:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:13 smithi098 ceph-mon[103124]: pgmap v20630: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 34 op/s 2023-09-10T09:59:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38385.892772 secs 2023-09-10T09:59:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38386.244686 secs 2023-09-10T09:59:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:59:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:59:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:59:15.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:14 smithi082 ceph-mon[98694]: osdmap e1358: 12 total, 12 up, 12 in 2023-09-10T09:59:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38385.892772 secs 2023-09-10T09:59:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38386.244686 secs 2023-09-10T09:59:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:59:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:59:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:59:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:14 smithi098 ceph-mon[103124]: osdmap e1358: 12 total, 12 up, 12 in 2023-09-10T09:59:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38385.892772 secs 2023-09-10T09:59:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38386.244686 secs 2023-09-10T09:59:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T09:59:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T09:59:15.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T09:59:15.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:14 smithi170 ceph-mon[102832]: osdmap e1358: 12 total, 12 up, 12 in 2023-09-10T09:59:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:15 smithi082 ceph-mon[98694]: pgmap v20632: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 630 KiB/s wr, 18 op/s 2023-09-10T09:59:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:15 smithi098 ceph-mon[103124]: pgmap v20632: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 630 KiB/s wr, 18 op/s 2023-09-10T09:59:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:15 smithi170 ceph-mon[102832]: pgmap v20632: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 630 KiB/s wr, 18 op/s 2023-09-10T09:59:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:18 smithi170 ceph-mon[102832]: pgmap v20633: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 571 KiB/s wr, 16 op/s 2023-09-10T09:59:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38389.661626 secs 2023-09-10T09:59:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:18 smithi082 ceph-mon[98694]: pgmap v20633: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 571 KiB/s wr, 16 op/s 2023-09-10T09:59:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38389.661626 secs 2023-09-10T09:59:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:18 smithi098 ceph-mon[103124]: pgmap v20633: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 571 KiB/s wr, 16 op/s 2023-09-10T09:59:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38389.661626 secs 2023-09-10T09:59:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38390.892829 secs 2023-09-10T09:59:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38391.244754 secs 2023-09-10T09:59:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38390.892829 secs 2023-09-10T09:59:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38391.244754 secs 2023-09-10T09:59:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38390.892829 secs 2023-09-10T09:59:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38391.244754 secs 2023-09-10T09:59:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:20 smithi170 ceph-mon[102832]: pgmap v20634: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 504 KiB/s wr, 14 op/s 2023-09-10T09:59:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:20 smithi082 ceph-mon[98694]: pgmap v20634: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 504 KiB/s wr, 14 op/s 2023-09-10T09:59:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:20 smithi098 ceph-mon[103124]: pgmap v20634: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 504 KiB/s wr, 14 op/s 2023-09-10T09:59:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:22 smithi170 ceph-mon[102832]: pgmap v20635: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 13 op/s 2023-09-10T09:59:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:22 smithi082 ceph-mon[98694]: pgmap v20635: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 13 op/s 2023-09-10T09:59:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:22 smithi098 ceph-mon[103124]: pgmap v20635: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 13 op/s 2023-09-10T09:59:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38394.661737 secs 2023-09-10T09:59:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38394.661737 secs 2023-09-10T09:59:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38394.661737 secs 2023-09-10T09:59:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38395.892912 secs 2023-09-10T09:59:25.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:24 smithi170 ceph-mon[102832]: pgmap v20636: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 13 op/s 2023-09-10T09:59:25.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38396.244816 secs 2023-09-10T09:59:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38395.892912 secs 2023-09-10T09:59:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:24 smithi082 ceph-mon[98694]: pgmap v20636: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 13 op/s 2023-09-10T09:59:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38396.244816 secs 2023-09-10T09:59:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38395.892912 secs 2023-09-10T09:59:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:24 smithi098 ceph-mon[103124]: pgmap v20636: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 169 KiB/s wr, 13 op/s 2023-09-10T09:59:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38396.244816 secs 2023-09-10T09:59:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:26 smithi170 ceph-mon[102832]: pgmap v20637: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 17 op/s 2023-09-10T09:59:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:26 smithi082 ceph-mon[98694]: pgmap v20637: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 17 op/s 2023-09-10T09:59:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:26 smithi098 ceph-mon[103124]: pgmap v20637: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 17 op/s 2023-09-10T09:59:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:28 smithi170 ceph-mon[102832]: pgmap v20638: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T09:59:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38399.661799 secs 2023-09-10T09:59:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:28 smithi082 ceph-mon[98694]: pgmap v20638: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T09:59:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38399.661799 secs 2023-09-10T09:59:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:28 smithi098 ceph-mon[103124]: pgmap v20638: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T09:59:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38399.661799 secs 2023-09-10T09:59:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38400.892969 secs 2023-09-10T09:59:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38401.244882 secs 2023-09-10T09:59:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38400.892969 secs 2023-09-10T09:59:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38401.244882 secs 2023-09-10T09:59:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38400.892969 secs 2023-09-10T09:59:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38401.244882 secs 2023-09-10T09:59:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:30 smithi170 ceph-mon[102832]: pgmap v20639: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T09:59:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:30 smithi082 ceph-mon[98694]: pgmap v20639: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T09:59:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:30 smithi098 ceph-mon[103124]: pgmap v20639: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-09-10T09:59:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:32 smithi170 ceph-mon[102832]: pgmap v20640: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 171 KiB/s wr, 17 op/s 2023-09-10T09:59:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:32 smithi082 ceph-mon[98694]: pgmap v20640: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 171 KiB/s wr, 17 op/s 2023-09-10T09:59:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:32 smithi098 ceph-mon[103124]: pgmap v20640: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 171 KiB/s wr, 17 op/s 2023-09-10T09:59:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38404.661857 secs 2023-09-10T09:59:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38404.661857 secs 2023-09-10T09:59:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38404.661857 secs 2023-09-10T09:59:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38405.893053 secs 2023-09-10T09:59:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:34 smithi170 ceph-mon[102832]: pgmap v20641: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T09:59:35.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38406.244944 secs 2023-09-10T09:59:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38405.893053 secs 2023-09-10T09:59:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:34 smithi082 ceph-mon[98694]: pgmap v20641: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T09:59:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38406.244944 secs 2023-09-10T09:59:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38405.893053 secs 2023-09-10T09:59:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:34 smithi098 ceph-mon[103124]: pgmap v20641: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T09:59:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38406.244944 secs 2023-09-10T09:59:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:36 smithi170 ceph-mon[102832]: pgmap v20642: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 142 KiB/s wr, 16 op/s 2023-09-10T09:59:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:36 smithi082 ceph-mon[98694]: pgmap v20642: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 142 KiB/s wr, 16 op/s 2023-09-10T09:59:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:36 smithi098 ceph-mon[103124]: pgmap v20642: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 142 KiB/s wr, 16 op/s 2023-09-10T09:59:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:38 smithi170 ceph-mon[102832]: pgmap v20643: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T09:59:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:59:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:38 smithi082 ceph-mon[98694]: pgmap v20643: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T09:59:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:59:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:38 smithi098 ceph-mon[103124]: pgmap v20643: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T09:59:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:59:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38410.893111 secs 2023-09-10T09:59:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38411.245009 secs 2023-09-10T09:59:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:59:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38410.893111 secs 2023-09-10T09:59:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38411.245009 secs 2023-09-10T09:59:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:59:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38410.893111 secs 2023-09-10T09:59:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38411.245009 secs 2023-09-10T09:59:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T09:59:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:40 smithi170 ceph-mon[102832]: pgmap v20644: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T09:59:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:59:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:59:41.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:40 smithi082 ceph-mon[98694]: pgmap v20644: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T09:59:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:59:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:59:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:40 smithi098 ceph-mon[103124]: pgmap v20644: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T09:59:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T09:59:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:59:41.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:42 smithi170 ceph-mon[102832]: pgmap v20645: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 481 KiB/s wr, 18 op/s 2023-09-10T09:59:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:42 smithi082 ceph-mon[98694]: pgmap v20645: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 481 KiB/s wr, 18 op/s 2023-09-10T09:59:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:42 smithi098 ceph-mon[103124]: pgmap v20645: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 481 KiB/s wr, 18 op/s 2023-09-10T09:59:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38415.893177 secs 2023-09-10T09:59:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:44 smithi170 ceph-mon[102832]: pgmap v20646: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 339 KiB/s wr, 12 op/s 2023-09-10T09:59:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38416.245078 secs 2023-09-10T09:59:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38415.893177 secs 2023-09-10T09:59:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:44 smithi082 ceph-mon[98694]: pgmap v20646: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 339 KiB/s wr, 12 op/s 2023-09-10T09:59:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38416.245078 secs 2023-09-10T09:59:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38415.893177 secs 2023-09-10T09:59:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:44 smithi098 ceph-mon[103124]: pgmap v20646: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 339 KiB/s wr, 12 op/s 2023-09-10T09:59:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38416.245078 secs 2023-09-10T09:59:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:46 smithi170 ceph-mon[102832]: pgmap v20647: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 674 KiB/s wr, 19 op/s 2023-09-10T09:59:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:46 smithi082 ceph-mon[98694]: pgmap v20647: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 674 KiB/s wr, 19 op/s 2023-09-10T09:59:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:46 smithi098 ceph-mon[103124]: pgmap v20647: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 674 KiB/s wr, 19 op/s 2023-09-10T09:59:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:48 smithi170 ceph-mon[102832]: pgmap v20648: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 674 KiB/s wr, 14 op/s 2023-09-10T09:59:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:48 smithi082 ceph-mon[98694]: pgmap v20648: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 674 KiB/s wr, 14 op/s 2023-09-10T09:59:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:48 smithi098 ceph-mon[103124]: pgmap v20648: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 674 KiB/s wr, 14 op/s 2023-09-10T09:59:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38420.893264 secs 2023-09-10T09:59:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38421.245144 secs 2023-09-10T09:59:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38420.893264 secs 2023-09-10T09:59:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38421.245144 secs 2023-09-10T09:59:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38420.893264 secs 2023-09-10T09:59:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38421.245144 secs 2023-09-10T09:59:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:50 smithi170 ceph-mon[102832]: pgmap v20649: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 674 KiB/s wr, 14 op/s 2023-09-10T09:59:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:59:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:50 smithi082 ceph-mon[98694]: pgmap v20649: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 674 KiB/s wr, 14 op/s 2023-09-10T09:59:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:59:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:50 smithi098 ceph-mon[103124]: pgmap v20649: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 674 KiB/s wr, 14 op/s 2023-09-10T09:59:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:59:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:52 smithi170 ceph-mon[102832]: pgmap v20650: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T09:59:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:59:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:52 smithi082 ceph-mon[98694]: pgmap v20650: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T09:59:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:59:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:52 smithi098 ceph-mon[103124]: pgmap v20650: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T09:59:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:59:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38425.893323 secs 2023-09-10T09:59:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:54 smithi170 ceph-mon[102832]: pgmap v20651: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 705 KiB/s wr, 14 op/s 2023-09-10T09:59:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:59:55.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:55.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38426.245207 secs 2023-09-10T09:59:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38425.893323 secs 2023-09-10T09:59:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:54 smithi082 ceph-mon[98694]: pgmap v20651: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 705 KiB/s wr, 14 op/s 2023-09-10T09:59:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:59:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38426.245207 secs 2023-09-10T09:59:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38425.893323 secs 2023-09-10T09:59:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:54 smithi098 ceph-mon[103124]: pgmap v20651: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 705 KiB/s wr, 14 op/s 2023-09-10T09:59:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:59:55.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:55.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38426.245207 secs 2023-09-10T09:59:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:56 smithi170 ceph-mon[102832]: pgmap v20652: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:59:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:59:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:56 smithi082 ceph-mon[98694]: pgmap v20652: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:59:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:59:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:56 smithi098 ceph-mon[103124]: pgmap v20652: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T09:59:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T09:59:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:58 smithi170 ceph-mon[102832]: pgmap v20653: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 741 KiB/s wr, 14 op/s 2023-09-10T09:59:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:59:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:59.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38429.662156 secs 2023-09-10T09:59:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:58 smithi082 ceph-mon[98694]: pgmap v20653: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 741 KiB/s wr, 14 op/s 2023-09-10T09:59:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:59:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38429.662156 secs 2023-09-10T09:59:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:58 smithi098 ceph-mon[103124]: pgmap v20653: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 741 KiB/s wr, 14 op/s 2023-09-10T09:59:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T09:59:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T09:59:59.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38429.662156 secs 2023-09-10T10:00:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38430.893405 secs 2023-09-10T10:00:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 09:59:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38431.245272 secs 2023-09-10T10:00:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38430.893405 secs 2023-09-10T10:00:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 09:59:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38431.245272 secs 2023-09-10T10:00:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38430.893405 secs 2023-09-10T10:00:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 09:59:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38431.245272 secs 2023-09-10T10:00:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:00 smithi170 ceph-mon[102832]: pgmap v20654: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 741 KiB/s wr, 14 op/s 2023-09-10T10:00:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:00:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:00:01.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:01.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:00 smithi170 ceph-mon[102832]: Health detail: HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T10:00:01.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:00 smithi170 ceph-mon[102832]: [WRN] MDS_SLOW_REQUEST: 3 MDSs report slow requests 2023-09-10T10:00:01.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:00 smithi170 ceph-mon[102832]: mds.l(mds.4): 3 slow requests are blocked > 180 secs 2023-09-10T10:00:01.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:00 smithi170 ceph-mon[102832]: mds.h(mds.1): 3 slow requests are blocked > 180 secs 2023-09-10T10:00:01.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:00 smithi170 ceph-mon[102832]: mds.b(mds.0): 1 slow requests are blocked > 180 secs 2023-09-10T10:00:01.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:00 smithi170 ceph-mon[102832]: [WRN] MDS_TRIM: 2 MDSs behind on trimming 2023-09-10T10:00:01.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:00 smithi170 ceph-mon[102832]: mds.e(mds.0): Behind on trimming (982/128) max_segments: 128, num_segments: 982 2023-09-10T10:00:01.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:00 smithi170 ceph-mon[102832]: mds.b(mds.0): Behind on trimming (982/128) max_segments: 128, num_segments: 982 2023-09-10T10:00:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:00 smithi082 ceph-mon[98694]: pgmap v20654: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 741 KiB/s wr, 14 op/s 2023-09-10T10:00:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:00:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:00:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:00 smithi082 ceph-mon[98694]: Health detail: HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T10:00:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:00 smithi082 ceph-mon[98694]: [WRN] MDS_SLOW_REQUEST: 3 MDSs report slow requests 2023-09-10T10:00:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:00 smithi082 ceph-mon[98694]: mds.l(mds.4): 3 slow requests are blocked > 180 secs 2023-09-10T10:00:01.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:00 smithi082 ceph-mon[98694]: mds.h(mds.1): 3 slow requests are blocked > 180 secs 2023-09-10T10:00:01.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:00 smithi082 ceph-mon[98694]: mds.b(mds.0): 1 slow requests are blocked > 180 secs 2023-09-10T10:00:01.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:00 smithi082 ceph-mon[98694]: [WRN] MDS_TRIM: 2 MDSs behind on trimming 2023-09-10T10:00:01.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:00 smithi082 ceph-mon[98694]: mds.e(mds.0): Behind on trimming (982/128) max_segments: 128, num_segments: 982 2023-09-10T10:00:01.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:00 smithi082 ceph-mon[98694]: mds.b(mds.0): Behind on trimming (982/128) max_segments: 128, num_segments: 982 2023-09-10T10:00:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:00 smithi098 ceph-mon[103124]: pgmap v20654: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 741 KiB/s wr, 14 op/s 2023-09-10T10:00:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:00:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:00:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:00 smithi098 ceph-mon[103124]: Health detail: HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T10:00:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:00 smithi098 ceph-mon[103124]: [WRN] MDS_SLOW_REQUEST: 3 MDSs report slow requests 2023-09-10T10:00:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:00 smithi098 ceph-mon[103124]: mds.l(mds.4): 3 slow requests are blocked > 180 secs 2023-09-10T10:00:01.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:00 smithi098 ceph-mon[103124]: mds.h(mds.1): 3 slow requests are blocked > 180 secs 2023-09-10T10:00:01.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:00 smithi098 ceph-mon[103124]: mds.b(mds.0): 1 slow requests are blocked > 180 secs 2023-09-10T10:00:01.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:00 smithi098 ceph-mon[103124]: [WRN] MDS_TRIM: 2 MDSs behind on trimming 2023-09-10T10:00:01.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:00 smithi098 ceph-mon[103124]: mds.e(mds.0): Behind on trimming (982/128) max_segments: 128, num_segments: 982 2023-09-10T10:00:01.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:00 smithi098 ceph-mon[103124]: mds.b(mds.0): Behind on trimming (982/128) max_segments: 128, num_segments: 982 2023-09-10T10:00:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:02 smithi170 ceph-mon[102832]: pgmap v20655: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 982 KiB/s wr, 21 op/s 2023-09-10T10:00:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:02 smithi082 ceph-mon[98694]: pgmap v20655: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 982 KiB/s wr, 21 op/s 2023-09-10T10:00:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:02 smithi098 ceph-mon[103124]: pgmap v20655: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 982 KiB/s wr, 21 op/s 2023-09-10T10:00:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38434.662216 secs 2023-09-10T10:00:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38434.662216 secs 2023-09-10T10:00:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38434.662216 secs 2023-09-10T10:00:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38435.893463 secs 2023-09-10T10:00:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:04 smithi170 ceph-mon[102832]: pgmap v20656: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 612 KiB/s wr, 14 op/s 2023-09-10T10:00:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:00:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:04 smithi170 ceph-mon[102832]: osdmap e1359: 12 total, 12 up, 12 in 2023-09-10T10:00:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38436.245338 secs 2023-09-10T10:00:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38435.893463 secs 2023-09-10T10:00:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:04 smithi082 ceph-mon[98694]: pgmap v20656: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 612 KiB/s wr, 14 op/s 2023-09-10T10:00:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:00:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:04 smithi082 ceph-mon[98694]: osdmap e1359: 12 total, 12 up, 12 in 2023-09-10T10:00:05.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38436.245338 secs 2023-09-10T10:00:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38435.893463 secs 2023-09-10T10:00:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:04 smithi098 ceph-mon[103124]: pgmap v20656: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 612 KiB/s wr, 14 op/s 2023-09-10T10:00:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:00:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:04 smithi098 ceph-mon[103124]: osdmap e1359: 12 total, 12 up, 12 in 2023-09-10T10:00:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38436.245338 secs 2023-09-10T10:00:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:06 smithi170 ceph-mon[102832]: pgmap v20658: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 747 KiB/s wr, 33 op/s 2023-09-10T10:00:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:06 smithi082 ceph-mon[98694]: pgmap v20658: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 747 KiB/s wr, 33 op/s 2023-09-10T10:00:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:06 smithi098 ceph-mon[103124]: pgmap v20658: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 747 KiB/s wr, 33 op/s 2023-09-10T10:00:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:08 smithi170 ceph-mon[102832]: pgmap v20659: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 747 KiB/s wr, 33 op/s 2023-09-10T10:00:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38439.662290 secs 2023-09-10T10:00:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:08 smithi082 ceph-mon[98694]: pgmap v20659: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 747 KiB/s wr, 33 op/s 2023-09-10T10:00:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38439.662290 secs 2023-09-10T10:00:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:08 smithi098 ceph-mon[103124]: pgmap v20659: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 747 KiB/s wr, 33 op/s 2023-09-10T10:00:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38439.662290 secs 2023-09-10T10:00:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38440.893526 secs 2023-09-10T10:00:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:09 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:00:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:09 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38441.245400 secs 2023-09-10T10:00:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:09 smithi082 ceph-mon[98694]: osdmap e1360: 12 total, 12 up, 12 in 2023-09-10T10:00:10.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:09 smithi082 ceph-mon[98694]: pgmap v20661: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 572 KiB/s wr, 31 op/s 2023-09-10T10:00:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38440.893526 secs 2023-09-10T10:00:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:09 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:00:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:09 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38441.245400 secs 2023-09-10T10:00:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:09 smithi098 ceph-mon[103124]: osdmap e1360: 12 total, 12 up, 12 in 2023-09-10T10:00:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:09 smithi098 ceph-mon[103124]: pgmap v20661: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 572 KiB/s wr, 31 op/s 2023-09-10T10:00:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38440.893526 secs 2023-09-10T10:00:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:09 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:00:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:09 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38441.245400 secs 2023-09-10T10:00:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:09 smithi170 ceph-mon[102832]: osdmap e1360: 12 total, 12 up, 12 in 2023-09-10T10:00:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:09 smithi170 ceph-mon[102832]: pgmap v20661: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 572 KiB/s wr, 31 op/s 2023-09-10T10:00:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:12 smithi170 ceph-mon[102832]: pgmap v20662: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 615 KiB/s wr, 40 op/s 2023-09-10T10:00:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:12 smithi082 ceph-mon[98694]: pgmap v20662: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 615 KiB/s wr, 40 op/s 2023-09-10T10:00:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:12 smithi098 ceph-mon[103124]: pgmap v20662: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 615 KiB/s wr, 40 op/s 2023-09-10T10:00:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38444.662349 secs 2023-09-10T10:00:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38444.662349 secs 2023-09-10T10:00:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38444.662349 secs 2023-09-10T10:00:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38445.893589 secs 2023-09-10T10:00:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:14 smithi170 ceph-mon[102832]: pgmap v20663: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 498 KiB/s wr, 32 op/s 2023-09-10T10:00:14.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38446.245474 secs 2023-09-10T10:00:14.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:00:14.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38445.893589 secs 2023-09-10T10:00:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:14 smithi082 ceph-mon[98694]: pgmap v20663: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 498 KiB/s wr, 32 op/s 2023-09-10T10:00:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38446.245474 secs 2023-09-10T10:00:14.960 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:00:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38445.893589 secs 2023-09-10T10:00:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:14 smithi098 ceph-mon[103124]: pgmap v20663: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 498 KiB/s wr, 32 op/s 2023-09-10T10:00:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38446.245474 secs 2023-09-10T10:00:15.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:00:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:00:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:00:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:00:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:00:16.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:00:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:00:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:00:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:00:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:00:16.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:00:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:00:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:00:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:00:16.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:00:16.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:00:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:16 smithi170 ceph-mon[102832]: pgmap v20664: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 316 KiB/s wr, 14 op/s 2023-09-10T10:00:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:16 smithi082 ceph-mon[98694]: pgmap v20664: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 316 KiB/s wr, 14 op/s 2023-09-10T10:00:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:16 smithi098 ceph-mon[103124]: pgmap v20664: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 316 KiB/s wr, 14 op/s 2023-09-10T10:00:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:18 smithi170 ceph-mon[102832]: pgmap v20665: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 316 KiB/s wr, 14 op/s 2023-09-10T10:00:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:00:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:19.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38449.662439 secs 2023-09-10T10:00:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:18 smithi082 ceph-mon[98694]: pgmap v20665: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 316 KiB/s wr, 14 op/s 2023-09-10T10:00:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:00:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38449.662439 secs 2023-09-10T10:00:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:18 smithi098 ceph-mon[103124]: pgmap v20665: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 316 KiB/s wr, 14 op/s 2023-09-10T10:00:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:00:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:19.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38449.662439 secs 2023-09-10T10:00:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38450.893651 secs 2023-09-10T10:00:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38451.245487 secs 2023-09-10T10:00:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38450.893651 secs 2023-09-10T10:00:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38451.245487 secs 2023-09-10T10:00:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38450.893651 secs 2023-09-10T10:00:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38451.245487 secs 2023-09-10T10:00:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:20 smithi170 ceph-mon[102832]: pgmap v20666: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 308 KiB/s wr, 14 op/s 2023-09-10T10:00:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:00:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:20 smithi082 ceph-mon[98694]: pgmap v20666: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 308 KiB/s wr, 14 op/s 2023-09-10T10:00:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:00:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:20 smithi098 ceph-mon[103124]: pgmap v20666: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 308 KiB/s wr, 14 op/s 2023-09-10T10:00:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:00:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:22 smithi170 ceph-mon[102832]: pgmap v20667: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 18 op/s 2023-09-10T10:00:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:22 smithi082 ceph-mon[98694]: pgmap v20667: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 18 op/s 2023-09-10T10:00:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:22 smithi098 ceph-mon[103124]: pgmap v20667: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 264 KiB/s wr, 18 op/s 2023-09-10T10:00:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38454.662524 secs 2023-09-10T10:00:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38454.662524 secs 2023-09-10T10:00:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38454.662524 secs 2023-09-10T10:00:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38455.893713 secs 2023-09-10T10:00:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:24 smithi170 ceph-mon[102832]: pgmap v20668: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 235 KiB/s wr, 12 op/s 2023-09-10T10:00:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38456.245553 secs 2023-09-10T10:00:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38455.893713 secs 2023-09-10T10:00:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:24 smithi082 ceph-mon[98694]: pgmap v20668: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 235 KiB/s wr, 12 op/s 2023-09-10T10:00:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38456.245553 secs 2023-09-10T10:00:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38455.893713 secs 2023-09-10T10:00:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:24 smithi098 ceph-mon[103124]: pgmap v20668: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 235 KiB/s wr, 12 op/s 2023-09-10T10:00:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38456.245553 secs 2023-09-10T10:00:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:26 smithi170 ceph-mon[102832]: pgmap v20669: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 376 KiB/s wr, 17 op/s 2023-09-10T10:00:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:26 smithi082 ceph-mon[98694]: pgmap v20669: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 376 KiB/s wr, 17 op/s 2023-09-10T10:00:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:26 smithi098 ceph-mon[103124]: pgmap v20669: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 376 KiB/s wr, 17 op/s 2023-09-10T10:00:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:28 smithi170 ceph-mon[102832]: pgmap v20670: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T10:00:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38459.662617 secs 2023-09-10T10:00:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:28 smithi082 ceph-mon[98694]: pgmap v20670: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T10:00:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38459.662617 secs 2023-09-10T10:00:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:28 smithi098 ceph-mon[103124]: pgmap v20670: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T10:00:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38459.662617 secs 2023-09-10T10:00:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38460.893807 secs 2023-09-10T10:00:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38461.245619 secs 2023-09-10T10:00:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38460.893807 secs 2023-09-10T10:00:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38461.245619 secs 2023-09-10T10:00:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38460.893807 secs 2023-09-10T10:00:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38461.245619 secs 2023-09-10T10:00:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:30 smithi170 ceph-mon[102832]: pgmap v20671: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T10:00:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:30 smithi082 ceph-mon[98694]: pgmap v20671: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T10:00:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:30 smithi098 ceph-mon[103124]: pgmap v20671: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 11 op/s 2023-09-10T10:00:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:32 smithi170 ceph-mon[102832]: pgmap v20672: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 142 KiB/s wr, 16 op/s 2023-09-10T10:00:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:32 smithi082 ceph-mon[98694]: pgmap v20672: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 142 KiB/s wr, 16 op/s 2023-09-10T10:00:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:32 smithi098 ceph-mon[103124]: pgmap v20672: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 142 KiB/s wr, 16 op/s 2023-09-10T10:00:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38464.662678 secs 2023-09-10T10:00:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38464.662678 secs 2023-09-10T10:00:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38464.662678 secs 2023-09-10T10:00:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38465.893870 secs 2023-09-10T10:00:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:34 smithi170 ceph-mon[102832]: pgmap v20673: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 10 op/s 2023-09-10T10:00:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38466.245707 secs 2023-09-10T10:00:35.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38465.893870 secs 2023-09-10T10:00:35.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:34 smithi082 ceph-mon[98694]: pgmap v20673: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 10 op/s 2023-09-10T10:00:35.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38466.245707 secs 2023-09-10T10:00:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38465.893870 secs 2023-09-10T10:00:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:34 smithi098 ceph-mon[103124]: pgmap v20673: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 10 op/s 2023-09-10T10:00:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38466.245707 secs 2023-09-10T10:00:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:36 smithi170 ceph-mon[102832]: pgmap v20674: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 283 KiB/s wr, 15 op/s 2023-09-10T10:00:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:36 smithi082 ceph-mon[98694]: pgmap v20674: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 283 KiB/s wr, 15 op/s 2023-09-10T10:00:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:36 smithi098 ceph-mon[103124]: pgmap v20674: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 283 KiB/s wr, 15 op/s 2023-09-10T10:00:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:37 smithi098 ceph-mon[103124]: pgmap v20675: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 10 op/s 2023-09-10T10:00:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:37 smithi170 ceph-mon[102832]: pgmap v20675: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 10 op/s 2023-09-10T10:00:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:37 smithi082 ceph-mon[98694]: pgmap v20675: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 10 op/s 2023-09-10T10:00:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:00:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:00:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:00:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38470.893955 secs 2023-09-10T10:00:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38471.245772 secs 2023-09-10T10:00:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:00:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:00:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:40 smithi098 ceph-mon[103124]: pgmap v20676: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 10 op/s 2023-09-10T10:00:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38470.893955 secs 2023-09-10T10:00:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38471.245772 secs 2023-09-10T10:00:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:00:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:00:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:40 smithi170 ceph-mon[102832]: pgmap v20676: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 10 op/s 2023-09-10T10:00:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38470.893955 secs 2023-09-10T10:00:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38471.245772 secs 2023-09-10T10:00:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:00:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:00:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:40 smithi082 ceph-mon[98694]: pgmap v20676: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 10 op/s 2023-09-10T10:00:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:42 smithi170 ceph-mon[102832]: pgmap v20677: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 281 KiB/s wr, 16 op/s 2023-09-10T10:00:43.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:42 smithi082 ceph-mon[98694]: pgmap v20677: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 281 KiB/s wr, 16 op/s 2023-09-10T10:00:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:42 smithi098 ceph-mon[103124]: pgmap v20677: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 281 KiB/s wr, 16 op/s 2023-09-10T10:00:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38475.894014 secs 2023-09-10T10:00:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:44 smithi170 ceph-mon[102832]: pgmap v20678: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 281 KiB/s wr, 11 op/s 2023-09-10T10:00:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38476.245835 secs 2023-09-10T10:00:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38475.894014 secs 2023-09-10T10:00:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:44 smithi082 ceph-mon[98694]: pgmap v20678: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 281 KiB/s wr, 11 op/s 2023-09-10T10:00:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38476.245835 secs 2023-09-10T10:00:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38475.894014 secs 2023-09-10T10:00:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:44 smithi098 ceph-mon[103124]: pgmap v20678: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 281 KiB/s wr, 11 op/s 2023-09-10T10:00:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38476.245835 secs 2023-09-10T10:00:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:46 smithi170 ceph-mon[102832]: pgmap v20679: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 584 KiB/s wr, 17 op/s 2023-09-10T10:00:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:00:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:46 smithi082 ceph-mon[98694]: pgmap v20679: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 584 KiB/s wr, 17 op/s 2023-09-10T10:00:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:00:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:46 smithi098 ceph-mon[103124]: pgmap v20679: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 584 KiB/s wr, 17 op/s 2023-09-10T10:00:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:00:47.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:48 smithi170 ceph-mon[102832]: pgmap v20680: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 443 KiB/s wr, 11 op/s 2023-09-10T10:00:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:48 smithi082 ceph-mon[98694]: pgmap v20680: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 443 KiB/s wr, 11 op/s 2023-09-10T10:00:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:48 smithi098 ceph-mon[103124]: pgmap v20680: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 443 KiB/s wr, 11 op/s 2023-09-10T10:00:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38480.894084 secs 2023-09-10T10:00:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38481.245899 secs 2023-09-10T10:00:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38480.894084 secs 2023-09-10T10:00:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38481.245899 secs 2023-09-10T10:00:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38480.894084 secs 2023-09-10T10:00:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38481.245899 secs 2023-09-10T10:00:51.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:50 smithi170 ceph-mon[102832]: pgmap v20681: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 443 KiB/s wr, 11 op/s 2023-09-10T10:00:51.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:00:51.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:50 smithi082 ceph-mon[98694]: pgmap v20681: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 443 KiB/s wr, 11 op/s 2023-09-10T10:00:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:00:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:50 smithi098 ceph-mon[103124]: pgmap v20681: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 443 KiB/s wr, 11 op/s 2023-09-10T10:00:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:00:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:52 smithi170 ceph-mon[102832]: pgmap v20682: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 500 KiB/s wr, 18 op/s 2023-09-10T10:00:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:00:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:52 smithi082 ceph-mon[98694]: pgmap v20682: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 500 KiB/s wr, 18 op/s 2023-09-10T10:00:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:00:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:52 smithi098 ceph-mon[103124]: pgmap v20682: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 500 KiB/s wr, 18 op/s 2023-09-10T10:00:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:00:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38485.894166 secs 2023-09-10T10:00:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:54 smithi170 ceph-mon[102832]: pgmap v20683: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 360 KiB/s wr, 13 op/s 2023-09-10T10:00:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:00:55.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:55.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38486.245960 secs 2023-09-10T10:00:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38485.894166 secs 2023-09-10T10:00:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:54 smithi082 ceph-mon[98694]: pgmap v20683: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 360 KiB/s wr, 13 op/s 2023-09-10T10:00:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:00:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38486.245960 secs 2023-09-10T10:00:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38485.894166 secs 2023-09-10T10:00:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:54 smithi098 ceph-mon[103124]: pgmap v20683: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 360 KiB/s wr, 13 op/s 2023-09-10T10:00:55.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:00:55.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:55.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38486.245960 secs 2023-09-10T10:00:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:56 smithi170 ceph-mon[102832]: pgmap v20684: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 862 KiB/s wr, 20 op/s 2023-09-10T10:00:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:00:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:56 smithi082 ceph-mon[98694]: pgmap v20684: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 862 KiB/s wr, 20 op/s 2023-09-10T10:00:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:00:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:56 smithi098 ceph-mon[103124]: pgmap v20684: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 862 KiB/s wr, 20 op/s 2023-09-10T10:00:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:00:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:00:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:58 smithi170 ceph-mon[102832]: pgmap v20685: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 559 KiB/s wr, 14 op/s 2023-09-10T10:00:59.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:58 smithi082 ceph-mon[98694]: pgmap v20685: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 559 KiB/s wr, 14 op/s 2023-09-10T10:00:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:58 smithi098 ceph-mon[103124]: pgmap v20685: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 559 KiB/s wr, 14 op/s 2023-09-10T10:01:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38490.894248 secs 2023-09-10T10:01:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:00:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38491.246026 secs 2023-09-10T10:01:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38490.894248 secs 2023-09-10T10:01:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:00:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38491.246026 secs 2023-09-10T10:01:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38490.894248 secs 2023-09-10T10:01:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:00:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38491.246026 secs 2023-09-10T10:01:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:00 smithi170 ceph-mon[102832]: pgmap v20686: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 559 KiB/s wr, 14 op/s 2023-09-10T10:01:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:00 smithi082 ceph-mon[98694]: pgmap v20686: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 559 KiB/s wr, 14 op/s 2023-09-10T10:01:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:00 smithi098 ceph-mon[103124]: pgmap v20686: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 559 KiB/s wr, 14 op/s 2023-09-10T10:01:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:02 smithi170 ceph-mon[102832]: pgmap v20687: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 657 KiB/s wr, 22 op/s 2023-09-10T10:01:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:02 smithi082 ceph-mon[98694]: pgmap v20687: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 657 KiB/s wr, 22 op/s 2023-09-10T10:01:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:02 smithi098 ceph-mon[103124]: pgmap v20687: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 657 KiB/s wr, 22 op/s 2023-09-10T10:01:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38495.894305 secs 2023-09-10T10:01:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:04 smithi170 ceph-mon[102832]: pgmap v20688: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 600 KiB/s wr, 15 op/s 2023-09-10T10:01:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:01:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38496.246091 secs 2023-09-10T10:01:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38495.894305 secs 2023-09-10T10:01:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:04 smithi082 ceph-mon[98694]: pgmap v20688: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 600 KiB/s wr, 15 op/s 2023-09-10T10:01:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:01:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38496.246091 secs 2023-09-10T10:01:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38495.894305 secs 2023-09-10T10:01:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:04 smithi098 ceph-mon[103124]: pgmap v20688: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 600 KiB/s wr, 15 op/s 2023-09-10T10:01:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:01:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38496.246091 secs 2023-09-10T10:01:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:06 smithi170 ceph-mon[102832]: pgmap v20689: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 33 op/s 2023-09-10T10:01:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:01:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:06 smithi082 ceph-mon[98694]: pgmap v20689: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 33 op/s 2023-09-10T10:01:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:01:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:06 smithi098 ceph-mon[103124]: pgmap v20689: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 33 op/s 2023-09-10T10:01:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:01:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:08 smithi170 ceph-mon[102832]: pgmap v20690: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 572 KiB/s wr, 25 op/s 2023-09-10T10:01:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:08 smithi082 ceph-mon[98694]: pgmap v20690: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 572 KiB/s wr, 25 op/s 2023-09-10T10:01:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:08 smithi098 ceph-mon[103124]: pgmap v20690: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 572 KiB/s wr, 25 op/s 2023-09-10T10:01:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38500.894394 secs 2023-09-10T10:01:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38501.246157 secs 2023-09-10T10:01:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38500.894394 secs 2023-09-10T10:01:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38501.246157 secs 2023-09-10T10:01:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38500.894394 secs 2023-09-10T10:01:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38501.246157 secs 2023-09-10T10:01:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:10 smithi170 ceph-mon[102832]: pgmap v20691: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 572 KiB/s wr, 25 op/s 2023-09-10T10:01:11.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:01:11.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:11.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:10 smithi082 ceph-mon[98694]: pgmap v20691: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 572 KiB/s wr, 25 op/s 2023-09-10T10:01:11.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:01:11.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:10 smithi098 ceph-mon[103124]: pgmap v20691: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 572 KiB/s wr, 25 op/s 2023-09-10T10:01:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:01:11.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:12 smithi170 ceph-mon[102832]: pgmap v20692: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 851 KiB/s wr, 31 op/s 2023-09-10T10:01:13.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:01:13.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:12 smithi082 ceph-mon[98694]: pgmap v20692: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 851 KiB/s wr, 31 op/s 2023-09-10T10:01:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:01:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:12 smithi098 ceph-mon[103124]: pgmap v20692: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 851 KiB/s wr, 31 op/s 2023-09-10T10:01:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:01:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38505.894463 secs 2023-09-10T10:01:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:14 smithi170 ceph-mon[102832]: pgmap v20693: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 753 KiB/s wr, 24 op/s 2023-09-10T10:01:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38506.246223 secs 2023-09-10T10:01:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38505.894463 secs 2023-09-10T10:01:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:14 smithi082 ceph-mon[98694]: pgmap v20693: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 753 KiB/s wr, 24 op/s 2023-09-10T10:01:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38506.246223 secs 2023-09-10T10:01:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38505.894463 secs 2023-09-10T10:01:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:14 smithi098 ceph-mon[103124]: pgmap v20693: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 753 KiB/s wr, 24 op/s 2023-09-10T10:01:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38506.246223 secs 2023-09-10T10:01:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:01:16.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:01:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:01:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:16 smithi170 ceph-mon[102832]: pgmap v20694: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 890 KiB/s wr, 30 op/s 2023-09-10T10:01:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:01:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:01:17.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:01:17.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:01:17.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:01:17.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:01:17.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:01:17.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:16 smithi082 ceph-mon[98694]: pgmap v20694: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 890 KiB/s wr, 30 op/s 2023-09-10T10:01:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:01:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:01:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:01:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:01:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:01:17.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:01:17.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:01:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:16 smithi098 ceph-mon[103124]: pgmap v20694: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 890 KiB/s wr, 30 op/s 2023-09-10T10:01:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:01:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:01:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:01:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:01:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:01:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:01:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:01:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:18 smithi170 ceph-mon[102832]: pgmap v20695: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 416 KiB/s wr, 12 op/s 2023-09-10T10:01:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:18 smithi082 ceph-mon[98694]: pgmap v20695: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 416 KiB/s wr, 12 op/s 2023-09-10T10:01:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:18 smithi098 ceph-mon[103124]: pgmap v20695: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 416 KiB/s wr, 12 op/s 2023-09-10T10:01:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38510.894521 secs 2023-09-10T10:01:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38511.246290 secs 2023-09-10T10:01:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38510.894521 secs 2023-09-10T10:01:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38511.246290 secs 2023-09-10T10:01:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38510.894521 secs 2023-09-10T10:01:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38511.246290 secs 2023-09-10T10:01:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:20 smithi082 ceph-mon[98694]: pgmap v20696: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 416 KiB/s wr, 12 op/s 2023-09-10T10:01:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:01:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:20 smithi098 ceph-mon[103124]: pgmap v20696: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 416 KiB/s wr, 12 op/s 2023-09-10T10:01:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:01:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:20 smithi170 ceph-mon[102832]: pgmap v20696: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 416 KiB/s wr, 12 op/s 2023-09-10T10:01:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:01:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:22 smithi082 ceph-mon[98694]: pgmap v20697: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 770 KiB/s wr, 21 op/s 2023-09-10T10:01:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:22 smithi098 ceph-mon[103124]: pgmap v20697: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 770 KiB/s wr, 21 op/s 2023-09-10T10:01:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:22 smithi170 ceph-mon[102832]: pgmap v20697: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 770 KiB/s wr, 21 op/s 2023-09-10T10:01:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38514.663541 secs 2023-09-10T10:01:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38514.663541 secs 2023-09-10T10:01:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38514.663541 secs 2023-09-10T10:01:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38515.894584 secs 2023-09-10T10:01:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:24 smithi082 ceph-mon[98694]: pgmap v20698: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 491 KiB/s wr, 15 op/s 2023-09-10T10:01:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:24 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:01:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:24 smithi082 ceph-mon[98694]: osdmap e1361: 12 total, 12 up, 12 in 2023-09-10T10:01:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:01:25.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:25.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38516.246359 secs 2023-09-10T10:01:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38515.894584 secs 2023-09-10T10:01:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:24 smithi098 ceph-mon[103124]: pgmap v20698: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 491 KiB/s wr, 15 op/s 2023-09-10T10:01:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:24 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:01:25.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:24 smithi098 ceph-mon[103124]: osdmap e1361: 12 total, 12 up, 12 in 2023-09-10T10:01:25.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:01:25.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:25.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38516.246359 secs 2023-09-10T10:01:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38515.894584 secs 2023-09-10T10:01:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:24 smithi170 ceph-mon[102832]: pgmap v20698: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 491 KiB/s wr, 15 op/s 2023-09-10T10:01:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:24 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:01:25.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:24 smithi170 ceph-mon[102832]: osdmap e1361: 12 total, 12 up, 12 in 2023-09-10T10:01:25.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:01:25.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:25.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38516.246359 secs 2023-09-10T10:01:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:26 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:01:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:26 smithi170 ceph-mon[102832]: pgmap v20700: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 426 KiB/s wr, 18 op/s 2023-09-10T10:01:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:26 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:01:26.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:26.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:26 smithi082 ceph-mon[98694]: pgmap v20700: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 426 KiB/s wr, 18 op/s 2023-09-10T10:01:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:26 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:01:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:26 smithi098 ceph-mon[103124]: pgmap v20700: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 426 KiB/s wr, 18 op/s 2023-09-10T10:01:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:28 smithi170 ceph-mon[102832]: pgmap v20701: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 426 KiB/s wr, 18 op/s 2023-09-10T10:01:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38519.663630 secs 2023-09-10T10:01:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:28 smithi082 ceph-mon[98694]: pgmap v20701: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 426 KiB/s wr, 18 op/s 2023-09-10T10:01:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38519.663630 secs 2023-09-10T10:01:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:28 smithi098 ceph-mon[103124]: pgmap v20701: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 426 KiB/s wr, 18 op/s 2023-09-10T10:01:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38519.663630 secs 2023-09-10T10:01:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38520.894643 secs 2023-09-10T10:01:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38521.246423 secs 2023-09-10T10:01:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:29 smithi170 ceph-mon[102832]: osdmap e1362: 12 total, 12 up, 12 in 2023-09-10T10:01:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38520.894643 secs 2023-09-10T10:01:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38521.246423 secs 2023-09-10T10:01:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:29 smithi082 ceph-mon[98694]: osdmap e1362: 12 total, 12 up, 12 in 2023-09-10T10:01:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38520.894643 secs 2023-09-10T10:01:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38521.246423 secs 2023-09-10T10:01:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:29 smithi098 ceph-mon[103124]: osdmap e1362: 12 total, 12 up, 12 in 2023-09-10T10:01:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:30 smithi170 ceph-mon[102832]: pgmap v20703: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2023-09-10T10:01:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:30 smithi082 ceph-mon[98694]: pgmap v20703: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2023-09-10T10:01:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:30 smithi098 ceph-mon[103124]: pgmap v20703: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2023-09-10T10:01:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:32 smithi170 ceph-mon[102832]: pgmap v20704: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 537 KiB/s wr, 19 op/s 2023-09-10T10:01:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:32 smithi082 ceph-mon[98694]: pgmap v20704: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 537 KiB/s wr, 19 op/s 2023-09-10T10:01:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:32 smithi098 ceph-mon[103124]: pgmap v20704: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 537 KiB/s wr, 19 op/s 2023-09-10T10:01:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38524.663690 secs 2023-09-10T10:01:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38524.663690 secs 2023-09-10T10:01:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38524.663690 secs 2023-09-10T10:01:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38525.894703 secs 2023-09-10T10:01:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:34 smithi170 ceph-mon[102832]: pgmap v20705: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 437 KiB/s wr, 16 op/s 2023-09-10T10:01:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38526.246487 secs 2023-09-10T10:01:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38525.894703 secs 2023-09-10T10:01:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:34 smithi082 ceph-mon[98694]: pgmap v20705: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 437 KiB/s wr, 16 op/s 2023-09-10T10:01:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38526.246487 secs 2023-09-10T10:01:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38525.894703 secs 2023-09-10T10:01:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:34 smithi098 ceph-mon[103124]: pgmap v20705: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 437 KiB/s wr, 16 op/s 2023-09-10T10:01:35.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38526.246487 secs 2023-09-10T10:01:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:36 smithi170 ceph-mon[102832]: pgmap v20706: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 429 KiB/s wr, 14 op/s 2023-09-10T10:01:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:36 smithi082 ceph-mon[98694]: pgmap v20706: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 429 KiB/s wr, 14 op/s 2023-09-10T10:01:37.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:36 smithi098 ceph-mon[103124]: pgmap v20706: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 429 KiB/s wr, 14 op/s 2023-09-10T10:01:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:38 smithi170 ceph-mon[102832]: pgmap v20707: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 429 KiB/s wr, 14 op/s 2023-09-10T10:01:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:01:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:39.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:38 smithi082 ceph-mon[98694]: pgmap v20707: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 429 KiB/s wr, 14 op/s 2023-09-10T10:01:39.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:01:39.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:38 smithi098 ceph-mon[103124]: pgmap v20707: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 429 KiB/s wr, 14 op/s 2023-09-10T10:01:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:01:39.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38530.894761 secs 2023-09-10T10:01:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38531.246551 secs 2023-09-10T10:01:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:01:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38530.894761 secs 2023-09-10T10:01:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38531.246551 secs 2023-09-10T10:01:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:01:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38530.894761 secs 2023-09-10T10:01:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38531.246551 secs 2023-09-10T10:01:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:01:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:40 smithi170 ceph-mon[102832]: pgmap v20708: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 418 KiB/s wr, 14 op/s 2023-09-10T10:01:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:01:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:01:41.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:40 smithi082 ceph-mon[98694]: pgmap v20708: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 418 KiB/s wr, 14 op/s 2023-09-10T10:01:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:01:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:01:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:40 smithi098 ceph-mon[103124]: pgmap v20708: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 418 KiB/s wr, 14 op/s 2023-09-10T10:01:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:01:41.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:01:41.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:42 smithi170 ceph-mon[102832]: pgmap v20709: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 669 KiB/s wr, 18 op/s 2023-09-10T10:01:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:42 smithi082 ceph-mon[98694]: pgmap v20709: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 669 KiB/s wr, 18 op/s 2023-09-10T10:01:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:42 smithi098 ceph-mon[103124]: pgmap v20709: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 669 KiB/s wr, 18 op/s 2023-09-10T10:01:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38535.894841 secs 2023-09-10T10:01:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:44 smithi170 ceph-mon[102832]: pgmap v20710: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 312 KiB/s wr, 11 op/s 2023-09-10T10:01:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38536.246615 secs 2023-09-10T10:01:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38535.894841 secs 2023-09-10T10:01:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:44 smithi082 ceph-mon[98694]: pgmap v20710: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 312 KiB/s wr, 11 op/s 2023-09-10T10:01:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38536.246615 secs 2023-09-10T10:01:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38535.894841 secs 2023-09-10T10:01:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:44 smithi098 ceph-mon[103124]: pgmap v20710: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 312 KiB/s wr, 11 op/s 2023-09-10T10:01:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38536.246615 secs 2023-09-10T10:01:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:46 smithi170 ceph-mon[102832]: pgmap v20711: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 640 KiB/s wr, 18 op/s 2023-09-10T10:01:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:01:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:46 smithi082 ceph-mon[98694]: pgmap v20711: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 640 KiB/s wr, 18 op/s 2023-09-10T10:01:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:01:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:46 smithi098 ceph-mon[103124]: pgmap v20711: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 640 KiB/s wr, 18 op/s 2023-09-10T10:01:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:01:47.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:48 smithi170 ceph-mon[102832]: pgmap v20712: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 640 KiB/s wr, 13 op/s 2023-09-10T10:01:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:48 smithi082 ceph-mon[98694]: pgmap v20712: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 640 KiB/s wr, 13 op/s 2023-09-10T10:01:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:48 smithi098 ceph-mon[103124]: pgmap v20712: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 640 KiB/s wr, 13 op/s 2023-09-10T10:01:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38540.894919 secs 2023-09-10T10:01:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38541.246681 secs 2023-09-10T10:01:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38540.894919 secs 2023-09-10T10:01:50.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38541.246681 secs 2023-09-10T10:01:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38540.894919 secs 2023-09-10T10:01:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38541.246681 secs 2023-09-10T10:01:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:50 smithi170 ceph-mon[102832]: pgmap v20713: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 640 KiB/s wr, 13 op/s 2023-09-10T10:01:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:01:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:50 smithi082 ceph-mon[98694]: pgmap v20713: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 640 KiB/s wr, 13 op/s 2023-09-10T10:01:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:01:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:50 smithi098 ceph-mon[103124]: pgmap v20713: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 640 KiB/s wr, 13 op/s 2023-09-10T10:01:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:01:51.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:52 smithi170 ceph-mon[102832]: pgmap v20714: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 849 KiB/s wr, 20 op/s 2023-09-10T10:01:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:01:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:52 smithi082 ceph-mon[98694]: pgmap v20714: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 849 KiB/s wr, 20 op/s 2023-09-10T10:01:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:01:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:52 smithi098 ceph-mon[103124]: pgmap v20714: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 849 KiB/s wr, 20 op/s 2023-09-10T10:01:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:01:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38545.894988 secs 2023-09-10T10:01:55.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:54 smithi170 ceph-mon[102832]: pgmap v20715: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 538 KiB/s wr, 13 op/s 2023-09-10T10:01:55.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38546.246747 secs 2023-09-10T10:01:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38545.894988 secs 2023-09-10T10:01:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:54 smithi082 ceph-mon[98694]: pgmap v20715: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 538 KiB/s wr, 13 op/s 2023-09-10T10:01:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38546.246747 secs 2023-09-10T10:01:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38545.894988 secs 2023-09-10T10:01:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:54 smithi098 ceph-mon[103124]: pgmap v20715: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 538 KiB/s wr, 13 op/s 2023-09-10T10:01:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38546.246747 secs 2023-09-10T10:01:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:56 smithi170 ceph-mon[102832]: pgmap v20716: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1007 KiB/s wr, 21 op/s 2023-09-10T10:01:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:01:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:56 smithi082 ceph-mon[98694]: pgmap v20716: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1007 KiB/s wr, 21 op/s 2023-09-10T10:01:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:01:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:56 smithi098 ceph-mon[103124]: pgmap v20716: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1007 KiB/s wr, 21 op/s 2023-09-10T10:01:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:01:57.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:01:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:01:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:01:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:01:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:58 smithi170 ceph-mon[102832]: pgmap v20717: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 679 KiB/s wr, 14 op/s 2023-09-10T10:01:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38549.663986 secs 2023-09-10T10:01:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:58 smithi082 ceph-mon[98694]: pgmap v20717: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 679 KiB/s wr, 14 op/s 2023-09-10T10:01:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38549.663986 secs 2023-09-10T10:01:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:58 smithi098 ceph-mon[103124]: pgmap v20717: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 679 KiB/s wr, 14 op/s 2023-09-10T10:01:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38549.663986 secs 2023-09-10T10:02:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38550.895046 secs 2023-09-10T10:02:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:01:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38551.246812 secs 2023-09-10T10:02:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38550.895046 secs 2023-09-10T10:02:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:01:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38551.246812 secs 2023-09-10T10:02:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38550.895046 secs 2023-09-10T10:02:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:01:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38551.246812 secs 2023-09-10T10:02:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:00 smithi170 ceph-mon[102832]: pgmap v20718: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 679 KiB/s wr, 14 op/s 2023-09-10T10:02:01.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:02:01.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:02:01.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:00 smithi082 ceph-mon[98694]: pgmap v20718: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 679 KiB/s wr, 14 op/s 2023-09-10T10:02:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:02:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:02:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:00 smithi098 ceph-mon[103124]: pgmap v20718: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 679 KiB/s wr, 14 op/s 2023-09-10T10:02:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:02:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:02:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:02 smithi170 ceph-mon[102832]: pgmap v20719: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 930 KiB/s wr, 21 op/s 2023-09-10T10:02:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:02 smithi082 ceph-mon[98694]: pgmap v20719: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 930 KiB/s wr, 21 op/s 2023-09-10T10:02:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:02 smithi098 ceph-mon[103124]: pgmap v20719: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 930 KiB/s wr, 21 op/s 2023-09-10T10:02:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38554.664065 secs 2023-09-10T10:02:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38554.664065 secs 2023-09-10T10:02:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38554.664065 secs 2023-09-10T10:02:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38555.895107 secs 2023-09-10T10:02:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:04 smithi082 ceph-mon[98694]: pgmap v20720: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 721 KiB/s wr, 14 op/s 2023-09-10T10:02:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38556.246877 secs 2023-09-10T10:02:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38555.895107 secs 2023-09-10T10:02:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:04 smithi098 ceph-mon[103124]: pgmap v20720: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 721 KiB/s wr, 14 op/s 2023-09-10T10:02:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38556.246877 secs 2023-09-10T10:02:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38555.895107 secs 2023-09-10T10:02:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:04 smithi170 ceph-mon[102832]: pgmap v20720: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 721 KiB/s wr, 14 op/s 2023-09-10T10:02:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38556.246877 secs 2023-09-10T10:02:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:06 smithi082 ceph-mon[98694]: pgmap v20721: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 952 KiB/s wr, 34 op/s 2023-09-10T10:02:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:06 smithi098 ceph-mon[103124]: pgmap v20721: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 952 KiB/s wr, 34 op/s 2023-09-10T10:02:07.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:06 smithi170 ceph-mon[102832]: pgmap v20721: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 952 KiB/s wr, 34 op/s 2023-09-10T10:02:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:08 smithi082 ceph-mon[98694]: pgmap v20722: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 483 KiB/s wr, 27 op/s 2023-09-10T10:02:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38559.664122 secs 2023-09-10T10:02:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:08 smithi098 ceph-mon[103124]: pgmap v20722: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 483 KiB/s wr, 27 op/s 2023-09-10T10:02:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38559.664122 secs 2023-09-10T10:02:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:08 smithi170 ceph-mon[102832]: pgmap v20722: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 483 KiB/s wr, 27 op/s 2023-09-10T10:02:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38559.664122 secs 2023-09-10T10:02:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38560.895165 secs 2023-09-10T10:02:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:09 smithi082 ceph-mon[98694]: osdmap e1363: 12 total, 12 up, 12 in 2023-09-10T10:02:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38561.246944 secs 2023-09-10T10:02:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38560.895165 secs 2023-09-10T10:02:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:09 smithi098 ceph-mon[103124]: osdmap e1363: 12 total, 12 up, 12 in 2023-09-10T10:02:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38561.246944 secs 2023-09-10T10:02:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38560.895165 secs 2023-09-10T10:02:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:09 smithi170 ceph-mon[102832]: osdmap e1363: 12 total, 12 up, 12 in 2023-09-10T10:02:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38561.246944 secs 2023-09-10T10:02:11.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:10 smithi082 ceph-mon[98694]: pgmap v20724: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 580 KiB/s wr, 32 op/s 2023-09-10T10:02:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:10 smithi098 ceph-mon[103124]: pgmap v20724: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 580 KiB/s wr, 32 op/s 2023-09-10T10:02:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:10 smithi170 ceph-mon[102832]: pgmap v20724: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 580 KiB/s wr, 32 op/s 2023-09-10T10:02:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:11 smithi082 ceph-mon[98694]: pgmap v20725: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 281 KiB/s wr, 31 op/s 2023-09-10T10:02:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:11 smithi098 ceph-mon[103124]: pgmap v20725: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 281 KiB/s wr, 31 op/s 2023-09-10T10:02:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:11 smithi170 ceph-mon[102832]: pgmap v20725: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 281 KiB/s wr, 31 op/s 2023-09-10T10:02:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38564.664197 secs 2023-09-10T10:02:13.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38564.664197 secs 2023-09-10T10:02:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38564.664197 secs 2023-09-10T10:02:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:14 smithi170 ceph-mon[102832]: pgmap v20726: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 281 KiB/s wr, 31 op/s 2023-09-10T10:02:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:14 smithi082 ceph-mon[98694]: pgmap v20726: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 281 KiB/s wr, 31 op/s 2023-09-10T10:02:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:14 smithi098 ceph-mon[103124]: pgmap v20726: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 281 KiB/s wr, 31 op/s 2023-09-10T10:02:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38565.895221 secs 2023-09-10T10:02:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38566.247004 secs 2023-09-10T10:02:15.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:15 smithi170 ceph-mon[102832]: osdmap e1364: 12 total, 12 up, 12 in 2023-09-10T10:02:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38565.895221 secs 2023-09-10T10:02:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38566.247004 secs 2023-09-10T10:02:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:15 smithi082 ceph-mon[98694]: osdmap e1364: 12 total, 12 up, 12 in 2023-09-10T10:02:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38565.895221 secs 2023-09-10T10:02:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38566.247004 secs 2023-09-10T10:02:15.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:15 smithi098 ceph-mon[103124]: osdmap e1364: 12 total, 12 up, 12 in 2023-09-10T10:02:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:16 smithi170 ceph-mon[102832]: pgmap v20728: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 387 KiB/s wr, 18 op/s 2023-09-10T10:02:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:16 smithi082 ceph-mon[98694]: pgmap v20728: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 387 KiB/s wr, 18 op/s 2023-09-10T10:02:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:16 smithi098 ceph-mon[103124]: pgmap v20728: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 387 KiB/s wr, 18 op/s 2023-09-10T10:02:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:02:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:02:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:02:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:02:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:18.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:02:18.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:02:18.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:02:18.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:18 smithi170 ceph-mon[102832]: pgmap v20729: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 350 KiB/s wr, 17 op/s 2023-09-10T10:02:18.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38569.664253 secs 2023-09-10T10:02:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:02:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:02:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:02:18.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:02:18.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:18 smithi082 ceph-mon[98694]: pgmap v20729: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 350 KiB/s wr, 17 op/s 2023-09-10T10:02:18.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38569.664253 secs 2023-09-10T10:02:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:02:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:18.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:02:18.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:02:18.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:02:18.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:18 smithi098 ceph-mon[103124]: pgmap v20729: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 350 KiB/s wr, 17 op/s 2023-09-10T10:02:18.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38569.664253 secs 2023-09-10T10:02:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38570.895279 secs 2023-09-10T10:02:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38571.247069 secs 2023-09-10T10:02:20.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:02:20.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:20.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:20 smithi170 ceph-mon[102832]: pgmap v20730: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 310 KiB/s wr, 15 op/s 2023-09-10T10:02:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38570.895279 secs 2023-09-10T10:02:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38571.247069 secs 2023-09-10T10:02:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:02:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:20 smithi082 ceph-mon[98694]: pgmap v20730: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 310 KiB/s wr, 15 op/s 2023-09-10T10:02:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38570.895279 secs 2023-09-10T10:02:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38571.247069 secs 2023-09-10T10:02:20.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:02:20.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:20.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:20 smithi098 ceph-mon[103124]: pgmap v20730: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 310 KiB/s wr, 15 op/s 2023-09-10T10:02:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:22 smithi170 ceph-mon[102832]: pgmap v20731: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 470 KiB/s wr, 15 op/s 2023-09-10T10:02:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:22 smithi082 ceph-mon[98694]: pgmap v20731: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 470 KiB/s wr, 15 op/s 2023-09-10T10:02:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:22 smithi098 ceph-mon[103124]: pgmap v20731: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 470 KiB/s wr, 15 op/s 2023-09-10T10:02:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38574.664358 secs 2023-09-10T10:02:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38574.664358 secs 2023-09-10T10:02:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38574.664358 secs 2023-09-10T10:02:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38575.895338 secs 2023-09-10T10:02:25.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:24 smithi170 ceph-mon[102832]: pgmap v20732: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 470 KiB/s wr, 15 op/s 2023-09-10T10:02:25.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:02:25.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:25.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38576.247132 secs 2023-09-10T10:02:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38575.895338 secs 2023-09-10T10:02:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:24 smithi082 ceph-mon[98694]: pgmap v20732: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 470 KiB/s wr, 15 op/s 2023-09-10T10:02:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:02:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38576.247132 secs 2023-09-10T10:02:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38575.895338 secs 2023-09-10T10:02:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:24 smithi098 ceph-mon[103124]: pgmap v20732: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 470 KiB/s wr, 15 op/s 2023-09-10T10:02:25.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:02:25.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:25.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38576.247132 secs 2023-09-10T10:02:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:26 smithi170 ceph-mon[102832]: pgmap v20733: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 570 KiB/s wr, 20 op/s 2023-09-10T10:02:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:26 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:02:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:26 smithi082 ceph-mon[98694]: pgmap v20733: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 570 KiB/s wr, 20 op/s 2023-09-10T10:02:27.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:26 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:02:27.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:26 smithi098 ceph-mon[103124]: pgmap v20733: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 570 KiB/s wr, 20 op/s 2023-09-10T10:02:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:26 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:02:27.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:28 smithi170 ceph-mon[102832]: pgmap v20734: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T10:02:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38579.664453 secs 2023-09-10T10:02:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:28 smithi082 ceph-mon[98694]: pgmap v20734: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T10:02:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38579.664453 secs 2023-09-10T10:02:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:28 smithi098 ceph-mon[103124]: pgmap v20734: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T10:02:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38579.664453 secs 2023-09-10T10:02:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38580.895423 secs 2023-09-10T10:02:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38581.247194 secs 2023-09-10T10:02:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38580.895423 secs 2023-09-10T10:02:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38581.247194 secs 2023-09-10T10:02:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38580.895423 secs 2023-09-10T10:02:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38581.247194 secs 2023-09-10T10:02:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:30 smithi170 ceph-mon[102832]: pgmap v20735: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T10:02:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:30 smithi082 ceph-mon[98694]: pgmap v20735: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T10:02:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:30 smithi098 ceph-mon[103124]: pgmap v20735: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 279 KiB/s wr, 12 op/s 2023-09-10T10:02:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:32 smithi170 ceph-mon[102832]: pgmap v20736: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 280 KiB/s wr, 17 op/s 2023-09-10T10:02:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:32 smithi082 ceph-mon[98694]: pgmap v20736: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 280 KiB/s wr, 17 op/s 2023-09-10T10:02:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:32 smithi098 ceph-mon[103124]: pgmap v20736: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 280 KiB/s wr, 17 op/s 2023-09-10T10:02:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38584.664513 secs 2023-09-10T10:02:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38584.664513 secs 2023-09-10T10:02:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38584.664513 secs 2023-09-10T10:02:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38585.895503 secs 2023-09-10T10:02:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:34 smithi170 ceph-mon[102832]: pgmap v20737: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 144 KiB/s wr, 11 op/s 2023-09-10T10:02:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38586.247259 secs 2023-09-10T10:02:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38585.895503 secs 2023-09-10T10:02:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:34 smithi082 ceph-mon[98694]: pgmap v20737: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 144 KiB/s wr, 11 op/s 2023-09-10T10:02:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38586.247259 secs 2023-09-10T10:02:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38585.895503 secs 2023-09-10T10:02:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:34 smithi098 ceph-mon[103124]: pgmap v20737: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 144 KiB/s wr, 11 op/s 2023-09-10T10:02:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38586.247259 secs 2023-09-10T10:02:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:36 smithi170 ceph-mon[102832]: pgmap v20738: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 343 KiB/s wr, 17 op/s 2023-09-10T10:02:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:36 smithi082 ceph-mon[98694]: pgmap v20738: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 343 KiB/s wr, 17 op/s 2023-09-10T10:02:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:36 smithi098 ceph-mon[103124]: pgmap v20738: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 343 KiB/s wr, 17 op/s 2023-09-10T10:02:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:38 smithi170 ceph-mon[102832]: pgmap v20739: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 199 KiB/s wr, 11 op/s 2023-09-10T10:02:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38589.664571 secs 2023-09-10T10:02:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:38 smithi082 ceph-mon[98694]: pgmap v20739: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 199 KiB/s wr, 11 op/s 2023-09-10T10:02:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38589.664571 secs 2023-09-10T10:02:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:38 smithi098 ceph-mon[103124]: pgmap v20739: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 199 KiB/s wr, 11 op/s 2023-09-10T10:02:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38589.664571 secs 2023-09-10T10:02:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38590.895559 secs 2023-09-10T10:02:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38591.247322 secs 2023-09-10T10:02:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38590.895559 secs 2023-09-10T10:02:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38591.247322 secs 2023-09-10T10:02:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38590.895559 secs 2023-09-10T10:02:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38591.247322 secs 2023-09-10T10:02:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:40 smithi170 ceph-mon[102832]: pgmap v20740: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 199 KiB/s wr, 11 op/s 2023-09-10T10:02:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:40 smithi082 ceph-mon[98694]: pgmap v20740: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 199 KiB/s wr, 11 op/s 2023-09-10T10:02:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:40 smithi098 ceph-mon[103124]: pgmap v20740: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 199 KiB/s wr, 11 op/s 2023-09-10T10:02:43.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:42 smithi170 ceph-mon[102832]: pgmap v20741: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 283 KiB/s wr, 17 op/s 2023-09-10T10:02:43.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:42 smithi082 ceph-mon[98694]: pgmap v20741: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 283 KiB/s wr, 17 op/s 2023-09-10T10:02:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:42 smithi098 ceph-mon[103124]: pgmap v20741: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 283 KiB/s wr, 17 op/s 2023-09-10T10:02:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:43 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:02:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:43 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:02:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:43 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:02:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38595.895619 secs 2023-09-10T10:02:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:44 smithi170 ceph-mon[102832]: pgmap v20742: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 283 KiB/s wr, 11 op/s 2023-09-10T10:02:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:44 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:02:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:02:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38596.247384 secs 2023-09-10T10:02:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38595.895619 secs 2023-09-10T10:02:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:44 smithi082 ceph-mon[98694]: pgmap v20742: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 283 KiB/s wr, 11 op/s 2023-09-10T10:02:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:44 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:02:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:02:45.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:45.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38596.247384 secs 2023-09-10T10:02:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38595.895619 secs 2023-09-10T10:02:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:44 smithi098 ceph-mon[103124]: pgmap v20742: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 283 KiB/s wr, 11 op/s 2023-09-10T10:02:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:44 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:02:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:02:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38596.247384 secs 2023-09-10T10:02:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:46 smithi170 ceph-mon[102832]: pgmap v20743: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 588 KiB/s wr, 17 op/s 2023-09-10T10:02:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:46 smithi082 ceph-mon[98694]: pgmap v20743: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 588 KiB/s wr, 17 op/s 2023-09-10T10:02:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:46 smithi098 ceph-mon[103124]: pgmap v20743: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 588 KiB/s wr, 17 op/s 2023-09-10T10:02:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:48 smithi170 ceph-mon[102832]: pgmap v20744: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 389 KiB/s wr, 12 op/s 2023-09-10T10:02:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:48 smithi082 ceph-mon[98694]: pgmap v20744: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 389 KiB/s wr, 12 op/s 2023-09-10T10:02:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:48 smithi098 ceph-mon[103124]: pgmap v20744: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 389 KiB/s wr, 12 op/s 2023-09-10T10:02:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38600.895677 secs 2023-09-10T10:02:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:02:50.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:50.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38601.247479 secs 2023-09-10T10:02:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38600.895677 secs 2023-09-10T10:02:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:02:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38601.247479 secs 2023-09-10T10:02:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38600.895677 secs 2023-09-10T10:02:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:02:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38601.247479 secs 2023-09-10T10:02:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:50 smithi170 ceph-mon[102832]: pgmap v20745: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 389 KiB/s wr, 12 op/s 2023-09-10T10:02:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:50 smithi082 ceph-mon[98694]: pgmap v20745: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 389 KiB/s wr, 12 op/s 2023-09-10T10:02:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:50 smithi098 ceph-mon[103124]: pgmap v20745: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 389 KiB/s wr, 12 op/s 2023-09-10T10:02:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:52 smithi170 ceph-mon[102832]: pgmap v20746: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 582 KiB/s wr, 19 op/s 2023-09-10T10:02:53.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:02:53.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:52 smithi082 ceph-mon[98694]: pgmap v20746: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 582 KiB/s wr, 19 op/s 2023-09-10T10:02:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:02:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:52 smithi098 ceph-mon[103124]: pgmap v20746: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 582 KiB/s wr, 19 op/s 2023-09-10T10:02:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:02:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38605.895737 secs 2023-09-10T10:02:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:54 smithi170 ceph-mon[102832]: pgmap v20747: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 498 KiB/s wr, 13 op/s 2023-09-10T10:02:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38606.247544 secs 2023-09-10T10:02:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38605.895737 secs 2023-09-10T10:02:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:54 smithi082 ceph-mon[98694]: pgmap v20747: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 498 KiB/s wr, 13 op/s 2023-09-10T10:02:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38606.247544 secs 2023-09-10T10:02:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38605.895737 secs 2023-09-10T10:02:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:54 smithi098 ceph-mon[103124]: pgmap v20747: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 498 KiB/s wr, 13 op/s 2023-09-10T10:02:55.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38606.247544 secs 2023-09-10T10:02:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:56 smithi082 ceph-mon[98694]: pgmap v20748: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 872 KiB/s wr, 21 op/s 2023-09-10T10:02:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:02:57.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:56 smithi098 ceph-mon[103124]: pgmap v20748: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 872 KiB/s wr, 21 op/s 2023-09-10T10:02:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:02:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:56 smithi170 ceph-mon[102832]: pgmap v20748: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 872 KiB/s wr, 21 op/s 2023-09-10T10:02:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:02:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:02:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:02:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:02:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:02:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:58 smithi082 ceph-mon[98694]: pgmap v20749: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 567 KiB/s wr, 14 op/s 2023-09-10T10:02:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:58 smithi098 ceph-mon[103124]: pgmap v20749: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 567 KiB/s wr, 14 op/s 2023-09-10T10:02:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:58 smithi170 ceph-mon[102832]: pgmap v20749: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 567 KiB/s wr, 14 op/s 2023-09-10T10:03:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38610.895813 secs 2023-09-10T10:03:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:02:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38611.247610 secs 2023-09-10T10:03:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38610.895813 secs 2023-09-10T10:03:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:02:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38611.247610 secs 2023-09-10T10:03:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38610.895813 secs 2023-09-10T10:03:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:02:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38611.247610 secs 2023-09-10T10:03:01.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:00 smithi082 ceph-mon[98694]: pgmap v20750: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 567 KiB/s wr, 14 op/s 2023-09-10T10:03:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:00 smithi098 ceph-mon[103124]: pgmap v20750: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 567 KiB/s wr, 14 op/s 2023-09-10T10:03:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:00 smithi170 ceph-mon[102832]: pgmap v20750: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 567 KiB/s wr, 14 op/s 2023-09-10T10:03:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:02 smithi082 ceph-mon[98694]: pgmap v20751: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 728 KiB/s wr, 22 op/s 2023-09-10T10:03:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:02 smithi098 ceph-mon[103124]: pgmap v20751: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 728 KiB/s wr, 22 op/s 2023-09-10T10:03:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:02 smithi170 ceph-mon[102832]: pgmap v20751: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 728 KiB/s wr, 22 op/s 2023-09-10T10:03:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38615.895870 secs 2023-09-10T10:03:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:04 smithi082 ceph-mon[98694]: pgmap v20752: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 535 KiB/s wr, 15 op/s 2023-09-10T10:03:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:03:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:05.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38616.247677 secs 2023-09-10T10:03:05.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38615.895870 secs 2023-09-10T10:03:05.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:04 smithi098 ceph-mon[103124]: pgmap v20752: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 535 KiB/s wr, 15 op/s 2023-09-10T10:03:05.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:03:05.258 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:05.258 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38616.247677 secs 2023-09-10T10:03:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38615.895870 secs 2023-09-10T10:03:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:04 smithi170 ceph-mon[102832]: pgmap v20752: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 535 KiB/s wr, 15 op/s 2023-09-10T10:03:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:03:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38616.247677 secs 2023-09-10T10:03:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:03:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:06 smithi170 ceph-mon[102832]: pgmap v20753: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 949 KiB/s wr, 33 op/s 2023-09-10T10:03:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:03:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:06 smithi082 ceph-mon[98694]: pgmap v20753: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 949 KiB/s wr, 33 op/s 2023-09-10T10:03:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:03:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:07.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:06 smithi098 ceph-mon[103124]: pgmap v20753: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 949 KiB/s wr, 33 op/s 2023-09-10T10:03:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:08 smithi170 ceph-mon[102832]: pgmap v20754: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 576 KiB/s wr, 25 op/s 2023-09-10T10:03:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:08 smithi082 ceph-mon[98694]: pgmap v20754: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 576 KiB/s wr, 25 op/s 2023-09-10T10:03:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:08 smithi098 ceph-mon[103124]: pgmap v20754: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 576 KiB/s wr, 25 op/s 2023-09-10T10:03:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38620.895936 secs 2023-09-10T10:03:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38621.247742 secs 2023-09-10T10:03:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38620.895936 secs 2023-09-10T10:03:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38621.247742 secs 2023-09-10T10:03:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38620.895936 secs 2023-09-10T10:03:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38621.247742 secs 2023-09-10T10:03:11.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:10 smithi170 ceph-mon[102832]: pgmap v20755: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 576 KiB/s wr, 25 op/s 2023-09-10T10:03:11.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:03:11.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:10 smithi082 ceph-mon[98694]: pgmap v20755: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 576 KiB/s wr, 25 op/s 2023-09-10T10:03:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:03:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:11.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:10 smithi098 ceph-mon[103124]: pgmap v20755: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 576 KiB/s wr, 25 op/s 2023-09-10T10:03:11.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:03:11.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:12 smithi170 ceph-mon[102832]: pgmap v20756: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 856 KiB/s wr, 31 op/s 2023-09-10T10:03:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:03:13.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:12 smithi082 ceph-mon[98694]: pgmap v20756: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 856 KiB/s wr, 31 op/s 2023-09-10T10:03:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:03:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:12 smithi098 ceph-mon[103124]: pgmap v20756: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 856 KiB/s wr, 31 op/s 2023-09-10T10:03:13.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:03:13.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38625.895988 secs 2023-09-10T10:03:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:14 smithi170 ceph-mon[102832]: pgmap v20757: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 695 KiB/s wr, 24 op/s 2023-09-10T10:03:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38626.247805 secs 2023-09-10T10:03:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38625.895988 secs 2023-09-10T10:03:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:14 smithi082 ceph-mon[98694]: pgmap v20757: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 695 KiB/s wr, 24 op/s 2023-09-10T10:03:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38626.247805 secs 2023-09-10T10:03:15.262 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38625.895988 secs 2023-09-10T10:03:15.263 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:14 smithi098 ceph-mon[103124]: pgmap v20757: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 695 KiB/s wr, 24 op/s 2023-09-10T10:03:15.263 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38626.247805 secs 2023-09-10T10:03:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:16 smithi170 ceph-mon[102832]: pgmap v20758: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 763 KiB/s wr, 29 op/s 2023-09-10T10:03:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:16 smithi082 ceph-mon[98694]: pgmap v20758: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 763 KiB/s wr, 29 op/s 2023-09-10T10:03:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:16 smithi098 ceph-mon[103124]: pgmap v20758: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 763 KiB/s wr, 29 op/s 2023-09-10T10:03:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:03:18.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:03:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:03:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:18 smithi170 ceph-mon[102832]: pgmap v20759: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 348 KiB/s wr, 11 op/s 2023-09-10T10:03:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:03:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:03:19.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:03:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:18 smithi082 ceph-mon[98694]: pgmap v20759: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 348 KiB/s wr, 11 op/s 2023-09-10T10:03:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:03:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:03:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:03:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:18 smithi098 ceph-mon[103124]: pgmap v20759: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 348 KiB/s wr, 11 op/s 2023-09-10T10:03:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:03:19.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:03:19.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:03:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38630.896049 secs 2023-09-10T10:03:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38631.247877 secs 2023-09-10T10:03:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38630.896049 secs 2023-09-10T10:03:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38631.247877 secs 2023-09-10T10:03:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38630.896049 secs 2023-09-10T10:03:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38631.247877 secs 2023-09-10T10:03:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:20 smithi170 ceph-mon[102832]: pgmap v20760: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 348 KiB/s wr, 11 op/s 2023-09-10T10:03:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:20 smithi082 ceph-mon[98694]: pgmap v20760: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 348 KiB/s wr, 11 op/s 2023-09-10T10:03:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:20 smithi098 ceph-mon[103124]: pgmap v20760: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 348 KiB/s wr, 11 op/s 2023-09-10T10:03:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:22 smithi170 ceph-mon[102832]: pgmap v20761: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 611 KiB/s wr, 20 op/s 2023-09-10T10:03:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:22 smithi082 ceph-mon[98694]: pgmap v20761: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 611 KiB/s wr, 20 op/s 2023-09-10T10:03:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:22 smithi098 ceph-mon[103124]: pgmap v20761: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 611 KiB/s wr, 20 op/s 2023-09-10T10:03:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38634.665232 secs 2023-09-10T10:03:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38634.665232 secs 2023-09-10T10:03:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38634.665232 secs 2023-09-10T10:03:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38635.896104 secs 2023-09-10T10:03:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:24 smithi170 ceph-mon[102832]: pgmap v20762: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 331 KiB/s wr, 13 op/s 2023-09-10T10:03:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:24 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:03:25.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:03:25.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:25.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38636.247946 secs 2023-09-10T10:03:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38635.896104 secs 2023-09-10T10:03:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:24 smithi082 ceph-mon[98694]: pgmap v20762: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 331 KiB/s wr, 13 op/s 2023-09-10T10:03:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:24 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:03:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:03:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:25.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38636.247946 secs 2023-09-10T10:03:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38635.896104 secs 2023-09-10T10:03:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:24 smithi098 ceph-mon[103124]: pgmap v20762: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 331 KiB/s wr, 13 op/s 2023-09-10T10:03:25.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:24 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:03:25.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:03:25.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:25.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38636.247946 secs 2023-09-10T10:03:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:26 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:03:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:26 smithi170 ceph-mon[102832]: pgmap v20763: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 469 KiB/s wr, 19 op/s 2023-09-10T10:03:26.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:26 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:03:26.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:26.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:26 smithi082 ceph-mon[98694]: pgmap v20763: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 469 KiB/s wr, 19 op/s 2023-09-10T10:03:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:26 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:03:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:27.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:26 smithi098 ceph-mon[103124]: pgmap v20763: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 469 KiB/s wr, 19 op/s 2023-09-10T10:03:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:28 smithi170 ceph-mon[102832]: pgmap v20764: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 401 KiB/s wr, 14 op/s 2023-09-10T10:03:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38639.665290 secs 2023-09-10T10:03:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:28 smithi082 ceph-mon[98694]: pgmap v20764: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 401 KiB/s wr, 14 op/s 2023-09-10T10:03:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38639.665290 secs 2023-09-10T10:03:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:28 smithi098 ceph-mon[103124]: pgmap v20764: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 401 KiB/s wr, 14 op/s 2023-09-10T10:03:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38639.665290 secs 2023-09-10T10:03:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38640.896164 secs 2023-09-10T10:03:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:29 smithi170 ceph-mon[102832]: osdmap e1365: 12 total, 12 up, 12 in 2023-09-10T10:03:30.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38641.248012 secs 2023-09-10T10:03:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38640.896164 secs 2023-09-10T10:03:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:29 smithi082 ceph-mon[98694]: osdmap e1365: 12 total, 12 up, 12 in 2023-09-10T10:03:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38641.248012 secs 2023-09-10T10:03:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38640.896164 secs 2023-09-10T10:03:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:29 smithi098 ceph-mon[103124]: osdmap e1365: 12 total, 12 up, 12 in 2023-09-10T10:03:30.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38641.248012 secs 2023-09-10T10:03:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:30 smithi170 ceph-mon[102832]: pgmap v20766: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 481 KiB/s wr, 17 op/s 2023-09-10T10:03:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:03:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:30 smithi082 ceph-mon[98694]: pgmap v20766: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 481 KiB/s wr, 17 op/s 2023-09-10T10:03:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:03:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:30 smithi098 ceph-mon[103124]: pgmap v20766: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 481 KiB/s wr, 17 op/s 2023-09-10T10:03:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:03:31.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:32 smithi170 ceph-mon[102832]: pgmap v20767: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 373 KiB/s wr, 14 op/s 2023-09-10T10:03:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:03:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:32 smithi082 ceph-mon[98694]: pgmap v20767: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 373 KiB/s wr, 14 op/s 2023-09-10T10:03:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:03:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:32 smithi098 ceph-mon[103124]: pgmap v20767: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 373 KiB/s wr, 14 op/s 2023-09-10T10:03:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:03:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38644.665350 secs 2023-09-10T10:03:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38644.665350 secs 2023-09-10T10:03:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38644.665350 secs 2023-09-10T10:03:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38645.896221 secs 2023-09-10T10:03:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:34 smithi170 ceph-mon[102832]: pgmap v20768: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 373 KiB/s wr, 14 op/s 2023-09-10T10:03:35.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38646.248079 secs 2023-09-10T10:03:35.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:34 smithi170 ceph-mon[102832]: osdmap e1366: 12 total, 12 up, 12 in 2023-09-10T10:03:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38645.896221 secs 2023-09-10T10:03:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:34 smithi082 ceph-mon[98694]: pgmap v20768: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 373 KiB/s wr, 14 op/s 2023-09-10T10:03:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38646.248079 secs 2023-09-10T10:03:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:34 smithi082 ceph-mon[98694]: osdmap e1366: 12 total, 12 up, 12 in 2023-09-10T10:03:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38645.896221 secs 2023-09-10T10:03:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:34 smithi098 ceph-mon[103124]: pgmap v20768: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 373 KiB/s wr, 14 op/s 2023-09-10T10:03:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38646.248079 secs 2023-09-10T10:03:35.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:34 smithi098 ceph-mon[103124]: osdmap e1366: 12 total, 12 up, 12 in 2023-09-10T10:03:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:36 smithi170 ceph-mon[102832]: pgmap v20770: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 503 KiB/s wr, 17 op/s 2023-09-10T10:03:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:36 smithi082 ceph-mon[98694]: pgmap v20770: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 503 KiB/s wr, 17 op/s 2023-09-10T10:03:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:36 smithi098 ceph-mon[103124]: pgmap v20770: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 503 KiB/s wr, 17 op/s 2023-09-10T10:03:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:03:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:38.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:03:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:03:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:38 smithi170 ceph-mon[102832]: pgmap v20771: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 450 KiB/s wr, 15 op/s 2023-09-10T10:03:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38649.665410 secs 2023-09-10T10:03:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:38 smithi082 ceph-mon[98694]: pgmap v20771: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 450 KiB/s wr, 15 op/s 2023-09-10T10:03:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38649.665410 secs 2023-09-10T10:03:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:38 smithi098 ceph-mon[103124]: pgmap v20771: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 450 KiB/s wr, 15 op/s 2023-09-10T10:03:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38649.665410 secs 2023-09-10T10:03:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38650.896277 secs 2023-09-10T10:03:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38651.248144 secs 2023-09-10T10:03:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38650.896277 secs 2023-09-10T10:03:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38651.248144 secs 2023-09-10T10:03:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38650.896277 secs 2023-09-10T10:03:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38651.248144 secs 2023-09-10T10:03:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:40 smithi082 ceph-mon[98694]: pgmap v20772: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 402 KiB/s wr, 13 op/s 2023-09-10T10:03:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:03:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:40 smithi098 ceph-mon[103124]: pgmap v20772: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 402 KiB/s wr, 13 op/s 2023-09-10T10:03:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:03:41.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:40 smithi170 ceph-mon[102832]: pgmap v20772: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 402 KiB/s wr, 13 op/s 2023-09-10T10:03:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:03:41.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:43.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:42 smithi082 ceph-mon[98694]: pgmap v20773: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 405 KiB/s wr, 14 op/s 2023-09-10T10:03:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:42 smithi098 ceph-mon[103124]: pgmap v20773: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 405 KiB/s wr, 14 op/s 2023-09-10T10:03:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:42 smithi170 ceph-mon[102832]: pgmap v20773: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 405 KiB/s wr, 14 op/s 2023-09-10T10:03:44.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:43 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38654.665469 secs 2023-09-10T10:03:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:43 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38654.665469 secs 2023-09-10T10:03:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:43 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38654.665469 secs 2023-09-10T10:03:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38655.896337 secs 2023-09-10T10:03:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:44 smithi082 ceph-mon[98694]: pgmap v20774: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 405 KiB/s wr, 14 op/s 2023-09-10T10:03:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38656.248209 secs 2023-09-10T10:03:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38655.896337 secs 2023-09-10T10:03:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:44 smithi098 ceph-mon[103124]: pgmap v20774: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 405 KiB/s wr, 14 op/s 2023-09-10T10:03:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38656.248209 secs 2023-09-10T10:03:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38655.896337 secs 2023-09-10T10:03:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:44 smithi170 ceph-mon[102832]: pgmap v20774: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 405 KiB/s wr, 14 op/s 2023-09-10T10:03:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38656.248209 secs 2023-09-10T10:03:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:46 smithi082 ceph-mon[98694]: pgmap v20775: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 686 KiB/s wr, 20 op/s 2023-09-10T10:03:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:46 smithi098 ceph-mon[103124]: pgmap v20775: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 686 KiB/s wr, 20 op/s 2023-09-10T10:03:47.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:46 smithi170 ceph-mon[102832]: pgmap v20775: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 686 KiB/s wr, 20 op/s 2023-09-10T10:03:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:48 smithi082 ceph-mon[98694]: pgmap v20776: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 483 KiB/s wr, 13 op/s 2023-09-10T10:03:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:48 smithi098 ceph-mon[103124]: pgmap v20776: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 483 KiB/s wr, 13 op/s 2023-09-10T10:03:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:48 smithi170 ceph-mon[102832]: pgmap v20776: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 483 KiB/s wr, 13 op/s 2023-09-10T10:03:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38660.896393 secs 2023-09-10T10:03:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:03:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38661.248270 secs 2023-09-10T10:03:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38660.896393 secs 2023-09-10T10:03:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:03:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38661.248270 secs 2023-09-10T10:03:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38660.896393 secs 2023-09-10T10:03:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:03:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38661.248270 secs 2023-09-10T10:03:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:50 smithi082 ceph-mon[98694]: pgmap v20777: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 483 KiB/s wr, 13 op/s 2023-09-10T10:03:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:50 smithi098 ceph-mon[103124]: pgmap v20777: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 483 KiB/s wr, 13 op/s 2023-09-10T10:03:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:50 smithi170 ceph-mon[102832]: pgmap v20777: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 483 KiB/s wr, 13 op/s 2023-09-10T10:03:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:51 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:03:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:51 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:03:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:51 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:03:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:52 smithi082 ceph-mon[98694]: pgmap v20778: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 771 KiB/s wr, 20 op/s 2023-09-10T10:03:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:52 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:03:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:03:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:52 smithi098 ceph-mon[103124]: pgmap v20778: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 771 KiB/s wr, 20 op/s 2023-09-10T10:03:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:52 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:03:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:03:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:52 smithi170 ceph-mon[102832]: pgmap v20778: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 771 KiB/s wr, 20 op/s 2023-09-10T10:03:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:52 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:03:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:03:53.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:03:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:54.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:53 smithi098 ceph-mon[103124]: pgmap v20779: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 595 KiB/s wr, 14 op/s 2023-09-10T10:03:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:03:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:53 smithi170 ceph-mon[102832]: pgmap v20779: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 595 KiB/s wr, 14 op/s 2023-09-10T10:03:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:03:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:53 smithi082 ceph-mon[98694]: pgmap v20779: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 595 KiB/s wr, 14 op/s 2023-09-10T10:03:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38665.896453 secs 2023-09-10T10:03:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38666.248337 secs 2023-09-10T10:03:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38665.896453 secs 2023-09-10T10:03:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38666.248337 secs 2023-09-10T10:03:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38665.896453 secs 2023-09-10T10:03:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38666.248337 secs 2023-09-10T10:03:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:55 smithi098 ceph-mon[103124]: pgmap v20780: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1013 KiB/s wr, 22 op/s 2023-09-10T10:03:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:55 smithi170 ceph-mon[102832]: pgmap v20780: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1013 KiB/s wr, 22 op/s 2023-09-10T10:03:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:55 smithi082 ceph-mon[98694]: pgmap v20780: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1013 KiB/s wr, 22 op/s 2023-09-10T10:03:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:03:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:03:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:03:57.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:03:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:58.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:57 smithi098 ceph-mon[103124]: pgmap v20781: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 707 KiB/s wr, 15 op/s 2023-09-10T10:03:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:03:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:57 smithi170 ceph-mon[102832]: pgmap v20781: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 707 KiB/s wr, 15 op/s 2023-09-10T10:03:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:03:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:03:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:57 smithi082 ceph-mon[98694]: pgmap v20781: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 707 KiB/s wr, 15 op/s 2023-09-10T10:03:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38669.665671 secs 2023-09-10T10:03:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38669.665671 secs 2023-09-10T10:03:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38669.665671 secs 2023-09-10T10:04:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38670.896534 secs 2023-09-10T10:04:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38671.248399 secs 2023-09-10T10:04:00.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:03:59 smithi098 ceph-mon[103124]: pgmap v20782: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 707 KiB/s wr, 15 op/s 2023-09-10T10:04:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38670.896534 secs 2023-09-10T10:04:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38671.248399 secs 2023-09-10T10:04:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:03:59 smithi170 ceph-mon[102832]: pgmap v20782: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 707 KiB/s wr, 15 op/s 2023-09-10T10:04:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38670.896534 secs 2023-09-10T10:04:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38671.248399 secs 2023-09-10T10:04:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:03:59 smithi082 ceph-mon[98694]: pgmap v20782: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 707 KiB/s wr, 15 op/s 2023-09-10T10:04:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:04:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:04:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:04:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:04:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:04:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:04:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:02 smithi098 ceph-mon[103124]: pgmap v20783: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 981 KiB/s wr, 22 op/s 2023-09-10T10:04:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:01 smithi170 ceph-mon[102832]: pgmap v20783: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 981 KiB/s wr, 22 op/s 2023-09-10T10:04:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:01 smithi082 ceph-mon[98694]: pgmap v20783: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 981 KiB/s wr, 22 op/s 2023-09-10T10:04:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38674.665729 secs 2023-09-10T10:04:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38674.665729 secs 2023-09-10T10:04:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38674.665729 secs 2023-09-10T10:04:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:04 smithi170 ceph-mon[102832]: pgmap v20784: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 693 KiB/s wr, 15 op/s 2023-09-10T10:04:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:04 smithi082 ceph-mon[98694]: pgmap v20784: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 693 KiB/s wr, 15 op/s 2023-09-10T10:04:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:04 smithi098 ceph-mon[103124]: pgmap v20784: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 693 KiB/s wr, 15 op/s 2023-09-10T10:04:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38675.896600 secs 2023-09-10T10:04:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38676.248460 secs 2023-09-10T10:04:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38675.896600 secs 2023-09-10T10:04:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38676.248460 secs 2023-09-10T10:04:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38675.896600 secs 2023-09-10T10:04:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38676.248460 secs 2023-09-10T10:04:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:06 smithi170 ceph-mon[102832]: pgmap v20785: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 941 KiB/s wr, 36 op/s 2023-09-10T10:04:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:06 smithi082 ceph-mon[98694]: pgmap v20785: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 941 KiB/s wr, 36 op/s 2023-09-10T10:04:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:06 smithi098 ceph-mon[103124]: pgmap v20785: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 941 KiB/s wr, 36 op/s 2023-09-10T10:04:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:08 smithi170 ceph-mon[102832]: pgmap v20786: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 522 KiB/s wr, 28 op/s 2023-09-10T10:04:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38679.665777 secs 2023-09-10T10:04:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:08 smithi082 ceph-mon[98694]: pgmap v20786: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 522 KiB/s wr, 28 op/s 2023-09-10T10:04:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38679.665777 secs 2023-09-10T10:04:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:08 smithi098 ceph-mon[103124]: pgmap v20786: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 522 KiB/s wr, 28 op/s 2023-09-10T10:04:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38679.665777 secs 2023-09-10T10:04:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38680.896684 secs 2023-09-10T10:04:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:09 smithi170 ceph-mon[102832]: osdmap e1367: 12 total, 12 up, 12 in 2023-09-10T10:04:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38681.248492 secs 2023-09-10T10:04:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38680.896684 secs 2023-09-10T10:04:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:09 smithi082 ceph-mon[98694]: osdmap e1367: 12 total, 12 up, 12 in 2023-09-10T10:04:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38681.248492 secs 2023-09-10T10:04:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38680.896684 secs 2023-09-10T10:04:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:09 smithi098 ceph-mon[103124]: osdmap e1367: 12 total, 12 up, 12 in 2023-09-10T10:04:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38681.248492 secs 2023-09-10T10:04:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:10 smithi170 ceph-mon[102832]: pgmap v20788: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 627 KiB/s wr, 34 op/s 2023-09-10T10:04:11.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:04:11.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:10 smithi082 ceph-mon[98694]: pgmap v20788: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 627 KiB/s wr, 34 op/s 2023-09-10T10:04:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:04:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:10 smithi098 ceph-mon[103124]: pgmap v20788: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 627 KiB/s wr, 34 op/s 2023-09-10T10:04:11.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:04:11.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:12 smithi170 ceph-mon[102832]: pgmap v20789: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 500 KiB/s wr, 32 op/s 2023-09-10T10:04:13.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:04:13.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:12 smithi082 ceph-mon[98694]: pgmap v20789: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 500 KiB/s wr, 32 op/s 2023-09-10T10:04:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:04:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:12 smithi098 ceph-mon[103124]: pgmap v20789: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 500 KiB/s wr, 32 op/s 2023-09-10T10:04:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:04:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38684.665842 secs 2023-09-10T10:04:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38684.665842 secs 2023-09-10T10:04:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38684.665842 secs 2023-09-10T10:04:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38685.896745 secs 2023-09-10T10:04:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:14 smithi170 ceph-mon[102832]: pgmap v20790: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 500 KiB/s wr, 32 op/s 2023-09-10T10:04:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38686.248555 secs 2023-09-10T10:04:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:14 smithi170 ceph-mon[102832]: osdmap e1368: 12 total, 12 up, 12 in 2023-09-10T10:04:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38685.896745 secs 2023-09-10T10:04:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:14 smithi082 ceph-mon[98694]: pgmap v20790: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 500 KiB/s wr, 32 op/s 2023-09-10T10:04:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38686.248555 secs 2023-09-10T10:04:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:14 smithi082 ceph-mon[98694]: osdmap e1368: 12 total, 12 up, 12 in 2023-09-10T10:04:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38685.896745 secs 2023-09-10T10:04:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:14 smithi098 ceph-mon[103124]: pgmap v20790: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 500 KiB/s wr, 32 op/s 2023-09-10T10:04:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38686.248555 secs 2023-09-10T10:04:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:14 smithi098 ceph-mon[103124]: osdmap e1368: 12 total, 12 up, 12 in 2023-09-10T10:04:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:16 smithi170 ceph-mon[102832]: pgmap v20792: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 398 KiB/s wr, 18 op/s 2023-09-10T10:04:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:16 smithi082 ceph-mon[98694]: pgmap v20792: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 398 KiB/s wr, 18 op/s 2023-09-10T10:04:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:16 smithi098 ceph-mon[103124]: pgmap v20792: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 398 KiB/s wr, 18 op/s 2023-09-10T10:04:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:18 smithi170 ceph-mon[102832]: pgmap v20793: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 356 KiB/s wr, 16 op/s 2023-09-10T10:04:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38689.665928 secs 2023-09-10T10:04:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:04:19.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:04:19.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:04:19.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:18 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:04:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:18 smithi082 ceph-mon[98694]: pgmap v20793: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 356 KiB/s wr, 16 op/s 2023-09-10T10:04:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38689.665928 secs 2023-09-10T10:04:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:04:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:04:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:04:19.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:18 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:04:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:18 smithi098 ceph-mon[103124]: pgmap v20793: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 356 KiB/s wr, 16 op/s 2023-09-10T10:04:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38689.665928 secs 2023-09-10T10:04:19.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:04:19.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:04:19.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:04:19.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:18 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:04:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38690.896814 secs 2023-09-10T10:04:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38691.248622 secs 2023-09-10T10:04:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38690.896814 secs 2023-09-10T10:04:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38691.248622 secs 2023-09-10T10:04:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38690.896814 secs 2023-09-10T10:04:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38691.248622 secs 2023-09-10T10:04:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:20 smithi170 ceph-mon[102832]: pgmap v20794: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 318 KiB/s wr, 15 op/s 2023-09-10T10:04:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:20 smithi082 ceph-mon[98694]: pgmap v20794: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 318 KiB/s wr, 15 op/s 2023-09-10T10:04:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:20 smithi098 ceph-mon[103124]: pgmap v20794: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 318 KiB/s wr, 15 op/s 2023-09-10T10:04:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:22 smithi170 ceph-mon[102832]: pgmap v20795: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 14 op/s 2023-09-10T10:04:23.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:22 smithi082 ceph-mon[98694]: pgmap v20795: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 14 op/s 2023-09-10T10:04:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:22 smithi098 ceph-mon[103124]: pgmap v20795: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 14 op/s 2023-09-10T10:04:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38694.665986 secs 2023-09-10T10:04:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38694.665986 secs 2023-09-10T10:04:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38694.665986 secs 2023-09-10T10:04:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38695.896872 secs 2023-09-10T10:04:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:24 smithi082 ceph-mon[98694]: pgmap v20796: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 14 op/s 2023-09-10T10:04:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38696.248692 secs 2023-09-10T10:04:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38695.896872 secs 2023-09-10T10:04:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:24 smithi098 ceph-mon[103124]: pgmap v20796: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 14 op/s 2023-09-10T10:04:25.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38696.248692 secs 2023-09-10T10:04:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38695.896872 secs 2023-09-10T10:04:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:24 smithi170 ceph-mon[102832]: pgmap v20796: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 14 op/s 2023-09-10T10:04:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38696.248692 secs 2023-09-10T10:04:27.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:26 smithi082 ceph-mon[98694]: pgmap v20797: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 105 KiB/s wr, 19 op/s 2023-09-10T10:04:27.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:26 smithi098 ceph-mon[103124]: pgmap v20797: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 105 KiB/s wr, 19 op/s 2023-09-10T10:04:27.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:26 smithi170 ceph-mon[102832]: pgmap v20797: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 105 KiB/s wr, 19 op/s 2023-09-10T10:04:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:28 smithi082 ceph-mon[98694]: pgmap v20798: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-09-10T10:04:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38699.666058 secs 2023-09-10T10:04:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:28 smithi098 ceph-mon[103124]: pgmap v20798: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-09-10T10:04:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38699.666058 secs 2023-09-10T10:04:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:28 smithi170 ceph-mon[102832]: pgmap v20798: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-09-10T10:04:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38699.666058 secs 2023-09-10T10:04:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38700.896931 secs 2023-09-10T10:04:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:29 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:04:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:29 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38701.248755 secs 2023-09-10T10:04:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38700.896931 secs 2023-09-10T10:04:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:29 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:04:30.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:29 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:30.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38701.248755 secs 2023-09-10T10:04:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38700.896931 secs 2023-09-10T10:04:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:29 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:04:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:29 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:30.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38701.248755 secs 2023-09-10T10:04:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:30 smithi082 ceph-mon[98694]: pgmap v20799: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-09-10T10:04:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:30 smithi098 ceph-mon[103124]: pgmap v20799: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-09-10T10:04:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:30 smithi170 ceph-mon[102832]: pgmap v20799: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-09-10T10:04:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:32 smithi082 ceph-mon[98694]: pgmap v20800: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 17 op/s 2023-09-10T10:04:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:04:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:32 smithi098 ceph-mon[103124]: pgmap v20800: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 17 op/s 2023-09-10T10:04:33.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:04:33.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:32 smithi170 ceph-mon[102832]: pgmap v20800: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 17 op/s 2023-09-10T10:04:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:04:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38704.666143 secs 2023-09-10T10:04:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38704.666143 secs 2023-09-10T10:04:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38704.666143 secs 2023-09-10T10:04:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38705.897004 secs 2023-09-10T10:04:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:34 smithi082 ceph-mon[98694]: pgmap v20801: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 144 KiB/s wr, 11 op/s 2023-09-10T10:04:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38706.248844 secs 2023-09-10T10:04:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38705.897004 secs 2023-09-10T10:04:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:34 smithi098 ceph-mon[103124]: pgmap v20801: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 144 KiB/s wr, 11 op/s 2023-09-10T10:04:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38706.248844 secs 2023-09-10T10:04:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38705.897004 secs 2023-09-10T10:04:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:34 smithi170 ceph-mon[102832]: pgmap v20801: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 144 KiB/s wr, 11 op/s 2023-09-10T10:04:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38706.248844 secs 2023-09-10T10:04:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:36 smithi082 ceph-mon[98694]: pgmap v20802: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 144 KiB/s wr, 16 op/s 2023-09-10T10:04:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:36 smithi098 ceph-mon[103124]: pgmap v20802: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 144 KiB/s wr, 16 op/s 2023-09-10T10:04:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:36 smithi170 ceph-mon[102832]: pgmap v20802: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 144 KiB/s wr, 16 op/s 2023-09-10T10:04:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:38 smithi098 ceph-mon[103124]: pgmap v20803: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 10 op/s 2023-09-10T10:04:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:38 smithi170 ceph-mon[102832]: pgmap v20803: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 10 op/s 2023-09-10T10:04:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:38 smithi082 ceph-mon[98694]: pgmap v20803: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 10 op/s 2023-09-10T10:04:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38710.897077 secs 2023-09-10T10:04:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38711.248910 secs 2023-09-10T10:04:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:04:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38710.897077 secs 2023-09-10T10:04:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38711.248910 secs 2023-09-10T10:04:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:04:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38710.897077 secs 2023-09-10T10:04:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38711.248910 secs 2023-09-10T10:04:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:04:40.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:40 smithi082 ceph-mon[98694]: pgmap v20804: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 10 op/s 2023-09-10T10:04:40.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:04:40.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:04:40.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:40 smithi098 ceph-mon[103124]: pgmap v20804: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 10 op/s 2023-09-10T10:04:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:04:41.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:04:41.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:40 smithi170 ceph-mon[102832]: pgmap v20804: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 10 op/s 2023-09-10T10:04:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:04:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:04:41.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:42 smithi098 ceph-mon[103124]: pgmap v20805: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 462 KiB/s wr, 16 op/s 2023-09-10T10:04:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:42 smithi170 ceph-mon[102832]: pgmap v20805: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 462 KiB/s wr, 16 op/s 2023-09-10T10:04:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:42 smithi082 ceph-mon[98694]: pgmap v20805: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 462 KiB/s wr, 16 op/s 2023-09-10T10:04:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38715.897156 secs 2023-09-10T10:04:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:44 smithi098 ceph-mon[103124]: pgmap v20806: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 319 KiB/s wr, 11 op/s 2023-09-10T10:04:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38716.248978 secs 2023-09-10T10:04:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38715.897156 secs 2023-09-10T10:04:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:44 smithi170 ceph-mon[102832]: pgmap v20806: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 319 KiB/s wr, 11 op/s 2023-09-10T10:04:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38716.248978 secs 2023-09-10T10:04:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38715.897156 secs 2023-09-10T10:04:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:44 smithi082 ceph-mon[98694]: pgmap v20806: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 319 KiB/s wr, 11 op/s 2023-09-10T10:04:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38716.248978 secs 2023-09-10T10:04:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:04:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:46.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:46 smithi170 ceph-mon[102832]: pgmap v20807: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 627 KiB/s wr, 18 op/s 2023-09-10T10:04:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:04:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:46 smithi082 ceph-mon[98694]: pgmap v20807: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 627 KiB/s wr, 18 op/s 2023-09-10T10:04:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:04:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:46 smithi098 ceph-mon[103124]: pgmap v20807: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 627 KiB/s wr, 18 op/s 2023-09-10T10:04:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:48 smithi170 ceph-mon[102832]: pgmap v20808: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 626 KiB/s wr, 13 op/s 2023-09-10T10:04:49.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:48 smithi082 ceph-mon[98694]: pgmap v20808: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 626 KiB/s wr, 13 op/s 2023-09-10T10:04:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:48 smithi098 ceph-mon[103124]: pgmap v20808: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 626 KiB/s wr, 13 op/s 2023-09-10T10:04:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38720.897214 secs 2023-09-10T10:04:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38721.249041 secs 2023-09-10T10:04:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38720.897214 secs 2023-09-10T10:04:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38721.249041 secs 2023-09-10T10:04:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38720.897214 secs 2023-09-10T10:04:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38721.249041 secs 2023-09-10T10:04:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:50 smithi170 ceph-mon[102832]: pgmap v20809: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 626 KiB/s wr, 13 op/s 2023-09-10T10:04:51.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:04:51.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:50 smithi082 ceph-mon[98694]: pgmap v20809: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 626 KiB/s wr, 13 op/s 2023-09-10T10:04:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:04:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:50 smithi098 ceph-mon[103124]: pgmap v20809: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 626 KiB/s wr, 13 op/s 2023-09-10T10:04:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:04:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:52 smithi170 ceph-mon[102832]: pgmap v20810: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 950 KiB/s wr, 20 op/s 2023-09-10T10:04:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:04:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:52 smithi082 ceph-mon[98694]: pgmap v20810: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 950 KiB/s wr, 20 op/s 2023-09-10T10:04:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:04:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:52 smithi098 ceph-mon[103124]: pgmap v20810: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 950 KiB/s wr, 20 op/s 2023-09-10T10:04:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:04:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38725.897280 secs 2023-09-10T10:04:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:54 smithi170 ceph-mon[102832]: pgmap v20811: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 631 KiB/s wr, 13 op/s 2023-09-10T10:04:55.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38726.249103 secs 2023-09-10T10:04:55.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38725.897280 secs 2023-09-10T10:04:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:54 smithi082 ceph-mon[98694]: pgmap v20811: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 631 KiB/s wr, 13 op/s 2023-09-10T10:04:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38726.249103 secs 2023-09-10T10:04:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38725.897280 secs 2023-09-10T10:04:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:54 smithi098 ceph-mon[103124]: pgmap v20811: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 631 KiB/s wr, 13 op/s 2023-09-10T10:04:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38726.249103 secs 2023-09-10T10:04:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:56 smithi170 ceph-mon[102832]: pgmap v20812: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 844 KiB/s wr, 20 op/s 2023-09-10T10:04:57.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:56 smithi082 ceph-mon[98694]: pgmap v20812: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 844 KiB/s wr, 20 op/s 2023-09-10T10:04:57.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:56 smithi098 ceph-mon[103124]: pgmap v20812: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 844 KiB/s wr, 20 op/s 2023-09-10T10:04:58.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:04:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:04:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:04:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:04:59.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:58 smithi082 ceph-mon[98694]: pgmap v20813: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 536 KiB/s wr, 13 op/s 2023-09-10T10:04:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:58 smithi098 ceph-mon[103124]: pgmap v20813: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 536 KiB/s wr, 13 op/s 2023-09-10T10:04:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:58 smithi170 ceph-mon[102832]: pgmap v20813: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 536 KiB/s wr, 13 op/s 2023-09-10T10:05:00.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38730.897379 secs 2023-09-10T10:05:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:04:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38731.249166 secs 2023-09-10T10:05:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38730.897379 secs 2023-09-10T10:05:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:04:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38731.249166 secs 2023-09-10T10:05:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38730.897379 secs 2023-09-10T10:05:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:04:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38731.249166 secs 2023-09-10T10:05:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:00 smithi082 ceph-mon[98694]: pgmap v20814: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 536 KiB/s wr, 13 op/s 2023-09-10T10:05:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:05:01.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:00 smithi098 ceph-mon[103124]: pgmap v20814: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 536 KiB/s wr, 13 op/s 2023-09-10T10:05:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:05:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:00 smithi170 ceph-mon[102832]: pgmap v20814: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 536 KiB/s wr, 13 op/s 2023-09-10T10:05:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:05:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:02 smithi082 ceph-mon[98694]: pgmap v20815: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 846 KiB/s wr, 35 op/s 2023-09-10T10:05:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:02 smithi098 ceph-mon[103124]: pgmap v20815: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 846 KiB/s wr, 35 op/s 2023-09-10T10:05:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:02 smithi170 ceph-mon[102832]: pgmap v20815: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 846 KiB/s wr, 35 op/s 2023-09-10T10:05:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38734.666613 secs 2023-09-10T10:05:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38734.666613 secs 2023-09-10T10:05:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38734.666613 secs 2023-09-10T10:05:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38735.897436 secs 2023-09-10T10:05:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:04 smithi082 ceph-mon[98694]: pgmap v20816: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 522 KiB/s wr, 28 op/s 2023-09-10T10:05:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:05:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:05:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:05.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:04 smithi082 ceph-mon[98694]: osdmap e1369: 12 total, 12 up, 12 in 2023-09-10T10:05:05.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38736.249228 secs 2023-09-10T10:05:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38735.897436 secs 2023-09-10T10:05:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:04 smithi098 ceph-mon[103124]: pgmap v20816: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 522 KiB/s wr, 28 op/s 2023-09-10T10:05:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:05:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:05:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:04 smithi098 ceph-mon[103124]: osdmap e1369: 12 total, 12 up, 12 in 2023-09-10T10:05:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38736.249228 secs 2023-09-10T10:05:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38735.897436 secs 2023-09-10T10:05:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:04 smithi170 ceph-mon[102832]: pgmap v20816: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 522 KiB/s wr, 28 op/s 2023-09-10T10:05:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:05:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:05:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:04 smithi170 ceph-mon[102832]: osdmap e1369: 12 total, 12 up, 12 in 2023-09-10T10:05:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38736.249228 secs 2023-09-10T10:05:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:06 smithi082 ceph-mon[98694]: pgmap v20818: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 704 KiB/s wr, 33 op/s 2023-09-10T10:05:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:06 smithi098 ceph-mon[103124]: pgmap v20818: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 704 KiB/s wr, 33 op/s 2023-09-10T10:05:07.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:06 smithi170 ceph-mon[102832]: pgmap v20818: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 704 KiB/s wr, 33 op/s 2023-09-10T10:05:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:08 smithi082 ceph-mon[98694]: pgmap v20819: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 704 KiB/s wr, 33 op/s 2023-09-10T10:05:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38739.666700 secs 2023-09-10T10:05:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:08 smithi098 ceph-mon[103124]: pgmap v20819: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 704 KiB/s wr, 33 op/s 2023-09-10T10:05:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38739.666700 secs 2023-09-10T10:05:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:08 smithi170 ceph-mon[102832]: pgmap v20819: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 704 KiB/s wr, 33 op/s 2023-09-10T10:05:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38739.666700 secs 2023-09-10T10:05:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38740.897492 secs 2023-09-10T10:05:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:09 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:05:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:09 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38741.249291 secs 2023-09-10T10:05:10.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:09 smithi082 ceph-mon[98694]: osdmap e1370: 12 total, 12 up, 12 in 2023-09-10T10:05:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38740.897492 secs 2023-09-10T10:05:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:09 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:05:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:09 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38741.249291 secs 2023-09-10T10:05:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:09 smithi098 ceph-mon[103124]: osdmap e1370: 12 total, 12 up, 12 in 2023-09-10T10:05:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38740.897492 secs 2023-09-10T10:05:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:09 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:05:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:09 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38741.249291 secs 2023-09-10T10:05:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:09 smithi170 ceph-mon[102832]: osdmap e1370: 12 total, 12 up, 12 in 2023-09-10T10:05:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:10 smithi082 ceph-mon[98694]: pgmap v20821: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 416 KiB/s wr, 9 op/s 2023-09-10T10:05:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:10 smithi098 ceph-mon[103124]: pgmap v20821: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 416 KiB/s wr, 9 op/s 2023-09-10T10:05:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:10 smithi170 ceph-mon[102832]: pgmap v20821: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 416 KiB/s wr, 9 op/s 2023-09-10T10:05:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:12 smithi098 ceph-mon[103124]: pgmap v20822: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 719 KiB/s wr, 18 op/s 2023-09-10T10:05:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:05:13.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:12 smithi170 ceph-mon[102832]: pgmap v20822: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 719 KiB/s wr, 18 op/s 2023-09-10T10:05:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:05:13.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:12 smithi082 ceph-mon[98694]: pgmap v20822: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 719 KiB/s wr, 18 op/s 2023-09-10T10:05:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:05:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38744.666766 secs 2023-09-10T10:05:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38744.666766 secs 2023-09-10T10:05:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38744.666766 secs 2023-09-10T10:05:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38745.897574 secs 2023-09-10T10:05:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:14 smithi098 ceph-mon[103124]: pgmap v20823: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 581 KiB/s wr, 15 op/s 2023-09-10T10:05:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38746.249355 secs 2023-09-10T10:05:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38745.897574 secs 2023-09-10T10:05:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:14 smithi170 ceph-mon[102832]: pgmap v20823: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 581 KiB/s wr, 15 op/s 2023-09-10T10:05:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38746.249355 secs 2023-09-10T10:05:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38745.897574 secs 2023-09-10T10:05:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:14 smithi082 ceph-mon[98694]: pgmap v20823: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 581 KiB/s wr, 15 op/s 2023-09-10T10:05:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38746.249355 secs 2023-09-10T10:05:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:16 smithi170 ceph-mon[102832]: pgmap v20824: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 415 KiB/s wr, 14 op/s 2023-09-10T10:05:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:16 smithi082 ceph-mon[98694]: pgmap v20824: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 415 KiB/s wr, 14 op/s 2023-09-10T10:05:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:16 smithi098 ceph-mon[103124]: pgmap v20824: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 415 KiB/s wr, 14 op/s 2023-09-10T10:05:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:18 smithi170 ceph-mon[102832]: pgmap v20825: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 415 KiB/s wr, 14 op/s 2023-09-10T10:05:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38749.666776 secs 2023-09-10T10:05:19.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:18 smithi082 ceph-mon[98694]: pgmap v20825: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 415 KiB/s wr, 14 op/s 2023-09-10T10:05:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38749.666776 secs 2023-09-10T10:05:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:18 smithi098 ceph-mon[103124]: pgmap v20825: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 415 KiB/s wr, 14 op/s 2023-09-10T10:05:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38749.666776 secs 2023-09-10T10:05:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38750.897638 secs 2023-09-10T10:05:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:05:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38751.249424 secs 2023-09-10T10:05:20.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:05:20.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:05:20.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:05:20.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38750.897638 secs 2023-09-10T10:05:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:05:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38751.249424 secs 2023-09-10T10:05:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:05:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:05:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:05:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38750.897638 secs 2023-09-10T10:05:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:05:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38751.249424 secs 2023-09-10T10:05:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:05:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:05:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:05:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:20 smithi170 ceph-mon[102832]: pgmap v20826: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 404 KiB/s wr, 14 op/s 2023-09-10T10:05:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:20 smithi082 ceph-mon[98694]: pgmap v20826: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 404 KiB/s wr, 14 op/s 2023-09-10T10:05:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:20 smithi098 ceph-mon[103124]: pgmap v20826: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 404 KiB/s wr, 14 op/s 2023-09-10T10:05:23.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:22 smithi170 ceph-mon[102832]: pgmap v20827: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 346 KiB/s wr, 16 op/s 2023-09-10T10:05:23.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:22 smithi082 ceph-mon[98694]: pgmap v20827: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 346 KiB/s wr, 16 op/s 2023-09-10T10:05:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:22 smithi098 ceph-mon[103124]: pgmap v20827: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 346 KiB/s wr, 16 op/s 2023-09-10T10:05:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38754.666854 secs 2023-09-10T10:05:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38754.666854 secs 2023-09-10T10:05:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38754.666854 secs 2023-09-10T10:05:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38755.897698 secs 2023-09-10T10:05:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:24 smithi170 ceph-mon[102832]: pgmap v20828: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:05:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38756.249489 secs 2023-09-10T10:05:25.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38755.897698 secs 2023-09-10T10:05:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:24 smithi082 ceph-mon[98694]: pgmap v20828: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:05:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38756.249489 secs 2023-09-10T10:05:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38755.897698 secs 2023-09-10T10:05:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:24 smithi098 ceph-mon[103124]: pgmap v20828: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:05:25.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38756.249489 secs 2023-09-10T10:05:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:26 smithi170 ceph-mon[102832]: pgmap v20829: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 288 KiB/s wr, 16 op/s 2023-09-10T10:05:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:26 smithi082 ceph-mon[98694]: pgmap v20829: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 288 KiB/s wr, 16 op/s 2023-09-10T10:05:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:26 smithi098 ceph-mon[103124]: pgmap v20829: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 288 KiB/s wr, 16 op/s 2023-09-10T10:05:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:28 smithi170 ceph-mon[102832]: pgmap v20830: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:05:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38759.666932 secs 2023-09-10T10:05:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:28 smithi082 ceph-mon[98694]: pgmap v20830: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:05:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38759.666932 secs 2023-09-10T10:05:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:28 smithi098 ceph-mon[103124]: pgmap v20830: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:05:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38759.666932 secs 2023-09-10T10:05:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38760.897755 secs 2023-09-10T10:05:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38761.249551 secs 2023-09-10T10:05:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38760.897755 secs 2023-09-10T10:05:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38761.249551 secs 2023-09-10T10:05:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38760.897755 secs 2023-09-10T10:05:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38761.249551 secs 2023-09-10T10:05:31.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:30 smithi082 ceph-mon[98694]: pgmap v20831: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:05:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:30 smithi098 ceph-mon[103124]: pgmap v20831: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:05:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:30 smithi170 ceph-mon[102832]: pgmap v20831: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:05:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:32 smithi082 ceph-mon[98694]: pgmap v20832: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 144 KiB/s wr, 16 op/s 2023-09-10T10:05:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:32 smithi098 ceph-mon[103124]: pgmap v20832: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 144 KiB/s wr, 16 op/s 2023-09-10T10:05:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:32 smithi170 ceph-mon[102832]: pgmap v20832: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 144 KiB/s wr, 16 op/s 2023-09-10T10:05:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:05:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38764.666992 secs 2023-09-10T10:05:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:05:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38764.666992 secs 2023-09-10T10:05:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:05:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38764.666992 secs 2023-09-10T10:05:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38765.897823 secs 2023-09-10T10:05:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:34 smithi082 ceph-mon[98694]: pgmap v20833: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 11 op/s 2023-09-10T10:05:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38766.249615 secs 2023-09-10T10:05:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38765.897823 secs 2023-09-10T10:05:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:34 smithi098 ceph-mon[103124]: pgmap v20833: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 11 op/s 2023-09-10T10:05:35.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38766.249615 secs 2023-09-10T10:05:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38765.897823 secs 2023-09-10T10:05:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:34 smithi170 ceph-mon[102832]: pgmap v20833: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 11 op/s 2023-09-10T10:05:35.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38766.249615 secs 2023-09-10T10:05:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:36 smithi098 ceph-mon[103124]: pgmap v20834: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 385 KiB/s wr, 16 op/s 2023-09-10T10:05:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:05:37.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:36 smithi170 ceph-mon[102832]: pgmap v20834: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 385 KiB/s wr, 16 op/s 2023-09-10T10:05:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:05:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:36 smithi082 ceph-mon[98694]: pgmap v20834: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 385 KiB/s wr, 16 op/s 2023-09-10T10:05:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:05:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:05:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:05:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:05:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:38 smithi098 ceph-mon[103124]: pgmap v20835: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 241 KiB/s wr, 11 op/s 2023-09-10T10:05:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:38 smithi170 ceph-mon[102832]: pgmap v20835: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 241 KiB/s wr, 11 op/s 2023-09-10T10:05:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:38 smithi082 ceph-mon[98694]: pgmap v20835: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 241 KiB/s wr, 11 op/s 2023-09-10T10:05:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38770.897882 secs 2023-09-10T10:05:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38771.249682 secs 2023-09-10T10:05:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:05:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38770.897882 secs 2023-09-10T10:05:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38771.249682 secs 2023-09-10T10:05:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:05:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38770.897882 secs 2023-09-10T10:05:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38771.249682 secs 2023-09-10T10:05:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:05:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:40 smithi098 ceph-mon[103124]: pgmap v20836: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 241 KiB/s wr, 11 op/s 2023-09-10T10:05:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:05:41.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:05:41.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:40 smithi170 ceph-mon[102832]: pgmap v20836: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 241 KiB/s wr, 11 op/s 2023-09-10T10:05:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:05:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:05:41.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:40 smithi082 ceph-mon[98694]: pgmap v20836: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 241 KiB/s wr, 11 op/s 2023-09-10T10:05:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:05:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:05:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:42 smithi098 ceph-mon[103124]: pgmap v20837: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 462 KiB/s wr, 17 op/s 2023-09-10T10:05:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:42 smithi170 ceph-mon[102832]: pgmap v20837: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 462 KiB/s wr, 17 op/s 2023-09-10T10:05:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:42 smithi082 ceph-mon[98694]: pgmap v20837: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 462 KiB/s wr, 17 op/s 2023-09-10T10:05:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:44 smithi170 ceph-mon[102832]: pgmap v20838: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 462 KiB/s wr, 12 op/s 2023-09-10T10:05:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:44 smithi082 ceph-mon[98694]: pgmap v20838: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 462 KiB/s wr, 12 op/s 2023-09-10T10:05:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:44 smithi098 ceph-mon[103124]: pgmap v20838: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 462 KiB/s wr, 12 op/s 2023-09-10T10:05:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38775.897943 secs 2023-09-10T10:05:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38776.249745 secs 2023-09-10T10:05:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38775.897943 secs 2023-09-10T10:05:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38776.249745 secs 2023-09-10T10:05:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38775.897943 secs 2023-09-10T10:05:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38776.249745 secs 2023-09-10T10:05:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:46 smithi170 ceph-mon[102832]: pgmap v20839: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 770 KiB/s wr, 19 op/s 2023-09-10T10:05:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:46 smithi082 ceph-mon[98694]: pgmap v20839: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 770 KiB/s wr, 19 op/s 2023-09-10T10:05:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:46 smithi098 ceph-mon[103124]: pgmap v20839: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 770 KiB/s wr, 19 op/s 2023-09-10T10:05:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:48 smithi170 ceph-mon[102832]: pgmap v20840: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 529 KiB/s wr, 13 op/s 2023-09-10T10:05:49.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38779.667225 secs 2023-09-10T10:05:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:48 smithi082 ceph-mon[98694]: pgmap v20840: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 529 KiB/s wr, 13 op/s 2023-09-10T10:05:49.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38779.667225 secs 2023-09-10T10:05:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:48 smithi098 ceph-mon[103124]: pgmap v20840: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 529 KiB/s wr, 13 op/s 2023-09-10T10:05:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38779.667225 secs 2023-09-10T10:05:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38780.898000 secs 2023-09-10T10:05:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:05:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:50.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38781.249808 secs 2023-09-10T10:05:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38780.898000 secs 2023-09-10T10:05:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:05:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38781.249808 secs 2023-09-10T10:05:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38780.898000 secs 2023-09-10T10:05:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:05:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38781.249808 secs 2023-09-10T10:05:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:50 smithi170 ceph-mon[102832]: pgmap v20841: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 529 KiB/s wr, 13 op/s 2023-09-10T10:05:51.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:50 smithi082 ceph-mon[98694]: pgmap v20841: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 529 KiB/s wr, 13 op/s 2023-09-10T10:05:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:50 smithi098 ceph-mon[103124]: pgmap v20841: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 529 KiB/s wr, 13 op/s 2023-09-10T10:05:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:05:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:05:52.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:05:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:05:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:05:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:05:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:52 smithi098 ceph-mon[103124]: pgmap v20842: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 792 KiB/s wr, 21 op/s 2023-09-10T10:05:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:52 smithi170 ceph-mon[102832]: pgmap v20842: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 792 KiB/s wr, 21 op/s 2023-09-10T10:05:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:52 smithi082 ceph-mon[98694]: pgmap v20842: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 792 KiB/s wr, 21 op/s 2023-09-10T10:05:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38784.667285 secs 2023-09-10T10:05:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:05:54.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:54.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:53 smithi098 ceph-mon[103124]: pgmap v20843: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 572 KiB/s wr, 15 op/s 2023-09-10T10:05:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38784.667285 secs 2023-09-10T10:05:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:05:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:54.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:53 smithi170 ceph-mon[102832]: pgmap v20843: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 572 KiB/s wr, 15 op/s 2023-09-10T10:05:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38784.667285 secs 2023-09-10T10:05:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:05:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:53 smithi082 ceph-mon[98694]: pgmap v20843: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 572 KiB/s wr, 15 op/s 2023-09-10T10:05:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38785.898078 secs 2023-09-10T10:05:55.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38786.249872 secs 2023-09-10T10:05:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38785.898078 secs 2023-09-10T10:05:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38786.249872 secs 2023-09-10T10:05:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38785.898078 secs 2023-09-10T10:05:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38786.249872 secs 2023-09-10T10:05:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:55 smithi098 ceph-mon[103124]: pgmap v20844: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 894 KiB/s wr, 21 op/s 2023-09-10T10:05:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:55 smithi170 ceph-mon[102832]: pgmap v20844: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 894 KiB/s wr, 21 op/s 2023-09-10T10:05:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:55 smithi082 ceph-mon[98694]: pgmap v20844: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 894 KiB/s wr, 21 op/s 2023-09-10T10:05:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:05:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:58.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:57 smithi098 ceph-mon[103124]: pgmap v20845: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 586 KiB/s wr, 14 op/s 2023-09-10T10:05:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:05:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:57 smithi170 ceph-mon[102832]: pgmap v20845: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 586 KiB/s wr, 14 op/s 2023-09-10T10:05:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:05:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:05:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:57 smithi082 ceph-mon[98694]: pgmap v20845: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 586 KiB/s wr, 14 op/s 2023-09-10T10:05:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:05:59 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:05:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:05:59 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:05:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:05:59 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:06:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38790.898173 secs 2023-09-10T10:06:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38791.249937 secs 2023-09-10T10:06:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:00 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:06:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:06:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:00 smithi098 ceph-mon[103124]: pgmap v20846: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 586 KiB/s wr, 14 op/s 2023-09-10T10:06:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38790.898173 secs 2023-09-10T10:06:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38791.249937 secs 2023-09-10T10:06:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:00 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:06:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:06:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:00 smithi170 ceph-mon[102832]: pgmap v20846: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 586 KiB/s wr, 14 op/s 2023-09-10T10:06:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38790.898173 secs 2023-09-10T10:06:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38791.249937 secs 2023-09-10T10:06:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:00 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:06:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:06:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:00.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:00 smithi082 ceph-mon[98694]: pgmap v20846: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 586 KiB/s wr, 14 op/s 2023-09-10T10:06:03.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:02 smithi170 ceph-mon[102832]: pgmap v20847: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 845 KiB/s wr, 24 op/s 2023-09-10T10:06:03.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:02 smithi082 ceph-mon[98694]: pgmap v20847: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 845 KiB/s wr, 24 op/s 2023-09-10T10:06:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:02 smithi098 ceph-mon[103124]: pgmap v20847: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 845 KiB/s wr, 24 op/s 2023-09-10T10:06:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38794.667405 secs 2023-09-10T10:06:04.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38794.667405 secs 2023-09-10T10:06:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38794.667405 secs 2023-09-10T10:06:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38795.898261 secs 2023-09-10T10:06:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:04 smithi082 ceph-mon[98694]: pgmap v20848: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 582 KiB/s wr, 16 op/s 2023-09-10T10:06:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:06:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:05.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38796.250004 secs 2023-09-10T10:06:05.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:06:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38795.898261 secs 2023-09-10T10:06:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:04 smithi098 ceph-mon[103124]: pgmap v20848: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 582 KiB/s wr, 16 op/s 2023-09-10T10:06:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:06:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38796.250004 secs 2023-09-10T10:06:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:06:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38795.898261 secs 2023-09-10T10:06:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:04 smithi170 ceph-mon[102832]: pgmap v20848: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 582 KiB/s wr, 16 op/s 2023-09-10T10:06:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:06:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38796.250004 secs 2023-09-10T10:06:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:06:07.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:06 smithi082 ceph-mon[98694]: pgmap v20849: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 748 KiB/s wr, 34 op/s 2023-09-10T10:06:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:06 smithi098 ceph-mon[103124]: pgmap v20849: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 748 KiB/s wr, 34 op/s 2023-09-10T10:06:07.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:06 smithi170 ceph-mon[102832]: pgmap v20849: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 748 KiB/s wr, 34 op/s 2023-09-10T10:06:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:08 smithi082 ceph-mon[98694]: pgmap v20850: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 425 KiB/s wr, 28 op/s 2023-09-10T10:06:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38799.667465 secs 2023-09-10T10:06:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:08 smithi098 ceph-mon[103124]: pgmap v20850: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 425 KiB/s wr, 28 op/s 2023-09-10T10:06:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38799.667465 secs 2023-09-10T10:06:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:08 smithi170 ceph-mon[102832]: pgmap v20850: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 425 KiB/s wr, 28 op/s 2023-09-10T10:06:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38799.667465 secs 2023-09-10T10:06:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38800.898345 secs 2023-09-10T10:06:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:09 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:06:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:09 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:09 smithi082 ceph-mon[98694]: osdmap e1371: 12 total, 12 up, 12 in 2023-09-10T10:06:10.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38801.250071 secs 2023-09-10T10:06:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38800.898345 secs 2023-09-10T10:06:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:09 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:06:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:09 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:09 smithi098 ceph-mon[103124]: osdmap e1371: 12 total, 12 up, 12 in 2023-09-10T10:06:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38801.250071 secs 2023-09-10T10:06:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38800.898345 secs 2023-09-10T10:06:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:09 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:06:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:09 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:09 smithi170 ceph-mon[102832]: osdmap e1371: 12 total, 12 up, 12 in 2023-09-10T10:06:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38801.250071 secs 2023-09-10T10:06:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:10 smithi082 ceph-mon[98694]: pgmap v20852: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 511 KiB/s wr, 33 op/s 2023-09-10T10:06:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:10 smithi098 ceph-mon[103124]: pgmap v20852: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 511 KiB/s wr, 33 op/s 2023-09-10T10:06:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:10 smithi170 ceph-mon[102832]: pgmap v20852: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 511 KiB/s wr, 33 op/s 2023-09-10T10:06:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:12 smithi082 ceph-mon[98694]: pgmap v20853: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 442 KiB/s wr, 29 op/s 2023-09-10T10:06:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:06:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:12 smithi098 ceph-mon[103124]: pgmap v20853: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 442 KiB/s wr, 29 op/s 2023-09-10T10:06:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:06:13.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:12 smithi170 ceph-mon[102832]: pgmap v20853: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 442 KiB/s wr, 29 op/s 2023-09-10T10:06:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:06:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38804.667526 secs 2023-09-10T10:06:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38804.667526 secs 2023-09-10T10:06:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38804.667526 secs 2023-09-10T10:06:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38805.898432 secs 2023-09-10T10:06:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:14 smithi098 ceph-mon[103124]: pgmap v20854: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 442 KiB/s wr, 29 op/s 2023-09-10T10:06:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38806.250138 secs 2023-09-10T10:06:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:14 smithi098 ceph-mon[103124]: osdmap e1372: 12 total, 12 up, 12 in 2023-09-10T10:06:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38805.898432 secs 2023-09-10T10:06:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:14 smithi170 ceph-mon[102832]: pgmap v20854: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 442 KiB/s wr, 29 op/s 2023-09-10T10:06:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38806.250138 secs 2023-09-10T10:06:15.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:14 smithi170 ceph-mon[102832]: osdmap e1372: 12 total, 12 up, 12 in 2023-09-10T10:06:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38805.898432 secs 2023-09-10T10:06:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:14 smithi082 ceph-mon[98694]: pgmap v20854: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 442 KiB/s wr, 29 op/s 2023-09-10T10:06:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38806.250138 secs 2023-09-10T10:06:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:14 smithi082 ceph-mon[98694]: osdmap e1372: 12 total, 12 up, 12 in 2023-09-10T10:06:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:16 smithi098 ceph-mon[103124]: pgmap v20856: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 519 KiB/s wr, 18 op/s 2023-09-10T10:06:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:16 smithi170 ceph-mon[102832]: pgmap v20856: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 519 KiB/s wr, 18 op/s 2023-09-10T10:06:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:16 smithi082 ceph-mon[98694]: pgmap v20856: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 519 KiB/s wr, 18 op/s 2023-09-10T10:06:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:18 smithi098 ceph-mon[103124]: pgmap v20857: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 465 KiB/s wr, 16 op/s 2023-09-10T10:06:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38809.667587 secs 2023-09-10T10:06:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:18 smithi170 ceph-mon[102832]: pgmap v20857: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 465 KiB/s wr, 16 op/s 2023-09-10T10:06:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38809.667587 secs 2023-09-10T10:06:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:18 smithi082 ceph-mon[98694]: pgmap v20857: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 465 KiB/s wr, 16 op/s 2023-09-10T10:06:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38809.667587 secs 2023-09-10T10:06:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38810.898494 secs 2023-09-10T10:06:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38811.250204 secs 2023-09-10T10:06:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:19 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:06:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38810.898494 secs 2023-09-10T10:06:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38811.250204 secs 2023-09-10T10:06:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:19 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:06:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38810.898494 secs 2023-09-10T10:06:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38811.250204 secs 2023-09-10T10:06:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:19 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:06:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:20 smithi098 ceph-mon[103124]: pgmap v20858: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 415 KiB/s wr, 14 op/s 2023-09-10T10:06:21.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:06:21.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:06:21.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:06:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:20 smithi170 ceph-mon[102832]: pgmap v20858: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 415 KiB/s wr, 14 op/s 2023-09-10T10:06:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:06:21.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:06:21.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:06:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:20 smithi082 ceph-mon[98694]: pgmap v20858: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 415 KiB/s wr, 14 op/s 2023-09-10T10:06:21.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:06:21.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:06:21.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:06:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:22 smithi098 ceph-mon[103124]: pgmap v20859: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 173 KiB/s wr, 13 op/s 2023-09-10T10:06:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:22 smithi170 ceph-mon[102832]: pgmap v20859: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 173 KiB/s wr, 13 op/s 2023-09-10T10:06:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:22 smithi082 ceph-mon[98694]: pgmap v20859: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 173 KiB/s wr, 13 op/s 2023-09-10T10:06:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38814.667672 secs 2023-09-10T10:06:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38814.667672 secs 2023-09-10T10:06:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38814.667672 secs 2023-09-10T10:06:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38815.898555 secs 2023-09-10T10:06:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:24 smithi098 ceph-mon[103124]: pgmap v20860: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 173 KiB/s wr, 13 op/s 2023-09-10T10:06:25.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38816.250271 secs 2023-09-10T10:06:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38815.898555 secs 2023-09-10T10:06:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:24 smithi170 ceph-mon[102832]: pgmap v20860: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 173 KiB/s wr, 13 op/s 2023-09-10T10:06:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38816.250271 secs 2023-09-10T10:06:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38815.898555 secs 2023-09-10T10:06:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:24 smithi082 ceph-mon[98694]: pgmap v20860: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 173 KiB/s wr, 13 op/s 2023-09-10T10:06:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38816.250271 secs 2023-09-10T10:06:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:26 smithi098 ceph-mon[103124]: pgmap v20861: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 307 KiB/s wr, 18 op/s 2023-09-10T10:06:27.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:26 smithi170 ceph-mon[102832]: pgmap v20861: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 307 KiB/s wr, 18 op/s 2023-09-10T10:06:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:26 smithi082 ceph-mon[98694]: pgmap v20861: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 307 KiB/s wr, 18 op/s 2023-09-10T10:06:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:28 smithi098 ceph-mon[103124]: pgmap v20862: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T10:06:29.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38819.667764 secs 2023-09-10T10:06:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:28 smithi170 ceph-mon[102832]: pgmap v20862: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T10:06:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38819.667764 secs 2023-09-10T10:06:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:28 smithi082 ceph-mon[98694]: pgmap v20862: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T10:06:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38819.667764 secs 2023-09-10T10:06:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38820.898613 secs 2023-09-10T10:06:30.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38821.250337 secs 2023-09-10T10:06:30.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:29 smithi098 ceph-mon[103124]: pgmap v20863: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T10:06:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38820.898613 secs 2023-09-10T10:06:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38821.250337 secs 2023-09-10T10:06:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:29 smithi170 ceph-mon[102832]: pgmap v20863: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T10:06:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38820.898613 secs 2023-09-10T10:06:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38821.250337 secs 2023-09-10T10:06:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:29 smithi082 ceph-mon[98694]: pgmap v20863: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T10:06:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:32 smithi082 ceph-mon[98694]: pgmap v20864: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 145 KiB/s wr, 16 op/s 2023-09-10T10:06:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:32 smithi098 ceph-mon[103124]: pgmap v20864: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 145 KiB/s wr, 16 op/s 2023-09-10T10:06:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:32 smithi170 ceph-mon[102832]: pgmap v20864: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 145 KiB/s wr, 16 op/s 2023-09-10T10:06:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:06:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38824.667800 secs 2023-09-10T10:06:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:06:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38824.667800 secs 2023-09-10T10:06:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:06:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38824.667800 secs 2023-09-10T10:06:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38825.898671 secs 2023-09-10T10:06:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:34 smithi082 ceph-mon[98694]: pgmap v20865: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 11 op/s 2023-09-10T10:06:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38826.250402 secs 2023-09-10T10:06:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38825.898671 secs 2023-09-10T10:06:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:34 smithi098 ceph-mon[103124]: pgmap v20865: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 11 op/s 2023-09-10T10:06:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38826.250402 secs 2023-09-10T10:06:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38825.898671 secs 2023-09-10T10:06:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:34 smithi170 ceph-mon[102832]: pgmap v20865: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 11 op/s 2023-09-10T10:06:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38826.250402 secs 2023-09-10T10:06:37.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:36 smithi082 ceph-mon[98694]: pgmap v20866: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 416 KiB/s wr, 17 op/s 2023-09-10T10:06:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:06:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:36 smithi098 ceph-mon[103124]: pgmap v20866: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 416 KiB/s wr, 17 op/s 2023-09-10T10:06:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:06:37.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:36 smithi170 ceph-mon[102832]: pgmap v20866: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 416 KiB/s wr, 17 op/s 2023-09-10T10:06:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:06:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:06:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:06:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:06:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:38 smithi098 ceph-mon[103124]: pgmap v20867: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T10:06:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:38 smithi170 ceph-mon[102832]: pgmap v20867: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T10:06:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:38 smithi082 ceph-mon[98694]: pgmap v20867: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T10:06:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38830.898728 secs 2023-09-10T10:06:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38831.250467 secs 2023-09-10T10:06:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:06:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38830.898728 secs 2023-09-10T10:06:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38831.250467 secs 2023-09-10T10:06:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:06:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38830.898728 secs 2023-09-10T10:06:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38831.250467 secs 2023-09-10T10:06:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:06:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:40 smithi098 ceph-mon[103124]: pgmap v20868: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T10:06:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:06:41.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:06:41.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:40 smithi170 ceph-mon[102832]: pgmap v20868: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T10:06:41.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:06:41.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:06:41.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:40 smithi082 ceph-mon[98694]: pgmap v20868: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 272 KiB/s wr, 11 op/s 2023-09-10T10:06:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:06:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:06:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:42 smithi098 ceph-mon[103124]: pgmap v20869: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 463 KiB/s wr, 18 op/s 2023-09-10T10:06:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:42 smithi170 ceph-mon[102832]: pgmap v20869: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 463 KiB/s wr, 18 op/s 2023-09-10T10:06:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:42 smithi082 ceph-mon[98694]: pgmap v20869: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 463 KiB/s wr, 18 op/s 2023-09-10T10:06:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38835.898787 secs 2023-09-10T10:06:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:44 smithi098 ceph-mon[103124]: pgmap v20870: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 463 KiB/s wr, 12 op/s 2023-09-10T10:06:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38836.250490 secs 2023-09-10T10:06:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38835.898787 secs 2023-09-10T10:06:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:44 smithi170 ceph-mon[102832]: pgmap v20870: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 463 KiB/s wr, 12 op/s 2023-09-10T10:06:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38836.250490 secs 2023-09-10T10:06:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38835.898787 secs 2023-09-10T10:06:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:44 smithi082 ceph-mon[98694]: pgmap v20870: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 463 KiB/s wr, 12 op/s 2023-09-10T10:06:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38836.250490 secs 2023-09-10T10:06:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:46 smithi098 ceph-mon[103124]: pgmap v20871: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 772 KiB/s wr, 19 op/s 2023-09-10T10:06:47.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:46 smithi170 ceph-mon[102832]: pgmap v20871: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 772 KiB/s wr, 19 op/s 2023-09-10T10:06:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:46 smithi082 ceph-mon[98694]: pgmap v20871: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 772 KiB/s wr, 19 op/s 2023-09-10T10:06:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:48 smithi098 ceph-mon[103124]: pgmap v20872: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 500 KiB/s wr, 13 op/s 2023-09-10T10:06:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38839.667993 secs 2023-09-10T10:06:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:48 smithi170 ceph-mon[102832]: pgmap v20872: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 500 KiB/s wr, 13 op/s 2023-09-10T10:06:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38839.667993 secs 2023-09-10T10:06:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:48 smithi082 ceph-mon[98694]: pgmap v20872: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 500 KiB/s wr, 13 op/s 2023-09-10T10:06:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38839.667993 secs 2023-09-10T10:06:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38840.898847 secs 2023-09-10T10:06:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:06:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38841.250553 secs 2023-09-10T10:06:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38840.898847 secs 2023-09-10T10:06:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:06:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38841.250553 secs 2023-09-10T10:06:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38840.898847 secs 2023-09-10T10:06:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:06:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38841.250553 secs 2023-09-10T10:06:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:50 smithi098 ceph-mon[103124]: pgmap v20873: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 500 KiB/s wr, 13 op/s 2023-09-10T10:06:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:50 smithi170 ceph-mon[102832]: pgmap v20873: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 500 KiB/s wr, 13 op/s 2023-09-10T10:06:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:50 smithi082 ceph-mon[98694]: pgmap v20873: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 500 KiB/s wr, 13 op/s 2023-09-10T10:06:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:52 smithi170 ceph-mon[102832]: pgmap v20874: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 773 KiB/s wr, 20 op/s 2023-09-10T10:06:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:06:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:06:52.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:52 smithi082 ceph-mon[98694]: pgmap v20874: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 773 KiB/s wr, 20 op/s 2023-09-10T10:06:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:06:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:06:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:52 smithi098 ceph-mon[103124]: pgmap v20874: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 773 KiB/s wr, 20 op/s 2023-09-10T10:06:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:06:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:06:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38844.668084 secs 2023-09-10T10:06:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38844.668084 secs 2023-09-10T10:06:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38844.668084 secs 2023-09-10T10:06:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:54 smithi170 ceph-mon[102832]: pgmap v20875: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 582 KiB/s wr, 14 op/s 2023-09-10T10:06:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:54 smithi082 ceph-mon[98694]: pgmap v20875: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 582 KiB/s wr, 14 op/s 2023-09-10T10:06:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:54 smithi098 ceph-mon[103124]: pgmap v20875: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 582 KiB/s wr, 14 op/s 2023-09-10T10:06:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38845.898904 secs 2023-09-10T10:06:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38846.250618 secs 2023-09-10T10:06:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38845.898904 secs 2023-09-10T10:06:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38846.250618 secs 2023-09-10T10:06:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38845.898904 secs 2023-09-10T10:06:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38846.250618 secs 2023-09-10T10:06:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:56 smithi170 ceph-mon[102832]: pgmap v20876: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 874 KiB/s wr, 20 op/s 2023-09-10T10:06:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:56 smithi082 ceph-mon[98694]: pgmap v20876: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 874 KiB/s wr, 20 op/s 2023-09-10T10:06:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:56 smithi098 ceph-mon[103124]: pgmap v20876: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 874 KiB/s wr, 20 op/s 2023-09-10T10:06:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:06:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:58 smithi170 ceph-mon[102832]: pgmap v20877: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 565 KiB/s wr, 13 op/s 2023-09-10T10:06:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:06:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:58 smithi082 ceph-mon[98694]: pgmap v20877: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 565 KiB/s wr, 13 op/s 2023-09-10T10:06:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:06:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:06:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:58 smithi098 ceph-mon[103124]: pgmap v20877: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 565 KiB/s wr, 13 op/s 2023-09-10T10:06:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:06:59 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:06:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:06:59 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:06:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:06:59 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:07:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38850.898988 secs 2023-09-10T10:07:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38851.250683 secs 2023-09-10T10:07:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:00 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:07:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:07:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:00 smithi170 ceph-mon[102832]: pgmap v20878: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 565 KiB/s wr, 13 op/s 2023-09-10T10:07:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38850.898988 secs 2023-09-10T10:07:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38851.250683 secs 2023-09-10T10:07:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:00 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:07:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:07:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:00 smithi082 ceph-mon[98694]: pgmap v20878: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 565 KiB/s wr, 13 op/s 2023-09-10T10:07:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38850.898988 secs 2023-09-10T10:07:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38851.250683 secs 2023-09-10T10:07:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:00 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:07:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:07:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:00 smithi098 ceph-mon[103124]: pgmap v20878: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 565 KiB/s wr, 13 op/s 2023-09-10T10:07:03.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:02 smithi082 ceph-mon[98694]: pgmap v20879: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 831 KiB/s wr, 20 op/s 2023-09-10T10:07:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:02 smithi098 ceph-mon[103124]: pgmap v20879: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 831 KiB/s wr, 20 op/s 2023-09-10T10:07:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:02 smithi170 ceph-mon[102832]: pgmap v20879: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 831 KiB/s wr, 20 op/s 2023-09-10T10:07:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38854.668209 secs 2023-09-10T10:07:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38854.668209 secs 2023-09-10T10:07:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38854.668209 secs 2023-09-10T10:07:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38855.899052 secs 2023-09-10T10:07:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:04 smithi082 ceph-mon[98694]: pgmap v20880: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 557 KiB/s wr, 13 op/s 2023-09-10T10:07:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:07:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:05.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38856.250750 secs 2023-09-10T10:07:05.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:07:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38855.899052 secs 2023-09-10T10:07:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:04 smithi098 ceph-mon[103124]: pgmap v20880: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 557 KiB/s wr, 13 op/s 2023-09-10T10:07:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:07:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38856.250750 secs 2023-09-10T10:07:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:07:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38855.899052 secs 2023-09-10T10:07:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:04 smithi170 ceph-mon[102832]: pgmap v20880: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 557 KiB/s wr, 13 op/s 2023-09-10T10:07:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:07:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38856.250750 secs 2023-09-10T10:07:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:07:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:06 smithi082 ceph-mon[98694]: pgmap v20881: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 740 KiB/s wr, 34 op/s 2023-09-10T10:07:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:06 smithi098 ceph-mon[103124]: pgmap v20881: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 740 KiB/s wr, 34 op/s 2023-09-10T10:07:07.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:06 smithi170 ceph-mon[102832]: pgmap v20881: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 740 KiB/s wr, 34 op/s 2023-09-10T10:07:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:08 smithi082 ceph-mon[98694]: pgmap v20882: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 449 KiB/s wr, 28 op/s 2023-09-10T10:07:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38859.668270 secs 2023-09-10T10:07:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:08 smithi098 ceph-mon[103124]: pgmap v20882: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 449 KiB/s wr, 28 op/s 2023-09-10T10:07:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38859.668270 secs 2023-09-10T10:07:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:08 smithi170 ceph-mon[102832]: pgmap v20882: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 449 KiB/s wr, 28 op/s 2023-09-10T10:07:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38859.668270 secs 2023-09-10T10:07:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38860.899112 secs 2023-09-10T10:07:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:09 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:07:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:09 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:09 smithi098 ceph-mon[103124]: osdmap e1373: 12 total, 12 up, 12 in 2023-09-10T10:07:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38861.250816 secs 2023-09-10T10:07:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38860.899112 secs 2023-09-10T10:07:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:09 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:07:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:09 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:09 smithi170 ceph-mon[102832]: osdmap e1373: 12 total, 12 up, 12 in 2023-09-10T10:07:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38861.250816 secs 2023-09-10T10:07:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38860.899112 secs 2023-09-10T10:07:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:09 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:07:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:09 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:09 smithi082 ceph-mon[98694]: osdmap e1373: 12 total, 12 up, 12 in 2023-09-10T10:07:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38861.250816 secs 2023-09-10T10:07:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:10 smithi098 ceph-mon[103124]: pgmap v20884: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 538 KiB/s wr, 34 op/s 2023-09-10T10:07:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:10 smithi170 ceph-mon[102832]: pgmap v20884: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 538 KiB/s wr, 34 op/s 2023-09-10T10:07:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:10 smithi082 ceph-mon[98694]: pgmap v20884: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 538 KiB/s wr, 34 op/s 2023-09-10T10:07:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:12 smithi098 ceph-mon[103124]: pgmap v20885: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 463 KiB/s wr, 34 op/s 2023-09-10T10:07:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:07:13.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:12 smithi170 ceph-mon[102832]: pgmap v20885: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 463 KiB/s wr, 34 op/s 2023-09-10T10:07:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:07:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:12 smithi082 ceph-mon[98694]: pgmap v20885: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 463 KiB/s wr, 34 op/s 2023-09-10T10:07:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:07:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38864.668335 secs 2023-09-10T10:07:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38864.668335 secs 2023-09-10T10:07:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38864.668335 secs 2023-09-10T10:07:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38865.899170 secs 2023-09-10T10:07:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:14 smithi098 ceph-mon[103124]: pgmap v20886: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 463 KiB/s wr, 34 op/s 2023-09-10T10:07:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38866.250880 secs 2023-09-10T10:07:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:14 smithi098 ceph-mon[103124]: osdmap e1374: 12 total, 12 up, 12 in 2023-09-10T10:07:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38865.899170 secs 2023-09-10T10:07:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:14 smithi170 ceph-mon[102832]: pgmap v20886: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 463 KiB/s wr, 34 op/s 2023-09-10T10:07:15.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38866.250880 secs 2023-09-10T10:07:15.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:14 smithi170 ceph-mon[102832]: osdmap e1374: 12 total, 12 up, 12 in 2023-09-10T10:07:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38865.899170 secs 2023-09-10T10:07:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:14 smithi082 ceph-mon[98694]: pgmap v20886: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 463 KiB/s wr, 34 op/s 2023-09-10T10:07:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38866.250880 secs 2023-09-10T10:07:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:14 smithi082 ceph-mon[98694]: osdmap e1374: 12 total, 12 up, 12 in 2023-09-10T10:07:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:16 smithi098 ceph-mon[103124]: pgmap v20888: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 477 KiB/s wr, 19 op/s 2023-09-10T10:07:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:16 smithi170 ceph-mon[102832]: pgmap v20888: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 477 KiB/s wr, 19 op/s 2023-09-10T10:07:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:16 smithi082 ceph-mon[98694]: pgmap v20888: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 477 KiB/s wr, 19 op/s 2023-09-10T10:07:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:18 smithi098 ceph-mon[103124]: pgmap v20889: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 427 KiB/s wr, 17 op/s 2023-09-10T10:07:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38869.668393 secs 2023-09-10T10:07:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:18 smithi170 ceph-mon[102832]: pgmap v20889: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 427 KiB/s wr, 17 op/s 2023-09-10T10:07:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38869.668393 secs 2023-09-10T10:07:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:18 smithi082 ceph-mon[98694]: pgmap v20889: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 427 KiB/s wr, 17 op/s 2023-09-10T10:07:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38869.668393 secs 2023-09-10T10:07:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38870.899256 secs 2023-09-10T10:07:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38871.250943 secs 2023-09-10T10:07:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38870.899256 secs 2023-09-10T10:07:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38871.250943 secs 2023-09-10T10:07:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38870.899256 secs 2023-09-10T10:07:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38871.250943 secs 2023-09-10T10:07:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:20 smithi098 ceph-mon[103124]: pgmap v20890: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 381 KiB/s wr, 15 op/s 2023-09-10T10:07:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:07:21.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:07:21.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:07:21.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:20 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:07:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:20 smithi170 ceph-mon[102832]: pgmap v20890: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 381 KiB/s wr, 15 op/s 2023-09-10T10:07:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:07:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:07:21.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:07:21.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:20 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:07:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:20 smithi082 ceph-mon[98694]: pgmap v20890: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 381 KiB/s wr, 15 op/s 2023-09-10T10:07:21.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:07:21.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:07:21.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:07:21.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:20 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:07:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:22 smithi170 ceph-mon[102832]: pgmap v20891: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 13 op/s 2023-09-10T10:07:22.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:22 smithi082 ceph-mon[98694]: pgmap v20891: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 13 op/s 2023-09-10T10:07:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:22 smithi098 ceph-mon[103124]: pgmap v20891: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 13 op/s 2023-09-10T10:07:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38874.668476 secs 2023-09-10T10:07:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38874.668476 secs 2023-09-10T10:07:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38874.668476 secs 2023-09-10T10:07:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:24 smithi170 ceph-mon[102832]: pgmap v20892: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 13 op/s 2023-09-10T10:07:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:24 smithi082 ceph-mon[98694]: pgmap v20892: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 13 op/s 2023-09-10T10:07:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:24 smithi098 ceph-mon[103124]: pgmap v20892: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 13 op/s 2023-09-10T10:07:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38875.899313 secs 2023-09-10T10:07:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38876.251008 secs 2023-09-10T10:07:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38875.899313 secs 2023-09-10T10:07:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38876.251008 secs 2023-09-10T10:07:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38875.899313 secs 2023-09-10T10:07:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38876.251008 secs 2023-09-10T10:07:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:26 smithi170 ceph-mon[102832]: pgmap v20893: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 276 KiB/s wr, 18 op/s 2023-09-10T10:07:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:26 smithi082 ceph-mon[98694]: pgmap v20893: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 276 KiB/s wr, 18 op/s 2023-09-10T10:07:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:26 smithi098 ceph-mon[103124]: pgmap v20893: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 276 KiB/s wr, 18 op/s 2023-09-10T10:07:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:28 smithi082 ceph-mon[98694]: pgmap v20894: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 10 op/s 2023-09-10T10:07:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38879.668577 secs 2023-09-10T10:07:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:28 smithi098 ceph-mon[103124]: pgmap v20894: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 10 op/s 2023-09-10T10:07:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38879.668577 secs 2023-09-10T10:07:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:28 smithi170 ceph-mon[102832]: pgmap v20894: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 10 op/s 2023-09-10T10:07:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38879.668577 secs 2023-09-10T10:07:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38880.899372 secs 2023-09-10T10:07:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38881.251073 secs 2023-09-10T10:07:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38880.899372 secs 2023-09-10T10:07:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38881.251073 secs 2023-09-10T10:07:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38880.899372 secs 2023-09-10T10:07:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38881.251073 secs 2023-09-10T10:07:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:30 smithi082 ceph-mon[98694]: pgmap v20895: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 10 op/s 2023-09-10T10:07:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:30 smithi098 ceph-mon[103124]: pgmap v20895: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 10 op/s 2023-09-10T10:07:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:30 smithi170 ceph-mon[102832]: pgmap v20895: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 10 op/s 2023-09-10T10:07:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:32 smithi082 ceph-mon[98694]: pgmap v20896: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 145 KiB/s wr, 16 op/s 2023-09-10T10:07:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:32 smithi098 ceph-mon[103124]: pgmap v20896: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 145 KiB/s wr, 16 op/s 2023-09-10T10:07:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:32 smithi170 ceph-mon[102832]: pgmap v20896: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 145 KiB/s wr, 16 op/s 2023-09-10T10:07:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:07:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38884.668639 secs 2023-09-10T10:07:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:07:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38884.668639 secs 2023-09-10T10:07:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:07:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38884.668639 secs 2023-09-10T10:07:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38885.899434 secs 2023-09-10T10:07:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:34 smithi098 ceph-mon[103124]: pgmap v20897: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T10:07:35.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38886.251140 secs 2023-09-10T10:07:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38885.899434 secs 2023-09-10T10:07:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:34 smithi170 ceph-mon[102832]: pgmap v20897: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T10:07:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38886.251140 secs 2023-09-10T10:07:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38885.899434 secs 2023-09-10T10:07:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:34 smithi082 ceph-mon[98694]: pgmap v20897: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T10:07:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38886.251140 secs 2023-09-10T10:07:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:36 smithi098 ceph-mon[103124]: pgmap v20898: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 285 KiB/s wr, 16 op/s 2023-09-10T10:07:37.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:07:37.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:36 smithi170 ceph-mon[102832]: pgmap v20898: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 285 KiB/s wr, 16 op/s 2023-09-10T10:07:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:07:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:36 smithi082 ceph-mon[98694]: pgmap v20898: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 285 KiB/s wr, 16 op/s 2023-09-10T10:07:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:07:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:38 smithi098 ceph-mon[103124]: pgmap v20899: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 10 op/s 2023-09-10T10:07:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:38 smithi170 ceph-mon[102832]: pgmap v20899: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 10 op/s 2023-09-10T10:07:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:38 smithi082 ceph-mon[98694]: pgmap v20899: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 10 op/s 2023-09-10T10:07:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38890.899494 secs 2023-09-10T10:07:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38891.251206 secs 2023-09-10T10:07:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:07:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38890.899494 secs 2023-09-10T10:07:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38891.251206 secs 2023-09-10T10:07:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:07:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38890.899494 secs 2023-09-10T10:07:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38891.251206 secs 2023-09-10T10:07:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:07:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:40 smithi098 ceph-mon[103124]: pgmap v20900: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 10 op/s 2023-09-10T10:07:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:07:41.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:07:41.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:40 smithi170 ceph-mon[102832]: pgmap v20900: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 10 op/s 2023-09-10T10:07:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:07:41.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:07:41.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:40 smithi082 ceph-mon[98694]: pgmap v20900: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 10 op/s 2023-09-10T10:07:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:07:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:07:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:42 smithi098 ceph-mon[103124]: pgmap v20901: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 464 KiB/s wr, 17 op/s 2023-09-10T10:07:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:42 smithi170 ceph-mon[102832]: pgmap v20901: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 464 KiB/s wr, 17 op/s 2023-09-10T10:07:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:42 smithi082 ceph-mon[98694]: pgmap v20901: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 464 KiB/s wr, 17 op/s 2023-09-10T10:07:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38895.899578 secs 2023-09-10T10:07:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:44 smithi098 ceph-mon[103124]: pgmap v20902: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 464 KiB/s wr, 12 op/s 2023-09-10T10:07:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38896.251272 secs 2023-09-10T10:07:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38895.899578 secs 2023-09-10T10:07:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:44 smithi170 ceph-mon[102832]: pgmap v20902: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 464 KiB/s wr, 12 op/s 2023-09-10T10:07:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38896.251272 secs 2023-09-10T10:07:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38895.899578 secs 2023-09-10T10:07:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:44 smithi082 ceph-mon[98694]: pgmap v20902: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 464 KiB/s wr, 12 op/s 2023-09-10T10:07:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38896.251272 secs 2023-09-10T10:07:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:07:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:46 smithi170 ceph-mon[102832]: pgmap v20903: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 773 KiB/s wr, 19 op/s 2023-09-10T10:07:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:07:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:46.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:46 smithi082 ceph-mon[98694]: pgmap v20903: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 773 KiB/s wr, 19 op/s 2023-09-10T10:07:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:07:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:47.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:46 smithi098 ceph-mon[103124]: pgmap v20903: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 773 KiB/s wr, 19 op/s 2023-09-10T10:07:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:48 smithi098 ceph-mon[103124]: pgmap v20904: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 633 KiB/s wr, 13 op/s 2023-09-10T10:07:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38899.668930 secs 2023-09-10T10:07:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:48 smithi170 ceph-mon[102832]: pgmap v20904: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 633 KiB/s wr, 13 op/s 2023-09-10T10:07:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38899.668930 secs 2023-09-10T10:07:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:48 smithi082 ceph-mon[98694]: pgmap v20904: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 633 KiB/s wr, 13 op/s 2023-09-10T10:07:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38899.668930 secs 2023-09-10T10:07:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38900.899636 secs 2023-09-10T10:07:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:07:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38901.251357 secs 2023-09-10T10:07:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38900.899636 secs 2023-09-10T10:07:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:07:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38901.251357 secs 2023-09-10T10:07:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38900.899636 secs 2023-09-10T10:07:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:07:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38901.251357 secs 2023-09-10T10:07:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:50 smithi098 ceph-mon[103124]: pgmap v20905: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 633 KiB/s wr, 13 op/s 2023-09-10T10:07:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:50 smithi170 ceph-mon[102832]: pgmap v20905: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 633 KiB/s wr, 13 op/s 2023-09-10T10:07:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:50 smithi082 ceph-mon[98694]: pgmap v20905: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 633 KiB/s wr, 13 op/s 2023-09-10T10:07:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:52 smithi170 ceph-mon[102832]: pgmap v20906: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 931 KiB/s wr, 20 op/s 2023-09-10T10:07:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:07:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:07:52.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:52 smithi082 ceph-mon[98694]: pgmap v20906: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 931 KiB/s wr, 20 op/s 2023-09-10T10:07:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:07:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:07:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:52 smithi098 ceph-mon[103124]: pgmap v20906: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 931 KiB/s wr, 20 op/s 2023-09-10T10:07:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:07:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:07:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38904.668993 secs 2023-09-10T10:07:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38904.668993 secs 2023-09-10T10:07:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38904.668993 secs 2023-09-10T10:07:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:54 smithi170 ceph-mon[102832]: pgmap v20907: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 607 KiB/s wr, 13 op/s 2023-09-10T10:07:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:54 smithi082 ceph-mon[98694]: pgmap v20907: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 607 KiB/s wr, 13 op/s 2023-09-10T10:07:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:54 smithi098 ceph-mon[103124]: pgmap v20907: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 607 KiB/s wr, 13 op/s 2023-09-10T10:07:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38905.899695 secs 2023-09-10T10:07:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38906.251421 secs 2023-09-10T10:07:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38905.899695 secs 2023-09-10T10:07:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38906.251421 secs 2023-09-10T10:07:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38905.899695 secs 2023-09-10T10:07:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38906.251421 secs 2023-09-10T10:07:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:56 smithi170 ceph-mon[102832]: pgmap v20908: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 876 KiB/s wr, 20 op/s 2023-09-10T10:07:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:56 smithi082 ceph-mon[98694]: pgmap v20908: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 876 KiB/s wr, 20 op/s 2023-09-10T10:07:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:56 smithi098 ceph-mon[103124]: pgmap v20908: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 876 KiB/s wr, 20 op/s 2023-09-10T10:07:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:07:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:58 smithi170 ceph-mon[102832]: pgmap v20909: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 566 KiB/s wr, 13 op/s 2023-09-10T10:07:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:07:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38909.669054 secs 2023-09-10T10:07:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:07:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:58 smithi082 ceph-mon[98694]: pgmap v20909: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 566 KiB/s wr, 13 op/s 2023-09-10T10:07:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:07:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38909.669054 secs 2023-09-10T10:07:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:07:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:07:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:58 smithi098 ceph-mon[103124]: pgmap v20909: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 566 KiB/s wr, 13 op/s 2023-09-10T10:07:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:07:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38909.669054 secs 2023-09-10T10:08:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38910.899788 secs 2023-09-10T10:08:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38911.251486 secs 2023-09-10T10:08:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:08:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:00 smithi170 ceph-mon[102832]: pgmap v20910: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 566 KiB/s wr, 13 op/s 2023-09-10T10:08:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38910.899788 secs 2023-09-10T10:08:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38911.251486 secs 2023-09-10T10:08:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:08:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:00 smithi082 ceph-mon[98694]: pgmap v20910: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 566 KiB/s wr, 13 op/s 2023-09-10T10:08:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38910.899788 secs 2023-09-10T10:08:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38911.251486 secs 2023-09-10T10:08:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:08:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:00 smithi098 ceph-mon[103124]: pgmap v20910: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 566 KiB/s wr, 13 op/s 2023-09-10T10:08:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:02 smithi098 ceph-mon[103124]: pgmap v20911: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 832 KiB/s wr, 20 op/s 2023-09-10T10:08:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:02 smithi170 ceph-mon[102832]: pgmap v20911: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 832 KiB/s wr, 20 op/s 2023-09-10T10:08:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:02 smithi082 ceph-mon[98694]: pgmap v20911: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 832 KiB/s wr, 20 op/s 2023-09-10T10:08:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38914.669135 secs 2023-09-10T10:08:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38914.669135 secs 2023-09-10T10:08:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38914.669135 secs 2023-09-10T10:08:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38915.899846 secs 2023-09-10T10:08:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:04 smithi098 ceph-mon[103124]: pgmap v20912: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 534 KiB/s wr, 13 op/s 2023-09-10T10:08:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38916.251549 secs 2023-09-10T10:08:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38915.899846 secs 2023-09-10T10:08:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:04 smithi170 ceph-mon[102832]: pgmap v20912: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 534 KiB/s wr, 13 op/s 2023-09-10T10:08:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38916.251549 secs 2023-09-10T10:08:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38915.899846 secs 2023-09-10T10:08:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:04 smithi082 ceph-mon[98694]: pgmap v20912: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 534 KiB/s wr, 13 op/s 2023-09-10T10:08:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38916.251549 secs 2023-09-10T10:08:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:06 smithi098 ceph-mon[103124]: pgmap v20913: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 718 KiB/s wr, 34 op/s 2023-09-10T10:08:07.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:06 smithi170 ceph-mon[102832]: pgmap v20913: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 718 KiB/s wr, 34 op/s 2023-09-10T10:08:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:06 smithi082 ceph-mon[98694]: pgmap v20913: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 718 KiB/s wr, 34 op/s 2023-09-10T10:08:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:08 smithi098 ceph-mon[103124]: pgmap v20914: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 450 KiB/s wr, 28 op/s 2023-09-10T10:08:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38919.669191 secs 2023-09-10T10:08:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:08 smithi170 ceph-mon[102832]: pgmap v20914: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 450 KiB/s wr, 28 op/s 2023-09-10T10:08:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38919.669191 secs 2023-09-10T10:08:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:08 smithi082 ceph-mon[98694]: pgmap v20914: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 450 KiB/s wr, 28 op/s 2023-09-10T10:08:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38919.669191 secs 2023-09-10T10:08:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38920.899928 secs 2023-09-10T10:08:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:09 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:08:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:09 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:09 smithi098 ceph-mon[103124]: osdmap e1375: 12 total, 12 up, 12 in 2023-09-10T10:08:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38921.251614 secs 2023-09-10T10:08:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38920.899928 secs 2023-09-10T10:08:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:09 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:08:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:09 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:09 smithi170 ceph-mon[102832]: osdmap e1375: 12 total, 12 up, 12 in 2023-09-10T10:08:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38921.251614 secs 2023-09-10T10:08:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38920.899928 secs 2023-09-10T10:08:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:09 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:08:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:09 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:09 smithi082 ceph-mon[98694]: osdmap e1375: 12 total, 12 up, 12 in 2023-09-10T10:08:10.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38921.251614 secs 2023-09-10T10:08:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:10 smithi098 ceph-mon[103124]: pgmap v20916: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 540 KiB/s wr, 34 op/s 2023-09-10T10:08:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:10 smithi170 ceph-mon[102832]: pgmap v20916: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 540 KiB/s wr, 34 op/s 2023-09-10T10:08:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:10 smithi082 ceph-mon[98694]: pgmap v20916: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 540 KiB/s wr, 34 op/s 2023-09-10T10:08:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:12 smithi098 ceph-mon[103124]: pgmap v20917: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 493 KiB/s wr, 33 op/s 2023-09-10T10:08:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:08:13.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:12 smithi170 ceph-mon[102832]: pgmap v20917: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 493 KiB/s wr, 33 op/s 2023-09-10T10:08:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:08:13.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:12 smithi082 ceph-mon[98694]: pgmap v20917: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 493 KiB/s wr, 33 op/s 2023-09-10T10:08:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:08:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38924.669251 secs 2023-09-10T10:08:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38924.669251 secs 2023-09-10T10:08:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38924.669251 secs 2023-09-10T10:08:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38925.899988 secs 2023-09-10T10:08:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:14 smithi098 ceph-mon[103124]: pgmap v20918: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 493 KiB/s wr, 33 op/s 2023-09-10T10:08:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38926.251679 secs 2023-09-10T10:08:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:14 smithi098 ceph-mon[103124]: osdmap e1376: 12 total, 12 up, 12 in 2023-09-10T10:08:15.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38925.899988 secs 2023-09-10T10:08:15.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:14 smithi170 ceph-mon[102832]: pgmap v20918: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 493 KiB/s wr, 33 op/s 2023-09-10T10:08:15.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38926.251679 secs 2023-09-10T10:08:15.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:14 smithi170 ceph-mon[102832]: osdmap e1376: 12 total, 12 up, 12 in 2023-09-10T10:08:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38925.899988 secs 2023-09-10T10:08:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:14 smithi082 ceph-mon[98694]: pgmap v20918: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 493 KiB/s wr, 33 op/s 2023-09-10T10:08:15.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38926.251679 secs 2023-09-10T10:08:15.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:14 smithi082 ceph-mon[98694]: osdmap e1376: 12 total, 12 up, 12 in 2023-09-10T10:08:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:17 smithi098 ceph-mon[103124]: pgmap v20920: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 513 KiB/s wr, 18 op/s 2023-09-10T10:08:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:17 smithi170 ceph-mon[102832]: pgmap v20920: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 513 KiB/s wr, 18 op/s 2023-09-10T10:08:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:17 smithi082 ceph-mon[98694]: pgmap v20920: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 513 KiB/s wr, 18 op/s 2023-09-10T10:08:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:18 smithi170 ceph-mon[102832]: pgmap v20921: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 460 KiB/s wr, 16 op/s 2023-09-10T10:08:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:18 smithi082 ceph-mon[98694]: pgmap v20921: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 460 KiB/s wr, 16 op/s 2023-09-10T10:08:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:18 smithi098 ceph-mon[103124]: pgmap v20921: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 460 KiB/s wr, 16 op/s 2023-09-10T10:08:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38929.669309 secs 2023-09-10T10:08:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38929.669309 secs 2023-09-10T10:08:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38929.669309 secs 2023-09-10T10:08:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38930.900071 secs 2023-09-10T10:08:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38931.251743 secs 2023-09-10T10:08:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:20 smithi170 ceph-mon[102832]: pgmap v20922: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 411 KiB/s wr, 14 op/s 2023-09-10T10:08:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38930.900071 secs 2023-09-10T10:08:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38931.251743 secs 2023-09-10T10:08:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:20 smithi082 ceph-mon[98694]: pgmap v20922: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 411 KiB/s wr, 14 op/s 2023-09-10T10:08:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38930.900071 secs 2023-09-10T10:08:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38931.251743 secs 2023-09-10T10:08:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:20 smithi098 ceph-mon[103124]: pgmap v20922: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 411 KiB/s wr, 14 op/s 2023-09-10T10:08:21.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:21 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:08:21.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:21 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:08:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:21 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:08:22.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:22 smithi170 ceph-mon[102832]: pgmap v20923: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 13 op/s 2023-09-10T10:08:22.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:22 smithi082 ceph-mon[98694]: pgmap v20923: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 13 op/s 2023-09-10T10:08:23.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:22 smithi098 ceph-mon[103124]: pgmap v20923: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 13 op/s 2023-09-10T10:08:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38934.669378 secs 2023-09-10T10:08:23.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38934.669378 secs 2023-09-10T10:08:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38934.669378 secs 2023-09-10T10:08:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38935.900144 secs 2023-09-10T10:08:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:08:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:08:25.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:08:25.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:08:25.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:24 smithi098 ceph-mon[103124]: pgmap v20924: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 13 op/s 2023-09-10T10:08:25.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38936.251807 secs 2023-09-10T10:08:25.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:08:25.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:08:25.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:08:25.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:08:25.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:08:25.007 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:08:25.008 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:08:25.008 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:08:25.008 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:08:25.008 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:08:25.008 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:08:25.009 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:08:25.009 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:08:25.009 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:08:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38935.900144 secs 2023-09-10T10:08:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:08:25.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:08:25.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:08:25.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:08:25.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:24 smithi170 ceph-mon[102832]: pgmap v20924: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 13 op/s 2023-09-10T10:08:25.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38936.251807 secs 2023-09-10T10:08:25.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:08:25.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:08:25.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:08:25.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:08:25.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:08:25.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:08:25.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:08:25.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:08:25.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:08:25.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:08:25.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:08:25.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:08:25.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:08:25.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:08:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38935.900144 secs 2023-09-10T10:08:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:08:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:08:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:08:25.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:08:25.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:24 smithi082 ceph-mon[98694]: pgmap v20924: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 138 KiB/s wr, 13 op/s 2023-09-10T10:08:25.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38936.251807 secs 2023-09-10T10:08:25.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:08:25.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:08:25.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:08:25.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:08:25.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:08:25.210 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:08:25.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:08:25.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:08:25.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:08:25.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:08:25.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:08:25.212 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:08:25.212 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:08:25.212 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:08:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:25 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T10:08:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:25 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T10:08:26.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:25 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T10:08:26.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:25 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T10:08:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:25 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T10:08:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:25 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T10:08:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:25 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T10:08:26.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:25 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T10:08:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:25 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T10:08:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:25 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T10:08:26.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:25 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T10:08:26.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:25 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T10:08:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:26 smithi170 ceph-mon[102832]: pgmap v20925: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 18 op/s 2023-09-10T10:08:27.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:26 smithi082 ceph-mon[98694]: pgmap v20925: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 18 op/s 2023-09-10T10:08:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:26 smithi098 ceph-mon[103124]: pgmap v20925: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 277 KiB/s wr, 18 op/s 2023-09-10T10:08:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:28 smithi098 ceph-mon[103124]: pgmap v20926: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T10:08:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38939.669449 secs 2023-09-10T10:08:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:28 smithi170 ceph-mon[102832]: pgmap v20926: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T10:08:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38939.669449 secs 2023-09-10T10:08:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:28 smithi082 ceph-mon[98694]: pgmap v20926: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T10:08:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38939.669449 secs 2023-09-10T10:08:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38940.900224 secs 2023-09-10T10:08:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38941.251871 secs 2023-09-10T10:08:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38940.900224 secs 2023-09-10T10:08:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38941.251871 secs 2023-09-10T10:08:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38940.900224 secs 2023-09-10T10:08:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38941.251871 secs 2023-09-10T10:08:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:30 smithi098 ceph-mon[103124]: pgmap v20927: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T10:08:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:30 smithi170 ceph-mon[102832]: pgmap v20927: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T10:08:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:30 smithi082 ceph-mon[98694]: pgmap v20927: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T10:08:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:32 smithi098 ceph-mon[103124]: pgmap v20928: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 145 KiB/s wr, 16 op/s 2023-09-10T10:08:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:32 smithi170 ceph-mon[102832]: pgmap v20928: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 145 KiB/s wr, 16 op/s 2023-09-10T10:08:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:32 smithi082 ceph-mon[98694]: pgmap v20928: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 145 KiB/s wr, 16 op/s 2023-09-10T10:08:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38944.669558 secs 2023-09-10T10:08:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:08:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38944.669558 secs 2023-09-10T10:08:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:08:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38944.669558 secs 2023-09-10T10:08:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:08:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38945.900277 secs 2023-09-10T10:08:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:34 smithi098 ceph-mon[103124]: pgmap v20929: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T10:08:35.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38946.251936 secs 2023-09-10T10:08:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38945.900277 secs 2023-09-10T10:08:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:34 smithi170 ceph-mon[102832]: pgmap v20929: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T10:08:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38946.251936 secs 2023-09-10T10:08:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38945.900277 secs 2023-09-10T10:08:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:34 smithi082 ceph-mon[98694]: pgmap v20929: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T10:08:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38946.251936 secs 2023-09-10T10:08:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:36 smithi098 ceph-mon[103124]: pgmap v20930: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 286 KiB/s wr, 17 op/s 2023-09-10T10:08:37.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:08:37.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:36 smithi170 ceph-mon[102832]: pgmap v20930: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 286 KiB/s wr, 17 op/s 2023-09-10T10:08:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:08:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:36 smithi082 ceph-mon[98694]: pgmap v20930: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 286 KiB/s wr, 17 op/s 2023-09-10T10:08:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:08:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:38 smithi098 ceph-mon[103124]: pgmap v20931: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T10:08:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:38 smithi170 ceph-mon[102832]: pgmap v20931: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T10:08:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:38 smithi082 ceph-mon[98694]: pgmap v20931: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T10:08:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38950.900333 secs 2023-09-10T10:08:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38951.252001 secs 2023-09-10T10:08:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:08:40.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38950.900333 secs 2023-09-10T10:08:40.836 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38951.252001 secs 2023-09-10T10:08:40.836 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:08:40.837 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38950.900333 secs 2023-09-10T10:08:40.838 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38951.252001 secs 2023-09-10T10:08:40.838 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:08:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:40 smithi098 ceph-mon[103124]: pgmap v20932: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T10:08:41.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:08:41.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:08:41.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:40 smithi170 ceph-mon[102832]: pgmap v20932: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T10:08:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:08:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:08:41.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:40 smithi082 ceph-mon[98694]: pgmap v20932: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 11 op/s 2023-09-10T10:08:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:08:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:08:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:43 smithi098 ceph-mon[103124]: pgmap v20933: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 465 KiB/s wr, 17 op/s 2023-09-10T10:08:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:43 smithi170 ceph-mon[102832]: pgmap v20933: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 465 KiB/s wr, 17 op/s 2023-09-10T10:08:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:43 smithi082 ceph-mon[98694]: pgmap v20933: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 465 KiB/s wr, 17 op/s 2023-09-10T10:08:45.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38955.900391 secs 2023-09-10T10:08:45.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:45 smithi170 ceph-mon[102832]: pgmap v20934: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 465 KiB/s wr, 12 op/s 2023-09-10T10:08:45.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38956.252065 secs 2023-09-10T10:08:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38955.900391 secs 2023-09-10T10:08:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:45 smithi082 ceph-mon[98694]: pgmap v20934: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 465 KiB/s wr, 12 op/s 2023-09-10T10:08:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38956.252065 secs 2023-09-10T10:08:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38955.900391 secs 2023-09-10T10:08:45.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:45 smithi098 ceph-mon[103124]: pgmap v20934: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 465 KiB/s wr, 12 op/s 2023-09-10T10:08:45.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38956.252065 secs 2023-09-10T10:08:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:08:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:46.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:46 smithi170 ceph-mon[102832]: pgmap v20935: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 775 KiB/s wr, 18 op/s 2023-09-10T10:08:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:08:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:46 smithi082 ceph-mon[98694]: pgmap v20935: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 775 KiB/s wr, 18 op/s 2023-09-10T10:08:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:08:47.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:47.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:46 smithi098 ceph-mon[103124]: pgmap v20935: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 775 KiB/s wr, 18 op/s 2023-09-10T10:08:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:48 smithi098 ceph-mon[103124]: pgmap v20936: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 634 KiB/s wr, 13 op/s 2023-09-10T10:08:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38959.669755 secs 2023-09-10T10:08:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:48 smithi170 ceph-mon[102832]: pgmap v20936: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 634 KiB/s wr, 13 op/s 2023-09-10T10:08:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38959.669755 secs 2023-09-10T10:08:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:48 smithi082 ceph-mon[98694]: pgmap v20936: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 634 KiB/s wr, 13 op/s 2023-09-10T10:08:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38959.669755 secs 2023-09-10T10:08:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38960.900447 secs 2023-09-10T10:08:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:08:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38961.252127 secs 2023-09-10T10:08:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38960.900447 secs 2023-09-10T10:08:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:08:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38961.252127 secs 2023-09-10T10:08:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38960.900447 secs 2023-09-10T10:08:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:08:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38961.252127 secs 2023-09-10T10:08:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:50 smithi098 ceph-mon[103124]: pgmap v20937: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 634 KiB/s wr, 13 op/s 2023-09-10T10:08:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:50 smithi170 ceph-mon[102832]: pgmap v20937: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 634 KiB/s wr, 13 op/s 2023-09-10T10:08:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:50 smithi082 ceph-mon[98694]: pgmap v20937: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 634 KiB/s wr, 13 op/s 2023-09-10T10:08:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:52 smithi170 ceph-mon[102832]: pgmap v20938: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 928 KiB/s wr, 20 op/s 2023-09-10T10:08:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:08:52.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:08:52.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:52 smithi082 ceph-mon[98694]: pgmap v20938: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 928 KiB/s wr, 20 op/s 2023-09-10T10:08:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:08:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:08:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:52 smithi098 ceph-mon[103124]: pgmap v20938: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 928 KiB/s wr, 20 op/s 2023-09-10T10:08:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:08:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:08:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38964.669781 secs 2023-09-10T10:08:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38964.669781 secs 2023-09-10T10:08:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38964.669781 secs 2023-09-10T10:08:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:54 smithi170 ceph-mon[102832]: pgmap v20939: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 604 KiB/s wr, 13 op/s 2023-09-10T10:08:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:54 smithi082 ceph-mon[98694]: pgmap v20939: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 604 KiB/s wr, 13 op/s 2023-09-10T10:08:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:54 smithi098 ceph-mon[103124]: pgmap v20939: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 604 KiB/s wr, 13 op/s 2023-09-10T10:08:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38965.900494 secs 2023-09-10T10:08:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38966.252191 secs 2023-09-10T10:08:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38965.900494 secs 2023-09-10T10:08:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38966.252191 secs 2023-09-10T10:08:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38965.900494 secs 2023-09-10T10:08:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38966.252191 secs 2023-09-10T10:08:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:56 smithi170 ceph-mon[102832]: pgmap v20940: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 877 KiB/s wr, 20 op/s 2023-09-10T10:08:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:56 smithi082 ceph-mon[98694]: pgmap v20940: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 877 KiB/s wr, 20 op/s 2023-09-10T10:08:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:56 smithi098 ceph-mon[103124]: pgmap v20940: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 877 KiB/s wr, 20 op/s 2023-09-10T10:08:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:08:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:58 smithi170 ceph-mon[102832]: pgmap v20941: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 568 KiB/s wr, 14 op/s 2023-09-10T10:08:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:08:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38969.669899 secs 2023-09-10T10:08:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:08:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:58 smithi082 ceph-mon[98694]: pgmap v20941: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 568 KiB/s wr, 14 op/s 2023-09-10T10:08:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:08:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38969.669899 secs 2023-09-10T10:08:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:08:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:08:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:58 smithi098 ceph-mon[103124]: pgmap v20941: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 568 KiB/s wr, 14 op/s 2023-09-10T10:08:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:08:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38969.669899 secs 2023-09-10T10:09:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38970.900551 secs 2023-09-10T10:09:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38971.252253 secs 2023-09-10T10:09:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:09:00.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:00.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:00 smithi170 ceph-mon[102832]: pgmap v20942: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 568 KiB/s wr, 14 op/s 2023-09-10T10:09:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38970.900551 secs 2023-09-10T10:09:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38971.252253 secs 2023-09-10T10:09:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:09:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:00 smithi082 ceph-mon[98694]: pgmap v20942: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 568 KiB/s wr, 14 op/s 2023-09-10T10:09:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38970.900551 secs 2023-09-10T10:09:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38971.252253 secs 2023-09-10T10:09:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:09:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:00 smithi098 ceph-mon[103124]: pgmap v20942: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 568 KiB/s wr, 14 op/s 2023-09-10T10:09:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:02 smithi098 ceph-mon[103124]: pgmap v20943: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 859 KiB/s wr, 21 op/s 2023-09-10T10:09:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:02 smithi170 ceph-mon[102832]: pgmap v20943: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 859 KiB/s wr, 21 op/s 2023-09-10T10:09:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:02 smithi082 ceph-mon[98694]: pgmap v20943: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1009 KiB/s rd, 859 KiB/s wr, 21 op/s 2023-09-10T10:09:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38974.670001 secs 2023-09-10T10:09:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38974.670001 secs 2023-09-10T10:09:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38974.670001 secs 2023-09-10T10:09:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38975.900616 secs 2023-09-10T10:09:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:04 smithi098 ceph-mon[103124]: pgmap v20944: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 564 KiB/s wr, 13 op/s 2023-09-10T10:09:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38976.252319 secs 2023-09-10T10:09:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38975.900616 secs 2023-09-10T10:09:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:04 smithi170 ceph-mon[102832]: pgmap v20944: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 564 KiB/s wr, 13 op/s 2023-09-10T10:09:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38976.252319 secs 2023-09-10T10:09:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38975.900616 secs 2023-09-10T10:09:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:04 smithi082 ceph-mon[98694]: pgmap v20944: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 564 KiB/s wr, 13 op/s 2023-09-10T10:09:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38976.252319 secs 2023-09-10T10:09:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:06 smithi098 ceph-mon[103124]: pgmap v20945: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 748 KiB/s wr, 35 op/s 2023-09-10T10:09:07.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:06 smithi170 ceph-mon[102832]: pgmap v20945: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 748 KiB/s wr, 35 op/s 2023-09-10T10:09:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:06 smithi082 ceph-mon[98694]: pgmap v20945: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 748 KiB/s wr, 35 op/s 2023-09-10T10:09:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:08 smithi098 ceph-mon[103124]: pgmap v20946: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 474 KiB/s wr, 28 op/s 2023-09-10T10:09:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38979.670059 secs 2023-09-10T10:09:09.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:08 smithi098 ceph-mon[103124]: osdmap e1377: 12 total, 12 up, 12 in 2023-09-10T10:09:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:08 smithi170 ceph-mon[102832]: pgmap v20946: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 474 KiB/s wr, 28 op/s 2023-09-10T10:09:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38979.670059 secs 2023-09-10T10:09:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:08 smithi170 ceph-mon[102832]: osdmap e1377: 12 total, 12 up, 12 in 2023-09-10T10:09:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:08 smithi082 ceph-mon[98694]: pgmap v20946: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 474 KiB/s wr, 28 op/s 2023-09-10T10:09:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38979.670059 secs 2023-09-10T10:09:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:08 smithi082 ceph-mon[98694]: osdmap e1377: 12 total, 12 up, 12 in 2023-09-10T10:09:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38980.900705 secs 2023-09-10T10:09:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38981.252381 secs 2023-09-10T10:09:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38980.900705 secs 2023-09-10T10:09:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38981.252381 secs 2023-09-10T10:09:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38980.900705 secs 2023-09-10T10:09:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38981.252381 secs 2023-09-10T10:09:11.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:11 smithi098 ceph-mon[103124]: pgmap v20948: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 569 KiB/s wr, 33 op/s 2023-09-10T10:09:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:11 smithi170 ceph-mon[102832]: pgmap v20948: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 569 KiB/s wr, 33 op/s 2023-09-10T10:09:11.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:11 smithi082 ceph-mon[98694]: pgmap v20948: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 569 KiB/s wr, 33 op/s 2023-09-10T10:09:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:13 smithi098 ceph-mon[103124]: pgmap v20949: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 493 KiB/s wr, 34 op/s 2023-09-10T10:09:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:13 smithi170 ceph-mon[102832]: pgmap v20949: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 493 KiB/s wr, 34 op/s 2023-09-10T10:09:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:13 smithi082 ceph-mon[98694]: pgmap v20949: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 493 KiB/s wr, 34 op/s 2023-09-10T10:09:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38984.670120 secs 2023-09-10T10:09:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38984.670120 secs 2023-09-10T10:09:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38984.670120 secs 2023-09-10T10:09:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38985.900759 secs 2023-09-10T10:09:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:15 smithi170 ceph-mon[102832]: pgmap v20950: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 493 KiB/s wr, 34 op/s 2023-09-10T10:09:15.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38986.252445 secs 2023-09-10T10:09:15.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:15 smithi170 ceph-mon[102832]: osdmap e1378: 12 total, 12 up, 12 in 2023-09-10T10:09:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38985.900759 secs 2023-09-10T10:09:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:15 smithi082 ceph-mon[98694]: pgmap v20950: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 493 KiB/s wr, 34 op/s 2023-09-10T10:09:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38986.252445 secs 2023-09-10T10:09:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:15 smithi082 ceph-mon[98694]: osdmap e1378: 12 total, 12 up, 12 in 2023-09-10T10:09:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38985.900759 secs 2023-09-10T10:09:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:15 smithi098 ceph-mon[103124]: pgmap v20950: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 493 KiB/s wr, 34 op/s 2023-09-10T10:09:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38986.252445 secs 2023-09-10T10:09:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:15 smithi098 ceph-mon[103124]: osdmap e1378: 12 total, 12 up, 12 in 2023-09-10T10:09:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:17 smithi170 ceph-mon[102832]: pgmap v20952: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 522 KiB/s wr, 21 op/s 2023-09-10T10:09:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:17 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:09:17.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:17 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:17 smithi082 ceph-mon[98694]: pgmap v20952: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 522 KiB/s wr, 21 op/s 2023-09-10T10:09:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:17 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:09:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:17 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:17 smithi098 ceph-mon[103124]: pgmap v20952: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 522 KiB/s wr, 21 op/s 2023-09-10T10:09:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:17 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:09:17.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:17 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:09:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:18 smithi170 ceph-mon[102832]: pgmap v20953: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 421 KiB/s wr, 17 op/s 2023-09-10T10:09:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:09:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:18 smithi082 ceph-mon[98694]: pgmap v20953: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 421 KiB/s wr, 17 op/s 2023-09-10T10:09:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:09:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:18 smithi098 ceph-mon[103124]: pgmap v20953: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 421 KiB/s wr, 17 op/s 2023-09-10T10:09:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38989.670205 secs 2023-09-10T10:09:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38989.670205 secs 2023-09-10T10:09:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38989.670205 secs 2023-09-10T10:09:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38990.900844 secs 2023-09-10T10:09:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38991.252507 secs 2023-09-10T10:09:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:20 smithi170 ceph-mon[102832]: pgmap v20954: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 418 KiB/s wr, 16 op/s 2023-09-10T10:09:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38990.900844 secs 2023-09-10T10:09:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38991.252507 secs 2023-09-10T10:09:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:20 smithi082 ceph-mon[98694]: pgmap v20954: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 418 KiB/s wr, 16 op/s 2023-09-10T10:09:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38990.900844 secs 2023-09-10T10:09:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38991.252507 secs 2023-09-10T10:09:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:20 smithi098 ceph-mon[103124]: pgmap v20954: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 418 KiB/s wr, 16 op/s 2023-09-10T10:09:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:22 smithi098 ceph-mon[103124]: pgmap v20955: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 145 KiB/s wr, 14 op/s 2023-09-10T10:09:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:22 smithi170 ceph-mon[102832]: pgmap v20955: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 145 KiB/s wr, 14 op/s 2023-09-10T10:09:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:22 smithi082 ceph-mon[98694]: pgmap v20955: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 145 KiB/s wr, 14 op/s 2023-09-10T10:09:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38994.670267 secs 2023-09-10T10:09:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38994.670267 secs 2023-09-10T10:09:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38994.670267 secs 2023-09-10T10:09:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38995.900901 secs 2023-09-10T10:09:25.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:24 smithi098 ceph-mon[103124]: pgmap v20956: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 145 KiB/s wr, 14 op/s 2023-09-10T10:09:25.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 38996.252571 secs 2023-09-10T10:09:25.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:24 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:09:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38995.900901 secs 2023-09-10T10:09:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:24 smithi170 ceph-mon[102832]: pgmap v20956: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 145 KiB/s wr, 14 op/s 2023-09-10T10:09:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 38996.252571 secs 2023-09-10T10:09:25.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:24 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:09:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38995.900901 secs 2023-09-10T10:09:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:24 smithi082 ceph-mon[98694]: pgmap v20956: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 145 KiB/s wr, 14 op/s 2023-09-10T10:09:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 38996.252571 secs 2023-09-10T10:09:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:24 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:09:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:09:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:09:26.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:25 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:09:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:09:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:09:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:25 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:09:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:09:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:09:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:25 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:09:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:26 smithi098 ceph-mon[103124]: pgmap v20957: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 283 KiB/s wr, 18 op/s 2023-09-10T10:09:27.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:26 smithi170 ceph-mon[102832]: pgmap v20957: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 283 KiB/s wr, 18 op/s 2023-09-10T10:09:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:26 smithi082 ceph-mon[98694]: pgmap v20957: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 283 KiB/s wr, 18 op/s 2023-09-10T10:09:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:29 smithi098 ceph-mon[103124]: pgmap v20958: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T10:09:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 38999.670324 secs 2023-09-10T10:09:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:29 smithi170 ceph-mon[102832]: pgmap v20958: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T10:09:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 38999.670324 secs 2023-09-10T10:09:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:29 smithi082 ceph-mon[98694]: pgmap v20958: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T10:09:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 38999.670324 secs 2023-09-10T10:09:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39000.900961 secs 2023-09-10T10:09:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39001.252635 secs 2023-09-10T10:09:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39000.900961 secs 2023-09-10T10:09:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39001.252635 secs 2023-09-10T10:09:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39000.900961 secs 2023-09-10T10:09:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39001.252635 secs 2023-09-10T10:09:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:31 smithi098 ceph-mon[103124]: pgmap v20959: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T10:09:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:31 smithi170 ceph-mon[102832]: pgmap v20959: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T10:09:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:31 smithi082 ceph-mon[98694]: pgmap v20959: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T10:09:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:33 smithi170 ceph-mon[102832]: pgmap v20960: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 16 op/s 2023-09-10T10:09:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:33 smithi082 ceph-mon[98694]: pgmap v20960: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 16 op/s 2023-09-10T10:09:33.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:33 smithi098 ceph-mon[103124]: pgmap v20960: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 16 op/s 2023-09-10T10:09:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39004.670443 secs 2023-09-10T10:09:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:09:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39004.670443 secs 2023-09-10T10:09:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:09:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39004.670443 secs 2023-09-10T10:09:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:09:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39005.901020 secs 2023-09-10T10:09:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:35 smithi170 ceph-mon[102832]: pgmap v20961: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T10:09:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39006.252698 secs 2023-09-10T10:09:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39005.901020 secs 2023-09-10T10:09:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:35 smithi082 ceph-mon[98694]: pgmap v20961: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T10:09:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39006.252698 secs 2023-09-10T10:09:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39005.901020 secs 2023-09-10T10:09:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:35 smithi098 ceph-mon[103124]: pgmap v20961: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 11 op/s 2023-09-10T10:09:35.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39006.252698 secs 2023-09-10T10:09:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:36 smithi170 ceph-mon[102832]: pgmap v20962: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 286 KiB/s wr, 16 op/s 2023-09-10T10:09:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:36 smithi082 ceph-mon[98694]: pgmap v20962: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 286 KiB/s wr, 16 op/s 2023-09-10T10:09:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:36 smithi098 ceph-mon[103124]: pgmap v20962: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 286 KiB/s wr, 16 op/s 2023-09-10T10:09:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:09:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:09:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:09:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:38 smithi170 ceph-mon[102832]: pgmap v20963: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 10 op/s 2023-09-10T10:09:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:38 smithi082 ceph-mon[98694]: pgmap v20963: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 10 op/s 2023-09-10T10:09:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:38 smithi098 ceph-mon[103124]: pgmap v20963: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 10 op/s 2023-09-10T10:09:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:09:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:09:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:09:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39010.901073 secs 2023-09-10T10:09:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39011.252761 secs 2023-09-10T10:09:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:09:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:09:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:40 smithi098 ceph-mon[103124]: pgmap v20964: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 10 op/s 2023-09-10T10:09:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39010.901073 secs 2023-09-10T10:09:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39011.252761 secs 2023-09-10T10:09:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:09:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:09:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:40 smithi170 ceph-mon[102832]: pgmap v20964: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 10 op/s 2023-09-10T10:09:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39010.901073 secs 2023-09-10T10:09:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39011.252761 secs 2023-09-10T10:09:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:09:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:09:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:40 smithi082 ceph-mon[98694]: pgmap v20964: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 10 op/s 2023-09-10T10:09:43.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:42 smithi098 ceph-mon[103124]: pgmap v20965: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 466 KiB/s wr, 17 op/s 2023-09-10T10:09:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:42 smithi170 ceph-mon[102832]: pgmap v20965: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 466 KiB/s wr, 17 op/s 2023-09-10T10:09:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:42 smithi082 ceph-mon[98694]: pgmap v20965: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 466 KiB/s wr, 17 op/s 2023-09-10T10:09:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39015.901155 secs 2023-09-10T10:09:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:44 smithi098 ceph-mon[103124]: pgmap v20966: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 466 KiB/s wr, 11 op/s 2023-09-10T10:09:45.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39016.252824 secs 2023-09-10T10:09:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39015.901155 secs 2023-09-10T10:09:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:44 smithi170 ceph-mon[102832]: pgmap v20966: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 466 KiB/s wr, 11 op/s 2023-09-10T10:09:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39016.252824 secs 2023-09-10T10:09:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39015.901155 secs 2023-09-10T10:09:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:44 smithi082 ceph-mon[98694]: pgmap v20966: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 466 KiB/s wr, 11 op/s 2023-09-10T10:09:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39016.252824 secs 2023-09-10T10:09:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:09:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:46.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:46 smithi170 ceph-mon[102832]: pgmap v20967: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 776 KiB/s wr, 18 op/s 2023-09-10T10:09:46.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:09:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:46 smithi082 ceph-mon[98694]: pgmap v20967: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 776 KiB/s wr, 18 op/s 2023-09-10T10:09:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:09:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:47.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:46 smithi098 ceph-mon[103124]: pgmap v20967: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 776 KiB/s wr, 18 op/s 2023-09-10T10:09:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:48 smithi098 ceph-mon[103124]: pgmap v20968: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 635 KiB/s wr, 13 op/s 2023-09-10T10:09:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39019.670683 secs 2023-09-10T10:09:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:48 smithi170 ceph-mon[102832]: pgmap v20968: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 635 KiB/s wr, 13 op/s 2023-09-10T10:09:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39019.670683 secs 2023-09-10T10:09:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:48 smithi082 ceph-mon[98694]: pgmap v20968: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 635 KiB/s wr, 13 op/s 2023-09-10T10:09:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39019.670683 secs 2023-09-10T10:09:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39020.901242 secs 2023-09-10T10:09:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39021.252886 secs 2023-09-10T10:09:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:09:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39020.901242 secs 2023-09-10T10:09:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39021.252886 secs 2023-09-10T10:09:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:09:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39020.901242 secs 2023-09-10T10:09:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39021.252886 secs 2023-09-10T10:09:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:09:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:51 smithi098 ceph-mon[103124]: pgmap v20969: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 635 KiB/s wr, 13 op/s 2023-09-10T10:09:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:51 smithi170 ceph-mon[102832]: pgmap v20969: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 635 KiB/s wr, 13 op/s 2023-09-10T10:09:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:51 smithi082 ceph-mon[98694]: pgmap v20969: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 635 KiB/s wr, 13 op/s 2023-09-10T10:09:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:52 smithi170 ceph-mon[102832]: pgmap v20970: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 921 KiB/s wr, 20 op/s 2023-09-10T10:09:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:09:52.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:09:52.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:52 smithi082 ceph-mon[98694]: pgmap v20970: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 921 KiB/s wr, 20 op/s 2023-09-10T10:09:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:09:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:09:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:52 smithi098 ceph-mon[103124]: pgmap v20970: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 921 KiB/s wr, 20 op/s 2023-09-10T10:09:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:09:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:09:52.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39024.670755 secs 2023-09-10T10:09:53.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39024.670755 secs 2023-09-10T10:09:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39024.670755 secs 2023-09-10T10:09:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:54 smithi170 ceph-mon[102832]: pgmap v20971: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 597 KiB/s wr, 13 op/s 2023-09-10T10:09:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:54 smithi082 ceph-mon[98694]: pgmap v20971: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 597 KiB/s wr, 13 op/s 2023-09-10T10:09:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:54 smithi098 ceph-mon[103124]: pgmap v20971: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 597 KiB/s wr, 13 op/s 2023-09-10T10:09:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39025.901297 secs 2023-09-10T10:09:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39026.252953 secs 2023-09-10T10:09:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39025.901297 secs 2023-09-10T10:09:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39026.252953 secs 2023-09-10T10:09:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39025.901297 secs 2023-09-10T10:09:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39026.252953 secs 2023-09-10T10:09:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:56 smithi170 ceph-mon[102832]: pgmap v20972: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 879 KiB/s wr, 20 op/s 2023-09-10T10:09:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:56 smithi082 ceph-mon[98694]: pgmap v20972: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 879 KiB/s wr, 20 op/s 2023-09-10T10:09:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:56 smithi098 ceph-mon[103124]: pgmap v20972: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 879 KiB/s wr, 20 op/s 2023-09-10T10:09:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:09:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:58 smithi170 ceph-mon[102832]: pgmap v20973: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 569 KiB/s wr, 13 op/s 2023-09-10T10:09:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:09:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39029.670777 secs 2023-09-10T10:09:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:09:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:58 smithi082 ceph-mon[98694]: pgmap v20973: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 569 KiB/s wr, 13 op/s 2023-09-10T10:09:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:09:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39029.670777 secs 2023-09-10T10:09:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:09:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:09:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:58 smithi098 ceph-mon[103124]: pgmap v20973: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 569 KiB/s wr, 13 op/s 2023-09-10T10:09:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:09:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39029.670777 secs 2023-09-10T10:10:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39030.901357 secs 2023-09-10T10:10:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39031.253016 secs 2023-09-10T10:10:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:10:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:00 smithi170 ceph-mon[102832]: pgmap v20974: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 569 KiB/s wr, 13 op/s 2023-09-10T10:10:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T10:10:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39030.901357 secs 2023-09-10T10:10:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39031.253016 secs 2023-09-10T10:10:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:10:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:00.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:00 smithi082 ceph-mon[98694]: pgmap v20974: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 569 KiB/s wr, 13 op/s 2023-09-10T10:10:00.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T10:10:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39030.901357 secs 2023-09-10T10:10:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39031.253016 secs 2023-09-10T10:10:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:10:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:00 smithi098 ceph-mon[103124]: pgmap v20974: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 569 KiB/s wr, 13 op/s 2023-09-10T10:10:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 3 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T10:10:03.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:02 smithi098 ceph-mon[103124]: pgmap v20975: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 836 KiB/s wr, 20 op/s 2023-09-10T10:10:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:02 smithi170 ceph-mon[102832]: pgmap v20975: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 836 KiB/s wr, 20 op/s 2023-09-10T10:10:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:02 smithi082 ceph-mon[98694]: pgmap v20975: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 836 KiB/s wr, 20 op/s 2023-09-10T10:10:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39034.670890 secs 2023-09-10T10:10:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39034.670890 secs 2023-09-10T10:10:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39034.670890 secs 2023-09-10T10:10:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39035.901432 secs 2023-09-10T10:10:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:05 smithi170 ceph-mon[102832]: pgmap v20976: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 550 KiB/s wr, 13 op/s 2023-09-10T10:10:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39036.253103 secs 2023-09-10T10:10:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39035.901432 secs 2023-09-10T10:10:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:05 smithi082 ceph-mon[98694]: pgmap v20976: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 550 KiB/s wr, 13 op/s 2023-09-10T10:10:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39036.253103 secs 2023-09-10T10:10:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39035.901432 secs 2023-09-10T10:10:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:05 smithi098 ceph-mon[103124]: pgmap v20976: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 550 KiB/s wr, 13 op/s 2023-09-10T10:10:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39036.253103 secs 2023-09-10T10:10:07.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:07 smithi170 ceph-mon[102832]: pgmap v20977: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 734 KiB/s wr, 34 op/s 2023-09-10T10:10:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:07 smithi082 ceph-mon[98694]: pgmap v20977: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 734 KiB/s wr, 34 op/s 2023-09-10T10:10:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:07 smithi098 ceph-mon[103124]: pgmap v20977: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 734 KiB/s wr, 34 op/s 2023-09-10T10:10:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:09 smithi170 ceph-mon[102832]: pgmap v20978: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 451 KiB/s wr, 27 op/s 2023-09-10T10:10:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39039.670950 secs 2023-09-10T10:10:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:09 smithi170 ceph-mon[102832]: osdmap e1379: 12 total, 12 up, 12 in 2023-09-10T10:10:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:09 smithi082 ceph-mon[98694]: pgmap v20978: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 451 KiB/s wr, 27 op/s 2023-09-10T10:10:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39039.670950 secs 2023-09-10T10:10:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:09 smithi082 ceph-mon[98694]: osdmap e1379: 12 total, 12 up, 12 in 2023-09-10T10:10:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:09 smithi098 ceph-mon[103124]: pgmap v20978: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 451 KiB/s wr, 27 op/s 2023-09-10T10:10:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39039.670950 secs 2023-09-10T10:10:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:09 smithi098 ceph-mon[103124]: osdmap e1379: 12 total, 12 up, 12 in 2023-09-10T10:10:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39040.901512 secs 2023-09-10T10:10:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39041.253166 secs 2023-09-10T10:10:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:10:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39040.901512 secs 2023-09-10T10:10:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39041.253166 secs 2023-09-10T10:10:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:10:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39040.901512 secs 2023-09-10T10:10:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39041.253166 secs 2023-09-10T10:10:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:10:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:11 smithi170 ceph-mon[102832]: pgmap v20980: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 541 KiB/s wr, 33 op/s 2023-09-10T10:10:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:11 smithi082 ceph-mon[98694]: pgmap v20980: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 541 KiB/s wr, 33 op/s 2023-09-10T10:10:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:11 smithi098 ceph-mon[103124]: pgmap v20980: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 541 KiB/s wr, 33 op/s 2023-09-10T10:10:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:13 smithi170 ceph-mon[102832]: pgmap v20981: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 465 KiB/s wr, 32 op/s 2023-09-10T10:10:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:10:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:13 smithi082 ceph-mon[98694]: pgmap v20981: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 465 KiB/s wr, 32 op/s 2023-09-10T10:10:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:10:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:13 smithi098 ceph-mon[103124]: pgmap v20981: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 465 KiB/s wr, 32 op/s 2023-09-10T10:10:13.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:10:13.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39044.671013 secs 2023-09-10T10:10:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:14 smithi170 ceph-mon[102832]: pgmap v20982: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 465 KiB/s wr, 32 op/s 2023-09-10T10:10:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39044.671013 secs 2023-09-10T10:10:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:14 smithi082 ceph-mon[98694]: pgmap v20982: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 465 KiB/s wr, 32 op/s 2023-09-10T10:10:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39044.671013 secs 2023-09-10T10:10:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:14 smithi098 ceph-mon[103124]: pgmap v20982: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 465 KiB/s wr, 32 op/s 2023-09-10T10:10:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39045.901571 secs 2023-09-10T10:10:15.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39046.253228 secs 2023-09-10T10:10:15.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:15 smithi170 ceph-mon[102832]: osdmap e1380: 12 total, 12 up, 12 in 2023-09-10T10:10:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39045.901571 secs 2023-09-10T10:10:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39046.253228 secs 2023-09-10T10:10:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:15 smithi082 ceph-mon[98694]: osdmap e1380: 12 total, 12 up, 12 in 2023-09-10T10:10:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39045.901571 secs 2023-09-10T10:10:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39046.253228 secs 2023-09-10T10:10:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:15 smithi098 ceph-mon[103124]: osdmap e1380: 12 total, 12 up, 12 in 2023-09-10T10:10:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:16 smithi170 ceph-mon[102832]: pgmap v20984: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 479 KiB/s wr, 17 op/s 2023-09-10T10:10:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:16 smithi082 ceph-mon[98694]: pgmap v20984: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 479 KiB/s wr, 17 op/s 2023-09-10T10:10:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:16 smithi098 ceph-mon[103124]: pgmap v20984: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 479 KiB/s wr, 17 op/s 2023-09-10T10:10:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:18 smithi098 ceph-mon[103124]: pgmap v20985: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 386 KiB/s wr, 14 op/s 2023-09-10T10:10:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39049.671073 secs 2023-09-10T10:10:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:18 smithi170 ceph-mon[102832]: pgmap v20985: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 386 KiB/s wr, 14 op/s 2023-09-10T10:10:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39049.671073 secs 2023-09-10T10:10:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:18 smithi082 ceph-mon[98694]: pgmap v20985: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 386 KiB/s wr, 14 op/s 2023-09-10T10:10:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39049.671073 secs 2023-09-10T10:10:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39050.901650 secs 2023-09-10T10:10:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39051.253293 secs 2023-09-10T10:10:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39050.901650 secs 2023-09-10T10:10:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39051.253293 secs 2023-09-10T10:10:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39050.901650 secs 2023-09-10T10:10:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39051.253293 secs 2023-09-10T10:10:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:21 smithi098 ceph-mon[103124]: pgmap v20986: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 383 KiB/s wr, 14 op/s 2023-09-10T10:10:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:21 smithi170 ceph-mon[102832]: pgmap v20986: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 383 KiB/s wr, 14 op/s 2023-09-10T10:10:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:21 smithi082 ceph-mon[98694]: pgmap v20986: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 383 KiB/s wr, 14 op/s 2023-09-10T10:10:23.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:23 smithi098 ceph-mon[103124]: pgmap v20987: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 13 op/s 2023-09-10T10:10:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:23 smithi170 ceph-mon[102832]: pgmap v20987: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 13 op/s 2023-09-10T10:10:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:23 smithi082 ceph-mon[98694]: pgmap v20987: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 13 op/s 2023-09-10T10:10:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39054.671152 secs 2023-09-10T10:10:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39054.671152 secs 2023-09-10T10:10:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39054.671152 secs 2023-09-10T10:10:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39055.901706 secs 2023-09-10T10:10:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:25 smithi170 ceph-mon[102832]: pgmap v20988: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 13 op/s 2023-09-10T10:10:25.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39056.253357 secs 2023-09-10T10:10:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39055.901706 secs 2023-09-10T10:10:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:25 smithi082 ceph-mon[98694]: pgmap v20988: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 13 op/s 2023-09-10T10:10:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39056.253357 secs 2023-09-10T10:10:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39055.901706 secs 2023-09-10T10:10:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:25 smithi098 ceph-mon[103124]: pgmap v20988: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 139 KiB/s wr, 13 op/s 2023-09-10T10:10:25.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39056.253357 secs 2023-09-10T10:10:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:10:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:10:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:26 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:10:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:10:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:10:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:26 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:10:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:10:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:10:26.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:26 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:10:27.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:27 smithi170 ceph-mon[102832]: pgmap v20989: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 278 KiB/s wr, 18 op/s 2023-09-10T10:10:27.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:10:27.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:10:27.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:10:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:27 smithi082 ceph-mon[98694]: pgmap v20989: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 278 KiB/s wr, 18 op/s 2023-09-10T10:10:27.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:10:27.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:10:27.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:10:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:27 smithi098 ceph-mon[103124]: pgmap v20989: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 278 KiB/s wr, 18 op/s 2023-09-10T10:10:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:10:27.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:10:27.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:10:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:29 smithi170 ceph-mon[102832]: pgmap v20990: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T10:10:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39059.671214 secs 2023-09-10T10:10:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:29 smithi082 ceph-mon[98694]: pgmap v20990: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T10:10:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39059.671214 secs 2023-09-10T10:10:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:29 smithi098 ceph-mon[103124]: pgmap v20990: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T10:10:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39059.671214 secs 2023-09-10T10:10:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39060.901765 secs 2023-09-10T10:10:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39061.253419 secs 2023-09-10T10:10:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39060.901765 secs 2023-09-10T10:10:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39061.253419 secs 2023-09-10T10:10:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39060.901765 secs 2023-09-10T10:10:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39061.253419 secs 2023-09-10T10:10:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:31 smithi170 ceph-mon[102832]: pgmap v20991: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T10:10:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:31 smithi082 ceph-mon[98694]: pgmap v20991: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T10:10:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:31 smithi098 ceph-mon[103124]: pgmap v20991: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T10:10:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:32 smithi170 ceph-mon[102832]: pgmap v20992: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 16 op/s 2023-09-10T10:10:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:32 smithi082 ceph-mon[98694]: pgmap v20992: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 16 op/s 2023-09-10T10:10:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:32 smithi098 ceph-mon[103124]: pgmap v20992: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 16 op/s 2023-09-10T10:10:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39064.671274 secs 2023-09-10T10:10:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:10:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:34 smithi170 ceph-mon[102832]: pgmap v20993: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T10:10:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39064.671274 secs 2023-09-10T10:10:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:10:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:34 smithi082 ceph-mon[98694]: pgmap v20993: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T10:10:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39064.671274 secs 2023-09-10T10:10:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:10:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:34 smithi098 ceph-mon[103124]: pgmap v20993: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T10:10:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39065.901846 secs 2023-09-10T10:10:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39066.253487 secs 2023-09-10T10:10:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39065.901846 secs 2023-09-10T10:10:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39066.253487 secs 2023-09-10T10:10:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39065.901846 secs 2023-09-10T10:10:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39066.253487 secs 2023-09-10T10:10:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:36 smithi170 ceph-mon[102832]: pgmap v20994: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 287 KiB/s wr, 17 op/s 2023-09-10T10:10:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:36 smithi082 ceph-mon[98694]: pgmap v20994: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 287 KiB/s wr, 17 op/s 2023-09-10T10:10:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:36 smithi098 ceph-mon[103124]: pgmap v20994: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 287 KiB/s wr, 17 op/s 2023-09-10T10:10:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:10:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:10:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:10:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:38 smithi170 ceph-mon[102832]: pgmap v20995: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 10 op/s 2023-09-10T10:10:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:38 smithi082 ceph-mon[98694]: pgmap v20995: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 10 op/s 2023-09-10T10:10:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:38 smithi098 ceph-mon[103124]: pgmap v20995: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 10 op/s 2023-09-10T10:10:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:10:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:10:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:10:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39070.901906 secs 2023-09-10T10:10:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39071.253552 secs 2023-09-10T10:10:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:10:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:10:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:40 smithi098 ceph-mon[103124]: pgmap v20996: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 10 op/s 2023-09-10T10:10:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39070.901906 secs 2023-09-10T10:10:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39071.253552 secs 2023-09-10T10:10:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:10:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:10:40.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:40.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:40 smithi170 ceph-mon[102832]: pgmap v20996: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 10 op/s 2023-09-10T10:10:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39070.901906 secs 2023-09-10T10:10:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39071.253552 secs 2023-09-10T10:10:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:10:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:10:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:40 smithi082 ceph-mon[98694]: pgmap v20996: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 10 op/s 2023-09-10T10:10:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:43 smithi170 ceph-mon[102832]: pgmap v20997: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 464 KiB/s wr, 17 op/s 2023-09-10T10:10:43.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:43 smithi082 ceph-mon[98694]: pgmap v20997: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 464 KiB/s wr, 17 op/s 2023-09-10T10:10:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:43 smithi098 ceph-mon[103124]: pgmap v20997: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 464 KiB/s wr, 17 op/s 2023-09-10T10:10:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39075.901964 secs 2023-09-10T10:10:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:45 smithi170 ceph-mon[102832]: pgmap v20998: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 464 KiB/s wr, 12 op/s 2023-09-10T10:10:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39076.253619 secs 2023-09-10T10:10:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39075.901964 secs 2023-09-10T10:10:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:45 smithi082 ceph-mon[98694]: pgmap v20998: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 464 KiB/s wr, 12 op/s 2023-09-10T10:10:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39076.253619 secs 2023-09-10T10:10:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39075.901964 secs 2023-09-10T10:10:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:45 smithi098 ceph-mon[103124]: pgmap v20998: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 464 KiB/s wr, 12 op/s 2023-09-10T10:10:45.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39076.253619 secs 2023-09-10T10:10:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:10:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:46 smithi082 ceph-mon[98694]: pgmap v20999: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 775 KiB/s wr, 18 op/s 2023-09-10T10:10:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:10:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:47.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:46 smithi098 ceph-mon[103124]: pgmap v20999: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 775 KiB/s wr, 18 op/s 2023-09-10T10:10:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:10:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:47.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:46 smithi170 ceph-mon[102832]: pgmap v20999: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 775 KiB/s wr, 18 op/s 2023-09-10T10:10:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:49 smithi170 ceph-mon[102832]: pgmap v21000: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 634 KiB/s wr, 12 op/s 2023-09-10T10:10:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:49 smithi082 ceph-mon[98694]: pgmap v21000: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 634 KiB/s wr, 12 op/s 2023-09-10T10:10:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:49 smithi098 ceph-mon[103124]: pgmap v21000: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 634 KiB/s wr, 12 op/s 2023-09-10T10:10:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39080.902058 secs 2023-09-10T10:10:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39081.253684 secs 2023-09-10T10:10:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:10:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39080.902058 secs 2023-09-10T10:10:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39081.253684 secs 2023-09-10T10:10:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:10:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39080.902058 secs 2023-09-10T10:10:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39081.253684 secs 2023-09-10T10:10:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:10:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:51 smithi170 ceph-mon[102832]: pgmap v21001: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 634 KiB/s wr, 12 op/s 2023-09-10T10:10:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:51 smithi082 ceph-mon[98694]: pgmap v21001: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 634 KiB/s wr, 12 op/s 2023-09-10T10:10:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:51 smithi098 ceph-mon[103124]: pgmap v21001: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 634 KiB/s wr, 12 op/s 2023-09-10T10:10:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:53 smithi170 ceph-mon[102832]: pgmap v21002: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 934 KiB/s wr, 20 op/s 2023-09-10T10:10:53.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:10:53.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:53 smithi082 ceph-mon[98694]: pgmap v21002: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 934 KiB/s wr, 20 op/s 2023-09-10T10:10:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:10:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:53 smithi098 ceph-mon[103124]: pgmap v21002: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 934 KiB/s wr, 20 op/s 2023-09-10T10:10:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:10:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39085.902117 secs 2023-09-10T10:10:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:55 smithi170 ceph-mon[102832]: pgmap v21003: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 611 KiB/s wr, 13 op/s 2023-09-10T10:10:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39086.253747 secs 2023-09-10T10:10:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39085.902117 secs 2023-09-10T10:10:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:55 smithi082 ceph-mon[98694]: pgmap v21003: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 611 KiB/s wr, 13 op/s 2023-09-10T10:10:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39086.253747 secs 2023-09-10T10:10:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39085.902117 secs 2023-09-10T10:10:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:55 smithi098 ceph-mon[103124]: pgmap v21003: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 611 KiB/s wr, 13 op/s 2023-09-10T10:10:55.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39086.253747 secs 2023-09-10T10:10:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:57 smithi170 ceph-mon[102832]: pgmap v21004: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 881 KiB/s wr, 20 op/s 2023-09-10T10:10:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:57 smithi082 ceph-mon[98694]: pgmap v21004: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 881 KiB/s wr, 20 op/s 2023-09-10T10:10:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:57 smithi098 ceph-mon[103124]: pgmap v21004: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 881 KiB/s wr, 20 op/s 2023-09-10T10:10:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:10:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:10:58 smithi170 ceph-mon[102832]: pgmap v21005: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 570 KiB/s wr, 14 op/s 2023-09-10T10:10:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:10:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:10:58 smithi082 ceph-mon[98694]: pgmap v21005: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 570 KiB/s wr, 14 op/s 2023-09-10T10:10:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:10:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:10:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:10:58 smithi098 ceph-mon[103124]: pgmap v21005: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 570 KiB/s wr, 14 op/s 2023-09-10T10:11:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39090.902175 secs 2023-09-10T10:11:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39091.253813 secs 2023-09-10T10:11:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:11:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:00 smithi098 ceph-mon[103124]: pgmap v21006: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 570 KiB/s wr, 14 op/s 2023-09-10T10:11:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39090.902175 secs 2023-09-10T10:11:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39091.253813 secs 2023-09-10T10:11:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:11:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:00 smithi170 ceph-mon[102832]: pgmap v21006: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 570 KiB/s wr, 14 op/s 2023-09-10T10:11:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39090.902175 secs 2023-09-10T10:11:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39091.253813 secs 2023-09-10T10:11:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:11:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:00 smithi082 ceph-mon[98694]: pgmap v21006: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 570 KiB/s wr, 14 op/s 2023-09-10T10:11:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:03 smithi170 ceph-mon[102832]: pgmap v21007: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 838 KiB/s wr, 21 op/s 2023-09-10T10:11:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:03 smithi082 ceph-mon[98694]: pgmap v21007: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 838 KiB/s wr, 21 op/s 2023-09-10T10:11:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:03 smithi098 ceph-mon[103124]: pgmap v21007: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 838 KiB/s wr, 21 op/s 2023-09-10T10:11:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39094.671710 secs 2023-09-10T10:11:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39094.671710 secs 2023-09-10T10:11:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39094.671710 secs 2023-09-10T10:11:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39095.902234 secs 2023-09-10T10:11:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:05 smithi170 ceph-mon[102832]: pgmap v21008: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 538 KiB/s wr, 13 op/s 2023-09-10T10:11:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39096.253881 secs 2023-09-10T10:11:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:05 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:11:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:11:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39095.902234 secs 2023-09-10T10:11:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:05 smithi082 ceph-mon[98694]: pgmap v21008: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 538 KiB/s wr, 13 op/s 2023-09-10T10:11:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39096.253881 secs 2023-09-10T10:11:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:05 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:11:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:11:05.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39095.902234 secs 2023-09-10T10:11:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:05 smithi098 ceph-mon[103124]: pgmap v21008: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 538 KiB/s wr, 13 op/s 2023-09-10T10:11:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39096.253881 secs 2023-09-10T10:11:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:05 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:11:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:11:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:07.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:07 smithi170 ceph-mon[102832]: pgmap v21009: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 736 KiB/s wr, 35 op/s 2023-09-10T10:11:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:07 smithi082 ceph-mon[98694]: pgmap v21009: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 736 KiB/s wr, 35 op/s 2023-09-10T10:11:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:07 smithi098 ceph-mon[103124]: pgmap v21009: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 736 KiB/s wr, 35 op/s 2023-09-10T10:11:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:09 smithi170 ceph-mon[102832]: pgmap v21010: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 465 KiB/s wr, 28 op/s 2023-09-10T10:11:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39099.671791 secs 2023-09-10T10:11:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:09 smithi170 ceph-mon[102832]: osdmap e1381: 12 total, 12 up, 12 in 2023-09-10T10:11:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:09 smithi082 ceph-mon[98694]: pgmap v21010: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 465 KiB/s wr, 28 op/s 2023-09-10T10:11:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39099.671791 secs 2023-09-10T10:11:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:09 smithi082 ceph-mon[98694]: osdmap e1381: 12 total, 12 up, 12 in 2023-09-10T10:11:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:09 smithi098 ceph-mon[103124]: pgmap v21010: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 465 KiB/s wr, 28 op/s 2023-09-10T10:11:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39099.671791 secs 2023-09-10T10:11:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:09 smithi098 ceph-mon[103124]: osdmap e1381: 12 total, 12 up, 12 in 2023-09-10T10:11:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39100.902319 secs 2023-09-10T10:11:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39101.253945 secs 2023-09-10T10:11:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:11:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:10 smithi170 ceph-mon[102832]: pgmap v21012: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 558 KiB/s wr, 34 op/s 2023-09-10T10:11:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39100.902319 secs 2023-09-10T10:11:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39101.253945 secs 2023-09-10T10:11:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:11:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:10 smithi082 ceph-mon[98694]: pgmap v21012: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 558 KiB/s wr, 34 op/s 2023-09-10T10:11:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39100.902319 secs 2023-09-10T10:11:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39101.253945 secs 2023-09-10T10:11:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:11:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:10 smithi098 ceph-mon[103124]: pgmap v21012: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 558 KiB/s wr, 34 op/s 2023-09-10T10:11:12.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:11:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:12 smithi082 ceph-mon[98694]: pgmap v21013: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 466 KiB/s wr, 34 op/s 2023-09-10T10:11:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:11:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:13.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:12 smithi098 ceph-mon[103124]: pgmap v21013: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 466 KiB/s wr, 34 op/s 2023-09-10T10:11:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:11:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:12 smithi170 ceph-mon[102832]: pgmap v21013: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 466 KiB/s wr, 34 op/s 2023-09-10T10:11:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39104.671853 secs 2023-09-10T10:11:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39104.671853 secs 2023-09-10T10:11:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39104.671853 secs 2023-09-10T10:11:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39105.902378 secs 2023-09-10T10:11:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:14 smithi082 ceph-mon[98694]: pgmap v21014: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 466 KiB/s wr, 34 op/s 2023-09-10T10:11:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39106.254012 secs 2023-09-10T10:11:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:14 smithi082 ceph-mon[98694]: osdmap e1382: 12 total, 12 up, 12 in 2023-09-10T10:11:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39105.902378 secs 2023-09-10T10:11:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:14 smithi098 ceph-mon[103124]: pgmap v21014: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 466 KiB/s wr, 34 op/s 2023-09-10T10:11:15.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39106.254012 secs 2023-09-10T10:11:15.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:14 smithi098 ceph-mon[103124]: osdmap e1382: 12 total, 12 up, 12 in 2023-09-10T10:11:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39105.902378 secs 2023-09-10T10:11:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:14 smithi170 ceph-mon[102832]: pgmap v21014: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 466 KiB/s wr, 34 op/s 2023-09-10T10:11:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39106.254012 secs 2023-09-10T10:11:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:14 smithi170 ceph-mon[102832]: osdmap e1382: 12 total, 12 up, 12 in 2023-09-10T10:11:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:17 smithi170 ceph-mon[102832]: pgmap v21016: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 504 KiB/s wr, 21 op/s 2023-09-10T10:11:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:17 smithi082 ceph-mon[98694]: pgmap v21016: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 504 KiB/s wr, 21 op/s 2023-09-10T10:11:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:17 smithi098 ceph-mon[103124]: pgmap v21016: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 504 KiB/s wr, 21 op/s 2023-09-10T10:11:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:19 smithi170 ceph-mon[102832]: pgmap v21017: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 406 KiB/s wr, 17 op/s 2023-09-10T10:11:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39109.671933 secs 2023-09-10T10:11:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:19 smithi082 ceph-mon[98694]: pgmap v21017: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 406 KiB/s wr, 17 op/s 2023-09-10T10:11:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39109.671933 secs 2023-09-10T10:11:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:19 smithi098 ceph-mon[103124]: pgmap v21017: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 406 KiB/s wr, 17 op/s 2023-09-10T10:11:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39109.671933 secs 2023-09-10T10:11:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39110.902437 secs 2023-09-10T10:11:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39111.254078 secs 2023-09-10T10:11:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39110.902437 secs 2023-09-10T10:11:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39111.254078 secs 2023-09-10T10:11:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39110.902437 secs 2023-09-10T10:11:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39111.254078 secs 2023-09-10T10:11:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:21 smithi170 ceph-mon[102832]: pgmap v21018: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 403 KiB/s wr, 17 op/s 2023-09-10T10:11:21.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:21 smithi082 ceph-mon[98694]: pgmap v21018: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 403 KiB/s wr, 17 op/s 2023-09-10T10:11:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:21 smithi098 ceph-mon[103124]: pgmap v21018: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 403 KiB/s wr, 17 op/s 2023-09-10T10:11:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:23 smithi170 ceph-mon[102832]: pgmap v21019: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 14 op/s 2023-09-10T10:11:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:23 smithi082 ceph-mon[98694]: pgmap v21019: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 14 op/s 2023-09-10T10:11:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:23 smithi098 ceph-mon[103124]: pgmap v21019: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 14 op/s 2023-09-10T10:11:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39114.671994 secs 2023-09-10T10:11:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39114.671994 secs 2023-09-10T10:11:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39114.671994 secs 2023-09-10T10:11:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39115.902496 secs 2023-09-10T10:11:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:25 smithi170 ceph-mon[102832]: pgmap v21020: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 14 op/s 2023-09-10T10:11:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39116.254144 secs 2023-09-10T10:11:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39115.902496 secs 2023-09-10T10:11:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:25 smithi082 ceph-mon[98694]: pgmap v21020: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 14 op/s 2023-09-10T10:11:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39116.254144 secs 2023-09-10T10:11:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39115.902496 secs 2023-09-10T10:11:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:25 smithi098 ceph-mon[103124]: pgmap v21020: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 14 op/s 2023-09-10T10:11:25.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39116.254144 secs 2023-09-10T10:11:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:26 smithi170 ceph-mon[102832]: pgmap v21021: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 309 KiB/s wr, 18 op/s 2023-09-10T10:11:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:26 smithi082 ceph-mon[98694]: pgmap v21021: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 309 KiB/s wr, 18 op/s 2023-09-10T10:11:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:26 smithi098 ceph-mon[103124]: pgmap v21021: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 309 KiB/s wr, 18 op/s 2023-09-10T10:11:27.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:11:27.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:11:27.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:11:27.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:11:27.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:11:27.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:11:27.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:11:27.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:11:27.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:11:27.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:11:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:11:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:11:27.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:11:27.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:11:27.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:11:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:11:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:11:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:11:28.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:28 smithi170 ceph-mon[102832]: pgmap v21022: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 10 op/s 2023-09-10T10:11:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:11:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:11:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:11:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:28 smithi082 ceph-mon[98694]: pgmap v21022: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 10 op/s 2023-09-10T10:11:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:11:28.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:11:28.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:11:28.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:28 smithi098 ceph-mon[103124]: pgmap v21022: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 10 op/s 2023-09-10T10:11:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39119.672052 secs 2023-09-10T10:11:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39119.672052 secs 2023-09-10T10:11:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39119.672052 secs 2023-09-10T10:11:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39120.902557 secs 2023-09-10T10:11:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39121.254208 secs 2023-09-10T10:11:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:30 smithi170 ceph-mon[102832]: pgmap v21023: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 10 op/s 2023-09-10T10:11:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39120.902557 secs 2023-09-10T10:11:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39121.254208 secs 2023-09-10T10:11:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:30 smithi082 ceph-mon[98694]: pgmap v21023: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 10 op/s 2023-09-10T10:11:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39120.902557 secs 2023-09-10T10:11:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39121.254208 secs 2023-09-10T10:11:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:30 smithi098 ceph-mon[103124]: pgmap v21023: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 10 op/s 2023-09-10T10:11:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:33 smithi170 ceph-mon[102832]: pgmap v21024: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 146 KiB/s wr, 15 op/s 2023-09-10T10:11:33.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:33 smithi082 ceph-mon[98694]: pgmap v21024: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 146 KiB/s wr, 15 op/s 2023-09-10T10:11:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:33 smithi098 ceph-mon[103124]: pgmap v21024: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 146 KiB/s wr, 15 op/s 2023-09-10T10:11:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39124.672110 secs 2023-09-10T10:11:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:11:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39124.672110 secs 2023-09-10T10:11:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:11:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39124.672110 secs 2023-09-10T10:11:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:11:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39125.902637 secs 2023-09-10T10:11:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:35 smithi170 ceph-mon[102832]: pgmap v21025: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 10 op/s 2023-09-10T10:11:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39126.254275 secs 2023-09-10T10:11:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39125.902637 secs 2023-09-10T10:11:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:35 smithi082 ceph-mon[98694]: pgmap v21025: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 10 op/s 2023-09-10T10:11:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39126.254275 secs 2023-09-10T10:11:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39125.902637 secs 2023-09-10T10:11:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:35 smithi098 ceph-mon[103124]: pgmap v21025: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 10 op/s 2023-09-10T10:11:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39126.254275 secs 2023-09-10T10:11:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:36 smithi170 ceph-mon[102832]: pgmap v21026: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 288 KiB/s wr, 16 op/s 2023-09-10T10:11:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:11:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:36 smithi082 ceph-mon[98694]: pgmap v21026: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 288 KiB/s wr, 16 op/s 2023-09-10T10:11:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:11:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:36 smithi098 ceph-mon[103124]: pgmap v21026: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 288 KiB/s wr, 16 op/s 2023-09-10T10:11:36.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:11:36.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:39 smithi170 ceph-mon[102832]: pgmap v21027: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 10 op/s 2023-09-10T10:11:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:39 smithi082 ceph-mon[98694]: pgmap v21027: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 10 op/s 2023-09-10T10:11:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:39 smithi098 ceph-mon[103124]: pgmap v21027: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 10 op/s 2023-09-10T10:11:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39130.902700 secs 2023-09-10T10:11:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39131.254342 secs 2023-09-10T10:11:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:40 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:11:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39130.902700 secs 2023-09-10T10:11:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39131.254342 secs 2023-09-10T10:11:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:40 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:11:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39130.902700 secs 2023-09-10T10:11:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39131.254342 secs 2023-09-10T10:11:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:40 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:11:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:41 smithi170 ceph-mon[102832]: pgmap v21028: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 10 op/s 2023-09-10T10:11:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:41 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:11:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:41 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:11:41.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:41 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:41 smithi082 ceph-mon[98694]: pgmap v21028: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 10 op/s 2023-09-10T10:11:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:41 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:11:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:41 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:11:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:41 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:41 smithi098 ceph-mon[103124]: pgmap v21028: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 10 op/s 2023-09-10T10:11:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:41 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:11:41.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:41 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:11:41.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:41 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:43 smithi170 ceph-mon[102832]: pgmap v21029: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 465 KiB/s wr, 16 op/s 2023-09-10T10:11:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:43 smithi082 ceph-mon[98694]: pgmap v21029: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 465 KiB/s wr, 16 op/s 2023-09-10T10:11:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:43 smithi098 ceph-mon[103124]: pgmap v21029: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 465 KiB/s wr, 16 op/s 2023-09-10T10:11:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:44 smithi170 ceph-mon[102832]: pgmap v21030: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 465 KiB/s wr, 11 op/s 2023-09-10T10:11:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:44 smithi082 ceph-mon[98694]: pgmap v21030: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 465 KiB/s wr, 11 op/s 2023-09-10T10:11:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:44 smithi098 ceph-mon[103124]: pgmap v21030: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 465 KiB/s wr, 11 op/s 2023-09-10T10:11:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39135.902768 secs 2023-09-10T10:11:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39136.254410 secs 2023-09-10T10:11:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39135.902768 secs 2023-09-10T10:11:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39136.254410 secs 2023-09-10T10:11:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39135.902768 secs 2023-09-10T10:11:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39136.254410 secs 2023-09-10T10:11:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:11:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:46.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:46 smithi082 ceph-mon[98694]: pgmap v21031: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 777 KiB/s wr, 18 op/s 2023-09-10T10:11:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:11:47.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:47.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:46 smithi098 ceph-mon[103124]: pgmap v21031: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 777 KiB/s wr, 18 op/s 2023-09-10T10:11:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:11:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:46 smithi170 ceph-mon[102832]: pgmap v21031: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 777 KiB/s wr, 18 op/s 2023-09-10T10:11:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:49 smithi170 ceph-mon[102832]: pgmap v21032: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 635 KiB/s wr, 13 op/s 2023-09-10T10:11:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:49 smithi082 ceph-mon[98694]: pgmap v21032: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 635 KiB/s wr, 13 op/s 2023-09-10T10:11:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:49 smithi098 ceph-mon[103124]: pgmap v21032: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 635 KiB/s wr, 13 op/s 2023-09-10T10:11:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39140.902826 secs 2023-09-10T10:11:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39141.254481 secs 2023-09-10T10:11:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:11:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39140.902826 secs 2023-09-10T10:11:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39141.254481 secs 2023-09-10T10:11:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:11:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39140.902826 secs 2023-09-10T10:11:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39141.254481 secs 2023-09-10T10:11:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:11:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:51 smithi170 ceph-mon[102832]: pgmap v21033: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 635 KiB/s wr, 13 op/s 2023-09-10T10:11:51.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:51 smithi082 ceph-mon[98694]: pgmap v21033: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 635 KiB/s wr, 13 op/s 2023-09-10T10:11:51.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:51 smithi098 ceph-mon[103124]: pgmap v21033: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 635 KiB/s wr, 13 op/s 2023-09-10T10:11:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:53 smithi170 ceph-mon[102832]: pgmap v21034: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 944 KiB/s wr, 20 op/s 2023-09-10T10:11:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:11:53.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:53 smithi082 ceph-mon[98694]: pgmap v21034: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 944 KiB/s wr, 20 op/s 2023-09-10T10:11:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:11:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:53 smithi098 ceph-mon[103124]: pgmap v21034: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 944 KiB/s wr, 20 op/s 2023-09-10T10:11:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:11:53.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:54 smithi170 ceph-mon[102832]: pgmap v21035: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 621 KiB/s wr, 13 op/s 2023-09-10T10:11:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:54 smithi082 ceph-mon[98694]: pgmap v21035: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 621 KiB/s wr, 13 op/s 2023-09-10T10:11:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:54 smithi098 ceph-mon[103124]: pgmap v21035: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 621 KiB/s wr, 13 op/s 2023-09-10T10:11:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39145.902885 secs 2023-09-10T10:11:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39146.254550 secs 2023-09-10T10:11:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39145.902885 secs 2023-09-10T10:11:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39146.254550 secs 2023-09-10T10:11:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39145.902885 secs 2023-09-10T10:11:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39146.254550 secs 2023-09-10T10:11:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:56 smithi170 ceph-mon[102832]: pgmap v21036: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 883 KiB/s wr, 20 op/s 2023-09-10T10:11:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:56 smithi082 ceph-mon[98694]: pgmap v21036: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 883 KiB/s wr, 20 op/s 2023-09-10T10:11:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:56 smithi098 ceph-mon[103124]: pgmap v21036: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 883 KiB/s wr, 20 op/s 2023-09-10T10:11:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:11:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:11:58 smithi170 ceph-mon[102832]: pgmap v21037: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 571 KiB/s wr, 13 op/s 2023-09-10T10:11:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:11:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:11:58 smithi082 ceph-mon[98694]: pgmap v21037: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 571 KiB/s wr, 13 op/s 2023-09-10T10:11:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:11:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:11:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:11:58 smithi098 ceph-mon[103124]: pgmap v21037: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 571 KiB/s wr, 13 op/s 2023-09-10T10:12:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39150.902943 secs 2023-09-10T10:12:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39151.254617 secs 2023-09-10T10:12:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:12:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:00 smithi098 ceph-mon[103124]: pgmap v21038: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 571 KiB/s wr, 13 op/s 2023-09-10T10:12:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39150.902943 secs 2023-09-10T10:12:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39151.254617 secs 2023-09-10T10:12:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:12:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:00 smithi170 ceph-mon[102832]: pgmap v21038: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 571 KiB/s wr, 13 op/s 2023-09-10T10:12:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39150.902943 secs 2023-09-10T10:12:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39151.254617 secs 2023-09-10T10:12:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:12:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:00 smithi082 ceph-mon[98694]: pgmap v21038: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 571 KiB/s wr, 13 op/s 2023-09-10T10:12:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:03 smithi170 ceph-mon[102832]: pgmap v21039: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 862 KiB/s wr, 33 op/s 2023-09-10T10:12:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:03 smithi082 ceph-mon[98694]: pgmap v21039: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 862 KiB/s wr, 33 op/s 2023-09-10T10:12:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:03 smithi098 ceph-mon[103124]: pgmap v21039: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 862 KiB/s wr, 33 op/s 2023-09-10T10:12:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39154.672513 secs 2023-09-10T10:12:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39154.672513 secs 2023-09-10T10:12:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39154.672513 secs 2023-09-10T10:12:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39155.903027 secs 2023-09-10T10:12:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:05 smithi170 ceph-mon[102832]: pgmap v21040: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 553 KiB/s wr, 26 op/s 2023-09-10T10:12:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39156.254683 secs 2023-09-10T10:12:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:05 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:12:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:12:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39155.903027 secs 2023-09-10T10:12:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:05 smithi082 ceph-mon[98694]: pgmap v21040: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 553 KiB/s wr, 26 op/s 2023-09-10T10:12:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39156.254683 secs 2023-09-10T10:12:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:05 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:12:05.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:12:05.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39155.903027 secs 2023-09-10T10:12:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:05 smithi098 ceph-mon[103124]: pgmap v21040: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 553 KiB/s wr, 26 op/s 2023-09-10T10:12:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39156.254683 secs 2023-09-10T10:12:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:05 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:12:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:12:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:07.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:07 smithi170 ceph-mon[102832]: pgmap v21041: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 715 KiB/s wr, 34 op/s 2023-09-10T10:12:07.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:07 smithi082 ceph-mon[98694]: pgmap v21041: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 715 KiB/s wr, 34 op/s 2023-09-10T10:12:07.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:07 smithi098 ceph-mon[103124]: pgmap v21041: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 715 KiB/s wr, 34 op/s 2023-09-10T10:12:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:08 smithi170 ceph-mon[102832]: pgmap v21042: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 453 KiB/s wr, 28 op/s 2023-09-10T10:12:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:08 smithi082 ceph-mon[98694]: pgmap v21042: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 453 KiB/s wr, 28 op/s 2023-09-10T10:12:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:08 smithi098 ceph-mon[103124]: pgmap v21042: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 453 KiB/s wr, 28 op/s 2023-09-10T10:12:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39159.672576 secs 2023-09-10T10:12:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:09 smithi170 ceph-mon[102832]: osdmap e1383: 12 total, 12 up, 12 in 2023-09-10T10:12:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39159.672576 secs 2023-09-10T10:12:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:09 smithi082 ceph-mon[98694]: osdmap e1383: 12 total, 12 up, 12 in 2023-09-10T10:12:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39159.672576 secs 2023-09-10T10:12:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:09 smithi098 ceph-mon[103124]: osdmap e1383: 12 total, 12 up, 12 in 2023-09-10T10:12:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39160.903085 secs 2023-09-10T10:12:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39161.254749 secs 2023-09-10T10:12:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:10 smithi170 ceph-mon[102832]: pgmap v21044: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 543 KiB/s wr, 33 op/s 2023-09-10T10:12:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39160.903085 secs 2023-09-10T10:12:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39161.254749 secs 2023-09-10T10:12:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:10 smithi082 ceph-mon[98694]: pgmap v21044: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 543 KiB/s wr, 33 op/s 2023-09-10T10:12:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39160.903085 secs 2023-09-10T10:12:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39161.254749 secs 2023-09-10T10:12:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:10 smithi098 ceph-mon[103124]: pgmap v21044: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 543 KiB/s wr, 33 op/s 2023-09-10T10:12:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:13 smithi170 ceph-mon[102832]: pgmap v21045: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 440 KiB/s wr, 16 op/s 2023-09-10T10:12:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:13 smithi082 ceph-mon[98694]: pgmap v21045: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 440 KiB/s wr, 16 op/s 2023-09-10T10:12:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:13 smithi098 ceph-mon[103124]: pgmap v21045: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 440 KiB/s wr, 16 op/s 2023-09-10T10:12:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39164.672671 secs 2023-09-10T10:12:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39164.672671 secs 2023-09-10T10:12:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39164.672671 secs 2023-09-10T10:12:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39165.903168 secs 2023-09-10T10:12:15.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:15 smithi170 ceph-mon[102832]: pgmap v21046: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 440 KiB/s wr, 16 op/s 2023-09-10T10:12:15.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39166.254814 secs 2023-09-10T10:12:15.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:15 smithi170 ceph-mon[102832]: osdmap e1384: 12 total, 12 up, 12 in 2023-09-10T10:12:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39165.903168 secs 2023-09-10T10:12:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:15 smithi082 ceph-mon[98694]: pgmap v21046: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 440 KiB/s wr, 16 op/s 2023-09-10T10:12:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39166.254814 secs 2023-09-10T10:12:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:15 smithi082 ceph-mon[98694]: osdmap e1384: 12 total, 12 up, 12 in 2023-09-10T10:12:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39165.903168 secs 2023-09-10T10:12:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:15 smithi098 ceph-mon[103124]: pgmap v21046: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 440 KiB/s wr, 16 op/s 2023-09-10T10:12:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39166.254814 secs 2023-09-10T10:12:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:15 smithi098 ceph-mon[103124]: osdmap e1384: 12 total, 12 up, 12 in 2023-09-10T10:12:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:16 smithi170 ceph-mon[102832]: pgmap v21048: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 524 KiB/s wr, 18 op/s 2023-09-10T10:12:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:16 smithi082 ceph-mon[98694]: pgmap v21048: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 524 KiB/s wr, 18 op/s 2023-09-10T10:12:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:16 smithi098 ceph-mon[103124]: pgmap v21048: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 524 KiB/s wr, 18 op/s 2023-09-10T10:12:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:17 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:12:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:17 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:17 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:12:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:17 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:17 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:12:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:17 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:12:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:18 smithi170 ceph-mon[102832]: pgmap v21049: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 422 KiB/s wr, 14 op/s 2023-09-10T10:12:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:12:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:18 smithi082 ceph-mon[98694]: pgmap v21049: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 422 KiB/s wr, 14 op/s 2023-09-10T10:12:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:12:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:18 smithi098 ceph-mon[103124]: pgmap v21049: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 422 KiB/s wr, 14 op/s 2023-09-10T10:12:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39169.672779 secs 2023-09-10T10:12:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39169.672779 secs 2023-09-10T10:12:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39169.672779 secs 2023-09-10T10:12:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39170.903225 secs 2023-09-10T10:12:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39171.254882 secs 2023-09-10T10:12:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:20 smithi170 ceph-mon[102832]: pgmap v21050: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 419 KiB/s wr, 14 op/s 2023-09-10T10:12:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39170.903225 secs 2023-09-10T10:12:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39171.254882 secs 2023-09-10T10:12:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:20 smithi082 ceph-mon[98694]: pgmap v21050: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 419 KiB/s wr, 14 op/s 2023-09-10T10:12:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39170.903225 secs 2023-09-10T10:12:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39171.254882 secs 2023-09-10T10:12:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:20 smithi098 ceph-mon[103124]: pgmap v21050: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 419 KiB/s wr, 14 op/s 2023-09-10T10:12:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:23 smithi170 ceph-mon[102832]: pgmap v21051: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 13 op/s 2023-09-10T10:12:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:23 smithi082 ceph-mon[98694]: pgmap v21051: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 13 op/s 2023-09-10T10:12:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:23 smithi098 ceph-mon[103124]: pgmap v21051: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 13 op/s 2023-09-10T10:12:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39174.672850 secs 2023-09-10T10:12:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39174.672850 secs 2023-09-10T10:12:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39174.672850 secs 2023-09-10T10:12:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39175.903282 secs 2023-09-10T10:12:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:25 smithi170 ceph-mon[102832]: pgmap v21052: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 13 op/s 2023-09-10T10:12:25.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39176.254944 secs 2023-09-10T10:12:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39175.903282 secs 2023-09-10T10:12:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:25 smithi082 ceph-mon[98694]: pgmap v21052: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 13 op/s 2023-09-10T10:12:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39176.254944 secs 2023-09-10T10:12:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39175.903282 secs 2023-09-10T10:12:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:25 smithi098 ceph-mon[103124]: pgmap v21052: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 174 KiB/s wr, 13 op/s 2023-09-10T10:12:25.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39176.254944 secs 2023-09-10T10:12:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:26 smithi170 ceph-mon[102832]: pgmap v21053: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 310 KiB/s wr, 18 op/s 2023-09-10T10:12:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:26 smithi082 ceph-mon[98694]: pgmap v21053: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 310 KiB/s wr, 18 op/s 2023-09-10T10:12:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:26 smithi098 ceph-mon[103124]: pgmap v21053: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 310 KiB/s wr, 18 op/s 2023-09-10T10:12:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:27 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:12:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:27 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:12:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:27 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:12:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39179.672913 secs 2023-09-10T10:12:29.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:28 smithi170 ceph-mon[102832]: pgmap v21054: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T10:12:29.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:12:29.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:12:29.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:28 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:12:29.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39179.672913 secs 2023-09-10T10:12:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:28 smithi082 ceph-mon[98694]: pgmap v21054: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T10:12:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:12:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:12:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:28 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:12:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39179.672913 secs 2023-09-10T10:12:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:28 smithi098 ceph-mon[103124]: pgmap v21054: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T10:12:29.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:12:29.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:12:29.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:28 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:12:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39180.903339 secs 2023-09-10T10:12:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39181.255007 secs 2023-09-10T10:12:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39180.903339 secs 2023-09-10T10:12:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39181.255007 secs 2023-09-10T10:12:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39180.903339 secs 2023-09-10T10:12:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39181.255007 secs 2023-09-10T10:12:31.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:30 smithi170 ceph-mon[102832]: pgmap v21055: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T10:12:31.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:30 smithi082 ceph-mon[98694]: pgmap v21055: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T10:12:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:30 smithi098 ceph-mon[103124]: pgmap v21055: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T10:12:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:33 smithi170 ceph-mon[102832]: pgmap v21056: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T10:12:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:33 smithi082 ceph-mon[98694]: pgmap v21056: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T10:12:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:33 smithi098 ceph-mon[103124]: pgmap v21056: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T10:12:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39184.672974 secs 2023-09-10T10:12:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:12:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:34 smithi170 ceph-mon[102832]: pgmap v21057: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T10:12:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39184.672974 secs 2023-09-10T10:12:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:12:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:34 smithi082 ceph-mon[98694]: pgmap v21057: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T10:12:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39184.672974 secs 2023-09-10T10:12:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:12:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:34 smithi098 ceph-mon[103124]: pgmap v21057: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 11 op/s 2023-09-10T10:12:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39185.903399 secs 2023-09-10T10:12:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39186.255074 secs 2023-09-10T10:12:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39185.903399 secs 2023-09-10T10:12:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39186.255074 secs 2023-09-10T10:12:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39185.903399 secs 2023-09-10T10:12:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39186.255074 secs 2023-09-10T10:12:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:36 smithi170 ceph-mon[102832]: pgmap v21058: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 288 KiB/s wr, 16 op/s 2023-09-10T10:12:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:36 smithi082 ceph-mon[98694]: pgmap v21058: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 288 KiB/s wr, 16 op/s 2023-09-10T10:12:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:36 smithi098 ceph-mon[103124]: pgmap v21058: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 288 KiB/s wr, 16 op/s 2023-09-10T10:12:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:12:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:12:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:12:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:38 smithi170 ceph-mon[102832]: pgmap v21059: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 10 op/s 2023-09-10T10:12:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:38 smithi082 ceph-mon[98694]: pgmap v21059: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 10 op/s 2023-09-10T10:12:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:38 smithi098 ceph-mon[103124]: pgmap v21059: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 10 op/s 2023-09-10T10:12:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:12:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:12:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:12:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39190.903456 secs 2023-09-10T10:12:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39191.255139 secs 2023-09-10T10:12:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:12:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:12:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:40 smithi098 ceph-mon[103124]: pgmap v21060: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 10 op/s 2023-09-10T10:12:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39190.903456 secs 2023-09-10T10:12:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39191.255139 secs 2023-09-10T10:12:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:12:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:12:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:40 smithi170 ceph-mon[102832]: pgmap v21060: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 10 op/s 2023-09-10T10:12:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39190.903456 secs 2023-09-10T10:12:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39191.255139 secs 2023-09-10T10:12:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:12:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:12:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:40 smithi082 ceph-mon[98694]: pgmap v21060: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 142 KiB/s wr, 10 op/s 2023-09-10T10:12:43.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:43 smithi170 ceph-mon[102832]: pgmap v21061: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 469 KiB/s wr, 17 op/s 2023-09-10T10:12:43.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:43 smithi082 ceph-mon[98694]: pgmap v21061: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 469 KiB/s wr, 17 op/s 2023-09-10T10:12:43.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:43 smithi098 ceph-mon[103124]: pgmap v21061: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 469 KiB/s wr, 17 op/s 2023-09-10T10:12:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39195.903502 secs 2023-09-10T10:12:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:45 smithi170 ceph-mon[102832]: pgmap v21062: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 469 KiB/s wr, 12 op/s 2023-09-10T10:12:45.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39196.255204 secs 2023-09-10T10:12:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39195.903502 secs 2023-09-10T10:12:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:45 smithi082 ceph-mon[98694]: pgmap v21062: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 469 KiB/s wr, 12 op/s 2023-09-10T10:12:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39196.255204 secs 2023-09-10T10:12:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39195.903502 secs 2023-09-10T10:12:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:45 smithi098 ceph-mon[103124]: pgmap v21062: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 469 KiB/s wr, 12 op/s 2023-09-10T10:12:45.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39196.255204 secs 2023-09-10T10:12:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:12:47.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:47.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:46 smithi098 ceph-mon[103124]: pgmap v21063: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 781 KiB/s wr, 19 op/s 2023-09-10T10:12:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:12:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:46 smithi170 ceph-mon[102832]: pgmap v21063: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 781 KiB/s wr, 19 op/s 2023-09-10T10:12:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:12:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:46 smithi082 ceph-mon[98694]: pgmap v21063: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 781 KiB/s wr, 19 op/s 2023-09-10T10:12:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:49 smithi170 ceph-mon[102832]: pgmap v21064: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 639 KiB/s wr, 13 op/s 2023-09-10T10:12:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:49 smithi082 ceph-mon[98694]: pgmap v21064: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 639 KiB/s wr, 13 op/s 2023-09-10T10:12:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:49 smithi098 ceph-mon[103124]: pgmap v21064: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 639 KiB/s wr, 13 op/s 2023-09-10T10:12:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39200.903559 secs 2023-09-10T10:12:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39201.255271 secs 2023-09-10T10:12:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:12:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:50 smithi170 ceph-mon[102832]: pgmap v21065: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 639 KiB/s wr, 13 op/s 2023-09-10T10:12:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39200.903559 secs 2023-09-10T10:12:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39201.255271 secs 2023-09-10T10:12:50.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:12:50.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:50.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:50 smithi082 ceph-mon[98694]: pgmap v21065: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 639 KiB/s wr, 13 op/s 2023-09-10T10:12:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39200.903559 secs 2023-09-10T10:12:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39201.255271 secs 2023-09-10T10:12:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:12:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:50.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:50 smithi098 ceph-mon[103124]: pgmap v21065: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 639 KiB/s wr, 13 op/s 2023-09-10T10:12:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:12:53.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:53.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:52 smithi098 ceph-mon[103124]: pgmap v21066: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 950 KiB/s wr, 20 op/s 2023-09-10T10:12:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:12:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:52 smithi170 ceph-mon[102832]: pgmap v21066: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 950 KiB/s wr, 20 op/s 2023-09-10T10:12:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:12:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:52 smithi082 ceph-mon[98694]: pgmap v21066: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 950 KiB/s wr, 20 op/s 2023-09-10T10:12:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39205.903616 secs 2023-09-10T10:12:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:55 smithi170 ceph-mon[102832]: pgmap v21067: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 623 KiB/s wr, 14 op/s 2023-09-10T10:12:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39206.255340 secs 2023-09-10T10:12:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39205.903616 secs 2023-09-10T10:12:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:55 smithi082 ceph-mon[98694]: pgmap v21067: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 623 KiB/s wr, 14 op/s 2023-09-10T10:12:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39206.255340 secs 2023-09-10T10:12:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39205.903616 secs 2023-09-10T10:12:55.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:55 smithi098 ceph-mon[103124]: pgmap v21067: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 623 KiB/s wr, 14 op/s 2023-09-10T10:12:55.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39206.255340 secs 2023-09-10T10:12:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:56 smithi170 ceph-mon[102832]: pgmap v21068: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 885 KiB/s wr, 20 op/s 2023-09-10T10:12:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:56 smithi082 ceph-mon[98694]: pgmap v21068: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 885 KiB/s wr, 20 op/s 2023-09-10T10:12:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:56 smithi098 ceph-mon[103124]: pgmap v21068: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 885 KiB/s wr, 20 op/s 2023-09-10T10:12:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:12:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:12:58 smithi170 ceph-mon[102832]: pgmap v21069: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 572 KiB/s wr, 13 op/s 2023-09-10T10:12:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:12:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:12:58 smithi082 ceph-mon[98694]: pgmap v21069: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 572 KiB/s wr, 13 op/s 2023-09-10T10:12:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:12:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:12:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:12:58 smithi098 ceph-mon[103124]: pgmap v21069: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 572 KiB/s wr, 13 op/s 2023-09-10T10:13:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39210.903675 secs 2023-09-10T10:13:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39211.255421 secs 2023-09-10T10:13:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:13:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:00 smithi098 ceph-mon[103124]: pgmap v21070: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 572 KiB/s wr, 13 op/s 2023-09-10T10:13:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39210.903675 secs 2023-09-10T10:13:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39211.255421 secs 2023-09-10T10:13:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:13:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:00 smithi170 ceph-mon[102832]: pgmap v21070: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 572 KiB/s wr, 13 op/s 2023-09-10T10:13:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39210.903675 secs 2023-09-10T10:13:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39211.255421 secs 2023-09-10T10:13:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:13:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:00 smithi082 ceph-mon[98694]: pgmap v21070: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 572 KiB/s wr, 13 op/s 2023-09-10T10:13:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:03 smithi170 ceph-mon[102832]: pgmap v21071: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 842 KiB/s wr, 23 op/s 2023-09-10T10:13:03.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:03 smithi082 ceph-mon[98694]: pgmap v21071: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 842 KiB/s wr, 23 op/s 2023-09-10T10:13:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:03 smithi098 ceph-mon[103124]: pgmap v21071: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 842 KiB/s wr, 23 op/s 2023-09-10T10:13:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39214.673426 secs 2023-09-10T10:13:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:04 smithi170 ceph-mon[102832]: pgmap v21072: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 531 KiB/s wr, 16 op/s 2023-09-10T10:13:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39214.673426 secs 2023-09-10T10:13:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:04 smithi082 ceph-mon[98694]: pgmap v21072: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 531 KiB/s wr, 16 op/s 2023-09-10T10:13:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39214.673426 secs 2023-09-10T10:13:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:04 smithi098 ceph-mon[103124]: pgmap v21072: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 531 KiB/s wr, 16 op/s 2023-09-10T10:13:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39215.903731 secs 2023-09-10T10:13:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39216.255493 secs 2023-09-10T10:13:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:05 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:13:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:13:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39215.903731 secs 2023-09-10T10:13:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39216.255493 secs 2023-09-10T10:13:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:05 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:13:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:13:05.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39215.903731 secs 2023-09-10T10:13:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39216.255493 secs 2023-09-10T10:13:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:05 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:13:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:13:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:06 smithi170 ceph-mon[102832]: pgmap v21073: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 787 KiB/s wr, 35 op/s 2023-09-10T10:13:06.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:06 smithi082 ceph-mon[98694]: pgmap v21073: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 787 KiB/s wr, 35 op/s 2023-09-10T10:13:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:06 smithi098 ceph-mon[103124]: pgmap v21073: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 787 KiB/s wr, 35 op/s 2023-09-10T10:13:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39219.673484 secs 2023-09-10T10:13:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:09 smithi170 ceph-mon[102832]: pgmap v21074: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 525 KiB/s wr, 29 op/s 2023-09-10T10:13:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:09 smithi170 ceph-mon[102832]: osdmap e1385: 12 total, 12 up, 12 in 2023-09-10T10:13:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39219.673484 secs 2023-09-10T10:13:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:09 smithi082 ceph-mon[98694]: pgmap v21074: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 525 KiB/s wr, 29 op/s 2023-09-10T10:13:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:09 smithi082 ceph-mon[98694]: osdmap e1385: 12 total, 12 up, 12 in 2023-09-10T10:13:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39219.673484 secs 2023-09-10T10:13:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:09 smithi098 ceph-mon[103124]: pgmap v21074: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 525 KiB/s wr, 29 op/s 2023-09-10T10:13:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:09 smithi098 ceph-mon[103124]: osdmap e1385: 12 total, 12 up, 12 in 2023-09-10T10:13:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39220.903788 secs 2023-09-10T10:13:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39221.255559 secs 2023-09-10T10:13:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:13:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:10.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:10 smithi170 ceph-mon[102832]: pgmap v21076: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 630 KiB/s wr, 35 op/s 2023-09-10T10:13:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39220.903788 secs 2023-09-10T10:13:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39221.255559 secs 2023-09-10T10:13:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:13:10.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:10.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:10 smithi082 ceph-mon[98694]: pgmap v21076: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 630 KiB/s wr, 35 op/s 2023-09-10T10:13:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39220.903788 secs 2023-09-10T10:13:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39221.255559 secs 2023-09-10T10:13:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:13:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:10 smithi098 ceph-mon[103124]: pgmap v21076: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 630 KiB/s wr, 35 op/s 2023-09-10T10:13:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:13:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:13.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:12 smithi098 ceph-mon[103124]: pgmap v21077: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 634 KiB/s wr, 31 op/s 2023-09-10T10:13:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:13:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:13.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:12 smithi170 ceph-mon[102832]: pgmap v21077: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 634 KiB/s wr, 31 op/s 2023-09-10T10:13:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:13:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:12 smithi082 ceph-mon[98694]: pgmap v21077: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 634 KiB/s wr, 31 op/s 2023-09-10T10:13:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39224.673544 secs 2023-09-10T10:13:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39224.673544 secs 2023-09-10T10:13:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39224.673544 secs 2023-09-10T10:13:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39225.903847 secs 2023-09-10T10:13:15.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39226.255625 secs 2023-09-10T10:13:15.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:14 smithi098 ceph-mon[103124]: pgmap v21078: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 634 KiB/s wr, 31 op/s 2023-09-10T10:13:15.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:14 smithi098 ceph-mon[103124]: osdmap e1386: 12 total, 12 up, 12 in 2023-09-10T10:13:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39225.903847 secs 2023-09-10T10:13:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39226.255625 secs 2023-09-10T10:13:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:14 smithi170 ceph-mon[102832]: pgmap v21078: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 634 KiB/s wr, 31 op/s 2023-09-10T10:13:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:14 smithi170 ceph-mon[102832]: osdmap e1386: 12 total, 12 up, 12 in 2023-09-10T10:13:15.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39225.903847 secs 2023-09-10T10:13:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39226.255625 secs 2023-09-10T10:13:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:14 smithi082 ceph-mon[98694]: pgmap v21078: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 634 KiB/s wr, 31 op/s 2023-09-10T10:13:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:14 smithi082 ceph-mon[98694]: osdmap e1386: 12 total, 12 up, 12 in 2023-09-10T10:13:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:16 smithi170 ceph-mon[102832]: pgmap v21080: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 582 KiB/s wr, 19 op/s 2023-09-10T10:13:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:16 smithi082 ceph-mon[98694]: pgmap v21080: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 582 KiB/s wr, 19 op/s 2023-09-10T10:13:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:16 smithi098 ceph-mon[103124]: pgmap v21080: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 582 KiB/s wr, 19 op/s 2023-09-10T10:13:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:18 smithi170 ceph-mon[102832]: pgmap v21081: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 468 KiB/s wr, 15 op/s 2023-09-10T10:13:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:18 smithi082 ceph-mon[98694]: pgmap v21081: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 468 KiB/s wr, 15 op/s 2023-09-10T10:13:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:18 smithi098 ceph-mon[103124]: pgmap v21081: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 468 KiB/s wr, 15 op/s 2023-09-10T10:13:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39229.673605 secs 2023-09-10T10:13:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39229.673605 secs 2023-09-10T10:13:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39229.673605 secs 2023-09-10T10:13:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39230.903904 secs 2023-09-10T10:13:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39231.255714 secs 2023-09-10T10:13:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:20 smithi170 ceph-mon[102832]: pgmap v21082: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 466 KiB/s wr, 15 op/s 2023-09-10T10:13:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39230.903904 secs 2023-09-10T10:13:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39231.255714 secs 2023-09-10T10:13:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:20 smithi082 ceph-mon[98694]: pgmap v21082: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 466 KiB/s wr, 15 op/s 2023-09-10T10:13:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39230.903904 secs 2023-09-10T10:13:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39231.255714 secs 2023-09-10T10:13:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:20 smithi098 ceph-mon[103124]: pgmap v21082: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 466 KiB/s wr, 15 op/s 2023-09-10T10:13:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:22 smithi170 ceph-mon[102832]: pgmap v21083: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 140 KiB/s wr, 13 op/s 2023-09-10T10:13:22.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:22 smithi082 ceph-mon[98694]: pgmap v21083: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 140 KiB/s wr, 13 op/s 2023-09-10T10:13:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:22 smithi098 ceph-mon[103124]: pgmap v21083: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 140 KiB/s wr, 13 op/s 2023-09-10T10:13:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39234.673669 secs 2023-09-10T10:13:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:13:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:24 smithi098 ceph-mon[103124]: pgmap v21084: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 140 KiB/s wr, 13 op/s 2023-09-10T10:13:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39234.673669 secs 2023-09-10T10:13:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:13:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:24 smithi170 ceph-mon[102832]: pgmap v21084: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 140 KiB/s wr, 13 op/s 2023-09-10T10:13:24.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39234.673669 secs 2023-09-10T10:13:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:13:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:24 smithi082 ceph-mon[98694]: pgmap v21084: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 140 KiB/s wr, 13 op/s 2023-09-10T10:13:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39235.903965 secs 2023-09-10T10:13:25.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39236.255781 secs 2023-09-10T10:13:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39235.903965 secs 2023-09-10T10:13:25.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39236.255781 secs 2023-09-10T10:13:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39235.903965 secs 2023-09-10T10:13:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39236.255781 secs 2023-09-10T10:13:27.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:26 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:13:27.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:27.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:26 smithi098 ceph-mon[103124]: pgmap v21085: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 279 KiB/s wr, 18 op/s 2023-09-10T10:13:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:26 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:13:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:26 smithi170 ceph-mon[102832]: pgmap v21085: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 279 KiB/s wr, 18 op/s 2023-09-10T10:13:27.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:26 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:13:27.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:27.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:26 smithi082 ceph-mon[98694]: pgmap v21085: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 279 KiB/s wr, 18 op/s 2023-09-10T10:13:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:28 smithi170 ceph-mon[102832]: pgmap v21086: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:13:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:28 smithi082 ceph-mon[98694]: pgmap v21086: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:13:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:28 smithi098 ceph-mon[103124]: pgmap v21086: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:13:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39239.673729 secs 2023-09-10T10:13:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:13:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:13:29.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:13:29.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:29 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:13:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39239.673729 secs 2023-09-10T10:13:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:13:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:13:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:13:29.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:29 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:13:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39239.673729 secs 2023-09-10T10:13:29.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:13:29.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:13:29.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:13:29.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:29 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:13:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39240.904025 secs 2023-09-10T10:13:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39241.255846 secs 2023-09-10T10:13:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:30 smithi170 ceph-mon[102832]: pgmap v21087: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:13:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39240.904025 secs 2023-09-10T10:13:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39241.255846 secs 2023-09-10T10:13:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:30 smithi082 ceph-mon[98694]: pgmap v21087: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:13:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39240.904025 secs 2023-09-10T10:13:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39241.255846 secs 2023-09-10T10:13:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:30 smithi098 ceph-mon[103124]: pgmap v21087: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:13:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:32 smithi170 ceph-mon[102832]: pgmap v21088: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T10:13:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:32 smithi082 ceph-mon[98694]: pgmap v21088: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T10:13:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:32 smithi098 ceph-mon[103124]: pgmap v21088: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T10:13:33.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39244.673786 secs 2023-09-10T10:13:33.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39244.673786 secs 2023-09-10T10:13:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39244.673786 secs 2023-09-10T10:13:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:34 smithi170 ceph-mon[102832]: pgmap v21089: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 10 op/s 2023-09-10T10:13:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:34 smithi082 ceph-mon[98694]: pgmap v21089: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 10 op/s 2023-09-10T10:13:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:34 smithi098 ceph-mon[103124]: pgmap v21089: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 10 op/s 2023-09-10T10:13:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39245.904082 secs 2023-09-10T10:13:35.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39246.255910 secs 2023-09-10T10:13:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39245.904082 secs 2023-09-10T10:13:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39246.255910 secs 2023-09-10T10:13:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39245.904082 secs 2023-09-10T10:13:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39246.255910 secs 2023-09-10T10:13:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:36 smithi170 ceph-mon[102832]: pgmap v21090: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 326 KiB/s wr, 16 op/s 2023-09-10T10:13:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:36 smithi082 ceph-mon[98694]: pgmap v21090: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 326 KiB/s wr, 16 op/s 2023-09-10T10:13:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:36 smithi098 ceph-mon[103124]: pgmap v21090: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 326 KiB/s wr, 16 op/s 2023-09-10T10:13:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:13:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:38.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:38 smithi170 ceph-mon[102832]: pgmap v21091: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 180 KiB/s wr, 10 op/s 2023-09-10T10:13:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:13:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:38 smithi082 ceph-mon[98694]: pgmap v21091: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 180 KiB/s wr, 10 op/s 2023-09-10T10:13:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:13:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:38.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:38 smithi098 ceph-mon[103124]: pgmap v21091: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 180 KiB/s wr, 10 op/s 2023-09-10T10:13:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:13:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:13:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:13:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39250.904144 secs 2023-09-10T10:13:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39251.255978 secs 2023-09-10T10:13:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:13:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:13:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:40 smithi170 ceph-mon[102832]: pgmap v21092: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 180 KiB/s wr, 10 op/s 2023-09-10T10:13:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39250.904144 secs 2023-09-10T10:13:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39251.255978 secs 2023-09-10T10:13:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:13:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:13:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:40 smithi082 ceph-mon[98694]: pgmap v21092: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 180 KiB/s wr, 10 op/s 2023-09-10T10:13:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39250.904144 secs 2023-09-10T10:13:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39251.255978 secs 2023-09-10T10:13:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:13:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:13:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:40 smithi098 ceph-mon[103124]: pgmap v21092: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 180 KiB/s wr, 10 op/s 2023-09-10T10:13:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:42 smithi170 ceph-mon[102832]: pgmap v21093: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 551 KiB/s wr, 17 op/s 2023-09-10T10:13:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:42 smithi082 ceph-mon[98694]: pgmap v21093: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 551 KiB/s wr, 17 op/s 2023-09-10T10:13:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:42 smithi098 ceph-mon[103124]: pgmap v21093: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 551 KiB/s wr, 17 op/s 2023-09-10T10:13:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:44 smithi170 ceph-mon[102832]: pgmap v21094: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 551 KiB/s wr, 12 op/s 2023-09-10T10:13:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:44 smithi082 ceph-mon[98694]: pgmap v21094: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 551 KiB/s wr, 12 op/s 2023-09-10T10:13:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:44 smithi098 ceph-mon[103124]: pgmap v21094: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 551 KiB/s wr, 12 op/s 2023-09-10T10:13:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39255.904228 secs 2023-09-10T10:13:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39256.256044 secs 2023-09-10T10:13:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39255.904228 secs 2023-09-10T10:13:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39256.256044 secs 2023-09-10T10:13:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39255.904228 secs 2023-09-10T10:13:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39256.256044 secs 2023-09-10T10:13:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:46 smithi170 ceph-mon[102832]: pgmap v21095: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 780 KiB/s wr, 19 op/s 2023-09-10T10:13:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:46 smithi082 ceph-mon[98694]: pgmap v21095: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 780 KiB/s wr, 19 op/s 2023-09-10T10:13:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:46 smithi098 ceph-mon[103124]: pgmap v21095: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 780 KiB/s wr, 19 op/s 2023-09-10T10:13:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:48 smithi170 ceph-mon[102832]: pgmap v21096: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 600 KiB/s wr, 13 op/s 2023-09-10T10:13:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:48 smithi082 ceph-mon[98694]: pgmap v21096: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 600 KiB/s wr, 13 op/s 2023-09-10T10:13:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:48 smithi098 ceph-mon[103124]: pgmap v21096: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 600 KiB/s wr, 13 op/s 2023-09-10T10:13:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39260.904288 secs 2023-09-10T10:13:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39261.256110 secs 2023-09-10T10:13:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:13:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:50 smithi170 ceph-mon[102832]: pgmap v21097: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 600 KiB/s wr, 13 op/s 2023-09-10T10:13:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39260.904288 secs 2023-09-10T10:13:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39261.256110 secs 2023-09-10T10:13:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:13:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:50.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:50 smithi082 ceph-mon[98694]: pgmap v21097: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 600 KiB/s wr, 13 op/s 2023-09-10T10:13:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39260.904288 secs 2023-09-10T10:13:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39261.256110 secs 2023-09-10T10:13:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:13:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:50 smithi098 ceph-mon[103124]: pgmap v21097: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 600 KiB/s wr, 13 op/s 2023-09-10T10:13:53.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:13:53.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:53.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:52 smithi098 ceph-mon[103124]: pgmap v21098: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 869 KiB/s wr, 21 op/s 2023-09-10T10:13:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:13:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:53.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:52 smithi170 ceph-mon[102832]: pgmap v21098: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 869 KiB/s wr, 21 op/s 2023-09-10T10:13:53.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:13:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:52 smithi082 ceph-mon[98694]: pgmap v21098: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 869 KiB/s wr, 21 op/s 2023-09-10T10:13:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:54 smithi170 ceph-mon[102832]: pgmap v21099: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 498 KiB/s wr, 14 op/s 2023-09-10T10:13:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:54 smithi082 ceph-mon[98694]: pgmap v21099: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 498 KiB/s wr, 14 op/s 2023-09-10T10:13:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:54 smithi098 ceph-mon[103124]: pgmap v21099: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 498 KiB/s wr, 14 op/s 2023-09-10T10:13:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39265.904348 secs 2023-09-10T10:13:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39266.256176 secs 2023-09-10T10:13:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39265.904348 secs 2023-09-10T10:13:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39266.256176 secs 2023-09-10T10:13:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39265.904348 secs 2023-09-10T10:13:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39266.256176 secs 2023-09-10T10:13:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:56 smithi170 ceph-mon[102832]: pgmap v21100: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 803 KiB/s wr, 20 op/s 2023-09-10T10:13:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:56 smithi082 ceph-mon[98694]: pgmap v21100: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 803 KiB/s wr, 20 op/s 2023-09-10T10:13:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:56 smithi098 ceph-mon[103124]: pgmap v21100: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 803 KiB/s wr, 20 op/s 2023-09-10T10:13:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:13:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:13:58 smithi170 ceph-mon[102832]: pgmap v21101: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 574 KiB/s wr, 13 op/s 2023-09-10T10:13:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:13:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:13:58 smithi082 ceph-mon[98694]: pgmap v21101: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 574 KiB/s wr, 13 op/s 2023-09-10T10:13:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:13:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:13:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:13:58 smithi098 ceph-mon[103124]: pgmap v21101: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 574 KiB/s wr, 13 op/s 2023-09-10T10:14:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39270.904404 secs 2023-09-10T10:14:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39271.256242 secs 2023-09-10T10:14:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:14:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:00 smithi098 ceph-mon[103124]: pgmap v21102: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 574 KiB/s wr, 13 op/s 2023-09-10T10:14:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39270.904404 secs 2023-09-10T10:14:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39271.256242 secs 2023-09-10T10:14:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:14:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:00 smithi170 ceph-mon[102832]: pgmap v21102: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 574 KiB/s wr, 13 op/s 2023-09-10T10:14:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39270.904404 secs 2023-09-10T10:14:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39271.256242 secs 2023-09-10T10:14:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:14:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:00 smithi082 ceph-mon[98694]: pgmap v21102: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 574 KiB/s wr, 13 op/s 2023-09-10T10:14:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:02 smithi170 ceph-mon[102832]: pgmap v21103: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 892 KiB/s wr, 20 op/s 2023-09-10T10:14:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:02 smithi082 ceph-mon[98694]: pgmap v21103: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 892 KiB/s wr, 20 op/s 2023-09-10T10:14:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:02 smithi098 ceph-mon[103124]: pgmap v21103: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 892 KiB/s wr, 20 op/s 2023-09-10T10:14:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:03 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:14:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:03 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:14:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:03 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:03 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:14:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:03 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:14:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:03 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:03 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:14:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:03 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:14:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:03 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39274.674279 secs 2023-09-10T10:14:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:04 smithi170 ceph-mon[102832]: pgmap v21104: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 623 KiB/s wr, 13 op/s 2023-09-10T10:14:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39274.674279 secs 2023-09-10T10:14:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:04 smithi082 ceph-mon[98694]: pgmap v21104: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 623 KiB/s wr, 13 op/s 2023-09-10T10:14:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39274.674279 secs 2023-09-10T10:14:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:04 smithi098 ceph-mon[103124]: pgmap v21104: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 623 KiB/s wr, 13 op/s 2023-09-10T10:14:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39275.904464 secs 2023-09-10T10:14:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39276.256308 secs 2023-09-10T10:14:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39275.904464 secs 2023-09-10T10:14:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39276.256308 secs 2023-09-10T10:14:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39275.904464 secs 2023-09-10T10:14:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39276.256308 secs 2023-09-10T10:14:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:06 smithi170 ceph-mon[102832]: pgmap v21105: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 832 KiB/s wr, 34 op/s 2023-09-10T10:14:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:06 smithi082 ceph-mon[98694]: pgmap v21105: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 832 KiB/s wr, 34 op/s 2023-09-10T10:14:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:06 smithi098 ceph-mon[103124]: pgmap v21105: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 832 KiB/s wr, 34 op/s 2023-09-10T10:14:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39279.674361 secs 2023-09-10T10:14:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:09 smithi170 ceph-mon[102832]: pgmap v21106: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 527 KiB/s wr, 28 op/s 2023-09-10T10:14:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:09 smithi170 ceph-mon[102832]: osdmap e1387: 12 total, 12 up, 12 in 2023-09-10T10:14:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39279.674361 secs 2023-09-10T10:14:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:09 smithi082 ceph-mon[98694]: pgmap v21106: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 527 KiB/s wr, 28 op/s 2023-09-10T10:14:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:09 smithi082 ceph-mon[98694]: osdmap e1387: 12 total, 12 up, 12 in 2023-09-10T10:14:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39279.674361 secs 2023-09-10T10:14:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:09 smithi098 ceph-mon[103124]: pgmap v21106: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 527 KiB/s wr, 28 op/s 2023-09-10T10:14:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:09 smithi098 ceph-mon[103124]: osdmap e1387: 12 total, 12 up, 12 in 2023-09-10T10:14:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39280.904515 secs 2023-09-10T10:14:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39281.256374 secs 2023-09-10T10:14:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:14:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:10 smithi170 ceph-mon[102832]: pgmap v21108: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 633 KiB/s wr, 33 op/s 2023-09-10T10:14:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39280.904515 secs 2023-09-10T10:14:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39281.256374 secs 2023-09-10T10:14:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:14:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:10.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:10 smithi082 ceph-mon[98694]: pgmap v21108: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 633 KiB/s wr, 33 op/s 2023-09-10T10:14:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39280.904515 secs 2023-09-10T10:14:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39281.256374 secs 2023-09-10T10:14:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:14:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:10 smithi098 ceph-mon[103124]: pgmap v21108: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 633 KiB/s wr, 33 op/s 2023-09-10T10:14:13.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:14:13.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:13.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:12 smithi098 ceph-mon[103124]: pgmap v21109: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 529 KiB/s wr, 33 op/s 2023-09-10T10:14:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:14:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:12 smithi170 ceph-mon[102832]: pgmap v21109: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 529 KiB/s wr, 33 op/s 2023-09-10T10:14:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:14:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:12 smithi082 ceph-mon[98694]: pgmap v21109: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 529 KiB/s wr, 33 op/s 2023-09-10T10:14:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39284.674421 secs 2023-09-10T10:14:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39284.674421 secs 2023-09-10T10:14:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39284.674421 secs 2023-09-10T10:14:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39285.904575 secs 2023-09-10T10:14:15.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39286.256438 secs 2023-09-10T10:14:15.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:14 smithi098 ceph-mon[103124]: pgmap v21110: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 529 KiB/s wr, 33 op/s 2023-09-10T10:14:15.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:14 smithi098 ceph-mon[103124]: osdmap e1388: 12 total, 12 up, 12 in 2023-09-10T10:14:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39285.904575 secs 2023-09-10T10:14:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39286.256438 secs 2023-09-10T10:14:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:14 smithi170 ceph-mon[102832]: pgmap v21110: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 529 KiB/s wr, 33 op/s 2023-09-10T10:14:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:14 smithi170 ceph-mon[102832]: osdmap e1388: 12 total, 12 up, 12 in 2023-09-10T10:14:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39285.904575 secs 2023-09-10T10:14:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39286.256438 secs 2023-09-10T10:14:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:14 smithi082 ceph-mon[98694]: pgmap v21110: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 529 KiB/s wr, 33 op/s 2023-09-10T10:14:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:14 smithi082 ceph-mon[98694]: osdmap e1388: 12 total, 12 up, 12 in 2023-09-10T10:14:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:14:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:17.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:16 smithi170 ceph-mon[102832]: pgmap v21112: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 566 KiB/s wr, 19 op/s 2023-09-10T10:14:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:14:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:16 smithi082 ceph-mon[98694]: pgmap v21112: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 566 KiB/s wr, 19 op/s 2023-09-10T10:14:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:14:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:16 smithi098 ceph-mon[103124]: pgmap v21112: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 566 KiB/s wr, 19 op/s 2023-09-10T10:14:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:14:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:19.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39289.674519 secs 2023-09-10T10:14:19.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:18 smithi170 ceph-mon[102832]: pgmap v21113: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 454 KiB/s wr, 16 op/s 2023-09-10T10:14:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:14:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39289.674519 secs 2023-09-10T10:14:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:18 smithi082 ceph-mon[98694]: pgmap v21113: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 454 KiB/s wr, 16 op/s 2023-09-10T10:14:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:14:19.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:19.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39289.674519 secs 2023-09-10T10:14:19.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:18 smithi098 ceph-mon[103124]: pgmap v21113: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 454 KiB/s wr, 16 op/s 2023-09-10T10:14:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39290.904633 secs 2023-09-10T10:14:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39291.256491 secs 2023-09-10T10:14:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39290.904633 secs 2023-09-10T10:14:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39291.256491 secs 2023-09-10T10:14:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39290.904633 secs 2023-09-10T10:14:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39291.256491 secs 2023-09-10T10:14:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:20 smithi170 ceph-mon[102832]: pgmap v21114: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 453 KiB/s wr, 15 op/s 2023-09-10T10:14:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:20 smithi082 ceph-mon[98694]: pgmap v21114: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 453 KiB/s wr, 15 op/s 2023-09-10T10:14:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:20 smithi098 ceph-mon[103124]: pgmap v21114: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 453 KiB/s wr, 15 op/s 2023-09-10T10:14:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:22 smithi170 ceph-mon[102832]: pgmap v21115: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 175 KiB/s wr, 14 op/s 2023-09-10T10:14:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:22 smithi082 ceph-mon[98694]: pgmap v21115: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 175 KiB/s wr, 14 op/s 2023-09-10T10:14:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:22 smithi098 ceph-mon[103124]: pgmap v21115: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 175 KiB/s wr, 14 op/s 2023-09-10T10:14:23.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39294.674638 secs 2023-09-10T10:14:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39294.674638 secs 2023-09-10T10:14:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39294.674638 secs 2023-09-10T10:14:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:24 smithi170 ceph-mon[102832]: pgmap v21116: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 175 KiB/s wr, 14 op/s 2023-09-10T10:14:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:24 smithi082 ceph-mon[98694]: pgmap v21116: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 175 KiB/s wr, 14 op/s 2023-09-10T10:14:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:24 smithi098 ceph-mon[103124]: pgmap v21116: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 175 KiB/s wr, 14 op/s 2023-09-10T10:14:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39295.904692 secs 2023-09-10T10:14:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39296.256555 secs 2023-09-10T10:14:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39295.904692 secs 2023-09-10T10:14:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39296.256555 secs 2023-09-10T10:14:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39295.904692 secs 2023-09-10T10:14:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39296.256555 secs 2023-09-10T10:14:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:26 smithi170 ceph-mon[102832]: pgmap v21117: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 407 KiB/s wr, 19 op/s 2023-09-10T10:14:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:26 smithi082 ceph-mon[98694]: pgmap v21117: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 407 KiB/s wr, 19 op/s 2023-09-10T10:14:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:26 smithi098 ceph-mon[103124]: pgmap v21117: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 407 KiB/s wr, 19 op/s 2023-09-10T10:14:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:28 smithi170 ceph-mon[102832]: pgmap v21118: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 238 KiB/s wr, 11 op/s 2023-09-10T10:14:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:28 smithi082 ceph-mon[98694]: pgmap v21118: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 238 KiB/s wr, 11 op/s 2023-09-10T10:14:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:28 smithi098 ceph-mon[103124]: pgmap v21118: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 238 KiB/s wr, 11 op/s 2023-09-10T10:14:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39299.674699 secs 2023-09-10T10:14:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39299.674699 secs 2023-09-10T10:14:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39299.674699 secs 2023-09-10T10:14:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39300.904751 secs 2023-09-10T10:14:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39301.256616 secs 2023-09-10T10:14:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:14:30.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:14:30.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:14:30.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:14:30.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:30 smithi170 ceph-mon[102832]: pgmap v21119: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 238 KiB/s wr, 11 op/s 2023-09-10T10:14:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39300.904751 secs 2023-09-10T10:14:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39301.256616 secs 2023-09-10T10:14:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:14:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:14:30.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:14:30.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:14:30.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:30 smithi082 ceph-mon[98694]: pgmap v21119: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 238 KiB/s wr, 11 op/s 2023-09-10T10:14:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39300.904751 secs 2023-09-10T10:14:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39301.256616 secs 2023-09-10T10:14:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:14:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:14:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:14:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:14:30.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:30 smithi098 ceph-mon[103124]: pgmap v21119: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 238 KiB/s wr, 11 op/s 2023-09-10T10:14:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:32 smithi170 ceph-mon[102832]: pgmap v21120: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 238 KiB/s wr, 17 op/s 2023-09-10T10:14:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:32 smithi082 ceph-mon[98694]: pgmap v21120: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 238 KiB/s wr, 17 op/s 2023-09-10T10:14:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:32 smithi098 ceph-mon[103124]: pgmap v21120: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 238 KiB/s wr, 17 op/s 2023-09-10T10:14:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39304.674813 secs 2023-09-10T10:14:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:14:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:34 smithi170 ceph-mon[102832]: pgmap v21121: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 237 KiB/s wr, 11 op/s 2023-09-10T10:14:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39304.674813 secs 2023-09-10T10:14:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:14:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:34 smithi082 ceph-mon[98694]: pgmap v21121: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 237 KiB/s wr, 11 op/s 2023-09-10T10:14:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39304.674813 secs 2023-09-10T10:14:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:14:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:34 smithi098 ceph-mon[103124]: pgmap v21121: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 237 KiB/s wr, 11 op/s 2023-09-10T10:14:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39305.904811 secs 2023-09-10T10:14:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39306.256678 secs 2023-09-10T10:14:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39305.904811 secs 2023-09-10T10:14:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39306.256678 secs 2023-09-10T10:14:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39305.904811 secs 2023-09-10T10:14:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39306.256678 secs 2023-09-10T10:14:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:36 smithi170 ceph-mon[102832]: pgmap v21122: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 380 KiB/s wr, 17 op/s 2023-09-10T10:14:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:36 smithi082 ceph-mon[98694]: pgmap v21122: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 380 KiB/s wr, 17 op/s 2023-09-10T10:14:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:36 smithi098 ceph-mon[103124]: pgmap v21122: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 380 KiB/s wr, 17 op/s 2023-09-10T10:14:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:14:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:14:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:14:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:38 smithi170 ceph-mon[102832]: pgmap v21123: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 11 op/s 2023-09-10T10:14:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:38 smithi082 ceph-mon[98694]: pgmap v21123: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 11 op/s 2023-09-10T10:14:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:38 smithi098 ceph-mon[103124]: pgmap v21123: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 11 op/s 2023-09-10T10:14:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:14:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:39 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:14:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:14:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:39 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:14:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:14:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:39 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:14:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39310.904870 secs 2023-09-10T10:14:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39311.256744 secs 2023-09-10T10:14:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:14:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:40 smithi098 ceph-mon[103124]: pgmap v21124: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 11 op/s 2023-09-10T10:14:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39310.904870 secs 2023-09-10T10:14:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39311.256744 secs 2023-09-10T10:14:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:14:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:40 smithi170 ceph-mon[102832]: pgmap v21124: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 11 op/s 2023-09-10T10:14:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39310.904870 secs 2023-09-10T10:14:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39311.256744 secs 2023-09-10T10:14:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:14:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:40 smithi082 ceph-mon[98694]: pgmap v21124: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 11 op/s 2023-09-10T10:14:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:42 smithi170 ceph-mon[102832]: pgmap v21125: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 469 KiB/s wr, 17 op/s 2023-09-10T10:14:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:42 smithi082 ceph-mon[98694]: pgmap v21125: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 469 KiB/s wr, 17 op/s 2023-09-10T10:14:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:42 smithi098 ceph-mon[103124]: pgmap v21125: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 469 KiB/s wr, 17 op/s 2023-09-10T10:14:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:44 smithi170 ceph-mon[102832]: pgmap v21126: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 469 KiB/s wr, 12 op/s 2023-09-10T10:14:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:44 smithi082 ceph-mon[98694]: pgmap v21126: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 469 KiB/s wr, 12 op/s 2023-09-10T10:14:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:44 smithi098 ceph-mon[103124]: pgmap v21126: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 469 KiB/s wr, 12 op/s 2023-09-10T10:14:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39315.904948 secs 2023-09-10T10:14:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39316.256807 secs 2023-09-10T10:14:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39315.904948 secs 2023-09-10T10:14:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39316.256807 secs 2023-09-10T10:14:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39315.904948 secs 2023-09-10T10:14:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39316.256807 secs 2023-09-10T10:14:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:14:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:47.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:46 smithi170 ceph-mon[102832]: pgmap v21127: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 782 KiB/s wr, 19 op/s 2023-09-10T10:14:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:14:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:46 smithi082 ceph-mon[98694]: pgmap v21127: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 782 KiB/s wr, 19 op/s 2023-09-10T10:14:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:14:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:47.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:46 smithi098 ceph-mon[103124]: pgmap v21127: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 782 KiB/s wr, 19 op/s 2023-09-10T10:14:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:48 smithi170 ceph-mon[102832]: pgmap v21128: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 639 KiB/s wr, 13 op/s 2023-09-10T10:14:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:48 smithi082 ceph-mon[98694]: pgmap v21128: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 639 KiB/s wr, 13 op/s 2023-09-10T10:14:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:48 smithi098 ceph-mon[103124]: pgmap v21128: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 639 KiB/s wr, 13 op/s 2023-09-10T10:14:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39320.905027 secs 2023-09-10T10:14:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39321.256873 secs 2023-09-10T10:14:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:14:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:50 smithi170 ceph-mon[102832]: pgmap v21129: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 639 KiB/s wr, 13 op/s 2023-09-10T10:14:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39320.905027 secs 2023-09-10T10:14:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39321.256873 secs 2023-09-10T10:14:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:14:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:50.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:50 smithi082 ceph-mon[98694]: pgmap v21129: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 639 KiB/s wr, 13 op/s 2023-09-10T10:14:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39320.905027 secs 2023-09-10T10:14:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39321.256873 secs 2023-09-10T10:14:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:14:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:50 smithi098 ceph-mon[103124]: pgmap v21129: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 639 KiB/s wr, 13 op/s 2023-09-10T10:14:53.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:14:53.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:53.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:52 smithi170 ceph-mon[102832]: pgmap v21130: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 959 KiB/s wr, 20 op/s 2023-09-10T10:14:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:14:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:52 smithi082 ceph-mon[98694]: pgmap v21130: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 959 KiB/s wr, 20 op/s 2023-09-10T10:14:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:14:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:52 smithi098 ceph-mon[103124]: pgmap v21130: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 959 KiB/s wr, 20 op/s 2023-09-10T10:14:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:54 smithi170 ceph-mon[102832]: pgmap v21131: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 633 KiB/s wr, 14 op/s 2023-09-10T10:14:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:54 smithi082 ceph-mon[98694]: pgmap v21131: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 633 KiB/s wr, 14 op/s 2023-09-10T10:14:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:54 smithi098 ceph-mon[103124]: pgmap v21131: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 633 KiB/s wr, 14 op/s 2023-09-10T10:14:55.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39325.905089 secs 2023-09-10T10:14:55.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39326.256938 secs 2023-09-10T10:14:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39325.905089 secs 2023-09-10T10:14:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39326.256938 secs 2023-09-10T10:14:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39325.905089 secs 2023-09-10T10:14:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39326.256938 secs 2023-09-10T10:14:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:56 smithi170 ceph-mon[102832]: pgmap v21132: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 888 KiB/s wr, 20 op/s 2023-09-10T10:14:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:56 smithi082 ceph-mon[98694]: pgmap v21132: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 888 KiB/s wr, 20 op/s 2023-09-10T10:14:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:56 smithi098 ceph-mon[103124]: pgmap v21132: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 888 KiB/s wr, 20 op/s 2023-09-10T10:14:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:14:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:14:58 smithi170 ceph-mon[102832]: pgmap v21133: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 574 KiB/s wr, 13 op/s 2023-09-10T10:14:58.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:14:58.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:58.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:14:58 smithi082 ceph-mon[98694]: pgmap v21133: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 574 KiB/s wr, 13 op/s 2023-09-10T10:14:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:14:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:14:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:14:58 smithi098 ceph-mon[103124]: pgmap v21133: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 574 KiB/s wr, 13 op/s 2023-09-10T10:15:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39330.905170 secs 2023-09-10T10:15:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39331.257001 secs 2023-09-10T10:15:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:15:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:00 smithi098 ceph-mon[103124]: pgmap v21134: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 574 KiB/s wr, 13 op/s 2023-09-10T10:15:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39330.905170 secs 2023-09-10T10:15:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39331.257001 secs 2023-09-10T10:15:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:15:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:00 smithi170 ceph-mon[102832]: pgmap v21134: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 574 KiB/s wr, 13 op/s 2023-09-10T10:15:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39330.905170 secs 2023-09-10T10:15:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39331.257001 secs 2023-09-10T10:15:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:15:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:00.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:00 smithi082 ceph-mon[98694]: pgmap v21134: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 574 KiB/s wr, 13 op/s 2023-09-10T10:15:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:02 smithi170 ceph-mon[102832]: pgmap v21135: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 845 KiB/s wr, 20 op/s 2023-09-10T10:15:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:02 smithi082 ceph-mon[98694]: pgmap v21135: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 845 KiB/s wr, 20 op/s 2023-09-10T10:15:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:02 smithi098 ceph-mon[103124]: pgmap v21135: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 845 KiB/s wr, 20 op/s 2023-09-10T10:15:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:03 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:15:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:03 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:15:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:03 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:03 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:15:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:03 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:15:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:03 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:03 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:15:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:03 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:15:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:03 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39334.675172 secs 2023-09-10T10:15:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:04 smithi170 ceph-mon[102832]: pgmap v21136: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 525 KiB/s wr, 13 op/s 2023-09-10T10:15:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39334.675172 secs 2023-09-10T10:15:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:04 smithi082 ceph-mon[98694]: pgmap v21136: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 525 KiB/s wr, 13 op/s 2023-09-10T10:15:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39334.675172 secs 2023-09-10T10:15:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:04 smithi098 ceph-mon[103124]: pgmap v21136: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 525 KiB/s wr, 13 op/s 2023-09-10T10:15:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39335.905226 secs 2023-09-10T10:15:05.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39336.257068 secs 2023-09-10T10:15:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39335.905226 secs 2023-09-10T10:15:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39336.257068 secs 2023-09-10T10:15:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39335.905226 secs 2023-09-10T10:15:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39336.257068 secs 2023-09-10T10:15:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:06 smithi170 ceph-mon[102832]: pgmap v21137: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 710 KiB/s wr, 34 op/s 2023-09-10T10:15:06.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:06 smithi082 ceph-mon[98694]: pgmap v21137: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 710 KiB/s wr, 34 op/s 2023-09-10T10:15:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:06 smithi098 ceph-mon[103124]: pgmap v21137: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 710 KiB/s wr, 34 op/s 2023-09-10T10:15:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39339.675229 secs 2023-09-10T10:15:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:09 smithi170 ceph-mon[102832]: pgmap v21138: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 455 KiB/s wr, 27 op/s 2023-09-10T10:15:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:09 smithi170 ceph-mon[102832]: osdmap e1389: 12 total, 12 up, 12 in 2023-09-10T10:15:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39339.675229 secs 2023-09-10T10:15:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:09 smithi082 ceph-mon[98694]: pgmap v21138: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 455 KiB/s wr, 27 op/s 2023-09-10T10:15:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:09 smithi082 ceph-mon[98694]: osdmap e1389: 12 total, 12 up, 12 in 2023-09-10T10:15:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39339.675229 secs 2023-09-10T10:15:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:09 smithi098 ceph-mon[103124]: pgmap v21138: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 455 KiB/s wr, 27 op/s 2023-09-10T10:15:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:09 smithi098 ceph-mon[103124]: osdmap e1389: 12 total, 12 up, 12 in 2023-09-10T10:15:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39340.905283 secs 2023-09-10T10:15:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39341.257137 secs 2023-09-10T10:15:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:15:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39340.905283 secs 2023-09-10T10:15:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39341.257137 secs 2023-09-10T10:15:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:15:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39340.905283 secs 2023-09-10T10:15:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39341.257137 secs 2023-09-10T10:15:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:15:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:11 smithi170 ceph-mon[102832]: pgmap v21140: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 546 KiB/s wr, 33 op/s 2023-09-10T10:15:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:11 smithi082 ceph-mon[98694]: pgmap v21140: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 546 KiB/s wr, 33 op/s 2023-09-10T10:15:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:11 smithi098 ceph-mon[103124]: pgmap v21140: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 546 KiB/s wr, 33 op/s 2023-09-10T10:15:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:13 smithi170 ceph-mon[102832]: pgmap v21141: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 469 KiB/s wr, 33 op/s 2023-09-10T10:15:13.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:15:13.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:13 smithi082 ceph-mon[98694]: pgmap v21141: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 469 KiB/s wr, 33 op/s 2023-09-10T10:15:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:15:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:13 smithi098 ceph-mon[103124]: pgmap v21141: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 469 KiB/s wr, 33 op/s 2023-09-10T10:15:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:15:13.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39344.675343 secs 2023-09-10T10:15:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39344.675343 secs 2023-09-10T10:15:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39344.675343 secs 2023-09-10T10:15:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39345.905340 secs 2023-09-10T10:15:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39346.257199 secs 2023-09-10T10:15:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:15 smithi170 ceph-mon[102832]: pgmap v21142: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 469 KiB/s wr, 33 op/s 2023-09-10T10:15:15.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:15 smithi170 ceph-mon[102832]: osdmap e1390: 12 total, 12 up, 12 in 2023-09-10T10:15:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39345.905340 secs 2023-09-10T10:15:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39346.257199 secs 2023-09-10T10:15:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:15 smithi082 ceph-mon[98694]: pgmap v21142: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 469 KiB/s wr, 33 op/s 2023-09-10T10:15:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:15 smithi082 ceph-mon[98694]: osdmap e1390: 12 total, 12 up, 12 in 2023-09-10T10:15:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39345.905340 secs 2023-09-10T10:15:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39346.257199 secs 2023-09-10T10:15:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:15 smithi098 ceph-mon[103124]: pgmap v21142: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 469 KiB/s wr, 33 op/s 2023-09-10T10:15:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:15 smithi098 ceph-mon[103124]: osdmap e1390: 12 total, 12 up, 12 in 2023-09-10T10:15:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:17 smithi170 ceph-mon[102832]: pgmap v21144: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 527 KiB/s wr, 20 op/s 2023-09-10T10:15:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:17 smithi082 ceph-mon[98694]: pgmap v21144: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 527 KiB/s wr, 20 op/s 2023-09-10T10:15:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:17 smithi098 ceph-mon[103124]: pgmap v21144: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 527 KiB/s wr, 20 op/s 2023-09-10T10:15:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39349.675426 secs 2023-09-10T10:15:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:19 smithi170 ceph-mon[102832]: pgmap v21145: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 422 KiB/s wr, 16 op/s 2023-09-10T10:15:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39349.675426 secs 2023-09-10T10:15:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:19 smithi082 ceph-mon[98694]: pgmap v21145: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 422 KiB/s wr, 16 op/s 2023-09-10T10:15:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39349.675426 secs 2023-09-10T10:15:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:19 smithi098 ceph-mon[103124]: pgmap v21145: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 422 KiB/s wr, 16 op/s 2023-09-10T10:15:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39350.905397 secs 2023-09-10T10:15:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39351.257266 secs 2023-09-10T10:15:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39350.905397 secs 2023-09-10T10:15:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39351.257266 secs 2023-09-10T10:15:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39350.905397 secs 2023-09-10T10:15:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39351.257266 secs 2023-09-10T10:15:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:21 smithi170 ceph-mon[102832]: pgmap v21146: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 422 KiB/s wr, 16 op/s 2023-09-10T10:15:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:21 smithi082 ceph-mon[98694]: pgmap v21146: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 422 KiB/s wr, 16 op/s 2023-09-10T10:15:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:21 smithi098 ceph-mon[103124]: pgmap v21146: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 422 KiB/s wr, 16 op/s 2023-09-10T10:15:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:23 smithi170 ceph-mon[102832]: pgmap v21147: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 175 KiB/s wr, 13 op/s 2023-09-10T10:15:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:23 smithi082 ceph-mon[98694]: pgmap v21147: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 175 KiB/s wr, 13 op/s 2023-09-10T10:15:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:23 smithi098 ceph-mon[103124]: pgmap v21147: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 175 KiB/s wr, 13 op/s 2023-09-10T10:15:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39354.675483 secs 2023-09-10T10:15:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39354.675483 secs 2023-09-10T10:15:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39354.675483 secs 2023-09-10T10:15:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39355.905504 secs 2023-09-10T10:15:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39356.257330 secs 2023-09-10T10:15:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:25 smithi170 ceph-mon[102832]: pgmap v21148: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 175 KiB/s wr, 13 op/s 2023-09-10T10:15:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39355.905504 secs 2023-09-10T10:15:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39356.257330 secs 2023-09-10T10:15:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:25 smithi082 ceph-mon[98694]: pgmap v21148: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 175 KiB/s wr, 13 op/s 2023-09-10T10:15:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39355.905504 secs 2023-09-10T10:15:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39356.257330 secs 2023-09-10T10:15:25.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:25 smithi098 ceph-mon[103124]: pgmap v21148: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 175 KiB/s wr, 13 op/s 2023-09-10T10:15:27.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:27 smithi170 ceph-mon[102832]: pgmap v21149: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 311 KiB/s wr, 18 op/s 2023-09-10T10:15:27.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:27 smithi082 ceph-mon[98694]: pgmap v21149: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 311 KiB/s wr, 18 op/s 2023-09-10T10:15:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:27 smithi098 ceph-mon[103124]: pgmap v21149: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 311 KiB/s wr, 18 op/s 2023-09-10T10:15:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:28 smithi170 ceph-mon[102832]: pgmap v21150: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 10 op/s 2023-09-10T10:15:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:28 smithi082 ceph-mon[98694]: pgmap v21150: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 10 op/s 2023-09-10T10:15:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:28 smithi098 ceph-mon[103124]: pgmap v21150: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 10 op/s 2023-09-10T10:15:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39359.675540 secs 2023-09-10T10:15:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39359.675540 secs 2023-09-10T10:15:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39359.675540 secs 2023-09-10T10:15:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39360.905562 secs 2023-09-10T10:15:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39361.257393 secs 2023-09-10T10:15:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:30 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:15:30.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:30 smithi170 ceph-mon[102832]: pgmap v21151: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 10 op/s 2023-09-10T10:15:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39360.905562 secs 2023-09-10T10:15:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39361.257393 secs 2023-09-10T10:15:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:30 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:15:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:30 smithi082 ceph-mon[98694]: pgmap v21151: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 10 op/s 2023-09-10T10:15:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39360.905562 secs 2023-09-10T10:15:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39361.257393 secs 2023-09-10T10:15:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:30 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:15:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:30 smithi098 ceph-mon[103124]: pgmap v21151: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 10 op/s 2023-09-10T10:15:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:15:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:15:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:15:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:15:31.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:15:31.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:15:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:15:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:15:31.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:15:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:32 smithi170 ceph-mon[102832]: pgmap v21152: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T10:15:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:32 smithi082 ceph-mon[98694]: pgmap v21152: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T10:15:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:32 smithi098 ceph-mon[103124]: pgmap v21152: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T10:15:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39364.675639 secs 2023-09-10T10:15:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:15:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:34 smithi170 ceph-mon[102832]: pgmap v21153: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:15:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39364.675639 secs 2023-09-10T10:15:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:15:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:34.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:34 smithi082 ceph-mon[98694]: pgmap v21153: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:15:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39364.675639 secs 2023-09-10T10:15:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:15:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:34 smithi098 ceph-mon[103124]: pgmap v21153: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:15:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39365.905620 secs 2023-09-10T10:15:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39366.257458 secs 2023-09-10T10:15:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39365.905620 secs 2023-09-10T10:15:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39366.257458 secs 2023-09-10T10:15:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39365.905620 secs 2023-09-10T10:15:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39366.257458 secs 2023-09-10T10:15:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:36 smithi170 ceph-mon[102832]: pgmap v21154: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 290 KiB/s wr, 16 op/s 2023-09-10T10:15:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:36 smithi082 ceph-mon[98694]: pgmap v21154: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 290 KiB/s wr, 16 op/s 2023-09-10T10:15:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:36 smithi098 ceph-mon[103124]: pgmap v21154: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 290 KiB/s wr, 16 op/s 2023-09-10T10:15:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:15:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:15:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:15:37.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:38 smithi170 ceph-mon[102832]: pgmap v21155: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 10 op/s 2023-09-10T10:15:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:38 smithi082 ceph-mon[98694]: pgmap v21155: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 10 op/s 2023-09-10T10:15:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:38 smithi098 ceph-mon[103124]: pgmap v21155: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 10 op/s 2023-09-10T10:15:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:15:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:15:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:15:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39370.905677 secs 2023-09-10T10:15:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39371.257487 secs 2023-09-10T10:15:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:15:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:15:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:40.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:40 smithi098 ceph-mon[103124]: pgmap v21156: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 10 op/s 2023-09-10T10:15:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39370.905677 secs 2023-09-10T10:15:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39371.257487 secs 2023-09-10T10:15:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:15:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:15:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:40.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:40 smithi170 ceph-mon[102832]: pgmap v21156: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 10 op/s 2023-09-10T10:15:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39370.905677 secs 2023-09-10T10:15:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39371.257487 secs 2023-09-10T10:15:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:15:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:15:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:40 smithi082 ceph-mon[98694]: pgmap v21156: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 10 op/s 2023-09-10T10:15:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:42 smithi170 ceph-mon[102832]: pgmap v21157: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 469 KiB/s wr, 17 op/s 2023-09-10T10:15:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:42 smithi082 ceph-mon[98694]: pgmap v21157: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 469 KiB/s wr, 17 op/s 2023-09-10T10:15:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:42 smithi098 ceph-mon[103124]: pgmap v21157: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 469 KiB/s wr, 17 op/s 2023-09-10T10:15:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:44 smithi170 ceph-mon[102832]: pgmap v21158: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 469 KiB/s wr, 12 op/s 2023-09-10T10:15:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:44 smithi082 ceph-mon[98694]: pgmap v21158: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 469 KiB/s wr, 12 op/s 2023-09-10T10:15:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:44 smithi098 ceph-mon[103124]: pgmap v21158: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 469 KiB/s wr, 12 op/s 2023-09-10T10:15:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39375.905735 secs 2023-09-10T10:15:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39376.257554 secs 2023-09-10T10:15:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39375.905735 secs 2023-09-10T10:15:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39376.257554 secs 2023-09-10T10:15:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39375.905735 secs 2023-09-10T10:15:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39376.257554 secs 2023-09-10T10:15:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:15:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:47.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:46 smithi170 ceph-mon[102832]: pgmap v21159: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 783 KiB/s wr, 19 op/s 2023-09-10T10:15:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:15:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:46 smithi082 ceph-mon[98694]: pgmap v21159: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 783 KiB/s wr, 19 op/s 2023-09-10T10:15:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:15:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:47.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:46 smithi098 ceph-mon[103124]: pgmap v21159: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 783 KiB/s wr, 19 op/s 2023-09-10T10:15:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:48 smithi170 ceph-mon[102832]: pgmap v21160: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 640 KiB/s wr, 13 op/s 2023-09-10T10:15:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:48 smithi082 ceph-mon[98694]: pgmap v21160: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 640 KiB/s wr, 13 op/s 2023-09-10T10:15:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:48 smithi098 ceph-mon[103124]: pgmap v21160: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 640 KiB/s wr, 13 op/s 2023-09-10T10:15:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39380.905798 secs 2023-09-10T10:15:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39381.257619 secs 2023-09-10T10:15:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:15:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:50 smithi170 ceph-mon[102832]: pgmap v21161: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 640 KiB/s wr, 13 op/s 2023-09-10T10:15:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39380.905798 secs 2023-09-10T10:15:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39381.257619 secs 2023-09-10T10:15:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:15:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:50.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:50 smithi082 ceph-mon[98694]: pgmap v21161: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 640 KiB/s wr, 13 op/s 2023-09-10T10:15:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39380.905798 secs 2023-09-10T10:15:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39381.257619 secs 2023-09-10T10:15:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:15:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:50 smithi098 ceph-mon[103124]: pgmap v21161: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 640 KiB/s wr, 13 op/s 2023-09-10T10:15:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:15:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:53.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:52 smithi170 ceph-mon[102832]: pgmap v21162: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 952 KiB/s wr, 20 op/s 2023-09-10T10:15:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:15:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:52 smithi082 ceph-mon[98694]: pgmap v21162: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 952 KiB/s wr, 20 op/s 2023-09-10T10:15:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:15:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:52 smithi098 ceph-mon[103124]: pgmap v21162: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 952 KiB/s wr, 20 op/s 2023-09-10T10:15:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:54 smithi170 ceph-mon[102832]: pgmap v21163: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 625 KiB/s wr, 13 op/s 2023-09-10T10:15:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:54 smithi082 ceph-mon[98694]: pgmap v21163: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 625 KiB/s wr, 13 op/s 2023-09-10T10:15:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:54 smithi098 ceph-mon[103124]: pgmap v21163: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 625 KiB/s wr, 13 op/s 2023-09-10T10:15:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39385.905855 secs 2023-09-10T10:15:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39386.257687 secs 2023-09-10T10:15:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39385.905855 secs 2023-09-10T10:15:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39386.257687 secs 2023-09-10T10:15:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39385.905855 secs 2023-09-10T10:15:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39386.257687 secs 2023-09-10T10:15:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:56 smithi170 ceph-mon[102832]: pgmap v21164: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 890 KiB/s wr, 19 op/s 2023-09-10T10:15:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:56 smithi082 ceph-mon[98694]: pgmap v21164: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 890 KiB/s wr, 19 op/s 2023-09-10T10:15:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:56 smithi098 ceph-mon[103124]: pgmap v21164: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 890 KiB/s wr, 19 op/s 2023-09-10T10:15:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:15:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:15:58 smithi170 ceph-mon[102832]: pgmap v21165: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 576 KiB/s wr, 12 op/s 2023-09-10T10:15:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:15:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:15:58 smithi082 ceph-mon[98694]: pgmap v21165: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 576 KiB/s wr, 12 op/s 2023-09-10T10:15:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:15:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:15:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:15:58 smithi098 ceph-mon[103124]: pgmap v21165: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 576 KiB/s wr, 12 op/s 2023-09-10T10:16:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39390.905913 secs 2023-09-10T10:16:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39391.257749 secs 2023-09-10T10:16:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:16:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:00 smithi098 ceph-mon[103124]: pgmap v21166: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 576 KiB/s wr, 12 op/s 2023-09-10T10:16:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39390.905913 secs 2023-09-10T10:16:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39391.257749 secs 2023-09-10T10:16:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:16:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:00 smithi170 ceph-mon[102832]: pgmap v21166: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 576 KiB/s wr, 12 op/s 2023-09-10T10:16:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39390.905913 secs 2023-09-10T10:16:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39391.257749 secs 2023-09-10T10:16:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:16:00.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:00.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:00 smithi082 ceph-mon[98694]: pgmap v21166: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 576 KiB/s wr, 12 op/s 2023-09-10T10:16:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:02 smithi170 ceph-mon[102832]: pgmap v21167: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 847 KiB/s wr, 20 op/s 2023-09-10T10:16:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:02 smithi082 ceph-mon[98694]: pgmap v21167: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 847 KiB/s wr, 20 op/s 2023-09-10T10:16:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:02 smithi098 ceph-mon[103124]: pgmap v21167: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 847 KiB/s wr, 20 op/s 2023-09-10T10:16:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:03 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:16:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:03 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:16:03.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:03 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:03 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:16:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:03 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:16:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:03 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:03 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:16:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:03 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:16:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:03 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39394.675979 secs 2023-09-10T10:16:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:04 smithi170 ceph-mon[102832]: pgmap v21168: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 536 KiB/s wr, 13 op/s 2023-09-10T10:16:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39394.675979 secs 2023-09-10T10:16:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:04 smithi082 ceph-mon[98694]: pgmap v21168: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 536 KiB/s wr, 13 op/s 2023-09-10T10:16:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39394.675979 secs 2023-09-10T10:16:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:04 smithi098 ceph-mon[103124]: pgmap v21168: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 536 KiB/s wr, 13 op/s 2023-09-10T10:16:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39395.905973 secs 2023-09-10T10:16:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39396.257817 secs 2023-09-10T10:16:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39395.905973 secs 2023-09-10T10:16:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39396.257817 secs 2023-09-10T10:16:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39395.905973 secs 2023-09-10T10:16:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39396.257817 secs 2023-09-10T10:16:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:06 smithi170 ceph-mon[102832]: pgmap v21169: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 860 KiB/s wr, 33 op/s 2023-09-10T10:16:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:06 smithi082 ceph-mon[98694]: pgmap v21169: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 860 KiB/s wr, 33 op/s 2023-09-10T10:16:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:06 smithi098 ceph-mon[103124]: pgmap v21169: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 860 KiB/s wr, 33 op/s 2023-09-10T10:16:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39399.676035 secs 2023-09-10T10:16:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:09 smithi170 ceph-mon[102832]: osdmap e1391: 12 total, 12 up, 12 in 2023-09-10T10:16:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:09 smithi170 ceph-mon[102832]: pgmap v21171: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 715 KiB/s wr, 33 op/s 2023-09-10T10:16:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39399.676035 secs 2023-09-10T10:16:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:09 smithi082 ceph-mon[98694]: osdmap e1391: 12 total, 12 up, 12 in 2023-09-10T10:16:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:09 smithi082 ceph-mon[98694]: pgmap v21171: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 715 KiB/s wr, 33 op/s 2023-09-10T10:16:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39399.676035 secs 2023-09-10T10:16:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:09 smithi098 ceph-mon[103124]: osdmap e1391: 12 total, 12 up, 12 in 2023-09-10T10:16:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:09 smithi098 ceph-mon[103124]: pgmap v21171: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 715 KiB/s wr, 33 op/s 2023-09-10T10:16:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39400.906044 secs 2023-09-10T10:16:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39401.257880 secs 2023-09-10T10:16:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:16:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39400.906044 secs 2023-09-10T10:16:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39401.257880 secs 2023-09-10T10:16:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:16:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39400.906044 secs 2023-09-10T10:16:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39401.257880 secs 2023-09-10T10:16:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:16:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:11 smithi170 ceph-mon[102832]: pgmap v21172: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 715 KiB/s wr, 33 op/s 2023-09-10T10:16:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:11 smithi082 ceph-mon[98694]: pgmap v21172: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 715 KiB/s wr, 33 op/s 2023-09-10T10:16:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:11 smithi098 ceph-mon[103124]: pgmap v21172: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 715 KiB/s wr, 33 op/s 2023-09-10T10:16:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:16:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:13 smithi170 ceph-mon[102832]: pgmap v21173: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 652 KiB/s wr, 32 op/s 2023-09-10T10:16:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:16:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:13 smithi082 ceph-mon[98694]: pgmap v21173: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 652 KiB/s wr, 32 op/s 2023-09-10T10:16:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:16:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:13.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:13 smithi098 ceph-mon[103124]: pgmap v21173: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 652 KiB/s wr, 32 op/s 2023-09-10T10:16:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39404.676097 secs 2023-09-10T10:16:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39404.676097 secs 2023-09-10T10:16:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39404.676097 secs 2023-09-10T10:16:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39405.906101 secs 2023-09-10T10:16:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39406.257947 secs 2023-09-10T10:16:15.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:15 smithi170 ceph-mon[102832]: pgmap v21174: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 652 KiB/s wr, 32 op/s 2023-09-10T10:16:15.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:15 smithi170 ceph-mon[102832]: osdmap e1392: 12 total, 12 up, 12 in 2023-09-10T10:16:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39405.906101 secs 2023-09-10T10:16:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39406.257947 secs 2023-09-10T10:16:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:15 smithi082 ceph-mon[98694]: pgmap v21174: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 652 KiB/s wr, 32 op/s 2023-09-10T10:16:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:15 smithi082 ceph-mon[98694]: osdmap e1392: 12 total, 12 up, 12 in 2023-09-10T10:16:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39405.906101 secs 2023-09-10T10:16:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39406.257947 secs 2023-09-10T10:16:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:15 smithi098 ceph-mon[103124]: pgmap v21174: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 652 KiB/s wr, 32 op/s 2023-09-10T10:16:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:15 smithi098 ceph-mon[103124]: osdmap e1392: 12 total, 12 up, 12 in 2023-09-10T10:16:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:17 smithi170 ceph-mon[102832]: pgmap v21176: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 527 KiB/s wr, 19 op/s 2023-09-10T10:16:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:17 smithi082 ceph-mon[98694]: pgmap v21176: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 527 KiB/s wr, 19 op/s 2023-09-10T10:16:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:17 smithi098 ceph-mon[103124]: pgmap v21176: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 527 KiB/s wr, 19 op/s 2023-09-10T10:16:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39409.676153 secs 2023-09-10T10:16:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:19 smithi170 ceph-mon[102832]: pgmap v21177: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 422 KiB/s wr, 15 op/s 2023-09-10T10:16:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39409.676153 secs 2023-09-10T10:16:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:19 smithi082 ceph-mon[98694]: pgmap v21177: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 422 KiB/s wr, 15 op/s 2023-09-10T10:16:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39409.676153 secs 2023-09-10T10:16:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:19 smithi098 ceph-mon[103124]: pgmap v21177: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 422 KiB/s wr, 15 op/s 2023-09-10T10:16:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39410.906189 secs 2023-09-10T10:16:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39411.258011 secs 2023-09-10T10:16:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39410.906189 secs 2023-09-10T10:16:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39411.258011 secs 2023-09-10T10:16:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39410.906189 secs 2023-09-10T10:16:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39411.258011 secs 2023-09-10T10:16:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:21 smithi170 ceph-mon[102832]: pgmap v21178: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 422 KiB/s wr, 15 op/s 2023-09-10T10:16:21.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:21 smithi082 ceph-mon[98694]: pgmap v21178: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 422 KiB/s wr, 15 op/s 2023-09-10T10:16:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:21 smithi098 ceph-mon[103124]: pgmap v21178: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 422 KiB/s wr, 15 op/s 2023-09-10T10:16:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:23 smithi170 ceph-mon[102832]: pgmap v21179: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 160 KiB/s wr, 15 op/s 2023-09-10T10:16:23.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:23 smithi082 ceph-mon[98694]: pgmap v21179: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 160 KiB/s wr, 15 op/s 2023-09-10T10:16:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:23 smithi098 ceph-mon[103124]: pgmap v21179: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 160 KiB/s wr, 15 op/s 2023-09-10T10:16:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39414.676214 secs 2023-09-10T10:16:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39414.676214 secs 2023-09-10T10:16:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39414.676214 secs 2023-09-10T10:16:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39415.906249 secs 2023-09-10T10:16:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39416.258078 secs 2023-09-10T10:16:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:25 smithi170 ceph-mon[102832]: pgmap v21180: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 160 KiB/s wr, 15 op/s 2023-09-10T10:16:25.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:25 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:16:25.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:25 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39415.906249 secs 2023-09-10T10:16:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39416.258078 secs 2023-09-10T10:16:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:25 smithi082 ceph-mon[98694]: pgmap v21180: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 160 KiB/s wr, 15 op/s 2023-09-10T10:16:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:25 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:16:25.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:25 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39415.906249 secs 2023-09-10T10:16:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39416.258078 secs 2023-09-10T10:16:25.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:25 smithi098 ceph-mon[103124]: pgmap v21180: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 160 KiB/s wr, 15 op/s 2023-09-10T10:16:25.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:25 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:16:25.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:25 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:26 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:16:27.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:27.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:26 smithi170 ceph-mon[102832]: pgmap v21181: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 298 KiB/s wr, 19 op/s 2023-09-10T10:16:27.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:26 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:16:27.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:27.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:26 smithi082 ceph-mon[98694]: pgmap v21181: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 298 KiB/s wr, 19 op/s 2023-09-10T10:16:27.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:26 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:16:27.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:27.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:26 smithi098 ceph-mon[103124]: pgmap v21181: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 298 KiB/s wr, 19 op/s 2023-09-10T10:16:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:28 smithi170 ceph-mon[102832]: pgmap v21182: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:16:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:28 smithi082 ceph-mon[98694]: pgmap v21182: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:16:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:28 smithi098 ceph-mon[103124]: pgmap v21182: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:16:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39419.676276 secs 2023-09-10T10:16:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39419.676276 secs 2023-09-10T10:16:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39419.676276 secs 2023-09-10T10:16:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39420.906332 secs 2023-09-10T10:16:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39421.258141 secs 2023-09-10T10:16:30.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:30 smithi170 ceph-mon[102832]: pgmap v21183: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:16:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39420.906332 secs 2023-09-10T10:16:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39421.258141 secs 2023-09-10T10:16:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:30 smithi082 ceph-mon[98694]: pgmap v21183: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:16:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39420.906332 secs 2023-09-10T10:16:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39421.258141 secs 2023-09-10T10:16:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:30 smithi098 ceph-mon[103124]: pgmap v21183: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:16:31.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:16:31.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:16:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:16:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:16:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:16:32.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:16:32.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:32 smithi170 ceph-mon[102832]: pgmap v21184: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 16 op/s 2023-09-10T10:16:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:16:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:16:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:16:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:32 smithi082 ceph-mon[98694]: pgmap v21184: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 16 op/s 2023-09-10T10:16:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:16:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:16:32.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:16:32.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:32 smithi098 ceph-mon[103124]: pgmap v21184: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 16 op/s 2023-09-10T10:16:33.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39424.676341 secs 2023-09-10T10:16:33.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39424.676341 secs 2023-09-10T10:16:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39424.676341 secs 2023-09-10T10:16:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:34 smithi170 ceph-mon[102832]: pgmap v21185: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 10 op/s 2023-09-10T10:16:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:34 smithi082 ceph-mon[98694]: pgmap v21185: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 10 op/s 2023-09-10T10:16:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:34 smithi098 ceph-mon[103124]: pgmap v21185: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 10 op/s 2023-09-10T10:16:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39425.906392 secs 2023-09-10T10:16:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39426.258209 secs 2023-09-10T10:16:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39425.906392 secs 2023-09-10T10:16:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39426.258209 secs 2023-09-10T10:16:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39425.906392 secs 2023-09-10T10:16:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39426.258209 secs 2023-09-10T10:16:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:36 smithi170 ceph-mon[102832]: pgmap v21186: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 318 KiB/s wr, 15 op/s 2023-09-10T10:16:36.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:36 smithi082 ceph-mon[98694]: pgmap v21186: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 318 KiB/s wr, 15 op/s 2023-09-10T10:16:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:36 smithi098 ceph-mon[103124]: pgmap v21186: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 318 KiB/s wr, 15 op/s 2023-09-10T10:16:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:16:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:38.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:38 smithi170 ceph-mon[102832]: pgmap v21187: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 171 KiB/s wr, 10 op/s 2023-09-10T10:16:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:16:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:38 smithi082 ceph-mon[98694]: pgmap v21187: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 171 KiB/s wr, 10 op/s 2023-09-10T10:16:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:16:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:38.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:38 smithi098 ceph-mon[103124]: pgmap v21187: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 171 KiB/s wr, 10 op/s 2023-09-10T10:16:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:16:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:16:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:16:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39430.906476 secs 2023-09-10T10:16:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39431.258279 secs 2023-09-10T10:16:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:16:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:16:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:40 smithi170 ceph-mon[102832]: pgmap v21188: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 171 KiB/s wr, 10 op/s 2023-09-10T10:16:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39430.906476 secs 2023-09-10T10:16:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39431.258279 secs 2023-09-10T10:16:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:16:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:16:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:40 smithi082 ceph-mon[98694]: pgmap v21188: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 171 KiB/s wr, 10 op/s 2023-09-10T10:16:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39430.906476 secs 2023-09-10T10:16:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39431.258279 secs 2023-09-10T10:16:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:16:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:16:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:40 smithi098 ceph-mon[103124]: pgmap v21188: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 171 KiB/s wr, 10 op/s 2023-09-10T10:16:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:42 smithi170 ceph-mon[102832]: pgmap v21189: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 542 KiB/s wr, 16 op/s 2023-09-10T10:16:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:42 smithi082 ceph-mon[98694]: pgmap v21189: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 542 KiB/s wr, 16 op/s 2023-09-10T10:16:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:42 smithi098 ceph-mon[103124]: pgmap v21189: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 542 KiB/s wr, 16 op/s 2023-09-10T10:16:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:44 smithi170 ceph-mon[102832]: pgmap v21190: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 542 KiB/s wr, 12 op/s 2023-09-10T10:16:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:44 smithi082 ceph-mon[98694]: pgmap v21190: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 542 KiB/s wr, 12 op/s 2023-09-10T10:16:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:44 smithi098 ceph-mon[103124]: pgmap v21190: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 542 KiB/s wr, 12 op/s 2023-09-10T10:16:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39435.906494 secs 2023-09-10T10:16:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39436.258347 secs 2023-09-10T10:16:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39435.906494 secs 2023-09-10T10:16:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39436.258347 secs 2023-09-10T10:16:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39435.906494 secs 2023-09-10T10:16:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39436.258347 secs 2023-09-10T10:16:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:46 smithi170 ceph-mon[102832]: pgmap v21191: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 788 KiB/s wr, 18 op/s 2023-09-10T10:16:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:46 smithi082 ceph-mon[98694]: pgmap v21191: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 788 KiB/s wr, 18 op/s 2023-09-10T10:16:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:46 smithi098 ceph-mon[103124]: pgmap v21191: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 788 KiB/s wr, 18 op/s 2023-09-10T10:16:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:48 smithi170 ceph-mon[102832]: pgmap v21192: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 617 KiB/s wr, 12 op/s 2023-09-10T10:16:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:48 smithi082 ceph-mon[98694]: pgmap v21192: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 617 KiB/s wr, 12 op/s 2023-09-10T10:16:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:48 smithi098 ceph-mon[103124]: pgmap v21192: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 617 KiB/s wr, 12 op/s 2023-09-10T10:16:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39440.906554 secs 2023-09-10T10:16:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39441.258414 secs 2023-09-10T10:16:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:16:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:50 smithi170 ceph-mon[102832]: pgmap v21193: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 617 KiB/s wr, 12 op/s 2023-09-10T10:16:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39440.906554 secs 2023-09-10T10:16:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39441.258414 secs 2023-09-10T10:16:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:16:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:50 smithi082 ceph-mon[98694]: pgmap v21193: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 617 KiB/s wr, 12 op/s 2023-09-10T10:16:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39440.906554 secs 2023-09-10T10:16:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39441.258414 secs 2023-09-10T10:16:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:16:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:50 smithi098 ceph-mon[103124]: pgmap v21193: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 617 KiB/s wr, 12 op/s 2023-09-10T10:16:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:16:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:53.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:52 smithi170 ceph-mon[102832]: pgmap v21194: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 888 KiB/s wr, 20 op/s 2023-09-10T10:16:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:16:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:52 smithi082 ceph-mon[98694]: pgmap v21194: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 888 KiB/s wr, 20 op/s 2023-09-10T10:16:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:16:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:52 smithi098 ceph-mon[103124]: pgmap v21194: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 888 KiB/s wr, 20 op/s 2023-09-10T10:16:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:54 smithi170 ceph-mon[102832]: pgmap v21195: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 516 KiB/s wr, 14 op/s 2023-09-10T10:16:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:54 smithi082 ceph-mon[98694]: pgmap v21195: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 516 KiB/s wr, 14 op/s 2023-09-10T10:16:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:54 smithi098 ceph-mon[103124]: pgmap v21195: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 516 KiB/s wr, 14 op/s 2023-09-10T10:16:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39445.906614 secs 2023-09-10T10:16:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39446.258503 secs 2023-09-10T10:16:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39445.906614 secs 2023-09-10T10:16:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39446.258503 secs 2023-09-10T10:16:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39445.906614 secs 2023-09-10T10:16:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39446.258503 secs 2023-09-10T10:16:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:56 smithi170 ceph-mon[102832]: pgmap v21196: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 823 KiB/s wr, 21 op/s 2023-09-10T10:16:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:56 smithi082 ceph-mon[98694]: pgmap v21196: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 823 KiB/s wr, 21 op/s 2023-09-10T10:16:56.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:56 smithi098 ceph-mon[103124]: pgmap v21196: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 823 KiB/s wr, 21 op/s 2023-09-10T10:16:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:16:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:16:58 smithi170 ceph-mon[102832]: pgmap v21197: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 578 KiB/s wr, 14 op/s 2023-09-10T10:16:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:16:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:16:58 smithi082 ceph-mon[98694]: pgmap v21197: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 578 KiB/s wr, 14 op/s 2023-09-10T10:16:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:16:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:16:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:16:58 smithi098 ceph-mon[103124]: pgmap v21197: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 578 KiB/s wr, 14 op/s 2023-09-10T10:17:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39450.906672 secs 2023-09-10T10:17:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39451.258568 secs 2023-09-10T10:17:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:17:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:00 smithi098 ceph-mon[103124]: pgmap v21198: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 578 KiB/s wr, 14 op/s 2023-09-10T10:17:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39450.906672 secs 2023-09-10T10:17:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39451.258568 secs 2023-09-10T10:17:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:17:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:00 smithi170 ceph-mon[102832]: pgmap v21198: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 578 KiB/s wr, 14 op/s 2023-09-10T10:17:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39450.906672 secs 2023-09-10T10:17:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39451.258568 secs 2023-09-10T10:17:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:17:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:00 smithi082 ceph-mon[98694]: pgmap v21198: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 578 KiB/s wr, 14 op/s 2023-09-10T10:17:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:02 smithi170 ceph-mon[102832]: pgmap v21199: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 874 KiB/s wr, 21 op/s 2023-09-10T10:17:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:02 smithi082 ceph-mon[98694]: pgmap v21199: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 874 KiB/s wr, 21 op/s 2023-09-10T10:17:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:02 smithi098 ceph-mon[103124]: pgmap v21199: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 874 KiB/s wr, 21 op/s 2023-09-10T10:17:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:03 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:17:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:03 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:17:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:03 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:03 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:17:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:03 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:17:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:03 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:03 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:17:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:03 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:17:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:03 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39454.676736 secs 2023-09-10T10:17:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:04 smithi170 ceph-mon[102832]: pgmap v21200: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 603 KiB/s wr, 13 op/s 2023-09-10T10:17:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39454.676736 secs 2023-09-10T10:17:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:04 smithi082 ceph-mon[98694]: pgmap v21200: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 603 KiB/s wr, 13 op/s 2023-09-10T10:17:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39454.676736 secs 2023-09-10T10:17:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:04 smithi098 ceph-mon[103124]: pgmap v21200: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 603 KiB/s wr, 13 op/s 2023-09-10T10:17:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39455.906768 secs 2023-09-10T10:17:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39456.258635 secs 2023-09-10T10:17:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39455.906768 secs 2023-09-10T10:17:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39456.258635 secs 2023-09-10T10:17:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39455.906768 secs 2023-09-10T10:17:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39456.258635 secs 2023-09-10T10:17:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:06 smithi170 ceph-mon[102832]: pgmap v21201: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 788 KiB/s wr, 34 op/s 2023-09-10T10:17:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:06 smithi082 ceph-mon[98694]: pgmap v21201: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 788 KiB/s wr, 34 op/s 2023-09-10T10:17:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:06 smithi098 ceph-mon[103124]: pgmap v21201: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 788 KiB/s wr, 34 op/s 2023-09-10T10:17:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39459.676778 secs 2023-09-10T10:17:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:09 smithi170 ceph-mon[102832]: osdmap e1393: 12 total, 12 up, 12 in 2023-09-10T10:17:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:09 smithi170 ceph-mon[102832]: pgmap v21203: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 578 KiB/s wr, 33 op/s 2023-09-10T10:17:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39459.676778 secs 2023-09-10T10:17:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:09 smithi082 ceph-mon[98694]: osdmap e1393: 12 total, 12 up, 12 in 2023-09-10T10:17:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:09 smithi082 ceph-mon[98694]: pgmap v21203: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 578 KiB/s wr, 33 op/s 2023-09-10T10:17:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39459.676778 secs 2023-09-10T10:17:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:09 smithi098 ceph-mon[103124]: osdmap e1393: 12 total, 12 up, 12 in 2023-09-10T10:17:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:09 smithi098 ceph-mon[103124]: pgmap v21203: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 578 KiB/s wr, 33 op/s 2023-09-10T10:17:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39460.906852 secs 2023-09-10T10:17:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39461.258703 secs 2023-09-10T10:17:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:17:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39460.906852 secs 2023-09-10T10:17:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39461.258703 secs 2023-09-10T10:17:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:17:10.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39460.906852 secs 2023-09-10T10:17:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39461.258703 secs 2023-09-10T10:17:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:17:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:11 smithi170 ceph-mon[102832]: pgmap v21204: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 578 KiB/s wr, 33 op/s 2023-09-10T10:17:11.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:11 smithi082 ceph-mon[98694]: pgmap v21204: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 578 KiB/s wr, 33 op/s 2023-09-10T10:17:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:11 smithi098 ceph-mon[103124]: pgmap v21204: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 578 KiB/s wr, 33 op/s 2023-09-10T10:17:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:17:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:13.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:13 smithi170 ceph-mon[102832]: pgmap v21205: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 529 KiB/s wr, 33 op/s 2023-09-10T10:17:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:17:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:13 smithi082 ceph-mon[98694]: pgmap v21205: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 529 KiB/s wr, 33 op/s 2023-09-10T10:17:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:17:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:13.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:13 smithi098 ceph-mon[103124]: pgmap v21205: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 529 KiB/s wr, 33 op/s 2023-09-10T10:17:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39464.676836 secs 2023-09-10T10:17:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39464.676836 secs 2023-09-10T10:17:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39464.676836 secs 2023-09-10T10:17:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39465.906934 secs 2023-09-10T10:17:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39466.258772 secs 2023-09-10T10:17:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:15 smithi170 ceph-mon[102832]: pgmap v21206: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 529 KiB/s wr, 33 op/s 2023-09-10T10:17:15.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:15 smithi170 ceph-mon[102832]: osdmap e1394: 12 total, 12 up, 12 in 2023-09-10T10:17:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39465.906934 secs 2023-09-10T10:17:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39466.258772 secs 2023-09-10T10:17:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:15 smithi082 ceph-mon[98694]: pgmap v21206: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 529 KiB/s wr, 33 op/s 2023-09-10T10:17:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:15 smithi082 ceph-mon[98694]: osdmap e1394: 12 total, 12 up, 12 in 2023-09-10T10:17:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39465.906934 secs 2023-09-10T10:17:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39466.258772 secs 2023-09-10T10:17:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:15 smithi098 ceph-mon[103124]: pgmap v21206: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 529 KiB/s wr, 33 op/s 2023-09-10T10:17:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:15 smithi098 ceph-mon[103124]: osdmap e1394: 12 total, 12 up, 12 in 2023-09-10T10:17:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:17 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:17:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:17 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:17.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:17 smithi170 ceph-mon[102832]: pgmap v21208: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 555 KiB/s wr, 18 op/s 2023-09-10T10:17:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:17 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:17:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:17 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:17 smithi082 ceph-mon[98694]: pgmap v21208: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 555 KiB/s wr, 18 op/s 2023-09-10T10:17:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:17 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:17:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:17 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:17.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:17 smithi098 ceph-mon[103124]: pgmap v21208: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 555 KiB/s wr, 18 op/s 2023-09-10T10:17:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:17:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:17:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:17:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39469.676896 secs 2023-09-10T10:17:19.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:19 smithi170 ceph-mon[102832]: pgmap v21209: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 446 KiB/s wr, 15 op/s 2023-09-10T10:17:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39469.676896 secs 2023-09-10T10:17:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:19 smithi082 ceph-mon[98694]: pgmap v21209: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 446 KiB/s wr, 15 op/s 2023-09-10T10:17:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39469.676896 secs 2023-09-10T10:17:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:19 smithi098 ceph-mon[103124]: pgmap v21209: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 446 KiB/s wr, 15 op/s 2023-09-10T10:17:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39470.906991 secs 2023-09-10T10:17:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39471.258839 secs 2023-09-10T10:17:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39470.906991 secs 2023-09-10T10:17:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39471.258839 secs 2023-09-10T10:17:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39470.906991 secs 2023-09-10T10:17:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39471.258839 secs 2023-09-10T10:17:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:21 smithi170 ceph-mon[102832]: pgmap v21210: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 446 KiB/s wr, 15 op/s 2023-09-10T10:17:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:21 smithi082 ceph-mon[98694]: pgmap v21210: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 446 KiB/s wr, 15 op/s 2023-09-10T10:17:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:21 smithi098 ceph-mon[103124]: pgmap v21210: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 446 KiB/s wr, 15 op/s 2023-09-10T10:17:23.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:23 smithi170 ceph-mon[102832]: pgmap v21211: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 13 op/s 2023-09-10T10:17:23.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:23 smithi082 ceph-mon[98694]: pgmap v21211: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 13 op/s 2023-09-10T10:17:23.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:23 smithi098 ceph-mon[103124]: pgmap v21211: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 13 op/s 2023-09-10T10:17:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39474.676957 secs 2023-09-10T10:17:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39474.676957 secs 2023-09-10T10:17:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39474.676957 secs 2023-09-10T10:17:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39475.907049 secs 2023-09-10T10:17:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39476.258904 secs 2023-09-10T10:17:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:25 smithi170 ceph-mon[102832]: pgmap v21212: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 13 op/s 2023-09-10T10:17:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39475.907049 secs 2023-09-10T10:17:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39476.258904 secs 2023-09-10T10:17:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:25 smithi082 ceph-mon[98694]: pgmap v21212: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 13 op/s 2023-09-10T10:17:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39475.907049 secs 2023-09-10T10:17:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39476.258904 secs 2023-09-10T10:17:25.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:25 smithi098 ceph-mon[103124]: pgmap v21212: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 140 KiB/s wr, 13 op/s 2023-09-10T10:17:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:26 smithi170 ceph-mon[102832]: pgmap v21213: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 281 KiB/s wr, 18 op/s 2023-09-10T10:17:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:26 smithi082 ceph-mon[98694]: pgmap v21213: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 281 KiB/s wr, 18 op/s 2023-09-10T10:17:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:26 smithi098 ceph-mon[103124]: pgmap v21213: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 281 KiB/s wr, 18 op/s 2023-09-10T10:17:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:28 smithi170 ceph-mon[102832]: pgmap v21214: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T10:17:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:28 smithi082 ceph-mon[98694]: pgmap v21214: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T10:17:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:28 smithi098 ceph-mon[103124]: pgmap v21214: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T10:17:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39479.677056 secs 2023-09-10T10:17:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39479.677056 secs 2023-09-10T10:17:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39479.677056 secs 2023-09-10T10:17:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39480.907109 secs 2023-09-10T10:17:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39481.258967 secs 2023-09-10T10:17:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:30 smithi170 ceph-mon[102832]: pgmap v21215: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T10:17:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39480.907109 secs 2023-09-10T10:17:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39481.258967 secs 2023-09-10T10:17:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:30 smithi082 ceph-mon[98694]: pgmap v21215: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T10:17:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39480.907109 secs 2023-09-10T10:17:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39481.258967 secs 2023-09-10T10:17:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:30 smithi098 ceph-mon[103124]: pgmap v21215: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T10:17:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:31 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:17:32.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:31 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:17:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:31 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:17:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:17:33.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:17:33.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:17:33.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:32 smithi170 ceph-mon[102832]: pgmap v21216: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 16 op/s 2023-09-10T10:17:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:17:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:17:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:17:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:32 smithi082 ceph-mon[98694]: pgmap v21216: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 16 op/s 2023-09-10T10:17:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:17:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:17:33.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:17:33.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:32 smithi098 ceph-mon[103124]: pgmap v21216: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 16 op/s 2023-09-10T10:17:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39484.677114 secs 2023-09-10T10:17:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39484.677114 secs 2023-09-10T10:17:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39484.677114 secs 2023-09-10T10:17:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39485.907192 secs 2023-09-10T10:17:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39486.259032 secs 2023-09-10T10:17:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:34 smithi170 ceph-mon[102832]: pgmap v21217: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:17:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39485.907192 secs 2023-09-10T10:17:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39486.259032 secs 2023-09-10T10:17:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:34 smithi082 ceph-mon[98694]: pgmap v21217: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:17:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39485.907192 secs 2023-09-10T10:17:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39486.259032 secs 2023-09-10T10:17:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:34 smithi098 ceph-mon[103124]: pgmap v21217: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:17:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:36 smithi170 ceph-mon[102832]: pgmap v21218: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 291 KiB/s wr, 16 op/s 2023-09-10T10:17:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:36 smithi082 ceph-mon[98694]: pgmap v21218: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 291 KiB/s wr, 16 op/s 2023-09-10T10:17:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:36 smithi098 ceph-mon[103124]: pgmap v21218: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 291 KiB/s wr, 16 op/s 2023-09-10T10:17:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:17:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:38 smithi170 ceph-mon[102832]: pgmap v21219: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 11 op/s 2023-09-10T10:17:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:17:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:38 smithi082 ceph-mon[98694]: pgmap v21219: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 11 op/s 2023-09-10T10:17:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:17:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:38 smithi098 ceph-mon[103124]: pgmap v21219: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 11 op/s 2023-09-10T10:17:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:17:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:17:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:17:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39490.907248 secs 2023-09-10T10:17:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39491.259096 secs 2023-09-10T10:17:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:17:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:17:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:40 smithi098 ceph-mon[103124]: pgmap v21220: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 11 op/s 2023-09-10T10:17:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39490.907248 secs 2023-09-10T10:17:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39491.259096 secs 2023-09-10T10:17:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:17:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:17:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:40 smithi170 ceph-mon[102832]: pgmap v21220: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 11 op/s 2023-09-10T10:17:40.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39490.907248 secs 2023-09-10T10:17:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39491.259096 secs 2023-09-10T10:17:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:17:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:17:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:40 smithi082 ceph-mon[98694]: pgmap v21220: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 11 op/s 2023-09-10T10:17:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:42 smithi170 ceph-mon[102832]: pgmap v21221: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 471 KiB/s wr, 18 op/s 2023-09-10T10:17:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:42 smithi082 ceph-mon[98694]: pgmap v21221: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 471 KiB/s wr, 18 op/s 2023-09-10T10:17:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:42 smithi098 ceph-mon[103124]: pgmap v21221: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 471 KiB/s wr, 18 op/s 2023-09-10T10:17:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:44 smithi170 ceph-mon[102832]: pgmap v21222: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 471 KiB/s wr, 12 op/s 2023-09-10T10:17:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:44 smithi082 ceph-mon[98694]: pgmap v21222: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 471 KiB/s wr, 12 op/s 2023-09-10T10:17:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:44 smithi098 ceph-mon[103124]: pgmap v21222: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 471 KiB/s wr, 12 op/s 2023-09-10T10:17:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39495.907308 secs 2023-09-10T10:17:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39496.259162 secs 2023-09-10T10:17:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39495.907308 secs 2023-09-10T10:17:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39496.259162 secs 2023-09-10T10:17:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39495.907308 secs 2023-09-10T10:17:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39496.259162 secs 2023-09-10T10:17:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:17:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:47.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:46 smithi170 ceph-mon[102832]: pgmap v21223: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 787 KiB/s wr, 19 op/s 2023-09-10T10:17:47.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:17:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:46 smithi082 ceph-mon[98694]: pgmap v21223: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 787 KiB/s wr, 19 op/s 2023-09-10T10:17:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:17:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:47.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:46 smithi098 ceph-mon[103124]: pgmap v21223: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 787 KiB/s wr, 19 op/s 2023-09-10T10:17:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:48 smithi170 ceph-mon[102832]: pgmap v21224: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 643 KiB/s wr, 13 op/s 2023-09-10T10:17:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:48 smithi082 ceph-mon[98694]: pgmap v21224: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 643 KiB/s wr, 13 op/s 2023-09-10T10:17:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:48 smithi098 ceph-mon[103124]: pgmap v21224: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 643 KiB/s wr, 13 op/s 2023-09-10T10:17:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39500.907412 secs 2023-09-10T10:17:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39501.259225 secs 2023-09-10T10:17:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:17:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:50.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:50 smithi170 ceph-mon[102832]: pgmap v21225: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 643 KiB/s wr, 13 op/s 2023-09-10T10:17:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39500.907412 secs 2023-09-10T10:17:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39501.259225 secs 2023-09-10T10:17:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:17:50.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:50.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:50 smithi082 ceph-mon[98694]: pgmap v21225: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 643 KiB/s wr, 13 op/s 2023-09-10T10:17:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39500.907412 secs 2023-09-10T10:17:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39501.259225 secs 2023-09-10T10:17:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:17:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:50 smithi098 ceph-mon[103124]: pgmap v21225: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 643 KiB/s wr, 13 op/s 2023-09-10T10:17:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:17:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:53.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:52 smithi170 ceph-mon[102832]: pgmap v21226: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T10:17:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:17:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:52 smithi082 ceph-mon[98694]: pgmap v21226: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T10:17:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:17:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:52 smithi098 ceph-mon[103124]: pgmap v21226: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T10:17:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:54 smithi170 ceph-mon[102832]: pgmap v21227: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 722 KiB/s wr, 14 op/s 2023-09-10T10:17:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:54 smithi082 ceph-mon[98694]: pgmap v21227: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 722 KiB/s wr, 14 op/s 2023-09-10T10:17:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:54 smithi098 ceph-mon[103124]: pgmap v21227: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 722 KiB/s wr, 14 op/s 2023-09-10T10:17:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39505.907469 secs 2023-09-10T10:17:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39506.259288 secs 2023-09-10T10:17:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39505.907469 secs 2023-09-10T10:17:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39506.259288 secs 2023-09-10T10:17:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39505.907469 secs 2023-09-10T10:17:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39506.259288 secs 2023-09-10T10:17:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:56 smithi170 ceph-mon[102832]: pgmap v21228: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 894 KiB/s wr, 21 op/s 2023-09-10T10:17:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:56 smithi082 ceph-mon[98694]: pgmap v21228: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 894 KiB/s wr, 21 op/s 2023-09-10T10:17:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:56 smithi098 ceph-mon[103124]: pgmap v21228: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 894 KiB/s wr, 21 op/s 2023-09-10T10:17:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:17:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:17:58 smithi170 ceph-mon[102832]: pgmap v21229: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 579 KiB/s wr, 14 op/s 2023-09-10T10:17:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:17:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:17:58 smithi082 ceph-mon[98694]: pgmap v21229: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 579 KiB/s wr, 14 op/s 2023-09-10T10:17:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:17:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:17:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:17:58 smithi098 ceph-mon[103124]: pgmap v21229: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 579 KiB/s wr, 14 op/s 2023-09-10T10:18:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39510.907496 secs 2023-09-10T10:18:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39511.259361 secs 2023-09-10T10:18:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:18:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:00 smithi098 ceph-mon[103124]: pgmap v21230: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 579 KiB/s wr, 14 op/s 2023-09-10T10:18:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39510.907496 secs 2023-09-10T10:18:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39511.259361 secs 2023-09-10T10:18:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:18:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:00 smithi170 ceph-mon[102832]: pgmap v21230: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 579 KiB/s wr, 14 op/s 2023-09-10T10:18:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39510.907496 secs 2023-09-10T10:18:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39511.259361 secs 2023-09-10T10:18:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:18:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:00 smithi082 ceph-mon[98694]: pgmap v21230: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 579 KiB/s wr, 14 op/s 2023-09-10T10:18:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:02 smithi170 ceph-mon[102832]: pgmap v21231: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 886 KiB/s wr, 35 op/s 2023-09-10T10:18:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:02 smithi082 ceph-mon[98694]: pgmap v21231: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 886 KiB/s wr, 35 op/s 2023-09-10T10:18:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:02 smithi098 ceph-mon[103124]: pgmap v21231: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 886 KiB/s wr, 35 op/s 2023-09-10T10:18:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:03 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:18:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:03 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:18:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:03 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:03 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:18:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:03 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:18:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:03 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:03 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:18:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:03 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:18:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:03 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39514.677545 secs 2023-09-10T10:18:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:04 smithi170 ceph-mon[102832]: pgmap v21232: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 478 KiB/s wr, 28 op/s 2023-09-10T10:18:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39514.677545 secs 2023-09-10T10:18:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:04 smithi082 ceph-mon[98694]: pgmap v21232: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 478 KiB/s wr, 28 op/s 2023-09-10T10:18:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39514.677545 secs 2023-09-10T10:18:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:04 smithi098 ceph-mon[103124]: pgmap v21232: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 478 KiB/s wr, 28 op/s 2023-09-10T10:18:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39515.907558 secs 2023-09-10T10:18:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39516.259421 secs 2023-09-10T10:18:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39515.907558 secs 2023-09-10T10:18:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39516.259421 secs 2023-09-10T10:18:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39515.907558 secs 2023-09-10T10:18:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39516.259421 secs 2023-09-10T10:18:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:06 smithi170 ceph-mon[102832]: pgmap v21233: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 630 KiB/s wr, 35 op/s 2023-09-10T10:18:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:06 smithi082 ceph-mon[98694]: pgmap v21233: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 630 KiB/s wr, 35 op/s 2023-09-10T10:18:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:06 smithi098 ceph-mon[103124]: pgmap v21233: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 630 KiB/s wr, 35 op/s 2023-09-10T10:18:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39519.677603 secs 2023-09-10T10:18:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:09 smithi170 ceph-mon[102832]: osdmap e1395: 12 total, 12 up, 12 in 2023-09-10T10:18:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:09 smithi170 ceph-mon[102832]: pgmap v21235: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 550 KiB/s wr, 34 op/s 2023-09-10T10:18:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39519.677603 secs 2023-09-10T10:18:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:09 smithi082 ceph-mon[98694]: osdmap e1395: 12 total, 12 up, 12 in 2023-09-10T10:18:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:09 smithi082 ceph-mon[98694]: pgmap v21235: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 550 KiB/s wr, 34 op/s 2023-09-10T10:18:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39519.677603 secs 2023-09-10T10:18:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:09 smithi098 ceph-mon[103124]: osdmap e1395: 12 total, 12 up, 12 in 2023-09-10T10:18:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:09 smithi098 ceph-mon[103124]: pgmap v21235: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 550 KiB/s wr, 34 op/s 2023-09-10T10:18:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39520.907648 secs 2023-09-10T10:18:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39521.259487 secs 2023-09-10T10:18:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39520.907648 secs 2023-09-10T10:18:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39521.259487 secs 2023-09-10T10:18:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39520.907648 secs 2023-09-10T10:18:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39521.259487 secs 2023-09-10T10:18:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:11 smithi170 ceph-mon[102832]: pgmap v21236: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 550 KiB/s wr, 34 op/s 2023-09-10T10:18:11.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:11 smithi082 ceph-mon[98694]: pgmap v21236: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 550 KiB/s wr, 34 op/s 2023-09-10T10:18:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:11 smithi098 ceph-mon[103124]: pgmap v21236: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 550 KiB/s wr, 34 op/s 2023-09-10T10:18:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:12 smithi170 ceph-mon[102832]: pgmap v21237: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 431 KiB/s wr, 16 op/s 2023-09-10T10:18:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:12 smithi082 ceph-mon[98694]: pgmap v21237: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 431 KiB/s wr, 16 op/s 2023-09-10T10:18:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:12 smithi098 ceph-mon[103124]: pgmap v21237: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 431 KiB/s wr, 16 op/s 2023-09-10T10:18:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39524.677666 secs 2023-09-10T10:18:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39524.677666 secs 2023-09-10T10:18:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39524.677666 secs 2023-09-10T10:18:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:14 smithi170 ceph-mon[102832]: pgmap v21238: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 431 KiB/s wr, 16 op/s 2023-09-10T10:18:14.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:14 smithi082 ceph-mon[98694]: pgmap v21238: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 431 KiB/s wr, 16 op/s 2023-09-10T10:18:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:14 smithi098 ceph-mon[103124]: pgmap v21238: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 431 KiB/s wr, 16 op/s 2023-09-10T10:18:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39525.907706 secs 2023-09-10T10:18:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39526.259552 secs 2023-09-10T10:18:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:15 smithi170 ceph-mon[102832]: osdmap e1396: 12 total, 12 up, 12 in 2023-09-10T10:18:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39525.907706 secs 2023-09-10T10:18:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39526.259552 secs 2023-09-10T10:18:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:15 smithi082 ceph-mon[98694]: osdmap e1396: 12 total, 12 up, 12 in 2023-09-10T10:18:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39525.907706 secs 2023-09-10T10:18:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39526.259552 secs 2023-09-10T10:18:15.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:15 smithi098 ceph-mon[103124]: osdmap e1396: 12 total, 12 up, 12 in 2023-09-10T10:18:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:18:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:17.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:16 smithi170 ceph-mon[102832]: pgmap v21240: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 483 KiB/s wr, 19 op/s 2023-09-10T10:18:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:18:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:17.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:16 smithi082 ceph-mon[98694]: pgmap v21240: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 483 KiB/s wr, 19 op/s 2023-09-10T10:18:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:18:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:16 smithi098 ceph-mon[103124]: pgmap v21240: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 483 KiB/s wr, 19 op/s 2023-09-10T10:18:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:18:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39529.677726 secs 2023-09-10T10:18:19.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:18 smithi170 ceph-mon[102832]: pgmap v21241: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 389 KiB/s wr, 15 op/s 2023-09-10T10:18:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:18:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39529.677726 secs 2023-09-10T10:18:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:18 smithi082 ceph-mon[98694]: pgmap v21241: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 389 KiB/s wr, 15 op/s 2023-09-10T10:18:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:18:19.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:19.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39529.677726 secs 2023-09-10T10:18:19.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:18 smithi098 ceph-mon[103124]: pgmap v21241: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 389 KiB/s wr, 15 op/s 2023-09-10T10:18:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39530.907765 secs 2023-09-10T10:18:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39531.259616 secs 2023-09-10T10:18:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39530.907765 secs 2023-09-10T10:18:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39531.259616 secs 2023-09-10T10:18:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39530.907765 secs 2023-09-10T10:18:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39531.259616 secs 2023-09-10T10:18:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:20 smithi170 ceph-mon[102832]: pgmap v21242: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 389 KiB/s wr, 15 op/s 2023-09-10T10:18:21.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:20 smithi082 ceph-mon[98694]: pgmap v21242: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 389 KiB/s wr, 15 op/s 2023-09-10T10:18:21.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:20 smithi098 ceph-mon[103124]: pgmap v21242: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 389 KiB/s wr, 15 op/s 2023-09-10T10:18:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:22 smithi170 ceph-mon[102832]: pgmap v21243: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 141 KiB/s wr, 14 op/s 2023-09-10T10:18:22.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:22 smithi082 ceph-mon[98694]: pgmap v21243: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 141 KiB/s wr, 14 op/s 2023-09-10T10:18:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:22 smithi098 ceph-mon[103124]: pgmap v21243: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 141 KiB/s wr, 14 op/s 2023-09-10T10:18:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39534.677810 secs 2023-09-10T10:18:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39534.677810 secs 2023-09-10T10:18:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39534.677810 secs 2023-09-10T10:18:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:24 smithi170 ceph-mon[102832]: pgmap v21244: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 141 KiB/s wr, 14 op/s 2023-09-10T10:18:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:24 smithi082 ceph-mon[98694]: pgmap v21244: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 141 KiB/s wr, 14 op/s 2023-09-10T10:18:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:24 smithi098 ceph-mon[103124]: pgmap v21244: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 141 KiB/s wr, 14 op/s 2023-09-10T10:18:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39535.907844 secs 2023-09-10T10:18:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39536.259707 secs 2023-09-10T10:18:25.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39535.907844 secs 2023-09-10T10:18:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39536.259707 secs 2023-09-10T10:18:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39535.907844 secs 2023-09-10T10:18:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39536.259707 secs 2023-09-10T10:18:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:26 smithi170 ceph-mon[102832]: pgmap v21245: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 282 KiB/s wr, 18 op/s 2023-09-10T10:18:26.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:26 smithi082 ceph-mon[98694]: pgmap v21245: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 282 KiB/s wr, 18 op/s 2023-09-10T10:18:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:26 smithi098 ceph-mon[103124]: pgmap v21245: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 282 KiB/s wr, 18 op/s 2023-09-10T10:18:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:28 smithi170 ceph-mon[102832]: pgmap v21246: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:18:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:28 smithi082 ceph-mon[98694]: pgmap v21246: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:18:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:28 smithi098 ceph-mon[103124]: pgmap v21246: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:18:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39539.677868 secs 2023-09-10T10:18:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39539.677868 secs 2023-09-10T10:18:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39539.677868 secs 2023-09-10T10:18:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39540.907903 secs 2023-09-10T10:18:30.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39541.259774 secs 2023-09-10T10:18:30.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:30 smithi170 ceph-mon[102832]: pgmap v21247: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:18:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39540.907903 secs 2023-09-10T10:18:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39541.259774 secs 2023-09-10T10:18:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:30 smithi082 ceph-mon[98694]: pgmap v21247: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:18:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39540.907903 secs 2023-09-10T10:18:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39541.259774 secs 2023-09-10T10:18:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:30 smithi098 ceph-mon[103124]: pgmap v21247: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:18:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:32 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:18:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:32 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:18:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:32 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:18:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:33 smithi170 ceph-mon[102832]: pgmap v21248: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 148 KiB/s wr, 15 op/s 2023-09-10T10:18:33.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:33 smithi082 ceph-mon[98694]: pgmap v21248: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 148 KiB/s wr, 15 op/s 2023-09-10T10:18:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:33 smithi098 ceph-mon[103124]: pgmap v21248: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 148 KiB/s wr, 15 op/s 2023-09-10T10:18:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39544.677928 secs 2023-09-10T10:18:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39544.677928 secs 2023-09-10T10:18:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39544.677928 secs 2023-09-10T10:18:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39545.907961 secs 2023-09-10T10:18:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39546.259846 secs 2023-09-10T10:18:35.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:35 smithi170 ceph-mon[102832]: pgmap v21249: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:18:35.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:18:35.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:18:35.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:18:35.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:18:35.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:18:35.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:18:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39545.907961 secs 2023-09-10T10:18:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39546.259846 secs 2023-09-10T10:18:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:35 smithi082 ceph-mon[98694]: pgmap v21249: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:18:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:18:35.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:18:35.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:18:35.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:18:35.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:18:35.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:18:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39545.907961 secs 2023-09-10T10:18:35.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39546.259846 secs 2023-09-10T10:18:35.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:35 smithi098 ceph-mon[103124]: pgmap v21249: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:18:35.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:18:35.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:18:35.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:18:35.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:18:35.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:18:35.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:18:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:18:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:18:36.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:18:36.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:18:36.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:36 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T10:18:36.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:36 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T10:18:36.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:18:36.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:18:36.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:18:36.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:18:36.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:18:36.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:36 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T10:18:36.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:36 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T10:18:36.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:18:36.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:18:36.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:18:36.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:36 smithi170 ceph-mon[102832]: pgmap v21250: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 292 KiB/s wr, 16 op/s 2023-09-10T10:18:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:18:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:18:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:18:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:18:36.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:36 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T10:18:36.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:36 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T10:18:36.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:18:36.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:18:36.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:18:36.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:18:36.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:18:36.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:36 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T10:18:36.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:36 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T10:18:36.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:18:36.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:18:36.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:18:36.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:36 smithi082 ceph-mon[98694]: pgmap v21250: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 292 KiB/s wr, 16 op/s 2023-09-10T10:18:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:18:36.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:18:36.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:18:36.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:18:36.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:36 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T10:18:36.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:36 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T10:18:36.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:18:36.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:18:36.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:18:36.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:18:36.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:18:36.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:36 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T10:18:36.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:36 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T10:18:36.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:18:36.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:18:36.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:18:36.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:36 smithi098 ceph-mon[103124]: pgmap v21250: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 292 KiB/s wr, 16 op/s 2023-09-10T10:18:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:18:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:38 smithi170 ceph-mon[102832]: pgmap v21251: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:18:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:18:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:38 smithi082 ceph-mon[98694]: pgmap v21251: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:18:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:18:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:38.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:38 smithi098 ceph-mon[103124]: pgmap v21251: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:18:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:18:39.747 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:18:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:18:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39550.908039 secs 2023-09-10T10:18:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39551.259910 secs 2023-09-10T10:18:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:18:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:18:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:40 smithi170 ceph-mon[102832]: pgmap v21252: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:18:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39550.908039 secs 2023-09-10T10:18:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39551.259910 secs 2023-09-10T10:18:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:18:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:18:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:40 smithi082 ceph-mon[98694]: pgmap v21252: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:18:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39550.908039 secs 2023-09-10T10:18:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39551.259910 secs 2023-09-10T10:18:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:18:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:18:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:40 smithi098 ceph-mon[103124]: pgmap v21252: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:18:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:42 smithi170 ceph-mon[102832]: pgmap v21253: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 472 KiB/s wr, 17 op/s 2023-09-10T10:18:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:42 smithi082 ceph-mon[98694]: pgmap v21253: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 472 KiB/s wr, 17 op/s 2023-09-10T10:18:42.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:42 smithi098 ceph-mon[103124]: pgmap v21253: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 472 KiB/s wr, 17 op/s 2023-09-10T10:18:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:44 smithi170 ceph-mon[102832]: pgmap v21254: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 472 KiB/s wr, 12 op/s 2023-09-10T10:18:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:44 smithi082 ceph-mon[98694]: pgmap v21254: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 472 KiB/s wr, 12 op/s 2023-09-10T10:18:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:44 smithi098 ceph-mon[103124]: pgmap v21254: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 472 KiB/s wr, 12 op/s 2023-09-10T10:18:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39555.908096 secs 2023-09-10T10:18:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39556.259976 secs 2023-09-10T10:18:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39555.908096 secs 2023-09-10T10:18:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39556.259976 secs 2023-09-10T10:18:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39555.908096 secs 2023-09-10T10:18:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39556.259976 secs 2023-09-10T10:18:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:18:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:47.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:46 smithi170 ceph-mon[102832]: pgmap v21255: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 801 KiB/s wr, 18 op/s 2023-09-10T10:18:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:18:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:46 smithi082 ceph-mon[98694]: pgmap v21255: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 801 KiB/s wr, 18 op/s 2023-09-10T10:18:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:18:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:47.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:46 smithi098 ceph-mon[103124]: pgmap v21255: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 801 KiB/s wr, 18 op/s 2023-09-10T10:18:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:48 smithi170 ceph-mon[102832]: pgmap v21256: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 657 KiB/s wr, 12 op/s 2023-09-10T10:18:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:48 smithi082 ceph-mon[98694]: pgmap v21256: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 657 KiB/s wr, 12 op/s 2023-09-10T10:18:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:48 smithi098 ceph-mon[103124]: pgmap v21256: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 657 KiB/s wr, 12 op/s 2023-09-10T10:18:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39560.908153 secs 2023-09-10T10:18:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39561.260040 secs 2023-09-10T10:18:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:18:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:50 smithi170 ceph-mon[102832]: pgmap v21257: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 657 KiB/s wr, 12 op/s 2023-09-10T10:18:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39560.908153 secs 2023-09-10T10:18:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39561.260040 secs 2023-09-10T10:18:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:18:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:50 smithi082 ceph-mon[98694]: pgmap v21257: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 657 KiB/s wr, 12 op/s 2023-09-10T10:18:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39560.908153 secs 2023-09-10T10:18:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39561.260040 secs 2023-09-10T10:18:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:18:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:50 smithi098 ceph-mon[103124]: pgmap v21257: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 657 KiB/s wr, 12 op/s 2023-09-10T10:18:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:18:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:52 smithi170 ceph-mon[102832]: pgmap v21258: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 20 op/s 2023-09-10T10:18:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:18:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:52 smithi082 ceph-mon[98694]: pgmap v21258: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 20 op/s 2023-09-10T10:18:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:18:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:52 smithi098 ceph-mon[103124]: pgmap v21258: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 20 op/s 2023-09-10T10:18:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:54 smithi170 ceph-mon[102832]: pgmap v21259: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 701 KiB/s wr, 13 op/s 2023-09-10T10:18:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:54 smithi082 ceph-mon[98694]: pgmap v21259: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 701 KiB/s wr, 13 op/s 2023-09-10T10:18:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:54 smithi098 ceph-mon[103124]: pgmap v21259: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 701 KiB/s wr, 13 op/s 2023-09-10T10:18:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39565.908238 secs 2023-09-10T10:18:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39566.260104 secs 2023-09-10T10:18:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39565.908238 secs 2023-09-10T10:18:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39566.260104 secs 2023-09-10T10:18:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39565.908238 secs 2023-09-10T10:18:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39566.260104 secs 2023-09-10T10:18:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:56 smithi170 ceph-mon[102832]: pgmap v21260: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 896 KiB/s wr, 20 op/s 2023-09-10T10:18:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:56 smithi082 ceph-mon[98694]: pgmap v21260: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 896 KiB/s wr, 20 op/s 2023-09-10T10:18:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:56 smithi098 ceph-mon[103124]: pgmap v21260: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 896 KiB/s wr, 20 op/s 2023-09-10T10:18:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:18:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:18:58 smithi170 ceph-mon[102832]: pgmap v21261: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 567 KiB/s wr, 14 op/s 2023-09-10T10:18:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:18:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:18:58 smithi082 ceph-mon[98694]: pgmap v21261: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 567 KiB/s wr, 14 op/s 2023-09-10T10:18:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:18:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:18:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:18:58 smithi098 ceph-mon[103124]: pgmap v21261: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 567 KiB/s wr, 14 op/s 2023-09-10T10:19:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39570.908298 secs 2023-09-10T10:19:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39571.260168 secs 2023-09-10T10:19:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:19:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:00 smithi098 ceph-mon[103124]: pgmap v21262: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 567 KiB/s wr, 14 op/s 2023-09-10T10:19:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39570.908298 secs 2023-09-10T10:19:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39571.260168 secs 2023-09-10T10:19:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:19:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:00 smithi170 ceph-mon[102832]: pgmap v21262: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 567 KiB/s wr, 14 op/s 2023-09-10T10:19:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39570.908298 secs 2023-09-10T10:19:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39571.260168 secs 2023-09-10T10:19:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:19:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:00 smithi082 ceph-mon[98694]: pgmap v21262: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 567 KiB/s wr, 14 op/s 2023-09-10T10:19:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:02 smithi170 ceph-mon[102832]: pgmap v21263: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 852 KiB/s wr, 24 op/s 2023-09-10T10:19:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:02 smithi082 ceph-mon[98694]: pgmap v21263: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 852 KiB/s wr, 24 op/s 2023-09-10T10:19:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:02 smithi098 ceph-mon[103124]: pgmap v21263: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 852 KiB/s wr, 24 op/s 2023-09-10T10:19:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:03 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:19:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:03 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:19:03.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:03 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:03 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:19:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:03 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:19:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:03 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:03 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:19:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:03 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:19:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:03 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39574.678370 secs 2023-09-10T10:19:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:04 smithi170 ceph-mon[102832]: pgmap v21264: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 480 KiB/s wr, 16 op/s 2023-09-10T10:19:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39574.678370 secs 2023-09-10T10:19:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:04 smithi082 ceph-mon[98694]: pgmap v21264: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 480 KiB/s wr, 16 op/s 2023-09-10T10:19:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39574.678370 secs 2023-09-10T10:19:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:04 smithi098 ceph-mon[103124]: pgmap v21264: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 480 KiB/s wr, 16 op/s 2023-09-10T10:19:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39575.908357 secs 2023-09-10T10:19:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39576.260236 secs 2023-09-10T10:19:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39575.908357 secs 2023-09-10T10:19:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39576.260236 secs 2023-09-10T10:19:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39575.908357 secs 2023-09-10T10:19:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39576.260236 secs 2023-09-10T10:19:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:06 smithi170 ceph-mon[102832]: pgmap v21265: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 654 KiB/s wr, 34 op/s 2023-09-10T10:19:06.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:06 smithi082 ceph-mon[98694]: pgmap v21265: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 654 KiB/s wr, 34 op/s 2023-09-10T10:19:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:06 smithi098 ceph-mon[103124]: pgmap v21265: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 654 KiB/s wr, 34 op/s 2023-09-10T10:19:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39579.678428 secs 2023-09-10T10:19:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:09 smithi170 ceph-mon[102832]: osdmap e1397: 12 total, 12 up, 12 in 2023-09-10T10:19:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:09 smithi170 ceph-mon[102832]: pgmap v21267: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 550 KiB/s wr, 33 op/s 2023-09-10T10:19:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39579.678428 secs 2023-09-10T10:19:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:09 smithi082 ceph-mon[98694]: osdmap e1397: 12 total, 12 up, 12 in 2023-09-10T10:19:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:09 smithi082 ceph-mon[98694]: pgmap v21267: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 550 KiB/s wr, 33 op/s 2023-09-10T10:19:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39579.678428 secs 2023-09-10T10:19:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:09 smithi098 ceph-mon[103124]: osdmap e1397: 12 total, 12 up, 12 in 2023-09-10T10:19:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:09 smithi098 ceph-mon[103124]: pgmap v21267: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 550 KiB/s wr, 33 op/s 2023-09-10T10:19:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39580.908440 secs 2023-09-10T10:19:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39581.260301 secs 2023-09-10T10:19:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:19:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39580.908440 secs 2023-09-10T10:19:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39581.260301 secs 2023-09-10T10:19:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:19:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39580.908440 secs 2023-09-10T10:19:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39581.260301 secs 2023-09-10T10:19:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:19:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:11.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:11 smithi170 ceph-mon[102832]: pgmap v21268: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 550 KiB/s wr, 33 op/s 2023-09-10T10:19:11.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:11 smithi082 ceph-mon[98694]: pgmap v21268: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 550 KiB/s wr, 33 op/s 2023-09-10T10:19:11.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:11 smithi098 ceph-mon[103124]: pgmap v21268: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 550 KiB/s wr, 33 op/s 2023-09-10T10:19:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:19:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:19:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:19:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:14 smithi170 ceph-mon[102832]: pgmap v21269: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 458 KiB/s wr, 30 op/s 2023-09-10T10:19:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39584.678492 secs 2023-09-10T10:19:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:14 smithi082 ceph-mon[98694]: pgmap v21269: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 458 KiB/s wr, 30 op/s 2023-09-10T10:19:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39584.678492 secs 2023-09-10T10:19:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:14 smithi098 ceph-mon[103124]: pgmap v21269: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 458 KiB/s wr, 30 op/s 2023-09-10T10:19:14.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39584.678492 secs 2023-09-10T10:19:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39585.908497 secs 2023-09-10T10:19:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39586.260368 secs 2023-09-10T10:19:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:15 smithi170 ceph-mon[102832]: osdmap e1398: 12 total, 12 up, 12 in 2023-09-10T10:19:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39585.908497 secs 2023-09-10T10:19:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39586.260368 secs 2023-09-10T10:19:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:15 smithi082 ceph-mon[98694]: osdmap e1398: 12 total, 12 up, 12 in 2023-09-10T10:19:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39585.908497 secs 2023-09-10T10:19:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39586.260368 secs 2023-09-10T10:19:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:15 smithi098 ceph-mon[103124]: osdmap e1398: 12 total, 12 up, 12 in 2023-09-10T10:19:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:16 smithi170 ceph-mon[102832]: pgmap v21270: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 458 KiB/s wr, 30 op/s 2023-09-10T10:19:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:16 smithi082 ceph-mon[98694]: pgmap v21270: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 458 KiB/s wr, 30 op/s 2023-09-10T10:19:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:16 smithi098 ceph-mon[103124]: pgmap v21270: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 458 KiB/s wr, 30 op/s 2023-09-10T10:19:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:18 smithi170 ceph-mon[102832]: pgmap v21272: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 527 KiB/s wr, 20 op/s 2023-09-10T10:19:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:18 smithi082 ceph-mon[98694]: pgmap v21272: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 527 KiB/s wr, 20 op/s 2023-09-10T10:19:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:18 smithi098 ceph-mon[103124]: pgmap v21272: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 527 KiB/s wr, 20 op/s 2023-09-10T10:19:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39589.678574 secs 2023-09-10T10:19:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39589.678574 secs 2023-09-10T10:19:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39589.678574 secs 2023-09-10T10:19:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:20 smithi170 ceph-mon[102832]: pgmap v21273: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 425 KiB/s wr, 16 op/s 2023-09-10T10:19:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39590.908555 secs 2023-09-10T10:19:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39591.260434 secs 2023-09-10T10:19:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:20 smithi082 ceph-mon[98694]: pgmap v21273: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 425 KiB/s wr, 16 op/s 2023-09-10T10:19:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39590.908555 secs 2023-09-10T10:19:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39591.260434 secs 2023-09-10T10:19:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:20 smithi098 ceph-mon[103124]: pgmap v21273: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 425 KiB/s wr, 16 op/s 2023-09-10T10:19:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39590.908555 secs 2023-09-10T10:19:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39591.260434 secs 2023-09-10T10:19:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:22 smithi170 ceph-mon[102832]: pgmap v21274: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 425 KiB/s wr, 16 op/s 2023-09-10T10:19:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:22 smithi082 ceph-mon[98694]: pgmap v21274: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 425 KiB/s wr, 16 op/s 2023-09-10T10:19:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:22 smithi098 ceph-mon[103124]: pgmap v21274: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 425 KiB/s wr, 16 op/s 2023-09-10T10:19:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:24 smithi170 ceph-mon[102832]: pgmap v21275: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 13 op/s 2023-09-10T10:19:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39594.678666 secs 2023-09-10T10:19:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:24 smithi082 ceph-mon[98694]: pgmap v21275: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 13 op/s 2023-09-10T10:19:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39594.678666 secs 2023-09-10T10:19:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:24 smithi098 ceph-mon[103124]: pgmap v21275: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 13 op/s 2023-09-10T10:19:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39594.678666 secs 2023-09-10T10:19:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39595.908616 secs 2023-09-10T10:19:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39596.260486 secs 2023-09-10T10:19:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39595.908616 secs 2023-09-10T10:19:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39596.260486 secs 2023-09-10T10:19:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39595.908616 secs 2023-09-10T10:19:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39596.260486 secs 2023-09-10T10:19:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:26 smithi170 ceph-mon[102832]: pgmap v21276: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 13 op/s 2023-09-10T10:19:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:26 smithi082 ceph-mon[98694]: pgmap v21276: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 13 op/s 2023-09-10T10:19:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:26 smithi098 ceph-mon[103124]: pgmap v21276: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 176 KiB/s wr, 13 op/s 2023-09-10T10:19:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:28 smithi170 ceph-mon[102832]: pgmap v21277: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 314 KiB/s wr, 17 op/s 2023-09-10T10:19:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:28 smithi082 ceph-mon[98694]: pgmap v21277: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 314 KiB/s wr, 17 op/s 2023-09-10T10:19:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:28 smithi098 ceph-mon[103124]: pgmap v21277: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 314 KiB/s wr, 17 op/s 2023-09-10T10:19:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39599.678724 secs 2023-09-10T10:19:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39599.678724 secs 2023-09-10T10:19:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39599.678724 secs 2023-09-10T10:19:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:30 smithi170 ceph-mon[102832]: pgmap v21278: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:19:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39600.908675 secs 2023-09-10T10:19:30.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39601.260552 secs 2023-09-10T10:19:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:30 smithi082 ceph-mon[98694]: pgmap v21278: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:19:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39600.908675 secs 2023-09-10T10:19:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39601.260552 secs 2023-09-10T10:19:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:30 smithi098 ceph-mon[103124]: pgmap v21278: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:19:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39600.908675 secs 2023-09-10T10:19:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39601.260552 secs 2023-09-10T10:19:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:31 smithi170 ceph-mon[102832]: pgmap v21279: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:19:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:31 smithi082 ceph-mon[98694]: pgmap v21279: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:19:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:31 smithi098 ceph-mon[103124]: pgmap v21279: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:19:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:33 smithi170 ceph-mon[102832]: pgmap v21280: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 149 KiB/s wr, 15 op/s 2023-09-10T10:19:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39604.678778 secs 2023-09-10T10:19:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:33 smithi082 ceph-mon[98694]: pgmap v21280: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 149 KiB/s wr, 15 op/s 2023-09-10T10:19:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39604.678778 secs 2023-09-10T10:19:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:33 smithi098 ceph-mon[103124]: pgmap v21280: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 149 KiB/s wr, 15 op/s 2023-09-10T10:19:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39604.678778 secs 2023-09-10T10:19:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39605.908738 secs 2023-09-10T10:19:35.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39606.260615 secs 2023-09-10T10:19:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39605.908738 secs 2023-09-10T10:19:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39606.260615 secs 2023-09-10T10:19:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39605.908738 secs 2023-09-10T10:19:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39606.260615 secs 2023-09-10T10:19:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:35 smithi170 ceph-mon[102832]: pgmap v21281: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:19:36.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:35 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:19:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:35 smithi082 ceph-mon[98694]: pgmap v21281: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:19:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:35 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:19:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:35 smithi098 ceph-mon[103124]: pgmap v21281: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:19:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:35 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:19:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:19:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:19:37.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:36 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:19:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:19:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:19:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:36 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:19:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:19:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:19:37.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:36 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:19:38.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:37 smithi170 ceph-mon[102832]: pgmap v21282: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 292 KiB/s wr, 16 op/s 2023-09-10T10:19:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:37 smithi082 ceph-mon[98694]: pgmap v21282: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 292 KiB/s wr, 16 op/s 2023-09-10T10:19:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:37 smithi098 ceph-mon[103124]: pgmap v21282: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 292 KiB/s wr, 16 op/s 2023-09-10T10:19:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:19:39.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:19:39.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:19:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:39 smithi170 ceph-mon[102832]: pgmap v21283: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:19:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39610.908796 secs 2023-09-10T10:19:40.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39611.260679 secs 2023-09-10T10:19:40.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:19:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:39 smithi082 ceph-mon[98694]: pgmap v21283: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:19:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39610.908796 secs 2023-09-10T10:19:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39611.260679 secs 2023-09-10T10:19:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:19:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:39 smithi098 ceph-mon[103124]: pgmap v21283: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:19:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39610.908796 secs 2023-09-10T10:19:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39611.260679 secs 2023-09-10T10:19:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:19:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:19:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:19:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:41.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:19:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:19:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:19:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:19:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:42.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:41 smithi170 ceph-mon[102832]: pgmap v21284: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:19:42.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:41 smithi082 ceph-mon[98694]: pgmap v21284: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:19:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:41 smithi098 ceph-mon[103124]: pgmap v21284: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:19:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:43 smithi170 ceph-mon[102832]: pgmap v21285: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 473 KiB/s wr, 17 op/s 2023-09-10T10:19:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:43 smithi082 ceph-mon[98694]: pgmap v21285: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 473 KiB/s wr, 17 op/s 2023-09-10T10:19:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:43 smithi098 ceph-mon[103124]: pgmap v21285: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 473 KiB/s wr, 17 op/s 2023-09-10T10:19:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39615.908853 secs 2023-09-10T10:19:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39616.260747 secs 2023-09-10T10:19:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39615.908853 secs 2023-09-10T10:19:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39616.260747 secs 2023-09-10T10:19:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39615.908853 secs 2023-09-10T10:19:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39616.260747 secs 2023-09-10T10:19:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:45 smithi170 ceph-mon[102832]: pgmap v21286: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 473 KiB/s wr, 12 op/s 2023-09-10T10:19:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:45 smithi082 ceph-mon[98694]: pgmap v21286: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 473 KiB/s wr, 12 op/s 2023-09-10T10:19:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:45 smithi098 ceph-mon[103124]: pgmap v21286: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 473 KiB/s wr, 12 op/s 2023-09-10T10:19:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:19:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:19:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:19:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:47 smithi170 ceph-mon[102832]: pgmap v21287: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 790 KiB/s wr, 20 op/s 2023-09-10T10:19:48.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:47 smithi082 ceph-mon[98694]: pgmap v21287: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 790 KiB/s wr, 20 op/s 2023-09-10T10:19:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:47 smithi098 ceph-mon[103124]: pgmap v21287: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 790 KiB/s wr, 20 op/s 2023-09-10T10:19:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:49 smithi170 ceph-mon[102832]: pgmap v21288: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 646 KiB/s wr, 14 op/s 2023-09-10T10:19:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39620.908938 secs 2023-09-10T10:19:50.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39621.260814 secs 2023-09-10T10:19:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:49 smithi082 ceph-mon[98694]: pgmap v21288: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 646 KiB/s wr, 14 op/s 2023-09-10T10:19:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39620.908938 secs 2023-09-10T10:19:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39621.260814 secs 2023-09-10T10:19:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:49 smithi098 ceph-mon[103124]: pgmap v21288: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 646 KiB/s wr, 14 op/s 2023-09-10T10:19:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39620.908938 secs 2023-09-10T10:19:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39621.260814 secs 2023-09-10T10:19:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:19:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:19:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:19:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:52.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:51 smithi170 ceph-mon[102832]: pgmap v21289: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 646 KiB/s wr, 14 op/s 2023-09-10T10:19:52.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:51 smithi082 ceph-mon[98694]: pgmap v21289: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 646 KiB/s wr, 14 op/s 2023-09-10T10:19:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:51 smithi098 ceph-mon[103124]: pgmap v21289: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 646 KiB/s wr, 14 op/s 2023-09-10T10:19:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:19:53.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:19:53.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:19:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:54.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:53 smithi170 ceph-mon[102832]: pgmap v21290: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1004 KiB/s wr, 21 op/s 2023-09-10T10:19:54.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:53 smithi082 ceph-mon[98694]: pgmap v21290: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1004 KiB/s wr, 21 op/s 2023-09-10T10:19:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:53 smithi098 ceph-mon[103124]: pgmap v21290: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1004 KiB/s wr, 21 op/s 2023-09-10T10:19:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39625.908998 secs 2023-09-10T10:19:55.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39626.260879 secs 2023-09-10T10:19:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39625.908998 secs 2023-09-10T10:19:55.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39626.260879 secs 2023-09-10T10:19:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39625.908998 secs 2023-09-10T10:19:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39626.260879 secs 2023-09-10T10:19:56.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:55 smithi082 ceph-mon[98694]: pgmap v21291: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 675 KiB/s wr, 14 op/s 2023-09-10T10:19:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:55 smithi098 ceph-mon[103124]: pgmap v21291: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 675 KiB/s wr, 14 op/s 2023-09-10T10:19:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:55 smithi170 ceph-mon[102832]: pgmap v21291: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 675 KiB/s wr, 14 op/s 2023-09-10T10:19:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:57 smithi082 ceph-mon[98694]: pgmap v21292: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 898 KiB/s wr, 20 op/s 2023-09-10T10:19:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:19:58.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:57 smithi098 ceph-mon[103124]: pgmap v21292: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 898 KiB/s wr, 20 op/s 2023-09-10T10:19:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:19:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:19:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:57 smithi170 ceph-mon[102832]: pgmap v21292: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 898 KiB/s wr, 20 op/s 2023-09-10T10:19:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:19:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:59 smithi082 ceph-mon[98694]: pgmap v21293: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 581 KiB/s wr, 13 op/s 2023-09-10T10:20:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39630.909078 secs 2023-09-10T10:20:00.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:19:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39631.260944 secs 2023-09-10T10:20:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:59 smithi098 ceph-mon[103124]: pgmap v21293: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 581 KiB/s wr, 13 op/s 2023-09-10T10:20:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39630.909078 secs 2023-09-10T10:20:00.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:19:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39631.260944 secs 2023-09-10T10:20:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:59 smithi170 ceph-mon[102832]: pgmap v21293: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 581 KiB/s wr, 13 op/s 2023-09-10T10:20:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39630.909078 secs 2023-09-10T10:20:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:19:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39631.260944 secs 2023-09-10T10:20:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:20:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:00 smithi098 ceph-mon[103124]: Health detail: HEALTH_WARN 2 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T10:20:01.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:00 smithi098 ceph-mon[103124]: [WRN] MDS_SLOW_REQUEST: 2 MDSs report slow requests 2023-09-10T10:20:01.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:00 smithi098 ceph-mon[103124]: mds.l(mds.4): 3 slow requests are blocked > 180 secs 2023-09-10T10:20:01.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:00 smithi098 ceph-mon[103124]: mds.h(mds.1): 3 slow requests are blocked > 180 secs 2023-09-10T10:20:01.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:00 smithi098 ceph-mon[103124]: [WRN] MDS_TRIM: 2 MDSs behind on trimming 2023-09-10T10:20:01.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:00 smithi098 ceph-mon[103124]: mds.e(mds.0): Behind on trimming (1107/128) max_segments: 128, num_segments: 1107 2023-09-10T10:20:01.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:00 smithi098 ceph-mon[103124]: mds.b(mds.0): Behind on trimming (1107/128) max_segments: 128, num_segments: 1107 2023-09-10T10:20:01.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:20:01.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:01.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:00 smithi170 ceph-mon[102832]: Health detail: HEALTH_WARN 2 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T10:20:01.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:00 smithi170 ceph-mon[102832]: [WRN] MDS_SLOW_REQUEST: 2 MDSs report slow requests 2023-09-10T10:20:01.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:00 smithi170 ceph-mon[102832]: mds.l(mds.4): 3 slow requests are blocked > 180 secs 2023-09-10T10:20:01.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:00 smithi170 ceph-mon[102832]: mds.h(mds.1): 3 slow requests are blocked > 180 secs 2023-09-10T10:20:01.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:00 smithi170 ceph-mon[102832]: [WRN] MDS_TRIM: 2 MDSs behind on trimming 2023-09-10T10:20:01.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:00 smithi170 ceph-mon[102832]: mds.e(mds.0): Behind on trimming (1107/128) max_segments: 128, num_segments: 1107 2023-09-10T10:20:01.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:00 smithi170 ceph-mon[102832]: mds.b(mds.0): Behind on trimming (1107/128) max_segments: 128, num_segments: 1107 2023-09-10T10:20:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:20:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:00 smithi082 ceph-mon[98694]: Health detail: HEALTH_WARN 2 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T10:20:01.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:00 smithi082 ceph-mon[98694]: [WRN] MDS_SLOW_REQUEST: 2 MDSs report slow requests 2023-09-10T10:20:01.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:00 smithi082 ceph-mon[98694]: mds.l(mds.4): 3 slow requests are blocked > 180 secs 2023-09-10T10:20:01.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:00 smithi082 ceph-mon[98694]: mds.h(mds.1): 3 slow requests are blocked > 180 secs 2023-09-10T10:20:01.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:00 smithi082 ceph-mon[98694]: [WRN] MDS_TRIM: 2 MDSs behind on trimming 2023-09-10T10:20:01.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:00 smithi082 ceph-mon[98694]: mds.e(mds.0): Behind on trimming (1107/128) max_segments: 128, num_segments: 1107 2023-09-10T10:20:01.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:00 smithi082 ceph-mon[98694]: mds.b(mds.0): Behind on trimming (1107/128) max_segments: 128, num_segments: 1107 2023-09-10T10:20:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:01 smithi098 ceph-mon[103124]: pgmap v21294: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 581 KiB/s wr, 13 op/s 2023-09-10T10:20:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:01 smithi170 ceph-mon[102832]: pgmap v21294: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 581 KiB/s wr, 13 op/s 2023-09-10T10:20:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:01 smithi082 ceph-mon[98694]: pgmap v21294: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 581 KiB/s wr, 13 op/s 2023-09-10T10:20:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:03 smithi098 ceph-mon[103124]: pgmap v21295: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 855 KiB/s wr, 21 op/s 2023-09-10T10:20:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39634.679207 secs 2023-09-10T10:20:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:03 smithi170 ceph-mon[102832]: pgmap v21295: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 855 KiB/s wr, 21 op/s 2023-09-10T10:20:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39634.679207 secs 2023-09-10T10:20:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:03 smithi082 ceph-mon[98694]: pgmap v21295: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 855 KiB/s wr, 21 op/s 2023-09-10T10:20:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39634.679207 secs 2023-09-10T10:20:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39635.909156 secs 2023-09-10T10:20:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:20:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:20:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39636.261014 secs 2023-09-10T10:20:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39635.909156 secs 2023-09-10T10:20:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:20:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:20:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39636.261014 secs 2023-09-10T10:20:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39635.909156 secs 2023-09-10T10:20:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:20:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:20:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39636.261014 secs 2023-09-10T10:20:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:05 smithi098 ceph-mon[103124]: pgmap v21296: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 497 KiB/s wr, 14 op/s 2023-09-10T10:20:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:05 smithi170 ceph-mon[102832]: pgmap v21296: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 497 KiB/s wr, 14 op/s 2023-09-10T10:20:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:05 smithi082 ceph-mon[98694]: pgmap v21296: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 497 KiB/s wr, 14 op/s 2023-09-10T10:20:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:07 smithi098 ceph-mon[103124]: pgmap v21297: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 683 KiB/s wr, 35 op/s 2023-09-10T10:20:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:07 smithi170 ceph-mon[102832]: pgmap v21297: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 683 KiB/s wr, 35 op/s 2023-09-10T10:20:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:07 smithi082 ceph-mon[98694]: pgmap v21297: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 683 KiB/s wr, 35 op/s 2023-09-10T10:20:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39639.679300 secs 2023-09-10T10:20:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39639.679300 secs 2023-09-10T10:20:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39639.679300 secs 2023-09-10T10:20:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:09 smithi098 ceph-mon[103124]: pgmap v21298: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 460 KiB/s wr, 28 op/s 2023-09-10T10:20:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39640.909236 secs 2023-09-10T10:20:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:09 smithi098 ceph-mon[103124]: osdmap e1399: 12 total, 12 up, 12 in 2023-09-10T10:20:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:09 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:20:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:09 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39641.261081 secs 2023-09-10T10:20:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:09 smithi170 ceph-mon[102832]: pgmap v21298: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 460 KiB/s wr, 28 op/s 2023-09-10T10:20:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39640.909236 secs 2023-09-10T10:20:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:09 smithi170 ceph-mon[102832]: osdmap e1399: 12 total, 12 up, 12 in 2023-09-10T10:20:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:09 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:20:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:09 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39641.261081 secs 2023-09-10T10:20:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:09 smithi082 ceph-mon[98694]: pgmap v21298: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 460 KiB/s wr, 28 op/s 2023-09-10T10:20:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39640.909236 secs 2023-09-10T10:20:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:09 smithi082 ceph-mon[98694]: osdmap e1399: 12 total, 12 up, 12 in 2023-09-10T10:20:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:09 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:20:10.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:09 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:10.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39641.261081 secs 2023-09-10T10:20:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:11 smithi098 ceph-mon[103124]: pgmap v21300: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 552 KiB/s wr, 34 op/s 2023-09-10T10:20:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:11 smithi170 ceph-mon[102832]: pgmap v21300: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 552 KiB/s wr, 34 op/s 2023-09-10T10:20:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:11 smithi082 ceph-mon[98694]: pgmap v21300: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 552 KiB/s wr, 34 op/s 2023-09-10T10:20:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:20:13.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:20:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:20:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:14 smithi098 ceph-mon[103124]: pgmap v21301: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 473 KiB/s wr, 32 op/s 2023-09-10T10:20:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39644.679360 secs 2023-09-10T10:20:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:14 smithi170 ceph-mon[102832]: pgmap v21301: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 473 KiB/s wr, 32 op/s 2023-09-10T10:20:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39644.679360 secs 2023-09-10T10:20:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:14 smithi082 ceph-mon[98694]: pgmap v21301: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 473 KiB/s wr, 32 op/s 2023-09-10T10:20:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39644.679360 secs 2023-09-10T10:20:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39645.909323 secs 2023-09-10T10:20:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39646.261144 secs 2023-09-10T10:20:15.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:15 smithi170 ceph-mon[102832]: osdmap e1400: 12 total, 12 up, 12 in 2023-09-10T10:20:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39645.909323 secs 2023-09-10T10:20:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39646.261144 secs 2023-09-10T10:20:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:15 smithi082 ceph-mon[98694]: osdmap e1400: 12 total, 12 up, 12 in 2023-09-10T10:20:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39645.909323 secs 2023-09-10T10:20:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39646.261144 secs 2023-09-10T10:20:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:15 smithi098 ceph-mon[103124]: osdmap e1400: 12 total, 12 up, 12 in 2023-09-10T10:20:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:16 smithi170 ceph-mon[102832]: pgmap v21302: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 473 KiB/s wr, 32 op/s 2023-09-10T10:20:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:16 smithi082 ceph-mon[98694]: pgmap v21302: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 473 KiB/s wr, 32 op/s 2023-09-10T10:20:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:16 smithi098 ceph-mon[103124]: pgmap v21302: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 473 KiB/s wr, 32 op/s 2023-09-10T10:20:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:18 smithi170 ceph-mon[102832]: pgmap v21304: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 488 KiB/s wr, 17 op/s 2023-09-10T10:20:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:18 smithi082 ceph-mon[98694]: pgmap v21304: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 488 KiB/s wr, 17 op/s 2023-09-10T10:20:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:18 smithi098 ceph-mon[103124]: pgmap v21304: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 488 KiB/s wr, 17 op/s 2023-09-10T10:20:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39649.679417 secs 2023-09-10T10:20:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39649.679417 secs 2023-09-10T10:20:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39649.679417 secs 2023-09-10T10:20:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:20 smithi170 ceph-mon[102832]: pgmap v21305: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 427 KiB/s wr, 15 op/s 2023-09-10T10:20:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39650.909413 secs 2023-09-10T10:20:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39651.261211 secs 2023-09-10T10:20:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:20 smithi082 ceph-mon[98694]: pgmap v21305: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 427 KiB/s wr, 15 op/s 2023-09-10T10:20:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39650.909413 secs 2023-09-10T10:20:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39651.261211 secs 2023-09-10T10:20:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:20 smithi098 ceph-mon[103124]: pgmap v21305: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 427 KiB/s wr, 15 op/s 2023-09-10T10:20:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39650.909413 secs 2023-09-10T10:20:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39651.261211 secs 2023-09-10T10:20:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:22 smithi170 ceph-mon[102832]: pgmap v21306: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 390 KiB/s wr, 14 op/s 2023-09-10T10:20:22.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:22 smithi082 ceph-mon[98694]: pgmap v21306: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 390 KiB/s wr, 14 op/s 2023-09-10T10:20:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:22 smithi098 ceph-mon[103124]: pgmap v21306: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 390 KiB/s wr, 14 op/s 2023-09-10T10:20:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:24 smithi170 ceph-mon[102832]: pgmap v21307: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 13 op/s 2023-09-10T10:20:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39654.679505 secs 2023-09-10T10:20:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:24 smithi082 ceph-mon[98694]: pgmap v21307: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 13 op/s 2023-09-10T10:20:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39654.679505 secs 2023-09-10T10:20:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:24 smithi098 ceph-mon[103124]: pgmap v21307: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 13 op/s 2023-09-10T10:20:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39654.679505 secs 2023-09-10T10:20:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39655.909472 secs 2023-09-10T10:20:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39656.261277 secs 2023-09-10T10:20:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39655.909472 secs 2023-09-10T10:20:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39656.261277 secs 2023-09-10T10:20:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39655.909472 secs 2023-09-10T10:20:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39656.261277 secs 2023-09-10T10:20:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:26 smithi170 ceph-mon[102832]: pgmap v21308: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 13 op/s 2023-09-10T10:20:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:26 smithi082 ceph-mon[98694]: pgmap v21308: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 13 op/s 2023-09-10T10:20:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:26 smithi098 ceph-mon[103124]: pgmap v21308: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 141 KiB/s wr, 13 op/s 2023-09-10T10:20:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:28 smithi170 ceph-mon[102832]: pgmap v21309: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 17 op/s 2023-09-10T10:20:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:28 smithi082 ceph-mon[98694]: pgmap v21309: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 17 op/s 2023-09-10T10:20:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:28 smithi098 ceph-mon[103124]: pgmap v21309: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 283 KiB/s wr, 17 op/s 2023-09-10T10:20:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39659.679566 secs 2023-09-10T10:20:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39659.679566 secs 2023-09-10T10:20:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39659.679566 secs 2023-09-10T10:20:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:30 smithi170 ceph-mon[102832]: pgmap v21310: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:20:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39660.909490 secs 2023-09-10T10:20:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39661.261342 secs 2023-09-10T10:20:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:30 smithi082 ceph-mon[98694]: pgmap v21310: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:20:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39660.909490 secs 2023-09-10T10:20:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39661.261342 secs 2023-09-10T10:20:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:30 smithi098 ceph-mon[103124]: pgmap v21310: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:20:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39660.909490 secs 2023-09-10T10:20:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39661.261342 secs 2023-09-10T10:20:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:32 smithi170 ceph-mon[102832]: pgmap v21311: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:20:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:32 smithi082 ceph-mon[98694]: pgmap v21311: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:20:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:32 smithi098 ceph-mon[103124]: pgmap v21311: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:20:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:34 smithi170 ceph-mon[102832]: pgmap v21312: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 149 KiB/s wr, 15 op/s 2023-09-10T10:20:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39664.679627 secs 2023-09-10T10:20:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:34 smithi082 ceph-mon[98694]: pgmap v21312: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 149 KiB/s wr, 15 op/s 2023-09-10T10:20:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39664.679627 secs 2023-09-10T10:20:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:34 smithi098 ceph-mon[103124]: pgmap v21312: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 149 KiB/s wr, 15 op/s 2023-09-10T10:20:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39664.679627 secs 2023-09-10T10:20:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39665.909573 secs 2023-09-10T10:20:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39666.261407 secs 2023-09-10T10:20:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39665.909573 secs 2023-09-10T10:20:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39666.261407 secs 2023-09-10T10:20:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39665.909573 secs 2023-09-10T10:20:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39666.261407 secs 2023-09-10T10:20:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:36 smithi170 ceph-mon[102832]: pgmap v21313: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:20:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:36 smithi082 ceph-mon[98694]: pgmap v21313: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:20:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:36 smithi098 ceph-mon[103124]: pgmap v21313: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:20:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:20:37.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:20:37.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:37 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:20:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:20:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:20:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:37 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:20:37.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:20:37.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:20:37.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:37 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:20:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:38 smithi170 ceph-mon[102832]: pgmap v21314: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 16 op/s 2023-09-10T10:20:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:20:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:20:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:20:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:20:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:38 smithi082 ceph-mon[98694]: pgmap v21314: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 16 op/s 2023-09-10T10:20:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:20:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:20:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:20:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:20:38.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:38 smithi098 ceph-mon[103124]: pgmap v21314: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 293 KiB/s wr, 16 op/s 2023-09-10T10:20:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:20:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:20:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:20:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:20:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:39 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39669.679687 secs 2023-09-10T10:20:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:39 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39669.679687 secs 2023-09-10T10:20:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:39 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39669.679687 secs 2023-09-10T10:20:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:40 smithi170 ceph-mon[102832]: pgmap v21315: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:20:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39670.909631 secs 2023-09-10T10:20:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39671.261476 secs 2023-09-10T10:20:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:40 smithi082 ceph-mon[98694]: pgmap v21315: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:20:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39670.909631 secs 2023-09-10T10:20:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39671.261476 secs 2023-09-10T10:20:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:40 smithi098 ceph-mon[103124]: pgmap v21315: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:20:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39670.909631 secs 2023-09-10T10:20:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39671.261476 secs 2023-09-10T10:20:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:41 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:20:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:41 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:41 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:20:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:41 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:41 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:20:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:41 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:42 smithi170 ceph-mon[102832]: pgmap v21316: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:20:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:42 smithi082 ceph-mon[98694]: pgmap v21316: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:20:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:42 smithi098 ceph-mon[103124]: pgmap v21316: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:20:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:44 smithi170 ceph-mon[102832]: pgmap v21317: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 475 KiB/s wr, 17 op/s 2023-09-10T10:20:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:44 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:20:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:44 smithi082 ceph-mon[98694]: pgmap v21317: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 475 KiB/s wr, 17 op/s 2023-09-10T10:20:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:44 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:20:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:44 smithi098 ceph-mon[103124]: pgmap v21317: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 475 KiB/s wr, 17 op/s 2023-09-10T10:20:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:44 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:20:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39675.909711 secs 2023-09-10T10:20:45.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39676.261543 secs 2023-09-10T10:20:45.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:45 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:20:45.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:20:45.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39675.909711 secs 2023-09-10T10:20:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39676.261543 secs 2023-09-10T10:20:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:45 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:20:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:20:45.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39675.909711 secs 2023-09-10T10:20:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39676.261543 secs 2023-09-10T10:20:45.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:45 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:20:45.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:20:45.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:46 smithi170 ceph-mon[102832]: pgmap v21318: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 474 KiB/s wr, 12 op/s 2023-09-10T10:20:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:20:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:46 smithi082 ceph-mon[98694]: pgmap v21318: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 474 KiB/s wr, 12 op/s 2023-09-10T10:20:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:20:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:46 smithi098 ceph-mon[103124]: pgmap v21318: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 474 KiB/s wr, 12 op/s 2023-09-10T10:20:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:20:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:48 smithi170 ceph-mon[102832]: pgmap v21319: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 791 KiB/s wr, 19 op/s 2023-09-10T10:20:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:48 smithi082 ceph-mon[98694]: pgmap v21319: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 791 KiB/s wr, 19 op/s 2023-09-10T10:20:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:48 smithi098 ceph-mon[103124]: pgmap v21319: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 791 KiB/s wr, 19 op/s 2023-09-10T10:20:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:50 smithi170 ceph-mon[102832]: pgmap v21320: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 647 KiB/s wr, 13 op/s 2023-09-10T10:20:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39680.909765 secs 2023-09-10T10:20:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39681.261608 secs 2023-09-10T10:20:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:20:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:50 smithi082 ceph-mon[98694]: pgmap v21320: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 647 KiB/s wr, 13 op/s 2023-09-10T10:20:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39680.909765 secs 2023-09-10T10:20:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39681.261608 secs 2023-09-10T10:20:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:20:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:50 smithi098 ceph-mon[103124]: pgmap v21320: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 647 KiB/s wr, 13 op/s 2023-09-10T10:20:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39680.909765 secs 2023-09-10T10:20:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39681.261608 secs 2023-09-10T10:20:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:20:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:52 smithi170 ceph-mon[102832]: pgmap v21321: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 647 KiB/s wr, 13 op/s 2023-09-10T10:20:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:52 smithi082 ceph-mon[98694]: pgmap v21321: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 647 KiB/s wr, 13 op/s 2023-09-10T10:20:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:52 smithi098 ceph-mon[103124]: pgmap v21321: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 647 KiB/s wr, 13 op/s 2023-09-10T10:20:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:20:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:20:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:20:53.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:54 smithi170 ceph-mon[102832]: pgmap v21322: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 974 KiB/s wr, 21 op/s 2023-09-10T10:20:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:54 smithi082 ceph-mon[98694]: pgmap v21322: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 974 KiB/s wr, 21 op/s 2023-09-10T10:20:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:54 smithi098 ceph-mon[103124]: pgmap v21322: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 974 KiB/s wr, 21 op/s 2023-09-10T10:20:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39685.909824 secs 2023-09-10T10:20:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39686.261675 secs 2023-09-10T10:20:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39685.909824 secs 2023-09-10T10:20:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39686.261675 secs 2023-09-10T10:20:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39685.909824 secs 2023-09-10T10:20:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39686.261675 secs 2023-09-10T10:20:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:56 smithi170 ceph-mon[102832]: pgmap v21323: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 644 KiB/s wr, 14 op/s 2023-09-10T10:20:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:56 smithi082 ceph-mon[98694]: pgmap v21323: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 644 KiB/s wr, 14 op/s 2023-09-10T10:20:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:56 smithi098 ceph-mon[103124]: pgmap v21323: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 644 KiB/s wr, 14 op/s 2023-09-10T10:20:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:57 smithi170 ceph-mon[102832]: pgmap v21324: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 899 KiB/s wr, 21 op/s 2023-09-10T10:20:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:57 smithi082 ceph-mon[98694]: pgmap v21324: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 899 KiB/s wr, 21 op/s 2023-09-10T10:20:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:57 smithi098 ceph-mon[103124]: pgmap v21324: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 899 KiB/s wr, 21 op/s 2023-09-10T10:20:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:20:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:20:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:20:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:20:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:20:59 smithi082 ceph-mon[98694]: pgmap v21325: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 854 KiB/s rd, 582 KiB/s wr, 14 op/s 2023-09-10T10:20:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:20:59 smithi098 ceph-mon[103124]: pgmap v21325: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 854 KiB/s rd, 582 KiB/s wr, 14 op/s 2023-09-10T10:20:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:20:59 smithi170 ceph-mon[102832]: pgmap v21325: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 854 KiB/s rd, 582 KiB/s wr, 14 op/s 2023-09-10T10:21:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39690.909908 secs 2023-09-10T10:21:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39691.261739 secs 2023-09-10T10:21:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:21:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39690.909908 secs 2023-09-10T10:21:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39691.261739 secs 2023-09-10T10:21:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:21:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39690.909908 secs 2023-09-10T10:21:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39691.261739 secs 2023-09-10T10:21:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:21:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:01 smithi170 ceph-mon[102832]: pgmap v21326: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 854 KiB/s rd, 582 KiB/s wr, 14 op/s 2023-09-10T10:21:02.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:01 smithi082 ceph-mon[98694]: pgmap v21326: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 854 KiB/s rd, 582 KiB/s wr, 14 op/s 2023-09-10T10:21:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:01 smithi098 ceph-mon[103124]: pgmap v21326: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 854 KiB/s rd, 582 KiB/s wr, 14 op/s 2023-09-10T10:21:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:03 smithi170 ceph-mon[102832]: pgmap v21327: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 857 KiB/s wr, 21 op/s 2023-09-10T10:21:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39694.680008 secs 2023-09-10T10:21:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:03 smithi082 ceph-mon[98694]: pgmap v21327: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 857 KiB/s wr, 21 op/s 2023-09-10T10:21:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39694.680008 secs 2023-09-10T10:21:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:03 smithi098 ceph-mon[103124]: pgmap v21327: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 857 KiB/s wr, 21 op/s 2023-09-10T10:21:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39694.680008 secs 2023-09-10T10:21:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39695.909993 secs 2023-09-10T10:21:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:21:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:21:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39696.261804 secs 2023-09-10T10:21:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39695.909993 secs 2023-09-10T10:21:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:21:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:21:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39696.261804 secs 2023-09-10T10:21:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39695.909993 secs 2023-09-10T10:21:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:21:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:21:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39696.261804 secs 2023-09-10T10:21:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:05 smithi170 ceph-mon[102832]: pgmap v21328: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 529 KiB/s wr, 14 op/s 2023-09-10T10:21:06.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:05 smithi082 ceph-mon[98694]: pgmap v21328: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 529 KiB/s wr, 14 op/s 2023-09-10T10:21:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:05 smithi098 ceph-mon[103124]: pgmap v21328: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 529 KiB/s wr, 14 op/s 2023-09-10T10:21:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:07 smithi170 ceph-mon[102832]: pgmap v21329: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 716 KiB/s wr, 35 op/s 2023-09-10T10:21:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:07 smithi082 ceph-mon[98694]: pgmap v21329: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 716 KiB/s wr, 35 op/s 2023-09-10T10:21:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:07 smithi098 ceph-mon[103124]: pgmap v21329: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 716 KiB/s wr, 35 op/s 2023-09-10T10:21:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39699.680065 secs 2023-09-10T10:21:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39699.680065 secs 2023-09-10T10:21:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39699.680065 secs 2023-09-10T10:21:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:09 smithi170 ceph-mon[102832]: pgmap v21330: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 460 KiB/s wr, 28 op/s 2023-09-10T10:21:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39700.910085 secs 2023-09-10T10:21:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:09 smithi170 ceph-mon[102832]: osdmap e1401: 12 total, 12 up, 12 in 2023-09-10T10:21:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39701.261870 secs 2023-09-10T10:21:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:09 smithi082 ceph-mon[98694]: pgmap v21330: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 460 KiB/s wr, 28 op/s 2023-09-10T10:21:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39700.910085 secs 2023-09-10T10:21:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:09 smithi082 ceph-mon[98694]: osdmap e1401: 12 total, 12 up, 12 in 2023-09-10T10:21:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39701.261870 secs 2023-09-10T10:21:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:09 smithi098 ceph-mon[103124]: pgmap v21330: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 460 KiB/s wr, 28 op/s 2023-09-10T10:21:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39700.910085 secs 2023-09-10T10:21:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:09 smithi098 ceph-mon[103124]: osdmap e1401: 12 total, 12 up, 12 in 2023-09-10T10:21:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39701.261870 secs 2023-09-10T10:21:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:21:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:21:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:21:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:11 smithi170 ceph-mon[102832]: pgmap v21332: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 553 KiB/s wr, 33 op/s 2023-09-10T10:21:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:11 smithi082 ceph-mon[98694]: pgmap v21332: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 553 KiB/s wr, 33 op/s 2023-09-10T10:21:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:11 smithi098 ceph-mon[103124]: pgmap v21332: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 553 KiB/s wr, 33 op/s 2023-09-10T10:21:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:21:13.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:13.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:21:13.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:21:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:13 smithi170 ceph-mon[102832]: pgmap v21333: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 474 KiB/s wr, 33 op/s 2023-09-10T10:21:14.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39704.680136 secs 2023-09-10T10:21:14.212 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:13 smithi082 ceph-mon[98694]: pgmap v21333: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 474 KiB/s wr, 33 op/s 2023-09-10T10:21:14.212 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39704.680136 secs 2023-09-10T10:21:14.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:13 smithi098 ceph-mon[103124]: pgmap v21333: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 474 KiB/s wr, 33 op/s 2023-09-10T10:21:14.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39704.680136 secs 2023-09-10T10:21:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39705.910170 secs 2023-09-10T10:21:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39706.261934 secs 2023-09-10T10:21:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:14 smithi082 ceph-mon[98694]: osdmap e1402: 12 total, 12 up, 12 in 2023-09-10T10:21:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39705.910170 secs 2023-09-10T10:21:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39706.261934 secs 2023-09-10T10:21:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:14 smithi098 ceph-mon[103124]: osdmap e1402: 12 total, 12 up, 12 in 2023-09-10T10:21:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39705.910170 secs 2023-09-10T10:21:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39706.261934 secs 2023-09-10T10:21:15.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:14 smithi170 ceph-mon[102832]: osdmap e1402: 12 total, 12 up, 12 in 2023-09-10T10:21:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:15 smithi082 ceph-mon[98694]: pgmap v21334: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 474 KiB/s wr, 33 op/s 2023-09-10T10:21:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:15 smithi098 ceph-mon[103124]: pgmap v21334: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 474 KiB/s wr, 33 op/s 2023-09-10T10:21:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:15 smithi170 ceph-mon[102832]: pgmap v21334: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 474 KiB/s wr, 33 op/s 2023-09-10T10:21:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:17 smithi082 ceph-mon[98694]: pgmap v21336: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 533 KiB/s wr, 20 op/s 2023-09-10T10:21:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:17 smithi098 ceph-mon[103124]: pgmap v21336: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 533 KiB/s wr, 20 op/s 2023-09-10T10:21:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:17 smithi170 ceph-mon[102832]: pgmap v21336: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 533 KiB/s wr, 20 op/s 2023-09-10T10:21:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39709.680195 secs 2023-09-10T10:21:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39709.680195 secs 2023-09-10T10:21:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39709.680195 secs 2023-09-10T10:21:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:19 smithi098 ceph-mon[103124]: pgmap v21337: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 461 KiB/s wr, 17 op/s 2023-09-10T10:21:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39710.910227 secs 2023-09-10T10:21:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39711.262001 secs 2023-09-10T10:21:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:19 smithi170 ceph-mon[102832]: pgmap v21337: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 461 KiB/s wr, 17 op/s 2023-09-10T10:21:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39710.910227 secs 2023-09-10T10:21:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39711.262001 secs 2023-09-10T10:21:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:19 smithi082 ceph-mon[98694]: pgmap v21337: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 461 KiB/s wr, 17 op/s 2023-09-10T10:21:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39710.910227 secs 2023-09-10T10:21:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39711.262001 secs 2023-09-10T10:21:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:21 smithi098 ceph-mon[103124]: pgmap v21338: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 426 KiB/s wr, 16 op/s 2023-09-10T10:21:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:21 smithi170 ceph-mon[102832]: pgmap v21338: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 426 KiB/s wr, 16 op/s 2023-09-10T10:21:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:21 smithi082 ceph-mon[98694]: pgmap v21338: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 426 KiB/s wr, 16 op/s 2023-09-10T10:21:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:23 smithi098 ceph-mon[103124]: pgmap v21339: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 177 KiB/s wr, 13 op/s 2023-09-10T10:21:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39714.680255 secs 2023-09-10T10:21:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:23 smithi170 ceph-mon[102832]: pgmap v21339: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 177 KiB/s wr, 13 op/s 2023-09-10T10:21:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39714.680255 secs 2023-09-10T10:21:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:23 smithi082 ceph-mon[98694]: pgmap v21339: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 177 KiB/s wr, 13 op/s 2023-09-10T10:21:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39714.680255 secs 2023-09-10T10:21:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39715.910313 secs 2023-09-10T10:21:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39716.262066 secs 2023-09-10T10:21:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39715.910313 secs 2023-09-10T10:21:25.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39716.262066 secs 2023-09-10T10:21:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39715.910313 secs 2023-09-10T10:21:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39716.262066 secs 2023-09-10T10:21:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:25 smithi098 ceph-mon[103124]: pgmap v21340: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 177 KiB/s wr, 13 op/s 2023-09-10T10:21:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:25 smithi170 ceph-mon[102832]: pgmap v21340: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 177 KiB/s wr, 13 op/s 2023-09-10T10:21:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:25 smithi082 ceph-mon[98694]: pgmap v21340: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 177 KiB/s wr, 13 op/s 2023-09-10T10:21:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:27 smithi098 ceph-mon[103124]: pgmap v21341: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 315 KiB/s wr, 18 op/s 2023-09-10T10:21:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:27 smithi170 ceph-mon[102832]: pgmap v21341: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 315 KiB/s wr, 18 op/s 2023-09-10T10:21:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:27 smithi082 ceph-mon[98694]: pgmap v21341: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 315 KiB/s wr, 18 op/s 2023-09-10T10:21:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39719.680362 secs 2023-09-10T10:21:29.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39719.680362 secs 2023-09-10T10:21:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39719.680362 secs 2023-09-10T10:21:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:29 smithi098 ceph-mon[103124]: pgmap v21342: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 10 op/s 2023-09-10T10:21:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39720.910370 secs 2023-09-10T10:21:30.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39721.262131 secs 2023-09-10T10:21:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:29 smithi170 ceph-mon[102832]: pgmap v21342: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 10 op/s 2023-09-10T10:21:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39720.910370 secs 2023-09-10T10:21:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39721.262131 secs 2023-09-10T10:21:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:29 smithi082 ceph-mon[98694]: pgmap v21342: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 10 op/s 2023-09-10T10:21:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39720.910370 secs 2023-09-10T10:21:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39721.262131 secs 2023-09-10T10:21:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:31 smithi098 ceph-mon[103124]: pgmap v21343: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 10 op/s 2023-09-10T10:21:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:31 smithi170 ceph-mon[102832]: pgmap v21343: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 10 op/s 2023-09-10T10:21:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:31 smithi082 ceph-mon[98694]: pgmap v21343: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 10 op/s 2023-09-10T10:21:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:33 smithi098 ceph-mon[103124]: pgmap v21344: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 149 KiB/s wr, 16 op/s 2023-09-10T10:21:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39724.680422 secs 2023-09-10T10:21:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:21:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:33 smithi170 ceph-mon[102832]: pgmap v21344: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 149 KiB/s wr, 16 op/s 2023-09-10T10:21:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39724.680422 secs 2023-09-10T10:21:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:21:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:33 smithi082 ceph-mon[98694]: pgmap v21344: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 149 KiB/s wr, 16 op/s 2023-09-10T10:21:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39724.680422 secs 2023-09-10T10:21:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:21:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39725.910458 secs 2023-09-10T10:21:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39726.262197 secs 2023-09-10T10:21:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39725.910458 secs 2023-09-10T10:21:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39726.262197 secs 2023-09-10T10:21:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39725.910458 secs 2023-09-10T10:21:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39726.262197 secs 2023-09-10T10:21:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:35 smithi098 ceph-mon[103124]: pgmap v21345: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:21:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:35 smithi170 ceph-mon[102832]: pgmap v21345: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:21:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:35 smithi082 ceph-mon[98694]: pgmap v21345: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:21:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:21:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:21:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:21:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:38 smithi170 ceph-mon[102832]: pgmap v21346: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 293 KiB/s wr, 16 op/s 2023-09-10T10:21:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:21:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:21:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:21:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:21:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:21:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:38 smithi082 ceph-mon[98694]: pgmap v21346: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 293 KiB/s wr, 16 op/s 2023-09-10T10:21:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:21:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:21:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:21:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:21:38.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:21:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:38 smithi098 ceph-mon[103124]: pgmap v21346: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 293 KiB/s wr, 16 op/s 2023-09-10T10:21:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:21:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:21:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:21:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:21:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:21:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:39 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39729.680493 secs 2023-09-10T10:21:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:21:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:21:39.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:21:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:39 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39729.680493 secs 2023-09-10T10:21:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:21:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:21:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:21:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:39 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39729.680493 secs 2023-09-10T10:21:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:21:39.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:21:39.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:21:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:40 smithi170 ceph-mon[102832]: pgmap v21347: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 10 op/s 2023-09-10T10:21:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39730.910526 secs 2023-09-10T10:21:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39731.262263 secs 2023-09-10T10:21:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:40 smithi082 ceph-mon[98694]: pgmap v21347: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 10 op/s 2023-09-10T10:21:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39730.910526 secs 2023-09-10T10:21:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39731.262263 secs 2023-09-10T10:21:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:40 smithi098 ceph-mon[103124]: pgmap v21347: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 10 op/s 2023-09-10T10:21:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39730.910526 secs 2023-09-10T10:21:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39731.262263 secs 2023-09-10T10:21:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:41 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:21:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:41 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:41 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:21:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:41 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:41 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:21:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:41 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:42 smithi170 ceph-mon[102832]: pgmap v21348: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 10 op/s 2023-09-10T10:21:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:42 smithi082 ceph-mon[98694]: pgmap v21348: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 10 op/s 2023-09-10T10:21:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:42 smithi098 ceph-mon[103124]: pgmap v21348: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 10 op/s 2023-09-10T10:21:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:44 smithi170 ceph-mon[102832]: pgmap v21349: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 479 KiB/s wr, 17 op/s 2023-09-10T10:21:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:44 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:21:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:44 smithi082 ceph-mon[98694]: pgmap v21349: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 479 KiB/s wr, 17 op/s 2023-09-10T10:21:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:44 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:21:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:44 smithi098 ceph-mon[103124]: pgmap v21349: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 479 KiB/s wr, 17 op/s 2023-09-10T10:21:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:44 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:21:45.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39735.910587 secs 2023-09-10T10:21:45.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39736.262329 secs 2023-09-10T10:21:45.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:45 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:21:45.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:21:45.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39735.910587 secs 2023-09-10T10:21:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39736.262329 secs 2023-09-10T10:21:45.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:45 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:21:45.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:21:45.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39735.910587 secs 2023-09-10T10:21:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39736.262329 secs 2023-09-10T10:21:45.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:45 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:21:45.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:21:45.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:46 smithi170 ceph-mon[102832]: pgmap v21350: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 479 KiB/s wr, 11 op/s 2023-09-10T10:21:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:21:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:46 smithi082 ceph-mon[98694]: pgmap v21350: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 479 KiB/s wr, 11 op/s 2023-09-10T10:21:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:21:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:46 smithi098 ceph-mon[103124]: pgmap v21350: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 479 KiB/s wr, 11 op/s 2023-09-10T10:21:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:21:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:48 smithi170 ceph-mon[102832]: pgmap v21351: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 796 KiB/s wr, 19 op/s 2023-09-10T10:21:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:48 smithi082 ceph-mon[98694]: pgmap v21351: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 796 KiB/s wr, 19 op/s 2023-09-10T10:21:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:48 smithi098 ceph-mon[103124]: pgmap v21351: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 796 KiB/s wr, 19 op/s 2023-09-10T10:21:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:50 smithi170 ceph-mon[102832]: pgmap v21352: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 652 KiB/s wr, 13 op/s 2023-09-10T10:21:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39740.910670 secs 2023-09-10T10:21:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39741.262394 secs 2023-09-10T10:21:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:21:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:50 smithi082 ceph-mon[98694]: pgmap v21352: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 652 KiB/s wr, 13 op/s 2023-09-10T10:21:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39740.910670 secs 2023-09-10T10:21:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39741.262394 secs 2023-09-10T10:21:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:21:50.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:50 smithi098 ceph-mon[103124]: pgmap v21352: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 652 KiB/s wr, 13 op/s 2023-09-10T10:21:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39740.910670 secs 2023-09-10T10:21:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39741.262394 secs 2023-09-10T10:21:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:21:50.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:52 smithi170 ceph-mon[102832]: pgmap v21353: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 652 KiB/s wr, 13 op/s 2023-09-10T10:21:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:52 smithi082 ceph-mon[98694]: pgmap v21353: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 652 KiB/s wr, 13 op/s 2023-09-10T10:21:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:52 smithi098 ceph-mon[103124]: pgmap v21353: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 652 KiB/s wr, 13 op/s 2023-09-10T10:21:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:21:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:21:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:21:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:54 smithi170 ceph-mon[102832]: pgmap v21354: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 941 KiB/s wr, 21 op/s 2023-09-10T10:21:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:54 smithi082 ceph-mon[98694]: pgmap v21354: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 941 KiB/s wr, 21 op/s 2023-09-10T10:21:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:54 smithi098 ceph-mon[103124]: pgmap v21354: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 941 KiB/s wr, 21 op/s 2023-09-10T10:21:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39745.910732 secs 2023-09-10T10:21:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39746.262463 secs 2023-09-10T10:21:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39745.910732 secs 2023-09-10T10:21:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39746.262463 secs 2023-09-10T10:21:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39745.910732 secs 2023-09-10T10:21:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39746.262463 secs 2023-09-10T10:21:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:56 smithi170 ceph-mon[102832]: pgmap v21355: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 607 KiB/s wr, 14 op/s 2023-09-10T10:21:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:56 smithi082 ceph-mon[98694]: pgmap v21355: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 607 KiB/s wr, 14 op/s 2023-09-10T10:21:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:56 smithi098 ceph-mon[103124]: pgmap v21355: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 607 KiB/s wr, 14 op/s 2023-09-10T10:21:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:58 smithi170 ceph-mon[102832]: pgmap v21356: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 902 KiB/s wr, 21 op/s 2023-09-10T10:21:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:21:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:21:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:58 smithi082 ceph-mon[98694]: pgmap v21356: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 902 KiB/s wr, 21 op/s 2023-09-10T10:21:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:21:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:21:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:21:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:58 smithi098 ceph-mon[103124]: pgmap v21356: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 902 KiB/s wr, 21 op/s 2023-09-10T10:21:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:21:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:21:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:00 smithi170 ceph-mon[102832]: pgmap v21357: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 584 KiB/s wr, 13 op/s 2023-09-10T10:22:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39750.910791 secs 2023-09-10T10:22:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39751.262487 secs 2023-09-10T10:22:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:00 smithi082 ceph-mon[98694]: pgmap v21357: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 584 KiB/s wr, 13 op/s 2023-09-10T10:22:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39750.910791 secs 2023-09-10T10:22:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39751.262487 secs 2023-09-10T10:22:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:00 smithi098 ceph-mon[103124]: pgmap v21357: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 584 KiB/s wr, 13 op/s 2023-09-10T10:22:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39750.910791 secs 2023-09-10T10:22:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39751.262487 secs 2023-09-10T10:22:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:01 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:22:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:01 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:01 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:22:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:01 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:01 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:22:01.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:01 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:02 smithi170 ceph-mon[102832]: pgmap v21358: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 584 KiB/s wr, 13 op/s 2023-09-10T10:22:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:02 smithi082 ceph-mon[98694]: pgmap v21358: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 584 KiB/s wr, 13 op/s 2023-09-10T10:22:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:02 smithi098 ceph-mon[103124]: pgmap v21358: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 584 KiB/s wr, 13 op/s 2023-09-10T10:22:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:04 smithi170 ceph-mon[102832]: pgmap v21359: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 859 KiB/s wr, 21 op/s 2023-09-10T10:22:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39754.680879 secs 2023-09-10T10:22:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:04 smithi082 ceph-mon[98694]: pgmap v21359: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 859 KiB/s wr, 21 op/s 2023-09-10T10:22:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39754.680879 secs 2023-09-10T10:22:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:04 smithi098 ceph-mon[103124]: pgmap v21359: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 859 KiB/s wr, 21 op/s 2023-09-10T10:22:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39754.680879 secs 2023-09-10T10:22:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39755.910873 secs 2023-09-10T10:22:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39756.262555 secs 2023-09-10T10:22:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:05 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:22:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:22:05.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39755.910873 secs 2023-09-10T10:22:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39756.262555 secs 2023-09-10T10:22:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:05 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:22:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:22:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39755.910873 secs 2023-09-10T10:22:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39756.262555 secs 2023-09-10T10:22:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:05 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:22:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:22:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:06 smithi170 ceph-mon[102832]: pgmap v21360: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 570 KiB/s wr, 14 op/s 2023-09-10T10:22:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:06 smithi082 ceph-mon[98694]: pgmap v21360: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 570 KiB/s wr, 14 op/s 2023-09-10T10:22:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:06 smithi098 ceph-mon[103124]: pgmap v21360: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 570 KiB/s wr, 14 op/s 2023-09-10T10:22:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:08 smithi170 ceph-mon[102832]: pgmap v21361: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 756 KiB/s wr, 34 op/s 2023-09-10T10:22:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:08 smithi082 ceph-mon[98694]: pgmap v21361: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 756 KiB/s wr, 34 op/s 2023-09-10T10:22:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:08 smithi098 ceph-mon[103124]: pgmap v21361: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 756 KiB/s wr, 34 op/s 2023-09-10T10:22:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39759.680936 secs 2023-09-10T10:22:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:09 smithi170 ceph-mon[102832]: osdmap e1403: 12 total, 12 up, 12 in 2023-09-10T10:22:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39759.680936 secs 2023-09-10T10:22:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:09 smithi082 ceph-mon[98694]: osdmap e1403: 12 total, 12 up, 12 in 2023-09-10T10:22:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39759.680936 secs 2023-09-10T10:22:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:09 smithi098 ceph-mon[103124]: osdmap e1403: 12 total, 12 up, 12 in 2023-09-10T10:22:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:10 smithi170 ceph-mon[102832]: pgmap v21363: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 554 KiB/s wr, 34 op/s 2023-09-10T10:22:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39760.910954 secs 2023-09-10T10:22:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39761.262623 secs 2023-09-10T10:22:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:22:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:10 smithi082 ceph-mon[98694]: pgmap v21363: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 554 KiB/s wr, 34 op/s 2023-09-10T10:22:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39760.910954 secs 2023-09-10T10:22:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39761.262623 secs 2023-09-10T10:22:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:22:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:10 smithi098 ceph-mon[103124]: pgmap v21363: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 554 KiB/s wr, 34 op/s 2023-09-10T10:22:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39760.910954 secs 2023-09-10T10:22:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39761.262623 secs 2023-09-10T10:22:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:22:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:12 smithi170 ceph-mon[102832]: pgmap v21364: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 554 KiB/s wr, 34 op/s 2023-09-10T10:22:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:12 smithi082 ceph-mon[98694]: pgmap v21364: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 554 KiB/s wr, 34 op/s 2023-09-10T10:22:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:12 smithi098 ceph-mon[103124]: pgmap v21364: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 554 KiB/s wr, 34 op/s 2023-09-10T10:22:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:22:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:22:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:22:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:14 smithi170 ceph-mon[102832]: pgmap v21365: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 469 KiB/s wr, 31 op/s 2023-09-10T10:22:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39764.681006 secs 2023-09-10T10:22:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:14 smithi082 ceph-mon[98694]: pgmap v21365: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 469 KiB/s wr, 31 op/s 2023-09-10T10:22:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39764.681006 secs 2023-09-10T10:22:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:14 smithi098 ceph-mon[103124]: pgmap v21365: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 469 KiB/s wr, 31 op/s 2023-09-10T10:22:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39764.681006 secs 2023-09-10T10:22:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39765.911012 secs 2023-09-10T10:22:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39766.262686 secs 2023-09-10T10:22:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:15 smithi170 ceph-mon[102832]: pgmap v21366: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 469 KiB/s wr, 31 op/s 2023-09-10T10:22:15.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:15 smithi170 ceph-mon[102832]: osdmap e1404: 12 total, 12 up, 12 in 2023-09-10T10:22:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39765.911012 secs 2023-09-10T10:22:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39766.262686 secs 2023-09-10T10:22:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:15 smithi082 ceph-mon[98694]: pgmap v21366: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 469 KiB/s wr, 31 op/s 2023-09-10T10:22:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:15 smithi082 ceph-mon[98694]: osdmap e1404: 12 total, 12 up, 12 in 2023-09-10T10:22:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39765.911012 secs 2023-09-10T10:22:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39766.262686 secs 2023-09-10T10:22:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:15 smithi098 ceph-mon[103124]: pgmap v21366: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 469 KiB/s wr, 31 op/s 2023-09-10T10:22:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:15 smithi098 ceph-mon[103124]: osdmap e1404: 12 total, 12 up, 12 in 2023-09-10T10:22:17.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:17 smithi098 ceph-mon[103124]: pgmap v21368: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 525 KiB/s wr, 18 op/s 2023-09-10T10:22:17.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:17 smithi170 ceph-mon[102832]: pgmap v21368: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 525 KiB/s wr, 18 op/s 2023-09-10T10:22:17.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:17 smithi082 ceph-mon[98694]: pgmap v21368: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 525 KiB/s wr, 18 op/s 2023-09-10T10:22:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39769.681127 secs 2023-09-10T10:22:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:19 smithi098 ceph-mon[103124]: pgmap v21369: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 422 KiB/s wr, 14 op/s 2023-09-10T10:22:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39769.681127 secs 2023-09-10T10:22:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:19 smithi170 ceph-mon[102832]: pgmap v21369: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 422 KiB/s wr, 14 op/s 2023-09-10T10:22:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39769.681127 secs 2023-09-10T10:22:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:19 smithi082 ceph-mon[98694]: pgmap v21369: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 422 KiB/s wr, 14 op/s 2023-09-10T10:22:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39770.911069 secs 2023-09-10T10:22:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39771.262751 secs 2023-09-10T10:22:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39770.911069 secs 2023-09-10T10:22:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39771.262751 secs 2023-09-10T10:22:20.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39770.911069 secs 2023-09-10T10:22:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39771.262751 secs 2023-09-10T10:22:22.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:21 smithi170 ceph-mon[102832]: pgmap v21370: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 422 KiB/s wr, 14 op/s 2023-09-10T10:22:22.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:21 smithi082 ceph-mon[98694]: pgmap v21370: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 422 KiB/s wr, 14 op/s 2023-09-10T10:22:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:21 smithi098 ceph-mon[103124]: pgmap v21370: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 422 KiB/s wr, 14 op/s 2023-09-10T10:22:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:23 smithi170 ceph-mon[102832]: pgmap v21371: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 13 op/s 2023-09-10T10:22:24.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39774.681224 secs 2023-09-10T10:22:24.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:23 smithi082 ceph-mon[98694]: pgmap v21371: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 13 op/s 2023-09-10T10:22:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39774.681224 secs 2023-09-10T10:22:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:23 smithi098 ceph-mon[103124]: pgmap v21371: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 13 op/s 2023-09-10T10:22:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39774.681224 secs 2023-09-10T10:22:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39775.911128 secs 2023-09-10T10:22:25.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39776.262817 secs 2023-09-10T10:22:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39775.911128 secs 2023-09-10T10:22:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39776.262817 secs 2023-09-10T10:22:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39775.911128 secs 2023-09-10T10:22:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39776.262817 secs 2023-09-10T10:22:26.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:25 smithi170 ceph-mon[102832]: pgmap v21372: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 13 op/s 2023-09-10T10:22:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:25 smithi082 ceph-mon[98694]: pgmap v21372: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 13 op/s 2023-09-10T10:22:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:25 smithi098 ceph-mon[103124]: pgmap v21372: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 177 KiB/s wr, 13 op/s 2023-09-10T10:22:28.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:27 smithi170 ceph-mon[102832]: pgmap v21373: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 313 KiB/s wr, 18 op/s 2023-09-10T10:22:28.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:27 smithi082 ceph-mon[98694]: pgmap v21373: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 313 KiB/s wr, 18 op/s 2023-09-10T10:22:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:27 smithi098 ceph-mon[103124]: pgmap v21373: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 313 KiB/s wr, 18 op/s 2023-09-10T10:22:29.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39779.681283 secs 2023-09-10T10:22:29.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39779.681283 secs 2023-09-10T10:22:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39779.681283 secs 2023-09-10T10:22:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:29 smithi170 ceph-mon[102832]: pgmap v21374: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:22:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39780.911212 secs 2023-09-10T10:22:30.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39781.262902 secs 2023-09-10T10:22:30.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:29 smithi082 ceph-mon[98694]: pgmap v21374: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:22:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39780.911212 secs 2023-09-10T10:22:30.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39781.262902 secs 2023-09-10T10:22:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:29 smithi098 ceph-mon[103124]: pgmap v21374: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:22:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39780.911212 secs 2023-09-10T10:22:30.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39781.262902 secs 2023-09-10T10:22:32.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:31 smithi170 ceph-mon[102832]: pgmap v21375: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:22:32.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:31 smithi082 ceph-mon[98694]: pgmap v21375: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:22:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:31 smithi098 ceph-mon[103124]: pgmap v21375: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:22:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:33 smithi170 ceph-mon[102832]: pgmap v21376: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T10:22:34.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39784.681344 secs 2023-09-10T10:22:34.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:33 smithi082 ceph-mon[98694]: pgmap v21376: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T10:22:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39784.681344 secs 2023-09-10T10:22:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:33 smithi098 ceph-mon[103124]: pgmap v21376: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 147 KiB/s wr, 16 op/s 2023-09-10T10:22:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39784.681344 secs 2023-09-10T10:22:35.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39785.911269 secs 2023-09-10T10:22:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:22:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39786.262966 secs 2023-09-10T10:22:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39785.911269 secs 2023-09-10T10:22:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:22:35.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:35.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39786.262966 secs 2023-09-10T10:22:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39785.911269 secs 2023-09-10T10:22:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:22:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:35.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39786.262966 secs 2023-09-10T10:22:36.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:35 smithi082 ceph-mon[98694]: pgmap v21377: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:22:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:35 smithi098 ceph-mon[103124]: pgmap v21377: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:22:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:35 smithi170 ceph-mon[102832]: pgmap v21377: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:22:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:22:37.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:22:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:22:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:37 smithi082 ceph-mon[98694]: pgmap v21378: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 427 KiB/s wr, 17 op/s 2023-09-10T10:22:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:22:38.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:37 smithi098 ceph-mon[103124]: pgmap v21378: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 427 KiB/s wr, 17 op/s 2023-09-10T10:22:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:22:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:37 smithi170 ceph-mon[102832]: pgmap v21378: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 427 KiB/s wr, 17 op/s 2023-09-10T10:22:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:22:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:39.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:38 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:22:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:38 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:22:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:38 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:22:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:39 smithi082 ceph-mon[98694]: pgmap v21379: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 280 KiB/s wr, 11 op/s 2023-09-10T10:22:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39790.911327 secs 2023-09-10T10:22:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39791.263032 secs 2023-09-10T10:22:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:22:40.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:22:40.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:22:40.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:22:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:39 smithi098 ceph-mon[103124]: pgmap v21379: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 280 KiB/s wr, 11 op/s 2023-09-10T10:22:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39790.911327 secs 2023-09-10T10:22:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39791.263032 secs 2023-09-10T10:22:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:22:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:22:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:22:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:22:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:39 smithi170 ceph-mon[102832]: pgmap v21379: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 280 KiB/s wr, 11 op/s 2023-09-10T10:22:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39790.911327 secs 2023-09-10T10:22:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39791.263032 secs 2023-09-10T10:22:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:22:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:22:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:22:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:22:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:22:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:22:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:22:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:22:41.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:22:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:22:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:42 smithi170 ceph-mon[102832]: pgmap v21380: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 280 KiB/s wr, 11 op/s 2023-09-10T10:22:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:42 smithi082 ceph-mon[98694]: pgmap v21380: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 280 KiB/s wr, 11 op/s 2023-09-10T10:22:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:42 smithi098 ceph-mon[103124]: pgmap v21380: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 280 KiB/s wr, 11 op/s 2023-09-10T10:22:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:44 smithi170 ceph-mon[102832]: pgmap v21381: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 463 KiB/s wr, 18 op/s 2023-09-10T10:22:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:44 smithi082 ceph-mon[98694]: pgmap v21381: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 463 KiB/s wr, 18 op/s 2023-09-10T10:22:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:44 smithi098 ceph-mon[103124]: pgmap v21381: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 463 KiB/s wr, 18 op/s 2023-09-10T10:22:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39795.911388 secs 2023-09-10T10:22:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39796.263095 secs 2023-09-10T10:22:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39795.911388 secs 2023-09-10T10:22:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39796.263095 secs 2023-09-10T10:22:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39795.911388 secs 2023-09-10T10:22:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39796.263095 secs 2023-09-10T10:22:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:46 smithi170 ceph-mon[102832]: pgmap v21382: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 463 KiB/s wr, 12 op/s 2023-09-10T10:22:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:46 smithi082 ceph-mon[98694]: pgmap v21382: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 463 KiB/s wr, 12 op/s 2023-09-10T10:22:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:46 smithi098 ceph-mon[103124]: pgmap v21382: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 463 KiB/s wr, 12 op/s 2023-09-10T10:22:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:48 smithi170 ceph-mon[102832]: pgmap v21383: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 975 KiB/s wr, 20 op/s 2023-09-10T10:22:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:48 smithi082 ceph-mon[98694]: pgmap v21383: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 975 KiB/s wr, 20 op/s 2023-09-10T10:22:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:48 smithi098 ceph-mon[103124]: pgmap v21383: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 975 KiB/s wr, 20 op/s 2023-09-10T10:22:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:50 smithi170 ceph-mon[102832]: pgmap v21384: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 695 KiB/s wr, 14 op/s 2023-09-10T10:22:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39800.911448 secs 2023-09-10T10:22:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39801.263158 secs 2023-09-10T10:22:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:22:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:50 smithi082 ceph-mon[98694]: pgmap v21384: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 695 KiB/s wr, 14 op/s 2023-09-10T10:22:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39800.911448 secs 2023-09-10T10:22:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39801.263158 secs 2023-09-10T10:22:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:22:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:50 smithi098 ceph-mon[103124]: pgmap v21384: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 695 KiB/s wr, 14 op/s 2023-09-10T10:22:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39800.911448 secs 2023-09-10T10:22:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39801.263158 secs 2023-09-10T10:22:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:22:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:52 smithi170 ceph-mon[102832]: pgmap v21385: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 695 KiB/s wr, 14 op/s 2023-09-10T10:22:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:52 smithi082 ceph-mon[98694]: pgmap v21385: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 695 KiB/s wr, 14 op/s 2023-09-10T10:22:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:52 smithi098 ceph-mon[103124]: pgmap v21385: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 695 KiB/s wr, 14 op/s 2023-09-10T10:22:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:22:53.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:53.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:53 smithi170 ceph-mon[102832]: pgmap v21386: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 928 KiB/s wr, 22 op/s 2023-09-10T10:22:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:22:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:53 smithi082 ceph-mon[98694]: pgmap v21386: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 928 KiB/s wr, 22 op/s 2023-09-10T10:22:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:22:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:53.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:53 smithi098 ceph-mon[103124]: pgmap v21386: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 928 KiB/s wr, 22 op/s 2023-09-10T10:22:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39805.911505 secs 2023-09-10T10:22:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39806.263222 secs 2023-09-10T10:22:55.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:55 smithi098 ceph-mon[103124]: pgmap v21387: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 746 KiB/s wr, 15 op/s 2023-09-10T10:22:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39805.911505 secs 2023-09-10T10:22:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39806.263222 secs 2023-09-10T10:22:55.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:55 smithi170 ceph-mon[102832]: pgmap v21387: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 746 KiB/s wr, 15 op/s 2023-09-10T10:22:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39805.911505 secs 2023-09-10T10:22:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39806.263222 secs 2023-09-10T10:22:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:55 smithi082 ceph-mon[98694]: pgmap v21387: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 746 KiB/s wr, 15 op/s 2023-09-10T10:22:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:57 smithi098 ceph-mon[103124]: pgmap v21388: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T10:22:57.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:57 smithi170 ceph-mon[102832]: pgmap v21388: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T10:22:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:57 smithi082 ceph-mon[98694]: pgmap v21388: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 22 op/s 2023-09-10T10:22:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:22:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:22:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:22:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:22:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39809.681751 secs 2023-09-10T10:22:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:22:59 smithi098 ceph-mon[103124]: pgmap v21389: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 571 KiB/s wr, 15 op/s 2023-09-10T10:22:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39809.681751 secs 2023-09-10T10:22:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:22:59 smithi170 ceph-mon[102832]: pgmap v21389: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 571 KiB/s wr, 15 op/s 2023-09-10T10:22:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39809.681751 secs 2023-09-10T10:22:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:22:59 smithi082 ceph-mon[98694]: pgmap v21389: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 571 KiB/s wr, 15 op/s 2023-09-10T10:23:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39810.911564 secs 2023-09-10T10:23:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39811.263288 secs 2023-09-10T10:23:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:23:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:23:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39810.911564 secs 2023-09-10T10:23:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39811.263288 secs 2023-09-10T10:23:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:23:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:23:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39810.911564 secs 2023-09-10T10:23:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39811.263288 secs 2023-09-10T10:23:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:23:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:23:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:02.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:01 smithi170 ceph-mon[102832]: pgmap v21390: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 572 KiB/s wr, 15 op/s 2023-09-10T10:23:02.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:01 smithi082 ceph-mon[98694]: pgmap v21390: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 572 KiB/s wr, 15 op/s 2023-09-10T10:23:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:01 smithi098 ceph-mon[103124]: pgmap v21390: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 572 KiB/s wr, 15 op/s 2023-09-10T10:23:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:03 smithi170 ceph-mon[102832]: pgmap v21391: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 746 KiB/s wr, 21 op/s 2023-09-10T10:23:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39814.681775 secs 2023-09-10T10:23:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:03 smithi082 ceph-mon[98694]: pgmap v21391: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 746 KiB/s wr, 21 op/s 2023-09-10T10:23:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39814.681775 secs 2023-09-10T10:23:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:03 smithi098 ceph-mon[103124]: pgmap v21391: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 746 KiB/s wr, 21 op/s 2023-09-10T10:23:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39814.681775 secs 2023-09-10T10:23:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39815.911623 secs 2023-09-10T10:23:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39816.263354 secs 2023-09-10T10:23:05.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39815.911623 secs 2023-09-10T10:23:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39816.263354 secs 2023-09-10T10:23:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39815.911623 secs 2023-09-10T10:23:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39816.263354 secs 2023-09-10T10:23:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:05 smithi170 ceph-mon[102832]: pgmap v21392: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 513 KiB/s wr, 13 op/s 2023-09-10T10:23:06.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:05 smithi082 ceph-mon[98694]: pgmap v21392: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 513 KiB/s wr, 13 op/s 2023-09-10T10:23:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:05 smithi098 ceph-mon[103124]: pgmap v21392: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 513 KiB/s wr, 13 op/s 2023-09-10T10:23:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:07 smithi170 ceph-mon[102832]: pgmap v21393: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 724 KiB/s wr, 33 op/s 2023-09-10T10:23:08.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:07 smithi082 ceph-mon[98694]: pgmap v21393: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 724 KiB/s wr, 33 op/s 2023-09-10T10:23:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:07 smithi098 ceph-mon[103124]: pgmap v21393: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 724 KiB/s wr, 33 op/s 2023-09-10T10:23:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39819.681898 secs 2023-09-10T10:23:09.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39819.681898 secs 2023-09-10T10:23:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39819.681898 secs 2023-09-10T10:23:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:09 smithi170 ceph-mon[102832]: pgmap v21394: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 386 KiB/s wr, 26 op/s 2023-09-10T10:23:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39820.911683 secs 2023-09-10T10:23:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:09 smithi170 ceph-mon[102832]: osdmap e1405: 12 total, 12 up, 12 in 2023-09-10T10:23:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39821.263422 secs 2023-09-10T10:23:10.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:09 smithi082 ceph-mon[98694]: pgmap v21394: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 386 KiB/s wr, 26 op/s 2023-09-10T10:23:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39820.911683 secs 2023-09-10T10:23:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:09 smithi082 ceph-mon[98694]: osdmap e1405: 12 total, 12 up, 12 in 2023-09-10T10:23:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39821.263422 secs 2023-09-10T10:23:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:09 smithi098 ceph-mon[103124]: pgmap v21394: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 386 KiB/s wr, 26 op/s 2023-09-10T10:23:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39820.911683 secs 2023-09-10T10:23:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:09 smithi098 ceph-mon[103124]: osdmap e1405: 12 total, 12 up, 12 in 2023-09-10T10:23:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39821.263422 secs 2023-09-10T10:23:12.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:11 smithi170 ceph-mon[102832]: pgmap v21396: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 463 KiB/s wr, 31 op/s 2023-09-10T10:23:12.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:11 smithi082 ceph-mon[98694]: pgmap v21396: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 463 KiB/s wr, 31 op/s 2023-09-10T10:23:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:11 smithi098 ceph-mon[103124]: pgmap v21396: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 463 KiB/s wr, 31 op/s 2023-09-10T10:23:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:13 smithi170 ceph-mon[102832]: pgmap v21397: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 254 KiB/s wr, 31 op/s 2023-09-10T10:23:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39824.681958 secs 2023-09-10T10:23:14.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:13 smithi082 ceph-mon[98694]: pgmap v21397: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 254 KiB/s wr, 31 op/s 2023-09-10T10:23:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39824.681958 secs 2023-09-10T10:23:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:13 smithi098 ceph-mon[103124]: pgmap v21397: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 254 KiB/s wr, 31 op/s 2023-09-10T10:23:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39824.681958 secs 2023-09-10T10:23:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39825.911742 secs 2023-09-10T10:23:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39826.263489 secs 2023-09-10T10:23:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:14 smithi170 ceph-mon[102832]: osdmap e1406: 12 total, 12 up, 12 in 2023-09-10T10:23:15.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39825.911742 secs 2023-09-10T10:23:15.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39826.263489 secs 2023-09-10T10:23:15.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:14 smithi082 ceph-mon[98694]: osdmap e1406: 12 total, 12 up, 12 in 2023-09-10T10:23:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39825.911742 secs 2023-09-10T10:23:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39826.263489 secs 2023-09-10T10:23:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:14 smithi098 ceph-mon[103124]: osdmap e1406: 12 total, 12 up, 12 in 2023-09-10T10:23:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:15 smithi082 ceph-mon[98694]: pgmap v21398: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 254 KiB/s wr, 31 op/s 2023-09-10T10:23:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:15 smithi098 ceph-mon[103124]: pgmap v21398: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 254 KiB/s wr, 31 op/s 2023-09-10T10:23:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:15 smithi170 ceph-mon[102832]: pgmap v21398: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 254 KiB/s wr, 31 op/s 2023-09-10T10:23:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:23:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:23:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:23:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:17 smithi082 ceph-mon[98694]: pgmap v21400: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 639 B/s wr, 16 op/s 2023-09-10T10:23:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:17 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:23:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:17 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:17 smithi098 ceph-mon[103124]: pgmap v21400: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 639 B/s wr, 16 op/s 2023-09-10T10:23:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:17 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:23:18.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:17 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:17 smithi170 ceph-mon[102832]: pgmap v21400: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 639 B/s wr, 16 op/s 2023-09-10T10:23:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:17 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:23:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:17 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39829.682019 secs 2023-09-10T10:23:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39829.682019 secs 2023-09-10T10:23:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39829.682019 secs 2023-09-10T10:23:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:19 smithi082 ceph-mon[98694]: pgmap v21401: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 552 B/s wr, 14 op/s 2023-09-10T10:23:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39830.911799 secs 2023-09-10T10:23:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39831.263556 secs 2023-09-10T10:23:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:19 smithi098 ceph-mon[103124]: pgmap v21401: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 552 B/s wr, 14 op/s 2023-09-10T10:23:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39830.911799 secs 2023-09-10T10:23:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39831.263556 secs 2023-09-10T10:23:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:19 smithi170 ceph-mon[102832]: pgmap v21401: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 552 B/s wr, 14 op/s 2023-09-10T10:23:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39830.911799 secs 2023-09-10T10:23:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39831.263556 secs 2023-09-10T10:23:22.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:21 smithi082 ceph-mon[98694]: pgmap v21402: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 13 op/s 2023-09-10T10:23:22.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:21 smithi098 ceph-mon[103124]: pgmap v21402: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 13 op/s 2023-09-10T10:23:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:21 smithi170 ceph-mon[102832]: pgmap v21402: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 13 op/s 2023-09-10T10:23:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:23 smithi082 ceph-mon[98694]: pgmap v21403: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 12 op/s 2023-09-10T10:23:24.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39834.682079 secs 2023-09-10T10:23:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:23 smithi098 ceph-mon[103124]: pgmap v21403: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 12 op/s 2023-09-10T10:23:24.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39834.682079 secs 2023-09-10T10:23:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:23 smithi170 ceph-mon[102832]: pgmap v21403: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 12 op/s 2023-09-10T10:23:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39834.682079 secs 2023-09-10T10:23:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39835.911868 secs 2023-09-10T10:23:25.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39836.263624 secs 2023-09-10T10:23:25.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39835.911868 secs 2023-09-10T10:23:25.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39836.263624 secs 2023-09-10T10:23:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39835.911868 secs 2023-09-10T10:23:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39836.263624 secs 2023-09-10T10:23:26.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:25 smithi082 ceph-mon[98694]: pgmap v21404: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 12 op/s 2023-09-10T10:23:26.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:25 smithi098 ceph-mon[103124]: pgmap v21404: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 12 op/s 2023-09-10T10:23:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:25 smithi170 ceph-mon[102832]: pgmap v21404: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 12 op/s 2023-09-10T10:23:28.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:27 smithi098 ceph-mon[103124]: pgmap v21405: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 306 KiB/s wr, 16 op/s 2023-09-10T10:23:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:27 smithi170 ceph-mon[102832]: pgmap v21405: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 306 KiB/s wr, 16 op/s 2023-09-10T10:23:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:27 smithi082 ceph-mon[98694]: pgmap v21405: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 306 KiB/s wr, 16 op/s 2023-09-10T10:23:29.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39839.682172 secs 2023-09-10T10:23:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39839.682172 secs 2023-09-10T10:23:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39839.682172 secs 2023-09-10T10:23:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:29 smithi098 ceph-mon[103124]: pgmap v21406: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 287 KiB/s wr, 10 op/s 2023-09-10T10:23:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39840.911952 secs 2023-09-10T10:23:30.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39841.263689 secs 2023-09-10T10:23:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:29 smithi170 ceph-mon[102832]: pgmap v21406: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 287 KiB/s wr, 10 op/s 2023-09-10T10:23:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39840.911952 secs 2023-09-10T10:23:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39841.263689 secs 2023-09-10T10:23:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:29 smithi082 ceph-mon[98694]: pgmap v21406: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 287 KiB/s wr, 10 op/s 2023-09-10T10:23:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39840.911952 secs 2023-09-10T10:23:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39841.263689 secs 2023-09-10T10:23:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:31 smithi098 ceph-mon[103124]: pgmap v21407: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 287 KiB/s wr, 10 op/s 2023-09-10T10:23:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:31 smithi170 ceph-mon[102832]: pgmap v21407: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 287 KiB/s wr, 10 op/s 2023-09-10T10:23:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:31 smithi082 ceph-mon[98694]: pgmap v21407: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 287 KiB/s wr, 10 op/s 2023-09-10T10:23:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:33 smithi098 ceph-mon[103124]: pgmap v21408: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 288 KiB/s wr, 15 op/s 2023-09-10T10:23:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:33 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:23:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:33 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:34.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39844.682230 secs 2023-09-10T10:23:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:33 smithi170 ceph-mon[102832]: pgmap v21408: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 288 KiB/s wr, 15 op/s 2023-09-10T10:23:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:33 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:23:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:33 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39844.682230 secs 2023-09-10T10:23:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:33 smithi082 ceph-mon[98694]: pgmap v21408: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 288 KiB/s wr, 15 op/s 2023-09-10T10:23:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:33 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:23:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:33 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39844.682230 secs 2023-09-10T10:23:35.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39845.912013 secs 2023-09-10T10:23:35.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39846.263751 secs 2023-09-10T10:23:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39845.912013 secs 2023-09-10T10:23:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39846.263751 secs 2023-09-10T10:23:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39845.912013 secs 2023-09-10T10:23:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39846.263751 secs 2023-09-10T10:23:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:35 smithi098 ceph-mon[103124]: pgmap v21409: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 287 KiB/s wr, 10 op/s 2023-09-10T10:23:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:35 smithi170 ceph-mon[102832]: pgmap v21409: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 287 KiB/s wr, 10 op/s 2023-09-10T10:23:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:35 smithi082 ceph-mon[98694]: pgmap v21409: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 287 KiB/s wr, 10 op/s 2023-09-10T10:23:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:23:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:23:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:23:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:37 smithi098 ceph-mon[103124]: pgmap v21410: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 596 KiB/s wr, 16 op/s 2023-09-10T10:23:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:37 smithi170 ceph-mon[102832]: pgmap v21410: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 596 KiB/s wr, 16 op/s 2023-09-10T10:23:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:37 smithi082 ceph-mon[98694]: pgmap v21410: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 596 KiB/s wr, 16 op/s 2023-09-10T10:23:39.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:38 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39849.682309 secs 2023-09-10T10:23:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:38 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39849.682309 secs 2023-09-10T10:23:39.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:38 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39849.682309 secs 2023-09-10T10:23:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:39 smithi098 ceph-mon[103124]: pgmap v21411: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 309 KiB/s wr, 11 op/s 2023-09-10T10:23:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39850.912094 secs 2023-09-10T10:23:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39851.263826 secs 2023-09-10T10:23:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:23:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:23:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:23:40.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:39 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:23:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:39 smithi170 ceph-mon[102832]: pgmap v21411: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 309 KiB/s wr, 11 op/s 2023-09-10T10:23:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39850.912094 secs 2023-09-10T10:23:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39851.263826 secs 2023-09-10T10:23:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:23:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:23:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:23:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:39 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:23:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:39 smithi082 ceph-mon[98694]: pgmap v21411: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 309 KiB/s wr, 11 op/s 2023-09-10T10:23:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39850.912094 secs 2023-09-10T10:23:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39851.263826 secs 2023-09-10T10:23:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:23:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:23:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:23:40.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:39 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:23:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:42 smithi098 ceph-mon[103124]: pgmap v21412: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 309 KiB/s wr, 11 op/s 2023-09-10T10:23:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:42 smithi170 ceph-mon[102832]: pgmap v21412: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 309 KiB/s wr, 11 op/s 2023-09-10T10:23:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:42 smithi082 ceph-mon[98694]: pgmap v21412: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 309 KiB/s wr, 11 op/s 2023-09-10T10:23:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:44 smithi170 ceph-mon[102832]: pgmap v21413: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 432 KiB/s wr, 16 op/s 2023-09-10T10:23:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:44 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39854.682371 secs 2023-09-10T10:23:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:44 smithi082 ceph-mon[98694]: pgmap v21413: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 432 KiB/s wr, 16 op/s 2023-09-10T10:23:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:44 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39854.682371 secs 2023-09-10T10:23:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:44 smithi098 ceph-mon[103124]: pgmap v21413: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 432 KiB/s wr, 16 op/s 2023-09-10T10:23:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:44 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39854.682371 secs 2023-09-10T10:23:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39855.912156 secs 2023-09-10T10:23:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39856.263888 secs 2023-09-10T10:23:45.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:23:45.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39855.912156 secs 2023-09-10T10:23:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39856.263888 secs 2023-09-10T10:23:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:23:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39855.912156 secs 2023-09-10T10:23:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39856.263888 secs 2023-09-10T10:23:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:23:45.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:46 smithi170 ceph-mon[102832]: pgmap v21414: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 431 KiB/s wr, 11 op/s 2023-09-10T10:23:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:23:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:46 smithi082 ceph-mon[98694]: pgmap v21414: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 431 KiB/s wr, 11 op/s 2023-09-10T10:23:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:23:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:46 smithi098 ceph-mon[103124]: pgmap v21414: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 431 KiB/s wr, 11 op/s 2023-09-10T10:23:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:23:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:48 smithi170 ceph-mon[102832]: pgmap v21415: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 629 KiB/s wr, 17 op/s 2023-09-10T10:23:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:48 smithi082 ceph-mon[98694]: pgmap v21415: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 629 KiB/s wr, 17 op/s 2023-09-10T10:23:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:48 smithi098 ceph-mon[103124]: pgmap v21415: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 629 KiB/s wr, 17 op/s 2023-09-10T10:23:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:49 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39859.682429 secs 2023-09-10T10:23:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:49 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39859.682429 secs 2023-09-10T10:23:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:49 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39859.682429 secs 2023-09-10T10:23:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:50 smithi170 ceph-mon[102832]: pgmap v21416: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 321 KiB/s wr, 11 op/s 2023-09-10T10:23:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39860.912214 secs 2023-09-10T10:23:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39861.263953 secs 2023-09-10T10:23:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:50 smithi082 ceph-mon[98694]: pgmap v21416: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 321 KiB/s wr, 11 op/s 2023-09-10T10:23:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39860.912214 secs 2023-09-10T10:23:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39861.263953 secs 2023-09-10T10:23:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:50 smithi098 ceph-mon[103124]: pgmap v21416: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 321 KiB/s wr, 11 op/s 2023-09-10T10:23:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39860.912214 secs 2023-09-10T10:23:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39861.263953 secs 2023-09-10T10:23:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:52 smithi170 ceph-mon[102832]: pgmap v21417: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 321 KiB/s wr, 11 op/s 2023-09-10T10:23:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:52 smithi082 ceph-mon[98694]: pgmap v21417: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 321 KiB/s wr, 11 op/s 2023-09-10T10:23:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:52 smithi098 ceph-mon[103124]: pgmap v21417: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 321 KiB/s wr, 11 op/s 2023-09-10T10:23:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:54 smithi170 ceph-mon[102832]: pgmap v21418: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 543 KiB/s wr, 19 op/s 2023-09-10T10:23:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:23:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:54 smithi082 ceph-mon[98694]: pgmap v21418: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 543 KiB/s wr, 19 op/s 2023-09-10T10:23:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:23:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:54 smithi098 ceph-mon[103124]: pgmap v21418: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 543 KiB/s wr, 19 op/s 2023-09-10T10:23:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:23:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39865.912300 secs 2023-09-10T10:23:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39866.264017 secs 2023-09-10T10:23:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39865.912300 secs 2023-09-10T10:23:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39866.264017 secs 2023-09-10T10:23:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39865.912300 secs 2023-09-10T10:23:55.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39866.264017 secs 2023-09-10T10:23:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:56 smithi170 ceph-mon[102832]: pgmap v21419: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 420 KiB/s wr, 13 op/s 2023-09-10T10:23:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:56 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:23:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:56 smithi082 ceph-mon[98694]: pgmap v21419: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 420 KiB/s wr, 13 op/s 2023-09-10T10:23:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:56 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:23:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:56 smithi098 ceph-mon[103124]: pgmap v21419: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 420 KiB/s wr, 13 op/s 2023-09-10T10:23:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:56 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:23:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:57 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:23:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:23:57.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:57.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:57 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:23:57.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:23:57.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:57 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:23:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:23:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:58 smithi170 ceph-mon[102832]: pgmap v21420: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 773 KiB/s wr, 20 op/s 2023-09-10T10:23:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:23:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:23:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:58 smithi082 ceph-mon[98694]: pgmap v21420: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 773 KiB/s wr, 20 op/s 2023-09-10T10:23:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:23:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:23:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:23:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:58 smithi098 ceph-mon[103124]: pgmap v21420: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 773 KiB/s wr, 20 op/s 2023-09-10T10:23:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:23:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:23:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:00 smithi170 ceph-mon[102832]: pgmap v21421: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 576 KiB/s wr, 15 op/s 2023-09-10T10:24:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39870.912368 secs 2023-09-10T10:24:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39871.264104 secs 2023-09-10T10:24:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:00 smithi082 ceph-mon[98694]: pgmap v21421: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 576 KiB/s wr, 15 op/s 2023-09-10T10:24:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39870.912368 secs 2023-09-10T10:24:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39871.264104 secs 2023-09-10T10:24:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:00 smithi098 ceph-mon[103124]: pgmap v21421: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 576 KiB/s wr, 15 op/s 2023-09-10T10:24:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39870.912368 secs 2023-09-10T10:24:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39871.264104 secs 2023-09-10T10:24:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:01 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:24:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:01 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:01 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:24:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:01 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:01 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:24:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:01 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:02 smithi170 ceph-mon[102832]: pgmap v21422: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 576 KiB/s wr, 15 op/s 2023-09-10T10:24:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:02 smithi082 ceph-mon[98694]: pgmap v21422: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 576 KiB/s wr, 15 op/s 2023-09-10T10:24:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:02 smithi098 ceph-mon[103124]: pgmap v21422: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 576 KiB/s wr, 15 op/s 2023-09-10T10:24:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:04 smithi170 ceph-mon[102832]: pgmap v21423: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 719 KiB/s wr, 21 op/s 2023-09-10T10:24:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:04 smithi082 ceph-mon[98694]: pgmap v21423: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 719 KiB/s wr, 21 op/s 2023-09-10T10:24:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:04 smithi098 ceph-mon[103124]: pgmap v21423: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 719 KiB/s wr, 21 op/s 2023-09-10T10:24:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39875.912424 secs 2023-09-10T10:24:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39876.264169 secs 2023-09-10T10:24:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:24:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39875.912424 secs 2023-09-10T10:24:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39876.264169 secs 2023-09-10T10:24:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:24:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39875.912424 secs 2023-09-10T10:24:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39876.264169 secs 2023-09-10T10:24:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:24:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:06 smithi170 ceph-mon[102832]: pgmap v21424: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 496 KiB/s wr, 14 op/s 2023-09-10T10:24:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:24:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:06 smithi082 ceph-mon[98694]: pgmap v21424: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 496 KiB/s wr, 14 op/s 2023-09-10T10:24:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:24:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:06 smithi098 ceph-mon[103124]: pgmap v21424: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 496 KiB/s wr, 14 op/s 2023-09-10T10:24:06.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:24:06.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:08 smithi170 ceph-mon[102832]: pgmap v21425: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 991 KiB/s wr, 36 op/s 2023-09-10T10:24:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:08 smithi082 ceph-mon[98694]: pgmap v21425: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 991 KiB/s wr, 36 op/s 2023-09-10T10:24:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:08 smithi098 ceph-mon[103124]: pgmap v21425: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 991 KiB/s wr, 36 op/s 2023-09-10T10:24:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39879.682713 secs 2023-09-10T10:24:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:09 smithi170 ceph-mon[102832]: osdmap e1407: 12 total, 12 up, 12 in 2023-09-10T10:24:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39879.682713 secs 2023-09-10T10:24:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:09 smithi082 ceph-mon[98694]: osdmap e1407: 12 total, 12 up, 12 in 2023-09-10T10:24:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39879.682713 secs 2023-09-10T10:24:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:09 smithi098 ceph-mon[103124]: osdmap e1407: 12 total, 12 up, 12 in 2023-09-10T10:24:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:10 smithi170 ceph-mon[102832]: pgmap v21427: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 765 KiB/s wr, 34 op/s 2023-09-10T10:24:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39880.912500 secs 2023-09-10T10:24:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39881.264236 secs 2023-09-10T10:24:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:10 smithi082 ceph-mon[98694]: pgmap v21427: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 765 KiB/s wr, 34 op/s 2023-09-10T10:24:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39880.912500 secs 2023-09-10T10:24:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39881.264236 secs 2023-09-10T10:24:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:10 smithi098 ceph-mon[103124]: pgmap v21427: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 765 KiB/s wr, 34 op/s 2023-09-10T10:24:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39880.912500 secs 2023-09-10T10:24:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39881.264236 secs 2023-09-10T10:24:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:12 smithi170 ceph-mon[102832]: pgmap v21428: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 765 KiB/s wr, 34 op/s 2023-09-10T10:24:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:12 smithi082 ceph-mon[98694]: pgmap v21428: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 765 KiB/s wr, 34 op/s 2023-09-10T10:24:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:12 smithi098 ceph-mon[103124]: pgmap v21428: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 765 KiB/s wr, 34 op/s 2023-09-10T10:24:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:13 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:24:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:24:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:13 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:24:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:24:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:13 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:24:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:24:13.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:14 smithi170 ceph-mon[102832]: pgmap v21429: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 994 KiB/s wr, 34 op/s 2023-09-10T10:24:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39884.682777 secs 2023-09-10T10:24:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:14 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:24:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:14 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:14 smithi082 ceph-mon[98694]: pgmap v21429: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 994 KiB/s wr, 34 op/s 2023-09-10T10:24:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39884.682777 secs 2023-09-10T10:24:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:14 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:24:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:14 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:14 smithi098 ceph-mon[103124]: pgmap v21429: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 994 KiB/s wr, 34 op/s 2023-09-10T10:24:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39884.682777 secs 2023-09-10T10:24:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:14 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:24:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:14 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39885.912560 secs 2023-09-10T10:24:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39886.264301 secs 2023-09-10T10:24:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:15 smithi170 ceph-mon[102832]: osdmap e1408: 12 total, 12 up, 12 in 2023-09-10T10:24:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39885.912560 secs 2023-09-10T10:24:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39886.264301 secs 2023-09-10T10:24:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:15 smithi082 ceph-mon[98694]: osdmap e1408: 12 total, 12 up, 12 in 2023-09-10T10:24:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39885.912560 secs 2023-09-10T10:24:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39886.264301 secs 2023-09-10T10:24:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:15 smithi098 ceph-mon[103124]: osdmap e1408: 12 total, 12 up, 12 in 2023-09-10T10:24:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:16 smithi170 ceph-mon[102832]: pgmap v21430: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 994 KiB/s wr, 34 op/s 2023-09-10T10:24:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:16 smithi082 ceph-mon[98694]: pgmap v21430: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 994 KiB/s wr, 34 op/s 2023-09-10T10:24:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:16 smithi098 ceph-mon[103124]: pgmap v21430: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 994 KiB/s wr, 34 op/s 2023-09-10T10:24:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:17 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:24:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:17 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:17.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:17 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:24:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:17 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:17 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:24:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:17 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:18 smithi170 ceph-mon[102832]: pgmap v21432: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 965 KiB/s wr, 19 op/s 2023-09-10T10:24:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:18 smithi082 ceph-mon[98694]: pgmap v21432: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 965 KiB/s wr, 19 op/s 2023-09-10T10:24:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:18 smithi098 ceph-mon[103124]: pgmap v21432: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 965 KiB/s wr, 19 op/s 2023-09-10T10:24:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39889.682850 secs 2023-09-10T10:24:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39889.682850 secs 2023-09-10T10:24:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39889.682850 secs 2023-09-10T10:24:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:20 smithi170 ceph-mon[102832]: pgmap v21433: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 781 KiB/s wr, 15 op/s 2023-09-10T10:24:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39890.912615 secs 2023-09-10T10:24:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39891.264369 secs 2023-09-10T10:24:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:20 smithi082 ceph-mon[98694]: pgmap v21433: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 781 KiB/s wr, 15 op/s 2023-09-10T10:24:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39890.912615 secs 2023-09-10T10:24:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39891.264369 secs 2023-09-10T10:24:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:20 smithi098 ceph-mon[103124]: pgmap v21433: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 781 KiB/s wr, 15 op/s 2023-09-10T10:24:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39890.912615 secs 2023-09-10T10:24:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39891.264369 secs 2023-09-10T10:24:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:22 smithi170 ceph-mon[102832]: pgmap v21434: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 781 KiB/s wr, 15 op/s 2023-09-10T10:24:22.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:22 smithi082 ceph-mon[98694]: pgmap v21434: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 781 KiB/s wr, 15 op/s 2023-09-10T10:24:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:22 smithi098 ceph-mon[103124]: pgmap v21434: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 781 KiB/s wr, 15 op/s 2023-09-10T10:24:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:24 smithi170 ceph-mon[102832]: pgmap v21435: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 558 KiB/s wr, 15 op/s 2023-09-10T10:24:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39894.682940 secs 2023-09-10T10:24:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:24 smithi082 ceph-mon[98694]: pgmap v21435: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 558 KiB/s wr, 15 op/s 2023-09-10T10:24:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39894.682940 secs 2023-09-10T10:24:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:24 smithi098 ceph-mon[103124]: pgmap v21435: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 558 KiB/s wr, 15 op/s 2023-09-10T10:24:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39894.682940 secs 2023-09-10T10:24:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39895.912687 secs 2023-09-10T10:24:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39896.264433 secs 2023-09-10T10:24:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39895.912687 secs 2023-09-10T10:24:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39896.264433 secs 2023-09-10T10:24:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39895.912687 secs 2023-09-10T10:24:25.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39896.264433 secs 2023-09-10T10:24:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:26 smithi170 ceph-mon[102832]: pgmap v21436: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 558 KiB/s wr, 15 op/s 2023-09-10T10:24:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:26 smithi082 ceph-mon[98694]: pgmap v21436: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 558 KiB/s wr, 15 op/s 2023-09-10T10:24:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:26 smithi098 ceph-mon[103124]: pgmap v21436: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 558 KiB/s wr, 15 op/s 2023-09-10T10:24:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:27 smithi098 ceph-mon[103124]: pgmap v21437: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 495 KiB/s wr, 19 op/s 2023-09-10T10:24:27.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:27 smithi170 ceph-mon[102832]: pgmap v21437: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 495 KiB/s wr, 19 op/s 2023-09-10T10:24:27.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:27 smithi082 ceph-mon[98694]: pgmap v21437: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 495 KiB/s wr, 19 op/s 2023-09-10T10:24:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39899.683000 secs 2023-09-10T10:24:29.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:29 smithi098 ceph-mon[103124]: pgmap v21438: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T10:24:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39899.683000 secs 2023-09-10T10:24:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:29 smithi170 ceph-mon[102832]: pgmap v21438: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T10:24:29.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39899.683000 secs 2023-09-10T10:24:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:29 smithi082 ceph-mon[98694]: pgmap v21438: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T10:24:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39900.912742 secs 2023-09-10T10:24:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39901.264492 secs 2023-09-10T10:24:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:24:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39900.912742 secs 2023-09-10T10:24:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39901.264492 secs 2023-09-10T10:24:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:24:30.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:30.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39900.912742 secs 2023-09-10T10:24:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39901.264492 secs 2023-09-10T10:24:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:24:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:32.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:31 smithi082 ceph-mon[98694]: pgmap v21439: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T10:24:32.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:31 smithi098 ceph-mon[103124]: pgmap v21439: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T10:24:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:31 smithi170 ceph-mon[102832]: pgmap v21439: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T10:24:33.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:24:33.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:24:33.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:24:33.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:34.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:33 smithi082 ceph-mon[98694]: pgmap v21440: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 366 KiB/s wr, 17 op/s 2023-09-10T10:24:34.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39904.683060 secs 2023-09-10T10:24:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:33 smithi098 ceph-mon[103124]: pgmap v21440: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 366 KiB/s wr, 17 op/s 2023-09-10T10:24:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39904.683060 secs 2023-09-10T10:24:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:33 smithi170 ceph-mon[102832]: pgmap v21440: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 366 KiB/s wr, 17 op/s 2023-09-10T10:24:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39904.683060 secs 2023-09-10T10:24:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39905.912828 secs 2023-09-10T10:24:35.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39906.264555 secs 2023-09-10T10:24:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39905.912828 secs 2023-09-10T10:24:35.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39906.264555 secs 2023-09-10T10:24:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39905.912828 secs 2023-09-10T10:24:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39906.264555 secs 2023-09-10T10:24:36.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:35 smithi082 ceph-mon[98694]: pgmap v21441: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 218 KiB/s wr, 11 op/s 2023-09-10T10:24:36.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:35 smithi098 ceph-mon[103124]: pgmap v21441: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 218 KiB/s wr, 11 op/s 2023-09-10T10:24:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:35 smithi170 ceph-mon[102832]: pgmap v21441: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 218 KiB/s wr, 11 op/s 2023-09-10T10:24:38.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:37 smithi082 ceph-mon[98694]: pgmap v21442: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 322 KiB/s wr, 16 op/s 2023-09-10T10:24:38.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:24:38.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:37 smithi098 ceph-mon[103124]: pgmap v21442: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 322 KiB/s wr, 16 op/s 2023-09-10T10:24:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:24:38.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:37 smithi170 ceph-mon[102832]: pgmap v21442: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 322 KiB/s wr, 16 op/s 2023-09-10T10:24:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:24:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:40.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:39 smithi082 ceph-mon[98694]: pgmap v21443: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 322 KiB/s wr, 11 op/s 2023-09-10T10:24:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39910.912911 secs 2023-09-10T10:24:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39911.264622 secs 2023-09-10T10:24:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:24:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:39 smithi098 ceph-mon[103124]: pgmap v21443: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 322 KiB/s wr, 11 op/s 2023-09-10T10:24:40.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39910.912911 secs 2023-09-10T10:24:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39911.264622 secs 2023-09-10T10:24:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:24:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:39 smithi170 ceph-mon[102832]: pgmap v21443: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 322 KiB/s wr, 11 op/s 2023-09-10T10:24:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39910.912911 secs 2023-09-10T10:24:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39911.264622 secs 2023-09-10T10:24:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:24:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:24:41.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:24:41.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:41.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:24:41.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:24:41.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:24:41.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:40 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:24:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:24:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:24:41.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:41.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:24:41.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:24:41.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:24:41.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:40 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:24:41.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:24:41.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:24:41.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:41.462 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:24:41.462 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:24:41.462 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:24:41.462 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:40 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:24:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:41 smithi098 ceph-mon[103124]: pgmap v21444: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 322 KiB/s wr, 11 op/s 2023-09-10T10:24:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:41 smithi170 ceph-mon[102832]: pgmap v21444: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 322 KiB/s wr, 11 op/s 2023-09-10T10:24:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:41 smithi082 ceph-mon[98694]: pgmap v21444: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 322 KiB/s wr, 11 op/s 2023-09-10T10:24:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:43 smithi098 ceph-mon[103124]: pgmap v21445: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 535 KiB/s wr, 17 op/s 2023-09-10T10:24:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:43 smithi170 ceph-mon[102832]: pgmap v21445: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 535 KiB/s wr, 17 op/s 2023-09-10T10:24:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:43 smithi082 ceph-mon[98694]: pgmap v21445: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 535 KiB/s wr, 17 op/s 2023-09-10T10:24:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39915.912968 secs 2023-09-10T10:24:45.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39916.264687 secs 2023-09-10T10:24:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39915.912968 secs 2023-09-10T10:24:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39916.264687 secs 2023-09-10T10:24:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39915.912968 secs 2023-09-10T10:24:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39916.264687 secs 2023-09-10T10:24:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:45 smithi098 ceph-mon[103124]: pgmap v21446: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 318 KiB/s wr, 11 op/s 2023-09-10T10:24:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:45 smithi170 ceph-mon[102832]: pgmap v21446: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 318 KiB/s wr, 11 op/s 2023-09-10T10:24:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:45 smithi082 ceph-mon[98694]: pgmap v21446: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 318 KiB/s wr, 11 op/s 2023-09-10T10:24:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:47 smithi098 ceph-mon[103124]: pgmap v21447: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 632 KiB/s wr, 18 op/s 2023-09-10T10:24:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:47 smithi170 ceph-mon[102832]: pgmap v21447: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 632 KiB/s wr, 18 op/s 2023-09-10T10:24:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:47 smithi082 ceph-mon[98694]: pgmap v21447: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 632 KiB/s wr, 18 op/s 2023-09-10T10:24:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:48 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39919.683280 secs 2023-09-10T10:24:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:48 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39919.683280 secs 2023-09-10T10:24:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:48 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39919.683280 secs 2023-09-10T10:24:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:49 smithi098 ceph-mon[103124]: pgmap v21448: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 528 KiB/s wr, 13 op/s 2023-09-10T10:24:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39920.913026 secs 2023-09-10T10:24:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:49 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:24:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:49 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:50.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39921.264751 secs 2023-09-10T10:24:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:49 smithi170 ceph-mon[102832]: pgmap v21448: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 528 KiB/s wr, 13 op/s 2023-09-10T10:24:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39920.913026 secs 2023-09-10T10:24:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:49 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:24:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:49 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39921.264751 secs 2023-09-10T10:24:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:49 smithi082 ceph-mon[98694]: pgmap v21448: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 528 KiB/s wr, 13 op/s 2023-09-10T10:24:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39920.913026 secs 2023-09-10T10:24:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:49 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:24:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:49 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39921.264751 secs 2023-09-10T10:24:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:51 smithi098 ceph-mon[103124]: pgmap v21449: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 528 KiB/s wr, 13 op/s 2023-09-10T10:24:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:51 smithi170 ceph-mon[102832]: pgmap v21449: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 528 KiB/s wr, 13 op/s 2023-09-10T10:24:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:51 smithi082 ceph-mon[98694]: pgmap v21449: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 528 KiB/s wr, 13 op/s 2023-09-10T10:24:53.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:52 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:24:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:24:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:52 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:24:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:24:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:52 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:24:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:24:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:54 smithi098 ceph-mon[103124]: pgmap v21450: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 775 KiB/s wr, 20 op/s 2023-09-10T10:24:54.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39924.683383 secs 2023-09-10T10:24:54.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:24:54.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:54 smithi170 ceph-mon[102832]: pgmap v21450: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 775 KiB/s wr, 20 op/s 2023-09-10T10:24:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39924.683383 secs 2023-09-10T10:24:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:24:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:54 smithi082 ceph-mon[98694]: pgmap v21450: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 775 KiB/s wr, 20 op/s 2023-09-10T10:24:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39924.683383 secs 2023-09-10T10:24:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:24:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39925.913085 secs 2023-09-10T10:24:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39926.264815 secs 2023-09-10T10:24:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39925.913085 secs 2023-09-10T10:24:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39926.264815 secs 2023-09-10T10:24:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39925.913085 secs 2023-09-10T10:24:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39926.264815 secs 2023-09-10T10:24:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:56 smithi170 ceph-mon[102832]: pgmap v21451: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 561 KiB/s wr, 14 op/s 2023-09-10T10:24:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:56 smithi082 ceph-mon[98694]: pgmap v21451: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 561 KiB/s wr, 14 op/s 2023-09-10T10:24:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:56 smithi098 ceph-mon[103124]: pgmap v21451: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 561 KiB/s wr, 14 op/s 2023-09-10T10:24:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:24:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:57.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:24:57.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:24:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:58 smithi170 ceph-mon[102832]: pgmap v21452: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T10:24:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:24:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:58 smithi082 ceph-mon[98694]: pgmap v21452: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T10:24:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:24:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:58 smithi098 ceph-mon[103124]: pgmap v21452: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T10:24:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:24:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:24:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:24:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39929.683440 secs 2023-09-10T10:24:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:24:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39929.683440 secs 2023-09-10T10:24:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:24:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39929.683440 secs 2023-09-10T10:25:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:00 smithi170 ceph-mon[102832]: pgmap v21453: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 716 KiB/s wr, 14 op/s 2023-09-10T10:25:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39930.913142 secs 2023-09-10T10:25:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39931.264881 secs 2023-09-10T10:25:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:00 smithi082 ceph-mon[98694]: pgmap v21453: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 716 KiB/s wr, 14 op/s 2023-09-10T10:25:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39930.913142 secs 2023-09-10T10:25:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39931.264881 secs 2023-09-10T10:25:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:00 smithi098 ceph-mon[103124]: pgmap v21453: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 716 KiB/s wr, 14 op/s 2023-09-10T10:25:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39930.913142 secs 2023-09-10T10:25:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39931.264881 secs 2023-09-10T10:25:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:01 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:25:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:01 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:01 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:25:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:01 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:01 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:25:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:01 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:02 smithi170 ceph-mon[102832]: pgmap v21454: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 716 KiB/s wr, 14 op/s 2023-09-10T10:25:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:02 smithi082 ceph-mon[98694]: pgmap v21454: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 716 KiB/s wr, 14 op/s 2023-09-10T10:25:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:02 smithi098 ceph-mon[103124]: pgmap v21454: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 716 KiB/s wr, 14 op/s 2023-09-10T10:25:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:04 smithi170 ceph-mon[102832]: pgmap v21455: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1000 KiB/s wr, 31 op/s 2023-09-10T10:25:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39934.683500 secs 2023-09-10T10:25:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:04 smithi170 ceph-mon[102832]: osdmap e1409: 12 total, 12 up, 12 in 2023-09-10T10:25:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:04 smithi082 ceph-mon[98694]: pgmap v21455: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1000 KiB/s wr, 31 op/s 2023-09-10T10:25:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39934.683500 secs 2023-09-10T10:25:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:04 smithi082 ceph-mon[98694]: osdmap e1409: 12 total, 12 up, 12 in 2023-09-10T10:25:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:04 smithi098 ceph-mon[103124]: pgmap v21455: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1000 KiB/s wr, 31 op/s 2023-09-10T10:25:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39934.683500 secs 2023-09-10T10:25:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:04 smithi098 ceph-mon[103124]: osdmap e1409: 12 total, 12 up, 12 in 2023-09-10T10:25:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39935.913202 secs 2023-09-10T10:25:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39936.264949 secs 2023-09-10T10:25:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:25:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:05.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39935.913202 secs 2023-09-10T10:25:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39936.264949 secs 2023-09-10T10:25:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:25:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39935.913202 secs 2023-09-10T10:25:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39936.264949 secs 2023-09-10T10:25:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:25:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:06 smithi170 ceph-mon[102832]: pgmap v21457: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 904 KiB/s wr, 28 op/s 2023-09-10T10:25:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:06 smithi082 ceph-mon[98694]: pgmap v21457: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 904 KiB/s wr, 28 op/s 2023-09-10T10:25:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:06 smithi098 ceph-mon[103124]: pgmap v21457: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 904 KiB/s wr, 28 op/s 2023-09-10T10:25:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:08 smithi170 ceph-mon[102832]: pgmap v21458: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 731 KiB/s wr, 33 op/s 2023-09-10T10:25:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:08 smithi082 ceph-mon[98694]: pgmap v21458: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 731 KiB/s wr, 33 op/s 2023-09-10T10:25:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:08 smithi098 ceph-mon[103124]: pgmap v21458: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 731 KiB/s wr, 33 op/s 2023-09-10T10:25:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39939.683566 secs 2023-09-10T10:25:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39939.683566 secs 2023-09-10T10:25:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39939.683566 secs 2023-09-10T10:25:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:10 smithi170 ceph-mon[102832]: pgmap v21459: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 731 KiB/s wr, 33 op/s 2023-09-10T10:25:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39940.913261 secs 2023-09-10T10:25:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39941.265011 secs 2023-09-10T10:25:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:25:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:10 smithi170 ceph-mon[102832]: osdmap e1410: 12 total, 12 up, 12 in 2023-09-10T10:25:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:10 smithi082 ceph-mon[98694]: pgmap v21459: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 731 KiB/s wr, 33 op/s 2023-09-10T10:25:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39940.913261 secs 2023-09-10T10:25:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39941.265011 secs 2023-09-10T10:25:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:25:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:10.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:10 smithi082 ceph-mon[98694]: osdmap e1410: 12 total, 12 up, 12 in 2023-09-10T10:25:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:10 smithi098 ceph-mon[103124]: pgmap v21459: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 731 KiB/s wr, 33 op/s 2023-09-10T10:25:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39940.913261 secs 2023-09-10T10:25:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39941.265011 secs 2023-09-10T10:25:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:25:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:10 smithi098 ceph-mon[103124]: osdmap e1410: 12 total, 12 up, 12 in 2023-09-10T10:25:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:12 smithi170 ceph-mon[102832]: pgmap v21461: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 488 KiB/s wr, 17 op/s 2023-09-10T10:25:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:12 smithi082 ceph-mon[98694]: pgmap v21461: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 488 KiB/s wr, 17 op/s 2023-09-10T10:25:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:12 smithi098 ceph-mon[103124]: pgmap v21461: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 488 KiB/s wr, 17 op/s 2023-09-10T10:25:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:14 smithi170 ceph-mon[102832]: pgmap v21462: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 459 KiB/s wr, 23 op/s 2023-09-10T10:25:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39944.683636 secs 2023-09-10T10:25:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:14 smithi082 ceph-mon[98694]: pgmap v21462: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 459 KiB/s wr, 23 op/s 2023-09-10T10:25:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39944.683636 secs 2023-09-10T10:25:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:14 smithi098 ceph-mon[103124]: pgmap v21462: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 459 KiB/s wr, 23 op/s 2023-09-10T10:25:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39944.683636 secs 2023-09-10T10:25:15.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39945.913320 secs 2023-09-10T10:25:15.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39946.265077 secs 2023-09-10T10:25:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39945.913320 secs 2023-09-10T10:25:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39946.265077 secs 2023-09-10T10:25:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39945.913320 secs 2023-09-10T10:25:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39946.265077 secs 2023-09-10T10:25:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:16 smithi170 ceph-mon[102832]: pgmap v21463: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 421 KiB/s wr, 21 op/s 2023-09-10T10:25:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:16 smithi082 ceph-mon[98694]: pgmap v21463: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 421 KiB/s wr, 21 op/s 2023-09-10T10:25:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:16 smithi098 ceph-mon[103124]: pgmap v21463: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 421 KiB/s wr, 21 op/s 2023-09-10T10:25:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:18 smithi170 ceph-mon[102832]: pgmap v21464: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 319 KiB/s wr, 15 op/s 2023-09-10T10:25:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:25:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:18 smithi082 ceph-mon[98694]: pgmap v21464: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 319 KiB/s wr, 15 op/s 2023-09-10T10:25:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:25:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:18 smithi098 ceph-mon[103124]: pgmap v21464: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 319 KiB/s wr, 15 op/s 2023-09-10T10:25:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:25:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39949.683695 secs 2023-09-10T10:25:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39949.683695 secs 2023-09-10T10:25:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39949.683695 secs 2023-09-10T10:25:20.095 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:20 smithi098 ceph-mon[103124]: pgmap v21465: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 319 KiB/s wr, 15 op/s 2023-09-10T10:25:20.096 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39950.913378 secs 2023-09-10T10:25:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:20 smithi170 ceph-mon[102832]: pgmap v21465: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 319 KiB/s wr, 15 op/s 2023-09-10T10:25:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39950.913378 secs 2023-09-10T10:25:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39951.265139 secs 2023-09-10T10:25:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:20 smithi082 ceph-mon[98694]: pgmap v21465: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 319 KiB/s wr, 15 op/s 2023-09-10T10:25:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39950.913378 secs 2023-09-10T10:25:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39951.265139 secs 2023-09-10T10:25:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:21 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:25:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:21 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:21.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:21 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:25:21.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:21 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39951.265139 secs 2023-09-10T10:25:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:21 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:25:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:21 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:22 smithi170 ceph-mon[102832]: pgmap v21466: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 311 KiB/s wr, 15 op/s 2023-09-10T10:25:22.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:22 smithi082 ceph-mon[98694]: pgmap v21466: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 311 KiB/s wr, 15 op/s 2023-09-10T10:25:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:22 smithi098 ceph-mon[103124]: pgmap v21466: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 311 KiB/s wr, 15 op/s 2023-09-10T10:25:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:24 smithi170 ceph-mon[102832]: pgmap v21467: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 297 KiB/s wr, 19 op/s 2023-09-10T10:25:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39954.683787 secs 2023-09-10T10:25:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:24 smithi082 ceph-mon[98694]: pgmap v21467: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 297 KiB/s wr, 19 op/s 2023-09-10T10:25:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39954.683787 secs 2023-09-10T10:25:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:24 smithi098 ceph-mon[103124]: pgmap v21467: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 297 KiB/s wr, 19 op/s 2023-09-10T10:25:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39954.683787 secs 2023-09-10T10:25:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39955.913436 secs 2023-09-10T10:25:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39956.265203 secs 2023-09-10T10:25:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39955.913436 secs 2023-09-10T10:25:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39956.265203 secs 2023-09-10T10:25:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39955.913436 secs 2023-09-10T10:25:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39956.265203 secs 2023-09-10T10:25:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:26 smithi170 ceph-mon[102832]: pgmap v21468: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 272 KiB/s wr, 12 op/s 2023-09-10T10:25:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:26 smithi082 ceph-mon[98694]: pgmap v21468: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 272 KiB/s wr, 12 op/s 2023-09-10T10:25:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:26 smithi098 ceph-mon[103124]: pgmap v21468: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 272 KiB/s wr, 12 op/s 2023-09-10T10:25:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:28 smithi170 ceph-mon[102832]: pgmap v21469: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 272 KiB/s wr, 18 op/s 2023-09-10T10:25:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:28 smithi082 ceph-mon[98694]: pgmap v21469: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 272 KiB/s wr, 18 op/s 2023-09-10T10:25:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:28 smithi098 ceph-mon[103124]: pgmap v21469: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 272 KiB/s wr, 18 op/s 2023-09-10T10:25:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39959.683877 secs 2023-09-10T10:25:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39959.683877 secs 2023-09-10T10:25:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39959.683877 secs 2023-09-10T10:25:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:30 smithi170 ceph-mon[102832]: pgmap v21470: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-10T10:25:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39960.913497 secs 2023-09-10T10:25:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39961.265266 secs 2023-09-10T10:25:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:30 smithi082 ceph-mon[98694]: pgmap v21470: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-10T10:25:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39960.913497 secs 2023-09-10T10:25:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39961.265266 secs 2023-09-10T10:25:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:30 smithi098 ceph-mon[103124]: pgmap v21470: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-10T10:25:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39960.913497 secs 2023-09-10T10:25:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39961.265266 secs 2023-09-10T10:25:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:32 smithi170 ceph-mon[102832]: pgmap v21471: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-10T10:25:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:32 smithi082 ceph-mon[98694]: pgmap v21471: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-10T10:25:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:32 smithi098 ceph-mon[103124]: pgmap v21471: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 12 op/s 2023-09-10T10:25:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:34 smithi170 ceph-mon[102832]: pgmap v21472: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 178 KiB/s wr, 17 op/s 2023-09-10T10:25:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39964.683938 secs 2023-09-10T10:25:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:34 smithi082 ceph-mon[98694]: pgmap v21472: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 178 KiB/s wr, 17 op/s 2023-09-10T10:25:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39964.683938 secs 2023-09-10T10:25:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:34 smithi098 ceph-mon[103124]: pgmap v21472: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 178 KiB/s wr, 17 op/s 2023-09-10T10:25:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39964.683938 secs 2023-09-10T10:25:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39965.913554 secs 2023-09-10T10:25:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39966.265330 secs 2023-09-10T10:25:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39965.913554 secs 2023-09-10T10:25:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39966.265330 secs 2023-09-10T10:25:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39965.913554 secs 2023-09-10T10:25:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39966.265330 secs 2023-09-10T10:25:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:36 smithi170 ceph-mon[102832]: pgmap v21473: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:25:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:36 smithi082 ceph-mon[98694]: pgmap v21473: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:25:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:36 smithi098 ceph-mon[103124]: pgmap v21473: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-09-10T10:25:38.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:38 smithi170 ceph-mon[102832]: pgmap v21474: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 157 KiB/s wr, 16 op/s 2023-09-10T10:25:38.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:25:38.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:38 smithi082 ceph-mon[98694]: pgmap v21474: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 157 KiB/s wr, 16 op/s 2023-09-10T10:25:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:25:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:38 smithi098 ceph-mon[103124]: pgmap v21474: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 157 KiB/s wr, 16 op/s 2023-09-10T10:25:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:25:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:40 smithi170 ceph-mon[102832]: pgmap v21475: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 157 KiB/s wr, 10 op/s 2023-09-10T10:25:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39970.913613 secs 2023-09-10T10:25:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39971.265396 secs 2023-09-10T10:25:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:40 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:25:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:40 smithi082 ceph-mon[98694]: pgmap v21475: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 157 KiB/s wr, 10 op/s 2023-09-10T10:25:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39970.913613 secs 2023-09-10T10:25:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39971.265396 secs 2023-09-10T10:25:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:40 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:25:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:40 smithi098 ceph-mon[103124]: pgmap v21475: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 157 KiB/s wr, 10 op/s 2023-09-10T10:25:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39970.913613 secs 2023-09-10T10:25:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39971.265396 secs 2023-09-10T10:25:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:40 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:25:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:41 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:25:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:41 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:25:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:41 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:41.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:41 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:25:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:41 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:25:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:41 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:25:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:41 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:41 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:25:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:41 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:25:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:41 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:25:41.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:41 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:41.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:41 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:25:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:42 smithi170 ceph-mon[102832]: pgmap v21476: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 157 KiB/s wr, 10 op/s 2023-09-10T10:25:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:42 smithi082 ceph-mon[98694]: pgmap v21476: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 157 KiB/s wr, 10 op/s 2023-09-10T10:25:42.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:42 smithi098 ceph-mon[103124]: pgmap v21476: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 157 KiB/s wr, 10 op/s 2023-09-10T10:25:44.424 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:44 smithi082 ceph-mon[98694]: pgmap v21477: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 529 KiB/s wr, 17 op/s 2023-09-10T10:25:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:44 smithi170 ceph-mon[102832]: pgmap v21477: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 529 KiB/s wr, 17 op/s 2023-09-10T10:25:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:44 smithi098 ceph-mon[103124]: pgmap v21477: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 529 KiB/s wr, 17 op/s 2023-09-10T10:25:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39975.913675 secs 2023-09-10T10:25:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39976.265464 secs 2023-09-10T10:25:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:45 smithi082 ceph-mon[98694]: pgmap v21478: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 382 KiB/s wr, 11 op/s 2023-09-10T10:25:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:25:45.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:25:45.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:25:45.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:25:45.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:25:45.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:25:45.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:25:45.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:25:45.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:25:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39975.913675 secs 2023-09-10T10:25:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39976.265464 secs 2023-09-10T10:25:45.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:45 smithi098 ceph-mon[103124]: pgmap v21478: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 382 KiB/s wr, 11 op/s 2023-09-10T10:25:45.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:25:45.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:25:45.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:25:45.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:25:45.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:25:45.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:25:45.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:25:45.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:25:45.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:25:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39975.913675 secs 2023-09-10T10:25:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39976.265464 secs 2023-09-10T10:25:45.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:45 smithi170 ceph-mon[102832]: pgmap v21478: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 382 KiB/s wr, 11 op/s 2023-09-10T10:25:45.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:25:45.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:25:45.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:25:45.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:25:45.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:25:45.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:25:45.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:25:45.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:25:45.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:25:47.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:47 smithi082 ceph-mon[98694]: pgmap v21479: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 637 KiB/s wr, 18 op/s 2023-09-10T10:25:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:47 smithi098 ceph-mon[103124]: pgmap v21479: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 637 KiB/s wr, 18 op/s 2023-09-10T10:25:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:47 smithi170 ceph-mon[102832]: pgmap v21479: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 637 KiB/s wr, 18 op/s 2023-09-10T10:25:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:49 smithi098 ceph-mon[103124]: pgmap v21480: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 627 KiB/s wr, 13 op/s 2023-09-10T10:25:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:49 smithi170 ceph-mon[102832]: pgmap v21480: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 627 KiB/s wr, 13 op/s 2023-09-10T10:25:49.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:49 smithi082 ceph-mon[98694]: pgmap v21480: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 627 KiB/s wr, 13 op/s 2023-09-10T10:25:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39980.913750 secs 2023-09-10T10:25:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39981.265488 secs 2023-09-10T10:25:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:25:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39980.913750 secs 2023-09-10T10:25:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39981.265488 secs 2023-09-10T10:25:50.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:25:50.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:50.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39980.913750 secs 2023-09-10T10:25:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39981.265488 secs 2023-09-10T10:25:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:25:50.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:52.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:51 smithi082 ceph-mon[98694]: pgmap v21481: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 627 KiB/s wr, 13 op/s 2023-09-10T10:25:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:51 smithi098 ceph-mon[103124]: pgmap v21481: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 627 KiB/s wr, 13 op/s 2023-09-10T10:25:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:51 smithi170 ceph-mon[102832]: pgmap v21481: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 627 KiB/s wr, 13 op/s 2023-09-10T10:25:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:25:53.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:25:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:25:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:54.257 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:53 smithi098 ceph-mon[103124]: pgmap v21482: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 861 KiB/s wr, 21 op/s 2023-09-10T10:25:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:53 smithi170 ceph-mon[102832]: pgmap v21482: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 861 KiB/s wr, 21 op/s 2023-09-10T10:25:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:53 smithi082 ceph-mon[98694]: pgmap v21482: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 861 KiB/s wr, 21 op/s 2023-09-10T10:25:55.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39985.913811 secs 2023-09-10T10:25:55.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39986.265553 secs 2023-09-10T10:25:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39985.913811 secs 2023-09-10T10:25:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39986.265553 secs 2023-09-10T10:25:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39985.913811 secs 2023-09-10T10:25:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39986.265553 secs 2023-09-10T10:25:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:55 smithi098 ceph-mon[103124]: pgmap v21483: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 489 KiB/s wr, 14 op/s 2023-09-10T10:25:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:55 smithi170 ceph-mon[102832]: pgmap v21483: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 489 KiB/s wr, 14 op/s 2023-09-10T10:25:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:55 smithi082 ceph-mon[98694]: pgmap v21483: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 489 KiB/s wr, 14 op/s 2023-09-10T10:25:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:57 smithi098 ceph-mon[103124]: pgmap v21484: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 799 KiB/s wr, 21 op/s 2023-09-10T10:25:58.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:25:58.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:57 smithi170 ceph-mon[102832]: pgmap v21484: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 799 KiB/s wr, 21 op/s 2023-09-10T10:25:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:25:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:57 smithi082 ceph-mon[98694]: pgmap v21484: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 799 KiB/s wr, 21 op/s 2023-09-10T10:25:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:25:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:25:59.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39989.684308 secs 2023-09-10T10:25:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39989.684308 secs 2023-09-10T10:25:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39989.684308 secs 2023-09-10T10:26:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:59 smithi098 ceph-mon[103124]: pgmap v21485: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 544 KiB/s wr, 14 op/s 2023-09-10T10:26:00.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39990.913896 secs 2023-09-10T10:26:00.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:25:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39991.265621 secs 2023-09-10T10:26:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:59 smithi170 ceph-mon[102832]: pgmap v21485: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 544 KiB/s wr, 14 op/s 2023-09-10T10:26:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39990.913896 secs 2023-09-10T10:26:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:25:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39991.265621 secs 2023-09-10T10:26:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:59 smithi082 ceph-mon[98694]: pgmap v21485: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 544 KiB/s wr, 14 op/s 2023-09-10T10:26:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39990.913896 secs 2023-09-10T10:26:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:25:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39991.265621 secs 2023-09-10T10:26:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:26:01.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:01 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:26:01.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:01 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:26:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:26:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:26:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:01 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:26:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:26:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:26:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:26:02.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:02 smithi098 ceph-mon[103124]: pgmap v21486: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 544 KiB/s wr, 14 op/s 2023-09-10T10:26:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:02 smithi170 ceph-mon[102832]: pgmap v21486: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 544 KiB/s wr, 14 op/s 2023-09-10T10:26:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:02 smithi082 ceph-mon[98694]: pgmap v21486: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 544 KiB/s wr, 14 op/s 2023-09-10T10:26:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:04 smithi170 ceph-mon[102832]: pgmap v21487: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 832 KiB/s wr, 22 op/s 2023-09-10T10:26:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39994.684398 secs 2023-09-10T10:26:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:04 smithi170 ceph-mon[102832]: osdmap e1411: 12 total, 12 up, 12 in 2023-09-10T10:26:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:04 smithi082 ceph-mon[98694]: pgmap v21487: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 832 KiB/s wr, 22 op/s 2023-09-10T10:26:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39994.684398 secs 2023-09-10T10:26:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:04 smithi082 ceph-mon[98694]: osdmap e1411: 12 total, 12 up, 12 in 2023-09-10T10:26:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:04 smithi098 ceph-mon[103124]: pgmap v21487: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 832 KiB/s wr, 22 op/s 2023-09-10T10:26:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39994.684398 secs 2023-09-10T10:26:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:04 smithi098 ceph-mon[103124]: osdmap e1411: 12 total, 12 up, 12 in 2023-09-10T10:26:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39995.913954 secs 2023-09-10T10:26:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 39996.265685 secs 2023-09-10T10:26:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39995.913954 secs 2023-09-10T10:26:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 39996.265685 secs 2023-09-10T10:26:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39995.913954 secs 2023-09-10T10:26:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 39996.265685 secs 2023-09-10T10:26:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:06 smithi170 ceph-mon[102832]: pgmap v21489: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 717 KiB/s wr, 17 op/s 2023-09-10T10:26:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:06 smithi082 ceph-mon[98694]: pgmap v21489: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 717 KiB/s wr, 17 op/s 2023-09-10T10:26:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:06 smithi098 ceph-mon[103124]: pgmap v21489: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 717 KiB/s wr, 17 op/s 2023-09-10T10:26:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:08 smithi170 ceph-mon[102832]: pgmap v21490: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 743 KiB/s wr, 33 op/s 2023-09-10T10:26:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:08 smithi082 ceph-mon[98694]: pgmap v21490: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 743 KiB/s wr, 33 op/s 2023-09-10T10:26:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:08 smithi098 ceph-mon[103124]: pgmap v21490: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 743 KiB/s wr, 33 op/s 2023-09-10T10:26:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 39999.684454 secs 2023-09-10T10:26:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 39999.684454 secs 2023-09-10T10:26:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 39999.684454 secs 2023-09-10T10:26:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:10 smithi170 ceph-mon[102832]: pgmap v21491: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 743 KiB/s wr, 33 op/s 2023-09-10T10:26:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40000.914013 secs 2023-09-10T10:26:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40001.265750 secs 2023-09-10T10:26:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:26:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:26:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:10 smithi170 ceph-mon[102832]: osdmap e1412: 12 total, 12 up, 12 in 2023-09-10T10:26:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:10 smithi082 ceph-mon[98694]: pgmap v21491: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 743 KiB/s wr, 33 op/s 2023-09-10T10:26:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40000.914013 secs 2023-09-10T10:26:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40001.265750 secs 2023-09-10T10:26:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:26:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:26:10.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:10 smithi082 ceph-mon[98694]: osdmap e1412: 12 total, 12 up, 12 in 2023-09-10T10:26:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:10 smithi098 ceph-mon[103124]: pgmap v21491: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 743 KiB/s wr, 33 op/s 2023-09-10T10:26:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40000.914013 secs 2023-09-10T10:26:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40001.265750 secs 2023-09-10T10:26:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:26:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:26:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:10 smithi098 ceph-mon[103124]: osdmap e1412: 12 total, 12 up, 12 in 2023-09-10T10:26:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:12 smithi170 ceph-mon[102832]: pgmap v21493: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 496 KiB/s wr, 31 op/s 2023-09-10T10:26:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:12 smithi082 ceph-mon[98694]: pgmap v21493: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 496 KiB/s wr, 31 op/s 2023-09-10T10:26:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:12 smithi098 ceph-mon[103124]: pgmap v21493: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 496 KiB/s wr, 31 op/s 2023-09-10T10:26:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:26:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:26:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:26:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:26:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:26:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:26:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:14 smithi170 ceph-mon[102832]: pgmap v21494: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 699 KiB/s wr, 35 op/s 2023-09-10T10:26:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40004.684513 secs 2023-09-10T10:26:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:14 smithi082 ceph-mon[98694]: pgmap v21494: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 699 KiB/s wr, 35 op/s 2023-09-10T10:26:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40004.684513 secs 2023-09-10T10:26:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:14 smithi098 ceph-mon[103124]: pgmap v21494: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 699 KiB/s wr, 35 op/s 2023-09-10T10:26:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40004.684513 secs 2023-09-10T10:26:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40005.914070 secs 2023-09-10T10:26:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40006.265814 secs 2023-09-10T10:26:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40005.914070 secs 2023-09-10T10:26:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40006.265814 secs 2023-09-10T10:26:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40005.914070 secs 2023-09-10T10:26:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40006.265814 secs 2023-09-10T10:26:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:16 smithi170 ceph-mon[102832]: pgmap v21495: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 644 KiB/s wr, 32 op/s 2023-09-10T10:26:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:16 smithi082 ceph-mon[98694]: pgmap v21495: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 644 KiB/s wr, 32 op/s 2023-09-10T10:26:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:16 smithi098 ceph-mon[103124]: pgmap v21495: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 644 KiB/s wr, 32 op/s 2023-09-10T10:26:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:18 smithi170 ceph-mon[102832]: pgmap v21496: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 425 KiB/s wr, 15 op/s 2023-09-10T10:26:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:18 smithi082 ceph-mon[98694]: pgmap v21496: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 425 KiB/s wr, 15 op/s 2023-09-10T10:26:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:18 smithi098 ceph-mon[103124]: pgmap v21496: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 425 KiB/s wr, 15 op/s 2023-09-10T10:26:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40009.684624 secs 2023-09-10T10:26:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40009.684624 secs 2023-09-10T10:26:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40009.684624 secs 2023-09-10T10:26:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:20 smithi170 ceph-mon[102832]: pgmap v21497: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 425 KiB/s wr, 15 op/s 2023-09-10T10:26:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40010.914129 secs 2023-09-10T10:26:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40011.265882 secs 2023-09-10T10:26:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:20 smithi082 ceph-mon[98694]: pgmap v21497: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 425 KiB/s wr, 15 op/s 2023-09-10T10:26:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40010.914129 secs 2023-09-10T10:26:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40011.265882 secs 2023-09-10T10:26:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:20 smithi098 ceph-mon[103124]: pgmap v21497: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 425 KiB/s wr, 15 op/s 2023-09-10T10:26:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40010.914129 secs 2023-09-10T10:26:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40011.265882 secs 2023-09-10T10:26:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:22 smithi170 ceph-mon[102832]: pgmap v21498: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 413 KiB/s wr, 14 op/s 2023-09-10T10:26:22.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:22 smithi082 ceph-mon[98694]: pgmap v21498: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 413 KiB/s wr, 14 op/s 2023-09-10T10:26:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:22 smithi098 ceph-mon[103124]: pgmap v21498: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 413 KiB/s wr, 14 op/s 2023-09-10T10:26:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:24 smithi170 ceph-mon[102832]: pgmap v21499: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 355 KiB/s wr, 17 op/s 2023-09-10T10:26:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40014.684692 secs 2023-09-10T10:26:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:24 smithi082 ceph-mon[98694]: pgmap v21499: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 355 KiB/s wr, 17 op/s 2023-09-10T10:26:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40014.684692 secs 2023-09-10T10:26:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:24 smithi098 ceph-mon[103124]: pgmap v21499: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 355 KiB/s wr, 17 op/s 2023-09-10T10:26:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40014.684692 secs 2023-09-10T10:26:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40015.914192 secs 2023-09-10T10:26:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40016.265946 secs 2023-09-10T10:26:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40015.914192 secs 2023-09-10T10:26:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40016.265946 secs 2023-09-10T10:26:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40015.914192 secs 2023-09-10T10:26:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40016.265946 secs 2023-09-10T10:26:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:26 smithi170 ceph-mon[102832]: pgmap v21500: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:26:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:26 smithi082 ceph-mon[98694]: pgmap v21500: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:26:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:26 smithi098 ceph-mon[103124]: pgmap v21500: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:26:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:28 smithi170 ceph-mon[102832]: pgmap v21501: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 296 KiB/s wr, 16 op/s 2023-09-10T10:26:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:28 smithi082 ceph-mon[98694]: pgmap v21501: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 296 KiB/s wr, 16 op/s 2023-09-10T10:26:28.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:28 smithi098 ceph-mon[103124]: pgmap v21501: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 296 KiB/s wr, 16 op/s 2023-09-10T10:26:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40019.684750 secs 2023-09-10T10:26:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40019.684750 secs 2023-09-10T10:26:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40019.684750 secs 2023-09-10T10:26:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:30 smithi170 ceph-mon[102832]: pgmap v21502: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:26:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40020.914252 secs 2023-09-10T10:26:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40021.266012 secs 2023-09-10T10:26:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:30 smithi082 ceph-mon[98694]: pgmap v21502: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:26:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40020.914252 secs 2023-09-10T10:26:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40021.266012 secs 2023-09-10T10:26:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:30 smithi098 ceph-mon[103124]: pgmap v21502: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:26:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40020.914252 secs 2023-09-10T10:26:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40021.266012 secs 2023-09-10T10:26:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:32 smithi170 ceph-mon[102832]: pgmap v21503: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:26:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:32 smithi082 ceph-mon[98694]: pgmap v21503: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:26:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:32 smithi098 ceph-mon[103124]: pgmap v21503: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:26:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:34 smithi170 ceph-mon[102832]: pgmap v21504: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 148 KiB/s wr, 15 op/s 2023-09-10T10:26:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40024.684858 secs 2023-09-10T10:26:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:34 smithi082 ceph-mon[98694]: pgmap v21504: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 148 KiB/s wr, 15 op/s 2023-09-10T10:26:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40024.684858 secs 2023-09-10T10:26:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:34 smithi098 ceph-mon[103124]: pgmap v21504: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 148 KiB/s wr, 15 op/s 2023-09-10T10:26:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40024.684858 secs 2023-09-10T10:26:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40025.914311 secs 2023-09-10T10:26:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40026.266076 secs 2023-09-10T10:26:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40025.914311 secs 2023-09-10T10:26:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40026.266076 secs 2023-09-10T10:26:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40025.914311 secs 2023-09-10T10:26:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40026.266076 secs 2023-09-10T10:26:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:36 smithi170 ceph-mon[102832]: pgmap v21505: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:26:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:36 smithi082 ceph-mon[98694]: pgmap v21505: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:26:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:36 smithi098 ceph-mon[103124]: pgmap v21505: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:26:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:38 smithi170 ceph-mon[102832]: pgmap v21506: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 291 KiB/s wr, 16 op/s 2023-09-10T10:26:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:26:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:26:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:38 smithi082 ceph-mon[98694]: pgmap v21506: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 291 KiB/s wr, 16 op/s 2023-09-10T10:26:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:26:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:26:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:38 smithi098 ceph-mon[103124]: pgmap v21506: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 291 KiB/s wr, 16 op/s 2023-09-10T10:26:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:26:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:26:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:40 smithi170 ceph-mon[102832]: pgmap v21507: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:26:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40030.914394 secs 2023-09-10T10:26:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40031.266139 secs 2023-09-10T10:26:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:40 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:26:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:40 smithi082 ceph-mon[98694]: pgmap v21507: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:26:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40030.914394 secs 2023-09-10T10:26:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40031.266139 secs 2023-09-10T10:26:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:40 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:26:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:40 smithi098 ceph-mon[103124]: pgmap v21507: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:26:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40030.914394 secs 2023-09-10T10:26:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40031.266139 secs 2023-09-10T10:26:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:40 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:26:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:41 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:26:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:41 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:26:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:41 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:26:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:41 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:26:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:41 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:26:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:41 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:26:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:41 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:26:41.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:41 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:26:41.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:41 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:26:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:42 smithi170 ceph-mon[102832]: pgmap v21508: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:26:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:42 smithi082 ceph-mon[98694]: pgmap v21508: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:26:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:42 smithi098 ceph-mon[103124]: pgmap v21508: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:26:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:44 smithi170 ceph-mon[102832]: pgmap v21509: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 466 KiB/s wr, 17 op/s 2023-09-10T10:26:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40035.914489 secs 2023-09-10T10:26:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:44 smithi082 ceph-mon[98694]: pgmap v21509: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 466 KiB/s wr, 17 op/s 2023-09-10T10:26:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40035.914489 secs 2023-09-10T10:26:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:44 smithi098 ceph-mon[103124]: pgmap v21509: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 466 KiB/s wr, 17 op/s 2023-09-10T10:26:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40035.914489 secs 2023-09-10T10:26:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40036.266204 secs 2023-09-10T10:26:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:45 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:26:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40036.266204 secs 2023-09-10T10:26:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:45 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:26:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40036.266204 secs 2023-09-10T10:26:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:45 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:26:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:46 smithi170 ceph-mon[102832]: pgmap v21510: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 466 KiB/s wr, 12 op/s 2023-09-10T10:26:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:26:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:26:46.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:26:46.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:26:46.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:26:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:46 smithi082 ceph-mon[98694]: pgmap v21510: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 466 KiB/s wr, 12 op/s 2023-09-10T10:26:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:26:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:26:46.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:26:46.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:26:46.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:26:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:46 smithi098 ceph-mon[103124]: pgmap v21510: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 466 KiB/s wr, 12 op/s 2023-09-10T10:26:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:26:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:26:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:26:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:26:46.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:26:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:48 smithi170 ceph-mon[102832]: pgmap v21511: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 782 KiB/s wr, 20 op/s 2023-09-10T10:26:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:48 smithi082 ceph-mon[98694]: pgmap v21511: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 782 KiB/s wr, 20 op/s 2023-09-10T10:26:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:48 smithi098 ceph-mon[103124]: pgmap v21511: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 782 KiB/s wr, 20 op/s 2023-09-10T10:26:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40040.914494 secs 2023-09-10T10:26:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40040.914494 secs 2023-09-10T10:26:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40040.914494 secs 2023-09-10T10:26:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:50 smithi170 ceph-mon[102832]: pgmap v21512: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 638 KiB/s wr, 14 op/s 2023-09-10T10:26:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40041.266271 secs 2023-09-10T10:26:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:26:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:26:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:50 smithi082 ceph-mon[98694]: pgmap v21512: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 638 KiB/s wr, 14 op/s 2023-09-10T10:26:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40041.266271 secs 2023-09-10T10:26:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:26:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:26:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:50 smithi098 ceph-mon[103124]: pgmap v21512: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 638 KiB/s wr, 14 op/s 2023-09-10T10:26:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40041.266271 secs 2023-09-10T10:26:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:26:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:26:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:52 smithi170 ceph-mon[102832]: pgmap v21513: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 638 KiB/s wr, 14 op/s 2023-09-10T10:26:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:52 smithi082 ceph-mon[98694]: pgmap v21513: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 638 KiB/s wr, 14 op/s 2023-09-10T10:26:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:52 smithi098 ceph-mon[103124]: pgmap v21513: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 638 KiB/s wr, 14 op/s 2023-09-10T10:26:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:26:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:26:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:26:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:26:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:26:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:26:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:54 smithi170 ceph-mon[102832]: pgmap v21514: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 990 KiB/s wr, 21 op/s 2023-09-10T10:26:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40045.914552 secs 2023-09-10T10:26:54.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:54 smithi082 ceph-mon[98694]: pgmap v21514: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 990 KiB/s wr, 21 op/s 2023-09-10T10:26:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40045.914552 secs 2023-09-10T10:26:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:54 smithi098 ceph-mon[103124]: pgmap v21514: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 990 KiB/s wr, 21 op/s 2023-09-10T10:26:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40045.914552 secs 2023-09-10T10:26:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40046.266333 secs 2023-09-10T10:26:55.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40046.266333 secs 2023-09-10T10:26:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40046.266333 secs 2023-09-10T10:26:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:56 smithi170 ceph-mon[102832]: pgmap v21515: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 668 KiB/s wr, 14 op/s 2023-09-10T10:26:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:56 smithi082 ceph-mon[98694]: pgmap v21515: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 668 KiB/s wr, 14 op/s 2023-09-10T10:26:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:56 smithi098 ceph-mon[103124]: pgmap v21515: 97 pgs: 97 active+clean; 14 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 668 KiB/s wr, 14 op/s 2023-09-10T10:26:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:58 smithi082 ceph-mon[98694]: pgmap v21516: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 887 KiB/s wr, 20 op/s 2023-09-10T10:26:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:26:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:26:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:58 smithi098 ceph-mon[103124]: pgmap v21516: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 887 KiB/s wr, 20 op/s 2023-09-10T10:26:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:26:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:26:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:58 smithi170 ceph-mon[102832]: pgmap v21516: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 887 KiB/s wr, 20 op/s 2023-09-10T10:26:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:26:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:26:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40049.685199 secs 2023-09-10T10:26:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:26:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40050.914610 secs 2023-09-10T10:26:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40049.685199 secs 2023-09-10T10:26:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:26:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40050.914610 secs 2023-09-10T10:26:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40049.685199 secs 2023-09-10T10:26:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:26:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40050.914610 secs 2023-09-10T10:27:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:00 smithi082 ceph-mon[98694]: pgmap v21517: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 571 KiB/s wr, 13 op/s 2023-09-10T10:27:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40051.266398 secs 2023-09-10T10:27:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:27:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:27:00.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:27:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:00 smithi098 ceph-mon[103124]: pgmap v21517: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 571 KiB/s wr, 13 op/s 2023-09-10T10:27:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40051.266398 secs 2023-09-10T10:27:00.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:27:00.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:27:00.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:27:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:00 smithi170 ceph-mon[102832]: pgmap v21517: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 571 KiB/s wr, 13 op/s 2023-09-10T10:27:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40051.266398 secs 2023-09-10T10:27:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:27:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:27:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:27:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:01 smithi082 ceph-mon[98694]: pgmap v21518: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 571 KiB/s wr, 13 op/s 2023-09-10T10:27:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:01 smithi098 ceph-mon[103124]: pgmap v21518: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 571 KiB/s wr, 13 op/s 2023-09-10T10:27:01.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:01 smithi170 ceph-mon[102832]: pgmap v21518: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 571 KiB/s wr, 13 op/s 2023-09-10T10:27:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:03 smithi170 ceph-mon[102832]: pgmap v21519: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 835 KiB/s wr, 21 op/s 2023-09-10T10:27:03.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:03 smithi082 ceph-mon[98694]: pgmap v21519: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 835 KiB/s wr, 21 op/s 2023-09-10T10:27:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:03 smithi098 ceph-mon[103124]: pgmap v21519: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 835 KiB/s wr, 21 op/s 2023-09-10T10:27:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40054.685259 secs 2023-09-10T10:27:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40055.914694 secs 2023-09-10T10:27:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40054.685259 secs 2023-09-10T10:27:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40055.914694 secs 2023-09-10T10:27:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40054.685259 secs 2023-09-10T10:27:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40055.914694 secs 2023-09-10T10:27:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40056.266465 secs 2023-09-10T10:27:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:05 smithi170 ceph-mon[102832]: pgmap v21520: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 482 KiB/s wr, 14 op/s 2023-09-10T10:27:05.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40056.266465 secs 2023-09-10T10:27:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:05 smithi082 ceph-mon[98694]: pgmap v21520: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 482 KiB/s wr, 14 op/s 2023-09-10T10:27:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40056.266465 secs 2023-09-10T10:27:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:05 smithi098 ceph-mon[103124]: pgmap v21520: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 482 KiB/s wr, 14 op/s 2023-09-10T10:27:07.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:07 smithi170 ceph-mon[102832]: pgmap v21521: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 670 KiB/s wr, 34 op/s 2023-09-10T10:27:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:07 smithi082 ceph-mon[98694]: pgmap v21521: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 670 KiB/s wr, 34 op/s 2023-09-10T10:27:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:07 smithi098 ceph-mon[103124]: pgmap v21521: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 670 KiB/s wr, 34 op/s 2023-09-10T10:27:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40059.685385 secs 2023-09-10T10:27:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:09 smithi170 ceph-mon[102832]: pgmap v21522: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 452 KiB/s wr, 28 op/s 2023-09-10T10:27:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:09 smithi170 ceph-mon[102832]: osdmap e1413: 12 total, 12 up, 12 in 2023-09-10T10:27:09.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40060.914754 secs 2023-09-10T10:27:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40059.685385 secs 2023-09-10T10:27:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:09 smithi082 ceph-mon[98694]: pgmap v21522: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 452 KiB/s wr, 28 op/s 2023-09-10T10:27:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:09 smithi082 ceph-mon[98694]: osdmap e1413: 12 total, 12 up, 12 in 2023-09-10T10:27:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40060.914754 secs 2023-09-10T10:27:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40059.685385 secs 2023-09-10T10:27:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:09 smithi098 ceph-mon[103124]: pgmap v21522: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 452 KiB/s wr, 28 op/s 2023-09-10T10:27:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:09 smithi098 ceph-mon[103124]: osdmap e1413: 12 total, 12 up, 12 in 2023-09-10T10:27:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40060.914754 secs 2023-09-10T10:27:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40061.266490 secs 2023-09-10T10:27:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:27:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:27:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40061.266490 secs 2023-09-10T10:27:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:27:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:27:10.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40061.266490 secs 2023-09-10T10:27:10.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:27:10.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:27:12.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:11 smithi098 ceph-mon[103124]: pgmap v21524: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 542 KiB/s wr, 34 op/s 2023-09-10T10:27:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:11 smithi170 ceph-mon[102832]: pgmap v21524: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 542 KiB/s wr, 34 op/s 2023-09-10T10:27:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:11 smithi082 ceph-mon[98694]: pgmap v21524: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 542 KiB/s wr, 34 op/s 2023-09-10T10:27:13.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:27:13.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:27:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:27:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:27:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:27:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:27:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:13 smithi098 ceph-mon[103124]: pgmap v21525: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 473 KiB/s wr, 32 op/s 2023-09-10T10:27:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40064.685503 secs 2023-09-10T10:27:14.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:13 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40065.914812 secs 2023-09-10T10:27:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:13 smithi170 ceph-mon[102832]: pgmap v21525: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 473 KiB/s wr, 32 op/s 2023-09-10T10:27:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40064.685503 secs 2023-09-10T10:27:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:13 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40065.914812 secs 2023-09-10T10:27:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:13 smithi082 ceph-mon[98694]: pgmap v21525: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 473 KiB/s wr, 32 op/s 2023-09-10T10:27:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40064.685503 secs 2023-09-10T10:27:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:13 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40065.914812 secs 2023-09-10T10:27:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40066.266555 secs 2023-09-10T10:27:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:14 smithi098 ceph-mon[103124]: osdmap e1414: 12 total, 12 up, 12 in 2023-09-10T10:27:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40066.266555 secs 2023-09-10T10:27:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:14 smithi170 ceph-mon[102832]: osdmap e1414: 12 total, 12 up, 12 in 2023-09-10T10:27:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40066.266555 secs 2023-09-10T10:27:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:14 smithi082 ceph-mon[98694]: osdmap e1414: 12 total, 12 up, 12 in 2023-09-10T10:27:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:15 smithi098 ceph-mon[103124]: pgmap v21526: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 473 KiB/s wr, 32 op/s 2023-09-10T10:27:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:15 smithi170 ceph-mon[102832]: pgmap v21526: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 473 KiB/s wr, 32 op/s 2023-09-10T10:27:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:15 smithi082 ceph-mon[98694]: pgmap v21526: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 473 KiB/s wr, 32 op/s 2023-09-10T10:27:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:17 smithi098 ceph-mon[103124]: pgmap v21528: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 532 KiB/s wr, 18 op/s 2023-09-10T10:27:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:17 smithi170 ceph-mon[102832]: pgmap v21528: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 532 KiB/s wr, 18 op/s 2023-09-10T10:27:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:17 smithi082 ceph-mon[98694]: pgmap v21528: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 532 KiB/s wr, 18 op/s 2023-09-10T10:27:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40069.685562 secs 2023-09-10T10:27:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40070.914894 secs 2023-09-10T10:27:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40069.685562 secs 2023-09-10T10:27:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40070.914894 secs 2023-09-10T10:27:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40069.685562 secs 2023-09-10T10:27:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40070.914894 secs 2023-09-10T10:27:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:20 smithi098 ceph-mon[103124]: pgmap v21529: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 429 KiB/s wr, 14 op/s 2023-09-10T10:27:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40071.266621 secs 2023-09-10T10:27:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:20 smithi170 ceph-mon[102832]: pgmap v21529: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 429 KiB/s wr, 14 op/s 2023-09-10T10:27:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40071.266621 secs 2023-09-10T10:27:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:20 smithi082 ceph-mon[98694]: pgmap v21529: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 429 KiB/s wr, 14 op/s 2023-09-10T10:27:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40071.266621 secs 2023-09-10T10:27:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:22 smithi170 ceph-mon[102832]: pgmap v21530: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 426 KiB/s wr, 14 op/s 2023-09-10T10:27:22.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:22 smithi082 ceph-mon[98694]: pgmap v21530: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 426 KiB/s wr, 14 op/s 2023-09-10T10:27:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:22 smithi098 ceph-mon[103124]: pgmap v21530: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 426 KiB/s wr, 14 op/s 2023-09-10T10:27:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:24 smithi170 ceph-mon[102832]: pgmap v21531: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 179 KiB/s wr, 13 op/s 2023-09-10T10:27:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40074.685680 secs 2023-09-10T10:27:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40075.914963 secs 2023-09-10T10:27:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:24 smithi082 ceph-mon[98694]: pgmap v21531: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 179 KiB/s wr, 13 op/s 2023-09-10T10:27:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40074.685680 secs 2023-09-10T10:27:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40075.914963 secs 2023-09-10T10:27:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:24 smithi098 ceph-mon[103124]: pgmap v21531: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 179 KiB/s wr, 13 op/s 2023-09-10T10:27:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40074.685680 secs 2023-09-10T10:27:24.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40075.914963 secs 2023-09-10T10:27:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40076.266685 secs 2023-09-10T10:27:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40076.266685 secs 2023-09-10T10:27:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40076.266685 secs 2023-09-10T10:27:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:26 smithi170 ceph-mon[102832]: pgmap v21532: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 179 KiB/s wr, 13 op/s 2023-09-10T10:27:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:26 smithi082 ceph-mon[98694]: pgmap v21532: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 179 KiB/s wr, 13 op/s 2023-09-10T10:27:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:26 smithi098 ceph-mon[103124]: pgmap v21532: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 179 KiB/s wr, 13 op/s 2023-09-10T10:27:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:28 smithi170 ceph-mon[102832]: pgmap v21533: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 316 KiB/s wr, 17 op/s 2023-09-10T10:27:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:28 smithi082 ceph-mon[98694]: pgmap v21533: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 316 KiB/s wr, 17 op/s 2023-09-10T10:27:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:28 smithi098 ceph-mon[103124]: pgmap v21533: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 316 KiB/s wr, 17 op/s 2023-09-10T10:27:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40079.685739 secs 2023-09-10T10:27:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40080.915022 secs 2023-09-10T10:27:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40079.685739 secs 2023-09-10T10:27:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40080.915022 secs 2023-09-10T10:27:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40079.685739 secs 2023-09-10T10:27:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40080.915022 secs 2023-09-10T10:27:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:30 smithi170 ceph-mon[102832]: pgmap v21534: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:27:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40081.266752 secs 2023-09-10T10:27:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:30 smithi082 ceph-mon[98694]: pgmap v21534: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:27:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40081.266752 secs 2023-09-10T10:27:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:30 smithi098 ceph-mon[103124]: pgmap v21534: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:27:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40081.266752 secs 2023-09-10T10:27:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:32 smithi170 ceph-mon[102832]: pgmap v21535: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:27:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:32 smithi082 ceph-mon[98694]: pgmap v21535: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:27:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:32 smithi098 ceph-mon[103124]: pgmap v21535: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:27:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:34 smithi170 ceph-mon[102832]: pgmap v21536: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 148 KiB/s wr, 15 op/s 2023-09-10T10:27:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40084.685832 secs 2023-09-10T10:27:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40085.915082 secs 2023-09-10T10:27:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:34 smithi082 ceph-mon[98694]: pgmap v21536: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 148 KiB/s wr, 15 op/s 2023-09-10T10:27:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40084.685832 secs 2023-09-10T10:27:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40085.915082 secs 2023-09-10T10:27:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:34 smithi098 ceph-mon[103124]: pgmap v21536: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 148 KiB/s wr, 15 op/s 2023-09-10T10:27:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40084.685832 secs 2023-09-10T10:27:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40085.915082 secs 2023-09-10T10:27:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40086.266817 secs 2023-09-10T10:27:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40086.266817 secs 2023-09-10T10:27:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40086.266817 secs 2023-09-10T10:27:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:36 smithi170 ceph-mon[102832]: pgmap v21537: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:27:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:36 smithi082 ceph-mon[98694]: pgmap v21537: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:27:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:36 smithi098 ceph-mon[103124]: pgmap v21537: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:27:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:38 smithi170 ceph-mon[102832]: pgmap v21538: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 292 KiB/s wr, 16 op/s 2023-09-10T10:27:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:27:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:27:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:38 smithi082 ceph-mon[98694]: pgmap v21538: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 292 KiB/s wr, 16 op/s 2023-09-10T10:27:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:27:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:27:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:38 smithi098 ceph-mon[103124]: pgmap v21538: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 292 KiB/s wr, 16 op/s 2023-09-10T10:27:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:27:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:27:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40090.915140 secs 2023-09-10T10:27:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40090.915140 secs 2023-09-10T10:27:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40090.915140 secs 2023-09-10T10:27:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:40 smithi170 ceph-mon[102832]: pgmap v21539: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:27:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40091.266886 secs 2023-09-10T10:27:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:40 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:27:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:40 smithi082 ceph-mon[98694]: pgmap v21539: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:27:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40091.266886 secs 2023-09-10T10:27:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:40 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:27:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:40 smithi098 ceph-mon[103124]: pgmap v21539: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:27:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40091.266886 secs 2023-09-10T10:27:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:40 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:27:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:41 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:27:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:41 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:27:41.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:41 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:27:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:41 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:27:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:41 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:27:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:41 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:27:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:41 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:27:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:41 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:27:41.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:41 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:27:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:42 smithi170 ceph-mon[102832]: pgmap v21540: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:27:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:42 smithi082 ceph-mon[98694]: pgmap v21540: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:27:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:42 smithi098 ceph-mon[103124]: pgmap v21540: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:27:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:44 smithi170 ceph-mon[102832]: pgmap v21541: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 467 KiB/s wr, 16 op/s 2023-09-10T10:27:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40095.915199 secs 2023-09-10T10:27:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:44 smithi082 ceph-mon[98694]: pgmap v21541: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 467 KiB/s wr, 16 op/s 2023-09-10T10:27:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40095.915199 secs 2023-09-10T10:27:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:44 smithi098 ceph-mon[103124]: pgmap v21541: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 467 KiB/s wr, 16 op/s 2023-09-10T10:27:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40095.915199 secs 2023-09-10T10:27:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40096.266952 secs 2023-09-10T10:27:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40096.266952 secs 2023-09-10T10:27:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40096.266952 secs 2023-09-10T10:27:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:46 smithi170 ceph-mon[102832]: pgmap v21542: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 467 KiB/s wr, 12 op/s 2023-09-10T10:27:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:27:46.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:27:46.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:46 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:27:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:46 smithi082 ceph-mon[98694]: pgmap v21542: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 467 KiB/s wr, 12 op/s 2023-09-10T10:27:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:27:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:27:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:46 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:27:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:46 smithi098 ceph-mon[103124]: pgmap v21542: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 467 KiB/s wr, 12 op/s 2023-09-10T10:27:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:27:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:27:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:46 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:27:47.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:27:47.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:27:47.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:27:47.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:27:47.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:27:47.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:27:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:27:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:27:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:27:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:48 smithi170 ceph-mon[102832]: pgmap v21543: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 783 KiB/s wr, 18 op/s 2023-09-10T10:27:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:48 smithi082 ceph-mon[98694]: pgmap v21543: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 783 KiB/s wr, 18 op/s 2023-09-10T10:27:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:48 smithi098 ceph-mon[103124]: pgmap v21543: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 783 KiB/s wr, 18 op/s 2023-09-10T10:27:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40100.915260 secs 2023-09-10T10:27:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40100.915260 secs 2023-09-10T10:27:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40100.915260 secs 2023-09-10T10:27:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:50 smithi170 ceph-mon[102832]: pgmap v21544: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 639 KiB/s wr, 12 op/s 2023-09-10T10:27:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40101.267014 secs 2023-09-10T10:27:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:27:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:27:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:50 smithi082 ceph-mon[98694]: pgmap v21544: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 639 KiB/s wr, 12 op/s 2023-09-10T10:27:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40101.267014 secs 2023-09-10T10:27:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:27:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:27:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:50 smithi098 ceph-mon[103124]: pgmap v21544: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 639 KiB/s wr, 12 op/s 2023-09-10T10:27:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40101.267014 secs 2023-09-10T10:27:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:27:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:27:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:52 smithi170 ceph-mon[102832]: pgmap v21545: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 639 KiB/s wr, 12 op/s 2023-09-10T10:27:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:52 smithi082 ceph-mon[98694]: pgmap v21545: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 639 KiB/s wr, 12 op/s 2023-09-10T10:27:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:52 smithi098 ceph-mon[103124]: pgmap v21545: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 639 KiB/s wr, 12 op/s 2023-09-10T10:27:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:27:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:27:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:27:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:27:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:27:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:27:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:54 smithi170 ceph-mon[102832]: pgmap v21546: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 992 KiB/s wr, 20 op/s 2023-09-10T10:27:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40105.915318 secs 2023-09-10T10:27:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:54 smithi082 ceph-mon[98694]: pgmap v21546: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 992 KiB/s wr, 20 op/s 2023-09-10T10:27:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40105.915318 secs 2023-09-10T10:27:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:54 smithi098 ceph-mon[103124]: pgmap v21546: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 992 KiB/s wr, 20 op/s 2023-09-10T10:27:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40105.915318 secs 2023-09-10T10:27:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40106.267077 secs 2023-09-10T10:27:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40106.267077 secs 2023-09-10T10:27:55.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40106.267077 secs 2023-09-10T10:27:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:56 smithi170 ceph-mon[102832]: pgmap v21547: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 669 KiB/s wr, 14 op/s 2023-09-10T10:27:56.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:56 smithi082 ceph-mon[98694]: pgmap v21547: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 669 KiB/s wr, 14 op/s 2023-09-10T10:27:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:56 smithi098 ceph-mon[103124]: pgmap v21547: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 669 KiB/s wr, 14 op/s 2023-09-10T10:27:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:58 smithi170 ceph-mon[102832]: pgmap v21548: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 889 KiB/s wr, 20 op/s 2023-09-10T10:27:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:27:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:27:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:58 smithi082 ceph-mon[98694]: pgmap v21548: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 889 KiB/s wr, 20 op/s 2023-09-10T10:27:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:27:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:27:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:58 smithi098 ceph-mon[103124]: pgmap v21548: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 889 KiB/s wr, 20 op/s 2023-09-10T10:27:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:27:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:27:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40109.686236 secs 2023-09-10T10:27:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:27:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40110.915379 secs 2023-09-10T10:27:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40109.686236 secs 2023-09-10T10:27:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:27:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40110.915379 secs 2023-09-10T10:27:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40109.686236 secs 2023-09-10T10:27:59.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:27:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40110.915379 secs 2023-09-10T10:28:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:00 smithi170 ceph-mon[102832]: pgmap v21549: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 572 KiB/s wr, 14 op/s 2023-09-10T10:28:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40111.267141 secs 2023-09-10T10:28:00.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:00 smithi082 ceph-mon[98694]: pgmap v21549: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 572 KiB/s wr, 14 op/s 2023-09-10T10:28:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40111.267141 secs 2023-09-10T10:28:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:00 smithi098 ceph-mon[103124]: pgmap v21549: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 572 KiB/s wr, 14 op/s 2023-09-10T10:28:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40111.267141 secs 2023-09-10T10:28:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:01 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:28:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:01 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:28:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:01 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:28:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:01 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:28:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:01 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:28:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:01 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:28:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:01 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:28:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:01 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:28:01.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:01 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:28:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:02 smithi170 ceph-mon[102832]: pgmap v21550: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 572 KiB/s wr, 14 op/s 2023-09-10T10:28:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:02 smithi082 ceph-mon[98694]: pgmap v21550: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 572 KiB/s wr, 14 op/s 2023-09-10T10:28:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:02 smithi098 ceph-mon[103124]: pgmap v21550: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 572 KiB/s wr, 14 op/s 2023-09-10T10:28:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:04 smithi170 ceph-mon[102832]: pgmap v21551: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 837 KiB/s wr, 21 op/s 2023-09-10T10:28:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40114.686297 secs 2023-09-10T10:28:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40115.915436 secs 2023-09-10T10:28:04.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:04 smithi082 ceph-mon[98694]: pgmap v21551: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 837 KiB/s wr, 21 op/s 2023-09-10T10:28:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40114.686297 secs 2023-09-10T10:28:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40115.915436 secs 2023-09-10T10:28:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:04 smithi098 ceph-mon[103124]: pgmap v21551: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 837 KiB/s wr, 21 op/s 2023-09-10T10:28:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40114.686297 secs 2023-09-10T10:28:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40115.915436 secs 2023-09-10T10:28:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40116.267206 secs 2023-09-10T10:28:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40116.267206 secs 2023-09-10T10:28:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40116.267206 secs 2023-09-10T10:28:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:06 smithi170 ceph-mon[102832]: pgmap v21552: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 484 KiB/s wr, 14 op/s 2023-09-10T10:28:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:06 smithi082 ceph-mon[98694]: pgmap v21552: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 484 KiB/s wr, 14 op/s 2023-09-10T10:28:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:06 smithi098 ceph-mon[103124]: pgmap v21552: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 484 KiB/s wr, 14 op/s 2023-09-10T10:28:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:08 smithi170 ceph-mon[102832]: pgmap v21553: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 672 KiB/s wr, 35 op/s 2023-09-10T10:28:08.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:08 smithi082 ceph-mon[98694]: pgmap v21553: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 672 KiB/s wr, 35 op/s 2023-09-10T10:28:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:08 smithi098 ceph-mon[103124]: pgmap v21553: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 672 KiB/s wr, 35 op/s 2023-09-10T10:28:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40119.686362 secs 2023-09-10T10:28:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:09 smithi170 ceph-mon[102832]: osdmap e1415: 12 total, 12 up, 12 in 2023-09-10T10:28:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40120.915498 secs 2023-09-10T10:28:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40119.686362 secs 2023-09-10T10:28:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:09 smithi082 ceph-mon[98694]: osdmap e1415: 12 total, 12 up, 12 in 2023-09-10T10:28:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40120.915498 secs 2023-09-10T10:28:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40119.686362 secs 2023-09-10T10:28:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:09 smithi098 ceph-mon[103124]: osdmap e1415: 12 total, 12 up, 12 in 2023-09-10T10:28:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40120.915498 secs 2023-09-10T10:28:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:10 smithi170 ceph-mon[102832]: pgmap v21555: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 543 KiB/s wr, 33 op/s 2023-09-10T10:28:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40121.267271 secs 2023-09-10T10:28:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:28:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:28:10.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:10 smithi082 ceph-mon[98694]: pgmap v21555: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 543 KiB/s wr, 33 op/s 2023-09-10T10:28:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40121.267271 secs 2023-09-10T10:28:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:28:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:28:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:10 smithi098 ceph-mon[103124]: pgmap v21555: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 543 KiB/s wr, 33 op/s 2023-09-10T10:28:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40121.267271 secs 2023-09-10T10:28:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:28:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:28:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:12 smithi170 ceph-mon[102832]: pgmap v21556: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 543 KiB/s wr, 33 op/s 2023-09-10T10:28:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:12 smithi082 ceph-mon[98694]: pgmap v21556: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 543 KiB/s wr, 33 op/s 2023-09-10T10:28:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:12 smithi098 ceph-mon[103124]: pgmap v21556: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 543 KiB/s wr, 33 op/s 2023-09-10T10:28:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:28:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:28:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:28:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:28:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:28:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:28:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:14 smithi170 ceph-mon[102832]: pgmap v21557: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 473 KiB/s wr, 33 op/s 2023-09-10T10:28:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40124.686424 secs 2023-09-10T10:28:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40125.915557 secs 2023-09-10T10:28:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:14 smithi082 ceph-mon[98694]: pgmap v21557: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 473 KiB/s wr, 33 op/s 2023-09-10T10:28:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40124.686424 secs 2023-09-10T10:28:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40125.915557 secs 2023-09-10T10:28:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:14 smithi098 ceph-mon[103124]: pgmap v21557: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 473 KiB/s wr, 33 op/s 2023-09-10T10:28:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40124.686424 secs 2023-09-10T10:28:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40125.915557 secs 2023-09-10T10:28:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40126.267336 secs 2023-09-10T10:28:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:15 smithi170 ceph-mon[102832]: osdmap e1416: 12 total, 12 up, 12 in 2023-09-10T10:28:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40126.267336 secs 2023-09-10T10:28:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:15 smithi082 ceph-mon[98694]: osdmap e1416: 12 total, 12 up, 12 in 2023-09-10T10:28:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40126.267336 secs 2023-09-10T10:28:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:15 smithi098 ceph-mon[103124]: osdmap e1416: 12 total, 12 up, 12 in 2023-09-10T10:28:16.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:16 smithi170 ceph-mon[102832]: pgmap v21558: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 473 KiB/s wr, 33 op/s 2023-09-10T10:28:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:16 smithi082 ceph-mon[98694]: pgmap v21558: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 473 KiB/s wr, 33 op/s 2023-09-10T10:28:16.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:16 smithi098 ceph-mon[103124]: pgmap v21558: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 473 KiB/s wr, 33 op/s 2023-09-10T10:28:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:18 smithi170 ceph-mon[102832]: pgmap v21560: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 480 KiB/s wr, 18 op/s 2023-09-10T10:28:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:18 smithi082 ceph-mon[98694]: pgmap v21560: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 480 KiB/s wr, 18 op/s 2023-09-10T10:28:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:18 smithi098 ceph-mon[103124]: pgmap v21560: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 480 KiB/s wr, 18 op/s 2023-09-10T10:28:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40129.686485 secs 2023-09-10T10:28:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40130.915616 secs 2023-09-10T10:28:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40129.686485 secs 2023-09-10T10:28:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40130.915616 secs 2023-09-10T10:28:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40129.686485 secs 2023-09-10T10:28:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40130.915616 secs 2023-09-10T10:28:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:20 smithi082 ceph-mon[98694]: pgmap v21561: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 390 KiB/s wr, 15 op/s 2023-09-10T10:28:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40131.267400 secs 2023-09-10T10:28:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:20 smithi098 ceph-mon[103124]: pgmap v21561: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 390 KiB/s wr, 15 op/s 2023-09-10T10:28:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40131.267400 secs 2023-09-10T10:28:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:20 smithi170 ceph-mon[102832]: pgmap v21561: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 390 KiB/s wr, 15 op/s 2023-09-10T10:28:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40131.267400 secs 2023-09-10T10:28:22.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:22 smithi082 ceph-mon[98694]: pgmap v21562: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 390 KiB/s wr, 15 op/s 2023-09-10T10:28:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:22 smithi098 ceph-mon[103124]: pgmap v21562: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 390 KiB/s wr, 15 op/s 2023-09-10T10:28:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:22 smithi170 ceph-mon[102832]: pgmap v21562: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 390 KiB/s wr, 15 op/s 2023-09-10T10:28:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:24 smithi082 ceph-mon[98694]: pgmap v21563: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 13 op/s 2023-09-10T10:28:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40134.686545 secs 2023-09-10T10:28:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40135.915693 secs 2023-09-10T10:28:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:24 smithi098 ceph-mon[103124]: pgmap v21563: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 13 op/s 2023-09-10T10:28:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40134.686545 secs 2023-09-10T10:28:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40135.915693 secs 2023-09-10T10:28:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:24 smithi170 ceph-mon[102832]: pgmap v21563: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 13 op/s 2023-09-10T10:28:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40134.686545 secs 2023-09-10T10:28:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40135.915693 secs 2023-09-10T10:28:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40136.267467 secs 2023-09-10T10:28:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40136.267467 secs 2023-09-10T10:28:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40136.267467 secs 2023-09-10T10:28:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:26 smithi082 ceph-mon[98694]: pgmap v21564: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 13 op/s 2023-09-10T10:28:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:26 smithi098 ceph-mon[103124]: pgmap v21564: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 13 op/s 2023-09-10T10:28:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:26 smithi170 ceph-mon[102832]: pgmap v21564: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 143 KiB/s wr, 13 op/s 2023-09-10T10:28:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:28 smithi082 ceph-mon[98694]: pgmap v21565: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 284 KiB/s wr, 17 op/s 2023-09-10T10:28:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:28 smithi098 ceph-mon[103124]: pgmap v21565: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 284 KiB/s wr, 17 op/s 2023-09-10T10:28:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:28 smithi170 ceph-mon[102832]: pgmap v21565: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 284 KiB/s wr, 17 op/s 2023-09-10T10:28:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40139.686602 secs 2023-09-10T10:28:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40140.915751 secs 2023-09-10T10:28:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40139.686602 secs 2023-09-10T10:28:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40140.915751 secs 2023-09-10T10:28:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40139.686602 secs 2023-09-10T10:28:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40140.915751 secs 2023-09-10T10:28:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:30 smithi082 ceph-mon[98694]: pgmap v21566: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:28:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40141.267487 secs 2023-09-10T10:28:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:30 smithi098 ceph-mon[103124]: pgmap v21566: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:28:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40141.267487 secs 2023-09-10T10:28:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:30 smithi170 ceph-mon[102832]: pgmap v21566: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:28:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40141.267487 secs 2023-09-10T10:28:31.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:31 smithi098 ceph-mon[103124]: pgmap v21567: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:28:31.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:31 smithi170 ceph-mon[102832]: pgmap v21567: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:28:31.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:31 smithi082 ceph-mon[98694]: pgmap v21567: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 10 op/s 2023-09-10T10:28:33.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:33 smithi170 ceph-mon[102832]: pgmap v21568: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 148 KiB/s wr, 15 op/s 2023-09-10T10:28:33.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:33 smithi082 ceph-mon[98694]: pgmap v21568: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 148 KiB/s wr, 15 op/s 2023-09-10T10:28:33.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:33 smithi098 ceph-mon[103124]: pgmap v21568: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 148 KiB/s wr, 15 op/s 2023-09-10T10:28:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40144.686665 secs 2023-09-10T10:28:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40145.915819 secs 2023-09-10T10:28:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40144.686665 secs 2023-09-10T10:28:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40145.915819 secs 2023-09-10T10:28:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40144.686665 secs 2023-09-10T10:28:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40145.915819 secs 2023-09-10T10:28:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40146.267550 secs 2023-09-10T10:28:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:35 smithi170 ceph-mon[102832]: pgmap v21569: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 9 op/s 2023-09-10T10:28:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40146.267550 secs 2023-09-10T10:28:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:35 smithi082 ceph-mon[98694]: pgmap v21569: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 9 op/s 2023-09-10T10:28:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40146.267550 secs 2023-09-10T10:28:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:35 smithi098 ceph-mon[103124]: pgmap v21569: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 9 op/s 2023-09-10T10:28:37.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:37 smithi170 ceph-mon[102832]: pgmap v21570: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 292 KiB/s wr, 15 op/s 2023-09-10T10:28:37.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:37 smithi082 ceph-mon[98694]: pgmap v21570: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 292 KiB/s wr, 15 op/s 2023-09-10T10:28:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:37 smithi098 ceph-mon[103124]: pgmap v21570: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 292 KiB/s wr, 15 op/s 2023-09-10T10:28:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:28:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:28:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:28:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:28:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:28:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:28:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:39 smithi170 ceph-mon[102832]: pgmap v21571: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:28:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40150.915870 secs 2023-09-10T10:28:39.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:28:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:39 smithi082 ceph-mon[98694]: pgmap v21571: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:28:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40150.915870 secs 2023-09-10T10:28:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:28:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:39 smithi098 ceph-mon[103124]: pgmap v21571: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:28:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40150.915870 secs 2023-09-10T10:28:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:28:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40151.267616 secs 2023-09-10T10:28:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:28:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:28:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:28:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40151.267616 secs 2023-09-10T10:28:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:28:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:28:40.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:28:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40151.267616 secs 2023-09-10T10:28:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:28:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:28:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:28:42.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:41 smithi098 ceph-mon[103124]: pgmap v21572: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:28:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:41 smithi170 ceph-mon[102832]: pgmap v21572: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:28:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:41 smithi082 ceph-mon[98694]: pgmap v21572: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:28:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:44 smithi098 ceph-mon[103124]: pgmap v21573: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 468 KiB/s wr, 17 op/s 2023-09-10T10:28:44.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40155.915955 secs 2023-09-10T10:28:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:44 smithi170 ceph-mon[102832]: pgmap v21573: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 468 KiB/s wr, 17 op/s 2023-09-10T10:28:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40155.915955 secs 2023-09-10T10:28:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:44 smithi082 ceph-mon[98694]: pgmap v21573: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 468 KiB/s wr, 17 op/s 2023-09-10T10:28:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40155.915955 secs 2023-09-10T10:28:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40156.267681 secs 2023-09-10T10:28:45.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40156.267681 secs 2023-09-10T10:28:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40156.267681 secs 2023-09-10T10:28:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:46 smithi170 ceph-mon[102832]: pgmap v21574: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 468 KiB/s wr, 12 op/s 2023-09-10T10:28:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:28:46.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:28:46.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:46 smithi082 ceph-mon[98694]: pgmap v21574: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 468 KiB/s wr, 12 op/s 2023-09-10T10:28:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:28:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:28:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:46 smithi098 ceph-mon[103124]: pgmap v21574: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 468 KiB/s wr, 12 op/s 2023-09-10T10:28:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:28:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:28:47.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:47 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:28:47.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:47 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:28:47.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:47 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:28:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:48 smithi170 ceph-mon[102832]: pgmap v21575: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 785 KiB/s wr, 20 op/s 2023-09-10T10:28:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:48 smithi082 ceph-mon[98694]: pgmap v21575: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 785 KiB/s wr, 20 op/s 2023-09-10T10:28:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:48 smithi098 ceph-mon[103124]: pgmap v21575: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 785 KiB/s wr, 20 op/s 2023-09-10T10:28:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40160.916014 secs 2023-09-10T10:28:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40160.916014 secs 2023-09-10T10:28:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40160.916014 secs 2023-09-10T10:28:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:50 smithi170 ceph-mon[102832]: pgmap v21576: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 641 KiB/s wr, 14 op/s 2023-09-10T10:28:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40161.267751 secs 2023-09-10T10:28:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:28:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:28:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:50.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:50.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:50.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:50.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:50 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:50 smithi082 ceph-mon[98694]: pgmap v21576: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 641 KiB/s wr, 14 op/s 2023-09-10T10:28:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40161.267751 secs 2023-09-10T10:28:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:28:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:28:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:50.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:50.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:50.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:50.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:50.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:50 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:50 smithi098 ceph-mon[103124]: pgmap v21576: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 641 KiB/s wr, 14 op/s 2023-09-10T10:28:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40161.267751 secs 2023-09-10T10:28:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:28:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:28:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:50.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:50.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:50 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:51.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:51.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:28:51.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:51.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:28:51.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:28:51.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:28:51.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:28:51.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:51 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T10:28:51.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:51 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T10:28:51.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:51.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:51.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:28:51.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:28:51.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:28:51.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:28:51.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:28:51.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:28:51.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:28:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:51.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:28:51.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:28:51.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:28:51.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:28:51.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:51 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T10:28:51.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:51 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T10:28:51.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:51.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:51.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:28:51.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:28:51.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:28:51.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:28:51.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:28:51.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:28:51.462 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:51.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:51.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:51.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:28:51.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:51.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:28:51.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:28:51.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:28:51.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:28:51.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:51 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T10:28:51.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:51 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T10:28:51.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:51.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:51.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:28:51.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:28:51.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:28:51.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:28:51.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:28:51.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:28:51.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:28:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:52 smithi170 ceph-mon[102832]: pgmap v21577: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 641 KiB/s wr, 14 op/s 2023-09-10T10:28:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:52 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T10:28:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:52 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T10:28:52.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:52 smithi082 ceph-mon[98694]: pgmap v21577: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 641 KiB/s wr, 14 op/s 2023-09-10T10:28:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:52 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T10:28:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:52 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T10:28:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:52 smithi098 ceph-mon[103124]: pgmap v21577: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 641 KiB/s wr, 14 op/s 2023-09-10T10:28:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:52 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T10:28:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:52 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T10:28:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:28:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:28:53.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:28:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:28:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:28:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:28:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:54 smithi170 ceph-mon[102832]: pgmap v21578: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 989 KiB/s wr, 21 op/s 2023-09-10T10:28:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40165.916094 secs 2023-09-10T10:28:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:54 smithi082 ceph-mon[98694]: pgmap v21578: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 989 KiB/s wr, 21 op/s 2023-09-10T10:28:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40165.916094 secs 2023-09-10T10:28:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:54 smithi098 ceph-mon[103124]: pgmap v21578: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 989 KiB/s wr, 21 op/s 2023-09-10T10:28:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40165.916094 secs 2023-09-10T10:28:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40166.267814 secs 2023-09-10T10:28:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40166.267814 secs 2023-09-10T10:28:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40166.267814 secs 2023-09-10T10:28:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:56 smithi082 ceph-mon[98694]: pgmap v21579: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 665 KiB/s wr, 15 op/s 2023-09-10T10:28:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:56 smithi098 ceph-mon[103124]: pgmap v21579: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 665 KiB/s wr, 15 op/s 2023-09-10T10:28:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:56 smithi170 ceph-mon[102832]: pgmap v21579: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 665 KiB/s wr, 15 op/s 2023-09-10T10:28:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:58 smithi082 ceph-mon[98694]: pgmap v21580: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 890 KiB/s wr, 21 op/s 2023-09-10T10:28:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:28:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:28:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:58 smithi098 ceph-mon[103124]: pgmap v21580: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 890 KiB/s wr, 21 op/s 2023-09-10T10:28:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:28:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:28:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:58 smithi170 ceph-mon[102832]: pgmap v21580: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 890 KiB/s wr, 21 op/s 2023-09-10T10:28:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:28:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:28:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:28:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40170.916152 secs 2023-09-10T10:28:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:28:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40170.916152 secs 2023-09-10T10:28:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:28:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40170.916152 secs 2023-09-10T10:29:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:00 smithi082 ceph-mon[98694]: pgmap v21581: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 573 KiB/s wr, 13 op/s 2023-09-10T10:29:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40171.267879 secs 2023-09-10T10:29:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:00 smithi098 ceph-mon[103124]: pgmap v21581: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 573 KiB/s wr, 13 op/s 2023-09-10T10:29:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40171.267879 secs 2023-09-10T10:29:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:00 smithi170 ceph-mon[102832]: pgmap v21581: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 573 KiB/s wr, 13 op/s 2023-09-10T10:29:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40171.267879 secs 2023-09-10T10:29:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:01 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:29:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:01 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:01 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:29:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:01 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:01.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:01 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:29:01.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:01 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:02 smithi082 ceph-mon[98694]: pgmap v21582: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 750 KiB/s wr, 17 op/s 2023-09-10T10:29:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:02 smithi098 ceph-mon[103124]: pgmap v21582: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 750 KiB/s wr, 17 op/s 2023-09-10T10:29:02.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:02 smithi170 ceph-mon[102832]: pgmap v21582: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 750 KiB/s wr, 17 op/s 2023-09-10T10:29:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:03 smithi098 ceph-mon[103124]: pgmap v21583: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 838 KiB/s wr, 20 op/s 2023-09-10T10:29:03.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:03 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:29:03.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:03 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:29:03.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:03 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:03 smithi170 ceph-mon[102832]: pgmap v21583: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 838 KiB/s wr, 20 op/s 2023-09-10T10:29:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:03 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:29:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:03 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:29:03.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:03 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:03 smithi082 ceph-mon[98694]: pgmap v21583: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 838 KiB/s wr, 20 op/s 2023-09-10T10:29:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:03 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:29:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:03 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:29:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:03 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40174.687130 secs 2023-09-10T10:29:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40175.916212 secs 2023-09-10T10:29:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40174.687130 secs 2023-09-10T10:29:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40175.916212 secs 2023-09-10T10:29:04.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40174.687130 secs 2023-09-10T10:29:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40175.916212 secs 2023-09-10T10:29:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40176.267946 secs 2023-09-10T10:29:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:05 smithi170 ceph-mon[102832]: pgmap v21584: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 490 KiB/s wr, 13 op/s 2023-09-10T10:29:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40176.267946 secs 2023-09-10T10:29:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:05 smithi082 ceph-mon[98694]: pgmap v21584: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 490 KiB/s wr, 13 op/s 2023-09-10T10:29:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40176.267946 secs 2023-09-10T10:29:05.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:05 smithi098 ceph-mon[103124]: pgmap v21584: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 490 KiB/s wr, 13 op/s 2023-09-10T10:29:07.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:07 smithi170 ceph-mon[102832]: pgmap v21585: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 788 KiB/s wr, 33 op/s 2023-09-10T10:29:07.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:07 smithi082 ceph-mon[98694]: pgmap v21585: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 788 KiB/s wr, 33 op/s 2023-09-10T10:29:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:07 smithi098 ceph-mon[103124]: pgmap v21585: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 788 KiB/s wr, 33 op/s 2023-09-10T10:29:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40179.687188 secs 2023-09-10T10:29:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:09 smithi170 ceph-mon[102832]: pgmap v21586: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 563 KiB/s wr, 27 op/s 2023-09-10T10:29:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:09 smithi170 ceph-mon[102832]: osdmap e1417: 12 total, 12 up, 12 in 2023-09-10T10:29:09.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40180.916273 secs 2023-09-10T10:29:09.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40179.687188 secs 2023-09-10T10:29:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:09 smithi082 ceph-mon[98694]: pgmap v21586: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 563 KiB/s wr, 27 op/s 2023-09-10T10:29:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:09 smithi082 ceph-mon[98694]: osdmap e1417: 12 total, 12 up, 12 in 2023-09-10T10:29:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40180.916273 secs 2023-09-10T10:29:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40179.687188 secs 2023-09-10T10:29:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:09 smithi098 ceph-mon[103124]: pgmap v21586: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 563 KiB/s wr, 27 op/s 2023-09-10T10:29:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:09 smithi098 ceph-mon[103124]: osdmap e1417: 12 total, 12 up, 12 in 2023-09-10T10:29:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40180.916273 secs 2023-09-10T10:29:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40181.268010 secs 2023-09-10T10:29:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:29:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40181.268010 secs 2023-09-10T10:29:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:29:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40181.268010 secs 2023-09-10T10:29:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:29:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:12 smithi170 ceph-mon[102832]: pgmap v21588: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 751 KiB/s wr, 32 op/s 2023-09-10T10:29:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:12 smithi082 ceph-mon[98694]: pgmap v21588: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 751 KiB/s wr, 32 op/s 2023-09-10T10:29:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:12 smithi098 ceph-mon[103124]: pgmap v21588: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 751 KiB/s wr, 32 op/s 2023-09-10T10:29:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:29:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:13.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:29:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:29:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:14 smithi170 ceph-mon[102832]: pgmap v21589: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 645 KiB/s wr, 32 op/s 2023-09-10T10:29:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40184.687245 secs 2023-09-10T10:29:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40185.916336 secs 2023-09-10T10:29:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:14 smithi082 ceph-mon[98694]: pgmap v21589: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 645 KiB/s wr, 32 op/s 2023-09-10T10:29:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40184.687245 secs 2023-09-10T10:29:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40185.916336 secs 2023-09-10T10:29:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:14 smithi098 ceph-mon[103124]: pgmap v21589: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 645 KiB/s wr, 32 op/s 2023-09-10T10:29:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40184.687245 secs 2023-09-10T10:29:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40185.916336 secs 2023-09-10T10:29:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40186.268079 secs 2023-09-10T10:29:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:15 smithi170 ceph-mon[102832]: osdmap e1418: 12 total, 12 up, 12 in 2023-09-10T10:29:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40186.268079 secs 2023-09-10T10:29:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:15 smithi082 ceph-mon[98694]: osdmap e1418: 12 total, 12 up, 12 in 2023-09-10T10:29:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40186.268079 secs 2023-09-10T10:29:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:15 smithi098 ceph-mon[103124]: osdmap e1418: 12 total, 12 up, 12 in 2023-09-10T10:29:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:16 smithi170 ceph-mon[102832]: pgmap v21590: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 645 KiB/s wr, 32 op/s 2023-09-10T10:29:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:16 smithi082 ceph-mon[98694]: pgmap v21590: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 645 KiB/s wr, 32 op/s 2023-09-10T10:29:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:16 smithi098 ceph-mon[103124]: pgmap v21590: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 645 KiB/s wr, 32 op/s 2023-09-10T10:29:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:18 smithi170 ceph-mon[102832]: pgmap v21592: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 583 KiB/s wr, 18 op/s 2023-09-10T10:29:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:18 smithi082 ceph-mon[98694]: pgmap v21592: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 583 KiB/s wr, 18 op/s 2023-09-10T10:29:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:18 smithi098 ceph-mon[103124]: pgmap v21592: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 583 KiB/s wr, 18 op/s 2023-09-10T10:29:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40189.687317 secs 2023-09-10T10:29:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40190.916393 secs 2023-09-10T10:29:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40189.687317 secs 2023-09-10T10:29:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40190.916393 secs 2023-09-10T10:29:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40189.687317 secs 2023-09-10T10:29:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40190.916393 secs 2023-09-10T10:29:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:20 smithi170 ceph-mon[102832]: pgmap v21593: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 469 KiB/s wr, 14 op/s 2023-09-10T10:29:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40191.268144 secs 2023-09-10T10:29:20.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:20 smithi082 ceph-mon[98694]: pgmap v21593: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 469 KiB/s wr, 14 op/s 2023-09-10T10:29:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40191.268144 secs 2023-09-10T10:29:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:20 smithi098 ceph-mon[103124]: pgmap v21593: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 469 KiB/s wr, 14 op/s 2023-09-10T10:29:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40191.268144 secs 2023-09-10T10:29:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:22 smithi170 ceph-mon[102832]: pgmap v21594: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 179 KiB/s wr, 14 op/s 2023-09-10T10:29:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:22 smithi082 ceph-mon[98694]: pgmap v21594: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 179 KiB/s wr, 14 op/s 2023-09-10T10:29:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:22 smithi098 ceph-mon[103124]: pgmap v21594: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 179 KiB/s wr, 14 op/s 2023-09-10T10:29:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:24 smithi170 ceph-mon[102832]: pgmap v21595: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 13 op/s 2023-09-10T10:29:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40194.687406 secs 2023-09-10T10:29:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40195.916452 secs 2023-09-10T10:29:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:24 smithi082 ceph-mon[98694]: pgmap v21595: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 13 op/s 2023-09-10T10:29:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40194.687406 secs 2023-09-10T10:29:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40195.916452 secs 2023-09-10T10:29:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:24 smithi098 ceph-mon[103124]: pgmap v21595: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 13 op/s 2023-09-10T10:29:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40194.687406 secs 2023-09-10T10:29:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40195.916452 secs 2023-09-10T10:29:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40196.268209 secs 2023-09-10T10:29:25.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40196.268209 secs 2023-09-10T10:29:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40196.268209 secs 2023-09-10T10:29:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:26 smithi170 ceph-mon[102832]: pgmap v21596: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 13 op/s 2023-09-10T10:29:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:26 smithi082 ceph-mon[98694]: pgmap v21596: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 13 op/s 2023-09-10T10:29:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:26 smithi098 ceph-mon[103124]: pgmap v21596: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 179 KiB/s wr, 13 op/s 2023-09-10T10:29:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:28 smithi170 ceph-mon[102832]: pgmap v21597: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 316 KiB/s wr, 18 op/s 2023-09-10T10:29:28.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:28 smithi082 ceph-mon[98694]: pgmap v21597: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 316 KiB/s wr, 18 op/s 2023-09-10T10:29:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:28 smithi098 ceph-mon[103124]: pgmap v21597: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 316 KiB/s wr, 18 op/s 2023-09-10T10:29:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40199.687495 secs 2023-09-10T10:29:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40200.916492 secs 2023-09-10T10:29:29.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40199.687495 secs 2023-09-10T10:29:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40200.916492 secs 2023-09-10T10:29:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40199.687495 secs 2023-09-10T10:29:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40200.916492 secs 2023-09-10T10:29:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:30 smithi170 ceph-mon[102832]: pgmap v21598: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:29:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40201.268275 secs 2023-09-10T10:29:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:30 smithi082 ceph-mon[98694]: pgmap v21598: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:29:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40201.268275 secs 2023-09-10T10:29:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:30 smithi098 ceph-mon[103124]: pgmap v21598: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:29:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40201.268275 secs 2023-09-10T10:29:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:32 smithi170 ceph-mon[102832]: pgmap v21599: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 14 op/s 2023-09-10T10:29:32.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:32 smithi082 ceph-mon[98694]: pgmap v21599: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 14 op/s 2023-09-10T10:29:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:32 smithi098 ceph-mon[103124]: pgmap v21599: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 14 op/s 2023-09-10T10:29:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:34 smithi170 ceph-mon[102832]: pgmap v21600: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 13 op/s 2023-09-10T10:29:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40204.687610 secs 2023-09-10T10:29:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:29:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40205.916549 secs 2023-09-10T10:29:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:34 smithi082 ceph-mon[98694]: pgmap v21600: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 13 op/s 2023-09-10T10:29:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40204.687610 secs 2023-09-10T10:29:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:29:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40205.916549 secs 2023-09-10T10:29:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:34 smithi098 ceph-mon[103124]: pgmap v21600: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 13 op/s 2023-09-10T10:29:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40204.687610 secs 2023-09-10T10:29:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:29:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40205.916549 secs 2023-09-10T10:29:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40206.268339 secs 2023-09-10T10:29:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40206.268339 secs 2023-09-10T10:29:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40206.268339 secs 2023-09-10T10:29:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:36 smithi170 ceph-mon[102832]: pgmap v21601: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T10:29:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:36 smithi082 ceph-mon[98694]: pgmap v21601: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T10:29:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:36 smithi098 ceph-mon[103124]: pgmap v21601: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T10:29:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:29:37.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:37.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:29:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:29:37.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:38 smithi170 ceph-mon[102832]: pgmap v21602: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 356 KiB/s wr, 17 op/s 2023-09-10T10:29:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:29:38.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:38.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:38 smithi082 ceph-mon[98694]: pgmap v21602: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 356 KiB/s wr, 17 op/s 2023-09-10T10:29:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:29:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:38 smithi098 ceph-mon[103124]: pgmap v21602: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 356 KiB/s wr, 17 op/s 2023-09-10T10:29:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:29:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40210.916610 secs 2023-09-10T10:29:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40210.916610 secs 2023-09-10T10:29:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40210.916610 secs 2023-09-10T10:29:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:40 smithi170 ceph-mon[102832]: pgmap v21603: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 208 KiB/s wr, 11 op/s 2023-09-10T10:29:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40211.268403 secs 2023-09-10T10:29:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:40 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:29:40.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:40 smithi082 ceph-mon[98694]: pgmap v21603: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 208 KiB/s wr, 11 op/s 2023-09-10T10:29:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40211.268403 secs 2023-09-10T10:29:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:40 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:29:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:40 smithi098 ceph-mon[103124]: pgmap v21603: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 208 KiB/s wr, 11 op/s 2023-09-10T10:29:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40211.268403 secs 2023-09-10T10:29:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:40 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:29:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:41 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:29:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:41 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:29:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:41 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:41 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:29:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:41 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:29:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:41 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:41 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:29:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:41 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:29:41.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:41 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:42 smithi170 ceph-mon[102832]: pgmap v21604: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 580 KiB/s wr, 16 op/s 2023-09-10T10:29:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:42 smithi082 ceph-mon[98694]: pgmap v21604: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 580 KiB/s wr, 16 op/s 2023-09-10T10:29:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:42 smithi098 ceph-mon[103124]: pgmap v21604: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 580 KiB/s wr, 16 op/s 2023-09-10T10:29:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:44 smithi170 ceph-mon[102832]: pgmap v21605: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 580 KiB/s wr, 14 op/s 2023-09-10T10:29:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40215.916672 secs 2023-09-10T10:29:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:44 smithi082 ceph-mon[98694]: pgmap v21605: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 580 KiB/s wr, 14 op/s 2023-09-10T10:29:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40215.916672 secs 2023-09-10T10:29:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:44 smithi098 ceph-mon[103124]: pgmap v21605: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 580 KiB/s wr, 14 op/s 2023-09-10T10:29:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40215.916672 secs 2023-09-10T10:29:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40216.268478 secs 2023-09-10T10:29:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40216.268478 secs 2023-09-10T10:29:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40216.268478 secs 2023-09-10T10:29:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:46 smithi170 ceph-mon[102832]: pgmap v21606: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 580 KiB/s wr, 12 op/s 2023-09-10T10:29:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:46 smithi082 ceph-mon[98694]: pgmap v21606: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 580 KiB/s wr, 12 op/s 2023-09-10T10:29:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:46 smithi098 ceph-mon[103124]: pgmap v21606: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 580 KiB/s wr, 12 op/s 2023-09-10T10:29:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:48 smithi170 ceph-mon[102832]: pgmap v21607: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 786 KiB/s wr, 19 op/s 2023-09-10T10:29:48.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:48 smithi082 ceph-mon[98694]: pgmap v21607: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 786 KiB/s wr, 19 op/s 2023-09-10T10:29:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:48 smithi098 ceph-mon[103124]: pgmap v21607: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 786 KiB/s wr, 19 op/s 2023-09-10T10:29:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40220.916757 secs 2023-09-10T10:29:49.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40220.916757 secs 2023-09-10T10:29:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40220.916757 secs 2023-09-10T10:29:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:50 smithi170 ceph-mon[102832]: pgmap v21608: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 578 KiB/s wr, 13 op/s 2023-09-10T10:29:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40221.268542 secs 2023-09-10T10:29:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:29:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:50 smithi082 ceph-mon[98694]: pgmap v21608: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 578 KiB/s wr, 13 op/s 2023-09-10T10:29:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40221.268542 secs 2023-09-10T10:29:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:29:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:50 smithi098 ceph-mon[103124]: pgmap v21608: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 578 KiB/s wr, 13 op/s 2023-09-10T10:29:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40221.268542 secs 2023-09-10T10:29:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:29:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:51.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:29:51.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:29:51.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:29:51.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:51 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:29:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:29:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:29:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:29:51.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:51 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:29:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:29:51.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:29:51.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:29:51.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:51 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:29:52.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:52 smithi170 ceph-mon[102832]: pgmap v21609: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 841 KiB/s wr, 20 op/s 2023-09-10T10:29:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:52 smithi082 ceph-mon[98694]: pgmap v21609: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 841 KiB/s wr, 20 op/s 2023-09-10T10:29:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:52 smithi098 ceph-mon[103124]: pgmap v21609: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 841 KiB/s wr, 20 op/s 2023-09-10T10:29:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:29:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:29:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:29:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:54 smithi170 ceph-mon[102832]: pgmap v21610: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 469 KiB/s wr, 16 op/s 2023-09-10T10:29:54.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40225.916829 secs 2023-09-10T10:29:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:54 smithi082 ceph-mon[98694]: pgmap v21610: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 469 KiB/s wr, 16 op/s 2023-09-10T10:29:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40225.916829 secs 2023-09-10T10:29:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:54 smithi098 ceph-mon[103124]: pgmap v21610: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 469 KiB/s wr, 16 op/s 2023-09-10T10:29:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40225.916829 secs 2023-09-10T10:29:55.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40226.268607 secs 2023-09-10T10:29:55.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40226.268607 secs 2023-09-10T10:29:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40226.268607 secs 2023-09-10T10:29:56.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:56 smithi170 ceph-mon[102832]: pgmap v21611: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 469 KiB/s wr, 14 op/s 2023-09-10T10:29:56.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:56 smithi082 ceph-mon[98694]: pgmap v21611: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 469 KiB/s wr, 14 op/s 2023-09-10T10:29:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:56 smithi098 ceph-mon[103124]: pgmap v21611: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 469 KiB/s wr, 14 op/s 2023-09-10T10:29:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:58 smithi170 ceph-mon[102832]: pgmap v21612: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 781 KiB/s wr, 21 op/s 2023-09-10T10:29:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:29:58.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:58 smithi082 ceph-mon[98694]: pgmap v21612: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 781 KiB/s wr, 21 op/s 2023-09-10T10:29:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:29:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:58 smithi098 ceph-mon[103124]: pgmap v21612: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 781 KiB/s wr, 21 op/s 2023-09-10T10:29:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:29:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:29:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:29:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40230.916915 secs 2023-09-10T10:29:59.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:29:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40230.916915 secs 2023-09-10T10:29:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:29:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40230.916915 secs 2023-09-10T10:30:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:00 smithi170 ceph-mon[102832]: pgmap v21613: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 575 KiB/s wr, 14 op/s 2023-09-10T10:30:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40231.268672 secs 2023-09-10T10:30:00.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 2 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T10:30:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:00 smithi082 ceph-mon[98694]: pgmap v21613: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 575 KiB/s wr, 14 op/s 2023-09-10T10:30:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40231.268672 secs 2023-09-10T10:30:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 2 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T10:30:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:00 smithi098 ceph-mon[103124]: pgmap v21613: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 575 KiB/s wr, 14 op/s 2023-09-10T10:30:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40231.268672 secs 2023-09-10T10:30:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 2 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T10:30:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:01 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:30:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:01 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:01.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:01 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:30:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:01 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:01 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:30:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:01 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:02 smithi170 ceph-mon[102832]: pgmap v21614: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 635 KiB/s wr, 19 op/s 2023-09-10T10:30:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:02 smithi082 ceph-mon[98694]: pgmap v21614: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 635 KiB/s wr, 19 op/s 2023-09-10T10:30:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:02 smithi098 ceph-mon[103124]: pgmap v21614: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 635 KiB/s wr, 19 op/s 2023-09-10T10:30:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:04 smithi170 ceph-mon[102832]: pgmap v21615: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 602 KiB/s wr, 13 op/s 2023-09-10T10:30:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40234.688057 secs 2023-09-10T10:30:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40235.916973 secs 2023-09-10T10:30:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:04 smithi082 ceph-mon[98694]: pgmap v21615: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 602 KiB/s wr, 13 op/s 2023-09-10T10:30:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40234.688057 secs 2023-09-10T10:30:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40235.916973 secs 2023-09-10T10:30:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:04 smithi098 ceph-mon[103124]: pgmap v21615: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 602 KiB/s wr, 13 op/s 2023-09-10T10:30:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40234.688057 secs 2023-09-10T10:30:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40235.916973 secs 2023-09-10T10:30:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40236.268738 secs 2023-09-10T10:30:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:05 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:30:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:30:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40236.268738 secs 2023-09-10T10:30:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:05 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:30:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:30:05.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40236.268738 secs 2023-09-10T10:30:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:05 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:30:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:30:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:06 smithi082 ceph-mon[98694]: pgmap v21616: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 602 KiB/s wr, 13 op/s 2023-09-10T10:30:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:06 smithi098 ceph-mon[103124]: pgmap v21616: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 602 KiB/s wr, 13 op/s 2023-09-10T10:30:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:06 smithi170 ceph-mon[102832]: pgmap v21616: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 602 KiB/s wr, 13 op/s 2023-09-10T10:30:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:08 smithi082 ceph-mon[98694]: pgmap v21617: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 817 KiB/s wr, 34 op/s 2023-09-10T10:30:08.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:08 smithi098 ceph-mon[103124]: pgmap v21617: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 817 KiB/s wr, 34 op/s 2023-09-10T10:30:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:08 smithi170 ceph-mon[102832]: pgmap v21617: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 817 KiB/s wr, 34 op/s 2023-09-10T10:30:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40239.688114 secs 2023-09-10T10:30:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:09 smithi082 ceph-mon[98694]: osdmap e1419: 12 total, 12 up, 12 in 2023-09-10T10:30:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40240.917060 secs 2023-09-10T10:30:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40239.688114 secs 2023-09-10T10:30:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:09 smithi098 ceph-mon[103124]: osdmap e1419: 12 total, 12 up, 12 in 2023-09-10T10:30:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40240.917060 secs 2023-09-10T10:30:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40239.688114 secs 2023-09-10T10:30:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:09 smithi170 ceph-mon[102832]: osdmap e1419: 12 total, 12 up, 12 in 2023-09-10T10:30:09.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40240.917060 secs 2023-09-10T10:30:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:10 smithi082 ceph-mon[98694]: pgmap v21619: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 606 KiB/s wr, 33 op/s 2023-09-10T10:30:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40241.268762 secs 2023-09-10T10:30:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:30:10.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:10 smithi098 ceph-mon[103124]: pgmap v21619: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 606 KiB/s wr, 33 op/s 2023-09-10T10:30:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40241.268762 secs 2023-09-10T10:30:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:30:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:10.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:10 smithi170 ceph-mon[102832]: pgmap v21619: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 606 KiB/s wr, 33 op/s 2023-09-10T10:30:10.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40241.268762 secs 2023-09-10T10:30:10.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:30:10.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:12 smithi082 ceph-mon[98694]: pgmap v21620: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 751 KiB/s wr, 34 op/s 2023-09-10T10:30:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:30:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:12 smithi098 ceph-mon[103124]: pgmap v21620: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 751 KiB/s wr, 34 op/s 2023-09-10T10:30:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:30:12.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:12 smithi170 ceph-mon[102832]: pgmap v21620: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 751 KiB/s wr, 34 op/s 2023-09-10T10:30:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:30:12.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:14 smithi082 ceph-mon[98694]: pgmap v21621: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 475 KiB/s wr, 33 op/s 2023-09-10T10:30:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40244.688181 secs 2023-09-10T10:30:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40245.917120 secs 2023-09-10T10:30:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:14 smithi098 ceph-mon[103124]: pgmap v21621: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 475 KiB/s wr, 33 op/s 2023-09-10T10:30:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40244.688181 secs 2023-09-10T10:30:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40245.917120 secs 2023-09-10T10:30:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:14 smithi170 ceph-mon[102832]: pgmap v21621: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 475 KiB/s wr, 33 op/s 2023-09-10T10:30:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40244.688181 secs 2023-09-10T10:30:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40245.917120 secs 2023-09-10T10:30:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40246.268832 secs 2023-09-10T10:30:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:15 smithi098 ceph-mon[103124]: osdmap e1420: 12 total, 12 up, 12 in 2023-09-10T10:30:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40246.268832 secs 2023-09-10T10:30:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:15 smithi170 ceph-mon[102832]: osdmap e1420: 12 total, 12 up, 12 in 2023-09-10T10:30:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40246.268832 secs 2023-09-10T10:30:15.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:15 smithi082 ceph-mon[98694]: osdmap e1420: 12 total, 12 up, 12 in 2023-09-10T10:30:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:16 smithi098 ceph-mon[103124]: pgmap v21622: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 475 KiB/s wr, 33 op/s 2023-09-10T10:30:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:16 smithi170 ceph-mon[102832]: pgmap v21622: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 475 KiB/s wr, 33 op/s 2023-09-10T10:30:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:16 smithi082 ceph-mon[98694]: pgmap v21622: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 475 KiB/s wr, 33 op/s 2023-09-10T10:30:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:18 smithi098 ceph-mon[103124]: pgmap v21624: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 488 KiB/s wr, 20 op/s 2023-09-10T10:30:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:18 smithi170 ceph-mon[102832]: pgmap v21624: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 488 KiB/s wr, 20 op/s 2023-09-10T10:30:18.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:18 smithi082 ceph-mon[98694]: pgmap v21624: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 488 KiB/s wr, 20 op/s 2023-09-10T10:30:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40249.688249 secs 2023-09-10T10:30:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40250.917181 secs 2023-09-10T10:30:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40249.688249 secs 2023-09-10T10:30:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40250.917181 secs 2023-09-10T10:30:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40249.688249 secs 2023-09-10T10:30:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40250.917181 secs 2023-09-10T10:30:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:20 smithi098 ceph-mon[103124]: pgmap v21625: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 396 KiB/s wr, 16 op/s 2023-09-10T10:30:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40251.268897 secs 2023-09-10T10:30:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:20 smithi170 ceph-mon[102832]: pgmap v21625: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 396 KiB/s wr, 16 op/s 2023-09-10T10:30:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40251.268897 secs 2023-09-10T10:30:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:20 smithi082 ceph-mon[98694]: pgmap v21625: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 396 KiB/s wr, 16 op/s 2023-09-10T10:30:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40251.268897 secs 2023-09-10T10:30:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:21 smithi098 ceph-mon[103124]: pgmap v21626: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 15 op/s 2023-09-10T10:30:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:21 smithi170 ceph-mon[102832]: pgmap v21626: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 15 op/s 2023-09-10T10:30:21.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:21 smithi082 ceph-mon[98694]: pgmap v21626: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 15 op/s 2023-09-10T10:30:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:23 smithi170 ceph-mon[102832]: pgmap v21627: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 14 op/s 2023-09-10T10:30:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:23 smithi082 ceph-mon[98694]: pgmap v21627: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 14 op/s 2023-09-10T10:30:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:23 smithi098 ceph-mon[103124]: pgmap v21627: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 14 op/s 2023-09-10T10:30:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40254.688360 secs 2023-09-10T10:30:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:24 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:30:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:24 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40255.917266 secs 2023-09-10T10:30:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40254.688360 secs 2023-09-10T10:30:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:24 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:30:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:24 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:24.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40255.917266 secs 2023-09-10T10:30:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40254.688360 secs 2023-09-10T10:30:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:24 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:30:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:24 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40255.917266 secs 2023-09-10T10:30:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40256.268962 secs 2023-09-10T10:30:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:25 smithi170 ceph-mon[102832]: pgmap v21628: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 14 op/s 2023-09-10T10:30:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40256.268962 secs 2023-09-10T10:30:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:25 smithi082 ceph-mon[98694]: pgmap v21628: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 14 op/s 2023-09-10T10:30:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40256.268962 secs 2023-09-10T10:30:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:25 smithi098 ceph-mon[103124]: pgmap v21628: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 180 KiB/s wr, 14 op/s 2023-09-10T10:30:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:26 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:30:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:26 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:30:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:26 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:30:26.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:27.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:27 smithi170 ceph-mon[102832]: pgmap v21629: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 317 KiB/s wr, 19 op/s 2023-09-10T10:30:27.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:27 smithi082 ceph-mon[98694]: pgmap v21629: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 317 KiB/s wr, 19 op/s 2023-09-10T10:30:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:27 smithi098 ceph-mon[103124]: pgmap v21629: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 317 KiB/s wr, 19 op/s 2023-09-10T10:30:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40259.688422 secs 2023-09-10T10:30:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40259.688422 secs 2023-09-10T10:30:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40259.688422 secs 2023-09-10T10:30:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:29 smithi170 ceph-mon[102832]: pgmap v21630: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:30:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40260.917343 secs 2023-09-10T10:30:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:29 smithi082 ceph-mon[98694]: pgmap v21630: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:30:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40260.917343 secs 2023-09-10T10:30:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:29 smithi098 ceph-mon[103124]: pgmap v21630: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:30:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40260.917343 secs 2023-09-10T10:30:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40261.269028 secs 2023-09-10T10:30:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40261.269028 secs 2023-09-10T10:30:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40261.269028 secs 2023-09-10T10:30:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:32 smithi170 ceph-mon[102832]: pgmap v21631: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 149 KiB/s wr, 16 op/s 2023-09-10T10:30:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:32 smithi082 ceph-mon[98694]: pgmap v21631: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 149 KiB/s wr, 16 op/s 2023-09-10T10:30:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:32 smithi098 ceph-mon[103124]: pgmap v21631: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 149 KiB/s wr, 16 op/s 2023-09-10T10:30:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:34 smithi170 ceph-mon[102832]: pgmap v21632: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T10:30:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40264.688511 secs 2023-09-10T10:30:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40265.917402 secs 2023-09-10T10:30:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:34 smithi082 ceph-mon[98694]: pgmap v21632: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T10:30:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40264.688511 secs 2023-09-10T10:30:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40265.917402 secs 2023-09-10T10:30:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:34 smithi098 ceph-mon[103124]: pgmap v21632: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T10:30:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40264.688511 secs 2023-09-10T10:30:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40265.917402 secs 2023-09-10T10:30:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40266.269095 secs 2023-09-10T10:30:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40266.269095 secs 2023-09-10T10:30:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40266.269095 secs 2023-09-10T10:30:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:36 smithi170 ceph-mon[102832]: pgmap v21633: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T10:30:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:36 smithi082 ceph-mon[98694]: pgmap v21633: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T10:30:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:36 smithi098 ceph-mon[103124]: pgmap v21633: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 148 KiB/s wr, 11 op/s 2023-09-10T10:30:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:38 smithi170 ceph-mon[102832]: pgmap v21634: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 320 KiB/s wr, 16 op/s 2023-09-10T10:30:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:30:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:38 smithi082 ceph-mon[98694]: pgmap v21634: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 320 KiB/s wr, 16 op/s 2023-09-10T10:30:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:30:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:38 smithi098 ceph-mon[103124]: pgmap v21634: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 320 KiB/s wr, 16 op/s 2023-09-10T10:30:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:30:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40270.917460 secs 2023-09-10T10:30:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40270.917460 secs 2023-09-10T10:30:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40270.917460 secs 2023-09-10T10:30:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:40 smithi170 ceph-mon[102832]: pgmap v21635: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 172 KiB/s wr, 10 op/s 2023-09-10T10:30:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40271.269163 secs 2023-09-10T10:30:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:40 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:30:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:40 smithi082 ceph-mon[98694]: pgmap v21635: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 172 KiB/s wr, 10 op/s 2023-09-10T10:30:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40271.269163 secs 2023-09-10T10:30:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:40 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:30:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:40 smithi098 ceph-mon[103124]: pgmap v21635: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 172 KiB/s wr, 10 op/s 2023-09-10T10:30:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40271.269163 secs 2023-09-10T10:30:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:40 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:30:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:41 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:30:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:41 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:30:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:41 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:41.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:41 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:30:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:41 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:30:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:41 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:41.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:41 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:30:41.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:41 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:30:41.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:41 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:42 smithi170 ceph-mon[102832]: pgmap v21636: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 544 KiB/s wr, 17 op/s 2023-09-10T10:30:42.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:42 smithi082 ceph-mon[98694]: pgmap v21636: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 544 KiB/s wr, 17 op/s 2023-09-10T10:30:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:42 smithi098 ceph-mon[103124]: pgmap v21636: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 544 KiB/s wr, 17 op/s 2023-09-10T10:30:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:44 smithi170 ceph-mon[102832]: pgmap v21637: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 543 KiB/s wr, 12 op/s 2023-09-10T10:30:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40275.917548 secs 2023-09-10T10:30:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:44 smithi082 ceph-mon[98694]: pgmap v21637: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 543 KiB/s wr, 12 op/s 2023-09-10T10:30:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40275.917548 secs 2023-09-10T10:30:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:44 smithi098 ceph-mon[103124]: pgmap v21637: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 543 KiB/s wr, 12 op/s 2023-09-10T10:30:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40275.917548 secs 2023-09-10T10:30:45.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40276.269227 secs 2023-09-10T10:30:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40276.269227 secs 2023-09-10T10:30:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40276.269227 secs 2023-09-10T10:30:46.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:46 smithi170 ceph-mon[102832]: pgmap v21638: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 544 KiB/s wr, 12 op/s 2023-09-10T10:30:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:46 smithi082 ceph-mon[98694]: pgmap v21638: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 544 KiB/s wr, 12 op/s 2023-09-10T10:30:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:46 smithi098 ceph-mon[103124]: pgmap v21638: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 544 KiB/s wr, 12 op/s 2023-09-10T10:30:48.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:48 smithi170 ceph-mon[102832]: pgmap v21639: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 789 KiB/s wr, 19 op/s 2023-09-10T10:30:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:48 smithi082 ceph-mon[98694]: pgmap v21639: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 789 KiB/s wr, 19 op/s 2023-09-10T10:30:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:48 smithi098 ceph-mon[103124]: pgmap v21639: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 789 KiB/s wr, 19 op/s 2023-09-10T10:30:49.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40280.917603 secs 2023-09-10T10:30:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40280.917603 secs 2023-09-10T10:30:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40280.917603 secs 2023-09-10T10:30:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:50 smithi170 ceph-mon[102832]: pgmap v21640: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 617 KiB/s wr, 13 op/s 2023-09-10T10:30:50.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40281.269293 secs 2023-09-10T10:30:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:30:50.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:50 smithi082 ceph-mon[98694]: pgmap v21640: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 617 KiB/s wr, 13 op/s 2023-09-10T10:30:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40281.269293 secs 2023-09-10T10:30:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:30:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:50.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:50 smithi098 ceph-mon[103124]: pgmap v21640: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 617 KiB/s wr, 13 op/s 2023-09-10T10:30:50.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40281.269293 secs 2023-09-10T10:30:50.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:30:50.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:52.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:52 smithi170 ceph-mon[102832]: pgmap v21641: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 880 KiB/s wr, 20 op/s 2023-09-10T10:30:52.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:30:52.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:30:52.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:52 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:30:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:52 smithi082 ceph-mon[98694]: pgmap v21641: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 880 KiB/s wr, 20 op/s 2023-09-10T10:30:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:30:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:30:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:52 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:30:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:52 smithi098 ceph-mon[103124]: pgmap v21641: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 880 KiB/s wr, 20 op/s 2023-09-10T10:30:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:30:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:30:52.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:52 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:30:53.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:30:53.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:53.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:30:53.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:30:53.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:30:53.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:30:53.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:53.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:30:53.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:30:53.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:30:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:30:53.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:53.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:30:53.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:30:53.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:30:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:54 smithi098 ceph-mon[103124]: pgmap v21642: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 508 KiB/s wr, 14 op/s 2023-09-10T10:30:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40285.917663 secs 2023-09-10T10:30:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:54 smithi170 ceph-mon[102832]: pgmap v21642: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 508 KiB/s wr, 14 op/s 2023-09-10T10:30:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40285.917663 secs 2023-09-10T10:30:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:54 smithi082 ceph-mon[98694]: pgmap v21642: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 508 KiB/s wr, 14 op/s 2023-09-10T10:30:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40285.917663 secs 2023-09-10T10:30:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40286.269360 secs 2023-09-10T10:30:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:55 smithi098 ceph-mon[103124]: pgmap v21643: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 508 KiB/s wr, 14 op/s 2023-09-10T10:30:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40286.269360 secs 2023-09-10T10:30:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:55 smithi170 ceph-mon[102832]: pgmap v21643: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 508 KiB/s wr, 14 op/s 2023-09-10T10:30:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40286.269360 secs 2023-09-10T10:30:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:55 smithi082 ceph-mon[98694]: pgmap v21643: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 508 KiB/s wr, 14 op/s 2023-09-10T10:30:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:30:57.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:57.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:30:57.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:30:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:58 smithi170 ceph-mon[102832]: pgmap v21644: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T10:30:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:30:58.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:58.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:58 smithi082 ceph-mon[98694]: pgmap v21644: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T10:30:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:30:58.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:58 smithi098 ceph-mon[103124]: pgmap v21644: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T10:30:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:30:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:30:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40289.688835 secs 2023-09-10T10:30:59.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:30:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40290.917721 secs 2023-09-10T10:30:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40289.688835 secs 2023-09-10T10:30:59.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:30:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40290.917721 secs 2023-09-10T10:30:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40289.688835 secs 2023-09-10T10:30:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:30:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40290.917721 secs 2023-09-10T10:31:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:00 smithi170 ceph-mon[102832]: pgmap v21645: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 784 KiB/s wr, 14 op/s 2023-09-10T10:31:00.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40291.269427 secs 2023-09-10T10:31:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:00 smithi082 ceph-mon[98694]: pgmap v21645: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 784 KiB/s wr, 14 op/s 2023-09-10T10:31:00.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40291.269427 secs 2023-09-10T10:31:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:00 smithi098 ceph-mon[103124]: pgmap v21645: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 784 KiB/s wr, 14 op/s 2023-09-10T10:31:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40291.269427 secs 2023-09-10T10:31:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:01 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:31:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:01 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:31:01.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:01 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:31:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:01 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:31:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:01 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:31:01.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:01 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:31:01.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:01 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:31:01.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:01 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:31:01.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:01 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:31:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:02 smithi170 ceph-mon[102832]: pgmap v21646: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 984 KiB/s wr, 32 op/s 2023-09-10T10:31:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:02 smithi082 ceph-mon[98694]: pgmap v21646: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 984 KiB/s wr, 32 op/s 2023-09-10T10:31:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:02 smithi098 ceph-mon[103124]: pgmap v21646: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 984 KiB/s wr, 32 op/s 2023-09-10T10:31:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:04 smithi170 ceph-mon[102832]: pgmap v21647: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 721 KiB/s wr, 25 op/s 2023-09-10T10:31:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40294.688905 secs 2023-09-10T10:31:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40295.917805 secs 2023-09-10T10:31:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:04 smithi082 ceph-mon[98694]: pgmap v21647: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 721 KiB/s wr, 25 op/s 2023-09-10T10:31:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40294.688905 secs 2023-09-10T10:31:04.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40295.917805 secs 2023-09-10T10:31:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:04 smithi098 ceph-mon[103124]: pgmap v21647: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 721 KiB/s wr, 25 op/s 2023-09-10T10:31:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40294.688905 secs 2023-09-10T10:31:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40295.917805 secs 2023-09-10T10:31:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40296.269492 secs 2023-09-10T10:31:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40296.269492 secs 2023-09-10T10:31:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40296.269492 secs 2023-09-10T10:31:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:06 smithi170 ceph-mon[102832]: pgmap v21648: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 721 KiB/s wr, 25 op/s 2023-09-10T10:31:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:06 smithi082 ceph-mon[98694]: pgmap v21648: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 721 KiB/s wr, 25 op/s 2023-09-10T10:31:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:06 smithi098 ceph-mon[103124]: pgmap v21648: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 721 KiB/s wr, 25 op/s 2023-09-10T10:31:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:08 smithi170 ceph-mon[102832]: pgmap v21649: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 862 KiB/s wr, 35 op/s 2023-09-10T10:31:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:08 smithi082 ceph-mon[98694]: pgmap v21649: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 862 KiB/s wr, 35 op/s 2023-09-10T10:31:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:08 smithi098 ceph-mon[103124]: pgmap v21649: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 862 KiB/s wr, 35 op/s 2023-09-10T10:31:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40299.688964 secs 2023-09-10T10:31:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:09 smithi170 ceph-mon[102832]: osdmap e1421: 12 total, 12 up, 12 in 2023-09-10T10:31:09.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40300.917865 secs 2023-09-10T10:31:09.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40299.688964 secs 2023-09-10T10:31:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:09 smithi082 ceph-mon[98694]: osdmap e1421: 12 total, 12 up, 12 in 2023-09-10T10:31:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40300.917865 secs 2023-09-10T10:31:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40299.688964 secs 2023-09-10T10:31:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:09 smithi098 ceph-mon[103124]: osdmap e1421: 12 total, 12 up, 12 in 2023-09-10T10:31:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40300.917865 secs 2023-09-10T10:31:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:10 smithi170 ceph-mon[102832]: pgmap v21651: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 410 KiB/s wr, 33 op/s 2023-09-10T10:31:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40301.269554 secs 2023-09-10T10:31:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:10 smithi082 ceph-mon[98694]: pgmap v21651: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 410 KiB/s wr, 33 op/s 2023-09-10T10:31:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40301.269554 secs 2023-09-10T10:31:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:10 smithi098 ceph-mon[103124]: pgmap v21651: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 410 KiB/s wr, 33 op/s 2023-09-10T10:31:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40301.269554 secs 2023-09-10T10:31:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:12 smithi170 ceph-mon[102832]: pgmap v21652: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 170 KiB/s wr, 17 op/s 2023-09-10T10:31:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:12 smithi082 ceph-mon[98694]: pgmap v21652: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 170 KiB/s wr, 17 op/s 2023-09-10T10:31:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:12 smithi098 ceph-mon[103124]: pgmap v21652: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 170 KiB/s wr, 17 op/s 2023-09-10T10:31:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:14 smithi170 ceph-mon[102832]: pgmap v21653: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 170 KiB/s wr, 17 op/s 2023-09-10T10:31:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40304.689038 secs 2023-09-10T10:31:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40305.917923 secs 2023-09-10T10:31:14.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:14 smithi082 ceph-mon[98694]: pgmap v21653: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 170 KiB/s wr, 17 op/s 2023-09-10T10:31:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40304.689038 secs 2023-09-10T10:31:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40305.917923 secs 2023-09-10T10:31:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:14 smithi098 ceph-mon[103124]: pgmap v21653: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 170 KiB/s wr, 17 op/s 2023-09-10T10:31:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40304.689038 secs 2023-09-10T10:31:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40305.917923 secs 2023-09-10T10:31:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40306.269618 secs 2023-09-10T10:31:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:15 smithi170 ceph-mon[102832]: osdmap e1422: 12 total, 12 up, 12 in 2023-09-10T10:31:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40306.269618 secs 2023-09-10T10:31:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:15 smithi082 ceph-mon[98694]: osdmap e1422: 12 total, 12 up, 12 in 2023-09-10T10:31:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40306.269618 secs 2023-09-10T10:31:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:15 smithi098 ceph-mon[103124]: osdmap e1422: 12 total, 12 up, 12 in 2023-09-10T10:31:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:16 smithi170 ceph-mon[102832]: pgmap v21654: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 170 KiB/s wr, 17 op/s 2023-09-10T10:31:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:16 smithi082 ceph-mon[98694]: pgmap v21654: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 170 KiB/s wr, 17 op/s 2023-09-10T10:31:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:16 smithi098 ceph-mon[103124]: pgmap v21654: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 170 KiB/s wr, 17 op/s 2023-09-10T10:31:18.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:18 smithi170 ceph-mon[102832]: pgmap v21656: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 248 B/s wr, 13 op/s 2023-09-10T10:31:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:18 smithi082 ceph-mon[98694]: pgmap v21656: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 248 B/s wr, 13 op/s 2023-09-10T10:31:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:18 smithi098 ceph-mon[103124]: pgmap v21656: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 248 B/s wr, 13 op/s 2023-09-10T10:31:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40309.689110 secs 2023-09-10T10:31:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40310.917985 secs 2023-09-10T10:31:19.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40309.689110 secs 2023-09-10T10:31:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40310.917985 secs 2023-09-10T10:31:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40309.689110 secs 2023-09-10T10:31:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40310.917985 secs 2023-09-10T10:31:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:20 smithi170 ceph-mon[102832]: pgmap v21657: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T10:31:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40311.269683 secs 2023-09-10T10:31:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:20 smithi082 ceph-mon[98694]: pgmap v21657: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T10:31:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40311.269683 secs 2023-09-10T10:31:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:20 smithi098 ceph-mon[103124]: pgmap v21657: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-09-10T10:31:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40311.269683 secs 2023-09-10T10:31:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:22 smithi170 ceph-mon[102832]: pgmap v21658: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T10:31:22.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:22 smithi082 ceph-mon[98694]: pgmap v21658: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T10:31:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:22 smithi098 ceph-mon[103124]: pgmap v21658: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T10:31:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:24 smithi170 ceph-mon[102832]: pgmap v21659: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T10:31:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40314.689186 secs 2023-09-10T10:31:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40315.918069 secs 2023-09-10T10:31:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:24 smithi082 ceph-mon[98694]: pgmap v21659: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T10:31:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40314.689186 secs 2023-09-10T10:31:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40315.918069 secs 2023-09-10T10:31:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:24 smithi098 ceph-mon[103124]: pgmap v21659: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T10:31:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40314.689186 secs 2023-09-10T10:31:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40315.918069 secs 2023-09-10T10:31:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40316.269747 secs 2023-09-10T10:31:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40316.269747 secs 2023-09-10T10:31:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40316.269747 secs 2023-09-10T10:31:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:26 smithi170 ceph-mon[102832]: pgmap v21660: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T10:31:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:26 smithi082 ceph-mon[98694]: pgmap v21660: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T10:31:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:26 smithi098 ceph-mon[103124]: pgmap v21660: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-09-10T10:31:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:28 smithi170 ceph-mon[102832]: pgmap v21661: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 181 B/s wr, 14 op/s 2023-09-10T10:31:28.469 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:28 smithi082 ceph-mon[98694]: pgmap v21661: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 181 B/s wr, 14 op/s 2023-09-10T10:31:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:28 smithi098 ceph-mon[103124]: pgmap v21661: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 181 B/s wr, 14 op/s 2023-09-10T10:31:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40319.689243 secs 2023-09-10T10:31:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40320.918169 secs 2023-09-10T10:31:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40319.689243 secs 2023-09-10T10:31:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40320.918169 secs 2023-09-10T10:31:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40319.689243 secs 2023-09-10T10:31:29.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40320.918169 secs 2023-09-10T10:31:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:30 smithi170 ceph-mon[102832]: pgmap v21662: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-10T10:31:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40321.269812 secs 2023-09-10T10:31:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:30 smithi082 ceph-mon[98694]: pgmap v21662: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-10T10:31:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40321.269812 secs 2023-09-10T10:31:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:30 smithi098 ceph-mon[103124]: pgmap v21662: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-10T10:31:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40321.269812 secs 2023-09-10T10:31:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:32 smithi170 ceph-mon[102832]: pgmap v21663: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 144 KiB/s wr, 14 op/s 2023-09-10T10:31:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:32 smithi082 ceph-mon[98694]: pgmap v21663: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 144 KiB/s wr, 14 op/s 2023-09-10T10:31:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:32 smithi098 ceph-mon[103124]: pgmap v21663: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 144 KiB/s wr, 14 op/s 2023-09-10T10:31:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:34 smithi170 ceph-mon[102832]: pgmap v21664: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:31:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40324.689306 secs 2023-09-10T10:31:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40325.918225 secs 2023-09-10T10:31:34.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:34 smithi082 ceph-mon[98694]: pgmap v21664: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:31:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40324.689306 secs 2023-09-10T10:31:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40325.918225 secs 2023-09-10T10:31:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:34 smithi098 ceph-mon[103124]: pgmap v21664: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:31:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40324.689306 secs 2023-09-10T10:31:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40325.918225 secs 2023-09-10T10:31:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40326.269877 secs 2023-09-10T10:31:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40326.269877 secs 2023-09-10T10:31:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40326.269877 secs 2023-09-10T10:31:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:36 smithi170 ceph-mon[102832]: pgmap v21665: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:31:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:36 smithi082 ceph-mon[98694]: pgmap v21665: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:31:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:36 smithi098 ceph-mon[103124]: pgmap v21665: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 144 KiB/s wr, 10 op/s 2023-09-10T10:31:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:38 smithi170 ceph-mon[102832]: pgmap v21666: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 292 KiB/s wr, 16 op/s 2023-09-10T10:31:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:31:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:31:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:38 smithi082 ceph-mon[98694]: pgmap v21666: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 292 KiB/s wr, 16 op/s 2023-09-10T10:31:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:31:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:31:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:38 smithi098 ceph-mon[103124]: pgmap v21666: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 292 KiB/s wr, 16 op/s 2023-09-10T10:31:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:31:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:31:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40330.918299 secs 2023-09-10T10:31:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40330.918299 secs 2023-09-10T10:31:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40330.918299 secs 2023-09-10T10:31:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:40 smithi082 ceph-mon[98694]: pgmap v21667: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 292 KiB/s wr, 11 op/s 2023-09-10T10:31:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40331.269942 secs 2023-09-10T10:31:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:40 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:31:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:40 smithi098 ceph-mon[103124]: pgmap v21667: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 292 KiB/s wr, 11 op/s 2023-09-10T10:31:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40331.269942 secs 2023-09-10T10:31:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:40 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:31:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:40 smithi170 ceph-mon[102832]: pgmap v21667: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 292 KiB/s wr, 11 op/s 2023-09-10T10:31:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40331.269942 secs 2023-09-10T10:31:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:40 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:31:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:41 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:31:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:41 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:31:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:41 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:31:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:41 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:31:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:41 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:31:41.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:41 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:31:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:41 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:31:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:41 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:31:41.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:41 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:31:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:42 smithi082 ceph-mon[98694]: pgmap v21668: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 614 KiB/s wr, 18 op/s 2023-09-10T10:31:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:42 smithi098 ceph-mon[103124]: pgmap v21668: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 614 KiB/s wr, 18 op/s 2023-09-10T10:31:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:42 smithi170 ceph-mon[102832]: pgmap v21668: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 614 KiB/s wr, 18 op/s 2023-09-10T10:31:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:44 smithi082 ceph-mon[98694]: pgmap v21669: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 471 KiB/s wr, 13 op/s 2023-09-10T10:31:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40335.918357 secs 2023-09-10T10:31:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:44 smithi098 ceph-mon[103124]: pgmap v21669: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 471 KiB/s wr, 13 op/s 2023-09-10T10:31:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40335.918357 secs 2023-09-10T10:31:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:44 smithi170 ceph-mon[102832]: pgmap v21669: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 471 KiB/s wr, 13 op/s 2023-09-10T10:31:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40335.918357 secs 2023-09-10T10:31:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40336.270009 secs 2023-09-10T10:31:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40336.270009 secs 2023-09-10T10:31:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40336.270009 secs 2023-09-10T10:31:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:46 smithi082 ceph-mon[98694]: pgmap v21670: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 471 KiB/s wr, 13 op/s 2023-09-10T10:31:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:31:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:31:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:46 smithi098 ceph-mon[103124]: pgmap v21670: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 471 KiB/s wr, 13 op/s 2023-09-10T10:31:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:31:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:31:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:46 smithi170 ceph-mon[102832]: pgmap v21670: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 471 KiB/s wr, 13 op/s 2023-09-10T10:31:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:31:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:31:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:48 smithi082 ceph-mon[98694]: pgmap v21671: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 815 KiB/s wr, 20 op/s 2023-09-10T10:31:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:48 smithi098 ceph-mon[103124]: pgmap v21671: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 815 KiB/s wr, 20 op/s 2023-09-10T10:31:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:48 smithi170 ceph-mon[102832]: pgmap v21671: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 815 KiB/s wr, 20 op/s 2023-09-10T10:31:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40340.918416 secs 2023-09-10T10:31:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40340.918416 secs 2023-09-10T10:31:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40340.918416 secs 2023-09-10T10:31:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:50 smithi098 ceph-mon[103124]: pgmap v21672: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 667 KiB/s wr, 14 op/s 2023-09-10T10:31:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40341.270076 secs 2023-09-10T10:31:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:31:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:31:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:50 smithi170 ceph-mon[102832]: pgmap v21672: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 667 KiB/s wr, 14 op/s 2023-09-10T10:31:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40341.270076 secs 2023-09-10T10:31:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:31:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:31:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:50 smithi082 ceph-mon[98694]: pgmap v21672: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 667 KiB/s wr, 14 op/s 2023-09-10T10:31:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40341.270076 secs 2023-09-10T10:31:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:31:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:31:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:52 smithi098 ceph-mon[103124]: pgmap v21673: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 988 KiB/s wr, 21 op/s 2023-09-10T10:31:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:31:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:31:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:52 smithi170 ceph-mon[102832]: pgmap v21673: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 988 KiB/s wr, 21 op/s 2023-09-10T10:31:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:31:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:31:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:52 smithi082 ceph-mon[98694]: pgmap v21673: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 988 KiB/s wr, 21 op/s 2023-09-10T10:31:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:31:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:31:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:31:53.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:31:53.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:31:53.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:31:53.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:53 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:31:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:31:53.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:31:53.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:31:53.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:31:53.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:53 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:31:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:31:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:31:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:31:53.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:31:53.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:53 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:31:54.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:54 smithi098 ceph-mon[103124]: pgmap v21674: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 665 KiB/s wr, 14 op/s 2023-09-10T10:31:54.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:31:54.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:31:54.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:31:54.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40345.918478 secs 2023-09-10T10:31:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:54 smithi170 ceph-mon[102832]: pgmap v21674: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 665 KiB/s wr, 14 op/s 2023-09-10T10:31:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:31:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:31:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:31:54.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40345.918478 secs 2023-09-10T10:31:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:54 smithi082 ceph-mon[98694]: pgmap v21674: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 665 KiB/s wr, 14 op/s 2023-09-10T10:31:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:31:54.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:31:54.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:31:54.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40345.918478 secs 2023-09-10T10:31:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40346.270140 secs 2023-09-10T10:31:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40346.270140 secs 2023-09-10T10:31:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40346.270140 secs 2023-09-10T10:31:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:56 smithi098 ceph-mon[103124]: pgmap v21675: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 665 KiB/s wr, 14 op/s 2023-09-10T10:31:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:56 smithi170 ceph-mon[102832]: pgmap v21675: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 665 KiB/s wr, 14 op/s 2023-09-10T10:31:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:56 smithi082 ceph-mon[98694]: pgmap v21675: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 665 KiB/s wr, 14 op/s 2023-09-10T10:31:57.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:57 smithi170 ceph-mon[102832]: pgmap v21676: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 869 KiB/s wr, 21 op/s 2023-09-10T10:31:57.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:57 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:31:57.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:57 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:31:57.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:57 smithi082 ceph-mon[98694]: pgmap v21676: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 869 KiB/s wr, 21 op/s 2023-09-10T10:31:57.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:57 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:31:57.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:57 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:31:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:57 smithi098 ceph-mon[103124]: pgmap v21676: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 869 KiB/s wr, 21 op/s 2023-09-10T10:31:57.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:57 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:31:57.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:57 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:31:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:59 smithi170 ceph-mon[102832]: pgmap v21677: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 525 KiB/s wr, 13 op/s 2023-09-10T10:31:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:31:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40350.918492 secs 2023-09-10T10:31:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:59 smithi082 ceph-mon[98694]: pgmap v21677: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 525 KiB/s wr, 13 op/s 2023-09-10T10:31:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:31:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40350.918492 secs 2023-09-10T10:31:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:59 smithi098 ceph-mon[103124]: pgmap v21677: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 525 KiB/s wr, 13 op/s 2023-09-10T10:31:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:31:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40350.918492 secs 2023-09-10T10:32:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40351.270202 secs 2023-09-10T10:32:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:32:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:00.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40351.270202 secs 2023-09-10T10:32:00.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:32:00.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40351.270202 secs 2023-09-10T10:32:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:32:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:02 smithi170 ceph-mon[102832]: pgmap v21678: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 792 KiB/s wr, 20 op/s 2023-09-10T10:32:02.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:02 smithi082 ceph-mon[98694]: pgmap v21678: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 792 KiB/s wr, 20 op/s 2023-09-10T10:32:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:02 smithi098 ceph-mon[103124]: pgmap v21678: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 792 KiB/s wr, 20 op/s 2023-09-10T10:32:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:04 smithi170 ceph-mon[102832]: pgmap v21679: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 471 KiB/s wr, 13 op/s 2023-09-10T10:32:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40354.689690 secs 2023-09-10T10:32:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:04 smithi170 ceph-mon[102832]: osdmap e1423: 12 total, 12 up, 12 in 2023-09-10T10:32:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40355.918549 secs 2023-09-10T10:32:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:04 smithi082 ceph-mon[98694]: pgmap v21679: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 471 KiB/s wr, 13 op/s 2023-09-10T10:32:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40354.689690 secs 2023-09-10T10:32:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:04 smithi082 ceph-mon[98694]: osdmap e1423: 12 total, 12 up, 12 in 2023-09-10T10:32:04.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40355.918549 secs 2023-09-10T10:32:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:04 smithi098 ceph-mon[103124]: pgmap v21679: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 471 KiB/s wr, 13 op/s 2023-09-10T10:32:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40354.689690 secs 2023-09-10T10:32:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:04 smithi098 ceph-mon[103124]: osdmap e1423: 12 total, 12 up, 12 in 2023-09-10T10:32:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40355.918549 secs 2023-09-10T10:32:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40356.270289 secs 2023-09-10T10:32:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:05 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:32:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:32:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40356.270289 secs 2023-09-10T10:32:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:05 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:32:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:32:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40356.270289 secs 2023-09-10T10:32:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:05 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:32:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:32:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:06.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:06 smithi170 ceph-mon[102832]: pgmap v21681: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 565 KiB/s wr, 16 op/s 2023-09-10T10:32:06.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:06 smithi082 ceph-mon[98694]: pgmap v21681: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 565 KiB/s wr, 16 op/s 2023-09-10T10:32:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:06 smithi098 ceph-mon[103124]: pgmap v21681: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 565 KiB/s wr, 16 op/s 2023-09-10T10:32:08.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:08 smithi170 ceph-mon[102832]: pgmap v21682: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 729 KiB/s wr, 33 op/s 2023-09-10T10:32:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:08 smithi082 ceph-mon[98694]: pgmap v21682: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 729 KiB/s wr, 33 op/s 2023-09-10T10:32:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:08 smithi098 ceph-mon[103124]: pgmap v21682: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 729 KiB/s wr, 33 op/s 2023-09-10T10:32:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40359.689769 secs 2023-09-10T10:32:09.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40360.918631 secs 2023-09-10T10:32:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40359.689769 secs 2023-09-10T10:32:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40360.918631 secs 2023-09-10T10:32:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40359.689769 secs 2023-09-10T10:32:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40360.918631 secs 2023-09-10T10:32:10.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:10 smithi170 ceph-mon[102832]: pgmap v21683: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 729 KiB/s wr, 33 op/s 2023-09-10T10:32:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40361.270352 secs 2023-09-10T10:32:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:32:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:10.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:10 smithi170 ceph-mon[102832]: osdmap e1424: 12 total, 12 up, 12 in 2023-09-10T10:32:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:10 smithi082 ceph-mon[98694]: pgmap v21683: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 729 KiB/s wr, 33 op/s 2023-09-10T10:32:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40361.270352 secs 2023-09-10T10:32:10.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:32:10.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:10.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:10 smithi082 ceph-mon[98694]: osdmap e1424: 12 total, 12 up, 12 in 2023-09-10T10:32:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:10 smithi098 ceph-mon[103124]: pgmap v21683: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 729 KiB/s wr, 33 op/s 2023-09-10T10:32:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40361.270352 secs 2023-09-10T10:32:10.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:32:10.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:10.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:10 smithi098 ceph-mon[103124]: osdmap e1424: 12 total, 12 up, 12 in 2023-09-10T10:32:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:12 smithi170 ceph-mon[102832]: pgmap v21685: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 813 KiB/s wr, 41 op/s 2023-09-10T10:32:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:12 smithi082 ceph-mon[98694]: pgmap v21685: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 813 KiB/s wr, 41 op/s 2023-09-10T10:32:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:12 smithi098 ceph-mon[103124]: pgmap v21685: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 813 KiB/s wr, 41 op/s 2023-09-10T10:32:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:32:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:32:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:32:13.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:14 smithi170 ceph-mon[102832]: pgmap v21686: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 701 KiB/s wr, 35 op/s 2023-09-10T10:32:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40364.689794 secs 2023-09-10T10:32:14.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40365.918688 secs 2023-09-10T10:32:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:14 smithi082 ceph-mon[98694]: pgmap v21686: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 701 KiB/s wr, 35 op/s 2023-09-10T10:32:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40364.689794 secs 2023-09-10T10:32:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40365.918688 secs 2023-09-10T10:32:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:14 smithi098 ceph-mon[103124]: pgmap v21686: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 701 KiB/s wr, 35 op/s 2023-09-10T10:32:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40364.689794 secs 2023-09-10T10:32:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40365.918688 secs 2023-09-10T10:32:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40366.270416 secs 2023-09-10T10:32:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40366.270416 secs 2023-09-10T10:32:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40366.270416 secs 2023-09-10T10:32:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:16 smithi170 ceph-mon[102832]: pgmap v21687: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 650 KiB/s wr, 33 op/s 2023-09-10T10:32:16.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:16 smithi082 ceph-mon[98694]: pgmap v21687: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 650 KiB/s wr, 33 op/s 2023-09-10T10:32:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:16 smithi098 ceph-mon[103124]: pgmap v21687: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 650 KiB/s wr, 33 op/s 2023-09-10T10:32:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:18 smithi170 ceph-mon[102832]: pgmap v21688: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 386 KiB/s wr, 14 op/s 2023-09-10T10:32:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:18 smithi082 ceph-mon[98694]: pgmap v21688: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 386 KiB/s wr, 14 op/s 2023-09-10T10:32:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:18 smithi098 ceph-mon[103124]: pgmap v21688: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 386 KiB/s wr, 14 op/s 2023-09-10T10:32:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40369.689855 secs 2023-09-10T10:32:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40370.918761 secs 2023-09-10T10:32:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40369.689855 secs 2023-09-10T10:32:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40370.918761 secs 2023-09-10T10:32:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40369.689855 secs 2023-09-10T10:32:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40370.918761 secs 2023-09-10T10:32:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:20 smithi170 ceph-mon[102832]: pgmap v21689: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 386 KiB/s wr, 14 op/s 2023-09-10T10:32:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40371.270484 secs 2023-09-10T10:32:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:20 smithi082 ceph-mon[98694]: pgmap v21689: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 386 KiB/s wr, 14 op/s 2023-09-10T10:32:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40371.270484 secs 2023-09-10T10:32:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:20 smithi098 ceph-mon[103124]: pgmap v21689: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 386 KiB/s wr, 14 op/s 2023-09-10T10:32:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40371.270484 secs 2023-09-10T10:32:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:22 smithi170 ceph-mon[102832]: pgmap v21690: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 376 KiB/s wr, 20 op/s 2023-09-10T10:32:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:22 smithi082 ceph-mon[98694]: pgmap v21690: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 376 KiB/s wr, 20 op/s 2023-09-10T10:32:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:22 smithi098 ceph-mon[103124]: pgmap v21690: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 376 KiB/s wr, 20 op/s 2023-09-10T10:32:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:24 smithi170 ceph-mon[102832]: pgmap v21691: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T10:32:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40374.689913 secs 2023-09-10T10:32:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40375.918819 secs 2023-09-10T10:32:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:24 smithi082 ceph-mon[98694]: pgmap v21691: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T10:32:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40374.689913 secs 2023-09-10T10:32:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40375.918819 secs 2023-09-10T10:32:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:24 smithi098 ceph-mon[103124]: pgmap v21691: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T10:32:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40374.689913 secs 2023-09-10T10:32:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40375.918819 secs 2023-09-10T10:32:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40376.270550 secs 2023-09-10T10:32:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40376.270550 secs 2023-09-10T10:32:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40376.270550 secs 2023-09-10T10:32:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:26 smithi170 ceph-mon[102832]: pgmap v21692: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T10:32:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:26 smithi082 ceph-mon[98694]: pgmap v21692: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T10:32:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:26 smithi098 ceph-mon[103124]: pgmap v21692: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 11 op/s 2023-09-10T10:32:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:28 smithi170 ceph-mon[102832]: pgmap v21693: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 268 KiB/s wr, 16 op/s 2023-09-10T10:32:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:28 smithi082 ceph-mon[98694]: pgmap v21693: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 268 KiB/s wr, 16 op/s 2023-09-10T10:32:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:28 smithi098 ceph-mon[103124]: pgmap v21693: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 268 KiB/s wr, 16 op/s 2023-09-10T10:32:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40379.689974 secs 2023-09-10T10:32:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40380.918901 secs 2023-09-10T10:32:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40379.689974 secs 2023-09-10T10:32:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40380.918901 secs 2023-09-10T10:32:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40379.689974 secs 2023-09-10T10:32:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40380.918901 secs 2023-09-10T10:32:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:30 smithi170 ceph-mon[102832]: pgmap v21694: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 10 op/s 2023-09-10T10:32:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40381.270616 secs 2023-09-10T10:32:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:30 smithi082 ceph-mon[98694]: pgmap v21694: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 10 op/s 2023-09-10T10:32:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40381.270616 secs 2023-09-10T10:32:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:30 smithi098 ceph-mon[103124]: pgmap v21694: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 10 op/s 2023-09-10T10:32:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40381.270616 secs 2023-09-10T10:32:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:32 smithi170 ceph-mon[102832]: pgmap v21695: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 149 KiB/s wr, 16 op/s 2023-09-10T10:32:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:32 smithi082 ceph-mon[98694]: pgmap v21695: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 149 KiB/s wr, 16 op/s 2023-09-10T10:32:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:32 smithi098 ceph-mon[103124]: pgmap v21695: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 149 KiB/s wr, 16 op/s 2023-09-10T10:32:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:34 smithi170 ceph-mon[102832]: pgmap v21696: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:32:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40384.690029 secs 2023-09-10T10:32:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:32:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40385.918961 secs 2023-09-10T10:32:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:34 smithi082 ceph-mon[98694]: pgmap v21696: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:32:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40384.690029 secs 2023-09-10T10:32:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:32:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:34.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40385.918961 secs 2023-09-10T10:32:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:34 smithi098 ceph-mon[103124]: pgmap v21696: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:32:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40384.690029 secs 2023-09-10T10:32:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:32:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40385.918961 secs 2023-09-10T10:32:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40386.270683 secs 2023-09-10T10:32:35.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40386.270683 secs 2023-09-10T10:32:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40386.270683 secs 2023-09-10T10:32:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:36 smithi170 ceph-mon[102832]: pgmap v21697: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:32:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:36 smithi082 ceph-mon[98694]: pgmap v21697: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:32:36.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:36 smithi098 ceph-mon[103124]: pgmap v21697: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:32:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:32:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:32:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:37.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:32:37.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:38 smithi082 ceph-mon[98694]: pgmap v21698: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 371 KiB/s wr, 17 op/s 2023-09-10T10:32:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:32:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:38 smithi098 ceph-mon[103124]: pgmap v21698: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 371 KiB/s wr, 17 op/s 2023-09-10T10:32:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:32:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:38 smithi170 ceph-mon[102832]: pgmap v21698: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 371 KiB/s wr, 17 op/s 2023-09-10T10:32:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:32:38.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40390.919037 secs 2023-09-10T10:32:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40390.919037 secs 2023-09-10T10:32:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40390.919037 secs 2023-09-10T10:32:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:40 smithi082 ceph-mon[98694]: pgmap v21699: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T10:32:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40391.270748 secs 2023-09-10T10:32:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:40 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:32:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:32:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:40 smithi098 ceph-mon[103124]: pgmap v21699: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T10:32:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40391.270748 secs 2023-09-10T10:32:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:40 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:32:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:32:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:40 smithi170 ceph-mon[102832]: pgmap v21699: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 223 KiB/s wr, 11 op/s 2023-09-10T10:32:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40391.270748 secs 2023-09-10T10:32:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:40 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:32:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:32:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:41 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:32:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:41 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:41 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:32:41.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:41 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:41 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:32:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:41 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:42 smithi082 ceph-mon[98694]: pgmap v21700: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 594 KiB/s wr, 17 op/s 2023-09-10T10:32:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:42 smithi098 ceph-mon[103124]: pgmap v21700: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 594 KiB/s wr, 17 op/s 2023-09-10T10:32:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:42 smithi170 ceph-mon[102832]: pgmap v21700: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 594 KiB/s wr, 17 op/s 2023-09-10T10:32:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:44 smithi082 ceph-mon[98694]: pgmap v21701: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 594 KiB/s wr, 12 op/s 2023-09-10T10:32:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40395.919094 secs 2023-09-10T10:32:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:44 smithi098 ceph-mon[103124]: pgmap v21701: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 594 KiB/s wr, 12 op/s 2023-09-10T10:32:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40395.919094 secs 2023-09-10T10:32:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:44 smithi170 ceph-mon[102832]: pgmap v21701: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 594 KiB/s wr, 12 op/s 2023-09-10T10:32:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40395.919094 secs 2023-09-10T10:32:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40396.270812 secs 2023-09-10T10:32:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40396.270812 secs 2023-09-10T10:32:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40396.270812 secs 2023-09-10T10:32:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:46 smithi098 ceph-mon[103124]: pgmap v21702: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 594 KiB/s wr, 12 op/s 2023-09-10T10:32:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:46 smithi170 ceph-mon[102832]: pgmap v21702: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 594 KiB/s wr, 12 op/s 2023-09-10T10:32:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:46 smithi082 ceph-mon[98694]: pgmap v21702: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 594 KiB/s wr, 12 op/s 2023-09-10T10:32:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:48 smithi098 ceph-mon[103124]: pgmap v21703: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 791 KiB/s wr, 19 op/s 2023-09-10T10:32:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:48 smithi170 ceph-mon[102832]: pgmap v21703: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 791 KiB/s wr, 19 op/s 2023-09-10T10:32:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:48 smithi082 ceph-mon[98694]: pgmap v21703: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 791 KiB/s wr, 19 op/s 2023-09-10T10:32:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40400.919152 secs 2023-09-10T10:32:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40400.919152 secs 2023-09-10T10:32:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40400.919152 secs 2023-09-10T10:32:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:50 smithi098 ceph-mon[103124]: pgmap v21704: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 569 KiB/s wr, 13 op/s 2023-09-10T10:32:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40401.270875 secs 2023-09-10T10:32:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:32:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:50 smithi170 ceph-mon[102832]: pgmap v21704: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 569 KiB/s wr, 13 op/s 2023-09-10T10:32:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40401.270875 secs 2023-09-10T10:32:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:32:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:50 smithi082 ceph-mon[98694]: pgmap v21704: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 569 KiB/s wr, 13 op/s 2023-09-10T10:32:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40401.270875 secs 2023-09-10T10:32:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:32:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:52 smithi098 ceph-mon[103124]: pgmap v21705: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 834 KiB/s wr, 20 op/s 2023-09-10T10:32:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:32:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:52 smithi170 ceph-mon[102832]: pgmap v21705: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 834 KiB/s wr, 20 op/s 2023-09-10T10:32:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:32:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:52 smithi082 ceph-mon[98694]: pgmap v21705: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 834 KiB/s wr, 20 op/s 2023-09-10T10:32:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:32:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:54 smithi098 ceph-mon[103124]: pgmap v21706: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 462 KiB/s wr, 14 op/s 2023-09-10T10:32:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:54 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:32:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40405.919212 secs 2023-09-10T10:32:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:54 smithi170 ceph-mon[102832]: pgmap v21706: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 462 KiB/s wr, 14 op/s 2023-09-10T10:32:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:54 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:32:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40405.919212 secs 2023-09-10T10:32:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:54 smithi082 ceph-mon[98694]: pgmap v21706: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 462 KiB/s wr, 14 op/s 2023-09-10T10:32:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:54 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:32:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40405.919212 secs 2023-09-10T10:32:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40406.270941 secs 2023-09-10T10:32:55.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:32:55.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:32:55.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:32:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40406.270941 secs 2023-09-10T10:32:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:32:55.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:32:55.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:32:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40406.270941 secs 2023-09-10T10:32:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:32:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:32:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:32:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:56 smithi098 ceph-mon[103124]: pgmap v21707: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 462 KiB/s wr, 14 op/s 2023-09-10T10:32:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:56 smithi170 ceph-mon[102832]: pgmap v21707: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 462 KiB/s wr, 14 op/s 2023-09-10T10:32:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:56 smithi082 ceph-mon[98694]: pgmap v21707: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 462 KiB/s wr, 14 op/s 2023-09-10T10:32:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:58 smithi098 ceph-mon[103124]: pgmap v21708: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 776 KiB/s wr, 21 op/s 2023-09-10T10:32:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:32:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:58 smithi170 ceph-mon[102832]: pgmap v21708: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 776 KiB/s wr, 21 op/s 2023-09-10T10:32:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:32:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:58 smithi082 ceph-mon[98694]: pgmap v21708: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 776 KiB/s wr, 21 op/s 2023-09-10T10:32:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:32:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:32:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:32:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40410.919296 secs 2023-09-10T10:32:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:32:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40410.919296 secs 2023-09-10T10:32:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:32:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40410.919296 secs 2023-09-10T10:33:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:00 smithi098 ceph-mon[103124]: pgmap v21709: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 579 KiB/s wr, 14 op/s 2023-09-10T10:33:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40411.271005 secs 2023-09-10T10:33:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:33:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:00 smithi170 ceph-mon[102832]: pgmap v21709: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 579 KiB/s wr, 14 op/s 2023-09-10T10:33:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40411.271005 secs 2023-09-10T10:33:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:33:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:00 smithi082 ceph-mon[98694]: pgmap v21709: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 579 KiB/s wr, 14 op/s 2023-09-10T10:33:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40411.271005 secs 2023-09-10T10:33:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:33:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:01.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:01 smithi170 ceph-mon[102832]: pgmap v21710: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 848 KiB/s wr, 21 op/s 2023-09-10T10:33:01.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:01 smithi082 ceph-mon[98694]: pgmap v21710: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 848 KiB/s wr, 21 op/s 2023-09-10T10:33:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:01 smithi098 ceph-mon[103124]: pgmap v21710: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 848 KiB/s wr, 21 op/s 2023-09-10T10:33:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:03 smithi170 ceph-mon[102832]: pgmap v21711: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 583 KiB/s wr, 14 op/s 2023-09-10T10:33:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40414.690473 secs 2023-09-10T10:33:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:03 smithi082 ceph-mon[98694]: pgmap v21711: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 583 KiB/s wr, 14 op/s 2023-09-10T10:33:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40414.690473 secs 2023-09-10T10:33:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:03 smithi098 ceph-mon[103124]: pgmap v21711: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 583 KiB/s wr, 14 op/s 2023-09-10T10:33:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40414.690473 secs 2023-09-10T10:33:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:33:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:33:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:04 smithi170 ceph-mon[102832]: osdmap e1425: 12 total, 12 up, 12 in 2023-09-10T10:33:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40415.919354 secs 2023-09-10T10:33:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:33:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:33:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:04.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:04 smithi082 ceph-mon[98694]: osdmap e1425: 12 total, 12 up, 12 in 2023-09-10T10:33:04.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40415.919354 secs 2023-09-10T10:33:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:33:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:33:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:04 smithi098 ceph-mon[103124]: osdmap e1425: 12 total, 12 up, 12 in 2023-09-10T10:33:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40415.919354 secs 2023-09-10T10:33:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40416.271071 secs 2023-09-10T10:33:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:05 smithi170 ceph-mon[102832]: pgmap v21713: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 700 KiB/s wr, 17 op/s 2023-09-10T10:33:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40416.271071 secs 2023-09-10T10:33:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:05 smithi082 ceph-mon[98694]: pgmap v21713: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 700 KiB/s wr, 17 op/s 2023-09-10T10:33:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40416.271071 secs 2023-09-10T10:33:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:05 smithi098 ceph-mon[103124]: pgmap v21713: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 700 KiB/s wr, 17 op/s 2023-09-10T10:33:07.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:07 smithi170 ceph-mon[102832]: pgmap v21714: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 724 KiB/s wr, 34 op/s 2023-09-10T10:33:07.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:07 smithi082 ceph-mon[98694]: pgmap v21714: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 724 KiB/s wr, 34 op/s 2023-09-10T10:33:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:07 smithi098 ceph-mon[103124]: pgmap v21714: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 724 KiB/s wr, 34 op/s 2023-09-10T10:33:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40419.690532 secs 2023-09-10T10:33:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40419.690532 secs 2023-09-10T10:33:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40419.690532 secs 2023-09-10T10:33:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:09 smithi082 ceph-mon[98694]: pgmap v21715: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 724 KiB/s wr, 34 op/s 2023-09-10T10:33:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40420.919409 secs 2023-09-10T10:33:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:09 smithi098 ceph-mon[103124]: pgmap v21715: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 724 KiB/s wr, 34 op/s 2023-09-10T10:33:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40420.919409 secs 2023-09-10T10:33:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:09 smithi170 ceph-mon[102832]: pgmap v21715: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 724 KiB/s wr, 34 op/s 2023-09-10T10:33:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40420.919409 secs 2023-09-10T10:33:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40421.271135 secs 2023-09-10T10:33:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:33:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:10.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:10 smithi082 ceph-mon[98694]: osdmap e1426: 12 total, 12 up, 12 in 2023-09-10T10:33:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40421.271135 secs 2023-09-10T10:33:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:33:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:10 smithi098 ceph-mon[103124]: osdmap e1426: 12 total, 12 up, 12 in 2023-09-10T10:33:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40421.271135 secs 2023-09-10T10:33:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:33:10.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:10.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:10 smithi170 ceph-mon[102832]: osdmap e1426: 12 total, 12 up, 12 in 2023-09-10T10:33:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:12 smithi170 ceph-mon[102832]: pgmap v21717: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 770 KiB/s wr, 41 op/s 2023-09-10T10:33:12.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:12 smithi082 ceph-mon[98694]: pgmap v21717: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 770 KiB/s wr, 41 op/s 2023-09-10T10:33:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:12 smithi098 ceph-mon[103124]: pgmap v21717: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 770 KiB/s wr, 41 op/s 2023-09-10T10:33:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:33:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:33:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:33:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:14 smithi170 ceph-mon[102832]: pgmap v21718: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 689 KiB/s wr, 36 op/s 2023-09-10T10:33:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40424.690591 secs 2023-09-10T10:33:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40425.919467 secs 2023-09-10T10:33:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:14 smithi082 ceph-mon[98694]: pgmap v21718: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 689 KiB/s wr, 36 op/s 2023-09-10T10:33:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40424.690591 secs 2023-09-10T10:33:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40425.919467 secs 2023-09-10T10:33:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:14 smithi098 ceph-mon[103124]: pgmap v21718: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 689 KiB/s wr, 36 op/s 2023-09-10T10:33:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40424.690591 secs 2023-09-10T10:33:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40425.919467 secs 2023-09-10T10:33:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40426.271199 secs 2023-09-10T10:33:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40426.271199 secs 2023-09-10T10:33:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40426.271199 secs 2023-09-10T10:33:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:16 smithi170 ceph-mon[102832]: pgmap v21719: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 616 KiB/s wr, 32 op/s 2023-09-10T10:33:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:16 smithi082 ceph-mon[98694]: pgmap v21719: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 616 KiB/s wr, 32 op/s 2023-09-10T10:33:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:16 smithi098 ceph-mon[103124]: pgmap v21719: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 616 KiB/s wr, 32 op/s 2023-09-10T10:33:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:18 smithi170 ceph-mon[102832]: pgmap v21720: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 357 KiB/s wr, 13 op/s 2023-09-10T10:33:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:33:18.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:18 smithi082 ceph-mon[98694]: pgmap v21720: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 357 KiB/s wr, 13 op/s 2023-09-10T10:33:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:33:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:18 smithi098 ceph-mon[103124]: pgmap v21720: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 357 KiB/s wr, 13 op/s 2023-09-10T10:33:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:33:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40429.690650 secs 2023-09-10T10:33:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40430.919546 secs 2023-09-10T10:33:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40429.690650 secs 2023-09-10T10:33:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40430.919546 secs 2023-09-10T10:33:19.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40429.690650 secs 2023-09-10T10:33:19.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40430.919546 secs 2023-09-10T10:33:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:20 smithi170 ceph-mon[102832]: pgmap v21721: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 357 KiB/s wr, 13 op/s 2023-09-10T10:33:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40431.271263 secs 2023-09-10T10:33:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:20 smithi082 ceph-mon[98694]: pgmap v21721: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 357 KiB/s wr, 13 op/s 2023-09-10T10:33:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40431.271263 secs 2023-09-10T10:33:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:20 smithi098 ceph-mon[103124]: pgmap v21721: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 357 KiB/s wr, 13 op/s 2023-09-10T10:33:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40431.271263 secs 2023-09-10T10:33:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:21 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:33:21.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:21 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:21.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:21 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:33:21.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:21 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:21 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:33:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:21 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:22 smithi170 ceph-mon[102832]: pgmap v21722: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 347 KiB/s wr, 19 op/s 2023-09-10T10:33:22.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:22 smithi082 ceph-mon[98694]: pgmap v21722: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 347 KiB/s wr, 19 op/s 2023-09-10T10:33:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:22 smithi098 ceph-mon[103124]: pgmap v21722: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 347 KiB/s wr, 19 op/s 2023-09-10T10:33:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:24 smithi170 ceph-mon[102832]: pgmap v21723: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T10:33:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40434.690712 secs 2023-09-10T10:33:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40435.919603 secs 2023-09-10T10:33:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:24 smithi082 ceph-mon[98694]: pgmap v21723: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T10:33:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40434.690712 secs 2023-09-10T10:33:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40435.919603 secs 2023-09-10T10:33:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:24 smithi098 ceph-mon[103124]: pgmap v21723: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T10:33:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40434.690712 secs 2023-09-10T10:33:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40435.919603 secs 2023-09-10T10:33:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40436.271327 secs 2023-09-10T10:33:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40436.271327 secs 2023-09-10T10:33:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40436.271327 secs 2023-09-10T10:33:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:26 smithi170 ceph-mon[102832]: pgmap v21724: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T10:33:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:26 smithi082 ceph-mon[98694]: pgmap v21724: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T10:33:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:26 smithi098 ceph-mon[103124]: pgmap v21724: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 119 KiB/s wr, 10 op/s 2023-09-10T10:33:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:28 smithi170 ceph-mon[102832]: pgmap v21725: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 119 KiB/s wr, 15 op/s 2023-09-10T10:33:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:28 smithi082 ceph-mon[98694]: pgmap v21725: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 119 KiB/s wr, 15 op/s 2023-09-10T10:33:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:28 smithi098 ceph-mon[103124]: pgmap v21725: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 119 KiB/s wr, 15 op/s 2023-09-10T10:33:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40439.690831 secs 2023-09-10T10:33:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40440.919661 secs 2023-09-10T10:33:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40439.690831 secs 2023-09-10T10:33:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40440.919661 secs 2023-09-10T10:33:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40439.690831 secs 2023-09-10T10:33:29.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40440.919661 secs 2023-09-10T10:33:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:30 smithi170 ceph-mon[102832]: pgmap v21726: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 10 op/s 2023-09-10T10:33:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40441.271393 secs 2023-09-10T10:33:30.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:30 smithi082 ceph-mon[98694]: pgmap v21726: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 10 op/s 2023-09-10T10:33:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40441.271393 secs 2023-09-10T10:33:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:30 smithi098 ceph-mon[103124]: pgmap v21726: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 10 op/s 2023-09-10T10:33:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40441.271393 secs 2023-09-10T10:33:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:32 smithi170 ceph-mon[102832]: pgmap v21727: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 15 op/s 2023-09-10T10:33:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:32 smithi082 ceph-mon[98694]: pgmap v21727: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 15 op/s 2023-09-10T10:33:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:32 smithi098 ceph-mon[103124]: pgmap v21727: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 15 op/s 2023-09-10T10:33:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:34 smithi170 ceph-mon[102832]: pgmap v21728: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 10 op/s 2023-09-10T10:33:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40444.690892 secs 2023-09-10T10:33:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40445.919745 secs 2023-09-10T10:33:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:34 smithi082 ceph-mon[98694]: pgmap v21728: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 10 op/s 2023-09-10T10:33:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40444.690892 secs 2023-09-10T10:33:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40445.919745 secs 2023-09-10T10:33:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:34 smithi098 ceph-mon[103124]: pgmap v21728: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 10 op/s 2023-09-10T10:33:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40444.690892 secs 2023-09-10T10:33:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40445.919745 secs 2023-09-10T10:33:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40446.271460 secs 2023-09-10T10:33:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40446.271460 secs 2023-09-10T10:33:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40446.271460 secs 2023-09-10T10:33:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:36 smithi170 ceph-mon[102832]: pgmap v21729: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 10 op/s 2023-09-10T10:33:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:36 smithi082 ceph-mon[98694]: pgmap v21729: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 10 op/s 2023-09-10T10:33:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:36 smithi098 ceph-mon[103124]: pgmap v21729: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 10 op/s 2023-09-10T10:33:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:38 smithi170 ceph-mon[102832]: pgmap v21730: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 200 KiB/s wr, 16 op/s 2023-09-10T10:33:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:38 smithi082 ceph-mon[98694]: pgmap v21730: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 200 KiB/s wr, 16 op/s 2023-09-10T10:33:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:38 smithi098 ceph-mon[103124]: pgmap v21730: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 200 KiB/s wr, 16 op/s 2023-09-10T10:33:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40450.919805 secs 2023-09-10T10:33:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40450.919805 secs 2023-09-10T10:33:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40450.919805 secs 2023-09-10T10:33:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:40 smithi170 ceph-mon[102832]: pgmap v21731: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 200 KiB/s wr, 11 op/s 2023-09-10T10:33:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40451.271490 secs 2023-09-10T10:33:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:40 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:33:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:40 smithi082 ceph-mon[98694]: pgmap v21731: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 200 KiB/s wr, 11 op/s 2023-09-10T10:33:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40451.271490 secs 2023-09-10T10:33:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:40 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:33:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:40 smithi098 ceph-mon[103124]: pgmap v21731: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 200 KiB/s wr, 11 op/s 2023-09-10T10:33:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40451.271490 secs 2023-09-10T10:33:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:40 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:33:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:41 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:33:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:41 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:33:41.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:41 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:41 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:33:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:41 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:33:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:41 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:41 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:33:41.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:41 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:33:41.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:41 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:42 smithi082 ceph-mon[98694]: pgmap v21732: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 296 KiB/s wr, 16 op/s 2023-09-10T10:33:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:42 smithi098 ceph-mon[103124]: pgmap v21732: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 296 KiB/s wr, 16 op/s 2023-09-10T10:33:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:42 smithi170 ceph-mon[102832]: pgmap v21732: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 296 KiB/s wr, 16 op/s 2023-09-10T10:33:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:44 smithi082 ceph-mon[98694]: pgmap v21733: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 295 KiB/s wr, 11 op/s 2023-09-10T10:33:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40455.919880 secs 2023-09-10T10:33:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:44 smithi098 ceph-mon[103124]: pgmap v21733: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 295 KiB/s wr, 11 op/s 2023-09-10T10:33:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40455.919880 secs 2023-09-10T10:33:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:44 smithi170 ceph-mon[102832]: pgmap v21733: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 295 KiB/s wr, 11 op/s 2023-09-10T10:33:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40455.919880 secs 2023-09-10T10:33:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40456.271554 secs 2023-09-10T10:33:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:45 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:33:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:45 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40456.271554 secs 2023-09-10T10:33:45.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:45 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:33:45.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:45 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40456.271554 secs 2023-09-10T10:33:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:45 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:33:45.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:45 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:46 smithi082 ceph-mon[98694]: pgmap v21734: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 295 KiB/s wr, 11 op/s 2023-09-10T10:33:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:46 smithi098 ceph-mon[103124]: pgmap v21734: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 295 KiB/s wr, 11 op/s 2023-09-10T10:33:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:46 smithi170 ceph-mon[102832]: pgmap v21734: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 295 KiB/s wr, 11 op/s 2023-09-10T10:33:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:48 smithi082 ceph-mon[98694]: pgmap v21735: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 615 KiB/s wr, 17 op/s 2023-09-10T10:33:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:48 smithi098 ceph-mon[103124]: pgmap v21735: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 615 KiB/s wr, 17 op/s 2023-09-10T10:33:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:48 smithi170 ceph-mon[102832]: pgmap v21735: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 615 KiB/s wr, 17 op/s 2023-09-10T10:33:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40460.919939 secs 2023-09-10T10:33:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40460.919939 secs 2023-09-10T10:33:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40460.919939 secs 2023-09-10T10:33:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:50 smithi082 ceph-mon[98694]: pgmap v21736: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 416 KiB/s wr, 11 op/s 2023-09-10T10:33:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40461.271619 secs 2023-09-10T10:33:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:33:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:50 smithi098 ceph-mon[103124]: pgmap v21736: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 416 KiB/s wr, 11 op/s 2023-09-10T10:33:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40461.271619 secs 2023-09-10T10:33:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:33:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:50 smithi170 ceph-mon[102832]: pgmap v21736: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 416 KiB/s wr, 11 op/s 2023-09-10T10:33:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40461.271619 secs 2023-09-10T10:33:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:33:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:52 smithi098 ceph-mon[103124]: pgmap v21737: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 473 KiB/s wr, 18 op/s 2023-09-10T10:33:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:33:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:52 smithi170 ceph-mon[102832]: pgmap v21737: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 473 KiB/s wr, 18 op/s 2023-09-10T10:33:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:33:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:52 smithi082 ceph-mon[98694]: pgmap v21737: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 473 KiB/s wr, 18 op/s 2023-09-10T10:33:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:33:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:54 smithi098 ceph-mon[103124]: pgmap v21738: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 377 KiB/s wr, 12 op/s 2023-09-10T10:33:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:33:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40465.920001 secs 2023-09-10T10:33:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:54 smithi170 ceph-mon[102832]: pgmap v21738: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 377 KiB/s wr, 12 op/s 2023-09-10T10:33:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:33:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40465.920001 secs 2023-09-10T10:33:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:54 smithi082 ceph-mon[98694]: pgmap v21738: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 377 KiB/s wr, 12 op/s 2023-09-10T10:33:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:33:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:54.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40465.920001 secs 2023-09-10T10:33:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40466.271685 secs 2023-09-10T10:33:55.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:33:55.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:33:55.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:33:55.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:55 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:33:55.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40466.271685 secs 2023-09-10T10:33:55.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:33:55.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:33:55.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:33:55.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:55 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:33:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40466.271685 secs 2023-09-10T10:33:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:33:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:33:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:33:55.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:55 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:33:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:56 smithi098 ceph-mon[103124]: pgmap v21739: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 377 KiB/s wr, 12 op/s 2023-09-10T10:33:56.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:33:56.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:56 smithi170 ceph-mon[102832]: pgmap v21739: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 377 KiB/s wr, 12 op/s 2023-09-10T10:33:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:33:56.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:56 smithi082 ceph-mon[98694]: pgmap v21739: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 377 KiB/s wr, 12 op/s 2023-09-10T10:33:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:33:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:33:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:58 smithi098 ceph-mon[103124]: pgmap v21740: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 903 KiB/s wr, 19 op/s 2023-09-10T10:33:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:58 smithi170 ceph-mon[102832]: pgmap v21740: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 903 KiB/s wr, 19 op/s 2023-09-10T10:33:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:58 smithi082 ceph-mon[98694]: pgmap v21740: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 903 KiB/s wr, 19 op/s 2023-09-10T10:33:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:33:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40470.920060 secs 2023-09-10T10:33:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:33:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40470.920060 secs 2023-09-10T10:33:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:33:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40470.920060 secs 2023-09-10T10:34:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:00 smithi098 ceph-mon[103124]: pgmap v21741: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 583 KiB/s wr, 13 op/s 2023-09-10T10:34:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40471.271758 secs 2023-09-10T10:34:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:34:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:00 smithi170 ceph-mon[102832]: pgmap v21741: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 583 KiB/s wr, 13 op/s 2023-09-10T10:34:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40471.271758 secs 2023-09-10T10:34:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:34:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:00 smithi082 ceph-mon[98694]: pgmap v21741: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 583 KiB/s wr, 13 op/s 2023-09-10T10:34:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40471.271758 secs 2023-09-10T10:34:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:34:00.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:02 smithi098 ceph-mon[103124]: pgmap v21742: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 786 KiB/s wr, 20 op/s 2023-09-10T10:34:02.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:02 smithi170 ceph-mon[102832]: pgmap v21742: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 786 KiB/s wr, 20 op/s 2023-09-10T10:34:02.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:02 smithi082 ceph-mon[98694]: pgmap v21742: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 786 KiB/s wr, 20 op/s 2023-09-10T10:34:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:03 smithi170 ceph-mon[102832]: pgmap v21743: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 728 KiB/s wr, 13 op/s 2023-09-10T10:34:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:03 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:34:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:03 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:34:03.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:03 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:03 smithi082 ceph-mon[98694]: pgmap v21743: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 728 KiB/s wr, 13 op/s 2023-09-10T10:34:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:03 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:34:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:03 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:34:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:03 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:03 smithi098 ceph-mon[103124]: pgmap v21743: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 728 KiB/s wr, 13 op/s 2023-09-10T10:34:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:03 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:34:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:03 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:34:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:03 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40474.691300 secs 2023-09-10T10:34:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40475.920119 secs 2023-09-10T10:34:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40474.691300 secs 2023-09-10T10:34:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40475.920119 secs 2023-09-10T10:34:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40474.691300 secs 2023-09-10T10:34:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40475.920119 secs 2023-09-10T10:34:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40476.271814 secs 2023-09-10T10:34:05.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:05 smithi170 ceph-mon[102832]: pgmap v21744: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 728 KiB/s wr, 13 op/s 2023-09-10T10:34:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40476.271814 secs 2023-09-10T10:34:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:05 smithi082 ceph-mon[98694]: pgmap v21744: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 728 KiB/s wr, 13 op/s 2023-09-10T10:34:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40476.271814 secs 2023-09-10T10:34:05.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:05 smithi098 ceph-mon[103124]: pgmap v21744: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 728 KiB/s wr, 13 op/s 2023-09-10T10:34:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:34:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:34:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:34:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:07.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:07 smithi170 ceph-mon[102832]: pgmap v21745: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 36 op/s 2023-09-10T10:34:07.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:07 smithi082 ceph-mon[98694]: pgmap v21745: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 36 op/s 2023-09-10T10:34:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:07 smithi098 ceph-mon[103124]: pgmap v21745: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 36 op/s 2023-09-10T10:34:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40479.691400 secs 2023-09-10T10:34:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:09 smithi170 ceph-mon[102832]: osdmap e1427: 12 total, 12 up, 12 in 2023-09-10T10:34:09.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:09 smithi170 ceph-mon[102832]: pgmap v21747: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 944 KiB/s wr, 34 op/s 2023-09-10T10:34:09.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40480.920179 secs 2023-09-10T10:34:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40479.691400 secs 2023-09-10T10:34:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:09 smithi082 ceph-mon[98694]: osdmap e1427: 12 total, 12 up, 12 in 2023-09-10T10:34:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:09 smithi082 ceph-mon[98694]: pgmap v21747: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 944 KiB/s wr, 34 op/s 2023-09-10T10:34:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40480.920179 secs 2023-09-10T10:34:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40479.691400 secs 2023-09-10T10:34:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:09 smithi098 ceph-mon[103124]: osdmap e1427: 12 total, 12 up, 12 in 2023-09-10T10:34:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:09 smithi098 ceph-mon[103124]: pgmap v21747: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 944 KiB/s wr, 34 op/s 2023-09-10T10:34:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40480.920179 secs 2023-09-10T10:34:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40481.271877 secs 2023-09-10T10:34:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40481.271877 secs 2023-09-10T10:34:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40481.271877 secs 2023-09-10T10:34:11.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:11 smithi170 ceph-mon[102832]: pgmap v21748: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 925 KiB/s wr, 34 op/s 2023-09-10T10:34:11.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:11 smithi082 ceph-mon[98694]: pgmap v21748: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 925 KiB/s wr, 34 op/s 2023-09-10T10:34:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:11 smithi098 ceph-mon[103124]: pgmap v21748: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 925 KiB/s wr, 34 op/s 2023-09-10T10:34:13.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:13 smithi082 ceph-mon[98694]: pgmap v21749: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 925 KiB/s wr, 34 op/s 2023-09-10T10:34:13.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40484.691487 secs 2023-09-10T10:34:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:13 smithi098 ceph-mon[103124]: pgmap v21749: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 925 KiB/s wr, 34 op/s 2023-09-10T10:34:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40484.691487 secs 2023-09-10T10:34:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:13 smithi170 ceph-mon[102832]: pgmap v21749: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 925 KiB/s wr, 34 op/s 2023-09-10T10:34:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40484.691487 secs 2023-09-10T10:34:14.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40485.920235 secs 2023-09-10T10:34:14.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40485.920235 secs 2023-09-10T10:34:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40485.920235 secs 2023-09-10T10:34:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40486.271942 secs 2023-09-10T10:34:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:15 smithi082 ceph-mon[98694]: pgmap v21750: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 925 KiB/s wr, 34 op/s 2023-09-10T10:34:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:15 smithi082 ceph-mon[98694]: osdmap e1428: 12 total, 12 up, 12 in 2023-09-10T10:34:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40486.271942 secs 2023-09-10T10:34:15.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:15 smithi098 ceph-mon[103124]: pgmap v21750: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 925 KiB/s wr, 34 op/s 2023-09-10T10:34:15.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:15 smithi098 ceph-mon[103124]: osdmap e1428: 12 total, 12 up, 12 in 2023-09-10T10:34:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40486.271942 secs 2023-09-10T10:34:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:15 smithi170 ceph-mon[102832]: pgmap v21750: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 925 KiB/s wr, 34 op/s 2023-09-10T10:34:15.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:15 smithi170 ceph-mon[102832]: osdmap e1428: 12 total, 12 up, 12 in 2023-09-10T10:34:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:17 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:34:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:17 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:17 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:34:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:17 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:17 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:34:17.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:17 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:18 smithi170 ceph-mon[102832]: pgmap v21752: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 531 KiB/s wr, 19 op/s 2023-09-10T10:34:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:34:18.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:18 smithi082 ceph-mon[98694]: pgmap v21752: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 531 KiB/s wr, 19 op/s 2023-09-10T10:34:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:34:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:18 smithi098 ceph-mon[103124]: pgmap v21752: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 531 KiB/s wr, 19 op/s 2023-09-10T10:34:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:34:18.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40489.691544 secs 2023-09-10T10:34:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40490.920294 secs 2023-09-10T10:34:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40489.691544 secs 2023-09-10T10:34:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40490.920294 secs 2023-09-10T10:34:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40489.691544 secs 2023-09-10T10:34:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40490.920294 secs 2023-09-10T10:34:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:20 smithi170 ceph-mon[102832]: pgmap v21753: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 428 KiB/s wr, 15 op/s 2023-09-10T10:34:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40491.272007 secs 2023-09-10T10:34:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:20 smithi082 ceph-mon[98694]: pgmap v21753: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 428 KiB/s wr, 15 op/s 2023-09-10T10:34:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40491.272007 secs 2023-09-10T10:34:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:20 smithi098 ceph-mon[103124]: pgmap v21753: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 428 KiB/s wr, 15 op/s 2023-09-10T10:34:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40491.272007 secs 2023-09-10T10:34:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:22 smithi170 ceph-mon[102832]: pgmap v21754: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 241 KiB/s wr, 14 op/s 2023-09-10T10:34:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:22 smithi082 ceph-mon[98694]: pgmap v21754: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 241 KiB/s wr, 14 op/s 2023-09-10T10:34:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:22 smithi098 ceph-mon[103124]: pgmap v21754: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 241 KiB/s wr, 14 op/s 2023-09-10T10:34:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:24 smithi170 ceph-mon[102832]: pgmap v21755: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 241 KiB/s wr, 14 op/s 2023-09-10T10:34:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40494.691609 secs 2023-09-10T10:34:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40495.920354 secs 2023-09-10T10:34:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:24 smithi082 ceph-mon[98694]: pgmap v21755: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 241 KiB/s wr, 14 op/s 2023-09-10T10:34:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40494.691609 secs 2023-09-10T10:34:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40495.920354 secs 2023-09-10T10:34:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:24 smithi098 ceph-mon[103124]: pgmap v21755: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 241 KiB/s wr, 14 op/s 2023-09-10T10:34:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40494.691609 secs 2023-09-10T10:34:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40495.920354 secs 2023-09-10T10:34:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40496.272071 secs 2023-09-10T10:34:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40496.272071 secs 2023-09-10T10:34:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40496.272071 secs 2023-09-10T10:34:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:26 smithi170 ceph-mon[102832]: pgmap v21756: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 241 KiB/s wr, 14 op/s 2023-09-10T10:34:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:26 smithi082 ceph-mon[98694]: pgmap v21756: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 241 KiB/s wr, 14 op/s 2023-09-10T10:34:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:26 smithi098 ceph-mon[103124]: pgmap v21756: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 241 KiB/s wr, 14 op/s 2023-09-10T10:34:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:28 smithi170 ceph-mon[102832]: pgmap v21757: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 214 KiB/s wr, 18 op/s 2023-09-10T10:34:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:28 smithi082 ceph-mon[98694]: pgmap v21757: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 214 KiB/s wr, 18 op/s 2023-09-10T10:34:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:28 smithi098 ceph-mon[103124]: pgmap v21757: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 214 KiB/s wr, 18 op/s 2023-09-10T10:34:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40499.691722 secs 2023-09-10T10:34:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40500.920439 secs 2023-09-10T10:34:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40499.691722 secs 2023-09-10T10:34:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40500.920439 secs 2023-09-10T10:34:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40499.691722 secs 2023-09-10T10:34:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40500.920439 secs 2023-09-10T10:34:30.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:30 smithi170 ceph-mon[102832]: pgmap v21758: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T10:34:30.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40501.272136 secs 2023-09-10T10:34:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:30 smithi082 ceph-mon[98694]: pgmap v21758: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T10:34:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40501.272136 secs 2023-09-10T10:34:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:30 smithi098 ceph-mon[103124]: pgmap v21758: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 11 op/s 2023-09-10T10:34:30.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40501.272136 secs 2023-09-10T10:34:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:32 smithi170 ceph-mon[102832]: pgmap v21759: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 17 op/s 2023-09-10T10:34:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:32 smithi082 ceph-mon[98694]: pgmap v21759: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 17 op/s 2023-09-10T10:34:32.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:32 smithi098 ceph-mon[103124]: pgmap v21759: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 180 KiB/s wr, 17 op/s 2023-09-10T10:34:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:34 smithi170 ceph-mon[102832]: pgmap v21760: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T10:34:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40504.691776 secs 2023-09-10T10:34:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40505.920497 secs 2023-09-10T10:34:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:34 smithi082 ceph-mon[98694]: pgmap v21760: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T10:34:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40504.691776 secs 2023-09-10T10:34:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40505.920497 secs 2023-09-10T10:34:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:34 smithi098 ceph-mon[103124]: pgmap v21760: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T10:34:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40504.691776 secs 2023-09-10T10:34:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40505.920497 secs 2023-09-10T10:34:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40506.272200 secs 2023-09-10T10:34:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40506.272200 secs 2023-09-10T10:34:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40506.272200 secs 2023-09-10T10:34:36.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:36 smithi170 ceph-mon[102832]: pgmap v21761: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T10:34:36.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:36 smithi082 ceph-mon[98694]: pgmap v21761: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T10:34:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:36 smithi098 ceph-mon[103124]: pgmap v21761: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T10:34:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:38 smithi170 ceph-mon[102832]: pgmap v21762: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 150 KiB/s wr, 16 op/s 2023-09-10T10:34:38.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:34:38.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:38 smithi082 ceph-mon[98694]: pgmap v21762: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 150 KiB/s wr, 16 op/s 2023-09-10T10:34:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:34:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:38 smithi098 ceph-mon[103124]: pgmap v21762: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 150 KiB/s wr, 16 op/s 2023-09-10T10:34:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:34:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:39.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40510.920583 secs 2023-09-10T10:34:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40510.920583 secs 2023-09-10T10:34:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40510.920583 secs 2023-09-10T10:34:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:40 smithi170 ceph-mon[102832]: pgmap v21763: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T10:34:40.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40511.272264 secs 2023-09-10T10:34:40.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:40 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:34:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:40 smithi082 ceph-mon[98694]: pgmap v21763: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T10:34:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40511.272264 secs 2023-09-10T10:34:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:40 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:34:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:40 smithi098 ceph-mon[103124]: pgmap v21763: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T10:34:40.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40511.272264 secs 2023-09-10T10:34:40.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:40 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:34:41.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:41 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:34:41.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:41 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:34:41.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:41 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:41 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:34:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:41 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:34:41.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:41 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:41.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:41 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:34:41.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:41 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:34:41.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:41 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:42.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:42 smithi170 ceph-mon[102832]: pgmap v21764: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 507 KiB/s wr, 17 op/s 2023-09-10T10:34:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:42 smithi082 ceph-mon[98694]: pgmap v21764: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 507 KiB/s wr, 17 op/s 2023-09-10T10:34:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:42 smithi098 ceph-mon[103124]: pgmap v21764: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 507 KiB/s wr, 17 op/s 2023-09-10T10:34:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:44 smithi170 ceph-mon[102832]: pgmap v21765: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 358 KiB/s wr, 11 op/s 2023-09-10T10:34:44.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40515.920642 secs 2023-09-10T10:34:44.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:44 smithi082 ceph-mon[98694]: pgmap v21765: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 358 KiB/s wr, 11 op/s 2023-09-10T10:34:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40515.920642 secs 2023-09-10T10:34:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:44 smithi098 ceph-mon[103124]: pgmap v21765: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 358 KiB/s wr, 11 op/s 2023-09-10T10:34:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40515.920642 secs 2023-09-10T10:34:45.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40516.272329 secs 2023-09-10T10:34:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40516.272329 secs 2023-09-10T10:34:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40516.272329 secs 2023-09-10T10:34:46.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:46 smithi082 ceph-mon[98694]: pgmap v21766: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 358 KiB/s wr, 11 op/s 2023-09-10T10:34:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:46 smithi098 ceph-mon[103124]: pgmap v21766: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 358 KiB/s wr, 11 op/s 2023-09-10T10:34:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:46 smithi170 ceph-mon[102832]: pgmap v21766: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 358 KiB/s wr, 11 op/s 2023-09-10T10:34:48.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:48 smithi082 ceph-mon[98694]: pgmap v21767: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 649 KiB/s wr, 18 op/s 2023-09-10T10:34:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:48 smithi098 ceph-mon[103124]: pgmap v21767: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 649 KiB/s wr, 18 op/s 2023-09-10T10:34:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:48 smithi170 ceph-mon[102832]: pgmap v21767: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 649 KiB/s wr, 18 op/s 2023-09-10T10:34:49.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40520.920704 secs 2023-09-10T10:34:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40520.920704 secs 2023-09-10T10:34:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40520.920704 secs 2023-09-10T10:34:50.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:50 smithi082 ceph-mon[98694]: pgmap v21768: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 648 KiB/s wr, 13 op/s 2023-09-10T10:34:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40521.272391 secs 2023-09-10T10:34:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:34:50.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:50 smithi098 ceph-mon[103124]: pgmap v21768: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 648 KiB/s wr, 13 op/s 2023-09-10T10:34:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40521.272391 secs 2023-09-10T10:34:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:34:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:50 smithi170 ceph-mon[102832]: pgmap v21768: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 648 KiB/s wr, 13 op/s 2023-09-10T10:34:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40521.272391 secs 2023-09-10T10:34:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:34:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:52 smithi082 ceph-mon[98694]: pgmap v21769: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 888 KiB/s wr, 21 op/s 2023-09-10T10:34:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:34:52.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:52 smithi098 ceph-mon[103124]: pgmap v21769: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 888 KiB/s wr, 21 op/s 2023-09-10T10:34:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:34:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:52 smithi170 ceph-mon[102832]: pgmap v21769: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 888 KiB/s wr, 21 op/s 2023-09-10T10:34:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:34:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:54 smithi082 ceph-mon[98694]: pgmap v21770: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 530 KiB/s wr, 14 op/s 2023-09-10T10:34:54.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40525.920787 secs 2023-09-10T10:34:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:54 smithi098 ceph-mon[103124]: pgmap v21770: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 530 KiB/s wr, 14 op/s 2023-09-10T10:34:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40525.920787 secs 2023-09-10T10:34:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:54 smithi170 ceph-mon[102832]: pgmap v21770: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 530 KiB/s wr, 14 op/s 2023-09-10T10:34:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40525.920787 secs 2023-09-10T10:34:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40526.272457 secs 2023-09-10T10:34:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40526.272457 secs 2023-09-10T10:34:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40526.272457 secs 2023-09-10T10:34:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:56 smithi098 ceph-mon[103124]: pgmap v21771: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 530 KiB/s wr, 14 op/s 2023-09-10T10:34:56.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:34:56.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:34:56.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:34:56.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:34:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:56 smithi170 ceph-mon[102832]: pgmap v21771: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 530 KiB/s wr, 14 op/s 2023-09-10T10:34:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:34:56.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:34:56.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:34:56.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:34:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:56 smithi082 ceph-mon[98694]: pgmap v21771: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 530 KiB/s wr, 14 op/s 2023-09-10T10:34:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:34:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:34:56.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:34:56.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:34:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:58 smithi098 ceph-mon[103124]: pgmap v21772: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 845 KiB/s wr, 22 op/s 2023-09-10T10:34:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:34:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:58 smithi170 ceph-mon[102832]: pgmap v21772: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 845 KiB/s wr, 22 op/s 2023-09-10T10:34:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:34:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:58 smithi082 ceph-mon[98694]: pgmap v21772: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 845 KiB/s wr, 22 op/s 2023-09-10T10:34:58.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:34:58.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:34:59.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:34:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40530.920855 secs 2023-09-10T10:34:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:34:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40530.920855 secs 2023-09-10T10:34:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:34:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40530.920855 secs 2023-09-10T10:35:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:00 smithi098 ceph-mon[103124]: pgmap v21773: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 554 KiB/s wr, 14 op/s 2023-09-10T10:35:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40531.272488 secs 2023-09-10T10:35:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:35:00.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:00 smithi170 ceph-mon[102832]: pgmap v21773: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 554 KiB/s wr, 14 op/s 2023-09-10T10:35:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40531.272488 secs 2023-09-10T10:35:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:35:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:00 smithi082 ceph-mon[98694]: pgmap v21773: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 554 KiB/s wr, 14 op/s 2023-09-10T10:35:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40531.272488 secs 2023-09-10T10:35:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:35:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:02 smithi098 ceph-mon[103124]: pgmap v21774: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 23 op/s 2023-09-10T10:35:02.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:02 smithi170 ceph-mon[102832]: pgmap v21774: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 23 op/s 2023-09-10T10:35:02.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:02 smithi082 ceph-mon[98694]: pgmap v21774: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 23 op/s 2023-09-10T10:35:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:03 smithi170 ceph-mon[102832]: pgmap v21775: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 899 KiB/s wr, 15 op/s 2023-09-10T10:35:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:03 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:35:03.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:03 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:35:03.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:03 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:03 smithi082 ceph-mon[98694]: pgmap v21775: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 899 KiB/s wr, 15 op/s 2023-09-10T10:35:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:03 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:35:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:03 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:35:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:03 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:03 smithi098 ceph-mon[103124]: pgmap v21775: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 899 KiB/s wr, 15 op/s 2023-09-10T10:35:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:03 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:35:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:03 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:35:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:03 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40534.692191 secs 2023-09-10T10:35:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40535.920922 secs 2023-09-10T10:35:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40534.692191 secs 2023-09-10T10:35:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40535.920922 secs 2023-09-10T10:35:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40534.692191 secs 2023-09-10T10:35:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40535.920922 secs 2023-09-10T10:35:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40536.272554 secs 2023-09-10T10:35:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:05 smithi170 ceph-mon[102832]: pgmap v21776: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 899 KiB/s wr, 15 op/s 2023-09-10T10:35:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40536.272554 secs 2023-09-10T10:35:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:05 smithi082 ceph-mon[98694]: pgmap v21776: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 899 KiB/s wr, 15 op/s 2023-09-10T10:35:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40536.272554 secs 2023-09-10T10:35:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:05 smithi098 ceph-mon[103124]: pgmap v21776: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 899 KiB/s wr, 15 op/s 2023-09-10T10:35:07.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:07 smithi082 ceph-mon[98694]: pgmap v21777: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 36 op/s 2023-09-10T10:35:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:07 smithi098 ceph-mon[103124]: pgmap v21777: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 36 op/s 2023-09-10T10:35:07.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:07 smithi170 ceph-mon[102832]: pgmap v21777: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 36 op/s 2023-09-10T10:35:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40539.692294 secs 2023-09-10T10:35:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40539.692294 secs 2023-09-10T10:35:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40539.692294 secs 2023-09-10T10:35:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:09 smithi098 ceph-mon[103124]: pgmap v21778: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 920 KiB/s wr, 29 op/s 2023-09-10T10:35:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:09 smithi098 ceph-mon[103124]: osdmap e1429: 12 total, 12 up, 12 in 2023-09-10T10:35:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40540.921008 secs 2023-09-10T10:35:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:09 smithi170 ceph-mon[102832]: pgmap v21778: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 920 KiB/s wr, 29 op/s 2023-09-10T10:35:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:09 smithi170 ceph-mon[102832]: osdmap e1429: 12 total, 12 up, 12 in 2023-09-10T10:35:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40540.921008 secs 2023-09-10T10:35:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:09 smithi082 ceph-mon[98694]: pgmap v21778: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 920 KiB/s wr, 29 op/s 2023-09-10T10:35:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:09 smithi082 ceph-mon[98694]: osdmap e1429: 12 total, 12 up, 12 in 2023-09-10T10:35:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40540.921008 secs 2023-09-10T10:35:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40541.272619 secs 2023-09-10T10:35:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:35:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40541.272619 secs 2023-09-10T10:35:10.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:35:10.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40541.272619 secs 2023-09-10T10:35:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:35:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:12 smithi170 ceph-mon[102832]: pgmap v21780: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 617 KiB/s wr, 32 op/s 2023-09-10T10:35:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:12 smithi082 ceph-mon[98694]: pgmap v21780: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 617 KiB/s wr, 32 op/s 2023-09-10T10:35:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:12 smithi098 ceph-mon[103124]: pgmap v21780: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 617 KiB/s wr, 32 op/s 2023-09-10T10:35:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:35:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:35:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:35:13.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:14 smithi170 ceph-mon[102832]: pgmap v21781: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 617 KiB/s wr, 32 op/s 2023-09-10T10:35:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40544.692402 secs 2023-09-10T10:35:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40545.921066 secs 2023-09-10T10:35:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:14 smithi082 ceph-mon[98694]: pgmap v21781: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 617 KiB/s wr, 32 op/s 2023-09-10T10:35:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40544.692402 secs 2023-09-10T10:35:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40545.921066 secs 2023-09-10T10:35:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:14 smithi098 ceph-mon[103124]: pgmap v21781: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 617 KiB/s wr, 32 op/s 2023-09-10T10:35:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40544.692402 secs 2023-09-10T10:35:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40545.921066 secs 2023-09-10T10:35:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40546.272686 secs 2023-09-10T10:35:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:15 smithi170 ceph-mon[102832]: osdmap e1430: 12 total, 12 up, 12 in 2023-09-10T10:35:15.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40546.272686 secs 2023-09-10T10:35:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:15 smithi082 ceph-mon[98694]: osdmap e1430: 12 total, 12 up, 12 in 2023-09-10T10:35:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40546.272686 secs 2023-09-10T10:35:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:15 smithi098 ceph-mon[103124]: osdmap e1430: 12 total, 12 up, 12 in 2023-09-10T10:35:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:16 smithi170 ceph-mon[102832]: pgmap v21782: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 617 KiB/s wr, 32 op/s 2023-09-10T10:35:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:16 smithi082 ceph-mon[98694]: pgmap v21782: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 617 KiB/s wr, 32 op/s 2023-09-10T10:35:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:16 smithi098 ceph-mon[103124]: pgmap v21782: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 617 KiB/s wr, 32 op/s 2023-09-10T10:35:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:18 smithi170 ceph-mon[102832]: pgmap v21784: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 449 KiB/s wr, 17 op/s 2023-09-10T10:35:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:18 smithi082 ceph-mon[98694]: pgmap v21784: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 449 KiB/s wr, 17 op/s 2023-09-10T10:35:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:18 smithi098 ceph-mon[103124]: pgmap v21784: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 449 KiB/s wr, 17 op/s 2023-09-10T10:35:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40549.692466 secs 2023-09-10T10:35:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40550.921139 secs 2023-09-10T10:35:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40549.692466 secs 2023-09-10T10:35:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40550.921139 secs 2023-09-10T10:35:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40549.692466 secs 2023-09-10T10:35:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40550.921139 secs 2023-09-10T10:35:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:20 smithi170 ceph-mon[102832]: pgmap v21785: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 361 KiB/s wr, 14 op/s 2023-09-10T10:35:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40551.272749 secs 2023-09-10T10:35:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:20 smithi082 ceph-mon[98694]: pgmap v21785: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 361 KiB/s wr, 14 op/s 2023-09-10T10:35:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40551.272749 secs 2023-09-10T10:35:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:20 smithi098 ceph-mon[103124]: pgmap v21785: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 361 KiB/s wr, 14 op/s 2023-09-10T10:35:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40551.272749 secs 2023-09-10T10:35:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:22 smithi170 ceph-mon[102832]: pgmap v21786: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 319 KiB/s wr, 12 op/s 2023-09-10T10:35:22.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:22 smithi082 ceph-mon[98694]: pgmap v21786: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 319 KiB/s wr, 12 op/s 2023-09-10T10:35:22.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:22 smithi098 ceph-mon[103124]: pgmap v21786: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 319 KiB/s wr, 12 op/s 2023-09-10T10:35:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:24 smithi170 ceph-mon[102832]: pgmap v21787: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 319 KiB/s wr, 12 op/s 2023-09-10T10:35:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40554.692537 secs 2023-09-10T10:35:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40555.921222 secs 2023-09-10T10:35:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:24 smithi082 ceph-mon[98694]: pgmap v21787: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 319 KiB/s wr, 12 op/s 2023-09-10T10:35:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40554.692537 secs 2023-09-10T10:35:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40555.921222 secs 2023-09-10T10:35:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:24 smithi098 ceph-mon[103124]: pgmap v21787: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 319 KiB/s wr, 12 op/s 2023-09-10T10:35:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40554.692537 secs 2023-09-10T10:35:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40555.921222 secs 2023-09-10T10:35:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40556.272811 secs 2023-09-10T10:35:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40556.272811 secs 2023-09-10T10:35:25.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40556.272811 secs 2023-09-10T10:35:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:26 smithi170 ceph-mon[102832]: pgmap v21788: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 319 KiB/s wr, 12 op/s 2023-09-10T10:35:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:26 smithi082 ceph-mon[98694]: pgmap v21788: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 319 KiB/s wr, 12 op/s 2023-09-10T10:35:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:26 smithi098 ceph-mon[103124]: pgmap v21788: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 319 KiB/s wr, 12 op/s 2023-09-10T10:35:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:28 smithi170 ceph-mon[102832]: pgmap v21789: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 283 KiB/s wr, 16 op/s 2023-09-10T10:35:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:28 smithi082 ceph-mon[98694]: pgmap v21789: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 283 KiB/s wr, 16 op/s 2023-09-10T10:35:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:28 smithi098 ceph-mon[103124]: pgmap v21789: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 283 KiB/s wr, 16 op/s 2023-09-10T10:35:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40559.692594 secs 2023-09-10T10:35:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40560.921279 secs 2023-09-10T10:35:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40559.692594 secs 2023-09-10T10:35:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40560.921279 secs 2023-09-10T10:35:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40559.692594 secs 2023-09-10T10:35:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40560.921279 secs 2023-09-10T10:35:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:30 smithi170 ceph-mon[102832]: pgmap v21790: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 9 op/s 2023-09-10T10:35:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40561.272877 secs 2023-09-10T10:35:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:30 smithi082 ceph-mon[98694]: pgmap v21790: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 9 op/s 2023-09-10T10:35:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40561.272877 secs 2023-09-10T10:35:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:30 smithi098 ceph-mon[103124]: pgmap v21790: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 9 op/s 2023-09-10T10:35:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40561.272877 secs 2023-09-10T10:35:32.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:32 smithi170 ceph-mon[102832]: pgmap v21791: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 296 KiB/s wr, 15 op/s 2023-09-10T10:35:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:32 smithi082 ceph-mon[98694]: pgmap v21791: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 296 KiB/s wr, 15 op/s 2023-09-10T10:35:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:32 smithi098 ceph-mon[103124]: pgmap v21791: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 296 KiB/s wr, 15 op/s 2023-09-10T10:35:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:34 smithi170 ceph-mon[102832]: pgmap v21792: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 10 op/s 2023-09-10T10:35:34.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40564.692668 secs 2023-09-10T10:35:34.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40565.921362 secs 2023-09-10T10:35:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:34 smithi082 ceph-mon[98694]: pgmap v21792: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 10 op/s 2023-09-10T10:35:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40564.692668 secs 2023-09-10T10:35:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40565.921362 secs 2023-09-10T10:35:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:34 smithi098 ceph-mon[103124]: pgmap v21792: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 10 op/s 2023-09-10T10:35:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40564.692668 secs 2023-09-10T10:35:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40565.921362 secs 2023-09-10T10:35:35.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40566.272939 secs 2023-09-10T10:35:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40566.272939 secs 2023-09-10T10:35:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40566.272939 secs 2023-09-10T10:35:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:36 smithi082 ceph-mon[98694]: pgmap v21793: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 10 op/s 2023-09-10T10:35:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:36 smithi098 ceph-mon[103124]: pgmap v21793: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 10 op/s 2023-09-10T10:35:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:36 smithi170 ceph-mon[102832]: pgmap v21793: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 10 op/s 2023-09-10T10:35:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:38 smithi082 ceph-mon[98694]: pgmap v21794: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 150 KiB/s wr, 16 op/s 2023-09-10T10:35:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:35:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:38 smithi098 ceph-mon[103124]: pgmap v21794: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 150 KiB/s wr, 16 op/s 2023-09-10T10:35:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:35:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:38 smithi170 ceph-mon[102832]: pgmap v21794: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 150 KiB/s wr, 16 op/s 2023-09-10T10:35:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:35:38.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:39.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40570.921445 secs 2023-09-10T10:35:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40570.921445 secs 2023-09-10T10:35:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40570.921445 secs 2023-09-10T10:35:40.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:40 smithi082 ceph-mon[98694]: pgmap v21795: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T10:35:40.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40571.273007 secs 2023-09-10T10:35:40.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:40 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:35:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:40 smithi098 ceph-mon[103124]: pgmap v21795: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T10:35:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40571.273007 secs 2023-09-10T10:35:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:40 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:35:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:40 smithi170 ceph-mon[102832]: pgmap v21795: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T10:35:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40571.273007 secs 2023-09-10T10:35:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:40 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:35:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:41 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:35:41.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:41 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:35:41.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:41 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:41 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:35:41.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:41 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:35:41.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:41 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:41 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:35:41.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:41 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:35:41.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:41 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:42.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:42 smithi082 ceph-mon[98694]: pgmap v21796: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 478 KiB/s wr, 17 op/s 2023-09-10T10:35:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:42 smithi098 ceph-mon[103124]: pgmap v21796: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 478 KiB/s wr, 17 op/s 2023-09-10T10:35:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:42 smithi170 ceph-mon[102832]: pgmap v21796: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 478 KiB/s wr, 17 op/s 2023-09-10T10:35:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:44 smithi082 ceph-mon[98694]: pgmap v21797: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 329 KiB/s wr, 12 op/s 2023-09-10T10:35:44.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40575.921495 secs 2023-09-10T10:35:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:44 smithi098 ceph-mon[103124]: pgmap v21797: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 329 KiB/s wr, 12 op/s 2023-09-10T10:35:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40575.921495 secs 2023-09-10T10:35:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:44 smithi170 ceph-mon[102832]: pgmap v21797: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 329 KiB/s wr, 12 op/s 2023-09-10T10:35:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40575.921495 secs 2023-09-10T10:35:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40576.273071 secs 2023-09-10T10:35:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40576.273071 secs 2023-09-10T10:35:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40576.273071 secs 2023-09-10T10:35:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:46 smithi098 ceph-mon[103124]: pgmap v21798: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 329 KiB/s wr, 12 op/s 2023-09-10T10:35:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:35:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:46 smithi170 ceph-mon[102832]: pgmap v21798: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 329 KiB/s wr, 12 op/s 2023-09-10T10:35:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:35:46.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:46 smithi082 ceph-mon[98694]: pgmap v21798: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 329 KiB/s wr, 12 op/s 2023-09-10T10:35:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:35:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:48 smithi098 ceph-mon[103124]: pgmap v21799: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 650 KiB/s wr, 18 op/s 2023-09-10T10:35:48.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:48 smithi170 ceph-mon[102832]: pgmap v21799: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 650 KiB/s wr, 18 op/s 2023-09-10T10:35:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:48 smithi082 ceph-mon[98694]: pgmap v21799: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 650 KiB/s wr, 18 op/s 2023-09-10T10:35:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40580.921580 secs 2023-09-10T10:35:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40580.921580 secs 2023-09-10T10:35:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40580.921580 secs 2023-09-10T10:35:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:50 smithi098 ceph-mon[103124]: pgmap v21800: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 650 KiB/s wr, 13 op/s 2023-09-10T10:35:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40581.273138 secs 2023-09-10T10:35:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:35:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:50 smithi170 ceph-mon[102832]: pgmap v21800: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 650 KiB/s wr, 13 op/s 2023-09-10T10:35:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40581.273138 secs 2023-09-10T10:35:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:35:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:50 smithi082 ceph-mon[98694]: pgmap v21800: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 650 KiB/s wr, 13 op/s 2023-09-10T10:35:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40581.273138 secs 2023-09-10T10:35:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:35:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:52 smithi098 ceph-mon[103124]: pgmap v21801: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 976 KiB/s wr, 20 op/s 2023-09-10T10:35:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:35:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:52 smithi170 ceph-mon[102832]: pgmap v21801: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 976 KiB/s wr, 20 op/s 2023-09-10T10:35:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:35:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:52 smithi082 ceph-mon[98694]: pgmap v21801: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 976 KiB/s wr, 20 op/s 2023-09-10T10:35:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:35:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:54 smithi098 ceph-mon[103124]: pgmap v21802: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 648 KiB/s wr, 14 op/s 2023-09-10T10:35:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40585.921640 secs 2023-09-10T10:35:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:54 smithi170 ceph-mon[102832]: pgmap v21802: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 648 KiB/s wr, 14 op/s 2023-09-10T10:35:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40585.921640 secs 2023-09-10T10:35:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:54 smithi082 ceph-mon[98694]: pgmap v21802: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 648 KiB/s wr, 14 op/s 2023-09-10T10:35:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40585.921640 secs 2023-09-10T10:35:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40586.273201 secs 2023-09-10T10:35:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40586.273201 secs 2023-09-10T10:35:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40586.273201 secs 2023-09-10T10:35:56.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:56 smithi098 ceph-mon[103124]: pgmap v21803: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 648 KiB/s wr, 14 op/s 2023-09-10T10:35:56.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:56 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:35:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:56 smithi170 ceph-mon[102832]: pgmap v21803: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 648 KiB/s wr, 14 op/s 2023-09-10T10:35:56.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:56 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:35:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:56 smithi082 ceph-mon[98694]: pgmap v21803: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 648 KiB/s wr, 14 op/s 2023-09-10T10:35:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:56 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:35:57.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:35:57.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:35:57.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:35:57.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:35:57.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:35:57.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:35:57.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:35:57.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:35:57.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:35:58.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:58 smithi098 ceph-mon[103124]: pgmap v21804: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 876 KiB/s wr, 21 op/s 2023-09-10T10:35:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:35:58.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:58 smithi170 ceph-mon[102832]: pgmap v21804: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 876 KiB/s wr, 21 op/s 2023-09-10T10:35:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:35:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:58 smithi082 ceph-mon[98694]: pgmap v21804: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 876 KiB/s wr, 21 op/s 2023-09-10T10:35:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:35:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:35:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:35:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40590.921690 secs 2023-09-10T10:35:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:35:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40590.921690 secs 2023-09-10T10:35:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:35:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40590.921690 secs 2023-09-10T10:36:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:00 smithi170 ceph-mon[102832]: pgmap v21805: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 555 KiB/s wr, 14 op/s 2023-09-10T10:36:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40591.273263 secs 2023-09-10T10:36:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:36:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:00 smithi082 ceph-mon[98694]: pgmap v21805: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 555 KiB/s wr, 14 op/s 2023-09-10T10:36:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40591.273263 secs 2023-09-10T10:36:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:36:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:00 smithi098 ceph-mon[103124]: pgmap v21805: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 555 KiB/s wr, 14 op/s 2023-09-10T10:36:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40591.273263 secs 2023-09-10T10:36:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:36:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:02.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:02 smithi170 ceph-mon[102832]: pgmap v21806: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 864 KiB/s wr, 36 op/s 2023-09-10T10:36:02.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:02 smithi082 ceph-mon[98694]: pgmap v21806: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 864 KiB/s wr, 36 op/s 2023-09-10T10:36:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:02 smithi098 ceph-mon[103124]: pgmap v21806: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 864 KiB/s wr, 36 op/s 2023-09-10T10:36:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:03 smithi170 ceph-mon[102832]: pgmap v21807: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 537 KiB/s wr, 28 op/s 2023-09-10T10:36:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:03 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:36:03.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:03 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:36:03.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:03 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:03 smithi082 ceph-mon[98694]: pgmap v21807: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 537 KiB/s wr, 28 op/s 2023-09-10T10:36:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:03 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:36:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:03 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:36:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:03 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:03 smithi098 ceph-mon[103124]: pgmap v21807: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 537 KiB/s wr, 28 op/s 2023-09-10T10:36:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:03 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:36:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:03 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:36:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:03 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40594.693276 secs 2023-09-10T10:36:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40595.921773 secs 2023-09-10T10:36:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40594.693276 secs 2023-09-10T10:36:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40595.921773 secs 2023-09-10T10:36:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40594.693276 secs 2023-09-10T10:36:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40595.921773 secs 2023-09-10T10:36:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40596.273327 secs 2023-09-10T10:36:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:05 smithi170 ceph-mon[102832]: pgmap v21808: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 537 KiB/s wr, 28 op/s 2023-09-10T10:36:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40596.273327 secs 2023-09-10T10:36:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:05 smithi082 ceph-mon[98694]: pgmap v21808: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 537 KiB/s wr, 28 op/s 2023-09-10T10:36:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40596.273327 secs 2023-09-10T10:36:05.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:05 smithi098 ceph-mon[103124]: pgmap v21808: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 537 KiB/s wr, 28 op/s 2023-09-10T10:36:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:07 smithi098 ceph-mon[103124]: pgmap v21809: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 687 KiB/s wr, 36 op/s 2023-09-10T10:36:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:07 smithi170 ceph-mon[102832]: pgmap v21809: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 687 KiB/s wr, 36 op/s 2023-09-10T10:36:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:07 smithi082 ceph-mon[98694]: pgmap v21809: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 687 KiB/s wr, 36 op/s 2023-09-10T10:36:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40599.693336 secs 2023-09-10T10:36:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40599.693336 secs 2023-09-10T10:36:08.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40599.693336 secs 2023-09-10T10:36:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:09 smithi098 ceph-mon[103124]: pgmap v21810: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 459 KiB/s wr, 29 op/s 2023-09-10T10:36:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:09 smithi098 ceph-mon[103124]: osdmap e1431: 12 total, 12 up, 12 in 2023-09-10T10:36:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40600.921832 secs 2023-09-10T10:36:09.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:09 smithi170 ceph-mon[102832]: pgmap v21810: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 459 KiB/s wr, 29 op/s 2023-09-10T10:36:09.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:09 smithi170 ceph-mon[102832]: osdmap e1431: 12 total, 12 up, 12 in 2023-09-10T10:36:09.930 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40600.921832 secs 2023-09-10T10:36:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:09 smithi082 ceph-mon[98694]: pgmap v21810: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 459 KiB/s wr, 29 op/s 2023-09-10T10:36:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:09 smithi082 ceph-mon[98694]: osdmap e1431: 12 total, 12 up, 12 in 2023-09-10T10:36:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40600.921832 secs 2023-09-10T10:36:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40601.273407 secs 2023-09-10T10:36:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:36:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40601.273407 secs 2023-09-10T10:36:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:36:10.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40601.273407 secs 2023-09-10T10:36:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:36:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:12 smithi170 ceph-mon[102832]: pgmap v21812: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 432 KiB/s wr, 18 op/s 2023-09-10T10:36:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:12 smithi082 ceph-mon[98694]: pgmap v21812: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 432 KiB/s wr, 18 op/s 2023-09-10T10:36:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:12 smithi098 ceph-mon[103124]: pgmap v21812: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 432 KiB/s wr, 18 op/s 2023-09-10T10:36:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:36:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:36:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:36:13.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:14 smithi170 ceph-mon[102832]: pgmap v21813: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 432 KiB/s wr, 18 op/s 2023-09-10T10:36:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40604.693397 secs 2023-09-10T10:36:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40605.921913 secs 2023-09-10T10:36:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:14 smithi082 ceph-mon[98694]: pgmap v21813: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 432 KiB/s wr, 18 op/s 2023-09-10T10:36:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40604.693397 secs 2023-09-10T10:36:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40605.921913 secs 2023-09-10T10:36:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:14 smithi098 ceph-mon[103124]: pgmap v21813: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 432 KiB/s wr, 18 op/s 2023-09-10T10:36:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40604.693397 secs 2023-09-10T10:36:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40605.921913 secs 2023-09-10T10:36:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40606.273475 secs 2023-09-10T10:36:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:15 smithi170 ceph-mon[102832]: osdmap e1432: 12 total, 12 up, 12 in 2023-09-10T10:36:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40606.273475 secs 2023-09-10T10:36:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:15 smithi082 ceph-mon[98694]: osdmap e1432: 12 total, 12 up, 12 in 2023-09-10T10:36:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40606.273475 secs 2023-09-10T10:36:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:15 smithi098 ceph-mon[103124]: osdmap e1432: 12 total, 12 up, 12 in 2023-09-10T10:36:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:16 smithi170 ceph-mon[102832]: pgmap v21814: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 432 KiB/s wr, 18 op/s 2023-09-10T10:36:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:16 smithi082 ceph-mon[98694]: pgmap v21814: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 432 KiB/s wr, 18 op/s 2023-09-10T10:36:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:16 smithi098 ceph-mon[103124]: pgmap v21814: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 432 KiB/s wr, 18 op/s 2023-09-10T10:36:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:18 smithi170 ceph-mon[102832]: pgmap v21816: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 540 KiB/s wr, 20 op/s 2023-09-10T10:36:18.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:18 smithi082 ceph-mon[98694]: pgmap v21816: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 540 KiB/s wr, 20 op/s 2023-09-10T10:36:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:18 smithi098 ceph-mon[103124]: pgmap v21816: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 540 KiB/s wr, 20 op/s 2023-09-10T10:36:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40609.693468 secs 2023-09-10T10:36:19.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40610.921972 secs 2023-09-10T10:36:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40609.693468 secs 2023-09-10T10:36:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40610.921972 secs 2023-09-10T10:36:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40609.693468 secs 2023-09-10T10:36:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40610.921972 secs 2023-09-10T10:36:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:20 smithi170 ceph-mon[102832]: pgmap v21817: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 435 KiB/s wr, 16 op/s 2023-09-10T10:36:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40611.273490 secs 2023-09-10T10:36:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:20 smithi082 ceph-mon[98694]: pgmap v21817: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 435 KiB/s wr, 16 op/s 2023-09-10T10:36:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40611.273490 secs 2023-09-10T10:36:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:20 smithi098 ceph-mon[103124]: pgmap v21817: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 435 KiB/s wr, 16 op/s 2023-09-10T10:36:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40611.273490 secs 2023-09-10T10:36:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:22 smithi170 ceph-mon[102832]: pgmap v21818: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 181 KiB/s wr, 13 op/s 2023-09-10T10:36:22.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:22 smithi082 ceph-mon[98694]: pgmap v21818: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 181 KiB/s wr, 13 op/s 2023-09-10T10:36:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:22 smithi098 ceph-mon[103124]: pgmap v21818: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 181 KiB/s wr, 13 op/s 2023-09-10T10:36:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:24 smithi170 ceph-mon[102832]: pgmap v21819: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 181 KiB/s wr, 13 op/s 2023-09-10T10:36:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40614.693548 secs 2023-09-10T10:36:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40615.922048 secs 2023-09-10T10:36:24.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:24 smithi082 ceph-mon[98694]: pgmap v21819: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 181 KiB/s wr, 13 op/s 2023-09-10T10:36:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40614.693548 secs 2023-09-10T10:36:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40615.922048 secs 2023-09-10T10:36:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:24 smithi098 ceph-mon[103124]: pgmap v21819: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 181 KiB/s wr, 13 op/s 2023-09-10T10:36:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40614.693548 secs 2023-09-10T10:36:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40615.922048 secs 2023-09-10T10:36:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40616.273555 secs 2023-09-10T10:36:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40616.273555 secs 2023-09-10T10:36:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40616.273555 secs 2023-09-10T10:36:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:26 smithi170 ceph-mon[102832]: pgmap v21820: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 181 KiB/s wr, 13 op/s 2023-09-10T10:36:26.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:26 smithi082 ceph-mon[98694]: pgmap v21820: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 181 KiB/s wr, 13 op/s 2023-09-10T10:36:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:26 smithi098 ceph-mon[103124]: pgmap v21820: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 181 KiB/s wr, 13 op/s 2023-09-10T10:36:28.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:28 smithi170 ceph-mon[102832]: pgmap v21821: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 319 KiB/s wr, 18 op/s 2023-09-10T10:36:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:28 smithi082 ceph-mon[98694]: pgmap v21821: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 319 KiB/s wr, 18 op/s 2023-09-10T10:36:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:28 smithi098 ceph-mon[103124]: pgmap v21821: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 319 KiB/s wr, 18 op/s 2023-09-10T10:36:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40619.693641 secs 2023-09-10T10:36:29.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40620.922104 secs 2023-09-10T10:36:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40619.693641 secs 2023-09-10T10:36:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40620.922104 secs 2023-09-10T10:36:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40619.693641 secs 2023-09-10T10:36:29.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40620.922104 secs 2023-09-10T10:36:30.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:30 smithi170 ceph-mon[102832]: pgmap v21822: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 10 op/s 2023-09-10T10:36:30.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40621.273616 secs 2023-09-10T10:36:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:30 smithi082 ceph-mon[98694]: pgmap v21822: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 10 op/s 2023-09-10T10:36:30.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40621.273616 secs 2023-09-10T10:36:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:30 smithi098 ceph-mon[103124]: pgmap v21822: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 10 op/s 2023-09-10T10:36:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40621.273616 secs 2023-09-10T10:36:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:32 smithi082 ceph-mon[98694]: pgmap v21823: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 16 op/s 2023-09-10T10:36:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:32 smithi098 ceph-mon[103124]: pgmap v21823: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 16 op/s 2023-09-10T10:36:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:32 smithi170 ceph-mon[102832]: pgmap v21823: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 151 KiB/s wr, 16 op/s 2023-09-10T10:36:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:34 smithi082 ceph-mon[98694]: pgmap v21824: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T10:36:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40624.693700 secs 2023-09-10T10:36:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:36:34.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:34.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40625.922165 secs 2023-09-10T10:36:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:34 smithi098 ceph-mon[103124]: pgmap v21824: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T10:36:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40624.693700 secs 2023-09-10T10:36:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:36:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40625.922165 secs 2023-09-10T10:36:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:34 smithi170 ceph-mon[102832]: pgmap v21824: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T10:36:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40624.693700 secs 2023-09-10T10:36:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:36:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40625.922165 secs 2023-09-10T10:36:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40626.273681 secs 2023-09-10T10:36:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40626.273681 secs 2023-09-10T10:36:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40626.273681 secs 2023-09-10T10:36:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:36 smithi082 ceph-mon[98694]: pgmap v21825: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T10:36:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:36 smithi098 ceph-mon[103124]: pgmap v21825: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T10:36:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:36 smithi170 ceph-mon[102832]: pgmap v21825: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 11 op/s 2023-09-10T10:36:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:37 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:36:37.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:37 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:37.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:37 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:36:37.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:37 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:37.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:37 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:36:37.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:37 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:38.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:38 smithi082 ceph-mon[98694]: pgmap v21826: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 297 KiB/s wr, 16 op/s 2023-09-10T10:36:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:38 smithi098 ceph-mon[103124]: pgmap v21826: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 297 KiB/s wr, 16 op/s 2023-09-10T10:36:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:38 smithi170 ceph-mon[102832]: pgmap v21826: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 297 KiB/s wr, 16 op/s 2023-09-10T10:36:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40630.922224 secs 2023-09-10T10:36:39.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:36:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40630.922224 secs 2023-09-10T10:36:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:36:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40630.922224 secs 2023-09-10T10:36:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:36:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:40 smithi098 ceph-mon[103124]: pgmap v21827: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 10 op/s 2023-09-10T10:36:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40631.273742 secs 2023-09-10T10:36:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:36:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:36:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:40 smithi170 ceph-mon[102832]: pgmap v21827: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 10 op/s 2023-09-10T10:36:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40631.273742 secs 2023-09-10T10:36:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:36:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:36:40.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:40 smithi082 ceph-mon[98694]: pgmap v21827: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 147 KiB/s wr, 10 op/s 2023-09-10T10:36:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40631.273742 secs 2023-09-10T10:36:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:36:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:36:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:42 smithi098 ceph-mon[103124]: pgmap v21828: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 477 KiB/s wr, 17 op/s 2023-09-10T10:36:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:42 smithi170 ceph-mon[102832]: pgmap v21828: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 477 KiB/s wr, 17 op/s 2023-09-10T10:36:42.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:42 smithi082 ceph-mon[98694]: pgmap v21828: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 477 KiB/s wr, 17 op/s 2023-09-10T10:36:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:44 smithi098 ceph-mon[103124]: pgmap v21829: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 476 KiB/s wr, 12 op/s 2023-09-10T10:36:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40635.922283 secs 2023-09-10T10:36:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:44 smithi170 ceph-mon[102832]: pgmap v21829: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 476 KiB/s wr, 12 op/s 2023-09-10T10:36:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40635.922283 secs 2023-09-10T10:36:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:44 smithi082 ceph-mon[98694]: pgmap v21829: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 476 KiB/s wr, 12 op/s 2023-09-10T10:36:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40635.922283 secs 2023-09-10T10:36:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40636.273804 secs 2023-09-10T10:36:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40636.273804 secs 2023-09-10T10:36:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40636.273804 secs 2023-09-10T10:36:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:46 smithi098 ceph-mon[103124]: pgmap v21830: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 476 KiB/s wr, 12 op/s 2023-09-10T10:36:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:36:46.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:46 smithi170 ceph-mon[102832]: pgmap v21830: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 476 KiB/s wr, 12 op/s 2023-09-10T10:36:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:36:46.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:46 smithi082 ceph-mon[98694]: pgmap v21830: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 476 KiB/s wr, 12 op/s 2023-09-10T10:36:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:36:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:48 smithi098 ceph-mon[103124]: pgmap v21831: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 798 KiB/s wr, 19 op/s 2023-09-10T10:36:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:48 smithi170 ceph-mon[102832]: pgmap v21831: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 798 KiB/s wr, 19 op/s 2023-09-10T10:36:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:48 smithi082 ceph-mon[98694]: pgmap v21831: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 798 KiB/s wr, 19 op/s 2023-09-10T10:36:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40640.922356 secs 2023-09-10T10:36:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40640.922356 secs 2023-09-10T10:36:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40640.922356 secs 2023-09-10T10:36:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:50 smithi098 ceph-mon[103124]: pgmap v21832: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 652 KiB/s wr, 13 op/s 2023-09-10T10:36:50.514 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40641.273867 secs 2023-09-10T10:36:50.515 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:36:50.515 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:50 smithi170 ceph-mon[102832]: pgmap v21832: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 652 KiB/s wr, 13 op/s 2023-09-10T10:36:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40641.273867 secs 2023-09-10T10:36:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:36:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:50 smithi082 ceph-mon[98694]: pgmap v21832: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 652 KiB/s wr, 13 op/s 2023-09-10T10:36:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40641.273867 secs 2023-09-10T10:36:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:36:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:52 smithi098 ceph-mon[103124]: pgmap v21833: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 966 KiB/s wr, 20 op/s 2023-09-10T10:36:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:36:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:52 smithi170 ceph-mon[102832]: pgmap v21833: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 966 KiB/s wr, 20 op/s 2023-09-10T10:36:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:36:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:52 smithi082 ceph-mon[98694]: pgmap v21833: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 966 KiB/s wr, 20 op/s 2023-09-10T10:36:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:36:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:54 smithi098 ceph-mon[103124]: pgmap v21834: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 636 KiB/s wr, 13 op/s 2023-09-10T10:36:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40645.922419 secs 2023-09-10T10:36:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:54 smithi170 ceph-mon[102832]: pgmap v21834: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 636 KiB/s wr, 13 op/s 2023-09-10T10:36:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40645.922419 secs 2023-09-10T10:36:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:54 smithi082 ceph-mon[98694]: pgmap v21834: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 636 KiB/s wr, 13 op/s 2023-09-10T10:36:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40645.922419 secs 2023-09-10T10:36:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40646.273931 secs 2023-09-10T10:36:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40646.273931 secs 2023-09-10T10:36:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40646.273931 secs 2023-09-10T10:36:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:56 smithi170 ceph-mon[102832]: pgmap v21835: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 636 KiB/s wr, 13 op/s 2023-09-10T10:36:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:56 smithi082 ceph-mon[98694]: pgmap v21835: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 636 KiB/s wr, 13 op/s 2023-09-10T10:36:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:56 smithi098 ceph-mon[103124]: pgmap v21835: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 636 KiB/s wr, 13 op/s 2023-09-10T10:36:57.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:57 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:36:57.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:57 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:36:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:57 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:36:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:58 smithi170 ceph-mon[102832]: pgmap v21836: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 905 KiB/s wr, 20 op/s 2023-09-10T10:36:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:36:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:36:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:36:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:36:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:58 smithi082 ceph-mon[98694]: pgmap v21836: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 905 KiB/s wr, 20 op/s 2023-09-10T10:36:58.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:36:58.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:58.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:36:58.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:36:58.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:36:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:58 smithi098 ceph-mon[103124]: pgmap v21836: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 905 KiB/s wr, 20 op/s 2023-09-10T10:36:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:36:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:36:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:36:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:36:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:36:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:36:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40650.922506 secs 2023-09-10T10:36:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:36:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40650.922506 secs 2023-09-10T10:36:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:36:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40650.922506 secs 2023-09-10T10:37:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:00 smithi170 ceph-mon[102832]: pgmap v21837: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 583 KiB/s wr, 13 op/s 2023-09-10T10:37:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40651.273995 secs 2023-09-10T10:37:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:37:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:00 smithi082 ceph-mon[98694]: pgmap v21837: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 583 KiB/s wr, 13 op/s 2023-09-10T10:37:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40651.273995 secs 2023-09-10T10:37:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:37:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:00 smithi098 ceph-mon[103124]: pgmap v21837: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 583 KiB/s wr, 13 op/s 2023-09-10T10:37:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40651.273995 secs 2023-09-10T10:37:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:37:00.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:02.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:02 smithi170 ceph-mon[102832]: pgmap v21838: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 871 KiB/s wr, 23 op/s 2023-09-10T10:37:02.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:02 smithi082 ceph-mon[98694]: pgmap v21838: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 871 KiB/s wr, 23 op/s 2023-09-10T10:37:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:02 smithi098 ceph-mon[103124]: pgmap v21838: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 871 KiB/s wr, 23 op/s 2023-09-10T10:37:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:03 smithi170 ceph-mon[102832]: pgmap v21839: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 557 KiB/s wr, 16 op/s 2023-09-10T10:37:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:03 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:37:03.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:03 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:37:03.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:03 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:03 smithi082 ceph-mon[98694]: pgmap v21839: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 557 KiB/s wr, 16 op/s 2023-09-10T10:37:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:03 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:37:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:03 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:37:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:03 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:03 smithi098 ceph-mon[103124]: pgmap v21839: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 557 KiB/s wr, 16 op/s 2023-09-10T10:37:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:03 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:37:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:03 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:37:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:03 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40654.694192 secs 2023-09-10T10:37:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40655.922589 secs 2023-09-10T10:37:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40654.694192 secs 2023-09-10T10:37:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40655.922589 secs 2023-09-10T10:37:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40654.694192 secs 2023-09-10T10:37:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40655.922589 secs 2023-09-10T10:37:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40656.274058 secs 2023-09-10T10:37:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:05 smithi170 ceph-mon[102832]: pgmap v21840: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 557 KiB/s wr, 16 op/s 2023-09-10T10:37:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40656.274058 secs 2023-09-10T10:37:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:05 smithi082 ceph-mon[98694]: pgmap v21840: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 557 KiB/s wr, 16 op/s 2023-09-10T10:37:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40656.274058 secs 2023-09-10T10:37:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:05 smithi098 ceph-mon[103124]: pgmap v21840: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 557 KiB/s wr, 16 op/s 2023-09-10T10:37:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:07 smithi098 ceph-mon[103124]: pgmap v21841: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 730 KiB/s wr, 34 op/s 2023-09-10T10:37:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:07 smithi170 ceph-mon[102832]: pgmap v21841: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 730 KiB/s wr, 34 op/s 2023-09-10T10:37:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:07 smithi082 ceph-mon[98694]: pgmap v21841: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 730 KiB/s wr, 34 op/s 2023-09-10T10:37:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40659.694249 secs 2023-09-10T10:37:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40659.694249 secs 2023-09-10T10:37:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40659.694249 secs 2023-09-10T10:37:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:09 smithi098 ceph-mon[103124]: pgmap v21842: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 461 KiB/s wr, 28 op/s 2023-09-10T10:37:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:09 smithi098 ceph-mon[103124]: osdmap e1433: 12 total, 12 up, 12 in 2023-09-10T10:37:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40660.922649 secs 2023-09-10T10:37:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:09 smithi170 ceph-mon[102832]: pgmap v21842: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 461 KiB/s wr, 28 op/s 2023-09-10T10:37:09.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:09 smithi170 ceph-mon[102832]: osdmap e1433: 12 total, 12 up, 12 in 2023-09-10T10:37:09.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40660.922649 secs 2023-09-10T10:37:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:09 smithi082 ceph-mon[98694]: pgmap v21842: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 461 KiB/s wr, 28 op/s 2023-09-10T10:37:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:09 smithi082 ceph-mon[98694]: osdmap e1433: 12 total, 12 up, 12 in 2023-09-10T10:37:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40660.922649 secs 2023-09-10T10:37:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40661.274121 secs 2023-09-10T10:37:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:37:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40661.274121 secs 2023-09-10T10:37:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:37:10.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40661.274121 secs 2023-09-10T10:37:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:37:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:12 smithi170 ceph-mon[102832]: pgmap v21844: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 424 KiB/s wr, 28 op/s 2023-09-10T10:37:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:12 smithi082 ceph-mon[98694]: pgmap v21844: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 424 KiB/s wr, 28 op/s 2023-09-10T10:37:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:12 smithi098 ceph-mon[103124]: pgmap v21844: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 424 KiB/s wr, 28 op/s 2023-09-10T10:37:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:37:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:37:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:37:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:14 smithi170 ceph-mon[102832]: pgmap v21845: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 424 KiB/s wr, 28 op/s 2023-09-10T10:37:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40664.694310 secs 2023-09-10T10:37:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40665.922707 secs 2023-09-10T10:37:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:14 smithi082 ceph-mon[98694]: pgmap v21845: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 424 KiB/s wr, 28 op/s 2023-09-10T10:37:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40664.694310 secs 2023-09-10T10:37:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40665.922707 secs 2023-09-10T10:37:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:14 smithi098 ceph-mon[103124]: pgmap v21845: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 424 KiB/s wr, 28 op/s 2023-09-10T10:37:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40664.694310 secs 2023-09-10T10:37:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40665.922707 secs 2023-09-10T10:37:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40666.274183 secs 2023-09-10T10:37:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:15 smithi170 ceph-mon[102832]: osdmap e1434: 12 total, 12 up, 12 in 2023-09-10T10:37:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40666.274183 secs 2023-09-10T10:37:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:15 smithi082 ceph-mon[98694]: osdmap e1434: 12 total, 12 up, 12 in 2023-09-10T10:37:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40666.274183 secs 2023-09-10T10:37:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:15 smithi098 ceph-mon[103124]: osdmap e1434: 12 total, 12 up, 12 in 2023-09-10T10:37:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:16 smithi170 ceph-mon[102832]: pgmap v21846: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 424 KiB/s wr, 28 op/s 2023-09-10T10:37:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:16 smithi082 ceph-mon[98694]: pgmap v21846: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 424 KiB/s wr, 28 op/s 2023-09-10T10:37:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:16 smithi098 ceph-mon[103124]: pgmap v21846: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 424 KiB/s wr, 28 op/s 2023-09-10T10:37:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:18 smithi170 ceph-mon[102832]: pgmap v21848: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 451 KiB/s wr, 16 op/s 2023-09-10T10:37:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:18 smithi082 ceph-mon[98694]: pgmap v21848: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 451 KiB/s wr, 16 op/s 2023-09-10T10:37:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:18 smithi098 ceph-mon[103124]: pgmap v21848: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 451 KiB/s wr, 16 op/s 2023-09-10T10:37:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40669.694372 secs 2023-09-10T10:37:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40670.922770 secs 2023-09-10T10:37:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40669.694372 secs 2023-09-10T10:37:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40670.922770 secs 2023-09-10T10:37:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40669.694372 secs 2023-09-10T10:37:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40670.922770 secs 2023-09-10T10:37:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:20 smithi170 ceph-mon[102832]: pgmap v21849: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 363 KiB/s wr, 13 op/s 2023-09-10T10:37:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40671.274245 secs 2023-09-10T10:37:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:20 smithi082 ceph-mon[98694]: pgmap v21849: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 363 KiB/s wr, 13 op/s 2023-09-10T10:37:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40671.274245 secs 2023-09-10T10:37:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:20 smithi098 ceph-mon[103124]: pgmap v21849: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 363 KiB/s wr, 13 op/s 2023-09-10T10:37:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40671.274245 secs 2023-09-10T10:37:22.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:22 smithi170 ceph-mon[102832]: pgmap v21850: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 13 op/s 2023-09-10T10:37:22.457 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:22 smithi082 ceph-mon[98694]: pgmap v21850: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 13 op/s 2023-09-10T10:37:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:22 smithi098 ceph-mon[103124]: pgmap v21850: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 13 op/s 2023-09-10T10:37:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:24 smithi170 ceph-mon[102832]: pgmap v21851: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 13 op/s 2023-09-10T10:37:24.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40674.694426 secs 2023-09-10T10:37:24.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40675.922827 secs 2023-09-10T10:37:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:24 smithi082 ceph-mon[98694]: pgmap v21851: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 13 op/s 2023-09-10T10:37:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40674.694426 secs 2023-09-10T10:37:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40675.922827 secs 2023-09-10T10:37:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:24 smithi098 ceph-mon[103124]: pgmap v21851: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 13 op/s 2023-09-10T10:37:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40674.694426 secs 2023-09-10T10:37:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40675.922827 secs 2023-09-10T10:37:25.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40676.274308 secs 2023-09-10T10:37:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40676.274308 secs 2023-09-10T10:37:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40676.274308 secs 2023-09-10T10:37:26.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:26 smithi170 ceph-mon[102832]: pgmap v21852: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 13 op/s 2023-09-10T10:37:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:26 smithi082 ceph-mon[98694]: pgmap v21852: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 13 op/s 2023-09-10T10:37:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:26 smithi098 ceph-mon[103124]: pgmap v21852: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 13 op/s 2023-09-10T10:37:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:28 smithi082 ceph-mon[98694]: pgmap v21853: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 288 KiB/s wr, 18 op/s 2023-09-10T10:37:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:28 smithi098 ceph-mon[103124]: pgmap v21853: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 288 KiB/s wr, 18 op/s 2023-09-10T10:37:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:28 smithi170 ceph-mon[102832]: pgmap v21853: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 288 KiB/s wr, 18 op/s 2023-09-10T10:37:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40679.694486 secs 2023-09-10T10:37:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40680.922889 secs 2023-09-10T10:37:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40679.694486 secs 2023-09-10T10:37:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40680.922889 secs 2023-09-10T10:37:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40679.694486 secs 2023-09-10T10:37:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40680.922889 secs 2023-09-10T10:37:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:30 smithi082 ceph-mon[98694]: pgmap v21854: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:37:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40681.274370 secs 2023-09-10T10:37:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:30 smithi098 ceph-mon[103124]: pgmap v21854: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:37:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40681.274370 secs 2023-09-10T10:37:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:30 smithi170 ceph-mon[102832]: pgmap v21854: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:37:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40681.274370 secs 2023-09-10T10:37:32.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:32 smithi082 ceph-mon[98694]: pgmap v21855: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 151 KiB/s wr, 16 op/s 2023-09-10T10:37:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:32 smithi098 ceph-mon[103124]: pgmap v21855: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 151 KiB/s wr, 16 op/s 2023-09-10T10:37:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:32 smithi170 ceph-mon[102832]: pgmap v21855: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 151 KiB/s wr, 16 op/s 2023-09-10T10:37:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:34 smithi082 ceph-mon[98694]: pgmap v21856: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:37:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40684.694559 secs 2023-09-10T10:37:34.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:37:34.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:34.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40685.922946 secs 2023-09-10T10:37:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:34 smithi098 ceph-mon[103124]: pgmap v21856: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:37:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40684.694559 secs 2023-09-10T10:37:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:37:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40685.922946 secs 2023-09-10T10:37:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:34 smithi170 ceph-mon[102832]: pgmap v21856: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:37:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40684.694559 secs 2023-09-10T10:37:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:37:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40685.922946 secs 2023-09-10T10:37:35.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40686.274437 secs 2023-09-10T10:37:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40686.274437 secs 2023-09-10T10:37:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40686.274437 secs 2023-09-10T10:37:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:36 smithi082 ceph-mon[98694]: pgmap v21857: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:37:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:37:36.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:36 smithi098 ceph-mon[103124]: pgmap v21857: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:37:36.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:37:36.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:36 smithi170 ceph-mon[102832]: pgmap v21857: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:37:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:37:36.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:38 smithi098 ceph-mon[103124]: pgmap v21858: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 444 KiB/s wr, 17 op/s 2023-09-10T10:37:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:37:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:38 smithi170 ceph-mon[102832]: pgmap v21858: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 444 KiB/s wr, 17 op/s 2023-09-10T10:37:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:37:38.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:38 smithi082 ceph-mon[98694]: pgmap v21858: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 444 KiB/s wr, 17 op/s 2023-09-10T10:37:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:37:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:39 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40689.694644 secs 2023-09-10T10:37:39.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40690.923006 secs 2023-09-10T10:37:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:39 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40689.694644 secs 2023-09-10T10:37:39.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40690.923006 secs 2023-09-10T10:37:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:39 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40689.694644 secs 2023-09-10T10:37:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40690.923006 secs 2023-09-10T10:37:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:40 smithi098 ceph-mon[103124]: pgmap v21859: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T10:37:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40691.274492 secs 2023-09-10T10:37:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:37:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:40 smithi170 ceph-mon[102832]: pgmap v21859: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T10:37:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40691.274492 secs 2023-09-10T10:37:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:37:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:40 smithi082 ceph-mon[98694]: pgmap v21859: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T10:37:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40691.274492 secs 2023-09-10T10:37:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:37:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:42 smithi098 ceph-mon[103124]: pgmap v21860: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 477 KiB/s wr, 17 op/s 2023-09-10T10:37:42.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:42 smithi170 ceph-mon[102832]: pgmap v21860: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 477 KiB/s wr, 17 op/s 2023-09-10T10:37:42.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:42 smithi082 ceph-mon[98694]: pgmap v21860: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 477 KiB/s wr, 17 op/s 2023-09-10T10:37:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:44 smithi098 ceph-mon[103124]: pgmap v21861: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 477 KiB/s wr, 12 op/s 2023-09-10T10:37:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:44 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40694.694796 secs 2023-09-10T10:37:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40695.923064 secs 2023-09-10T10:37:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:44 smithi170 ceph-mon[102832]: pgmap v21861: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 477 KiB/s wr, 12 op/s 2023-09-10T10:37:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:44 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40694.694796 secs 2023-09-10T10:37:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40695.923064 secs 2023-09-10T10:37:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:44 smithi082 ceph-mon[98694]: pgmap v21861: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 477 KiB/s wr, 12 op/s 2023-09-10T10:37:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:44 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40694.694796 secs 2023-09-10T10:37:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40695.923064 secs 2023-09-10T10:37:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40696.274556 secs 2023-09-10T10:37:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40696.274556 secs 2023-09-10T10:37:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40696.274556 secs 2023-09-10T10:37:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:46 smithi098 ceph-mon[103124]: pgmap v21862: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 477 KiB/s wr, 12 op/s 2023-09-10T10:37:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:46 smithi170 ceph-mon[102832]: pgmap v21862: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 477 KiB/s wr, 12 op/s 2023-09-10T10:37:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:46 smithi082 ceph-mon[98694]: pgmap v21862: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 477 KiB/s wr, 12 op/s 2023-09-10T10:37:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:48 smithi098 ceph-mon[103124]: pgmap v21863: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 799 KiB/s wr, 19 op/s 2023-09-10T10:37:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:48 smithi170 ceph-mon[102832]: pgmap v21863: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 799 KiB/s wr, 19 op/s 2023-09-10T10:37:48.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:48 smithi082 ceph-mon[98694]: pgmap v21863: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 799 KiB/s wr, 19 op/s 2023-09-10T10:37:49.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40700.923140 secs 2023-09-10T10:37:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40700.923140 secs 2023-09-10T10:37:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40700.923140 secs 2023-09-10T10:37:50.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:50 smithi098 ceph-mon[103124]: pgmap v21864: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 506 KiB/s wr, 13 op/s 2023-09-10T10:37:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40701.274621 secs 2023-09-10T10:37:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:37:50.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:50 smithi170 ceph-mon[102832]: pgmap v21864: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 506 KiB/s wr, 13 op/s 2023-09-10T10:37:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40701.274621 secs 2023-09-10T10:37:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:37:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:50 smithi082 ceph-mon[98694]: pgmap v21864: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 506 KiB/s wr, 13 op/s 2023-09-10T10:37:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40701.274621 secs 2023-09-10T10:37:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:37:50.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:51.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:51 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:37:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:51 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:37:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:51 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:37:52.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:52 smithi098 ceph-mon[103124]: pgmap v21865: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 776 KiB/s wr, 20 op/s 2023-09-10T10:37:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:52 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:37:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:37:52.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:52 smithi170 ceph-mon[102832]: pgmap v21865: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 776 KiB/s wr, 20 op/s 2023-09-10T10:37:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:52 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:37:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:37:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:52 smithi082 ceph-mon[98694]: pgmap v21865: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 776 KiB/s wr, 20 op/s 2023-09-10T10:37:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:52 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:37:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:37:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:54.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:54 smithi098 ceph-mon[103124]: pgmap v21866: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 592 KiB/s wr, 13 op/s 2023-09-10T10:37:54.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40705.923198 secs 2023-09-10T10:37:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:54 smithi170 ceph-mon[102832]: pgmap v21866: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 592 KiB/s wr, 13 op/s 2023-09-10T10:37:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40705.923198 secs 2023-09-10T10:37:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:54 smithi082 ceph-mon[98694]: pgmap v21866: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 592 KiB/s wr, 13 op/s 2023-09-10T10:37:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40705.923198 secs 2023-09-10T10:37:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40706.274684 secs 2023-09-10T10:37:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40706.274684 secs 2023-09-10T10:37:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40706.274684 secs 2023-09-10T10:37:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:56 smithi170 ceph-mon[102832]: pgmap v21867: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 592 KiB/s wr, 13 op/s 2023-09-10T10:37:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:56 smithi082 ceph-mon[98694]: pgmap v21867: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 592 KiB/s wr, 13 op/s 2023-09-10T10:37:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:56 smithi098 ceph-mon[103124]: pgmap v21867: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 592 KiB/s wr, 13 op/s 2023-09-10T10:37:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:58 smithi170 ceph-mon[102832]: pgmap v21868: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 907 KiB/s wr, 20 op/s 2023-09-10T10:37:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:37:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:58 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:37:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:58 smithi082 ceph-mon[98694]: pgmap v21868: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 907 KiB/s wr, 20 op/s 2023-09-10T10:37:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:37:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:58 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:37:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:58 smithi098 ceph-mon[103124]: pgmap v21868: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 907 KiB/s wr, 20 op/s 2023-09-10T10:37:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:37:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:37:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:58 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:37:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:37:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:37:59.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:37:59.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:37:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40710.923283 secs 2023-09-10T10:37:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:37:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:37:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:37:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:37:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40710.923283 secs 2023-09-10T10:37:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:37:59.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:37:59.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:37:59.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:37:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40710.923283 secs 2023-09-10T10:38:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:00 smithi170 ceph-mon[102832]: pgmap v21869: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 584 KiB/s wr, 13 op/s 2023-09-10T10:38:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40711.274746 secs 2023-09-10T10:38:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:38:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:00 smithi082 ceph-mon[98694]: pgmap v21869: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 584 KiB/s wr, 13 op/s 2023-09-10T10:38:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40711.274746 secs 2023-09-10T10:38:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:38:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:00 smithi098 ceph-mon[103124]: pgmap v21869: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 584 KiB/s wr, 13 op/s 2023-09-10T10:38:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40711.274746 secs 2023-09-10T10:38:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:38:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:02.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:02 smithi170 ceph-mon[102832]: pgmap v21870: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 855 KiB/s wr, 21 op/s 2023-09-10T10:38:02.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:02 smithi082 ceph-mon[98694]: pgmap v21870: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 855 KiB/s wr, 21 op/s 2023-09-10T10:38:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:02 smithi098 ceph-mon[103124]: pgmap v21870: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 855 KiB/s wr, 21 op/s 2023-09-10T10:38:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:03 smithi170 ceph-mon[102832]: pgmap v21871: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 586 KiB/s wr, 13 op/s 2023-09-10T10:38:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:03 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:38:03.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:03 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:38:03.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:03 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:03 smithi082 ceph-mon[98694]: pgmap v21871: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 586 KiB/s wr, 13 op/s 2023-09-10T10:38:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:03 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:38:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:03 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:38:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:03 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:03 smithi098 ceph-mon[103124]: pgmap v21871: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 586 KiB/s wr, 13 op/s 2023-09-10T10:38:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:03 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:38:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:03 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:38:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:03 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40714.695061 secs 2023-09-10T10:38:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40715.923362 secs 2023-09-10T10:38:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40714.695061 secs 2023-09-10T10:38:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40715.923362 secs 2023-09-10T10:38:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40714.695061 secs 2023-09-10T10:38:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40715.923362 secs 2023-09-10T10:38:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40716.274807 secs 2023-09-10T10:38:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:05 smithi170 ceph-mon[102832]: pgmap v21872: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 586 KiB/s wr, 13 op/s 2023-09-10T10:38:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40716.274807 secs 2023-09-10T10:38:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:05 smithi082 ceph-mon[98694]: pgmap v21872: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 586 KiB/s wr, 13 op/s 2023-09-10T10:38:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40716.274807 secs 2023-09-10T10:38:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:05 smithi098 ceph-mon[103124]: pgmap v21872: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 586 KiB/s wr, 13 op/s 2023-09-10T10:38:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:07 smithi098 ceph-mon[103124]: pgmap v21873: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 776 KiB/s wr, 34 op/s 2023-09-10T10:38:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:07 smithi170 ceph-mon[102832]: pgmap v21873: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 776 KiB/s wr, 34 op/s 2023-09-10T10:38:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:07 smithi082 ceph-mon[98694]: pgmap v21873: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 776 KiB/s wr, 34 op/s 2023-09-10T10:38:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40719.695145 secs 2023-09-10T10:38:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40719.695145 secs 2023-09-10T10:38:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40719.695145 secs 2023-09-10T10:38:09.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:09 smithi170 ceph-mon[102832]: pgmap v21874: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 461 KiB/s wr, 28 op/s 2023-09-10T10:38:09.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:09 smithi170 ceph-mon[102832]: osdmap e1435: 12 total, 12 up, 12 in 2023-09-10T10:38:09.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40720.923444 secs 2023-09-10T10:38:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:09 smithi082 ceph-mon[98694]: pgmap v21874: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 461 KiB/s wr, 28 op/s 2023-09-10T10:38:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:09 smithi082 ceph-mon[98694]: osdmap e1435: 12 total, 12 up, 12 in 2023-09-10T10:38:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40720.923444 secs 2023-09-10T10:38:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:09 smithi098 ceph-mon[103124]: pgmap v21874: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 461 KiB/s wr, 28 op/s 2023-09-10T10:38:10.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:09 smithi098 ceph-mon[103124]: osdmap e1435: 12 total, 12 up, 12 in 2023-09-10T10:38:10.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40720.923444 secs 2023-09-10T10:38:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40721.274870 secs 2023-09-10T10:38:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:38:10.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40721.274870 secs 2023-09-10T10:38:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:38:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40721.274870 secs 2023-09-10T10:38:11.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:38:11.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:12.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:12 smithi170 ceph-mon[102832]: pgmap v21876: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 481 KiB/s wr, 33 op/s 2023-09-10T10:38:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:12 smithi082 ceph-mon[98694]: pgmap v21876: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 481 KiB/s wr, 33 op/s 2023-09-10T10:38:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:12 smithi098 ceph-mon[103124]: pgmap v21876: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 481 KiB/s wr, 33 op/s 2023-09-10T10:38:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:13 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:38:13.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:13 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:13 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:38:13.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:13 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:13 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:38:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:13 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:14 smithi170 ceph-mon[102832]: pgmap v21877: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 481 KiB/s wr, 33 op/s 2023-09-10T10:38:14.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40724.695217 secs 2023-09-10T10:38:14.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40725.923503 secs 2023-09-10T10:38:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:14 smithi082 ceph-mon[98694]: pgmap v21877: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 481 KiB/s wr, 33 op/s 2023-09-10T10:38:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40724.695217 secs 2023-09-10T10:38:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40725.923503 secs 2023-09-10T10:38:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:14 smithi098 ceph-mon[103124]: pgmap v21877: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 481 KiB/s wr, 33 op/s 2023-09-10T10:38:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40724.695217 secs 2023-09-10T10:38:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40725.923503 secs 2023-09-10T10:38:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40726.274930 secs 2023-09-10T10:38:15.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:15 smithi170 ceph-mon[102832]: osdmap e1436: 12 total, 12 up, 12 in 2023-09-10T10:38:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40726.274930 secs 2023-09-10T10:38:15.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:15 smithi082 ceph-mon[98694]: osdmap e1436: 12 total, 12 up, 12 in 2023-09-10T10:38:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40726.274930 secs 2023-09-10T10:38:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:15 smithi098 ceph-mon[103124]: osdmap e1436: 12 total, 12 up, 12 in 2023-09-10T10:38:16.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:16 smithi170 ceph-mon[102832]: pgmap v21878: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 481 KiB/s wr, 33 op/s 2023-09-10T10:38:16.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:16 smithi082 ceph-mon[98694]: pgmap v21878: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 481 KiB/s wr, 33 op/s 2023-09-10T10:38:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:16 smithi098 ceph-mon[103124]: pgmap v21878: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 481 KiB/s wr, 33 op/s 2023-09-10T10:38:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:18 smithi082 ceph-mon[98694]: pgmap v21880: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 542 KiB/s wr, 20 op/s 2023-09-10T10:38:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:18 smithi098 ceph-mon[103124]: pgmap v21880: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 542 KiB/s wr, 20 op/s 2023-09-10T10:38:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:18 smithi170 ceph-mon[102832]: pgmap v21880: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 542 KiB/s wr, 20 op/s 2023-09-10T10:38:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40729.695275 secs 2023-09-10T10:38:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40730.923586 secs 2023-09-10T10:38:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40729.695275 secs 2023-09-10T10:38:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40730.923586 secs 2023-09-10T10:38:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40729.695275 secs 2023-09-10T10:38:19.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40730.923586 secs 2023-09-10T10:38:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:20 smithi082 ceph-mon[98694]: pgmap v21881: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 436 KiB/s wr, 16 op/s 2023-09-10T10:38:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40731.274992 secs 2023-09-10T10:38:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:20 smithi098 ceph-mon[103124]: pgmap v21881: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 436 KiB/s wr, 16 op/s 2023-09-10T10:38:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40731.274992 secs 2023-09-10T10:38:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:20 smithi170 ceph-mon[102832]: pgmap v21881: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 436 KiB/s wr, 16 op/s 2023-09-10T10:38:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40731.274992 secs 2023-09-10T10:38:22.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:22 smithi082 ceph-mon[98694]: pgmap v21882: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 182 KiB/s wr, 13 op/s 2023-09-10T10:38:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:22 smithi098 ceph-mon[103124]: pgmap v21882: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 182 KiB/s wr, 13 op/s 2023-09-10T10:38:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:22 smithi170 ceph-mon[102832]: pgmap v21882: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 182 KiB/s wr, 13 op/s 2023-09-10T10:38:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:24 smithi082 ceph-mon[98694]: pgmap v21883: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 182 KiB/s wr, 13 op/s 2023-09-10T10:38:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40734.695347 secs 2023-09-10T10:38:24.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40735.923645 secs 2023-09-10T10:38:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:24 smithi098 ceph-mon[103124]: pgmap v21883: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 182 KiB/s wr, 13 op/s 2023-09-10T10:38:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40734.695347 secs 2023-09-10T10:38:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40735.923645 secs 2023-09-10T10:38:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:24 smithi170 ceph-mon[102832]: pgmap v21883: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 182 KiB/s wr, 13 op/s 2023-09-10T10:38:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40734.695347 secs 2023-09-10T10:38:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40735.923645 secs 2023-09-10T10:38:25.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40736.275056 secs 2023-09-10T10:38:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40736.275056 secs 2023-09-10T10:38:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40736.275056 secs 2023-09-10T10:38:26.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:26 smithi082 ceph-mon[98694]: pgmap v21884: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 182 KiB/s wr, 13 op/s 2023-09-10T10:38:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:26 smithi098 ceph-mon[103124]: pgmap v21884: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 182 KiB/s wr, 13 op/s 2023-09-10T10:38:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:26 smithi170 ceph-mon[102832]: pgmap v21884: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 182 KiB/s wr, 13 op/s 2023-09-10T10:38:28.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:28 smithi082 ceph-mon[98694]: pgmap v21885: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 321 KiB/s wr, 18 op/s 2023-09-10T10:38:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:28 smithi098 ceph-mon[103124]: pgmap v21885: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 321 KiB/s wr, 18 op/s 2023-09-10T10:38:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:28 smithi170 ceph-mon[102832]: pgmap v21885: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 321 KiB/s wr, 18 op/s 2023-09-10T10:38:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40739.695407 secs 2023-09-10T10:38:29.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40740.923704 secs 2023-09-10T10:38:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40739.695407 secs 2023-09-10T10:38:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40740.923704 secs 2023-09-10T10:38:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40739.695407 secs 2023-09-10T10:38:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40740.923704 secs 2023-09-10T10:38:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:30 smithi082 ceph-mon[98694]: pgmap v21886: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T10:38:30.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40741.275119 secs 2023-09-10T10:38:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:30 smithi098 ceph-mon[103124]: pgmap v21886: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T10:38:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40741.275119 secs 2023-09-10T10:38:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:30 smithi170 ceph-mon[102832]: pgmap v21886: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T10:38:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40741.275119 secs 2023-09-10T10:38:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:32 smithi098 ceph-mon[103124]: pgmap v21887: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 151 KiB/s wr, 16 op/s 2023-09-10T10:38:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:32 smithi170 ceph-mon[102832]: pgmap v21887: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 151 KiB/s wr, 16 op/s 2023-09-10T10:38:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:32 smithi082 ceph-mon[98694]: pgmap v21887: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 151 KiB/s wr, 16 op/s 2023-09-10T10:38:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:34 smithi098 ceph-mon[103124]: pgmap v21888: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:38:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40744.695469 secs 2023-09-10T10:38:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:38:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40745.923783 secs 2023-09-10T10:38:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:34 smithi170 ceph-mon[102832]: pgmap v21888: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:38:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40744.695469 secs 2023-09-10T10:38:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:38:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40745.923783 secs 2023-09-10T10:38:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:34 smithi082 ceph-mon[98694]: pgmap v21888: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:38:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40744.695469 secs 2023-09-10T10:38:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:38:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:34.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40745.923783 secs 2023-09-10T10:38:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40746.275180 secs 2023-09-10T10:38:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40746.275180 secs 2023-09-10T10:38:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40746.275180 secs 2023-09-10T10:38:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:36 smithi098 ceph-mon[103124]: pgmap v21889: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:38:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:38:36.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:36 smithi170 ceph-mon[102832]: pgmap v21889: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:38:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:38:36.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:36 smithi082 ceph-mon[98694]: pgmap v21889: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:38:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:38:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:38.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:38 smithi098 ceph-mon[103124]: pgmap v21890: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 396 KiB/s wr, 16 op/s 2023-09-10T10:38:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:38:38.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:38 smithi170 ceph-mon[102832]: pgmap v21890: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 396 KiB/s wr, 16 op/s 2023-09-10T10:38:38.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:38:38.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:38.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:38 smithi082 ceph-mon[98694]: pgmap v21890: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 396 KiB/s wr, 16 op/s 2023-09-10T10:38:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:38:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:39 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40749.695545 secs 2023-09-10T10:38:39.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40750.923825 secs 2023-09-10T10:38:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:39 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40749.695545 secs 2023-09-10T10:38:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40750.923825 secs 2023-09-10T10:38:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:39 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40749.695545 secs 2023-09-10T10:38:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40750.923825 secs 2023-09-10T10:38:40.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:40 smithi098 ceph-mon[103124]: pgmap v21891: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 246 KiB/s wr, 11 op/s 2023-09-10T10:38:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40751.275247 secs 2023-09-10T10:38:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:38:40.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:40 smithi170 ceph-mon[102832]: pgmap v21891: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 246 KiB/s wr, 11 op/s 2023-09-10T10:38:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40751.275247 secs 2023-09-10T10:38:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:38:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:40 smithi082 ceph-mon[98694]: pgmap v21891: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 246 KiB/s wr, 11 op/s 2023-09-10T10:38:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40751.275247 secs 2023-09-10T10:38:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:38:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:42.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:42 smithi098 ceph-mon[103124]: pgmap v21892: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 618 KiB/s wr, 17 op/s 2023-09-10T10:38:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:42 smithi170 ceph-mon[102832]: pgmap v21892: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 618 KiB/s wr, 17 op/s 2023-09-10T10:38:42.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:42 smithi082 ceph-mon[98694]: pgmap v21892: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 618 KiB/s wr, 17 op/s 2023-09-10T10:38:44.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:44 smithi098 ceph-mon[103124]: pgmap v21893: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 618 KiB/s wr, 12 op/s 2023-09-10T10:38:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:44 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40754.695612 secs 2023-09-10T10:38:44.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40755.923886 secs 2023-09-10T10:38:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:44 smithi170 ceph-mon[102832]: pgmap v21893: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 618 KiB/s wr, 12 op/s 2023-09-10T10:38:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:44 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40754.695612 secs 2023-09-10T10:38:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40755.923886 secs 2023-09-10T10:38:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:44 smithi082 ceph-mon[98694]: pgmap v21893: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 618 KiB/s wr, 12 op/s 2023-09-10T10:38:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:44 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40754.695612 secs 2023-09-10T10:38:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40755.923886 secs 2023-09-10T10:38:45.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40756.275308 secs 2023-09-10T10:38:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40756.275308 secs 2023-09-10T10:38:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40756.275308 secs 2023-09-10T10:38:46.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:46 smithi098 ceph-mon[103124]: pgmap v21894: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 618 KiB/s wr, 12 op/s 2023-09-10T10:38:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:46 smithi170 ceph-mon[102832]: pgmap v21894: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 618 KiB/s wr, 12 op/s 2023-09-10T10:38:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:46 smithi082 ceph-mon[98694]: pgmap v21894: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 618 KiB/s wr, 12 op/s 2023-09-10T10:38:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:48 smithi098 ceph-mon[103124]: pgmap v21895: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 802 KiB/s wr, 19 op/s 2023-09-10T10:38:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:48 smithi170 ceph-mon[102832]: pgmap v21895: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 802 KiB/s wr, 19 op/s 2023-09-10T10:38:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:48 smithi082 ceph-mon[98694]: pgmap v21895: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 802 KiB/s wr, 19 op/s 2023-09-10T10:38:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40760.923943 secs 2023-09-10T10:38:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40760.923943 secs 2023-09-10T10:38:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40760.923943 secs 2023-09-10T10:38:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:50 smithi170 ceph-mon[102832]: pgmap v21896: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 556 KiB/s wr, 14 op/s 2023-09-10T10:38:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40761.275369 secs 2023-09-10T10:38:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:38:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:50 smithi082 ceph-mon[98694]: pgmap v21896: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 556 KiB/s wr, 14 op/s 2023-09-10T10:38:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40761.275369 secs 2023-09-10T10:38:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:38:50.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:50 smithi098 ceph-mon[103124]: pgmap v21896: 97 pgs: 97 active+clean; 14 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 556 KiB/s wr, 14 op/s 2023-09-10T10:38:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40761.275369 secs 2023-09-10T10:38:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:38:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:51 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:38:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:51 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:38:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:51 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:38:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:52 smithi170 ceph-mon[102832]: pgmap v21897: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 826 KiB/s wr, 21 op/s 2023-09-10T10:38:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:52 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:38:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:38:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:52 smithi082 ceph-mon[98694]: pgmap v21897: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 826 KiB/s wr, 21 op/s 2023-09-10T10:38:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:52 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:38:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:38:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:52 smithi098 ceph-mon[103124]: pgmap v21897: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 826 KiB/s wr, 21 op/s 2023-09-10T10:38:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:52 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:38:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:38:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:54 smithi170 ceph-mon[102832]: pgmap v21898: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 454 KiB/s wr, 14 op/s 2023-09-10T10:38:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40765.924015 secs 2023-09-10T10:38:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:54 smithi082 ceph-mon[98694]: pgmap v21898: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 454 KiB/s wr, 14 op/s 2023-09-10T10:38:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40765.924015 secs 2023-09-10T10:38:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:54 smithi098 ceph-mon[103124]: pgmap v21898: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 454 KiB/s wr, 14 op/s 2023-09-10T10:38:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40765.924015 secs 2023-09-10T10:38:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40766.275430 secs 2023-09-10T10:38:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40766.275430 secs 2023-09-10T10:38:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40766.275430 secs 2023-09-10T10:38:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:56 smithi170 ceph-mon[102832]: pgmap v21899: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 454 KiB/s wr, 14 op/s 2023-09-10T10:38:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:56 smithi082 ceph-mon[98694]: pgmap v21899: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 454 KiB/s wr, 14 op/s 2023-09-10T10:38:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:56 smithi098 ceph-mon[103124]: pgmap v21899: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 454 KiB/s wr, 14 op/s 2023-09-10T10:38:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:58 smithi170 ceph-mon[102832]: pgmap v21900: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 770 KiB/s wr, 20 op/s 2023-09-10T10:38:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:38:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:58 smithi082 ceph-mon[98694]: pgmap v21900: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 770 KiB/s wr, 20 op/s 2023-09-10T10:38:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:38:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:58 smithi098 ceph-mon[103124]: pgmap v21900: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 770 KiB/s wr, 20 op/s 2023-09-10T10:38:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:38:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:38:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:59 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:38:59.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:38:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40770.924077 secs 2023-09-10T10:38:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:59 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:38:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:38:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40770.924077 secs 2023-09-10T10:38:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:59 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:38:59.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:38:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40770.924077 secs 2023-09-10T10:39:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:00 smithi170 ceph-mon[102832]: pgmap v21901: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 586 KiB/s wr, 13 op/s 2023-09-10T10:39:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40771.275485 secs 2023-09-10T10:39:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:39:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:00 smithi082 ceph-mon[98694]: pgmap v21901: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 586 KiB/s wr, 13 op/s 2023-09-10T10:39:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40771.275485 secs 2023-09-10T10:39:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:39:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:00 smithi098 ceph-mon[103124]: pgmap v21901: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 586 KiB/s wr, 13 op/s 2023-09-10T10:39:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40771.275485 secs 2023-09-10T10:39:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:39:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:02 smithi170 ceph-mon[102832]: pgmap v21902: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 857 KiB/s wr, 19 op/s 2023-09-10T10:39:02.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:39:02.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:39:02.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:39:02.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:39:02.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:39:02.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:39:02.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:39:02.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:39:02.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:39:02.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:39:02.430 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:39:02.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:39:02.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:39:02.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:39:02.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:39:02.431 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:39:02.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:39:02.432 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:39:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:02 smithi082 ceph-mon[98694]: pgmap v21902: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 857 KiB/s wr, 19 op/s 2023-09-10T10:39:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:39:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:39:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:39:02.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:39:02.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:39:02.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:39:02.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:39:02.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:39:02.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:39:02.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:39:02.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:39:02.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:39:02.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:39:02.460 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:39:02.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:39:02.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:39:02.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:39:02.461 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:39:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:02 smithi098 ceph-mon[103124]: pgmap v21902: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 857 KiB/s wr, 19 op/s 2023-09-10T10:39:02.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:39:02.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:39:02.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:39:02.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:39:02.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:39:02.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:39:02.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:39:02.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:39:02.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:39:02.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:39:02.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:39:02.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:39:02.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:39:02.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:39:02.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:39:02.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:39:02.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:39:02.508 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:39:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:03 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T10:39:03.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:03 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T10:39:03.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:03 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T10:39:03.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:03 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T10:39:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:03 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T10:39:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:03 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T10:39:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:03 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T10:39:03.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:03 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T10:39:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:03 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T10:39:03.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:03 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T10:39:03.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:03 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T10:39:03.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:03 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T10:39:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:04 smithi170 ceph-mon[102832]: pgmap v21903: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 588 KiB/s wr, 12 op/s 2023-09-10T10:39:04.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40774.695910 secs 2023-09-10T10:39:04.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40775.924137 secs 2023-09-10T10:39:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:04 smithi082 ceph-mon[98694]: pgmap v21903: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 588 KiB/s wr, 12 op/s 2023-09-10T10:39:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40774.695910 secs 2023-09-10T10:39:04.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40775.924137 secs 2023-09-10T10:39:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:04 smithi098 ceph-mon[103124]: pgmap v21903: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 588 KiB/s wr, 12 op/s 2023-09-10T10:39:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40774.695910 secs 2023-09-10T10:39:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40775.924137 secs 2023-09-10T10:39:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40776.275549 secs 2023-09-10T10:39:05.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:05 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:39:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:05 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:39:05.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:05 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40776.275549 secs 2023-09-10T10:39:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:05 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:39:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:05 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:39:05.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:05 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40776.275549 secs 2023-09-10T10:39:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:05 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:39:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:05 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:39:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:05 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:06.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:06 smithi082 ceph-mon[98694]: pgmap v21904: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 588 KiB/s wr, 12 op/s 2023-09-10T10:39:06.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:06 smithi098 ceph-mon[103124]: pgmap v21904: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 588 KiB/s wr, 12 op/s 2023-09-10T10:39:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:06 smithi170 ceph-mon[102832]: pgmap v21904: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 588 KiB/s wr, 12 op/s 2023-09-10T10:39:08.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:08 smithi082 ceph-mon[98694]: pgmap v21905: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 779 KiB/s wr, 33 op/s 2023-09-10T10:39:08.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:08 smithi098 ceph-mon[103124]: pgmap v21905: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 779 KiB/s wr, 33 op/s 2023-09-10T10:39:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:08 smithi170 ceph-mon[102832]: pgmap v21905: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 779 KiB/s wr, 33 op/s 2023-09-10T10:39:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40779.695967 secs 2023-09-10T10:39:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:09 smithi082 ceph-mon[98694]: osdmap e1437: 12 total, 12 up, 12 in 2023-09-10T10:39:09.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40780.924221 secs 2023-09-10T10:39:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40779.695967 secs 2023-09-10T10:39:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:09 smithi098 ceph-mon[103124]: osdmap e1437: 12 total, 12 up, 12 in 2023-09-10T10:39:09.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40780.924221 secs 2023-09-10T10:39:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40779.695967 secs 2023-09-10T10:39:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:09 smithi170 ceph-mon[102832]: osdmap e1437: 12 total, 12 up, 12 in 2023-09-10T10:39:09.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40780.924221 secs 2023-09-10T10:39:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:10 smithi082 ceph-mon[98694]: pgmap v21907: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 555 KiB/s wr, 33 op/s 2023-09-10T10:39:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40781.275609 secs 2023-09-10T10:39:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:39:10.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:10 smithi098 ceph-mon[103124]: pgmap v21907: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 555 KiB/s wr, 33 op/s 2023-09-10T10:39:10.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40781.275609 secs 2023-09-10T10:39:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:39:10.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:10 smithi170 ceph-mon[102832]: pgmap v21907: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 555 KiB/s wr, 33 op/s 2023-09-10T10:39:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40781.275609 secs 2023-09-10T10:39:10.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:39:10.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:12 smithi082 ceph-mon[98694]: pgmap v21908: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 482 KiB/s wr, 32 op/s 2023-09-10T10:39:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:39:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:12 smithi098 ceph-mon[103124]: pgmap v21908: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 482 KiB/s wr, 32 op/s 2023-09-10T10:39:12.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:39:12.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:12 smithi170 ceph-mon[102832]: pgmap v21908: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 482 KiB/s wr, 32 op/s 2023-09-10T10:39:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:39:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:14 smithi082 ceph-mon[98694]: pgmap v21909: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 482 KiB/s wr, 32 op/s 2023-09-10T10:39:14.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40784.696047 secs 2023-09-10T10:39:14.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40785.924281 secs 2023-09-10T10:39:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:14 smithi098 ceph-mon[103124]: pgmap v21909: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 482 KiB/s wr, 32 op/s 2023-09-10T10:39:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40784.696047 secs 2023-09-10T10:39:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40785.924281 secs 2023-09-10T10:39:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:14 smithi170 ceph-mon[102832]: pgmap v21909: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 482 KiB/s wr, 32 op/s 2023-09-10T10:39:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40784.696047 secs 2023-09-10T10:39:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40785.924281 secs 2023-09-10T10:39:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40786.275673 secs 2023-09-10T10:39:15.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:15 smithi098 ceph-mon[103124]: osdmap e1438: 12 total, 12 up, 12 in 2023-09-10T10:39:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40786.275673 secs 2023-09-10T10:39:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:15 smithi170 ceph-mon[102832]: osdmap e1438: 12 total, 12 up, 12 in 2023-09-10T10:39:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40786.275673 secs 2023-09-10T10:39:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:15 smithi082 ceph-mon[98694]: osdmap e1438: 12 total, 12 up, 12 in 2023-09-10T10:39:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:16 smithi098 ceph-mon[103124]: pgmap v21910: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 482 KiB/s wr, 32 op/s 2023-09-10T10:39:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:16 smithi170 ceph-mon[102832]: pgmap v21910: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 482 KiB/s wr, 32 op/s 2023-09-10T10:39:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:16 smithi082 ceph-mon[98694]: pgmap v21910: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 482 KiB/s wr, 32 op/s 2023-09-10T10:39:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:18 smithi098 ceph-mon[103124]: pgmap v21912: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 481 KiB/s wr, 16 op/s 2023-09-10T10:39:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:18 smithi170 ceph-mon[102832]: pgmap v21912: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 481 KiB/s wr, 16 op/s 2023-09-10T10:39:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:18 smithi082 ceph-mon[98694]: pgmap v21912: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 481 KiB/s wr, 16 op/s 2023-09-10T10:39:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40789.696158 secs 2023-09-10T10:39:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40790.924340 secs 2023-09-10T10:39:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40789.696158 secs 2023-09-10T10:39:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40790.924340 secs 2023-09-10T10:39:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40789.696158 secs 2023-09-10T10:39:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40790.924340 secs 2023-09-10T10:39:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:20 smithi098 ceph-mon[103124]: pgmap v21913: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 398 KiB/s wr, 13 op/s 2023-09-10T10:39:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40791.275733 secs 2023-09-10T10:39:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:20 smithi170 ceph-mon[102832]: pgmap v21913: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 398 KiB/s wr, 13 op/s 2023-09-10T10:39:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40791.275733 secs 2023-09-10T10:39:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:20 smithi082 ceph-mon[98694]: pgmap v21913: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 398 KiB/s wr, 13 op/s 2023-09-10T10:39:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40791.275733 secs 2023-09-10T10:39:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:22 smithi098 ceph-mon[103124]: pgmap v21914: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T10:39:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:22 smithi170 ceph-mon[102832]: pgmap v21914: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T10:39:22.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:22 smithi082 ceph-mon[98694]: pgmap v21914: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T10:39:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:24 smithi098 ceph-mon[103124]: pgmap v21915: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T10:39:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40794.696239 secs 2023-09-10T10:39:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40795.924398 secs 2023-09-10T10:39:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:24 smithi170 ceph-mon[102832]: pgmap v21915: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T10:39:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40794.696239 secs 2023-09-10T10:39:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40795.924398 secs 2023-09-10T10:39:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:24 smithi082 ceph-mon[98694]: pgmap v21915: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T10:39:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40794.696239 secs 2023-09-10T10:39:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40795.924398 secs 2023-09-10T10:39:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40796.275801 secs 2023-09-10T10:39:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40796.275801 secs 2023-09-10T10:39:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40796.275801 secs 2023-09-10T10:39:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:26 smithi098 ceph-mon[103124]: pgmap v21916: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T10:39:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:26 smithi170 ceph-mon[102832]: pgmap v21916: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T10:39:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:26 smithi082 ceph-mon[98694]: pgmap v21916: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T10:39:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:28 smithi098 ceph-mon[103124]: pgmap v21917: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 290 KiB/s wr, 17 op/s 2023-09-10T10:39:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:28 smithi170 ceph-mon[102832]: pgmap v21917: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 290 KiB/s wr, 17 op/s 2023-09-10T10:39:28.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:28 smithi082 ceph-mon[98694]: pgmap v21917: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 290 KiB/s wr, 17 op/s 2023-09-10T10:39:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40799.696296 secs 2023-09-10T10:39:29.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40800.924476 secs 2023-09-10T10:39:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40799.696296 secs 2023-09-10T10:39:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40800.924476 secs 2023-09-10T10:39:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40799.696296 secs 2023-09-10T10:39:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40800.924476 secs 2023-09-10T10:39:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:30 smithi098 ceph-mon[103124]: pgmap v21918: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:39:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40801.275856 secs 2023-09-10T10:39:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:30 smithi170 ceph-mon[102832]: pgmap v21918: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:39:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40801.275856 secs 2023-09-10T10:39:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:30 smithi082 ceph-mon[98694]: pgmap v21918: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:39:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40801.275856 secs 2023-09-10T10:39:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:32 smithi098 ceph-mon[103124]: pgmap v21919: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 152 KiB/s wr, 16 op/s 2023-09-10T10:39:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:32 smithi170 ceph-mon[102832]: pgmap v21919: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 152 KiB/s wr, 16 op/s 2023-09-10T10:39:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:32 smithi082 ceph-mon[98694]: pgmap v21919: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 152 KiB/s wr, 16 op/s 2023-09-10T10:39:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:34 smithi098 ceph-mon[103124]: pgmap v21920: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:39:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40804.696356 secs 2023-09-10T10:39:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:39:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:34.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40805.924536 secs 2023-09-10T10:39:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:34 smithi170 ceph-mon[102832]: pgmap v21920: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:39:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40804.696356 secs 2023-09-10T10:39:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:39:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40805.924536 secs 2023-09-10T10:39:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:34 smithi082 ceph-mon[98694]: pgmap v21920: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:39:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40804.696356 secs 2023-09-10T10:39:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:39:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40805.924536 secs 2023-09-10T10:39:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40806.275918 secs 2023-09-10T10:39:35.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40806.275918 secs 2023-09-10T10:39:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40806.275918 secs 2023-09-10T10:39:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:36 smithi170 ceph-mon[102832]: pgmap v21921: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:39:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:39:36.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:36 smithi082 ceph-mon[98694]: pgmap v21921: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:39:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:39:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:36 smithi098 ceph-mon[103124]: pgmap v21921: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:39:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:39:36.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:38 smithi170 ceph-mon[102832]: pgmap v21922: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 344 KiB/s wr, 17 op/s 2023-09-10T10:39:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:39:38.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:38 smithi082 ceph-mon[98694]: pgmap v21922: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 344 KiB/s wr, 17 op/s 2023-09-10T10:39:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:39:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:38 smithi098 ceph-mon[103124]: pgmap v21922: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 344 KiB/s wr, 17 op/s 2023-09-10T10:39:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:39:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:39 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40809.696457 secs 2023-09-10T10:39:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40810.924593 secs 2023-09-10T10:39:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:39 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40809.696457 secs 2023-09-10T10:39:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40810.924593 secs 2023-09-10T10:39:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:39 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40809.696457 secs 2023-09-10T10:39:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40810.924593 secs 2023-09-10T10:39:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:40 smithi170 ceph-mon[102832]: pgmap v21923: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T10:39:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40811.275982 secs 2023-09-10T10:39:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:39:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:40 smithi082 ceph-mon[98694]: pgmap v21923: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T10:39:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40811.275982 secs 2023-09-10T10:39:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:39:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:40 smithi098 ceph-mon[103124]: pgmap v21923: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 193 KiB/s wr, 11 op/s 2023-09-10T10:39:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40811.275982 secs 2023-09-10T10:39:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:39:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:42 smithi170 ceph-mon[102832]: pgmap v21924: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 565 KiB/s wr, 18 op/s 2023-09-10T10:39:42.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:42 smithi082 ceph-mon[98694]: pgmap v21924: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 565 KiB/s wr, 18 op/s 2023-09-10T10:39:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:42 smithi098 ceph-mon[103124]: pgmap v21924: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 565 KiB/s wr, 18 op/s 2023-09-10T10:39:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:44 smithi170 ceph-mon[102832]: pgmap v21925: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 565 KiB/s wr, 12 op/s 2023-09-10T10:39:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:44 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40814.696532 secs 2023-09-10T10:39:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40815.924653 secs 2023-09-10T10:39:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:44 smithi082 ceph-mon[98694]: pgmap v21925: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 565 KiB/s wr, 12 op/s 2023-09-10T10:39:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:44 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40814.696532 secs 2023-09-10T10:39:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40815.924653 secs 2023-09-10T10:39:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:44 smithi098 ceph-mon[103124]: pgmap v21925: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 565 KiB/s wr, 12 op/s 2023-09-10T10:39:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:44 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40814.696532 secs 2023-09-10T10:39:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40815.924653 secs 2023-09-10T10:39:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40816.276042 secs 2023-09-10T10:39:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40816.276042 secs 2023-09-10T10:39:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40816.276042 secs 2023-09-10T10:39:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:46 smithi170 ceph-mon[102832]: pgmap v21926: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 565 KiB/s wr, 12 op/s 2023-09-10T10:39:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:46 smithi082 ceph-mon[98694]: pgmap v21926: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 565 KiB/s wr, 12 op/s 2023-09-10T10:39:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:46 smithi098 ceph-mon[103124]: pgmap v21926: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 565 KiB/s wr, 12 op/s 2023-09-10T10:39:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:48 smithi170 ceph-mon[102832]: pgmap v21927: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 804 KiB/s wr, 19 op/s 2023-09-10T10:39:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:48 smithi082 ceph-mon[98694]: pgmap v21927: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 804 KiB/s wr, 19 op/s 2023-09-10T10:39:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:48 smithi098 ceph-mon[103124]: pgmap v21927: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 804 KiB/s wr, 19 op/s 2023-09-10T10:39:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40820.924702 secs 2023-09-10T10:39:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40820.924702 secs 2023-09-10T10:39:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40820.924702 secs 2023-09-10T10:39:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:50 smithi170 ceph-mon[102832]: pgmap v21928: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 611 KiB/s wr, 13 op/s 2023-09-10T10:39:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40821.276102 secs 2023-09-10T10:39:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:39:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:50 smithi082 ceph-mon[98694]: pgmap v21928: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 611 KiB/s wr, 13 op/s 2023-09-10T10:39:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40821.276102 secs 2023-09-10T10:39:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:39:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:50 smithi098 ceph-mon[103124]: pgmap v21928: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 611 KiB/s wr, 13 op/s 2023-09-10T10:39:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40821.276102 secs 2023-09-10T10:39:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:39:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:51 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:39:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:51 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:39:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:51 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:39:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:52 smithi170 ceph-mon[102832]: pgmap v21929: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T10:39:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:52 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:39:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:39:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:52 smithi082 ceph-mon[98694]: pgmap v21929: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T10:39:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:52 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:39:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:39:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:52 smithi098 ceph-mon[103124]: pgmap v21929: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T10:39:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:52 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:39:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:39:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:54 smithi170 ceph-mon[102832]: pgmap v21930: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 745 KiB/s wr, 14 op/s 2023-09-10T10:39:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40825.924760 secs 2023-09-10T10:39:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:54 smithi082 ceph-mon[98694]: pgmap v21930: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 745 KiB/s wr, 14 op/s 2023-09-10T10:39:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40825.924760 secs 2023-09-10T10:39:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:54 smithi098 ceph-mon[103124]: pgmap v21930: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 745 KiB/s wr, 14 op/s 2023-09-10T10:39:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40825.924760 secs 2023-09-10T10:39:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40826.276164 secs 2023-09-10T10:39:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40826.276164 secs 2023-09-10T10:39:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40826.276164 secs 2023-09-10T10:39:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:56 smithi170 ceph-mon[102832]: pgmap v21931: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 745 KiB/s wr, 14 op/s 2023-09-10T10:39:56.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:56 smithi082 ceph-mon[98694]: pgmap v21931: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 745 KiB/s wr, 14 op/s 2023-09-10T10:39:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:56 smithi098 ceph-mon[103124]: pgmap v21931: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 745 KiB/s wr, 14 op/s 2023-09-10T10:39:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:58 smithi170 ceph-mon[102832]: pgmap v21932: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 985 KiB/s wr, 22 op/s 2023-09-10T10:39:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:39:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:58 smithi082 ceph-mon[98694]: pgmap v21932: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 985 KiB/s wr, 22 op/s 2023-09-10T10:39:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:39:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:58 smithi098 ceph-mon[103124]: pgmap v21932: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 985 KiB/s wr, 22 op/s 2023-09-10T10:39:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:39:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:39:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:39:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40830.924844 secs 2023-09-10T10:39:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:39:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40830.924844 secs 2023-09-10T10:39:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:39:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40830.924844 secs 2023-09-10T10:40:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:00 smithi170 ceph-mon[102832]: pgmap v21933: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 746 KiB/s wr, 15 op/s 2023-09-10T10:40:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40831.276225 secs 2023-09-10T10:40:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:40:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 2 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T10:40:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:00 smithi082 ceph-mon[98694]: pgmap v21933: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 746 KiB/s wr, 15 op/s 2023-09-10T10:40:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40831.276225 secs 2023-09-10T10:40:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:40:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:00.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 2 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T10:40:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:00 smithi098 ceph-mon[103124]: pgmap v21933: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 746 KiB/s wr, 15 op/s 2023-09-10T10:40:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40831.276225 secs 2023-09-10T10:40:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:40:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 2 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T10:40:01.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:01 smithi170 ceph-mon[102832]: pgmap v21934: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1019 KiB/s wr, 23 op/s 2023-09-10T10:40:01.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:01 smithi082 ceph-mon[98694]: pgmap v21934: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1019 KiB/s wr, 23 op/s 2023-09-10T10:40:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:01 smithi098 ceph-mon[103124]: pgmap v21934: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1019 KiB/s wr, 23 op/s 2023-09-10T10:40:02.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:02 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:40:02.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:02 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:40:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:02 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:40:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:03 smithi170 ceph-mon[102832]: pgmap v21935: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 513 KiB/s wr, 15 op/s 2023-09-10T10:40:03.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:40:03.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:40:03.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:40:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:03 smithi082 ceph-mon[98694]: pgmap v21935: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 513 KiB/s wr, 15 op/s 2023-09-10T10:40:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:40:03.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:40:03.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:40:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:03 smithi098 ceph-mon[103124]: pgmap v21935: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 513 KiB/s wr, 15 op/s 2023-09-10T10:40:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:40:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:40:03.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:40:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40834.696785 secs 2023-09-10T10:40:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:40:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:40:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40835.924926 secs 2023-09-10T10:40:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40834.696785 secs 2023-09-10T10:40:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:40:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:40:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:04.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40835.924926 secs 2023-09-10T10:40:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40834.696785 secs 2023-09-10T10:40:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:40:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:40:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40835.924926 secs 2023-09-10T10:40:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40836.276286 secs 2023-09-10T10:40:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:05 smithi170 ceph-mon[102832]: pgmap v21936: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 513 KiB/s wr, 15 op/s 2023-09-10T10:40:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40836.276286 secs 2023-09-10T10:40:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:05 smithi082 ceph-mon[98694]: pgmap v21936: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 513 KiB/s wr, 15 op/s 2023-09-10T10:40:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40836.276286 secs 2023-09-10T10:40:05.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:05 smithi098 ceph-mon[103124]: pgmap v21936: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 513 KiB/s wr, 15 op/s 2023-09-10T10:40:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:07 smithi170 ceph-mon[102832]: pgmap v21937: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 704 KiB/s wr, 36 op/s 2023-09-10T10:40:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:07 smithi082 ceph-mon[98694]: pgmap v21937: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 704 KiB/s wr, 36 op/s 2023-09-10T10:40:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:07 smithi098 ceph-mon[103124]: pgmap v21937: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 704 KiB/s wr, 36 op/s 2023-09-10T10:40:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40839.696886 secs 2023-09-10T10:40:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40839.696886 secs 2023-09-10T10:40:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40839.696886 secs 2023-09-10T10:40:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:09 smithi170 ceph-mon[102832]: pgmap v21938: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 464 KiB/s wr, 29 op/s 2023-09-10T10:40:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:09 smithi170 ceph-mon[102832]: osdmap e1439: 12 total, 12 up, 12 in 2023-09-10T10:40:09.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40840.924984 secs 2023-09-10T10:40:09.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:09 smithi082 ceph-mon[98694]: pgmap v21938: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 464 KiB/s wr, 29 op/s 2023-09-10T10:40:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:09 smithi082 ceph-mon[98694]: osdmap e1439: 12 total, 12 up, 12 in 2023-09-10T10:40:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40840.924984 secs 2023-09-10T10:40:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:09 smithi098 ceph-mon[103124]: pgmap v21938: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 464 KiB/s wr, 29 op/s 2023-09-10T10:40:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:09 smithi098 ceph-mon[103124]: osdmap e1439: 12 total, 12 up, 12 in 2023-09-10T10:40:10.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40840.924984 secs 2023-09-10T10:40:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40841.276349 secs 2023-09-10T10:40:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:40:10.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40841.276349 secs 2023-09-10T10:40:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:40:10.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40841.276349 secs 2023-09-10T10:40:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:40:11.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:12 smithi082 ceph-mon[98694]: pgmap v21940: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 514 KiB/s wr, 33 op/s 2023-09-10T10:40:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:40:12.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:12 smithi098 ceph-mon[103124]: pgmap v21940: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 514 KiB/s wr, 33 op/s 2023-09-10T10:40:12.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:40:12.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:12 smithi170 ceph-mon[102832]: pgmap v21940: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 514 KiB/s wr, 33 op/s 2023-09-10T10:40:12.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:40:12.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:14.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:14 smithi082 ceph-mon[98694]: pgmap v21941: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 514 KiB/s wr, 33 op/s 2023-09-10T10:40:14.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40844.696945 secs 2023-09-10T10:40:14.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40845.925042 secs 2023-09-10T10:40:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:14 smithi098 ceph-mon[103124]: pgmap v21941: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 514 KiB/s wr, 33 op/s 2023-09-10T10:40:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40844.696945 secs 2023-09-10T10:40:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40845.925042 secs 2023-09-10T10:40:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:14 smithi170 ceph-mon[102832]: pgmap v21941: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 514 KiB/s wr, 33 op/s 2023-09-10T10:40:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40844.696945 secs 2023-09-10T10:40:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40845.925042 secs 2023-09-10T10:40:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40846.276408 secs 2023-09-10T10:40:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40846.276408 secs 2023-09-10T10:40:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40846.276408 secs 2023-09-10T10:40:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:16 smithi098 ceph-mon[103124]: pgmap v21942: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 514 KiB/s wr, 33 op/s 2023-09-10T10:40:16.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:16 smithi098 ceph-mon[103124]: osdmap e1440: 12 total, 12 up, 12 in 2023-09-10T10:40:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:16 smithi170 ceph-mon[102832]: pgmap v21942: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 514 KiB/s wr, 33 op/s 2023-09-10T10:40:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:16 smithi170 ceph-mon[102832]: osdmap e1440: 12 total, 12 up, 12 in 2023-09-10T10:40:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:16 smithi082 ceph-mon[98694]: pgmap v21942: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 514 KiB/s wr, 33 op/s 2023-09-10T10:40:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:16 smithi082 ceph-mon[98694]: osdmap e1440: 12 total, 12 up, 12 in 2023-09-10T10:40:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:18 smithi098 ceph-mon[103124]: pgmap v21944: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 538 KiB/s wr, 17 op/s 2023-09-10T10:40:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:18 smithi170 ceph-mon[102832]: pgmap v21944: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 538 KiB/s wr, 17 op/s 2023-09-10T10:40:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:18 smithi082 ceph-mon[98694]: pgmap v21944: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 538 KiB/s wr, 17 op/s 2023-09-10T10:40:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40849.697031 secs 2023-09-10T10:40:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40850.925103 secs 2023-09-10T10:40:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40849.697031 secs 2023-09-10T10:40:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40850.925103 secs 2023-09-10T10:40:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40849.697031 secs 2023-09-10T10:40:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40850.925103 secs 2023-09-10T10:40:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:20 smithi098 ceph-mon[103124]: pgmap v21945: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 433 KiB/s wr, 14 op/s 2023-09-10T10:40:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40851.276474 secs 2023-09-10T10:40:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:20 smithi170 ceph-mon[102832]: pgmap v21945: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 433 KiB/s wr, 14 op/s 2023-09-10T10:40:20.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40851.276474 secs 2023-09-10T10:40:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:20 smithi082 ceph-mon[98694]: pgmap v21945: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 433 KiB/s wr, 14 op/s 2023-09-10T10:40:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40851.276474 secs 2023-09-10T10:40:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:22 smithi098 ceph-mon[103124]: pgmap v21946: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T10:40:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:22 smithi170 ceph-mon[102832]: pgmap v21946: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T10:40:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:22 smithi082 ceph-mon[98694]: pgmap v21946: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T10:40:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:24 smithi098 ceph-mon[103124]: pgmap v21947: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T10:40:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40854.697093 secs 2023-09-10T10:40:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40855.925160 secs 2023-09-10T10:40:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:24 smithi170 ceph-mon[102832]: pgmap v21947: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T10:40:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40854.697093 secs 2023-09-10T10:40:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40855.925160 secs 2023-09-10T10:40:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:24 smithi082 ceph-mon[98694]: pgmap v21947: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T10:40:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40854.697093 secs 2023-09-10T10:40:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40855.925160 secs 2023-09-10T10:40:25.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40856.276537 secs 2023-09-10T10:40:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40856.276537 secs 2023-09-10T10:40:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40856.276537 secs 2023-09-10T10:40:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:26 smithi098 ceph-mon[103124]: pgmap v21948: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T10:40:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:26 smithi170 ceph-mon[102832]: pgmap v21948: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T10:40:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:26 smithi082 ceph-mon[98694]: pgmap v21948: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T10:40:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:28 smithi098 ceph-mon[103124]: pgmap v21949: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 289 KiB/s wr, 16 op/s 2023-09-10T10:40:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:28 smithi170 ceph-mon[102832]: pgmap v21949: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 289 KiB/s wr, 16 op/s 2023-09-10T10:40:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:28 smithi082 ceph-mon[98694]: pgmap v21949: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 289 KiB/s wr, 16 op/s 2023-09-10T10:40:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40859.697154 secs 2023-09-10T10:40:29.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40860.925220 secs 2023-09-10T10:40:29.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40859.697154 secs 2023-09-10T10:40:29.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40860.925220 secs 2023-09-10T10:40:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40859.697154 secs 2023-09-10T10:40:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40860.925220 secs 2023-09-10T10:40:30.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:30 smithi098 ceph-mon[103124]: pgmap v21950: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T10:40:30.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40861.276600 secs 2023-09-10T10:40:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:30 smithi170 ceph-mon[102832]: pgmap v21950: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T10:40:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40861.276600 secs 2023-09-10T10:40:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:30 smithi082 ceph-mon[98694]: pgmap v21950: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T10:40:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40861.276600 secs 2023-09-10T10:40:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:32 smithi098 ceph-mon[103124]: pgmap v21951: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 152 KiB/s wr, 16 op/s 2023-09-10T10:40:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:32 smithi170 ceph-mon[102832]: pgmap v21951: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 152 KiB/s wr, 16 op/s 2023-09-10T10:40:32.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:32 smithi082 ceph-mon[98694]: pgmap v21951: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 152 KiB/s wr, 16 op/s 2023-09-10T10:40:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:34 smithi170 ceph-mon[102832]: pgmap v21952: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:40:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40864.697271 secs 2023-09-10T10:40:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:40:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40865.925277 secs 2023-09-10T10:40:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:34 smithi082 ceph-mon[98694]: pgmap v21952: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:40:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40864.697271 secs 2023-09-10T10:40:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:40:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:34.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40865.925277 secs 2023-09-10T10:40:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:34 smithi098 ceph-mon[103124]: pgmap v21952: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:40:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40864.697271 secs 2023-09-10T10:40:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:40:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40865.925277 secs 2023-09-10T10:40:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40866.276661 secs 2023-09-10T10:40:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40866.276661 secs 2023-09-10T10:40:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40866.276661 secs 2023-09-10T10:40:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:36 smithi170 ceph-mon[102832]: pgmap v21953: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:40:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:40:36.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:36 smithi082 ceph-mon[98694]: pgmap v21953: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:40:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:40:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:36 smithi098 ceph-mon[103124]: pgmap v21953: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 11 op/s 2023-09-10T10:40:36.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:40:36.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:38 smithi170 ceph-mon[102832]: pgmap v21954: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 444 KiB/s wr, 17 op/s 2023-09-10T10:40:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:40:38.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:38 smithi082 ceph-mon[98694]: pgmap v21954: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 444 KiB/s wr, 17 op/s 2023-09-10T10:40:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:40:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:38 smithi098 ceph-mon[103124]: pgmap v21954: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 444 KiB/s wr, 17 op/s 2023-09-10T10:40:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:40:38.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:39 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40869.697334 secs 2023-09-10T10:40:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40870.925337 secs 2023-09-10T10:40:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:39 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40869.697334 secs 2023-09-10T10:40:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40870.925337 secs 2023-09-10T10:40:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:39 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40869.697334 secs 2023-09-10T10:40:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40870.925337 secs 2023-09-10T10:40:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:40 smithi170 ceph-mon[102832]: pgmap v21955: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T10:40:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40871.276723 secs 2023-09-10T10:40:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:40:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:40 smithi082 ceph-mon[98694]: pgmap v21955: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T10:40:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40871.276723 secs 2023-09-10T10:40:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:40:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:40 smithi098 ceph-mon[103124]: pgmap v21955: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 293 KiB/s wr, 11 op/s 2023-09-10T10:40:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40871.276723 secs 2023-09-10T10:40:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:40:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:42 smithi170 ceph-mon[102832]: pgmap v21956: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 483 KiB/s wr, 18 op/s 2023-09-10T10:40:42.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:42 smithi082 ceph-mon[98694]: pgmap v21956: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 483 KiB/s wr, 18 op/s 2023-09-10T10:40:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:42 smithi098 ceph-mon[103124]: pgmap v21956: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 483 KiB/s wr, 18 op/s 2023-09-10T10:40:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:44 smithi170 ceph-mon[102832]: pgmap v21957: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 482 KiB/s wr, 12 op/s 2023-09-10T10:40:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:44 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40874.697421 secs 2023-09-10T10:40:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40875.925396 secs 2023-09-10T10:40:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:44 smithi082 ceph-mon[98694]: pgmap v21957: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 482 KiB/s wr, 12 op/s 2023-09-10T10:40:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:44 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40874.697421 secs 2023-09-10T10:40:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40875.925396 secs 2023-09-10T10:40:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:44 smithi098 ceph-mon[103124]: pgmap v21957: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 482 KiB/s wr, 12 op/s 2023-09-10T10:40:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:44 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40874.697421 secs 2023-09-10T10:40:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40875.925396 secs 2023-09-10T10:40:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40876.276783 secs 2023-09-10T10:40:45.860 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40876.276783 secs 2023-09-10T10:40:45.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40876.276783 secs 2023-09-10T10:40:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:46 smithi170 ceph-mon[102832]: pgmap v21958: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 482 KiB/s wr, 12 op/s 2023-09-10T10:40:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:46 smithi082 ceph-mon[98694]: pgmap v21958: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 482 KiB/s wr, 12 op/s 2023-09-10T10:40:46.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:46 smithi098 ceph-mon[103124]: pgmap v21958: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 482 KiB/s wr, 12 op/s 2023-09-10T10:40:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:48 smithi170 ceph-mon[102832]: pgmap v21959: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 806 KiB/s wr, 19 op/s 2023-09-10T10:40:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:48 smithi082 ceph-mon[98694]: pgmap v21959: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 806 KiB/s wr, 19 op/s 2023-09-10T10:40:48.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:48 smithi098 ceph-mon[103124]: pgmap v21959: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 806 KiB/s wr, 19 op/s 2023-09-10T10:40:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40880.925453 secs 2023-09-10T10:40:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40880.925453 secs 2023-09-10T10:40:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40880.925453 secs 2023-09-10T10:40:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:50 smithi170 ceph-mon[102832]: pgmap v21960: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 514 KiB/s wr, 13 op/s 2023-09-10T10:40:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40881.276845 secs 2023-09-10T10:40:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:40:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:50 smithi082 ceph-mon[98694]: pgmap v21960: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 514 KiB/s wr, 13 op/s 2023-09-10T10:40:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40881.276845 secs 2023-09-10T10:40:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:40:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:50 smithi098 ceph-mon[103124]: pgmap v21960: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 514 KiB/s wr, 13 op/s 2023-09-10T10:40:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40881.276845 secs 2023-09-10T10:40:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:40:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:51 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:40:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:51 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:40:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:51 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:40:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:52 smithi170 ceph-mon[102832]: pgmap v21961: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 786 KiB/s wr, 20 op/s 2023-09-10T10:40:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:52 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:40:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:40:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:52 smithi082 ceph-mon[98694]: pgmap v21961: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 786 KiB/s wr, 20 op/s 2023-09-10T10:40:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:52 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:40:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:40:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:52 smithi098 ceph-mon[103124]: pgmap v21961: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 786 KiB/s wr, 20 op/s 2023-09-10T10:40:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:52 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:40:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:40:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:54 smithi170 ceph-mon[102832]: pgmap v21962: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 596 KiB/s wr, 14 op/s 2023-09-10T10:40:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:40:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40885.925495 secs 2023-09-10T10:40:54.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:54 smithi082 ceph-mon[98694]: pgmap v21962: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 596 KiB/s wr, 14 op/s 2023-09-10T10:40:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:40:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40885.925495 secs 2023-09-10T10:40:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:54 smithi098 ceph-mon[103124]: pgmap v21962: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 596 KiB/s wr, 14 op/s 2023-09-10T10:40:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:40:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40885.925495 secs 2023-09-10T10:40:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40886.276908 secs 2023-09-10T10:40:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40886.276908 secs 2023-09-10T10:40:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40886.276908 secs 2023-09-10T10:40:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:56 smithi170 ceph-mon[102832]: pgmap v21963: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 596 KiB/s wr, 14 op/s 2023-09-10T10:40:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:56 smithi082 ceph-mon[98694]: pgmap v21963: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 596 KiB/s wr, 14 op/s 2023-09-10T10:40:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:56 smithi098 ceph-mon[103124]: pgmap v21963: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 596 KiB/s wr, 14 op/s 2023-09-10T10:40:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:58 smithi170 ceph-mon[102832]: pgmap v21964: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 930 KiB/s wr, 20 op/s 2023-09-10T10:40:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:40:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:58 smithi082 ceph-mon[98694]: pgmap v21964: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 930 KiB/s wr, 20 op/s 2023-09-10T10:40:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:40:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:58 smithi098 ceph-mon[103124]: pgmap v21964: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 930 KiB/s wr, 20 op/s 2023-09-10T10:40:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:40:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:40:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:40:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40890.925565 secs 2023-09-10T10:40:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:40:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40890.925565 secs 2023-09-10T10:40:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:40:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40890.925565 secs 2023-09-10T10:41:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:00 smithi170 ceph-mon[102832]: pgmap v21965: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 605 KiB/s wr, 14 op/s 2023-09-10T10:41:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40891.276968 secs 2023-09-10T10:41:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:41:00.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:00 smithi082 ceph-mon[98694]: pgmap v21965: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 605 KiB/s wr, 14 op/s 2023-09-10T10:41:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40891.276968 secs 2023-09-10T10:41:00.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:41:00.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:00 smithi098 ceph-mon[103124]: pgmap v21965: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 605 KiB/s wr, 14 op/s 2023-09-10T10:41:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40891.276968 secs 2023-09-10T10:41:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:41:00.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:02.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:02 smithi170 ceph-mon[102832]: pgmap v21966: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 888 KiB/s wr, 21 op/s 2023-09-10T10:41:02.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:02 smithi082 ceph-mon[98694]: pgmap v21966: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 888 KiB/s wr, 21 op/s 2023-09-10T10:41:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:02 smithi098 ceph-mon[103124]: pgmap v21966: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 888 KiB/s wr, 21 op/s 2023-09-10T10:41:03.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:41:03.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:41:03.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:03 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:41:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:41:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:41:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:03 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:41:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:41:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:41:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:03 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:41:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:04 smithi170 ceph-mon[102832]: pgmap v21967: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 617 KiB/s wr, 13 op/s 2023-09-10T10:41:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40894.697687 secs 2023-09-10T10:41:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40895.925647 secs 2023-09-10T10:41:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:41:04.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:41:04.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:41:04.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:41:04.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:41:04.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:04 smithi082 ceph-mon[98694]: pgmap v21967: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 617 KiB/s wr, 13 op/s 2023-09-10T10:41:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40894.697687 secs 2023-09-10T10:41:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40895.925647 secs 2023-09-10T10:41:04.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:41:04.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:41:04.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:41:04.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:41:04.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:41:04.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:04 smithi098 ceph-mon[103124]: pgmap v21967: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 617 KiB/s wr, 13 op/s 2023-09-10T10:41:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40894.697687 secs 2023-09-10T10:41:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40895.925647 secs 2023-09-10T10:41:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:41:04.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:41:04.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:41:04.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:41:04.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:41:04.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40896.277028 secs 2023-09-10T10:41:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:05 smithi170 ceph-mon[102832]: pgmap v21968: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 617 KiB/s wr, 13 op/s 2023-09-10T10:41:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40896.277028 secs 2023-09-10T10:41:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:05 smithi082 ceph-mon[98694]: pgmap v21968: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 617 KiB/s wr, 13 op/s 2023-09-10T10:41:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40896.277028 secs 2023-09-10T10:41:05.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:05 smithi098 ceph-mon[103124]: pgmap v21968: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 617 KiB/s wr, 13 op/s 2023-09-10T10:41:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:07 smithi170 ceph-mon[102832]: pgmap v21969: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 808 KiB/s wr, 34 op/s 2023-09-10T10:41:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:07 smithi082 ceph-mon[98694]: pgmap v21969: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 808 KiB/s wr, 34 op/s 2023-09-10T10:41:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:07 smithi098 ceph-mon[103124]: pgmap v21969: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 808 KiB/s wr, 34 op/s 2023-09-10T10:41:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40899.697749 secs 2023-09-10T10:41:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40899.697749 secs 2023-09-10T10:41:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40899.697749 secs 2023-09-10T10:41:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:09 smithi170 ceph-mon[102832]: pgmap v21970: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 474 KiB/s wr, 28 op/s 2023-09-10T10:41:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:09 smithi170 ceph-mon[102832]: osdmap e1441: 12 total, 12 up, 12 in 2023-09-10T10:41:09.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40900.925711 secs 2023-09-10T10:41:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:09 smithi082 ceph-mon[98694]: pgmap v21970: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 474 KiB/s wr, 28 op/s 2023-09-10T10:41:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:09 smithi082 ceph-mon[98694]: osdmap e1441: 12 total, 12 up, 12 in 2023-09-10T10:41:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40900.925711 secs 2023-09-10T10:41:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:09 smithi098 ceph-mon[103124]: pgmap v21970: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 474 KiB/s wr, 28 op/s 2023-09-10T10:41:10.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:09 smithi098 ceph-mon[103124]: osdmap e1441: 12 total, 12 up, 12 in 2023-09-10T10:41:10.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40900.925711 secs 2023-09-10T10:41:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40901.277088 secs 2023-09-10T10:41:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:41:10.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40901.277088 secs 2023-09-10T10:41:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:41:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40901.277088 secs 2023-09-10T10:41:11.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:41:11.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:12 smithi098 ceph-mon[103124]: pgmap v21972: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 484 KiB/s wr, 32 op/s 2023-09-10T10:41:12.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:41:12.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:12 smithi170 ceph-mon[102832]: pgmap v21972: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 484 KiB/s wr, 32 op/s 2023-09-10T10:41:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:41:12.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:12 smithi082 ceph-mon[98694]: pgmap v21972: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 484 KiB/s wr, 32 op/s 2023-09-10T10:41:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:41:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:14.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:14 smithi098 ceph-mon[103124]: pgmap v21973: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 484 KiB/s wr, 32 op/s 2023-09-10T10:41:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40904.697824 secs 2023-09-10T10:41:14.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40905.925781 secs 2023-09-10T10:41:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:14 smithi170 ceph-mon[102832]: pgmap v21973: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 484 KiB/s wr, 32 op/s 2023-09-10T10:41:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40904.697824 secs 2023-09-10T10:41:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40905.925781 secs 2023-09-10T10:41:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:14 smithi082 ceph-mon[98694]: pgmap v21973: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 484 KiB/s wr, 32 op/s 2023-09-10T10:41:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40904.697824 secs 2023-09-10T10:41:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40905.925781 secs 2023-09-10T10:41:15.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40906.277151 secs 2023-09-10T10:41:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40906.277151 secs 2023-09-10T10:41:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40906.277151 secs 2023-09-10T10:41:16.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:16 smithi098 ceph-mon[103124]: pgmap v21974: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 484 KiB/s wr, 32 op/s 2023-09-10T10:41:16.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:16 smithi098 ceph-mon[103124]: osdmap e1442: 12 total, 12 up, 12 in 2023-09-10T10:41:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:16 smithi170 ceph-mon[102832]: pgmap v21974: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 484 KiB/s wr, 32 op/s 2023-09-10T10:41:16.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:16 smithi170 ceph-mon[102832]: osdmap e1442: 12 total, 12 up, 12 in 2023-09-10T10:41:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:16 smithi082 ceph-mon[98694]: pgmap v21974: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 484 KiB/s wr, 32 op/s 2023-09-10T10:41:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:16 smithi082 ceph-mon[98694]: osdmap e1442: 12 total, 12 up, 12 in 2023-09-10T10:41:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:18 smithi098 ceph-mon[103124]: pgmap v21976: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 500 KiB/s wr, 17 op/s 2023-09-10T10:41:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:18 smithi170 ceph-mon[102832]: pgmap v21976: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 500 KiB/s wr, 17 op/s 2023-09-10T10:41:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:18 smithi082 ceph-mon[98694]: pgmap v21976: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 500 KiB/s wr, 17 op/s 2023-09-10T10:41:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40909.697883 secs 2023-09-10T10:41:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40910.925841 secs 2023-09-10T10:41:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40909.697883 secs 2023-09-10T10:41:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40910.925841 secs 2023-09-10T10:41:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40909.697883 secs 2023-09-10T10:41:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40910.925841 secs 2023-09-10T10:41:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:20 smithi098 ceph-mon[103124]: pgmap v21977: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 403 KiB/s wr, 14 op/s 2023-09-10T10:41:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40911.277218 secs 2023-09-10T10:41:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:20 smithi170 ceph-mon[102832]: pgmap v21977: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 403 KiB/s wr, 14 op/s 2023-09-10T10:41:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40911.277218 secs 2023-09-10T10:41:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:20 smithi082 ceph-mon[98694]: pgmap v21977: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 403 KiB/s wr, 14 op/s 2023-09-10T10:41:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40911.277218 secs 2023-09-10T10:41:22.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:22 smithi098 ceph-mon[103124]: pgmap v21978: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T10:41:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:22 smithi170 ceph-mon[102832]: pgmap v21978: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T10:41:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:22 smithi082 ceph-mon[98694]: pgmap v21978: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T10:41:24.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:24 smithi098 ceph-mon[103124]: pgmap v21979: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T10:41:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40914.697956 secs 2023-09-10T10:41:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40915.925901 secs 2023-09-10T10:41:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:24 smithi170 ceph-mon[102832]: pgmap v21979: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T10:41:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40914.697956 secs 2023-09-10T10:41:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40915.925901 secs 2023-09-10T10:41:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:24 smithi082 ceph-mon[98694]: pgmap v21979: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T10:41:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40914.697956 secs 2023-09-10T10:41:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40915.925901 secs 2023-09-10T10:41:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40916.277283 secs 2023-09-10T10:41:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40916.277283 secs 2023-09-10T10:41:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40916.277283 secs 2023-09-10T10:41:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:26 smithi098 ceph-mon[103124]: pgmap v21980: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T10:41:26.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:26 smithi170 ceph-mon[102832]: pgmap v21980: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T10:41:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:26 smithi082 ceph-mon[98694]: pgmap v21980: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 146 KiB/s wr, 13 op/s 2023-09-10T10:41:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:28 smithi170 ceph-mon[102832]: pgmap v21981: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 290 KiB/s wr, 17 op/s 2023-09-10T10:41:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:28 smithi082 ceph-mon[98694]: pgmap v21981: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 290 KiB/s wr, 17 op/s 2023-09-10T10:41:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:28 smithi098 ceph-mon[103124]: pgmap v21981: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 290 KiB/s wr, 17 op/s 2023-09-10T10:41:29.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40919.698017 secs 2023-09-10T10:41:29.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40920.925960 secs 2023-09-10T10:41:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40919.698017 secs 2023-09-10T10:41:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40920.925960 secs 2023-09-10T10:41:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40919.698017 secs 2023-09-10T10:41:29.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40920.925960 secs 2023-09-10T10:41:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:30 smithi170 ceph-mon[102832]: pgmap v21982: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T10:41:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40921.277352 secs 2023-09-10T10:41:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:30 smithi082 ceph-mon[98694]: pgmap v21982: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T10:41:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40921.277352 secs 2023-09-10T10:41:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:30 smithi098 ceph-mon[103124]: pgmap v21982: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T10:41:30.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40921.277352 secs 2023-09-10T10:41:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:32 smithi170 ceph-mon[102832]: pgmap v21983: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 152 KiB/s wr, 16 op/s 2023-09-10T10:41:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:32 smithi082 ceph-mon[98694]: pgmap v21983: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 152 KiB/s wr, 16 op/s 2023-09-10T10:41:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:32 smithi098 ceph-mon[103124]: pgmap v21983: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 152 KiB/s wr, 16 op/s 2023-09-10T10:41:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:34 smithi170 ceph-mon[102832]: pgmap v21984: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T10:41:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40924.698079 secs 2023-09-10T10:41:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:41:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40925.926050 secs 2023-09-10T10:41:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:34 smithi082 ceph-mon[98694]: pgmap v21984: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T10:41:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40924.698079 secs 2023-09-10T10:41:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:41:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:34.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40925.926050 secs 2023-09-10T10:41:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:34 smithi098 ceph-mon[103124]: pgmap v21984: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T10:41:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40924.698079 secs 2023-09-10T10:41:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:41:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40925.926050 secs 2023-09-10T10:41:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40926.277415 secs 2023-09-10T10:41:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40926.277415 secs 2023-09-10T10:41:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40926.277415 secs 2023-09-10T10:41:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:36 smithi170 ceph-mon[102832]: pgmap v21985: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T10:41:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:41:36.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:36 smithi082 ceph-mon[98694]: pgmap v21985: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T10:41:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:41:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:36 smithi098 ceph-mon[103124]: pgmap v21985: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T10:41:36.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:41:36.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:38 smithi170 ceph-mon[102832]: pgmap v21986: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 356 KiB/s wr, 17 op/s 2023-09-10T10:41:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:41:38.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:38 smithi082 ceph-mon[98694]: pgmap v21986: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 356 KiB/s wr, 17 op/s 2023-09-10T10:41:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:41:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:38 smithi098 ceph-mon[103124]: pgmap v21986: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 356 KiB/s wr, 17 op/s 2023-09-10T10:41:38.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:41:38.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40930.926136 secs 2023-09-10T10:41:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:41:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40930.926136 secs 2023-09-10T10:41:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:41:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40930.926136 secs 2023-09-10T10:41:39.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:41:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:40 smithi170 ceph-mon[102832]: pgmap v21987: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 205 KiB/s wr, 11 op/s 2023-09-10T10:41:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40931.277491 secs 2023-09-10T10:41:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:41:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:41:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:40 smithi082 ceph-mon[98694]: pgmap v21987: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 205 KiB/s wr, 11 op/s 2023-09-10T10:41:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40931.277491 secs 2023-09-10T10:41:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:41:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:41:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:40 smithi098 ceph-mon[103124]: pgmap v21987: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 205 KiB/s wr, 11 op/s 2023-09-10T10:41:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40931.277491 secs 2023-09-10T10:41:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:41:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:41:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:42.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:42 smithi170 ceph-mon[102832]: pgmap v21988: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 578 KiB/s wr, 17 op/s 2023-09-10T10:41:42.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:42 smithi082 ceph-mon[98694]: pgmap v21988: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 578 KiB/s wr, 17 op/s 2023-09-10T10:41:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:42 smithi098 ceph-mon[103124]: pgmap v21988: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 578 KiB/s wr, 17 op/s 2023-09-10T10:41:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:44 smithi170 ceph-mon[102832]: pgmap v21989: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 578 KiB/s wr, 12 op/s 2023-09-10T10:41:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40935.926193 secs 2023-09-10T10:41:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:44 smithi082 ceph-mon[98694]: pgmap v21989: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 578 KiB/s wr, 12 op/s 2023-09-10T10:41:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40935.926193 secs 2023-09-10T10:41:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:44 smithi098 ceph-mon[103124]: pgmap v21989: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 578 KiB/s wr, 12 op/s 2023-09-10T10:41:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40935.926193 secs 2023-09-10T10:41:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40936.277555 secs 2023-09-10T10:41:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40936.277555 secs 2023-09-10T10:41:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40936.277555 secs 2023-09-10T10:41:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:46 smithi170 ceph-mon[102832]: pgmap v21990: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 578 KiB/s wr, 12 op/s 2023-09-10T10:41:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:46 smithi082 ceph-mon[98694]: pgmap v21990: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 578 KiB/s wr, 12 op/s 2023-09-10T10:41:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:46 smithi098 ceph-mon[103124]: pgmap v21990: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 578 KiB/s wr, 12 op/s 2023-09-10T10:41:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:48 smithi170 ceph-mon[102832]: pgmap v21991: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 807 KiB/s wr, 19 op/s 2023-09-10T10:41:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:48 smithi082 ceph-mon[98694]: pgmap v21991: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 807 KiB/s wr, 19 op/s 2023-09-10T10:41:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:48 smithi098 ceph-mon[103124]: pgmap v21991: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 807 KiB/s wr, 19 op/s 2023-09-10T10:41:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:49 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40939.698318 secs 2023-09-10T10:41:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40940.926251 secs 2023-09-10T10:41:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:49 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40939.698318 secs 2023-09-10T10:41:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40940.926251 secs 2023-09-10T10:41:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:49 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40939.698318 secs 2023-09-10T10:41:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40940.926251 secs 2023-09-10T10:41:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:50 smithi170 ceph-mon[102832]: pgmap v21992: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 602 KiB/s wr, 13 op/s 2023-09-10T10:41:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40941.277620 secs 2023-09-10T10:41:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:41:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:50 smithi082 ceph-mon[98694]: pgmap v21992: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 602 KiB/s wr, 13 op/s 2023-09-10T10:41:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40941.277620 secs 2023-09-10T10:41:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:41:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:50 smithi098 ceph-mon[103124]: pgmap v21992: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 602 KiB/s wr, 13 op/s 2023-09-10T10:41:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40941.277620 secs 2023-09-10T10:41:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:41:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:51 smithi170 ceph-mon[102832]: pgmap v21993: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 874 KiB/s wr, 20 op/s 2023-09-10T10:41:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:41:51.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:41:51.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:51 smithi082 ceph-mon[98694]: pgmap v21993: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 874 KiB/s wr, 20 op/s 2023-09-10T10:41:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:41:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:41:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:51 smithi098 ceph-mon[103124]: pgmap v21993: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 874 KiB/s wr, 20 op/s 2023-09-10T10:41:51.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:41:51.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:41:51.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:53 smithi170 ceph-mon[102832]: pgmap v21994: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 501 KiB/s wr, 14 op/s 2023-09-10T10:41:53.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:53 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40944.698459 secs 2023-09-10T10:41:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:53 smithi082 ceph-mon[98694]: pgmap v21994: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 501 KiB/s wr, 14 op/s 2023-09-10T10:41:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:53 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40944.698459 secs 2023-09-10T10:41:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:53 smithi098 ceph-mon[103124]: pgmap v21994: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 501 KiB/s wr, 14 op/s 2023-09-10T10:41:54.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:53 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40944.698459 secs 2023-09-10T10:41:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40945.926312 secs 2023-09-10T10:41:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40945.926312 secs 2023-09-10T10:41:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40945.926312 secs 2023-09-10T10:41:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40946.277684 secs 2023-09-10T10:41:55.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:55 smithi170 ceph-mon[102832]: pgmap v21995: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 501 KiB/s wr, 14 op/s 2023-09-10T10:41:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40946.277684 secs 2023-09-10T10:41:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:55 smithi082 ceph-mon[98694]: pgmap v21995: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 501 KiB/s wr, 14 op/s 2023-09-10T10:41:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40946.277684 secs 2023-09-10T10:41:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:55 smithi098 ceph-mon[103124]: pgmap v21995: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 501 KiB/s wr, 14 op/s 2023-09-10T10:41:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:57 smithi170 ceph-mon[102832]: pgmap v21996: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 819 KiB/s wr, 21 op/s 2023-09-10T10:41:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:57 smithi082 ceph-mon[98694]: pgmap v21996: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 819 KiB/s wr, 21 op/s 2023-09-10T10:41:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:57 smithi098 ceph-mon[103124]: pgmap v21996: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 819 KiB/s wr, 21 op/s 2023-09-10T10:41:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:41:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:41:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:41:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:41:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:59 smithi170 ceph-mon[102832]: pgmap v21997: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 590 KiB/s wr, 14 op/s 2023-09-10T10:41:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40950.926370 secs 2023-09-10T10:41:59.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:41:59 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:41:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:59 smithi082 ceph-mon[98694]: pgmap v21997: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 590 KiB/s wr, 14 op/s 2023-09-10T10:41:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40950.926370 secs 2023-09-10T10:41:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:41:59 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:42:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:59 smithi098 ceph-mon[103124]: pgmap v21997: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 590 KiB/s wr, 14 op/s 2023-09-10T10:42:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40950.926370 secs 2023-09-10T10:42:00.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:41:59 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:42:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40951.277748 secs 2023-09-10T10:42:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:00 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:42:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:42:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40951.277748 secs 2023-09-10T10:42:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:00 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:42:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:42:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40951.277748 secs 2023-09-10T10:42:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:00 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:42:01.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:42:01.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:02.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:02 smithi098 ceph-mon[103124]: pgmap v21998: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 889 KiB/s wr, 20 op/s 2023-09-10T10:42:02.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:02 smithi170 ceph-mon[102832]: pgmap v21998: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 889 KiB/s wr, 20 op/s 2023-09-10T10:42:02.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:02 smithi082 ceph-mon[98694]: pgmap v21998: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 889 KiB/s wr, 20 op/s 2023-09-10T10:42:04.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:04 smithi098 ceph-mon[103124]: pgmap v21999: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 618 KiB/s wr, 13 op/s 2023-09-10T10:42:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40954.698581 secs 2023-09-10T10:42:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:42:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:04.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40955.926425 secs 2023-09-10T10:42:04.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:42:04.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:04 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:42:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:04 smithi170 ceph-mon[102832]: pgmap v21999: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 618 KiB/s wr, 13 op/s 2023-09-10T10:42:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40954.698581 secs 2023-09-10T10:42:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:42:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40955.926425 secs 2023-09-10T10:42:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:42:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:04 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:42:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:04 smithi082 ceph-mon[98694]: pgmap v21999: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 618 KiB/s wr, 13 op/s 2023-09-10T10:42:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40954.698581 secs 2023-09-10T10:42:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:42:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:04.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40955.926425 secs 2023-09-10T10:42:04.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:42:04.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:04 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:42:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40956.277811 secs 2023-09-10T10:42:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:42:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:42:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:42:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:42:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:42:05.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:42:05.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:05 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:42:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40956.277811 secs 2023-09-10T10:42:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:42:05.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:42:05.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:42:05.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:42:05.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:42:05.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:42:05.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:05 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:42:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40956.277811 secs 2023-09-10T10:42:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:42:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:42:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:42:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:42:05.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:42:05.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:42:05.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:05 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:42:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:06 smithi170 ceph-mon[102832]: pgmap v22000: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 618 KiB/s wr, 13 op/s 2023-09-10T10:42:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:06 smithi082 ceph-mon[98694]: pgmap v22000: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 618 KiB/s wr, 13 op/s 2023-09-10T10:42:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:06 smithi098 ceph-mon[103124]: pgmap v22000: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 618 KiB/s wr, 13 op/s 2023-09-10T10:42:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:08 smithi170 ceph-mon[102832]: pgmap v22001: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 835 KiB/s wr, 34 op/s 2023-09-10T10:42:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:08 smithi082 ceph-mon[98694]: pgmap v22001: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 835 KiB/s wr, 34 op/s 2023-09-10T10:42:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:08 smithi098 ceph-mon[103124]: pgmap v22001: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 835 KiB/s wr, 34 op/s 2023-09-10T10:42:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40959.698642 secs 2023-09-10T10:42:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:09 smithi170 ceph-mon[102832]: osdmap e1443: 12 total, 12 up, 12 in 2023-09-10T10:42:09.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40960.926494 secs 2023-09-10T10:42:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40959.698642 secs 2023-09-10T10:42:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:09 smithi082 ceph-mon[98694]: osdmap e1443: 12 total, 12 up, 12 in 2023-09-10T10:42:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40960.926494 secs 2023-09-10T10:42:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40959.698642 secs 2023-09-10T10:42:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:09 smithi098 ceph-mon[103124]: osdmap e1443: 12 total, 12 up, 12 in 2023-09-10T10:42:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40960.926494 secs 2023-09-10T10:42:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:10 smithi170 ceph-mon[102832]: pgmap v22003: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 620 KiB/s wr, 33 op/s 2023-09-10T10:42:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40961.277879 secs 2023-09-10T10:42:10.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:42:10.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:10 smithi082 ceph-mon[98694]: pgmap v22003: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 620 KiB/s wr, 33 op/s 2023-09-10T10:42:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40961.277879 secs 2023-09-10T10:42:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:42:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:10 smithi098 ceph-mon[103124]: pgmap v22003: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 620 KiB/s wr, 33 op/s 2023-09-10T10:42:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40961.277879 secs 2023-09-10T10:42:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:42:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:12 smithi170 ceph-mon[102832]: pgmap v22004: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 515 KiB/s wr, 33 op/s 2023-09-10T10:42:12.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:42:12.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:12 smithi082 ceph-mon[98694]: pgmap v22004: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 515 KiB/s wr, 33 op/s 2023-09-10T10:42:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:42:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:12 smithi098 ceph-mon[103124]: pgmap v22004: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 515 KiB/s wr, 33 op/s 2023-09-10T10:42:12.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:42:12.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:14 smithi170 ceph-mon[102832]: pgmap v22005: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 515 KiB/s wr, 33 op/s 2023-09-10T10:42:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40964.698702 secs 2023-09-10T10:42:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40965.926577 secs 2023-09-10T10:42:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:14 smithi082 ceph-mon[98694]: pgmap v22005: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 515 KiB/s wr, 33 op/s 2023-09-10T10:42:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40964.698702 secs 2023-09-10T10:42:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40965.926577 secs 2023-09-10T10:42:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:14 smithi098 ceph-mon[103124]: pgmap v22005: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 515 KiB/s wr, 33 op/s 2023-09-10T10:42:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40964.698702 secs 2023-09-10T10:42:14.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40965.926577 secs 2023-09-10T10:42:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40966.277941 secs 2023-09-10T10:42:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:15 smithi170 ceph-mon[102832]: osdmap e1444: 12 total, 12 up, 12 in 2023-09-10T10:42:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40966.277941 secs 2023-09-10T10:42:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:15 smithi082 ceph-mon[98694]: osdmap e1444: 12 total, 12 up, 12 in 2023-09-10T10:42:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40966.277941 secs 2023-09-10T10:42:15.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:15 smithi098 ceph-mon[103124]: osdmap e1444: 12 total, 12 up, 12 in 2023-09-10T10:42:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:16 smithi170 ceph-mon[102832]: pgmap v22006: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 515 KiB/s wr, 33 op/s 2023-09-10T10:42:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:16 smithi082 ceph-mon[98694]: pgmap v22006: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 515 KiB/s wr, 33 op/s 2023-09-10T10:42:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:16 smithi098 ceph-mon[103124]: pgmap v22006: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 515 KiB/s wr, 33 op/s 2023-09-10T10:42:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:18 smithi170 ceph-mon[102832]: pgmap v22008: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 533 KiB/s wr, 19 op/s 2023-09-10T10:42:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:42:18.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:18 smithi082 ceph-mon[98694]: pgmap v22008: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 533 KiB/s wr, 19 op/s 2023-09-10T10:42:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:42:18.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:18.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:18 smithi098 ceph-mon[103124]: pgmap v22008: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 538 KiB/s rd, 533 KiB/s wr, 19 op/s 2023-09-10T10:42:18.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:42:18.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40969.698784 secs 2023-09-10T10:42:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40970.926636 secs 2023-09-10T10:42:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40969.698784 secs 2023-09-10T10:42:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40970.926636 secs 2023-09-10T10:42:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40969.698784 secs 2023-09-10T10:42:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40970.926636 secs 2023-09-10T10:42:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:20 smithi170 ceph-mon[102832]: pgmap v22009: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 438 KiB/s wr, 15 op/s 2023-09-10T10:42:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40971.278011 secs 2023-09-10T10:42:20.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:42:20.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:20 smithi082 ceph-mon[98694]: pgmap v22009: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 438 KiB/s wr, 15 op/s 2023-09-10T10:42:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40971.278011 secs 2023-09-10T10:42:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:42:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:20 smithi098 ceph-mon[103124]: pgmap v22009: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 438 KiB/s wr, 15 op/s 2023-09-10T10:42:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40971.278011 secs 2023-09-10T10:42:20.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:42:20.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:21.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:21 smithi170 ceph-mon[102832]: pgmap v22010: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T10:42:21.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:21 smithi082 ceph-mon[98694]: pgmap v22010: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T10:42:21.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:21 smithi098 ceph-mon[103124]: pgmap v22010: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T10:42:23.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:23 smithi170 ceph-mon[102832]: pgmap v22011: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T10:42:23.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:23 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40974.698876 secs 2023-09-10T10:42:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:23 smithi082 ceph-mon[98694]: pgmap v22011: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T10:42:23.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:23 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40974.698876 secs 2023-09-10T10:42:24.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:23 smithi098 ceph-mon[103124]: pgmap v22011: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T10:42:24.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:23 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40974.698876 secs 2023-09-10T10:42:24.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40975.926694 secs 2023-09-10T10:42:24.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40975.926694 secs 2023-09-10T10:42:25.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40975.926694 secs 2023-09-10T10:42:25.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40976.278075 secs 2023-09-10T10:42:25.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:25 smithi170 ceph-mon[102832]: pgmap v22012: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T10:42:25.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40976.278075 secs 2023-09-10T10:42:25.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:25 smithi082 ceph-mon[98694]: pgmap v22012: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T10:42:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40976.278075 secs 2023-09-10T10:42:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:25 smithi098 ceph-mon[103124]: pgmap v22012: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T10:42:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:27 smithi170 ceph-mon[102832]: pgmap v22013: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 424 KiB/s wr, 19 op/s 2023-09-10T10:42:27.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:27 smithi082 ceph-mon[98694]: pgmap v22013: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 424 KiB/s wr, 19 op/s 2023-09-10T10:42:28.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:27 smithi098 ceph-mon[103124]: pgmap v22013: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 424 KiB/s wr, 19 op/s 2023-09-10T10:42:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40979.698934 secs 2023-09-10T10:42:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40979.698934 secs 2023-09-10T10:42:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40979.698934 secs 2023-09-10T10:42:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:29 smithi170 ceph-mon[102832]: pgmap v22014: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 247 KiB/s wr, 11 op/s 2023-09-10T10:42:29.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40980.926754 secs 2023-09-10T10:42:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:29 smithi082 ceph-mon[98694]: pgmap v22014: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 247 KiB/s wr, 11 op/s 2023-09-10T10:42:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40980.926754 secs 2023-09-10T10:42:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:29 smithi098 ceph-mon[103124]: pgmap v22014: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 247 KiB/s wr, 11 op/s 2023-09-10T10:42:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40980.926754 secs 2023-09-10T10:42:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40981.278141 secs 2023-09-10T10:42:30.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40981.278141 secs 2023-09-10T10:42:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40981.278141 secs 2023-09-10T10:42:32.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:32 smithi098 ceph-mon[103124]: pgmap v22015: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 247 KiB/s wr, 17 op/s 2023-09-10T10:42:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:32 smithi170 ceph-mon[102832]: pgmap v22015: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 247 KiB/s wr, 17 op/s 2023-09-10T10:42:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:32 smithi082 ceph-mon[98694]: pgmap v22015: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 247 KiB/s wr, 17 op/s 2023-09-10T10:42:34.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:34 smithi098 ceph-mon[103124]: pgmap v22016: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 247 KiB/s wr, 11 op/s 2023-09-10T10:42:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40984.698994 secs 2023-09-10T10:42:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:42:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:34.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40985.926811 secs 2023-09-10T10:42:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:34 smithi170 ceph-mon[102832]: pgmap v22016: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 247 KiB/s wr, 11 op/s 2023-09-10T10:42:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40984.698994 secs 2023-09-10T10:42:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:42:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40985.926811 secs 2023-09-10T10:42:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:34 smithi082 ceph-mon[98694]: pgmap v22016: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 247 KiB/s wr, 11 op/s 2023-09-10T10:42:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40984.698994 secs 2023-09-10T10:42:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:42:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40985.926811 secs 2023-09-10T10:42:35.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40986.278206 secs 2023-09-10T10:42:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40986.278206 secs 2023-09-10T10:42:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40986.278206 secs 2023-09-10T10:42:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:36 smithi098 ceph-mon[103124]: pgmap v22017: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 247 KiB/s wr, 11 op/s 2023-09-10T10:42:36.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:42:36.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:36 smithi170 ceph-mon[102832]: pgmap v22017: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 247 KiB/s wr, 11 op/s 2023-09-10T10:42:36.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:42:36.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:36 smithi082 ceph-mon[98694]: pgmap v22017: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 247 KiB/s wr, 11 op/s 2023-09-10T10:42:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:42:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:38.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:38 smithi170 ceph-mon[102832]: pgmap v22018: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 395 KiB/s wr, 17 op/s 2023-09-10T10:42:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:38 smithi082 ceph-mon[98694]: pgmap v22018: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 395 KiB/s wr, 17 op/s 2023-09-10T10:42:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:38 smithi098 ceph-mon[103124]: pgmap v22018: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 395 KiB/s wr, 17 op/s 2023-09-10T10:42:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40990.926889 secs 2023-09-10T10:42:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:42:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40990.926889 secs 2023-09-10T10:42:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:42:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40990.926889 secs 2023-09-10T10:42:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:42:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:40 smithi170 ceph-mon[102832]: pgmap v22019: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:42:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40991.278268 secs 2023-09-10T10:42:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:42:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:42:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:40 smithi082 ceph-mon[98694]: pgmap v22019: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:42:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40991.278268 secs 2023-09-10T10:42:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:42:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:42:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:40 smithi098 ceph-mon[103124]: pgmap v22019: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:42:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40991.278268 secs 2023-09-10T10:42:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:42:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:42:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:42 smithi170 ceph-mon[102832]: pgmap v22020: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 483 KiB/s wr, 17 op/s 2023-09-10T10:42:42.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:42 smithi082 ceph-mon[98694]: pgmap v22020: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 483 KiB/s wr, 17 op/s 2023-09-10T10:42:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:42 smithi098 ceph-mon[103124]: pgmap v22020: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 483 KiB/s wr, 17 op/s 2023-09-10T10:42:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:44 smithi170 ceph-mon[102832]: pgmap v22021: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 483 KiB/s wr, 12 op/s 2023-09-10T10:42:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40995.926949 secs 2023-09-10T10:42:44.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:44 smithi082 ceph-mon[98694]: pgmap v22021: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 483 KiB/s wr, 12 op/s 2023-09-10T10:42:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40995.926949 secs 2023-09-10T10:42:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:44 smithi098 ceph-mon[103124]: pgmap v22021: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 483 KiB/s wr, 12 op/s 2023-09-10T10:42:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40995.926949 secs 2023-09-10T10:42:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 40996.278330 secs 2023-09-10T10:42:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 40996.278330 secs 2023-09-10T10:42:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 40996.278330 secs 2023-09-10T10:42:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:46 smithi170 ceph-mon[102832]: pgmap v22022: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 483 KiB/s wr, 12 op/s 2023-09-10T10:42:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:42:46.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:46 smithi082 ceph-mon[98694]: pgmap v22022: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 483 KiB/s wr, 12 op/s 2023-09-10T10:42:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:42:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:46 smithi098 ceph-mon[103124]: pgmap v22022: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 483 KiB/s wr, 12 op/s 2023-09-10T10:42:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:42:46.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:48 smithi170 ceph-mon[102832]: pgmap v22023: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 808 KiB/s wr, 18 op/s 2023-09-10T10:42:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:48 smithi082 ceph-mon[98694]: pgmap v22023: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 808 KiB/s wr, 18 op/s 2023-09-10T10:42:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:48 smithi098 ceph-mon[103124]: pgmap v22023: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 808 KiB/s wr, 18 op/s 2023-09-10T10:42:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:49 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 40999.699239 secs 2023-09-10T10:42:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41000.927032 secs 2023-09-10T10:42:49.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:49 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 40999.699239 secs 2023-09-10T10:42:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41000.927032 secs 2023-09-10T10:42:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:49 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 40999.699239 secs 2023-09-10T10:42:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41000.927032 secs 2023-09-10T10:42:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:50 smithi170 ceph-mon[102832]: pgmap v22024: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 660 KiB/s wr, 13 op/s 2023-09-10T10:42:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41001.278394 secs 2023-09-10T10:42:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:42:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:50 smithi082 ceph-mon[98694]: pgmap v22024: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 660 KiB/s wr, 13 op/s 2023-09-10T10:42:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41001.278394 secs 2023-09-10T10:42:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:42:50.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:50 smithi098 ceph-mon[103124]: pgmap v22024: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 660 KiB/s wr, 13 op/s 2023-09-10T10:42:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41001.278394 secs 2023-09-10T10:42:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:42:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:51 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:42:51.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:51 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:42:51.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:51 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:51 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:42:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:51 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:42:51.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:51 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:51 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:42:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:51 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:42:51.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:51 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:52 smithi170 ceph-mon[102832]: pgmap v22025: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 964 KiB/s wr, 20 op/s 2023-09-10T10:42:52.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:52 smithi082 ceph-mon[98694]: pgmap v22025: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 964 KiB/s wr, 20 op/s 2023-09-10T10:42:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:52 smithi098 ceph-mon[103124]: pgmap v22025: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 964 KiB/s wr, 20 op/s 2023-09-10T10:42:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:54 smithi170 ceph-mon[102832]: pgmap v22026: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 630 KiB/s wr, 13 op/s 2023-09-10T10:42:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41004.699300 secs 2023-09-10T10:42:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41005.927109 secs 2023-09-10T10:42:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:54 smithi082 ceph-mon[98694]: pgmap v22026: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 630 KiB/s wr, 13 op/s 2023-09-10T10:42:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41004.699300 secs 2023-09-10T10:42:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41005.927109 secs 2023-09-10T10:42:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:54 smithi098 ceph-mon[103124]: pgmap v22026: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 630 KiB/s wr, 13 op/s 2023-09-10T10:42:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41004.699300 secs 2023-09-10T10:42:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41005.927109 secs 2023-09-10T10:42:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41006.278461 secs 2023-09-10T10:42:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41006.278461 secs 2023-09-10T10:42:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41006.278461 secs 2023-09-10T10:42:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:56 smithi170 ceph-mon[102832]: pgmap v22027: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 630 KiB/s wr, 13 op/s 2023-09-10T10:42:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:56 smithi082 ceph-mon[98694]: pgmap v22027: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 630 KiB/s wr, 13 op/s 2023-09-10T10:42:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:56 smithi098 ceph-mon[103124]: pgmap v22027: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 630 KiB/s wr, 13 op/s 2023-09-10T10:42:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:58 smithi170 ceph-mon[102832]: pgmap v22028: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 917 KiB/s wr, 20 op/s 2023-09-10T10:42:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:42:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:58 smithi082 ceph-mon[98694]: pgmap v22028: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 917 KiB/s wr, 20 op/s 2023-09-10T10:42:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:42:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:58 smithi098 ceph-mon[103124]: pgmap v22028: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 917 KiB/s wr, 20 op/s 2023-09-10T10:42:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:42:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:42:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41009.699370 secs 2023-09-10T10:42:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:42:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41010.927169 secs 2023-09-10T10:42:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41009.699370 secs 2023-09-10T10:42:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:42:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41010.927169 secs 2023-09-10T10:42:59.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41009.699370 secs 2023-09-10T10:42:59.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:42:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41010.927169 secs 2023-09-10T10:43:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:00 smithi170 ceph-mon[102832]: pgmap v22029: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 591 KiB/s wr, 13 op/s 2023-09-10T10:43:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41011.278489 secs 2023-09-10T10:43:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:43:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:00 smithi082 ceph-mon[98694]: pgmap v22029: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 591 KiB/s wr, 13 op/s 2023-09-10T10:43:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41011.278489 secs 2023-09-10T10:43:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:43:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:00 smithi098 ceph-mon[103124]: pgmap v22029: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 591 KiB/s wr, 13 op/s 2023-09-10T10:43:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41011.278489 secs 2023-09-10T10:43:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:43:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:01.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:01 smithi170 ceph-mon[102832]: pgmap v22030: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 889 KiB/s wr, 33 op/s 2023-09-10T10:43:01.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:01 smithi082 ceph-mon[98694]: pgmap v22030: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 889 KiB/s wr, 33 op/s 2023-09-10T10:43:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:01 smithi098 ceph-mon[103124]: pgmap v22030: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 889 KiB/s wr, 33 op/s 2023-09-10T10:43:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:03 smithi170 ceph-mon[102832]: pgmap v22031: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 585 KiB/s wr, 26 op/s 2023-09-10T10:43:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41014.699430 secs 2023-09-10T10:43:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:03 smithi082 ceph-mon[98694]: pgmap v22031: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 585 KiB/s wr, 26 op/s 2023-09-10T10:43:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41014.699430 secs 2023-09-10T10:43:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:03 smithi098 ceph-mon[103124]: pgmap v22031: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 585 KiB/s wr, 26 op/s 2023-09-10T10:43:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41014.699430 secs 2023-09-10T10:43:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41015.927233 secs 2023-09-10T10:43:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41015.927233 secs 2023-09-10T10:43:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41015.927233 secs 2023-09-10T10:43:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41016.278554 secs 2023-09-10T10:43:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:05 smithi170 ceph-mon[102832]: pgmap v22032: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 585 KiB/s wr, 26 op/s 2023-09-10T10:43:05.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:43:05.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:43:05.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:43:05.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:05 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:43:05.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41016.278554 secs 2023-09-10T10:43:05.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:05 smithi082 ceph-mon[98694]: pgmap v22032: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 585 KiB/s wr, 26 op/s 2023-09-10T10:43:05.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:43:05.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:43:05.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:43:05.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:05 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:43:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41016.278554 secs 2023-09-10T10:43:06.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:05 smithi098 ceph-mon[103124]: pgmap v22032: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 585 KiB/s wr, 26 op/s 2023-09-10T10:43:06.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:43:06.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:43:06.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:43:06.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:05 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:43:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:07 smithi170 ceph-mon[102832]: pgmap v22033: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 764 KiB/s wr, 34 op/s 2023-09-10T10:43:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:07 smithi082 ceph-mon[98694]: pgmap v22033: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 764 KiB/s wr, 34 op/s 2023-09-10T10:43:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:07 smithi098 ceph-mon[103124]: pgmap v22033: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 849 KiB/s rd, 764 KiB/s wr, 34 op/s 2023-09-10T10:43:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41019.699510 secs 2023-09-10T10:43:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41019.699510 secs 2023-09-10T10:43:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41019.699510 secs 2023-09-10T10:43:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:09 smithi170 ceph-mon[102832]: pgmap v22034: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 477 KiB/s wr, 27 op/s 2023-09-10T10:43:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:09 smithi170 ceph-mon[102832]: osdmap e1445: 12 total, 12 up, 12 in 2023-09-10T10:43:09.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41020.927333 secs 2023-09-10T10:43:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:09 smithi082 ceph-mon[98694]: pgmap v22034: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 477 KiB/s wr, 27 op/s 2023-09-10T10:43:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:09 smithi082 ceph-mon[98694]: osdmap e1445: 12 total, 12 up, 12 in 2023-09-10T10:43:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41020.927333 secs 2023-09-10T10:43:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:09 smithi098 ceph-mon[103124]: pgmap v22034: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 477 KiB/s wr, 27 op/s 2023-09-10T10:43:10.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:09 smithi098 ceph-mon[103124]: osdmap e1445: 12 total, 12 up, 12 in 2023-09-10T10:43:10.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41020.927333 secs 2023-09-10T10:43:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41021.278619 secs 2023-09-10T10:43:10.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:43:10.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41021.278619 secs 2023-09-10T10:43:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:43:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41021.278619 secs 2023-09-10T10:43:11.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:43:11.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:12.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:12 smithi098 ceph-mon[103124]: pgmap v22036: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 459 KiB/s wr, 17 op/s 2023-09-10T10:43:12.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:43:12.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:12 smithi170 ceph-mon[102832]: pgmap v22036: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 459 KiB/s wr, 17 op/s 2023-09-10T10:43:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:43:12.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:12 smithi082 ceph-mon[98694]: pgmap v22036: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 459 KiB/s wr, 17 op/s 2023-09-10T10:43:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:43:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:14 smithi170 ceph-mon[102832]: pgmap v22037: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 459 KiB/s wr, 17 op/s 2023-09-10T10:43:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41024.699570 secs 2023-09-10T10:43:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41025.927392 secs 2023-09-10T10:43:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:14 smithi082 ceph-mon[98694]: pgmap v22037: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 459 KiB/s wr, 17 op/s 2023-09-10T10:43:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41024.699570 secs 2023-09-10T10:43:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41025.927392 secs 2023-09-10T10:43:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:14 smithi098 ceph-mon[103124]: pgmap v22037: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 459 KiB/s wr, 17 op/s 2023-09-10T10:43:14.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41024.699570 secs 2023-09-10T10:43:14.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41025.927392 secs 2023-09-10T10:43:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41026.278685 secs 2023-09-10T10:43:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:15 smithi170 ceph-mon[102832]: osdmap e1446: 12 total, 12 up, 12 in 2023-09-10T10:43:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41026.278685 secs 2023-09-10T10:43:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:15 smithi082 ceph-mon[98694]: osdmap e1446: 12 total, 12 up, 12 in 2023-09-10T10:43:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41026.278685 secs 2023-09-10T10:43:15.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:15 smithi098 ceph-mon[103124]: osdmap e1446: 12 total, 12 up, 12 in 2023-09-10T10:43:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:16 smithi170 ceph-mon[102832]: pgmap v22038: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 459 KiB/s wr, 17 op/s 2023-09-10T10:43:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:16 smithi082 ceph-mon[98694]: pgmap v22038: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 459 KiB/s wr, 17 op/s 2023-09-10T10:43:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:16 smithi098 ceph-mon[103124]: pgmap v22038: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 459 KiB/s wr, 17 op/s 2023-09-10T10:43:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:18 smithi170 ceph-mon[102832]: pgmap v22040: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 533 KiB/s wr, 19 op/s 2023-09-10T10:43:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:18 smithi082 ceph-mon[98694]: pgmap v22040: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 533 KiB/s wr, 19 op/s 2023-09-10T10:43:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:18 smithi098 ceph-mon[103124]: pgmap v22040: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 533 KiB/s wr, 19 op/s 2023-09-10T10:43:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41029.699630 secs 2023-09-10T10:43:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41030.927448 secs 2023-09-10T10:43:19.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41029.699630 secs 2023-09-10T10:43:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41030.927448 secs 2023-09-10T10:43:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41029.699630 secs 2023-09-10T10:43:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41030.927448 secs 2023-09-10T10:43:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:20 smithi170 ceph-mon[102832]: pgmap v22041: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 429 KiB/s wr, 15 op/s 2023-09-10T10:43:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41031.278751 secs 2023-09-10T10:43:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:20 smithi082 ceph-mon[98694]: pgmap v22041: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 429 KiB/s wr, 15 op/s 2023-09-10T10:43:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41031.278751 secs 2023-09-10T10:43:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:20 smithi098 ceph-mon[103124]: pgmap v22041: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 429 KiB/s wr, 15 op/s 2023-09-10T10:43:20.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41031.278751 secs 2023-09-10T10:43:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:22 smithi170 ceph-mon[102832]: pgmap v22042: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T10:43:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:22 smithi082 ceph-mon[98694]: pgmap v22042: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T10:43:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:22 smithi098 ceph-mon[103124]: pgmap v22042: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T10:43:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:24 smithi170 ceph-mon[102832]: pgmap v22043: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T10:43:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41034.699690 secs 2023-09-10T10:43:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41035.927493 secs 2023-09-10T10:43:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:24 smithi082 ceph-mon[98694]: pgmap v22043: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T10:43:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41034.699690 secs 2023-09-10T10:43:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41035.927493 secs 2023-09-10T10:43:24.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:24 smithi098 ceph-mon[103124]: pgmap v22043: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T10:43:24.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41034.699690 secs 2023-09-10T10:43:24.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41035.927493 secs 2023-09-10T10:43:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41036.278814 secs 2023-09-10T10:43:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41036.278814 secs 2023-09-10T10:43:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41036.278814 secs 2023-09-10T10:43:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:26 smithi170 ceph-mon[102832]: pgmap v22044: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T10:43:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:26 smithi082 ceph-mon[98694]: pgmap v22044: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T10:43:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:26 smithi098 ceph-mon[103124]: pgmap v22044: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 14 op/s 2023-09-10T10:43:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:28 smithi170 ceph-mon[102832]: pgmap v22045: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 324 KiB/s wr, 19 op/s 2023-09-10T10:43:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:28 smithi082 ceph-mon[98694]: pgmap v22045: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 324 KiB/s wr, 19 op/s 2023-09-10T10:43:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:28 smithi098 ceph-mon[103124]: pgmap v22045: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 324 KiB/s wr, 19 op/s 2023-09-10T10:43:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41039.699750 secs 2023-09-10T10:43:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41040.927553 secs 2023-09-10T10:43:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41039.699750 secs 2023-09-10T10:43:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41040.927553 secs 2023-09-10T10:43:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41039.699750 secs 2023-09-10T10:43:29.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41040.927553 secs 2023-09-10T10:43:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:30 smithi170 ceph-mon[102832]: pgmap v22046: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T10:43:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41041.278879 secs 2023-09-10T10:43:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:30 smithi082 ceph-mon[98694]: pgmap v22046: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T10:43:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41041.278879 secs 2023-09-10T10:43:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:30 smithi098 ceph-mon[103124]: pgmap v22046: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 153 KiB/s wr, 11 op/s 2023-09-10T10:43:30.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41041.278879 secs 2023-09-10T10:43:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:32 smithi170 ceph-mon[102832]: pgmap v22047: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 16 op/s 2023-09-10T10:43:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:32 smithi082 ceph-mon[98694]: pgmap v22047: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 16 op/s 2023-09-10T10:43:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:32 smithi098 ceph-mon[103124]: pgmap v22047: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 16 op/s 2023-09-10T10:43:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:34 smithi170 ceph-mon[102832]: pgmap v22048: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T10:43:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41044.699900 secs 2023-09-10T10:43:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:43:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41045.927619 secs 2023-09-10T10:43:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:34 smithi082 ceph-mon[98694]: pgmap v22048: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T10:43:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41044.699900 secs 2023-09-10T10:43:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:43:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41045.927619 secs 2023-09-10T10:43:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:34 smithi098 ceph-mon[103124]: pgmap v22048: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T10:43:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41044.699900 secs 2023-09-10T10:43:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:43:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41045.927619 secs 2023-09-10T10:43:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41046.278947 secs 2023-09-10T10:43:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41046.278947 secs 2023-09-10T10:43:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41046.278947 secs 2023-09-10T10:43:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:36 smithi170 ceph-mon[102832]: pgmap v22049: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T10:43:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:43:36.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:36 smithi082 ceph-mon[98694]: pgmap v22049: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T10:43:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:43:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:36 smithi098 ceph-mon[103124]: pgmap v22049: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T10:43:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:43:36.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:38 smithi170 ceph-mon[102832]: pgmap v22050: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 417 KiB/s wr, 17 op/s 2023-09-10T10:43:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:38 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:43:38.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:38 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:38 smithi082 ceph-mon[98694]: pgmap v22050: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 417 KiB/s wr, 17 op/s 2023-09-10T10:43:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:38 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:43:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:38 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:38 smithi098 ceph-mon[103124]: pgmap v22050: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 417 KiB/s wr, 17 op/s 2023-09-10T10:43:38.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:38 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:43:38.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:38 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41050.927678 secs 2023-09-10T10:43:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:43:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41050.927678 secs 2023-09-10T10:43:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:43:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41050.927678 secs 2023-09-10T10:43:39.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:43:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:40 smithi170 ceph-mon[102832]: pgmap v22051: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T10:43:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41051.279012 secs 2023-09-10T10:43:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:43:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:43:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:40 smithi082 ceph-mon[98694]: pgmap v22051: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T10:43:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41051.279012 secs 2023-09-10T10:43:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:43:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:43:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:40 smithi098 ceph-mon[103124]: pgmap v22051: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 265 KiB/s wr, 11 op/s 2023-09-10T10:43:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41051.279012 secs 2023-09-10T10:43:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:43:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:43:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:42 smithi170 ceph-mon[102832]: pgmap v22052: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 484 KiB/s wr, 17 op/s 2023-09-10T10:43:42.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:42 smithi082 ceph-mon[98694]: pgmap v22052: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 484 KiB/s wr, 17 op/s 2023-09-10T10:43:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:42 smithi098 ceph-mon[103124]: pgmap v22052: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 484 KiB/s wr, 17 op/s 2023-09-10T10:43:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:44 smithi170 ceph-mon[102832]: pgmap v22053: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 483 KiB/s wr, 12 op/s 2023-09-10T10:43:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41055.927738 secs 2023-09-10T10:43:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:44 smithi082 ceph-mon[98694]: pgmap v22053: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 483 KiB/s wr, 12 op/s 2023-09-10T10:43:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41055.927738 secs 2023-09-10T10:43:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:44 smithi098 ceph-mon[103124]: pgmap v22053: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 483 KiB/s wr, 12 op/s 2023-09-10T10:43:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41055.927738 secs 2023-09-10T10:43:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41056.279087 secs 2023-09-10T10:43:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41056.279087 secs 2023-09-10T10:43:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41056.279087 secs 2023-09-10T10:43:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:46 smithi170 ceph-mon[102832]: pgmap v22054: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 483 KiB/s wr, 12 op/s 2023-09-10T10:43:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:46 smithi082 ceph-mon[98694]: pgmap v22054: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 483 KiB/s wr, 12 op/s 2023-09-10T10:43:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:46 smithi098 ceph-mon[103124]: pgmap v22054: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 483 KiB/s wr, 12 op/s 2023-09-10T10:43:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:48 smithi170 ceph-mon[102832]: pgmap v22055: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 810 KiB/s wr, 18 op/s 2023-09-10T10:43:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:48 smithi082 ceph-mon[98694]: pgmap v22055: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 810 KiB/s wr, 18 op/s 2023-09-10T10:43:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:48 smithi098 ceph-mon[103124]: pgmap v22055: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 810 KiB/s wr, 18 op/s 2023-09-10T10:43:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41060.927797 secs 2023-09-10T10:43:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41060.927797 secs 2023-09-10T10:43:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41060.927797 secs 2023-09-10T10:43:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:50 smithi170 ceph-mon[102832]: pgmap v22056: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 545 KiB/s wr, 12 op/s 2023-09-10T10:43:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41061.279148 secs 2023-09-10T10:43:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:43:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:50 smithi082 ceph-mon[98694]: pgmap v22056: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 545 KiB/s wr, 12 op/s 2023-09-10T10:43:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41061.279148 secs 2023-09-10T10:43:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:43:50.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:50 smithi098 ceph-mon[103124]: pgmap v22056: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 545 KiB/s wr, 12 op/s 2023-09-10T10:43:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41061.279148 secs 2023-09-10T10:43:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:43:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:52 smithi170 ceph-mon[102832]: pgmap v22057: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 818 KiB/s wr, 20 op/s 2023-09-10T10:43:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:43:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:52 smithi082 ceph-mon[98694]: pgmap v22057: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 818 KiB/s wr, 20 op/s 2023-09-10T10:43:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:43:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:52 smithi098 ceph-mon[103124]: pgmap v22057: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 818 KiB/s wr, 20 op/s 2023-09-10T10:43:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:43:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:54 smithi170 ceph-mon[102832]: pgmap v22058: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 599 KiB/s wr, 13 op/s 2023-09-10T10:43:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41065.927857 secs 2023-09-10T10:43:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:54 smithi082 ceph-mon[98694]: pgmap v22058: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 599 KiB/s wr, 13 op/s 2023-09-10T10:43:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41065.927857 secs 2023-09-10T10:43:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:54 smithi098 ceph-mon[103124]: pgmap v22058: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 599 KiB/s wr, 13 op/s 2023-09-10T10:43:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41065.927857 secs 2023-09-10T10:43:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41066.279216 secs 2023-09-10T10:43:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41066.279216 secs 2023-09-10T10:43:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41066.279216 secs 2023-09-10T10:43:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:56 smithi170 ceph-mon[102832]: pgmap v22059: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 599 KiB/s wr, 13 op/s 2023-09-10T10:43:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:56 smithi082 ceph-mon[98694]: pgmap v22059: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 599 KiB/s wr, 13 op/s 2023-09-10T10:43:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:56 smithi098 ceph-mon[103124]: pgmap v22059: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 599 KiB/s wr, 13 op/s 2023-09-10T10:43:57.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:57 smithi170 ceph-mon[102832]: pgmap v22060: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 919 KiB/s wr, 20 op/s 2023-09-10T10:43:57.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:57 smithi082 ceph-mon[98694]: pgmap v22060: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 919 KiB/s wr, 20 op/s 2023-09-10T10:43:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:57 smithi098 ceph-mon[103124]: pgmap v22060: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 919 KiB/s wr, 20 op/s 2023-09-10T10:43:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:43:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:43:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:43:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:43:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41069.700316 secs 2023-09-10T10:43:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:59 smithi170 ceph-mon[102832]: pgmap v22061: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 593 KiB/s wr, 14 op/s 2023-09-10T10:43:59.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:43:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41070.927917 secs 2023-09-10T10:43:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41069.700316 secs 2023-09-10T10:43:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:59 smithi082 ceph-mon[98694]: pgmap v22061: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 593 KiB/s wr, 14 op/s 2023-09-10T10:43:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:43:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41070.927917 secs 2023-09-10T10:43:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41069.700316 secs 2023-09-10T10:43:59.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:59 smithi098 ceph-mon[103124]: pgmap v22061: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 593 KiB/s wr, 14 op/s 2023-09-10T10:43:59.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:43:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41070.927917 secs 2023-09-10T10:44:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41071.279304 secs 2023-09-10T10:44:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:44:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:44:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41071.279304 secs 2023-09-10T10:44:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:44:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:44:00.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41071.279304 secs 2023-09-10T10:44:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:44:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:44:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:01.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:01 smithi170 ceph-mon[102832]: pgmap v22062: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 868 KiB/s wr, 22 op/s 2023-09-10T10:44:01.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:01 smithi082 ceph-mon[98694]: pgmap v22062: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 868 KiB/s wr, 22 op/s 2023-09-10T10:44:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:01 smithi098 ceph-mon[103124]: pgmap v22062: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 868 KiB/s wr, 22 op/s 2023-09-10T10:44:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:03 smithi170 ceph-mon[102832]: pgmap v22063: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 595 KiB/s wr, 14 op/s 2023-09-10T10:44:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41074.700378 secs 2023-09-10T10:44:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:03 smithi082 ceph-mon[98694]: pgmap v22063: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 595 KiB/s wr, 14 op/s 2023-09-10T10:44:03.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41074.700378 secs 2023-09-10T10:44:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:03 smithi098 ceph-mon[103124]: pgmap v22063: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 595 KiB/s wr, 14 op/s 2023-09-10T10:44:04.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41074.700378 secs 2023-09-10T10:44:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41075.927975 secs 2023-09-10T10:44:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41075.927975 secs 2023-09-10T10:44:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41075.927975 secs 2023-09-10T10:44:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41076.279371 secs 2023-09-10T10:44:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:05 smithi170 ceph-mon[102832]: pgmap v22064: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 595 KiB/s wr, 14 op/s 2023-09-10T10:44:05.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41076.279371 secs 2023-09-10T10:44:05.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:05 smithi082 ceph-mon[98694]: pgmap v22064: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 595 KiB/s wr, 14 op/s 2023-09-10T10:44:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41076.279371 secs 2023-09-10T10:44:06.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:05 smithi098 ceph-mon[103124]: pgmap v22064: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 595 KiB/s wr, 14 op/s 2023-09-10T10:44:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:44:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:44:06.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:44:06.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:06 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:44:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:44:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:44:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:44:06.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:06 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:44:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:44:07.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:44:07.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:44:07.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:06 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:44:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:08 smithi170 ceph-mon[102832]: pgmap v22065: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 787 KiB/s wr, 34 op/s 2023-09-10T10:44:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:08 smithi082 ceph-mon[98694]: pgmap v22065: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 787 KiB/s wr, 34 op/s 2023-09-10T10:44:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:08 smithi098 ceph-mon[103124]: pgmap v22065: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 787 KiB/s wr, 34 op/s 2023-09-10T10:44:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41079.700478 secs 2023-09-10T10:44:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:09 smithi170 ceph-mon[102832]: osdmap e1447: 12 total, 12 up, 12 in 2023-09-10T10:44:09.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41080.928033 secs 2023-09-10T10:44:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41079.700478 secs 2023-09-10T10:44:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:09 smithi082 ceph-mon[98694]: osdmap e1447: 12 total, 12 up, 12 in 2023-09-10T10:44:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41080.928033 secs 2023-09-10T10:44:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41079.700478 secs 2023-09-10T10:44:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:09 smithi098 ceph-mon[103124]: osdmap e1447: 12 total, 12 up, 12 in 2023-09-10T10:44:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41080.928033 secs 2023-09-10T10:44:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:10 smithi170 ceph-mon[102832]: pgmap v22067: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 561 KiB/s wr, 33 op/s 2023-09-10T10:44:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41081.279436 secs 2023-09-10T10:44:10.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:44:10.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:10 smithi082 ceph-mon[98694]: pgmap v22067: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 561 KiB/s wr, 33 op/s 2023-09-10T10:44:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41081.279436 secs 2023-09-10T10:44:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:44:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:10 smithi098 ceph-mon[103124]: pgmap v22067: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 561 KiB/s wr, 33 op/s 2023-09-10T10:44:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41081.279436 secs 2023-09-10T10:44:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:44:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:12 smithi170 ceph-mon[102832]: pgmap v22068: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 498 KiB/s wr, 32 op/s 2023-09-10T10:44:12.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:44:12.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:12 smithi082 ceph-mon[98694]: pgmap v22068: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 498 KiB/s wr, 32 op/s 2023-09-10T10:44:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:44:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:12 smithi098 ceph-mon[103124]: pgmap v22068: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 498 KiB/s wr, 32 op/s 2023-09-10T10:44:12.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:44:12.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:14 smithi170 ceph-mon[102832]: pgmap v22069: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 498 KiB/s wr, 32 op/s 2023-09-10T10:44:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41084.700539 secs 2023-09-10T10:44:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41085.928092 secs 2023-09-10T10:44:14.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:14 smithi082 ceph-mon[98694]: pgmap v22069: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 498 KiB/s wr, 32 op/s 2023-09-10T10:44:14.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41084.700539 secs 2023-09-10T10:44:14.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41085.928092 secs 2023-09-10T10:44:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:14 smithi098 ceph-mon[103124]: pgmap v22069: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 498 KiB/s wr, 32 op/s 2023-09-10T10:44:14.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41084.700539 secs 2023-09-10T10:44:14.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41085.928092 secs 2023-09-10T10:44:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41086.279511 secs 2023-09-10T10:44:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:15 smithi170 ceph-mon[102832]: osdmap e1448: 12 total, 12 up, 12 in 2023-09-10T10:44:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41086.279511 secs 2023-09-10T10:44:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:15 smithi082 ceph-mon[98694]: osdmap e1448: 12 total, 12 up, 12 in 2023-09-10T10:44:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41086.279511 secs 2023-09-10T10:44:15.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:15 smithi098 ceph-mon[103124]: osdmap e1448: 12 total, 12 up, 12 in 2023-09-10T10:44:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:16 smithi170 ceph-mon[102832]: pgmap v22070: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 498 KiB/s wr, 32 op/s 2023-09-10T10:44:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:16 smithi082 ceph-mon[98694]: pgmap v22070: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 498 KiB/s wr, 32 op/s 2023-09-10T10:44:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:16 smithi098 ceph-mon[103124]: pgmap v22070: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 498 KiB/s wr, 32 op/s 2023-09-10T10:44:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:18 smithi170 ceph-mon[102832]: pgmap v22072: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 502 KiB/s wr, 19 op/s 2023-09-10T10:44:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:44:18.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:18 smithi082 ceph-mon[98694]: pgmap v22072: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 502 KiB/s wr, 19 op/s 2023-09-10T10:44:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:44:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:18 smithi098 ceph-mon[103124]: pgmap v22072: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 502 KiB/s wr, 19 op/s 2023-09-10T10:44:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:44:18.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41089.700601 secs 2023-09-10T10:44:19.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41090.928148 secs 2023-09-10T10:44:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41089.700601 secs 2023-09-10T10:44:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41090.928148 secs 2023-09-10T10:44:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41089.700601 secs 2023-09-10T10:44:19.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41090.928148 secs 2023-09-10T10:44:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:20 smithi170 ceph-mon[102832]: pgmap v22073: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 414 KiB/s wr, 15 op/s 2023-09-10T10:44:20.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41091.279576 secs 2023-09-10T10:44:20.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:44:20.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:20 smithi082 ceph-mon[98694]: pgmap v22073: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 414 KiB/s wr, 15 op/s 2023-09-10T10:44:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41091.279576 secs 2023-09-10T10:44:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:44:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:20 smithi098 ceph-mon[103124]: pgmap v22073: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 414 KiB/s wr, 15 op/s 2023-09-10T10:44:20.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41091.279576 secs 2023-09-10T10:44:20.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:44:20.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:22 smithi170 ceph-mon[102832]: pgmap v22074: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T10:44:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:22 smithi082 ceph-mon[98694]: pgmap v22074: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T10:44:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:22 smithi098 ceph-mon[103124]: pgmap v22074: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T10:44:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:24 smithi170 ceph-mon[102832]: pgmap v22075: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T10:44:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41094.700664 secs 2023-09-10T10:44:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41095.928209 secs 2023-09-10T10:44:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:24 smithi082 ceph-mon[98694]: pgmap v22075: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T10:44:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41094.700664 secs 2023-09-10T10:44:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41095.928209 secs 2023-09-10T10:44:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:24 smithi098 ceph-mon[103124]: pgmap v22075: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T10:44:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41094.700664 secs 2023-09-10T10:44:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41095.928209 secs 2023-09-10T10:44:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41096.279641 secs 2023-09-10T10:44:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41096.279641 secs 2023-09-10T10:44:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41096.279641 secs 2023-09-10T10:44:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:26 smithi170 ceph-mon[102832]: pgmap v22076: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T10:44:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:26 smithi082 ceph-mon[98694]: pgmap v22076: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T10:44:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:26 smithi098 ceph-mon[103124]: pgmap v22076: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 393 KiB/s wr, 13 op/s 2023-09-10T10:44:27.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:27 smithi170 ceph-mon[102832]: pgmap v22077: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 348 KiB/s wr, 13 op/s 2023-09-10T10:44:27.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:27 smithi082 ceph-mon[98694]: pgmap v22077: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 348 KiB/s wr, 13 op/s 2023-09-10T10:44:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:27 smithi098 ceph-mon[103124]: pgmap v22077: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 348 KiB/s wr, 13 op/s 2023-09-10T10:44:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41099.700723 secs 2023-09-10T10:44:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41099.700723 secs 2023-09-10T10:44:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41099.700723 secs 2023-09-10T10:44:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:29 smithi098 ceph-mon[103124]: pgmap v22078: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 327 KiB/s wr, 11 op/s 2023-09-10T10:44:29.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41100.928297 secs 2023-09-10T10:44:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:29 smithi170 ceph-mon[102832]: pgmap v22078: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 327 KiB/s wr, 11 op/s 2023-09-10T10:44:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41100.928297 secs 2023-09-10T10:44:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:29 smithi082 ceph-mon[98694]: pgmap v22078: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 327 KiB/s wr, 11 op/s 2023-09-10T10:44:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41100.928297 secs 2023-09-10T10:44:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41101.279703 secs 2023-09-10T10:44:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41101.279703 secs 2023-09-10T10:44:30.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41101.279703 secs 2023-09-10T10:44:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:31 smithi170 ceph-mon[102832]: pgmap v22079: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 328 KiB/s wr, 16 op/s 2023-09-10T10:44:31.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:31 smithi082 ceph-mon[98694]: pgmap v22079: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 328 KiB/s wr, 16 op/s 2023-09-10T10:44:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:31 smithi098 ceph-mon[103124]: pgmap v22079: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 328 KiB/s wr, 16 op/s 2023-09-10T10:44:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:33 smithi170 ceph-mon[102832]: pgmap v22080: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 10 op/s 2023-09-10T10:44:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41104.700823 secs 2023-09-10T10:44:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:33 smithi082 ceph-mon[98694]: pgmap v22080: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 10 op/s 2023-09-10T10:44:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41104.700823 secs 2023-09-10T10:44:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:33 smithi098 ceph-mon[103124]: pgmap v22080: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 10 op/s 2023-09-10T10:44:34.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41104.700823 secs 2023-09-10T10:44:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41105.928399 secs 2023-09-10T10:44:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41105.928399 secs 2023-09-10T10:44:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41105.928399 secs 2023-09-10T10:44:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41106.279769 secs 2023-09-10T10:44:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:35 smithi170 ceph-mon[102832]: pgmap v22081: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 10 op/s 2023-09-10T10:44:35.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41106.279769 secs 2023-09-10T10:44:35.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:35 smithi082 ceph-mon[98694]: pgmap v22081: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 10 op/s 2023-09-10T10:44:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41106.279769 secs 2023-09-10T10:44:36.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:35 smithi098 ceph-mon[103124]: pgmap v22081: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 121 KiB/s wr, 10 op/s 2023-09-10T10:44:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:37 smithi170 ceph-mon[102832]: pgmap v22082: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 270 KiB/s wr, 16 op/s 2023-09-10T10:44:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:37 smithi082 ceph-mon[98694]: pgmap v22082: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 270 KiB/s wr, 16 op/s 2023-09-10T10:44:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:37 smithi098 ceph-mon[103124]: pgmap v22082: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 270 KiB/s wr, 16 op/s 2023-09-10T10:44:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:39 smithi170 ceph-mon[102832]: pgmap v22083: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 10 op/s 2023-09-10T10:44:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41110.928456 secs 2023-09-10T10:44:39.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:44:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:39 smithi082 ceph-mon[98694]: pgmap v22083: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 10 op/s 2023-09-10T10:44:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41110.928456 secs 2023-09-10T10:44:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:44:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:39 smithi098 ceph-mon[103124]: pgmap v22083: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 10 op/s 2023-09-10T10:44:40.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41110.928456 secs 2023-09-10T10:44:40.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:44:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41111.279834 secs 2023-09-10T10:44:40.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:44:40.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:44:40.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41111.279834 secs 2023-09-10T10:44:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:44:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:44:40.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41111.279834 secs 2023-09-10T10:44:41.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:44:41.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:44:41.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:42 smithi170 ceph-mon[102832]: pgmap v22084: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 384 KiB/s wr, 16 op/s 2023-09-10T10:44:42.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:42 smithi082 ceph-mon[98694]: pgmap v22084: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 384 KiB/s wr, 16 op/s 2023-09-10T10:44:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:42 smithi098 ceph-mon[103124]: pgmap v22084: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 384 KiB/s wr, 16 op/s 2023-09-10T10:44:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:44 smithi170 ceph-mon[102832]: pgmap v22085: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 384 KiB/s wr, 11 op/s 2023-09-10T10:44:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41115.928494 secs 2023-09-10T10:44:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:44 smithi082 ceph-mon[98694]: pgmap v22085: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 384 KiB/s wr, 11 op/s 2023-09-10T10:44:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41115.928494 secs 2023-09-10T10:44:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:44 smithi098 ceph-mon[103124]: pgmap v22085: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 384 KiB/s wr, 11 op/s 2023-09-10T10:44:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41115.928494 secs 2023-09-10T10:44:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41116.279898 secs 2023-09-10T10:44:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41116.279898 secs 2023-09-10T10:44:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41116.279898 secs 2023-09-10T10:44:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:46 smithi170 ceph-mon[102832]: pgmap v22086: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 384 KiB/s wr, 11 op/s 2023-09-10T10:44:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:44:46.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:46 smithi082 ceph-mon[98694]: pgmap v22086: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 384 KiB/s wr, 11 op/s 2023-09-10T10:44:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:44:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:46.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:46 smithi098 ceph-mon[103124]: pgmap v22086: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 384 KiB/s wr, 11 op/s 2023-09-10T10:44:46.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:44:46.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:48.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:48 smithi170 ceph-mon[102832]: pgmap v22087: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 626 KiB/s wr, 17 op/s 2023-09-10T10:44:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:48 smithi082 ceph-mon[98694]: pgmap v22087: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 626 KiB/s wr, 17 op/s 2023-09-10T10:44:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:48 smithi098 ceph-mon[103124]: pgmap v22087: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 626 KiB/s wr, 17 op/s 2023-09-10T10:44:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41120.928555 secs 2023-09-10T10:44:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41120.928555 secs 2023-09-10T10:44:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41120.928555 secs 2023-09-10T10:44:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:50 smithi170 ceph-mon[102832]: pgmap v22088: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 478 KiB/s wr, 12 op/s 2023-09-10T10:44:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41121.279961 secs 2023-09-10T10:44:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:44:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:50 smithi082 ceph-mon[98694]: pgmap v22088: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 478 KiB/s wr, 12 op/s 2023-09-10T10:44:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41121.279961 secs 2023-09-10T10:44:50.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:44:50.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:50 smithi098 ceph-mon[103124]: pgmap v22088: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 478 KiB/s wr, 12 op/s 2023-09-10T10:44:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41121.279961 secs 2023-09-10T10:44:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:44:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:52 smithi170 ceph-mon[102832]: pgmap v22089: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 639 KiB/s wr, 19 op/s 2023-09-10T10:44:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:44:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:52 smithi082 ceph-mon[98694]: pgmap v22089: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 639 KiB/s wr, 19 op/s 2023-09-10T10:44:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:44:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:52 smithi098 ceph-mon[103124]: pgmap v22089: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 639 KiB/s wr, 19 op/s 2023-09-10T10:44:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:44:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:54 smithi170 ceph-mon[102832]: pgmap v22090: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 404 KiB/s wr, 13 op/s 2023-09-10T10:44:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41124.701114 secs 2023-09-10T10:44:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:44:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41125.928614 secs 2023-09-10T10:44:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:54 smithi082 ceph-mon[98694]: pgmap v22090: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 404 KiB/s wr, 13 op/s 2023-09-10T10:44:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41124.701114 secs 2023-09-10T10:44:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:44:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:54.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41125.928614 secs 2023-09-10T10:44:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:54 smithi098 ceph-mon[103124]: pgmap v22090: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 404 KiB/s wr, 13 op/s 2023-09-10T10:44:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41124.701114 secs 2023-09-10T10:44:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:44:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41125.928614 secs 2023-09-10T10:44:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41126.280084 secs 2023-09-10T10:44:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41126.280084 secs 2023-09-10T10:44:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41126.280084 secs 2023-09-10T10:44:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:56 smithi170 ceph-mon[102832]: pgmap v22091: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 404 KiB/s wr, 13 op/s 2023-09-10T10:44:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:44:56.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:44:56.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:56 smithi082 ceph-mon[98694]: pgmap v22091: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 404 KiB/s wr, 13 op/s 2023-09-10T10:44:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:44:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:44:56.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:56 smithi098 ceph-mon[103124]: pgmap v22091: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 404 KiB/s wr, 13 op/s 2023-09-10T10:44:56.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:44:56.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:44:56.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:58 smithi170 ceph-mon[102832]: pgmap v22092: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 870 KiB/s wr, 21 op/s 2023-09-10T10:44:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:44:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:58 smithi082 ceph-mon[98694]: pgmap v22092: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 870 KiB/s wr, 21 op/s 2023-09-10T10:44:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:44:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:58 smithi098 ceph-mon[103124]: pgmap v22092: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 870 KiB/s wr, 21 op/s 2023-09-10T10:44:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:44:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:44:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41129.701172 secs 2023-09-10T10:44:59.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:44:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41130.928679 secs 2023-09-10T10:44:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41129.701172 secs 2023-09-10T10:44:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:44:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41130.928679 secs 2023-09-10T10:44:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41129.701172 secs 2023-09-10T10:44:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:44:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41130.928679 secs 2023-09-10T10:45:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:00 smithi170 ceph-mon[102832]: pgmap v22093: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 627 KiB/s wr, 15 op/s 2023-09-10T10:45:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41131.280149 secs 2023-09-10T10:45:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:45:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:00 smithi082 ceph-mon[98694]: pgmap v22093: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 627 KiB/s wr, 15 op/s 2023-09-10T10:45:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41131.280149 secs 2023-09-10T10:45:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:45:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:00 smithi098 ceph-mon[103124]: pgmap v22093: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 627 KiB/s wr, 15 op/s 2023-09-10T10:45:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41131.280149 secs 2023-09-10T10:45:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:45:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:02.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:02 smithi170 ceph-mon[102832]: pgmap v22094: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 662 KiB/s wr, 22 op/s 2023-09-10T10:45:02.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:02 smithi082 ceph-mon[98694]: pgmap v22094: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 662 KiB/s wr, 22 op/s 2023-09-10T10:45:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:02 smithi098 ceph-mon[103124]: pgmap v22094: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 662 KiB/s wr, 22 op/s 2023-09-10T10:45:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:04 smithi170 ceph-mon[102832]: pgmap v22095: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 500 KiB/s wr, 15 op/s 2023-09-10T10:45:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41134.701232 secs 2023-09-10T10:45:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:45:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41135.928782 secs 2023-09-10T10:45:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:04 smithi082 ceph-mon[98694]: pgmap v22095: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 500 KiB/s wr, 15 op/s 2023-09-10T10:45:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41134.701232 secs 2023-09-10T10:45:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:45:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41135.928782 secs 2023-09-10T10:45:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:04 smithi098 ceph-mon[103124]: pgmap v22095: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 500 KiB/s wr, 15 op/s 2023-09-10T10:45:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41134.701232 secs 2023-09-10T10:45:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:45:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41135.928782 secs 2023-09-10T10:45:05.683 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41136.280214 secs 2023-09-10T10:45:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41136.280214 secs 2023-09-10T10:45:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41136.280214 secs 2023-09-10T10:45:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:06 smithi170 ceph-mon[102832]: pgmap v22096: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 500 KiB/s wr, 15 op/s 2023-09-10T10:45:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:45:06.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:06 smithi082 ceph-mon[98694]: pgmap v22096: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 500 KiB/s wr, 15 op/s 2023-09-10T10:45:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:45:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:06 smithi098 ceph-mon[103124]: pgmap v22096: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 500 KiB/s wr, 15 op/s 2023-09-10T10:45:06.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:45:06.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:07.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:45:07.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:45:07.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:45:07.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:45:07.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:45:07.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:45:07.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:45:07.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:45:07.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:45:07.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:45:07.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:45:07.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:45:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:08 smithi170 ceph-mon[102832]: pgmap v22097: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 38 op/s 2023-09-10T10:45:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:08 smithi082 ceph-mon[98694]: pgmap v22097: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 38 op/s 2023-09-10T10:45:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:08 smithi098 ceph-mon[103124]: pgmap v22097: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 38 op/s 2023-09-10T10:45:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41139.701328 secs 2023-09-10T10:45:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:09 smithi170 ceph-mon[102832]: osdmap e1449: 12 total, 12 up, 12 in 2023-09-10T10:45:09.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41140.928842 secs 2023-09-10T10:45:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41139.701328 secs 2023-09-10T10:45:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:09 smithi082 ceph-mon[98694]: osdmap e1449: 12 total, 12 up, 12 in 2023-09-10T10:45:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41140.928842 secs 2023-09-10T10:45:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41139.701328 secs 2023-09-10T10:45:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:09 smithi098 ceph-mon[103124]: osdmap e1449: 12 total, 12 up, 12 in 2023-09-10T10:45:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41140.928842 secs 2023-09-10T10:45:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:10 smithi170 ceph-mon[102832]: pgmap v22099: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1.0 MiB/s wr, 35 op/s 2023-09-10T10:45:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41141.280280 secs 2023-09-10T10:45:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:10 smithi082 ceph-mon[98694]: pgmap v22099: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1.0 MiB/s wr, 35 op/s 2023-09-10T10:45:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41141.280280 secs 2023-09-10T10:45:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:10 smithi098 ceph-mon[103124]: pgmap v22099: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1.0 MiB/s wr, 35 op/s 2023-09-10T10:45:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41141.280280 secs 2023-09-10T10:45:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:12 smithi170 ceph-mon[102832]: pgmap v22100: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 34 op/s 2023-09-10T10:45:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:12 smithi082 ceph-mon[98694]: pgmap v22100: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 34 op/s 2023-09-10T10:45:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:12 smithi098 ceph-mon[103124]: pgmap v22100: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 34 op/s 2023-09-10T10:45:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:13 smithi170 ceph-mon[102832]: pgmap v22101: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 34 op/s 2023-09-10T10:45:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:13 smithi082 ceph-mon[98694]: pgmap v22101: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 34 op/s 2023-09-10T10:45:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:13 smithi098 ceph-mon[103124]: pgmap v22101: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 34 op/s 2023-09-10T10:45:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41144.701409 secs 2023-09-10T10:45:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41145.928905 secs 2023-09-10T10:45:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41144.701409 secs 2023-09-10T10:45:14.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41145.928905 secs 2023-09-10T10:45:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41144.701409 secs 2023-09-10T10:45:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41145.928905 secs 2023-09-10T10:45:15.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41146.280341 secs 2023-09-10T10:45:15.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:15 smithi082 ceph-mon[98694]: pgmap v22102: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 34 op/s 2023-09-10T10:45:15.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:15 smithi082 ceph-mon[98694]: osdmap e1450: 12 total, 12 up, 12 in 2023-09-10T10:45:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41146.280341 secs 2023-09-10T10:45:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:15 smithi098 ceph-mon[103124]: pgmap v22102: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 34 op/s 2023-09-10T10:45:15.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:15 smithi098 ceph-mon[103124]: osdmap e1450: 12 total, 12 up, 12 in 2023-09-10T10:45:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41146.280341 secs 2023-09-10T10:45:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:15 smithi170 ceph-mon[102832]: pgmap v22102: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 34 op/s 2023-09-10T10:45:15.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:15 smithi170 ceph-mon[102832]: osdmap e1450: 12 total, 12 up, 12 in 2023-09-10T10:45:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:17 smithi170 ceph-mon[102832]: pgmap v22104: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 437 KiB/s wr, 16 op/s 2023-09-10T10:45:17.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:17 smithi082 ceph-mon[98694]: pgmap v22104: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 437 KiB/s wr, 16 op/s 2023-09-10T10:45:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:17 smithi098 ceph-mon[103124]: pgmap v22104: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 437 KiB/s wr, 16 op/s 2023-09-10T10:45:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41149.701448 secs 2023-09-10T10:45:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41149.701448 secs 2023-09-10T10:45:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41149.701448 secs 2023-09-10T10:45:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:19 smithi170 ceph-mon[102832]: pgmap v22105: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 359 KiB/s wr, 13 op/s 2023-09-10T10:45:19.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41150.928962 secs 2023-09-10T10:45:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:19 smithi082 ceph-mon[98694]: pgmap v22105: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 359 KiB/s wr, 13 op/s 2023-09-10T10:45:19.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41150.928962 secs 2023-09-10T10:45:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:19 smithi098 ceph-mon[103124]: pgmap v22105: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 359 KiB/s wr, 13 op/s 2023-09-10T10:45:20.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41150.928962 secs 2023-09-10T10:45:20.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41151.280405 secs 2023-09-10T10:45:20.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41151.280405 secs 2023-09-10T10:45:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41151.280405 secs 2023-09-10T10:45:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:22 smithi170 ceph-mon[102832]: pgmap v22106: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T10:45:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:22 smithi082 ceph-mon[98694]: pgmap v22106: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T10:45:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:22 smithi098 ceph-mon[103124]: pgmap v22106: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T10:45:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:24 smithi170 ceph-mon[102832]: pgmap v22107: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T10:45:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41154.701507 secs 2023-09-10T10:45:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41155.929042 secs 2023-09-10T10:45:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:24 smithi082 ceph-mon[98694]: pgmap v22107: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T10:45:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41154.701507 secs 2023-09-10T10:45:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41155.929042 secs 2023-09-10T10:45:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:24 smithi098 ceph-mon[103124]: pgmap v22107: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T10:45:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41154.701507 secs 2023-09-10T10:45:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41155.929042 secs 2023-09-10T10:45:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41156.280472 secs 2023-09-10T10:45:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41156.280472 secs 2023-09-10T10:45:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41156.280472 secs 2023-09-10T10:45:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:26 smithi170 ceph-mon[102832]: pgmap v22108: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T10:45:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:26 smithi082 ceph-mon[98694]: pgmap v22108: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T10:45:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:26 smithi098 ceph-mon[103124]: pgmap v22108: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 146 KiB/s wr, 12 op/s 2023-09-10T10:45:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:28 smithi170 ceph-mon[102832]: pgmap v22109: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-09-10T10:45:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:28 smithi082 ceph-mon[98694]: pgmap v22109: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-09-10T10:45:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:28 smithi098 ceph-mon[103124]: pgmap v22109: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-09-10T10:45:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41159.701571 secs 2023-09-10T10:45:29.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41160.929099 secs 2023-09-10T10:45:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41159.701571 secs 2023-09-10T10:45:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41160.929099 secs 2023-09-10T10:45:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41159.701571 secs 2023-09-10T10:45:29.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41160.929099 secs 2023-09-10T10:45:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:30 smithi170 ceph-mon[102832]: pgmap v22110: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 10 op/s 2023-09-10T10:45:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41161.280484 secs 2023-09-10T10:45:30.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:45:30.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:30 smithi082 ceph-mon[98694]: pgmap v22110: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 10 op/s 2023-09-10T10:45:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41161.280484 secs 2023-09-10T10:45:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:45:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:30 smithi098 ceph-mon[103124]: pgmap v22110: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 10 op/s 2023-09-10T10:45:30.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41161.280484 secs 2023-09-10T10:45:30.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:45:30.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:32 smithi170 ceph-mon[102832]: pgmap v22111: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 15 op/s 2023-09-10T10:45:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:45:32.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:32 smithi082 ceph-mon[98694]: pgmap v22111: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 15 op/s 2023-09-10T10:45:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:45:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:32 smithi098 ceph-mon[103124]: pgmap v22111: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 153 KiB/s wr, 15 op/s 2023-09-10T10:45:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:45:32.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:34 smithi170 ceph-mon[102832]: pgmap v22112: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 153 KiB/s wr, 10 op/s 2023-09-10T10:45:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41164.701655 secs 2023-09-10T10:45:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41165.929159 secs 2023-09-10T10:45:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:34 smithi082 ceph-mon[98694]: pgmap v22112: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 153 KiB/s wr, 10 op/s 2023-09-10T10:45:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41164.701655 secs 2023-09-10T10:45:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41165.929159 secs 2023-09-10T10:45:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:34 smithi098 ceph-mon[103124]: pgmap v22112: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 153 KiB/s wr, 10 op/s 2023-09-10T10:45:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41164.701655 secs 2023-09-10T10:45:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41165.929159 secs 2023-09-10T10:45:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41166.280547 secs 2023-09-10T10:45:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41166.280547 secs 2023-09-10T10:45:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41166.280547 secs 2023-09-10T10:45:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:36 smithi170 ceph-mon[102832]: pgmap v22113: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 153 KiB/s wr, 10 op/s 2023-09-10T10:45:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:36 smithi082 ceph-mon[98694]: pgmap v22113: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 153 KiB/s wr, 10 op/s 2023-09-10T10:45:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:36 smithi098 ceph-mon[103124]: pgmap v22113: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 153 KiB/s wr, 10 op/s 2023-09-10T10:45:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:38 smithi170 ceph-mon[102832]: pgmap v22114: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 229 KiB/s wr, 16 op/s 2023-09-10T10:45:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:38 smithi082 ceph-mon[98694]: pgmap v22114: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 229 KiB/s wr, 16 op/s 2023-09-10T10:45:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:38 smithi098 ceph-mon[103124]: pgmap v22114: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 229 KiB/s wr, 16 op/s 2023-09-10T10:45:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41170.929219 secs 2023-09-10T10:45:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:45:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41170.929219 secs 2023-09-10T10:45:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:45:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41170.929219 secs 2023-09-10T10:45:39.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:45:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:40 smithi170 ceph-mon[102832]: pgmap v22115: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 229 KiB/s wr, 10 op/s 2023-09-10T10:45:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41171.280611 secs 2023-09-10T10:45:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:45:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:45:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:40 smithi082 ceph-mon[98694]: pgmap v22115: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 229 KiB/s wr, 10 op/s 2023-09-10T10:45:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41171.280611 secs 2023-09-10T10:45:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:45:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:45:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:40 smithi098 ceph-mon[103124]: pgmap v22115: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 229 KiB/s wr, 10 op/s 2023-09-10T10:45:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41171.280611 secs 2023-09-10T10:45:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:45:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:45:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:42 smithi170 ceph-mon[102832]: pgmap v22116: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 304 KiB/s wr, 16 op/s 2023-09-10T10:45:42.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:42 smithi082 ceph-mon[98694]: pgmap v22116: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 304 KiB/s wr, 16 op/s 2023-09-10T10:45:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:42 smithi098 ceph-mon[103124]: pgmap v22116: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 304 KiB/s wr, 16 op/s 2023-09-10T10:45:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:44 smithi170 ceph-mon[102832]: pgmap v22117: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T10:45:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:45:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41175.929280 secs 2023-09-10T10:45:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:44 smithi082 ceph-mon[98694]: pgmap v22117: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T10:45:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:45:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41175.929280 secs 2023-09-10T10:45:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:44 smithi098 ceph-mon[103124]: pgmap v22117: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T10:45:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:45:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:44.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41175.929280 secs 2023-09-10T10:45:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41176.280673 secs 2023-09-10T10:45:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41176.280673 secs 2023-09-10T10:45:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41176.280673 secs 2023-09-10T10:45:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:46 smithi170 ceph-mon[102832]: pgmap v22118: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T10:45:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:45:46.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:46 smithi082 ceph-mon[98694]: pgmap v22118: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T10:45:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:45:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:46 smithi098 ceph-mon[103124]: pgmap v22118: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 152 KiB/s wr, 11 op/s 2023-09-10T10:45:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:45:46.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:48 smithi170 ceph-mon[102832]: pgmap v22119: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 666 KiB/s wr, 18 op/s 2023-09-10T10:45:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:48 smithi082 ceph-mon[98694]: pgmap v22119: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 666 KiB/s wr, 18 op/s 2023-09-10T10:45:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:48 smithi098 ceph-mon[103124]: pgmap v22119: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 666 KiB/s wr, 18 op/s 2023-09-10T10:45:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41180.929339 secs 2023-09-10T10:45:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41180.929339 secs 2023-09-10T10:45:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41180.929339 secs 2023-09-10T10:45:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:50 smithi170 ceph-mon[102832]: pgmap v22120: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 590 KiB/s wr, 12 op/s 2023-09-10T10:45:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41181.280739 secs 2023-09-10T10:45:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:45:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:50 smithi082 ceph-mon[98694]: pgmap v22120: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 590 KiB/s wr, 12 op/s 2023-09-10T10:45:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41181.280739 secs 2023-09-10T10:45:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:45:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:50 smithi098 ceph-mon[103124]: pgmap v22120: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 590 KiB/s wr, 12 op/s 2023-09-10T10:45:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41181.280739 secs 2023-09-10T10:45:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:45:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:52 smithi170 ceph-mon[102832]: pgmap v22121: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 746 KiB/s wr, 19 op/s 2023-09-10T10:45:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:52 smithi082 ceph-mon[98694]: pgmap v22121: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 746 KiB/s wr, 19 op/s 2023-09-10T10:45:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:52 smithi098 ceph-mon[103124]: pgmap v22121: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 746 KiB/s wr, 19 op/s 2023-09-10T10:45:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:54 smithi170 ceph-mon[102832]: pgmap v22122: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 670 KiB/s wr, 13 op/s 2023-09-10T10:45:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:54 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41184.701927 secs 2023-09-10T10:45:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:45:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41185.929394 secs 2023-09-10T10:45:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:54 smithi082 ceph-mon[98694]: pgmap v22122: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 670 KiB/s wr, 13 op/s 2023-09-10T10:45:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:54 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41184.701927 secs 2023-09-10T10:45:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:45:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41185.929394 secs 2023-09-10T10:45:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:54 smithi098 ceph-mon[103124]: pgmap v22122: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 670 KiB/s wr, 13 op/s 2023-09-10T10:45:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:54 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41184.701927 secs 2023-09-10T10:45:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:45:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41185.929394 secs 2023-09-10T10:45:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41186.280805 secs 2023-09-10T10:45:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41186.280805 secs 2023-09-10T10:45:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41186.280805 secs 2023-09-10T10:45:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:56 smithi170 ceph-mon[102832]: pgmap v22123: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 670 KiB/s wr, 13 op/s 2023-09-10T10:45:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:56 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:45:56.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:45:56.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:56 smithi082 ceph-mon[98694]: pgmap v22123: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 670 KiB/s wr, 13 op/s 2023-09-10T10:45:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:56 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:45:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:45:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:56 smithi098 ceph-mon[103124]: pgmap v22123: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 670 KiB/s wr, 13 op/s 2023-09-10T10:45:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:56 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:45:56.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:45:56.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:45:57.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:57 smithi170 ceph-mon[102832]: pgmap v22124: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T10:45:57.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:57 smithi082 ceph-mon[98694]: pgmap v22124: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T10:45:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:57 smithi098 ceph-mon[103124]: pgmap v22124: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T10:45:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:58 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41189.701988 secs 2023-09-10T10:45:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:58 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41189.701988 secs 2023-09-10T10:45:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:58 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41189.701988 secs 2023-09-10T10:45:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:59 smithi170 ceph-mon[102832]: pgmap v22125: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 567 KiB/s wr, 14 op/s 2023-09-10T10:45:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:45:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41190.929478 secs 2023-09-10T10:45:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:59 smithi082 ceph-mon[98694]: pgmap v22125: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 567 KiB/s wr, 14 op/s 2023-09-10T10:45:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:45:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41190.929478 secs 2023-09-10T10:46:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:59 smithi098 ceph-mon[103124]: pgmap v22125: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 567 KiB/s wr, 14 op/s 2023-09-10T10:46:00.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:45:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41190.929478 secs 2023-09-10T10:46:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41191.280868 secs 2023-09-10T10:46:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:46:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41191.280868 secs 2023-09-10T10:46:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:46:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41191.280868 secs 2023-09-10T10:46:01.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:46:01.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:01 smithi170 ceph-mon[102832]: pgmap v22126: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 627 KiB/s wr, 20 op/s 2023-09-10T10:46:01.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:01 smithi082 ceph-mon[98694]: pgmap v22126: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 627 KiB/s wr, 20 op/s 2023-09-10T10:46:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:01 smithi098 ceph-mon[103124]: pgmap v22126: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 627 KiB/s wr, 20 op/s 2023-09-10T10:46:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:03 smithi170 ceph-mon[102832]: pgmap v22127: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 471 KiB/s wr, 14 op/s 2023-09-10T10:46:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41194.702067 secs 2023-09-10T10:46:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:03 smithi082 ceph-mon[98694]: pgmap v22127: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 471 KiB/s wr, 14 op/s 2023-09-10T10:46:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41194.702067 secs 2023-09-10T10:46:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:03 smithi098 ceph-mon[103124]: pgmap v22127: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 471 KiB/s wr, 14 op/s 2023-09-10T10:46:04.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41194.702067 secs 2023-09-10T10:46:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:46:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41195.929493 secs 2023-09-10T10:46:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:46:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41195.929493 secs 2023-09-10T10:46:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:46:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41195.929493 secs 2023-09-10T10:46:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41196.280935 secs 2023-09-10T10:46:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:05 smithi170 ceph-mon[102832]: pgmap v22128: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 471 KiB/s wr, 14 op/s 2023-09-10T10:46:05.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41196.280935 secs 2023-09-10T10:46:05.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:05 smithi082 ceph-mon[98694]: pgmap v22128: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 471 KiB/s wr, 14 op/s 2023-09-10T10:46:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41196.280935 secs 2023-09-10T10:46:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:05 smithi098 ceph-mon[103124]: pgmap v22128: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 471 KiB/s wr, 14 op/s 2023-09-10T10:46:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:46:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:46:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:46:07.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:07 smithi170 ceph-mon[102832]: pgmap v22129: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 35 op/s 2023-09-10T10:46:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:07 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:46:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:07 smithi082 ceph-mon[98694]: pgmap v22129: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 35 op/s 2023-09-10T10:46:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:07 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:46:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:07 smithi098 ceph-mon[103124]: pgmap v22129: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 35 op/s 2023-09-10T10:46:08.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:07 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:46:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:46:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:46:08.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:46:08.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41199.702137 secs 2023-09-10T10:46:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:46:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:46:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:46:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41199.702137 secs 2023-09-10T10:46:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:46:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:46:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:46:09.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41199.702137 secs 2023-09-10T10:46:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:09 smithi098 ceph-mon[103124]: pgmap v22130: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 702 KiB/s wr, 27 op/s 2023-09-10T10:46:10.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41200.929553 secs 2023-09-10T10:46:10.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:09 smithi098 ceph-mon[103124]: osdmap e1451: 12 total, 12 up, 12 in 2023-09-10T10:46:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:09 smithi170 ceph-mon[102832]: pgmap v22130: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 702 KiB/s wr, 27 op/s 2023-09-10T10:46:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41200.929553 secs 2023-09-10T10:46:10.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:09 smithi170 ceph-mon[102832]: osdmap e1451: 12 total, 12 up, 12 in 2023-09-10T10:46:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:09 smithi082 ceph-mon[98694]: pgmap v22130: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 702 KiB/s wr, 27 op/s 2023-09-10T10:46:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41200.929553 secs 2023-09-10T10:46:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:09 smithi082 ceph-mon[98694]: osdmap e1451: 12 total, 12 up, 12 in 2023-09-10T10:46:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41201.281000 secs 2023-09-10T10:46:11.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:46:11.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41201.281000 secs 2023-09-10T10:46:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:46:11.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41201.281000 secs 2023-09-10T10:46:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:46:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:12 smithi170 ceph-mon[102832]: pgmap v22132: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 34 op/s 2023-09-10T10:46:12.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:46:12.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:12 smithi082 ceph-mon[98694]: pgmap v22132: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 34 op/s 2023-09-10T10:46:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:46:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:12 smithi098 ceph-mon[103124]: pgmap v22132: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 34 op/s 2023-09-10T10:46:12.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:46:12.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:14 smithi170 ceph-mon[102832]: pgmap v22133: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 34 op/s 2023-09-10T10:46:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41204.702197 secs 2023-09-10T10:46:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41205.929612 secs 2023-09-10T10:46:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:14 smithi082 ceph-mon[98694]: pgmap v22133: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 34 op/s 2023-09-10T10:46:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41204.702197 secs 2023-09-10T10:46:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41205.929612 secs 2023-09-10T10:46:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:14 smithi098 ceph-mon[103124]: pgmap v22133: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 34 op/s 2023-09-10T10:46:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41204.702197 secs 2023-09-10T10:46:14.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41205.929612 secs 2023-09-10T10:46:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41206.281064 secs 2023-09-10T10:46:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41206.281064 secs 2023-09-10T10:46:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41206.281064 secs 2023-09-10T10:46:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:16 smithi170 ceph-mon[102832]: pgmap v22134: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 34 op/s 2023-09-10T10:46:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:16 smithi170 ceph-mon[102832]: osdmap e1452: 12 total, 12 up, 12 in 2023-09-10T10:46:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:16 smithi082 ceph-mon[98694]: pgmap v22134: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 34 op/s 2023-09-10T10:46:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:16 smithi082 ceph-mon[98694]: osdmap e1452: 12 total, 12 up, 12 in 2023-09-10T10:46:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:16 smithi098 ceph-mon[103124]: pgmap v22134: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 34 op/s 2023-09-10T10:46:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:16 smithi098 ceph-mon[103124]: osdmap e1452: 12 total, 12 up, 12 in 2023-09-10T10:46:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:18 smithi170 ceph-mon[102832]: pgmap v22136: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 811 KiB/s wr, 21 op/s 2023-09-10T10:46:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:46:18.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:18 smithi082 ceph-mon[98694]: pgmap v22136: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 811 KiB/s wr, 21 op/s 2023-09-10T10:46:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:46:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:18 smithi098 ceph-mon[103124]: pgmap v22136: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 811 KiB/s wr, 21 op/s 2023-09-10T10:46:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:46:18.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41209.702256 secs 2023-09-10T10:46:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41210.929676 secs 2023-09-10T10:46:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41209.702256 secs 2023-09-10T10:46:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41210.929676 secs 2023-09-10T10:46:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41209.702256 secs 2023-09-10T10:46:19.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41210.929676 secs 2023-09-10T10:46:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:20 smithi170 ceph-mon[102832]: pgmap v22137: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 658 KiB/s wr, 17 op/s 2023-09-10T10:46:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41211.281131 secs 2023-09-10T10:46:20.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:46:20.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:20 smithi082 ceph-mon[98694]: pgmap v22137: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 658 KiB/s wr, 17 op/s 2023-09-10T10:46:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41211.281131 secs 2023-09-10T10:46:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:46:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:20 smithi098 ceph-mon[103124]: pgmap v22137: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 658 KiB/s wr, 17 op/s 2023-09-10T10:46:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41211.281131 secs 2023-09-10T10:46:20.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:46:20.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:22 smithi170 ceph-mon[102832]: pgmap v22138: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 399 KiB/s wr, 14 op/s 2023-09-10T10:46:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:22 smithi082 ceph-mon[98694]: pgmap v22138: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 399 KiB/s wr, 14 op/s 2023-09-10T10:46:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:22 smithi098 ceph-mon[103124]: pgmap v22138: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 399 KiB/s wr, 14 op/s 2023-09-10T10:46:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:24 smithi170 ceph-mon[102832]: pgmap v22139: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 399 KiB/s wr, 14 op/s 2023-09-10T10:46:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41214.702317 secs 2023-09-10T10:46:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41215.929744 secs 2023-09-10T10:46:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:24 smithi082 ceph-mon[98694]: pgmap v22139: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 399 KiB/s wr, 14 op/s 2023-09-10T10:46:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41214.702317 secs 2023-09-10T10:46:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41215.929744 secs 2023-09-10T10:46:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:24 smithi098 ceph-mon[103124]: pgmap v22139: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 399 KiB/s wr, 14 op/s 2023-09-10T10:46:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41214.702317 secs 2023-09-10T10:46:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41215.929744 secs 2023-09-10T10:46:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41216.281195 secs 2023-09-10T10:46:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41216.281195 secs 2023-09-10T10:46:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41216.281195 secs 2023-09-10T10:46:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:26 smithi170 ceph-mon[102832]: pgmap v22140: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 399 KiB/s wr, 14 op/s 2023-09-10T10:46:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:26 smithi082 ceph-mon[98694]: pgmap v22140: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 399 KiB/s wr, 14 op/s 2023-09-10T10:46:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:26 smithi098 ceph-mon[103124]: pgmap v22140: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 399 KiB/s wr, 14 op/s 2023-09-10T10:46:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:28 smithi170 ceph-mon[102832]: pgmap v22141: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 353 KiB/s wr, 13 op/s 2023-09-10T10:46:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:28 smithi082 ceph-mon[98694]: pgmap v22141: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 353 KiB/s wr, 13 op/s 2023-09-10T10:46:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:28 smithi098 ceph-mon[103124]: pgmap v22141: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 353 KiB/s wr, 13 op/s 2023-09-10T10:46:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41219.702377 secs 2023-09-10T10:46:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41220.929832 secs 2023-09-10T10:46:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41219.702377 secs 2023-09-10T10:46:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41220.929832 secs 2023-09-10T10:46:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41219.702377 secs 2023-09-10T10:46:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41220.929832 secs 2023-09-10T10:46:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:30 smithi170 ceph-mon[102832]: pgmap v22142: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 302 KiB/s wr, 11 op/s 2023-09-10T10:46:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41221.281260 secs 2023-09-10T10:46:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:30 smithi082 ceph-mon[98694]: pgmap v22142: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 302 KiB/s wr, 11 op/s 2023-09-10T10:46:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41221.281260 secs 2023-09-10T10:46:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:30 smithi098 ceph-mon[103124]: pgmap v22142: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 302 KiB/s wr, 11 op/s 2023-09-10T10:46:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41221.281260 secs 2023-09-10T10:46:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:32 smithi170 ceph-mon[102832]: pgmap v22143: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 302 KiB/s wr, 17 op/s 2023-09-10T10:46:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:32 smithi082 ceph-mon[98694]: pgmap v22143: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 302 KiB/s wr, 17 op/s 2023-09-10T10:46:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:32 smithi098 ceph-mon[103124]: pgmap v22143: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 302 KiB/s wr, 17 op/s 2023-09-10T10:46:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:34 smithi170 ceph-mon[102832]: pgmap v22144: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T10:46:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41224.702485 secs 2023-09-10T10:46:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:46:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41225.929886 secs 2023-09-10T10:46:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:34 smithi082 ceph-mon[98694]: pgmap v22144: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T10:46:34.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41224.702485 secs 2023-09-10T10:46:34.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:46:34.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:34.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41225.929886 secs 2023-09-10T10:46:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:34 smithi098 ceph-mon[103124]: pgmap v22144: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T10:46:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41224.702485 secs 2023-09-10T10:46:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:46:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41225.929886 secs 2023-09-10T10:46:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41226.281325 secs 2023-09-10T10:46:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41226.281325 secs 2023-09-10T10:46:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41226.281325 secs 2023-09-10T10:46:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:36 smithi170 ceph-mon[102832]: pgmap v22145: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T10:46:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:46:36.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:36 smithi082 ceph-mon[98694]: pgmap v22145: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T10:46:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:46:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:36 smithi098 ceph-mon[103124]: pgmap v22145: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 122 KiB/s wr, 11 op/s 2023-09-10T10:46:36.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:46:36.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:38 smithi170 ceph-mon[102832]: pgmap v22146: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 417 KiB/s wr, 16 op/s 2023-09-10T10:46:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:38 smithi082 ceph-mon[98694]: pgmap v22146: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 417 KiB/s wr, 16 op/s 2023-09-10T10:46:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:38 smithi098 ceph-mon[103124]: pgmap v22146: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 417 KiB/s wr, 16 op/s 2023-09-10T10:46:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41230.929943 secs 2023-09-10T10:46:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:46:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41230.929943 secs 2023-09-10T10:46:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:46:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41230.929943 secs 2023-09-10T10:46:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:46:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:40 smithi170 ceph-mon[102832]: pgmap v22147: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 295 KiB/s wr, 11 op/s 2023-09-10T10:46:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41231.281390 secs 2023-09-10T10:46:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:46:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:46:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:40 smithi082 ceph-mon[98694]: pgmap v22147: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 295 KiB/s wr, 11 op/s 2023-09-10T10:46:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41231.281390 secs 2023-09-10T10:46:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:46:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:46:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:40 smithi098 ceph-mon[103124]: pgmap v22147: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 295 KiB/s wr, 11 op/s 2023-09-10T10:46:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41231.281390 secs 2023-09-10T10:46:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:46:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:46:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:42 smithi170 ceph-mon[102832]: pgmap v22148: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 300 KiB/s wr, 17 op/s 2023-09-10T10:46:42.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:42 smithi082 ceph-mon[98694]: pgmap v22148: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 300 KiB/s wr, 17 op/s 2023-09-10T10:46:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:42 smithi098 ceph-mon[103124]: pgmap v22148: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 300 KiB/s wr, 17 op/s 2023-09-10T10:46:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:44 smithi170 ceph-mon[102832]: pgmap v22149: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 300 KiB/s wr, 11 op/s 2023-09-10T10:46:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:46:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:44.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41235.930026 secs 2023-09-10T10:46:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:44 smithi082 ceph-mon[98694]: pgmap v22149: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 300 KiB/s wr, 11 op/s 2023-09-10T10:46:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:46:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41235.930026 secs 2023-09-10T10:46:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:44 smithi098 ceph-mon[103124]: pgmap v22149: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 300 KiB/s wr, 11 op/s 2023-09-10T10:46:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:46:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41235.930026 secs 2023-09-10T10:46:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41236.281456 secs 2023-09-10T10:46:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41236.281456 secs 2023-09-10T10:46:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41236.281456 secs 2023-09-10T10:46:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:46 smithi170 ceph-mon[102832]: pgmap v22150: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 300 KiB/s wr, 11 op/s 2023-09-10T10:46:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:46 smithi082 ceph-mon[98694]: pgmap v22150: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 300 KiB/s wr, 11 op/s 2023-09-10T10:46:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:46 smithi098 ceph-mon[103124]: pgmap v22150: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 300 KiB/s wr, 11 op/s 2023-09-10T10:46:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:48 smithi170 ceph-mon[102832]: pgmap v22151: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 628 KiB/s wr, 17 op/s 2023-09-10T10:46:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:48 smithi082 ceph-mon[98694]: pgmap v22151: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 628 KiB/s wr, 17 op/s 2023-09-10T10:46:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:48 smithi098 ceph-mon[103124]: pgmap v22151: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 628 KiB/s wr, 17 op/s 2023-09-10T10:46:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41240.930084 secs 2023-09-10T10:46:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41240.930084 secs 2023-09-10T10:46:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41240.930084 secs 2023-09-10T10:46:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:50 smithi170 ceph-mon[102832]: pgmap v22152: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T10:46:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41241.281487 secs 2023-09-10T10:46:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:46:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:50 smithi082 ceph-mon[98694]: pgmap v22152: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T10:46:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41241.281487 secs 2023-09-10T10:46:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:46:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:50 smithi098 ceph-mon[103124]: pgmap v22152: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T10:46:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41241.281487 secs 2023-09-10T10:46:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:46:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:51.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:51 smithi170 ceph-mon[102832]: pgmap v22153: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 540 KiB/s wr, 19 op/s 2023-09-10T10:46:51.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:51 smithi082 ceph-mon[98694]: pgmap v22153: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 540 KiB/s wr, 19 op/s 2023-09-10T10:46:51.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:51 smithi098 ceph-mon[103124]: pgmap v22153: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 540 KiB/s wr, 19 op/s 2023-09-10T10:46:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:53 smithi170 ceph-mon[102832]: pgmap v22154: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 535 KiB/s wr, 13 op/s 2023-09-10T10:46:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:53 smithi082 ceph-mon[98694]: pgmap v22154: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 535 KiB/s wr, 13 op/s 2023-09-10T10:46:54.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:53 smithi098 ceph-mon[103124]: pgmap v22154: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 535 KiB/s wr, 13 op/s 2023-09-10T10:46:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:46:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:54.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41245.930144 secs 2023-09-10T10:46:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:46:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41245.930144 secs 2023-09-10T10:46:55.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:46:55.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:55.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41245.930144 secs 2023-09-10T10:46:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41246.281555 secs 2023-09-10T10:46:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:55 smithi170 ceph-mon[102832]: pgmap v22155: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 535 KiB/s wr, 13 op/s 2023-09-10T10:46:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41246.281555 secs 2023-09-10T10:46:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:55 smithi082 ceph-mon[98694]: pgmap v22155: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 535 KiB/s wr, 13 op/s 2023-09-10T10:46:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41246.281555 secs 2023-09-10T10:46:56.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:55 smithi098 ceph-mon[103124]: pgmap v22155: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 535 KiB/s wr, 13 op/s 2023-09-10T10:46:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:46:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:46:57.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:57.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:46:57.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:57 smithi082 ceph-mon[98694]: pgmap v22156: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 929 KiB/s wr, 21 op/s 2023-09-10T10:46:58.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:57 smithi098 ceph-mon[103124]: pgmap v22156: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 929 KiB/s wr, 21 op/s 2023-09-10T10:46:58.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:57 smithi170 ceph-mon[102832]: pgmap v22156: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 929 KiB/s wr, 21 op/s 2023-09-10T10:46:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:46:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:46:59.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:46:59.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:46:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:59 smithi082 ceph-mon[98694]: pgmap v22157: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 601 KiB/s wr, 14 op/s 2023-09-10T10:46:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:46:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41250.930202 secs 2023-09-10T10:47:00.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:59 smithi098 ceph-mon[103124]: pgmap v22157: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 601 KiB/s wr, 14 op/s 2023-09-10T10:47:00.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:46:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41250.930202 secs 2023-09-10T10:47:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:59 smithi170 ceph-mon[102832]: pgmap v22157: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 601 KiB/s wr, 14 op/s 2023-09-10T10:47:00.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:46:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41250.930202 secs 2023-09-10T10:47:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41251.281617 secs 2023-09-10T10:47:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:47:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:01.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41251.281617 secs 2023-09-10T10:47:01.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:47:01.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41251.281617 secs 2023-09-10T10:47:01.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:47:01.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:02.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:02 smithi170 ceph-mon[102832]: pgmap v22158: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 629 KiB/s wr, 22 op/s 2023-09-10T10:47:02.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:02 smithi082 ceph-mon[98694]: pgmap v22158: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 629 KiB/s wr, 22 op/s 2023-09-10T10:47:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:02 smithi098 ceph-mon[103124]: pgmap v22158: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 629 KiB/s wr, 22 op/s 2023-09-10T10:47:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:04 smithi170 ceph-mon[102832]: pgmap v22159: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 422 KiB/s wr, 15 op/s 2023-09-10T10:47:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:47:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41255.930285 secs 2023-09-10T10:47:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:04 smithi082 ceph-mon[98694]: pgmap v22159: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 422 KiB/s wr, 15 op/s 2023-09-10T10:47:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:47:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:04.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41255.930285 secs 2023-09-10T10:47:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:04 smithi098 ceph-mon[103124]: pgmap v22159: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 422 KiB/s wr, 15 op/s 2023-09-10T10:47:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:47:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41255.930285 secs 2023-09-10T10:47:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41256.281680 secs 2023-09-10T10:47:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41256.281680 secs 2023-09-10T10:47:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41256.281680 secs 2023-09-10T10:47:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:06 smithi170 ceph-mon[102832]: pgmap v22160: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 422 KiB/s wr, 15 op/s 2023-09-10T10:47:06.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:47:06.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:06 smithi082 ceph-mon[98694]: pgmap v22160: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 422 KiB/s wr, 15 op/s 2023-09-10T10:47:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:47:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:06 smithi098 ceph-mon[103124]: pgmap v22160: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 422 KiB/s wr, 15 op/s 2023-09-10T10:47:06.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:47:06.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:08 smithi170 ceph-mon[102832]: pgmap v22161: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 33 op/s 2023-09-10T10:47:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:08 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:47:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:08 smithi082 ceph-mon[98694]: pgmap v22161: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 33 op/s 2023-09-10T10:47:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:08 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:47:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:08 smithi098 ceph-mon[103124]: pgmap v22161: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 33 op/s 2023-09-10T10:47:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:08 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:47:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:47:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:47:09.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:47:09.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41260.930340 secs 2023-09-10T10:47:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:47:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:47:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:47:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41260.930340 secs 2023-09-10T10:47:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:47:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:47:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:47:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41260.930340 secs 2023-09-10T10:47:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:10 smithi170 ceph-mon[102832]: pgmap v22162: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 885 KiB/s wr, 26 op/s 2023-09-10T10:47:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41261.281743 secs 2023-09-10T10:47:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:10 smithi082 ceph-mon[98694]: pgmap v22162: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 885 KiB/s wr, 26 op/s 2023-09-10T10:47:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41261.281743 secs 2023-09-10T10:47:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:10 smithi098 ceph-mon[103124]: pgmap v22162: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 885 KiB/s wr, 26 op/s 2023-09-10T10:47:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41261.281743 secs 2023-09-10T10:47:12.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:12 smithi170 ceph-mon[102832]: pgmap v22163: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 916 KiB/s wr, 31 op/s 2023-09-10T10:47:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:12 smithi082 ceph-mon[98694]: pgmap v22163: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 916 KiB/s wr, 31 op/s 2023-09-10T10:47:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:12 smithi098 ceph-mon[103124]: pgmap v22163: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 916 KiB/s wr, 31 op/s 2023-09-10T10:47:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:14 smithi170 ceph-mon[102832]: pgmap v22164: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 888 KiB/s wr, 24 op/s 2023-09-10T10:47:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41265.930423 secs 2023-09-10T10:47:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:14 smithi082 ceph-mon[98694]: pgmap v22164: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 888 KiB/s wr, 24 op/s 2023-09-10T10:47:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41265.930423 secs 2023-09-10T10:47:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:14 smithi098 ceph-mon[103124]: pgmap v22164: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 888 KiB/s wr, 24 op/s 2023-09-10T10:47:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41265.930423 secs 2023-09-10T10:47:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41266.281807 secs 2023-09-10T10:47:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41266.281807 secs 2023-09-10T10:47:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41266.281807 secs 2023-09-10T10:47:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:16 smithi170 ceph-mon[102832]: pgmap v22165: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 888 KiB/s wr, 24 op/s 2023-09-10T10:47:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:16 smithi082 ceph-mon[98694]: pgmap v22165: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 888 KiB/s wr, 24 op/s 2023-09-10T10:47:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:16 smithi098 ceph-mon[103124]: pgmap v22165: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 888 KiB/s wr, 24 op/s 2023-09-10T10:47:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:18 smithi170 ceph-mon[102832]: pgmap v22166: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 925 KiB/s wr, 29 op/s 2023-09-10T10:47:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:18 smithi082 ceph-mon[98694]: pgmap v22166: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 925 KiB/s wr, 29 op/s 2023-09-10T10:47:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:18 smithi098 ceph-mon[103124]: pgmap v22166: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 925 KiB/s wr, 29 op/s 2023-09-10T10:47:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41270.930490 secs 2023-09-10T10:47:19.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41270.930490 secs 2023-09-10T10:47:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41270.930490 secs 2023-09-10T10:47:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:20 smithi170 ceph-mon[102832]: pgmap v22167: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 68 KiB/s wr, 10 op/s 2023-09-10T10:47:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41271.281871 secs 2023-09-10T10:47:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:20 smithi082 ceph-mon[98694]: pgmap v22167: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 68 KiB/s wr, 10 op/s 2023-09-10T10:47:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41271.281871 secs 2023-09-10T10:47:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:20 smithi098 ceph-mon[103124]: pgmap v22167: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 68 KiB/s wr, 10 op/s 2023-09-10T10:47:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41271.281871 secs 2023-09-10T10:47:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:22 smithi170 ceph-mon[102832]: pgmap v22168: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 384 KiB/s wr, 19 op/s 2023-09-10T10:47:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:22 smithi082 ceph-mon[98694]: pgmap v22168: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 384 KiB/s wr, 19 op/s 2023-09-10T10:47:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:22 smithi098 ceph-mon[103124]: pgmap v22168: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 384 KiB/s wr, 19 op/s 2023-09-10T10:47:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:23 smithi170 ceph-mon[102832]: pgmap v22169: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 353 KiB/s wr, 13 op/s 2023-09-10T10:47:23.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:23 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:47:23.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:23 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:47:23.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:23 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:23.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:23 smithi170 ceph-mon[102832]: osdmap e1453: 12 total, 12 up, 12 in 2023-09-10T10:47:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:23 smithi082 ceph-mon[98694]: pgmap v22169: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 353 KiB/s wr, 13 op/s 2023-09-10T10:47:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:23 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:47:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:23 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:47:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:23 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:23.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:23 smithi082 ceph-mon[98694]: osdmap e1453: 12 total, 12 up, 12 in 2023-09-10T10:47:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:23 smithi098 ceph-mon[103124]: pgmap v22169: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 353 KiB/s wr, 13 op/s 2023-09-10T10:47:23.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:23 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:47:23.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:23 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:47:23.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:23 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:23.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:23 smithi098 ceph-mon[103124]: osdmap e1453: 12 total, 12 up, 12 in 2023-09-10T10:47:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41274.703187 secs 2023-09-10T10:47:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41275.930550 secs 2023-09-10T10:47:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41274.703187 secs 2023-09-10T10:47:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41275.930550 secs 2023-09-10T10:47:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41274.703187 secs 2023-09-10T10:47:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41275.930550 secs 2023-09-10T10:47:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41276.281937 secs 2023-09-10T10:47:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:25 smithi170 ceph-mon[102832]: pgmap v22171: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 424 KiB/s wr, 16 op/s 2023-09-10T10:47:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41276.281937 secs 2023-09-10T10:47:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:25 smithi082 ceph-mon[98694]: pgmap v22171: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 424 KiB/s wr, 16 op/s 2023-09-10T10:47:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41276.281937 secs 2023-09-10T10:47:25.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:25 smithi098 ceph-mon[103124]: pgmap v22171: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 424 KiB/s wr, 16 op/s 2023-09-10T10:47:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:26 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:47:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:26 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:26 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:47:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:26 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:26 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:47:26.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:26 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:27.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:27 smithi082 ceph-mon[98694]: pgmap v22172: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 734 KiB/s wr, 18 op/s 2023-09-10T10:47:27.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:27 smithi098 ceph-mon[103124]: pgmap v22172: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 734 KiB/s wr, 18 op/s 2023-09-10T10:47:27.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:27 smithi170 ceph-mon[102832]: pgmap v22172: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 734 KiB/s wr, 18 op/s 2023-09-10T10:47:28.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:28 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41279.703246 secs 2023-09-10T10:47:28.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:28 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41279.703246 secs 2023-09-10T10:47:29.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:28 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41279.703246 secs 2023-09-10T10:47:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:29 smithi170 ceph-mon[102832]: pgmap v22173: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 734 KiB/s wr, 18 op/s 2023-09-10T10:47:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41280.930610 secs 2023-09-10T10:47:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:29 smithi082 ceph-mon[98694]: pgmap v22173: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 734 KiB/s wr, 18 op/s 2023-09-10T10:47:29.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41280.930610 secs 2023-09-10T10:47:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:29 smithi098 ceph-mon[103124]: pgmap v22173: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 734 KiB/s wr, 18 op/s 2023-09-10T10:47:30.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41280.930610 secs 2023-09-10T10:47:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41281.282000 secs 2023-09-10T10:47:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:47:30.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:30.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:30 smithi170 ceph-mon[102832]: osdmap e1454: 12 total, 12 up, 12 in 2023-09-10T10:47:30.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41281.282000 secs 2023-09-10T10:47:30.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:47:30.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:30.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:30 smithi082 ceph-mon[98694]: osdmap e1454: 12 total, 12 up, 12 in 2023-09-10T10:47:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41281.282000 secs 2023-09-10T10:47:31.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:47:31.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:31.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:30 smithi098 ceph-mon[103124]: osdmap e1454: 12 total, 12 up, 12 in 2023-09-10T10:47:31.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:31 smithi170 ceph-mon[102832]: pgmap v22175: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 722 KiB/s wr, 20 op/s 2023-09-10T10:47:31.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:31 smithi082 ceph-mon[98694]: pgmap v22175: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 722 KiB/s wr, 20 op/s 2023-09-10T10:47:32.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:31 smithi098 ceph-mon[103124]: pgmap v22175: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 722 KiB/s wr, 20 op/s 2023-09-10T10:47:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:47:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:47:32.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:47:33.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:33 smithi170 ceph-mon[102832]: pgmap v22176: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 625 KiB/s wr, 18 op/s 2023-09-10T10:47:33.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:33 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41284.703308 secs 2023-09-10T10:47:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:33 smithi082 ceph-mon[98694]: pgmap v22176: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 625 KiB/s wr, 18 op/s 2023-09-10T10:47:33.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:33 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41284.703308 secs 2023-09-10T10:47:34.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:33 smithi098 ceph-mon[103124]: pgmap v22176: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 625 KiB/s wr, 18 op/s 2023-09-10T10:47:34.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:33 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41284.703308 secs 2023-09-10T10:47:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41285.930669 secs 2023-09-10T10:47:34.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41285.930669 secs 2023-09-10T10:47:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41285.930669 secs 2023-09-10T10:47:35.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41286.282067 secs 2023-09-10T10:47:35.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:35 smithi170 ceph-mon[102832]: pgmap v22177: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 578 KiB/s wr, 16 op/s 2023-09-10T10:47:35.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41286.282067 secs 2023-09-10T10:47:35.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:35 smithi082 ceph-mon[98694]: pgmap v22177: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 578 KiB/s wr, 16 op/s 2023-09-10T10:47:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41286.282067 secs 2023-09-10T10:47:36.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:35 smithi098 ceph-mon[103124]: pgmap v22177: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 578 KiB/s wr, 16 op/s 2023-09-10T10:47:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:37 smithi170 ceph-mon[102832]: pgmap v22178: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 369 KiB/s wr, 15 op/s 2023-09-10T10:47:37.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:37 smithi082 ceph-mon[98694]: pgmap v22178: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 369 KiB/s wr, 15 op/s 2023-09-10T10:47:38.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:37 smithi098 ceph-mon[103124]: pgmap v22178: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 369 KiB/s wr, 15 op/s 2023-09-10T10:47:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:39 smithi082 ceph-mon[98694]: pgmap v22179: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 369 KiB/s wr, 15 op/s 2023-09-10T10:47:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41290.930727 secs 2023-09-10T10:47:39.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:47:40.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:39 smithi098 ceph-mon[103124]: pgmap v22179: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 369 KiB/s wr, 15 op/s 2023-09-10T10:47:40.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41290.930727 secs 2023-09-10T10:47:40.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:47:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:39 smithi170 ceph-mon[102832]: pgmap v22179: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 369 KiB/s wr, 15 op/s 2023-09-10T10:47:40.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41290.930727 secs 2023-09-10T10:47:40.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:47:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41291.282133 secs 2023-09-10T10:47:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:47:40.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:47:40.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41291.282133 secs 2023-09-10T10:47:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:47:41.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:47:41.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41291.282133 secs 2023-09-10T10:47:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:47:41.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:47:41.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:42 smithi170 ceph-mon[102832]: pgmap v22180: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 534 KiB/s wr, 16 op/s 2023-09-10T10:47:42.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:42 smithi082 ceph-mon[98694]: pgmap v22180: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 534 KiB/s wr, 16 op/s 2023-09-10T10:47:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:42 smithi098 ceph-mon[103124]: pgmap v22180: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 534 KiB/s wr, 16 op/s 2023-09-10T10:47:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:44 smithi170 ceph-mon[102832]: pgmap v22181: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 458 KiB/s wr, 12 op/s 2023-09-10T10:47:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41295.930784 secs 2023-09-10T10:47:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:44 smithi082 ceph-mon[98694]: pgmap v22181: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 458 KiB/s wr, 12 op/s 2023-09-10T10:47:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41295.930784 secs 2023-09-10T10:47:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:44 smithi098 ceph-mon[103124]: pgmap v22181: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 458 KiB/s wr, 12 op/s 2023-09-10T10:47:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41295.930784 secs 2023-09-10T10:47:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41296.282200 secs 2023-09-10T10:47:45.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41296.282200 secs 2023-09-10T10:47:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41296.282200 secs 2023-09-10T10:47:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:46 smithi170 ceph-mon[102832]: pgmap v22182: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 458 KiB/s wr, 12 op/s 2023-09-10T10:47:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:47:46.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:46.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:46 smithi082 ceph-mon[98694]: pgmap v22182: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 458 KiB/s wr, 12 op/s 2023-09-10T10:47:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:47:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:46 smithi098 ceph-mon[103124]: pgmap v22182: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 458 KiB/s wr, 12 op/s 2023-09-10T10:47:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:47:46.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:48 smithi170 ceph-mon[102832]: pgmap v22183: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 787 KiB/s wr, 19 op/s 2023-09-10T10:47:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:48 smithi082 ceph-mon[98694]: pgmap v22183: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 787 KiB/s wr, 19 op/s 2023-09-10T10:47:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:48 smithi098 ceph-mon[103124]: pgmap v22183: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 787 KiB/s wr, 19 op/s 2023-09-10T10:47:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41300.930845 secs 2023-09-10T10:47:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41300.930845 secs 2023-09-10T10:47:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41300.930845 secs 2023-09-10T10:47:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:50 smithi170 ceph-mon[102832]: pgmap v22184: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 665 KiB/s wr, 13 op/s 2023-09-10T10:47:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41301.282263 secs 2023-09-10T10:47:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:47:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:50 smithi082 ceph-mon[98694]: pgmap v22184: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 665 KiB/s wr, 13 op/s 2023-09-10T10:47:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41301.282263 secs 2023-09-10T10:47:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:47:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:50 smithi098 ceph-mon[103124]: pgmap v22184: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 665 KiB/s wr, 13 op/s 2023-09-10T10:47:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41301.282263 secs 2023-09-10T10:47:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:47:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:52 smithi170 ceph-mon[102832]: pgmap v22185: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 938 KiB/s wr, 20 op/s 2023-09-10T10:47:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:47:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:52 smithi082 ceph-mon[98694]: pgmap v22185: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 938 KiB/s wr, 20 op/s 2023-09-10T10:47:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:47:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:52 smithi098 ceph-mon[103124]: pgmap v22185: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 938 KiB/s wr, 20 op/s 2023-09-10T10:47:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:47:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:54 smithi170 ceph-mon[102832]: pgmap v22186: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 603 KiB/s wr, 13 op/s 2023-09-10T10:47:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:47:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41305.930906 secs 2023-09-10T10:47:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:54 smithi082 ceph-mon[98694]: pgmap v22186: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 603 KiB/s wr, 13 op/s 2023-09-10T10:47:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:47:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41305.930906 secs 2023-09-10T10:47:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:54 smithi098 ceph-mon[103124]: pgmap v22186: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 603 KiB/s wr, 13 op/s 2023-09-10T10:47:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:47:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41305.930906 secs 2023-09-10T10:47:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41306.282327 secs 2023-09-10T10:47:55.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41306.282327 secs 2023-09-10T10:47:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41306.282327 secs 2023-09-10T10:47:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:56 smithi170 ceph-mon[102832]: pgmap v22187: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 603 KiB/s wr, 13 op/s 2023-09-10T10:47:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:47:56.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:56 smithi082 ceph-mon[98694]: pgmap v22187: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 603 KiB/s wr, 13 op/s 2023-09-10T10:47:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:47:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:56 smithi098 ceph-mon[103124]: pgmap v22187: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 603 KiB/s wr, 13 op/s 2023-09-10T10:47:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:47:56.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:58 smithi170 ceph-mon[102832]: pgmap v22188: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T10:47:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:47:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:58 smithi082 ceph-mon[98694]: pgmap v22188: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T10:47:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:47:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:58 smithi098 ceph-mon[103124]: pgmap v22188: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T10:47:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:47:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:47:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41309.703641 secs 2023-09-10T10:47:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:47:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41310.930963 secs 2023-09-10T10:47:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41309.703641 secs 2023-09-10T10:47:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:47:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41310.930963 secs 2023-09-10T10:47:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41309.703641 secs 2023-09-10T10:47:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:47:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41310.930963 secs 2023-09-10T10:48:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:00 smithi170 ceph-mon[102832]: pgmap v22189: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 756 KiB/s wr, 14 op/s 2023-09-10T10:48:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41311.282393 secs 2023-09-10T10:48:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:48:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:48:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:00.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:00 smithi082 ceph-mon[98694]: pgmap v22189: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 756 KiB/s wr, 14 op/s 2023-09-10T10:48:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41311.282393 secs 2023-09-10T10:48:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:48:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:48:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:00 smithi098 ceph-mon[103124]: pgmap v22189: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 756 KiB/s wr, 14 op/s 2023-09-10T10:48:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41311.282393 secs 2023-09-10T10:48:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:48:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:48:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:02.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:02 smithi170 ceph-mon[102832]: pgmap v22190: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T10:48:02.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:02 smithi082 ceph-mon[98694]: pgmap v22190: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T10:48:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:02 smithi098 ceph-mon[103124]: pgmap v22190: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 21 op/s 2023-09-10T10:48:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:04 smithi170 ceph-mon[102832]: pgmap v22191: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 753 KiB/s wr, 14 op/s 2023-09-10T10:48:04.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41314.703732 secs 2023-09-10T10:48:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:48:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:04.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41315.931061 secs 2023-09-10T10:48:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:04 smithi082 ceph-mon[98694]: pgmap v22191: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 753 KiB/s wr, 14 op/s 2023-09-10T10:48:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41314.703732 secs 2023-09-10T10:48:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:48:04.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:04.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41315.931061 secs 2023-09-10T10:48:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:04 smithi098 ceph-mon[103124]: pgmap v22191: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 753 KiB/s wr, 14 op/s 2023-09-10T10:48:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41314.703732 secs 2023-09-10T10:48:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:48:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41315.931061 secs 2023-09-10T10:48:05.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41316.282459 secs 2023-09-10T10:48:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41316.282459 secs 2023-09-10T10:48:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41316.282459 secs 2023-09-10T10:48:06.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:06 smithi170 ceph-mon[102832]: pgmap v22192: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 753 KiB/s wr, 14 op/s 2023-09-10T10:48:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:06 smithi082 ceph-mon[98694]: pgmap v22192: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 753 KiB/s wr, 14 op/s 2023-09-10T10:48:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:06 smithi098 ceph-mon[103124]: pgmap v22192: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 753 KiB/s wr, 14 op/s 2023-09-10T10:48:08.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:08 smithi170 ceph-mon[102832]: pgmap v22193: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 35 op/s 2023-09-10T10:48:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:08 smithi082 ceph-mon[98694]: pgmap v22193: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 35 op/s 2023-09-10T10:48:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:08 smithi098 ceph-mon[103124]: pgmap v22193: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 35 op/s 2023-09-10T10:48:09.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41319.703779 secs 2023-09-10T10:48:09.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:09 smithi170 ceph-mon[102832]: osdmap e1455: 12 total, 12 up, 12 in 2023-09-10T10:48:09.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:48:09.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41320.931161 secs 2023-09-10T10:48:09.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:48:09.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:48:09.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:48:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41319.703779 secs 2023-09-10T10:48:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:09 smithi082 ceph-mon[98694]: osdmap e1455: 12 total, 12 up, 12 in 2023-09-10T10:48:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:48:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41320.931161 secs 2023-09-10T10:48:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:48:09.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:48:09.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:48:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41319.703779 secs 2023-09-10T10:48:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:09 smithi098 ceph-mon[103124]: osdmap e1455: 12 total, 12 up, 12 in 2023-09-10T10:48:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:48:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41320.931161 secs 2023-09-10T10:48:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:48:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:48:09.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:48:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:10 smithi170 ceph-mon[102832]: pgmap v22195: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 763 KiB/s wr, 34 op/s 2023-09-10T10:48:10.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41321.282489 secs 2023-09-10T10:48:10.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:48:10.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:10.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:10 smithi082 ceph-mon[98694]: pgmap v22195: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 763 KiB/s wr, 34 op/s 2023-09-10T10:48:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41321.282489 secs 2023-09-10T10:48:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:48:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:10 smithi098 ceph-mon[103124]: pgmap v22195: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 763 KiB/s wr, 34 op/s 2023-09-10T10:48:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41321.282489 secs 2023-09-10T10:48:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:48:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:11.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:11 smithi170 ceph-mon[102832]: pgmap v22196: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 457 KiB/s wr, 34 op/s 2023-09-10T10:48:11.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:11 smithi082 ceph-mon[98694]: pgmap v22196: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 457 KiB/s wr, 34 op/s 2023-09-10T10:48:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:11 smithi098 ceph-mon[103124]: pgmap v22196: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 457 KiB/s wr, 34 op/s 2023-09-10T10:48:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:13 smithi082 ceph-mon[98694]: pgmap v22197: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 457 KiB/s wr, 34 op/s 2023-09-10T10:48:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41324.703842 secs 2023-09-10T10:48:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:13 smithi098 ceph-mon[103124]: pgmap v22197: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 457 KiB/s wr, 34 op/s 2023-09-10T10:48:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41324.703842 secs 2023-09-10T10:48:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:13 smithi170 ceph-mon[102832]: pgmap v22197: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 457 KiB/s wr, 34 op/s 2023-09-10T10:48:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41324.703842 secs 2023-09-10T10:48:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41325.931219 secs 2023-09-10T10:48:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41325.931219 secs 2023-09-10T10:48:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41325.931219 secs 2023-09-10T10:48:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41326.282554 secs 2023-09-10T10:48:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:15 smithi082 ceph-mon[98694]: pgmap v22198: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 457 KiB/s wr, 34 op/s 2023-09-10T10:48:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:15 smithi082 ceph-mon[98694]: osdmap e1456: 12 total, 12 up, 12 in 2023-09-10T10:48:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41326.282554 secs 2023-09-10T10:48:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:15 smithi098 ceph-mon[103124]: pgmap v22198: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 457 KiB/s wr, 34 op/s 2023-09-10T10:48:16.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:15 smithi098 ceph-mon[103124]: osdmap e1456: 12 total, 12 up, 12 in 2023-09-10T10:48:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41326.282554 secs 2023-09-10T10:48:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:15 smithi170 ceph-mon[102832]: pgmap v22198: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 457 KiB/s wr, 34 op/s 2023-09-10T10:48:16.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:15 smithi170 ceph-mon[102832]: osdmap e1456: 12 total, 12 up, 12 in 2023-09-10T10:48:17.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:17 smithi082 ceph-mon[98694]: pgmap v22200: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 451 KiB/s wr, 21 op/s 2023-09-10T10:48:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:17 smithi098 ceph-mon[103124]: pgmap v22200: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 451 KiB/s wr, 21 op/s 2023-09-10T10:48:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:17 smithi170 ceph-mon[102832]: pgmap v22200: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 451 KiB/s wr, 21 op/s 2023-09-10T10:48:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:48:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:18.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41329.703899 secs 2023-09-10T10:48:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:48:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:19.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41329.703899 secs 2023-09-10T10:48:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:48:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:19.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41329.703899 secs 2023-09-10T10:48:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:19 smithi098 ceph-mon[103124]: pgmap v22201: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 371 KiB/s wr, 17 op/s 2023-09-10T10:48:20.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41330.931303 secs 2023-09-10T10:48:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:19 smithi170 ceph-mon[102832]: pgmap v22201: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 371 KiB/s wr, 17 op/s 2023-09-10T10:48:20.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41330.931303 secs 2023-09-10T10:48:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:19 smithi082 ceph-mon[98694]: pgmap v22201: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 371 KiB/s wr, 17 op/s 2023-09-10T10:48:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41330.931303 secs 2023-09-10T10:48:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41331.282617 secs 2023-09-10T10:48:21.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:48:21.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41331.282617 secs 2023-09-10T10:48:21.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:48:21.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41331.282617 secs 2023-09-10T10:48:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:48:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:22 smithi170 ceph-mon[102832]: pgmap v22202: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 537 KiB/s wr, 16 op/s 2023-09-10T10:48:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:22 smithi082 ceph-mon[98694]: pgmap v22202: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 537 KiB/s wr, 16 op/s 2023-09-10T10:48:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:22 smithi098 ceph-mon[103124]: pgmap v22202: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 537 KiB/s wr, 16 op/s 2023-09-10T10:48:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:24 smithi170 ceph-mon[102832]: pgmap v22203: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 537 KiB/s wr, 16 op/s 2023-09-10T10:48:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41334.703958 secs 2023-09-10T10:48:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41335.931364 secs 2023-09-10T10:48:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:24 smithi082 ceph-mon[98694]: pgmap v22203: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 537 KiB/s wr, 16 op/s 2023-09-10T10:48:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41334.703958 secs 2023-09-10T10:48:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41335.931364 secs 2023-09-10T10:48:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:24 smithi098 ceph-mon[103124]: pgmap v22203: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 537 KiB/s wr, 16 op/s 2023-09-10T10:48:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41334.703958 secs 2023-09-10T10:48:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41335.931364 secs 2023-09-10T10:48:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41336.282683 secs 2023-09-10T10:48:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41336.282683 secs 2023-09-10T10:48:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41336.282683 secs 2023-09-10T10:48:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:26 smithi170 ceph-mon[102832]: pgmap v22204: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 537 KiB/s wr, 16 op/s 2023-09-10T10:48:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:26 smithi082 ceph-mon[98694]: pgmap v22204: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 537 KiB/s wr, 16 op/s 2023-09-10T10:48:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:26 smithi098 ceph-mon[103124]: pgmap v22204: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 537 KiB/s wr, 16 op/s 2023-09-10T10:48:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:28 smithi170 ceph-mon[102832]: pgmap v22205: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 163 KiB/s wr, 13 op/s 2023-09-10T10:48:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:28 smithi082 ceph-mon[98694]: pgmap v22205: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 163 KiB/s wr, 13 op/s 2023-09-10T10:48:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:28 smithi098 ceph-mon[103124]: pgmap v22205: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 163 KiB/s wr, 13 op/s 2023-09-10T10:48:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41339.704070 secs 2023-09-10T10:48:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41340.931416 secs 2023-09-10T10:48:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41339.704070 secs 2023-09-10T10:48:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41340.931416 secs 2023-09-10T10:48:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41339.704070 secs 2023-09-10T10:48:29.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41340.931416 secs 2023-09-10T10:48:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:30 smithi170 ceph-mon[102832]: pgmap v22206: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 154 KiB/s wr, 12 op/s 2023-09-10T10:48:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41341.282747 secs 2023-09-10T10:48:30.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:30 smithi082 ceph-mon[98694]: pgmap v22206: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 154 KiB/s wr, 12 op/s 2023-09-10T10:48:30.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41341.282747 secs 2023-09-10T10:48:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:30 smithi098 ceph-mon[103124]: pgmap v22206: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 154 KiB/s wr, 12 op/s 2023-09-10T10:48:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41341.282747 secs 2023-09-10T10:48:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:32 smithi170 ceph-mon[102832]: pgmap v22207: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 154 KiB/s wr, 17 op/s 2023-09-10T10:48:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:32 smithi082 ceph-mon[98694]: pgmap v22207: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 154 KiB/s wr, 17 op/s 2023-09-10T10:48:32.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:32 smithi098 ceph-mon[103124]: pgmap v22207: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 154 KiB/s wr, 17 op/s 2023-09-10T10:48:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:34 smithi170 ceph-mon[102832]: pgmap v22208: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 11 op/s 2023-09-10T10:48:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41344.704129 secs 2023-09-10T10:48:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41345.931501 secs 2023-09-10T10:48:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:34 smithi082 ceph-mon[98694]: pgmap v22208: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 11 op/s 2023-09-10T10:48:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41344.704129 secs 2023-09-10T10:48:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41345.931501 secs 2023-09-10T10:48:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:34 smithi098 ceph-mon[103124]: pgmap v22208: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 11 op/s 2023-09-10T10:48:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41344.704129 secs 2023-09-10T10:48:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41345.931501 secs 2023-09-10T10:48:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41346.282813 secs 2023-09-10T10:48:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41346.282813 secs 2023-09-10T10:48:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41346.282813 secs 2023-09-10T10:48:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:36 smithi170 ceph-mon[102832]: pgmap v22209: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 11 op/s 2023-09-10T10:48:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:36 smithi082 ceph-mon[98694]: pgmap v22209: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 11 op/s 2023-09-10T10:48:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:36 smithi098 ceph-mon[103124]: pgmap v22209: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 11 op/s 2023-09-10T10:48:38.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:38 smithi170 ceph-mon[102832]: pgmap v22210: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 426 B/s wr, 16 op/s 2023-09-10T10:48:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:38 smithi082 ceph-mon[98694]: pgmap v22210: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 426 B/s wr, 16 op/s 2023-09-10T10:48:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:38 smithi098 ceph-mon[103124]: pgmap v22210: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 426 B/s wr, 16 op/s 2023-09-10T10:48:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41350.931561 secs 2023-09-10T10:48:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:48:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41350.931561 secs 2023-09-10T10:48:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:48:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41350.931561 secs 2023-09-10T10:48:39.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:48:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:40 smithi170 ceph-mon[102832]: pgmap v22211: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 11 op/s 2023-09-10T10:48:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41351.282877 secs 2023-09-10T10:48:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:48:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:48:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:40 smithi082 ceph-mon[98694]: pgmap v22211: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 11 op/s 2023-09-10T10:48:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41351.282877 secs 2023-09-10T10:48:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:48:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:48:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:40 smithi098 ceph-mon[103124]: pgmap v22211: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 11 op/s 2023-09-10T10:48:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41351.282877 secs 2023-09-10T10:48:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:48:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:48:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:42 smithi170 ceph-mon[102832]: pgmap v22212: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 297 KiB/s wr, 17 op/s 2023-09-10T10:48:42.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:42 smithi082 ceph-mon[98694]: pgmap v22212: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 297 KiB/s wr, 17 op/s 2023-09-10T10:48:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:42 smithi098 ceph-mon[103124]: pgmap v22212: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 297 KiB/s wr, 17 op/s 2023-09-10T10:48:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:44 smithi170 ceph-mon[102832]: pgmap v22213: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T10:48:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41355.931621 secs 2023-09-10T10:48:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:44 smithi082 ceph-mon[98694]: pgmap v22213: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T10:48:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41355.931621 secs 2023-09-10T10:48:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:44 smithi098 ceph-mon[103124]: pgmap v22213: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T10:48:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41355.931621 secs 2023-09-10T10:48:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41356.282944 secs 2023-09-10T10:48:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41356.282944 secs 2023-09-10T10:48:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41356.282944 secs 2023-09-10T10:48:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:46 smithi170 ceph-mon[102832]: pgmap v22214: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T10:48:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:48:46.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:46 smithi082 ceph-mon[98694]: pgmap v22214: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T10:48:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:48:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:46 smithi098 ceph-mon[103124]: pgmap v22214: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 296 KiB/s wr, 11 op/s 2023-09-10T10:48:46.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:48:46.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:48 smithi170 ceph-mon[102832]: pgmap v22215: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 481 KiB/s wr, 17 op/s 2023-09-10T10:48:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:48 smithi082 ceph-mon[98694]: pgmap v22215: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 481 KiB/s wr, 17 op/s 2023-09-10T10:48:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:48 smithi098 ceph-mon[103124]: pgmap v22215: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 481 KiB/s wr, 17 op/s 2023-09-10T10:48:49.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41360.931702 secs 2023-09-10T10:48:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41360.931702 secs 2023-09-10T10:48:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41360.931702 secs 2023-09-10T10:48:50.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:50 smithi170 ceph-mon[102832]: pgmap v22216: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 481 KiB/s wr, 11 op/s 2023-09-10T10:48:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41361.283009 secs 2023-09-10T10:48:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:48:50.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:50 smithi082 ceph-mon[98694]: pgmap v22216: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 481 KiB/s wr, 11 op/s 2023-09-10T10:48:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41361.283009 secs 2023-09-10T10:48:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:48:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:50 smithi098 ceph-mon[103124]: pgmap v22216: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 481 KiB/s wr, 11 op/s 2023-09-10T10:48:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41361.283009 secs 2023-09-10T10:48:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:48:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:52 smithi170 ceph-mon[102832]: pgmap v22217: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 541 KiB/s wr, 18 op/s 2023-09-10T10:48:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:48:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:52 smithi082 ceph-mon[98694]: pgmap v22217: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 541 KiB/s wr, 18 op/s 2023-09-10T10:48:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:48:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:52 smithi098 ceph-mon[103124]: pgmap v22217: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 541 KiB/s wr, 18 op/s 2023-09-10T10:48:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:48:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:53.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:53 smithi098 ceph-mon[103124]: pgmap v22218: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 245 KiB/s wr, 12 op/s 2023-09-10T10:48:53.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:53 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:48:53.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:53 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:53 smithi170 ceph-mon[102832]: pgmap v22218: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 245 KiB/s wr, 12 op/s 2023-09-10T10:48:53.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:53 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:48:53.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:53 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:53 smithi082 ceph-mon[98694]: pgmap v22218: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 245 KiB/s wr, 12 op/s 2023-09-10T10:48:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:53 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:48:53.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:53 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41365.931761 secs 2023-09-10T10:48:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41365.931761 secs 2023-09-10T10:48:54.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41365.931761 secs 2023-09-10T10:48:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41366.283073 secs 2023-09-10T10:48:55.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:55 smithi098 ceph-mon[103124]: pgmap v22219: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 245 KiB/s wr, 12 op/s 2023-09-10T10:48:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41366.283073 secs 2023-09-10T10:48:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:55 smithi170 ceph-mon[102832]: pgmap v22219: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 245 KiB/s wr, 12 op/s 2023-09-10T10:48:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41366.283073 secs 2023-09-10T10:48:55.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:55 smithi082 ceph-mon[98694]: pgmap v22219: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 245 KiB/s wr, 12 op/s 2023-09-10T10:48:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:48:56.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:48:56.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:48:56.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:57 smithi098 ceph-mon[103124]: pgmap v22220: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 818 KiB/s wr, 20 op/s 2023-09-10T10:48:57.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:57 smithi170 ceph-mon[102832]: pgmap v22220: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 818 KiB/s wr, 20 op/s 2023-09-10T10:48:57.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:57 smithi082 ceph-mon[98694]: pgmap v22220: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 818 KiB/s wr, 20 op/s 2023-09-10T10:48:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:48:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:48:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:48:58.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:48:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:59 smithi098 ceph-mon[103124]: pgmap v22221: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 634 KiB/s wr, 14 op/s 2023-09-10T10:48:59.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:48:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41370.931821 secs 2023-09-10T10:48:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:59 smithi170 ceph-mon[102832]: pgmap v22221: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 634 KiB/s wr, 14 op/s 2023-09-10T10:48:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:48:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41370.931821 secs 2023-09-10T10:48:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:59 smithi082 ceph-mon[98694]: pgmap v22221: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 634 KiB/s wr, 14 op/s 2023-09-10T10:48:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:48:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41370.931821 secs 2023-09-10T10:49:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41371.283141 secs 2023-09-10T10:49:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:49:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41371.283141 secs 2023-09-10T10:49:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:49:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:00.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41371.283141 secs 2023-09-10T10:49:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:49:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:01 smithi098 ceph-mon[103124]: pgmap v22222: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 669 KiB/s wr, 22 op/s 2023-09-10T10:49:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:01 smithi170 ceph-mon[102832]: pgmap v22222: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 669 KiB/s wr, 22 op/s 2023-09-10T10:49:01.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:01 smithi082 ceph-mon[98694]: pgmap v22222: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 669 KiB/s wr, 22 op/s 2023-09-10T10:49:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:03 smithi082 ceph-mon[98694]: pgmap v22223: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 608 KiB/s wr, 15 op/s 2023-09-10T10:49:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41374.704625 secs 2023-09-10T10:49:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:03 smithi098 ceph-mon[103124]: pgmap v22223: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 608 KiB/s wr, 15 op/s 2023-09-10T10:49:04.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41374.704625 secs 2023-09-10T10:49:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:03 smithi170 ceph-mon[102832]: pgmap v22223: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 608 KiB/s wr, 15 op/s 2023-09-10T10:49:04.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41374.704625 secs 2023-09-10T10:49:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41375.931905 secs 2023-09-10T10:49:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:49:04.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:49:04.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41375.931905 secs 2023-09-10T10:49:05.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:49:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:49:05.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41375.931905 secs 2023-09-10T10:49:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:49:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:49:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:05.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41376.283203 secs 2023-09-10T10:49:05.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:05 smithi082 ceph-mon[98694]: pgmap v22224: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 608 KiB/s wr, 15 op/s 2023-09-10T10:49:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41376.283203 secs 2023-09-10T10:49:06.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:05 smithi098 ceph-mon[103124]: pgmap v22224: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 608 KiB/s wr, 15 op/s 2023-09-10T10:49:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41376.283203 secs 2023-09-10T10:49:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:05 smithi170 ceph-mon[102832]: pgmap v22224: 97 pgs: 97 active+clean; 14 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 608 KiB/s wr, 15 op/s 2023-09-10T10:49:07.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:49:07.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:49:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:49:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:07 smithi098 ceph-mon[103124]: pgmap v22225: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 36 op/s 2023-09-10T10:49:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:07 smithi170 ceph-mon[102832]: pgmap v22225: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 36 op/s 2023-09-10T10:49:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:07 smithi082 ceph-mon[98694]: pgmap v22225: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 36 op/s 2023-09-10T10:49:09.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41379.704687 secs 2023-09-10T10:49:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41379.704687 secs 2023-09-10T10:49:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41379.704687 secs 2023-09-10T10:49:10.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:09 smithi098 ceph-mon[103124]: pgmap v22226: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 620 KiB/s wr, 28 op/s 2023-09-10T10:49:10.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41380.931965 secs 2023-09-10T10:49:10.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:09 smithi098 ceph-mon[103124]: osdmap e1457: 12 total, 12 up, 12 in 2023-09-10T10:49:10.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:09 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:49:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:09 smithi170 ceph-mon[102832]: pgmap v22226: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 620 KiB/s wr, 28 op/s 2023-09-10T10:49:10.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41380.931965 secs 2023-09-10T10:49:10.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:09 smithi170 ceph-mon[102832]: osdmap e1457: 12 total, 12 up, 12 in 2023-09-10T10:49:10.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:09 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:49:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:09 smithi082 ceph-mon[98694]: pgmap v22226: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 620 KiB/s wr, 28 op/s 2023-09-10T10:49:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41380.931965 secs 2023-09-10T10:49:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:09 smithi082 ceph-mon[98694]: osdmap e1457: 12 total, 12 up, 12 in 2023-09-10T10:49:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:09 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:49:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41381.283267 secs 2023-09-10T10:49:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41381.283267 secs 2023-09-10T10:49:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41381.283267 secs 2023-09-10T10:49:12.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:12 smithi170 ceph-mon[102832]: pgmap v22228: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 739 KiB/s wr, 31 op/s 2023-09-10T10:49:12.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:49:12.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:49:12.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:49:12.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:49:12.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:49:12.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:12 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:49:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:12 smithi082 ceph-mon[98694]: pgmap v22228: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 739 KiB/s wr, 31 op/s 2023-09-10T10:49:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:49:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:49:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:49:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:49:12.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:49:12.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:12 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:49:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:12 smithi098 ceph-mon[103124]: pgmap v22228: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 739 KiB/s wr, 31 op/s 2023-09-10T10:49:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:49:12.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:49:12.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:49:12.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:49:12.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:49:12.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:12 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:49:13.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:49:13.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:49:13.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:49:13.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:49:13.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:49:13.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:49:13.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:49:13.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:49:13.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:49:13.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:49:13.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:49:13.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:49:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:49:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:49:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:49:13.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:49:13.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:49:13.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:49:13.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:49:13.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:49:13.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:49:13.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:49:13.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:49:13.710 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:49:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:49:13.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:49:13.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:49:13.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:49:13.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:49:13.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:49:13.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:49:13.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:49:13.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-10T10:49:13.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:49:13.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:49:13.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:49:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:14 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T10:49:14.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:14 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T10:49:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:14 smithi170 ceph-mon[102832]: pgmap v22229: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 739 KiB/s wr, 31 op/s 2023-09-10T10:49:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:14 smithi170 ceph-mon[102832]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T10:49:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:14 smithi170 ceph-mon[102832]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T10:49:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41384.704746 secs 2023-09-10T10:49:14.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41385.932028 secs 2023-09-10T10:49:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:14 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T10:49:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:14 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T10:49:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:14 smithi082 ceph-mon[98694]: pgmap v22229: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 739 KiB/s wr, 31 op/s 2023-09-10T10:49:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:14 smithi082 ceph-mon[98694]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T10:49:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:14 smithi082 ceph-mon[98694]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T10:49:14.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41384.704746 secs 2023-09-10T10:49:14.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41385.932028 secs 2023-09-10T10:49:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:14 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-09-10T10:49:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:14 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-09-10T10:49:14.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:14 smithi098 ceph-mon[103124]: pgmap v22229: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 739 KiB/s wr, 31 op/s 2023-09-10T10:49:14.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:14 smithi098 ceph-mon[103124]: Adjusting osd_memory_target on smithi098 to 196.2M 2023-09-10T10:49:14.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:14 smithi098 ceph-mon[103124]: Unable to set osd_memory_target on smithi098 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-10T10:49:14.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41384.704746 secs 2023-09-10T10:49:14.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41385.932028 secs 2023-09-10T10:49:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41386.283334 secs 2023-09-10T10:49:15.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:15 smithi170 ceph-mon[102832]: osdmap e1458: 12 total, 12 up, 12 in 2023-09-10T10:49:15.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41386.283334 secs 2023-09-10T10:49:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:15 smithi082 ceph-mon[98694]: osdmap e1458: 12 total, 12 up, 12 in 2023-09-10T10:49:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41386.283334 secs 2023-09-10T10:49:15.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:15 smithi098 ceph-mon[103124]: osdmap e1458: 12 total, 12 up, 12 in 2023-09-10T10:49:16.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:16 smithi170 ceph-mon[102832]: pgmap v22230: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 739 KiB/s wr, 31 op/s 2023-09-10T10:49:16.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:16 smithi082 ceph-mon[98694]: pgmap v22230: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 739 KiB/s wr, 31 op/s 2023-09-10T10:49:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:16 smithi098 ceph-mon[103124]: pgmap v22230: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 739 KiB/s wr, 31 op/s 2023-09-10T10:49:18.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:18 smithi170 ceph-mon[102832]: pgmap v22232: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 269 KiB/s wr, 17 op/s 2023-09-10T10:49:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:18 smithi082 ceph-mon[98694]: pgmap v22232: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 269 KiB/s wr, 17 op/s 2023-09-10T10:49:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:18 smithi098 ceph-mon[103124]: pgmap v22232: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 269 KiB/s wr, 17 op/s 2023-09-10T10:49:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41389.704773 secs 2023-09-10T10:49:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41390.932088 secs 2023-09-10T10:49:19.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41389.704773 secs 2023-09-10T10:49:19.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41390.932088 secs 2023-09-10T10:49:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41389.704773 secs 2023-09-10T10:49:19.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41390.932088 secs 2023-09-10T10:49:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:20 smithi170 ceph-mon[102832]: pgmap v22233: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 217 KiB/s wr, 14 op/s 2023-09-10T10:49:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41391.283397 secs 2023-09-10T10:49:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:20 smithi082 ceph-mon[98694]: pgmap v22233: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 217 KiB/s wr, 14 op/s 2023-09-10T10:49:20.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41391.283397 secs 2023-09-10T10:49:20.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:20 smithi098 ceph-mon[103124]: pgmap v22233: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 217 KiB/s wr, 14 op/s 2023-09-10T10:49:20.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41391.283397 secs 2023-09-10T10:49:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:22 smithi170 ceph-mon[102832]: pgmap v22234: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 215 KiB/s wr, 13 op/s 2023-09-10T10:49:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:22 smithi082 ceph-mon[98694]: pgmap v22234: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 215 KiB/s wr, 13 op/s 2023-09-10T10:49:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:22 smithi098 ceph-mon[103124]: pgmap v22234: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 215 KiB/s wr, 13 op/s 2023-09-10T10:49:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:24 smithi170 ceph-mon[102832]: pgmap v22235: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 215 KiB/s wr, 13 op/s 2023-09-10T10:49:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41394.704832 secs 2023-09-10T10:49:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41395.932148 secs 2023-09-10T10:49:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:24 smithi082 ceph-mon[98694]: pgmap v22235: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 215 KiB/s wr, 13 op/s 2023-09-10T10:49:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41394.704832 secs 2023-09-10T10:49:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41395.932148 secs 2023-09-10T10:49:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:24 smithi098 ceph-mon[103124]: pgmap v22235: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 215 KiB/s wr, 13 op/s 2023-09-10T10:49:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41394.704832 secs 2023-09-10T10:49:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41395.932148 secs 2023-09-10T10:49:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41396.283463 secs 2023-09-10T10:49:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41396.283463 secs 2023-09-10T10:49:25.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41396.283463 secs 2023-09-10T10:49:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:26 smithi170 ceph-mon[102832]: pgmap v22236: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 215 KiB/s wr, 13 op/s 2023-09-10T10:49:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:26 smithi082 ceph-mon[98694]: pgmap v22236: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 215 KiB/s wr, 13 op/s 2023-09-10T10:49:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:26 smithi098 ceph-mon[103124]: pgmap v22236: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 215 KiB/s wr, 13 op/s 2023-09-10T10:49:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:28 smithi170 ceph-mon[102832]: pgmap v22237: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 12 op/s 2023-09-10T10:49:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:28 smithi082 ceph-mon[98694]: pgmap v22237: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 12 op/s 2023-09-10T10:49:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:28 smithi098 ceph-mon[103124]: pgmap v22237: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 12 op/s 2023-09-10T10:49:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41399.704894 secs 2023-09-10T10:49:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41400.932214 secs 2023-09-10T10:49:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41399.704894 secs 2023-09-10T10:49:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41400.932214 secs 2023-09-10T10:49:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41399.704894 secs 2023-09-10T10:49:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41400.932214 secs 2023-09-10T10:49:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:30 smithi170 ceph-mon[102832]: pgmap v22238: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T10:49:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41401.283487 secs 2023-09-10T10:49:30.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:49:30.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:30 smithi082 ceph-mon[98694]: pgmap v22238: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T10:49:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41401.283487 secs 2023-09-10T10:49:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:49:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:30 smithi098 ceph-mon[103124]: pgmap v22238: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 11 op/s 2023-09-10T10:49:30.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41401.283487 secs 2023-09-10T10:49:30.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:49:30.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:32 smithi170 ceph-mon[102832]: pgmap v22239: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 185 KiB/s wr, 16 op/s 2023-09-10T10:49:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:49:32.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:32 smithi082 ceph-mon[98694]: pgmap v22239: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 185 KiB/s wr, 16 op/s 2023-09-10T10:49:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:49:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:32 smithi098 ceph-mon[103124]: pgmap v22239: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 185 KiB/s wr, 16 op/s 2023-09-10T10:49:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:49:32.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:34 smithi170 ceph-mon[102832]: pgmap v22240: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T10:49:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41404.704957 secs 2023-09-10T10:49:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41405.932297 secs 2023-09-10T10:49:34.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:34 smithi082 ceph-mon[98694]: pgmap v22240: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T10:49:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41404.704957 secs 2023-09-10T10:49:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41405.932297 secs 2023-09-10T10:49:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:34 smithi098 ceph-mon[103124]: pgmap v22240: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T10:49:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41404.704957 secs 2023-09-10T10:49:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41405.932297 secs 2023-09-10T10:49:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41406.283551 secs 2023-09-10T10:49:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41406.283551 secs 2023-09-10T10:49:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41406.283551 secs 2023-09-10T10:49:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:36 smithi170 ceph-mon[102832]: pgmap v22241: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T10:49:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:36 smithi082 ceph-mon[98694]: pgmap v22241: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T10:49:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:36 smithi098 ceph-mon[103124]: pgmap v22241: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 155 KiB/s wr, 11 op/s 2023-09-10T10:49:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:38 smithi170 ceph-mon[102832]: pgmap v22242: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 16 op/s 2023-09-10T10:49:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:38 smithi082 ceph-mon[98694]: pgmap v22242: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 16 op/s 2023-09-10T10:49:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:38 smithi098 ceph-mon[103124]: pgmap v22242: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 16 op/s 2023-09-10T10:49:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41410.932356 secs 2023-09-10T10:49:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:49:39.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41410.932356 secs 2023-09-10T10:49:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:49:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41410.932356 secs 2023-09-10T10:49:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:49:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:40 smithi170 ceph-mon[102832]: pgmap v22243: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T10:49:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41411.283616 secs 2023-09-10T10:49:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:49:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:49:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:40 smithi082 ceph-mon[98694]: pgmap v22243: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T10:49:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41411.283616 secs 2023-09-10T10:49:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:49:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:49:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:40 smithi098 ceph-mon[103124]: pgmap v22243: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 154 KiB/s wr, 11 op/s 2023-09-10T10:49:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41411.283616 secs 2023-09-10T10:49:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:49:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:49:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:41.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:41 smithi082 ceph-mon[98694]: pgmap v22244: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 323 KiB/s wr, 17 op/s 2023-09-10T10:49:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:41 smithi098 ceph-mon[103124]: pgmap v22244: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 323 KiB/s wr, 17 op/s 2023-09-10T10:49:41.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:41 smithi170 ceph-mon[102832]: pgmap v22244: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 323 KiB/s wr, 17 op/s 2023-09-10T10:49:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:43 smithi098 ceph-mon[103124]: pgmap v22245: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 169 KiB/s wr, 11 op/s 2023-09-10T10:49:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:43 smithi170 ceph-mon[102832]: pgmap v22245: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 169 KiB/s wr, 11 op/s 2023-09-10T10:49:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:43 smithi082 ceph-mon[98694]: pgmap v22245: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 169 KiB/s wr, 11 op/s 2023-09-10T10:49:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41415.932425 secs 2023-09-10T10:49:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:49:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41415.932425 secs 2023-09-10T10:49:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:49:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:45.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41415.932425 secs 2023-09-10T10:49:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:49:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:46.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41416.283680 secs 2023-09-10T10:49:46.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:45 smithi098 ceph-mon[103124]: pgmap v22246: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 169 KiB/s wr, 11 op/s 2023-09-10T10:49:46.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41416.283680 secs 2023-09-10T10:49:46.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:45 smithi170 ceph-mon[102832]: pgmap v22246: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 169 KiB/s wr, 11 op/s 2023-09-10T10:49:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41416.283680 secs 2023-09-10T10:49:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:45 smithi082 ceph-mon[98694]: pgmap v22246: 97 pgs: 97 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 169 KiB/s wr, 11 op/s 2023-09-10T10:49:47.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:49:47.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:49:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:49:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:48.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:47 smithi098 ceph-mon[103124]: pgmap v22247: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 672 KiB/s wr, 18 op/s 2023-09-10T10:49:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:47 smithi170 ceph-mon[102832]: pgmap v22247: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 672 KiB/s wr, 18 op/s 2023-09-10T10:49:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:47 smithi082 ceph-mon[98694]: pgmap v22247: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 672 KiB/s wr, 18 op/s 2023-09-10T10:49:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:49 smithi098 ceph-mon[103124]: pgmap v22248: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 671 KiB/s wr, 12 op/s 2023-09-10T10:49:50.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41420.932486 secs 2023-09-10T10:49:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:49 smithi170 ceph-mon[102832]: pgmap v22248: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 671 KiB/s wr, 12 op/s 2023-09-10T10:49:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41420.932486 secs 2023-09-10T10:49:50.207 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:49 smithi082 ceph-mon[98694]: pgmap v22248: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 671 KiB/s wr, 12 op/s 2023-09-10T10:49:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41420.932486 secs 2023-09-10T10:49:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41421.283741 secs 2023-09-10T10:49:51.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:49:51.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41421.283741 secs 2023-09-10T10:49:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:49:51.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41421.283741 secs 2023-09-10T10:49:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:49:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:52 smithi170 ceph-mon[102832]: pgmap v22249: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 702 KiB/s wr, 19 op/s 2023-09-10T10:49:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:49:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:52 smithi082 ceph-mon[98694]: pgmap v22249: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 702 KiB/s wr, 19 op/s 2023-09-10T10:49:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:49:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:52 smithi098 ceph-mon[103124]: pgmap v22249: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 702 KiB/s wr, 19 op/s 2023-09-10T10:49:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:49:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:54 smithi170 ceph-mon[102832]: pgmap v22250: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 533 KiB/s wr, 13 op/s 2023-09-10T10:49:54.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:49:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41425.932543 secs 2023-09-10T10:49:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:54 smithi082 ceph-mon[98694]: pgmap v22250: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 533 KiB/s wr, 13 op/s 2023-09-10T10:49:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:49:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41425.932543 secs 2023-09-10T10:49:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:54 smithi098 ceph-mon[103124]: pgmap v22250: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 533 KiB/s wr, 13 op/s 2023-09-10T10:49:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:49:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41425.932543 secs 2023-09-10T10:49:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41426.283807 secs 2023-09-10T10:49:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41426.283807 secs 2023-09-10T10:49:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41426.283807 secs 2023-09-10T10:49:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:56 smithi170 ceph-mon[102832]: pgmap v22251: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 533 KiB/s wr, 13 op/s 2023-09-10T10:49:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:49:56.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:56 smithi082 ceph-mon[98694]: pgmap v22251: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 533 KiB/s wr, 13 op/s 2023-09-10T10:49:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:49:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:56 smithi098 ceph-mon[103124]: pgmap v22251: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 533 KiB/s wr, 13 op/s 2023-09-10T10:49:56.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:49:56.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:49:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:58 smithi170 ceph-mon[102832]: pgmap v22252: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T10:49:58.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:58 smithi082 ceph-mon[98694]: pgmap v22252: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T10:49:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:58 smithi098 ceph-mon[103124]: pgmap v22252: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T10:49:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:49:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41430.932601 secs 2023-09-10T10:49:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:49:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41430.932601 secs 2023-09-10T10:49:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:49:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41430.932601 secs 2023-09-10T10:50:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:00 smithi170 ceph-mon[102832]: pgmap v22253: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 574 KiB/s wr, 14 op/s 2023-09-10T10:50:00.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41431.283879 secs 2023-09-10T10:50:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:00 smithi170 ceph-mon[102832]: overall HEALTH_WARN 2 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T10:50:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:00 smithi082 ceph-mon[98694]: pgmap v22253: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 574 KiB/s wr, 14 op/s 2023-09-10T10:50:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41431.283879 secs 2023-09-10T10:50:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:00 smithi082 ceph-mon[98694]: overall HEALTH_WARN 2 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T10:50:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:00 smithi098 ceph-mon[103124]: pgmap v22253: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 574 KiB/s wr, 14 op/s 2023-09-10T10:50:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41431.283879 secs 2023-09-10T10:50:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:00 smithi098 ceph-mon[103124]: overall HEALTH_WARN 2 MDSs report slow requests; 2 MDSs behind on trimming 2023-09-10T10:50:02.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:02 smithi170 ceph-mon[102832]: pgmap v22254: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 647 KiB/s wr, 20 op/s 2023-09-10T10:50:02.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:02 smithi082 ceph-mon[98694]: pgmap v22254: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 647 KiB/s wr, 20 op/s 2023-09-10T10:50:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:02 smithi098 ceph-mon[103124]: pgmap v22254: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 647 KiB/s wr, 20 op/s 2023-09-10T10:50:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:03 smithi082 ceph-mon[98694]: pgmap v22255: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 616 KiB/s wr, 13 op/s 2023-09-10T10:50:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:03 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:50:03.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:03 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:50:03.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:03 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:03 smithi098 ceph-mon[103124]: pgmap v22255: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 616 KiB/s wr, 13 op/s 2023-09-10T10:50:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:03 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:50:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:03 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:50:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:03 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:03 smithi170 ceph-mon[102832]: pgmap v22255: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 616 KiB/s wr, 13 op/s 2023-09-10T10:50:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:03 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:50:03.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:03 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:50:03.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:03 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41434.705463 secs 2023-09-10T10:50:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41435.932672 secs 2023-09-10T10:50:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41434.705463 secs 2023-09-10T10:50:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41435.932672 secs 2023-09-10T10:50:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41434.705463 secs 2023-09-10T10:50:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41435.932672 secs 2023-09-10T10:50:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41436.283940 secs 2023-09-10T10:50:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:05 smithi082 ceph-mon[98694]: pgmap v22256: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 616 KiB/s wr, 13 op/s 2023-09-10T10:50:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41436.283940 secs 2023-09-10T10:50:05.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:05 smithi098 ceph-mon[103124]: pgmap v22256: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 616 KiB/s wr, 13 op/s 2023-09-10T10:50:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41436.283940 secs 2023-09-10T10:50:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:05 smithi170 ceph-mon[102832]: pgmap v22256: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 616 KiB/s wr, 13 op/s 2023-09-10T10:50:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:50:06.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:50:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:50:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:07 smithi098 ceph-mon[103124]: pgmap v22257: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 35 op/s 2023-09-10T10:50:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:07 smithi170 ceph-mon[102832]: pgmap v22257: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 35 op/s 2023-09-10T10:50:07.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:07 smithi082 ceph-mon[98694]: pgmap v22257: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 35 op/s 2023-09-10T10:50:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41439.705526 secs 2023-09-10T10:50:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:09 smithi098 ceph-mon[103124]: osdmap e1459: 12 total, 12 up, 12 in 2023-09-10T10:50:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:09 smithi098 ceph-mon[103124]: pgmap v22259: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 939 KiB/s wr, 33 op/s 2023-09-10T10:50:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41440.932736 secs 2023-09-10T10:50:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41439.705526 secs 2023-09-10T10:50:09.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:09 smithi170 ceph-mon[102832]: osdmap e1459: 12 total, 12 up, 12 in 2023-09-10T10:50:09.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:09 smithi170 ceph-mon[102832]: pgmap v22259: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 939 KiB/s wr, 33 op/s 2023-09-10T10:50:09.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41440.932736 secs 2023-09-10T10:50:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41439.705526 secs 2023-09-10T10:50:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:09 smithi082 ceph-mon[98694]: osdmap e1459: 12 total, 12 up, 12 in 2023-09-10T10:50:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:09 smithi082 ceph-mon[98694]: pgmap v22259: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 939 KiB/s wr, 33 op/s 2023-09-10T10:50:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41440.932736 secs 2023-09-10T10:50:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41441.284003 secs 2023-09-10T10:50:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:10 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:50:10.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:10 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41441.284003 secs 2023-09-10T10:50:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:10 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:50:10.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:10 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41441.284003 secs 2023-09-10T10:50:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:10 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:50:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:10 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:11 smithi098 ceph-mon[103124]: pgmap v22260: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 34 op/s 2023-09-10T10:50:11.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:11 smithi170 ceph-mon[102832]: pgmap v22260: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 34 op/s 2023-09-10T10:50:11.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:11 smithi082 ceph-mon[98694]: pgmap v22260: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 34 op/s 2023-09-10T10:50:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:50:12.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:50:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:50:12.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:13 smithi098 ceph-mon[103124]: pgmap v22261: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 34 op/s 2023-09-10T10:50:13.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:50:13.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41444.705591 secs 2023-09-10T10:50:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:13 smithi170 ceph-mon[102832]: pgmap v22261: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 34 op/s 2023-09-10T10:50:13.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:50:13.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41444.705591 secs 2023-09-10T10:50:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:13 smithi082 ceph-mon[98694]: pgmap v22261: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 34 op/s 2023-09-10T10:50:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:50:13.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41444.705591 secs 2023-09-10T10:50:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:50:14.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:50:14.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:50:14.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41445.932795 secs 2023-09-10T10:50:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:50:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:50:14.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:50:14.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41445.932795 secs 2023-09-10T10:50:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:50:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:50:14.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:50:14.959 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41445.932795 secs 2023-09-10T10:50:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41446.284066 secs 2023-09-10T10:50:15.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:15 smithi098 ceph-mon[103124]: pgmap v22262: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 34 op/s 2023-09-10T10:50:15.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:15 smithi098 ceph-mon[103124]: osdmap e1460: 12 total, 12 up, 12 in 2023-09-10T10:50:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41446.284066 secs 2023-09-10T10:50:15.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:15 smithi170 ceph-mon[102832]: pgmap v22262: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 34 op/s 2023-09-10T10:50:15.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:15 smithi170 ceph-mon[102832]: osdmap e1460: 12 total, 12 up, 12 in 2023-09-10T10:50:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41446.284066 secs 2023-09-10T10:50:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:15 smithi082 ceph-mon[98694]: pgmap v22262: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 34 op/s 2023-09-10T10:50:15.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:15 smithi082 ceph-mon[98694]: osdmap e1460: 12 total, 12 up, 12 in 2023-09-10T10:50:18.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:17 smithi098 ceph-mon[103124]: pgmap v22264: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 712 KiB/s wr, 17 op/s 2023-09-10T10:50:18.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:17 smithi170 ceph-mon[102832]: pgmap v22264: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 712 KiB/s wr, 17 op/s 2023-09-10T10:50:18.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:17 smithi082 ceph-mon[98694]: pgmap v22264: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 712 KiB/s wr, 17 op/s 2023-09-10T10:50:19.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:18 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:50:19.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:18 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:19.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41449.705650 secs 2023-09-10T10:50:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:18 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:50:19.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:18 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:19.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41449.705650 secs 2023-09-10T10:50:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:18 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:50:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:18 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:19.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41449.705650 secs 2023-09-10T10:50:20.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:19 smithi098 ceph-mon[103124]: pgmap v22265: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 584 KiB/s wr, 14 op/s 2023-09-10T10:50:20.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41450.932852 secs 2023-09-10T10:50:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:19 smithi170 ceph-mon[102832]: pgmap v22265: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 584 KiB/s wr, 14 op/s 2023-09-10T10:50:20.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41450.932852 secs 2023-09-10T10:50:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:19 smithi082 ceph-mon[98694]: pgmap v22265: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 584 KiB/s wr, 14 op/s 2023-09-10T10:50:20.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41450.932852 secs 2023-09-10T10:50:21.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41451.284131 secs 2023-09-10T10:50:21.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:20 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:50:21.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:20 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41451.284131 secs 2023-09-10T10:50:21.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:20 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:50:21.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:20 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41451.284131 secs 2023-09-10T10:50:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:20 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:50:21.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:20 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:22 smithi170 ceph-mon[102832]: pgmap v22266: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 13 op/s 2023-09-10T10:50:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:22 smithi082 ceph-mon[98694]: pgmap v22266: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 13 op/s 2023-09-10T10:50:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:22 smithi098 ceph-mon[103124]: pgmap v22266: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 13 op/s 2023-09-10T10:50:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:24 smithi170 ceph-mon[102832]: pgmap v22267: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 13 op/s 2023-09-10T10:50:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41454.705752 secs 2023-09-10T10:50:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41455.932911 secs 2023-09-10T10:50:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:24 smithi082 ceph-mon[98694]: pgmap v22267: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 13 op/s 2023-09-10T10:50:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41454.705752 secs 2023-09-10T10:50:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41455.932911 secs 2023-09-10T10:50:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:24 smithi098 ceph-mon[103124]: pgmap v22267: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 13 op/s 2023-09-10T10:50:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41454.705752 secs 2023-09-10T10:50:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41455.932911 secs 2023-09-10T10:50:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41456.284200 secs 2023-09-10T10:50:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41456.284200 secs 2023-09-10T10:50:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41456.284200 secs 2023-09-10T10:50:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:26 smithi170 ceph-mon[102832]: pgmap v22268: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 13 op/s 2023-09-10T10:50:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:26 smithi082 ceph-mon[98694]: pgmap v22268: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 13 op/s 2023-09-10T10:50:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:26 smithi098 ceph-mon[103124]: pgmap v22268: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 185 KiB/s wr, 13 op/s 2023-09-10T10:50:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:28 smithi170 ceph-mon[102832]: pgmap v22269: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 275 KiB/s wr, 12 op/s 2023-09-10T10:50:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:28 smithi082 ceph-mon[98694]: pgmap v22269: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 275 KiB/s wr, 12 op/s 2023-09-10T10:50:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:28 smithi098 ceph-mon[103124]: pgmap v22269: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 275 KiB/s wr, 12 op/s 2023-09-10T10:50:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41459.705777 secs 2023-09-10T10:50:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41460.932996 secs 2023-09-10T10:50:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41459.705777 secs 2023-09-10T10:50:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41460.932996 secs 2023-09-10T10:50:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41459.705777 secs 2023-09-10T10:50:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41460.932996 secs 2023-09-10T10:50:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:30 smithi170 ceph-mon[102832]: pgmap v22270: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 259 KiB/s wr, 11 op/s 2023-09-10T10:50:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41461.284265 secs 2023-09-10T10:50:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:30 smithi082 ceph-mon[98694]: pgmap v22270: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 259 KiB/s wr, 11 op/s 2023-09-10T10:50:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41461.284265 secs 2023-09-10T10:50:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:30 smithi098 ceph-mon[103124]: pgmap v22270: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 259 KiB/s wr, 11 op/s 2023-09-10T10:50:30.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41461.284265 secs 2023-09-10T10:50:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:32 smithi170 ceph-mon[102832]: pgmap v22271: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 260 KiB/s wr, 17 op/s 2023-09-10T10:50:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:32 smithi082 ceph-mon[98694]: pgmap v22271: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 260 KiB/s wr, 17 op/s 2023-09-10T10:50:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:32 smithi098 ceph-mon[103124]: pgmap v22271: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 260 KiB/s wr, 17 op/s 2023-09-10T10:50:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:34 smithi170 ceph-mon[102832]: pgmap v22272: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 229 KiB/s wr, 11 op/s 2023-09-10T10:50:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41464.705856 secs 2023-09-10T10:50:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:34 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:50:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:34 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41465.933057 secs 2023-09-10T10:50:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:34 smithi082 ceph-mon[98694]: pgmap v22272: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 229 KiB/s wr, 11 op/s 2023-09-10T10:50:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41464.705856 secs 2023-09-10T10:50:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:34 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:50:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:34 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:34.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41465.933057 secs 2023-09-10T10:50:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:34 smithi098 ceph-mon[103124]: pgmap v22272: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 229 KiB/s wr, 11 op/s 2023-09-10T10:50:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41464.705856 secs 2023-09-10T10:50:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:34 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:50:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:34 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41465.933057 secs 2023-09-10T10:50:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41466.284328 secs 2023-09-10T10:50:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41466.284328 secs 2023-09-10T10:50:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41466.284328 secs 2023-09-10T10:50:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:36 smithi170 ceph-mon[102832]: pgmap v22273: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 229 KiB/s wr, 11 op/s 2023-09-10T10:50:36.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:36 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:50:36.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:36 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:36 smithi082 ceph-mon[98694]: pgmap v22273: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 229 KiB/s wr, 11 op/s 2023-09-10T10:50:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:36 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:50:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:36 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:36 smithi098 ceph-mon[103124]: pgmap v22273: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 229 KiB/s wr, 11 op/s 2023-09-10T10:50:36.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:36 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:50:36.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:36 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:38.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:38 smithi170 ceph-mon[102832]: pgmap v22274: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 411 KiB/s wr, 17 op/s 2023-09-10T10:50:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:38 smithi082 ceph-mon[98694]: pgmap v22274: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 411 KiB/s wr, 17 op/s 2023-09-10T10:50:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:38 smithi098 ceph-mon[103124]: pgmap v22274: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 411 KiB/s wr, 17 op/s 2023-09-10T10:50:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41470.933131 secs 2023-09-10T10:50:39.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:50:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41470.933131 secs 2023-09-10T10:50:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:50:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41470.933131 secs 2023-09-10T10:50:39.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:50:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:40 smithi170 ceph-mon[102832]: pgmap v22275: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T10:50:40.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41471.284392 secs 2023-09-10T10:50:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:50:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:50:40.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:40 smithi082 ceph-mon[98694]: pgmap v22275: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T10:50:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41471.284392 secs 2023-09-10T10:50:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:50:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:50:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:40 smithi098 ceph-mon[103124]: pgmap v22275: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 183 KiB/s wr, 11 op/s 2023-09-10T10:50:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41471.284392 secs 2023-09-10T10:50:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:50:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:50:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:42 smithi170 ceph-mon[102832]: pgmap v22276: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 460 KiB/s wr, 17 op/s 2023-09-10T10:50:42.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:42 smithi082 ceph-mon[98694]: pgmap v22276: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 460 KiB/s wr, 17 op/s 2023-09-10T10:50:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:42 smithi098 ceph-mon[103124]: pgmap v22276: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 460 KiB/s wr, 17 op/s 2023-09-10T10:50:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:44 smithi082 ceph-mon[98694]: pgmap v22277: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 459 KiB/s wr, 12 op/s 2023-09-10T10:50:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:50:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41475.933192 secs 2023-09-10T10:50:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:44 smithi098 ceph-mon[103124]: pgmap v22277: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 459 KiB/s wr, 12 op/s 2023-09-10T10:50:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:50:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41475.933192 secs 2023-09-10T10:50:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:44 smithi170 ceph-mon[102832]: pgmap v22277: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 459 KiB/s wr, 12 op/s 2023-09-10T10:50:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:50:44.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:44.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41475.933192 secs 2023-09-10T10:50:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41476.284456 secs 2023-09-10T10:50:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41476.284456 secs 2023-09-10T10:50:45.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41476.284456 secs 2023-09-10T10:50:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:46 smithi082 ceph-mon[98694]: pgmap v22278: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 459 KiB/s wr, 12 op/s 2023-09-10T10:50:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:46 smithi098 ceph-mon[103124]: pgmap v22278: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 459 KiB/s wr, 12 op/s 2023-09-10T10:50:46.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:46 smithi170 ceph-mon[102832]: pgmap v22278: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 459 KiB/s wr, 12 op/s 2023-09-10T10:50:47.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:47 smithi082 ceph-mon[98694]: pgmap v22279: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 817 KiB/s wr, 18 op/s 2023-09-10T10:50:47.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:47 smithi098 ceph-mon[103124]: pgmap v22279: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 817 KiB/s wr, 18 op/s 2023-09-10T10:50:47.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:47 smithi170 ceph-mon[102832]: pgmap v22279: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 817 KiB/s wr, 18 op/s 2023-09-10T10:50:50.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41480.933290 secs 2023-09-10T10:50:50.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:49 smithi098 ceph-mon[103124]: pgmap v22280: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 635 KiB/s wr, 12 op/s 2023-09-10T10:50:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41480.933290 secs 2023-09-10T10:50:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:49 smithi170 ceph-mon[102832]: pgmap v22280: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 635 KiB/s wr, 12 op/s 2023-09-10T10:50:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41480.933290 secs 2023-09-10T10:50:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:49 smithi082 ceph-mon[98694]: pgmap v22280: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 635 KiB/s wr, 12 op/s 2023-09-10T10:50:51.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41481.284488 secs 2023-09-10T10:50:51.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:50:51.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41481.284488 secs 2023-09-10T10:50:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:50:51.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41481.284488 secs 2023-09-10T10:50:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:50:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:52 smithi170 ceph-mon[102832]: pgmap v22281: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 21 op/s 2023-09-10T10:50:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:50:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:52 smithi082 ceph-mon[98694]: pgmap v22281: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 21 op/s 2023-09-10T10:50:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:50:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:52 smithi098 ceph-mon[103124]: pgmap v22281: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 21 op/s 2023-09-10T10:50:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:50:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:54 smithi170 ceph-mon[102832]: pgmap v22282: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1020 KiB/s wr, 14 op/s 2023-09-10T10:50:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:50:54.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:54.680 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41485.933349 secs 2023-09-10T10:50:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:54 smithi082 ceph-mon[98694]: pgmap v22282: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1020 KiB/s wr, 14 op/s 2023-09-10T10:50:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:50:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:54.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41485.933349 secs 2023-09-10T10:50:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:54 smithi098 ceph-mon[103124]: pgmap v22282: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1020 KiB/s wr, 14 op/s 2023-09-10T10:50:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:50:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41485.933349 secs 2023-09-10T10:50:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41486.284553 secs 2023-09-10T10:50:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41486.284553 secs 2023-09-10T10:50:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41486.284553 secs 2023-09-10T10:50:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:56 smithi170 ceph-mon[102832]: pgmap v22283: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1020 KiB/s wr, 14 op/s 2023-09-10T10:50:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:50:56.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:56 smithi082 ceph-mon[98694]: pgmap v22283: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1020 KiB/s wr, 14 op/s 2023-09-10T10:50:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:50:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:56 smithi098 ceph-mon[103124]: pgmap v22283: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1020 KiB/s wr, 14 op/s 2023-09-10T10:50:56.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:50:56.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:50:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:58 smithi170 ceph-mon[102832]: pgmap v22284: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 22 op/s 2023-09-10T10:50:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:58 smithi082 ceph-mon[98694]: pgmap v22284: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 22 op/s 2023-09-10T10:50:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:58 smithi098 ceph-mon[103124]: pgmap v22284: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 22 op/s 2023-09-10T10:50:59.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:50:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41490.933424 secs 2023-09-10T10:50:59.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:50:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41490.933424 secs 2023-09-10T10:50:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:50:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41490.933424 secs 2023-09-10T10:51:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:00 smithi170 ceph-mon[102832]: pgmap v22285: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 15 op/s 2023-09-10T10:51:00.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41491.284617 secs 2023-09-10T10:51:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:00 smithi082 ceph-mon[98694]: pgmap v22285: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 15 op/s 2023-09-10T10:51:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41491.284617 secs 2023-09-10T10:51:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:00 smithi098 ceph-mon[103124]: pgmap v22285: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 15 op/s 2023-09-10T10:51:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41491.284617 secs 2023-09-10T10:51:02.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:02 smithi170 ceph-mon[102832]: pgmap v22286: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 24 op/s 2023-09-10T10:51:02.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:02 smithi082 ceph-mon[98694]: pgmap v22286: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 24 op/s 2023-09-10T10:51:02.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:02 smithi098 ceph-mon[103124]: pgmap v22286: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 24 op/s 2023-09-10T10:51:03.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:03 smithi098 ceph-mon[103124]: pgmap v22287: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 746 KiB/s wr, 16 op/s 2023-09-10T10:51:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:03 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:51:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:03 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:51:03.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:03 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:03 smithi170 ceph-mon[102832]: pgmap v22287: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 746 KiB/s wr, 16 op/s 2023-09-10T10:51:03.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:03 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:51:03.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:03 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:51:03.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:03 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:03 smithi082 ceph-mon[98694]: pgmap v22287: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 746 KiB/s wr, 16 op/s 2023-09-10T10:51:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:03 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:51:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:03 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:51:03.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:03 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41494.706230 secs 2023-09-10T10:51:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41495.933487 secs 2023-09-10T10:51:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41494.706230 secs 2023-09-10T10:51:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41495.933487 secs 2023-09-10T10:51:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41494.706230 secs 2023-09-10T10:51:04.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41495.933487 secs 2023-09-10T10:51:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41496.284683 secs 2023-09-10T10:51:05.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:05 smithi098 ceph-mon[103124]: pgmap v22288: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 746 KiB/s wr, 16 op/s 2023-09-10T10:51:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41496.284683 secs 2023-09-10T10:51:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:05 smithi170 ceph-mon[102832]: pgmap v22288: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 746 KiB/s wr, 16 op/s 2023-09-10T10:51:05.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41496.284683 secs 2023-09-10T10:51:05.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:05 smithi082 ceph-mon[98694]: pgmap v22288: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 746 KiB/s wr, 16 op/s 2023-09-10T10:51:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:51:06.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:51:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:51:06.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:07 smithi098 ceph-mon[103124]: pgmap v22289: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 36 op/s 2023-09-10T10:51:07.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:07 smithi170 ceph-mon[102832]: pgmap v22289: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 36 op/s 2023-09-10T10:51:07.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:07 smithi082 ceph-mon[98694]: pgmap v22289: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 36 op/s 2023-09-10T10:51:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41499.706291 secs 2023-09-10T10:51:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41499.706291 secs 2023-09-10T10:51:08.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41499.706291 secs 2023-09-10T10:51:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:09 smithi098 ceph-mon[103124]: osdmap e1461: 12 total, 12 up, 12 in 2023-09-10T10:51:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41500.933546 secs 2023-09-10T10:51:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:09 smithi098 ceph-mon[103124]: pgmap v22291: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 921 KiB/s wr, 35 op/s 2023-09-10T10:51:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:09 smithi170 ceph-mon[102832]: osdmap e1461: 12 total, 12 up, 12 in 2023-09-10T10:51:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41500.933546 secs 2023-09-10T10:51:09.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:09 smithi170 ceph-mon[102832]: pgmap v22291: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 921 KiB/s wr, 35 op/s 2023-09-10T10:51:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:09 smithi082 ceph-mon[98694]: osdmap e1461: 12 total, 12 up, 12 in 2023-09-10T10:51:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41500.933546 secs 2023-09-10T10:51:09.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:09 smithi082 ceph-mon[98694]: pgmap v22291: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 921 KiB/s wr, 35 op/s 2023-09-10T10:51:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41501.284747 secs 2023-09-10T10:51:10.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41501.284747 secs 2023-09-10T10:51:11.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41501.284747 secs 2023-09-10T10:51:11.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:11 smithi170 ceph-mon[102832]: pgmap v22292: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 832 KiB/s wr, 31 op/s 2023-09-10T10:51:11.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:11 smithi082 ceph-mon[98694]: pgmap v22292: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 832 KiB/s wr, 31 op/s 2023-09-10T10:51:12.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:11 smithi098 ceph-mon[103124]: pgmap v22292: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 832 KiB/s wr, 31 op/s 2023-09-10T10:51:14.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:13 smithi098 ceph-mon[103124]: pgmap v22293: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 832 KiB/s wr, 31 op/s 2023-09-10T10:51:14.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41504.706352 secs 2023-09-10T10:51:14.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:13 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:51:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:13 smithi170 ceph-mon[102832]: pgmap v22293: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 832 KiB/s wr, 31 op/s 2023-09-10T10:51:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41504.706352 secs 2023-09-10T10:51:14.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:13 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:51:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:13 smithi082 ceph-mon[98694]: pgmap v22293: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 832 KiB/s wr, 31 op/s 2023-09-10T10:51:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41504.706352 secs 2023-09-10T10:51:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:13 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:51:15.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41505.933604 secs 2023-09-10T10:51:15.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:14 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:51:15.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:14 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:15.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:51:15.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:14 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:51:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41505.933604 secs 2023-09-10T10:51:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:14 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:51:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:14 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:51:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:14 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:51:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41505.933604 secs 2023-09-10T10:51:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:14 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:51:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:14 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:51:15.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:14 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:51:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41506.284812 secs 2023-09-10T10:51:16.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:15 smithi098 ceph-mon[103124]: pgmap v22294: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 832 KiB/s wr, 31 op/s 2023-09-10T10:51:16.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:51:16.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:51:16.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:51:16.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:15 smithi098 ceph-mon[103124]: osdmap e1462: 12 total, 12 up, 12 in 2023-09-10T10:51:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41506.284812 secs 2023-09-10T10:51:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:15 smithi170 ceph-mon[102832]: pgmap v22294: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 832 KiB/s wr, 31 op/s 2023-09-10T10:51:16.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:51:16.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:51:16.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:51:16.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:15 smithi170 ceph-mon[102832]: osdmap e1462: 12 total, 12 up, 12 in 2023-09-10T10:51:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41506.284812 secs 2023-09-10T10:51:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:15 smithi082 ceph-mon[98694]: pgmap v22294: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 832 KiB/s wr, 31 op/s 2023-09-10T10:51:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:51:16.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:51:16.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:51:16.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:15 smithi082 ceph-mon[98694]: osdmap e1462: 12 total, 12 up, 12 in 2023-09-10T10:51:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:17 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:51:17.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:17 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:17 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:51:17.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:17 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:17.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:17 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:51:17.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:17 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:18 smithi170 ceph-mon[102832]: pgmap v22296: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 665 KiB/s wr, 17 op/s 2023-09-10T10:51:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:18 smithi082 ceph-mon[98694]: pgmap v22296: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 665 KiB/s wr, 17 op/s 2023-09-10T10:51:18.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:18 smithi098 ceph-mon[103124]: pgmap v22296: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 665 KiB/s wr, 17 op/s 2023-09-10T10:51:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:19 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41509.706457 secs 2023-09-10T10:51:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41510.933687 secs 2023-09-10T10:51:19.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41509.706457 secs 2023-09-10T10:51:19.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41510.933687 secs 2023-09-10T10:51:19.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:19 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41509.706457 secs 2023-09-10T10:51:19.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41510.933687 secs 2023-09-10T10:51:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:20 smithi082 ceph-mon[98694]: pgmap v22297: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 544 KiB/s wr, 14 op/s 2023-09-10T10:51:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:20 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41511.284873 secs 2023-09-10T10:51:20.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:20 smithi098 ceph-mon[103124]: pgmap v22297: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 544 KiB/s wr, 14 op/s 2023-09-10T10:51:20.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:20 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41511.284873 secs 2023-09-10T10:51:20.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:20 smithi170 ceph-mon[102832]: pgmap v22297: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 544 KiB/s wr, 14 op/s 2023-09-10T10:51:20.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:20 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41511.284873 secs 2023-09-10T10:51:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:22 smithi170 ceph-mon[102832]: pgmap v22298: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 181 KiB/s wr, 14 op/s 2023-09-10T10:51:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:22 smithi082 ceph-mon[98694]: pgmap v22298: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 181 KiB/s wr, 14 op/s 2023-09-10T10:51:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:22 smithi098 ceph-mon[103124]: pgmap v22298: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 181 KiB/s wr, 14 op/s 2023-09-10T10:51:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:24 smithi170 ceph-mon[102832]: pgmap v22299: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 181 KiB/s wr, 14 op/s 2023-09-10T10:51:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41514.706553 secs 2023-09-10T10:51:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41515.933745 secs 2023-09-10T10:51:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:24 smithi082 ceph-mon[98694]: pgmap v22299: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 181 KiB/s wr, 14 op/s 2023-09-10T10:51:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41514.706553 secs 2023-09-10T10:51:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41515.933745 secs 2023-09-10T10:51:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:24 smithi098 ceph-mon[103124]: pgmap v22299: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 181 KiB/s wr, 14 op/s 2023-09-10T10:51:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41514.706553 secs 2023-09-10T10:51:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41515.933745 secs 2023-09-10T10:51:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41516.284938 secs 2023-09-10T10:51:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41516.284938 secs 2023-09-10T10:51:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41516.284938 secs 2023-09-10T10:51:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:26 smithi170 ceph-mon[102832]: pgmap v22300: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 181 KiB/s wr, 14 op/s 2023-09-10T10:51:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:26 smithi082 ceph-mon[98694]: pgmap v22300: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 181 KiB/s wr, 14 op/s 2023-09-10T10:51:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:26 smithi098 ceph-mon[103124]: pgmap v22300: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 181 KiB/s wr, 14 op/s 2023-09-10T10:51:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:28 smithi170 ceph-mon[102832]: pgmap v22301: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 453 B/s wr, 12 op/s 2023-09-10T10:51:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:28 smithi082 ceph-mon[98694]: pgmap v22301: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 453 B/s wr, 12 op/s 2023-09-10T10:51:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:28 smithi098 ceph-mon[103124]: pgmap v22301: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 453 B/s wr, 12 op/s 2023-09-10T10:51:29.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41519.706613 secs 2023-09-10T10:51:29.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41520.933862 secs 2023-09-10T10:51:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41519.706613 secs 2023-09-10T10:51:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41520.933862 secs 2023-09-10T10:51:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41519.706613 secs 2023-09-10T10:51:29.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41520.933862 secs 2023-09-10T10:51:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:30 smithi170 ceph-mon[102832]: pgmap v22302: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 11 op/s 2023-09-10T10:51:30.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41521.285003 secs 2023-09-10T10:51:30.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:51:30.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:30 smithi082 ceph-mon[98694]: pgmap v22302: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 11 op/s 2023-09-10T10:51:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41521.285003 secs 2023-09-10T10:51:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:51:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:30 smithi098 ceph-mon[103124]: pgmap v22302: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 11 op/s 2023-09-10T10:51:30.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41521.285003 secs 2023-09-10T10:51:30.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:51:30.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:32.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:32 smithi170 ceph-mon[102832]: pgmap v22303: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 151 KiB/s wr, 16 op/s 2023-09-10T10:51:32.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:51:32.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:32 smithi082 ceph-mon[98694]: pgmap v22303: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 151 KiB/s wr, 16 op/s 2023-09-10T10:51:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:51:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:32 smithi098 ceph-mon[103124]: pgmap v22303: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 151 KiB/s wr, 16 op/s 2023-09-10T10:51:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:51:32.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:34.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:34 smithi170 ceph-mon[102832]: pgmap v22304: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T10:51:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41524.706673 secs 2023-09-10T10:51:34.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41525.933923 secs 2023-09-10T10:51:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:34 smithi082 ceph-mon[98694]: pgmap v22304: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T10:51:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41524.706673 secs 2023-09-10T10:51:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41525.933923 secs 2023-09-10T10:51:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:34 smithi098 ceph-mon[103124]: pgmap v22304: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T10:51:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41524.706673 secs 2023-09-10T10:51:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41525.933923 secs 2023-09-10T10:51:35.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41526.285066 secs 2023-09-10T10:51:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41526.285066 secs 2023-09-10T10:51:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41526.285066 secs 2023-09-10T10:51:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:36 smithi082 ceph-mon[98694]: pgmap v22305: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T10:51:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:36 smithi098 ceph-mon[103124]: pgmap v22305: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T10:51:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:36 smithi170 ceph-mon[102832]: pgmap v22305: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 151 KiB/s wr, 10 op/s 2023-09-10T10:51:38.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:38 smithi082 ceph-mon[98694]: pgmap v22306: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 257 KiB/s wr, 15 op/s 2023-09-10T10:51:38.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:38 smithi098 ceph-mon[103124]: pgmap v22306: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 257 KiB/s wr, 15 op/s 2023-09-10T10:51:38.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:38 smithi170 ceph-mon[102832]: pgmap v22306: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 257 KiB/s wr, 15 op/s 2023-09-10T10:51:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41530.934020 secs 2023-09-10T10:51:39.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:51:39.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41530.934020 secs 2023-09-10T10:51:39.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:51:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41530.934020 secs 2023-09-10T10:51:39.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:51:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:40 smithi082 ceph-mon[98694]: pgmap v22307: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 257 KiB/s wr, 10 op/s 2023-09-10T10:51:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41531.285132 secs 2023-09-10T10:51:40.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:51:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:51:40.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:40 smithi098 ceph-mon[103124]: pgmap v22307: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 257 KiB/s wr, 10 op/s 2023-09-10T10:51:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41531.285132 secs 2023-09-10T10:51:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:51:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:51:40.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:40.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:40 smithi170 ceph-mon[102832]: pgmap v22307: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 257 KiB/s wr, 10 op/s 2023-09-10T10:51:40.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41531.285132 secs 2023-09-10T10:51:40.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:51:40.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:51:40.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:41.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:41 smithi082 ceph-mon[98694]: pgmap v22308: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 599 KiB/s wr, 16 op/s 2023-09-10T10:51:41.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:41 smithi098 ceph-mon[103124]: pgmap v22308: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 599 KiB/s wr, 16 op/s 2023-09-10T10:51:41.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:41 smithi170 ceph-mon[102832]: pgmap v22308: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 599 KiB/s wr, 16 op/s 2023-09-10T10:51:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:43 smithi098 ceph-mon[103124]: pgmap v22309: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 448 KiB/s wr, 10 op/s 2023-09-10T10:51:44.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:43 smithi170 ceph-mon[102832]: pgmap v22309: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 448 KiB/s wr, 10 op/s 2023-09-10T10:51:44.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:43 smithi082 ceph-mon[98694]: pgmap v22309: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 448 KiB/s wr, 10 op/s 2023-09-10T10:51:45.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41535.934078 secs 2023-09-10T10:51:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:44 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:51:45.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:44 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:45.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41535.934078 secs 2023-09-10T10:51:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:44 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:51:45.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:44 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41535.934078 secs 2023-09-10T10:51:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:44 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:51:45.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:44 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41536.285196 secs 2023-09-10T10:51:46.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:45 smithi170 ceph-mon[102832]: pgmap v22310: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 448 KiB/s wr, 10 op/s 2023-09-10T10:51:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41536.285196 secs 2023-09-10T10:51:46.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:45 smithi082 ceph-mon[98694]: pgmap v22310: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 448 KiB/s wr, 10 op/s 2023-09-10T10:51:46.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41536.285196 secs 2023-09-10T10:51:46.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:45 smithi098 ceph-mon[103124]: pgmap v22310: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 448 KiB/s wr, 10 op/s 2023-09-10T10:51:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:51:47.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:51:47.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:51:47.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:48.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:47 smithi170 ceph-mon[102832]: pgmap v22311: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 972 KiB/s wr, 18 op/s 2023-09-10T10:51:48.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:47 smithi082 ceph-mon[98694]: pgmap v22311: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 972 KiB/s wr, 18 op/s 2023-09-10T10:51:48.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:47 smithi098 ceph-mon[103124]: pgmap v22311: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 972 KiB/s wr, 18 op/s 2023-09-10T10:51:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41540.934147 secs 2023-09-10T10:51:50.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:49 smithi170 ceph-mon[102832]: pgmap v22312: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 866 KiB/s wr, 13 op/s 2023-09-10T10:51:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41540.934147 secs 2023-09-10T10:51:50.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:49 smithi082 ceph-mon[98694]: pgmap v22312: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 866 KiB/s wr, 13 op/s 2023-09-10T10:51:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41540.934147 secs 2023-09-10T10:51:50.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:49 smithi098 ceph-mon[103124]: pgmap v22312: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 866 KiB/s wr, 13 op/s 2023-09-10T10:51:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41541.285265 secs 2023-09-10T10:51:51.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:51:51.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41541.285265 secs 2023-09-10T10:51:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:51:51.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:51.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41541.285265 secs 2023-09-10T10:51:51.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:51:51.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:52 smithi170 ceph-mon[102832]: pgmap v22313: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T10:51:52.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:51:52.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:52 smithi082 ceph-mon[98694]: pgmap v22313: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T10:51:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:51:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:52 smithi098 ceph-mon[103124]: pgmap v22313: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 21 op/s 2023-09-10T10:51:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:51:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:54.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:54 smithi170 ceph-mon[102832]: pgmap v22314: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 777 KiB/s wr, 15 op/s 2023-09-10T10:51:54.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:51:54.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:54.682 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41545.934207 secs 2023-09-10T10:51:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:54 smithi082 ceph-mon[98694]: pgmap v22314: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 777 KiB/s wr, 15 op/s 2023-09-10T10:51:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:51:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41545.934207 secs 2023-09-10T10:51:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:54 smithi098 ceph-mon[103124]: pgmap v22314: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 777 KiB/s wr, 15 op/s 2023-09-10T10:51:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:51:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41545.934207 secs 2023-09-10T10:51:55.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41546.285326 secs 2023-09-10T10:51:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41546.285326 secs 2023-09-10T10:51:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41546.285326 secs 2023-09-10T10:51:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:56 smithi170 ceph-mon[102832]: pgmap v22315: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 777 KiB/s wr, 15 op/s 2023-09-10T10:51:56.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:51:56.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:56 smithi082 ceph-mon[98694]: pgmap v22315: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 777 KiB/s wr, 15 op/s 2023-09-10T10:51:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:51:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:56 smithi098 ceph-mon[103124]: pgmap v22315: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 777 KiB/s wr, 15 op/s 2023-09-10T10:51:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:51:56.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:58 smithi170 ceph-mon[102832]: pgmap v22316: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 23 op/s 2023-09-10T10:51:58.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:51:58.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:58 smithi082 ceph-mon[98694]: pgmap v22316: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 23 op/s 2023-09-10T10:51:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:51:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:58 smithi098 ceph-mon[103124]: pgmap v22316: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 23 op/s 2023-09-10T10:51:58.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:51:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:51:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:59 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41549.707061 secs 2023-09-10T10:51:59.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:51:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41550.934279 secs 2023-09-10T10:51:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:59 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41549.707061 secs 2023-09-10T10:51:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:51:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41550.934279 secs 2023-09-10T10:51:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:59 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41549.707061 secs 2023-09-10T10:51:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:51:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41550.934279 secs 2023-09-10T10:52:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:00 smithi082 ceph-mon[98694]: pgmap v22317: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 795 KiB/s wr, 16 op/s 2023-09-10T10:52:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41551.285391 secs 2023-09-10T10:52:00.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:00 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:52:00.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:52:00.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:00 smithi098 ceph-mon[103124]: pgmap v22317: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 795 KiB/s wr, 16 op/s 2023-09-10T10:52:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41551.285391 secs 2023-09-10T10:52:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:00 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:52:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:52:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:00 smithi170 ceph-mon[102832]: pgmap v22317: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 795 KiB/s wr, 16 op/s 2023-09-10T10:52:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41551.285391 secs 2023-09-10T10:52:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:00 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:52:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:52:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:02.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:02 smithi082 ceph-mon[98694]: pgmap v22318: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 795 KiB/s wr, 22 op/s 2023-09-10T10:52:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:02 smithi098 ceph-mon[103124]: pgmap v22318: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 795 KiB/s wr, 22 op/s 2023-09-10T10:52:02.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:02 smithi170 ceph-mon[102832]: pgmap v22318: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 795 KiB/s wr, 22 op/s 2023-09-10T10:52:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:04 smithi082 ceph-mon[98694]: pgmap v22319: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 542 KiB/s wr, 15 op/s 2023-09-10T10:52:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:04 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41554.707139 secs 2023-09-10T10:52:04.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41555.934364 secs 2023-09-10T10:52:04.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:04 smithi098 ceph-mon[103124]: pgmap v22319: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 542 KiB/s wr, 15 op/s 2023-09-10T10:52:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:04 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41554.707139 secs 2023-09-10T10:52:04.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41555.934364 secs 2023-09-10T10:52:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:04 smithi170 ceph-mon[102832]: pgmap v22319: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 542 KiB/s wr, 15 op/s 2023-09-10T10:52:04.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:04 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41554.707139 secs 2023-09-10T10:52:04.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41555.934364 secs 2023-09-10T10:52:05.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41556.285455 secs 2023-09-10T10:52:05.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41556.285455 secs 2023-09-10T10:52:05.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41556.285455 secs 2023-09-10T10:52:06.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:06 smithi082 ceph-mon[98694]: pgmap v22320: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 542 KiB/s wr, 15 op/s 2023-09-10T10:52:06.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:06 smithi098 ceph-mon[103124]: pgmap v22320: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 542 KiB/s wr, 15 op/s 2023-09-10T10:52:06.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:06 smithi170 ceph-mon[102832]: pgmap v22320: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 542 KiB/s wr, 15 op/s 2023-09-10T10:52:08.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:08 smithi082 ceph-mon[98694]: pgmap v22321: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 921 KiB/s wr, 35 op/s 2023-09-10T10:52:08.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:08 smithi098 ceph-mon[103124]: pgmap v22321: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 921 KiB/s wr, 35 op/s 2023-09-10T10:52:08.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:08 smithi170 ceph-mon[102832]: pgmap v22321: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 921 KiB/s wr, 35 op/s 2023-09-10T10:52:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:09 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41559.707202 secs 2023-09-10T10:52:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:09 smithi082 ceph-mon[98694]: osdmap e1463: 12 total, 12 up, 12 in 2023-09-10T10:52:09.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41560.934423 secs 2023-09-10T10:52:09.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:09 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41559.707202 secs 2023-09-10T10:52:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:09 smithi098 ceph-mon[103124]: osdmap e1463: 12 total, 12 up, 12 in 2023-09-10T10:52:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41560.934423 secs 2023-09-10T10:52:09.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:09 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41559.707202 secs 2023-09-10T10:52:09.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:09 smithi170 ceph-mon[102832]: osdmap e1463: 12 total, 12 up, 12 in 2023-09-10T10:52:09.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41560.934423 secs 2023-09-10T10:52:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:10 smithi082 ceph-mon[98694]: pgmap v22323: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 455 KiB/s wr, 31 op/s 2023-09-10T10:52:10.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41561.285487 secs 2023-09-10T10:52:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:10 smithi098 ceph-mon[103124]: pgmap v22323: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 455 KiB/s wr, 31 op/s 2023-09-10T10:52:10.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41561.285487 secs 2023-09-10T10:52:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:10 smithi170 ceph-mon[102832]: pgmap v22323: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 455 KiB/s wr, 31 op/s 2023-09-10T10:52:10.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41561.285487 secs 2023-09-10T10:52:11.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:11 smithi098 ceph-mon[103124]: pgmap v22324: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 603 KiB/s wr, 30 op/s 2023-09-10T10:52:11.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:11 smithi170 ceph-mon[102832]: pgmap v22324: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 603 KiB/s wr, 30 op/s 2023-09-10T10:52:11.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:11 smithi082 ceph-mon[98694]: pgmap v22324: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 603 KiB/s wr, 30 op/s 2023-09-10T10:52:14.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:13 smithi170 ceph-mon[102832]: pgmap v22325: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 603 KiB/s wr, 30 op/s 2023-09-10T10:52:14.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:13 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41564.707270 secs 2023-09-10T10:52:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:13 smithi082 ceph-mon[98694]: pgmap v22325: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 603 KiB/s wr, 30 op/s 2023-09-10T10:52:14.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:13 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41564.707270 secs 2023-09-10T10:52:14.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:13 smithi098 ceph-mon[103124]: pgmap v22325: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 603 KiB/s wr, 30 op/s 2023-09-10T10:52:14.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:13 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41564.707270 secs 2023-09-10T10:52:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41565.934487 secs 2023-09-10T10:52:15.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:14 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:52:15.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:14 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41565.934487 secs 2023-09-10T10:52:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:14 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:52:15.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:14 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:15.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41565.934487 secs 2023-09-10T10:52:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:14 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:52:15.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:14 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:16.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41566.285550 secs 2023-09-10T10:52:16.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:15 smithi170 ceph-mon[102832]: pgmap v22326: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 603 KiB/s wr, 30 op/s 2023-09-10T10:52:16.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:52:16.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:52:16.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:52:16.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:52:16.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:15 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:52:16.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:15 smithi170 ceph-mon[102832]: osdmap e1464: 12 total, 12 up, 12 in 2023-09-10T10:52:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41566.285550 secs 2023-09-10T10:52:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:15 smithi082 ceph-mon[98694]: pgmap v22326: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 603 KiB/s wr, 30 op/s 2023-09-10T10:52:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:52:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:52:16.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:52:16.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:52:16.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:15 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:52:16.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:15 smithi082 ceph-mon[98694]: osdmap e1464: 12 total, 12 up, 12 in 2023-09-10T10:52:16.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41566.285550 secs 2023-09-10T10:52:16.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:15 smithi098 ceph-mon[103124]: pgmap v22326: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 603 KiB/s wr, 30 op/s 2023-09-10T10:52:16.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:52:16.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:52:16.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:52:16.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:52:16.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:15 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:52:16.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:15 smithi098 ceph-mon[103124]: osdmap e1464: 12 total, 12 up, 12 in 2023-09-10T10:52:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:16 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:52:17.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:16 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:17.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:52:17.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:52:17.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:52:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:16 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:52:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:16 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:52:17.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:52:17.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:52:17.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:16 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:52:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:16 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:52:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:52:17.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:52:18.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:17 smithi098 ceph-mon[103124]: pgmap v22328: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 179 KiB/s wr, 15 op/s 2023-09-10T10:52:18.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:17 smithi170 ceph-mon[102832]: pgmap v22328: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 179 KiB/s wr, 15 op/s 2023-09-10T10:52:18.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:17 smithi082 ceph-mon[98694]: pgmap v22328: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 179 KiB/s wr, 15 op/s 2023-09-10T10:52:19.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:18 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41569.707348 secs 2023-09-10T10:52:19.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:18 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41569.707348 secs 2023-09-10T10:52:19.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:18 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41569.707348 secs 2023-09-10T10:52:20.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41570.934593 secs 2023-09-10T10:52:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:19 smithi098 ceph-mon[103124]: pgmap v22329: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 12 op/s 2023-09-10T10:52:20.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41571.285619 secs 2023-09-10T10:52:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41570.934593 secs 2023-09-10T10:52:20.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:19 smithi170 ceph-mon[102832]: pgmap v22329: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 12 op/s 2023-09-10T10:52:20.429 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:19 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41571.285619 secs 2023-09-10T10:52:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41570.934593 secs 2023-09-10T10:52:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:19 smithi082 ceph-mon[98694]: pgmap v22329: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 148 KiB/s wr, 12 op/s 2023-09-10T10:52:20.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:19 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41571.285619 secs 2023-09-10T10:52:22.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:22 smithi170 ceph-mon[102832]: pgmap v22330: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2023-09-10T10:52:22.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:22 smithi082 ceph-mon[98694]: pgmap v22330: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2023-09-10T10:52:22.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:22 smithi098 ceph-mon[103124]: pgmap v22330: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2023-09-10T10:52:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:24 smithi170 ceph-mon[102832]: pgmap v22331: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2023-09-10T10:52:24.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:24 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41574.707447 secs 2023-09-10T10:52:24.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:24 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41575.934650 secs 2023-09-10T10:52:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:24 smithi082 ceph-mon[98694]: pgmap v22331: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2023-09-10T10:52:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:24 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41574.707447 secs 2023-09-10T10:52:24.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:24 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41575.934650 secs 2023-09-10T10:52:24.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:24 smithi098 ceph-mon[103124]: pgmap v22331: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2023-09-10T10:52:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:24 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41574.707447 secs 2023-09-10T10:52:24.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:24 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41575.934650 secs 2023-09-10T10:52:25.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:25 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41576.285682 secs 2023-09-10T10:52:25.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:25 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41576.285682 secs 2023-09-10T10:52:25.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:25 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41576.285682 secs 2023-09-10T10:52:26.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:26 smithi170 ceph-mon[102832]: pgmap v22332: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2023-09-10T10:52:26.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:26 smithi082 ceph-mon[98694]: pgmap v22332: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2023-09-10T10:52:26.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:26 smithi098 ceph-mon[103124]: pgmap v22332: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 12 op/s 2023-09-10T10:52:28.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:28 smithi170 ceph-mon[102832]: pgmap v22333: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 159 KiB/s wr, 12 op/s 2023-09-10T10:52:28.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:28 smithi082 ceph-mon[98694]: pgmap v22333: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 159 KiB/s wr, 12 op/s 2023-09-10T10:52:28.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:28 smithi098 ceph-mon[103124]: pgmap v22333: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 159 KiB/s wr, 12 op/s 2023-09-10T10:52:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:29 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41579.707518 secs 2023-09-10T10:52:29.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:29 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41580.934734 secs 2023-09-10T10:52:29.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:29 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41579.707518 secs 2023-09-10T10:52:29.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:29 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41580.934734 secs 2023-09-10T10:52:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:29 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41579.707518 secs 2023-09-10T10:52:29.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:29 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41580.934734 secs 2023-09-10T10:52:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:30 smithi082 ceph-mon[98694]: pgmap v22334: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:52:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:30 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41581.285744 secs 2023-09-10T10:52:30.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:30 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:52:30.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:30 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:30 smithi098 ceph-mon[103124]: pgmap v22334: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:52:30.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:30 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41581.285744 secs 2023-09-10T10:52:30.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:30 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:52:30.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:30 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:30 smithi170 ceph-mon[102832]: pgmap v22334: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 149 KiB/s wr, 11 op/s 2023-09-10T10:52:30.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:30 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41581.285744 secs 2023-09-10T10:52:30.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:30 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:52:30.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:30 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:32 smithi082 ceph-mon[98694]: pgmap v22335: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 332 KiB/s wr, 17 op/s 2023-09-10T10:52:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:32 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:52:32.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:32 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:32.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:32 smithi098 ceph-mon[103124]: pgmap v22335: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 332 KiB/s wr, 17 op/s 2023-09-10T10:52:32.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:32 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:52:32.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:32 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:32.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:32 smithi170 ceph-mon[102832]: pgmap v22335: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 332 KiB/s wr, 17 op/s 2023-09-10T10:52:32.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:32 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:52:32.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:32 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:34 smithi082 ceph-mon[98694]: pgmap v22336: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T10:52:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:34 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41584.707581 secs 2023-09-10T10:52:34.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:34 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41585.934817 secs 2023-09-10T10:52:34.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:34 smithi098 ceph-mon[103124]: pgmap v22336: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T10:52:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:34 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41584.707581 secs 2023-09-10T10:52:34.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:34 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41585.934817 secs 2023-09-10T10:52:34.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:34 smithi170 ceph-mon[102832]: pgmap v22336: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T10:52:34.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:34 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41584.707581 secs 2023-09-10T10:52:34.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:34 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41585.934817 secs 2023-09-10T10:52:35.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:35 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41586.285810 secs 2023-09-10T10:52:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:35 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41586.285810 secs 2023-09-10T10:52:35.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:35 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41586.285810 secs 2023-09-10T10:52:36.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:36 smithi082 ceph-mon[98694]: pgmap v22337: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T10:52:36.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:36 smithi098 ceph-mon[103124]: pgmap v22337: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T10:52:36.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:36 smithi170 ceph-mon[102832]: pgmap v22337: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 332 KiB/s wr, 12 op/s 2023-09-10T10:52:37.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:37 smithi098 ceph-mon[103124]: pgmap v22338: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 455 KiB/s wr, 17 op/s 2023-09-10T10:52:37.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:37 smithi170 ceph-mon[102832]: pgmap v22338: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 455 KiB/s wr, 17 op/s 2023-09-10T10:52:37.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:37 smithi082 ceph-mon[98694]: pgmap v22338: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 455 KiB/s wr, 17 op/s 2023-09-10T10:52:40.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:39 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41590.934873 secs 2023-09-10T10:52:40.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:39 smithi170 ceph-mon[102832]: pgmap v22339: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 306 KiB/s wr, 11 op/s 2023-09-10T10:52:40.180 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:39 smithi170 ceph-mon[102832]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:52:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:39 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41590.934873 secs 2023-09-10T10:52:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:39 smithi082 ceph-mon[98694]: pgmap v22339: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 306 KiB/s wr, 11 op/s 2023-09-10T10:52:40.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:39 smithi082 ceph-mon[98694]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:52:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:39 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41590.934873 secs 2023-09-10T10:52:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:39 smithi098 ceph-mon[103124]: pgmap v22339: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 306 KiB/s wr, 11 op/s 2023-09-10T10:52:40.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:39 smithi098 ceph-mon[103124]: MDS health message cleared (mds.?): 1 slow requests are blocked > 180 secs 2023-09-10T10:52:41.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:40 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41591.285875 secs 2023-09-10T10:52:41.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:40 smithi170 ceph-mon[102832]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:52:41.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:40 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:52:41.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:40 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:40 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41591.285875 secs 2023-09-10T10:52:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:40 smithi082 ceph-mon[98694]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:52:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:40 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:52:41.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:40 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:41.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:40 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41591.285875 secs 2023-09-10T10:52:41.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:40 smithi098 ceph-mon[103124]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:52:41.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:40 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:52:41.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:40 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:42.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:42 smithi170 ceph-mon[102832]: pgmap v22340: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 624 KiB/s wr, 17 op/s 2023-09-10T10:52:42.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:42 smithi082 ceph-mon[98694]: pgmap v22340: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 624 KiB/s wr, 17 op/s 2023-09-10T10:52:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:42 smithi098 ceph-mon[103124]: pgmap v22340: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 624 KiB/s wr, 17 op/s 2023-09-10T10:52:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:44 smithi170 ceph-mon[102832]: pgmap v22341: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 441 KiB/s wr, 11 op/s 2023-09-10T10:52:44.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:44 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41595.934957 secs 2023-09-10T10:52:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:44 smithi082 ceph-mon[98694]: pgmap v22341: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 441 KiB/s wr, 11 op/s 2023-09-10T10:52:44.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:44 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41595.934957 secs 2023-09-10T10:52:44.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:44 smithi098 ceph-mon[103124]: pgmap v22341: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 441 KiB/s wr, 11 op/s 2023-09-10T10:52:44.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:44 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41595.934957 secs 2023-09-10T10:52:45.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:45 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41596.285940 secs 2023-09-10T10:52:45.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:45 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41596.285940 secs 2023-09-10T10:52:45.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:45 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41596.285940 secs 2023-09-10T10:52:46.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:46 smithi170 ceph-mon[102832]: pgmap v22342: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 441 KiB/s wr, 11 op/s 2023-09-10T10:52:46.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:46 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:52:46.679 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:46 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:46 smithi082 ceph-mon[98694]: pgmap v22342: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 441 KiB/s wr, 11 op/s 2023-09-10T10:52:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:46 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:52:46.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:46 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:46 smithi098 ceph-mon[103124]: pgmap v22342: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 441 KiB/s wr, 11 op/s 2023-09-10T10:52:46.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:46 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:52:46.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:46 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:48.678 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:48 smithi170 ceph-mon[102832]: pgmap v22343: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 758 KiB/s wr, 17 op/s 2023-09-10T10:52:48.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:48 smithi082 ceph-mon[98694]: pgmap v22343: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 758 KiB/s wr, 17 op/s 2023-09-10T10:52:48.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:48 smithi098 ceph-mon[103124]: pgmap v22343: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 758 KiB/s wr, 17 op/s 2023-09-10T10:52:49.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:49 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41600.935023 secs 2023-09-10T10:52:49.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:49 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41600.935023 secs 2023-09-10T10:52:49.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:49 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41600.935023 secs 2023-09-10T10:52:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:50 smithi082 ceph-mon[98694]: pgmap v22344: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 635 KiB/s wr, 11 op/s 2023-09-10T10:52:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:50 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41601.286004 secs 2023-09-10T10:52:50.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:50 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:52:50.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:50 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:50.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:50 smithi098 ceph-mon[103124]: pgmap v22344: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 635 KiB/s wr, 11 op/s 2023-09-10T10:52:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:50 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41601.286004 secs 2023-09-10T10:52:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:50 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:52:50.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:50 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:50.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:50 smithi170 ceph-mon[102832]: pgmap v22344: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 635 KiB/s wr, 11 op/s 2023-09-10T10:52:50.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:50 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41601.286004 secs 2023-09-10T10:52:50.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:50 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:52:50.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:50 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:52 smithi082 ceph-mon[98694]: pgmap v22345: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 857 KiB/s wr, 19 op/s 2023-09-10T10:52:52.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:52 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:52:52.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:52 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:52.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:52 smithi098 ceph-mon[103124]: pgmap v22345: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 857 KiB/s wr, 19 op/s 2023-09-10T10:52:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:52 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:52:52.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:52 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:52 smithi170 ceph-mon[102832]: pgmap v22345: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 857 KiB/s wr, 19 op/s 2023-09-10T10:52:52.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:52 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:52:52.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:52 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:54 smithi082 ceph-mon[98694]: pgmap v22346: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 539 KiB/s wr, 13 op/s 2023-09-10T10:52:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:54 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:52:54.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:54 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:54.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:54 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41605.935084 secs 2023-09-10T10:52:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:54 smithi098 ceph-mon[103124]: pgmap v22346: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 539 KiB/s wr, 13 op/s 2023-09-10T10:52:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:54 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:52:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:54 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:54.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:54 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41605.935084 secs 2023-09-10T10:52:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:54 smithi170 ceph-mon[102832]: pgmap v22346: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 539 KiB/s wr, 13 op/s 2023-09-10T10:52:54.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:54 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:52:54.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:54 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:54.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:54 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41605.935084 secs 2023-09-10T10:52:55.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:55 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41606.286070 secs 2023-09-10T10:52:55.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:55 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41606.286070 secs 2023-09-10T10:52:55.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:55 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41606.286070 secs 2023-09-10T10:52:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:56 smithi082 ceph-mon[98694]: pgmap v22347: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 539 KiB/s wr, 13 op/s 2023-09-10T10:52:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:56 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:52:56.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:56 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:56 smithi098 ceph-mon[103124]: pgmap v22347: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 539 KiB/s wr, 13 op/s 2023-09-10T10:52:56.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:56 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:52:56.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:56 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:56.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:56 smithi170 ceph-mon[102832]: pgmap v22347: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 539 KiB/s wr, 13 op/s 2023-09-10T10:52:56.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:56 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:52:56.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:56 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:58 smithi082 ceph-mon[98694]: pgmap v22348: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 938 KiB/s wr, 21 op/s 2023-09-10T10:52:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:58 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:52:58.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:58 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:58 smithi098 ceph-mon[103124]: pgmap v22348: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 938 KiB/s wr, 21 op/s 2023-09-10T10:52:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:58 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:52:58.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:58 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:58.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:58 smithi170 ceph-mon[102832]: pgmap v22348: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 938 KiB/s wr, 21 op/s 2023-09-10T10:52:58.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:58 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:52:58.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:58 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:52:59.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:52:59 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41610.935189 secs 2023-09-10T10:52:59.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:52:59 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41610.935189 secs 2023-09-10T10:52:59.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:52:59 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41610.935189 secs 2023-09-10T10:53:00.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:00 smithi098 ceph-mon[103124]: pgmap v22349: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 621 KiB/s wr, 15 op/s 2023-09-10T10:53:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:00 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41611.286137 secs 2023-09-10T10:53:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:00 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:53:00.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:00 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:53:00.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:00 smithi170 ceph-mon[102832]: pgmap v22349: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 621 KiB/s wr, 15 op/s 2023-09-10T10:53:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:00 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41611.286137 secs 2023-09-10T10:53:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:00 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:53:00.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:00 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:53:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:00 smithi082 ceph-mon[98694]: pgmap v22349: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 621 KiB/s wr, 15 op/s 2023-09-10T10:53:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:00 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41611.286137 secs 2023-09-10T10:53:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:00 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:53:00.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:00 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:53:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:01 smithi098 ceph-mon[103124]: pgmap v22350: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 892 KiB/s wr, 36 op/s 2023-09-10T10:53:01.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:01 smithi170 ceph-mon[102832]: pgmap v22350: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 892 KiB/s wr, 36 op/s 2023-09-10T10:53:01.958 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:01 smithi082 ceph-mon[98694]: pgmap v22350: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 892 KiB/s wr, 36 op/s 2023-09-10T10:53:04.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:03 smithi170 ceph-mon[102832]: pgmap v22351: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 669 KiB/s wr, 28 op/s 2023-09-10T10:53:04.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:03 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41614.707982 secs 2023-09-10T10:53:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:03 smithi082 ceph-mon[98694]: pgmap v22351: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 669 KiB/s wr, 28 op/s 2023-09-10T10:53:04.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:03 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41614.707982 secs 2023-09-10T10:53:04.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:03 smithi098 ceph-mon[103124]: pgmap v22351: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 669 KiB/s wr, 28 op/s 2023-09-10T10:53:04.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:03 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41614.707982 secs 2023-09-10T10:53:05.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:04 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41615.935297 secs 2023-09-10T10:53:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:04 smithi170 ceph-mon[102832]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:53:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:04 smithi170 ceph-mon[102832]: osdmap e1465: 12 total, 12 up, 12 in 2023-09-10T10:53:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:04 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:53:05.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:04 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:53:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:04 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41615.935297 secs 2023-09-10T10:53:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:04 smithi082 ceph-mon[98694]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:53:05.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:04 smithi082 ceph-mon[98694]: osdmap e1465: 12 total, 12 up, 12 in 2023-09-10T10:53:05.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:04 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:53:05.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:04 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:53:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:04 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41615.935297 secs 2023-09-10T10:53:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:04 smithi098 ceph-mon[103124]: Health check update: 3 MDSs report slow requests (MDS_SLOW_REQUEST) 2023-09-10T10:53:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:04 smithi098 ceph-mon[103124]: osdmap e1465: 12 total, 12 up, 12 in 2023-09-10T10:53:05.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:04 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:53:05.256 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:04 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:53:06.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:05 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41616.286203 secs 2023-09-10T10:53:06.179 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:05 smithi170 ceph-mon[102832]: pgmap v22353: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 803 KiB/s wr, 34 op/s 2023-09-10T10:53:06.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:05 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41616.286203 secs 2023-09-10T10:53:06.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:05 smithi082 ceph-mon[98694]: pgmap v22353: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 803 KiB/s wr, 34 op/s 2023-09-10T10:53:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:05 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41616.286203 secs 2023-09-10T10:53:06.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:05 smithi098 ceph-mon[103124]: pgmap v22353: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 803 KiB/s wr, 34 op/s 2023-09-10T10:53:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:06 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:53:07.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:06 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:53:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:06 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:53:07.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:06 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:53:07.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:06 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:53:07.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:06 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:53:08.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:07 smithi170 ceph-mon[102832]: pgmap v22354: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 774 KiB/s wr, 34 op/s 2023-09-10T10:53:08.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:07 smithi082 ceph-mon[98694]: pgmap v22354: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 774 KiB/s wr, 34 op/s 2023-09-10T10:53:08.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:07 smithi098 ceph-mon[103124]: pgmap v22354: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 774 KiB/s wr, 34 op/s 2023-09-10T10:53:09.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:08 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41619.708043 secs 2023-09-10T10:53:09.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:08 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41619.708043 secs 2023-09-10T10:53:09.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:08 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41619.708043 secs 2023-09-10T10:53:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:09 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41620.935355 secs 2023-09-10T10:53:10.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:09 smithi170 ceph-mon[102832]: pgmap v22355: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 774 KiB/s wr, 34 op/s 2023-09-10T10:53:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:09 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41620.935355 secs 2023-09-10T10:53:10.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:09 smithi082 ceph-mon[98694]: pgmap v22355: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 774 KiB/s wr, 34 op/s 2023-09-10T10:53:10.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:09 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41620.935355 secs 2023-09-10T10:53:10.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:09 smithi098 ceph-mon[103124]: pgmap v22355: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 774 KiB/s wr, 34 op/s 2023-09-10T10:53:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:10 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41621.286266 secs 2023-09-10T10:53:11.178 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:10 smithi170 ceph-mon[102832]: osdmap e1466: 12 total, 12 up, 12 in 2023-09-10T10:53:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:10 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41621.286266 secs 2023-09-10T10:53:11.208 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:10 smithi082 ceph-mon[98694]: osdmap e1466: 12 total, 12 up, 12 in 2023-09-10T10:53:11.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:10 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41621.286266 secs 2023-09-10T10:53:11.255 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:10 smithi098 ceph-mon[103124]: osdmap e1466: 12 total, 12 up, 12 in 2023-09-10T10:53:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:12 smithi082 ceph-mon[98694]: pgmap v22357: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 562 KiB/s wr, 22 op/s 2023-09-10T10:53:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:12 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:53:12.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:12 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:53:12.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:12 smithi098 ceph-mon[103124]: pgmap v22357: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 562 KiB/s wr, 22 op/s 2023-09-10T10:53:12.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:12 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:53:12.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:12 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:53:12.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:12 smithi170 ceph-mon[102832]: pgmap v22357: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 562 KiB/s wr, 22 op/s 2023-09-10T10:53:12.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:12 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6838/2785086371,v1:172.21.15.98:6839/2785086371] up:active 2023-09-10T10:53:12.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:12 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:53:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:14 smithi082 ceph-mon[98694]: pgmap v22358: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 503 KiB/s wr, 20 op/s 2023-09-10T10:53:14.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:14 smithi082 ceph-mon[98694]: 1 slow requests, 0 included below; oldest blocked for > 41624.708127 secs 2023-09-10T10:53:14.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:14 smithi082 ceph-mon[98694]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:53:14.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:14 smithi082 ceph-mon[98694]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:53:14.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:14 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41625.935433 secs 2023-09-10T10:53:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:14 smithi098 ceph-mon[103124]: pgmap v22358: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 503 KiB/s wr, 20 op/s 2023-09-10T10:53:14.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:14 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41624.708127 secs 2023-09-10T10:53:14.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:14 smithi098 ceph-mon[103124]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:53:14.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:14 smithi098 ceph-mon[103124]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:53:14.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:14 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41625.935433 secs 2023-09-10T10:53:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:14 smithi170 ceph-mon[102832]: pgmap v22358: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 503 KiB/s wr, 20 op/s 2023-09-10T10:53:14.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:14 smithi170 ceph-mon[102832]: 1 slow requests, 0 included below; oldest blocked for > 41624.708127 secs 2023-09-10T10:53:14.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:14 smithi170 ceph-mon[102832]: mds.? [v2:172.21.15.98:6834/1356474828,v1:172.21.15.98:6835/1356474828] up:standby-replay 2023-09-10T10:53:14.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:14 smithi170 ceph-mon[102832]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-10T10:53:14.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:14 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41625.935433 secs 2023-09-10T10:53:15.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:15 smithi082 ceph-mon[98694]: 3 slow requests, 0 included below; oldest blocked for > 41626.286330 secs 2023-09-10T10:53:15.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:15 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41626.286330 secs 2023-09-10T10:53:15.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:15 smithi170 ceph-mon[102832]: 3 slow requests, 0 included below; oldest blocked for > 41626.286330 secs 2023-09-10T10:53:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:16 smithi082 ceph-mon[98694]: pgmap v22359: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 450 KiB/s wr, 17 op/s 2023-09-10T10:53:16.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:16 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:53:16.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:16 smithi098 ceph-mon[103124]: pgmap v22359: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 450 KiB/s wr, 17 op/s 2023-09-10T10:53:16.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:16 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:53:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:16 smithi170 ceph-mon[102832]: pgmap v22359: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 450 KiB/s wr, 17 op/s 2023-09-10T10:53:16.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:16 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-10T10:53:17.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:53:17.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:53:17.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:17 smithi082 ceph-mon[98694]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:53:17.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:53:17.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:53:17.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:17 smithi098 ceph-mon[103124]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:53:17.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-10T10:53:17.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-10T10:53:17.929 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:17 smithi170 ceph-mon[102832]: from='mgr.15597 172.21.15.82:0/4258387338' entity='mgr.x' 2023-09-10T10:53:18.708 INFO:journalctl@ceph.mon.a.smithi082.stdout:Sep 10 10:53:18 smithi082 ceph-mon[98694]: pgmap v22360: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 14 op/s 2023-09-10T10:53:18.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:18 smithi098 ceph-mon[103124]: pgmap v22360: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 14 op/s 2023-09-10T10:53:18.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Sep 10 10:53:18 smithi170 ceph-mon[102832]: pgmap v22360: 97 pgs: 97 active+clean; 15 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 14 op/s 2023-09-10T10:53:19.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:19 smithi098 ceph-mon[103124]: 1 slow requests, 0 included below; oldest blocked for > 41629.708187 secs 2023-09-10T10:53:19.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Sep 10 10:53:19 smithi098 ceph-mon[103124]: 3 slow requests, 0 included below; oldest blocked for > 41630.935514 secs 2023-09-10T10:53:19.833 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2023-09-10T10:53:19.834 DEBUG:teuthology.task.console_log:Killing console logger for smithi082 2023-09-10T10:53:19.834 DEBUG:teuthology.task.console_log:Killing console logger for smithi098 2023-09-10T10:53:19.835 DEBUG:teuthology.task.console_log:Killing console logger for smithi170 2023-09-10T10:53:19.835 DEBUG:teuthology.exit:Finished running handlers